Three-dimensional measurement of object surface by using ellipse binary defocusing projection

Background

The accuracy of three-dimensional measurement of object surface is always affected by the nonlinear gamma of the projector. The defocusing binary projection can overcome the nonlinear gamma distortion of the projector and reduce the effect of high harmonics without gamma calibration. Although researches have already reduced the errors to get a clear sinusoidal curve, there still leave room for improvement, especially when wide stripes are applied during the measurement.

Methods

This paper presents a kind of ellipse binary pattern. By analyzing the property and spectrum, the binary pattern can produce high quality sinusoidal curve and produce smaller errors. It has a better effect to use an ellipse pattern to overcome the nonlinear gamma distortion of the projector and it is suit to be used as wide stripes.

Results

Simulation and comparison experiments plus three-step phase shifted method are conducted to verify feasibility and accuracy of this binary pattern. The experimental results have indicated that this binary pattern can increase the accuracy of 3D measurement and reduce phase errors caused by the nonlinear gamma of the projector. The defocusing ellipse pattern is superior to traditional methods. In addition, the defocusing binary stripe is robust and suit to measure object with large period.

Conclusions

In this paper, an binary ellipse pattern is proposed for high-accuracy 3D measurement profilometry. It is easy to generate high-quality sinusoidal fringe pattern. Experiment results have demonstrated the feasibility and accuracy of the improved binary pattern besides it proved that the period of fringe pattern has little impact on the accuracy of measurement.

Three-dimensional (3D) profilometry measurement is very important in both scientific studies and industrial manufacture [1]. Conventionally, sinusoidal pattern is usually used in a digital fringe projection system. However, the gamma distortion of the projector causes non-sine of the pattern which will lead high harmonics components and high-frequency noise. Eventually, the harmonics will affect the accuracy of 3D measurement [2,3,4,5,6]. In order to solve the above problem, binary defocusing technology has been widely researched. The binary defocusing techniques have been proposed recently, including (SBM) [7], SPWM [8], OPWM [9], etc.

These methods have demonstrated its potential to overcome the gamma distortion without calibration. However the technologies aforementioned are limited to narrow fringe stripes, because wide fringe stripes decrease the quality of sinusoidal curve generated by defocusing technology [10]. In order to solve this problem, Recently more improved methods has been researched. Wang Y [11] brings dithering technique into profilometry, and shows its ability to improve the fringe quality, especially for wide fringes. However the dithering techniques are originally developed for global similarity. Jiang H [12] proposes to use a kind of triangle pattern to get the sinusoidal curve. It has the potential to be used as the wide stripes pattern, but the boundary of triangle is a straight line which will affect the quality of defocusing stripes. Xiao Y [13] proposes to combine intensity and phase to solve wide stripes. Besides the accuracy of the narrow stripes and wide stripes are compared. In order to reduce the errors caused by wide stripes, Fu Y, et al. [14] propose a non-uniform periodical binary fringe defocusing projection. Xian and Su [15] develop the area modulation grating technique. This pattern make use of the two-dimensional information to solve the bottleneck of wide stripes. Other modified technologies are also researched. Pérez O G, et al. [16] propose a two-step trapezoidal-pattern phase-shifting method for 3D shape measurements. The process of shifting is based on the two trapezoid with phase shift T/2 where T is the period of every trapezoid. Because it only need two patterns, it is more applicable to be used as wide stripes. But this pattern should be combined into at least eight RGB color images. Su [17] uses the area method to generate wide stripes. The advantage of this method can get high quality sinusoidal, but the process is time-costing.

In this paper, we propose an ellipse binary pattern to solve this problem caused by large period. By using defocusing technology, the harmonics and high frequency noise can be effectively suppressed. With the increasing of defocusing amounts, a similar sinusoidal pattern can be gradually obtained from the improved pattern, besides the phase errors can be reduced rapidly until the phase error is close to zero. Simulations and experimental results demonstrate the proposed pattern can get more accurate results compared with other methods besides it is robust to different periods.

Phase-shifted algorithm

In this paper, three-step phase-shifted algorithm with a phase shift 2π/3 is applied, because it need the least formulas for its simplicity and speed. The sinusoidal light intensity can be described as:

Where A(x, y) is the average intensity, B(x, y) is the intensity modulation, and φ(x, y) is the phase needed to be calculated. Based on the Eqs. (1)-(3), the φ(x, y) can be gotten.

Equations (4)-(5) show the relationship between the wrapped phase and the absolute phase. Where k(x, y) is the order of wrapped phase.

The theory of improved ellipse binary pattern

In order to explain the principle and advantage of ellipse binary pattern, firstly the theory about sinusoidal curve will be illustrated. The sinusoidal curve distribution can be presented as:

Where c(x) ranges from 0 to 1, p is the period of every sinusoidal curve. It can be divided into a series of parts with the same pixels along x direction. As shown in Fig. 1, assuming there are N _x equal parts in a period. The sinusoidal curve can be presented by a series of rectangles approximately. Because the input image is discrete, it can be presented as:

The sinusoidal curve can be presented by a series of rectangles

Full size image

Where ω = p/N _x which is the width of every part, and sinc(x) = sin x/x.

Based on the theory of reference [17], the meaning of Eq. (7) is that the intensity of sinusoidal curve can be presented by performing integral of area. The width of rectangle is set to have w pixels and the height is set to have h pixels. The area of every part is S = w*h. The area of every rectangle is set to S _i (i ∈ (0, 1, 2 .. …)), then the intensity can be can calculated as Eq. (8):

When the wide stripes are used, a binary pattern is designed to get the ideal sinusoidal curve which consists of the binary ellipse pattern. This pattern is an improved technology of binary triangle fringe pattern which have proved that it has the ability to generate sinusoidal pattern [12]. Because the boundary of triangle is a straight line, the integral value is approximate linear relation. From the Fig. 2, we can find that triangle can not get perfect sinusoidal curve. Figure 2a is the triangle binary pattern, Fig. 2b is the relationship between frequency and amplitude. Figure 2c is the comparison between ideal sinusoidal curve and defocusing triangle pattern. The process of defocusing makes the intensity suffer from attenuation especially at the crest and trough.

The triangle binary pattern. a The triangle pattern. b The relationship between frequency and amplitude. c The comparison between ideal sinusoidal curve and defocusing triangle pattern

Full size image

The binary pattern is a unit which will be transformed into a sinusoidal pattern when the edge is defocused. Every unit can be expressed as I _unit (x, y), here x and y represent the image coordinate in the projector. As shown in Fig. 3, the ellipse binary pattern consists of a series of rectangles. In a unit, the width of every rectangle is set to have W pixels and the height is set to have H pixels. Because every pixel value is discrete, the area of this part is S = W*H. If there are N1 pixels with zero intensity, the output intensity can be described by the ratio between the N1 and S. In order to get accurate intensity, we use every column as an area which means the ratio between the number of pixels in the ellipse and all the pixels in the same column can be used to present the intensity in this column.

The ellipse binary pattern can be represented by a series of rectangles

Full size image

The edge of binary unit I _unit (x, y) is blurred along x or y direction in the high frequency. The length of every unit should be blurred based on the height of binary pattern to get an ideal sinusoidal curve. A smooth gray intensity can be gotten by defocusing technology or filter effect of binary ellipse pattern. The defocusing of ellipse along x direction can be expressed as Eq. (8):

Here x and y are the coordination of the image in the projector, b represents the length of semi-minor axis of every ellipse unit. A similar approach can be used to get the dithering pattern along y direction, as shown in Eq. 9.

Similarly, x and y are the coordination of the image in the projector and a represents the length of semi-major axis. In order to express the property by using formula, a special unit is analyzed, the semi-major axis is set to x direction and the width of this unit is a, similarly the semi-minor axis is set to y direction and the height of it is set to b, as shown in Fig. 4.

The improved binary pattern. The width and height is a and b respectively

Full size image

Based on the mathematical theory, the gray intensity of every pixel in the unit can be expressed as follows:

Here x ∈ [−a, a], y ∈ [0, b], when the improved pattern is blured horizontally, the gray intensity in each column would be the same, which can be derived using polar coordinates as follows:

θis the polar angle, the range is from 0 to π. In every column, the gray intensity in the vertical direction is determined as shown in Eq. (12):

Because the pixels are discrete quantity, the intensity can be described by the ratio between areas. The intensity can be indicated as discrete equation, as shown in Eq. (13):

To provide a concrete fringe pattern, here the semi-major axis and semi-minor axis are set a = 6 and b = 4 respectively, then the range of x direction is x ∈ [−6, 6] and the range of y direction is y ∈ [0, 4].

The process of defocusing is similar to a lens system with a 2D Gaussian function which can be represented as Eq. (15).

Where σ is the standard deviation, it determines the width of Gaussian function. For extensive defocusing, this parameter is large so that the size of filter is narrow. Based on the Eqs. (1)-(3), the output intensity can be gotten by convolution between the input image and the Gaussian function, the process can be represented as:

Where I _c (x, y) is the output image, g(x, y) is the Gaussian function, because Gaussian filter is isotropic, so the I _c (x, y) can be gotten by convolution along x axis and y axis, respectively. In this paper, we use the 2D Gaussian function, so the output intensity can be calculated as:

Equation (17) shows the blurred fringe pattern after the defocusing technology. The process of convolution can be presented in frequency domain. The projected sinusoidal fringes in Eqs. (1)-(3) can be presented in time domain as:

Where I _a is the average intensity, I _m is the modulation intensity. ω ₀ = 2π/T and T is the period of fringe pattern, Δφ is the shifted phase value. The fringe pattern can be described in frequency domain, as shown in Eqs. (19)-(21):

By using inverse Fourier transform, the output intensity can be calculate as Eq. (22):

In this paper, the proposed pattern need extensive defocusing but the projector can still be modeled as a 2D Gaussian function, the defocusing process is also suit to ellipse binary pattern.

Phase unwrapping

In Methods section, Eq. (4) calculates wrapped phase with the range of (−π, π). The traditional method is based on temporal phase unwrapping algorithm or spatial phase unwrapping algorithm. The aim of unwrapping phase is to get the order of every wrapped phase. We use a stair intensity image, and the period of different intensity is perfectly aligned with the period of wrapped phase.

Figure 5 demonstrates the principle. A stair image can be generated as:

a The intensity corresponding to wrapped phase; b The order of stair and wrapped phase; c The relationship between wrapped phase and absolute phase

Full size image

Where P represents the period of every wrapped phase. floor keeps the integral part. S is the intensity level for each stair. In order to overcome the albedo, it is necessary to normalize the intensity. The normalization process need the maximum and minimum intensity for each pixel which can be calculated as:

Where A(x, y) and B(x, y) can be gotten from Eqs. (1)-(3), so the stair pattern can be normalized using the following equation:

The order of every wrapped phase can be gotten:

In the formula, R is the intensity captured by the camera and S is the intensity level of each stair.

Simulation

The process of defocusing can be modeled as a 2D Gaussian smoothing filter. Based on the Eq. (15), the process of simulation are shown in Fig. 6. Figure 6a shows the original ellipse binary pattern. The 2D Fourier transformation of the original pattern is shown in Fig. 6b. After using Gaussian smoothing filter and inverse Fourier transformation, a clear sinusoidal curve can be generated, as shown in Fig. 6c. From the results, we can find high-frequency harmonics are almost completely suppressed besides there is little difference between them.

a The improved binary pattern; b The 2D Fourier transformation results; c The results generated by using 2-D Gaussian smoothing filter

Full size image

Moreover, different defocusing levels (the filter sizes are set 4, 12, 20) are stimulated by using Gaussian filter. The simulation results are shown in Fig. 7a-c. From left to right, the amount of defocusing increases and corresponding phase errors are shown in Fig. 7d. From the simulation results, we can find that the phase error reduces when the amount of defocusing increases, but the distinction is not large. This means that the proposed technique can be used to generate clear sinusoidal curve besides it is robust to the change of different defocusing levels.

a-c The sinusoidal curve generated under different defocusing levels; d Phase errors comparison

Full size image

Experiment 1. Generating sinusoidal curve

The phase error caused by different amounts of defocusing are experimented first. Based on the theory of improved pattern, an ellipse binary pattern are projected under different defocusing amounts. As shown in Fig. 8, when the defocusing level increases, the binary pattern gradually becomes a sinusoidal pattern. Because the influence of defocusing is similar to using Gaussian filter, the edge of binary pattern can be blurred along y direction. The first row of Fig. 8 shows the different defocusing amounts of binary pattern, the second row of Fig. 8 shows the phase errors. From the results, we can find that when the projector is in focus, we can get the binary pattern, but there is the largest the phase error. With the increasing of defocusing amounts, a similar sinusoidal pattern can be gradually obtained from the improved pattern, besides the phase errors can be reduced, until the phase error is close to zero. The projector was adjusted from slightly defocusing to extensively defocusing.

Sinusoidal fringe generation from sightly defocusing to significant defocusing. The left column a-e show binary pattern under different defocusing levels. The right column f-j show the 100th cross section phase errors

Full size image

Experiment 2. Comparison between the improved method and traditional methods

In order to verify the feasibility and accuracy of improved binary pattern in this experiment, a white house is measured. Based on the experimental results from Experiment 1, a clear modulated sinusoidal fringe pattern can be gotten. For all the measurements, the projector and the camera are fixed. To test the proposed pattern, we compare the results with three traditional methods: SBM, SPWM, and OPWM. All the binary pattern is in the same defocusing amount and with the same periods (50 pixels). As shown in Fig. 9, it shows the measurement results reconstructed by defocusing ellipse binary pattern.

The measurement results. Based on the three-step phase shifted, the phase map and 3D point cloud can be reconstructed

Full size image

In the comparison experiments, the distance of four experiments measurement is 917 mm and comparison results are shown in Fig. 10, which is based on the SBM, SPWM, OPWM and the proposed pattern, respectively. From the measurement results, we can find that the measurement accuracy using the improved technology is better than that of traditional methods. There is little noise for the proposed method.

Experimental results comparison by using different binary defocusing pattern. a The measurement result based on SBM. b The measurement result based on SPWM. c The measurement result based on OPWM. d The measurement result based on the proposed method

Full size image

To evaluate the measurement accuracy, phase errors are also measured to compare the accuracy of different binary patterns. Two absolute phase maps are needed. One absolute phase is based on the projector calibration and the other is based on the binary defocusing technology. The difference between these two absolute phase were calculated as measurement errors. The final experimental results about the absolute phase errors are plotted in Fig. 11.

Phase errors comparison among different binary patterns

Full size image

By analyzing phase errors, we can find when the defocusing binary patterns are at the same defocusing level and they have the same period, the range of the phase error agrees well with the simulation analysis. The average phase errors proved that OPWM performs better than SBM and SPWM, whose maximum phase error is 0.23 rad, 0.15 rad, 0.08 rad, respectively. Obviously, the maximum phase error of ellipse pattern is 0.02 rad which performs the best in this experiment.

Experiment 3. The accuracy of different period

In this experiment, the lens of projector keep proper defocusing and the system is calibrated. The projector keeps the focus unchanged and projects ellipse binary pattern while the camera captures the defocusing modulation stripes. After the absolute phase is calculated, the phase-height mapping relationship is used to get the 3D point clouds. The fringe number is set to 2,4,8 where the fringe periods are 200 pixels, 50 pixels, and 25 pixels respectively. As shown in Fig. 12.

Comparison results between different periods

Full size image

The comparison measurement experiment in different periods are captured respectively, as shown in the first row of Fig. 12. The corresponding 3D reconstruction results are shown in the second row of Fig. 12. In order to give a specified comparison, the enlarged view of reconstruction results are shown in the third row of Fig. 12. From the results, we can find that the accuracy of measurement increases as the period becomes smaller and the surface become more smooth, but the difference is very small.

The measurement comparison with different periods are listed in Table 1, which includes average height, root-mean-square (RMS), average error and maximum error. When the fringe periods are 200 pixels, 50 pixels, and 25 pixels, the RMS is 0.062 mm, 0.042 mm 0.042 mm, respectively. We also find that the average error range changes from 0.050 mm to 0.024 mm when the period decreases from 200 pixels to 25 pixels. It indicates that the proposed method can successfully reconstruct the 3D point cloud and the accuracy of our proposed method is robust and suit to large period of fringe pattern.

We aim to further to testify its ability to measure the step block, the step-height is about 60 mm, 50 mm, 30 mm respectively and compare it with the ideal sinusoidal curve with projector’s nonlinear gamma correction. The experimental results are shown in Fig. 13. Figure 13a shows the object with modulation information. Figure 13b shows the measurement result. Table 2 lists the ideal height, measured height, absolute error and RMS error for different measuring height. From this table, we can find that the absolute error is 0.078 mm, 0.051 mm, 0.031 mm, moreover RMS is 0.025 mm, 0.018 mm, 0.011 mm, respectively. We can find the difference between measured results and the ideal sinusoidal results is very small. This further verify the success of the proposed pattern.

The measurement of step block. a The object with modulation information; b The measurement results

Full size image

Based on the experiments, we can find high frequency can cause severe phase errors. The experimental results verify the simulation analysis. This means when the harmonics are suppressed, the phase errors are eliminated well. The proposed pattern can get better measurement results. The improved defocusing ellipse pattern do not require to calibrate the nonlinear gamma of the projector because the binary pattern only need two grayscale values. It can suppressed the high-order harmonics effectively by defocusing and the high-frequency noise are eliminated because of the low-pass filtering effects of defocusing. From the comparison results, we can find that when the projector is in a significantly defocusing, the ellipse pattern performs better compared with traditional methods. Moreover, The accuracy of the measurement will not be affected by the large period of stripes which are robust and suit to high-accuracy measurement. However, there are the following disadvantages. Because this improved binary pattern uses three steps phase-shifted to obtain one three-dimensional shape and therefore its measurement speed is low as well as the accuracy of sinusoidal fringe determine the precise of measurement. Any error in the measurement system and calibration parameters will add phase error into the final results. Therefore, better hardware is needed to get high quality sinusoidal curve.

In this paper, an improvement binary ellipse pattern is proposed for high-accuracy 3D measurement profilometry. By explaining the defocusing theory combining the phase-shifting technique, it is easy to generate high-quality sinusoidal fringe pattern, besides the process is similar to the low-pass filtering which eliminated the high harmonics and high-frequency noise. By analyzing the property of proposed pattern, the simulation results are given. Experiment results have demonstrated the feasibility and accuracy of the improved binary pattern besides it proved that the period of fringe pattern has little impact on the accuracy of measurement.

3D:

Three-dimensional

OPWM:

Optimum pulse width modulation

SBM:

Squared binary defocusing technique

SPWM:

Sinusoidal pulse width modulation

This work was supported by the Fundamental Research Funds for the Central Universities (N150403009) and the Fundamental Research Funds for the Central Universities (N162610004).

Funding

Faculty of Robot and Engineering of Northeastern University provides the funding for this research.

Availability of data and materials

Details data has been provided in this paper.

Affiliations

1. Faculty of Robot Science and Engineering, Northeastern University, Shenyang, China

   • Feng Lu
   •  & Chengdong Wu

Contributions

Both authors contributed equally in all the sections of this work. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Feng Lu.

Authors’ information

Feng Lu, a doctor of Faculty of Robot Science and Engineering, Northeastern University, Shenyang, China. His interest includes Structured light, Robot and image processing.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Reprints and Permissions