- Research
- Open Access

# High-speed 3D shape measurement using Fourier transform and stereo vision

- Feng Lu
^{1}Email authorView ORCID ID profile, - Chengdong Wu
^{1}and - Jikun Yang
^{2}

**14**:22

https://doi.org/10.1186/s41476-018-0090-z

© The Author(s) 2018

**Received:**28 May 2018**Accepted:**24 September 2018**Published:**3 October 2018

## Abstract

### Background

In the fast 3D shape measurement, it is an important factor to use the least number of fringe patterns to get the wrapped phase and the wrapped phase is always required to be retrieved to absolute phase. But the process of phase unwrapping may affect the quality of absolute phase. Besides absolute phase retrieval is time-costing especially for high-speed 3D shape measurement.

### Methods

This paper proposes a fast measurement method based on Fourier transform profilometry and stereo vision. Because of the property of stereo vision, every point can find its corresponding point in the wrapped phase. This method can reconstruct 3D surface without phase unwrapping. In order to further increase the measurement speed and overcome the gamma effect of projector, the dithering fringe pattern is used. To resist false matching, the original image matching constraint gives a rough parallax. Phase matching and sub-pixel parallax optimization are used to reduce the matching errors. In order to confirm the phase value of points which are at the edge of wrapped phase, average phase value is calculated.

### Results

A white mask is measured based on the proposed method. The results from every step show the effect of different functions. To better show the effect of the proposed method, a white house is measured and the measurement is compared with LSSM and MFH. In order to further to show the advantage of the proposed method, discontinuous blocks and continuous blocks are measured. The average height, RMS, average error, maximum error are compared with the method which uses absolute phase to match the corresponding points. Experiment results verify the feasibility of the proposed method and it can measure complex objects with high speed and accuracy.

### Conclusion

This paper uses Fourier transform and stereo vision to get wrapped phase and reconstruct 3D shape without phase unwrapping. The proposed method contains three part: phase matching, edge points detection and sub-pixel parallax optimization. By comparing matching precision, the proposed method can get high quality surface. Because the process of measurement only need one frame of deformed fringe pattern to get the wrapped phase and the matching process does not need phase unwrapping, the proposed method has the potential to be used in fast measurement.

## Keywords

- Fringe analysis
- Three-dimensional shape measurement
- Fourier transform
- Stereo vision

## Background

High-speed 3D shape measurement has been an important technology in the industrial manufacturing such as quality inspection, reverse engineering, 3-D sensing, object recognition, etc. [1]. In order to get accurate 3D shape measurement results, a lot of technologies have been researched such as Moire technique (MT) [2], phase-measuring profilometry (PMP) [3–6], and Fourier transform profilometry (FTP) [7–9].

In general, multi-frame fringe patterns can get more accurate and reliable results, but it is significant to use fewer fringe patterns to get the results in high-speed measurement [10]. Among these technologies, Fourier transform profilometry only need one frame of fringe pattern to get the wrapped phase, so it is suitable to be used in high-speed measurement [11].

Though FTP has the advantage, it still has shortcomings. Because FTP only need one frame of fringe pattern, it is sensitive to the quality of fringe pattern [12–15]. It is always affected by the nonlinearity of projector and the wrapped phase will exist noise which will have impact on the measurement directly [16]. Moreover, gamma calibration is required but this process will increase the computation complexity [17]. The phase error compensation is also required to remove the phase error of reconstruction results [18]. But there is not a standard method which can be used in different applications. Another problem which FTP suffers from is that the wrapped phase need to be unwrapped to get the absolute phase. However, the process of phase unwrapping will also affect the accuracy of absolute phase and it will also increase computing time of the measurement [19].

To balance the accuracy and speed, this paper proposes a high-speed 3D shape measurement method based on Fourier transform profilometry and stereo vision. Instead of using 8-bit gray sinusoidal fringe pattern, 1-bit dithering fringe pattern is used. Because it is binary pattern, it can resolve the problem caused by gamma effect by using defocusing technology. In this paper, the fringe pattern is generated based on reference [16]. By using the property of stereo vision, original image matching constraint can get a rough parallax. In order to get the accurate corresponding point, phase matching and sub-pixel parallax optimization are used to preclude false points. Phase matching is used to confirm candidate points. When points are at the edge of wrapped phase where the phase value is −*π*or *π*, there are missing points or wrong corresponding points. An average phase value is used to increase the robustness of the high-speed measurement. Sub-pixel parallax optimization is used to find the true corresponding point based on the coordinate of sub-pixel.

This paper is organized as follows. In Section Methods, the flow of the proposed method and the principle of the proposed method are introduced. Section Results and discussion verifies the feasibility and accuracy of the proposed method. By comparing measurement results, the matching precision is confirmed. Section Conclusion gives the summary of this paper.

## Methods

### The process of the proposed method

Step 1. Preparation before measurement. It includes the generation of dithering pattern and stereo vision calibration. The dithering fringe pattern is insensitive to the gamma of projector which uses 1-bit binary instead of 8-bit gray information to approximate the sinusoidal fringe pattern. Dithering fringe pattern can be used for high speed measurement without projector calibration. Another merit of dithering fringe pattern is that it is suitable to measure objects when wide fringe pattern is used. In this paper, the dithering fringe pattern is generated based on the reference [16].

Step 2. Original image matching constraint. The original image can be captured from left and right cameras respectively. Based on the feature of stereo vision, a rough matching can be implemented. It can be used to provide a rough parallax as a constraint condition.

Step 3. Calculation of the wrapped phase. FTP is applied to the captured image from left camera and right camera respectively. Then the fundamental component is extracted by using a filter and inverse FTP is used to obtain the wrapped phase.

Step 4. The wrapped phase matching. Based on the original image matching constraint, the wrapped phase matching can be performed to find the candidate points. Because FTP uses only one frame of fringe pattern to get the phase map, perhaps there are some phase errors at the boundary of wrapped phase where the phase is or ‐*π*or *π*.

Step 5. Sub-pixel parallax optimization. In order to find the corresponding points precisely, the sub-pixel parallax optimization is used. The optimized parallax will correspond the target point. Once the stereo vision system is calibrated, the height of object can be calculated.

In this paper, Step 4 and Step 5 are the mainly proposed method so the principle of them will be introduced in the following part.

### The principle of the proposed method

*π*to

*π*with periodical change. Because the dithering fringe pattern is used in this system, the gamma effect of projector can be neglected. The original image matching constraint narrows the range of candidate points within the epipolar line. Without original image matching constraint, the false points will be considered and reconstructed which will occupy the process time [21]. Traditionally the absolute phase is required to find the correct corresponding points. The algorithm of retrieving absolute phase can be classified into spatial method and temporal method. But both methods have demerits. Spatial method can not be used to retrieve the phase of isolate objects and the phase error will spread along the direction of phase unwrapping. Temporal method need multiple frames to unwrap the phase which will occupy the measurement speed. The process of wrapped phase matching without any absolute phase is shown in Fig.2. Because absolute phase is not used, there are some false corresponding points with the same phase value. These points can be defined as candidate points which are shown in Fig.2a.

*x*

_{L},

*y*

_{L}) in the left camera corresponds the point (

*x*

_{R},

*y*

_{R}) in the right camera, as shown in Fig.2b. The original image matching constraint provides a rough corresponding parallax

*Par*, which can be expressed as:

*x*

_{L},

*y*

_{L}) in the left wrapped phase is

*Phase*

_{L}(

*x*

_{L},

*y*

_{L}), the corresponding phase of right wrapped phase is

*Phase*

_{R}(

*x*

_{R},

*y*

_{R}), as shown in Eq.(2):

*x*

_{R},

*y*

_{R}) and adjacent points are set as candidate points. Considering the computation complexity, adjacent candidate points are shown in Fig.3.

*Phase*

_{L}(

*x*

_{L},

*y*

_{L}) subtracts the phase of these candidate points

*Phase*

_{R}(

*x*

_{R}+

*s*,

*y*

_{R}) and the absolute difference can be expressed as:

Where *s* is an integer and *s* ∈ [−2, 2]. Δ*Phase* is the difference of the points phase. The least difference Δ*Phase*(*x*_{R _ min}, *y*_{R _ min}) is used and (*x*_{R _ min}, *y*_{R _ min}) is the coordinate of candidate point which has the least phase difference.

As mentioned above, because two cameras capture the deformed fringe patterns, the wrapped phase can be generated with deformed information. Based on the principle of stereo vision, the parallax can be used in the wrapped phase, as shown in Eq.(2). When the parallax is applied into the phase matching, a basic point can be obtained. The proposed method chooses the basic matching point and its two left-and-right neighborhood points as candidate matching points. The wrapped phase is monotonous in a period so these candidate points have different phase value. Theoretically the basic point and true point should have the closest phase value so the point with the least difference is used as the optimal point, as shown in Eq.(3).

*Par*

_{min}can be calculated as:

Phase value matching belongs to pixel matching and the parallax of sub-pixel is required.

*π*or

*π*, it is difficult to find the matching points precisely based on Δ

*Phase*. In order to find the corresponding points accurately, the average phase value is used. As shown in Fig.3, the middle point (

*x*

_{R},

*y*

_{R}) is on the edge of wrapped phase and the phase

*Phase*

_{R}(

*x*

_{R},

*y*

_{R}) is required to be calculated. The average phase value

*Phase*

_{ave}(

*x*

_{R},

*y*

_{R}) can be described as:

*Phase*

_{ave}(

*x*

_{R},

*y*

_{R}) is greater than zero, it can be designed as

*π*otherwise it can be designed as −

*π*which can be expressed as:

*π*to

*π*and it changes periodically. By using this feature, the coordinate of sub-pixel can be obtained. If the coordinate of original image matching point is (

*x*

_{R _ o},

*y*

_{R}), the coordinate of sub-pixel (

*x*

_{R _ sub},

*y*

_{R}) can be obtained. Because the phase value is monotone in every period, the coordinate of sub-pixel (

*x*

_{R _ sub},

*y*

_{R}) is located between the best corresponding point (

*x*

_{R _ min},

*y*

_{R _ min}) and candidate point (

*x*

_{R _ o},

*y*

_{R}). The phase value of the sub-pixel in the right wrapped phase can be set to

*Phase*

_{R}(

*x*

_{R _ sub},

*y*

_{R}). Because

*Phase*

_{R}(

*x*

_{R _ sub},

*y*

_{R}) represents the best matching point in the wrapped phase, it should be equivalent to the phase

*Phase*

_{L}(

*x*

_{L},

*y*

_{L}) in the left wrapped phase. Then it can be expressed as:

*x*

_{R _ o},

*y*

_{R}) is on the left of the best point (

*x*

_{R _ min},

*y*

_{R _ min}), it means

*x*

_{R _ o}<

*x*

_{R _ min}. Because phase information is robust to the noise from the texture on the surface, the phase difference and ratio is used to find the coordinate of sub-pixel. The phase difference between (

*x*

_{R _ sub},

*y*

_{R})and (

*x*

_{R _ o},

*y*

_{R}) can be calculated as

*Phase*

_{R}(

*x*

_{R _ sub},

*y*

_{R}) −

*Phase*

_{R}(

*x*

_{R _ o},

*y*

_{R}). Similarly the phase difference between (

*x*

_{R _ min},

*y*

_{R}) and (

*x*

_{R _ o},

*y*

_{R}) can also be expressed as

*Phase*

_{R}(

*x*

_{R _ min},

*y*

_{R}) −

*Phase*

_{R}(

*x*

_{R _ o},

*y*

_{R}). Instead of calculating the ratio between the different x-coordinate, the ratio of the phase difference is used. Based on the relationship, the coordinate of sub-pixel can be expressed as:

*x*

_{R _ o},

*y*

_{R}) has the same x-coordinate of the best point (

*x*

_{R _ min},

*y*

_{R _ min}), it means

*x*

_{R _ o}=

*x*

_{R _ min}. As shown in Fig.4b, the coordinate of sub-pixel can be expressed as:

When the original image matching point (*x*_{R _ o}, *y*_{R}) is on the right of the best point (*x*_{R _ min}, *y*_{R _ min}), it means *x*_{R _ o} > *x*_{R _ min}, as shown in Fig.4c.

*ParPhase*

_{sub}can be obtained as:

Based on the baseline and sub-pixel parallax, the height of the object can be calculated.

## Results and Discussion

In order to verify the proposed method, a 3D shape measurement system is developed. The system contains a projector (Samsung SP-P310MEMX) and two digital CCD cameras (Daheng MER-500-14U3M/C-L). Every camera is attached with a 16 mm focal length lens (Computar M1614-MP) and every camera resolution is 1024 × 768. The projector resolution is 800 × 600 and it has 0.49–2.80 m projection distance.

The blocks measurement results comparison from MFH. (Units:mm)

Objects | Ideal height | 30 | 50 | 60 |
---|---|---|---|---|

Discontinuous | Average. height | 28.43 | 52.11 | 61.92 |

Blocks | RMS | 0.77 | 0.58 | 0.43 |

Measurement | Average error | 0.52 | 0.53 | 0.39 |

Maximum error | 0.58 | 0.71 | 0.47 | |

Continuous | Average. height | 28.35 | 52.32 | 61.81 |

Blocks | RMS | 0.75 | 0.58 | 0.45 |

Measurement | Average error | 0.53 | 0.53 | 0.36 |

Maximum error | 0.60 | 0.75 | 0.45 |

The blocks measurement results comparison from the proposed method. (Units:mm)

Objects | Ideal height | 30 | 50 | 60 |
---|---|---|---|---|

Discontinuous | Average. height | 30.21 | 50.23 | 59.28 |

Blocks | RMS | 0.34 | 0.31 | 0.25 |

Measurement | Average error | 0.40 | 0.36 | 0.24 |

Maximum error | 0.46 | 0.41 | 0.37 | |

Continuous | Average. height | 30.15 | 50.19 | 60.01 |

Blocks | RMS | 0.33 | 0.35 | 0.24 |

Measurement | Average error | 0.41 | 0.35 | 0.26 |

Maximum error | 0.48 | 0.43 | 0.40 |

## Conclusion

This paper proposes a high-speed 3D shape measurement algorithm based on FTP and stereo vision. It can use one-shot fringe pattern to get wrapped phase and 3D shape can be reconstructed without phase unwrapping. The original image matching constraint and sub-pixel phase matching are used to find the correct corresponding points. In order to increase the robustness and reduce missing rate of the boundary of the wrapped phase where phase value is *π* or −*π*, the average value is used based on the adjacent phase value. By comparing the matching precision, the proposed method can get high quality surface. Because the process of measurement only need one frame of deformed fringe pattern to get the wrapped phase and the matching process does not need phase unwrapping, the proposed method can be used in fast measurement.

## Declarations

### Acknowledgments

This work was supported by National Key R&D Program of China (2017YBF1300900), the the National Natural Science Foundation of China (U1713216), the Fund of Shenyang (17-87-0-00) and the Fundamental Research Funds for the Central Universities(N172604004).

### Funding

Faculty of Robot Science and Engineering of Northeastern University, Shenyang, China provides the funding for this research.

### Availability of data and materials

Details data has been provided in this paper.

### Authors’ contributions

All authors contributed equally in all the sections of this work. All authors read and approved the final manuscript.

### Authors’ information

Feng Lu, a doctor of Faculty of Robot Science and Engineering, Northeastern University, Shenyang, China. His interest includes Structured light, Robot and image processing.

### Ethics approval and consent to participate

Not applicable.

### Consent for publication

Not applicable.

### Competing interests

The authors declare that they have no competing interests.

### Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

## Authors’ Affiliations

## References

- Lin, H., Gao, J., Mei, Q., et al.: Three-dimensional shape measurement technique for shiny surfaces by adaptive pixel-wise projection intensity adjustment[J]. Opt. Lasers Eng.
**91**, 206–215 (2017)View ArticleGoogle Scholar - Kaura, S.K., Chhachhia, D.P., Aggarwal, A.K.: Interferometric moiré pattern encoded security holograms[J]. J. Opt. A Pure Appl. Opt.
**8**(1), 67 (2006)ADSView ArticleGoogle Scholar - Peng, J., Liu, X., Deng, D., et al.: Suppression of projector distortion in phase-measuring profilometry by projecting adaptive fringe patterns[J]. Opt. Express.
**24**(19), 21846 (2016)ADSView ArticleGoogle Scholar - Zhou, P., Liu, X., He, Y., et al.: Phase error analysis and compensation considering ambient light for phase measuring profilometry[J]. Opt. Lasers Eng.
**55**(7), 99–104 (2014)View ArticleGoogle Scholar - Zhang, Z., Wang, Y., Huang, S., et al.: Three-dimensional shape measurements of specular objects using phase-measuring Deflectometry:[J]. Sensors.
**17**(12), 2835 (2017)View ArticleGoogle Scholar - Lee, H., Min, Y.K., Moon, J.I.: Three-dimensional sensing methodology combining stereo vision and phase-measuring profilometry based on dynamic programming[J]. Opt. Eng.
**56**(12), 1 (2017)View ArticleGoogle Scholar - Yun, H., Li, B., Zhang, S.: Pixel-by-pixel absolute three-dimensional shape measurement with modified Fourier transform profilometry[J]. Appl. Opt.
**56**(5), 1472 (2017)ADSView ArticleGoogle Scholar - Li, H., Hu, Y., Tao, T., et al.: Optimal wavelength selection strategy in temporal phase unwrapping with projection distance minimization[J]. Appl. Opt.
**57**(10), 2352 (2018)ADSView ArticleGoogle Scholar - Sun, W., Wang, T., Zhao, Y., et al.: Advanced method of global phase shift estimation from two linear carrier interferograms[J]. J. Eur. Opt. Soc. Rapid Publ.
**14**(1), 10 (2018)View ArticleGoogle Scholar - Hu, Y., Chen, Q., Zhang, Y., et al.: Dynamic microscopic 3D shape measurement based on marker-embedded Fourier transform profilometry[J]. Appl. Opt.
**57**(4), 772 (2018)ADSView ArticleGoogle Scholar - Zhou, Y., Tang, Y., Yang, Y., et al.: Topography measurement of large-range microstructures through advanced Fourier-transform method and phase stitching in scanning broadband light interferometry[J]. Micromachines.
**8**(11), 319 (2017)View ArticleGoogle Scholar - Su, X., Zhang, Q.: Dynamic 3-D shape measurement method: a review[J]. Opt. Lasers Eng.
**48**(2), 191–204 (2010)ADSView ArticleGoogle Scholar - Li, B., Zhang, S.: High-resolution, real-time to superfast 3D imaging techniques[C]// IEEE international conference on advanced intelligent mechatronics. IEEE. 1252–1257 (2016)Google Scholar
- Jeught, S.V.D., Dirckx, J.J.J.: Real-time structured light profilometry: a review[J]. Opt. Lasers Eng.
**87**, 18–31 (2016)View ArticleGoogle Scholar - Zhang, Z.H.: Review of single-shot 3D shape measurement by phase calculation-based fringe projection techniques[J]. Opt. Lasers Eng.
**50**(8), 1097–1106 (2012)View ArticleGoogle Scholar - Xiao, Y., Li, Y.: High-quality binary fringe generation via joint optimization on intensity and phase[J]. Opt. Lasers Eng.
**97**, 19–26 (2017)View ArticleGoogle Scholar - Zhang, S.: Comparative study on passive and active projector nonlinear gamma calibration[J]. Appl. Opt.
**54**(13), 3834–3841 (2015)ADSView ArticleGoogle Scholar - Lu, F., Wu, C.: Three-dimensional measurement of object surface by using ellipse binary defocusing projection. Journal of the European Optical Society-Rapid Publications. 13(1), 29 (2017)Google Scholar
- Dai, J., An, Y., Zhang, S.: Absolute three-dimensional shape measurement with a known object[J]. Opt. Express.
**25**(9), 10384 (2017)ADSView ArticleGoogle Scholar - Geiger, A., Roser, M., Urtasun, R.: Efficient large-scale stereo matching. Comput. Vision – Accv.
**6492**, 25–38 (2011)Google Scholar - Li, Z.W., Shi, Y.S., Wang, C.J., Wang, Y.Y., et al.: Accurate calibration method for a structured light system. Opt. Eng.
**47**, 053604 (2008)ADSView ArticleGoogle Scholar