- Open Access
High-speed 3D shape measurement using Fourier transform and stereo vision
© The Author(s) 2018
- Received: 28 May 2018
- Accepted: 24 September 2018
- Published: 3 October 2018
In the fast 3D shape measurement, it is an important factor to use the least number of fringe patterns to get the wrapped phase and the wrapped phase is always required to be retrieved to absolute phase. But the process of phase unwrapping may affect the quality of absolute phase. Besides absolute phase retrieval is time-costing especially for high-speed 3D shape measurement.
This paper proposes a fast measurement method based on Fourier transform profilometry and stereo vision. Because of the property of stereo vision, every point can find its corresponding point in the wrapped phase. This method can reconstruct 3D surface without phase unwrapping. In order to further increase the measurement speed and overcome the gamma effect of projector, the dithering fringe pattern is used. To resist false matching, the original image matching constraint gives a rough parallax. Phase matching and sub-pixel parallax optimization are used to reduce the matching errors. In order to confirm the phase value of points which are at the edge of wrapped phase, average phase value is calculated.
A white mask is measured based on the proposed method. The results from every step show the effect of different functions. To better show the effect of the proposed method, a white house is measured and the measurement is compared with LSSM and MFH. In order to further to show the advantage of the proposed method, discontinuous blocks and continuous blocks are measured. The average height, RMS, average error, maximum error are compared with the method which uses absolute phase to match the corresponding points. Experiment results verify the feasibility of the proposed method and it can measure complex objects with high speed and accuracy.
This paper uses Fourier transform and stereo vision to get wrapped phase and reconstruct 3D shape without phase unwrapping. The proposed method contains three part: phase matching, edge points detection and sub-pixel parallax optimization. By comparing matching precision, the proposed method can get high quality surface. Because the process of measurement only need one frame of deformed fringe pattern to get the wrapped phase and the matching process does not need phase unwrapping, the proposed method has the potential to be used in fast measurement.
- Fringe analysis
- Three-dimensional shape measurement
- Fourier transform
- Stereo vision
High-speed 3D shape measurement has been an important technology in the industrial manufacturing such as quality inspection, reverse engineering, 3-D sensing, object recognition, etc. . In order to get accurate 3D shape measurement results, a lot of technologies have been researched such as Moire technique (MT) , phase-measuring profilometry (PMP) [3–6], and Fourier transform profilometry (FTP) [7–9].
In general, multi-frame fringe patterns can get more accurate and reliable results, but it is significant to use fewer fringe patterns to get the results in high-speed measurement . Among these technologies, Fourier transform profilometry only need one frame of fringe pattern to get the wrapped phase, so it is suitable to be used in high-speed measurement .
Though FTP has the advantage, it still has shortcomings. Because FTP only need one frame of fringe pattern, it is sensitive to the quality of fringe pattern [12–15]. It is always affected by the nonlinearity of projector and the wrapped phase will exist noise which will have impact on the measurement directly . Moreover, gamma calibration is required but this process will increase the computation complexity . The phase error compensation is also required to remove the phase error of reconstruction results . But there is not a standard method which can be used in different applications. Another problem which FTP suffers from is that the wrapped phase need to be unwrapped to get the absolute phase. However, the process of phase unwrapping will also affect the accuracy of absolute phase and it will also increase computing time of the measurement .
To balance the accuracy and speed, this paper proposes a high-speed 3D shape measurement method based on Fourier transform profilometry and stereo vision. Instead of using 8-bit gray sinusoidal fringe pattern, 1-bit dithering fringe pattern is used. Because it is binary pattern, it can resolve the problem caused by gamma effect by using defocusing technology. In this paper, the fringe pattern is generated based on reference . By using the property of stereo vision, original image matching constraint can get a rough parallax. In order to get the accurate corresponding point, phase matching and sub-pixel parallax optimization are used to preclude false points. Phase matching is used to confirm candidate points. When points are at the edge of wrapped phase where the phase value is −πor π, there are missing points or wrong corresponding points. An average phase value is used to increase the robustness of the high-speed measurement. Sub-pixel parallax optimization is used to find the true corresponding point based on the coordinate of sub-pixel.
This paper is organized as follows. In Section Methods, the flow of the proposed method and the principle of the proposed method are introduced. Section Results and discussion verifies the feasibility and accuracy of the proposed method. By comparing measurement results, the matching precision is confirmed. Section Conclusion gives the summary of this paper.
The process of the proposed method
Step 1. Preparation before measurement. It includes the generation of dithering pattern and stereo vision calibration. The dithering fringe pattern is insensitive to the gamma of projector which uses 1-bit binary instead of 8-bit gray information to approximate the sinusoidal fringe pattern. Dithering fringe pattern can be used for high speed measurement without projector calibration. Another merit of dithering fringe pattern is that it is suitable to measure objects when wide fringe pattern is used. In this paper, the dithering fringe pattern is generated based on the reference .
Step 2. Original image matching constraint. The original image can be captured from left and right cameras respectively. Based on the feature of stereo vision, a rough matching can be implemented. It can be used to provide a rough parallax as a constraint condition.
Step 3. Calculation of the wrapped phase. FTP is applied to the captured image from left camera and right camera respectively. Then the fundamental component is extracted by using a filter and inverse FTP is used to obtain the wrapped phase.
Step 4. The wrapped phase matching. Based on the original image matching constraint, the wrapped phase matching can be performed to find the candidate points. Because FTP uses only one frame of fringe pattern to get the phase map, perhaps there are some phase errors at the boundary of wrapped phase where the phase is or ‐πor π.
Step 5. Sub-pixel parallax optimization. In order to find the corresponding points precisely, the sub-pixel parallax optimization is used. The optimized parallax will correspond the target point. Once the stereo vision system is calibrated, the height of object can be calculated.
In this paper, Step 4 and Step 5 are the mainly proposed method so the principle of them will be introduced in the following part.
The principle of the proposed method
Where s is an integer and s ∈ [−2, 2]. ΔPhase is the difference of the points phase. The least difference ΔPhase(xR _ min, yR _ min) is used and (xR _ min, yR _ min) is the coordinate of candidate point which has the least phase difference.
As mentioned above, because two cameras capture the deformed fringe patterns, the wrapped phase can be generated with deformed information. Based on the principle of stereo vision, the parallax can be used in the wrapped phase, as shown in Eq.(2). When the parallax is applied into the phase matching, a basic point can be obtained. The proposed method chooses the basic matching point and its two left-and-right neighborhood points as candidate matching points. The wrapped phase is monotonous in a period so these candidate points have different phase value. Theoretically the basic point and true point should have the closest phase value so the point with the least difference is used as the optimal point, as shown in Eq.(3).
Phase value matching belongs to pixel matching and the parallax of sub-pixel is required.
When the original image matching point (xR _ o, yR) is on the right of the best point (xR _ min, yR _ min), it means xR _ o > xR _ min, as shown in Fig.4c.
Based on the baseline and sub-pixel parallax, the height of the object can be calculated.
In order to verify the proposed method, a 3D shape measurement system is developed. The system contains a projector (Samsung SP-P310MEMX) and two digital CCD cameras (Daheng MER-500-14U3M/C-L). Every camera is attached with a 16 mm focal length lens (Computar M1614-MP) and every camera resolution is 1024 × 768. The projector resolution is 800 × 600 and it has 0.49–2.80 m projection distance.
The blocks measurement results comparison from MFH. (Units:mm)
The blocks measurement results comparison from the proposed method. (Units:mm)
This paper proposes a high-speed 3D shape measurement algorithm based on FTP and stereo vision. It can use one-shot fringe pattern to get wrapped phase and 3D shape can be reconstructed without phase unwrapping. The original image matching constraint and sub-pixel phase matching are used to find the correct corresponding points. In order to increase the robustness and reduce missing rate of the boundary of the wrapped phase where phase value is π or −π, the average value is used based on the adjacent phase value. By comparing the matching precision, the proposed method can get high quality surface. Because the process of measurement only need one frame of deformed fringe pattern to get the wrapped phase and the matching process does not need phase unwrapping, the proposed method can be used in fast measurement.
This work was supported by National Key R&D Program of China (2017YBF1300900), the the National Natural Science Foundation of China (U1713216), the Fund of Shenyang (17-87-0-00) and the Fundamental Research Funds for the Central Universities(N172604004).
Faculty of Robot Science and Engineering of Northeastern University, Shenyang, China provides the funding for this research.
Availability of data and materials
Details data has been provided in this paper.
All authors contributed equally in all the sections of this work. All authors read and approved the final manuscript.
Feng Lu, a doctor of Faculty of Robot Science and Engineering, Northeastern University, Shenyang, China. His interest includes Structured light, Robot and image processing.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Lin, H., Gao, J., Mei, Q., et al.: Three-dimensional shape measurement technique for shiny surfaces by adaptive pixel-wise projection intensity adjustment[J]. Opt. Lasers Eng. 91, 206–215 (2017)View ArticleGoogle Scholar
- Kaura, S.K., Chhachhia, D.P., Aggarwal, A.K.: Interferometric moiré pattern encoded security holograms[J]. J. Opt. A Pure Appl. Opt. 8(1), 67 (2006)ADSView ArticleGoogle Scholar
- Peng, J., Liu, X., Deng, D., et al.: Suppression of projector distortion in phase-measuring profilometry by projecting adaptive fringe patterns[J]. Opt. Express. 24(19), 21846 (2016)ADSView ArticleGoogle Scholar
- Zhou, P., Liu, X., He, Y., et al.: Phase error analysis and compensation considering ambient light for phase measuring profilometry[J]. Opt. Lasers Eng. 55(7), 99–104 (2014)View ArticleGoogle Scholar
- Zhang, Z., Wang, Y., Huang, S., et al.: Three-dimensional shape measurements of specular objects using phase-measuring Deflectometry:[J]. Sensors. 17(12), 2835 (2017)View ArticleGoogle Scholar
- Lee, H., Min, Y.K., Moon, J.I.: Three-dimensional sensing methodology combining stereo vision and phase-measuring profilometry based on dynamic programming[J]. Opt. Eng. 56(12), 1 (2017)View ArticleGoogle Scholar
- Yun, H., Li, B., Zhang, S.: Pixel-by-pixel absolute three-dimensional shape measurement with modified Fourier transform profilometry[J]. Appl. Opt. 56(5), 1472 (2017)ADSView ArticleGoogle Scholar
- Li, H., Hu, Y., Tao, T., et al.: Optimal wavelength selection strategy in temporal phase unwrapping with projection distance minimization[J]. Appl. Opt. 57(10), 2352 (2018)ADSView ArticleGoogle Scholar
- Sun, W., Wang, T., Zhao, Y., et al.: Advanced method of global phase shift estimation from two linear carrier interferograms[J]. J. Eur. Opt. Soc. Rapid Publ. 14(1), 10 (2018)View ArticleGoogle Scholar
- Hu, Y., Chen, Q., Zhang, Y., et al.: Dynamic microscopic 3D shape measurement based on marker-embedded Fourier transform profilometry[J]. Appl. Opt. 57(4), 772 (2018)ADSView ArticleGoogle Scholar
- Zhou, Y., Tang, Y., Yang, Y., et al.: Topography measurement of large-range microstructures through advanced Fourier-transform method and phase stitching in scanning broadband light interferometry[J]. Micromachines. 8(11), 319 (2017)View ArticleGoogle Scholar
- Su, X., Zhang, Q.: Dynamic 3-D shape measurement method: a review[J]. Opt. Lasers Eng. 48(2), 191–204 (2010)ADSView ArticleGoogle Scholar
- Li, B., Zhang, S.: High-resolution, real-time to superfast 3D imaging techniques[C]// IEEE international conference on advanced intelligent mechatronics. IEEE. 1252–1257 (2016)Google Scholar
- Jeught, S.V.D., Dirckx, J.J.J.: Real-time structured light profilometry: a review[J]. Opt. Lasers Eng. 87, 18–31 (2016)View ArticleGoogle Scholar
- Zhang, Z.H.: Review of single-shot 3D shape measurement by phase calculation-based fringe projection techniques[J]. Opt. Lasers Eng. 50(8), 1097–1106 (2012)View ArticleGoogle Scholar
- Xiao, Y., Li, Y.: High-quality binary fringe generation via joint optimization on intensity and phase[J]. Opt. Lasers Eng. 97, 19–26 (2017)View ArticleGoogle Scholar
- Zhang, S.: Comparative study on passive and active projector nonlinear gamma calibration[J]. Appl. Opt. 54(13), 3834–3841 (2015)ADSView ArticleGoogle Scholar
- Lu, F., Wu, C.: Three-dimensional measurement of object surface by using ellipse binary defocusing projection. Journal of the European Optical Society-Rapid Publications. 13(1), 29 (2017)Google Scholar
- Dai, J., An, Y., Zhang, S.: Absolute three-dimensional shape measurement with a known object[J]. Opt. Express. 25(9), 10384 (2017)ADSView ArticleGoogle Scholar
- Geiger, A., Roser, M., Urtasun, R.: Efficient large-scale stereo matching. Comput. Vision – Accv. 6492, 25–38 (2011)Google Scholar
- Li, Z.W., Shi, Y.S., Wang, C.J., Wang, Y.Y., et al.: Accurate calibration method for a structured light system. Opt. Eng. 47, 053604 (2008)ADSView ArticleGoogle Scholar