### Phase-to-height relationship by LSM

#### Geometric analysis

Figure 1 illustrates a typical setup of a generalized FPP system [8]. The reference plane *Oxy*, camera imaging plane *O’x’y’* , and projection plane *O”x”y”* are arbitrarily arranged. *P* represents an arbitrary point on the object, *B* is the imaging point of *P*, *D* is the original fringe point projected at *P*, and *A* and *C* are the lens centers of the camera and the projector, respectively. For convenience and clarification, the coordinates of a point in a coordinate system are denoted by the corresponding coordinate symbols, and the symbol of the point is chosen as the subscript. For example, point *P* is denoted as (*x*
_{
p
}, *y*
_{
p
}, *z*
_{
p
}), (*x*
_{
p
}’ , *y*
_{
p
}’, *z*
_{
p
}’), and (*x*
_{
p
}”, *y*
_{
p
}”, *z*
_{
p
}”) in coordinate systems *Oxyz*, *O’x’y’z’* , and *O”x”y”z”*, respectively. Based on the coordinate relations among points *P*, *A*, and *B* in the system *Oxyz*, we obtain:

$$ \frac{x_P-{x}_A}{x_B-{x}_A}=\frac{y_P-{y}_A}{y_B-{y}_A}=\frac{z_P-{z}_A}{z_B-{z}_A}. $$

(1)

A typical coordinate transformation of point *B* from system *O’x’y’z’* to system *Oxyz* is as follows:

$$ \left[\begin{array}{c}\hfill {x}_B\hfill \\ {}\hfill {y}_B\hfill \\ {}\hfill {z}_B\hfill \end{array}\right]=\left[\begin{array}{c}\hfill {x}_{O\hbox{'}}\hfill \\ {}\hfill {y}_{O\hbox{'}}\hfill \\ {}\hfill {z}_{O\hbox{'}}\hfill \end{array}\right]+ Rot\left(z,\gamma \right) Rot\left(y,\beta \right) Rot\left(x,\alpha \right)\left[\begin{array}{c}\hfill {x}_B^{\prime}\hfill \\ {}\hfill {y}_B^{\prime}\hfill \\ {}\hfill {z}_B^{\prime}\hfill \end{array}\right], $$

(2)

where *Rot*(*z*, *γ*), *Rot*(*y*, *β*), and *Rot*(*x*, *α*) are the coordinate transformation matrices, and *α*, *β*, and *γ* are the rotation angles of the *x*’ , *y*’ , and *z*’ axes based on the reference coordinate system *Oxyz*, respectively. Finally, the height of the object based on the least squares method is as follows [8]:

$$ {z}_p=\frac{1+{c}_1\phi +\left({c}_2+{c}_3\phi \right){x}_B^{\prime }+\left({c}_4+{c}_5\phi \right){y}_B^{\prime }}{d_0+{d}_1\phi +\left({d}_2+{d}_3\phi \right){x}_B^{\prime }+\left({d}_4+{d}_5\phi \right){y}_B^{\prime }}, $$

(3)

where *z*
_{
p
} is the out-of-reference-plane height at point (*x,y,z*) on the object, coefficients *c*
_{1} to *c*
_{5} and *d*
_{0} to *d*
_{5} are constant coefficients that need to be determined from geometric information such as the position and direction of the camera and the projector, and *ϕ* is the fringe phase at the same point. Eq. (3) cannot consider the lens distortion because it is derived from geometric analysis. Generally, if the lens distortion is not considered, it is not difficult to reduce the modeling error for the phase-to-height relationship. Because the lens distortion consists of the radial lens distortion and the tangential lens distortion, the new normalized point coordinate (*x*
_{
d
}, *y*
_{
d
}) is defined as follows [10]:

$$ {x}_d={x}_B^{\prime }+d{x}_r+d{x}_t, $$

(4)

$$ {y}_d={y}_B^{\prime }+d{y}_r+d{y}_t, $$

(5)

where *dx*
_{
r
} and *dy*
_{
r
} are the position error caused by the radial lens distortion, while *dx*
_{
t
} and *dy*
_{
t
} are the position error from the tangential lens distortion, which are defined as:

$$ d{x}_r=\left[{k}_{c1}{R}^2+{k}_{c2}{R}^4\right]{x}_B^{\prime }, $$

(6)

$$ d{y}_r=\left[{k}_{c1}{R}^2+{k}_{c2}{R}^4\right]{y}_B^{\prime }, $$

(7)

$$ d{x}_t=2{k}_{c3}{x}_B^{\prime }{y}_B^{\prime }+{k}_{c4}\left({R}^2+2{x^{\prime}}_B^2\right), $$

(8)

$$ d{y}_t={k}_{c3}\left({R}^2+2{y^{\prime}}_B^2\right)+2{k}_{c4}{x}_B^{\prime }{y}_B^{\prime }. $$

(9)

where *k*
_{
c1}, *k*
_{
c2}, *k*
_{
c3}, and *k*
_{
c4} are the constant coefficients of camera lens distortion, and *R* is defined as \( {R}^2={x^{\prime}}_B^2+{y^{\prime}}_B^2 \). Since the radial distortion is usually much larger than the tangential distortion in modern optics, *k*
_{
c3} and *k*
_{
c4} can be reasonably discarded. It is very difficult to calculate the inverse function for the variables *x’*
_{
B
} and *y’*
_{
B
} because the tangential distortion includes the term of the multiplication of *x’*
_{
B
} and *y’*
_{
B
}. Therefore, if the tangential distortion (*dx*
_{
t
} and *dy*
_{
t
}) is ignored, the point coordinate is simplified as:

$$ {x}_B^{\prime }=\frac{x_d}{1+{k}_{c1}{R}^2+{k}_{c2}{R}^4}\approx \left[1-{k}_{c1}{R}^2-{k}_{c2}{R}^4\right]{x}_d, $$

(10)

$$ {y}_B^{\prime }=\frac{y_d}{1+{k}_{c1}{R}^2+{k}_{c2}{R}^4}\approx \left[1-{k}_{c1}{R}^2-{k}_{c2}{R}^4\right]{y}_d $$

(11)

The height of the object considering the radial distortion is rewritten as follows [10]:

$$ {z}_p=\frac{\begin{array}{l}1+{c}_1\phi +\left({c}_2+{c}_3\phi \right){x}_d+\left({c}_4+{c}_5\phi \right){y}_d+\left({c}_6+{c}_7\phi \right){R}^2{x}_d\\ {}+\left({c}_8+{c}_9\phi \right){R}^2{y}_d+\left({c}_{10}+{c}_{11}\phi \right){R}^4{x}_d+\left({c}_{12}+{c}_{13}\phi \right){R}^4{y}_d\end{array}}{\begin{array}{l}{d}_0+{d}_1\phi +\left({d}_2+{d}_3\phi \right){x}_d+\left({d}_4+{d}_5\phi \right){y}_d+\left({d}_6+{d}_7\phi \right){R}^2{x}_d\\ {}+\left({d}_8+{d}_9\phi \right){R}^2{y}_d+\left({d}_{10}+{d}_{11}\phi \right){R}^4{x}_d+\left({d}_{12}+{d}_{13}\phi \right){R}^4{y}_d\end{array}}, $$

(12)

where the coefficients *c*
_{0} to *c*
_{13} and *d*
_{0} to *d*
_{13} are constants. The coefficients should be determined in advance using the least-squares method based on the phase data for the several reference planes for which the heights are known.

#### Experimental results

Figure 2 shows 3-D measurement equipment using cameras and a beam projector [15]. The equipment consists of black and white CCD cameras (AOS MPX1350, 1280× 1024 pixels, and 8-bit data depth), a digital light processing (DLP) projector (LG HS200G, 800 × 600 pixels), a personal computer for image processing, and a three-axis stage for camera calibration. The stage has a repeatability accuracy of 0.001 mm, and the z-axis is only used to obtain focuses of the projector and camera during the calibration for 3-D measurement. In the experiments, the basic period of the fringe pattern was set to eight pixels, and the eight-bucket algorithm with eight different phases was used in the phase shift method. Because the horizontal resolution of the projector is 800 pixels, the gray code patterns of seven bits were necessary to distinguish 100 different periods. The measuring range was set to 100 × 100 × 50 mm for the *x*-*y*-*z* axis because the focal depth of the *z*-axis is relatively sensitive to the *x*- and *y*-axes.

A training set is necessary for the training process of the coefficients in the LSM and is formed by training pair vectors. In the phase-to-height relationship, 3-D points are used as the training pair vectors. To obtain the related coefficients, the LSM needs more training pair vectors than the number of the coefficients. Eqs. (3) and (12) need at least 11 and 27 training pair vectors, respectively. If the training pair vectors use 5 different heights for the z-axis (every 12.5 mm from -25 to +25 mm) and 5 different positions with 200 pixel intervals for each axis of the camera, the number of the training members *N* is 125 (5 × 5 × 5). Using the training set, each coefficient is adjusted to minimize the modeling error by the gradient descent method. However, it is very easy to obtain the coefficients if the pseudo-inverse matrix in Matlab is used for the 125 points.

The camera coordinates (*x*
_{
d
}, *y*
_{
d
}) are obtained by normalizing the camera pixel coordinates (*u*, *v*) for the camera center. Figure 3 shows the modeling errors of the horizontal line (v = 500) in the phase-to-height relationship obtained using Eq. (3). The modeling error is the difference between the real height and the modeling height. Because lens distortion is neglected in Eq. (3), the errors greatly change according to the *u*-coordinate. Eq. (3) is not suitable as a phase-to-height model because the maximum errors are as large as ±2 mm. Eq. (12) considers the radial lens distortion. Figure 4 shows the modeling errors for when 27 coefficients of Eq. (12) are obtained using 125 reference points and Matlab functions, and the errors are fairly reduced to ±0.15 mm. However, even with 6,237 (63 × 9 × 11) more reference points than 125, the modeling errors do not improve any further. For less error, it is necessary to derive an equation that includes the tangential lens distortion, which is very difficult to achieve using geometric analysis.