Three-step phase-shifting algorithm
In this paper, three-step phase-shifting algorithm with a phase shift 2π/3is applied. Because it need the least formulas for its simplicity and speed. The sinusoidal light intensity can be described as:
$$ {I}_1\left(x,y\right)=A\left(x,y\right)+B\left(x,y\right)\cos \left[\varphi \left(x,y\right)\right] $$
(1)
$$ {I}_2\left(x,y\right)=A\left(x,y\right)+B\left(x,y\right)\cos \left[\varphi \left(x,y\right)+\pi /3\right] $$
(2)
$$ {I}_3\left(x,y\right)=A\left(x,y\right)+B\left(x,y\right)\cos \left[\varphi \left(x,y\right)+2\pi /3\right] $$
(3)
WhereA(x, y) is the average intensity. B(x, y) is the intensity modulation. φ(x, y) is the phase needed to be calculated. Based on the Eqs.(1–3), the φ(x, y) can be gotten.
$$ \varphi \left(x,y\right)={\tan}^{-1}\left[\sqrt{3}\left({I}_1-{I}_3\right)/\left(2{I}_2-{I}_1-{I}_3\right)\right] $$
(4)
Notably, the environmental noise can be eliminated by subtraction of different patterns. The solved phase from Eq. (4) is wrapped in range (−π, π]. After φ(x, y) is unwrapped, the absolute phase can be obtained.
Objection function
Dithering optimization technique has been researched in reducing its overall phase error. The objective function of all these optimization techniques is to get the best fit of the binary patterns to the corresponding ideal sinusoidal pattern [14]. The optimization process can be described as the norm function such as norm Frobenius function:
$$ \underset{B,G}{\min}\left\Vert I\left(x,y\right)-G\left(x,y\right)\ast B\left(x,y\right)\right\Vert $$
(5)
Where ‖·‖ represents the norm Frobenius. I(x, y) is the ideal sinusoidal intensity pattern. G(x, y) represents a 2D Gaussian function. B(x, y) is the 2D binary pattern. * represents convolution. The optimization function evaluates the global intensity similarity between the dithering pattern and its corresponding ideal pattern, but it just considers the global similarity without focusing on local structure similarity. In addition, the global optimization is a very time-consuming process and it is difficult to solve the problem by using the objective function because of the NP problem [18].
The local detailed structure is one of the important information resources in an image. It contains particular high-frequency components. In order to contain the local similarity in the objective function, we propose a weighted synthetic function (WSF). The objective function combines two parts: a global intensity part and a local structure part. The global intensity measurement is based on the normalized mean squared error (NMSE). The local structure measurement is based on the structural similarity index measure (SSIM). The WSF can be expressed as follows:
$$ WSF={\omega}_n NMSE\left(I,{I}_d\right)+{\omega}_s\left(1- SSIM\left(I,{I}_d\right)\right) $$
(6)
Where NMSE(I, I
d
) measures the global similarity and SSIM(I, I
d
) measures the local similarity, respectively. WSF gets the synthetic error between the ideal pattern I and the dithering pattern I
d
. The ω
n
and ω
s
are the weight factors (ω
n
+ ω
s
= 1). Because ω
n
is the coefficient of the normalized mean squared error, it represents the weight of global similarity, such as period, peak, trough and symmetric axis and so on. While ω
s
is the coefficient of structural similarity index measure which represents the weight of local similarity such as local luminance, contrast and structure and so on. In this paper, ω
n
and ω
s
are empirically set to be 0.7 and 0.3 respectively.
First a global intensity similarity is introduced into the WSF expression. The NMSE(I, I
d
) can evaluate the global intensity similarity between the dithering pattern and the corresponding ideal pattern. The NMSE(I, I
d
) can be shown as:
$$ NMSE\left(I,{I}_d\right)=\frac{\sum_H{\sum}_W{\left(I\left(x,y\right)-{I}_d\left(x,y\right)\right)}^2}{\sum_H{\sum}_W{\left(I\left(x,y\right)\right)}^2} $$
(7)
Where H and W respectively represent height and width of the pattern. The NMSE(I, I
d
) method reflects the whole fringe pattern intensity statistical errors. Thus, the smaller the NMSE(I, I
d
) is, the better the whole pattern quality becomes. However NMSE(I, I
d
) only focuses on the global similarity, so SSIM(I, I
d
) is introduced as an optimization term to evaluate the local structure similarity between the dithering pattern and the corresponding ideal pattern. The whole fringe pattern is divided into P elements by the neighborhood window. SSIM(I, I
d
) is calculated in the neighborhood window and the window size is designed based on the whole pattern. For example, in this paper, the size of captured image is 768*1024. Because the vertical fringe pattern is used, the window size is designed according to the width of fringe pattern. 24 pixels are neglected from the 1024 pixels and 1000 pixels are kept. For the whole fringe pattern, it can be divided into 20 parts where every part contains 50 pixels. Every part is analyzed by using luminance, contrast and structure. At last every optimized section is connected directly without complex computing because we optimize the fringe pattern in a whole pattern. In the local structure, the comparison between the ideal pattern and dithering pattern focuses on three parts: luminance, contrast and structure. First the local structure function (LSF) should be generated and the structural intensity error is calculated by comparing the LSF between I(x, y) andI
d
(x, y).
The calculation process of LSF(x, y) is shown as following and we set I(x, y) as an example:
Luminance: Luminance function can be designed as l(x, y), as shown in Eq.(8):
$$ l\left(x,y\right)=\frac{2{S}_x{S}_y+{c}_1}{S_x^2+{S}_y^2+{c}_1} $$
(8)
Where c1 is a constant (here c1 = 1) to avoid singularity. Based on the theory of references [22, 23], every S
x
and S
y
can respectively be calculated by using formula: \( {S}_x=\sum \limits_{i=1}^W{x}_i \) and \( {S}_y=\sum \limits_{i=1}^H{y}_i \) where x
i
is the intensity of every row and y
i
is the intensity of every column. W and H is the width and height of every part.
Contrast: Contrast function can be designed as c(x, y). It is similar to Eq.(8), but it uses σ
x
and σ
y
to express the estimation of the contrast, as show in Eq.(9):
$$ c\left(x,y\right)=\frac{2{\sigma}_x{\sigma}_y+{c}_2}{\sigma_x^2+{\sigma}_y^2+{c}_2} $$
(9)
Where c2 is also constant (here c2 = 1) to avoid singularity.
If the intensity average value of every row is S
i
, the standard deviation σ
x
can be calculated by using formula: \( {\sigma}_x=\sqrt{\frac{1}{N-1}\sum \limits_{i=1}^N\left({x}_i-{S}_i\right)} \) and σ
y
can be gotten with the same form.
Structure: Structural function can be designed as s(x, y). It uses the correlation between the ideal pattern and dithering pattern to measure the structural similarity, as shown in Eq.(10):
$$ s\left(x,y\right)=\frac{\sigma_{xy}+{c}_3}{\sigma_x+{\sigma}_y+{c}_3} $$
(10)
Where the c3 is similar to c1 and c2 (here c3 = 1) and the standard deviation σ
xy
is solved for \( {\sigma}_{xy}=\frac{1}{N-1}\sum \limits_{i=1}^N\left({x}_i-{S}_x\right)\left({y}_i-{S}_y\right) \).
The local weighted function (LWF) is generated by weighted combing the luminance function、contrast function and structural function, as shown in Eq. (11)
$$ LSF\left(x,y\right)=l{\left(x,y\right)}^{\alpha}\cdotp c{\left(x,y\right)}^{\beta}\cdotp s{\left(x,y\right)}^{\gamma } $$
(11)
Where α, β and γ are used to be weighted between luminance function、contrast function and structural function. Usually α, β and γ are set to 1.
The ideal sinusoidal fringe pattern I(x, y) and dithering pattern I
d
(x, y) have the same process to calculate the LSF and LSF
d
, and finally the function SSIM(I, I
d
) can be calculated as shown in Eq.(12):
$$ SSIM\left(I,{I}_d\right)=\sqrt{\sum \limits_{x=1}^N\sum \limits_{y=1}^M{\left( LSF\left(x,y\right)-{LSF}_d\left(x,y\right)\right)}^2} $$
(12)
Where I(x, y) is the pixel intensity of desired ideal sinusoidal fringe. I
d
(x, y) is the pixel intensity of the Gaussian filtered dithering fringe.
Phase-based optimization (p-opt) and intensity-based optimization (i-opt) are also two main categories to improve the quality of fringe patterns. In order to show the superiority of the proposed optimization method, we compare it with the norm Frobenius (NF) [17], the p-opt [18], i-opt [19] and ideal sinusoidal fringe patterns under the same condition. Figure 1 shows the effectiveness of the proposed method. Figure 1a shows the cross-section comparison between the ideal sinusoidal curve and fringe patterns with different optimization. Figure 1b and c give the intensity difference and phase difference with p-opt, i-opt, NF and the proposed method. From the results we can find that the fringe pattern based on phase optimization has less intensity difference than that based on intensity optimization. It means phase-based optimization can generate better quality fringe pattern than intensity-based optimization. Compared with these two methods, the proposed method has the least intensity difference which means the quality is the best among these optimization methods, while the quality based on the norm Frobenius is the worst because intensity difference is the largest. By analyzing the phase errors, it can also be found that the phase error based on NF is the largest. Phase-based optimization is superior to the intensity-based optimization while the proposed optimization can generate the least phase errors. It proves the advantage of dithering fringe pattern after optimization of WSF. It can generate better quality fringe pattern with less intensity difference and less phase errors.
Because NF just compares the global similarity without considering the local structure similarity and other methods are designed for the local details, in the following section we just compare the intensity-based optimization, phase-based optimization and the proposed optimization method.
Implementation in the whole fringe optimization framework
Figure 2 is a flowchart showing the proposed algorithm implemented in the whole fringe optimization and the optimization process is implemented as:
Step 1: Intensity error pixel detection. Intensity absolute error map ΔI(x, y) is calculated by ΔI(x, y) = |I(x, y) − I
d
(x, y)|, where I(x, y) is the intensity of ideal sinusoidal fringe and I
d
(x, y) is the intensity of Gaussian filtered binary fringe. When the pixels with intensity error are bigger than a certain threshold, they can be defined as error pixels. In this paper, ten fringe patterns are projected to a flat and a series of images are captured. The same period of sinusoidal fringe pattern is designed and compared with captured image. The mean difference value is used as the threshold.
Step 2: Error pixel reassignment. Error pixels are mutated from 0 to1 or from 1 to 0. After mutation of each error pixel, the binary fringe is filtered and global intensity RMS error is calculated. Only good mutations are kept, and bad mutations are discarded. Here good mutation means after this mutation, the global intensity RMS error is reduced.
Step 3: Iteration. As Gaussian filter is applied to the fringe patterns, the error pixel mutation of each pixel will influence its surrounding pixels. So after error pixel reassignment optimization of the whole fringe, steps 1 and 2 are carried out iteratively. Here the iteration process will continue until the intensity RMS error improvement is less than 0.1% in one round or the iteration reaches maximum iterating times.
Step 4: Local error pixel detection. LSF is applied on the ideal sinusoidal fringe patterns to obtain the LSF of ideal sinusoidal fringe patterns. LSF is also applied on the filtered sinusoidal fringe patterns to obtain the LSF
d
of dithering fringe patterns. The absolute error between LSF and LSF
d
is calculated by using SSIM(I, I
d
). When absolute error of the pixels are larger than the threshold, they are defined as error pixels.
Step 5: Local error pixel reassignment based on the WSF. Pixels in three shifted fringe patterns are optimized simultaneously. If local error pixel is marked as an error pixel, the corresponding pixels in the three dithering fringes will mutate their binary status from 0 to 1 or from 1 to 0. There are eight kinds of combination:(0,0,0), (0,01), (0,1,0), (0,1,1), (1,0,0), (1,0,1), (1,1,0), (1,1,1). The WSF error is calculated for every combination and the smallest error combination is kept. This algorithm is applied to each local error pixel.
Step 6: Iteration. After local structure optimization for the whole fringe patterns, steps 4 and 5 are carried iteratively, until the intensity RMS error improvement is less than 0.1% in one round or the iteration reaches maximum iterating times.