Fringe projection profilometry (FPP)^[1] is one of the most widely used optical metrological techniques for three-dimensional (3D) surface measurement. The accuracy of 3D surface measurement profoundly depends on the quality of the acquired fringe images, especially in truncation, due to the limited dynamic range of cameras in FPP systems. Fringe truncation might happen from the high- and/or low-reflectance surface of objects, thus leading to considerable phase errors, and this is a crucial issue in a practical environment. To solve the fringe truncation problem^[2] in FPP, researchers have developed correction methods for eliminating the phase errors engendered by objects with shiny surfaces. In many industrial applications, objects with high-reflectance surfaces are diffused by sprinkling a thin layer of powder directly before the 3D surface measurement^[3]. However, the procedure is time-consuming, and its measurement accuracy is affected by the powder thickness. In contrast to the sprinkling powder method, various chemical-free processes have been studied. Unlike conventional FPP methods, many techniques would entail adding, modifying, or replacing some hardware and photometric adjustment to eliminate image saturation for measuring shiny surfaces^{[4–8" target="_self" style="display: inline;">–8]}. However, these approaches increase the complexity of hardware or photometric adjustment for reconstructing 3D model of objects. In contrast to hardware-based methods, many strategies have been developed to remove saturation problems caused by high reflectance with no additional hardware^{[9–15" target="_self" style="display: inline;">–15]}. In the preceding methods, either multiple exposures or many prior rounds of projection were required to guarantee that one or more projections would not cause saturation for highly reflective objects. Nevertheless, these techniques need no additional hardware, and their systems can be performed by using conventional FPP approaches. However, when the objects with shiny surfaces are measured, the projection intensity for the adjustment of the projector and camera should be kept low enough to avoid saturation. But, the low-intensity projection may lead to phase retrieval failure in the low-reflectance surface region. Some requirements, such as efficient measurement, easy implementation, changing system parameters quickly, are key issues for fringe truncation problems. However, changing multiple exposures quickly is usually not a practical approach. Thus, it is a challenge to develop an efficient and accurate method for truncated fringes to conduct high-quality measurement across the 3D surface. Therefore, this study proposes a fringe-shaping method based on the particle swarm optimization (PSO)^[16] algorithm for 3D surface reconstruction with high-/low-reflectance, which only uses the original four phase-shifting saturated fringe images (i.e., fringes of single-trial fringe projection) without many time-consuming reprojected fringes and photometric adjustment.

The proposed approach for 3D reconstruction in FPP-based measurement systems is briefly described as follows. Let In(x,y) represent a 3D object’s fringe images using a conventional N-step equally shifted phase FPP: In(x,y)=A(x,y)+B(x,y)cos[φ(x,y)+2nπ/N],(1)where n(=1,2,…,N) denotes the phase-shift step, and A(x,y), B(x,y), and φ(x,y) denote average intensity, intensity modulation, and phase distribution at each image coordinate (x,y), respectively. The phase term φ(x,y) can be determined by using N-step phase-shifting projected patterns and written as φ(x,y)=−arctan∑n=1NIn(x,y)sin(2nπN)∑n=1NIn(x,y)cos(2nπN).(2)

One can easily derive the phase distribution φ(x,y) that contains surface information of the 3D object from the phase-shifted fringe images. The wrapped phase of the measured object ranges from 0 to 2π. For removing 2π discontinuities to achieve a continuous phase distribution, phase-unwrapping algorithms^[17] are used to widen the range for the wrapped phase distribution. The absolute phase that corresponds to the surface profile can be extracted through phase-shifting algorithms. Then, the unwrapped phase distribution of objects can be further converted into 3D surface profiles after system calibration^[18].

To perform 3D reconstruction of an object by using the modulated fringe images, the phase shifting FPP requires not only an accurate phase shift but also perfect sinusoidal fringe patterns, as described in Eq. (1), during fringe data acquisition. However, in a practical environment, surface reflectance of an object is dependent on light exposure, material properties, and the illuminance limited by the dynamic range of the camera. These might lead the acquired fringes in the region of higher and/or weaker surface reflectance to be truncated. Furthermore, when the object with severe highlights is measured, the fringe intensity should be adjusted to extremely low for saturation avoidance. But, the quality of captured fringe images may be poor, thus leading to significant phase errors in 3D surface measurement. Consider that fringe patterns are projected onto objects, and an m-bit digital camera is used for fringe image acquisition. The measured 3D surface is affected by various reflectance, and the acquired camera fringe image In,tru(x,y) can be written as $$\begin{gathered} In,tru(x,y) \\ ={0,ifInissmallthanlowerlimitIn(x,y),if0<In(x,y)<2m−12m−1,ifInislargethanupperlimit. \end{gathered}$$(3)

Note that the fringe image intensity can be adjusted by camera aperture and exposure time. When the high-reflectance surface of objects leads to the fringe image intensity exceeding the upper limit (2m−1, for m-bit levels of the digital camera), or it is too dark down to the lower limit for the weak reflectance, the fringe truncation might produce significant phase errors in reconstructing the 3D model of objects. Consider that the captured fringe from the ideal sinusoidal fringe pattern in a camera-captured image is essential for accurate 3D surface measurement. Thus, the fringe truncation problem heavily deteriorates the reconstructed fidelity of the 3D object surface; this study proposes a strategy of fringe shaping for reshaping the truncated fringes to match an ideal sinusoidal fringe model, as in Eq. (1), after only single-trial fringe projection and acquisition. Figure 1 is shown to illustrate the flowchart of the proposed single-trial fringe projection. A set of phase-shifting sinusoidal fringe patterns is projected onto the measured object surface, and then the fringe images with 3D surface profile information are captured by the camera. The regional fringe analysis method^[19] is applied first to detect the target object region in the acquired fringe images and then remove the background. If the intensities of fringe images are between the upper and lower limits of the camera, then the fringes are kept and directly used for the 3D phase reconstruction. By contrast, when the surface of the measured object has high/low reflectance, truncation occurs, as described in Eq. (3). It will lead to 3D phase reconstruction errors. The bright/dark regions in the truncated fringe images are detected and labeled first, then filled with the labeled parts, and connected by a flood-fill algorithm. To alleviate the effect of truncation, the fringes are then reshaped by the PSO-based fringe-shaping process, as shown in the dash-line block of Fig. 1. The function of the PSO algorithm is applied to find the optimal fitting for different truncated images in the fringe-shaping process. The operation details of the PSO-based fringe-shaping algorithm will be described in the following section. After the process, the fringes that can be compensated and restored from the truncation are merged into the unsaturated fringe regions to synthesize whole fringe images for obtaining accurate 3D information of objects. In contrast to previous methods, the proposed method accomplishes fringe analysis for fringe truncation problems with only single-trial fringe projection without any time-consuming reprojection.

Fig. 1. Flowchart of the proposed single-trial fringe projection.

下载图片 查看所有图片

Based on the ideal sinusoidal fringe model as in Eq. (1), a set of optimal fitting parameters of the PSO algorithm is required to calculate the coefficients of A(x,y), B(x,y), and Φ(x,y)[=φ(x,y)+2nπ/N]. Due to the substantially high computational efficiency of PSO with a low number of function evaluations in this study, the PSO algorithm is employed in the optimization process to reshape the truncated fringes on the detected bright and/or dark regions. Thus, accurate phase recovery can be improved to achieve good results of 3D surface reconstruction by using the following reconstruction process. In the proposed method, the surface variation of the 3D object is assumed to be smooth without abrupt jumps. Consequently, the average intensity, modulation intensity, and phase function can be estimated using polynomials. Thus, the fringe image is estimated using PSO, denoted as ItruPSO(x,y), and can be written as ItruPSO(x,y)=AtruPSO(x,y)+BtruPSO(x,y)cosΦtruPSO(x,y),(4)where AtruPSO(x,y)=∑m=0p∑k=0mak+12[m(m+1)]xkym−k,(5)BtruPSO(x,y)=∑m=0q∑k=0mbk+12[m(m+1)]xkym−k,(6)ΦtruPSO(x,y)=∑m=0r∑k=0mck+12[m(m+1)]xkym−k.(7)

The degrees of the polynomial approximation in Eqs. (4)–(7) for the average intensity AtruPSO(x,y), intensity modulation BtruPSO(x,y), and phase function ΦtruPSO(x,y) are set to p, q, and r, respectively. Low-degree polynomials are good enough for the smooth surface of 3D objects or slow variation of ambient light in the PSO-based fringe shaping. In contrast, higher degree polynomials for the parameters (p,q,r) will be required and implemented by parallel processing to overcome the high computational complexity for finding the optimal solution of the 3D object with complex surfaces^[20].

In the PSO algorithm, each trial solution is a particle in the problem space. Each particle has a velocity vector vi, and a position pi (pi=[a,b,c], where a=[a0,a1,…,a(p+1)(p+2)2], b=[b0,b1,…,b(q+1)(q+2)2], and c=[c0,c1,…,c(r+1)(r+2)2]). The algorithm is initialized by using a group of random particles, and then, it can search for an optimal solution through the problem space. All of the particles in the swarm have suitable fitting values to be evaluated by optimizing an objective function in the algorithm. Each particle moves through the problem space with the local best known position, but it is also guided toward the best known positions in the search space. The position and velocity of each particle are determined on the basis of the following equations using the constriction factor approach^[16]: $$\begin{gathered} vi(t+1)=χ·{vi(t)+s1·ξ1·[pbesti(t)−pi(t)] \\ +s2·ξ2·[gbest−pi(t)]}, \end{gathered}$$(8)pi(t+1)=pi(t)+vi(t+1),(9)where χ=2|2−ψ−ψ2−4ψ|, and ψ=s1+s2>4^[16]. Here, s1 and s2 are learning coefficients utilized to control the effect of the global best velocity component described in Eq. (8). The parameters of vi and pi represent the velocity and position of the ith particle, respectively; ξ1 and ξ2 are random numbers between 0 and 1. Each particle adjusts its velocity according to its best experience, denoted by gbest, and the best particle in the swarm, denoted by pbest. The parameters of [a,b,c] for average intensity, intensity modulation, and phase functions, respectively, in Eqs. (5)–(7) are the adaptation parameters used for the analysis on the PSO-based fringe-shaping algorithm. This study quantified the quality of fitness by minimizing the difference between the approximation polynomial ItruPSO(x,y) and the acquired fringe In,tru(x,y) in the untruncated region. The best experience is evaluated by the objective function F(x,y) and formulated as F(x,y)=∑(x,y∈Ns)[In,tru(x,y)−ItruPSO(x,y)]2,(10)where Ns represents the coordinates of the pixel location in the untruncated region of In,tru(x,y). For the criterion of minimizing the difference between the reshaping fringe ItruPSO(x,y) and the untruncated region of acquired fringe In,tru(x,y), the PSO algorithm is used to find the optimal solution for ItruPSO(x,y). Thus, the PSO algorithm can be viewed as an optimal fitting based on the sinusoidal modeling of the fringe images. Consequently, the truncated fringe can be reshaped by the proposed algorithm, and then, the compensated fringe image (m bit) is described as $$\begin{gathered} In,com \\ ={A(x,y)+B(x,y)cosΦ(x,y),0<In(x,y)<2m−1ItruPSO(x,y),otherwise. \end{gathered}$$(11)

The truncated regions of the saturated/weak fringes are estimated with the optimal fitting data in the untruncated region through polynomial modeling of the PSO algorithm.

To validate the proposed fringe-shaping technique, the performance of the proposed algorithm was verified by projecting fringes on the glossy acrylic hemisphere shown in Fig. 2. Our experimental setup mainly consists of a digital camera (uEye UI-358 xLE) with a resolution of 2560pixel×1920pixel, a digital projector (Optoma ML750) with a resolution of 1280pixel×800pixel (WXGA), and a Dell workstation (Precision Tower 5810). After the nonlinear distortion is corrected in our camera–projector pair^[21], four π/2 phase-shifting sinusoidal fringes patterns with 20 pixels per period are sequentially projected onto the test objects. If the acquired fringes are truncated, the parameters of p, q, and r in Eqs. (4)–(11) will be selected, and their corresponding 3D reconstructed phase for various parameters of optimal fitting are acquired. Figure 2(a) is the 3D surface reconstruction with unsaturated fringes. The performances derived from different parameters of fringe-shaping optimization are shown in the figures. The experimental results of fringe compensation acquired by saturated fringes using the parameters (p,q,r) of fringe shaping of (1,2,1), (1,2,2), and (2,2,2) based on Eqs. (4)–(11) are shown in Figs. 2(b)–2(d), respectively. The average root mean square errors (RMSEs) of the fringe difference in the untruncated region between the approximating polynomials and the acquired saturated fringe for parameters (p,q,r) in Figs. 2(b)–2(d) are 50.39, 27.07, and 4.54 gray levels, respectively. Accordingly, the mean RMSEs of phase difference for Figs. 2(b)–2(d) compared to the 3D reconstruction shown in Fig. 2(a) can be obtained as follows: 2.80, 2.68×10−1, and 2.53×10−3rad, respectively. As expected, if saturated fringes can be compensated more, the resulting residual fringe structure on the surface can be decreased substantially, as shown in Fig. 2(d), and then, we acquire higher quality 3D phase reconstruction.

Fig. 2. (a) 3D phase reconstruction using unsaturated fringes. (b)–(d) Fringe shaping for (p,q,r): (b) (1,2,1), (c) (1,2,2), and (d) (2,2,2) (color bar unit: rad).

下载图片 查看所有图片

In Fig. 2(d), the parameters (p,q,r) for optimal fitting are (2,2,2), which have no apparent phase errors on the measured hemisphere surface. Consequently, the proposed method can effectively optimize the truncated fringes and successfully retrieve the phase information with enough degrees for approximate functions. In addition, the 3D reconstruction shown in Fig. 2(b) looks like a cylinder, but that in Fig. 2(c) looks like a sphere. The experimental results illustrate that the 3D reconstruction is especially sensitive to the degree of phase function ΦtruPSO(x,y); it must be higher than two to obtain a high-quality 3D surface for the measured glossy hemisphere. Accordingly, as noted in Eq. (1), an accurate approximation for the phase function is crucial for exacting 3D surface information. In addition, population particles and generations of PSO are also discussed for (p,q,r)=(2,2,2) in our experiment. When the population particles are set to 30, 50, and 100, the mean RMSEs of phase differences between the reconstructed 3D phase and Fig. 2(a) for 500 generations are 1.12×10−1, 9.45×10−3, and 9.05×10−3rad. By contrast, the minor differences can be performed to 8.62×10−3, 8.09×10−3, and 2.53×10−3rad, respectively, when the number of generation is set to 1000. Consequently, the truncated fringes can be mostly compensated by the proposed PSO algorithm with optimized parameters for reducing 3D phase errors.

Furthermore, Fig. 3(a) shows the 3D phase reconstruction based on the conventional phase-shifting method for the acquired saturated or weak fringe images. Obviously, there are some phase errors (missing holes) on their surface, because the intensities of the acquired fringe images are greater than the upper limit of the camera. The saturated fringe degraded the 3D measurement accuracy because of the high and low reflectance of objects. However, if these truncated images are compensated by the proposed process, the phase difference for 3D surface reconstruction can improve by 8.41%, 11.15%, 5.71%, and 9.01%, respectively. The 3D surface reconstructions achieve less corrupted phase images, as shown in Fig. 3(b).

Fig. 3. 3D surface reconstruction for high- and low-reflectance surface (a) without fringe shaping and (b) using fringe shaping.

下载图片 查看所有图片

To recover the fringe truncation engendered by high-/low-reflectance surface simultaneously, the proposed fringe-shaping algorithm is examined. The experiment using a complex surface with three different reflectance materials consisted of a plaster figure, a bright coin (dashed ellipse), and a dark knife (dashed rectangle), as shown in Fig. 4(a). Its 3D phase reconstruction is shown in Fig. 4(b). We can obviously find that there are many pixels reaching the maximum and minimum settings of the camera because of the high-/low-reflectance surface in Fig. 4(a). Figure 4(b) shows the reconstructed phase without fringe shaping, which caused considerable measurement errors in the regions of the saturated and weak fringes.

Fig. 4. Experimental results for a complex surface with high-/low-reflectance surfaces simultaneously: (a) saturated fringe and (b) 3D reconstruction without fringe shaping.

下载图片 查看所有图片

The cross-sectional profiles of the detected bright and dark regions near the y axis at y=80 and 600 in Fig. 4(b) are shown in Figs. 5(a) and 5(b), respectively, indicated by the blue curves. Their compensations using the proposed method for the saturated and weak fringes generated and demonstrated using red curves are shown in Figs. 5(a) and 5(b), respectively.

Fig. 5. Fringe shaping for the complex surface with high-/low-reflectance surface materials in Fig. 4(a): (a) high-reflectance surface and (b) low-reflectance surface.

下载图片 查看所有图片

Figure 6(a) is the 3D phase reconstruction based on conventional phase-shifting method; there are some missing parts because of significant phase errors on the high-/low-reflectance surface. If we employ the proposed fringe-shaping technique for the saturated and weak fringe images in Fig. 4(a), we can reshape the truncated fringes and then recover the missing 3D phase information shown in Fig. 6(b). It is performed almost the same as in Fig. 6(c), which is reconstructed from the unsaturated projection fringes. Obviously, these results indicate that once the truncated fringes have been compensated, the fringe truncation can be alleviated to reduce the phase errors by the proposed technique.

Fig. 6. Comparison of 3D reconstruction results between two methods: (a) conventional phase-shifting method, (b) fringe-shaping method, and (c) unsaturated phase-shifting projection.

下载图片 查看所有图片

In summary, a novel single-trial phase-shifting profilometry based on the PSO-based fringe-shaping method has been simulated and experimentally demonstrated. In the proposed method, single-trial fringe projection is employed with only a set of four phase-shifting fringes to recover the truncation for 3D surface measurement. Consequently, the influence of truncation engendered by high and weak surface reflectance can be appropriately compensated. Compared to previous work, the proposed technique can be implemented without time-consuming reprojection, special hardware, or photometric adjustment for the phase recovery of distorted saturated/absorbed fringes. Experimental results validate that significant 3D measurement errors originating from saturated and/or weak fringes can be efficiently and substantially alleviated with a single-trial fringe projection. Thus, the proposed method has potential to conquer one of the challenges for improving 3D surface measurement accuracy in a wide range of reflectivity for both saturated and weak reflectance surfaces.