Creating a floating three-dimensional (3D) image in space has been the dream of human beings for a long time, providing natural scenes in an intuitive and natural manner. Due to the realistic and natural display quality of the floating 3D display, such technology has increasingly drawn interests of investigators and scientists in the past several decades. Recently, various floating 3D display methods have been demonstrated. A hologram can reconstruct the 3D object with both amplitude and phase information^[1,2], and it can also be utilized to realize the floating 3D effect^[3]. However, holographic technology is difficult to realize the full-color and dynamic 3D effect currently. Integral imaging (II) is an attractive method to build the 3D light field, which can easily realize full natural color stereoscopic image^[4]. There are many research efforts implemented for this technique, such as the extension of the depth of field (DOF)^[5,6], the enlarging of the viewing angle^{[79" target="_self" style="display: inline;">–9]}, the highly efficient computer-generated encryption image method^[10], and the improvement of resolution^[11,12]. On the base of the above-mentioned studies, it can be concluded that II has the potential to realize the floating 3D effect^[13]. According to the primary aberration theory, lens distortion seriously deteriorates the 3D image quality. Through experiments, it can be found that when the reconstructed 3D scene is displayed far away from the lens array, the distortion of the lens increases sharply. So, the distortion correction for the elemental images of II is very important to realize the floating 3D effect. Here, we demonstrate the distortion correction method for the elemental images of II by introducing the directional diffuser. The directional diffuser has been widely applied in a large body of projection light field displays to obtain smooth parallax^[14,15]. In the contrast experiments, when the distance between the reconstructed image and lens array is equal to 120 mm, the largest lens distortion is decreased from 46.6% to 3.3%.

In general, the II system is composed of a lens array and a liquid crystal display (LCD), as shown in Fig. 1(a). The LCD is used to load the elemental images. Each lens in the lens array is utilized to enlarge the corresponding elemental image individually. The focused image plane is determined by the Gaussian imaging law, and its location can be obtained by substituting the focal length of the lens and the gap between the LCD and lens array into the formula as follows: 1lg+1lc=1f,(1)where f is the focal length of the lens array, lg is the gap between the LCD and lens array, and lc is the distance from the focused image plane to the main plane of the lens. According to the natural trend of the visual system, when the observer looks at the II system, the eye will adjust its accommodation to the image plane of the lens so that sharp image can be focused on the retina^[16].

Fig. 1. Schematics of (a) the traditional II and (b) the separated optical system.

下载图片 查看所有图片

When the observer looks towards the II system, different portions of the reconstructed 3D image are observed through each lens. The observed pixels are a part of the elemental image. As the aberration, the observed pixels are severely distorted. To get an intuitive understanding of the degradation of image quality caused by lens distortion, we separate one lens from the traditional II system, as shown in Fig. 1(b), and analyze the process of an elemental image captured by the retina. The separated optical system is composed of pixels, a lens, and human eyes. According to the optics definition, the eye’s pupil is the aperture stop. As shown in Fig. 1(b), A is the point on the optical axis, and B is the point off the optical axis. Ray 1 is the light ray emitting from point A, and it passes through the edge of the aperture. Ray 2 is the light ray that emits from point B and passes through the center of the aperture. h is the height of the Ray 1 on the lens. hz is the height of the Ray 2 on the lens. u and u′ are the aperture angles. On the basis of the primary aberration theory, the PW-Seidel distortion (S ) can be expressed by the following equations: S=∑hz3h2P−3J∑hz2h2W+J2∑hzhφ(3+μ),(2)P=∑(u−u′1n−1n′)2(un−u′n′),(3)W=∑(u−u′1n−1n′)(un−u′n′).(4)

In the above equations, J is the Lagrange–Helmholtz invariant, and φ is the diopter of the lens. The value of μ is usually set as 0.7. n and n′ are the refractive indexes of the lens and the air, respectively. To indicate the degradation of image quality caused by lens distortion, a traditional II system with a lens array and an LCD is employed. The focal length and the pitch of the lens are 16 and 10 mm, respectively. The LCD with the resolution of 3840×2160 and the size of 23 in. is used. In the experiments, through adjusting the gap between the LCD and lens array and changing the elemental images, two letters “A” and “B” are displayed from 30 to 120 mm. There are four experiments implemented by utilizing the described system. The display results and the details are shown as follows.

Figure 2 illustrates the visual effect with different distances from the lens array to the reconstructed 3D image. Through observing the variation of a partial image displayed by one lens, when the distance is above 90 mm, the distortion is obvious, and the 3D image quality seriously deteriorates. Through simulation, the distortion grid maps of the separated optical system with different parameters are shown in Fig. 3.

Fig. 2. Reconstructed 3D image of the integral imaging. The distances between the reconstructed images and lens array are (a) 30, (b) 60, (c) 90, and (d) 120 mm.

下载图片 查看所有图片

Fig. 3. Distortion grid maps of the separated optical system. The distances between the reconstructed images and lens array are (a) 30, (b) 60, (c) 90, and (d) 120 mm.

下载图片 查看所有图片

Through the above analysis and experiments, it can be concluded that the farther the 3D image is displayed from the lens array, the more serious the distortion is. Obviously, the degradation of image quality caused by the lens distortion makes it very difficult to use the traditional II system to show the floating 3D visual effect.

Here, the directional diffuser is introduced to correct the lens distortion, and it is located at the image plane of the II system. To analyze the process of eliminating lens distortion, the separated optical system of the improved II system is shown in Fig. 4(b). The elemental image and corresponding lens make up a so-called projector. Each pixel of the element image is imaged on the directional diffuser. The diffuser diffuses and limits the incident light in a solid diffusing angle. The human eyes observe the pixels from the directional diffuser.

Fig. 4. Schematics of (a) the improved traditional II and (b) the separated optical system with the directional diffuser.

下载图片 查看所有图片

According to the optics definition, in the separated optical system with the directional diffuser, the lens is the aperture stop. The values of h and hz in Eq. (2) vary significantly by changing the II system. As shown in Fig. 4(b), hz reduces to zero, and h increases as the radius of lens. According to Eq. (2), we can draw the conclusion that the distortion (S ) of the improved II display system decreases sharply by introducing the directional diffuser.

To demonstrate the effect of correction brought by the directional diffuser, the contrast experiments are implemented, as shown in Fig. 5. The 3D images are reconstructed with different distances to the lens array. Compared with the result in Fig. 2, through observing the partial image displayed by one lens, it can be found that the distortion is corrected, and the 3D image quality is obviously improved. By introducing the directional diffuser, the distortion grid maps of the separated optical system are changed, as illustrated in Fig. 6. The diagram of the numerical value of distortion for two display systems is shown in Fig. 7. For example, the largest distortion is decreased from 46.6% to 3.3% at 120 mm, and the largest distortion is decreased from 70.0% to 3.9% at 200 mm.

Fig. 5. Reconstructed 3D image of the II system with the directional diffuser. The distances between the reconstructed images and lens array are (a) 30, (b) 60, (c) 90, and (d) 120 mm.

下载图片 查看所有图片

Fig. 6. Distortion grid maps of the separated optical system with the directional diffuser. The distances between the reconstructed images and lens array are (a) 30, (b) 60, (c) 90, and (d) 120 mm.

下载图片 查看所有图片

Fig. 7. Diagram of the numerical value of distortion.

下载图片 查看所有图片

Through the above analysis and contrast experiments, it can be concluded that the proposed II display system effectively eliminates the lens distortion. However, by introducing the diffuser, the aperture of the display system changes from the pupil to the lens. The aperture angle u obviously increases, as shown in Figs. 1 and 4. The spherical aberration δL can be expressed by the following equation: δL=a1u2+a2u4+a3u6+….(5)

a1toa3 are the corresponding coefficients, which can be calculated by lens parameters. The enlargement of the aperture angle increases the spherical aberration, which sacrifices the display sharpness. By comparing the captured images in Figs. 2 and 5, a slight blur appears after introducing the diffuser. According to the geometrical analysis, the longer the distance between the reconstructed images and lens array, the larger the aperture angle u will be. In the spherical aberration, the display sharpness reduces with the increasing of the distance between the reconstructed images and lens array.

To guarantee the reconstructed 3D image is consecutive and uniform, the expanding angle of the directional diffuser is required to exactly eliminate the gap between the lenses. As shown in Fig. 8, in the improved II display system, g is the center distance of two adjacent lenses, and p is the pitch of the lens. The distance lc is much larger than the value of parameters g and p. According to the geometric relationship, the expanding angle θ can be calculated by the following equation: θ=arctanglc−arctanplc.(6)

Fig. 8. Schematic of the proposed floating 3D display system.

下载图片 查看所有图片

Figure 8(b) illustrates the result of diffusion caused by the directional diffuser. The blue area (triangle above) represents the distribution of light rays emitted from the lens array. The red area (triangle below) represents the distribution of light rays passing through the directional diffuser. As shown in Fig. 8, by introducing the directional diffuser with the designed expanding angle, the light rays with 3D information are distributed uniformly in the visual space.

In our experimental setup, an LCD with the size of 23 in. and the resolution of 3840×2160 is employed. The pitch and the focal length are 10 and 16 mm, respectively. The image plane is set 200 mm from the lens array. The DOF of the proposed display system is the same as the II display in real mode, which is determined by the tolerable limit of the overlaps among voxels. A letter “A” is displayed at five depth planes, successively. The captured images are shown in Fig. 9. When the display depth is less than 150 mm or more than 250 mm, the displayed letters are blurred. The DOF of the employed system is from 150 to 250 mm. To ensure the display quality, the reconstructed 3D image should be located in the DOF.

Fig. 9. Photographs of reconstructed images at different depth layers.

下载图片 查看所有图片

The elemental images of a blue car model are coded and loaded on the display panel. The displayed depth is 20 cm, and the center of the reconstructed 3D image is 20 cm from the lens array. Photographs of the displayed 3D image taken at different positions are shown in Fig. 10. Obviously, by introducing the diffuser, the quality of the reconstructed 3D image is improved.

Fig. 10. Photographs taken at different viewing angles (see Visualization 1).

下载图片 查看所有图片

A distortion correction method for II display is successfully demonstrated. The lens distortion of the traditional II seriously degrades the image quality. The farther the 3D image is displayed from the lens array, the worse the visual effect is. To analyze the process of eliminating lens distortion, one lens is separated from the traditional II. By introducing the directional diffuser, the aperture stop of the separated optical system changes from the eye’s pupil to the lens. The values of hz and h are decreased and increased, respectively. According to the expression of the PW-Seidel distortion, it can be concluded that the lens distortion is corrected effectively. When the distance between the reconstructed image and lens array is equal to 200 mm, the largest lens distortion is decreased from 70.0% to 3.9%. In the experiment, the display quality of the II display system with a directional diffuser is obviously improved.