Wide-viewing integral imaging system using polarizers and light barriers array

Abstract

Background

Integral imaging is considered one of the most promising three-dimensional display technologies, while the limited viewing angle is regarded as a primary disadvantage of integral imaging display to reach a commercial level. This paper proposes a viewing angle enhancement method for both liquid crystal display (LCD) and the projection-type three-dimensional integral imaging system.

Methods

The proposed wide-viewing integral imaging system is established by using the lenslet array coupling with the polarizers, light barriers, and enlarged elemental images array. The size of light barrier and polarizer is equal to the pitch of each lenslet, the light barrier prevents light rays from passing through, and the polarizer controls the passage of light ray in the specific polarization direction. In the projection-type integral imaging system, two orthogonal elemental images arrays (EIA) are projected onto the projection screen simultaneously by the corresponding projectors. In LCD integral imaging system, two orthogonal EIAs are displayed by use of an LCD screen which can switch the polarization direction of the EIA by time-multiplexed technology within the time constant of the eyes’ response time.

Results

The viewing angle can be enlarged dramatically by the improvement of the size of each elemental image according to the integral imaging principles. The experimental result shows that the proposed method exhibits approximately four times the viewing angle of conventional integral imaging with the same lens array.

Conclusions

The increment of viewing angle can be determined by the number of light barriers between two adjacently orthogonal polarizers, the more the light barrier, the larger the viewing angle.

Background

Integral imaging, proposed by Lippman in 1908, is a promising three-dimensional (3D) technique for its full-parallax, continuous-viewing 3D images and without any special glasses [1,2,3]. Integral imaging uses lens array to collect and reproduce the light field of the three-dimensional scene, affording full color, full parallax and quasi-continuous viewing points within the viewing angle, working with incoherent light and can be viewed with the naked eye. Despite its many advantages, integral imaging suffers from the inherent drawbacks such as pseudoscopic effect [4], limited viewing angle [5,6,7,8,9,10], low viewing resolution [11,12,13,14,15] and small image depth [16, 17]. Focusing on the improvement method of field of view, the researchers have proposed a variety of solutions.

The viewing angle of integral imaging system is limited by the size of each elemental image and the distance between the lenslet and the EIA. In an integral imaging system, the lenslet array acts as a spatial beam splitter. Each lenslet unit is equivalent to a macro pixel so the lenslet unit is always not large. Most of the commercially available lenslet arrays are made of single-layer optical glass or optical plastic. They have no aberration correction capability and the monolayer lenslet array has a small viewing angle of about 30 degrees. To overcome the limitation of viewing angle, various methods have been proposed [5,6,7,8,9,10, 14, 15]. However, most of these algorithms have some limitations and their performances heavily depend on some specific conditions. The viewer tracking has a complex control system and modifying the configuration of the lens array can also extend the viewing angle, such as the fresnel lens array and negative index planoconcave lens array [18, 19]. Conventional fisheye lenses can map flat image onto wide field of view, but suppressing field curvature and their aberrations needs multi-layer components, and thus imposes harsh design tradeoffs.

In order to improve the viewing angle of the integral imaging system, this paper presents a method of placing the polarizers and light barriers array in front of the lenslet array. By jointly controlling LCD/projection screen, the lenslet array, the polarizer, and light barriers array, different polarized light ray can pass through corresponding lenslets. The proposed method can increase the size of the elemental images (EI), eliminate the crosstalk phenomenon of the light through the adjacent lenslets, and thus improve the viewing angle of the 3D display system.

Method

Viewing angle of the conventional integral imaging

A typical integral imaging system consists of two parts: the pickup process and the display process. In the pickup process, the EIs to record 3D object from different perspectives are captured by a camera with a lenslet array. In the display process, the captured EIs are back projected to reconstruct the 3D images either by optical or computational method. The basic configuration of the pickup process is that the recording medium has the same size as the lenslet array, as shown in Fig. 1.

Fig. 1

Principle of the integral imaging system: (a) the pickup process and (b) the display process

The viewing angle of integral imaging is one of the key indicators of integral imaging system, which has a direct impact on the 3D viewing experience. The conventional integral imaging display system by refractive type lenslet array has a large aberration, the display area is small, and there is a jumping phenomenon at the edge of the viewing area. Since the lenslet array has multiple channels and the size of each lenslet is small, it is difficult to use multi-lens combination for image quality optimization which is often used in traditional optics. In conventional integral imaging display, the pitch of the elemental lens is equal to the size of the corresponding elemental image, which is displayed by a flat-panel monitor directly. According to paraxial optics theorem, the viewing angle is limited by the elemental lens pitch and the gap between the lens array and the display device. Thus, the viewing angle of the integral imaging shown in Fig. 2 can be expressed as:

$$ \theta =2\arctan \frac{p}{2g} $$

(1)

where p is the pitch of the elemental lens, and g is its focal length. According to Eq. (1), an easy way to improve the viewing angle is to enlarge the pitch of the elemental lens or shorten the focal length. However, enlarging the pitch of adjacent lenslets degrades the viewing resolution. Thus, the pitch of the elemental lenslet should be no larger than several millimeters to avoid observing the grid structure effect.

Fig. 2

Viewing angle of the conventional integral imaging

Wide-viewing integral imaging system

In order to improve the viewing angle of the integral imaging system, this paper presents a method of placing the polarizers and light barriers array in front of the lenslet array. By jointly controlling liquid crystal display,the polarizers and the light barriers array, different polarized light ray can pass through the corresponding lenslets. Theoretically, the size of the elemental image corresponding to each lenslet will be several times larger than conventional method. The proposed method, as shown in Fig. 3, can increase the size of the EIs, eliminate the crosstalk phenomenon of the light through the adjacent lenslets, and improve the viewing angle of the 3D display system. Figure 3(a) is the projection-type 3D integral imaging system and Fig. 3(b) is the LCD integral imaging system with the polarizers and light barriers array, respectively.

Fig. 3

Wide-viewing LCD/projection-type integral imaging system using the polarizers and light barriers array: (a) projection-type and (b) LCD integral imaging system

In Fig. 3(a), two orthogonal polarizers are placed in front of two projectors with the same parameters and then two EIAs with orthogonal polarization states are projected onto the projection screen by the corresponding projectors. The polarization direction of the EIA is the same as that of the polarizer in front of the projectors. The projection screen is a rear projection screen which is able to maintain the light polarization characteristics. In addition, by adding a full light barrier between different polarizers before the lenslet array, we can increase the viewing angle further. The size of each light barrier is the same as that of each lenslet. The width of each EI is 4 times that of conventional integral imaging system, and its height remains unchanged. The polarizers and light barriers are placed in close proximity to the lenslet array and the polarization directions of adjacent polarizers are orthogonal. The light ray of the EIA is filtered through polarizers array and light barriers array and then reaches the lenslets array. The 3D reconstructed image can be obtained by the observer.

Figure 3(b) shows the variation of EIA with different polarization state on the polarization switchable LCD screen. At the time t1, the LCD screen display the EIA 1 which can pass through the yellow polarizers. At the time t2, the LCD screen display the EIA 2 which can pass through the green polarizers. The elemental images array for the corresponding mode is displayed in synchronization with the change of the polarization state. The different EIAs can be displayed repeatedly by time-multiplexed technology within the time constant of the eyes’ response time. In practice, a polarization-switching device, such as the polarization shutter screen which is widely applied in the field of stereoscopy, electrically switchable polarization laser based on metasurface [20], and polarization switchable lens [21], can apply in the proposed wide-viewing integral imaging system for polarization-switching. If one light barrier is placed between the two orthogonal polarizers, the size of the elemental image should be magnified by a factor of 4 correspondingly.

The size of each light barrier and polarizer is equal to the pitch of lenslet, the light barrier prevents light ray from passing through, and the polarizer controls the passage of light ray in the specific polarization direction. By adjusting the number of full light barriers, you can increase the display viewing angle by any multiples. The more the light barrier, the lager the viewing angle is. There is the following relationship:

$$ \theta =2\arctan \frac{\left(N+1\right)p}{g} $$

(2)

where N represents the number of light barriers between two adjacent polarizers. Comparing the Eq. (1) and (2), it can be seen that the integral imaging display system based on the polarizers and light barriers array expands the display area of the EIs and increases the viewing angle. In both modes, the viewing angle which can be enlarged more than 2 times is the main difference between the proposed system and the previous methods.

One problem encountered with integral imaging is the pseudoscopic effect of the 3D reconstructed image when the captured elemental images used for display do not receive pre-processing. There is no direct relationship between the viewing angle and the pseudoscopic effect, and this paper cannot eliminate the pseudoscopic effect. Moreover, the imaging depth of integral imaging is different in real/virtual display mode and focused display mode, which is determined by the light wavelength, object distance, and the size of each lenslet [22]. Thus, the method proposed does not affect the imaging depth.

Hoshino et al. [23] define the viewing resolution of integral photography as the minimum value between the maximum viewing spatial frequency of elemental images and the Nyquist frequency determined by the distance between two adjacent lenslets q:

$$ {f}_{\mathrm{max}}=\min \left({f}_{i\max },{f}_{\mathrm{nyq}}\right)=\min \left({\alpha}_{i\max}\frac{z_i}{L-{z}_i},\frac{L}{2q}\right) $$

(3)

where f _imax is the maximum viewing spatial frequency of elemental images through the lenslet array, L is the distance from the lens array to the observer position, α _imax is the maximum projectable frequency of an elemental image through lenslet array, and z _i is the distance between the integral image and the observer; f _nyq is the Nyquist frequency. When the maximum viewing spatial frequency f _imax is higher than f _nyq, the higher frequency component over the Nyquist frequency should be removed from the elemental image, and the viewing resolution of an integral image is determined by f _nyq. The viewing resolution will be decreased due to the increasement of the distance between two adjacent lenslets. Admittedly, more light barriers employed will cause the signal noise ratio to decrease and this is one of the main disadvantages of our proposed method.

Results and discussion

In this section, computational integral imaging reconstruction is used to verify the validity of the proposed method. The computational processing framework of integral imaging system using ray tracing are presented, which can generate EIs and reconstruct 3D images avoiding suffering from the diffraction and device limitations. Figure 4 illustrates the procedure of computational 3D integral imaging method, which can generate the enlarged EIAs according to different system parameters.

Fig. 4

Procedure of computational 3D integral imaging system

The 3D target object is a parrot positioned at depth of 8 cm. First, the planar EIA of a parrot is captured with our self-developed integral imaging pickup software. A lens array, 60 × 60, with 1.2 cm × 1.2 cm rectangular aperture is used in the pickup process. The focal length of the lenslet array is 1 cm and the pixel number of each elemental image is 75 × 75. Figure 5 shows the EIAs captured by the pickup system. In order to increase the viewing angle, we generate the EIAs with the same height and 4 times the width of conventional EIs, that is, the number of EIs is 15 × 60, the focal length of 1 cm, the number of pixels per elemental image is 300 × 75. Figure 5(a) is the EIA of the conventional integral imaging system, while Figs. 5(b) and (c) are the EIAs with different polarization states captured by the proposed integral imaging system. The viewing angle of traditional integral imaging system is 8.58° calculated by Eq. (1) and the viewing angle of the proposed integral imaging system with orthogonal polarizer array and light barriers is 33.40°.

Fig. 5

(a) is the elemental images array of the conventional integral imaging system; (b) and (c) are the EIAs displayed in different polarization states

Figure 6 shows the reconstructed display viewing angle using an integral imaging system with a polarizers and light barriers array. The observer observes the three-dimensional reconstructed image with respect to the center position (0 °) of the central lens optical axis, the left side (angle is -), and the right side (angle +), respectively. It can be seen from the figure that the correct reconstructed image of the entire scene can be seen in the range of ± 33°, while an error reconstructed image appears at ± 34°. The wrong area is marked with a small yellow box. The perspectives can be observed continuous within an angle of 8.58° by conventional computational integral imaging system. In contrast, the displayed 3D images on different viewpoints using the proposed method are shown in Fig. 6. The different perspectives of the images can be seen continuously up to 33°, which is in accordance with the theoretical result according to Eq. (2).

Fig. 6

Different viewpoints of the reconstruction image of the proposed 3D integral imaging system

It can be seen in Fig. 6 that the viewing angle of the integral imaging system based on polarizers and light barriers array has been significantly improved. By comparing the different view images horizontally, different parallax information can be obtained. In the range of ±33°, you can get the correct reconstructed image of the whole scene. The proposed method is about approximately 4 times the viewing angle of the conventional integral imaging. The different wide-viewing integral three-dimensional imaging methods proposed by Refs. [9, 10] can improve the field angle to a certain extent, but cannot more than 2 times the original viewing angle. In both modes of our manuscript, the viewing angle can be enlarged more than 2 times which is the main difference between the proposed system and the previous methods. The increment of viewing angle can be determined by the number of light barriers, the more the light barrier, the lager the viewing angle. When one light baffles array is used, the field angle will be enlarged 4 times as conventional method. The size of each elemental image can be enlarged as any positive integer multiples as the pitch of the lenslet. Thus, the viewing angle of the integral imaging system can be improved dramatically by the improvement of the size of each elemental image according to the integral imaging principles.

Conclusions

This paper proposes a viewing angle enhancement method for both the LCD and projection-type 3D integral imaging system. The proposed integral imaging system is established by using the lenslet array coupling with the polarizer array, light barrier, and the enlarged EIs. In projection-type integral imaging system, two orthogonal EIA are projected onto the projection screen simultaneously by the corresponding projectors. In LCD integral imaging system, two orthogonal EIA are displayed by the use of a LCD screen which can switch the polarization direction of the display images by time-multiplexed technology within the time constant of the eyes’ response time. The increment of viewing angle can be determined by the number of light barriers, the more the light barriers, the lager the viewing angle. The size of each elemental image can be enlarged as any positive integer multiples. Thus, the viewing angle of the integral imaging system can be improved dramatically by the improvement of the size of each elemental image according to the integral imaging principles.