Focal plane model for flat refractive geometry

Abstract

Background

Flow visualization techniques such as uPIV and droplet imaging determine the measurement volume by the focal plane. Thus, an understanding of how the focal plane moves in reference to the camera is necessary when planar interfaces are present between the camera and the focal plane.

Methods

Using geometric optics, a focus model for a camera imaging through multiple parallel interfaces with different refractive indices is derived. This model is based on the thin lens camera model and gives the location of the focal plane, the depth of field, and the change in the location of the focal plane for a change of camera position. The theoretical model is validated by both simulation and experimental results.

Results

Significant results are that while the magnification of a camera for an in-focus object does not vary for changes in the camera position, the position of the focal plane does. The change of the focal plane location depends only on the refractive indices of the media surrounding the camera and the focal plane regardless of the number or type of other media in between.

Conclusion

The derived model provides a simple, accurate relationship between the focal plane location and the number and location of planar interfaces, thus avoiding potentially incorrect results for measurement plane depth.

Background

Particle image velocimetry (PIV) is one of the most prominent measurement techniques for full-field flow quantification. Of particular recent interest, with the rise of micro-electromechanical systems (MEMS) technology and microfluidics, is the field of μPIV—measuring velocity fields of small channel flows at high magnification (see [1] for a review). Various challenges arise when applying this technique to small scales, including diffraction effects, errors due to Brownian motion and spurious reflections, and the necessity of setting the measurement volume using the depth of field instead of light sheet thickness. The latter of these challenges shifts the responsibility of accurately locating the measurement volume to the imaging rather than light sheet optics. Thus, the measurement volume location and thickness are determined by the parameters of the camera and lens combination such as numerical aperture (NA), focal length, and magnification. The thickness of the measurement volume for volume-illuminated μPIV has been thoroughly studied. Meinhart et al. [2] derived the depth of measurement volume accounting for diffraction, geometric optics, and particle size. Olsen and Adrian [3] derived the depth of correlation for the measurement volume—the thickness outside of which a particle does not make a significant contribution to the cross-correlation of the interrogation regions. On the other hand, very little has been said about the location of the focal plane. Both of these considerations become more complicated when multiple interfaces are situated between the camera and the focal plane (e.g. the camera views through a window into a medium other than air), which is a common situation in μPIV applications.

For the macro-scale PIV case, various models have been proposed to correct the distortions from such changes of optical media. Wieneke proposed a self-calibration scheme where the location of the laser light sheet was inferred without a calibration target using the disparity map from de-warped images from two stereoscopic cameras [4]. He showed that the Tsai pin hole camera model [5] could accurately model the distortion induced by the planar interface, although the parameters of the model no longer took on physical values. For his proposed self-calibration scheme when the camera system was initially calibrated in air and then self-calibrated in water, he proposed the three interface model of Maas [6] and was able to accurately self-calibrate as long as the distance and misalignment between the window and the light sheet could otherwise be accurately inferred. He also verified that using Tsai’s model along with Maas’ three interface model allowed the Tsai intrinsic parameters to assume physically realistic values.

For applications other than PIV, such as underwater photography and photogrammetry, several models and calibration techniques have been proposed to explicitly correct for the distortion due to a change of refractive media. Kwon proposed a modification of the direct linear transform (DLT) model using Snell’s law to explicitly correct for this [7, 8]. Simulations using this model showed a significantly smaller error than the DLT model. However, his calibration method required the prior knowledge of the location and orientation of the planar interface. Treibitz et al. show that a camera viewing a scene through multiple refractive geometry is not a single viewpoint system [9]. They proposed a camera calibration method for estimating the focal length of the camera and the distance from the camera center to the interface based on minimizing the length in the world space of a known straight object. Agrawal et al. further extended this area, showing the system is essentially an axial camera [10]. They proposed a calibration method based on 2D-3D point correspondences where the 3D calibration points have an unknown pose. The calibration scheme allows them to estimate the orientation of the camera with respect to the interfaces, the thicknesses of the interfaces, and the indices of refraction.

However, none of these models address the effect of multiple refractive geometry on the location of the focal plane or the depth of the measurement volume. While effects of refractive geometry are perhaps understood by the community (e.g. Meinhart et al. [2] mention adjusting the movement of a microscope by the ratio of the refractive indices), the authors are not aware of an organized exposition of these effects and their practical implications anywhere in the literature. Thus, there is warrant for revisiting this topic to provide better clarity for further research in the area of μPIV and other experimental flow visualization methods.

Therefore, this paper presents the relationship between the location of the focal plane and the orientation of the camera with respect to m parallel planar index of refraction changes using a first order geometric model. This relationship is demonstrated without complicated optical theory to make the results usable to experimentalists from domains other than optics. The model is an extension of the thin lens, paraxial model and is derived for the case where the optical axis is perpendicular to the planar interfaces.

Methods

Model development

A thin lens is described by two idealizations. First, all rays traced through the lens are assumed to be close enough to the optical axis so that the trigonometric functions of the angle between the ray and the axis can be approximated accurately with a first order Taylor series expansion. Second, the thickness of the lens is small enough that the two principle planes of the lens are assumed to be superimposed. Thus, a ray intersecting the center of the lens at the optical axis will continue in the same direction without deviation. These two assumptions ensure that all rays from object points will intersect at a single image point, and thus the thin lens model can exactly describe a pinhole camera model. However, the additional parameter—the distance of the image plane from the camera center, allows information about focus to be inferred. For many imaging scenarios, the thin lens assumptions are justified the distance of the object to the camera is large compared to the size of the aperture. Additionally, most modern optical systems are designed to minimize the aberrations caused by the limitations in this model.

In this paper, the thin lens model will be extended to include the effect of multiple parallel interfaces between the camera and the focal plane. The analytical expressions regarding the focal plane location will be derived using the paraxial assumption for the rays passing through the planar interfaces as well as through the thin lens.

Figure 1 shows a ray leaving an object, passing through the parallel interfaces, and being focused by the thin lens to potentially form an image on the image plane. Under the paraxial assumption, an arbitrary ray passing through a thin lens may be considered to continue directly to the principle plane and be bent according to the relation [11]

$$ \theta^{\prime} = \theta + y\phi, $$

(1)

Fig. 1

Ray tracing diagram for focal plane model with m changes in index of refraction

where θ and θ ^′ are the small angle between the ray and the normal to the principle plane for the entering and leaving ray respectively (a ray with a positive slope has a positive angle), y is the height above the optical axis where the ray intersects the principle plane, and ϕ is the power of the refracting system, which is the reciprocal of the focal length of the lens or 1/f.

Starting from the object point, a general expression will be derived for the distance between the optical axis and the image point as a function of z as shown in Fig. 1. From Fig. 1, the height where the ray hits the principle plane of the lens is

$$ y_{0} = h_{0} - \sum\limits^{m}_{i=0}{\Delta z_{i} \theta_{i}}, $$

(2)

where the small angle approximation tan(θ _i )≈θ _i has been used. The negative sign comes from the sign convention assumed for the slope of the ray: the angle that the ray makes with the interface is positive if the slope of the ray is positive with respect to the camera coordinate system. For positive ray slopes with respect to the coordinate system, the intersection point of the ray with a surface closer to the origin will be lower.

Snell’s Law can be used to write θ _i in terms of θ _m and the refractive indices. By applying Snell’s Law, in first order, recursively at each interface, the ith angle is given as

$$ \theta_{i} = \frac{n_{m}}{n_{i}}\theta_{m}. $$

(3)

Thus, the angle of the ray where it hits the principle plane of the lens is

$$ \theta_{0} = \frac{n_{m}}{n_{0}}\theta_{m}. $$

(4)

and the height of this point is given by

$$ y_{0} = h_{O} - \sum\limits^{m}_{i=0}{\Delta z_{i} \frac{n_{m}}{n_{i}} \theta_{m}}. $$

(5)

The ray is then bent by the lens according to Eq. (1) so that the new angle is

$$ \theta_{I} = \frac{n_{m}}{n_{0}}\theta_{m}+\left(h_{O} - \sum\limits^{m}_{i=0}{\Delta z_{i} \frac{n_{m}}{n_{i}} \theta_{m}}\right)\frac{1}{f}. $$

(6)

Because of the paraxial assumption, this angle is the same as the slope of the ray forming the image. Thus the image height can be expressed as a linear equation in z where θ _I is the slope and y ₀ is the intercept,

$$ {}\begin{aligned} h_{I}(z,\theta_{m}) &= \left[\frac{n_{m}}{n_{0}}\theta_{m}+\left(h_{O} - \sum\limits^{m}_{i=0}{\Delta z_{i} \frac{n_{m}}{n_{i}} \theta_{m}}\right)\frac{1}{f}\right]z+h_{O} \\ & \quad - \sum\limits^{m}_{i=0}{\Delta z_{i} \frac{n_{m}}{n_{i}} \theta_{m}}. \end{aligned} $$

(7)

For clarity, Γ _m is defined as

$$ \Gamma_{m} \triangleq \sum\limits^{m}_{i=0}{\Delta z_{i} \frac{n_{0}}{n_{i}}}, $$

(8)

which when substituted yields

$$ \begin{aligned} h_{I}(z,\theta_{m})& = \left[\frac{n_{m}}{n_{0}}\theta_{m}+\left(h_{O} - \frac{n_{m}}{n_{0}}\Gamma_{m}\theta_{m}\right)\frac{1}{f}\right]z+h_{O}\\ &\quad - \frac{n_{m}}{n_{0}}\Gamma_{m}\theta_{m}. \end{aligned} $$

(9)

Any ray leaving the object will create a ray in the image space with a height given by this equation. Any two rays leaving the object will intersect to form an image point. By solving for the image point using two arbitrary rays, it can be shown that all rays leaving the object point and captured by the lens will intersect to form a single image point. The rays will be distinguished by the angles leaving the object point θ _m and $\theta _{m}^{\prime }$. The system is

$$ {}\begin{aligned} h_{I} = \left[\frac{n_{m}}{n_{0}}\theta_{m}+\left(h_{O} - \frac{n_{m}}{n_{0}}\Gamma_{m}\theta_{m}\right)\frac{1}{f}\right]z+h_{O} - \frac{n_{m}}{n_{0}}\Gamma_{m}\theta_{m} \\ \end{aligned} $$

(10)

$$ {}\begin{aligned} h_{I} = \left[\frac{n_{m}}{n_{0}}\theta_{m}^{\prime}+\left(h_{O} - \frac{n_{m}}{n_{0}}\Gamma_{m}\theta_{m}^{\prime}\right)\frac{1}{f}\right]z+h_{O} - \frac{n_{m}}{n_{0}}\Gamma_{m}\theta_{m}^{\prime}. \end{aligned} $$

(11)

Solving for z and h _I yields

$$ z =\frac{\Gamma_{m}f}{f-\Gamma_{m}} $$

(12)

and

$$ h_{I} = \frac{h_{O}f}{f-\Gamma_{m}}. $$

(13)

Note that neither of these expressions depend on the value of θ or θ ^′. Thus, all rays leaving an object and captured by the lens intersect at a single image point.

Let d _I be the distance of the image plane from the principle point (e.g. z). From Eq. (12), Γ _m can be expressed as

$$ \Gamma_{m} = \frac{d_{I}f}{d_{I}-f}. $$

(14)

Using the definition of Γ _m , the distance from the final interface to the object plane is

$$ \Delta z_{m} = \frac{n_{m}}{n_{0}}\left[\frac{d_{I}f}{d_{I}-f}-\sum\limits^{m-1}_{i=0}{\Delta z_{i} \frac{n_{0}}{n_{i}}}\right]. $$

(15)

This result is almost the same as a thin lens by itself, and it simplifies to this as the indices of refraction converge. The difference is that the distance from the camera to the object is modified by the ratios of refractive indices of the different segments.

Using this expression, the movement of the focal plane location with changes in camera location can be demonstrated. For image measurement techniques, it is critical to be able to evaluate this relationship in order to record data from the proper slice of a flow. First consider from Eq. (15) that the location of the focal plane with respect to the mth interface is a function of Δ z ₀. Thus an incremental change in Δ z ₀ will lead to a differential change in Δ z _m . Expressly,

$$ \delta\Delta z_{m} = \Delta z_{m}(\Delta z_{0}-\delta\Delta z_{0})-\Delta z_{m}(\Delta z_{0}) $$

(16)

A negative differential change indicated in Eq. (16) corresponds to the camera moving towards the interfaces. Using Eq. (15), δ Δ z _m is equivalent simply to

$$ \delta \Delta z_{m} = \frac{n_{m}}{n_{0}}\delta\Delta z_{0} $$

(17)

Simulations

In the above derivations, the paraxial assumption was made for the rays passing through the refractive planes. This is only a reasonable assumption for object points close to the optical axis at a sufficient distance from the lens. Thus, it is important to validate this assumption by simulating the full physics of the rays for typical optical systems to see how the full solution differs from the approximation. Without the paraxial assumption at the parallel interfaces, the expression for the image height can be shown to be, using the same nomenclature as in Fig. 1,

$$ h_{I}(z) = \left[\frac{\frac{n_{m}}{n_{0}}\sin(\theta_{m})}{\sqrt{1-\left(\frac{n_{m}}{n_{0}}\right)^{2}\sin^{2}(\theta_{m})}} +\frac{y_{0}}{f}\right] z+y_{0}, $$

(18)

where y ₀ is the height of the ray where it intersects the principle plane of the lens and is given by

$$ y_{0}=h_{O}-\sum\limits^{m}_{i=0}{\frac{\Delta z_{i} \sin(\theta_{m}) \frac{n_{m}}{n_{i}}}{\sqrt{1-\left(\frac{n_{m}}{n_{i}}\right)^{2}\sin(\theta_{m})^{2}}}}, $$

(19)

and where θ _m is the angle between the ray leaving the object point and the optical axis. In general, there is no image point that is common to all rays leaving the object under this model.

Because of this, focus information must be inferred by finding the location of the image plane that results in the least blur of the image. This location was computed using the envelope of the curve (caustic) formed by Eq. (18) for all rays captured by the camera. The intersection points of this envelope with the rays passing through the aperture limits on the top and bottom of the lens gave the likely locations of the smallest bundle of rays. For the simulation, these intersection points were solved for numerically and all possible combinations were computed to find the image location for the smallest bundle.

This was performed for two representative cases with equipment currently used in the authors’ lab. The first case was a μPIV situation with a long-distance microscope (TSI, Inc PowerView Plus 4 MP with K2/SC Infinity lens with CF-4 objective). Here there were two planar interfaces and the focal length was 55 mm. The indices of refraction from the camera to the focal plane were 1.0, 1.58, and 1.33, simulating air, glass, and water respectively. The camera center was 64 mm from the first interface, the glass thickness was 1 mm, and system was initially focused at a location 2 mm from the last surface. The camera was moved 150 μm towards the interfaces. Table 1 gives the results of the simulation. The object height is given as a fraction of the field of view (FOV) of the camera with 0.5 representing the maximum distance from the optical axis.

Table 1 Simulation results for μPIV case

The second simulation case was that of a macro-scale droplet imaging situation. The camera and optics were set to match a machine vision camera with a standard 100 mm lens. The focal length was 100 mm, and the aperture spans f/2.8 to f/32. There were three interfaces with four mediums having refractive indices of 1.0, 1.518, 1.474, and 1.02. The distances between the interfaces were 304.8, 15.75, 33.02 mm, and the focal plane was at 33.34 mm beyond the final interface. The camera was moved 10 mm towards the interfaces. Table 2 gives the results of the simulation.

Table 2 Simulation results for droplet imaging case

In both of these simulations, the accuracy of the first order model decreases as the object height and aperture increases. However, even at the largest aperture and object height, the relative error between the full model and the paraxial model is less than 1% indicating that the first-order model can be accurately be used for these imaging situations. However, as indicated by the simulations, the error induced by this approximation increases with FOV and aperture. Imaging setups with FOV or aperture much larger than those used in the above simulations could result in significant errors.

Experimental validation

The model was validated by performing experiments on a long distance μPIV setup. Figure 2 gives a schematic for the experimental facility. The camera was mounted over a square polycarbonate duct with a copper bottom. A 25 μm [0.001in] resolution stage was used to move the camera with respect to the camera interface. Water was circulated in the duct. Light was supplied by means of an LED lamp. The camera used for the experiments was a machine vision camera (TSI PowerView Plus 4 MP) with a long distance microscope (K2/SC Infinity with CF-4).

Fig. 2

Schematic of experimental setup

The procedure of the experiment was as follows:

• 1. The camera was focused on the outside of the polycarbonate duct using the focus adjustment on the K2 lens.

• 2. A focused image of the top of the polycarbonate was captured.

• 3. Using the traverse, the camera was moved towards the duct until the bottom surface of the polycarbonate wall was in focus. The displacement required was recorded, and an image was captured.

• 4. The camera was then lowered until the bottom of the duct was in-focus. The displacement required was recorded and an image was captured.

Figure 3 shows the focused images of the top, bottom of top wall, and bottom of duct locations. The measured displacements of the camera required to achieve these sharp images in comparison to the model are shown in Table 3. The indices of refraction for air, polycarbonate, and water were assumed to be n=1.0, n=1.6, and n=1.33.

Fig. 3

Images taken during experiments. a Top of duct (outside), b Top of duct (inside), c Bottom of duct

Table 3 Experimental results

As shown in the table, the model predicts the experimental values within ± 2%, giving good confidence in the model. Notice that neglecting to use any model at all would result in a 28% error in the focal plane location.

Results and discussion

From this focus model, the location of the focal plane, implications about the magnification of the system, and how these factors change with movement of the camera or for different orientations of the planar interfaces can be inferred.

Location of focal plane

From Eqs. (14) and (17), several important conclusions can be derived. First, movement of the camera will not produce equal movement of the focal plane unless $\frac {n_{m}}{n_{0}} = 1$. Secondly, the amplification of the motion of the camera does not depend on the location of the focal plane. Thus, Eq. (17) holds regardless of the location of the camera with respect to the interface. Also from these equations, the only factor that affects this relationship is the refractive indices of the media surrounding the lens and the focal plane. Thus, the number or type of media changes in between the camera and the focal plane have no effect on the relationship between camera and focal plane movement.

A more complete understanding of this result can be seen by realizing that the right hand side of Eq. (14) only depends on the type and adjustment of the lens. Thus, if the focus adjustment of the lens remains fixed, Γ _m is constant, despite the number of interfaces, the indices of refraction, or the relative location of the camera to the interfaces changing. If the optical system has been focused on a known location to determine the left hand side of Eq. (14), any of the parameters of the left hand side can be solved for after a change in the location or type of the interfaces.

Magnification of image

From the results in Eq. (13) immediately follows that the magnification of the system is

$$ M \triangleq \frac{h_{I}}{h_{O}} = \frac{f}{f-\Gamma_{m}}. $$

(20)

The magnification written in terms of the distance to the image, $d_{I}\triangleq -z$ from Eq. (12) is

$$ M = \frac{f-d_{I}}{f}. $$

(21)

Note that here the magnification is defined to be negative if the image is inverted.

Equation (21) is identical to the magnification of a single thin lens [11]. However, Eq. (20) is different. This leads to the important conclusion that, from the perspective of the camera, the magnification of the image is not changed by the insertion of the refractive interfaces. However, the location where the camera is in focus and where the magnification holds, does change. For example, consider a camera that is focused on an object in air. If a window is placed between the object and the camera and the space surrounding the object filled with water, the location of the center of the focal plane would no longer be at the object. However, if the object was moved to the new location of the focal plane, the magnification would be the same as before the window was put in place. If instead the camera was re-focused on the current location of the object after the insertion of the window, the magnification would change according to how the re-focusing changes d _I in Eq. (21).

Further implications of this result are that as long as the camera is not re-focused, movement of the camera with respect to the interfaces will not change the magnification, though, depending on the depth of field, the camera will not still be focused at the same location.

Conclusions

In this paper, a first order, geometrical optics focus model for a camera looking through multiple planar interfaces was presented. The model demonstrates the following relationships between the object focal plane and the orientation of the camera to the planar interfaces:

• 1. For fixed focus, the magnification of the camera is not changed by the addition or removal of planar interfaces between the camera and the object; however, the location of the focal plane where that magnification holds, does change.

• 2. Motion of the camera towards or away from the refractive interfaces results in a movement of the focal plane of the camera that is magnified by the ratio of the refractive indices of the media surrounding the object and the camera regardless of the number or type of media between the object and the camera.

• 3. For a fixed lens focus, the optical path length between the camera center and the focal plane is fixed regardless of changes in number of interfaces, index of refraction, or relative location of the camera to the interfaces.

The paraxial assumption of the model has been validated for typical imaging cases through simulation, and the main consequences of the model have been validated experimentally.