- Research
- Open Access
- Published:

# Orthogonally coupled geometrical constraints in optics design

*Journal of the European Optical Society-Rapid Publications*
**volume 15**, Article number: 25 (2019)

## Abstract

Modern optical multi-camera systems require integrating many camera modules in a small volume. A new space-saving concept for such imaging systems is presented, based on intersecting optical paths that utilize one or more common elements for the respective optical paths. The principles for the optimization for such systems is examined, providing the theory for geometric optimization constraints. These principles can become useful in designing *e. g.* spatially challenging 360-degree imaging systems for surveillance and consumer applications.

## Introduction

Imaging refractive optical systems are most often designed by using sequential raytracing software, and the underlying principle in design is to optimize the optical surfaces to a merit function that is tailored to the specification of the optical system under consideration [1]. While this approach is proven and quite straightforward, there are design problems that cannot be satisfied with simple barrel-based optics where the optimization is done with respect to one optical axis. For example, multi-camera imaging systems for extreme wideangle photography aim to provide a full 360-degree spatial imaging capability, combining the image space of several optical barrel-sensor systems by software means, such as stitching [2]. These systems are typically more than 15 cm in diameter, where it is assumed that individual entrance pupils of the lenses are ca. 60 mm or more apart, approximating or slightly exceeding the human interpupillary distance (IPD) between the optical axes of the lenses [2]. This matching is important for achieving realistic three-dimensional (3-D) imaging capability, to present the imaged content to the user with virtual reality viewing devices.

In order to save space, a design framework based on intersecting optical axes is proposed. The underlying principle in this design is to utilize, for both optical axes, one or more common optical elements to provide an optical function simultaneously for both optical paths. The easiest way to think of this type of design is to intersect the optical axes orthogonally to each other at the location of the common element.

Previously, common-element based imaging arrays have been designed for gigapixel imaging for instance, in the DARPA (Defense Advanced Research Projects Agency) SCENICC (Soldier Centric Imaging via Computational Cameras) program [3,4,5]. In the AWARE (Advanced Wide field of view Architectures for Image Reconstruction and Exploitation) program within SCENICC, a monocentric multicamera array was designed to provide 2-gigapixel [4] and 100-gigapixel [5] imagery by arranging a multicamera array at the back focal region behind a single or dual element spherical lens system. For this approach to work for all the individual cameras, all surfaces in the spherical lens system were concentric, forming, in effect, a multilayer ball. The cameras each had a relay optic to convey a partially overlapping image field on the array of individual image sensors. Free-space or fiber-optic relay optics were used for each individual camera to provide focusing on the flat image sensor [5].

The immediate effect of applying a common element for two optical paths in the same design is that the optimization needs to dimension the common element in such a way that the propagation of the full field needs to be able to pass through this element, in both orthogonal directions. The overall effect of designing systems with intersecting optical axes is that the optimization parameters for each axis as well as their tolerances are singly or multiply coupled to each other. This article examines the taxonomy and optimization principles of such systems and discusses practical design workflow aspects of optical dual-axis systems with orthogonally coupled constraints. To the Author’s knowledge, this is the first report where the theoretical limit to minimize the dimensions of an optical design, where the optical paths intersect orthogonally within a common element, is described.

## Taxonomy of optical designs for orthogonally coupled optical systems

Limiting the approach of this study to orthogonally coupled dual optics where the optical design of both intersecting axes is identical, a taxonomy can be developed, taking into account the common aspects applied in the design (Fig. 1). One major differentiator in the designs is the sign of the marginal ray angle at the leading surface of the common element. The marginal ray can be converging or diverging, *i. e.* the first surface can have a larger or smaller semidiameter than the second surface of the common element, whether it is an air gap or an optical glass element. Taking first the case of converging marginal rays, the simplest system utilizes a central air gap in the design (Fig. 1a). If the air gap is replaced with an optical material, we arrive at the design of Fig. 1b). Considering these designs, but replacing the marginal ray with a diverging one, in Fig. 1c) the central element is again an air space, and in Fig. 1d), the central element is an optical material. These four proto designs constitute the basis of the taxonomy for finding the geometrical constraints to minimize the distance between the first surface of the common element and the intersecting orthogonal optical axis. Further design considerations arise with regard to the positioning of the stop in the optical paths which is shown in Fig. 2 that presents the taxonomy tree. The taxonomy tree shows all theoretically possible configurations, however, some of these may not be feasible with regard to making practicable imaging optics. In the following, the stop is only regarded as any arbitrary surface within the design.

There are differences as to how the optimization constraints imposed on the design will be made in order to force the optimization process to realize a given topology of the taxonomy tree. The common feature, however, is the dimensioning of the central element, whether it is made of air or an optical material. The topology outlined in the taxonomy will determine the mathematical and the theoretical limit to which these designs can be minimized in size. In addition to constraining the central, common element, also limiting constraints for the surrounding elements in the complete design will need to be imposed.

## Mathematical principles in the constraint definition for the dual orthogonal design

In the following treatment, it is assumed that in the dual orthogonal design, both optical functions, in each of the orthogonal optical paths, are identical. The treatment is also limited to spherical surfaces constituting the common, central element. It is of course possible to design space-saving systems where the optical designs for either axis are different, or where the intersecting axes are non-orthogonally oriented. Similar mathematics can be developed for these cases, using the principles outlined in this article. First, the mathematical expressions or the constraints of the element are developed (Constraints for the common central element section) and then, the constraints of the surrounding elements are described (Constraints for the surrounding elements section).

### Constraints for the common central element

The constraints that apply in the optical design of orthogonally coupled systems relate to the dimensions of the central element whether it is air (Fig. 1a and c) or an optical material (Fig. 1b and d). Furthermore, the constraints limiting the geometry can be defined in terms of the signs of the leading and trailing surfaces of the common central element and the sign of the marginal ray angle *β*, as is conceptually shown in Fig. 3. From Fig. 3, we can see that for each optical path arrangement utilizing a central, common element, the leading surface *S* can either be positive (convex toward the left) or negative (convex toward the right). Similarly, for the trailing surface *T*, the same conditions regarding the signs of the surface apply. For the leading surface *S*, the marginal ray leaving *S* at angle *β* can either be converging (*β* < 0) or diverging (*β* > 0). The limiting calculations must be done for both the leading surfaces *S* and *S′*, as well as the trailing surfaces *T* and *T’*, where the primed symbols refer to the orthogonal axis and its respective surfaces, as either surface, when placed too close, can potentially partially block the passage of light from the orthogonal light train. In total, there are eight conditions for which the mathematical expressions limiting the thickness of the central element need to be developed (see Table 1).

Table 1 presents the taxonomy of Fig. 1 as a table with abbreviations that signify the eight main cases that guide the placement of the common central element, now taking into account also the signs of the curvatures of the leading and trailing surfaces.

The mathematical framework is the same, irrespective of whether the central element is an air space or an optical material. The key principle is in ensuring that the surfaces of the orthogonal train of light passage do not interfere with the propagation of light of the other main direction. Therefore, in Table 1, the surfaces *S* and *S′* as well as *T* and *T’* are designated to be either positive or negative, following the conventions used in optical simulation software [6]. The constraints also only differ with regard to the sign of the marginal ray angle (converging rays or diverging rays).

It is helpful to think of the central point in the design as the intersection of the orthogonal optical axes. This intersection is therefore located within the common element. As it does not represent a real surface in the design, it can be incorporated in optimization as a “dummy surface” *D*, which in Zemax OpticStudio is a surface with no effect on light propagation [6]. Therefore, in the following calculations, the distance *t* after the surface *S* and *S′* (the distance between *S* and *T* as well as between *S′* and *T’*) will be the sum of the distances *e*_{S} and *e*_{T} to and from the intersecting optical axis (dummy surface *D*) (subscripts *S* and *T* refer to the leading surface *S* and the trailing surface *T*, respectively):

From Fig. 3, it can be seen that there are conditions where the surfaces never can overlap the light path of the orthogonal light train. These occur when the leading surface *S* is positive, and the trailing surface *T* is negative (cases C + - and D + - in Table 1). In these cases, if the marginal ray is converging (*β* < 0), an adequate condition is that the near edges of the surfaces *S* and *S′* do not intersect:

where *h*_{S} is the marginal ray height at the surface *S* (or, clear semidiameter of surface *S*). When *β* > 0, respectively, the far edge of surface S′ must not intersect with the marginal ray. From trigonometry, it is easy to obtain:

Where \( {s}_S={r}_S-\sqrt{r_S^2-{h}_S^2} \) is the sag of surface *S* and *r*_{S} is the radius of the surface *S*.

At the trailing surface, conditions expressed by (1) and (2) are applied conversely. For the diverging case (*β* > 0),

and for the converging case (*β* < 0),

where the quantities *h*_{T} and *s*_{T} refer to the marginal ray height and sag, respectively, of surface T and T’.

The calculation for the conditions where the leading surface *S* is negative leads to a more complicated expression, which can be solved as an abstract trigonometric problem (see Fig. 4). Mathematically, this can be expressed in terms of *h*_{S}, the marginal ray height at surface *S* and *r*_{S}, the radius of the surface *S*, as well as the marginal ray angle *β* of the surface *S*. These quantities are obtained in sequential optical simulation from the optimization of the leading surface. Starting with the following trigonometric identities from Fig. 4, where the angle *α* is the angle defined by the height *h*_{S} of the marginal ray leaving surface *S* (point *q*) and the radius *r*_{S} of surface *S* (*α* = arcsin (*h*_{S}/*r*_{S})). The fundamental equations for line *A*, line *B* perpendicular to line *A*, and circle *S′* are:

denoting *m* = tan*β* and *d*_{S}^{2} = *r*_{S}^{2} – *h*_{S}^{2}, *x* and *y* can be eliminated, and for *c*_{S}, we obtain the solution of a quadratic equation (for full derivation, see Appendix):

where the coefficients are

To obtain the distance from surface *S* to the orthogonal optical axis, we identify for our final result from Fig. 4:

In the calculations, there arise conditions that exclude some angles due to the geometry of the problem (Fig. 4) as well as due to that the sides of the equation cannot be divided by zero: *β* ≠ 0^{o}, *β* < 45^{o}, *β* > −45^{o}. At *β* = 0^{o}, we arrive at the trivial solution *e*_{S} = *h*_{S}. Note that the above derivation leading to (13) applies both to the converging and diverging marginal rays.

Conversely, again, a similar calculation can be applied to the positive sign trailing surfaces to obtain *e*_{T} (the “thickness” [7] of the dummy surface *D* to the trailing surface *T*), substituting the relevant quantities *h*_{T} (marginal ray height at the surface *T*), *d*_{T} (distance of the marginal ray intersection point with surface *T* in the horizontal direction), and *r*_{T} (radius of surface *T*) for the marginal ray angle *β*.

For designs dimensioning the central element by (13), first and foremost, the optimization constrains the thickness of the central element to equal its diameter adding the sag of the surfaces (the sign of the sag included). Here, the marginal ray of the field corresponding to the largest field of view (FOV) of the lens design needs to be able to pass through the central element without even tangentially hitting the side surfaces of the element. Therefore, the clear semidiameter of the central element, as well as its thickness, may need to be larger than is necessary for conventional single-barrel designs, to pass this marginal ray.

### Constraints for the surrounding elements

If the maximum diameter of the first surface in the lens is specified, several dimensions for the design can be calculated. Often, the maximum optical track length is also defined. For instance, the common central element can be placed at half the maximum track length, and the upper limit of the semidiameters of all elements can then be calculated.

If the central element is the only common optical part in the dual orthogonal design, the positioning of this central element is not constrained by the surrounding surfaces, provided that the semidiameters of the surfaces *n* of the preceding elements prior to the intersection of the orthogonal axes are not, respectively, larger than their distance *e*_{n} subtracting their sag *s*_{n}, from the central dummy surface *D* of the common element. For each preceding element, therefore:

where *h*_{n} is the semidiameter of surface *n* (note that the sign of the sag as defined in Zemax OpticStudio [6] is important). The center of the common element or the corresponding air space (*i. e.* the intersection of the optical axes) must therefore be placed at least the distance *h*_{n} from any and all preceding elements.

The center of the common element is also a constraint for the optimization of the optical elements placed after the common element in the optical design, limiting the semidiameter of each subsequent element to be

where the same sign convention as in Eq. (14) is used for the sag *s*_{n}.

It should be noted that the constraining Eqs. (13, 14 and 15) apply for both axes in the dual orthogonal design, simultaneously. Therefore, only one optical path needs to be solved, as the optical designs will be identical in both directions, independently of each other, only constrained by the geometry of the orthogonal design. All parameters required for the dual-orthogonal design originate from either one of these orthogonal axes, as all single elements and optical groups in the design are identical, along both optical paths.

## Discussion

The optical designs developed in this article that allow optical paths to intersect present additional degrees of difficulty arising from the coupled constraints affecting both of the respective orthogonal optical paths simultaneously. The problem of dimensioning the central element can be solved using the principles presented in Eq. (13), and rules for dimensioning the other elements can be found in Eqs. (14) and (15). A flowchart for the design process can then be developed, depending on the set of initial requirements and specifications, as shown in Fig. 5.

As the flowchart shows, the design first starts with the fundamental specifications for the optical design. Some of the geometrical considerations can then be directly derived from the specified values for the lens diameter and optical track length. Notably, the numerical aperture and viewing angle as well as focal length of the objective will not play a role in specifying the properties of the central element. Assuming that the most suitable application of optics is in wideangle multicamera systems, hyperhemispherical optical designs commonly apply telecentric back focal systems [7]. Therefore, the common element can very well be placed behind the aperture stop, making it simple to design the central element using the constraint calculations of constraints for the common central element and constraints for the surrounding elements sections. Indeed, the main design consideration in the lens specification will be the choice for the starting design. Systems with many surfaces in the middle and toward the back focal range of the design will probably not be suitable to apply the orthogonal crossed-path design outlined in this article.

The second main consideration in the design specification concerns the image sensor to be applied in the design. There are many design drivers that depend on the application of the system to be designed. Some applications aim at a high image quality with low distortions, and others are optimized with regard to high efficiency in light throughput and even near infrared performance. Sensors that suit these applications must be selected, and the optical design subjected to the requirements imposed by the sensor.

Once the specification is defined, the semidiameter upper limit for the preceding surfaces (before the common element) are written out as geometric expressions in the merit function, for each surface using (14). Chip zone width [6] must be taken into account in physically realistic design for each complete element so as not to make the lens large enough to intersect with the corresponding element in the orthogonal path. After this, the common element is dimensioned according to the principles presented in Chapter 3 (Eq. 13), giving us the thickness of surface *S* (to the dummy surface at the intersection of the orthogonal optical paths). The semidiameter of the dummy, central surface is calculated by the Zemax OpticStudio optimization routine, as the marginal ray angle *β* stays the same when passing through the dummy surface. This calculation is written as a Zemax Programming Language (ZPL) routine in the Lens Data Editor (for optimization in Zemax OpticStudio [6]). Finally, the maximum semidiameters of the trailing elements (those after the common element) are written out in the merit function, similarly to how the preceding elements were limited in size by Eq. (15).

The optimization itself must make use of the common element and its lower thickness limit. As this will be a new constraint in the design, most likely, additional surfaces must be used to balance the optical performance with a similar design without a constraining common element.

If more than one common elements are included in the design, they involve additional surfaces jointly utilized by both of the orthogonal optical paths in the system to be designed. Let us name one of these surfaces *C*. Then, an additional constraint is imposed on this common surface *C*:

where *r*_{C} refers to the radius of the common surface *C*, and *t*_{CD} refers to the track length from the surface *C* to the dummy surface *D*, *i. e*. the radius of the common surface *C* must equal its distance from the intersection of the two orthogonal paths. This approach has been already used in the design of the monocentric multiscale imagers [4, 5], where the common spherical element surfaces are all concentric. This new constraint ties the optical system tightly together with little room for tolerancing. A prospective application for this approach might be a hyperhemispherical camera system, incorporating an outermost, joint spherical lens surface as the first surface of the design. Such a system with three dual-camera modules is conceptually presented in Fig. 6.

In the system of Fig. 6, orthogonally arranged imaging paths cross at a common element within each of the three dual-camera modules. In addition, a joint outer surface concentric with the central, common element has been designed as the outermost surface of the dual-camera module. Placing these modules 120 degrees apart provides a 360-degree camera view of the environment, with a possibility of viewing the scene in three dimensions, as the camera views overlap to provide the right-eye and left-eye views from all directions.

## Conclusions

The principles outlined in this paper present a way to design space-saving optical systems that may make it possible to reduce the size of, for instance, multicamera systems for security, surveillance, and virtual reality 3-D imaging, making these systems better accessible to the consumer. Previous efforts to design military multiscale imagers [3,4,5] relied on dense packing of the imager array [4], resulting in significant space savings compared to hemispherical imagers composed of dense-packed individual cameras with no common elements [4]. No theoretical limits for placing the optical elements in relation to the common element were given, as the imager array size was limited by the packing constraints of placing the cameras as close to each other as spatially possible [4, 5]. For the first time, in this research, the theoretical limit to minimize the space between the first and second surface in a common, central optical element is reported.

In optical simulation, the mathematical expressions for the constraints can be incorporated in the merit function and as hard limits in the lens data editor [6] (in Zemax OpticStudio as Zemax Programming Language (ZPL) routines).

Further work will involve realistic optical designs using the principles presented in this work, as well as developing the theory for tolerancing these orthogonally coupled designs. The ZPL routines will also be developed, in order to properly optimize the designs under orthogonally coupled geometrical constraints.

## Availability of data and materials

The mathematical derivation is available by e-mail.

## References

Fischer, R.E., et al.: Optical System Design, 2nd edn. SPIE Press, McGraw-Hill Education (2008)

Kimmel, J.S., Baldwin, A.R., Rantanen, V.P.: Optics for virtual reality applications. In: Proc. European Optical Society Annual Meeting 2016 (EOSAM 2016), pp. 82–83 (2016)

SCENICC, https://www.fbo.gov/ (2019). Accessed 23 Aug 2019

Tremblay, E.J., Marks, D.L., Brady, D.J., Ford, J.E.: Design and scaling of monocentric multiscale imagers. Appl. Opt.

**51**(20), 4691–4702 (2012)Stamenov, I., Agurok, I.P., Ford, J.E.: Optimization of two-glass monocentric lenses for compact panoramic imagers: general aberration analysis and specific designs. Appl. Opt.

**51**(31), 7648–76661 (2012)Zemax OpticStudio 18.9, https://www.zemax.com/products/opticstudio (2019). Accessed 19 Jan 2019

Martin, C.B.: Design issues of a hyperfield fisheye lens. In: Proc. SPIE 5524, Novel Optical Systems Design and Optimization VII (2004)

## Acknowledgements

The Author is thankful for M. Sc. Peter Eskolin (Nokia Solutions and Networks) for the three-dimensional model of Fig. 6.

## Funding

No funding beyond my salary from Nokia has been applied for or used.

## Author information

### Authors and Affiliations

### Contributions

The Author is the sole contributor to this work. The author read and approved the final manuscript.

### Authors’ information

Jyrki Kimmel is a Principal Researcher at Nokia Bell Laboratories in Tampere, Finland. He holds a DSc (Tech.) degree from Tampere University of Technology. He is active in the European Optical Society as a Program Committee Member of Topical Meeting 3 for the AGM and a member of the Industrial Advisory Committee. He is also a member of the Society for Information Display (SID) where he has served in committee and board positions, as well as of IEEE. His research interests include new technologies for displays for mobile and wearable applications as well as optics design, imaging and display technologies for the emerging mixed reality field. Dr. Kimmel has several peer-refereed articles, book chapters, conference papers, and patents in these disciplines.

### Corresponding author

## Ethics declarations

### Competing interests

The author declares that he has no competing interests.

## Additional information

### Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Appendix

### Appendix

For the full derivation of the trigonometric solution to the smallest distance the orthogonally oriented optical surfaces *S* and *S’* can be placed, refer to Fig. 4 in the body of this article.

The problem can be abstracted by: “Find the distance *e*_{S} for which line *A*, intersecting circle *S* at point *q* = (*d*_{S}, *h*_{S}) at angle *β* tangentially intersects the circle S’ at point *p*. The center of circle *S* is at (0, 0), and the center of circle *S’* is at point *o* = (*r*_{S} + *e*_{S}, *r*_{S} + *e*_{S}), where *r*_{S} is the radius of both *S* and *S’*, and \( {d}_S=\sqrt{h_S^2-{r}_S^2} \).”

From sequential optical simulation, the known quantities are the radius *r*_{S} of circle (surface) *S*, the marginal ray height at surface *S h*_{S}, and the marginal ray angle *β*, since these will be solved for surface *S* before the optimization proceeds any further.

From Fig. 4, we can identify

and from the fundamental equations of a line

(note that line *B* is perpendicular to line *A*, and *β* ≠ 0) In addition, we have the equation for a circle

Let us denote tan*β* = *m*. At point *q*, for line *A*:

from which, solving for *b*:

Likewise, at point *o*, for line *B*:

and solving for *f*:

The relations (20-23) give us for line *A*:

And for Line *B*:

We also have the equation for circle *S’* (20).

Now, we have three equations and three unknowns *x*, *y*, and *c*_{S}. Solving first for *x* at point *p* (intersection of lines *A* and *B* on circle *S’*):

from which

Substituting *x* to the equation for line A (18) we obtain

Substituting *x* from (28) as well as *y* from (30) which both contain *c*_{S} to (20) we get:

Recognizing *r*^{2}_{S} = *h*^{2}_{S} + *d*^{2}_{S} and that *m*^{2} + 1 > 0 for all *β* > -45^{o}, and solving for *c*_{S} from the numerator of the resulting expression, we finally obtain a quadratic equation:

where the coefficients are

To obtain the distance from surface *S* to the orthogonal optical axis, we identify from Fig. 4:

which is our solution to the problem. Note that the positive root of (32) is the only meaningful solution to our problem. In the trivial case where *β* = 0^{o}, it is obvious that *e*_{S} = *h*_{S}, and in this problem, *β* > -45^{o}, since the distance *e*_{T} from the dummy surface *D* to the trailing surface *T* cannot be negative; also, *β* < 45^{o}, since the light path from the orthogonal path cannot pass through surface *S*.

## Rights and permissions

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

## About this article

### Cite this article

Kimmel, J. Orthogonally coupled geometrical constraints in optics design.
*J. Eur. Opt. Soc.-Rapid Publ.* **15**, 25 (2019). https://doi.org/10.1186/s41476-019-0121-4

Received:

Accepted:

Published:

DOI: https://doi.org/10.1186/s41476-019-0121-4

### Keywords

- Optics design
- Design theory
- Orthogonal constraints
- Coupled constraints
- 360-degree imaging