Figure 1a shows the optical layout of the orthogonal-dispersion prism-grating device (*PGD*) that combines a prism with a transmission grating. The dispersion directions of the prism and transmission grating are orthogonal. The transmission grating is superimposed on the hypotenuse surface of an optical wedge, and the right-angle surface of the optical wedge is directly attached to one surface of the prism. The grating normal is parallel to y-axis, the grating grooves are parallel to z-axis, and the y´-axis is perpendicular to the right-angle surface of the optical wedge. The prism disperses the incident light along the vertical direction (i.e., z-axis), then the transmission grating disperses the separated beams along the horizontal direction (i.e., x-axis). The spectrum is spread out in two dimensions.

Suppose that *γ* denotes the vertex angle of the prism, *γ*_{1} denotes the vertex angle of the optical wedge, *n*_{
p
} (λ_{
i
}) is the refractive index of the prism for wavelength λ_{
i
}, and *n*_{
g
} (λ_{
i
}) is the refractive index of the grating material for wavelength λ_{
i
}.

Figure 1b shows the ray tracing in the sagittal plane (i.e., x´-y´ plane), in which *δ* is the angle between the inward normal vector of the grating and the projection of the incident wave vector \( \overrightarrow{k} \) onto the normal plane of the grating (i.e., x-y plane). It is easily obtained that *δ=γ*_{1}.

Figure 1c depicts the ray tracing in the meridian plane (i.e., y´-z´ plane), in which ε is the angle between the incident wave vector \( \overrightarrow{k} \) and the normal plane of the grating. A ray enters the prism at angle *θ*, is refracted at angle *θ*_{1}, leaves the prism at angle *θ*_{2} , and is refracted into the grating material at angle *θ*_{3}. It is easily obtained that

$$ \sin \theta ={n}_p\left({\lambda}_i\right)\sin {\theta}_1, $$

(1a)

$$ {n}_p\left({\lambda}_i\right)\sin {\theta}_2={n}_g\left({\lambda}_1\right)\sin {\theta}_3, $$

(1b)

$$ \varepsilon ={\theta}_3, $$

(1c)

$$ {\theta}_1+{\theta}_2=\gamma $$

(1d)

From Eq. (1a-d), the angle ε is given by

$$ {\displaystyle \begin{array}{c}{n}_g\left({\lambda}_i\right)\sin \varepsilon ={n}_p\left({\lambda}_i\right)\sin \left(\gamma -{\theta}_1\right)\\ {}={n}_p\left({\lambda}_i\right)\left(\sin \gamma \cos {\theta}_1-\cos \gamma \sin {\theta}_1\right)\\ {}={n}_p\left({\lambda}_i\right)\left(\sin \gamma \sqrt{1-\frac{\sin^2\theta }{n_p^2\left({\lambda}_i\right)}}-\frac{\cos \gamma \sin \theta }{n_p\left({\lambda}_i\right)}\right)\\ {}=\sin \gamma \sqrt{n_p^2\left({\lambda}_i\right)-{\sin}^2\theta }-\cos \gamma \sin \theta .\end{array}} $$

(2)

Let the plane of incidence be the plane that is made up of the incident light ray and the grating normal. Let the normal plane of the grating be the plane that is perpendicular to the grating grooves. In the conical diffraction case, the grating equation of a plane transmission grating is given by [1].

$$ m{\lambda}_i=d\left[{n}_g\left({\lambda}_i\right)\sin \alpha -\sin {\beta}_m\right]\cos \varepsilon $$

(3)

where *m* is the diffraction order, λ_{
i
} is the wavelength of light, *d* is the groove spacing of grating, ε is the angle between the incident light path and the plane perpendicular to the grating grooves, *α* is the angle of incidence measured from the grating normal, *β*_{
m
} is the diffraction angle measured from the grating normal. The angle sign convention is that angles measured counter-clockwise from the normal are positive and angles measured clockwise from the normal are negative.

Figure 2(a) shows the conical diffraction by a plane transmission grating, in which the grating lies in the x-z plane, the grating grooves are parallel to z-axis, and the grating normal is parallel to y-axis. Both the incident ray and the reflected rays lie in the y < 0 half-space; the transmitted diffracted rays lie in the y > 0 half-spaceThat is, the incident light is dispersed on the opposite side of the transmission grating. All diffracted rays fall on the surface of a half-cone whose cone axis is parallel to the grating grooves.

Figure 2(b) shows the geometry diagram of a diffracted light ray. The direction of the diffracted ray with a wave vector \( {\overrightarrow{k}}_m={k}_{mx}{\overrightarrow{e}}_x+{k}_{my}{\overrightarrow{e}}_y+{k}_{mz}{\overrightarrow{e}}_z \) is given by two parameters: (1) *φ*_{
m
} is the angle between the diffracted ray and the normal plane of the grating and (2) *ρ*_{
m
} is the angle between the outward normal vector of the grating and the projection of the diffracted ray onto the normal plane of the grating. We can get

$$ \sin {\beta}_m=\sqrt{k_{mx}^2+{k}_{mz}^2}/{k}_m, $$

(4a)

$$ \sin {\rho}_m={k}_{mx}/\sqrt{k_{mx}^2+{k}_{my}^2}, $$

(4b)

$$ \cos {\varphi}_m=\sqrt{k_{mx}^2+{k}_{my}^2}/{k}_m, $$

(4c)

$$ \sin {\varphi}_m={k}_{mz}/{k}_m $$

(4d)

Based on Eqs. (4a-d), it can be obtained that

$$ \sin {\rho}_m=\sqrt{\frac{\sin^2{\beta}_m-{\sin}^2{\varphi}_m}{1-{\sin}^2{\varphi}_m}} $$

(5)

Figure 2(c) shows the geometry diagram of an incident wave vector. A light ray, with a wave vector \( \overrightarrow{k}={k}_x{\overrightarrow{e}}_x+{k}_y{\overrightarrow{e}}_y+{k}_z{\overrightarrow{e}}_z \) and its modulus *k* = 2π/*λ*, is incident on the grating at an off-plane direction (*ε* ≠ 0). We can obtain

$$ \tan \delta ={k}_x/{k}_y, $$

(6a)

$$ \cos \alpha ={k}_y/k, $$

(6b)

$$ \sin \varepsilon ={k}_z/k, $$

(6c)

$$ {k}^2={k}_x^2+{k}_y^2+{k}_z^2 $$

(6d)

From Eqs. (6a-d), it can be obtained that

$$ {\displaystyle \begin{array}{l}\sin \alpha =\sqrt{\sin^2\delta +{\cos}^2\delta {\sin}^2\varepsilon}\\ {}\kern1.75em =\sqrt{\sin^2{\gamma}_1+{\cos}^2{\gamma}_1{\sin}^2\varepsilon}\end{array}} $$

(7)

According to the law of refraction and the geometry, we can get

$$ \sin {\varphi}_m={n}_g\left({\lambda}_i\right)\sin \varepsilon $$

(8)

Based on Eqs. (2) and (8), the elevation angle *φ*_{
m
}, for the **PGD**, is given by

$$ \sin {\varphi}_m=\sin \gamma \sqrt{n_p^2\left({\lambda}_i\right)-{\sin}^2\theta }-\cos \gamma \sin \theta $$

(9)

Based on Eqs. (3), (5), (7) and (8)**,** the azimuth angle *ρ*_{
m
}, for the **PGD**, is given by

$$ {\displaystyle \begin{array}{c}\sin {\rho}_m=\sqrt{\frac{\sin^2{\beta}_m-{\sin}^2{\varphi}_m}{1-{\sin}^2{\varphi}_m}}\\ {}=\sqrt{\frac{{\left(\frac{m{\lambda}_i}{d\cos \varepsilon }-{n}_g\left({\lambda}_i\right)\sin \alpha \right)}^2-{\sin}^2{\varphi}_m}{1-{\sin}^2{\varphi}_m}}\\ {}=\sqrt{\frac{{\left(\frac{m{\lambda}_i}{d\cos \varepsilon }-{n}_g\left({\lambda}_i\right)\sqrt{\sin^2{\gamma}_1+{\cos}^2{\gamma}_1{\sin}^2\varepsilon}\right)}^2-{\sin}^2{\varphi}_m}{1-{\sin}^2{\varphi}_m}}\\ {}=\sqrt{\frac{{\left(\frac{m{\lambda}_1}{d\sqrt{1-\frac{\sin^2{\varphi}_m}{n_g^2\left({\lambda}_i\right)}}}-{n}_g\left({\lambda}_i\right)\sqrt{\sin^2}{\gamma}_1+{\cos}^2{\gamma}_1\frac{\sin^2{\varphi}_m}{n_g^2\left({\lambda}_i\right)}\right)}^2-{\sin}^2{\varphi}_m}{1-{\sin}^2{\varphi}_m}}\\ {}=\sqrt{\frac{{\left(\frac{n_g\left({\lambda}_i\right)m{\lambda}_i}{d\sqrt{n_g^2\left({\lambda}_i\right)-{\sin}^2{\varphi}_m}}-\sqrt{n_g^2\left({\lambda}_i\right){\sin}^2{\gamma}_1+{\cos}^2{\gamma}_1{\sin}^2{\varphi}_m}\right)}^2-{\sin}^2{\varphi}_m}{1-{\sin}^2{\varphi}_m}}\end{array}} $$

(10)

Eqs. (9) and (10) completely specify the diffracted ray angles in terms of the incident ray angles. For an incident ray of a given wavelength, the diffracted rays will all have the same value of *φ*_{
m
} given in Eq. (9) and will therefore all lie in a common conical surface, as shown in Fig. 2(a). For reference, if ε = 0, the conical diffraction grating equation [Eq. (3)] simplifies to yield the classical in-plane diffraction grating equation, *mλ*_{
i
} = *d*[*n*_{
g
}(*λ*_{
i
}) sin *α* − sin *β*_{
m
}], for a plane transmission grating.