Guide star based deconvolution for imaging behind turbid media

Abstract

Background

If structures of interest are hidden beneath turbid layers such as biological tissues, imaging becomes challenging, even impossible. However, if the point spread function of the system is known from the presence of a guide star, application of common deconvolution algorithms can be a convenient approach to reconstruct even heavily scrambled images. In this work, we present the severity of scattering and the capability of deconvolution techniques in optical settings realistically mimicking biological applications.

Methods

We determine the point spread function (PSF) of the optical path using a single fluorescent bead hidden behind a scattering layer. Once the PSF is obtained, a scene containing several beads is brought to the exact the same position behind the scattering layer. The scrambled image of the scene is then deconvoluted with the PSF. Plastic films and thin slices of chicken tissues are used as scattering layers.

Results

Despite the low signal provided by small fluorescent particles and their short distance of a few millimeters to the turbid media, the reconstructed images reproduced the original scenes successfully. The spatial variance of the PSF caused by the inhomogeneous scattering layer mainly limited the size of the reconstructed area.

Conclusion

Our method overcomes the negative effects of scattering on the detection side of an imaging system. However, it can be combined with wavefront shaping methods optimizing the illumination path as well leading to even further increase of signal to noise ratio and image quality. The required guide star can be brought inside the biological sample to a desired position using optical fibers as a light guide or using capillaries filled with bright fluorescent molecules.

Background

Most optical imaging systems suffer from sample induced, as well as system induced aberrations and scattering. Hence, great efforts in terms of adaptive optics and digital image processing have been made in order to correct imperfections within the optical path and to restore the original scene from the acquired image.

Traditional adaptive optics techniques in fluorescence microscopy rely on direct wavefront measurements, employing e.g. a Hartmann-Shack sensor [1,2,3]. This sensor detects and analyses either the aberrant wavefront of the backscattered excitation light [4, 5] or the wavefront of a fluorescent guide star [6,7,8]. The compensation of the aberrations is then performed by a spatial light modulator or a deformable mirror device. These techniques can achieve correction speeds on the sub-second time scale if the guide star is bright enough. However, direct wavefront sensing with conventional sensors requires ballistic photons. Thus, they cannot be employed in the case of multiple scattering, which leads to the complete loss of phase information.

Alternatively, iterative wavefront shaping methods can be used to create a certain image metric such as maximum brightness or maximum sharpness [9,10,11]. These methods require only wavefront shaping devices making the overall system cheaper and less complicated. But achieving the desired optimum involves capturing of large number of frames or necessitates dedicated techniques working with less number of iterations [12,13,14].

Also, Fresnel guide star based approaches can correct aberrations on the illumination path as well as the detection path [15, 16].

An easier approach in case of severe scattering is directly to apply deconvolution within the range of the optical memory effect as stated in the work of Edrei et al. [17]. Memory effect describes wavefront correlation valid within a certain angular range [18]. Even though speckle patterns behind a turbid medium look random, they are highly correlated if the incoming light hits the scattering layer by only a slightly different angle. The area at image plane which can be scanned without losing the speckle correlation dictates the region where deconvolution would perform well despite scattering. In general, deconvolution operations [19,20,21] aim to remove or reverse the effects of blurring, noise and scattering in microscopy images. It is one of the key image processing tools enhancing the contrast of 3-D image stacks of confocal and widefield microscopes. Deconvolution enables a more accurate quantitative data analysis, increases the signal to noise ratio and restores fine details [22,23,24]. Common algorithms require the knowledge of, or at least a good estimation of the system’s response to a single point source – i.e. the point spread function (PSF). A final image containing the signal of many fluorescence emitters mathematically corresponds to the convolution of the original scene with the PSF, which allows the application of the inverse deconvolution operation.

In this paper, we present how we traced back the fluorescence signal behind scattering media by applying deconvolution schemes. The main difference compared to the work of Edrei et al. is the optical setup which mimics a realistic biological scene. Biologists mostly apply elaborate techniques like clearing or slicing during sample preparation to minimize image degradation due to scattering. We saw in the work of Edrei et al. a great potential for image restoration of biological turbid samples which might reduce the sample preparation steps and even enable in vivo imaging. Instead of direct laser illumination, we employed small fluorescent particles as a signal source providing far less signal. We also reduced the distance between the signal source and the scattering layer from several centimeters down to a few millimeters resulting in a reduced area in the image field where the memory effect holds. The determination of the PSF, based on the guide star approach still has led to successful reconstruction as shown below.

Methods

Optical setup and the measurement procedure

Our samples consisted of 4 μm yellow-green fluorescent beads (Invitrogen FluoSpheres®, USA). The beads were diluted 1:5000 in distilled water and squeezed between a coverslip and a glass slide as a thin film embedded in water. The arrangement was then mounted on a manual translation stage (MS1S, Thorlabs Inc., USA) for three axis translation. A cyan laser (488 nm, Spectra-Physics, USA) illuminated the sample after passing a 20 μm pinhole (Thorlabs Inc., USA). A neutral density filter wheel (Thorlabs Inc., USA) allowed the adjustment of the laser power. The power applied to the beads was a few tens of microwatts without the scattering layer and up to 2 milliwatt with the scattering layer. An objective lens with long working distance of 5.2 mm (Zeiss Plan-Neofluar, 10×/0.3) collected the fluorescence signal and imaged the beads directly onto the camera (ORCA-Flash 4.0 LT, Hamamatsu, Japan). We employed no other lenses than the objective lens. The magnification was 20× (calculated as image distance divided by the object distance) as given by the classical lens equation. The typical camera exposure times were set to 30 ms for imaging without the scattering layer and 300 ms to 1 s in the presence of the scattering layer (see also Fig. 1. for the schematic of the setup). A bandpass filter (FF01–525/45, Semrock Inc., USA) and a notch filter (NF488–15, Thorlabs Inc., USA) were placed in front of the camera to eliminate laser light and ambient light.

Fig. 1

The experimental setup. ND filter: neutral density filter. The blue line represents the excitation light; the green line represents the fluorescence emission. The scattered fluorescence signal is directly imaged onto the camera via objective lens. The magnification is given as the image distance divided by the object distance. The filters in front of the camera transmit the fluorescence and block the laser light

First, we determined a location within the sample with only a single illuminated bead to be used as a guide star. Second, we determined another location with several beads serving as a more complicated fluorescent scene. Finally, we placed a scattering layer between the sample and detection objective. Both locations within the sample are then brought to the exact same position in front of the scattering layer one after another. As expected, the scattering layer scrambled the fluorescence signals from the guide star, as well as from the scene, heavily. We used the scattered fluorescent image of the guide star as the point spread function (PSF). We then deconvoluted the scrambled image of the scene using this PSF.

Deconvolution operations were performed with Matlab® R2015a (The MathWorks Inc., USA) using the built-in functions [25,26,27] and with the ImageJ DeconvolutionLab Plugin (Biomedical Imaging Group, EPFL, Switzerland, [28]). We applied Richardson-Lucy algorithms as well as blind deconvolution.

Results

Plastic film as a scattering layer

We started our experiments with a piece of plastic bag of a thickness of 75 μm as a technical scattering layer. To determine the scattering properties of the plastic film; we measured the total reflectance, total transmittance and the collimated transmittance of the scattering layer as suggested by Prahl et al. [29, 30]. We inputted the measured values into his open access “inverse adding-doubling program” [31]. A detailed explanation of the characterization of our technical layer is provided in the Additional file 1. We found the scattering mean free path on different positions in the plastic film to be in the range of 40 μm to 70 μm. The anisotropy factor (average cosine of the scattering angle) varied from 0.94 to 0.99 indicating strong forward scattering. The scattering mean free paths of many biological tissues vary between 30 μm to 300 μm by anisotropy factors of 0.8 to 0.98 [32]. Thus, our technical layer suitably mimicked a thin slice of biological tissue.

We took images of our guide star and another fluorescent scene behind the plastic film with the procedure explained in the previous chapter. Exemplary sets of results are shown in Figs. 2 and 3.

Fig. 2

a The image of a single fluorescent bead used as guide star. b The same scene as in (a) scrambled by the scattering layer indicating the PSF of the system. c The image of a fluorescent scene. d The same scene as in (c) scrambled after passing the same part of the turbid medium. e Reconstruction of the fluorescent scene seen in (c) by deconvoluting image (d) with image (b) using blind deconvolution with 80 iterations. The distance between the fluorescent sample and the scattering medium was set to 5 mm. All images are contrast-adjusted for better visibility as can be seen on individual colormap scales. The colormaps in parts (a) – (d) are scaled in 16-bit arbitrary units determined by the camera readout and represent the physically measured fluorescence intensity. The colormap in part (e) represents the mathematically calculated pixel values after deconvolution. The scale is normalized since the values vary depending on the number of iterations and the chosen algorithm. This holds true for the following figures as well

Fig. 3

Another data set acquired with a plastic bag as turbid medium placed 5 mm in front of the fluorescent sample. a The image of a single fluorescent bead used as guide star. b The same scene as in (a) scrambled by the scattering layer used as PSF of the system. c The image of a fluorescent scene. d The same scene as in (c) scrambled after passing the same part of the turbid medium. e Reconstruction of the fluorescent scene seen in (c): Image (d) is deconvoluted with image (b) using Richardson-Lucy deconvolution with 120 iterations. All images are contrast-adjusted for better visibility as can be seen on individual colormap scales

Chicken tissue as a scattering layer

After these encouraging outcomes with a technical scattering layer, we moved on to include a biological tissue as the turbid medium. Also in this case, the reconstruction of the original scenes succeeded, as depicted in Figs. 4 and 5. The thicknesses of the chicken tissues were approximately 100 μm and 300 μm respectively. Figure 5 demonstrates the dependence of the deconvolution success on the memory effect range. Structures within 25 μm around the center were reconstructed whereas information from the particles further away got lost. On samples thicker than 300 μm, the fluorescence signal reaching the camera dropped to the noise level hindering the evaluation of our technique.

Fig. 4

A data set acquired with a dry chicken tissue with a thickness of about 100 μm, taking the role of turbid medium, and placed 1 mm in front of the fluorescent sample. (a) The image of a single fluorescent bead used as guide star. (b) The same scene as in (a) scrambled by the scattering layer indicating the PSF of the system. (c) The image of a fluorescent scene. (d) The same scene as in (c) scrambled after passing the same part of the turbid medium. (e) Reconstruction of the fluorescent scene seen in (c): Image (d) is deconvoluted with image (b) using Richardson-Lucy deconvolution with 200 iterations. All images are contrast-adjusted for better visibility as can be seen on individual colormap scales

Fig. 5

A data set acquired with a chicken tissue with a thickness of about 300 μm as turbid medium placed 1 mm in front of the fluorescent sample. (a) The image of a single fluorescent bead used as guide star. (b) The same scene as in (a) scrambled by the scattering layer used as PSF of the system. (c) The image of a fluorescent scene. (d) The same scene as in (c) scrambled after passing the same part of the turbid medium. (e) Reconstruction of the fluorescent scene seen in (c): Image (d) is deconvoluted with image (b) using Richardson-Lucy deconvolution with 150 iterations. The reconstruction succeeded over a field of approximately 25 μm as marked in (c) with a white circle. All images are contrast-adjusted for better visibility as can be seen on individual colormap scales

Discussion

The determination of the PSF of a scattering layer using a guide star has led to successful reconstruction of drastically degraded fluorescent scenes as long as the signal to noise ratio was sufficient. The incoherent nature of the fluorescence allowed a straight-forward deconvolution because the intensity distribution of the camera image consisted of the superposition of the intensity distributions of adjacent points in the image field. In case of coherence light sources, characterization of the interference effects would have been required.

We recorded the PSF of the optical path using fluorescent particles of 4 μm diameter to achieve a sufficient signal to noise ratio at the image plane despite scattering. Strictly speaking, these rather big particles cannot be considered as a point source. If smaller particles were behind the scattering layer, deconvolution of their image with the PSF recorded by these big particles would have produced artefacts. Smearing effects depending on the number of the iterations would have occurred. Therefore, the deconvolution operations shown here perform reliably for reconstruction of objects with a size of > 4 μm.

The main bottleneck of this technique is the spatial variance of the PSF as given in every type of guide star based method. The range of spatially invariance PSF determines the size of the reconstructed area and is directly related to the memory effect range. However, compared to isotropically scattering media, the anisotropically forward scattering biological tissues predict an extended range of the memory effect [33] where the PSF is still valid making this technique suitable for imaging in biology.

The required guide star can be brought inside the biological sample to a desired position using optical fibers as a light guide or using capillaries filled with bright fluorescent molecules. If no artificial structures are desired at all, one can fluorescently label a structure of the specimen as well. However, such a labeling via expression of fluorescent proteins or utilizing antibodies might not provide enough signal.

Edrei et al. used either a diffuser (negligible thickness of scattering layer but completely isotropic scattering) or 1 mm scotch tape as turbid medium. Both turbid layers used in his study would cause more severe scattering than our chicken tissues. Since the fluorescence signal is very weak compared to direct laser light as used by his work, the signal to noise ratio at the camera would drop quickly on such high degrees of scattering. Here, the main limiting component from the technical side is the objective lens with the limited numerical aperture. The higher the scattering (due to high scattering coefficient and/or the thickness of the scattering layer) the bigger is the angular area the photons leave the scattering layer and the higher is the number of photons which cannot be collected by the objective lens. As stated in the Methods Section, we employed an air objective lens with long working distance of 5.2 mm in our setup with a moderate numerical aperture of 0.3. This objective lens allowed us to insert and evaluate different scattering layers easily. For further studies, an objective lens with higher numerical aperture with appropriate embedding medium is recommended to collect scattered photons from higher angles (e. g. glycerol objectives with numerical aperture up to 1.3). This would increase the penetration depth as well which has been 300 μm in our study so far. Besides, one should note that the anisotropic scattering in biological samples is more beneficial compared to the isotropic scattering in technical samples in terms of signal collection.

If the guide star is bright and stable enough, i.e. if the size of light emitting particle is big enough and still not disturbing the physiology of the sample, some in vivo applications can be performed as well. For this purpose, the guide star image and image of the scene in its close vicinity can be captured continuously with the camera with short exposure times in tens of milliseconds range. Image data at video rate can be handled and processed online or offline with a dedicated computer. A big challenge of such an application would be the differentiation of the signals coming from the guide star and the structures of interest. The guide star can be labeled with a slightly different color and a synchronized filter wheel in front of the camera can enable the differentiation of the images. Since the scattering properties depend on the wavelength, the labeling of the guide star and the biological structure must be as similar as possible. Another possibility could be to label the structures of interest with switchable fluorophores which can be turned on and off depending on the wavelength and timing of the excitation laser. However, if the in vivo dynamics change the scattering properties within a few milliseconds or faster, this technique would most probably suffer from low signal. Additionally, fast transfer of image data would be challenging.

In most experiments, Richardson-Lucy as well as blind deconvolution algorithms could reconstruct - at least the outlines of the original scene - similarly well. Richardson-Lucy algorithms produced increasingly negative values from the image corners towards the center with more iterations. In the corresponding figures above, these negative values were set to zero.

The inverse adding-doubling method used to characterize the technical scattering layer assumes conditions like homogenous optical properties and infinite slab geometry. These conditions were not fully satisfied. However, our samples had an area several times bigger than the illumination spot avoiding any edge effects. Movement of the scattering layer within a few millimeters resulted in a variation of less than 10% of measured values. The variation became bigger by displacements exceeding 5 mm. Therefore, we mentioned the mean free path and the anisotropy factor of the sample not as absolute values but as a range as mentioned in the Results Section.

Conclusion

The structures of interest in biological tissues are covered with scattering layers both on the illumination path and detection path. Also, a camera cannot be placed behind the tissue making a backscattering setup necessary. This paper primarily presents a method to overcome the scattering on the detection side but it also includes a convenient combination possibility with wavefront shaping techniques optimizing the illumination path. Many of wavefront shaping methods make use of a feedback signal coming from behind the scattering layer [34,35,36,37,38,39,40,41,42]. The guide star required in this work can be used as such a feedback signal source for the optimization on the illumination path as well. For example, the overall increase of fluorescence signal at a certain point within the ROI can be used as a metric for the illumination optimization resulting in an adequately excited small fluorescent structure. The scrambled image of this structure can then be used as the PSF of the detection path. If the wavefront is optimized to create a single focus, it can be scanned within the same memory range valid for the deconvolution area as well. Instead of fluorescent particles, also an optical fiber as a light guide can be brought into the sample.

Besides a backscattering setup, a light sheet microscope would benefit from our approach to a great extent. Here, the illumination and imaging paths are decoupled allowing every type of improvement on the excitation side. Deconvolution can then be applied on the detection side independently. Bourgenot et al. has already showed the power of adaptive optics for the imaging arm of a light sheet microscope which could have been applied on the illumination arm as well [43].

Our method can be combined not only with the already mentioned methods but also with techniques based on optical phase conjugation [44, 45]. Optical phase conjugation proved to be a powerful tool for focusing light through tissues of thicknesses up to 10 cm [46] and is also capable for high speed applications [47]. Optical phase conjugation of fluorescence signal behind turbid media have been showed as well [48]. Complementing these techniques with deconvolution schemes would lead to an overall advance in the imaging of biological samples.