- Open Access
Adaptive particle image velocimetry based on sharpness metrics
Journal of the European Optical Society-Rapid Publications volume 14, Article number: 5 (2018)
Optical distortions can significantly deteriorate the measurement accuracy in particle image velocimetry systems. Such distortions can occur at fluctuating phase boundaries during flow measurement and result from the accompanied refractive index changes. The usage of a wavefront sensor can be hindered by disturbing light reflexes or scattering.
A combination of sharpness metric image evaluation and iterative optimization is demonstrated. The sharpness metric is used as an indicator for wavefront aberrations in order to correct low-order Zernike modes that influence the image quality of particle image velocimetry.
In this work we outline a sharpness metric based aberration correction with a deformable mirror, applied for the first time to particle image velocimetry. The proposed method allows for the reduction of systematic measurement uncertainties in particle image velocimetry.
Our approach offers a new way to reduce static or slowly changing wavefront distortions in a fluid flow measurement setup in which a wavefront sensor is not applicable.
Adaptive optics has developed to an established method in many research fields of applied optics. Adaptive optics is used in astronomy  for compensating atmospheric turbulence. Another field of application is ophthalmology  in order to get sharp retina-images within the human eye or effectively perform vision correction. In microscopy  for biomedical applications, aberrations mainly result from different cell layers. Aberrations can also influence the point spread function (PSF) for high-resolution confocal microscopy . The drawback of intrinsic aberration effects in a measurement system or from a measurement object can be compensated by applying spatial light modulators. Several methods exist to determine the wavefront of a light wave. Beside interferometric methods and holography-based modal wavefront sensing , the Hartmann-Shack wavefront sensor  is a widely spread tool. Beside active in-situ correction of disturbed wavefronts, also post-processing approaches can be applied to improve the image quality, e.g. for removing turbulence effects  or distortion correction based on affine transformation algorithms. If the optical access for a wavefront sensor is not available, i.e. a transmission measurement is not performable, then a sensor based closed-loop setup cannot be applied , except a Fresnel guide star approach is applicable [9, 10].
Wavefront sensor-less adaptive optics is commonly used with a pinhole and a single photodetector [11, 12] allowing single light spot correction i.e. the intensity is maximized and the spot shape is improved (e.g. for confocal microscopy). There are also sensor-less concepts in terms of FFT-based aberration correction of low spatial frequencies  and by using the second moment of the image Fourier transform .
However, image based wide-field correction requires a CCD-camera and a more complex optimization strategy . Usually an iterative approach is chosen, which does not require an additional sensor beside the imaging camera. Iterative methods are expected to be time-consuming due to the large parameter space. Either single elements of the actuator can be addressed or models for the expected optical distortion can be assumed. The latter one leads to a model based sensor-less correction  which often uses Zernike modes.
Particle imaging velocimetry  is an image-based measurement technique that is used for fluid flow measurements. Seeding particles that follow the flow are added to the fluid. Usually a light-sheet illumination is generated, producing a homogenous light plane which is the measurement area from where the seeding particles scatter light to the detector (CCD camera). By evaluating the position shift of the cross-correlation (CC) peak between two consecutive frames, the particle movement and therefore the flow velocity can be determined . The quality of the particle images is essential for low systematic uncertainties. The less sharp a particle image is, the broader is the cross-correlation peak and the larger the uncertainty in determining the spatial shift between two frames (see also Fig. 1).
In the present paper we show that it is possible to effectively reduce measurement uncertainties in particle image velocimetry (PIV) by using a sharpness metric and a systematic linear search algorithm which finds the optimal solution within several seconds by cycling through the deformable mirror modes. Similar approaches have been demonstrated in the past for static microscopy applications , but not for cross-correlation based flow measurements. In our work the source of aberration is the deformable mirror itself  since it gives a reproducible control on the experimental outcome in order to verify the best combination of sharpness metric and search algorithm for practical application.
The setup uses a large bandwidth actuator, namely a deformable mirror with a single-element settling rate of 1 kHz. For single-element testing of an 8-bit deformable mirror with 69 elements, the parameter space has a size of 25669 which is far too large for a systemic search algorithm. Therefore we use Zernike mode testing  of seven modes after Noll index enumeration (from Z4 to Z10, see also Fig. 2). By applying 100 amplitude values for each mode it results in a much smaller search space of 1007 possible combinations. As a measure of image quality we tested five different sharpness metrics. Many sharpness metrics exist [20, 21]. It has to be noted that the sharpness metric has to fit to the observed scene and type of image .
Search algorithms can be divided into a systematic type as the linear search algorithm is, and non-systematic type as the Nelder-Mead simplex  or the stochastic gradient descent  are. All these algorithms show substantially different behavior in terms of the time needed to identify global extrema.
Setup and software
As it is depicted in Fig. 3, the experimental setup consists of a laser source (continuous wave, wavelength: 561 nm) which is combined with a cylindrical lens in order to generate a light sheet for illumination of the seeding particles (with a size of 10 μm in diameter) within a water basin. A microscope objective (Plan Apo, WD = 34 mm, magnification = 2, NA = 0.2, infinity corrected) observes the light sheet plane from above. The image path is combined with a correction element namely a deformable membrane mirror (DM) (Alpao-DM69) with 69 elements, which is located behind the conjugate plane of the microscope objective thus introducing and also correcting geometrical distortions of the image. Therefore the correction is mainly anisoplanatic, which is expected to be advantageous for an air-water interface in practical applications even though it is connected with a loss of light intensity. After reflection on the mirror membrane the light is focused onto a CCD camera (Basler Pilot piA640-210gm, 648 × 488 pixel, 210 Hz frame-rate). The image data are evaluated on a workstation PC (CPU Intel Xenon CPU E5-1650 3.6 GHz, 12 Cores, GPU Nvidia Quadro K1200, 32 GB RAM, Win10 64bit) in order to determine the sharpness metric value in dependence on the Zernike-mode amplitude that was set to the deformable mirror. For data processing and sharpness metric evaluation MATLAB R2016a was used in combination with parts of the Image Acquisition and Optimization Toolbox. The membrane settling time is given by 10 ms whereas single element settling time is about 1 ms. The measured duration time values for sharpness metric calculation were 3 to 13 ms depending on the type of sharpness metric (see Table 1). A time of 9 ms was needed for the acquisition of one PIV image (with GigE in MATLAB).
For PIV image distortion seven Zernike modes have been applied to the deformable mirror. The sharpness evaluation was performed with five different sharpness metrics under test regarding the calculation speed and the suitability for correct global extremum estimation (see also Table 1). An open-source PIV algorithm was applied to the measured flow data  for three cases. The first one is the undisturbed case with a flat DM. The second case is the disturbed one where a set of Zernike mode amplitudes was loaded to the DM which deteriorate the image quality significantly. And the third case is the corrected one where a linear search algorithm was applied to systematically correct for the applied Zernike mode amplitudes.
Cross-correlation peak quality
In Fig. 2 is depicted the Zernike-mode enumeration after Noll index  that have been considered for image distortion and correction. Seven Zernike modes (from Z4 to Z10) have been considered for image correction. In Fig. 2(a) is shown the qualitative phase change within a circular pupil delivered by a deformable mirror. In Fig. 2(b) is shown the impact on a Gaussian light distribution, namely an imaged scattering particle. The intensity distribution of the PSF is modulated by the applied phase change. Defocus (Z4), astigmatism (Z5, Z6), coma (Z7, Z8) and trefoil (Z9, Z10) can be set to the deformable membrane mirror with RMS values from (− 5 to + 5) μm. This results in peak-to-valley values of about 30 μm depending on the mode (compare Fig. 2(a)).
Image distortions result in deteriorated cross-correlation functions, which lead to large systematic uncertainties for the velocity measurement. This behavior is demonstrated in Fig. 1. A sharp PIV image sequence (Ref) and disturbed image sequences (Z4, Z7) were evaluated concerning their cross-correlation (CC) peaks. At first the auto-correlation (AC) was calculated for each image. The position change of the cross-correlation peak with respect to the AC center gives the average movement of the seeding particles in the observation window (commonly known as interrogation window). For degraded images that are disturbed by aberrations (here shown for Z4, Z7), the position change measurement is worse since the cross-correlation peaks is broadened and deformed depending on the Zernike mode or a superposition of them.
Different types of sharpness metrics  have been applied in order to estimate sharpness and image quality of particle images. The following types of sharpness metrics have been used for feedback control of a deformable mirror. They are discussed in the following.
The sum of the squared Laplace images S1 is a metric that weights large grey-value differences in x and y direction. The image is convoluted with the Laplace-Kernel and the borders of the image are deleted. Here, M and N are the maximal available pixel number in x and y direction (M = 648, N = 488).
The histogram evaluation corresponds to a method by Mendelsohn und Mayall  and is a well-established approach. It is based on the fact that only a few grey-values exist for blurred images. Sharp images possess a large gray value distribution. The metric is a weighted sum of histogram intensities Hk with a threshold T operating as a low intensity noise-filter.
The Riemann tensor metric S3  investigates the curve variation in Riemann space since the intensity distribution of an image can be mapped to Riemann space.
The steepness metric S4 was designed by us for the needs of the experiment. It is a sum of the quotient of maximal grey value Imax,Dk for a particle and the value of its longest axis Dk. This metric averages over all steepness values found in the image.
The metric of maximal gradients S5  evaluates the occurrence (frequency) of grey-value differences di within the surrounding area of a particle (square of 21 by 21 pixel). The metric sums up all grey-value differences for the number of areas A around detected particles.
In order to have a time-efficient feedback loop, the computational cost for sharpness metric calculation of each image is important. The computation time was measured with MATLAB Profiler (in ms) for the tested metrics. Note that the metric calculation time is not supposed to be much larger than the mirror membrane settling time (10 ms) and the image acquisition time (9 ms).
Results and discussion
Several Zernike mode amplitude scans have been performed for different sharpness metric calculation methods. This has been done for PIV particle images during fluid flow. Figure 4 shows the normalized metric behavior of five different metrics. The scans have been done in the available amplitude RMS range from (− 5 to + 5) μm. The behavior of all five metrics is symmetric around RMS = 0 μm. Note that the metrics possess different widths of decline, which mainly determines the number of necessary amplitude values that have to be set in order to find the extremum. The width of decline is therefore important for the overall correction speed in a feedback loop. Important is also that the reduction of noise is crucial for precise extremum estimation. Here a defocus (Z4) of 2.7 μm was introduced to the measurement system by a centered lens with a long focal length of 140 mm. All applied metrics match the defocus value during the Z4 scan. By looking at the astigmatism behavior, which should give zero amplitude value for Z5 and Z6 (Fig. 4(b) and (c)) it turns out that only metric S4 (steepness) performs well since the other metrics detect two symmetric Z5 and Z6 values as optimum that do not correspond to sharp images. Higher modes up to Z10 show similar problems for all metrics except S4 (data are not shown here). Therefore S4 (steepness) was selected as a reliable indicator for aberration correction during PIV measurements.
A systematic optimization algorithm has been used, the linear search. Linear search goes step by step through the parameter space and is testing one Zernike mode after another. The problem, that can arise, is, that the algorithm can be trapped in a local extremum for the applied metric. The advantage is, that it is relatively fast (a few seconds).
The correction method is currently not real-time capable for a free water surface in a basin, but it can be used for static aberration in an optical access or for slowly varying aberrations e.g. on a droplet or bubble surface.
In Fig. 5 is shown such a linear search optimization procedure, which was performed during laminar water flow in the basin. While continuously seeding particles are streaming through the field of view, the sharpness metric S4 is iteratively optimized. The water surface was kept calm and was therefore not disturbing the measurement here. A set of randomly selected Z amplitudes from mode 4 to 10 was applied to the deformable mirror. The resulting PIV images appeared heavily blurred and deteriorated. The linear search needed 19.6 s to iteratively compensate the artificially introduced distortion by the deformable mirror. After correction the PIV images recovered in sharpness and the scattering particles appeared as symmetric Gaussian intensity distributions. Note that depending on the abort condition which mainly influences the time consumption, the recovered Z amplitude values after correction can vary by within 10% in comparison to the introduced distortion.
Flow profile measurement have been performed in the water basin directly after a nozzle. A characteristic velocity field after the nozzle is formed. It was measured with PIV as it is depicted in Fig. 6a (undisturbed case). The flow direction is indicated here by the arrow located at the center of each interrogation window. The length of the arrows represents the velocity magnitude v. The peak velocity in the flow field had a value of 14 mm/s. By inserting aberrations in form of randomly generated Z amplitude values (compare Fig. 5), the PIV measurement is significantly deteriorated (disturbed). The flow profile cannot be resolved anymore and the peak velocity drops to 9 mm/s. After optimization the measurement quality has improved (corrected case) and the flow profile is measurable with the same accuracy as in the undisturbed case.
The relative standard deviation of the velocity magnitude σv/v was calculated and is plotted in Fig. 6b. It is proportional to a system-intrinsic turbulence value arising from PIV evaluation. For very low velocity magnitudes in the shear layers σv/v is high (up to 150%), whereas it is low for large velocity magnitudes (a few percent). In the disturbed case the measured velocity magnitude decreases and σv/v increases in the corresponding interrogation windows (velocity arrows are overlaid here for better orientation in the flow field). A large σv/v value leads to a lower detection rate of real turbulences in a measured flow field. After correction σv/v is comparable to the undisturbed case over the whole field of view.
In this paper we demonstrated an iterative approach for aberration correction i.e. without using a wavefront sensor. The technique was tested with five different sharpness metrics. Seven Zernike modes have been combined in a superposition to generate an optical distortion. The applied sharpness metrics were characterized in terms of calculation time, decline broadening and the ability in correctly determining sharp images. The metrics of squared Laplace image (3 ms), Mendelsohn and Mayall (3 ms), Riemann tensor (5 ms), steepness (8 ms) and maximal gradients (13 ms) have been tested. Only the steepness metric (S4) was able to determine sharp images for all seven Zernike modes.
A linear search algorithm was demonstrated as a reliable optimization method which requires about 20 s on a workstation PC for a complete optimization run.
We emphasize the importance of undisturbed particle images for a reliable PIV evaluation that is based on cross-correlation peak determination. For optical distortions that deteriorate the point-spread function of the system, the cross-correlation peak is broadened and deformed which leads to large systematic and statistic uncertainties in the velocity measurement.
Several attempts have been already undertaken in the past in order to reduce uncertainties in fluid flow measurements through an optical distortion  and even a fluctuating water surface . In latter case a beam steering correction for a 0D measurement volume was accomplished. An interference pattern was optimized within the focal region in order to reduce measurement uncertainty. Smooth surface water waves are correctable due to their moderate wave height (200 μm) and frequency (300 Hz) that are within reach of the available actuator amplitudes and frequency range. Our findings indicate that water surface waves can become correctable with low order Zernike modes for the improvement of PIV measurements through fluctuating air-liquid interfaces. An important problem that has to be solved is reduction of the time budget in the correction system. A personal computer based operating system is not real-time capable. Therefore a field-programmable gate array (FPGA) could be of advantage in terms of image acquisition and metric calculation time.
Particle image velocimetry
Hardy, J.W.: Active optics: a new technology for the control of light. In: Proceedings of the IEEE, vol. 66, pp. 651–697 (1978)
Fernandez, E., Artal, P.: Membrane deformable mirror for adaptive optics: performance limits in visual optics. Opt. Express. 11, 1056–1069 (2003)
Booth, M.: Adaptive optics in microscopy. In: Optical and Digital Image Processing: Fundamentals and Applications, pp. 295–321. Wiley-VCH Verlag, Weinheim (2011)
Yoo, H.W., Van Royen, M.E., Van Cappellen, W.A., Houtsmuller, A.B., Verhaegen, M., Schitter, G.: Automated spherical aberration correction in scanning confocal microscopy. Rev. Sci. Instrum. 85, 123706 (2014)
Dong, S., Haist, T., Osten, W.: Hybrid wavefront sensor for the fast detection of wavefront disturbances. Appl. Opt. 51, 6268–6274 (2012)
Haber, A., Polo, A., Smith, C.S., Pereira, S.F., Urbach, P., Verhaegen, M.: Iterative learning control of a membrane deformable mirror for optimal wavefront correction. Appl. Opt. 52, 2363–2373 (2013)
Xie, Y., Zhang, W., Tao, D., Hu, W., Qu, Y., Wang, H.: Removing turbulence effect via hybrid total variation and deformation-guided kernel regression. IEEE Trans. Image Process. 25, 4943–4958 (2016)
Kulcsár, C., Raynaud, H.-F., Petit, C., Conan, J.-M.: Minimum variance prediction and control for adaptive optics. Automatica. 48, 1939–1954 (2012)
Koukourakis, N., Fregin, B., König, J., Büttner, L., Czarske, J.W.: Wavefront shaping for imaging-based flow velocity measurements through distortions using a Fresnel guide star. Opt. Express. 24(19), 22074–22087 (2016)
Bourgenot, C., Saunter, C.D., Love, G.D., Girkin, J.M.: Comparison of closed loop and sensorless adaptive optics in widefield optical microscopy. J. Eur. Opt. Soc. 8, 13027 (2013)
Booth, M.: Wavefront sensor-less adaptive optics: a model-based approach using sphere packings. Opt. Express. 14, 1339–1352 (2006)
Booth, M.J.: Wavefront sensorless adaptive optics for large aberrations. Opt. Lett. 32, 5–7 (2007)
Debarre, D., Booth, M.J., Wilson, T.: Image based adaptive optics through optimisation of low spatial frequencies. Opt. Express. 15, 8176–8190 (2007)
Burke, D., Patton, B., Huang, F., Bewersdorf, J., Booth, M.J.: Adaptive optics correction of specimen-induced aberrations in single-molecule switching microscopy. Optica. 2, 177–185 (2015)
Warber, M., Maier, S., Haist, T., Osten, W.: Combination of scene-based and stochastic measurement for wide-field aberration correction in microscopic imaging. Appl. Opt. 49, 5474–5479 (2010)
Antonello, J., Verhaegen, M., Fraanje, R., van Werkhoven, T., Gerritsen, H.C., Keller, C.U.: Semidefinite programming for model-based sensorless adaptive optics. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 29, 2428–2438 (2012)
Thielicke, W., Stamhuis, E.J.: PIVlab - towards user-friendly, affordable and accurate digital particle image velocimetry in MATLAB. J. Open Res. Softw. 2, e30 (2014)
Murray, L.P.: Smart Optics: Wavefront Sensor-less Adaptive Optics-Image Correction through Sharpness Maximisation (2006)
Facomprez, A., Beaurepaire, E., Débarre, D.: Accuracy of correction in modal sensorless adaptive optics. Opt. Express. 20, 2598 (2012)
Doble, N.: Image Sharpness Metrics and Search Strategies for Indirect Adaptive Optics, Durham Theses. Durham University, Durham (2000)
Muller, R.A., Buffington, A.: Through image sharpening. J. Opt. Soc. Am. 64, 1200–1210 (1974)
Fienup, J.R., Miller, J.J.: Aberration correction by maximizing generalized sharpness metrics. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 20, 609–620 (2003)
Lagarias, J.C., Reeds, J.A., Wright, M.H., Wright, P.E.: Convergence properties of the Nelder--Mead simplex method in low dimensions. SIAM J. Optim. 9, 112–147 (1998)
Noll, R.J.: Zernike polynomials and atmospheric turbulence. J. Opt. Soc. Am. 66, 207 (1976)
Mendelsohn, M.L., Hungerford, D.A., Mayall, B.H., Perrv, B., Conway, T., Prewitt, J.M.S.: Computer-oriented analysis of human chromosomes. II. integrated optical density as a single parameter for karyotype analysis. Ann. N. Y. Acad. Sci. 376–392 (1969)
Ferzli, R., Karam, L.J.: A no reference objective sharpness metric using riemannian tensor. In: IEEE 3rd International Workshop on Video Processing and Quality Metrics for Consumer Electronics (2007)
Büttner, L., Leithold, C., Czarske, J.: Interferometric velocity measurements through a fluctuating gas-liquid interface employing adaptive optics. Opt. Express. 21, 30653–30663 (2013)
Radner, H., Büttner, L., Czarske, J.: Interferometric velocity measurements through a fluctuating phase boundary using two Fresnel guide stars. Opt. Lett. 40, 3766–3769 (2015)
Reinhart-Koselleck project (CZ 55/30) of the German Research Foundation.
Availability of data and materials
As given in the present paper.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.