Chinese Optics Letters, 2023, 21 (6): 060101, Published Online: Jun. 9, 2023  

Target-independent dynamic wavefront sensing method based on distorted grating and deep learning

Author Affiliations
1 Key Laboratory on Adaptive Optics, Chinese Academy of Sciences, Chengdu 610209, China
2 Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu 610209, China
3 School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100049, China
Abstract
A real-time wavefront sensing method for arbitrary targets is proposed, which provides an effective way for diversified wavefront sensing application scenarios. By using a distorted grating, the positive and negative defocus images are simultaneously acquired on a single detector. A fine feature, which is independent of the target itself but corresponding to the wavefront aberration, is defined. A lightweight and efficient network combined with an attention mechanism (AM-EffNet) is proposed to establish an accurate mapping between the features and the incident wavefronts. Comparison results show that the proposed method has superior performance compared to other methods and can achieve high-accuracy wavefront sensing in varied target scenes only by using the point target dataset to train the network well.

1. Introduction

Phase difference (PD) methods eliminate the uncertainty of phase recovery through two (or multiple) images, which have the advantages of having a simple structure. Such methods are suitable for point and extended source imaging, and can recover continuous and discontinuous phase distribution. Therefore, the PD algorithm has been widely used in the fields of image reconstruction[1,2] and image-based wavefront sensing[36]. However, the PD algorithm based on numerical iteration leads to heavy calculation and low efficiency, which is limited in some scenarios where real-time wavefront sensing is required. Moreover, such a method is easily getting into a local optimum and results in inaccurate results under a large-scale phase aberration situation.

In recent years, deep learning has been introduced into the field of image-based wavefront sensing[712], which can be used to fit the nonlinear mapping relationship between the wavefront phase and the intensity image. These methods avoid a time-consuming iterative process so that they provide an effective way to restore a larger wavefront phase without local minimum problem after appropriate training. Current studies mainly focus on either point target imaging scenes[79,11] or on specific extended target imaging scenes (such as MNIST dataset)[10,12]. However, once the imaging scene is switched, the neural network model can hardly complete the wavefront sensing task effectively. There are also some network models that improve the generalization ability by increasing the diversity of training scenes[10,13]. However, these models lead to huge calculations, which are time-consuming and difficult to apply to real-world situations. When deep learning is used for extended target wavefront sensing, it is impossible to restore the phase directly if both the imaging target and the wavefront phase are unknown. Therefore, it is significant and meaningful to develop an image-based wavefront sensing method that is independent of the imaging target. Such a method can be applied to both a point source target and any extended source target imaging, especially for special application scenarios where the target on the image is dynamically changing, e.g. remote imaging or laser beam transmission where the distance between the target and the observer is changing.

Qi et al.[14] proposed a time-domain feature through mathematical operations to realize target-independent wavefront sensing. Li et al.[15] also used sharpness information of the in-focus and defocus images to get rid of the dependence of the images. These methods provide a feasible way for studying extended target wavefront sensing. However, in order to collect the in-focus and defocus images, it is usually necessary to use a beamsplitter and two cameras to work together at the same frequency or move one camera to collect the images twice. These methods may lead to some limitations in application scenarios with a high demand for dynamic wavefront sensing. Blanchard et al. first proposed a diffractive element called distorted grating and applied it to curvature wavefront sensing[16,17]. Distorted grating is widely used in multi-plane imaging, fluid velocity measurement, and particle tracking due to its strong real-time performance and high efficiency. Compared with a beamsplitter, the distorted grating ensures that the captured images are strictly corresponding.

In this Letter, we propose a target-independent wavefront sensing method based on distorted grating and deep learning. The distorted grating is designed to acquire positive and negative defocus images on an imaging plane simultaneously. Then a normalized fine feature is defined, which eliminates the information of the target and keeps the wavefront information, and we provide a lightweight and efficient network combined with an attention mechanism (AM-EffNet) to invert the unique wavefront aberration. In particular, targets in different scenarios are considered, and the optimal defocus amount of distorted grating is quantitatively analyzed. It is verified that our method has irreplaceable advantages in accuracy and efficiency, which provides potential for much more complicated wavefront sensing tasks.

2. Design of Distorted Grating

When using the PD method for wavefront sensing, it is necessary to ensure that more than two images are acquired synchronously in the same exposure time. In this Letter, the distorted grating is designed in detail to realize it. Such an optical element is, in fact, an off-axis Fresnel zone plate[16].

In Ref. [16], the defocus amount W20 represents the defocusing ability of the grating, W20=R22fg, and the off-axis amount x0 is the distance from the pupil center to the center of the Fresnel zone plate, x0=λR22W20d0, where R is the radius of the grating, fg is the focal length of the grating, λ represents the incident wavelength, and d0 represents the grating period at the center of the aperture.

In this Letter, we further consider two key parameters, lateral distance and splitting ratio. As shown in Fig. 1, when the incident light is vertical to the distorted grating, the grating equation is mλ=d0sinθm;the lateral distance Δy of the ±1st-order diffracted spots from the 0th-order spot on the detection plane is determined by the focal length of the lens fL and diffraction angle θm, Δy=fLtanθm;combining Eqs. (1), (2), and the equation of x0, after determining λ, W20, R, fL, and Δy, the corresponding θm, d0, and x0 can be calculated in turn. Note that if the Δy is too small, then there will be image crosstalk, and if it is too large, then it will overflow to the detector target surface. In the actual process, we need to determine an appropriate lateral distance according to the size of the selected camera target surface.

Fig. 1. (a) Schematic of the distorted grating and (b) geometric relationship of the ±1st diffraction order imaging positions.

下载图片 查看所有图片

Another key parameter is the splitting ratio, which determines the diffraction efficiency of the three diffraction orders of ±1 and 0. In order to make the light intensity distribution of the three images as uniform as possible under the same exposure intensity, the step width is set as a bisection, and the height of the phase step is set as 0.639π[18].

In our work, the distorted grating is designed based on the principles described above. The designed parameters are the wavelength λ=520nm, the lens focal length fL=200mm, the defocus amount W20=1λ, the entrance pupil radius R=6mm, the lateral distance Δy=3mm, and the off-axis amount x0=0.519m.

The structure diagram of the proposed target-independent wavefront sensing method is shown in Fig. 2. First, Zernike coefficients conforming to the Kolmogorov turbulence distribution are randomly simulated to generate the phase screen. Then targets with aberration are imaged by the distorted grating, and the features irrelevant to the targets are extracted as the input of the neural network. Finally Zernike coefficients for wavefront sensing are output.

Fig. 2. Schematic of target-independent wavefront sensing.

下载图片 查看所有图片

3. Feature Extraction of Arbitrary Target

The intensity distribution of the image captured by a camera can be expressed as i=o*psf,where * denotes the convolution operation, i is the image captured by the camera, o is the imaging target, and psf is the point spread function of the optical system, which is proportional to the Fourier transform of the expanded pupil function, ignoring the scaling factors of the coordinates, psf(x,y)=|F[p(x,y)ejϕ(x,y)]|2,where ϕ(x,y) is the wavefront aberration through the aperture, which can be represented by Zernike polynomials. By the Fourier transform of Eq. (3), we can get I=O·OTF,where I and O represent the frequency domain information of the acquired image and the imaging target itself, respectively, and OTF is the optical transfer function of the optical system. The power matrix Mpower and sharpness matrix Msharpness can be calculated from the spectrum of the positive and negative defocus images: Mpower=Id(u,v)·Id*(u,v)Id+(u,v)·Id+*(u,v)Id(u,v)·Id*(u,v)+Id+(u,v)·Id+*(u,v)=F(psfd)·F*(psfd)F(psfd+)·F*(psfd+)F(psfd)·F*(psfd)+F(psfd+)·F*(psfd+),Msharpness=Id(u,v)·Id+*(u,v)Id*(u,v)·Id+(u,v)Id(u,v)·Id*(u,v)+Id+(u,v)·Id+*(u,v)=F(psfd)·F*(psfd+)F*(psfd)·F(psfd+)F(psfd)·F*(psfd)+F(psfd+)·F*(psfd+),where Id+*(u,v) and Id*(u,v) are the complex conjugates of the spectrum of the positive and negative defocused images. Combining Mpower and Msharpness, the normalized fine feature Mfine is defined as Mfine=|Mpower|·|Msharpness|4,where the multiplication operation is used to fuse sharpness features and power features, and the fourth root operation is used to enhance the blurred details. By substituting Eqs. (4), (6), and (7) into Eq. (8), it can be proved that the target-related feature is eliminated in Mfine where the aberration feature still remains. Using fine features as the input, the neural network can be used to invert the unique wavefront aberration. Figure 2 shows the results of extracting normalized fine features from the positive and negative defocus images by replacing different targets under the same wavefront aberrations. The structure similarity index measure (SSIM) results have proved that the proposed features of the different imaging targets are the same (SSIM=1), and once the wavefront aberration changes, the feature also changes, but they are still consistent, which means the proposed features are independent of the target but related to the wavefront aberration.

4. Proposed Neural Network

A lightweight and efficient network based on attention mechanism (AM-EffNet) is proposed to obtain an efficient dynamic wavefront restoration capability. As shown in Fig. 3(a), the proposed network consists of three EffNet-blocks, three attention mechanisms, and one full connection layer. Since the feature map is a centrosymmetric figure, using half the image (224×128) as input provides the same information with fewer network parameters and computations than the original size image. Piston and tip/tilt aberrations are not considered in our work, and the network outputs the 4th to 35th order Zernike coefficients.

Fig. 3. Structure of the AM-EffNet for target-independent wavefront sensing.

下载图片 查看所有图片

Figure 3(b) shows the structure of the EffNet-block and the overall attention process. EffNet-block uses depth-wise separable convolution to replace the traditional convolution operation, which reduces the computational complexity and makes the network more real-time. Channel and spatial attention modules are sequentially applied to learn “what” and “where” to attend in the channel and spatial axes, respectively[19]. FRC×H×W represents the input feature maps, McRC×1×1 expresses the channel attention module, MsR1×H×W expresses the spatial attention module, denotes the element-wise multiplication, and F and F express the result of the channel attention and the final refined output, respectively.

The AM-EffNet is trained with an Adam optimizer at an initial learning rate of 0.001 over 150 epochs, which is based on the PyTorch deep learning framework with NVIDIA GeForce 3070 GPU. The mean-square error (MSE) loss (regression error of the predicted Zernike coefficients) is used as the loss function, and the batch size is set to 32.

Designed distorted grating generated 24,000 pairs of data, which are divided into 10:1:1 for training, validation, and testing, respectively. Each set of data consists of the feature image and corresponding label, which is the wavefront aberration represented as the 4th to 35th order Zernike coefficients conforming to the Kolmogorov turbulence distribution with different degrees of distortion [original wavefront RMS (root-mean-square) ranging from 0 through 1.5λ].

In particular, for the method proposed in this paper, it does not need to use multiple targets for training. On the contrary, it only relies on the simple point target scenes to collect positive and negative defocus images for fine feature extraction, and then uses the AM-EffNet to establish an accurate mapping between the features and the incident wavefronts. It can be used to achieve wavefront sensing of multiple scenes while the network is well trained.

5. Comparison of Feature Extraction Methods

The effectiveness of the proposed normalized fine feature for the wavefront sensing task is verified under a variety of features. Figure 4 shows the comparison results of the different features extracted from the same group of positive and negative defocus extended target images.

Fig. 4. (a) Time domain feature. The red boxes in (b) are the enumerated power features, and the blue boxes in (c) are the sharpness features. We can see that the normalized fine feature in (d) includes the information of the sharpness and the power features with data distributed between 0 and 1. (e) Comparison of loss for different features in training.

下载图片 查看所有图片

In Fig. 4, the normalized fine features realize normalization while combining the sharpness feature and the power feature, which increases both the feature’s numerical stability and the detail representation ability. Four feature extraction methods are used to realize wavefront sensing for the same sets of test data (labels are the same) and the same network (AM-EffNet). Figure 4(e) shows the variation of loss for training using different features from Figs. 4(a)4(d). After testing, the RMSE between the groundtruth and the recovered wavefront phase of methods [Figs. 4(a)4(d)] is 0.101λ, 0.086λ, 0.071λ, and 0.042λ, respectively, which has demonstrated that the proposed feature can better achieve wavefront sensing with higher accuracy.

6. Comparison of Defocus Degree

When using the PD method for wavefront sensing, the selection of the defocus amount is critical. It can be known from Refs. [20,21] that in order to obtain accurate wavefront sensing results, the PV value of defocus aberration is generally within the range of 1λ±λ/2. Based on this, we simulate and design five different defocus amounts of distorted grating with a step size of 0.25λ referring to this range.

For the same network, hyperparameters, and environment, the datasets obtained by distorted grating imaging with five different defocus amounts were used for training and testing, and the results are shown in Fig. 5. We can see that the distorted grating with a defocus amount of 1.0λ has the highest wavefront sensing accuracy, and from the outliers, the value of the RMSE shows that our method still has relatively stable wavefront sensing ability for major wavefront distortion with this defocus amount.

Fig. 5. Residual wavefront at different defocus amount. The letters A, B, C, D, and E in the abscissa express gratings with different defocusing degrees, and the numbers 1, 2, and 3 represent three different degrees of atmospheric turbulence in the range of 0–0.5λ, 0.5λ–1.0λ, and 1.0λ–1.5λ of the original wavefront RMS, respectively. The red circles represent outliers.

下载图片 查看所有图片

7. Applicability of Different Scenarios

In order to explore the applicability of this method to other imaging scenes, 2000 groups of mixed data from different targets are used for further testing. The targets include a point target and four types of extended targets with 400 groups each. The positive and negative defocus extended target images can be obtained by convolving with the positive and negative defocus point spread functions respectively. In order to closely simulate the application scenarios where targets are constantly switched and changed, the five types of targets are randomly and uniformly distributed.

Figure 6 shows a group of the wavefront sensing results with our method in each scenario, and Table 1 shows the average restoration results among five scenarios. After testing, the average SSIM between the restored wavefronts and the original wavefronts is 0.948. The RMSE of the residual wavefront is 0.042, the RMSE/Truth is 5.6%, and the inference time is about 2.0 ms. The simulation results show that our method has robust and high-accuracy wavefront sensing capability in a variety of extended target scenes.

Table 1. Testing Results Based on Our Method

Target typeRMS (λ)Time (ms)SSIMRMSE/Truth
Point source0.0401.960.9525.5%
Resolution chart0.0461.990.9495.6%
Remote sensing0.0352.050.9525.3%
Bird0.0382.040.9485.4%
Nebula0.0511.960.9436.1%

查看所有表

Fig. 6. A group of wavefront sensing results of targets in five different scenarios with our method. The RMSE/Truth represents the ratio of the residual wavefront to the true wavefront.

下载图片 查看所有图片

8. Conclusion

In this Letter, a distorted grating is designed to acquire the positive and negative defocus images of arbitrary targets simultaneously, and the normalized fine features irrelevant to the imaging target itself are extracted as the input of the AM-EffNet, which can achieve fast wavefront sensing with an inference time of about 2.0 ms. Furthermore, the quantitative analysis shows that the distorted grating with 1λ defocus amount has the best wavefront sensing accuracy when the RMS of the original wavefront is within 1.5λ. It has been proved that our method has an effective wavefront sensing ability under diversified application scenarios, which provides potential for future applications like image restoration and real-time distortion correction.

References

[1] FienupC.DaintyJ., “Phase retrieval and image reconstruction for astronomy,” in Image Recovery: Theory and Application (Elsevier, 1987), p. 275.

[2] Y. Rivenson, Y. Zhang, H. Günaydın, D. Teng, A. Ozcan. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci. Appl., 2018, 7: 17141.

[3] R. A. Gonsalves, R. Chidlaw. Wavefront sensing by phase retrieval. Proc. SPIE, 1979, 207: 32.

[4] R. G. Paxman, T. J. Schulz, J. R. Fienup. Joint estimation of object and aberrations by using phase diversity. J. Opt. Soc. Am. A, 1992, 9: 1072.

[5] B. H. Dean, D. L. Aronstein, J. S. Smith, R. Shiri, D. S. Acton. Phase retrieval algorithm for JWST flight and Testbed Telescope. Proc. SPIE, 2006, 6265: 626511.

[6] J. J. Dolne, P. Menicucci, D. Miccolis, K. Widen, H. Seiden, F. Vachss, H. Schall. Advanced image processing and wavefront sensing with real-time phase diversity. Appl. Opt., 2009, 48: A30.

[7] G. Ju, X. Qi, H. Ma, C. Yan. Feature-based phase retrieval wavefront sensing approach using machine learning. Opt. Express, 2018, 26: 31767.

[8] S. W. Paine, J. R. Fienup. Machine learning for improved image-based wavefront sensing. Opt. Lett., 2018, 43: 1235.

[9] H. Guo, Y. Xu, Q. Li, S. Du, D. He, Q. Wang, Y. Huang. Improved machine learning approach for wavefront sensing. Sensors, 2019, 19: 3533.

[10] Y. Nishizaki, M. Valdivia, R. Horisaki, K. Kitaguchi, M. Saito, J. Tanida, E. Vera. Deep learning wavefront sensing. Opt. Express, 2019, 27: 240.

[11] Y. Wu, Y. Guo, H. Bao, C. Rao. Sub-millisecond phase retrieval for phase-diversity wavefront sensor. Sensors, 2020, 20: 4877.

[12] C. Lu, Q. Tian, L. Zhu, R. Gao, H. Yao, F. Tian, Q. Zhang, X. Xin. Mitigating the ambiguity problem in the CNN-based wavefront correction. Opt. Lett., 2022, 47: 3251.

[13] Z. Zhou, J. Xia, J. Wu, C. Chang, X. Ye, S. Li, B. Du, H. Zhang, G. Tong. Learning-based phase imaging using a low-bit-depth pattern. Photon. Res., 2020, 8: 1624.

[14] Q. Xin, G. Ju, C. Zhang, S. Xu. Object-independent image-based wavefront sensing approach using phase diversity images and deep learning. Opt. Express, 2019, 27: 26102.

[15] D. Li, S. Xu, D. Wang, D. Yan. Large-scale piston error detection technology for segmented optical mirrors via convolutional neural networks. Opt. Lett., 2019, 44: 1170.

[16] P. M. Blanchard, A. H. Greenaway. Simultaneous multiplane imaging with a distorted diffraction grating. Appl. Opt., 1999, 38: 6692.

[17] P. M. Blanchard, D. J. Fisher, S. C. Woods, A. H. Greenaway. Phase-diversity wave-front sensing with a distorted diffraction grating. Appl. Opt., 2000, 39: 6649.

[18] F. Xi, Z. Jiang, X. Xu, Y. Geng. High-diffractive-efficiency defocus grating for wavefront curvature sensing. J. Opt. Soc. Am. A, 2007, 24: 3444.

[19] WooS.ParkJ.LeeJ.-Y.KweonI. S., “CBAM: convolutional block attention module,” in Proceedings of the European Conference on Computer Vision (ECCV) (2018), p. 3.

[20] J. R. Fienup, B. J. Thelen, R. G. Paxman, D. A. Carrara. Comparison of phase diversity and curvature wavefront sensing. Proc. SPIE, 1998, 3353: 930.

[21] L. Meynadier, V. Michau, M.-T. Velluet, J.-M. Conan, L. M. Mugnier, G. Rousset. Noise propagation in wave-front sensing with phase diversity. Appl. Opt., 1999, 38: 4967.

Xinlan Ge, Licheng Zhu, Zeyu Gao, Ning Wang, Wang Zhao, Hongwei Ye, Shuai Wang, Ping Yang. Target-independent dynamic wavefront sensing method based on distorted grating and deep learning[J]. Chinese Optics Letters, 2023, 21(6): 060101.

引用该论文: TXT   |   EndNote

相关论文

加载中...

关于本站 Cookie 的使用提示

中国光学期刊网使用基于 cookie 的技术来更好地为您提供各项服务,点击此处了解我们的隐私策略。 如您需继续使用本网站,请您授权我们使用本地 cookie 来保存部分信息。
全站搜索
您最值得信赖的光电行业旗舰网络服务平台!