Opto-Electronic Science, 2023, 2 (4): 220023, Published Online: Sep. 21, 2023  

Deep learning assisted variational Hilbert quantitative phase imaging

Author Affiliations
Abstract
We propose a high-accuracy artifacts-free single-frame digital holographic phase demodulation scheme for relatively low-carrier frequency holograms—deep learning assisted variational Hilbert quantitative phase imaging (DL-VHQPI). The method, incorporating a conventional deep neural network into a complete physical model utilizing the idea of residual compensation, reliably and robustly recovers the quantitative phase information of the test objects. It can significantly alleviate spectrum-overlapping-caused phase artifacts under the slightly off-axis digital holographic system. Compared to the conventional end-to-end networks (without a physical model), the proposed method can reduce the dataset size dramatically while maintaining the imaging quality and model generalization. The DL-VHQPI is quantitatively studied by numerical simulation. The live-cell experiment is designed to demonstrate the method's practicality in biological research. The proposed idea of the deep learning-assisted physical model might be extended to diverse computational imaging techniques.

1 Introduction

Quantitative phase imaging (QPI), as a powerful label-free imaging technique, enables dynamic 2D and 3D non-destructive imaging of completely transparent structures1-3. It uses the refractive index as an endogenous contrast agent to generate subcellular-specific quantitative maps of analyzed live bio-structure4, 5. QPI solutions based on digital holographic microscopy (DHM) encode a complex wavefront information into intensity modulations by the interference of a scattered sample wave and a reference wave6-9. And it can robustly perform the quantitative analysis of wave-matter interactions by decoding phase delay from a hologram. DHM has emerged as a valuable means in the biomedical fields, such as measurements for stain-free biological cells3, 10 , optical metrology of nanostructures11-14, and drug release monitoring invitro15.

Regarding the phase demodulation strategy employed, there are two main configurations for holographic wavefront acquisition in DHM, i.e., in-line and off-axis digital holography (DH). In-line DH records complete wavefront information by the interference of the object light and the reference light on the same optical axis, which can realize full detector-bandwidth phase reconstruction. However, due to the superimposed twin image, the phase retrieval results of samples are severely impacted by imaging artifacts. It always needs to be processed via iterative phase retrieval16, 17 or noniterative phase-shifting methods18-20, which dramatically sacrifices the temporal resolution. Therefore, it is difficult for the in-line DH, which is vulnerable to external disturbance and vibration, to be applied to dynamic measurement. Alternatively, off-axis DH implements twin-image separation by introducing a slight angle between the object beam and reference beam and recovers the complex wavefront of the sample from the single-frame off-axis hologram. Whereas, for achieving the separation of autocorrelation and cross-correlation terms in the spatial frequency domain (SFD), the off-axis DH needs to provide a sufficiently high carrier frequency at the expense of the space-bandwidth product (SBP) of the imaging system21. The slightly off-axis DH regime, as a single-frame high-SBP DH imaging solutions, is therefore proposed22-24. It optimizes SBP through full spectral separation of conjugated object lobes while leaving the autocorrelation term partially overlapped with information-carrying cross-correlation terms. Under this configuration, the inevitable spectrum overlapping causes phase artifacts, which greatly degrades the imaging quality and impairs the practicality of the slightly off-axis DH configuration.

High-accuracy artifacts-free phase recovery from the low-carrier frequency holograms is the key to slightly off-axis DH application. This process is presently implemented by suppressing autocorrelation term iteratively25, utilizing dual-frame decoding scheme26, 27, employing second wavelength assistance28 and performing the 1D limited processing29, 30. With inspiration from the theory of “cepstrum” and homomorphic filtering31, a slightly off-axis DH demodulation scheme based on the Kramers-Kronig (KK) relations is proposed, which utilizes the half-space bandwidth of the sensor to achieve high-SBP imaging32, 33. Although it is able to increase the SBP of full complex field recovery significantly, it inevitably requires intensity restrictions on the object and reference beams and the separation of the cross-correlation terms of the interferogram in the extended SFD. Noteworthily, an exquisite low-carrier frequency fringe demodulation approach has been presented recently, namely variational Hilbert quantitative phase imaging (VHQPI)34. The VHQPI, as an end-to-end pure numerical add-on module, deploys the merger of tailored variational image decomposition35 and enhanced Hilbert spiral transform36 to achieve quantitative phase recovery. It adaptively alleviates the overlapped-spectrum problem and robustly demodulates high-quality phase information, performing excellent practicality in biological applications.

Although VHQPI has demonstrated excellent low-carrier frequency fringe demodulation capability, the algorithm-inherent limitations (e.g., parameter robustness and iterative stability) still cause non-sufficient image frequency component extraction, resulting in imaging artifacts in the phase reconstruction results. Deep learning (DL), as a subfield of machine learning, has currently gained extensive attention in the field of optical metrology and demonstrated great potential in solving optical metrology tasks37-46. When sufficient training data is collected in an environment that reproduces real experimental conditions, the trained model may have advantages over physics-model-based approaches on some issues (e.g., computing speed, parameter adaptivity, algorithm complexity)37. Specifically, in terms of a series of ill-posed inverse phase retrieval problems, the traditional physical model tends to exhibit higher physics complexity and time consumption. Driven by a large dataset, the deep neural network (DNN) can directly and efficiently reconstruct the phase and amplitude images of the objects from the captured holograms47-49. Nevertheless, in DL-based phase recovery tasks, it is pretty tricky and laborious to capture massive datasets and generate the corresponding ground truth, especially when applied to bio-samples. Deep image prior (DIP) applies an untrained network to the solution of several inverse problems without a massive training dataset and ground truth, which can fit a randomly initialized DNN to a single corrupted image50. Inspired by the DIP, an untrained network model named “PhysenNet” is proposed, which incorporates a complete physical model into the conventional DNN to achieve phase retrieval from a single intensity image51.

Inspired by the successful application of the interplay between DNN and the physical model, in this work, we propose a DL-assisted variational Hilbert quantitative phase imaging approach (DL-VHQPI). Unlike the massive-data-driven DL training model, DL-VHQPI, which utilizes DNN to compensate and optimize the possible solutions of the physics-driven model, can achieve high-precision artifacts-free phase recovery using only a small fraction datasets. Specifically, VHQPI, as the underlying physical model, can complete the preliminary extraction of the background components of the fringes to provide a physical prior for the deep learning model. The DNN compensates for the image frequencies that cannot be extracted by the physical model using the idea of residual compensation. Due to the physical model reducing the information entropy of the dataset, the DL-VHQPI performs higher reconstruction accuracy utilizing less than one-tenth of the dataset of the conventional end-to-end model (without the physical model). The simulation experiments quantitatively demonstrate that the proposed method can achieve high-accuracy artifacts-free quantitative phase imaging from single-frame low-carrier frequency holograms. And the results of live-cell experiments demonstrate the practicality of the method in biological research.

2 Principle of VHQPI

The VHQPI, as the physical model of the DL-VHQPI, adaptively and effectively completes the low-carrier frequency fringe demodulation employing the unsupervised variational image decomposition (uVID) and enhanced Hilbert spiral transform (HST). This section will focus on describing the process details and physical limitations of this method. In the DH wavefront recording, the interferogram containing the required object information is constructed upon the coherent superimposition of the object and reference beams. The intensity distribution of the recorded hologram can be expressed as:

I=I1+I2+2I1I2cos(θ)+n=a+bcos(θ)+n.

It consists of a sum of three fundamental intensity components: background (a, incoherent sum of intensitiesI1 andI2 of interfering beams), high-frequency noise (n), and coherent interference fringes term comprised by a cosine function modulated in phase (θ) and amplitude (b,2I1I2). Acquiring the accurate fringes term from the three components is the prerequisite of high-precision artifacts-free phase recovery. The uVID approach achieves image frequency components extraction, which is based on the notion of the classical variational image decomposition to separate the information components of the image with two steps in terms of methodology52, 53: 1) A block-matching 3D (BM3D) algorithm is employed to remove noise with remarkable efficiency54; 2) Background-fringes differentiation is performed using modified Chambolle projection algorithm with an automatic stopping criterion to set the number of projections, and there is no need to pre-set any parameter values35. The based-on uVID image frequency components extraction process is shown in Step 1 of Fig. 1. Although the uVID provides a robust and automatic one-stop-shop solution for single-frame fringe pattern analysis, there are physical limitations in the process of frequency component extraction, i.e., iterative instability and parameter robustness, which directly cause non-sufficient background term removal and then impair phase recovery accuracy and artifacts-suppression effect52.

Fig. 1. Flow chart of slightly off-axis interferometric fringe demodulation based on VHQPI.

下载图片 查看所有图片

To recover the phase information of the object, the uVID-filtered noise-free zero-mean-valued interferogram is then analyzed using the HST algorithm36, as shown in Step 2 of Fig. 1. The HST is the two-dimensional variant of the Hilbert transform (HT), in which the complex analytic signal can be constructed, whereas several requirements must be fulfilled. First, the processed interferogram must be of zero mean value, which is satisfied based on background term removal using the uVID approach. And the amplitude term (b in Eq. (1)) has to be a slowly varying function. This is the so-called Bedrosian theorem which can be applied to general pure-phase objects at relatively low carrier frequencies55. The complex analytic signal constructed by HST can be expressed as

AFP=2I1I2cos(θ)iexp(iβ)F1{SPF*F[2I1I2cos(θ)]},

where,AFP denotes the analytic fringe pattern andSPF is the spiral phase function;F andF1 denote Fourier transform (FT) and inverse Fourier transform (IFT) operator respectively. It is important to emphasize that carrier-free single-shot interferogram analysis is a fully 2D phase demodulation problem, whereas carrier-based FT phase demodulation is a 1D simplification of the HT analytic relation. The HST, therefore, requires the local fringe direction map (β, modulo2π)56. The modulus value and angle of the 2D complex analytic signal constitute the intensity and phase in QPI, respectively.SPF is defined as

SPF(u,v)=u+ivu2+v2=exp[iϕ(x,y)],

where(u,v) is the coordinate of(x,y) corresponding to the SFD.ϕ(x,y) is the polar coordinate phase expression. Figure 1 specifically showcases the schematic diagram of the VHQPI-based low-carrier interferogram quantitative phase demodulation algorithm.

3 Deep learning assisted VHQPI model

VHQPI has been proven to have excellent robustness and practicality in low-carrier frequency fringe demodulation issue though34. However, the algorithm-inherent iterative instability and parameter robustness restrict the image frequency component extraction capability, which will cause the non-perfect background term removal. DL methods driven by massive datasets provide a new route to address this problem by virtue of their high-powerful image feature extraction characteristics. Whereas, when encountering insufficient training data, which is very common, the DL method based on massive datasets may have a poor effect. A feasible scheme is to train the DNN on a stronger-constrained available standardized dataset57. Here, we employ Shannon entropy theory of the images in the dataset for that purpose: the lower the entropy of the datasets is, the more constrained prior information is, giving it a better same-domain generalization ability58, 59. Therefore, in the proposed DL-assisted VHQPI model, the uVID is utilized to extract the image background term as the physical prior of the network to reduce the dataset's entropy. The first convolutional neural network (CNN1) is used to “learn” the residual terms and assists the physical model to complete the preliminary estimation of the background components of the fringes. Furthermore, to further improve the imaging accuracy, the original hologram and the preliminary estimation background are re-fed into the model (CNN2) for advanced component extraction. Dual-channel input is used because the preliminarily estimated background terms have been very close to the ground truth after the first residual compensation by CNN1. Hence, the preliminary estimated background can be used to provide the network with feature guidance and helps CNN2 achieve the advanced component extraction.

As depicted in Fig. 2, with the original hologram as input, CNN1 completes the preliminary background component extraction by compensating for the residual (ϵ1) of the background component acquired by uVID, as shown in Fig. 2(b). With the preliminary estimated background term and the original hologram, the CNN2 (as shown in Fig. 2(c)) uses the two as dual-channel inputs to implement the more advanced background residual (ϵ2) compensation. After the high-accuracy fringes terms extraction, the complete complex analytic signal can be constructed by HST. And then the final phase results are recovered by calculating the angle of the 2D complex analytic signal. The whole method flow chart is shown in Fig. 2(a).

Fig. 2. Deep learning-assisted VHQPI. (a) Total network structure, combining uVID and HST with CNN respectively for phase reconstruction. (b) CNN1 takes a hologram as input and consists of three convolutional layers and a group of residual blocks to achieve compensation of background residuals by learning. (c) The CNN2 network structure is the same as CNN1, except that CNN2 combines the original hologram and the result of the first process into a two-channel input for advanced background compensation.

下载图片 查看所有图片

Moreover, both CNN1 and CNN2 networks are composed of a convolutional layer (Conv), a group of residual blocks (containing four residual blocks), and two convolutional layers. Each residual block comprises two sets of Convs stacked one above the other. The network architecture uses Batch Normalization60 and ReLU activation61 to accelerate the model convergence. It establishes a shortcut between input and output, which can solve the problem of accuracy decline as the network deepens, thereby easing the training process. The output of the Convs is a 3-D tensor of shape(H,W,C), where H andW are the height and width of pixels of the hologram respectively, andC is the number of channels. The hyperparameters of the two networks, i.e., the weights, bias, and convolutional kernels, are trained using backpropagation on mean-squared errors between the results of the network output and the ground truth. The loss function is computed as

Loss(ω)=1H×WYoutputωG2,

whereω represents the parameter space of the model,Youtput is the results predicted by the model, andG is the ground truth.

4 Experiments and results

In this section, we demonstrate the performance of the proposed DL-VHQPI method over the conventional physics-driven low-carrier frequency fringe demodulation techniques and pure DL approach without a physical model (DL-noPhy) through numerical simulation and live-cell experiment. A rich set of paired training data is the prerequisite for network generalization during DL training. It is challenging to acquire a reliable ground truth in the real-world DH system due to environment-induced instability and system-inherent speckle noise. Consequently, we simulated low-carrier frequency holograms and the corresponding ground truth for training and quantitative analysis. We separately constructed the complex amplitude distributions of the object and reference light waves, and then the holograms can be constructed by solving the square of the modulus of the sum of the two. The sum of the squares of the modulus values of the two was calculated to obtain the background (ground truth) needed for training. The more specific process can be found in Supplementary information Section 1.

In the live-cell experiment, we used the Digital holographic smart computational light microscope (DH-SCLM) developed by SCILab, and turned it to a slightly off-axis state for hologram acquisition1. In the DH-SCLM, the object wave transmitting the objective lens (UPLanSAPO ×20/0.45NA, Olympus, Japan) interferes with the reference light and is recorded by the camera (The Imaging Source DMK 23U274, 1600×1200, 4.4 μm). The central wavelength of the illumination is 532 nm. The used sample is Henrietta Lacks (HeLa) human cervical cancer cells cultured in DMEM medium with 10% fetal bovine serum under standard cell culture conditions (37.2 °C in 5% CO2 in a humidified incubator). To acquire the ground truth from the configuration, each intensity map of the object and reference light paths needs to be captured separately under the highly stable condition of the holographic system (Refer to Section 2 of the Supplementary Information for detailed processing). The complete training process was implemented using the TensorFlow framework (Google) and was computed on a GTX Titan graphics card (NVIDIA). A fixed learning rate of 0.0001 for the experiment is adopted for the Adam optimizer62.

4.3 Simulation

Figure 3 presents the experimental results under the numerical simulation, demonstrating the quantitative analysis between DL-VHQPI and the conventional single-frame fringe demodulation techniques. Figure 3(a) shows the phase result recovered by the conventional Fourier transform (FT) method. It can be seen that the phase artifacts severely disturb imaging results due to the spectrum-overlapped problem in the SFD. Although reducing the filtering window size can attenuate the phase artifacts, this will sacrifice the SBP of the system while causing blurred imaging. More details about it can be found in Supplementary Section 3. The size of the filtering window used in the FT-based phase reconstruction results shown in Fig. 3(a) is calculated under the simulated numerical aperture (NA), as shown in the red filtering window in Fig. S2(a) of Supplementary Section 3. In VHQPI, the uVID can extract the fringes term from the hologram; however, the non-perfect background term removal still inevitably causes imaging artifacts, as shown in Fig. 3(b). In contrast, as presented in Fig. 3(c), the experimental results demonstrate DL-VHQPI’s excellent performance, in terms of artifacts suppression, over two physics-driven methods: FT and VHQPI. Figure 3(d) is the ground truth recovered by phase recovery after theoretical background removal. The magnified views of the corresponding rectangular boxes in Fig. 3(a–d), as shown in Fig. 3(j1–j4), are the phase gradient images by digital differential interference contrast (DIC) for them. To discuss the performance of methods intuitively and quantitatively, we respectively calculated the Mean Absolute Error (MAE), Structural Similarity Index (SSIM), and Peak Signal-to-Noise Ratio (PSNR) between the FT, VHQPI, and DL-VHQPI phase recovery results and the ground truth. Compared with the FT and VHQPI methods, Fig. 3(h) quantitatively demonstrates that DL-VHQPI has an excellent phase recovery accuracy and artifacts-suppression effect (More than 10 times improvement in precision.). The background-part cross-section of the four phase results depicted in Fig. 3(i) shows the phase result reconstructed by DL-VHQPI has a higher similarity to the ground truth, which also demonstrates that it can be more effective in suppressing the fringe-like error of the background part.

Fig. 3. The experiment results under the numerical simulation. (a) The FT method phase recovery result. (b) The phase recovery result of VHQPI. (c) The phase result reconstructed by DL-VHQPI. (d) The ground truth. (eg) The difference between the phase results of the three methods (i.e. FT, VHQPI, DL-VHQPI) and the ground truth. (h) Quantitative error analysis of three methods. (i) The cross-section of the phase results of FT, VHQPI, DL-VHQPI, and ground truth, and (j1j4) are the DIC views of the partially enlarged views of their corresponding phase maps respectively.

下载图片 查看所有图片

In addition, we also designed a comparison experiment with DL-noPhy (The specific network is provided in the Section 4 of Supplementary information) to demonstrate the high-efficiency and high-accuracy characteristics exhibited by the proposed method. Table 1 quantitatively shows the comparison results of the DL-VHQPI and DL-noPhy; DL-VHQPI performs a higher phase reconstruction accuracy while only utilizing one-tenth of the datasets of DL-noPhy. The reason is that DL-VHQPI adopts a physical model (uVID) to the background-component extraction process of the fringe pattern and acquires the residual components for training, which is inherently a process of image entropy reduction. According to the Shannon entropy theory, lower image entropy implies more image constraints, which provides DNN with a more powerful same-domain generalization ability. The simulated holograms with the size of 160×160 were fed to the network. During the training process, the CNN1 of DL-VHQPI over 150 epochs took 1 hour and 20 minutes, and CNN2 over 150 epochs took 1.5 hours; in contrast, DL-noPhy over 150 epochs took 7 hours and 50 minutes. Fewer training datasets for the same DNN model naturally mean shorter training time, so our method performs higher training efficiency than DL-noPhy while ensuring excellent imaging quality.

Table 1. The quantitative comparison results of DL-VHQPI and DL-noPhy.

Evaluation IndexGroup1Group2Group3Group4
DL-noPhyDL-VHQPIDL-noPhyDL-VHQPIDL-noPhyDL-VHQPIDL-noPhyDL-VHQPI
MAE0.01050.00650.01720.01420.02020.01710.01910.0170
SSIM0.99140.99690.97120.97810.96370.97180.96460.9697
PSNR84.819688.684679.727681.575778.112379.908379.467980.0348
Size of dataset3600324360032436003243600324

查看所有表

4.4 Live-cell experiment on HeLa cells

We performed holographic biological experiments on HeLa cells under a ×20/0.45NA lens to demonstrate the application of the method in biological research. The denoised interferogram presented in Fig. 4(a) is of overall low spatial carrier frequency, which results in a spectrum overlapping of cross-correlation and autocorrelation terms, as shown in Fig. 4(b). Figure 4(c) and 4(d) respectively show the phase reconstruction of captured low-carrier frequency holograms utilizing the FT and DL-VHQPI methods for HeLa cells. The field of view (FoV) of Fig. 4(c) and 4(d) is 0.093 mm2 (The Imaging Source DMK 23U274, 1600×1200, 4.4 μm), and the SBP of the complex amplitude image is 210000 pixels [the area of the FoV, multiplied by the area of the spatial frequency band,π(NA/λ)2]. To compare the imaging results of the two methods in detail, we selected two regions of interest (ROI, Area1 and Area2) on the specimens, and their magnified views are shown in Fig. 4(e1, e3, f1, f3). Additionally, Fig. 4(e2, e4, f2, f4) vividly depict the reconstructed phase gradient images by digital DIC. It can be revealed that spectrum-overlapping-caused fringe-like error dramatically degrades the phase recovery quality. The selected regions in the red rectangle box of Fig. 4(c) and 4(d) highlight the artifacts-suppression capability on the phase background. And the enlarged views after DIC processing are shown in Fig. 4(g) and 4(h), respectively. The background part of the FT-based reconstructed phase result features many coarse diagonal-fringe distributions; in comparison, that of DL-VHQPI is much smoother. The calculated Standard Deviation (STD) quantitatively demonstrates that DL-VHQPI performs a better flatness distribution. As can be readily observed in the cross-section presented in Fig. 4(i), in the FT phase recovery, the reconstruction errors brought by the autocorrelation term will introduce noticeable artifacts to the correct phase result. The results demonstrate that DL-VHQPI can excellently suppress phase artifacts and own the effectiveness and applicability for a practical slightly off-axis DH system.

Fig. 4. Results of holographic experiments on HeLa cells. (a) Low-carrier-frequency high-contrast hologram collected by slightly off-axis interferometry system. (b) Corresponding spatial frequency spectrum. (c) The result of phase recovery by slightly off-axis holography using FT method under ×20 lens. (d) The result of phase recovery using DL-VHQPI. (e1e4) and (f1f4) correspond to the local amplification results of “Area1” and “Area2” for the two samples under different phase recovery methods. Where (e2, e4, f2, f4) are the corresponding DIC views, respectively. (g) and (h) The DIC views after partial magnification of the phase map in the corresponding red box. (i) The numerical distribution of the cross-section and detail-preservation feature of the DL-VHQPI.

下载图片 查看所有图片

Indeed, reducing the size of the FT filter window may also be a good way to alleviate artifacts, but this will not fundamentally address the problem of the overlapped spectrum and will cause phase imaging blur. The reason is that reducing the filtering window is at the expense of the system’s SBP and the high-frequency information of the object cannot be enclosed in the limited filtering window. In the Section 3 of Supplementary information, we experimentally present the imaging effects under different FT filtering windows for living cells. To verify the generalization of DL-VHQPI, we supplemented a new group of experimental results for living cells in Supplementary Section 5, in which we added a comparison and discussion with the VHQPI method and the traditional FT method. The results demonstrate that DL-VHQPI still performs the best artifact-suppression ability and generalizability under a new group of biological applications.

5 Conclusions and discussions

In summary, we proposed a high-accuracy artifacts-free single-frame low-carrier frequency fringe demodulation approach for the slightly off-axis DH system, i.e., a model using the DNN-assisted physical process. When the cross-correlation and autocorrelation are inevitably aliased in the SFD, the phase reconstruction based on the conventional FT method cannot eliminate the effect of phase artifacts caused by zero-order term6. Although reducing the size of the FT filter window may alleviate the problem of imaging artifacts, the high-frequency information loss of the object caused by the limited filtering window will cause imaging blur. The method based on Kramers-Kronig relation is proposed on the basis of the concept of “cepstrum” and homomorphic filtering31, however, this method must depend on the limited condition of the object-reference ratio and need the separation of the high-order terms in the extended SFD32, 33. Furthermore, the VHQPI implements the background component removal of single-frame hologram utilizing the principle of image frequency components extraction, while it inevitably suffers from the non-sufficient background term removal caused by the physical method34. In contrast, DL-VHQPI, a novel DL-assisted physical model method, can better suppress phase artifacts while improving imaging accuracy. The simulation result quantitatively demonstrates that the phase recovery accuracy obtained by DL-VHQPI is greatly superior to that by FT and VHQPI. Moreover, the live-cell experiment results demonstrate that our method is applicable in biological research.

In addition, it is noteworthy that in the classical end-to-end DNN model (without a physical model), massive data pairs are required to train the network model for a higher reconstruction precision. However, it may be prohibitively laborious and time-consuming for the real-world DH system to collect datasets and generate the corresponding ground truth. Conversely, the proposed DL-VHQPI can perform better same-domain generalization ability and image data-feature extraction capability without a large of datasets. Compared to the classical end-to-end DNN model (i.e., DL-noPhy), DL-VHQPI can achieve a higher reconstruction accuracy utilizing only a small fraction of the datasets due to the physical model reducing the information entropy of DL training objects. Meanwhile, fewer datasets mean shorter training time and higher training efficiency.

The significance of our work lies in the multiple possibilities of applying the proposed DL-assisted physical model idea to the QPI. This idea can be applied to many scenarios in which deep learning methods are applied to the QPI, e.g., addressing a series of ill-posed inverse phase retrieval problems and holography-based high-throughput optical diffraction tomography (ODT) problems63-65. Specifically, the artifacts-free low-carrier-frequency fringe demodulation capability of the proposed method has application possibilities for ODT imaging of wide-bandwidth objects. In addition, it has also implications for high-throughput studies of high-robust common-path off-axis interferometer systems66, 67. We envision that the idea presented in this research can be applicable to a diverse range of future computational imaging techniques, not just limited to what we discussed here.

6 Acknowledgements

We are grateful for financial supports from the National Natural Science Foundation of China (61905115, 62105151, 62175109, U21B2033, 62227818), Leading Technology of Jiangsu Basic Research Plan (BK20192003), Youth Foundation of Jiangsu Province (BK20190445, BK20210338), Biomedical Competition Foundation of Jiangsu Province (BE2022847), Key National Industrial Technology Cooperation Foundation of Jiangsu Province (BZ2022039), Fundamental Research Funds for the Central Universities (30920032101), and Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense (JSGP202105, JSGP202201), National Science Center, Poland (2020/37/B/ST7/03629). The authors thank F. Sun for her contribution to this paper in terms of language expression and grammatical correction.

The authors declare no competing financial interests.

Supplementary information for this paper is available. https://doi.org/10.29026/oes.2023.220023

References

[1] FanY, LiJJ, LuLP, SunJS, HuYet alSmart computational light microscopes (SCLMs) of smart computational imaging laboratory (SCILab)PhotoniX202121910.1186/s43074-021-00040-2

[2] LeeK, KimK, JungJ, HeoJ, ChoSet alQuantitative phase imaging techniques for the study of cell pathophysiology: from principles to applicationsSensors2013134170419110.3390/s130404170

[3] ParkY, DepeursingeC, PopescuGQuantitative phase imaging in biomedicineNat Photonics20181257858910.1038/s41566-018-0253-x

[4] VicarT, BalvanJ, JarosJ, JugF, KolarRet alCell segmentation methods for label-free contrast microscopy: review and comprehensive comparisonBMC Bioinformatics20192036010.1186/s12859-019-2880-8

[5] GaoP, WirthR, LacknerJ, SunbulM, JaeschkeAet alSuperresolution imaging of live cells with genetically encoded silicon rhodamine-binding RNA aptamersBiophys J2020118145A10.1016/j.bpj.2019.11.916

[6] LiZS, FanY, SunJS, ZuoC, ChenQA commercialized digital holographic microscope with complete software supportingProc SPIE202011571115711C10.1117/12.2581220

[7] KimMKPrinciples and techniques of digital holographic microscopySPIE Rev2010101800510.1117/6.0000006

[8] KemperB, von BallyGDigital holographic microscopy for live cell applications and technical inspectionAppl Opt200847A52A6110.1364/AO.47.000A52

[9] GaoP, YuanCJResolution enhancement of digital holographic microscopy via synthetic aperture: a reviewLight Adv Manuf2022310512010.37188/lam.2022.006

[10] BettenworthD, LenzP, KrausewitzP, BrücknerM, KetelhutSet alQuantitative stain-free and continuous multimodal monitoring of wound healing in vitro with digital holographic microscopy PLoS One20149e10731710.1371/journal.pone.0107317

[11] CoppolaG, FerraroP, IodiceM, De NicolaS, FinizioAet alA digital holographic microscope for complete characterization of microelectromechanical systemsMeas Sci Technol20041552953910.1088/0957-0233/15/3/005

[12] AnandV, HanML, MaksimovicJ, NgSH, KatkusTet alSingle-shot mid-infrared incoherent holography using Lucy-Richardson-Rosen algorithmOpto-Electron Sci2022121000610.29026/oes.2022.210006

[13] XuK, Wang XE, Fan XH, et alMeta-holography: from concept to realizationOpto-Electron Eng20224922018310.12086/oee.2022.220183

[14] GaoH, FanXH, XiongW, HongMHRecent advances in optical dynamic meta-holographyOpto-Electron Adv2021421003010.29026/oea.2021.210030

[15] GabaiH, Baranes-ZeeviM, ZilbermanM, ShakedNTContinuous wide-field characterization of drug release from skin substitute using off-axis interferometryOpt Lett2013383017302010.1364/OL.38.003017

[16] HuangZZ, MemmoloP, FerraroP, CaoLCDual-plane coupled phase retrieval for non-prior holographic imagingPhotoniX20223310.1186/s43074-021-00046-w

[17] WuXJ, SunJS, ZhangJL, LuLP, ChenRet alWavelength-scanning lensfree on-chip microscopy for wide-field pixel-super-resolved quantitative phase imagingOpt Lett2021462023202610.1364/OL.421869

[18] WangHD, GöröcsZ, LuoW, ZhangYB, RivensonYet alComputational out-of-focus imaging increases the space–bandwidth product in lens-based coherent microscopyOptica201631422142910.1364/OPTICA.3.001422

[19] MicóV, GarcíaJ, ZalevskyZ, JavidiBPhase-shifting Gabor holographyOpt Lett2009341492149410.1364/OL.34.001492

[20] PoonTCDigitalHolographyandThree-DimensionalDisplay: PrinciplesandApplications (Springer, New York, 2006).

[21] ClausD, IliescuD, Bryanston-CrossPQuantitative space-bandwidth product analysis in digital holographyAppl Opt201150H116H12710.1364/AO.50.00H116

[22] ZhongZ, BaiHY, ShanMG, ZhangYB, GuoLLFast phase retrieval in slightly off-axis digital holographyOpt Lasers Eng20179791810.1016/j.optlaseng.2017.05.004

[23] XueL, LaiJC, WangSY, LiZHSingle-shot slightly-off-axis interferometry based Hilbert phase microscopy of red blood cellsBiomed Opt Express2011298799510.1364/BOE.2.000987

[24] ShakedNT, ZhuYZ, RinehartMT, WaxATwo-step-only phase-shifting interferometry with optimized detector bandwidth for microscopy of live cellsOpt Express200917155851559110.1364/OE.17.015585

[25] PavillonN, ArfireC, BergoëndI, DepeursingeCIterative method for zero-order suppression in off-axis digital holographyOpt Express201018153181533110.1364/OE.18.015318

[26] TrusiakM, Picazo-BuenoJA, PatorskiK, ZdankowskiP, MicoVSingle-shot two-frame π-shifted spatially multiplexed interference phase microscopy J Biomed Opt20192409600410.1117/1.JBO.24.9.096004

[27] León-RodríguezM, RayasJA, CorderoRR, Martínez-GarcíaA, Martínez-GonzalezAet alDual-plane slightly off-axis digital holography based on a single cube beam splitterAppl Opt2018572727273510.1364/AO.57.002727

[28] HanJH, GaoP, YaoBL, GuYZ, HuangMJSlightly off-axis interferometry for microscopy with second wavelength assistanceAppl Opt2011502793279810.1364/AO.50.002793

[29] IkedaT, PopescuG, DasariRR, FeldMSHilbert phase microscopy for investigating fast dynamics in transparent systemsOpt Lett2005301165116710.1364/OL.30.001165

[30] GuoCS, WangBY, ShaB, LuYJ, XuMYPhase derivative method for reconstruction of slightly off-axis digital hologramsOpt Express201422305533055810.1364/OE.22.030553

[31] PavillonN, SeelamantulaCS, KühnJ, UnserM, DepeursingeCSuppression of the zero-order term in off-axis digital holography through nonlinear filteringAppl Opt200948H186H19510.1364/AO.48.00H186

[32] BaekY, LeeK, ShinS, ParkYKramers–Kronig holographic imaging for high-space-bandwidth productOptica20196455110.1364/OPTICA.6.000045

[33] BaekY, ParkYIntensity-based holographic imaging via space-domain Kramers–Kronig relationsNat Photonics20211535436010.1038/s41566-021-00760-8

[34] TrusiakM, CywińskaM, MicóV, Picazo-BuenoJÁ, ZuoCet alVariational Hilbert quantitative phase imagingSci Rep2020101395510.1038/s41598-020-69717-1

[35] CywińskaM, TrusiakM, PatorskiKAutomatized fringe pattern preprocessing using unsupervised variational image decompositionOpt Express201927225422256210.1364/OE.27.022542

[36] LarkinKG, BoneDJ, OldfieldMANatural demodulation of two-dimensional fringe patterns. I. General background of the spiral phase quadrature transformJ Opt Soc Am A2001181862187010.1364/JOSAA.18.001862

[37] ZuoC, QianJM, FengSJ, YinW, LiYXet alDeep learning in optical metrology: a reviewLight Sci Appl2022113910.1038/s41377-022-00714-x

[38] FengSJ, ChenQ, GuGH, TaoTY, ZhangLet alFringe pattern analysis using deep learningAdv Photonics2019102500110.1117/1.AP.1.2.025001

[39] FengSJ, ZuoC, HuY, LiYX, ChenQDeep-learning-based fringe-pattern analysis with uncertainty estimationOptica202181507151010.1364/OPTICA.434311

[40] CywińskaM, BrzeskiF, KrajnikW, PatorskiK, ZuoCet alDeepDensity: convolutional neural network based estimation of local fringe pattern densityOpt Lasers Eng202114510667510.1016/j.optlaseng.2021.106675

[41] PanBOptical metrology embraces deep learning: keeping an open mindLight Sci Appl20221113910.1038/s41377-022-00829-1

[42] ZuoC, QianJM, FengSJ, YinW, LiYXet alCorrection: deep learning in optical metrology: a reviewLight Sci Appl2022117410.1038/s41377-022-00757-0

[43] FengSJ, ZuoC, ZhangL, YinW, ChenQGeneralized framework for non-sinusoidal fringe analysis using deep learningPhotonics Res202191084109810.1364/PRJ.420944

[44] LiYX, QianJM, FengSJ, ChenQ, ZuoCDeep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurementOpto-Electron Adv2022521002110.29026/oea.2022.210021

[45] ZhengCH, WangTS, LiuZQ, et alDeep transfer learning method to identify orbital angular momentum beamsOpto-Electron Eng20224921040910.12086/oee.2022.210409

[46] ZhengZH, ZhuSK, ChenY, ChenHY, ChenJHTowards integrated mode-division demultiplexing spectrometer by deep learningOpto-Electron Sci2022122001210.29026/oes.2022.220012

[47] RivensonY, ZhangYB, GünaydınH, TengD, OzcanAPhase recovery and holographic image reconstruction using deep learning in neural networksLight Sci Appl201871714110.1038/lsa.2017.141

[48] RivensonY, WuYC, OzcanADeep learning in holography and coherent imagingLight Sci Appl201988510.1038/s41377-019-0196-0

[49] ChenHL, HuangLZ, LiuTR, OzcanAFourier Imager Network (FIN): a deep neural network for hologram reconstruction with superior external generalizationLight Sci Appl20221125410.1038/s41377-022-00949-8

[50] LempitskyV, VedaldiA, UlyanovD.Deep image prior. In Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 9446–9454 (IEEE, 2018);http://doi.org/10.1109/CVPR.2018.00984.

[51] WangF, BianYM, WangHC, LyuM, PedriniGet alPhase imaging with an untrained neural networkLight Sci Appl202097710.1038/s41377-020-0302-3

[52] DuranJ, CollB, SbertCChambolle’s projection algorithm for total variation denoisingImage Process Line2013331133110.5201/ipol.2013.61

[53] ZhuXJ, ChenZQ, TangCVariational image decomposition for automatic background and noise removal of fringe patternsOpt Lett20133827527710.1364/OL.38.000275

[54] BiancoV, MemmoloP, PaturzoM, FinizioA, JavidiBet alQuasi noise-free digital holographyLight Sci Appl20165e1614210.1038/lsa.2016.142

[55] KluverJWElimination of slip and instability effects in certain M-type electron beams Proc IEEE19635186886810.1109/proc.1963.2309

[56] YangX, YuQF, FuSHA combined method for obtaining fringe orientations of ESPIOpt Commun2007273606610.1016/j.optcom.2006.12.026

[57] DengM, LiS, ZhangZY, KangI, FangNXet alOn the interplay between physical and content priors in deep learning for computational imagingOpt Express202028241522417010.1364/OE.395204

[58] ShannonCEA mathematical theory of communicationACM SIGMOBILE Mob Comput Commun Rev2001535510.1145/584091.584093

[59] CoverTMElementsofInformationTheory. John Wiley & Sons, 1999).

[60] IoffeS, SzegedyC.Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning (JMLR.org, 2015).

[61] NairV, HintonGE.Rectified linear units improve restricted Boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning 807–814 (Omnipress, 2010).

[62] KingmaDP, BaJ.Adam: a method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations. https://arxiv.org/abs/1412.6980 (2015).

[63] ChoiW, Fang-YenC, OhS, LueN, DasariRRet alTomographic phase microscopy: quantitative 3D-mapping of refractive index in live cellsImaging Microsc2008104850

[64] SungY, ChoiW, Fang-YenC, BadizadeganK, DasariRRet alOptical diffraction tomography for high resolution live cell imagingOpt Express20091726627710.1364/OE.17.000266

[65] LiJJ, MatlockAC, LiYZ, ChenQ, ZuoCet alHigh-speed in vitro intensity diffraction tomography Adv Photonics2019106600410.1117/1.AP.1.6.066004

[66] MicoV, ZalevskyZ, GarcíaJSuperresolution optical system by common-path interferometryOpt Express2006145168517710.1364/OE.14.005168

[67] ZhangJW, DaiSQ, MaCJ, XiTL, DiJLet alA review of common-path off-axis digital holography: towards high stable optical instrument manufacturingLight Adv Manuf2021233334910.37188/lam.2021.023

Zhuoshi Li, Jiasong Sun, Yao Fan, Yanbo Jin, Qian Shen, Maciej Trusiak, Maria Cywińska, Peng Gao, Qian Chen, Chao Zuo. Deep learning assisted variational Hilbert quantitative phase imaging[J]. Opto-Electronic Science, 2023, 2(4): 220023.

引用该论文: TXT   |   EndNote

相关论文

加载中...

关于本站 Cookie 的使用提示

中国光学期刊网使用基于 cookie 的技术来更好地为您提供各项服务,点击此处了解我们的隐私策略。 如您需继续使用本网站,请您授权我们使用本地 cookie 来保存部分信息。
全站搜索
您最值得信赖的光电行业旗舰网络服务平台!