1 Introduction

In the past, holography, and three-dimensional imaging (3D) techniques were mostly powered by coherent light sources since spatially incoherent illumination demanded cumbersome optical architectures such as rotational shearing interferometers, triangle interferometers and conoscopic holography¹. The need for such optical architectures is derived from the requirement that two-beam interference (TBI) was needed to compress 3D information in a two-dimensional (2D) hologram. The birth of active optical devices such as spatial light modulators and computer processing methods, eased the constraints of optical architectures of incoherent holography with TBI. One milestone in this research direction was the development of Fresnel incoherent correlation holography (FINCH), which is one of the widely used incoherent holography architectures today^2-4. In 1968, deconvolution based 2D coded aperture imaging (CAI) using randomly arranged pinholes was demonstrated by Dicke and Ables to overcome the low-intensity problem when using a single pinhole for X-ray and gamma-ray imaging^{5, 6}. However, the study was not explored beyond 2D imaging. In 2008, a 3D imaging technique based on CAI was demonstrated with wavelength as the third dimension⁷. Later, the CAI method was extended to 3D imaging and four-dimensional (4D) imaging with depth as the third dimension^8-10, and depth and wavelength as the third and fourth dimensions, respectively^{11, 12}. Therefore, CAI offers multidimensional imaging capabilities without TBI and with a minimum number of optical components, which makes them more attractive than FINCH-based techniques.

This research project began when we attempted the CAI method employing the benchtop Fourier transform infrared microspectroscopy (FTIRm) system at the Australian Synchrotron operating with a conventional Globar^TM IR source. This instrument, which comprises a Bruker Hyperion 3000 IR microscope and V70 FTIR spectrometer, is used in support of many applications at the Infrared Microspectroscopy (IRM) beamline, whereby the identification and analysis of functional groups within a sample enable a better understanding of diverse materials from diseased tissue to composite materials and foods to paintings^13-15. We applied the CAI method to the benchtop FTIR instrument at the IRM beamline by manufacturing scattering lenses on calcium fluoride (CaF₂) substrates. A matching 15× (NA = 0.4) Cassegrain objective lens (COL) and a 64×64 focal plane array (FPA) imaging detector (pixel size = 40 μm) were used. During this study, we noticed that COL exhibited interesting focal characteristics with sharp autocorrelation functions and low cross-correlation with respect to other planes (see Supplementary information Section 1, Section 2).

Let us ask this fundamental question – why is scattering based coded apertures preferred for 3D imaging? The reason is that the average speckle size formed by scattering is approximately the diffraction limited spot size¹⁶. Consequently, the autocorrelation function is sharp and about twice that of the size of the focal spot that can be obtained with a lens of the same NA¹. Secondly, the speckle patterns change with depth resulting in the capability to discriminate different planes along depth by cross-correlation. Therefore, for 3D imaging, a sharp autocorrelation and low cross-correlation along depth (SALCAD) is needed. The fields generated using scatterers can be classified as random SALCAD fields and the one from COL as a deterministic SALCAD field. Deterministic SALCAD fields require a lower photon budget than the random counterparts and the point spread functions (PSFs) can be calculated, unlike random SALCAD fields where PSFs are recorded⁹. So, this imaging concept can be considered as correlation holography when the SALCAD condition is satisfied and the object is illuminated by a spatially incoherent source. Recently, a modified approach was employed where sparse randomly arranged focal spots were generated instead of scattering to improve the signal to noise ratio¹⁷. However, this method is suitable only for imaging a single plane, and to image multiple planes, many such phase masks are needed to be spatially multiplexed. Fresnel zone aperture based 2D imaging systems have been reported in the past^18-20.

2 Results and discussion

The FTIRm consists of collinear dual beams generated from two sources: (i) a standard white light source for bright-field visible light observation of the sample plane and (ii) a broadband Globar^TM IR source that allows the acquisition of mid-IR (MIR) spectral images in the range ~ 899 cm^–1–3845 cm^–1 as shown in Fig. 1(a). The FTIR single-beam spectrum obtained from the Globar^TM source is shown in Fig. 1(b) in which a total of 765 spectral channels equally separated by an average value of 3.85 cm^–1 were used. The MIR beam from the FTIR spectrometer is focused on the sample plane and the scattered signal is recorded by the FPA detector. With 15 × COL, the pixel pitch is ~2.7 μm in the sample plane and the field of view is around ~172 × 172 μm².

Fig. 1. Simplified schematic of the FTIR microspectroscopic system and the measured spatial and spectral characteristics. (a) Abbreviations: COL – Cassegrain objective lens, IR – Infrared, BS – beam splitter, M – mirror, L – lens, FTIR – Fourier transform infrared spectrometer, FPA – focal plane array, MSP – Motorized sliding plate, MCT - mercury-cadmium-telluride, A – aperture, MIR – mid infrared, PSF – point spread function. The MIR from the FTIR spectrometer is sent into the IR/VISIBLE microscope. The microscope is aligned collinearly for both MIR and reference visible light. There are two modes of operation: (reflection and transmission) and two modes of data recording (mapping using single element MCT detector and imaging using multi-element FPA detector). (b) Spectral profile of the Globar^TM source with 14 spectral channels marked in blue are selected for the study. (c–e) 4D plot of the PSFs recorded for an axial aberration Δz = 0, 125 μm and 250 μm, respectively. Spectral cube data PSF(x, y, ν) was formed from the 14 spectral channels each averaged over 100 spectral channels –50 to +50 channels. The non-changing pattern along ν indicates spectral aberrations suppressed by the COL. (f–h). Autocorrelation cube data of (c–e), respectively. The thin cylindrical structures obtained for Δz = 125 μm and 250 μm indicates the possibility of imaging in deconvolution mode with a similar resolving power as Δz = 0 μm. (i) Depth cube data PSF(x, y, Δz) from 0 to 250 μm. Unlike a regular lens where the intensity distribution is filled, the COL generates hollow intensity distribution with four distinct lobes causing a sharp autocorrelation function. (j) The spectral and spatial cube data were combined to form 8415 spatio-spectral images corresponding to the 14 spectral channels and aberrations 0 to 250 μm in steps of 25 μm and was cross-correlated with the image corresponding to the central spatio-spectral point (2307 cm^–1, 125 μm) and cross-correlation value at the origin (x=0, y=0) is plotted. The FWHM along z was ~50 μm indicating that the intensity of a plane 50 μm apart has a loss of about 50% of intensity which is ideal for 3D MIR imaging applications.

下载图片 查看所有图片

The IR light produced by the Globar^TM source is spatially incoherent and so the light emitted from one object point does not interfere with the light emitted from another point, instead their intensities add up. As a result, this FTIRm system is a shift-invariant imager, linear in intensity, i.e., PSF and the intensity distribution obtained for an object O can be expressed as OH=O⊗PSF+σ, where σ is the noise and ‘ ⊗’ is a 2D convolutional operator (see Supplementary information Section 1, Section 2). Therefore, if the PSF is known for all axial shifts Δz, then the 3D image of O can be reconstructed by solving the above equation. For instance, object O=O(z1,λa)+O(z2,λa)+OT, which absorbs at λa at z₁ and z₂ while OT refers to non-absorbing layers. The reconstruction of the chemical image of O at plane z1 is given as OR(λa)=OH*PSF(z1)=O(z1,λa)⊗PSF(z1)*PSF(z1)+O(z2,λa)⊗PSF(z2)*PSF(z1)+N, where ‘ *’ is a 2D correlational operator and N is the reconstruction noise. The reconstructed image is O(z1,λa) sampled by the autocorrelation function PSF(z1)*PSF(z1), which is a Delta-like function with a minimum width ~1.22λ/NA added to O(z2,λa) sampled by PSF(z1)*PSF(z2), which is a blurred version of O(z2,λa).

The PSFs were recorded using a 15 μm pinhole for Δz = 0 to 250 μm in steps of 25 μm. The lateral resolution of the system is therefore ~ 15 μm. The spectral data cubes PSF(x,y,ν) for Δz = 0, 125 μm and 250 μm along 14 averaged (~100 channels) spectral channels are shown in Fig. 1(c–e), respectively, whose non-changing images along ν indicate the suppression of spectral aberrations. The data structure of the output generated by the OPUS 8.0 software was transformed into cube data for processing in MATLAB. The COL generates sharp annular intensity distributions with four peaks, which are responsible for the sharp autocorrelation functions as shown in Fig. 1(f–h). Only a slight increase in the width of the autocorrelation functions was noticed for larger values of Δz. The depth cube data PSF(x,y,Δz) was generated from recorded PSFs for Δz = 0 to 250 μm and averaged over all the 765 spectral channels as shown in Fig. 1(i). The spatial and spectral cubes were combined, cross-correlated with the central spatio-spectral point (2307 cm^–1, 125 μm) as shown in Fig. 1(j). It is seen that the FWHM along the depth axis is sharp at ~50 μm with mild fluctuations along the spectral axis. The above spatio-spectral characteristics of the FTIRm system are highly desirable for 3D chemical imaging.

The next important aspect of imaging is the reconstruction technique. It is well-established from previous studies that the PSF is not the optimal reconstruction function by cross-correlation²¹, and a non-linear reconstruction (NLR) method was introduced by Rosen in which the magnitudes of the spatial frequency spectrum of the PSF and OH are tuned in powers of α and β respectively in R=|F−1exp[−iarg(OH˜)]}| until minimum entropy was obtained (see Supplementary information Section 3)²². Another approach is the Lucy-Richardson algorithm (LRA), which consists of a forward convolution between the initial guess solution and the PSF, which is compared with the recorded OH and the ratio is backward cross-correlated with the PSF and the result is multiplied to the initial guess. This process is iterated until the maximum likelihood object function is reconstructed^{23, 24}. The (n+1)^th reconstructed image is given as Rn+1=Rn{OHRn⊗PSFPSF'}, where PSF' is the 180 degrees rotated version and the complex conjugate of PSF. In this study, it was noticed that neither NLR nor LRA were optimal; however, when the linearity was broken of the backward cross-correlation of LRA by the NLR approach, a rapid convergence and significantly better estimation was obtained. The schematic of the Lucy-Richardson-Rosen algorithm (LRRA) is shown in Fig. 2(a) (see Supplementary information Section 3).

Fig. 2. Reconstruction results using Lucy-Richardson algorithm, Non-linear reconstruction and Lucy-Richardson-Rosen algorithm. (a) Lucy-Richardson-Rosen algorithm, O – object, OH – object hologram, PSF – point spread function, OTF – optical transfer function, × - element by element product, ⊗ - 2D convolution, ℑ - Fourier transform, ℑ* - complex conjugate operation following a Fourier transform, ℑ−1- inverse Fourier transform, ~ Fourier transform of a variable, Rⁿ and R⁽ⁿ⁺¹⁾ are the n^th and (n+1)^th solutions and OH was used as the initial guess solution R¹, ML(O) – maximum likelihood solution of the object. (b) Direct imaging of the cross object using a visible light channel of the microscope. Average image of all the 765 spectral channels of the cross object at (c) Δz = 0. (d) Δz = 100 μm, (e) The PSF recorded for Δz = 100 μm. Reconstruction results of the cross object using (f) LRA (150 iterations), (g) NLR (α = 0, β = 0.6) and (h) LRRA (20 iterations, α = 0.2, β = 1). (i) Direct MIR imaging result of four pinholes Δz = 0. (j) Intensity pattern recorded for four pinholes for Δz = 150 μm and (k). its reconstruction result. (l) Intensity pattern recorded for four pinholes for Δz = 200 μm and (m). its reconstruction result. (n) Bar plots of RMSE and entropy for LRA, NLR and LRRA. The performance of LRRA is found to be significantly higher than LRA and NLR with RMSE ratio of 13 and 5 times, respectively. The LRRA has a rapid convergence with 7 times faster than LRA for the cross object. It must be noted that the above values are for only the cross object. The performances vary with the complexity of the object with the minimum difference for simple objects and significant difference for complicated objects. The scale bar is 50 μm.

下载图片 查看所有图片

The spatio-spectral aberrations of the system have been studied²⁵. A cross shaped object (150 μm × 150 μm) and four random pinholes (50 μm in diameter) ablated on chromium-gold layer coated calcium fluoride substrate were used as test objects. The visible light image and IR image when Δz = 0 are shown in Fig. 2(b) and 2(c) respectively. The IR image when Δz = 100 μm is shown in Fig. 2(d). The PSF recorded for Δz = 100 μm is shown in Fig. 2(e). The reconstruction results with LRA (150 iterations) (Supplementary Video 1), NLR (α = 0, β = 0.6) and LRRA (20 iterations, α = 0.2, β = 1) (Supplementary Video 2) are shown in Fig. 2(f–h), respectively. The recorded IR images for Δz = 0, 150 μm and 200 μm are shown in Fig. 2(i), 2(j)and 2(l), respectively, and the reconstructions using LRRA for Δz = 150 μm and 200 μm are shown in Fig. 2(k) and 2(m), respectively. The RMSE and entropy for LRA, NLR and LRRA are compared for the cross object and LRRA was found to perform significantly better than both LRA and NLR as shown in Fig. 2(n).

The experiment was then repeated on a bundle of silk fibers. The absorption spectrum of silk is shown in Fig. 3(a) and the chemical image (Δz = 0) is shown in Fig. 3(b). The reconstruction results of Fig. 3(b) using LRRA with PSFs recorded at Δz = 25 μm, 50 μm and 75 μm are shown in Fig. 3(c–e), respectively. The image directly recorded at Δz = 50 μm is shown in Fig. 3(f). Comparing Fig. 3(d) and 3(f), a good overlap can be observed. A single strand of silk fiber was imaged as shown in Fig. 3(g) whose varying blur indicates varying depth. It was reconstructed using a PSF recorded at Δz = 50 μm as shown in Fig. 3(h) and the corresponding direct image is shown in Fig. 3(i). A polymer test pattern, prepared on a CaF₂ substrate, using a chrome-on-glass United States Air Force (USAF) 1954 target as a lithography mask was examined next, whose absorption spectrum is shown in Fig. 3(j). The direct imaging and reconstruction of object ‘3’ from the USAF target using LRRA are shown in Fig. 3(k) and 3(l), respectively. In some cases, it was necessary to adjust the magnification of the PSFs before reconstruction to achieve optimal reconstruction and for some cases, synthetic PSFs generated from recorded PSFs were used for reconstruction (see supplementary information Section 4).

Fig. 3. Reconstruction results of thick and strong IR absorbing materials. (a) Normalised absorbance of the silk fiber sample. The characteristic absorption peaks are noted at 3297 cm^–1 and 1600 cm^–1. The spectral matrices for Δz = 0 from 1708 cm^–1 to 1558 cm^–1 were averaged to obtain Fig. 3(b). (b) Direct imaging of a bundle of silk threads in 3D space, showing focused and out-of-focus objects. Reconstruction result using LRRA (α = 0.5, β = 1, iterations = 18) using PSFs recorded at (c) Δz = 25 μm, (d) Δz = 50 μm and (e), Δz =75 μm. (f) Direct imaging of a bundle of silk fibers for Δz ≈ 50 μm. The magnified regions of the direct imaging and reconstruction results indicate that the LRRA could obtain a good reconstruction even for thick objects and strong IR absorbing objects. (g) Direct imaging of a single silk fiber in 3D space. (h) Reconstruction results using PSF of Δz = 50 μm. (i) Direct imaging of the single silk fiber for Δz ≈ 50 μm. (j) Normalised absorbance of the polymer material of USAF target with characteristic peaks at 1500 cm^–1 and 1238 cm^–1. (k) Direct imaging of the USAF object ‘3’ located at Δz ≈ 50 μm. (l) Reconstruction result of the USAF object ‘3’ using LRRA (α = 0.6, β = 1, iterations = 14). The scale bar is 50 μm. The lateral resolution of the imaging system is ~15 μm.

下载图片 查看所有图片

3 Conclusion

In conclusion, we demonstrated 3D imaging without TBI using deterministic fields. Unlike scattering based SALCAD fields, where the PSFs have to be recorded at all axial planes, with deterministic SALCAD fields it is possible to calculate the PSFs from the complex amplitude of the optical modulator. Consequently, the reconstruction in this case mimics that of conventional incoherent holography. Furthermore, we have invented a new reconstruction method by introducing non-linearity into the LRA using the NLR approach, which performs significantly better than both LRA and NLR. The method and reconstruction have been demonstrated in the FTIRm chemical imaging system by introducing necessary data structure conversions (see Supplementary information Section 5). We believe that the approach can be easily adapted to power sensitive areas such as astronomical, fluorescence and biomedical imaging. Above all, we also believe that this research work will benefit the users of synchrotron based FTIRm technique to be able to investigate multiple samples during the limited beamtime in the future which was the motivation that led to this research work. The proposed method opens a new direction where deterministic fields and structured light can be engineered for rapid 3D imaging with a low photon budget, high lateral, and axial resolutions.

The evolution of incoherent holography over the years from the complicated architectures such as rotational shearing interferometer²⁶, conoscopic holography²⁷, and FINCH^{2, 3}, to this version of holography with deterministic SALCAD fields using Lucy-Richardson-Rosen algorithm is interesting. We believe that the proposed method will compete with the existing coherent holography methods such as MIR digital holography and holographic interferometry using quantum cascade laser²⁸, single shot Raman holography²⁹, and label-free second harmonic phase imagers³⁰.

This research was undertaken on the IRM beamline at the Australian Synchrotron (Victoria, Australia), part of ANSTO (Proposal ID. 15775, Reference No. AS1/IRM/15775 and Proposal ID. M17333, Reference No. AS2/IRM/17333). This work was performed in part at the Swinburne's Nanofabrication Facility (Nanolab). Funded by European Union’s Horizon 2020 research and innovation programme under grant agreement No. 857627 (CIPHR).

The authors declare no competing financial interests.

Supplementary information for this paper is available at https://doi.org/10.29026/oes.2022.210006