1. INTRODUCTION

Over the last decade, image sensors have undergone dramatic miniaturization, thanks to advances in optical packaging and semiconductor-based photodetector technology. Even further size reduction, however, is required for emerging areas of machine vision, autonomous transportation, and augmented reality visors [13" target="_self" style="display: inline;">–3]. Such miniaturization using traditional refractive optics is difficult, as the optical elements themselves occupy a significant volume. An attractive solution to reduce the overall volume of these imaging systems is to use diffractive optical elements such as kinoforms. These diffractive lenses, however, suffer from severe chromatic aberrations and higher-order diffraction. While efforts have been made to mitigate these aberrations by utilizing the combination of an extended depth of focus (EDOF) and post-capture image reconstruction [4], the higher-order diffraction cannot be avoided over a broad wavelength range.

To eliminate higher-order diffraction, subwavelength diffractive optics, also known as metasurfaces, can be used [5]. Metasurfaces are two-dimensional optical elements consisting of quasi-periodic subwavelength resonators that are capable of abruptly introducing phase shifts onto a wavefront, enabling ultrathin optics and lenses [5–23" target="_self" style="display: inline;">–23]. Unfortunately, these metasurfaces also exhibit severe chromatic aberrations. For a metalens, the focal length is inversely proportional to the wavelength, originating primarily from the fixed positions of phase-wrapping discontinuities as the wavelength changes [24]. This shift in focal length with a change in wavelength causes chromatic aberrations that blur the image. Recent works have attempted to mitigate this chromatic aberration through dispersion engineering [25–30" target="_self" style="display: inline;">–30], which employs scatterers that compensate for the chromatic phase dispersion. The phase delay at each scatterer has a wavelength dependence that is effectively corrected using higher-order terms in the Taylor expansion of the phase with respect to the wavelength. Dispersion-engineered metasurfaces, however, are limited to a small aperture for a fixed numerical aperture (NA) [25]; a larger aperture would require a higher maximum phase dispersion, which requires the optical resonators to have higher quality factors. This would imply that the scatterers would need ever higher aspect ratios for increased aperture size, exceeding current high-throughput nanofabrication [31,32] capabilities. A different approach is to design a metalens capable of focusing discrete colors of red, green, and blue, using a composite of three layers of Fresnel binary zone plates on frequency-selective plasmonic metasurfaces [33]. Though this achieves focusing at visible frequencies, the diffraction efficiency takes a toll due to the partially opaque design of the binary Fresnel zone plates for each of the designated colors. The imaging capability of this method is also limited to discrete wavelengths and is not broadband in nature.

Another technique for mitigating chromatic aberrations is to employ freeform metasurfaces and computational imaging [34–38" target="_self" style="display: inline;">–38], a paradigm that does not have the same scaling challenges although this process entails additional energy consumption and latency due to the need of computational reconstruction. Computational imaging is a technique that employs software to generate images from measurements. With the availability of fast and efficient computing, this can be executed swiftly and with minimal energy consumption. Computational imaging can reduce the complexity of the optical system by offloading the aberration correction to the post-processing stage [39,40]. One specific example is that of an EDOF lens, where the depth of focus of a refractive lens is extended by an auxiliary phase mask with the help of computational imaging [41,42], and the chromatic aberrations can be corrected with software post-processing [4,43,44]. Thus, computational imaging in conjunction with freeform metasurfaces is a promising avenue for mitigating metalens aberrations.

Recently, full-color imaging in the visible wavelength regime was demonstrated using an EDOF metasurface and post-processing deconvolution [35]. Here, a rectangularly separable cubic phase mask (CPM) [45] was added to the standard hyperboloidal metalens phase, generating a non-rotationally symmetric extended focal spot. The longitudinally extended nature of the focal spots at different wavelengths is sufficient to compensate for the chromatic shift in the focal length. The EDOF property of the CPM enables the imaging system to capture useful spatial frequency information of the colored image so that computational reconstruction is possible [35]. The CPM is limited, however, in that it produces a transversely asymmetric point spread function (PSF) that makes imaging sensitive to the orientation of the element, often manifesting as asymmetric artifacts even after deconvolution. Additionally, the CPM produces a lateral shift of the PSF with a change in wavelength (due to creation of an accelerating Airy beam), which can contribute to distortions in imaging. One potential solution to this limitation is to utilize a rotationally symmetric PSF. Although there are previous works on symmetric, EDOF refractive optics using log-aspheres [46] and axicon-based lenses [47], they have not been used for correcting the strong chromatic aberrations encountered in a metasurface. In this paper, we extend the family of EDOF metasurfaces beyond a simple CPM. We design and fabricate four different types of EDOF metasurface lenses operating in the visible regime, including both rotationally symmetric and asymmetric phase profiles. We characterize the modulation transfer function (MTF) for all these lenses and demonstrate full-color imaging. A comparative analysis of all these lenses is also provided, evaluating in terms of optical bandwidth and image quality. All our EDOF metasurfaces demonstrate at least an order of magnitude larger optical bandwidth compared to a standard metalens. Full-color imaging in the visible range is achieved using all the EDOF lenses, outperforming the traditional metalens in terms of chromatic aberrations.

2. METHODS

An imaging system behaves as a linear system that maps the incoming light from a scene to the sensor. This mapping function is modeled by an object’s convolution with the element’s PSF. For imaging under incoherent light, the system provides a linear mapping from the input intensity to the output intensity, which is captured on the sensor array. By scanning a point source throughout the object volume and measuring the resultant intensity across the image volume, a 3D intensity impulse response, or PSF, can be measured. This method can fully characterize the imaging function of the system. Assuming the PSF is shift-invariant, we can treat the PSF as a kernel that convolves with the input to produce an image on the sensor plane. The PSF of an ideal lens is an Airy disk that, when the aperture is large, resembles a point, which enables capturing an almost exact replica of the scene. When the PSF deviates from a point, the captured image becomes blurry; however, with a PSF known a priori, an in-focus image can be retrieved via post-capture deconvolution if sufficient spatial frequency information at a high enough signal-to-noise ratio (SNR) is collected by the sensor. This information retrievability through computational imaging can be expressed in terms of the MTF of the optical element, which is given by the magnitude of the Fourier transform of the PSF. A broad MTF, one that does not drop to zero rapidly, signifies that a wide range of spatial frequencies from the object plane is captured at the sensor plane, corresponding to a PSF with a small spot size. On the other hand, a narrow MTF, one that decays rapidly, captures only a limited range of spatial frequency content, which precludes the possibility of computational reconstruction due to the zeros in the spatial frequency spectrum. A conventional, in-focus metalens exhibits a broad MTF when imaging with narrowband light, resulting in high-quality images. When imaging with a different wavelength at the same sensor plane, however, the spatial bandwidth of the MTF drastically decreases and the collected spatial frequencies from the scene are attenuated or eliminated. As some of the spatial frequencies are not collected, this results in an uncorrectable blur. With EDOF lenses, we can realize a similar PSF at the sensor plane for a broad and continuous range of wavelengths [48]. Moreover, the resulting MTF can capture a wider range of spatial frequencies, which is key to generating full-color images at high resolution.

Here, we designed four different EDOF lenses, namely cubic [45], shifted axicon, log-asphere [46], and SQUBIC [49] lenses. All, except for the cubic, are axially symmetric. We fix the aperture for each of these metasurfaces at 200 μm and select a nominal focal length of 200 μm, making the NA close to 0.45 for all designs. We emphasize that the small aperture of the metalens is due to the prohibitive cost and time for fabricating large-aperture lenses based on electron-beam lithography, and not due to the same scaling limitations as encountered with dispersion-engineered metalenses. At the time of writing, the highest demonstrated NA with a dispersion-engineered metalens is 0.35, the diameter of which was only 30 μm [50,51]. The cubic metalens utilizes a focusing phase mask combined with a cubic term to produce an MTF insensitive to wavelength [35], and the phase mask is where λ denotes the operating wavelength, x and y are the coordinates in plane, f is the nominal focal length, α represents the strength of the cubic term, and R represents the radius of the phase mask. We chose λ=550nm, f=200μm, R=100μm, and α=55π for the design of the cubic metasurface. To compare the performance of the EDOF metasurfaces against the standard singlet metalens, we include a design with α=0 that imparts no cubic term to the wavefront. The log-asphere phase mask was inspired by a prior work [46] that divides the phase mask into annular zones with continuously varying focal lengths. The central annular zone has a focal length of s1 and the outermost annular zone has a focal length of s2. This design effectively extends the focal length from s1 to s2. The log-asphere phase mask is governed by the relation where r=x2+y2 and R is the aperture radius of the phase mask. The parameter n changes the intensity distribution over the optical axis. For the log-asphere phase mask, we set n=2, making the intensity distribution uniform across the line of foci. The parameters we chose for the log-asphere metasurface design are λ=550nm, s1=80μm, s2=300μm, and R=100μm. Similar to the log-asphere phase mask, the shifted axicon phase mask takes the same form as Eq. (2). In this case, we set n=1, λ=550nm, s1=80μm, and s2=300μm. The axial intensity distribution of the shifted axicon resembles that of a diffractive axicon lens [52], hence the nomenclature. For a fixed NA, both log-asphere and shifted axicon metasurfaces can be scaled to arbitrary aperture sizes without increasing the maximum phase gradient. To ensure each phase zone has at minimum two phase steps, we need to enforce a maximum gradient of π/Λ, where Λ is the periodicity of the unit cell. The maximum phase gradient for both the log-asphere and shifted axicon is where z=S2/R, which is a constant given a fixed NA. By guaranteeing multiple phase steps per 2π zone, we ensure a lower bound on the theoretical diffraction efficiency per zone.

The SQUBIC metasurface was reported to have EDOF properties [49]. The phase mask directs a collimated light beam into a line of discrete foci, achieving EDOF, and is given by the equation where α=arcsin(NA), given that NA is the nominal numerical aperture of the optics. Here, A is a design parameter that determines the strength of the phase mask [49]. In our design, we set A=50 and NA=0.45. Figure 1(A) shows the wrapped phase distributions of all the metasurfaces.

Fig. 1. EDOF metasurface design and measurements. (A) The phase masks of an ordinary metalens and four different EDOF metasurfaces. (B) Scanning electron micrographs of the fabricated metasurfaces. Inset shows the pillar distribution. (C) We experimentally measured the intensity along the optical axis where panels from top to bottom represent illumination by 625 nm, 530 nm, and 455 nm wavelengths. A cross section on the y–z plane is taken for each of the 3D PSFs.

下载图片 查看所有图片

These EDOF lenses are then implemented using cylindrical Si3N4 nanopillars [5,53] to ensure polarization insensitivity. These nanopillars are arranged on a square lattice. By varying the diameters of these nanoposts, the transmission coefficient imparted on incident light is modified as the coupling to and among different supported modes by the nanoposts changes, resulting in different phase shifts (Fig. 6, Appendix A.2). Rigorous coupled-wave analysis (RCWA) is used to construct a library consisting of the diameters of the nanoposts and the corresponding phase shift and amplitude.

3. EXPERIMENT

A fiber-coupled LED light source (Thorlabs M625F2, M530F2, M455F1) is used to illuminate the fabricated metasurfaces (one ordinary metalens and four EDOF metasurfaces). A custom microscope mounted on a computer-controlled translation stage is used to take snapshots of the imaging plane. The focal length of the metalens is found to be 231 μm with the green LED. This deviation from the designed focal length (200 μm) could be due to the difference in the designed wavelength (550 nm) and the experimental light source wavelength (530 nm). The imperfect fabrication could also contribute to this deviation. The nominal focal length is thereafter set to 231 μm for all metasurfaces to accommodate this shift. To verify the EDOF, PSFs in different x–y planes are measured along the optical axis (z-axis) starting from the metasurface (z=0) to z=400μm to construct a 3D PSF [y–z plane is shown in Fig. 1(C)]. The wavelength-dependent PSFs on the x–y plane at z=231μm are shown in Figs. 2(A)–2(C). For a standard metalens, the PSFs are very different for different colors, whereas for the EDOF lenses the variation is significantly reduced. As under white light illumination, all wavelengths are captured at the sensor array, a similar PSF for all colors enables computational reconstruction of the image. Additionally, we need the optical element to capture a wide band of spatial frequencies. To quantitatively understand this behavior, we calculate the MTF of all metasurfaces at the nominal focal plane. As seen in Fig. 2(D), the MTF of an ordinary metalens preserves spatial frequency information for green light (when focused at the sensor plane) but fails to preserve high-frequency components for blue or red. On the other hand, EDOF metasurfaces retain a broad range of spatial frequencies, exhibiting a higher cutoff frequency than the standard metalens. We note that, while some of the MTFs are oscillatory and have some zeros, for a wider range of higher spatial frequencies, we have non-zero MTF compared to the standard metalens. As discussed before, it is desirable to have an invariant PSF across the full visible wavelength range to achieve full-color imaging. We quantify this via calculating correlation between PSFs (see Appendix A.11). Table 1 shows the bandwidth (measured as the optical bandwidth where the correlation values fall to half of the maximum [54]) relative to the central wavelength of all the metasurfaces. Compared with the standard metalens, EDOF metasurfaces have a much wider spectral bandwidth.

Fig. 2. Characterization of the metasurfaces. The PSFs of the singlet metasurfaces were measured under (A) 455 nm blue, (B) 530 nm green, and (C) 625 nm red. The corresponding x–y plane cross sections of the experimental MTFs are displayed with red lines from its PSF measured under red light, green lines under green light, and blue lines under blue light. (D) The x–y plane cross sections of the theoretical MTFs are displayed in row (E). The scale bar signifies 25 μm. The MTF plots have a spatial frequency normalized to 560 cycles/mm.

下载图片 查看所有图片

Our EDOF imaging system presents a problem in the form of O=Kx+n, in which O is the observed image, K is the PSF or blur kernel, x is the latent image, and n is noise that corrupts the captured data. Although several methods exist to restore the image x given K and O, we chose to use Wiener deconvolution due to its effectiveness and simplicity. We note that even though we design K to be wavelength-invariant, it still has some residual wavelength dependence. The EDOF property that we engineer ensures that this variance is sufficiently small across the design bandwidth. Given the three primary colors in a typical camera sensor, we sample K at three wavelengths using separate sources at 625 nm, 530 nm, and 455 nm. We simulate the images by convolving the experimental PSFs with the ground truths and adding Gaussian white noise (Fig. 7, Appendix A.5). The simulated images are restored using the Wiener–Hunt deconvolution with the theoretical PSFs (Fig. 3). The structural similarities (SSIMs) and peak signal-to-noise ratios (PSNRs) are then calculated on the simulated images of Fig. 3.

Fig. 3. Simulated imaging performance after deconvolution. Deconvolved images captured by the EDOF imaging system, using the simulated images and PSFs. The experimental counterpart can be found in Fig. 4.

下载图片 查看所有图片

We then captured colored images projected by a SmallHD 5.5 in. Focus OLED HDMI Monitor using all the metasurfaces. We first record the spectra of the OLED monitor as well as the fiber-coupled LEDs. The individual spectra are recorded when displaying a single color at a time using a spectrograph (IsoPlane SCT320). The OLED monitor is placed ∼15cm away from the metasurface, which displays ground-truth images shown in Fig. 4. The raw images are captured using a camera (Allied Vision GT1930 C) and then computationally processed to create the actual image. To restore the captured raw images given PSF, a 3×3 median filter is first applied to the raw image to remove defective pixels. We then apply Wiener deconvolution (via the Wiener–Hunt algorithm [55]) on the images. For deconvolution, the PSF is first normalized by dividing by the sum of all PSF pixels. The Wiener filter estimates the desired image x^ for each color channel as follows: where ΛH is the optical transfer function, ΛD is a Laplacian filter that penalizes high frequencies, and L is the parameter that tunes the regularization. The L used for the deconvolutions is 1×10−4. We note that the normal metalens exhibits such a high level of chromatic aberration that we were unable to restore the raw images with any reasonably good results.

Fig. 4. Imaging performance. Restored images taken from (A) an OLED display of colored letters in ROYGBVWG, (B) a colorful neighborhood, and (C) vibrant umbrellas against the sky. The scale bar signifies 20 μm. Note that the metalens images are raw and unrestored.

下载图片 查看所有图片

We then characterized the experimentally captured images by calculating the SSIM for the “ROYGBVWG” image shown in Fig. 4(A). SSIM calculations are sensitive to the subject’s translation, scaling, and rotation, which are difficult to eliminate in an experimental setup. Hence, an intensity-based image registration method in MATLAB is utilized to transform the captured image to align with the ground truth using a similarity transformation [56]. This operation is performed for all three color channels simultaneously for each capture. The SSIMs are then calculated between the ground truth and the output images for the red, green, and blue channels, respectively. The values of the SSIM range from 0 to 1, 1 being a perfect match. As shown in Fig. 5, the imaging system with the singlet metalens exhibits the lowest SSIM score compared to the rest of the metasurfaces. Even though the SSIM score is at most 0.55, the scores on the devices with EDOF properties are significantly higher than that of the standard metalens. The metalens obtained poor imaging quality throughout all colors, although designed to work for green. The low SSIM of the metalens for green is due to the fact that the OLED display used has a nominal wavelength (511 nm) different from that of the light source (531 nm) used to measure the PSF for all the colors. The OLED monitor also has a much larger bandwidth (∼35nm for green) compared to the simulated bandwidth for the metalens, leading to a blurry image even for the intended wavelength. We attribute the higher SSIM values for color blue and lower values for color red in metasurfaces to the difference in luminance and contrast between the individual channels in the ground truths. This trend correlates to that of the simulated images (Table 5, Appendix A.6) and the trend disappears in the simulation when we use a gray-scale ground truth (Table 7, Appendix A.6).

Fig. 5. Full color SSIM. The restored captures are scaled, rotated, and translated to align with the ground truth; then SSIM is calculated for each color channel for the metasurface.

下载图片 查看所有图片

Fig. 6. Phase (dashed lines) and amplitude (solid lines) response of the nanopillars, simulated using RCWA.

下载图片 查看所有图片

Fig. 7. Simulated captured images before deconvolution. The experimental counterpart is shown in Fig. 9.

下载图片 查看所有图片

4. DISCUSSION

The SSIM results agree with the poor imaging quality exhibited by the metalens in the visible regime, caused by its strong chromatic aberrations. In contrast, the EDOF metasurfaces demonstrate an impressive ability to maintain a highly invariant PSF across a large spectral range. The imaging results as well as the SSIM calculations indicate that EDOF metasurfaces significantly outperform the standard metalens in full-color imaging. The log-asphere and shifted axicon designs both demonstrate the highest optical bandwidth for imaging. We note that recently a fundamental limit on the achievable optical bandwidth is reported given a thickness and numerical aperture of the lens [57]. For our parameters, the fundamental limit is ∼100nm, which is smaller than all the bandwidths demonstrated using EDOF lenses. While it is indeed possible to increase the optical bandwidth at the expense of image quality, it is difficult to estimate the Strehl ratio [58] for our EDOF lenses as the PSFs are very different from a lens and the final image is obtained only after deconvolution. Although previous works have demonstrated achromatic imaging with dispersion-engineered metalenses, our imaging system does not rely on any input polarization state and is generalizable to larger-aperture applications. A larger aperture will enable higher signal-to-noise ratio and faster shutter speed, which are crucial for practical applications. Moreover, dispersion-engineered metalenses require high aspect ratio scatterers and also multiple scatterers per unit cell, making the fabrication very challenging. In our approach the aspect ratios of the scatterers are relatively small, making the fabrication constraints more relaxed. Our image restoration process takes ∼2s to restore a single image (pixel dimension: 1936×1216; bit depth: 12 per color channel) on a single CPU. This process can, however, be significantly accelerated by utilizing multiple CPUs, graphics processing units (GPUs),or a dedicated field-programmable gate array. Preloading the MTF into memory will also speed up the computational reconstruction process. While this work uses Wiener deconvolution due to its speed, sophisticated deconvolution algorithms can be used to improve the image quality at the expense of time [59,60].

It is also important to emphasize the utility of rotational symmetric EDOF metasurfaces. A CPM is asymmetric and creates an accelerating Airy beam, which causes the PSF to shift for different wavelengths. This can be seen in the cross section of the 3D PSF in Fig. 1(C) as the CPM produces a curved focal line that does not coincide with the optical axis. This can cause misalignment across the color channels in the form of lateral chromatic aberration. Although the image registration corrects some of the misalignment across the color channels, residual artifacts can still be seen in Fig. 4(A), cubic column, in which the letters seem to have a red halo around them. In contrast, the other EDOF metasurfaces have a straight focal line that coincides with the optical axis. Moreover, the CPM adds asymmetric artifacts to images due to its asymmetric PSF (see Fig. 2, cubic column). In theory, the deconvolution can undo this, but because of noise, there will always be some residual component of the PSF as we need to balance noise amplification. Symmetric phase masks also have this problem, but the result is more aesthetically pleasing as the shape of the blur is symmetric. Finally, a symmetric phase mask is beneficial for simplified packaging as the relative orientation between the metasurface and the sensor does not affect the ultimate imaging performance.

5. SUMMARY

In summary, our imaging platform combines the form factor of ultrathin metasurfaces and the flexibility of computational imaging, making this an attractive solution for novel imaging applications. The CMOS compatibility of our silicon nitride platform, combined with a high NA of ∼0.45, makes this approach ideal for miniaturized microscopy, smartphone cameras, and endoscopy. Moreover, it is possible to increase the aperture of our EDOF hybrid system while maintaining the same NA and imaging characteristics. This aperture scalability combined with extremely lightweight and small size may open avenues to applications such as planar cameras as well as satellite imaging.

6 Acknowledgment

Acknowledgment. Part of this work was conducted at the Washington Nanofabrication Facility/Molecular Analysis Facility, a National Nanotechnology Coordinated Infrastructure (NNCI) site at the University of Washington, which is supported in part by funds from the National Science Foundation (NNCI-1542101, 1337840, and 0335765), the National Institutes of Health, the Molecular Engineering & Sciences Institute, the Clean Energy Institute, the Washington Research Foundation, the M. J. Murdock Charitable Trust, Altatech, ClassOne Technology, GCE Market, Google, and SPTS.