深度压缩新方法大幅提高单像素相机的成像速度

提及光学成像，大家通常想到的工具就是具有数百万个像素的相机。但在不可见光谱下，其灵敏度较差，且只有造价非常昂贵的相机才有可能克服这个缺点，从而在此种条件下具有较高灵敏度且经济实惠的单个探测器很快受到了人们的广泛关注。

通过单个探测器成像的最直接也是最常见的方法就是对物体进行扫描，逐个像素对其进行测量。另一种非常有趣的方法是利用压缩感测，通常也称为单像素相机成像。与逐个像素采样不同，这种方式通过对不同像素的组合来对物体进行采样。单像素成像使用某一图案照射物体，并用探测器测量那些受到图案照射的物体像素的总强度，通过变换不同的照明图案并进行测量，进而重建物体。

在常规相机和基于点扫描的成像系统中，实际的测量次数与像素总数相同（前者通过相机上的所有探测器一次性测量所有像素，而后者则通过单个探测器逐个测量像素）。然而，在压缩感测中，测量的次数与照明图案的数量相同，其远小于像素的数量（1%~10%），同时还可以高质量地恢复成像物体，从而压缩感测具有很高的测量效率。

但是，利用压缩感测的单像素相机需要使用DMD（Digital Micromirror Device）去切换不同的照明图案，而这个切换速度限制了整体的成像速度。从而导致在许多情况下，这种单像素相机的成像速度低于基于点扫描或常规相机的成像速度。

针对上述研究现状，美国加州大学戴维斯分校的杨暐健教授课题组在Photonics Research 2021年第3期（Kangning Zhang, Junjie Hu, Weijian Yang. Deep compressed imaging via optimized pattern scanning[J]. Photonics Research, 2021, 9(3): 03000B57 ）发表的文章中展示了一种新的压缩成像方式（DeCIOPS）。该成像方式通过在物体上投影并扫描一个被经过优化的图案，再进行深度压缩成像，可以解决现有单像素相机中遇到的困难，大大提高压缩成像的成像速度。

DeCIOPS的示意图。DeCIOPS通过一个照明图案对物体进行区块采样, 并利用ISTA-Net（一种基于压缩感测的深度神经网络）重建高分辨率物体（照明图案和ISTA-Net中的参数由自编码器的端到端训练进行优化）

DeCIOPS把一个照明图案投影到物体的一个区块上，然后通过照明图案对这个区块上的所有像素进行加权求和而得到一个测量值。只需扫描照明图案，就可以测量物体的不同区块。接着，研究者们开发了一种基于压缩感知的深度神经网络，可以从这种看似低分辨率的区块测量中恢复高分辨率的图像，其从本质上实现了超分辨率成像。

此外，研究人员通过自编码器对DeCIOPS进行建模，并对该编码器以端到端的方式进行训练，其在实现了用于图像采集的照明图案和物体重建的深度神经网络优化的同时，也确保了物体的高质量重建。

整体而言，DeCIOPS综合了点扫描中的高采集速度和压缩感测中的高测量效率的优势。与点扫描系统相比，DeCIOPS通过高效的采样方式提高了成像速度（几倍到一个数量级）。与需要在许多照明图案之间进行切换的常规压缩成像相比，DeCIOPS只需要在整个物体上扫描单个照明图案，从而将整体成像速度提高多个数量级。

杨暐健教授表示，“DeCIOPS紧密集成了成像系统和计算算法，是计算成像中一个很好的例子，并且该成像方法提供的高成像速度使其在生物医学、监视和消费电子领域都有非常大的应用空间。”

New imaging modality through deep compressed sensing enables high speed imaging

Optical imaging typically relies on a camera with millions of photodetectors. By contrast, imaging can also take place with a single detector. This is particularly advantageous for imaging at non-visible spectrum where the conventional pixelated cameras lose the sensitivity or become very costly for a good performance.

A most straightforward way to image through a single detector is to scan the object and measure it pixel by pixel. Yet, another interesting approach, commonly known as single-pixel camera, is to leverage compressed sensing. Instead of being sampled pixel by pixel, the object is sampled through the sum of different combinations of pixels. Typically, the object is illuminated with a pattern, and the detector captures the sum intensity of the object pixels illuminated by the pattern. By performing the measurement through different patterns, the entire object can be reconstructed.

In both the conventional camera and the point-scanning-based system, the actual number of measurements is the same as the total number of pixels, though the former measures all pixels at once through different detectors and the latter measures individual pixel sequentially through a single detector. However, the number of measurements, i.e. the number of illumination patterns, can be much smaller than the number of pixels (for example 10~100 times smaller) in compressed sensing. The computation algorithm can then recover the object in high quality through a small number of measurements, by exploiting the natural relationship between adjacent pixels. As a result, compressed sensing has a very high measurement efficiency. The fewer measurements make it promising to reduce the data acquisition time and thus increase the imaging speed.

However, the existing implementation of compressed sensing in imaging relies on switching different illumination masks, and the limited speed in the switching device (digital mirror device) restraints the overall speed. In many scenarios, the imaging speed of the switching-mask-based single-pixel camera is actually lower than that based on point scanning or conventional cameras.

The research group led by Prof. Weijian Yang from University of California, Davis recently demonstrated a new compressed imaging modality, termed "deep compressed imaging via optimized pattern scanning" (DeCIOPS), which could overcome the challenges in the existing single-pixel cameras, and greatly improve the imaging speed of compressed imaging. The research results are published in Photonics Research, Vol. 9, No. 3, 2021 (Kangning Zhang, Junjie Hu, Weijian Yang. Deep compressed imaging via optimized pattern scanning[J]. Photonics Research, 2021, 9(3): 03000B57 ).

Schematics of DeCIOPS (deep compressed imaging via optimized pattern scanning). The object is sampled block by block by an illumination pattern. ISTA-Net, a compressed-sensing-inspired deep neural network is used to reconstruct the object. The illumination pattern and the parameters in ISTA-Net are optimized through an end-to-end trained auto-encoder.

In this new imaging modality (DeCIOPS), a block of pixels, weighted by an illumination pattern, are sampled and summed into a single measurement. By scanning the illumination pattern across the object, different blocks of pixels could be measured. A reconstruction algorithm based on a compressed-sensing-inspired deep neural network was developed to recover the high resolution image from this seemingly low-resolution block-by-block measurement. Since the number of measurement blocks can be much smaller than the number of total pixels, the measurement speed can be very high.

In addition, the researchers modeled DeCIOPS through an auto-encoder, which was trained in an end-to-end manner and jointly optimized the illumination pattern for image acquisition and the deep neural network for object reconstruction. The end-to-end optimization ensures an overall optimal performance of the entire imaging system and thus a high quality in object reconstruction.

Essentially, DeCIOPS synthesizes the strengths of the high acquisition speed of point scanning and the high measurement efficiency of compressed sensing. Compared to the point scanning system, DeCIOPS increases the imaging speed (by a few folds to one order of magnitude) through a highly efficient sampling scheme. Compared to the conventional compressed imaging which requires switching between many illumination patterns, DeCIOPS scans a single illumination pattern across the object, and could increase the overall acquisition speed by orders of magnitude.

"DeCIOPS represents a nice example in computational imaging with a tight integration of the imaging system and the computation algorithm," said Prof. Weijian Yang. He believes that the high imaging speed offered in DeCIOPS makes it very promising for applications in biomedicine, surveillance, and consumer electronics.