激光与光电子学进展, 2021, 58 (18): 1811013, 网络出版: 2021-09-03
多相机系统:成像增强及应用 
下载： 1486次特邀综述
Multi-Camera System: Imaging Enhancement and Application
图 & 表
图 2. 多相机系统对模拟场景的多维多尺度采样说明[15]。(a) 模拟场景下,三相机视点示意图; (b) 三相机对场景各维各尺度均匀采样,即三相机的成像控制参数一致;(c)三相机对场景各维各尺度非均匀采样,即三相机的成像控制参数不一致,例如#1相机采用低空间分辨率,#2相机未保留色彩信息
Fig. 2. Illustration of various dimensional and multi-scale data acquisition with multi-camera systems in the toy model[15]. (a) The illustration of three cameras’ viewpoints in simulation scene; (b) uniform sampling in the multi-dimensional scene features, namely same imaging control setting for all cameras; (c) non-uniform sampling in the multi-dimensional scene features, namely various imaging control settings among cameras. For example, camera #1 captures at the low spatial resolution and camera #2 doesn’t record color information
图 3. 单相机扫描拍摄及拼接结果[24]。(a)直接拼接的结果;(b)对(a)进行亮度对齐和色调映射处理后的结果
Fig. 3. Mosaic results acquired with single camera through scanning[24]. (a) Captured images with only geometric alignment; (b) result after radiometric alignment and tone mapping
图 4. 一种宽视场高清成像相机阵列及其合成结果[27]。(a)相机阵列结构图;(b)由该阵列相机拍摄得到的宽视场超高清图像;(c) 图(b)的局部放大视野
Fig. 4. A wide-field high-resolution imaging array camera and its synthesized image[27]. (a) Architecture of the array camera; (b) captured image using the array camera; (c) zoomed-in view of (b)
图 6. AWARE-2的光学结构及重建结果。(a)(b)AWARE-2的并行微型相机结构图[25];(c)AWARE-2 拍摄的宽视场超高清图像及其局部放大[31]
Fig. 6. Optical structure and image reconstruction of AWARE-2. (a)(b) Structure of AWARE-2’s parallel array of micro-camera [25]; (c) an image captured using AWARE-2, of which the insets are digitally magnified images[31]
图 7. 非结构化阵列相机[26]。(a)相机系统及子模块结构图,相机和镜头的性质根据目标性质非均匀选择;(b) 使用非结构化相机捕获的十亿像素级视频帧,红色和蓝色框分别表示全局和局部摄影机的分布
Fig. 7. The UnstructuredCam[26]. (a) Schematic of UnstructuredCam system consisting of multiple columns of subarrays and UnstructuredCam module, and the cameras and lenses are selected heterogeneously based on the nature of the targets; (b) gigapixel-level videography captured using the UnstructuredCam, where the red and blue frames represent the distributions of the global and local cameras
图 8. 运动混叠现象[34]。(a)红球以正弦状的轨迹在运动;(b)相机以较低帧率对该运动状态进行捕获所获得的帧序列,该帧序列中显示红球沿直线运动,这种错误的感知现象被称为运动混叠现象;(c)利用已有序列在时间维度上理想插值获得的帧序列(蓝色虚线矩形代表预测帧)无法恢复正确的运动信息
Fig. 8. Motion aliasing[34]. (a) A ball moving in a sinusoidal trajectory; (b) an image sequence of the ball is captured at low frame-rate, the perceived motion is along a straight line, this false perception is referred to as “motion aliasing”; (c) the filled-in frames (indicated with blue dashed lines) which are obtained via an ideal temporal interpolation can not produce the correct motion
图 9. 同帧率的异步触发相机组合[34]。(a)~(d)不同相机在不同时刻开始拍摄的帧序列;(e)恢复的高帧率帧序列
Fig. 9. Illustration of multi-cameras triggered with the same frame rate[34]. (a)--(d)Captured sequences from different cameras at different time; (e) reconstructed sequences with high frame rate
图 10. Wilburn等的相机阵列示意图[35]。(a)相机系统;(b)相机阵列的触发顺序;(c)从不同帧中获取相同时刻的数据切片生成最终图片,抑制电子卷帘快门引起的失真;(d)重建的气球被击破的高速视频片段
Fig. 10. Illustration of Wilburn et al.’s multiple camera array[35]. (a) Detailed structure; (b) firing order for camera array; (c) the final view consists of different rows (shown in gray) in each camera at the same time, which suppresses the distortion caused by rolling shutter; (d) reconstructed high-speed video of a popping balloon
图 11. Wang等所提方案的系统结构及输入、输出[36]。(a)系统结构;(b)系统捕获的标准30 frame/s视频内容和3 frame/s光场序列;(c)最终生成的30 frame/s光场视频
Fig. 11. System setup and input/output proposed by Wang et al.[36]. (a) Camera setup; (b) the inputs consist of a standard 30 frame/s video and a 3 frame/s light field sequence; (c) finally generated 30 frame/s light field video
图 12. Wang等提出的图像处理框架[36]。(a)估测关键帧间的视差以及视频帧在时间维度上的运动信息,从而预测目标帧时刻两路视角对应的视差图,实现从高帧率视频内容向低帧率视点的映射;(b)融合所有参考图像形成最终的重建图像
Fig. 12. Overall image processing architecture proposed by Wang et al.[36]. (a) Estimates the disparity at the key frames, as well as the temporal flow in the 2D video to generate the disparity at the target frame and then warps high frame-rate video to the low frame-rate viewpoint; (b) fuses all the reference images to output the final reconstruction
图 13. Cheng等方案的具体阐述[37]。(a)双相机结构;(b)高分辨率信息在时间维度上的传递过程;(c)高、低分辨率融合的具体流程
Fig. 13. Illustration of Cheng et al.’s proposal[37]. (a) Dual camera setup; (b) propagation of high-resolution map along with the temporal dimension; (c) the detailed pipeline of fusing the high resolution map and the low resolution map
图 14. Cheng等方案的可视化效果[37]。(a) 高分辨率、低帧率的输入图Iref; (b) 上采样的低分辨率、高帧率输入图ILSR↑;(c)(d)Iref和ILSR↑中对应的局部图片特写; (e) 该方案的重建图
Fig. 14. Visualized results of Cheng et al.’s proposal[37]. (a) Input frame (Iref) with high resolution and low frame rate; (b) input frame (ILSR↑) with low resolution and high frame rate; (c)(d) close-up of patches in Iref and ILSR↑; (e) reconstructed frame with the proposal
图 16. 基于多曝光图像序列的高动态重建和基于反色调映射的高动态重建示意图[39]。(a)基于多曝光图像序列的高动态重建;(b)基于反色调映射的高动态重建
Fig. 16. High dynamic range imaging based on multi-exposure image sequence and high dynamic range imaging based on inverse tone mapping[39]. (a) High dynamic range imaging based on multi-exposure image sequence; (b) high dynamic range imaging based on inverse tone mapping
图 17. 基于分光镜的高动态相机示意图及重建结果[41]。(a)1∶16透过率的分光镜同步拍摄;(b)50%透过率的分光镜配合减光滤光镜拍摄; (c)高动态相机重建的场景
Fig. 17. Schematic and reconstructed frames of high dynamic range camera based on beam splitter[41]. (a) Capture with 1∶16 beam splitter; (b) capture with 50% semitransparent mirror and camera mounted with a neutral density filter; (c) reconstructed scene using the high dynamic range camera
图 18. 多传感器高动态相机及一组拍摄结果[42]。(a)相机的光学结构,其中LE、ME、HE分别代表传感器接收的光的量为低、中、高;(b)最终的重建图像以及由HE、ME、LE传感器得到的低动态范围(LDR)图像
Fig. 18. Multi-sensor high dynamic range camera and a group of captured images[42]. (a) Optical architecture of the high dynamic range camera, the terms high, medium, and low exposure (HE, ME, LE, respectively) refer to the sensors based on the amount of light each sensor receives; (b) final reconstructed image, where the inset photos show the low dynamic range images from the high, medium, and low-exposure sensors, respectively
图 19. 基于可变形卷积的高动态重建[50]。(a)重建网络流程图;(b)多相机多曝光输入的高动态重建结果
Fig. 19. High dynamic range (HDR) reconstruction based on deformable convolution[50]. (a) Flow chart of HDR reconstruction network; (b) HDR reconstruction on multi-exposure multi-camera input
图 20. 无外置光源成像与有外置光源成像的对比[55]。(a) 原始无外置光源成像、有外置光源成像以及两者结合的成像结果;(b)~(d) 三种成像结果的局部细节效果图
Fig. 20. Illustration of imaging with flash and without flash[55]. (a) Standard photography, flash photography and the combined photography;(b)--(d) enlarged views of the local details
图 21. 传统RGB成像与暗闪光成像的光谱响应曲线说明及不同通道对应亮度的对比[56]。(a)(c)传统RGB成像和暗闪光成像的光谱响应曲线,j∈{1,2,3,4,5}分别对应可见光红色、绿色、蓝色以及红外、紫外波段;(b)暗闪光光源在1 m处对应的绝对辐照度;(d)~(l)低光照条件下,长曝光RGB成像、环境光RGB成像以暗闪光成像在相同位置处可见光红色和红外波段的亮度及其梯度对比
Fig. 21. Illustration of spectral response curves for traditional RGB imaging and dark flash imaging along with the luminance comparison between different channels[56]. (a) (c) Spectral response curves for RGB imaging and dark flash imaging, j∈{1,2,3,4,5} for red, green, blue, IR, and UV channels; (b) absolute irradiance at 1 m from the dark flash; (d)--(l) under low-light condition, luminance and its gradient comparison between the red channel and IR channel of the corresponding pixel in the long-exposure imaging, ambient light imaging, and dark flash imaging
图 22. 可见光与不可见光波段成像组合示意图[57]。(a)理想相机原型;(b)实际搭建系统及光谱曲线
Fig. 22. Illustration of hybrid imaging setup with RGB and NIR-G-NUV cameras[57]. (a) An idealized prototype; (b) an actual camera system and spectral curves
图 23. Wang 等方案的效果图[57]。(a)低照度RGB相机输入(这里显示了5倍数字增益后的图像,便于观察);(b)暗闪光相机输入;(c)输出
Fig. 23. Visualization of Wang et al.’s method[57]. (a) RGB input (visualized here with a digital gain of 5×); (b) dark-flash input;(c) output
图 24. 混合尺度的单通道相机与彩色相机实现的低照度高质量彩色成像流程图[59]
Fig. 24. Workflow of low-light and high-quality color imaging with hybrid monochrome and color cameras[59]
图 25. Guo等的方案在低照度实拍数据集上的效果细节可视化[59]。(a)低分辨率RGB输入图像(此处为了方便对比,将低分辨率输入放大到高分辨率大小);(b)高分辨率单通道输入图像;(c)重建后的高分辨率RGB图像; (d) 对(a)图进行数字增益, 用以展示噪声与模糊现象
Fig. 25. Close-up performance of Guo et al.’s method on the real scenes in the low-light condition[59]. (a) Low-resolution RGB input images (for comparison, the low-resolution inputs are enlarged to the high-resolution sizes); (b) high-resolution monochromic images; (c) reconstructed high-resolution RGB images; (d) the value-amplified low-resolution images from (a) for showing noise and blur
郭珮瑶, 蒲志远, 马展. 多相机系统:成像增强及应用[J]. 激光与光电子学进展, 2021, 58(18): 1811013. Peiyao Guo, Zhiyuan Pu, Zhan Ma. Multi-Camera System: Imaging Enhancement and Application[J]. Laser & Optoelectronics Progress, 2021, 58(18): 1811013.
PDF全文
