红外与激光工程, 2024, 53 (2): 20230252, 网络出版: 2024-03-27  

基于空间非一致模糊核标定的红外图像超分辨率重建方法

Infrared image super-resolution based on spatially variant blur kernel calibration
曹军峰 1,2,3,4丁庆海 5罗海波 1,2,3
作者单位
1 中国科学院光电信息处理重点实验室,辽宁 沈阳 110016
2 中国科学院沈阳自动化研究所,辽宁 沈阳 110016
3 中国科学院机器人与智能制造创新研究院,辽宁 沈阳 110169
4 中国科学院大学,北京 100049
5 航天恒星科技有限公司,北京 100086
摘要
近年来,红外成像系统在工业、安防、遥感等领域获得了广泛的应用,但由于制造工艺及成本制约,红外系统的分辨率仍然较低。基于深度神经网络的单帧图像超分辨率重建技术是提高红外图像分辨率的有效方法,获得了广泛研究,并在仿真图像上取得了显著进展,但应用于实际场景图像时容易出现伪影或图像模糊等现象。造成这种性能差异的主要原因是目前方法大多假定造成图像退化的模糊核是空间一致的,然而实际红外光学系统不可避免地存在像差、热离焦等,由此造成的图像模糊的模糊核并非空间一致的。针对这一问题,提出了一种非盲模糊核估计方法,通过采集特定的靶标图像,并设计模糊核估计网络,求解空间非一致模糊核;设计基于图像分块的超分辨率重建方法,将图像块和对应区域的模糊核一起输入非盲超分辨率重建网络进行子块图像重建,再通过子块合并和重叠区域图像融合,得到最终的高分辨率图像。实验结果表明,光学系统自身引起了模糊核随空间位置缓慢变化,在实验室条件下标定模糊核并基于图像分块进行超分辨率重建的方法可显著提高红外图像超分辨率重建的效果。
Abstract
ObjectiveIn recent years, infrared imaging systems have been increasingly used in industry, security, and remote sensing. However, the resolution of infrared devices is still quite limited due to its cost and manufacturing technology restrictions. To increase image resolution, deep learning-based single image super-resolution (SISR) has gained much interest and made significant progress in simulated images. However, when applied to real-world images, most approaches suffer a performance drop, such as over-sharpening or over-smoothing. The main reason is that these methods assume that blur kernels are spatially invariant across the whole image. But such an assumption is rarely applicable for infrared images, whose blur kernels are usually spatially variant due to factors such as lens aberrations and thermal defocus. To address this issue, a blur kernel calibration method is proposed to estimate spatially-variant blur kernels, and a patch-based super-resolution (SR) algorithm is designed to reconstruct super-resolution images. MethodsParallel light tube and motorized rotating platform are used to establish target image acquisition environment, and then images of multi-circle target at different positions are gathered (Fig.1). Based on sub-pixel accurate circle center detection, the camera pose parameters are solved, and high-resolution target images are synthesized according to the parameters. High-resolution and low-resolution target image pairs are fed into the blur kernel estimation network to obtain accurate blur kernels (Fig.3). In addition, a patch-based super-resolution algorithm is designed, which decomposes the test image into overlapping patches, reconstructs each of them separately using estimated kernels, and finally merges them according to Euclidean distances (Fig.4). Results and Discussions The experimental results show that the blur caused by the optical system is not negligible and varies slowly with spatial position (Fig.6). The proposed method, which calibrates blur kernels in a laboratory setting, can obtain a more accurate blur kernel estimation result. As a consequence, the proposed patch-based super-resolution algorithm can produce more visually pleasant results with more reliable details (Fig.7-8), and can also boost objective quality evaluation indicators such as natural image quality evaluator (NIQE), perception based image quality evaluator (PIQE), and blind/referenceless image spatial quality evaluator (BRISQUE) (Tab.1). SR experiments on 4-bar targets with different spatial frequencies show that the proposed method can distinguish the target with spatial frequency of 3.57 cycles/mrad, while comparison methods can just distinguish that of 3.05 cycles/mrad under the same conditions (Fig.9). ConclusionsA blur kernel calibration method is proposed to estimate spatially-variant blur kernels, and a patch-based super-resolution algorithm is designed to implement super-resolution reconstruction. The experimental results show that image blur caused by the optical system changes slowly with the spatial position. As a result, one blur kernel can be estimated for each image patch, instead of densely estimated for each pixel, thereby reducing the complexity of calibration and memory consumption during reconstruction. Thanks to the accurate blur kernel estimation, the proposed super-resolution algorithm outperforms the comparison methods in both qualitative and quantitative results. Furthermore, the blur kernel calibration method is easy to implement in engineering applications. For any infrared camera, only dozens of multi-circle target images covering all areas of the focal plane are needed to complete the calibration process. When real-time performance is required, the proposed blur kernel calibration method can also be combined with other lightweight non-blind super-resolution methods to achieve a real-time performance. In the future, the problem of image blur caused by thermal defocusing will be studied to expand the scope of the method.

曹军峰, 丁庆海, 罗海波. 基于空间非一致模糊核标定的红外图像超分辨率重建方法[J]. 红外与激光工程, 2024, 53(2): 20230252. Junfeng Cao, Qinghai Ding, Haibo Luo. Infrared image super-resolution based on spatially variant blur kernel calibration[J]. Infrared and Laser Engineering, 2024, 53(2): 20230252.

引用该论文: TXT   |   EndNote

相关论文

加载中...

关于本站 Cookie 的使用提示

中国光学期刊网使用基于 cookie 的技术来更好地为您提供各项服务,点击此处了解我们的隐私策略。 如您需继续使用本网站,请您授权我们使用本地 cookie 来保存部分信息。
全站搜索
您最值得信赖的光电行业旗舰网络服务平台!