首页 > 论文 > 光学学报 > 40卷 > 18期(pp:1810001--1)

基于级联多尺度信息融合对抗网络的红外仿真

Infrared Simulation Based on Cascade Multi-Scale Information Fusion Adversarial Network

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

提出了一种应用于红外图像仿真的级联多尺度信息融合生成对抗网络,能由可见光图像估计对应的红外图像。针对可见光与红外图像特征之间的关联与区别,该网络采用级联的对抗网络结构:第一级对抗网络以语义分割图像为辅助任务,使用大感受野的卷积网络结构,重建红外图像的结构信息;第二级对抗网络以可见光的灰度反转图像为辅助任务,采用小感受野的网络结构,补充红外仿真图像的细节纹理信息,并使用多尺度融合模块整合多感受野信息以提升算法精度。在先进算法的通用数据集上进行实验,结果表明,级联多尺度信息融合对抗网络能够实现可见光到红外图像的转换,可得到结构与纹理都较正确的红外仿真图像,在多种客观指标与主观感受上均优于其他类似算法。

Abstract

In this paper, we propose a cascade multi-scale information fusion generative adversarial network (CMIF-GAN) for infrared image simulation, which can estimate the infrared map from a visible image. Inspired by the connections and differences between visible and infrared features, CMIF-GAN adopts a cascaded structure composed of two levels of adversarial networks. With a large overall receptive field, the first-level adversarial network focuses on reconstructing structural information of the infrared image, and adds a semantic segmentation image task as auxiliary information. To enrich detailed texture information of the infrared image, the second-level adversarial network uses the grayscale inverted visible (GIV) images as auxiliary information and adopts a small overall receptive field network. Otherwise, the second-level adversarial network can integrate the multiple receptive information by a multi-scale fusion module (MFM) to improve algorithm accuracy. Experiments on public dataset demonstrate that CMIF-GAN can efficiently translate visible images to corresponding infrared images, and outperform previous methods in objective metrics and subjective vision.

广告组1 - 空间光调制器+DMD
补充资料

中图分类号:TP391

DOI:10.3788/AOS202040.1810001

所属栏目:图像处理

基金项目:国家重点研发计划、国家自然科学基金青年基金、北方工业大学学生科技活动项目;

收稿日期:2020-04-07

修改稿日期:2020-06-03

网络出版日期:2020-09-01

作者单位    点击查看

贾瑞明:北方工业大学信息学院, 北京 100144
李彤:北方工业大学信息学院, 北京 100144
刘圣杰:北方工业大学信息学院, 北京 100144
崔家礼:北方工业大学信息学院, 北京 100144
袁飞:中国科学院自动化研究所数字内容技术与服务研究中心, 北京 100190

联系人作者:贾瑞明(jiaruiming@ncut.edu.cn)

备注:国家重点研发计划、国家自然科学基金青年基金、北方工业大学学生科技活动项目;

【1】Mu C P, Peng M S, Dong Q X, et al. Infrared image simulation of ground maneuver target and scene based on OGRE [J]. Applied Mechanics and Materials. 2015, 3752(716): 932-935.

【2】Ma Y, Tian Y. Modeling method of warship radiation model for infrared simulation Tactical Missile Technology[J]. 0, 2013(3): 67-70,75.
马艳, 田宇. 红外仿真中舰船辐射模型建模方法 战术导弹技术[J]. 0, 2013(3): 67-70, 75.

【3】Yang M, Li M, Yi Y X, et al. Infrared simulation of ship target on the sea based on OGRE [J]. Laser & Infrared. 2017, 47(1): 53-57.
杨敏, 李敏, 易亚星, 等. 基于OGRE的海面舰船目标红外仿真方法 [J]. 激光与红外. 2017, 47(1): 53-57.

【4】Xie J R, Li F M, Wei H, et al. Infrared target simulation method based on generative adversarial neural networks [J]. Acta Optica Sinica. 2019, 39(3): 0311002.
谢江荣, 李范鸣, 卫红, 等. 基于生成对抗式神经网络的红外目标仿真方法 [J]. 光学学报. 2019, 39(3): 0311002.

【5】Zhou Q, Bai T Z, Liu M Q, et al. Near infrared scene simulation based on visual image [J]. Infrared Technology. 2015, 37(1): 11-15.
周强, 白廷柱, 刘明奇, 等. 基于可见光图像的近红外场景仿真 [J]. 红外技术. 2015, 37(1): 11-15.

【6】Li M, Xu Z W, Xie H W, et al. Infrared image generation method and detail modulation based on visible light images [J]. Infrared Technology. 2018, 40(1): 34-38.
李敏, 徐中外, 解鸿文, 等. 基于可见光图像的红外图像生成方法及其细节调制 [J]. 红外技术. 2018, 40(1): 34-38.

【7】Eigen D, Puhrsch C, Fergus R. Depth map prediction from a single image using a multi-scale deep network . [C]∥International Conference on Neural Information Processing Systems. 2014, 2366-2374.

【8】Wang P, Shen X H, Lin Z, et al. Towards unified depth and semantic prediction from a single image[C]∥2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 7-12 June 2015, Boston, MA, USA. New York: , 2015, 2800-2809.

【9】Xu D, Ricci E, Wanli O Y, et al. Multi-scale continuous CRFs as sequential deep networks for monocular depth estimation[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 21-26 July 2017, Honolulu, HI, USA. New York: , 2017, 161-169.

【10】Xu D, Wang W, Tang H, et al. Structured attention guided convolutional neural fields for monocular depth estimation[C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 18-23 June 2018, Salt Lake City, UT, USA. New York: , 2018, 3917-3925.

【11】Laina I, Rupprecht C, Belagiannis V, et al. Deeper depth prediction with fully convolutional residual networks[C]∥2016 Fourth International Conference on 3D Vision (3DV). 25-28 Oct. 2016, Stanford, CA, USA. New York: , 2016, 239-248.

【12】Jia R M, Liu L Q, Liu S J, et al. Single image depth estimation based on encoder-decoder convolution neural network [J]. Journal of Graphics. 2019, 40(4): 718-724.
贾瑞明, 刘立强, 刘圣杰, 等. 基于编解码卷积神经网络的单张图像深度估计 [J]. 图学学报. 2019, 40(4): 718-724.

【13】Jia R M, Li Y, Li T, et al. - . 1000, 3428: 0056477.
贾瑞明, 李阳, 李彤, 等. - [2020-04-01]. . 多层级特征融合结构的单目图像深度估计网络. 计算机工程[2020-04-01]. https: ∥doi.org/10.19678/j.issn. 1000, 3428: 0056477.

【14】Qi X J, Liao R J, Liu Z Z, et al. GeoNet: , 2018, 283-291.

【15】Yin Z C, Shi J P. GeoNet: , 2018, 1983-1992.

【16】Ranjan A, Jampani V, Balles L, et al. Competitive collaboration: , 2019, 12232-12241.

【17】Jiao J B, Cao Y, Song Y B, et al. Look deeper into depth: monocular depth estimation with semantic booster and attention-driven loss [J]. Computer Vision-ECCV. 2018, 2018: 53-69.

【18】Mirza M. -11-06)[2020-04-01] . https: ∥arxiv. 2014, org/abs/1411: 1784.

【19】Hu L M, Zhang Y. Facial image translation in short-wavelength infrared and visible light based on generative adversarial network [J]. Acta Optica Sinica. 2020, 40(5): 0510001.
胡麟苗, 张湧. 基于生成对抗网络的短波红外-可见光人脸图像翻译 [J]. 光学学报. 2020, 40(5): 0510001.

【20】Isola P, Zhu J Y, Zhou T H, et al. Image-to-image translation with conditional adversarial networks[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 21-26 July 2017, Honolulu, HI, USA. New York: , 2017, 5967-5976.

【21】Ma S, Fu J L, Chen C W, et al. DA-. GAN: , 2018, 5657-5666.

【22】Mejjati Y A, Richardt C, Tompkin J, et al. -11-08)[2020-04-01] . https: ∥arxiv. 2018, org/abs/1806: 02311.

【23】Tang H, Xu D, Sebe N, et al. Multi-channel attention selection GAN with cascaded semantic guidance for cross-view image translation[C]∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 15-20 June 2019, Long Beach, CA, USA. New York: , 2019, 2412-2421.

【24】Luo W J, Li Y J, Urtasun R, et al. -01-25)[2020-04-01] . https: ∥arxiv. 2017, org/abs/1701: 04128.

【25】Hwang S, Park J, Kim N, et al. Multispectral pedestrian detection: , 2015, 1037-1045.

【26】Ioffe S. -03-02)[2020-04-01] . https: ∥arxiv. 2015, org/abs/1502: 03167.

【27】Jia R M, Qiu Z Z, Cui J L, et al. Deep multi-scale encoder-decoder convolutional network for blind deblurring [J]. Journal of Computer Applications. 2019, 39(9): 2552-2557.
贾瑞明, 邱桢芝, 崔家礼, 等. 盲去模糊的多尺度编解码深度卷积网络 [J]. 计算机应用. 2019, 39(9): 2552-2557.

【28】Lin G S, Milan A, Shen C H, et al. RefineNet: , 2017, 5168-5177.

引用该论文

Jia Ruiming,Li Tong,Liu Shengjie,Cui Jiali,Yuan Fei. Infrared Simulation Based on Cascade Multi-Scale Information Fusion Adversarial Network[J]. Acta Optica Sinica, 2020, 40(18): 1810001

贾瑞明,李彤,刘圣杰,崔家礼,袁飞. 基于级联多尺度信息融合对抗网络的红外仿真[J]. 光学学报, 2020, 40(18): 1810001

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF