首页 > 论文 > 激光与光电子学进展 > 55卷 > 9期(pp:91505--1)

基于双通道循环生成对抗网络的无人车夜视红外视频彩色化

Unmanned Vehicle Night Infrared Video Colorization Based on Dual-Channel Cycle-Consistent Adversarial Networks

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

在无人车夜视红外视频彩色化问题中, 考虑到可同时利用单帧图像的信息和视频的帧间信息, 提出了一种双通道循环生成对抗网络(DcCCAN)对夜视红外视频进行彩色化。DcCCAN是在循环一致生成对抗网络(CCAN)的基础上提出的双通道生成网络。双通道生成网络具有良好的图像特征提取能力, 能够自动提取视频中待处理图像的特征, 同时提取先前模型所生成图像的特征, 然后将特征信息整合后生成一幅目标图像。通过在生成对抗性训练中引入循环一致性训练机制, 可无监督地学习得到红外域图像到彩色域图像的映射关系, 从而实现红外视频的彩色化。实验表明该方法能够为视频中的红外图像赋予自然的色彩信息和纹理信息, 且满足实时性要求。

Abstract

In the task of unmanned vehicle infrared video colorization, considering the uniqueness of a single frame and the continuity of the entire infrared video, a dual-channel cycle-consistent adversarial network (DcCCAN) based colorization method is proposed. The dual-channel generation network we proposed is on the basis of cycle-consistent adversarial network (CCAN) and has good image feature extraction ability, which can automatically extract the features of the frame in the video, and at the same time can extract the features of the previous frame generated. By joint training of adversarial loss and cycle-consistent loss, the function from infrared domain image to color domain image can be learned by unsupervised learning methods and the colorization of the infrared video can be realized. The experimental results show that the proposed method can provide natural color information and texture information for the infrared images in the video, and meet the real-time requirements.

Newport宣传-MKS新实验室计划
补充资料

中图分类号:TP391

DOI:10.3788/lop55.091505

所属栏目:机器视觉

基金项目:上海市科委基础研究项目(15JC1400600)、国家自然科学基金青年科学基金(61603089)、上海市青年科技英才扬帆计划(16YF1400100)

收稿日期:2018-03-28

修改稿日期:2018-04-17

网络出版日期:2018-04-23

作者单位    点击查看

李佳豪:东华大学信息科学与技术学院, 上海 201620东华大学数字化纺织服装技术教育部工程研究中心, 上海 201620
孙韶媛:东华大学信息科学与技术学院, 上海 201620东华大学数字化纺织服装技术教育部工程研究中心, 上海 201620
吴雪平:东华大学信息科学与技术学院, 上海 201620东华大学数字化纺织服装技术教育部工程研究中心, 上海 201620
李大威:东华大学信息科学与技术学院, 上海 201620东华大学数字化纺织服装技术教育部工程研究中心, 上海 201620

联系人作者:孙韶媛(shysun@dhu.edu.cn); 李佳豪(jiahaoli0305@gmail.com);

【1】Gao K J, Sun S Y, Yao G S, et al. Semantic segmentation of night vision images for unmanned vehicles based on deep learning[J]. Journal of Applied Optics, 2017, 38(3): 421-428.
高凯珺, 孙韶媛, 姚广顺, 等. 基于深度学习的无人车夜视图像语义分割[J]. 应用光学, 2017, 38(3): 421-428.

【2】Zhu L B, Sun S Y, Gu X J, et al. Image colorization based on color transfer and propagation[J]. Journal of Image and Graphics, 2010, 15(2): 200-205.
朱黎博, 孙韶媛, 谷小婧, 等. 基于色彩传递与扩展的图像着色算法[J]. 中国图象图形学报, 2010, 15(2): 200-205.

【3】Dai Z H, Sun S Y, Xu Z, et al. A colorization algorithm for vehicle infrared video[J]. Infrared Technology, 2010, 32(10): 595-600.
代中华, 孙韶媛, 许真, 等. 一种车载红外视频彩色化算法[J]. 红外技术, 2010, 32(10): 595-600.

【4】Do M N, Vetterli M. The contourlet transform: an efficient directional multiresolution image representation[J]. IEEE Transactions on Image Processing, 2005, 14(12): 2091-2106.

【5】Weyrich N, Warhola G T. Wavelet shrinkage and generalized cross validation for image denoising[J]. IEEE Transactions on Image Processing, 1998, 7(1): 82-90.

【6】Qiao S, Sun S Y, Gu X J, et al. Vehicle infrared video colorization based on contour tracking[J]. Journal of Applied Optics, 2012, 33(4): 727-732.
谯帅, 孙韶媛, 谷小婧, 等. 基于轮廓跟踪的车载红外视频彩色化方法[J]. 应用光学, 2012, 33(4): 727-732.

【7】Gu X, Sun S, Fang J, et al. Kernel based color estimation for night vision imagery[J]. Optics Communications, 2012, 285(7): 1697-1703.

【8】GoodfellowI J, Pougetabadie J, Mirza M, et al. Generative adversarial networks[J]. Advances in Neural Information Processing Systems, 2014, 3: 2672-2680.

【9】Isola P, Zhu J Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks[EB/OL]. (2017-11-22)[2018-03-01]. https:∥arxiv.org/abs/1611.07004.

【10】Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[EB/OL]. (2018-02-19)[2018-03-01]. https:∥arxiv.org/abs/1703.10593.

【11】Yao G S, Sun S Y, Fang J A, et al. Depth estimation of night driverless vehicle scene based on infrared and radar[J]. Laser & Optoelectronics Progress, 2017, 54(12): 121003.
姚广顺, 孙韶媛, 方建安, 等. 基于红外与雷达的夜间无人车场景深度估计[J]. 激光与光电子学进展, 2017, 54(12): 121003.

【12】Wu S C, Zhao H T, Sun S Y. Depth estimation from monocular infrared video based on bi-recursive convolutional neural network[J]. Acta Optica Sinica, 2017, 37(12): 1215003.
吴寿川, 赵海涛, 孙韶媛. 基于双向递归卷积神经网络的单目红外视频深度估计[J]. 光学学报, 2017, 37(12): 1215003.

【13】Li Z B, Guo C, Zhao P F, et al. Mode instability mitigation by counter-pumped scheme in high power fiber laser[J]. Chinese Journal of Lasers, 2017, 44(8): 0801010.
李泽标, 郭超, 赵鹏飞, 等. 高功率光纤激光器中反向抽运结构对模式不稳定的抑制[J]. 中国激光, 2017, 44(8): 0801010.

【14】Pal N R. On minimum cross-entropy thresholding[J]. Pattern Recognition, 1996, 29(4): 575-580

引用该论文

Li Jiahao,Sun Shaoyuan,Wu Xueping,Li Dawei. Unmanned Vehicle Night Infrared Video Colorization Based on Dual-Channel Cycle-Consistent Adversarial Networks[J]. Laser & Optoelectronics Progress, 2018, 55(9): 091505

李佳豪,孙韶媛,吴雪平,李大威. 基于双通道循环生成对抗网络的无人车夜视红外视频彩色化[J]. 激光与光电子学进展, 2018, 55(9): 091505

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF