首页 > 论文 > 光学学报 > 40卷 > 20期(pp:2010001--1)

基于改进WGAN-GP的多波段图像同步超分与融合方法

Multi-Band Image Synchronous Super-Resolution and Fusion Method Based on Improved WGAN-GP

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

针对低分辨率源图像的融合结果质量低下不利于后续目标提取的问题,提出一种基于梯度惩罚Wasserstein生成对抗网络(WGAN-GP)的多波段图像同步超分与融合方法。首先,基于双三次插值法将多波段低分辨率源图像分别放大至目标尺寸;其次,将该放大结果输入特征提取(编码)网络分别提取特征并在高层特征空间进行组合;然后,通过解码网络重构出初步融合图像;最后,经过生成器和判别器的动态博弈得到高分辨率的融合图像。实验结果表明,所提方法不仅可以同步实现多波段图像的超分和融合,而且融合图像的信息量、清晰度和视觉质量明显高于其他代表性方法。

Abstract

Aiming at the problem that the fused results of low resolution source images are not good for the subsequent target extraction, a multi-band image synchronous super-resolution and fusion method based on Wasserstein generative adversarial network with gradient penalty (WGAN-GP) is proposed. Firstly, the multi-band low-resolution source images are enlarged to the target size respectively based on the bicubic interpolation method. Secondly, the enlarged results are input to a feature extraction (encoding) network to extract features respectively and combine them in a high-level feature space. Then, the initial fused images are reconstructed by decoding network. Finally, a high-resolution fused image is obtained through a dynamic game between the generator and the discriminator. The experimental results show that the proposed method can not only achieve multi-band images super-resolution and fusion simultaneously, but also the information amount, clarity, and visual quality of the fused images are significantly higher than other representative methods.

广告组1 - 空间光调制器+DMD
补充资料

中图分类号:TP391

DOI:10.3788/AOS202040.2010001

所属栏目:图像处理

基金项目:山西省应用基础研究项目、山西省研究生创新项目、中北大学第十六届研究生科技立项;

收稿日期:2020-04-09

修改稿日期:2020-07-03

网络出版日期:2020-10-01

作者单位    点击查看

田嵩旺:中北大学大数据学院, 山西 太原 030051
蔺素珍:中北大学大数据学院, 山西 太原 030051
雷海卫:中北大学大数据学院, 山西 太原 030051
李大威:中北大学大数据学院, 山西 太原 030051
王丽芳:中北大学大数据学院, 山西 太原 030051

联系人作者:蔺素珍(lsz@nuc.edu.cn); 雷海卫(lsz@nuc.edu.cn);

备注:山西省应用基础研究项目、山西省研究生创新项目、中北大学第十六届研究生科技立项;

【1】Zhou X L, Jiang Z T. Infrared and visible image fusion combining pulse-coupled neural network and guided filtering [J]. Acta Optica Sinica. 2019, 39(11): 1110003.
周哓玲, 江泽涛. 结合脉冲耦合神经网络与引导滤波的红外与可见光图像融合 [J]. 光学学报. 2019, 39(11): 1110003.

【2】Yan H B, Li Z M. Infrared and visual image fusion based on multi-scale feature decomposition [J]. Optik. 2020, 203: 163900.Yan H B, Li Z M. Infrared and visual image fusion based on multi-scale feature decomposition [J]. Optik. 2020, 203: 163900.

【3】Wang W Q, Gao Y D, Liu H, et al. Infrared and visible image sequence fusion via low-rank and sparse representation [J]. Journal of Xi''''an University of Technology. 2019, 35(3): 276-283.
王文卿, 高钰迪, 刘涵, 等. 基于低秩稀疏表示的红外与可见光图像序列融合方法 [J]. 西安理工大学学报. 2019, 35(3): 276-283.
Wang W Q, Gao Y D, Liu H, et al. Infrared and visible image sequence fusion via low-rank and sparse representation [J]. Journal of Xi''''an University of Technology. 2019, 35(3): 276-283.
王文卿, 高钰迪, 刘涵, 等. 基于低秩稀疏表示的红外与可见光图像序列融合方法 [J]. 西安理工大学学报. 2019, 35(3): 276-283.

【4】Zhang L B, Zhang J. A new adaptive fusion method based on saliency analysis for remote sensing images [J]. Chinese Journal of Lasers. 2015, 42(1): 0114001.
张立保, 章珏. 基于显著性分析的自适应遥感图像融合 [J]. 中国激光. 2015, 42(1): 0114001.

【5】Liu Y, Chen X, Cheng J, et al. Infrared and visible image fusion with convolutional neural networks [J]. International Journal of Wavelets, Multiresolution and Information Processing. 2017, 16(3): 1850018.

【6】Ma J Y, Yu W, Liang P W, et al. FusionGAN: a generative adversarial network for infrared and visible image fusion [J]. Information Fusion. 2019, 48: 11-26.

【7】Li H, Wu X J. DenseFuse: a fusion approach to infrared and visible images [J]. IEEE Transactions on Image Processing. 2019, 28(5): 2614-2623.

【8】Lin S Z, Han Z. Images fusion based on deep stack convolutional neural network [J]. Chinese Journal of Computers. 2017, 40(11): 2506-2518.
蔺素珍, 韩泽. 基于深度堆叠卷积神经网络的图像融合 [J]. 计算机学报. 2017, 40(11): 2506-2518.

【9】Lin S Z, Han Z, Li D W, et al. Integrating model- and data-driven methods for synchronous adaptive multi-band image fusion [J]. Information Fusion. 2020, 54: 145-160.

【10】Lyu Q, Shan H M, Wang G. Multi-contrast super-resolution MRI through a progressive network [EB/OL]. 2019.Lyu Q, Shan H M, Wang G. Multi-contrast super-resolution MRI through a progressive network [EB/OL]. 2019.

【11】Dong C, Loy C C, He K M, et al. Image super-resolution using deep convolutional networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2016, 38(2): 295-307.

【12】Zhang Y L, Tian Y P, Kong Y, et al. Residual dense network for image super-resolution[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. New York: , 2018, 2472-2481.

【13】Sajjadi M S M, Sch?lkopf B, Hirsch M. EnhanceNet: , 2017, 4501-4510.

【14】Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. New York: , 2017, 105-114.

【15】Zheng H, Zeng K, Guo D, et al. Multi-contrast brain MRI image super-resolution with gradient-guided edge enhancement [J]. IEEE Access. 2018, 6: 57856-57867.

【16】Zheng H, Qu X B, Bai Z J, et al. Multi-contrast brain magnetic resonance image super-resolution using the local weight similarity [J]. BMC Medical Imaging. 2017, 17(1): 1-13.

【17】Zhong J Y, Yang B, Li Y H, et al. Image fusion and super-resolution with convolutional neural network [M]. Singapore: Springer Singapore. 2016, 78-88.

【18】Chen L C, Zhu Y K, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[M]. Cham: , 2018, 833-851.

【19】Lei B Y, Huang S, Li R, et al. Segmentation of breast anatomy for automated whole breast ultrasound images with boundary regularized convolutional encoder-decoder network [J]. Neurocomputing. 2018, 321: 178-186.

【20】Gulrajani I, Ahmed F, Arjovsky M, et al. -12-25)[2020-05-14] . 2017, org/abs/1704: 00028.Gulrajani I, Ahmed F, Arjovsky M, et al. -12-25)[2020-05-14] . 2017, org/abs/1704: 00028.

【21】Han Z. Research on multi-band image adaptive fusion method based on deep learning [D]. Taiyuan: North University of China. 2018.
韩泽. 基于深度学习的多波段图像自适应融合方法研究 [D]. 太原: 中北大学. 2018.

【22】Sun Y W, Li L T, Cong P, et al. Super-resolution method for radiation image based on deep learning [J]. Atomic Energy Science and Technology. 2017, 51(5): 890-895.
孙跃文, 李立涛, 丛鹏, 等. 基于深度学习的辐射图像超分辨率重建方法 [J]. 原子能科学技术. 2017, 51(5): 890-895.

【23】Wang W, Zhang T, Wang X. Survey of single image super-resolution based on deep learning [J]. Journal of Chinese Computer Systems. 2019, 40(9): 1891-1896.
王威, 张彤, 王新. 用于图像超分辨率重构的深度学习方法综述 [J]. 小型微型计算机系统. 2019, 40(9): 1891-1896.

【24】Toet A, Hogervorst M A, Pinkus A R. The TRICLOBS dynamic multi-band image data set for the development and evaluation of image fusion methods [J]. Plos One. 2016, 11(12): e0165016.

引用该论文

Tian Songwang,Lin Suzhen,Lei Haiwei,Li Dawei,Wang Lifang. Multi-Band Image Synchronous Super-Resolution and Fusion Method Based on Improved WGAN-GP[J]. Acta Optica Sinica, 2020, 40(20): 2010001

田嵩旺,蔺素珍,雷海卫,李大威,王丽芳. 基于改进WGAN-GP的多波段图像同步超分与融合方法[J]. 光学学报, 2020, 40(20): 2010001

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF