首页 > 论文 > 激光与光电子学进展 > 56卷 > 23期(pp:231010--1)

双判别器生成对抗网络图像的超分辨率重建方法

Image Super-Resolution Reconstruction Method Using Dual Discriminator Based on Generative Adversarial Networks

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

提出一种可用于改进图像超分辨率重建质量的双判别器超分辨率重建网络(DDSRRN)。该网络在生成式对抗网络(GAN)的基础上增加一个判别器,将Kullback-Leibler(KL)和反向KL散度组合成一个统一的目标函数来训练两个判别器,利用这两种散度的互补统计特性,能在多模式下分散预估计密度,从而避免重建过程中网络模型的崩溃问题,提高模型训练的稳定性。针对模型损失函数的设计部分,首先使用Charbonnier损失函数来构建内容损失,利用网络中间层的特征信息来设计感知损失和风格损失,最后为缩减图像重建时间,在网络结构中引入反卷积来完成图像重建操作。实验结果表明本文方法在主观视觉上具有丰富的细节,获得了更好的主观视觉评价和客观量化评价,网络泛化能力好。

Abstract

In this study, we propose a dual discriminator super-resolution reconstruction network (DDSRRN) that can improve the super-resolution reconstruction quality of images. By adding a discriminator based on generative adversarial networks, the DDSRRN combines the Kullback-Leibler (KL) divergence and reverse KL divergence into a unified objective function for training two discriminators. Thus, the complementary statistical properties obtained from these divergences can be exploited to effectively diversify the pre-estimated density under multiple modes. Additionally, model collapse is effectively avoided during the reconstruction process, and the model training stability is improved. The model loss function can be designed based on the Charbonnier loss function to estimate the content loss. Furthermore, the intermediate features of the network are used to design the perceptual loss and style loss. Finally, a deconvolution layer is designed to reconstruct the super-resolution images, thereby reducing the image reconstruction time. The proposed method is experimentally demonstrated to provide abundant details. Thus, the proposed method exhibits good generalization ability and obtains improved subjective visual evaluation and objective quantitative evaluation.

Newport宣传-MKS新实验室计划
补充资料

DOI:10.3788/LOP56.231010

所属栏目:图像处理

基金项目:国家自然科学基金、云南省万人计划青年拔尖人才项目;

收稿日期:2019-05-13

修改稿日期:2019-06-03

网络出版日期:2019-12-01

作者单位    点击查看

袁飘逸:云南师范大学信息学院, 云南 昆明 650500
张亚萍:云南师范大学信息学院, 云南 昆明 650500

联系人作者:袁飘逸(1970915834@qq.com); 张亚萍(zhangyp@ynnu.edu.cn);

备注:国家自然科学基金、云南省万人计划青年拔尖人才项目;

【1】Park S C, Park M K, Kang M G. Super-resolution image reconstruction: a technical overview [J]. IEEE Signal Processing Magazine. 2003, 20(3): 21-36.

【2】632: XIV [M]. Subhasis C. Super-resolution imaging. The springer international series in engineering, computer science. New York: Springer Science+Business Media. 2001, 280.

【3】Dong C, Loy C C, He K M, et al. Learning a deep convolutional network for image super-resolution [M]. ∥Fleet D, Pajdla T, Schiele B, et al. Computer vision-ECCV 2014. Lecture notes in computer science. Cham: Springer. 2014, 8692: 184-199.

【4】Kim J, Lee J K, Lee K M. Deeply-recursive convolutional network for image super-resolution . [C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 26-July 1, 2016, Las Vegas, Nevada. New York: IEEE. 2016, 1637-1645.

【5】Simonyan K. -04-10)[2019-05-12] . https:∥arxiv. 2015, org/abs/1409: 1556.

【6】Shi W Z, Caballero J, Huszar F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network . [C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE. 2016, 1874-1883.

【7】Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution . [C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), July 21-26, 2017, Honolulu, HI, USA. New York: IEEE. 2017, 1132-1140.

【8】Lai W S, Huang J B, Ahuja N, et al. Deep Laplacian pyramid networks for fast and accurate super-resolution . [C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA. New York: IEEE. 2017, 5835-5843.

【9】S?nderby C K, Caballero J, Theis L, et al. -02-21)[2019-05-12] . https:∥arxiv. 2017, org/abs/1610: 04490.

【10】Dahl R, Norouzi M, Shlens J. Pixel recursive super resolution . [C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA. New York: IEEE. 2017, 5439-5448.

【11】Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets . [C]∥Proceedings of the 27th International Conference on Neural Information Processing Systems, December 4-9, 2017, Long Beach, CA, USA. Canada: NIPS. 2017.

【12】Ledig C, Theis L, Huszar F, et al. Photo-realistic single image super-resolution using a generative adversarial network . [C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA. New York: IEEE. 2017, 4681-4690.

【13】Wang X T, Yu K, Wu S X, et al. ESRGAN: enhanced super-resolution generative adversarial networks [M]. ∥Leal-Taixé L, Roth S. Computer vision-ECCV 2018 Workshops. Lecture notes in computer science. Cham: Springer. 2019, 11133: 63-79.

【14】Nguyen T D, Le T, Vu H, et al. Dual discriminator generative adversarial nets . [C]∥Proceedings of the 27th International Conference on Neural Information Processing Systems, December 4-9, 2017, Long Beach, CA, USA. Canada: NIPS. 2017.

【15】He K M, Zhang X Y, Ren S Q, et al. Identity mappings in deep residual networks [M]. ∥Leibe B, Matas J, Sebe N, et al. Computer vision-ECCV 2016. Lecture notes in computer science. Cham: Springer. 2016, 9908: 630-645.

【16】Gulrajani I, Ahmed F, Arjovsky M, et al. Improved training of Wasserstein GANs . [C]∥Proceedings of the 27th International Conference on Neural Information Processing Systems, December 4-9, 2017, Long Beach, CA, USA. Canada: NIPS. 2017.

【17】Wang Z, Bovik A C. Mean squared error: love it or leave it? A new look at signal fidelity measures [J]. IEEE Signal Processing Magazine. 2009, 26(1): 98-117.

【18】Zhao H, Gallo O, Frosio I, et al. Loss functions for image restoration with neural networks [J]. IEEE Transactions on Computational Imaging. 2017, 3(1): 47-57.

【19】Bruhn A, Weickert J, Schn?rr C. Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods [J]. International Journal of Computer Vision. 2005, 61(3): 211-231.

【20】Wu B Z, Duan H D, Liu Z C, et al. -12-20)[2019-05-12] . https:∥arxiv. 2017, org/abs/1712: 05927.

【21】Liu G L, Reda F A, Shih K J, et al. Image inpainting for irregular holes using partial convolutions [M]. ∥Ferrari V, Hebert M, Sminchisescu C, et al. Computer vision-ECCV 2018. Lecture notes in computer science. Cham: Springer. 2018, 11215: 89-105.

【22】Agustsson E, Timofte R. NTIRE 2017 challenge on single image super-resolution: dataset and study . [C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), July 21-26, 2017, Honolulu, HI, USA. New York: IEEE. 2017, 1122-1131.

【23】Bevilacqua M, Roumy A, Guillemot C, et al. Neighbor embedding based single-image super-resolution using Semi-Nonnegative Matrix Factorization . [C]∥2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), March 25-30, 2012, Kyoto, Japan. New York: IEEE. 2012, 1289-1292.

【24】Yuan Y, Liu S Y, Zhang J W, et al. Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks . [C]∥2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 18-22, 2018, Salt Lake City, Utah. New York: IEEE. 2018, 814-823.

【25】Martin D, Fowlkes C, Tal D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics . [C]∥Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, July 7-14, 2001, Vancouver, BC, Canada. New York: IEEE. 2001, 416-423.

【26】Zhang Y L, Tian Y P, Kong Y, et al. -12-25)[2019-05-12] . https:∥arxiv. 2018, org/abs/1812: 10477.

引用该论文

Yuan Piaoyi,Zhang Yaping. Image Super-Resolution Reconstruction Method Using Dual Discriminator Based on Generative Adversarial Networks[J]. Laser & Optoelectronics Progress, 2019, 56(23): 231010

袁飘逸,张亚萍. 双判别器生成对抗网络图像的超分辨率重建方法[J]. 激光与光电子学进展, 2019, 56(23): 231010

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF