生成式对抗神经网络的多帧红外图像超分辨率重建

生成式对抗神经网络在约束图像生成表现出了巨大潜力, 使得其适合运用于图像超分辨率重建。但是使用生成式对抗神经网络重建后的超分辨率图像存在过度平滑, 缺少高频细节信息的缺点。针对单帧图像超分辨率重建方法不能有效利用图像序列间的时间-空间相关性的问题, 提出了一种基于生成式对抗神经网络的多帧红外图像超分辨率重建方法(M-GANs)。首先, 对低分辨率图像序列进行运动补偿; 其次, 使用权值表示卷积层对运动补偿后的图像序列进行权值转换计算; 最后, 将其输入生成式对抗重建网络, 输出重建后的高分辨率图像。实验结果表明: 文中方法在主观及客观评价中均优于当前代表性的超分辨率重建方法。

Abstract

Generative adversarial networks had shown promising potential in conditional image generation. It seemed that the GANs were particularly suitable for use in image super-resolution reconstruction. However, there was a shortcoming of excessive smoothness and lack of high frequency detail information for the reconstructed SR images by using GANs. Aiming at resolving the problem that the method of single image super-resolution reconstruction ignored the spatio-temporal relationship between image frames, a method of multiframe infrared image super-resolution reconstruction based on generative adversarial networks (M-GANs) was proposed in this paper. Firstly, motion compensation was proposed for registration low resolution image frames; Secondly, a weight representation convolutional layer was performed to calculate the weight transfer; Finally, the generative adversarial network was used to reconstruct the high resolution image. Experimental results demonstrate that the proposed method surpass current state-of-the-art performance of both subjective and objective evaluation.infrared imaging

参考文献

[1] Chandra M S, Rajan K, Srinivasan R. Locally adaptive regularization for robust multiframe super resolution reconstruction[J]. Advances in Computer Science, Eng & Appl, 2012, AISC166: 223-234.

[2] Xiao C, Yu J, Xue Y. A high-efficiency super-resolution reconstruction algorithm from image/video sequences[C]//Third International IEEE Conference on Signal-Image Technologies and Internet-Based System, 2008: 573-581.

[3] Luo W, Zhang Y, Feizi A, et al. Pixel super-resolution using wavelength scanning[J]. Light: Science & Application, 2016, 5(4): e16060.

[4] Zhaoghao K, Chen L, Yang X S, et al. Super-resolution dipole orientation mapping via polarization demodulation[J]. Light: Science & Application, 2016, 5(10): e16166.

[5] Ashikaga H, Estner H L, Herzka D A, et al. Quantitative assessment of single-image super-resolution in myocardical scar imaging[J]. Medical Imaging and Diagnostic Radiology, 2014, 2(1): 1-12.

[6] Lei Sen, Shi Zhenwei, Zou Zhengxia. Super-resolution for remote sensing images via local-global combined network[J].IEEE Geosciences and Remote Sensing Letters, 2017, 14(8): 1243-1247.

[7] Pan Z, Yu J, Huang H, et al. Super-resolution based on compressive sensing and structural self-similarity for remote sensing images[J]. IEEE Transactions Geosci and Remote Sensing Letters, 2013, 51(9): 4864-4876.

[8] Zhou F, Yang W M, Liao Q M. Interpolation-based image super-resolution using multisurface fitting[J]. IEEE Transactions on Image Processing, 2012, 21(7): 3312-3318.

[9] Freedman G, Fattal R. Image and video upscaling from local self-examples[J]. TOG, 2011, 28(2): 1-11.

[10] Liu Xueting, Song Daojin, Dong Chuandai, et al. MAP-based image superresolution reconstructionscience[J]. International Journal of Computer Science Engineering, 2008, 2(1): dai: 10.1999/1307-6892/2034.

[11] Simon Baker, Takeo Kanad. Limits on super-resolution and how to break them[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(9): 1167-1183.

[12] Marcia L S A, Neison D A. Mascarenhas, generalization of iterative restoration techniques for super-resolution[C]// SIBGRAPI Conference on Graphics, Patterns and Images, 2011: 258-265.

[13] Yang J C, Wright J, Huang T S, et al. Image super-resolution via sparse representation[J]. IEEE Signal Processing Magazine, 2010, 19(11): 2861-2873.

[14] Timofte R, Smet V D, Gool L V. Anchored neighborhood regression for fast example-based super-resolution[C]// IEEE International Conference on Computer Vision, 2013: 1920-1927.

[15] Wang Xinwei, Cao Yinan, Lu Dezhen, et al. Spatial difference shaping to improve range resolution in 3D super-resolution range-gated imaging[C]//2015 Inernational Conference on Optical Instruments and Technology: Optoelectronic Imaging and Processing Technology, 2015, 9622: 1-8.

[16] Zhang Di, He Jiazhong. Hybrid sparse-representation-based approach to image super-resolution reconstruction[J]. Journal of Electronic Imaging, 2017, 26(2): 02308.

[17] Dong Chao, Chen Change Loy, He Kaiming, et al. Image super-resolution using deep convolutianal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307.

[18] Kappeler A, Yoo Seunghwan, Dai Qiqin, et al. Video super-resolution with convolutional neural networks[J]. IEEE Transactions on Computational Imaging, 2016, 2(2): 109-122.

[19] Shi Wenzhe, Caballero J, Huszar F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2016: 1874-1883.

[20] Caballero J, Ledig C, Aitken A, et al. Real-time video super-resolution with spatio-temporal networks and motion compensation[C]//Computer Vision and Pattern Recognition, 2017, arXiv preprint arXiv: 1611. 05250v2.

[21] Ledig C, Theis L, Huszar F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Computer Vision and Pattern Recognition, 2016, arXiv preprint arXiv: 1609. 04802v1.

[22] Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks[C]//Machine Learning, 2014, arXiv preprint arXiv: 1406. 2661vl.

李方彪, 何昕, 魏仲慧, 何家维, 何丁龙. 生成式对抗神经网络的多帧红外图像超分辨率重建[J]. 红外与激光工程, 2018, 47(2): 0203003. Li Fangbiao, He Xin, Wei Zhonghui, He Jiawei, He Dinglong. Multiframe infrared image super-resolution reconstruction using generative adversarial networks[J]. Infrared and Laser Engineering, 2018, 47(2): 0203003.

生成式对抗神经网络的多帧红外图像超分辨率重建

关于本站 Cookie 的使用提示

全站搜索

生成式对抗神经网络的多帧红外图像超分辨率重建

相关论文

相关资讯

关于本站 Cookie 的使用提示

全站搜索