激光与光电子学进展, 2015, 52 (2): 020004, 网络出版: 2015-01-29   

图像超分辨率复原方法及应用 下载: 1314次

Methods and Applications of Image Super-Resolution Restoration
陈健 1,2,3,*高慧斌 1王伟国 1毕寻 1,2
作者单位
1 中国科学院长春光学精密机械与物理研究所, 吉林 长春 130033
2 中国科学院大学, 北京 100049
3 吉林大学通信工程学院, 吉林 长春 130012
摘要
介绍了超分辨率复原方法的概念和理论基础,重点总结了一些常用的超分辨率复原理论的国内外研究现状,并对它们的理论依据、优缺点和适用范围进行了详尽分析,对超分辨率复原理论的应用领域进行了介绍。超分辨率复原方法分为频域法和空域法。频域复原法原理简单清楚,计算方便,但是所建立的运动模型都是平移模型,不具有一般性,同时难以利用正则化约束,这就导致难以使用图像的先验信息来帮助进行超分辨率复原。空域复原法可以很方便地建立复杂的运动模型,同时考虑了几乎所有的图像降质因素,例如噪声、降采样、由非零孔径时间造成的模糊、光学系统降质和运动模糊等,还可以加入更完善的先验知识,相比于频域复原法,空域超分辨率复原模型更符合实际的图像退化过程,是目前应用最广泛的一类超分辨率复原方法。
Abstract
The basic concepts and theories of super-resolution restoration method is introduced. Some applications focused on common method of super-resolution restoration is summarized. Their theoretical basis, advantages and disadvantages, and scope of applications are exhaustively analyzed. The applications of super-resolution restoration theory is introduced. Overall, the super-resolution restoration methods are divided into frequency domain method and space domain method. Frequency domain recovery method is simple in principle and easy in calculation. But its motion model is shift model have no generality. Meanwhile it is difficult to use the priori information of the image to help super-resolution restoration. Space domain recovery method can be easily taken degradation and motion blur. More perfect priori knowledge is added. Compared with the frequency domain method, space domain superresolution restoration model is more close to actual degradation processes and currently the most widely used superresolution restoration method.
参考文献

[1] M R Banham, A K Katsaggelos, et al.. Digital image restoration[J]. IEEE Signal Processing Magazine, 1997, 14(2): 24-41

[2] H Trussel, M Civanlar. Feasible solution in signal restoration[J]. IEEE Trans Accoust Speech Signal Processing, 1984, ASSP-32: 201-212.

[3] B R Hunt. Super-resolution of images: Algorithms, principles, performance[J]. International J Imaging Systems Technol, 1995, 6(4): 297-304.

[4] X Gao, K Zhang, D Tao, et al.. Joint learning for single-image super-resolution via a coupled constraint[J]. IEEE Trans Image Process, 2012, 21(2): 469-480.

[5] H He, W C Siu. Single image super-resolution using Gaussian process regression[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2011. 449-456.

[6] K I Kim, Y Kwon. Single-image super-resolution using sparse regression and natural image prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(6): 1127-1133.

[7] Y W Tai, S Liu, M S Brown, et al.. Super resolution using edge prior and single image detail synthesis[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2010. 2400-2407.

[8] F Ahmed, S C Gutafsou, M A Karim. High-fidelity image interpolation using radial basis function neural networks[C]. Proc IEEE National Aerospace and Electronics Conference, 1995, 2: 588-592.

[9] N Plaziac. Image interpolation using neural networks[J]. IEEE Trans on image Processing, 1999, 8(11): 1647-1651.

[10] F M Candlneia, J C Principle. Super resolution of images based on local correlations[J]. IEEE Trans Neural Networks, 1999, 10(2): 372-380.

[11] R R Schultz, R L Stevenson. A Bayesian approach to image expansion for improved definition[J]. IEEE Trans on Image Processing, 1994, 3(3): 233-242.

[12] X Li, M T Orchard. New edge-directed interpolation[J]. IEEE Trans Image Processing, 2001, 10(10): 1521-1527.

[13] R Zeyde, M Elad, M Protter. On Single Image Scale-Up Using Sparse-Representations[M]. Berlin Heidelberg: Springer Press, 2012. 711-730.

[14] D Glasner, S Bagon, M Irani. Super-resolution from a single image[C]. IEEE Conference on Computer Vision, 2009. 349-356.

[15] M K Ng, N K Bose. Mathematical analysis of super-resolution methodology[J]. IEEE Signal Processing Magazine, 2003, 20(3): 62-74.

[16] N K Bose. Multi-Dimensional Systems Theory and App1ications[M]. Holland: Kluwer Academic Publishers Press, 2003.

[17] R Y Tsai, T Huang. Multi frame image restoration and registration[J]. Advances in Computer Vision and Image Processing, 1984, 1(2): 317-339.

[18] A M Tekalp, M K Ozkan, M L Sezan. High-resolution image reconstruction for lower-resolution image restoration[C]. Proceedings of the IEEE international Conference on Acoustics, Speech and Signal Processing, 1992, 3: 169-172.

[19] S P Kim, W Y Su. Recursive high- resolution reconstruction of blurred multi- frame images[J]. IEEE Trans Image Processing, 1993, 2(4): 534-539.

[20] M Elad, A Feuer. Super- resolution restoration of an image sequence: adaptive filtering approach[J]. IEEE Trans Image Processing, 1999, 8(3): 387-395.

[21] C E Davila. Efficient recursive total least squares algorithms for FIR adaptive filtering[J]. IEEE Trans Signal Processing, 1994, 42(2): 268-280.

[22] N X Ngugen. Numerical Algorithm for Image Super Restoration[D]. California: Stanford University, 2000.

[23] 周芳. 图像超分辨率复原技术的现状与展望[J]. 自动化与仪表, 2006, (1): 10-14.

    Zhou Fang. A review of super-resolution image restoration[J]. Automation and Instrument, 2006, (1): 10-14.

[24] F M Candlneia, J C Principle. Super resolution of images based on local correlations[J]. IEEE Trans Neural Networks, 1999, 10(2): 372-380.

[25] 张新明, 沈兰荪. 超分辨率复原技术的发展[J]. 测控技术, 2002, 21(5): 33-35.

    Zhang Xinming, Shen Lansun. The development of super- resolution restoration from image sequence[J]. Control Technology, 2002, 21(5): 33-35.

[26] H Ur, D Gross. Improved resolution from sub-pixel shifted pictures[J]. CVGIP: Graphical Models and Image Processing, 1992, 54(2): 181-186.

[27] A Papoulis. Generalized sampling expansion[J]. IEEE Trans Circuits Syst, 1977, 24(11): 652-654.

[28] J L Brown. Multi-channel sampling of low pass signals[J]. IEEE Trans Circuits Syst, 1981, CAS-28: 101-106.

[29] R C Hardie, K J Barnard, E E Armstrong. Joint MAP registration and high-resolution image estimation using a sequence of under-sampled images[J]. IEEE Trans Image Processing, 1997, 6(12): 1621-1633.

[30] N Nguyen. Numerical Algorithms for Image Super-Resolution[D]. California:Stanford University, 2000.

[31] N Nguyen, P Milanfar. A wavelet-based interpolation-restoration method for super-resolution[J]. Circuits Systems Signal Process, 2000, 19(4): 321-338.

[32] S Lertrattanapanich. Super- Resolution from Degraded Image Sequence Using Spatial Tessellations and Wavelets[D]. University Park: Pennsylvania State University, 2003.

[33] A J Patti, M Sezan, A M Tekalp. A new motion compensated reduced order model kalman filter for space- varying restoration of progressive and interlaced video[J]. IEEE Trans Image Processing, 1998, 7(4): 543-554.

[34] M Elad, A Feuer. Super- resolution restoration of image sequence[J]. IEEE Trans Pattern Anal. Machine Intelligence, 1999, 21(9): 817-834.

[35] M Elad, A Feuer. Super-resolution reconstruction of continuous image sequences[C]. International Conference on Image Processing, Kobe, Japan, 1999. 459-463.

[36] M S Alam, J G Bognar, R C Hardie, et al.. Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames[J]. IEEE Trans Instrumentation Measurement, 2000, 49(5): 915-923.

[37] B R Frieden, H G Aumann. Image reconstruction from multiple 11-D scans using filtered localized projection[J]. Appl Optics, 1987, 26(3): 223-226.

[38] M Irani, S Peleg. Improving resolution by image registration[J]. CVGIP: Graphical Models and Image Processing, 1991, 53(3): 231-239.

[39] 郭伟伟, 章品正. 基于迭代反投影的超分辨率图像重建[J]. 计算机科学与探索, 2009, 3(3): 321-329.

    Guo Weiwei, Zhang Pinzheng. Super-resolution image reconstruction with iterative back projection algorithm[J]. Journal of Frontiers of Computer Science and Technology, 2009, 3(3): 321-329.

[40] 覃凤清, 何小海, 陈为龙, 等. 一种基于子像素配准视频超分辨率重建方法[J]. 光电子·激光, 2009, 20(7): 972-976.

    Tan Fengqing, He Xiaohai, Chen Weilong, et al.. A video super-resolution reconstruction method based on sub-pixel registration[J]. Journal of Optoelectronics·Laser, 2009, 20(7): 972-976.

[41] 张永育, 李翠华, 余礼钹, 等. 基于Keren 改进配准算法的IBP 超分率重建[J]. 厦门大学学报(自然科学版), 2012, 51(4): 686-69.

    Zhang Yongyu, Li Cuihua, Yu Liba, et al.. IBP super-resolution reconstruction based on improvement approach of Keren registration method[J]. Journal of Xiamen University, 2012, 51(4): 686-69.

[42] S Maan, R W Picard. Virtual bellows: Constructing high quality stills from video[C]. Proc of International Conference on Image Processing, Austin, TX, 1994, 1: 363-367.

[43] B C Tom, A K Katsaggelos. Resolution enhancement of video sequences using motion compensation[C]. Proc of IEEE Int. Conf Image Processing, Lausanne, Switzerland, 1996, 1: 713-716.

[44] D C Youla, H Webb. Image restoration by the method of convex projections: Part I, theory[J]. IEEE Transactions on Medical Imaging, 1982, 1(2): 81-94.

[45] H Stark, P Oskoui. High resolution image recovery from image-plane arrays, using convex projection[J]. JOSA A, 1989, 6(11): 1715-1726.

[46] M Tekalp, M K Ozkan, M I Sezan. High-resolution image reconstruction from lower-resolution image sequences and space varying image restoration[C]. IEEE international Conference on Acoustics, Speech and Signal Processing, 1992, 3: 169-172.

[47] B K Gunturk, Y Altunbasak, R M Mersereau. Super- resolution reconstruction of compressed video using transformdomain statistics[J]. IEEE Trans Image Processing, 2004, 13(1): 33-43.

[48] 黄华, 孔玲莉, 齐春, 等. 基于凸集投影和线过程模型的超分辨率图像重建[J]. 西安交通大学学报, 2003, 37(10): 1059-1062.

    Huang Hua, Kong Lingli, Qi Chun, et al.. Super-resolution image reconstruction based on projections onto convex sets and line process modeling[J]. Journal of Xi′an Jiaotong University, 2003, 37(10): 1059-1062.

[49] 朱翔, 袁杰, 都思丹. 基于JPEG 序列的图像重建[J]. 电子与信息学报, 2007, 29(8): 1841-1844.

    Zhu Xiang, Yuan Jie, Du Sidan. Recovery of JPEG compressed image sequence[J]. Journal of Electronics and Information Technology, 2007, 29(8): 1841-1844.

[50] 肖创柏, 段娟, 禹晶. 序列图像的POCS 超分辨率重建方法[J]. 北京工业大学学报, 2009, 35(1): 108-113.

    Xiao Chuangbai, Duan Juan, Yu Jing. POCS super-resolution reconstruction from image sequences[J]. Journal of Beijing University of Technology, 2009, 35(1): 108-113.

[51] 张现, 徐昆, 李勇. 基于外存和凸集投影法的遥感图像超分辨率方法[J]. 清华大学学报(自然科学版), 2010, 50(10): 1743-1746.

    Zhang Xian, Xu Kun, Li Yong. Remote sensing image super-resolution based on POCS and out-of-core[J]. Journal of Tsinghua University, 2010, 50(10): 1743-1746.

[52] 张砚, 李先颖, 满益云. 基于凸集投影法和复数小波包域的遥感图像上釆样研究[J]. 计算机学报, 2011, 34(3): 482-488.

    Zhang Yan, Li Xianying, Man Yiyun. Remote sensing images upsampling based on projection onto convex sets and complex wavelet packet transfer[J]. Chinese Journal of Computers, 2011, 34(3): 482-488.

[53] J Patti, M Sezan, A M Tekalp. High-resolution image reconstruction from a low-resolution image sequence in the presence of time-varying motion blur[C]. Proc IEEE Int Conf Image Processing, Austin, TX, 1994, 1: 343-347.

[54] J Patti, M Sezan, A M Tekalp. Super-resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time[J]. IEEE Trans Image Processing, 1997, 6(8): 1064-1076.

[55] J Patti, M Sezan, A M Tekalp. Robust methods for high-quality stills from interlaced video in the presence of dominant motion[J]. IEEE Trans Circuits and Systems for Video technology, 1997, 7(2): 328-342.

[56] J Patti, Y Altunbasak. Artifact reduction for set theoretic super resolution image reconstruction with edge adaptive constraints and higher-order interpolants[J]. IEEE Trans Image Processing, 2001, 10(1): 179-186.

[57] B C Tom, A K Katsaggelos. An Iterative Algorithm for Improving the Resolution of Video Sequence[A]. Proc SPIE Conf. Visual Communication and Image Processing, Orlando, FL, 1995. 1430-1438.

[58] R R Sehultz, R L Stevenson. A Bayesian approach to image expansion for improved definition[J]. IEEE Transactions on Image Processing, 1994, 3(2): 233-242.

[59] R C Hardie, K J Bamard, E E Armstrong. Joint MAP registration high-resolution image estimation using a sequence of under sampled image[J]. IEEE Transactions on Image Processing, 1997, 6(12): 1621-1633.

[60] P Cheeseman, B Kanefsky. Super- resolved surface reconstruction from multiple images[R]. NASA Ames Research Center, Moffett Field, CA, Tech ReP FIA-94-12, 1994.

[61] B R Hunt, P J Sementilli. Description of a Poisson imagery super-resolution algorithm[C]. Astronomical Data Analysis Software and System I, California, USA, 1992, 25: 196-199.

[62] P J Sementilli, M S Nadar, B R Hunt. Poisson MAP super-resolution estimator with smoothness constraint[C]. Proceedings of SPIE Neural and Stochastic Methods in Image and Signal Processing II, 1993, 2032: 2-13.

[63] G K Chantas, N P Galatsanos, N A Woods. Super- resolution based on fast registration and maximum a posteriori reconstruction[J]. IEEE Transactions on Image Processing, 2007, 16(7): 1821-1830.

[64] S P Belekos, N P Galatsanos, A K Katsaggelos. Maximum a posteriori video super-resolution using a new multichannel image prior[J]. IEEE Transactions on Image Processing, 2010, 19(6): 1451-1464.

[65] L J Karam, N G Sadaka. An Efficient selective perceptual- based super- resolution estimator[J]. IEEE Trans Image Processing, 2011, 20(12): 3470-3481.

[66] D Wallach, F Lamare. Super-resolution in respiratory synchronized positron emission tomography[J]. IEEE Transactions on Medical Imaging, 2012, 31(2): 438-448.

[67] M Irani, S Peleg. Motion analysis for image enhancement, resolution, occlusion and transparency[J]. Journal of Visual Communication and Image Representation, 1993, 4(4): 324-336.

[68] M Irani, S Peleg. Super resolution from image sequences[C]. Piscataway, NJ, USA: Proceedings of international Conference on Pattern Recognition, 1990. 115-120.

[69] 鲜海莹, 傅志中, 万群, 等. 基于非冗余信息的超分辨率算法[J]. 电波科学学报, 2012, 27(2): 216-221.

    Xian Haiying, Fu Zhizhong, Wan Qun, et al.. Super resolution algorithm based on non- redundant information[J]. Chinese Journal of Radio Science, 2012, 27(2): 216-221.

[70] 王静, 章世平, 孙权森, 等. 基于MAP 估计的遥感图像频域校正超分辨率算法[J]. 东南大学学报, 2010, 40(1): 84-88.

    Wang Jing, Zhang Shiping, Sun Quansen, et al.. MAP based remote sensing image super- resolution with frequency domain correction[J]. Journal of Southeast University, 2010, 40(1): 84-88.

[71] 韩玉兵, 吴乐南. 基于自适应滤波的视频序列超分辨率重建[J]. 计算机学报, 2006, 29(4): 642-647.

    Han Yubing, Wu Lenan. Super resolution reconstruction of video sequence based on adaptive filter[J]. Chinese Journal of Computer, 2006, 29(4): 642-647.

[72] 韩华, 王洪剑, 彭思龙. 基于局部结构相似性的单幅图像超分辨率算法[J]. 计算机辅助设计与图形学学报, 2005, 17(5): 941-947.

    Han Hua, Wang Hongjian, Peng Silong. A new super-resolution algorithm for a single image based on local structure similarity[J]. Journal of Computer Aided Design and Computer Graphics, 2005, 17(5): 941-947.

[73] 陈华, 金伟其, 王霞, 等. 基于小波包分析的三维宽场显微图像复原方法[J]. 北京理工大学学报, 2006, 26(1): 72-75.

    Chen Hua, Jin Weiqi, Wang Xia, et al.. A method of restoration for the 3D image of wide- field microscope based on wavelet packet analysis denoising[J]. Transactions of Beijing Institute of Technology, 2006, 26(1): 72-75.

[74] M Elad, A Feuer. Restoration of a single super-resolution image from several blurrd, noisy and under sampled measured images[J]. IEEE Trans Image Processing, 1997, 6(12): 1646-1658.

[75] 苏秉华, 金伟其. 基于POCS-MPMAP 合成算法的超分辨率图像复原[J]. 光子学报, 2003, 32(4): 502-504.

    Su Binghua, Jing Weiqi. POCS-MPMAP based super-resolution image restoration[J]. Acta Photonica Sinica, 2003, 32(4): 502-504.

[76] D P Capel. Image Mosaicing and Super-Resolution[D]. London: University of Oxford, 2001.

[77] M Elad. On the bilateral fi1ter and ways to improve it[J]. IEEE Transactions on Image Processing, 2002, 11(10): 1141-1151.

[78] D Barash. Bilateral filtering and anisotropic diffusion: Towards a unified viewpoint[C]. Hewlett-Packard Laboratories Technical Report, 2000. 18.

[79] S Farsiu, M D Robinson. Fast and robust multiframe super resolution[J]. IEEE Transactions on Image Processing, 2004, 13(10): 1327-1344.

[80] C Tomasi, R Manduchi. Bilateral filtering for gray and color images[C]. Proc of the 6th international Conference on Computer Vision, 1998. 839-846.

[81] G Gilboa, N Soehen, Y Y Zeevi. Forward- and- backward diffusion processes for Adaptive image enhancement and denoising[J]. IEEE Trans Image Processing, 2002, 11(7): 689-703.

[82] E Shechtman, Y Caspi, M Irani. Space-time super-resolution[J]. IEEE Trans Pattern Analysis and Machine Intelligence, 2005, 27(4): 531-545.

[83] H Kim, K S Hong. Variational approaches to super-resolution with contrast enhancement and anisotropic diffusion[J]. J Electron Imaging, 2003, 12(2): 244-251.

[84] A Zomet, S Peleg. Multi-sensor super-resolution[C]. Proceedings of the IEEE Workshop on applications of computer Vision, 2001. 27-31.

[85] N Nguyen, P Milanfar, G Golub. A computationally efficient super resolution image Reconstruction algorithm[J]. IEEE Trans Image Processing, 2001, 10(4): 573-583.

[86] P D Wirawan, H Maitre. Multi-channel high resolution blind image restoration[C]. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, 1989. 3229-3232.

[87] S Baker, T Kanade. Limits on super-resolution and how to break them[C]. Proceedings of IEEE Conference Computer Vision and Patten Recognition, 2000. 372-379.

[88] Z C Lin, H Y Shum. Fundamental limits of reconstruction- based super- resolution algorithms under local translation [J]. IEEE Trans Pattern Analysis and Machine Intelligence, 2004, 26(1): 83-97.

[89] J Sun, Z Xu, H Y Shum. Image super-resolution using gradient profile prior[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2008. 1-8.

[90] T Tung, S Nobuhara, T Matsuyama. Simultaneous super-resolution and 3D video using graph-cuts[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2008. 1-8.

[91] K Nguyen, C Fookes, S Sridharan, et al.. Quality- driven super- resolution for less constrained iris recognition at a distance and on the move[J]. IEEE Trans Information Forensics and Security, 2011, 6(4): 248-1258.

[92] T K Nguyen, C B Fookes, S Sridharan, et al.. Feature- domain super- resolution for IRIS recognition[C]. Proceedings of The 18th International Conference on Image Processing, 2011. 3258-3261.

[93] C Liu, D Sun. A Bayesian approach to adaptive video super resolution[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2011. 209-216.

[94] S Wang, D Zhang, Y Liang, et al.. Semi- coupled dictionary learning with applications to image super- resolution and photo-sketch synthesis[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2012. 2216-2223.

[95] J Yang, Z Wang, Z Lin, et al.. Coupled dictionary training for image super-resolution[J]. IEEE Trans Image Processing, 2012, 21(8): 3467-3478.

[96] J Yang, J Wright, T S Huang, et al.. Image super-resolution via sparse representation[J]. IEEE Trans Image Processing, 2010, 19(11): 2861-2873.

[97] J Wang, S Zhu, Y Gong. Resolution enhancement based on learning the sparse association of image patches[J]. Pattern Recognition Letters, 2010, 31(1): 1-10.

[98] Y Hu, K M Lam, G Qiu, et al.. From local pixel structure to global image super- resolution: a new face hallucination framework[J]. IEEE Trans Image Processing, 2011, 20(2): 433-445.

[99] H Zhang, J Yang, Y Zhang, et al.. Non-Local Kernel Regression for Image and Video Restoration[M]. Berlin Heidelberg: Springer Press, 2010. 566-579.

[100] 陈健. 基于POCS 的红外弱小目标超分辨率复原算法研究[D]. 长春: 中国科学院大学, 2014.

    Chen Jian. Research on Infrared Dim- Small Target Super- Resolution Restoration Arithmetic Based on POCS[D]. Changchun:University of Chinese Academy of Sciences, 2014.

陈健, 高慧斌, 王伟国, 毕寻. 图像超分辨率复原方法及应用[J]. 激光与光电子学进展, 2015, 52(2): 020004. Chen Jian, Gao Huibin, Wang Weiguo, Bi Xun. Methods and Applications of Image Super-Resolution Restoration[J]. Laser & Optoelectronics Progress, 2015, 52(2): 020004.

本文已被 10 篇论文引用
被引统计数据来源于中国光学期刊网
引用该论文: TXT   |   EndNote

相关论文

加载中...

关于本站 Cookie 的使用提示

中国光学期刊网使用基于 cookie 的技术来更好地为您提供各项服务,点击此处了解我们的隐私策略。 如您需继续使用本网站,请您授权我们使用本地 cookie 来保存部分信息。
全站搜索
您最值得信赖的光电行业旗舰网络服务平台!