摘要

提出一种基于全变分投影和空洞修复的光场图像透视变换算法。该算法可估计输入光场图像每个子视点的视差图,通过图像反向投影和重投影优化生成指定虚拟相机位置和姿态的光场图像,接着使用基于视差优先级的图像修复方法修复透视变换后的中心子视点图像,再通过中心子视点向水平和垂直方向进行内容传播,依次修复其他子视点图像。实验结果表明,所提方法生成的透视变换光场图像满足需求,可正确填充被遮挡区域,可以应用于多种光场图像编辑。

Abstract

This study proposes a perspective transformation algorithm for light field image based on total variational projection and hole restoration. The proposed method estimates disparity maps for all subaperture views of the input light field image. The light field image with the specified posture in the specified virtual camera position is generated via back projection and reprojection optimization. Using the image restoration algorithm based on the disparity priority, we restore the central subaperture view image after perspective transformation. The central subaperture views are used for content distribution in horizontal and vertical directions, and thus the rest subaperture view images are restored orderly. Experimental results show that the perspective transformed light field images generated by the proposed method meet requirements, and the covered regions are filled correctly. The proposed method can be applied to a series of light field image editing.

补充资料

DOI：10.3788/LOP56.151003

所属栏目：图像处理

基金项目：国家科技支撑计划课题(2015BAH54F01)、国家自然科学基金(61872166)、江苏省自然科学基金(BK20170197);

收稿日期：2019-01-31

修改稿日期：2019-03-05

网络出版日期：2019-08-01

作者单位    点击查看

联系人作者：晏涛(yantao.ustc@gmail.com)

备注：国家科技支撑计划课题(2015BAH54F01)、国家自然科学基金(61872166)、江苏省自然科学基金(BK20170197);

【1】Dai Q, Wang Y J and Han G L. Perspective image rectification based on improved Hough transformation and perspective transformation. Chinese Journal of Liquid Crystals and Displays. 27(4), 552-556(2012).
代勤, 王延杰, 韩广良. 基于改进Hough变换和透视变换的透视图像矫正. 液晶与显示. 27(4), 552-556(2012).

【2】Zhao T, Kang H L and Zhang Z P. Fast image mosaic algorithm based on area blocking and BRISK. Laser & Optoelectronics Progress. 55(3), (2018).
赵婷, 康海林, 张正平. 结合区域分块的快速BRISK图像拼接算法. 激光与光电子学进展. 55(3), (2018).

【3】Chen J H and Guo W S. Method of panoramic image stitching for theodolite-camera system. Laser & Optoelectronics Progress. 53(5), (2016).
陈继华, 郭文松. 基于摄影经纬仪的全景图像拼接方法. 激光与光电子学进展. 53(5), (2016).

【4】Du S P, Hu S M and Martin R R. Changing perspective in stereoscopic images. IEEE Transactions on Visualization and Computer Graphics. 19(8), 1288-1297(2013).

【5】Zhang L, Zhang Y H and Huang H. Efficient variational light field view synthesis for making stereoscopic 3D images. Computer Graphics Forum. 34(7), 183-191(2015).

【6】Chaurasia G, Sorkine O and Drettakis G. Silhouette-aware warping for image-based rendering. Computer Graphics Forum. 30(4), 1223-1232(2011).

【7】Chaurasia G, Duchene S, Sorkine-Hornung O et al. Depth synthesis and local warps for plausible image-based navigation. ACM Transactions on Graphics. 32(3), (2013).

【8】Wanner S and Goldluecke B. Spatial and angular variational super-resolution of 4D light fields. ∥Fitzgibbon A, Lazebnik S, Perona P, et al. Computer vision-ECCV 2012. Lecture notes in computer science. Berlin, Heidelberg: Springer. 7576, 608-621(2012).

【9】Pujades S, Devernay F and Goldluecke B. Bayesian view synthesis and image-based rendering principles. [C]∥2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 23-28, 2014, Columbus, OH, USA. New York: IEEE. 3906-3913(2014).

【10】Kalantari N K, Wang T C and Ramamoorthi R. Learning-based view synthesis for light field cameras. ACM Transactions on Graphics. 35(6), (2016).

【11】Flynn J, Neulander I, Philbin J et al. Deep stereo: learning to predict new views from the world''s imagery. [C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE. 5515-5524(2016).

【12】Gul M S K and Gunturk B K. Spatial and angular resolution enhancement of light fields using convolutional neural networks. IEEE Transactions on Image Processing. 27(5), 2146-2159(2018).

【13】Wang Y L, Liu F, Wang Z L et al. End-to-end view synthesis for light field imaging with pseudo 4DCNN. ∥Ferrari V, Hebert M, Sminchisescu C, et al. Computer vision-ECCV 2018. Lecture notes in computer science. Cham: Springer. 11206, 340-355(2018).

【14】Wanner S and Goldluecke B. Globally consistent depth labeling of 4D light fields. [C]∥2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 16-21, 2012, Providence, RI, USA. New York: IEEE. 41-48(2012).

【15】Yuan R F, Liu M, Hui M et al. Depth map stitching based on binocular vision. Laser & Optoelectronics Progress. 55(12), (2018).
袁瑞峰, 刘明, 惠梅 等. 基于双目视觉的深度图拼接. 激光与光电子学进展. 55(12), (2018).

【16】Beck A and Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences. 2(1), 183-202(2009).

【17】Criminisi A, Pérez P and Toyama K. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing. 13(9), 1200-1212(2004).

【18】Xue H Y, Zhang S M and Cai D. Depth image inpainting: improving low rank matrix completion with low gradient regularization. IEEE Transactions on Image Processing. 26(9), 4311-4320(2017).

【19】Dansereau D G, Pizarro O and Williams S B. Decoding, calibration and rectification for lenselet-based plenoptic cameras. [C]∥2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 23-28, 2013, Portland, OR, USA. New York: IEEE. 1027-1034(2013).

引用该论文

Jianming Wang, Yiming Mao, Tao Yan, Yuan Liu. Perspective Transformation Algorithm for Light Field Image[J]. Laser & Optoelectronics Progress, 2019, 56(15): 151003

王建明, 毛一鸣, 晏涛, 刘渊. 光场图像透视变换算法[J]. 激光与光电子学进展, 2019, 56(15): 151003