首页 > 论文 > 光学学报 > 40卷 > 1期(pp:0111002--1)

深度学习在计算成像中的应用 (特邀综述)

Applications of Deep Learning in Computational Imaging (Invited)

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

近年来,深度学习被广泛应用于计算成像中,并取得了令人瞩目的成果,已成为该领域的研究热点。为了深入了解现有基于深度学习的方法是如何解决众多计算成像问题的,主要介绍了该方法的基本理论和实施步骤,然后以散射成像、数字全息及计算鬼成像中的应用为例具体介绍该方法的有效性和优越性。汇总对比了一些典型应用,并对基于深度学习的计算成像方法进行了总结和展望。

Abstract

In recent years, deep learning (DL) has been widely used in computational imaging (CI) and has achieved remarkable results; as such, DL has become a research hotspot in this field. To gain an in-depth understanding of how DL-based CI works, this manuscript mainly introduces the basic theory and implementation steps of DL as well as its applications in scattering imaging, digital holography, and computational ghost imaging to demonstrate its effectiveness and superiority. Some typical applications of DL in CI are summarized and compared herein, and the CI methods based on deep learning are prospected.

Newport宣传-MKS新实验室计划
补充资料

中图分类号:O439

DOI:10.3788/AOS202040.0111002

所属栏目:“计算光学成像"专题

基金项目:中国科学院前沿科学重点研究计划、中德合作小组;

收稿日期:2019-10-15

修改稿日期:2019-11-26

网络出版日期:2020-01-01

作者单位    点击查看

王飞:中国科学院上海光学精密机械研究所信息光学与光电技术实验室, 上海 201800中国科学院大学材料与光电研究中心, 北京 100049
王昊:中国科学院上海光学精密机械研究所信息光学与光电技术实验室, 上海 201800中国科学院大学材料与光电研究中心, 北京 100049
卞耀明:中国科学院上海光学精密机械研究所信息光学与光电技术实验室, 上海 201800中国科学院大学材料与光电研究中心, 北京 100049
司徒国海:中国科学院上海光学精密机械研究所信息光学与光电技术实验室, 上海 201800中国科学院大学材料与光电研究中心, 北京 100049

联系人作者:司徒国海(ghsitu@siom.ac.cn)

备注:中国科学院前沿科学重点研究计划、中德合作小组;

【1】Mait J N, Euliss G W, Athale R A. Computational imaging [J]. Advances in Optics and Photonics. 2018, 10(2): 409-483.

【2】Barbastathis G, Ozcan A, Situ G H. On the use of deep learning for computational imaging [J]. Optica. 2019, 6(8): 921-943.

【3】Fienup J R. Reconstruction of an object from the modulus of its Fourier transform [J]. Optics Letters. 1978, 3(1): 27-29.

【4】Fienup J R. Phase retrieval algorithms: a comparison [J]. Applied Optics. 1982, 21(15): 2758-2769.

【5】Pittman T B, Shih Y H, Strekalov D V, et al. Optical imaging by means of two-photon quantum entanglement [J]. Physical Review A. 1995, 52(5): R3429-R3432.

【6】Shapiro J H, Boyd R W. The physics of ghost imaging [J]. Quantum Information Processing. 2012, 11(4): 949-993.

【7】Natterer F. The mathematics of computerized tomography (classics in applied mathematics, vol. 32) [J]. Inverse Problems. 2001, 18: 283-284.

【8】Lauterbur P C. Image formation by induced local interactions: examples employing nuclear magnetic resonance [J]. Nature. 1973, 242(5394): 190-191.

【9】Mansfield P, Grannell P K. “Diffraction” and microscopy in solids and liquids by NMR [J]. Physical Review B. 1975, 12(9): 3618-3634.

【10】Arce G R, Brady D J, Carin L, et al. Compressive coded aperture spectral imaging: an introduction [J]. IEEE Signal Processing Magazine. 2014, 31(1): 105-115.

【11】H?usler G, Willomitzer F. A stroll through 3D imaging and measurement [J]. ICO Newslett. 2015, 104.

【12】Willomitzer F, Ettl S, Faber C, et al. Single-shot three-dimensional sensing with improved data density [J]. Applied Optics. 2015, 54(3): 408-417.

【13】Arimoto H, Javidi B. Integral three-dimensional imaging with digital reconstruction [J]. Optics Letters. 2001, 26(3): 157-159.

【14】Xiao X, Javidi B, Martinez-Corral M, et al. Advances in three-dimensional integral imaging: sensing, display, and applications [Invited] [J]. Applied Optics. 2013, 52(4): 546-560.

【15】Ng R. Digital light field photography [M]. USA: Stanford University. 2006.

【16】Cutrona L J, Vivian W E, Leith E N, et al. MIL- . 1961, 5(2): 127-131.

【17】Betzig E, Patterson G H, Sougrat R, et al. Imaging intracellular fluorescent proteins at nanometer resolution [J]. Science. 2006, 313(5793): 1642-1645.

【18】Zheng G A, Horstmeyer R, Yang C. Wide-field, high-resolution Fourier ptychographic microscopy [J]. Nature Photonics. 2013, 7(9): 739-745.

【19】Gustafsson M G L. Nonlinear structured-illumination microscopy:wide-field fluorescence imaging with theoretically unlimited resolution [J]. Proceedings of the National Academy of Sciences of the United States of America. 2005, 102(37): 13081-13086.

【20】Baraniuk R. Compressive sensing [lecture notes] [J]. IEEE Signal Processing Magazine. 2007, 24(4): 118-121.

【21】LeCun Y, Bengio Y, Hinton G. Deep learning [J]. Nature. 2015, 521(7553): 436-444.

【22】Goodfellow I, Bengio Y, Courville A. Deep learning[M]. USA: , 2016.

【23】Sinha A, Lee J, Li S, et al. Lensless computational imaging through deep learning [J]. Optica. 2017, 4(9): 1117-1125.

【24】Xue Y, Cheng S, Li Y, et al. Reliable deep-learning-based phase imaging with uncertainty quantification [J]. Optica. 2019, 6(5): 618-629.

【25】Rivenson Y, Liu T R, Wei Z S, et al. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning [J]. Light: Science & Applications. 2019, 8: 23.

【26】Li S, Deng M, Lee J, et al. Imaging through glass diffusers using densely connected convolutional networks [J]. Optica. 2018, 5(7): 803-813.

【27】Li Y Z, Xue Y J, Tian L. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media [J]. Optica. 2018, 5(10): 1181-1190.

【28】Lyu M, Wang H, Li G W, et al. Learning-based lensless imaging through optically thick scattering media [J]. Advanced Photonics. 2019, 1(3): 036002.

【29】Lyu M, Wang W, Wang H, et al. Deep-learning-based ghost imaging [J]. Scientific Reports. 2017, 7: 17865.

【30】He Y C, Wang G, Dong G X, et al. Ghost imaging based on deep learning [J]. Scientific Reports. 2018, 8: 6469.

【31】Wang F, Wang H, Wang H C, et al. Learning from simulation: an end-to-end deep-learning approach for computational ghost imaging [J]. Optics Express. 2019, 27(18): 25560-25572.

【32】Rivenson Y, Zhang Y B, Günayd?n H, et al. Phase recovery and holographic image reconstruction using deep learning in neural networks [J]. Light: Science & Applications. 2018, 7: 17141.

【33】Ren Z B, Xu Z M, Lam E Y. Learning-based nonparametric autofocusing for digital holography [J]. Optica. 2018, 5(4): 337-344.

【34】Wang H, Lyu M. Situ G H. eHoloNet: a learning-based end-to-end approach for in-line digital holographic reconstruction [J]. Optics Express. 2018, 26(18): 22603-22614.

【35】Nguyen T, Xue Y J, Li Y Z, et al. Deep learning approach for Fourier ptychography microscopy [J]. Optics Express. 2018, 26(20): 26470-26484.

【36】Zhang J Z, Xu T F, Shen Z Y, et al. Fourier ptychographic microscopy reconstruction with multiscale deep residual network [J]. Optics Express. 2019, 27(6): 8612-8625.

【37】Nehme E, Weiss L E, Michaeli T, et al. Deep-STORM: super-resolution single-molecule microscopy by deep learning [J]. Optica. 2018, 5(4): 458-464.

【38】Wang H D, Rivenson Y, Jin Y Y, et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy [J]. Nature Methods. 2019, 16(1): 103-110.

【39】Liu T R, de Haan K, Rivenson Y, et al. Deep learning-based super-resolution in coherent imaging systems [J]. Scientific Reports. 2019, 9: 3926.

【40】Goy A, Arthur K, Li S, et al. Low photon count phase retrieval using deep learning [J]. Physical Review Letters. 2018, 121(24): 243902.

【41】Niu Z Z, Shi J H, Sun L, et al. Photon-limited face image super-resolution based on deep learning [J]. Optics Express. 2018, 26(18): 22773-22782.

【42】Wang K Q, Li Y, Qian K M, et al. One-step robust deep learning phase unwrapping [J]. Optics Express. 2019, 27(10): 15100-15115.

【43】Zhang T, Jiang S W, Zhao Z X, et al. Rapid and robust two-dimensional phase unwrapping via deep learning [J]. Optics Express. 2019, 27(16): 23173-23185.

【44】Feng S J, Chen Q, Gu G H, et al. Fringe pattern analysis using deep learning [J]. Advanced Photonics. 2019, 1(2): 025001.

【45】Yan K T, Yu Y J, Huang C T, et al. Fringe pattern denoising based on deep learning [J]. Optics Communications. 2019, 437: 148-152.

【46】Halupka K J, Antony B J, Lee M H, et al. Retinal optical coherence tomography image enhancement via deep learning [J]. Biomedical Optics Express. 2018, 9(12): 6205-6221.

【47】Choi G, Ryu D, Jo Y, et al. Cycle-consistent deep learning approach to coherent noise reduction in optical diffraction tomography [J]. Optics Express. 2019, 27(4): 4927-4943.

【48】Huang Y Q, Lu Z X, Shao Z M, et al. Simultaneous denoising and super-resolution of optical coherence tomography images based on generative adversarial network [J]. Optics Express. 2019, 27(9): 12289-12307.

【49】Borhani N, Kakkava E, Moser C, et al. Learning to see through multimode fibers [J]. Optica. 2018, 5(8): 960-966.

【50】Rahmani B, Loterie D, Konstantinou G, et al. Multimode optical fiber transmission with a deep learning network [J]. Light: Science & Applications. 2018, 7: 69.

【51】Bishop C M. Pattern recognition and machine learning [M]. New York: Springer-Verlag. 2006.

【52】Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets . [C]//Advances in Neural Information Processing Systems, December 8-13, 2014, Montreal, Quebec, Canada. Canada: NIPS. 2014, 2672-2680.

【53】Hornik K, Stinchcombe M, White H. Multilayer feedforward networks are universal approximators [J]. Neural Networks. 1989, 2(5): 359-366.

【54】Cybenko G. Approximation by superpositions of a sigmoidal function [J]. Mathematics of Control, Signals, and Systems. 1989, 2(4): 303-314.

【55】Nair V, Hinton G E. Rectified linear units improve restricted Boltzmann machines . [C]//Proceedings of the 27th international conference on machine learning (ICML-10), June 21-24, 2010, Haifa, Israel. [S.l.: s.n.]. 2010, 807-814.

【56】Bottou L. Large-scale machine learning with stochastic gradient descent [M]. //Lechevallier Y, Saporta G. Proceedings of COMPSTAT''''2010. Heidelberg: Physica-Verlag HD. 2010, 177-186.

【57】Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks[C]//Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, April 11-13, 2011, Fort Lauderdale, USA. USA: , 2011, 315-323.

【58】Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation . [C]//2015 IEEE International Conference on Computer Vision (ICCV), December 7-13, 2015, Santiago, Chile. New York: IEEE. 2015, 1520-1528.

【59】LeCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition [J]. Proceedings of the IEEE. 1998, 86(11): 2278-2324.

【60】Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting [J]. Journal of Machine Learning Research. 2014, 15(1): 1929-1958.

【61】Ioffe S. -03-02)[2019-10-14] [J/OL]. Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. 2015, top/abs/1502: 03167.

【62】He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition . [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 27-30, 2016, Las Vegas, NV, USA. New York: IEEE. 2016, 770-778.

【63】-06-15)[2019-10-14] [J/OL]. Ruder S. An overview of gradient descent optimization algorithms. 2017, top/abs/1609: 04747.

【64】Rumelhart D E, Hinton G E, Williams R J. Learning representations by back-propagating errors [J]. Nature. 1986, 323(6088): 533-536.

【65】Nielsen M A. Neural networks and deep learning[M]. San Francisco: , 2015.

【66】Wang K Q, Dou J Z, Qian K M, et al. Y-Net: a one-to-two deep learning framework for digital holographic reconstruction [J]. Optics Letters. 2019, 44(19): 4765-4768.

【67】Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks [J]. Communications of the ACM. 2017, 60(6): 84-90.

【68】Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions . [C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 7-12, 2015, Boston, MA, USA. New York: IEEE. 2015, 15523970.

【69】Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks . [C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, May 13-15, 2010, Chia Laguna Resort, Sardinia, Italy. [S.l.: s.n.]. 2010, 249-256.

【70】Kuka ka J, Golkov V. -10-29)[2019-10-14] [J/OL]. Cremers D. Regularization for deep learning: a taxonomy. 2017, top/abs/1710: 10686.

【71】Wang L, Ho P P, Liu C, et al. Ballistic 2-D imaging through scattering walls using an ultrafast optical Kerr gate [J]. Science. 1991, 253(5021): 769-771.

【72】Vellekoop I M, Mosk A P. Focusing coherent light through opaque strongly scattering media [J]. Optics Letters. 2007, 32(16): 2309-2311.

【73】Bertolotti J, van Putten E G, Blum C, et al. Non-invasive imaging through opaque scattering layers [J]. Nature. 2012, 491(7423): 232-234.

【74】Lyu M, Wang H, Li G, et al. -08-09)[2019-10-14] . 2017, top/abs/1708: 07881.

【75】Gabor D. A new microscopic principle [J]. Nature. 1948, 161(4098): 777-778.

【76】Leith E N, Upatnieks J. Reconstructed wavefronts and communication theory [J]. Journal of the Optical Society of America. 1962, 52(10): 1123-1130.

【77】Rivenson Y, Wu Y C, Ozcan A. Deep learning in holography and coherent imaging [J]. Light: Science & Applications. 2019, 8: 85.

【78】Shapiro J H. Computational ghost imaging [J]. Physical Review A. 2008, 78(6): 061802.

【79】Clemente P, Durán V, Torres-Company V, et al. Optical encryption based on computational ghost imaging [J]. Optics Letters. 2010, 35(14): 2391-2393.

【80】Cheng J. Ghost imaging through turbulent atmosphere [J]. Optics Express. 2009, 17(10): 7916-7921.

【81】Gong W L, Zhao C Q, Yu H, et al. Three-dimensional ghost imaging lidar via sparsity constraint [J]. Scientific Reports. 2016, 6: 26133.

【82】Ceddia D, Paganin D M. Random-matrix bases, ghost imaging, and X-ray phase contrast computational ghost imaging [J]. Physical Review A. 2018, 97(6): 062119.

【83】Katz O, Bromberg Y, Silberberg Y. Compressive ghost imaging [J]. Applied Physics Letters. 2009, 95(13): 131110.

引用该论文

Wang Fei,Wang Hao,Bian Yaoming,Situ Guohai. Applications of Deep Learning in Computational Imaging[J]. Acta Optica Sinica, 2020, 40(1): 0111002

王飞,王昊,卞耀明,司徒国海. 深度学习在计算成像中的应用[J]. 光学学报, 2020, 40(1): 0111002

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF