光学学报, 2020, 40 (1): 0111003, 网络出版: 2020-01-06   

深度学习下的计算成像:现状、挑战与未来 下载: 11290次特邀综述

Deep Learning Based Computational Imaging: Status, Challenges, and Future
左超 1,2冯世杰 1,2张翔宇 1,2韩静 2陈钱 2,*
作者单位
1 南京理工大学电子工程与光电技术学院,智能计算成像实验室(SCILab), 江苏 南京 210094
2 南京理工大学江苏省光谱成像与智能感知重点实验室, 江苏 南京 210094
引用该论文

左超, 冯世杰, 张翔宇, 韩静, 陈钱. 深度学习下的计算成像:现状、挑战与未来[J]. 光学学报, 2020, 40(1): 0111003.

Chao Zuo, Shijie Feng, Xiangyu Zhang, Jing Han, Chen Qian. Deep Learning Based Computational Imaging: Status, Challenges, and Future[J]. Acta Optica Sinica, 2020, 40(1): 0111003.

参考文献

[1] Wikipedia. Computationalimaging[EB/OL]. ( 2019-10-15)[2019-11-20]. https://en.wikipedia.org/wiki/Computational_imaging.

[2] Wikipedia. AlphaGo versus Lee Sedol[EB/OL]. ( 2019-11-06)[2019-11-20]. https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol.

[3] Russakovsky O, Deng J, Su H, et al. Imagenet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252.

[4] Rivenson Y, Zhang Y B, Günaydın H, et al. Phase recovery and holographic image reconstruction using deep learning in neural networks[J]. Light: Science & Applications, 2018, 7(2): 17141.

[5] Wu Y C, Rivenson Y, Zhang Y B, et al. Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery[J]. Optica, 2018, 5(6): 704-710.

[6] Ren Z B, Xu Z M, Lam E Y. Learning-based nonparametric autofocusing for digital holography[J]. Optica, 2018, 5(4): 337-344.

[7] Wang K Q, Dou J Z, Qian K M, et al. Y-Net: a one-to-two deep learning framework for digital holographic reconstruction[J]. Optics Letters, 2019, 44(19): 4765-4768.

[8] Nguyen T, Bui V, Lam V, et al. Automatic phase aberration compensation for digital holographic microscopy based on deep learning background detection[J]. Optics Express, 2017, 25(13): 15043-15057.

[9] Zhang G, Guan T, Shen Z Y, et al. Fast phase retrieval in off-axis digital holographic microscopy through deep learning[J]. Optics Express, 2018, 26(15): 19388-19405.

[10] Nguyen T, Xue Y J, Li Y Z, et al. Deep learning approach for Fourier ptychography microscopy[J]. Optics Express, 2018, 26(20): 26470-26484.

[11] KappelerA, GhoshS, HollowayJ, et al. Ptychnet: CNN based Fourier ptychography[C]∥2017 IEEE International Conference on Image Processing (ICIP), September 17-20, 2017, Beijing, China. New York: IEEE, 2017: 1712- 1716.

[12] Jiang S W, Guo K K, Liao J, et al. Solving Fourier ptychographic imaging problems via neural network modeling and TensorFlow[J]. Biomedical Optics Express, 2018, 9(7): 3306-3319.

[13] Cheng Y F, Strachan M, Weiss Z, et al. Illumination pattern design with deep learning for single-shot Fourier ptychographic microscopy[J]. Optics Express, 2019, 27(2): 644-656.

[14] Lü M, Wang W, Wang H, et al. Deep-learning-based ghost imaging[J]. Scientific Reports, 2017, 7: 17865.

[15] He Y C, Wang G, Dong G X, et al. Ghost imaging based on deep learning[J]. Scientific Reports, 2018, 8: 6469.

[16] Shimobaba T, Endo Y, Nishitsuji T, et al. Computational ghost imaging using deep learning[J]. Optics Communications, 2018, 413: 147-151.

[17] Wang H D, Rivenson Y, Jin Y Y, et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy[J]. Nature Methods, 2019, 16(1): 103-110.

[18] Nehme E, Weiss L E, Michaeli T, et al. Deep-STORM: super-resolution single-molecule microscopy by deep learning[J]. Optica, 2018, 5(4): 458-464.

[19] Ouyang W, Aristov A, Lelek M, et al. Deep learning massively accelerates super-resolution localization microscopy[J]. Nature Biotechnology, 2018, 36(5): 460-468.

[20] Rivenson Y, Göröcs Z, Günaydin H, et al. Deep learning microscopy[J]. Optica, 2017, 4(11): 1437-1443.

[21] HeinrichL, Bogovic JA, SaalfeldS. Deep learning for isotropic super-resolution from non-isotropic 3D electron microscopy[M] ∥Descoteaux M, Maier-Hein L, Franz A, et al. Medical image computing and computer-assisted intervention-MICCAI 2017. Lecture notes in computer science. Cham: Springer, 2017, 10434: 135- 143.

[22] Wang H, Rivenson Y, Jin Y, et al. Deep learning achieves super-resolution in fluorescence microscopy[J]. Biorxiv, 2018, 309641.

[23] Fang L Y, Cunefare D, Wang C, et al. Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search[J]. Biomedical Optics Express, 2017, 8(5): 2732-2744.

[24] Lee C S, Baughman D M, Lee A Y. Deep learning is effective for classifying normal versus age-related macular degeneration OCT images[J]. Ophthalmology Retina, 2017, 1(4): 322-327.

[25] Schlegl T, Waldstein S M, Bogunovic H, et al. Fully automated detection and quantification of macular fluid in OCT using deep learning[J]. Ophthalmology, 2018, 125(4): 549-558.

[26] Waller L, Tian L. Machine learning for 3D microscopy[J]. Nature, 2015, 523(7561): 416-417.

[27] Nguyen T, Bui V, Nehmetallah G. Computational optical tomography using 3-D deep convolutional neural networks[J]. Optical Engineering, 2018, 57(4): 043111.

[28] Lü M, Wang H, Li G, et al. Learning-based lensless imaging through optically thick scattering media[J]. Advanced Photonics, 2019, 1(3): 036002.

[29] Li S, Deng M, Lee J, et al. Imaging through glass diffusers using densely connected convolutional networks[J]. Optica, 2018, 5(7): 803-813.

[30] Horisaki R, Takagi R, Tanida J. Learning-based imaging through scattering media[J]. Optics Express, 2016, 24(13): 13738-13743.

[31] Satat G, Tancik M, Gupta O, et al. Object classification through scattering media with deep learning on time resolved measurement[J]. Optics Express, 2017, 25(15): 17466-17479.

[32] Cheng S F, Li H H, Luo Y Q, et al. Artificial intelligence-assisted light control and computational imaging through scattering media[J]. Journal of Innovative Optical Health Sciences, 2019, 12(4): 1930006.

[33] Goy A, Arthur K, Li S, et al. Low photon count phase retrieval using deep learning[J]. Physical Review Letters, 2018, 121(24): 243902.

[34] ChenC, Chen QF, XuJ, et al. Learning to see in the dark[C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA. New York: IEEE, 2018: 3291- 3300.

[35] Rivenson Y, Wang H D, Wei Z S, et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning[J]. Nature Biomedical Engineering, 2019, 3(6): 466-477.

[36] Rivenson Y, Liu T R, Wei Z S, et al. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning[J]. Light: Science & Applications, 2019, 8: 23.

[37] Feng S J, Chen Q, Gu G H, et al. Fringe pattern analysis using deep learning[J]. Advanced Photonics, 2019, 1(2): 025001.

[38] Yan K T, Yu Y J, Huang C T, et al. Fringe pattern denoising based on deep learning[J]. Optics Communications, 2019, 437: 148-152.

[39] Wang K Q, Li Y, Qian K M, et al. One-step robust deep learning phase unwrapping[J]. Optics Express, 2019, 27(10): 15100-15115.

[40] Feng S J, Zuo C, Yin W, et al. Micro deep learning profilometry for high-speed 3D surface imaging[J]. Optics and Lasers in Engineering, 2019, 121: 416-427.

[41] LuoW, Schwing AG, UrtasunR. Efficient deep learning for stereo matching[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 26-July 1, 2016, Las Vegas, Nevada. New York: IEEE, 2016: 5695- 5703.

[42] KuznietsovY, StucklerJ, LeibeB. Semi-supervised deep learning for monocular depth map prediction[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA. New York: IEEE, 2017: 2215- 2223.

[43] Gerchberg R W, Saxton W O. A practical algorithm for the determination of phase from image and diffraction plane pictures[J]. Optik, 1972, 35(2): 237-250.

[44] Gerchberg R W. Phase determination from image and diffraction plane pictures in the electron microscope[J]. Optik, 1971, 34(3): 275-284.

[45] Fienup J R, Wackerman C C. Phase-retrieval stagnation problems and solutions[J]. Journal of the Optical Society of America A, 1986, 3(11): 1897-1907.

[46] Seldin J H, Fienup J R. Numerical investigation of the uniqueness of phase retrieval[J]. Journal of the Optical Society of America A, 1990, 7(3): 412-427.

[47] Guizar-Sicairos M, Fienup J R. Understanding the twin-image problem in phase retrieval[J]. Journal of the Optical Society of America A, 2012, 29(11): 2367-2375.

[48] Wackerman C C, Yagle A E. Use of Fourier domain real-plane zeros to overcome a phase retrieval stagnation[J]. Journal of the Optical Society of America A, 1991, 8(12): 1898-1904.

[49] Lu G, Zhang Z. Yu F T S, et al. Pendulum iterative algorithm for phase retrieval from modulus data[J]. Optical Engineering, 1994, 33(2): 548-555.

[50] Takajo H, Takahashi T, Kawanami H, et al. Numerical investigation of the iterative phase-retrieval stagnation problem: territories of convergence objects and holes in their boundaries[J]. Journal of the Optical Society of America A, 1997, 14(12): 3175-3187.

[51] Misell D L. A method for the solution of the phase problem in electron microscopy[J]. Journal of Physics D: Applied Physics, 1973, 6(1): L6-L9.

[52] Zheng G A, Horstmeyer R, Yang C. Wide-field, high-resolution Fourier ptychographic microscopy[J]. Nature Photonics, 2013, 7(9): 739-745.

[53] Horstmeyer R, Chen R Y, Ou X Z, et al. Solving ptychography with a convex relaxation[J]. New Journal of Physics, 2015, 17(5): 053044.

[54] Yeh L H, Dong J, Zhong J S, et al. Experimental robustness of Fourier ptychography phase retrieval algorithms[J]. Optics Express, 2015, 23(26): 33214-33240.

[55] Zuo C, Sun J S, Chen Q. Adaptive step-size strategy for noise-robust Fourier ptychographic microscopy[J]. Optics Express, 2016, 24(18): 20724-20744.

[56] Pittman T B, Shih Y H, Strekalov D V, et al. Optical imaging by means of two-photon quantum entanglement[J]. Physical Review A, 1995, 52(5): R3429-R3432.

[57] Bennink R S, Bentley S J, Boyd R W. “Two-photon” coincidence imaging with a classical source[J]. Physical Review Letters, 2002, 89(11): 113601.

[58] Gatti A, Brambilla E, Bache M, et al. Ghost imaging with thermal light: comparing entanglement and classical correlation[J]. Physical Review Letters, 2004, 93(9): 093602.

[59] SenP, ChenB, GargG, et al. Dual photography[C]∥ACM SIGGRAPH 2005 Papers on-SIGGRAPH'05, July 31-August 4, 2005, Los Angeles, California. New York: ACM, 2005: 745- 755.

[60] Dharmpal T, Laska J N, Michael B, et al. A new compressive imaging camera architecture using optical-domain compression[J]. Proceedings of SPIE, 2006, 6065: 606509.

[61] Duarte M F, Davenport M A, Takhar D, et al. Single-pixel imaging via compressive sampling[J]. IEEE Signal Processing Magazine, 2008, 25(2): 83-91.

[62] Sun B, Edgar M P, Bowman R, et al. 3D computational imaging with single-pixel detectors[J]. Science, 2013, 340(6134): 844-847.

[63] Vellekoop I M, Mosk A P. Focusing coherent light through opaque strongly scattering media[J]. Optics Letters, 2007, 32(16): 2309-2311.

[64] Popoff S M, Lerosey G, Carminati R, et al. Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media[J]. Physical Review Letters, 2010, 104(10): 100601.

[65] Leith E N, Upatnieks J. Holographic imagery through diffusing media[J]. Journal of the Optical Society of America, 1966, 56(4): 523.

[66] Yaqoob Z, Psaltis D, Feld M S, et al. Optical phase conjugation for turbidity suppression in biological samples[J]. Nature Photonics, 2008, 2(2): 110-115.

[67] Bertolotti J, van Putten E G, Blum C, et al. Non-invasive imaging through opaque scattering layers[J]. Nature, 2012, 491(7423): 232-234.

[68] Yang W Q, Li G W, Situ G H. Imaging through scattering media with the auxiliary of a known reference object[J]. Scientific Reports, 2018, 8: 9614.

[69] Lü M, Wang H, Li G, et al. Learning-based lensless imaging through optically thick scattering media[J]. Advanced Photonics, 2019, 1(3): 036002.

[70] Wolf E. Three-dimensional structure determination of semi-transparent objects from holographic data[J]. Optics Communications, 1969, 1(4): 153-156.

[71] Kak AC, SlaneyM. Principles of computerized tomographic imaging[M]. New York: IEEE Press, 2001.

[72] Haeberlé O, Belkebir K, Giovaninni H, et al. Tomographic diffractive microscopy: basics, techniques and perspectives[J]. Journal of Modern Optics, 2010, 57(9): 686-699.

[73] Rappaz B, Marquet P, Cuche E, et al. Measurement of the integral refractive index and dynamic cell morphometry of living cells with digital holographic microscopy[J]. Optics Express, 2005, 13(23): 9361-9373.

[74] Lauer V. New approach to optical diffraction tomography yielding a vector equation of diffraction tomography and a novel tomographic microscope[J]. Journal of Microscopy, 2002, 205(2): 165-176.

[75] Choi W. Tomographic phase microscopy and its biological applications[J]. 3D Research, 2012, 3(4): 5.

[76] Charrière F, Marian A, Montfort F, et al. Cell refractive index tomography by digital holographic microscopy[J]. Optics Letters, 2006, 31(2): 178-180.

[77] Charrière F, Pavillon N, Colomb T, et al. Living specimen tomography by digital holographic microscopy: morphometry of testate amoeba[J]. Optics Express, 2006, 14(16): 7005-7013.

[78] Choi W, Fang-Yen C, Badizadegan K, et al. Tomographic phase microscopy[J]. Nature Methods, 2007, 4(9): 717-719.

[79] Sung Y, Choi W, Fang-Yen C, et al. Optical diffraction tomography for high resolution live cell imaging[J]. Optics Express, 2009, 17(1): 266-277.

[80] Kim K, Yoon H, Diez-Silva M, et al. High-resolution three-dimensional imaging of red blood cells parasitized by Plasmodium falciparum and in situ hemozoin crystals using optical diffraction tomography[J]. Journal of Biomedical Optics, 2014, 19(1): 011005.

[81] Cotte Y, Toy F, Jourdain P, et al. Marker-free phase nanoscopy[J]. Nature Photonics, 2013, 7(2): 113-117.

[82] Devaney A. A filtered backpropagation algorithm for diffraction tomography[J]. Ultrasonic Imaging, 1982, 4(4): 336-350.

[83] Barty A, Nugent K A, Roberts A, et al. Quantitative phase tomography[J]. Optics Communications, 2000, 175(4/5/6): 329-336.

[84] Soto J M, Rodrigo J A, Alieva T. Label-free quantitative 3D tomographic imaging for partially coherent light microscopy[J]. Optics Express, 2017, 25(14): 15699-15712.

[85] Li J, Chen Q, Sun J, et al. Three-dimensional tomographic microscopy technique with multi-frequency combination with partially coherent illuminations[J]. Biomedical Optics Express, 2018, 9(6): 2526-2542.

[86] Soto J M, Rodrigo J A, Alieva T. Optical diffraction tomography with fully and partially coherent illumination in high numerical aperture label-free microscopy [Invited][J]. Applied Optics, 2018, 57(1): A205-A214.

[87] Horstmeyer R, Chung J, Ou X, et al. Diffraction tomography with Fourier ptychography[J]. Optica, 2016, 3(8): 827-835.

[88] Tian L, Waller L. 3D intensity and phase imaging from light field measurements in an LED array microscope[J]. Optica, 2015, 2(2): 104-111.

[89] ZuoC, SunJ, LiJ, et al. ( 2019-05-26)[2019-11-20]. org/abs/1904. 09386. https://arxiv.

[90] Kamilov U S, Papadopoulos I N, Shoreh M H, et al. Learning approach to optical tomography[J]. Optica, 2015, 2(6): 517-522.

[91] Abbe E. Beiträge zur theorie des Mikroskops und der Mikroskopischen Wahrnehmung[J]. Archiv Für Mikroskopische Anatomie, 1873, 9( 1): 413- 468.

[92] Hell S W, Wichmann J. Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy[J]. Optics Letters, 1994, 19(11): 780-782.

[93] Betzig E, Patterson G H, Sougrat R, et al. Imaging intracellular fluorescent proteins at nanometer resolution[J]. Science, 2006, 313(5793): 1642-1645.

[94] Rust M J, Bates M, Zhuang X W. Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM)[J]. Nature Methods, 2006, 3(10): 793-796.

[95] Zeiler MD, FergusR. Visualizing and understanding convolutional networks[M] ∥Fleet D, Pajdla T, Schiele B, et al. Computer vision-ECCV 2014. Lecture notes in computer science. Cham: Springer, 2014, 8689: 818- 833.

[96] Shwartz-ZivR, Tishby N. Opening the black box of deep neural networks via information[J/OL]. ( 2017-04-29)[2019-11-20]. org/abs/1703. 00810. https://arxiv.

[97] GoodfellowI, Pouget-AbadieJ, MirzaM, et al. Generative adversarial nets[C]∥Advances in Neural Information Processing Systems, December 8-13, 2014, Montreal, Quebec, Canada. Canada: NIPS, 2014: 2672- 2680.

[98] Xu Z B, Sun J. Model-driven deep-learning[J]. National Science Review, 2018, 5(1): 22-24.

[99] Lin H W, Tegmark M, Rolnick D. Why does deep and cheap learning work so well?[J]. Journal of Statistical Physics, 2017, 168(6): 1223-1247.

[100] Behera B. Status of a deep learning based measurement of the inclusive muon neutrino charged-current cross section in the NOvA near detector[J/OL]. ( 2017-10-10)[2019-11-20]. org/abs/1710. 03766. https://arxiv.

[101] Nguyen T Q, Weitekamp D, Anderson D, et al. Topology classification with deep learning to improve real-time event selection at the LHC[J]. Computing and Software for Big Science, 2019, 3(1): 12.

[102] PereraP, Patel VM. Deep transfer learning for multiple class novelty detection[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 16-20, 2019, Long Beach, CA, USA. New York: IEEE, 2019: 11544- 11552.

[103] PereraP, NallapatiR, XiangB. OCGAN: one-class novelty detection using GANs with constrained latent representations[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 16-20, 2019, Long Beach, CA, USA. New York: IEEE, 2019: 2898- 2906.

[104] Goodfellow IJ, ShlensJ, Szegedy C.Explaining and harnessing adversarialexamples[J/OL]. ( 2015-03-20)[2019-11-20]. org/abs/1412. 6572. https://arxiv.

[105] DeVries P M R, Viégas F, Wattenberg M, et al. Deep learning of aftershock patterns following large earthquakes[J]. Nature, 2018, 560(7720): 632-634.

[106] MachineLearning. Deep learning of aftershock patterns following large earthquakes[R/OL]. [2019-11-20].https://www.reddit.com/r/MachineLearning/comments/9bo9i9/r_deep_learning_of_aftershock_patterns_following/.

左超, 冯世杰, 张翔宇, 韩静, 陈钱. 深度学习下的计算成像:现状、挑战与未来[J]. 光学学报, 2020, 40(1): 0111003. Chao Zuo, Shijie Feng, Xiangyu Zhang, Jing Han, Chen Qian. Deep Learning Based Computational Imaging: Status, Challenges, and Future[J]. Acta Optica Sinica, 2020, 40(1): 0111003.

本文已被 35 篇论文引用
被引统计数据来源于中国光学期刊网
引用该论文: TXT   |   EndNote

相关论文

加载中...

关于本站 Cookie 的使用提示

中国光学期刊网使用基于 cookie 的技术来更好地为您提供各项服务,点击此处了解我们的隐私策略。 如您需继续使用本网站,请您授权我们使用本地 cookie 来保存部分信息。
全站搜索
您最值得信赖的光电行业旗舰网络服务平台!