深度学习下的计算成像:现状、挑战与未来 下载: 11290次特邀综述
左超, 冯世杰, 张翔宇, 韩静, 陈钱. 深度学习下的计算成像:现状、挑战与未来[J]. 光学学报, 2020, 40(1): 0111003.
Chao Zuo, Shijie Feng, Xiangyu Zhang, Jing Han, Chen Qian. Deep Learning Based Computational Imaging: Status, Challenges, and Future[J]. Acta Optica Sinica, 2020, 40(1): 0111003.
[2] Wikipedia. AlphaGo versus Lee Sedol[EB/OL]. ( 2019-11-06)[2019-11-20]. https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol.
[3] Russakovsky O, Deng J, Su H, et al. Imagenet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252.
[4] Rivenson Y, Zhang Y B, Günaydın H, et al. Phase recovery and holographic image reconstruction using deep learning in neural networks[J]. Light: Science & Applications, 2018, 7(2): 17141.
[5] Wu Y C, Rivenson Y, Zhang Y B, et al. Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery[J]. Optica, 2018, 5(6): 704-710.
[6] Ren Z B, Xu Z M, Lam E Y. Learning-based nonparametric autofocusing for digital holography[J]. Optica, 2018, 5(4): 337-344.
[7] Wang K Q, Dou J Z, Qian K M, et al. Y-Net: a one-to-two deep learning framework for digital holographic reconstruction[J]. Optics Letters, 2019, 44(19): 4765-4768.
[8] Nguyen T, Bui V, Lam V, et al. Automatic phase aberration compensation for digital holographic microscopy based on deep learning background detection[J]. Optics Express, 2017, 25(13): 15043-15057.
[9] Zhang G, Guan T, Shen Z Y, et al. Fast phase retrieval in off-axis digital holographic microscopy through deep learning[J]. Optics Express, 2018, 26(15): 19388-19405.
[10] Nguyen T, Xue Y J, Li Y Z, et al. Deep learning approach for Fourier ptychography microscopy[J]. Optics Express, 2018, 26(20): 26470-26484.
[11] KappelerA, GhoshS, HollowayJ, et al. Ptychnet: CNN based Fourier ptychography[C]∥2017 IEEE International Conference on Image Processing (ICIP), September 17-20, 2017, Beijing, China. New York: IEEE, 2017: 1712- 1716.
[12] Jiang S W, Guo K K, Liao J, et al. Solving Fourier ptychographic imaging problems via neural network modeling and TensorFlow[J]. Biomedical Optics Express, 2018, 9(7): 3306-3319.
[13] Cheng Y F, Strachan M, Weiss Z, et al. Illumination pattern design with deep learning for single-shot Fourier ptychographic microscopy[J]. Optics Express, 2019, 27(2): 644-656.
[14] Lü M, Wang W, Wang H, et al. Deep-learning-based ghost imaging[J]. Scientific Reports, 2017, 7: 17865.
[15] He Y C, Wang G, Dong G X, et al. Ghost imaging based on deep learning[J]. Scientific Reports, 2018, 8: 6469.
[16] Shimobaba T, Endo Y, Nishitsuji T, et al. Computational ghost imaging using deep learning[J]. Optics Communications, 2018, 413: 147-151.
[17] Wang H D, Rivenson Y, Jin Y Y, et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy[J]. Nature Methods, 2019, 16(1): 103-110.
[18] Nehme E, Weiss L E, Michaeli T, et al. Deep-STORM: super-resolution single-molecule microscopy by deep learning[J]. Optica, 2018, 5(4): 458-464.
[19] Ouyang W, Aristov A, Lelek M, et al. Deep learning massively accelerates super-resolution localization microscopy[J]. Nature Biotechnology, 2018, 36(5): 460-468.
[20] Rivenson Y, Göröcs Z, Günaydin H, et al. Deep learning microscopy[J]. Optica, 2017, 4(11): 1437-1443.
[21] HeinrichL, Bogovic JA, SaalfeldS. Deep learning for isotropic super-resolution from non-isotropic 3D electron microscopy[M] ∥Descoteaux M, Maier-Hein L, Franz A, et al. Medical image computing and computer-assisted intervention-MICCAI 2017. Lecture notes in computer science. Cham: Springer, 2017, 10434: 135- 143.
[22] Wang H, Rivenson Y, Jin Y, et al. Deep learning achieves super-resolution in fluorescence microscopy[J]. Biorxiv, 2018, 309641.
[23] Fang L Y, Cunefare D, Wang C, et al. Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search[J]. Biomedical Optics Express, 2017, 8(5): 2732-2744.
[24] Lee C S, Baughman D M, Lee A Y. Deep learning is effective for classifying normal versus age-related macular degeneration OCT images[J]. Ophthalmology Retina, 2017, 1(4): 322-327.
[25] Schlegl T, Waldstein S M, Bogunovic H, et al. Fully automated detection and quantification of macular fluid in OCT using deep learning[J]. Ophthalmology, 2018, 125(4): 549-558.
[26] Waller L, Tian L. Machine learning for 3D microscopy[J]. Nature, 2015, 523(7561): 416-417.
[27] Nguyen T, Bui V, Nehmetallah G. Computational optical tomography using 3-D deep convolutional neural networks[J]. Optical Engineering, 2018, 57(4): 043111.
[29] Li S, Deng M, Lee J, et al. Imaging through glass diffusers using densely connected convolutional networks[J]. Optica, 2018, 5(7): 803-813.
[30] Horisaki R, Takagi R, Tanida J. Learning-based imaging through scattering media[J]. Optics Express, 2016, 24(13): 13738-13743.
[31] Satat G, Tancik M, Gupta O, et al. Object classification through scattering media with deep learning on time resolved measurement[J]. Optics Express, 2017, 25(15): 17466-17479.
[33] Goy A, Arthur K, Li S, et al. Low photon count phase retrieval using deep learning[J]. Physical Review Letters, 2018, 121(24): 243902.
[34] ChenC, Chen QF, XuJ, et al. Learning to see in the dark[C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18-23, 2018, Salt Lake City, UT, USA. New York: IEEE, 2018: 3291- 3300.
[35] Rivenson Y, Wang H D, Wei Z S, et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning[J]. Nature Biomedical Engineering, 2019, 3(6): 466-477.
[36] Rivenson Y, Liu T R, Wei Z S, et al. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning[J]. Light: Science & Applications, 2019, 8: 23.
[38] Yan K T, Yu Y J, Huang C T, et al. Fringe pattern denoising based on deep learning[J]. Optics Communications, 2019, 437: 148-152.
[39] Wang K Q, Li Y, Qian K M, et al. One-step robust deep learning phase unwrapping[J]. Optics Express, 2019, 27(10): 15100-15115.
[40] Feng S J, Zuo C, Yin W, et al. Micro deep learning profilometry for high-speed 3D surface imaging[J]. Optics and Lasers in Engineering, 2019, 121: 416-427.
[41] LuoW, Schwing AG, UrtasunR. Efficient deep learning for stereo matching[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 26-July 1, 2016, Las Vegas, Nevada. New York: IEEE, 2016: 5695- 5703.
[42] KuznietsovY, StucklerJ, LeibeB. Semi-supervised deep learning for monocular depth map prediction[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 21-26, 2017, Honolulu, HI, USA. New York: IEEE, 2017: 2215- 2223.
[43] Gerchberg R W, Saxton W O. A practical algorithm for the determination of phase from image and diffraction plane pictures[J]. Optik, 1972, 35(2): 237-250.
[44] Gerchberg R W. Phase determination from image and diffraction plane pictures in the electron microscope[J]. Optik, 1971, 34(3): 275-284.
[45] Fienup J R, Wackerman C C. Phase-retrieval stagnation problems and solutions[J]. Journal of the Optical Society of America A, 1986, 3(11): 1897-1907.
[46] Seldin J H, Fienup J R. Numerical investigation of the uniqueness of phase retrieval[J]. Journal of the Optical Society of America A, 1990, 7(3): 412-427.
[47] Guizar-Sicairos M, Fienup J R. Understanding the twin-image problem in phase retrieval[J]. Journal of the Optical Society of America A, 2012, 29(11): 2367-2375.
[48] Wackerman C C, Yagle A E. Use of Fourier domain real-plane zeros to overcome a phase retrieval stagnation[J]. Journal of the Optical Society of America A, 1991, 8(12): 1898-1904.
[49] Lu G, Zhang Z. Yu F T S, et al. Pendulum iterative algorithm for phase retrieval from modulus data[J]. Optical Engineering, 1994, 33(2): 548-555.
[50] Takajo H, Takahashi T, Kawanami H, et al. Numerical investigation of the iterative phase-retrieval stagnation problem: territories of convergence objects and holes in their boundaries[J]. Journal of the Optical Society of America A, 1997, 14(12): 3175-3187.
[51] Misell D L. A method for the solution of the phase problem in electron microscopy[J]. Journal of Physics D: Applied Physics, 1973, 6(1): L6-L9.
[53] Horstmeyer R, Chen R Y, Ou X Z, et al. Solving ptychography with a convex relaxation[J]. New Journal of Physics, 2015, 17(5): 053044.
[54] Yeh L H, Dong J, Zhong J S, et al. Experimental robustness of Fourier ptychography phase retrieval algorithms[J]. Optics Express, 2015, 23(26): 33214-33240.
[55] Zuo C, Sun J S, Chen Q. Adaptive step-size strategy for noise-robust Fourier ptychographic microscopy[J]. Optics Express, 2016, 24(18): 20724-20744.
[56] Pittman T B, Shih Y H, Strekalov D V, et al. Optical imaging by means of two-photon quantum entanglement[J]. Physical Review A, 1995, 52(5): R3429-R3432.
[57] Bennink R S, Bentley S J, Boyd R W. “Two-photon” coincidence imaging with a classical source[J]. Physical Review Letters, 2002, 89(11): 113601.
[58] Gatti A, Brambilla E, Bache M, et al. Ghost imaging with thermal light: comparing entanglement and classical correlation[J]. Physical Review Letters, 2004, 93(9): 093602.
[59] SenP, ChenB, GargG, et al. Dual photography[C]∥ACM SIGGRAPH 2005 Papers on-SIGGRAPH'05, July 31-August 4, 2005, Los Angeles, California. New York: ACM, 2005: 745- 755.
[60] Dharmpal T, Laska J N, Michael B, et al. A new compressive imaging camera architecture using optical-domain compression[J]. Proceedings of SPIE, 2006, 6065: 606509.
[61] Duarte M F, Davenport M A, Takhar D, et al. Single-pixel imaging via compressive sampling[J]. IEEE Signal Processing Magazine, 2008, 25(2): 83-91.
[63] Vellekoop I M, Mosk A P. Focusing coherent light through opaque strongly scattering media[J]. Optics Letters, 2007, 32(16): 2309-2311.
[64] Popoff S M, Lerosey G, Carminati R, et al. Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media[J]. Physical Review Letters, 2010, 104(10): 100601.
[65] Leith E N, Upatnieks J. Holographic imagery through diffusing media[J]. Journal of the Optical Society of America, 1966, 56(4): 523.
[66] Yaqoob Z, Psaltis D, Feld M S, et al. Optical phase conjugation for turbidity suppression in biological samples[J]. Nature Photonics, 2008, 2(2): 110-115.
[68] Yang W Q, Li G W, Situ G H. Imaging through scattering media with the auxiliary of a known reference object[J]. Scientific Reports, 2018, 8: 9614.
[70] Wolf E. Three-dimensional structure determination of semi-transparent objects from holographic data[J]. Optics Communications, 1969, 1(4): 153-156.
[71] Kak AC, SlaneyM. Principles of computerized tomographic imaging[M]. New York: IEEE Press, 2001.
[72] Haeberlé O, Belkebir K, Giovaninni H, et al. Tomographic diffractive microscopy: basics, techniques and perspectives[J]. Journal of Modern Optics, 2010, 57(9): 686-699.
[73] Rappaz B, Marquet P, Cuche E, et al. Measurement of the integral refractive index and dynamic cell morphometry of living cells with digital holographic microscopy[J]. Optics Express, 2005, 13(23): 9361-9373.
[74] Lauer V. New approach to optical diffraction tomography yielding a vector equation of diffraction tomography and a novel tomographic microscope[J]. Journal of Microscopy, 2002, 205(2): 165-176.
[76] Charrière F, Marian A, Montfort F, et al. Cell refractive index tomography by digital holographic microscopy[J]. Optics Letters, 2006, 31(2): 178-180.
[77] Charrière F, Pavillon N, Colomb T, et al. Living specimen tomography by digital holographic microscopy: morphometry of testate amoeba[J]. Optics Express, 2006, 14(16): 7005-7013.
[78] Choi W, Fang-Yen C, Badizadegan K, et al. Tomographic phase microscopy[J]. Nature Methods, 2007, 4(9): 717-719.
[79] Sung Y, Choi W, Fang-Yen C, et al. Optical diffraction tomography for high resolution live cell imaging[J]. Optics Express, 2009, 17(1): 266-277.
[80] Kim K, Yoon H, Diez-Silva M, et al. High-resolution three-dimensional imaging of red blood cells parasitized by Plasmodium falciparum and in situ hemozoin crystals using optical diffraction tomography[J]. Journal of Biomedical Optics, 2014, 19(1): 011005.
[82] Devaney A. A filtered backpropagation algorithm for diffraction tomography[J]. Ultrasonic Imaging, 1982, 4(4): 336-350.
[83] Barty A, Nugent K A, Roberts A, et al. Quantitative phase tomography[J]. Optics Communications, 2000, 175(4/5/6): 329-336.
[84] Soto J M, Rodrigo J A, Alieva T. Label-free quantitative 3D tomographic imaging for partially coherent light microscopy[J]. Optics Express, 2017, 25(14): 15699-15712.
[85] Li J, Chen Q, Sun J, et al. Three-dimensional tomographic microscopy technique with multi-frequency combination with partially coherent illuminations[J]. Biomedical Optics Express, 2018, 9(6): 2526-2542.
[86] Soto J M, Rodrigo J A, Alieva T. Optical diffraction tomography with fully and partially coherent illumination in high numerical aperture label-free microscopy [Invited][J]. Applied Optics, 2018, 57(1): A205-A214.
[87] Horstmeyer R, Chung J, Ou X, et al. Diffraction tomography with Fourier ptychography[J]. Optica, 2016, 3(8): 827-835.
[88] Tian L, Waller L. 3D intensity and phase imaging from light field measurements in an LED array microscope[J]. Optica, 2015, 2(2): 104-111.
[89] ZuoC, SunJ, LiJ, et al. ( 2019-05-26)[2019-11-20]. org/abs/1904. 09386. https://arxiv.
[90] Kamilov U S, Papadopoulos I N, Shoreh M H, et al. Learning approach to optical tomography[J]. Optica, 2015, 2(6): 517-522.
[91] Abbe E. Beiträge zur theorie des Mikroskops und der Mikroskopischen Wahrnehmung[J]. Archiv Für Mikroskopische Anatomie, 1873, 9( 1): 413- 468.
[92] Hell S W, Wichmann J. Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy[J]. Optics Letters, 1994, 19(11): 780-782.
[93] Betzig E, Patterson G H, Sougrat R, et al. Imaging intracellular fluorescent proteins at nanometer resolution[J]. Science, 2006, 313(5793): 1642-1645.
[94] Rust M J, Bates M, Zhuang X W. Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM)[J]. Nature Methods, 2006, 3(10): 793-796.
[95] Zeiler MD, FergusR. Visualizing and understanding convolutional networks[M] ∥Fleet D, Pajdla T, Schiele B, et al. Computer vision-ECCV 2014. Lecture notes in computer science. Cham: Springer, 2014, 8689: 818- 833.
[97] GoodfellowI, Pouget-AbadieJ, MirzaM, et al. Generative adversarial nets[C]∥Advances in Neural Information Processing Systems, December 8-13, 2014, Montreal, Quebec, Canada. Canada: NIPS, 2014: 2672- 2680.
[98] Xu Z B, Sun J. Model-driven deep-learning[J]. National Science Review, 2018, 5(1): 22-24.
[99] Lin H W, Tegmark M, Rolnick D. Why does deep and cheap learning work so well?[J]. Journal of Statistical Physics, 2017, 168(6): 1223-1247.
[101] Nguyen T Q, Weitekamp D, Anderson D, et al. Topology classification with deep learning to improve real-time event selection at the LHC[J]. Computing and Software for Big Science, 2019, 3(1): 12.
[102] PereraP, Patel VM. Deep transfer learning for multiple class novelty detection[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 16-20, 2019, Long Beach, CA, USA. New York: IEEE, 2019: 11544- 11552.
[103] PereraP, NallapatiR, XiangB. OCGAN: one-class novelty detection using GANs with constrained latent representations[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 16-20, 2019, Long Beach, CA, USA. New York: IEEE, 2019: 2898- 2906.
[105] DeVries P M R, Viégas F, Wattenberg M, et al. Deep learning of aftershock patterns following large earthquakes[J]. Nature, 2018, 560(7720): 632-634.
[106] MachineLearning. Deep learning of aftershock patterns following large earthquakes[R/OL]. [2019-11-20].https://www.reddit.com/r/MachineLearning/comments/9bo9i9/r_deep_learning_of_aftershock_patterns_following/.
左超, 冯世杰, 张翔宇, 韩静, 陈钱. 深度学习下的计算成像:现状、挑战与未来[J]. 光学学报, 2020, 40(1): 0111003. Chao Zuo, Shijie Feng, Xiangyu Zhang, Jing Han, Chen Qian. Deep Learning Based Computational Imaging: Status, Challenges, and Future[J]. Acta Optica Sinica, 2020, 40(1): 0111003.