首页 > 论文 > 激光与光电子学进展 > 57卷 > 22期(pp:221106--1)

基于深度学习的傅里叶叠层显微成像

Fourier Ptychographic Microscopy Reconstruction Based on Deep Learning

  • 摘要
  • 论文信息
  • 参考文献
  • 被引情况
  • PDF全文
分享:

摘要

傅里叶叠层显微成像(FPM)是一种能够重建宽视场和高分辨率图像的新型成像技术。传统的FPM重建算法计算成本高,重建高质量的图像需要较大的图像采集量,这些缺点使得传统重建算法的成像性能和效率较低。因此,提出一种基于深度学习的傅里叶叠层显微成像的神经网络模型,对图像进行低分辨率到高分辨率的端到端映射,有效提高成像性能和效率。首先,借助菱形采样方法进行图像采集,加速低分辨图片采集过程。其次,结合残差结构、密集连接以及通道注意力机制等模块,拓展网络深度、挖掘有用特征,增强网络模型的表达能力和泛化能力。然后,使用子像素卷积进行高效地上采样,恢复高清图像。最后,采用主观和客观的评价方法对重建结果进行评估。结果显示,本文提出的网络模型对比传统重建算法重构效果更优,且降低了计算复杂度,平均重建时间更短。同时,在保证图像重建效果不变的情况下,低分辨率图像的采集数量比传统算法减少了约一半。

Abstract

Fourier ptychographic microscopy (FPM) is a newly developed imaging technology, which is capable of reconstructing images with a wide field of view and high resolution. However, the reconstruction based on traditional reconstruction algorithms has high calculation cost, large amount of image acquisition, and low efficiency. Therefore, we propose a deep learning-based neural network model of FPM that performs end-to-end mapping from low-resolution to high-resolution to effectively improve imaging performance and efficiency. First, the diamond sampling method is used to speed up the process of image acquisition. Second, the combination of residual structure, dense connection, and channel attention mechanism is used to expand the network depth, mine useful features, and enhance the expression and generalization ability of the network model. Then, sub-pixel convolution is used for efficient upsampling and restoring high-resolution images. Finally, subjective and objective evaluation methods are used to evaluate the reconstruction results. The results show that, compared with the traditional reconstruction algorithm, the proposed network model has better reconstruction effect, lower computational complexity, and shorter average reconstruction time. At the same time, the number of low-resolution images is reduced by about half compared with the traditional algorithm.

广告组1.2 - 空间光调制器+DMD
补充资料

中图分类号:O436

DOI:10.3788/LOP57.221106

所属栏目:成像系统

基金项目:国家自然科学基金;

收稿日期:2020-03-26

修改稿日期:2020-04-27

网络出版日期:2020-11-01

作者单位    点击查看

陈奕灿:暨南大学信息科学技术学院电子工程系, 广东 广州 510632
吴霞:暨南大学信息科学技术学院电子工程系, 广东 广州 510632
罗志:暨南大学信息科学技术学院电子工程系, 广东 广州 510632
杨恢东:暨南大学信息科学技术学院电子工程系, 广东 广州 510632
黄波:暨南大学信息科学技术学院电子工程系, 广东 广州 510632

联系人作者:吴霞(wuxia_liao@qq.com); 黄波(wuxia_liao@qq.com);

备注:国家自然科学基金;

【1】Sun J S, Zhang Y Z, Chen Q, et al. Fourier ptychographic microscopy: theory, advances, and applications [J]. Acta Optica Sinica. 2016, 36(10): 1011005.
孙佳嵩, 张玉珍, 陈钱, 等. 傅里叶叠层显微成像技术: 理论、发展和应用 [J]. 光学学报. 2016, 36(10): 1011005.

【2】Zheng G A, Horstmeyer R, Yang C. Wide-field, high-resolution Fourier ptychographic microscopy [J]. Nature Photonics. 2013, 7(9): 739-745.Zheng G A, Horstmeyer R, Yang C. Wide-field, high-resolution Fourier ptychographic microscopy [J]. Nature Photonics. 2013, 7(9): 739-745.

【3】Zheng G. Fourier ptychographic imaging: a MATLAB tutorial [M]. San Rafael: Morgan & Claypool Publishers. 2016, 10-22.

【4】Fienup J R. Phase retrieval algorithms: a personal tour [J]. Applied Optics. 2013, 52(1): 45-56.

【5】Gerchberg R W. A practical algorithm for the determination of phase from image and diffraction plane pictures [J]. Optik. 1972, 35(2): 237-246.

【6】Zhao M, Wang X M, Zhang X H, et al. Experimental research on macroscopic Fourier ptychography super-resolution imaging [J]. Laser & Optoelectronics Progress. 2019, 56(12): 121101.
赵明, 王希明, 张晓慧, 等. 宏观傅里叶叠层超分辨率成像实验研究 [J]. 激光与光电子学进展. 2019, 56(12): 121101.

【7】Tian L, Li X, Ramchandran K, et al. Multiplexed coded illumination for Fourier ptychography with an LED array microscope [J]. Biomedical Optics Express. 2014, 5(7): 2376-2389.

【8】Lin Z Q, Ma X, Lin J X, et al. Fourier ptychographic microscopy based on rotating arc-shaped array of LEDs [J]. Laser & Optoelectronics Progress. 2018, 55(7): 071102.
林子强, 马骁, 林锦新, 等. 基于弧形阵列LED光源旋转照明装置的傅里叶叠层显微术 [J]. 激光与光电子学进展. 2018, 55(7): 071102.

【9】Li T, Zhao J F, Mao H F, et al. An efficient Fourier ptychographic microscopy imaging method based on angle illumination optimization [J]. Laser & Optoelectronics Progress. 2020, 57(8): 081106.
李通, 赵巨峰, 毛海锋, 等. 基于角度照明优化的傅里叶叠层显微成像方法 [J]. 激光与光电子学进展. 2020, 57(8): 081106.

【10】Bian L H, Suo J L, Zheng G A, et al. Fourier ptychographic reconstruction using wirtinger flow optimization [J]. Optics Express. 2015, 23(4): 4856-4866.

【11】Zuo C, Sun J S, Chen Q. Adaptive step-size strategy for noise-robust Fourier ptychographic microscopy [J]. Optics Express. 2016, 24(18): 20724-20744.

【12】Kappeler A, Ghosh S, Holloway J, et al. Ptychnet: , 2017, 1712-1716.

【13】Jiang S W, Guo K K, Liao J, et al. Solving Fourier ptychographic imaging problems via neural network modeling and TensorFlow [J]. Biomedical Optics Express. 2018, 9(7): 3306-3319.

【14】Rivenson Y, Zhang Y B, Günayd?n H, et al. Phase recovery and holographic image reconstruction using deep learning in neural networks [J]. Light: Science & Applications. 2018, 7(2): 17141.

【15】Girshick R. Fast R-CNN[C]∥2015 IEEE International Conference on Computer Vision (ICCV). December 7-13, 2015, Santiago, Chile. New York: , 2015, 1440-1448.

【16】Dong C, Loy C C, He K M, et al. Image super-resolution using deep convolutional networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2016, 38(2): 295-307.

【17】Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. New York: , 2017, 105-114.

【18】Shi Z T, Wang Z R, Wang R, et al. Single image super-resolution based on convolutional neural network [J]. Laser & Optoelectronics Progress. 2018, 55(12): 121001.
史紫腾, 王知人, 王瑞, 等. 基于卷积神经网络的单幅图像超分辨 [J]. 激光与光电子学进展. 2018, 55(12): 121001.

【19】Sun C, Lü J W, Gong J, et al. Image super-resolution method combining wavelet transform with deep network [J]. Laser & Optoelectronics Progress. 2018, 55(12): 121006.
孙超, 吕俊伟, 宫剑, 等. 结合小波变换与深度网络的图像超分辨率方法 [J]. 激光与光电子学进展. 2018, 55(12): 121006.

【20】LeCun Y, Bengio Y, Hinton G. Deep learning [J]. Nature. 2015, 521(7553): 436-444.

【21】Schmidhuber J. Deep learning in neural networks: an overview [J]. Neural Networks. 2015, 61: 85-117.

【22】He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. New York: , 2016, 770-778.

【23】Huang G. Liu Z, van der Maaten L, et al. Densely connected convolutional networks[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. New York: , 2017, 2261-2269.

【24】Tong T, Li G, Liu X J, et al. Image super-resolution using dense skip connections[C]∥2017 IEEE International Conference on Computer Vision (ICCV). October 22-29, 2017, Venice, Italy. New York: , 2017, 4809-4817.

【25】Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018, Salt Lake City, UT, USA. New York: , 2018, 7132-7141.

【26】Lin M, Chen Q. -03-04) [2020-03-25] . https:∥arxiv. 2014, org/abs/1312: 4400.

【27】Shi W Z, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. New York: , 2016, 1874-1883.

【28】Ioffe S. -03-02) [2020-03-25] . https:∥arxiv. 2015, org/abs/1502: 03167.

【29】Bevilacqua M, Roumy A, Guillemot C, et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding . [C]∥Procedings of the British Machine Vision Conference 2012. Surrey. British Machine Vision Association. 2012, 135.

【30】Zeyde R, Elad M, Protter M. On single image scale-up using sparse-representations [M]. ∥Curves and surfaces. Berlin, Heidelberg: Springer Berlin Heidelberg. 2012, 711-730.

【31】Martin D, Fowlkes C, Tal D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]∥Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001. July 7-14, 2001, Vancouver, BC, Canada. New York: , 2001, 416-423.

【32】Huang J B, Singh A, Ahuja N. Single image super-resolution from transformed self-exemplars[C]∥2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 7-12, 2015, Boston, MA, USA. New York: , 2015, 5197-5206.

【33】Matsui Y, Ito K, Aramaki Y, et al. Sketch-based manga retrieval using manga109 dataset [J]. Multimedia Tools and Applications. 2017, 76(20): 21811-21838.

引用该论文

Chen Yican,Wu Xia,Luo Zhi,Yang Huidong,Huang Bo. Fourier Ptychographic Microscopy Reconstruction Based on Deep Learning[J]. Laser & Optoelectronics Progress, 2020, 57(22): 221106

陈奕灿,吴霞,罗志,杨恢东,黄波. 基于深度学习的傅里叶叠层显微成像[J]. 激光与光电子学进展, 2020, 57(22): 221106

您的浏览器不支持PDF插件,请使用最新的(Chrome/Fire Fox等)浏览器.或者您还可以点击此处下载该论文PDF