激光生物学报, 2022, 31 (6): 481, 网络出版: 2023-03-06  

组织学病理图像在深度学习中染色处理的研究进展

Advances in Staining Processing of Histological Pathology Images in Deep Learning
作者单位
华南师范大学生物光子学研究院,广州 510631
摘要
深度学习使辅助诊断的软件能够更积极有效地开发和应用,但是组织病理学图像的颜色变化降低了这些算法的性能。染色归一化可以解决扫描仪效应、不同的染色方法、患者的疾病状态、染色时间等因素产生的图像异质性。虚拟染色可以摆脱载玻片染色,减少载玻片的制备步骤,为临床缩短样本的制备时间,节省大量的成本。在缺乏注释训练数据的情况下,病理图像数据增强可用于创建具有纹理和颜色、样式逼真的人工样本来促进网络训练。本文就组织学病理图像在深度学习病理分析中染色处理的染色归一化、虚拟染色和数据增强等方面展开综述,为组织学病理图像在临床上的应用和研究提供参考。
Abstract
Deep learning allows software to assist in diagnosis to be developed and applied more aggressively and efficiently, whereas the color variability of histopathology images degrades the performance of these algorithms. Stain normalization can address image heterogeneity arising from scanner effects, different staining methods, patient’s disease states, staining times, and other factors. Virtual staining can eliminate slide staining and reduce slide preparation steps, reducing sample preparation time for the clinic and saving significant costs. In the absence of annotated training data, pathology image data augmentation is performed by creating artificial samples with realistic texture, color and style to facilitate network training. In this paper, we made a review on staining processing of histological pathology images in deep learning pathology analysis to provide a reference for histological pathology maps in clinical applications and research.
参考文献

[1] JOSE L, LIU S, RUSSO C, et al. Generative adversarial networks in digital pathology and histopathological image processing: a review [J]. Journal of Pathology Informatics, 2021, 12: 43.

[2] BIANCONI F, KATHER J N, REYES-ALDASORO C C. Experimental assessment of color deconvolution and color normalization for automated classification of histology images stained with hematoxylin and eosin [J]. Cancers, 2020, 12(11): 3337.

[3] REINHARD E, ADHIKHMIN M, GOOCH B, et al. Color transfer between images [J]. IEEE Computer Graphics and Applications, 2001, 21(4): 34-41.

[4] ROY S, PANDA S, JANGID M. Modified reinhard algorithm for color normalization of colorectal cancer histopathology images [C]. 2021 29th European Signal Processing Conference (EUSIPCO). IEEE, 2021: 1231-1235

[5] ASWATHY M A, JAGANNATH M. Dual stage normalization approach towards classification of breast cancer [J]. IETE Journal of Research, 2022, 68(4): 3074-3085

[6] NADEEM S, HOLLMANN T, TANNENBAUM A. Multimarginal Wasserstein barycenter for stain normalization and augmentation [C]. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2020: 362-371.

[7] RUIFROK A C, KATZ R L, JOHNSTON D A. Comparison of quantification of histochemical staining by hue-saturation-intensity (HSI) transformation and color-deconvolution [J]. Applied Immunohistochemistry & Molecular Morphology, 2003, 11(1): 85-91.

[8] ZHENG Y, JIANG Z, ZHANG H, et al. Adaptive color deconvolution for histological WSI normalization[J]. Computer Methods and Programs in Biomedicine, 2019, 170: 107-120.

[9] SALVI M, MICHIELLI N, MOLINARI F. Stain color adaptive normalization (SCAN) algorithm: separation and standardization of histological stains in digital pathology [J]. Computer Methods and Programs in Biomedicine, 2020, 193: 105506.

[10] MACENKO M, NIETHAMMER M, MARRON J S, et al. A method for normalizing histology slides for quantitative analysis [C]. 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro. IEEE, 2009: 1107-1110.

[11] VAHADANE A, PENG T, SETHI A, et al. Structure-preserving color normalization and sparse stain separation for histological images [J]. IEEE Transactions on Medical Imaging, 2016, 35(8): 1962-1971.

[12] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks [C]. Proceedings of the IEEE International Conference on Computer Vision, 2017: 2223-2232.

[13] JIAO Y, LI J, FEI S. Staining condition visualization in digital histopathological whole-slide images [J]. Multimedia Tools and Applications, 2022, 81(13): 17831-17847.

[14] SHABAN M T, BAUR C, NAVAB N, et al. StainGAN: stain style transfer for digital histological images [C]. 2019 Ieee 16th international symposium on biomedical imaging (Isbi 2019). IEEE, 2019: 953-956.

[15] KANG H, LUO D, FENG W, et al. StainNet: a fast and robust stain normalization network[J]. Frontiers in Medicine, 2021, 8: 746307.

[16] GUTIERREZ PEREZ J C, OTERO BAGUER D, MAASS P. StainCUT: stain normalization with contrastive learning [J]. Journal of Imaging, 2022, 8(7): 202.

[17] PARK T, EFROS A A, ZHANG R, et al. Contrastive learning for unpaired image-to-image translation [C]. European Conference on Computer Vision. Springer, Cham, 2020: 319-345.

[18] MAO X, WANG J, TAO X, et al. Single generative networks for stain normalization and quality enhancement of histological images in digital pathology [C]. 2021 14th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI). IEEE, 2021: 1-5.

[19] NAZKI H, ARANDJELOVI? O, UM I H, et al. MultiPathGAN: structure preserving stain normalization using unsupervised multi-domain adversarial network with perception loss [J]. arXiv, 2022: 2204.09782. https://doi.org/10.48550/arXiv:2204.09782.

[20] SHRIVASTAVA A, ADORNO W, SHARMA Y, et al. Self-attentive adversarial stain normalization[C]. International Conference on Pattern Recognition. Springer, Cham, 2021: 120-140.

[21] LIU Y, YUAN H, WANG Z, et al. Global pixel transformers for virtual staining of microscopy images [J]. IEEE Transactions on Medical Imaging, 2020, 39(6): 2256-2266.

[22] CHRISTIANSEN E M, YANG S J, ANDO D M, et al. In silico labeling: predicting fluorescent labels in unlabeled images [J]. Cell, 2018, 173(3): 792-803.e19.

[23] SIMONSON P D, REN X, FROMM J R. Creating virtual hematoxylin and eosin images using samples imaged on a commercial CODEX platform [J]. Journal of Pathology Informatics, 2021, 12(1): 52.

[24] BURLINGAME E A, MCDONNELL M, SCHAU G F, et al. SHIFT: speedy histological-to-immunofluorescent translation of a tumor signature enabled by deep learning [J]. Scientific Reports, 2020, 10(1): 17507.

[25] Rivenson Y, Wang H, Wei Z, et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning[J]. Nature Biomedical Engineering, 2019, 3(6): 466-477.

[26] CHEN Z, YU W, WONG I H M, et al. Deep-learning-assisted microscopy with ultraviolet surface excitation for rapid slide-free histological imaging [J]. Biomedical Optics Express, 2021, 12(9): 5920.

[27] LI J, GARFINKEL J, ZHANG X, et al. Biopsy-free in vivo virtual histology of skin using deep learning [J]. Light: Science & Applications, 2021, 10(1): 233.

[28] ZHANG R, CAO Y, LI Y, et al. MVFStain: multiple virtual functional stain histopathology images generation based on specific domain mapping [J]. Medical Image Analysis, 2022, 80: 102520.

[29] DE HAAN K, ZHANG Y, ZUCKERMAN J E, et al. Deep learning-based transformation of H&E stained tissues into special stains [J]. Nature Communications, 2021, 12(1): 4884.

[30] BELL K, ABBASI S, DINAKARAN D, et al. Reflection-mode virtual histology using photoacoustic remote sensing microscopy [J]. Scientific Reports, 2020, 10(1): 1-13.

[31] KANG L, LI X, ZHANG Y, et al. Deep learning enables ultraviolet photoacoustic microscopy based histological imaging with near real-time virtual staining [J]. Photoacoustics, 2022, 25: 100308.

[32] MARTELL M T, HAVEN N J M, MCALISTER E A, et al. Deep learning-enabled realistic virtual histology with ultraviolet scattering and photoacoustic remote sensing microscopy [C]. Microscopy Histopathology and Analytics. Optica Publishing Group, 2022: MS4A.2.

[33] GAREAU D S, JEON H, NEHAL K S, et al. Rapid screening of cancer margins in tissue with multimodal confocal microscopy [J]. Journal of Surgical Research, 2012, 178(2): 533-538.

[34] CARRASCO-ZEVALLOS O M, VIEHLAND C, KELLER B, et al. Review of intraoperative optical coherence tomography: technology and applications [Invited][J]. Biomedical Optics Express, 2017, 8(3): 1607-1637.

[35] RESTALL B S, CIKALUK B D, MARTELL M T, et al. Fast hybrid optomechanical scanning photoacoustic remote sensing microscopy for virtual histology [J]. Biomedical Optics Express, 2022, 13(1): 39-47.

[36] HOLLANDI R, SZKALISITY A, TOTH T, et al. NucleAIzer: a parameter-free deep learning framework for nucleus segmentation using image style transfer [J]. Cell Systems, 2020, 10(5): 453-458.e6.

[37] HUANG X, LIU M Y, BELONGIE S, et al. Multimodal unsupervised image-to-image translation [C]. Proceedings of the European Conference on Computer Vision (ECCV), 2018: 172-189.

[38] WANG H, ZHENG H, CHEN J, et al. Unlabeled data guided semi-supervised histopathology image segmentation [C]. 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2020: 815-820.

[39] YAMASHITA R, LONG J, BANDA S, et al. Learning domain-agnostic visual representation for computational pathology using medically-irrelevant style transfer augmentation [J]. IEEE Transactions on Medical Imaging, 2021, 40(12): 3945-3954.

[40] LIU Y, WAGNER S J, PENG T. Multi-modality microscopy image style augmentation for nuclei segmentation [J]. Journal of Imaging, 2022, 8(3): 71.

[41] MOGHADAM A Z, AZARNOUSH H, SEYYEDSALEHI S A, et al. Stain transfer using Generative Adversarial Networks and disentangled features [J]. Computers in Biology and Medicine, 2022, 142: 105219.

罗诗欢, 刘智明, 杨必文, 郭周义. 组织学病理图像在深度学习中染色处理的研究进展[J]. 激光生物学报, 2022, 31(6): 481. LUO Shihuan, LIU Zhiming, YANG Biwen, GUO Zhouyi. Advances in Staining Processing of Histological Pathology Images in Deep Learning[J]. Acta Laser Biology Sinica, 2022, 31(6): 481.

关于本站 Cookie 的使用提示

中国光学期刊网使用基于 cookie 的技术来更好地为您提供各项服务,点击此处了解我们的隐私策略。 如您需继续使用本网站,请您授权我们使用本地 cookie 来保存部分信息。
全站搜索
您最值得信赖的光电行业旗舰网络服务平台!