近年来，乳腺癌已成为威胁女性健康的头号恶性肿瘤，对其实现快速精准的筛查和诊断变得尤为重要。针对现有临床病理学检查需要染色突显细胞形态并依赖医生主观经验判断的局限性，立足于明场显微成像和偏振显微成像构建多模态显微成像技术，获取正常与癌变乳腺组织未染色冰冻切片的形态结构及其异质性信息并对之进行分析和诊断研究。首先对正常与癌变组织切片进行多角度正交偏振成像、明场显微成像并分析图像差异性；然后实施像素级图像融合；接着利用卷积神经网络模型，对多模态融合图像进行深度学习的特征提取与分类，有效提升准确率(0.8727)和受试者特征曲线下面积(AUC，0.9400)等参数，进而实现精准的乳腺癌智能诊断。该技术有助于医生进行快速准确的临床诊断，为实施乳腺癌术中快检以辅助精准手术治疗提供有效的技术手段，具有突出的临床潜力和应用前景。

Breast cancer has been the most common life-threatening cancer of women in recent years. Both diagnostic imaging and pathology are routinely employed to diagnose breast cancer, while the latter is considered the "gold standard" for cancer diagnosis. In clinical practice, however, the routine pathological diagnosis is strongly hindered by the complicated and time-consuming process of staining the biopsy sections with hematoxylin and eosin (H&E) for highlighting the fine structures of cells and tissues. Moreover, the diagnosis is manually performed by pathologists, therefore the diagnostic speed and accuracy are highly dependent on their knowledge and experience. In order to better serve the treatment of breast cancer, the pathological diagnosis is greatly expected to be accelerated and automated, especially providing the efficient intraoperative assessment for precision surgical therapy. In this study, a rapid, accurate and automatic diagnostic technique of breast cancer is proposed based on the polarization and bright-field multimodal microscopic imaging. To accelerate the pathological diagnosis, the breast biopsy sections are directly investigated without the H&E staining, while the cell structures are no longer distinct under the bright-field microscopic imaging. In this case, the polarization microscopic imaging is introduced to further extract the morphological difference between the normal and cancerous breast tissues. Collagen fiber is an important part of the extracellular matrix (ECM), and the organization of collagen fibers has been found to be closely related with the cancer progression. Due to the inherent optical anisotropy, collagen fibers can be examined by cross-polarization imaging. To perform the automatic diagnosis, deep learning is employed to distinguish the normal and cancerous breast tissues, where the convolutional neural network (CNN) classification model is established to extract features from the multimodal microscopic images and make the accurate and reliable judgements.

In this study, 23 breast biopsies from 16 patients were first rapidly frozen in liquid nitrogen and cut into sections with the thickness of 15 μm using a cryotome. Without the H&E staining, the bright-field microscopic imaging and polarization microscopic imaging were sequentially performed through switching the light source, polarizer and analyzer of the custom-made transmission polarization microscope. The polarization microscopic imaging was operated in the cross-polarization mode, where the polarizer was orthogonal to the analyzer. Since the period of cross-polarization imaging is 90°, the polarizer-analyzer pair was rotated by 0°, 30° and 60° to characterize the biopsy sections. Following the microscopic imaging, the pixel-level image fusion was conducted to transform the four multimodal images into a single fused image for breast cancer diagnosis. In order to perform the deep learning-assisted automatic diagnosis, the classical CNN ResNet34 was utilized to develop a classification model, whose input is the pixel-level fused image. In addition to the pixel-level fusion, the decision-level fusion was also examined for comparison, where two classification models were created based on the bright-field and polarization microscopic imaging, respectively. In this way, the final classification result was generated according to the weight coefficients determined using the logistic regression algorithm. To evaluate the performance of CNN classification model, four parameters including accuracy, sensitivity, specificity and AUC [area under receiver operating characteristic (ROC) curve] were calculated. Due to the rather small amount of breast biopsies, the leave-one-out cross validation was performed in order to avoid overfitting.

The cross-polarization microscopic imaging of normal breast tissues exhibits the distinct periodic change in the brightness with the rotation of the polarizer-analyzer pair from 0° to 90°, while that of cancerous breast tissues is almost unchanged (Fig. 3). This polarization-sensitive brightness change results from the anisotropic organization of collagen fibers in the normal breast tissues, which alters during the cancer progression. Further, the classification result of the multi-polarization imaging is better than that of the single-polarization imaging (Table 1). Following the pixel-level fusion of multimodal images, the CNN classification model based on ResNet34 was established. As a result, the accuracy of 0.8727 and AUC of 0.9400 were realized, better than bright-field (0.8540, 0.9013) and polarization (0.8575, 0.9307) microscopic imaging, respectively (Table 2). In addition, the decision-level fusion was also evaluated, achieving the accuracy of 0.8710 and AUC of 0.9367. The weight coefficients of bright-field and polarization imaging were calculated to be 0.3971 and 0.6029, respectively.

A deep learning-assisted multimodal microscopic imaging technique is proposed for the rapid, accurate and automatic diagnosis of breast cancer, combining bright-field and cross-polarization microscopic imaging. In this scheme, the time-consuming process of H&E staining can be removed to accelerate the breast cancer diagnosis, and the automatic diagnosis can be performed with the CNN classification model. Moreover, the accurate diagnosis can be realized, since the cross-polarization microscopic imaging can further extract the polarization-sensitive morphological change closely related with the cancer progression, such as the organization of the optically anisotropic collagen fibers. In this sense, the multimodal microscopic imaging diagnosis has great potential to better serve the surgical treatment of breast cancer with the efficient intraoperative assessment.