Single-molecule localization microscopy (SMLM) is widely used for 3D spatial resolution of subcellular structures. Traditional single-molecule localization analysis methods rely on point spread function (PSF) model fitting, which restricts the application of some more complex PSF models. The emergence of deep learning is gradually changing the analysis methods of single-molecule localization, which is expected to be used to analyze complex PSF models and can provide new directions and ideas.

To break through the diffraction limit, various super-resolution fluorescence microscopy techniques have been developed for nanoscale imaging in recent years, making it possible to observe the interactions between individual molecules inside cells. Several imaging methods have emerged to achieve high resolution beyond the optical diffraction limit, including structured illumination microscopy (SIM), stimulated emission depletion (STED) microscopy, and SMLM. Among them, SMLM is widely used in the study of organelles and protein structures. The main idea of SMLM imaging is to avoid the overlapping of multiple point light sources in the diffraction-limited area, and control the imaging area with activation light and excitation light to emit only a small number of random and discrete single fluorescent molecules each time.

With the development of deep learning, a large number of excellent network frameworks have emerged, which have achieved good results in the field of computer vision, for example, convolutional neural network (CNN) widely used in feature extraction, recurrent neural network (RNN) for time series data extraction, generative adversarial network (GAN) for generating data, and so on. Developed super-resolution microscopy techniques combined with deep learning have achieved good results. Depending on the type of input data and target output data, SMLM image data analysis employs various efficient and well-performing neural network architectures. Currently, deep learning has been widely used to extract single-molecule localization from diffraction-limited images. Compared with traditional model fitting methods, deep learning can greatly reduce the processing time while ensuring a certain accuracy. By learning the structural redundancy of similar images, deep learning can reconstruct high-quality super-resolution images from sparse images. This approach can minimize phototoxicity and facilitate super-resolution imaging of live cells. Deep learning solves the limitation of traditional methods in high-density single-molecule localization and improves the localization accuracy. Additionally, employing deep learning-based methods is of great help in extracting various hidden molecular information, such as single-molecule axial position and emission color information, which can improve the localization and spectral accuracy.

For specific application scenarios, deep learning is used to model complex mathematical models by data training method, and the network is optimized by loss function, which makes many complex PSF designs possible. PSFs usually contain rich multidimensional information, such as transverse and axial position, emission wavelength, color and so on. Using deep learning can fully extract the information contained in PSFs, thereby improving localization accuracy.

Deep learning is a data-driven feature extraction method. By learning a large number of samples, a deep and specific feature representation of the dataset can be obtained. So diversity of data is extremely important. Different from the traditional natural image dataset, the desired single-molecule image can be simulated by the simulator. The emergence of a large number of excellent single-molecule data simulators also provides a favorable tool for the combination of deep learning and single-molecule localization.

Deep learning has had a huge impact on computer vision and image processing, and has become an emerging and powerful tool for image analysis. Recently developed super-resolution microscopy techniques combined with deep learning have achieved good results. Depending on the type of input data and target output data, SMLM image data analysis employs various efficient and well-performing neural network architectures. For the combination of single-molecule localization and deep learning, we analyze it from three aspects: dataset, network framework, and loss function.

Deep learning is a method that relies on training data to extract features from a large amount of data to obtain target results. For example, deep learning algorithms based on sparse image reconstruction often rely on the biological sample structure of the training dataset. Data mismatches need to be taken into account when training with deep learning, while developing methods that are aware or robust to differences between test and training data is needed. On the other hand, the similarity between the data simulation and the real data is improved, and the error is reduced. Considering the characteristics of single-molecule datasets and the training difficulty, number of parameters and training time of the network, transfer learning is a feasible method. Considering the structural similarity of the single-molecule datasets, retraining on the pretrained network will greatly reduce the training time and training difficulty. In addition to the feature of structural similarity, the input of the network is a continuous frame with time sequence. According to this feature, the RNN-type network can better extract features. For the loss function, we need to consider these issues. When constructing the loss function, we need to consider whether the components of the loss function are correlated, whether a simple summation can be performed, and how to choose the weights of different parts. Deep learning algorithms have had a huge impact on many fields of scientific research. We expect deep learning to be more deeply integrated with single-molecule localization and achieve more remarkable achievements.