Optical neural networks (ONNs), enabling low latency and high parallel data processing without electromagnetic interference, have become a viable player for fast and energy-efficient processing and calculation to meet the increasing demand for hash rate. Photonic memories employing nonvolatile phase-change materials could achieve zero static power consumption, low thermal cross talk, large-scale, and high-energy-efficient photonic neural networks. Nevertheless, the switching speed and dynamic energy consumption of phase-change material-based photonic memories make them inapplicable for in situ training. Here, by integrating a patch of phase change thin film with a PIN-diode-embedded microring resonator, a bifunctional photonic memory enabling both 5-bit storage and nanoseconds volatile modulation was demonstrated. For the first time, a concept is presented for electrically programmable phase-change material-driven photonic memory integrated with nanosecond modulation to allow fast in situ training and zero static power consumption data processing in ONNs. ONNs with an optical convolution kernel constructed by our photonic memory theoretically achieved an accuracy of predictions higher than 95% when tested by the MNIST handwritten digit database. This provides a feasible solution to constructing large-scale nonvolatile ONNs with high-speed in situ training capability.

脑科学与类脑研究是国际必争战略性前沿。人工智能与深度学习的飞速发展对算力提出了迫切需求。而传统的冯诺依曼架构，由于存算分离导致功耗墙和内存墙，摩尔定律也逐渐放缓。光神经拟态计算充分融合高速光通信、光互连、光集成、硅基光电子与神经拟态计算的特点，具有超高速、大带宽、多维度等优势，在高性能计算、人工智能领域有广阔的应用前景，是突破后摩尔时代传统微电子计算极限极具竞争力的方案。本文回顾了国内外主要研究团队在光神经元、光突触、光神经网络的理论、算法及器件方面的工作，并提出了展望。

Optical computing provides unique opportunities in terms of parallelization, scalability, power efficiency, and computational speed and has attracted major interest for machine learning. Diffractive deep neural networks have been introduced earlier as an optical machine learning framework that uses task-specific diffractive surfaces designed by deep learning to all-optically perform inference, achieving promising performance for object classification and imaging. We demonstrate systematic improvements in diffractive optical neural networks, based on a differential measurement technique that mitigates the strict nonnegativity constraint of light intensity. In this differential detection scheme, each class is assigned to a separate pair of detectors, behind a diffractive optical network, and the class inference is made by maximizing the normalized signal difference between the photodetector pairs. Using this differential detection scheme, involving 10 photodetector pairs behind 5 diffractive layers with a total of 0.2 million neurons, we numerically achieved blind testing accuracies of 98.54%, 90.54%, and 48.51% for MNIST, Fashion-MNIST, and grayscale CIFAR-10 datasets, respectively. Moreover, by utilizing the inherent parallelization capability of optical systems, we reduced the cross-talk and optical signal coupling between the positive and negative detectors of each class by dividing the optical path into two jointly trained diffractive neural networks that work in parallel. We further made use of this parallelization approach and divided individual classes in a target dataset among multiple jointly trained diffractive neural networks. Using this class-specific differential detection in jointly optimized diffractive neural networks that operate in parallel, our simulations achieved blind testing accuracies of 98.52%, 91.48%, and 50.82% for MNIST, Fashion-MNIST, and grayscale CIFAR-10 datasets, respectively, coming close to the performance of some of the earlier generations of all-electronic deep neural networks, e.g., LeNet, which achieves classification accuracies of 98.77%, 90.27%, and 55.21% corresponding to the same datasets, respectively. In addition to these jointly optimized diffractive neural networks, we also independently optimized multiple diffractive networks and utilized them in a way that is similar to ensemble methods practiced in machine learning; using 3 independently optimized differential diffractive neural networks that optically project their light onto a common output/detector plane, we numerically achieved blind testing accuracies of 98.59%, 91.06%, and 51.44% for MNIST, Fashion-MNIST, and grayscale CIFAR-10 datasets, respectively. Through these systematic advances in designing diffractive neural networks, the reported classification accuracies set the state of the art for all-optical neural network design. The presented framework might be useful to bring optical neural network-based low power solutions for various machine learning applications and help us design new computational cameras that are task-specific.