Delay-weight plasticity-based supervised learning in optical spiking neural networks
We propose a modified supervised learning algorithm for optical spiking neural networks, which introduces synaptic time-delay plasticity on the basis of traditional weight training. Delay learning is combined with the remote supervised method that is incorporated with photonic spike-timing-dependent plasticity. A spike sequence learning task implemented via the proposed algorithm is found to have better performance than via the traditional weight-based method. Moreover, the proposed algorithm is also applied to two benchmark data sets for classification. In a simple network structure with only a few optical neurons, the classification accuracy based on the delay-weight learning algorithm is significantly improved compared with weight-based learning. The introduction of delay adjusting improves the learning efficiency and performance of the algorithm, which is helpful for photonic neuromorphic computing and is also important specifically for understanding information processing in the biological brain.
1. INTRODUCTION
As the improvement of traditional neural networks has gradually approached an upper limit, research focuses on neural networks with more biological reality. The spiking neural networks (SNNs), normally known as the third generation of artificial neural networks (ANNs), are more biologically plausible than previous ANNs [1] and have attracted more and more attention in recent decades [2–6]. Spikes transmitted in the biological neural networks enable the network to capture the rich dynamics of neurons and to integrate different information dimensions [3]. However, the information representation and processing manner has become a controversial issue and remains very challenging.
Rate coding is widely used in traditional SNNs; however, there is biological evidence that the precise timing of spikes also conveys information in nervous systems [7–9]. The precise timing of spikes enables higher information encoding capacity and lower power consumption [10–12], which are extremely important in information processing of the human brain. However, the exact learning mechanism still remains an open problem [13]. It has been shown that in the cerebellum and the cerebellar cortex, there exist signals that act like an instructor that helps the processing of information [14,15]. Several supervised learning algorithms have been proposed upon which specific problems that are tightly related to neural processing such as spike sequence learning and pattern recognition have been solved successfully [16–20]. Remote supervised method (ReSuMe) is one of the supervised learning algorithms originally derived from the well-known Widrow-Hoff rule [17]. Based on photonic spike-timing-dependent plasticity (STDP) and anti-STDP rules, the synaptic weights can be adjusted to train the output neuron to fire spikes at the desired time.
Time-delayed transmission is an intrinsic feature in neural networks. Biological evidence shows that the transmission velocities in the nervous system can be modulated [21,22], for example, by changing the length and thickness of dendrites and axons [23]. The adjustability of both delay and weight of a synapse is referred to as synaptic plasticity. Delay plasticity has also been found to be helpful for the neuron in changing its firing behavior and synchronization, and it helps to understand the process of learning [24,25]. Delay selection and delay shift are two basic approaches incorporated in delay learning works [26,27]. To be specific, delay selection is to strengthen the weight of an optimal synapse among multiple subconnections, and delay shift is a more biologically plausible method training neurons to fire with coincident input spikes and with constant weight. Recently, researchers found that combined adjustment of delay and weight enhances the performance of an SNN [28–32]. In 2015, a delay learning remote supervised method (DL-ReSuMe) for spiking neurons was proposed to merge the delay shift approach and weight adjustment based on ReSuMe, by which the learning accuracy and learning speed are enhanced [30]. In 2018, Taherkhani
For the sake of emulating realistic biological behaviors, SNN hardware realizations are designed to seek ultralow power consumption [5]. Devices for the implementation of basic elements of SNN, namely, artificial spiking neurons and synapses, have been achieved via complementary metal-oxide-semiconductors (CMOS), transistors, and the emerging nonvolatile memory technologies [33–38], etc. Photonic neuromorphic systems have attracted attention for being a potential candidate in applications of ultrafast processing. Despite its similarity with biological neurons [39], the semiconductor laser also exceeds its electronic counterpart in its ultrafast response and low power consumption. Numerous studies on photonic synapses and photonic neurons [40–53] have laid a solid foundation for significant progress in photonic neuromorphic computing based on both software and hardware implementations [54–59]. In 2019, an all-optical SNN with self-learning capacity was physically implemented on a nanophotonic chip, which is capable of supervised and unsupervised learning [55]. In 2020, we proposed to solve XOR in an all-optical neuromorphic system with inhibitory dynamics of a single photonic spiking neuron based on vertical-cavity surface-emitting lasers (VCSELs) with an embedding saturable absorber (VCSEL-SA) [58], and an all-optical spiking neural network based on VCSELs was also proposed for supervised learning and pattern classification [59]. However, as far as we know, delay learning has not yet been applied in photonic SNNs.
In this work, we propose to incorporate delay learning with the traditional algorithm based on an optical SNN. First, we propose a modified algorithm that combines delay learning with ReSuMe in a photonic SNN, which adjusts synaptic weight and delay simultaneously. By implementing a spike sequence learning task, better performance and learning efficiency of the proposed algorithm than that of its weight-based counterpart are verified. Then, the proposed algorithm is also implemented for classification, where two benchmarks, the Iris data set and the breast cancer data set, are adopted. By applying the delay-weight (DW)-based algorithm, the testing accuracy for both benchmarks is significantly improved (reaching 92%).
2. SYSTEM MODEL
2.1 A. Photonic Neurons and Synapses Based on VCSEL-SA and VCSOA
The photonic neurons and synapses are the basic elements in a photonic SNN. VCSEL-SA can mimic spiking dynamics of a biological neuron [51] and VCSEL that works below threshold value can serve as a vertical-cavity semiconductor optical amplifier (VCSOA) that is able to perform the STDP function [47], which provides possibilities of large-scale integration and low power consumption in a photonic SNN. In this work, the spiking dynamics are implemented via the excitable VCSEL-SA neurons. The rate equations of a VCSEL-SA are written as follows [57]:
The term in Eq. (2) describes the pre-coded square-wave pulses injected as external stimulus into PREs, and represents the weighted sum of all of the pre-synaptic spikes fed into the POST. is the adjustable transmission delay from PREi to the PSOT. , , and denote the strength, the central timing, and the temporal duration of the pulse, respectively. is the time interval between two adjacent input pulses. is the coupling weight between the PREi and the POST that can be tuned according to the supervised training method. The output power of PREs and POST can be calculated by . In practice, the is calculated as an initial weight multiplying a constant coefficient to match the optical system. The remaining parameters are the same for all neurons as in Ref. [56]. The rate equations are numerically solved by using the fourth-order Runge-Kutta method.
2.2 B. DW-Based Learning Algorithm
In most of the algorithms with tasks tightly related to neural processing, such as spike sequence learning and pattern recognition, only weight adjustment is considered. However, the time delay from the PREs to the POST is hardly considered, which may play a vital role in brain computing [31]. Under this consideration, the time-delay plasticity is combined with the ReSuMe algorithm. The weight and delay changes of the synapse after each training epoch are Schematic illustration of the ReSuMe incorporated with optical STDP rule. Schematic illustration of the ReSuMe incorporated with optical STDP rule.
and in Hebbian terms represent that this rule only modifies inputs that contribute to the neuron state before the desired or actual output firing time but neglect those inputs that fire afterward, which leads to better performance of the proposed DW-based algorithm.
The term () is the distance between and (). The delay adjustment is based on the distance between input spikes and output spikes (target spikes), in a way similar to that of synaptic weight. Note that both and approach 0 if the POST fires at the desired times, which ensures the convergence of the proposed algorithm. In addition, the synaptic weights (delays) are adjusted with a learning rate and within a learning window . The weight and delay of the synapse are adjusted only if the input spike distance is less than the learning window. Finally, the weight and delay of the synapse are updated by
A simple case for delay learning is illustrated in Fig. 3. Consider two PREs; each fires a spike at and , respectively. After delayed transmission through the synapses, the actual input spikes locate at and [see in Fig. 3(a1)]. The principle of delay learning is to shift the actual input spikes toward the desired time, also called coherence learning, in which several coherent input spikes trigger the POST to fire a spike very shortly after the input time [Figs. 3(a2) and 3(b2)], while a single input spike cannot enable the release of spikes at the desired time [Fig. 3(b1)]. (a1) and (b1) Input pattern and output pattern before delay adjustment. (a2) and (b2) After 7 training epochs. (a1) and (b1) Input pattern and output pattern before delay adjustment. (a2) and (b2) After 7 training epochs.
3. RESULTS AND DISCUSSION
The learning capability of a single neuron is highly related to the property of a neural network. In Fig. 4, we compare the performance of the weight-based ReSuMe and the DW-ReSuMe in recognizing a single spike, where , , and . To quantify convergence rate, the value of spike sequence distance (SSD) (defined in Ref. [57]) is adopted. The learning window is 3 ns, while is not constrained. The input spiking time varies from 4 ns to 7 ns. The time range within which an input pattern can be successfully learned, denoted as valid input window, suggests the learning ability of a neuron. The SSD at different training epochs is presented in Fig. 4 for the two algorithms. The valid input window for ReSuMe and DW-ReSuMe is 0.3 ns (4.4–4.7) and 2.4 ns (4.4–6.8) at the 50th training epoch, respectively, as shown in Figs. 4(a) and 4(b). It is shown that with DW-ReSuMe, the simple network is able to learn a single input spike with wider time range and with higher precision. Not surprisingly, with the increase of learning epochs, the valid input window also becomes a little wider. Note that weight learning is crucial when the single input is too weak to trigger an output spike in the POST. Here, we emphasize the importance of initial weight and weight learning rate on the performance of DW-based learning. When is too small, a larger learning rate of weight is required to generate an output spike before the delay learning has shifted the input spiking time right at that of the target. As can be seen in Fig. 4(c), the efficient input window is widened by increasing the learning rate from to . In addition, for large , smaller is required to avoid missing the optimal solution, as indicated in Fig. 4(d). The initial weight ranges from a unit value of 0.1 to 2, and smaller contributes to a larger efficient input window. We suggest relatively larger weight learning rate and smaller initial weight for spike sequence learning. For a classification problem, should be large enough to trigger a spike of the PREs, while smaller is necessary to obtain higher accuracy. Comparison of the learning capability of a single neuron based on (a) weight-based ReSuMe and (b) DW-ReSuMe. The value of SSD after the 50th, 100th, and 300th training epoch is presented for different Comparison of the learning capability of a single neuron based on (a) weight-based ReSuMe and (b) DW-ReSuMe. The value of SSD after the 50th, 100th, and 300th training epoch is presented for different
However, note that if the desired output contains more than one spike, the delay learning window has to be limited within the range of the minimum inter-spike interval (ISI) to maintain stable performance. Since some of the input spikes are shifted toward a certain target spike, there is an additional consideration in the DW-ReSuMe that the injection power of the output neuron should be limited to protect devices of photonic neurons and to ensure spiking dynamics.
3.3 A. Spike Sequence Learning
Then a spike sequence learning task is implemented via a single-layer photonic SNN. Both DW-ReSuMe and ReSuMe are considered for comparison. The PRE includes 60 input neurons stimulated via pre-coded rectangular pulses with a time interval of 0.1 ns, each connected to the POST with a photonic synapse. In Fig. 5 we show the learning process of the two algorithms. The black line in Fig. 5(a1) describes the carrier density of the POST after training of 300 epochs based on DW-ReSuMe. When exceeds a threshold value (marked in red dotted line), a spike is emitted. We can see that after training, the POST is able to fire accurately at the desired time (denoted by blue dotted line). The training process is further illustrated in Fig. 5(b1), where the SSD converges quickly from 2 to 0 within about 20 training epochs. (a1) Carrier density of the POST after training and (b1) the evolution of output spikes based on the DW-ReSuMe; (a2) and (b2) those based on ReSuMe. The black solid line is (a1) Carrier density of the POST after training and (b1) the evolution of output spikes based on the DW-ReSuMe; (a2) and (b2) those based on ReSuMe. The black solid line is
The evolution of synaptic weights and delays is shown in Fig. 6. The initial weights and delays of all neurons are identical, as indicated by Figs. 6(a) and 6(b), respectively. After training, some of the weights and delays are potentiated, while others are depressed or hardly changed. It is interesting to note that for some synapses, the weights and (or) delays change obviously at the first six training epochs. Such fluctuations during training are mainly caused by the combined effect of delay learning and weight learning. That is, when initially the output spiking time is far before the target , the update amount of both weight and delay is relatively large as the distance between and is large. In this case, the combined adjustment of weight and delay may make lag far behind but closer to than in the previous epoch. Note that the weight-based ReSuMe is not able to solve this problem in an SNN with 60 PREs, as shown in Figs. 5(a2) and 5(b2). The results show that combined with delay learning, DW-ReSuMe is more powerful than the weight-based algorithm. Evolution of (a1) synaptic weights Evolution of (a1) synaptic weights
Fig. 7. Learning spike sequences with ununiformed ISI. (a1) and (b1) The evolution of output spikes for spike sequence [10 ns, 12 ns, 14 ns, 18 ns, 20 ns, 22 ns, 24 ns, 26 ns, 29 ns] and [10 ns, 11 ns, 13 ns, 14.5 ns, 17 ns, 21 ns, 23 ns, 25.5 ns, 27 ns], respectively. (a2) and (b2) The evolution for the corresponding distance.
3.4 B. Fisher’s Iris Data Set
The Fisher’s Iris flower data set (Fisher, 1936) is a classic benchmark of pattern recognition that contains three classes of Iris flowers with a total of 150 case entries [31]. One of the classes is linearly separable from the other two, while the other two classes are linearly inseparable. Four measurements are used to describe and differentiate the three classes, and each measurement is directly encoded into single spike firing at different times and is linearly rescaled into the interval of [5 ns, 10 ns]. The network comprises 4 PREs and a single POST. The encoding spikes are fed into the 4 PREs of the SNN via synapses with adjustable delay and weight. The initial delays are 2 ns for all of the four input neurons, and the initial weight for each synapse is randomly selected as a constant coefficient (which is 0.1 here) multiplied by a random number from [0,1]. In this case, nearly all of the PRE neurons are capable of generating just a single spike in response of each input pattern. The weight learning rate is 0.01 and decays by half every 20 training epochs to enhance the convergence of learning. The output of the network is represented by the precise spiking time of the POST, which fires a desired spike at different times for different classes. Here, the target spikes for the three classes are 8 ns, 9 ns, and 10 ns, respectively. If the output spike locates within 40% the interval of target spikes around a certain , the input entry is classified into the corresponding class.
In our scheme, according to Ref. [31], 50% of the IRIS data are used for training and the rest for testing. The training and testing processes are implemented through program simulation based on MATLAB. For each training epoch, all of the entries in the training data set are injected into the input neurons of the SNN successively. Based on the learning algorithm, the () of the -th input synapse for each entry is calculated and summed as (). After each training epoch, the mean value of () is used for weight (delay) update, namely, the actual update amount for weight (delay) is (), where is the size of the training data set. After each training epoch, the updated weights and delays are also used for testing. The accuracy is defined as the number of correctly classified entries divided by the total entry number. Both training accuracy and testing accuracy with different learning algorithms are presented in Fig. 8. Based on the DW-ReSuMe, the accuracy arises rapidly at first and gradually approaches 96% for the training data set and 92% for the testing data set, as indicated by the red solid line. (a) Training accuracy and (b) testing accuracy varying with training epochs for weight-based ReSuMe (blue solid line) and DW-ReSuMe (red solid line). (a) Training accuracy and (b) testing accuracy varying with training epochs for weight-based ReSuMe (blue solid line) and DW-ReSuMe (red solid line).
For a detailed illustration of the classification results, a scattered plot of the target spiking time and actual output spiking time is shown in Figs. 9(a) and 9(b) for the training data set and testing data set, respectively. However, based on the same SNN architecture, the accuracy of the weight-based ReSuMe (blue solid line) only reaches 65%, with the same initial weights and delays, but a higher learning rate () is required to make the algorithm work efficiently. The results indicate that the performance can be greatly enhanced by introducing time-delay learning. Illustration of classification results for (a) training data set and (b) testing data set. The orange cycles denote target spiking time, the blue squares represent the actual spiking time, and misclassified samples are highlighted in bright blue. Illustration of classification results for (a) training data set and (b) testing data set. The orange cycles denote target spiking time, the blue squares represent the actual spiking time, and misclassified samples are highlighted in bright blue.
Moreover, we also investigate the effect of learning window on the classification accuracy. The testing accuracy as a function of weight learning window and delay learning window is shown in Figs. 10(a) and 10(b), respectively. We can see that a relatively larger is required for achieving higher accuracy. However, should be selected as the minimum ISI of target spikes, which is 1 ns. Note that the accuracy can reach more than 85% even without the adjustment of weight. The results indicate that delay learning is an extremely efficient learning algorithm in a photonic SNN with temporal encoding, which suggests that delay learning may be an essential mechanism in the biological spiking neuron systems. Testing accuracy as a function of (a) weight learning window Testing accuracy as a function of (a) weight learning window
3.5 C. Breast Cancer Data Set
The breast cancer data set contains 608 case entries, and, is divided into benign and malignant cases, each has nine measurements [62]. Five significant measurements are selected and pre-encoded into rectangular pulses, which trigger the input neurons to fire spikes at specific time within the range of 6.5 to 11 ns. The network contains 5 PREs and 1 POST. The initial delays are 0 ns for all of the 5 PREs, and the initial weights are randomly selected using the same method as for the Iris data set. The weight learning rate is 0.0001. The SNN is trained to fire a spike at 9 ns for the first case and at 13 ns for the second case. Four hundred eight entries are used to train the network, with the rest for testing. The accuracy on the training data set and testing data set reaches 93% and 92%, respectively, as shown in Figs. 11(a) and 11(b). The training data set is trained for over 100 epochs, and the accuracy converges quickly within 20 training epochs. However, with weight-based ReSuMe, the classification accuracy is less than 30% with the same operating parameters (not shown here). It is worth noting that in this case, the ReSuMe algorithm does not work efficiently, limited by the simple network structure and encoding schemes. The accuracy mainly depends on the initial synaptic delay . In Fig. 11, we also present the training results based on ReSuMe (blue solid line), with an initial delay of 0.5 ns. The weight learning rate here is 0.01. (a) Training accuracy and (b) testing accuracy varying with training epochs based on DW-ReSuMe (red solid line) and ReSuMe (blue solid line), respectively. (a) Training accuracy and (b) testing accuracy varying with training epochs based on DW-ReSuMe (red solid line) and ReSuMe (blue solid line), respectively.
Moreover, in consideration of the effect of initial delay on the training performance of ReSuMe and DW-ReSuMe, we show in Fig. 12 the accuracy of the two data sets after 60 training epochs as a function of different initial delays based on the two algorithms. Both training and testing results are considered. Figures 12(a1) and 12(a2) show the training and testing accuracy of the Iris data set, respectively, and the results of the breast cancer data set are shown in Figs. 12(b1) and 12(b2). Obviously, the proposed DW-based algorithm is less sensitive to varying initial delays. Note that in the Iris data set, the learning accuracy of DW-ReSuMe is small when is less than 1.5 ns, which is mainly constrained by the relatively small delay learning window (1 ns). (a1) Training accuracy and (a2) testing accuracy for the Iris data set after 60 training epochs with different initial delay (a1) Training accuracy and (a2) testing accuracy for the Iris data set after 60 training epochs with different initial delay
Finally, note that in Fig. 11 there is a downward fluctuation when the accuracy is close to the stable state. The fluctuations may be related to learning rate. A larger learning rate usually generates more obvious fluctuations, as illustrated in Figs. 13(a1) and 13(a2). The training accuracy fluctuates more obviously when the weight learning rate is larger. However, at about the 10th training cycle, the obvious downward fluctuation is not affected by weight learning rate. We can reasonably assume that the fluctuation is caused by the relatively larger constant delay learning rate (). As this value contributes to better training performance in different tasks, we consider a decaying that is reduced by half after about five training epochs. The results are presented in Figs. 13(b1) and 13(b2), from which we can see that the fluctuation disappears. However, not surprisingly, the accuracy grows much more slowly. Learning accuracy of the breast cancer data set based on DW-ReSuMe with different cases of Learning accuracy of the breast cancer data set based on DW-ReSuMe with different cases of
DW-based learning has shown excellent performance in single-layer networks. However, the real neural networks are usually hierarchical, and synaptic weights and delays can be modulated based on different biological mechanisms, which form the foundations of complex brain functions. It is quite interesting and challenging to consider how to effectively introduce delay adjustment into deep learning networks.
4. CONCLUSION
This paper proposed a supervised DW learning method in an optical SNN. Based on precise timing of spikes, delay learning trains coherent inputs in the input layer of an SNN via shifting the synaptic delays according to the desired and actual output timing. The proposed DW-ReSuMe is applied to spike sequence learning and two classification benchmarks, the Iris data set and breast cancer data set, successfully. The performance of the SNN is significantly improved compared with its weight-based counterpart. By introducing time-delay learning in a photonic SNN, fewer optical neurons are required to solve different tasks, which is significant to photonic neuromorphic computing. The results also suggest that synaptic delay and weight may be a combined learning mechanism in real biological neural networks, which deserves deeper investigation.
[1] S. Ghosh-Dastidar, H. Adeli. Spiking neural networks. Int. J. Neural Syst., 2009, 19: 295-308.
[2] R. Brette, M. Rudolph, T. Carnevale, M. Hines, D. Beeman, J. M. Bower, M. Diesmann, A. Morrison, P. H. Goodman, F. C. Harris, M. Zirpe, T. Natschläger, D. Pecevski, B. Ermentrout, M. Djurfeldt, A. Lansner, O. Rochel, T. Vieville, E. Muller, A. P. Davison, S. El Boustani, A. Destexhe. Simulation of networks of spiking neurons: a review of tools and strategies. J. Comput. Neurosci., 2007, 23: 349-398.
[3] N. Kasabov, K. Dhoble, N. Nuntalid, G. Indiveri. Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition. Neural Netw., 2013, 41: 188-201.
[4] K. Roy, J. Akhilesh, P. Priyadarshini. Towards spike-based machine intelligence with neuromorphic computing. Nature, 2019, 575: 607-617.
[5] W. Q. Zhang, B. Gao, J. S. Tang, P. Yao, H. Q. Wu. Neuro-inspired computing chips. Nat. Electron., 2020, 3: 371-382.
[6] T. Clarence, M. Arlija, N. Kasabov. Spiking neural networks: background, recent development and the NeuCube architecture. Neural Process. Lett., 2020, 3: 1675-1701.
[7] A. Cariani. Temporal codes and computations for sensory representation and scene analysis. IEEE Trans. Neural Netw., 2004, 15: 1100-1111.
[8] S. M. Bohte. The evidence for neural information processing with precise spike-times: a survey. Natural Comput., 2004, 3: 195-206.
[9] A. Mohemmed, S. Schliebs. Training spiking neural networks to associate spatio-temporal input-output spike patterns. Neurocomputing, 2013, 107: 3-10.
[10] S. B. Laughlin, R. R. de Ruyter van Steveninck, J. C. Anderson. The metabolic cost of neural information. Nat. Neurosci., 1998, 1: 36-41.
[11] S. B. Laughlin. Energy as a constraint on the coding and processing of sensory information. Curr. Opinion Neurobiol., 2001, 11: 475-480.
[13] J. Hu, H. Tang, K. C. Tan, H. Li, L. Shi. A spike-timing-based integrated model for pattern recognition. Neural Comput., 2013, 25: 450-472.
[14] F. Ponulak, A. Kasinski. Introduction to spiking neural networks: information processing, learning and applications. Acta Neurobiol. Experim., 2011, 71: 409-433.
[15] H. Jörntell, C. Hansel. Synaptic memories upside down: bidirectional plasticity at cerebellar parallel fiber-Purkinje cell synapses. Neuron, 2006, 52: 227-238.
[16] R. Gütig, H. Sompolinsky. The tempotron: a neuron that learns spike timing-based decisions. Nat. Neurosci., 2006, 9: 420-428.
[17] F. Ponulak, A. Kasinski. Supervised learning in spiking neural networks with ReSuMe: sequence learning, classification, and spike shifting. Neural Comput., 2010, 22: 467-510.
[18] I. Sporea, A. Grüning. Supervised learning in multilayer spiking neural networks. Neural Comput., 2013, 25: 473-509.
[19] S. R. Kulkarni, B. Rajendran. Spiking neural networks for handwritten digit recognition—supervised learning and network optimization. Neural Netw., 2018, 103: 118-127.
[20] C. Hong, X. Wei, J. Wang, B. Deng, H. Yu, Y. Che. Training spiking neural networks for cognitive tasks: a versatile framework compatible with various temporal codes. IEEE Trans. Neural Netw. Learn. Syst., 2020, 31: 1285-1296.
[21] S. Boudkkazi, L. Fronzaroli-Molinieres, D. Debanne. Presynaptic action potential waveform determines cortical synaptic latency. J. Physiol., 2011, 589: 1117-1131.
[22] J. W. Lin, D. S. Faber. Modulation of synaptic delay during synaptic plasticity. Trends Neurosci., 2002, 25: 449-455.
[23] C. W. Eurich, K. Pawelzik, U. Ernst, J. D. Cowan, J. G. Milton. Dynamics of self-organized delay adaptation. Phys. Rev. Lett., 1999, 82: 1594-1597.
[24] X. B. Gong, Y. B. Wang, B. Ying. Delay-induced firing behavior and transitions in adaptive neuronal networks with two types of synapses. Sci. China Chem., 2012, 56: 222-229.
[25] M. Dhamala, V. K. Jirsa, M. Ding. Enhancement of neural synchrony by time delay. Phys. Rev. Lett., 2004, 92: 074104.
[26] S. Ghosh-Dastidar, H. Adeli. Improved spiking neural networks for EEG classification and epilepsy and seizure detection. Integr. Comput.-Aided Eng., 2007, 14: 187-212.
[27] P. Adibi, M. R. Meybodi, R. Safabakhsh. Unsupervised learning of synaptic delays based on learning automata in an RBF-like network of spiking neurons for data clustering. Neurocomputing, 2005, 64: 335-357.
[28] S. Ghosh-Dastidar, H. Adeli. A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection. Neural Netw., 2009, 22: 1419-1431.
[30] A. Taherkhani, A. Belatreche, Y. Li, L. P. Maguire. DL-ReSuMe: a delay learning-based remote supervised method for spiking neurons. IEEE Trans. Neur. Netw. Learn. Syst., 2015, 26: 3137-3149.
[31] A. Taherkhani, A. Belatreche, Y. Li, L. P. Maguire. A supervised learning algorithm for learning precise timing of multiple spikes in multilayer spiking neural networks. IEEE Trans. Neur. Netw. Learn. Syst., 2018, 29: 5394-5407.
[32] M. L. Zhang, J. B. Wu, A. Belatreche, Z. H. Pan, X. R. Xie, Y. S. Chua. Supervised learning in spiking neural networks with synaptic delay-weight plasticity. Neurocomputing, 2020, 409: 103-118.
[33] W. Xu, S. Y. Min, H. Hwang, T. W. Lee. Organic core-sheath nanowire artificial synapses with femtojoule energy consumption. Sci. Adv., 2016, 2: e1501326.
[34] I. Sourikopoulos, S. Hedayat, C. Loyez, F. Danneville, V. Hoel, E. Mercier, A. Cappy. A 4-fJ/spike artificial neuron in 65 nm CMOS technology. Front. Neurosci., 2017, 11: 123.
[35] J. D. Zhu, Y. C. Yang, R. D. Jia, Z. X. Liang, W. Zhu, Z. A. Rehman, L. Bao, X. X. Zhang, Y. M. Cai, L. Song, R. Huang. Ion gated synaptic transistors based on 2D van der Waals crystals with tunable diffusive dynamics. Adv. Mater., 2018, 30: 1800195.
[36] B. Irem, L. G. Manuel, S. R. Nandakumar, M. Timoleon, P. Thomas, R. Bipin, L. Yusuf, S. Abu, E. Evangelos. Neuromorphic computing with multi-memristive synapses. Nat. Commun., 2018, 9: 2514.
[38] S. G. Hu, G. C. Qiao, Y. A. Liu, L. M. Rong, Q. Yu, Y. Liu. An improved memristor model connecting plastic synapse synapses and nonlinear spiking neuron. J. Phys. D, 2019, 52: 275402.
[39] J. Ohtsubo, R. Ozawa, M. Nanbu. Synchrony of small nonlinear networks in chaotic semiconductor lasers. Jpn. J. Appl. Phys., 2015, 54: 072702.
[40] A. Hurtado, I. D. Henning, M. J. Adams. Optical neuron using polarization switching in a 1550 nm-VCSEL. Opt. Express, 2010, 18: 25170-25176.
[41] A. Hurtado, K. Schires, I. Henning, M. Adams. Investigation of vertical cavity surface emitting laser dynamics for neuromorphic photonic systems. Appl. Phys. Lett., 2012, 100: 103703.
[42] M. P. Fok, Y. Tian, D. Rosenbluth, P. R. Prucnal. Pulse lead/lag timing detection for adaptive feedback and control based on optical spike-timing-dependent plasticity. Opt. Lett., 2013, 38: 419-421.
[43] Q. Li, Z. Wang, Y. Le, C. Sun, X. Song, C. Wu. Optical implementation of neural learning algorithms based on cross-gain modulation in a semiconductor optical amplifier. Proc. SPIE, 2016, 10019: 100190E.
[44] S. Y. Xiang, Y. H. Zhang, X. X. Guo, J. F. Li, A. J. Wen, W. Pan, Y. Hao. Cascadable neuron-like spiking dynamics in coupled VCSELs subject to orthogonally polarized optical pulse injection. IEEE J. Sel. Top. Quantum Electron., 2017, 23: 1700207.
[45] T. Deng, J. Robertson, A. Hurtado. Controlled propagation of spiking dynamics in vertical-cavity surface-emitting lasers: towards neuromorphic photonic networks. IEEE J. Sel. Top. Quantum Electron., 2017, 23: 1800408.
[46] Z. G. Chen, R. Carlos, W. H. P. Pernice, C. D. Wrigh, H. Bhaskara. On-chip photonic synapse. Sci. Adv., 2017, 3: e1700160.
[47] S. Y. Xiang, J. K. Gong, Y. H. Zhang, X. X. Guo, A. Wen, Y. Hao. Numerical implementation of wavelength-dependent photonic spike timing dependent plasticity based on VCSOA. IEEE J. Quantum Electron., 2018, 54: 8100107.
[48] Y. H. Zhang, S. Y. Xiang, J. K. Gong, X. X. Guo, A. J. Wen, Y. Hao. Spike encoding and storage properties in mutually coupled vertical-cavity surface-emitting lasers subject to optical pulse injection. Appl. Opt., 2018, 57: 1731-1737.
[49] I. Chakraborty, G. Saha, A. Sengupta, K. Roy. Toward fast neural computing using all-photonic phase change spiking neurons. Sci. Rep., 2018, 8: 12980.
[50] I. Chakraborty, G. Saha, K. Roy. Photonic in-memory computing primitive for spiking neural networks using phase-change materials. Phys. Rev. Appl., 2019, 11: 014063.
[51] Y. H. Zhang, S. Y. Xiang, X. Guo, A. Wen, Y. Hao. All-optical inhibitory dynamics in photonic neuron based on polarization mode competition in a VCSEL with an embedded saturable absorber. Opt. Lett., 2019, 44: 1548-1551.
[52] J. Robertson, E. Wade, Y. Kopp, J. Bueno, A. Hurtado. Toward neuromorphic photonic networks of ultrafast spiking laser neurons. IEEE J. Sel. Top. Quantum Electron., 2020, 26: 7700715.
[53] B. W. Ma, W. W. Zou. Demonstration of a distributed feedback laser diode working as a graded-potential-signaling photonic neuron and its application to neuromorphic information processing. Sci. China Inf. Sci., 2020, 63: 160408.
[54] S. Y. Xiang, Y. H. Zhang, J. K. Gong, X. X. Guo, L. Lin, Y. Hao. STDP-based unsupervised spike pattern learning in a photonic spiking neural network with VCSELs and VCSOAs. IEEE J. Sel. Top. Quantum Electron., 2019, 25: 1700109.
[55] J. Feldmann, N. Youngblood, C. D. Wright, H. Bhaskaran, W. H. P. Pernice. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature, 2019, 569: 208-215.
[56] C. Ríos, N. Youngblood, Z. Cheng, M. L. Gallo, W. H. P. Pernice, C. D. Wright, A. Sebastian, H. Bhaskaran. In-memory computing on a photonic platform. Sci. Adv., 2019, 5: eaau5759.
[57] Z. W. Song, S. Y. Xiang, Z. X. Ren, G. Q. Han, Y. Hao. Spike sequence learning in a photonic spiking neural network consisting of VCSELs-SA with supervised training. IEEE J. Sel. Top. Quantum Electron., 2020, 26: 1700209.
[58] S. Y. Xiang, Z. X. Ren, Y. H. Zhang, Z. W. Song, Y. Hao. All-optical neuromorphic XOR operation with inhibitory dynamics of a single photonic spiking neuron based on VCSEL-SA. Opt. Lett., 2020, 45: 1104-1107.
[61] G. Wetzstein, A. Ozcan, S. Gigan, S. Fan, D. Englund, M. Soljačić, C. Denz, D. A. B. Miller, D. Psaltis. Inference in artificial intelligence with deep optics and photonics. Nature, 2020, 588: 39-47.
[62] A. Roy, S. Govil, R. Miranda. An algorithm to generate radial basis function (RBF)-like nets for classification problems. Neural Netw., 1995, 8: 179-201.
Article Outline
Yanan Han, Shuiying Xiang, Zhenxing Ren, Chentao Fu, Aijun Wen, Yue Hao. Delay-weight plasticity-based supervised learning in optical spiking neural networks[J]. Photonics Research, 2021, 9(4): 0400B119.
PDF全文
