Chiral detection of biomolecules based on reinforcement learning
Chirality is one of important properties of subjects1, 2. Especially in the aspects of medicine and biology, it is very normal that the opposite handedness of enantiomers leads to entirely different pharmacological effects. How to discriminate chiral signals attracts wide attention in recent years, but most of chiral biomolecules present weak optical signals in ultraviolet band, which increases difficulties for detection. On the other hand, it is reported that surface plasmons enhance circular dichroism (CD)3-6, and different kinds of plasmonic metamaterial exhibit a giant nonlinear optical activity (NOA)7-11. They possess significant optical chirality around the resonant frequencies in the visible and near-infrared band. Therefore, chiral metasurfaces12-19, metamaterials19-22 and nanoparticles23-26 are widely used in chiral detection. The nearfield interaction between chiral molecules and metal nanostructures causes a far-field spectrum shift which can be considered as the basis of chiral discrimination27. Complex nanostructures usually possess intrinsic chirality, and the background CD signals of nanostructures cover the interaction effects from biomolecules. Emphasizing the CD signals of biomolecules depends on shapes and optical properties of the nanostructure. It is significant to design particular structures for accurate chiral detection. Diverse types of biomolecules require different nanostructures, but the relationships between biomolecules and corresponding nanostructures are very intricate. Finding appropriate nanostructures demands extensive theoretical simulation and explorative experiments.
In recent decades, it is found that various artificial nanostructures achieve the manipulation of light based on their unique geometric distributions. However, the structure design process consumes huge amounts of computing in iterative electromagnetic simulations28. Advanced optimization algorithms rather than simple parameter sweep methods are involved in the nanophotonic design to tackle these challenges. For example, genetic algorithm and topology optimization succeed in the design of metasurface and obtain exceptional optical features29-32. As the development of artificial intelligence, the application of intelligent algorithms has been an active topic for nanophotonics28, 33-35. Due to their huge advantages on large-scale data processing, intelligent algorithms deal with the complex problem which traditional optimization algorithms cannot solve. They contribute to the design of diverse functional nanostructures including meta-lens36, meta-grating37-39, coupler40-41, beam splitters42-43 and photonic crystals44, etc. As one of the most popular artificial intelligence algorithms, deep learning models successfully decrease the cost of nanostructure design, enlarge the computable parameter space and help search for better nanostructures than classic design methods provide. However, deep learning models are usually limited by the small volume and inappropriate distribution of raw dataset28.
In this work, we present a method of nanostructure design based on reinforcement learning, which changes data generation manners and replaces data fitting with explorations of parameters. The algorithm proposes automatically the nanostructures that it wants, and learns from the spectra of these nanostructures. Artificial neural networks (ANNs) guide searching for potentially better nanostructures which are about to be calculated by electromagnetic simulations, and these results are used to update the ANNs. With this algorithm, we obtain numerous different nanostructures with a sharp peak in CD spectra in order to emphasize the interaction effects between chiral biomolecules and plasmonic. The experiment proves that the microfluidic chips with the optimized chiral structures succeed in differentiation of enantiomers. The resonance wavelength shifts between enantiomers of glucose with opposite chirality reach 7 nm. According to the cathodoluminescence (CL) spectroscopy, the structures designed by artificial intelligence possess different near-field modes for RCP and LCP components.
2 The workflow of the optimization algorithm
The circular dichroism (CD) spectrum of a typical chiral reflective metasurface illustrates the difference in reflectivity (transmissivity) between left-circularly polarized light (LCP) and right-circularly polarized light (RCP), and it is defined as
where RRCP and RLCP denote the reflectivity of the metasurface with a right-circularly polarized input and a left-circularly polarized input.
The metasurface whose CD spectrum presents a sharp peak emphasizes the phenomenon of the frequency shift caused by chiral biomolecules. When a metasurface is covered by biomolecular solution, its CD spectrum will be modulated by the biomolecules which leads to the change of the resonance frequency. As for ordinary chiral metasurface, the modulation effect is limited, because the CD spectra shift is combined with the noise of background spectra. Therefore, it is effective to design a nanostructure with a sharp chiral peak for detection of the chiral molecules.
Reinforcement learning is involved to design structures with strong chirality. The specific optimization workflow for optical properties of nanostructures based on reinforcement learning is shown in
A design algorithm based on reinforcement learning. The entire workflow is a circle of four steps ( a- d). ( a) Dataset composed of nanostructures and corresponding spectra. The spectra are calculated with electromagnetic simulations. ( b) Train different ANNs. ( c) Possible optimized structures proposed by ANNs. ( d) Classifications for possible optimized structures. Compare the results from different ANNs. The spectra are accurate and the structures are optimized when the ANNs give the same predictions. Other structures are totally new for the existed data and added to dataset.
When involving classic supervised learning to design structures of nano-photonics, it is a possible contradiction that enough training data might consume so much computing resource that machine learning does not develop the design process. Data plays a crucial role in classic supervised learning. Trained deep learning models summarize the relationship between the input data and the output data, so they cannot achieve the prediction of inputs which have nothing in common with training dataset, even for the most advanced artificial neural networks. Obviously, sufficient data is the base of supervised learning, but the generation of data cost large amounts of computing resource and time. We obtain the training data through numerical electromagnetic simulations which take much more time than the calculation of ANNs. When the training dataset covers the whole parameter space densely, involving ANNs is meaningless, because the cost of training data generation reaches or even exceeds the traditional design method. In this work, it is difficult for normal supervised learning to accurately predict spectra with a dramatic change because of the above problem. It is necessary for accurate prediction for optical response of a structure with ANNs that there are similar structures in the training dataset. As the first step of the classic supervised machine learning, it is very difficult to estimate the optical responses of structures without electromagnetic simulation while the generation of training dataset, so the structures are usually selected randomly. However, the electromagnetic simulation of all kinds of nanostructures demands more computing costs than the classical nano-photonic design does. Normal supervised learning cannot achieve the design under the limitation of calculation resources.
Reinforcement learning is an effective way for the construction of a training dataset shown in
The exploration of the parameter space is achieved through ANNs and Bayesian optimization algorithms. The first part of the optimization is the generation of the initial dataset. We create different nanostructures randomly and calculate the corresponding CD spectra by solving the Maxwell’s equations through the finite difference time-domain (FDTD) method. As
The above steps compose a self-consisted cycle which searches for nanostructures with optimized chirality. Reinforcement learning achieves the combination of parameter explorations and model updates, and declines the cost of data generation. Through this method, we obtain lots of chiral structures to detect chiral biomolecules.
3 Results and discussion
Details of the deep learning algorithm involved. We show the geometry of nanostructures ( a– c), parameterization (c), the structure of ANNs ( d) and performances of the ANNs ( e– f). (a) Periodic arrays of chiral nanostructures. (b) The side view of a structure. (c) The top view of a structure and parameterization. The geometry is encoded to 0 and 1. (d) ANN structure. ANNs analyze parameterized geometry and predict reflective spectra with a right-circularly polarized incidence. ANNs mainly consist of multiple convolutional (Conv) layers and fully connection (FC) layers. (e) ANNs’ performance in prediction of reflective spectra with a right-circularly polarized incidence. (f) ANNs’ performance in prediction of CD spectra.
The above ANN models predict optical responses successfully after the whole design process. In the first step of the design process, 1000 random nanostructures are generated as the initial dataset. As the design process searches for new structures with a strong chirality continuously, the quantity of the dataset is enlarged to 6000. In order to estimate the performance of the ANNs involved, 80% of data composes of training data and 20% data left is used as test data. After data enhancement (See Supplementary information), the training loss defined as the mean square error between results of simulations and ANNs decline to 1.22×10−4, and the test loss is 3.31×10−4. A prediction example is shown in
In order to prove the chirality of structures designed by the algorithm proposed, we fabricate three structures whose chiral peaks distribute in different wavelengths.
Fig. 3. Experiment results for optimized structures at the target wavelength of 850 nm (
a– d), 750 nm ( e– h) and 925 nm ( i– l). ( a, e, i) SEM images. ( b, f, j) CD spectra of experiments and simulations. ( c, g, k) The LCP component of CL mapping. ( d, h, l) The RCP component of CL mapping.
The cathodoluminescence (CL) spectroscopy is used to analyze optical near-field resonance modes, and the CL mapping images shown in
The frequency shift of CD spectra caused by chiral biomolecules is detected in experiments. The nanostructures locate at the bottom of microchannel structure made of Polydimethylsiloxane (PDMS) as shown in
Differentiation of enantiomers. ( a) Structures of microfluidic chips. ( b– d) CD spectra of different solutions on metasurfaces with a CD peak at the wavelength of 750 nm, 850 nm and 925 nm. The contrast of solutions of L-glucose, NaCl and D-glucose present an obvious frequency shift
The experiment results based on three kinds of nanostructures illustrate consistently that the L-glucose and R-glucose triggers frequency shifts in different directions compared to the CD spectra of NaCl solution. The shifts originate from the coupling of chiral molecules and nanostructures rather than the change of refractive index. It is a simple theory explanation that the effective refractive indexes of chiral solution for LCP and RCP are different, which leads to distinct near-field electromagnetic modes. As the CL mapping images show, the chiral resonance peaks of the metal nanostructures appear when dipole modes excited by LCP abate significantly. Enantiomers with opposite chirality possess contrary variation of effective refractive index, so the orientations of frequency shifts are different.
As shown in
At present, the chiral detection methods of biomolecules include the chiral assemblies of plasmonic nanostructures enabled by biomolecules1-4, 46, chiral metasurfaces12-19, chiral metamaterials19-21, plasmonic nanocavities47-48 and chiroptical active nanoparticles. Most of them focus on the macromolecules like DNA, RNA and proteins. The chiral detection of small molecules mainly rely on nanoparticles and chiral assemblies. It is reported that chiral nanoparticle films display chiral binding to glucose enantiomers and induce selective catalysis and glucose enantiomers exhibit highly selective photooxidation under CPL illumination4, 46. This method obtains interfacial assemblies of the monolayer NP film through evaporating mixed solution and achieves a high sensitivity. Unlike the complex process of the above method, our approach utilizes microfluidic chips to detect the chiral biomolecules constantly, which is reusable and more convenient to practice. Compared to other chiral metasurfaces, our nanostructures differentiate one type of small molecules, which can inspire more exploration of chiral detection based on metasurfaces.
In conclusion, a nanophotonics design method based on reinforcement learning is proposed to devise microfluidic chips which achieve differentiation of chiral molecules. The algorithm combines parameter explorations with model updates and successfully obtains nanostructures which possesses sharp peaks in CD spectra. ANNs do not only learn from the given training data but propose the structures whose spectra they need to know, so the process is an automatic exploration instead of supervised learning. They accurately predict the reflective spectra with a right-circularly polarized incidence within 6000 simulations, even though the reflectivity is very sensitive to the wavelength. The CL spectroscopy analyzes the near-field electromagnetic modes of nanostructures designed by the above method, and it is considered as the origin of the chirality of the nanostructures that LCP and RCP excite different near-field modes. When the solution of enantiomers with opposite chirality flows into microfluidic chips, a pair of contrary resonance frequency shifts are detected in CD spectra. It is because the coupling of chiral biomolecules and nanostructures changes the near-field modes. Therefore, the microfluidic chips designed by artificial intelligence differentiate enantiomers correctly.
The above algorithm improves classic explorations of parameter spaces with machine learning and achieves the optimization for spectrum. However, the physical meanings of spectrum are not important for the algorithm, which means that other optical properties can be optimized through this method. The paradigm of reinforcement learning can be extended to design nanostructures for other functions. In this work, the algorithm is restricted by the physical limits of the system, but the designed nanostructures successfully differentiate the enantiomers of glucose. It is reported that coupled chiral plasmonic nanostructures including plasmonic nanocavities are more suitable for chiral detection47-48. The algorithm has the potential for diverse nanophotonic circumstances, and the combination with these plasmonic nanostructures probably form better chiral detection. As an instance, the study illuminates the physics mechanism and the practicability of nanostructures proposed by artificial intelligence. It can inspire new applications of deep learning algorithm in nanophotonics and help explore physical limits.
This work is supported by the National Science Foundation of China (Grant Nos. 12027807, 62225501, and 11974002), National Key Research and Development Program of China (Grant No. 2020YFA0211300, 2020YFA0906900, and 2021YFF1200500), PKU-Baidu Fund Project (Grant No. 2020BD023), and High-performance Computing Platform of Peking University.
Z. Y. Fang supervised the project. Y. X. Chen designed the algorithm, fabricated metal nanostructures and measured circular dichroism spectra. F. Y. Zhang fabricated microfluidics chips and participated in biomolecular testings. X. He contributed to the fabrication of nanostructures. Z. B. Dang measured CL signals. C. X. Luo proposed the implementation scheme of microfluidics chips. Z. C. Liu, P. Peng and Y. Li participated in the design of neural networks. All the authors contributed to the discussion and analysis of the project.
The authors declare no competing financial interests.
 Cao ZL, Gao H, Qiu M, Jin W, Dem sub-nanometer biochemical molecules to sub-micrometer plasmonic metastructures:physiochemical mechanisms, biosensing, and bioimaging opportuniting SZ et alChirality transfer froesAdv Mater202032190715110.1002/adma.201907151
 Chen SM, Zeuner F, Weismann M, Reineke B, Li GX et alGiant nonlinear optical activity of achiral origin in planar metasurfaces with quadratic and cubic nonlinearitiesAdv Mater2016282992299910.1002/adma.201505640
 Solomon ML, Hu J, Lawrence M, García-Etxarri A, Dionne JAEnantiospecific optical enhancement of chiral sensing and separation with dielectric metasurfacesACS Photonics201964349
 Ben-Moshe A, Teitelboim A, Oron D, Markovich GProbing the interaction of quantum dots with chiral capping molecules using circular dichroism spectroscopyNano Lett2016167467747310.1021/acs.nanolett.6b03143
 Palermo G, Lio GE, Esposito M, Ricciardi L, Manoccio M et alBiomolecular sensing at the interface between chiral metasurfaces and hyperbolic metamaterialsACS Appl Mater Interfaces202012301813018810.1021/acsami.0c07415
 Chen Z, Wang Q, Wu X, Li Z, Jiang YBOptical chirality sensing using macrocycles, synthetic and supramolecular oligomers/polymers, and nanoparticle based sensorsChem Soc Rev2015444249426310.1039/C4CS00531G
 Gwak J, Park SJ, Choi HY, Lee JH, Jeong KJ et alPlasmonic enhancement of chiroptical property in enantiomers using a helical array of magnetoplasmonic nanoparticles for ultrasensitive chiral recognitionACS Appl Mater Interfaces202113468864689310.1021/acsami.1c14047
 Hendry E, Carpy T, Johnston J, Popland M, Mikhaylovskiy RV et alUltrasensitive detection and characterization of biomolecules using superchiral fieldsNat Nanotechnol2010578378710.1038/nnano.2010.209
 Chen YQ, Hu YQ, Zhao JY, Deng YS, Wang ZL et alTopology optimization-based inverse design of plasmonic nanodimer with maximum near-field enhancementAdv Funct Mater202030200064210.1002/adfm.202000642
 Fan YL, Chen MK, Qiu M, Lin RJ, Xu YK et alExperimental demonstration of genetic algorithm based metalens design for generating side-lobe-suppressed, large depth-of-focus light sheetLaser Photonics Rev202216210042510.1002/lpor.202100425
 Mansouree M, McClung A, Samudrala S, Arbabi ALarge-scale parametrized metasurface design using adjoint optimizationACS Photonics20218455463
 Gostimirovic D, Ye WNAn open-source artificial neural network model for polarization-insensitive silicon-on-insulator subwavelength grating couplersIEEE J Sel Top Quantum Electron2019258200205
 Melati D, Grinberg Y, Dezfouli MK, Janz S, Cheben P et alMapping the global design space of nanophotonic components using machine learning pattern recognitionNat Commun201910477510.1038/s41467-019-12698-1
 Li GC, Zhang Q, Maier SA, Lei DYPlasmonic particle-on-film nanocavities: a versatile platform for Plasmon-enhanced spectroscopy and photochemistryNanophotonics201871865188910.1515/nanoph-2018-0162
 Bao ZY, Dai JY, Zhang Q, Ho KH, Li SQ et alGeometric modulation of induced plasmonic circular dichroism in nanoparticle assemblies based on backaction and field enhancementNanoscale201810196841969110.1039/C8NR07300G
Yuxiang Chen, Fengyu Zhang, Zhibo Dang, Xiao He, Chunxiong Luo, Zhengchang Liu, Pu Peng, Yuchen Dai, Yijing Huang, Yu Li, Zheyu Fang. Chiral detection of biomolecules based on reinforcement learning[J]. Opto-Electronic Science, 2023, 2(1): 220019.