PFN Internship 2023 / Hagai Masaya: Towards Neural Network Potential for Excited states

Towards Neural Network Potential for Excited states. PFN Internship 2023 by Hagai Masaya

Transcript

1. Towards Neural Network Potential for Excited states Hagai Masaya @ Preferred Networks Summer Internship 2023
2. Light - related phenomena 2 Photosynthesis[1] Human vision[2] Photodamage[3] [1] https://en.wikipedia.org/wiki/Photosynthesis [2] http://light.physics.auth.gr/enc/vision_en.html [3] Facial plastic surgery : FPS 25 5 (2009): 337-46 .
3. Light - related devices Solar cell[1] OLED[2] Bioimaging[3] 3 [1] https://www.science.org/content/article/amp-solar-cells-scientists-ditch-silicon [2] https://en.wikipedia.org/wiki/OLED [3] https://www.k-ishiilab.iis.u-tokyo.ac.jp/research/theme3/3-1_en.html
4. Theory of absorption and emission 4 S 0 (Ground State) S 1 (Excited State) S 2 (Excited State) Energy Nuclear Coordinates hν hν 1. Absorption 2. Structural relaxation in S 1 3. Emission 4. Structural relaxation in S 0 ① ② ③ ④ Radiative process Nonradiative process • Internal conversion • Intersystem crossing • Photoreaction
5. Problems in traditional ways of excited state calc. 5 Calculation time of excited state[1] Conventional methods v.s. NNP Calculation scaling against number of atoms in molecule[2] Conventional method takes much time, O(N4)~O(N!) (N is molecule size) Large number of molecules for new photo-related materials Big molecule such as protein and molecular complex [1] Chem. Rev. 121, 9873–9926 (2021). [2] http://www.chem.waseda.ac.jp/nakai/?page_id=1291
6. Conventional neural network potential (NNP) 6 Input: Molecules Atomic numbers + Coordinates Z Rx Ry Rz … Output : DFT’s property (Density Functional Theory) - Energy - Gradient - Dipole moment S 0 Ground State Neural Network[1] [1] Chem. Sci. 8, 3192–3203 (2017).
7. Objective of this study : NNP for excited states 7 7 Input: Molecules Atomic numbers + Coordinates Z Rx Ry Rz … Output: DFT’s & TDDFT’s property Neural Network[1] - Energy - Gradient - Dipole moment S 0 Ground State - Energy - Gradient - Dipole moment - Transition Dipole S 1 ,S 2 Excited State [1] ANI-1 : Chem. Sci. 8, 3192–3203 (2017).
8. Extensive property and Intensive property 8 Methane(n=1) Ethane (n=2) Hexane (n=6) (S 0 opt. / S 1 opt.) C n H 2n+ 2 Ground state energy [eV] Excitation energy [eV] -1100 12.86 -2168 11.12 -6440 / -6439 10.06 / 7.09 ~2 ~3 Ground state energy is extensive, Excitation energy is intensive O(N) O(1)
9. No size consistency for excited state 9 Excited state energy = Ground state energy + Excitation energy extensive property intensive property System A System B ΔEe=12.9 eV ΔEe=10.1 eV System A+B ΔEe=10.1 eV 1000Å
10. Overview of this study 10 Dataset (Made by myself) • n=1~6 Alkane (n_sample=4,000) • QM5 (n_sample=57,000) NNP model • SchNet • M3GNet ➔ Comfirm better performance M3GNet compared to SchNet Extrapolaion • Heptane (n=7) • Octane (n=8) Readout layer • Output E(S1) directry ◦ Sum model • Output ΔE(S1) as E(S1)=E(S0)+ΔE(S1) ◦ Softmin model ◦ Softmin + self-attention model (Didn’t improve) Loss 1. E(S1) loss 2. ΔE(S1) loss
11. Dataset preparation 11 S 0 (Ground State) S 1 (Excited State) S 2 Energy Nuclear Coordinates 1. Find equilibrium structure of S 0 and S 1 (　 ) 2. Sampling structures around each equilibrium structure (Wigner sampling[1] (≒Normal mode sampling)) [1] Wigner sampling : M. Pinheiro Jr, S. Zhang, P. O. Dral, M. Barbatti, Scientific Data. 10, 1–11 (2023).
12. Alkane dataset / QM5 dataset 12 Alkane dataset (total 4,000 data) C n H 2n+2 (n=1,2,3,4,5,6) • For n=1,2,3,4, sampling 500 structure from S0 opt. structure • For n=5,6, sampling 500 structures from each of the S0 and S1 opt. structures • TDDFT PBE0/6-31G(d) using Gaussian 16 QM5 (total 57,000 data) Max. 5 heavy atoms (C,N,O,F) • 177 S0 opt. structures and 108 S1 opt. structures • Sampling 200 structures from each optimized structre Heptane(n=7) and Octane(n=8) can be predicted from these data? Made TDDFT dataset for NNP by myself in this internship
13. M3GNet NNP model 13 M3GNnet: C. Chen, S. P. Ong, Nature Computational Science. 2, 718–728 (2022). Hyper parameter • N block = 3 • cutoff = 5.0Å • 3body cutoff = 4.0Å • node / edge embedding=64dim Readout layer
14. Readout layer 14 Node feature of atom i 1. Sum model 2. Softmin model 3. Softmin + SelfAttention Ground state energy Excited state energy
15. Test of ΔE(S1-S0) with Sum / Softmin readout 15 Sum readout (MAE=140meV) Softmin readout (MAE=66meV) ❏ Softmin readout shows smaller test error
16. Sum readout model (extrapolation) Heptane (n=7) Octane (n=8) ❏ Large MAE compared to TDDFT error (0.24eV) ❏ MAE error scales with the size of molecule
17. Softmin readout model (extrapolation) 17 Heptane (n=7) Octane (n=8) ❏ Small MAE (0.11~0.33 eV) ❏ MAE errors don’t scale with the size of molecule
18. ΔE(S1) loss & Softmin readout 18 Heptane (n=7) Octane (n=8) Test MAE = 18.6 meV ❏ Test dataset MAE (18.6meV) using ΔE(S1) loss is better than one (66meV) using E(S1) loss
19. App 1. Size consistency problem 19 A = Methane, B = Methane ΔE(S1) in A+B ΔE(S2) in A+B A = Methane, B = Pentane (Softmin readout) ❏ Failed to reproduce ΔE(S1) ❏ between ΔE(S1)(Methane) and ΔE(S1)(Pentane) ❏ Success to reproduce ΔE(S1) ❏ Failed to reproduce ΔE(S2) ❏ (Must be same as ΔE(S1))
20. App 2. Geometry optimization of Excited State 20 Geometry optimization of S1 takes long time Optimization using NNP for excited state C-C-C angle vary greatly (S0 opt. 113.5°, S1 opt. 96.5°) Hexane S1 opt. using NNP from S0 opt. init structure C-C-C angle of NNP S1 opt. is 97.8° Fail hydrogen position • RMSD between initial and S1(TDDFT) = 0.31Å • RMSD between S1(NNP) and S1(TDDFT) = 0.19Å
21. Test of QM5 result (Softmin readout model) 21 S0/S1 energy (MAE=54/58meV) S1 Excitation energy (MAE=22meV) ΔE(S1) Loss
22. QM5 & [QM5 + hexane] extrapolation 22 Heptane Heptane Octane Octane QM5 dataset QM5+Hexane dataset ❏ QM5 only, octane’s predictions are poor (MAE=0.47/0.75eV) ❏ By adding hexane, extrapolability improved (MAE=0.25/0.14eV)
23. TDDFT error & Summary 23 TDDFT/TZVP MSE [eV] MAE [eV] PBE0 -0.05 0.24 [1] A. D. Laurent, D. Jacquemin, Int. J. Quantum Chem. 113, 2019–2039 (2013). TDDFT vertical transition energy error[1](Ref. CASPT2/TZVP) Test dataset & Heptane & Octane ΔE(S1) MAE Type of Readout layer Type of Loss Sum readout E(S1) Loss Softmin E(S1) Loss Softmin ΔE(S1) Loss Softmin,ΔE(S1) QM5 data Softmin, ΔE(S1) QM5 + Hexane Test dataset MAE [meV] 140 66 19 22 Not yet Heptane/Octane MAE[eV] 0.73/1.40 0.33/0.32 0.40/0.19 0.47/0.75 0.25/0.14 ❏ Achieve errors comparable to TDDFT's own errors
24. Conclusion 24 Conclusion • Made TDDFT dataset by myself • Designing readout layer and loss function to reflect excited state property improve performance • Our best excited state NNP shows errors comparable to TDDFT’s own error (0.24eV) • Calculation of Octane S1 energy and force using NNP takes only 0.26s (GPU), TDDFT 10s (32cpu)