Towards Neural Network Potential for Excited states Hagai Masaya @ Preferred Networks Summer Internship 2023

Light - related phenomena 2 Photosynthesis[1] Human vision[2] Photodamage[3] [1] https://en.wikipedia.org/wiki/Photosynthesis [2] http://light.physics.auth.gr/enc/vision_en.html [3] Facial plastic surgery : FPS 25 5 (2009): 337-46 .

Light - related devices Solar cell[1] OLED[2] Bioimaging[3] 3 [1] https://www.science.org/content/article/amp-solar-cells-scientists-ditch-silicon [2] https://en.wikipedia.org/wiki/OLED [3] https://www.k-ishiilab.iis.u-tokyo.ac.jp/research/theme3/3-1_en.html

Theory of absorption and emission 4 S 0 (Ground State) S 1 (Excited State) S 2 (Excited State) Energy Nuclear Coordinates hν hν 1. Absorption 2. Structural relaxation in S 1 3. Emission 4. Structural relaxation in S 0 ① ② ③ ④ Radiative process Nonradiative process ● Internal conversion ● Intersystem crossing ● Photoreaction

Problems in traditional ways of excited state calc. 5 Calculation time of excited state[1] Conventional methods v.s. NNP Calculation scaling against number of atoms in molecule[2] Conventional method takes much time, O(N4)~O(N!) (N is molecule size) Large number of molecules for new photo-related materials Big molecule such as protein and molecular complex [1] Chem. Rev. 121, 9873–9926 (2021). [2] http://www.chem.waseda.ac.jp/nakai/?page_id=1291

Conventional neural network potential (NNP) 6 Input: Molecules Atomic numbers + Coordinates Z Rx Ry Rz … Output : DFT’s property (Density Functional Theory) - Energy - Gradient - Dipole moment S 0 Ground State Neural Network[1] [1] Chem. Sci. 8, 3192–3203 (2017).

Objective of this study : NNP for excited states 7 7 Input: Molecules Atomic numbers + Coordinates Z Rx Ry Rz … Output: DFT’s & TDDFT’s property Neural Network[1] - Energy - Gradient - Dipole moment S 0 Ground State - Energy - Gradient - Dipole moment - Transition Dipole S 1 ,S 2 Excited State [1] ANI-1 : Chem. Sci. 8, 3192–3203 (2017).

Extensive property and Intensive property 8 Methane(n=1) Ethane (n=2) Hexane (n=6) (S 0 opt. / S 1 opt.) C n H 2n+ 2 Ground state energy [eV] Excitation energy [eV] -1100 12.86 -2168 11.12 -6440 / -6439 10.06 / 7.09 ~2 ~3 Ground state energy is extensive, Excitation energy is intensive O(N) O(1)

No size consistency for excited state 9 Excited state energy = Ground state energy + Excitation energy extensive property intensive property System A System B ΔEe=12.9 eV ΔEe=10.1 eV System A+B ΔEe=10.1 eV 1000Å

Overview of this study 10 Dataset (Made by myself) ● n=1~6 Alkane (n_sample=4,000) ● QM5 (n_sample=57,000) NNP model ● SchNet ● M3GNet ➔ Comfirm better performance M3GNet compared to SchNet Extrapolaion ● Heptane (n=7) ● Octane (n=8) Readout layer ● Output E(S1) directry ○ Sum model ● Output ΔE(S1) as E(S1)=E(S0)+ΔE(S1) ○ Softmin model ○ Softmin + self-attention model (Didn’t improve) Loss 1. E(S1) loss 2. ΔE(S1) loss

Dataset preparation 11 S 0 (Ground State) S 1 (Excited State) S 2 Energy Nuclear Coordinates 1. Find equilibrium structure of S 0 and S 1 (　 ) 2. Sampling structures around each equilibrium structure (Wigner sampling[1] (≒Normal mode sampling)) [1] Wigner sampling : M. Pinheiro Jr, S. Zhang, P. O. Dral, M. Barbatti, Scientific Data. 10, 1–11 (2023).

Alkane dataset / QM5 dataset 12 Alkane dataset (total 4,000 data) C n H 2n+2 (n=1,2,3,4,5,6) ● For n=1,2,3,4, sampling 500 structure from S0 opt. structure ● For n=5,6, sampling 500 structures from each of the S0 and S1 opt. structures ● TDDFT PBE0/6-31G(d) using Gaussian 16 QM5 (total 57,000 data) Max. 5 heavy atoms (C,N,O,F) ● 177 S0 opt. structures and 108 S1 opt. structures ● Sampling 200 structures from each optimized structre Heptane(n=7) and Octane(n=8) can be predicted from these data? Made TDDFT dataset for NNP by myself in this internship

M3GNet NNP model 13 M3GNnet: C. Chen, S. P. Ong, Nature Computational Science. 2, 718–728 (2022). Hyper parameter ● N block = 3 ● cutoff = 5.0Å ● 3body cutoff = 4.0Å ● node / edge embedding=64dim Readout layer

Readout layer 14 Node feature of atom i 1. Sum model 2. Softmin model 3. Softmin + SelfAttention Ground state energy Excited state energy

Test of ΔE(S1-S0) with Sum / Softmin readout 15 Sum readout (MAE=140meV) Softmin readout (MAE=66meV) ❏ Softmin readout shows smaller test error

Sum readout model (extrapolation) Heptane (n=7) Octane (n=8) ❏ Large MAE compared to TDDFT error (0.24eV) ❏ MAE error scales with the size of molecule

Softmin readout model (extrapolation) 17 Heptane (n=7) Octane (n=8) ❏ Small MAE (0.11~0.33 eV) ❏ MAE errors don’t scale with the size of molecule

ΔE(S1) loss & Softmin readout 18 Heptane (n=7) Octane (n=8) Test MAE = 18.6 meV ❏ Test dataset MAE (18.6meV) using ΔE(S1) loss is better than one (66meV) using E(S1) loss

App 1. Size consistency problem 19 A = Methane, B = Methane ΔE(S1) in A+B ΔE(S2) in A+B A = Methane, B = Pentane (Softmin readout) ❏ Failed to reproduce ΔE(S1) ❏ between ΔE(S1)(Methane) and ΔE(S1)(Pentane) ❏ Success to reproduce ΔE(S1) ❏ Failed to reproduce ΔE(S2) ❏ (Must be same as ΔE(S1))

App 2. Geometry optimization of Excited State 20 Geometry optimization of S1 takes long time Optimization using NNP for excited state C-C-C angle vary greatly (S0 opt. 113.5°, S1 opt. 96.5°) Hexane S1 opt. using NNP from S0 opt. init structure C-C-C angle of NNP S1 opt. is 97.8° Fail hydrogen position ● RMSD between initial and S1(TDDFT) = 0.31Å ● RMSD between S1(NNP) and S1(TDDFT) = 0.19Å

Test of QM5 result (Softmin readout model) 21 S0/S1 energy (MAE=54/58meV) S1 Excitation energy (MAE=22meV) ΔE(S1) Loss

QM5 & [QM5 + hexane] extrapolation 22 Heptane Heptane Octane Octane QM5 dataset QM5+Hexane dataset ❏ QM5 only, octane’s predictions are poor (MAE=0.47/0.75eV) ❏ By adding hexane, extrapolability improved (MAE=0.25/0.14eV)

TDDFT error & Summary 23 TDDFT/TZVP MSE [eV] MAE [eV] PBE0 -0.05 0.24 [1] A. D. Laurent, D. Jacquemin, Int. J. Quantum Chem. 113, 2019–2039 (2013). TDDFT vertical transition energy error[1](Ref. CASPT2/TZVP) Test dataset & Heptane & Octane ΔE(S1) MAE Type of Readout layer Type of Loss Sum readout E(S1) Loss Softmin E(S1) Loss Softmin ΔE(S1) Loss Softmin,ΔE(S1) QM5 data Softmin, ΔE(S1) QM5 + Hexane Test dataset MAE [meV] 140 66 19 22 Not yet Heptane/Octane MAE[eV] 0.73/1.40 0.33/0.32 0.40/0.19 0.47/0.75 0.25/0.14 ❏ Achieve errors comparable to TDDFT's own errors

Conclusion 24 Conclusion ● Made TDDFT dataset by myself ● Designing readout layer and loss function to reflect excited state property improve performance ● Our best excited state NNP shows errors comparable to TDDFT’s own error (0.24eV) ● Calculation of Octane S1 energy and force using NNP takes only 0.26s (GPU), TDDFT 10s (32cpu)