© SAKURA internet Inc. 22nd International Conference on Machine Learning and Applications (ICMLA 2023) DeepCrysTet: A Deep Learning Approach Using Tetrahedral Mesh for Predicting Properties of Crystalline Materials Hirofumi Tsuruta1, Yukari Katsura2,3,4, Masaya Kumagai1,4,5 1SAKURA internet Inc., 2National Institute for Materials Science (NIMS), 3The University of Tokyo, 4RIKEN, 5Kyoto University

2 Background Materials Informatics (MI) Materials Discovery Machine Learning (ML) Traditional approaches rely on the researcher's experience and intuition. Researcher Researcher MI approaches are based on data-driven computational methods. Database

3 Material Properties One of the key issues in predicting the properties of crystalline materials is how to convert the crystal structure into the features for input to the ML model. Background Prediction of Crystalline Material Properties Input ML Model Output NaCl Formation energy Bulk modulus, etc. e.g. Chemical composition e.g. Crystal structure

4 [Xie 2018] Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties [Park 2020] Developing an improved crystal graph convolutional neural network framework for accelerated materials discovery [Choudhary 2021] Atomistic Line Graph Neural Network for improved materials property predictions Source: [Xie 2018] Figure 1 Related Work Graph Representation of a Crystal Structure Crystal Graph Convolutional Neural Network (CGCNN) [Xie 2018] Node: Constituent atoms Edge: Interatomic connections originally defined by interatomic distances Inspired by CGCNN, many GNN variants using a crystal graph have been rapidly developed to improve predictive performance, such as iCGCNN [Park 2020] and ALIGNN [Choudhary 2021]. Crystal Graph Recent Variants Using Crystal Graphs

5 [Xie 2018] Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties Source: [Xie 2018] Figure 1 Related Work Graph Representation of a Crystal Structure Crystal Graph Convolutional Neural Network (CGCNN) [Xie 2018] Node: Constituent atoms Edge: Interatomic connections originally defined by interatomic distances Crystal Graph may have the limitation of losing the crystal structure's three-dimensional (3D) information. Can the prediction accuracy be further improved by designing a data representation that takes advantage of the 3D crystal structure? Crystal Graph Research Question

6 Tetrahedral Mesh Representation of a Crystal Structure DeepCrysTet is the first work to predict material properties using a 3D mesh representation of a crystal structure. Crystal structure Graph DeepCrysTet’s approach Proposal Tetrahedral Mesh

7 Proposal Overview of DeepCrysTet

8 1. Mesh Generation 2. Feature Design 3. Neural Network Architecture Proposal Overview of DeepCrysTet

9 Few studies have applied Delaunay tetrahedralization to crystal structures. [Hinuma 2022] Categorization of inorganic crystal structures by Delaunay tetrahedralization Proposal 1. Mesh Generation Delaunay tetrahedralization [Delaunay 1934] Sur la sphère vide. A la mémoire de Georges Voronoï, Bulletin de l'Académie des Sciences de l'URSS [Delaunay 1934] Divide a 3D space into a set of tetrahedra [Hinuma 2022] 3D mesh Face Edge Vertex A 3D mesh, consisting of a collection of vertices, edges, and faces, can be more expressive than other 3D representations, such as point clouds and voxels.

10 Proposal 2. Feature Design Properties used in atomic features. Center: Coordinates of the centroid of the triangle. Edge: Lengths of triangle sides sorted in ascending order. Corner: Vectors from the centroid to three vertices. Atomic features: Average of the feature vectors of the three atoms constituting the triangle.

11 The input data to the model is a set of features for each triangular face. 1. Permutation invariant 2. Different input size Requirements PointNet architecture [Zaheer 2017] Deep Sets [Qi 2017] PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , , , . . . 1 2 𝑥! 𝑥" ・ ・ ・ 𝑥# shared weight MLP 𝑥′′ MLP Output 𝑥! ′ 𝑥" ′ ・ ・ ・ 𝑥# ′ Global Max Pooling Proposal 3. Neural Network Architecture [Qi 2017] ℎ 𝑔 𝑀𝐴𝑋 : multi-layer perceptron (MLP) 𝑔, ℎ 𝑀𝐴𝑋 : max pooling operation [Zaheer 2017]

12 [Feng 2019] MeshNet: Mesh Neural Network for 3D Shape Representation Source: [Feng 2019] Figure 4 Face Rotate Convolution [Feng 2019] Proposal 3. Neural Network Architecture

13 Materials Project※ (83,989 materials) Task 1. Crystal Structure Classification Experiments Experimental Settings • DeepCrysTet • CGCNN • ALIGNN Task 2. Material Property Prediction Training (80%) Validation (10%) Test (10%) Train Models Predict [Xie 2018] ※ https://figshare.com/articles/dataset/Materials_Project_Data/7227749 [Xie 2018] Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties [Choudhary 2021] Atomistic Line Graph Neural Network for improved materials property predictions [Choudhary 2021] Target properties • Crystal system • Space group • Formation energy • Band gap • Bulk modulus • Shear modulus

14 Experiments Task 1. Crystal Structure Classification Summary of the classification accuracies. Seven crystal systems [Shirokanev 2019] The study of effectiveness of a high-performance crystal lattice parametric identification algorithm based on CUDA technology Source: [Shirokanev 2019] Figure 1 These two classifications strongly reflect the 3D crystal structure.

15 Experiments Task 1. Crystal Structure Classification Confusion matrices for crystal systems. DeepCrysTet ALIGNN • Previous studies [Suzuki 2020, Li 2021] have also shown that it is more difficult to classify the triclinic and monoclinic systems than others. • DeepCrysTet can capture 3D crystal structures more accurately from 3D mesh representations than graph-based approaches. [Suzuki 2020] Symmetry prediction and knowledge discovery from {X}-ray diffraction patterns using an interpretable machine learning approach [Li 2021] Composition based crystal materials symmetry prediction using machine learning with enhanced descriptors

16 • DeepCrysTet achieved a prediction accuracy comparable to that of GNN-based models for predicting the elastic properties of materials. • The experimental results suggest that the 3D crystal structure captured by DeepCrysTet contributes more to predicting the elastic properties than the other properties. Experiments Task 2. Material Property Prediction Summary of the prediction performance (MAE).

17 Experiments Task 2. Material Property Prediction Previous studies [Rabiei 2020, Rabiei 2021] have shown that the elastic modulus is strongly correlated with the planar density, which is the fraction of the crystal plane area occupied by atoms. Crystal Plane [Rabiei 2020] Measurement Modulus of elasticity related to the atomic density of planes in unit cell of crystal lattices [Rabiei 2021] Relationship between Young's modulus and planar density of unit cell, super cells (2×2×2), symmetry cells of perovskite (CaTiO3 ) lattice DeepCrysTet’s features include the size and shape of each triangular face and radii of the constituent atoms, which can be regarded as features similar to the planar density. Triangular Face in DeepCrysTet

18 • DeepCrysTet is the first work to predict material properties using a 3D mesh representation of a crystal structure. This work was supported by JST CREST Grant Number JPMJCR19J1 and JSPS KAKENHI Grant-in- Aid for Early-Career Scientists Grant Number JP22K14474. Acknowledgment Thank You! Summary • DeepCrysTet significantly outperforms existing GNN approaches in classifying crystal structures and achieves state-of-the-art performance in predicting elastic properties. • In the future, we plan to improve DeepCrysTet further to achieve highly accurate predictions of the formation energy and band gap in addition to the elastic properties.