Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DeepCrysTet: A Deep Learning Approach Using Tetrahedral Mesh for Predicting Properties of Crystalline Materials

tsurubee
December 17, 2023

DeepCrysTet: A Deep Learning Approach Using Tetrahedral Mesh for Predicting Properties of Crystalline Materials

tsurubee

December 17, 2023
Tweet

More Decks by tsurubee

Other Decks in Research

Transcript

  1. © SAKURA internet Inc.
    22nd International Conference on Machine Learning and Applications (ICMLA 2023)
    DeepCrysTet: A Deep Learning Approach
    Using Tetrahedral Mesh for Predicting
    Properties of Crystalline Materials
    Hirofumi Tsuruta1, Yukari Katsura2,3,4, Masaya Kumagai1,4,5
    1SAKURA internet Inc., 2National Institute for Materials Science (NIMS),
    3The University of Tokyo, 4RIKEN, 5Kyoto University

    View full-size slide

  2. 2
    Background
    Materials Informatics (MI)
    Materials Discovery
    Machine Learning
    (ML)
    Traditional approaches
    rely on the researcher's
    experience and intuition.
    Researcher
    Researcher
    MI approaches are based on
    data-driven computational
    methods.
    Database

    View full-size slide

  3. 3
    Material Properties
    One of the key issues in predicting the properties of crystalline materials is how to
    convert the crystal structure into the features for input to the ML model.
    Background
    Prediction of Crystalline Material Properties
    Input ML Model Output
    NaCl
    Formation energy
    Bulk modulus, etc.
    e.g.
    Chemical composition
    e.g.
    Crystal structure

    View full-size slide

  4. 4
    [Xie 2018] Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
    [Park 2020] Developing an improved crystal graph convolutional neural network framework for accelerated materials discovery
    [Choudhary 2021] Atomistic Line Graph Neural Network for improved materials property predictions
    Source: [Xie 2018] Figure 1
    Related Work
    Graph Representation of a Crystal Structure
    Crystal Graph Convolutional Neural Network (CGCNN) [Xie 2018]
    Node: Constituent atoms
    Edge: Interatomic connections originally defined by
    interatomic distances
    Inspired by CGCNN, many GNN variants using a crystal graph have been rapidly developed to
    improve predictive performance, such as iCGCNN [Park 2020] and ALIGNN [Choudhary 2021].
    Crystal Graph
    Recent Variants Using Crystal Graphs

    View full-size slide

  5. 5
    [Xie 2018] Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
    Source: [Xie 2018] Figure 1
    Related Work
    Graph Representation of a Crystal Structure
    Crystal Graph Convolutional Neural Network (CGCNN) [Xie 2018]
    Node: Constituent atoms
    Edge: Interatomic connections originally defined by
    interatomic distances
    Crystal Graph may have the limitation of losing the crystal structure's three-dimensional (3D)
    information. Can the prediction accuracy be further improved by designing a data
    representation that takes advantage of the 3D crystal structure?
    Crystal Graph
    Research Question

    View full-size slide

  6. 6
    Tetrahedral Mesh Representation of a Crystal Structure
    DeepCrysTet is the first work to predict material properties using a 3D mesh representation
    of a crystal structure.
    Crystal structure
    Graph
    DeepCrysTet’s
    approach
    Proposal
    Tetrahedral Mesh

    View full-size slide

  7. 7
    Proposal
    Overview of DeepCrysTet

    View full-size slide

  8. 8
    1. Mesh Generation 2. Feature Design 3. Neural Network Architecture
    Proposal
    Overview of DeepCrysTet

    View full-size slide

  9. 9
    Few studies have applied Delaunay tetrahedralization to crystal structures.
    [Hinuma 2022] Categorization of inorganic crystal structures by Delaunay tetrahedralization
    Proposal
    1. Mesh Generation
    Delaunay tetrahedralization
    [Delaunay 1934] Sur la sphère vide. A la mémoire de Georges Voronoï, Bulletin de l'Académie des Sciences de l'URSS
    [Delaunay 1934]
    Divide a 3D space into
    a set of tetrahedra
    [Hinuma 2022]
    3D mesh
    Face
    Edge
    Vertex
    A 3D mesh, consisting of a collection of vertices, edges, and
    faces, can be more expressive than other 3D representations,
    such as point clouds and voxels.

    View full-size slide

  10. 10
    Proposal
    2. Feature Design
    Properties used in atomic features.
    Center: Coordinates of the centroid
    of the triangle.
    Edge: Lengths of triangle sides
    sorted in ascending order.
    Corner: Vectors from the centroid
    to three vertices.
    Atomic features: Average of the
    feature vectors of the three atoms
    constituting the triangle.

    View full-size slide

  11. 11
    The input data to the model is a set of features for each triangular face.
    1. Permutation invariant
    2. Different input size
    Requirements
    PointNet architecture
    [Zaheer 2017] Deep Sets
    [Qi 2017] PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
    , , , . . .
    1 2
    𝑥!
    𝑥"



    𝑥#
    shared weight
    MLP
    𝑥′′
    MLP
    Output
    𝑥!

    𝑥"




    𝑥#

    Global Max
    Pooling
    Proposal
    3. Neural Network Architecture
    [Qi 2017]
    ℎ 𝑔
    𝑀𝐴𝑋
    : multi-layer perceptron (MLP)
    𝑔, ℎ
    𝑀𝐴𝑋 : max pooling operation
    [Zaheer 2017]

    View full-size slide

  12. 12
    [Feng 2019] MeshNet: Mesh Neural Network for 3D Shape Representation
    Source: [Feng 2019] Figure 4
    Face Rotate Convolution [Feng 2019]
    Proposal
    3. Neural Network Architecture

    View full-size slide

  13. 13
    Materials Project※
    (83,989 materials)
    Task 1. Crystal Structure Classification
    Experiments
    Experimental Settings
    • DeepCrysTet
    • CGCNN
    • ALIGNN
    Task 2. Material Property Prediction
    Training
    (80%)
    Validation
    (10%)
    Test
    (10%)
    Train
    Models
    Predict
    [Xie 2018]
    ※ https://figshare.com/articles/dataset/Materials_Project_Data/7227749
    [Xie 2018] Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties
    [Choudhary 2021] Atomistic Line Graph Neural Network for improved materials property predictions
    [Choudhary 2021]
    Target properties
    • Crystal system
    • Space group
    • Formation energy
    • Band gap
    • Bulk modulus
    • Shear modulus

    View full-size slide

  14. 14
    Experiments
    Task 1. Crystal Structure Classification
    Summary of the classification accuracies.
    Seven crystal systems
    [Shirokanev 2019] The study of effectiveness of a high-performance crystal lattice parametric identification algorithm based on CUDA technology
    Source: [Shirokanev 2019] Figure 1
    These two classifications strongly
    reflect the 3D crystal structure.

    View full-size slide

  15. 15
    Experiments
    Task 1. Crystal Structure Classification
    Confusion matrices for crystal systems.
    DeepCrysTet ALIGNN
    • Previous studies [Suzuki 2020, Li 2021] have also shown that it is more difficult to classify the triclinic
    and monoclinic systems than others.
    • DeepCrysTet can capture 3D crystal structures more accurately from 3D mesh representations
    than graph-based approaches.
    [Suzuki 2020] Symmetry prediction and knowledge discovery from {X}-ray diffraction patterns using an interpretable machine learning approach
    [Li 2021] Composition based crystal materials symmetry prediction using machine learning with enhanced descriptors

    View full-size slide

  16. 16
    • DeepCrysTet achieved a prediction accuracy comparable to that of GNN-based models for
    predicting the elastic properties of materials.
    • The experimental results suggest that the 3D crystal structure captured by DeepCrysTet
    contributes more to predicting the elastic properties than the other properties.
    Experiments
    Task 2. Material Property Prediction
    Summary of the prediction performance (MAE).

    View full-size slide

  17. 17
    Experiments
    Task 2. Material Property Prediction
    Previous studies [Rabiei 2020, Rabiei 2021] have shown that the elastic modulus is strongly correlated
    with the planar density, which is the fraction of the crystal plane area occupied by atoms.
    Crystal Plane
    [Rabiei 2020] Measurement Modulus of elasticity related to the atomic density of planes in unit cell of crystal lattices
    [Rabiei 2021] Relationship between Young's modulus and planar density of unit cell, super cells (2×2×2), symmetry cells of perovskite (CaTiO3
    ) lattice
    DeepCrysTet’s features include the size and shape of each triangular face and radii of the
    constituent atoms, which can be regarded as features similar to the planar density.
    Triangular Face in DeepCrysTet

    View full-size slide

  18. 18
    • DeepCrysTet is the first work to predict material properties using a 3D mesh representation of a
    crystal structure.
    This work was supported by JST CREST Grant Number JPMJCR19J1 and JSPS KAKENHI Grant-in-
    Aid for Early-Career Scientists Grant Number JP22K14474.
    Acknowledgment
    Thank You!
    Summary
    • DeepCrysTet significantly outperforms existing GNN approaches in classifying crystal structures
    and achieves state-of-the-art performance in predicting elastic properties.
    • In the future, we plan to improve DeepCrysTet further to achieve highly accurate predictions of
    the formation energy and band gap in addition to the elastic properties.

    View full-size slide