Machine Learning for Materials (Lecture 2)

Slides linked to https://github.com/aronwalsh/MLforMaterials

Transcript

1. Aron Walsh Department of Materials Centre for Processable Electronics Machine Learning for Materials 2. Materials Modelling
2. Course Contents 1. Course Introduction 2. Materials Modelling 3. Machine Learning Basics 4. Materials Data and Representations 5. Classical Learning 6. Artificial Neural Networks 7. Building a Model from Scratch 8. Recent Advances in AI 9. and 10. Research Challenge
3. Materials Modelling Computational techniques to simulate & predict the structures and properties materials Common Workflows Input Output Composition Structure Structure Properties Properties Structure ML techniques are accelerating each of these
4. Class Outline Materials Modelling A. Analytical methods B. Numerical methods C. Statistical methods D. Automation and robotics
5. Building Blocks of Materials
6. Models of Interacting Atoms J. E. Jones, Proc. Roy. Soc. A 106, 463 (1923) Molecular mechanics (MM): application of classical mechanics to model chemical systems. Often an analytic formula (bond stretching, bending, etc.) 𝑉𝐿𝐽 = 4ε σ 𝑟 !" − σ 𝑟 # Repulsive Attractive John Lennard-Jones (1894–1954)
7. Models of Interacting Atoms J. E. Jones, Proc. Roy. Soc. A 106, 463 (1923)
8. Models of Interacting Atoms Two-body interatomic potentials from https://gulp.curtin.edu.au
9. Electrostatics in Ionic Solids Oxidation states in crystals: A. Walsh et al, Nature Materials 17, 958 (2018) Difference in electronegativity gives rise to positive and negative ions in compounds Mg(s) + 1 2 O 2 (g) → MgO ≈ Mg2+O2−                                                                           Potential at site i Ulattice = 39 eV / MgO 𝑉! = # "#! 𝑞" 4𝜋𝜀$𝑟!" Born-Haber Cycle
10. Models of Interacting Atoms Y. Shibuta et al, Nat. Comm. 8, 10 (2017) Molecular mechanics (MM): simulations of >1 billion atoms now possible (e.g. protein folding, radiation damage, nucleation, crystallisation) Solidification of Fe with Finnis-Sinclair potential using molecular dynamics
11. Models of Interacting Atoms Octahedral tilt correlation Traditional MM models are being complemented by machine learning force fields (MLFF) Simulation of the metal halide perovskite CsPbI3 A 69,120 atom molecular dynamics simulation within the atomic cluster expansion (ACE) formalism W. J. Baldwin et al, Small 2303565 (2023)
12. Class Outline Materials Modelling A. Analytical methods B. Numerical methods C. Statistical methods D. Automation and robotics
13. Models of Interacting Electrons Quantum mechanics (QM): numerical solutions of the Schrödinger, or relativistic Dirac, equation to describe electron distributions (chemical bonding) The master equations appear simple, but for >1 electron these partial differential equations cannot be solved exactly and require approximations Many-body wavefunction E. Schrödinger, Phys. Rev. 28, 1049 (1926)
14. Models of Interacting Electrons Density functional theory (DFT) replaces the many-body N-dimensional electronic wavefunction by the 3-dimensional electron density Walter Kohn (1923–2016) W. Kohn and L. J. Sham, Phys. Rev. 140, A1133 (1965)
15. Models of Interacting Electrons An iterative solution where the trial electron density is improved until convergence is reached Image from https://siesta-project.org
16. Models of Interacting Electrons O(N3) scaling due to matrix diagonalisation, e.g. 10⨉ atoms cost 1000⨉ more processing time Basis functions are used to expand the wavefunction Density functional theory scaling data in FHI-AIMS courtesy of Volker Blum https://aimsclub.fhi-berlin.mpg.de Using ELPA massively parallel eigensolvers https://gitlab.mpcdf.mpg.de/elpa Image from https://fhi-aims.org
17. Success of DFT Applicable to most types of solids and for wide ranging properties (from mechanical to optical) Image from http://www.materialsproject.org
18. Class Outline Materials Modelling A. Analytical methods B. Numerical methods C. Statistical methods D. Automation and robotics
19. Predict a category, e.g. decision trees to predict reaction outcome Predict a value, e.g. regression to extract a reaction rate Group by similarity, e.g. high-throughput crystallography Maximise reward, e.g. reaction conditions to optimise yield Statistical Modelling Images from https://vas3k.com/blog/machine_learning Machine learning (ML) methods identify and use correlations in multi-dimensional feature spaces
20. (Universal) Function Approximation ML models can describe a wide variety of functions with appropriate data, weights, and parameters Images from https://github.com/jermwatt/machine_learning_refined Linear combination of vectors Linear combination of functions
21. (Universal) Function Approximation Image from https://github.com/jermwatt/machine_learning_refined ML model selection, training, and testing tunes a “complexity dial” for your problem of interest Linear Highly non-linear
22. Function Approximation You should recognise the underlying function from earlier in this lecture…
23. Function Approximation My reference function to generate data for model training and testing
24. Function Approximation Underfitting Overfitting Fitting Default parameters with the scikit-learn Python package
25. Growing Ecosystem for Materials ML Structures & Properties Example workflow based on some of the (many) available toolkits Chemical Features Model Selection & Training Benchmarking
26. Matbench: A. Dunn et al, npj Comp. Mater. 138, 1 (2020) Mean Average Error (Regression) Receiver Operating Characteristic (Classification) Growing Ecosystem for Materials ML
27. https://matbench.materialsproject.org Growing Ecosystem for Materials ML
28. Train, Test, Repeat…
29. Class Outline Materials Modelling A. Analytical methods B. Numerical methods C. Statistical methods D. Automation and robotics
30. Combinatorial Chemistry Produce compound libraries (101-1013) by enumeration over a given parameter space Combinatorial Chemistry (Ed. G. Jung, Wiley 1999) Standard Reaction Combinatorial Approach
31. Modular hardware with computer-controlled synthesis and characterisation B. P. MacLeod et al, Science Advances 6, eaaz8867 (2020) Smart Automation
32. Smart Automation Modular hardware with computer-controlled synthesis and characterisation Y. Jiang, A. I. Cooper and colleagues, Digital Discovery 2, 1733 (2023)
33. Smart Optimisation Optimisation problems in large parameter spaces (e.g. temperature, pressure, concentrations, time) B. P. MacLeod et al, Science Advances 6, eaaz8867 (2020)
34. Smart Optimisation Bayesian optimisation (BO) uses prior data to decide which experiment to perform next J. Močkus, Optimisation Techniques 1, 400 (1974); More in Lecture 7 𝑃(A|B) = 𝑃(A) 𝑃(B|A) 𝑃(B) Thomas Bayes FRS (1701 – 1761) Bayes’ Theorem Updated probabilities of A given B occurred BO involves: Surrogate Model Approximation of the optimisation surface (e.g. Gaussian process) Acquisition Function Selection of the next sample point (e.g. Upper confidence bound)
35. Smart Characterisation Combine data from multiple characterisation techniques for better materials/device models S. J. Cooper and colleagues, ACS Energy Letters 7, 4368 (2022)
36. Smart Characterisation Utilise powerful image processing AI toolkits S. J. Cooper and colleagues, ACS Energy Letters 7, 4368 (2022) Labelling of objects (e.g. domains) Missing regions (e.g. corruptions) Upscale images (e.g. faster scans) Generate new datasets (e.g. conditions) Transform appearance (e.g. noise) Map from 2D to 3D (e.g. TEM data)
37. Class Outcomes 1. Define materials modelling 2. Compare and contrast analytical, numerical, and statistical methods 3. Describe an example of each Activity: Crystal electrostatics