A fourth paradigm? The challenges of using machine learning in physics

I look at some of the particular obsticles faced for applying modern ML techniques to physics problems. ML has been revolutionising fields such as machine vision and automated translation. Recently there has been a flurry of interest in applying ML to problems in physics and chemistry. While there are notable successes, there are certain particular issues that should be understood if ML is to be successfully used in the physical sciences. I look at how chooisng the right problems to study, choosing the right models to study them, understanding your data and finally interpreting your results will be critical for the success of data-driven physics.

Transcript

1. A FOURTH PARADIGM? THE CHALLENGES OF USING MACHINE LEARNING IN PHYSICS KEITH T. BUTLER IOP OCT 2019
2. IOP OCT 2019 THE MEANING OF A FOURTH PARADIGM(?) APL Materials 4, 053208 (2016)
3. IOP OCT 2019 HOW TO BENEFIT FROM ML IN PHYSICS PROBLEMS ▸ Choose the right problems ▸ Choose the right models ▸ Choose the right data ▸ Understand the results (if you care to…)
4. IOP OCT 2019 CHOOSING THE RIGHT PROBLEMS 4 “Computers are useless, they can only give you answers.” Picasso.
5. IOP OCT 2019 CHOOSING THE RIGHT PROBLEMS 5 KEY QUESTIONS BEFORE GOING ML ▸ What do I want to achieve? ▸ How much data do I have/can I get? ▸ What kind of data do I have? ▸ Do I care more about prediction or inference? ▸ What kind of hardware do I have? 5
6. IOP OCT 2019 CHOOSING THE RIGHT MODEL UNDERSTAND THE OPTIONS ▸ Example decision tree (classical); neural network (deep) 6 Robustness Scaling Interpretability Simplicity Speed Accuracy ANN DT Traditional ML Deep NN Performance Data
7. IOP OCT 2019 AN OVERVIEW OF CLASSICAL ML MODELS 7 https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html
8. IOP OCT 2019 SOME IMPORTANT CLASSES OF DEEP NEURAL NETWORKS 8 Fuller map available: https://towardsdatascience.com/the-mostly-complete-chart-of-neural-networks- explained-3fb6f2367464 Mulitlayer perceptrons Function approximation machine Recurrent Neural Networks Time series/sequential data eg text conversion Deep Convolutional Networks Image processing
9. IOP OCT 2019 CHOOSING THE RIGHT MODEL REPRESENTATIONS OF THE PROBLEM ▸ Hand crafted descriptors ▸ Learned representations ▸ E.g Convolutional neural networks 9 Smaller datasets, with very structured inputs, classical models Larger datasets, with very unstructured inputs, missing data, combined data sources, deep models Traditional ML Deep NN
10. IOP OCT 2019 REPRESENTATIONS OF THE PROBLEM EXAMPLES ▸ Clean structured data can often be converted/compressed 10 ▸ Noisy/incomplete datasets are difficult to convert analytically ?
11. IOP OCT 2019 OBTAINING (THE RIGHT) DATA 14,197,122 images!!
12. IOP OCT 2019 OBTAINING (THE RIGHT) DATA ▸ Image recognition exploded after the availability of the Imagenet challenge ▸ Physics data is not as easy to label as many other ‘simple’ data types
13. IOP OCT 2019 DEALING WITH DATA SCARCITY TRANSFER LEARNING Can we use what we learned from other examples to speed up learning?
14. IOP OCT 2019 DEALING WITH DATA SCARCITY TRANSFER LEARNING http://scs.ryerson.ca/~aharley/vis/conv/flat.html Example - convolutional networks learn very general features in early layers
15. IOP OCT 2019 DEALING WITH DATA SCARCITY TRANSFER LEARNING https://github.com/materialsvirtuallab/megnet Example - crystal neural network trained on energies can be used for dielectric properties DFPT calculation several hours on 10s to 100s of CPUs. MegNet calculation seconds on a desktop
16. IOP OCT 2019 DEALING WITH DATA SCARCITY ACTIVE LEARNING TRAIN ON LABELLED DATA TEST ON UNLABELLED DATA Committee of models Identify regions of uncertainty
17. IOP OCT 2019 BUILDING BETTER DATASETS Nature, 2018, 559, 547. The importance of metadata is becoming more apparent Initiatives like FAIRsFAIR https://www.force11.org/ group/fairgroup/fairprinciples
18. IOP OCT 2019 MAKING MODELS INTERPRETABLE ▸ Classical models are often easy to interpret ▸ Deep models, learned representations can be more opaque
19. IOP OCT 2019 MAKING MODELS INTERPRETABLE Model performance Interpretability use Sub-human Debug and improve Human Increase confidence Super-human Learn from successs
20. IOP OCT 2019 INTERPRETABLE MODELS FOR NEUTRON SCATTERING Finding the right signal can be a needle in a haystack
21. IOP OCT 2019 INTERPRETABLE MODELS FOR NEUTRON SCATTERING Build a model discrimination network - ask it WHY it makes the choice.
22. IOP OCT 2019 INTERPRETABLE MODELS FOR NEUTRON SCATTERING The network identifies the same regions of E/Q space as a trained physicist. Could, in future, guide experiments of the same type.
23. IOP OCT 2019 SUMMARY ▸ AI/ML/Logistic regression has potential to be a transformative tool ▸ To get the most from AI we should understand the relationships between models, data and physics ▸ Physics data tends to be scarce or messy or both, cutting edge research looks at how to deal with these issues ▸ Combining AI with traditional physics simulation offers a powerful route
24. IOP OCT 2019 ACKNOWLEDGMENTS ▸ SciML ▸ Rebecca, Tony, Jeyan, Sam, Patrick ▸ ICL ▸ Kazuki, Aron ▸ UCL ▸ Dan
25. IOP OCT 2019 THANK YOU 25 NATURE, 2018, 559, 547. @keeeto2000 @ml_sci keeeto.github.io www.scd.stfc.ac.uk/ Pages/Scientific- Machine-Learning.aspx