Machine Learning to Predict Chaos

Transcript

1. Machine Learning to Predict Chaos
2. YURY NIÑO Site Reliability Engineer Chaos Engineering Advocate @yurynino https://www.yurynino.com/
3. AGENDA • Cloud Native Transformation • Current status • Next: ML to predict Chaos • Machine Learning Foundations • ML to predict Chaos • Failures Taxonomy • Predicting failures www.yurynino.com
4. CLOUD NATIVE Transformation www.yurynino.com
5. Cloud Native is the future of computing. It’s going to allow you to deliver software more quickly and more cheaply. It looks like It’ll save the planet! www.yurynino.com
6. Cloud native is more than a tool set. It is a complete architecture, a philosophical approach for building applications taking advantage of cloud computing. Cloud native is an architecture for assembling all of the above cloud-based components in a way that is optimized for the cloud environment. It’s not about the servers, but the services. www.yurynino.com
7. www.yurynino.com
8. It is the future! Humans are defining the starting point for machines to learn and constantly improve. Self healing is the optimal way for systems to be operated and maintained. It is faster, more secure, and more reliable. www.yurynino.com
9. MACHINE LEARNING FOUNDATIONS www.yurynino.com
10. … what we want is a machine that can learn from experience ... Alan Turing, 1947 www.yurynino.com
11. Machine Learning Foundations www.yurynino.com
12. Machine Learning Foundations Machine learning is a research field at the intersection of statistics, artificial intelligence, and computer science for extracting knowledge from data. www.yurynino.com
13. www.yurynino.com
14. www.yurynino.com https://www.wordstream.com/blog/ws/2017/07/28/ma chine-learning-applications
15. Machine Learning has a lot applications in systems that we use daily! www.yurynino.com
16. Chaos Theory Half a century ago, the pioneers of chaos theory discovered that the “butterfly effect” makes long-term prediction impossible. www.yurynino.com
17. NOW! Machine Learning could change this! The effectiveness of using machine learning for model-free prediction of chaotic systems has been documented in several documents! Jaideep Pathak of the University of Maryland, who shown that ML is a powerful tool for predicting chaos. 4 www.yurynino.com
18. Holger Kantz “The machine-learning technique is almost as good as knowing the truth, If we have ignorance we should use the machine learning to fill in the gaps where the ignorance resides. Edward Ott www.yurynino.com
19. 3 Steps for Machine Learning Make the neural network learn the dynamics of the evolving flame! ML 1 The neural network essentially asks itself what will happen. Get Data Input Eg. measure the height of a flame at n different points. 2 3 Feed data-streams in to randomly artificial neurons Outputs are fed back in as new inputs. www.yurynino.com
20. 3 Steps for Machine Learning Make the neural network learn the dynamics of the evolving flame! ML 1 The neural network essentially asks itself what will happen. Get Data Input Eg. measure the height of a flame at n different points. 2 3 Feed data-streams in to randomly artificial neurons Outputs are fed back in as new inputs. www.yurynino.com
21. 3 Steps for Machine Learning Make the neural network learn the dynamics of the evolving flame! ML 1 The neural network essentially asks itself what will happen. Get Data Input Eg. measure the height of a flame at n different points. 2 3 Feed data-streams in to randomly artificial neurons Outputs are fed back in as new inputs. www.yurynino.com
22. 3 Steps for Machine Learning Make the neural network learn the dynamics of the evolving flame! ML 1 The neural network essentially asks itself what will happen. Get Data Input Eg. measure the height of a flame at n different points. 2 3 Feed data-streams in to randomly artificial neurons Outputs are fed back in as new inputs. www.yurynino.com
23. Humans are defining machines to learn and constantly improve. So can we extrapolate this to predict our chaos!
24. Classifying Operations Failures Identify and measure the characteristics of a failure: • Are we using an observability tool? • Do we have observability about SLOs? • Are we following an IM methodology? • Are we writing postmortems? • Did we practice gamedays? Get Data Input Eg. measure the height of a flame at n different points. 1 1 www.yurynino.com
25. Training Operations Failures Provide examples of previous projects: • If we don’t use an observability tool the response time is ... • If we cannot follow SLOs in an observability tool our KPIs are ... • If we don’t have an IM methodology we are attending the same. Make the neural network learn the dynamics of the evolving flame! 2 2 www.yurynino.com
26. Predicting Operations Failures Ask to neural network: • I lost the access to observability tool what will the impact in the response times ... • If I write postmortems how could improve my response times … • I will invest X money and time in a gameday next month what will my revenue ... The neural network essentially asks itself what will happen. 3 3 www.yurynino.com
27. NEXT: Challenges www.yurynino.com
28. Artificial intelligence is being used for providing resilience and reliability but is AI reliable a resilient? www.yurynino.com
29. Big Data is a form of AI. Organizations are using it to manage the customer experience, transform their products and deliver digital services. This data must be reliable! Challenge 1
30. Robust ML systems and hardware architectures are required to generate reliable and trustworthy results in the presence of hardware-level faults while also preserving security and privacy. Challenge 2
31. Natural Processing Language! Since computers exist people have tried to teach them how to process human language, however, the inconsistency and volatility of human language turns NLP into a complex task susceptible to fail. Challenge 3
32. The usual approach to predicting a chaotic system is to measure its conditions at one moment as accurately as possible, use these data to calibrate a physical model, and then evolve the model forward. Nataly Wolchover www.yurynino.com
33. Attack! Architectures & Models • Test models injecting failures. • Chaos as functions. • Use Artificial Intelligence to classify postmortems! www.yurynino.com
34. Remember! Failures are an inevitable part of making software products and services, however it is not necessary to repeat the mistakes of the past. www.yurynino.com
35. We are here! Humans are defining the starting point for machines to learn and constantly improve. Self healing is the optimal way for systems to be operated and maintained. It is faster, more secure, and more reliable. Systems learn on their own how to prevent failures by, for instance, automatically scaling up capacity. www.yurynino.com
36. Artificial intelligence is being used for providing resilience and reliability but is AI reliable a resilient? www.yurynino.com
37. Thank you! @yurynino https://www.yurynino.com www.yurynino.com