Tulp: Integrating Machine Learning and Chaos Engineering

Transcript

1. Tulp: Integrating Artificial Intelligence and Chaos Engineering to Learn from the Incidents.
2. YURY NIÑO ROA Site Reliability Engineer Chaos Engineering Advocate @yurynino https://www.yurynino.dev/
3. • Managing Incidents • Postmortems • Chaos Engineering • Approach with Machine Learning • Tulp: Classifying postmortems • Next steps Agenda
4. High Severity Incidents
5. High Severity Incidents www.gremlin.com
6. High Severity Incidents www.gremlin.com
7. Humans, are central to both the problem and the solution
10. After a storm, life tends to return to a state of normality. We have to write what? We are so tired, let me write it tomorrow! A friendly reminder, we have SLAs It is the time for the postmortem
11. A postmortem is an artifact with a detailed description of exactly what went wrong in an incident. A postmortem is a written record of an incident, its impact, the actions taken to mitigate it, the root cause, and the follow-up actions to prevent the incident.
13. Because
15. Why we don’t like write Postmortems?
18. Here Chaos Engineering is a proper approach to face this don’t like
19. It is the discipline of experimenting in production on a distributed system in order to reveal their weakness and to build confidence in their resilience capability. Chaos Engineering www.principleofchaos.com
20. 2008 Chaos Engineering began at Netflix 2010 Chaos Monkey & Simian Army were launched 2016 Gremlin born 2019 1 Book Chaos massification 2017 SRE USenix Chaos IQ born ChaosConf 2018 1 Book Chaos Monkey for Spring Boot 2020 1 Book was published History of Chaos Engineering
21. Hypothesize about Steady State Run Experiments Vary Real-World Events Automate Experiments Chaos Principles
22. CE is being used for providing resilience and reliability to Artificial Intelligence
23. Chaos for Artificial Intelligence
24. Resilience for Artificial Intelligence
25. Experiment: Hypothesis Validate that there is no interruption in computing metrics when the different Spark components fail. To simulate such failures, we employed a whack-a-mole approach and killed the various Spark components.
26. CE is being used for providing resilience and reliability to AI What about AI in CE
27. What is Artificial Intelligence?
28. Artificial Intelligence Intelligence demonstrated by machines. Area of computer science that studies how machines can perform tasks that would normally require a sentient agent. From Artificial Intelligence with Python
29. AI Process From Artificial Intelligence with Python
30. Tulp
31. Chaos Engineering could be improved with Artificial Intelligence! • Handle large amounts of data in an efficient way. • Ingest data from multiple sources without any lag. • Learn from new data and update constantly using the right learning algorithms. • Continue with tasks without getting tired or needing breaks. Post- mortems
32. Humans, operate differently when they expect things to fail
33. https://www.gremlin.com https://chaosengineering.slack.com https://github.com/dastergon/awesome-ch aos-engineering https://www.infoq.com/chaos-engineering
34. Thanks for coming @yurynino