Chaos Gamedays for training Engineering Teams

Transcript

1. Chaos Engineering Building Secure Systems using
2. Chaos Gamedays for Training Engineering Teams
3. YURY NIÑO ROA Site Reliability Engineer Chaos Engineering Advocate @yurynino https://www.yurynino.com/
4. Topics to Cover
5. If you know the enemy and know yourself, you need not fear the result of a hundred battles … The Art of War. Sun Tzu
6. How many of you have seen a black swan?
7. Black Swans 1. The event is a surprise. 2. The event has a major effect. 3. After the first recorded, it is rationalized by hindsight!
8. “Don't worry about the future. Or worry, but know that worrying is as effective as trying to solve an algebra equation by chewing a bubble gum. The real troubles in your life are things that never crossed your worried mind, the kind that blindside you at 4 p.m. on some idle Tuesday" Mary Schmich
11. Security Attacks Denial of Service Phishing Man in Middle Attacker overwhelms a system’s resources so that it cannot respond to service requests. DoS doesn’t provide direct benefits for attackers! Attacker hijacks a session between a trusted client and network server. Session hijacking, IP spoofing and replay! Attacker sends emails that appear to be from trusted sources to gain access. Social engineering and Technical trickery. https://blog.netwrix.com
12. Attacks Attacker executes a SQL query via an input data from the client to server. “SELECT * FROM users WHERE account = ‘’ or ‘1’ = ‘1’;” Attacker uses third-party web resources to run scripts in browsers or applications. Steal cookies, keystrokes and collect information. Attacker installs malicious software in the system without consentment of the owner. File infectors, trojans, worms, ransomware. https://blog.netwrix.com Security SQL Injection Malware Cross-Site Scripting
13. Cyberwar is everywhere! In the media, in the military, among politicians and in academia. https://www.yurynino.com/
14. The World is Chaotic! and Insecure Black swans take our systems down and keep them down for a long time. Laura Nolan, SRE in Slack
15. About software systems we can proactively prepare us for cyberattacks! Bring Order through Chaos!
16. What is Chaos Engineering? It is the discipline of experimenting failures in production in order to reveal their weakness and to build confidence in their resilience capability. https://principlesofchaos.org/
17. What is Security Chaos Engineering? It is the identification of security control failures through proactive experimentation to build confidence in the system’s ability to defend against malicious conditions in production. Chaos Engineering Book. 2020
18. Immunity Artificial Systems
19. History 1986 Artificial Immune Systems 2008 Chaos Engineering was born 2018 2020 Chapter dedicated to Security CE 2019 Aaron Rinehart first articles Artificial Intelligence for data security
20. Principles Chaos Engineering Principles Injecting failure to achieve resilience! Hypothesize about Steady State Run Experiments Vary Real-World Events Automate Experiments
21. 1. Pick a Hypothesis: Recipe! 2. Choose the tools: Ingredients! 3. Launch an attack: Cook! 4. Notify the Org: Invite! 5. Run the Experiment: Enjoy! 6. Analyze the Results 7. Automate Chaos Principles
22. More Chaos Security Engineering With Security Chaos Engineering we can introduce false positives into production, to check whether procedures are capable of identifying security failures under controlled conditions.
23. More Chaos Security Engineering www.thoughtworks.com
24. Human factors in cybersecurity are perhaps the biggest challenge when building an effective threat prevention strategy. Vircom
25. Chaos Science
26. Who is responsible for Security Chaos Engineering
27. What my mom thinks I do What my friends thinks I do What software engineers think I do What I really do Who is a Security Chaos Engineer? Help service owners to increase their security and resilience through education, tools and encouragement.
28. Build a Culture based on Security & Reliability Training Security Teams https://www.yurynino.com/
29. Avoid Toil NaLSD Curiosity
30. https://www.yurynino.com/
31. Red Team Exercises • They were originated with the US Armed Forces by Bryce Hoffman. • Adversarial approach that imitates the behaviors and techniques of attackers in the most realistic way possible. • Two common forms of Red Teaming seen in the enterprise are: • Ethical hacking • Penetration testing. • Blue Teams are the defensive counterparts to the Red teams in these exercises. • Recommendations: Think-Write-Share! https://whatis.techtarget.com Training
32. Purple Team Exercises • They were intended as an evolution of Red Team exercises by delivering a more cohesive experience between the offensive and defensive teams. • The “Purple” in Purple Teaming reflects the cohesion of Red and Blue Teaming. • The goal of these exercises is the collaboration of offensive and defensive tactics to improve the effectiveness of both groups in the event of an attempted compromise. • The intention is to increase transparency as well as provide a conduit for the security apparatus to learn about how effective their preparation is when subjected to a live fire exercise. https://whatis.techtarget.com Training
33. Strategies https://whatis.techtarget.com Training
34. Chaos GameDays GameDays are an interactive, real-world and learning exercises. They are designed to give players a chance to put their skills in a technology to test. GameDays were created by Jesse Robbins inspired by his experience & training as a firefighter.
35. Chaos: Plan Gameday Methodology
36. Before After During • Pick a hypothesis. • Pick a style. • Decide who. • Decide where. • Decide when. • Document. • Get approval! • Detect the situation. • Take a deep breath. • Communicate. • Visit dashboards. • Analyze data. • Propose solutions. • Apply and solve! • Write a postmortem. • What Happened • Impact • Duration • Resolution Time • Resolution • Timeline • Action Items Gameday Methodology
37. First on Call Monitors, triages, and tries to mitigate failures caused by the Master of Disaster. Master of Disaster Decides the failure and declares start of incident and attack!!! Team Find and solve the exhibited issues, and write up postmortem. Chaos Roles
38. Tools
39. ChaoSlingr • Serverless app in AWS. • Written in Python. • 100% Native in AWS. • Configuration as a Code. • Configurable Operational Mode. • Open Framework. • With example codes. Tools
40. • Introduce latency on security controls. • Drop a folder like a script would do in production. • Software secret clear text disclosure. • Permission collision in a shared IAM role policy. • Disable service event logging. • API gateway shutdown. • Unencrypted S3 Bucket. • Disable MFA. https://www.yurynino.dev/ Experiment
41. Hypothesis: After the owner of Root account in AWS left the company, we could use our cloud in a normal way. Result: Hypothesis disproved. In this experiment the access to AWS was connected to the Active Directory. When an employee left the company his account is dropped and we lost the access to AWS. Side Effect: Thinking in this scenario allows to consider another applications connected to Active Directory. https://www.yurynino.dev/ Experiment
42. As Henry Ford said, "Failure is only the opportunity to begin again, this time more intelligently." Security Chaos Engineering and Security Chaos Testing give us that opportunity. Taken from DevOpsSec by Jim Bird
43. Who
44. Chaos References
45. Chaos References
46. https://chaosengineering.slack.com https://github.com/dastergon/ awesome-chaos-engineering https://www.infoq.com/chaos-engineering @yurynino How to begin
47. Thanks for coming! @yurynino