Chaos Engineering @ Spring I/O 2019 Barcelona

Why we need Chaos Engineering
The complexity in modern and distributed architectures continues to increase. We have successfully broken down our application into small and maintainable components. Each individual component can be automated and brought into production at any time. A lot of effort was put into the development to keep the test coverage as high as possible. Every release has to successfully pass our pipeline and countless unit, integration and acceptance tests.

But why do we have this unpleasant feeling shortly before our arrival at the most beautiful place in the world (production)?

Transcript

1. WITHSTANDING TURBULENT CONDITIONS IN PRODUCTION
2. BENJAMIN WILMS CODECENTRIC @MRBWILMS
8. 80% - 90% ...
11. HOW OUR BABY BEHAVES IN PRODUCTION?
28. TOP 10 ECOMMERCE GERMANY 2017
34. CHAOS ENGINEERING IS NOT... ...TO CAUSE CHAOS!
35. CHAOS ENGINEERING IS NOT... ...BREAKING THINGS JUST TO BREAK THEM!
36. CHAOS ENGINEERING IS NOT... ...TO RUN A CHAOS MONKEY!
37. CHAOS ENGINEERING IS NOT... ...TO LET THE WHOLE SIMIAN ARMY OUT OF THE CAGE!
38. CHAOS ENGINEERING IS NOT... ...TO USE IT IN PRODUCTION!
39. CHAOS ENGINEERING IS NOT... ...TO REPLACE OTHER KINDS OF TESTS!
40. CHAOS ENGINEERING IS NOT... ...TO DO IT ALONE AND WITHOUT ANY ARRANGEMENT!
43. THIS IS BOB
44. HE IS RESPONSIBLE FOR SERVICE A
45. THESE ARE BOB'S TEAMMATES
46. THEY DEPEND ON BOB'S SERVICE
47. THEY BLAME BOB FOR THE BAD PERFORMANCE OF SERVICE A
48. BE SOCIAL AND COMMUNICATIVE SHARE YOUR EXPERIENCES AND THOUGHTS STOP BLAMING EACH OTHER WORK TOGETHER
51. CHAOS ENGINEERING IS THE DISCIPLINE OF EXPERIMENTING ON A DISTRIBUTED SYSTEM IN ORDER TO BUILD CONFIDENCE IN THE SYSTEM’S CAPABILITY TO WITHSTAND TURBULENT CONDITIONS IN PRODUCTION.
54. WHAT SHOULD HAPPEN WHEN...
57. IF YOU KNOW YOUR CHAOS EXPERIMENT WILL FAIL... ...DON'T DO IT!!!
61. STEADY STATE ORDERS PER MINUTE ON A TYPICAL MONDAY MORNING
62. STEADY STATE ORDERS PER MINUTE
63. STEADY STATE EXPERIMENT WAS CANCELED
65. CPU BURNING - INSPIRED BY TAMMY BUTOW # burn.zsh while true; do openssl speed; done EOF # cpu_burning.zsh for i in {1..32} do nohup /bin/zsh burn.zsh & done
66. STRESS CPU stress ­­cpu 2 ­­io 1 ­­vm 1 ­­vm­bytes 128M ­­timeout 10s ­­verbose
67. TRAFFIC CONTROL (TC) PACKAGE IPROUTE2
69. PLATFORMS
70. HOW DOES IT WORK <dependency> <groupid>de.codecentric</groupid> <artifactid>chaos­monkey­spring­boot</artifactid> <version>2.0.2</version> </dependency>
71. (SIDECAR PATTERN) java ­cp your­app.jar ­Dloader.path=chaos­monkey­spring­boot­2.0.2­jar­with­dependencies.jar org.springframework.boot.loader.PropertiesLauncher ­­spring.profiles.active=chaos­monkey ­­spring.config.location=file:./chaos­monkey.properties
72. WATCHER
73. ASSAULTS
74. SPRING BOOT ACTUATOR ENDPOINT CONTROL VIA REST ENDPOINT AT RUNTIME
97. BENJAMIN WILMS @MRBWILMS BENJAMIN.WILMS@CODECENTRIC.DE SHOPPING DEMO