Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Chaos Engineering - W-JAX 2018
Search
Benjamin Wilms
November 08, 2018
Technology
0
290
Chaos Engineering - W-JAX 2018
Chaos Engineering @ W-JAX 2018
- ChaosToolkit
- ChaosHub
- Chaos Monkey for Spring Boot
Benjamin Wilms
November 08, 2018
Tweet
Share
More Decks by Benjamin Wilms
See All by Benjamin Wilms
Chaos Engineering @ Spring I/O 2019 Barcelona
mrbw
1
390
Chaos Engineering - AWS Community Days 2018
mrbw
0
690
Chaos Engineering @ JUG Bonn
mrbw
0
130
Chaos Engineering @ Microservices Meetup Berlin
mrbw
0
95
Spring Cloud Netf
mrbw
1
130
Hysterie in verteilten Systemen - Hystrix im Einsatz
mrbw
0
280
Resilience mit Hystrix: Fehlertoleranz statt Hochverfügbarkeit
mrbw
0
200
Hystrix @ Vaadin Meetup Düsseldorf
mrbw
0
51
Hystrix Warstories
mrbw
0
48
Other Decks in Technology
See All in Technology
期待しすぎずに取り組む両面 TypeScript
shozawa
2
270
どう買う?Azure
kuniteru
1
190
TCA入門したてなので、自分が馴染みのある実装と比較しながらキャッチアップしてみる
fumiyasac0921
1
370
LLM + RAG を使った SORACOM Support Bot の裏側の歴史
soracom
PRO
1
630
実務への応用例から考える 変更に強いオブジェクト指向設計 / 20240324-ooc2024
bengo4com
7
4.5k
単回帰分析について数式を追いながら実装してみた
kentaitakura
0
490
やっていきテスト
k6s4i53rx
0
160
技術広報として2023年度に頑張ったこと / What we did well in FY2023 as a DevRel
pauli
5
450
Challenges - Open Farming Hackdays 2024
loleg
0
490
大規模データとの戦い方
knih
1
150
Tohoku.Tech #1 「EC-CUBE/AWSの構築をChatGPTに相談してみました」by テンダ
jun2882
0
140
ハイパフォーマンスな組織をつくるための開発生産性の考え方 / developer-productivity-high-performer-link-and-motivation
lmi
3
220
Featured
See All Featured
The Illustrated Children's Guide to Kubernetes
chrisshort
28
46k
The Cult of Friendly URLs
andyhume
72
5.6k
StorybookのUI Testing Handbookを読んだ
zakiyama
10
4.5k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
355
22k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
18
1.7k
Infographics Made Easy
chrislema
237
18k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
57
14k
A designer walks into a library…
pauljervisheath
199
23k
Typedesign – Prime Four
hannesfritz
36
2k
Raft: Consensus for Rubyists
vanstee
130
6.2k
Embracing the Ebb and Flow
colly
78
4.1k
What’s in a name? Adding method to the madness
productmarketing
PRO
14
2.6k
Transcript
WITHSTANDING TURBULENT CONDITIONS IN PRODUCTION
BENJAMIN WILMS CODECENTRIC @MRBWILMS GITHUB.COM/MRBW
None
None
FROM A DEVELOPER'S POINT OF VIEW
None
None
None
80% - 90% ...
HOW OUR BABY BEHAVES IN PRODUCTION?
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
JEPSEN IS AN EFFORT TO IMPROVE THE SAFETY OF DISTRIBUTED
DATABASES, QUEUES, CONSENSUS SYSTEMS, ETC.
None
None
CHAOS ENGINEERING IS NOT... ...TO CAUSE CHAOS!
CHAOS ENGINEERING IS NOT... ...BREAKING THINGS JUST TO BREAK THEM!
CHAOS ENGINEERING IS NOT... ...TO RUN A CHAOS MONKEY!
CHAOS ENGINEERING IS NOT... ...TO LET THE WHOLE SIMIAN ARMY
OUT OF THE CAGE!
CHAOS ENGINEERING IS NOT... ...TO USE IT IN PRODUCTION!
CHAOS ENGINEERING IS NOT... ...TO USE IT IN PRODUCTION, FIRST!
CHAOS ENGINEERING IS NOT... ...TO REPLACE OTHER KINDS OF TESTS!
CHAOS ENGINEERING IS NOT... ...TO DO IT ALONE AND WITHOUT
ANY ARRANGEMENT!
None
None
THIS IS BOB
HE IS RESPONSIBLE FOR SERVICE A
THESE ARE BOB'S TEAMMATES
THEY DEPEND ON BOB'S SERVICE
THEY BLAME BOB FOR THE BAD PERFORMANCE OF SERVICE A
BE SOCIAL AND COMMUNICATIVE SHARE YOUR EXPERIENCES AND THOUGHTS STOP
BLAMING EACH OTHER WORK TOGETHER
None
None
CHAOS ENGINEERING IS THE DISCIPLINE OF EXPERIMENTING ON A DISTRIBUTED
SYSTEM IN ORDER TO BUILD CONFIDENCE IN THE SYSTEM’S CAPABILITY TO WITHSTAND TURBULENT CONDITIONS IN PRODUCTION.
None
None
None
STEADY STATE ORDERS PER MINUTE ON A TYPICAL MONDAY MORNING
STEADY STATE ORDERS PER MINUTE
STEADY STATE EXPERIMENT WAS CANCELED
WHAT SHOULD HAPPEN WHEN...
None
None
IF YOU KNOW YOUR CHAOS EXPERIMENT WILL FAIL... ...DON'T DO
IT!!!
None
CPU BURNING - INSPIRED BY TAMMY BUTOW # burn.zsh while
true; do openssl speed; done EOF # cpu_burning.zsh for i in {1..32} do nohup /bin/zsh burn.zsh & done
STRESS CPU stress cpu 2 io 1 vm 1 vmbytes
128M timeout 10s verbose
TRAFFIC CONTROL (TC) PACKAGE IPROUTE2
None
PLATFORMS
CHAOS TOOLKIT FELT IN LOVE WITH CHAOS MONKEY FOR SPRING
BOOT
HOW DOES IT WORK <dependency> <groupid>de.codecentric</groupid> <artifactid>chaosmonkeyspringboot</artifactid> <version>2.0.1</version> </dependency>
(SIDECAR PATTERN) java cp yourapp.jar Dloader.path=chaosmonkeyspringboot2.0.1jarwithdependencies.jar org.springframework.boot.loader.PropertiesLauncher spring.profiles.active=chaosmonkey spring.config.location=file:./chaosmonkey.properties
WATCHER
ASSAULTS
SPRING BOOT ACTUATOR ENDPOINT CONTROL VIA REST ENDPOINT AT RUNTIME
None
None
None
None
None
None
None
None
None
None
None
FINDINGS?
None
None
None
None
BENJAMIN WILMS @MRBWILMS
[email protected]
SHOPPING DEMO CHAOS MONKEY FOR SPRING
BOOT CHAOSTOOLKIT & CHAOSHUB