Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Chaos Engineering - W-JAX 2018
Search
Benjamin Wilms
November 08, 2018
Technology
0
290
Chaos Engineering - W-JAX 2018
Chaos Engineering @ W-JAX 2018
- ChaosToolkit
- ChaosHub
- Chaos Monkey for Spring Boot
Benjamin Wilms
November 08, 2018
Tweet
Share
More Decks by Benjamin Wilms
See All by Benjamin Wilms
Chaos Engineering @ Spring I/O 2019 Barcelona
mrbw
1
420
Chaos Engineering - AWS Community Days 2018
mrbw
0
730
Chaos Engineering @ JUG Bonn
mrbw
0
140
Chaos Engineering @ Microservices Meetup Berlin
mrbw
0
110
Spring Cloud Netf
mrbw
1
160
Hysterie in verteilten Systemen - Hystrix im Einsatz
mrbw
0
300
Resilience mit Hystrix: Fehlertoleranz statt Hochverfügbarkeit
mrbw
0
230
Hystrix @ Vaadin Meetup Düsseldorf
mrbw
0
58
Hystrix Warstories
mrbw
0
54
Other Decks in Technology
See All in Technology
WACATE2024冬セッション資料(ユーザビリティ)
scarletplover
0
210
KnowledgeBaseDocuments APIでベクトルインデックス管理を自動化する
iidaxs
1
270
1等無人航空機操縦士一発試験 合格までの道のり ドローンミートアップ@大阪 2024/12/18
excdinc
0
160
成果を出しながら成長する、アウトプット駆動のキャッチアップ術 / Output-driven catch-up techniques to grow while producing results
aiandrox
0
350
どちらを使う?GitHub or Azure DevOps Ver. 24H2
kkamegawa
0
830
プロダクト開発を加速させるためのQA文化の築き方 / How to build QA culture to accelerate product development
mii3king
1
270
PHPerのための計算量入門/Complexity101 for PHPer
hanhan1978
5
170
ずっと昔に Star をつけたはずの思い出せない GitHub リポジトリを見つけたい!
rokuosan
0
150
AWS re:Invent 2024で発表された コードを書く開発者向け機能について
maruto
0
190
20241214_WACATE2024冬_テスト設計技法をチョット俯瞰してみよう
kzsuzuki
3
520
Opcodeを読んでいたら何故かphp-srcを読んでいた話
murashotaro
0
270
レンジャーシステムズ | 会社紹介(採用ピッチ)
rssytems
0
150
Featured
See All Featured
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
111
49k
Building Applications with DynamoDB
mza
91
6.1k
Making Projects Easy
brettharned
116
5.9k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
330
21k
How to Think Like a Performance Engineer
csswizardry
22
1.2k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
127
18k
Become a Pro
speakerdeck
PRO
26
5k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
No one is an island. Learnings from fostering a developers community.
thoeni
19
3k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
95
17k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
59k
Navigating Team Friction
lara
183
15k
Transcript
WITHSTANDING TURBULENT CONDITIONS IN PRODUCTION
BENJAMIN WILMS CODECENTRIC @MRBWILMS GITHUB.COM/MRBW
None
None
FROM A DEVELOPER'S POINT OF VIEW
None
None
None
80% - 90% ...
HOW OUR BABY BEHAVES IN PRODUCTION?
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
JEPSEN IS AN EFFORT TO IMPROVE THE SAFETY OF DISTRIBUTED
DATABASES, QUEUES, CONSENSUS SYSTEMS, ETC.
None
None
CHAOS ENGINEERING IS NOT... ...TO CAUSE CHAOS!
CHAOS ENGINEERING IS NOT... ...BREAKING THINGS JUST TO BREAK THEM!
CHAOS ENGINEERING IS NOT... ...TO RUN A CHAOS MONKEY!
CHAOS ENGINEERING IS NOT... ...TO LET THE WHOLE SIMIAN ARMY
OUT OF THE CAGE!
CHAOS ENGINEERING IS NOT... ...TO USE IT IN PRODUCTION!
CHAOS ENGINEERING IS NOT... ...TO USE IT IN PRODUCTION, FIRST!
CHAOS ENGINEERING IS NOT... ...TO REPLACE OTHER KINDS OF TESTS!
CHAOS ENGINEERING IS NOT... ...TO DO IT ALONE AND WITHOUT
ANY ARRANGEMENT!
None
None
THIS IS BOB
HE IS RESPONSIBLE FOR SERVICE A
THESE ARE BOB'S TEAMMATES
THEY DEPEND ON BOB'S SERVICE
THEY BLAME BOB FOR THE BAD PERFORMANCE OF SERVICE A
BE SOCIAL AND COMMUNICATIVE SHARE YOUR EXPERIENCES AND THOUGHTS STOP
BLAMING EACH OTHER WORK TOGETHER
None
None
CHAOS ENGINEERING IS THE DISCIPLINE OF EXPERIMENTING ON A DISTRIBUTED
SYSTEM IN ORDER TO BUILD CONFIDENCE IN THE SYSTEM’S CAPABILITY TO WITHSTAND TURBULENT CONDITIONS IN PRODUCTION.
None
None
None
STEADY STATE ORDERS PER MINUTE ON A TYPICAL MONDAY MORNING
STEADY STATE ORDERS PER MINUTE
STEADY STATE EXPERIMENT WAS CANCELED
WHAT SHOULD HAPPEN WHEN...
None
None
IF YOU KNOW YOUR CHAOS EXPERIMENT WILL FAIL... ...DON'T DO
IT!!!
None
CPU BURNING - INSPIRED BY TAMMY BUTOW # burn.zsh while
true; do openssl speed; done EOF # cpu_burning.zsh for i in {1..32} do nohup /bin/zsh burn.zsh & done
STRESS CPU stress cpu 2 io 1 vm 1 vmbytes
128M timeout 10s verbose
TRAFFIC CONTROL (TC) PACKAGE IPROUTE2
None
PLATFORMS
CHAOS TOOLKIT FELT IN LOVE WITH CHAOS MONKEY FOR SPRING
BOOT
HOW DOES IT WORK <dependency> <groupid>de.codecentric</groupid> <artifactid>chaosmonkeyspringboot</artifactid> <version>2.0.1</version> </dependency>
(SIDECAR PATTERN) java cp yourapp.jar Dloader.path=chaosmonkeyspringboot2.0.1jarwithdependencies.jar org.springframework.boot.loader.PropertiesLauncher spring.profiles.active=chaosmonkey spring.config.location=file:./chaosmonkey.properties
WATCHER
ASSAULTS
SPRING BOOT ACTUATOR ENDPOINT CONTROL VIA REST ENDPOINT AT RUNTIME
None
None
None
None
None
None
None
None
None
None
None
FINDINGS?
None
None
None
None
BENJAMIN WILMS @MRBWILMS
[email protected]
SHOPPING DEMO CHAOS MONKEY FOR SPRING
BOOT CHAOSTOOLKIT & CHAOSHUB