Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
dojo.pdf
Search
Rich Burroughs
April 18, 2019
Technology
0
120
dojo.pdf
Rich Burroughs
April 18, 2019
Tweet
Share
More Decks by Rich Burroughs
See All by Rich Burroughs
Virtual_Kubernetes_Clusters__Tips_and_Tricks_-_Rejekts.pdf
richburroughs
0
1k
What On-Call Does to Us
richburroughs
1
110
Other Decks in Technology
See All in Technology
家族アルバム みてねにおけるGrafana活用術 / Grafana Meetup Japan Vol.1 LT
isaoshimizu
1
910
エンジニア候補者向け資料2024.04.24.pdf
macloud
0
3.3k
認知症フレンドリーテックとスタックチャン
naokiuc
0
120
LayerXにおけるLLMプロダクト開発の今までとこれから
layerx
PRO
3
600
DMM.com アルファ室採用案内資料
hsugita
1
220
Amplify 🩷 Bedrock 〜生成AI入門〜
minorun365
PRO
8
400
リテール金融(キャッシュレス・ネット銀行・ネット証券)の競争環境と経済圏
8maki
0
1.5k
How to Lead? Testimonial of a Lead Android Engineer
oleur
1
110
KubeConにproposalを送りたい人へのアドバイス
sat
PRO
3
270
Handling focus in 2024
tahia910
0
210
Google Cloud Next '24 Recap(Cloud Run/k8s)
mokocm
0
320
一生覚えておきたい「システム開発=コミュニケーション」〜初めての実務案件振り返りLT〜
maimyyym
2
250
Featured
See All Featured
How to train your dragon (web standard)
notwaldorf
74
5.2k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
275
13k
Writing Fast Ruby
sferik
622
60k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
13
8.3k
In The Pink: A Labor of Love
frogandcode
138
21k
Put a Button on it: Removing Barriers to Going Fast.
kastner
58
3.1k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
7
1.3k
Why You Should Never Use an ORM
jnunemaker
PRO
51
8.6k
GraphQLの誤解/rethinking-graphql
sonatard
55
9.3k
Bootstrapping a Software Product
garrettdimon
PRO
302
110k
How to name files
jennybc
65
93k
The Brand Is Dead. Long Live the Brand.
mthomps
49
29k
Transcript
Learning Through Failure Rich Burroughs Community Manager Gremlin, Inc. @richburroughs
None
None
Complexity is constantly increasing
None
None
None
What's changed?
None
None
"Catastrophe is always just around the corner"
"Change introduces new forms of failure"
"All practitioner actions are gambles"
None
None
What are some ways we can learn more about systems?
None
None
None
Chaos Engineering
"The science of performing intentional experimentation on a system by
injecting precise and measured amounts of harm to observe how the system responds for the purpose of improving the system’s resilience."
None
Prerequisites —Observability —Blameless Culture
Scientific Method —Ask a question —Research —Form a hypothesis —Experiment
to test the hypothesis —Analyze data and draw a conclusion —Share the results
Types of attacks —Shutdown —CPU —Memory —I/O —Network Latency —Packet
Loss —DNS —Blackhole
None
The goal is to experiment in Production
None
Example experiment —Application: Front End —Attack: CPU —Hypothesis: Adding CPU
load will cause additional hosts to spin up in our Autoscaling Group —Abort condition: Latency increases by 20%
Example experiment #2 —Application: Front End —Attack: Blackhole —Hypothesis: Blackholing
the hostname for the Twilio API will cause the SMS transmissions to time out —Abort condition: Error rate increases by 20%
Don't experiment on things you know are broken
None
Questions —Were we able to measure the results? —Did the
system respond the way we expected? —Are there things we need to fix?
Run experiments to simulate an incident you've had
What comes after Game Days?
Continuous Chaos
Maturity model —Running manual experiments —Running experiments using Chaos Engineering
tools —Regularly scheduled Game Days —Experimenting in Production —Continuous Chaos
Next steps: —Join our Chaos Engineering Slack: gremlin.com/ slack —Read
tutorials: gremlin.com/community —Chaos Conf: chaosconf.io —Gremlin Free: go.gremlin.com/richchaos
Thank you! Twitter: @richburroughs Email:
[email protected]
Slides: https://github.com/richburroughs/ dojo201904