Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Chaos Monkey on my Laptop

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →
Avatar for Matthew Campbell Matthew Campbell
October 02, 2015
Programming
200
0

Chaos Monkey on my Laptop

Chaos testing/engineering talk about integrating chaos into integration test suites. Original talk done at GothamGO and Velocity NYC 2015. Videos only

Avatar for Matthew Campbell

Matthew Campbell

October 02, 2015

More Decks by Matthew Campbell

See All by Matthew Campbell
Building Distributed Timeseries Database in Go
Avatar for Matthew Campbell mattkanwisher
4
1.4k
Serious Gaming, Thai Typing Game
Avatar for Matthew Campbell mattkanwisher
0
95

Other Decks in Programming

See All in Programming
PHP でエミュレータを自作して Ubuntu を動かそう
Avatar for memory m3m0r7
PRO
2
160
安いハードウェアでVulkan
Avatar for Fadis fadis
1
870
The free-lunch guide to idea circularity
Avatar for Holly Cummins hollycummins
0
400
Geminiをパートナーに神社DXシステムを個人開発した話（いなめぐDX 開発振り返り）
Avatar for T. Fujiba fujiba
0
130
年間50登壇、単著出版、雑誌寄稿、Podcast出演、YouTube、CM、カンファレンス主催……全部やってみたので面白さ等を比較してみよう / I’ve tried them all, so let’s compare how interesting they are.
Avatar for nrs nrslib
4
660
20260320登壇資料
Avatar for 松井遼(まついはるか) pharct
0
150
RailsのValidatesをSwift Macrosで再現してみた
Avatar for Takuma Shimizu hokuron
0
150
AIと共にエンジニアとPMの “二刀流”を実現する
Avatar for Naruo Yoshiki naruogram
0
120
Ruby and LLM Ecosystem 2nd
Avatar for Koichi ITO koic
1
1.4k
CS教育のDX AIによる育成の効率化
Avatar for ニフティ株式会社 niftycorp
PRO
0
180
Codex CLI でつくる、Issue から merge までの開発フロー
Avatar for amata1219 amata1219
0
280
ファインチューニングせずメインコンペを解く方法
Avatar for pokutuna pokutuna
0
250

Featured

See All Featured
Paper Plane (Part 1)
Avatar for Katie Cordina Lindsey katiecoart
PRO
0
6.3k
The innovator’s Mindset - 
Leading Through an Era of Exponential Change - McGill University 2025
Avatar for Jean-Marc De Jonghe jdejongh
PRO
1
140
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
32
2.6k
Distributed Sagas: A Protocol for Coordinating Microservices
Avatar for Caitie McCaffrey caitiem20
333
22k
Breaking role norms: Why Content Design is so much more than writing copy - Taylor Woolridge
Avatar for UX Y'all uxyall
0
240
Product Roadmaps are Hard
Avatar for C. Todd Lombardo iamctodd
PRO
55
12k
How to Think Like a Performance Engineer
Avatar for Harry Roberts csswizardry
28
2.5k
Heart Work Chapter 1 - Part 1
Avatar for Lake Fama lfama
PRO
5
35k
A Soul's Torment
Avatar for Nat Ma seathinner
5
2.6k
What’s in a name? Adding method to the madness
Avatar for The Alliance productmarketing
PRO
24
4k
Bootstrapping a Software Product
Avatar for Garrett Dimon garrettdimon
PRO
307
120k
Stewardship and Sustainability of Urban and Community Forests
Avatar for Eric Wiseman pwiseman
0
170

Transcript

  1. Chaos Monkey on my Laptop

  2. What is chaos testing?

  3. Matt Campbell

  4. Eikon Messenger

  5. None

  6. Scale 300k financial traders Every major bank around the world

  7. Our testing setup

  8. –Said by no one “Sweet lets just start killing production

    databases”

  9. Lets learn how to crawl first

  10. Vagrant Docker Integration Tests

  11. Continuous Integration

  12. Failure integration tests

  13. Less Mocking

  14. Database failovers

  15. Microservice Problems

  16. Performance Downtime Discovery

  17. TheSlowNess

  18. Problems • Connection pooling in go • Dead tcp connections

    • Hung connections • Network splits / Congestion

  19. Production integration tests

  20. Chaos Gopher

  21. • Unix/GO Style tools • Simple utilities , Slowness, Failover,

    Assassin

  22. Where Next? Better integration test suites Allowing for flappy tests

    Statistically analysis of flapping tests

  23. Other cool projects • Chaos Monkey (the mac daddy, Netflix)

    • Chaos Lemur (from Pivotal)

  24. Questions? Matthew Campbell [email protected] @kanwisher github.com/mattkanwisher