Chaos Monkey on my Laptop

Chaos testing/engineering talk about integrating chaos into integration test suites. Original talk done at GothamGO and Velocity NYC 2015. Videos only

Transcript

1. Chaos Monkey
   on my Laptop
   

   View Slide

2. What is chaos testing?
   

   View Slide

3. Matt Campbell
   

   View Slide

4. Eikon Messenger
   

   View Slide

5. View Slide

6. Scale
   300k financial traders
   Every major bank around the world
   

   View Slide

7. Our testing setup
   

   View Slide

8. –Said by no one
   “Sweet lets just start killing
   production databases”
   

   View Slide

9. Lets learn how to crawl first
   

   View Slide

10. Vagrant
    Docker
    Integration Tests
    

    View Slide

11. Continuous
    Integration
    

    View Slide

12. Failure integration
    tests
    

    View Slide

13. Less Mocking
    

    View Slide

14. Database failovers
    

    View Slide

15. Microservice Problems
    

    View Slide

16. Performance
    Downtime
    Discovery
    

    View Slide

17. TheSlowNess
    

    View Slide

18. Problems
    • Connection pooling in go
    • Dead tcp connections
    • Hung connections
    • Network splits / Congestion
    

    View Slide

19. Production
    integration tests
    

    View Slide

20. Chaos Gopher
    

    View Slide

21. • Unix/GO Style tools
    • Simple utilities , Slowness, Failover, Assassin
    

    View Slide

22. Where Next?
    Better integration test suites
    Allowing for flappy tests
    Statistically analysis of flapping tests
    

    View Slide

23. Other cool projects
    • Chaos Monkey (the mac daddy, Netflix)
    • Chaos Lemur (from Pivotal)
    

    View Slide

24. Questions?
    Matthew Campbell
    hyper@hyperworks.nu
    @kanwisher
    github.com/mattkanwisher
    

    View Slide