Living without scheduled maintenance

A97a75c945507f70992f579a730b0657?s=47 Doug Barth
October 08, 2014

Living without scheduled maintenance

An ignite talk I gave at DevOpsDays Chicago 2014. Here's 4 strategies for avoiding scheduled maintenance.

Note: this is a slight variation from my ignite format, because the chat format is annoying to glance through.

Video of the talk is available on youtube: https://www.youtube.com/watch?v=01Qfk7d8CMg&feature=youtu.be&t=21m36s

A97a75c945507f70992f579a730b0657?s=128

Doug Barth

October 08, 2014
Tweet

Transcript

  1. 10/8/14 Living without scheduled maintenance DEVOPSDAYS CHICAGO 2014 @dougbarth

  2. 10/8/14 Why should you care? LIVING WITHOUT SCHEDULED MAINTENANCE You

    like sleep and weekends Customers hate maintenance Makes your system more reliable
  3. 10/8/14 #1 - Rock solid deployments LIVING WITHOUT SCHEDULED MAINTENANCE

    Scripted deployments/rollback Forward and backward compatibility Red/blue/green deployment Stateless app servers
  4. 10/8/14 #2 - Avoid knife edge changes LIVING WITHOUT SCHEDULED

    MAINTENANCE Canary deploys Feature gating, slow rollout Slowly bleed over traffic New critical path code on the side
  5. 10/8/14 #3 - Retry retry retry LIVING WITHOUT SCHEDULED MAINTENANCE

    Build retries into all service layer hops Idempotent requests Queues Circuit breakers
  6. 10/8/14 #4 - Distribute your data LIVING WITHOUT SCHEDULED MAINTENANCE

    Multi-master clusters - MySQL Galera Quorum systems - Cassandra Sharding
  7. 10/8/14 LIVING WITHOUT SCHEDULED MAINTENANCE #1 - Rock solid deployments

    #2 - Avoid knife edge changes #3 - Retry retry retry #4 - Distribute your data