DevOpsDays Silicon Valley 2016: Autoscaling Containers... with Math

DevOpsDays Silicon Valley 2016: Autoscaling Containers... with Math

Presented at DevOpsDays Silicon Valley 2016 http://dodsv2016.busyconf.com/schedule#activity_57282a2fd4bb1947a60000a8

Docker and Kubernetes provide delightful APIs to show various statuses of our applications. Whether CPU, Load average, HTTP response times, etc., we have all that we need to make sure our app is running healthily. When things are on fire, we Ops people twiddle some knobs like spin up more Pods to keep things going. We mostly use our experience and knowledge of the systems that we are running to know what to do.

However, if you look at everyday things like your air conditioner and thermostat, they don't have an Ops team that gets paged to set the correct level of the coolant to set your room to the right temperature. They use some math called Control Theory to keep your room's temperature stable. In this talk, I will show how we can use the same concepts to autoscale and manage the health of our applications on Kubernetes.

93ac6f98100c830506d8d451245635e1?s=128

Allan Espinosa

June 24, 2016
Tweet

Transcript

  1. 4.

    @AllanEspinosa Ruby on Rails • Redis • RDS • Node.js

    • Chef • Nginx • PHP Built and proven on over 1 Billion AWS hours. Reliability and support you can trust. We ARE DevOps use promo code “DEVOPSDAYS2016” for 500 free trial hours www.engineyard.com/trial
  2. 7.

    @AllanEspinosa Support questions • How many unicorn workers should I

    have? • Why is monit killing my sidekiq workers?
  3. 9.

    @AllanEspinosa Autoscaling Target HPA Number of Pods RC Utilization Heapster

    Utilization Traffic http://kubernetes.io/docs/user-guide/horizontal-pod-autoscaling/
  4. 10.
  5. 12.

    @AllanEspinosa The Royal Society is collaborating with JSTOR to digitize,

    preserve, and extend access to Proceedings of the Royal Society of London. www.jsto
  6. 13.
  7. 14.

    @AllanEspinosa • Regulatory Control • Disturbance rejection 0 0.1 0.2

    0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 2000 4000 6000 8000 10000 0 100 200 300 400 500 600 CPU Utilization Number of Instances Time (s) CPU Utilization Number of Instances
  8. 25.

    @AllanEspinosa Controllers () + () () () () () −

    () = () − () () = ∞ ∑ u=−∞ ( − )()
  9. 28.

    @AllanEspinosa Integral Control u () = u ( − 1)

    + u () • reduce steady-state error • increase settling times
  10. 29.

    @AllanEspinosa Derivative Control u () = u [() − (

    − 1)] • decrease settling times • sensitive to noise
  11. 30.

    @AllanEspinosa Non-linearity 0 0.2 0.4 0.6 0.8 1 0 20

    40 60 80 100 120 CPU Utilization Number of Instances
  12. 31.

    @AllanEspinosa Summary • iterate on feedback • effectiveness of feedback

    • linear models go a long way • re-evaluate your models!
  13. 33.

    @AllanEspinosa Fill out our survey by 3PM on the final

    day of the conference for a chance to win: Fighter Building Kit ey.io/EYprize