Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CloudNativeCon + KubeCon 2016: Autoscaling Containers... with Math

CloudNativeCon + KubeCon 2016: Autoscaling Containers... with Math

http://sched.co/8K3h

Docker and Kubernetes provides delightful APIs to show various statuses of our
applications. From CPU, Load average, HTTP response times, etc. we all have
what we need to make sure our app is running healthily. When things are on
fire, we Ops people twiddle some knobs like spin up more Pods to keep things
going. We use mostly our experience and knowledge of the systems that we are
running to know what to do.

However, if you look at everyday things like your airconditioners and
thermostats, they don't have an Ops team that gets paged to set the correct
level of the coolant to set your room to the right temperature. They use some
math behind called *Control Theory* to keep your room's temperature stable. In
this talk, I will show how we can use the same concepts to autoscale and manage
the health of our applications on Kubernetes.

93ac6f98100c830506d8d451245635e1?s=128

Allan Espinosa

November 08, 2016
Tweet

Transcript

  1. Autoscaling Containers…with Math @AllanEspinosa

  2. @AllanEspinosa @AllanEspinosa https://amzn.com/1785886800

  3. @AllanEspinosa

  4. @AllanEspinosa Your Distributed System Goal OPS Person Twiddle Server Farm

    CPU Utilization Monitoring Page Traffic
  5. @AllanEspinosa Your Aircon Set Temperature Thermostat Valve Coolant Actual Temperature

    Sensor The Weather
  6. @AllanEspinosa Autoscaling Target HPA Number of Pods RC Utilization Heapster

    Utilization Traffic http://kubernetes.io/docs/user-guide/horizontal-pod-autoscaling/
  7. @AllanEspinosa Control Theory • influencing dynamical systems • corrections based

    on feedback loops • math describes effectiveness
  8. @AllanEspinosa Centrifugal Governor R. Routledge, Discoveries & Inventions oftheNineteenthCentury,13thedition, 1901.

  9. @AllanEspinosa

  10. @AllanEspinosa • Target Output • Input • Output • Disturbance

    () + () () () () () − ()
  11. @AllanEspinosa

  12. @AllanEspinosa

  13. @AllanEspinosa Linear-Time Invariant Systems ∂ ∂ () + () =

    () ( + 1) = () + ()
  14. @AllanEspinosa Desired Properties • Stability • Accuracy • Settling time

    • Overshoot
  15. @AllanEspinosa Stability

  16. @AllanEspinosa

  17. @AllanEspinosa

  18. @AllanEspinosa

  19. @AllanEspinosa

  20. @AllanEspinosa Accuracy

  21. @AllanEspinosa Settling time

  22. @AllanEspinosa Overshoot

  23. @AllanEspinosa Controllers () + () () () () () −

    () = () − () () = ∞ ∑ =−∞ ( − )()
  24. @AllanEspinosa Proportional Control () = ()

  25. @AllanEspinosa () ()

  26. @AllanEspinosa Proportional Control • inherently inaccurate • increases overshoot and

    settling time
  27. @AllanEspinosa Integral Control () = ( − 1) + ()

    • reduce steady-state error • increase settling times
  28. @AllanEspinosa Derivative Control () = [() − ( − 1)]

    • decrease settling times • sensitive to noise
  29. @AllanEspinosa Non-linearity 40 50 60 70 80 90 100 300

    500 700 900 Number of Replicas CPU Utilization
  30. @AllanEspinosa Other Applications • response time vs keepalive timeout •

    memory vs worker connections • multiple input-multiple output Autotuning, Autoconfiguration
  31. @AllanEspinosa Summary • iterate on feedback • effectiveness of feedback

    • linear models go a long way • re-evaluate your models!
  32. @AllanEspinosa References P. Janert, Feedback Control for Computer Sys- tems,

    Sebastapol, CA: O’Reilly Media, 2014. J. Hellerstein, et. al., Feedback Control of Com- putingSystems,Hoboken,NJ:JohnWiley&Sons, 2004.
  33. @AllanEspinosa Acknowledgements Autoscaling experiments were run on GoogleContainerEnginewithGCPcredits c/o Kelsey

    Hightower.
  34. @AllanEspinosa Demo

  35. @AllanEspinosa Thank You! @AllanEspinosa https://github.com/aespinosa/control-theory