DevOpsDays Silicon Valley 2016: Autoscaling Containers... with Math

Presented at DevOpsDays Silicon Valley 2016 http://dodsv2016.busyconf.com/schedule#activity_57282a2fd4bb1947a60000a8

Docker and Kubernetes provide delightful APIs to show various statuses of our applications. Whether CPU, Load average, HTTP response times, etc., we have all that we need to make sure our app is running healthily. When things are on fire, we Ops people twiddle some knobs like spin up more Pods to keep things going. We mostly use our experience and knowledge of the systems that we are running to know what to do.

However, if you look at everyday things like your air conditioner and thermostat, they don't have an Ops team that gets paged to set the correct level of the coolant to set your room to the right temperature. They use some math called Control Theory to keep your room's temperature stable. In this talk, I will show how we can use the same concepts to autoscale and manage the health of our applications on Kubernetes.

Transcript

1. Autoscaling Containers…with Math
   @AllanEspinosa
   

   View Slide

2. @AllanEspinosa
   

   View Slide

3. @AllanEspinosa
   

   View Slide

4. @AllanEspinosa
   Ruby on Rails • Redis • RDS • Node.js • Chef • Nginx • PHP
   Built and proven on over 1 Billion AWS hours.
   Reliability and support you can trust.
   We ARE DevOps
   use promo code “DEVOPSDAYS2016”
   for 500 free trial hours
   www.engineyard.com/trial
   

   View Slide

5. @AllanEspinosa
   Your Distributed System
   Goal OPS Person
   Twiddle
   Server Farm
   CPU Utilization
   Monitoring
   Page
   Traffic
   

   View Slide

6. @AllanEspinosa
   Your Aircon
   Set Temperature
   Thermostat
   Valve
   Coolant
   Actual Temperature
   Sensor
   The Weather
   

   View Slide

7. @AllanEspinosa
   Support questions
   • How many unicorn workers should I have?
   • Why is monit killing my sidekiq workers?
   

   View Slide

8. @AllanEspinosa
   

   View Slide

9. @AllanEspinosa
   Autoscaling
   Target HPA
   Number of Pods
   RC
   Utilization
   Heapster
   Utilization
   Traffic
   http://kubernetes.io/docs/user-guide/horizontal-pod-autoscaling/
   

   View Slide

10. @AllanEspinosa
    Control Theory
    • influencing dynamical systems
    • corrections based on feedback loops
    • math describes effectiveness
    

    View Slide

11. @AllanEspinosa
    R. Routledge, Discoveries & Inventions of the Nineteenth Century, 13th edition, 1901.
    

    View Slide

12. @AllanEspinosa
    The Royal Society is collaborating with JSTOR to digitize, preserve, and extend access to
    Proceedings of the Royal Society of London.
    www.jsto
    

    View Slide

13. @AllanEspinosa
    Control Theory
    • Target Output
    • Input
    • Output
    • Disturbance
    () + ()
    ()
    ()
    () ()
    −
    ()
    

    View Slide

14. @AllanEspinosa
    • Regulatory Control • Disturbance rejection
    0
    0.1
    0.2
    0.3
    0.4
    0.5
    0.6
    0.7
    0.8
    0.9
    1
    0 2000 4000 6000 8000 10000
    0
    100
    200
    300
    400
    500
    600
    CPU Utilization
    Number of Instances
    Time (s)
    CPU Utilization
    Number of Instances
    

    View Slide

15. @AllanEspinosa
    Linear-Time Invariant Systems
    
    ∂
    ∂
    () + () = ()
    ( + 1) = () + ()
    

    View Slide

16. @AllanEspinosa
    Desired Properties
    • Stability
    • Accuracy
    • Settling time
    • Overshoot
    

    View Slide

17. @AllanEspinosa
    Stability
    

    View Slide

18. @AllanEspinosa
    

    View Slide

19. @AllanEspinosa
    

    View Slide

20. @AllanEspinosa
    

    View Slide

21. @AllanEspinosa
    

    View Slide

22. @AllanEspinosa
    Accuracy
    

    View Slide

23. @AllanEspinosa
    Settling time
    

    View Slide

24. @AllanEspinosa
    Overshoot
    
    

    View Slide

25. @AllanEspinosa
    Controllers
    () + ()
    ()
    ()
    () ()
    −
    () = () − ()
    () =
    ∞
    ∑
    u=−∞
    ( − )()
    

    View Slide

26. @AllanEspinosa
    Proportional Control
    u
    () = u
    ()
    ()
    u
    ()
    

    View Slide

27. @AllanEspinosa
    Proportional Control
    • inherently inaccurate
    • u
    increases overshoot and settling time
    

    View Slide

28. @AllanEspinosa
    Integral Control
    u
    () = u
    ( − 1) + u
    ()
    • reduce steady-state error
    • increase settling times
    

    View Slide

29. @AllanEspinosa
    Derivative Control
    u
    () = u
    [() − ( − 1)]
    • decrease settling times
    • sensitive to noise
    

    View Slide

30. @AllanEspinosa
    Non-linearity
    0
    0.2
    0.4
    0.6
    0.8
    1
    0 20 40 60 80 100 120
    CPU Utilization
    Number of Instances
    

    View Slide

31. @AllanEspinosa
    Summary
    • iterate on feedback
    • effectiveness of feedback
    • linear models go a long way
    • re-evaluate your models!
    

    View Slide

32. @AllanEspinosa
    

    View Slide

33. @AllanEspinosa
    Fill out our survey by 3PM on the final day
    of the conference for a chance to win:
    Fighter Building Kit
    ey.io/EYprize
    

    View Slide

34. Thank You!
    @AllanEspinosa
    https://ey.io/EYprize
    https://github.com/aespinosa/control-theory
    

    View Slide