DevOpsDays Minneapolis 2016: Autoscaling Containers... with Math

Presented at DevOpsDays Minneapolis. http://www.devopsdays.org/events/2016-minneapolis/program/allan-espinosa/

Also shows how to write your own Autoscaler in Kubernetes

Docker and Kubernetes provide delightful APIs to show various statuses of our applications. Whether CPU, Load average, HTTP response times, etc., we have all that we need to make sure our app is running healthily. When things are on fire, we Ops people twiddle some knobs like spin up more Pods to keep things going. We mostly use our experience and knowledge of the systems that we are running to know what to do.

However, if you look at everyday things like your air conditioner and thermostat, they don't have an Ops team that gets paged to set the correct level of the coolant to set your room to the right temperature. They use some math called Control Theory to keep your room's temperature stable. In this talk, I will show how we can use the same concepts to autoscale and manage the health of our applications on Kubernetes.

Transcript

1. Autoscaling Containers…with Math
   @AllanEspinosa
   

   View Slide

2. @AllanEspinosa
   

   View Slide

3. @AllanEspinosa
   Your Distributed System
   Goal OPS Person Twiddle Server Farm CPU Utilization
   Monitoring
   Page
   Traffic
   

   View Slide

4. @AllanEspinosa
   Your Aircon
   Set Temperature
   Thermostat
   Valve
   Coolant
   Actual Temperature
   Sensor
   The Weather
   

   View Slide

5. @AllanEspinosa
   Autoscaling
   Target HPA Number of Pods RC Utilization
   Heapster
   Utilization
   Traffic
   http://kubernetes.io/docs/user-guide/horizontal-pod-autoscaling/
   

   View Slide

6. @AllanEspinosa
   Control Theory
   • influencing dynamical systems
   • corrections based on feedback loops
   • math describes effectiveness
   

   View Slide

7. @AllanEspinosa
   R. Routledge, Discoveries & Inventions of the Nineteenth Century, 13th edition, 1901.
   

   View Slide

8. @AllanEspinosa
   

   View Slide

9. @AllanEspinosa
   Control Theory
   • Target Output
   • Input
   • Output
   • Disturbance
   () + ()
   ()
   ()
   () ()
   −
   ()
   

   View Slide

10. @AllanEspinosa
    

    View Slide

11. @AllanEspinosa
    

    View Slide

12. @AllanEspinosa
    Linear-Time Invariant Systems
    
    ∂
    ∂
    () + () = ()
    ( + 1) = () + ()
    

    View Slide

13. @AllanEspinosa
    Desired Properties
    • Stability
    • Accuracy
    • Settling time
    • Overshoot
    

    View Slide

14. @AllanEspinosa
    Stability
    

    View Slide

15. @AllanEspinosa
    

    View Slide

16. @AllanEspinosa
    

    View Slide

17. @AllanEspinosa
    

    View Slide

18. @AllanEspinosa
    

    View Slide

19. @AllanEspinosa
    Accuracy
    

    View Slide

20. @AllanEspinosa
    Settling time
    

    View Slide

21. @AllanEspinosa
    Overshoot
    
    

    View Slide

22. @AllanEspinosa
    Controllers
    () + ()
    ()
    ()
    () ()
    −
    () = () − ()
    () =
    ∞
    ∑
    u=−∞
    ( − )()
    

    View Slide

23. @AllanEspinosa
    Proportional Control
    u
    () = u
    ()
    ()
    u
    ()
    

    View Slide

24. @AllanEspinosa
    Proportional Control
    • inherently inaccurate
    • u
    increases overshoot and settling time
    

    View Slide

25. @AllanEspinosa
    Integral Control
    u
    () = u
    ( − 1) + u
    ()
    • reduce steady-state error
    • increase settling times
    

    View Slide

26. @AllanEspinosa
    Derivative Control
    u
    () = u
    [() − ( − 1)]
    • decrease settling times
    • sensitive to noise
    

    View Slide

27. @AllanEspinosa
    Non-linearity
    40 50 60 70 80 90 100
    0.5 0.7 0.9 1.1
    Number of Replicas
    CPU Utilization
    

    View Slide

28. @AllanEspinosa
    Summary
    • iterate on feedback
    • effectiveness of feedback
    • linear models go a long way
    • re-evaluate your models!
    

    View Slide

29. @AllanEspinosa
    P. Janert, Feedback Control for Computer Systems, Sebastapol, CA: O’Reilly Media, 2014.
    J. Hellerstein, et. al., Feedback Control of Computing Systems, Hoboken, NJ: John Wiley & Sons,
    2004.
    

    View Slide

30. @AllanEspinosa
    Thank You!
    @AllanEspinosa
    https://github.com/aespinosa/control-theory
    

    View Slide