Cluster Autoscaler: Dynamic & Automatic Reconfiguration

Transcript

1. Cluster Autoscaler Dynamic & Automatic Reconfiguration @mumoshu, primary maintainer of

   kube-aws/contributor of cluster- autoscaler Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 1

2. Կͷ࿩ • K8S on AWSͰΦʔτεέʔϦϯάΛ͢Δ • pod͡Όͳͯ͘nodeͷ΄͏ • cluster-autoscaler(CA)Λ࢖͏ Kubernetes

   Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 2

3. Cluster Autoscaler: FAQ • Ϋϥ΢υϓϩόΠμඪ४ͷΦʔτεέʔϦϯάͱͷҧ͍? • AWS AutoScaling: based on

   CPU/RequestCount(ALB) • Cluster Autoscaler: Autoscaling based on total resource requests • e.g. add node(s) when CA sees pending pods • Pod࡞Γ͗ͯᷓ͢ΕͨΒϊʔυ૿΍͢Α! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 3

4. Why: I was too lazy to operate CA • K8SΫϥελͷӡ༻΋΍Διϑτ΢ΣΞΤϯδχΞ

   • Φʔτεέʔϧ͸ར༻͍ͨ͠ • CAͷӡ༻͕஍ຯʹΊΜͲ͍͘͞ I'm responsible to run several production K8S clusters on AWS. I'd like to enable autoscaling on them but I'm too lazy to operate CA. I'm basically a software who loves sketching & coding, not operating ! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 4

5. Why: Syncing node pool config between cloud<->CA • ASG on

   AWSͱCA on K8Sͷઃఆ͸ผ • Cloud Provider: Node Pool Min/Max Size • CA: Node Pool Min/Max Size • Change(s) on AWS-side requires updating/restarting CA • ໃ६͕͋Δͱਖ਼͘͠ಈ͔ͳ͍ • AWSଆͰઃఆΛม͑ͨΒɺCAͷઃఆΛม͑ͯ࠶ىಈ͕ඞཁ Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 5

6. Node Pools/ϊʔυϓʔϧ • AutoScaling Group(AWS) aws autoscaling update-auto-scaling-group \ --auto-scaling-group-name

   $NAME \ --min-size $MIN \ --max-size $MAX • Instance Group in GCE? Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 6

7. CA 1. kubectl edit deployment cluster-autoscaler 2. cluster-autoscaler --nodes $MIN:$MAX:$NAME

   Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 7

8. 2౓खؒ • ࡞ۀ΋Εͦ͠͏ • Scripting? • ͜Ε͚ͩͷͨΊʹ…? Kubernetes Meetup Tokyo

   #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 8

9. ASGͱCAͷઃఆ • Ͳ͕ͬͪਖ਼ͳͷ? • ASGͷ৔߹ • CAͷmin >= ASGͷmin •

   CAͷmax <= ASGͷmax Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 9

10. How • ASGͷઃఆΛCA͕ࣗಈతʹಡΜͰ͘ΕΕ͹Α͘ͳ͍? Shouldn't CA automatically sync up with AWS

    about ASGs it is told to manage? Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 10

11. What • CAͷ৽ػೳ 1. Dynamic Reconfiguration 2. Node Group Automatic

    Discovery ! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 11

12. Dynamic Reconfiguration • CAͷઃఆΛConfigMapʹॻ͘ • ConfigMapΛॻ͖׵͑ΔͱCA͕ࣗಈϦϩʔυ(--configmap) • PR: github.com/kubernetes/contrib/pull/2226 Kubernetes

    Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 12

13. Dynamic Reconfigurationͷར༻ํ๏ kubectl create configmap cluster-autoscaler -f ca-config.yaml Kubernetes Meetup

    Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 13

14. kind: ConfigMap apiVersion: v1 metadata: name: cluster-autoscaler namespace: kube-system data:

    settings: |- { "nodeGroups": [ { "minSize": 1, "maxSize": 2, "name": "myasg" } ] } Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 14

15. ./cluster-autoscaler --configmap=cluster-autoscaler ... Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic

    & Automatic Reconfiguration © 2017 mumoshu 15

16. όʔδϣϯ • CA 0.5.2͔Β Kubernetes Meetup Tokyo #6 "Cluster Autoscaler:

    Dynamic & Automatic Reconfiguration © 2017 mumoshu 16

17. Dynamic ReconfigurationͷͭΒ͍ͱ͜Ζ • kubectl edit deployment ca͕kubectl edit configmap caʹมΘ͚ͬͨͩͰ͸

    • ͔֬ʹएׯ៉ྷʹεΫϦϓτԽ͸͠΍͘͢ͳΔ͚Ͳ • BEFORE: εΫϦϓτ͸deploymentͷ໊લͱશઃఆɺcaί ϚϯυϥΠϯͷॻ͖׵͑ํΛ஌Βͳ͍ͱ͍͚ͳ͍ • AFTER: configmapͷ໊લͱઃఆ஋͚ͩ஌͍ͬͯΕ͹͍͍ Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 17

18. ͦ͜Ͱ Node Group Automatic Discovery Kubernetes Meetup Tokyo #6 "Cluster

    Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 18

19. Automatic discovery in nutshell ҰݴͰ͍͏ͱɺ CA͕AWS APIΛ࢖ͬͯASGͷઃఆΛࣗಈಉظͯ͘͠ΕΔػೳ cluster-autoscaler: Re: AWS

    Autoscaler autodiscover ASG names and sizes github.com/kubernetes/autoscaler/pull/11 Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 19

20. How it works ͋ͳͨ: • ASGʹλάΛ͚ͭΔ • CAʹλάΛڭ͑Δ CA: •

    λάͷ͍ͭͨASGΛݕ஌͢Δ ! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 20

21. How operating CA improved? BEFORE: cluster-autoscaler --nodes 1:10:myasg-123456 & ASGઃఆมߋͷͨͼʹ͜ͷҾ਺ΛରԠͤ͞Δӡ༻

    AFTER: cluster-autoscaler --node-group-automatic-discovery asg:tag:key1=val1,key2=val2 & ӡ༻ͷखؒͳ͠ ! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 21

22. ࠓͷͱ͜ΖAWSͷΈ • ΠϯλϑΣʔε͸GCE΍Azureʹ΋ରԠͰ͖ΔΑ͏ʹ੾ͬͨ ͭ΋Γ The automatic discovery feature works only

    with the AWS provider as of today. However, the common interface is there for you to easily add GCE/Azure supports ! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 22

23. GCEରԠΠϝʔδ AWS(Supported): --cloudprovider=aws \ --node-group-automatic-discovery asg:tag:key1=val1,key2=val2 GCE(Unsupported/Imaginary interface): --cloudprovider=gce \

    --node-group-automatic-discovery gce:label:key1=val1,key2=val2 An instance group could be "labeled" in GCE, right? Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 23

24. GCE, AzureରԠٻΉ! Would anyone mind contributing supports for GCE and

    Azure to the automatic discovery feature?! Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 24

25. Supported since CA 0.6 • Automatic Discovery͸CA 0.6͔Β • kube-aws

    v0.9.8-rc.1͔Β • CA 0.6͕σϑΥϧτ • Automatic DiscoveryΛΑ͠ͳʹઃఆͯ͘͠ΕΔ Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 25

26. kube-aws: Configuring CA autoscaling: clusterAutoscaler: enabled: true Kubernetes Meetup Tokyo

    #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 26

27. kube-aws: Configuring CA per node pool worker: nodePools: # CA

    is disabled by default - name: static # CA is enabled only on this node pool - name: dynamic autoscaling: clusterAutoscaler: enabled: true Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 27

28. ·ͱΊ/Recap • CAͷӡ༻ࣗಈԽ/ָʹΦʔτεέʔϦϯά͕࢖͑Δ • ͍·ͷͱ͜ΖAWS޲͚ͷํ͕ॆ࣮ • Automate a boring operational

    task for CA • Dynamic Reconfiguration: supported since CA 0.5.2 • Automatic Discovery: supported on AWS since CA 0.6 • kube-aws supports CA & Automatic Discovery Kubernetes Meetup Tokyo #6 "Cluster Autoscaler: Dynamic & Automatic Reconfiguration © 2017 mumoshu 28