The Kubernetes API & Next Generation Automation Tools

The Kubernetes API provides new ways to build automation tools that are highly available and scale to serve clusters with many thousands of nodes. During this session attendees will learn how to move beyond shell scripts and configuration management tools and leverage cluster level APIs and distributed systems design patterns to build next generation automation tools. Attendees will learn how to extend the Kubernetes API and utilize their own custom declarative configuration to build controllers that automate job scheduling or schedulers that allow you to place jobs based on your rules.

Transcript

1. Ian Lewis
   Developer Advocate, Google Cloud Platform
   The Kubernetes API & Next
   Generation Automation Tools
   ContainerCon Beijing 2017
   

   View Slide

2. Confidential & Proprietary
   Google Cloud Platform 2
   Ian Lewis
   Developer Advocate - Google Cloud Platform
   Tokyo, Japan
   +Ian Lewis
   @IanMLewis
   

   View Slide

3. Kubernetes
   κυβερνήτης: Greek for “pilot” or “helmsman of a ship”
   the open source cluster manager from Google
   

   View Slide

4. Google Cloud Platform
   Cloud Native Computing Foundation
   

   View Slide

5. Google Cloud Platform
   Goal: Avoid vendor lock-in
   Runs in many environments, including
   “bare metal” and “your laptop”
   The API and the implementation are
   100% open
   The whole system is modular and
   replaceable
   Workload portability
   

   View Slide

6. Google Cloud Platform
   Goal: Write once, run anywhere*
   Don’t force apps to know about
   concepts that are
   cloud-provider-specific
   Examples of this:
   ● Network model
   ● Ingress
   ● Service load-balancers
   ● PersistentVolumes
   * approximately
   Workload portability
   

   View Slide

7. Google Cloud Platform
   Goal: Avoid coupling
   Don’t force apps to know about
   concepts that are Kubernetes-specific
   Examples of this:
   ● Namespaces
   ● Services / DNS
   ● Downward API
   ● Secrets / ConfigMaps
   Workload portability
   

   View Slide

8. Google Cloud Platform
   Result: Portability
   Build your apps on-prem, lift-and-shift
   into cloud when you are ready
   Don’t get stuck with a platform that
   doesn’t work for you
   Put your app on wheels and move it
   whenever and wherever you need
   Workload portability
   

   View Slide

9. Kubernetes is a
   platform for building
   distributed systems
   

   View Slide

10. Your
    App
    

    View Slide

11. Your
    Platform
    

    View Slide

12. Google Cloud Platform
    users
    SSH
    SSH
    SSH
    The Clusters of Old
    server
    server
    server
    

    View Slide

13. Google Cloud Platform
    users
    API
    Container Clusters
    Container
    Cluster
    

    View Slide

14. Google Cloud Platform
    UI
    CLI
    API
    users master nodes
    etcd
    scheduler
    controllers
    master
    The 10000 Foot View
    kubelet
    kubelet
    kubelet
    kubelet
    kubelet
    kubelet
    kubelet
    kubelet
    kubelet
    

    View Slide

15. Google Cloud Platform
    Small group of containers & volumes
    Tightly coupled
    The atom of scheduling & placement
    Shared namespace
    • share IP address & localhost
    • share IPC, etc.
    Managed lifecycle
    • bound to a node, restart in place
    • can die, cannot be reborn with same ID
    Example: data puller & web server
    Consumers
    Content
    Manager
    File
    Puller
    Web
    Server
    Volume
    Pod
    Pods
    

    View Slide

16. Google Cloud Platform
    Docker Containers
    IPC
    Network
    PID
    Hostname
    Mount
    nginx
    IPC
    Network
    PID
    Hostname
    Mount
    nginx
    IPC
    Network
    PID
    Hostname
    Mount
    nginx
    

    View Slide

17. Google Cloud Platform
    IPC
    Network
    Pods
    docker …
    --net=container:id
    --ipc=container:id
    Hostname
    cgroup
    Web
    Server
    Pod
    cgroup
    File
    Puller
    localhost
    

    View Slide

18. Google confidential │ Do not
    distribute
    Services
    A group of pods that work together
    • grouped by a selector
    Defines access policy
    • “load balanced” or “headless”
    Gets a stable virtual IP and port
    • sometimes called the service portal
    • also a DNS name
    VIP is managed by kube-proxy
    • watches all services
    • updates iptables when backends change
    Hides complexity - ideal for non-native apps
    Virtual IP
    Client
    

    View Slide

19. Google Cloud Platform
    Deployments
    ReplicaSet
    - replicas: 3
    - selector:
    - app: MyApp
    - version: v1
    Deployment
    - name: MyApp
    kubectl apply ...
    

    View Slide

20. Google Cloud Platform
    Deployments
    ReplicaSet
    - replicas: 4
    - selector:
    - app: MyApp
    - version: v1
    Deployment
    - name: MyApp
    

    View Slide

21. Google Cloud Platform
    Deployments
    ReplicaSet
    - replicas: 3
    - selector:
    - app: MyApp
    - version: v1
    Deployment
    - name: MyApp
    

    View Slide

22. Google Cloud Platform
    Deployments
    ReplicaSet
    - replicas: 3
    - selector:
    - app: MyApp
    - version: v1
    Deployment
    - name: MyApp
    

    View Slide

23. Google Cloud Platform
    Rolling Updates
    ReplicaSet
    - replicas: 3
    - selector:
    - app: MyApp
    - version: v1
    Deployment
    - name: MyApp
    kubectl apply ...
    

    View Slide

24. Google Cloud Platform
    ReplicaSet
    - replicas: 3
    - selector:
    - app: MyApp
    - version: v1
    Rolling Updates
    ReplicaSet
    - replicas: 0
    - selector:
    - app: MyApp
    - version: v2
    Deployment
    - name: MyApp
    

    View Slide

25. Google Cloud Platform
    ReplicaSet
    - replicas: 3
    - selector:
    - app: MyApp
    - version: v1
    ReplicaSet
    - replicas: 1
    - selector:
    - app: MyApp
    - version: v2
    Rolling Updates
    Deployment
    - name: MyApp
    

    View Slide

26. Google Cloud Platform
    ReplicaSet
    - replicas: 2
    - selector:
    - app: MyApp
    - version: v1
    ReplicaSet
    - replicas: 1
    - selector:
    - app: MyApp
    - version: v2
    Deployment
    - app: MyApp
    Rolling Updates
    

    View Slide

27. Google Cloud Platform
    ReplicaSet
    - replicas: 2
    - selector:
    - app: MyApp
    - version: v1
    ReplicaSet
    - replicas: 2
    - selector:
    - app: MyApp
    - version: v2
    Rolling Updates
    Deployment
    - name: MyApp
    

    View Slide

28. Google Cloud Platform
    ReplicaSet
    - replicas: 1
    - selector:
    - app: MyApp
    - version: v1
    ReplicaSet
    - replicas: 2
    - selector:
    - app: MyApp
    - version: v2
    Rolling Updates
    Deployment
    - name: MyApp
    

    View Slide

29. Google Cloud Platform
    ReplicaSet
    - replicas: 1
    - selector:
    - app: MyApp
    - version: v1
    ReplicaSet
    - replicas: 3
    - selector:
    - app: MyApp
    - version: v2
    Rolling Updates
    Deployment
    - name: MyApp
    

    View Slide

30. Google Cloud Platform
    ReplicaSet
    - replicas: 0
    - selector:
    - app: MyApp
    - version: v1
    ReplicaSet
    - replicas: 3
    - selector:
    - app: MyApp
    - version: v2
    Rolling Updates
    Deployment
    - name: MyApp
    

    View Slide

31. Google Cloud Platform
    ReplicaSet
    - replicas: 3
    - selector:
    - app: MyApp
    - version: v2
    Rolling Updates
    Deployment
    - name: MyApp
    

    View Slide

32. Google Cloud Platform
    observe
    diff
    act
    Controllers
    

    View Slide

33. Google Cloud Platform
    Pods
    Watch
    observe
    diff
    act
    Container
    Run/Stop
    Kubelet
    

    View Slide

34. Google Cloud Platform
    Services
    Watch
    observe
    diff
    act
    iptables
    Create/Update
    Kube-Proxy
    

    View Slide

35. Google Cloud Platform
    Service
    Watch
    observe
    diff
    act
    Load Balancer
    Create/Update
    Service/LB Controller
    

    View Slide

36. Google Cloud Platform
    ReplicaSet
    Watch
    observe
    diff
    act
    Pod
    Create/Delete
    ReplicaSet Controller
    

    View Slide

37. Google Cloud Platform
    The 10000 Foot View
    etcd
    scheduler
    controller-manager
    apiserver
    kubelet docker
    kube-proxy iptables
    1. User creates & pushes image
    2. User creates Deployment
    3. API server saves info to etcd
    4. CM finds Deployment and
    creates ReplicaSet, which
    creates Pods (unscheduled)
    5. Scheduler schedules pods.
    6. Kubelet sees pod scheduled to
    it and tells docker to run the
    container.
    7. Docker pulls and runs the
    container.
    Docker Hub
    / GCR
    

    View Slide

38. Google Cloud Platform
    controller-manager
    Deployment
    metadata.name: nginx
    Deployment
    Controller
    ReplicaSet
    Controller
    Scheduler
    

    View Slide

39. Google Cloud Platform
    controller-manager
    Deployment
    metadata.name: nginx
    Deployment
    Controller
    ReplicaSet
    Controller
    Scheduler
    

    View Slide

40. Google Cloud Platform
    controller-manager
    ReplicaSet
    metadata.name: nginx-xxxx
    Deployment
    metadata.name: nginx
    Deployment
    Controller
    ReplicaSet
    Controller
    Scheduler
    

    View Slide

41. Google Cloud Platform
    controller-manager
    ReplicaSet
    metadata.name: nginx-xxxx
    Deployment
    metadata.name: nginx
    Deployment
    Controller
    ReplicaSet
    Controller
    Scheduler
    

    View Slide

42. Google Cloud Platform
    controller-manager
    Pod
    metadata.name: nginx-xxxx-xxxx
    spec.nodeName:
    ReplicaSet
    metadata.name: nginx-xxxx
    Deployment
    metadata.name: nginx
    Deployment
    Controller
    ReplicaSet
    Controller
    Scheduler
    

    View Slide

43. Google Cloud Platform
    controller-manager
    Pod
    metadata.name: nginx-xxxx-xxxx
    spec.nodeName:
    ReplicaSet
    metadata.name: nginx-xxxx
    Deployment
    metadata.name: nginx
    Deployment
    Controller
    ReplicaSet
    Controller
    Scheduler
    

    View Slide

44. Google Cloud Platform
    controller-manager
    Pod
    metadata.name: nginx-xxxx-xxxx
    spec.nodeName: node1
    ReplicaSet
    metadata.name: nginx-xxxx
    Deployment
    metadata.name: nginx
    Deployment
    Controller
    ReplicaSet
    Controller
    Scheduler
    

    View Slide

45. Google Cloud Platform
    Pod
    metadata.name: nginx-xxxx-xxxx
    spec.nodeName: node1
    status: Pending
    node1
    kubelet docker
    

    View Slide

46. Google Cloud Platform
    Pod
    metadata.name: nginx-xxxx-xxxx
    spec.nodeName: node1
    status: ContainerCreating
    node1
    kubelet docker
    

    View Slide

47. Google Cloud Platform
    Pod
    metadata.name: nginx-xxxx-xxxx
    spec.nodeName: node1
    status: ContainerCreating
    node1
    kubelet docker
    

    View Slide

48. Google Cloud Platform
    Pod
    metadata.name: nginx-xxxx-xxxx
    spec.nodeName: node1
    status: ContainerCreating
    node1
    kubelet docker
    Docker Hub /
    GCR
    

    View Slide

49. Google Cloud Platform
    Pod
    metadata.name: nginx-xxxx-xxxx
    spec.nodeName: node1
    status: ContainerCreating
    node1
    kubelet docker
    nginx-xxxx-x
    xxx
    

    View Slide

50. Google Cloud Platform
    Pod
    metadata.name: nginx-xxxx-xxxx
    spec.nodeName: node1
    status: ContainerCreating
    node1
    kubelet docker
    nginx-xxxx-x
    xxx
    

    View Slide

51. Google Cloud Platform
    Pod
    metadata.name: nginx-xxxx-xxxx
    spec.nodeName: node1
    status: Running
    node1
    kubelet docker
    nginx-xxxx-x
    xxx
    

    View Slide

52. for {
    resp := http.Get(secretsEndpoint)
    list := json.Unmarshal(resp.Body)
    for _, secret := range list.Items {
    data := base64Decode(secret[“data”])
    if len(data[“password”]) < 10 {
    log.Printf(“Insecure secret: %s”, secret.Metadata.Name)
    http.Delete(secretsEndpoint + secret.Metadata.Name)
    }
    }
    }
    

    View Slide

53. for {
    resp := http.Get(crontabEndpoint)
    list := json.Unmarshal(resp.Body)
    for _, crontab := range list.Items {
    addIfNew(crontab)
    updateIfChanged(crontab)
    deleteIfOld(crontab)
    }
    }
    

    View Slide

54. Thank You
    

    View Slide