Upgrade to Pro — share decks privately, control downloads, hide ads and more …

2017 JFokus - Managing Cloud Native Applications with Kubernetes - End to End

Ray Tsang
February 06, 2017

2017 JFokus - Managing Cloud Native Applications with Kubernetes - End to End

Slides from JFokus 2017 3.5hr tutorial session: https://www.jfokus.se/jfokus/talks.jsp#ManagingCloudNativeA

The presentation is mostly composed of live demos and thus most of the slides were not used.

Ray Tsang

February 06, 2017
Tweet

More Decks by Ray Tsang

Other Decks in Technology

Transcript

  1. @saturnism @googlecloud @kubernetesio @jfokus
    Managing Cloud Native
    Application with Kubernetes
    End-to-End

    View Slide

  2. @saturnism @googlecloud @kubernetesio @jfokus
    Time Topics
    09:00 - 10:30 Containers, Orchestration
    Scheduling, Kubernetes
    10:30 - 11:00 Break. YAY!
    11:00 - 12:30 Kubernetes Federation
    CI/CD

    View Slide

  3. @saturnism @googlecloud @kubernetesio @jfokus
    Ray Tsang
    Developer Advocate
    Google Cloud Platform
    @saturnism | +RayTsang

    View Slide

  4. @saturnism @googlecloud @kubernetesio @jfokus
    Ray Tsang
    Developer
    Architect
    Traveler
    Photographer
    flickr.com/saturnism

    View Slide

  5. @saturnism @googlecloud @kubernetesio @jfokus

    View Slide

  6. @saturnism @googlecloud @kubernetesio @jfokus
    Cloud Native?
    != Running in the Cloud

    View Slide

  7. @saturnism @googlecloud @kubernetesio @jfokus
    Cloud Native?
    Innovate fast
    Iterate fast
    Focus on core business differentiators

    View Slide

  8. @saturnism @googlecloud @kubernetesio @jfokus
    Cloud Native?
    Infrastructure - elastic, automated
    Environment - consistent, agile
    Application - scalable, resilient, service oriented
    Deployment - continuous
    Everything - measurable

    View Slide

  9. @saturnism @googlecloud @kubernetesio @jfokus
    Cloud Native?
    Infrastructure - elastic, automated
    Environment - consistent, agile
    Application - scalable, resilient, service oriented
    Deployment - continuous
    Everything - measurable
    DevOps
    Machines as Cattles
    Containerization
    Orchestration
    Microservices
    CI/CD
    Orchestration
    Trace, Monitoring
    Metrics

    View Slide

  10. @saturnism @googlecloud @kubernetesio @jfokus

    View Slide

  11. @saturnism @googlecloud @kubernetesio @jfokus
    Hello World
    Service - Greet
    Guestbook
    Service - Create
    Guestbook Service -
    Retrieve

    View Slide

  12. @saturnism @googlecloud @kubernetesio @jfokus
    Guestbook
    UI
    Hello World
    Service
    Redis
    session replication
    greeting
    MySQL
    Guestbook
    Service
    CRUD

    View Slide

  13. @saturnism @googlecloud @kubernetesio @jfokus
    Package & Deployment
    application.ear
    app.war
    helloworld-service.jar
    guestbook-service.jar
    apache-xyz.jar
    application.war
    /...
    helloworld-service.jar
    guestbook-service.jar
    apache-xyz.jar

    View Slide

  14. @saturnism @googlecloud @kubernetesio @jfokus
    Machine
    app.war / app.ear
    Application Server
    Kernel
    Shell / CLI / Tools

    View Slide

  15. @saturnism @googlecloud @kubernetesio @jfokus
    Machine
    app.war / app.ear
    Application Server
    Kernel
    Shell / CLI / Tools
    Machine
    app.war / app.ear
    Application Server
    Kernel
    Shell / CLI / Tools
    Machine
    app.war / app.ear
    Application Server
    Kernel
    Shell / CLI / Tools

    View Slide

  16. @saturnism @googlecloud @kubernetesio @jfokus
    Let’s see write some code

    View Slide

  17. @saturnism @googlecloud @kubernetesio @jfokus
    Microservices
    helloworld-service.jar
    guestbook-service.jar
    app.jar

    View Slide

  18. @saturnism @googlecloud @kubernetesio @jfokus
    Deployment? Just run it!
    java -jar helloworld-service.jar
    java -jar guestbook-service.jar
    java -jar app.jar

    View Slide

  19. @saturnism @googlecloud @kubernetesio @jfokus
    So many services
    Deploy, Manage, Bin Pack, Ports, Discovery, Isolation… How?

    View Slide

  20. @saturnism @googlecloud @kubernetesio @jfokus
    Challenges
    Which host machine(s) should each service run on?
    Avoid port conflicts?
    How does the application code find the Guest Book service?
    How do we keep all of these service running?
    What happens if a host machine has trouble?
    Are the services healthy?
    How do we scale when load changes?
    Run this in another environment? QA, dev, another cloud, your servers?

    View Slide

  21. @saturnism @googlecloud @kubernetesio @jfokus
    Containers
    To the Rescue!

    View Slide

  22. @saturnism @googlecloud @kubernetesio @jfokus
    Old Way: Shared machines
    kernel
    libs
    app
    app app
    No isolation
    No namespacing
    Common libs
    Highly coupled apps and OS
    app

    View Slide

  23. @saturnism @googlecloud @kubernetesio @jfokus
    Old Way: Virtual machines
    Some isolation
    Expensive and inefficient
    Still highly coupled to the guest OS
    Hard to manage
    app
    libs
    kernel
    libs
    app app
    kernel
    app
    libs
    libs
    kernel
    kernel

    View Slide

  24. @saturnism @googlecloud @kubernetesio @jfokus
    New Way: Containers
    libs
    app
    kernel
    libs
    app
    libs
    app
    libs
    app

    View Slide

  25. @saturnism @googlecloud @kubernetesio @jfokus
    Containerize Option #1
    Dockerfile

    View Slide

  26. @saturnism @googlecloud @kubernetesio @jfokus
    Containerize Option #2
    spotify/docker-maven-plugin
    mvn docker:build
    or, spotify/dockerfile-maven-plugin
    or, fabric8 maven plugin

    View Slide

  27. @saturnism @googlecloud @kubernetesio @jfokus
    Containerize Option #3
    Docker Hub / GitHub
    saturnism/spring-boot

    View Slide

  28. @saturnism @googlecloud @kubernetesio @jfokus
    Static Binary!
    Runs anywhere

    View Slide

  29. @saturnism @googlecloud @kubernetesio @jfokus
    More on Containers
    Container Basics

    View Slide

  30. @saturnism @googlecloud @kubernetesio @jfokus
    Let’s run the container!
    docker run -ti -p 8080:8080 helloworld-service

    View Slide

  31. @saturnism @googlecloud @kubernetesio @jfokus
    Docker Compose
    docker-compose up

    View Slide

  32. @saturnism @googlecloud @kubernetesio @jfokus
    Versioning container image
    docker tag spring-boot-demo spring-boot-demo:1.0

    View Slide

  33. @saturnism @googlecloud @kubernetesio @jfokus
    Versioning container image
    docker build -t spring-boot-demo:${BUILDNO}

    View Slide

  34. @saturnism @googlecloud @kubernetesio @jfokus
    Don’t Log to Container Filesystem!
    Log to a volume… docker -v /tmp/log:/log
    Or
    Send it elsewhere!
    STDOUT is captured by Docker / Kubernetes

    View Slide

  35. @saturnism @googlecloud @kubernetesio @jfokus
    Clean up disk spaces
    Every image, layer, and, even containers litters
    docker rm $(docker ps -a -q)
    docker rmi $(docker images -q --filter dangling=true)

    View Slide

  36. @saturnism @googlecloud @kubernetesio @jfokus
    Combine RUN commands
    apt-get update && apt-get install xyz && apt-get clean
    Saves you space.

    View Slide

  37. @saturnism @googlecloud @kubernetesio @jfokus
    Consider Slim JARs
    Base Layer - java:8 (not frequently updated)
    Next Layer - Dependency JARs (not frequently updated)
    Last Layer - Application JAR (frequently updated)
    Use dockerfile-maven-plugin, copy-dependencies or slimfast

    View Slide

  38. @saturnism @googlecloud @kubernetesio @jfokus
    Run with --rm
    docker run -ti --rm saturnism/spring-boot-helloworld-service:1.0

    View Slide

  39. @saturnism @googlecloud @kubernetesio @jfokus
    Everything at Google
    runs in containers
    Launch over 2 billion
    containers per week.

    View Slide

  40. @saturnism @googlecloud @kubernetesio @jfokus

    View Slide

  41. @saturnism @googlecloud @kubernetesio @jfokus
    Control Plane
    Apps
    Software
    Engineers

    View Slide

  42. @saturnism @googlecloud @kubernetesio @jfokus
    job hello_world = {
    runtime = { cell = 'ic' } // Cell (cluster) to run in
    binary = '.../hello_world_webserver' // Program to run
    args = { port = '%port%' } // Command line parameters
    requirements = { // Resource requirements
    ram = 100M
    disk = 100M
    cpu = 0.1
    }
    replicas = 5 // Number of tasks
    }
    10000

    View Slide

  43. @saturnism @googlecloud @kubernetesio @jfokus
    web browsers
    BorgMaster
    link shard
    UI shard
    BorgMaster
    link shard
    UI shard
    BorgMaster
    link shard
    UI shard
    BorgMaster
    link shard
    UI shard
    Scheduler
    borgcfg web browsers
    scheduler
    Borglet Borglet Borglet Borglet
    Config
    file
    BorgMaster
    link shard
    UI shard
    persistent store
    (Paxos)
    Binary
    What just
    happened?

    View Slide

  44. @saturnism @googlecloud @kubernetesio @jfokus
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world! Hello
    world! Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Image by Connie
    Zhou
    Hello
    world!
    Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world!
    Hello
    world!

    View Slide

  45. @saturnism @googlecloud @kubernetesio @jfokus

    View Slide

  46. @saturnism @googlecloud @kubernetesio @jfokus
    Containers Orchestration

    View Slide

  47. @saturnism @googlecloud #kubernetes #devoxx

    View Slide

  48. @saturnism @googlecloud #kubernetes #devoxx
    Kubernetes
    Greek for “Helmsman”; also the root of the
    words “governor” and “cybernetic”
    • Infrastructure for containers
    • Schedules, runs, and manages containers
    on virtual and physical machines
    • Platform for automating deployment,
    scaling, and operations
    • Inspired and informed by Google’s
    experiences and internal systems
    • 100% Open source, written in Go

    View Slide

  49. @saturnism @googlecloud @kubernetesio @jfokus
    web browsers
    Scheduler
    kubectl web browsers
    scheduler
    Kubelet Kubelet Kubelet Kubelet
    Config
    file
    Kubernetes Master
    Container
    Image
    Developer View
    What just
    happened?

    View Slide

  50. @saturnism @googlecloud @kubernetesio @jfokus
    Developer View
    spec:
    containers:
    - name: myservice
    image: myservice
    resources:
    limits:
    memory: "128Mi"
    cpu: "0.1"
    ports:
    - containerPort: 3306
    protocol: TCP
    replicas: 1
    10000

    View Slide

  51. @saturnism @googlecloud @kubernetesio @jfokus
    Cluster of Machines as One

    View Slide

  52. @saturnism @googlecloud @kubernetesio @jfokus
    Live Demo

    View Slide

  53. @saturnism @googlecloud #kubernetes #devoxx

    View Slide

  54. @saturnism @googlecloud @kubernetesio @jfokus

    View Slide

  55. @saturnism @googlecloud @kubernetesio @jfokus
    Hello World
    Service - Greet
    Guestbook
    Service - Create
    Guestbook Service -
    Retrieve

    View Slide

  56. @saturnism @googlecloud @kubernetesio @jfokus
    Guestbook
    UI
    Hello World
    Service
    Redis
    session replication
    greeting
    MySQL
    Guestbook
    Service
    CRUD

    View Slide

  57. @saturnism @googlecloud @kubernetesio @jfokus
    Staging vs. production
    Use Namespaces - deploy in the same infrastructure

    View Slide

  58. @saturnism @googlecloud @kubernetesio @jfokus
    Canary
    Use service, and replication controllers to canary new versions

    View Slide

  59. @saturnism @googlecloud @kubernetesio @jfokus
    Rollback
    Super simple with versioned containers

    View Slide

  60. @saturnism @googlecloud @kubernetesio @jfokus
    Persistent Volumes

    View Slide

  61. @saturnism @googlecloud @kubernetesio @jfokus
    Rolling upgrade
    Similar to canary, but slowly let the new version take over

    View Slide

  62. @saturnism @googlecloud @kubernetesio @jfokus
    Health Checks

    View Slide

  63. @saturnism @googlecloud @kubernetesio @jfokus
    Configuration Map

    View Slide

  64. @saturnism @googlecloud @kubernetesio @jfokus
    Secrets

    View Slide

  65. @saturnism @googlecloud @kubernetesio @jfokus
    Namespace

    View Slide

  66. @saturnism @googlecloud @kubernetesio @jfokus
    Service discovery
    Read service IP addresses via environmental variables

    View Slide

  67. @saturnism @googlecloud @kubernetesio @jfokus
    Service discovery
    Kubernetes API
    or…
    DNS Lookups!
    ping redis

    View Slide

  68. @saturnism @googlecloud @kubernetesio @jfokus
    Before Stateful Set - Stateless Cattles
    Deployment
    Replicas → 1 Pod
    frontend
    Pod - cb-axk3u
    type = Couchbase
    version = 1.0
    Pod
    CB-Disk
    Volume
    Mount

    View Slide

  69. @saturnism @googlecloud @kubernetesio @jfokus
    Before Pet Set - Stateless Cattles
    Deployment
    Replicas → 2 Pod
    frontend
    Pod - cb-axk3u
    type = Couchbase
    version = 1.0
    Pod
    CB-Disk
    Pod
    frontend
    Pod - cb-a94kd
    type = Couchbase
    version = 1.0
    Volume
    Mount

    View Slide

  70. @saturnism @googlecloud @kubernetesio @jfokus
    Stateful Set
    Stateful Set
    Replicas → 1 Pod
    frontend
    Pod - cb-0
    type = Couchbase
    version = 1.0
    cb-0
    Volume
    Mount
    Automatic
    Provisioning

    View Slide

  71. @saturnism @googlecloud @kubernetesio @jfokus
    Stateful Set
    Stateful Set
    Replicas → 2 Pod
    frontend
    Pod - cb-0
    type = Couchbase
    version = 1.0
    Pod Pod
    frontend
    Pod - cb-1
    type = Couchbase
    version = 1.0
    cb-0
    Volume
    Mount
    cb-1
    Volume
    Mount

    View Slide

  72. @saturnism @googlecloud @kubernetesio @jfokus
    Wait a second, how about the disks?
    volumeClaimTemplates:
    - metadata:
    name: www
    annotations:
    volume.alpha.kubernetes.io/storage-class: anything
    spec:
    accessModes: [ "ReadWriteOnce" ]
    resources:
    requests:
    storage: 1Gi

    View Slide

  73. @saturnism @googlecloud @kubernetesio @jfokus
    Stateful Set
    Stateful Set
    Replicas → 2 Pod
    frontend
    Pod - cb-0
    type = Couchbase
    version = 1.0
    Pod Pod
    frontend
    Pod - cb-1
    type = Couchbase
    version = 1.0
    cb-0
    Volume
    Mount
    Automatic
    Provisioning
    cb-1
    Volume
    Mount

    View Slide

  74. @saturnism @googlecloud @kubernetesio @jfokus
    Next Step? Data Centers as One!

    View Slide

  75. @saturnism @googlecloud @kubernetesio @jfokus
    UI
    CLI
    API
    Control Plane Servers
    Admin
    containers
    containers
    containers
    containers
    containers
    containers
    containers
    containers
    containers
    containers
    containers
    containers
    containers
    containers
    containers
    Cluster / Data Center / Availability Zone

    View Slide

  76. @saturnism @googlecloud @kubernetesio @jfokus
    UI
    CLI
    API
    API
    Kubernetes on
    Kubernetes on
    Kubernetes on
    Premise
    Federation

    View Slide

  77. @saturnism @googlecloud @kubernetesio @jfokus
    Higher Availability
    Easy Application Migration
    Avoid Vendor Lock-in
    Capacity Overflow
    Cross-cluster
    Load Balancer
    Your
    users
    Cluster 1
    Cluster 2
    Cluster 3
    Use Cases

    View Slide

  78. @saturnism @googlecloud @kubernetesio @jfokus
    Provider 1
    Zone A
    Zone B
    Provider 2
    Zone C
    Provider 1
    Zone D
    Challenges

    View Slide

  79. @saturnism @googlecloud @kubernetesio @jfokus
    Geographically aware DNS gets clients to the
    "closest" healthy cluster.
    Standard Kubernetes service load balancing
    within each cluster.
    Can be extended to divert traffic away from
    "healthy-but-saturated" clusters.
    Cross-cluster Load Balancing

    View Slide

  80. @saturnism @googlecloud @kubernetesio @jfokus
    Location affinity
    Strictly coupled pods/applications
    • High bandwidth requirements
    • Low latency requirements
    • High fidelity requirements
    • Cannot easily span clusters
    Loosely coupled
    • Opposite of above
    • Relatively easily distributed across
    clusters
    Preferentially coupled
    • Strongly coupled but can be
    migrated piecemeal.

    View Slide

  81. @saturnism @googlecloud @kubernetesio @jfokus
    Location affinity continued...
    Negative Affinity
    • Don't run my replicas in the same
    failure domain (host/rack/zone)
    Topology
    • Same host
    • Same rack
    • Same zone
    • Same metro region
    • Same sub-continent
    Absolute affinity

    View Slide

  82. @saturnism @googlecloud @kubernetesio @jfokus
    What can we federate?
    As of Kubernetes 1.5
    Namespace
    Deployment
    Replica Set
    Secret
    ConfigMap
    Service
    Ingress

    View Slide

  83. @saturnism @googlecloud @kubernetesio @jfokus
    Federation Control Plane
    Cluster 2
    us-central1-b
    Cluster 1
    us-east1-b
    Cluster 3
    europe-west1-b
    Cluster 4
    asia-east1-b
    API API API API
    Cluster 2 - us-central1-b
    Federation API Server
    Federation Controller
    Federation Key/value store (etcd)
    Federation
    API
    contexts:
    - context:
    cluster: federation-cluster
    user: federation-cluster

    View Slide

  84. @saturnism @googlecloud @kubernetesio @jfokus
    Adding a Cluster
    Kubernetes Cluster
    (Asia)
    Federation Control Plane
    kubectl --context=federation-cluster create -f clusters/gcp-asia-east1.yaml
    apiVersion: federation/v1beta1
    kind: Cluster
    metadata:
    name: gce-asia-east1
    spec:
    serverAddressByClientCIDRs:
    - clientCIDR: "0.0.0.0/0"
    serverAddress: "https://..."
    secretRef:
    name: gce-asia-east1
    API
    kubeconfig

    View Slide

  85. @saturnism @googlecloud @kubernetesio @jfokus
    Cluster Federation
    Kubernetes Cluster 1
    (Google Cloud)
    Kubernetes Cluster 2
    (On-Prem)
    Kubernetes Cluster 3
    (Another Cloud)
    Federation Control Plane
    kubectl

    View Slide

  86. @saturnism @googlecloud @kubernetesio @jfokus
    Cluster Federation
    Kubernetes Cluster 1
    (Google Cloud)
    Kubernetes Cluster 2
    (On-Prem)
    Kubernetes Cluster 3
    (Another Cloud)
    Federation Control Plane
    kubectl create -f nginx-service.yaml
    nginx
    Service
    nginx
    Service
    nginx
    Service

    View Slide

  87. @saturnism @googlecloud @kubernetesio @jfokus
    Cluster Federation
    Kubernetes Cluster 1
    (Google Cloud)
    Kubernetes Cluster 2
    (On-Prem)
    Kubernetes Cluster 3
    (Another Cloud)
    Federation Control Plane
    kubectl create -f nginx-service.yaml
    nginx
    Service
    nginx
    Service
    nginx
    Service
    DNS

    View Slide

  88. @saturnism @googlecloud @kubernetesio @jfokus
    Kubernetes Cluster 1
    (Google Cloud)
    Kubernetes Cluster 2
    (On-Prem)
    Kubernetes Cluster 3
    (Another Cloud)
    svc app
    Clusters 1 and 3
    DNS
    nslookup app.default.federation.svc.federation.com
    svc app

    View Slide

  89. @saturnism @googlecloud @kubernetesio @jfokus
    Ecosystem

    View Slide

  90. @saturnism @googlecloud @kubernetesio @jfokus
    minikube

    View Slide

  91. @saturnism @googlecloud @kubernetesio @jfokus
    kompose

    View Slide

  92. @saturnism @googlecloud @kubernetesio @jfokus
    kubeadm

    View Slide

  93. @saturnism @googlecloud @kubernetesio @jfokus
    kops

    View Slide

  94. @saturnism @googlecloud @kubernetesio @jfokus
    helm

    View Slide

  95. @saturnism @googlecloud @kubernetesio @jfokus
    kubefed

    View Slide

  96. @saturnism @googlecloud @kubernetesio @jfokus
    Resources
    Spring Boot: https://github.com/saturnism/spring-boot-docker
    Federation: https://github.com/saturnism/buttonmasher
    Visualizer: https://github.com/saturnism/gcp-live-k8s-visualizer
    Hands-on Lab: http://bit.ly/k8s-lab
    Raspberry Pi Cluster: http://bit.ly/k8s-rpi
    Google Container Engine: https://cloud.google.com/container-engine/

    View Slide

  97. @saturnism @googlecloud @kubernetesio @jfokus
    Thanks!
    Images by Connie
    Zhou
    http://kubernetes.io
    http://bit.ly/1QLg5E1

    View Slide