Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Who are you and what did you do with my containers?

Who are you and what did you do with my containers?

Kubernetes talk at DevFest Dublin

GoogleCloudPlatform

November 07, 2015
Tweet

More Decks by GoogleCloudPlatform

Other Decks in Technology

Transcript

  1. Who are you and what have you done with
    my Containers?

    View Slide

  2. @tekgrrl #kubernetes #devfest
    Mandy Waite
    Developer Advocate
    +MandyWaite
    @tekgrrl

    View Slide

  3. Image by Connie
    Zhou

    View Slide

  4. @tekgrrl #kubernetes #devfest
    job hello_world = {
    runtime = { cell = 'ic' } // Cell (cluster) to run in
    binary = '.../hello_world_webserver' // Program to run
    args = { port = '%port%' } // Command line parameters
    requirements = { // Resource requirements
    ram = 100M
    disk = 100M
    cpu = 0.1
    }
    replicas = 5 // Number of tasks
    }
    10000
    Developer View

    View Slide

  5. @tekgrrl #kubernetes #devfest
    Developer View

    View Slide

  6. @tekgrrl #kubernetes #devfest
    web browsers
    BorgMaster
    link shard
    UI
    shard
    BorgMaster
    link shard
    UI
    shard
    BorgMaster
    link shard
    UI
    shard
    BorgMaster
    link shard
    UI
    shard
    Scheduler
    borgcfg web browsers
    scheduler
    Borglet Borglet Borglet Borglet
    Config
    file
    BorgMaster
    link shard
    UI
    shard
    persistent store
    (Paxos)
    Binary
    What just
    happened?
    Cell
    Storage
    Developer View

    View Slide

  7. Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world! Hello
    world! Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Image by Connie
    Zhou
    Hello
    world!
    Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world! Hello
    world!
    Hello
    world!
    Hello
    world!

    View Slide

  8. Google confidential │ Do not
    distribute
    Everything at Google runs in
    containers:
    • Gmail, Web Search, Maps, ...
    • MapReduce, batch, ...
    • GFS, Colossus, ...
    • Even Google Cloud Platform:
    VMs run in containers!

    View Slide

  9. Google confidential │ Do not
    distribute
    Everything at Google runs in
    containers:
    • Gmail, Web Search, Maps, ...
    • MapReduce, batch, ...
    • GFS, Colossus, ...
    • Even Google’s Cloud Platform:
    VMs run in containers!
    We launch over 2 billion
    containers per week

    View Slide

  10. Kubernetes

    View Slide

  11. @tekgrrl #kubernetes #devfest
    Kubernetes
    Greek for “Helmsman”; also the root of the
    words “governor” and “cybernetic”
    • Runs and manages containers
    • Inspired and informed by Google’s
    experiences and internal systems
    • Supports multiple cloud and bare-metal
    environments
    • Supports multiple container runtimes
    • 100% Open source, written in Go
    Manage applications, not machines

    View Slide

  12. @tekgrrl #kubernetes #devfest
    web browsers
    y
    Kubelet Kubelet Kubelet Kubelet
    Kubernetes Master
    Replication
    Controller
    Scheduler
    API Server
    Kube-UI
    Container
    Registry
    kubectl
    proxy

    web browsers
    Kubernetes Architecture

    View Slide

  13. @tekgrrl #kubernetes #devfest
    Setting up a cluster
    • Choose a platform: GCE, AWS, Azure, Rackspace, on-premises, ...
    • Choose a node OS: CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ...
    • Provision machines: Boot VMs, install and run kube components, ...
    • Configure networking: IP ranges for Pods, Services, SDN, ...
    • Start cluster services: DNS, logging, monitoring, ...
    • Manage nodes: kernel upgrades, OS updates, hardware failures...
    Not the easy or fun part, but unavoidable
    This is where things like Google Container Engine (GKE) really help

    View Slide

  14. @tekgrrl #kubernetes #devfest
    The atom of scheduling for containers
    Represents an application specific
    logical host
    Hosts containers and volumes
    Each has its own routable (no NAT) IP
    address
    Ephemeral
    • Pods are functionally identical and therefore
    ephemeral and replaceable
    Pod
    Web Server
    Volume
    Consumers
    Pods

    View Slide

  15. @tekgrrl #kubernetes #devfest
    Can be used to group multiple
    containers & shared volumes
    Containers within a pod are tightly
    coupled
    Shared namespaces
    • Containers in a pod share IP, port and IPC
    namespaces
    • Containers in a pod talk to each other
    through localhost
    Pods
    Pod
    Git
    Synchronizer
    Node.js App
    Container
    Volume
    Consumers
    git Repo

    View Slide

  16. @tekgrrl #kubernetes #devfest
    Developer View (Pods)
    spec:
    containers:
    - name: mysql
    image: mysql
    resources:
    limits:
    memory: "512Mi"
    cpu: "1000m"
    ports:
    - containerPort: 3306
    name: mysql
    volumeMounts:
    - name: mysql-persistent-storage
    mountPath: /var/lib/mysql
    volumes:
    - name: mysql-persistent-storage

    View Slide

  17. @tekgrrl @googlecloud #oscon
    Pod Networking (across nodes)
    Pods have IPs which are routable
    Pods can reach each other without NAT
    ● Even across nodes
    No Brokering of Port Numbers
    These are fundamental requirements
    Many solutions
    ● Flannel, Weave, OpenVSwitch,
    Cloud Provider
    10.1.2.0/24
    10.1.1.0/24
    10.1.1.211 10.1.1.2
    10.1.2.106
    10.1.3.0/24
    10.1.3.45
    10.1.3.17
    10.1.3.0/24

    View Slide

  18. @tekgrrl #kubernetes #devfest
    Dashboard
    show: type = FE
    Pod
    Pod
    frontend
    Pod
    frontend
    Pod Pod
    Dashboard
    show: version = v2
    type = FE
    version = v2
    type = FE version = v2
    ● Metadata with semantic meaning
    ● Membership identifier
    ● The only Grouping Mechanism
    Behavior Benefits
    ➔ Allow for intent of many users (e.g. dashboards)
    ➔ Build higher level systems …
    ➔ Queryable by Selectors
    Labels ← These are important

    View Slide

  19. @tekgrrl #kubernetes #devfest
    Developer View (pod with labels)
    metadata:
    name: frontend
    labels:
    type: frontend
    version: v2
    spec:
    containers:
    - name: php-guestbook
    image: php-guestbook:oscon-eu
    ...

    View Slide

  20. @tekgrrl #kubernetes #devfest
    Replication
    Controller
    Pod
    Pod
    frontend
    Pod
    frontend
    Pod Pod
    Replication
    Controller
    #pods = 1
    version = v2
    show: version = v2
    version= v1 version = v1 version = v2
    Replication
    Controller
    #pods = 2
    version = v1
    show: version = v2 Behavior Benefits
    ● Keeps Pods running
    ● Gives direct control of Pod #s
    ● Grouped by Label Selector
    ➔ Recreates Pods, maintains desired state
    ➔ Fine-grained control for scaling
    ➔ Standard grouping semantics
    Replication Controllers

    View Slide

  21. @tekgrrl #kubernetes #devfest
    Developer View (ReplicationController)
    replicas: 2
    selector:
    version: v1
    template:
    metadata:
    name: frontend
    labels:
    version: v1
    spec:
    containers:
    - name: php-guestbook
    image: php-guestbook:oscon-eu
    ...

    View Slide

  22. @tekgrrl #kubernetes #devfest
    Replication Controller
    Replication Controller
    - Name = “backend”
    - Selector = {“name”: “backend”}
    - Template = { ... }
    - NumReplicas = 4
    API Server
    3
    Start 1
    more
    OK 4
    How
    many?
    How
    many?
    Canonical example of control loops
    Have one job: ensure N copies of a pod
    ● if too few, start new ones
    ● if too many, kill some
    ● group == selector
    Replicated pods are fungible
    ● No implied order or identity
    Replication Controllers

    View Slide

  23. @tekgrrl #kubernetes #devfest
    Client
    Pod
    Container
    Pod
    Container
    Pod
    Container
    Container
    A logical grouping of pods that perform the
    same function
    • grouped by label selector
    Load balances incoming requests across
    constituent pods
    Choice of pod is random but supports
    session affinity (ClientIP)
    Gets a stable virtual IP and port
    • also a DNS nametype = FE

    Services
    Service
    Label selector:
    type = FE
    VIP
    type = FE type = FE type = FE

    View Slide

  24. @tekgrrl #kubernetes #devfest
    Developer View (Service)
    apiVersion: v1
    kind: Service
    metadata:
    name: frontend
    labels:
    name: frontend-svc
    spec:
    type: LoadBalancer
    ports:
    - port: 80
    targetPort: 80
    protocol: TCP
    selector:
    type: FE

    View Slide

  25. @tekgrrl #kubernetes #devfest
    Node3
    Kubelet Proxy
    Pod
    Container
    Container
    Container
    Container
    Pod
    Container
    Container
    Container
    Container
    Node3
    Kubelet Proxy
    Pod
    Container
    Container
    Container
    Container
    Pod
    Container
    Container
    Container
    Container
    Node1
    Kubelet Proxy
    Pod
    Container
    Container
    Pod
    $ kubectl proxy --www=k8s-visualizer/
    Visualizing Kubernetes
    Master
    APIs
    Scheduling
    REST
    (pods, services,
    controllers)
    AuthN
    Scheduler
    Replication
    Controller
    Container

    View Slide

  26. @tekgrrl #kubernetes #devfest
    Service
    Label selectors:
    version = 1.0
    type = Frontend
    Service
    name = frontend
    Label selector:
    type = BE
    Replication
    Controller Pod
    frontend
    Pod
    version= v1 version = v1
    Replication
    Controller
    version = v1
    #pods = 1
    show: version = v2
    type = FE type = FE
    Scaling Example
    Pod
    frontend
    Pod
    version = v1
    type = FE
    Replication
    Controller
    version = v1
    #pods = 2
    show: version = v2
    Pod Pod
    Replication
    Controller
    version = v1
    type = FE
    #pods = 4
    show: version = v2
    version = v1
    type = FE

    View Slide

  27. @tekgrrl #kubernetes #devfest
    Rolling Update Example
    Service
    Label selectors:
    version = 1.0
    type = Frontend
    Service
    name = backend
    Label selector:
    type = BE
    Replication
    Controller
    Pod
    Pod
    frontend
    Pod
    version= v1 version = v1
    Replication
    Controller
    version = v1
    type = BE
    #pods = 2
    show: version = v2
    type = BE type = BE
    Replication
    Controller
    version = v2
    type = BE
    #pods = 2
    show: version = v2
    Pod
    version = v2
    type = BE
    version = v2

    View Slide

  28. @tekgrrl #kubernetes #devfest
    Service
    Label selectors:
    version = 1.0
    type = Frontend
    Service
    name = backend
    Label selector:
    type = BE
    Replication
    Controller
    Pod
    Pod
    frontend
    Pod
    version= v1 version = v1
    Replication
    Controller
    version = v1
    type = BE
    #pods = 2
    show: version = v2
    type = BE type = BE
    Canary Example
    Replication
    Controller
    Replication
    Controller
    version = v2
    type = BE
    #pods = 1
    show: version = v2
    Pod
    frontend
    Pod
    version = v2
    type = BE

    View Slide

  29. Demo - Visualization

    View Slide

  30. @tekgrrl #kubernetes #devfest
    A quick guide to Cluster Nodes
    Cluster Node
    Kubelet Proxy
    disk = ssd
    Resources
    Labels
    Disks

    View Slide

  31. @tekgrrl #kubernetes #devfest
    What Resources does it need?
    What Disk(s) does it need?
    What node can it run on (NodeName)?
    What node(s) can it run on (Node
    Labels)?
    Finding Potential Nodes
    Cluster Node
    Kubelet Proxy
    disk = ssd

    View Slide

  32. @tekgrrl #kubernetes #devfest
    Prefer node with most free resource
    left after the pod is deployed
    Prefer nodes with the specified label
    Minimise number of Pods from the
    same service on the same node
    CPU and Memory is balanced after the
    Pod is deployed [Default]
    Ranking Potential Nodes
    Cluster Node
    Kubelet Proxy
    disk = ssd

    View Slide

  33. @tekgrrl #kubernetes #devfest
    Let’s explore that CPU and memory balancing

    View Slide

  34. @tekgrrl #kubernetes #devfest
    Machines (Virtual and Bare Metal) have shapes

    View Slide

  35. @tekgrrl #kubernetes #devfest
    Workloads have shapes too

    View Slide

  36. @tekgrrl #kubernetes #devfest
    In a container cluster the Machine becomes a Resource
    Boundary

    View Slide

  37. @tekgrrl #kubernetes #devfest
    Machine Shapes
    Workload Shapes

    View Slide

  38. @tekgrrl #kubernetes #devfest
    Computing Tetris
    5.5GB RAM
    Inaccessible
    CPU Fully
    Utilized
    5.5GB RAM
    Available
    1 CPU Core
    available
    Resource Stranding Efficient Bin-Packing
    Memory Fully
    Utilized
    CPU Fully
    Utilized

    View Slide

  39. @tekgrrl #kubernetes #devfest
    Efficient scheduling is key to container management

    View Slide

  40. @tekgrrl #kubernetes #devfest
    Open sourced in June, 2014
    v1.0 in July, 2015
    Google Container Engine (GKE)
    ● hosted Kubernetes - don’t think about cluster setup
    ● GA in August, 2015
    PaaSes:
    ● RedHat OpenShift, Deis, Stratos
    Distros:
    ● CoreOS Tectonic, Mirantis Murano (OpenStack),RedHat
    Atomic, Mesos
    Driving towards a 1.1 release
    Kubernetes status & plans

    View Slide

  41. @tekgrrl #kubernetes #devfest
    Google Container Engine (GA)
    Managed Kubernetes (Kubernetes v1)
    Manages Kubernetes master uptime
    Manages Updates
    Cluster Resize via Managed Instance
    Groups
    Centralised Logging
    Google Cloud VPN support

    View Slide

  42. @tekgrrl #kubernetes #devfest
    Kubernetes is Open Source
    We want your help!
    http://kubernetes.io
    https://github.com/GoogleCloudPlatform/kubernetes
    irc.freenode.net #google-containers
    @kubernetesio

    View Slide

  43. @tekgrrl #kubernetes #devfest
    Tweet questions to:
    @tekgrrl
    Slides: http://bit.ly/1i2PsgE
    Questions

    View Slide