Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Nomad Overview and 0.6 New Features

Nomad Overview and 0.6 New Features

We provide an overview of HashiCorp Nomad and the new features coming in 0.6, which is a cluster manager and scheduler. We explore what a cluster scheduler is and the benefits they provide, including higher resource utilization, decoupling developers and operators, and better quality of service. The high level design of Nomad is explained and how it supports running over a million containers.

The new 0.6 release adds support for first class deployments. This includes rolling deploys, canaries, promotions, blue/green, and auto-revert. Dynamic config values from Consul and Vault can be used as environment variables, and much more.

The talk and demo are also recorded as a webinar with Caius from Citadel discussing their use case.

Armon Dadgar

July 12, 2017
Tweet

More Decks by Armon Dadgar

Other Decks in Technology

Transcript

  1. Nomad Overview +
    0.6 New Features

    View full-size slide

  2. Armon Dadgar
    Founder and CTO
    @armon

    View full-size slide

  3. Copyright © 2017 HashiCorp
    CONNECT
    Infrastructure & applications
    RUN
    SECURE
    PROVISION
    Applications
    Infrastructure & applications
    Infrastructure
    OSS TOOL SUITE PRODUCT SUITE
    Consul
    Nomad
    Terraform
    Vault
    Consul
    Enterprise
    Nomad
    Enterprise
    Vault Enterprise
    Terraform
    Enterprise
    FOR TEAMS
    FOR INDIVIDUALS
    Provision, secure, connect, and run any infrastructure
    3
    Copyright © 2017 HashiCorp

    View full-size slide

  4. Copyright © 2017 HashiCorp
    ▪Nomad Overview
    ▪New Features of 0.6
    ▪Demo!
    4
    Agenda

    View full-size slide

  5. Nomad
    Cluster Manager
    Scheduler

    View full-size slide

  6. Nomad
    Cluster Manager
    Scheduler

    View full-size slide

  7. Schedulers map a set of
    work to a set of resources

    View full-size slide

  8. CPU Scheduler
    Web Server -Thread 1
    CPU - Core 1
    CPU - Core 2
    Web Server -Thread 2
    Redis -Thread 1
    Kernel -Thread 1
    Work (Input) Resources
    CPU
    Scheduler

    View full-size slide

  9. CPU Scheduler
    Web Server -Thread 1
    CPU - Core 1
    CPU - Core 2
    Web Server -Thread 2
    Redis -Thread 1
    Kernel -Thread 1
    Work (Input) Resources
    CPU
    Scheduler

    View full-size slide

  10. Schedulers in the Wild
    Type Work Resources
    CPU Scheduler Threads Physical Cores
    AWS EC2 /
    OpenStack Nova
    Virtual Machines Hypervisors
    Hadoop YARN MapReduce Jobs Client Nodes
    Cluster Scheduler Applications Servers

    View full-size slide

  11. User Nomad
    Servers
    Nomad
    Clients

    View full-size slide

  12. User Nomad
    Servers
    Submits
    Job
    Nomad
    Clients

    View full-size slide

  13. User Nomad
    Servers
    Submits
    Job
    Nomad
    Clients
    Deploy App

    View full-size slide

  14. User Nomad
    Servers
    Submits
    Job
    Nomad
    Clients
    Deploy App
    Skip (Busy)

    View full-size slide

  15. User Nomad
    Servers
    Submits
    Job
    Nomad
    Clients
    Deploy App
    Skip (Busy)
    Deploy App

    View full-size slide

  16. Advantages
    Higher Resource U.liza.on
    Decouple Work from Resources
    Be:er Quality of Service

    View full-size slide

  17. Advantages
    Higher Resource U.liza.on
    Decouple Work from Resources
    Be:er Quality of Service
    Bin Packing
    Over-Subscrip.on
    Job Queueing

    View full-size slide

  18. Advantages
    Higher Resource U.liza.on
    Decouple Work from Resources
    Be:er Quality of Service
    Abstrac.on
    API Contracts
    Packaging

    View full-size slide

  19. Advantages
    Higher Resource U.liza.on
    Decouple Work from Resources
    Be:er Quality of Service
    Priori.es
    Resource Isola.on
    Pre-emp.on

    View full-size slide


  20. Nomad
    @armon

    View full-size slide

  21. Nomad
    Cluster Scheduler
    Easily Deploy Applications
    Operationally Simple
    Built for Scale

    View full-size slide

  22. job "redis" {
    datacenters = ["us-east-1"]
    task "redis" {
    driver = "docker"
    config { image = "redis:latest" }
    resources {
    cpu = 500 # Mhz
    memory = 256 # MB
    network {
    mbits = 10
    port “redis" {}
    }
    }
    }
    }
    example.nomad

    View full-size slide

  23. Declares what to run

    View full-size slide

  24. Nomad determines where
    and manages how to run

    View full-size slide

  25. Nomad abstracts
    work from resources

    View full-size slide

  26. OS Workloads Drivers
    Windows Long Running Service Docker / Rkt / LXC
    Linux Short Lived Batch Qemu / KVM
    BSD Periodic Cron
    “exec”
    cgroups+chroot
    Solaris System Agents
    Static Binaries /
    Fat JARs

    View full-size slide

  27. Nomad
    Declarative Jobs
    Infrastructure as Code
    Consul Integration
    Vault Integration
    Composable vs Platform

    View full-size slide

  28. Empowers developers by
    de-coupling operators

    View full-size slide

  29. Operationally Simple
    &
    Scalable

    View full-size slide

  30. Nomad
    Single Binary
    No Dependencies
    Highly Available
    Multi-DC/Region Support

    View full-size slide

  31. Built on Research & Experience
    GOSSIP CONSENSUS
    Mature Libraries Proven Design Patterns

    View full-size slide

  32. Nomad
    Inspired by Google Omega
    Optimistic Concurrency
    Service & Batch workloads
    Pluggable Architecture

    View full-size slide

  33. Single Region Architecture
    SERVER SERVER SERVER
    CLIENT CLIENT CLIENT
    DC1 DC2 DC3
    FOLLOWER LEADER FOLLOWER
    REPLICATION
    FORWARDING
    REPLICATION
    FORWARDING
    RPC RPC RPC

    View full-size slide

  34. Multi Region Architecture
    SERVER SERVER SERVER
    FOLLOWER LEADER FOLLOWER
    REPLICATION
    FORWARDING
    REPLICATION
    REGION B
     GOSSIP
    REPLICATION REPLICATION
    FORWARDING
    REGION FORWARDING
     REGION A
    SERVER
    FOLLOWER
    SERVER SERVER
    LEADER FOLLOWER

    View full-size slide

  35. 100’s of Regions
    10,000’s of Clients per Region
    1000’s of Jobs per Region

    View full-size slide

  36. Nomad
    Million Container
    Challenge
    1,000 Jobs
    1,000 Tasks per Job
    5,000 Hosts on GCE
    1,000,000 Containers

    View full-size slide

  37. “640 KB ought to be enough for anybody.”
    - Bill Gates

    View full-size slide

  38. 2nd Largest Hedge Fund
    18K Cores
    5 Hours
    2,200 Containers/second

    View full-size slide

  39. Nomad
    Cluster Scheduler
    Easily Deploy Applications
    Operationally Simple
    Built for Scale

    View full-size slide

  40. Copyright © 2017 HashiCorp
    ▪Service Updates
    ▪Rolling Deploys with Health Checks
    ▪Canaries
    ▪Blue/Green
    ▪Dynamic Environment Variables
    ▪Multiple Job Versions & Rollbacks
    ▪Docker Networking Plugins
    ▪Much More (see CHANGELOG)
    42
    New Features in 0.6

    View full-size slide

  41. Service Updates
    • Changing a Service Job from Version A to Version B
    • Upgrading WebApp from 1.1.0 to 1.2.0
    • Controlled by update block
    • Supports many different approaches
    • Rolling Deploys, Canaries, Blue/Green

    View full-size slide

  42. Copyright © 2017 HashiCorp 44
    Rolling Deploys
    job "webapp" {
    update {
    max_parallel = 2
    }
    group "web" {
    count = 6
    task "app" {
    driver = "docker"
    config {
    image = "webapp:1.0"
    }
    }
    }
    }
    webapp.nomad

    View full-size slide

  43. webapp 1.0
    1.0 1.0 1.0 1.0 1.0
    1.0

    View full-size slide

  44. 1.0 1.0 1.0 1.0
    webapp 2.0
    2.0 2.0

    View full-size slide

  45. 1.0 1.0
    webapp 2.0
    2.0 2.0 2.0 2.0

    View full-size slide

  46. webapp 2.0
    2.0 2.0 2.0 2.0 2.0 2.0

    View full-size slide

  47. Copyright © 2017 HashiCorp 50
    Canaries
    job "webapp" {
    update {
    max_parallel = 2
    canary = 1
    }
    group "web" {
    count = 6
    task "app" {
    driver = "docker"
    config {
    image = "webapp:1.0"
    }
    }
    }
    }
    webapp.nomad

    View full-size slide

  48. webapp 1.0
    1.0 1.0 1.0 1.0 1.0
    1.0

    View full-size slide

  49. 1.0 1.0 1.0 1.0
    webapp 2.0
    2.0
    1.0 1.0
    Canary

    View full-size slide

  50. 1.0 1.0 1.0 1.0
    Promote
    webapp 2.0 2.0
    1.0

    View full-size slide

  51. 1.0 1.0 1.0
    Promote
    webapp 2.0 2.0 2.0 2.0

    View full-size slide

  52. 1.0
    Promote
    webapp 2.0 2.0 2.0 2.0 2.0 2.0

    View full-size slide

  53. Promote
    webapp 2.0 2.0 2.0 2.0 2.0 2.0 2.0

    View full-size slide

  54. Copyright © 2017 HashiCorp 57
    Blue/Green
    job "webapp" {
    update {
    max_parallel = 2
    canary = 6 # Match count of group
    }
    group "web" {
    count = 6
    task "app" {
    driver = "docker"
    config {
    image = "webapp:1.0"
    }
    }
    }
    }
    webapp.nomad

    View full-size slide

  55. webapp 1.0
    1.0 1.0 1.0 1.0 1.0
    1.0

    View full-size slide

  56. 1.0 1.0 1.0 1.0
    webapp 2.0
    2.0
    1.0 1.0
    Canary
    2.0 2.0 2.0 2.0 2.0
    Canary Canary Canary Canary Canary

    View full-size slide

  57. 2.0 2.0 2.0 2.0 2.0 2.0
    Promote
    webapp 2.0

    View full-size slide

  58. Multiple Job Versions & Rollbacks
    • Track Multiple Versions per Job
    • Rollbacks
    • “Auto Revert” failed deploys

    View full-size slide

  59. Copyright © 2017 HashiCorp 62
    Job Versions
    $ nomad job history example
    Version = 1
    Stable = true
    Submit Date = 07/08/17 23:29:31 UTC
    Version = 0
    Stable = true
    Submit Date = 07/08/17 23:27:54 UTC
    $ nomad job revert example 0
    ==> Monitoring evaluation "4e0662c7"

    ==> Evaluation "4e0662c7" finished with status “complete"
    $ nomad job history -p example
    Version = 2
    Stable = true
    Submit Date = 07/08/17 23:31:07 UTC
    Diff =
    +/- Job: "example"
    +/- Task Group: "cache"
    +/- Task: "redis"
    +/- Config {
    +/- image: "redis:3.0" => "redis:3.2"
    port_map[0][db]: "6379"
    }
    Terminal

    View full-size slide

  60. Copyright © 2017 HashiCorp 63
    Job Status with
    Versions
    $ nomad status example
    ID = example
    Name = example
    Submit Date = 07/08/17 23:31:07 UTC
    Type = service
    Priority = 50
    Datacenters = dc1
    Status = running
    Periodic = false
    Parameterized = false
    Summary
    Task Group Queued Starting Running Failed Complete Lost
    cache 0 0 1 0 2 0
    Latest Deployment
    ID = 07e88357
    Status = successful
    Description = Deployment completed successfully
    Deployed
    Task Group Desired Placed Healthy Unhealthy
    cache 1 1 1 0
    Allocations
    ID Node ID Task Group Version Desired Status Created At
    71c6f915 49a104fe cache 2 run running 07/08/17 23:31:07 UTC
    9b69e806 49a104fe cache 1 stop complete 07/08/17 23:29:31 UTC
    2e5550f0 49a104fe cache 0 stop complete 07/08/17 23:27:54 UTC
    Terminal

    View full-size slide

  61. Dynamic Environment Variables
    • The template block renders dynamic configuration
    • Integrates with Consul for app configuration
    • Integrates with Vault for secrets (DB credentials, TLS, etc)
    • New env parameter sets environment variables

    View full-size slide

  62. Copyright © 2017 HashiCorp 65
    Dynamic
    Environment
    Variables
    job "webapp" {
    task "app" {
    driver = "docker"
    config {
    image = "webapp:1.0"
    }
    template {
    data = <# Feature flag from Consul
    FEATURE_FLAG_FOO = {{ key "service/webapp/foo" }}
    # AWS credentials from Vault
    {{ with secret "service/webapp/aws_key" }}
    AWS_ACCESS_KEY = {{ .Data.access_key }}
    AWS_SECRET_KEY = {{ .Data.secret_key }}
    {{ end }}
    EOH
    destination = "local/file.env"
    env = true
    }
    webapp.nomad

    View full-size slide


  63. Thanks!
    @armon

    View full-size slide