Nomad Overview and 0.6 New Features

Nomad Overview and 0.6 New Features

We provide an overview of HashiCorp Nomad and the new features coming in 0.6, which is a cluster manager and scheduler. We explore what a cluster scheduler is and the benefits they provide, including higher resource utilization, decoupling developers and operators, and better quality of service. The high level design of Nomad is explained and how it supports running over a million containers.

The new 0.6 release adds support for first class deployments. This includes rolling deploys, canaries, promotions, blue/green, and auto-revert. Dynamic config values from Consul and Vault can be used as environment variables, and much more.

The talk and demo are also recorded as a webinar with Caius from Citadel discussing their use case.

11ba9630c9136eef9a70d26473d355d5?s=128

Armon Dadgar

July 12, 2017
Tweet

Transcript

  1. Nomad Overview + 0.6 New Features

  2. Armon Dadgar Founder and CTO @armon

  3. Copyright © 2017 HashiCorp CONNECT Infrastructure & applications RUN SECURE

    PROVISION Applications Infrastructure & applications Infrastructure OSS TOOL SUITE PRODUCT SUITE Consul Nomad Terraform Vault Consul Enterprise Nomad Enterprise Vault Enterprise Terraform Enterprise FOR TEAMS FOR INDIVIDUALS Provision, secure, connect, and run any infrastructure 3 Copyright © 2017 HashiCorp
  4. Copyright © 2017 HashiCorp ▪Nomad Overview ▪New Features of 0.6

    ▪Demo! 4 Agenda
  5. Nomad Cluster Manager Scheduler

  6. Nomad Cluster Manager Scheduler

  7. Schedulers map a set of work to a set of

    resources
  8. CPU Scheduler Web Server -Thread 1 CPU - Core 1

    CPU - Core 2 Web Server -Thread 2 Redis -Thread 1 Kernel -Thread 1 Work (Input) Resources CPU Scheduler
  9. CPU Scheduler Web Server -Thread 1 CPU - Core 1

    CPU - Core 2 Web Server -Thread 2 Redis -Thread 1 Kernel -Thread 1 Work (Input) Resources CPU Scheduler
  10. Schedulers in the Wild Type Work Resources CPU Scheduler Threads

    Physical Cores AWS EC2 / OpenStack Nova Virtual Machines Hypervisors Hadoop YARN MapReduce Jobs Client Nodes Cluster Scheduler Applications Servers
  11. User Nomad Servers Nomad Clients

  12. User Nomad Servers Submits Job Nomad Clients

  13. User Nomad Servers Submits Job Nomad Clients Deploy App

  14. User Nomad Servers Submits Job Nomad Clients Deploy App Skip

    (Busy)
  15. User Nomad Servers Submits Job Nomad Clients Deploy App Skip

    (Busy) Deploy App
  16. Advantages Higher Resource U.liza.on Decouple Work from Resources Be:er Quality

    of Service
  17. Advantages Higher Resource U.liza.on Decouple Work from Resources Be:er Quality

    of Service Bin Packing Over-Subscrip.on Job Queueing
  18. Advantages Higher Resource U.liza.on Decouple Work from Resources Be:er Quality

    of Service Abstrac.on API Contracts Packaging
  19. Advantages Higher Resource U.liza.on Decouple Work from Resources Be:er Quality

    of Service Priori.es Resource Isola.on Pre-emp.on
  20. None
  21.  Nomad @armon

  22. Nomad Cluster Scheduler Easily Deploy Applications Operationally Simple Built for

    Scale
  23. job "redis" { datacenters = ["us-east-1"] task "redis" { driver

    = "docker" config { image = "redis:latest" } resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10 port “redis" {} } } } } example.nomad
  24. Declares what to run

  25. Nomad determines where and manages how to run

  26. Nomad abstracts work from resources

  27. OS Workloads Drivers Windows Long Running Service Docker / Rkt

    / LXC Linux Short Lived Batch Qemu / KVM BSD Periodic Cron “exec” cgroups+chroot Solaris System Agents Static Binaries / Fat JARs
  28. Nomad Declarative Jobs Infrastructure as Code Consul Integration Vault Integration

    Composable vs Platform
  29. Empowers developers by de-coupling operators

  30. Operationally Simple & Scalable

  31. Nomad Single Binary No Dependencies Highly Available Multi-DC/Region Support

  32. Built on Research & Experience GOSSIP CONSENSUS Mature Libraries Proven

    Design Patterns
  33. Nomad Inspired by Google Omega Optimistic Concurrency Service & Batch

    workloads Pluggable Architecture
  34. Single Region Architecture SERVER SERVER SERVER CLIENT CLIENT CLIENT DC1

    DC2 DC3 FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION FORWARDING RPC RPC RPC
  35. Multi Region Architecture SERVER SERVER SERVER FOLLOWER LEADER FOLLOWER REPLICATION

    FORWARDING REPLICATION REGION B  GOSSIP REPLICATION REPLICATION FORWARDING REGION FORWARDING  REGION A SERVER FOLLOWER SERVER SERVER LEADER FOLLOWER
  36. 100’s of Regions 10,000’s of Clients per Region 1000’s of

    Jobs per Region
  37. Nomad Million Container Challenge 1,000 Jobs 1,000 Tasks per Job

    5,000 Hosts on GCE 1,000,000 Containers
  38. None
  39. “640 KB ought to be enough for anybody.” - Bill

    Gates
  40. 2nd Largest Hedge Fund 18K Cores 5 Hours 2,200 Containers/second

  41. Nomad Cluster Scheduler Easily Deploy Applications Operationally Simple Built for

    Scale
  42. Copyright © 2017 HashiCorp ▪Service Updates ▪Rolling Deploys with Health

    Checks ▪Canaries ▪Blue/Green ▪Dynamic Environment Variables ▪Multiple Job Versions & Rollbacks ▪Docker Networking Plugins ▪Much More (see CHANGELOG) 42 New Features in 0.6
  43. Service Updates • Changing a Service Job from Version A

    to Version B • Upgrading WebApp from 1.1.0 to 1.2.0 • Controlled by update block • Supports many different approaches • Rolling Deploys, Canaries, Blue/Green
  44. Copyright © 2017 HashiCorp 44 Rolling Deploys job "webapp" {

    update { max_parallel = 2 } group "web" { count = 6 task "app" { driver = "docker" config { image = "webapp:1.0" } } } } webapp.nomad
  45. webapp 1.0

  46. webapp 1.0 1.0 1.0 1.0 1.0 1.0 1.0

  47. 1.0 1.0 1.0 1.0 webapp 2.0 2.0 2.0

  48. 1.0 1.0 webapp 2.0 2.0 2.0 2.0 2.0

  49. webapp 2.0 2.0 2.0 2.0 2.0 2.0 2.0

  50. Copyright © 2017 HashiCorp 50 Canaries job "webapp" { update

    { max_parallel = 2 canary = 1 } group "web" { count = 6 task "app" { driver = "docker" config { image = "webapp:1.0" } } } } webapp.nomad
  51. webapp 1.0 1.0 1.0 1.0 1.0 1.0 1.0

  52. 1.0 1.0 1.0 1.0 webapp 2.0 2.0 1.0 1.0 Canary

  53. 1.0 1.0 1.0 1.0 Promote webapp 2.0 2.0 1.0

  54. 1.0 1.0 1.0 Promote webapp 2.0 2.0 2.0 2.0

  55. 1.0 Promote webapp 2.0 2.0 2.0 2.0 2.0 2.0

  56. Promote webapp 2.0 2.0 2.0 2.0 2.0 2.0 2.0

  57. Copyright © 2017 HashiCorp 57 Blue/Green job "webapp" { update

    { max_parallel = 2 canary = 6 # Match count of group } group "web" { count = 6 task "app" { driver = "docker" config { image = "webapp:1.0" } } } } webapp.nomad
  58. webapp 1.0 1.0 1.0 1.0 1.0 1.0 1.0

  59. 1.0 1.0 1.0 1.0 webapp 2.0 2.0 1.0 1.0 Canary

    2.0 2.0 2.0 2.0 2.0 Canary Canary Canary Canary Canary
  60. 2.0 2.0 2.0 2.0 2.0 2.0 Promote webapp 2.0

  61. Multiple Job Versions & Rollbacks • Track Multiple Versions per

    Job • Rollbacks • “Auto Revert” failed deploys
  62. Copyright © 2017 HashiCorp 62 Job Versions $ nomad job

    history example Version = 1 Stable = true Submit Date = 07/08/17 23:29:31 UTC Version = 0 Stable = true Submit Date = 07/08/17 23:27:54 UTC $ nomad job revert example 0 ==> Monitoring evaluation "4e0662c7" … ==> Evaluation "4e0662c7" finished with status “complete" $ nomad job history -p example Version = 2 Stable = true Submit Date = 07/08/17 23:31:07 UTC Diff = +/- Job: "example" +/- Task Group: "cache" +/- Task: "redis" +/- Config { +/- image: "redis:3.0" => "redis:3.2" port_map[0][db]: "6379" } Terminal
  63. Copyright © 2017 HashiCorp 63 Job Status with Versions $

    nomad status example ID = example Name = example Submit Date = 07/08/17 23:31:07 UTC Type = service Priority = 50 Datacenters = dc1 Status = running Periodic = false Parameterized = false Summary Task Group Queued Starting Running Failed Complete Lost cache 0 0 1 0 2 0 Latest Deployment ID = 07e88357 Status = successful Description = Deployment completed successfully Deployed Task Group Desired Placed Healthy Unhealthy cache 1 1 1 0 Allocations ID Node ID Task Group Version Desired Status Created At 71c6f915 49a104fe cache 2 run running 07/08/17 23:31:07 UTC 9b69e806 49a104fe cache 1 stop complete 07/08/17 23:29:31 UTC 2e5550f0 49a104fe cache 0 stop complete 07/08/17 23:27:54 UTC Terminal
  64. Dynamic Environment Variables • The template block renders dynamic configuration

    • Integrates with Consul for app configuration • Integrates with Vault for secrets (DB credentials, TLS, etc) • New env parameter sets environment variables
  65. Copyright © 2017 HashiCorp 65 Dynamic Environment Variables job "webapp"

    { task "app" { driver = "docker" config { image = "webapp:1.0" } template { data = <<EOH # Feature flag from Consul FEATURE_FLAG_FOO = {{ key "service/webapp/foo" }} # AWS credentials from Vault {{ with secret "service/webapp/aws_key" }} AWS_ACCESS_KEY = {{ .Data.access_key }} AWS_SECRET_KEY = {{ .Data.secret_key }} {{ end }} EOH destination = "local/file.env" env = true } webapp.nomad
  66. Demo!

  67.  Thanks! @armon