Scheduling at Scale

Scheduling at Scale

Docker and rkt have made it really easy to package and ship applications but running them at scale, remains a challenge. Also, not all organizations have the bandwidth to containerize their workloads. Nomad, a single binary cluster scheduler, can be used to build a multi region, self-healing production environment that runs a diverse set of workloads including non-containerized applications. Nomad is a powerful and feature-rich scheduler; we will see how it allows native execution of a diverse set of applications without the requirement to package them in a Docker image. We will also look at how we can integrate Nomad and Vault to provide dynamically generated TLS certificates and secrets to our application, how Consul and Consul Template allows us to provide configuration management and feature flagging, and how operationally simple it is to perform zero downtime updates with Blue / Green and Canary deployments. This talk will discuss the theory, and also showcase a live demo of running an application on Nomad, highlighting how simple it is to leverage the power of Nomad, Consul, and Vault to orchestrate your applications.

26896287bc831a13e768cea7efe29632?s=128

Anubhav Mishra

June 05, 2018
Tweet

Transcript

  1. Scheduling at Scale Photo by Sašo Tušar on Unsplash @anubhavm

  2. @anubhavm Anubhav Mishra Developer Advocate, HashiCorp

  3. @anubhavm Anubhav Mishra Developer Advocate, HashiCorp has stickers

  4. PROVISION, SECURE AND RUN ANY INFRASTRUCTURE Nomad Consul Vault Vagrant

    Packer Terraform Consul Enterprise Terraform Enterprise Vault Enterprise PRODUCT SUITE OSS TOOL SUITE RUN Applications SECURE Application Infrastructure PROVISION Infrastructure FOR INDIVIDUALS FOR TEAMS Nomad Enterprise
  5. None
  6. Copyright © 2017 HashiCorp @anubhavm  !6 Globally Distributed Optimistically

    Concurrent Scheduler
  7. Copyright © 2017 HashiCorp @anubhavm  !7 Globally Distributed Optimistically

    Concurrent Scheduler
  8. Scheduling Photo by Emma Matthews on Unsplash

  9. Copyright © 2017 HashiCorp @anubhavm  !9 [1] Assigning an

    appropriate number of workers to the jobs during each day of work. Scheduling [1] Read more: http://www.businessdictionary.com/definition/scheduling.html
  10. Copyright © 2017 HashiCorp @anubhavm  !10 A person or

    machine that helps scheduling during each day of work. Scheduler
  11. Copyright © 2017 HashiCorp @anubhavm  !11

  12. Copyright © 2017 HashiCorp @anubhavm  !12 A computer program

    that controls or manages the execution of jobs / processes / operations. Scheduler (Computing)
  13. Copyright © 2017 HashiCorp @anubhavm  !13

  14. Copyright © 2017 HashiCorp @anubhavm  !14 Schedulers map a

    set of work to a set of resources
  15. Copyright © 2017 HashiCorp @anubhavm  Scheduling !15 Traditional DATACENTER

  16. Copyright © 2017 HashiCorp @anubhavm  Scheduling !16 Traditional DATACENTER

    OPERATIONS ENGINEER
  17. Copyright © 2017 HashiCorp @anubhavm  Scheduling !17 Traditional DATACENTER

    OPERATIONS ENGINEER Gandalf Gollum Frodo Sam
  18. Copyright © 2017 HashiCorp @anubhavm  Scheduling !18 Traditional DATACENTER

    OPERATIONS ENGINEER Gandalf Gollum Frodo Sam
  19. Copyright © 2017 HashiCorp @anubhavm  Scheduling !19 Traditional DATACENTER

    OPERATIONS ENGINEER Gandalf Gollum Frodo Sam
  20. Copyright © 2017 HashiCorp @anubhavm  Scheduling !20 Traditional DATACENTER

    OPERATIONS ENGINEER Gandalf Gollum Frodo Sam
  21. Copyright © 2017 HashiCorp @anubhavm  !21

  22. Copyright © 2017 HashiCorp @anubhavm  Scheduling !22 Traditional DATACENTER

    OPERATIONS ENGINEER Gandalf Gollum Frodo Sam
  23. Copyright © 2017 HashiCorp @anubhavm  Scheduling !23 Traditional DATACENTER

    OPERATIONS ENGINEER Gandalf Gollum Frodo Sam
  24. Copyright © 2017 HashiCorp @anubhavm  Scheduling !24 Traditional DATACENTER

    OPERATIONS ENGINEER Gandalf Gollum Frodo Sam
  25. Copyright © 2017 HashiCorp @anubhavm  Scheduling !25 Traditional DATACENTER

    OPERATIONS ENGINEER Gandalf Gollum Frodo Sam
  26. Copyright © 2017 HashiCorp @anubhavm  !26

  27. None
  28. Copyright © 2017 HashiCorp @anubhavm  CPU Scheduler !28 CORE

    CORE CORE CORE CPU SCHEDULER KERNEL APACHE REDIS BASH
  29. Copyright © 2017 HashiCorp @anubhavm  CPU Scheduler !29 CORE

    CORE CPU SCHEDULER KERNEL APACHE REDIS BASH
  30. Copyright © 2017 HashiCorp @anubhavm  CPU Scheduler !30 CORE

    CORE CPU SCHEDULER KERNEL APACHE REDIS BASH
  31. Copyright © 2017 HashiCorp @anubhavm  CPU Scheduler !31 CORE

    CORE CPU SCHEDULER KERNEL APACHE REDIS BASH
  32. Copyright © 2017 HashiCorp @anubhavm  Scheduler Advantages !32 Higher

    Resource Utilization Decouple Work from Resources Better Quality of Service
  33. Copyright © 2017 HashiCorp @anubhavm  Scheduler Advantages !33 Bin

    Packing Over-Subscription Job Queueing Higher Resource Utilization Decouple Work from Resources Better Quality of Service
  34. Copyright © 2017 HashiCorp @anubhavm  Scheduler Advantages !34 Abstraction

    API Contracts Standardization Higher Resource Utilization Decouple Work from Resources Better Quality of Service
  35. Copyright © 2017 HashiCorp @anubhavm  Scheduler Advantages !35 Priorities

    Resource Isolation Pre-emption Higher Resource Utilization Decouple Work from Resources Better Quality of Service
  36. Copyright © 2017 HashiCorp @anubhavm  Nop! Schedulers Aren’t New

    Concept !36
  37. Copyright © 2017 HashiCorp @anubhavm  Landscape !37

  38. None
  39. @anubhavm  !39 Cluster Scheduler Deployments Job Specification

  40. @anubhavm  !X job "redis" { datacenters = ["us-east-1"] task

    "redis" { driver = "docker" config { image = "redis:latest" } resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10 port "redis" {} } } } }
  41. @anubhavm  !X job "webserver" { datacenters = ["us-east-1"] task

    "webserver" { driver = "exec" config { command = "yet-another-golang-webserver-linux_amd64" } artifact { source = "https://github.com/anubhavmishra/yet-another-golang-webserver/releases/ download/v1.0.0/yet-another-golang-webserver-linux_amd64" } resources { cpu = 500 # Mhz memory = 128 # MB network { port "http" { static = 8080 } } } } }
  42. Copyright © 2017 HashiCorp @anubhavm  !42 Job specification declares

    what to run
  43. Copyright © 2017 HashiCorp @anubhavm  !43 Nomad determines how

    and where to run
  44. Copyright © 2017 HashiCorp @anubhavm  !44 Nomad abstracts work

    from resources
  45. Designing

  46. @anubhavm  !46 Multi-Datacenter Multi-Region Flexible Workloads Job Priorities Bin

    Packing Large Scale Operationally Simple
  47. Copyright © 2017 HashiCorp @anubhavm  Thousands of regions Tens

    of thousands of clients per region Thousands of jobs per region Scaling Requirements !47
  48. Copyright © 2017 HashiCorp @anubhavm  Our Past Experience !48

    GOSSIP CONSENSUS
  49. @anubhavm  !49 Cluster Management Gossip Based (P2P) Membership Failure

    Detection Event System Serf
  50. @anubhavm  !50 Serf Gossip Protocol Large Scale Production Hardened

    Operationally Simple
  51. @anubhavm  !51 Service Discovery Configuration Coordination (Locking) Central Servers

    + Distributed Clients
  52. @anubhavm  !52 Multi-Datacenter Raft Consensus Large Scale Production Hardened

  53. Copyright © 2017 HashiCorp @anubhavm  Our Past Experience !53

    GOSSIP CONSENSUS Mature Libraries Proven Design Patterns
  54. Copyright © 2017 HashiCorp @anubhavm  Our Past Experience !54

    GOSSIP CONSENSUS Mature Libraries Proven Design Patterns ?
  55. Copyright © 2017 HashiCorp @anubhavm  Our Past Experience !55

    GOSSIP CONSENSUS
  56. None
  57. @anubhavm  !57 Optimistic vs Pessimistic Internal vs External State

    Single vs Multi Level Fixed vs Pluggable Service vs Batch Oriented
  58. @anubhavm  !58 Inspired by Google Omega Optimistic Concurrency State

    Coordination Service & Batch workloads Pluggable Architecture
  59. Copyright © 2017 HashiCorp @anubhavm  Consul Cluster !59 CLIENT

    CLIENT CLIENT CLIENT CLIENT CLIENT SERVER SERVER SERVER REPLICATION REPLICATION RPC RPC LAN GOSSIP SERVER SERVER SERVER REPLICATION REPLICATION WAN GOSSIP
  60. Copyright © 2017 HashiCorp @anubhavm  Single Region Architecture !60

    SERVER SERVER SERVER CLIENT CLIENT CLIENT DC1 DC2 DC3 FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION FORWARDING RPC RPC RPC
  61. Copyright © 2017 HashiCorp @anubhavm  Single Region Architecture !61

    SERVER SERVER SERVER FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION REGION B  GOSSIP REPLICATION REPLICATION FORWARDING REGION FORWARDING  REGION A SERVER FOLLOWER SERVER SERVER LEADER FOLLOWER
  62. Copyright © 2017 HashiCorp @anubhavm  !62 Region is Isolation

    Domain 1-N Datacenters Per Region Flexibility to do 1:1 (Consul) Scheduling Boundary
  63. Copyright © 2017 HashiCorp @anubhavm  Omega Class Scheduler Pluggable

    Logic Internal Coordination and State Multi-Region / Multi-Datacenter Server Architecture !63
  64. Copyright © 2017 HashiCorp @anubhavm  Broad OS Support Host

    Fingerprinting Pluggable Drivers Client Architecture !64
  65. Copyright © 2017 HashiCorp @anubhavm  Fingerprinting !65 Type Examples

    Operating System Kernel, OS, Version Hardware CPU, Memory, Disk Apps (Capabilities) Docker, Java, Consul Environment AWS, GCE
  66. Copyright © 2017 HashiCorp @anubhavm  !66 Constrain Placement and

    Bin Pack
  67. Copyright © 2017 HashiCorp @anubhavm  !67 “Task Requires Linux,

    Docker, and PCI-Compliant Hardware” expressed as constraints in job file
  68. Copyright © 2017 HashiCorp @anubhavm  !68 “Task needs 512MB

    RAM and 1 Core” expressed as resource in job file
  69. Drivers

  70. @anubhavm  !70 Containerized Virtualized Standalone Docker Qemu / KVM

    Java Jar Static Binaries rkt LXC
  71. @anubhavm  !71 Containerized Virtualized Standalone Docker Qemu / KVM

    Java Jar Static Binaries rkt LXC Windows Server Containers Hyper-V Xen C#
  72. Copyright © 2017 HashiCorp @anubhavm  !72 Schedulers Fingerprints Drivers

    Job Specification
  73. @anubhavm  !73 Single Binary No Dependencies Highly Available

  74. Nomad Million Container Challenge 1,000 Jobs 1,000 Tasks per Job

    5,000 Hosts on GCE 1,000,000 Containers
  75. Copyright © 2017 HashiCorp @anubhavm  !75

  76. None
  77. Copyright © 2017 HashiCorp @anubhavm  !77 “640 KB ought

    to be enough for anybody.” - Bill Gates
  78. Copyright © 2017 HashiCorp @anubhavm  !78 2nd Largest Hedge

    Fund 18K Cores 5 Hours 2,200 Containers/second
  79. Copyright © 2017 HashiCorp @anubhavm  !79 7+ Million Builds

    a Month Sustain 400-1000 Jobs a Minute Great Talk By Danielle Tomlinson: https://youtu.be/b8NQO_vFAYo
  80. s Copyright © 2017 HashiCorp @anubhavm  !80 DEMO

  81. Copyright © 2017 HashiCorp @anubhavm  !81 Globally Distributed Optimistically

    Concurrent Scheduler
  82. Copyright © 2017 HashiCorp @anubhavm  !82 Higher Resource Utilization

    Decouple Work from Resources Better Quality of Service
  83. | June 25-27, 2018

  84. October 22-24, 2018 | San Francisco

  85. Thank You! I have stickers! Ask me anything. @anubhavm www.hashicorp.com

    Anubhav Mishra