Scheduling Applications at Scale

Slide 1

Slide 1 text

Nomad HASHICORP

Slide 2

Slide 2 text

HASHICORP Armon Dadgar @armon

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

HASHICORP

Slide 5

Slide 5 text

Nomad HASHICORP Distributed Optimistically Concurrent Scheduler

Slide 6

Slide 6 text

Nomad HASHICORP Distributed Optimistically Concurrent Scheduler

Slide 7

Slide 7 text

HASHICORP Schedulers map a set of work to a set of resources

Slide 8

Slide 8 text

HASHICORP CPU Scheduler Web Server -Thread 1 CPU - Core 1 CPU - Core 2 Web Server -Thread 2 Redis -Thread 1 Kernel -Thread 1 Work (Input) Resources CPU Scheduler

Slide 9

Slide 9 text

HASHICORP CPU Scheduler Web Server -Thread 1 CPU - Core 1 CPU - Core 2 Web Server -Thread 2 Redis -Thread 1 Kernel -Thread 1 Work (Input) Resources CPU Scheduler

Slide 10

Slide 10 text

HASHICORP Schedulers In the Wild Type Work Resources CPU Scheduler Threads Physical Cores AWS EC2 / OpenStack Nova Virtual Machines Hypervisors Hadoop YARN MapReduce Jobs Client Nodes Cluster Scheduler Applications Servers

Slide 11

Slide 11 text

HASHICORP Advantages Higher Resource Utilization Decouple Work from Resources Better Quality of Service

Slide 12

Slide 12 text

HASHICORP Advantages Bin Packing Over-Subscription Job Queueing Higher Resource Utilization Decouple Work from Resources Better Quality of Service

Slide 13

Slide 13 text

HASHICORP Advantages Abstraction API Contracts Standardization Higher Resource Utilization Decouple Work from Resources Better Quality of Service

Slide 14

Slide 14 text

HASHICORP Advantages Priorities Resource Isolation Pre-emption Higher Resource Utilization Decouple Work from Resources Better Quality of Service

Slide 15

Slide 15 text

HASHICORP

Slide 16

Slide 16 text

Nomad HASHICORP

Slide 17

Slide 17 text

Nomad HASHICORP Cluster Scheduler Easily Deploy Applications Job Speciﬁcation

Slide 18

Slide 18 text

HASHICORP example.nomad # Define our simple redis job job "redis" { # Run only in us-east-1 datacenters = ["us-east-1"] # Define the single redis task using Docker task "redis" { driver = "docker" config { image = "redis:latest" } resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10 dynamic_ports = ["redis"] } } } }

Slide 19

Slide 19 text

HASHICORP Job Speciﬁcation Declares what to run

Slide 20

Slide 20 text

HASHICORP Job Speciﬁcation Nomad determines where and manages how to run

Slide 21

Slide 21 text

HASHICORP Job Speciﬁcation Powerful yet simple

Slide 22

Slide 22 text

HASHICORP # Define our simple redis job job "redis" { # Run only in us-east-1 datacenters = ["us-east-1"] # Define the single redis task using Docker task "redis" { driver = "docker" config { image = "redis:latest" } resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10 dynamic_ports = ["redis"] } } } }

Slide 23

Slide 23 text

HASHICORP Containerized Virtualized Standalone Docker Qemu / KVM Java Jar Static Binaries

Slide 24

Slide 24 text

HASHICORP Containerized Virtualized Standalone Docker Jetpack Windows Server Containers Qemu / KVM Hyper-V Xen Java Jar Static Binaries C#

Slide 25

Slide 25 text

Nomad HASHICORP Application Deployment Docker Multi-Datacenter and Multi-Region Flexible Workloads Bin Packing HCL Job Speciﬁcations

Slide 26

Slide 26 text

Nomad HASHICORP Easy for developers Operationally simple Built for scale

Slide 27

Slide 27 text

HASHICORP Easy for Developers

Slide 28

Slide 28 text

HASHICORP Nomad for Developers Simple Data Model Declarative Job Speciﬁcation Sane Defaults

Slide 29

Slide 29 text

HASHICORP job “foobar” { # Restrict the parallelism in updates update { stagger = “60s” max_parallel = 3 } … }

Slide 30

Slide 30 text

HASHICORP job “foobar” { group “api” { # Scale our service up count = 5 … } }

Slide 31

Slide 31 text

HASHICORP job “foobar” { group “api” { # Scale our service down count = 3 … } }

Slide 32

Slide 32 text

HASHICORP job “foobar” { group “hdfs-data-node” { # Ensure the scheduler does not put # multiple instances on one host constraint { distinct_hosts = true } … } }

Slide 33

Slide 33 text

HASHICORP job “foobar” { group “hdfs-data-node” { # Attempt restart of tasks if they # fail unexpectedly restart { attempts = 5 interval = “10m” delay = “30s” } … } }

Slide 34

Slide 34 text

HASHICORP job “foobar” { task “my-app” { # Ensure modern kernel available constraint { attribute = “kernel.version” version = “>= 3.14” } … } }

Slide 35

Slide 35 text

HASHICORP job “foobar” { task “my-app” { # Inject environment variables env { MY_FEATURE_FLAG = “ON” } … } }

Slide 36

Slide 36 text

HASHICORP job “foobar” { task “my-app” { # Register with Consul for service # discovery and health checking service { port = “http” check { type = “tcp” interval = “10s” } } … } }

Slide 37

Slide 37 text

HASHICORP job “foobar” { # Make sure this task runs everywhere type = “system” # Nothing should evict our collector priority = 100 task “stats-collector” { … } }

Slide 38

Slide 38 text

Terminal HASHICORP $ nomad agent -dev ==> Starting Nomad agent... ==> Nomad agent configuration: Atlas: Client: true Log Level: DEBUG Region: global (DC: dc1) Server: true ==> Nomad agent started! Log data will stream in below: [INFO] serf: EventMemberJoin: nomad.global 127.0.0.1 [INFO] nomad: starting 4 scheduling worker(s) for [service batch _core] [INFO] raft: Node at 127.0.0.1:4647 [Follower] entering Follower state [INFO] nomad: adding server nomad.global (Addr: 127.0.0.1:4647) (DC: dc1) [DEBUG] client: applied fingerprints [storage arch cpu host memory] [DEBUG] client: available drivers [docker exec]

Slide 39

Slide 39 text

Nomad HASHICORP Infrastructure As Code Declarative Jobs Desired State Emergent State

Slide 40

Slide 40 text

HASHICORP Operationally Simple

Slide 41

Slide 41 text

HASHICORP Client Server

Slide 42

Slide 42 text

HASHICORP Built for Scale

Slide 43

Slide 43 text

HASHICORP Built on Experience gossip consensus

Slide 44

Slide 44 text

HASHICORP Built on Research gossip consensus

Slide 45

Slide 45 text

HASHICORP Single Region Architecture SERVER SERVER SERVER CLIENT CLIENT CLIENT DC1 DC2 DC3 FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION FORWARDING RPC RPC RPC

Slide 46

Slide 46 text

HASHICORP Multi Region Architecture SERVER SERVER SERVER FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION REGION B  GOSSIP REPLICATION REPLICATION FORWARDING REGION FORWARDING  REGION A SERVER FOLLOWER SERVER SERVER LEADER FOLLOWER

Slide 47

Slide 47 text

Nomad HASHICORP Region is Isolation Domain 1-N Datacenters Per Region Flexibility to do 1:1 (Consul) Scheduling Boundary

Slide 48

Slide 48 text

HASHICORP Thousands of regions Tens of thousands of clients per region Thousands of jobs per region

Slide 49

Slide 49 text

HASHICORP Optimistically Concurrent

Slide 50

Slide 50 text

HASHICORP Data Model

Slide 51

Slide 51 text

HASHICORP Evaluations ~= State Change Event

Slide 52

Slide 52 text

HASHICORP Create / Update / Delete Job Node Up / Node Down Allocation Failed

Slide 53

Slide 53 text

HASHICORP “Scheduler” = func(Eval) => []AllocUpdates

Slide 54

Slide 54 text

HASHICORP Scheduler func’s can specialize (Service, Batch, System, etc)

Slide 55

Slide 55 text

HASHICORP Evaluation Enqueue

Slide 56

Slide 56 text

HASHICORP Evaluation Dequeue

Slide 57

Slide 57 text

HASHICORP Plan Generation

Slide 58

Slide 58 text

HASHICORP Plan Execution

Slide 59

Slide 59 text

HASHICORP External Event Evalua?on Crea?on Evalua?on Queuing Evalua?on Processing Op?mis?c Coordina?on State Updates

Slide 60

Slide 60 text

HASHICORP Server Architecture Omega Class Scheduler Pluggable Logic Internal Coordination and State Multi-Region / Multi-Datacenter

Slide 61

Slide 61 text

HASHICORP Client Architecture Broad OS Support Host Fingerprinting Pluggable Drivers

Slide 62

Slide 62 text

HASHICORP Fingerprinting Operating System Hardware Applications Environment Type Examples Kernel, OS, Versions CPU, Memory, Disk Java, Docker, Consul AWS, GCE

Slide 63

Slide 63 text

HASHICORP Fingerprinting Constrain Placement and Bin Pack

Slide 64

Slide 64 text

HASHICORP Drivers Execute Tasks Provide Resource Isolation

Slide 65

Slide 65 text

Nomad HASHICORP Workload Flexibility: Schedulers Fingerprints Drivers Job Speciﬁcation

Slide 66

Slide 66 text

Nomad HASHICORP Operational Simplicity: Single Binary No Dependencies Highly Available

Slide 67

Slide 67 text

HASHICORP

Slide 68

Slide 68 text

HASHICORP Nomad 0.1 Released in October Service and Batch Scheduler Docker, Qemu, Exec, Java Drivers

Slide 69

Slide 69 text

HASHICORP Case Study

Slide 70

Slide 70 text

HASHICORP Case Study 3 servers in NYC3 100 clients in NYC3, SFO1, AMS2/3 1000 Containers

Slide 71

Slide 71 text

HASHICORP Case Study <1s to schedule 1s to ﬁrst start 6s to 95% 8s to 99%

Slide 72

Slide 72 text

HASHICORP Nomad 0.2 - Service Workloads Service Discovery System Scheduler Restart Policies Enhanced Constraints

Slide 73

Slide 73 text

HASHICORP Nomad 0.3 - Batch Workloads Cron Job Queuing Latency-Aware Scheduling

Slide 74

Slide 74 text

Nomad HASHICORP Cluster Scheduler Easily Deploy Applications Job Speciﬁcation

Slide 75

Slide 75 text

Nomad HASHICORP Higher Resource Utilization Decouple Work from Resources Better Quality of Service

Slide 76

Slide 76 text

HASHICORP Thanks! Q/A