Tweet
Tweet

Slide 1

Slide 1 text

Nomad Overview + 0.6 New Features

Slide 2

Slide 2 text

Armon Dadgar Founder and CTO @armon

Slide 3

Slide 3 text

Copyright © 2017 HashiCorp CONNECT Infrastructure & applications RUN SECURE PROVISION Applications Infrastructure & applications Infrastructure OSS TOOL SUITE PRODUCT SUITE Consul Nomad Terraform Vault Consul Enterprise Nomad Enterprise Vault Enterprise Terraform Enterprise FOR TEAMS FOR INDIVIDUALS Provision, secure, connect, and run any infrastructure 3 Copyright © 2017 HashiCorp

Slide 4

Slide 4 text

Copyright © 2017 HashiCorp ▪Nomad Overview ▪New Features of 0.6 ▪Demo! 4 Agenda

Slide 5

Slide 5 text

Nomad Cluster Manager Scheduler

Slide 6

Slide 6 text

Nomad Cluster Manager Scheduler

Slide 7

Slide 7 text

Schedulers map a set of work to a set of resources

Slide 8

Slide 8 text

CPU Scheduler Web Server -Thread 1 CPU - Core 1 CPU - Core 2 Web Server -Thread 2 Redis -Thread 1 Kernel -Thread 1 Work (Input) Resources CPU Scheduler

Slide 9

Slide 9 text

CPU Scheduler Web Server -Thread 1 CPU - Core 1 CPU - Core 2 Web Server -Thread 2 Redis -Thread 1 Kernel -Thread 1 Work (Input) Resources CPU Scheduler

Slide 10

Slide 10 text

Schedulers in the Wild Type Work Resources CPU Scheduler Threads Physical Cores AWS EC2 / OpenStack Nova Virtual Machines Hypervisors Hadoop YARN MapReduce Jobs Client Nodes Cluster Scheduler Applications Servers

Slide 11

Slide 11 text

User Nomad Servers Nomad Clients

Slide 12

Slide 12 text

User Nomad Servers Submits Job Nomad Clients

Slide 13

Slide 13 text

User Nomad Servers Submits Job Nomad Clients Deploy App

Slide 14

Slide 14 text

User Nomad Servers Submits Job Nomad Clients Deploy App Skip (Busy)

Slide 15

Slide 15 text

User Nomad Servers Submits Job Nomad Clients Deploy App Skip (Busy) Deploy App

Slide 16

Slide 16 text

Advantages Higher Resource U.liza.on Decouple Work from Resources Be:er Quality of Service

Slide 17

Slide 17 text

Advantages Higher Resource U.liza.on Decouple Work from Resources Be:er Quality of Service Bin Packing Over-Subscrip.on Job Queueing

Slide 18

Slide 18 text

Advantages Higher Resource U.liza.on Decouple Work from Resources Be:er Quality of Service Abstrac.on API Contracts Packaging

Slide 19

Slide 19 text

Advantages Higher Resource U.liza.on Decouple Work from Resources Be:er Quality of Service Priori.es Resource Isola.on Pre-emp.on

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

 Nomad @armon

Slide 22

Slide 22 text

Nomad Cluster Scheduler Easily Deploy Applications Operationally Simple Built for Scale

Slide 23

Slide 23 text

job "redis" { datacenters = ["us-east-1"] task "redis" { driver = "docker" config { image = "redis:latest" } resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10 port “redis" {} } } } } example.nomad

Slide 24

Slide 24 text

Declares what to run

Slide 25

Slide 25 text

Nomad determines where and manages how to run

Slide 26

Slide 26 text

Nomad abstracts work from resources

Slide 27

Slide 27 text

OS Workloads Drivers Windows Long Running Service Docker / Rkt / LXC Linux Short Lived Batch Qemu / KVM BSD Periodic Cron “exec” cgroups+chroot Solaris System Agents Static Binaries / Fat JARs

Slide 28

Slide 28 text

Nomad Declarative Jobs Infrastructure as Code Consul Integration Vault Integration Composable vs Platform

Slide 29

Slide 29 text

Empowers developers by de-coupling operators

Slide 30

Slide 30 text

Operationally Simple & Scalable

Slide 31

Slide 31 text

Nomad Single Binary No Dependencies Highly Available Multi-DC/Region Support

Slide 32

Slide 32 text

Built on Research & Experience GOSSIP CONSENSUS Mature Libraries Proven Design Patterns

Slide 33

Slide 33 text

Nomad Inspired by Google Omega Optimistic Concurrency Service & Batch workloads Pluggable Architecture

Slide 34

Slide 34 text

Single Region Architecture SERVER SERVER SERVER CLIENT CLIENT CLIENT DC1 DC2 DC3 FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION FORWARDING RPC RPC RPC

Slide 35

Slide 35 text

Multi Region Architecture SERVER SERVER SERVER FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION REGION B  GOSSIP REPLICATION REPLICATION FORWARDING REGION FORWARDING  REGION A SERVER FOLLOWER SERVER SERVER LEADER FOLLOWER

Slide 36

Slide 36 text

100’s of Regions 10,000’s of Clients per Region 1000’s of Jobs per Region

Slide 37

Slide 37 text

Nomad Million Container Challenge 1,000 Jobs 1,000 Tasks per Job 5,000 Hosts on GCE 1,000,000 Containers

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

“640 KB ought to be enough for anybody.” - Bill Gates

Slide 40

Slide 40 text

2nd Largest Hedge Fund 18K Cores 5 Hours 2,200 Containers/second

Slide 41

Slide 41 text

Nomad Cluster Scheduler Easily Deploy Applications Operationally Simple Built for Scale

Slide 42

Slide 42 text

Copyright © 2017 HashiCorp ▪Service Updates ▪Rolling Deploys with Health Checks ▪Canaries ▪Blue/Green ▪Dynamic Environment Variables ▪Multiple Job Versions & Rollbacks ▪Docker Networking Plugins ▪Much More (see CHANGELOG) 42 New Features in 0.6

Slide 43

Slide 43 text

Service Updates • Changing a Service Job from Version A to Version B • Upgrading WebApp from 1.1.0 to 1.2.0 • Controlled by update block • Supports many different approaches • Rolling Deploys, Canaries, Blue/Green

Slide 44

Slide 44 text

Copyright © 2017 HashiCorp 44 Rolling Deploys job "webapp" { update { max_parallel = 2 } group "web" { count = 6 task "app" { driver = "docker" config { image = "webapp:1.0" } } } } webapp.nomad

Slide 45

Slide 45 text

webapp 1.0

Slide 46

Slide 46 text

webapp 1.0 1.0 1.0 1.0 1.0 1.0 1.0

Slide 47

Slide 47 text

1.0 1.0 1.0 1.0 webapp 2.0 2.0 2.0

Slide 48

Slide 48 text

1.0 1.0 webapp 2.0 2.0 2.0 2.0 2.0

Slide 49

Slide 49 text

webapp 2.0 2.0 2.0 2.0 2.0 2.0 2.0

Slide 50

Slide 50 text

Copyright © 2017 HashiCorp 50 Canaries job "webapp" { update { max_parallel = 2 canary = 1 } group "web" { count = 6 task "app" { driver = "docker" config { image = "webapp:1.0" } } } } webapp.nomad

Slide 51

Slide 51 text

webapp 1.0 1.0 1.0 1.0 1.0 1.0 1.0

Slide 52

Slide 52 text

1.0 1.0 1.0 1.0 webapp 2.0 2.0 1.0 1.0 Canary

Slide 53

Slide 53 text

1.0 1.0 1.0 1.0 Promote webapp 2.0 2.0 1.0

Slide 54

Slide 54 text

1.0 1.0 1.0 Promote webapp 2.0 2.0 2.0 2.0

Slide 55

Slide 55 text

1.0 Promote webapp 2.0 2.0 2.0 2.0 2.0 2.0

Slide 56

Slide 56 text

Promote webapp 2.0 2.0 2.0 2.0 2.0 2.0 2.0

Slide 57

Slide 57 text

Copyright © 2017 HashiCorp 57 Blue/Green job "webapp" { update { max_parallel = 2 canary = 6 # Match count of group } group "web" { count = 6 task "app" { driver = "docker" config { image = "webapp:1.0" } } } } webapp.nomad

Slide 58

Slide 58 text

webapp 1.0 1.0 1.0 1.0 1.0 1.0 1.0

Slide 59

Slide 59 text

1.0 1.0 1.0 1.0 webapp 2.0 2.0 1.0 1.0 Canary 2.0 2.0 2.0 2.0 2.0 Canary Canary Canary Canary Canary

Slide 60

Slide 60 text

2.0 2.0 2.0 2.0 2.0 2.0 Promote webapp 2.0

Slide 61

Slide 61 text

Multiple Job Versions & Rollbacks • Track Multiple Versions per Job • Rollbacks • “Auto Revert” failed deploys

Slide 62

Slide 62 text

Copyright © 2017 HashiCorp 62 Job Versions $ nomad job history example Version = 1 Stable = true Submit Date = 07/08/17 23:29:31 UTC Version = 0 Stable = true Submit Date = 07/08/17 23:27:54 UTC $ nomad job revert example 0 ==> Monitoring evaluation "4e0662c7" … ==> Evaluation "4e0662c7" finished with status “complete" $ nomad job history -p example Version = 2 Stable = true Submit Date = 07/08/17 23:31:07 UTC Diff = +/- Job: "example" +/- Task Group: "cache" +/- Task: "redis" +/- Config { +/- image: "redis:3.0" => "redis:3.2" port_map[0][db]: "6379" } Terminal

Slide 63

Slide 63 text

Copyright © 2017 HashiCorp 63 Job Status with Versions $ nomad status example ID = example Name = example Submit Date = 07/08/17 23:31:07 UTC Type = service Priority = 50 Datacenters = dc1 Status = running Periodic = false Parameterized = false Summary Task Group Queued Starting Running Failed Complete Lost cache 0 0 1 0 2 0 Latest Deployment ID = 07e88357 Status = successful Description = Deployment completed successfully Deployed Task Group Desired Placed Healthy Unhealthy cache 1 1 1 0 Allocations ID Node ID Task Group Version Desired Status Created At 71c6f915 49a104fe cache 2 run running 07/08/17 23:31:07 UTC 9b69e806 49a104fe cache 1 stop complete 07/08/17 23:29:31 UTC 2e5550f0 49a104fe cache 0 stop complete 07/08/17 23:27:54 UTC Terminal

Slide 64

Slide 64 text

Dynamic Environment Variables • The template block renders dynamic configuration • Integrates with Consul for app configuration • Integrates with Vault for secrets (DB credentials, TLS, etc) • New env parameter sets environment variables

Slide 65

Slide 65 text

Copyright © 2017 HashiCorp 65 Dynamic Environment Variables job "webapp" { task "app" { driver = "docker" config { image = "webapp:1.0" } template { data = <

Slide 66

Slide 66 text

Demo!

Slide 67

Slide 67 text

 Thanks! @armon