Nomad and next-generation
application architectures
Slide 2
Slide 2 text
Armon Dadgar
Founder and CTO
@armon
Slide 3
Slide 3 text
HashiCorp Suite
CONNECT
RUN
SECURE
PROVISION
Infrastructure & applications
Applications
Infrastructure & applications
Infrastructure
Consul
Nomad
Terraform
Vault
Packer
Vagrant
Consul Enterprise
Nomad Enterprise
Vault Enterprise
Terraform Enterprise
FOR TEAMS
OSS TOOL SUITE PRODUCT SUITE
Slide 4
Slide 4 text
Nomad
Cluster Manager
Scheduler
Slide 5
Slide 5 text
Nomad
Cluster Manager
Scheduler
Slide 6
Slide 6 text
Schedulers map a set of
work to a set of resources
Slide 7
Slide 7 text
CPU Scheduler
Web Server -Thread 1
CPU - Core 1
CPU - Core 2
Web Server -Thread 2
Redis -Thread 1
Kernel -Thread 1
Work (Input) Resources
CPU
Scheduler
Slide 8
Slide 8 text
CPU Scheduler
Web Server -Thread 1
CPU - Core 1
CPU - Core 2
Web Server -Thread 2
Redis -Thread 1
Kernel -Thread 1
Work (Input) Resources
CPU
Scheduler
Slide 9
Slide 9 text
Schedulers in the Wild
Type Work Resources
CPU Scheduler Threads Physical Cores
AWS EC2 /
OpenStack Nova
Virtual Machines Hypervisors
Hadoop YARN MapReduce Jobs Client Nodes
Cluster Scheduler Applications Servers
Slide 10
Slide 10 text
Advantages
Higher Resource U.liza.on
Decouple Work from Resources
Be:er Quality of Service
Slide 11
Slide 11 text
Advantages
Higher Resource U.liza.on
Decouple Work from Resources
Be:er Quality of Service
Bin Packing
Over-Subscrip.on
Job Queueing
Slide 12
Slide 12 text
Advantages
Higher Resource U.liza.on
Decouple Work from Resources
Be:er Quality of Service
Abstrac.on
API Contracts
Packaging
Slide 13
Slide 13 text
Advantages
Higher Resource U.liza.on
Decouple Work from Resources
Be:er Quality of Service
Priori.es
Resource Isola.on
Pre-emp.on
Slide 14
Slide 14 text
No content
Slide 15
Slide 15 text
Nomad
@armon
Slide 16
Slide 16 text
Nomad
Cluster Scheduler
Easily Deploy Applications
Operationally Simple
Built for Scale
API for Next-Gen Patterns
OS Workloads Drivers
Windows Long Running Service Docker / Rkt / LXC
Linux Short Lived Batch Qemu / KVM
BSD Periodic Cron
“exec”
cgroups+chroot
Solaris System Agents
Static Binaries /
Fat JARs
Slide 22
Slide 22 text
Nomad
Declarative Jobs
Infrastructure as Code
Consul Integration
Vault Integration
Composable vs Platform
Slide 23
Slide 23 text
Empowers developers by
de-coupling operators
Slide 24
Slide 24 text
Operationally Simple
Slide 25
Slide 25 text
Client Server
Slide 26
Slide 26 text
Built on Experience
GOSSIP CONSENSUS
Slide 27
Slide 27 text
Serf
Cluster Management
Gossip Based (P2P)
Membership
Failure Detection
Event System
Slide 28
Slide 28 text
Serf
Large Scale
Production Hardened
Simple Clustering and
Federation
Slide 29
Slide 29 text
Consul
Service Discovery
Configuration
Coordination (Locking)
Central Servers +
Distributed Clients
Slide 30
Slide 30 text
Consul
Multi-Datacenter
Raft Consensus
Large Scale
Production Hardened
Slide 31
Slide 31 text
Nomad
Single Binary
No Dependencies
Highly Available
Multi-DC/Region Support
Slide 32
Slide 32 text
Built for Scale
Slide 33
Slide 33 text
Built on Experience
GOSSIP CONSENSUS
Mature Libraries Proven Design Patterns
Lacking Scheduling Logic
Slide 34
Slide 34 text
Built on Research
GOSSIP CONSENSUS
Slide 35
Slide 35 text
No content
Slide 36
Slide 36 text
Single Region Architecture
SERVER SERVER SERVER
CLIENT CLIENT CLIENT
DC1 DC2 DC3
FOLLOWER LEADER FOLLOWER
REPLICATION
FORWARDING
REPLICATION
FORWARDING
RPC RPC RPC
Slide 37
Slide 37 text
Multi Region Architecture
SERVER SERVER SERVER
FOLLOWER LEADER FOLLOWER
REPLICATION
FORWARDING
REPLICATION
REGION B
GOSSIP
REPLICATION REPLICATION
FORWARDING
REGION FORWARDING
REGION A
SERVER
FOLLOWER
SERVER SERVER
LEADER FOLLOWER
Slide 38
Slide 38 text
100’s of Regions
10,000’s of Clients per Region
1000’s of Jobs per Region
Slide 39
Slide 39 text
Nomad
Inspired by Google Omega
Optimistic Concurrency
Service & Batch workloads
Pluggable Architecture
Slide 40
Slide 40 text
Nomad
Million Container
Challenge
1,000 Jobs
1,000 Tasks per Job
5,000 Hosts on GCE
1,000,000 Containers
Slide 41
Slide 41 text
No content
Slide 42
Slide 42 text
“640 KB ought to be enough for anybody.”
- Bill Gates
Slide 43
Slide 43 text
2nd Largest Hedge Fund
18K Cores
5 Hours
2,200 Containers/second
Slide 44
Slide 44 text
Next-Gen Patterns
Slide 45
Slide 45 text
Monolith
Micro-Service
SOA Spectrum
Slide 46
Slide 46 text
Monolith
Micro-Service
SOA Spectrum
Utility of Service
Slide 47
Slide 47 text
Monoliths have high
application complexity
Slide 48
Slide 48 text
Microservices have high
operational complexity
Slide 49
Slide 49 text
Abstractions allow us to
scale complexity
Slide 50
Slide 50 text
Frameworks :: Monoliths
Schedulers :: Services
Slide 51
Slide 51 text
Schedulers abstract details,
focus on service composition
Slide 52
Slide 52 text
Side Cars
• Sidecar or Co-Process Pattern
• Application that runs alongside “main” process
• Nomad “Task Group”
• Borg “Alloc”
• Kubernetes “Pod”
Nomad
Transparent Scheduling
API awareness
Dynamic Behavior
Slide 56
Slide 56 text
Queues
Slide 57
Slide 57 text
Queues
• Workers are online service doing batch work
• Workers provisioned in advance
• N+1 instances for high availability
• Typically idle or underutilized
Slide 58
Slide 58 text
Nomad Dispatch
• “Dispatch” a worker for each incoming event
• Consumer launched on-demand and terminates when done
• Publisher shielded from implementation detail
• Nomad job acts like a future, queues when busy
• Avoids underutilization