Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Towards Running Stateful Applications on Nomad
Search
Diptanu Choudhury
August 03, 2016
0
930
Towards Running Stateful Applications on Nomad
Diptanu Choudhury
August 03, 2016
Tweet
Share
More Decks by Diptanu Choudhury
See All by Diptanu Choudhury
Chaos Engineering and design principles for building Highly Available services on the cloud
diptanu
0
400
Taming the public and private clouds with Nomad
diptanu
1
190
Distributed Scheduling with Apache Mesos in the Cloud
diptanu
1
380
Reliably shipping containers in a resource rich world using Titan
diptanu
8
2.3k
Distributed Scheduling with Apache Mesos in the Cloud
diptanu
0
380
Featured
See All Featured
Why Our Code Smells
bkeepers
PRO
336
57k
Automating Front-end Workflow
addyosmani
1368
200k
Gamification - CAS2011
davidbonilla
80
5.2k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
59k
Build your cross-platform service in a week with App Engine
jlugia
229
18k
Adopting Sorbet at Scale
ufuk
74
9.2k
The Cult of Friendly URLs
andyhume
78
6.2k
Building Flexible Design Systems
yeseniaperezcruz
328
38k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.7k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
21
2.5k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
Transcript
HASHICORP Towards Running Stateful Applications on Nomad Diptanu Gon Choudhury
@diptanu HashiConf US Sep 2016
Quest to increase agility and reliability Develop Build Deploy Production
Fast turnaround High Availability Control Plane for operations
Quest to increase agility and reliability Develop Build Deploy Region
1 Region 2 DC 1 DC 2 DC 3 DC 3 DC 3 DC 3
Nomad Multi DC Multi Region Diverse workload Diverse platform Built
for scale Operationally simple
Nomad Optimistic Concurrency State Coordination Service, Batch and System workloads
Application Configuration Constraints job "redis" { datacenters = ["us-east-1"] group
“cache” { count = 100 task "redis" { driver = "docker" config { image = "redis:latest" } resources { cpu = 500 memory = 256 network { mbits = 10 dynamic_ports = ["redis"] } } } }
job "redis" { datacenters = ["us-east-1"] task "redis" { driver
= "docker" config { image = "redis:latest" } resources { cpu = 500 memory = 256 network { mbits = 10 dynamic_ports = ["redis"] } } } }
Batch Scheduler Service Scheduler Restart Policies System Scheduler Consul Integration
Log Management Runtime Stats Job Plans Vault Integration TLS Sticky Volumes Disk Watchers Volume Plugins Network Plugins ACL Priorities Quotas
Nomad is a great platform for stateless workloads
Improved support and features for Stateful Applications
Nomad Jobs Task Group Task Group Task A Task B
Task C Task D Allocation Allocation Job
Allocations Allocations are instances of a task group on a
compute node Ephemeral in nature Allocations manage the life cycle of tasks within them Provides the environment and file system for the tasks
Allocation Resources CPU Shares Memory Network Ports and IPs Task
Resources Shared Resources Disk Resources
Allocation Directory Allocation directory provides a shared data directory for
tasks stdout and stderr streams of tasks are written in logs dir Each Task has a task local directory
Allocation Directory Ephemeral Disk Allocation Directory Task Dir Task Dir
Allocation Directory
Allocation Directory Allocation Directory data logs tmp local tmp Task
Dir Task Dir local tmp
Allocations Allocations are ephemeral in nature Allocations can be restarted
on a different node Allocation directory not preserved when restarted on same node They are garbage collected by Nomad after they transition to terminal state
Sticky Volumes
Sticky Volumes Prefers to restart allocation on the same nodes
to avoid replication Replicate the shared data dir and task local dirs of allocations Best effort replication, not a Distributed File Systems
group “redis” { ephemeral_disk { sticky = true size =
20000 } task “redis” { … } task “backup-agent” { … } } cache.nomad
Ephemeral Disk Shared Allocation Directory Task A Task B
Ephemeral Disk Shared Allocation Directory Task A Task B Shared
Allocation Directory Task A Task B Move
Node A Allocation Allocation Node B P2P Replication
Disk Watcher Watch dog process to monitor disk usage Allocations
get killed if they use more disk resources than they are allocated If an allocation exceeds it’s disk quota, the allocation fails permanently
The Future Data Volumes to add additional volumes to allocations
Volume plugins for materializing volumes on storage services like EBS, EFS, NFS, etc File system drivers for supporting file systems such as ZFS
Thanks! https://www.nomadproject.io/ https://github.com/hashicorp/nomad