Towards Running Stateful Applications on Nomad

Transcript

1. HASHICORP Towards Running Stateful Applications on Nomad Diptanu Gon Choudhury @diptanu HashiConf US Sep 2016
2. Quest to increase agility and reliability Develop Build Deploy Production Fast turnaround High Availability
 Control Plane for operations
3. Quest to increase agility and reliability Develop Build Deploy Region 1 Region 2 DC 1 DC 2 DC 3 DC 3 DC 3 DC 3
4. Nomad Multi DC Multi Region Diverse workload Diverse platform Built for scale Operationally simple
5. Nomad Optimistic Concurrency State Coordination Service, Batch and System workloads
6. Application Configuration Constraints job "redis" { datacenters = ["us-east-1"] group “cache” { count = 100 task "redis" { driver = "docker" config { image = "redis:latest" } resources { cpu = 500 memory = 256 network { mbits = 10 dynamic_ports = ["redis"] } } } }
7. job "redis" { datacenters = ["us-east-1"] task "redis" { driver = "docker" config { image = "redis:latest" } resources { cpu = 500 memory = 256 network { mbits = 10 dynamic_ports = ["redis"] } } } }
8. Batch Scheduler
 Service Scheduler Restart Policies
 System Scheduler Consul Integration
 Log Management
 Runtime Stats
 Job Plans Vault Integration
 TLS Sticky Volumes
 Disk Watchers Volume Plugins
 Network Plugins ACL Priorities
 Quotas
9. Nomad is a great platform for stateless workloads
10. Improved support and features
 for Stateful Applications
11. Nomad Jobs Task Group Task Group Task A Task B Task C Task D Allocation Allocation Job
12. Allocations Allocations are instances of a task group on a compute node Ephemeral in nature Allocations manage the life cycle of tasks within them Provides the environment and file system for the tasks
13. Allocation Resources CPU Shares Memory Network Ports and IPs Task Resources Shared Resources Disk Resources
14. Allocation Directory Allocation directory provides a shared data directory for tasks stdout and stderr streams of tasks are written in logs dir Each Task has a task local directory
15. Allocation Directory Ephemeral Disk Allocation Directory Task Dir Task Dir Allocation Directory
16. Allocation Directory Allocation Directory data logs tmp local tmp Task Dir Task Dir local tmp
17. Allocations Allocations are ephemeral in nature Allocations can be restarted on a different node Allocation directory not preserved when restarted on same node They are garbage collected by Nomad after they transition to terminal state
18. Sticky Volumes
19. Sticky Volumes Prefers to restart allocation on the same nodes to avoid replication Replicate the shared data dir and task local dirs of allocations Best effort replication, not a Distributed File Systems
20. group “redis” { ephemeral_disk { sticky = true
 size = 20000 } task “redis” { … } 
 task “backup-agent” { … } } cache.nomad
21. Ephemeral Disk Shared Allocation Directory Task A Task B
22. Ephemeral Disk Shared Allocation Directory Task A Task B Shared Allocation Directory Task A Task B Move
23. Node A Allocation Allocation Node B P2P Replication
24. Disk Watcher Watch dog process to monitor disk usage Allocations get killed if they use more disk resources than they are allocated If an allocation exceeds it’s disk quota, the allocation fails permanently
25. The Future Data Volumes to add additional volumes to allocations Volume plugins for materializing volumes on storage services like EBS, EFS, NFS, etc File system drivers for supporting file systems such as ZFS
26. Thanks! https://www.nomadproject.io/
 https://github.com/hashicorp/nomad