Nomad and next generation application architectures

We provide an overview of HashiCorp Nomad, which is a cluster manager and scheduler. We explore what a cluster scheduler is and the benefits they provide, including higher resource utilization, decoupling developers and operators, and better quality of service. The high level design of Nomad is explained and how it supports running over a million containers. The scalability of Nomad sets the stage for using the API from applications to build next generation applications that blur the lines between applications and infrastructure.

Transcript

1. Nomad and next-generation application architectures
2. Armon Dadgar Founder and CTO @armon
3. HashiCorp Suite CONNECT RUN SECURE PROVISION Infrastructure & applications Applications Infrastructure & applications Infrastructure Consul Nomad Terraform Vault Packer Vagrant Consul Enterprise Nomad Enterprise Vault Enterprise Terraform Enterprise FOR TEAMS OSS TOOL SUITE PRODUCT SUITE
4. Nomad Cluster Manager Scheduler
5. Nomad Cluster Manager Scheduler
6. Schedulers map a set of work to a set of resources
7. CPU Scheduler Web Server -Thread 1 CPU - Core 1 CPU - Core 2 Web Server -Thread 2 Redis -Thread 1 Kernel -Thread 1 Work (Input) Resources CPU Scheduler
8. CPU Scheduler Web Server -Thread 1 CPU - Core 1 CPU - Core 2 Web Server -Thread 2 Redis -Thread 1 Kernel -Thread 1 Work (Input) Resources CPU Scheduler
9. Schedulers in the Wild Type Work Resources CPU Scheduler Threads Physical Cores AWS EC2 / OpenStack Nova Virtual Machines Hypervisors Hadoop YARN MapReduce Jobs Client Nodes Cluster Scheduler Applications Servers
10. Advantages Higher Resource U.liza.on Decouple Work from Resources Be:er Quality of Service
11. Advantages Higher Resource U.liza.on Decouple Work from Resources Be:er Quality of Service Bin Packing Over-Subscrip.on Job Queueing
12. Advantages Higher Resource U.liza.on Decouple Work from Resources Be:er Quality of Service Abstrac.on API Contracts Packaging
13. Advantages Higher Resource U.liza.on Decouple Work from Resources Be:er Quality of Service Priori.es Resource Isola.on Pre-emp.on
15.  Nomad @armon
16. Nomad Cluster Scheduler Easily Deploy Applications Operationally Simple Built for Scale API for Next-Gen Patterns
17. job "redis" { datacenters = ["us-east-1"] task "redis" { driver = "docker" config { image = "redis:latest" } resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10 port “redis" {} } } } } example.nomad
18. Declares what to run
19. Nomad determines where and manages how to run
20. Nomad abstracts work from resources
21. OS Workloads Drivers Windows Long Running Service Docker / Rkt / LXC Linux Short Lived Batch Qemu / KVM BSD Periodic Cron “exec” cgroups+chroot Solaris System Agents Static Binaries / Fat JARs
22. Nomad Declarative Jobs Infrastructure as Code Consul Integration Vault Integration Composable vs Platform
23. Empowers developers by de-coupling operators
24. Operationally Simple
25. Client Server
26. Built on Experience GOSSIP CONSENSUS
27. Serf Cluster Management Gossip Based (P2P) Membership Failure Detection Event System
28. Serf Large Scale Production Hardened Simple Clustering and Federation
29. Consul Service Discovery Configuration Coordination (Locking) Central Servers + Distributed Clients
30. Consul Multi-Datacenter Raft Consensus Large Scale Production Hardened
31. Nomad Single Binary No Dependencies Highly Available Multi-DC/Region Support
32. Built for Scale
33. Built on Experience GOSSIP CONSENSUS Mature Libraries Proven Design Patterns Lacking Scheduling Logic
34. Built on Research GOSSIP CONSENSUS
36. Single Region Architecture SERVER SERVER SERVER CLIENT CLIENT CLIENT DC1 DC2 DC3 FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION FORWARDING RPC RPC RPC
37. Multi Region Architecture SERVER SERVER SERVER FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION REGION B  GOSSIP REPLICATION REPLICATION FORWARDING REGION FORWARDING  REGION A SERVER FOLLOWER SERVER SERVER LEADER FOLLOWER
38. 100’s of Regions 10,000’s of Clients per Region 1000’s of Jobs per Region
39. Nomad Inspired by Google Omega Optimistic Concurrency Service & Batch workloads Pluggable Architecture
40. Nomad Million Container Challenge 1,000 Jobs 1,000 Tasks per Job 5,000 Hosts on GCE 1,000,000 Containers
42. “640 KB ought to be enough for anybody.” - Bill Gates
43. 2nd Largest Hedge Fund 18K Cores 5 Hours 2,200 Containers/second
44. Next-Gen Patterns
45. Monolith Micro-Service SOA Spectrum
46. Monolith Micro-Service SOA Spectrum Utility of Service
47. Monoliths have high application complexity
48. Microservices have high operational complexity
49. Abstractions allow us to scale complexity
50. Frameworks :: Monoliths Schedulers :: Services
51. Schedulers abstract details, focus on service composition
52. Side Cars • Sidecar or Co-Process Pattern • Application that runs alongside “main” process • Nomad “Task Group” • Borg “Alloc” • Kubernetes “Pod”
53. Routing Proxy Log Shipper App1 Routing Proxy Log Shipper App2 Client Node Allocation #1 Allocation #2 App3 Allocation #3
54. Side Cars • Configuration (Consul-Template) • Logging Agents (Splunk, CloudWatch) • Telemetry Agents (Datadog) • Service Mesh (Envoy, Linkerd) • Load Balancing (HAProxy, Nginx, Fabio)
55. Nomad Transparent Scheduling API awareness Dynamic Behavior
56. Queues
57. Queues • Workers are online service doing batch work • Workers provisioned in advance • N+1 instances for high availability • Typically idle or underutilized
58. Nomad Dispatch • “Dispatch” a worker for each incoming event • Consumer launched on-demand and terminates when done • Publisher shielded from implementation detail • Nomad job acts like a future, queues when busy • Avoids underutilization
59. job “my-dispatch” { datacenter = [“dc1”] type = “batch” parameterized { meta_required = [“input”] } task “worker” { driver = “docker” config = { image = “myworker:latest” args = [“—input”, “${NOMAD_META_INPUT}”] }
 } } my-dispatch.job
60. Nomad Server Register Dispatch Job
61. Nomad Server Register Dispatch Job Web Server Dispatch
62. Nomad Server Register Dispatch Job Web Server Dispatch Worker 1 Schedule
63. Nomad Server Register Dispatch Job Web Server Dispatch Worker 1 Schedule Worker 2 Worker N
64. Function-as-a-Service • AWS Lambda • Small Granularity • Low Volume, Latency Insensitive => Nomad Dispatch • High Volume, Latency Sensitive => Setup Overhead Prohibitive
65. FaaS / Serverless • Process multiple events per worker • Dynamically scale workers • Queue messages to avoid dropping
66. Nomad Server Register Dispatch Job Controller
67. Nomad Server Register Dispatch Job Web Server Push Controller Dispatch
68. Nomad Server Register Dispatch Job Web Server Push Worker 1 Schedule Controller Pull Dispatch
69. Nomad Server Register Dispatch Job Web Server Push Worker 1 Schedule Controller Pull Dispatch Deep Message Queue
70. Nomad Server Register Dispatch Job Web Server Push Worker 1 Schedule Worker 2 Worker N Controller Pull Dispatch
71. Big Data Processing • Large scale batch workload • Graph of processing steps • Each phase dynamic size • Programmatically setup/teardown workers • Native Spark Integration!
72. Nomad Server Executor 1 Schedule Executor 2 Executor N Launch Executors Submit Job
73. Large-scale cluster management at Google with Borg. Abhishek Verma, Luis Pedrosa, Madhukar R. Korupolu, David Oppenheimer, Eric Tune, John Wilkes.
74. Scheduler API blurs line between Application and Infrastructure
75. Nomad enables dynamic behavior while optimizing utilization
76. Nomad Cluster Scheduler Easily Deploy Applications Operationally Simple Built for Scale API for Next-Gen Patterns
77.  Thanks! @armon