HashiCorp Nomad

We provide an overview of HashiCorp Nomad, which is a cluster manager and scheduler. We explore what a cluster scheduler is and the benefits they provide, including higher resource utilization, decoupling developers and operators, and better quality of service. The high level design of Nomad is explained and how it supports running over a million containers. The scalability of Nomad sets the stage for using the API from applications to build next generation applications that blur the lines between applications and infrastructure.

Transcript

1. HashiCorp Nomad
2. Anubhav Mishra Developer Advocate @anubhavm
3. DEVOPS DELIVERED
4. PROVISION, SECURE AND RUN ANY INFRASTRUCTURE Nomad Consul Vault Vagrant Packer Terraform Consul Enterprise Terraform Enterprise Vault Enterprise PRODUCT SUITE OSS TOOL SUITE RUN Applications SECURE Application Infrastructure PROVISION Infrastructure FOR INDIVIDUALS FOR TEAMS Nomad Enterprise
5. Copyright © 2017 HashiCorp ▪Nomad Overview ▪Demo! 5 Agenda
6. Nomad Cluster Manager Scheduler
7. User Nomad Servers Nomad Clients
8. User Nomad Servers Submits Job Nomad Clients
9. User Nomad Servers Submits Job Nomad Clients Deploy App
10. User Nomad Servers Submits Job Nomad Clients Deploy App Skip (Busy)
11. User Nomad Servers Submits Job Nomad Clients Deploy App Skip (Busy) Deploy App
12. Advantages Higher Resource U.liza.on Decouple Work from Resources Be:er Quality of Service
13.  Nomad @anubhavm
14. Nomad Cluster Scheduler Easily Deploy Applications Operationally Simple Built for Scale
15. job "redis" { datacenters = ["us-east-1"] task "redis" { driver = "docker" config { image = "redis:latest" } resources { cpu = 500 # Mhz memory = 256 # MB network { mbits = 10 port “redis" {} } } } } example.nomad
16. Declares what to run
17. Nomad determines where and manages how to run
18. Nomad abstracts work from resources
19. OS Workloads Drivers Windows Long Running Service Docker / Rkt / LXC Linux Short Lived Batch Qemu / KVM BSD Periodic Cron “exec” cgroups+chroot Solaris System Agents Static Binaries / Fat JARs
20. Nomad Declarative Jobs Infrastructure as Code Consul Integration Vault Integration Composable vs Platform
21. Empowers developers by de-coupling operators
22. Operationally Simple & Scalable
23. Nomad Single Binary No Dependencies Highly Available Multi-DC/Region Support
24. Built on Research & Experience GOSSIP CONSENSUS Mature Libraries Proven Design Patterns
25. Nomad Inspired by Google Omega Optimistic Concurrency Service & Batch workloads Pluggable Architecture
26. Single Region Architecture SERVER SERVER SERVER CLIENT CLIENT CLIENT DC1 DC2 DC3 FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION FORWARDING RPC RPC RPC
27. Multi Region Architecture SERVER SERVER SERVER FOLLOWER LEADER FOLLOWER REPLICATION FORWARDING REPLICATION REGION B  GOSSIP REPLICATION REPLICATION FORWARDING REGION FORWARDING  REGION A SERVER FOLLOWER SERVER SERVER LEADER FOLLOWER
28. 100’s of Regions 10,000’s of Clients per Region 1000’s of Jobs per Region
29. Nomad Million Container Challenge 1,000 Jobs 1,000 Tasks per Job 5,000 Hosts on GCE 1,000,000 Containers
31. “640 KB ought to be enough for anybody.” - Bill Gates
32. 2nd Largest Hedge Fund 18K Cores 5 Hours 2,200 Containers/second
33. Nomad Cluster Scheduler Easily Deploy Applications Operationally Simple Built for Scale
34. Demo!
35.  Thanks! @anubhavm