Hybrid GPU Orchestration for Enterprise AI at Scale

In this presentation, I dive into how to think about hybrid GPU-enabled workloads and why HashiCorp Nomad is the right foundation for an Enterprise AI platform.

This version of the talk was given at IBM's booth at NVIDIA GTC in March 2026.

Transcript

1. Hybrid GPU Orchestration for Enterprise AI at Scale Kerim Satirli Senior Developer Advocate II HashiCorp, an IBM Company NVIDIA GTC 2026
2. ©2026 HASHICORP Your GPU is probably in three places at once.
3. ©2026 HASHICORP Most schedulers only see one of them.
4. Edge On-Prem Cloud Workload Locations
5. mobile deployments and disconnected locations • NVIDIA Jetson Series • disposable compute Edge traditional DCS and failover locations • Linux distributions • Windows Server On-Prem AWS, Azure, Google, IBM Cloud, and more • traditional compute • specialized compute Cloud Workload Locations
6. mobile deployments and disconnected locations • NVIDIA Jetson Series • disposable compute Edge traditional DCS and failover locations • Linux distributions • Windows Server On-Prem AWS, Azure, Google, IBM Cloud, and more • traditional compute • specialized compute Cloud Workload Locations HashiCorp Nomad
7. ©2026 HASHICORP AI workloads are not special
8. ©2026 HASHICORP • resource hungry • state aware • hardware constrained AI workloads are just
9. Workload Orchestration with HashiCorp Nomad
10. Executables Containers Runtimes Workload Support
11. Executables Containers Runtimes Workload Support • isolated and raw execution • Windows binaries • macOS binaries traditional workloads
12. Executables Containers Runtimes Workload Support • isolated and raw execution • Windows binaries • macOS binaries • Docker • Podman • containerd traditional workloads modern workloads
13. Executables Containers Runtimes Workload Support • isolated and raw execution • Windows binaries • macOS binaries • Docker • Podman • containerd • Java • QEMU • libvirt traditional workloads modern workloads specialized workloads
14. Executables Containers Runtimes Workload Support • isolated and raw execution • Windows binaries • macOS binaries • Docker • Podman • containerd • Java • QEMU • libvirt traditional workloads modern workloads specialized workloads HashiCorp Nomad
15. ©2026 HASHICORP Nomad is a general-purpose workload orchestrator
16. ©2026 HASHICORP Nomad is a general-purpose workload orchestrator (not a container runtime)
17. Orchestrating a Workload with HashiCorp Nomad
18. defines basic job properties • datacenter and region • type of job • update strategy Job Workload Specification
19. defines basic job properties • datacenter and region • type of job • update strategy defines how to co-locate tasks • network config • volume config • service discovery Job Group Workload Specification
20. defines basic job properties • datacenter and region • type of job • update strategy defines atomic units of work • driver selection • task environment • resource requirements defines how to co-locate tasks • network config • volume config • service discovery Task Job Group Workload Specification
21. GPU Workloads with Nomad job "docling" { datacenters = ["dc1"] type = "service" constraint { attribute = "${node.class}" value = "linux" } group "api" { task "docling-serve" { driver = "podman" config { image = "quay.io/docling-project/docling-serve:latest" command = "docling-serve" } device "nvidia/gpu" { count = 1 } } } } docling.nomad.hcl
22. GPU Workloads with Nomad job "docling" { datacenters = ["dc1"] type = "service" constraint { attribute = "${node.class}" value = "linux" } group "api" { task "docling-serve" { driver = "podman" config { image = "quay.io/docling-project/docling-serve:latest" command = "docling-serve" } device "nvidia/gpu" { count = 1 } } } } docling.nomad.hcl
23. GPU Workloads with Nomad job "docling" { datacenters = ["dc1"] type = "service" constraint { attribute = "${node.class}" value = "linux" } group "api" { task "docling-serve" { driver = "podman" config { image = "quay.io/docling-project/docling-serve:latest" command = "docling-serve" } device "nvidia/gpu" { count = 1 } } } } docling.nomad.hcl
24. GPU Workloads with Nomad NVIDIA H200 141GB GPU
25. GPU Workloads with Nomad 18GB 18GB 18GB 18GB 18GB 18GB 18GB 1 compute 1 compute 1 compute 1 compute 1 compute 1 compute NVIDIA H200 MIG Pro fi le 18GB 1 compute
26. GPU Workloads with Nomad 18GB 18GB 18GB 18GB 18GB 18GB 18GB 1 compute 1 compute 1 compute 1 compute 1 compute 1 compute NVIDIA H200 MIG Pro fi le 18GB 1 compute
27. GPU Workloads with Nomad 18GB 18GB 18GB 18GB 18GB 18GB 1 compute 1 compute 1 compute 1 compute 1 compute NVIDIA H200 MIG Pro fi le 18GB 1 compute 18GB 1 compute
28. GPU Workloads with Nomad 18GB 18GB 18GB 18GB 18GB 1 compute 1 compute 1 compute 1 compute I H200 18GB 1 compute 18GB 1 compute 18GB 1 compute
29. GPU Workloads with Nomad job "docling" { # other config hidden group "api" { task "docling-serve" { # other config hidden device "nvidia/gpu" { count = 1 } resources { device "nvidia/gpu/NVIDIA H200 MIG 1g.18gb" { count = 1 } } } } } docling.nomad.hcl
30. ©2026 HASHICORP
31. ©2026 HASHICORP Enterprise AI runs on hybrid infrastructure. Your orchestrator should too.
32. github.com/ksatirli/gpu-workloads-on-nomad Get the Code
33. speakerdeck.com/ksatirli Thank you