Industrial Mining at Scale

In this presentation, I explain workload orchestration concepts and how they relate to spending more time doing fun things in Minecraft.

This version of the talk was given at the 2023 Edition of All Things Open in October 2023.

---

Companion Code: github.com/workloads

Transcript

1. Industrial Mining at Scale (A Minecraft Story)
2. Industrial Mining at Scale (A Minecraft Story)
3. Sr. Developer Advocate at HashiCorp he / him @ksatirli Kerim Satirli
4. Aletta
6. Challenge: Dig less, build more.
7. Let's build a Minecraft Bot
8. chat interface and API (2 of 3) Laws of Robotics health endpoint
10. FROM !"platform=linux/amd64 node:18-alpine3.16 LABEL org.opencontainers.image.authors='ksatirli' LABEL org.opencontainers.image.url='github.com/workloads/minecraft-bot' LABEL org.opencontainers.image.description='Excavation Assistant' LABEL org.opencontainers.image.licenses='Apache-2.0' LABEL org.opencontainers.image.source='github.com/workloads/minecraft-bot' COPY ./ / WORKDIR / RUN npm install CMD [ "node", "dist/index.js" ] Dockerfile Container Image
11. AWS: us-west-2
12. job "minecraft" { datacenters = ["aws"] type = "service" namespace = "default" priority = 99 group "minecraft" { count = 1 ephemeral_disk { migrate = true size = 1024 sticky = true } network { mode = "host" Minecraft Server server.nomad.hcl
13. nomad run server.nomad.hcl Terminal Minecraft Server ==> Monitoring evaluation "e614d2f7" Evaluation triggered by job "minecraft" Allocation "82aa9f54" created Evaluation status: "complete" ==> Evaluation "e614d2f7" finished.
14. nomad-leader.svcs.dev:4646 Minecraft Server
15. Demo
16. Let's deploy a Minecraft Bot
17. job "minecraft-bot" { task "minecraft_bot" { driver = "docker" config { image = "ghcr.io/workloads/minecraft-bot:v20231010" ports = [ "api", "inventory", ] } env { # APP CONFIGURATION DEBUG = "false" LOG_COLOR = "true" Minecraft Bot bot.nomad.hcl
18. nomad-leader.svcs.dev:4646 Minecraft Bot
19. nomad run bot.nomad.hcl Terminal Multi-Region Deployment ==> Monitoring evaluation "c6118496" Evaluation triggered by job "bot" Allocation "8862fb0e" created Allocation "cae76d09" created Allocation "ff1f1972" created Evaluation status: "complete" ==> Evaluation "c6118496" finished.
20. EMEA AMER APAC Multi-Region Deployment
21. data "aws_regions" "main" {} regional_deployments.tf Multi-Region Deployment
22. aws_regions = toset([ "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ca-central-1", "eu-central-1", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "me-south-1", "sa-east-1", Terraform Console Multi-Region Deployment
23. module "regional_aws_workspaces" { for_each = toset(data.aws_regions.main.names) source = "ksatirli/regional-workspace/tfe" version = "1.2.0" region = { category = "terraform" identifier = each.key prefix = "aws" variable = "aws_region" } vcs_repo = { identifier = "workloads/regional-aws-deployment" branch = "main" } } regional_deployments.tf Multi-Region Deployment
24. Multi-Region Deployment
25. console.aws.amazon.com/support Multi-Region Deployment
26. ✉ from AWS Support
27. app.terraform.io/app/workloads/workspaces Multi-Region Deployment
28. Demo
29. deployed an army of bots to do our bidding learned about Cloud Geography provisioned resilient infrastructure built a bot and containerized it Recap
30. Demo Code github.com/workloads/
31. speakerdeck.com/ksatirli