Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Industrial Mining at Scale

Industrial Mining at Scale

In this presentation, I explain workload orchestration concepts and how they relate to spending more time doing fun things in Minecraft.

This version of the talk was given at the 2023 Edition of All Things Open in October 2023.

---

Companion Code: github.com/workloads

Kerim Satirli

October 17, 2023
Tweet

More Decks by Kerim Satirli

Other Decks in Technology

Transcript

  1. FROM !"platform=linux/amd64 node:18-alpine3.16 LABEL org.opencontainers.image.authors='ksatirli' LABEL org.opencontainers.image.url='github.com/workloads/minecraft-bot' LABEL org.opencontainers.image.description='Excavation Assistant'

    LABEL org.opencontainers.image.licenses='Apache-2.0' LABEL org.opencontainers.image.source='github.com/workloads/minecraft-bot' COPY ./ / WORKDIR / RUN npm install CMD [ "node", "dist/index.js" ] Dockerfile Container Image
  2. job "minecraft" { datacenters = ["aws"] type = "service" namespace

    = "default" priority = 99 group "minecraft" { count = 1 ephemeral_disk { migrate = true size = 1024 sticky = true } network { mode = "host" Minecraft Server server.nomad.hcl
  3. nomad run server.nomad.hcl Terminal Minecraft Server ==> Monitoring evaluation "e614d2f7"

    Evaluation triggered by job "minecraft" Allocation "82aa9f54" created Evaluation status: "complete" ==> Evaluation "e614d2f7" finished.
  4. job "minecraft-bot" { task "minecraft_bot" { driver = "docker" config

    { image = "ghcr.io/workloads/minecraft-bot:v20231010" ports = [ "api", "inventory", ] } env { # APP CONFIGURATION DEBUG = "false" LOG_COLOR = "true" Minecraft Bot bot.nomad.hcl
  5. nomad run bot.nomad.hcl Terminal Multi-Region Deployment ==> Monitoring evaluation "c6118496"

    Evaluation triggered by job "bot" Allocation "8862fb0e" created Allocation "cae76d09" created Allocation "ff1f1972" created Evaluation status: "complete" ==> Evaluation "c6118496" finished.
  6. aws_regions = toset([ "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-southeast-1",

    "ap-southeast-2", "ap-southeast-3", "ca-central-1", "eu-central-1", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "me-south-1", "sa-east-1", Terraform Console Multi-Region Deployment
  7. module "regional_aws_workspaces" { for_each = toset(data.aws_regions.main.names) source = "ksatirli/regional-workspace/tfe" version

    = "1.2.0" region = { category = "terraform" identifier = each.key prefix = "aws" variable = "aws_region" } vcs_repo = { identifier = "workloads/regional-aws-deployment" branch = "main" } } regional_deployments.tf Multi-Region Deployment
  8. deployed an army of bots to do our bidding learned

    about Cloud Geography provisioned resilient infrastructure built a bot and containerized it Recap