Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gaming at Scale

Gaming at Scale

In this presentation, I explain workload orchestration and how a good strategy can help you deal with regional- and provider-wide failures,.

This version of the talk was given at the 2023 Edition of Scale by the Bay in November 2023.

---

Companion Code: github.com/workloads

Kerim Satirli

November 15, 2023
Tweet

More Decks by Kerim Satirli

Other Decks in Programming

Transcript

  1. Sr. Developer Advocate at HashiCorp
    he / him
    @ksatirli
    Kerim
    Satirli

    View full-size slide

  2. Challenge:
    Build Collaboratively

    View full-size slide

  3. conversational interface
    Laws of Robotics (sort of)
    observability

    View full-size slide

  4. FROM !"platform=linux/amd64 node:18-alpine3.17
    LABEL org.opencontainers.image.authors='ksatirli'
    LABEL org.opencontainers.image.url='github.com/workloads/minecraft-bot'
    LABEL org.opencontainers.image.description='Excavation Assistant'
    LABEL org.opencontainers.image.licenses='Apache-2.0'
    LABEL org.opencontainers.image.source='github.com/workloads/minecraft-bot'
    COPY ./ /
    WORKDIR /
    RUN npm install
    CMD [ "node", "dist/index.js" ]
    Dockerfile
    Container Image

    View full-size slide

  5. AWS: us-west-2

    View full-size slide

  6. job "minecraft_server" {
    datacenters = ["aws"]
    type = "service"
    namespace = "default"
    priority = 99
    group "minecraft" {
    count = 1
    ephemeral_disk {
    migrate = true
    size = 1024
    sticky = true
    }
    network {
    mode = "host"
    Minecraft Server
    server.nomad.hcl

    View full-size slide

  7. nomad run server.nomad.hcl
    Terminal
    Minecraft Server
    ==> Monitoring evaluation "e614d2f7"
    Evaluation triggered by job "minecraft_server"
    Allocation "82aa9f54" created
    Evaluation status: "complete"
    ==> Evaluation "e614d2f7" finished.

    View full-size slide

  8. nomad-leader.svcs.dev:4646/ui/jobs
    Minecraft Server

    View full-size slide

  9. nomad-leader.svcs.dev:4646/ui/jobs/minecraft_server@default
    Minecraft Server

    View full-size slide

  10. nomad run bot.nomad.hcl
    Terminal
    Multi-Region Deployment
    ==> Monitoring evaluation "c6118496"
    Evaluation triggered by job "minecraft_bot"
    Allocation "8862fb0e" created
    Allocation "cae76d09" created
    Allocation "ff1f1972" created
    Evaluation status: "complete"
    ==> Evaluation "c6118496" finished.

    View full-size slide

  11. nomad-leader.svcs.dev:4646/ui/jobs/minecraft_bot@default
    Minecraft Bot and Server

    View full-size slide

  12. AWS: us-*
    AZ: *eu GCP: asia-*
    Multi-Cloud Deployment

    View full-size slide

  13. aws_regions = toset([
    "af-south-1",
    "ap-east-1",
    "ap-northeast-1",
    "ap-northeast-2",
    "ap-northeast-3",
    "ap-south-1",
    "ap-southeast-1",
    "ap-southeast-2",
    "ap-southeast-3",
    "ca-central-1",
    "eu-central-1",
    "eu-north-1",
    "eu-south-1",
    "eu-south-2",
    "eu-west-1",
    "eu-west-2",
    "eu-west-3",
    "me-south-1",
    "sa-east-1",
    Terraform Console
    Multi-Region Deployment

    View full-size slide

  14. module "regional_aws_workspaces" {
    for_each = toset(data.aws_regions.main.names)
    source = "ksatirli/regional-workspace/tfe"
    version = "1.2.0"
    region = {
    category = "terraform"
    identifier = each.key
    prefix = "aws"
    variable = "aws_region"
    }
    vcs_repo = {
    identifier = "workloads/regional-aws-deployment"
    branch = "main"
    }
    }
    regional_deployments.tf
    Multi-Region Deployment

    View full-size slide

  15. module "regional_aws_workspaces" {
    for_each = toset(data.aws_regions.main.names)
    source = "ksatirli/regional-workspace/tfe"
    version = "1.2.0"
    region = {
    category = "terraform"
    identifier = each.key
    prefix = "aws"
    variable = "aws_region"
    }
    vcs_repo = {
    identifier = "workloads/regional-aws-deployment"
    branch = "inactive"
    }
    }
    regional_deployments.tf
    Multi-Region Deployment

    View full-size slide

  16. variable "cidr_map" {
    type = object({
    geo = map(number)
    cardinality = map(number)
    })
    description = "Mapping of CIDR data by AWS Region."
    default = {
    geo = {
    # Asia Central
    ac = 22
    # Africa
    af = 23
    # Asia Pacific
    ap = 27
    Multi-Region Networking
    regional_cidrs.tf

    View full-size slide

  17. app.terraform.io/app/workloads/workspaces
    Multi-Region Deployment

    View full-size slide

  18. This talk wasn't
    about Minecraft.

    View full-size slide

  19. This talk was
    about fault tolerance.

    View full-size slide

  20. Take-aways
    ▪ train like you play - no manual !
    ▪ codify infrastructure and processes
    ▪ architect for awareness, don't depend on it

    View full-size slide

  21. Demo Code
    github.com/workloads/

    View full-size slide

  22. speakerdeck.com/ksatirli

    View full-size slide