Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Инфраструктура как TypeScript

Инфраструктура как TypeScript

Термин “Инфраструктура как код” известен уже многие годы, однако сотни инструментов скрывают свой секрет… в них нет настоящего “кода”! Мало кому нравится “программировать” YAML или JSON, и даже меньшему числу людей по душе пытаться втиснуть практики из мира программной разработки в инструменты, сопротивляющиеся этому любой ценой. А что, если программирование инфраструктуры было бы больше похоже на любое другое программирование с использованием настоящих языков, таких как TypeScript? Что, если бы AWS Lambda функции определялись как обычные функции, другие абстракции определялись бы с помощью системы типов, а в набор инструментов входили бы привычные нам модули, рефакторинг, линтеры, и тесты?
В этом докладе я познакомлю вас с Pulumi, движком с открытым кодом, который позволяет делать всё это с помощью TypeScript. Мы рассмотрим примеры кода для создания традиционной облачной инфраструктуры, кластеров Kubernetes, “serverless” приложений. А также посмотрим, как расширить своё приложение на несколько регионов или даже на “облака” нескольких провайдеров.

Mikhail Shilkov

November 20, 2019
Tweet

More Decks by Mikhail Shilkov

Other Decks in Programming

Transcript

  1. Инфраструктура
    как TypeScript

    View Slide

  2. План

    View Slide

  3. План

    View Slide

  4. План

    View Slide

  5. Автоматизация
    Cloud
    3

    View Slide

  6. API управления ресурсами

    View Slide

  7. Сервисы HTTP/REST
    Content-Type: application/x-www-form-urlencoded; charset=UTF-8
    X-Amz-Date: 20130813T150211Z
    Host: ec2.amazonaws.com
    Authorization: AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20130813/us-east-
    1/ec2/aws4_request, SignedHeaders=content-type;host;x-amz-
    date, Signature=ced6826de92d2bdeed8f846f0bf508e8559e98e4b0194b84example54174deb456c
    http://ec2.amazonaws.com/?Action=RunInstances
    ImageId=ami-2bb65342
    &MaxCount=3
    &MinCount=1
    &Monitoring.Enabled=true
    &Placement.AvailabilityZone=us-east-1a
    &Version=2016-11-15

    View Slide

  8. var launchRequest = new RunInstancesRequest()
    {
    ImageId = amiID,
    InstanceType = "t1.micro",
    MinCount = 1,
    MaxCount = 1,
    KeyName = keyPairName,
    SecurityGroupIds = groups
    };
    var launchResponse = ec2Client.RunInstances(launchRequest);
    var instances = launchResponse.Reservation.Instances;
    SDK для вызова REST сервисов

    View Slide

  9. az storage account create \
    --location westus \
    --name samplesa \
    --resource-group myrg \
    --sku LRS
    Командная строка

    View Slide

  10. Сложности со скриптами

    View Slide

  11. Desired State
    Configuration
    6

    View Slide

  12. Описание целевого состояния

    View Slide

  13. Управление графами ресурсов

    View Slide

  14. Характеристики

    View Slide

  15. Инструменты от вендоров

    View Slide

  16. Языки разметки
    9
    Yet
    Another
    Markup
    Language

    View Slide

  17. Azure ARM Templates: JSON
    "resources": [
    {
    "apiVersion": "2016-01-01",
    "type": "Microsoft.Storage/storageAccounts",
    "name": "mystorageaccount",
    "location": "westus",
    "sku": {
    "name": "Standard_LRS"
    },
    "kind": "Storage",
    "properties": {
    }
    }
    ]

    View Slide

  18. AWS CloudFormation: YAML
    Resources:
    S3BucketForURLs:
    Type: "AWS::S3::Bucket"
    DeletionPolicy: Delete
    Properties:
    BucketName: !If [ "CreateNewBucket", !Ref "AWS::NoValue", !Ref S3BucketName ]
    WebsiteConfiguration:
    IndexDocument: "index.html"
    LifecycleConfiguration:
    Rules:
    -
    Id: DisposeShortUrls
    ExpirationInDays: !Ref URLExpiration
    Prefix: "u"
    Status: Enabled

    View Slide

  19. resource "azurerm_storage_account" "sa" {
    name = "mysa"
    location = azurerm_resource_group.rg.location
    resource_group_name = azurerm_resource_group.rg.name
    account_tier = "Standard"
    account_replication_type = "LRS"
    }
    Terraform:
    HashiCorp Configuration Language

    View Slide

  20. Языки разметки

    View Slide

  21. Пример: Соотношение кода и разметки
    KV Store
    Table “URLs”
    Function
    “Add URL”
    Function
    “Open URL”

    View Slide

  22. {
    "Resources": {
    "InstanceSecurityGroup": {
    "Type": "AWS::EC2::SecurityGroup",
    "Properties": {
    "SecurityGroupIngress": [ ... ]
    }
    },
    "EC2Instance": {
    "Type": "AWS::EC2::Instance",
    "Properties": {
    "InstanceType": "t2.micro",
    "SecurityGroups": [ { "Ref": "InstanceSecurityGroup" } ],
    "ImageId": "ami-0080e4c5bc078760e"
    }
    }
    }
    }
    Ссылки на ресурсы

    View Slide

  23. {
    "condition": "[equals(parameters('production'), 'Yes')]",
    "type": "Microsoft.Compute/availabilitySets",
    "apiVersion": "2017-03-30",
    "name": "[variables('availabilitySetName')]",
    "location": "[resourceGroup().location]",
    "properties": {
    "platformFaultDomainCount": 2,
    "platformUpdateDomainCount": 3
    }
    }
    Условия

    View Slide

  24. locals {
    files = ["index.php","main.php"]
    }
    resource "aws_s3_bucket_object" "files" {
    count = "${length(local.files)}"
    key = "${local.files[count.index]}"
    bucket = "${aws_s3_bucket.examplebucket.id}"
    source = "./src/${local.files[count.index]}"
    }
    Циклы

    View Slide

  25. Повторное использование

    View Slide

  26. Тестирование

    View Slide

  27. Кажется,
    мы это
    уже
    когда-то
    делали?

    View Slide

  28. IDEs
    Code completion
    Unit tests
    Types
    Functions
    Package managers
    Versioning
    Classes
    Code reviews
    Contracts
    SDKs
    Workflows
    Refactoring

    View Slide

  29. Pulumi
    14

    View Slide

  30. Универсальные языки программирования

    View Slide






  31. Провайдеры





    View Slide

  32. const resourceGroup = new azure.core.ResourceGroup("rg");
    const storageAccount = new azure.storage.Account("storage", {
    resourceGroupName: resourceGroup.name,
    accountReplicationType: "LRS",
    accountTier: "Standard",
    });
    Пример TypeScript

    View Slide

  33. const resourceGroup = new azure.core.ResourceGroup("rg");
    const storageAccount = new azure.storage.Account("storage", {
    resourceGroupName: resourceGroup.name,
    accountReplicationType: "LRS",
    accountTier: "Standard",
    });
    Desired
    State

    View Slide

  34. Как
    работает
    Pulumi

    View Slide

  35. Static Website
    on AWS
    17

    View Slide

  36. N провайдеров, M языков

    View Slide

  37. N провайдеров, M языков

    View Slide

  38. Сложность O(N*M)

    View Slide

  39. Сложность O(N+M)

    View Slide

  40. Сложность O(N+M)

    View Slide

  41. Управление
    состоянием

    View Slide

  42. Serverless
    Functions
    в Azure
    22

    View Slide

  43. Ресурсы, нужные для Azure Function

    View Slide

  44. // A resource group to contain our Azure Functions
    const resourceGroup = new azure.core.ResourceGroup("rg");
    // A Node.js Azure Function
    const nodeApp = new azure.appservice.ArchiveFunctionApp("http", {
    resourceGroup,
    archive: new pulumi.asset.FileArchive("./javascript"),
    });
    export const nodeEndpoint = nodeApp.endpoint;
    Node.js Azure Function

    View Slide

  45. Запустим pulumi up …
    Type Name Plan
    + pulumi:pulumi:Stack functions-dev create
    + ├─ azure:appservice:ArchiveFunctionApp http create
    + │ ├─ azure:storage:Account http create
    + │ ├─ azure:appservice:Plan http create
    + │ ├─ azure:storage:Container http create
    + │ ├─ azure:storage:ZipBlob http create
    + │ └─ azure:appservice:FunctionApp http create
    + └─ azure:core:ResourceGroup rg create

    View Slide

  46. // A .NET (C#) Azure Function
    const dotnetApp = new azure.appservice.ArchiveFunctionApp("dotnet", {
    resourceGroup,
    archive: new pulumi.asset.FileArchive("./dotnet/publish/"),
    appSettings: {
    runtime: "dotnet",
    },
    });
    .NET Azure Function

    View Slide

  47. const linuxResourceGroup = new azure.core.ResourceGroup("linuxrg");
    const linuxPlan = new azure.appservice.Plan("linux-asp", {
    resourceGroupName: linuxResourceGroup.name,
    kind: "Linux",
    sku: { tier: "Dynamic", size: "Y1" },
    reserved: true,
    });
    const pythonApp = new azure.appservice.ArchiveFunctionApp(...);
    Python Azure Function on Linux

    View Slide

  48. const linuxResourceGroup = new azure.core.ResourceGroup("linuxrg");
    const linuxPlan = new azure.appservice.Plan("linux-asp", {...});
    const pythonApp = new azure.appservice.ArchiveFunctionApp("http", {
    resourceGroup: linuxResourceGroup,
    plan: linuxPlan,
    archive: new pulumi.asset.FileArchive("./python"),
    appSettings: {
    runtime: "python",
    },
    });
    Python Azure Function on Linux

    View Slide

  49. const f = new azure.appservice.HttpEventSubscription("greeting", {
    resourceGroup,
    callback: async (context, req) => {
    return {
    status: 200,
    body: `Hello ${req.params.name}!`,
    };
    },
    });
    Callbacks => Serverless Functions

    View Slide

  50. const queue = new azure.storage.Queue("myqueue", {
    storageAccountName: storageAccount.name,
    });
    queue.onEvent("MyMessageHandler", {
    callback: async (context, msg) => {
    doWork(msg);
    },
    });
    Callbacks => Serverless Functions

    View Slide

  51. Containers
    27

    View Slide

  52. Crosswalk for AWS

    View Slide

  53. Load-
    balanced
    container
    instances
    Load
    Balancer
    Container
    Container

    View Slide

  54. const listener = new awsx.elasticloadbalancingv2.NetworkListener("lb",{
    port: 80 });
    const service = new awsx.ecs.FargateService("nginx", {
    desiredCount: 2,
    taskDefinitionArgs: {
    containers: {
    nginx: {
    image: "jonashackt/spring-boot-vuejs",
    memory: 512,
    portMappings: [listener],
    }}}});
    Nginx + Fargate + Load Balancer

    View Slide

  55. Запустим pulumi up …
    Type Name Status
    + pulumi:pulumi:Stack pulumi-typescript-aws-fargate-dev created
    + ├─ awsx:x:ec2:SecurityGroup nginx created
    + │ └─ aws:ec2:SecurityGroup nginx created
    + ├─ awsx:x:ec2:Vpc default-vpc-accd9bc5 created
    + │ ├─ awsx:x:ec2:Subnet default-vpc-accd9bc5-public-1 created
    + │ └─ awsx:x:ec2:Subnet default-vpc-accd9bc5-public-0 created
    + ├─ awsx:x:ecs:FargateService spring-boot-vuejs created
    + │ └─ aws:ecs:Service spring-boot-vuejs created
    + ├─ awsx:x:ecs:FargateTaskDefinition spring-boot-vuejs created
    + │ ├─ aws:cloudwatch:LogGroup spring-boot-vuejs created
    + │ ├─ aws:iam:Role spring-boot-vuejs-execution created
    + │ ├─ aws:iam:Role spring-boot-vuejs-task created
    + │ ├─ aws:iam:RolePolicyAttachment spring-boot-vuejs-execution-9a42f520 created
    + │ ├─ aws:iam:RolePolicyAttachment spring-boot-vuejs-task-32be53a2 created
    + │ ├─ aws:iam:RolePolicyAttachment spring-boot-vuejs-task-fd1a00e5 created
    + │ └─ aws:ecs:TaskDefinition spring-boot-vuejs created
    + ├─ awsx:x:ecs:Cluster default-cluster created
    + │ ├─ awsx:x:ec2:SecurityGroup default-cluster created
    + │ │ ├─ awsx:x:ec2:IngressSecurityGroupRule default-cluster-ssh created
    + │ │ │ └─ aws:ec2:SecurityGroupRule default-cluster-ssh created
    + │ │ ├─ awsx:x:ec2:IngressSecurityGroupRule default-cluster-containers created
    + │ │ │ └─ aws:ec2:SecurityGroupRule default-cluster-containers created
    + │ │ ├─ awsx:x:ec2:EgressSecurityGroupRule default-cluster-egress created
    + │ │ │ └─ aws:ec2:SecurityGroupRule default-cluster-egress created
    + │ │ └─ aws:ec2:SecurityGroup default-cluster created
    + │ └─ aws:ecs:Cluster default-cluster created
    + └─ aws:lb:ApplicationLoadBalancer nginx created
    + ├─ awsx:lb:ApplicationTargetGroup nginx created
    + │ └─ aws:lb:TargetGroup nginx created
    + ├─ awsx:lb:ApplicationListener nginx created
    + │ ├─ awsx:x:ec2:IngressSecurityGroupRule nginx-external-0-ingress created
    + │ │ └─ aws:ec2:SecurityGroupRule nginx-external-0-ingress created
    + │ ├─ awsx:x:ec2:EgressSecurityGroupRule nginx-external-0-egress created
    + │ │ └─ aws:ec2:SecurityGroupRule nginx-external-0-egress created
    + │ └─ aws:lb:Listener nginx created
    + └─ aws:lb:LoadBalancer nginx created

    View Slide

  56. Cosmos App
    30

    View Slide

  57. Cosmos App
    (1)

    View Slide

  58. Cosmos App
    (2)

    View Slide

  59. Cosmos App
    (3)

    View Slide

  60. export const functions = new CosmosApp("urls", {
    resourceGroup,
    locations: ["WestEurope", "WestUS", "SouthEastAsia"],
    factory: ({location, cosmosdb}) => {
    const app = new azure.appservice.ArchiveFunctionApp("app", {
    location,
    archive: new pulumi.asset.FileArchive("./app"),
    appSettings: {
    COSMOSDB_ENDPOINT: cosmosdb.endpoint,
    },
    });
    return { id: app.functionApp.id };
    }});
    Глобальные приложения

    View Slide

  61. export const functions = new CosmosApp("urls", {
    resourceGroup,
    locations: ["WestEurope", "WestUS", "SouthEastAsia"],
    factory: ({location, cosmosdb}) => {
    const app = new azure.appservice.ArchiveFunctionApp("app", {
    location,
    archive: new pulumi.asset.FileArchive("./app"),
    appSettings: {
    COSMOSDB_ENDPOINT: cosmosdb.endpoint,
    },
    });
    return { id: app.functionApp.id };
    }});
    Глобальные приложения

    View Slide

  62. Kubernetes
    33

    View Slide

  63. class AksCluster extends pulumi.ComponentResource {
    constructor(name: string,
    args: AksClusterArgs,
    opts: pulumi.ComponentResourceOptions = {}) {
    super("examples:aks:AksCluster", name, args, opts);
    const password = new random.RandomPassword("password"/*, {...}*/);
    const sshPublicKey = new tls.PrivateKey("key"/*, {...}*/);
    const adApp = new azuread.Application("aks"/*, {...}*/);
    const adSp = new azuread.ServicePrincipal("aksSp"/*, {...}*/);
    const vnet = new azure.network.VirtualNetwork("vnet"/*, {...}*/);
    const subnet = new azure.network.Subnet("subnet"/*, {...}*/);
    const cluster = new azure.containerservice.KubernetesCluster("aksCluster"/*, {...}*/)
    }
    }
    Managed Clusters

    View Slide

  64. Crosswalk
    for
    Kubernetes

    View Slide

  65. // Define the application Pod.
    const appPod = new kx.PodBuilder({
    containers: [{
    image: "app:1.0.0",
    env: { "APP_CONFIG_PATH": "/app/config" },
    volumeMounts: [configs.mount("/app/config")],
    }],
    });
    // Create a Kubernetes Deployment using the previous Pod definition.
    const deployment = new kx.Deployment("nginx", {
    spec: appPod.asDeploymentSpec({ replicas: 3 }),
    });
    // Expose the Deployment with a load balancer using a Kubernetes Service.
    const service = deployment.createService({
    type: kx.types.ServiceType.LoadBalancer,
    });
    Библиотека Kx

    View Slide

  66. // Find all distinct versions of MySQL running in your cluster.
    const mySqlVersions = kq
    .list("v1", "Pod")
    .flatMap(pod => pod.spec.containers)
    .map(container => container.image)
    .filter(imageName => imageName.includes("mysql"))
    .distinct();
    mySqlVersions.forEach(console.log);
    Запросы

    View Slide

  67. Контроль
    качества
    36

    View Slide

  68. Языки
    программирования
    для
    инфраструктуры?
    Точно?

    View Slide

  69. const group = require("./index").group;
    // check: Instances must not have SSH open to the Internet.
    it("must not open port 22 (SSH) to the Internet", done => {
    pulumi.all([ group.urn, group.ingress ]).apply(([ urn, ingress ]) => {
    if (ingress.find(rule =>
    rule.fromPort == 22 && rule.cidrBlocks.find(block => block === "0.0.0.0/0"))) {
    done(new Error(`Illegal SSH port 22 open to the Internet on group ${urn}`));
    } else {
    done();
    }
    });
    });
    Unit Testing

    View Slide

  70. const policies = new PolicyPack("aws", {
    policies: [
    {
    name: "prohibited-public-internet",
    description: "Ingress rules with public internet access are prohibited.",
    enforcementLevel: "mandatory",
    validateResource: validateTypedResource(aws.ec2.SecurityGroup, (sg, args, report) => {
    const publicInternetRules = (sg.ingress || []).find(ingressRule =>
    (ingressRule.cidrBlocks || []).find(cidr => cidr === "0.0.0.0/0"));
    if (publicInternetRules) {
    report("Ingress rules with public internet access are prohibited.");
    }
    }),
    },
    ],
    });
    Policy as Code

    View Slide

  71. AWS Cloud
    Development Kit
    (CDK)?
    39

    View Slide

  72. м
    TypeScript
    Python
    Java
    .NET
    Транслируется в
    CloudFormation

    View Slide

  73. const hello = new lambda.Function(this, 'HelloHandler', {
    runtime: lambda.Runtime.NODEJS_8_10,
    code: lambda.Code.asset('lambda'),
    handler: 'hello.handler'
    });
    new apigw.LambdaRestApi(this, 'Endpoint', {
    handler: hello
    });
    Пример AWS CDK

    View Slide

  74. Примеры из моей презентации
    ✓ ✓
    ✓ 
    ✓ ✓
    ✓ 
    ✓ 
    ✓ 

    View Slide

  75. Выводы
    40

    View Slide

  76. Используйте
    Инфраструктуру как
    код
    Создаёте
    приложения
    для cloud?

    View Slide

  77. Меньшие
    компоненты
    Лучшая
    автоматизация

    View Slide

  78. Преимущества кода

    View Slide

  79. м
    Инфраструктура
    как код
    Архитектура
    как код

    View Slide

  80. View Slide