Инфраструктура как TypeScript

Термин “Инфраструктура как код” известен уже многие годы, однако сотни инструментов скрывают свой секрет… в них нет настоящего “кода”! Мало кому нравится “программировать” YAML или JSON, и даже меньшему числу людей по душе пытаться втиснуть практики из мира программной разработки в инструменты, сопротивляющиеся этому любой ценой. А что, если программирование инфраструктуры было бы больше похоже на любое другое программирование с использованием настоящих языков, таких как TypeScript? Что, если бы AWS Lambda функции определялись как обычные функции, другие абстракции определялись бы с помощью системы типов, а в набор инструментов входили бы привычные нам модули, рефакторинг, линтеры, и тесты?
В этом докладе я познакомлю вас с Pulumi, движком с открытым кодом, который позволяет делать всё это с помощью TypeScript. Мы рассмотрим примеры кода для создания традиционной облачной инфраструктуры, кластеров Kubernetes, “serverless” приложений. А также посмотрим, как расширить своё приложение на несколько регионов или даже на “облака” нескольких провайдеров.

Transcript

1. Инфраструктура
   как TypeScript
   

   View Slide

2. План
   

   View Slide

3. План
   

   View Slide

4. План
   

   View Slide

5. Автоматизация
   Cloud
   3
   

   View Slide

6. API управления ресурсами
   

   View Slide

7. Сервисы HTTP/REST
   Content-Type: application/x-www-form-urlencoded; charset=UTF-8
   X-Amz-Date: 20130813T150211Z
   Host: ec2.amazonaws.com
   Authorization: AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20130813/us-east-
   1/ec2/aws4_request, SignedHeaders=content-type;host;x-amz-
   date, Signature=ced6826de92d2bdeed8f846f0bf508e8559e98e4b0194b84example54174deb456c
   http://ec2.amazonaws.com/?Action=RunInstances
   ImageId=ami-2bb65342
   &MaxCount=3
   &MinCount=1
   &Monitoring.Enabled=true
   &Placement.AvailabilityZone=us-east-1a
   &Version=2016-11-15
   

   View Slide

8. var launchRequest = new RunInstancesRequest()
   {
   ImageId = amiID,
   InstanceType = "t1.micro",
   MinCount = 1,
   MaxCount = 1,
   KeyName = keyPairName,
   SecurityGroupIds = groups
   };
   var launchResponse = ec2Client.RunInstances(launchRequest);
   var instances = launchResponse.Reservation.Instances;
   SDK для вызова REST сервисов
   

   View Slide

9. az storage account create \
   --location westus \
   --name samplesa \
   --resource-group myrg \
   --sku LRS
   Командная строка
   

   View Slide

10. Сложности со скриптами
    

    View Slide

11. Desired State
    Configuration
    6
    

    View Slide

12. Описание целевого состояния
    

    View Slide

13. Управление графами ресурсов
    

    View Slide

14. Характеристики
    

    View Slide

15. Инструменты от вендоров
    

    View Slide

16. Языки разметки
    9
    Yet
    Another
    Markup
    Language
    

    View Slide

17. Azure ARM Templates: JSON
    "resources": [
    {
    "apiVersion": "2016-01-01",
    "type": "Microsoft.Storage/storageAccounts",
    "name": "mystorageaccount",
    "location": "westus",
    "sku": {
    "name": "Standard_LRS"
    },
    "kind": "Storage",
    "properties": {
    }
    }
    ]
    

    View Slide

18. AWS CloudFormation: YAML
    Resources:
    S3BucketForURLs:
    Type: "AWS::S3::Bucket"
    DeletionPolicy: Delete
    Properties:
    BucketName: !If [ "CreateNewBucket", !Ref "AWS::NoValue", !Ref S3BucketName ]
    WebsiteConfiguration:
    IndexDocument: "index.html"
    LifecycleConfiguration:
    Rules:
    -
    Id: DisposeShortUrls
    ExpirationInDays: !Ref URLExpiration
    Prefix: "u"
    Status: Enabled
    

    View Slide

19. resource "azurerm_storage_account" "sa" {
    name = "mysa"
    location = azurerm_resource_group.rg.location
    resource_group_name = azurerm_resource_group.rg.name
    account_tier = "Standard"
    account_replication_type = "LRS"
    }
    Terraform:
    HashiCorp Configuration Language
    

    View Slide

20. Языки разметки
    

    View Slide

21. Пример: Соотношение кода и разметки
    KV Store
    Table “URLs”
    Function
    “Add URL”
    Function
    “Open URL”
    

    View Slide

22. {
    "Resources": {
    "InstanceSecurityGroup": {
    "Type": "AWS::EC2::SecurityGroup",
    "Properties": {
    "SecurityGroupIngress": [ ... ]
    }
    },
    "EC2Instance": {
    "Type": "AWS::EC2::Instance",
    "Properties": {
    "InstanceType": "t2.micro",
    "SecurityGroups": [ { "Ref": "InstanceSecurityGroup" } ],
    "ImageId": "ami-0080e4c5bc078760e"
    }
    }
    }
    }
    Ссылки на ресурсы
    

    View Slide

23. {
    "condition": "[equals(parameters('production'), 'Yes')]",
    "type": "Microsoft.Compute/availabilitySets",
    "apiVersion": "2017-03-30",
    "name": "[variables('availabilitySetName')]",
    "location": "[resourceGroup().location]",
    "properties": {
    "platformFaultDomainCount": 2,
    "platformUpdateDomainCount": 3
    }
    }
    Условия
    

    View Slide

24. locals {
    files = ["index.php","main.php"]
    }
    resource "aws_s3_bucket_object" "files" {
    count = "${length(local.files)}"
    key = "${local.files[count.index]}"
    bucket = "${aws_s3_bucket.examplebucket.id}"
    source = "./src/${local.files[count.index]}"
    }
    Циклы
    

    View Slide

25. Повторное использование
    

    View Slide

26. Тестирование
    

    View Slide

27. Кажется,
    мы это
    уже
    когда-то
    делали?
    

    View Slide

28. IDEs
    Code completion
    Unit tests
    Types
    Functions
    Package managers
    Versioning
    Classes
    Code reviews
    Contracts
    SDKs
    Workflows
    Refactoring
    

    View Slide

29. Pulumi
    14
    

    View Slide

30. Универсальные языки программирования
    

    View Slide

31. •
    •
    •
    •
    •
    Провайдеры
    •
    •
    •
    •
    •
    

    View Slide

32. const resourceGroup = new azure.core.ResourceGroup("rg");
    const storageAccount = new azure.storage.Account("storage", {
    resourceGroupName: resourceGroup.name,
    accountReplicationType: "LRS",
    accountTier: "Standard",
    });
    Пример TypeScript
    

    View Slide

33. const resourceGroup = new azure.core.ResourceGroup("rg");
    const storageAccount = new azure.storage.Account("storage", {
    resourceGroupName: resourceGroup.name,
    accountReplicationType: "LRS",
    accountTier: "Standard",
    });
    Desired
    State
    

    View Slide

34. Как
    работает
    Pulumi
    

    View Slide

35. Static Website
    on AWS
    17
    

    View Slide

36. N провайдеров, M языков
    

    View Slide

37. N провайдеров, M языков
    

    View Slide

38. Сложность O(N*M)
    

    View Slide

39. Сложность O(N+M)
    

    View Slide

40. Сложность O(N+M)
    

    View Slide

41. Управление
    состоянием
    

    View Slide

42. Serverless
    Functions
    в Azure
    22
    

    View Slide

43. Ресурсы, нужные для Azure Function
    

    View Slide

44. // A resource group to contain our Azure Functions
    const resourceGroup = new azure.core.ResourceGroup("rg");
    // A Node.js Azure Function
    const nodeApp = new azure.appservice.ArchiveFunctionApp("http", {
    resourceGroup,
    archive: new pulumi.asset.FileArchive("./javascript"),
    });
    export const nodeEndpoint = nodeApp.endpoint;
    Node.js Azure Function
    

    View Slide

45. Запустим pulumi up …
    Type Name Plan
    + pulumi:pulumi:Stack functions-dev create
    + ├─ azure:appservice:ArchiveFunctionApp http create
    + │ ├─ azure:storage:Account http create
    + │ ├─ azure:appservice:Plan http create
    + │ ├─ azure:storage:Container http create
    + │ ├─ azure:storage:ZipBlob http create
    + │ └─ azure:appservice:FunctionApp http create
    + └─ azure:core:ResourceGroup rg create
    

    View Slide

46. // A .NET (C#) Azure Function
    const dotnetApp = new azure.appservice.ArchiveFunctionApp("dotnet", {
    resourceGroup,
    archive: new pulumi.asset.FileArchive("./dotnet/publish/"),
    appSettings: {
    runtime: "dotnet",
    },
    });
    .NET Azure Function
    

    View Slide

47. const linuxResourceGroup = new azure.core.ResourceGroup("linuxrg");
    const linuxPlan = new azure.appservice.Plan("linux-asp", {
    resourceGroupName: linuxResourceGroup.name,
    kind: "Linux",
    sku: { tier: "Dynamic", size: "Y1" },
    reserved: true,
    });
    const pythonApp = new azure.appservice.ArchiveFunctionApp(...);
    Python Azure Function on Linux
    

    View Slide

48. const linuxResourceGroup = new azure.core.ResourceGroup("linuxrg");
    const linuxPlan = new azure.appservice.Plan("linux-asp", {...});
    const pythonApp = new azure.appservice.ArchiveFunctionApp("http", {
    resourceGroup: linuxResourceGroup,
    plan: linuxPlan,
    archive: new pulumi.asset.FileArchive("./python"),
    appSettings: {
    runtime: "python",
    },
    });
    Python Azure Function on Linux
    

    View Slide

49. const f = new azure.appservice.HttpEventSubscription("greeting", {
    resourceGroup,
    callback: async (context, req) => {
    return {
    status: 200,
    body: `Hello ${req.params.name}!`,
    };
    },
    });
    Callbacks => Serverless Functions
    

    View Slide

50. const queue = new azure.storage.Queue("myqueue", {
    storageAccountName: storageAccount.name,
    });
    queue.onEvent("MyMessageHandler", {
    callback: async (context, msg) => {
    doWork(msg);
    },
    });
    Callbacks => Serverless Functions
    

    View Slide

51. Containers
    27
    

    View Slide

52. Crosswalk for AWS
    

    View Slide

53. Load-
    balanced
    container
    instances
    Load
    Balancer
    Container
    Container
    

    View Slide

54. const listener = new awsx.elasticloadbalancingv2.NetworkListener("lb",{
    port: 80 });
    const service = new awsx.ecs.FargateService("nginx", {
    desiredCount: 2,
    taskDefinitionArgs: {
    containers: {
    nginx: {
    image: "jonashackt/spring-boot-vuejs",
    memory: 512,
    portMappings: [listener],
    }}}});
    Nginx + Fargate + Load Balancer
    

    View Slide

55. Запустим pulumi up …
    Type Name Status
    + pulumi:pulumi:Stack pulumi-typescript-aws-fargate-dev created
    + ├─ awsx:x:ec2:SecurityGroup nginx created
    + │ └─ aws:ec2:SecurityGroup nginx created
    + ├─ awsx:x:ec2:Vpc default-vpc-accd9bc5 created
    + │ ├─ awsx:x:ec2:Subnet default-vpc-accd9bc5-public-1 created
    + │ └─ awsx:x:ec2:Subnet default-vpc-accd9bc5-public-0 created
    + ├─ awsx:x:ecs:FargateService spring-boot-vuejs created
    + │ └─ aws:ecs:Service spring-boot-vuejs created
    + ├─ awsx:x:ecs:FargateTaskDefinition spring-boot-vuejs created
    + │ ├─ aws:cloudwatch:LogGroup spring-boot-vuejs created
    + │ ├─ aws:iam:Role spring-boot-vuejs-execution created
    + │ ├─ aws:iam:Role spring-boot-vuejs-task created
    + │ ├─ aws:iam:RolePolicyAttachment spring-boot-vuejs-execution-9a42f520 created
    + │ ├─ aws:iam:RolePolicyAttachment spring-boot-vuejs-task-32be53a2 created
    + │ ├─ aws:iam:RolePolicyAttachment spring-boot-vuejs-task-fd1a00e5 created
    + │ └─ aws:ecs:TaskDefinition spring-boot-vuejs created
    + ├─ awsx:x:ecs:Cluster default-cluster created
    + │ ├─ awsx:x:ec2:SecurityGroup default-cluster created
    + │ │ ├─ awsx:x:ec2:IngressSecurityGroupRule default-cluster-ssh created
    + │ │ │ └─ aws:ec2:SecurityGroupRule default-cluster-ssh created
    + │ │ ├─ awsx:x:ec2:IngressSecurityGroupRule default-cluster-containers created
    + │ │ │ └─ aws:ec2:SecurityGroupRule default-cluster-containers created
    + │ │ ├─ awsx:x:ec2:EgressSecurityGroupRule default-cluster-egress created
    + │ │ │ └─ aws:ec2:SecurityGroupRule default-cluster-egress created
    + │ │ └─ aws:ec2:SecurityGroup default-cluster created
    + │ └─ aws:ecs:Cluster default-cluster created
    + └─ aws:lb:ApplicationLoadBalancer nginx created
    + ├─ awsx:lb:ApplicationTargetGroup nginx created
    + │ └─ aws:lb:TargetGroup nginx created
    + ├─ awsx:lb:ApplicationListener nginx created
    + │ ├─ awsx:x:ec2:IngressSecurityGroupRule nginx-external-0-ingress created
    + │ │ └─ aws:ec2:SecurityGroupRule nginx-external-0-ingress created
    + │ ├─ awsx:x:ec2:EgressSecurityGroupRule nginx-external-0-egress created
    + │ │ └─ aws:ec2:SecurityGroupRule nginx-external-0-egress created
    + │ └─ aws:lb:Listener nginx created
    + └─ aws:lb:LoadBalancer nginx created
    

    View Slide

56. Cosmos App
    30
    

    View Slide

57. Cosmos App
    (1)
    

    View Slide

58. Cosmos App
    (2)
    

    View Slide

59. Cosmos App
    (3)
    

    View Slide

60. export const functions = new CosmosApp("urls", {
    resourceGroup,
    locations: ["WestEurope", "WestUS", "SouthEastAsia"],
    factory: ({location, cosmosdb}) => {
    const app = new azure.appservice.ArchiveFunctionApp("app", {
    location,
    archive: new pulumi.asset.FileArchive("./app"),
    appSettings: {
    COSMOSDB_ENDPOINT: cosmosdb.endpoint,
    },
    });
    return { id: app.functionApp.id };
    }});
    Глобальные приложения
    

    View Slide

61. export const functions = new CosmosApp("urls", {
    resourceGroup,
    locations: ["WestEurope", "WestUS", "SouthEastAsia"],
    factory: ({location, cosmosdb}) => {
    const app = new azure.appservice.ArchiveFunctionApp("app", {
    location,
    archive: new pulumi.asset.FileArchive("./app"),
    appSettings: {
    COSMOSDB_ENDPOINT: cosmosdb.endpoint,
    },
    });
    return { id: app.functionApp.id };
    }});
    Глобальные приложения
    

    View Slide

62. Kubernetes
    33
    

    View Slide

63. class AksCluster extends pulumi.ComponentResource {
    constructor(name: string,
    args: AksClusterArgs,
    opts: pulumi.ComponentResourceOptions = {}) {
    super("examples:aks:AksCluster", name, args, opts);
    const password = new random.RandomPassword("password"/*, {...}*/);
    const sshPublicKey = new tls.PrivateKey("key"/*, {...}*/);
    const adApp = new azuread.Application("aks"/*, {...}*/);
    const adSp = new azuread.ServicePrincipal("aksSp"/*, {...}*/);
    const vnet = new azure.network.VirtualNetwork("vnet"/*, {...}*/);
    const subnet = new azure.network.Subnet("subnet"/*, {...}*/);
    const cluster = new azure.containerservice.KubernetesCluster("aksCluster"/*, {...}*/)
    }
    }
    Managed Clusters
    

    View Slide

64. Crosswalk
    for
    Kubernetes
    

    View Slide

65. // Define the application Pod.
    const appPod = new kx.PodBuilder({
    containers: [{
    image: "app:1.0.0",
    env: { "APP_CONFIG_PATH": "/app/config" },
    volumeMounts: [configs.mount("/app/config")],
    }],
    });
    // Create a Kubernetes Deployment using the previous Pod definition.
    const deployment = new kx.Deployment("nginx", {
    spec: appPod.asDeploymentSpec({ replicas: 3 }),
    });
    // Expose the Deployment with a load balancer using a Kubernetes Service.
    const service = deployment.createService({
    type: kx.types.ServiceType.LoadBalancer,
    });
    Библиотека Kx
    

    View Slide

66. // Find all distinct versions of MySQL running in your cluster.
    const mySqlVersions = kq
    .list("v1", "Pod")
    .flatMap(pod => pod.spec.containers)
    .map(container => container.image)
    .filter(imageName => imageName.includes("mysql"))
    .distinct();
    mySqlVersions.forEach(console.log);
    Запросы
    

    View Slide

67. Контроль
    качества
    36
    

    View Slide

68. Языки
    программирования
    для
    инфраструктуры?
    Точно?
    

    View Slide

69. const group = require("./index").group;
    // check: Instances must not have SSH open to the Internet.
    it("must not open port 22 (SSH) to the Internet", done => {
    pulumi.all([ group.urn, group.ingress ]).apply(([ urn, ingress ]) => {
    if (ingress.find(rule =>
    rule.fromPort == 22 && rule.cidrBlocks.find(block => block === "0.0.0.0/0"))) {
    done(new Error(`Illegal SSH port 22 open to the Internet on group ${urn}`));
    } else {
    done();
    }
    });
    });
    Unit Testing
    

    View Slide

70. const policies = new PolicyPack("aws", {
    policies: [
    {
    name: "prohibited-public-internet",
    description: "Ingress rules with public internet access are prohibited.",
    enforcementLevel: "mandatory",
    validateResource: validateTypedResource(aws.ec2.SecurityGroup, (sg, args, report) => {
    const publicInternetRules = (sg.ingress || []).find(ingressRule =>
    (ingressRule.cidrBlocks || []).find(cidr => cidr === "0.0.0.0/0"));
    if (publicInternetRules) {
    report("Ingress rules with public internet access are prohibited.");
    }
    }),
    },
    ],
    });
    Policy as Code
    

    View Slide

71. AWS Cloud
    Development Kit
    (CDK)?
    39
    

    View Slide

72. м
    TypeScript
    Python
    Java
    .NET
    Транслируется в
    CloudFormation
    

    View Slide

73. const hello = new lambda.Function(this, 'HelloHandler', {
    runtime: lambda.Runtime.NODEJS_8_10,
    code: lambda.Code.asset('lambda'),
    handler: 'hello.handler'
    });
    new apigw.LambdaRestApi(this, 'Endpoint', {
    handler: hello
    });
    Пример AWS CDK
    

    View Slide

74. Примеры из моей презентации
    ✓ ✓
    ✓ 
    ✓ ✓
    ✓ 
    ✓ 
    ✓ 
    

    View Slide

75. Выводы
    40
    

    View Slide

76. Используйте
    Инфраструктуру как
    код
    Создаёте
    приложения
    для cloud?
    

    View Slide

77. Меньшие
    компоненты
    Лучшая
    автоматизация
    

    View Slide

78. Преимущества кода
    

    View Slide

79. м
    Инфраструктура
    как код
    Архитектура
    как код
    

    View Slide

80. View Slide