Tweet
Tweet

Slide 1

Slide 1 text

From YAML to TypeScript: Developer’s View on Cloud Automation Ø | November 8 | 2019

Slide 2

Slide 2 text

Why Infrastructure as Code General- purpose Languages

Slide 3

Slide 3 text

On-Premises 2

Slide 4

Slide 4 text

On-premises Load Balancer Web Server Web Server Load Balancer Database Server Database Server

Slide 5

Slide 5 text

LB Web Web DB LB DB LB Web Web DB LB DB Web DB Web + DB

Slide 6

Slide 6 text

Issues with Manual Server Management

Slide 7

Slide 7 text

Configuration management tools: Puppet/Chef

Slide 8

Slide 8 text

Benefits of Infrastructure as Code

Slide 9

Slide 9 text

Cloud Automation 6

Slide 10

Slide 10 text

Unified Management API

Slide 11

Slide 11 text

REST calls Content-Type: application/x-www-form-urlencoded; charset=UTF-8 X-Amz-Date: 20130813T150211Z Host: ec2.amazonaws.com Authorization: AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20130813/us-east- 1/ec2/aws4_request, SignedHeaders=content-type;host;x-amz- date, Signature=ced6826de92d2bdeed8f846f0bf508e8559e98e4b0194b84example54174deb456c http://ec2.amazonaws.com/?Action=RunInstances ImageId=ami-2bb65342 &MaxCount=3 &MinCount=1 &Monitoring.Enabled=true &Placement.AvailabilityZone=us-east-1a &Version=2016-11-15

Slide 12

Slide 12 text

var launchRequest = new RunInstancesRequest() { ImageId = amiID, InstanceType = "t1.micro", MinCount = 1, MaxCount = 1, KeyName = keyPairName, SecurityGroupIds = groups }; var launchResponse = ec2Client.RunInstances(launchRequest); var instances = launchResponse.Reservation.Instances; SDKs to call those REST endpoints

Slide 13

Slide 13 text

az storage account create \ --location westus \ --name samplesa \ --resource-group myrg \ --sku LRS Command-line Tools

Slide 14

Slide 14 text

Scripting: Challenges

Slide 15

Slide 15 text

Desired State Configuration 8

Slide 16

Slide 16 text

Desired State Configuration

Slide 17

Slide 17 text

Managing Resource Graphs

Slide 18

Slide 18 text

Challenges

Slide 19

Slide 19 text

Desired State Provisioning

Slide 20

Slide 20 text

Markup Languages 12 Yet Another Markup Language

Slide 21

Slide 21 text

Azure ARM Templates: JSON "resources": [ { "apiVersion": "2016-01-01", "type": "Microsoft.Storage/storageAccounts", "name": "mystorageaccount", "location": "westus", "sku": { "name": "Standard_LRS" }, "kind": "Storage", "properties": { } } ]

Slide 22

Slide 22 text

Converted to REST calls PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/ {resourceGroupName}/providers/Microsoft.Storage/storageAccounts/mystorageaccount?api- version=2016-01-01 { "location": "westus", "properties": { } "sku": { "name": "Standard_LRS" }, "kind": "Storage" }

Slide 23

Slide 23 text

AWS CloudFormation: YAML Resources: S3BucketForURLs: Type: "AWS::S3::Bucket" DeletionPolicy: Delete Properties: BucketName: !If [ "CreateNewBucket", !Ref "AWS::NoValue", !Ref S3BucketName ] WebsiteConfiguration: IndexDocument: "index.html" LifecycleConfiguration: Rules: - Id: DisposeShortUrls ExpirationInDays: !Ref URLExpiration Prefix: "u" Status: Enabled

Slide 24

Slide 24 text

Challenges

Slide 25

Slide 25 text

resource "azurerm_storage_account" "sa" { name = "mysa" location = azurerm_resource_group.rg.location resource_group_name = azurerm_resource_group.rg.name account_tier = "Standard" account_replication_type = "LRS" } Terraform: DSL (HashiCorp Configuration Language)

Slide 26

Slide 26 text

• • • • • Providers • • • • •

Slide 27

Slide 27 text

Markup Authoring Experience

Slide 28

Slide 28 text

Example: Code to infrastructure ratio KV Store Table “URLs” Function “Add URL” Function “Open URL”

Slide 29

Slide 29 text

{ "Resources": { "InstanceSecurityGroup": { "Type": "AWS::EC2::SecurityGroup", "Properties": { "SecurityGroupIngress": [ ... ] } }, "EC2Instance": { "Type": "AWS::EC2::Instance", "Properties": { "InstanceType": "t2.micro", "SecurityGroups": [ { "Ref": "InstanceSecurityGroup" } ], "ImageId": "ami-0080e4c5bc078760e" } } } } References

Slide 30

Slide 30 text

{ "condition": "[equals(parameters('production'), 'Yes')]", "type": "Microsoft.Compute/availabilitySets", "apiVersion": "2017-03-30", "name": "[variables('availabilitySetName')]", "location": "[resourceGroup().location]", "properties": { "platformFaultDomainCount": 2, "platformUpdateDomainCount": 3 } } Conditionals

Slide 31

Slide 31 text

locals { files = ["index.php","main.php"] } resource "aws_s3_bucket_object" "files" { count = "${length(local.files)}" key = "${local.files[count.index]}" bucket = "${aws_s3_bucket.examplebucket.id}" source = "./src/${local.files[count.index]}" } Loops

Slide 32

Slide 32 text

Reusability

Slide 33

Slide 33 text

Testability

Slide 34

Slide 34 text

I think we solved these problems before?

Slide 35

Slide 35 text

IDEs Code completion Unit tests Types Functions Package managers Versioning Classes Code reviews Contracts SDKs Workflows Refactoring

Slide 36

Slide 36 text

What if developers did infrastructure?

Slide 37

Slide 37 text

Pulumi 20

Slide 38

Slide 38 text

General-Purpose Programming Languages

Slide 39

Slide 39 text

const resourceGroup = new azure.core.ResourceGroup("rg"); const storageAccount = new azure.storage.Account("storage", { resourceGroupName: resourceGroup.name, accountReplicationType: "LRS", accountTier: "Standard", }); TypeScript example

Slide 40

Slide 40 text

const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", environment: { variables: { "COUNTER_TABLE": counterTable.name, }}, }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); Example: Serverless HTTP

Slide 41

Slide 41 text

const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", environment: { variables: { "COUNTER_TABLE": counterTable.name, }}, }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); Example: Serverless HTTP

Slide 42

Slide 42 text

const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", environment: { variables: { "COUNTER_TABLE": counterTable.name, }}, }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); Example: Serverless HTTP

Slide 43

Slide 43 text

const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", environment: { variables: { "COUNTER_TABLE": counterTable.name, }}, }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); Still, Desired State!

Slide 44

Slide 44 text

How Pulumi works CLI and Engine Last deployed state index.ts Language host AWS Azure GCP Kubernetes new Resource() Create, update, delete

Slide 45

Slide 45 text

• • • • • Providers • • • • •

Slide 46

Slide 46 text

Benefits of Real Code

Slide 47

Slide 47 text

Auto- completion

Slide 48

Slide 48 text

Compile Errors 48

Slide 49

Slide 49 text

for (const az of aws.getAvailabilityZones().names) { const server = new aws.ec2.Instance(`web-${az}`, { instanceType: "t2.micro", securityGroups: [ group.id ], ami: "ami-0080e4c5bc078760e", availabilityZone: az, }); } Conditionals, loops

Slide 50

Slide 50 text

class AksCluster extends pulumi.ComponentResource { constructor(name: string, args: AksClusterArgs, opts: pulumi.ComponentResourceOptions = {}) { super("examples:aks:AksCluster", name, args, opts); const password = new random.RandomPassword("password"/*, {...}*/); const sshPublicKey = new tls.PrivateKey("key"/*, {...}*/); const adApp = new azuread.Application("aks"/*, {...}*/); const adSp = new azuread.ServicePrincipal("aksSp"/*, {...}*/); const vnet = new azure.network.VirtualNetwork("vnet"/*, {...}*/); const subnet = new azure.network.Subnet("subnet"/*, {...}*/); const cluster = new azure.containerservice.KubernetesCluster("aksCluster"/*, {...}*/) } } Components

Slide 51

Slide 51 text

AWS Cloud Development Kit (CDK) 31

Slide 52

Slide 52 text

м TypeScript Python Java .NET Compiles to CloudFormation

Slide 53

Slide 53 text

const hello = new lambda.Function(this, 'HelloHandler', { runtime: lambda.Runtime.NODEJS_8_10, code: lambda.Code.asset('lambda'), handler: 'hello.handler' }); new apigw.LambdaRestApi(this, 'Endpoint', { handler: hello }); AWS CDK example

Slide 54

Slide 54 text

Programming Languages for Infrastructure? You sure?

Slide 55

Slide 55 text

Architecture as Code 33

Slide 56

Slide 56 text

Scheduled Tasks

Slide 57

Slide 57 text

const ecsScheduledTask = new ScheduledEc2Task(stack, 'ScheduledTask’, { schedule: events.Schedule.expression('rate(1 minute)’) scheduledEc2TaskImageOptions: { image: ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'), memoryLimitMiB: 256, environment: { name: 'TRIGGER', value: 'CloudWatch Events' }, }, }); Scheduled Tasks

Slide 58

Slide 58 text

Zip Incoming Reports

Slide 59

Slide 59 text

const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips"); tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks

Slide 60

Slide 60 text

const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips"); tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks

Slide 61

Slide 61 text

Global Apps (1)

Slide 62

Slide 62 text

Global Apps (2)

Slide 63

Slide 63 text

Global Apps (3)

Slide 64

Slide 64 text

export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope", "WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps

Slide 65

Slide 65 text

export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope", "WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps

Slide 66

Slide 66 text

Demo! 37

Slide 67

Slide 67 text

Conclusions 37

Slide 68

Slide 68 text

Define Infrastructure as Code Making Cloud Apps?

Slide 69

Slide 69 text

Fine-grained components Better Automation

Slide 70

Slide 70 text

70 Infrastructure is a part of your applications. Use your developer skills to build it.

Slide 71

Slide 71 text

No content