Tweet
Tweet

Slide 1

Slide 1 text

Cloud Infrastructure as Code Cloud Republic Event | October 30 | 2019

Slide 2

Slide 2 text

Agenda

Slide 3

Slide 3 text

On-Premises

Slide 4

Slide 4 text

On-premises Load Balancer Web Server Web Server Load Balancer Database Server Database Server

Slide 5

Slide 5 text

LB Web Web DB LB DB LB Web Web DB LB DB Web DB Web + DB

Slide 6

Slide 6 text

Ad-hoc Management

Slide 7

Slide 7 text

Issues

Slide 8

Slide 8 text

Configuration management tools: Puppet/Chef

Slide 9

Slide 9 text

Benefits of Infrastructure as Code

Slide 10

Slide 10 text

Cloud

Slide 11

Slide 11 text

Compute on Demand

Slide 12

Slide 12 text

REST calls Content-Type: application/x-www-form-urlencoded; charset=UTF-8 X-Amz-Date: 20130813T150211Z Host: ec2.amazonaws.com Authorization: AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20130813/us-east- 1/ec2/aws4_request, SignedHeaders=content-type;host;x-amz- date, Signature=ced6826de92d2bdeed8f846f0bf508e8559e98e4b0194b84example54174deb456c http://ec2.amazonaws.com/?Action=RunInstances ImageId=ami-2bb65342 &MaxCount=3 &MinCount=1 &Monitoring.Enabled=true &Placement.AvailabilityZone=us-east-1a &Version=2016-11-15

Slide 13

Slide 13 text

var launchRequest = new RunInstancesRequest() { ImageId = amiID, InstanceType = "t1.micro", MinCount = 1, MaxCount = 1, KeyName = keyPairName, SecurityGroupIds = groups }; var launchResponse = ec2Client.RunInstances(launchRequest); var instances = launchResponse.Reservation.Instances; SDKs to call those REST endpoints

Slide 14

Slide 14 text

More Cloud services (PaaS, SaaS, FaaS)

Slide 15

Slide 15 text

Early Cloud API Challenges

Slide 16

Slide 16 text

Resource Managers

Slide 17

Slide 17 text

Unified API Layer

Slide 18

Slide 18 text

az storage account create \ --location westus \ --name samplesa \ --resource-group myrg \ --sku LRS Unified CLI

Slide 19

Slide 19 text

Desired State Provisioning

Slide 20

Slide 20 text

ARM Templates: JSON "resources": [ { "apiVersion": "2016-01-01", "type": "Microsoft.Storage/storageAccounts", "name": "mystorageaccount", "location": "westus", "sku": { "name": "Standard_LRS" }, "kind": "Storage", "properties": { } } ]

Slide 21

Slide 21 text

Converted to REST calls PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/ {resourceGroupName}/providers/Microsoft.Storage/storageAccounts/mystorageaccount?api- version=2016-01-01 { "location": "westus", "properties": { } "sku": { "name": "Standard_LRS" }, "kind": "Storage" }

Slide 22

Slide 22 text

Tracked Deployment

Slide 23

Slide 23 text

AWS CloudFormation: YAML Resources: S3BucketForURLs: Type: "AWS::S3::Bucket" DeletionPolicy: Delete Properties: BucketName: !If [ "CreateNewBucket", !Ref "AWS::NoValue", !Ref S3BucketName ] WebsiteConfiguration: IndexDocument: "index.html" LifecycleConfiguration: Rules: - Id: DisposeShortUrls ExpirationInDays: !Ref URLExpiration Prefix: "u" Status: Enabled

Slide 24

Slide 24 text

AWS CloudFormation Nice Features

Slide 25

Slide 25 text

Resource Graphs

Slide 26

Slide 26 text

Describes a list of Resources

Slide 27

Slide 27 text

Resource Graphs

Slide 28

Slide 28 text

Challenges

Slide 29

Slide 29 text

More Challenges

Slide 30

Slide 30 text

Terraform

Slide 31

Slide 31 text

Made by HashiCorp Also: Vagrant, Packer, Vault, Consul Written in Go, open source MPL 2.0 Since 2014 Terraform

Slide 32

Slide 32 text

resource "azurerm_storage_account" "sa" { name = "mysa" location = azurerm_resource_group.rg.location resource_group_name = azurerm_resource_group.rg.name account_tier = "Standard" account_replication_type = "LRS" } Custom DSL: HCL (HashiCorp Configuration Language)

Slide 33

Slide 33 text

• • • • • Providers • • • • •

Slide 34

Slide 34 text

Client-driven Engine Azure Resource Manager Azure Go SDK Azure TF Provider TF CLI HCL

Slide 35

Slide 35 text

resource "azurerm_function_app" "app" { name = var.app_name location = azurerm_resource_group.rg.location resource_group_name = azurerm_resource_group.rg.name app_service_plan_id = azurerm_app_service_plan.asp.id storage_connection_string = azurerm_storage_account.sa.primary_connection_string } Strong in dependency management

Slide 36

Slide 36 text

terraform plan # Creates a draft of changes, does nothing to the infra terraform apply # Rolls the infra from its current state to the desired one terraform destroy # Deletes all resources in the current workspace Workflow: Plan, Apply, Destroy

Slide 37

Slide 37 text

> terraform plan Terraform will perform the following actions: # azurerm_storage_account.sa must be replaced -/+ resource "azurerm_storage_account" "sa" { ... ~ location = "westus" -> "westus2" # forces replacement ... Preview Changes

Slide 38

Slide 38 text

Serverless

Slide 39

Slide 39 text

Serverless URL Shortener AWS DynamoDB Table “URLs” AWS Lambda “Add URL” AWS Lambda “Open URL”

Slide 40

Slide 40 text

AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs” API Gateway S3 Bucket Static site

Slide 41

Slide 41 text

AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs” API Gateway S3 Bucket Static site Stage Deployment REST endpoint Permissions Permissions Policy Bucket Objects

Slide 42

Slide 42 text

Need for higher abstraction and reusable components KV Store Table “URLs” Function “Add URL” Function “Open URL”

Slide 43

Slide 43 text

Cloud- Native Ecosystem

Slide 44

Slide 44 text

How?

Slide 45

Slide 45 text

Serverless Framework

Slide 46

Slide 46 text

Made by Serverless, Inc. Tooling for Serverless Applications Since 2015 Serverless Framework

Slide 47

Slide 47 text

service: serverless-simple-http-endpoint provider: name: aws runtime: nodejs8.10 functions: currentTime: handler: handler.endpoint events: - http: path: ping method: get YAML Definition Files

Slide 48

Slide 48 text

Focus on target scenarios

Slide 49

Slide 49 text

Pulumi

Slide 50

Slide 50 text

Made by Pulumi Corp. Written in Go, open source Apache License 2.0 Since 2018 Pulumi

Slide 51

Slide 51 text

General-Purpose Programming Languages

Slide 52

Slide 52 text

• • • • • Providers • • • • •

Slide 53

Slide 53 text

const storageAccount = new azure.storage.Account("storage", { resourceGroupName: resourceGroup.name, accountReplicationType: "LRS", accountTier: "Standard", }); TypeScript example

Slide 54

Slide 54 text

How Pulumi works CLI and Engine Last deployed state index.ts Language host AWS Azure GCP Kubernetes new Resource() Create, update, delete

Slide 55

Slide 55 text

Workflow: Preview, Up, Destroy

Slide 56

Slide 56 text

Workflow: Preview, Up, Destroy

Slide 57

Slide 57 text

Benefits of Real Code

Slide 58

Slide 58 text

const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener

Slide 59

Slide 59 text

const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener

Slide 60

Slide 60 text

const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener

Slide 61

Slide 61 text

AWS Cloud Development Kit (CDK)

Slide 62

Slide 62 text

м TypeScript Python Java .NET Compiles to CloudFormation

Slide 63

Slide 63 text

const hello = new lambda.Function(this, 'HelloHandler', { runtime: lambda.Runtime.NODEJS_8_10, code: lambda.Code.asset('lambda'), handler: 'hello.handler' }); new apigw.LambdaRestApi(this, 'Endpoint', { handler: hello }); AWS CDK example

Slide 64

Slide 64 text

// This is F# let myStorage = storageAccount { name "mystorage" // set account name sku Storage.Sku.PremiumLRS // use Premium LRS } Azure? ARM Generators (ex: Farmer)

Slide 65

Slide 65 text

Architecture as Code

Slide 66

Slide 66 text

Zip Incoming Reports

Slide 67

Slide 67 text

const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips"); tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks

Slide 68

Slide 68 text

const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips"); tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks

Slide 69

Slide 69 text

Scheduled Tasks

Slide 70

Slide 70 text

const ecsScheduledTask = new ScheduledEc2Task(stack, 'ScheduledTask', { schedule: events.Schedule.expression('rate(1 minute)') scheduledEc2TaskImageOptions: { image: ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'), memoryLimitMiB: 256, environment: { name: 'TRIGGER', value: 'CloudWatch Events' }, }, }); Scheduled Tasks

Slide 71

Slide 71 text

Global Apps

Slide 72

Slide 72 text

export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope", "WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps

Slide 73

Slide 73 text

export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope", "WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps

Slide 74

Slide 74 text

Conclusions

Slide 75

Slide 75 text

USE INFRASTRUCTURE- AS-CODE Making Cloud Apps?

Slide 76

Slide 76 text

Infrastructure as Code Tools: Tradeoffs

Slide 77

Slide 77 text

Fine-grained components Better Automation

Slide 78

Slide 78 text

In Search for Powerful Abstractions

Slide 79

Slide 79 text

No content