Tweet
Tweet

Slide 1

Slide 1 text

Serverless Infrastructure as Code Serverless Computing London | November 6 | 2019

Slide 2

Slide 2 text

Serverless Functions

Slide 3

Slide 3 text

Serverless Functions

Slide 4

Slide 4 text

Serverless Functions

Slide 5

Slide 5 text

Agenda

Slide 6

Slide 6 text

Agenda

Slide 7

Slide 7 text

URL Shortener 4

Slide 8

Slide 8 text

Serverless URL Shortener (AWS) AWS DynamoDB Table “URLs” AWS Lambda “Add URL” AWS Lambda “Open URL”

Slide 9

Slide 9 text

AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs” API Gateway S3 Bucket Static site

Slide 10

Slide 10 text

AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs” API Gateway S3 Bucket Static site Stage Deployment REST endpoint IAM IAM Role IAM Policy Bucket Objects CloudFront Route53

Slide 11

Slide 11 text

Global URL Shortener (Azure)

Slide 12

Slide 12 text

How do we create such an application?

Slide 13

Slide 13 text

Web Portals:

Slide 14

Slide 14 text

Goals of Infrastructure as Code

Slide 15

Slide 15 text

Cloud Automation 10

Slide 16

Slide 16 text

Unified Management API

Slide 17

Slide 17 text

REST calls Content-Type: application/x-www-form-urlencoded; charset=UTF-8 X-Amz-Date: 20130813T150211Z Host: ec2.amazonaws.com Authorization: AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20130813/us-east- 1/ec2/aws4_request, SignedHeaders=content-type;host;x-amz- date, Signature=ced6826de92d2bdeed8f846f0bf508e8559e98e4b0194b84example54174deb456c http://ec2.amazonaws.com/?Action=RunInstances ImageId=ami-2bb65342 &MaxCount=3 &MinCount=1 &Monitoring.Enabled=true &Placement.AvailabilityZone=us-east-1a &Version=2016-11-15

Slide 18

Slide 18 text

var launchRequest = new RunInstancesRequest() { ImageId = amiID, InstanceType = "t1.micro", MinCount = 1, MaxCount = 1, KeyName = keyPairName, SecurityGroupIds = groups }; var launchResponse = ec2Client.RunInstances(launchRequest); var instances = launchResponse.Reservation.Instances; SDKs to call those REST endpoints

Slide 19

Slide 19 text

az storage account create \ --location westus \ --name samplesa \ --resource-group myrg \ --sku LRS Unified CLI

Slide 20

Slide 20 text

Scripting: Challenges

Slide 21

Slide 21 text

Desired State Configuration 14

Slide 22

Slide 22 text

Desired State Configuration

Slide 23

Slide 23 text

Managing Resource Graphs

Slide 24

Slide 24 text

Challenges

Slide 25

Slide 25 text

More Challenges

Slide 26

Slide 26 text

Desired State Provisioning

Slide 27

Slide 27 text

ARM Templates: JSON "resources": [ { "apiVersion": "2016-01-01", "type": "Microsoft.Storage/storageAccounts", "name": "mystorageaccount", "location": "westus", "sku": { "name": "Standard_LRS" }, "kind": "Storage", "properties": { } } ]

Slide 28

Slide 28 text

Converted to REST calls PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/ {resourceGroupName}/providers/Microsoft.Storage/storageAccounts/mystorageaccount?api- version=2016-01-01 { "location": "westus", "properties": { } "sku": { "name": "Standard_LRS" }, "kind": "Storage" }

Slide 29

Slide 29 text

Tracked Deployment

Slide 30

Slide 30 text

AWS CloudFormation: YAML Resources: S3BucketForURLs: Type: "AWS::S3::Bucket" DeletionPolicy: Delete Properties: BucketName: !If [ "CreateNewBucket", !Ref "AWS::NoValue", !Ref S3BucketName ] WebsiteConfiguration: IndexDocument: "index.html" LifecycleConfiguration: Rules: - Id: DisposeShortUrls ExpirationInDays: !Ref URLExpiration Prefix: "u" Status: Enabled

Slide 31

Slide 31 text

AWS CloudFormation Features

Slide 32

Slide 32 text

Challenges

Slide 33

Slide 33 text

Terraform 22

Slide 34

Slide 34 text

resource "azurerm_storage_account" "sa" { name = "mysa" location = azurerm_resource_group.rg.location resource_group_name = azurerm_resource_group.rg.name account_tier = "Standard" account_replication_type = "LRS" } Custom DSL: HCL (HashiCorp Configuration Language)

Slide 35

Slide 35 text

• • • • • Providers • • • • •

Slide 36

Slide 36 text

Client-driven Engine Azure Resource Manager Azure Go SDK Azure TF Provider TF CLI HCL

Slide 37

Slide 37 text

terraform plan # Creates a draft of changes, does nothing to the infra terraform apply # Rolls the infra from its current state to the desired one terraform destroy # Deletes all resources in the current workspace Workflow: Plan, Apply, Destroy

Slide 38

Slide 38 text

> terraform plan Terraform will perform the following actions: # azurerm_storage_account.sa must be replaced -/+ resource "azurerm_storage_account" "sa" { ... ~ location = "westus" -> "westus2" # forces replacement ... Preview Changes

Slide 39

Slide 39 text

More Terraform today

Slide 40

Slide 40 text

Need for higher abstraction and reusable components KV Store Table “URLs” Function “Add URL” Function “Open URL”

Slide 41

Slide 41 text

How?

Slide 42

Slide 42 text

Serverless Framework 28

Slide 43

Slide 43 text

service: serverless-simple-http-endpoint provider: name: aws runtime: nodejs8.10 functions: currentTime: handler: handler.endpoint events: - http: path: ping method: get YAML Definition Files

Slide 44

Slide 44 text

Focus on target scenarios

Slide 45

Slide 45 text

Pulumi 30

Slide 46

Slide 46 text

General-Purpose Programming Languages

Slide 47

Slide 47 text

• • • • • Providers • • • • •

Slide 48

Slide 48 text

const storageAccount = new azure.storage.Account("storage", { resourceGroupName: resourceGroup.name, accountReplicationType: "LRS", accountTier: "Standard", }); TypeScript example

Slide 49

Slide 49 text

const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", environment: { variables: { "COUNTER_TABLE": counterTable.name, }}, }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener

Slide 50

Slide 50 text

const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", environment: { variables: { "COUNTER_TABLE": counterTable.name, }}, }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener

Slide 51

Slide 51 text

const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", environment: { variables: { "COUNTER_TABLE": counterTable.name, }}, }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener

Slide 52

Slide 52 text

const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", environment: { variables: { "COUNTER_TABLE": counterTable.name, }}, }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); Still, Desired State!

Slide 53

Slide 53 text

How Pulumi works CLI and Engine Last deployed state index.ts Language host AWS Azure GCP Kubernetes new Resource() Create, update, delete

Slide 54

Slide 54 text

Auto- completion

Slide 55

Slide 55 text

Compile Errors 55

Slide 56

Slide 56 text

for (const location of locations) { // Azure Function - one per region const fn = new appservice.FunctionApp(`GetUrl-${location}`, { resourceGroup, location, route: "{key}", archive: zip, }); } Conditionals, loops

Slide 57

Slide 57 text

class AksCluster extends pulumi.ComponentResource { constructor(name: string, args: AksClusterArgs, opts: pulumi.ComponentResourceOptions = {}) { super("examples:aks:AksCluster", name, args, opts); const password = new random.RandomPassword("password"/*, {...}*/); const sshPublicKey = new tls.PrivateKey("key"/*, {...}*/); const adApp = new azuread.Application("aks"/*, {...}*/); const adSp = new azuread.ServicePrincipal("aksSp"/*, {...}*/); const vnet = new azure.network.VirtualNetwork("vnet"/*, {...}*/); const subnet = new azure.network.Subnet("subnet"/*, {...}*/); const cluster = new azure.containerservice.KubernetesCluster("aksCluster"/*, {...}*/) } } Components

Slide 58

Slide 58 text

AWS Cloud Development Kit (CDK) 36

Slide 59

Slide 59 text

м TypeScript Python Java .NET Compiles to CloudFormation

Slide 60

Slide 60 text

const hello = new lambda.Function(this, 'HelloHandler', { runtime: lambda.Runtime.NODEJS_8_10, code: lambda.Code.asset('lambda'), handler: 'hello.handler' }); new apigw.LambdaRestApi(this, 'Endpoint', { handler: hello }); AWS CDK example

Slide 61

Slide 61 text

Want to talk CDK? Find this guy:

Slide 62

Slide 62 text

Programming Languages for Infrastructure? You sure?

Slide 63

Slide 63 text

Architecture as Code 36 38

Slide 64

Slide 64 text

Scheduled Tasks

Slide 65

Slide 65 text

const ecsScheduledTask = new ScheduledEc2Task(stack, 'ScheduledTask’, { schedule: events.Schedule.expression('rate(1 minute)’) scheduledEc2TaskImageOptions: { image: ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'), memoryLimitMiB: 256, environment: { name: 'TRIGGER', value: 'CloudWatch Events' }, }, }); Scheduled Tasks

Slide 66

Slide 66 text

Zip Incoming Reports

Slide 67

Slide 67 text

const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips"); tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks

Slide 68

Slide 68 text

const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips"); tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks

Slide 69

Slide 69 text

Global Apps

Slide 70

Slide 70 text

Global Apps

Slide 71

Slide 71 text

export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope", "WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps

Slide 72

Slide 72 text

export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope", "WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps

Slide 73

Slide 73 text

Conclusions 42

Slide 74

Slide 74 text

Define Infrastructure as Code Making Cloud Apps?

Slide 75

Slide 75 text

Infrastructure as Code Tools: Tradeoffs

Slide 76

Slide 76 text

Fine-grained components Better Automation

Slide 77

Slide 77 text

No content