Cloud Infrastructure as Code

History and tool comparison of Infrastructure as Code for cloud applications in AWS and Azure

Transcript

1. Cloud Infrastructure as Code Cloud Republic Event | October 30 | 2019
2. Agenda
3. On-Premises
4. On-premises Load Balancer Web Server Web Server Load Balancer Database Server Database Server
5. LB Web Web DB LB DB LB Web Web DB LB DB Web DB Web + DB
6. Ad-hoc Management
7. Issues
8. Configuration management tools: Puppet/Chef
9. Benefits of Infrastructure as Code
10. Cloud
11. Compute on Demand
12. REST calls Content-Type: application/x-www-form-urlencoded; charset=UTF-8 X-Amz-Date: 20130813T150211Z Host: ec2.amazonaws.com Authorization: AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20130813/us-east- 1/ec2/aws4_request, SignedHeaders=content-type;host;x-amz- date, Signature=ced6826de92d2bdeed8f846f0bf508e8559e98e4b0194b84example54174deb456c http://ec2.amazonaws.com/?Action=RunInstances ImageId=ami-2bb65342 &MaxCount=3 &MinCount=1 &Monitoring.Enabled=true &Placement.AvailabilityZone=us-east-1a &Version=2016-11-15
13. var launchRequest = new RunInstancesRequest() { ImageId = amiID, InstanceType = "t1.micro", MinCount = 1, MaxCount = 1, KeyName = keyPairName, SecurityGroupIds = groups }; var launchResponse = ec2Client.RunInstances(launchRequest); var instances = launchResponse.Reservation.Instances; SDKs to call those REST endpoints
14. More Cloud services (PaaS, SaaS, FaaS)
15. Early Cloud API Challenges
16. Resource Managers
17. Unified API Layer
18. az storage account create \ --location westus \ --name samplesa \ --resource-group myrg \ --sku LRS Unified CLI
19. Desired State Provisioning
20. ARM Templates: JSON "resources": [ { "apiVersion": "2016-01-01", "type": "Microsoft.Storage/storageAccounts", "name": "mystorageaccount", "location": "westus", "sku": { "name": "Standard_LRS" }, "kind": "Storage", "properties": { } } ]
21. Converted to REST calls PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/ {resourceGroupName}/providers/Microsoft.Storage/storageAccounts/mystorageaccount?api- version=2016-01-01 { "location": "westus", "properties": { } "sku": { "name": "Standard_LRS" }, "kind": "Storage" }
22. Tracked Deployment
23. AWS CloudFormation: YAML Resources: S3BucketForURLs: Type: "AWS::S3::Bucket" DeletionPolicy: Delete Properties: BucketName: !If [ "CreateNewBucket", !Ref "AWS::NoValue", !Ref S3BucketName ] WebsiteConfiguration: IndexDocument: "index.html" LifecycleConfiguration: Rules: - Id: DisposeShortUrls ExpirationInDays: !Ref URLExpiration Prefix: "u" Status: Enabled
24. AWS CloudFormation Nice Features
25. Resource Graphs
26. Describes a list of Resources
27. Resource Graphs
28. Challenges
29. More Challenges
30. Terraform
31. Made by HashiCorp Also: Vagrant, Packer, Vault, Consul Written in Go, open source MPL 2.0 Since 2014 Terraform
32. resource "azurerm_storage_account" "sa" { name = "mysa" location = azurerm_resource_group.rg.location resource_group_name = azurerm_resource_group.rg.name account_tier = "Standard" account_replication_type = "LRS" } Custom DSL: HCL (HashiCorp Configuration Language)
33. • • • • • Providers • • • • •
34. Client-driven Engine Azure Resource Manager Azure Go SDK Azure TF Provider TF CLI HCL
35. resource "azurerm_function_app" "app" { name = var.app_name location = azurerm_resource_group.rg.location resource_group_name = azurerm_resource_group.rg.name app_service_plan_id = azurerm_app_service_plan.asp.id storage_connection_string = azurerm_storage_account.sa.primary_connection_string } Strong in dependency management
36. terraform plan # Creates a draft of changes, does nothing to the infra terraform apply # Rolls the infra from its current state to the desired one terraform destroy # Deletes all resources in the current workspace Workflow: Plan, Apply, Destroy
37. > terraform plan Terraform will perform the following actions: # azurerm_storage_account.sa must be replaced -/+ resource "azurerm_storage_account" "sa" { ... ~ location = "westus" -> "westus2" # forces replacement ... Preview Changes
38. Serverless
39. Serverless URL Shortener AWS DynamoDB Table “URLs” AWS Lambda “Add URL” AWS Lambda “Open URL”
40. AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs” API Gateway S3 Bucket Static site
41. AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs” API Gateway S3 Bucket Static site Stage Deployment REST endpoint Permissions Permissions Policy Bucket Objects
42. Need for higher abstraction and reusable components KV Store Table “URLs” Function “Add URL” Function “Open URL”
43. Cloud- Native Ecosystem
44. How?
45. Serverless Framework
46. Made by Serverless, Inc. Tooling for Serverless Applications Since 2015 Serverless Framework
47. service: serverless-simple-http-endpoint provider: name: aws runtime: nodejs8.10 functions: currentTime: handler: handler.endpoint events: - http: path: ping method: get YAML Definition Files
48. Focus on target scenarios
49. Pulumi
50. Made by Pulumi Corp. Written in Go, open source Apache License 2.0 Since 2018 Pulumi
51. General-Purpose Programming Languages
52. • • • • • Providers • • • • •
53. const storageAccount = new azure.storage.Account("storage", { resourceGroupName: resourceGroup.name, accountReplicationType: "LRS", accountTier: "Standard", }); TypeScript example
54. How Pulumi works CLI and Engine Last deployed state index.ts Language host AWS Azure GCP Kubernetes new Resource() Create, update, delete
55. Workflow: Preview, Up, Destroy
56. Workflow: Preview, Up, Destroy
57. Benefits of Real Code
58. const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
59. const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
60. const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id", type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
61. AWS Cloud Development Kit (CDK)
62. м TypeScript Python Java .NET Compiles to CloudFormation
63. const hello = new lambda.Function(this, 'HelloHandler', { runtime: lambda.Runtime.NODEJS_8_10, code: lambda.Code.asset('lambda'), handler: 'hello.handler' }); new apigw.LambdaRestApi(this, 'Endpoint', { handler: hello }); AWS CDK example
64. // This is F# let myStorage = storageAccount { name "mystorage" // set account name sku Storage.Sku.PremiumLRS // use Premium LRS } Azure? ARM Generators (ex: Farmer)
65. Architecture as Code
66. Zip Incoming Reports
67. const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips"); tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks
68. const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips"); tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks
69. Scheduled Tasks
70. const ecsScheduledTask = new ScheduledEc2Task(stack, 'ScheduledTask', { schedule: events.Schedule.expression('rate(1 minute)') scheduledEc2TaskImageOptions: { image: ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'), memoryLimitMiB: 256, environment: { name: 'TRIGGER', value: 'CloudWatch Events' }, }, }); Scheduled Tasks
71. Global Apps
72. export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope", "WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps
73. export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope", "WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps
74. Conclusions
75. USE INFRASTRUCTURE- AS-CODE Making Cloud Apps?
76. Infrastructure as Code Tools: Tradeoffs
77. Fine-grained components Better Automation
78. In Search for Powerful Abstractions