Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Cloud Infrastructure as Code
Search
Mikhail Shilkov
October 30, 2019
Programming
0
220
Cloud Infrastructure as Code
History and tool comparison of Infrastructure as Code for cloud applications in AWS and Azure
Mikhail Shilkov
October 30, 2019
Tweet
Share
More Decks by Mikhail Shilkov
See All by Mikhail Shilkov
From YAML to TypeScript: Developer’s View on Cloud Automation
mikhailshilkov
0
78
Cloud Management Superpowers with Pulumi
mikhailshilkov
0
480
Cloud Superpowers with Pulumi and F#
mikhailshilkov
1
470
Cloud Management Superpowers with Pulumi and .NET
mikhailshilkov
0
110
Managing Any Cloud with .NET
mikhailshilkov
0
65
Azure Infrastructure as C# and F#
mikhailshilkov
0
240
Infrastructure as Software
mikhailshilkov
0
400
Azure Infrastructure as C# and F#
mikhailshilkov
0
300
Pulumi: Cloud Infrastructure as C# and F#
mikhailshilkov
0
780
Other Decks in Programming
See All in Programming
대규모 트래픽을 처리하는 프론트 개발자의 전략
maryang
0
120
バイブスあるコーディングで ~PHP~ 便利ツールをつくるプラクティス
uzulla
1
330
プロダクトという一杯を作る - プロダクトチームが味の責任を持つまでの煮込み奮闘記
hiliteeternal
0
450
QA x AIエコシステム段階構築作戦
osu
0
270
『リコリス・リコイル』に学ぶ!! 〜キャリア戦略における計画的偶発性理論と変わる勇気の重要性〜
wanko_it
1
510
GitHub Copilotの全体像と活用のヒント AI駆動開発の最初の一歩
74th
7
2.6k
Nuances on Kubernetes - RubyConf Taiwan 2025
envek
0
150
新しいモバイルアプリ勉強会(仮)について
uetyo
1
250
Terraform やるなら公式スタイルガイドを読もう 〜重要項目 10選〜
hiyanger
13
3.1k
iOS開発スターターキットの作り方
akidon0000
0
240
自作OSでDOOMを動かしてみた
zakki0925224
1
1.3k
中級グラフィックス入門~効率的なメッシュレット描画~
projectasura
4
2.6k
Featured
See All Featured
Building a Scalable Design System with Sketch
lauravandoore
462
33k
KATA
mclloyd
32
14k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
23
1.4k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
283
13k
Producing Creativity
orderedlist
PRO
347
40k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Java REST API Framework Comparison - PWX 2021
mraible
33
8.8k
Optimizing for Happiness
mojombo
379
70k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.4k
Site-Speed That Sticks
csswizardry
10
770
How to Ace a Technical Interview
jacobian
278
23k
Transcript
Cloud Infrastructure as Code Cloud Republic Event | October 30
| 2019
Agenda
On-Premises
On-premises Load Balancer Web Server Web Server Load Balancer Database
Server Database Server
LB Web Web DB LB DB LB Web Web DB
LB DB Web DB Web + DB
Ad-hoc Management
Issues
Configuration management tools: Puppet/Chef
Benefits of Infrastructure as Code
Cloud
Compute on Demand
REST calls Content-Type: application/x-www-form-urlencoded; charset=UTF-8 X-Amz-Date: 20130813T150211Z Host: ec2.amazonaws.com Authorization:
AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20130813/us-east- 1/ec2/aws4_request, SignedHeaders=content-type;host;x-amz- date, Signature=ced6826de92d2bdeed8f846f0bf508e8559e98e4b0194b84example54174deb456c http://ec2.amazonaws.com/?Action=RunInstances ImageId=ami-2bb65342 &MaxCount=3 &MinCount=1 &Monitoring.Enabled=true &Placement.AvailabilityZone=us-east-1a &Version=2016-11-15
var launchRequest = new RunInstancesRequest() { ImageId = amiID, InstanceType
= "t1.micro", MinCount = 1, MaxCount = 1, KeyName = keyPairName, SecurityGroupIds = groups }; var launchResponse = ec2Client.RunInstances(launchRequest); var instances = launchResponse.Reservation.Instances; SDKs to call those REST endpoints
More Cloud services (PaaS, SaaS, FaaS)
Early Cloud API Challenges
Resource Managers
Unified API Layer
az storage account create \ --location westus \ --name samplesa
\ --resource-group myrg \ --sku LRS Unified CLI
Desired State Provisioning
ARM Templates: JSON "resources": [ { "apiVersion": "2016-01-01", "type": "Microsoft.Storage/storageAccounts",
"name": "mystorageaccount", "location": "westus", "sku": { "name": "Standard_LRS" }, "kind": "Storage", "properties": { } } ]
Converted to REST calls PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/ {resourceGroupName}/providers/Microsoft.Storage/storageAccounts/mystorageaccount?api- version=2016-01-01 { "location":
"westus", "properties": { } "sku": { "name": "Standard_LRS" }, "kind": "Storage" }
Tracked Deployment
AWS CloudFormation: YAML Resources: S3BucketForURLs: Type: "AWS::S3::Bucket" DeletionPolicy: Delete Properties:
BucketName: !If [ "CreateNewBucket", !Ref "AWS::NoValue", !Ref S3BucketName ] WebsiteConfiguration: IndexDocument: "index.html" LifecycleConfiguration: Rules: - Id: DisposeShortUrls ExpirationInDays: !Ref URLExpiration Prefix: "u" Status: Enabled
AWS CloudFormation Nice Features
Resource Graphs
Describes a list of Resources
Resource Graphs
Challenges
More Challenges
Terraform
Made by HashiCorp Also: Vagrant, Packer, Vault, Consul Written in
Go, open source MPL 2.0 Since 2014 Terraform
resource "azurerm_storage_account" "sa" { name = "mysa" location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name account_tier = "Standard" account_replication_type = "LRS" } Custom DSL: HCL (HashiCorp Configuration Language)
• • • • • Providers • • • •
•
Client-driven Engine Azure Resource Manager Azure Go SDK Azure TF
Provider TF CLI HCL
resource "azurerm_function_app" "app" { name = var.app_name location = azurerm_resource_group.rg.location
resource_group_name = azurerm_resource_group.rg.name app_service_plan_id = azurerm_app_service_plan.asp.id storage_connection_string = azurerm_storage_account.sa.primary_connection_string } Strong in dependency management
terraform plan # Creates a draft of changes, does nothing
to the infra terraform apply # Rolls the infra from its current state to the desired one terraform destroy # Deletes all resources in the current workspace Workflow: Plan, Apply, Destroy
> terraform plan Terraform will perform the following actions: #
azurerm_storage_account.sa must be replaced -/+ resource "azurerm_storage_account" "sa" { ... ~ location = "westus" -> "westus2" # forces replacement ... Preview Changes
Serverless
Serverless URL Shortener AWS DynamoDB Table “URLs” AWS Lambda “Add
URL” AWS Lambda “Open URL”
AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs”
API Gateway S3 Bucket Static site
AWS Resources Lambda “Add URL” Lambda “Open URL” DynamoDB “URLs”
API Gateway S3 Bucket Static site Stage Deployment REST endpoint Permissions Permissions Policy Bucket Objects
Need for higher abstraction and reusable components KV Store Table
“URLs” Function “Add URL” Function “Open URL”
Cloud- Native Ecosystem
How?
Serverless Framework
Made by Serverless, Inc. Tooling for Serverless Applications Since 2015
Serverless Framework
service: serverless-simple-http-endpoint provider: name: aws runtime: nodejs8.10 functions: currentTime: handler:
handler.endpoint events: - http: path: ping method: get YAML Definition Files
Focus on target scenarios
Pulumi
Made by Pulumi Corp. Written in Go, open source Apache
License 2.0 Since 2018 Pulumi
General-Purpose Programming Languages
• • • • • Providers • • • •
•
const storageAccount = new azure.storage.Account("storage", { resourceGroupName: resourceGroup.name, accountReplicationType: "LRS",
accountTier: "Standard", }); TypeScript example
How Pulumi works CLI and Engine Last deployed state index.ts
Language host AWS Azure GCP Kubernetes new Resource() Create, update, delete
Workflow: Preview, Up, Destroy
Workflow: Preview, Up, Destroy
Benefits of Real Code
const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id",
type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id",
type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
const counterTable = new aws.dynamodb.Table("url-table", { attributes: [{ name: "id",
type: "S" }], hashKey: "id", readCapacity: 5, }); const lambda = new aws.lambda.Function("mylambda", { code: new pulumi.asset.FileArchive("./app"), handler: "index.handler", }); const endpoint = new awsx.apigateway.API("short-url", { routes: [{ path: "/{route+}", method: "GET", eventHandler: lambda, }], }); URL Shortener
AWS Cloud Development Kit (CDK)
м TypeScript Python Java .NET Compiles to CloudFormation
const hello = new lambda.Function(this, 'HelloHandler', { runtime: lambda.Runtime.NODEJS_8_10, code:
lambda.Code.asset('lambda'), handler: 'hello.handler' }); new apigw.LambdaRestApi(this, 'Endpoint', { handler: hello }); AWS CDK example
// This is F# let myStorage = storageAccount { name
"mystorage" // set account name sku Storage.Sku.PremiumLRS // use Premium LRS } Azure? ARM Generators (ex: Farmer)
Architecture as Code
Zip Incoming Reports
const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips");
tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks
const tpsReports = new aws.s3.Bucket("tpsReports"); const tpsZips = new aws.s3.Bucket("tpsZips");
tpsReports.onObjectCreated("zipTpsReports", (e) => { for (const rec of e.Records || []) { const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ]; const data = await s3.getObject({ Bucket: buck, Key: key })); zip.addFile(key, data.Body); await s3.putObject({ Bucket: tpsZips.bucket.get(), Key: `${key}.zip`, Body: zip.toBuffer(), }); } }); Functions = Callbacks
Scheduled Tasks
const ecsScheduledTask = new ScheduledEc2Task(stack, 'ScheduledTask', { schedule: events.Schedule.expression('rate(1 minute)')
scheduledEc2TaskImageOptions: { image: ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'), memoryLimitMiB: 256, environment: { name: 'TRIGGER', value: 'CloudWatch Events' }, }, }); Scheduled Tasks
Global Apps
export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope",
"WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps
export const functions = new CosmosApp("urls", { resourceGroup, locations: ["WestEurope",
"WestUS", "SouthEastAsia"], factory: ({location, cosmosdb}) => { const app = new azure.appservice.ArchiveFunctionApp("app", { location, archive: new pulumi.asset.FileArchive("./app"), appSettings: { COSMOSDB_ENDPOINT: cosmosdb.endpoint, }, }); return { id: app.functionApp.id }; }}); Global Apps
Conclusions
USE INFRASTRUCTURE- AS-CODE Making Cloud Apps?
Infrastructure as Code Tools: Tradeoffs
Fine-grained components Better Automation
In Search for Powerful Abstractions
None