Tweet
Tweet

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

DevOps in Cloud age Developer Cloud Infrastructure User IT

Slide 4

Slide 4 text

About me

Slide 5

Slide 5 text

Agenda

Slide 6

Slide 6 text

Agenda

Slide 7

Slide 7 text

Agenda

Slide 8

Slide 8 text

Cloud Automation 49

Slide 9

Slide 9 text

Unified Management API Management API (Azure Resource Manager) Cloud Service Cloud Service Cloud Service Web Portal CLI SDK 3rd Party

Slide 10

Slide 10 text

REST calls PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/ {resourceGroupName}/providers/Microsoft.Storage/storageAccounts/mystorageaccount ?api-version=2016-01-01 { "location": "westus", "properties": { } "sku": { "name": "Standard_LRS" }, "kind": "Storage" }

Slide 11

Slide 11 text

var storageAccount = az.StorageAccounts.Define(storageAccountName) .WithRegion(Region.US_EAST) .WithNewResourceGroup(resourceGroupName) .Create(); SDKs to call those REST endpoints

Slide 12

Slide 12 text

az storage account create \ --location westus \ --name samplesa \ --resource-group myrg \ --sku LRS Command-line Tools

Slide 13

Slide 13 text

Step-by-step provisioning process Depend on both the current and the target state How to handle failures? Imperative Complex Error-prone SDK and Scripting: Challenges

Slide 14

Slide 14 text

Desired State Configuration 51

Slide 15

Slide 15 text

Desired State Configuration Target Current Tool

Slide 16

Slide 16 text

Managing Resource Graphs Target Current Tool

Slide 17

Slide 17 text

Be able to describe the desired state in full, concisely, and correctly Determine the correct order of creation and destruction Preview the changes before running the deployment Expressiveness Correctness Transparency Challenges

Slide 18

Slide 18 text

ARM Templates: JSON "resources": [ { "apiVersion": "2016-01-01", "type": "Microsoft.Storage/storageAccounts", "name": "mystorageaccount", "location": "westus", "sku": { "name": "Standard_LRS" }, "kind": "Storage", "properties": { } } ]

Slide 19

Slide 19 text

resource "azurerm_storage_account" "sa" { name = "mysa" location = azurerm_resource_group.rg.location resource_group_name = azurerm_resource_group.rg.name account_tier = "Standard" account_replication_type = "LRS" } Terraform: DSL (HashiCorp Configuration Language)

Slide 20

Slide 20 text

Lack of fast feedback about errors Which options need to be filled and which values are possible Thousands of lines of markup are hard to write, read, and navigate Tooling Discovery Size Markup Authoring Experience

Slide 21

Slide 21 text

Example: Code to infrastructure ratio Storage Table “URLs” Function “Add URL” Function “Open URL” Lines of code Function Code ~20 ARM Templates ~160 Terraform ~140 Estimates based on public examples

Slide 22

Slide 22 text

{ "condition": "[equals(parameters('production'), 'Yes')]", "type": "Microsoft.Compute/availabilitySets", "apiVersion": "2017-03-30", "name": "[variables('availabilitySetName')]", "location": "[resourceGroup().location]", "properties": { "platformFaultDomainCount": 2, "platformUpdateDomainCount": 3 } } Conditionals Example for Azure RM Templates

Slide 23

Slide 23 text

locals { files = ["index.php","main.php"] } resource "aws_s3_bucket_object" "files" { count = "${length(local.files)}" key = "${local.files[count.index]}" bucket = "${aws_s3_bucket.examplebucket.id}" source = "./src/${local.files[count.index]}" } Loops Example for Terraform

Slide 24

Slide 24 text

Developer Tools

Slide 25

Slide 25 text

Pulumi 56

Slide 26

Slide 26 text

General-Purpose Programming Languages Node.js Go Python .NET

Slide 27

Slide 27 text

Providers

Slide 28

Slide 28 text

var resourceGroup = new ResourceGroup("rg"); var storageAccount = new Account("storage", new AccountArgs { ResourceGroupName = resourceGroup.Name, AccountReplicationType = "LRS", AccountTier = "Standard", }); C# example

Slide 29

Slide 29 text

Still, Desired State! var resourceGroup = new ResourceGroup("rg"); var storageAccount = new Account("storage", new AccountArgs { ResourceGroupName = resourceGroup.Name, AccountReplicationType = "LRS", AccountTier = "Standard", });

Slide 30

Slide 30 text

How Pulumi works CLI & engine Last deployed state Program.cs Language host Azure AWS GCP Kubernetes new Resource() CRUD

Slide 31

Slide 31 text

Static Website on Azure 59

Slide 32

Slide 32 text

Components: Serverless Functions 04

Slide 33

Slide 33 text

Resources to Provision an Azure Function Blob Plan Container Storage Account Function App

Slide 34

Slide 34 text

Component Resources

Slide 35

Slide 35 text

// A resource group to contain our Azure Functions var resourceGroup = new ResourceGroup("functions-rg"); // A .NET Azure Function var dotnet = new ArchiveFunctionApp("http", new ArchiveFunctionAppArgs { ResourceGroup = resourceGroup, Archive = new FileArchive("./dotnet/bin/Debug/netcoreapp3.1/publish") }); .NET Azure Function

Slide 36

Slide 36 text

Let’s run pulumi up … Type Name Plan + pulumi:pulumi:Stack functions-dev create + ├─ azure:appservice:ArchiveFunctionApp http create + │ ├─ azure:storage:Account http create + │ ├─ azure:appservice:Plan http create + │ ├─ azure:storage:Container http create + │ ├─ azure:storage:ZipBlob http create + │ └─ azure:appservice:FunctionApp http create + └─ azure:core:ResourceGroup rg create

Slide 37

Slide 37 text

// A Node.js Azure Function var js = new ArchiveFunctionApp("js", new ArchiveFunctionAppArgs { ResourceGroup = resourceGroup, Archive = new FileArchive("./javascript"), Runtime = "node" }); Node.js Azure Function

Slide 38

Slide 38 text

var linuxResourceGroup = new ResourceGroup("linux-rg"); var linuxPlan = new Plan("linux-asp", new PlanArgs { ResourceGroupName = linuxResourceGroup.Name, Kind = "Linux", Reserved = true, Sku = new PlanSkuArgs { Tier = "Dynamic", Size = "Y1" } }); var python = new ArchiveFunctionApp(...); Python Azure Function on Linux

Slide 39

Slide 39 text

var linuxResourceGroup = new ResourceGroup("linux-rg"); var linuxPlan = new Plan("linux-asp", new PlanArgs { ... }); var python = new ArchiveFunctionApp("py", new ArchiveFunctionAppArgs { ResourceGroup = linuxResourceGroup, Archive = new FileArchive("./python"), Plan = linuxPlan, Runtime = "python" }); Python Azure Function on Linux

Slide 40

Slide 40 text

Cosmos App 08

Slide 41

Slide 41 text

Cosmos App (1)

Slide 42

Slide 42 text

Cosmos App (2)

Slide 43

Slide 43 text

Cosmos App (3)

Slide 44

Slide 44 text

Cosmos App (4)

Slide 45

Slide 45 text

var app = new CosmosApp("functions", new CosmosAppArgs { ResourceGroup = resourceGroup, Locations = ["WestUS", "EastUS", "WestEurope"], Factory = global => region => { var func = new ArchiveFunctionApp("app", new ArchiveFunctionAppArgs { ResourceGroupName = resourceGroup.Name, Location = region.Location, Archive = new FileArchive("./app/bin/Debug/netcoreapp2.2/publish"), AppSettings = { { "CosmosDBConnection", global.ConnectionString } } }); return new AzureEndpoint(func.AppId); } }); Global Applications

Slide 46

Slide 46 text

var app = new CosmosApp("functions", new CosmosAppArgs { ResourceGroup = resourceGroup, Locations = ["WestUS", "EastUS", "WestEurope"], Factory = global => region => { var func = new ArchiveFunctionApp("app", new ArchiveFunctionAppArgs { ResourceGroupName = resourceGroup.Name, Location = region.Location, Archive = new FileArchive("./app/bin/Debug/netcoreapp2.2/publish"), AppSettings = { { "CosmosDBConnection", global.ConnectionString } } }); return new AzureEndpoint(func.AppId); } }); Global Applications

Slide 47

Slide 47 text

Provider models 12

Slide 48

Slide 48 text

N providers, M runtimes

Slide 49

Slide 49 text

N providers, M runtimes

Slide 50

Slide 50 text

O(N*M) complexity

Slide 51

Slide 51 text

O(N+M) complexity Intermediate Representation

Slide 52

Slide 52 text

O(N+M) complexity

Slide 53

Slide 53 text

State Management and Backends 14

Slide 54

Slide 54 text

State Management CLI and Engine Last deployed state Program.cs Language host AWS Azure GCP Kubernetes new Resource() CRUD Pulumi Service Blob Storage / S3 Local disk

Slide 55

Slide 55 text

Open-source Free for any use Cross-platform Written in Go Open-source Free for any use Can be created by 1st or 3rd party Web API + portal “GitHub for infra” Free for individuals Paid plans for teams Useful but not required CLI Providers SaaS backend Pulumi components

Slide 56

Slide 56 text

Cloud credentials must be configured for CLI to run az login Tokens can be set via environment variables CLI Local logins Environment Cloud Authentication

Slide 57

Slide 57 text

Deployment process Allow developers to iterate quickly on dev environments. Gain visibility and change management with Continuous Delivery.

Slide 58

Slide 58 text

Kubernetes 17

Slide 59

Slide 59 text

Kubernetes layers Managed Kubernetes cluster (AKS) Infrastructure Resources (networking, storage, identity) Managed Service Managed Service Application Application Application

Slide 60

Slide 60 text

Organizations, Projects, and Stacks Org: acme-corp Project: vpc Stack: dev env: dev region: us-east-1 Stack: prod env: prod region: us-west-2 Project: k8s-cluster Stack: dev env: dev region: us-east-1 Stack: prod env: prod region: us-west-2 Project: svc-userprofile Stack: dev env: dev region: us-east-1 Stack: prod env: prod region: us-west-2 Project: svc-email Stack: dev env: dev region: us-east-1 Stack: prod env: prod region: us-west-2

Slide 61

Slide 61 text

Stack References Org: acme-corp vpc Stack: dev env: dev region: us-east-1 k8s-cluster Stack: dev env: dev region: us-east-1 svc-userprofile Stack: dev env: dev region: us-east-1 svc-email Stack: dev env: dev region: us-east-1

Slide 62

Slide 62 text

Advanced scenarios 19

Slide 63

Slide 63 text

F# 22

Slide 64

Slide 64 text

let storageAccount = Account("sa", AccountArgs (ResourceGroupName = io resourceGroup.Name, AccountReplicationType = input "LRS", AccountTier = input "Standard")) let sku = PlanSkuArgs(Tier = input "Basic", Size = input "B1") let appServicePlan = Plan("asp", PlanArgs (ResourceGroupName = io resourceGroup.Name, Kind = input "App", Sku = input sku)) Resources in F#

Slide 65

Slide 65 text

let d = deployment "mydep" { deploymentSpec { replicas 3 selector { matchLabels appLabels } podTemplateSpec { metadata { labels appLabels } podSpec { container { name "nginx" image "nginx" containerPort 80 } } } } } Computation Expression DSL? https://github.com/pulumi/pulumi/issues/3644

Slide 66

Slide 66 text

Quality Control 24

Slide 67

Slide 67 text

[Fact] public async Task InstancesMustNotHaveSshPortOpenToInternet() { var result = await Deployment.TestAsync(); var resource = result.Resources.OfType().Single(); var ingress = await resource.GetAsync(sg => sg.Ingress); foreach (var rule in ingress) foreach (var cidrBlock in rule.CidrBlocks) { Assert.False(rule.FromPort == 22 && cidrBlock == "0.0.0.0/0", "Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0)"); } } Unit Testing

Slide 68

Slide 68 text

[Fact] public async Task InstancesMustNotHaveSshPortOpenToInternet() { var result = await Deployment.TestAsync(); var resource = result.Resources.OfType().Single(); var ingress = await resource.GetAsync(sg => sg.Ingress); foreach (var rule in ingress) foreach (var cidrBlock in rule.CidrBlocks) { Assert.False(rule.FromPort == 22 && cidrBlock == "0.0.0.0/0", "Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0)"); } } Unit Testing

Slide 69

Slide 69 text

[Fact] public async Task InstancesMustNotHaveSshPortOpenToInternet() { var result = await Deployment.TestAsync(); var resource = result.Resources.OfType().Single(); var ingress = await resource.GetAsync(sg => sg.Ingress); foreach (var rule in ingress) foreach (var cidrBlock in rule.CidrBlocks) { Assert.False(rule.FromPort == 22 && cidrBlock == "0.0.0.0/0", "Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0)"); } } Unit Testing

Slide 70

Slide 70 text

Policy as Code (.NET coming later) const policies = new PolicyPack("aws", { policies: [ { name: "prohibited-public-internet", description: "Ingress rules with public internet access are prohibited.", enforcementLevel: "mandatory", validateResource: validateTypedResource(aws.ec2.SecurityGroup, (sg, args, report) => { const publicInternetRules = (sg.ingress || []).find(ingressRule => (ingressRule.cidrBlocks || []).find(cidr => cidr === "0.0.0.0/0")); if (publicInternetRules) { report("Ingress rules with public internet access are prohibited."); } }), }, ], });

Slide 71

Slide 71 text

Conclusions 27

Slide 72

Slide 72 text

Working on Cloud Apps? Define your infrastructure as code!

Slide 73

Slide 73 text

Fine-grained components Better Automation

Slide 74

Slide 74 text

Familiar language, experience, toolchain, packages Conditionals, loops, functions, classes, components, modules Reusable components that encapsulate complex logic and provide the right level of abstraction Developer-friendly Logic Abstractions Benefits of Real Code

Slide 75

Slide 75 text

м Infrastructure as Code Architecture as Code

Slide 76

Slide 76 text

Useful Links http://bit.ly/pulumilinks