Azure Infrastructure as C# and F#

Azure cloud offers amazing capabilities for application developers. Cloud applications utilize multiple services and consist of many components, so they are hard to manage without employing Infrastructure as Code. Traditional tools like CLI scripts, ARM templates, and Terraform use text-based formats, which tend to be tedious, repetitive, and cumbersome to reuse. What if instead of configuration files you could use your favorite programming languages like C# or F#? See how you can bring your developer tools like code completion, types, components, and abstractions to cloud infrastructure definition.

Transcript

3. DevOps in Cloud age Developer Cloud Infrastructure User IT
4. About me
5. Agenda
6. Agenda
7. Agenda
8. Cloud Automation 49
9. Unified Management API Management API (Azure Resource Manager) Cloud Service Cloud Service Cloud Service Web Portal CLI SDK 3rd Party
10. REST calls PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/ {resourceGroupName}/providers/Microsoft.Storage/storageAccounts/mystorageaccount ?api-version=2016-01-01 { "location": "westus", "properties": { } "sku": { "name": "Standard_LRS" }, "kind": "Storage" }
11. var storageAccount = az.StorageAccounts.Define(storageAccountName) .WithRegion(Region.US_EAST) .WithNewResourceGroup(resourceGroupName) .Create(); SDKs to call those REST endpoints
12. az storage account create \ --location westus \ --name samplesa \ --resource-group myrg \ --sku LRS Command-line Tools
13. Step-by-step provisioning process Depend on both the current and the target state How to handle failures? Imperative Complex Error-prone SDK and Scripting: Challenges
14. Desired State Configuration 51
15. Desired State Configuration Target Current Tool
16. Managing Resource Graphs Target Current Tool
17. Be able to describe the desired state in full, concisely, and correctly Determine the correct order of creation and destruction Preview the changes before running the deployment Expressiveness Correctness Transparency Challenges
18. ARM Templates: JSON "resources": [ { "apiVersion": "2016-01-01", "type": "Microsoft.Storage/storageAccounts", "name": "mystorageaccount", "location": "westus", "sku": { "name": "Standard_LRS" }, "kind": "Storage", "properties": { } } ]
19. resource "azurerm_storage_account" "sa" { name = "mysa" location = azurerm_resource_group.rg.location resource_group_name = azurerm_resource_group.rg.name account_tier = "Standard" account_replication_type = "LRS" } Terraform: DSL (HashiCorp Configuration Language)
20. Lack of fast feedback about errors Which options need to be filled and which values are possible Thousands of lines of markup are hard to write, read, and navigate Tooling Discovery Size Markup Authoring Experience
21. Example: Code to infrastructure ratio Storage Table “URLs” Function “Add URL” Function “Open URL” Lines of code Function Code ~20 ARM Templates ~160 Terraform ~140 Estimates based on public examples
22. { "condition": "[equals(parameters('production'), 'Yes')]", "type": "Microsoft.Compute/availabilitySets", "apiVersion": "2017-03-30", "name": "[variables('availabilitySetName')]", "location": "[resourceGroup().location]", "properties": { "platformFaultDomainCount": 2, "platformUpdateDomainCount": 3 } } Conditionals Example for Azure RM Templates
23. locals { files = ["index.php","main.php"] } resource "aws_s3_bucket_object" "files" { count = "${length(local.files)}" key = "${local.files[count.index]}" bucket = "${aws_s3_bucket.examplebucket.id}" source = "./src/${local.files[count.index]}" } Loops Example for Terraform
24. Developer Tools
25. Pulumi 56
26. General-Purpose Programming Languages Node.js Go Python .NET
27. Providers
28. var resourceGroup = new ResourceGroup("rg"); var storageAccount = new Account("storage", new AccountArgs { ResourceGroupName = resourceGroup.Name, AccountReplicationType = "LRS", AccountTier = "Standard", }); C# example
29. Still, Desired State! var resourceGroup = new ResourceGroup("rg"); var storageAccount = new Account("storage", new AccountArgs { ResourceGroupName = resourceGroup.Name, AccountReplicationType = "LRS", AccountTier = "Standard", });
30. How Pulumi works CLI & engine Last deployed state Program.cs Language host Azure AWS GCP Kubernetes new Resource() CRUD
31. Static Website on Azure 59
32. Components: Serverless Functions 04
33. Resources to Provision an Azure Function Blob Plan Container Storage Account Function App
34. Component Resources
35. // A resource group to contain our Azure Functions var resourceGroup = new ResourceGroup("functions-rg"); // A .NET Azure Function var dotnet = new ArchiveFunctionApp("http", new ArchiveFunctionAppArgs { ResourceGroup = resourceGroup, Archive = new FileArchive("./dotnet/bin/Debug/netcoreapp3.1/publish") }); .NET Azure Function
36. Let’s run pulumi up … Type Name Plan + pulumi:pulumi:Stack functions-dev create + ├─ azure:appservice:ArchiveFunctionApp http create + │ ├─ azure:storage:Account http create + │ ├─ azure:appservice:Plan http create + │ ├─ azure:storage:Container http create + │ ├─ azure:storage:ZipBlob http create + │ └─ azure:appservice:FunctionApp http create + └─ azure:core:ResourceGroup rg create
37. // A Node.js Azure Function var js = new ArchiveFunctionApp("js", new ArchiveFunctionAppArgs { ResourceGroup = resourceGroup, Archive = new FileArchive("./javascript"), Runtime = "node" }); Node.js Azure Function
38. var linuxResourceGroup = new ResourceGroup("linux-rg"); var linuxPlan = new Plan("linux-asp", new PlanArgs { ResourceGroupName = linuxResourceGroup.Name, Kind = "Linux", Reserved = true, Sku = new PlanSkuArgs { Tier = "Dynamic", Size = "Y1" } }); var python = new ArchiveFunctionApp(...); Python Azure Function on Linux
39. var linuxResourceGroup = new ResourceGroup("linux-rg"); var linuxPlan = new Plan("linux-asp", new PlanArgs { ... }); var python = new ArchiveFunctionApp("py", new ArchiveFunctionAppArgs { ResourceGroup = linuxResourceGroup, Archive = new FileArchive("./python"), Plan = linuxPlan, Runtime = "python" }); Python Azure Function on Linux
40. Cosmos App 08
41. Cosmos App (1)
42. Cosmos App (2)
43. Cosmos App (3)
44. Cosmos App (4)
45. var app = new CosmosApp("functions", new CosmosAppArgs { ResourceGroup = resourceGroup, Locations = ["WestUS", "EastUS", "WestEurope"], Factory = global => region => { var func = new ArchiveFunctionApp("app", new ArchiveFunctionAppArgs { ResourceGroupName = resourceGroup.Name, Location = region.Location, Archive = new FileArchive("./app/bin/Debug/netcoreapp2.2/publish"), AppSettings = { { "CosmosDBConnection", global.ConnectionString } } }); return new AzureEndpoint(func.AppId); } }); Global Applications
46. var app = new CosmosApp("functions", new CosmosAppArgs { ResourceGroup = resourceGroup, Locations = ["WestUS", "EastUS", "WestEurope"], Factory = global => region => { var func = new ArchiveFunctionApp("app", new ArchiveFunctionAppArgs { ResourceGroupName = resourceGroup.Name, Location = region.Location, Archive = new FileArchive("./app/bin/Debug/netcoreapp2.2/publish"), AppSettings = { { "CosmosDBConnection", global.ConnectionString } } }); return new AzureEndpoint(func.AppId); } }); Global Applications
47. Provider models 12
48. N providers, M runtimes
49. N providers, M runtimes
50. O(N*M) complexity
51. O(N+M) complexity Intermediate Representation
52. O(N+M) complexity
53. State Management and Backends 14
54. State Management CLI and Engine Last deployed state Program.cs Language host AWS Azure GCP Kubernetes new Resource() CRUD Pulumi Service Blob Storage / S3 Local disk
55. Open-source Free for any use Cross-platform Written in Go Open-source Free for any use Can be created by 1st or 3rd party Web API + portal “GitHub for infra” Free for individuals Paid plans for teams Useful but not required CLI Providers SaaS backend Pulumi components
56. Cloud credentials must be configured for CLI to run az login Tokens can be set via environment variables CLI Local logins Environment Cloud Authentication
57. Deployment process Allow developers to iterate quickly on dev environments. Gain visibility and change management with Continuous Delivery.
58. Kubernetes 17
59. Kubernetes layers Managed Kubernetes cluster (AKS) Infrastructure Resources (networking, storage, identity) Managed Service Managed Service Application Application Application
60. Organizations, Projects, and Stacks Org: acme-corp Project: vpc Stack: dev env: dev region: us-east-1 Stack: prod env: prod region: us-west-2 Project: k8s-cluster Stack: dev env: dev region: us-east-1 Stack: prod env: prod region: us-west-2 Project: svc-userprofile Stack: dev env: dev region: us-east-1 Stack: prod env: prod region: us-west-2 Project: svc-email Stack: dev env: dev region: us-east-1 Stack: prod env: prod region: us-west-2
61. Stack References Org: acme-corp vpc Stack: dev env: dev region: us-east-1 k8s-cluster Stack: dev env: dev region: us-east-1 svc-userprofile Stack: dev env: dev region: us-east-1 svc-email Stack: dev env: dev region: us-east-1
62. Advanced scenarios 19
63. F# 22
64. let storageAccount = Account("sa", AccountArgs (ResourceGroupName = io resourceGroup.Name, AccountReplicationType = input "LRS", AccountTier = input "Standard")) let sku = PlanSkuArgs(Tier = input "Basic", Size = input "B1") let appServicePlan = Plan("asp", PlanArgs (ResourceGroupName = io resourceGroup.Name, Kind = input "App", Sku = input sku)) Resources in F#
65. let d = deployment "mydep" { deploymentSpec { replicas 3 selector { matchLabels appLabels } podTemplateSpec { metadata { labels appLabels } podSpec { container { name "nginx" image "nginx" containerPort 80 } } } } } Computation Expression DSL? https://github.com/pulumi/pulumi/issues/3644
66. Quality Control 24
67. [Fact] public async Task InstancesMustNotHaveSshPortOpenToInternet() { var result = await Deployment.TestAsync<MyStack>(); var resource = result.Resources.OfType<SecurityGroup>().Single(); var ingress = await resource.GetAsync(sg => sg.Ingress); foreach (var rule in ingress) foreach (var cidrBlock in rule.CidrBlocks) { Assert.False(rule.FromPort == 22 && cidrBlock == "0.0.0.0/0", "Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0)"); } } Unit Testing
68. [Fact] public async Task InstancesMustNotHaveSshPortOpenToInternet() { var result = await Deployment.TestAsync<MyStack>(); var resource = result.Resources.OfType<SecurityGroup>().Single(); var ingress = await resource.GetAsync(sg => sg.Ingress); foreach (var rule in ingress) foreach (var cidrBlock in rule.CidrBlocks) { Assert.False(rule.FromPort == 22 && cidrBlock == "0.0.0.0/0", "Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0)"); } } Unit Testing
69. [Fact] public async Task InstancesMustNotHaveSshPortOpenToInternet() { var result = await Deployment.TestAsync<MyStack>(); var resource = result.Resources.OfType<SecurityGroup>().Single(); var ingress = await resource.GetAsync(sg => sg.Ingress); foreach (var rule in ingress) foreach (var cidrBlock in rule.CidrBlocks) { Assert.False(rule.FromPort == 22 && cidrBlock == "0.0.0.0/0", "Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0)"); } } Unit Testing
70. Policy as Code (.NET coming later) const policies = new PolicyPack("aws", { policies: [ { name: "prohibited-public-internet", description: "Ingress rules with public internet access are prohibited.", enforcementLevel: "mandatory", validateResource: validateTypedResource(aws.ec2.SecurityGroup, (sg, args, report) => { const publicInternetRules = (sg.ingress || []).find(ingressRule => (ingressRule.cidrBlocks || []).find(cidr => cidr === "0.0.0.0/0")); if (publicInternetRules) { report("Ingress rules with public internet access are prohibited."); } }), }, ], });
71. Conclusions 27
72. Working on Cloud Apps? Define your infrastructure as code!
73. Fine-grained components Better Automation
74. Familiar language, experience, toolchain, packages Conditionals, loops, functions, classes, components, modules Reusable components that encapsulate complex logic and provide the right level of abstraction Developer-friendly Logic Abstractions Benefits of Real Code
75. м Infrastructure as Code Architecture as Code
76. Useful Links http://bit.ly/pulumilinks