Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Azure Infrastructure as C# and F#

Azure Infrastructure as C# and F#

Azure cloud platform offers amazing capabilities for application developers. Cloud apps utilize multiple services and consist of many components, so they are hard to manage without employing Infrastructure as Code.

Traditional tools like ARM templates and Terraform use text-based formats, which tend to be tedious, repetitive, and cumbersome to reuse.

What if instead of configuration files you could use your favourite programming languages like C# or F#? See how you can bring your developer tools like code completion, types, components, and abstractions to cloud infrastructure definition.

Mikhail Shilkov

March 31, 2020
Tweet

More Decks by Mikhail Shilkov

Other Decks in Programming

Transcript

  1. • Pulumi: Modern Infrastructure as Code • Open source and

    free • Use .NET languages to manage cloud resources • Desired state configuration • Tools that you love • Reusable abstractions • Automated testing • Multi-cloud Agenda
  2. About me • Software engineer at Pulumi .NET SDK, Azure,

    Core platform • Serverless • Functional programming, F# • Microsoft Azure MVP @MikhailShilkov [email protected] https://mikhail.io
  3. Providers • Azure • AWS • GCP • Digital Ocean

    • Cloudflare … and more • Docker • Kubernetes • OpenStack • PostgreSQL • New Relic
  4. Unified Management API Management API (Azure Resource Manager) Cloud Service

    Cloud Service Cloud Service Web Portal CLI SDK 3rd Party
  5. az storage account create \ --location westus \ --name samplesa

    \ --resource-group myrg \ --sku LRS Command-line Tools
  6. SDK and Scripting: Challenges 14 Imperative Step-by-step provisioning process Complex

    Depend on both the current and the target state Error-prone How to handle failures?
  7. Markup-based Languages 17 AWS CloudFormation YAML Azure RM Templates JSON

    Terraform HCL "resources": [ { "apiVersion": "2016-01-01", "type": "Microsoft.Storage/storageAccounts", "name": "mystorageaccount", "location": "westus", "sku": { "name": "Standard_LRS" }, "kind": "Storage", "properties": { } } ] Resources: S3BucketForURLs: Type: "AWS::S3::Bucket" DeletionPolicy: Delete Properties: BucketName: !If [ "CreateNewBucket", !Ref WebsiteConfiguration: IndexDocument: "index.html" LifecycleConfiguration: Rules: - Id: DisposeShortUrls ExpirationInDays: !Ref URLExpiration Prefix: "u" Status: Enabled resource "azurerm_storage_account" "sa" { name = "mysa" location = azurerm_resource_group .rg.location resource_group_name = azurerm_resource_group .rg.name account_tier = "Standard" account_replication_type = "LRS" } Every tool has its own language, own tools, own conventions
  8. Markup Authoring Experience 18 Tooling Lack of tooling on day

    1. No fast feedback from editors. Discovery Which properties can be set, and which values are valid? Size Thousands of lines of markup are hard to write, read, and maintain
  9. Example App: Serverless URL Shortener KV Store Table “URLs” Function

    “Add URL” Function “Open URL” Tool Lines of code Function Code ~20 CloudFormation ~240 ARM Templates ~160 Terraform ~220 Estimates based on public examples
  10. var resourceGroup = new ResourceGroup("rg"); var storageAccount = new Account("storage",

    new AccountArgs { ResourceGroupName = resourceGroup.Name, AccountReplicationType = "LRS", AccountTier = "Standard" }); C# Example
  11. Desired State! var resourceGroup = new ResourceGroup("rg"); var storageAccount =

    new Account("storage", new AccountArgs { ResourceGroupName = resourceGroup.Name, AccountReplicationType = "LRS", AccountTier = "Standard" });
  12. How Pulumi Works CLI & engine Last deployed state Program.cs

    Language host AWS Azure GCP Kubernetes new Resource() CRUD
  13. Pulumi ❤ .NET • .NET Core 3.1 • OS: Windows,

    macOS, Linux • Language: C#, F#, VB.NET • Editor and IDE: Visual Studio, Code, Rider • IntelliSense, ReSharper, StyleCop, DocFX • Package Manager: NuGet, MyGet, Paket • NUnit, xUnit.net, Moq
  14. var storageAccount = new Account("sa", new AccountArgs { ResourceGroupName =

    resourceGroup.Name, AccountReplicationType = "LRS", AccountTier = "Standard" }); var appServicePlan = new Plan("asp", new PlanArgs { ResourceGroupName = resourceGroup.Name, Kind = "App", Sku = new PlanSkuArgs { Tier = "Basic", Size = "B1” } }); Cloud Resources with C#
  15. let storageAccount = Account("sa", AccountArgs (ResourceGroupName = io resourceGroup.Name, AccountReplicationType

    = input "LRS", AccountTier = input "Standard")) let sku = PlanSkuArgs(Tier = input "Basic", Size = input "B1") let appServicePlan = Plan("asp", PlanArgs (ResourceGroupName = io resourceGroup.Name, Kind = input "App", Sku = input sku)) Cloud Resources with F#
  16. ' Create an Azure Resource Group Dim resourceGroup = New

    ResourceGroup("resourceGroup") ' Create an Azure Storage Account Dim storageAccount = New Account("storage", New AccountArgs With { .ResourceGroupName = resourceGroup.Name, .AccountReplicationType = "LRS", .AccountTier = "Standard" }) Cloud Resources with VB.NET
  17. // A resource group to contain our Azure Functions var

    resourceGroup = new ResourceGroup("functions-rg"); // A .NET Azure Function var dotnet = new ArchiveFunctionApp("http", new ArchiveFunctionAppArgs { ResourceGroup = resourceGroup, Archive = new FileArchive("./dotnet/bin/Debug/netcoreapp3.1/publish") }); .NET Azure Function
  18. Let’s run pulumi up … Type Name Plan + pulumi:pulumi:Stack

    functions-dev create + ├─ azure:appservice:ArchiveFunctionApp http create + │ ├─ azure:storage:Account http create + │ ├─ azure:appservice:Plan http create + │ ├─ azure:storage:Container http create + │ ├─ azure:storage:ZipBlob http create + │ └─ azure:appservice:FunctionApp http create + └─ azure:core:ResourceGroup rg create
  19. var app = new CosmosApp("functions", new CosmosAppArgs { ResourceGroup =

    resourceGroup, Locations = ["WestUS", "EastUS", "WestEurope"], Factory = global => region => { var func = new ArchiveFunctionApp("app", new ArchiveFunctionAppArgs { ResourceGroupName = resourceGroup.Name, Location = region.Location, Archive = new FileArchive("./app/bin/Debug/netcoreapp2.2/publish"), AppSettings = { { "CosmosDBConnection", global.ConnectionString } } }); return new AzureEndpoint(func.AppId); } }); Global Applications
  20. var app = new CosmosApp("functions", new CosmosAppArgs { ResourceGroup =

    resourceGroup, Locations = ["WestUS", "EastUS", "WestEurope"], Factory = global => region => { var func = new ArchiveFunctionApp("app", new ArchiveFunctionAppArgs { ResourceGroupName = resourceGroup.Name, Location = region.Location, Archive = new FileArchive("./app/bin/Debug/netcoreapp2.2/publish"), AppSettings = { { "CosmosDBConnection", global.ConnectionString } } }); return new AzureEndpoint(func.AppId); } }); Global Applications
  21. [Fact] public async Task InstancesMustNotHaveSshPortOpenToInternet() { var resources = await

    Deployment.TestAsync<MyStack>(); var resource = resources.OfType<SecurityGroup>().Single(); var ingress = await resource.GetAsync(sg => sg.Ingress); foreach (var rule in ingress) foreach (var cidrBlock in rule.CidrBlocks) { Assert.False(rule.FromPort == 22 && cidrBlock == "0.0.0.0/0", "Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0)"); } } Unit Testing
  22. [Fact] public async Task InstancesMustNotHaveSshPortOpenToInternet() { var resources = await

    Deployment.TestAsync<MyStack>(); var resource = resources.OfType<SecurityGroup>().Single(); var ingress = await resource.GetAsync(sg => sg.Ingress); foreach (var rule in ingress) foreach (var cidrBlock in rule.CidrBlocks) { Assert.False(rule.FromPort == 22 && cidrBlock == "0.0.0.0/0", "Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0)"); } } Unit Testing
  23. [Fact] public async Task InstancesMustNotHaveSshPortOpenToInternet() { var resources = await

    Deployment.TestAsync<MyStack>(); var resource = resources.OfType<SecurityGroup>().Single(); var ingress = await resource.GetAsync(sg => sg.Ingress); foreach (var rule in ingress) foreach (var cidrBlock in rule.CidrBlocks) { Assert.False(rule.FromPort == 22 && cidrBlock == "0.0.0.0/0", "Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0)"); } } Unit Testing
  24. Policy as Code (.NET coming later) const policies = new

    PolicyPack("network-policies", { policies: [ { name: "prohibited-public-internet", description: "Ingress rules with public internet access are prohibited.", enforcementLevel: "mandatory", validateResource: validateTypedResource(SecurityGroup, (sg, args, report) => { const publicInternetRules = (sg.ingress || []).find(ingressRule => (ingressRule.cidrBlocks || []).find(cidr => cidr === "0.0.0.0/0")); if (publicInternetRules) { report("Ingress rules with public internet access are prohibited."); } }), }, ], });
  25. Cloud Transition Kubernetes Azure Functions Azure Storage Azure Analytics Azure

    ML MySQL DataDog App Docker DataDog New Relic App MySQL EARLY CLOUD Mostly Static N-Tier Apps VMs Private Cloud CURRENT CLOUD Partly Dynamic Less Monolithic VMs and Containers* Hybrid - Public/Private *Experimentation FUTURE CLOUD Fully Dynamic Hyper-Connected Services Containers and Serverless Mostly Public Cloud
  26. Infrastructure Landscape Foundation Security IAM KMS Networking VPC Subnets Firewalls

    Load Balancing DNS Compute VMs Containers Clusters Registries APM Monitoring Logging AlerKng Serverless Functions API Gateways Data Object Stores Databases SQL NoSQL MQ Queues Pub/Sub Applications Images Container Images Code Packaging CI/CD
  27. Kubernetes layers Managed Kubernetes cluster Infrastructure Resources (networking, storage, identity)

    Managed Service Managed Service Application Application Application
  28. Stack References Org: acme-corp vpc Stack: dev env: dev region:

    us-east-1 k8s-cluster Stack: dev env: dev region: us-east-1 svc-userprofile Stack: dev env: dev region: us-east-1 svc-email Stack: dev env: dev region: us-east-1
  29. PROVISIONING Developer-friendly Familiar language experience, toolchain, packages – applied to

    cloud infrastructure. Developers and operators working in a team. Cloud Engineering Transformed TESTING Confidence and quality Unit testing and TDD with battle-tested tools to ensure correctness. Policy as Code for compliance, cost control, and company-wide best practices. ARCHITECTURE Logic and abstractions Conditionals, loops, functions, classes, and packages out of the box. Reusable components that encapsulate complex logic and provider the right level of abstraction. Modern Infrastructure as Code Capabilities to ship faster and with confidence