Cloud Superpowers with Pulumi and F#

Feeling daunted creating cloud resources by clicking around in web portals or automating your deployments with scripts, YAML files, or JSON templates?

In this talk, Mikhail will introduce Pulumi, an open-source tool that leverages programming languages to manage infrastructure, yet in a declarative manner! Using developer tools that you know and love, you can now apply unit testing, reusable architecture, and continuous delivery while provisioning resources in the cloud. Obtain the cloud superpowers while using F#!

Transcript

1. Cloud Superpowers with F#
   Mikhail Shilkov
   

   View Slide

2. About me
   • Mikhail Shilkov
   • Software engineer at Pulumi
   .NET SDK, Azure, Core platform
   • Serverless
   • Functional programming, F#
   • Microsoft Azure MVP
   @MikhailShilkov
   mikhail@pulumi.com
   

   View Slide

3. Intro
   ● Pulumi: Modern Infrastructure as Code
   ● F# to manage cloud resources
   Cloud Superpowers
   ● Provisioning
   ● Architecture
   ● Testing
   ● Delivery
   Agenda
   

   View Slide

4. General-purpose Programming Languages
   

   View Slide

5. F#!
   

   View Slide

6. Providers
   ● AWS
   ● Azure
   ● GCP
   ● Digital Ocean
   ● Cloudflare
   … and more
   ● Docker
   ● Kubernetes
   ● OpenStack
   ● PostgreSQL
   ● New Relic
   

   View Slide

7. Static Website
   on AWS
   Demo
   3
   

   View Slide

8. Provisioning
   with Desired State Configuration
   8
   

   View Slide

9. Desired State Configuration
   Target Current
   Tool
   

   View Slide

10. Managing Resource Graphs
    Target Current
    Tool
    

    View Slide

11. Tools that you love
    ● .NET Core 3.1
    ● OS: Windows, macOS, Linux
    ● Editor and IDE: Visual Studio, Code, Rider
    ● IntelliSense, ReSharper, StyleCop, DocFX
    ● Package Manager: Paket, NuGet, MyGet
    ● NUnit, xUnit.net, Moq, FsUnit, FsCheck
    

    View Slide

12. let resourceGroup = ResourceGroup "rg"
    let storageAccount =
    Account("storage",
    AccountArgs
    (ResourceGroupName = io resourceGroup.Name,
    AccountReplicationType = input "LRS",
    AccountTier = input "Standard"))
    F# Example
    

    View Slide

13. Desired
    State!
    let resourceGroup = ResourceGroup "rg"
    let storageAccount =
    Account("storage",
    AccountArgs
    (ResourceGroupName = io resourceGroup.Name,
    AccountReplicationType = input "LRS",
    AccountTier = input "Standard"))
    

    View Slide

14. How Pulumi Works
    CLI & engine
    Last
    deployed
    state
    Program.fs
    Language host
    AWS
    Azure
    GCP
    Kubernetes
    new
    Resource()
    CRUD
    

    View Slide

15. let resourceGroup = ResourceGroup "rg"
    let storageAccount =
    Account("storage",
    AccountArgs
    (ResourceGroupName = io resourceGroup.Name,
    AccountReplicationType = input "LRS",
    AccountTier = input "Standard"))
    Dependency Definitions
    

    View Slide

16. Architecture
    Reusable Abstractions
    12
    

    View Slide

17. Resources to Provision an Azure Function
    Blob
    App
    Plan
    Container
    Storage
    Account
    Function
    App
    

    View Slide

18. // A resource group to contain our Azure Functions
    let resourceGroup = ResourceGroup "functions-rg"
    // A .NET Azure Function
    let dotnet =
    ArchiveFunctionApp("http",
    ArchiveFunctionAppArgs
    (ResourceGroup = resourceGroup,
    Archive = new FileArchive("./dotnet/publish")))
    .NET Azure Function
    

    View Slide

19. Let’s run pulumi up …
    Type Name Plan
    + pulumi:pulumi:Stack functions-dev create
    + ├─ azure:appservice:ArchiveFunctionApp http create
    + │ ├─ azure:storage:Account http create
    + │ ├─ azure:appservice:Plan http create
    + │ ├─ azure:storage:Container http create
    + │ ├─ azure:storage:ZipBlob http create
    + │ └─ azure:appservice:FunctionApp http create
    + └─ azure:core:ResourceGroup rg create
    

    View Slide

20. // A resource group to contain our Azure Functions
    let resourceGroup = ResourceGroup "functions-rg"
    // A .NET Azure Function
    let dotnet =
    ArchiveFunctionApp("http",
    ArchiveFunctionAppArgs
    (ResourceGroup = resourceGroup,
    Archive = new FileArchive("./ javascript ")
    Runtime = "node"))
    Node.js Azure Function
    

    View Slide

21. let linuxResourceGroup = ResourceGroup "linux-rg"
    let linuxPlan =
    Plan("linux-asp",
    PlanArgs
    (ResourceGroupName = io linuxResourceGroup.Name,
    Kind = input "Linux",
    Reserved = input true))
    let python = ArchiveFunctionApp(...);
    Python Azure Function on Linux
    

    View Slide

22. let linuxResourceGroup = ResourceGroup "linux-rg"
    let linuxPlan = Plan("linux-asp", PlanArgs ( ... ))
    let python =
    ArchiveFunctionApp("py",
    ArchiveFunctionAppArgs
    (ResourceGroup = linuxResourceGroup,
    Archive = new FileArchive("./python"),
    Plan = linuxPlan,
    Runtime = "python"))
    Python Azure Function on Linux
    

    View Slide

23. Testing
    Validate infrastructure definition
    15
    

    View Slide

24. []
    member this.ResourceGroupHasEnvironmentTag() =
    let resources = Deployment.TestAsync() |> runSync
    let resourceGroup =
    resources.OfType() |> Seq.head
    let tags = getValue resourceGroup.Tags
    tags.Should().NotBeNull("Tags must be defined") |> ignore
    tags.Should().ContainKey("Environment", null) |> ignore
    Unit Testing
    

    View Slide

25. []
    member this.ResourceGroupHasEnvironmentTag() =
    let resources = Deployment.TestAsync() |> runSync
    let resourceGroup =
    resources.OfType() |> Seq.head
    let tags = getValue resourceGroup.Tags
    tags.Should().NotBeNull("Tags must be defined") |> ignore
    tags.Should().ContainKey("Environment", null) |> ignore
    Unit Testing
    

    View Slide

26. []
    member this.ResourceGroupHasEnvironmentTag() =
    let resources = Deployment.TestAsync() |> runSync
    let resourceGroup =
    resources.OfType() |> Seq.head
    let tags = getValue resourceGroup.Tags
    tags.Should().NotBeNull("Tags must be defined") |> ignore
    tags.Should().ContainKey("Environment", null) |> ignore
    Unit Testing
    

    View Slide

27. https://mikhail.io/2020/05/unit-testing-cloud-deployments-with-pulumi-in-fsharp/
    

    View Slide

28. Delivery
    Multi-cloud cross-stack automation
    17
    

    View Slide

29. Deployment process
    Allow developers to
    iterate quickly on dev
    environments.
    Gain visibility and change
    management with
    Continuous Delivery.
    

    View Slide

30. Cloud Transition
    Kubernetes
    Azure
    Functions
    AWS S3
    Azure
    Analytics
    Google ML
    MySQL
    DataDog
    App
    Docker
    DataDog
    New Relic
    App
    MySQL
    EARLY CLOUD
    Mostly Static
    N-Tier Apps
    VMs
    Private Cloud
    CURRENT CLOUD
    Partly Dynamic
    Less Monolithic
    VMs and Containers*
    Hybrid - Public/Private
    *Experimentation
    FUTURE CLOUD
    Fully Dynamic
    Hyper-Connected Services
    Containers and Serverless
    Mostly Public Cloud
    

    View Slide

31. Infrastructure Landscape
    Foundation
    Security
    IAM
    KMS
    Networking
    VPC
    Subnets
    Firewalls
    Load Balancing
    DNS
    Compute
    VMs
    Containers
    Clusters
    Registries
    APM
    Monitoring
    Logging
    Alerting
    Serverless
    Functions
    API Gateways
    Data
    Object Stores
    Databases
    SQL
    NoSQL
    MQ
    Queues
    Pub/Sub
    Applications
    Images Container Images Code Packaging
    CI/CD
    

    View Slide

32. Example: Kubernetes
    Managed Kubernetes cluster
    Infrastructure Resources (networking, storage, identity)
    Managed Service Managed Service
    Application Application Application
    

    View Slide

33. Conclusions
    20
    

    View Slide

34. PROVISIONING
    Developer-friendly
    Familiar language experience, toolchain,
    packages – applied to cloud infrastructure.
    Developers and operators working in a
    team.
    Cloud Engineering Transformed
    TESTING
    Confidence and quality
    Unit testing and TDD with battle-tested
    tools to ensure correctness.
    Policy as Code for compliance, cost
    control, and company-wide best
    practices.
    ARCHITECTURE
    Logic and abstractions
    Conditionals, loops, functions, classes,
    and packages out of the box.
    Reusable components that encapsulate
    complex logic and provider the right
    level of abstraction.
    Modern Infrastructure as Code Capabilities
    to ship faster and with confidence
    

    View Slide

35. Useful Links
    http://bit.ly/pulumilinks
    

    View Slide

36. Q&A
    Twitter: @MikhailShilkov
    Email: mikhail@pulumi.com
    

    View Slide