Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Principles, Patterns, and Practices for Effective Infrastructure as Code

Adarsh Shah
January 25, 2021
700

Principles, Patterns, and Practices for Effective Infrastructure as Code

Infrastructure as Code (IaC) is an approach that takes proven coding techniques used by software systems and extends it to infrastructure. It is one of the key DevOps practices that enable teams to deliver infrastructure, and the software running on it, rapidly and reliably, at scale.

In this presentation, we will look at Principles, Patterns, and Practices for Effective Infrastructure as Code that have helped me and the teams I have worked with over the years. We will look at anecdotes about deploying & operating infrastructure and applications in production.

Adarsh Shah

January 25, 2021
Tweet

Transcript

  1. Principles, Patterns, and
    Practices for Effective
    Infrastructure as Code
    Deliver Infrastructure and
    Software running on it Rapidly
    and Reliably at Scale
    Adarsh Sha
    h


    Engineering Leader, Coach, Hands-on Architec
    t


    Independent Consultan
    t


    @shahadarsh 

    https://shahadarsh.com
    Deck: https://bit.ly/IaC-FOSDEM

    View full-size slide

  2. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Infrastructure as Code
    Infrastructure as Code (IaC) is an approach that takes
    proven coding techniques used by software systems and
    extends it to infrastructure. It is one of the key DevOps
    practices that enable teams to deliver infrastructure,
    and the software running on it, rapidly and reliably,
    at scale.

    View full-size slide

  3. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Key Principles

    View full-size slide

  4. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Idempotency
    Idempotency means no matter how many times you run
    your IaC and, what your starting state is, you will end up
    with the same end state. This simpli
    f
    i
    es the provisioning
    of Infrastructure and reduces the chances of inconsistent
    results.

    View full-size slide

  5. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Non-Idempotent
    Provision
    VM VM VM
    { }
    IaC
    3 VMs
    End State

    Expected =
    3


    Actual = 6
    Idempotent
    End State

    Expected =
    3


    Actual = 3
    Reapply
    VM VM VM
    Provision
    VM VM VM
    Reapply
    No change
    { }
    IaC
    3 VMs

    View full-size slide

  6. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Immutability
    Immutable infrastructure means instead of changing
    an existing infrastructure you replace it with new. By
    provisioning new infrastructure every time, you are
    making sure it is reproducible and doesn’t allow for
    con
    f
    i
    guration drift over time.

    View full-size slide

  7. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Provision
    v1 v1 v1
    User
    Mutable Infrastructure
    { }
    IaC
    Apply
    changes
    v2
    v2 v2
    Change
    { }
    IaC
    v1 v1 v1
    Provision
    User
    Provision
    v2 v2 v2
    User
    Immutable Infrastructure
    { }
    IaC
    Change
    { }
    IaC
    Deploys v2 to same Infrastructure Provisions new Infrastructure with v2

    View full-size slide

  8. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Patterns and Practices

    View full-size slide

  9. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Source Control
    Everything in Source
    Control
    Accessible t
    o


    Everyone

    View full-size slide

  10. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Modularize and Version
    Maintenance
    Readability
    Ownership

    View full-size slide

  11. Application
    Owner: 

    Application
    Team
    App
    Deploy
    Platform
    Owner: 

    Platform Engg.
    Team
    Network
    Owner: 

    Networking



    Team
    Account
    Owner: 

    Security/Platform
    Engg. Team
    Bootstrap
    Owner: 

    Platform Engg.



    Team
    Common
    Storage
    Account
    Settings
    VPC/
    Subnet
    Common
    Network
    Database
    Cluster
    EKS
    App
    Database
    App
    Storage
    All Application Infrastructure gets
    provisioned along with Application
    Deployment in this Layer
    Application Platform Layer for
    provisioning anything needed to run
    applications on
    All Networking including connections
    to other networks gets provisioned in
    this Layer
    All AWS Account Level resources
    get provisioned in this Layer
    Any Bootstrapping needed for
    running above Layers like Service
    Account with permissions to run IaC
    & IaC State Bucket
    Infrastructure Layers Example
    State S3
    Bucket
    Service
    Account

    View full-size slide

  12. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Documentation
    Easily available
    Closer to the code
    Keep it updated

    View full-size slide

  13. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Static Analysis
    terraform validate, TFLint, puppet parser validate
    Cost and Duration
    Unit Tests
    Bats, chefspec
    Integration Tests
    InSpec, goss
    Smoke



    Tests



    w/ dummy app
    Selenium, JMeter
    Brittleness and Maintenance
    IaC Testing

    View full-size slide

  14. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Security and Compliance
    Identity & Access
    Management
    Secrets



    Management
    Security



    Scanning
    Compliance

    View full-size slide

  15. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Automate Execution from a
    Shared Environment

    View full-size slide

  16. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    VM
    VM
    VM
    Static Analysis Unit Tests
    Continuous Integration
    Compliance
    Integration
    Tests Security
    VM VM VM
    Testing & Validation
    Ephemeral Environment
    Provision
    Smoke
    Tests
    Smoke
    Tests
    Note: This example is for Platform Layer but
    similar pipeline should be added for other layers
    (account, network & application)
    IaC Pipeline

    View full-size slide

  17. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    GitOps
    GitOps = IaC + (Work
    f
    l
    ow + Control Loop)

    View full-size slide

  18. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Create Pull Request
    ,


    Plan & Tests
    Terraform
    Apply
    VM
    VM
    VM
    Terraform
    Plan & Tests
    Approve & 

    Merge PR
    Control Loop
    Verify Desired
    & Actual State
    Sync (if
    different)
    Provision
    GitOps

    View full-size slide

  19. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Talk based On
    Article on my website:



    http://bit.ly/IaC-PP
    Contributed to the new O’Reilly book

    View full-size slide

  20. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Adarsh Sha
    h


    Engineering Leader, Coach, Hands-on Architec
    t


    Independent Consultan
    t


    @shahadarsh 

    https://shahadarsh.com

    View full-size slide