Upgrade to Pro — share decks privately, control downloads, hide ads and more …

From Infrastructure as Code to Environment as Code – DevOps Days Boston

Adarsh Shah
September 13, 2022

From Infrastructure as Code to Environment as Code – DevOps Days Boston

Infrastructure as Code(IaC) has made managing infrastructure easier in a lot of ways, but there are many challenges that companies accept as the cost of adopting IaC especially when scaling. IaC is good at provisioning individual resources (or a few of them together) but engineering teams want an entire environment with various components like networking, platform (ec2/eks), database, s3 buckets, etc. to deploy and operate their applications.

To provision and tear down an entire environment, these teams have two options. They can either hand roll pipelines to manage individual resources and then manage complex dependencies between these resources within those pipelines or create a monolith IaC for the entire environment. These approaches are inefficient and slow down feature development and innovation. They also make replicating, visualizing & understanding environments difficult. What if there were a better way?

This talk digs into these challenges to try to better understand them and then look at how to resolve them. We will introduce Environment as Code (abstraction over IaC) that enables teams to provision & teardown entire Environments in an efficient way and promotes best practices like loosely coupled infrastructure resources.

Adarsh Shah

September 13, 2022
Tweet

More Decks by Adarsh Shah

Other Decks in Technology

Transcript

  1. From Infrastructure as
    Code to Environment as
    Code
    Challenges scaling IaC
    and how to resolve them
    Adarsh Shah
    Engineering Leader, Coach, Public Speaker
    Founder & CEO at zLifecycle
    @shahadarsh 

    https://zLifecycle.com
    @shahadarsh
    https://bit.ly/eac-dod-boston

    View full-size slide

  2. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Terminologies
    Components: A logical grouping of 1 or more Infrastructure Resources or
    Applications that get provisioned together. For example, Networking is an
    Infrastructure Component with various Infrastructure resources like Virtual Private
    Cloud(VPC), Subnets, Internet Gateways, Route Tables, etc.
    Environment: A logical grouping of all the Components that are needed to run
    business applications. The grouping includes components like networking, eks,
    database, k8s apps, etc.

    View full-size slide

  3. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Infrastructure as Code
    Infrastructure as Code (IaC) is an approach that takes
    proven coding techniques used by software systems and
    extends it to infrastructure. It is one of the key DevOps
    practices that enable teams to deliver infrastructure,
    and thereby software running on it, rapidly and
    reliably.

    View full-size slide

  4. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Evolution of IaC Setup
    networking
    +
    eks
    +
    ….
    Monolith IaC
    networking
    eks
    website
    postgres
    networking
    networking
    +
    eks
    +
    …. ec2 website
    eks postgres
    ec2
    Breaking into Separate & Smaller components
    +
    +

    View full-size slide

  5. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Evolution of IaC Execution
    From Engineers
    machine
    IaC Pipeline/GitOps execution
    from a Shared Environment

    View full-size slide

  6. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston

    View full-size slide

  7. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    • Option 1
    • Create a Monolith IaC
    • Option 2
    • Hand-roll Pipelines
    • Manage Complex Dependencies
    Options for Entire Environment Provisioning

    View full-size slide

  8. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Other Challenges scaling IaC
    Replicating Environments
    is a pain Not easy to visualize/understand
    Environments
    Drift Detection Not straightforward to 

    promote changes
    Dev QA Prod

    View full-size slide

  9. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    What is Environment as
    Code?

    View full-size slide

  10. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Environment as Code
    networking
    eks
    website
    postgres
    networking
    website
    eks postgres
    Declarative
    Environment as Code
    +

    View full-size slide

  11. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston

    View full-size slide

  12. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    IaC vs EaC
    Infrastructure as Code
    Automates various Lego Pieces 

    (i.e. Infrastructure Resources)
    Environment as Code
    Automates how those Lego Pieces 

    are connected to make up a Lego Toy 

    (i.e. Entire Environment)

    View full-size slide

  13. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Environment as Code
    Environment as Code (EaC) is an abstraction over Cloud
    native tools that provides a declarative way of de
    fi
    ning an
    entire Environment. It has a Control Plane that manages the
    state of the environment, including relationships between
    various resources, Detects Drift as well enables Reconciliation.

    View full-size slide

  14. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Provision Environment
    Environment as Code
    Environment
    State
    (Manages dependencies,
    status etc. )
    State
    networking
    eks
    State
    website
    postgres
    State
    networking
    website
    eks postgres
    Provision
    Reconcile
    Declarative
    Detect Drift
    Control Plane
    +

    View full-size slide

  15. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    networking
    website
    eks postgres
    Teardown Environment
    Environment as Code
    Environment
    State
    (Manages dependencies,
    status etc. )
    State
    networking
    State
    website
    eks postgres
    State
    Teardown
    Reconcile
    Declarative
    Detect Drift
    Control Plane
    +

    View full-size slide

  16. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Attributes of Environment as
    Code

    View full-size slide

  17. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Ability to de
    fi
    ne Entire Environment

    View full-size slide

  18. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Environment as Code
    networking
    platform-k8s
    k8s-addons
    postgres
    networking
    eks
    website
    postgres
    Declarative
    Loosely Coupled

    View full-size slide

  19. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Manage State for the entire Environment
    Example Environment State File

    View full-size slide

  20. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Idempotent and Immutable for
    entire Environment

    View full-size slide

  21. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Non-Idempotent
    Provision
    VM VM VM
    { }
    IaC
    3 VMs
    End State

    Expected = 3
    Actual = 6
    Idempotent
    End State

    Expected = 3
    Actual = 3
    Reapply
    VM VM VM
    Provision
    VM VM VM
    Reapply
    No change
    { }
    IaC
    3 VMs

    View full-size slide

  22. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Provision
    v1 v1 v1
    User
    Mutable Infrastructure
    { }
    IaC
    Apply
    changes
    v2
    v2 v2
    Change
    { }
    IaC
    v1 v1 v1
    Provision
    User
    Provision
    v2 v2 v2
    User
    Immutable Infrastructure
    { }
    IaC
    Change
    { }
    IaC
    Deploys v2 to same Infrastructure Provisions new Infrastructure with v2

    View full-size slide

  23. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Visualize and Understand Environments

    View full-size slide

  24. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Drift Detection and Reconciliation
    Reconcile
    (Preferably with Approval step
    that shows the plan)
    Desired State Current State
    Control Loop
    Observe/Detect Drift

    View full-size slide

  25. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Compare and Promote Changes between
    Environments
    Compare & promote changes
    across environments
    GitOps for Entire Environment

    View full-size slide

  26. @shahadarsh
    https://zLifecycle.com
    https://bit.ly/eac-dod-boston
    Create Pull Request,
    EaC Validate
    Validate
    Approve & 

    Merge PR
    Control
    Plane
    Reconcile
    GitOps for Environment
    EaC
    Trunk
    networking
    platform-k8s
    k8s-addons
    postgres
    Observe
    Branch
    networking
    eks
    website
    postgres

    View full-size slide

  27. Adarsh Shah
    Engineering Leader, Coach, Public Speaker
    Founder & CEO at zLifecycle
    @shahadarsh 

    https://zLifecycle.com
    Thank You
    @shahadarsh
    https://bit.ly/eac-dod-boston

    View full-size slide