From Infrastructure as Code to Environment as Code

From Infrastructure as Code to Environment as Code: Challenges scaling IaC and how to resolve them

Description: Implementing and maintaining Infrastructure as Code (IaC) to automate the provisioning of Environments, becomes challenging as you scale. IaC is good at provisioning individual resources (or a few of them together) but engineering teams want an entire environment with various components like networking, platform (ec2/eks), database, s3 buckets, etc. to deploy and operate their applications.

To provision and tear down an entire environment, these teams have two options. They can either hand roll pipelines to manage individual resources and then manage complex dependencies between these resources within those pipelines or create a monolith IaC for the entire environment. These approaches are inefficient and slow down feature development and innovation. What if there were a better way?

In this presentation, we will introduce Environment as Code (abstraction over IaC) that enables teams to provision & teardown entire Environments in an efficient way and promotes best practices like loosely coupled infrastructure resources.

** Key Takeaways: **

* Challenges scaling Infrastructure as Code
* What is Environment as Code?
* How Environment as Code can help resolve those challenges?

Transcript

1. From Infrastructure as
   Code to Environment as
   Code
   Challenges scaling IaC
   and how to resolve them
   Adarsh Sha
   h
   
   Engineering Leader, Coach, Public Speake
   r
   
   Founder & CEO at CompuZes
   t
   
   @shahadarsh 

   https://compuzest.com
   https://bit.ly/iac-to-eac
   

   View Slide

2. https://compuzest.com @shahadarsh
   Infrastructure as Code
   Infrastructure as Code (IaC) is an approach that takes
   proven coding techniques used by software systems and
   extends it to infrastructure. It is one of the key DevOps
   practices that enable teams to deliver infrastructure,
   and thereby software running on it, rapidly and
   reliably, at scale.
   

   View Slide

3. https://compuzest.com @shahadarsh
   IaC: Principles, Patterns, and Practices
   • Key Principles
   
   
   • Idempotenc
   y
   
   • Immutabilit
   y
   
   • Patterns and Practice
   s
   
   • Source Contro
   l
   
   • Modularize and Versio
   n
   
   • Documentatio
   n
   
   • Testin
   g
   
   • Security and Complianc
   e
   
   • Automate Execution from a Shared Environmen
   t
   
   • IaC Pipelin
   e
   
   • GitOps
   Article/Talk: https://bit.ly/iac-to-eac
   

   View Slide

4. https://compuzest.com @shahadarsh
   Idempotency
   Idempotency means no matter how many times you run
   your IaC and, what your starting state is, you will end up
   with the same end state. This simpli
   fi
   es the provisioning
   of Infrastructure and reduces the chances of inconsistent
   results.
   

   View Slide

5. https://compuzest.com @shahadarsh
   Non-Idempotent
   Provision
   VM VM VM
   { }
   IaC
   3 VMs
   End State

   Expected =
   3
   
   Actual = 6
   Idempotent
   End State

   Expected =
   3
   
   Actual = 3
   Reapply
   VM VM VM
   Provision
   VM VM VM
   Reapply
   No change
   { }
   IaC
   3 VMs
   

   View Slide

6. https://compuzest.com @shahadarsh
   Immutability
   Immutable infrastructure means instead of changing
   an existing infrastructure you replace it with new. By
   provisioning new infrastructure every time, you are
   making sure it is reproducible and doesn’t allow for
   con
   fi
   guration drift over time.
   

   View Slide

7. https://compuzest.com @shahadarsh
   Provision
   v1 v1 v1
   User
   Mutable Infrastructure
   { }
   IaC
   Apply
   changes
   v2
   v2 v2
   Change
   { }
   IaC
   v1 v1 v1
   Provision
   User
   Provision
   v2 v2 v2
   User
   Immutable Infrastructure
   { }
   IaC
   Change
   { }
   IaC
   Deploys v2 to same Infrastructure Provisions new Infrastructure with v2
   

   View Slide

8. https://compuzest.com @shahadarsh
   Automate Execution from a
   Shared Environment
   

   View Slide

9. https://compuzest.com @shahadarsh
   VM
   VM
   VM
   Static Analysis Unit Tests
   Continuous Integration
   Compliance
   Integration
   Tests Security
   VM VM VM
   Testing & Validation
   Ephemeral Environment
   Provision
   Smoke
   Tests
   Smoke
   Tests
   Note: This example is for Platform Layer but
   similar pipeline should be added for other layers
   (account, network & application)
   IaC Pipeline
   

   View Slide

10. https://compuzest.com @shahadarsh
    GitOps
    GitOps = IaC + (Work
    fl
    ow + Control Loop)
    

    View Slide

11. https://compuzest.com @shahadarsh
    Create Pull Request
    ,
    
    Plan & Tests
    Terraform
    Apply
    VM
    VM
    VM
    Terraform
    Plan & Tests
    Approve & 

    Merge PR
    Control Loop
    Verify Desired
    & Actual State
    Sync (if
    different)
    Provision
    GitOps
    

    View Slide

12. https://compuzest.com @shahadarsh
    

    View Slide

13. https://compuzest.com @shahadarsh
    

    View Slide

14. https://compuzest.com @shahadarsh
    Challenges scaling IaC
    • Hand-rolling Pipelines
    
    
    • Teams want an Entire Environment and not just
    individual resources
    
    
    • Replicating Environments
    
    
    • Visualizing and Understanding Environments
    
    
    • Infrastructure Drift
    Article/Talk: https://bit.ly/iac-to-eac
    

    View Slide

15. https://compuzest.com @shahadarsh
    Environment as Code
    Environment as Code (EaC) is an abstraction over
    Infrastructure as Code that provides a way of de
    fi
    ning an
    entire Infrastructure Environment. It manages state for
    the environment including dependences between various
    resources in the Environment and supports best
    practices like Loose coupling, Idempotency, Immutability
    etc. for an Environment. It allows teams to deliver entire
    environments (instead of individual infrastructure
    resources) rapidly and reliably, at scale.
    

    View Slide

16. https://compuzest.com @shahadarsh
    Environment as Code
    Example
    

    View Slide

17. https://compuzest.com @shahadarsh
    EaC features
    • De
    fi
    ne Entire Environment
    
    
    • Various Infrastructure Resource
    s
    
    • Dependencies between resource
    s
    
    • Manage State for the entire Environmen
    t
    
    • Idempotent and Immutable for entire Environmen
    t
    
    • Loose Couplin
    g
    
    • Visualize and Understand Environment
    s
    
    • Infrastructure Drift
    

    View Slide

18. Adarsh Sha
    h
    
    Engineering Leader, Coach, Public Speake
    r
    
    Founder & CEO at CompuZes
    t
    
    @shahadarsh 

    https://compuzest.com
    https://bit.ly/iac-to-eac
    

    View Slide