Principles, Patterns, and Practices for Effective Infrastructure as Code

Infrastructure as Code (IaC) is an approach that takes proven coding techniques used by software systems and extends it to infrastructure. It is one of the key DevOps practices that enable teams to deliver infrastructure, and the software running on it, rapidly and reliably, at scale.

In this presentation, we will look at Principles, Patterns, and Practices for Effective Infrastructure as Code that have helped me and the teams I have worked with over the years. We will look at anecdotes about deploying & operating infrastructure and applications in production.

Transcript

1. Principles, Patterns, and
   Practices for Effective
   Infrastructure as Code
   Deliver Infrastructure and
   Software running on it Rapidly
   and Reliably at Scale
   Adarsh Sha
   h
   
   
   Engineering Leader, Coach, Hands-on Architec
   t
   
   
   Independent Consultan
   t
   
   
   @shahadarsh 

   https://shahadarsh.com
   Deck: https://bit.ly/IaC-FOSDEM
   

   View Slide

2. https://shahadarsh.com @shahadarsh
   Deck: https://bit.ly/IaC-FOSDEM
   Infrastructure as Code
   Infrastructure as Code (IaC) is an approach that takes
   proven coding techniques used by software systems and
   extends it to infrastructure. It is one of the key DevOps
   practices that enable teams to deliver infrastructure,
   and the software running on it, rapidly and reliably,
   at scale.
   

   View Slide

3. https://shahadarsh.com @shahadarsh
   Deck: https://bit.ly/IaC-FOSDEM
   Key Principles
   

   View Slide

4. https://shahadarsh.com @shahadarsh
   Deck: https://bit.ly/IaC-FOSDEM
   Idempotency
   Idempotency means no matter how many times you run
   your IaC and, what your starting state is, you will end up
   with the same end state. This simpli
   f
   i
   es the provisioning
   of Infrastructure and reduces the chances of inconsistent
   results.
   

   View Slide

5. https://shahadarsh.com @shahadarsh
   Deck: https://bit.ly/IaC-FOSDEM
   Non-Idempotent
   Provision
   VM VM VM
   { }
   IaC
   3 VMs
   End State

   Expected =
   3
   
   
   Actual = 6
   Idempotent
   End State

   Expected =
   3
   
   
   Actual = 3
   Reapply
   VM VM VM
   Provision
   VM VM VM
   Reapply
   No change
   { }
   IaC
   3 VMs
   

   View Slide

6. https://shahadarsh.com @shahadarsh
   Deck: https://bit.ly/IaC-FOSDEM
   Immutability
   Immutable infrastructure means instead of changing
   an existing infrastructure you replace it with new. By
   provisioning new infrastructure every time, you are
   making sure it is reproducible and doesn’t allow for
   con
   f
   i
   guration drift over time.
   

   View Slide

7. https://shahadarsh.com @shahadarsh
   Deck: https://bit.ly/IaC-FOSDEM
   Provision
   v1 v1 v1
   User
   Mutable Infrastructure
   { }
   IaC
   Apply
   changes
   v2
   v2 v2
   Change
   { }
   IaC
   v1 v1 v1
   Provision
   User
   Provision
   v2 v2 v2
   User
   Immutable Infrastructure
   { }
   IaC
   Change
   { }
   IaC
   Deploys v2 to same Infrastructure Provisions new Infrastructure with v2
   

   View Slide

8. https://shahadarsh.com @shahadarsh
   Deck: https://bit.ly/IaC-FOSDEM
   Patterns and Practices
   

   View Slide

9. https://shahadarsh.com @shahadarsh
   Deck: https://bit.ly/IaC-FOSDEM
   Source Control
   Everything in Source
   Control
   Accessible t
   o
   
   
   Everyone
   

   View Slide

10. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Modularize and Version
    Maintenance
    Readability
    Ownership
    

    View Slide

11. Application
    Owner: 

    Application
    Team
    App
    Deploy
    Platform
    Owner: 

    Platform Engg.
    Team
    Network
    Owner: 

    Networking
    
    
    
    Team
    Account
    Owner: 

    Security/Platform
    Engg. Team
    Bootstrap
    Owner: 

    Platform Engg.
    
    
    
    Team
    Common
    Storage
    Account
    Settings
    VPC/
    Subnet
    Common
    Network
    Database
    Cluster
    EKS
    App
    Database
    App
    Storage
    All Application Infrastructure gets
    provisioned along with Application
    Deployment in this Layer
    Application Platform Layer for
    provisioning anything needed to run
    applications on
    All Networking including connections
    to other networks gets provisioned in
    this Layer
    All AWS Account Level resources
    get provisioned in this Layer
    Any Bootstrapping needed for
    running above Layers like Service
    Account with permissions to run IaC
    & IaC State Bucket
    Infrastructure Layers Example
    State S3
    Bucket
    Service
    Account
    

    View Slide

12. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Documentation
    Easily available
    Closer to the code
    Keep it updated
    

    View Slide

13. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Static Analysis
    terraform validate, TFLint, puppet parser validate
    Cost and Duration
    Unit Tests
    Bats, chefspec
    Integration Tests
    InSpec, goss
    Smoke
    
    
    
    Tests
    
    
    
    w/ dummy app
    Selenium, JMeter
    Brittleness and Maintenance
    IaC Testing
    

    View Slide

14. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Security and Compliance
    Identity & Access
    Management
    Secrets
    
    
    
    Management
    Security
    
    
    
    Scanning
    Compliance
    

    View Slide

15. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Automate Execution from a
    Shared Environment
    

    View Slide

16. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    VM
    VM
    VM
    Static Analysis Unit Tests
    Continuous Integration
    Compliance
    Integration
    Tests Security
    VM VM VM
    Testing & Validation
    Ephemeral Environment
    Provision
    Smoke
    Tests
    Smoke
    Tests
    Note: This example is for Platform Layer but
    similar pipeline should be added for other layers
    (account, network & application)
    IaC Pipeline
    

    View Slide

17. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    GitOps
    GitOps = IaC + (Work
    f
    l
    ow + Control Loop)
    

    View Slide

18. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Create Pull Request
    ,
    
    
    Plan & Tests
    Terraform
    Apply
    VM
    VM
    VM
    Terraform
    Plan & Tests
    Approve & 

    Merge PR
    Control Loop
    Verify Desired
    & Actual State
    Sync (if
    different)
    Provision
    GitOps
    

    View Slide

19. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Talk based On
    Article on my website:
    
    
    
    http://bit.ly/IaC-PP
    Contributed to the new O’Reilly book
    

    View Slide

20. https://shahadarsh.com @shahadarsh
    Deck: https://bit.ly/IaC-FOSDEM
    Adarsh Sha
    h
    
    
    Engineering Leader, Coach, Hands-on Architec
    t
    
    
    Independent Consultan
    t
    
    
    @shahadarsh 

    https://shahadarsh.com
    

    View Slide