Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Environment as Code: Moving away from Imperative Pipelines

Adarsh Shah
February 06, 2023

Environment as Code: Moving away from Imperative Pipelines

Cloud Native tools like Infrastructure as Code (IaC) & Helm have made managing infrastructure as well as cloud-native apps easier in a lot of ways, but there are many challenges that companies accept as the cost of adopting them. IaC is good at provisioning individual infrastructure components (a few resources together), and tools like helm are good at deploying applications, but engineering teams want an entire environment with various components like networking, platform (ec2/eks), database, k8s apps, etc. to run their business applications.

To provision and teardown an entire environment, these teams have two options. They can either hand-roll imperative pipelines to manage individual components and then manage complex dependencies between these components or create a monolith deployment for the entire environment. These approaches are inefficient and slow down feature development and innovation. They also make replicating, visualizing & understanding environments difficult. What if there were a better way?

This talk digs into these challenges and then looks at how to resolve them. We will introduce Environment as Code (EaC) which is an abstraction over cloud-native tools that provides a declarative way of defining an entire environment. It has a Control Plane that manages the state of the environment, including component dependencies, and detects drift, and enables reconciliation.

Key Takeaways:

* Challenges adopting Cloud Native tools like Infrastructure as Code, Helm, etc.
* What is Environment as Code?
* How can Environment as Code help resolve those challenges?

Adarsh Shah

February 06, 2023
Tweet

More Decks by Adarsh Shah

Other Decks in Technology

Transcript

  1. Environment as Code: Moving away from Imperative Pipelines Adarsh Shah

    Engineering Leader, Coach, Public Speaker Founder & CEO at CloudKnit @shahadarsh 
 https://cloudknit.io @shahadarsh https://bit.ly/eac-ghent Open Source Repo: https://github.com/cloudknit-io/cloudknit
  2. @shahadarsh https://bit.ly/eac-ghent https://github.com/cloudknit-io/cloudknit Evolution of Environment Setup networking + eks

    + …. Monolith Deployment networking eks website postgres networking networking + eks + …. ec2 website eks postgres ec2 Breaking into Separate & Smaller components + +
  3. @shahadarsh https://bit.ly/eac-ghent https://github.com/cloudknit-io/cloudknit • Option 1 • Create a Monolith

    IaC • Option 2 • Hand-roll Pipelines - Imperative • Manage Complex Dependencies Options for Entire Environment Provisioning
  4. @shahadarsh https://bit.ly/eac-ghent https://github.com/cloudknit-io/cloudknit Other Challenges scaling IaC Replicating Environments is

    a pain Not easy to visualize/understand Environments Drift Detection Not straightforward to 
 promote changes Dev QA Prod
  5. @shahadarsh https://bit.ly/eac-ghent https://github.com/cloudknit-io/cloudknit Terminologies Components: A logical grouping of 1

    or more Infrastructure Resources or Applications that get provisioned together. For example, Networking is an Infrastructure Component with various Infrastructure resources like Virtual Private Cloud(VPC), Subnets, Internet Gateways, Route Tables, etc. Environment: A logical grouping of all the Components that are needed to run business applications. The grouping includes components like networking, eks, database, k8s apps, etc.
  6. @shahadarsh https://bit.ly/eac-ghent https://github.com/cloudknit-io/cloudknit IaC vs EaC Infrastructure as Code Automates

    various Lego Pieces 
 (i.e. Infrastructure Resources) Environment as Code Automates how those Lego Pieces 
 are connected to make up a Lego Toy 
 (i.e. Entire Environment)
  7. @shahadarsh https://bit.ly/eac-ghent https://github.com/cloudknit-io/cloudknit Environment as Code Environment as Code (EaC)

    is an abstraction over Cloud native tools that provides a declarative way of de fi ning an entire Environment. It has a Control Plane that manages the state of the environment, including relationships between various resources, Detects Drift as well enables Reconciliation.
  8. @shahadarsh https://github.com/cloudknit-io/cloudknit https://bit.ly/eac-ghent Provision Environment Environment as Code Environment State

    (Manages dependencies, status etc. ) State networking eks State website postgres State networking website eks postgres Provision Reconcile Declarative Detect Drift Control Plane +
  9. @shahadarsh https://github.com/cloudknit-io/cloudknit https://bit.ly/eac-ghent networking website eks postgres Teardown Environment Environment

    as Code Environment State (Manages dependencies, status etc. ) State networking State website eks postgres State Teardown Reconcile Declarative Detect Drift Control Plane +
  10. @shahadarsh https://bit.ly/eac-ghent https://github.com/cloudknit-io/cloudknit Non-Idempotent Provision VM VM VM { }

    IaC 3 VMs End State
 Expected = 3 Actual = 6 Idempotent End State
 Expected = 3 Actual = 3 Reapply VM VM VM Provision VM VM VM Reapply No change { } IaC 3 VMs
  11. @shahadarsh https://bit.ly/eac-ghent https://github.com/cloudknit-io/cloudknit Provision v1 v1 v1 User Mutable Infrastructure

    { } IaC Apply changes v2 v2 v2 Change { } IaC v1 v1 v1 Provision User Provision v2 v2 v2 User Immutable Infrastructure { } IaC Change { } IaC Deploys v2 to same Infrastructure Provisions new Infrastructure with v2
  12. @shahadarsh https://bit.ly/eac-ghent https://github.com/cloudknit-io/cloudknit Drift Detection and Reconciliation Reconcile (Preferably with

    Approval step that shows the plan) Desired State Current State Control Loop Observe/Detect Drift
  13. @shahadarsh https://github.com/cloudknit-io/cloudknit https://bit.ly/eac-ghent Create Pull Request, EaC Validate Validate Approve

    & 
 Merge PR Control Plane Reconcile GitOps for Environment EaC Trunk networking platform-k8s k8s-addons postgres Observe Branch networking eks website postgres
  14. Thank You @shahadarsh https://bit.ly/eac-ghent Adarsh Shah Engineering Leader, Coach, Public

    Speaker Founder & CEO at CloudKnit @shahadarsh 
 https://cloudknit.io Open Source Repo: https://github.com/cloudknit-io/cloudknit