Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Config management 2.0

B78415ec052d8811a4f31c4746cbde9f?s=47 Senthil V S
January 22, 2019

Config management 2.0

The whole `DevOps` movement is about no separate `Dev` and `Ops` teams. But with using heavy config management tools, we're once again on the same place just that we've replaced bash scripts with yamls and central store. For a startup, this approach does not suit. We've taken a different approach and finding it fruitful. In this talk, we will share our journey and why our approach could be useful to you.

B78415ec052d8811a4f31c4746cbde9f?s=128

Senthil V S

January 22, 2019
Tweet

More Decks by Senthil V S

Other Decks in Technology

Transcript

  1. Config management 2.0 siliconsenthil Senthil V S

  2. Before we start…

  3. Used Config management tools?

  4. Used PaaS like Heroku?

  5. Is this one cake or multiple?

  6. make money simple, so that people can live well and

    do amazing things. API SDKs Mobile and web apps
  7. Simpl architecture Services Streaming Platform Data Pipeline Dashboards Analytics ML

    models Merchant Apps Simpl Website ML models feed back underwriting, UX decisions, etc services Event sourcing Kafka as streaming platform services
  8. Hard challenges vs less resources >100 merchants checkout 
 page

    90k req/min ~20 μservices Four tech stacks 
 (RoR, GoLang, Node, Python) >200GB of event data/day Continuous delivery Strict security needs Five developers. No DevOps No BW to manage Kubernetes AWS had only ECS then
  9. We wanted a solution, that’s lightweight

  10. Typical codebase Service A Service B Service C DevOps code

    Library X • Services could be polyglot • Developers own services • DevOps code by DevOps person • Puppet/Chef/Ansible
  11. DevOps: Dissection Service A Service B Service C Library X

    
 DevOps code Service topography Environ. config Deployment Observability Provisioning Network } } • Service specific • Frequent changes • Common across services • Changes less • Platform dependent
  12. DevOps: Service topography Service A Service B Service C Library

    X Dockerfile JSON store • As JSON config • Managed by developers at central store 
 DevOps code Service topography Environ. config Deployment Observability Provisioning Network
  13. DevOps: Env. config Service A Service B Service C Library

    X Dockerfile JSON store env.sample & parameter store • Secure central storage • Versioned file <> Central store 
 DevOps code Service topography Environ. config Deployment Observability Provisioning Network
  14. DevOps: Summary Service A Service B Service C Library X

    Dockerfile JSON store env.sample & parameter store } • AWS APIs • JSON store • AWS VPCs & ECS • Cloudformation & APIs 
 DevOps code Service topography Environ. config Deployment Observability Provisioning Network
  15. Solutions exists for all the layers. How to wire together

    in a simple way?
  16. Cloudlift Service A Service B Service C Library X Service

    topography Environ. config Observability Network Provisioning Deployment • Glues above and creates cloudformation templates • executable as CLI • Python • In production for 2 years • Open sourced. 
 https://github.com/GetSimpl/cloudlift Cloudlift
  17. Demo: Create service • Service config by user • cfn

    template building with trophosphere • Api call to AWS • dynamoDB as service config store Service Config cloudlift Cloudformation template
  18. Demo: Create service • One repo, multiple services • http

    interface or background • Internal (visible at VPC level) vs public • Stored in AWS dynamoDB
  19. Demo: Edit environment config • Git commit like editor flow

    to change config • Stored in AWS parameter store, encrypted
  20. Demo: Edit config & Create service

  21. Demo: deploy service Push local docker 
 image to ECR

    Ensure store has all required config env.sample Docker registry ECR Parameter store Update the task definition with image version & env vars ECS runs new containers with updated version Stops the old version once new are up ★ The service name is guessed from current dir ★ image name is same as repo name ★ latest version is picked to push 1 2 3 4 5
  22. Demo: Deploy service

  23. AWS Lambda Logs • 12-factor: Logs as event streams •

    Structured log to STDOUT • AWS lambda pipe • Searchable log viewer CloudWatch logs STDOUT logs
  24. Show me the code

  25. Learnings

  26. Microservices & DevOps Build-time dependency ▶ Library Run-time dependency ▶

    Another microservice Lifecycle dependency ▶ DevOps
  27. How to add features to cloudlift? Service A Cloudlift Do

    it microservices style v1 Service B v1 Service C v1 v1 Service A Cloudlift v1 Service B v1 Service C v1 v2 Service A Cloudlift v1 Service B v2 Service C v1 v2 Service A, B, C are running with cloudlift version v1 Feature gets added to Cloudlift 
 
 New version released Service uses new feature
 
 Cloudlift is backward compatible
  28. ECS is not batteries-included. Simple and does the job. Well.

  29. Bake-in monitoring and alerts

  30. Service should have only one way to configure Environment variables

    Do you smell?
  31. 12 factor apps - Env var config - Port binding

    - Dev/prod parity - Logs as event streams
  32. Treat secrets and other configurations same.

  33. Never hand-code Cloudformation template

  34. Testing infra code. How?

  35. Prefer CLI over GUI - Building CLI is easier -

    Easier to integrate in CI tools - Security
  36. Summary - DevOps as lifecycle dependency, 
 not just infrastructure

    as code - Extract and treat as a μService
  37. Q & A siliconsenthil Senthil V S siliconsenthil