Config management 2.0

The whole `DevOps` movement is about no separate `Dev` and `Ops` teams. But with using heavy config management tools, we're once again on the same place just that we've replaced bash scripts with yamls and central store. For a startup, this approach does not suit. We've taken a different approach and finding it fruitful. In this talk, we will share our journey and why our approach could be useful to you.

Transcript

1. Config management 2.0 siliconsenthil Senthil V S
2. Before we start…
3. Used Config management tools?
4. Used PaaS like Heroku?
5. Is this one cake or multiple?
6. make money simple, so that people can live well and do amazing things. API SDKs Mobile and web apps
7. Simpl architecture Services Streaming Platform Data Pipeline Dashboards Analytics ML models Merchant Apps Simpl Website ML models feed back underwriting, UX decisions, etc services Event sourcing Kafka as streaming platform services
8. Hard challenges vs less resources >100 merchants checkout 
 page 90k req/min ~20 μservices Four tech stacks 
 (RoR, GoLang, Node, Python) >200GB of event data/day Continuous delivery Strict security needs Five developers. No DevOps No BW to manage Kubernetes AWS had only ECS then
9. We wanted a solution, that’s lightweight
10. Typical codebase Service A Service B Service C DevOps code Library X • Services could be polyglot • Developers own services • DevOps code by DevOps person • Puppet/Chef/Ansible
11. DevOps: Dissection Service A Service B Service C Library X 
 DevOps code Service topography Environ. config Deployment Observability Provisioning Network } } • Service specific • Frequent changes • Common across services • Changes less • Platform dependent
12. DevOps: Service topography Service A Service B Service C Library X Dockerfile JSON store • As JSON config • Managed by developers at central store 
 DevOps code Service topography Environ. config Deployment Observability Provisioning Network
13. DevOps: Env. config Service A Service B Service C Library X Dockerfile JSON store env.sample & parameter store • Secure central storage • Versioned file <> Central store 
 DevOps code Service topography Environ. config Deployment Observability Provisioning Network
14. DevOps: Summary Service A Service B Service C Library X Dockerfile JSON store env.sample & parameter store } • AWS APIs • JSON store • AWS VPCs & ECS • Cloudformation & APIs 
 DevOps code Service topography Environ. config Deployment Observability Provisioning Network
15. Solutions exists for all the layers. How to wire together in a simple way?
16. Cloudlift Service A Service B Service C Library X Service topography Environ. config Observability Network Provisioning Deployment • Glues above and creates cloudformation templates • executable as CLI • Python • In production for 2 years • Open sourced. 
 https://github.com/GetSimpl/cloudlift Cloudlift
17. Demo: Create service • Service config by user • cfn template building with trophosphere • Api call to AWS • dynamoDB as service config store Service Config cloudlift Cloudformation template
18. Demo: Create service • One repo, multiple services • http interface or background • Internal (visible at VPC level) vs public • Stored in AWS dynamoDB
19. Demo: Edit environment config • Git commit like editor flow to change config • Stored in AWS parameter store, encrypted
20. Demo: Edit config & Create service
21. Demo: deploy service Push local docker 
 image to ECR Ensure store has all required config env.sample Docker registry ECR Parameter store Update the task definition with image version & env vars ECS runs new containers with updated version Stops the old version once new are up ★ The service name is guessed from current dir ★ image name is same as repo name ★ latest version is picked to push 1 2 3 4 5
22. Demo: Deploy service
23. AWS Lambda Logs • 12-factor: Logs as event streams • Structured log to STDOUT • AWS lambda pipe • Searchable log viewer CloudWatch logs STDOUT logs
24. Show me the code
25. Learnings
26. Microservices & DevOps Build-time dependency ▶ Library Run-time dependency ▶ Another microservice Lifecycle dependency ▶ DevOps
27. How to add features to cloudlift? Service A Cloudlift Do it microservices style v1 Service B v1 Service C v1 v1 Service A Cloudlift v1 Service B v1 Service C v1 v2 Service A Cloudlift v1 Service B v2 Service C v1 v2 Service A, B, C are running with cloudlift version v1 Feature gets added to Cloudlift 
 
 New version released Service uses new feature
 
 Cloudlift is backward compatible
28. ECS is not batteries-included. Simple and does the job. Well.
29. Bake-in monitoring and alerts
30. Service should have only one way to configure Environment variables Do you smell?
31. 12 factor apps - Env var config - Port binding - Dev/prod parity - Logs as event streams
32. Treat secrets and other configurations same.
33. Never hand-code Cloudformation template
34. Testing infra code. How?
35. Prefer CLI over GUI - Building CLI is easier - Easier to integrate in CI tools - Security
36. Summary - DevOps as lifecycle dependency, 
 not just infrastructure as code - Extract and treat as a μService
37. Q & A siliconsenthil Senthil V S siliconsenthil