Cloud Native Configuration Management

A review of tools and technologies currently in use across the cloud native community for managing configurations of their applications and infrastructure.

Keynote at Config Management Camp 2020

Transcript

1. Eric Sorenson // eric@puppet.com // {github,twitter,soundcloud}.com/ahpook Cloud Native Configuration Management: 2020 and beyond
3. IFTTT For Devops ...? puppet.com/project-nebula
4. • Platform-enabled • Resilient • Agile • Operable • Observable CN Characteristics
5. • Declarative config • Clean OS contract • Built for cloud • Enable CD 12 Facter Apps
6. • Use declarative formats for setup automation, to minimize time and cost for new developers joining the project; • Have a clean contract with the underlying operating system, offering maximum portability between execution environments; • Are suitable for deployment on modern cloud platforms, obviating the need for servers and systems administration; • Minimize divergence between development and production, enabling continuous deployment for maximum agility; • And can scale up without significant changes to tooling, architecture, or development practices. — Adam Wiggins, 12factor.net
7. Ephemerality Immutability Data...bility Cardinality
8. Infrastructure as code Infrastructure as data Infrastructure as software
9. TK Visualization of 110 Tools
10. The Configuration Complexity Clock (Mike Hadlow, via @garethr and Cedric Charly) 12 3 6 9 Hard Coded Config Values Rules Engine DSL Curse
11. 12 3 6 9 Raw Config Generators Frontends DSL Taxonomy of Cloud-Native CM Tools
12. Generators
13. Kapitan - kapitan.dev • ‘reclass’ as a source of truth for config data • Jinja2 or jsonnet templates for generation • Compiles output into folders for runtime injection and gitops love
14. Kr8 – kr8.rocks • “Oh **** we’re going to have to write something” – (blog: leebriggs.co.uk) • Uses Jsonnet for both data storage and templating • Maps ‘components’ (things you run) onto ‘clusters’ (where you run them)
15. Frontends
16. Tanka – tanka.dev • Philosophically like ksonnet, but simpler: “environments” only • Relatively thin layer of workflow over jsonnet, featuring deep-merges • Uses k8s 1.13+ server- side diffing
17. Helm – helm.sh • ”Package management” for Kubernetes • Variable substitution to templating to… Lua? • Value injection still a problem
18. P U P P E T O V E R V I E W 18 Pulumi – pulumi.io • “Terraform for Programmers” • Multiple language bindings over TF providers – js, python • Cloud-based state by default
19. Domain Specific Languages
20. Terraform – https://terraform.io
21. Gyro – getgyro.io • Ground-up Java IaC tool from PerfectSense • Custom DSL / CLI for cloud provisioning • Y THO?!
22. Starlark – bazelbuild/starlark • Dialect of Python made for configuration languages • Multiple implementations: java, go, rust • Proposal for Tekton pipelines
23. Cue – cuelang.org • “Configure, Unify, Execute” • Data constraint language • Builds templating, defaults, spec directly into the config language • DSL over the domain of configuration
24. Lessons Learned
25. Complexity will chase you ....but wait until the last minute to start running *HONK*
26. Constraints and specs > all else or: “Any sufficiently advanced YAML dialect is indistinguishable from a DSL” json-schema.org | @lyraproj/dgo | WTFM
27. Stop making new DSLs. ... please? cuelang.org | jsonnet.org | @bazelbuild/starlark
28. Of all the problems we have confronted, the ones over which the most brainpower, ink, and code have been spilled are related to managing configurations—the set of values supplied to applications, rather than hard-coded into them. In truth, we could have devoted this entire article to the subject and still have had more to say. Burns, Grant, Brewer et al - “Borg, Omega and Kubernetes”
29. Thank you! Eric Sorenson // eric@puppet.com @ahpook
30. Lessons Learned