The Field Guide to Understanding Declarative Systems

by Povilas Daukintis
DevOps Pro Vilnius 2016

Transcript

1. THE FIELD GUIDE TO UNDERSTANDING DECLARATIVE SYSTEMS IN OPS WORLD Povilas Daukintis (@pdaukintis)
2. $:~whoami (Mostly) operational stuff Configuration management evangelist Tackling infrastructural problems in Adform Current environment: 3000+ servers, 30+ product teams, heterogeneous stacks
3. TALK AGENDA Imperative vs. declarative ways of managing systems Lessons learned Intro to Promise Theory Conclusions
4. IMPERATIVE VS DECLARATIVE SYSTEMS
5. IMPERATIVE SYSTEMS AND INTERFACES Shells Scripting languages APIs CLIs
6. EXAMPLES OF DECLARATIVE SYSTEMS make ANSI SQL (Structured Query Language) Prolog (declarative logic programming language) XML (extensible markup language)
8. IMPERATIVE AND DECLARATIVE APPROACHES IN CONFIGURATION MANAGEMENT SYSTEMS
9. CONFIGURATION MANAGEMENT (CM) “CM is the practice of handling changes systematically so that a system maintains its integrity over time. CM implements the policies, procedures, techniques, and tools that are required to manage, evaluate proposed changes, track the status of changes, and to maintain an inventory of system and support documents as the system changes” - IEEE-SA 828-2012
10. DSLs DSL is a domain specific language Many CM tools have their own on the purpose It is a way to create appropriate layers of abstraction
11. DECLARATIVE(-ISH) TOOLS
12. IDEMPOTENCE
13. CONVERGENCE
14. EXAMPLES
15. IMPERATIVE / PROCEDURAL CODE
16. IMPERATIVE / PROCEDURAL CODE
17. DECLARATIVE CODE
18. MANAGING AWS RESOURCES
19. DIFFERENT ABSTRACTION LAYERS Currently existing CM tools manage configuration of a single node pretty well Still not the case if we speak about multi-node interdependencies What about dependencies of infrastructure dependencies?
21. ALL ABSTRACTION LAYERS ARE “LEAKY”
23. EXAMPLE OF “LEAKY ABSTRACTION”
24. LEARNED LESSONS Rollout your CM/IaC code in small steps When you see deviations from the existing model, adapt quickly “Do it once” approach does not work at scale Choose appropriate tools
25. PROMISE THEORY
26. PROMISE THEORY Formally proposed by Mark Burgess in 2004 Some of the ideas are 20+ years old CFEngine 3 is based on it
27. PROMISE THEORY Promise theory is a model of voluntary cooperation between autonomous actors Who publish their intentions to one another in the form of promises It can be viewed as a mathematical graph
28. IMPORTANT ASPECTS OF PROMISE THEORY Obligations vs. Intentions Handling failures: it must work vs. it will fail Automation: it must be this vs. it should look like this Deterministic vs. Indeterministic view of the system
29. PROMISE THEORY IN ONE PICTURE http://markburgess.org/TIpromises.html
30. PROMISE THEORY AND CM SYSTEMS
31. KEY PRINCIPLES OF PT Convergency - The promise is the desired state Embracing failures - Promises might be kept occasionally Autonomy - Not relying on other agents
33. PUPPET IN CONTEXT OF PT Every Puppet declaration is a sort of promise Everything is converging on the promised configuration state as much as possible at each Puppet run It will keep trying to move closer to the ideal state where all promises are kept
34. MOVING FORWARD … mgmt (https://github.com/purpleidea/mgmt)
35. MGMT “mgmt” feels like a very modern approach in CM systems space Most interesting properties: Parallel execution to run all the resources concurrently (where possible) Event driven to monitor and react dynamically only to changes (when needed) Distributed topology so that scale and centralisation problems are replaced with a robust distributed system Declarative DSL (reusing Puppet DSL for now)
36. CONCLUSIONS
37. JUST A REMINDER …
38. Any
39. @adforminsider