The Echidna pattern; scaling platforms without losing flexibility

In this talk i show you how the Echidna pattern, something i came up with during many engagements with customers, can help you scaling your platform, interacting with external systems but stay slim and agile and in control over your own backlog. Follow the party of happy developers and see how they fight the Wizards of external dependencies with the spell of Echidna

Transcript

2. Once upon a time…
4. There was a party of happy developers… • They successfully launched their platform • They told everyone about it • People liked it and started to use it with joy • And with their usage, their thirst for more features and integrations arose…
5. And so…
7. The wizards of external requirements came… • As people started to like our party’s platform, they came for audiences with extra requirements • The party started then to integrate this platform, adapt that protocol, get data from another system… • More and more requirements came in by external stakeholders from near and far…
9. And the happy developers became worried… • Gradually they lost control over their backlog, as they became more and more dependend on external systems. • Their testing became more and more complex. • Their velocity dropped and dropped. • In the end their shiny platform became a rigid system that was not fun anymore to maintain and grow. • The once happy party became sad and worried about their precious platform.
10. The dreaded big ball of mud
11. Luckily…
13. The No! solution • One option to prevent ending in the big ball of mud scenario was to say NO to external requests. • This worked but disappointed and angered important stakeholders • The solution stayed flexible and slim, but did also not grow • The party was not perceived as interested in embracing change anymore.
14. The sea of insignificance
15. Fear not dear developers…
16. The Echidna is here!
17. The Echidna-Basics • The Echidna protects its core at all costs (Black Box) • For interaction with external systems, the Echidna pattern uses 2 primary patterns: • APIs to receive, provide or modify data (client-invoked) • Hooks to inform external systems about relevant changes of specific data objects and –types (server-invoked) • Interaction beyond the above stated ways is not permitted / possible • External interactions must pass an anti-corruption layer to talk to our core.
18. Touch me gently, or get hurt…
19. The map of Echidna
21. Elements of Echidna • The Echidna is a system design pattern • It has a core system, which is the heart of the platform • The core is surrounded by an anti-corruption layer • Outside the anti-corruption layer there are communication elements to handle API and hook interactions. • An IAM and Webhook registration service limit access to defined usergroups Choose wisely and do not overdo it, fellow developer
22. The legendary artefacts of Echidna…
24. Contracts, contracts, contracts… • To ensure adhesion to hooks and APIs you need to publish and maintain contracts. • For API I do recommend OpenAPI as standard or a client SDK • For hooks I would recommend an SPI (Service Provider Interface) or an SDK for supported languages • All contracts need a proper versioning, lifecycle and schema • Contracts enable you to stay in charge in what way an external application can interact with you
26. Anti corruption layer • To ensure that your core is protected and independent from your external dependencies, you need to integrate and anti-corruption layer • This layer serves as a protection shield to ensure that only domain-valid data is entering the system • Furthermore, it also ensures that the core data is translated into the respective communication objects for the external dependency
27. Be cautious, fellow developer…
29. The fallacy of plugins • Plugins are a pattern that is living inside your application scope. • A plugin is aggregated or constructed by your system • If plugin code crashes or misbehaves, your system is affected directly • This does not solve your dependency problem and eventually slows you down as well • Try to avoid plugins or external modules if possible • In case you want to incorporate a dependency, make it part of your codebase.
30. And so, they lived happily ever after…
31. Summary • Echidna keeps you in charge of your codebase without losing flexibility to interact with other systems. • It is a system design pattern similar to Ports and Adapters • Interactions with external dependencies happen via APIs (client invoked interaction) or hooks (server-invoked interaction) • APIs and hooks should be versioned, have a schema and a lifecycle • Keep the amount of interaction points as low as possible, be rather hesitant to add new ones. • Do prefer to integrate features you want to incorporate over creating a plugin or module
33. The one who brought this tale to you
34. The End