[JUG Saxony Day 2017] Eine konkurrenzfähige Architektur für den Lebensmittelhandel

Vor drei Jahren haben wir begonnen einen monolithischen Shop in eine Microservice-basierte eCommerce-Plattform zu migrieren und sind dabei von 2 Teams auf heute 180 Entwickler gewachsen. In diesem Vortrag möchten wir unsere, manchmal auch schmerzhaften, Erfahrungen teilen.
Nachdem wir einige Begriffe zu Microservices geklärt haben wollen wir die folgenden Fragen beantworten:

- Welche Organisationsstruktur wird benötigt um einen vertikalen Softwareschnitt zu fördern und gleichzeitig schnell zu wachsen?
- Wie definiert man bounded-contexts mit vielen Teams und Features? Gibt es Wege die Teams auf allen Ebenen autonom arbeiten zu lassen?
- Wie können verschiedene Teams Features unabhängig voneinander entwickeln und in die Produktion bringen?
- Wie verändert asynchrone Kommunikation mittels Kafka die Art über Entitäten nachzudenken?
- Wie können verschiedenste Microservices zu einer gemeinsamen Website beitragen? (Und warum man dies vielleicht zweimal entwickelt…)

Wir möchten gerne unsere Interpretation einer Microservice Architektur teilen in der Entwickler die Fäden in der Hand halten und durch Teilhabe einen großen Einfluss auf die entstehende Architektur haben.

Transcript

1. A competitive food retail architecture with microservices A three year journey towards microservices / JUG Saxony Day 2017 Ansgar Brauner & Sebastian Gauder @a_brauner & @rattakresch
2. Agenda 1) Who we are and where we come from a) REWE Group & REWE digital 2) How to ... a) … scale monoliths? b) … determine boundaries? c) … guide developers? d) … access a microservice architecture? 3) Questions?
3. Our history
4. 4 Details REWE GROUP Turnover >54 bn History 90 years Employees >330.000 Inudstries Food Retail, Tourism, DIY Shops >15.000
5. Our history (Spoiler alert)
6. How to scale monoliths?
7. Design Goals Vertical Boundaries Decentralization
8. “Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.” Melvin Conway (1967) Conway’s law
9. How to organize teams to build a scalable system? Functional and vertical teams
10. How to determine boundaries?
11. Customer journey defines bounded contexts
12. How to guide developers?
13. Design Goals - Vertical boundaries - Decentralization Architectural principles - Collection of 9 basic ‘laws’ - Autonomy, Automation and Communication Guides - Practical manual for common tasks (RFC 2119) - e.g. Eventing, REST, Authentication Guarding rails for developers MUST SHOULD COULD
14. autonomy
15. Autonomy principles Deploy independently Ensure that the services can and are deployed by the teams themselves. Isolate failure Make the services as resilient as possible. Hide implementation Details Different verticals must not share state. Verticals hide their implementation details. Encapsulate Data Storage For any data resource exactly one service is responsible. If possible, the data supply should proceed asynchronously in the background
16. Autonomous Teams - challenge for product owners as they have to rethink their features - they might have to reduce the scope - split the original feature into smaller ones for each team / bounded context Data-integration pattern - Change your service behavior depending on the data you have to display - Enable others to deliver functionality later Evolving Implementations - Start with a fitting but simple solution and get better by every team implementing it Challenges
17. automation
18. Automation principles Be Scalable Services are scaled horizontally Embrace a Culture of Automation Test, deployment and operations are completely automated Be Highly Observable Use of semantic monitoring to see if the whole platform works correctly.
19. communication
20. Communication principles Follow REST Principles The API of a service follows the RESTful paradigm Standardize Service communication Inter-service communication is standardized and if possible asynchronous
21. Asynchronous > Synchronous - Have as much needed data locally as possible -> resilience at request time - Duplication of data is accepted and wanted - Only communicate synchronously if - process needs to be synchronous - data is not available asynchronously (yet) Communication in microservice worlds Having data is better than fetching data
22. Eventing with Apache Kafka • Eventing != Messaging ◦ Publish events that already happened ◦ One owning service per queue/topic ◦ Independence at request time • Complete entities - not deltas ◦ Atomic ◦ Allows log compaction • Re-writing and Re-reading ◦ needed in case of additional data ◦ useful in case of data loss
23. Kafka is an API as well - Events have to behave like APIs and avoid breaking changes. Writing APIs is hard - Teams tend to write APIs for special clients (e.g. mobile apps) - New clients often don’t match and require API changes Breaking changes require a strict procedure - Could be solved by new endpoints, new topics Lessons learned with microservice APIs
24. Breaking changes in APIs
25. How to access?
26. Usage of API gateways as entry points How to access Microservices?
27. Dynamic frontend composition How UIs in microservice architectures should be composed
28. Dynamic UI Composition with DUC Constraints - There is no dedicated FE Team - CSS and JavaScript are delivered from the services - A/B testing should be possible in all services - Authentication and request-enhancement is handled in the gateway component
29. Dynamic UI composition with DUC Where does my UI come from?
30. Thank you Questions ?
31. A competitive food retail architecture with microservices A three year journey towards microservices / JUG Saxony Day 2017 Ansgar Brauner & Sebastian Gauder @a_brauner & @rattakresch