Agile Architecture and Innovation

It’s well-known that software architecture, organisation, processes and the humans affected by them are interrelated. But how does this translate into advice for practitioners’ daily work?

In this session, we’ll look at the interplay between these disciplines, examining how architecture both constrains, shapes and is shaped by agile methods, and derive some lessons for both sides. We’ll look at some patterns and anti-patterns of architectural work in agile development efforts, and examine how a “good” architecture can help projects succeed.

Transcript

1. Agile Architecture and Innovation Stefan Tilkov
 stefan.tilkov@innoq.com
 @stilkov Agile Cambridge 2019 "Science Museum, ceiling" by Elecé is licensed under CC BY 2.0
2. Topics we’ll ignore today:
3. Whether you need “Architecture” (you have no choice)
4. Whether your Agile project needs an “Architect” (no, but someone will do that architecting)
5. What to document (structure, dependencies, principles, choices)
6. Whether there should be “Architecture Stories” (no)
7. Whether the whole discussion might actually be quite pointless (possibly)
8. Instead: Architecture
9. @stilkov Conscious trade-offs over emerging architecture Documented rationale over quick ad-hoc decisions Sustained changeability over easy initial development Design for replacement over design for re-use Simplicity over fast delivery Architecture Manifesto Attempt
10. @stilkov (Software) Architecture Definitions A system’s elements, their relationships, and the rules and principles that govern their design and evolution Whatever the architect considers important enough to merit their attention Decisions that you want to be correct because they are costly to change
11. Awesome Shop CMS Archive General Ledger Print Shop HR
12. Awesome Shop CMS Archive General Ledger Print Shop HR Context
13. Awesome Shop CMS Archive General Ledger Print Shop HR Invoicing Accounting Auth Catalog Checkout & Order Search
14. Awesome Shop CMS Archive General Ledger Print Shop HR Invoicing Accounting Auth Catalog Checkout & Order Search Domain Architecture
15. Invoicing Accounting Auth Catalog Checkout & Order Search
16. Macro Architecture
17. Ruby on Rails MySQL Java Spring Boot OSS Product COTS Java Spring Boot NodeJS ElasticSearch
18. Ruby on Rails MySQL Java Spring Boot OSS Product COTS Java Spring Boot NodeJS ElasticSearch Micro Architecture
19. Invoicing Accounting Auth Catalog Checkout & Order Search
20. Invoicing Accounting Auth Catalog Checkout & Order Search
21. Invoicing Accounting Auth Catalog Checkout & Order Search
22. Invoicing Accounting Auth Catalog Checkout & Order Search Team Architecture?
23. @stilkov If your goal is to support autonomous teams, architecture is an essential ingredient
24. @stilkov Size is the #1 enemy of agility. Keep your systems as small as you can.
25. @stilkov Coming up with the “right” system boundaries is an architecture activity that must be done first
26. @stilkov Managing dependencies is the most important ongoing architecture task
27. @stilkov t Simplicity Homogeneity Cohesion Decoupling Modularity (Support for) Heterogeneity Autonomy Ease of development
28. @stilkov number of
 developers strength of 
 decoupling methods modules components μservices systems
29. @stilkov From a layered system … System Logic Data UI Module Module Module
30. @stilkov … to a system of systems System System System Logic Data UI Logic Data UI Logic Data UI
31. Invoicing Accounting Auth Catalog Checkout & Order Search
32. Invoicing Accounting Auth Catalog Checkout & Order Search Team Architecture!
33. @stilkov Benefits 1. Isolation 2. Autonomy 3. Scalability 4. Resilience 5. Speed 6. Experimentation 7. Rapid Feedback 8. Flexibility 9. Replaceability 10. Ecosystem
34. @stilkov Centralization vs. Autonomy: Cases
35. @stilkov Context: • … Observation(s): • … Lesson(s) learned: • …
36. @stilkov Context: • E-Commerce/Online shop (Retail) • 100-120 developers, ~10 teams Observation(s): • Lack of front-end expertise led to central UI/design team, bottleneck for development, deployment, operations, evolution Lesson(s) learned: • Local optimization needs can trigger centralization • Full stack teams require full stack capabilities
37. @stilkov A general lack of specific skills, combined with a select few who have it, will sabotage any attempt at decentralizing anything requiring it
38. @stilkov Context: • E-Commerce/Online shop (Retail) • 100-120 developers, ~10 teams Observation(s): • Extremely inefficient UI integration runtime due to lack of standardization • Vast differences in API style, formats, documentation Lesson(s) learned: • Complete lack of guidance creates unproductive diversity
39. @stilkov You cannot decide to not have an architecture; if you don’t actively create it, be prepared to deal with the one that emerges
40. @stilkov There’s a fine line between diversity (that adds value) and chaos (that doesn’t)
41. @stilkov Context: • Insurance customer portal • 10-15 developers, 1 team Observation(s): • Potential for independent decisions in separated systems (almost) never exploited • Engineering effort spent on coordination Lesson(s) learned: • Premature modularization can lead to increased effort without matching benefits
42. @stilkov Context: • E-Commerce/Online shop (Retail) • 100-120 developers, ~10 teams Observation(s): • Common standard micro architecture at start of project • Gradual increase in degrees of freedom • Increase in actual diversity of tools, languages, architecture Lesson(s) learned: • Increased maturity allows for less dogma/fewer rules
43. Pattern: Regulated Market
44. @stilkov Dreyfus model of skill acquisition Novice Advanced Beginner Competence Proficient Expert Recollection Non- Situational Situational Situational Situational Situational Recognition Decomposed Decomposed Holistic Holistic Holistic Decision Analytical Analytical Analytical Intuitive Intuitive Awareness Monitoring Monitoring Monitoring Monitoring Absorbed Quality Stage
45. @stilkov Growing architectural maturity means less guidance and rules are needed
46. @stilkov The more experienced you are at (active and passive) architectural governance, the less you can do of it
47. Takeaways
48. @stilkov 1. There is no conflict between Agile and Architecture
49. @stilkov 2. Size matters
50. @stilkov 3. Architecture must match Organization
51. Stefan Tilkov @stilkov
 stefan.tilkov@innoq.com
 Phone: +49 170 471 2625 innoQ Deutschland GmbH Krischerstr. 100 40789 Monheim am Rhein Germany Phone: +49 2173 3366-0 innoQ Schweiz GmbH Gewerbestr. 11 CH-6330 Cham Switzerland Phone: +41 41 743 0116 www.innoq.com Ohlauer Straße 43 10999 Berlin Germany Phone: +49 2173 3366-0 Ludwigstr. 180E 63067 Offenbach Germany Phone: +49 2173 3366-0 Kreuzstraße 16
 80331 München Germany Phone: +49 2173 3366-0 @stilkov That’s all I have.
 Thanks for listening!
52. @stilkov www.innoq.com OFFICES Monheim Berlin Offenbach Munich Hamburg Zurich FACTS ~150 employees Privately owned Vendor-independent SERVICES Strategy & technology consulting Digital business models Software architecture & development Digital platforms & infrastructures Knowledge transfer, coaching & trainings CLIENTS Finance Telecommunications Logistics E-commerce Fortune 500 SMBs Startups
53. @stilkov Problems Some People Have Building features takes too long Technical debt is well-known, yet not addressed Deployment is way too complicated and slow Replacement would be way too expensive Scalability has reached its limit Architectural quality has degraded “-ility” problems abound