Two shockingly recurring problems

Developing Software architectures is a demanding task that is usually quite rewarding (aka "fun")

## Missing requirements

The whole thing would be even better, if somebody told the development team what exact problem they are supposed to solve.
In other words: Teams urgently need better requirements as the basis of their development work.
In our 20+ years of architecture and development experience, we consistently encountered teams around the world to complain about missing, incomplete or contradicting requirements.
Our brief and fast-paced tutorial summarizes some important and pragmatic approaches that architects and development teams can follow to improve their requirements. It starts with better stakeholder management, more efficient overview, elegant formulation of functional requirements and, last not least, a pragmatic approach to handling quality requirements.

## Technical and other Debt
Which leads to the second recurring issue: The existing systems which most of us regularly work on often carry a load of technical and organizational debt.

In other words: These existing systems are in serious need of improvements, to become maintainable and future-proof.

But how to start? Management does not listen to your pleas and demands more features.
Small refactorings don’t solve the root causes of evil problems.
That’s where systematic architecture improvement comes into play: A pragmatic and simple approach to methodically address all kinds of problems, technical and organizational.

In our workshop, we demonstrate the fundamental techniques and methods for architecture improvement, technology-agnostic and applicable to all kinds of development projects.

Transcript

1. Two Shockingly Recurring Problems Gernot Starke INNOQ Fellow Peter Hruschka Atlantic Systems Guild
2. Re • cur • ring adjective Happening many times, or happening again Foto: Tabitha Turner on unsplash
3. Re • cur • ring Have you ever made the same dumb mistake twice? Have you ever made the same mistake 100 times? Welcome to the real world! Welcome to software development Tom DeMarco
4. Dr. Peter Hruschka üSystem Engineer üCoach, Trainer ü arc42, req42 ü iSAQB, IREB
5. Dr. Gernot Starke INNOQ Fellow üArchitecture-Improver üCoach, Trainer ü arc42, aim42 ü iSAQB
6. https://ahaslides.com/VVNJI action required
7. Solutions to (some) recurring problems Agenda
8. Bad Requirements. Technical and other Debt. Disclaimer: Of course there are other recurring problems J
9. PROBLEM #1 Good (enough) Requirements?
10. garbage in, garbage out
11. Input Requirements Output Software
12. Input Output
13. Input Output ? Photo by John Cameron on Unsplash
14. Requirements-Chaos Instead of „true“ Product Owners: • competing business departments • constantly changing priorities • unclear requirements • diffuse Terms
15. Who is blamed? Development Teams L Photo by Aimee Vogelsang on Unsplash
16. On a scale from 1 (very good) to 5 (very bad) how would you rate your requirements?
17. Requirements-Responsibility (Business Analysts, Product Owner, Requirements Engineer, …) Stakeholder (Users, Clients, Customers, Security-People, Legal Staff, Data Protection Agencies, …) Development Team (Architects, Developers, Testers, …) Only 2 ways to improve the situation
18. Clarifying requirements (& constraints) is part of the tasks of architects evaluate architecture clarify requirements & constraints design technical concepts shepherd realization communicate architecture design structures
19. Architecturally Significant Requirements Requirements, that (can) have special impact on architecture (decisions). These are much less than 25% evaluate architecture clarify requirements & constraints design technical concepts shepherd realization communicate architecture design structures
20. Product Owner Universe *) *) www.req42.de Management Requirements Development Legend:
21. Architecturally Significant Requirements MUST !!! At the beginning only overview, then ”Just in Time” details Top 3 – 5 of each Top10
22. 1 Business Goals 2 Assets 3 Stakeholder 4 Business Scope 5 Product Backlog (Functional Requirements) 6 Domain Terminology 7 Supporting Models 8 Quality Requirements 9 Org. Constraints 10 Roadmaps 11 Risks/Assumptions 12 Teams req42 – The Requirements Framework linearized Form (e.g. for Confluence- Pages) MUST !!! At the beginning only overview, then ”Just in Time” details Top 3 – 5 of each Top10
23. needed (available) Visions/ Goals often implicit Stakeholders often incomplete, not documented Scope not at all or incomplete Functional Requirements Functions & Processes often given, mostly plain text Data often implicit, without glossary Quality Requirements often missing Constraints often known, but not documented
24. Scope (Defining your „Playground“) Stakeholder Vision /Goals The successful (Project) Start Even agile projects need some „upfront“ work Clean Start
25. Visions/Goals The vision describes why the project is being undertaken and what the desired end state is. (Ken Schwaber 2004) You should understand: • How does your product compare to existing product in your company or prodcuts of the competition. What are your „unique selling points“ (USPs) • What is critical for product success? Goals are Requirements that hopefully do not (constantly) change during the planned duration of the project (L) those Vision /Goals
26. Stakeholder Missing Stakeholders = Missing Requirements Photo by Yiran Ding on Unsplash • Persons or organizations that have a direct or indirect influence on the requirements of a system. • All persons or groups that have an interest in the product or the project. • Stakeholders in this context can be more than just customer: – Domain Experts – Subject Matter Experts (SME) – Groups or Organizations (e.g. for standardization) – ...
27. SCOPE = YOUR PLAYGROUND CONTEXT = NOT YOUR PLAYGROUND
28. GPS System position Example: Traffic Pursuit Unit
29. The minimum prerequisite for a clean start Know Your goals Your stakeholders Your scope
30. You will find requirements of different granularity § coarse grained § ..... § fine grained ... and stakeholder will constantly speak on these different levels Functional Requirements The final truth is in the implemented systems (i.e. its software and hardware Features User Stories Epics
31. Epics Features User Storys Refinement “just in time”, driven by needs.. The Product Backlog is a living repository, never finished You have to create a tree-like structure for your functional requirements
32. The T-Model Concentrate on overview (Epics, Features) Delay details (user stories) until needed for implementation Enough overview Details only for the next iteration Degree of details Product Scope Requirements Architecture/Design Coding/Unit Testing Integration/Acceptance Testing
33. You should at least agree on your 10 – 15 most important terms of your domain! Trend: Epics, Features, Stories using natural language, but … …everything that improves communication (Use-Cases, Activity Diagrams, EPKs, Data Models, Wireframes, …) is good practice!
34. Functional Requirements are only “one half of the story" Technical and organizational Constraints Often forgotten needed qualities ✓
35. If you don‘t get good enough requirements as development team ... ... help yourself! Clarify ASRs (Architecturally Significant Requirements)
36. #1 Ensure a „clean start“ Never start without knowing • your visions/goals • your stakeholders • your scope
37. #2 Get an overview of functional requirements to allow for planning and risk management
38. #3 Clarify Quality Requirements and Constraints You will need them for making your architecture decisions
39. PROBLEM #2 Legacy
40. system that is hard to maintain suffers from technical / organizational debt Legacy...
41. 3 https://www.flickr.com/photos/celestinechua/9661913835
42. ??
43. Action required
44. Cannabidiol 100mg/day treatment of neurological disorders: anxiety, chronic pain, epilepsy etc.
45. Therefore: start by getting an overview avoid technology-/solution-driven actionism
46. Systematic Improvement https://aim42.org
47. Some Details
48. Software Reviews ???
49. https://www.flickr.com/photos/jonasb/24341 322
50. https://www.flickr.com/photos/emiliano- iko/5354414276
51. The Microscope Trap If you ONLY look into code, you will only find problems THERE...
53. One man's problem is another man's friend The Relativity Trap
54. expect strange reactions...
55. Analyze: Find Issues analyze e valuate improve https://aim42.org
56. Iteration 2 Analyze: Getting Overview Iteration 1 Breadth-first search Stakeholder Code Context Quality Data Processes
57. VENOM Data Warehouse (Business Intelligence) Buchhaltung, Lohn/Gehalt DATEV Support (Call Center) Buchhaltung Sage etc. Personal- wirtschaft Partner- management Lieferanten- management EMail + Kalender Sharepoint + Wiki CITRIX Facilities Contracts LDAP / SSO Context analysis / Scoping Clarify scope and external interfaces
58. https://ahaslides.com/VVNJI action required
59. Analyze: Quality... Breadth-first search • Are quality requirements met? • Performance • Robustness • Security/Safety • etc... • Method: ATAM Software Product Quality Functional Suitability Reliability Performance efficiency Operability Security Compatibility Maintain- ability Transfer- ability Appropriate- ness Accuracy Compliance Availability Fault tolerance Recover- ability Compliance Time- behaviour Resource- utilisation Compliance Appropriate- ness Recognise- ability Learnability Ease-of-use Helpfulness Attractiveness Technical accessibility Compliance Confidential- ity Integrity Non- repudiation Account- ability Authenticity Compliance Replace- ability Co- existence Inter- operability Compliance Modularity Reusability Analyzability Changeability Modification stability Testability Compliance Portability Adaptability Installability Compliance Stakehold er Code Contex t Quality Data Processes
60. Analyze Quality... Software Product Quality Functional Suitability Reliability Performance efficiency Operability Security Compatibility Maintain- ability Transfer- ability Appropriate- ness Accuracy Compliance Availability Fault tolerance Recover- ability Compliance Time- behaviour Resource- utilisation Compliance Appropriate- ness Recognise- ability Learnability Ease-of-use Helpfulness Attractiveness Technical accessibility Compliance Confidential- ity Integrity Non- repudiation Account- ability Authenticity Compliance Replace- ability Co- existence Inter- operability Compliance Modularity Reusability Analyzability Changeability Modification stability Testability Compliance Portability Adaptability Installability Compliance Pri o Category Szenario 1 Performance Configurator displays potential add-on-parts + services < 5 sec 1 Operability Configuration (private customers) works on iOS & Android 2 Performance Binding offer price displayed < 10 sec 2 Changeabilit y New product category live < 30d ... <etc.> Approaches in Architecture / System storage of add-on parts in DB (no caching) multiple data sources for add- on and service-data Add-ons by wrong data in optical archive Configurator developed in Flash Binding offer price depends on client history, requires access to optical archive <u.v.a.m.> 1 2 3 4
61. Analyse application data 1. Structure 2. Types 3. Access • Read / write 4. Volume • Query-results + indices 5. Correctness 6. Security requirements 7. Distribution / decentralization 8. Duplication / redundancy 9. Throughput -> Runtime analysis Structure and types suitable for the problem read oder write optimization? Overly many of sth? Sth overly large? Wrong/incorrect data? sensitive data? duplicate data items? duplicate data items?
62. Analyse Component Structure
63. https://ahaslides.com/VVNJI action required
64. Analyse Code • (manual) Reviews • Collect Metrics • automated (e.g. SonarQube) • manual (e.g. performance) • Analyse Code • Dependency checks • Hotspots
65. Metrics • Coupling / Dependencies • Code complexity • Performance • … good
66. Metrics • Coupling / Dependencies • Code complexity • Performance • … • Bugs per component • Know-How per component • Change frequency • … good better
67. complex code, that‘s frequently modified. Consider Hotspots
68. complex code, that‘s frequently modified. try for yourself... https://codescene.io/projects/1737/jobs/4353/results/code/hotspots/system-map https://bit.ly/hotspot-sample Hotspot analysis of Linux kernel...
69. https://ahaslides.com/VVNJI action required
70. Analyse Tests • Adequate tests available: • automated... • Unit-, integrationstests • acceptance tests • Others ... • Load tests • Penetration tests • Test infrastructure • Test data Important prerequisite for (low-risk) modification..
71. Analyse Technology Base technology: • Hardware • Programming languages • Operating systems Frameworks, Libraries 3rd-Party Products • Build-/CI-Server • Database • Middleware • Container-/Hypervisor • <etc> current? adequate? used correctly? sufficient know-how
72. Analyse Processes Requirements processes • elicit, clarify, manage Development- /design processes • Architecture, Implementation, Documentation Operations • Deployment, Rollout, Administration, Monitoring Management • Team- and Taskmanagement, Risk management
73. https://ahaslides.com/VVNJI action required
74. now you have.. (large) collection of explicitly known problems Prob. 1 Prob. 1 Prob. 1 Prob. 1 Prob. 1 Prob. 10 Prob. 1 Prob. 1 Prob. 42
75. Evaluate: Prioritize Issues analyze improve e valuate https://aim42.org
76. minor Evaluate – Prioritize What problem is (currently) the worst: costing or hindering the most real minor really serious, dangerous, > 10.000 $ serious, > 1000 $ annoying, hindering a little, >> 100 $ trivial, ok to have, < 100 $ minor minor real real serious real extreme very bad
77. Improve: Resolve Issues e valuate analyze improve https://aim42.org
78. Improvement and Daily Business day-to-day development Practices Practices time Approaches Practices Practices Practices Practices
79. Strategic Improvement Approaches... Build new system Migrate old data to new formats, rewrite corresponding code Make it smaller Partly rework system re-distribute responsibilities in code Separate domain from technology «category» Long-Term Improvement Approach «category» Rewrite «category» Restructure «category» Data Migration «category» Brainsize «category» Improve Modularization «category» Supporting Patterns Supportie improvements, e.g. in operations, test, development.
80. Change by Extraction 2 1 Client Flawed (incohesive) System Client „other“ other features Client Flawed System Client „other“ other features Better other features Client (reduced) Flawed System Client „other“ «category» Restructure «category» Downsize
82. Decorating Collaborator Client Code Flawed Supplier «use» Client Code Flawed Supplier „Proxy“ «use» «delegate» 1. 2. New functionality Client Code Flawed Supplier „Proxy“ «use» «delegate» «decoration» 1. 2. 3. 1 2
83. Get your POs trained -> IREB Get yourself trained in requirements –> ISAQB REQ4ARC Establish a culture of IMPROVEMENT Problems can be solved, but need actions Foto: brett-Jordan on unsplash
84. THANX. Gernot Starke gernot.starke@innoq.com Twitter: @gernotstarke Peter Hruschka peter@systemsguild.com