Architecturally Aligned Testing

Architecture impacts heavily testing. The way you test a monolith is different than how you should test a loosely coupled system. Let’s look at microservices as a prominent exponent of loosely coupled systems.

1. Hier soll der Titel rein www.qs-tag.de Organisator: imbus AG www.qs-tag.de Continuous ‚Everything‘ Architecturally Aligned Testing Stefan Friese HERE Technologies @y3key linkedin.com/in/stefan-friese-12205
2. HERE in numbers 200 Countries mapped 30+ Years of experience transforming location technology 8,000+ Employees in 56 countries focused on delivering the world’s best map and location technologies HERE Maps on board of 100M vehicles and counting 28 TB map data collected per day 4 of 5In-car navigation systems in Europe and North America use HERE maps + collecting data for maps 400 HERE cars 3D data points per second per car 700,000 600k+ km for Autonomous Driving HD Live Map covering 2
3. Why shall we talk about architecture? 3
4. The Test Approach has to be aligned. 4
5. Architecture is abstract until operationalized, when it becomes a living thing. - Patrick Kua, Rebecca Parsons, Neal Ford (2017): Building Evolutionary Architectures. Tests can guide development & provide feedback about the architecture! 5
6. COUPLING COHESION Degree to which elements have knowledge of other elements Separation of concerns & Degree to which the elements belong together 6
7. Tomcat web server war archive persistence layer data layer business rules presentation layer Infrastructure: Physical HW + Full OS search order shipping login User Interface component component component component d o m a i n d i m e n s i o n s Tightly Coupled System: Monolith 7
8. People Coordination Problem In tightly coupled systems, introducing and releasing changes leads to a high synchronization effort. 8
9. 9 Monolith to Microservices
10. catalogCheckout searchCatalog team app team app shipToCustomer team app Container Container Container Loosely Coupled System: Microservices 10
11. “Microservices turn people coordination problems into software coordination problems” -- Kris Jenkins (2018): http://blog.jenkster.com/2018/07/microservices- check-size.html 11 Consider Microservices to scale the organization.
12. Monolith: “In a layered architecture, the focus is on the technical dimension […]: persistence, UI, business rules, etc.”* Microservices: Architecture where the domain dimension is the primary segregation of the architecture. *): https://www.safaribooksonline.com/library/view/building-evolutionary-architectures/9781491986356/ch04.html 12
13. Functional decomposition Independence Loosely coupled Autonomy principle Observability Resilience 6 Key aspects of Microservices 13
14. := Microservices are responsible for one functionality. Makes testing of business domains easier as there is an inherent advantage for testing of smaller bits of uncoupled logic. 6 Key aspects of Microservices Testability 1. FUNCTIONAL DECOMPOSITION 14
15. catalogCheckout searchCatalog team app team app Container Container “Ratio” of tests based on functional decomposition Many Unit Tests Comprehensive sets of Acceptance Tests Contract Tests small set of End User Journey Tests Performance and Fuzzing Tests 15
16. 2. INDEPENDENCE := Microservices can be deployed and released independently, i.e. with Continuous Integration & Continuous Delivery (CI&CD). L Makes integration scenario more dynamic as single microservices can be replaced. J Tests within CI&CD requires test automation and a shift left can ensure fast and meaningful feedback. 6 Key aspects of Microservices Testability 16
17. Cars have brakes so that they can go fast. 17
18. Faster Feedback More Stable Less Expensive to fix Later Feedback More Fragile More Expensive to fix commit release SHIFT LEFT Continuous Integration & Continuous Delivery (CI&CD) 18
19. 3. LOOSELY COUPLED := Microservices are autonomous and loosely coupled since each microservice is physically separated (running in its own container); each domain forms a physical bounded context. They shall communicate via APIs or message queues and shall not share a common data layer. J Importance of API and contract tests J Allows simulation and makes mocking easier L Number of failure points increases 6 Key aspects of Microservices Testability 19
20. Instance: 814 Instance: 243 Instance: 814 Instance: 243 By orchestrating CI&CD pipelines, releases can’t happen independently from each other! This leads to Deployment Coupling Integration Testing 20 Review carefully your integration tests: It’s not appropriate and often expensive to try to test everything in the way you test a monolith.
21. 21 What can you do to avoid an Integration Hell?
22. When you face problems while implementing tests, that’s an important test outcome. 22
23. "If it hurts, do it more frequently, and bring the pain forward." Jez Humble and David Farley (2011): Continuous Delivery. Reliable Software Releases through Build, Test, and Deployment Automation. 23
24. Contract Testing Provider Consumer Broker 1 The consumer captures the expectations of the provider in a separate contract. The contract is shared with the provider so that he can gain insights into the obligations he must fulfill for each consumer. 2 If the provider (upstream dependency) agrees on this contract, those tests are executed in the deployment pipeline of the provider. 3 Contract Tests let the provider and consumers stay agile and the deployment pipelines independent, following the "autonomy over coordination" principle: 24
25. Contract Testing Contract Tests let the provider and consumers stay agile and the deployment pipelines independent, following the "autonomy over coordination" principle: 25 https://raw.githubusercontent.com/wiki/pact-foundation/pact_broker/images/index.png
26. 4. AUTONOMY PRINCIPLE Everybody is responsible for the quality of the product: Microservices are typically owned by cross-functional and autonomous teams, responsible for implementing, testing, releasing, and operating. 6 Key aspects of Microservices Testability 26
27. Conway’s Law: • "Any organization that designs a system […] will inevitably produce a design whose structure is a copy of the organization's communication structure“ • Inverse Conway’s Maneuver Autonomy Principle 27
28. 5. OBSERVABILITY := For microservices, monitoring and logging are first-class concerns. Microservices must support observability as many instances of many different microservices are running in a distributed system. This can be for example done with semantic monitoring / synthetic transactions. 6 Key aspects of Microservices Testability 28
29. 6. RESILIENCE := Microservices isolate failures by acting autonomously. Decompose your system into independently deployable microservices for each failure domain. Design for recoverability. L The number of failure points increases. J Fault tolerance testing of microservices is easier as failures must be isolated. J Consider chaos experiments in production. 6 Key aspects of Microservices Testability 29
30. What comes after Microservices? 30 30
31. https://medium.com/@patrickmichelberger/building-a-serverless-e-commerce-app-with-aws-lambda- stripe-and-react-4663e241710b FaaS – Function as a Service BaaS – Backend as a Service Serverless Shop Frontend (static CDN site) Payments Shop API (AWS Lambda) Order Management JS Plugin 31
32. Serverless Testing The Serverless Architecture is an integration of separate, distributed services. Every single FaaS function needs to be treated as a microservice for testing. More information: • https://read.acloud.guru/testing-and-the-serverless-approach-495cef7495ea • https://serverless.com/framework/docs/providers/azure/guide/testing 32
33. Serverless Testing Challenges • Mocking Unit tests is easy with a FaaS function because the logic is often tiny. However, there is a tendency to an over reliance on mocks. • A FaaS function is dependent on internet/cloud services, which are hard to emulate locally. • Write your business logic so that it is separate from your FaaS provider, to keep it provider-independent, reusable and more easily testable. More information: • https://read.acloud.guru/testing-and-the-serverless-approach-495cef7495ea • https://serverless.com/framework/docs/providers/azure/guide/testing 33
34. Walking on Water and Testing Software are easy if both are frozen. Adapted from: Edward V Berard Painting: Hagemeister (https://bit.ly/2yqPlWx) 34
35. The cost of fixing a bug may be less than the cost of insuring no bugs appear. Try to understand the trade-off between putting more efforts into testing and detecting issues faster in production (Sam Newman 2015, p. 153) 35
36. But Keep in Mind: Every test adds a negative impact on the length of the feedback loop within CI&CD. Strong Test Automation is Fundamental 36
37. Automate Everything? Test Automation • “Shift Left”: fast and meaningful feedback by a comprehensive set of automated tests! Continuous Testing • Shift Left • Fast left to right flow 37
38. Based on: https://danashby.co.uk/2016/10/19/continuous-testing-in-devops Dev Build Code Plan Ops Test Here Test Here Test Here Test Here Test Here Test Here Test Here 38
39. (Semantic) Monitoring Some E2E Tests! small risk high risk that bugs are not detected Unit Tests! Acceptance Tests Incremental Release A/B Tests Alpha/Beta Releases Controlled Experiments (e.g. “ChaosMonkey”) Contract Tests Pre-Production Tests (Shift Left) Production Tests (Fast left to right flow) Based on: https://danashby.co.uk/2016/ 10/19/continuous-testing-in- devops/ Bug Filter 39 Exploratory Tests parallel to development
40. As tester, you cannot stop changes, but you can manage them! 40
41. Companion Article: https://www.infoq.com/articles/architecturally-aligned-testing @y3key linkedin.com/in/stefan-friese-12205 41