The C4 Test Pyramid An Architecture Driven Test Strategy DB Systel GmbH | Christian Fischer | OOP 2021

What are your QA pains? DB Systel GmbH | Christian Fischer | OOP 2021 2 With each new feature we have to refactor our tests. We have to write a lot of redundant tests. We still have blind spots. Failing tests are not quite significant. We need a test strategy to balance all QA aspects: ‒ Coverage ‒ Effort ‒ Robustness ‒ Significance

The Test Pyramid to the Rescue? DB Systel GmbH | Christian Fischer | OOP 2021 3 Service UI Unit Test Pyramid Principles 1. Define different scope levels 2. No redundancy: Test one aspect only on one scope level 3. Test on the lowest possible scope level Mike Cohn § E2E Tests § Manual Tests § System Tests § Integration Tests § Acceptance Tests

Everything clear now? DB Systel GmbH | Christian Fischer | OOP 2021 4 What is an integration test? Which pyramid should we choose? On which level do I test my use case? Which tools and fixtures do we use? We need a test strategy which defines: - on which scope level - to test which behaviour or capability - to accomplish which coverage - with which test methods/fixtures

It‘s an Engineering Process… DB Systel GmbH | Christian Fischer | OOP 2021 5 Test Strategy Define Scope Levels Select Test Methods/ Fixtures Determine Coverage Specify Behaviour/ Capabilities Scope Level C4 Model Test Engineering

C4 Model for Visualizing Software Architecture DB Systel GmbH | Christian Fischer | OOP 2021 6 Code Views The code elements of a component (e.g. Class, Interface etc.) + - Context View The system as a black box with all of its collaborators (external Systems, Users etc.) Component Views The software components of a container (e.g. Java Package, Framework Implementation etc) Container View The deployed runtime elements (e.g. Web Services, Mobile Apps etc.) of the system Not relevant for Test Scope

An Example: The Train Capacity Management System DB Systel GmbH | Christian Fischer | OOP 2021 7 Hamburg Hannover Frankfurt Würzburg München leaving: 32 entering: 64 leaving: 74 entering: 27 leaving: 22 entering: 177 Passengers: 70 Seats: 256 Passengers: 112 Seats: 256 Passengers: 65 Seats: 256 Passengers: 220 Seats: 347 XXX Train Stop

Context View DB Systel GmbH | Christian Fischer | OOP 2021 8 Train Capacity Management Service [Software System] Ticket System [Software System] User [Person] External Consumer [Software System] API Queries [HTTPS/JSON] Train Ride View Train Information [HTTPS/FileUpload] Ticket Information [AMQPS] Train Management System [Software System] BEHAVIOUR/CAPABILITIES System Level Use Cases § REST Query Use Cases § Web UI Use Cases § Train Data Upload § Ticket Information Message Delivery TEST METHODS Smoke Test/ Monitoring § REST Assured § Selenium § Log Monitoring

Context View DB Systel GmbH | Christian Fischer | OOP 2021 9 Capacity DB [Container: Postgresql Schema] Ticket System [Software System] Train Capacity Management System Train Capacity Service [Container: Spring Boot REST Service] Client Backend [Container: Spring Boot Rest Service] API Service [Container: Spring Boot REST Service] Capacity Queries [HTTPS/JSON] Capacity [SQL] Capacity Queries [HTTPS/JSON] User [Person] Client Queries [HTTPS/JSON] External Consumer [Software System] API Queries [HTTPS/JSON] Train Ride View Train Information [HTTPS/FileUpload] Ticket Information [AMQPS] Client [Container: ReactJS WebApp] Train Management System [Software System] COVERAGE § Communication Paths § Container Runtime Configuration BEHAVIOUR/CAPABILITIES System Level Use Cases § REST Query Use Cases § Web UI Use Cases § Train Data Upload § Ticket Information Message Delivery TEST METHODS Smoke Test/ Monitoring § REST Assured § Selenium § Log Monitoring

Container View (Excerpt) DB Systel GmbH | Christian Fischer | OOP 2021 10 Train Capacity Service [Container: Spring Boot REST Service] API Service [Container: Spring Boot REST Service] Client Backend [Container: Spring Boot Rest Service] Capacity Queries [HTTPS/JSON] Capacity DB [Container: Postgresql Schema] Train Management System [Software System] Train Information [HTTPS/FileUpload] Ticket System [Software System] Ticket Information [AMQPS] Capacities [SQL] Capacity Queries [HTTPS/JSON] BEHAVIOUR/CAPABILITIES Container Use Cases § REST Query Use Cases § Train Information Upload § Ticket Information Message Delivery TEST METHODS E2E Test § Spring Boot Test + Postgresql Testcontainer

Container View (Excerpt) DB Systel GmbH | Christian Fischer | OOP 2021 11 API Service [Container: Spring Boot REST Service] Client Backend [Container: Spring Boot Rest Service] Capacity Queries [HTTPS/JSON] Capacity DB [Container: Postgresql Schema] Train Management System [Software System] Train Information [HTTPS/FileUpload] Ticket System [Software System] Ticket Information [AMQPS] Capacities [SQL] Capacity Queries [HTTPS/JSON] Train Capacity Service Capacity REST API [Component: Spring REST Controller] Train REST API [Component: Spring REST Controller] Capacity Repository [Component: Spring Repository] Ticket Queue Consumer [Component: Spring Queue Listener] Query Manager [Component: Java Package] Train Importer [Component: Java Package] Ticket Importer [Component: Java Package] Oauth Access Config [Component: Spring Security Configuration] COVERAGE TEST METHODS BEHAVIOUR/CAPABILITIES Container Use Cases § REST Query Use Cases § Train Information Upload § Ticket Information Message Delivery E2E Test § Spring Boot Test + Postgresql Testcontainer § Component Composition § Application Configuration

Component View (Excerpts) DB Systel GmbH | Christian Fischer | OOP 2021 12 Capacity Queries [HTTPS/JSON] Query Manager [Component: Java Package] Capacity Queries [API] API Service [Container: Spring Boot REST Service] Capacity REST API [Component: Spring REST Controller] Query Manager [Component: Java Package] BEHAVIOUR/CAPABILITIES Technical & Business Specification § Use Cases § Business Rules § Protocols § Technical Capabilities TEST METHODS Unit & Integration Tests § Junit Tests § Spring Integration Tests + Mocks COVERAGE § All Execution Paths § Corner Cases

Test Strategy for TCMS DB Systel GmbH | Christian Fischer | OOP 2021 13 Tests for System Level Use Cases covering Communication Paths and Container Runtime Configurations with Smoke Test and Monitoring Tests for Container Use Cases covering Component Compositon and Application Configuration with E2E Tests Tests for Use Cases, Domain Policies, Protocols and Technical Capabilities covering all Execution Paths and Corner Cases with Integration and Unit Tests

And what about the other Test Shapes? DB Systel GmbH | Christian Fischer | OOP 2021 14 Nonfunctional Tests Write tests. Not too many. Mostly integration. Integration Tests should be preferred, because they … • have a higher Confidence Quotient • can replace lower level tests when fast,reliable and cheap If my high level tests are fast, reliable, and cheap to modify - then lower-level tests aren't needed. -- Martin Fowler in TestPyramid But everything comes with a price Tag! Integration Test have • lower singnificance • higher combinational Complexity

And what about the other Test Shapes? DB Systel GmbH | Christian Fischer | OOP 2021 15 A Test Strategy is an Engineering Process! You can make exceptions from your Principles but beware of the Tradeoffs. Nonfunctional Tests

Key Take Aways DB Systel GmbH | Christian Fischer | OOP 2021 16 - Your Test Strategy should define on which scope level to test which behaviour or capability to accomplish which coverage with which test methods/fixtures. - The C4 views and the principles of the test pyramid can be combined to define scope and coverage for each level of the test pyramid. ‒ Developing a test strategy is an engineering process. Know your Tradeoffs.

www.dbsystel.de Contact Christian Fischer Software Engineering Coach DB Systel GmbH @agiledojo Thank you for your attention