Testing Reconsidered

Given an opportunity to work on a green fields project, a team that I worked closely with challenged our current monolithic testing practices and attempted to follow a new tactical approach more suitable for the added complexity of shifting to micro services.

This is a story of how we attempted to implemented a continuous delivery pipeline that got us very close to the utopia of deploying every single commit to production at a click of a button. We've had to challenge the way we think and talk about tests, and most certainly the way that we write them. More focus on smaller and concise unit tests, integration tests that stop strictly short at boundaries, and the blurring of the boundaries between what should be covered by developers and testers.

Transcript

1. Testing Reconsidered A journey to Continuous Delivery @hadynz
2. Context is important
3. Key Takeaways: 1. Contract Driven Testing 2. Combination of testing strategies leads to a high test coverage 3. Everyone is responsible for testing
4. How the story starts....
5. Continuous Delivery Image Reference: http://www.theimaginativeconservative.org Continuous delivery is a series of practices designed to ensure that code can be rapidly and safely deployed to production by delivering every change to a production-like environment and ensuring business applications and services function as expected through rigorous automated testing.
6. Illustrations based on: https://puppet.com/blog/continuous-delivery-vs-continuous-deployment-what-s-diff
7. Working in Small Batches Image Reference: http://nydailynews.com
8. Rigorous Automated Testing Image Reference: http://athletics.wikia.com/wiki/US_Air_Force_Fitness_Test
9. With shift to “micro” services Monolithic testing practices need to be reconsidered Image Reference: http://www.bbc.com/earth/story/20170127-how-to-d ecide-which-extinct-species-we-should-resurrect Need new approaches for managing additional complexity: • Multiple independently deployable components • Scale and maintain each component • Multiple teams acting as guardians for different services
10. Monolith vs Microservices Data Access Layer Business Logic UI Microservice Microservice Microservice UI
11. Contract Driven Testing Image Reference: http://tumg.co.nz/new-standard-energy-contracts
12. Consumer Driven Contracts Service Provider (Client Service) Consumer (Client Web) Consumer (Activity Web) Implementation Boundary Remote Boundary Apiary API Blueprint Swagger Illustrations based on: http://thoughtworks.github.io/pacto/patterns/cdc/
13. Consumer Driven Contracts Service Provider (Client Service) Consumer (Client Web) Consumer (Activity Web) Implementation Boundary Remote Boundary Contract testing tools to guarantee fulfillment (e.g. Pact, Chappie) Stub Service Illustrations based on: http://thoughtworks.github.io/pacto/patterns/cdc/
14. Real Stub Real Stub Stub* + Stage Developer Pre-Prod Prod Environments Build
15. Testing Processes
16. Practice Service Organisation Service
17. Resources Service Layer Domain Repositories HTTP Client Data Mapper / ORM External Datastore External Service Illustration Reference: https://martinfowler.com/articles/micr oservice-testing/#anatomy-modules
18. Resources Service Layer Domain Repositories HTTP Client Data Mapper / ORM External Datastore External Service Unit Testing • Typically written at a class level, or a function level • Avoid mocks • Need for mocks highlight when a module should be broken • Powerful design tool • Does NOT test state changes
19. Execute(List events) { events .Where(IsPartnerSwap) .Select(Transform) .ForEach(Create); } private IsPartnerSwap(event) { ... } private Transform(event) { ... } private Create(practice) { ... } public public public But not on interface
20. Resources Service Layer Domain Repositories HTTP Client Data Mapper / ORM External Datastore External Service Integration Testing • Full microservice in-memory (OWIN) • Using real DB and SP’s • All interactions make use of real network calls • Test double help emulate unhappy paths • FED - Use a real browser on build agents Component Stub Service Pact spins up a real server to act as a test double Each test specifies pre-defined responses
21. Acceptance Testing
22. A combination of testing strategies leads to a high test coverage Image: https://martinfowler.com/articles/microservice-testing/#conclusion-test-pyramid
23. Testing Strategies
24. Working in small, coherent increments whilst preserving functionality Deployment to Production is part of Definition of Done.
25. You will be shipping bugs Mitigated by monitoring and fast deployment pipeline for rollback/rollforward
26. Testing Culture
27. Developers have to prove their stuff works And keeps working all the way in Production
28. Everyone is responsible for quality Testing scenarios and strategy part of all refinement discussions
29. In-Depth Code Reviews Looking for valuable tests, that give confidence and are easy on the eyes
30. No team member to be left out of the journey Shared understanding of the stack cultivates shared ownership
31. Where does this leave the QA role?
34. Key Takeaways: 1. Contract Driven Testing 2. Combination of testing strategies leads to a high test coverage 3. Everyone is responsible for testing