Slide 1
Slide 1 text
Testing Reconsidered
A journey to Continuous Delivery
@hadynz
Slide 2
Slide 2 text
Context is important
Slide 3
Slide 3 text
Key Takeaways:
1. Contract Driven Testing
2. Combination of testing strategies leads to a high test coverage
3. Everyone is responsible for testing
Slide 4
Slide 4 text
How the story starts....
Slide 5
Slide 5 text
Continuous Delivery
Image Reference: http://www.theimaginativeconservative.org
Continuous delivery is a series of practices designed to ensure
that code can be rapidly and safely deployed to production by
delivering every change to a production-like environment
and ensuring business applications and services function as
expected through rigorous automated testing.
Slide 6
Slide 6 text
Illustrations based on:
https://puppet.com/blog/continuous-delivery-vs-continuous-deployment-what-s-diff
Slide 7
Slide 7 text
Working in Small Batches
Image Reference: http://nydailynews.com
Slide 8
Slide 8 text
Rigorous Automated Testing
Image Reference: http://athletics.wikia.com/wiki/US_Air_Force_Fitness_Test
Slide 9
Slide 9 text
With shift to “micro” services
Monolithic testing practices
need to be reconsidered
Image Reference:
http://www.bbc.com/earth/story/20170127-how-to-d
ecide-which-extinct-species-we-should-resurrect
Need new approaches for managing
additional complexity:
● Multiple independently deployable
components
● Scale and maintain each component
● Multiple teams acting as guardians
for different services
Slide 10
Slide 10 text
Monolith vs Microservices
Data Access
Layer
Business
Logic
UI
Microservice Microservice Microservice
UI
Slide 11
Slide 11 text
Contract Driven Testing
Image Reference: http://tumg.co.nz/new-standard-energy-contracts
Slide 12
Slide 12 text
Consumer Driven Contracts
Service Provider
(Client Service)
Consumer
(Client Web)
Consumer
(Activity Web)
Implementation
Boundary
Remote
Boundary
Apiary API Blueprint
Swagger
Illustrations based on:
http://thoughtworks.github.io/pacto/patterns/cdc/
Slide 13
Slide 13 text
Consumer Driven Contracts
Service Provider
(Client Service)
Consumer
(Client Web)
Consumer
(Activity Web)
Implementation
Boundary
Remote
Boundary
Contract testing
tools to guarantee
fulfillment
(e.g. Pact, Chappie)
Stub
Service
Illustrations based on:
http://thoughtworks.github.io/pacto/patterns/cdc/
Slide 14
Slide 14 text
Real
Stub
Real
Stub Stub*
+
Stage
Developer Pre-Prod Prod
Environments
Build
Slide 15
Slide 15 text
Testing Processes
Slide 16
Slide 16 text
Practice
Service
Organisation
Service
Slide 17
Slide 17 text
Resources
Service Layer
Domain
Repositories
HTTP Client
Data Mapper / ORM
External
Datastore
External
Service
Illustration Reference:
https://martinfowler.com/articles/micr
oservice-testing/#anatomy-modules
Slide 18
Slide 18 text
Resources
Service Layer
Domain
Repositories
HTTP Client
Data Mapper / ORM
External
Datastore
External
Service
Unit Testing
● Typically written at a class
level, or a function level
● Avoid mocks
● Need for mocks highlight
when a module should be
broken
● Powerful design tool
● Does NOT test state
changes
Slide 19
Slide 19 text
Execute(List events) {
events
.Where(IsPartnerSwap)
.Select(Transform)
.ForEach(Create);
}
private IsPartnerSwap(event) {
...
}
private Transform(event) {
...
}
private Create(practice) {
...
}
public
public
public
But not on interface
Slide 20
Slide 20 text
Resources
Service Layer
Domain
Repositories
HTTP Client
Data Mapper / ORM
External
Datastore
External
Service
Integration Testing
● Full microservice
in-memory (OWIN)
● Using real DB and SP’s
● All interactions make use of
real network calls
● Test double help emulate
unhappy paths
● FED - Use a real browser on
build agents
Component
Stub
Service
Pact spins up a real server
to act as a test double
Each test specifies
pre-defined responses
Slide 21
Slide 21 text
Acceptance Testing
Slide 22
Slide 22 text
A combination of testing
strategies leads to a high
test coverage
Image: https://martinfowler.com/articles/microservice-testing/#conclusion-test-pyramid
Slide 23
Slide 23 text
Testing Strategies
Slide 24
Slide 24 text
Working in small, coherent increments whilst
preserving functionality
Deployment to Production is part of Definition of Done.
Slide 25
Slide 25 text
You will be shipping bugs
Mitigated by monitoring and fast deployment pipeline for
rollback/rollforward
Slide 26
Slide 26 text
Testing Culture
Slide 27
Slide 27 text
Developers have to prove their stuff works
And keeps working all the way in Production
Slide 28
Slide 28 text
Everyone is responsible for quality
Testing scenarios and strategy part of all refinement discussions
Slide 29
Slide 29 text
In-Depth Code Reviews
Looking for valuable tests, that give confidence and are easy on the eyes
Slide 30
Slide 30 text
No team member to be left out of the journey
Shared understanding of the stack cultivates shared ownership
Slide 31
Slide 31 text
Where does this leave the QA role?
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
No content
Slide 34
Slide 34 text
Key Takeaways:
1. Contract Driven Testing
2. Combination of testing strategies leads to a high test coverage
3. Everyone is responsible for testing
Slide 35
Slide 35 text
No content