Testing RESTful Service Providers

A presentation that explains end-to-end testing, why it's useful, some strategies for doing it, and a tool to make your life easier when creating testing doubles of external services.

Transcript

1. Testing RESTful Service Providers
2. Nick Presta Software Developer @ Wave
3. Sound familiar? • Application which talks to a 3rd party • Code which relies on services that aren’t built yet • Need to inspect 3rd party traffic due to poor docs
4. End-to-end tests • What are they • Why you want them • How to write them
5. Short Detour
6. Types of testing • Unit • Isolated from collaboration • Integration • Code in your system working together • End-to-end • Does everything work together
8. What are E2E tests? • Total black box • Interact with the system as a consumer would • Includes interactions with 3rd party services
9. Why E2E tests? • Interact with your service using public interfaces • Surface poor API decisions • Identity all points of failure with 3rd parties • Build confidence in a “bug detector”
10. Simple system architecture HAProxy NGINX NGINX ! App Server MySQL Redis MongoDB Elastic Search
11. What can go bad?
12. All these moving parts • Logging • Authentication • Routing • Configuration • Supervisors • Rewrite Rules • Environment Variables • App settings • Firewalls • Package versions • Replication • Failover All of this can (and will) go wrong
13. Hope is not a valid testing strategy
14. Writing E2E tests • We want our E2E tests to hit all the moving parts • Test “happy path” — these tests are slow • Test and validate business logic • Don’t reach inside the box
15. Most basic test #!/bin/sh ! RET=$(curl -s -o /dev/null -L \ -w "%{http_code}" https://myapi.com);
 if [ $RET -ne 200 ]; then echo "FAILED"; else echo "PASSED"; fi
16. Non-determinism in testing
17. –Martin Fowler “A test is non-deterministic when it passes sometimes and fails sometimes, without any noticeable change in the code, tests, or environment. Such tests fail, then you re-run them and they pass. Test failures for such tests are seemingly random.”
18. Why are NDT a problem? • E2E test provide bug detection by acting as regression tests • E2E give confidence to make big changes with broad impact • When NDT fail, you don’t know if you screwed up, or if that’s just part of the test’s behaviour.
19. Services within services • When your API service calls a 3rd party service, you have introduced a large point of failure • 3rd party system might not have a test instance, might be unreliable, slow, or unstable • Your tests are dependent on infrastructure you can’t control in any way
20. Test Doubles • Represent a real service, with real data • Often a snapshot of data at a particular time • Can be hand crafted, in the case where a part of the service doesn’t exist in test mode
21. http://martinfowler.com/bliki/SelfInitializingFake.html
22. chameleon
23. Your very own Test Double • https://github.com/NickPresta/chameleon • Caching reverse proxy used for testing • Creates a “test double” for a given service • Written in Go • Well tested
24. Using chameleon $ chameleon -data ./http_bin_data
 -url https://httpbin.org/
 -host localhost:10005 $ curl http://localhost:10005/ip
 {
 "origin": "101.145.34.25"
 }
 $ ls ./http_bin_data
 spec.json
 262076ab58b2423e21e681e7b710312c
25. chameleon for your E2E 1. Set up chameleon to proxy external services that communicate with your application 2. Instruct your application to connect to chameleon’s host:port instead of the external service 3. Make a request to your application that will communicate with the external service 4. Let chameleon do its thing and continue as normal 5. Check in responses to your source control to have deterministic tests
26. Further Reading • http://martinfowler.com/articles/microservice-testing/ • http://martinfowler.com/articles/ nonDeterminism.html • http://martinfowler.com/bliki/ IntegrationContractTest.html • http://martinfowler.com/bliki/TestDouble.html • Patterns of Enterprise Application Architecture
27. Thanks! Nick Presta @NickPresta https://github.com/NickPresta/chameleon