TDD for APIs in a Microservice World (extended Version) by Michael Kuehne-Schlinkert @ MuCon 2016 in London

It can be tough to test an apparently simple service comprehensively. A microservice architecture brings a new level of complexity to the question “How can we validate that our API is working as intended?”

In this talk Michael will explain how to use test driven development for APIs and even further how TDD can drive an API Design towards a more usable design, and how to build an well-tested ecosystem of microservices. This approach is applicable for different kinds of services (REST APIs, websockets, industrial protocols). Independent from the type of interface we always ran into similar problems when we build an ecosystem of services. We have to deal with dependency, asynchronous behaviours, fallback mechanisms, endpoint versioning and sometimes even shared databases. It's not trivial to apply TDD to these kinds of problems cause you have to think of scenarios. But there are ways of identify these scenarios and to test them.

As an API specialist Michael worked with various clients designing, building, testing, maintaining and even redesigning private and public services. Based on his project experience he developed a practical approach to apply TDD to APIs in microservice ecosystems.

Transcript

1. @michikuehne TDD for APIs in a Microservice World Michael Kuehne-Schlinkert Head of Development @ Cybus
2. @michikuehne Cybus brings untapped Data Sources to a secure API Access Control Pre-Processing & API Shopfloor Protocols Security Monitoring Maintenance Efficiency Sovereignty
3. @michikuehne Our Challenges   Security, Reliability & Scalability è Service Level Agreements Better Representation of Customer Needs through Tests & Codes Loosly coupled but highly connected Microservices Efficient TDD with limited resources
4. @michikuehne „How to validate that our API is working as intended?“ by Steve Klabnik
5. @michikuehne „How to efficiently validate that our microservice ecosystem is working as intended?“ by Michael Kuehne-Schlinkert
6. @michikuehne Simplified Ecosystem Device Service Data {...} Data {...} Data {...} UI Service Auth Service
7. @michikuehne Where to start? Story   As a <role>, __.I want <feature> __.so that <reason>. Example __.As an admin of the cybus middleware __.I want to control the read access to my devices __.so that users can only read the data they are allowed to Scenario   GIVEN <context> __.WHEN <event> __.THEN <outcome> Example __. GIVEN External Device __. GIVEN Device Service __. GIVEN Auth Service __. WHEN External Device provides new data __. WHEN Read Access to Device is granted __. THEN Device Service reads data from external device
8. @michikuehne Non-Success Scenarios Example __. GIVEN External Device __. GIVEN Device Service __. GIVEN Auth Service __. WHEN External Device provides new data __. WHEN Read Access to Device is not granted __. THEN Device Service fails to read data from external device Example __. GIVEN External Device __. GIVEN Device Service __. NOT GIVEN Auth Service __. WHEN External Device provides new data __. THEN ..... Error Scernarios Failover Scenarios
9. @michikuehne Test Scenario Device Service Data {...} Data {...} Data {...} UI Service Auth Service External Device Data {...} Stubbed
10. @michikuehne Acceptance Tests Platform Tests   Turn Scenarios into Tests   Focus on whole Microservice Ecosystem Verify Functionality of Ecosystem External / Third-Party Services are stubbed/mocked   Black Box Testing   Entry Point of Platform Test depends on Scenario   API   GUI Contract Tests   Turn API Specifications into Tests   Focus on only One Mircroservice Verify the Interface (General Contract)   Black Box Testing
11. @michikuehne Each Consumer forms a Contract Device Service Contract PUT /device/ { mapping: ... } UI Service Auth Service Contract GET /devices/ [ { mapping: ... } ]
12. @michikuehne Our Solution: One Contract for All Device Service Contract PUT /device/ GET /devices/ UI Service Auth Service
13. @michikuehne Integration Tests A.k.a. Component Tests & Service Tests   Focus on only One Microservice Verify Functionality of One Service   Break down Scenarios   End-to-End Tests – but not to end of ecosystem   Internal Services are stubbed/mocked   Gray Box Testing (Black & White) Resources Domain ORM External Client Stubbed Service Ext. DB Stb. Test Client Simplified Microservice Architecture
14. @michikuehne Functional Tests Controller Industrial Protocol Device Handler   Focus on 2+ Classes/Modules of Domain Verify Interaction between Modules   Internal Dependencies are stubbed/mocked   White Box Testing Example Domain of Microservice Stubbed
15. @michikuehne Unit Tests Controller Industrial Protocol Device Handler   Focus on One Unit (Class/Module) of Domain Verify Functionality of One Unit   Internal Dependencies are stubbed/mocked   White Box Testing Example Domain of Microservice Stubbed
16. @michikuehne Our TDD Cycle Acceptance Tests Integration Tests Functional Tests Unit Tests Code Black Box Gray Box White Box
17. @michikuehne Our TDD Cycle with aspired Test Coverage Acceptance Tests Integration Tests Functional Tests Unit Tests Code tcU à 100% tcI à 100% tcF à 100% - tcI tcI + tcF à 100% tcA à x% tc = test coverage x –
18. @michikuehne Testing Pyramid Acceptance Integration Unit Functional (optional)
19. @michikuehne Lessons Learned It works! We overcame Testing Hell Less Test Code and less Fragile Tests but high test coverage   Quality of Story Definitions increased with mandatory Scenarios Efficency comes with the right Tools Each Microservice must provide Test Library (e.g. Stubs) Deterministic Environments gained Complexity Asynchronous Services require Gray to White Box Testing Functional Tests became our Plan B Don‘t forget the Human Factory
20. @michikuehne Internal Adoption It‘s a process! Be patient & encourage   Support from Management   Trainings & Pair Programming on real World Problems One TDD Evengalist per Team   Start with Contract Tests Awesome Test Libraries are critical Measure progess – Make it visible
21. @michikuehne cybus.io @cybus_io hello@cybus.io Pssst! We‘re hiring! Michael Kuehne-Schlinkert @michikuehne kuehne@cybus.io Thx! ...and happy testing! TDD gives you more time for fun stuff
22. @michikuehne Appendix   Icons by Gregor Cresnar from www.flaticon.com is licensed by CC 3.0 BY   Icons by Freepik from www.flaticon.com is licensed by CC 3.0 BY