Testing micro-services made easy @ PyCon 2018

Transcript

1. A S L Devi PyCon India, 2018 Testing micro-services made easy
2. $ whoami Devi Lead engineer and Architect at PowerToFly @asldevi https://www.linkedin.com/in/asldevi/
3. Agenda • What micro-services are • Goodness and challenges with micro-services • Why testing micro-services is hard • Tools and techniques to test them effectively • Demo
4. Micro-services … particular way of designing software applications as suites of independently deployable services. (Martin Fowler)
5. A simple e-commerce application with micro-service architecture Web app E-Commerce backend Inventory Shopping cart Product search User accounts User request Mobile app User request
6. • Independent choice of technology stacks • Independently scale, deploy and maintain • Faster development and deployment cycles • Agile workflows with smaller teams among many other things Goodness of Micro-services
7. • Managing and monitoring distributed systems • Tracing and debugging • Fault Tolerance • Testing Challenges
8. Challenges in testing micro-services • Many moving parts • Changes in request/response formats, query params and the functionality • Different users using different versions of the same service • Unit-tests ? End-to-end tests ?
9. Are unit-tests good enough ? Service A Service C Service B Change request ?
10. • Need ALL the dependencies up and running - VERY expensive • No Isolation of failure What about end-to-end tests ? ¯\_(ツ)_/¯ All our unit tests PASS
11. • Simplicity and ease of unit tests • Coverage and confidence that comes with end-to-end tests • Quickly identify the source of the failure What do we want ?
12. A bit of terminology Web app E-Commerce backend inventory user accounts product search shopping cart User request Mobile app User request Providers Consumers Provider & Consumer
13. • Contract ◦ each consumer specifies how it uses provider’s services ▪ Which endpoints I use ▪ How do I use ▪ What do I expect in return • each provider verifies contracts provided by all consumers part of its deployment process Consumer Driven Contracts
14. • A consumer-driven contract testing tool • For consumer ◦ Provides a mock service ◦ Provides DSL to write contract tests • For provider ◦ Ability to play back tests and verification • Has support in many languages - python, ruby, java, JS, go, node, andriod and more Pact
15. Pact-python • Active repo on github • pip install pact-python • Generates pacts with unittest, pytest, nose and nose2 on consumer side • Verify them with pact-verifier on provider side
16. How it works ? • consumers run “consumer tests” with backend • auto-generate “pact file” • JSON file • contains request-response pairs • share them with provider • provider verifies all pact files during its deployment process Backend app Mobile app Web app Expectations Expectations
17. $ cat src/provider.py from flask import Flask app = Flask(__name__) @app.route("/<name>") def make_greeting(name): return "Hello {}".format(name) if __name__ == '__main__': app.run(host='0.0.0.0', port=1234) Demo - Provider
18. $ cat src/consumer.py import requests def get_greeting(name): url = 'http://localhost:1234/{}'.format(name) return requests.get(url) Demo: Consumer
19. $ cat src/test_consumer.py import atexit from consumer import get_greeting from pact import Consumer, Provider pact = Consumer('web_app').has_pact_with(Provider('backend_app')) pact.start_service() # starts Pact mock service will be localhost:1234 atexit.register(pact.stop_service) def test_consumer(): name = 'PyCon' expected = 'Hello {}'.format(name) pact.given( 'a name' ).upon_receiving( 'a request to provider with that name' ).with_request( 'get', '/{}'.format(name) ).will_respond_with(200, body=expected) with pact: result = get_greeting(name) assert result.status_code == 200 assert result.text == expected
20. $ pytest src ======= test session starts ==================================================================== ... collected 1 item src/test_consumer.py . $ cat web_app-backend_app.json { "consumer": {"name": "web_app"}, "provider": {"name": "backend_app"}, "interactions": [{ "description": "a request to provider with that name", "providerState": "a name", "request": { "method": "get", "path": "/PyCon" }, "response": { "status": 200, "headers": {}, "body": "Hello PyCon" } }], ... }
21. Demo: Provider side $ pact-verifier --provider-base-url=http://localhost:1234 --pact-url=web_app-backend_app.json INFO: Reading pact at web_app-backend_app.json Verifying a pact between web_app and backend_app Given a name a request to with that name with GET /PyCon returns a response which has status code 200 has a matching body 1 interaction, 0 failures
22. • Provider states • Variable content in response • Pact Broker More with Pact
23. References • https://docs.pact.io/ • https://github.com/pact-foundation/pact-python • https://martinfowler.com/articles/consumerDrivenContracts.html • https://github.com/pact-foundation/pact_broker
24. Thank you! @asldevi
25. Q & A