Test your API docs!

Today many API docs are driven by API description formats, such as Swagger/OpenAPI, API Blueprint, and so on. I'm working on an Open Source tool (Dredd), which verifies whether the implementation is according to the description. This enables a whole new workflow to API designers - they can design before implementing, and then ensure the implementation is in sync with the design.

https://fosdem.org/2018/schedule/event/test_api_docs_with_dredd/

Transcript

1. Test your API docs! It's tested or it's broken Honza Javorek FOSDEM 2018
2. Honza honzajavorek.cz @honzajavorek
6. PRAGUE
7. Czech Republic
8. PYCON.CZ
10. Interface of libraries #!/usr/bin/env python import django
11. Interface of systems curl https://api.github.com/
12. Technical?
13. User interface!
14. import urllib2 gh_url = 'https://api.github.com' req = urllib2.Request(gh_url) password_manager = urllib2.HTTPPasswordMgrWithDefaultRealm() password_manager.add_password(None, gh_url, 'user', 'pass') auth_manager = urllib2.HTTPBasicAuthHandler(password_manager) opener = urllib2.build_opener(auth_manager) urllib2.install_opener(opener) handler = urllib2.urlopen(req) print handler.getcode() print handler.headers.getheader('content-type') Do you like this interface?
15. Requests: HTTP for Humans
16. >>> r = requests.get('https://api.github.com/user', ... auth=('user', 'pass')) >>> r.status_code 200 >>> r.headers['content-type'] 'application/json; charset=utf8' >>> r.encoding 'utf-8' >>> r.text '{"type":"User"...' >>> r.json() {'disk_usage': 368627, 'private_gists': 484, ...} Do you like this interface?
17. How do you design the interface?
18. Eating your own dog food
19. def test_basic_building(): req = requests.Request() req.url = 'http://kennethreitz.org/' req.data = {'life': '42'} pr = req.prepare() assert pr.url == req.url assert pr.body == 'life=42'
20. Writing tests first helps to design the interface TDD
21. test > RED > implement > test > GREEN req = requests.Request() req.url = 'http://kennethreitz.org/' req.data = {'life': '42'} pr = req.prepare() assert pr.url == req.url assert pr.body == 'life=42'
22. Writing down behavior first helps to design the interface BDD
23. Feature: Status code Background: Given you expect HTTP status code "200" Scenario: Different real response status When real status code is "500" Then Gavel will set some error for "status code" And Request or Response is NOT valid Scenario: Response status code match When real status code is "200" Then Gavel will NOT set any errors for "status code" And Request or Response is valid Gherkin / Cucumber
24. Testable documentation!
25. 1. Design 2. Test 3. Implement
26. 1. Think, agree, promise 2. Test the promise 3. Fulfill the promise
27. How did Kenneth design requests?
29. Before I start writing a single line of code, I write the README and fill it with usage examples. I pretend that the module I want to build is already written and available, and I write some code with it. “ https://www.kennethreitz.org/essays/how-i-develop-things-and-why
30. RDD
31. RDD Readme Driven Development http://tom.preston-werner.com/2010/08/23/readme-driven- development.html
32. # Requests The `requests` library allows you to perform HTTP requests from your Python code. ## Example ```python >>> r = requests.get('https://github.com') >>> r.status_code 200 ``` ## License MIT README.md
33. Readme Driven Development • chance to think through the project first • docs are ready - no need to write them retroactively • your team can use the interface before it exists • easy to discuss the interface with everyone
34. Interface in README = Essential interface user expects
35. README must not get out of sync with code
36. How do we ensure implementation matches the design?
37. python -m doctest README.md doctest
38. language: "python" python: - "3.6" script: - "python -m doctest README.md" Continuous Integration
39. 1. Think, agree, README 2. Test the README 3. Fulfill the README
40. What if we could design and test web APIs like this?
41. # Calendar API The API gives you various means to work with date and time. ## GET /now Provides you with current date and time. - Response 200 (application/json) ```json { "day": 29, "month": 2, "year": 2017, "hour": 11, "minute": 45, "second": 38 } ``` API.md
44. dredd API.md http://localhost:8000 Dredd
52. Go JavaScript Perl PHP Python Ruby Rust
53. language: "python" python: - "3.6" before_install: - "npm install -g dredd" script: - "dredd API.md http://localhost:8000" Continuous Integration
56. What about the OpenAPI Spec? (fka Swagger)
57. YAML
58. 1. Think, agree, API desc. 2. Test the API desc. 3. Fulfill the API desc.
59. Testing implementation against design allows you designing before implementing
60. Designing before implementing allows you better design
61. Dredd allows you better design
62. Remember • think first, design first, docs first, test first • discuss the the interface design before implementing • use the interface before implementing (mocks, tests) • have your interface design as a single source of truth • test implementation against the design
64. github.com/apiaryio/dredd @honzajavorek