The Good and the Bad of the OpenStack REST APIs - OpenStack Summit Vancouver 2015

As a relatively recent new user of OpenStack, I have mixed feelings about the current state of the OpenStack APIs. On one side, it is exciting to see how modular OpenStack is, how each module sports its own REST API, and how all of these APIs can interact and function together. But also, I can't help but notice that there are inconsistencies in the design of these various APIs, and that best practices associated with REST architecture are frequently ignored or violated. The inconsistencies among the different APIs concern me very much. To help improve the situation, I have been an active member of the API Working Group since its inception. In this talk I want to share my personal views and pre-OpenStack experiences on REST API design, and contrast that to the state of the OpenStack APIs. Drawing from the reviews and discussions in the API Working Group, I plan to show actual examples of what I perceive are the problem areas. I'll be joined by one of the founding members of the API Working Group to talk about what we're doing to improve the APIs and the overall developer experience for users building applications and tools on OpenStack. More specifically, the topics we'll cover are: APIs vs. Web APIs vs. REST APIs. What's the difference? The GoodStateless servers Modular APIs Authentication The BadInconsistencies: pagination, sorting, searching. metadata handling Actions vs. Resources Lack of hypermedia Client/Server coupling OpenStack clients vs. OpenStack APIs The API Working GroupPurpose Scope Guidelines I hope by the end of the session you will have another perspective to consider when building applications and tools on the OpenStack APIs.

Transcript

1. The Good and the Bad of the OpenStack APIs Miguel Grinberg | Everett Toews rack.to/good-bad-apis
2. 2 About @miguelgrinberg • Software Developer with Rackspace ◦ Rackspace Private Cloud (os-ansible-deployment on Stackforge) ◦ Member of the API Working Group ◦ Heat & Horizon contributor • Author & Speaker ◦ O’Reilly’s Flask Web Development book, videos, webcasts ◦ Flask & REST presentations at PyCon 2014, 2015 • Blogger ◦ Personal: http://blog.miguelgrinberg.com ◦ Rackspace Dev Blog: https://developer.rackspace.com/blog/ www.rackspace.com
3. 3 About @everett_toews • Developer Advocate with Rackspace ◦ Developer Experience ◦ Founder of the API Working Group ◦ Python OpenStack SDK contributor • Author & Speaker ◦ O’Reilly’s OpenStack Operations Guide ◦ Presentations at OpenStack Summits, SXSW, OSCON, Gluecon, etc. • Blogger ◦ Personal: http://blog.phymata.com ◦ Rackspace Dev Blog: https://developer.rackspace.com/blog/ www.rackspace.com
4. Overview www.rackspace.com 4
5. 5 100 Ways to OpenStack www.rackspace.com
6. 6 100 Ways to OpenStack www.rackspace.com
7. 7 100 Ways to OpenStack www.rackspace.com
8. 8 100 Ways to OpenStack www.rackspace.com REST APIs • Everything goes through the APIs • If it’s not in the APIs, it can’t be done!
9. 9 100 Ways to OpenStack www.rackspace.com
10. 10 100 Ways to OpenStack www.rackspace.com
11. 11 100 Ways to OpenStack www.rackspace.com
12. 12 100 Ways to OpenStack www.rackspace.com Python Client Libraries / SDK • Runs in all Python platforms • Python only
13. 13 100 Ways to OpenStack www.rackspace.com
14. 14 100 Ways to OpenStack www.rackspace.com Unofficial Client Libraries • Not as complete as Python clients • YMMV
15. 15 100 Ways to OpenStack www.rackspace.com
16. 16 100 Ways to OpenStack www.rackspace.com
17. 17 100 Ways to OpenStack www.rackspace.com Command Line Clients • Scripting friendly • Output needs to be parsed or scraped
18. 18 100 Ways to OpenStack www.rackspace.com
19. 19 100 Ways to OpenStack www.rackspace.com Horizon Dashboard • Good for quick interactive tasks • No automation
20. 20 What Is A REST API? • REST: A software architecture style for creating highly scalable services. • Client-Server interaction is based on HTTP requests sent to resource URLs. • For a REST overview, watch my PyCon 2015 talk “Is Your REST API RESTful?” on YouTube. www.rackspace.com
21. 21 The Good www.rackspace.com
22. 22 Not Good… Great! Universal access All you need is an HTTP client! www.rackspace.com
23. 23 Modular APIs • Services are classified by type (identity, compute, image, network, etc.) • They can be combined like LEGO bricks www.rackspace.com
24. 24 The Catalog • Lists all the services • For each service its endpoints are defined • Awesome concept... Hypermedia! www.rackspace.com
25. 25 Authentication Flow • Connect to the authentication URL ◦ Send username, password and project ◦ Receive token and catalog • Connect to APIs from the catalog ◦ Pass token in X-Auth-Token header www.rackspace.com
26. The Bad www.rackspace.com 26
27. 27 Versioning Mess • Which version should I use? ◦ v2? v3? v2.1??? • APIs should be designed to be extensible • Versioning should be a last resort measure, reserved only for a major redesign www.rackspace.com
28. 28 Bypassing the Catalog • If a service is not in the catalog, then it should not be used • Example from heat: www.rackspace.com
29. 29 Client-Server Coupling • Same team works on the server, API, client library and command line client • APIs are mostly tested indirectly through the Python client libraries www.rackspace.com
30. 30 Lack of Hypermedia • Clients are forced to hardcode URLs • Examples from glance and nova clients: www.rackspace.com
31. 31 RPC Style Over REST • APIs abuse the concept of “actions” ◦ Glance: ▪ /v2/images/{image_id}/actions/deactivate ▪ /v2/images/{image_id}/actions/reactivate ◦ Heat: ▪ /v1/{tenant_id}/stacks/preview ▪ /v1/{tenant_id}/validate www.rackspace.com
32. 32 No Direct Browser Access • No CORS support (Swift is the exception) • Workarounds: ◦ Set up third party WSGI middleware for CORS ◦ Use an API proxy www.rackspace.com
33. 33 Inconsistencies I • Pagination ◦ Works only for select resource collections ◦ Nova, glance and cinder use limit and marker ◦ Swift uses limit, marker and end_marker ◦ Keystone uses page and per_page ◦ Neutron does not paginate at all ◦ No links for next and previous pages ◦ No counts in most cases www.rackspace.com
34. 34 Inconsistencies II • Filtering ◦ Most filters check for equality, very few support pattern searches ◦ No way to check for ranges or multiple values ◦ Lots of resource collections do not support it • Sorting ◦ Support across OpenStack is almost non-existant www.rackspace.com
35. 35 Inconsistencies III • Metadata ◦ Nova and glance use a /metadata appended to the resource URL ◦ Swift uses request and response headers, on the actual resource URL www.rackspace.com
36. 36 Inconsistencies IV • HTTP Response status codes ◦ Asynchronous requests: 200 (bad) vs. 202 (good) ◦ Incorrect request method: 404 (bad) vs. 405 (good) ◦ Resource creation: 200 (bad) vs. 201 (good) www.rackspace.com
37. 37 The Solution www.rackspace.com
38. The API Working Group “To improve the developer experience of API users by converging the OpenStack API to a consistent and pragmatic RESTful design. The working group creates guidelines that all OpenStack projects should follow for new development, and promotes convergence of new APIs and future versions of existing APIs.” 38 www.rackspace.com
39. 39 Deliverables • Analysis • Guidelines • Reviews ◦ APIImpact • Collaboration • Bugs? www.rackspace.com
40. 40 Traction www.rackspace.com • “I like this, many thanks.” ◦ John Garbutt, Nova PTL [1] • “I also seek feedback from the API-wg” ◦ Salvatore Orlando, Neutron Core [2] • “dovetails with developer focused API errors” ◦ Rochelle Grober, Logging WG [3] • “Everett, Thanks! Just joined!” ◦ Alex Xu, Nova [4] [5]
41. 41 How To Help • The API Working Group • Meet with us • Chat with us • State of the API Working Group ◦ 3:40pm - 4:20pm • Room 301 ◦ rack.to/api-wg www.rackspace.com
42. Thank You! @miguelgrinberg @everett_toews rack.to/good-bad-apis Did we mention we’re hiring? bit.ly/RackerTalent