API Design: It's Not Rocket Surgery (ODI)

15c49bd9f73317bf66952b9ee17414ef?s=47 Dave Ingram
December 06, 2012

API Design: It's Not Rocket Surgery (ODI)

An updated version of the API talk, given at the Open Data Institute on Thursday 6th December 2012.

15c49bd9f73317bf66952b9ee17414ef?s=128

Dave Ingram

December 06, 2012
Tweet

Transcript

  1. API design It’s Not Rocket Surgery Dave Ingram @dmi December

    6, 2012
  2. Who am I? • Was developer/RM at GroupSpaces for 41/2

    years • Now DevOps freelancer • Twitter: @dmi • Github: dingram • Way too many projects • Involved in open source, including Phabricator • Occasionally found at London Hackspace
  3. Why am I giving this talk? • I’ve built a

    number of APIs • Both for GroupSpaces and my own projects • Sadly most are not public (yet!) • I’m also a consumer of many other APIs • Twitter • Foursquare • Tumblr • TfL • Spotify • . . . blah blah blah. . .
  4. Feedback/thoughts via Twitter: #apirocketsurgery (or @dmi)

  5. What this isn’t

  6. T T REST REST REST REST REST REST REST EST

    EST ST ST ST T T REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST RES RES REST REST REST REST REST REST REST REST REST REST REST REST REST R R R RE RE RE RES RES REST R
  7. T T REST REST REST REST REST REST REST EST

    EST ST ST ST T T REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST RES RES REST REST REST REST REST REST REST REST REST REST REST REST REST R R R RE RE RE RES RES REST R
  8. T T REST REST REST REST REST REST REST EST

    EST ST ST ST T T REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST RES RES REST REST REST REST REST REST REST REST REST REST REST REST REST R R R RE RE RE RES RES REST R
  9. There’s so much more!

  10. URLs

  11. URLs verbs

  12. URLs verbs headers

  13. URLs verbs headers authentication

  14. URLs verbs headers authentication formats

  15. URLs verbs headers authentication formats integrity

  16. URLs verbs headers authentication formats integrity documentation

  17. DOCUMENTATION

  18. DOCUMENTATION (later)

  19. Part 1: URLs

  20. Make them versioned

  21. Make them versioned, hackable

  22. Make them versioned, hackable & meaningful

  23. http://api.com/v1/

  24. http://api.com/v1/users/

  25. http://api.com/v1/users/31337

  26. http://api.com/v1/users/31337/posts/

  27. Look at URLs from a consumer perspective

  28. They don’t care about internal implementation details

  29. Use common sense

  30. Don’t be afraid of the query string

  31. Use query arguments to filter output or for (some) alternate

    representations
  32. Good: searching & verbosity Bad: output format control (XML vs

    JSON)
  33. Good: http://api.com/v1/search?q=foobar Bad: http://api.com/v1/search/foobar

  34. Good: http://api.com/v1/foo.json Bad: http://api.com/v1/foo?format=json

  35. Part 2: Verbs

  36. • GET = get() • PUT = setAll() / new

    Obj($id) • POST = new Obj() / doStuff() / set() • DELETE = delete() • HEAD ≈ getMetadata() • OPTIONS ≈ reflection/permissions
  37. Must at least handle GET/POST

  38. Can emulate PUT/DELETE

  39. For example: POST http://api.com/v1/foo/42!PUT POST http://api.com/v1/foo/11!DELETE

  40. Remember that POST can have side-effects and is never cached

  41. PUT and DELETE are idempotent, which means they have the

    same effect even if they’re repeated
  42. HEAD is rare and can probably be ignored

  43. OPTIONS is used by CORS, but otherwise rare

  44. CORS?

  45. Cross-Origin Resource Sharing

  46. A way to allow in-browser cross-origin XMLHTTPRequests Support: FF3.5+, Chrome

    4+, Safari 4+, Opera 12+, IE8+ (partial), IE10+ (full), iOS 3.2+, Android 2.1+ http://www.w3.org/TR/cors/ http://caniuse.com/cors
  47. Origin & Allow-Origin A way to allow in-browser cross-origin XMLHTTPRequests

    Support: FF3.5+, Chrome 4+, Safari 4+, Opera 12+, IE8+ (partial), IE10+ (full), iOS 3.2+, Android 2.1+ http://www.w3.org/TR/cors/ http://caniuse.com/cors
  48. Part 3: Headers

  49. Headers are important too! (and you can be clever)

  50. Accept The MIME types the client will accept. No need

    to use file extensions to decide what content type to serve! Accept-Language The languages the client will accept. No need to ask clients or (worse) just assume English responses.
  51. Some headers reduce traffic (important for mobile)

  52. • ETag – A unique tag for the content •

    If-(None-)Match – Check ETag • If-(Un)Modified-Since – Is it newer? • Cache-Control – Can it be cached? • Expires – How long is it valid? • Vary – Additional caching rules
  53. Beware: some proxies may not pass custom headers

  54. Beware: even some standard headers are not safe

  55. Prefer headers, but accept other methods

  56. Part 4: Authentication & Authorization

  57. OAuth2 over HTTPS

  58. Much simpler than OAuth1 (similar to session cookies)

  59. Many libraries for OAuth2 for many platforms as it’s a

    popular standard
  60. For extra security use OAuth2-MAC

  61. OAuth2-MAC uses signatures instead of bearer tokens, so secrets stay

    secret
  62. Then again, the author of OAuth2 has now advised not

    upgrading from OAuth 1.0a and either using OAuth 1.0a for new sites or staying close to a large provider’s implementation
  63. No need to worry about rate-limiting (to start with*) 3scale,

    Apiaxle, Mashery can help
  64. Part 5: Formats

  65. Sane default: JSON, plus envelope with metadata

  66. { "meta": { "code": 2 , "dev_notes ": [ "This

    endpoint is deprecated" ] }, "response ": { ... } }
  67. Metadata envelope helps with JSONP and limited clients that discard

    errors
  68. Having metadata allows out-of-band information, like deprecation warnings

  69. What about XML?

  70. *Hypertext as the Engine of Application State HATEOAS* is a

    nice idea, although it’s very verbose (but useful for building an API explorer)
  71. HATEOAS adds links to related endpoints in the response

  72. At the very most, HATEOAS should be opt-in for clients

    that will use it
  73. Don’t forget: mobile bandwidth is still very limited

  74. Timestamps • ISO-8601 2 12- 5- 3T19: : Z Human-readable,

    but needs parsing • UTC seconds since epoch: 1336 716 Easily machine-usable
  75. Should your API really be human-readable? It’s better to help

    your consumers.
  76. Part 6: Data integrity

  77. Allow your consumers to cache data whenever possible

  78. Encourage use of request headers: • GET: • If-Modified-Since •

    If-None-Match • POST/PUT/DELETE: • If-Unmodified-Since • If-Match
  79. Return useful response headers: • Last-Modified • Expires • ETag

    • Cache-Control
  80. Deal with preconditions and give correct response codes

  81. Useful status codes: 4 5 Method Not Allowed 4 6

    Not Acceptable 412 Precondition Failed 428 Precondition Required* 429 Too Many Requests* *New in RFC6585
  82. For example: ETag and Last-Modified can help prevent race conditions

  83. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com Content -Type:

    text/html ... 428 Precondition Required ETag: "x-rev -11294" Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT ...
  84. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com Content -Type:

    text/html ... 428 Precondition Required ETag: "x-rev -11294" Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT ...
  85. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com Content -Type:

    text/html ... 428 Precondition Required ETag: "x-rev -11294" Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT ...
  86. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com Content -Type:

    text/html ... 428 Precondition Required ETag: "x-rev -11294" Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT ...
  87. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com If -Unmodified

    -Since: Sat , 18 Feb 2 12 11: 9:21 GMT If -Match: "x-rev -11294" Content -Type: text/html ... 412 Precondition Failed ETag: "x-rev -11467" Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT ...
  88. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com If -Unmodified

    -Since: Sat , 18 Feb 2 12 11: 9:21 GMT If -Match: "x-rev -11294" Content -Type: text/html ... 412 Precondition Failed ETag: "x-rev -11467" Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT ...
  89. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com If -Unmodified

    -Since: Sat , 18 Feb 2 12 11: 9:21 GMT If -Match: "x-rev -11294" Content -Type: text/html ... 412 Precondition Failed ETag: "x-rev -11467" Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT ...
  90. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com If -Unmodified

    -Since: Sat , 18 Feb 2 12 11: 9:21 GMT If -Match: "x-rev -11294" Content -Type: text/html ... 412 Precondition Failed ETag: "x-rev -11467" Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT ...
  91. Part 7: Documentation

  92. Documentation is important

  93. If people can’t understand your API, they won’t use it

  94. Use examples liberally. . . and make sure they’re both

    up-to-date and correct!
  95. Assume nothing; explain everything

  96. Try getting people you know to build something using only

    your own API docs
  97. Be helpful: provide an interactive console e.g. Apiary, Apigee console,

    Mashery’s iodocs
  98. And finally: Statistics

  99. Record numbers for anything and everything you can imagine

  100. Everybody loves graphs (and they’re important for understanding API usage

    and performance)
  101. Things to measure Counters: HTTP status (200, 400, 500, .

    . . ) Tokens issued Token refreshes Clients created Client usage Auth successes Auth failures Auth grants Auth rejections DB queries API version/release Client type . . . Avg response time: Auth requests General requests Specific subsystems . . . Other Memory usage Client location . . .
  102. Thanks! Any questions? Feedback via Twitter: #apirocketsurgery or @dmi Slides:

    http://www.dmi.me.uk/talks/ Built in L ATEX Inspired by: http://goo.gl/ mT55