API Design: It's Not Rocket Surgery (PHPUK13)

15c49bd9f73317bf66952b9ee17414ef?s=47 Dave Ingram
February 22, 2013

API Design: It's Not Rocket Surgery (PHPUK13)

The talk I gave at PHPUK13 on Friday 22nd February in (a very packed!) Track 3.

15c49bd9f73317bf66952b9ee17414ef?s=128

Dave Ingram

February 22, 2013
Tweet

Transcript

  1. API design It’s Not Rocket Surgery Dave Ingram @dmi PHP

    UK Conference 22nd February 2013
  2. Who am I? • Senior operations engineer at DataSift •

    Was developer/RM at GroupSpaces for 41/2 years • Way too many projects • Involved in open source, including Phabricator • Occasionally found at London Hackspace • Twitter: @dmi • Github: dingram
  3. Why am I giving this talk? • I’ve built a

    number of APIs • Both for GroupSpaces and my own projects • Sadly most are not public (yet!) • I’m also a consumer of many other APIs • Twitter • Foursquare • Tumblr • TfL • Spotify • . . . blah blah blah. . . • I’m opinionated
  4. Feedback/thoughts via Twitter: #apirocketsurgery (or @dmi) http://joind.in/8 45

  5. What this isn’t

  6. T T REST REST REST REST REST REST REST EST

    EST ST ST ST T T REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST RES RES REST REST REST REST REST REST REST REST REST REST REST REST REST R R R RE RE RE RES RES REST R
  7. T T REST REST REST REST REST REST REST EST

    EST ST ST ST T T REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST RES RES REST REST REST REST REST REST REST REST REST REST REST REST REST R R R RE RE RE RES RES REST R
  8. T T REST REST REST REST REST REST REST EST

    EST ST ST ST T T REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST REST R REST REST REST REST REST REST REST REST REST REST REST REST REST REST RES RES REST REST REST REST REST REST REST REST REST REST REST REST REST R R R RE RE RE RES RES REST R
  9. There’s so much more!

  10. URLs

  11. URLs verbs

  12. URLs verbs headers

  13. URLs verbs headers authentication

  14. URLs verbs headers authentication formats

  15. URLs verbs headers authentication formats integrity

  16. URLs verbs headers authentication formats integrity documentation

  17. DOCUMENTATION

  18. DOCUMENTATION (later)

  19. Part 1: URLs

  20. Make them versioned

  21. Make them versioned, hackable

  22. Make them versioned, hackable & meaningful

  23. http://api.com/v1/

  24. http://api.com/v1/users/

  25. http://api.com/v1/users/31337

  26. http://api.com/v1/users/31337/posts/

  27. Look at URLs from a consumer perspective

  28. They don’t care about internal implementation details

  29. Use common sense

  30. Don’t be afraid of the query string

  31. Use query arguments to filter output or for (some) alternate

    representations
  32. Good: searching & verbosity Bad: output format control (XML vs

    JSON)
  33. Good: http://api.com/v1/search?q=foobar Bad: http://api.com/v1/search/foobar

  34. Good: http://api.com/v1/foo.json Bad: http://api.com/v1/foo?format=json

  35. Part 2: Verbs

  36. • GET = get() • PUT = setAll() / new

    Obj($id) • POST = new Obj() / doStuff() / set() • DELETE = delete() • HEAD ≈ getMetadata() • OPTIONS ≈ reflection/permissions
  37. Must at least handle GET/POST

  38. Can emulate PUT/DELETE

  39. For example: POST http://api.com/v1/foo/42!PUT POST http://api.com/v1/foo/11!DELETE

  40. Remember that POST can have side-effects and is never cached

  41. PUT and DELETE are idempotent, which means they have the

    same effect even if they’re repeated
  42. HEAD is rare and can probably be ignored

  43. OPTIONS is used by CORS, but otherwise rare

  44. CORS?

  45. Cross-Origin Resource Sharing

  46. A way to allow in-browser cross-origin XMLHTTPRequests Support: FF3.5+, Chrome

    4+, Safari 4+, Opera 12+, IE8+ (partial), IE10+ (full), iOS 3.2+, Android 2.1+ http://www.w3.org/TR/cors/ http://caniuse.com/cors
  47. Origin & Allow-Origin A way to allow in-browser cross-origin XMLHTTPRequests

    Support: FF3.5+, Chrome 4+, Safari 4+, Opera 12+, IE8+ (partial), IE10+ (full), iOS 3.2+, Android 2.1+ http://www.w3.org/TR/cors/ http://caniuse.com/cors
  48. Part 3: Headers

  49. Headers are important too! (and you can be clever)

  50. Accept The MIME types the client will accept. No need

    to use file extensions to decide what content type to serve!* Accept-Language The languages the client will accept. No need to ask clients or (worse) just assume English responses.
  51. Some headers reduce traffic (important for mobile)

  52. • ETag – A unique tag for the content •

    If-(None-)Match – Check ETag • If-(Un)Modified-Since – Is it newer? • Cache-Control – Can it be cached? • Expires – How long is it valid? • Vary – Additional caching rules
  53. Beware: some proxies may not pass custom headers

  54. Beware: even some standard headers are not safe

  55. Prefer headers, but accept other methods

  56. Part 4: Authentication & Authorization

  57. OAuth2 over HTTPS

  58. Much simpler than OAuth1 (and similar to session cookies)

  59. Many libraries for OAuth2 for many platforms as it’s a

    popular standard
  60. For extra security use OAuth2-MAC

  61. OAuth2-MAC uses signatures instead of bearer tokens, so secrets stay

    secret
  62. Then again, the author of OAuth2 has now advised not

    upgrading from OAuth 1.0a and either using OAuth 1.0a for new sites or staying close to a large provider’s implementation
  63. No need to worry about rate-limiting (to start with*) 3scale,

    Apiaxle, Mashery can help
  64. Part 5: Formats

  65. Sane default: JSON, plus envelope with metadata

  66. { "meta": { "code": 2 , "dev_notes ": [ "This

    endpoint is deprecated" ] }, "response ": { ... } }
  67. Metadata envelope helps with JSONP and limited clients that discard

    errors
  68. Having metadata allows out-of-band information, like deprecation warnings

  69. What about XML?

  70. *Hypertext as the Engine of Application State HATEOAS* is a

    nice idea, although it’s very verbose (but useful for building an API explorer)
  71. HATEOAS adds links to related endpoints in the response

  72. Don’t forget: mobile bandwidth is still very limited

  73. Timestamps • ISO-8601 2 12- 5- 3T19: : Z Human-readable,

    but needs parsing • UTC seconds since epoch: 1336 716 Easily machine-usable
  74. Should your API really be human-readable? It’s better to help

    your consumers.
  75. Part 6: Data integrity

  76. Allow your consumers to cache data whenever possible

  77. Encourage use of request headers: • GET: • If-Modified-Since •

    If-None-Match • POST/PUT/DELETE: • If-Unmodified-Since • If-Match
  78. Return useful response headers: • Last-Modified • Expires • ETag

    • Cache-Control
  79. Deal with preconditions and give correct response codes

  80. Useful status codes: 4 5 Method Not Allowed 4 6

    Not Acceptable 412 Precondition Failed 428 Precondition Required* 429 Too Many Requests* *New in RFC6585
  81. For example: ETag and Last-Modified can help prevent race conditions

  82. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com Content -Type:

    text/html ... 428 Precondition Required ETag: "x-rev -11294" Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT ...
  83. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com Content -Type:

    text/html ... 428 Precondition Required ETag: "x-rev -11294" Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT ...
  84. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com Content -Type:

    text/html ... 428 Precondition Required ETag: "x-rev -11294" Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT ...
  85. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com Content -Type:

    text/html ... 428 Precondition Required ETag: "x-rev -11294" Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT ...
  86. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com If -Unmodified

    -Since: Sat , 18 Feb 2 12 11: 9:21 GMT If -Match: "x-rev -11294" Content -Type: text/html ... 412 Precondition Failed ETag: "x-rev -11467" Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT ...
  87. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com If -Unmodified

    -Since: Sat , 18 Feb 2 12 11: 9:21 GMT If -Match: "x-rev -11294" Content -Type: text/html ... 412 Precondition Failed ETag: "x-rev -11467" Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT ...
  88. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com If -Unmodified

    -Since: Sat , 18 Feb 2 12 11: 9:21 GMT If -Match: "x-rev -11294" Content -Type: text/html ... 412 Precondition Failed ETag: "x-rev -11467" Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT ...
  89. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1 Host: api.com If -Unmodified

    -Since: Sat , 18 Feb 2 12 11: 9:21 GMT If -Match: "x-rev -11294" Content -Type: text/html ... 412 Precondition Failed ETag: "x-rev -11467" Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT ...
  90. Part 7: Documentation

  91. Documentation is important

  92. If people can’t understand your API, they won’t use it

  93. Use examples liberally. . . and make sure they’re both

    up-to-date and correct!
  94. Consider linking your examples to your automated tests (You do

    have them, right?)
  95. Assume nothing; explain everything (What is an ID? string[8]? int32?

    uint64?)
  96. Be helpful: provide an interactive console e.g. Apiary, Apigee console,

    Mashery’s iodocs
  97. And finally: Statistics

  98. Record numbers for anything and everything you can imagine

  99. Everybody loves graphs (and they’re important for understanding API usage

    and performance)
  100. Things to measure Counters: HTTP status (200, 400, 500, .

    . . ) Tokens issued Token refreshes Clients created Client usage Auth successes Auth failures Auth grants Auth rejections DB queries API version/release Client type . . . Avg response time: Auth requests General requests Specific subsystems . . . Other Memory usage Client location . . .
  101. Thanks! Any questions? Feedback via joind.in: http://joind.in/8 45 or #apirocketsurgery

    or @dmi Slides: http://www.dmi.me.uk/talks/ Built in L ATEX Inspired by: http://goo.gl/ mT55