Upgrade to Pro — share decks privately, control downloads, hide ads and more …

API Design: It's Not Rocket Surgery (ODI)

Dave Ingram
December 06, 2012

API Design: It's Not Rocket Surgery (ODI)

An updated version of the API talk, given at the Open Data Institute on Thursday 6th December 2012.

Dave Ingram

December 06, 2012
Tweet

More Decks by Dave Ingram

Other Decks in Programming

Transcript

  1. API design
    It’s Not Rocket Surgery
    Dave Ingram
    @dmi
    December 6, 2012

    View Slide

  2. Who am I?
    • Was developer/RM at GroupSpaces for 41/2 years
    • Now DevOps freelancer
    • Twitter: @dmi
    • Github: dingram
    • Way too many projects
    • Involved in open source, including Phabricator
    • Occasionally found at London Hackspace

    View Slide

  3. Why am I giving this talk?
    • I’ve built a number of APIs
    • Both for GroupSpaces and my own projects
    • Sadly most are not public (yet!)
    • I’m also a consumer of many other APIs
    • Twitter
    • Foursquare
    • Tumblr
    • TfL
    • Spotify
    • . . . blah blah blah. . .

    View Slide

  4. Feedback/thoughts
    via Twitter:
    #apirocketsurgery
    (or @dmi)

    View Slide

  5. What this isn’t

    View Slide

  6. T
    T
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    EST
    EST
    ST
    ST
    ST
    T
    T
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    R
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    R
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    R
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    RES
    RES
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    R
    R
    R
    RE
    RE
    RE
    RES
    RES
    REST
    R

    View Slide

  7. T
    T
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    EST
    EST
    ST
    ST
    ST
    T
    T
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    R
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    R
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    R
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    RES
    RES
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    R
    R
    R
    RE
    RE
    RE
    RES
    RES
    REST
    R

    View Slide

  8. T
    T
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    EST
    EST
    ST
    ST
    ST
    T
    T
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    R
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    R
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    R
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    RES
    RES
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    REST
    R
    R
    R
    RE
    RE
    RE
    RES
    RES
    REST
    R

    View Slide

  9. There’s so much more!

    View Slide

  10. URLs

    View Slide

  11. URLs
    verbs

    View Slide

  12. URLs
    verbs
    headers

    View Slide

  13. URLs
    verbs
    headers
    authentication

    View Slide

  14. URLs
    verbs
    headers
    authentication
    formats

    View Slide

  15. URLs
    verbs
    headers
    authentication
    formats
    integrity

    View Slide

  16. URLs
    verbs
    headers
    authentication
    formats
    integrity
    documentation

    View Slide

  17. DOCUMENTATION

    View Slide

  18. DOCUMENTATION
    (later)

    View Slide

  19. Part 1: URLs

    View Slide

  20. Make them
    versioned

    View Slide

  21. Make them
    versioned,
    hackable

    View Slide

  22. Make them
    versioned,
    hackable &
    meaningful

    View Slide

  23. http://api.com/v1/

    View Slide

  24. http://api.com/v1/users/

    View Slide

  25. http://api.com/v1/users/31337

    View Slide

  26. http://api.com/v1/users/31337/posts/

    View Slide

  27. Look at URLs from a
    consumer perspective

    View Slide

  28. They don’t care about
    internal implementation
    details

    View Slide

  29. Use common sense

    View Slide

  30. Don’t be afraid of
    the query string

    View Slide

  31. Use query arguments to
    filter output or for (some)
    alternate representations

    View Slide

  32. Good: searching & verbosity
    Bad: output format control
    (XML vs JSON)

    View Slide

  33. Good:
    http://api.com/v1/search?q=foobar
    Bad:
    http://api.com/v1/search/foobar

    View Slide

  34. Good:
    http://api.com/v1/foo.json
    Bad:
    http://api.com/v1/foo?format=json

    View Slide

  35. Part 2: Verbs

    View Slide

  36. • GET = get()
    • PUT = setAll() / new Obj($id)
    • POST = new Obj() / doStuff() / set()
    • DELETE = delete()
    • HEAD ≈ getMetadata()
    • OPTIONS ≈ reflection/permissions

    View Slide

  37. Must at least handle GET/POST

    View Slide

  38. Can emulate PUT/DELETE

    View Slide

  39. For example:
    POST http://api.com/v1/foo/42!PUT
    POST http://api.com/v1/foo/11!DELETE

    View Slide

  40. Remember that POST can
    have side-effects and is
    never cached

    View Slide

  41. PUT and DELETE are
    idempotent, which means
    they have the same effect
    even if they’re repeated

    View Slide

  42. HEAD is rare and
    can probably be ignored

    View Slide

  43. OPTIONS is used by CORS,
    but otherwise rare

    View Slide

  44. CORS?

    View Slide

  45. Cross-Origin Resource Sharing

    View Slide

  46. A way to allow in-browser
    cross-origin XMLHTTPRequests
    Support: FF3.5+, Chrome 4+, Safari 4+, Opera 12+,
    IE8+ (partial), IE10+ (full), iOS 3.2+, Android 2.1+
    http://www.w3.org/TR/cors/
    http://caniuse.com/cors

    View Slide

  47. Origin & Allow-Origin
    A way to allow in-browser
    cross-origin XMLHTTPRequests
    Support: FF3.5+, Chrome 4+, Safari 4+, Opera 12+,
    IE8+ (partial), IE10+ (full), iOS 3.2+, Android 2.1+
    http://www.w3.org/TR/cors/
    http://caniuse.com/cors

    View Slide

  48. Part 3: Headers

    View Slide

  49. Headers are important too!
    (and you can be clever)

    View Slide

  50. Accept
    The MIME types the client will accept.
    No need to use file extensions to decide
    what content type to serve!
    Accept-Language
    The languages the client will accept.
    No need to ask clients or (worse) just
    assume English responses.

    View Slide

  51. Some headers reduce traffic
    (important for mobile)

    View Slide

  52. • ETag – A unique tag for the content
    • If-(None-)Match – Check ETag
    • If-(Un)Modified-Since – Is it newer?
    • Cache-Control – Can it be cached?
    • Expires – How long is it valid?
    • Vary – Additional caching rules

    View Slide

  53. Beware: some proxies may
    not pass custom headers

    View Slide

  54. Beware: even some standard
    headers are not safe

    View Slide

  55. Prefer headers, but accept
    other methods

    View Slide

  56. Part 4: Authentication &
    Authorization

    View Slide

  57. OAuth2 over HTTPS

    View Slide

  58. Much simpler than OAuth1
    (similar to session cookies)

    View Slide

  59. Many libraries for OAuth2
    for many platforms as
    it’s a popular standard

    View Slide

  60. For extra security
    use OAuth2-MAC

    View Slide

  61. OAuth2-MAC uses signatures
    instead of bearer tokens,
    so secrets stay secret

    View Slide

  62. Then again, the author of OAuth2 has now
    advised not upgrading from OAuth 1.0a
    and either using OAuth 1.0a for new sites
    or staying close to a large provider’s
    implementation

    View Slide

  63. No need to worry about
    rate-limiting (to start with*)
    3scale, Apiaxle, Mashery can help

    View Slide

  64. Part 5: Formats

    View Slide

  65. Sane default: JSON, plus
    envelope with metadata

    View Slide

  66. {
    "meta": {
    "code": 2 ,
    "dev_notes ": [
    "This endpoint is deprecated"
    ]
    },
    "response ": {
    ...
    }
    }

    View Slide

  67. Metadata envelope helps
    with JSONP and limited
    clients that discard errors

    View Slide

  68. Having metadata allows
    out-of-band information, like
    deprecation warnings

    View Slide

  69. What about XML?

    View Slide

  70. *Hypertext as the Engine of Application State
    HATEOAS* is a nice idea,
    although it’s very verbose
    (but useful for building an API explorer)

    View Slide

  71. HATEOAS adds links to related
    endpoints in the response

    View Slide

  72. At the very most,
    HATEOAS should be opt-in
    for clients that will use it

    View Slide

  73. Don’t forget: mobile
    bandwidth is still very limited

    View Slide

  74. Timestamps
    • ISO-8601 2 12- 5- 3T19: : Z
    Human-readable, but needs parsing
    • UTC seconds since epoch: 1336 716
    Easily machine-usable

    View Slide

  75. Should your API really be
    human-readable?
    It’s better to help your
    consumers.

    View Slide

  76. Part 6: Data integrity

    View Slide

  77. Allow your consumers
    to cache data
    whenever possible

    View Slide

  78. Encourage use of request headers:
    • GET:
    • If-Modified-Since
    • If-None-Match
    • POST/PUT/DELETE:
    • If-Unmodified-Since
    • If-Match

    View Slide

  79. Return useful response headers:
    • Last-Modified
    • Expires
    • ETag
    • Cache-Control

    View Slide

  80. Deal with preconditions and
    give correct response codes

    View Slide

  81. Useful status codes:
    4 5 Method Not Allowed
    4 6 Not Acceptable
    412 Precondition Failed
    428 Precondition Required*
    429 Too Many Requests*
    *New in RFC6585

    View Slide

  82. For example:
    ETag and Last-Modified
    can help prevent
    race conditions

    View Slide

  83. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1
    Host: api.com
    Content -Type: text/html
    ...
    428 Precondition Required
    ETag: "x-rev -11294"
    Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT
    ...

    View Slide

  84. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1
    Host: api.com
    Content -Type: text/html
    ...
    428 Precondition Required
    ETag: "x-rev -11294"
    Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT
    ...

    View Slide

  85. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1
    Host: api.com
    Content -Type: text/html
    ...
    428 Precondition Required
    ETag: "x-rev -11294"
    Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT
    ...

    View Slide

  86. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1
    Host: api.com
    Content -Type: text/html
    ...
    428 Precondition Required
    ETag: "x-rev -11294"
    Last -Modified: Sat , 18 Feb 2 12 11: 9:21 GMT
    ...

    View Slide

  87. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1
    Host: api.com
    If -Unmodified -Since: Sat , 18 Feb 2 12 11: 9:21 GMT
    If -Match: "x-rev -11294"
    Content -Type: text/html
    ...
    412 Precondition Failed
    ETag: "x-rev -11467"
    Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT
    ...

    View Slide

  88. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1
    Host: api.com
    If -Unmodified -Since: Sat , 18 Feb 2 12 11: 9:21 GMT
    If -Match: "x-rev -11294"
    Content -Type: text/html
    ...
    412 Precondition Failed
    ETag: "x-rev -11467"
    Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT
    ...

    View Slide

  89. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1
    Host: api.com
    If -Unmodified -Since: Sat , 18 Feb 2 12 11: 9:21 GMT
    If -Match: "x-rev -11294"
    Content -Type: text/html
    ...
    412 Precondition Failed
    ETag: "x-rev -11467"
    Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT
    ...

    View Slide

  90. PUT /v1/wiki/dealing -with -conflicts HTTP /1.1
    Host: api.com
    If -Unmodified -Since: Sat , 18 Feb 2 12 11: 9:21 GMT
    If -Match: "x-rev -11294"
    Content -Type: text/html
    ...
    412 Precondition Failed
    ETag: "x-rev -11467"
    Last -Modified: Sat , 25 Feb 2 12 14:42:53 GMT
    ...

    View Slide

  91. Part 7: Documentation

    View Slide

  92. Documentation is important

    View Slide

  93. If people can’t understand
    your API, they won’t use it

    View Slide

  94. Use examples liberally. . .
    and make sure they’re both
    up-to-date and correct!

    View Slide

  95. Assume nothing;
    explain everything

    View Slide

  96. Try getting people you know
    to build something using
    only your own API docs

    View Slide

  97. Be helpful: provide an
    interactive console
    e.g. Apiary, Apigee console, Mashery’s iodocs

    View Slide

  98. And finally: Statistics

    View Slide

  99. Record numbers for anything
    and everything you can
    imagine

    View Slide

  100. Everybody loves graphs
    (and they’re important for understanding
    API usage and performance)

    View Slide

  101. Things to measure
    Counters:
    HTTP status (200, 400, 500, . . . )
    Tokens issued
    Token refreshes
    Clients created
    Client usage
    Auth successes
    Auth failures
    Auth grants
    Auth rejections
    DB queries
    API version/release
    Client type
    . . .
    Avg response time:
    Auth requests
    General requests
    Specific subsystems
    . . .
    Other
    Memory usage
    Client location
    . . .

    View Slide

  102. Thanks!
    Any questions?
    Feedback via Twitter:
    #apirocketsurgery or @dmi
    Slides: http://www.dmi.me.uk/talks/
    Built in L
    ATEX Inspired by: http://goo.gl/ mT55

    View Slide