API Standards 2.0

We're all familiar with things like HTTP codes and content types, but there's so much more we can do when developing an API to make life easier for consumers. How many times have you used an API only to find out that every endpoint is slightly different – some use `snake_case`, others `camelCase`, sometimes the field is called `id`, sometimes it’s `user_id`. How about pagination? Error responses? What about API documentation? Trying to standardise on all these things can kill an engineering team. There are so many options out there it’s difficult to know where to start. Come along and learn what works for our team! We’ll cover contentious topics (should the version be in the URL or a header?), lesser-known standards that are great (RFC 7807 springs to mind) and a couple of things that aren't an issue right up until they’re a really big issue (like pagination).

Transcript

1. API STANDARDS 2.0
   

   View Slide

2. 1.
   API Deprecation
   Before you build it, know how you're going to turn it off
   2
   

   View Slide

3. API Deprecation
   3
   Beta
   Versions
   

   View Slide

4. API Deprecation
   4
   Major
   Versions
   Beta
   Versions
   

   View Slide

5. API Deprecation
   5
   Major
   Versions
   Beta
   Versions
   API

   Retirement
   

   View Slide

6. Have a deprecation policy. It
   doesn't matter what it is, but be
   consistent.
   Deprecation Policies
   6
   

   View Slide

7. When the decision has been taken to deprecate an
   API:
   • For beta products the deprecation period must be
   at least 30 days (60 days recommended)
   • For GA products the deprecation period must be at
   least 1 year
   • Warning emails will be sent to the API at regular
   intervals before the deprecation time
   • A guide will be supplied to customers explaining
   how to migrate to the replacement API with the
   initial deprecation notice.
   Deprecation Policies: Nexmo
   7
   

   View Slide

8. Use the Sunset Header
   Sunset: Sat, 31 Dec 2018 23:59:59
   GMT
   Sunset Header
   8
   https://tools.ietf.org/html/draft-wilde-sunset-header-03
   

   View Slide

9. $ composer require hskrasek/guzzle-sunset
   ---
   $stack = new \GuzzleHttp\HandlerStack(\GuzzleHttp\choose_handler());
   $stack->push(new \HSkrasek\Sunset\SunsetMiddleware($logger));
   $client = new \GuzzleHttp\Client(['handler' => $stack]);
   API Deprecation: Sunset Header
   9
   

   View Slide

10. 2.
    Versioning
    Header/URL, Versioning Schemes, Deprecation
    10
    

    View Slide

11. “
    The reason to make a real REST API
    is to get evolvability … a "v1" is a
    middle finger to your API customers,
    indicating RPC/HTTP (not REST)
    https://twitter.com/fielding/status/376835835670167552
    

    View Slide

12. “With a sufficient number of users of an
    interface, it doesn’t matter what you
    promised in the interface contracts, all
    observable behaviors of your class or
    function or whatnot will be depended
    upon by somebody.
    Hyrum's Law
    

    View Slide

13. Accept: application/vnd.nexmo+json;
    version=3
    URI vs Header
    13
    

    View Slide

14. Accept: application/vnd.nexmo+json;
    version=3
    

    X-Nexmo-Version: 3
    URI vs Header
    14
    

    View Slide

15. Accept: application/vnd.nexmo+json;
    version=3
    

    X-Nexmo-Version: 3
    https://api.nexmo.com/v3
    URI vs Header
    15
    

    View Slide

16. https://api.nexmo.com/v1/calls
    Global vs Endpoint versioning
    16
    

    View Slide

17. https://api.nexmo.com/v1/calls
    https://api.nexmo.com/v1/media
    Global vs Endpoint versioning
    17
    

    View Slide

18. https://api.nexmo.com/v1/calls
    https://api.nexmo.com/v3/media
    Global vs Endpoint versioning
    18
    

    View Slide

19. https://api.nexmo.com/v1/calls
    Versioning Scheme
    19
    

    View Slide

20. https://api.nexmo.com/v1/calls
    https://api.example.com/2010-04-01
    Versioning Scheme
    20
    

    View Slide

21. https://api.nexmo.com/v1/calls
    https://api.example.com/2010-04-01
    https://api.example.com/1.2.3
    Versioning Scheme
    21
    

    View Slide

22. https://api.nexmo.com/v3.new-
    feature/calls
    Version Variants
    22
    

    View Slide

23. 3.
    HTTP Verbs
    GET, POST, PUT, DELETE, PATCH
    23
    

    View Slide

24. GET /users
    24
    

    View Slide

25. GET /users
    25
    

    View Slide

26. GET /users
    26
    

    View Slide

27. GET /users
    27
    

    View Slide

28. GET /users
    28
    

    View Slide

29. GET /users
    29
    

    View Slide

30. GET /users
    30
    200 OK
    301 Moved Permanently
    302 Found
    404 Not Found
    406 Not Acceptable
    

    View Slide

31. POST /users
    31
    

    View Slide

32. POST /users
    32
    201 Created
    202 Accepted
    415 Unsupported Media Type
    422 Unprocessable Entity
    

    View Slide

33. PUT /users/1
    33
    200 OK
    

    View Slide

34. DELETE /users/1
    34
    204 No Content
    

    View Slide

35. PATCH /users/1
    35
    

    View Slide

36. PATCH /users/1
    36
    RFC 7396 - JSON Merge Patch
    {
    "name":"Michael",
    "address": {
    "line_2": null
    }
    }
    

    View Slide

37. PATCH /users/1
    37
    RFC 6902 - JavaScript Object Notation
    (JSON) Patch
    [
    { "op": "replace", "path": "/name",
    "value": "Michael" },
    { "op": "remove", "path": "/
    address/line_2" }
    ]
    

    View Slide

38. 4.
    Resource Design
    URLs, HTTP Requests/Responses + Webhooks
    38
    

    View Slide

39. GET /users
    GET /users/1
    POST /users
    PUT /users/1
    PATCH /users/1
    DELETE /users/1
    URL Design
    39
    

    View Slide

40. GET /users/1
    Subresources
    40
    

    View Slide

41. GET /users/1
    GET /users/1/calls
    Subresources
    41
    

    View Slide

42. GET /users/1
    GET /users/1/calls
    GET /users/1/calls/JSDB-1837A
    Subresources
    42
    

    View Slide

43. GET /users/1
    GET /users/1/calls
    GET /users/1/calls/JSDB-1837A
    GET /calls/JSDB-1837A
    Subresources
    43
    

    View Slide

44. GET /users/1
    GET /users/1/calls
    Subresources
    44
    

    View Slide

45. GET /users/1
    GET /users/1/calls
    GET /calls?user=1
    Subresources
    45
    

    View Slide

46. GET /users/1
    GET /addresses/33-90210
    GET /users/1/addresses/33-90210
    Subresources
    46
    

    View Slide

47. GET /users/1
    GET /addresses/33-90210
    GET /users/1/addresses/33-90210
    GET /user_addresses/1-33-90210
    Subresources
    47
    

    View Slide

48. POST /machines/1/shutdown
    Actions
    48
    

    View Slide

49. POST /machines/1/shutdown
    POST /machines/1/actions/shutdown
    Actions
    49
    

    View Slide

50. POST /machines/1/shutdown
    POST /machines/1/actions/shutdown
    POST /machines/1/action:shutdown
    Actions
    50
    

    View Slide

51. POST /machines/1/shutdown
    POST /machines/1/actions/shutdown
    POST /machines/1/action:shutdown
    POST /machines/1/actions {"type":
    "shutdown"}
    Actions
    51
    

    View Slide

52. POST /machine-shutdown
    Actions as Resources
    52
    

    View Slide

53. POST /machine-shutdown
    PUT /article-locks/{article-id}
    DELETE /article-locks/{article-id}
    POST /balance-transfer
    Actions as Resources
    53
    

    View Slide

54. Resource Design
    54
    • Use snake_case for key names
    • [resource]_id - A string that identifies a specific resource
    • [description]_url - A url that is necessary for communication.
    For example, callback_url.
    • [description]_method - Send the callback using either GET or
    POST.
    • start - The time the communication started in https://
    en.wikipedia.org/wiki/ISO_8601 format.
    • end - The time the communication ended in https://
    en.wikipedia.org/wiki/ISO_8601 format.
    

    View Slide

55. Webhooks
    55
    • Use snake_case for key names
    • start - The time the communication started in https://
    en.wikipedia.org/wiki/ISO_8601 format.
    • end - The time the communication ended in https://
    en.wikipedia.org/wiki/ISO_8601 format.
    

    View Slide

56. 5.
    Errors
    Status Codes, RFC7807 + Extensions
    56
    

    View Slide

57. “
    Errors are easy! People should just
    use status codes
    

    View Slide

58. 401 Unauthorized
    Error Design
    58
    

    View Slide

59. 401 Unauthorized
    {
    "error": "No credentials provided"
    }
    Error Design
    59
    

    View Slide

60. 401 Unauthorized
    {
    "error": "Invalid credentials"
    }
    Error Design
    60
    

    View Slide

61. 401 Unauthorized
    {
    "error": "Invalid credentials",
    "more_help": "http://example.com/
    auth#invalid-credentials"
    }
    Error Design
    61
    

    View Slide

62. 401 Unauthorized
    {
    "msg": "Invalid credentials"
    }
    Error Design
    62
    

    View Slide

63. 401 Unauthorized
    {
    "errorCode": 118118,
    "errorMessage": "Invalid credentials"
    }
    Error Design
    63
    

    View Slide

64. 401 Unauthorized
    "Invalid credentials"
    Error Design
    64
    

    View Slide

65. RFC7807
    Type
    A URI
    reference that
    identifies the
    problem type.
    When followed,
    it provides
    human-readable
    documentation
    for the
    problem type
    Title
    A short,
    human-readable
    summary of the
    problem type
    Detail
    A human-
    readable
    explanation
    specific to
    this
    occurrence of
    the problem
    65
    Instance
    A URI
    reference that
    identifies the
    specific
    occurrence of
    the problem
    

    View Slide

66. RFC7807
    Type
    A URI
    reference that
    identifies the
    problem type.
    When followed,
    it provides
    human-readable
    documentation
    for the
    problem type
    Title
    A short,
    human-readable
    summary of the
    problem type
    Detail
    A human-
    readable
    explanation
    specific to
    this
    occurrence of
    the problem
    66
    Instance
    A URI
    reference that
    identifies the
    specific
    occurrence of
    the problem
    Status
    The HTTP
    status code
    generated by
    the origin
    server for
    this
    occurrence of
    the problem.
    

    View Slide

67. RFC7807
    Type
    A URI
    reference that
    identifies the
    problem type.
    When followed,
    it provides
    human-readable
    documentation
    for the
    problem type
    Title
    A short,
    human-readable
    summary of the
    problem type
    Detail
    A human-
    readable
    explanation
    specific to
    this
    occurrence of
    the problem
    67
    Instance
    A URI
    reference that
    identifies the
    specific
    occurrence of
    the problem
    

    View Slide

68. {
    "type": "https://developer.nexmo.com/api-
    errors#unauthorized",
    "title": "Invalid credentials supplied",
    "detail": "You did not provide correct credentials.",
    "instance": "797a8f199c45014ab7b08bfe9cc1c12c"
    }
    RFC7807
    68
    

    View Slide

69. 403 Forbidden
    {
    "type": "https://example.com/probs/out-of-
    credit",
    "title": "You do not have enough credit.",
    "detail": "Your current balance is 30, but
    that costs 50.",
    "instance": "/account/12345/msgs/abc"
    }
    Extension Members
    69
    

    View Slide

70. 403 Forbidden
    {
    "type": "https://example.com/probs/out-of-credit",
    "title": "You do not have enough credit.",
    "detail": "Your current balance is 30, but that
    costs 50.",
    "instance": "/account/12345/msgs/abc",
    "balance": 30,
    "accounts": ["/account/12345",
    "/account/67890"]
    }
    Extension Members
    70
    

    View Slide

71. 400 Bad Request
    {
    "type": "https://developer.nexmo.com/api-errors/account/secret-
    management#validation",
    "title": "Bad Request",
    "detail": "The request failed due to validation errors",
    "invalid_parameters": [
    {
    "name": "secret",
    "reason": "must contain 1 upper case character"
    }
    ],
    "instance": "797a8f199c45014ab7b08bfe9cc1c12c"
    }
    Extension Members
    71
    

    View Slide

72. 401: No Credentials, Invalid
    Credentials
    403: Feature disabled, Exceeded
    calls-per-second limit
    422: Invalid product specified
    + more!
    Response Library
    72
    

    View Slide

73. “
    As a consumer of the APIs, this has
    already dramatically reduced the
    amount of code I have to write for
    each endpoint, and I can't wait until
    all endpoints are standardised
    

    View Slide

74. 6.
    Design First APIs
    Consistency through specifications
    74
    

    View Slide

75. '400':
    $ref: '#/components/responses/
    InvalidPayloadError'
    '401':
    $ref: '../shared_errors.yml#/components/
    responses/BadCredentialsError'
    '405':
    $ref: '../shared_errors.yml#/components/
    responses/InvalidRequestMethod'
    '406':
    $ref: '../shared_errors.yml#/components/
    responses/InvalidAcceptHeader'
    Design First APIs
    75
    

    View Slide

76. Design First APIs
    76
    

    View Slide

77. Design First APIs
    77
    

    View Slide

78. 7.
    Collection Management
    Sorting, Filtering, Searching, Pagination
    78
    

    View Slide

79. Sorting
    79
    

    View Slide

80. GET /users?
    sort_by=email&order_by=asc
    Sorting
    80
    

    View Slide

81. GET /users?sort_by=email.asc
    Sorting
    81
    

    View Slide

82. GET /users?sort_by=email.asc
    &sort_by=status.desc
    Sorting
    82
    

    View Slide

83. GET /users?
    sort_by=email.asc,status.desc
    Sorting
    83
    

    View Slide

84. GET /users?sort_by=+email,-status
    Sorting
    84
    

    View Slide

85. GET /users?sort_by=+email,-status
    GET /users?
    sort_by=email.asc,status.desc
    Sorting
    85
    

    View Slide

86. Filtering
    86
    

    View Slide

87. Filtering
    87
    Discrete data e.g. user role
    Continuous data e.g. subscription
    expiry time
    

    View Slide

88. Discrete Data
    88
    /users?role=admin
    /orders?shipped=true
    /calls?status=active
    

    View Slide

89. Discrete Data
    89
    /users?role=admin
    /orders?shipped=true
    /calls?status=active
    403 Forbidden
    /users?is_fbi_informant=true
    

    View Slide

90. Discrete Data: JSON API
    90
    GET /employees?
    filter[role]=internal&filter[title]=
    senior
    

    View Slide

91. Discrete Data: JSON API
    91
    GET /employees?
    filter[role]=internal&filter[title]=
    senior
    GET /employees?
    role=internal&title=senior
    

    View Slide

92. Continuous Data
    92
    

    View Slide

93. Continuous Data
    93
    GET /orders?
    start_date=2018-01-01&end-
    date=2018=01-31
    

    View Slide

94. Continuous Data
    94
    GET /orders?
    start_date=2018-01-01&end-
    date=2018=01-31
    GET /orders?
    date[gte]=2018-01-01&date[lte]=2018=
    01-31
    

    View Slide

95. Industry Standards
    95
    

    View Slide

96. Industry Standards: SCIM
    96
    filter=userName eq "bjensen"
    filter=name.familyName co "O'Malley"
    filter=userName sw "J"
    filter=urn:ietf:params:scim:schemas:core:
    2.0:User:userName sw "J"
    filter=title pr
    filter=meta.lastModified gt "2011-05-13T04:42:34Z"
    https://tools.ietf.org/html/rfc7644#section-3.4.2.2
    

    View Slide

97. Searching
    97
    

    View Slide

98. Searching
    98
    GET /items?q=title:red chair AND
    price:[10 TO 100]
    

    View Slide

99. Pagination
    99
    Offset based
    Cursor based
    

    View Slide

100. Pagination: Offset
     100
     GET /calls?page=3&page_size=100
     

     View Slide

101. Pagination: Cursor
     101
     GET /calls
     GET /calls?
     cursor=2018-01-19T12:33:51
     GET /calls?
     cursor=2018-01-18T03:00:18
     

     View Slide

102. Pagination: Links
     102
     {
     "_links": {
     "next": {
     "href": "/calls?cursor=9274"
     }
     }
     }
     Link: cursor=9274>; rel="next",
     

     View Slide

103. 8.
     Hypermedia
     Specifically, HAL+JSON
     103
     

     View Slide

104. [
     {
     "id": "78d335fa323d01149c3dd6f0d489",
     "name": "My Application",
     }
     ]
     GET /v2/applications/78d335fa323d01149c3dd6f0d489
     104
     

     View Slide

105. [
     {
     "id": "78d335fa323d01149c3dd6f0d48968cf",
     "name": "My Application",
     "_links": {
     "self": {
     "href": "/v2/applications/
     78d335fa323d01149c3dd6f0d48968cf"
     }
     }
     }
     ]
     GET /v2/applications/78d335fa323d01149c3dd6f0d489
     105
     

     View Slide

106. [
     {
     "id": "78d335fa323d01149c3dd6f0d48968cf",
     "name": "My Application",
     "_links": {
     "self": {
     "href": "/v2/applications/
     78d335fa323d01149c3dd6f0d48968cf"
     },
     "numbers": {
     "href": "/v2/numbers?
     application=78d335fa323d01149c3dd6f0d48968cf"
     }
     }
     }
     ]
     GET /v2/applications/78d335fa323d01149c3dd6f0d489
     106
     

     View Slide

107. "hints": {
     "allow": [
     "GET",
     "PUT",
     "DELETE",
     "PATCH"
     ],
     "formats": {
     "application/json": {}
     }
     }
     Hints Format
     107
     

     View Slide

108. {
     "numbers": {
     "href": "/v2/numbers?
     application=78d335fa323d01149c3dd6f0d48968cf",
     "hints": {
     "allow": [
     "GET",
     "PUT",
     "DELETE",
     "PATCH"
     ],
     "formats": {
     "application/json": {}
     }
     }
     }
     Hints Format
     108
     

     View Slide

109. 9.
     Rate Limiting
     Should we add rate limiting?
     109
     

     View Slide

110. Yes.
     Rate Limiting
     110
     

     View Slide

111. Options:
     • Per-server
     • Per-region
     • Per-user
     Rate Limiting
     111
     

     View Slide

112. Refresh Rates:
     • Per-second limits
     • Interval
     • Bucket
     Rate Limiting
     112
     

     View Slide

113. How about queueing?
     Rate Limiting
     113
     

     View Slide

114. 10.
     Documentation
     This isn't an optional task for your API
     114
     

     View Slide

115. Open API Specification: Docs
     115
     

     View Slide

116. Open API Specification: Postman
     116
     

     View Slide

117. Open API Specification: Generated Code
     117
     

     View Slide

118. Written Docs
     118
     

     View Slide

119. Open API Specification: Generated Code
     119
     

     View Slide

120. 11.
     Conclusion
     What have we learned?
     120
     

     View Slide

121. @mheap
     m@michaelheap.com
     

     View Slide