Broken APIs Break Trust: Tips for Creating and Updating APIs

Broken APIs Break Trust: Tips for Creating and Updating APIs

For many of us, APIs and their client libraries are the face of our applications to the world. We need to innovate with new features, but breaking changes are toxic to customer trust.

In this session you will pick-up concrete design patterns that you can use and anti-patterns that you can recognize so that your service API or library can continue to grow without breaking the world. Using real APIs and open source libraries as a case study, we’ll look at actual examples of specific strategies that you can employ to keep your API backwards compatible without painting yourself into a corner.

Fceb24dfe052368eec601f5a0934fa42?s=128

Alex Wood

April 18, 2018
Tweet

Transcript

  1. BROKEN APIS BREAK TRUST TIPS FOR CREATING AND UPDATING APIS

  2. None
  3. Instagram: @pug_katsu

  4. WHO AM I? ▸ AWS SDK for Ruby ▸ Twitter:

    @alexwwood ▸ Please do tweet your comments and questions about the talk! ▸ Bad Jokes
  5. AGENDA ▸ Defining APIs and Backwards Compatibility ▸ Safe API

    Changes ▸ Constraints and Validation ▸ Exceptions ▸ Undocumented API Contracts ▸ Conclusion: Backwards Compatibility Rules
  6. DEFINITIONS

  7. "THERE IS A THEORY WHICH STATES THAT IF EVER ANYONE

    DISCOVERS EXACTLY WHAT THE UNIVERSE IS FOR AND WHY IT IS HERE, IT WILL INSTANTLY DISAPPEAR AND BE REPLACED BY SOMETHING EVEN MORE BIZARRE AND INEXPLICABLE. THERE IS ANOTHER THEORY WHICH STATES THAT THIS HAS ALREADY HAPPENED." Douglas Adams, "Hitchhiker's Guide to the Galaxy"
  8. ANATOMY OF A WEB API ▸ Resources: Things that an

    API manages. ▸ Shape: A structure with members. ▸ Input Shape: A top-level request shape passed to an operation. ▸ Output Shape: A top-level response shape returned from an operation. ▸ Error Shape: A structure that represents an error response from an operation. ▸ Member: Belongs to a shape, represents a property of the shape. ▸ Operation: Something exposed by the service that can be invoked. ▸ NOTE: Can all of this apply to libraries which expose an API but do not make calls to the internet? Absolutely.
  9. EVOLUTION OF CLIENTS RELATIVE TO APIS CLIENT API

  10. EVOLUTION OF CLIENTS RELATIVE TO APIS CLIENT V2 API V2

    CLIENT V1 API V1
  11. EVOLUTION OF CLIENTS RELATIVE TO APIS CLIENT V2 API V2

    CLIENT V1 API V1
  12. EVOLUTION OF CLIENTS RELATIVE TO APIS CLIENT V3 API V3

    CLIENT V2 API V2 CLIENT V1 API V1
  13. EVOLUTION OF CLIENTS RELATIVE TO APIS CLIENT V3 API V3

    CLIENT V2 API V2 CLIENT V1 API V1 CLIENT V4 API V4
  14. EVOLUTION OF CLIENTS RELATIVE TO APIS CLIENT V3 API V3

    CLIENT V2 API V2 CLIENT V1 API V1 CLIENT V4 API V4 CLIENT ∞ API ∞
  15. EVOLUTION OF CLIENTS RELATIVE TO APIS API' CLIENT CLIENT'

  16. API Backward Compatibility + Client Forward Compatibility =

  17. ANATOMY OF A WEB API Trip travelers [string array] description

    [string] flight [Flight] ...
  18. ANATOMY OF A WEB API Trip travelers [string array] description

    [string] flight [Flight] ... SHAPE MEMBERS
  19. ANATOMY OF A WEB API Trip travelers [string array] description

    [string] flight [Flight] ... SHAPE MEMBERS SPECIAL MEMBER
  20. ANATOMY OF A WEB API Trip travelers [string array] description

    [string] flight [Flight] ... Flight flight_number [string] airline [string] status [string] ...
  21. ANATOMY OF A WEB API ▸ GetTrip ▸ CreateTrip ▸

    UpdateTrip ▸ ListTrips ▸ DeleteTrip
  22. ANATOMY OF A WEB API ▸ GetTrip ▸ CreateTrip ▸

    UpdateTrip ▸ ListTrips ▸ DeleteTrip ▸ GET /trips/:id ▸ POST /trips ▸ PATCH /trips/:id ▸ GET /trips ▸ DELETE /trips/:id
  23. ANATOMY OF A WEB API ▸ GetTrip ▸ CreateTrip ▸

    UpdateTrip ▸ ListTrips ▸ DeleteTrip ▸ GET /trips/:id ▸ POST /trips ▸ PATCH /trips/:id ▸ GET /trips ▸ DELETE /trips/:id ▸ 'trips#show' ▸ 'trips#create' ▸ 'trips#update' ▸ 'trips#index' ▸ 'trips#destroy'
  24. ANATOMY OF A WEB API trip_client.get_trip(id: 123) GET /trips/123 {

    trip: { travelers: ["Alex"], description: "RailsConf 2018", flight: { flight_number: "DL3378", airline: "Delta", status: "Landed" } } }
  25. ANATOMY OF A WEB API trip_client.get_trip(id: 123) GET /trips/123 {

    trip: { travelers: ["Alex"], description: "RailsConf 2018", flight: { flight_number: "DL3378", airline: "Delta", status: "Landed" } } }
  26. ANATOMY OF A WEB API trip_client.get_trip(id: 123) GET /trips/123 {

    trip: { travelers: ["Alex"], description: "RailsConf 2018", flight: { flight_number: "DL3378", airline: "Delta", status: "Delayed" # Because of course it was. } } }
  27. ANATOMY OF A WEB API trip_client.get_trip(id: 321) GET /trips/321 ResourceNotFound

  28. ANATOMY OF A WEB API ▸ Resources: Things that an

    API manages. ▸ Shape: A structure with members. ▸ Input Shape: A top-level request shape passed to an operation. ▸ Output Shape: A top-level response shape returned from an operation. ▸ Error Shape: A structure that represents an error response from an operation. ▸ Member: Belongs to a shape, represents a property of the shape. ▸ Operation: Something exposed by the service that can be invoked. ▸ NOTE: Can all of this apply to libraries which expose an API but do not make calls to the internet? Absolutely.
  29. SAFE API CHANGES

  30. I MIGHT DROP THE WORLD ...IF I CHANGE HANDS. Lil'

    Wayne
  31. ADDING & REMOVING Trip travelers [string array] description [string] flight

    [Flight] ...
  32. ADDING & REMOVING Trip travelers [string array] description [string] flight

    [Flight] confirmed [boolean]
  33. ADDING & REMOVING Trip travelers [string array] description [string] flight

    [Flight] confirmed [boolean]
  34. ADDING & REMOVING Trip travelers [string array] description [string] flight

    [Flight] confirmed [boolean]
  35. ADDING & REMOVING Trip travelers [string array] description [string] flight

    [Flight] confirmed [boolean] WRONG
  36. ADDING & REMOVING Trip travelers [string array] description [string] flight

    [Flight] confirmed [booleanstring]
  37. ADDING & REMOVING Trip travelers [string array] description [string] flight

    [Flight] confirmed [booleanstring] WRONG
  38. ADDING MEMBERS Trip travelers [string array] description [string] flight [Flight]

    confirmed [boolean]
  39. ADDING MEMBERS - NEW CLIENT trip_client = TripClient.new resp =

    trip_client.get_trip(id: 321) resp.travelers # => ["Alex"] resp.confirmed # => true
  40. ADDING MEMBERS - OLD CLIENT trip_client = TripClient.new resp =

    trip_client.get_trip(id: 321) resp.travelers # => ["Alex"]
  41. INPUT SHAPES - ADDING AND REMOVING ListTripsRequest next_token [string] max_results

    [integer] ... ...
  42. INPUT SHAPES - ADDING AND REMOVING ListTripsRequest next_token [string] max_results

    [integer] confirmed [boolean] ...
  43. INPUT SHAPES - ADDING AND REMOVING ListTripsRequest next_token [string] max_results

    [integer] confirmed [boolean] ...
  44. INPUT SHAPES - ADDING AND REMOVING ListTripsRequest next_token [string] max_results

    [integer] confirmed [boolean] (Required) ...
  45. INPUT SHAPES - ADDING AND REMOVING ListTripsRequest next_token [string] max_results

    [integer] confirmed [boolean] (Required) ... WRONG
  46. INPUT SHAPES - ADDING AND REMOVING trip_client = TripClient.new resp

    = trip_client.list_trips
  47. INPUT SHAPES - ADDING AND REMOVING trip_client = TripClient.new resp

    = trip_client.list_trips # TripClient::Errors::MissingRequiredParameter
  48. INPUT SHAPES - ADDING AND REMOVING trip_client = TripClient.new resp

    = trip_client.list_trips # TripClient::Errors::MissingRequiredParameter WRONG
  49. INPUT SHAPES - ADDING AND REMOVING trip_client = TripClient.new resp

    = trip_client.list_trips
  50. INPUT SHAPES - ADDING AND REMOVING trip_client = TripClient.new resp

    = trip_client.list_trips # ArgumentError: missing required parameter # :confirmed
  51. INPUT SHAPES - ADDING AND REMOVING trip_client = TripClient.new resp

    = trip_client.list_trips # ArgumentError: missing required parameter # :confirmed WRONG
  52. KEY TAKEAWAYS ‣ Having your code break on you is

    a jarring, painful experience. Make serious efforts to avoid doing this to your users. ‣ Since some changes are harder than others, think ahead when initially launching your API to avoid the likelihood of needing to make "bad changes". ‣ For Web APIs, there is no complete way around this no matter how clever you are with clients. ‣ Community clients will be built. ‣ People will make raw, well-formed HTTP requests outside of your client libraries. ‣ Ruby does NOT get a free lunch.
  53. CONSTRAINTS, VALIDATION, AND EXCEPTIONS

  54. DID YOU HAVE TO DO THIS? I WAS THINKING THAT

    YOU COULD BE TRUSTED. Taylor Swift
  55. AGAIN: NO ADDING NEW REQUIRED PARAMETERS ▸ Breaks existing code

    AND old clients. ▸ Removing "required" attribute from parameters is okay.
  56. CONSTRAINT VALIDATION ListTripsRequest next_token [string] max_results [integer] (Maximum: 25) confirmed

    [boolean] ...
  57. CONSTRAINT VALIDATION ListTripsRequest next_token [string] max_results [integer] (Maximum: 20) confirmed

    [boolean] ...
  58. CONSTRAINT VALIDATION ListTripsRequest next_token [string] max_results [integer] (Maximum: 20) confirmed

    [boolean] ... WRONG
  59. CONSTRAINT VALIDATION ListTripsRequest next_token [string] max_results [integer] (Maximum: 50) confirmed

    [boolean] ...
  60. TERMINAL STATES Flight flight_number [string] airline [string] status [string] ...

  61. TERMINAL STATES ▸ Flight.status ▸ Interim: OnTime, Delayed, Boarding, InFlight

    ▸ Terminal: Landed, Cancelled
  62. TERMINAL STATES ▸ Flight.status ▸ Interim: OnTime, Delayed, Boarding, InFlight

    ▸ Terminal: Landed, LandedOnTime, LandedLate, Cancelled
  63. TERMINAL STATES ▸ Flight.status ▸ Interim: OnTime, Delayed, Boarding, InFlight

    ▸ Terminal: Landed, LandedLate, Cancelled
  64. TERMINAL STATES ▸ Flight.status ▸ Interim: OnTime, Delayed, Boarding, InFlight

    ▸ Terminal: Landed, LandedLate, Cancelled WRONG
  65. TERMINAL STATES ▸ Flight.status ▸ Interim: OnTime, Delayed, Boarding, InFlight

    ▸ Terminal: Landed, Cancelled
  66. TERMINAL STATES ▸ Flight.status ▸ Interim: OnTime, Delayed, Boarding, InFlight,

    OnApproach ▸ Terminal: Landed, Cancelled
  67. EXCEPTIONS trip_client.get_trip(id: 321) GET /trips/321 ResourceNotFound

  68. EXCEPTIONS begin resp = trip_client.get_trip(id: id) rescue ResourceNotFound => e

    logger.info("Trip #{id} does not exist.") nil end
  69. EXCEPTIONS trip_client.get_trip(id: 321) GET /trips/321 ResourceNotFound || ResourceDeleted

  70. EXCEPTIONS begin resp = trip_client.get_trip(id: id) rescue ResourceNotFound => e

    logger.info("Trip #{id} does not exist.") nil end # Suddenly this raises ResourceDeleted...
  71. EXCEPTIONS begin resp = trip_client.get_trip(id: id) rescue ResourceNotFound => e

    logger.info("Trip #{id} does not exist.") nil end # Suddenly this raises ResourceDeleted... WRONG
  72. EXCEPTIONS begin resp = trip_client.get_trip(id: id) rescue ResourceNotFound => e

    if e.previously_existed? logger.info("Trip #{id} previously deleted.") else logger.info("Trip #{id} does not exist.") end nil end
  73. UNDOCUMENTED API CONTRACTS

  74. WHEN GIVEN A CHOICE BETWEEN TWO OPTIONS, CHOOSE ALL THREE.

    John Carlyle
  75. WITH A SUFFICIENT NUMBER OF USERS OF AN API, IT

    DOES NOT MATTER WHAT YOU PROMISE IN THE CONTRACT, ALL OBSERVABLE BEHAVIORS OF YOUR SYSTEM WILL BE DEPENDED ON BY SOMEBODY. Hyrum's Law
  76. CONCLUSION API DESIGN RULES YOU CAN USE!

  77. RELATED TALKS ▸ AWS re:Invent 2017: Embracing Change without Breaking

    the World (DEV319) ▸ AWS re:Invent 2017: Deploying and Managing Ruby Applications on AWS (DEV207) ▸ RailsConf 2018: The GraphQL Way: A new path for JSON APIs. ▸ Note: This talk already happened, but you can view later on YouTube for a different POV on some of these topics.
  78. FOUR OUT OF FIVE DENTISTS RECOMMEND SAVING THIS SLIDE FOR

    REFERENCE DO ▸ APIs ▸ Add members/shapes ▸ Add intermediate workflow states ▸ Add detail to exceptions ▸ Add new opt-in exceptions ▸ Loosening constraints ▸ Clients ▸ Forward compatible (support all of the above) ▸ Focus on discoverability ▸ APIs ▸ Remove/rename members and shapes. ▸ Change member types ▸ Add new terminal workflow states ▸ Add new exceptions without opt-in ▸ Tighten constraints ▸ Clients ▸ Validate API constraints DO NOT