Creating Solid APIs

Presented at DjangoCon Europe 2018

Creating APIs that other developers love to use isn't trivial. I'll share some tips and best practices.

Transcript

1. Creating Solid APIs
   DjangoCon Europe 2018
   ⋅Rivo Laks 2018-05-23
   

   View Slide

2. Background
   

   View Slide

3. What is an API?
   

   View Slide

4. What is an API?
   application programming interface
   

   View Slide

5. What is an API?
   application programming interface
   application programmer interface
   

   View Slide

6. What is an API?
   application programming interface
   application programmer interface
   API is user interface for developers
   

   View Slide

7. What Makes an API Good?
   

   View Slide

8. What Makes an API Good?
   Documentation
   

   View Slide

9. What Makes an API Good?
   Documentation
   Familiarity
   

   View Slide

10. What Makes an API Good?
    Documentation
    Familiarity
    Lack of friction
    

    View Slide

11. Documentation
    Often overlooked
    

    View Slide

12. Documentation
    Often overlooked
    Gives the rst impression
    

    View Slide

13. Documentation
    Often overlooked
    Gives the rst impression
    The e ort is worth it!
    

    View Slide

14. Creating
    Awesome Docs
    

    View Slide

15. What Should Go In There?
    

    View Slide

16. What Should Go In There?
    How do I access it?
    

    View Slide

17. What Should Go In There?
    How do I access it?
    Do I need developer account?
    

    View Slide

18. What Should Go In There?
    How do I access it?
    Do I need developer account?
    Root URL, etc
    

    View Slide

19. What Should Go In There?
    How do I access it?
    Do I need developer account?
    Root URL, etc
    Authentication info
    

    View Slide

20. What Should Go In There?
    General encodings, formats, etc
    

    View Slide

21. What Should Go In There?
    General encodings, formats, etc
    Pagination, versioning, etc
    

    View Slide

22. What Should Go In There?
    General encodings, formats, etc
    Pagination, versioning, etc
    Common errors
    

    View Slide

23. What Should Go In There?
    General encodings, formats, etc
    Pagination, versioning, etc
    Common errors
    Code for getting started
    

    View Slide

24. The Endpoints
    

    View Slide

25. The Endpoints
    URL & operations
    

    View Slide

26. The Endpoints
    URL & operations
    Request/response data
    

    View Slide

27. The Endpoints
    URL & operations
    Request/response data
    Optional parameters
    

    View Slide

28. The Endpoints
    URL & operations
    Request/response data
    Optional parameters
    Permissions etc
    

    View Slide

29. Keep it Fresh!
    

    View Slide

30. Keep it Fresh!
    Obsolete docs are the worst
    

    View Slide

31. Keep it Fresh!
    Obsolete docs are the worst
    Always autogenerate!
    

    View Slide

32. Keep it Fresh!
    Obsolete docs are the worst
    Always autogenerate!
    Usually code » schema » docs
    

    View Slide

33. Schema & Autogeneration
    

    View Slide

34. Schema & Autogeneration
    OpenAPI, Swagger, etc
    

    View Slide

35. Schema & Autogeneration
    OpenAPI, Swagger, etc
    Use your tools
    

    View Slide

36. Schema & Autogeneration
    OpenAPI, Swagger, etc
    Use your tools
    Combine docs & code examples
    

    View Slide

37. Schema & Autogeneration
    OpenAPI, Swagger, etc
    Use your tools
    Combine docs & code examples
    Client libs autogeneration
    

    View Slide

38. Standardize!
    

    View Slide

39. JSON API
    http://jsonapi.org/
    one potential standard to use
    

    View Slide

40. JSON API
    http://jsonapi.org/
    one potential standard to use
    GraphQL is another option
    

    View Slide

41. Standardize Structure
    

    View Slide

42. Standardize Structure
    Responses have predictable, familiar
    structure
    

    View Slide

43. GET https://example.com/api/v1/projects
    {
    "links": {
    "next": "https://example.com/api/v1/projects?cursor=cD0yMDE4L",
    "prev": null
    },
    "data": [...],
    "included": [...]
    }
    

    View Slide

44. "data": [
    {
    "type": "project",
    "id": "289",
    "links": {
    "self": "https://example.com/api/v1/projects/289"
    },
    "attributes": {
    "created": "2018-04-28T22:52:08.690486Z",
    "name": "Allison-Patterson",
    "description": "aggregate collaborative models"
    },
    "relationships": {...}
    },
    ...
    ],
    

    View Slide

45. "data": [{ ...
    "relationships": {
    "created_by": {
    "data": {
    "type": "user",
    "id": "199"
    }
    },
    "epics": {
    "data": [
    {
    "type": "epic",
    "id": "3101"
    }
    ],
    }
    } }, ...
    ],
    

    View Slide

46. "included": [
    {
    "type": "epic",
    "id": "3101",
    "attributes": {
    "created": "2018-04-28T22:50:45.885691Z",
    "name": "Ergonomic background extranet"
    },
    "links": {
    "self": "https://example.com/api/v1/epics/3101"
    }
    },
    {
    "type": "user",
    "id": "199",
    "attributes": {...}
    }
    ]
    

    View Slide

47. Impressions?
    

    View Slide

48. Flexibility
    Con gurable elds:
    GET https://example.com/api/v1/projects
    GET https://example.com/api/v1/projects \
    ?included=comments
    GET https://example.com/api/v1/projects \
    ?included=comments&fields[project]=name,comments
    

    View Slide

49. Pagination
    List responses have next / prev links
    {
    "links": {
    "next": "https://example.com/api/v1/projects?cursor=cD0yMDE4L",
    "prev": null
    },
    "data": [...],
    }
    

    View Slide

50. Pagination
    List responses have next / prev links
    {
    "links": {
    "next": "https://example.com/api/v1/projects?cursor=cD0yMDE4L",
    "prev": null
    },
    "data": [...],
    }
    Cursor-base pagination FTW (but YMMV)
    

    View Slide

51. There is More ...
    Filtering
    Ordering
    

    View Slide

52. Errors
    

    View Slide

53. Errors
    POST https://example.com/api/v1/projects
    {
    "errors": [
    {
    "title": "Invalid Attribute",
    "detail": "Name must contain at least three letters.",
    "source": { "pointer": "/data/attributes/name" },
    "status": "422"
    }
    ]
    }
    

    View Slide

54. Special Cases
    For when you have LOTS of data
    

    View Slide

55. Special Cases
    For when you have LOTS of data
    out-of-band approach
    

    View Slide

56. GET https://example.com/api/v1/datasets/123
    {
    "data": {
    "type": "dataset",
    "id": "123",
    "attributes": {
    "name": "CIFAR10 dataset",
    },
    "links": {
    "data_tgz": "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz",
    "self": "https://example.com/api/v1/datasets/123"
    }
    }
    }
    

    View Slide

57. Standardization Matters
    the speci c standard isn't that important
    GraphQL, etc are also good options
    

    View Slide

58. Authentication &
    Authorization
    

    View Slide

59. Token Authentication
    Clients send HTTP header, ala
    Authorization: Token 9944b09199c62bcf9418
    

    View Slide

60. OAuth 2.0
    

    View Slide

61. OAuth 2.0
    For creating platforms
    

    View Slide

62. OAuth 2.0
    For creating platforms
    Complex, but solves many issues
    

    View Slide

63. OAuth 2.0
    For creating platforms
    Complex, but solves many issues
    Many packages, e.g. Django OAuth Toolkit,
    OAuthLib
    

    View Slide

64. Versioning
    

    View Slide

65. Versioning Schemes
    

    View Slide

66. Versioning Schemes
    AcceptHeaderVersioning (DRF)
    GET /projects HTTP/1.0
    Accept: application/json; version=1.0
    

    View Slide

67. Versioning Schemes
    AcceptHeaderVersioning (DRF)
    GET /projects HTTP/1.0
    Accept: application/json; version=1.0
    URLPathVersioning (DRF)
    GET /v1/projects HTTP/1.0
    Accept: application/json
    

    View Slide

68. Versioning Schemes Cont.
    Integers ( v1 ) vs dates ( 2018-05-23 )?
    

    View Slide

69. Versioning Schemes Cont.
    Integers ( v1 ) vs dates ( 2018-05-23 )?
    Dates are less emotional
    

    View Slide

70. Versioning Schemes Cont.
    Integers ( v1 ) vs dates ( 2018-05-23 )?
    Dates are less emotional
    Make upgrades easy
    

    View Slide

71. Version Transformers
    

    View Slide

72. Version Transformers
    » » Requests into newer version » »
    Core code is for latest version
    « « Responses into older version « «
    

    View Slide

73. Version Transformers
    » » Requests into newer version » »
    Core code is for latest version
    « « Responses into older version « «
    Won't work for big, breaking changes
    

    View Slide

74. tg-apicore
    

    View Slide

75. Thorgate API Core
    pip install tg-apicore
    Addon for Django REST Framework
    

    View Slide

76. Thorgate API Core
    pip install tg-apicore
    Addon for Django REST Framework
    Documentation generation
    Pre-con gured
    Viewset/serializer utils
    

    View Slide

77. Client's Perspective
    

    View Slide

78. The Scenario
    Let's try speech recognition...
    

    View Slide

79. The Scenario
    Let's try speech recognition...
    ... using AWS and GCP
    

    View Slide

80. Getting Started
    Documentation
    

    View Slide

81. Getting Started
    Documentation
    Quite easy to nd
    A bit overwhelming
    

    View Slide

82. Getting Started
    Documentation
    Quite easy to nd
    A bit overwhelming
    Code examples
    

    View Slide

83. Comprehensive Clients
    Install Python client
    GCP: google-cloud package
    AWS: boto3 package
    

    View Slide

84. Comprehensive Clients
    Install Python client
    GCP: google-cloud package
    AWS: boto3 package
    Authenticate
    

    View Slide

85. Comprehensive Clients
    Install Python client
    GCP: google-cloud package
    AWS: boto3 package
    Authenticate
    Thorough docs
    

    View Slide

86. Amazon
    import boto3
    client = boto3.client('transcribe')
    response = client.start_transcription_job(...)
    

    View Slide

87. Amazon
    import boto3
    client = boto3.client('transcribe')
    response = client.start_transcription_job(...)
    Google
    from google.cloud import speech
    client = speech.SpeechClient()
    results = client.recognize(...)
    

    View Slide

88. In Summary
    Invest in documentation
    Embrace standards (e.g. JSON API)
    Use automation
    Reduce friction
    

    View Slide

89. Thanks!
    Rivo Laks ⋅ @rivolaks ⋅ rivolaks.com
    Lead Engineer at Thorgate ⋅ thorgate.eu
    API framework for Django/DRF ⋅ tg-apicore (WIP)
    

    View Slide