Eve - REST API for Humans™

Introducing the Eve REST API Framework.

PyCon Web 2018, Munich
PyCon Belarus 2018, Minsk
PiterPy 2016, St. Petersburg
EuroPython 2014, Berlin
Python Meetup, Helsinki
PyCon Italy 2014, Florence
PyCon Sweden 2014, Stockholm
FOSDEM 2014, Brussels

Transcript

1. REST API FOR HUMANS
   eve
   REST WEB API
   eve
   

   View Slide

2. NICOLA IAROCCI
   Co-Founder @ CIR2000
   Lead Developer @ gestionaleamica.com
   Microsoft MVP
   MongoDB Master
   Open Source Author
   CoderDojo
   DevRomagna
   @nicolaiarocci / nicolaiarocci.com / eve@nicolaiarocci.com
   

   View Slide

3. EVE IS POWERED BY
   

   View Slide

4. View Slide

5. View Slide

6. View Slide

7. QUICKSTART
   

   View Slide

8. RUN.PY
   from eve import Eve
   app = Eve()
   app.run()
   

   View Slide

9. SETTINGS.PY
   # just a couple API endpoints with no
   # custom options and validation rules
   DOMAIN = {
   'people': {}
   'books': {}
   }
   

   View Slide

10. LAUNCH THE API
    $ python run.py
    * Running on http://127.0.0.1:5000/
    

    View Slide

11. TEST YOUR API
    $ curl -i http://localhost:5000/
    {
    "_items": [],
    "_links": {
    "self": {
    "href": "127.0.0.1:5000/people",
    "title": "people" },
    "parent": {
    "href": "127.0.0.1:5000",
    "title": "home"}
    }
    }
    

    View Slide

12. $ curl -i http://localhost:5000/
    {
    "_items": [],
    "_links": {
    "self": {
    "href": "127.0.0.1:5000/people",
    "title": "people" },
    "parent": {
    "href": "127.0.0.1:5000",
    "title": "home"}
    }
    }
    HATEOAS AT WORK HERE
    TEST YOUR API
    

    View Slide

13. $ curl -i http://localhost:5000/
    {
    "_items": [],
    "_links": {
    "self": {
    "href": "127.0.0.1:5000/people",
    "title": "people" },
    "parent": {
    "href": "127.0.0.1:5000",
    "title": "home"}
    }
    }
    TEST YOUR API
    CLIENTS CAN EXPLORE THE API PROGRAMMATICALLY
    

    View Slide

14. $ curl -i http://localhost:5000/
    {
    "_items": [],
    "_links": {
    "self": {
    "href": "127.0.0.1:5000/people",
    "title": "people" },
    "parent": {
    "href": "127.0.0.1:5000",
    "title": "home"}
    }
    }
    TEST YOUR API
    AND EVENTUALLY FILL THEIR UI
    

    View Slide

15. $ curl -i http://localhost:5000/
    {
    "_items": [],
    "_links": {
    "self": {
    "href": "127.0.0.1:5000/people",
    "title": "people" },
    "parent": {
    "href": "127.0.0.1:5000",
    "title": "home"}
    }
    }
    EMTPY RESOURCE AS WE DIDN’T CONNECT ANY DATASOURCE
    TEST YOUR API
    

    View Slide

16. SETTINGS.PY
    # connect to mongo
    MONGO_HOST = 'localhost'
    MONGO_PORT = 27017
    MONGO_USERNAME = 'user'
    MONGO_PASSWORD = 'pw'
    MONGO_DBNAME = 'apitest'
    # or (better):
    MONGO_URI = ‘mongodb://user:pw@localhost:27017/apitest'
    

    View Slide

17. DEMO TIME!
    

    View Slide

18. # add fields and validation rules for 'people' endpoint
    DOMAIN['people']['schema'] = {
    'name': {
    'type': 'string',
    'maxlength': 50,
    'unique': True}
    'email': {
    'type': 'string',
    'regex': '^\S+@\S+$'},
    'location': {
    'type': 'dict',
    'schema': {
    'address': {'type': 'string'},
    'city': {'type': 'string'}}},
    'born': {'type': 'datetime'}}
    USE BETTER REGEX IN PRODUCTION
    SETTINGS.PY
    

    View Slide

19. SETTINGS.PY
    # enable writes. default is ['GET']
    # /people
    RESOURCE_METHODS = ['GET','POST']
    # /people/
    ITEM_METHODS = ['GET','PATCH','PUT','DELETE']
    

    View Slide

20. # enable writes. default is ['GET']
    # /people
    RESOURCE_METHODS = ['GET','POST']
    # /people/
    ITEM_METHODS = ['GET','PATCH','PUT','DELETE']
    SETTINGS.PY
    ADD/CREATE ONE OR MORE ITEMS
    

    View Slide

21. # enable writes. default is ['GET']
    # /people
    RESOURCE_METHODS = ['GET','POST']
    # /people/
    ITEM_METHODS = ['GET','PATCH','PUT','DELETE']
    SETTINGS.PY
    UPDATE DOCUMENT
    

    View Slide

22. # enable writes. default is ['GET']
    # /people
    RESOURCE_METHODS = ['GET','POST']
    # /people/
    ITEM_METHODS = ['GET','PATCH','PUT','DELETE']
    SETTINGS.PY
    REPLACE DOCUMENT
    

    View Slide

23. # enable writes. default is ['GET']
    # /people
    RESOURCE_METHODS = ['GET','POST']
    # /people/
    ITEM_METHODS = ['GET','PATCH','PUT','DELETE']
    SETTINGS.PY
    YOU GUESSED IT
    

    View Slide

24. SETTINGS.PY
    # a few additional configuration options
    DOMAIN['people'].update(
    {
    'item_title': 'person',
    'cache_control': 'max-age=10,must-revalidate',
    'cache_expires': 10,
    'additional_lookup': {
    'url': 'regex("[\w]+")',
    'field': 'name'
    }
    )
    

    View Slide

25. FEATURES
    

    View Slide

26. MONGO FILTERS
    ?where={“lastname”: “Doe”}
    

    View Slide

27. PYTHON FILTERS
    ?where=lastname==“Doe”
    

    View Slide

28. SORTING
    ?sort=-total
    SORT BY ‘TOTAL’, DESCENDING
    DEMO
    

    View Slide

29. SORTING
    ?sort=[(“total”: -1)]
    SAME, MONGODB-STYLE
    

    View Slide

30. PAGINATION
    ?max_results=20&page=2
    MAX 20 RESULTS/PAGE; PAGE 2
    

    View Slide

31. PROJECTIONS
    ?projection={"avatar": 0}
    RETURN ALL FIELDS BUT ‘AVATAR’
    

    View Slide

32. PROJECTIONS
    ?projection={"lastname": 1}
    ONLY RETURN ‘LASTNAME’
    

    View Slide

33. EMBEDDED RESOURCES
    ?embedded={"author": 1}
    DEMO
    

    View Slide

34. NOT EMBEDDED
    $ curl -i
    HTTP/1.1 200 OK
    {
    "title": "Book Title",
    "description": "book description",
    "author": "52da465a5610320002660f94"
    }
    RAW FOREIGN KEY
    

    View Slide

35. EMBEDDED
    $ curl -i ?embedded={"author": 1}
    HTTP/1.1 200 OK
    {
    "title": "Book Title",
    "description": "book description",
    "author": {
    "firstname": "Mark",
    "lastname": "Green",
    }
    }
    REQUEST EMBEDDED AUTHOR
    

    View Slide

36. EMBEDDED
    $ curl -i ?embedded={"author": 1}
    HTTP/1.1 200 OK
    {
    "title": "Book Title",
    "description": "book description",
    "author": {
    "firstname": "Mark",
    "lastname": "Green",
    }
    }
    EMBEDDED DOCUMENT
    

    View Slide

37. JSON AND XML
    SERIALIZATION FOR ALL RESPONSES
    DEMO
    

    View Slide

38. APPLICATION/JSON
    [
    {
    "firstname": "Mark",
    "lastname": "Green",
    "born": "Sat, 23 Feb 1985 12:00:00 GMT",
    "role": ["copy", "author"],
    "location": {"city": "New York", "address": "4925 Lacross Road"},
    "_id": "50bf198338345b1c604faf31",
    "_updated": "Wed, 05 Dec 2012 09:53:07 GMT",
    "_created": "Wed, 05 Dec 2012 09:53:07 GMT",
    "_etag": "ec5e8200b8fa0596afe9ca71a87f23e71ca30e2d",
    },
    {
    "firstname": "John",
    ...
    },
    ]
    

    View Slide

39. [
    {
    "firstname": "Mark",
    "lastname": "Green",
    "born": "Sat, 23 Feb 1985 12:00:00 GMT",
    "role": ["copy", "author"],
    "location": {"city": "New York", "address": "4925 Lacross Road"},
    "_id": "50bf198338345b1c604faf31",
    "_updated": "Wed, 05 Dec 2012 09:53:07 GMT",
    "_created": "Wed, 05 Dec 2012 09:53:07 GMT",
    "_etag": "ec5e8200b8fa0596afe9ca71a87f23e71ca30e2d",
    },
    {
    "firstname": "John",
    ...
    },
    ]
    APPLICATION/JSON
    METAFIELDS ARE CUSTOMIZABLE
    

    View Slide

40. APPLICATION/XML
    
    
    Green
    Mark
    Wed, 05 Dec 2012 09:53:07 GMT
    author
    copy
    
    4925 Lacross Road
    New York
    
    <_id>50bf198338345b1c604faf31
    Wed, 05 Dec 2012 09:53:07 GMT
    Sat, 18 Jan 2014 09:16:10 GMT
    ec5e8200b8fa0596afe9ca71a87f23e71ca30e2d
    
    ...
    
    

    View Slide

41. HATEOAS
    HYPERMEDIA AS THE ENGINE OF APPLICATION STATE
    

    View Slide

42. HATEOAS
    {
    "_links": {
    "self": {
    "href": "/people",
    "title": "people" },
    "parent": {
    "href": "/",
    "title": "home"},
    "next": {
    "href": "/people?page=2",
    "title": "next page"},
    "last": {
    "href": "/people?page=10",
    "title": "last page"}
    }
    }
    

    View Slide

43. DOCUMENT VERSIONS
    ?version=3
    ?version=all
    ?version=diffs
    

    View Slide

44. FILE STORAGE
    FILES ARE STORED IN GRIDFS BY DEFAULT
    

    View Slide

45. FILE STORAGE / SETTINGS
    accounts = {
    "name": {"type": "string"},
    "pic": {"type": "media"},
    }
    

    View Slide

46. FILE UPLOAD
    $ curl F "name=doe" -F "pic=@profile.jpg"
    HTTP/1.1 200 OK
    $ curl -i
    HTTP/1.1 200 OK
    {
    "name": "john",
    "pic": "/9j/4QAYRXhpZgAASUkqAAgAAAAAAAAA…"
    }
    MULTIPART/DATA-FORM POST
    

    View Slide

47. FILE UPLOAD
    $ curl F "name=doe" -F "pic=@profile.jpg"
    HTTP/1.1 200 OK
    $ curl -i
    HTTP/1.1 200 OK
    {
    "name": "john",
    "pic": "/9j/4QAYRXhpZgAASUkqAAgAAAAAAAAA…"
    }
    FILES RETURNED AS BASE64 STRINGS
    

    View Slide

48. FILE STORAGE (WITH META)
    $ curl -i
    HTTP/1.1 200 OK
    {
    "name": "john",
    "pic": {
    "file": "/9j/4QAYRXhpZgAASUkqAAgAAAAAAAAA",
    "content_type": "image/jpeg",
    "name": "profile.jpg",
    "length": 8129
    }
    }
    EXTENDED_MEDIA_INFO: [‘content_type, ‘name’, ‘length’]
    

    View Slide

49. FILE STORAGE (DEDICATED ENDPOINT)
    $ curl -i
    HTTP/1.1 200 OK
    {
    "name": "john",
    "pic": "/media/profile.jpg"
    }
    }
    RETURN_MEDIA_AS_URL = True
    MEDIA_ENDPOINT = "media"
    

    View Slide

50. RATE LIMITING
    POWERED
    

    View Slide

51. RATE LIMITING / SETTINGS
    # Set rate limit on GET requests:
    # 1 requests 1 minute window (per client)
    RATE_LIMIT_GET = (1, 60)
    

    View Slide

52. RATE LIMITING / FIRST REQUEST
    $ curl -i
    HTTP/1.1 200 OK
    X-RateLimit-Limit: 1
    X-RateLimit-Remaining: 0
    X-RateLimit-Reset: 1390486659
    

    View Slide

53. RATE LIMITING / SECOND REQUEST
    $ curl -i
    HTTP/1.1 429 TOO MANY REQUESTS
    

    View Slide

54. CONDITIONAL REQUESTS
    ALLOW CLIENTS TO ONLY REQUEST NON-CACHED CONTENT
    DEMO
    

    View Slide

55. IF-MODIFIED-SINCE
    If-Modified-Since: Wed, 05 Dec 2012 09:53:07 GMT
    “Return document is changed since , or 304”
    

    View Slide

56. IF-NONE-MATCH
    If-None-Match:1234567890123456789012345678901234567890
    “Return data if it has changed (ETAG differs from mine), or 304”
    >
    

    View Slide

57. BULK INSERTS
    INSERT MULTIPLE DOCUMENTS WITH A SINGLE REQUEST
    DEMO
    

    View Slide

58. BULK INSERTS / REQUEST
    $ curl -d '
    [
    {
    "firstname": "barack",
    "lastname": "obama"
    },
    {
    "firstname": "mitt",
    "lastname": "romney"
    }
    ]'
    -H 'Content-Type: application/json'
    

    View Slide

59. BULK INSERTS / RESPONSE
    [
    {
    "_status": "OK",
    "_updated": "Thu, 22 Nov 2012 15:22:27 GMT",
    "_id": "50ae43339fa12500024def5b",
    "_etag": "749093d334ebd05cf7f2b7dbfb7868605578db2c"
    "_links": {"self": {"href": "", "title": "person"}}
    },
    {
    "_status": "OK",
    "_updated": "Thu, 22 Nov 2012 15:22:27 GMT",
    "_id": "50ae43339fa12500024def5c",
    "_etag": "62d356f623c7d9dc864ffa5facc47dced4ba6907"
    "_links": {"self": {"href": "", "title": "person"}}
    }
    ]
    COHERENCE MODE OFF: ONLY META FIELDS ARE RETURNED
    

    View Slide

60. BULK INSERTS / RESPONSE
    [
    {
    "_status": "OK",
    "_updated": "Thu, 22 Nov 2012 15:22:27 GMT",
    "_id": "50ae43339fa12500024def5b",
    "_etag": "749093d334ebd05cf7f2b7dbfb7868605578db2c"
    "_links": {"self": {"href": "", "title": }},
    "firstname": "barack",
    "lastname": "obama",
    …
    },
    {
    "_status": "OK",
    "_updated": "Thu, 22 Nov 2012 15:22:27 GMT",
    "_id": "50ae43339fa12500024def5c",
    "_etag": "62d356f623c7d9dc864ffa5facc47dced4ba6907"
    "_links": {"self": {"href": "", "title": "person"}}
    "firstname": "mitt",
    "lastname": "romney",
    …
    }
    ] COHERENCE MODE ON: ALL FIELDS RETURNED
    

    View Slide

61. DATA INTEGRITY AND
    CONCURRENCY CONTROL
    NO OVERWRITING OF ANY DOCUMENT WITH OBSOLETE VERSIONS
    DEMO
    

    View Slide

62. DATA INTEGRITY AND CONSISTENCY
    $ curl -X PATCH -i
    -d '{"firstname": "ronald"}'
    HTTP/1.1 428 PRECONDITION REQUIRED
    IF-MATCH MISSING
    

    View Slide

63. $ curl -X PATCH -i
    -H "If-Match: "
    -d '{"firstname": "ronald"}'
    HTTP/1.1 412 PRECONDITION FAILED
    ETAG MISMATCH
    DATA INTEGRITY AND CONSISTENCY
    

    View Slide

64. $ curl -X PATCH -i
    -H "If-Match: 206fb4a39815cc0ebf48b2b52d7…”
    -d '{"firstname": "ronald"}'
    HTTP/1.1 200 OK
    UPDATE ALLOWED AS CLIENT AND SERVER ETAG ARE MATCHING
    DATA INTEGRITY AND CONSISTENCY
    

    View Slide

65. DATA VALIDATION
    [
    {
    "_status": "ERR",
    "_issues": {"name": "value 'clinton' not unique"}
    },
    {
    "_status": "OK",
    "_updated": "Thu, 22 Nov 2012 15:22:27 GMT",
    "_id": "50ae43339fa12500024def5c",
    "_etag": "62d356f623c7d9dc864ffa5facc47dced4ba6907"
    "_links": {
    "self": {
    "href": "",
    "title": "person"
    }
    }
    }
    ]
    powered by
    Cerberus
    python-cerberus.org
    

    View Slide

66. GEOJSON
    SUPPORT AND VALIDATION FOR GEOJSON TYPES
    POINT, LINE-STRING, POLYGON, MULTI-POINT,
    MULTILINE-STRING, MULTI-POLYGON, GEOMETRICAL COLLECTION
    

    View Slide

67. AUTHENTICATION AND AUTHORIZATION
    BASIC, TOKEN AND HMAC AUTH SUPPORTED
    

    View Slide

68. RUNS ON ALL PYTHONS
    2.7 / 3.4+ / PYPY 3
    

    View Slide

69. AND MUCH MORE
    CORS / CACHE CONTROL / OPLOG / SOFT DELETES
    LOGGING / CUSTOM ID FIELDS / JSONP / MULTI-DB / AGGREGATION / ETC.
    

    View Slide

70. BSD LICENSED
    USE IN BOTH OPEN SOURCE AND COMMERCIAL
    

    View Slide

71. DEVELOPERS
    

    View Slide

72. CUSTOM DATA LAYERS
    BUILD YOUR OWN DATA LAYER
    

    View Slide

73. SQL ALCHEMY
    

    View Slide

74. ELASTICSERCH
    

    View Slide

75. AUTHENTICATION
    BASIC | TOKEN | HMAC
    DEMO
    

    View Slide

76. SECURITY AT A GLANCE
    • global authentication
    • endpoint authentication
    • public enpoints and methods
    • role based access control
    • user restricted resource access
    

    View Slide

77. THREE STEPS
    AUTH TUTORIAL
    

    View Slide

78. ONE.
    IMPORT BASE AUTH CLASS
    

    View Slide

79. TWO.
    OVERRIDE CHECK_AUTH() METHOD
    

    View Slide

80. THREE.
    PASS CUSTOM CLASS TO THE EVE APP
    

    View Slide

81. CUSTOM VALIDATION
    EXTEND THE BUILT-IN VALIDATION SYSTEM
    

    View Slide

82. CUSTOM VALIDATION
    • add custom data types
    • add custom validation logic
    • normalization
    

    View Slide

83. EVENT HOOKS
    PLUG CUSTOM ACTIONS INTO THE API LOOP
    DEMO
    

    View Slide

84. EVENT HOOKS AT A GLANCE
    • POST on_insert/on_inserted
    • GET on_fetch/on_fetched
    • PATCH on_update/on_updated
    • PUT on_replace/on_replaced
    • DELETE on_delete/on_deteled
    • on_pre_; on_post_
    

    View Slide

85. TRANSFORM INCOMING DOCUMENTS
    

    View Slide

86. EVE IS FLASK
    ‘CLASSIC’ FLASK POWER IS AT YOUR FINGERTIPS
    DEMO
    

    View Slide

87. FLASK AT YOUR FINGERTIPS
    from eve import Eve
    app = Eve()
    # add a regular Flask endpoint
    @app.route('/hello')
    def hello_world():
    return 'Hello World!'
    if __name__ == '__main__':
    app.run()
    

    View Slide

88. FLASK AT YOUR FINGERTIPS
    from eve import Eve
    from eve.auth import requires_auth
    app = Eve()
    # add Eve auth to Flask endpoint
    @app.route('/hello')
    @requires_auth('resource')
    def hello_world():
    return 'Hello World!'
    if __name__ == '__main__':
    app.run()
    

    View Slide

89. CUSTOM FILE STORAGE
    CUSTOM STORAGE CLASSES: S3 / FILE SYSTEM / ETC.
    

    View Slide

90. COMMUNITY
    

    View Slide

91. EVE-SWAGGER
    SWAGGER FOR EVE
    

    View Slide

92. EVE-SWAGGER
    

    View Slide

93. EVE-SQLALCHEMY
    SQL DATA LAYER FOR EVE
    

    View Slide

94. EVE-ELASTIC
    ELASTICSEARCH DATA LAYER FOR EVE REST FRAMEWORK
    

    View Slide

95. EVE.NET
    CROSS PLATFORM ASYNC C# CLIENT FOR .NET APPS
    PETR JASEK
    

    View Slide

96. EVE-AUTH-JWT
    OAUTH 2.0 JWT VALIDATION AUTHENTICATION
    THOMAS SILEO
    

    View Slide

97. FLASK-SENTINEL
    OAUTH2 SERVER
    THOMAS SILEO
    

    View Slide

98. {137: }
    Bryan Cattle Christoph Witzany Daniele Pizzolli
    dccrazyboy Dong Wei Ming Florian Rathgeber Francisco
    Corrales Morales Garrin Kimmell Gianfranco Palumbo Jaroslav
    Semančík Jean Boussier John Deng Jorge Puente Sarrín
    Josh Villbrandt Julien Barbot Ken Carpenter Kevin
    Bowrin Kracekumar Nicolas Bazire Nicolas Carlier Ondrej
    Slinták Petr Jašek Paul Doucet Robert Wlodarczyk Roberto Pasini
    Ronan Delacroix Roy Smith Ryan Shea Samuel Sutch
    Stanislav Heller Thomas Sileo Tomasz Jezierski Xavi Cubillas
    

    View Slide

99. PYTHON-EVE.ORG
    eve
    REST WEB API
    eve
    @nicolaiarocci / nicolaiarocci.com / eve@nicolaiarocci.com
    thanks!
    

    View Slide

100. View Slide