Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Eve - REST API for Humans™

Eve - REST API for Humans™

Introducing the Eve REST API Framework.

PyCon Web 2018, Munich
PyCon Belarus 2018, Minsk
PiterPy 2016, St. Petersburg
EuroPython 2014, Berlin
Python Meetup, Helsinki
PyCon Italy 2014, Florence
PyCon Sweden 2014, Stockholm
FOSDEM 2014, Brussels

Nicola Iarocci

February 02, 2014
Tweet

More Decks by Nicola Iarocci

Other Decks in Programming

Transcript

  1. REST API FOR HUMANS
    eve
    REST WEB API
    eve

    View full-size slide

  2. NICOLA IAROCCI
    Co-Founder @ CIR2000
    Lead Developer @ gestionaleamica.com
    Microsoft MVP
    MongoDB Master
    Open Source Author
    CoderDojo
    DevRomagna
    @nicolaiarocci / nicolaiarocci.com / [email protected]

    View full-size slide

  3. EVE IS POWERED BY

    View full-size slide

  4. RUN.PY
    from eve import Eve
    app = Eve()
    app.run()

    View full-size slide

  5. SETTINGS.PY
    # just a couple API endpoints with no
    # custom options and validation rules
    DOMAIN = {
    'people': {}
    'books': {}
    }

    View full-size slide

  6. LAUNCH THE API
    $ python run.py
    * Running on http://127.0.0.1:5000/

    View full-size slide

  7. TEST YOUR API
    $ curl -i http://localhost:5000/
    {
    "_items": [],
    "_links": {
    "self": {
    "href": "127.0.0.1:5000/people",
    "title": "people" },
    "parent": {
    "href": "127.0.0.1:5000",
    "title": "home"}
    }
    }

    View full-size slide

  8. $ curl -i http://localhost:5000/
    {
    "_items": [],
    "_links": {
    "self": {
    "href": "127.0.0.1:5000/people",
    "title": "people" },
    "parent": {
    "href": "127.0.0.1:5000",
    "title": "home"}
    }
    }
    HATEOAS AT WORK HERE
    TEST YOUR API

    View full-size slide

  9. $ curl -i http://localhost:5000/
    {
    "_items": [],
    "_links": {
    "self": {
    "href": "127.0.0.1:5000/people",
    "title": "people" },
    "parent": {
    "href": "127.0.0.1:5000",
    "title": "home"}
    }
    }
    TEST YOUR API
    CLIENTS CAN EXPLORE THE API PROGRAMMATICALLY

    View full-size slide

  10. $ curl -i http://localhost:5000/
    {
    "_items": [],
    "_links": {
    "self": {
    "href": "127.0.0.1:5000/people",
    "title": "people" },
    "parent": {
    "href": "127.0.0.1:5000",
    "title": "home"}
    }
    }
    TEST YOUR API
    AND EVENTUALLY FILL THEIR UI

    View full-size slide

  11. $ curl -i http://localhost:5000/
    {
    "_items": [],
    "_links": {
    "self": {
    "href": "127.0.0.1:5000/people",
    "title": "people" },
    "parent": {
    "href": "127.0.0.1:5000",
    "title": "home"}
    }
    }
    EMTPY RESOURCE AS WE DIDN’T CONNECT ANY DATASOURCE
    TEST YOUR API

    View full-size slide

  12. SETTINGS.PY
    # connect to mongo
    MONGO_HOST = 'localhost'
    MONGO_PORT = 27017
    MONGO_USERNAME = 'user'
    MONGO_PASSWORD = 'pw'
    MONGO_DBNAME = 'apitest'
    # or (better):
    MONGO_URI = ‘mongodb://user:pw@localhost:27017/apitest'

    View full-size slide

  13. # add fields and validation rules for 'people' endpoint
    DOMAIN['people']['schema'] = {
    'name': {
    'type': 'string',
    'maxlength': 50,
    'unique': True}
    'email': {
    'type': 'string',
    'regex': '^\S+@\S+$'},
    'location': {
    'type': 'dict',
    'schema': {
    'address': {'type': 'string'},
    'city': {'type': 'string'}}},
    'born': {'type': 'datetime'}}
    USE BETTER REGEX IN PRODUCTION
    SETTINGS.PY

    View full-size slide

  14. SETTINGS.PY
    # enable writes. default is ['GET']
    # /people
    RESOURCE_METHODS = ['GET','POST']
    # /people/
    ITEM_METHODS = ['GET','PATCH','PUT','DELETE']

    View full-size slide

  15. # enable writes. default is ['GET']
    # /people
    RESOURCE_METHODS = ['GET','POST']
    # /people/
    ITEM_METHODS = ['GET','PATCH','PUT','DELETE']
    SETTINGS.PY
    ADD/CREATE ONE OR MORE ITEMS

    View full-size slide

  16. # enable writes. default is ['GET']
    # /people
    RESOURCE_METHODS = ['GET','POST']
    # /people/
    ITEM_METHODS = ['GET','PATCH','PUT','DELETE']
    SETTINGS.PY
    UPDATE DOCUMENT

    View full-size slide

  17. # enable writes. default is ['GET']
    # /people
    RESOURCE_METHODS = ['GET','POST']
    # /people/
    ITEM_METHODS = ['GET','PATCH','PUT','DELETE']
    SETTINGS.PY
    REPLACE DOCUMENT

    View full-size slide

  18. # enable writes. default is ['GET']
    # /people
    RESOURCE_METHODS = ['GET','POST']
    # /people/
    ITEM_METHODS = ['GET','PATCH','PUT','DELETE']
    SETTINGS.PY
    YOU GUESSED IT

    View full-size slide

  19. SETTINGS.PY
    # a few additional configuration options
    DOMAIN['people'].update(
    {
    'item_title': 'person',
    'cache_control': 'max-age=10,must-revalidate',
    'cache_expires': 10,
    'additional_lookup': {
    'url': 'regex("[\w]+")',
    'field': 'name'
    }
    )

    View full-size slide

  20. MONGO FILTERS
    ?where={“lastname”: “Doe”}

    View full-size slide

  21. PYTHON FILTERS
    ?where=lastname==“Doe”

    View full-size slide

  22. SORTING
    ?sort=-total
    SORT BY ‘TOTAL’, DESCENDING
    DEMO

    View full-size slide

  23. SORTING
    ?sort=[(“total”: -1)]
    SAME, MONGODB-STYLE

    View full-size slide

  24. PAGINATION
    ?max_results=20&page=2
    MAX 20 RESULTS/PAGE; PAGE 2

    View full-size slide

  25. PROJECTIONS
    ?projection={"avatar": 0}
    RETURN ALL FIELDS BUT ‘AVATAR’

    View full-size slide

  26. PROJECTIONS
    ?projection={"lastname": 1}
    ONLY RETURN ‘LASTNAME’

    View full-size slide

  27. EMBEDDED RESOURCES
    ?embedded={"author": 1}
    DEMO

    View full-size slide

  28. NOT EMBEDDED
    $ curl -i
    HTTP/1.1 200 OK
    {
    "title": "Book Title",
    "description": "book description",
    "author": "52da465a5610320002660f94"
    }
    RAW FOREIGN KEY

    View full-size slide

  29. EMBEDDED
    $ curl -i ?embedded={"author": 1}
    HTTP/1.1 200 OK
    {
    "title": "Book Title",
    "description": "book description",
    "author": {
    "firstname": "Mark",
    "lastname": "Green",
    }
    }
    REQUEST EMBEDDED AUTHOR

    View full-size slide

  30. EMBEDDED
    $ curl -i ?embedded={"author": 1}
    HTTP/1.1 200 OK
    {
    "title": "Book Title",
    "description": "book description",
    "author": {
    "firstname": "Mark",
    "lastname": "Green",
    }
    }
    EMBEDDED DOCUMENT

    View full-size slide

  31. JSON AND XML
    SERIALIZATION FOR ALL RESPONSES
    DEMO

    View full-size slide

  32. APPLICATION/JSON
    [
    {
    "firstname": "Mark",
    "lastname": "Green",
    "born": "Sat, 23 Feb 1985 12:00:00 GMT",
    "role": ["copy", "author"],
    "location": {"city": "New York", "address": "4925 Lacross Road"},
    "_id": "50bf198338345b1c604faf31",
    "_updated": "Wed, 05 Dec 2012 09:53:07 GMT",
    "_created": "Wed, 05 Dec 2012 09:53:07 GMT",
    "_etag": "ec5e8200b8fa0596afe9ca71a87f23e71ca30e2d",
    },
    {
    "firstname": "John",
    ...
    },
    ]

    View full-size slide

  33. [
    {
    "firstname": "Mark",
    "lastname": "Green",
    "born": "Sat, 23 Feb 1985 12:00:00 GMT",
    "role": ["copy", "author"],
    "location": {"city": "New York", "address": "4925 Lacross Road"},
    "_id": "50bf198338345b1c604faf31",
    "_updated": "Wed, 05 Dec 2012 09:53:07 GMT",
    "_created": "Wed, 05 Dec 2012 09:53:07 GMT",
    "_etag": "ec5e8200b8fa0596afe9ca71a87f23e71ca30e2d",
    },
    {
    "firstname": "John",
    ...
    },
    ]
    APPLICATION/JSON
    METAFIELDS ARE CUSTOMIZABLE

    View full-size slide

  34. APPLICATION/XML


    Green
    Mark
    Wed, 05 Dec 2012 09:53:07 GMT
    author
    copy

    4925 Lacross Road
    New York

    <_id>50bf198338345b1c604faf31
    Wed, 05 Dec 2012 09:53:07 GMT
    Sat, 18 Jan 2014 09:16:10 GMT
    ec5e8200b8fa0596afe9ca71a87f23e71ca30e2d

    ...

    View full-size slide

  35. HATEOAS
    HYPERMEDIA AS THE ENGINE OF APPLICATION STATE

    View full-size slide

  36. HATEOAS
    {
    "_links": {
    "self": {
    "href": "/people",
    "title": "people" },
    "parent": {
    "href": "/",
    "title": "home"},
    "next": {
    "href": "/people?page=2",
    "title": "next page"},
    "last": {
    "href": "/people?page=10",
    "title": "last page"}
    }
    }

    View full-size slide

  37. DOCUMENT VERSIONS
    ?version=3
    ?version=all
    ?version=diffs

    View full-size slide

  38. FILE STORAGE
    FILES ARE STORED IN GRIDFS BY DEFAULT

    View full-size slide

  39. FILE STORAGE / SETTINGS
    accounts = {
    "name": {"type": "string"},
    "pic": {"type": "media"},
    }

    View full-size slide

  40. FILE UPLOAD
    $ curl F "name=doe" -F "[email protected]"
    HTTP/1.1 200 OK
    $ curl -i
    HTTP/1.1 200 OK
    {
    "name": "john",
    "pic": "/9j/4QAYRXhpZgAASUkqAAgAAAAAAAAA…"
    }
    MULTIPART/DATA-FORM POST

    View full-size slide

  41. FILE UPLOAD
    $ curl F "name=doe" -F "[email protected]"
    HTTP/1.1 200 OK
    $ curl -i
    HTTP/1.1 200 OK
    {
    "name": "john",
    "pic": "/9j/4QAYRXhpZgAASUkqAAgAAAAAAAAA…"
    }
    FILES RETURNED AS BASE64 STRINGS

    View full-size slide

  42. FILE STORAGE (WITH META)
    $ curl -i
    HTTP/1.1 200 OK
    {
    "name": "john",
    "pic": {
    "file": "/9j/4QAYRXhpZgAASUkqAAgAAAAAAAAA",
    "content_type": "image/jpeg",
    "name": "profile.jpg",
    "length": 8129
    }
    }
    EXTENDED_MEDIA_INFO: [‘content_type, ‘name’, ‘length’]

    View full-size slide

  43. FILE STORAGE (DEDICATED ENDPOINT)
    $ curl -i
    HTTP/1.1 200 OK
    {
    "name": "john",
    "pic": "/media/profile.jpg"
    }
    }
    RETURN_MEDIA_AS_URL = True
    MEDIA_ENDPOINT = "media"

    View full-size slide

  44. RATE LIMITING
    POWERED

    View full-size slide

  45. RATE LIMITING / SETTINGS
    # Set rate limit on GET requests:
    # 1 requests 1 minute window (per client)
    RATE_LIMIT_GET = (1, 60)

    View full-size slide

  46. RATE LIMITING / FIRST REQUEST
    $ curl -i
    HTTP/1.1 200 OK
    X-RateLimit-Limit: 1
    X-RateLimit-Remaining: 0
    X-RateLimit-Reset: 1390486659

    View full-size slide

  47. RATE LIMITING / SECOND REQUEST
    $ curl -i
    HTTP/1.1 429 TOO MANY REQUESTS

    View full-size slide

  48. CONDITIONAL REQUESTS
    ALLOW CLIENTS TO ONLY REQUEST NON-CACHED CONTENT
    DEMO

    View full-size slide

  49. IF-MODIFIED-SINCE
    If-Modified-Since: Wed, 05 Dec 2012 09:53:07 GMT
    “Return document is changed since , or 304”

    View full-size slide

  50. IF-NONE-MATCH
    If-None-Match:1234567890123456789012345678901234567890
    “Return data if it has changed (ETAG differs from mine), or 304”
    >

    View full-size slide

  51. BULK INSERTS
    INSERT MULTIPLE DOCUMENTS WITH A SINGLE REQUEST
    DEMO

    View full-size slide

  52. BULK INSERTS / REQUEST
    $ curl -d '
    [
    {
    "firstname": "barack",
    "lastname": "obama"
    },
    {
    "firstname": "mitt",
    "lastname": "romney"
    }
    ]'
    -H 'Content-Type: application/json'

    View full-size slide

  53. BULK INSERTS / RESPONSE
    [
    {
    "_status": "OK",
    "_updated": "Thu, 22 Nov 2012 15:22:27 GMT",
    "_id": "50ae43339fa12500024def5b",
    "_etag": "749093d334ebd05cf7f2b7dbfb7868605578db2c"
    "_links": {"self": {"href": "", "title": "person"}}
    },
    {
    "_status": "OK",
    "_updated": "Thu, 22 Nov 2012 15:22:27 GMT",
    "_id": "50ae43339fa12500024def5c",
    "_etag": "62d356f623c7d9dc864ffa5facc47dced4ba6907"
    "_links": {"self": {"href": "", "title": "person"}}
    }
    ]
    COHERENCE MODE OFF: ONLY META FIELDS ARE RETURNED

    View full-size slide

  54. BULK INSERTS / RESPONSE
    [
    {
    "_status": "OK",
    "_updated": "Thu, 22 Nov 2012 15:22:27 GMT",
    "_id": "50ae43339fa12500024def5b",
    "_etag": "749093d334ebd05cf7f2b7dbfb7868605578db2c"
    "_links": {"self": {"href": "", "title": }},
    "firstname": "barack",
    "lastname": "obama",

    },
    {
    "_status": "OK",
    "_updated": "Thu, 22 Nov 2012 15:22:27 GMT",
    "_id": "50ae43339fa12500024def5c",
    "_etag": "62d356f623c7d9dc864ffa5facc47dced4ba6907"
    "_links": {"self": {"href": "", "title": "person"}}
    "firstname": "mitt",
    "lastname": "romney",

    }
    ] COHERENCE MODE ON: ALL FIELDS RETURNED

    View full-size slide

  55. DATA INTEGRITY AND
    CONCURRENCY CONTROL
    NO OVERWRITING OF ANY DOCUMENT WITH OBSOLETE VERSIONS
    DEMO

    View full-size slide

  56. DATA INTEGRITY AND CONSISTENCY
    $ curl -X PATCH -i
    -d '{"firstname": "ronald"}'
    HTTP/1.1 428 PRECONDITION REQUIRED
    IF-MATCH MISSING

    View full-size slide

  57. $ curl -X PATCH -i
    -H "If-Match: "
    -d '{"firstname": "ronald"}'
    HTTP/1.1 412 PRECONDITION FAILED
    ETAG MISMATCH
    DATA INTEGRITY AND CONSISTENCY

    View full-size slide

  58. $ curl -X PATCH -i
    -H "If-Match: 206fb4a39815cc0ebf48b2b52d7…”
    -d '{"firstname": "ronald"}'
    HTTP/1.1 200 OK
    UPDATE ALLOWED AS CLIENT AND SERVER ETAG ARE MATCHING
    DATA INTEGRITY AND CONSISTENCY

    View full-size slide

  59. DATA VALIDATION
    [
    {
    "_status": "ERR",
    "_issues": {"name": "value 'clinton' not unique"}
    },
    {
    "_status": "OK",
    "_updated": "Thu, 22 Nov 2012 15:22:27 GMT",
    "_id": "50ae43339fa12500024def5c",
    "_etag": "62d356f623c7d9dc864ffa5facc47dced4ba6907"
    "_links": {
    "self": {
    "href": "",
    "title": "person"
    }
    }
    }
    ]
    powered by
    Cerberus
    python-cerberus.org

    View full-size slide

  60. GEOJSON
    SUPPORT AND VALIDATION FOR GEOJSON TYPES
    POINT, LINE-STRING, POLYGON, MULTI-POINT,
    MULTILINE-STRING, MULTI-POLYGON, GEOMETRICAL COLLECTION

    View full-size slide

  61. AUTHENTICATION AND AUTHORIZATION
    BASIC, TOKEN AND HMAC AUTH SUPPORTED

    View full-size slide

  62. RUNS ON ALL PYTHONS
    2.7 / 3.4+ / PYPY 3

    View full-size slide

  63. AND MUCH MORE
    CORS / CACHE CONTROL / OPLOG / SOFT DELETES
    LOGGING / CUSTOM ID FIELDS / JSONP / MULTI-DB / AGGREGATION / ETC.

    View full-size slide

  64. BSD LICENSED
    USE IN BOTH OPEN SOURCE AND COMMERCIAL

    View full-size slide

  65. CUSTOM DATA LAYERS
    BUILD YOUR OWN DATA LAYER

    View full-size slide

  66. ELASTICSERCH

    View full-size slide

  67. AUTHENTICATION
    BASIC | TOKEN | HMAC
    DEMO

    View full-size slide

  68. SECURITY AT A GLANCE
    • global authentication
    • endpoint authentication
    • public enpoints and methods
    • role based access control
    • user restricted resource access

    View full-size slide

  69. THREE STEPS
    AUTH TUTORIAL

    View full-size slide

  70. ONE.
    IMPORT BASE AUTH CLASS

    View full-size slide

  71. TWO.
    OVERRIDE CHECK_AUTH() METHOD

    View full-size slide

  72. THREE.
    PASS CUSTOM CLASS TO THE EVE APP

    View full-size slide

  73. CUSTOM VALIDATION
    EXTEND THE BUILT-IN VALIDATION SYSTEM

    View full-size slide

  74. CUSTOM VALIDATION
    • add custom data types
    • add custom validation logic
    • normalization

    View full-size slide

  75. EVENT HOOKS
    PLUG CUSTOM ACTIONS INTO THE API LOOP
    DEMO

    View full-size slide

  76. EVENT HOOKS AT A GLANCE
    • POST on_insert/on_inserted
    • GET on_fetch/on_fetched
    • PATCH on_update/on_updated
    • PUT on_replace/on_replaced
    • DELETE on_delete/on_deteled
    • on_pre_; on_post_

    View full-size slide

  77. TRANSFORM INCOMING DOCUMENTS

    View full-size slide

  78. EVE IS FLASK
    ‘CLASSIC’ FLASK POWER IS AT YOUR FINGERTIPS
    DEMO

    View full-size slide

  79. FLASK AT YOUR FINGERTIPS
    from eve import Eve
    app = Eve()
    # add a regular Flask endpoint
    @app.route('/hello')
    def hello_world():
    return 'Hello World!'
    if __name__ == '__main__':
    app.run()

    View full-size slide

  80. FLASK AT YOUR FINGERTIPS
    from eve import Eve
    from eve.auth import requires_auth
    app = Eve()
    # add Eve auth to Flask endpoint
    @app.route('/hello')
    @requires_auth('resource')
    def hello_world():
    return 'Hello World!'
    if __name__ == '__main__':
    app.run()

    View full-size slide

  81. CUSTOM FILE STORAGE
    CUSTOM STORAGE CLASSES: S3 / FILE SYSTEM / ETC.

    View full-size slide

  82. EVE-SWAGGER
    SWAGGER FOR EVE

    View full-size slide

  83. EVE-SQLALCHEMY
    SQL DATA LAYER FOR EVE

    View full-size slide

  84. EVE-ELASTIC
    ELASTICSEARCH DATA LAYER FOR EVE REST FRAMEWORK

    View full-size slide

  85. EVE.NET
    CROSS PLATFORM ASYNC C# CLIENT FOR .NET APPS
    PETR JASEK

    View full-size slide

  86. EVE-AUTH-JWT
    OAUTH 2.0 JWT VALIDATION AUTHENTICATION
    THOMAS SILEO

    View full-size slide

  87. FLASK-SENTINEL
    OAUTH2 SERVER
    THOMAS SILEO

    View full-size slide

  88. {137: }
    Bryan Cattle Christoph Witzany Daniele Pizzolli
    dccrazyboy Dong Wei Ming Florian Rathgeber Francisco
    Corrales Morales Garrin Kimmell Gianfranco Palumbo Jaroslav
    Semančík Jean Boussier John Deng Jorge Puente Sarrín
    Josh Villbrandt Julien Barbot Ken Carpenter Kevin
    Bowrin Kracekumar Nicolas Bazire Nicolas Carlier Ondrej
    Slinták Petr Jašek Paul Doucet Robert Wlodarczyk Roberto Pasini
    Ronan Delacroix Roy Smith Ryan Shea Samuel Sutch
    Stanislav Heller Thomas Sileo Tomasz Jezierski Xavi Cubillas

    View full-size slide

  89. PYTHON-EVE.ORG
    eve
    REST WEB API
    eve
    @nicolaiarocci / nicolaiarocci.com / [email protected]
    thanks!

    View full-size slide