Hit the Flask and Get some REST

Learn design decisions that lead Cloudant to rewrite the supporting API layer to all its backend systems in Python using the Flask microframework to create a unified, RESTFUL API.

Transcript

1. •
   

   View Slide

2. 
   
   
   
   

   View Slide

3. View Slide

4. 
   
   
   
   
   

   View Slide

5. View Slide

6. View Slide

7. View Slide

8. View Slide

9. View Slide

10. View Slide

11. {
    "data-layer":{
    "bytes":144421583215226,
    "I/Os":8076267344802,
    "databases":3463156
    }
    }
    

    View Slide

12. View Slide

13. View Slide

14. View Slide

15. View Slide

16. c:\users\joan\desktop>clou user info wohali6
    Username: wohali6
    Email: joant@ieee.org
    Cluster(s): meritage (was on lagoon until 2012-07-19T13:45:30,
    meritage until 2012-07-19T13:45:58,
    lagoon until 2012-08-30T17:43:50)
    Plan: oxygen
    Date: 2012-06-07 20:25:30
    Data size: 10.1 MB (disk) 4.0 KB (real)
    View size: 16.8 KB (disk) 436.0 B (real)
    Total size: 10.1 MB (disk) 4.4 KB (real)
    

    View Slide

17. c:\users\joan\desktop>clou user move wohali6 julep
    >>> Moving data from meritage to julep...
    Databases to replicate: animaldb, oakland_assessor.
    Replicating db animaldb...
    animaldb 9% |---- | ETA: 0:00:26
    Document count matches.
    Checking for auth doc...
    Replicating db oakland_assessor...
    oakland_a 9% |---- | ETA: 0:03:58
    Document count matches.
    Checking for auth doc...
    >>> Updating user’s record to point to new cluster...
    >>> Refreshing load balancers...
    Refreshing 4 LBs asynchronously, please wait...
    Host Code
    ===============================================
    lb1.julep.cloudant.com: 200
    lb2.julep.cloudant.com: 200
    lb1.meritage.cloudant.com: 200
    lb2.meritage.cloudant.com: 200
    >>> Updating DNS record...
    Using token <…>
    u'{"status": "success", "data": {"zone": "cloudant.com", "ttl": 300, "fqdn": "wohali6.cloudant.com",
    "record_type": "CNAME", "rdata"
    : {"cname": "julep.cloudant.com."}, "record_id": 0}, "job_id": 205794996, "msgs": [{"INFO": "update: Record
    updated", "SOURCE": "BLL
    ", "ERR_CD": null, "LVL": "INFO"}]}'
    u'{"status": "success", "data": {"zone_type": "Primary", "serial_style": "increment", "serial": 27877,
    "zone": "cloudant.com"}, "job
    _id": 205795008, "msgs": [{"INFO": "publish: cloudant.com published", "SOURCE": "BLL", "ERR_CD": null, "LVL":
    "INFO"}]}'
    

    View Slide

18. View Slide

19. 
    
    
    
    
    

    View Slide

20. 
    
    
    
    
    
    

    View Slide

21. 
    
    
    
    

    View Slide

22. 
    
    

    View Slide

23. View Slide

24. View Slide

25. 
    
    
    
    
    

    View Slide

26. 
    
    
    
    
    
    
    

    View Slide

27. @users.route('/service', methods=['GET', 'POST'])
    def service(parameter):
    ...
    return jsonify(thingy)
    

    View Slide

28. from flask import current_app
    import utils
    class Tickets(MethodView):
    """
    This is an interface into our support system. It allows users to
    query open tickets and create new tickets via a RESTful API.
    """
    decorators = [utils.user_required]
    def __init__(self):
    """
    'Connect' to fogbugz, wither the real one or a mock object.
    """
    user = current_app.config['FB_USER']
    passw = current_app.config['FB_PASS']
    if current_app.config['BACKEND'] == "Mock":
    import mock.fogbugz as fogbugz
    assert fogbugz # silence pyflakes
    else:
    import fogbugz
    self.fogbugz = fogbugz.FogBugz("https://cloudant.fogbugz.com")
    self.fogbugz.logon(user, passw)
    

    View Slide

29. from flask import current_app, request, jsonify, session
    class Tickets(MethodView):
    …
    def get(self):
    """
    List open tickets for the user. Response is a json structure like:
    {"tickets":[{"id": TICKET_ID, 'title': TICKET_TITLE}, ...]}
    User name is taken from the session, there must be a valid session to
    call the api.
    """
    query = '%s status:active OrderBy:Opened' % session['username'],
    resp = self.fogbugz.search(q=query, cols='ixBug,sTitle')
    cases = resp.cases.childGenerator()
    tickets = {'tickets': []}
    limit = int(request.args.get('limit', 5))
    current_app.logger.debug(limit)
    for case in itertools.islice(cases, limit):
    tickets['tickets'].append({'id': int(case['ixbug']),
    'title': case.stitle.string})
    return jsonify(tickets)
    

    View Slide

30. 
    
    
    $SYSTEM
    
    

    View Slide

31. 
    
    
    
    
    
    
    

    View Slide

32. View Slide

33. 
    
    
    
    

    View Slide

34. 
    
    
    

    View Slide

35. View Slide

36. 
    
    
    

    View Slide

37. “hi chris\n”
    42
    [“hi chris\n”, 42]
    { “saying”: “hi chris\n”,
    “lucky_number”: 42 }
    

    View Slide

38. US “Node”
    Single-tenant
    cluster
    Horizontally Scalable DB
    • Fault tolerant
    • Always consistent
    • Schemaless (NoSQL)
    • Automatic sharding
    • Distributed, parallel analytics
    • Incremental, chainable
    MapReduce
    • Full-text search
    Multi-tenant
    cluster
    PUT {document}
    Edge Database Cluster
    Disconnected Devices
    Japan
    Filtered
    Replication
    & Sync
    Secondary Data Centers
    (for DR & distributed access)
    EU-NL
    

    View Slide