Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CouchDB, PouchDB and Offline-Tolerant Apps

CouchDB, PouchDB and Offline-Tolerant Apps

From Dutch PHP Conference

Lorna Mitchell

July 01, 2017
Tweet

More Decks by Lorna Mitchell

Other Decks in Technology

Transcript

  1. CouchDB, PouchDB
    and Offline-Tolerant
    Apps
    Lorna Mitchell, IBM

    View full-size slide

  2. NoSQL?
    @lornajane

    View full-size slide

  3. Document Databases
    Store collections of schemaless documents
    @lornajane

    View full-size slide

  4. CouchDB
    Cluster Of Unreliable Commodity Hardware
    • HTTP API
    • JSON data format
    • MapReduce views in JavaScript
    • Mango is JSON-format also
    CouchDB 2.0 has clustering and sharding features
    @lornajane

    View full-size slide

  5. CouchDB HTTP API
    Hello, CouchDB
    ~ $ curl http://localhost:5984
    {"couchdb":"Welcome","version":"2.0.0",
    "vendor":{"name":"The Apache Software Foundation"}}
    List databases
    ~ $ curl http://localhost:5984/_all_dbs
    []
    @lornajane

    View full-size slide

  6. CouchDB HTTP API
    New database
    ~ $ curl -X PUT http://localhost:5984/shopping
    {"ok":true}
    List databases again
    ~ $ curl http://localhost:5984/_all_dbs
    ["shopping"]
    @lornajane

    View full-size slide

  7. Curl and Not-Curl
    • love curl? (https://curl.haxx.se/)
    • try jq (https://stedolan.github.io/jq/)
    • hate curl? Try one of these
    • http-console https://github.com/cloudhead/http-console
    • HTTPie https://httpie.org/
    • Postman https://www.getpostman.com/
    @lornajane

    View full-size slide

  8. Try http-console
    Connect:
    $ http-console http://localhost:5984
    Set database in path (to avoid typing it lots):
    http://localhost:5984> /shopping
    Set JSON header:
    http://localhost:5984/shopping> Content-Type: application/json
    @lornajane

    View full-size slide

  9. Try http-console
    Create item:
    http://localhost:5984/shopping> PUT /hat
    ... {"colour": "white"}
    HTTP/1.1 201 Created
    Location: http://localhost/shopping/hat
    Etag: "1-12cb52ad7565b6248c4dbe09f1377e2b"
    {
    ok: true,
    id: 'hat',
    rev: '1-12cb52ad7565b6248c4dbe09f1377e2b'
    }
    @lornajane

    View full-size slide

  10. Try http-console
    Update item:
    http://localhost:5984/shopping> PUT /hat
    ... {"colour": "blue"}
    HTTP/1.1 409 Conflict
    { error: 'conflict', reason: 'Document update conflict.' }
    To update, include the revision ...
    @lornajane

    View full-size slide

  11. Try http-console
    Update item:
    http://localhost:5984/shopping> PUT /hat
    ... {"_rev": "1-12cb52ad7565b6248c4dbe09f1377e2b", "colour": "blue
    HTTP/1.1 201 Created
    Location: http://localhost/shopping/hat
    Etag: "2-15b01d3c5cd2ed475e6ce4cf84b51990"
    {
    ok: true,
    id: 'hat',
    rev: '2-15b01d3c5cd2ed475e6ce4cf84b51990'
    }
    @lornajane

    View full-size slide

  12. Fauxton
    Friendly web interface
    @lornajane

    View full-size slide

  13. Fauxton
    @lornajane

    View full-size slide

  14. Fauxton
    @lornajane

    View full-size slide

  15. CouchDB: Lovely Doc DB
    I could stop here:
    • JSON format
    • HTTP interface and nice web UI
    • Scales well
    • Modern, performant document database
    @lornajane

    View full-size slide

  16. Changes Feed
    A feed containing all database changes.
    GET /_changes
    @lornajane

    View full-size slide

  17. Replication
    @lornajane

    View full-size slide

  18. Replication
    • Replication can be in either direction - or both
    • Can be one-off, or continuous
    • Other CouchDB-compatible storage also exists
    • e.g. PouchDB, a JavaScript implementation
    @lornajane

    View full-size slide

  19. Conflicts
    Change docs in both places, replicate again:
    87bf-bluemix.cloudant.com:443/shopping> GET /hat?conflicts=true
    {
    _id: 'hat',
    _rev: '4-ecbc38075f9a8535c123e523519613b9',
    colour: 'green',
    in_stock: 1,
    _conflicts: [ '3-0bb689d59034fb769d99dcf697ae2de7' ]
    }
    CouchDB will always choose the same "winning" doc
    @lornajane

    View full-size slide

  20. Conflicts
    Fetch the "losing" doc(s) with ?rev= parameter
    87bf-bluemix.cloudant.com:443/shopping> GET /hat?rev=3-0bb689d5903
    {
    _id: 'hat',
    _rev: '3-0bb689d59034fb769d99dcf697ae2de7',
    colour: 'blue',
    size: 'L'
    }
    CouchDB doesn't store old revisions forever
    @lornajane

    View full-size slide

  21. Mango
    (sample data: https://www.ibm.com/communities/analy
    tics/watson-analytics-blog/sales-products-sample-data/)
    @lornajane

    View full-size slide

  22. Mango: CouchDB Queries
    Mango is a mongo-like query language, useful for ad-hoc
    querying
    It is a JSON structure containing:
    • Selector: the criteria to match records on
    • Fields: which fields to return
    • Sort: what order you'd like that in (use with Skip)
    • Limit: how many records (default = 25)
    @lornajane

    View full-size slide

  23. Mango: CouchDB Queries
    @lornajane

    View full-size slide

  24. Mango: Example Query
    Use a query like this with the _find endpoint
    {
    "selector": {
    "Year": {"$eq": "2012"}
    },
    "fields": ["Quarter", "Product line"],
    "limit": 5
    }
    @lornajane

    View full-size slide

  25. Mango: Example Query
    $ curl -X POST -H Content-Type:application/json \
    http://localhost:5984/products/_find --data @mango.json
    {"warning":"no matching index found, create an index to optimize q
    "docs":[
    {"Quarter":"Q1 2012","Product line":"Mountaineering Equipment"},
    {"Quarter":"Q1 2012","Product line":"Mountaineering Equipment"},
    {"Quarter":"Q1 2012","Product line":"Mountaineering Equipment"},
    {"Quarter":"Q1 2012","Product line":"Mountaineering Equipment"},
    {"Quarter":"Q1 2012","Product line":"Mountaineering Equipment"}
    ]}
    @lornajane

    View full-size slide

  26. Mango: Indexes
    Describe the index in JSON, then use the _index endpoint
    {
    "index": {
    "fields": ["Year"]
    },
    "name": "Year"
    }
    @lornajane

    View full-size slide

  27. Mango: Indexes
    $ curl -X POST -H Content-Type:application/json \
    http://localhost:5984/products/_index --data @index.json
    {
    "result": "created",
    "id": "_design/e9b54f2ac34b8823ccbe8aaf6f406d464f50f521",
    "name": "Year"
    }
    Check which indexes are used by putting _explain where the
    _find normally goes!
    @lornajane

    View full-size slide

  28. Views
    @lornajane

    View full-size slide

  29. Views
    • Written in Javascript
    • Use MapReduce
    • The map results are stored
    • Can be used either for filtering, or for aggregation
    • Geospatial features also available (not in today's talk, sorry)
    @lornajane

    View full-size slide

  30. MapReduce Primer: Map
    • Examine each document, "emit" 0+ keys/value pairs
    • Scales well because each document is independent
    • To filter a collection of documents, use map step only
    @lornajane

    View full-size slide

  31. MapReduce Primer: Map
    @lornajane

    View full-size slide

  32. MapReduce Primer: Map
    @lornajane

    View full-size slide

  33. MapReduce Primer: Map
    @lornajane

    View full-size slide

  34. MapReduce Primer: Map
    @lornajane

    View full-size slide

  35. MapReduce Primer: Reduce
    @lornajane

    View full-size slide

  36. MapReduce Primer: Reduce
    • "Reduce" values in batches with the same key
    • CouchDB has useful built in functions for most things
    • Use reduce step when you want aggregate data
    • (SQL equivalent: a query with GROUP BY)
    @lornajane

    View full-size slide

  37. Views Example
    @lornajane

    View full-size slide

  38. CouchDB Sync Enables
    OfflineFirst Apps
    @lornajane

    View full-size slide

  39. PouchDB
    • https://pouchdb.com/
    • A database that your client-side javascript can use
    • Can also sync to CouchDB (-compatible) databases
    • https://tinyurl.com/fauxton-pouchdb
    • UI for PouchDB in your browser
    @lornajane

    View full-size slide

  40. PouchDB
    @lornajane

    View full-size slide

  41. PouchDB in Action
    In index.html:


    shopping.js is where my client-side JavaScript lives
    Code is here:
    https://github.com/lornajane/robust-shopping-list
    @lornajane

    View full-size slide

  42. PouchDB: Replication
    var db = new PouchDB('shopping');
    var remoteDB = new PouchDB('http://localhost:5984/shopping');
    window.onload = function() {
    db.sync(remoteDB, { live: true, retry: true }
    ).on('change', function (change) {
    return getItemList().then(function (contents) {
    document.getElementById('itemList').innerHTML = conten
    })
    }).on('active', function (info) {
    return getItemList().then(function (contents) {
    document.getElementById('itemList').innerHTML = conten
    });
    });
    @lornajane

    View full-size slide

  43. PouchDB in Action
    OfflineFirst for robust web applications
    @lornajane

    View full-size slide

  44. CouchDB: Scalable NoSQL
    Document database: and so much more
    @lornajane

    View full-size slide

  45. Questions?
    Feedback please! https://joind.in/talk/0746b
    Resources:
    • https://lornajane.net
    • https://couchdb.apache.org/
    • https://pouchdb.com/
    • https://offlinefirst.org
    If you liked it, tweet: @CouchDB, @PouchDB
    @lornajane

    View full-size slide

  46. Emergency Backup Slides
    @lornajane

    View full-size slide