Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Taking Cloud Databases to the Edge at All Things Open

Taking Cloud Databases to the Edge at All Things Open

Database apps on mobile devices or Internet of Things (IoT) devices shouldn’t stop working when there’s limited or no network connectivity. Learn how to bring data stored in a cloud database to the edge of the network (and back again), whenever an Internet connection is available. This talk will demonstrate techniques for replicating cloud databases with devices in order to build offline-enabled apps that can provide a better, faster user experience, both offline and online. The focus of this talk will be on IBM Cloudant, Apache CouchDB and PouchDB, an embeddable JavaScript database.

Attendees will learn how to use Apache CouchDB and PouchDB to access and synchronize data on devices with data in IBM Cloudant. They will learn how to create a PouchDB database, create a new document, update a document, delete a document, query a database, synchronize PouchDB with Cloudant, and live update a user interface based on database changes.

Bradley Holt

October 19, 2015
Tweet

More Decks by Bradley Holt

Other Decks in Programming

Transcript

  1. @jsloyer! @BradleyHolt! Taking Cloud Databases to the Edge All Things

    Open Monday, October 19, 2015
  2. @jsloyer! @BradleyHolt! Taking Cloud Databases to the Edge Bradley Holt

    Developer Advocate IBM Cloudant Jeff Sloyer Developer Advocate IBM Bluemix
  3. @jsloyer! @BradleyHolt! Image Credit: Joan Touzet (@wohali), ASF Member, CouchDB

    PMC Member
  4. @jsloyer! @BradleyHolt! Image Credit: Device landscape by Jeremy Keith, on

    Flickr Not just mobile first…!
  5. @jsloyer! @BradleyHolt! Image Credit: Cloud Formation Over the Adirondacks by

    Bradley Holt Offline First! Because being offline shouldn't be an error condition.!
  6. @jsloyer! @BradleyHolt! Doesn't ubiquitous connectivity make offline- enabled apps unnecessary?!

    Image Credit: Cell phone tower by Gary Lerude, on Flickr
  7. @jsloyer! @BradleyHolt! No.! Image Credit: Internet Splat Map by Steve

    Jurvetson, on Flickr
  8. @jsloyer! @BradleyHolt! Quite the opposite, in fact.! Image Credit: Tangled

    Network by Bruno Girin, on Flickr
  9. @jsloyer! @BradleyHolt! Ubiquitous connectivity is driving the demand for 


    offline capabilities.! Image Credit: Wired by Alexandre Duret-Lutz, on Flickr
  10. @jsloyer! @BradleyHolt! How?! Image Credit: connect me to BR549 by

    frankieleon, on Flickr
  11. @jsloyer! @BradleyHolt! The Eight Fallacies of Distributed Computing 1.  The

    network is reliable 2.  Latency is zero 3.  Bandwidth is infinite 4.  The network is secure 5.  Topology doesn't change 6.  There is one administrator 7.  Transport cost is zero 8.  The network is homogeneous Text Credit: The Eight Fallacies of Distributed Computing by Peter Deutsch | Image Credit: Pneumatic Central by Sleestak, on Flickr
  12. @jsloyer! @BradleyHolt! Image Credit: Connected version 2 by Hans Kylberg,

    on Flickr Offline-first is the only way 
 to achieve a true, 100% 
 always-on user experience.* ! *assuming the device is reliable!
  13. @jsloyer! @BradleyHolt! Faster User Experience! Image Credit: Speed DLR on

    Doklands by Umberto Rotundo, on Flickr
  14. @jsloyer! @BradleyHolt! Works Offline! Image Credit: Lynn Camp Prong (Explored)

    by AllieKF, on Flickr
  15. @jsloyer! @BradleyHolt! Battery and Bandwidth! Image Credit: Panorama of the

    Molasses Disaster site by Boston Public Library, on Flickr
  16. @jsloyer! @BradleyHolt! Offline Patterns & Anti-Patterns •  Don't return an

    error for no reason •  Do let users view cached/saved data •  Do synchronize data when connected
  17. @jsloyer! @BradleyHolt! Offline Patterns & Anti-Patterns •  Don't return an

    error for no reason •  Do let users view cached/saved data •  Do synchronize data when connected
  18. @jsloyer! @BradleyHolt! Offline Patterns & Anti-Patterns •  Don't return an

    error for no reason •  Do let users view cached/saved data •  Do synchronize data when connected
  19. @jsloyer! @BradleyHolt! Offline Patterns & Anti-Patterns •  Don't return an

    error for no reason •  Do let users view cached/saved data •  Do synchronize data when connected
  20. @jsloyer! @BradleyHolt! Introducing PouchDB

  21. @jsloyer! @BradleyHolt! PouchDB •  A database in your web browser—

    and Node.js! •  Can synchronize with any database that implements the CouchDB Replication Protocol •  Makes create, read, update and delete operations extremely fast
  22. @jsloyer! @BradleyHolt! JSON Documents {
 _id: "6EF9D2B0-13D3-1378-8D30-39E3CE0B36C2",
 _rev: "1-0b457efcf82fb29492ef927ba5b6ee15",
 type:

    "Feature",
 geometry: {
 type: "Point",
 coordinates: [
 -71.1028,
 42.3691
 ]
 },
 properties: {
 session_id: "3486b13f-7b8a-8a96-dfbf-9b82800e367f",
 timestamp: 1422928591717
 }
 }
  23. @jsloyer! @BradleyHolt! Responsive Mobile Web Apps, Hybrid Mobile Web Apps,

    or Internet of Things (IoT)
  24. @jsloyer! @BradleyHolt! Responsive Mobile Web Apps •  HTML5, CSS and

    JavaScript mobile web apps •  Responsive design •  Enhanced to enable offline usage
  25. @jsloyer! @BradleyHolt! Hybrid Mobile Web Apps •  Native mobile web

    apps built with HTML5, CSS and JavaScript •  Good for: •  Fully-featured, cross-platform native apps •  High-fidelity prototypes
  26. @jsloyer! @BradleyHolt! Internet of Things (IoT) •  Headless Node.js apps

    •  PouchDB includes a LevelDB adapter for use in Node.js •  Redis, Riak, and in-memory adapters are also available •  Good for: •  Internet of Things (IoT) applications •  Content delivery networks (CDN) •  Purpose-built devices Image Credit: Ethernet IoT Starter Kit
  27. @jsloyer! @BradleyHolt! Apache CouchDB

  28. @jsloyer! @BradleyHolt! Apache CouchDB •  JSON document database •  HTTP

    API •  Master-master replication
  29. @jsloyer! @BradleyHolt! IBM Cloudant •  Globally distributed data layer for

    web and mobile applications •  MongoDB-style queries •  Advanced geospatial capabilities •  Full text search indexing
  30. @jsloyer! @BradleyHolt! CouchDB Replication Protocol •  One-off, one way operation

    •  Peer-to-peer (masterless) •  Incremental •  Conflict detection
  31. @jsloyer! @BradleyHolt! Apache Cordova, PhoneGap and Ionic

  32. @jsloyer! @BradleyHolt! Apache Cordova and PhoneGap Image Credit: build.phonegap by

    Andrés Álvarez Iglesias, on Flickr
  33. @jsloyer! @BradleyHolt! Ionic •  Mobile-optimized HTML, CSS and JavaScript components

    •  Builds on Apache Cordova •  Utilizes AngularJS •  CLI installable via npm
  34. @jsloyer! @BradleyHolt! Using Cordova and Ionic $ npm install -g

    cordova ionic" $ ionic start energy-monitor tabs" $ cd energy-monitor" $ ionic platform add ios" $ ionic build ios" $ ionic emulate ios" Image Credit: Getting Started with Ionic
  35. @jsloyer! @BradleyHolt! HTML5 Offline Application Cache

  36. @jsloyer! @BradleyHolt! HTML5 Offline Application Cache •  Enables fully-functional offline

    web apps •  Stores files and assets for offline browsing •  Makes page loads very fast, even when online
  37. @jsloyer! @BradleyHolt! Cache Manifest File <html manifest="example.appcache">
 …
 </html>
 CACHE

    MANIFEST
 # v1 - 2015-01-08
 index.html
 logo.png
 app.css
 app.js
  38. @jsloyer! @BradleyHolt! PouchDB Code Examples

  39. @jsloyer! @BradleyHolt! Installing PouchDB <script src=
 "https://cdn.jsdelivr.net/pouchdb/3.5.0/pouchdb.min.js"
 ></script>
 CACHE MANIFEST


    # v2 - 2015-01-08
 index.html
 logo.png
 app.css
 app.js
 https://cdn.jsdelivr.net/pouchdb/3.5.0/pouchdb.min.js
  40. @jsloyer! @BradleyHolt! Creating a Local PouchDB Database var db =

    new PouchDB("smart-meter");" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/01-create-local-database.js
  41. @jsloyer! @BradleyHolt! Creating a Remote PouchDB Database var remoteDb =

    new PouchDB("https://bradley-holt.cloudant.com/smart-meter");" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/02-create-remote-database.js
  42. @jsloyer! @BradleyHolt! Cross-Origin Resource Sharing (CORS) •  Enable Cross-Origin Resource

    Sharing (CORS) on remote database •  Browsers place security restrictions on cross-site HTTP requests •  If you run into a problem, remember this warning! Image Credit: Grunge Warning Sign - Do Not Read This Sign by Nicolas Raymond, on Flickr
  43. @jsloyer! @BradleyHolt! Creating a New Document var db = new

    PouchDB("smart-meter");" db.put({" _id: "2014-11-12T23:27:03.794Z"," kilowatt_hours: 14" }).then(function() {" console.log("Document created");" }).catch(function(error) {" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/04-create-document-put.js
  44. @jsloyer! @BradleyHolt! Updating a Document db.put({" _id: "2014-11-12T23:27:03.794Z"," kilowatt_hours: 14"

    }).then(function() {" return db.get("2014-11-12T23:27:03.794Z");" }).then(function(doc) {" // Update the value for kilowatt hours" doc.kilowatt_hours = 15;" // Put the document back to the database" return db.put(doc);" }).catch(function(error) {" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/05-update-document.js
  45. @jsloyer! @BradleyHolt! Deleting a Document db.put({" _id: "2014-11-12T23:27:03.794Z"," kilowatt_hours: 14"

    }).then(function() {" // Get the document" return db.get("2014-11-12T23:27:03.794Z");" }).then(function(doc) {" // Remove the document from the database" return db.remove(doc);" }).catch(function(error) {" console.log(error);" });" " https://github.com/bradley-holt/offline-first/blob/master/pouchdb/06-delete-document.js
  46. @jsloyer! @BradleyHolt! Querying a Database with allDocs" db.bulkDocs([" {_id: "2014-11-12T23:27:03.794Z",

    kilowatt_hours: 14}," {_id: "2014-11-13T00:52:01.471Z", kilowatt_hours: 15}," {_id: "2014-11-13T01:39:28.911Z", kilowatt_hours: 16}," {_id: "2014-11-13T02:52:01.471Z", kilowatt_hours: 17}" ]).then(function(result) {" // Get all documents" return db.allDocs({include_docs: true});" }).then(function(response) {" console.log(response);" }).catch(function(error) {" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/07-query-database-all-docs.js
  47. @jsloyer! @BradleyHolt! allDocs Options •  include_docs" •  conflicts" •  attachments"

    •  startkey" •  endkey" •  inclusive_end (true by default) •  limit" •  skip" •  descending" •  key" •  keys"
  48. @jsloyer! @BradleyHolt! Querying a Database with Map/Reduce •  Most queries

    can be done with allDocs (in PouchDB) •  Map functions transform documents into indexes •  Reduce functions aggregate results of Map functions •  _sum" •  _count" •  _stats"
  49. @jsloyer! @BradleyHolt! Querying a Database with PouchDB Find •  Based

    on Cloudant Query (Mango) •  MongoDB-style query language •  Define fields to index Image Credit: Mango with section on a white background by bangdoll, on Flickr
  50. @jsloyer! @BradleyHolt! Replication Patterns

  51. @jsloyer! @BradleyHolt! Clemmie Danyel Shelba Manuel Francis Marissa Mitchel Georgianne

    Garnet Audrey Kalyn Image Credit: database by Tim Morgan, on Flickr
  52. @jsloyer! @BradleyHolt! 1b40d0a4 c28e6965 3fa0bee7 e83a0aeb cbdca6de ef9443e2 8e4d5a03 2e070be6

    271dc425 9a278e5e f4215dfa Image Credit: database by Tim Morgan, on Flickr
  53. @jsloyer! @BradleyHolt! Write-Only Replication •  Data generated on the device

    •  Replicate this data to the cloud from multiple users and/or devices •  Example uses: •  User updates •  Sensor data
  54. @jsloyer! @BradleyHolt! Write-Only Replication var db = new PouchDB("smart-meter");" var

    remoteDb = new PouchDB(" "https://bradley-holt.cloudant.com/smart-meter"" );" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/08-replicate-database.js
  55. @jsloyer! @BradleyHolt! Write-Only Replication db.bulkDocs([" {_id: "2014-11-12T23:27:03.794Z", kilowatt_hours: 14}," {_id:

    "2014-11-13T00:52:01.471Z", kilowatt_hours: 15}," {_id: "2014-11-13T01:39:28.911Z", kilowatt_hours: 16}," {_id: "2014-11-13T02:52:01.471Z", kilowatt_hours: 17}" ]).then(function(result) {" …" }).catch(function(error) {" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/08-replicate-database.js
  56. @jsloyer! @BradleyHolt! Write-Only Replication db.replicate.to(remoteDb, {" live: false," retry: false"

    }).on("change", function(info) {" // Replication has written a new document" console.log(info);" }).on("complete", function(info) {" // Replication has complete or been cancelled" console.log(info);" }).on("error", function(error) {" // Replication has stopped due to an unrecoverable failure" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/08-replicate-database.js
  57. @jsloyer! @BradleyHolt! Read-Only Replication •  Data generated in the cloud

    •  Replicate this data to multiple users and/or devices from the cloud •  Example uses: •  Weather •  Notifications
  58. @jsloyer! @BradleyHolt! Read-Only Replication remoteDb.replicate.to(db, {" live: false," retry: false"

    }).on("change", function(info) {" // Replication has written a new document" console.log(info);" }).on("complete", function(info) {" // Replication has complete or been cancelled" console.log(info);" }).on("error", function(error) {" // Replication has stopped due to an unrecoverable failure" console.log(error);" });"
  59. @jsloyer! @BradleyHolt! Bidirectional Replication •  Data generated on both the

    device and in the cloud •  Replicate this data to/from the cloud and to/from multiple users and/or devices •  Useful when you need to share data between users and/or devices
  60. @jsloyer! @BradleyHolt! Bidirectional Replication db.sync(remoteDb, {" live: false," retry: false"

    }).on("change", function(info) {" // Replication has written a new document" console.log(info);" }).on("complete", function(info) {" // Replication has complete or been cancelled" console.log(info);" }).on("error", function(error) {" // Replication has stopped due to an unrecoverable failure" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/09-replicate-database-bidirectional.js
  61. @jsloyer! @BradleyHolt! Live Replication var sync = db.sync(remoteDb, {" live:

    true," retry: true" }).on("change", function(info) {" // Replication has written a new document" console.log(info);" }).on("complete", function(info) {" // Replication has complete or been cancelled" console.log(info);" }).on("error", function(error) {" // Replication has stopped due to an unrecoverable failure" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/10-replicate-database-live.js
  62. @jsloyer! @BradleyHolt! Filtered Replication •  Select (with a function) which

    documents to replicate •  Filter can be defined locally within PouchDB, or remotely on Cloudant
  63. @jsloyer! @BradleyHolt! Filtered Replication db.replicate.to(remoteDb, {" filter: function(doc) {" return

    doc._id >= "2014-11-13T00:00:00.000Z";" }" }).on("change", function(info) {" // Replication has written a new document" console.log(info);" }).on("complete", function(info) {" // Replication has complete or been cancelled" console.log(info);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/12-replicate-database-filtered.js
  64. @jsloyer! @BradleyHolt! Image Credits •  Joan Touzet (@wohali), ASF Member,

    CouchDB PMC Member <https://twitter.com/wohali/status/595689720933445632> •  Device landscape by Jeremy Keith, on Flickr <https://www.flickr.com/photos/adactio/6153481666> •  Cloud Formation Over the Adirondacks by Bradley Holt <https://twitter.com/BradleyHolt/status/623030107679002624> •  Cell phone tower by Gary Lerude, on Flickr <https://www.flickr.com/photos/garylerude/7511464618/> •  Internet Splat Map by Steve Jurvetson, on Flickr <https://www.flickr.com/photos/jurvetson/916142/> •  Tangled Network by Bruno Girin, on Flickr <https://www.flickr.com/photos/brunogirin/73014722/> •  Wired by Alexandre Duret-Lutz, on Flickr <https://www.flickr.com/photos/gadl/1308489717/> •  connect me to BR549 by frankieleon, on Flickr <https://www.flickr.com/photos/armydre2008/5914251874/> •  Pneumatic Central by Sleestak, on Flickr <https://www.flickr.com/photos/dlanod/235990854> •  Connected version 2 by Hans Kylberg, on Flickr <https://www.flickr.com/photos/visulogik/2340179011/> •  Speed DLR on Doklands by Umberto Rotundo, on Flickr <https://www.flickr.com/photos/turyddu/4394742730/> •  Lynn Camp Prong (Explored) by AllieKF, on Flickr <https://www.flickr.com/photos/allie_k/14501117120> •  Panorama of the Molasses Disaster site by Boston Public Library, on Flickr <https://www.flickr.com/photos/boston_public_library/4901555337/> •  Ethernet IoT Starter Kit <https://developer.mbed.org/platforms/IBMEthernetKit/> •  build.phonegap by Andrés Álvarez Iglesias, on Flickr <https://www.flickr.com/photos/doctorserone/5682929553> •  Getting Started with Ionic <http://ionicframework.com/getting-started/> •  Grunge Warning Sign - Do Not Read This Sign by Nicolas Raymond, on Flickr <https://www.flickr.com/photos/80497449@N04/7417352980> •  Mango with section on a white background by bangdoll, on Flickr <https://www.flickr.com/photos/bangdoll/5665235102> •  database by Tim Morgan, on Flickr <https://www.flickr.com/photos/timothymorgan/75294154>
  65. @jsloyer! @BradleyHolt! Bradley Holt Developer Advocate IBM Cloudant @BradleyHolt github.com/bradley-holt

    Jeff Sloyer Developer Advocate IBM Bluemix @jsloyer github.com/jsloyer