Taking Cloud Databases to the Edge at All Things Open

Database apps on mobile devices or Internet of Things (IoT) devices shouldn’t stop working when there’s limited or no network connectivity. Learn how to bring data stored in a cloud database to the edge of the network (and back again), whenever an Internet connection is available. This talk will demonstrate techniques for replicating cloud databases with devices in order to build offline-enabled apps that can provide a better, faster user experience, both offline and online. The focus of this talk will be on IBM Cloudant, Apache CouchDB and PouchDB, an embeddable JavaScript database.

Attendees will learn how to use Apache CouchDB and PouchDB to access and synchronize data on devices with data in IBM Cloudant. They will learn how to create a PouchDB database, create a new document, update a document, delete a document, query a database, synchronize PouchDB with Cloudant, and live update a user interface based on database changes.

Transcript

1. @jsloyer! @BradleyHolt! Taking Cloud Databases to the Edge All Things Open Monday, October 19, 2015
2. @jsloyer! @BradleyHolt! Taking Cloud Databases to the Edge Bradley Holt Developer Advocate IBM Cloudant Jeff Sloyer Developer Advocate IBM Bluemix
3. @jsloyer! @BradleyHolt! Image Credit: Joan Touzet (@wohali), ASF Member, CouchDB PMC Member
4. @jsloyer! @BradleyHolt! Image Credit: Device landscape by Jeremy Keith, on Flickr Not just mobile first…!
5. @jsloyer! @BradleyHolt! Image Credit: Cloud Formation Over the Adirondacks by Bradley Holt Offline First! Because being offline shouldn't be an error condition.!
6. @jsloyer! @BradleyHolt! Doesn't ubiquitous connectivity make offline- enabled apps unnecessary?! Image Credit: Cell phone tower by Gary Lerude, on Flickr
7. @jsloyer! @BradleyHolt! No.! Image Credit: Internet Splat Map by Steve Jurvetson, on Flickr
8. @jsloyer! @BradleyHolt! Quite the opposite, in fact.! Image Credit: Tangled Network by Bruno Girin, on Flickr
9. @jsloyer! @BradleyHolt! Ubiquitous connectivity is driving the demand for 
 offline capabilities.! Image Credit: Wired by Alexandre Duret-Lutz, on Flickr
10. @jsloyer! @BradleyHolt! How?! Image Credit: connect me to BR549 by frankieleon, on Flickr
11. @jsloyer! @BradleyHolt! The Eight Fallacies of Distributed Computing 1.  The network is reliable 2.  Latency is zero 3.  Bandwidth is infinite 4.  The network is secure 5.  Topology doesn't change 6.  There is one administrator 7.  Transport cost is zero 8.  The network is homogeneous Text Credit: The Eight Fallacies of Distributed Computing by Peter Deutsch | Image Credit: Pneumatic Central by Sleestak, on Flickr
12. @jsloyer! @BradleyHolt! Image Credit: Connected version 2 by Hans Kylberg, on Flickr Offline-first is the only way 
 to achieve a true, 100% 
 always-on user experience.* ! *assuming the device is reliable!
13. @jsloyer! @BradleyHolt! Faster User Experience! Image Credit: Speed DLR on Doklands by Umberto Rotundo, on Flickr
14. @jsloyer! @BradleyHolt! Works Offline! Image Credit: Lynn Camp Prong (Explored) by AllieKF, on Flickr
15. @jsloyer! @BradleyHolt! Battery and Bandwidth! Image Credit: Panorama of the Molasses Disaster site by Boston Public Library, on Flickr
16. @jsloyer! @BradleyHolt! Offline Patterns & Anti-Patterns •  Don't return an error for no reason •  Do let users view cached/saved data •  Do synchronize data when connected
17. @jsloyer! @BradleyHolt! Offline Patterns & Anti-Patterns •  Don't return an error for no reason •  Do let users view cached/saved data •  Do synchronize data when connected
18. @jsloyer! @BradleyHolt! Offline Patterns & Anti-Patterns •  Don't return an error for no reason •  Do let users view cached/saved data •  Do synchronize data when connected
19. @jsloyer! @BradleyHolt! Offline Patterns & Anti-Patterns •  Don't return an error for no reason •  Do let users view cached/saved data •  Do synchronize data when connected
20. @jsloyer! @BradleyHolt! Introducing PouchDB
21. @jsloyer! @BradleyHolt! PouchDB •  A database in your web browser— and Node.js! •  Can synchronize with any database that implements the CouchDB Replication Protocol •  Makes create, read, update and delete operations extremely fast
22. @jsloyer! @BradleyHolt! JSON Documents {
 _id: "6EF9D2B0-13D3-1378-8D30-39E3CE0B36C2",
 _rev: "1-0b457efcf82fb29492ef927ba5b6ee15",
 type: "Feature",
 geometry: {
 type: "Point",
 coordinates: [
 -71.1028,
 42.3691
 ]
 },
 properties: {
 session_id: "3486b13f-7b8a-8a96-dfbf-9b82800e367f",
 timestamp: 1422928591717
 }
 }
23. @jsloyer! @BradleyHolt! Responsive Mobile Web Apps, Hybrid Mobile Web Apps, or Internet of Things (IoT)
24. @jsloyer! @BradleyHolt! Responsive Mobile Web Apps •  HTML5, CSS and JavaScript mobile web apps •  Responsive design •  Enhanced to enable offline usage
25. @jsloyer! @BradleyHolt! Hybrid Mobile Web Apps •  Native mobile web apps built with HTML5, CSS and JavaScript •  Good for: •  Fully-featured, cross-platform native apps •  High-fidelity prototypes
26. @jsloyer! @BradleyHolt! Internet of Things (IoT) •  Headless Node.js apps •  PouchDB includes a LevelDB adapter for use in Node.js •  Redis, Riak, and in-memory adapters are also available •  Good for: •  Internet of Things (IoT) applications •  Content delivery networks (CDN) •  Purpose-built devices Image Credit: Ethernet IoT Starter Kit
27. @jsloyer! @BradleyHolt! Apache CouchDB
28. @jsloyer! @BradleyHolt! Apache CouchDB •  JSON document database •  HTTP API •  Master-master replication
29. @jsloyer! @BradleyHolt! IBM Cloudant •  Globally distributed data layer for web and mobile applications •  MongoDB-style queries •  Advanced geospatial capabilities •  Full text search indexing
30. @jsloyer! @BradleyHolt! CouchDB Replication Protocol •  One-off, one way operation •  Peer-to-peer (masterless) •  Incremental •  Conflict detection
31. @jsloyer! @BradleyHolt! Apache Cordova, PhoneGap and Ionic
32. @jsloyer! @BradleyHolt! Apache Cordova and PhoneGap Image Credit: build.phonegap by Andrés Álvarez Iglesias, on Flickr
33. @jsloyer! @BradleyHolt! Ionic •  Mobile-optimized HTML, CSS and JavaScript components •  Builds on Apache Cordova •  Utilizes AngularJS •  CLI installable via npm
34. @jsloyer! @BradleyHolt! Using Cordova and Ionic $ npm install -g cordova ionic" $ ionic start energy-monitor tabs" $ cd energy-monitor" $ ionic platform add ios" $ ionic build ios" $ ionic emulate ios" Image Credit: Getting Started with Ionic
35. @jsloyer! @BradleyHolt! HTML5 Offline Application Cache
36. @jsloyer! @BradleyHolt! HTML5 Offline Application Cache •  Enables fully-functional offline web apps •  Stores files and assets for offline browsing •  Makes page loads very fast, even when online
37. @jsloyer! @BradleyHolt! Cache Manifest File <html manifest="example.appcache">
 …
 </html>
 CACHE MANIFEST
 # v1 - 2015-01-08
 index.html
 logo.png
 app.css
 app.js
38. @jsloyer! @BradleyHolt! PouchDB Code Examples
39. @jsloyer! @BradleyHolt! Installing PouchDB <script src=
 "https://cdn.jsdelivr.net/pouchdb/3.5.0/pouchdb.min.js"
 ></script>
 CACHE MANIFEST
 # v2 - 2015-01-08
 index.html
 logo.png
 app.css
 app.js
 https://cdn.jsdelivr.net/pouchdb/3.5.0/pouchdb.min.js
40. @jsloyer! @BradleyHolt! Creating a Local PouchDB Database var db = new PouchDB("smart-meter");" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/01-create-local-database.js
41. @jsloyer! @BradleyHolt! Creating a Remote PouchDB Database var remoteDb = new PouchDB("https://bradley-holt.cloudant.com/smart-meter");" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/02-create-remote-database.js
42. @jsloyer! @BradleyHolt! Cross-Origin Resource Sharing (CORS) •  Enable Cross-Origin Resource Sharing (CORS) on remote database •  Browsers place security restrictions on cross-site HTTP requests •  If you run into a problem, remember this warning! Image Credit: Grunge Warning Sign - Do Not Read This Sign by Nicolas Raymond, on Flickr
43. @jsloyer! @BradleyHolt! Creating a New Document var db = new PouchDB("smart-meter");" db.put({" _id: "2014-11-12T23:27:03.794Z"," kilowatt_hours: 14" }).then(function() {" console.log("Document created");" }).catch(function(error) {" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/04-create-document-put.js
44. @jsloyer! @BradleyHolt! Updating a Document db.put({" _id: "2014-11-12T23:27:03.794Z"," kilowatt_hours: 14" }).then(function() {" return db.get("2014-11-12T23:27:03.794Z");" }).then(function(doc) {" // Update the value for kilowatt hours" doc.kilowatt_hours = 15;" // Put the document back to the database" return db.put(doc);" }).catch(function(error) {" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/05-update-document.js
45. @jsloyer! @BradleyHolt! Deleting a Document db.put({" _id: "2014-11-12T23:27:03.794Z"," kilowatt_hours: 14" }).then(function() {" // Get the document" return db.get("2014-11-12T23:27:03.794Z");" }).then(function(doc) {" // Remove the document from the database" return db.remove(doc);" }).catch(function(error) {" console.log(error);" });" " https://github.com/bradley-holt/offline-first/blob/master/pouchdb/06-delete-document.js
46. @jsloyer! @BradleyHolt! Querying a Database with allDocs" db.bulkDocs([" {_id: "2014-11-12T23:27:03.794Z", kilowatt_hours: 14}," {_id: "2014-11-13T00:52:01.471Z", kilowatt_hours: 15}," {_id: "2014-11-13T01:39:28.911Z", kilowatt_hours: 16}," {_id: "2014-11-13T02:52:01.471Z", kilowatt_hours: 17}" ]).then(function(result) {" // Get all documents" return db.allDocs({include_docs: true});" }).then(function(response) {" console.log(response);" }).catch(function(error) {" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/07-query-database-all-docs.js
47. @jsloyer! @BradleyHolt! allDocs Options •  include_docs" •  conflicts" •  attachments" •  startkey" •  endkey" •  inclusive_end (true by default) •  limit" •  skip" •  descending" •  key" •  keys"
48. @jsloyer! @BradleyHolt! Querying a Database with Map/Reduce •  Most queries can be done with allDocs (in PouchDB) •  Map functions transform documents into indexes •  Reduce functions aggregate results of Map functions •  _sum" •  _count" •  _stats"
49. @jsloyer! @BradleyHolt! Querying a Database with PouchDB Find •  Based on Cloudant Query (Mango) •  MongoDB-style query language •  Define fields to index Image Credit: Mango with section on a white background by bangdoll, on Flickr
50. @jsloyer! @BradleyHolt! Replication Patterns
51. @jsloyer! @BradleyHolt! Clemmie Danyel Shelba Manuel Francis Marissa Mitchel Georgianne Garnet Audrey Kalyn Image Credit: database by Tim Morgan, on Flickr
52. @jsloyer! @BradleyHolt! 1b40d0a4 c28e6965 3fa0bee7 e83a0aeb cbdca6de ef9443e2 8e4d5a03 2e070be6 271dc425 9a278e5e f4215dfa Image Credit: database by Tim Morgan, on Flickr
53. @jsloyer! @BradleyHolt! Write-Only Replication •  Data generated on the device •  Replicate this data to the cloud from multiple users and/or devices •  Example uses: •  User updates •  Sensor data
54. @jsloyer! @BradleyHolt! Write-Only Replication var db = new PouchDB("smart-meter");" var remoteDb = new PouchDB(" "https://bradley-holt.cloudant.com/smart-meter"" );" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/08-replicate-database.js
55. @jsloyer! @BradleyHolt! Write-Only Replication db.bulkDocs([" {_id: "2014-11-12T23:27:03.794Z", kilowatt_hours: 14}," {_id: "2014-11-13T00:52:01.471Z", kilowatt_hours: 15}," {_id: "2014-11-13T01:39:28.911Z", kilowatt_hours: 16}," {_id: "2014-11-13T02:52:01.471Z", kilowatt_hours: 17}" ]).then(function(result) {" …" }).catch(function(error) {" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/08-replicate-database.js
56. @jsloyer! @BradleyHolt! Write-Only Replication db.replicate.to(remoteDb, {" live: false," retry: false" }).on("change", function(info) {" // Replication has written a new document" console.log(info);" }).on("complete", function(info) {" // Replication has complete or been cancelled" console.log(info);" }).on("error", function(error) {" // Replication has stopped due to an unrecoverable failure" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/08-replicate-database.js
57. @jsloyer! @BradleyHolt! Read-Only Replication •  Data generated in the cloud •  Replicate this data to multiple users and/or devices from the cloud •  Example uses: •  Weather •  Notifications
58. @jsloyer! @BradleyHolt! Read-Only Replication remoteDb.replicate.to(db, {" live: false," retry: false" }).on("change", function(info) {" // Replication has written a new document" console.log(info);" }).on("complete", function(info) {" // Replication has complete or been cancelled" console.log(info);" }).on("error", function(error) {" // Replication has stopped due to an unrecoverable failure" console.log(error);" });"
59. @jsloyer! @BradleyHolt! Bidirectional Replication •  Data generated on both the device and in the cloud •  Replicate this data to/from the cloud and to/from multiple users and/or devices •  Useful when you need to share data between users and/or devices
60. @jsloyer! @BradleyHolt! Bidirectional Replication db.sync(remoteDb, {" live: false," retry: false" }).on("change", function(info) {" // Replication has written a new document" console.log(info);" }).on("complete", function(info) {" // Replication has complete or been cancelled" console.log(info);" }).on("error", function(error) {" // Replication has stopped due to an unrecoverable failure" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/09-replicate-database-bidirectional.js
61. @jsloyer! @BradleyHolt! Live Replication var sync = db.sync(remoteDb, {" live: true," retry: true" }).on("change", function(info) {" // Replication has written a new document" console.log(info);" }).on("complete", function(info) {" // Replication has complete or been cancelled" console.log(info);" }).on("error", function(error) {" // Replication has stopped due to an unrecoverable failure" console.log(error);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/10-replicate-database-live.js
62. @jsloyer! @BradleyHolt! Filtered Replication •  Select (with a function) which documents to replicate •  Filter can be defined locally within PouchDB, or remotely on Cloudant
63. @jsloyer! @BradleyHolt! Filtered Replication db.replicate.to(remoteDb, {" filter: function(doc) {" return doc._id >= "2014-11-13T00:00:00.000Z";" }" }).on("change", function(info) {" // Replication has written a new document" console.log(info);" }).on("complete", function(info) {" // Replication has complete or been cancelled" console.log(info);" });" https://github.com/bradley-holt/offline-first/blob/master/pouchdb/12-replicate-database-filtered.js
64. @jsloyer! @BradleyHolt! Image Credits •  Joan Touzet (@wohali), ASF Member, CouchDB PMC Member <https://twitter.com/wohali/status/595689720933445632> •  Device landscape by Jeremy Keith, on Flickr <https://www.flickr.com/photos/adactio/6153481666> •  Cloud Formation Over the Adirondacks by Bradley Holt <https://twitter.com/BradleyHolt/status/623030107679002624> •  Cell phone tower by Gary Lerude, on Flickr <https://www.flickr.com/photos/garylerude/7511464618/> •  Internet Splat Map by Steve Jurvetson, on Flickr <https://www.flickr.com/photos/jurvetson/916142/> •  Tangled Network by Bruno Girin, on Flickr <https://www.flickr.com/photos/brunogirin/73014722/> •  Wired by Alexandre Duret-Lutz, on Flickr <https://www.flickr.com/photos/gadl/1308489717/> •  connect me to BR549 by frankieleon, on Flickr <https://www.flickr.com/photos/armydre2008/5914251874/> •  Pneumatic Central by Sleestak, on Flickr <https://www.flickr.com/photos/dlanod/235990854> •  Connected version 2 by Hans Kylberg, on Flickr <https://www.flickr.com/photos/visulogik/2340179011/> •  Speed DLR on Doklands by Umberto Rotundo, on Flickr <https://www.flickr.com/photos/turyddu/4394742730/> •  Lynn Camp Prong (Explored) by AllieKF, on Flickr <https://www.flickr.com/photos/allie_k/14501117120> •  Panorama of the Molasses Disaster site by Boston Public Library, on Flickr <https://www.flickr.com/photos/boston_public_library/4901555337/> •  Ethernet IoT Starter Kit <https://developer.mbed.org/platforms/IBMEthernetKit/> •  build.phonegap by Andrés Álvarez Iglesias, on Flickr <https://www.flickr.com/photos/doctorserone/5682929553> •  Getting Started with Ionic <http://ionicframework.com/getting-started/> •  Grunge Warning Sign - Do Not Read This Sign by Nicolas Raymond, on Flickr <https://www.flickr.com/photos/80497449@N04/7417352980> •  Mango with section on a white background by bangdoll, on Flickr <https://www.flickr.com/photos/bangdoll/5665235102> •  database by Tim Morgan, on Flickr <https://www.flickr.com/photos/timothymorgan/75294154>
65. @jsloyer! @BradleyHolt! Bradley Holt Developer Advocate IBM Cloudant @BradleyHolt github.com/bradley-holt Jeff Sloyer Developer Advocate IBM Bluemix @jsloyer github.com/jsloyer