Offline Sync for Progressive Web Apps Workshop at ZendCon

With the introduction of Progressive Web Apps and web platform APIs such as persistent storage, payments, and geolocation, the web platform is taking on native apps. A Progressive Web App combines the best parts both a web app and a native app. You browse to a Progressive Web App just like you browse to any other website. It lives at a URL, can be indexed by search engines, and can be linked to from anywhere else on the web. As you continue to use a Progressive Web App it gains additional native app-like capabilities. For example, the app could be installed to the home screen on your device. You might also grant the app the ability to send you push notifications, or the ability to access your camera, your microphone, or other device resources.

One important aspect of Progressive Web Apps is the concept of building your app to be Offline First. With an Offline First approach, you design your app for the most resource-constrained environment first. This approach provides a consistent user experience whether the user's device has no connectivity, limited connectivity, or great connectivity. One of the biggest benefits of Offline First apps is that they can be very fast, as they provide zero-latency access to content and data stored directly on the device.

The Service Worker API can do most of the heavy lifting when it comes to storing content and assets for Offline First Progressive Web Apps. A bigger challenge can be storing and sync'ing your app's data. One of the best answers today for offline sync for Progressive Web Apps is a combination of Apache CouchDB (an open source document database), Hoodie (an open source Node.js backend for Offline First apps), and PouchDB (an open source JavaScript database that syncs). This stack can provide you with the starting point for your own Progressive Web App mobile backend and frontend, and it's entirely open source! You'll leave this workshop having built a fully-functional Offline First Progressive Web Apps using Polymer, Web Components, Service Workers, Apache CouchDB, Hoodie, and PouchDB.

Transcript

1. ZendCon Bradley Holt, Developer Advocate Monday, October 23, 2017 Offline Sync for Progressive Web Apps ⌨
3. Prerequisite Knowledge & Skills §  Ability to write JavaScript at a novice level, at minimum §  A basic understanding of JavaScript promises https://pouchdb.com/2015/05/18/we-have-a-problem-with-promises.html §  A basic understanding of HTML §  Ability to work with an application programming interface (API) https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb
4. Tutorial Outline Background Key Concepts Initial Set Up ⌨ Creating the Polymer App Syncing Data What's next?
5. Background
6. https://mobilefirstplatform.ibmcloud.com/blog/2016/01/13/ionic-hybrid-mobile-app-using-mobilefirst-platform-7-1-cli/ Native Mobile Apps
7. organic growth by Steve Jurvetson, on Flickr (CC BY 2.0). The Web Platform
8. W3C HTML5 Logo (CC BY 3.0).
9. Mobile First & the Mobile Web Screen size visualisation by Orde Saunders, on Flickr (CC BY 2.0).
10. https://responsivedesign.is/examples/css-conf-europe/
11. Evergreen by Basheer Tome, on Flickr (CC BY 2.0). Evergreen Browsers
12. mettalic weave by Joel Ormsby, on Flickr (CC BY 2.0). Polyfills
13. By Diseño Web Valencia, via Wikimedia Commons (CC BY-SA 4.0).
15. https://doesmysiteneedhttps.com/
16. Percentage of Web Pages Loaded by Firefox Using HTTPS https://letsencrypt.org/stats/
17. Web or Mobile App? iPhone 4 - Bottom View with Bumper by William Hook, on Flickr (CC BY-SA 2.0).
18. http://gph.is/2dhW35c
19. Progressive Web Apps (or how to have your cake and eat it too) German Chocolate Cake by Kimberly Vardeman, on Flickr (CC BY 2.0).
20. Progressive Web Apps A Progressive Web App provides both the discoverability of a web app and the reliable, fast, and engaging user experience of a native mobile app https://www.pokedex.org/
21. "I don’t want the web to equal native; I want the web to surpass it. I, for one, would prefer a reality where my home screen isn’t filled with the icons of startups and companies that have fulfilled the criteria of the gatekeepers. But a home screen filled with the faces of people who didn’t have to ask anyone’s permission to publish? That’s what I want!" –Jeremy Keith Adactio: Journal—Progressing the web | Jeremy Keith by Matthew Oliphant, on Flickr (CC BY-ND 2.0).
22. https://www.pwastats.com/
23. https://developers.google.com/web/showcase/2017/twitter
24. Geolocation API Network Information API IndexedDB WebRTC API Push API Notifications API Device Orientation API Screen Orientation API Permissions API Proximity API Pointer Lock API Vibration API Payment Request API Battery Status API Ambient Light Sensor API Service Worker API IBM Design Icons by International Business Machines Corporation (CC BY 4.0).
25. Offline First Progressive Web Apps must be Offline First in order to provide a reliable, fast, and engaging user experience regardless of network connectivity No Signal by Steve Hodgson, on Flickr (CC BY-SA 2.0).
26. speed 3 by Monkieyes, on Flickr (CC BY 2.0). Zero Latency Access to Content & Data
27. Offline, Online & ¯\_(ツ)_/¯
28. The Next Billion Internet Users the galaxy by tommy@chau, on Flickr (CC BY 2.0).
29. Cache content & assets with Service Workers Service Workers fetch diagram by Mozilla Contributors (CC-BY-SA 2.5).
30. IndexedDB Use IndexedDB or localForage (a polyfill that uses WebSQL or localStorage if IndexedDB is not supported) to make application data available while offline https://caniuse.com/#feat=indexeddb
31. Technological devices design created by D3images - Freepik.com | Texture 94 by Ellen van Deelen, on Flickr (CC BY 2.0). Sync is Hard
32. IBM Cloudant Apache CouchDB PouchDB
33. "CouchDB’s superpower is sync. Sometimes I even try to explain it to people by saying, 'CouchDB isn’t a database; it’s a sync engine.' It’s a way of efficiently transferring data from one place to another, while intelligently managing conflicts and revisions. It’s very similar to Git. When I make that analogy, the light bulb often goes off." –Nolan Lawson PouchDB & CouchDB: An interview with Nolan Lawson – CouchDB Blog
34. Apache CouchDB An open source document database featuring an HTTP API, JSON documents, clustering capabilities for horizontal scalability, and peer-to-peer replication http://couchdb.apache.org/
35. PouchDB An open source JavaScript database that syncs with anything that implements the CouchDB Replication Protocol https://pouchdb.com/
36. IBM Cloudant A fully-managed database-as-a-service (DBaaS) based on Apache CouchDB with additional full text and geospatial search capabilities https://www.ibm.com/analytics/us/en/technology/cloud-data-services/cloudant/
37. Apache CouchDB Devices by Darwin Laganzon, on Pixabay (CC0 Public Domain). | Cloud by Raphael Silva, on Pixabay (CC0 Public Domain). PouchDB Browser-to- Cloud Sync
38. userdb-d76846 userdb-905cec userdb-adc95b userdb-c082f2 userdb-730bba userdb-c3d3e5 userdb-a1ec1f userdb-85a327 userdb-9b9aba userdb-85bcfe userdb-da3d25 One Database Per User database by Tim Morgan, on Flickr (CC BY 2.0). userdb-43ecb5 userdb-8a5dfa
40. Apache CouchDB Devices by Darwin Laganzon, on Pixabay (CC0 Public Domain). | Cloud by Raphael Silva, on Pixabay (CC0 Public Domain). PouchDB
41. Summary Build on the web platform with Progressive Web Apps Offline First makes for a reliable, fast, and engaging user experience Cache URL-addressable resources and content with Service Workers Store app data in PouchDB to easily sync your Offline First data Use Hoodie as the backend for your multi-user Offline First app
42. Tiny Steps by Jim Whimpey, on Flickr (CC BY 2.0). Progressive Web Apps, Progressively
43. Progressive Web Apps, Progressively Ensure your website is mobile friendly Improve an existing website Create a native app-like experience Leverage the web platform
44. Progressive Web Apps, Progressively •  Implement responsive web design •  Ensure cross-browser compatibility •  Test and improve web performance Ensure your website is mobile friendly Improve an existing website Create a native app-like experience Leverage the web platform
45. Progressive Web Apps, Progressively Ensure your website is mobile friendly •  Use an SSL/TLS certificate (Let's Encrypt is your friend) •  Add a Web App Manifest (name, author, icon, description, etc.) •  Use a Service Worker to make your app cache-first Improve an existing website Create a native app-like experience Leverage the web platform
46. Progressive Web Apps, Progressively Ensure your website is mobile friendly Improve an existing website •  Provide instant page transitions with a single-page (client-rendered) app •  Minimize time-to-interactive with the PRPL pattern •  Use a local database (IndexedDB, localForage, or PouchDB) Create a native app-like experience Leverage the web platform
47. Progressive Web Apps, Progressively Ensure your website is mobile friendly Improve an existing website Create a native app-like experience •  Ensure that each page can be addressed via a URL •  Implement server-side rendering for users without JavaScript •  Build a hybrid mobile app with a tool like Ionic or Apache Cordova Leverage the web platform
48. Offline Camp From the Catskill Mountains, to the Central Coast of California, to Berlin, we're building the Offline First community, one campfire at a time Image credit: Aaron Ross
50. Get Involved in the Offline First Community! §  Join the Offline First Slack team: offlinefirst.org/chat/ §  Follow @OfflineCamp on Twitter §  Read the Offline Camp Medium publication: medium.com/offline-camp §  Join us at an upcoming Offline Camp: http://offlinefirst.org/camp/
51. Key Concepts
52. Shopping List Offline First Demo Apps (work in progress) A series of demo Progressive Web Apps, hybrid mobile apps, native mobile apps, and a desktop app—each built using a different stack https://github.com/ibm-watson-data-lab/shopping-list
53. https://github.com/ibm-watson-data-lab/shopping-list
54. Polymer Libraries, tools, and patterns for building Progressive Web Apps using web platform features such as Web Components, Service Workers, and HTTP/2 https://www.polymer-project.org/
55. Web Components Open standard for components and widgets that are customizable, reusable, and encapsulated https://www.webcomponents.org/
56. Polymer App Toolbox Components, tools, and templates for building Progressive Web Apps with Polymer and Web Components https://www.polymer-project.org/2.0/toolbox/
57. Material Design A unified visual design system that combines theory, resources, and tools for crafting digital experiences https://material.io/guidelines/
58. Mobile First Screen size visualisation by Orde Saunders, on Flickr (CC BY 2.0).
59. The Completed Shopping List App Screenshots of the Offline First Shopping List Progressive Web App built using Polymer and PouchDB as it will appear when you have completed this tutorial
60. Initial Set Up ⌨
61. Check for Node.js Version 6 or Higher $ node -v! v8.6.0! https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
62. Install Node.js (if not already installed) § Download https://nodejs.org/en/download/ –or– § Install via a package manager https://nodejs.org/en/download/package-manager/ https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
63. Install Bower $ npm install -g bower! https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
64. Install Polymer CLI $ npm install -g polymer-cli! https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
65. Initialize the Polymer App $ mkdir shopping-list-polymer-pouchdb! $ cd shopping-list-polymer-pouchdb! $ polymer init polymer-2-starter-kit! info: Running template polymer-init-polymer-2-starter-kit:app...! info: Finding latest ^3.0.0 release of PolymerElements/polymer-starter-kit! info: Downloading v3.1.0 of PolymerElements/polymer-starter-kit! info: Unpacking template files! info: Finished writing template files! ! ! I'm all done. Running bower install for you to install the required dependencies. If this fails, try running the command yourself.! https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
66. Start the Polymer Development Server $ polymer serve! info: Files in this directory are available under the following URLs! applications: http://127.0.0.1:8081! reusable components: http://127.0.0.1:8081/components/polymer-starter-kit/! https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
67. Polymer Starter Kit Screenshots of the Polymer Starter Kit app immediately after initialization https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
68. Creating the Polymer App
69. Building the Basic Components
70. The Shopping List App Screenshots of the Shopping List app after updating the app metadata and creating stubs of the shopping lists and shopping list items components https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#building-the-basic-components
71. Building the Basic Components (20 minutes) 1.  Update the app title, description, and related metadata 2.  Remove views from the Starter Kit that we will not be using 3.  Create a stub of the shopping lists component 4.  Create a stub of the shopping list items component https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#building-the-basic-components
72. Adding the Domain Model
74. Immutable.js The Shopping List domain model JavaScript implementation uses Immutable.js Records and Lists https://facebook.github.io/immutable-js/
75. The Shopping List App Screenshots of the Shopping List app with an empty state indicator, a loading spinner, and using the shopping list domain model with stubbed-out data https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#adding-the-shopping-list-domain-model
76. Adding the Domain Model (30 minutes) 1.  Install the shopping list domain model 2.  Create a component to encapsulate the shopping list domain model 3.  Use the shopping list domain model for one-way data binding of shopping lists 4.  Create an empty state indicator for shopping lists 5.  Add stub data to the shopping lists component 6.  Add a loading spinner to the shopping lists component https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#adding-the-shopping-list-domain-model
77. Adding a PouchDB Database
78. The Shopping List App Screenshots of the Shopping List app using PouchDB and pouchdb-find to find a List of Shopping Lists and to create a new Shopping List entity https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#adding-a-pouchdb-database
79. Adding a PouchDB Database (30 minutes) 1.  Install PouchDB and pouchdb-find 2.  Create a component to encapsulate PouchDB with pouchdb-find 3.  Create a shared database property within the app component 4.  Use the shopping list repository to find a list of shopping lists 5.  Use PouchDB to listen for and propagate changes to shopping lists 6.  Create a dialog with a form for creating a new shopping list 7.  Create a new shopping list when the create new shopping list form is submitted https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#adding-a-pouchdb-database
80. Completing the App
81. The Completed Shopping List App Screenshots of the completed Offline First Shopping List Progressive Web App built using Polymer and PouchDB https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#completing-the-app
82. Completing the App (40 minutes) 1.  Use the shopping list domain model for one-way data binding of shopping list items 2.  Create an empty state indicator for shopping list items 3.  Add stub data to the shopping list items component 4.  Add a loading spinner to the shopping list items component 5.  Observe route changes in shopping list items component 6.  Use the shopping list repository to find a list of shopping list items 7.  Use PouchDB to listen for and propagate changes to shopping list items 8.  Create a dialog with a form for creating a new shopping list item 9.  Create a new shopping list item when the create new shopping list item form is submitted 10.  Update database when a list item is checked or unchecked https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#completing-the-app
83. Syncing Data
84. The Shopping List App Screenshots of the Shopping List app replicating with a remote database https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#syncing-data
85. Configure a Database •  Install CouchDB 2.1 •  Configure CouchDB for a single-node setup Option 1: Apache CouchDB •  Sign up for an IBM Bluemix account •  Create a new Cloudant service instance Option 2: IBM Cloudant •  Download the Cloudant Developer Edition from Docker Hub •  Start the Docker container Option 3: Cloudant Developer Edition https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#syncing-data
86. Enable Live Replication with a Remote Database ready() { super.ready(); if (this.remoteDb) { this.replicationHandler = this.db.sync(this.remoteDb, { live: true, retry: true }); this.$.toast.text = "Live replication with remote database started"; this.$.toast.open(); } } ! https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#enable-live-replication-with-a-remote-database
87. What's next?
88. Next Steps for the Shopping List App §  Handle replication conflicts §  Ability to edit and delete Shopping List and Shopping List Item entities §  Multi-user features with Hoodie §  Geolocation features §  Deploy to IBM Bluemix (and use IBM Cloudant)
89. Further Reading and Resources §  Offline Sync for Progressive Web Apps – IBM Watson Data Lab §  Voice of InterConnect – IBM Watson Data Lab §  Deploying the Hoodie Tracker demo app to IBM Bluemix §  Hoodie documentation on storing data with IBM Cloudant §  Offline Camp Medium publication §  Offline First resources §  Offline First on YouTube §  Maureen McElaney Presents Go Offline First to Save The World at JSConf EU 2017 §  Make&Model (consultancy specializing in user experience design for Offline First apps) §  Neighbourhoodie Software (IBM Business Partner specializing in architecting Offline First apps)
90. https://joind.in/talk/31a98 Fin