Offline Sync for Progressive Web Apps Workshop at ZendCon

With the introduction of Progressive Web Apps and web platform APIs such as persistent storage, payments, and geolocation, the web platform is taking on native apps. A Progressive Web App combines the best parts both a web app and a native app. You browse to a Progressive Web App just like you browse to any other website. It lives at a URL, can be indexed by search engines, and can be linked to from anywhere else on the web. As you continue to use a Progressive Web App it gains additional native app-like capabilities. For example, the app could be installed to the home screen on your device. You might also grant the app the ability to send you push notifications, or the ability to access your camera, your microphone, or other device resources.

One important aspect of Progressive Web Apps is the concept of building your app to be Offline First. With an Offline First approach, you design your app for the most resource-constrained environment first. This approach provides a consistent user experience whether the user's device has no connectivity, limited connectivity, or great connectivity. One of the biggest benefits of Offline First apps is that they can be very fast, as they provide zero-latency access to content and data stored directly on the device.

The Service Worker API can do most of the heavy lifting when it comes to storing content and assets for Offline First Progressive Web Apps. A bigger challenge can be storing and sync'ing your app's data. One of the best answers today for offline sync for Progressive Web Apps is a combination of Apache CouchDB (an open source document database), Hoodie (an open source Node.js backend for Offline First apps), and PouchDB (an open source JavaScript database that syncs). This stack can provide you with the starting point for your own Progressive Web App mobile backend and frontend, and it's entirely open source! You'll leave this workshop having built a fully-functional Offline First Progressive Web Apps using Polymer, Web Components, Service Workers, Apache CouchDB, Hoodie, and PouchDB.

Transcript

1. ZendCon
   Bradley Holt, Developer Advocate
   Monday, October 23, 2017
   Offline Sync for
   Progressive Web Apps
   ⌨
   

   View Slide

2. View Slide

3. Prerequisite Knowledge & Skills
   §  Ability to write JavaScript at a novice level, at minimum
   §  A basic understanding of JavaScript promises
   https://pouchdb.com/2015/05/18/we-have-a-problem-with-promises.html
   §  A basic understanding of HTML
   §  Ability to work with an application programming interface (API)
   https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb
   

   View Slide

4. Tutorial Outline
   Background
   Key Concepts
   Initial Set Up ⌨
   Creating the Polymer App
   Syncing Data
   What's next?
   

   View Slide

5. Background
   

   View Slide

6. https://mobilefirstplatform.ibmcloud.com/blog/2016/01/13/ionic-hybrid-mobile-app-using-mobilefirst-platform-7-1-cli/
   Native
   Mobile
   Apps
   

   View Slide

7. organic growth by Steve Jurvetson, on Flickr (CC BY 2.0).
   The
   Web
   Platform
   

   View Slide

8. W3C HTML5 Logo (CC BY 3.0).
   

   View Slide

9. Mobile First
   & the
   Mobile Web
   Screen size visualisation by Orde Saunders, on Flickr (CC BY 2.0).
   

   View Slide

10. https://responsivedesign.is/examples/css-conf-europe/
    

    View Slide

11. Evergreen by Basheer Tome, on Flickr (CC BY 2.0).
    Evergreen
    Browsers
    

    View Slide

12. mettalic weave by Joel Ormsby, on Flickr (CC BY 2.0).
    Polyfills
    

    View Slide

13. By Diseño Web Valencia, via Wikimedia Commons (CC BY-SA 4.0).
    

    View Slide

14. View Slide

15. https://doesmysiteneedhttps.com/
    

    View Slide

16. Percentage of Web Pages Loaded by Firefox Using HTTPS
    https://letsencrypt.org/stats/
    

    View Slide

17. Web or
    Mobile
    App?
    iPhone 4 - Bottom View with Bumper by William Hook, on Flickr (CC BY-SA 2.0).
    

    View Slide

18. http://gph.is/2dhW35c
    

    View Slide

19. Progressive
    Web Apps
    (or how to have your cake and eat it too)
    German Chocolate Cake by Kimberly Vardeman, on Flickr (CC BY 2.0).
    

    View Slide

20. Progressive Web Apps
    A Progressive Web App provides both the discoverability of a web app and the
    reliable, fast, and engaging user experience of a native mobile app
    https://www.pokedex.org/
    

    View Slide

21. "I don’t want the web to equal native; I want the web to
    surpass it. I, for one, would prefer a reality where my home screen
    isn’t filled with the icons of startups and companies that have
    fulfilled the criteria of the gatekeepers. But a home screen filled
    with the faces of people who didn’t have to ask anyone’s
    permission to publish? That’s what I want!" –Jeremy Keith
    Adactio: Journal—Progressing the web | Jeremy Keith by Matthew Oliphant, on Flickr (CC BY-ND 2.0).
    

    View Slide

22. https://www.pwastats.com/
    

    View Slide

23. https://developers.google.com/web/showcase/2017/twitter
    

    View Slide

24. Geolocation API
    Network Information API
    IndexedDB
    WebRTC API
    Push API
    Notifications API
    Device Orientation API
    Screen Orientation API
    Permissions API
    Proximity API
    Pointer Lock API
    Vibration API
    Payment Request API
    Battery Status API
    Ambient Light Sensor API
    Service Worker API
    IBM Design Icons by International Business Machines Corporation (CC BY 4.0).
    

    View Slide

25. Offline First
    Progressive Web Apps must be Offline First in order to provide a reliable, fast,
    and engaging user experience regardless of network connectivity
    No Signal by Steve Hodgson, on Flickr (CC BY-SA 2.0).
    

    View Slide

26. speed 3 by Monkieyes, on Flickr (CC BY 2.0).
    Zero Latency
    Access to
    Content & Data
    

    View Slide

27. Offline, Online & ¯\_(ツ)_/¯
    

    View Slide

28. The Next
    Billion
    Internet Users
    the galaxy by tommy@chau, on Flickr (CC BY 2.0).
    

    View Slide

29. Cache
    content
    & assets
    with
    Service
    Workers
    Service Workers fetch diagram by Mozilla Contributors (CC-BY-SA 2.5).
    

    View Slide

30. IndexedDB
    Use IndexedDB or localForage (a polyfill that uses WebSQL or localStorage if
    IndexedDB is not supported) to make application data available while offline
    https://caniuse.com/#feat=indexeddb
    

    View Slide

31. Technological devices design created by D3images - Freepik.com | Texture 94 by Ellen van Deelen, on Flickr (CC BY 2.0).
    Sync
    is
    Hard
    

    View Slide

32. IBM Cloudant
    Apache CouchDB
    PouchDB
    

    View Slide

33. "CouchDB’s superpower is sync. Sometimes I even try to explain it
    to people by saying, 'CouchDB isn’t a database; it’s a sync
    engine.' It’s a way of efficiently transferring data from one place to
    another, while intelligently managing conflicts and revisions. It’s
    very similar to Git. When I make that analogy, the light bulb often
    goes off." –Nolan Lawson
    PouchDB & CouchDB: An interview with Nolan Lawson – CouchDB Blog
    

    View Slide

34. Apache CouchDB
    An open source document database featuring an HTTP API, JSON documents,
    clustering capabilities for horizontal scalability, and peer-to-peer replication
    http://couchdb.apache.org/
    

    View Slide

35. PouchDB
    An open source JavaScript database that syncs with anything that implements
    the CouchDB Replication Protocol
    https://pouchdb.com/
    

    View Slide

36. IBM Cloudant
    A fully-managed database-as-a-service (DBaaS) based on Apache CouchDB
    with additional full text and geospatial search capabilities
    https://www.ibm.com/analytics/us/en/technology/cloud-data-services/cloudant/
    

    View Slide

37. Apache CouchDB
    Devices by Darwin Laganzon, on Pixabay (CC0 Public Domain). | Cloud by Raphael Silva, on Pixabay (CC0 Public Domain).
    PouchDB
    Browser-to-
    Cloud Sync
    

    View Slide

38. userdb-d76846
    userdb-905cec
    userdb-adc95b
    userdb-c082f2
    userdb-730bba
    userdb-c3d3e5
    userdb-a1ec1f
    userdb-85a327
    userdb-9b9aba
    userdb-85bcfe
    userdb-da3d25
    One Database
    Per User
    database by Tim Morgan, on Flickr (CC BY 2.0). userdb-43ecb5
    userdb-8a5dfa
    

    View Slide

39. View Slide

40. Apache CouchDB
    Devices by Darwin Laganzon, on Pixabay (CC0 Public Domain). | Cloud by Raphael Silva, on Pixabay (CC0 Public Domain).
    PouchDB
    

    View Slide

41. Summary
    Build on the web platform with Progressive Web Apps
    Offline First makes for a reliable, fast, and engaging user experience
    Cache URL-addressable resources and content with Service Workers
    Store app data in PouchDB to easily sync your Offline First data
    Use Hoodie as the backend for your multi-user Offline First app
    

    View Slide

42. Tiny Steps by Jim Whimpey, on Flickr (CC BY 2.0).
    Progressive
    Web Apps,
    Progressively
    

    View Slide

43. Progressive Web Apps, Progressively
    Ensure your website is mobile friendly
    Improve an existing website
    Create a native app-like experience
    Leverage the web platform
    

    View Slide

44. Progressive Web Apps, Progressively
    •  Implement responsive web design
    •  Ensure cross-browser compatibility
    •  Test and improve web performance
    Ensure your website is mobile friendly
    Improve an existing website
    Create a native app-like experience
    Leverage the web platform
    

    View Slide

45. Progressive Web Apps, Progressively
    Ensure your website is mobile friendly
    •  Use an SSL/TLS certificate (Let's Encrypt is your friend)
    •  Add a Web App Manifest (name, author, icon, description, etc.)
    •  Use a Service Worker to make your app cache-first
    Improve an existing website
    Create a native app-like experience
    Leverage the web platform
    

    View Slide

46. Progressive Web Apps, Progressively
    Ensure your website is mobile friendly
    Improve an existing website
    •  Provide instant page transitions with a single-page (client-rendered) app
    •  Minimize time-to-interactive with the PRPL pattern
    •  Use a local database (IndexedDB, localForage, or PouchDB)
    Create a native app-like experience
    Leverage the web platform
    

    View Slide

47. Progressive Web Apps, Progressively
    Ensure your website is mobile friendly
    Improve an existing website
    Create a native app-like experience
    •  Ensure that each page can be addressed via a URL
    •  Implement server-side rendering for users without JavaScript
    •  Build a hybrid mobile app with a tool like Ionic or Apache Cordova
    Leverage the web platform
    

    View Slide

48. Offline Camp
    From the Catskill Mountains, to the Central Coast of California, to Berlin, we're building the
    Offline First community, one campfire at a time
    Image credit: Aaron Ross
    

    View Slide

49. View Slide

50. Get Involved in the Offline First Community!
    §  Join the Offline First Slack team:
    offlinefirst.org/chat/
    §  Follow @OfflineCamp on Twitter
    §  Read the Offline Camp Medium publication:
    medium.com/offline-camp
    §  Join us at an upcoming Offline Camp:
    http://offlinefirst.org/camp/
    

    View Slide

51. Key Concepts
    

    View Slide

52. Shopping List Offline First Demo Apps (work in progress)
    A series of demo Progressive Web Apps, hybrid mobile apps, native mobile
    apps, and a desktop app—each built using a different stack
    https://github.com/ibm-watson-data-lab/shopping-list
    

    View Slide

53. https://github.com/ibm-watson-data-lab/shopping-list
    

    View Slide

54. Polymer
    Libraries, tools, and patterns for building Progressive Web Apps using web
    platform features such as Web Components, Service Workers, and HTTP/2
    https://www.polymer-project.org/
    

    View Slide

55. Web Components
    Open standard for components and widgets that are customizable, reusable,
    and encapsulated
    https://www.webcomponents.org/
    

    View Slide

56. Polymer App Toolbox
    Components, tools, and templates for building Progressive Web Apps
    with Polymer and Web Components
    https://www.polymer-project.org/2.0/toolbox/
    

    View Slide

57. Material Design
    A unified visual design system that combines theory, resources, and tools for
    crafting digital experiences
    https://material.io/guidelines/
    

    View Slide

58. Mobile First
    Screen size visualisation by Orde Saunders, on Flickr (CC BY 2.0).
    

    View Slide

59. The Completed Shopping List App
    Screenshots of the Offline First Shopping List Progressive Web App built using
    Polymer and PouchDB as it will appear when you have completed this tutorial
    

    View Slide

60. Initial Set Up ⌨
    

    View Slide

61. Check for Node.js Version 6 or Higher
    $ node -v!
    v8.6.0!
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
    

    View Slide

62. Install Node.js (if not already installed)
    § Download
    https://nodejs.org/en/download/
    –or–
    § Install via a package manager
    https://nodejs.org/en/download/package-manager/
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
    

    View Slide

63. Install Bower
    $ npm install -g bower!
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
    

    View Slide

64. Install Polymer CLI
    $ npm install -g polymer-cli!
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
    

    View Slide

65. Initialize the Polymer App
    $ mkdir shopping-list-polymer-pouchdb!
    $ cd shopping-list-polymer-pouchdb!
    $ polymer init polymer-2-starter-kit!
    info: Running template polymer-init-polymer-2-starter-kit:app...!
    info: Finding latest ^3.0.0 release of PolymerElements/polymer-starter-kit!
    info: Downloading v3.1.0 of PolymerElements/polymer-starter-kit!
    info: Unpacking template files!
    info: Finished writing template files!
    !
    !
    I'm all done. Running bower install for you to install the required
    dependencies. If this fails, try running the command yourself.!
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
    

    View Slide

66. Start the Polymer Development Server
    $ polymer serve!
    info: Files in this directory are available under the following URLs!
    applications: http://127.0.0.1:8081!
    reusable components: http://127.0.0.1:8081/components/polymer-starter-kit/!
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
    

    View Slide

67. Polymer Starter Kit
    Screenshots of the Polymer Starter Kit app immediately after initialization
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#initial-set-up
    

    View Slide

68. Creating the Polymer App
    

    View Slide

69. Building the Basic Components
    

    View Slide

70. The Shopping List App
    Screenshots of the Shopping List app after updating the app metadata and
    creating stubs of the shopping lists and shopping list items components
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#building-the-basic-components
    

    View Slide

71. Building the Basic Components (20 minutes)
    1.  Update the app title, description, and related metadata
    2.  Remove views from the Starter Kit that we will not be using
    3.  Create a stub of the shopping lists component
    4.  Create a stub of the shopping list items component
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#building-the-basic-components
    

    View Slide

72. Adding the Domain Model
    

    View Slide

73. View Slide

74. Immutable.js
    The Shopping List domain model JavaScript implementation uses Immutable.js
    Records and Lists
    https://facebook.github.io/immutable-js/
    

    View Slide

75. The Shopping List App
    Screenshots of the Shopping List app with an empty state indicator, a loading
    spinner, and using the shopping list domain model with stubbed-out data
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#adding-the-shopping-list-domain-model
    

    View Slide

76. Adding the Domain Model (30 minutes)
    1.  Install the shopping list domain model
    2.  Create a component to encapsulate the shopping list domain model
    3.  Use the shopping list domain model for one-way data binding of shopping lists
    4.  Create an empty state indicator for shopping lists
    5.  Add stub data to the shopping lists component
    6.  Add a loading spinner to the shopping lists component
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#adding-the-shopping-list-domain-model
    

    View Slide

77. Adding a PouchDB Database
    

    View Slide

78. The Shopping List App
    Screenshots of the Shopping List app using PouchDB and pouchdb-find to find a
    List of Shopping Lists and to create a new Shopping List entity
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#adding-a-pouchdb-database
    

    View Slide

79. Adding a PouchDB Database (30 minutes)
    1.  Install PouchDB and pouchdb-find
    2.  Create a component to encapsulate PouchDB with pouchdb-find
    3.  Create a shared database property within the app component
    4.  Use the shopping list repository to find a list of shopping lists
    5.  Use PouchDB to listen for and propagate changes to shopping lists
    6.  Create a dialog with a form for creating a new shopping list
    7.  Create a new shopping list when the create new shopping list form is submitted
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#adding-a-pouchdb-database
    

    View Slide

80. Completing the App
    

    View Slide

81. The Completed Shopping List App
    Screenshots of the completed Offline First Shopping List Progressive Web App
    built using Polymer and PouchDB
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#completing-the-app
    

    View Slide

82. Completing the App (40 minutes)
    1.  Use the shopping list domain model for one-way data binding of shopping list items
    2.  Create an empty state indicator for shopping list items
    3.  Add stub data to the shopping list items component
    4.  Add a loading spinner to the shopping list items component
    5.  Observe route changes in shopping list items component
    6.  Use the shopping list repository to find a list of shopping list items
    7.  Use PouchDB to listen for and propagate changes to shopping list items
    8.  Create a dialog with a form for creating a new shopping list item
    9.  Create a new shopping list item when the create new shopping list item form is submitted
    10.  Update database when a list item is checked or unchecked
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#completing-the-app
    

    View Slide

83. Syncing Data
    

    View Slide

84. The Shopping List App
    Screenshots of the Shopping List app replicating with a remote database
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#syncing-data
    

    View Slide

85. Configure a Database
    •  Install CouchDB 2.1
    •  Configure CouchDB for a single-node setup
    Option 1: Apache CouchDB
    •  Sign up for an IBM Bluemix account
    •  Create a new Cloudant service instance
    Option 2: IBM Cloudant
    •  Download the Cloudant Developer Edition from Docker Hub
    •  Start the Docker container
    Option 3: Cloudant Developer Edition
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#syncing-data
    

    View Slide

86. Enable Live Replication with a Remote Database
    ready() {
    super.ready();
    if (this.remoteDb) {
    this.replicationHandler = this.db.sync(this.remoteDb, {
    live: true,
    retry: true
    });
    this.$.toast.text = "Live replication with remote database started";
    this.$.toast.open();
    }
    }
    !
    https://github.com/ibm-watson-data-lab/shopping-list-polymer-pouchdb#enable-live-replication-with-a-remote-database
    

    View Slide

87. What's next?
    

    View Slide

88. Next Steps for the Shopping List App
    §  Handle replication conflicts
    §  Ability to edit and delete Shopping List and Shopping List Item entities
    §  Multi-user features with Hoodie
    §  Geolocation features
    §  Deploy to IBM Bluemix (and use IBM Cloudant)
    

    View Slide

89. Further Reading and Resources
    §  Offline Sync for Progressive Web Apps – IBM Watson Data Lab
    §  Voice of InterConnect – IBM Watson Data Lab
    §  Deploying the Hoodie Tracker demo app to IBM Bluemix
    §  Hoodie documentation on storing data with IBM Cloudant
    §  Offline Camp Medium publication
    §  Offline First resources
    §  Offline First on YouTube
    §  Maureen McElaney Presents Go Offline First to Save The World at JSConf EU 2017
    §  Make&Model (consultancy specializing in user experience design for Offline First apps)
    §  Neighbourhoodie Software (IBM Business Partner specializing in architecting Offline First apps)
    

    View Slide

90. https://joind.in/talk/31a98
    Fin
    

    View Slide