Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Strategies for Working with Offline Data

Strategies for Working with Offline Data

When your app needs to operate without a network connection, you will need to consider your options for offline or cached data. This session will explore methods for storing data locally and techniques for synchronizing with the backend when your app is online again.

6d0ac0a0024c8bb4ba9cb3eefaabc3f2?s=128

Andrew Duncan

July 18, 2013
Tweet

More Decks by Andrew Duncan

Other Decks in Technology

Transcript

  1. Andrew Duncan, Co-founder, SwarmOnline @andrewmduncan andrew@swarmonline.com Strategies for Working with

    Offline Data
  2. None
  3. Agenda • Application Cache vs Offline Storage • Data storage

    options • Connecting offline storage to the server • Considerations - Managing Conflicts - Deltas - When to Sync - Sync Frameworks - Security/Privacy - Hybrid App Containers - Performance
  4. Application Cache (AppCache) • Application logic • Browser caches files

    • Specified by developer • Advantages - App works without connectivity - Increased load times - Reduced server load • Cache Manifest • Hybrid App Containers - package resources, e.g. static images, large documents etc...
  5. - Eric Bidelman Offline storage is about capturing specific data

    generated by the user, or resources the user has expressed interest in.
  6. Reasons for going o ine

  7. Make the app work o ine

  8. Improve performance

  9. Where can we store our Data?

  10. LocalStorage Cookies WebSQL IndexedDB SessionStorage

  11. Memory • Reads data from local memory • Contents are

    not retained between sessions • For reading only • Useful to help you translate your data into the correct format in your model.
  12. Memory Proxy Ext.define('MyApp.store.Location',  {        extend:  'Ext.data.Store'  

           autoLoad:  true,        model:  'Location',        proxy:  {                type:  'memory',                reader:  {                        type:  'json'                }        } });
  13. SessionStorage Proxy • Extends WebStorage Proxy • SessionStorage property on

    Window object - accessible from window.SessionStorage • String Key/Value Pairs • Lifetime - data deleted when browsing session ends • Scope - document origin - per-window basis • Size limit - system memory
  14. Session Proxy Ext.define('MyApp.store.Location',  {        extend:  'Ext.data.Store'  

           autoLoad:  true,        model:  'Location',        proxy:  {                type:  'sessionstorage',                id:  'LocationsProxy',                reader:  {                        type:  'json'                }        } });
  15. LocalStorage • LocalStorage property on Window object • Lifetime -

    permanent - deleted by the app - deleted by the user • Scope - document origin - protocol, hostname & port must be the same - browser vendor • Size Limit - 5MB
  16. LocalStorage Proxy Ext.define('MyApp.store.Location',  {        extend:  'Ext.data.Store'  

           autoLoad:  true,        model:  'Location',        proxy:  {                type:  'localstorage',                id:  'LocationsProxy',                reader:  {                        type:  'json'                }        } });
  17. WebSQL Database • Proxy added in Sencha Touch 2.1 •

    Size - defaults to 5MB - developer can request more space from user • Lifetime - permanent - deleted by the app - deleted by the user • Scope - document origin - protocol, hostname & port must be the same - browser vendor • Rollbacks & Data Integrity
  18. SQL Proxy Ext.define('MyApp.store.Location',  {        extend:  'Ext.data.Store'  

           autoLoad:  true,        model:  'Location',        proxy:  {                type:  'sql',                database:  'Assets',                table:  'AssetLocations',                reader:  {                        type:  'json'                }        } });
  19. IndexedDB • No SQL • Key-value mapping • Better performance

    through indexes • Limited Support
  20. Enabling O ine Working

  21. Central Storage Local Storage Mobile Application Web Server SYNC

  22. Multiple Proxies may help • Data can be changed in

    two places - In the backend e.g. by external systems - Requires a connection via a Server Proxy - On the client - Requires a Client Proxy • Benefits are more than just offline working - Perceived performance boost as syncing is local and server syncing can be done in the background - Alleviates slow network connectivity or slow backend • We need a way of knowing/merging the data between the proxies
  23. Ext.ux.O ineSyncStore https://github.com/SwarmOnline/Ext.ux.OfflineSyncStore

  24. Overview • Sencha Touch plugin • Extends Ext.data.Store • Two

    proxies - LocalProxy - ServerProxy • Persists a local copy • Keeps a record of added, updated and deleted records • Syncs these changes with your server • Enables you to control when syncing takes place
  25. OfflineSyncStore Ext.data.Store LocalProxy Model Ext.data.Model Client Side Storage ServerProxy Backend

    Systems ModifiedRecordsCollection
  26. Ext.define('Example.store.Person',  {        extend:  'Ext.ux.OfflineSyncStore',      

     config:  {                model:  'Example.model.Person',                localProxy:  {                        type:  'localstorage',                        id:  'offline-­‐sync-­‐store'                },                serverProxy:  {                        type:  'ajax',                        api:  {                                read:  '../../server/select.php',                                create:  '../../server/save.php'                        },                        reader:  {                                type:  'json',                                rootProperty:  'rows'                        },                        writer:  {                                allowSingle:  false                        }                }        } });
  27. Conflicts between the Proxies • Dirty and phantom records are

    no longer known • We keep our own record of these - to avoid overwriting data between the proxies and a server load
  28. ServerProxy is synced New model added to store in ‘offline’

    mode
  29. Managing Conflicts

  30. Conflicts with the Backend • Timestamp all data • Keep

    a version number • Possible Solutions - Last write wins - trust the local time to be truthful - Privilege • Application dependent, no right or wrong answer
  31. When Should I Sync?

  32. When Should I Sync? • Try all the time •

    Ajax request will fail if no connectivity. • Stack up failures and resend • Time driven • Event/Action driven • Monitoring Connectivity • Phonegap • navigator.connection.type • Sencha Touch • Ext.device.Connection • HTML5 • navigator.onLine • definition of ‘online’ varies • local network or internet • Solution is detect a heartbeat Poor Connectivity As soon as connectivity is regained? Good Connectivity All the time?
  33. Data Loss During the Sync • Problem - New data

    can be lost if the store is currently ‘syncing’ • Cause - Slower & older devices - large datasets • Possible Solution - Pause user interaction while sync is in progress. - Framework Override to stop the date clear happening so early.
  34. Cross Platform Considerations

  35. Deltas

  36. Deltas • Improves performance / Reduces Bandwidth • Require your

    app to have knowledge of when its data was last updated • Server only sends newly updated between the current time and the last sync • Data is merged with store • Sencha Touch handles this well for us already
  37. Other tools that might help?

  38. Is the hardwork done for us? • RhoConnect • WebSqlSync

    - https://github.com/orbitaloop/WebSqlSync/ • PouchDB
  39. Storage, Privacy & Security Considerations

  40. HTML5 Storage Security • Do not store session identifiers -

    Cookies can use the httpOnly flag - SessionStorage might be ok for • Encryption is an option - Stanford Javascript Crypto Library - http://crypto.stanford.edu/sjcl/ - AES - Override the proxy
  41. Hybrid App Containers • RhoMobile - Rhom Local database -

    SQLite - Database encryption • PhoneGap - Use SQLLite Plugin - SQLCipher - Open Source - 256-bit encryption - http://brodyspark.blogspot.co.uk/ - Don’t store the key - derive from users password • Sencha Space
  42. Remote Wiping Data • Use a mobile device management suite

    - AirWatch - Soti MobiControl • Sencha Space
  43. None
  44. Hackathon

  45. Take the Survey! • Session Survey - Available on the

    SenchaCon mobile app - http://app.senchacon.com • Be Social! - @SenchaCon - #SenchaCon - @andrewmduncan
  46. Questions?