Andrew Duncan, Co-founder, SwarmOnline @andrewmduncan andrew@swarmonline.com Strategies for Working with Offline Data

No content

Agenda • Application Cache vs Offline Storage • Data storage options • Connecting offline storage to the server • Considerations - Managing Conflicts - Deltas - When to Sync - Sync Frameworks - Security/Privacy - Hybrid App Containers - Performance

Application Cache (AppCache) • Application logic • Browser caches files • Specified by developer • Advantages - App works without connectivity - Increased load times - Reduced server load • Cache Manifest • Hybrid App Containers - package resources, e.g. static images, large documents etc...

- Eric Bidelman Offline storage is about capturing specific data generated by the user, or resources the user has expressed interest in.

Reasons for going o ine

Make the app work o ine

Improve performance

Where can we store our Data?

LocalStorage Cookies WebSQL IndexedDB SessionStorage

Memory • Reads data from local memory • Contents are not retained between sessions • For reading only • Useful to help you translate your data into the correct format in your model.

Memory Proxy Ext.define('MyApp.store.Location',  {        extend:  'Ext.data.Store'          autoLoad:  true,        model:  'Location',        proxy:  {                type:  'memory',                reader:  {                        type:  'json'                }        } });

SessionStorage Proxy • Extends WebStorage Proxy • SessionStorage property on Window object - accessible from window.SessionStorage • String Key/Value Pairs • Lifetime - data deleted when browsing session ends • Scope - document origin - per-window basis • Size limit - system memory

Session Proxy Ext.define('MyApp.store.Location',  {        extend:  'Ext.data.Store'          autoLoad:  true,        model:  'Location',        proxy:  {                type:  'sessionstorage',                id:  'LocationsProxy',                reader:  {                        type:  'json'                }        } });

LocalStorage • LocalStorage property on Window object • Lifetime - permanent - deleted by the app - deleted by the user • Scope - document origin - protocol, hostname & port must be the same - browser vendor • Size Limit - 5MB

LocalStorage Proxy Ext.define('MyApp.store.Location',  {        extend:  'Ext.data.Store'          autoLoad:  true,        model:  'Location',        proxy:  {                type:  'localstorage',                id:  'LocationsProxy',                reader:  {                        type:  'json'                }        } });

WebSQL Database • Proxy added in Sencha Touch 2.1 • Size - defaults to 5MB - developer can request more space from user • Lifetime - permanent - deleted by the app - deleted by the user • Scope - document origin - protocol, hostname & port must be the same - browser vendor • Rollbacks & Data Integrity

SQL Proxy Ext.define('MyApp.store.Location',  {        extend:  'Ext.data.Store'          autoLoad:  true,        model:  'Location',        proxy:  {                type:  'sql',                database:  'Assets',                table:  'AssetLocations',                reader:  {                        type:  'json'                }        } });

IndexedDB • No SQL • Key-value mapping • Better performance through indexes • Limited Support

Enabling O ine Working

Central Storage Local Storage Mobile Application Web Server SYNC

Multiple Proxies may help • Data can be changed in two places - In the backend e.g. by external systems - Requires a connection via a Server Proxy - On the client - Requires a Client Proxy • Benefits are more than just offline working - Perceived performance boost as syncing is local and server syncing can be done in the background - Alleviates slow network connectivity or slow backend • We need a way of knowing/merging the data between the proxies

Ext.ux.O ineSyncStore https://github.com/SwarmOnline/Ext.ux.OfflineSyncStore

Overview • Sencha Touch plugin • Extends Ext.data.Store • Two proxies - LocalProxy - ServerProxy • Persists a local copy • Keeps a record of added, updated and deleted records • Syncs these changes with your server • Enables you to control when syncing takes place

OfflineSyncStore Ext.data.Store LocalProxy Model Ext.data.Model Client Side Storage ServerProxy Backend Systems ModifiedRecordsCollection

Ext.define('Example.store.Person',  {        extend:  'Ext.ux.OfflineSyncStore',        config:  {                model:  'Example.model.Person',                localProxy:  {                        type:  'localstorage',                        id:  'offline-­‐sync-­‐store'                },                serverProxy:  {                        type:  'ajax',                        api:  {                                read:  '../../server/select.php',                                create:  '../../server/save.php'                        },                        reader:  {                                type:  'json',                                rootProperty:  'rows'                        },                        writer:  {                                allowSingle:  false                        }                }        } });

Conflicts between the Proxies • Dirty and phantom records are no longer known • We keep our own record of these - to avoid overwriting data between the proxies and a server load

ServerProxy is synced New model added to store in ‘offline’ mode

Managing Conflicts

Conflicts with the Backend • Timestamp all data • Keep a version number • Possible Solutions - Last write wins - trust the local time to be truthful - Privilege • Application dependent, no right or wrong answer

When Should I Sync?

When Should I Sync? • Try all the time • Ajax request will fail if no connectivity. • Stack up failures and resend • Time driven • Event/Action driven • Monitoring Connectivity • Phonegap • navigator.connection.type • Sencha Touch • Ext.device.Connection • HTML5 • navigator.onLine • definition of ‘online’ varies • local network or internet • Solution is detect a heartbeat Poor Connectivity As soon as connectivity is regained? Good Connectivity All the time?

Data Loss During the Sync • Problem - New data can be lost if the store is currently ‘syncing’ • Cause - Slower & older devices - large datasets • Possible Solution - Pause user interaction while sync is in progress. - Framework Override to stop the date clear happening so early.

Cross Platform Considerations

Deltas

Deltas • Improves performance / Reduces Bandwidth • Require your app to have knowledge of when its data was last updated • Server only sends newly updated between the current time and the last sync • Data is merged with store • Sencha Touch handles this well for us already

Other tools that might help?

Is the hardwork done for us? • RhoConnect • WebSqlSync - https://github.com/orbitaloop/WebSqlSync/ • PouchDB

Storage, Privacy & Security Considerations

HTML5 Storage Security • Do not store session identifiers - Cookies can use the httpOnly flag - SessionStorage might be ok for • Encryption is an option - Stanford Javascript Crypto Library - http://crypto.stanford.edu/sjcl/ - AES - Override the proxy

Hybrid App Containers • RhoMobile - Rhom Local database - SQLite - Database encryption • PhoneGap - Use SQLLite Plugin - SQLCipher - Open Source - 256-bit encryption - http://brodyspark.blogspot.co.uk/ - Don’t store the key - derive from users password • Sencha Space

Remote Wiping Data • Use a mobile device management suite - AirWatch - Soti MobiControl • Sencha Space

No content

Hackathon

Take the Survey! • Session Survey - Available on the SenchaCon mobile app - http://app.senchacon.com • Be Social! - @SenchaCon - #SenchaCon - @andrewmduncan

Questions?