Tweet
Tweet

Slide 1

Slide 1 text

Andrew Duncan, Co-founder, SwarmOnline @andrewmduncan [email protected] Strategies for Working with Offline Data

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

Agenda • Application Cache vs Offline Storage • Data storage options • Connecting offline storage to the server • Considerations - Managing Conflicts - Deltas - When to Sync - Sync Frameworks - Security/Privacy - Hybrid App Containers - Performance

Slide 4

Slide 4 text

Application Cache (AppCache) • Application logic • Browser caches files • Specified by developer • Advantages - App works without connectivity - Increased load times - Reduced server load • Cache Manifest • Hybrid App Containers - package resources, e.g. static images, large documents etc...

Slide 5

Slide 5 text

- Eric Bidelman Offline storage is about capturing specific data generated by the user, or resources the user has expressed interest in.

Slide 6

Slide 6 text

Reasons for going o ine

Slide 7

Slide 7 text

Make the app work o ine

Slide 8

Slide 8 text

Improve performance

Slide 9

Slide 9 text

Where can we store our Data?

Slide 10

Slide 10 text

LocalStorage Cookies WebSQL IndexedDB SessionStorage

Slide 11

Slide 11 text

Memory • Reads data from local memory • Contents are not retained between sessions • For reading only • Useful to help you translate your data into the correct format in your model.

Slide 12

Slide 12 text

Memory Proxy Ext.define('MyApp.store.Location',  {        extend:  'Ext.data.Store'          autoLoad:  true,        model:  'Location',        proxy:  {                type:  'memory',                reader:  {                        type:  'json'                }        } });

Slide 13

Slide 13 text

SessionStorage Proxy • Extends WebStorage Proxy • SessionStorage property on Window object - accessible from window.SessionStorage • String Key/Value Pairs • Lifetime - data deleted when browsing session ends • Scope - document origin - per-window basis • Size limit - system memory

Slide 14

Slide 14 text

Session Proxy Ext.define('MyApp.store.Location',  {        extend:  'Ext.data.Store'          autoLoad:  true,        model:  'Location',        proxy:  {                type:  'sessionstorage',                id:  'LocationsProxy',                reader:  {                        type:  'json'                }        } });

Slide 15

Slide 15 text

LocalStorage • LocalStorage property on Window object • Lifetime - permanent - deleted by the app - deleted by the user • Scope - document origin - protocol, hostname & port must be the same - browser vendor • Size Limit - 5MB

Slide 16

Slide 16 text

LocalStorage Proxy Ext.define('MyApp.store.Location',  {        extend:  'Ext.data.Store'          autoLoad:  true,        model:  'Location',        proxy:  {                type:  'localstorage',                id:  'LocationsProxy',                reader:  {                        type:  'json'                }        } });

Slide 17

Slide 17 text

WebSQL Database • Proxy added in Sencha Touch 2.1 • Size - defaults to 5MB - developer can request more space from user • Lifetime - permanent - deleted by the app - deleted by the user • Scope - document origin - protocol, hostname & port must be the same - browser vendor • Rollbacks & Data Integrity

Slide 18

Slide 18 text

SQL Proxy Ext.define('MyApp.store.Location',  {        extend:  'Ext.data.Store'          autoLoad:  true,        model:  'Location',        proxy:  {                type:  'sql',                database:  'Assets',                table:  'AssetLocations',                reader:  {                        type:  'json'                }        } });

Slide 19

Slide 19 text

IndexedDB • No SQL • Key-value mapping • Better performance through indexes • Limited Support

Slide 20

Slide 20 text

Enabling O ine Working

Slide 21

Slide 21 text

Central Storage Local Storage Mobile Application Web Server SYNC

Slide 22

Slide 22 text

Multiple Proxies may help • Data can be changed in two places - In the backend e.g. by external systems - Requires a connection via a Server Proxy - On the client - Requires a Client Proxy • Benefits are more than just offline working - Perceived performance boost as syncing is local and server syncing can be done in the background - Alleviates slow network connectivity or slow backend • We need a way of knowing/merging the data between the proxies

Slide 23

Slide 23 text

Ext.ux.O ineSyncStore https://github.com/SwarmOnline/Ext.ux.OfflineSyncStore

Slide 24

Slide 24 text

Overview • Sencha Touch plugin • Extends Ext.data.Store • Two proxies - LocalProxy - ServerProxy • Persists a local copy • Keeps a record of added, updated and deleted records • Syncs these changes with your server • Enables you to control when syncing takes place

Slide 25

Slide 25 text

OfflineSyncStore Ext.data.Store LocalProxy Model Ext.data.Model Client Side Storage ServerProxy Backend Systems ModifiedRecordsCollection

Slide 26

Slide 26 text

Ext.define('Example.store.Person',  {        extend:  'Ext.ux.OfflineSyncStore',        config:  {                model:  'Example.model.Person',                localProxy:  {                        type:  'localstorage',                        id:  'offline-­‐sync-­‐store'                },                serverProxy:  {                        type:  'ajax',                        api:  {                                read:  '../../server/select.php',                                create:  '../../server/save.php'                        },                        reader:  {                                type:  'json',                                rootProperty:  'rows'                        },                        writer:  {                                allowSingle:  false                        }                }        } });

Slide 27

Slide 27 text

Conflicts between the Proxies • Dirty and phantom records are no longer known • We keep our own record of these - to avoid overwriting data between the proxies and a server load

Slide 28

Slide 28 text

ServerProxy is synced New model added to store in ‘offline’ mode

Slide 29

Slide 29 text

Managing Conflicts

Slide 30

Slide 30 text

Conflicts with the Backend • Timestamp all data • Keep a version number • Possible Solutions - Last write wins - trust the local time to be truthful - Privilege • Application dependent, no right or wrong answer

Slide 31

Slide 31 text

When Should I Sync?

Slide 32

Slide 32 text

When Should I Sync? • Try all the time • Ajax request will fail if no connectivity. • Stack up failures and resend • Time driven • Event/Action driven • Monitoring Connectivity • Phonegap • navigator.connection.type • Sencha Touch • Ext.device.Connection • HTML5 • navigator.onLine • definition of ‘online’ varies • local network or internet • Solution is detect a heartbeat Poor Connectivity As soon as connectivity is regained? Good Connectivity All the time?

Slide 33

Slide 33 text

Data Loss During the Sync • Problem - New data can be lost if the store is currently ‘syncing’ • Cause - Slower & older devices - large datasets • Possible Solution - Pause user interaction while sync is in progress. - Framework Override to stop the date clear happening so early.

Slide 34

Slide 34 text

Cross Platform Considerations

Slide 35

Slide 35 text

Deltas

Slide 36

Slide 36 text

Deltas • Improves performance / Reduces Bandwidth • Require your app to have knowledge of when its data was last updated • Server only sends newly updated between the current time and the last sync • Data is merged with store • Sencha Touch handles this well for us already

Slide 37

Slide 37 text

Other tools that might help?

Slide 38

Slide 38 text

Is the hardwork done for us? • RhoConnect • WebSqlSync - https://github.com/orbitaloop/WebSqlSync/ • PouchDB

Slide 39

Slide 39 text

Storage, Privacy & Security Considerations

Slide 40

Slide 40 text

HTML5 Storage Security • Do not store session identifiers - Cookies can use the httpOnly flag - SessionStorage might be ok for • Encryption is an option - Stanford Javascript Crypto Library - http://crypto.stanford.edu/sjcl/ - AES - Override the proxy

Slide 41

Slide 41 text

Hybrid App Containers • RhoMobile - Rhom Local database - SQLite - Database encryption • PhoneGap - Use SQLLite Plugin - SQLCipher - Open Source - 256-bit encryption - http://brodyspark.blogspot.co.uk/ - Don’t store the key - derive from users password • Sencha Space

Slide 42

Slide 42 text

Remote Wiping Data • Use a mobile device management suite - AirWatch - Soti MobiControl • Sencha Space

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

Hackathon

Slide 45

Slide 45 text

Take the Survey! • Session Survey - Available on the SenchaCon mobile app - http://app.senchacon.com • Be Social! - @SenchaCon - #SenchaCon - @andrewmduncan

Slide 46

Slide 46 text

Questions?