Immer und überall: offlinefähige Progressive Web Apps – am Beispiel Angular

"Keine Internetverbindung" – ein Satz, den wir alle aus genutzten Apps kennen, sei es Desktopanwendung oder Smartphone App. Oftmals sind die Clients auf dem jeweiligen Zielgerät installiert, bedienen sich aber einer externen Datenquelle, bspw. in Form eines Web API. Bei einer offline-fähigen Implementierung sind nicht nur Lese-, sondern vor allem auch Schreibzugriffe wichtig. Sprich, die Daten müssen in beide Richtungen synchronisiert werden können, sobald die Internetverbindung wiederhergestellt wurde. Gerade in der heutigen Zeit, in der Menschen immer mobiler werden, in Autos, Zügen und Flugzeugen unterwegs sind und arbeiten, benötigt man oftmals offline Zugriff auf die Anwendungsdaten. Wie man diese echt offline-fähigen Single-Page Applications/Progressive Web Apps realisiert, zeigen Thomas Hilzendegen und Manuel Rauber von Thinktecture in diesem Ganztagesworkshop. Allgemein werden Themen wie Synchronisationsstrategien von Anwendungsdaten und großen Binärdaten, Konfliktmanagement oder Datenbankarchitektur sowohl für neue als auch bestehende Anwendungen diskutiert. Am Beispiel einer Angular-basierten App mit einem exemplarischen Backend in .NET Core werden Client und Server zur Umsetzung der besprochenen Konzepte und Patterns implementiert.

GitHub: https://github.com/thinktecture/thinktecture-boardist

Transcript

1. Offlinefähige Progressive Web Apps Immer & überall Thomas Hilzendegen @hilzendegent Consultant Manuel Rauber @ManuelRauber Consultant
2. Thomas Hilzendegen Speakers Manuel Rauber thomas.hilzendegen@thinktecture.com @hilzendegent manuel.rauber@thinktecture.com @manuelrauber https://manuel-rauber.com Microsoft MVP
3. Timetable Time Doing 09:00 - 10:30 Part I 10:30 - 11:00 ☕ 11:00 - 12:30 Part II 12:30 - 13:30 13:30 - 15:00 Part III 15:00 - 15:30 ☕ 15:30 - 17:00 Part IV
4. • Progressive Web Apps • Offline availability • Offline synchronization • Security considerations • Pitfalls Agenda
5. Intro
6. Offline… but why?
7. • Just because the phone has a connection,
 e.g. Edge, it does not mean we are online • Depends on the use case which connection
 quality is required to determine, if the 
 app is reliable online • Connection quality could be measured
 by the time an exclusive request takes • Duration < 150 ms: online • Duration !>=150 ms: bad connection What does “offline” mean?
8. • A lot of reasons to be offline • Traveling, Train, Flights (bad or no signal) • Server is not available • Routing problems • Roaming • Costs Motivation
9. Demo Application
10. • A little application to manage boardgames • Angular, .NET Core, MS SQL Server • Brownfield application, started as a pure online application • As typical for any good demo: no security • Available on Azure: https://tt-boardist.azurewebsites.net • GitHub: https://github.com/thinktecture/thinktecture-boardist Demo Application “Thinktecture Boardist”
11. Demo Application Database Scheme
12. Live Demo
13. Now let’s go offline!
14. Duh! And now?!
15. Progressive Web Apps
16. • Motivation: Get rid of app stores! • Web App = App App • Support native features like: Push notifications, offline availability, installable • Backwards-compatibility: runs in non-PWA browsers, with a reduced feature set Progressive Web Apps
17. PWA Technology Overview Progressive Web Apps HTML5, JavaScript, CSS3 Service Worker API Web App Manifest HTTPS Fetch API Web Notifications Web Workers Push API
18. PWA Platform Support 40 44 11.1 17 4.1 11.3
19. Progressive Web Apps 
 are not a technology , 
 but a collection of features an application must/should support
20. PWA Collection Of Features Responsive Linkable Discoverable Installable App-Like Connectivity
 Independent Fresh Safe Re-engageable Progressive
21. • A JavaScript running in its own thread • No access to the DOM • Communicates via postMessage-API • Acts as a controller/proxy/interceptor • Can perform background tasks • Has to be installed before usage • Runs whenever the browser is running PWA ServiceWorker
22. PWA ServiceWorker Website Internet Cache Service Worker
23. PWA ServiceWorker Lifecycle Parse Installing Error Activated Idle Terminated fetch/message
24. PWA Connectivity Independent Website Internet Cache storage Server Remote storage ServiceWorker
25. • Cache only • Network only • Cache falling back to network • Cache & network race • Network falling back to cache • Cache then network • Generic fallback • ServiceWorker-side templating PWA Caching Strategies
26. PWA Cache Then Network Website Internet Cache storage Server Remote storage ServiceWorker HTTP 1. Lookup 2. fetch
27. • “One-size-fits-all” ServiceWorker implementation • Instrumented via ngsw-config.json • Restricted feature set (e.g. no communication between app and SW) • Initial caching of static content • Caching external content • Dynamic data caching • Push notifications PWA Angular ServiceWorker
28. Are we offline now?
29. Nope.
30. Real offline availability
31. • ServiceWorker is only able to take data offline which has been requested by the application • If all the URLs are known beforehand, the ServiceWorker could cache them all • Data which was not requested, is not available offline (no real offline synchronisation) • But what about … the Background Sync API? What’s the problem?
32. • Name is misleading • It does not offer any data synchronisation possibilities, but offers the possibility to start a (periodic) “sync” event • It’s totally up to the developer what to do in the “sync” event • In case the user agent is offline, the sync will be automatically scheduled to be sent, when the user agent is online again • Since the sync is done in the ServiceWorker, the page can be closed, the sync will be fulfilled anyway • It does not help anything with syncing your actual data! Background Sync API
33. Background Sync API
34. • Cookies (“old & outdated”, not meant for large data or binaries) • Web Storage API (Session Storage, Local Storage, not meant for large data or binaries) • IndexedDB • Cache Storage Storage capabilities
35. User can clear anything at any time by “Remove temporary internet files”
36. • Key-Value database within the browser • Stores data permanently • ServiceWorker and Web App share access to the same IndexedDB • Possibility of scenarios, where the ServiceWorker (or Web App) stores synchronised data in the IndexedDB and the Web App reads the data IndexedDB
37. IndexedDB Availability 11 4 7.1 10 4.4 8
38. • The standard API of IndexedDB is inconvenient to use (lots of callback) • Dexie.js is a minimalistic wrapper for IndexedDB • Operations are promise-based instead of callback • Near native performance (even for bulk inserts) • Open Source @ GitHub: https://github.com/dfahlander/Dexie.js IndexedDB API
39. Offline Sync
40. • Offline Sync means to download all data available to client into a persistent offline storage, without the user having to explicitly request the data • Depending on the scenario, client can do CRUD on the offline data • Data will be synced back to the server, whenever a connection is possible Offline Sync Basics
41. • Online/offline recognition • Conflict management • Binary data • Local changes • Update interval (incoming new data) • Error handling Offline Sync Challenges
42. • Client needs to be online for write operations • Locks the data, so no other client can overwrite it • Data stays locked, until the client either saves or discards changes • Last One (Write) Wins • Visual conflict management (diffing like in Git, SVN, etc.) Offline Sync Conflict Management Scenarios
43. Backend Preparation
44. • All syncable entities need to have rowversion column • rowversion is updated by MS SQL Server automatically whenever the row is changed (created & updated) • For deleted entities • Either set a IsDeleted flag to true (never delete any rows physically) • Or save the deleted IDs of the entities somewhere else (by trigger) Backend Preparation - MS SQL Server
45. Backend Preparation - MS SQL Server public class Syncable : ISyncable { public Guid Id { get; set; } public byte[] RowVersion { get; set; } } public abstract class SyncableEntityTypeConfiguration<T> : IEntityTypeConfiguration<T> where T : class, ISyncable { public virtual void Configure(EntityTypeBuilder<T> builder) { builder.HasKey(p !=> p.Id); builder.Property(p !=> p.RowVersion).IsRowVersion(); } }
46. Backend Preparation - MS SQL Server public async Task<SyncDto<TResult!>> SyncAsync<TSource, TResult>(string timestamp) where TSource : Syncable where TResult : SyncableDto { var rowVersion = Convert.FromBase64String(timestamp !?? string.Empty); var baseQuery = _context.Set<TSource>() .Where(p !=> (ulong)(object)p.RowVersion !>= (ulong)(object)rowVersion); var changed = await _mapper.ProjectTo<TResult>(baseQuery.WithoutDeleted()).ToListAsync(); var deleted = await baseQuery.Where(p !=> p.IsDeleted).Select(p !=> p.Id).ToListAsync(); return new SyncDto<TResult>() { Timestamp = await _context.GetMinActiveRowVersionAsync(), Changed = changed, Deleted = deleted }; }
47. Backend Preparation - MS SQL Server public class Context : DbContext { private DbQuery<DbQueryValue> DbQueryValue { get; set; } public async Task<byte[]> GetMinActiveRowVersionAsync() { return await DbQueryValue .FromSql("SELECT MIN_ACTIVE_ROWVERSION() AS Value") .Select(p !=> p.Value) .FirstAsync(); } }
48. • Use equivalents of rowversion and triggers • Manual implement mechanism in business logic (error-prone!) • Update tracking column manually by incrementing a database global number (during one transaction!) • Will be very hard für multi-row updates/inserts • Manual implement mechanism in triggers (if available) Backend Preparation - Other Database Systems
49. Frontend Preparation
50. • Choose storage area for data (e.g. IndexedDB) • Write all the code • Periodic data synchronization • Binary synchronization when data changes • Tracking of timestamps Frontend Preparation
51. Security
52. Typical token-based security Security API level Browser Identity Provider Web API
53. • User is only able to see data based on this security level • Permissions • Roles • Policies • What happens, if the user rights change leading to different data visible to the user? • What about data, which he does not see due to rights, but is connected to other data? Security Data Level
54. Pitfalls
55. • Highest rowversion within result • @@DBTS • MIN_ACTIVE_ROWVERSION() Change Tracking - MS SQL Server’s ROWVERSION
56. • Not reliable – a later committed (but started earlier) transaction results in skipped values Change Tracking - Highest ROWVERSION within result Timestamp Transaction #1 Transaction #2 Sync T1 MAX !=> ROWVERSION(1) T2 INSERT !=> ROWVERSION(2) UPDATE !=> ROWVERSION(3) MAX !=> ROWVERSION(1) T3 COMMIT MAX !=> ROWVERSION(3) … Tn COMMIT ROW LOST!
57. • Not reliable – represents the maximum rowversion of the database (including not yet committed transactions ) Change Tracking - @@DBTS Timestamp Transaction #1 Transaction #2 Sync T1 @@DBTS !=> ROWVERSION(1) T2 INSERT !=> ROWVERSION(2) UPDATE !=> ROWVERSION(3) @@DBTS !=> ROWVERSION(3) T3 COMMIT @@DBTS !=> ROWVERSION(3) … Tn COMMIT ROW LOST!
58. • Reliable – represents the minimum rowversion of the database (including not yet committed transactions) • Small drawback: could result in repetitive selection of same data Change Tracking - MIN_ACTIVE_ROWVERSION() Timestamp Transaction #1 Transaction #2 Sync T1 SELECT !=> ROWVERSION(2) T2 INSERT !=> ROWVERSION(2) UPDATE !=> ROWVERSION(3) SELECT !=> ROWVERSION(2) T3 COMMIT SELECT !=> ROWVERSION(2) … Tn COMMIT SELECT !=> ROWVERSION(4)
59. • De-normalize the relational data (document style) • Results in multiple rowversion for one entry (use most recent one) • Multiplies the data (more traffic) • Consistent data for one entry • Keep the relations up to the frontend • Explicit rowversion for each entry • Partial consistency (related data may not be synced yet) Relational Data
60. • Keep some relational data and de-normalize some of it • De-normalize many-to-many relations • Needs trigger or business logic to change main entry’s rowversion when relation changes • Partial consistency (related data may not be synced yet) Relational Data
61. • Native Cordova app out-of-the box possible • Native features could be used • Store restrictions apply • What about Electron? Native Packaging??
62. • PWA helps bringing offline the application, but not the data • Online !== Edge is available • Offline data is “temporary” (“Remove temporary internet files”) • Use MIN_ACTIVE_ROWVERSION and “greater than” operator • RxJS helps building a sync engine, but a lot knowledge is needed • Think about data level security (permissions, roles, etc.) • Think about conflict management • Depends heavily on your use case Summary
63. And… now? • Slides: https://speakerdeck.com/manuelrauber • Repository: https://github.com/thinktecture/thinktecture-boardist