Immer und überall: offlinefähige Progressive Web Apps – am Beispiel Angular

"Keine Internetverbindung" – ein Satz, den wir alle aus genutzten Apps kennen, sei es Desktopanwendung oder Smartphone App. Oftmals sind die Clients auf dem jeweiligen Zielgerät installiert, bedienen sich aber einer externen Datenquelle, bspw. in Form eines Web API. Bei einer offline-fähigen Implementierung sind nicht nur Lese-, sondern vor allem auch Schreibzugriffe wichtig. Sprich, die Daten müssen in beide Richtungen synchronisiert werden können, sobald die Internetverbindung wiederhergestellt wurde. Gerade in der heutigen Zeit, in der Menschen immer mobiler werden, in Autos, Zügen und Flugzeugen unterwegs sind und arbeiten, benötigt man oftmals offline Zugriff auf die Anwendungsdaten. Wie man diese echt offline-fähigen Single-Page Applications/Progressive Web Apps realisiert, zeigen Thomas Hilzendegen und Manuel Rauber von Thinktecture in diesem Ganztagesworkshop. Allgemein werden Themen wie Synchronisationsstrategien von Anwendungsdaten und großen Binärdaten, Konfliktmanagement oder Datenbankarchitektur sowohl für neue als auch bestehende Anwendungen diskutiert. Am Beispiel einer Angular-basierten App mit einem exemplarischen Backend in .NET Core werden Client und Server zur Umsetzung der besprochenen Konzepte und Patterns implementiert.

GitHub: https://github.com/thinktecture/thinktecture-boardist

Transcript

1. Offlinefähige Progressive Web Apps
   Immer & überall
   Thomas Hilzendegen
   @hilzendegent
   Consultant
   Manuel Rauber
   @ManuelRauber
   Consultant
   

   View Slide

2. Thomas Hilzendegen
   Speakers
   Manuel Rauber
   thomas.hilzendegen@thinktecture.com
   @hilzendegent
   manuel.rauber@thinktecture.com
   @manuelrauber
   https://manuel-rauber.com
   Microsoft MVP
   

   View Slide

3. Timetable
   Time Doing
   09:00 - 10:30 Part I
   10:30 - 11:00 ☕
   11:00 - 12:30 Part II
   12:30 - 13:30
   13:30 - 15:00 Part III
   15:00 - 15:30 ☕
   15:30 - 17:00 Part IV
   

   View Slide

4. • Progressive Web Apps
   • Offline availability
   • Offline synchronization
   • Security considerations
   • Pitfalls
   Agenda
   

   View Slide

5. Intro
   

   View Slide

6. Offline… but why?
   

   View Slide

7. • Just because the phone has a connection,

   e.g. Edge, it does not mean we are online
   • Depends on the use case which connection

   quality is required to determine, if the 

   app is reliable online
   • Connection quality could be measured

   by the time an exclusive request takes
   • Duration < 150 ms: online
   • Duration !>=150 ms: bad connection
   What does “offline” mean?
   

   View Slide

8. • A lot of reasons to be offline
   • Traveling, Train, Flights (bad or no signal)
   • Server is not available
   • Routing problems
   • Roaming
   • Costs
   Motivation
   

   View Slide

9. Demo Application
   

   View Slide

10. • A little application to manage boardgames
    • Angular, .NET Core, MS SQL Server
    • Brownfield application, started as a pure online application
    • As typical for any good demo: no security
    • Available on Azure: https://tt-boardist.azurewebsites.net
    • GitHub: https://github.com/thinktecture/thinktecture-boardist
    Demo Application “Thinktecture Boardist”
    

    View Slide

11. Demo Application Database Scheme
    

    View Slide

12. Live Demo
    

    View Slide

13. Now let’s go offline!
    

    View Slide

14. Duh! And now?!
    

    View Slide

15. Progressive Web Apps
    

    View Slide

16. • Motivation: Get rid of app stores!
    • Web App = App App
    • Support native features like: Push notifications,
    offline availability, installable
    • Backwards-compatibility: runs in non-PWA
    browsers, with a reduced feature set
    Progressive Web Apps
    

    View Slide

17. PWA Technology Overview
    Progressive Web Apps
    HTML5, JavaScript, CSS3
    Service Worker API Web App Manifest HTTPS
    Fetch API
    Web
    Notifications
    Web
    Workers
    Push API
    

    View Slide

18. PWA Platform Support
    40 44 11.1 17 4.1 11.3
    

    View Slide

19. Progressive Web Apps 

    are not a technology , 

    but a collection of features an
    application must/should support
    

    View Slide

20. PWA Collection Of Features
    Responsive Linkable Discoverable Installable App-Like
    Connectivity

    Independent
    Fresh Safe Re-engageable Progressive
    

    View Slide

21. • A JavaScript running in its own thread
    • No access to the DOM
    • Communicates via postMessage-API
    • Acts as a controller/proxy/interceptor
    • Can perform background tasks
    • Has to be installed before usage
    • Runs whenever the browser is running
    PWA ServiceWorker
    

    View Slide

22. PWA ServiceWorker
    Website Internet
    Cache Service
    Worker
    

    View Slide

23. PWA ServiceWorker Lifecycle
    Parse Installing
    Error
    Activated Idle Terminated
    fetch/message
    

    View Slide

24. PWA Connectivity Independent
    Website Internet
    Cache storage
    Server
    Remote storage
    ServiceWorker
    

    View Slide

25. • Cache only
    • Network only
    • Cache falling back to network
    • Cache & network race
    • Network falling back to cache
    • Cache then network
    • Generic fallback
    • ServiceWorker-side templating
    PWA Caching Strategies
    

    View Slide

26. PWA Cache Then Network
    Website Internet
    Cache storage
    Server
    Remote storage
    ServiceWorker
    HTTP
    1. Lookup
    2. fetch
    

    View Slide

27. • “One-size-fits-all” ServiceWorker implementation
    • Instrumented via ngsw-config.json
    • Restricted feature set (e.g. no communication between app and SW)
    • Initial caching of static content
    • Caching external content
    • Dynamic data caching
    • Push notifications
    PWA Angular ServiceWorker
    

    View Slide

28. Are we offline now?
    

    View Slide

29. Nope.
    

    View Slide

30. Real offline availability
    

    View Slide

31. • ServiceWorker is only able to take data offline which has been requested by
    the application
    • If all the URLs are known beforehand, the ServiceWorker could cache them
    all
    • Data which was not requested, is not available offline (no real offline
    synchronisation)
    • But what about … the Background Sync API?
    What’s the problem?
    

    View Slide

32. • Name is misleading
    • It does not offer any data synchronisation possibilities, but offers the
    possibility to start a (periodic) “sync” event
    • It’s totally up to the developer what to do in the “sync” event
    • In case the user agent is offline, the sync will be automatically scheduled to
    be sent, when the user agent is online again
    • Since the sync is done in the ServiceWorker, the page can be closed, the
    sync will be fulfilled anyway
    • It does not help anything with syncing your actual data!
    Background Sync API
    

    View Slide

33. Background Sync API
    

    View Slide

34. • Cookies (“old & outdated”, not meant for large data or binaries)
    • Web Storage API (Session Storage, Local Storage, not meant for large data
    or binaries)
    • IndexedDB
    • Cache Storage
    Storage capabilities
    

    View Slide

35. User can clear anything
    at any time by
    “Remove temporary internet files”
    

    View Slide

36. • Key-Value database within the browser
    • Stores data permanently
    • ServiceWorker and Web App share access to the same IndexedDB
    • Possibility of scenarios, where the ServiceWorker (or Web App) stores
    synchronised data in the IndexedDB and the Web App reads the data
    IndexedDB
    

    View Slide

37. IndexedDB Availability
    11 4 7.1 10 4.4 8
    

    View Slide

38. • The standard API of IndexedDB is inconvenient to use (lots of callback)
    • Dexie.js is a minimalistic wrapper for IndexedDB
    • Operations are promise-based instead of callback
    • Near native performance (even for bulk inserts)
    • Open Source @ GitHub: https://github.com/dfahlander/Dexie.js
    IndexedDB API
    

    View Slide

39. Offline Sync
    

    View Slide

40. • Offline Sync means to download all data available to client into a persistent
    offline storage, without the user having to explicitly request the data
    • Depending on the scenario, client can do CRUD on the offline data
    • Data will be synced back to the server, whenever a connection is possible
    Offline Sync Basics
    

    View Slide

41. • Online/offline recognition
    • Conflict management
    • Binary data
    • Local changes
    • Update interval (incoming new data)
    • Error handling
    Offline Sync Challenges
    

    View Slide

42. • Client needs to be online for write operations
    • Locks the data, so no other client can overwrite it
    • Data stays locked, until the client either saves or discards changes
    • Last One (Write) Wins
    • Visual conflict management (diffing like in Git, SVN, etc.)
    Offline Sync Conflict Management Scenarios
    

    View Slide

43. Backend Preparation
    

    View Slide

44. • All syncable entities need to have rowversion column
    • rowversion is updated by MS SQL Server automatically whenever the row
    is changed (created & updated)
    • For deleted entities
    • Either set a IsDeleted flag to true (never delete any rows physically)
    • Or save the deleted IDs of the entities somewhere else (by trigger)
    Backend Preparation - MS SQL Server
    

    View Slide

45. Backend Preparation - MS SQL Server
    public class Syncable : ISyncable
    {
    public Guid Id { get; set; }
    public byte[] RowVersion { get; set; }
    }
    public abstract class SyncableEntityTypeConfiguration : IEntityTypeConfiguration
    where T : class, ISyncable
    {
    public virtual void Configure(EntityTypeBuilder builder)
    {
    builder.HasKey(p !=> p.Id);
    builder.Property(p !=> p.RowVersion).IsRowVersion();
    }
    }
    

    View Slide

46. Backend Preparation - MS SQL Server
    public async Task> SyncAsync(string timestamp)
    where TSource : Syncable
    where TResult : SyncableDto
    {
    var rowVersion = Convert.FromBase64String(timestamp !?? string.Empty);
    var baseQuery = _context.Set()
    .Where(p !=> (ulong)(object)p.RowVersion !>= (ulong)(object)rowVersion);
    var changed = await _mapper.ProjectTo(baseQuery.WithoutDeleted()).ToListAsync();
    var deleted = await baseQuery.Where(p !=> p.IsDeleted).Select(p !=> p.Id).ToListAsync();
    return new SyncDto()
    {
    Timestamp = await _context.GetMinActiveRowVersionAsync(),
    Changed = changed,
    Deleted = deleted
    };
    }
    

    View Slide

47. Backend Preparation - MS SQL Server
    public class Context : DbContext {
    private DbQuery DbQueryValue { get; set; }
    public async Task GetMinActiveRowVersionAsync()
    {
    return await DbQueryValue
    .FromSql("SELECT MIN_ACTIVE_ROWVERSION() AS Value")
    .Select(p !=> p.Value)
    .FirstAsync();
    }
    }
    

    View Slide

48. • Use equivalents of rowversion and triggers
    • Manual implement mechanism in business logic (error-prone!)
    • Update tracking column manually by incrementing a database global
    number (during one transaction!)
    • Will be very hard für multi-row updates/inserts
    • Manual implement mechanism in triggers (if available)
    Backend Preparation - Other Database Systems
    

    View Slide

49. Frontend Preparation
    

    View Slide

50. • Choose storage area for data (e.g. IndexedDB)
    • Write all the code
    • Periodic data synchronization
    • Binary synchronization when data changes
    • Tracking of timestamps
    Frontend Preparation
    

    View Slide

51. Security
    

    View Slide

52. Typical token-based security
    Security API level
    Browser
    Identity
    Provider
    Web API
    

    View Slide

53. • User is only able to see data based on this security level
    • Permissions
    • Roles
    • Policies
    • What happens, if the user rights change leading to different data visible to
    the user?
    • What about data, which he does not see due to rights, but is connected to
    other data?
    Security Data Level
    

    View Slide

54. Pitfalls
    

    View Slide

55. • Highest rowversion within result
    • @@DBTS
    • MIN_ACTIVE_ROWVERSION()
    Change Tracking - MS SQL Server’s ROWVERSION
    

    View Slide

56. • Not reliable – a later committed (but started earlier) transaction results in
    skipped values
    Change Tracking - Highest ROWVERSION within result
    Timestamp Transaction #1 Transaction #2 Sync
    T1
    MAX !=> ROWVERSION(1)
    T2
    INSERT !=> ROWVERSION(2) UPDATE !=> ROWVERSION(3) MAX !=> ROWVERSION(1)
    T3
    COMMIT MAX !=> ROWVERSION(3)
    …
    Tn
    COMMIT ROW LOST!
    

    View Slide

57. • Not reliable – represents the maximum rowversion of the database
    (including not yet committed transactions )
    Change Tracking - @@DBTS
    Timestamp Transaction #1 Transaction #2 Sync
    T1
    @@DBTS !=> ROWVERSION(1)
    T2
    INSERT !=> ROWVERSION(2) UPDATE !=> ROWVERSION(3) @@DBTS !=> ROWVERSION(3)
    T3
    COMMIT @@DBTS !=> ROWVERSION(3)
    …
    Tn
    COMMIT ROW LOST!
    

    View Slide

58. • Reliable – represents the minimum rowversion of the database (including
    not yet committed transactions)
    • Small drawback: could result in repetitive selection of same data
    Change Tracking - MIN_ACTIVE_ROWVERSION()
    Timestamp Transaction #1 Transaction #2 Sync
    T1
    SELECT !=> ROWVERSION(2)
    T2
    INSERT !=> ROWVERSION(2) UPDATE !=> ROWVERSION(3) SELECT !=> ROWVERSION(2)
    T3
    COMMIT SELECT !=> ROWVERSION(2)
    …
    Tn
    COMMIT SELECT !=> ROWVERSION(4)
    

    View Slide

59. • De-normalize the relational data (document style)
    • Results in multiple rowversion for one entry (use most recent one)
    • Multiplies the data (more traffic)
    • Consistent data for one entry
    • Keep the relations up to the frontend
    • Explicit rowversion for each entry
    • Partial consistency (related data may not be synced yet)
    Relational Data
    

    View Slide

60. • Keep some relational data and de-normalize some of it
    • De-normalize many-to-many relations
    • Needs trigger or business logic to change main entry’s rowversion when
    relation changes
    • Partial consistency (related data may not be synced yet)
    Relational Data
    

    View Slide

61. • Native Cordova app out-of-the box possible
    • Native features could be used
    • Store restrictions apply
    • What about Electron?
    Native Packaging??
    

    View Slide

62. • PWA helps bringing offline the application, but not the data
    • Online !== Edge is available
    • Offline data is “temporary” (“Remove temporary internet files”)
    • Use MIN_ACTIVE_ROWVERSION and “greater than” operator
    • RxJS helps building a sync engine, but a lot knowledge is needed
    • Think about data level security (permissions, roles, etc.)
    • Think about conflict management
    • Depends heavily on your use case
    Summary
    

    View Slide

63. And… now?
    • Slides: https://speakerdeck.com/manuelrauber
    • Repository: https://github.com/thinktecture/thinktecture-boardist
    

    View Slide