Stockage offline & synchro automatique pour vos applications web et mobiles

Stockage offline & synchro automatique pour vos applications web et mobiles

De plus en plus de développeurs web et mobiles utilisent du stockage local pour assurer de bons temps de réponse et que les applications continuent de fonctionner en absence de connection réseau. Mais comment synchronisez vous le stockage local avec le backend ? Comment resolvez-vous les conflits ? Dans ce talk, je vous présente un framework open source (Amplify DataStore) qui vous permet de développer des applications webs et mobiles modernes qui répondent à ces questions avec un minimum de ligne de code.

Transcript

  1. OFFLINE FIRST.

  2. OFFLINE FIRST Core features should function with or without an

    internet connection. WHAT DOES IT MEAN? #01
  3. OFFLINE FIRST Data is written to and queried locally from

    the end user’s device and periodically uploaded and replicated in the database. WHAT DOES IT MEAN? #02
  4. OFFLINE FIRST Provide end users with a consistent UX when

    internet connectivity is slow or non-existent. WHAT DOES IT MEAN? #03
  5. REAL TIME.

  6. REAL TIME An application that functions within a time frame

    that the user senses as immediate or very close to it. WHAT DOES IT MEAN #01
  7. REAL TIME Cohesive system where data is in sync across

    all connected devices. WHAT DOES IT MEAN #02
  8. THE FUTURE OF REAL-TIME | OFFLINE | DATA

  9. #01 Sebastien Stormacq INTRO. DUCTION. sebsto

  10. CLIENT TECHNOLOGIES AWS AMPLIFY AWS APPSYNC

  11. WHY 11 REAL TIME & OFFLINE APPLICATIONS

  12. WHY MODERN APPLICATION DEVELOPMENT

  13. WHY MODERN APPLICATION DEVELOPMENT • Chat • Banking • Collaborative

    • Games • Social
  14. WHY SLOW or NONEXISTENT NETWORKS

  15. WHY USER EXPERIENCE

  16. CONSIDERATIONS REAL TIME & OFFLINE APPLICATION

  17. CACHING. 16

  18. 17 There are only two hard things in Computer Science:

    cache invalidation and naming things.
  19. 17 There are only two hard things in Computer Science:

    cache invalidation and naming things.
  20. CACHE !== DATABASE

  21. REAL TIME.

  22. CONFLICT DETECTION.

  23. CONFLICT RESOLUTION.

  24. MULTIPLE PLATFORMS.

  25. SCALABILITY.

  26. SOLUTIONS REAL TIME & OFFLINE APPLICATION

  27. CACHING

  28. None
  29. Create mutation

  30. Create mutation Create optimistic response

  31. Create mutation Create optimistic response Write to cache

  32. CACHING mutation create($item: InputType) { createItem(input: $item) { id name

    status } } Writes
  33. CACHING mutation create($item: InputType) { createItem(input: $item) { id name

    status } } database cache Writes
  34. Cache hydration query listItems { items { id name location

    } } database
  35. Querying query listItems { items(filter: { eq: { location: “NYC”

    } }) { id name location } } cache
  36. Querying query listItems { items(filter: { eq: { location: “NYC”

    } }) { id name location } } cache
  37. None
  38. MENTAL MODEL

  39. AMPLIFY DATASTORE Multi-platform on-device persistent storage engine that automatically synchronizes

    data between mobile/web apps and the cloud using GraphQL.
  40. Storage Engine Storage Adapter CRUD OBSERVES WRITES READS WRITES OBSERVES

    Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer
  41. DATA MODELING WITH GRAPHQL Developer

  42. DATA MODELING WITH GRAPHQL Developer AUTHORING

  43. DATA MODELING WITH GRAPHQL Developer AUTHORING BUILD

  44. DATA MODELING WITH GRAPHQL Developer AUTHORING BUILD NPM script, gradle

    task, or Xcode build phase
  45. DATA MODELING WITH GRAPHQL Developer AUTHORING BUILD NPM script, gradle

    task, or Xcode build phase
  46. DATASTORE Fluent interface Mutations / writes

  47. DATASTORE Fluent interface Queries / reads

  48. DATASTORE Fluent interface - with predicates

  49. DATASTORE Fluent interface - with predicates & chaining

  50. DATASTORE Mutations - with predicates & chaining

  51. ARCHITECTURE Developer

  52. Storage Engine CRUD OBSERVES ARCHITECTURE DataStore API Developer Model converted

    to / from the storage adapter of choice
  53. Storage Engine Storage Adapter CRUD OBSERVES WRITES READS ARCHITECTURE DataStore

    API Developer Model converted to / from the storage adapter of choice Object and action persistence with specific storage engine (SQL Lite, IndexDB, etc..)
  54. Storage Engine Storage Adapter CRUD OBSERVES WRITES READS ARCHITECTURE DataStore

    API Developer
  55. Storage Engine Storage Adapter CRUD OBSERVES WRITES READS OBSERVES WRITES

    / READS Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer
  56. Storage Engine Storage Adapter CRUD OBSERVES WRITES READS OBSERVES WRITES

    / READS Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer Model converted to GraphQL statement Response converted to Model and written to storage engine.
  57. REAL TIME

  58. REAL TIME OPTIONS

  59. LONG POLLING

  60. SERVER SENT EVENTS

  61. WEB SOCKETS

  62. GRAPHQL

  63. LOCAL STORE + SUBSCRIPTIONS Fluent interface - observing data models

  64. SINGLE REHYDRATE QUERY

  65. SUBSCRIPTIONS

  66. SINGLE REHYDRATE QUERY

  67. CONFLICT DETECTION AND RESOLUTION

  68. Provide a way to merge concurrent modifications, always, in any

    order. CONFLICT DETECTION & RESOLUTION
  69. None
  70. update

  71. update

  72. update operation operation

  73. update operation operation

  74. update operation operation update

  75. update operation operation update

  76. update operation operation update

  77. update operation operation update

  78. update update

  79. update update

  80. update operation operation update

  81. update update update

  82. update update update

  83. update update update

  84. CONFLICT DETECTION & RESOLUTION Objects tagged with metadata

  85. Conflict Free Replicated Data Types (CRDTs) Operational Transforms Google Docs,

    MS Office Online Redis, Facebook, CosmosDB CONFLICT DETECTION & RESOLUTION 2006 1989
  86. Vector Hybrid Logical Clock (HLC) Monotonic CONFLICT RESOLUTION Clock

  87. person = { name: “Jennifer”, age: 32 } original item

    CONFLICT RESOLUTION
  88. person = { name: “Jennifer”, age: 32 } person =

    { name: “Jennifer”, age: 32, title: “Developer”, } person = { name: “Jennifer”, age: 32, title: “Engineer” } original item client a client b CONFLICT RESOLUTION
  89. person = { name: “Jennifer”, age: 32, title: “Developer” }

    person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION
  90. person = { name: “Jennifer”, age: 32, title: “Developer” }

    person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION person = { name: “Jennifer”, age: 32, title: “Developer” } new item
  91. person = { name: “Jennifer”, age: 32 } original item

    CONFLICT RESOLUTION
  92. person = { name: “Jennifer”, age: 32 } person =

    { name: “Jennifer”, age: 32, location: “Nebraska”, } person = { name: “Jennifer”, age: 32, title: “Engineer” } original item client a client b CONFLICT RESOLUTION
  93. person = { name: “Jennifer”, age: 32, location: “Nebraska” }

    person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION
  94. person = { name: “Jennifer”, age: 32, location: “Nebraska” }

    person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION person = { name: “Jennifer”, age: 32, location: “Nebraska”, title: “Engineer” } new item
  95. person = { name: “Jennifer”, age: 32 hobbies: [“climbing"] }

    original item CONFLICT RESOLUTION Lists
  96. person = { name: “Jennifer”, age: 32 hobbies: [“climbing"] }

    person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”] } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “karaoke”] } original item client a client b CONFLICT RESOLUTION Lists
  97. person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”]

    } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”,“karaoke”] } client a client b CONFLICT RESOLUTION Lists
  98. person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”]

    } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”,“karaoke”] } client a client b CONFLICT RESOLUTION Lists person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “karaoke”, “reading”] } new item
  99. AMPLIFY DATASTORE CONFLICT DETECTION & RESOLUTION • Monotonic counters •

    Controlled by the system itself (AppSync) • GraphQL types with versions used for merge or reject decisions • Base table & change table with soft deletes & TTL
  100. MONOTONIC COUNTER person = { id: 100, age: 30, _version:

    1 } person = { id: 100, age: 31, _version: 2 } person = { id: 100, age: 32, _version: 3 } id age name _version 100 32 Amy 3 101 28 Charles 9 102 47 Melissa 8 Base Table
  101. ds_pk ds_sk age name _version _lastChangedAt Person-9386:2020-01-23 13:16:08.127:9386:1 32 Amy

    1 1579785368127 Person-9386:2020-01-23 13:16:21.785:9386:2 33 Amy 2 1579785381785 Person-9386:2020-01-23 13:16:37.859:9386:3 34 Amy 3 1579785397859 Change Table CONFLICT RESOLUTION
  102. ds_pk ds_sk age name _version _lastChangedAt Person-9386:2020-01-23 13:16:08.127:9386:1 32 Amy

    1 1579785368127 Person-9386:2020-01-23 13:16:21.785:9386:2 33 Amy 2 1579785381785 Person-9386:2020-01-23 13:16:37.859:9386:3 34 Amy 3 1579785397859 Change Table typename + uuid + date CONFLICT RESOLUTION
  103. ds_pk ds_sk age name _version _lastChangedAt Person-9386:2020-01-23 13:16:08.127:9386:1 32 Amy

    1 1579785368127 Person-9386:2020-01-23 13:16:21.785:9386:2 33 Amy 2 1579785381785 Person-9386:2020-01-23 13:16:37.859:9386:3 34 Amy 3 1579785397859 Change Table typename + uuid + date timestamp + uuid + version CONFLICT RESOLUTION
  104. AUTO MERGE CONFLICT RESOLUTION

  105. User A User B person = { age: 32, _version:

    1 } person = { age: 32, _version: 1 }
  106. User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 }
  107. User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 }
  108. User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } OFFLINE
  109. User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } OFFLINE person = { age: 34, location: “Seattle” _version: 3 }
  110. User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE person = { age: 34, location: “Seattle” _version: 3 }
  111. User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 }
  112. person = { age: 34, age: 35, location: “Seattle” _version:

    4 } User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 }
  113. User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 } person = { age: 34, location: “Seattle” _version: 4 } person = { age: 35, location: “Seattle” _version: 4 }
  114. AUTO MERGE SYNC ENABLED GRAPHQL RESOLVERS Base table + change

    table + GraphQL types with versions used for merge or reject decisions
  115. AWS APPSYNC CONFLICT RESOLUTION • Automerge • AWS Lambda •

    Optimistic concurrency
  116. SYNCHRONIZATION

  117. id age name _version 100 32 Amy 8 101 28

    Charles 9 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3
  118. id age name _version 100 32 Amy 8 101 28

    Charles 9 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3
  119. id age name _version 100 32 Amy 8 101 28

    Charles 9 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3
  120. id age name _version 100 32 Amy 8 101 28

    Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3
  121. id age name _version 100 32 Amy 8 101 28

    Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION
  122. id age name _version 100 32 Amy 8 101 28

    Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION
  123. DELTA SYNC SYNCHRONIZATION

  124. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC
  125. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC
  126. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC
  127. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859
  128. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859
  129. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC syncPerson(lastSync: 13:16:22.127) Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859
  130. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC syncPerson(lastSync: 13:16:22.127) Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859 Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859
  131. TIME TRAVEL SYNCHRONIZATION

  132. id age name _version _deleted _ttl 100 32 Amy 8

    101 28 Charles 9 102 47 Melissa 3 TRUE 1582384003 SYNCHRONIZATION ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1 32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003 Change Table Base Table
  133. DB OPTIMIZATION SYNCHRONIZATION

  134. TIME TO LIVE ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1

    32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003
  135. TIME TO LIVE ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1

    32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003 1582384003 1582384003 1582384003
  136. PLATFORMS

  137. PLATFORMS

  138. https://aws-amplify.github.io/docs/js/datastore

  139. THANK YOU! /sebAWS @sebsto /sebsto /sebsto