Stockage offline & synchro automatique pour vos applications web et mobiles

De plus en plus de développeurs web et mobiles utilisent du stockage local pour assurer de bons temps de réponse et que les applications continuent de fonctionner en absence de connection réseau. Mais comment synchronisez vous le stockage local avec le backend ? Comment resolvez-vous les conflits ? Dans ce talk, je vous présente un framework open source (Amplify DataStore) qui vous permet de développer des applications webs et mobiles modernes qui répondent à ces questions avec un minimum de ligne de code.

Transcript

1. OFFLINE FIRST.
2. OFFLINE FIRST Core features should function with or without an internet connection. WHAT DOES IT MEAN? #01
3. OFFLINE FIRST Data is written to and queried locally from the end user’s device and periodically uploaded and replicated in the database. WHAT DOES IT MEAN? #02
4. OFFLINE FIRST Provide end users with a consistent UX when internet connectivity is slow or non-existent. WHAT DOES IT MEAN? #03
5. REAL TIME.
6. REAL TIME An application that functions within a time frame that the user senses as immediate or very close to it. WHAT DOES IT MEAN #01
7. REAL TIME Cohesive system where data is in sync across all connected devices. WHAT DOES IT MEAN #02
8. THE FUTURE OF REAL-TIME | OFFLINE | DATA
9. #01 Sebastien Stormacq INTRO. DUCTION. sebsto
10. CLIENT TECHNOLOGIES AWS AMPLIFY AWS APPSYNC
11. WHY 11 REAL TIME & OFFLINE APPLICATIONS
12. WHY MODERN APPLICATION DEVELOPMENT
13. WHY MODERN APPLICATION DEVELOPMENT • Chat • Banking • Collaborative • Games • Social
14. WHY SLOW or NONEXISTENT NETWORKS
15. WHY USER EXPERIENCE
16. CONSIDERATIONS REAL TIME & OFFLINE APPLICATION
17. CACHING. 16
18. 17 There are only two hard things in Computer Science: cache invalidation and naming things.
19. 17 There are only two hard things in Computer Science: cache invalidation and naming things.
20. CACHE !== DATABASE
21. REAL TIME.
22. CONFLICT DETECTION.
23. CONFLICT RESOLUTION.
24. MULTIPLE PLATFORMS.
25. SCALABILITY.
26. SOLUTIONS REAL TIME & OFFLINE APPLICATION
27. CACHING
29. Create mutation
30. Create mutation Create optimistic response
31. Create mutation Create optimistic response Write to cache
32. CACHING mutation create($item: InputType) { createItem(input: $item) { id name status } } Writes
33. CACHING mutation create($item: InputType) { createItem(input: $item) { id name status } } database cache Writes
34. Cache hydration query listItems { items { id name location } } database
35. Querying query listItems { items(filter: { eq: { location: “NYC” } }) { id name location } } cache
36. Querying query listItems { items(filter: { eq: { location: “NYC” } }) { id name location } } cache
38. MENTAL MODEL
39. AMPLIFY DATASTORE Multi-platform on-device persistent storage engine that automatically synchronizes data between mobile/web apps and the cloud using GraphQL.
40. Storage Engine Storage Adapter CRUD OBSERVES WRITES READS WRITES OBSERVES Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer
41. DATA MODELING WITH GRAPHQL Developer
42. DATA MODELING WITH GRAPHQL Developer AUTHORING
43. DATA MODELING WITH GRAPHQL Developer AUTHORING BUILD
44. DATA MODELING WITH GRAPHQL Developer AUTHORING BUILD NPM script, gradle task, or Xcode build phase
45. DATA MODELING WITH GRAPHQL Developer AUTHORING BUILD NPM script, gradle task, or Xcode build phase
46. DATASTORE Fluent interface Mutations / writes
47. DATASTORE Fluent interface Queries / reads
48. DATASTORE Fluent interface - with predicates
49. DATASTORE Fluent interface - with predicates & chaining
50. DATASTORE Mutations - with predicates & chaining
51. ARCHITECTURE Developer
52. Storage Engine CRUD OBSERVES ARCHITECTURE DataStore API Developer Model converted to / from the storage adapter of choice
53. Storage Engine Storage Adapter CRUD OBSERVES WRITES READS ARCHITECTURE DataStore API Developer Model converted to / from the storage adapter of choice Object and action persistence with specific storage engine (SQL Lite, IndexDB, etc..)
54. Storage Engine Storage Adapter CRUD OBSERVES WRITES READS ARCHITECTURE DataStore API Developer
55. Storage Engine Storage Adapter CRUD OBSERVES WRITES READS OBSERVES WRITES / READS Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer
56. Storage Engine Storage Adapter CRUD OBSERVES WRITES READS OBSERVES WRITES / READS Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer Model converted to GraphQL statement Response converted to Model and written to storage engine.
57. REAL TIME
58. REAL TIME OPTIONS
59. LONG POLLING
60. SERVER SENT EVENTS
61. WEB SOCKETS
62. GRAPHQL
63. LOCAL STORE + SUBSCRIPTIONS Fluent interface - observing data models
64. SINGLE REHYDRATE QUERY
65. SUBSCRIPTIONS
66. SINGLE REHYDRATE QUERY
67. CONFLICT DETECTION AND RESOLUTION
68. Provide a way to merge concurrent modifications, always, in any order. CONFLICT DETECTION & RESOLUTION
70. update
71. update
72. update operation operation
73. update operation operation
74. update operation operation update
75. update operation operation update
76. update operation operation update
77. update operation operation update
78. update update
79. update update
80. update operation operation update
81. update update update
82. update update update
83. update update update
84. CONFLICT DETECTION & RESOLUTION Objects tagged with metadata
85. Conflict Free Replicated Data Types (CRDTs) Operational Transforms Google Docs, MS Office Online Redis, Facebook, CosmosDB CONFLICT DETECTION & RESOLUTION 2006 1989
86. Vector Hybrid Logical Clock (HLC) Monotonic CONFLICT RESOLUTION Clock
87. person = { name: “Jennifer”, age: 32 } original item CONFLICT RESOLUTION
88. person = { name: “Jennifer”, age: 32 } person = { name: “Jennifer”, age: 32, title: “Developer”, } person = { name: “Jennifer”, age: 32, title: “Engineer” } original item client a client b CONFLICT RESOLUTION
89. person = { name: “Jennifer”, age: 32, title: “Developer” } person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION
90. person = { name: “Jennifer”, age: 32, title: “Developer” } person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION person = { name: “Jennifer”, age: 32, title: “Developer” } new item
91. person = { name: “Jennifer”, age: 32 } original item CONFLICT RESOLUTION
92. person = { name: “Jennifer”, age: 32 } person = { name: “Jennifer”, age: 32, location: “Nebraska”, } person = { name: “Jennifer”, age: 32, title: “Engineer” } original item client a client b CONFLICT RESOLUTION
93. person = { name: “Jennifer”, age: 32, location: “Nebraska” } person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION
94. person = { name: “Jennifer”, age: 32, location: “Nebraska” } person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION person = { name: “Jennifer”, age: 32, location: “Nebraska”, title: “Engineer” } new item
95. person = { name: “Jennifer”, age: 32 hobbies: [“climbing"] } original item CONFLICT RESOLUTION Lists
96. person = { name: “Jennifer”, age: 32 hobbies: [“climbing"] } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”] } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “karaoke”] } original item client a client b CONFLICT RESOLUTION Lists
97. person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”] } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”,“karaoke”] } client a client b CONFLICT RESOLUTION Lists
98. person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”] } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”,“karaoke”] } client a client b CONFLICT RESOLUTION Lists person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “karaoke”, “reading”] } new item
99. AMPLIFY DATASTORE CONFLICT DETECTION & RESOLUTION • Monotonic counters • Controlled by the system itself (AppSync) • GraphQL types with versions used for merge or reject decisions • Base table & change table with soft deletes & TTL
100. MONOTONIC COUNTER person = { id: 100, age: 30, _version: 1 } person = { id: 100, age: 31, _version: 2 } person = { id: 100, age: 32, _version: 3 } id age name _version 100 32 Amy 3 101 28 Charles 9 102 47 Melissa 8 Base Table
101. ds_pk ds_sk age name _version _lastChangedAt Person-9386:2020-01-23 13:16:08.127:9386:1 32 Amy 1 1579785368127 Person-9386:2020-01-23 13:16:21.785:9386:2 33 Amy 2 1579785381785 Person-9386:2020-01-23 13:16:37.859:9386:3 34 Amy 3 1579785397859 Change Table CONFLICT RESOLUTION
102. ds_pk ds_sk age name _version _lastChangedAt Person-9386:2020-01-23 13:16:08.127:9386:1 32 Amy 1 1579785368127 Person-9386:2020-01-23 13:16:21.785:9386:2 33 Amy 2 1579785381785 Person-9386:2020-01-23 13:16:37.859:9386:3 34 Amy 3 1579785397859 Change Table typename + uuid + date CONFLICT RESOLUTION
103. ds_pk ds_sk age name _version _lastChangedAt Person-9386:2020-01-23 13:16:08.127:9386:1 32 Amy 1 1579785368127 Person-9386:2020-01-23 13:16:21.785:9386:2 33 Amy 2 1579785381785 Person-9386:2020-01-23 13:16:37.859:9386:3 34 Amy 3 1579785397859 Change Table typename + uuid + date timestamp + uuid + version CONFLICT RESOLUTION
104. AUTO MERGE CONFLICT RESOLUTION
105. User A User B person = { age: 32, _version: 1 } person = { age: 32, _version: 1 }
106. User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 }
107. User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 }
108. User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } OFFLINE
109. User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } OFFLINE person = { age: 34, location: “Seattle” _version: 3 }
110. User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE person = { age: 34, location: “Seattle” _version: 3 }
111. User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 }
112. person = { age: 34, age: 35, location: “Seattle” _version: 4 } User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 }
113. User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 } person = { age: 34, location: “Seattle” _version: 4 } person = { age: 35, location: “Seattle” _version: 4 }
114. AUTO MERGE SYNC ENABLED GRAPHQL RESOLVERS Base table + change table + GraphQL types with versions used for merge or reject decisions
115. AWS APPSYNC CONFLICT RESOLUTION • Automerge • AWS Lambda • Optimistic concurrency
116. SYNCHRONIZATION
117. id age name _version 100 32 Amy 8 101 28 Charles 9 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3
118. id age name _version 100 32 Amy 8 101 28 Charles 9 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3
119. id age name _version 100 32 Amy 8 101 28 Charles 9 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3
120. id age name _version 100 32 Amy 8 101 28 Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3
121. id age name _version 100 32 Amy 8 101 28 Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION
122. id age name _version 100 32 Amy 8 101 28 Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION
123. DELTA SYNC SYNCHRONIZATION
124. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James 7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC
125. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James 7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC
126. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James 7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC
127. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James 7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859
128. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James 7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859
129. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James 7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC syncPerson(lastSync: 13:16:22.127) Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859
130. ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James 7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC syncPerson(lastSync: 13:16:22.127) Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859 Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859
131. TIME TRAVEL SYNCHRONIZATION
132. id age name _version _deleted _ttl 100 32 Amy 8 101 28 Charles 9 102 47 Melissa 3 TRUE 1582384003 SYNCHRONIZATION ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1 32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003 Change Table Base Table
133. DB OPTIMIZATION SYNCHRONIZATION
134. TIME TO LIVE ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1 32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003
135. TIME TO LIVE ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1 32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003 1582384003 1582384003 1582384003
136. PLATFORMS
137. PLATFORMS
138. https://aws-amplify.github.io/docs/js/datastore
139. THANK YOU! /sebAWS @sebsto /sebsto /sebsto