Stockage offline & synchro automatique pour vos applications web et mobiles

Stockage offline & synchro automatique pour vos applications web et mobiles

De plus en plus de développeurs web et mobiles utilisent du stockage local pour assurer de bons temps de réponse et que les applications continuent de fonctionner en absence de connection réseau. Mais comment synchronisez vous le stockage local avec le backend ? Comment resolvez-vous les conflits ? Dans ce talk, je vous présente un framework open source (Amplify DataStore) qui vous permet de développer des applications webs et mobiles modernes qui répondent à ces questions avec un minimum de ligne de code.

Transcript

  1. 2.

    OFFLINE FIRST Core features should function with or without an

    internet connection. WHAT DOES IT MEAN? #01
  2. 3.

    OFFLINE FIRST Data is written to and queried locally from

    the end user’s device and periodically uploaded and replicated in the database. WHAT DOES IT MEAN? #02
  3. 4.

    OFFLINE FIRST Provide end users with a consistent UX when

    internet connectivity is slow or non-existent. WHAT DOES IT MEAN? #03
  4. 6.

    REAL TIME An application that functions within a time frame

    that the user senses as immediate or very close to it. WHAT DOES IT MEAN #01
  5. 7.

    REAL TIME Cohesive system where data is in sync across

    all connected devices. WHAT DOES IT MEAN #02
  6. 18.

    17 There are only two hard things in Computer Science:

    cache invalidation and naming things.
  7. 19.

    17 There are only two hard things in Computer Science:

    cache invalidation and naming things.
  8. 27.
  9. 28.
  10. 37.
  11. 40.

    Storage Engine Storage Adapter CRUD OBSERVES WRITES READS WRITES OBSERVES

    Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer
  12. 53.

    Storage Engine Storage Adapter CRUD OBSERVES WRITES READS ARCHITECTURE DataStore

    API Developer Model converted to / from the storage adapter of choice Object and action persistence with specific storage engine (SQL Lite, IndexDB, etc..)
  13. 55.

    Storage Engine Storage Adapter CRUD OBSERVES WRITES READS OBSERVES WRITES

    / READS Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer
  14. 56.

    Storage Engine Storage Adapter CRUD OBSERVES WRITES READS OBSERVES WRITES

    / READS Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer Model converted to GraphQL statement Response converted to Model and written to storage engine.
  15. 57.
  16. 62.
  17. 68.
  18. 69.
  19. 70.
  20. 71.
  21. 85.

    Conflict Free Replicated Data Types (CRDTs) Operational Transforms Google Docs,

    MS Office Online Redis, Facebook, CosmosDB CONFLICT DETECTION & RESOLUTION 2006 1989
  22. 88.

    person = { name: “Jennifer”, age: 32 } person =

    { name: “Jennifer”, age: 32, title: “Developer”, } person = { name: “Jennifer”, age: 32, title: “Engineer” } original item client a client b CONFLICT RESOLUTION
  23. 89.

    person = { name: “Jennifer”, age: 32, title: “Developer” }

    person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION
  24. 90.

    person = { name: “Jennifer”, age: 32, title: “Developer” }

    person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION person = { name: “Jennifer”, age: 32, title: “Developer” } new item
  25. 92.

    person = { name: “Jennifer”, age: 32 } person =

    { name: “Jennifer”, age: 32, location: “Nebraska”, } person = { name: “Jennifer”, age: 32, title: “Engineer” } original item client a client b CONFLICT RESOLUTION
  26. 93.

    person = { name: “Jennifer”, age: 32, location: “Nebraska” }

    person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION
  27. 94.

    person = { name: “Jennifer”, age: 32, location: “Nebraska” }

    person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION person = { name: “Jennifer”, age: 32, location: “Nebraska”, title: “Engineer” } new item
  28. 95.
  29. 96.

    person = { name: “Jennifer”, age: 32 hobbies: [“climbing"] }

    person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”] } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “karaoke”] } original item client a client b CONFLICT RESOLUTION Lists
  30. 97.

    person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”]

    } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”,“karaoke”] } client a client b CONFLICT RESOLUTION Lists
  31. 98.

    person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”]

    } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”,“karaoke”] } client a client b CONFLICT RESOLUTION Lists person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “karaoke”, “reading”] } new item
  32. 99.

    AMPLIFY DATASTORE CONFLICT DETECTION & RESOLUTION • Monotonic counters •

    Controlled by the system itself (AppSync) • GraphQL types with versions used for merge or reject decisions • Base table & change table with soft deletes & TTL
  33. 100.

    MONOTONIC COUNTER person = { id: 100, age: 30, _version:

    1 } person = { id: 100, age: 31, _version: 2 } person = { id: 100, age: 32, _version: 3 } id age name _version 100 32 Amy 3 101 28 Charles 9 102 47 Melissa 8 Base Table
  34. 101.

    ds_pk ds_sk age name _version _lastChangedAt Person-9386:2020-01-23 13:16:08.127:9386:1 32 Amy

    1 1579785368127 Person-9386:2020-01-23 13:16:21.785:9386:2 33 Amy 2 1579785381785 Person-9386:2020-01-23 13:16:37.859:9386:3 34 Amy 3 1579785397859 Change Table CONFLICT RESOLUTION
  35. 102.

    ds_pk ds_sk age name _version _lastChangedAt Person-9386:2020-01-23 13:16:08.127:9386:1 32 Amy

    1 1579785368127 Person-9386:2020-01-23 13:16:21.785:9386:2 33 Amy 2 1579785381785 Person-9386:2020-01-23 13:16:37.859:9386:3 34 Amy 3 1579785397859 Change Table typename + uuid + date CONFLICT RESOLUTION
  36. 103.

    ds_pk ds_sk age name _version _lastChangedAt Person-9386:2020-01-23 13:16:08.127:9386:1 32 Amy

    1 1579785368127 Person-9386:2020-01-23 13:16:21.785:9386:2 33 Amy 2 1579785381785 Person-9386:2020-01-23 13:16:37.859:9386:3 34 Amy 3 1579785397859 Change Table typename + uuid + date timestamp + uuid + version CONFLICT RESOLUTION
  37. 105.

    User A User B person = { age: 32, _version:

    1 } person = { age: 32, _version: 1 }
  38. 106.

    User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 }
  39. 107.

    User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 }
  40. 108.

    User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } OFFLINE
  41. 109.

    User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } OFFLINE person = { age: 34, location: “Seattle” _version: 3 }
  42. 110.

    User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE person = { age: 34, location: “Seattle” _version: 3 }
  43. 111.

    User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 }
  44. 112.

    person = { age: 34, age: 35, location: “Seattle” _version:

    4 } User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 }
  45. 113.

    User A User B person = { age: 32, _version:

    1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 } person = { age: 34, location: “Seattle” _version: 4 } person = { age: 35, location: “Seattle” _version: 4 }
  46. 114.

    AUTO MERGE SYNC ENABLED GRAPHQL RESOLVERS Base table + change

    table + GraphQL types with versions used for merge or reject decisions
  47. 117.

    id age name _version 100 32 Amy 8 101 28

    Charles 9 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3
  48. 118.

    id age name _version 100 32 Amy 8 101 28

    Charles 9 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3
  49. 119.

    id age name _version 100 32 Amy 8 101 28

    Charles 9 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3
  50. 120.

    id age name _version 100 32 Amy 8 101 28

    Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3
  51. 121.

    id age name _version 100 32 Amy 8 101 28

    Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION
  52. 122.

    id age name _version 100 32 Amy 8 101 28

    Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION
  53. 124.

    ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC
  54. 125.

    ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC
  55. 126.

    ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC
  56. 127.

    ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859
  57. 128.

    ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859
  58. 129.

    ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC syncPerson(lastSync: 13:16:22.127) Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859
  59. 130.

    ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James

    7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC syncPerson(lastSync: 13:16:22.127) Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859 Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859
  60. 132.

    id age name _version _deleted _ttl 100 32 Amy 8

    101 28 Charles 9 102 47 Melissa 3 TRUE 1582384003 SYNCHRONIZATION ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1 32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003 Change Table Base Table
  61. 134.

    TIME TO LIVE ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1

    32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003
  62. 135.

    TIME TO LIVE ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1

    32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003 1582384003 1582384003 1582384003
  63. 136.
  64. 137.