Stockage offline & synchro automatique pour vos applications web et mobiles

Slide 1

Slide 1 text

OFFLINE FIRST.

Slide 2

Slide 2 text

OFFLINE FIRST Core features should function with or without an internet connection. WHAT DOES IT MEAN? #01

Slide 3

Slide 3 text

OFFLINE FIRST Data is written to and queried locally from the end user’s device and periodically uploaded and replicated in the database. WHAT DOES IT MEAN? #02

Slide 4

Slide 4 text

OFFLINE FIRST Provide end users with a consistent UX when internet connectivity is slow or non-existent. WHAT DOES IT MEAN? #03

Slide 5

Slide 5 text

REAL TIME.

Slide 6

Slide 6 text

REAL TIME An application that functions within a time frame that the user senses as immediate or very close to it. WHAT DOES IT MEAN #01

Slide 7

Slide 7 text

REAL TIME Cohesive system where data is in sync across all connected devices. WHAT DOES IT MEAN #02

Slide 8

Slide 8 text

THE FUTURE OF REAL-TIME | OFFLINE | DATA

Slide 9

Slide 9 text

#01 Sebastien Stormacq INTRO. DUCTION. sebsto

Slide 10

Slide 10 text

CLIENT TECHNOLOGIES AWS AMPLIFY AWS APPSYNC

Slide 11

Slide 11 text

WHY 11 REAL TIME & OFFLINE APPLICATIONS

Slide 12

Slide 12 text

WHY MODERN APPLICATION DEVELOPMENT

Slide 13

Slide 13 text

WHY MODERN APPLICATION DEVELOPMENT • Chat • Banking • Collaborative • Games • Social

Slide 14

Slide 14 text

WHY SLOW or NONEXISTENT NETWORKS

Slide 15

Slide 15 text

WHY USER EXPERIENCE

Slide 16

Slide 16 text

CONSIDERATIONS REAL TIME & OFFLINE APPLICATION

Slide 17

Slide 17 text

CACHING. 16

Slide 18

Slide 18 text

17 There are only two hard things in Computer Science: cache invalidation and naming things.

Slide 19

Slide 19 text

17 There are only two hard things in Computer Science: cache invalidation and naming things.

Slide 20

Slide 20 text

CACHE !== DATABASE

Slide 21

Slide 21 text

REAL TIME.

Slide 22

Slide 22 text

CONFLICT DETECTION.

Slide 23

Slide 23 text

CONFLICT RESOLUTION.

Slide 24

Slide 24 text

MULTIPLE PLATFORMS.

Slide 25

Slide 25 text

SCALABILITY.

Slide 26

Slide 26 text

SOLUTIONS REAL TIME & OFFLINE APPLICATION

Slide 27

Slide 27 text

CACHING

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

Create mutation

Slide 30

Slide 30 text

Create mutation Create optimistic response

Slide 31

Slide 31 text

Create mutation Create optimistic response Write to cache

Slide 32

Slide 32 text

CACHING mutation create($item: InputType) { createItem(input: $item) { id name status } } Writes

Slide 33

Slide 33 text

CACHING mutation create($item: InputType) { createItem(input: $item) { id name status } } database cache Writes

Slide 34

Slide 34 text

Cache hydration query listItems { items { id name location } } database

Slide 35

Slide 35 text

Querying query listItems { items(filter: { eq: { location: “NYC” } }) { id name location } } cache

Slide 36

Slide 36 text

Querying query listItems { items(filter: { eq: { location: “NYC” } }) { id name location } } cache

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

MENTAL MODEL

Slide 39

Slide 39 text

AMPLIFY DATASTORE Multi-platform on-device persistent storage engine that automatically synchronizes data between mobile/web apps and the cloud using GraphQL.

Slide 40

Slide 40 text

Storage Engine Storage Adapter CRUD OBSERVES WRITES READS WRITES OBSERVES Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer

Slide 41

Slide 41 text

DATA MODELING WITH GRAPHQL Developer

Slide 42

Slide 42 text

DATA MODELING WITH GRAPHQL Developer AUTHORING

Slide 43

Slide 43 text

DATA MODELING WITH GRAPHQL Developer AUTHORING BUILD

Slide 44

Slide 44 text

DATA MODELING WITH GRAPHQL Developer AUTHORING BUILD NPM script, gradle task, or Xcode build phase

Slide 45

Slide 45 text

DATA MODELING WITH GRAPHQL Developer AUTHORING BUILD NPM script, gradle task, or Xcode build phase

Slide 46

Slide 46 text

DATASTORE Fluent interface Mutations / writes

Slide 47

Slide 47 text

DATASTORE Fluent interface Queries / reads

Slide 48

Slide 48 text

DATASTORE Fluent interface - with predicates

Slide 49

Slide 49 text

DATASTORE Fluent interface - with predicates & chaining

Slide 50

Slide 50 text

DATASTORE Mutations - with predicates & chaining

Slide 51

Slide 51 text

ARCHITECTURE Developer

Slide 52

Slide 52 text

Storage Engine CRUD OBSERVES ARCHITECTURE DataStore API Developer Model converted to / from the storage adapter of choice

Slide 53

Slide 53 text

Storage Engine Storage Adapter CRUD OBSERVES WRITES READS ARCHITECTURE DataStore API Developer Model converted to / from the storage adapter of choice Object and action persistence with specific storage engine (SQL Lite, IndexDB, etc..)

Slide 54

Slide 54 text

Storage Engine Storage Adapter CRUD OBSERVES WRITES READS ARCHITECTURE DataStore API Developer

Slide 55

Slide 55 text

Storage Engine Storage Adapter CRUD OBSERVES WRITES READS OBSERVES WRITES / READS Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer

Slide 56

Slide 56 text

Storage Engine Storage Adapter CRUD OBSERVES WRITES READS OBSERVES WRITES / READS Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer Model converted to GraphQL statement Response converted to Model and written to storage engine.

Slide 57

Slide 57 text

REAL TIME

Slide 58

Slide 58 text

REAL TIME OPTIONS

Slide 59

Slide 59 text

LONG POLLING

Slide 60

Slide 60 text

SERVER SENT EVENTS

Slide 61

Slide 61 text

WEB SOCKETS

Slide 62

Slide 62 text

GRAPHQL

Slide 63

Slide 63 text

LOCAL STORE + SUBSCRIPTIONS Fluent interface - observing data models

Slide 64

Slide 64 text

SINGLE REHYDRATE QUERY

Slide 65

Slide 65 text

SUBSCRIPTIONS

Slide 66

Slide 66 text

SINGLE REHYDRATE QUERY

Slide 67

Slide 67 text

CONFLICT DETECTION AND RESOLUTION

Slide 68

Slide 68 text

Provide a way to merge concurrent modiﬁcations, always, in any order. CONFLICT DETECTION & RESOLUTION

Slide 69

Slide 69 text

No content

Slide 70

Slide 70 text

update

Slide 71

Slide 71 text

update

Slide 72

Slide 72 text

update operation operation

Slide 73

Slide 73 text

update operation operation

Slide 74

Slide 74 text

update operation operation update

Slide 75

Slide 75 text

update operation operation update

Slide 76

Slide 76 text

update operation operation update

Slide 77

Slide 77 text

update operation operation update

Slide 78

Slide 78 text

update update

Slide 79

Slide 79 text

update update

Slide 80

Slide 80 text

update operation operation update

Slide 81

Slide 81 text

update update update

Slide 82

Slide 82 text

update update update

Slide 83

Slide 83 text

update update update

Slide 84

Slide 84 text

CONFLICT DETECTION & RESOLUTION Objects tagged with metadata

Slide 85

Slide 85 text

Conﬂict Free Replicated Data Types (CRDTs) Operational Transforms Google Docs, MS Office Online Redis, Facebook, CosmosDB CONFLICT DETECTION & RESOLUTION 2006 1989

Slide 86

Slide 86 text

Vector Hybrid Logical Clock (HLC) Monotonic CONFLICT RESOLUTION Clock

Slide 87

Slide 87 text

person = { name: “Jennifer”, age: 32 } original item CONFLICT RESOLUTION

Slide 88

Slide 88 text

person = { name: “Jennifer”, age: 32 } person = { name: “Jennifer”, age: 32, title: “Developer”, } person = { name: “Jennifer”, age: 32, title: “Engineer” } original item client a client b CONFLICT RESOLUTION

Slide 89

Slide 89 text

person = { name: “Jennifer”, age: 32, title: “Developer” } person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION

Slide 90

Slide 90 text

person = { name: “Jennifer”, age: 32, title: “Developer” } person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION person = { name: “Jennifer”, age: 32, title: “Developer” } new item

Slide 91

Slide 91 text

person = { name: “Jennifer”, age: 32 } original item CONFLICT RESOLUTION

Slide 92

Slide 92 text

person = { name: “Jennifer”, age: 32 } person = { name: “Jennifer”, age: 32, location: “Nebraska”, } person = { name: “Jennifer”, age: 32, title: “Engineer” } original item client a client b CONFLICT RESOLUTION

Slide 93

Slide 93 text

person = { name: “Jennifer”, age: 32, location: “Nebraska” } person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION

Slide 94

Slide 94 text

person = { name: “Jennifer”, age: 32, location: “Nebraska” } person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION person = { name: “Jennifer”, age: 32, location: “Nebraska”, title: “Engineer” } new item

Slide 95

Slide 95 text

person = { name: “Jennifer”, age: 32 hobbies: [“climbing"] } original item CONFLICT RESOLUTION Lists

Slide 96

Slide 96 text

person = { name: “Jennifer”, age: 32 hobbies: [“climbing"] } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”] } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “karaoke”] } original item client a client b CONFLICT RESOLUTION Lists

Slide 97

Slide 97 text

Slide 98

Slide 98 text

person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”] } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”,“karaoke”] } client a client b CONFLICT RESOLUTION Lists person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “karaoke”, “reading”] } new item

Slide 99

Slide 99 text

AMPLIFY DATASTORE CONFLICT DETECTION & RESOLUTION • Monotonic counters • Controlled by the system itself (AppSync) • GraphQL types with versions used for merge or reject decisions • Base table & change table with soft deletes & TTL

Slide 100

Slide 100 text

MONOTONIC COUNTER person = { id: 100, age: 30, _version: 1 } person = { id: 100, age: 31, _version: 2 } person = { id: 100, age: 32, _version: 3 } id age name _version 100 32 Amy 3 101 28 Charles 9 102 47 Melissa 8 Base Table

Slide 101

Slide 101 text

ds_pk ds_sk age name _version _lastChangedAt Person-9386:2020-01-23 13:16:08.127:9386:1 32 Amy 1 1579785368127 Person-9386:2020-01-23 13:16:21.785:9386:2 33 Amy 2 1579785381785 Person-9386:2020-01-23 13:16:37.859:9386:3 34 Amy 3 1579785397859 Change Table CONFLICT RESOLUTION

Slide 102

Slide 102 text

Slide 103

Slide 103 text

Slide 104

Slide 104 text

AUTO MERGE CONFLICT RESOLUTION

Slide 105

Slide 105 text

User A User B person = { age: 32, _version: 1 } person = { age: 32, _version: 1 }

Slide 106

Slide 106 text

User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 }

Slide 107

Slide 107 text

User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 }

Slide 108

Slide 108 text

User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } OFFLINE

Slide 109

Slide 109 text

User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } OFFLINE person = { age: 34, location: “Seattle” _version: 3 }

Slide 110

Slide 110 text

User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE person = { age: 34, location: “Seattle” _version: 3 }

Slide 111

Slide 111 text

Slide 112

Slide 112 text

person = { age: 34, age: 35, location: “Seattle” _version: 4 } User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 }

Slide 113

Slide 113 text

User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 } person = { age: 34, location: “Seattle” _version: 4 } person = { age: 35, location: “Seattle” _version: 4 }

Slide 114

Slide 114 text

AUTO MERGE SYNC ENABLED GRAPHQL RESOLVERS Base table + change table + GraphQL types with versions used for merge or reject decisions

Slide 115

Slide 115 text

AWS APPSYNC CONFLICT RESOLUTION • Automerge • AWS Lambda • Optimistic concurrency

Slide 116

Slide 116 text

SYNCHRONIZATION

Slide 117

Slide 117 text

id age name _version 100 32 Amy 8 101 28 Charles 9 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3

Slide 118

Slide 118 text

id age name _version 100 32 Amy 8 101 28 Charles 9 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3

Slide 119

Slide 119 text

id age name _version 100 32 Amy 8 101 28 Charles 9 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3

Slide 120

Slide 120 text

id age name _version 100 32 Amy 8 101 28 Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3

Slide 121

Slide 121 text

id age name _version 100 32 Amy 8 101 28 Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION

Slide 122

Slide 122 text

id age name _version 100 32 Amy 8 101 28 Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION

Slide 123

Slide 123 text

DELTA SYNC SYNCHRONIZATION

Slide 124

Slide 124 text

Slide 125

Slide 125 text

Slide 126

Slide 126 text

Slide 127

Slide 127 text

ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James 7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859

Slide 128

Slide 128 text

Slide 129

Slide 129 text

ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James 7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC syncPerson(lastSync: 13:16:22.127) Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859

Slide 130

Slide 130 text

ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James 7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC syncPerson(lastSync: 13:16:22.127) Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859 Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859

Slide 131

Slide 131 text

TIME TRAVEL SYNCHRONIZATION

Slide 132

Slide 132 text

id age name _version _deleted _ttl 100 32 Amy 8 101 28 Charles 9 102 47 Melissa 3 TRUE 1582384003 SYNCHRONIZATION ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1 32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003 Change Table Base Table

Slide 133

Slide 133 text

DB OPTIMIZATION SYNCHRONIZATION

Slide 134

Slide 134 text

TIME TO LIVE ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1 32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003