Offline storage & automatic data synchronization for your web and mobile applications

Slide 1

Slide 1 text

OFFLINE FIRST.

Slide 2

Slide 2 text

OFFLINE FIRST Core features should function with or without an internet connection. WHAT DOES IT MEAN? #01

Slide 3

Slide 3 text

OFFLINE FIRST Data is written to and queried locally from the end user’s device and periodically uploaded and replicated in the database. WHAT DOES IT MEAN? #02

Slide 4

Slide 4 text

OFFLINE FIRST Provide end users with a consistent UX when internet connectivity is slow or non-existent. WHAT DOES IT MEAN? #03

Slide 5

Slide 5 text

REAL TIME.

Slide 6

Slide 6 text

REAL TIME An application that functions within a time frame that the user senses as immediate or very close to it. WHAT DOES IT MEAN #01

Slide 7

Slide 7 text

REAL TIME Cohesive system where data is in sync across all connected devices. WHAT DOES IT MEAN #02

Slide 8

Slide 8 text

THE FUTURE OF REAL-TIME | OFFLINE | DATA

Slide 9

Slide 9 text

#01 Sébastien Stormacq INTRODUCTION. @sebsto /sebsto

Slide 10

Slide 10 text

CLIENT TECHNOLOGIES AWS AMPLIFY AWS APPSYNC

Slide 11

Slide 11 text

DEMO WITH .

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

WHY REAL TIME & OFFLINE APPLICATIONS

Slide 15

Slide 15 text

WHY MODERN APPLICATION DEVELOPMENT

Slide 16

Slide 16 text

WHY SLOW or NONEXISTENT   NETWORKS

Slide 17

Slide 17 text

WHY USER EXPERIENCE

Slide 18

Slide 18 text

CHALLENGES REAL TIME & OFFLINE APPLICATION

Slide 19

Slide 19 text

CACHING.

Slide 20

Slide 20 text

CACHE !== DATABASE

Slide 21

Slide 21 text

REAL TIME.

Slide 22

Slide 22 text

CONFLICT DETECTION.

Slide 23

Slide 23 text

CONFLICT RESOLUTION.

Slide 24

Slide 24 text

MULTIPLE PLATFORMS.

Slide 25

Slide 25 text

SCALABILITY.

Slide 26

Slide 26 text

SOLUTIONS REAL TIME & OFFLINE APPLICATION

Slide 27

Slide 27 text

CACHING

Slide 28

Slide 28 text

CACHING mutation create($item: InputType) { createItem(input: $item) { id name status } } database cache Writes

Slide 29

Slide 29 text

Cache hydration query listItems { items { id name location } } database

Slide 30

Slide 30 text

Querying query listItems { items(filter: { eq: { location: “NYC” } }) { id name location } } cache

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

MENTAL MODEL struct Episode: Identifiable, Hashable, Codable { let id: String let date: String let title: String let duration: String let description: String? } let newEpisode = Episode(id: UUID().uuidString, date: Date(), title: “New Episode", duration: “00:12:14”, description: "Description")

Slide 33

Slide 33 text

AMPLIFY DATASTORE Multi-platform on-device persistent storage engine that automatically synchronizes data between mobile/web apps and the cloud using GraphQL.

Slide 34

Slide 34 text

Storage Engine Storage Adapter CRUD OBSERVES WRITES READS WRITES OBSERVES Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer

Slide 35

Slide 35 text

DATA MODELING WITH GRAPHQL Developer AUTHORING BUILD codegen typescript, java, kotlin, swift type EpisodeData { id: ID! date: String! title: String! duration: String! description: String } struct Episode { let id: String let date: String let title: String let duration: String let description: String? }

Slide 36

Slide 36 text

DATASTORE Fluent interface Mutations / writes let episode = new Episode(date: "14 nov 2022", title: "New Episode", duration: "New Duration") try await Amplify.DataStore.save(episode)

Slide 37

Slide 37 text

DATASTORE Fluent interface Queries / reads Amplify.DataStore .query(for: Episode.self)

Slide 38

Slide 38 text

DATASTORE Fluent interface - with predicates Amplify.DataStore .query( for: Episode.self, where: e.podcastEpisodesId == podcast.id)

Slide 39

Slide 39 text

DATASTORE Mutations - with predicates try await Amplify.DataStore.delete(Episode.self, withId: episode.id)

Slide 40

Slide 40 text

Storage Engine Storage Adapter CRUD OBSERVES WRITES READS ARCHITECTURE DataStore API Developer Model converted to / from the storage adapter of choice Object and action persistence with specific storage engine (SQL Lite, IndexDB, etc..)

Slide 41

Slide 41 text

Storage Engine Storage Adapter CRUD OBSERVES WRITES READS OBSERVES WRITES / READS Sync Engine QUERIES/MUTATIONS SUBSCRIPTIONS ARCHITECTURE DataStore API Developer Model converted to GraphQL statement Response converted to Model and written to storage engine.

Slide 42

Slide 42 text

REAL TIME

Slide 43

Slide 43 text

LOCAL STORE + SUBSCRIPTIONS Fluent interface - observing data models let e = EpisodeData.keys Amplify.DataStore .observeQuery( for: EpisodeData.self, where: e.podcastDataEpisodesId == podcast.id)

Slide 44

Slide 44 text

SINGLE REHYDRATE QUERY

Slide 45

Slide 45 text

SINGLE REHYDRATE QUERY let e = EpisodeData.keys Amplify.DataStore .observeQuery( for: EpisodeData.self, where: e.podcastDataEpisodesId == podcast.id)

Slide 46

Slide 46 text

CONFLICT DETECTION AND   RESOLUTION

Slide 47

Slide 47 text

Provide a way to merge concurrent modi fi cations, always, in any order. CONFLICT DETECTION & RESOLUTION

Slide 48

Slide 48 text

CONFLICT DETECTION & RESOLUTION Objects tagged with metadata

Slide 49

Slide 49 text

person = { name: “Jennifer”, age: 32 } person = { name: “Jennifer”, age: 32, title: “Developer”, } person = { name: “Jennifer”, age: 32, title: “Engineer” } original item client a client b CONFLICT RESOLUTION - AUTO MERGE

Slide 50

Slide 50 text

person = { name: “Jennifer”, age: 32, title: “Developer” } person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION - AUTO MERGE person = { name: “Jennifer”, age: 32, title: “Developer” } new item

Slide 51

Slide 51 text

person = { name: “Jennifer”, age: 32 } person = { name: “Jennifer”, age: 32, location: “Nebraska”, } person = { name: “Jennifer”, age: 32, title: “Engineer” } original item client a client b CONFLICT RESOLUTION - AUTO MERGE

Slide 52

Slide 52 text

person = { name: “Jennifer”, age: 32, location: “Nebraska” } person = { name: “Jennifer”, age: 32, title: “Engineer” } client a client b CONFLICT RESOLUTION - AUTO MERGE person = { name: “Jennifer”, age: 32, location: “Nebraska”, title: “Engineer” } new item

Slide 53

Slide 53 text

person = { name: “Jennifer”, age: 32 hobbies: [“climbing"] } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”] } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “karaoke”] } original item client a client b CONFLICT RESOLUTION - AUTO MERGE Lists

Slide 54

Slide 54 text

person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “reading”] } person = { name: “Jennifer”, age: 32, hobbies: [“climbing”,“karaoke”] } client a client b CONFLICT RESOLUTION - AUTO MERGE Lists person = { name: “Jennifer”, age: 32, hobbies: [“climbing”, “karaoke”, “reading”] } new item

Slide 55

Slide 55 text

AMPLIFY DATASTORE CONFLICT DETECTION & RESOLUTION • Monotonic counters • Controlled by the backend (AppSync) • GraphQL types with versions used for merge or reject decisions • Base table & change table with soft deletes & TTL

Slide 56

Slide 56 text

MONOTONIC COUNTER person = { id: 100, age: 30, _version: 1 } person = { id: 100, age: 31, _version: 2 } person = { id: 100, age: 32, _version: 3 } id age name _version 100 32 Amy 3 101 28 Charles 9 102 47 Melissa 8 Base Table

Slide 57

Slide 57 text

ds_pk ds_sk age name _version _lastChangedAt Person-9386:2020-01-23 13:16:08.127:9386:1 32 Amy 1 1579785368127 Person-9386:2020-01-23 13:16:21.785:9386:2 33 Amy 2 1579785381785 Person-9386:2020-01-23 13:16:37.859:9386:3 34 Amy 3 1579785397859 Change Table typename + uuid + date timestamp + uuid + version CONFLICT RESOLUTION

Slide 58

Slide 58 text

AUTO MERGE CONFLICT RESOLUTION

Slide 59

Slide 59 text

person = { age: 34, age: 35, location: “Seattle” _version: 4 } User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 }

Slide 60

Slide 60 text

User A User B person = { age: 32, _version: 1 } person = { age: 33, _version: 2 } person = { age: 33, _version: 2 } person = { age: 32, _version: 1 } person = { age: 35, _version: 3 } OFFLINE ONLINE person = { age: 34, location: “Seattle” _version: 3 } person = { age: 34, location: “Seattle” _version: 4 } person = { age: 35, location: “Seattle” _version: 4 }

Slide 61

Slide 61 text

AUTO MERGE SYNC ENABLED GRAPHQL RESOLVERS Base table + change table + GraphQL types with versions used for merge or reject decisions

Slide 62

Slide 62 text

AWS APPSYNC CONFLICT RESOLUTION STRATEGIES Automerge AWS Lambda Optimistic concurrency

Slide 63

Slide 63 text

SYNCHRONIZATION

Slide 64

Slide 64 text

id age name _version 100 32 Amy 8 101 28 Charles 9 DELETE ID 102 SYNCHRONIZATION - BASIC IMPLEMENTATION 102 47 Melissa 3

Slide 65

Slide 65 text

DELTA SYNC SYNCHRONIZATION

Slide 66

Slide 66 text

ds_pk ds_sk age name _version _lastChangedAt Person-9387:2020-01-23 13:16:08.127:9387:7 22 James 7 1579785368127 Person-9388:2020-01-23 13:16:21.785:9388:3 41 Amy 3 1579785381785 Change Table SYNCHRONIZATION - DELTA SYNC syncPerson(lastSync: 13:16:22.127) Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859 Person-9389:2020-01-23 13:16:37.859:9389:1 34 Jennifer 1 1579785397859

Slide 67

Slide 67 text

TIME TRAVEL SYNCHRONIZATION

Slide 68

Slide 68 text

id age name _version _deleted _ttl 100 32 Amy 8 101 28 Charles 9 102 47 Melissa 3 TRUE 1582384003 SYNCHRONIZATION ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1 32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003 Change Table Base Table

Slide 69

Slide 69 text

DB OPTIMIZATION SYNCHRONIZATION

Slide 70

Slide 70 text

TIME TO LIVE ds_pk ds_sk age _deleted _ttl Person-9386:2020-01-23 13:16:08.127:9386:1 32 1582384003 Person-9386:2020-01-23 13:16:21.785:9386:2 33 1582384003 Person-9386:2020-01-23 13:16:37.859:9386:3 47 TRUE 1582384003 1582384003 1582384003 1582384003

Slide 71

Slide 71 text

PLATFORMS

Slide 72

Slide 72 text

PLATFORMS

Slide 73

Slide 73 text

https://docs.amplify.aws/

Slide 74

Slide 74 text

https://github.com/sebsto

Slide 75

Slide 75 text

THANK YOU! /sebAWS @sebsto /sebsto /sebsto

Slide 76

Slide 76 text

https://eventbox.dev/survey/ADK80QX ANONYMOUS SURVEY $25 AWS VOUCHERS