Meet Store 5 DroidconSF 2023

Transcript

1. MobileNativeFoundation/Store meet Store 5 mattramotar@androidDev.social friendlyMike@androidDev.Social Droidcon SF 2023
2. What is Store?
3. Store is a Java library for effortless, reactive data loading. Guarantee things [you don’t want to think about] are done in the same way What is Store?
4. Contracts + RELIABILITY Guarantee things are done in the same way What is Store?
5. Store A brief history 2017 2019 2021 2022 2023
6. Store A brief history 2017 2019 1.0 Released Jan 17th, 2017 by NY Times
7. 2017 2019 Why did we need Store? We wanted to do complicated thing once
8. Store Version 1.0 Api 2017 2019 Store [was] a Java library for effortless, reactive data loading built with RxJava Store<Article> Store = ParsingStoreBuilder.<BufferedSource, String>builder() .fetcher(this::ResponseAsSource) .persister(SourcePersisterFactory.create(context.getFilesDir()) .parser(GsonParserFactory.createSourceParser(gson, Article.class)) .open(); Barcode barcode = new Barcode("Article", "42"); store.get(barcode).subscribe(onNext, onError, onComplete)
9. 2017 2019 Fresh == skip memory/persister go directly to fetcher store.fresh(param).subscribe(onNext, onError, onComplete) Fresh Skip Caches
10. Store Why You Need User Research 2.0 Released 1 month later (oops!) 2017 2019
11. 3.0 Api Rxjava2 RxSingle instead of Observable Store<Article> Store = ParsingStoreBuilder.<BufferedSource,ArticleParam, String>builder() .fetcher(this::ResponseAsSource) //responseBody.source() .persister(SourcePersisterFactory.create(context.getFilesDir()) .parser(GsonParserFactory.createSourceParser(gson, Article.class)) .open(); ArticleParam param = new ArticleParam("42"); store.get(param).subscribe(onNext, onError) 2017 2019 Store
12. Coroutines have hit stable 2017 2019 2021 Fast forward to 2019
13. Bye bye RxJava Hello Kotlin 2017 2019 2021 Store4 Fast forward to 2019
14. Hello Kotlin 2017 2019 2021 Store4 StoreBuilder.from { api.fetchSubreddit(it, "10")} .sourceOfTruth( reader = db.postDao()::loadPosts, writer = db.postDao()::insertPosts, delete = db.postDao()::clearFeed) .cachePolicy(MemoryPolicy) .build() Api
15. Streaming as first class citizen 2017 2019 2021 fun stream(request: StoreRequest<Key>): Flow<StoreResponse>Output>> lifecycleScope.launchWhenStarted { store.stream(StoreRequest.cached(3, refresh = false)) .collect{ } store.stream(StoreRequest.get(3)) .collect{ storeResponse -> } Store4 Api
16. Loading|Content|Error Return Types 2017 2019 2021 Store 4 store.stream(StoreRequest.cached(key = key, refresh=true)).collect { response -> when(response) { is StoreResponse.Loading -> showLoadingSpinner() is StoreResponse.Data -> { if (response.origin == ResponseOrigin.Fetcher) hideLoadingSpinner() updateUI(response.value) } is StoreResponse.Error -> { if (response.origin == ResponseOrigin.Fetcher) hideLoadingSpinner() showError(response.error) }}}} LCE
17. Using Store 4 Examples are from Firefly for Mastodon https://github.com/digitalbuddha/Firefly 2017 2019 2021
18. You create a Store using a builder. The only requirement is to include a Fetcher which is just a typealias to a function that returns a Flow<FetcherResult<ReturnType>>. //build it val store = StoreBuilder.from( Fetcher.of { key: Unit -> userApi.noti fi cations( authHeader = " Bearer ${oauthRepository.getCurrent()}", offset = null) }).build() //use it store.stream(StoreRequest.cached(Unit, refresh = true)).map { it.dataOrNull() } .collect{response-> } Fetcher as a Function 2017 2019 2021
19. How we get data 2017 2019 2021
20. Do the hard thing once How to work with local persistence
21. Source of Truth suspend fun fetcher(key:String): Token = api.createAccessToken(key) fun sourceOfTruth = SourceOfTruth.of<String, Token, String>( reader = {dataStore.data.map { it.currentUser?.accessToken }}, writer = { _, token -> dataStore.updateData { it.user.copy(accessToken = token.accessToken) }}) private val userTokenStore: Store<String, String> = StoreBuilder.from( fetcher = Fetcher.of { key: String -> fetcher(key) }, sourceOfTruth = sourceOfTruth ).build() 2017 2019 2021
22. 2017 2019 2021 Store Data Flow
23. Fast forward to 2022 Where we were 365 days ago 2021 2022 2023
24. Store 4 is stable Less than a dozen open bugs 2021 2022 2023
25. API was perfect As long as you only asked me 2021 2022 2023
26. We had LOTS of feature requests 2021 2022 2023
27. That I was ignoring 2021 2022 2023
28. Write Support (CRUD) 2021 2022 2023
29. Conflict Resolution 2021 2022 2023
30. Fallback Mechanisms 2021 2022 2023
31. Support for Collections 2021 2022 2023
32. Store 5 Introducing Store now with batteries included
34. Mission-Critical Server-Driven Architecture
35. Sync
36. Collaboration
37. Upload + Download
38. Data Protection
39. Offline Access
40. Campaigns
41. Server-driven UI
42. Don’t worry Store 5 is a superset of Store4 Which targets JVM/iOS/Web
43. How we got here…
44. Try 1…Adding Write Support Rewrite what you don’t understand
45. Mike: What is the bene fi t of starting over? Matt: Hmm Mike: Is it easier to add old to new or new to old?
46. Some time later… (half hour) Matt: safeguards will be easier to add to new Matt: because I don’t understand old Mike: SGTM as long as all tests are migrated, I also don’t understand old Matt: Sure how hard can it be?
47. Many days later: Mike: …. Matt: …
48. More time passes Mike: My spider sense is tingling, I found more gaps in new Mike: let’s sit together and go through Store4 code
49. Try 2…Extending the old Rather than boiling the ocean
50. Few weeks later… Matt:Just finished last test cases everything passes Mike: Everything? Matt: Yup in iOS, android, JVM and JS, I took all the work https://github.com/aclassen did and got it to pass tests
52. What’s in Store 5
53. Mutations Con fl ict Resolution Fallback Mechanisms List Decomposition Cache Delegation Validation Concepts
54. meet Trails
55. An offline-first companion Trails
59. User Story - Hiking without Internet
60. Trails Architecture TLDR
61. Data Pulled & Pushed
62. Source of Truth to Repository Data Flow
63. Can’t do with Store 4 Source of Truth to Repository Data Flow
64. interface Updater<Key, Output, Response> { suspend fun post(key: Key, value: Output): UpdaterResult val onCompletion: OnUpdaterCompletion<Response>?} Store 5 New Mutation Api Creating an Updater (like a fetcher)
65. Updating Remote Data sample
66. val updater = Updater.by( post = { _, hike -> try { val latest = api.updateHike(hike) UpdaterResult.Success.Typed(latest) } catch (error: Throwable) { UpdaterResult.Error.Exception(error) }}) Store 5 Updater Factory
67. val mutableStore = store.asMutableStore( updater = updater, bookkeeper = bookkeeper) Store 5 Mutable Store Builder
68. class RealHikeManager @Inject constructor( private val mutableStore: MutableStore<Int, Hike>, private val location:locationRepository, private val map:mapRepository ): HikeManager { private suspend fun track() { combine(location.current(), map.current()) { location, map -> val request = createStoreWriteRequest(location, map) val response = mutableStore.write(request) handleResponse(response) }}} Store 5 Usage of MutableStore
69. class RealHikeManager @Inject constructor( private val mutableStore: MutableStore<Int, Hike>, private val location:locationRepository, private val map:mapRepository ): HikeManager { private suspend fun track() { combine(location.current(), map.current()) { location, map -> val request = createStoreWriteRequest(location, map) val response = mutableStore.write(request) handleResponse(response) }}} Store 5 Usage of MutableStore
70. What happens if write fails or a competing write?
71. Con fl ict Resolution Store 5 Concepts
72. https://developer.android.com/topic/architecture/data-layer/o ff l ine- fi rst#last-write Non Happy Path == Hard
74. How Does Store 5 Resolves Con fl icts? Implementing Principles
75. How Store 5 Resolves Conflicts We Don’t You Do! With Delegation
76. What we need from you A Bookkeeper (delegate)
77. Bookkeeper? Tracks when local changes fail to sync with network
78. Store 5 Bookkeeper Api interface Bookkeeper<Key : Any> { suspend fun getLastFailedSync(key: Key): Long? suspend fun setLastFailedSync(key: Key, timestamp: Long): Boolean suspend fun clear(key: Key): Boolean suspend fun clearAll(): Boolean}
79. MutableStore + Bookkeeper Before completing a read request, MutableStore asks Bookkeeper whether any conflicts might exist
80. Store 5 - Any Conflicts? RealMutableStore private suspend fun con fl ictsMightExist(key: Key): Boolean { val lastFailedSync = bookkeeper?.getLastFailedSync(key) return lastFailedSync != null || writeRequestsQueueIsEmpty(key).not() }
81. MutableStore + Bookkeeper And if conflicts might exist, MutableStore tries to eagerly resolve them
82. What we think you should do Resolve conflicts on remote
83. On each write request 1. Init thread safety (key-scoped) 2. Add request to stack (key-scoped) 3. Write to memory cache + SOT!!! 4. Try to update network (posting latest) Mutations In the Weeds
84. On each write request 1. Init thread safety (key-scoped) 2. Add request to stack (key-scoped) 3. Write to memory cache + SOT!!! 4. Try to update network (posting latest) In the Weeds Mutations
85. On each write request 1. Init thread safety (key-scoped) 2. Add request to stack (key-scoped) 3. Write to memory cache + SOT!!! 4. Try to update network (posting latest) In the Weeds Mutations
86. On each write request 1. Init thread safety (key-scoped) 2. Add request to stack (key-scoped) 3. Write to memory cache + SOT!!! 4. Try to update network (posting latest) In the Weeds Mutations
87. When network response 1. Success 1. Reset stack 2. Reset bookkeeper 2. Failure 1. Log with bookkeeper Try to update network
88. When network response 1. Success 1. Reset stack 2. Reset bookkeeper 2. Failure 1. Log with bookkeeper Try to update network
89. When network response 1. Success 1. Reset stack 2. Reset bookkeeper 2. Failure 1. Log with bookkeeper Try to update network
90. When network response 1. Success 1. Reset stack 2. Reset bookkeeper 2. Failure 1. Log with bookkeeper Try to update network
91. When network response 1. Success 1. Reset stack 2. Reset bookkeeper 2. Failure 1. Log with bookkeeper Try to update network
92. Fallback Mechanisms
93. Always have a backup plan! Fallback Mechanisms
94. Store 5 Fetcher Api fun <Key : Any, Network : Any> of( name: String? = null, fetch: suspend (key: Key) -> Network ): Fetcher<Key, Network> = ofFlow(name, fetch.asFlow())
95. fun <Key : Any, Network : Any> of( name: String? = null, fallback: Fetcher<Key, Network>, fetch: suspend (key: Key) -> Network ): Fetcher<Key, Network> = ofFlowWithFallback(name, fallback, fetch.asFlow()) Store 5 Fetcher Api with fallback
96. User Story - A/B Testing
97. data class FeatureFlag( val key: String, val name: String, val description: String, val kind: Kind, val version: Int, val creationDate: Long, ) : Identi fi able<String> { enum class Kind { Boolean, Multivariate}} Trails Feature Flag Model
98. Trails Feature Flag Status Model sealed class FeatureFlagStatus: Identi fi able<String> { data class Multivariate( val key: String, val value: FeatureFlagVariation, val lastRequested: Long, val links: Links)}
99. Trails Feature Flag Architecture
100. Fallback Mechanisms sample
101. Network Feature Flag Status Fetcher With Fallback val networkFetcher = Fetcher.ofWithFallback( name = “networkFetcher”, fallback = hardcodedFetcher ) { key -> when (key) { is Collection -> fetchFeatureFlagStatuses(key.userId) is Single -> fetchFeatureFlagStatus(key.userId, key. fl agId)}}
102. Hardcoded Feature Flag Status Fetcher val hardcodedFetcher = Fetcher.of( name = “hardcodedFetcher” ) { key -> when (key) { is Collection -> loadFeatureFlagStatuses(key.userId) is Single -> loadFeatureFlagStatus(key.userId, key. fl agId)}}
103. Build Store as normal val store = StoreBuilder.from( fetcher = networkFetcher, sourceOfTruth = sourceOfTruth, memoryCache = multiCache ).toMutableStoreBuilder<FeatureFlagStatusData, FeatureFlagStatusData>().build( updater = updater, bookkeeper = bookkeeper)
104. Concepts List Decomposition Cache Delegation
105. Working With Collections sample
106. Just works Store Loves KMP We target Android/ Desktop/iOS/JS
107. Store 5 Beta is out now github.com/mobilenativefoundation/store