Storing app data reliably, performantly and easily

Almost every single app needs to store important pieces of data such as a session token on disk. Learn about the pitfalls of SharedPreferences, what to do for data needed for app launch, when to use something like Room for relational data, and how storage impacts the user experience especially in emerging markets. Learn how Google and Uber approach storage to ensure their apps are reliable.

Transcript

1. Storing app data reliably, performantly and easily Kurt Nelson Mobile Pla1orm at Uber
2. Last &me at DroidconSF...
3. Last &me at DroidconSF... The Dangers of SharedPreferences
4. SharedPreferences It's s&ll in the framework, and s&ll kind of works. S"ll avoided by both Uber and Google
5. Apps wri(en at Uber, Google & Facebook are deployed at scale.
6. When working on an app "at scale", there are some extra issues you have to worry about.
7. Storage at Scale
8. What is storage at scale? Performing I/O opera0ons on millions of devices, millions of 0mes a day, globally.
10. Storage Reliability
11. Why is storage reliability important?
12. Storage errors should not crash the user
13. Storage Reliability Applica'ons need to know when IO fails in a 'mely manner
14. Async callbacks improve the user experience
15. Explain fixable errors to the user When SharedPreferences#apply() fails, you never find out
16. Storage Reliability SharedPreferences sharedPref = context.getSharedPreferences("app", Context.MODE_PRIVATE); SharedPreferences.Editor editor = sharedPref.edit(); editor.putString("auth_token", authToken); editor.apply(); context.startActivity(MainScreenActivity.class); finish();
17. Block progress on failure when appropriate Essen%al for legal and privacy status, such as a EULA or GDPR opt- out.
18. Storage Performance
19. Why is storage performance important?
20. The user experience should always be smooth. Storage can cause frame drops.
23. It is easy to drop frames Running at 60fps gives the device 16 milliseconds to render each frame. Even on a modern device, the p95 storage read latency is already in this neighborhood. On low end devices, this can be dras;cally slower.
24. Storage Performance Different types of data have different performance needs.
25. Cache Performance Op/miza/ons If a cache read is latent, maybe it should be eagerly treated as a miss? If a cache write fails, does anyone care?
26. Storage Performance Cri$cal disk opera$ons should block progress with a spinner un$l complete
27. API Ergonomics
28. API Ergonomics • Understandable by new developers • Common async abstrac.on • Using correctly is easy • Consistant behavior for primi2ves and objects • Future proof serializa2on
29. I want a key-value store that ensures all of this.
30. I want a key-value store that ensures all of this. So I built it. Twice.
32. SimpleStore Simple yet performant asynchronous file storage for Android
33. SimpleStore h"ps:/ /github.com/uber/simple-store
34. Write a key val store = SimpleStoreFactory.create(context, MY_NAMESPACE) store.use { Futures.addCallback( it.putString("key", "value"), object : FutureCallback<String> { override fun onSuccess(result: String) { // update the UI } override fun onFailure(t: Throwable) { Log.e("MyActivity", "Something went wrong", t) } }, mainExecutor() ) }
35. Namespaces A namespace is the base unit that can be opened and closed. Only one reference to a specific namespace is allowed to exist at a 5me. Ordering is guarenteed within a namespace.
36. Write a key (RxJava) SimpleStoreFactory.create(context, MY_NAMESPACE).use { Single.fromFuture(it.putString("some_key", "Foo value")) .observeOn(uiThread()) .subscribe(updateView, handleError) }
37. Fetch a key (RxJava) SimpleStoreFactory.create(context, MY_NAMESPACE).use { Single.fromFuture(simpleStore.getString("some_key")) .observeOn(uiThread()) .subscribe(updateView, showError) }
38. Design Principles 1. Fundamentally async 2. Never emits null 3. Serializes as a byte[] 4. Releases all resources when desired 5. Supports configuring based on data type
39. Fundamentally Async IO opera)ons are interrupt driven.
40. Fundamentally Async Shared thread pool to enqueue IO opera1ons across all callsites Internally uses OrderedExecutor
41. Why not RxJava? Single.fromFuture instantly goes from ListenableFuture to RxJava.
42. Never Emits Null null in an Rx-stream or ListenableFuture is undesirable 0-length byte[] equivalent of the value instead
43. Serializes to byte[] Base class only handles String and byte[] Allows choosing your own object serializa5on
44. Releases Resources A"er close() is called, SimpleStore releases the in-memory cache.
45. Releases Resources When a namespace is closed, resources are freed Op#onal LRU mode for in-memory cache
46. Supports Configura.ons Caches Non-cri(cal Cri$cal The Next New Thing
47. Supports Configura.ons When opening a namespace, an op/onal configura/on can be passed. Used to place on disk
48. Supports Configura.ons Cache SimpleStoreFactory.create(context, "<namespace>", NamespaceConfig.CACHE) In the future, we can op0mize memory use for you.
49. Supports Configura.ons Non-cri(cal SimpleStoreFactory.create(context, "<namespace>") Default mode, good for data that is not on a hot-path.
50. Supports Configura.ons Performance Cri-cal SimpleStoreFactory.create(context, "<namespace>", NamespaceConfig.CRITICAL) Keeps data in memory as long as the store is open
51. Supports Configura.ons The Next New Thing Android Q's media scoping can be implemented down the line.
52. Primi%ves are in the box If you want to store more than just String and byte[], PrimitiveSimpleStore implements basic serializa9on of primi9ves for you.
53. If you're only interested in storing primi3ves, you can leave the talk now.
54. More than primi-ves!
55. SimpleStore for Protocol Buffers
56. SimpleStore for Protocol Buffers Companion ar*fact of simplestore-proto enables storing protoc generated POJOs directly.
57. SimpleStore for Protocol Buffers Atomically load complex models from disk. gRPC users are already using protos for models.
58. SimpleStore for Protocol Buffers Uses Google's Lite protoc plugin, con3bu3ons of other run3mes welcome.
59. val proto = Demo.Data.newBuilder().setField(editText.text.toString()).build() Futures.addCallback( simpleStore.put("some_thing", proto), object : FutureCallback<Demo.Data> { override fun onSuccess(payload: Demo.Data?) { editText.setText("") button.isEnabled = true editText.isEnabled = true loadMessage() } override fun onFailure(t: Throwable) { textView.text = t.toString() button.isEnabled = true editText.isEnabled = true } }, mainExecutor() )
60. Modular & flexible. Use whatever serializa.on format you like.
61. Possible Enhancements • Co-rou'ne wrapper • Kotlin na've compa'ble interface • Automa(c cache evic(on • Pipelining within a namespace • Key audi*ng & cleanup
62. Contribu)ons welcome!
63. Thanks! Kurt Nelson, Uber Mobile Pla3orm @kur%snelson h"p:/ /github.com/uber/simple-store