Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ADDC 2019 - Gopal Sharma - Building great offline-first apps

ADDC 2019 - Gopal Sharma - Building great offline-first apps

This talk will go through what an “offline-first” app is, why you should build offline-first apps, what some of the challenges of building them are, the problems (which appear simple on the surface, but are subtly complex) my team has faced building them, and how we solved those problems. We’ll look at the problem from a UX perspective first, and talk about some of the things to think about when designing offline-first apps. I will then look at the problem through a developer’s lens and talk about the architecture and techniques we used. We’ll finally briefly get into specific tools for iOS and Android that helped us get everything working.

Recordings & more: https://addconf.com/

Transcript

  1. Building Great Offline-First Apps 26 June 2019 Gopal Sharma @gopalkri

    https://bohr.in
  2. What Is an “Offline-First” App? • Work without internet connection

    • “Catch up” to a server later
  3. Example: Notes

  4. None
  5. Do You Need to Be Offline-First? • Ubiquitous networks •

    Being offline-first is non-trivial • You may not need to be offline-first • Unless required, don’t do it!
  6. Why Build Offline-First Apps?

  7. Networks • Networks are (still) slow, flaky or expensive •

    International roaming • Rural areas • Networks are not as pervasive as we think • Airplanes • Basements • Elevators • Hospitals
  8. None
  9. 14,000 KM from BLR to SFO 47ms At speed of

    light 240ms Verizon: India ↔ North America 360ms Me: India ↔ North America 0.05ms Flash storage latency https://enterprise.verizon.com/terms/latency/ https://www.snia.org/sites/default/education/tutorials/2010/spring/solid/LeviNorman_Latency_The_Heartbeat_SSD.pdf
  10. “You should never wait for your phone, your phone should

    wait for you” -Me
  11. User Experience • > 100 ms latency ➡ waiting on

    your phone * • People really dislike slow apps *https://www.youtube.com/watch?v=vOvQCPLkPt4
  12. Challenges

  13. Challenges • Conflicts • Inconsistencies • Errors • Performance

  14. Conflicts Device A Device B 09:00 Offline Offline 09:02 Note

    1 Device B 09:03 Note 1 Device A 09:04 Online 09:05 Online 09:06 ??? ??? Note 1 Server 09:01 Note 1 Note 1 Note 1 Device A ??? ???
  15. Options? • Pick a version, discard other • Duplicate, keep

    both • Combine both
  16. Pick A Version • Data loss ⚠ • Even if

    that’s okay, which version? • First? • Which one was first? • Client timestamps can’t be trusted • Server timestamps are better, but still not 100% accurate
  17. Duplicate • No data loss • Possibly annoying

  18. Combine • How? • No generic answer • Options •

    diff3 • CRDTs http://www.cis.upenn.edu/~bcpierce/papers/diff3-short.pdf https://pages.lip6.fr/Marc.Shapiro/papers/RR-7687.pdf
  19. It Gets Worse…

  20. Example: Calendar Device B Merged Device A

  21. Inconsistencies • Records are conflict free • Data across records

    is inconsistent
  22. Example: Timing • Each block of time is a “Timer”

    record • Start time • End time • Project id • Can’t be doing two things at once • Records may not be in state of conflict • But, two records may be inconsistent • 9-10AM Project 1, 9:30 to 10:30 AM Project 2
  23. Building Consensus • Multiple devices • Server • Canonical source

    of truth? (~ SVN) • Distributed truth? (~ git)
  24. CloudKit Model http://www.vldb.org/pvldb/vol11/p540-shraer.pdf

  25. Server Device A Device B Note - id - changeToken

    - text - etc. Note id: 1 ct: 1 text: ABC Note id: 1 ct: 1 text: ABC Note id: 1 ct: 1 text: ABC Time: t0
  26. Server Device A Device B Note id: 1 ct: 1

    text: ABCD Note id: 1 ct: 1 text: ABC Note id: 1 ct: 1 text: ABCE Time: t1
  27. Note id: 1 ct: 2 text: ABCD Note id: 1

    ct: 1 text: ABCD Server Device A Device B Note id: 1 ct: 1 text: ABC Note id: 1 ct: 1 text: ABCE Time: t2 Note id: 1 ct: 1 text: ABCD Note id: 1 ct: 2 text: ABCD Note id: 1 ct: 2 text: ABCD
  28. Note id: 1 ct: 2 text: ABCD Server Device A

    Device B Note id: 1 ct: 2 text: ABCD Note id: 1 ct: 1 text: ABCE Time: t3 Note id: 1 ct: 1 text: ABCE Note id: 1 ct: 2 text: ABCD Note id: 1 ct: 1 text: ABCE Note id: 1 ct: 1 text: ABC
  29. Note id: 1 ct: 1 text: ABCE Note id: 1

    ct: 2 text: ABCD Server Device A Device B Note id: 1 ct: 2 text: ABCD Note id: 1 ct: 2 text: ABCDE Time: t4 Note id: 1 ct: 2 text: ABCD Note id: 1 ct: 1 text: ABCE Note id: 1 ct: 1 text: ABC
  30. Note id: 1 ct: 2 text: ABCDE Note id: 1

    ct: 2 text: ABCD Server Device A Device B Note id: 1 ct: 2 text: ABCD Note id: 1 ct: 3 text: ABCDE Time: t5 Note id: 1 ct: 2 text: ABCDE Note id: 1 ct: 3 text: ABCDE Note id: 1 ct: 3 text: ABCDE
  31. Note id: 1 ct: 2 text: ABCDE Note id: 1

    ct: 2 text: ABCD Server Device A Device B Note id: 1 ct: 3 text: ABCDE Note id: 1 ct: 3 text: ABCDE Time: t6 Note id: 1 ct: 3 text: ABCDE getLatest?since=2 Note id: 1 ct: 3 text: ABCDE
  32. Error Handling

  33. Traditional Error Handling - Sync enum GarbageError: Error { case

    garbage(value: UInt32) } func maybeThrow() throws -> Int { let random = arc4random() if random > 10 { throw GarbageError.garbage(value: random) } return random } do { let value = try maybeThrow() print("Value is: \(value)") } catch { print("Error is: \(error)") } func maybeError() -> Int { let random = arc4random() if random > 10 { return -1 } return random } let value = maybeError() if value < 0 { print("Error!") } else { print("Value: \(value)") }
  34. Traditional Error Handling - Async enum GarbageError: Error { case

    garbage(value: UInt32) } func maybeError(callback: (Result<UInt32, GarbageError>) -> Void) { let random = arc4random() if random > 10 { callback(.failure(.garbage(value: random))) } else { callback(.success(random)) } } maybeError { result in switch result { case .success(let value): print("\(value)") case .failure(let error): print("Error: \(error)") } }
  35. Device A Device B ???

  36. A Solution • Persist errors in database • Create observable

    that fires when an error is found • Make common UI to get user to resolve error • Opt screens into observing error observable and triggering UI when it happens • Remove error from DB when resolved • Also works for detecting inconsistencies
  37. Performance

  38. Performance • How does data move between devices? • Cannot

    send complete data set back & forth • Need to sync deltas • Forward sync • Start with first record, move forwards • Get new records since last known change token
  39. Note id: 1 ct: 3 text: ABCDE Note id: 1

    ct: 2 text: ABCD Note id: 1 ct: 2 text: ABCDE Server Device A Device B Note id: 1 ct: 3 text: ABCDE Note id: 1 ct: 3 text: ABCDE Note id: 1 ct: 3 text: ABCDE getLatest?since=2
  40. Architecture

  41. Primary DB ViewController Store Rx Observable<ViewData> Observable<Error> View 1 View

    2 View 1 Data View 2 Data Read Path
  42. Write Path Primary DB ViewController Store Model View 1 View

    2 View 1 Event View 2 Event
  43. Architecture - Sync Primary DB SyncManager Store ViewController Start sync

    APIClient Model Model
  44. Libraries & Tools • iOS • GRDB • RxSwift •

    RxGRDB • Android • SQLDelight • RxJava/Kotlin • Architecture components
  45. Summary • Building offline-first apps is hard • Offline-first apps

    can be great for your users • Superior user experience • Flaky internet connections • Challenges & solutions • Conflicts • Inconsistencies • Error handling • Performance
  46. Thank You! Questions? @gopalkri https://bohr.in