$30 off During Our Annual Pro Sale. View Details »

ADDC 2019 - Gopal Sharma - Building great offline-first apps

ADDC 2019 - Gopal Sharma - Building great offline-first apps

This talk will go through what an “offline-first” app is, why you should build offline-first apps, what some of the challenges of building them are, the problems (which appear simple on the surface, but are subtly complex) my team has faced building them, and how we solved those problems. We’ll look at the problem from a UX perspective first, and talk about some of the things to think about when designing offline-first apps. I will then look at the problem through a developer’s lens and talk about the architecture and techniques we used. We’ll finally briefly get into specific tools for iOS and Android that helped us get everything working.

Recordings & more: https://addconf.com/

More Decks by ADDC - App Design & Development Conference

Other Decks in Technology

Transcript

  1. Building Great Offline-First Apps
    26 June 2019
    Gopal Sharma
    @gopalkri
    https://bohr.in

    View Slide

  2. What Is an “Offline-First” App?
    • Work without internet connection
    • “Catch up” to a server later

    View Slide

  3. Example: Notes

    View Slide

  4. View Slide

  5. Do You Need to Be Offline-First?
    • Ubiquitous networks
    • Being offline-first is non-trivial
    • You may not need to be offline-first
    • Unless required, don’t do it!

    View Slide

  6. Why Build Offline-First Apps?

    View Slide

  7. Networks
    • Networks are (still) slow, flaky or expensive
    • International roaming
    • Rural areas
    • Networks are not as pervasive as we think
    • Airplanes
    • Basements
    • Elevators
    • Hospitals

    View Slide

  8. View Slide

  9. 14,000
    KM from BLR to SFO
    47ms
    At speed of light
    240ms
    Verizon: India ↔ North America
    360ms
    Me: India ↔ North America
    0.05ms
    Flash storage latency
    https://enterprise.verizon.com/terms/latency/
    https://www.snia.org/sites/default/education/tutorials/2010/spring/solid/LeviNorman_Latency_The_Heartbeat_SSD.pdf

    View Slide

  10. “You should never wait for your phone,
    your phone should wait for you”
    -Me

    View Slide

  11. User Experience
    • > 100 ms latency ➡ waiting on your phone *
    • People really dislike slow apps
    *https://www.youtube.com/watch?v=vOvQCPLkPt4

    View Slide

  12. Challenges

    View Slide

  13. Challenges
    • Conflicts
    • Inconsistencies
    • Errors
    • Performance

    View Slide

  14. Conflicts
    Device A
    Device B
    09:00
    Offline
    Offline
    09:02
    Note 1
    Device B
    09:03
    Note 1
    Device A
    09:04
    Online
    09:05
    Online
    09:06
    ???
    ???
    Note 1
    Server
    09:01
    Note 1
    Note 1
    Note 1
    Device A
    ??? ???

    View Slide

  15. Options?
    • Pick a version, discard other
    • Duplicate, keep both
    • Combine both

    View Slide

  16. Pick A Version
    • Data loss ⚠
    • Even if that’s okay, which version?
    • First?
    • Which one was first?
    • Client timestamps can’t be trusted
    • Server timestamps are better, but still not 100%
    accurate

    View Slide

  17. Duplicate
    • No data loss
    • Possibly annoying

    View Slide

  18. Combine
    • How?
    • No generic answer
    • Options
    • diff3
    • CRDTs
    http://www.cis.upenn.edu/~bcpierce/papers/diff3-short.pdf
    https://pages.lip6.fr/Marc.Shapiro/papers/RR-7687.pdf

    View Slide

  19. It Gets Worse…

    View Slide

  20. Example: Calendar
    Device B Merged
    Device A

    View Slide

  21. Inconsistencies
    • Records are conflict free
    • Data across records is inconsistent

    View Slide

  22. Example: Timing
    • Each block of time is a “Timer” record
    • Start time
    • End time
    • Project id
    • Can’t be doing two things at once
    • Records may not be in state of conflict
    • But, two records may be inconsistent
    • 9-10AM Project 1, 9:30 to 10:30 AM Project 2

    View Slide

  23. Building Consensus
    • Multiple devices
    • Server
    • Canonical source of truth? (~ SVN)
    • Distributed truth? (~ git)

    View Slide

  24. CloudKit Model
    http://www.vldb.org/pvldb/vol11/p540-shraer.pdf

    View Slide

  25. Server
    Device A Device B
    Note
    - id
    - changeToken
    - text
    - etc.
    Note
    id: 1
    ct: 1
    text: ABC
    Note
    id: 1
    ct: 1
    text: ABC
    Note
    id: 1
    ct: 1
    text: ABC
    Time: t0

    View Slide

  26. Server
    Device A Device B
    Note
    id: 1
    ct: 1
    text: ABCD
    Note
    id: 1
    ct: 1
    text: ABC
    Note
    id: 1
    ct: 1
    text: ABCE
    Time: t1

    View Slide

  27. Note
    id: 1
    ct: 2
    text: ABCD
    Note
    id: 1
    ct: 1
    text: ABCD
    Server
    Device A Device B
    Note
    id: 1
    ct: 1
    text: ABC
    Note
    id: 1
    ct: 1
    text: ABCE
    Time: t2
    Note
    id: 1
    ct: 1
    text: ABCD
    Note
    id: 1
    ct: 2
    text: ABCD
    Note
    id: 1
    ct: 2
    text: ABCD

    View Slide

  28. Note
    id: 1
    ct: 2
    text: ABCD
    Server
    Device A Device B
    Note
    id: 1
    ct: 2
    text: ABCD
    Note
    id: 1
    ct: 1
    text: ABCE
    Time: t3
    Note
    id: 1
    ct: 1
    text: ABCE
    Note
    id: 1
    ct: 2
    text: ABCD
    Note
    id: 1
    ct: 1
    text: ABCE
    Note
    id: 1
    ct: 1
    text: ABC

    View Slide

  29. Note
    id: 1
    ct: 1
    text: ABCE
    Note
    id: 1
    ct: 2
    text: ABCD
    Server
    Device A Device B
    Note
    id: 1
    ct: 2
    text: ABCD
    Note
    id: 1
    ct: 2
    text: ABCDE
    Time: t4
    Note
    id: 1
    ct: 2
    text: ABCD
    Note
    id: 1
    ct: 1
    text: ABCE
    Note
    id: 1
    ct: 1
    text: ABC

    View Slide

  30. Note
    id: 1
    ct: 2
    text: ABCDE
    Note
    id: 1
    ct: 2
    text: ABCD
    Server
    Device A Device B
    Note
    id: 1
    ct: 2
    text: ABCD
    Note
    id: 1
    ct: 3
    text: ABCDE
    Time: t5
    Note
    id: 1
    ct: 2
    text: ABCDE
    Note
    id: 1
    ct: 3
    text: ABCDE
    Note
    id: 1
    ct: 3
    text: ABCDE

    View Slide

  31. Note
    id: 1
    ct: 2
    text: ABCDE
    Note
    id: 1
    ct: 2
    text: ABCD
    Server
    Device A Device B
    Note
    id: 1
    ct: 3
    text: ABCDE
    Note
    id: 1
    ct: 3
    text: ABCDE
    Time: t6
    Note
    id: 1
    ct: 3
    text: ABCDE
    getLatest?since=2
    Note
    id: 1
    ct: 3
    text: ABCDE

    View Slide

  32. Error Handling

    View Slide

  33. Traditional Error Handling - Sync
    enum GarbageError: Error {
    case garbage(value: UInt32)
    }
    func maybeThrow() throws -> Int {
    let random = arc4random()
    if random > 10 {
    throw GarbageError.garbage(value: random)
    }
    return random
    }
    do {
    let value = try maybeThrow()
    print("Value is: \(value)")
    } catch {
    print("Error is: \(error)")
    }
    func maybeError() -> Int {
    let random = arc4random()
    if random > 10 {
    return -1
    }
    return random
    }
    let value = maybeError()
    if value < 0 {
    print("Error!")
    } else {
    print("Value: \(value)")
    }

    View Slide

  34. Traditional Error Handling - Async
    enum GarbageError: Error {
    case garbage(value: UInt32)
    }
    func maybeError(callback: (Result) -> Void) {
    let random = arc4random()
    if random > 10 {
    callback(.failure(.garbage(value: random)))
    } else {
    callback(.success(random))
    }
    }
    maybeError { result in
    switch result {
    case .success(let value):
    print("\(value)")
    case .failure(let error):
    print("Error: \(error)")
    }
    }

    View Slide

  35. Device A
    Device B
    ???

    View Slide

  36. A Solution
    • Persist errors in database
    • Create observable that fires when an error is found
    • Make common UI to get user to resolve error
    • Opt screens into observing error observable and
    triggering UI when it happens
    • Remove error from DB when resolved
    • Also works for detecting inconsistencies

    View Slide

  37. Performance

    View Slide

  38. Performance
    • How does data move between devices?
    • Cannot send complete data set back & forth
    • Need to sync deltas
    • Forward sync
    • Start with first record, move forwards
    • Get new records since last known change token

    View Slide

  39. Note
    id: 1
    ct: 3
    text: ABCDE
    Note
    id: 1
    ct: 2
    text: ABCD
    Note
    id: 1
    ct: 2
    text: ABCDE
    Server
    Device A Device B
    Note
    id: 1
    ct: 3
    text: ABCDE
    Note
    id: 1
    ct: 3
    text: ABCDE
    Note
    id: 1
    ct: 3
    text: ABCDE
    getLatest?since=2

    View Slide

  40. Architecture

    View Slide

  41. Primary DB ViewController
    Store
    Rx
    Observable
    Observable
    View 1
    View 2
    View 1
    Data
    View 2
    Data
    Read Path

    View Slide

  42. Write Path
    Primary DB ViewController
    Store
    Model
    View 1
    View 2
    View 1
    Event
    View 2
    Event

    View Slide

  43. Architecture - Sync
    Primary DB SyncManager
    Store
    ViewController
    Start sync
    APIClient
    Model
    Model

    View Slide

  44. Libraries & Tools
    • iOS
    • GRDB
    • RxSwift
    • RxGRDB
    • Android
    • SQLDelight
    • RxJava/Kotlin
    • Architecture components

    View Slide

  45. Summary
    • Building offline-first apps is hard
    • Offline-first apps can be great for your users
    • Superior user experience
    • Flaky internet connections
    • Challenges & solutions
    • Conflicts
    • Inconsistencies
    • Error handling
    • Performance

    View Slide

  46. Thank You!
    Questions?
    @gopalkri
    https://bohr.in

    View Slide