Offline First (Web) Apps - Coldfront 2015

A couple of years ago, mobile devices became A Thing™, and it took us all a couple more years to realise what the design implications were. From this, we got responsive webdesign. It took a whole lot longer to grasp the structural implications: all of our (web)apps and sites have been potentially distributed systems for quite a while now, running on devices with varying degrees of connectivity.

However, we've mostly been building them with a pre-2007 desktop understanding of the internet. This has led to widespread frustration, a lot of wasted time, and many stupid error messages. This is a talk both about how that happened and why it's time to change. About what it actually means to design, structure and build an offline-first app. About the benefits, opportunities and challenges of the approach, and how it can make your apps faster, more fun, and more robust.

Transcript

1. Offline-First
   (WEB) APPS
   Alex Feyerke / @espylaub
   Coldfront 2015, Copenhagen
   Good morning!
   This is „Offline First Web Apps“, my name is Alex, I‘m a web designer from
   Berlin, and I‘m very grateful to Kenneth and the Coldfront team for inviting me
   here. I‘m part of the team that makes Hoodie:
   

   View Slide

2. HOOD.IE
   an open-source server that lets you quickly build complete, data-driven web
   apps from the frontend. It‘s like jquery for data storage. And Hoodie is a bit
   special, because these apps are offline capable by default, meaning they won‘t
   break when the connection goes down, and this is basically why we‘re so
   interested in this whole topic. This isn‘t a talk about Hoodie, although it does
   come up a few times, it‘s about this idea of
   

   View Slide

3. Offline-First
   Concepts
   Interfaces
   Experiences
   Problems
   Offline First.
   So this is going to be dense and fast, but also not technical. This one is about
   higher level things like
   Concepts, Interfaces, Experiences, and about understanding what the
   problem is in the first place.
   There are actually several talks today should be a lot more hands-on than this
   one.
   But before we get started, let‘s take a small detour and talk about building
   things.
   

   View Slide

4. From 1933 to 1957, there was a place called the Black Mountain College in
   North Carolina. Great thinkers and artists from all over the world wanted to
   remake education, make it holistic, give arts and sciences and crafts equal
   emphasis. One of the teachers there was…
   

   View Slide

5. Bucky
   Buckminster Fuller. The Dome Guy. Seen here with a dome. He thought
   domes were the future of architecture.
   

   View Slide

6. Bucky
   So he built one for the 1967 World Fair in Montreal. The future was coming,
   and he wanted to build in a way that reflected this: new forms of living, new
   forms of community, new materials. And the dome had lots of advantages: it
   was sturdier, used less material, needed less energy to heat, and seemed like
   the structure of the future. Efficient, robust, highly aesthetic, and geometrically
   perfect.
   So why don‘t we all live in domes?
   

   View Slide

7. This one, for example, caught fire. But what about the idea in general? We‘re
   in the future now, after all.
   Turns out what people really want is a building they can make their own, that is
   robust, adaptable, and flexible. Brick, wood and concrete boxes: simple,
   dependable, proven. Utterly traditional. You may despair at how conservative
   people and their choices are, but you'll have to admit that in the end, you'd
   rather have the thing that works, the thing that acknowledges how your actual
   reality is. Doesn‘t have to be perfect. You‘re not perfect either. And ironically,
   when the Black Mountain College had to expand its facilities, this is what they
   built:
   

   View Slide

8. A box.
   People want to have doors, and cupboards, and windows to lean out of.
   Awnings to sit under, and walls to screw shelves into. They'd like to be able to
   put in a dividing wall or extend the building. They want to make good use of the
   space they have to build on. And it turns out that, paradoxically, the most
   conservative option turns out to be the most adaptable, the one most suited to
   everyday life.
   That box? Could be any number of things for any number of people. It‘s
   adaptable to everyday needs.
   

   View Slide

9. Now keep that in mind for a second as we zoom back into the present.
   Let‘s look at how we build, how we work.
   

   View Slide

10. How we work
    IT‘S BASICALLY
    SCIENCE FICTION
    We make web apps. Or native apps, for our purposes, it doesn't make a
    difference. But we make, and while we make, we imagine those things we
    make in their completed, working, usable state. We imagine them in the future,
    where all the cool people are using our stuff and making us rich and famous.
    But we‘re always saying: „In the near future, when this is ready, your life will
    improve a tiny bit“
    And too often, our industry will communicate this like glossy science fiction:
    

    View Slide

11. Small companies like plastc, this „20 credit cards in one“-hardware startup, and
    like most pre-order/crowdfunding companies, they have a vision, and they
    have a video. And it‘s basically a tiny slice of science fiction, a year or two from
    now: „here‘s how we imagine real human beings behaving in the near future“.
    And of course they show the tech and all, but it doesn‘t really matter what the
    product is, they all work by showing this evocative shiny, perfect and slick
    future.
    [plastc.com]
    

    View Slide

12. And this scales throughout the industry. Microsoft, also loves imagining the
    future. They regularly release a so-called "Productivity Future Vision". Now, the
    name sounds as if someone just pulled three human resources buzzwords
    from a hat, but let‘s take a look at the current one:
    

    View Slide

13. https://www.youtube.com/watch?v=w-tFdreZB94
    Microsoft‘s Productivity Future Vision 2015
    But it feels… wrong. It's too smooth. It's not normal enough.
    But it‘s the future! Every flat surface is a screen! You‘re always connected,
    under water, in rural Cambodia, everywhere. The network is perfect, so your
    devices have no signal indicators. Your batteries are perfect, so your devices
    have no battery indicators either.
    Source: https://www.youtube.com/watch?v=w-tFdreZB94
    

    View Slide

14. And this is how it envisions you talking to your dad, who, as far as I can tell
    from the video, is in the same bloody building. The whole thing is a
    choreography of wealth and status, and it‘s utterly improbable in its perfection.
    

    View Slide

15. Everyone's happy, nothing is broken or
    smudged. There is infinite wi-fi with
    infinite bandwidth, no batteries run out,
    no traffic, no delays.
    —Tobias Revell
    Critical Design / Design Fiction
    All the systems in these videos are always flawless.
    „Everyone‘s happy…“, even old grandad with his wonky knee, because the
    humans may fail, but the systems always work.
    And by the way, you should absolutely look up Tobias Revells‘s work, it's eye-
    opening and very well done, which is why I'm stealing a bit more of it.
    Now, Google Glass used to be the future, now it‘s the past, but while it was still
    a thing, of course Google made a near-future sci fi video for it:
    [source: http://blog.tobiasrevell.com/2013/12/critical-design-design-fiction-
    lecture.html]
    

    View Slide

16. https://www.youtube.com/watch?v=v1uyQZNg2vE
    Google Glass Promo Video
    This is marginally better: there‘s normal stuff like playing with dogs or a being
    late for a flight, but the basic symptoms are the same: the people, the lives, the
    circumstances: most of them are outliers, all very privileged people in a kind of
    post-scarcity scenario where, as Revell puts it, "it's a classic case of everyone
    suddenly being an ice sculptor and flying around in balloons". It's also not
    normal. And again: the systems are perfect where the humans aren‘t.
    It again assumes a perfect, global network, even in the sky, so it‘s still
    improbable, but has hints of normality.
    And now:
    

    View Slide

17. Let‘s go back a bit
    This next one is from 1969, with a vision of the 1990s. Same era as the
    Montral Dome from a minute ago. So I‘d just like you to picture that in your
    mind for a second. The future, seen from the 60‘s. You‘ve all seen the movies.
    Ok?
    Now consider that this was made by the British Post Office…
    

    View Slide

18. https://www.youtube.com/watch?v=ONESDY9KMes
    Post Office in the 1960s - A glance 30 years into the future
    from Dollis Hill Research Station
    It seems rather normal. Why is that? Because it‘s not perfect. We see normal
    people at normal desks in an normal office. Also: the very first thing that
    happens: it doesn't work properly. The guy at the other end can‘t understand
    anything. And the weather's bad. Because it's London. It‘s normal. Listen:
    "suppose she makes an error". People don‘t make mistakes in Microsoft‘s
    future vision. Nobody has to ask glass to do something twice in the google
    thing.
    And the topics? Leases, bank statements, mortgages… normal normal normal.
    It's mundane, it's a bit wonky, but it's everyday life.
    What this shows is that thinking about the future is obviously a bit hard,
    because…
    

    View Slide

19. […] for every miraculous iPad there are
    countless partly broken realities: WiFi
    passwords, connectivity, battery life,
    privacy and compatibility amongst
    others.
    

    View Slide

20. The real skill of creating a compelling
    and engaging view of the future lies not
    in designing the gloss, but in seeing
    beyond the gloss to the truths behind it.
    —Nick Foster
    The Future Mundane
    Source: http://www.core77.com/blog/columns/the_future_mundane_25678.asp
    

    View Slide

21. And I just said this when speaking about this product sci-fi:
    „here‘s how we imagine real human beings behaving in the near future“
    But what I really mean, and what you really want, is
    „here‘s how we imagine something utterly normal happening in the near future“
    

    View Slide

22. We‘re not normal
    I'm showing you all this because we, as, and I'm sad to have to say this from
    up here, as well-educated, well paid people in the tech industry in a super-
    wealthy country, we're also not normal.
    

    View Slide

23. And the circumstances in which we work aren't normal either, they usually bear
    little relationship to how and where our products will probably be used. We
    build from behind shiny new macs with full-sized keyboards in well-equipped
    offices with fast connections, but the target environments are radically diverse
    and different.
    And that‘s just us in our normal offices. The closer you get to San Francisco,
    the exponentially less normal things get.
    

    View Slide

24. Thing is: It used to be that if your device could consume the web, it could also
    make it. That‘s no longer true for the majority of people.
    So you had to learn that people‘s devices were different than your dev
    machine, and now you realize that their circumstances are also very different,
    and require a bit of an empathic leap.
    Because their experience of the web is not only different to ours, it‘s also
    different to our historical experience of the web. Their first contact is radically
    different than ours was.
    But what counts for us, now, is that:
    

    View Slide

25. Our circumstances
    define our mindset
    We have the best tools and the best infrastructure available to us, and of
    course, that informs our mindset when we build all the new shiny future things.
    

    View Slide

26. BART 2
    An extreme example: If you take a google/apple/ebay/facebook bus to work,
    you don't get the usual public transport experience. You've got free corporate
    wifi, air conditioning, leather seats… your commuting experience is far, far
    removed from that of basically everyone else. That‘s probably your baseline
    now.
    But in a way, we‘re all on the google bus. We‘re super privileged. And we
    frequently forget that we have to…
    [http://www.wired.com/2013/09/mapping-silicon-valleys-corporate-shuttle-
    problem/]
    

    View Slide

27. Build for
    a normal world
    Normality is where most of life happens.
    

    View Slide

28. Build for
    a mobile world
    And the normal world, with all the normal people it, that have average budgets
    and average resources, that world, from your perspective as a developer, is
    overwhelmingly mobile. Phones are mobile, tablets are mobile, this thing is
    mobile. This is a mobile device. Hundreds of thousands of people use these
    with 3G or LTE sticks. You might not, but they do.
    The big grey box is a relic. Mobile is normal.
    

    View Slide

29. Normality
    is imperfect
    And you all know this: it's imperfect. Normality is imperfect. The web is very,
    very imperfect. The experience you're building? It's going to break for people.
    It's imperfect in ways you can't influence. You can write all the tests you want,
    but you can‘t fix 3G on the subway.
    And we do make attempts to import this reality into our work environments:
    

    View Slide

30. I saw a talk the other day that showed this image, stating that „this is the
    internet now“.
    But that‘s not true. That‘s superficially what it looks like, on the surface, but
    what that is is a bunch of devices connected to a local dev server that‘s
    propably in the same room. But that iPad‘s never going on a plane, and that
    kindle will never be on the subway, and that game boy… doesn‘t matter.
    The point is not that you have to carry all those devices around in the world for
    testing, but rather that…
    

    View Slide

31. …you can now build experiences to accommodate this normality, experiences
    that go beyond your apps just breaking, becoming unresponsive, showing
    empty views, losing data, making people nervous with panicky error messages.
    You can now build your apps in ways that acknowledge the fact that the web is
    imperfect, that your connections to it are also imperfect, and you can provide
    good experiences regardless.
    

    View Slide

32. Offline is
    not an error
    because Offline is not an error.
    

    View Slide

33. Offline is
    a fact of mobile life
    and therefore…
    

    View Slide

34. Offline is
    a fact of normal life
    It‘s just a state your app can be in. And a fairly likely one at that. It‘s not exactly
    an edge case.
    So here‘s the basic idea behind Offline First:
    

    View Slide

35. Treat the network as
    an enhancement
    OFFLINE-FIRST
    IN A NUTSHELL
    Beyond the first load, just don‘t assume you have a network connection.
    So how does that work, conceptually? Let‘s take a quick look at Hoodie‘s
    

    View Slide

36. Offline-First
    Architecture
    A SUPER SHORT OVERVIEW
    offline-first architecture. Just to see one way to do his whole thing.
    

    View Slide

37. WHAT NEEDS TO BE OFFLINE?
    An App's offline capability is guaranteed through two separate aspects:
    

    View Slide

38. • The app and its assets
    WHAT NEEDS TO BE OFFLINE?
    - the app itself and its assets must be available offline. This is done through
    Appcache or Serviceworker, or through a native wrapper such as phonegap or
    electron. In a native app, this is a given.
    

    View Slide

39. • The app and its assets
    • The app‘s data
    WHAT NEEDS TO BE OFFLINE?
    more interestingly, the app must handle data in a way that doesn't require a
    connection. This is best done by reading and writing through a local data store
    which syncs, when possible, to the server.
    Let‘s see how hoodie handles this.
    

    View Slide

40. FRONTEND BACKEND
    Hoodie Sync
    

    View Slide

41. FRONTEND
    App
    BACKEND
    Hoodie Sync
    

    View Slide

42. FRONTEND
    App
    hoodie.store
    BACKEND
    Hoodie Sync
    The app only ever talks to the hoodie API, never directly to the server-side
    code, the database, or even in-browser storage.
    

    View Slide

43. FRONTEND
    App
    hoodie.store
    browser store
    BACKEND
    Hoodie Sync
    [explain slide]
    As far as this concept goes, feel free to replace localstorage with any in-
    browser storage of your choice.
    This, by itself, is enough for an app. You can wrap that in node-webkit and sell
    it to people who don‘t trust the cloud. But let‘s assume you want more, you
    want user management and syncing.
    

    View Slide

44. FRONTEND
    App
    hoodie.store
    Sync
    CouchDB
    REST
    BACKEND
    Hoodie Sync
    browser store
    [explain slide]
    We can do syncing because each user has their own little private database
    which only they can access. So it‘s very simple to decide what gets synced. It‘s
    the user‘s private data. Of course it should be on their machine, and we can
    easily keep people‘s data separate on the server.
    

    View Slide

45. FRONTEND
    App
    hoodie.store
    Sync
    Plugins
    (node.js)
    CouchDB
    REST
    BACKEND
    Hoodie Sync
    browser store
    So, how does it work?
    [Explain a direct message moving through the system]
    So we never have element talk directly to each other. They only leave each
    other messages and tasks, like passive aggressive roommates. It‘s all very
    loosely coupled and event-based. Which means that it can be interrupted at
    any stage without breaking. Messages and tasks will be delivered and acted
    upon whenever possible. It‘s designed for eventual consistency.
    

    View Slide

46. FRONTEND
    App
    hoodie.store
    BACKEND
    Hoodie Sync
    Sync
    Anyone home?
    Out for lunch,
    BRB
    browser store
    The nice thing is: in most cases, the frontend doesn‘t care whether the
    backend is actually there or not. It will hang on to your data and sync whenever
    it can, and if your UI allows it, users can keep working without interruption.
    

    View Slide

47. FRONTEND
    App
    hoodie.store
    BACKEND
    Hoodie Sync
    Sync
    I don‘t care!
    Out for lunch,
    BRB
    browser store
    offline first apps are the web's honey badgers: they don't care, they just do their
    thing. Bees, tunnels, bad connectivity: the offline badger just keeps on going.
    It's how things should be on the internet: robust and fault-tolerant.
    Anyway, the point was:
    

    View Slide

48. FRONTEND
    App
    hoodie.store
    Sync
    Plugins
    (node.js)
    CouchDB
    REST
    BACKEND
    Hoodie Sync
    browser store
    If you want true offline capability, your app shouldn‘t try to talk to the server
    directly, only through sync. And you always want to keep a local copy in
    browser storage. That‘s really the central point: the user‘s data on the user‘s
    device, always.
    Sync is pretty hard. Really. You probably don‘t want to be implementing this
    yourself, there‘s just so much to get wrong.
    But whatever tech you choose to solve this with:
    

    View Slide

49. Getting into the offline mindset requires only
    one key realisation: when you leave the world
    of timely, reliable communication, the local
    database, not the server’s, must be the
    gateway for all persistent changes in
    application state.
    — Aanand Prasad
    Offline Support is Valuable, and You Can’t Add it Later
    So then, the big question:
    [http://aanandprasad.com/articles/offline/]
    

    View Slide

50. How do I make this
    offline magic work?
    Magiiiiiiic spaaarkles
    

    View Slide

51. //  Store  some  data
    hoodie.store.add('todo',  {name:  'fly  to  copenhagen'})
    Using Hoodie
    No magic, just a line of code.
    This is what using hoodie looks like. There‘s a hoodie object, with a store
    object, which has a couple of methods, like „add“. Add expects a type (todo)
    and an object with data in it.
    

    View Slide

52. //  Store  some  data
    hoodie.store.add('todo',  {name:  'fly  to  copenhagen'})
    //  Sign  up  a  new  user
    hoodie.account.signUp('yay@hood.ie',  'secret');
    Using Hoodie
    Sign up a new user: use the API
    

    View Slide

53. //  Store  some  data
    hoodie.store.add('todo',  {name:  'fly  to  copenhagen'})
    //  Sign  up  a  new  user
    hoodie.account.signUp('yay@hood.ie',  'secret');
    //  Listen  for  store  events
    hoodie.store.on('add',  handleNewObject)
    Using Hoodie
    Want to know when something changed so you can update the UI?
    Listen to events from the library
    

    View Slide

54. //  Store  some  data
    hoodie.store.add('todo',  {name:  'fly  to  copenhagen'})
    //  Sign  up  a  new  user
    hoodie.account.signUp('yay@hood.ie',  'secret');
    //  Listen  for  store  events
    hoodie.store.on('add',  handleNewObject)
    Using Hoodie
    Wait, so how do I offline? 0_o
    So how do you make this work offine?
    

    View Slide

55. //  Store  some  data
    hoodie.store.add('todo',  {name:  'eat  smørrebrød'})
    //  Sign  up  a  new  user
    hoodie.account.signUp('yay@hood.ie',  'secret');
    //  Listen  for  store  events
    hoodie.store.on('add',  handleNewObject)
    Using Hoodie
    That‘s it.
    All you need to do is embrace a decoupled, event based architecture, which
    you probably do anyway. Talk to the api, let it sync for you, and listen to events
    from Hoodie to see if anything happened.
    And that‘s it. It‘s not a special feature you have to explicitly invoke, it‘s how the
    entire architecture works. You get offline for free.
    Let‘s just see that in action:
    

    View Slide

56. Hoodie
    Demo
    Middle: So you see, you don‘t only get offline for free, you also get multi-
    device-sync for free.
    End: this illustrates our underlying strategy: if you can make it unobtrusive,
    make it unobtrusive. The simpler it is, the more people can use it.
    [haven‘t uploaded the video for this yet, sorry]
    

    View Slide

57. Sweet.
    Now what?
    So, we felt like we'd solved a lot of the technical aspects, and now we were
    wondering: what does this mean? Which opportunities does this give us,
    exactly? Which problems can we now solve?
    And in my research I noticed I was suddenly reading articles from an area of
    computer sciences I really wasn‘t expecting. Because if you have several data
    stores with the same data in your system…
    

    View Slide

58. You‘re all in the
    distributed systems
    business now
    … you‘ve got a distributed system, for better or worse. And now a whole new
    range of interesting problems and opportunities arise. But before we even go
    there, we can look at some insights from the distributed systems world and see
    whether they can help us.
    And as we all know, there are only two hard problems in distributed systems:
    

    View Slide

59. Seriously though.
    There are three things that pop up again and again:
    

    View Slide

60. — Tyler Treat
    Distributed Systems are a UX Problem
    1. The cost of knowing the „truth“ is
    prohibitively expensive
    A vendor is selling you a thing, but the vendor's system is not actually checking
    whether the item is actually present in the warehouse, or even if the
    warehouse still exists. It might be on fire, for all the web interface knows. But
    doing this check is prohibitively expensive and also quite challenging. There
    are other factors. Five other people may simultaneously buy something that is
    only in stock once. You could throw lots of money at engineering and try and
    make this perfect, and prevent four of those sales. But you're really pushing
    the limits of practicality. It is much more sensible to have your systems make
    educated guesses and assumptions, and make trade-offs on the few occasions
    when they're wrong. And then apologise to people, and recover from these
    edge cases and race conditions.
    

    View Slide

61. — Tyler Treat
    Distributed Systems are a UX Problem
    1. The cost of knowing the „truth“ is
    prohibitively expensive
    2. There is no „now“
    This is just physics. The parts of your system are not just distributed
    conceptually, there are actual physical distances between them, and there is
    latency between them. Your data realities can partition and end up on separate
    timelines that you have to merge again.
    And that means that…
    

    View Slide

62. — Tyler Treat
    Distributed Systems are a UX Problem
    1. The cost of knowing the „truth“ is
    prohibitively expensive
    2. There is no „now“
    3. Failure is not only an option, it‘s an
    inevitability
    Warehouses catch fire. Connections go down. Networks partition. Hardware
    fails. The software is buggy.
    We can‘t know everything, we can‘t control everything. So how can we gain
    certainty about anything?
    

    View Slide

63. — Tyler Treat
    Distributed Systems are a UX Problem
    1. The cost of knowing the „truth“ is
    prohibitively expensive
    2. There is no „now“
    3. Failure is not only an option, it‘s an
    inevitability
    Now Tyler here is urging us to stop trying to build perfect systems, but this is
    only half of our problem:
    You're building your app to be perfect, but at the same time, you're expecting
    the systems it runs on to be perfect, too. Just like in the Microsoft and Google
    videos assume a perfect world for their apps and services to run on.
    Anway, these three points aligned nicely with the problems people were having
    with offline scenarios:
    

    View Slide

64. Because we held workshops and talks, wrote blog posts and spoke to lots of
    people about this. We were surprised at how many developers suddenly
    opened up about their offline troubles—realizing they’d had similar problems in
    the past, but never spoken to others about them, because it wasn‘t a thing.
    People had veeeery interesting and specific problems, ranging from building,
    and I quote, „runkeeper for fish“, for anglers in the norwegian wilderness, to
    several people trying to build super robust point-of-sale systems for places with
    bad connectivity.
    But we identified some central themes:
    

    View Slide

65. Trust and
    reliability
    OFFLINE PROBLEMS
    The first and most pressing issue people seem to have is that they can‘t trust
    their apps to do the right thing when the connection gets dodgy. You can
    summarize this with the following user story:
    

    View Slide

66. As a mobile app user,
    I want to be sure
    that my data is actually
    available when I need it
    USER STORY
    „As a mobile app user…“ and we‘ve seen that mobile is the new normal, so…
    

    View Slide

67. As a mobile app user,
    I want to be sure
    that my data is actually
    available when I need it
    USER STORY
    And this is quite fundamental. It‘s so fundamental it probably gets forgotten
    most of the time. Of course users want this. But it used to be a painful
    problem, and it was easier to just fail, show errors and let people blame it on
    the telekom.
    But you can often do a lot better, and offline first is the way to go. But first, let
    me illustrate the problem:
    

    View Slide

68. So here‘s the german railway advertising that you can apparently book tickets
    and use their app under any circumstances, even when free soloing in the alps
    (remember, in the tech future, everyone is into extreme sports and ice carving)
    

    View Slide

69. vs.
    and then there‘s the normal world with this guy who can‘t access his ticket,
    because the app won‘t start up when the connection is bad. It‘ll work in
    airplane mode, but on an actual high-speed train, where the connection is slow
    and intermittent, it fails, and it fails slowly. While the ticket inspector is
    breathing down your neck. A german ticket inspector. Bad user experience.
    The solution would be to store personal data and tickets locally, and not make
    the initial communication with the server blocking.
    

    View Slide

70. Another symptom: screenshotting apps, because you can‘t trust them to retain
    their state.
    Who here does this?
    

    View Slide

71. Lots of people do this. This is my camera roll from a recent trip to Japan. Most
    of my photos are screenshots from where I was afraid that the data I‘d just
    fetched would vanish the next time I opened my phone. Because this kind of
    thing happens a lot:
    

    View Slide

72. Those views were full of data just moments before I took these screenshots,
    but then I made the mistake of, I don‘t know, switching apps or putting my
    phone away for a minute. And what happens? The apps ditch any data they
    may have had, then fail at loading new data, and then the user suddenly has
    no data at all. That‘s just not cool.
    So, we can use offline first to get…
    

    View Slide

73. More trustworthy
    and reliable apps
    OFFLINE FIRST OPPORTUNITY
    

    View Slide

74. Always-online
    architecture
    OFFLINE PROBLEMS
    The second issue is that many apps are designed to only work while
    connected, even if there‘s absolutely no intrinsic requirement to do so.
    This is a relic from the age of the big grey box, of the terminal and the thin
    client, where all the computing power was probably far away from you. Now it‘s
    in your pocket, and we can do things differently.
    

    View Slide

75. As a mobile app user, when I‘m
    offline, I want to be able to use
    features that don‘t require a
    connection
    USER STORY
    

    View Slide

76. If you‘re only viewing your own data, or adding data yourself, why doesn‘t this
    just always work, online or offline? You should be able to add a meeting, mark
    a task as done, write a message, check in, post a picture… create stuff. Store
    now, sync later, but don‘t get in the way of getting things done if it isn‘t
    necessary. And again, this is an opportunity:
    

    View Slide

77. More useful and
    more usable apps
    OFFLINE FIRST OPPORTUNITY
    This is directly linked to the next one:
    

    View Slide

78. My data
    isn‘t with me
    OFFLINE PROBLEMS
    It‘s probably on a server in the US somewhere, if you‘re lucky it‘s on a CDN on
    the same continent as you.
    

    View Slide

79. As a mobile app user, I want
    to have my personal data
    on my device at all times
    USER STORY
    It is my personal data, why should it not be on here? I might need it, and you
    don‘t know when. Plus, in most syncing scenarios, I‘m the authority over my
    data, so just let me keep a local copy. Or rather, your server only gets to back
    up my data.
    Granted, this gets problematic if your app is highly collaborative, but if you
    have a use case where user data is fairly well isolated, you can easily do this.
    So, the potential benefit is:
    

    View Slide

80. Always-accessible
    personal data
    OFFLINE FIRST OPPORTUNITY
    Super fast, always available and simply reliable.
    

    View Slide

81. Apps require
    advance planning
    to be useful
    OFFLINE PROBLEMS
    One more thing that frequently irked people when we talked to them about their
    offline woes: apps not being smart about what data they store locally. In short:
    

    View Slide

82. As a mobile app user, I want
    apps to pre-empt my needs
    in a sensible manner
    USER STORY
    Because I don‘t know in advance when I‘ll be offline. If I knew, it wouldn‘t be a
    problem, because I could prepare for it.
    

    View Slide

83. Apps should ideally
    have recent data in them
    when you open them
    Nowadays you expect your podcast app for example to download new stuff as
    it appears, so it‘s there when you need it, without delay. And this is wonderful
    for offline, too. So think about whether your app could hang on to data, or even
    pre-fetch new data for your users before they need it.
    Similarly:
    

    View Slide

84. Obviously important data
    should probably
    be stored locally
    Dropbox is good in this regard. Fave something on the phone? It‘s now on the
    phone, always. And that‘s a good assumption to make on their part.
    You could take this further though…
    

    View Slide

85. Smart, offline maps
    WOULDN‘T THAT BE NICE
    So sure, you can have offline maps, but you need to know that in advance. Get
    an offline maps app, or get google maps to cache your surroundings.
    But… maps knows where you live, right? Imagine if it would pre-emptively
    download your surroundings as soon as it notices you‘re on wifi in a different
    city than usual.
    

    View Slide

86. Smart, offline maps
    ALSO A BIT SPOOKY
    And on the other hand, this nicely demonstrates some of the issues with
    caching local data: you need new UI elements, plus you need to answer some
    pressing questions: how long will the cached map be retained? Does it get
    flushed at some point? Can I remove it when I don‘t need it anymore? Can I
    cache more than one at a time, and if yes, is there some sort of limit? In short:
    how does it work and can I trust it?
    Or, for you as a developer: can you make it trustworthy while hiding all the
    complexity? Interesting questions, but a cool opportunity:
    

    View Slide

87. Smart apps that
    pre-empt user needs
    OFFLINE FIRST OPPORTUNITY
    But that was already pretty tricky, and brings us to:
    

    View Slide

88. Offline-First
    Challenges
    It‘s becoming obvious that there are a number of new things you could be
    doing, and that these new things require new interfaces and in some parts
    even a new design language, because you have to communicate stuff to
    people you haven‘t had to communicate before
    

    View Slide

89. If your app behaves differently depending on whether it‘s connected or not,
    you‘ll probably have to make this clear somehow.
    Threema does this by displaying an always-visible connectivity stripe at the
    top, red for disconnected, yellow for connecting and green for connected, and
    teaches you beforehand what the consequences of each are.
    Another big issue:
    

    View Slide

90. save vs. sync
    How do we communicate the states data can be in? Stored locally, scheduled
    for sync, synced, possibly out of date, conflicting…
    possibly out of date is a state you‘ll get quite often in offline-capable apps,
    and you‘ll have to decide how crucial the age of data is, and how you
    communicate it. Because old data can still be useful while potentially being
    wrong. Think train schedules. Could be wrong, but communicate intent. And
    that‘s valuable.
    

    View Slide

91. Save
    But you could just as well selectively disable, hide or re-phrase features.
    Imagine a save button that knows whether it‘s connected to the server, and
    changes accordingly:
    

    View Slide

92. Save
    AND SYNC LATER
    Or you could just let users save either way and inform them about the state of
    things after the fact.
    Or, you could be super-secretive…
    

    View Slide

93. …and just handling everything in the background, autosaving constantly and
    auto-syncing whenever you can.
    And if it doesn‘t work…
    

    View Slide

94. Saved locally
    (offline, syncing later)
    …apologise and deal with it.
    There are loads of ways to solve this problem, depending on the use case,
    who you‘re developing for, and how crucial the data is to them.
    But one thing you‘ll probably have to do is
    

    View Slide

95. Informing users
    about sync
    outcomes
    …inform users about sync outcomes. Not just whether they succeeded or not,
    but how and possibly why data has changed in their browser. This is probably
    one of the hardest UI problems related to sync, for example when it comes to
    chronologically sorted data, like chats or other streams/threads.
    

    View Slide

96. Informing users
    about sync
    outcomes
    So, I said that if users only add data, why not always just let them do that and
    sync later? Well, it turns out that things aren‘t quite as simple, and that context
    really matters.
    Here‘s a fairly simple chat example that illustrates some of the things you‘ll
    have to keep in mind:
    

    View Slide

97. 10:00 - Hi
    10:01 - Yo
    A
    B
    10:02 - Meet on Thursday?
    A
    10:03 - Sure, I‘m free.
    B
    10:04 - Ah wait, meant Tuesday.
    A
    Now imagine that B‘s second message was written while one of the two was
    offline. On a train, in a tunnel or something.
    

    View Slide

98. 10:00 - Hi
    10:01 - Yo
    A
    B
    10:02 - Meet on Thursday?
    A
    10:04 - Ah wait, meant Tuesday.
    A
    …
    Then the offline user reconnects and the messages sync up again. A receives
    B‘s missing offline message. So where do you put it?
    You can put it in the chronologically correct place, which makes sense in a
    thread context, because the order carries meaning. But that might mean the
    message appears somewhere out of the user‘s current view, way up there
    somewhere. That‘s a new UI challenge in itself. Or you could do what
    Imessage sometimes seems to do: display it in the flow according to the time it
    arrives at.
    

    View Slide

99. 10:00 - Hi
    10:01 - Yo
    A
    B
    10:02 - Meet on Thursday?
    A
    10:04 - Ah wait, meant Tuesday.
    A
    …
    10:05 - Hey A
    C
    Then the offline user reconnects and the messages sync up again. A receives
    B‘s missing offline message. So where do you put it?
    You can put it in the chronologically correct place, which makes sense in a
    thread context, because the order carries meaning. But that might mean the
    message appears somewhere out of the user‘s current view, way up there
    somewhere. That‘s a new UI challenge in itself. Or you could do what
    Imessage sometimes seems to do: display it in the flow according to the time it
    arrives at.
    

    View Slide

100. 10:01 - Yo
     B
     10:02 - Meet on Thursday?
     A
     10:04 - Ah wait, meant Tuesday.
     A
     …
     10:05 - Hey A
     C
     10:06 - Hi C
     A
     Then the offline user reconnects and the messages sync up again. A receives
     B‘s missing offline message. So where do you put it?
     You can put it in the chronologically correct place, which makes sense in a
     thread context, because the order carries meaning. But that might mean the
     message appears somewhere out of the user‘s current view, way up there
     somewhere. That‘s a new UI challenge in itself. Or you could do what
     Imessage sometimes seems to do: display it in the flow according to the time it
     arrives at.
     

     View Slide

101. 10:02 - Meet on Thursday?
     A
     10:04 - Ah wait, meant Tuesday.
     A
     …
     10:05 - Hey A
     C
     10:06 - Hi C
     A
     10:07 - Hey, you free on Tuesday?
     A
     Then the offline user reconnects and the messages sync up again. A receives
     B‘s missing offline message. So where do you put it?
     You can put it in the chronologically correct place, which makes sense in a
     thread context, because the order carries meaning. But that might mean the
     message appears somewhere out of the user‘s current view, way up there
     somewhere. That‘s a new UI challenge in itself. Or you could do what
     Imessage sometimes seems to do: display it in the flow according to the time it
     arrives at.
     

     View Slide

102. 10:04 - Ah wait, meant Tuesday.
     A
     …
     10:05 - Hey A
     C
     10:06 - Hi C
     A
     10:07 - Hey, you free on Tuesday?
     A
     10:08 - Lemme see…
     C
     Then the offline user reconnects and the messages sync up again. A receives
     B‘s missing offline message. So where do you put it?
     You can put it in the chronologically correct place, which makes sense in a
     thread context, because the order carries meaning. But that might mean the
     message appears somewhere out of the user‘s current view, way up there
     somewhere. That‘s a new UI challenge in itself. Or you could do what
     Imessage sometimes seems to do: display it in the flow according to the time it
     arrives at.
     

     View Slide

103. 10:04 - Ah wait, meant Tuesday.
     A
     10:05 - Hey A
     C
     10:06 - Hi C
     A
     10:07 - Hey, you free on Tuesday?
     A
     10:08 - Lemme see…
     C
     10:09 - I asked B too, btw
     A
     Then the offline user reconnects and the messages sync up again. A receives
     B‘s missing offline message. So where do you put it?
     You can put it in the chronologically correct place, which makes sense in a
     thread context, because the order carries meaning. But that might mean the
     message appears somewhere out of the user‘s current view, way up there
     somewhere. That‘s a new UI challenge in itself. Or you could do what
     Imessage sometimes seems to do: display it in the flow according to the time it
     arrives at.
     

     View Slide

104. 10:00 - Hi
     10:01 - Yo
     A
     B
     10:02 - Meet on Thursday?
     A
     10:03 - Sure, I‘m free.
     B
     10:04 - Ah wait, meant Tuesday.
     A
     This guarantees that the message will actually be seen by A, but this approach
     has the potential to change meaning, because message order is meaningful.
     And this is only text-based, one-dimensional data. A simple chat, one of the
     simplest examples I could think of.
     What to do with deleted items, things that can’t be organised in lists, objects
     that aren’t in themselves immutable? There‘s a lot of potential for complexity
     here, so that‘s something to be aware of.
     

     View Slide

105. 10:00 - Hi
     10:01 - Yo
     A
     B
     10:02 - Meet on Thursday?
     A
     10:03 - Sure, I‘m free.
     B
     10:04 - Ah wait, meant Tuesday.
     A
     This guarantees that the message will actually be seen by A, but this approach
     has the potential to change meaning, because message order is meaningful.
     And this is only text-based, one-dimensional data. A simple chat, one of the
     simplest examples I could think of.
     What to do with deleted items, things that can’t be organised in lists, objects
     that aren’t in themselves immutable? There‘s a lot of potential for complexity
     here, so that‘s something to be aware of.
     

     View Slide

106. Because the web platform is becoming more an more powerful and capable.
     Here‘s a photo editor. It‘s not photoshop, nor is it trying to be: it‘s quick, easy,
     no installation, cross-platform, auto-updates, and does most of the things you
     want in a simple photo editor.
     You‘d probably want it to work offline, though.
     

     View Slide

107. This is an experimental collaborative multitrack recording app, with synths and
     drumkits and audio, and it runs in the browser. This is part of a masters thesis
     by Jan Monschke, and it‘s really bleeding edge, but just imagine where this
     tech will be in a year. Cross-platform garage band? That‘s pretty damn cool.
     But again, it can‘t be used offline, and that‘s a disadvantage in comparison to
     the native apps it competes with.
     But aside from competing with native apps, what are the…
     

     View Slide

108. Offline-First
     Advantages
     advantages of offline first?
     

     View Slide

109. • Performance
     OFFLINE-FIRST ADVANTAGES
     First: Performance. We put stuff in CDNs to move it closer to the user, but the
     closest thing to me right now is this (phone in pocket).
     Put your app in here.
     Put your data in here.
     

     View Slide

110. • Performance
     ZERO LATENCY
     OFFLINE-FIRST ADVANTAGES
     That's zero latency for me. This is the way to the snappiest experience. The
     data may be old, but at least it‘s there. And in many cases, time isn‘t doesn‘t
     even invalidate the data, so there‘s still a benefit to having it.
     

     View Slide

111. • Performance
     ZERO LATENCY
     • Robustness
     OFFLINE-FIRST ADVANTAGES
     Robustness. Offline capability protects from service interruptions. Interestingly,
     we hadn‘t even anticipated this. We have a fairly large service running on
     Hoodie and had to briefly take it down for maintenance, and most people using
     the app at that very moment didn‘t notice.
     Turns out: It doesn‘t matter if your app can‘t reach the server because the user
     is on the subway, or because the server is down. It‘ll still work.
     

     View Slide

112. • Performance
     ZERO LATENCY
     • Robustness
     SERVER DOWN? I DON‘T CARE
     OFFLINE-FIRST ADVANTAGES
     

     View Slide

113. • Performance
     ZERO LATENCY
     • Robustness
     SERVER DOWN? I DON‘T CARE
     • Better experiences
     OFFLINE-FIRST ADVANTAGES
     Better experiences. Apps don‘t lose data. Apps are more trustworthy. Apps are
     more usable and useful. Apps cause less frustration.
     

     View Slide

114. • Performance
     ZERO LATENCY
     • Robustness
     SERVER DOWN? I DON‘T CARE
     • Better experiences
     SAVE ALL THE TIME
     OFFLINE-FIRST ADVANTAGES
     And remember, you‘re saving to a local store first. You can save after every
     single keystroke if you want, and sync to the server every couple of seconds.
     Forget save buttons altogether.
     There‘s a lot to be gained from an offline-first architecture that‘s not completely
     obvious at first glance.
     Anyway:
     

     View Slide

115. The web platform
     is amazing
     This is just the start. Image manipulation, maturing web audio APIs, and there‘s
     a lot more to come. At JSConf EU last year, there was a lot about
     ServiceWorker, client side storage and web crypto: all just coming in to
     existence then, cutting edge, but wildly promising. And now?
     

     View Slide

116. The very next talk today will show you how to use these things to build apps.
     Today. And it‘s only going to get better.
     

     View Slide

117. Any application that can be written in
     JavaScript, will eventually be written in
     JavaScript.
     —Jeff Atwood
     „Atwood‘s Law“
     And if it‘s in the browser it‘s going to be used in a mobile context, and it‘s going
     to be offline at some point.
     And it‘s not just about increased mobility any more:
     

     View Slide

118. At some point recently, the browser
     transformed from being an awesome
     interactive document viewer into being
     the world’s most advanced, widely-
     distributed application runtime.
     —Tom Dale
     Progressive Enhancement is Dead
     The web is very attractive platform for developers for many reasons, and as
     web apps gain complexity and mobile native app usage increases further,
     users will simply expect a mature and solid experience.
     Waiting for more cell towers to be built won‘t help them. Your apps have to
     work regardless of the circumstances.
     Here‘s the conclusion:
     Source: http://tomdale.net/2013/09/progressive-enhancement-is-dead/
     

     View Slide

119. We can’t keep building apps with the
     desktop mindset of permanent, fast
     connectivity, where a temporary
     disconnection or slow service
     is regarded as a problem
     and communicated as an error.
     So, in closing:
     

     View Slide

120. The future is already here — it's just not
     very evenly distributed
     —William Gibson
     Don‘t forget you‘re from the future, and what‘s normal to you probably isn‘t
     normal for the rest of the world
     

     View Slide

121. • The world is imperfect
     • Your app is imperfect
     • Make the best of it
     FORGET PERFECTION
     Learn to let go a bit more. Find a sensible middle ground between certainty
     and assumption.
     

     View Slide

122. Dive in
     Last year, I finished a prototype web app that works completely offline, it has
     offline maps, and it can even take photos while offline and store them until
     you‘re back online. It runs well on my dodgy, nearly 4 year old android phone.
     It‘s only going to get easier, faster and more stable from here on out.
     

     View Slide

123. Raise awareness
     that this is a thing. Among developers, yes, but also among users: People need
     to get used to their browsers and websites working offline.
     And finally:
     

     View Slide

124. Please join in!
     

     View Slide

125. We started offlinefirst.org to start bringing people together on this. We‘ve got a
     pretty active slack channel and there‘s a newsletter I can really recommend. In
     any case, come join us!
     So finally:
     

     View Slide

126. be proud of that dependable boxy thing!
     And also:
     

     View Slide

127. Don‘t panic,
     use Hoodie
     Don‘t panic, use hoodie. And come talk to me if you want to know more about
     offline first or hoodie.
     

     View Slide

128. Thanks!
     Please come see me for info and stickers!
     @espylaub
     Slides: tinyurl.com/offline-first-coldfront
     And please check out http://hood.ie
     ❤
     I‘ve also got really sweet stickers :)
     Thank you and enjoy the rest of the day!
     

     View Slide