Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a Modern Memex

Andrew Louis
September 06, 2018

Building a Modern Memex

The Memex was proposed in 1945 as the ultimate organizational tool. The desk-sized device would store a user’s personal library and allow for information to be searched, organized, connected together with hyperlinks, and shared.

Without a device like this, its creator suggested, our species would drown in information overload and come to a premature end.

Tragically, zero of these devices were ever produced, but 72 years later, I've built a Memex in JavaScript. I've built importers for dozens of different sources of personal data and data consumptions and put it all behind a graph-based API.

This presentations goes over the history of the Memex and demoes my own version by showing what type of queries, organizing tools, and visualization are available.

Andrew Louis

September 06, 2018
Tweet

Other Decks in Technology

Transcript

  1. Building a modern
    Memex
    Andrew Louis (@hyfen)
    https://hyfen.net/memex

    View full-size slide

  2. History in tech:
    var foo = 'bar';
    let foo = 'bar';

    View full-size slide

  3. History (ancient):
    $(document).ready(function(){
    $('#header').addClass('foo')
    .css('color', 'red')
    .fadeIn('slow');

    View full-size slide

  4. History (pre-):

    Click Me

    View full-size slide

  5. History!
    (The kind with lots of black and
    white photos)

    View full-size slide

  6. Vannevar Bush

    View full-size slide

  7. “We are being buried under
    our own product.”

    View full-size slide

  8. “As We May Think”
    Essay published July 1945

    View full-size slide

  9. “Consider a future device for individual
    use, which is a sort of mechanized
    private file and library. It needs a name,
    and, to coin one at random, "memex"
    will do.

    View full-size slide

  10. “A memex is a device in which an
    individual stores all their books, records,
    and communications, and which is
    mechanized so that it may be consulted
    with exceeding speed and flexibility.”

    View full-size slide

  11. “The human brain files by
    association. The Memex could
    do this mechanically”

    View full-size slide

  12. Number of memexes
    produced:
    0

    View full-size slide

  13. I am an information
    pack rat.
    A little about me:

    View full-size slide

  14. Overwhelming amounts of
    personal data to save! ☹

    View full-size slide

  15. Talk to a therapist
    Talk to a therapist
    The solution:
    Build a Memex

    View full-size slide

  16. Building a Memex

    View full-size slide

  17. Gathering data:
    • Reading and browsing from browser history,
    ebook reader, twitter, RSS reader
    • Digital consumptions like photos viewed, videos
    watched, music/podcasts listened to
    • Location history from phone
    • Messaging and social interactions
    • Journalling, personal photography, notes

    View full-size slide

  18. Data details:
    • data from 1995-present organized as a
    graph
    • 6M activities (things I’ve done)
    • 5M entities (people, places, thoughts,
    messages, etc)
    • 17M relationships linking everything
    together

    View full-size slide

  19. Live demo time!
    "

    View full-size slide

  20. The nitty gritty

    View full-size slide

  21. Fancy graph database
    • Postgres.
    • That’s it!
    • one Nodes table; one Relationships table
    • database handles graph traversals,
    fulltext search, geospatial queries, JSON
    data, and more.

    View full-size slide

  22. Querying
    verb:listened
    provider:spotify
    involved:(
    represents:song
    creators:(
    “Aretha Franklin”
    )
    )

    View full-size slide

  23. API
    • a single endpoint for querying
    • a single endpoint for importing
    data (write-only access with de-
    duping system)

    View full-size slide

  24. Desktop app
    Importers
    Interface
    API

    View full-size slide

  25. Where’s this project going?

    View full-size slide

  26. my Memex as a weird
    art project

    View full-size slide

  27. before i regret it
    u don't know how many regret i have about school.
    dunno. might regret it
    do you regret how you used your year
    i think i'll really regret the chinstrap later in
    no regret
    usually i regret it
    cancelled. with some regret i have to warn you tha
    day. not that there's regrets about how things went
    quick before you get regrets
    maybe she even regrets it in some way but ag
    t i just hope i won't regret it later
    why regret?
    your two feet. won't regret it.
    respect. i sincerely regret having stressed her ou

    View full-size slide

  28. my Memex as long-
    running experiment

    View full-size slide

  29. And open source soon!
    You should sign up for my newsletter:
    https://hyfen.net/memex

    View full-size slide

  30. Now back to the story
    of the first Memex

    View full-size slide

  31. The Memex was weird!

    View full-size slide

  32. The Memex was weird
    for its form factor

    View full-size slide

  33. This was a computer from
    the 1930s:

    View full-size slide

  34. http://trevor.smith.name/memex/

    View full-size slide

  35. The Memex was weird
    for the problem it solved

    View full-size slide

  36. The Memex was weird
    for the technologies it used (or didn’t
    use)

    View full-size slide

  37. Extrasensory perception, crystals,
    and mind-control. Oh my!

    View full-size slide

  38. The Memex was weird
    for how it was published

    View full-size slide

  39. The Memex was weird
    for its form factor
    for the problem it solved
    for the technologies it used
    for how it was published

    View full-size slide

  40. The Memex was weird
    because for the creator, Vannevar
    Bush, this was just one of many side
    projects

    View full-size slide

  41. Memex as an “image of
    potentiality”
    — Linda Smith (1981)

    View full-size slide

  42. Side projects
    Make them weird! Make them
    experimental!

    View full-size slide

  43. Using my own
    weird side project

    View full-size slide

  44. Questions?
    I have a newsletter! Subscribe if you want to get
    project updates or be notified when a beta version
    becomes available: https://hyfen.net/memex
    Andrew Louis (@hyfen)

    View full-size slide