Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a Modern Memex

Andrew Louis
September 06, 2018

Building a Modern Memex

The Memex was proposed in 1945 as the ultimate organizational tool. The desk-sized device would store a user’s personal library and allow for information to be searched, organized, connected together with hyperlinks, and shared.

Without a device like this, its creator suggested, our species would drown in information overload and come to a premature end.

Tragically, zero of these devices were ever produced, but 72 years later, I've built a Memex in JavaScript. I've built importers for dozens of different sources of personal data and data consumptions and put it all behind a graph-based API.

This presentations goes over the history of the Memex and demoes my own version by showing what type of queries, organizing tools, and visualization are available.

Andrew Louis

September 06, 2018

Other Decks in Technology


  1. Building a modern Memex Andrew Louis (@hyfen) https://hyfen.net/memex

  2. History

  3. History in tech: var foo = 'bar'; let foo =

  4. History (ancient): $(document).ready(function(){ $('#header').addClass('foo') .css('color', 'red') .fadeIn('slow');

  5. History (pre-): <button onclick=“myFunction()"> Click Me </button>

  6. History! (The kind with lots of black and white photos)

  7. 1930s

  8. Vannevar Bush

  9. None
  10. None
  11. None
  12. 1940s

  13. None
  14. None
  15. None
  16. None
  17. None
  18. None
  19. “We are being buried under our own product.”

  20. “As We May Think” Essay published July 1945

  21. “Consider a future device for individual use, which is a

    sort of mechanized private file and library. It needs a name, and, to coin one at random, "memex" will do.
  22. The Memex

  23. “A memex is a device in which an individual stores

    all their books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility.”
  24. None
  25. None
  26. None
  27. None
  28. “The human brain files by association. The Memex could do

    this mechanically”
  29. None
  30. Number of memexes produced: 0

  31. I am an information pack rat. A little about me:

  32. None
  33. None
  34. None
  35. None
  36. None
  37. None
  38. Overwhelming amounts of personal data to save! ☹

  39. Talk to a therapist Talk to a therapist The solution:

    Build a Memex
  40. Building a Memex

  41. Gathering data: • Reading and browsing from browser history, ebook

    reader, twitter, RSS reader • Digital consumptions like photos viewed, videos watched, music/podcasts listened to • Location history from phone • Messaging and social interactions • Journalling, personal photography, notes
  42. Data details: • data from 1995-present organized as a graph

    • 6M activities (things I’ve done) • 5M entities (people, places, thoughts, messages, etc) • 17M relationships linking everything together
  43. Live demo time! "

  44. </demo>

  45. The nitty gritty

  46. Fancy graph database • Postgres. • That’s it! • one

    Nodes table; one Relationships table • database handles graph traversals, fulltext search, geospatial queries, JSON data, and more.
  47. Querying verb:listened provider:spotify involved:( represents:song creators:( “Aretha Franklin” ) )

  48. API • a single endpoint for querying • a single

    endpoint for importing data (write-only access with de- duping system)
  49. Desktop app Importers Interface API

  50. Why ?

  51. None
  52. None
  53. None
  54. None
  55. Where’s this project going?

  56. None
  57. None
  58. None
  59. None
  60. my Memex as a weird art project

  61. before i regret it u don't know how many regret

    i have about school. dunno. might regret it do you regret how you used your year i think i'll really regret the chinstrap later in no regret usually i regret it cancelled. with some regret i have to warn you tha day. not that there's regrets about how things went quick before you get regrets maybe she even regrets it in some way but ag t i just hope i won't regret it later why regret? your two feet. won't regret it. respect. i sincerely regret having stressed her ou
  62. None
  63. None
  64. my Memex as long- running experiment

  65. None
  66. None
  67. None
  68. And open source soon! You should sign up for my

    newsletter: https://hyfen.net/memex
  69. Now back to the story of the first Memex

  70. The Memex was weird!

  71. None
  72. The Memex was weird for its form factor

  73. This was a computer from the 1930s:

  74. None
  75. None
  76. None
  77. None
  78. None
  79. http://trevor.smith.name/memex/

  80. The Memex was weird for the problem it solved

  81. None
  82. None
  83. None
  84. The Memex was weird for the technologies it used (or

    didn’t use)
  85. None
  86. None
  87. None
  88. Extrasensory perception, crystals, and mind-control. Oh my!

  89. The Memex was weird for how it was published

  90. None
  91. None
  92. None
  93. None
  94. None
  95. None
  96. None
  97. None
  98. The Memex was weird for its form factor for the

    problem it solved for the technologies it used for how it was published
  99. The Memex was weird because for the creator, Vannevar Bush,

    this was just one of many side projects
  100. None
  101. None
  102. None
  103. None
  104. Memex as an “image of potentiality” — Linda Smith (1981)

  105. None
  106. None
  107. None
  108. None
  109. None
  110. None
  111. None
  112. Side projects Make them weird! Make them experimental!

  113. Using my own weird side project

  114. None
  115. None
  116. None
  117. None
  118. Thanks!

  119. Questions? I have a newsletter! Subscribe if you want to

    get project updates or be notified when a beta version becomes available: https://hyfen.net/memex Andrew Louis (@hyfen)