Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Human-Computer Collaboration at NYPL Labs

Human-Computer Collaboration at NYPL Labs

This talk is part of the Culture Analytics and User Experience Design Workshop organized by the UCLA's Institute for Pure & Applied Mathematics, April 11 - 15, 2016.

Abstract:
This talk will provide an insight into the Building Inspector, Oral History, and other projects developed at New York Public Library Labs, with an emphasis on design and HCI-related challenges and iteration processes. It will cover the rationale as to when NYPL Labs uses (or does not use) computational tools for data extraction and improvement, and how the library is learning to collaborate with its users on the creation of digital resources.

Mauricio Giraldo

April 14, 2016
Tweet

More Decks by Mauricio Giraldo

Other Decks in Design

Transcript

  1. View Slide

  2. human-computer collaboration
    mauricio giraldo arteaga
    @mgiraldo
    @nypl_labs
    IPAM Culture Analytics and User Experience Design, April 2016

    View Slide

  3. hello

    View Slide

  4. not a real library scientist

    View Slide

  5. flickr.com/photos/wallyg/6133216510

    View Slide

  6. View Slide

  7. View Slide

  8. Eric Shows

    View Slide

  9. NYPL Labs

    View Slide

  10. access
    digitization metadata
    public
    traditional digital library program

    View Slide

  11. access
    digitization metadata
    public
    engagement
    r+d

    View Slide

  12. View Slide

  13. View Slide

  14. what happens after digitization?

    View Slide

  15. human-computer collaboration

    View Slide

  16. ¿ ?

    View Slide

  17. embrace imperfection
    corollary of “perfect is the enemy of good”

    View Slide

  18. « A designer’s definition of
    ‘perfect’ is different for
    computational designers. »
    because it is not achievable
    John Maeda

    View Slide

  19. human-computer collaboration

    View Slide

  20. computers are good at some things…

    View Slide

  21. Randall Munroe - xkcd.com/1140

    View Slide

  22. David Hagen - drhagen.com/blog/the-missing-11th-of-the-month

    View Slide

  23. people overestimate OCR quality

    View Slide

  24. OCR result

    View Slide

  25. okay… so maybe computers are not that good

    View Slide

  26. people are good at other things

    View Slide

  27. human-computer collaboration
    i avoid the term “crowdsourcing”

    View Slide

  28. two examples

    View Slide

  29. View Slide

  30. View Slide

  31. View Slide

  32. View Slide

  33. footprint
    material
    use type
    street names
    address
    floors
    name
    class
    geo location
    year
    skylights
    backyards

    View Slide

  34. like Google Maps for the 19th century
    but Google Maps cannot answer questions about the 19th century

    View Slide

  35. View Slide

  36. View Slide

  37. View Slide

  38. *this is a simulation. actual process is intensive. consult your mathematician before trying

    View Slide

  39. View Slide

  40. View Slide

  41. and now you start tracing those buildings by hand
    (˽°□°)˽Ɨ ˍʓʓˍ

    View Slide

  42. View Slide

  43. 1852-1854

    View Slide

  44. 1852-1854

    View Slide

  45. can we automate this?

    View Slide

  46. computers are good at some things…

    View Slide

  47. View Slide

  48. View Slide

  49. View Slide

  50. View Slide

  51. View Slide

  52. View Slide

  53. View Slide

  54. yay footprints!
    60,000+ of those!

    View Slide

  55. like OCR for maps!™
    (not really trademarked)

    View Slide

  56. but OCR is pretty bad
    ಠ_ಠ

    View Slide

  57. View Slide

  58. people are good at other things!

    View Slide

  59. View Slide

  60. View Slide

  61. people don’t choose to complete these

    View Slide

  62. we have over 60,000 footprints to check!
    will people want to do this?

    View Slide

  63. what is the minimum contribution we need?
    we want the lowest friction possible so people will want to contribute

    View Slide


  64. this was 2013, touch-screen mobile had taken off

    View Slide

  65. View Slide

  66. what about malicious users?
    or even well-meaning ones who make mistakes

    View Slide

  67. View Slide

  68. 75% or more agreement between 3 or more people
    arbitrary numbers that have worked for us

    View Slide

  69. View Slide

  70. YES is on the right side because most people are right-handed and the algorithm is right most of the time

    View Slide

  71. Building Inspector
    buildinginspector.nypl.org

    View Slide

  72. will people participate?
    remember that little tweet button?

    View Slide

  73. View Slide

  74. View Slide

  75. View Slide

  76. footprint
    material
    use type
    street names
    address
    floors
    name
    class
    geo location
    year
    skylights
    backyards

    View Slide

  77. View Slide

  78. View Slide

  79. check
    YES FIX
    address color fix
    *footprints marked as “NO” go to polygon heaven

    View Slide

  80. address
    had to use full keyboard on mobile because fractions

    View Slide

  81. classify

    View Slide

  82. fix

    View Slide

  83. place names

    View Slide

  84. View Slide

  85. we add new maps as old ones are completed
    the bottleneck now became geo-rectifying those maps ¯\_(ϑ)_/¯

    View Slide

  86. this is actually version 2

    View Slide

  87. (the magic of git)

    View Slide

  88. View Slide

  89. View Slide

  90. View Slide

  91. good tutorials are hard

    View Slide

  92. View Slide

  93. Super Mario Bros. (Nintendo, 1985)

    View Slide

  94. we have too many edge cases
    or: how i learned to stop worrying and embrace imperfection

    View Slide

  95. View Slide

  96. ¯\_(ϑ)_/¯
    people skip them anyway

    View Slide

  97. View Slide

  98. coming soon

    View Slide

  99. View Slide

  100. View Slide

  101. View Slide

  102. View Slide

  103. NYPL Community Oral History Project
    oralhistory.nypl.org

    View Slide

  104. View Slide

  105. View Slide

  106. make these stories more accessible

    View Slide

  107. View Slide

  108. mark
    transcribe

    View Slide

  109. by brian foo @beefoo

    View Slide

  110. allows for basic text search
    but it’s not a proper transcript

    View Slide

  111. View Slide

  112. View Slide

  113. we felt we needed something different

    View Slide

  114. View Slide

  115. computers are good at some things…

    View Slide

  116. like OCR for audio!™
    (not sure if they trademarked that)

    View Slide

  117. we get transcription “snippets”
    from 1 to about 6 seconds long in varying levels of quality

    View Slide

  118. View Slide

  119. people are good at other things…

    View Slide

  120. View Slide

  121. View Slide

  122. by brian foo @beefoo

    View Slide

  123. we conducted a few usability studies

    View Slide

  124. by brian foo @beefoo

    View Slide

  125. View Slide

  126. View Slide

  127. it’s hard to reach consensus
    ಠ_ಠ

    View Slide

  128. embrace imperfection

    View Slide

  129. transcribe.oralhistory.nypl.org

    View Slide

  130. transcribe.oralhistory.nypl.org

    View Slide

  131. made with customizability in mind

    View Slide

  132. storyscribe.themoth.org

    View Slide

  133. this is one week after launch

    View Slide

  134. it is still being improved

    View Slide

  135. two of several projects we’ve worked on so far

    View Slide

  136. of human-computer collaboration

    View Slide

  137. it’s a collaborative process
    Willa Armstrong, Shawn Averkamp, Paul Beaudoin, Brian Foo, Josh
    Hadro, Elizabeth Hummer, Ara Kim, Shana Kimball, Tom Listanti,
    Matthew Miller, Eric Shows, Bert Spaan, and more at NYPL…

    View Slide

  138. one more thing…

    View Slide

  139. View Slide

  140. View Slide

  141. View Slide

  142. View Slide

  143. lala.cursivebuildings.com

    View Slide

  144. how to decode the 3D data?
    in the browser

    View Slide

  145. View Slide

  146. View Slide

  147. View Slide

  148. View Slide

  149. View Slide

  150. stereo.nypl.org

    View Slide

  151. View Slide

  152. View Slide

  153. View Slide

  154. View Slide

  155. View Slide

  156. Boston Public Library Boston Public Library U.S. Geological Survey
    U.S. Geological Survey

    View Slide

  157. thank you!
    mauricio giraldo arteaga
    @mgiraldo
    @nypl_labs
    IPAM Culture Analytics and User Experience Design, April 2016

    View Slide