Human-Computer Collaboration at NYPL Labs

This talk is part of the Culture Analytics and User Experience Design Workshop organized by the UCLA's Institute for Pure & Applied Mathematics, April 11 - 15, 2016.

Abstract:
This talk will provide an insight into the Building Inspector, Oral History, and other projects developed at New York Public Library Labs, with an emphasis on design and HCI-related challenges and iteration processes. It will cover the rationale as to when NYPL Labs uses (or does not use) computational tools for data extraction and improvement, and how the library is learning to collaborate with its users on the creation of digital resources.

Transcript

1. View Slide

2. human-computer collaboration
   mauricio giraldo arteaga
   @mgiraldo
   @nypl_labs
   IPAM Culture Analytics and User Experience Design, April 2016
   

   View Slide

3. hello
   

   View Slide

4. not a real library scientist
   

   View Slide

5. flickr.com/photos/wallyg/6133216510
   

   View Slide

6. View Slide

7. View Slide

8. Eric Shows
   

   View Slide

9. NYPL Labs
   

   View Slide

10. access
    digitization metadata
    public
    traditional digital library program
    

    View Slide

11. access
    digitization metadata
    public
    engagement
    r+d
    

    View Slide

12. View Slide

13. View Slide

14. what happens after digitization?
    

    View Slide

15. human-computer collaboration
    

    View Slide

16. ¿ ?
    

    View Slide

17. embrace imperfection
    corollary of “perfect is the enemy of good”
    

    View Slide

18. « A designer’s definition of
    ‘perfect’ is different for
    computational designers. »
    because it is not achievable
    John Maeda
    

    View Slide

19. human-computer collaboration
    

    View Slide

20. computers are good at some things…
    

    View Slide

21. Randall Munroe - xkcd.com/1140
    

    View Slide

22. David Hagen - drhagen.com/blog/the-missing-11th-of-the-month
    

    View Slide

23. people overestimate OCR quality
    

    View Slide

24. OCR result
    

    View Slide

25. okay… so maybe computers are not that good
    

    View Slide

26. people are good at other things
    

    View Slide

27. human-computer collaboration
    i avoid the term “crowdsourcing”
    

    View Slide

28. two examples
    

    View Slide

29. View Slide

30. View Slide

31. View Slide

32. View Slide

33. footprint
    material
    use type
    street names
    address
    floors
    name
    class
    geo location
    year
    skylights
    backyards
    

    View Slide

34. like Google Maps for the 19th century
    but Google Maps cannot answer questions about the 19th century
    

    View Slide

35. View Slide

36. View Slide

37. View Slide

38. *this is a simulation. actual process is intensive. consult your mathematician before trying
    

    View Slide

39. View Slide

40. View Slide

41. and now you start tracing those buildings by hand
    (˽°□°)˽Ɨ ˍʓʓˍ
    

    View Slide

42. View Slide

43. 1852-1854
    

    View Slide

44. 1852-1854
    

    View Slide

45. can we automate this?
    

    View Slide

46. computers are good at some things…
    

    View Slide

47. View Slide

48. View Slide

49. View Slide

50. View Slide

51. View Slide

52. View Slide

53. View Slide

54. yay footprints!
    60,000+ of those!
    

    View Slide

55. like OCR for maps!™
    (not really trademarked)
    

    View Slide

56. but OCR is pretty bad
    ಠ_ಠ
    

    View Slide

57. View Slide

58. people are good at other things!
    

    View Slide

59. View Slide

60. View Slide

61. people don’t choose to complete these
    

    View Slide

62. we have over 60,000 footprints to check!
    will people want to do this?
    

    View Slide

63. what is the minimum contribution we need?
    we want the lowest friction possible so people will want to contribute
    

    View Slide

64. 
    this was 2013, touch-screen mobile had taken off
    

    View Slide

65. View Slide

66. what about malicious users?
    or even well-meaning ones who make mistakes
    

    View Slide

67. View Slide

68. 75% or more agreement between 3 or more people
    arbitrary numbers that have worked for us
    

    View Slide

69. View Slide

70. YES is on the right side because most people are right-handed and the algorithm is right most of the time
    

    View Slide

71. Building Inspector
    buildinginspector.nypl.org
    

    View Slide

72. will people participate?
    remember that little tweet button?
    

    View Slide

73. View Slide

74. View Slide

75. View Slide

76. footprint
    material
    use type
    street names
    address
    floors
    name
    class
    geo location
    year
    skylights
    backyards
    

    View Slide

77. View Slide

78. View Slide

79. check
    YES FIX
    address color fix
    *footprints marked as “NO” go to polygon heaven
    

    View Slide

80. address
    had to use full keyboard on mobile because fractions
    

    View Slide

81. classify
    

    View Slide

82. fix
    

    View Slide

83. place names
    

    View Slide

84. View Slide

85. we add new maps as old ones are completed
    the bottleneck now became geo-rectifying those maps ¯\_(ϑ)_/¯
    

    View Slide

86. this is actually version 2
    

    View Slide

87. (the magic of git)
    

    View Slide

88. View Slide

89. View Slide

90. View Slide

91. good tutorials are hard
    

    View Slide

92. View Slide

93. Super Mario Bros. (Nintendo, 1985)
    

    View Slide

94. we have too many edge cases
    or: how i learned to stop worrying and embrace imperfection
    

    View Slide

95. View Slide

96. ¯\_(ϑ)_/¯
    people skip them anyway
    

    View Slide

97. View Slide

98. coming soon
    

    View Slide

99. View Slide

100. View Slide

101. View Slide

102. View Slide

103. NYPL Community Oral History Project
     oralhistory.nypl.org
     

     View Slide

104. View Slide

105. View Slide

106. make these stories more accessible
     

     View Slide

107. View Slide

108. mark
     transcribe
     

     View Slide

109. by brian foo @beefoo
     

     View Slide

110. allows for basic text search
     but it’s not a proper transcript
     

     View Slide

111. View Slide

112. View Slide

113. we felt we needed something different
     

     View Slide

114. View Slide

115. computers are good at some things…
     

     View Slide

116. like OCR for audio!™
     (not sure if they trademarked that)
     

     View Slide

117. we get transcription “snippets”
     from 1 to about 6 seconds long in varying levels of quality
     

     View Slide

118. View Slide

119. people are good at other things…
     

     View Slide

120. View Slide

121. View Slide

122. by brian foo @beefoo
     

     View Slide

123. we conducted a few usability studies
     

     View Slide

124. by brian foo @beefoo
     

     View Slide

125. View Slide

126. View Slide

127. it’s hard to reach consensus
     ಠ_ಠ
     

     View Slide

128. embrace imperfection
     

     View Slide

129. transcribe.oralhistory.nypl.org
     

     View Slide

130. transcribe.oralhistory.nypl.org
     

     View Slide

131. made with customizability in mind
     

     View Slide

132. storyscribe.themoth.org
     

     View Slide

133. this is one week after launch
     

     View Slide

134. it is still being improved
     

     View Slide

135. two of several projects we’ve worked on so far
     

     View Slide

136. of human-computer collaboration
     

     View Slide

137. it’s a collaborative process
     Willa Armstrong, Shawn Averkamp, Paul Beaudoin, Brian Foo, Josh
     Hadro, Elizabeth Hummer, Ara Kim, Shana Kimball, Tom Listanti,
     Matthew Miller, Eric Shows, Bert Spaan, and more at NYPL…
     

     View Slide

138. one more thing…
     

     View Slide

139. View Slide

140. View Slide

141. View Slide

142. View Slide

143. lala.cursivebuildings.com
     

     View Slide

144. how to decode the 3D data?
     in the browser
     

     View Slide

145. View Slide

146. View Slide

147. View Slide

148. View Slide

149. View Slide

150. stereo.nypl.org
     

     View Slide

151. View Slide

152. View Slide

153. View Slide

154. View Slide

155. View Slide

156. Boston Public Library Boston Public Library U.S. Geological Survey
     U.S. Geological Survey
     

     View Slide

157. thank you!
     mauricio giraldo arteaga
     @mgiraldo
     @nypl_labs
     IPAM Culture Analytics and User Experience Design, April 2016
     

     View Slide