Designing Prosthetic Memory: Audio or Transcript, That is the Question

Presentation given at AVI 2018, International Working Conference on Advanced Visual Interfaces, Grosseto, Italy.

ABSTRACT: Audio recordings and the corresponding transcripts are often used as prosthetic memory (PM) after meetings and lectures. While current research is mainly developing novel features for prosthetic memory, less is known on how and why audio recordings and transcripts are used. We investigate how users interact with audio and transcripts as prosthetic memory, whether interaction strategies change over time, and analyse potential differences in accuracy and efficiency. In contrast to the subjective user perception, our results show that audio recordings and transcripts are equally efficient, but that transcripts are generally preferred due to their easily accessible contextual information. We further identified that prosthetic memory is not only used as a recall aid but frequently also consulted for verifying information that has been recalled from organic memory (OM). Our findings are summarised in a number of design implications for prosthetic memory solutions.

Research paper: https://beatsigner.com/publications/designing-prosthetic-memory-audio-or-transcript-that-is-the-question.pdf

Transcript

1. 2 December 2005 Designing Prosthetic Memory: Audio or Transcript, That is the Question Sandra Trullemans, Payam Ebrahimi and Beat Signer Web & Information Systems Engineering Lab Department of Computer Science Vrije Universiteit Brussel WEB & INFORMATION SYSTEMS ENGINEERING
2. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 2 May 30, 2018
3. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 3 May 30, 2018 Cross-Indexed Prosthetic Memory ChittyChatty Kalnikaité et al. 2007 Ferret browser Wellner et al. 2004 Use Digital Prosthetic Memory a Kalnikaitė eld University , Sheffield, S1 4DP, UK. e@sheffield.ac.uk Steve Whittaker Sheffield University 211 Portobello St, Sheffield, S1 4DP, UK. s.whittaker@sheffield.ac.uk orable and important moments, as f information. The last few years on of digital devices intended to ory (PM), to help users recall ons and retrieve personal theless have little systematic nd why people might use such heir own organic memory (OM). , it may be more efficient than om a complex PM device. We udy which investigates when and d OM. We found that PM use tion of the quality of their OM, as rties. In particular, we found that and Efficiency, preferring rapid curate information over laborious ation. We discuss the implications e PM design and theory. Rather e PM designs need to focus on work in synergy. mory, Digital Memory, Speech al, Notes, Remembering. words ces and presentation (e.g., HCI): In his oft-cited vision of the future, Vannevar Bush [2] proposed Memex, a tool designed to help users remember and index information they have previously encountered. In the last 15 years this vision has become a reality. Reductions in the cost of digital storage and the emergence of sophisticated recording technologies have led to the development of many different PM systems. One type of Fig 1. ChittyChatty Graphical User Interface Livescribe 3
4. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 4 May 30, 2018 RQ1: Is there a difference in the way users interact with notes, transcripts and audio recordings and do the interaction strategies change when time elapses?
5. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 5 May 30, 2018 RQ2: Is there a difference in accuracy and efficiency between audio and transcript PM?
6. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 6 May 30, 2018 RQ3: Do factors such as confidence influence the way users interact with PM?
7. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 7 May 30, 2018 Case 1 - Audio Only
8. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 8 May 30, 2018 Case 2 - Transcript Only
9. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 9 May 30, 2018 Procedure
10. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 10 May 30, 2018 Procedure - Phases Phase 1 Phase 2 Open Interview
11. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 11 May 30, 2018 RQ1: Is there a difference in the way users interact with notes, transcripts and audio recordings and do the interaction strategies change when time elapses?
12. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 12 May 30, 2018 Overall Usage 0 20 40 60 80 100 0 7 30 Likelihood of Usage (%) Time Interval (days) OM Notes Audio 0 20 40 60 80 100 0 7 30 Likelihood of Usage (%) Time Interval (days) OM Notes Transcript Audio Only Transcript Only
13. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 13 May 30, 2018 Overall Usage 0 20 40 60 80 100 0 7 30 Likelihood of Usage (%) Time Interval (days) OM Notes Audio Transcript Audio and Transcript Combined
14. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 14 May 30, 2018 0 20 40 60 80 100 0 7 30 0 7 30 Time Interval (days) Time Interval (days) Likelihood of Usage (%) Transcript Navigation Skimming Audio Different Ways of Interacting with PM Audio Only Transcript Only
15. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 15 May 30, 2018 Different Purposes of PM Use 0 20 40 60 80 100 0 7 30 0 7 30 Time Interval (days) Time Interval (days) Likelihood of Usage (%) Transcript Recall Verifying Audio Audio Only Transcript Only
16. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 16 May 30, 2018 RQ2: Is there a difference in accuracy and efficiency between audio and transcript PM?
17. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 17 May 30, 2018 Accuracy ▪ No significant difference between audio and transcript in accuracy ▪ Accuracy does not change over time intervals ▪ Choice of interaction has no impact on accuracy
18. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 18 May 30, 2018 Efficiency 0 20 40 60 80 100 120 140 0 7 30 Efficiency (s) Time Interval (days) Transcript Navigating Transcript Skimming Audio Navigating Audio Skimming
19. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 19 May 30, 2018 RQ3: Do factors such as confidence influence the way users interact with PM?
20. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 20 May 30, 2018 Confidence Influence on Interactions
21. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 21 May 30, 2018 Design Implications Co-indexing PMs Efficiency and Accuracy Recall and Verification
22. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 22 May 30, 2018 Conclusion ▪ Investigated the overall usage of audio and transcript PM ▪ Audio PM is not less efficient than transcript PM ▪ Three design implications ▪ Designing for individual or combined audio and transcript ▪ PM is not only chosen based on efficiency and accuracy ▪ Designing for recall and verification