Designing Prosthetic Memory: Audio or Transcript, That is the Question

Presentation given at AVI 2018, International Working Conference on Advanced Visual Interfaces, Grosseto, Italy.

ABSTRACT: Audio recordings and the corresponding transcripts are often used as prosthetic memory (PM) after meetings and lectures. While current research is mainly developing novel features for prosthetic memory, less is known on how and why audio recordings and transcripts are used. We investigate how users interact with audio and transcripts as prosthetic memory, whether interaction strategies change over time, and analyse potential differences in accuracy and efficiency. In contrast to the subjective user perception, our results show that audio recordings and transcripts are equally efficient, but that transcripts are generally preferred due to their easily accessible contextual information. We further identified that prosthetic memory is not only used as a recall aid but frequently also consulted for verifying information that has been recalled from organic memory (OM). Our findings are summarised in a number of design implications for prosthetic memory solutions.

Research paper: https://beatsigner.com/publications/designing-prosthetic-memory-audio-or-transcript-that-is-the-question.pdf

Transcript

1. 2 December 2005
   Designing Prosthetic Memory:
   Audio or Transcript, That is the Question
   Sandra Trullemans, Payam Ebrahimi and Beat Signer
   Web & Information Systems Engineering Lab
   Department of Computer Science
   Vrije Universiteit Brussel
   WEB & INFORMATION
   SYSTEMS ENGINEERING
   

   View Slide

2. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 2
   May 30, 2018
   

   View Slide

3. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 3
   May 30, 2018
   Cross-Indexed Prosthetic Memory
   ChittyChatty
   Kalnikaité et al. 2007
   Ferret browser
   Wellner et al. 2004
   Use Digital Prosthetic Memory
   a Kalnikaitė
   eld University
   , Sheffield, S1 4DP, UK.
   e@sheffield.ac.uk
   Steve Whittaker
   Sheffield University
   211 Portobello St, Sheffield, S1 4DP, UK.
   s.whittaker@sheffield.ac.uk
   orable and important moments, as
   f information. The last few years
   on of digital devices intended to
   ory (PM), to help users recall
   ons and retrieve personal
   theless have little systematic
   nd why people might use such
   heir own organic memory (OM).
   , it may be more efficient than
   om a complex PM device. We
   udy which investigates when and
   d OM. We found that PM use
   tion of the quality of their OM, as
   rties. In particular, we found that
   and Efficiency, preferring rapid
   curate information over laborious
   ation. We discuss the implications
   e PM design and theory. Rather
   e PM designs need to focus on
   work in synergy.
   mory, Digital Memory, Speech
   al, Notes, Remembering.
   words
   ces and presentation (e.g., HCI):
   In his oft-cited vision of the future, Vannevar Bush [2]
   proposed Memex, a tool designed to help users remember
   and index information they have previously encountered. In
   the last 15 years this vision has become a reality.
   Reductions in the cost of digital storage and the emergence
   of sophisticated recording technologies have led to the
   development of many different PM systems. One type of
   Fig 1. ChittyChatty Graphical User Interface
   Livescribe 3
   

   View Slide

4. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 4
   May 30, 2018
   RQ1: Is there a difference in the way
   users interact with notes, transcripts and
   audio recordings and do the interaction
   strategies change when time elapses?
   

   View Slide

5. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 5
   May 30, 2018
   RQ2: Is there a difference in accuracy and
   efficiency between audio and transcript PM?
   

   View Slide

6. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 6
   May 30, 2018
   RQ3: Do factors such as confidence
   influence the way users interact with PM?
   

   View Slide

7. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 7
   May 30, 2018
   Case 1 - Audio Only
   

   View Slide

8. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 8
   May 30, 2018
   Case 2 - Transcript Only
   

   View Slide

9. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 9
   May 30, 2018
   Procedure
   

   View Slide

10. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 10
    May 30, 2018
    Procedure - Phases
    Phase 1
    Phase 2
    Open
    Interview
    

    View Slide

11. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 11
    May 30, 2018
    RQ1: Is there a difference in the way
    users interact with notes, transcripts and
    audio recordings and do the interaction
    strategies change when time elapses?
    

    View Slide

12. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 12
    May 30, 2018
    Overall Usage
    0
    20
    40
    60
    80
    100
    0 7 30
    Likelihood of Usage (%)
    Time Interval (days)
    OM Notes Audio
    0
    20
    40
    60
    80
    100
    0 7 30
    Likelihood of Usage (%)
    Time Interval (days)
    OM Notes Transcript
    Audio Only Transcript Only
    

    View Slide

13. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 13
    May 30, 2018
    Overall Usage
    0
    20
    40
    60
    80
    100
    0 7 30
    Likelihood of Usage (%)
    Time Interval (days)
    OM Notes Audio Transcript
    Audio and Transcript Combined
    

    View Slide

14. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 14
    May 30, 2018
    0
    20
    40
    60
    80
    100
    0 7 30 0 7 30
    Time Interval (days) Time Interval (days)
    Likelihood of Usage (%)
    Transcript
    Navigation
    Skimming
    Audio
    Different Ways of Interacting with PM
    Audio Only Transcript Only
    

    View Slide

15. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 15
    May 30, 2018
    Different Purposes of PM Use
    0
    20
    40
    60
    80
    100
    0 7 30 0 7 30
    Time Interval (days) Time Interval (days)
    Likelihood of Usage (%)
    Transcript
    Recall
    Verifying
    Audio
    Audio Only Transcript Only
    

    View Slide

16. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 16
    May 30, 2018
    RQ2: Is there a difference in accuracy and
    efficiency between audio and transcript PM?
    

    View Slide

17. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 17
    May 30, 2018
    Accuracy
    ▪ No significant difference between audio and transcript in
    accuracy
    ▪ Accuracy does not change over time intervals
    ▪ Choice of interaction has no impact on accuracy
    

    View Slide

18. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 18
    May 30, 2018
    Efficiency
    0
    20
    40
    60
    80
    100
    120
    140
    0 7 30
    Efficiency (s)
    Time Interval (days)
    Transcript Navigating Transcript Skimming
    Audio Navigating Audio Skimming
    

    View Slide

19. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 19
    May 30, 2018
    RQ3: Do factors such as confidence
    influence the way users interact with PM?
    

    View Slide

20. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 20
    May 30, 2018
    Confidence Influence on Interactions
    

    View Slide

21. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 21
    May 30, 2018
    Design Implications
    Co-indexing PMs Efficiency and Accuracy Recall and Verification
    

    View Slide

22. Payam Ebrahimi - Department of Computer Science - pebrahim@vub.be 22
    May 30, 2018
    Conclusion
    ▪ Investigated the overall usage of audio and transcript PM
    ▪ Audio PM is not less efficient than transcript PM
    ▪ Three design implications
    ▪ Designing for individual or combined audio and transcript
    ▪ PM is not only chosen based on efficiency and accuracy
    ▪ Designing for recall and verification
    

    View Slide