Reading Activity Recognition using an Off-the-shelf EEG Kai Kunze, Yuki Shiga, Shoya Ishimaru, Koichi Kise Osaka Prefecture University Detecting Reading Activities and Distinguishing Genres of Documents 

Kai Kunze - Reading Activity Recognition using an Off-the-shelf EEG Overview Motivation Modalities to Explore Reading Activities Approach Experiments Results Conclusion and Future Work 2 

Kai Kunze - Reading Activity Recognition using an Off-the-shelf EEG Towards “user-centered” Document Analysis ... Traditionally, Document Analysis focuses on the object 3 Documents as a structured source of information 

Kai Kunze - Reading Activity Recognition using an Off-the-shelf EEG “User-centered” Document Analysis Let’s focus on the readers 1. Analyze the document through the users ... Are they reading? What did they read? How much?How fast? How often? How much do they understand? 2. Analyze the users through the documents/ their reading behavior 4 Similar to Kindle Highlights Imagine annotating documents with: “x readers stopped reading after this sentence.” “most readers felt sad after this paragraph” “the reader never saw this warning label” ... 

Kai Kunze - Reading Activity Recognition using an Off-the-shelf EEG Interesting Modalities for Tracking Reading Habits Computer Vision: Egocentric Camera Monitoring Brain Activity Electroencephalography (EEG), Near-Infrared Spectroscopy (fNIS) Eye-Tracking (see the “Wordometer” talk this morning) ... we are actively exploring other sensing modalities. Important: Unobtrusive and Robust Tracking. 5 

Kai Kunze - Reading Activity Recognition using an Off-the-shelf EEG Case Study: Evaluating an Off-the-Shelf EEG Can we distinguish different types of reading activities using a relatively cheap, commercially available EEG device? Is it feasible to segment reading from non-reading activities with such a device? EGG gives us also some additional useful information ... Emotional State, Concentration etc. 6 

Kai Kunze - Reading Activity Recognition using an Off-the-shelf EEG Approach EEG signal analysis bandpass filter between 1-30Hz. Fast Fourier Transform (FFT) sliding window 500 msec., step size 20 msec. Sum the Frequencies according to the Classification of Brain Bands: Beta (13-30Hz), Alpha (8-12 Hz), Theta (4-8 Hz), and Delta (less than 4 Hz) other features: RMS, FFT Mean, FFT Variance K-nearest Neighbor Classifier 7 

Kai Kunze - Reading Activity Recognition using an Off-the-shelf EEG Experiments Disclaimer: Very few participants (in the paper all recorded on one day) 3 participants over 5 days recording Emotiv 14 Channel Wet Electrode EEG 10 min. per activity 3 reading: read a science paper, manga, internet news 3 not reading: listening to music, watching a video, playing chess(draw) doing nothing as reference 8 

Kai Kunze - Reading Activity Recognition using an Off-the-shelf EEG Experimental Results Reading vs. Not Reading Recognition (leave-out-one day): Train on 4 evaluate on 1 day User dependent: 90 % User independent: 69 % Recognition for the Individual Tasks (leave-out-one day): User Dependent: 80 % User Independent: 62.25% 9 a b c d e f g ←)classified)as 86 0 12 1 0 0 0 a=Paper 6 90 0 0 1 2 0 b=Watch 22 1 73 3 0 0 0 c=Manga 9 7 20 63 0 0 0 d=News 0 12 6 1 76 4 0 e=Music 0 14 1 0 26 58 0 f=Nothing 0 0 6 0 0 0 93 g=Chess 

Kai Kunze - Reading Activity Recognition using an Off-the-shelf EEG Conclusion and Future Work Of course, larger scale experimental setup Segmentation of Reading Activity Emotiv: records for 3 hours charging while recording is not possible needs a laptop (usb plug, proprietary wireless protocol) Evaluate the best electrode positions for tracking reading. Use devices with fewer electrodes long-term recordings 10 

Kai Kunze - Reading Activity Recognition using an Off-the-shelf EEG Questions? Answers! http://kaikunze.de twitter: @k_garten facebook: kai.kunze app.net: @kkai [email protected] https://github.com/kkai/ 11