CSC570 Lecture 08

Applied Affective Computing
Gaze Tracking
(202304)

Transcript

1. jgs
   CSC 570
   Current Topics in Computer Science
   Applied Affective Computing
   Lecture 08:
   Gaze Tracking
   Dr. Javier Gonzalez-Sanchez
   javiergs@calpoly.edu
   www.javiergs.com
   Building 14 -227
   Office Hours: By appointment
   

   View Slide

2. jgs
   Previously …
   Lab 01 and Assignment 02
   

   View Slide

3. jgs
   Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 3
   Assignment 02
   • Open or Closed Eyes VS. Brain
   • 5 diverse stimulation scenarios VS. Brain
   • 5 diverse stimulation scenarios VS. Affect
   Follow 2 approaches:
   a) Clustering as described today (EM, K-means, Density)
   b) Explore another solution to the best of your knowledge
   (Machine Learning, Data mining, Statistics)
   Due: Monday (April 24)
   

   View Slide

4. jgs
   Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 4
   Lab 01
   • 5 diverse stimulation scenarios VS. Brain
   • 5 diverse stimulation scenarios VS. Affect
   Follow Random Forest approach
   Due: Monday (April 26)
   

   View Slide

5. jgs
   Preprocessing the Dataset
   UCI Emotiv
   

   View Slide

6. jgs
   Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 6
   1) EEG.AF3
   2) EEG.F7
   3) EEG.F3
   4) EEG.FC5
   5) EEG.T7
   6) EEG.P7
   7) EEG.O1
   8) EEG.O2
   9) EEG.P8
   10)EEG.T8
   11)EEG.FC6
   12)EEG.F4
   13)EEG.F8
   14)EEG.AF4
   Dataset Attributes | X
   

   View Slide

7. jgs
   Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 7
   Dataset Attributes | X
   1) PM.Engagement.Scaled
   2) PM.Excitement.Scaled
   3) PM.Stress.Scaled
   4) PM.Relaxation.Scaled
   5) PM.Interest.Scaled
   6) PM.Focus.Scaled
   

   View Slide

8. jgs
   Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 8
   § EEG.MarkerValueInt
   Dataset Attributes | Y
   

   View Slide

9. jgs
   Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 9
   § https://webgazer.cs.brown.edu
   § eye tracking library using common webcams to infer the eye-gaze locations
   of web visitors on a page in real-time.
   § written in JavaScript
   § can be integrated into a website
   § runs entirely in the client browser, so no video data needs to be sent to a
   server, and it requires the user's consent to access their webcam.
   WebGazer
   

   View Slide

10. jgs
    Assignment 03
    

    View Slide

11. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 11
    Assignment 03
    • Collect EEG Data (Emotiv PRO)
    • Collect Gaze Tracking Data (WebGazer)
    • Save the integrated data in a Dataset
    (CSV or ARFF file)
    

    View Slide

12. jgs
    Assignment 03 | Part 01
    

    View Slide

13. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 13
    Add Lab 2 into the New Assignment 03
    Imagen (photo)
    Observer 1 minute
    Record data
    What was the part that catch attention?
    

    View Slide

14. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 14
    § https://github.com/brownhci/WebGazer/blob/master/www/calibration.html
    Callibration
    

    View Slide

15. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 15
    Example
    

    View Slide

16. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 16
    
    
    
    
    
    
    <br/>webgazer.resume();<br/>webgazer.setGazeListener(function(data, elapsedTime) {<br/>if (data != null) {<br/>var x = data.x;<br/>var y = data.y;<br/>document.getElementById("gazeData").innerHTML = "Gaze coordinates: x=" + x + ", y=" + y;<br/>}<br/>}).begin();<br/>
    
    Template
    

    View Slide

17. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 17
    
    <br/>html, body {height: 100%;margin: 0; padding: 0; }<br/>table {width: 100%;height: 100%;border-collapse: collapse;}<br/>td {border: 1px solid black;}<br/>
    
    
    Cell 1
    Cell 2
    Cell 3
    
    
    Cell 4
    Cell 5
    Cell 6
    
    
    Cell 7
    Cell 8
    Cell 9
    
    
    

    View Slide

18. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 18
    Towards This Scenario
    

    View Slide

19. jgs
    Assignment 03 | Part 02
    

    View Slide

20. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 20
    Emotiv
    

    View Slide

21. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 21
    Architecture
    Bluetooth
    EMOTIV PRO Your App
    WebSocket
    (server)
    WebSocket
    (client)
    EMOTIV BrainViz
    EMOTIV BCI
    

    View Slide

22. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 22
    Let’s Work
    

    View Slide

23. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 23
    Questions
    

    View Slide

24. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 24
    Office Hours
    Tuesday and Thursday 3 - 5 pm
    But an appointment required
    Sent me an email – javiergs@calpoly.edu
    

    View Slide

25. jgs
    CSC 570 Applied Affective Computing
    Javier Gonzalez-Sanchez, Ph.D.
    javiergs@calpoly.edu
    Spring 2023
    Copyright. These slides can only be used as study material for the class CSC308 at Cal Poly.
    They cannot be distributed or used for another purpose.
    

    View Slide