Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CSC570 Lecture 10

CSC570 Lecture 10

Applied Affective Computing
Face Gestures
(202305)

Javier Gonzalez-Sanchez
PRO

May 07, 2023
Tweet

More Decks by Javier Gonzalez-Sanchez

Other Decks in Research

Transcript

  1. jgs
    CSC 570
    Current Topics in Computer Science
    Applied Affective Computing
    Lecture 10:
    Face Gestures
    Dr. Javier Gonzalez-Sanchez
    [email protected]
    www.javiergs.com
    Building 14 -227
    Office Hours: By appointment

    View Slide

  2. jgs
    Previously

    View Slide

  3. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 3
    Face
    https://doi.org/10.1371/journal.pone.0251057

    View Slide

  4. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 4
    Emotiv EPOC X

    View Slide

  5. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 5
    Towards This Scenario

    View Slide

  6. jgs
    Face Gestures

    View Slide

  7. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 7
    § EEG
    § Inertial Measurement Unit (IMU) sensors to detect movements of the head and
    face.
    § IMU measures changes in acceleration, orientation, and rotation, which allow the
    headset to detect head movements, such as tilting or nodding, as well as
    movements of the face, such as jaw clenching or mouth opening.
    Headset

    View Slide

  8. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 8
    1. Smile: The software recognizes a smile as an upward movement of the
    corners of the mouth (low-quality happy/agreement)
    3. Clench: The software recognizes the clenching of the jaw
    4. Mouth open: The software recognizes an open mouth
    Data

    View Slide

  9. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 9
    1.Raise Brow: The software recognizes the movement of raising the eyebrows
    (low-quality surprise)
    2. Furrow Brow: The software recognizes the movement of raising the
    eyebrows (low-quality frustration)
    1.Blink: The software recognizes a blink as a momentary closing of the eyes
    3. Wink: The software recognizes a wink as a momentary closing of one eye
    Data

    View Slide

  10. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 10
    Data

    View Slide

  11. jgs
    Software

    View Slide

  12. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 12
    Maven

    View Slide

  13. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 13
    Main.java

    View Slide

  14. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 14
    EmotivSocket.java (Part 1/2)

    View Slide

  15. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 15
    EmotivSocket.java (Part 2/2)

    View Slide

  16. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 16
    EmotivDelegate.java (Part 1/5)

    View Slide

  17. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 17
    EmotivDelegate.java (Part 2/5)

    View Slide

  18. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 18
    EmotivDelegate.java (Part 3/5)

    View Slide

  19. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 19
    EmotivDelegate.java (Part 4/5)

    View Slide

  20. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 20
    EmotivDelegate.java (Part 5/5)

    View Slide

  21. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 21
    Let’s Work

    View Slide

  22. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 22
    met

    View Slide

  23. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 23
    PAD
    +P+A+D
    Engagement
    +P-A+D
    Meditation
    Concentration
    Thought
    Relaxation
    +P+A-D
    Excitement
    Interest
    Dependence
    +P-A-D
    Starting
    Agreement
    Docility
    -P+A+D
    Disagreement
    Hostility
    -P-A+D
    Disdain
    -P+A-D
    Frustration
    Unsureness
    Anxiety
    -P-A-D
    Boredom
    Pleasure
    (Valence)
    Arousal
    (Activation)
    Dominance

    View Slide

  24. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 24
    fac

    View Slide

  25. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 25
    WebSocket (in Java)
    https://github.com/javiergs/CSC570
    Java Emotiv Client

    View Slide

  26. jgs
    Assignment 04

    View Slide

  27. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 27
    Towards This Scenario

    View Slide

  28. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 28
    Architecture
    Bluetooth
    EMOTIV PRO Your App
    WebSocket
    (server)
    WebSocket
    (client)
    Your App
    WebGazer
    Facial Gestures
    Affect

    View Slide

  29. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 29
    Simulator

    View Slide

  30. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 30
    Power On

    View Slide

  31. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 31
    Connect

    View Slide

  32. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 32

    View Slide

  33. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 33
    Configuration

    View Slide

  34. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 34
    Data

    View Slide

  35. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 35
    Questions

    View Slide

  36. jgs
    Javier Gonzalez-Sanchez | CSC 309 | Winter 2023 | 36
    Office Hours
    Tuesday and Thursday 3 - 5 pm
    But an appointment required
    Sent me an email – [email protected]

    View Slide

  37. jgs
    CSC 570 Applied Affective Computing
    Javier Gonzalez-Sanchez, Ph.D.
    [email protected]
    Spring 2023
    Copyright. These slides can only be used as study material for the class CSC308 at Cal Poly.
    They cannot be distributed or used for another purpose.

    View Slide