Dr. Javier Gonzalez-Sanchez javiergs@calpoly.edu www.javiergs.info o ffi ce: 14 -227 CSC 486 Human-Computer Interaction Lecture 03. Affective Factors II

Previously On Week 01

PAD 3

PAD Zones 4

Work f low 5

Brain 6

Brain 7 Timestamp AF3 F7 F3 FC5 T7 P7 O1 O2 P8 T8 FC6 F4 F8 AF4 AccX AccY 101116112544901 4542.05 4831.79 4247.18 4690.26 4282.56 4395.38 4591.79 4569.23 4360 4570.77 4297.44 4311.28 4282.56 4367.18 1660 2003 101116112544901 4536.92 4802.05 4243.08 4673.85 4272.31 4393.33 4592.82 4570.26 4354.87 4570.26 4292.31 4309.74 4277.95 4370.77 1658 2002 101116112545010 4533.33 4798.97 4234.87 4669.74 4301.03 4396.92 4592.31 4570.77 4351.28 4561.03 4281.54 4301.54 4271.28 4363.59 1659 2003 101116112545010 4549.23 4839.49 4241.03 4691.28 4333.85 4397.95 4596.41 4567.18 4355.9 4556.41 4286.15 4306.15 4277.95 4369.74 1659 2003 101116112545010 4580 4865.64 4251.79 4710.26 4340 4401.54 4603.59 4572.82 4360 4558.46 4298.97 4324.62 4296.41 4395.9 1657 2004 101116112545010 4597.44 4860 4252.82 4705.64 4350.26 4412.31 4603.59 4577.44 4357.44 4555.9 4295.38 4329.23 4296.41 4414.36 1656 2005 101116112545010 4584.62 4847.69 4246.67 4690.26 4360 4409.23 4597.44 4569.74 4351.79 4549.74 4278.97 4316.92 4272.82 4399.49 1656 2006 101116112545010 4566.15 4842.05 4238.46 4684.1 4322.05 4389.74 4592.82 4566.67 4351.79 4549.74 4274.36 4310.26 4262.05 4370.77 1655 2005 101116112545010 4563.59 4844.62 4231.79 4687.69 4267.69 4387.69 4594.36 4580 4361.03 4556.41 4278.97 4310.77 4274.36 4370.77 1653 2006 101116112545010 4567.18 4847.18 4233.33 4688.72 4285.13 4409.23 4602.05 4589.23 4368.21 4560 4280.51 4310.77 4281.54 4390.26 1655 2004 101116112545010 4562.05 4840.51 4227.18 4673.85 4300 4405.13 4611.28 4601.03 4376.41 4561.54 4280 4303.59 4279.49 4374.87 1652 2000

Brain 8 14 channels 128 samples per second 1,792 values por second 107,520 values per minute 6,451,200 values per hour

Brain 9 Timestamp Short Term Excitement Long Term Excitement Engagement Meditation Frustration 101116091145065 0.447595 0.54871 0.834476 0.333844 0.536197 101116091145190 0.447595 0.54871 0.834476 0.333844 0.536197 101116091145315 0.447595 0.54871 0.834476 0.333844 0.536197 101116091145440 0.487864 0.546877 0.834146 0.339548 0.54851 101116091145565 0.487864 0.546877 0.834146 0.339548 0.54851 101116091145690 0.487864 0.546877 0.834146 0.339548 0.54851 101116091145815 0.487864 0.546877 0.834146 0.339548 0.54851 101116091145940 0.521663 0.545609 0.839321 0.348321 0.558228 101116091146065 0.521663 0.545609 0.839321 0.348321 0.558228 101116091146190 0.521663 0.545609 0.839321 0.348321 0.558228 101116091146315 0.521663 0.545609 0.839321 0.348321 0.558228 101116091146440 0.509297 0.544131 0.84401 0.358717 0.546771 101116091146565 0.509297 0.544131 0.84401 0.358717 0.546771 101116091146690 0.509297 0.544131 0.84401 0.358717 0.546771 101116091146815 0.509297 0.544131 0.84401 0.358717 0.546771 101116091146941 0.451885 0.541695 0.848087 0.368071 0.533919

Brain 10 24 5 samples per second 5 affective states 25 values per second 1,500 values per minute 90,000 values per hour

Data File • First, time information is represented by the attribute time. • The device information includes metrics such as batteryLevel, batteryPercent, wirelessSignal, and sensorsQuality. •For facial actions, attributes like actionEye, actionUpperFace, powerUpperFace, actionLowerFace, and powerLowerFace capture various facial expressions and their intensities. •The emotional metrics encompass a range of states, including engagement, excitement, longTermExcitement, stress, relaxation, interest, and focus. In addition, boolean fl ags, such as isENG, isEXC, isSTR, isREL, isINT, and isFOC, indicate whether speci fi c emotional states are active. •The Pleasure, Arousal, and Dominance (PAD) model is represented through the attributes Pleasure, Arousal, and Dominance. Finally, ignore SAM. 11

Eye Tracking

Eye 13

Eye 14 Timestamp GPX GPY Pupil Left Validity L Pupil Right Validity R Fixation Event AOI 101124162405582 636 199 2.759313 0 2.88406 0 48 Content 101124162405599 641 207 2.684893 0 2.855817 0 48 Content 101124162405615 659 211 2.624458 0 2.903861 0 48 Content 101124162405632 644 201 2.636186 0 2.916132 0 48 Content 101124162405649 644 213 2.690685 0 2.831013 0 48 Content 101124162405666 628 194 2.651784 0 2.869714 0 48 Content 101124162405682 614 177 2.829281 0 2.899828 0 48 Content 101124162405699 701 249 2.780344 0 2.907665 0 49 Content 101124162405716 906 341 2.853761 0 2.916398 0 49 Content 101124162405732 947 398 2.829427 0 2.889944 0 49 Content 101124162405749 941 400 2.826602 0 2.881179 0 49 Content 101124162405766 938 403 2.78699 0 2.87948 0 49 KeyPress Content 101124162405782 937 411 2.803387 0 2.821803 0 49 Content 101124162405799 934 397 2.819166 0 2.871547 0 49 Content 101124162405816 941 407 2.811687 0 2.817927 0 49 Content 101124162405832 946 405 2.857419 0 2.857427 0 49 Content 101124162405849 0 0 -1 4 -1 4 49 Content

Eye 15 30 o 60 frames per second 30 o 60 inferences per second 1,800 o 3,600 values per minute 108,000 o 216, 000 values per hour

Eye 16

Gestures

Face 18 (Ekman and Friesen 1978) – Facial Action Coding System, 46 actions (plus head movements). 19 Lip Corner Depressor 26 Jaw Drop 27 Mouth Stretch

Face 19

Face 20 Timestamp Agreement Concentrating Disagreement Interested Thinking Unsure 101116112838516 0.001836032 0.999917 1.79E-04 0.16485406 0.57114255 0.04595062 101116112838578 0.001447654 0.9999516 1.29E-04 0.16310683 0.5958921 0.042706452 101116112838672 5.97E-04 0 1.5E-04 0.44996294 0.45527613 0.00789697 101116112838766 2.46E-04 0 1.75E-04 0.77445686 0.32144752 0.001418217 101116112838860 1.01E-04 0 2.04E-04 0.93511915 0.21167138 2.53E-04 101116112838953 4.18E-05 0 2.38E-04 0.983739 0.13208677 4.52E-05 101116112839016 1.72E-05 0 2.78E-04 0.9960774 0.07941038 8.07E-06 101116112839110 7.1E-06 0 3.24E-04 0.99906266 0.046613157 1.44E-06 101116112839156 2.92E-06 0 3.77E-04 0.99977654 0.026964737 2.57E-07 101116112839250 1.21E-06 0 4.4E-04 0.9999467 0.015464196 4.58E-08 101116112839391 4.97E-07 0 5.12E-04 0.9999873 0.008824189 8.18E-09 101116112839438 2.05E-07 0 5.97E-04 0.999997 0.005020725 1.46E-09 101116112839547 8.43E-08 0 6.96E-04 0.9999993 0.002851939 2.6E-10 101116112839578 3.47E-08 0 8.11E-04 0.9999999 0.001618473 4.64E-11 101116112839688 1.43E-08 0 9.45E-04 0.99999994 9.18E-04 8.29E-12 101116112839781 5.9E-09 0 0.001101404 1 5.21E-04 1.48E-12 101116112839828 2.43E-09 0 0.001283521 1 2.95E-04 2.64E-13

Face 21 31 30 frames per second 10 inferences per second 600 values per minute 36,000 values per hour

Face 22

We have a problem Well, You will have a problem

timestamp fixationIndex gazePointX gazePointY mappedFixationPointX mappedFixationPointY fixationDuration Short Term Excitement Long Term Excitement Engagement/Boredom Meditation Frustration Conductance agreement concentrating 4135755652 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 4135755659 213 573 408 570 408 216 4135755668 0.436697 0.521059 0.550011 0.335825 0.498908 4135755676 213 566 412 570 408 216 4135755692 213 565 404 570 408 216 4135755709 213 567 404 570 408 216 4135755714 4135755726 213 568 411 570 408 216 4135755742 213 568 409 570 408 216 4135755759 213 563 411 570 408 216 4135755761 4135755776 213 574 413 570 408 216 4135755792 213 554 402 570 408 216 4135755809 214 603 409 696 405 216 4135755824 4135755826 214 701 407 696 405 216 4135755842 214 697 403 696 405 216 4135755859 214 693 401 696 405 216 4135755876 214 700 402 696 405 216 4135755892 214 701 411 696 405 216 4135755909 214 686 398 696 405 216 4135755918 4135755926 214 694 399 696 405 216 4135755942 214 694 407 696 405 216 4135755959 214 698 404 696 405 216 4135755964 4135756027 0.436697 0.521059 0.550011 0.335825 0.498908 1 1

timestamp fixationIndex gazePointX gazePointY mappedFixationPointX mappedFixationPointY fixationDuration Short Term Excitement Long Term Excitement Engagement/Boredom Meditation Frustration Conductance agreement concentrating 4135755652 213 574 414 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755659 213 573 408 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755668 213 573 408 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755676 213 566 412 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755692 213 565 404 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755709 213 567 404 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755714 213 567 404 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755726 213 568 411 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755742 213 568 409 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755759 213 563 411 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755761 213 563 411 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755776 213 574 413 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755792 213 554 402 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755809 214 603 409 696 405 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135756027 215 728 406 804 387 183 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1

Questions 26

Lab 03

What can you tell me about him? 28

Data File • First, time information is represented by the attribute time. • The device information includes metrics such as batteryLevel, batteryPercent, wirelessSignal, and sensorsQuality. •For facial actions, attributes like actionEye, actionUpperFace, powerUpperFace, actionLowerFace, and powerLowerFace capture various facial expressions and their intensities. •The emotional metrics encompass a range of states, including engagement, excitement, longTermExcitement, stress, relaxation, interest, and focus. In addition, boolean fl ags, such as isENG, isEXC, isSTR, isREL, isINT, and isFOC, indicate whether speci fi c emotional states are active. •The Pleasure, Arousal, and Dominance (PAD) model is represented through the attributes Pleasure, Arousal, and Dominance. Finally, ignore SAM. 29

PAD Zones 30

Lab 03 What can you said about the participant whose file I am sharing with you?

CSC 486 Human-Computer Interaction Javier Gonzalez-Sanchez, Ph.D. javiergs@calpoly.edu Winter 2025 Copyright. These slides can only be used as study material for the class CSC 486 at Cal Poly. They cannot be distributed or used for another purpose.