Dr. Javier Gonzalez-Sanchez javiergs@calpoly.edu www.javiergs.info o ffi ce: 14 -227 CSC 486 Human-Computer Interaction Lecture 04. Final Project

Previously

Interaction 3 Thinking Input Output Data Processing Input Output Reciprocal Action or Influence Affective Factors Cognitive Factors

Work f low 4

PAD 5

Data File • First, time information is represented by the attribute time. • The device information includes metrics such as batteryLevel, batteryPercent, wirelessSignal, and sensorsQuality. •For facial actions, attributes like actionEye, actionUpperFace, powerUpperFace, actionLowerFace, and powerLowerFace capture various facial expressions and their intensities. •The emotional metrics encompass a range of states, including engagement, excitement, longTermExcitement, stress, relaxation, interest, and focus. In addition, boolean fl ags, such as isENG, isEXC, isSTR, isREL, isINT, and isFOC, indicate whether speci fi c emotional states are active. •The Pleasure, Arousal, and Dominance (PAD) model is represented through the attributes Pleasure, Arousal, and Dominance. Finally, ignore SAM. 6

timestamp fixationIndex gazePointX gazePointY mappedFixationPointX mappedFixationPointY fixationDuration Short Term Excitement Long Term Excitement Engagement/Boredom Meditation Frustration Conductance agreement concentrating 4135755652 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 4135755659 213 573 408 570 408 216 4135755668 0.436697 0.521059 0.550011 0.335825 0.498908 4135755676 213 566 412 570 408 216 4135755692 213 565 404 570 408 216 4135755709 213 567 404 570 408 216 4135755714 4135755726 213 568 411 570 408 216 4135755742 213 568 409 570 408 216 4135755759 213 563 411 570 408 216 4135755761 4135755776 213 574 413 570 408 216 4135755792 213 554 402 570 408 216 4135755809 214 603 409 696 405 216 4135755824 4135755826 214 701 407 696 405 216 4135755842 214 697 403 696 405 216 4135755859 214 693 401 696 405 216 4135755876 214 700 402 696 405 216 4135755892 214 701 411 696 405 216 4135755909 214 686 398 696 405 216 4135755918 4135755926 214 694 399 696 405 216 4135755942 214 694 407 696 405 216 4135755959 214 698 404 696 405 216 4135755964 4135756027 0.436697 0.521059 0.550011 0.335825 0.498908 1 1

timestamp fixationIndex gazePointX gazePointY mappedFixationPointX mappedFixationPointY fixationDuration Short Term Excitement Long Term Excitement Engagement/Boredom Meditation Frustration Conductance agreement concentrating 4135755652 213 574 414 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755659 213 573 408 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755668 213 573 408 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755676 213 566 412 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755692 213 565 404 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755709 213 567 404 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755714 213 567 404 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755726 213 568 411 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755742 213 568 409 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755759 213 563 411 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755761 213 563 411 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755776 213 574 413 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755792 213 554 402 570 408 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135755809 214 603 409 696 405 216 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1 4135756027 215 728 406 804 387 183 0.436697 0.521059 0.550011 0.335825 0.498908 0.401690628 1 1

Projects

Affect Recognition 10 BCI and Gaze Points engagement

Affect Recognition 11 BCI and Gaze Points frustration

Affect Recognition 12 BCI and Gaze Points engagement

Affect Recognition 13 BCI and Gaze Points frustration

Neuromarketing 14 Chavez, M., Christopherson, R., Gonzalez-Sanchez, J., Atkinson, R. User Experience. 2018

Projects 15 Gonzalez-Sanchez, J., Chavez, M., Gibson, D., and Atkinson, R. Multimodal Affect Recognition in Virtual Worlds. ACII 2013

Projects 16 Harris, A., Hoch, A., Kral, R., Teposte, M., Villa, A., et. al. Including affect-driven adaptation to the Pac-Man video game. ACM ISWC 2014

Projects 17 Bernays, R., Mone, J., Yau, P., Murcia, M., Gonzalez-Sanchez, J., et al. Lost in the dark: ACM UIST 2012

Projects 18 Hang, B., Loucks, S., Patel, P., Wiseman, K. Capstone Project 2021

Projects 19 VanLehn, K., Burleson, W., Chavez, M., Gonzalez-Sanchez, J., et al. The Affective Meta-Tutoring project ITS 2014 - 2018

Projects 20 Rodriguez, J., Gonzalez-Sanchez, J., Del-Valle, C. Affect-Driven Robot-assisted Walking Therapy 2020-2022

Projects 21

Tools

Questions 23

Lab 04

Create a Document 1. Te a m of 2 or 3. Write Your N a mes 2. Title 3. Keywords 4. Problem th a t you w a nt to a ddress. Wh a t is the problem or need? Be sure to expl a in the technic a l problem or need. (800 ch a r a cters m a ximum) 5. Justi f ic a tion. Why should people c a re? (800 ch a r a cters m a ximum) 6.Go a ls/Appro a ch. Keep in mind th a t this is only a n 8-week project. Your go a ls a nd a ppro a ch should be scoped a ppropri a tely. (2500 ch a r a cters m a ximum) 7. Expertise or Intended responsibilities per te a m member 25

CSC 486 Human-Computer Interaction Javier Gonzalez-Sanchez, Ph.D. javiergs@calpoly.edu Winter 2025 Copyright. These slides can only be used as study material for the class CSC 486 at Cal Poly. They cannot be distributed or used for another purpose.