Yusuke Sugano (The University of Tokyo) IVI Lab. Research Introduction

• Computer vision and human-computer interaction ‣ How can we achieve intelligent interaction between humans and computers through videos/images? Our Research Interests 2 Machine Human HCI CV Vision-based Human Understanding Understanding people from images Human-in-the-loop Interactive ML Understanding media together with people 3D models Latent appearance vector 𝒛 𝒛1 𝒛2 𝒛3 𝒛𝟎 Input Normal map rendering Images generated by SCGAN User-Adaptive Image Generation Generating images that match the user's intentions Distractiveness Noticeability Gaze distance Size Blink Opacity Movement User-Adaptive UI/Visualization Presenting information with the user's intentions

• Technique to measure where a person is looking at • Many applications ‣ As Measurement: advertisement and marketing ‣ As Input: gaze interaction, assistive technologies, attentive user interfaces, VR and e-sports ‣ As Feature: activity and intention recognition, medical diagnosis Eye Tracking and Gaze Estimation 3 https://www.youtube.com/watch?v=RpQVSmGvbMo

• Estimating gaze directions from ordinary cameras (webcams, wearable cameras, … ) ‣ Calibration-free estimation from low-resolution images • Technically more challenging than other existing approaches Camera-based Gaze Estimation 4

• Building techniques to adapt gaze estimation models to novel environments ‣ Use 3 D face reconstruction to synthesize training data for unseen head poses ‣ Train multi-view gaze estimator that can handle different camera configurations Adaptive Training for Gaze Estimation 5 Synthetic Training Data Multi-view Gaze Estimation {"!"# } {"$%$ } Training Test Multi-View Gaze Estimator {$} … Multi-View Gaze Estimation {"} … … … Single-View Gaze Estimation From novel camera pairs Single-View Gaze Estimator

• Gamified workshop for democratizing gaze estimation research ‣ Provide game-like experience where users participate in and understand the data collection process ‣ Joint project with DLX design lab, UTokyo-IIS AICOM Project 6 https://vimeo.com/ 4 29 881 90 7

• Real-world computer vision and machine learning applications with end-users in the loop CV Applications & Interactive Machine Learning 7 human: 27% natural: 24% ML workshop with novice users with disabilities Interactive Machine Learning

• Public workshops as social implementation of research ‣ Analyze how the perception of technology and sound change through interactions with ML technologies Workshop Events using Interactive ML 8 ML Workshop at Science Museum