2018.07-2018.10 Jane Yung-jen Hsu Juho Kim Lin-Lin Chen 2018.06 Research Topics: Human-AI Interaction, CSCW, Computer-Assisted Learning, Creativity Support Tools, AI Toolkits for Design, Crowdsourcing 2 Hao-Chuan Wang Yi-Ching (Janet) Huang AI, Multi-Agents Systems, Activity Recognition, Human Computation CSCW, HCI, Computer- Supported Creativity HCI, Learning at Scale, Data-Driven Interaction, Crowdsourcing Design Research, Design Innovation Strategy Professor Acting Associate Professor Assistant Professor Full Professor Chair of Design Innovation Strategy Dean of the Industrial Design Dept. Director of the NTU IoX Center
is all blank, just like everything I do in life. Only one word-“index” Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning
is all blank, just like everything I do in life. I feel useless Only one word-“index” … And nobody loves me Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning
is usually evaluated by multiple criteria 4. Quality can be improved by iterative refinement 2. Answer is not true or false, but how good the answer is 11 (summarized from prior literature) Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning
Systems | Human-AI Co-Learning Human Intelligence Machine Intelligence Common Sense Empathy & Creativity Pattern Recognition Probabilistic Consistency Speed & Efficiency Analytic Intuitive Flexibility & Transer Annotation of Arbitrary Data Dellermann et al., Hybrid Intelligence, Business & Information Systems Engineering, 2019. 13
Intelligence systems outperform a single machine or human, even an expert? 1 2 Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning 15
Support Human-Machine System for Drawing Support Human-AI System for Smart Environment Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning 16
Human-AI System for Smart Environment Three Contexts of Hybrid Intelligence Systems Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning 17
Jane Yung-jen Hsu. Supporting ESL writing by prompting crowdsourced structural feedback. In Proceedings of the Fifth AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2017) Yi-Ching Huang, Hao-Chuan Wang, and Jane Yung-jen Hsu. Bridging learning gap in writing education with a crowd-powered system. CHI 2017 Workshop on Designing for Curiosity, Denver, Colorado, USA, 2017. 18 StructFeed (Hybrid Intelligence System) Revision Feedback Crowd Machine Revision annotation User Writing workflow Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning
sentence prediction Study 3: writing revision based on feedback Hybrid Intelligence vs Machine Intelligence Hybrid Intelligence vs Human Intelligence Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning 20
US$2 $1-$2 Time 1-2 days 30 mins 1-5 hrs Avg diff rating 0.15 0.21 0.43 Avg standard deviation 0.32 0.32 0.44 # of decreased diff rating 3 2 0 # of equal rating 7 7 5 ! ! ! ! ! ! 24 Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning Expert Random Crowd Worker vs vs Writing improvement based on three types of feedback StructFeed
Human-AI System for Smart Environment Three Contexts of Hybrid Intelligence Systems Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning 25
Human-AI System for Smart Environment Three Contexts of Hybrid Intelligence Systems Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning
2. machine collects, aggregates, and generates gazed-based guidance 3. human uses these guidances to facilitate their drawing Mon-Chu Chen, Yi-Ching Huang, and Kuan-Ying Wu. Gaze-based drawing assistant. In ACM SIGGRAPH 2014 Posters. Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning
Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning subject #1 subject #2 subject #3 with guidance with guidance with guidance Help proportion and perspective
Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning subject #1 subject #2 subject #3 with guidance with guidance with guidance Create new style
Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning subject #1 subject #2 subject #3 with guidance with guidance with guidance Create new style Machine is not replacing human in creative drawing, but only supporting their activity.
Human-AI System for Smart Environment Three Contexts of Hybrid Intelligence Systems Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning
Human-AI System for Smart Environment Three Contexts of Hybrid Intelligence Systems Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning
facilitating system performance Open Simple Lightwe CrowdButton A micro-volunteering situated platform which allows passersby to report a room status by clicking a button. 35 15-20 contributions per device, per day 56% prediction accuracy Yi-Ching Huang. Designing a micro-volunteering platform for situated crowdsourcing. In Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work and Social Computing, CSCW’15 Companion. What kind of activity is happening in the room? Empty 1 2 3 4 Meeting Lecture Study Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning
a prediction, and passersby can confirm or correct the predicted answer, and LED display show the feedback Happy Surprise when human input is as same as machine’s prediction when human input is different from machine’s prediction 36 Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning 3 1 4 2
0.86 0.87 0.56 0.80 0.79 0.67 0.62 0.82 0.62 0.82 0.68 Contribution Aggregated Contribution Prediction Experiment: - People generate more accurate labels by getting machine feedback (#2, #3, #4) than without feedback (#1). - The accuracy of room status prediction increases from 56% (without feedback) to 87% (with feedback) + Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning 37 Machine feedback helps generate accurate labels
Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning Feedback Artifact drawing design writing User Revision Human Machine Hybrid Intelligence System knowledge, patterns, strategies workflow
Hybrid Intelligence System Structure Leveraging “structure” to separate, distribute, and aggregate tasks 1 knowledge, patterns, strategies workflow Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning (1) Separate the complex task into small pieces (2) Distribute these small pieces to humans or machines (3) Aggregate results from difference resources
Hybrid Intelligence System Techniques for enabling “collaborations” between users and hybrid intelligence systems 2 knowledge, patterns, strategies workflow Intro | Hybrid Intelligence | Three Contexts of Hybrid Intelligence Systems | Human-AI Co-Learning Collaboration
intelligibility transparency Explainable AI interact with new materials understand people Accessible ML uncertainty evolving learning trust control user expectation understand acceptance bias Engineer sketching prototyping understand data Designing Data and AI Vision for Education
janetyc
t2kasa
kayceesrk
pshiko
izuna385
chokkan
usaito
shinyorke
sorami
scatterlab
ttsunoda
shiftpn
danielanewman
shpigford
erikaheidi
tenderlove
garrettdimon
3n
jnunemaker
qrush
brianwarren
kneath
swwweet
jponch
