Paul Fu

AI Design and the Future of Experience Design

Transcript

1. AI Design and the Future of Experience Design Alibaba International User Experience Head DAMO Natural HCI Lab Sr. Director Paul Fu Ph.D.
2. Alibaba Ecosystem OPERATING SYSTEM CLOUD COMPUTING DATA TECHNOLOGY TEAM Building the Infrastructure of Commerce Buyers Sellers MARKETPLACES Marketing Financing & Payment Solution Professional Service Providers Digital Entertainment Logistics Mobile Browser ENABLING 
 SERVICES Location-Based Social Media Weibo ® $
3. Perception Affective Computing Emotion & Affection Decision making Consumer Psychology Cognitive Science Behavioral Economics Server/ Chip CDN Quantum Computing Database/ Storage AI Algorithm Big Data Virtual/ Scheduling Natural Human-Computer Interaction Lab
4. From machine language to human natural language CUI Command line GUI Interaction design UX Design Visual Design Creative Design User Research NUI Graphic User Interface Natural User Interface Voice Interaction Design AI Design Space Design Emotional Computing Tangible Interaction Design
5. USER 170 Million Keyboard 700 Million Touch Screen 1.4 Billion Natural HCI
6. HUMAN Input Output COMPUTER
7. Vision Through sense digitization and AI, enhance human capability to interact with machine in natural ways
8. User Impact Scalability Visual, Voice, Haptics, and Olfactory interaction technologies, affective computing, human-field interaction, AI design Research Topics Project Selection Criteria
10. AI Fonts and Alibaba Wood Refinity Smart Touch Olfactory Interaction
11. 6763 characters 26 letters
12. Knowledge Map of fonts based on trend and style Training Optimization Optimization Training Optimization Representative Character Design Saved over 50% of the time 6763 Characters Font Style Selection Font Generation through ML QA & Launch Requirement Analysis AI Font Generation
13. Alibaba Wood - AI Short Video Pictures and Text Pictures and Videos Video Auto Clipping
14. Technology Used in Alibaba Wood
15. Application Scenarios Product Details Instagram AD Social Media Postings Video Site Content Social Media AD Short Video News
18. Short Video Generation from Product Live Show P r o d u c t E x p o s u r e I n c r e a s e d 2 4 % 1 2 m i l l i o n s h o r t v i d e o s w i l l b e g e n e r a t e d f o r 1 6 8 8 . c o m
19. • Interactive Experience • Create Infinite Possibilities Beyond Virtual Reality • Multi-sensory + Natural Human Computer Interaction REFINITY Infinite Reality
20. REFINITY： Multimodal interaction engine Interaction: Mid Air Mixed Reality Display: Naked eye 3D (partner) Technology Core: Body Gesture Haptic Smell + + Refinity provides a cross-platform naked-eye MR interaction engine that combines five senses for different applications
22. Digital Factory Interactive IOT
23. Voice interaction order machine - COSTA
24. Voice interaction order machine - KFC
26. Smell Human Limited to individual perception Visualization Smell Generation Detection Digitization Natural New Tech Machine Olfactory based search Health detection Olfactory AR Quality control OLFACTORY INTERACTION
27. SCENT DIGITALIZATION Convert scent to digital information to be perceived, transmitted, received and played over the Internet Scent Data Collection Process Capture Odor Data by the Device Transform the Digital Data into the Smell Label Build the Scent Database
28. SCENT DIGITALIZATION Similar Scent Product Recommendations Scent Data Collection Recommend Products with Similar Smell Recommend Products with Differentiated Smell Smell Label
29. Multimodal Design For Visually Impaired Inclusive Design
31. Challenges Missing Content Difficult to Click Learning Curve
32. Solutions Smart UI Proxy Multimodal Output Different Apps
33. Smart UI Proxy Read UI Back to Homepage Stop UI Reading Dynamic Buttons Function Add to Shopping Cart Add to Favorite Read Product Details
37. Total User Experience 手机上市时间：2018年4月 机型上安装为例 手机外观尺寸 贴膜器在 SmartFit贴合手机 正视图 SmartFit贴合手机 右视图 SmartFit贴合手机 左视图 SmartFit 结构爆炸示意图 安装完成，下载并启动软件 进行适配 定位适配原理：分别 触摸不同位置触觉膜， 软件会根据分辨率等 因素计算出位置坐标。 进行适配后将出现灰色悬浮窗 用户在触觉膜上的交互行为 通过手机屏幕上的悬浮窗感应接收 使用 SmartFit 固定贴合 好触觉膜。 即可运行软件，软件会指 引视障用户，使触觉膜与 智能手机操作适配。 P20 149.1mm*70.8mm*7.65mm HUAWEI P20 下 压 固 定 SmartFit贴合手机 后视图 分体式贴膜器有两个部件，分别固定于手机底部两侧。 Smart Fit “盲键”贴膜器包装造型方案一：分体式 SmartFit贴合手机 上视图 SmartFit贴合手机 下视图 A B C 可按压触发模块 开槽设计，待触觉膜 固定屏幕上，方便抽出 触觉反馈膜 根据工艺可采用 背胶纸或无背胶纸
38. Future of Experience Design: Natural HCI（CHI 2018） Natural UI is a human computer interaction system, which can leverage human’s natural perception, attention, cognition, and emotion capabilities • 3D Visual • Gesture • Natual language processing and understanding • More natural language express • Multi-model emotional computing • Multi-model interaction • Machine self-learning capability • Ability to handle ambiguity • Understand Individual Difference • Adaptivity • Ambiguity handling • Learning Capability Characteristics Technology Needed
39. Alibaba International User Experience Head DAMO Natural HCI Lab Sr. Director Paul Fu Ph.D. AI Design and the Future of Experience Design