Tweet
Tweet

Slide 1

Slide 1 text

AI Design and the Future of Experience Design Alibaba International User Experience Head DAMO Natural HCI Lab Sr. Director Paul Fu Ph.D.

Slide 2

Slide 2 text

Alibaba Ecosystem OPERATING SYSTEM CLOUD COMPUTING DATA TECHNOLOGY TEAM Building the Infrastructure of Commerce Buyers Sellers MARKETPLACES Marketing Financing & Payment Solution Professional Service Providers Digital Entertainment Logistics Mobile Browser ENABLING 
 SERVICES Location-Based Social Media Weibo ® $

Slide 3

Slide 3 text

Perception Affective Computing Emotion & Affection Decision making Consumer Psychology Cognitive Science Behavioral Economics Server/ Chip CDN Quantum Computing Database/ Storage AI Algorithm Big Data Virtual/ Scheduling Natural Human-Computer Interaction Lab

Slide 4

Slide 4 text

From machine language to human natural language CUI Command line GUI Interaction design UX Design Visual Design Creative Design User Research NUI Graphic User Interface Natural User Interface Voice Interaction Design AI Design Space Design Emotional Computing Tangible Interaction Design

Slide 5

Slide 5 text

USER 170 Million Keyboard 700 Million Touch Screen 1.4 Billion Natural HCI

Slide 6

Slide 6 text

HUMAN Input Output COMPUTER

Slide 7

Slide 7 text

Vision Through sense digitization and AI, enhance human capability to interact with machine in natural ways

Slide 8

Slide 8 text

User Impact Scalability Visual, Voice, Haptics, and Olfactory interaction technologies, affective computing, human-field interaction, AI design Research Topics Project Selection Criteria

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

AI Fonts and Alibaba Wood Refinity Smart Touch Olfactory Interaction

Slide 11

Slide 11 text

6763 characters 26 letters

Slide 12

Slide 12 text

Knowledge Map of fonts based on trend and style Training Optimization Optimization Training Optimization Representative Character Design Saved over 50% of the time 6763 Characters Font Style Selection Font Generation through ML QA & Launch Requirement Analysis AI Font Generation

Slide 13

Slide 13 text

Alibaba Wood - AI Short Video Pictures and Text Pictures and Videos Video Auto Clipping

Slide 14

Slide 14 text

Technology Used in Alibaba Wood

Slide 15

Slide 15 text

Application Scenarios Product Details Instagram AD Social Media Postings Video Site Content Social Media AD Short Video News

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

Short Video Generation from Product Live Show P r o d u c t E x p o s u r e I n c r e a s e d 2 4 % 1 2 m i l l i o n s h o r t v i d e o s w i l l b e g e n e r a t e d f o r 1 6 8 8 . c o m

Slide 19

Slide 19 text

• Interactive Experience • Create Infinite Possibilities Beyond Virtual Reality • Multi-sensory + Natural Human Computer Interaction REFINITY Infinite Reality

Slide 20

Slide 20 text

REFINITY： Multimodal interaction engine Interaction: Mid Air Mixed Reality Display: Naked eye 3D (partner) Technology Core: Body Gesture Haptic Smell + + Refinity provides a cross-platform naked-eye MR interaction engine that combines five senses for different applications

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

Digital Factory Interactive IOT

Slide 23

Slide 23 text

Voice interaction order machine - COSTA

Slide 24

Slide 24 text

Voice interaction order machine - KFC

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

Smell Human Limited to individual perception Visualization Smell Generation Detection Digitization Natural New Tech Machine Olfactory based search Health detection Olfactory AR Quality control OLFACTORY INTERACTION

Slide 27

Slide 27 text

SCENT DIGITALIZATION Convert scent to digital information to be perceived, transmitted, received and played over the Internet Scent Data Collection Process Capture Odor Data by the Device Transform the Digital Data into the Smell Label Build the Scent Database

Slide 28

Slide 28 text

SCENT DIGITALIZATION Similar Scent Product Recommendations Scent Data Collection Recommend Products with Similar Smell Recommend Products with Differentiated Smell Smell Label

Slide 29

Slide 29 text

Multimodal Design For Visually Impaired Inclusive Design

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

Challenges Missing Content Difficult to Click Learning Curve

Slide 32

Slide 32 text

Solutions Smart UI Proxy Multimodal Output Different Apps

Slide 33

Slide 33 text

Smart UI Proxy Read UI Back to Homepage Stop UI Reading Dynamic Buttons Function Add to Shopping Cart Add to Favorite Read Product Details

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

Total User Experience 手机上市时间：2018年4月 机型上安装为例 手机外观尺寸 贴膜器在 SmartFit贴合手机 正视图 SmartFit贴合手机 右视图 SmartFit贴合手机 左视图 SmartFit 结构爆炸示意图 安装完成，下载并启动软件 进行适配 定位适配原理：分别 触摸不同位置触觉膜， 软件会根据分辨率等 因素计算出位置坐标。 进行适配后将出现灰色悬浮窗 用户在触觉膜上的交互行为 通过手机屏幕上的悬浮窗感应接收 使用 SmartFit 固定贴合 好触觉膜。 即可运行软件，软件会指 引视障用户，使触觉膜与 智能手机操作适配。 P20 149.1mm*70.8mm*7.65mm HUAWEI P20 下 压 固 定 SmartFit贴合手机 后视图 分体式贴膜器有两个部件，分别固定于手机底部两侧。 Smart Fit “盲键”贴膜器包装造型方案一：分体式 SmartFit贴合手机 上视图 SmartFit贴合手机 下视图 A B C 可按压触发模块 开槽设计，待触觉膜 固定屏幕上，方便抽出 触觉反馈膜 根据工艺可采用 背胶纸或无背胶纸

Slide 38

Slide 38 text

Future of Experience Design: Natural HCI（CHI 2018） Natural UI is a human computer interaction system, which can leverage human’s natural perception, attention, cognition, and emotion capabilities • 3D Visual • Gesture • Natual language processing and understanding • More natural language express • Multi-model emotional computing • Multi-model interaction • Machine self-learning capability • Ability to handle ambiguity • Understand Individual Difference • Adaptivity • Ambiguity handling • Learning Capability Characteristics Technology Needed

Slide 39

Slide 39 text

Alibaba International User Experience Head DAMO Natural HCI Lab Sr. Director Paul Fu Ph.D. AI Design and the Future of Experience Design