Paul Fu

AI Design and the Future of Experience Design

Transcript

1. AI Design and the Future of
   Experience Design
   Alibaba International User Experience Head
   DAMO Natural HCI Lab Sr. Director
   Paul Fu Ph.D.
   

   View Slide

2. Alibaba Ecosystem
   OPERATING SYSTEM
   CLOUD COMPUTING
   DATA
   TECHNOLOGY TEAM
   Building the Infrastructure of Commerce
   Buyers
   Sellers
   MARKETPLACES
   Marketing
   Financing & Payment
   Solution
   Professional
   
   Service Providers
   Digital Entertainment
   Logistics
   Mobile Browser
   ENABLING 

   SERVICES
   Location-Based
   Social Media
   Weibo
   ®
   $
   

   View Slide

3. Perception
   Affective
   Computing
   Emotion
   &
   Affection
   Decision
   making
   Consumer
   Psychology
   Cognitive
   Science
   Behavioral
   Economics
   Server/
   Chip
   CDN
   Quantum
   Computing
   Database/
   Storage
   AI
   Algorithm
   Big Data
   Virtual/
   Scheduling
   Natural Human-Computer Interaction Lab
   

   View Slide

4. From machine language to human natural language
   CUI
   Command line
   GUI
   Interaction design
   UX Design
   Visual Design
   Creative Design
   User Research
   NUI
   Graphic User Interface Natural User Interface
   Voice Interaction Design
   AI Design
   Space Design
   Emotional Computing
   Tangible Interaction Design
   

   View Slide

5. USER 170 Million
   Keyboard
   700 Million
   Touch Screen
   1.4 Billion
   Natural HCI
   

   View Slide

6. HUMAN
   Input
   Output
   COMPUTER
   

   View Slide

7. Vision
   Through sense digitization and AI, enhance human
   capability to interact with machine in natural ways
   

   View Slide

8. User Impact Scalability
   Visual, Voice, Haptics, and Olfactory interaction technologies,
   affective computing, human-field interaction, AI design
   Research Topics
   Project Selection Criteria
   

   View Slide

9. View Slide

10. AI Fonts and
    Alibaba Wood
    Refinity Smart Touch
    Olfactory
    Interaction
    

    View Slide

11. 6763 characters
    26 letters
    

    View Slide

12. Knowledge Map of fonts
    based on trend and style
    Training
    Optimization
    Optimization
    Training
    Optimization
    Representative
    Character Design
    Saved over 50%
    of the time
    6763 Characters
    Font Style
    Selection
    Font Generation
    through ML
    QA & Launch
    Requirement
    Analysis
    AI Font Generation
    

    View Slide

13. Alibaba Wood - AI Short Video
    Pictures and Text Pictures and Videos Video Auto Clipping
    

    View Slide

14. Technology Used in Alibaba Wood
    

    View Slide

15. Application Scenarios
    Product Details
    Instagram AD
    Social Media Postings
    Video Site Content
    Social Media AD
    Short Video News
    

    View Slide

16. View Slide

17. View Slide

18. Short Video Generation from Product Live Show
    P r o d u c t
    E x p o s u r e I n c r e a s e d 2 4 %
    1 2 m i l l i o n s h o r t v i d e o s
    w i l l b e g e n e r a t e d
    f o r 1 6 8 8 . c o m
    

    View Slide

19. • Interactive Experience
    • Create Infinite Possibilities Beyond Virtual Reality
    • Multi-sensory + Natural Human Computer Interaction
    REFINITY
    Infinite Reality
    

    View Slide

20. REFINITY：
    Multimodal interaction engine
    Interaction:
    Mid Air Mixed Reality
    Display:
    Naked eye 3D (partner)
    Technology Core:
    Body Gesture Haptic Smell
    + +
    Refinity provides a cross-platform naked-eye MR interaction engine
    that combines five senses for different applications
    

    View Slide

21. View Slide

22. Digital Factory Interactive IOT
    

    View Slide

23. Voice interaction order machine - COSTA
    

    View Slide

24. Voice interaction order machine - KFC
    

    View Slide

25. View Slide

26. Smell
    Human
    Limited to individual
    perception
    Visualization
    Smell
    Generation
    Detection Digitization
    Natural New Tech
    Machine
    Olfactory based search
    Health detection
    Olfactory AR
    Quality control
    OLFACTORY INTERACTION
    

    View Slide

27. SCENT DIGITALIZATION
    Convert scent to digital information to be perceived, transmitted, received and played over the Internet
    Scent Data Collection Process
    Capture Odor Data
    by the Device
    Transform the
    Digital Data into
    the Smell Label
    Build the Scent Database
    

    View Slide

28. SCENT DIGITALIZATION
    Similar Scent Product Recommendations
    Scent Data
    Collection
    Recommend Products with
    Similar Smell
    Recommend Products with
    Differentiated Smell
    Smell Label
    

    View Slide

29. Multimodal Design For Visually Impaired
    Inclusive Design
    

    View Slide

30. View Slide

31. Challenges
    Missing
    Content
    Difficult
    to Click
    Learning
    Curve
    

    View Slide

32. Solutions
    Smart UI Proxy
    Multimodal Output
    Different Apps
    

    View Slide

33. Smart UI Proxy
    Read UI
    Back to Homepage
    Stop UI Reading
    Dynamic
    Buttons
    Function
    Add to Shopping Cart
    Add to Favorite
    Read Product Details
    

    View Slide

34. View Slide

35. View Slide

36. View Slide

37. Total User Experience
    手机上市时间：2018年4月
    机型上安装为例 手机外观尺寸
    贴膜器在
    SmartFit贴合手机
    正视图
    SmartFit贴合手机
    右视图
    SmartFit贴合手机
    左视图
    SmartFit 结构爆炸示意图
    安装完成，下载并启动软件
    进行适配
    定位适配原理：分别
    触摸不同位置触觉膜，
    软件会根据分辨率等
    因素计算出位置坐标。
    进行适配后将出现灰色悬浮窗
    用户在触觉膜上的交互行为
    通过手机屏幕上的悬浮窗感应接收
    使用 SmartFit 固定贴合
    好触觉膜。
    即可运行软件，软件会指
    引视障用户，使触觉膜与
    智能手机操作适配。
    P20
    149.1mm*70.8mm*7.65mm
    HUAWEI P20
    下
    压
    固
    定
    SmartFit贴合手机
    后视图
    分体式贴膜器有两个部件，分别固定于手机底部两侧。
    Smart Fit “盲键”贴膜器包装造型方案一：分体式
    SmartFit贴合手机
    上视图
    SmartFit贴合手机
    下视图
    A B
    C
    可按压触发模块
    开槽设计，待触觉膜
    固定屏幕上，方便抽出
    触觉反馈膜
    根据工艺可采用
    背胶纸或无背胶纸
    

    View Slide

38. Future of Experience Design:
    Natural HCI（CHI 2018）
    Natural UI is a human computer interaction system, which can leverage human’s
    natural perception, attention, cognition, and emotion capabilities
    • 3D Visual
    • Gesture
    • Natual language processing and
    understanding
    • More natural language express
    • Multi-model emotional computing
    • Multi-model interaction
    • Machine self-learning capability
    • Ability to handle ambiguity
    • Understand Individual Difference
    • Adaptivity
    • Ambiguity handling
    • Learning Capability
    Characteristics Technology Needed
    

    View Slide

39. Alibaba International User Experience Head
    DAMO Natural HCI Lab Sr. Director
    Paul Fu Ph.D.
    AI Design and the Future of
    Experience Design
    

    View Slide