Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Familiarisation (IUI 2018, Tokyo, Japan)

Familiarisation (IUI 2018, Tokyo, Japan)

Slides from the talk on "Familiarisation: Restructuring Layouts with Visual Learning Models" given at ACM IUI 2018, in Tokyo, Japan.

Fb45b475e4810a23707a7d1c68bd6334?s=128

Kashyap Todi

March 16, 2018
Tweet

Other Decks in Research

Transcript

  1. Restructuring Visual Layouts with Learning Kashyap Todi Jussi Jokinen Kris

    Luyten Antti Oulasvirta Familiarisation Models
  2. Familiariser New/Unvisited Page Familiarised Page

  3. page designs can be quite diverse

  4. Frequently visited pages

  5. New/Unvisited Frequently visited pages

  6. None
  7. increased visual search time reduces performace/usability

  8. New/Unvisited Page Familiarised Page Familiariser

  9. Familiariser User History History Logging

  10. Visit Page User History History Logging Familiariser

  11. User History Familiariser History Logging

  12. User History Visit Page Familiariser History Logging

  13. User History Familiariser History Logging

  14. User History Visit Page Familiariser History Logging

  15. Familiarity Models x y Time Frequency Page Frequency I. Frequency

    II. Serial-Position Curve III. Visual Statistical Learning IV. Cognitive Model History Logging Familiarity Modelling Familiariser
  16. History Logging Familiarity Modelling FAMILIAR DESIGN Familiariser

  17. New/Unvisited Page History Logging Familiarity Modelling FAMILIAR DESIGN Familiariser

  18. New/Unvisited Page Familiarised Page History Logging Familiarity Modelling Page Restructuring

    FAMILIAR DESIGN Familiariser
  19. Layout Generation Run-time Adaptations

  20. Layout Generation DesignScape [CHI ’15] ….. Sketchplore [DIS ’17] and

    many more generate/optimise designs for the general population
  21. Run-time Adaptations Split / Moving / Visual Popout Interface [CHI

    ’11] Walking User Interface [TACCESS ’11] ….. Actively adapt known interfaces as they are being used
  22. Familiarisation models per-user familiarity to generate a familiar design, and

    uses this to adapt unvisited designs by restructuring
  23. Familiarisation History Logging Familiarity Modelling Page Restructuring

  24. History Logging Parse for Features

  25. History Logging User History • Order of page visits •

    Parsed page layouts • Visit frequency, duration, etc
  26. History Logging Familiarity Modelling Page Restructuring Familiarisation

  27. Familiarity Modelling I. Frequency-Based II. Serial-Position Curve III. Visual Statistical

    Learning IV. Cognitive Model
  28. I. FREQUENCY-BASED Page Frequency

  29. I. FREQUENCY-BASED Page with highest visit frequency selected fpage =

    nvisits ∗ taverage
  30. II. SERIAL-POSITION CURVE Time Frequency

  31. II. SERIAL-POSITION CURVE Page selected based on function of long-term

    recall Frequency Recency Primacy npage / ntotal 1 – rpage / rtotal 1−((ppage −1)/npages)
  32. III. VISUAL-STATISTICAL LEARNIng x y

  33. III. VISUAL-STATISTICAL LEARNIng Page generated based on feature distributions Logo

    Search Social Menu
  34. IV. COGNITIVE MODEL

  35. IV. COGNITIVE MODEL Page generated based on cognitive model of

    visual learning
  36. Familiarity Modelling FAMILIAR DESIGN As a result, a familiar design

    is selected/generated
  37. Familiarity Modelling Logo As a result, a familiar design is

    selected/generated Login Cart Search Newsletter Social Content
  38. History Logging Familiarity Modelling Page Restructuring Familiarisation

  39. Page Restructuring Bricolage [CHI ’11] — Requires element size information

    — Hard to deal with feature mismatches — Content might be cropped due to mismatch
  40. Page Restructuring FAMILIAR DESIGN ORIGINAL PAGE + FAMILIARISED PAGE —>

  41. Page Restructuring FAMILIAR DESIGN ORIGINAL PAGE Logo Login Cart Search

    Newsletter Social Content ORIGINAL PAGE FAMILIAR DESIGN
  42. Page Restructuring FAMILIAR DESIGN ORIGINAL PAGE ORIGINAL PAGE RESTRUCTURED FAMILIAR

    DESIGN Page restructured but has overlaps!
  43. Page Restructuring FAMILIAR DESIGN ORIGINAL PAGE ORIGINAL PAGE RESTRUCTURED FAMILIAR

    DESIGN Page restructured but has overlaps! Overlap Resolution LP solver with constraints to: • Eliminate overlaps • Align elements
  44. Page Restructuring FAMILIAR DESIGN ORIGINAL PAGE ORIGINAL PAGE FAMILIARISED FAMILIAR

    DESIGN
  45. An Example

  46. User History

  47. User History Unvisited Page

  48. User History Unvisited Page Model Results Frequency-Based Serial-Position Curve Visual

    Statistical Learning Cognitive Model
  49. User Study Evaluating how familiarisation impacts visual search time during

    page browsing tasks
  50. 1. Learning Phase 5 page designs 25 visits Original design

    displayed
  51. 2. Test Phase 25 page designs 100 visits Original or

    familiarised design displayed
  52. Visual search time reduced by 10% (2.8 vs 2.5 seconds)

    23% fewer eye-gaze fixations (3.4 vs 2.6 fixations) Results =16
  53. A principled approach to restructure new layouts using visual learning

    models
  54. Designer Global Consistency Population

  55. Designer Global Consistency Familiariser History Logging Familiarity Modelling Page Restructuring

    Local Consistency Population Individual
  56. Familiarisation Kashyap Todi Jussi Jokinen Kris Luyten Antti Oulasvirta Unvisited

    Familiarised History Logging Familiarity Modelling Page Restructuring II. Serial-Position Curve I. Frequency-Based III. Visual Statistical Learning IV. Cognitive Model www.kashyaptodi.com/familiarisation