Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Analysis and Validation - Lecture 4 - Information Visualisation (4019538FNR)

Analysis and Validation - Lecture 4 - Information Visualisation (4019538FNR)

This lecture forms part of the course Information Visualisation given at the Vrije Universiteit Brussel.

Beat Signer
PRO

March 10, 2022
Tweet

More Decks by Beat Signer

Other Decks in Education

Transcript

  1. 2 December 2005 Information Visualisation Analysis and Validation Prof. Beat

    Signer Department of Computer Science Vrije Universiteit Brussel beatsigner.com
  2. Beat Signer - Department of Computer Science - [email protected] 2

    March 10, 2022 Four Nested Levels of Vis Design
  3. Beat Signer - Department of Computer Science - [email protected] 3

    March 10, 2022 Validation ▪ Huge vis design space and most designs are ineffective ▪ Validate choices right from the beginning of the design process ▪ top-down design (problem-driven work) - start at top situation domain level - major challenge at data/task abstraction; mainly use existing idioms ▪ bottom up design (technique-driven work) - invention of new idioms or algorithms ▪ Validate all four levels of the design ▪ domain validation ▪ abstraction validation (what and why) ▪ idiom validation (how) ▪ algorithm validation
  4. Beat Signer - Department of Computer Science - [email protected] 4

    March 10, 2022 Four Nested Levels of Vis Design ▪ Output from upstream level is input to downstream level ▪ errors at upstream levels propagate to downstream levels ▪ highly iterative design process
  5. Beat Signer - Department of Computer Science - [email protected] 5

    March 10, 2022 Domain Situation ▪ A domain situation is defined by ▪ target users ▪ domain of interest of target users - each domain might have its own vocabulary ▪ data of target users ▪ questions (tasks) of target users ▪ Outcome of design process ▪ understanding of user needs (user-centred design) - e.g. via observations or interviews ▪ Challenges and risks ▪ users can often not clearly specify their analysis needs ▪ designers make assumptions (rather than engaging with users)
  6. Beat Signer - Department of Computer Science - [email protected] 6

    March 10, 2022 Data & Task Abstraction ▪ Abstract from answers to domain-specific questions at upstream to a generic representation ▪ questions from different domain situations can map to same abstract vis tasks - e.g. browsing, comparing or summarising ▪ Design abstract data ▪ data from upstream is often transformed to something different ▪ determine which data type supports a visual representation that addresses a user's problem
  7. Beat Signer - Department of Computer Science - [email protected] 7

    March 10, 2022 Visual Encoding & Interaction Idiom ▪ Specific way (idiom) to create and manipulate the visual representation of abstract data ▪ visual encoding idiom - create a "picture" out of the data (what do users see?) ▪ interaction idiom - how do users change what they see? ▪ Design space of the combination of visual encoding and interaction idioms is very large ▪ data and task abstractions help to reduce the number of potential visual encoding and interaction idioms ▪ decision about good or bad matches based on human abilities (visual perception and memory)
  8. Beat Signer - Department of Computer Science - [email protected] 8

    March 10, 2022 Word Tree Example
  9. Beat Signer - Department of Computer Science - [email protected] 9

    March 10, 2022 Algorithm ▪ Implementation of visual encoding and interaction idioms ▪ can design different algorithms to realise the same idiom ▪ Various factors might impact the choice of a specific algorithm ▪ computational complexity (performance) ▪ memory usage ▪ level of match with visual encoding idiom ▪ Separate algorithm design (computational issues) from idiom design (human perception issues)
  10. Beat Signer - Department of Computer Science - [email protected] 10

    March 10, 2022 Threats to Validity ▪ Each design level has their own threats to validity ▪ wrong problem, wrong abstraction, wrong idiom or wrong algorithm
  11. Beat Signer - Department of Computer Science - [email protected] 11

    March 10, 2022 Validation Approaches
  12. Beat Signer - Department of Computer Science - [email protected] 12

    March 10, 2022 Validation Approaches … ▪ Can perform an immediate or downstream validation ▪ downstream dependencies add to the difficulty of validation - e.g. poor algorithm design may have a negative effect when validating an interaction technique ▪ use of mock-ups for early downstream evaluation ▪ Mismatches ▪ mismatch between the level at which the benefit is claimed and the chosen validation methodology - e.g. benefit of new visual encoding idiom cannot be validated by measuring the performance of the algorithm used downstream ▪ carefully select the subset of validation methods matching the levels of design where contributions are claimed
  13. Beat Signer - Department of Computer Science - [email protected] 13

    March 10, 2022 Domain Validation ▪ A field study can help to validate that we are going to address real user needs ▪ observe people in real-world settings ▪ semi-structured interviews (e.g. contextual inquiry) ▪ Downstream validation can for example investigate a solution's adoption rate by the target audience ▪ see what target users do (without brining them into a lab)
  14. Beat Signer - Department of Computer Science - [email protected] 14

    March 10, 2022 Abstraction Validation ▪ Identified task abstraction and data abstraction might not solve the target audience's problems ▪ Downstream validation includes testing the solution with members of the target audience ▪ anecdotal (qualitative) feedback whether the tool is useful ▪ field study to observe and document how the target audience uses the tool in their real-world workflow - observe changes in behaviour rather than documenting existing work practices
  15. Beat Signer - Department of Computer Science - [email protected] 15

    March 10, 2022 Idiom Validation ▪ Justify the design of the idiom with respect to known perceptual and cognitive principles ▪ Heuristic evaluation or expert reviews may be used to ensure that no known guidelines are violated ▪ Downstream validation ▪ controlled experiments in a lab setting (lab study) - controlled experiments for testing the performance of specific idioms - measure time and errors for given tasks ▪ presentation and qualitative discussion of results - show images or videos of the solution to the target audience ▪ quantitative measurement of resulting visualisations (quality metrics) such as the number of edge crossings for node-link graph ▪ usability studies
  16. Beat Signer - Department of Computer Science - [email protected] 16

    March 10, 2022 Algorithm Validation ▪ Analyse computational complexity of algorithms ▪ number of items in the dataset, number of pixels, … ▪ Downstream validation ▪ execution time ▪ memory consumption ▪ scalability ▪ Correctness of algorithm ▪ does implementation meet the idiom specification ▪ Standard benchmarks might help to compare algorithms
  17. Beat Signer - Department of Computer Science - [email protected] 17

    March 10, 2022 MatrixExplorer
  18. Beat Signer - Department of Computer Science - [email protected] 18

    March 10, 2022 MatrixExplorer Validation Methods
  19. Beat Signer - Department of Computer Science - [email protected] 19

    March 10, 2022 Genealogical Graphs
  20. Beat Signer - Department of Computer Science - [email protected] 20

    March 10, 2022 Genealogical Graphs Validation Methods
  21. Beat Signer - Department of Computer Science - [email protected] 21

    March 10, 2022 Flow Maps Migration from California Top ten states that sent migrants to California (green) and to New York (blue)
  22. Beat Signer - Department of Computer Science - [email protected] 22

    March 10, 2022 Flow Maps Validation Methods
  23. Beat Signer - Department of Computer Science - [email protected] 23

    March 10, 2022 LiveRAC
  24. Beat Signer - Department of Computer Science - [email protected] 24

    March 10, 2022 LiveRAC Validation Methods
  25. Beat Signer - Department of Computer Science - [email protected] 26

    March 10, 2022 Sizing the Horizon Validation Methods
  26. Beat Signer - Department of Computer Science - [email protected] 27

    March 10, 2022 Exercise 4 ▪ Analysis and Validation
  27. Beat Signer - Department of Computer Science - [email protected] 28

    March 10, 2022 Further Reading ▪ This lecture is mainly based on the book Visualization Analysis & Design ▪ chapter 4 - Analysis: Four Levels for Validation
  28. Beat Signer - Department of Computer Science - [email protected] 29

    March 10, 2022 References ▪ Visualization Analysis & Design, Tamara Munzner, Taylor & Francis Inc, (Har/Psc edition), May, November 2014, ISBN-13: 978-1466508910
  29. 2 December 2005 Next Lecture Data Presentation