$30 off During Our Annual Pro Sale. View Details »

Making Sense of (Big) Data with Visual Analytics

Kai
July 18, 2017

Making Sense of (Big) Data with Visual Analytics

An overview of the research by Kai Xu on making sense of data with visual analytics

Kai

July 18, 2017
Tweet

Other Decks in Technology

Transcript

  1. Making Sense of (Big) Data 

    with Visual Analytics
    Dr Kai Xu

    Associate Professor in Data Analytics

    Middlesex University, London, UK

    [email protected]

    View Slide

  2. https://kaixu.me

    View Slide

  3. http://vis4sense.github.io/

    View Slide

  4. Outline
    • What is Sensemaking

    • Why do we need Visual Analytics

    • Demo – SAVI: Social Analytics Visualisation

    • Demo – SenseMap: A ‘Map’ for Sensemaking

    View Slide

  5. What is Sensemaking?
    • Making sense of data

    • Collecting, understanding, analysing, reasoning, and
    making decisions

    • It is something we do everyday:

    • Plan a holiday, buy a house, understand an illness, …

    • Defence, policing, investment, medical diagnosis, …

    • Scientific research (from conception to paper)

    • How is it different from data analysis?

    • While the goal may be clear, how to get there is often
    not

    • Exploratory and takes a long time

    View Slide

  6. Example: what is the best camera for about £500?
    What is the best
    camera for £500?
    Pixel number
    Sensor size
    Image quality
    chromatic
    aberration?!
    Noise reduction
    What does
    experts say?
    Online reviews
    What does my
    friend say?
    Smart phone
    Compact
    Full frame?
    Micro 4/3?
    Sony RX100
    Nikon D750
    Samsung
    Galaxy S7
    What are the
    price? How do I
    compare?
    Panasonic 

    LX100
    Form factor
    Models
    Camera Lens
    Aperture

    View Slide

  7. An Hour Later …
    • What is relevant and what is not?

    • Where is the information about ‘chromatic
    aberration’?

    • What are the factors important to image quality?

    • How to compare the models?

    • Where did I left off two days ago?

    • How do I explain to my wife?

    View Slide

  8. Not just in browser

    View Slide

  9. Sensemaking Model
    All the information
    about camera
    The information
    relevant to my needs
    Important
    photography
    Concepts
    A understanding
    of how camera
    works
    Candidate
    models
    Decision and
    convince
    others

    View Slide

  10. Making Sense of (Big) Data
    Lots of tools 

    and support
    Very few tools 

    and support

    View Slide

  11. Why AlphaGo alone can’t do it
    • The Go game is very complex and difficult, but

    • The goal and rules are very well defined, and the
    results are easily measurable

    • However, the £500 camera task is ill defined and
    not easily measurable

    • How many people have the knowledge and
    resource to build a deep neural network, collect
    all the training data, and then train and tune it,
    just to find a camera?

    View Slide

  12. Who is the best chess player in the world?
    • Deep Blue, was in 1997

    • Currently, probably a human-machine
    team

    • And the two people on the team are
    not even professional chess players

    • The power of integrating the
    complementary strength of human and
    machine

    View Slide

  13. Data Visualisation – 

    Human Cognition for Pattern Discovery
    Confirm the expected and discover the unexpected

    View Slide

  14. Visual Analytics = Human + Artificial Intelligence
    Visualisation
    Data 

    Analysis
    Interaction
    Information Retrieval
    Machine Learning
    Data Mining
    Information Visualisation
    Scientific Visualisation
    Computer Graphics
    Human-Computer Interaction
    Cognitive Psychology
    Perception

    View Slide

  15. Some work in the last five years

    View Slide

  16. Example - SAVI: Social Analytics Visualisation
    • IEEE Visual Analytics Science & Technology (VAST)
    Challenge

    • Provide dataset and analysis tasks

    • Entry: visual analytics systems

    • Leading research groups and companies

    • VAST Challenge 2014 – Mini Challenge 3

    • Data: tweets

    • Task: detect and describe a crime

    View Slide

  17. The Data

    View Slide

  18. SAVI: Social Analytics Visualisation

    View Slide

  19. Map Visualisation and Sensemaking Support

    View Slide

  20. The Final Findings
    • Still a long time before AI can do such
    sensemaking

    • Difficult for human, too: almost impossible without
    the tool

    • Human leads, the tool supports

    • The tool does not provide answer,

    • Reveal pattern, help with organisation and
    reasoning, and many more

    • Limited sensemaking support

    View Slide

  21. A ‘Map’ of Sensemaking
    • Sensemaking is kind of like exploring a maze …

    • What may be helpful is something like this …

    View Slide

  22. SenseMap – A ‘Map’ for Online Sensemaking
    Browser enhancement
    History 

    Map
    Knowledge 

    Map

    View Slide

  23. Comparison: Before and After

    View Slide

  24. There is much to do …
    • Mostly for the lower stages of
    sensemaking so far

    • How (machine learning) algorithm can
    help

    • Understand sensemaking actions from
    the relatively lower level data
    (provenance)

    • And then provide better support

    • A ‘coverage map’ for data and solution
    space.

    View Slide

  25. Takeaway Messages
    • Sensemaking is how people understand, reason, and make decisions
    with data

    • It is important to Big Data, but there is limited support available

    • Visual Analytics combines data visualisation with analytics

    • A promising approach for sensemaking support

    More details on github.com: http://vis4sense.github.io/

    View Slide