Making Sense of (Big) Data with Visual Analytics

An overview of the research by Kai Xu on making sense of data with visual analytics

Transcript

1. Making Sense of (Big) Data 

   with Visual Analytics
   Dr Kai Xu
   
   Associate Professor in Data Analytics
   
   Middlesex University, London, UK
   
   k.xu@mdx.ac.uk
   

   View Slide

2. https://kaixu.me
   

   View Slide

3. http://vis4sense.github.io/
   

   View Slide

4. Outline
   • What is Sensemaking
   
   • Why do we need Visual Analytics
   
   • Demo – SAVI: Social Analytics Visualisation
   
   • Demo – SenseMap: A ‘Map’ for Sensemaking
   

   View Slide

5. What is Sensemaking?
   • Making sense of data
   
   • Collecting, understanding, analysing, reasoning, and
   making decisions
   
   • It is something we do everyday:
   
   • Plan a holiday, buy a house, understand an illness, …
   
   • Defence, policing, investment, medical diagnosis, …
   
   • Scientific research (from conception to paper)
   
   • How is it different from data analysis?
   
   • While the goal may be clear, how to get there is often
   not
   
   • Exploratory and takes a long time
   

   View Slide

6. Example: what is the best camera for about £500?
   What is the best
   camera for £500?
   Pixel number
   Sensor size
   Image quality
   chromatic
   aberration?!
   Noise reduction
   What does
   experts say?
   Online reviews
   What does my
   friend say?
   Smart phone
   Compact
   Full frame?
   Micro 4/3?
   Sony RX100
   Nikon D750
   Samsung
   Galaxy S7
   What are the
   price? How do I
   compare?
   Panasonic 

   LX100
   Form factor
   Models
   Camera Lens
   Aperture
   

   View Slide

7. An Hour Later …
   • What is relevant and what is not?
   
   • Where is the information about ‘chromatic
   aberration’?
   
   • What are the factors important to image quality?
   
   • How to compare the models?
   
   • Where did I left off two days ago?
   
   • How do I explain to my wife?
   

   View Slide

8. Not just in browser
   

   View Slide

9. Sensemaking Model
   All the information
   about camera
   The information
   relevant to my needs
   Important
   photography
   Concepts
   A understanding
   of how camera
   works
   Candidate
   models
   Decision and
   convince
   others
   

   View Slide

10. Making Sense of (Big) Data
    Lots of tools 

    and support
    Very few tools 

    and support
    

    View Slide

11. Why AlphaGo alone can’t do it
    • The Go game is very complex and difficult, but
    
    • The goal and rules are very well defined, and the
    results are easily measurable
    
    • However, the £500 camera task is ill defined and
    not easily measurable
    
    • How many people have the knowledge and
    resource to build a deep neural network, collect
    all the training data, and then train and tune it,
    just to find a camera?
    

    View Slide

12. Who is the best chess player in the world?
    • Deep Blue, was in 1997
    
    • Currently, probably a human-machine
    team
    
    • And the two people on the team are
    not even professional chess players
    
    • The power of integrating the
    complementary strength of human and
    machine
    

    View Slide

13. Data Visualisation – 

    Human Cognition for Pattern Discovery
    Confirm the expected and discover the unexpected
    

    View Slide

14. Visual Analytics = Human + Artificial Intelligence
    Visualisation
    Data 

    Analysis
    Interaction
    Information Retrieval
    Machine Learning
    Data Mining
    Information Visualisation
    Scientific Visualisation
    Computer Graphics
    Human-Computer Interaction
    Cognitive Psychology
    Perception
    

    View Slide

15. Some work in the last five years
    

    View Slide

16. Example - SAVI: Social Analytics Visualisation
    • IEEE Visual Analytics Science & Technology (VAST)
    Challenge
    
    • Provide dataset and analysis tasks
    
    • Entry: visual analytics systems
    
    • Leading research groups and companies
    
    • VAST Challenge 2014 – Mini Challenge 3
    
    • Data: tweets
    
    • Task: detect and describe a crime
    

    View Slide

17. The Data
    

    View Slide

18. SAVI: Social Analytics Visualisation
    

    View Slide

19. Map Visualisation and Sensemaking Support
    

    View Slide

20. The Final Findings
    • Still a long time before AI can do such
    sensemaking
    
    • Difficult for human, too: almost impossible without
    the tool
    
    • Human leads, the tool supports
    
    • The tool does not provide answer,
    
    • Reveal pattern, help with organisation and
    reasoning, and many more
    
    • Limited sensemaking support
    

    View Slide

21. A ‘Map’ of Sensemaking
    • Sensemaking is kind of like exploring a maze …
    
    • What may be helpful is something like this …
    

    View Slide

22. SenseMap – A ‘Map’ for Online Sensemaking
    Browser enhancement
    History 

    Map
    Knowledge 

    Map
    

    View Slide

23. Comparison: Before and After
    

    View Slide

24. There is much to do …
    • Mostly for the lower stages of
    sensemaking so far
    
    • How (machine learning) algorithm can
    help
    
    • Understand sensemaking actions from
    the relatively lower level data
    (provenance)
    
    • And then provide better support
    
    • A ‘coverage map’ for data and solution
    space.
    

    View Slide

25. Takeaway Messages
    • Sensemaking is how people understand, reason, and make decisions
    with data
    
    • It is important to Big Data, but there is limited support available
    
    • Visual Analytics combines data visualisation with analytics
    
    • A promising approach for sensemaking support
    
    More details on github.com: http://vis4sense.github.io/
    

    View Slide