Making Sense of (Big) Data with Visual Analytics

An overview of the research by Kai Xu on making sense of data with visual analytics

Transcript

1. Making Sense of (Big) Data 
 with Visual Analytics Dr Kai Xu Associate Professor in Data Analytics Middlesex University, London, UK k.xu@mdx.ac.uk
2. https://kaixu.me
3. http://vis4sense.github.io/
4. Outline • What is Sensemaking • Why do we need Visual Analytics • Demo – SAVI: Social Analytics Visualisation • Demo – SenseMap: A ‘Map’ for Sensemaking
5. What is Sensemaking? • Making sense of data • Collecting, understanding, analysing, reasoning, and making decisions • It is something we do everyday: • Plan a holiday, buy a house, understand an illness, … • Defence, policing, investment, medical diagnosis, … • Scientific research (from conception to paper) • How is it different from data analysis? • While the goal may be clear, how to get there is often not • Exploratory and takes a long time
6. Example: what is the best camera for about £500? What is the best camera for £500? Pixel number Sensor size Image quality chromatic aberration?! Noise reduction What does experts say? Online reviews What does my friend say? Smart phone Compact Full frame? Micro 4/3? Sony RX100 Nikon D750 Samsung Galaxy S7 What are the price? How do I compare? Panasonic 
 LX100 Form factor Models Camera Lens Aperture
7. An Hour Later … • What is relevant and what is not? • Where is the information about ‘chromatic aberration’? • What are the factors important to image quality? • How to compare the models? • Where did I left off two days ago? • How do I explain to my wife?
8. Not just in browser
9. Sensemaking Model All the information about camera The information relevant to my needs Important photography Concepts A understanding of how camera works Candidate models Decision and convince others
10. Making Sense of (Big) Data Lots of tools 
 and support Very few tools 
 and support
11. Why AlphaGo alone can’t do it • The Go game is very complex and difficult, but • The goal and rules are very well defined, and the results are easily measurable • However, the £500 camera task is ill defined and not easily measurable • How many people have the knowledge and resource to build a deep neural network, collect all the training data, and then train and tune it, just to find a camera?
12. Who is the best chess player in the world? • Deep Blue, was in 1997 • Currently, probably a human-machine team • And the two people on the team are not even professional chess players • The power of integrating the complementary strength of human and machine
13. Data Visualisation – 
 Human Cognition for Pattern Discovery Confirm the expected and discover the unexpected
14. Visual Analytics = Human + Artificial Intelligence Visualisation Data 
 Analysis Interaction Information Retrieval Machine Learning Data Mining Information Visualisation Scientific Visualisation Computer Graphics Human-Computer Interaction Cognitive Psychology Perception
15. Some work in the last five years
16. Example - SAVI: Social Analytics Visualisation • IEEE Visual Analytics Science & Technology (VAST) Challenge • Provide dataset and analysis tasks • Entry: visual analytics systems • Leading research groups and companies • VAST Challenge 2014 – Mini Challenge 3 • Data: tweets • Task: detect and describe a crime
17. The Data
18. SAVI: Social Analytics Visualisation
19. Map Visualisation and Sensemaking Support
20. The Final Findings • Still a long time before AI can do such sensemaking • Difficult for human, too: almost impossible without the tool • Human leads, the tool supports • The tool does not provide answer, • Reveal pattern, help with organisation and reasoning, and many more • Limited sensemaking support
21. A ‘Map’ of Sensemaking • Sensemaking is kind of like exploring a maze … • What may be helpful is something like this …
22. SenseMap – A ‘Map’ for Online Sensemaking Browser enhancement History 
 Map Knowledge 
 Map
23. Comparison: Before and After
24. There is much to do … • Mostly for the lower stages of sensemaking so far • How (machine learning) algorithm can help • Understand sensemaking actions from the relatively lower level data (provenance) • And then provide better support • A ‘coverage map’ for data and solution space.
25. Takeaway Messages • Sensemaking is how people understand, reason, and make decisions with data • It is important to Big Data, but there is limited support available • Visual Analytics combines data visualisation with analytics • A promising approach for sensemaking support More details on github.com: http://vis4sense.github.io/