Making Sense of (Big) Data with Visual Analytics

Combine the power of computational analysis with human intelligence through interactive data visualisation to tackle the most difficult data challenges.

Transcript

1. Making Sense of (Big) Data with Visual Analytics Dr Kai Xu Associate Professor in Data Analytics Middlesex University, London, UK k.xu@mdx.ac.uk https://kaixu.me
2. Last 20 years Bachelor, Shanghai Jiao Tong PhD, University of Queensland Postdoc, University of Sydney Research Scientist, CSIRO, Hobart Bioinformatics Specialist, ANU, Canberra A/Prof, Middlesex University London
3. Outline • What is Sensemaking • Why do we need Visual Analytics • Demo – SAVI: Social Analytics Visualisation • Demo – SenseMap: A ‘Map’ for Sensemaking
4. What is Sensemaking? • Making sense of data • Collecting, understanding, analysing, reasoning, and making decisions • It is something we do everyday: • Plan a holiday, buy a house, understand an illness, … • Defence, policing, investment, medical diagnosis, … • Scientific research (from conception to paper) • How is it different from data analysis? • Usually not well defined • Exploratory and takes a long time • Human led
5. Example: what is the best camera for about £500? What is the best camera for £500? Pixel number Sensor size Image quality chromatic aberration?! Noise reduction What does experts say? Online reviews What does my friend say? Smart phone Compact Full frame? Micro 4/3? Sony RX100 Nikon D750 Samsung Galaxy S7 What are the price? How do I compare? Panasonic LX100 Form factor Models Camera Lens Aperture
6. This is usually what it looks like after one hour • What is relevant and what is not? • Where is the information about ‘chromatic aberration’? • What are the factors important to image quality? • How to compare the models? • Where did I left off two days ago? • How do I explain to my wife?
7. Not just in browser
8. Sensemaking Model All the existing literature Related work Important Concepts A understanding of the research problem and potential techniques Different solutions Write and present the paper
9. Making Sense of (Big) Data
10. Why AlphaGo alone can’t do it • The Go game is very complex and difficult, but • The goal and rules are very well defined, and the results are easily measurable • However, the £500 camera task is ill defined and not easily measurable • How many people have the knowledge and resource to build a deep neural network, collect all the training data, and then train and tune it, just to find a camera?
11. Who is the best chess player in the world? • Deep Blue, was in 1997 • Currently, probably a human-machine team • And the two people on the team are not even professional chess players • The power of integrating the complementary strength of human and machine
12. Visual Analytics = Human + Computing Intelligence Visualisation Data Analysis Interaction Information Retrieval Machine Learning Data Mining Information Visualisation Scientific Visualisation Computer Graphics Human-Computer Interaction Cognitive Psychology Perception
13. Some work in the last five years
14. Example - SAVI: Social Analytics Visualisation • IEEE Visual Analytics Science & Technology (VAST) Challenge • Provide dataset and analysis tasks • Entry: visual analytics systems • Leading research groups and companies • VAST Challenge 2014 – Mini Challenge 3 • Data: tweets • Task: detect and describe a crime
15. The Data
16. SAVI: Social Analytics Visualisation
17. Map Visualisation and Sensemaking Support
18. The Final Findings • Still a long time before AI can do such sensemaking • Difficult for human, too: almost impossible without the tool • Human leads, the tool supports • The tool does not provide answer, • Reveal pattern, help with organisation and reasoning, and many more • Limited sensemaking support
19. A ‘Map’ of Sensemaking • Sensemaking is kind of like exploring a maze … • What may be helpful is something like this …
20. SenseMap – A ‘Map’ for Online Sensemaking Browser enhancement History Map Knowledge Map
21. Comparison: Before and After
22. There is much to do … • Mostly for the lower stages of sensemaking so far • How (machine learning) algorithm can help • Understand sensemaking actions from the relatively lower level data (provenance) • And then provide better support • A ‘coverage map’ for data and solution space.
23. Takeaway Messages • Sensemaking is how people understand, reason, and make decisions with data • It is important to Big Data, but there is limited support available • Visual Analytics combines data visualisation with analytics • A promising approach for sensemaking support More details about SAVI and SenseMap: http://vis4sense.github.io/