Architecture Design
for Interactive Visualisation
Amit Kapoor
amitkaps.com
Bargava Subramanian
bargava.com
1
Slide 2
Slide 2 text
Exemplar
2
Slide 3
Slide 3 text
Layers of Abstraction
— Raw Data
— Transform Layer
— Visual Layer
— Interaction Layer
3
Slide 4
Slide 4 text
Interactive Data Visualisation0
0 Grammar of Visualisation
4
Slide 5
Slide 5 text
Architectural Design Trade-offs
1. Rendering for Data Scale
2. Computation for Interaction Speed
3. Adaptive to Data Complexity
4. Responsive to Data Velocity
5
Slide 6
Slide 6 text
1. Rendering for Data Scale
"How do you render interactive visualization when
there are millions or billions of data points?"
6
Slide 7
Slide 7 text
Visualise a Million Points
Show all the Data
Same order as the number of
pixels on my MacBook Air: 1400
x 900
Problems with overplotting
7
Slide 8
Slide 8 text
Visualise a Million Points
Sample the Data
Sampling can be effective (with
overweighting unusual values)
Require multiple plots or
careful tuning parameters
8
Slide 9
Slide 9 text
Visualise a Million Points
Model the Data
Models are great as they scale
nicely.
But, visualisation is needed
to answer the question:
“I don’t know, what I don’t
know.”
9
Slide 10
Slide 10 text
Visualise a Million Points
Bin the Data
Reduce the size of the data to
complement the pixel
resolution on screen
10
Slide 11
Slide 11 text
“Visualising data at scale
is the process of creating
generalized histograms”
11
Slide 12
Slide 12 text
Layers of Visualisation
12
Slide 13
Slide 13 text
Rendering: How many data points?
— SVG: ~10^3
— Canvas: ~10^4
— Web.gl: ~10^6
13
Slide 14
Slide 14 text
Web.gl based Rendering: Deck.gl1
1 UK Road Accident Data using deck.gl
14
Slide 15
Slide 15 text
Bin and Web.gl Rendering
15
Slide 16
Slide 16 text
Visualise more than Million Points
Bin-Summarize-Smooth
Shift the transfer of data from raw data to binned-
summarised-smoothed data
16
Slide 17
Slide 17 text
BigVis: Bin-Summarize-Smooth-Visualise
17
Slide 18
Slide 18 text
Why not sent an image of the Data: DataShader3
3 DataShader: Turn largest data to image, accurately
18
Slide 19
Slide 19 text
DataShader: Encode-Bin-Summarise-ColorMap
19
Slide 20
Slide 20 text
2. Computation for Interaction Speed
"How do you reduce the latency of the query at the
interaction layer, so that the user can interact
with the visualisation?"
20
Approximate Query Processing e.g. VerdictDB5
5 Speed Comparison: AQP is 50x+ faster
23
Slide 24
Slide 24 text
Approximate Query Processing
24
Slide 25
Slide 25 text
Challenges in Visualise Uncertainty 6
6
25
Slide 26
Slide 26 text
Faster Databases e.g. MapD7
7 TweetMap Demo by MapD
26
Slide 27
Slide 27 text
Faster Databases
27
Slide 28
Slide 28 text
3. Adaptive to Data Complexity
Choosing a good visualisation design for a singular
dataset is possible after a few experiments and
iteration.
But how do you ensure that the visualisation will
adapt to the variety, volume and edge cases in the
real data?
28
Slide 29
Slide 29 text
Responsive Visualisation to Space & Data8
8 Responsive Data Visualisation - Nick Rabinowitz
29
Slide 30
Slide 30 text
Handling High Cardinality e.g. Facet-Dive9
9 FacetDive from PAIR Research
30
Slide 31
Slide 31 text
Handling High Cardinality e.g. DataComb10
10 Datacomb: Interactive Table Plot
31
Slide 32
Slide 32 text
Dimension Reduction e.g. Embedding Projector11
11 Embedding Projector
32
Slide 33
Slide 33 text
4. Responsive to Data Velocity
"How do you trade-offs between real-time vs. near
real-time data and its impact on refreshing
visualization"
33
Slide 34
Slide 34 text
Optimizing for near real-time visual refreshes
34
Slide 35
Slide 35 text
Architectural Design Trade-offs
1. Rendering for Data Scale
2. Computation for Interaction Speed
3. Adaptive to Data Complexity
4. Responsive to Data Velocity
35
Slide 36
Slide 36 text
Architecture Design
for Interactive Visualisation
Amit Kapoor
amitkaps.com
Bargava Subramanian
bargava.com
36