Slide 1

Slide 1 text

Architecture Design for Interactive Visualisation Amit Kapoor amitkaps.com Bargava Subramanian bargava.com 1

Slide 2

Slide 2 text

Exemplar 2

Slide 3

Slide 3 text

Layers of Abstraction — Raw Data — Transform Layer — Visual Layer — Interaction Layer 3

Slide 4

Slide 4 text

Interactive Data Visualisation0 0 Grammar of Visualisation 4

Slide 5

Slide 5 text

Architectural Design Trade-offs 1. Rendering for Data Scale 2. Computation for Interaction Speed 3. Adaptive to Data Complexity 4. Responsive to Data Velocity 5

Slide 6

Slide 6 text

1. Rendering for Data Scale "How do you render interactive visualization when there are millions or billions of data points?" 6

Slide 7

Slide 7 text

Visualise a Million Points Show all the Data Same order as the number of pixels on my MacBook Air: 1400 x 900 Problems with overplotting 7

Slide 8

Slide 8 text

Visualise a Million Points Sample the Data Sampling can be effective (with overweighting unusual values) Require multiple plots or careful tuning parameters 8

Slide 9

Slide 9 text

Visualise a Million Points Model the Data Models are great as they scale nicely. But, visualisation is needed to answer the question: “I don’t know, what I don’t know.” 9

Slide 10

Slide 10 text

Visualise a Million Points Bin the Data Reduce the size of the data to complement the pixel resolution on screen 10

Slide 11

Slide 11 text

“Visualising data at scale is the process of creating generalized histograms” 11

Slide 12

Slide 12 text

Layers of Visualisation 12

Slide 13

Slide 13 text

Rendering: How many data points? — SVG: ~10^3 — Canvas: ~10^4 — Web.gl: ~10^6 13

Slide 14

Slide 14 text

Web.gl based Rendering: Deck.gl1 1 UK Road Accident Data using deck.gl 14

Slide 15

Slide 15 text

Bin and Web.gl Rendering 15

Slide 16

Slide 16 text

Visualise more than Million Points Bin-Summarize-Smooth Shift the transfer of data from raw data to binned- summarised-smoothed data 16

Slide 17

Slide 17 text

BigVis: Bin-Summarize-Smooth-Visualise 17

Slide 18

Slide 18 text

Why not sent an image of the Data: DataShader3 3 DataShader: Turn largest data to image, accurately 18

Slide 19

Slide 19 text

DataShader: Encode-Bin-Summarise-ColorMap 19

Slide 20

Slide 20 text

2. Computation for Interaction Speed "How do you reduce the latency of the query at the interaction layer, so that the user can interact with the visualisation?" 20

Slide 21

Slide 21 text

Aggregation & In-Memory Cubes e.g imMems4 4 ImMems Demo 21

Slide 22

Slide 22 text

In-memory Cubes 22

Slide 23

Slide 23 text

Approximate Query Processing e.g. VerdictDB5 5 Speed Comparison: AQP is 50x+ faster 23

Slide 24

Slide 24 text

Approximate Query Processing 24

Slide 25

Slide 25 text

Challenges in Visualise Uncertainty 6 6 25

Slide 26

Slide 26 text

Faster Databases e.g. MapD7 7 TweetMap Demo by MapD 26

Slide 27

Slide 27 text

Faster Databases 27

Slide 28

Slide 28 text

3. Adaptive to Data Complexity Choosing a good visualisation design for a singular dataset is possible after a few experiments and iteration. But how do you ensure that the visualisation will adapt to the variety, volume and edge cases in the real data? 28

Slide 29

Slide 29 text

Responsive Visualisation to Space & Data8 8 Responsive Data Visualisation - Nick Rabinowitz 29

Slide 30

Slide 30 text

Handling High Cardinality e.g. Facet-Dive9 9 FacetDive from PAIR Research 30

Slide 31

Slide 31 text

Handling High Cardinality e.g. DataComb10 10 Datacomb: Interactive Table Plot 31

Slide 32

Slide 32 text

Dimension Reduction e.g. Embedding Projector11 11 Embedding Projector 32

Slide 33

Slide 33 text

4. Responsive to Data Velocity "How do you trade-offs between real-time vs. near real-time data and its impact on refreshing visualization" 33

Slide 34

Slide 34 text

Optimizing for near real-time visual refreshes 34

Slide 35

Slide 35 text

Architectural Design Trade-offs 1. Rendering for Data Scale 2. Computation for Interaction Speed 3. Adaptive to Data Complexity 4. Responsive to Data Velocity 35

Slide 36

Slide 36 text

Architecture Design for Interactive Visualisation Amit Kapoor amitkaps.com Bargava Subramanian bargava.com 36