High Performance Data Visualizaitons in JavaScript [Topconf 2013]

If you thought that building rich, interactive and mobile-friendly visualizations of high volume data with 100,000+ points just using the power of browser-side JavaScript was impossible, this talk will prove you wrong.

We’ll review every important aspect of achieving peak performance and responsiveness for these types of applications, including search indexes and tree structures, computational geometry and clustering algorithms, real-time data simplification, fast collision detection, advanced use of Web Workers and mixing Canvas with SVG and HTML.

Transcript

1. Vladimir Agafonkin High Performance Data Visualizations in JavaScript November 2013
2. /mourner
3. data visualization
10. not static anymore
11. responding to user actions clicking, hovering, scrolling, touch gestures, etc.
15. navigating through data
16. filtering
17. demand for real-time interactivity is increasing
18. responsibility for processing data is shifting from the server
    to the browser
19. lots of data
    +
    lots of rendering
    =
    big performance problem
20. pure JS
    fast rendering, DOM
    slow
21. rule #1
    the less stuff we render
    the better
22. data processing rendering
23. data processing rendering
25. data reduction
26. search
27. loading data
    once
    ! searching data
    lots of times
28. search index
29. grid
30. tree data structures •binary heap •binary search tree •range tree •k-d tree •quadtree •R-tree
31. points – quadtree
32. rectangles – R-tree
33. var tree = rbush();
    ! tree.insert([5, 10, 15, 25, obj]);
    ! ...
    ! tree.search([7, 7, 12, 12]); github.com/mourner/rbush
34. greedy rendering
35. •avoids rendering many objects in the same spot •free index for instant mouse/touch interaction greedy rendering
36. kothic.org/js
37. collision detection
38. Crossfilter (many dimensions)
39. geometric clipping
40. polyline clipping Cohen-Sutherland algorithm
41. polygon clipping Sutherland-Hodgeman algorithm
42. polyline simplification
43. distance-based simplification
44. Douglas-Peucker simplification
45. mourner.github.com/simplify-js
46. clustering
    grouping objects which are
    close to each other
49. hierarchical clustering once for all zoom levels
50. data loading and processing
51. UI JS browser freezes
    on heavy calculations UI
52. Web Workers
53. pure JS
    fast
    ! rendering, DOM
    slow
54. Worker •isolated DOM-less environment, freaking fast •runs in its own thread, doesn’t lock the UI •sends and receives messages
55. Web Workers
    you’ve been using it all wrong
56. <script src='data.js'></script>
    ! <script>
    ...
    worker.postMessage(HUGE_DATA_ARRAY);
    ...
    </script> loading and sending to Worker
57. UI Worker JS browser freezes on data loading and sending
    data loading UI
58. importScripts('data.js');
    ! ...
    ! onmessage = function (e) {
    var result =
    searchData(e.data.query);
    ! postMessage(result);
    } Loading in Worker
59. UI Worker JS browser freezes when receiving data from Worker
    data loading UI UI
60. var array = new Float16Array(len);
    ...
    ! var buffer = array.buffer;
    ! postMessage(buffer, [buffer]);
    ! // buffer stops being available transferable objects (all browsers except IE)
61. UI Worker JS browser doesn't freeze,
    data is sent as ArrayBuffer
    data loading UI UI
62. function addNumbers(a, b) {
    'use asm';
    ! a = a | 0; // int
    b = +b; // double
    ! return +(a + b); // double
    } asm.js
63. asm.js •useful for computational bottlenecks •supported in FF only •backwards compatibility! •V8 optimizes without annotations
64. V8: lets make it as fast as asm.js but without the need for special syntax
65. rendering backends
    SVG, Canvas, WebGL
66. SVG •fast native events for interactivity •easy to update separate objects •easy to animate •slows down the browser (with a large number of objects)
67. Canvas •doesn't affect browser performance after rendering •you can draw something once and copy •pixel data can be manipulated or generated in a Worker
68. Canvas •expensive to redraw with each update •hard to implement interactivity
69. WebGL •main way to visualize in 3D •very fast in 2D if you need to draw lots of sprites •performance gain vs Canvas-2D is questionable in other cases •much more difficult to use; limitations •no support in iOS and IE9-10, difficult in IE11
70. low number of objects:
    use SVG
    ! lots of stuff to draw:
    use Canvas
71. Canvas Performance
72. redraw partially
73. use several Canvas layers
74. draw once to an offscreen canvas and copy
75. function drawStar(ctx, x, y) {
    ...
    }
    ! drawStar(ctx, 10, 20);
    drawStar(ctx, 50, 70);
    ...
76. function drawStar() {
    ...
    return canvas;
    }
    ! var star = drawStar();
    ! ctx.drawImage(star, 10, 20);
    ctx.drawImage(star, 50, 70);
    ...
77. minimize stroke/fill
78. function drawLine(x1, x2, y1, y2) {
    ! ctx.strokeStyle = 'red';
    ! ctx.beginPath();
    ctx.moveTo(x1, y1);
    ctx.lineTo(x2, y2);
    ! ctx.stroke();
    }
    ! drawLine(10, 20, 30, 40);
    drawLine(200, 10, 0, 50);
    drawLine(30, 40, 70, 0);
79. function drawLine(x1, x2, y1, y2) {
    ! ctx.beginPath();
    ctx.moveTo(x1, y1);
    ctx.lineTo(x2, y2);
    }
    ! ctx.strokeStyle = 'red';
    ! drawLine(10, 20, 30, 40);
    drawLine(200, 10, 0, 50);
    drawLine(30, 40, 70, 0);
    ! ctx.stroke();
80. generate or manipulate
    raw pixel data in a Worker
81. var data =
    ctx.getImageData(0, 0, width, height).data;
    ! worker.postMessage(data.buffer, [data.buffer]);
    ! ...
    ! worker.onmessage = function (e) {
    var imageData =
    ctx.createImageData(width, height);
    ! imageData.data.set(e.data);
    ! ctx.putImageData(imageData, 0, 0);
    } Canvas + Worker
82. var pixels = new Uint8ClampedArray(
    width * height);
    ! function drawPixel(x, y, r, g, b, a) {
    var i = 4 * (256 * y + x);
    ! pixels[i] = r;
    pixels[i + 1] = g;
    pixels[i + 2] = b;
    pixels[i + 3] = a;
    }
    ! ...
    ! postMessage(pixels.buffer, [pixels.buffer]); drawing pixels in a Worker
83. var pixels = new Uint8ClampedArray(data);
    ! for (var x = 0; x < width; x++) {
    for (var y = 0; y < height; y++) {
    var i = 4 * (256 * y + x);
    ! pixels[i] = 2 * pixels[i];
    pixels[i + 1] = 2 * pixels[i + 1];
    pixels[i + 2] = 2 * pixels[i + 2];
    }
    }
    ! ...
    ! postMessage(pixels.buffer, [pixels.buffer]); processing pixels in a Worker
84. dynamic hill shading
85. still too much for the browsers? resorting to the server
86. Cavnas lookup table
87. UTFGrid •65535 symbols •each symbol is 4х4 pixels •1-3 KB per 256х256 tile in average
88. Thanks!
    Questions? Vladimir Agafonkin agafonkin@gmail.com