High Performance Data Visualizaitons in JavaScript [Topconf 2013]

High Performance Data Visualizaitons in JavaScript [Topconf 2013]

If you thought that building rich, interactive and mobile-friendly visualizations of high volume data with 100,000+ points just using the power of browser-side JavaScript was impossible, this talk will prove you wrong.

We’ll review every important aspect of achieving peak performance and responsiveness for these types of applications, including search indexes and tree structures, computational geometry and clustering algorithms, real-time data simplification, fast collision detection, advanced use of Web Workers and mixing Canvas with SVG and HTML.

6d07e6d95a43357254698ce9723350e6?s=128

Vladimir Agafonkin

November 06, 2013
Tweet

Transcript

  1. Vladimir Agafonkin High Performance Data Visualizations in JavaScript November 2013

  2. /mourner

  3. data visualization

  4. None
  5. None
  6. None
  7. None
  8. None
  9. None
  10. not static anymore

  11. responding to user actions clicking, hovering, scrolling, touch gestures, etc.

  12. None
  13. None
  14. None
  15. navigating through data

  16. filtering

  17. demand for real-time interactivity is increasing

  18. responsibility for processing data is shifting from the server 

    to the browser
  19. lots of data + lots of rendering = big performance

    problem
  20. pure JS fast rendering, DOM slow

  21. rule #1 the less stuff we render  the better

  22. data processing rendering

  23. data processing rendering

  24. None
  25. data reduction

  26. search

  27. loading data  once ! searching data  lots of

    times
  28. search index

  29. grid

  30. tree data structures •binary heap •binary search tree •range tree

    •k-d tree •quadtree •R-tree
  31. points – quadtree

  32. rectangles – R-tree

  33. var tree = rbush(); ! tree.insert([5, 10, 15, 25, obj]);

    ! ... ! tree.search([7, 7, 12, 12]); github.com/mourner/rbush
  34. greedy rendering

  35. •avoids rendering many objects in the same spot •free index

    for instant mouse/touch interaction greedy rendering
  36. kothic.org/js

  37. collision detection

  38. Crossfilter (many dimensions)

  39. geometric clipping

  40. polyline clipping Cohen-Sutherland algorithm

  41. polygon clipping Sutherland-Hodgeman algorithm

  42. polyline simplification

  43. distance-based simplification

  44. Douglas-Peucker simplification

  45. mourner.github.com/simplify-js

  46. clustering grouping objects which are  close to each other

  47. None
  48. None
  49. hierarchical clustering once for all zoom levels

  50. data loading and processing

  51. UI JS browser freezes on heavy calculations UI

  52. Web Workers

  53. pure JS fast ! rendering, DOM slow

  54. Worker •isolated DOM-less environment, freaking fast •runs in its own

    thread, doesn’t lock the UI •sends and receives messages
  55. Web Workers you’ve been using it all wrong

  56. <script src='data.js'></script> ! <script> ... worker.postMessage(HUGE_DATA_ARRAY); ... </script> loading and

    sending to Worker
  57. UI Worker JS browser freezes on data loading and sending

    data loading UI
  58. importScripts('data.js'); ! ... ! onmessage = function (e) { var

    result =  searchData(e.data.query); ! postMessage(result); } Loading in Worker
  59. UI Worker JS browser freezes when receiving data from Worker

    data loading UI UI
  60. var array = new Float16Array(len); ... ! var buffer =

    array.buffer; ! postMessage(buffer, [buffer]); ! // buffer stops being available transferable objects (all browsers except IE)
  61. UI Worker JS browser doesn't freeze,  data is sent

    as ArrayBuffer data loading UI UI
  62. function addNumbers(a, b) { 'use asm'; ! a = a

    | 0; // int b = +b; // double ! return +(a + b); // double } asm.js
  63. asm.js •useful for computational bottlenecks •supported in FF only •backwards

    compatibility! •V8 optimizes without annotations
  64. V8: lets make it as fast as asm.js but without

    the need for special syntax
  65. rendering backends SVG, Canvas, WebGL

  66. SVG •fast native events for interactivity •easy to update separate

    objects •easy to animate •slows down the browser (with a large number of objects)
  67. Canvas •doesn't affect browser performance after rendering •you can draw

    something once and copy •pixel data can be manipulated or generated in a Worker
  68. Canvas •expensive to redraw with each update •hard to implement

    interactivity
  69. WebGL •main way to visualize in 3D •very fast in

    2D if you need to draw lots of sprites •performance gain vs Canvas-2D is questionable in other cases •much more difficult to use; limitations •no support in iOS and IE9-10, difficult in IE11
  70. low number of objects:  use SVG ! lots of

    stuff to draw:  use Canvas
  71. Canvas Performance

  72. redraw partially

  73. use several Canvas layers

  74. draw once to an offscreen canvas and copy

  75. function drawStar(ctx, x, y) { ... } ! drawStar(ctx, 10,

    20); drawStar(ctx, 50, 70); ...
  76. function drawStar() { ... return canvas; } ! var star

    = drawStar(); ! ctx.drawImage(star, 10, 20); ctx.drawImage(star, 50, 70); ...
  77. minimize stroke/fill

  78. function drawLine(x1, x2, y1, y2) { ! ctx.strokeStyle = 'red';

    ! ctx.beginPath(); ctx.moveTo(x1, y1); ctx.lineTo(x2, y2); ! ctx.stroke(); } ! drawLine(10, 20, 30, 40); drawLine(200, 10, 0, 50); drawLine(30, 40, 70, 0);
  79. function drawLine(x1, x2, y1, y2) { ! ctx.beginPath(); ctx.moveTo(x1, y1);

    ctx.lineTo(x2, y2); } ! ctx.strokeStyle = 'red'; ! drawLine(10, 20, 30, 40); drawLine(200, 10, 0, 50); drawLine(30, 40, 70, 0); ! ctx.stroke();
  80. generate or manipulate  raw pixel data in a Worker

  81. var data =  ctx.getImageData(0, 0, width, height).data; ! worker.postMessage(data.buffer,

    [data.buffer]); ! ... ! worker.onmessage = function (e) { var imageData =  ctx.createImageData(width, height); ! imageData.data.set(e.data); ! ctx.putImageData(imageData, 0, 0); } Canvas + Worker
  82. var pixels = new Uint8ClampedArray( width * height); ! function

    drawPixel(x, y, r, g, b, a) { var i = 4 * (256 * y + x); ! pixels[i] = r; pixels[i + 1] = g; pixels[i + 2] = b; pixels[i + 3] = a; } ! ... ! postMessage(pixels.buffer, [pixels.buffer]); drawing pixels in a Worker
  83. var pixels = new Uint8ClampedArray(data); ! for (var x =

    0; x < width; x++) { for (var y = 0; y < height; y++) { var i = 4 * (256 * y + x); ! pixels[i] = 2 * pixels[i]; pixels[i + 1] = 2 * pixels[i + 1]; pixels[i + 2] = 2 * pixels[i + 2]; } } ! ... ! postMessage(pixels.buffer, [pixels.buffer]); processing pixels in a Worker
  84. dynamic hill shading

  85. still too much for the browsers? resorting to the server

  86. Cavnas lookup table

  87. UTFGrid •65535 symbols •each symbol is 4х4 pixels •1-3 KB

    per 256х256 tile in average
  88. Thanks! Questions? Vladimir Agafonkin agafonkin@gmail.com