Building Ambitious Data Visualisations

Building Ambitious Data Visualisations

Say you're building a chart, what's the most natural tool for specifying a visualisation? A configurable chart component? Abstract components which can be used together to create something larger? Or low level primitives which can give you fine grained control over your presentation?
In this talk I will introduce D3 Shape, and demonstrate how you can compose reusable Components which solve all of the above requirements for quickly visualising information in Ember.

9db8544717ba3d5a90afb1b5b9bd50bf?s=128

Ivan Vanderbyl

March 30, 2016
Tweet

Transcript

  1. Building Ambitious Data Visualisations IVAN VANDERBYL Co-founder Flood IO

  2. None
  3. None
  4. None
  5. en-AU

  6. None
  7. None
  8. What is a data visualisation?

  9. “Visual representations of abstract data to amplify cognition” — Readings

    in information visualization: using vision to think. Morgan Kaufmann, 1999.
  10. What's the most natural tool for specifying a visualization?

  11. A configurable chart?

  12. High level charting library?

  13. Low level geometric shapes and graphical markings?

  14. efficiency expressiveness

  15. Efficiency

  16. None
  17. Configurable Charting Package® XP Pro

  18. Configurable Charting Package® XP Pro

  19. “If we endeavour to develop a charting instead of a

    graphing program, we will accomplish two things. First, we inevitably will offer fewer charts than people want. Second, our package will have no deep structure. Our computer program will be unnecessarily complex, because we will fail to reuse objects or routines that function similarly in different charts. And we will have no way to add new charts to our system without generating complex new code. Elegant design requires us to think about a theory of graphics, not charts.” — Leland Wilkinson, The Grammar of Graphics
  20. “Elegant design requires us to think about a theory of

    graphics, not charts.”
  21. Separation of concerns

  22. series: [{ data: [29.9, 71.5, 106.4, 129.2, 144.0, 176.0, 135.6,

    148.5, {y: 216.4, marker: { fillColor: '#BF0B23', radius: 10 } }, 194.4] }] series: [{ data: [29.9, 71.5, 106.4, 129.2, 144.0, 176.0, 135.6, 148.5, {y: 216.4, marker: { fillColor: '#BF0B23', radius: 10 } }, 194.1, 54.4] }] Data Presentation
  23. Source: http://blog.codinghorror.com/the-php-singularity/

  24. Single Responsibility Principle

  25. scaleGridLineColor : "rgba(0,0,0,.05)", //Number - Width of the grid lines

    scaleGridLineWidth : 1, //Boolean - Whether to show horizontal lines (except X axis) scaleShowHorizontalLines: true, //Boolean - Whether to show vertical lines (except Y axis) scaleShowVerticalLines: true, //Boolean - Whether the line is curved between points bezierCurve : true, //Number - Tension of the bezier curve between points bezierCurveTension : 0.4, //Boolean - Whether to show a dot for each point pointDot : true, //Number - Radius of each point dot in pixels pointDotRadius : 4, //Number - Pixel width of point dot stroke pointDotStrokeWidth : 1, //Number - amount extra to add to the radius to cater for hit detection outside the drawn point pointHitDetectionRadius : 20, //Boolean - Whether to show a stroke for datasets datasetStroke : true, //Number - Pixel width of dataset stroke datasetStrokeWidth : 2, //Boolean - Whether to fill the dataset with a colour datasetFill : true, //String - A legend template legendTemplate : "<ul class=\"<%=name.toLowerCase()%>-legend\"><% for (var i=0; i<datasets.
  26. import { technicalDebt } from "charting-package";

  27. let chartConfig = { bigRedLine: true, bigRoundDotsOnLines: true, poniesEnabled: false,

    puppiesEnabled: true, thatFeatureMyBossAskedFor: true, notSureWhatThisOptionOptions: false, accuracy: “good” };
  28. Expressiveness

  29. var stations = []; // lazily loaded var formatTime =

    d3.time.format("%I:%M%p"); var margin = {top: 20, right: 30, bottom: 20, left: 100}, width = 960 - margin.left - margin.right, height = 500 - margin.top - margin.bottom; var x = d3.time.scale() .domain([parseTime("5:30AM"), parseTime("11:30AM")]) .range([0, width]); var y = d3.scale.linear() .range([0, height]); var xAxis = d3.svg.axis() .scale(x) .ticks(8) .tickFormat(formatTime); var line = d3.svg.line() .x(function(d) { return x(d.time); }) .y(function(d) { return y(d.station.distance); }); var svg = d3.select("body").append("svg") .attr("width", width + margin.left + margin.right) .attr("height", height + margin.top + margin.bottom) .append("g")
  30. None
  31. d3-shape “A small JavaScript library for drawing geometric shapes commonly

    found in data visualizations.”
  32. LIVE EXAMPLE: ec16.tomster.io/curves/linear

  33. Live Example App: ec16.tomster.io/line-test

  34. None
  35. ember install ember-cli-d3-shape Use it with ember, today.

  36. import { arc, pie } from 'd3-shape'; let arcFn =

    arc() .cornerRadius(8) .innerRadius(200) .outerRadius(232); let pieFn = pie().padAngle(5/360); arcFn(pieFn([80,20])[0]); arcFn(pieFn([80,20])[1]); //"M2.1271150941153607...
  37. None
  38. import { quantile } from 'd3-array'; quantile([0, 4, 7, 9,

    12, 18, 22, 25, 28, 31], 0.95); // 29.65
  39. import { deviation, extent, histogram, thresholdFreedmanDiaconis, thresholdScott, thresholdSturges, max, mean,

    median, min, permute, quantile, range, scan, shuffle, sum, ticks, transpose, variance, Zip, ... } from 'd3-array';
  40. Theory of Graphics

  41. "concurrency_mean": [ { "timestamp": 1450345920000, "flood_id": 1697, "grid_id": 553, "project_id":

    1, "value": 200, "label": null }, { "timestamp": 1450345935000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 200, "label": null }, { "timestamp": 1450345950000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 400, "label": null }, { "timestamp": 1450345965000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 400, "label": null }, "response_time_mean": [ { "timestamp": 1450345920000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1885, "label": null }, { "timestamp": 1450345935000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 2023, "label": null }, { "timestamp": 1450345950000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1938, "label": null }, { "timestamp": 1450345965000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1940, "label": null }, "transaction_rate_mean": [ { "timestamp": 1450345920000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 465, "label": null }, { "timestamp": 1450345935000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1074, "label": null }, { "timestamp": 1450345950000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1256, "label": null }, { "timestamp": 1450345965000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 2090, "label": null },
  42. Response Time

  43. Response Time Concurrency

  44. Response Time Concurrency Passed / RPM Failed / RPM

  45. http://www.caltrain.com/schedules/PDF_Schedules.html

  46. None
  47. https://bl.ocks.org/mbostock/5544008

  48. The correct representation doesn’t matter as much as the accuracy

    of the visualisation.
  49. None
  50. None
  51. Fast & Furious 6 - Universal Pictures

  52. You can’t exaggerate the presentation without disregarding the underlying data.

  53. None
  54. None
  55. The Grammar of Graphics Wilkinson, Leland. The grammar of graphics.

    Springer Science & Business Media, 2006. 1. Data 2. Transforms 3. Scales 4. Coordinates 5. Elements 6. Guides
  56. None
  57. None
  58. Empirical Data E.g. Events observed in the real world Abstract

    data E.g. Data generated by a modeling function. range(0, 100, 0.25), etc. Metadata E.g. Data about data
  59. import Route from "ember-route"; import fetch from "ember-network/fetch"; export default

    Route.extend({ model() { return fetch("/api/metrics").then((response) => response.json()); }, setupController(controller, metrics) { controller.setProperties({ metrics }); } });
  60. None
  61. "response_time_mean": [ { "timestamp": 1450345920000.0, "flood_id": 1697, "grid_id": 553, "project_id":

    1, "value": 1885, "label": null }, { "timestamp": 1450345935000.0, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 2023, "label": null }, { "timestamp": 1450345950000.0, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1938, "label": null }, { "timestamp": 1450345965000.0, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1940, "label": null }, [ [ 1450345920000, 1885 ], [ 1450345935000, 2023 ], [ 1450345950000, 1938 ], [ 1450345965000, 1940 ], [ 1450345980000, 1914 ], [ 1450345995000, 1996 ], [ 1450346010000, 1912 ], [ 1450346025000, 2171 ],
  62. export default Controller.extend({ /** * Represents all the metrics we

    want to display. * Example: * {response_time_mean: [{timestamp: TIMESTAMP, value: VALUE, label...}, ...]} * * @type {Object} */ metrics: {}, /** * Computes response times collection containing [TIMESTAMP, VALUE] pairs. */ responseTimeValues: computed.map('metrics.response_time_mean.[]', (d) => [d.timestamp, d.value]), });
  63. None
  64. 1500 54 SCALE Input domain=[1200,4000] Output domain=[0,500]

  65. None
  66. d3-scale github.com/d3/d3-scale

  67. export default Component.extend(Coordinates, { values: [], xValues: computed.map('values.[]', (d) =>

    new Date(d[0])), yValues: computed.mapBy('values.[]', 'lastObject'), xDomain: computedExtent('xValues.[]'), yDomain: computedExtent('yValues.[]'), xRange: computedExtent('plotArea.left', 'plotArea.width'), yRange: computedExtent('plotArea.top', 'plotArea.height'), xScale: computed('xRange', 'xDomain', { get() { const { xDomain: domain, xRange: range } = this.getProperties('xRange', 'xDomain'); return scaleTime().domain(domain).rangeRound(range); } }), yScale: computed('yRange', 'yDomain', { get() { const { yDomain: domain, yRange: range } = this.getProperties('yRange', 'yDomain'); return scaleLinear().domain(domain).rangeRound(range.reverse()); } }), });
  68. None
  69. Boilerplate

  70. None
  71. ...

  72. github.com/ivanvanderbyl/maximum-plaid

  73. <svg width="720" height="320"> {{plaid-line values=values xScale=xScale yScale=yScale stroke="#2196F3" strokeWidth="2" fill="none"

    }} </svg> <svg width="720" height="320"> <g id="ember4528" transform="translate(0,32)" class=" ember-view"> <path class="line" d="M0,162C2..." stroke="#2196F3" stroke-width="2" stroke-opacity="1" fill="none"></path> </g> </svg>
  74. {{#plaid-plot xScale yScale plotArea as |plot|}} {{plot.line values}} {{/plaid-plot}}

  75. // components/plaid-plot/template.hbs {{yield (hash line=(component "plaid-line" xScale=xScale yScale=yScale x=plotArea.left y=plotA

    scatter=(component "plaid-scatter" xScale=xScale yScale=yScale x=plotArea.left y symbol=(component "plaid-symbol" x=plotArea.left y=plotArea.top) bottom-axis=(component "plaid-axis" orientation="bottom" scale=xScale top=plotAr )}}
  76. {{#plaid-plot xScale yScale plotArea as |plot|}} {{plot.bottom-axis}} {{plot.line values stroke=stroke

    strokeWidth="2"}} {{#plot.scatter values as |x y|}} <circle cx={{x}} cy={{y}} r="2" fill-opacity="0.54"/> {{/plot.scatter}} {{/plaid-plot}}
  77. LIVE EXAMPLE: ec16.tomster.io/timeline

  78. {{plot.line values curve=(curve-step-after) }}

  79. d3-shape features implemented: - [x] Lines (plaid-line) - [x] Symbols

    (plaid-symbol) - [ ] Areas (plaid-area) - [ ] Pie - [ ] Donut - [ ] Arc - [ ] Stack Layout - [-] Curves Additions: - [x] Computed data transforms https://github.com/ivanvanderbyl/maximum-plaid/issues/1
  80. None
  81. None
  82. “All sufficiently ambitious applications will eventually require data visualisation”

  83. Thank You

  84. CODE: github.com/ivanvanderbyl/maximum-plaid DEMO: ec16.tomster.io SLIDES: ec16.tomster.io/slides NEXT: HallwayConf (Next to

    this room) Ember Community Slack “@ivan”