Building Ambitious Data Visualisations

Say you're building a chart, what's the most natural tool for specifying a visualisation? A configurable chart component? Abstract components which can be used together to create something larger? Or low level primitives which can give you fine grained control over your presentation?
In this talk I will introduce D3 Shape, and demonstrate how you can compose reusable Components which solve all of the above requirements for quickly visualising information in Ember.

Transcript

1. Building Ambitious Data Visualisations IVAN VANDERBYL Co-founder Flood IO
5. en-AU
8. What is a data visualisation?
9. “Visual representations of abstract data to amplify cognition” — Readings in information visualization: using vision to think. Morgan Kaufmann, 1999.
10. What's the most natural tool for specifying a visualization?
11. A configurable chart?
12. High level charting library?
13. Low level geometric shapes and graphical markings?
14. efficiency expressiveness
15. Efficiency
17. Configurable Charting Package® XP Pro
18. Configurable Charting Package® XP Pro
19. “If we endeavour to develop a charting instead of a graphing program, we will accomplish two things. First, we inevitably will offer fewer charts than people want. Second, our package will have no deep structure. Our computer program will be unnecessarily complex, because we will fail to reuse objects or routines that function similarly in different charts. And we will have no way to add new charts to our system without generating complex new code. Elegant design requires us to think about a theory of graphics, not charts.” — Leland Wilkinson, The Grammar of Graphics
20. “Elegant design requires us to think about a theory of graphics, not charts.”
21. Separation of concerns
22. series: [{ data: [29.9, 71.5, 106.4, 129.2, 144.0, 176.0, 135.6, 148.5, {y: 216.4, marker: { fillColor: '#BF0B23', radius: 10 } }, 194.4] }] series: [{ data: [29.9, 71.5, 106.4, 129.2, 144.0, 176.0, 135.6, 148.5, {y: 216.4, marker: { fillColor: '#BF0B23', radius: 10 } }, 194.1, 54.4] }] Data Presentation
23. Source: http://blog.codinghorror.com/the-php-singularity/
24. Single Responsibility Principle
25. scaleGridLineColor : "rgba(0,0,0,.05)", //Number - Width of the grid lines scaleGridLineWidth : 1, //Boolean - Whether to show horizontal lines (except X axis) scaleShowHorizontalLines: true, //Boolean - Whether to show vertical lines (except Y axis) scaleShowVerticalLines: true, //Boolean - Whether the line is curved between points bezierCurve : true, //Number - Tension of the bezier curve between points bezierCurveTension : 0.4, //Boolean - Whether to show a dot for each point pointDot : true, //Number - Radius of each point dot in pixels pointDotRadius : 4, //Number - Pixel width of point dot stroke pointDotStrokeWidth : 1, //Number - amount extra to add to the radius to cater for hit detection outside the drawn point pointHitDetectionRadius : 20, //Boolean - Whether to show a stroke for datasets datasetStroke : true, //Number - Pixel width of dataset stroke datasetStrokeWidth : 2, //Boolean - Whether to fill the dataset with a colour datasetFill : true, //String - A legend template legendTemplate : "<ul class=\"<%=name.toLowerCase()%>-legend\"><% for (var i=0; i<datasets.
26. import { technicalDebt } from "charting-package";
27. let chartConfig = { bigRedLine: true, bigRoundDotsOnLines: true, poniesEnabled: false, puppiesEnabled: true, thatFeatureMyBossAskedFor: true, notSureWhatThisOptionOptions: false, accuracy: “good” };
28. Expressiveness
29. var stations = []; // lazily loaded var formatTime = d3.time.format("%I:%M%p"); var margin = {top: 20, right: 30, bottom: 20, left: 100}, width = 960 - margin.left - margin.right, height = 500 - margin.top - margin.bottom; var x = d3.time.scale() .domain([parseTime("5:30AM"), parseTime("11:30AM")]) .range([0, width]); var y = d3.scale.linear() .range([0, height]); var xAxis = d3.svg.axis() .scale(x) .ticks(8) .tickFormat(formatTime); var line = d3.svg.line() .x(function(d) { return x(d.time); }) .y(function(d) { return y(d.station.distance); }); var svg = d3.select("body").append("svg") .attr("width", width + margin.left + margin.right) .attr("height", height + margin.top + margin.bottom) .append("g")
31. d3-shape “A small JavaScript library for drawing geometric shapes commonly found in data visualizations.”
32. LIVE EXAMPLE: ec16.tomster.io/curves/linear
33. Live Example App: ec16.tomster.io/line-test
35. ember install ember-cli-d3-shape Use it with ember, today.
36. import { arc, pie } from 'd3-shape'; let arcFn = arc() .cornerRadius(8) .innerRadius(200) .outerRadius(232); let pieFn = pie().padAngle(5/360); arcFn(pieFn([80,20])[0]); arcFn(pieFn([80,20])[1]); //"M2.1271150941153607...
38. import { quantile } from 'd3-array'; quantile([0, 4, 7, 9, 12, 18, 22, 25, 28, 31], 0.95); // 29.65
39. import { deviation, extent, histogram, thresholdFreedmanDiaconis, thresholdScott, thresholdSturges, max, mean, median, min, permute, quantile, range, scan, shuffle, sum, ticks, transpose, variance, Zip, ... } from 'd3-array';
40. Theory of Graphics
41. "concurrency_mean": [ { "timestamp": 1450345920000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 200, "label": null }, { "timestamp": 1450345935000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 200, "label": null }, { "timestamp": 1450345950000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 400, "label": null }, { "timestamp": 1450345965000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 400, "label": null }, "response_time_mean": [ { "timestamp": 1450345920000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1885, "label": null }, { "timestamp": 1450345935000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 2023, "label": null }, { "timestamp": 1450345950000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1938, "label": null }, { "timestamp": 1450345965000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1940, "label": null }, "transaction_rate_mean": [ { "timestamp": 1450345920000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 465, "label": null }, { "timestamp": 1450345935000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1074, "label": null }, { "timestamp": 1450345950000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1256, "label": null }, { "timestamp": 1450345965000, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 2090, "label": null },
42. Response Time
43. Response Time Concurrency
44. Response Time Concurrency Passed / RPM Failed / RPM
45. http://www.caltrain.com/schedules/PDF_Schedules.html
47. https://bl.ocks.org/mbostock/5544008
48. The correct representation doesn’t matter as much as the accuracy of the visualisation.
51. Fast & Furious 6 - Universal Pictures
52. You can’t exaggerate the presentation without disregarding the underlying data.
55. The Grammar of Graphics Wilkinson, Leland. The grammar of graphics. Springer Science & Business Media, 2006. 1. Data 2. Transforms 3. Scales 4. Coordinates 5. Elements 6. Guides
58. Empirical Data E.g. Events observed in the real world Abstract data E.g. Data generated by a modeling function. range(0, 100, 0.25), etc. Metadata E.g. Data about data
59. import Route from "ember-route"; import fetch from "ember-network/fetch"; export default Route.extend({ model() { return fetch("/api/metrics").then((response) => response.json()); }, setupController(controller, metrics) { controller.setProperties({ metrics }); } });
61. "response_time_mean": [ { "timestamp": 1450345920000.0, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1885, "label": null }, { "timestamp": 1450345935000.0, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 2023, "label": null }, { "timestamp": 1450345950000.0, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1938, "label": null }, { "timestamp": 1450345965000.0, "flood_id": 1697, "grid_id": 553, "project_id": 1, "value": 1940, "label": null }, [ [ 1450345920000, 1885 ], [ 1450345935000, 2023 ], [ 1450345950000, 1938 ], [ 1450345965000, 1940 ], [ 1450345980000, 1914 ], [ 1450345995000, 1996 ], [ 1450346010000, 1912 ], [ 1450346025000, 2171 ],
62. export default Controller.extend({ /** * Represents all the metrics we want to display. * Example: * {response_time_mean: [{timestamp: TIMESTAMP, value: VALUE, label...}, ...]} * * @type {Object} */ metrics: {}, /** * Computes response times collection containing [TIMESTAMP, VALUE] pairs. */ responseTimeValues: computed.map('metrics.response_time_mean.[]', (d) => [d.timestamp, d.value]), });
64. 1500 54 SCALE Input domain=[1200,4000] Output domain=[0,500]
66. d3-scale github.com/d3/d3-scale
67. export default Component.extend(Coordinates, { values: [], xValues: computed.map('values.[]', (d) => new Date(d[0])), yValues: computed.mapBy('values.[]', 'lastObject'), xDomain: computedExtent('xValues.[]'), yDomain: computedExtent('yValues.[]'), xRange: computedExtent('plotArea.left', 'plotArea.width'), yRange: computedExtent('plotArea.top', 'plotArea.height'), xScale: computed('xRange', 'xDomain', { get() { const { xDomain: domain, xRange: range } = this.getProperties('xRange', 'xDomain'); return scaleTime().domain(domain).rangeRound(range); } }), yScale: computed('yRange', 'yDomain', { get() { const { yDomain: domain, yRange: range } = this.getProperties('yRange', 'yDomain'); return scaleLinear().domain(domain).rangeRound(range.reverse()); } }), });
69. Boilerplate
71. ...
72. github.com/ivanvanderbyl/maximum-plaid
73. <svg width="720" height="320"> {{plaid-line values=values xScale=xScale yScale=yScale stroke="#2196F3" strokeWidth="2" fill="none" }} </svg> <svg width="720" height="320"> <g id="ember4528" transform="translate(0,32)" class=" ember-view"> <path class="line" d="M0,162C2..." stroke="#2196F3" stroke-width="2" stroke-opacity="1" fill="none"></path> </g> </svg>
74. {{#plaid-plot xScale yScale plotArea as |plot|}} {{plot.line values}} {{/plaid-plot}}
75. // components/plaid-plot/template.hbs {{yield (hash line=(component "plaid-line" xScale=xScale yScale=yScale x=plotArea.left y=plotA scatter=(component "plaid-scatter" xScale=xScale yScale=yScale x=plotArea.left y symbol=(component "plaid-symbol" x=plotArea.left y=plotArea.top) bottom-axis=(component "plaid-axis" orientation="bottom" scale=xScale top=plotAr )}}
76. {{#plaid-plot xScale yScale plotArea as |plot|}} {{plot.bottom-axis}} {{plot.line values stroke=stroke strokeWidth="2"}} {{#plot.scatter values as |x y|}} <circle cx={{x}} cy={{y}} r="2" fill-opacity="0.54"/> {{/plot.scatter}} {{/plaid-plot}}
77. LIVE EXAMPLE: ec16.tomster.io/timeline
78. {{plot.line values curve=(curve-step-after) }}
79. d3-shape features implemented: - [x] Lines (plaid-line) - [x] Symbols (plaid-symbol) - [ ] Areas (plaid-area) - [ ] Pie - [ ] Donut - [ ] Arc - [ ] Stack Layout - [-] Curves Additions: - [x] Computed data transforms https://github.com/ivanvanderbyl/maximum-plaid/issues/1
82. “All sufficiently ambitious applications will eventually require data visualisation”
83. Thank You
84. CODE: github.com/ivanvanderbyl/maximum-plaid DEMO: ec16.tomster.io SLIDES: ec16.tomster.io/slides NEXT: HallwayConf (Next to this room) Ember Community Slack “@ivan”