Shareable Data Objects - Visual Data Integration with d3

Talk at London d3.js User Group, 19 March 2014.
Shareable data objects from Sibdocity (www.sibdo.com).

Transcript

1. Shareable Data Objects Visual Data Integration with D3 Brian Gough @ Sibdocity 19 March 2014 Brian Gough @ Sibdocity Shareable Data Objects Visual Data Integration with D3 19 March 2014 1 / 12
2. Sibdocity Sibdocity develops visual data integration technologies (using d3) startup at IDEALondon in the City (CISCO/DC Thomson/UCL) available for commercial licensing (free widget on www.sibdo.com) contact founder Adrian.Hillary@sibdocity.com. The idea SIBDO = shareable interactive bindable data object Brian Gough @ Sibdocity Shareable Data Objects Visual Data Integration with D3 19 March 2014 3 / 12
3. The problem The problem Large organisations have multiple databases/applications Cannot compare data across "silos" Cannot compare with external data (from the internet) Typical solutions: combine databases into single data warehouse (expensive) CSV export/import -> Excel (inconvenient, lossy, uncontrolled) Brian Gough @ Sibdocity Shareable Data Objects Visual Data Integration with D3 19 March 2014 4 / 12
4. Sibdocity approach Sibdocity approach Integrate data across client applications directly in the front end Comments Tactical Can preserve metadata Accept data from any source/url Brian Gough @ Sibdocity Shareable Data Objects Visual Data Integration with D3 19 March 2014 5 / 12
5. Example Loading from difference formats/sources: <div class="sibdo" data-url="example.sib"> <div class="sibdo" data-url="example.csv"> <div class="sibdo" data-url="example.json"> <div class="sibdo" data-url= "http://s3.amazonaws.com/sibfx/XUDLGBD.sib"> 1 Get existing data into portable format by some method (via API, modify application) 2 Use portable format in new applications (render data via SIBDOs) Brian Gough @ Sibdocity Shareable Data Objects Visual Data Integration with D3 19 March 2014 6 / 12
6. Sibdo format { "_class": "sibdo", "factlets": [ { "_class": "sibfactlet", "col_names": [ "time", "data" ], "col_types": [ "Date", "Number"], "id": "831835a3-2486-4a22-c4db-a7bed700705 "properties": { "description": "Monthly average Spot e "label": "XUMAGBD" }, "table": [ { "data": 0.6097, "time": "2000-01" }, Brian Gough @ Sibdocity Shareable Data Objects Visual Data Integration with D3 19 March 2014 7 / 12
7. SIBDO Design Principles Treat data as objects, that can be shared between applications Universal runtime - object environment Javascript - inject into webpages Render objects in any page - MVC (MVVM) Handle business rules - semantic layer metadata, JSON-LD + OWL ontologies e.g. automatic currency and unit conversions Portable dataformat combining data and visualisation aka declarative chart format, SIBDO like VizJSON, VEGA, . . . HTML5 drag and drop serialise/deserialise objects works across browsers/tabs and to desktop Brian Gough @ Sibdocity Shareable Data Objects Visual Data Integration with D3 19 March 2014 8 / 12
8. Benefits Benefits for developers visualisation decoupled from back-end speed up development/deployment add new visualisations to runtime, available everywhere line, dot, bar, text, pie, . . . link object to any page element (not just charts) Benefits for users automatically combine data from different applications send data/visualisation as a file (->email, chat) drag to/from desktop Objects are lossless Brian Gough @ Sibdocity Shareable Data Objects Visual Data Integration with D3 19 March 2014 9 / 12
9. Technologies D3 Charting - natural mapping from object to visualisation Not just SVG! .enter()/.exit() pattern good for any container widgets example: info panel YUI3 does (nearly) everything you need, widgets runs in browser and under NodeJS (why aren’t more people using it?) Brian Gough @ Sibdocity Shareable Data Objects Visual Data Integration with D3 19 March 2014 10 / 12
10. Doing more Workspace - use decoupling to create independent application for analysis Container objects Dashboards, tabs, grids "Git for Data" Define a hash over JSON object representation Can now support git-like revision history of objects Server works like git repository for object revisions (MongoDB) Share and modify objects Object branches accessible as URLs e.g. http://sibdo.com/repo/user/object-id/rev-id Brian Gough @ Sibdocity Shareable Data Objects Visual Data Integration with D3 19 March 2014 11 / 12
11. Summary Shareable objects - a useful design pattern simplify visualisation architecture enable cross-application sharing Brian Gough @ Sibdocity Shareable Data Objects Visual Data Integration with D3 19 March 2014 12 / 12