Big data meets scalable visualizations

Presentation given at Big Data Spain http://www.bigdataspain.org/2013/conference/big-data-meets-scalable-visualizations

Transcript

1. Big data meets scalable visualizations JAVIER DE LA TORRE
3. 3 picture  on  big  data  awesomeness Big data awesomeness!!!!
7. Big data without data visualization = #fail
8. Maps are the most popular type of data visualization Everything happens somewhere ! Where are your clients? IP=location ! So everything can be analyzed and visualized on maps
9. Everybody wants to see data on maps, But making good maps is very hard! Ugly map!
10. Making maps is hard because… Tools are not there yet. They are for GIS experts ! Handling 100 points is easy, 1Million is hard ! Data chages! Is not about printing maps online!
11. 11 Demo  on  meteorites
12. Wall Street Journal US election maps
13. Big data analysis and reporting tool - UNEP Carbon calculator
14. Narrative maps / Story telling - The Hobbit filming Locations map
15. Narrative maps / Story telling - The Rolling Stones tour maps
16. German elections real time maps
17. Visualizing NYC Open Data
18. Animated geotemporal maps. Everything happens somewhere and at some time. Navy of WWI map
19. Visual analysis - Economic impact of the Mobile World Congress 2012 in Barcelona
20. All meteorites fallen on earth
21. Animated city traffic maps
22. Mobile ready.
24. Big data analysis of deforestation How we can track deforestation on real time Global Forest Watch
46. http://en.wikipedia.org/wiki/Bakun_Dam
47. Most people don’t need Big Data technologies But when you can’t…. when it really explodes… You just need to start collecting and analyzing data. Don’t focus on technology, probably your database can already do it ! You are not Facebook, don’t be cheat
48. Foreign Data wrappers Connect PostgreSQL to almost anything Oracle Hadoop MySQL MongoDB CouchDB Redis …. Twitter Email S3
49. 49 CartoDB Hadoop HBase
50. Geo-temporal visualizations CartoDB and Torque
52. WITH%hgrid% %%%%%AS%(SELECT%Cdb_rectanglegrid(Cdb_xyz_extent(8,%12,%5),% %%%%%%%%%%%%%%%%Cdb_xyz_resolution(5)%*%4,% %%%%%%%%%%%%%%%%%%%%%%%%%%%Cdb_xyz_resolution(5)%*%4)%AS%cell)% SELECT%x,% %%%%%%%y,% %%%%%%%Array_agg(c)%vals,% %%%%%%%Array_agg(d)%dates% FROM%%%(SELECT%St_xmax(hgrid.cell)%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%x,% %%%%%%%%%%%%%%%St_ymax(hgrid.cell)%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%y,% %%%%%%%%%%%%%%%Count(i.cartodb_id)%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%c,% %%%%%%%%%%%%%%%Floor((%Date_part('epoch',%built)%Q%Q10418716800%)%/%32837875)%d% %%%%%%%%FROM%%%hgrid,% %%%%%%%%%%%%%%%us_po_offices%i% %%%%%%%%WHERE%%St_intersects(i.the_geom_webmercator,%hgrid.cell)% %%%%%%%%GROUP%%BY%hgrid.cell,% %%%%%%%%%%%%%%%%%%Floor((%Date_part('epoch',%built)%Q%Q10418716800%)%/%32837875)% %%%%%%%)%f% GROUP%%BY%x,% %%%%%%%%%%y
53. { %%rows:%[ %%{ %%%%x:%0, %%%%y:%0, %%%%vals:%[2], %%%%dates:%[457] %%}, %%{ %%%%x:%1, %%%%y:%0, %%%%vals:%[1,1,4], %%%%dates:%[2,3,4] %%%%} %%] }
54. 1 10 100 1000 3mb 70mb 300mb 1.5 2 1.2 300 70 3 Raw Datacube Payload sizes
55. Think on the value of location on your data, and use it! Is very likely you have geospatial data already ! Complete the big data cycle: Don't forget data visualization ! Find the stories inside the data and show them!