Big data meets scalable visualizations

Slide 1

Slide 1 text

Big data meets scalable visualizations JAVIER DE LA TORRE

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

3 picture on big data awesomeness Big data awesomeness!!!!

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Big data without data visualization = #fail

Slide 8

Slide 8 text

Maps are the most popular type of data visualization Everything happens somewhere ! Where are your clients? IP=location ! So everything can be analyzed and visualized on maps

Slide 9

Slide 9 text

Everybody wants to see data on maps, But making good maps is very hard! Ugly map!

Slide 10

Slide 10 text

Making maps is hard because… Tools are not there yet. They are for GIS experts ! Handling 100 points is easy, 1Million is hard ! Data chages! Is not about printing maps online!

Slide 11

Slide 11 text

11 Demo on meteorites

Slide 12

Slide 12 text

Wall Street Journal US election maps

Slide 13

Slide 13 text

Big data analysis and reporting tool - UNEP Carbon calculator

Slide 14

Slide 14 text

Narrative maps / Story telling - The Hobbit filming Locations map

Slide 15

Slide 15 text

Narrative maps / Story telling - The Rolling Stones tour maps

Slide 16

Slide 16 text

German elections real time maps

Slide 17

Slide 17 text

Visualizing NYC Open Data

Slide 18

Slide 18 text

Animated geotemporal maps. Everything happens somewhere and at some time. Navy of WWI map

Slide 19

Slide 19 text

Visual analysis - Economic impact of the Mobile World Congress 2012 in Barcelona

Slide 20

Slide 20 text

All meteorites fallen on earth

Slide 21

Slide 21 text

Animated city traffic maps

Slide 22

Slide 22 text

Mobile ready.

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Big data analysis of deforestation How we can track deforestation on real time Global Forest Watch

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

http://en.wikipedia.org/wiki/Bakun_Dam

Slide 47

Slide 47 text

Most people don’t need Big Data technologies But when you can’t…. when it really explodes… You just need to start collecting and analyzing data. Don’t focus on technology, probably your database can already do it ! You are not Facebook, don’t be cheat

Slide 48

Slide 48 text

Foreign Data wrappers Connect PostgreSQL to almost anything Oracle Hadoop MySQL MongoDB CouchDB Redis …. Twitter Email S3

Slide 49

Slide 49 text

49 CartoDB Hadoop HBase

Slide 50

Slide 50 text

Geo-temporal visualizations CartoDB and Torque

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

WITH%hgrid% %%%%%AS%(SELECT%Cdb_rectanglegrid(Cdb_xyz_extent(8,%12,%5),% %%%%%%%%%%%%%%%%Cdb_xyz_resolution(5)%*%4,% %%%%%%%%%%%%%%%%%%%%%%%%%%%Cdb_xyz_resolution(5)%*%4)%AS%cell)% SELECT%x,% %%%%%%%y,% %%%%%%%Array_agg(c)%vals,% %%%%%%%Array_agg(d)%dates% FROM%%%(SELECT%St_xmax(hgrid.cell)%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%x,% %%%%%%%%%%%%%%%St_ymax(hgrid.cell)%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%y,% %%%%%%%%%%%%%%%Count(i.cartodb_id)%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%c,% %%%%%%%%%%%%%%%Floor((%Date_part('epoch',%built)%Q%Q10418716800%)%/%32837875)%d% %%%%%%%%FROM%%%hgrid,% %%%%%%%%%%%%%%%us_po_offices%i% %%%%%%%%WHERE%%St_intersects(i.the_geom_webmercator,%hgrid.cell)% %%%%%%%%GROUP%%BY%hgrid.cell,% %%%%%%%%%%%%%%%%%%Floor((%Date_part('epoch',%built)%Q%Q10418716800%)%/%32837875)% %%%%%%%)%f% GROUP%%BY%x,% %%%%%%%%%%y

Slide 53

Slide 53 text

{ %%rows:%[ %%{ %%%%x:%0, %%%%y:%0, %%%%vals:%[2], %%%%dates:%[457] %%}, %%{ %%%%x:%1, %%%%y:%0, %%%%vals:%[1,1,4], %%%%dates:%[2,3,4] %%%%} %%] }

Slide 54

Slide 54 text

1 10 100 1000 3mb 70mb 300mb 1.5 2 1.2 300 70 3 Raw Datacube Payload sizes

Slide 55

Slide 55 text

Think on the value of location on your data, and use it! Is very likely you have geospatial data already ! Complete the big data cycle: Don't forget data visualization ! Find the stories inside the data and show them!