Location analytics by MARC PLANAGUMÀ at Big Data Spain 2014

Location analytics by MARC PLANAGUMÀ at Big Data Spain 2014

While the implementation of analytic operations on distributed computing frameworks has been widely describing, enabling the computational core of a Big Data system with capabilities for supporting geospatial querying on data is yet a challenging issue. This session aims to target that specific aspect by reviewing how researchers at BDigital Technology Centre have designed and implemented a stack for advanced Machine Learning on Urban Data by providing a way to geoquery massive amounts of HDFS data from Spark processes without hindering the overall system performance.

Cb6e6da05b5b943d2691ceefa3381cad?s=128

Big Data Spain

November 25, 2014
Tweet

Transcript

  1. SPATIAL DATA ANALYTICS MARC PLANAGUMÀ LEAD DATA ENGINEER AND RESEARCHER

    BDIGITAL
  2. Ciudad 2020 Exploratory lines and further work Joana Simoes, BDigital

    27/05/2014 Scaling up a in a world of geolocated data Marc Planagumà, BDigital Technology Centre
  3. Our team

  4. Our work

  5. A world of geolocated data

  6. Applications

  7. None
  8. None
  9. Spatial data analytics Spatial analysis or spatial statistics includes any

    of the formal techniques which studies entities using their topological, geometric or geographic properties
  10. Spatial data analytics Basic Spatial Types Line Point Polygon *

    plus others
  11. Spatial data analytics Geometries

  12. Spatial data analytics Operations Contains Within

  13. Spatial data analytics Operations Equals Disjoint

  14. Spatial data analytics Operations Crosses Touches Overlaps

  15. Spatial data analytics Operations Distance

  16. Spatial data analytics Operations Union Difference

  17. Spatial data analytics Operations Intersection Symmetric Diff

  18. Spatial data analytics Complex operations Buffering Convex Hull

  19. Spatial data analytics Complex operations Area

  20. Spatial data analytics Geospatial data mining techniques Density-based clustering Hierarchy-based

    clustering
  21. Spatial data analytics Geospatial data mining techniques Heatmaps

  22. Spatial data analytics Geospatial data mining techniques Trajectory mining

  23. Spatial data analytics Geospatial data mining techniques Geospatial querying Spatial

    anomalies Geographically weighted regressions Self-organizing Maps Agent-based modeling
  24. Our Use Cases Customer activity tracking Where does people buy?

    What is the profile of the buyer? Traffic agencies Can we predict the traffic status? Can we predict the traffic incidences? Spatial data analytics
  25. Our Datasets Twitter (Point) Meteo (Polygon) Traffic status (Multi-Line-String) Traffic

    incidence (Multi-Point) Credit card ops (Multi-Point) Demographic (Point & Polygon) Shapefiles (Multi-Polygon) Spatial data analytics
  26. Our Tools Queries Clustering Heat maps Trajectory Anomaly detection Spatial

    data analytics
  27. Our Stacks

  28. Old School Stack - Why Not? PostGIS Hadoop + GIS

    Tools ElasticSearch Kibana Spatial data analytics
  29. None
  30. Hadoop Stack - The new Classic PostGIS Hadoop + GIS

    Tools ElasticSearch Kibana Spatial data analytics
  31. None
  32. Complete Stack - Big Data GIS PostGIS Hadoop + GIS

    Tools ElasticSearch Kibana Spatial data analytics
  33. None
  34. Do’s and Don’ts Geo Batch Near Real-Time Basic Geo Operations

    Workflow Composition Geo Streams Real-Time Advanced Geo Op. Unattended Workflow Spatial data analytics
  35. What’s next?

  36. RFC ;) Spatial data analytics

  37. Moltes gràcies! nom@bdigital.org mplanaguma@bdigital.org @djkram @bdigital

  38. 17TH ~ 18th NOV 2014 MADRID (SPAIN)