Atlanta Data Visualization Meetup: Hands-on Workshop

https://www.meetup.com/Atlanta-Data-Viz/events/252942224/
MapD's platform returns instant big data query results and interactive visualizations, removing the technology obstacles to immersive data exploration. MapD's open source GPU database and Immerse visualization platform leverages GPUs to execute
SQL queries and render complex visualizations with billions of rows in milliseconds. Participants in the workshop will get to experience firsthand the power of GPUs for data analytics, with instantaneous interaction on very large datasets.

Transcript

1. © MapD 2018 MapD Workshop Atlanta Data Visualization Meetup | August 14, 2018
2. © MapD 2018 Aaron Williams VP of Global Community @_arw_ aaron@mapd.com /in/aaronwilliams/ /williamsaaron slides: https://speakerdeck.com/mapd
3. © MapD 2018 © MapD 2018 Agenda • Intro to GPU Analytics and MapD • Tutorial A: Importing Data into MapD Cloud • Tutorial B: Rendering with Vega 3
4. © MapD 2018 Do This Now If you want to participate in the tutorials, sign up for a free trial account on MapD Cloud http://mapd.cloud 4
5. © MapD 2018 © MapD 2018 5 “Every business will become a software business, build applications, use advanced analytics and provide SaaS services.” - Smart CEO Guy has
6. © MapD 2018 The Evolution of Competitive Data 6 Collect It Make It Actionable Make it Predictive
7. Core Density Makes a Huge Difference 7 GPU Processing CPU Processing 40,000 Cores 20 Cores *fictitious example Latency Throughput CPU 1 ns per task (1 task/ns) x (20 cores) = 20 tasks/ns GPU 10 ns per task (0.1 task per ns) x (40,000 cores) = 4,000 task per ns Latency: Time to do a task. | Throughput: Number of tasks per unit time.
8. © MapD 2018 MapD is the analytics platform created for GPUs
9. © MapD 2018 © MapD 2018 9 DEMO TIME!
10. © MapD 2018 Advanced memory management Three-tier caching to GPU RAM for speed and to SSDs for persistent storage 1 0 SSD or NVRAM STORAGE (L3) 250GB to 20TB 1-2 GB/sec CPU RAM (L2) 32GB to 3TB 70-120 GB/sec GPU RAM (L1) 24GB to 256GB 1000-6000 GB/sec Hot Data Speedup = 1500x to 5000x Over Cold Data Warm Data Speedup = 35x to 120x Over Cold Data Cold Data COMPUTE LAYER STORAGE LAYER Data Lake/Data Warehouse/System Of Record
11. © MapD 2018 MapD Core: Query Compilation with LLVM
12. © MapD 2018 MapD Immerse: Hybrid Rendering
13. © MapD 2018 Categories of common MapD use cases 13 Operational Analytics • Thwart Banking Fraud • Scan for Cyber Threats • Fine-tune Advertising • Maintain the Utility Grid Geospatial Analytics • Monitor Networks • Ready Logistics • Forecast Micro-weather Data Science • Model Financial Markets • Predict Maintenance • Predict Staffing Levels
14. © MapD 2018 The GPU Open Analytics Initiative (GOAI) Creating common data frameworks to accelerate data science on GPUs 1 4 /mapd/pymapd /gpuopenanalytics/pygdf
15. © MapD 2018 • We’ve published a few notebooks showing how to connect to a MapD database and use an ML algorithm to make predictions • We’ve also shared a real-world example of churn, which we implemented with VW 15 ML Examples /gpuopenanalytics/demo-docker /mapd/mapd-ml-demo
16. © MapD 2018 Tutorial A: Importing Data into MapD Cloud 16
17. © MapD 2018 Last Chance ... If you want to participate in the tutorials, sign up for a free trial account on MapD Cloud http://mapd.cloud 17
18. © MapD 2018 © MapD 2018 18 SHORT BREAK?
19. © MapD 2018 Step 1: MapD Immerse Basics 1. View your data in the Data Manager 2. Import data a. Local CSV b. S3 Bucket 3. View your dashboards 4. Create a new dashboard a. SAVE! 19
20. © MapD 2018 Geospatial Objects Type Description POINT A point described by two coordinates. LINESTRING A sequence of 2 or more points and the lines that connect them. POLYGON A set of one or more rings (closed line strings), with the first representing the shape (external ring) and the rest representing holes in that shape (internal rings) MULTIPOLYGON A set of one or more polygons.
21. © MapD 2018 Step 2: Loading MapD Shapefiles into Immerse 1. Polygons SF City and County Subdivision Parcels MULTIPOLYGONS in GeoJSON https://s3.amazonaws.com/mapd-data/geodata/citylots.json 2. Points SF City-owned Critical Facilities POINTS in ESRI Shapefile https://s3.amazonaws.com/mapd-data/geodata/sffacs_current.zip 21
22. © MapD 2018 Tutorial B: Rendering with Vega 22
23. © MapD 2018 Load Using MapDQL • Clone the repo • git clone https://github.com/omveda/mapd-vega-mapboxgl-geo • Run the example (from the repo folder) • yarn install • npm start • Go to http://localhost:8080 • Change vegaspec.js to see points that are a specific distance from the utility lines • “name”: "sf_facilities", “sql”: “select conv_4326_900913_x(ST_X(A.mapd_geo)) as x, conv_4326_900913_y(ST_Y(A.mapd_geo)) as y, A.rowid from sf_facilities A, utility_lines B WHERE ST_Distance(ST_Transform(A.mapd_geo), ST_Transform(B.orig_geom)) < 20” 23
24. © MapD 2018 MapD VEGA MapD Vega is based on the open-source Vega Specification, it has been adapted to drive the rendering engine directly on the result set of a SQL query without ever requiring the data to leave the GPU. The MapD Connector API makes it easy to send the Vega JSON to the backend, which renders the visualization and returns a base64-encoded PNG image to the client. A Vega specification includes: • a data property that specifies and filters data source(s). • a marks property that defines the basic visualization graphic of a data item. • a scales property that defines geometry or applies additional attributes to the data item visualization. • viewing area dimensions.
25. © MapD 2018 Geospatial Functions Spatial Relationship and Measurement Functions Description ST_Distance Returns shortest planar distance between geometries. Returns shortest geodesic distance between geographies (in meters, limited support). ST_Contains Returns true if first geometry contains the second one.
26. © MapD 2018 Geospatial Functions Geometry Constructor Functions Description ST_GeomFromText Return a specified geometry from Well-Known Text representation (WKT) ST_GeogFromText Return a specified geography from Well-Known Text representation (WKT)
27. © MapD 2018 Geospatial Functions Geometry Editor Functions Description ST_Transform Return a new geometry with its coordinates transformed to a different spatial reference. The only supported transform in this release is WGS84 to Web Mercator, e.g. ST_Transform(ST_GeogFromText('POINT(-71.064544 42.28787)',4326),900913) ST_SetSRID Set the SRID on a geometry to a particular integer value.
28. © MapD 2018 Geospatial Functions Geometry Accessor Functions Description ST_XMin Returns X minima of a geometry. ST_XMax Returns X maxima of a geometry. ST_YMin Returns Y minima of a geometry. ST_YMax Returns Y maxima of a geometry. ST_StartPoint Returns the first point of a LINESTRING as a POINT. ST_EndPoint Returns the last point of a LINESTRING as a POINT. ST_PointN Return the Nth point of a LINESTRING as a POINT. ST_SRID Returns the spatial reference identifier for the underlying object
29. © MapD 2018 © MapD 2018 • mapd.com/demos Play with our demos • mapd.cloud Get a MapD instance in less than 60 seconds • mapd.com/platform/download-community/ Download the Community Edition • community.mapd.com Ask questions and share your experiences 29 Next Steps
30. © MapD 2018 Aaron Williams VP of Global Community @_arw_ aaron@mapd.com /in/aaronwilliams/ /williamsaaron slides: https://speakerdeck.com/mapd Thank you! Questions?