GPU-Accelerated Instance for Interactive Exploratory Analysis

Presented at AI NextCon in Seattle

Transcript

1. GPU-Accelerated Instance for Interactive Exploratory Analysis AI NextCon ‘19, Seattle Veda Shankar, OmniSci | January 24th, 2019
2. Veda Shankar Senior Developer Advocate , OmniSci Community veda.shankar@omnisci.com slides: https://speakerdeck.com/omnisci
3. © OmniSci 2018 Data Grows Faster Than CPU Processing Data Growth 40% per year CPU Processing Power 20% per year
4. © OmniSci 2018 GPU Processing Keeps Moore’s Law Alive
5. © OmniSci 2018 5 Core Density Makes a Huge Difference
6. © OmniSci 2018 6 Advanced Memory Management
7. The Fastest Software Designed for the Fastest Hardware HARNESS GPUs
8. © OmniSci 2018 8 OmniSci Innovations Powering Extreme Analytics 3-Tier Memory Caching Query Compilation In-Situ Rendering
9. © OmniSci 2018 9
10. DEMO : OmniSci Immerse
11. © OmniSci 2018 11 OmniSci Geospatial Features • Geospatial objects ◦ POINT, LINESTRING, POLYGON, MULTIPOLYGON • Geospatial File Formats ◦ GeoJSON, ESRI Shapefile, KML and CSV/TSV with WKT • Geospatial Functions ◦ Geometry Constructors ◦ Geometry Editors ◦ Geometry Accessors ◦ Spatial Relationships and Measurements ▪ ST_Distance, ST_Contains, ST_Within, ST_Area, ST_Perimeter, ST_Length
12. © OmniSci 2018 12 Ingesting LIDAR Data • Coordinate Systems & LIDAR ◦ LIDAR data is typically available in local coordinate systems ◦ OmniSci ver4 internally supports Web Mercator & Geographic Coordinates (WGS84 lat/lon) ◦ Back-projection of local into Geo-coordinates is default • LIDAR to OmniSci Readable Format ◦ PDAL is the best tool to convert LIDAR files to formats readable by OmniSci ◦ PDAL is an open source library installable locally using Conda ▪ conda install -c mathieu pdal ◦ PDAL Docker image available for easy execution ▪ !sudo docker run -v {laz_dir}:/data:z pdal/pdal:1.7 pdal translate -i /data/{laz_file} -o /data/{csv_file} -f filters.reprojection --filters.reprojection.out_srs="EPSG:4326" • Reference blogs ◦ https://www.omnisci.com/blog/taming-the-lidar-beast-part-1/ ◦ https://www.omnisci.com/blog/3d-lidar-with-mapd-and-ubers-deck-gl/
13. © OmniSci 2018 13 Building Custom Apps with MapD Charting • OmniSci provides mapd-charting - a superfast charting library that is based on dc.js, and is designed to work with MapD-Connector and MapD-Crossfilter to create charts instantly using OmniSci's Core SQL Database as the backend. • Reference blogs ◦ Creating OmniSci Custom Apps for Oil & Gas Applications
14. © OmniSci 2018 14 pymapd • The pymapd client interface provides a python DB API 2.0-compliant OmniSci interface. • pymapd provides methods to get results in the Apache Arrow-based GDF format for efficient data interchange with ML Libraries (XGBoost, H2O) • Reference blogs ◦ Using pymapd to Load Data to OmniSci Cloud
15. © OmniSci 2018 15 GPU Open Analytics Initiative (GOAI) Seamless data interchange framework in GPU memory
16. © OmniSci 2018 16 GPU Open Analytics Initiative (GOAI) Creating common data frameworks to accelerate data science on GPUs
17. © OmniSci 2018 17 OmniSci ML Lab : Jupyter Notebooks https://github.com/mapd/mapd-ml-demo
18. © OmniSci 2018 Four Ways to Get Started GitHub repo OPEN SOURCE Website download COMMUNITY OmniSci as a service OMNISCI CLOUD Contact sales ENTERPRISE 18 Ask questions and share your experiences @ https://community.omnisci.com
19. THANK YOU