End-to-End Open Source Data Science Workflow Using OmniSci and Nvidia RAPIDS

Presented at 7th Annual Global Big Data Conference on Aug 22nd, 2019 at Santa Clara Convention Center

Transcript

1. End-to-End Open Source Data Science Workflow Using OmniSci and Nvidia RAPIDS 7th Annual Global Big Data Conference Veda Shankar, OmniSci | Aug 22nd, 2019
2. Veda Shankar Senior Developer Advocate , OmniSci Community veda.shankar@omnisci.com slides: https://speakerdeck.com/omnisci
3. © OmniSci 2018 Data Grows Faster Than CPU Processing Data Growth 40% per year CPU Processing Power 20% per year
4. © OmniSci 2018 GPU Processing Keeps Moore’s Law Alive
5. © OmniSci 2018 5 Core Density Makes a Huge Difference
6. © OmniSci 2018 6 Advanced Memory Management
7. © OmniSci 2018 7 Advantages of a Common Data Layer from: https://arrow.apache.org
8. © OmniSci 2018 8 RAPIDS: The New GPU Data Science Pipeline from: https://github.com/rapidsai
9. © OmniSci 2018 Unifying GPU-accelerated Analytics and Data Science OmniSci query result set in-GPU to RAPIDS GPU-resident outputs from RAPIDS ML algorithms
10. © OmniSci 2018 10 OmniSci Innovations Powering Extreme Analytics 3-Tier Memory Caching Query Compilation In-Situ Rendering
11. © OmniSci 2018 11 * open source for single node github.com/omnisci/mapd-core
12. © OmniSci 2018 Taxi Demo 1B Taxi Rides + 1M Buildings
13. © OmniSci 2018 Three Ways to Get Started GitHub repo OPEN SOURCE OmniSci as a service OMNISCI CLOUD Contact sales ENTERPRISE 13
14. © OmniSci 2018 14 pymapd • The pymapd client interface provides a python DB API 2.0-compliant OmniSci interface. • pymapd provides methods to get results in the Apache Arrow-based GDF format for efficient data interchange with ML Libraries (XGBoost, H2O) • Reference blogs ◦ Using pymapd to Load Data to OmniSci Cloud
15. OmniSci Pymapd Demo • Jupyter Notebook https://github.com/omnisci/pymapd-workshop/blob/master/pymapd_usage.ipynb • Connect to OmniSci database • List tables in the database • Get table details • Run query and save results in a dataframe • Create table • Load data to table
16. OmniSci Pymapd ML Demo • Jupyter Notebook https://github.com/omnisci/pymapd-workshop/blob/master/flights_depdelay_cudf.ipynb • Connect to OmniSci database • Query departure delay & other features from flights table • Read data from query into CuDF dataframe • Prepping dataframe for model analysis • Using OLS (Ordinary Least Squares) to find feature impact on departure delay
17. © OmniSci 2018 © OmniSci 2018 • omnisci.com/blog Read interesting stories on product usage • omnisci.com/demos Play with our live demos for yourself! • omnisci.cloud Get an OmniSci instance in 60 seconds • omnisci.com/platform/downloads/ Download a 30-day trial of OmniSci • community.omnisci.com Ask questions and share your experiences Next Steps
18. Community Day (October 21st) is free to all attendees. Use Community50 for 50% off the entire conference! Date: October 21 - 23 Location: Computer History Museum 1401 N Shoreline Blvd Mountain View, CA 94043 converge.omnisci.com
19. THANK YOU Ask questions and share your experiences @ https://community.omnisci.com