Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PyData Cardiff - RAPIDS 0.11: Open GPU Data Science

PyData Cardiff - RAPIDS 0.11: Open GPU Data Science

A remix of the RAPIDS 0.11 release deck targeted at the PyData Cardiff community.

All release decks
https://docs.rapids.ai/overview

Meetup
https://www.meetup.com/PyData-Cardiff-Meetup/events/268066478/

Abstract
The RAPIDS suite of open source software libraries (https://rapids.ai/) allow you to run data science and analytics pipelines entirely on GPUs, but following familiar Python APIs including Numpy, Pandas and SciKit Learn.

RAPIDS relies on NVIDIA® CUDA® primitives for low-level compute optimization, but exposes that GPU parallelism and high-bandwidth memory speed through user-friendly Python interfaces.

RAPIDS also focuses on common data preparation tasks for analytics and data science. This includes a familiar DataFrame API that integrates with a variety of machine learning algorithms for end-to-end pipeline accelerations without paying typical serialization costs. RAPIDS also includes support for multi-node, multi-GPU deployments, enabling vastly accelerated processing and training on much larger dataset sizes.

Ca3d0556227d66b3c15be1eadf69473b?s=128

Jacob Tomlinson

February 11, 2020
Tweet

Transcript

  1. Jacob Tomlinson Senior Software Engineer, RAPIDS Engineering Open GPU Data

    Science
  2. 2 Jacob Tomlinson

  3. 3 What is RAPIDS?

  4. 4 RAPIDS https://github.com/rapidsai

  5. 5 Data Processing Evolution Faster data access, less data movement

    25-100x Improvement Less code Language flexible Primarily In-Memory HDFS Read HDFS Write HDFS Read HDFS Write HDFS Read Query ETL ML Train HDFS Read Query ETL ML Train HDFS Read GPU Read Query CPU Write GPU Read ETL CPU Write GPU Read ML Train Arrow Read ETL ML Train 5-10x Improvement More code Language rigid Substantially on GPU 50-100x Improvement Same code Language flexible Primarily on GPU RAPIDS Traditional GPU Processing Hadoop Processing, Reading from disk Spark In-Memory Processing Query
  6. 6 Jake VanderPlas - PyCon 2017

  7. 7 Pandas Analytics CPU Memory Data Preparation Visualization Model Training

    Scikit-Learn Machine Learning NetworkX Graph Analytics PyTorch Chainer MxNet Deep Learning Matplotlib/Plotly Visualization Open Source Data Science Ecosystem Familiar Python APIs Dask
  8. 8 cuDF cuIO Analytics GPU Memory Data Preparation Visualization Model

    Training cuML Machine Learning cuGraph Graph Analytics PyTorch Chainer MxNet Deep Learning cuXfilter <> pyViz Visualization RAPIDS End-to-End Accelerated GPU Data Science Dask
  9. 9 Ecosystem Partners

  10. 10 Faster Speeds, Real-World Benefits cuIO/cuDF – Load and Data

    Preparation XGBoost Machine Learning Time in seconds (shorter is better) cuIO/cuDF (Load and Data Prep) Data Conversion XGBoost Benchmark 200GB CSV dataset; Data prep includes joins, variable transformations CPU Cluster Configuration CPU nodes (61 GiB memory, 8 vCPUs, 64-bit platform), Apache Spark DGX Cluster Configuration 5x DGX-1 on InfiniBand network 8762 6148 3925 3221 322 213 End-to-End
  11. 11 Technologies

  12. 12 RAPIDS End-to-End Accelerated GPU Data Science cuDF cuIO Analytics

    GPU Memory Data Preparation Visualization Model Training cuML Machine Learning cuGraph Graph Analytics PyTorch Chainer MxNet Deep Learning cuXfilter <> pyViz Visualization Dask
  13. 13 cuDF

  14. 14 cuDF cuIO Analytics GPU Memory Data Preparation Visualization Model

    Training cuML Machine Learning cuGraph Graph Analytics PyTorch Chainer MxNet Deep Learning cuXfilter <> pyViz Visualization RAPIDS GPU Accelerated data wrangling and feature engineering Dask
  15. 15 ETL - the Backbone of Data Science cuDF is…

    Python Library • A Python library for manipulating GPU DataFrames following the Pandas API • Python interface to CUDA C++ library with additional functionality • Creating GPU DataFrames from Numpy arrays, Pandas DataFrames, and PyArrow Tables • JIT compilation of User-Defined Functions (UDFs) using Numba
  16. 16 cuDF v0.10, Pandas 0.24.2 Running on NVIDIA DGX-1: GPU:

    NVIDIA Tesla V100 32GB CPU: Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz Benchmark Setup: DataFrames: 2x int32 columns key columns, 3x int32 value columns Merge: inner GroupBy: count, sum, min, max calculated for each value column Benchmarks: single-GPU Speedup vs. Pandas
  17. 17 • Follow Pandas APIs and provide >10x speedup •

    CSV Reader - v0.2, CSV Writer v0.8 • Parquet Reader – v0.7, Parquet Writer v0.12 • ORC Reader – v0.7, ORC Writer v0.10 • JSON Reader - v0.8 • Avro Reader - v0.9 • GPU Direct Storage integration in progress for bypassing PCIe bottlenecks! • Key is GPU-accelerating both parsing and decompression wherever possible Source: Apache Crail blog: SQL Performance: Part 1 - Input File Formats Extraction is the Cornerstone cuIO for Faster Data Loading
  18. 18 CuPy

  19. 19 GPU Memory Data Preparation Visualization Model Training RAPIDS Building

    bridges into the array ecosystem Dask cuDF cuIO Analytics cuML Machine Learning cuGraph Graph Analytics PyTorch Chainer MxNet Deep Learning cuXfilter <> pyViz Visualization
  20. 20

  21. 21 Benchmark: single-GPU CuPy vs NumPy More details: https://blog.dask.org/2019/06/27/single-gpu-cupy-benchmarks

  22. 22 SVD Benchmark Dask and CuPy Doing Complex Workflows

  23. 23 cuML

  24. 24 GPU Memory Data Preparation Visualization Model Training Dask Machine

    Learning More models more problems cuDF cuIO Analytics cuML Machine Learning cuGraph Graph Analytics PyTorch Chainer MxNet Deep Learning cuXfilter <> pyViz Visualization
  25. 25 Algorithms GPU-accelerated Scikit-Learn Classification / Regression Inference Clustering Decomposition

    & Dimensionality Reduction Time Series Decision Trees / Random Forests Linear Regression Logistic Regression K-Nearest Neighbors Support Vector Machine Classification Random forest / GBDT inference K-Means DBSCAN Spectral Clustering Principal Components Singular Value Decomposition UMAP Spectral Embedding T-SNE Holt-Winters Kalman Filtering ARIMA Cross Validation More to come! Hyper-parameter Tuning Key: • Preexisting • NEW or enhanced for 0.11
  26. 26 RAPIDS matches common Python APIs from sklearn.cluster import DBSCAN

    dbscan = DBSCAN(eps = 0.3, min_samples = 5) dbscan.fit(X) y_hat = dbscan.predict(X) Find Clusters from sklearn.datasets import make_moons import pandas X, y = make_moons(n_samples=int(1e2), noise=0.05, random_state=0) X = pandas.DataFrame({'fea%d'%i: X[:, i] for i in range(X.shape[1])}) CPU-Based Clustering
  27. 27 RAPIDS matches common Python APIs from cuml import DBSCAN

    dbscan = DBSCAN(eps = 0.3, min_samples = 5) dbscan.fit(X) y_hat = dbscan.predict(X) Find Clusters from sklearn.datasets import make_moons import cudf X, y = make_moons(n_samples=int(1e2), noise=0.05, random_state=0) X = cudf.DataFrame({'fea%d'%i: X[:, i] for i in range(X.shape[1])}) GPU-Accelerated Clustering
  28. 28 Benchmarks: single-GPU cuML vs scikit-learn 1x V100 vs. 2x

    20 core CPU
  29. 29 cuML’s Forest Inference Library accelerates prediction (inference) for random

    forests and boosted decision trees: • Works with existing saved models (XGBoost, LightGBM, scikit-learn RF cuML RF soon) • Lightweight Python API • Single V100 GPU can infer up to 34x faster than XGBoost dual-CPU node • Over 100 million forest inferences per sec (with 1000 trees) on a DGX-1 Forest Inference Taking models from training to production 23x 36x 34x 23x
  30. 30 cuGraph

  31. 31 GPU Memory Data Preparation Visualization Model Training Dask Graph

    Analytics More connections more insights cuDF cuIO Analytics cuML Machine Learning cuGraph Graph Analytics PyTorch Chainer MxNet Deep Learning cuXfilter <> pyViz Visualization
  32. 32 Algorithms GPU-accelerated NetworkX Community Components Link Analysis Link Prediction

    Traversal Structure Spectral Clustering Balanced-Cut Modularity Maximization Louvain Subgraph Extraction KCore and KCore Number Jaccard Weighted Jaccard Overlap Coefficient Single Source Shortest Path (SSSP) Breadth First Search (BFS) Triangle Counting COO-to-CSR (Multi-GPU) Transpose Multi-GPU More to come! Utilities Weakly Connected Components Strongly Connected Components Page Rank (Multi-GPU) Personal Page Rank Katz Query Language Renumbering
  33. 33 GOALS AND BENEFITS OF CUGRAPH Focus on Features and

    User Experience • Property Graph support via DataFrames Seamless Integration with cuDF and cuML • Up to 500 million edges on a single 32GB GPU • Multi-GPU support for scaling into the billions of edges Breakthrough Performance • Python: Familiar NetworkX-like API • C/C++: lower-level granular control for application developers Multiple APIs • Extensive collection of algorithm, primitive, and utility functions Growing Functionality
  34. 34 Benchmarks: single-GPU cuGraph vs NetworkX Dataset Nodes Edges preferentialAttachment

    100,000 999,970 caidaRouterLevel 192,244 1,218,132 coAuthorsDBLP 299,067 299,067 dblp-2010 326,186 1,615,400 citationCiteseer 268,495 2,313,294 coPapersDBLP 540,486 30,491,458 coPapersCiteseer 434,102 32,073,440 as-Skitter 1,696,415 22,190,596
  35. 35 cuSpatial

  36. 36 cuSpatial • cuDF for data loading, cuGraph for routing

    optimization, and cuML for clustering are just a few examples Seamless Integration into RAPIDS • Extensive collection of algorithm, primitive, and utility functions for spatial analytics Growing Functionality • Up to 1000x faster than CPU spatial libraries • Python and C++ APIs for maximum usability and integration Breakthrough Performance & Ease of Use
  37. 37 cuSpatial cuSpatial Operation Input data cuSpatial Runtime Reference Runtime

    Speedup Point-in-Polygon Test 1.3+ million vehicle point locations and 27 Region of Interests 1.11 ms (C++) 1.50 ms (Python) [Nvidia Titan V] 334 ms (C++, optimized serial) 130468.2 ms (python Shapely API, serial) [Intel i7-7800X] 301X (C++) 86,978X (Python) Haversine Distance Computation 13+ million Monthly NYC taxi trip pickup and drop-off locations 7.61 ms (Python) [Nvidia T4] 416.9 ms (Numba) [Nvidia T4] 54.7X (Python) Hausdorff Distance Computation (for clustering) 52,800 trajectories with 1.3+ million points 13.5s [Quadro V100] 19227.5s (Python SciPy API, serial) [Intel i7-6700K] 1,400X (Python) Performance at a Glance
  38. 38 Scaling and interoperability

  39. 39 Dask

  40. 40 cuDF cuIO Analytics GPU Memory Data Preparation Visualization Model

    Training cuML Machine Learning cuGraph Graph Analytics PyTorch Chainer MxNet Deep Learning cuXfilter <> pyViz Visualization RAPIDS Scaling RAPIDS with Dask Dask
  41. 41 Why Dask? • Easy Migration: Built on top of

    NumPy, Pandas Scikit-Learn, etc. • Easy Training: With the same APIs • Trusted: With the same developer community PyData Native • Easy to install and use on a laptop • Scales out to thousand-node clusters Easy Scalability • Most common parallelism framework today in the PyData and SciPy community Popular • HPC: SLURM, PBS, LSF, SGE • Cloud: Kubernetes, AWS, Azure • Hadoop/Spark: Yarn Deployable
  42. 42 Dask Dask scales arrays, dataframes and ML APIs

  43. 43 Scale up with RAPIDS Accelerated on single GPU NumPy

    -> CuPy/PyTorch/.. Pandas -> cuDF Scikit-Learn -> cuML Numba -> Numba RAPIDS and Others NumPy, Pandas, Scikit-Learn, Numba and many more Single CPU core In-memory data PyData Scale Up / Accelerate
  44. 44 Scale out with RAPIDS + Dask with OpenUCX Accelerated

    on single GPU NumPy -> CuPy/PyTorch/.. Pandas -> cuDF Scikit-Learn -> cuML Numba -> Numba RAPIDS and Others Multi-GPU On single Node (DGX) Or across a cluster RAPIDS + Dask with OpenUCX Scale Up / Accelerate Scale out / Parallelize NumPy, Pandas, Scikit-Learn, Numba and many more Single CPU core In-memory data PyData Multi-core and Distributed PyData NumPy -> Dask Array Pandas -> Dask DataFrame Scikit-Learn -> Dask-ML … -> Dask Futures Dask
  45. 45 CUDA Array Interface

  46. 46 APP A Data Movement and Transformation Data Movement and

    Transformation The bane of productivity and performance CPU APP B Copy & Convert Copy & Convert Copy & Convert APP A GPU Data APP B GPU Data Read Data Load Data APP B APP A GPU
  47. 47 Data Movement and Transformation Data Movement and Transformation What

    if we could keep data on the GPU? APP A APP B Copy & Convert Copy & Convert Copy & Convert Read Data Load Data CPU APP A GPU Data APP B GPU Data APP B APP A GPU Copy & Convert
  48. 48 Interoperability for the Win DLPack and __cuda_array_interface__ mpi4py

  49. 49 Array function and dispatching

  50. 50 NEP18 import numpy as np import cupy x =

    np.random.random((10000, 1000)) y = cupy.array(x) u, s, v = np.linalg.svd(x) # 3min 11s u, s, v = np.linalg.svd(y) # 19.1 s Numpy __array_function__ protocol
  51. 51 UCX

  52. 52 Why OpenUCX? • TCP sockets are slow! • UCX

    provides uniform access to transports (TCP, InfiniBand, shared memory, NVLink) • Alpha Python bindings for UCX (ucx-py) • Will provide best communication performance, to Dask based on available hardware on nodes/cluster Bringing hardware accelerated communications to Dask conda install -c conda-forge -c rapidsai \ cudatoolkit=<CUDA version> ucx-proc=*=gpu ucx ucx-py
  53. 53 cuDF v0.11, UCX-PY 0.11 Running on NVIDIA DGX-2: GPU:

    NVIDIA Tesla V100 32GB CPU: Intel(R) Xeon(R) CPU 8168 @ 2.70GHz Benchmark Setup: DataFrames: Left/Right 1x int64 column key column, 1x int64 value columns Merge: inner 30% of matching data balanced across each partition Benchmarks: Distributed cuDF Random Merge
  54. 54 Getting started

  55. 55 RAPIDS Docs https://docs.rapids.ai

  56. 56 Easy Installation Interactive Installation Guide

  57. 57

  58. 58 Deploy RAPIDS Everywhere Focused on robust functionality, deployment, and

    user experience Integration with major cloud providers Both containers and cloud specific machine instances Support for Enterprise and HPC Orchestration Layers Cloud Dataproc Azure Machine Learning
  59. 59 Join the Movement Everyone can help! Integrations, feedback, documentation

    support, pull requests, new issues, or code donations welcomed! APACHE ARROW GPU Open Analytics Initiative https://arrow.apache.org/ @ApacheArrow http://gpuopenanalytics.com/ @GPUOAI RAPIDS https://rapids.ai @RAPIDSAI Dask https://dask.org @Dask_dev
  60. THANK YOU Jacob Tomlinson @_jacobtomlinson jtomlinson@nvidia.com