Jacob Tomlinson | Bristech Nov 2021 Intro to RAPIDS and GPU development in Python 

No content

3 RAPIDS Github https://github.com/rapidsai 

4 Jake VanderPlas - PyCon 2017 

5 Pandas Analytics CPU Memory Data Preparation Visualization Model Training Scikit-Learn Machine Learning NetworkX Graph Analytics PyTorch, TensorFlow, MxNet Deep Learning Matplotlib Visualization Dask Open Source Data Science Ecosystem Familiar Python APIs 

6 cuDF cuIO Analytics GPU Memory Data Preparation Visualization Model Training cuML Machine Learning cuGraph Graph Analytics PyTorch, TensorFlow, MxNet Deep Learning cuxfilter, pyViz, plotly Visualization Dask RAPIDS End-to-End Accelerated GPU Data Science 

7 Dask EASY SCALABILITY ▸ Easy to install and use on a laptop ▸ Scales out to thousand node clusters ▸ Modularly built for acceleration DEPLOYABLE ▸ HPC: SLURM, PBS, LSF, SGE ▸ Cloud: Kubernetes ▸ Hadoop/Spark: Yarn PYDATA NATIVE ▸ Easy Migration: Built on top of NumPy, Pandas Scikit-Learn, etc ▸ Easy Training: With the same API POPULAR ▸ Most Common parallelism framework today in the PyData and SciPy community ▸ Millions of monthly Downloads and Dozens of Integrations NumPy, Pandas, Scikit-Learn, Numba and many more Single CPU core In-memory data PYDATA Multi-core and distributed PyData NumPy -> Dask Array Pandas -> Dask DataFrame Scikit-Learn -> Dask-ML … -> Dask Futures DASK Scale Out / Parallelize 

8 Accelerated on single GPU NumPy -> CuPy/PyTorch/.. Pandas -> cuDF Scikit-Learn -> cuML NetworkX -> cuGraph Numba -> Numba RAPIDS AND OTHERS Multi-GPU On single Node (DGX) Or across a cluster RAPIDS + DASK WITH OPENUCX NumPy, Pandas, Scikit-Learn, Numba and many more Single CPU core In-memory data PYDATA Multi-core and distributed PyData NumPy -> Dask Array Pandas -> Dask DataFrame Scikit-Learn -> Dask-ML … -> Dask Futures DASK Scale Up / Accelerate Scale Out / Parallelize Scale Out with RAPIDS + Dask with OpenUCX 

9 Time in seconds (shorter is better) cuIO/cuDF (Load and Data Prep) Data Conversion XGBoost Faster Speeds, Real World Benefits Faster Data Access, Less Data Movement cuIO/cuDF – Load and Data Preparation XGBoost Machine Learning End-to-End Benchmark 200GB CSV dataset; Data prep includes joins, variable transformations CPU Cluster Configuration CPU nodes (61 GiB memory, 8 vCPUs, 64-bit platform), Apache Spark RAPIDS Version RAPIDS 0.17 A100 Cluster Configuration 16 A100 GPUs (40GB each) 

10 What are GPUs? 

11 Gaming Hardware For pwning n00bs 

12 Mysterious Machine Learning Hardware For things like GauGAN Semantic Image Synthesis with Spatially-Adaptive Normalization Taesung Park, Ming-Yu Liu, Ting-Chun Wang, Jun-Yan Zhu arXiv:1903.07291 [cs.CV] 

13 CPU GPU 

14 https://youtu.be/-P28LKWTzrI 

15 GPU vs CPU https://docs.nvidia.com/cuda/cuda-c-programming-guide/ 

16 Using a GPU is like using two computers Icons made by Freepik from Flaticon Network SSH VNC RD SCP FTP SFTP Robocopy 

17 Using a GPU is like using two computers PCI CUDA 

18 What is CUDA? 

19 CUDA 

20 I don’t write C/C++ 

21 What does CUDA do? Construct GPU code with CUDA C/C++ language extensions Copy data from RAM to GPU Copy compiled code to GPU Execute code Copy data from GPU to RAM How do we run stuff on the GPU? 

22 Let’s do it in Python 

23 

24 

25 Live coding 󰛢 

26 Writing a Kernel Differences between a kernel and a function ● A kernel cannot return anything, it must instead modify memory ● A kernel must specify its thread hierarchy (threads and blocks) A kernel is a GPU function 

27 Threads, blocks, grids and warps https://docs.nvidia.com/cuda/cuda-c-programming-guide/ 

28 What? Rules of thumb for threads per block: ● Should be a round multiple of the warp size (32) ● A good place to start is 128-512 but benchmarking is required to determine the optimal value. 

29 Imports 

30 Data arrays 

31 Example kernel 

32 Running the kernel 

33 Absolute positions 

34 Thread and block positions 

35 Example kernel (again) 

36 How did the GPU update our numpy array? If you call a Numba CUDA kernel with data that isn’t on the GPU it will be copied to the GPU before running the kernel and copied back after. This isn’t always ideal as copying data can waste time. 

37 Create a GPU array 

38 Simplified position kernel 

39 GPU Array 

40 Copy to host 

41 Higher level APIs 

42 

43 cuDF 

44 cuDF cuIO Analytics Data Preparation Visualization Model Training cuML Machine Learning cuGraph Graph Analytics PyTorch, TensorFlow, MxNet Deep Learning cuxfilter, pyViz, plotly Visualization Dask GPU Memory RAPIDS End-to-End GPU Accelerated Data Science 

45 Interoperability for the Win DLPack and __cuda_array_interface__ mpi4py 

46 __cuda_array_interface__ 

47 Recap 

48 Recap GPUs run the same function (kernel) many times in parallel When being called each function gets a unique index CUDA/C++ is used to write kernels, but high level languages like Python can also compile to it Memory must be copied between the CPU (host) and GPU (device) Many familiar Python APIs have GPU accelerated implementations to abstract all this away Takeaways on GPU computing 

THANK YOU Jacob Tomlinson @_jacobtomlinson