Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Accelerated Deep Learning - Part 2

Accelerated Deep Learning - Part 2

Slide deck of Jeremy Purches and Alison Lowndes talk at the Deep Learning London event on 04 Nov 2015. Talk was focused on explaining how graphical processing units (GPUs) enable various deep learning techniques. He will include use cases across a wide area of industry plus the latest news on NVIDIAs toolkits and software, including DIGITS, their open-source Deep Learning platform. Further information can be found here: https://developer.nvidia.com/deep-learning

Deep Learning London Meetup

November 04, 2015
Tweet

More Decks by Deep Learning London Meetup

Other Decks in Technology

Transcript

  1. 2 “ using deep learning to understand the genome and

    how genetic mutations lead to disease takes deep learning to a new level: understanding things that human’s cannot” Dr Brendan Frey, CEO Deep Genomics
  2. 6 Tree Cat Dog Machine Learning Software “turtle” Forward Propagation

    Compute weight update to nudge from “turtle” towards “dog” Backward Propagation Trained Model “cat” Repeat Training Inference
  3. 9 Long short-term memory (LSTM) Hochreiter (1991) analysed vanishing gradient

    “LSTM falls out of this almost naturally” Gates control importance of the corresponding activations Training via backprop unfolded in time LSTM: input gate output gate Long time dependencies are preserved until input gate is closed (-) and forget gate is open (O) forget gate Fig from Vinyals et al, Google April 2015 NIC Generator Fig from Graves, Schmidhuber et al, Supervised Sequence Labelling with RNNs
  4. 10 GPUs and Deep Learning GPUs deliver -- - same

    or better prediction accuracy - faster results - smaller footprint - lower power 72% 74% [VALUE ] 88% 93% 2010 2011 2012 2013 2014 ImageNet Challenge Accuracy NVIDIA CUDA GPU NEURAL NETWORKS GPUS Inherently Parallel   Matrix Operations   FLOPS   Bandwidth  
  5. 11 NVIDIA DEEP LEARNING PLATFORM DEPLOYMENT Hardware Systems Software DEVELOPMENT

    Systems Software Hardware Titan X Tesla DIGITS DevBox cuDNN Applications DIGITS Tools Deep Learning Frameworks System Management
  6. 12 • From development through deployment • Highly optimized multi-GPU

    training • Major deep learning frameworks GPU-accelerated • Supports embedded and the cloud Jetson GPU HW Cloud Tesla Titan X DIGITS Visualize Layers Configure Network Process Data Monitor Progress Theano Torch Caffe cuDNN, cuBLAS CUDA Deep Learning Solutions Design, train, and deploy deep neural networks
  7. 13 NVIDIA DIGITS Studio Interactive Deep Learning GPU Training System

    Test Image Monitor Progress Configure DNN Process Data Visualize Layers http://developer.nvidia.com/digits
  8. 14 Automatic Multi-GPU Training Automatic multi-GPU scaling up to 4

    GPUs DIGITS 2 Interactive Deep Learning Training System DIGITS 2 performance vs. previous version on an NVIDIA DIGITS DevBox system 0.0x 0.5x 1.0x 1.5x 2.0x 2.5x 1-GPU 2-GPUs 4-GPUs DIGITS 2 Trains Models up to 2x Faster with Multi-GPU Scaling
  9. 15 cuDNN Deep Learning Primitives IGNITING ARTIFICIAL INTELLIGENCE  GPU-accelerated

    Deep Learning subroutines  High performance neural network training  Accelerates Major Deep Learning frameworks: Caffe, Theano, Torch  Up to 2x faster on AlexNet than baseline GPU implementation NVIDIA GPU cuDNN Frameworks Applications 0 20 40 60 80 cuDNN 1 cuDNN 2 cuDNN 3 Millions of images trained per day 0.0x 0.5x 1.0x 1.5x 2.0x 2.5x Alexnet OverFeat VGG Train 2x faster than cuDNN 2
  10. 16 GPU Accelerated Deep Learning Frameworks CAFFE TORCH THEANO MINERVA

    KALDI Deep Learning Framework Scientific Computing Framework Math Expression Compiler Deep Learning Framework Speech RecognitionToolkit cuDNN 3 3 3 3 -- Multi-GPU   In Progress  (nnet2) Multi-Node     (nnet2) License BSD-2 BSD BSD Apache 2.0 Apache 2.0 Interface(s) Text-based definition files, Python, MATLAB Python, Lua, MATLAB Python C++ C++, Shell scripts Embedded     
  11. 17

  12. 18 Practical Examples of Deep Learning Image Classification, Object Detection,

    Localization, Action Recognition Speech Recognition, Speech Translation, Natural Language Processing Breast Cancer Cell Mitosis Detection, Volumetric Brain Image Segmentation Pedestrian Detection, Lane Detection, Traffic Sign Recognition
  13. 19 DEEP LEARNING REVOLUTIONIZING MEDICAL RESEARCH Detecting Mitosis in Breast

    Cancer Cells — IDSIA Predicting the Toxicity of New Drugs — Johannes Kepler University Understanding Gene Mutation to Prevent Disease — University of Toronto Molecular Activity Prediction for Drug Discovery — Merck
  14. 20 April 4-7, 2016 | Silicon Valley | #GTC16 www.gputechconf.com

    CONNECT Connect with technology experts from NVIDIA and other leading organizations LEARN Gain insight and hands-on training through the hundreds of sessions and research posters DISCOVER See how GPU technologies are creating amazing breakthroughs in important fields such as deep learning INNOVATE Hear about disruptive innovations as early-stage companies and startups present their work The world’s most important event for GPU developers