Docker and Deep Learning

Docker and Deep Learning

Presentation courtesy of Bennett Wineholt.

Scaling up systems for resource-intensive machine learning tasks demands convenient methods to manage computations distributed across multiple servers. Come and learn about both the processes underlying new Deep Learning techniques that have been applied to piloting drones and driving autonomous vehicles as well as the Docker containerization tools used to train these systems at scale.

Presented at SSW: https://cornell-ssw.github.io/meetings/2017-04-17

Cd1ce6ba8c288b05df99982f6f7097dc?s=128

CUSSW Hosted

April 17, 2017
Tweet

Transcript

  1. Docker and Deep Learning

  2. Summary What is Docker? What is Deep Learning? How can

    I use Deep Learning today?
  3. Outline • What is Docker? ◦ run code in containers

    ◦ isolated virtual environment ◦ automate setup for repeatability • What is Deep Learning? ◦ example functions and applications • How does Deep Learning work? ◦ take in lots of example input - output pairs ◦ learn a function, there are many techniques • How can I use Deep Learning today? ◦ Demo
  4. What is Docker?

  5. In their own words

  6. Docker containers are Linux in a box

  7. Why? Example: Run lots of identical web servers

  8. Why? Example: Run lots of machine learning

  9. Why? Example: Run research code locally then run the same

    thing on Google Cloud Engine
  10. The applications in the box are up to you

  11. How? Containers use host server resources Docker runs application code

    inside containers The Docker daemon (within the illustrated Container Engine) controls how containers can access resources of the host like disk to store programs or data and CPU to execute code
  12. How? Resource Isolation Think of a container like a box

    inside of which you can run any flavor of the Linux operating system The Docker daemon controls access to host resources like restricting how containers can access networking functions to send and receive traffic Namespaces prevent process interference
  13. Vagrant vs Docker

  14. Docker saves space

  15. Docker saves time* Compared to virtual machines, containers • startup

    faster • use only resources needed at that time • support direct hardware access *Overall performance depends on the application
  16. Docker saves you time Building a Docker container that you

    can run anywhere saves you time setting up servers Using one that someone on your team (or out on the world wide web) already built and published can save you a lot of time and effort
  17. First make a Dockerfile Specify a base image like your

    favorite Linux OS Write & install code then build the container and next run the application That makes one Dockerfile which describes how to run some application(s) in a container
  18. Test and publish your container You can run containers on

    your own laptop after installing the Docker Engine for your OS Then push it to a registry Later pull the container image file down to any host server running Docker to run anywhere
  19. Make a docker-compose file Write a docker-compose file which can

    pick multiple containers to run on one host server For example we can start up an in-memory network cache as well as a web server
  20. Make a Docker Swarm service Write a Docker Swarm service

    which can run containers automatically on many Docker host servers provisioned using Docker Machine For example we can deploy a web server application to many server nodes in some cloud
  21. Reuse setup to run on the cloud

  22. What gets run on Docker?

  23. What is Deep Learning?

  24. What is Deep Learning?

  25. Deep Neural Network Training and Inference

  26. Example Function : Digit Classification a training dataset with lots

    of labeled examples presented individually handwritten digits to integers 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ...
  27. Train Handwritten Digit Classification Model

  28. Digit Classification Model Inference given new input, infer what digit

    we are seeing image of digit to predicted integer 3
  29. What else can Deep Learning do? ◦ Object Recognition ◦

    Image Captioning ◦ Obstacle Detection ◦ Speech Recognition ◦ Scene Recognition ◦ Reinforcement Learning ◦ … etc.
  30. Example Function: Object Recognition image pixels to semantic object labels

    and bounding boxes
  31. Example Function: Image Captioning image pixels to semantic object labels

    bounding boxes and image caption
  32. Example: Obstacle Detection Online learning to iteratively refine the function

    model NVidia Jetson TK1 GPU on a drone
  33. Example: Speech Recognition mic recorded sound to words The training

    examples should be in the same language, otherwise there might not be a good match This is a fine day today.
  34. Example: Scene Segmentation Fidler et al. Scene Tutorial Reconstruction, Localization,

    Semantics in RGB-D CVPR'15 Depth camera video data stream to segmented 3D map
  35. Example: Coordinated Reinforcement Learning RL sharing knowledge between many learning

    systems to grasp and pick up objects Behaviors that successfully pick up objects are rewarded and performed more often, this is positive reinforcement Levine et al. Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection 2016
  36. Example: Learning Robot Motion Strategy Motor dynamics are known, robot

    learns movements to pick up and move objects Wong and Takahashi et al. Self-Supervised Deep Visuomotor Learning from Motor Unit Feedback 2016
  37. Example: AlphaGo Training policy network suggests the best next move

    trained on a large history of games value network predicts probability of winning given some position, trained similarly
  38. Example: AlphaGo Inference trained networks are used to search tree

    of possible futures for the game
  39. How does Deep Learning work?

  40. Training a function from input data

  41. Supervised Training with Backpropagation 0 0 0 0 0 0

    8 0 0 0 0 0 0 0 0 1 1 1 1 1 2 1 7 1 1 1 1 1 1 ...
  42. Supervised Training Sees a Misclassification 2 ≠ 1

  43. Supervised Training Corrected Prediction 1 = 1

  44. Deep Learning likes matrices Wu et al. Enabling Ubiquitous Visual

    Intelligence Through Deep Learning Baidu 2015
  45. Deep Learning does not like the unexpected

  46. Training takes a lot of input data

  47. Training produces a model - large and slow

  48. Training and Inference

  49. Evaluate learned model to perform inference

  50. Inference might require lots of hardware NVidia Titan X graphics

    card
  51. Inference might be lightweight Compressed models can preserve accuracy and

    run on mobile platforms NVidia Jetson TK1 GPU on a drone
  52. How can I use Deep Learning today?

  53. How can we get started? For supervised learning such as

    classification and object detection ◦ Pick a framework ◦ Utilize pre-trained models -or- ◦ Record/download data to train our own models ◦ Make evaluations of the model on new data
  54. Popular frameworks • Tensorflow • Caffe • Theano • Torch

    • MXNet • Matlab Deep Learning • and more
  55. Download pre-trained network models

  56. Or train your own rental GPU instances This will be

    in the demo
  57. Summary What is Docker? • virtualized containers for isolated and

    repeatable code execution What is Deep Learning? • deep neural network function approximation driven by big data How can I use Deep Learning today? • demo time
  58. Thanks

  59. Special Thanks Cornell Center for Advanced Computing Adrian Sampson and

    the Cornell Computer Systems Lab Reading Group for Deep Learning Hardware Song Han from Stanford University
  60. References • Docker Overview • Edureka What is Docker •

    Vagrant Vs Docker by Jonathan Chase • Docker vs VMs Christopher Tozzi • https://stanford.edu/~songhan/ • https://www.dropbox.com/s/p7lvelt0aihrwtl/FPGA%2717%20tutorial%20Song%20Han.pdf?dl=0 • https://arxiv.org/abs/1602.07360 • https://www.slideshare.net/embeddedvision/enabling-ubiquitous-visual-intelligence-through-deep-le arning-a-keynote-presentation-from-baidu • Fidler et al. Scene Tutorial Reconstruction, Localization, Semantics in RGB-D CVPR'15 • A 'Brief' History of Game AI Up To AlphaGo, Andrey Kurenkov • Zhang et al Efficient and Accurate Approximations of Nonlinear Convolutional Networks CVPR’15 • http://www.neurala.com
  61. References cont. • https://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-imagenet-challenge-upc-2016 • https://pdollar.wordpress.com/2015/01/21/image-captioning/ • https://github.com/BVLC/caffe/wiki/Model-Zoo • https://github.com/tensorflow/models

    • http://deeplearning.net/tutorial/gettingstarted.html • http://deeplearning.net/tutorial/gettingstarted.html#early-stopping • https://arxiv.org/abs/1603.02199 • https://arxiv.org/abs/1501.02530 • https://en.wikipedia.org/wiki/Backpropagation • Goodfellow et al. 2016 http://www.deeplearningbook.org/ • Where's the Bear Elias et al. https://www.cs.ucsb.edu/sites/cs.ucsb.edu/files/docs/reports/tr.pdf
  62. Image and Figure References Testing Docker for Mac by Dave

    Kerr Edureka What is Docker InfoWorld Containers 101 Swarm Goes Stable by Docker Inc Docker community ecosystem Docker Compose Example by Racquel Pau Continuous Delivery with Docker by Julia Mateo See Song Han FPGA '17 Tutorial in References above See listed References for selected figures ImageNet Wikimedia Commons Flickr ResearchGate Zhang et al. ResearchGate Abedini et al. Nextxen Jetsonhacks Robohub VGG in Tensorflow by Davi Frossard RSIP Vision
  63. Author Bennett Wineholt Center for Advanced Computing Cornell University Scientific

    Software Club at Cornell 04/17/2017
  64. Demo Time