Docker and Deep Learning

Presentation courtesy of Bennett Wineholt.

Scaling up systems for resource-intensive machine learning tasks demands convenient methods to manage computations distributed across multiple servers. Come and learn about both the processes underlying new Deep Learning techniques that have been applied to piloting drones and driving autonomous vehicles as well as the Docker containerization tools used to train these systems at scale.

Presented at SSW: https://cornell-ssw.github.io/meetings/2017-04-17

Transcript

1. Docker and Deep Learning
2. Summary What is Docker? What is Deep Learning? How can I use Deep Learning today?
3. Outline • What is Docker? ◦ run code in containers ◦ isolated virtual environment ◦ automate setup for repeatability • What is Deep Learning? ◦ example functions and applications • How does Deep Learning work? ◦ take in lots of example input - output pairs ◦ learn a function, there are many techniques • How can I use Deep Learning today? ◦ Demo
4. What is Docker?
5. In their own words
6. Docker containers are Linux in a box
7. Why? Example: Run lots of identical web servers
8. Why? Example: Run lots of machine learning
9. Why? Example: Run research code locally then run the same thing on Google Cloud Engine
10. The applications in the box are up to you
11. How? Containers use host server resources Docker runs application code inside containers The Docker daemon (within the illustrated Container Engine) controls how containers can access resources of the host like disk to store programs or data and CPU to execute code
12. How? Resource Isolation Think of a container like a box inside of which you can run any flavor of the Linux operating system The Docker daemon controls access to host resources like restricting how containers can access networking functions to send and receive traffic Namespaces prevent process interference
13. Vagrant vs Docker
14. Docker saves space
15. Docker saves time* Compared to virtual machines, containers • startup faster • use only resources needed at that time • support direct hardware access *Overall performance depends on the application
16. Docker saves you time Building a Docker container that you can run anywhere saves you time setting up servers Using one that someone on your team (or out on the world wide web) already built and published can save you a lot of time and effort
17. First make a Dockerfile Specify a base image like your favorite Linux OS Write & install code then build the container and next run the application That makes one Dockerfile which describes how to run some application(s) in a container
18. Test and publish your container You can run containers on your own laptop after installing the Docker Engine for your OS Then push it to a registry Later pull the container image file down to any host server running Docker to run anywhere
19. Make a docker-compose file Write a docker-compose file which can pick multiple containers to run on one host server For example we can start up an in-memory network cache as well as a web server
20. Make a Docker Swarm service Write a Docker Swarm service which can run containers automatically on many Docker host servers provisioned using Docker Machine For example we can deploy a web server application to many server nodes in some cloud
21. Reuse setup to run on the cloud
22. What gets run on Docker?
23. What is Deep Learning?
24. What is Deep Learning?
25. Deep Neural Network Training and Inference
26. Example Function : Digit Classification a training dataset with lots of labeled examples presented individually handwritten digits to integers 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ...
27. Train Handwritten Digit Classification Model
28. Digit Classification Model Inference given new input, infer what digit we are seeing image of digit to predicted integer 3
29. What else can Deep Learning do? ◦ Object Recognition ◦ Image Captioning ◦ Obstacle Detection ◦ Speech Recognition ◦ Scene Recognition ◦ Reinforcement Learning ◦ … etc.
30. Example Function: Object Recognition image pixels to semantic object labels and bounding boxes
31. Example Function: Image Captioning image pixels to semantic object labels bounding boxes and image caption
32. Example: Obstacle Detection Online learning to iteratively refine the function model NVidia Jetson TK1 GPU on a drone
33. Example: Speech Recognition mic recorded sound to words The training examples should be in the same language, otherwise there might not be a good match This is a fine day today.
34. Example: Scene Segmentation Fidler et al. Scene Tutorial Reconstruction, Localization, Semantics in RGB-D CVPR'15 Depth camera video data stream to segmented 3D map
35. Example: Coordinated Reinforcement Learning RL sharing knowledge between many learning systems to grasp and pick up objects Behaviors that successfully pick up objects are rewarded and performed more often, this is positive reinforcement Levine et al. Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection 2016
36. Example: Learning Robot Motion Strategy Motor dynamics are known, robot learns movements to pick up and move objects Wong and Takahashi et al. Self-Supervised Deep Visuomotor Learning from Motor Unit Feedback 2016
37. Example: AlphaGo Training policy network suggests the best next move trained on a large history of games value network predicts probability of winning given some position, trained similarly
38. Example: AlphaGo Inference trained networks are used to search tree of possible futures for the game
39. How does Deep Learning work?
40. Training a function from input data
41. Supervised Training with Backpropagation 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0 1 1 1 1 1 2 1 7 1 1 1 1 1 1 ...
42. Supervised Training Sees a Misclassification 2 ≠ 1
43. Supervised Training Corrected Prediction 1 = 1
44. Deep Learning likes matrices Wu et al. Enabling Ubiquitous Visual Intelligence Through Deep Learning Baidu 2015
45. Deep Learning does not like the unexpected
46. Training takes a lot of input data
47. Training produces a model - large and slow
48. Training and Inference
49. Evaluate learned model to perform inference
50. Inference might require lots of hardware NVidia Titan X graphics card
51. Inference might be lightweight Compressed models can preserve accuracy and run on mobile platforms NVidia Jetson TK1 GPU on a drone
52. How can I use Deep Learning today?
53. How can we get started? For supervised learning such as classification and object detection ◦ Pick a framework ◦ Utilize pre-trained models -or- ◦ Record/download data to train our own models ◦ Make evaluations of the model on new data
54. Popular frameworks • Tensorflow • Caffe • Theano • Torch • MXNet • Matlab Deep Learning • and more
55. Download pre-trained network models
56. Or train your own rental GPU instances This will be in the demo
57. Summary What is Docker? • virtualized containers for isolated and repeatable code execution What is Deep Learning? • deep neural network function approximation driven by big data How can I use Deep Learning today? • demo time
58. Thanks
59. Special Thanks Cornell Center for Advanced Computing Adrian Sampson and the Cornell Computer Systems Lab Reading Group for Deep Learning Hardware Song Han from Stanford University
60. References • Docker Overview • Edureka What is Docker • Vagrant Vs Docker by Jonathan Chase • Docker vs VMs Christopher Tozzi • https://stanford.edu/~songhan/ • https://www.dropbox.com/s/p7lvelt0aihrwtl/FPGA%2717%20tutorial%20Song%20Han.pdf?dl=0 • https://arxiv.org/abs/1602.07360 • https://www.slideshare.net/embeddedvision/enabling-ubiquitous-visual-intelligence-through-deep-le arning-a-keynote-presentation-from-baidu • Fidler et al. Scene Tutorial Reconstruction, Localization, Semantics in RGB-D CVPR'15 • A 'Brief' History of Game AI Up To AlphaGo, Andrey Kurenkov • Zhang et al Efficient and Accurate Approximations of Nonlinear Convolutional Networks CVPR’15 • http://www.neurala.com
61. References cont. • https://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-imagenet-challenge-upc-2016 • https://pdollar.wordpress.com/2015/01/21/image-captioning/ • https://github.com/BVLC/caffe/wiki/Model-Zoo • https://github.com/tensorflow/models • http://deeplearning.net/tutorial/gettingstarted.html • http://deeplearning.net/tutorial/gettingstarted.html#early-stopping • https://arxiv.org/abs/1603.02199 • https://arxiv.org/abs/1501.02530 • https://en.wikipedia.org/wiki/Backpropagation • Goodfellow et al. 2016 http://www.deeplearningbook.org/ • Where's the Bear Elias et al. https://www.cs.ucsb.edu/sites/cs.ucsb.edu/files/docs/reports/tr.pdf
62. Image and Figure References Testing Docker for Mac by Dave Kerr Edureka What is Docker InfoWorld Containers 101 Swarm Goes Stable by Docker Inc Docker community ecosystem Docker Compose Example by Racquel Pau Continuous Delivery with Docker by Julia Mateo See Song Han FPGA '17 Tutorial in References above See listed References for selected figures ImageNet Wikimedia Commons Flickr ResearchGate Zhang et al. ResearchGate Abedini et al. Nextxen Jetsonhacks Robohub VGG in Tensorflow by Davi Frossard RSIP Vision
63. Author Bennett Wineholt Center for Advanced Computing Cornell University Scientific Software Club at Cornell 04/17/2017
64. Demo Time