Deep Learning at Scale

Presented by Alex Kern (CTO and Cofounder at Pavlov) at Big Data Day LA 2016.

The advent of modern deep learning techniques has given organizations new tools to understand, query, and structure their data. However, maintaining complex pipelines, versioning models, and tracking accuracy regressions over time remain ongoing struggles of even the most advanced data engineering teams. This talk presents a simple architecture for deploying machine learning at scale and offer suggestions for how companies can get their feet wet with open source technologies they already deploy.

Transcript

1. Deep Learning at Scale
2. Hi, I’m Kern. @kern @KernCanCode
4. deep learning?!
5. AlphaGo achieves honorary 9 dan rank.
6. Google’s DeepDream or: scary fish-dog things everywhere
7. deep learning is a kind of machine learning.
8. machine learning is like pattern recognition. (kind of)
9. deep learning has been applied to images, text, audio, video, and games with great success!
10. DEEP
 CONVOLUTIONAL
 NEURAL
 NETWORKS
11. 14.2 million images
12. Challenge Winners top-5 classification error (lower is better) 0 0.075 0.15 0.225 0.3 0.036 0.05 0.074 0.112 0.153 0.258 0.282 2010 2011 2012 2014 2013 Human 2015
13. 0 0.075 0.15 0.225 0.3 0.036 0.05 0.074 0.112 0.153 0.258 0.282 2010 2011 2012 2014 2013 Human 2015 Challenge Winners top-5 classification error (lower is better)
14. 0 0.075 0.15 0.225 0.3 0.036 0.05 0.074 0.112 0.153 0.258 0.282 2010 2011 2012 2014 2013 Human 2015 Challenge Winners top-5 classification error (lower is better) BEFORE DEEP LEARNING AFTER DEEP LEARNING
15. 0 0.075 0.15 0.225 0.3 0.036 0.05 0.074 0.112 0.153 0.258 0.282 2010 2011 2012 2014 2013 Human 2015 Challenge Winners top-5 classification error (lower is better) }much wow AlexNet BEFORE DEEP LEARNING AFTER DEEP LEARNING
16. 0 0.075 0.15 0.225 0.3 0.036 0.05 0.074 0.112 0.153 0.258 0.282 2010 2011 2012 2014 2013 Human 2015 Challenge Winners top-5 classification error (lower is better) }much wow AlexNet BEFORE DEEP LEARNING AFTER DEEP LEARNING
18. how does deep learning work?
20. it’s easy! lol just kidding
21. it’s calculus, but you don’t need to know much.
22. label score dog 97.1% shiba inu 2.5% meme 0.4% neural networks are classifiers
23. supervised learning inferring a function from labeled training data Wikipedia
24. you provide “ground truth” examples and labels.
25. perceptrons input output weights
26. is this a dog? T F features classes weights
27. is this a dog? T F features classes weights
28. is this a dog? T F features classes weights
29. PERCEPTRONS ARE TOO LINEAR WE MUST GO DEEPER
30. deep convolutional neural networks affectionately known as “convnets” input output convolution, activation, & pooling layers fully-connected layers (perceptrons)
31. how are the optimal weights determined?
32. stochastic gradient descent find the minimum error
33. training takes a while. anywhere from 8hrs to 2wks
34. how do you scale deep learning?
35. data collection & cleaning more clean data, the better 1
36. data collection & cleaning more clean data, the better 1 2 model training & selection anywhere from 8hrs to 2wks
37. data collection & cleaning more clean data, the better 1 2 model training & selection anywhere from 8hrs to 2wks 3 serving in production with real-time or batch requests
38. data collection & cleaning more clean data, the better 1 ∞ rinse & repeat keep models fresh with new data 2 model training & selection anywhere from 8hrs to 2wks 3 serving in production with real-time or batch requests
39. you’re gonna need GPUs for NVIDIA CUDA & cuDNN
40. NVIDIA GeForce GTX TITAN X is the gold standard
41. AWS G2 instances mid-tier on-demand GPUs Model GPUs vCPU Mem (GiB) SSD Storage (GB) g2.2xlarge 1 8 15 1 x 60 g2.8xlarge 4 32 60 2 x 120
42. AWS G2 GPUs are also great for model serving.
43. use HDFS + Spark for data storage and processing
44. TensorFlow with Keras is a good choice.
45. from keras.models import Sequential from keras.layers import Dense, Activation model = Sequential() model.add(Dense(output_dim=64, input_dim=100)) model.add(Activation("relu")) model.add(Dense(output_dim=10)) model.add(Activation("softmax")) model.compile( loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy']) model.fit(X_train, Y_train, nb_epoch=5, batch_size=32)
46. distributed training with CaffeOnSpark https://github.com/yahoo/CaffeOnSpark
47. it’s often better to peer high-end GPUs on a single machine.
48. parallelize training on hyperparameters. (e.g. learning rate, momentum)
49. hyperparamter optimization with MOE, hyperopt, or custom scripts hyperopt MOE custom scripts
50. wrapping up
51. get your feet wet TensorFlow MNIST Walkthrough bit.ly/pavlovtensor Andrej Karpathy’s CS231n bit.ly/pavlov231
52. suggested technologies • Neural Network Libraries • Caffe & CaffeOnSpark • TensorFlow • Torch • Keras • Hyperparameter Optimization • MOE • hyperopt • Spearmint • Infrastructure and Hardware • Apache Spark & HDFS • NVIDIA CUDA • Amazon Web Services G2 instances such scale much wow
53. references • icons by John Caserta, Liau Jian Jie, Garrett Knoll, Luboš Volkov, Noe Araujo from the Noun Project • images from Andrej Karpathy • Alex Kern, co-founder & CTO of • we help you structure image & video w/ deep learning • @KernCanCode on Twitter • @kern on GitHub about me • deep learning is great for many kinds of media • you can scale a deep learning system on Spark & AWS • get started @ bit.ly/pavlovtensor & bit.ly/pavlov231 in summary