Deep Learning at Scale

The advent of modern deep learning techniques has given organizations new tools to understand, query, and structure their data. However, maintaining complex pipelines, versioning models, and tracking accuracy regressions over time remain ongoing struggles of even the most advanced data engineering teams. This talk presents a simple architecture for deploying machine learning at scale and offer suggestions for how companies can get their feet wet with open source technologies they already deploy.

Presented at Big Data Day LA 2016.

Transcript

1. Deep Learning at Scale
2. Hi, I’m Kern. @kern @KernCanCode
4. deep learning?!
5. AlphaGo achieves honorary 9 dan rank.
6. Google’s DeepDream or: scary fish-dog things everywhere
7. deep learning is a kind of machine learning.
8. machine learning is like pattern recognition. (kind of)
9. deep learning has been applied to images, text, audio, video, and games with great success!
10. DEEP
 CONVOLUTIONAL
 NEURAL
 NETWORKS
11. 14.2 million images
12. Challenge Winners top-5 classification error (lower is better) 0 0.075 0.15 0.225 0.3 0.036 0.05 0.074 0.112 0.153 0.258 0.282 2010 2011 2012 2014 2013 Human 2015
13. 0 0.075 0.15 0.225 0.3 0.036 0.05 0.074 0.112 0.153 0.258 0.282 2010 2011 2012 2014 2013 Human 2015 Challenge Winners top-5 classification error (lower is better) }much wow AlexNet BEFORE DEEP LEARNING AFTER DEEP LEARNING
15. how does deep learning work?
16. it’s easy! lol just kidding
17. it’s calculus, but you don’t need to know much.
18. label score dog 97.1% shiba inu 2.5% meme 0.4% neural networks are classifiers
19. supervised learning inferring a function from labeled training data Wikipedia
20. you provide “ground truth” examples and labels.
21. perceptrons input output weights
22. is this a dog? T F features classes weights
23. is this a dog? T F features classes weights
24. is this a dog? T F features classes weights
25. PERCEPTRONS ARE TOO LINEAR WE MUST GO DEEPER
26. deep convolutional neural networks affectionately known as “convnets” input output convolution, activation, & pooling layers fully-connected layers (perceptrons)
27. how are the optimal weights determined?
28. stochastic gradient descent find the minimum error
29. training takes a while. anywhere from 8hrs to 2wks
30. how do you scale deep learning?
31. data collection & cleaning more clean data, the better 1
32. data collection & cleaning more clean data, the better 1 2 model training & selection anywhere from 8hrs to 2wks
33. data collection & cleaning more clean data, the better 1 2 model training & selection anywhere from 8hrs to 2wks 3 serving in production with real-time or batch requests
34. data collection & cleaning more clean data, the better 1 ∞ rinse & repeat keep models fresh with new data 2 model training & selection anywhere from 8hrs to 2wks 3 serving in production with real-time or batch requests
35. you’re gonna need GPUs for NVIDIA CUDA & cuDNN
36. NVIDIA GeForce GTX TITAN X is the gold standard
37. AWS G2 instances mid-tier on-demand GPUs Model GPUs vCPU Mem (GiB) SSD Storage (GB) g2.2xlarge 1 8 15 1 x 60 g2.8xlarge 4 32 60 2 x 120
38. AWS G2 GPUs are also great for model serving.
39. use HDFS + Spark for data storage and processing
40. TensorFlow with Keras is a good choice.
41. from keras.models import Sequential from keras.layers import Dense, Activation model = Sequential() model.add(Dense(output_dim=64, input_dim=100)) model.add(Activation("relu")) model.add(Dense(output_dim=10)) model.add(Activation("softmax")) model.compile( loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy']) model.fit(X_train, Y_train, nb_epoch=5, batch_size=32)
42. distributed training with CaffeOnSpark https://github.com/yahoo/CaffeOnSpark
43. it’s often better to peer high-end GPUs on a single machine.
44. parallelize training on hyperparameters. (e.g. learning rate, momentum)
45. hyperparamter optimization with MOE, hyperopt, or custom scripts hyperopt MOE custom scripts
46. wrapping up
47. get your feet wet TensorFlow MNIST Walkthrough bit.ly/pavlovtensor Andrej Karpathy’s CS231n bit.ly/pavlov231
48. suggested technologies • Neural Network Libraries • Caffe & CaffeOnSpark • TensorFlow • Torch • Keras • Hyperparameter Optimization • MOE • hyperopt • Spearmint • Infrastructure and Hardware • Apache Spark & HDFS • NVIDIA CUDA • Amazon Web Services G2 instances such scale much wow
49. references • icons by John Caserta, Liau Jian Jie, Garrett Knoll, Luboš Volkov, Noe Araujo from the Noun Project • images from Andrej Karpathy • Alex Kern, co-founder & CTO of • we help you structure image & video w/ deep learning • @KernCanCode on Twitter • @kern on GitHub about me • deep learning is great for many kinds of media • you can scale a deep learning system on Spark & AWS • get started @ bit.ly/pavlovtensor & bit.ly/pavlov231 in summary