Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machine Learning

Machine Learning

Overview of machine learning emphasizing deep learning. Covers Keras example using CIFAR-10 and training and production pipelines.


Melanie Warrick

August 13, 2017


  1. Machine Learning Melanie Warrick @nyghtowl

  2. Who am I?

  3. @nyghtowl Input Computation Result Artificial Intelligence

  4. @nyghtowl AI Fields • Machine Learning • Statistics & Probability

    Models • Generative & Adversarial Models • Graph Theory • Quantum Computing
  5. @nyghtowl Artificial Intelligence Machine Learning Deep Learning

  6. @nyghtowl Machine Learning * Rosenfeld Media

  7. @nyghtowl “DUKE VINCENTIO: Well, your wit is in the care

    of side and that. Second Lord: They would be ruled after this chamber, and my fair nues begun out of the fact, to be conveyed, Whose noble souls I'll have the heart of the wars. Clown: Come, sir, I will make did behold your worship. VIOLA: I'll drink it.” - Karpathy: http://karpathy.github.io/2015/05/21/rnn-effectiveness/ - Shakespeare image {{PD-US}}
  8. None
  9. Proprietary & Confidential

  10. @nyghtowl Machine Learning...

  11. @nyghtowl ML Algorithms • Linear & Logistic Regression • SVM

    • Random Forest • Neural Networks • Reinforcement Learning
  12. @nyghtowl Linear Model / Equation y = mx + b

    Linear Regression Model Example coefficients
  13. @nyghtowl Activation Func: • sigmoid • rectified linear • softmax

    • binary step Artificial Neural Net y = (W*x + b)
  14. @nyghtowl Deep Neural Nets Hidden Input Layer 2 Layer 3

    Layer 4
  15. @nyghtowl Convolutional Neural Networks Source: mNeuron: A Matlab Plugin to

    Visualize Neurons from Deep Models, Donglai Wei et. al.
  16. @nyghtowl Supervised Learning Test Train

  17. @nyghtowl Minimize Loss Function Loss Functions • mean sqr. error

    • negative log likelihood • cross entropy Error classific. vs. real label dog vs. flower Output
  18. @nyghtowl Optimization | Backprop Run until error stops improving =

    converge flower
  19. @nyghtowl Evaluation NN

  20. @nyghtowl Google Cloud Platform Virtual Machines Data Storage Analysis &

    Pipelines Machine Learning APIs & Engine w/
  21. @nyghtowl ML Pipeline | Training DB Data Train Algorithm(s) Test

    Model DB Serial Model Raw Data Cloud SQL Compute Engine protobuf Container Engine Cloud Storage Cloud Dataproc
  22. @nyghtowl ML Pipeline | Production Data Serial New Data Model

    Cloud Pub/Sub Cloud Dataflow Kafka Compute Engine Container Engine Mobile apps
  23. @nyghtowl Machine Learning Pipeline DB Data Train Algorithm(s) Test Model

    DB Serial Data Model Serial Raw Data New Data Model Cloud Pub/Sub Cloud Dataflow Kafka Cloud SQL Compute Engine Training Production protobuf Container Engine Cloud Storage Cloud Dataproc Mobile apps
  24. @nyghtowl Cifar -10 • 60K images • 10 classes •

    32 x 32 pixels (each one is an input) - CIFAR-10 http://www.cs.toronto.edu/~kriz/cifar.html | Learning Multiple Layers of Features from Tiny Images, Alex Krizhevsky, 2009 - Kaggle https://www.kaggle.com/c/cifar-10
  25. Video Intelligence https://github.com/sararob/video-intelligence-demo

  26. @nyghtowl Last Points ML in real world applications Leaders in

    field drive stable solutions Never trust the data or models
  27. @nyghtowl Resources & References • How to run any ML

    algorithm on GCP nyghtowl.com • Gorner, Martin (2017 Jan 19) Learn TensorFlow and deep learning without a Ph.D. https://cloud.google.com/blog/big-data/2017/01/learn-tensorflow-and-deep-learning-without-a-phd • Muic, Tommy (2016 Sep 25) Exception Models explained and implemented https://hacktilldawn.com/2016/09/25/inception-modules-explained-and-implemented/ • Lewis-Kraus, Gideon (2016 Dec. 14) The Great AI Awakening https://www.nytimes.com/2016/12/14/magazine/the-great-ai-awakening.html?smprod=nytcore-ipad&smid=nytcore-ipad-sha re&_r=0 • Nielsen, Michael Neural Nets and Deep Learning http://neuralnetworksanddeeplearning.com/ • Karpathy, Andrej (2015) Hacker’s Guide to Neural Nets https://karpathy.github.io/neuralnets/ • Convolutional Networks for Visual Recognition http://cs231n.github.io/ • Hobbs, Paul (2013 Oct 4) How can the various machine learning algorithms be classified summarized... https://www.quora.com/How-can-the-various-machine-learning-algorithms-be-classified-summarized-according-to-the-probl ems-they-solve • Chollet, Francois (2017 July 18) The limitations of deep learning https://blog.keras.io/the-limitations-of-deep-learning.html “All code snippets in this presentation are licensed under Apache 2.0 license” • Keras:https://github.com/fchollet/keras/blob/master/examples/cifar10_cnn.py • Keras model to TF:https://github.com/amir-abdi/keras_to_tensorflow/blob/master/keras_to_tensorflow.ipynb
  28. @nyghtowl References: Images • iStock.com/adventtr • iStock.com/4x-image • iStock.com/vectortatu •

    iStock.com/d1sk • iStock.com/georgeclerk • iStock.com/JurgaR • iStock.com/ranckreporter • https://static.pexels.com/photos/6069/grass-lawn-green-wooden-6069.jpg • https://www.kaggle.com/c/cifar-10 & http://www.cs.toronto.edu/~kriz/cifar.htm • https://en.wikipedia.org/wiki/Precision_and_recall • https://www.digitaltrends.com/mobile/google-lens-google-io-2017/ • Google IO • https://code.flickr.net/2014/10/20/introducing-flickr-park-or-bird/ • http://www.texample.net/tikz/examples/neural-network/ • http://www.texample.net/tikz/examples/neural-network/ • https://en.wikipedia.org/wiki/General_linear_model • https://en.wikipedia.org/wiki/Convolutional_neural_network • https://commons.wikimedia.org/wiki/File:Perceptron.png • https://en.wikipedia.org/wiki/File:Shakes.png • https://www.computer.org/web/awards/pioneer-jean-sammet • Rosenfeld Media https://www.flickr.com/photos/rosenfeldmedia/6949089460 • Copyright and disclaimer notice: https://creativecommons.org/licenses/by/2.0/ • License notice: https://creativecommons.org/licenses/by/2.0/legalcode
  29. @nyghtowl Machine Learning Melanie Warrick @nyghtowl