A Gentle Introduction to Deep Learning for Developers at ConFoo Montreal

While we are a long way out from machines that can perform artificial general intelligence (AGI) tasks, deep learning can be used for many narrow artificial intelligence (AI) tasks including image or audio classification, facial recognition, object recognition, image caption generation, and natural language processing. We will explore how developers can easily integrate open source deep learning models into their applications.

Transcript

1. A Gentle Introduction
   to Deep Learning
   for Developers
   Bradley Holt
   Program Manager, Developer Advocacy
   Center for Open-Source Data and AI Technologies
   @BradleyHolt
   DOC ID / Month XX, 2018 / © 2018 IBM Corporation
   

   View Slide

2. Is AI overhyped?
   

   View Slide

3. Yes
   

   View Slide

4. Thank you
   for coming
   to my talk
   

   View Slide

5. General AI
   Metal Skull With Terminator Eye by L.C. Nøttaasen, on Flickr (CC BY-SA 2.0).
   

   View Slide

6. Broad AI
   -[ electrIc b88Gal88 ]- by JD Hancock, on Flickr (CC BY 2.0).
   

   View Slide

7. Narrow AI
   Danbo on the Lookout by IQRemix, on Flickr (CC BY-SA 2.0).
   

   View Slide

8. Deep
   Learning
   http://codait.org/
   

   View Slide

9. Demo: Object
   Identification
   and Image
   Segmentation
   with magicat
   https://github.com/CODAIT/magicat
   

   View Slide

10. https://github.com/CODAIT/magicat
    

    View Slide

11. Install magicat
    $ npm install -g magicat
    + magicat@1.4.9
    added 255 packages from 209 contributors in 11.798s
    

    View Slide

12. 23895621638_535be71dee_k.jpg
    37038669284_899d7784a9_k.jpg
    37489697170_31d05aa027_k.jpg
    37699459356_24fd526a5e_k.jpg
    37699976806_5ce694be36_k.jpg
    Animal photos by Susanne Nilsson, on Flickr (CC BY-SA 2.0).
    

    View Slide

13. Scan Directory with magicat
    $ magicat .
    Scanning directory '/Users/bradleydholt/tfjs-demos/magicat'...
    The image '23895621638_535be71dee_k.jpg' contains the following
    segments: background, cat.
    The image '37038669284_899d7784a9_k.jpg' contains the following
    segments: background, sheep.
    The image '37489697170_31d05aa027_k.jpg' contains the following
    segments: background, sheep.
    The image '37699459356_24fd526a5e_k.jpg' contains the following
    segments: background, cat.
    The image '37699976806_5ce694be36_k.jpg' contains the following
    segments: background, horse.
    

    View Slide

14. background, cat
    background, sheep
    background, sheep
    background, cat
    background, horse
    Animal photos by Susanne Nilsson, on Flickr (CC BY-SA 2.0).
    

    View Slide

15. Save Image Segment with magicat
    $ magicat 37699976806_5ce694be36_k.jpg --save horse
    The image '37699976806_5ce694be36_k.jpg' contains the following
    segments: background, horse.
    saved 37699976806_5ce694be36_k-horse.png
    Animal photos by Susanne Nilsson, on Flickr (CC BY-SA 2.0).
    

    View Slide

16. Demo: Object
    Identification
    and Image
    Segmentation
    with the Magic
    Cropping Tool
    https://developer.ibm.com/patterns/max-image-segmenter-magic-cropping-tool-web-app/
    

    View Slide

17. https://developer.ibm.com/patterns/max-image-segmenter-magic-cropping-tool-web-app/
    

    View Slide

18. View Slide

19. View Slide

20. Deep Learning Use Cases
    https://ibm.biz/max-developers
    

    View Slide

21. Software
    Programming
    Untitled by Marcin Wichary, on Flickr (CC BY 2.0).
    

    View Slide

22. 0101100100101100000110101101
    0101001001100000110101101011
    0110010010101010011101001011
    0101010011100101010101010000
    1010101011100000011010101010
    1001000110000011010011010101
    0010001011010101011101000101
    Software Program
    Business Logic
    

    View Slide

23. 0101100100101100000110101101
    0101001001100000110101101011
    0110010010101010011101001011
    0101010011100101010101010000
    1010101011100000011010101010
    1001000110000011010011010101
    0010001011010101011101000101
    Software Program
    Input
    Program Execution
    Output
    

    View Slide

24. Deep
    Learning
    

    View Slide

25. Input Layer Hidden Layers Output Layer
    

    View Slide

26. Labeled Training Data
    Backpropagation
    Animal photos by Susanne Nilsson, on Flickr (CC BY-SA 2.0).
    Output Errors
    Cat
    ❌
    Not Cat
    ❌
    Cat ❌
    Cat
    Cat
    

    View Slide

27. Input
    Neural Network Inferencing
    Output
    Cat
    Animal photos by Susanne Nilsson, on Flickr (CC BY-SA 2.0).
    

    View Slide

28. Neural Network Inferencing
    Output
    Not Cat
    Input
    Animal photos by Susanne Nilsson, on Flickr (CC BY-SA 2.0).
    

    View Slide

29. Machine
    Learning
    Libraries
    The Leeds Library by Michael D Beckwith, on Flickr (CC0 1.0).
    

    View Slide

30. View Slide

31. View Slide

32. Demo: Basic
    Classification
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

33. Goal & Context
    • Train a neural network to classify images
    of clothing
    • Classifications include dress, sneaker, etc.
    • We will use the Keras high-level API
    within TensorFlow
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

34. Fashion-MNIST
    • Training set of 60,000 article of clothing images
    • Test set of 10,000 examples can be used to
    evaluate image classification accuracy
    • Each example is a 28x28 grayscale image
    • Each example has an associated label from 10
    classes (T-shirt/top, trouser, pullover, etc.)
    https://github.com/zalandoresearch/fashion-mnist
    

    View Slide

35. Fashion-MNIST Images
    https://github.com/zalandoresearch/fashion-mnist
    

    View Slide

36. Fashion-MNIST Image Labels
    Label Class
    0 T-shirt/top
    1 Trouser
    2 Pullover
    3 Dress
    4 Coat
    5 Sandal
    6 Shirt
    7 Sneaker
    8 Bag
    9 Ankle boot
    https://github.com/zalandoresearch/fashion-mnist
    

    View Slide

37. https://jupyter.org/
    

    View Slide

38. Import Libraries
    In [3]: # TensorFlow and tf.keras
    import tensorflow as tf
    from tensorflow import keras
    # Helper libraries
    import numpy as np
    import matplotlib.pyplot as plt
    print(tf.__version__)
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

39. Import Fashion-MNIST Dataset
    In [4]: fashion_mnist = keras.datasets.fashion_mnist
    (train_images, train_labels), (test_images, test_labels) =
    fashion_mnist.load_data()
    In [5]: class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat’,
    'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

40. Explore the Data
    In [6]: train_images.shape
    Out[6]: (60000, 28, 28)
    In [7]: len(train_labels)
    Out[7]: 60000
    In [8]: train_labels
    Out[8]: array([9, 0, 0, ..., 3, 0, 5], dtype=uint8)
    In [9]: test_images.shape
    Out[9]: (10000, 28, 28)
    In [10]: len(test_labels)
    Out[10]: 10000
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

41. Preprocess the Data
    In [11]: plt.figure()
    plt.imshow(train_images[0])
    plt.colorbar()
    plt.grid(False)
    plt.show()
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

42. Preprocess the Data
    In [12]: train_images = train_images / 255.0
    test_images = test_images / 255.0
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

43. Preprocess the Data
    In [13]: plt.figure(figsize=(10,10))
    for i in range(25):
    plt.subplot(5,5,i+1)
    plt.xticks([])
    plt.yticks([])
    plt.grid(False)
    plt.imshow(train_images[i], cmap=plt.cm.binary)
    plt.xlabel(class_names[train_labels[i]])
    plt.show()
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

44. https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

45. Set Up the Model Layers
    In [14]: model = keras.Sequential([
    keras.layers.Flatten(input_shape=(28, 28)),
    keras.layers.Dense(128, activation=tf.nn.relu),
    keras.layers.Dense(10, activation=tf.nn.softmax)
    ])
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

46. Compile the Model
    In [15]: model.compile(optimizer='adam’,
    loss='sparse_categorical_crossentropy’,
    metrics=['accuracy'])
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

47. Train the Model
    In [16]: model.fit(train_images, train_labels, epochs=5)
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

48. Train the Model
    Epoch 1/5
    60000/60000 [==============================] - 4s 61us/step - loss: 0.4985 - acc: 0.8261
    Epoch 2/5
    60000/60000 [==============================] - 3s 54us/step - loss: 0.3747 - acc: 0.8646
    Epoch 3/5
    60000/60000 [==============================] - 4s 58us/step - loss: 0.3359 - acc: 0.8771
    Epoch 4/5
    60000/60000 [==============================] - 3s 55us/step - loss: 0.3123 - acc: 0.8846
    Epoch 5/5
    60000/60000 [==============================] - 4s 59us/step - loss: 0.2945 - acc: 0.8912
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

49. Evaluate Accuracy
    In [17]: test_loss, test_acc = model.evaluate(test_images, test_labels)
    print('Test accuracy:', test_acc)
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

50. Evaluate Accuracy
    10000/10000 [==============================] - 0s 26us/stepTest accuracy: 0.8748
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

51. Make Predictions
    In [18]: predictions = model.predict(test_images)
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

52. Make Predictions
    In [19]: predictions[0]
    Out[19]: array([9.6376725e-06, 4.5565642e-07, 5.0994299e-06, 4.8001135e-07,
    4.9554501e-06, 2.0314518e-02, 5.0287876e-05, 1.6105843e-01,
    2.2055059e-04, 8.1833553e-01], dtype=float32)
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

53. Make Predictions
    In [20]: np.argmax(predictions[0])
    Out[20]: 9
    In [21]: test_labels[0]
    Out[21]: 9
    https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

54. https://www.tensorflow.org/tutorials/keras/basic_classification
    

    View Slide

55. Applying
    Deep
    Learning
    Data Science
    Expertise
    Computing
    Resources
    High-Quality
    Training Data
    Model Deployment
    Time
    Model Integration
    Inferencing Code
    And more…
    

    View Slide

56. Model Asset Exchange (MAX)
    Classify
    Generate
    Recognize
    https://developer.ibm.com/exchanges/models/
    

    View Slide

57. Choose deployable model
    Deep Learning asset on MAX
    Deploy
    Swagger specification Inference endpoint Metadata endpoint
    Microservice
    Input pre-processing, model execution, and output post-processing
    Deploy
    model
    Use
    model
    https://developer.ibm.com/exchanges/models/
    

    View Slide

58. AI Lifecycle
    Deploy
    Operate
    Build
    & Train
    

    View Slide

59. Building and Training AI Models
    Model Asset Exchange (MAX)
    

    View Slide

60. View Slide

61. Deploying AI Models
    ONNX.js
    Model Asset Exchange (MAX)
    

    View Slide

62. View Slide

63. Operating AI Systems
    Fabric for Deep Learning (FfDL)
    Train and deploy deep learning models on Kubernetes using TensorFlow,
    Caffe2, PyTorch, and other frameworks
    AI Fairness 360 (AIF360)
    Comprehensive set of fairness metrics for machine learning models,
    explanations for these metrics, and algorithms to mitigate bias in models
    Adversarial Robustness Toolbox (ART)
    Python library for adversarial attacks and defenses for neural networks
    with multiple framework support
    

    View Slide

64. View Slide

65. What will you
    build using
    deep learning?
    Dynamic Earth - Continental Shelf by NASA Goddard Space Flight Center, on Flickr (CC BY 2.0).
    

    View Slide

66. Resources: Machine Learning Libraries
    • TensorFlow
    https://www.tensorflow.org/
    • Train your first neural network: basic classification | TensorFlow
    https://www.tensorflow.org/tutorials/keras/basic_classification
    • Keras
    https://keras.io/
    • PyTorch
    https://pytorch.org/
    • Caffe2
    https://caffe2.ai/
    

    View Slide

67. Resources: IBM Developer
    • IBM Cloud
    https://ibm.biz/Bd2A5f
    • Center for Open-Source Data & AI Technologies (CODAIT)
    http://codait.org/
    • IBM Developer Model Asset Exchange (MAX)
    https://developer.ibm.com/exchanges/models/
    • Model Asset Exchange (MAX) Code Patterns
    https://ibm.biz/max-developers
    • Deploy a deep learning-powered ‘Magic cropping tool’
    https://developer.ibm.com/patterns/max-image-segmenter-magic-cropping-tool-web-app/
    • !" magicat
    https://github.com/CODAIT/magicat
    

    View Slide

68. Resources: Building and Training
    AI Models
    • Project Jupyter
    https://jupyter.org/
    • IBM Developer Model Asset Exchange (MAX)
    https://developer.ibm.com/exchanges/models/
    • IBM Watson Studio
    https://www.ibm.com/cloud/watson-studio
    

    View Slide

69. Resources: Deploying AI Models
    • Data Mining Group (PMML & PFA)
    http://dmg.org/
    • ONNX
    https://onnx.ai/
    • ONNX.js
    https://github.com/Microsoft/onnxjs
    • TensorFlow.js
    https://js.tensorflow.org/
    • TensorFlow Lite
    https://www.tensorflow.org/lite
    • Core ML
    https://developer.apple.com/machine-learning/
    • IBM Watson Machine Learning
    https://www.ibm.com/cloud/machine-learning
    

    View Slide

70. Resources: Operating AI Systems
    • Fabric for Deep Learning (FfDL)
    https://github.com/IBM/FfDL
    • AI Fairness 360 (AIF360)
    https://github.com/IBM/AIF360
    • Adversarial Robustness Toolbox (ART)
    https://github.com/IBM/adversarial-robustness-toolbox
    • IBM Watson OpenScale
    https://www.ibm.com/cloud/watson-openscale
    

    View Slide

71. Thank you
    codait.org
    twitter.com/codait_datalab
    medium.com/ibm-watson-data-lab
    github.com/codait
    developer.ibm.com
    

    View Slide

72. View Slide