$30 off During Our Annual Pro Sale. View Details »

Dive into Deep Learning with Scala

Sören Brunk
May 16, 2018
7.4k

Dive into Deep Learning with Scala

Deep learning is an emerging field with the potential to transform whole industries, and to significantly affect software development.
This talk is an introduction to deep learning for Scala developers. We'll see why deep learning enables a new programming paradigm, and how it's related to functional programming.
We'll discuss what makes Scala an interesting choice for deep learning, and look at the current landscape of deep learning libraries for Scala. Equipped with that knowledge, we'll walk through a concrete example that shows you how to create, train and deploy a neural network using Scala.

Talk from Scala Days Europe 2018.

Sören Brunk

May 16, 2018
Tweet

Transcript

  1. Dive into Deep Learning -
    with Scala
    Sören Brunk @soebrunk
    https://brunk.io

    View Slide

  2. “AI is the new Electricity”
    Andrew Ng

    View Slide

  3. Source: https://petewarden.com/2017/11/13/deep-learning-is-eating-software/
    Source: https://medium.com/intuitionmachine/is-deep-learning-software-2-0-cc7ad46b138f

    View Slide

  4. Source: Dukesy68 CC-BY-SA 4.0

    View Slide

  5. (might look slightly different in your country)

    View Slide

  6. (might look slightly different in your country)

    View Slide

  7. import awesome.deeplearn.library.magic
    val result = magic(input)
    (might look slightly different in your country)

    View Slide

  8. Let's Discover Deep Learning from a Scala
    Perspective

    View Slide

  9. View Slide

  10. Deep Learning ≈ Artificial Neural
    Networks

    View Slide

  11. Neural Network Examples

    View Slide

  12. Computer Vision
    =>

    View Slide

  13. Speech Recognition
    “The Quick brown fox jumps
    over the lazy dog.”
    =>

    View Slide

  14. Machine Translation
    =>

    View Slide

  15. What do these Examples have in Common?
    => “The Quick brown fox
    jumps over the lazy dog.”
    =>
    =>

    View Slide

  16. =>
    =>
    What do these Examples have in Common?
    “The Quick brown fox jumps
    over the lazy dog.”

    View Slide

  17. =>
    A => B
    A Neural Network is a Function

    View Slide

  18. A Neural Network is a Function
    type Model[A, B] = A => B

    View Slide

  19. val classifier: Model[Image, ImageClass] = ???
    A Neural Network is a Function
    How do we implement
    that thing?

    View Slide

  20. Neural Network Mechanics

    View Slide

  21. Neural Networks Work on Numeric Tensors

    View Slide

  22. Representing Values as Tensors
    Model[Tensor, Tensor]
    encoder: A => Tensor
    decoder: Tensor => B
    Model[A, B]
    64
    432
    67
    62
    7 9 203
    90
    56
    64
    230
    54 76 4
    51
    83
    24
    8
    41
    40
    17
    5
    230
    98
    23
    7
    129
    34
    93
    73

    View Slide

  23. The Artificial Neuron
    w1

    x1
    w2
    x2
    w3
    x3
    wn
    xn


    Apply
    Activation
    Function
    y
    Multiply inputs and weights
    Sum up
    results

    View Slide

  24. The Artificial Neuron
    w
    1

    x
    1
    w
    2
    x
    2
    w
    3
    x
    3
    w
    n
    x
    n


    y

    View Slide

  25. Neural Networks are Layers of Neurons


    Input
    Layer
    1. Layer 2. Layer
    (Output)

    View Slide

  26. Source: https://github.com/tensorflow/models/tree/master/research/inception
    Neural Networks are Layers of Neurons

    View Slide

  27. But How Do We Set Those Weights?


    Input
    Layer
    1. Layer 2. Layer
    (Output)

    View Slide

  28. “What we want is a machine that can
    learn from experience.”
    Alan Turing, 1947

    View Slide

  29. A Different Programming Paradigm
    Traditional Programming Machine Learning

    View Slide

  30. A Neural Network Learns its Function
    def train(examples: Map[A, B]): Model[A, B]
    Supervised Learning

    View Slide

  31. Training Data: Examples with Answers
    Example
    Label (Answer)
    (1 = cat, 0 = not cat)
    1
    0
    1
    val examples: Map[Image, Either[Cat, NotCat]]

    View Slide

  32. Training a Neural Network
    Forward Pass
    Compute Loss
    Backpropagation
    Update Weights
    Initialize
    Randomly

    View Slide

  33. Initialize Randomly

    View Slide

  34. Initialize Randomly

    View Slide

  35. Forward Pass Forward Pass
    Compute Loss
    Backpropagation
    Update Weights

    View Slide

  36. Forward Pass
    Cat
    Probability
    = 0.4
    Forward Pass
    Compute Loss
    Backpropagation
    Update Weights

    View Slide

  37. Compute the Loss
    Loss (simplified) =
    correct answer - predicted answer
    Cat = 1 1
    0.4
    0.6
    Forward Pass
    Compute Loss
    Backpropagation
    Update Weights

    View Slide

  38. Backpropagation Forward Pass
    Compute Loss
    Backpropagation
    Update Weights

    View Slide

  39. Update Weights Forward Pass
    Compute Loss
    Backpropagation
    Update Weights

    View Slide

  40. Update Weights Forward Pass
    Compute Loss
    Backpropagation
    Update Weights

    View Slide

  41. Error
    Weight
    Loss Function
    Gradient Descent Forward Pass
    Compute Loss
    Backpropagation
    Update Weights
    Source: https://www.coursera.org/learn/machine-learning

    View Slide

  42. What Makes Deep Learning so
    powerful?

    View Slide

  43. Performance / Accuracy
    Amount of data
    Traditional Machine
    Learning
    Deep Learning
    Neural Networks Scale

    View Slide

  44. Neural Networks Compose
    Source: Vinyals et al. Show and Tell: A Neural Image Caption Generator

    View Slide

  45. Limitations of Deep Learning
    Source: https://www.technologyreview.com/s/608321/this-image-is-why-self-driving-cars-
    come-loaded-with-many-types-of-sensors
    Source: Brown et al, Adversarial Patch (https://arxiv.org/abs/1712.09665)

    View Slide

  46. View Slide

  47. Why Scala?
    Prototyping vs. Production

    View Slide

  48. Why Scala?
    Type safe tensor operations
    case class Tensor(
    dataType: DataType,
    shape: Shape
    )
    class TypedTensor[DataType, Shape]
    val tensor: TypedTensor[Float, (Width, Height, Channel)] = ...

    View Slide

  49. Deep Learning Libraries for Scala

    View Slide

  50. Deep Learning Libraries for Scala
    Source: https://brunk.io/deep-learning-in-scala-part-1-basics-and-libraries.html
    • Mature, Large
    Community &
    Commercial Support
    • Java API
    • Native Scala API
    (ScalNet) in alpha
    • Scala API part of main
    project
    • API very pythonic
    • Most idiomatic & type
    safe Scala API
    • Not part of main
    TensorFlow (yet)

    View Slide

  51. Deep Learning Libraries for Scala
    Source: https://brunk.io/deep-learning-in-scala-part-1-basics-and-libraries.html
    • Mature, Large
    Community &
    Commercial Support
    • Java API
    • Native Scala API
    (ScalNet) in alpha
    • Scala API part of main
    project
    • API very pythonic
    • Most idiomatic & type
    safe Scala API
    • Not part of main
    TensorFlow (yet)

    View Slide

  52. View Slide

  53. ? => ?
    1. Frame Your Problem

    View Slide

  54. 1. Frame Your Problem
    =>

    View Slide

  55. 1. Frame Your Problem
    =>Either[ScalaLogo,
    NotScalaLogo]

    View Slide

  56. 2. Look for Existing Solutions

    View Slide

  57. 3. Collect and Prepare Training Data
    Filter Resize Normalize Vectorize ...
    encoder: A => Tensor
    decoder: Tensor => B
    Training Set Test Set

    View Slide

  58. 3. Collect and Prepare Training Data
    def readImage(filename: Output): Output = {
    val rawImage = tf.data.readFile(filename)
    val image = tf.image.decodeJpeg(rawImage, numChannels = 3)
    tf.image.resizeBilinear(image.expandDims(axis=0), Seq(250, 250))
    .squeeze(Seq(0)).cast(UINT8)
    }
    val trainData: Dataset[(Tensor, Tensor), (Output, Output),
    (DataType, DataType), (Shape, Shape)] =
    tf.data.TensorSlicesDataset(filenamesWithLabels(trainDir))
    .shuffle(bufferSize = 30000, Some(seed))
    .map({ case (filename, label) => (readImage(filename), label)})
    .repeat()
    .batch(128)
    .prefetch(100)

    View Slide

  59. 4. Choose the Right Architecture
    By Aphex34 - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=45679374
    Source: https://karpathy.github.io/2015/05/21/rnn-effectiveness/
    Fully Connected Neural Network Convolutional Neural Network
    Recurrent Neural Network



    View Slide

  60. 4. Choose the Right Architecture
    By Aphex34 - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=45679374
    1. Layer
    2. Layer
    3. Layer
    4. Layer
    Convolutional Neural Network

    View Slide

  61. 5. Train, Evaluate, Refine
    Create /
    Improve
    Train
    Evaluate
    The Model
    Development
    Cycle

    View Slide

  62. Build a Simple Model
    Source: https://www.flickr.com/photos/foto_db/12163399363 CC-BY-SA

    View Slide

  63. Build a Simple Model
    import org.platanios.tensorflow.api._
    val input = tf.learn.Input(UINT8, Shape(-1, 250, 250, 3)) // type and shape of our images
    val labelInput = tf.learn.Input(UINT8, Shape(-1)) // type and shape of our labels
    val layer =
    tf.learn.Flatten("Input/Flatten") >> // flatten the images into a single vector
    tf.learn.Cast("Input/Cast", FLOAT32) >> // cast input to float
    tf.learn.Linear("Layer_1/Linear", units = 64) >> // hidden layer with 512 neurons
    tf.learn.ReLU("Layer_1/ReLU“, 0.01f) >> // hidden layer activation
    tf.learn.Linear("OutputLayer/Linear", units = 2) // output layer
    val trainInputLayer = tf.learn.Cast("TrainInput/Cast", INT64) // cast labels to long
    val loss =
    tf.learn.SparseSoftmaxCrossEntropy("Loss/CrossEntropy") >> // loss/error function
    tf.learn.Mean("Loss/Mean") >>
    tf.learn.ScalarSummary("Loss/Summary", "Loss")
    val optimizer = tf.train.GradientDescent(learningRate = 0.001) // the optimizer updates our weights
    // create a Model for training
    val model = tf.learn.Model.supervised(input, layer, labelInput, trainInputLayer, loss, optimizer)

    View Slide

  64. Train and Evaluate the Model
    val estimator = tf.learn.InMemoryEstimator(model)
    estimator.train(() => trainData)
    Learn / Hooks / Evaluation - Step 0 Evaluator:
    Learn / Hooks / Evaluation - ╔═══════╤════════════╗
    Learn / Hooks / Evaluation - ║ │ Accuracy ║
    Learn / Hooks / Evaluation - ╟───────┼────────────╢
    Learn / Hooks / Evaluation - ║ Train │ 0.1219 ║
    Learn / Hooks / Evaluation - ║ Test │ 0.1217 ║
    Learn / Hooks / Evaluation - ╚═══════╧════════════╝
    Learn / Hooks / Loss Logger - ( 2.831 s) Step: 100, Loss: 2250.2437
    Learn / Hooks / Loss Logger - ( 0.561 s) Step: 200, Loss: 2735.0537
    ...
    Learn / Hooks / Loss Logger - ( 0.537 s) Step: 900, Loss: 877.2033
    Learn / Hooks / Loss Logger - ( 0.699 s) Step: 1000, Loss: 1207.4133
    Learn / Hooks / Evaluation - Step 1000 Evaluator:
    Learn / Hooks / Evaluation - ╔═══════╤════════════╗
    Learn / Hooks / Evaluation - ║ │ Accuracy ║
    Learn / Hooks / Evaluation - ╟───────┼────────────╢
    Learn / Hooks / Evaluation - ║ Train │ 0.8105 ║
    Learn / Hooks / Evaluation - ║ Test │ 0.7888 ║
    Learn / Hooks / Evaluation - ╚═══════╧════════════╝
    Learn / Hooks / Loss Logger - ( 2.599 s) Step: 1100, Loss: 989.0901
    Learn / Hooks / Loss Logger - ( 0.529 s) Step: 1200, Loss: 440.6418
    ...

    View Slide

  65. Refine the Model
    val layer = tf.learn.Cast("Input/Cast", FLOAT32) >>
    tf.learn.Flatten("Input/Flatten") >>
    tf.learn.Linear("Layer_1/Linear", units = 64) >> // hidden layer
    tf.learn.ReLU("Layer_1/ReLU", 0.01f) >> // hidden layer activation
    tf.learn.Linear("OutputLayer/Linear", units = 2) // output layer

    View Slide

  66. Refine the Model
    val layer = tf.learn.Cast("Input/Cast", FLOAT32) >>
    tf.learn.Flatten("Input/Flatten") >>
    tf.learn.Linear("Layer_1/Linear", units = 128) >> // hidden layer
    tf.learn.ReLU("Layer_1/ReLU", 0.01f) >> // hidden layer activation
    tf.learn.Linear("OutputLayer/Linear", units = 2) // output layer

    View Slide

  67. Refine the Model
    val layer = tf.learn.Cast("Input/Cast", FLOAT32) >>
    tf.learn.Flatten("Input/Flatten") >>
    tf.learn.Linear("Layer_1/Linear", units = 128) >> // hidden layer
    tf.learn.ReLU("Layer_1/ReLU", 0.01f) >> // hidden layer activation
    tf.learn.Linear("OutputLayer/Linear", units = 2) // output layer
    val estimator = tf.learn.InMemoryEstimator(model)
    estimator.train(() => trainData)

    View Slide

  68. Refine the Model
    val layer = tf.learn.Cast("Input/Cast", FLOAT32) >>
    tf.learn.Flatten("Input/Flatten") >>
    tf.learn.Linear("Layer_1/Linear", units = 512) >> // hidden layer
    tf.learn.ReLU("Layer_1/ReLU", 0.01f) >> // hidden layer activation
    tf.learn.Linear("OutputLayer/Linear", units = 2) // output layer
    val estimator = tf.learn.InMemoryEstimator(model)
    estimator.train(() => trainData)

    View Slide

  69. Refine the Model
    tf.learn.Conv2D("Layer_0/Conv2D", Shape(3, 3, 1, 32), stride1 = 2, stride2 = 2, ValidConvPadding) >>
    tf.learn.AddBias("Layer_0/Bias") >>
    tf.learn.ReLU("Layer_0/ReLU", 0.01f) >>
    tf.learn.MaxPool("Layer_0/MaxPool", windowSize = Seq(1, 2, 2, 1), stride1 = 2, stride2 = 2, ValidConvPadding) >>
    val layer = tf.learn.Cast("Input/Cast", FLOAT32) >>
    tf.learn.Flatten("Input/Flatten") >>
    tf.learn.Linear("Layer_1/Linear", units = 512) >>
    tf.learn.ReLU("Layer_1/ReLU") >>
    tf.learn.Linear("OutputLayer/Linear", units = 2)

    View Slide

  70. Refine the Model
    tf.learn.Conv2D("Layer_0/Conv2D", Shape(3, 3, 1, 32), stride1 = 2, stride2 = 2, ValidConvPadding) >>
    tf.learn.AddBias("Layer_0/Bias") >>
    tf.learn.ReLU("Layer_0/ReLU", 0.01f) >>
    tf.learn.MaxPool("Layer_0/MaxPool", windowSize = Seq(1, 2, 2, 1), stride1 = 2, stride2 = 2, ValidConvPadding) >>
    val layer = tf.learn.Cast("Input/Cast", FLOAT32) >>
    tf.learn.Flatten("Input/Flatten") >>
    tf.learn.Linear("Layer_1/Linear", units = 512) >>
    tf.learn.ReLU("Layer_1/ReLU") >>
    tf.learn.Linear("OutputLayer/Linear", units = 2)
    val estimator = tf.learn.InMemoryEstimator(model)
    estimator.train(() => trainData)

    View Slide

  71. Run the Model
    val image: Tensor = readImage(filename)
    val decoder = Seq("not_scala", "scala")
    val result: Tensor = estimator.infer(() => image)
    val label = getLabel(result))
    drawLabel(image, s"Class: $label ($decoder(label))")

    View Slide

  72. 7. Deploy to Production

    View Slide

  73. Takeaways
    ● Neural networks are functions
    ● We usually learn them from known examples
    ● Training can be challenging
    ● Scala is a great language for Deep Learning
    ● Interested? Give it a try
    ● Get involved to make it even better

    View Slide

  74. Thank You for Listening
    Sören Brunk @soebrunk
    https://brunk.io

    View Slide