Dive into Deep Learning with Scala

Deep learning is an emerging field with the potential to transform whole industries, and to significantly affect software development.
This talk is an introduction to deep learning for Scala developers. We'll see why deep learning enables a new programming paradigm, and how it's related to functional programming.
We'll discuss what makes Scala an interesting choice for deep learning, and look at the current landscape of deep learning libraries for Scala. Equipped with that knowledge, we'll walk through a concrete example that shows you how to create, train and deploy a neural network using Scala.

Talk from Scala Days Europe 2018.

Transcript

1. Dive into Deep Learning -
   with Scala
   Sören Brunk @soebrunk
   https://brunk.io
   

   View Slide

2. “AI is the new Electricity”
   Andrew Ng
   

   View Slide

3. Source: https://petewarden.com/2017/11/13/deep-learning-is-eating-software/
   Source: https://medium.com/intuitionmachine/is-deep-learning-software-2-0-cc7ad46b138f
   

   View Slide

4. Source: Dukesy68 CC-BY-SA 4.0
   

   View Slide

5. (might look slightly different in your country)
   

   View Slide

6. (might look slightly different in your country)
   

   View Slide

7. import awesome.deeplearn.library.magic
   val result = magic(input)
   (might look slightly different in your country)
   

   View Slide

8. Let's Discover Deep Learning from a Scala
   Perspective
   

   View Slide

9. View Slide

10. Deep Learning ≈ Artificial Neural
    Networks
    

    View Slide

11. Neural Network Examples
    

    View Slide

12. Computer Vision
    =>
    

    View Slide

13. Speech Recognition
    “The Quick brown fox jumps
    over the lazy dog.”
    =>
    

    View Slide

14. Machine Translation
    =>
    

    View Slide

15. What do these Examples have in Common?
    => “The Quick brown fox
    jumps over the lazy dog.”
    =>
    =>
    

    View Slide

16. =>
    =>
    What do these Examples have in Common?
    “The Quick brown fox jumps
    over the lazy dog.”
    

    View Slide

17. =>
    A => B
    A Neural Network is a Function
    

    View Slide

18. A Neural Network is a Function
    type Model[A, B] = A => B
    

    View Slide

19. val classifier: Model[Image, ImageClass] = ???
    A Neural Network is a Function
    How do we implement
    that thing?
    

    View Slide

20. Neural Network Mechanics
    

    View Slide

21. Neural Networks Work on Numeric Tensors
    

    View Slide

22. Representing Values as Tensors
    Model[Tensor, Tensor]
    encoder: A => Tensor
    decoder: Tensor => B
    Model[A, B]
    64
    432
    67
    62
    7 9 203
    90
    56
    64
    230
    54 76 4
    51
    83
    24
    8
    41
    40
    17
    5
    230
    98
    23
    7
    129
    34
    93
    73
    

    View Slide

23. The Artificial Neuron
    w1
    ∑
    x1
    w2
    x2
    w3
    x3
    wn
    xn
    …
    …
    Apply
    Activation
    Function
    y
    Multiply inputs and weights
    Sum up
    results
    

    View Slide

24. The Artificial Neuron
    w
    1
    ∑
    x
    1
    w
    2
    x
    2
    w
    3
    x
    3
    w
    n
    x
    n
    …
    …
    y
    ∑
    

    View Slide

25. Neural Networks are Layers of Neurons
    ∑
    ∑
    Input
    Layer
    1. Layer 2. Layer
    (Output)
    ∑
    

    View Slide

26. Source: https://github.com/tensorflow/models/tree/master/research/inception
    Neural Networks are Layers of Neurons
    

    View Slide

27. But How Do We Set Those Weights?
    ∑
    ∑
    Input
    Layer
    1. Layer 2. Layer
    (Output)
    ∑
    

    View Slide

28. “What we want is a machine that can
    learn from experience.”
    Alan Turing, 1947
    

    View Slide

29. A Different Programming Paradigm
    Traditional Programming Machine Learning
    

    View Slide

30. A Neural Network Learns its Function
    def train(examples: Map[A, B]): Model[A, B]
    Supervised Learning
    

    View Slide

31. Training Data: Examples with Answers
    Example
    Label (Answer)
    (1 = cat, 0 = not cat)
    1
    0
    1
    val examples: Map[Image, Either[Cat, NotCat]]
    

    View Slide

32. Training a Neural Network
    Forward Pass
    Compute Loss
    Backpropagation
    Update Weights
    Initialize
    Randomly
    

    View Slide

33. Initialize Randomly
    

    View Slide

34. Initialize Randomly
    

    View Slide

35. Forward Pass Forward Pass
    Compute Loss
    Backpropagation
    Update Weights
    

    View Slide

36. Forward Pass
    Cat
    Probability
    = 0.4
    Forward Pass
    Compute Loss
    Backpropagation
    Update Weights
    

    View Slide

37. Compute the Loss
    Loss (simplified) =
    correct answer - predicted answer
    Cat = 1 1
    0.4
    0.6
    Forward Pass
    Compute Loss
    Backpropagation
    Update Weights
    

    View Slide

38. Backpropagation Forward Pass
    Compute Loss
    Backpropagation
    Update Weights
    

    View Slide

39. Update Weights Forward Pass
    Compute Loss
    Backpropagation
    Update Weights
    

    View Slide

40. Update Weights Forward Pass
    Compute Loss
    Backpropagation
    Update Weights
    

    View Slide

41. Error
    Weight
    Loss Function
    Gradient Descent Forward Pass
    Compute Loss
    Backpropagation
    Update Weights
    Source: https://www.coursera.org/learn/machine-learning
    

    View Slide

42. What Makes Deep Learning so
    powerful?
    

    View Slide

43. Performance / Accuracy
    Amount of data
    Traditional Machine
    Learning
    Deep Learning
    Neural Networks Scale
    

    View Slide

44. Neural Networks Compose
    Source: Vinyals et al. Show and Tell: A Neural Image Caption Generator
    

    View Slide

45. Limitations of Deep Learning
    Source: https://www.technologyreview.com/s/608321/this-image-is-why-self-driving-cars-
    come-loaded-with-many-types-of-sensors
    Source: Brown et al, Adversarial Patch (https://arxiv.org/abs/1712.09665)
    

    View Slide

46. View Slide

47. Why Scala?
    Prototyping vs. Production
    

    View Slide

48. Why Scala?
    Type safe tensor operations
    case class Tensor(
    dataType: DataType,
    shape: Shape
    )
    class TypedTensor[DataType, Shape]
    val tensor: TypedTensor[Float, (Width, Height, Channel)] = ...
    

    View Slide

49. Deep Learning Libraries for Scala
    

    View Slide

50. Deep Learning Libraries for Scala
    Source: https://brunk.io/deep-learning-in-scala-part-1-basics-and-libraries.html
    • Mature, Large
    Community &
    Commercial Support
    • Java API
    • Native Scala API
    (ScalNet) in alpha
    • Scala API part of main
    project
    • API very pythonic
    • Most idiomatic & type
    safe Scala API
    • Not part of main
    TensorFlow (yet)
    

    View Slide

51. Deep Learning Libraries for Scala
    Source: https://brunk.io/deep-learning-in-scala-part-1-basics-and-libraries.html
    • Mature, Large
    Community &
    Commercial Support
    • Java API
    • Native Scala API
    (ScalNet) in alpha
    • Scala API part of main
    project
    • API very pythonic
    • Most idiomatic & type
    safe Scala API
    • Not part of main
    TensorFlow (yet)
    

    View Slide

52. View Slide

53. ? => ?
    1. Frame Your Problem
    

    View Slide

54. 1. Frame Your Problem
    =>
    

    View Slide

55. 1. Frame Your Problem
    =>Either[ScalaLogo,
    NotScalaLogo]
    

    View Slide

56. 2. Look for Existing Solutions
    

    View Slide

57. 3. Collect and Prepare Training Data
    Filter Resize Normalize Vectorize ...
    encoder: A => Tensor
    decoder: Tensor => B
    Training Set Test Set
    

    View Slide

58. 3. Collect and Prepare Training Data
    def readImage(filename: Output): Output = {
    val rawImage = tf.data.readFile(filename)
    val image = tf.image.decodeJpeg(rawImage, numChannels = 3)
    tf.image.resizeBilinear(image.expandDims(axis=0), Seq(250, 250))
    .squeeze(Seq(0)).cast(UINT8)
    }
    val trainData: Dataset[(Tensor, Tensor), (Output, Output),
    (DataType, DataType), (Shape, Shape)] =
    tf.data.TensorSlicesDataset(filenamesWithLabels(trainDir))
    .shuffle(bufferSize = 30000, Some(seed))
    .map({ case (filename, label) => (readImage(filename), label)})
    .repeat()
    .batch(128)
    .prefetch(100)
    

    View Slide

59. 4. Choose the Right Architecture
    By Aphex34 - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=45679374
    Source: https://karpathy.github.io/2015/05/21/rnn-effectiveness/
    Fully Connected Neural Network Convolutional Neural Network
    Recurrent Neural Network
    ∑
    ∑
    ∑
    

    View Slide

60. 4. Choose the Right Architecture
    By Aphex34 - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=45679374
    1. Layer
    2. Layer
    3. Layer
    4. Layer
    Convolutional Neural Network
    

    View Slide

61. 5. Train, Evaluate, Refine
    Create /
    Improve
    Train
    Evaluate
    The Model
    Development
    Cycle
    

    View Slide

62. Build a Simple Model
    Source: https://www.flickr.com/photos/foto_db/12163399363 CC-BY-SA
    

    View Slide

63. Build a Simple Model
    import org.platanios.tensorflow.api._
    val input = tf.learn.Input(UINT8, Shape(-1, 250, 250, 3)) // type and shape of our images
    val labelInput = tf.learn.Input(UINT8, Shape(-1)) // type and shape of our labels
    val layer =
    tf.learn.Flatten("Input/Flatten") >> // flatten the images into a single vector
    tf.learn.Cast("Input/Cast", FLOAT32) >> // cast input to float
    tf.learn.Linear("Layer_1/Linear", units = 64) >> // hidden layer with 512 neurons
    tf.learn.ReLU("Layer_1/ReLU“, 0.01f) >> // hidden layer activation
    tf.learn.Linear("OutputLayer/Linear", units = 2) // output layer
    val trainInputLayer = tf.learn.Cast("TrainInput/Cast", INT64) // cast labels to long
    val loss =
    tf.learn.SparseSoftmaxCrossEntropy("Loss/CrossEntropy") >> // loss/error function
    tf.learn.Mean("Loss/Mean") >>
    tf.learn.ScalarSummary("Loss/Summary", "Loss")
    val optimizer = tf.train.GradientDescent(learningRate = 0.001) // the optimizer updates our weights
    // create a Model for training
    val model = tf.learn.Model.supervised(input, layer, labelInput, trainInputLayer, loss, optimizer)
    

    View Slide

64. Train and Evaluate the Model
    val estimator = tf.learn.InMemoryEstimator(model)
    estimator.train(() => trainData)
    Learn / Hooks / Evaluation - Step 0 Evaluator:
    Learn / Hooks / Evaluation - ╔═══════╤════════════╗
    Learn / Hooks / Evaluation - ║ │ Accuracy ║
    Learn / Hooks / Evaluation - ╟───────┼────────────╢
    Learn / Hooks / Evaluation - ║ Train │ 0.1219 ║
    Learn / Hooks / Evaluation - ║ Test │ 0.1217 ║
    Learn / Hooks / Evaluation - ╚═══════╧════════════╝
    Learn / Hooks / Loss Logger - ( 2.831 s) Step: 100, Loss: 2250.2437
    Learn / Hooks / Loss Logger - ( 0.561 s) Step: 200, Loss: 2735.0537
    ...
    Learn / Hooks / Loss Logger - ( 0.537 s) Step: 900, Loss: 877.2033
    Learn / Hooks / Loss Logger - ( 0.699 s) Step: 1000, Loss: 1207.4133
    Learn / Hooks / Evaluation - Step 1000 Evaluator:
    Learn / Hooks / Evaluation - ╔═══════╤════════════╗
    Learn / Hooks / Evaluation - ║ │ Accuracy ║
    Learn / Hooks / Evaluation - ╟───────┼────────────╢
    Learn / Hooks / Evaluation - ║ Train │ 0.8105 ║
    Learn / Hooks / Evaluation - ║ Test │ 0.7888 ║
    Learn / Hooks / Evaluation - ╚═══════╧════════════╝
    Learn / Hooks / Loss Logger - ( 2.599 s) Step: 1100, Loss: 989.0901
    Learn / Hooks / Loss Logger - ( 0.529 s) Step: 1200, Loss: 440.6418
    ...
    

    View Slide

65. Refine the Model
    val layer = tf.learn.Cast("Input/Cast", FLOAT32) >>
    tf.learn.Flatten("Input/Flatten") >>
    tf.learn.Linear("Layer_1/Linear", units = 64) >> // hidden layer
    tf.learn.ReLU("Layer_1/ReLU", 0.01f) >> // hidden layer activation
    tf.learn.Linear("OutputLayer/Linear", units = 2) // output layer
    

    View Slide

66. Refine the Model
    val layer = tf.learn.Cast("Input/Cast", FLOAT32) >>
    tf.learn.Flatten("Input/Flatten") >>
    tf.learn.Linear("Layer_1/Linear", units = 128) >> // hidden layer
    tf.learn.ReLU("Layer_1/ReLU", 0.01f) >> // hidden layer activation
    tf.learn.Linear("OutputLayer/Linear", units = 2) // output layer
    

    View Slide

67. Refine the Model
    val layer = tf.learn.Cast("Input/Cast", FLOAT32) >>
    tf.learn.Flatten("Input/Flatten") >>
    tf.learn.Linear("Layer_1/Linear", units = 128) >> // hidden layer
    tf.learn.ReLU("Layer_1/ReLU", 0.01f) >> // hidden layer activation
    tf.learn.Linear("OutputLayer/Linear", units = 2) // output layer
    val estimator = tf.learn.InMemoryEstimator(model)
    estimator.train(() => trainData)
    

    View Slide

68. Refine the Model
    val layer = tf.learn.Cast("Input/Cast", FLOAT32) >>
    tf.learn.Flatten("Input/Flatten") >>
    tf.learn.Linear("Layer_1/Linear", units = 512) >> // hidden layer
    tf.learn.ReLU("Layer_1/ReLU", 0.01f) >> // hidden layer activation
    tf.learn.Linear("OutputLayer/Linear", units = 2) // output layer
    val estimator = tf.learn.InMemoryEstimator(model)
    estimator.train(() => trainData)
    

    View Slide

69. Refine the Model
    tf.learn.Conv2D("Layer_0/Conv2D", Shape(3, 3, 1, 32), stride1 = 2, stride2 = 2, ValidConvPadding) >>
    tf.learn.AddBias("Layer_0/Bias") >>
    tf.learn.ReLU("Layer_0/ReLU", 0.01f) >>
    tf.learn.MaxPool("Layer_0/MaxPool", windowSize = Seq(1, 2, 2, 1), stride1 = 2, stride2 = 2, ValidConvPadding) >>
    val layer = tf.learn.Cast("Input/Cast", FLOAT32) >>
    tf.learn.Flatten("Input/Flatten") >>
    tf.learn.Linear("Layer_1/Linear", units = 512) >>
    tf.learn.ReLU("Layer_1/ReLU") >>
    tf.learn.Linear("OutputLayer/Linear", units = 2)
    

    View Slide

70. Refine the Model
    tf.learn.Conv2D("Layer_0/Conv2D", Shape(3, 3, 1, 32), stride1 = 2, stride2 = 2, ValidConvPadding) >>
    tf.learn.AddBias("Layer_0/Bias") >>
    tf.learn.ReLU("Layer_0/ReLU", 0.01f) >>
    tf.learn.MaxPool("Layer_0/MaxPool", windowSize = Seq(1, 2, 2, 1), stride1 = 2, stride2 = 2, ValidConvPadding) >>
    val layer = tf.learn.Cast("Input/Cast", FLOAT32) >>
    tf.learn.Flatten("Input/Flatten") >>
    tf.learn.Linear("Layer_1/Linear", units = 512) >>
    tf.learn.ReLU("Layer_1/ReLU") >>
    tf.learn.Linear("OutputLayer/Linear", units = 2)
    val estimator = tf.learn.InMemoryEstimator(model)
    estimator.train(() => trainData)
    

    View Slide

71. Run the Model
    val image: Tensor = readImage(filename)
    val decoder = Seq("not_scala", "scala")
    val result: Tensor = estimator.infer(() => image)
    val label = getLabel(result))
    drawLabel(image, s"Class: $label ($decoder(label))")
    

    View Slide

72. 7. Deploy to Production
    

    View Slide

73. Takeaways
    ● Neural networks are functions
    ● We usually learn them from known examples
    ● Training can be challenging
    ● Scala is a great language for Deep Learning
    ● Interested? Give it a try
    ● Get involved to make it even better
    

    View Slide

74. Thank You for Listening
    Sören Brunk @soebrunk
    https://brunk.io
    

    View Slide