Over Mt. Stupid to Deep Learning

My slides for my talk at Code.Talks 2018 about MAchine Learning, Deep Learning and my journey through this topics. Video of talk may come later.

Transcript

1. OVER MT. STUPID TO
   DEEP LEARNING…
   Carsten Sandtner \\ @casarock
   Photo by Archie Binamira from Pexels
   https://www.pexels.com/photo/man-wearing-white-shirt-brown-shorts-and-green-backpack-standing-on-hill-672358/
   

   View Slide

2. ABOUT://ME
   My name is Carsten and I’m Technical Director at
   mediaman GmbH in Mayence (Mainz).
   I’m a moz://a Techspeaker and I love the open web and
   everything about open standards for the web!
   I’m tweeting as @casarock
   

   

   … and I asked myself: What is this AI thing?
   

   View Slide

3. View Slide

4. „ “
   –Amy Webb - SXSW 2018
   The robots are going to come and kill us all
   but not before they take over all of our jobs
   

   View Slide

5. „ “
   –Amy Webb - SXSW 2018
   The artificial intelligence ecosystem — 
   flooded with capital, hungry for
   commercial applications, and yet polluted
   with widespread, misplaced optimism and
   fear — will continue to swell
   

   View Slide

6. „ “
   –Google CEO Sundar Pichai - Google I/O 2017
   From mobile first to AI first
   

   View Slide

7. Confidence
   Wisdom
   Mt. Stupid
   Valley of
   despair
   Slope of
   enlightenment
   Plateau of
   sustainability
   

   View Slide

8. Photo by Markus Spiske temporausch.com from Pexels
   https://www.pexels.com/photo/aerial-photography-of-white-mountains-987573/
   

   View Slide

9. BASICS!
   

   View Slide

10. View Slide

11. SOME HISTORY!
    

    View Slide

12. HISTORY
    1950: Neuronal Networks!
    ~1980: Machine Learning
    Today: Deep Learning
    Photo by Dick Thomas Johnson
    https://www.flickr.com/photos/31029865@N06/14810867549/
    

    View Slide

13. AI
    Machine
    Learning
    Deep
    Learning
    

    View Slide

14. MACHINE LEARNING
    

    View Slide

15. MACHINE LEARNING
    Statistics!
    Correlation
    Regression
    a => weights
    Const => Offset/Bias
    Y = Const + aX1
    + bX2
    + cX3
    + ... + zXn
    

    View Slide

16. Activation
    Function
    Output
    Input Weight
    Offset
    z = ∑w0x0
    w0
    x0
    +b
    y = f(z)
    y0
    

    View Slide

17. ACTIVATION FUNCTIONS
    0 2 4 6
    -4
    -6 -2
    0.0
    0.2
    0.4
    0.6
    0.8
    1.0
    0 2 4 6
    -4
    -6 -2
    0
    1
    2
    3
    4
    5
    Sigmoid ReLu
    R(z) = max(0, z)
    sig(z) = 1/(1+e-z)
    Rectifier Linear Unit
    

    View Slide

18. EXAMPLE:
    PREDICTIVE MAINTENANCE
    

    View Slide

19. SIMPLE EXAMPLE
    Machine
    Sensors x1, x2 and x3

    Weights a, b and c
    Y = Const + ax1
    + bx2
    + cx3
    Const: Value when x1, x2 and x3 are 0
    Training data: 100.000 Datasets.
    Keep 25.000 for validation
    Train with 75.000 -> Vary weights until result (Y) is ok
    Verify your model with the 25.000 sets for validation
    

    View Slide

20. View Slide

21. HOW MACHINES LEARN
    

    View Slide

22. HOW MACHINES LEARN
    Supervised Learning
    Unsupervised Learning
    Reinforcement Learning
    

    View Slide

23. SUPERVISED
    Useful for predictions
    and classifications
    Popular use case: 

    Image recognition
    Needs classified
    training sets.
    

    View Slide

24. UNSUPERVISED
    Useful for
    segmentation and
    clustering
    Clustered data needs
    revision by a human
    Good for dimensional
    reduction
    

    View Slide

25. REINFORCEMENT
    has not a defined result
    for training data.
    Using rewards for good
    results - if it isn’t good do
    it never again, bad boy!
    Example: Learn how to
    play a game just while
    analyse every pixel
    Popular Example: 

    Alpha Go
    

    View Slide

26. MY JOURNEY!
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability
    

    View Slide

27. Photo by Paula May on Unsplash
    https://unsplash.com/photos/AJqeO_-ifx0
    

    View Slide

28. DEEP LEARNING (DL)
    

    View Slide

29. NEURONAL NETWORKS
    

    View Slide

30. ME.
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability
    

    View Slide

31. Photo by Mario Álvarez on Unsplash
    https://unsplash.com/photos/M1YdS0g8SRA
    

    View Slide

32. NEURON
    Neuron
    Activation
    Function
    Output
    Input Weight
    Offset
    z = ∑wixi
    w0
    x0
    +b
    y = f(z)
    y0
    x1
    xn
    w1
    wn
    y1
    yn
    LAYER
    

    View Slide

33. . . . .
    Input Layer Output Layer
    Hidden Layers
    

    View Slide

34. . . . .
    Input Layer Output Layer
    Hidden Layers
    Forward pass
    Init with random weights and offsets
    

    View Slide

35. DETERMINE THE ERROR
    

    View Slide

36. ERROR FUNCTION
    f(x) = mx + b
    

    View Slide

37. ERROR FUNCTION
    f(x) = mx + b
    Mean Squared Error = Avg(Error2)
    

    View Slide

38. . . . .
    Input Layer Output Layer
    Hidden Layers
    Back propagation
    Propagate the error into every neuron
    err
    err
    err
    err
    err
    err
    err
    err
    err
    err
    err
    

    View Slide

39. BACKPROPAGATION
    For every neuron from output to input
    Check neurons share in the error (using weights)
    Fine tune weights (in very small steps eg. 0.001)
    This is a Hyperparameter: Learning Rate!
    Applies to the whole network
    Every weight adjusted?
    Start a new run
    This is called an new Epoch (next Hyperparameter!)
    

    View Slide

40. GRADIENT DESCENT
    

    View Slide

41. OUR GOAL: CONVERGENCE
    

    View Slide

42. CONVERGENCE
    Always take a look at your error.
    Is it minimal? Stop learning!
    Validate your model with other data
    (not the learning data!)
    Try to avoid overfitting!
    

    View Slide

43. ME.
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability
    

    View Slide

44. Photo by Will Langenberg on Unsplash
    https://unsplash.com/photos/S9uZOfeYi1U
    

    View Slide

45. CONVOLUTIONAL NEURAL
    NETWORKS
    

    View Slide

46. CONVOLUTIONS
    

    View Slide

47. CONVOLUTIONS
    

    View Slide

48. CONVOLUTIONS
    We train this
    little thing!
    

    View Slide

49. CONVOLUTIONAL NEURAL NETWORK
    Conv
    ReLu
    Pool
    Conv
    Conv
    Conv
    Conv
    Conv
    ReLu
    ReLu
    ReLu
    ReLu
    ReLu
    Pool
    Pool
    Dog
    Cat
    Mouse
    FC
    

    View Slide

50. Photo by Will Langenberg on Unsplash
    https://unsplash.com/photos/S9uZOfeYi1U
    

    View Slide

51. ME.
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability
    

    View Slide

52. Photo by Natalia_Kollegova
    https://pixabay.com/de/fata-morgana-das-gebirge-berge-2738131/
    

    View Slide

53. THOUGHT ABOUT MY GOALS
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability
    

    View Slide

54. WHAT IS IT
    SUITABLE FOR?
    

    View Slide

55. View Slide

56. View Slide

57. View Slide

58. SOME BETTER EXAMPLES
    Categorization of products in an online shop
    using their images.
    Using cognitive services for Natural Language
    Processing (NLP) voice or text based
    Using a cloud based AI Service!
    

    View Slide

59. AZURE COGNITIVE SERVICES
    https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/
    

    View Slide

60. „ “
    No matter what you do, you need
    sh*tloads of data!
    

    View Slide

61. EVERYTHING IS GREAT!
    

    View Slide

62. DEPENDS…
    

    View Slide

63. . . . .
    Input Layer Output Layer
    Hidden Layers
    

    View Slide

64. WHO IS RESPONSIBLE?
    

    View Slide

65. Photo by Katie Salerno
    https://www.pexels.com/photo/love-people-romance-engagement-18396/
    

    View Slide

66. Photo by Cade Roberts on Unsplash
    https://unsplash.com/photos/H0Aud5lhupc
    

    View Slide

67. Photo by Martin Shreder on Unsplash
    https://unsplash.com/photos/5Xwaj9gaR0g
    

    View Slide

68. AI (ML/DL) IS AWESOME.
    USE IT WISELY.
    Thank you!
    @casarock for
    

    View Slide