Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Over Mt. Stupid to Deep Learning

Over Mt. Stupid to Deep Learning

My slides for my talk at Code.Talks 2018 about MAchine Learning, Deep Learning and my journey through this topics. Video of talk may come later.

Carsten Sandtner

October 18, 2018
Tweet

More Decks by Carsten Sandtner

Other Decks in Technology

Transcript

  1. OVER MT. STUPID TO
    DEEP LEARNING…
    Carsten Sandtner \\ @casarock
    Photo by Archie Binamira from Pexels
    https://www.pexels.com/photo/man-wearing-white-shirt-brown-shorts-and-green-backpack-standing-on-hill-672358/

    View Slide

  2. ABOUT://ME
    My name is Carsten and I’m Technical Director at
    mediaman GmbH in Mayence (Mainz).
    I’m a moz://a Techspeaker and I love the open web and
    everything about open standards for the web!
    I’m tweeting as @casarock


    … and I asked myself: What is this AI thing?

    View Slide

  3. View Slide

  4. „ “
    –Amy Webb - SXSW 2018
    The robots are going to come and kill us all
    but not before they take over all of our jobs

    View Slide

  5. „ “
    –Amy Webb - SXSW 2018
    The artificial intelligence ecosystem — 
    flooded with capital, hungry for
    commercial applications, and yet polluted
    with widespread, misplaced optimism and
    fear — will continue to swell

    View Slide

  6. „ “
    –Google CEO Sundar Pichai - Google I/O 2017
    From mobile first to AI first

    View Slide

  7. Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability

    View Slide

  8. Photo by Markus Spiske temporausch.com from Pexels
    https://www.pexels.com/photo/aerial-photography-of-white-mountains-987573/

    View Slide

  9. BASICS!

    View Slide

  10. View Slide

  11. SOME HISTORY!

    View Slide

  12. HISTORY
    1950: Neuronal Networks!
    ~1980: Machine Learning
    Today: Deep Learning
    Photo by Dick Thomas Johnson
    https://www.flickr.com/photos/[email protected]/14810867549/

    View Slide

  13. AI
    Machine
    Learning
    Deep
    Learning

    View Slide

  14. MACHINE LEARNING

    View Slide

  15. MACHINE LEARNING
    Statistics!
    Correlation
    Regression
    a => weights
    Const => Offset/Bias
    Y = Const + aX1
    + bX2
    + cX3
    + ... + zXn

    View Slide

  16. Activation
    Function
    Output
    Input Weight
    Offset
    z = ∑w0x0
    w0
    x0
    +b
    y = f(z)
    y0

    View Slide

  17. ACTIVATION FUNCTIONS
    0 2 4 6
    -4
    -6 -2
    0.0
    0.2
    0.4
    0.6
    0.8
    1.0
    0 2 4 6
    -4
    -6 -2
    0
    1
    2
    3
    4
    5
    Sigmoid ReLu
    R(z) = max(0, z)
    sig(z) = 1/(1+e-z)
    Rectifier Linear Unit

    View Slide

  18. EXAMPLE:
    PREDICTIVE MAINTENANCE

    View Slide

  19. SIMPLE EXAMPLE
    Machine
    Sensors x1, x2 and x3

    Weights a, b and c
    Y = Const + ax1
    + bx2
    + cx3
    Const: Value when x1, x2 and x3 are 0
    Training data: 100.000 Datasets.
    Keep 25.000 for validation
    Train with 75.000 -> Vary weights until result (Y) is ok
    Verify your model with the 25.000 sets for validation

    View Slide

  20. View Slide

  21. HOW MACHINES LEARN

    View Slide

  22. HOW MACHINES LEARN
    Supervised Learning
    Unsupervised Learning
    Reinforcement Learning

    View Slide

  23. SUPERVISED
    Useful for predictions
    and classifications
    Popular use case: 

    Image recognition
    Needs classified
    training sets.

    View Slide

  24. UNSUPERVISED
    Useful for
    segmentation and
    clustering
    Clustered data needs
    revision by a human
    Good for dimensional
    reduction

    View Slide

  25. REINFORCEMENT
    has not a defined result
    for training data.
    Using rewards for good
    results - if it isn’t good do
    it never again, bad boy!
    Example: Learn how to
    play a game just while
    analyse every pixel
    Popular Example: 

    Alpha Go

    View Slide

  26. MY JOURNEY!
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability

    View Slide

  27. Photo by Paula May on Unsplash
    https://unsplash.com/photos/AJqeO_-ifx0

    View Slide

  28. DEEP LEARNING (DL)

    View Slide

  29. NEURONAL NETWORKS

    View Slide

  30. ME.
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability

    View Slide

  31. Photo by Mario Álvarez on Unsplash
    https://unsplash.com/photos/M1YdS0g8SRA

    View Slide

  32. NEURON
    Neuron
    Activation
    Function
    Output
    Input Weight
    Offset
    z = ∑wixi
    w0
    x0
    +b
    y = f(z)
    y0
    x1
    xn
    w1
    wn
    y1
    yn
    LAYER

    View Slide

  33. . . . .
    Input Layer Output Layer
    Hidden Layers

    View Slide

  34. . . . .
    Input Layer Output Layer
    Hidden Layers
    Forward pass
    Init with random weights and offsets

    View Slide

  35. DETERMINE THE ERROR

    View Slide

  36. ERROR FUNCTION
    f(x) = mx + b

    View Slide

  37. ERROR FUNCTION
    f(x) = mx + b
    Mean Squared Error = Avg(Error2)

    View Slide

  38. . . . .
    Input Layer Output Layer
    Hidden Layers
    Back propagation
    Propagate the error into every neuron
    err
    err
    err
    err
    err
    err
    err
    err
    err
    err
    err

    View Slide

  39. BACKPROPAGATION
    For every neuron from output to input
    Check neurons share in the error (using weights)
    Fine tune weights (in very small steps eg. 0.001)
    This is a Hyperparameter: Learning Rate!
    Applies to the whole network
    Every weight adjusted?
    Start a new run
    This is called an new Epoch (next Hyperparameter!)

    View Slide

  40. GRADIENT DESCENT

    View Slide

  41. OUR GOAL: CONVERGENCE

    View Slide

  42. CONVERGENCE
    Always take a look at your error.
    Is it minimal? Stop learning!
    Validate your model with other data
    (not the learning data!)
    Try to avoid overfitting!

    View Slide

  43. ME.
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability

    View Slide

  44. Photo by Will Langenberg on Unsplash
    https://unsplash.com/photos/S9uZOfeYi1U

    View Slide

  45. CONVOLUTIONAL NEURAL
    NETWORKS

    View Slide

  46. CONVOLUTIONS

    View Slide

  47. CONVOLUTIONS

    View Slide

  48. CONVOLUTIONS
    We train this
    little thing!

    View Slide

  49. CONVOLUTIONAL NEURAL NETWORK
    Conv
    ReLu
    Pool
    Conv
    Conv
    Conv
    Conv
    Conv
    ReLu
    ReLu
    ReLu
    ReLu
    ReLu
    Pool
    Pool
    Dog
    Cat
    Mouse
    FC

    View Slide

  50. Photo by Will Langenberg on Unsplash
    https://unsplash.com/photos/S9uZOfeYi1U

    View Slide

  51. ME.
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability

    View Slide

  52. Photo by Natalia_Kollegova
    https://pixabay.com/de/fata-morgana-das-gebirge-berge-2738131/

    View Slide

  53. THOUGHT ABOUT MY GOALS
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability

    View Slide

  54. WHAT IS IT
    SUITABLE FOR?

    View Slide

  55. View Slide

  56. View Slide

  57. View Slide

  58. SOME BETTER EXAMPLES
    Categorization of products in an online shop
    using their images.
    Using cognitive services for Natural Language
    Processing (NLP) voice or text based
    Using a cloud based AI Service!

    View Slide

  59. AZURE COGNITIVE SERVICES
    https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/

    View Slide

  60. „ “
    No matter what you do, you need
    sh*tloads of data!

    View Slide

  61. EVERYTHING IS GREAT!

    View Slide

  62. DEPENDS…

    View Slide

  63. . . . .
    Input Layer Output Layer
    Hidden Layers

    View Slide

  64. WHO IS RESPONSIBLE?

    View Slide

  65. Photo by Katie Salerno
    https://www.pexels.com/photo/love-people-romance-engagement-18396/

    View Slide

  66. Photo by Cade Roberts on Unsplash
    https://unsplash.com/photos/H0Aud5lhupc

    View Slide

  67. Photo by Martin Shreder on Unsplash
    https://unsplash.com/photos/5Xwaj9gaR0g

    View Slide

  68. AI (ML/DL) IS AWESOME.
    USE IT WISELY.
    Thank you!
    @casarock for

    View Slide