Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Over Mt. Stupid to Deep Learning

Over Mt. Stupid to Deep Learning

My slides for my talk at Code.Talks 2018 about MAchine Learning, Deep Learning and my journey through this topics. Video of talk may come later.

Carsten Sandtner

October 18, 2018
Tweet

More Decks by Carsten Sandtner

Other Decks in Technology

Transcript

  1. OVER MT. STUPID TO
    DEEP LEARNING…
    Carsten Sandtner \\ @casarock
    Photo by Archie Binamira from Pexels
    https://www.pexels.com/photo/man-wearing-white-shirt-brown-shorts-and-green-backpack-standing-on-hill-672358/

    View full-size slide

  2. ABOUT://ME
    My name is Carsten and I’m Technical Director at
    mediaman GmbH in Mayence (Mainz).
    I’m a moz://a Techspeaker and I love the open web and
    everything about open standards for the web!
    I’m tweeting as @casarock


    … and I asked myself: What is this AI thing?

    View full-size slide

  3. „ “
    –Amy Webb - SXSW 2018
    The robots are going to come and kill us all
    but not before they take over all of our jobs

    View full-size slide

  4. „ “
    –Amy Webb - SXSW 2018
    The artificial intelligence ecosystem — 
    flooded with capital, hungry for
    commercial applications, and yet polluted
    with widespread, misplaced optimism and
    fear — will continue to swell

    View full-size slide

  5. „ “
    –Google CEO Sundar Pichai - Google I/O 2017
    From mobile first to AI first

    View full-size slide

  6. Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability

    View full-size slide

  7. Photo by Markus Spiske temporausch.com from Pexels
    https://www.pexels.com/photo/aerial-photography-of-white-mountains-987573/

    View full-size slide

  8. SOME HISTORY!

    View full-size slide

  9. HISTORY
    1950: Neuronal Networks!
    ~1980: Machine Learning
    Today: Deep Learning
    Photo by Dick Thomas Johnson
    https://www.flickr.com/photos/31029865@N06/14810867549/

    View full-size slide

  10. AI
    Machine
    Learning
    Deep
    Learning

    View full-size slide

  11. MACHINE LEARNING

    View full-size slide

  12. MACHINE LEARNING
    Statistics!
    Correlation
    Regression
    a => weights
    Const => Offset/Bias
    Y = Const + aX1
    + bX2
    + cX3
    + ... + zXn

    View full-size slide

  13. Activation
    Function
    Output
    Input Weight
    Offset
    z = ∑w0x0
    w0
    x0
    +b
    y = f(z)
    y0

    View full-size slide

  14. ACTIVATION FUNCTIONS
    0 2 4 6
    -4
    -6 -2
    0.0
    0.2
    0.4
    0.6
    0.8
    1.0
    0 2 4 6
    -4
    -6 -2
    0
    1
    2
    3
    4
    5
    Sigmoid ReLu
    R(z) = max(0, z)
    sig(z) = 1/(1+e-z)
    Rectifier Linear Unit

    View full-size slide

  15. EXAMPLE:
    PREDICTIVE MAINTENANCE

    View full-size slide

  16. SIMPLE EXAMPLE
    Machine
    Sensors x1, x2 and x3

    Weights a, b and c
    Y = Const + ax1
    + bx2
    + cx3
    Const: Value when x1, x2 and x3 are 0
    Training data: 100.000 Datasets.
    Keep 25.000 for validation
    Train with 75.000 -> Vary weights until result (Y) is ok
    Verify your model with the 25.000 sets for validation

    View full-size slide

  17. HOW MACHINES LEARN

    View full-size slide

  18. HOW MACHINES LEARN
    Supervised Learning
    Unsupervised Learning
    Reinforcement Learning

    View full-size slide

  19. SUPERVISED
    Useful for predictions
    and classifications
    Popular use case: 

    Image recognition
    Needs classified
    training sets.

    View full-size slide

  20. UNSUPERVISED
    Useful for
    segmentation and
    clustering
    Clustered data needs
    revision by a human
    Good for dimensional
    reduction

    View full-size slide

  21. REINFORCEMENT
    has not a defined result
    for training data.
    Using rewards for good
    results - if it isn’t good do
    it never again, bad boy!
    Example: Learn how to
    play a game just while
    analyse every pixel
    Popular Example: 

    Alpha Go

    View full-size slide

  22. MY JOURNEY!
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability

    View full-size slide

  23. Photo by Paula May on Unsplash
    https://unsplash.com/photos/AJqeO_-ifx0

    View full-size slide

  24. DEEP LEARNING (DL)

    View full-size slide

  25. NEURONAL NETWORKS

    View full-size slide

  26. ME.
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability

    View full-size slide

  27. Photo by Mario Álvarez on Unsplash
    https://unsplash.com/photos/M1YdS0g8SRA

    View full-size slide

  28. NEURON
    Neuron
    Activation
    Function
    Output
    Input Weight
    Offset
    z = ∑wixi
    w0
    x0
    +b
    y = f(z)
    y0
    x1
    xn
    w1
    wn
    y1
    yn
    LAYER

    View full-size slide

  29. . . . .
    Input Layer Output Layer
    Hidden Layers

    View full-size slide

  30. . . . .
    Input Layer Output Layer
    Hidden Layers
    Forward pass
    Init with random weights and offsets

    View full-size slide

  31. DETERMINE THE ERROR

    View full-size slide

  32. ERROR FUNCTION
    f(x) = mx + b

    View full-size slide

  33. ERROR FUNCTION
    f(x) = mx + b
    Mean Squared Error = Avg(Error2)

    View full-size slide

  34. . . . .
    Input Layer Output Layer
    Hidden Layers
    Back propagation
    Propagate the error into every neuron
    err
    err
    err
    err
    err
    err
    err
    err
    err
    err
    err

    View full-size slide

  35. BACKPROPAGATION
    For every neuron from output to input
    Check neurons share in the error (using weights)
    Fine tune weights (in very small steps eg. 0.001)
    This is a Hyperparameter: Learning Rate!
    Applies to the whole network
    Every weight adjusted?
    Start a new run
    This is called an new Epoch (next Hyperparameter!)

    View full-size slide

  36. GRADIENT DESCENT

    View full-size slide

  37. OUR GOAL: CONVERGENCE

    View full-size slide

  38. CONVERGENCE
    Always take a look at your error.
    Is it minimal? Stop learning!
    Validate your model with other data
    (not the learning data!)
    Try to avoid overfitting!

    View full-size slide

  39. ME.
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability

    View full-size slide

  40. Photo by Will Langenberg on Unsplash
    https://unsplash.com/photos/S9uZOfeYi1U

    View full-size slide

  41. CONVOLUTIONAL NEURAL
    NETWORKS

    View full-size slide

  42. CONVOLUTIONS

    View full-size slide

  43. CONVOLUTIONS

    View full-size slide

  44. CONVOLUTIONS
    We train this
    little thing!

    View full-size slide

  45. CONVOLUTIONAL NEURAL NETWORK
    Conv
    ReLu
    Pool
    Conv
    Conv
    Conv
    Conv
    Conv
    ReLu
    ReLu
    ReLu
    ReLu
    ReLu
    Pool
    Pool
    Dog
    Cat
    Mouse
    FC

    View full-size slide

  46. Photo by Will Langenberg on Unsplash
    https://unsplash.com/photos/S9uZOfeYi1U

    View full-size slide

  47. ME.
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability

    View full-size slide

  48. Photo by Natalia_Kollegova
    https://pixabay.com/de/fata-morgana-das-gebirge-berge-2738131/

    View full-size slide

  49. THOUGHT ABOUT MY GOALS
    Confidence
    Wisdom
    Mt. Stupid
    Valley of
    despair
    Slope of
    enlightenment
    Plateau of
    sustainability

    View full-size slide

  50. WHAT IS IT
    SUITABLE FOR?

    View full-size slide

  51. SOME BETTER EXAMPLES
    Categorization of products in an online shop
    using their images.
    Using cognitive services for Natural Language
    Processing (NLP) voice or text based
    Using a cloud based AI Service!

    View full-size slide

  52. AZURE COGNITIVE SERVICES
    https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/

    View full-size slide

  53. „ “
    No matter what you do, you need
    sh*tloads of data!

    View full-size slide

  54. EVERYTHING IS GREAT!

    View full-size slide

  55. . . . .
    Input Layer Output Layer
    Hidden Layers

    View full-size slide

  56. WHO IS RESPONSIBLE?

    View full-size slide

  57. Photo by Katie Salerno
    https://www.pexels.com/photo/love-people-romance-engagement-18396/

    View full-size slide

  58. Photo by Cade Roberts on Unsplash
    https://unsplash.com/photos/H0Aud5lhupc

    View full-size slide

  59. Photo by Martin Shreder on Unsplash
    https://unsplash.com/photos/5Xwaj9gaR0g

    View full-size slide

  60. AI (ML/DL) IS AWESOME.
    USE IT WISELY.
    Thank you!
    @casarock for

    View full-size slide