Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Learning and Natural Language Processing with Spark - Berlin

Deep Learning and Natural Language Processing with Spark - Berlin

Melanie Warrick

June 17, 2016
Tweet

More Decks by Melanie Warrick

Other Decks in Technology

Transcript

  1. Deep Learning and
    Natural Language
    Processing with Spark
    Melanie Warrick | Skymind | @nyghtowl
    Andy Petrella | Data Fellas | @noootsab

    View full-size slide

  2. @nyghtowl
    Machine Learning

    View full-size slide

  3. Natural Language Processing
    @nyghtowl
    ● Question Answer
    ● Image Captioning
    ● Topic Modeling/Sentiment Analysis
    ● Language | Machine Translation
    ● Text Generation
    NLP is hard

    View full-size slide

  4. @nyghtowl
    Artificial Neural Nets
    Output | y
    Hidden
    Loss Function
    Output
    k j
    X
    M
    kj
    W
    y
    Run until error stops improving = converge
    Input | X

    View full-size slide

  5. @nyghtowl
    Recurrent Neural Net

    View full-size slide

  6. @nyghtowl
    Long short-term Memory (LSTM)

    View full-size slide

  7. @nyghtowl
    Sequence to Sequence

    View full-size slide

  8. @nyghtowl
    Example: Word2Vec
    Word embeddings represent context
    King – Man + Woman ~ Queen

    View full-size slide

  9. @nyghtowl
    Example: Image Captioning

    View full-size slide

  10. @nyghtowl
    Sequence to Sequence

    View full-size slide

  11. @nyghtowl
    Sentiment Analysis
    Reviews
    “Best part of the movie is the end credits”
    “It should have been a great movie…”
    Sentiment

    View full-size slide

  12. @nyghtowl
    Hadoop
    Spark AWS
    Skymind
    ND4J
    DeepLearning4J
    Native & JavaCPP & OpenMP & Cuda 7.5
    Canova
    Data
    Neural Nets
    Linear Algebra
    LIBND4J
    C Backend

    View full-size slide

  13. @nyghtowl
    Data Fellas
    - Spark-Notebook only Scala based notebook that is
    - scalable and enables interactive work on Spark,
    Akka, Cassandra, & Kafka
    - plotting interactive plots in any Scala type
    - Data Fellas enables data-driven business, bringing
    productivity to data science in enterprise

    View full-size slide

  14. @nyghtowl
    Cluster
    Juju bundle including:
    ● DL4J
    ● Mesos
    ● Spark
    ● Spark Notebook

    View full-size slide

  15. @nyghtowl
    Blog
    Making deep learning accessible on Openstack

    View full-size slide

  16. @nyghtowl
    Research References
    RNNs
    ● DL4J Overview:
    ○ RNN & LSTM Overview: http://deeplearning4j.org/recurrentnetwork
    ○ Using RNNs: http://deeplearning4j.org/usingrnns.html
    ● Karpathy: https://karpathy.github.io/2015/05/21/rnn-effectiveness/
    ● Intro: http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/
    Github Repos
    ● Spark Notebook: https://github.com/andypetrella/spark-notebook
    ● DL4J Examples: https://github.com/deeplearning4j/dl4j-0.4-examples
    ● DL4J Spark Examples: https://github.com/deeplearning4j/dl4j-spark-cdh5-examples
    General ML Projects (referenced in presentation)
    ● AlphaGo: http://i.dailymail.co.uk/i/pix/2016/03/09/09/320583D500000578-3483569-
    Google_has_confirmed_its_AlphaGo_computer_has_taken_the_first_vi-a-11_1457516282972.jpg
    ● Switzerland SAR: http://www.forensicmag.com/article/2016/02/autonomous-drones-fly-search-and-rescue-operations
    ● VIV: http://www.dailytech.com/ExSiri+CEO+Poaches+Apple+to+Create+Viv+The+Global+Brain/article36387.htm

    View full-size slide

  17. @nyghtowl
    Image References
    ● http://www.dailytech.
    com/ExSiri+CEO+Poaches+Apple+to+Create+Viv+The+Global+Brain/article36387.htm
    ● http://3.bp.blogspot.com/-
    mMPT3tgVWaQ/U5qVs64HbRI/AAAAAAAAJCM/lEE4OiJmRSY/s1600/thumb-down-smiley.png
    ● http://4.bp.blogspot.com/-pUoO5oOuzOc/VcomU6qKT4I/AAAAAAAAAsg/TonkgL1iEjE/s1600/Screen%
    2BShot%2B2015-08-11%2Bat%2B9.43.21%2BAM.png
    ● http://www.ucreative.com/inspiration/interesting-patterns-and-fractals-from-nature/
    ● http://i.telegraph.co.uk/multimedia/archive/02122/WILLIAM-SHAKESPEAR_2122089b.jpg
    ● https://karpathy.github.io/2015/05/21/rnn-effectiveness/
    ● https://pbs.twimg.com/media/CJm9HmfVEAEXU0c.jpg:large
    ● http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/
    ● http://i.dailymail.co.uk/i/pix/2016/03/09/09/320583D500000578-3483569-
    Google_has_confirmed_its_AlphaGo_computer_has_taken_the_first_vi-a-11_1457516282972.jpg
    ● http://www.forensicmag.com/article/2016/02/autonomous-drones-fly-search-and-rescue-operations
    ● Susan Eraly

    View full-size slide

  18. @nyghtowl
    Deep Learning and Natural
    Language Processing with Spark
    Andy Petrella | Data Fellas | @noootsab
    Melanie Warrick | Skymind | @nyghtowl

    View full-size slide