Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Learning and Natural Language Processing with Spark

Deep Learning and Natural Language Processing with Spark

Deep learning is taking data science by storm, due to the combination of stable distributed computing technologies, increasing amounts of data, and available computing resources. The presentation covers how to implement a Spark­-ready version of the long short­-term memory (LSTM) neural network, widely used in natural language processing.

Melanie Warrick

June 02, 2016

More Decks by Melanie Warrick

Other Decks in Technology


  1. Deep Learning and Natural Language Processing with Spark Melanie Warrick

    | Skymind | @nyghtowl Andy Petrella | Data Fellas | @noootsab
  2. @nyghtowl Machine Learning

  3. None
  4. @nyghtowl

  5. @nyghtowl

  6. Natural Language Processing @nyghtowl • Question Answer • Image Captioning

    • Topic Modeling/Sentiment Analysis • Language | Machine Translation • Text Generation NLP is hard
  7. @nyghtowl Artificial Neural Nets Input | X Output | y

  8. @nyghtowl Recurrent Neural Net

  9. @nyghtowl Long short-term Memory (LSTM)

  10. @nyghtowl Sequence to Sequence

  11. @nyghtowl Example: Word2Vec Word embeddings represent context King – Man

    + Woman ~ Queen
  12. @nyghtowl Example: Image Captioning

  13. @nyghtowl Sequence to Sequence

  14. @nyghtowl Sentiment Analysis Reviews “Best part of the movie is

    the end credits” “It should have been a great movie…” Sentiment
  15. @nyghtowl Hadoop Spark AWS Skymind ND4J DeepLearning4J Native & JavaCPP

    & OpenMP & Cuda 7.5 Canova Data Neural Nets Linear Algebra LIBND4J C Backend
  16. @nyghtowl Data Fellas - Spark-Notebook only Scala based notebook that

    is - scalable and enables interactive work on Spark, Akka, Cassandra, & Kafka - plotting interactive plots in any Scala type - Data Fellas enables data-driven business, bringing productivity to data science in enterprise
  17. @nyghtowl Research References RNNs • DL4J Overview: ◦ RNN &

    LSTM Overview: http://deeplearning4j.org/recurrentnetwork ◦ Using RNNs: http://deeplearning4j.org/usingrnns.html • Karpathy: https://karpathy.github.io/2015/05/21/rnn-effectiveness/ • Intro: http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/ Github Repos • Spark Notebook: https://github.com/andypetrella/spark-notebook • DL4J Examples: https://github.com/deeplearning4j/dl4j-0.4-examples • DL4J Spark Examples: https://github.com/deeplearning4j/dl4j-spark-cdh5-examples General ML Projects (referenced in presentation) • AlphaGo: http://i.dailymail.co.uk/i/pix/2016/03/09/09/320583D500000578-3483569- Google_has_confirmed_its_AlphaGo_computer_has_taken_the_first_vi-a-11_1457516282972.jpg • Switzerland SAR: http://www.forensicmag.com/article/2016/02/autonomous-drones-fly-search-and-rescue-operations • VIV: http://www.dailytech.com/ExSiri+CEO+Poaches+Apple+to+Create+Viv+The+Global+Brain/article36387.htm
  18. @nyghtowl Image References • http://www.dailytech. com/ExSiri+CEO+Poaches+Apple+to+Create+Viv+The+Global+Brain/article36387.htm • http://3.bp.blogspot.com/- mMPT3tgVWaQ/U5qVs64HbRI/AAAAAAAAJCM/lEE4OiJmRSY/s1600/thumb-down-smiley.png •

    http://4.bp.blogspot.com/-pUoO5oOuzOc/VcomU6qKT4I/AAAAAAAAAsg/TonkgL1iEjE/s1600/Screen% 2BShot%2B2015-08-11%2Bat%2B9.43.21%2BAM.png • http://www.ucreative.com/inspiration/interesting-patterns-and-fractals-from-nature/ • http://i.telegraph.co.uk/multimedia/archive/02122/WILLIAM-SHAKESPEAR_2122089b.jpg • https://karpathy.github.io/2015/05/21/rnn-effectiveness/ • https://pbs.twimg.com/media/CJm9HmfVEAEXU0c.jpg:large • http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/ • http://i.dailymail.co.uk/i/pix/2016/03/09/09/320583D500000578-3483569- Google_has_confirmed_its_AlphaGo_computer_has_taken_the_first_vi-a-11_1457516282972.jpg • http://www.forensicmag.com/article/2016/02/autonomous-drones-fly-search-and-rescue-operations • Susan Eraly
  19. @nyghtowl Deep Learning and Natural Language Processing with Spark Andy

    Petrella | Data Fellas | @noootsab Melanie Warrick | Skymind | @nyghtowl