Deep Learning and Natural Language Processing with Spark

Deep learning is taking data science by storm, due to the combination of stable distributed computing technologies, increasing amounts of data, and available computing resources. The presentation covers how to implement a Spark­-ready version of the long short­-term memory (LSTM) neural network, widely used in natural language processing.

Transcript

1. Deep Learning and Natural Language Processing with Spark Melanie Warrick | Skymind | @nyghtowl Andy Petrella | Data Fellas | @noootsab
2. @nyghtowl Machine Learning
4. @nyghtowl
5. @nyghtowl
6. Natural Language Processing @nyghtowl • Question Answer • Image Captioning • Topic Modeling/Sentiment Analysis • Language | Machine Translation • Text Generation NLP is hard
7. @nyghtowl Artificial Neural Nets Input | X Output | y Hidden
8. @nyghtowl Recurrent Neural Net
9. @nyghtowl Long short-term Memory (LSTM)
10. @nyghtowl Sequence to Sequence
11. @nyghtowl Example: Word2Vec Word embeddings represent context King – Man + Woman ~ Queen
12. @nyghtowl Example: Image Captioning
13. @nyghtowl Sequence to Sequence
14. @nyghtowl Sentiment Analysis Reviews “Best part of the movie is the end credits” “It should have been a great movie…” Sentiment
15. @nyghtowl Hadoop Spark AWS Skymind ND4J DeepLearning4J Native & JavaCPP & OpenMP & Cuda 7.5 Canova Data Neural Nets Linear Algebra LIBND4J C Backend
16. @nyghtowl Data Fellas - Spark-Notebook only Scala based notebook that is - scalable and enables interactive work on Spark, Akka, Cassandra, & Kafka - plotting interactive plots in any Scala type - Data Fellas enables data-driven business, bringing productivity to data science in enterprise
17. @nyghtowl Research References RNNs • DL4J Overview: ◦ RNN & LSTM Overview: http://deeplearning4j.org/recurrentnetwork ◦ Using RNNs: http://deeplearning4j.org/usingrnns.html • Karpathy: https://karpathy.github.io/2015/05/21/rnn-effectiveness/ • Intro: http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/ Github Repos • Spark Notebook: https://github.com/andypetrella/spark-notebook • DL4J Examples: https://github.com/deeplearning4j/dl4j-0.4-examples • DL4J Spark Examples: https://github.com/deeplearning4j/dl4j-spark-cdh5-examples General ML Projects (referenced in presentation) • AlphaGo: http://i.dailymail.co.uk/i/pix/2016/03/09/09/320583D500000578-3483569- Google_has_confirmed_its_AlphaGo_computer_has_taken_the_first_vi-a-11_1457516282972.jpg • Switzerland SAR: http://www.forensicmag.com/article/2016/02/autonomous-drones-fly-search-and-rescue-operations • VIV: http://www.dailytech.com/ExSiri+CEO+Poaches+Apple+to+Create+Viv+The+Global+Brain/article36387.htm
18. @nyghtowl Image References • http://www.dailytech. com/ExSiri+CEO+Poaches+Apple+to+Create+Viv+The+Global+Brain/article36387.htm • http://3.bp.blogspot.com/- mMPT3tgVWaQ/U5qVs64HbRI/AAAAAAAAJCM/lEE4OiJmRSY/s1600/thumb-down-smiley.png • http://4.bp.blogspot.com/-pUoO5oOuzOc/VcomU6qKT4I/AAAAAAAAAsg/TonkgL1iEjE/s1600/Screen% 2BShot%2B2015-08-11%2Bat%2B9.43.21%2BAM.png • http://www.ucreative.com/inspiration/interesting-patterns-and-fractals-from-nature/ • http://i.telegraph.co.uk/multimedia/archive/02122/WILLIAM-SHAKESPEAR_2122089b.jpg • https://karpathy.github.io/2015/05/21/rnn-effectiveness/ • https://pbs.twimg.com/media/CJm9HmfVEAEXU0c.jpg:large • http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/ • http://i.dailymail.co.uk/i/pix/2016/03/09/09/320583D500000578-3483569- Google_has_confirmed_its_AlphaGo_computer_has_taken_the_first_vi-a-11_1457516282972.jpg • http://www.forensicmag.com/article/2016/02/autonomous-drones-fly-search-and-rescue-operations • Susan Eraly
19. @nyghtowl Deep Learning and Natural Language Processing with Spark Andy Petrella | Data Fellas | @noootsab Melanie Warrick | Skymind | @nyghtowl