All Aboard the Natural Language Processing Train

Transcript

1. All Aboard the Natural Language Processing Train José Marcelino
2. José Marcelino Data Scientist @Farfetch github.com/josemarcelino linkedin.com/in/josemarcelino1
3. What’s the deal with Natural Language Processing?
4. Can you hear me? Yes! I’m here!
5. Can you hear me? Yes! I’m here!
6. I’m going to the bank. An analysis of the river bank erosion.
7. I’m going to the bank. An analysis of the river bank erosion.
8. John told the woman that Bill was dating.
9. John told the woman that Bill was dating.
10. John told the woman that Bill was dating.
11. We gave the monkeys the bananas because they were hungry. We gave the monkeys the bananas because they were over-ripe.
12. We gave the monkeys the bananas because they were hungry. We gave the monkeys the bananas because they were over-ripe.
13. Red Red Valentino dress. Golden shoes from Golden Goose.
14. Red Red Valentino dress. Golden shoes from Golden Goose.
15. Au revoir! Goodbye!
16. Text-Normalization
17. am, are, is -> be T-shirt, t-shirt, T-Shirt -> t-shirt car, cars, cars’ -> car Dolce&Gabbana? lower and uppercase?
18. I’m at the car festival. I’m buying a X.
19. I’m at the car festival. I’m buying a X. car computer
20. I’m at the car festival. I’m buying a X. P(“car”) > P(“computer”)?
21. I’m at the car festival. I’m buying a X. P(“X”) = P(“X” | “I’m at the car festival. I’m buying a”)
22. “I want red dresses.”
23. “I want red dresses.” [“I”,”want”,”red”,”dresses”,”.”]
24. “I want red dresses.” [“I”,”want”,”red”,”dresses”,”.”] P(“dresses” | “want”,”red”)
25. P(“I” | “<s>”,”<s>”) P(“want” | “<s>”,”I”) P(“red” | “I”,”want”) P(“dresses” | “want”,”red”) P(“.” | “red”,”dresses”)
26. Language Modeling
27. [“I”,”want”,”red”,”dresses”,”.”] “I” = 0 “want” = 1 “red” = 2 “dresses” = 3 “.” = 4
28. Words that appear in the same contexts share semantic meaning
29. Using vector representations can overcome some of these obstacles.
30. http://projector.tensorflow.org/
31. “I want red dresses.” [“I”,”want”,”red”,”dresses”,”.”]
32. Part-of-Speech Tagging
33. “I” - pronoun ”want” - verb “red” - adjective “dresses” - noun “.” - punctuation
34. Named Entity Recognition
35. “I” - Nothing. ”want” - Nothing. “red” - is a Color. “dresses” - is a Category. “.” - Nothing.
36. Machine Learning 101
38. Machine Learning 101
39. End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF https://arxiv.org/pdf/1603.01354.pdf
40. End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF input:Sentence -> output:Annotation Architecture
41. “I want red dresses.” [“I”,”want”,”red”,”dresses”,”.”]
42. [“I”,”want”,”red”,”dresses”,”.”] [“pron”,”verb”,”adj”,”noun”,”punct”]
43. [“I”,”want”,”red”,”dresses”,”.”] [“0”,”0”,”COLOR”,”CATEGORY”,”0”]
44. Architecture
45. “I want red dresses.” [“I”,”want”,”red”,”dresses”,”.”]
46. “I want red dresses.” [“I”,”want”,”red”,”dresses”,”.”] [(0.1, .. ,0.2), (0.3, .. ,0.4) … (0.1, .. 0.1)]
47. Architecture
48. Char Representation
49. “I want red dresses.” [“I”,”want”,”red”,”dresses”,”.”] [(0.1, .. ,0.2, 0.3, 0.4, 0.5), (0.3, .. ,0.4, 0.4, 0.1, 0.3) … (0.1, .. 0.1, 0.4, 0.2, 0.3)]
50. Architecture
51. Recurrent Neural Networks
52. Long Short Term Memory http://colah.github.io/posts/2015-08-Understanding-LSTMs/
53. Architecture
54. Conditional Random Fields
55. Annotated Dataset
56. “I” - pronoun ”want” - verb “red” - adjective “dresses” - noun “.” - punctuation
57. “Penn Treebank WSJ corpus for part-of-speech (POS) tagging and CoNLL 2003 corpus for named entity recognition (NER).”
58. Our end-to-end model outperforms previous state-of-the-art systems, obtaining 97.55% accuracy for POS tagging and 91.21% F1 for NER.
59. “I want red dresses.” [“I”,”want”,”red”,”dresses”,”.”] [“pron”,”verb”,”adj”,”noun”,”punct”]