Interpretable Representations for Affect and Semantics

While NLP has advanced considerably in recent years, state-of-the-art models can no longer easily be inspected and understood directly. This talk presents a series of interpretable approaches for affect and semantics-related settings.

The first part of the talk will present interpretable vectors custom-tailored for emotions and their connections to words, fonts, and colors. Subsequently, I will discuss NLP approaches for emojis and their connection to emotion. The final part of the talk will consider structured representations to better model the structure of a sentence or other kinds of knowledge that may be useful in downstream applications.

Transcript

1. Interpretable Representations for Affect and Semantics Gerard de Melo http://gerard.demelo.org Interpretable Representations for Affect and Semantics Gerard de Melo http://gerard.demelo.org
2. Machine Learning Machine Learning Examples Negative Positive
3. Machine Learning Machine Learning Classifier Model Model Examples Learning Learning Negative Positive
4. Machine Learning Machine Learning Classifier Model Model Examples Learning Learning ??? Negative Positive
5. Machine Learning Machine Learning Probably Positive! Prediction Prediction Classifier Model Model Examples Learning Learning Negative Positive
6. Machine Learning Machine Learning Probably Positive! Prediction Prediction Classifier Model Model Examples Learning Learning Negative Positive
7. Probably Incorrect! Prediction Prediction Classifier Model Model Examples Learning Learning Negative Positive Machine Learning Machine Learning Image: Method Matters Blog
8. Machine Learning Nowadays Machine Learning Nowadays Probably Positive! Classifier Model Model Examples Negative Positive Prediction Prediction Learning Learning Deep neural network Deep neural network
9. AI as a Black Box AI as a Black Box Probably Positive! Prediction Prediction Classifier Model Model Labelled Examples Learning Learning Negative Positive
10. Why Explainable AI (XAI)? Why Explainable AI (XAI)? Why Explainable AI (XAI)? Why Explainable AI (XAI)? https://www.idealrole.com/blog/cv-bias
11. Why Explainable AI (XAI)? Why Explainable AI (XAI)? Why Explainable AI (XAI)? Why Explainable AI (XAI)?
12. Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion Vectors for Emojis Trees for Sentences Graphs for Knowledge
13. Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion Vectors for Emojis Trees for Sentences Graphs for Knowledge
14. Modern AI Modern AI Probably Positive! Classifier Model Model Examples Negative Positive Prediction Prediction Learning Learning Deep neural network Deep neural network
15. Modern AI Modern AI Probably Positive! Classifier Model Model Examples Negative Positive Prediction Prediction Learning Learning Deep neural network Deep neural network
16. 0.00 0.23 0.03 0.31 0.01 0.03 0.91 ... 0.31 0.50 Embeddings Embeddings 0.01 0.23 0.01 0.29 0.00 0.03 0.92 ... 0.30 0.51 “dry” “arid” ≈ x x x x amazing awesome unbearable horrible x terrible xstupendous Vector space in which similar words have similar vectors Vector space in which similar words have similar vectors
17. Question Question Question Question Image: https://www.synthesio.com/blog/social-sentiment-and-your-business/ Can we have vector representations that are interpretable and provide detailed information about the sentiment/emotion of a word? Can we have vector representations that are interpretable and provide detailed information about the sentiment/emotion of a word? 0.00 0.23 0.03 0.31 0.01 0.03 0.91 ... 0.31 0.50 0.01 0.23 0.01 0.29 0.00 0.03 0.92 ... 0.30 0.51 “stupendous” “amazing” ≈ ??
18. Multi-Dimensional Multi-Dimensional Sentiment Embeddings Sentiment Embeddings Multi-Dimensional Multi-Dimensional Sentiment Embeddings Sentiment Embeddings Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 Music Laptops Image: http://www.thewilliamnyc.com/william-gallery/ 0.97 -0.30 ... 0.90 0.94 ... “hot” “amazing”
19. Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Transfer Learning using Supervised Linear Models Train n linear models Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018
20. Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Transfer Learning using Supervised Linear Models Train n linear models Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 f Feature Value “horrible” 1 “great” 2 “inside” 1 “cat” 3 ... ...
21. Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Transfer Learning using Supervised Linear Models Train n linear models Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 f Feature Value Weight “horrible” 1 -5.2 “great” 2 4.4 “inside” 1 -0.1 “cat” 3 1.2 ... ... ...
22. Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 Train n linear models Used 25 different domains from Amazon Books, Electronics, Movies, Kitchen, etc. Used 25 different domains from Amazon Books, Electronics, Movies, Kitchen, etc.
23. Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 0.0 0.8 0.2 ... –0.8 0.1 hot Train n linear models For each word, turn its linear coefficients across different n models into a single vector ...
24. Emotion Analysis Emotion Analysis Emotion Analysis Emotion Analysis “What the hell is going on?! I've been waiting for 2 weeks now!!” “What the hell is going on?! I've been waiting for 2 weeks now!!”
25. Emotion Models Emotion Models Emotion Models Emotion Models Image: Paul Ekman Ekman's 6 Basic Emotion Ekman's 6 Basic Emotion
26. Emotion Models Emotion Models Emotion Models Emotion Models Image: https://commons.wikimedia.org/wiki/File:Plutchik-wheel.svg Plutchik's Wheel of Emotions Plutchik's Wheel of Emotions
27. Even More Emotions Even More Emotions Even More Emotions Even More Emotions Cowen & Keltner (2017) Are 8 basic emotions really sufficient? Are 8 basic emotions really sufficient?
28. AffectVec AffectVec AffectVec AffectVec https://www.flickr.com/photos/ill-padrino/6437837857/ joy sadness anger guilt suspense 0.5 0.1 0.4 ... 0.3 0.1 prank Shahab Raji, Gerard de Melo. What Sparks Joy: The AffectVec Emotion Database Matej Kren: Idiom. Prague Municipal Library Prank Image: CollegeHumor
29. AffectVec AffectVec AffectVec AffectVec Shahab Raji, Gerard de Melo. What Sparks Joy: The AffectVec Emotion Database
30. AffectVec AffectVec AffectVec AffectVec Shahab Raji, Gerard de Melo. What Sparks Joy: The AffectVec Emotion Database
31. AffectVec AffectVec AffectVec AffectVec Download: http://emotionlexicon.org/ Download: http://emotionlexicon.org/ Shahab Raji, Gerard de Melo. What Sparks Joy: The AffectVec Emotion Database Also: Experiments on Unsupervised Document-Level Emotion Prediction
32. Cross-Lingual Cross-Lingual Emotion Lexicon Induction Emotion Lexicon Induction Cross-Lingual Cross-Lingual Emotion Lexicon Induction Emotion Lexicon Induction Over 300 languages http://emotionlexicon.org/ Over 300 languages http://emotionlexicon.org/ Arun Ramachandran, Gerard de Melo. COLING 2020
33. Application: Application: Fonts Fonts Application: Application: Fonts Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts Van Rompay and Pruyn (2011) Choice of font affects assumed brand credibility price expectations Choice of font affects assumed brand credibility price expectations
34. Application: Application: Fonts Fonts Application: Application: Fonts Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts de Sousa et al. (2020). Journal of Sensory Studies Choice of font affects intent to purchase and assumed acidity, sweetness Choice of font affects intent to purchase and assumed acidity, sweetness
35. Application: Application: Fonts Fonts Application: Application: Fonts Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts Shaikh (2007) O’Brien Louch & Stork (2014) Choice of font affects assumed attributes of people using them Choice of font affects assumed attributes of people using them
36. Font Selection Font Selection Font Selection Font Selection Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts
37. Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts
38. Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts
39. Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts fontjoy.com
40. Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts http://fontjoy.com/projector/ Visual Representation Space Visual Representation Space
41. Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts fontjoy.com Infer Attribute-Based Representations Infer Attribute-Based Representations
42. Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts
43. FontLex FontLex FontLex FontLex Tugba Kulahcioglu, Gerard de Melo. FontLex: A Typographical Lexicon based on Affective Associations Infer font–emotion mapping using seed data + word2vec Infer font–emotion mapping using seed data + word2vec
44. FontLex FontLex FontLex FontLex via Sentiment/Emotions associations of words Tugba Kulahcioglu, Gerard de Melo. FontLex: A Typographical Lexicon based on Affective Associations
45. FontLex: FontLex: Complex Emotions Complex Emotions FontLex: FontLex: Complex Emotions Complex Emotions Tugba Kulahcioglu, Gerard de Melo. Semantics-Aware Typographical Choices via Affective Associations Extension by using more fonts for FontLex data
46. Emotions vs. Emotions vs. Font and Color Font and Color Emotions vs. Emotions vs. Font and Color Font and Color Tugba Kulahcioglu, Gerard de Melo. Paralinguistic Recommendations for Affective Word Clouds. Proc. IUI Fonts: best for serious, trustworthy, disturbing Color palettes: best for calm, negative, playful
47. Emotions vs. Emotions vs. Font and Color Font and Color Emotions vs. Emotions vs. Font and Color Font and Color Tugba Kulahcioglu, Gerard de Melo. Paralinguistic Recommendations for Affective Word Clouds. Proc. IUI United Nations
48. Emotions vs. Emotions vs. Font and Color Font and Color Emotions vs. Emotions vs. Font and Color Font and Color Tugba Kulahcioglu, Gerard de Melo. FontLex: A Typographical Lexicon based on Affective Associations / Tugba Kulahcioglu, Gerard de Melo. Paralinguistic Recommendations for Affective Word Clouds. Proc. IUI The Smurfs Scream
49. Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion Vectors for Emojis Trees for Sentences Graphs for Knowledge
50. Emojis are ubiquitous Emojis are ubiquitous Emojis are ubiquitous Emojis are ubiquitous Image: https://www.theverge.com/2016/2/24/11105250/facebook-reactions-emoji-how-to Very prominent on social media, instant messaging email subject lines etc. Very prominent on social media, instant messaging email subject lines etc. Face with Tears of Joy: Oxford Dictionaries Word of the Year 2015 Face with Tears of Joy: Oxford Dictionaries Word of the Year 2015
51. Emojis vs. Emotions Emojis vs. Emotions Emojis vs. Emotions Emojis vs. Emotions Abu Awal Md Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP Same parts of brain activated as when seeing a real face Same parts of brain activated as when seeing a real face “Emoji” is Japanese for “picture character” but clearly closely linked to emotions Image: from Petra Kralj Novak. Sentiment of Emojis
52. Emoji-Based Emoji-Based Word Embeddings Word Embeddings Emoji-Based Emoji-Based Word Embeddings Word Embeddings Image: https://www.theverge.com/2016/2/24/11105250/facebook-reactions-emoji-how-to 0.8 0.4 0.0 ... 0.0 0.1 prank Abu Awal Md Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP
53. Use Case: Deep Neural Network for Use Case: Deep Neural Network for Tweet Emotion Classification Tweet Emotion Classification Use Case: Deep Neural Network for Use Case: Deep Neural Network for Tweet Emotion Classification Tweet Emotion Classification Abu Awal Md Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP Methods Anger Fear Joy Sadness Average Dim Interpretable Affective Tweets 0.65 0.66 0.60 0.69 0.65 n/a EmoTag 0.70 0.73 0.69 0.75 0.72 620 Non-Interpretable Random Int. 0.68 0.72 0.66 0.73 0.70 300 word2vec 0.70 0.72 0.67 0.75 0.71 300 GloVe 0.70 0.73 0.68 0.76 0.72 300 GloVe Twitter 0.72 0.74 0.68 0.76 0.73 200
54. Use Case: Emoji Sentiment Use Case: Emoji Sentiment Use Case: Emoji Sentiment Use Case: Emoji Sentiment Abu Awal Md Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP
55. Dataset and Study: Dataset and Study: Emoji Emotions Emoji Emotions Dataset and Study: Dataset and Study: Emoji Emotions Emoji Emotions Abu Shoeb, Gerard de Melo. EmoTag1200 : Understanding the Association between Emojis and Emotions . EMNLP 👍 😄 😻 Abu Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP Online: emoji.nlproc.org Online: emoji.nlproc.org
56. Dataset and Study: Dataset and Study: Emoji Emotions Emoji Emotions Dataset and Study: Dataset and Study: Emoji Emotions Emoji Emotions Abu Shoeb, Gerard de Melo. EmoTag1200 : Understanding the Association between Emojis and Emotions . EMNLP 👍 😄 😻 Abu Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP
57. Emojis in NLP: Emojis in NLP: Sentiment Analysis Sentiment Analysis Emojis in NLP: Emojis in NLP: Sentiment Analysis Sentiment Analysis Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021
58. Emojis in NLP: Emojis in NLP: Tokenization Tokenization Emojis in NLP: Emojis in NLP: Tokenization Tokenization Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021
59. Complex Emojis Complex Emojis Complex Emojis Complex Emojis Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021
60. Emojis in NLP: Emojis in NLP: POS Tagging POS Tagging Emojis in NLP: Emojis in NLP: POS Tagging POS Tagging Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021
61. Emojis in NLP: Emojis in NLP: Dependency Parsing Dependency Parsing Emojis in NLP: Emojis in NLP: Dependency Parsing Dependency Parsing Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021
62. Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion Vectors for Emojis Trees for Sentences Graphs for Knowledge
63. What are they capturing? What are they capturing? What are they capturing? What are they capturing? Image: Adapted from http://www.cse.unt.edu/people/StudentNewsletters/2013_Apr_StudentEmailNewsletter.html you can't cram the meaning of a whole ***ing sentence into a single ***ing vector
64. Evaluation via Evaluation via Probing Probing Evaluation via Evaluation via Probing Probing Also: Adi et al. ICLR 2016 These test whether enough information is kept to learn something from 100,000 training examples. These test whether enough information is kept to learn something from 100,000 training examples.
65. Our Approach: Our Approach: Inspect Proximity Structure Inspect Proximity Structure Our Approach: Our Approach: Inspect Proximity Structure Inspect Proximity Structure A person is slicing an onion. A person is cutting an onion. A person is not slicing an onion. S0 S= S* sim(S0,S=) sim(S0,S=) Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018
66. Our Approach: Our Approach: Inspect Proximity Structure Inspect Proximity Structure Our Approach: Our Approach: Inspect Proximity Structure Inspect Proximity Structure A person is slicing an onion. A person is cutting an onion. A person is not slicing an onion. S0 S= S* sim(S0,S=) sim(S0,S=) sim(S0,S*) sim(S0,S*) sim(S0,S=) > sim(S0,S*) ?
67. Negation Detection Negation Detection Negation Detection Negation Detection A person is slicing an onion. A person is cutting an onion. A person is not slicing an onion. S0 S= S* Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018
68. Negation Variant Negation Variant Negation Variant Negation Variant A man is not standing on his head under water. There is no man standing on his head under water. A man is standing on his head under water. S0 S= S* (Negation) (Negated Existential) Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018
69. Clause Relatedness Clause Relatedness Clause Relatedness Clause Relatedness Octel said the purchase was expected. The purchase was expected. Octel said the purchase was not expected. S0 S= S* Clause Extraction (for suitable head verbs only) Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018
70. Argument Sensitivity Argument Sensitivity Argument Sensitivity Argument Sensitivity Francesca teaches Adam to adjust the microphone on his stage. Adam is taught to adjust the microphone on his stage. Adam teaches Francesca to adjust the microphone on his stage. S0 S= S* (Passive) (Argument Inversion) Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018
71. Argument Sensitivity Argument Sensitivity Argument Sensitivity Argument Sensitivity Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018 Who-did-what- to-whom not reflected in topology Who-did-what- to-whom not reflected in topology
72. R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling
73. R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling black cat black cat the black the black cat the Which composition makes more sense? Which composition makes more sense?
74. R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away black cat black cat the black the black cat the
75. R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away black cat black cat the black the black cat the
76. R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away
77. R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away
78. R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away
79. R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away
80. R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling Local Probability Compositional Embedding For each split point k of T1,3
81. R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling Local Probability Compositional Embedding For each split point k of T1,3 Weighted sum over all split points
82. R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling Unsupervised Language Model Pre-Training Unsupervised Language Model Pre-Training
83. R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling
84. Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion Vectors for Emojis Trees for Sentences Graphs for Knowledge
85. Universal Wordnet Universal Wordnet Universal Wordnet Universal Wordnet cmn: “ ” 机构 http://lexvo.org/uwn/ Gerard de Melo. Towards a Universal Wordnet by Learning from Combined Evidence.
86. Explainable Inference in Graphs Explainable Inference in Graphs Fu et al. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In: Proceedings of SIGIR 2020. http://gerard.demelo.org/publications.html
87. Explainable Inference in Graphs Explainable Inference in Graphs Fu et al. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In: Proceedings of SIGIR 2020. http://gerard.demelo.org/publications.html
88. Explainable Inference in Graphs Explainable Inference in Graphs Fu et al. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In: Proceedings of SIGIR 2020. http://gerard.demelo.org/publications.html In reality, there are many possible paths. Which to choose? In reality, there are many possible paths. Which to choose?
89. Explainable Inference in Graphs Explainable Inference in Graphs Fu et al. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In: Proceedings of SIGIR 2020. http://gerard.demelo.org/publications.html Reward if you guessed correctly! Reward if you guessed correctly!
90. Explainable Inference in Graphs Explainable Inference in Graphs Xian et al. Reinforcement Knowledge Graph Reasoning for Explainable Recommendation. SIGIR 2019
91. x x 감사합니다 gracias xthank you Summary Summary Summary Summary Vectors for Sentiment and Emotion ►Interpretable Vectors ►Font Affect Embeddings Emojis ►Interpretable Vectors ►NLP Analysis Trees for Sentences ►Analysis ►R2D2 Differentiable Tree Model Graphs for Knowledge ►Reinforcement Learning over Knowledge Graphs for Explainable AI Get in Touch! http://gerard.demelo.org gdm@demelo.org Get in Touch! http://gerard.demelo.org gdm@demelo.org x 谢谢 x Vielen Dank спаси́бо x x obrigado