Slide 1

Slide 1 text

Interpretable Representations for Affect and Semantics Gerard de Melo http://gerard.demelo.org Interpretable Representations for Affect and Semantics Gerard de Melo http://gerard.demelo.org

Slide 2

Slide 2 text

Machine Learning Machine Learning Examples Negative Positive

Slide 3

Slide 3 text

Machine Learning Machine Learning Classifier Model Model Examples Learning Learning Negative Positive

Slide 4

Slide 4 text

Machine Learning Machine Learning Classifier Model Model Examples Learning Learning ??? Negative Positive

Slide 5

Slide 5 text

Machine Learning Machine Learning Probably Positive! Prediction Prediction Classifier Model Model Examples Learning Learning Negative Positive

Slide 6

Slide 6 text

Machine Learning Machine Learning Probably Positive! Prediction Prediction Classifier Model Model Examples Learning Learning Negative Positive

Slide 7

Slide 7 text

Probably Incorrect! Prediction Prediction Classifier Model Model Examples Learning Learning Negative Positive Machine Learning Machine Learning Image: Method Matters Blog

Slide 8

Slide 8 text

Machine Learning Nowadays Machine Learning Nowadays Probably Positive! Classifier Model Model Examples Negative Positive Prediction Prediction Learning Learning Deep neural network Deep neural network

Slide 9

Slide 9 text

AI as a Black Box AI as a Black Box Probably Positive! Prediction Prediction Classifier Model Model Labelled Examples Learning Learning Negative Positive

Slide 10

Slide 10 text

Why Explainable AI (XAI)? Why Explainable AI (XAI)? Why Explainable AI (XAI)? Why Explainable AI (XAI)? https://www.idealrole.com/blog/cv-bias

Slide 11

Slide 11 text

Why Explainable AI (XAI)? Why Explainable AI (XAI)? Why Explainable AI (XAI)? Why Explainable AI (XAI)?

Slide 12

Slide 12 text

Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion Vectors for Emojis Trees for Sentences Graphs for Knowledge

Slide 13

Slide 13 text

Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion Vectors for Emojis Trees for Sentences Graphs for Knowledge

Slide 14

Slide 14 text

Modern AI Modern AI Probably Positive! Classifier Model Model Examples Negative Positive Prediction Prediction Learning Learning Deep neural network Deep neural network

Slide 15

Slide 15 text

Modern AI Modern AI Probably Positive! Classifier Model Model Examples Negative Positive Prediction Prediction Learning Learning Deep neural network Deep neural network

Slide 16

Slide 16 text

0.00 0.23 0.03 0.31 0.01 0.03 0.91 ... 0.31 0.50 Embeddings Embeddings 0.01 0.23 0.01 0.29 0.00 0.03 0.92 ... 0.30 0.51 “dry” “arid” ≈ x x x x amazing awesome unbearable horrible x terrible xstupendous Vector space in which similar words have similar vectors Vector space in which similar words have similar vectors

Slide 17

Slide 17 text

Question Question Question Question Image: https://www.synthesio.com/blog/social-sentiment-and-your-business/ Can we have vector representations that are interpretable and provide detailed information about the sentiment/emotion of a word? Can we have vector representations that are interpretable and provide detailed information about the sentiment/emotion of a word? 0.00 0.23 0.03 0.31 0.01 0.03 0.91 ... 0.31 0.50 0.01 0.23 0.01 0.29 0.00 0.03 0.92 ... 0.30 0.51 “stupendous” “amazing” ≈ ??

Slide 18

Slide 18 text

Multi-Dimensional Multi-Dimensional Sentiment Embeddings Sentiment Embeddings Multi-Dimensional Multi-Dimensional Sentiment Embeddings Sentiment Embeddings Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 Music Laptops Image: http://www.thewilliamnyc.com/william-gallery/ 0.97 -0.30 ... 0.90 0.94 ... “hot” “amazing”

Slide 19

Slide 19 text

Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Transfer Learning using Supervised Linear Models Train n linear models Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018

Slide 20

Slide 20 text

Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Transfer Learning using Supervised Linear Models Train n linear models Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 f Feature Value “horrible” 1 “great” 2 “inside” 1 “cat” 3 ... ...

Slide 21

Slide 21 text

Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Transfer Learning using Supervised Linear Models Train n linear models Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 f Feature Value Weight “horrible” 1 -5.2 “great” 2 4.4 “inside” 1 -0.1 “cat” 3 1.2 ... ... ...

Slide 22

Slide 22 text

Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 Train n linear models Used 25 different domains from Amazon Books, Electronics, Movies, Kitchen, etc. Used 25 different domains from Amazon Books, Electronics, Movies, Kitchen, etc.

Slide 23

Slide 23 text

Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Method: Data-Driven Method: Data-Driven Sentiment Embeddings Sentiment Embeddings Xin Dong, Gerard de Melo. Cross-Lingual Propagation for Deep Sentiment Analysis. AAAI 2018 0.0 0.8 0.2 ... –0.8 0.1 hot Train n linear models For each word, turn its linear coefficients across different n models into a single vector ...

Slide 24

Slide 24 text

Emotion Analysis Emotion Analysis Emotion Analysis Emotion Analysis “What the hell is going on?! I've been waiting for 2 weeks now!!” “What the hell is going on?! I've been waiting for 2 weeks now!!”

Slide 25

Slide 25 text

Emotion Models Emotion Models Emotion Models Emotion Models Image: Paul Ekman Ekman's 6 Basic Emotion Ekman's 6 Basic Emotion

Slide 26

Slide 26 text

Emotion Models Emotion Models Emotion Models Emotion Models Image: https://commons.wikimedia.org/wiki/File:Plutchik-wheel.svg Plutchik's Wheel of Emotions Plutchik's Wheel of Emotions

Slide 27

Slide 27 text

Even More Emotions Even More Emotions Even More Emotions Even More Emotions Cowen & Keltner (2017) Are 8 basic emotions really sufficient? Are 8 basic emotions really sufficient?

Slide 28

Slide 28 text

AffectVec AffectVec AffectVec AffectVec https://www.flickr.com/photos/ill-padrino/6437837857/ joy sadness anger guilt suspense 0.5 0.1 0.4 ... 0.3 0.1 prank Shahab Raji, Gerard de Melo. What Sparks Joy: The AffectVec Emotion Database Matej Kren: Idiom. Prague Municipal Library Prank Image: CollegeHumor

Slide 29

Slide 29 text

AffectVec AffectVec AffectVec AffectVec Shahab Raji, Gerard de Melo. What Sparks Joy: The AffectVec Emotion Database

Slide 30

Slide 30 text

AffectVec AffectVec AffectVec AffectVec Shahab Raji, Gerard de Melo. What Sparks Joy: The AffectVec Emotion Database

Slide 31

Slide 31 text

AffectVec AffectVec AffectVec AffectVec Download: http://emotionlexicon.org/ Download: http://emotionlexicon.org/ Shahab Raji, Gerard de Melo. What Sparks Joy: The AffectVec Emotion Database Also: Experiments on Unsupervised Document-Level Emotion Prediction

Slide 32

Slide 32 text

Cross-Lingual Cross-Lingual Emotion Lexicon Induction Emotion Lexicon Induction Cross-Lingual Cross-Lingual Emotion Lexicon Induction Emotion Lexicon Induction Over 300 languages http://emotionlexicon.org/ Over 300 languages http://emotionlexicon.org/ Arun Ramachandran, Gerard de Melo. COLING 2020

Slide 33

Slide 33 text

Application: Application: Fonts Fonts Application: Application: Fonts Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts Van Rompay and Pruyn (2011) Choice of font affects assumed brand credibility price expectations Choice of font affects assumed brand credibility price expectations

Slide 34

Slide 34 text

Application: Application: Fonts Fonts Application: Application: Fonts Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts de Sousa et al. (2020). Journal of Sensory Studies Choice of font affects intent to purchase and assumed acidity, sweetness Choice of font affects intent to purchase and assumed acidity, sweetness

Slide 35

Slide 35 text

Application: Application: Fonts Fonts Application: Application: Fonts Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts Shaikh (2007) O’Brien Louch & Stork (2014) Choice of font affects assumed attributes of people using them Choice of font affects assumed attributes of people using them

Slide 36

Slide 36 text

Font Selection Font Selection Font Selection Font Selection Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts

Slide 37

Slide 37 text

Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts

Slide 38

Slide 38 text

Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts

Slide 39

Slide 39 text

Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts fontjoy.com

Slide 40

Slide 40 text

Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts http://fontjoy.com/projector/ Visual Representation Space Visual Representation Space

Slide 41

Slide 41 text

Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts fontjoy.com Infer Attribute-Based Representations Infer Attribute-Based Representations

Slide 42

Slide 42 text

Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Learning Representations of Fonts Tugba Kulahcioglu, Gerard de Melo. Predicting Semantic Signatures of Fonts

Slide 43

Slide 43 text

FontLex FontLex FontLex FontLex Tugba Kulahcioglu, Gerard de Melo. FontLex: A Typographical Lexicon based on Affective Associations Infer font–emotion mapping using seed data + word2vec Infer font–emotion mapping using seed data + word2vec

Slide 44

Slide 44 text

FontLex FontLex FontLex FontLex via Sentiment/Emotions associations of words Tugba Kulahcioglu, Gerard de Melo. FontLex: A Typographical Lexicon based on Affective Associations

Slide 45

Slide 45 text

FontLex: FontLex: Complex Emotions Complex Emotions FontLex: FontLex: Complex Emotions Complex Emotions Tugba Kulahcioglu, Gerard de Melo. Semantics-Aware Typographical Choices via Affective Associations Extension by using more fonts for FontLex data

Slide 46

Slide 46 text

Emotions vs. Emotions vs. Font and Color Font and Color Emotions vs. Emotions vs. Font and Color Font and Color Tugba Kulahcioglu, Gerard de Melo. Paralinguistic Recommendations for Affective Word Clouds. Proc. IUI Fonts: best for serious, trustworthy, disturbing Color palettes: best for calm, negative, playful

Slide 47

Slide 47 text

Emotions vs. Emotions vs. Font and Color Font and Color Emotions vs. Emotions vs. Font and Color Font and Color Tugba Kulahcioglu, Gerard de Melo. Paralinguistic Recommendations for Affective Word Clouds. Proc. IUI United Nations

Slide 48

Slide 48 text

Emotions vs. Emotions vs. Font and Color Font and Color Emotions vs. Emotions vs. Font and Color Font and Color Tugba Kulahcioglu, Gerard de Melo. FontLex: A Typographical Lexicon based on Affective Associations / Tugba Kulahcioglu, Gerard de Melo. Paralinguistic Recommendations for Affective Word Clouds. Proc. IUI The Smurfs Scream

Slide 49

Slide 49 text

Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion Vectors for Emojis Trees for Sentences Graphs for Knowledge

Slide 50

Slide 50 text

Emojis are ubiquitous Emojis are ubiquitous Emojis are ubiquitous Emojis are ubiquitous Image: https://www.theverge.com/2016/2/24/11105250/facebook-reactions-emoji-how-to Very prominent on social media, instant messaging email subject lines etc. Very prominent on social media, instant messaging email subject lines etc. Face with Tears of Joy: Oxford Dictionaries Word of the Year 2015 Face with Tears of Joy: Oxford Dictionaries Word of the Year 2015

Slide 51

Slide 51 text

Emojis vs. Emotions Emojis vs. Emotions Emojis vs. Emotions Emojis vs. Emotions Abu Awal Md Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP Same parts of brain activated as when seeing a real face Same parts of brain activated as when seeing a real face “Emoji” is Japanese for “picture character” but clearly closely linked to emotions Image: from Petra Kralj Novak. Sentiment of Emojis

Slide 52

Slide 52 text

Emoji-Based Emoji-Based Word Embeddings Word Embeddings Emoji-Based Emoji-Based Word Embeddings Word Embeddings Image: https://www.theverge.com/2016/2/24/11105250/facebook-reactions-emoji-how-to 0.8 0.4 0.0 ... 0.0 0.1 prank Abu Awal Md Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP

Slide 53

Slide 53 text

Use Case: Deep Neural Network for Use Case: Deep Neural Network for Tweet Emotion Classification Tweet Emotion Classification Use Case: Deep Neural Network for Use Case: Deep Neural Network for Tweet Emotion Classification Tweet Emotion Classification Abu Awal Md Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP Methods Anger Fear Joy Sadness Average Dim Interpretable Affective Tweets 0.65 0.66 0.60 0.69 0.65 n/a EmoTag 0.70 0.73 0.69 0.75 0.72 620 Non-Interpretable Random Int. 0.68 0.72 0.66 0.73 0.70 300 word2vec 0.70 0.72 0.67 0.75 0.71 300 GloVe 0.70 0.73 0.68 0.76 0.72 300 GloVe Twitter 0.72 0.74 0.68 0.76 0.73 200

Slide 54

Slide 54 text

Use Case: Emoji Sentiment Use Case: Emoji Sentiment Use Case: Emoji Sentiment Use Case: Emoji Sentiment Abu Awal Md Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP

Slide 55

Slide 55 text

Dataset and Study: Dataset and Study: Emoji Emotions Emoji Emotions Dataset and Study: Dataset and Study: Emoji Emotions Emoji Emotions Abu Shoeb, Gerard de Melo. EmoTag1200 : Understanding the Association between Emojis and Emotions . EMNLP 👍 😄 😻 Abu Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP Online: emoji.nlproc.org Online: emoji.nlproc.org

Slide 56

Slide 56 text

Dataset and Study: Dataset and Study: Emoji Emotions Emoji Emotions Dataset and Study: Dataset and Study: Emoji Emotions Emoji Emotions Abu Shoeb, Gerard de Melo. EmoTag1200 : Understanding the Association between Emojis and Emotions . EMNLP 👍 😄 😻 Abu Shoeb, Shahab Raji, Gerard de Melo. EmoTag – Towards an Emotion-Based Analysis of Emojis. Proc. RANLP

Slide 57

Slide 57 text

Emojis in NLP: Emojis in NLP: Sentiment Analysis Sentiment Analysis Emojis in NLP: Emojis in NLP: Sentiment Analysis Sentiment Analysis Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021

Slide 58

Slide 58 text

Emojis in NLP: Emojis in NLP: Tokenization Tokenization Emojis in NLP: Emojis in NLP: Tokenization Tokenization Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021

Slide 59

Slide 59 text

Complex Emojis Complex Emojis Complex Emojis Complex Emojis Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021

Slide 60

Slide 60 text

Emojis in NLP: Emojis in NLP: POS Tagging POS Tagging Emojis in NLP: Emojis in NLP: POS Tagging POS Tagging Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021

Slide 61

Slide 61 text

Emojis in NLP: Emojis in NLP: Dependency Parsing Dependency Parsing Emojis in NLP: Emojis in NLP: Dependency Parsing Dependency Parsing Abu Shoeb, Gerard de Melo. Assessing Emoji Use in Modern Text Processing Tools . ACL-IJCNLP 2021

Slide 62

Slide 62 text

Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion Vectors for Emojis Trees for Sentences Graphs for Knowledge

Slide 63

Slide 63 text

What are they capturing? What are they capturing? What are they capturing? What are they capturing? Image: Adapted from http://www.cse.unt.edu/people/StudentNewsletters/2013_Apr_StudentEmailNewsletter.html you can't cram the meaning of a whole ***ing sentence into a single ***ing vector

Slide 64

Slide 64 text

Evaluation via Evaluation via Probing Probing Evaluation via Evaluation via Probing Probing Also: Adi et al. ICLR 2016 These test whether enough information is kept to learn something from 100,000 training examples. These test whether enough information is kept to learn something from 100,000 training examples.

Slide 65

Slide 65 text

Our Approach: Our Approach: Inspect Proximity Structure Inspect Proximity Structure Our Approach: Our Approach: Inspect Proximity Structure Inspect Proximity Structure A person is slicing an onion. A person is cutting an onion. A person is not slicing an onion. S0 S= S* sim(S0,S=) sim(S0,S=) Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018

Slide 66

Slide 66 text

Our Approach: Our Approach: Inspect Proximity Structure Inspect Proximity Structure Our Approach: Our Approach: Inspect Proximity Structure Inspect Proximity Structure A person is slicing an onion. A person is cutting an onion. A person is not slicing an onion. S0 S= S* sim(S0,S=) sim(S0,S=) sim(S0,S*) sim(S0,S*) sim(S0,S=) > sim(S0,S*) ?

Slide 67

Slide 67 text

Negation Detection Negation Detection Negation Detection Negation Detection A person is slicing an onion. A person is cutting an onion. A person is not slicing an onion. S0 S= S* Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018

Slide 68

Slide 68 text

Negation Variant Negation Variant Negation Variant Negation Variant A man is not standing on his head under water. There is no man standing on his head under water. A man is standing on his head under water. S0 S= S* (Negation) (Negated Existential) Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018

Slide 69

Slide 69 text

Clause Relatedness Clause Relatedness Clause Relatedness Clause Relatedness Octel said the purchase was expected. The purchase was expected. Octel said the purchase was not expected. S0 S= S* Clause Extraction (for suitable head verbs only) Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018

Slide 70

Slide 70 text

Argument Sensitivity Argument Sensitivity Argument Sensitivity Argument Sensitivity Francesca teaches Adam to adjust the microphone on his stage. Adam is taught to adjust the microphone on his stage. Adam teaches Francesca to adjust the microphone on his stage. S0 S= S* (Passive) (Argument Inversion) Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018

Slide 71

Slide 71 text

Argument Sensitivity Argument Sensitivity Argument Sensitivity Argument Sensitivity Zhu, Li, de Melo. Exploring Semantic Properties of Sentence Embeddings. Proc. ACL 2018 Who-did-what- to-whom not reflected in topology Who-did-what- to-whom not reflected in topology

Slide 72

Slide 72 text

R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling

Slide 73

Slide 73 text

R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling black cat black cat the black the black cat the Which composition makes more sense? Which composition makes more sense?

Slide 74

Slide 74 text

R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away black cat black cat the black the black cat the

Slide 75

Slide 75 text

R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away black cat black cat the black the black cat the

Slide 76

Slide 76 text

R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away

Slide 77

Slide 77 text

R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away

Slide 78

Slide 78 text

R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away

Slide 79

Slide 79 text

R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling the black cat black cat jumped away the black cat jumped the black the black cat jumped away black cat jumped cat jumped away black cat cat jumped jumped away the black cat jumped away

Slide 80

Slide 80 text

R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling Local Probability Compositional Embedding For each split point k of T1,3

Slide 81

Slide 81 text

R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling Local Probability Compositional Embedding For each split point k of T1,3 Weighted sum over all split points

Slide 82

Slide 82 text

R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling Unsupervised Language Model Pre-Training Unsupervised Language Model Pre-Training

Slide 83

Slide 83 text

R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model R2D2: R2D2: Differentiable Tree Language Model Differentiable Tree Language Model Hu et al. R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling

Slide 84

Slide 84 text

Public Domain Image from https://pixabay.com/en/book-text-read-paper-education-451067/ Outline Outline Vectors for Sentiment/Emotion Vectors for Emojis Trees for Sentences Graphs for Knowledge

Slide 85

Slide 85 text

Universal Wordnet Universal Wordnet Universal Wordnet Universal Wordnet cmn: “ ” 机构 http://lexvo.org/uwn/ Gerard de Melo. Towards a Universal Wordnet by Learning from Combined Evidence.

Slide 86

Slide 86 text

Explainable Inference in Graphs Explainable Inference in Graphs Fu et al. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In: Proceedings of SIGIR 2020. http://gerard.demelo.org/publications.html

Slide 87

Slide 87 text

Explainable Inference in Graphs Explainable Inference in Graphs Fu et al. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In: Proceedings of SIGIR 2020. http://gerard.demelo.org/publications.html

Slide 88

Slide 88 text

Explainable Inference in Graphs Explainable Inference in Graphs Fu et al. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In: Proceedings of SIGIR 2020. http://gerard.demelo.org/publications.html In reality, there are many possible paths. Which to choose? In reality, there are many possible paths. Which to choose?

Slide 89

Slide 89 text

Explainable Inference in Graphs Explainable Inference in Graphs Fu et al. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In: Proceedings of SIGIR 2020. http://gerard.demelo.org/publications.html Reward if you guessed correctly! Reward if you guessed correctly!

Slide 90

Slide 90 text

Explainable Inference in Graphs Explainable Inference in Graphs Xian et al. Reinforcement Knowledge Graph Reasoning for Explainable Recommendation. SIGIR 2019

Slide 91

Slide 91 text

x x 감사합니다 gracias xthank you Summary Summary Summary Summary Vectors for Sentiment and Emotion ►Interpretable Vectors ►Font Affect Embeddings Emojis ►Interpretable Vectors ►NLP Analysis Trees for Sentences ►Analysis ►R2D2 Differentiable Tree Model Graphs for Knowledge ►Reinforcement Learning over Knowledge Graphs for Explainable AI Get in Touch! http://gerard.demelo.org [email protected] Get in Touch! http://gerard.demelo.org [email protected] x 谢谢 x Vielen Dank спаси́бо x x obrigado