Slide 1

Slide 1 text

BIDIRECTIONAL ENCODER REPRESENTIONS FROM TRANSFORMERS For Sentiment Analysis *Images generated by stable-diffusion Prompt: Bert from sesame street dressed as a synthesizer FINE TUNING

Slide 2

Slide 2 text

Why are you here? Me? Prompt: Sesame street character sitting at a desk with a laptop

Slide 3

Slide 3 text

I want to specialize in NLP! Chris Mccormick Prompt: A polyglot sesame street character sitting at a desk with a laptop Jay Alammar Andrew Ng

Slide 4

Slide 4 text

Queen – woman + 7*man = ?

Slide 5

Slide 5 text

Queen – woman + 7*man = snow white

Slide 6

Slide 6 text

1. https://www.who.int/news/item/17-06-2021-one-in-100-deaths-is-by-suicide 2. https://www.ijcai.org/proceedings/2022/0704.pdf “a code-mixed text has a dominant language or matrix language (here, Hindi) and an inserted language or embedded language (here, English)”

Slide 7

Slide 7 text

Sentiment analysis is on our roadmap for Q3. Prompt: A sesame street character dressed as a manager sitting at a desk with a laptop https://arxiv.org/pdf/1908.10063.pdf https://github.com/ProsusAI/finBERT https://medium.com/prosus-ai-tech-blog/finbert- financial-sentiment-analysis-with-bert-b277a3607101

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

I want to apply what I learn today to improve society Prompt: A superhero sesame street character sitting at a desk with a laptop

Slide 11

Slide 11 text

Detecting Potentially Harmful and Protective Suicide-related Content on Twitter: A Machine Learning Approach Metzler, H., Baginski, H., Niederkrotenthaler, T., & Garcia, D. (2022). Journal of medical Internet research, 24(8), e34705. https://doi.org/10.2196/34705 1. https://www.who.int/news/item/17-06-2021-one-in-100-deaths-is-by-suicide 2. https://arxiv.org/pdf/2112.04796.pdf

Slide 12

Slide 12 text

[2]

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

I’m a software engineer and I want to create ML apps with vision and generative models too. Prompt: A sesame street character who is a software engineer sitting at a desk 👩🎓 Train Sentence Transformers with little labeled data using a technique called SetFit 🐭 Compress models with knowledge distillation 🏎 Accelerate inference with quantization and 🤗 Optimum and Intel® Neural Compressor 🐳 Deploy models with Inference Endpoints For more details about SetFit, check out the library 👉: https://github.com/huggingface/setfit

Slide 15

Slide 15 text

Transfer Learning Prompt: sesame street character amazed by understanding a new concept Ultimately, we all follow tutorials hoping to transfer what we learn to use-cases we care about

Slide 16

Slide 16 text

From Wikipedia Transfer learning (TL) is a research problem in machine learning (ML) that focuses on applying knowledge gained while solving one task to a related task.[1] For example, knowledge gained while learning to recognize cars could be applied when trying to recognize trucks. This topic is related to the psychological literature on transfer of learning, although practical ties between the two fields are limited. Reusing/transferring information from previously learned tasks to new tasks has the potential to significantly improve learning efficiency.[2]

Slide 17

Slide 17 text

Machines From Wikipedia Transfer learning (TL) is a research problem in machine learning (ML) that focuses on applying knowledge gained while solving one task to a related task.[1] From Wikipedia Transfer of learning occurs when people apply information, strategies, and skills they have learned to a new situation or context. Humans

Slide 18

Slide 18 text

BERT YOU Embeddings Intuition Tokenizer Gut Masked Language Modeling and Next Sentence Prediction Finetuning Objective: How to classify the sentiment of a piece of text Fine Tuning Objective: How to finetune a BERT based model

Slide 19

Slide 19 text

Manager Polyglot Superhero Software Engineer https://huggingface.co/models Human base models

Slide 20

Slide 20 text

["Jack", "Jack", "Jack", "Jack", "There's a boat, Jack"] Interpreting Sentiment Positive or Negative?

Slide 21

Slide 21 text

Interpreting Sentiment (Advanced!) Plutchik, 1982

Slide 22

Slide 22 text

Interpreting Sentiment (Advanced!)

Slide 23

Slide 23 text

Interpreting Sentiment (Advanced!) ["Jack", "Jack", "Jack", "Jack", "There's a boat”, “Jack”, “Jack”, “JACK!”, “Jack”, “There’s a boat, Jack”, “Jack”] "Jack… Jack…Jack…Jack…There's a boat…Jack…Jack…JACK! Jack? There’s a boat, Jack. Jack” Relief Despair Ecstatic Confusion Guilt Fear Anger Time

Slide 24

Slide 24 text

Interpreting Sentiment (Advanced!) ["Jack", "Jack", "Jack", "Jack", "There's a boat”, “Jack”, “Jack”, “JACK!”, “Jack”, “There’s a boat, Jack”, “Jack”] Relief Ecstatic Confusion Despair Anger Guilt Fear Time [ .95, .00, .00, .00, .00, .00, .00] [ .95, .95, .00, .00, .00, .00, .00] [ .80, .50, .50, .00, .00, .00, .00] [ .00, .00, .20, .99, .00, .00, .00] [ .00, .00, .00, .95, .30, .00, .00] [ .00, .00, .00, .95, .02, .99, .00] [ .00, .00, .00, .00, .00, .00, .99]

Slide 25

Slide 25 text

Emotion Embeddings https://arxiv.org/pdf/2211.00171.pdf https://github.com/gchochla/Demux-MEmo