Applied NLP in the Age of Generative AI

Slide 1

Slide 1 text

Ines Montani Explosion LLM

Slide 2

Slide 2 text

270m+ 270m+ spaC y Open-source library for industrial- strength natural language processing spacy.io downloads

Slide 3

Slide 3 text

270m+ 270m+ spaC y ChatGPT can write spaCy code! Open-source library for industrial- strength natural language processing spacy.io downloads

Slide 4

Slide 4 text

900+ 10k+ Prodi g y Modern scriptable annotation tool for machine learning developers prodigy.ai 900+ companies 10k+ users

Slide 5

Slide 5 text

900+ 10k+ Prodi g y Modern scriptable annotation tool for machine learning developers prodigy.ai Alex Smith Developer Kim Miller Analyst GPT-4 API 900+ companies 10k+ users

Slide 6

Slide 6 text

B ack to our r oots! explosion.ai/blog/back-to-our-roots We’re back to running Explosion as a smaller, independent-minded and self-su ff icient company.

Slide 7

Slide 7 text

B ack to our r oots! explosion.ai/blog/back-to-our-roots We’re back to running Explosion as a smaller, independent-minded and self-su ff icient company. Consulting open source developer tools

Slide 8

Slide 8 text

Falcon MIXTRAL GPT-4 LLM

Slide 9

Slide 9 text

Falcon MIXTRAL GPT-4 good contextual results LLM

Slide 10

Slide 10 text

Falcon MIXTRAL GPT-4 good contextual results easy to use & configure LLM

Slide 11

Slide 11 text

Falcon MIXTRAL GPT-4 good contextual results easy to use & configure fast prototyping LLM

Slide 12

Slide 12 text

Falcon MIXTRAL GPT-4 good contextual results ⚠ transparency easy to use & configure fast prototyping LLM

Slide 13

Slide 13 text

Falcon MIXTRAL GPT-4 good contextual results ⚠ transparency ⚠ e iciency easy to use & configure fast prototyping LLM

Slide 14

Slide 14 text

Falcon MIXTRAL GPT-4 good contextual results ⚠ data privacy ⚠ transparency ⚠ e iciency easy to use & configure fast prototyping LLM

Slide 15

Slide 15 text

de fi nition s E volution

Slide 16

Slide 16 text

de fi nition s E volution rules or instructions ✍ programming & rules

Slide 17

Slide 17 text

de fi nition s E volution rules or instructions ✍ programming & rules machine learning examples 📝 supervised learning

Slide 18

Slide 18 text

de fi nition s E volution rules or instructions ✍ programming & rules machine learning examples 📝 supervised learning in-context learning rules or instructions ✍ LLM prompt engineering

Slide 19

Slide 19 text

de fi nition s E volution rules or instructions ✍ programming & rules machine learning examples 📝 supervised learning in-context learning rules or instructions ✍ LLM prompt engineering instructions: human-shaped, easy for non-experts, risk of data drift ✍

Slide 20

Slide 20 text

Slide 21

Slide 21 text

de fi nition s E volution rules or instructions ✍ programming & rules machine learning examples 📝 supervised learning in-context learning rules or instructions ✍ LLM prompt engineering ? ? LLM instructions: human-shaped, easy for non-experts, risk of data drift ✍ 📝 examples: nuanced and intuitive behaviors, specific to use case, labor-intensive

Slide 22

Slide 22 text

P rototype task-specific output 💬 prompt 📖 text LLM GPT-4 API

Slide 23

Slide 23 text

P rototype task-specific output 💬 prompt 📖 text LLM prompt model & transform output to structured data github.com/explosion/spacy-llm GPT-4 API

Slide 24

Slide 24 text

📖 text task-specific output P roduction P rototype task-specific output 💬 prompt 📖 text LLM prompt model & transform output to structured data github.com/explosion/spacy-llm GPT-4 API

Slide 25

Slide 25 text

📖 text task-specific output P roduction P rototype task-specific output 💬 prompt 📖 text LLM distilled task-specific components prompt model & transform output to structured data github.com/explosion/spacy-llm GPT-4 API

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation LLM

Slide 30

Slide 30 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM

Slide 31

Slide 31 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting

Slide 32

Slide 32 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting

Slide 33

Slide 33 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting transfer learning CO M PO N EN T

Slide 34

Slide 34 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting transfer learning CO M PO N EN T distilled model

Slide 35

Slide 35 text

99% 99% Case Stud y : S&P Global • real-time commodities trading insights by extracting structured attributes 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 36

Slide 36 text

99% 99% Case Stud y : S&P Global • real-time commodities trading insights by extracting structured attributes • high-security environment 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 37

Slide 37 text

99% 99% Case Stud y : S&P Global • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 38

Slide 38 text

99% 99% Case Stud y : S&P Global • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 39

Slide 39 text

99% 99% Case Stud y : S&P Global • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop • 8 market pipelines in production 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 40

Slide 40 text

99% 99% Case Stud y : S&P Global • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop • 8 market pipelines in production 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 41

Slide 41 text

Refactor your code and data.

Slide 42

Slide 42 text

Software 1.0 Software 1.0 📄 code 💾 program compiler

Slide 43

Slide 43 text

Software 1.0 Software 1.0 📄 code 💾 program compiler Software 2.0 Software 2.0 📊 data 🔮 model algorithm

Slide 44

Slide 44 text

Software 1.0 Software 1.0 📄 code 💾 program compiler Software 2.0 Software 2.0 📊 data 🔮 model algorithm ✅ tests 📈 evaluation

Slide 45

Slide 45 text

Software 1.0 Software 1.0 📄 code 💾 program compiler Software 2.0 Software 2.0 📊 data 🔮 model algorithm ✅ tests 📈 evaluation refactoring refactoring

Slide 46

Slide 46 text

Software 1.0 Software 1.0 📄 code 💾 program compiler Software 2.0 Software 2.0 📊 data 🔮 model algorithm ✅ tests 📈 evaluation refactoring refactoring iteration iteration

Slide 47

Slide 47 text

I lo v e cats. SIMILAR OR NOT? I ha t e cats.

Slide 48

Slide 48 text

I lo v e cats. SIMILAR OR NOT? I ha t e cats. Your application context always matters!

Slide 49

Slide 49 text

Serve with a cold beer and a small bowl of Cheetos on the side. spacy.fyi/pydata-nyc Mix the Cheetos with the breadcrumbs and crush them with a rolling pin. INGREDIENT DISH EQUIPMENT WHICH LABEL?

Slide 50

Slide 50 text

Slide 51

Slide 51 text

Slide 52

Slide 52 text

Serve with a cold beer and a small bowl of Cheetos on the side. spacy.fyi/pydata-nyc Mix the Cheetos with the breadcrumbs and crush them with a rolling pin. INGREDIENT DISH EQUIPMENT WHICH LABEL? Serve with a cold beer and a small bowl of Cheetos on the side. Mix the Cheetos with the breadcrumbs and crush them with a rolling pin. EQUIPMENT We beat few-shot GPT baseline with 20× speedup!

Slide 53

Slide 53 text

Slide 54

Slide 54 text

F actor out busi n ess logic MODEL

Slide 55

Slide 55 text

F actor out busi n ess logic result = business_logic(classification(text)) MODEL

Slide 56

Slide 56 text

F actor out busi n ess logic result = business_logic(classification(text)) MODEL words, grammar, syntax information in the text

Slide 57

Slide 57 text

F actor out busi n ess logic result = business_logic(classification(text)) MODEL external knowledge facts that can change over time words, grammar, syntax information in the text

Slide 58

Slide 58 text

F actor out busi n ess logic result = business_logic(classification(text)) P ro tip: Try to think about the text from the model’s point of view! MODEL external knowledge facts that can change over time words, grammar, syntax information in the text

Slide 59

Slide 59 text

1 year 1 year 6× Case Study: GitLab Case Stud y : GitLab • extract actionable insights from support tickets and usage questions 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 60

Slide 60 text

1 year 1 year 6× Case Study: GitLab Case Stud y : GitLab • extract actionable insights from support tickets and usage questions • high-security environment 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 61

Slide 61 text

1 year 1 year 6× Case Study: GitLab Case Stud y : GitLab • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 62

Slide 62 text

1 year 1 year 6× Case Study: GitLab Case Stud y : GitLab • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions • separated general-purpose features from product-specific logic 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 63

Slide 63 text

1 year 1 year 6× Case Study: GitLab Case Stud y : GitLab • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions • separated general-purpose features from product-specific logic 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 64

Slide 64 text

spacy.fyi/ie-bootstrapping 💬 question ⚙ vectorizer query answers 📚 vector DB 📖 snippets + ⚙ vectorizer RAG RAG Retrieval-Augmented Generation

Slide 65

Slide 65 text

spacy.fyi/ie-bootstrapping 💬 question ⚙ vectorizer query answers 📚 vector DB 📖 snippets + ⚙ vectorizer 💬 question ⚙ text-to-SQL query data 📦 NLP pipeline 📖 texts + RIE RIE Retrieval via Information Extraction RAG RAG Retrieval-Augmented Generation

Slide 66

Slide 66 text

Slide 67

Slide 67 text

Language is just another interface.

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

“knocker-uppers”

Slide 70

Slide 70 text

The Window K nocking Machine Tes t ines.io/blog/window-knocking-machine-test “knocker-uppers”

Slide 71

Slide 71 text

The Window K nocking Machine Tes t ines.io/blog/window-knocking-machine-test Are you designing a window-knocking machine or an alarm clock? “knocker-uppers”

Slide 72

Slide 72 text

Hello, I ’ m Toni ’ s virtual assistant and I help schedule appointments. Do you have time at 1pm on Monday? No, but Tuesday would work for me. Okay, please confirm: Tuesday at 1pm? 1pm is unideal but 3pm would work. Toni doesn ’ t have availability at 3pm but I could offer a slot at 4pm or 5 : 30pm. Which time zone is this by the way? I ’ m in CET. ines.io/blog/window-knocking-machine-test

Slide 73

Slide 73 text

Slide 74

Slide 74 text

Slide 75

Slide 75 text

What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ ines.io/blog/window-knocking-machine-test

Slide 76

Slide 76 text

What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation ines.io/blog/window-knocking-machine-test

Slide 77

Slide 77 text

2023 Year Services Type ACME Inc. FooBar GmbH NLPCorp XKCD Ltd. Python AG 432,032 82,000 1,500 193,000 91,320 $ 2,625,032 Clients (28) Revenue What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation ines.io/blog/window-knocking-machine-test

Slide 78

Slide 78 text

2023 Year Services Type ACME Inc. FooBar GmbH NLPCorp XKCD Ltd. Python AG 432,032 82,000 1,500 193,000 91,320 $ 2,625,032 Clients (28) Revenue A I still needs produc t decisions! Kim Miller Analyst What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation ines.io/blog/window-knocking-machine-test

Slide 79

Slide 79 text

Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI

Slide 80

Slide 80 text

Reason and refactor. The key to success lies in your data and may surprise you! Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI

Slide 81

Slide 81 text

Reason and refactor. The key to success lies in your data and may surprise you! Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI Think beyond chat bots. You don’t want to build a “window-knocking machine”.

Slide 82

Slide 82 text

Reason and refactor. The key to success lies in your data and may surprise you! LLM Stay ambitious. Don’t compromise on best practices, e iciency and privacy. Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI Think beyond chat bots. You don’t want to build a “window-knocking machine”.

Slide 83

Slide 83 text

Explosion spaCy Prodigy Twitter Mastodon Bluesky explosion.ai spacy.io prodigy.ai @_inesmontani @[email protected] @inesmontani.bsky.social LinkedIn