Reality is not an End-to-End Prediction Problem: Applied NLP in the Age of Generative AI

Slide 1

Slide 1 text

Ines Montani Explosion LLM

Slide 2

Slide 2 text

de fi nition s E volution

Slide 3

Slide 3 text

de fi nition s E volution rules or instructions ✍ programming & rules

Slide 4

Slide 4 text

de fi nition s E volution rules or instructions ✍ programming & rules machine learning examples 📝 supervised learning

Slide 5

Slide 5 text

de fi nition s E volution rules or instructions ✍ programming & rules machine learning examples 📝 supervised learning in-context learning rules or instructions ✍ LLM prompt engineering

Slide 6

Slide 6 text

de fi nition s E volution rules or instructions ✍ programming & rules machine learning examples 📝 supervised learning in-context learning rules or instructions ✍ LLM prompt engineering instructions: human-shaped, easy for non-experts, risk of data drift ✍

Slide 7

Slide 7 text

Slide 8

Slide 8 text

de fi nition s E volution rules or instructions ✍ programming & rules machine learning examples 📝 supervised learning in-context learning rules or instructions ✍ LLM prompt engineering ? ? LLM instructions: human-shaped, easy for non-experts, risk of data drift ✍ 📝 examples: nuanced and intuitive behaviors, specific to use case, labor-intensive

Slide 9

Slide 9 text

Falcon MIXTRAL GPT-4 LLM

Slide 10

Slide 10 text

Falcon MIXTRAL GPT-4 good contextual results LLM

Slide 11

Slide 11 text

Falcon MIXTRAL GPT-4 good contextual results easy to use & configure LLM

Slide 12

Slide 12 text

Falcon MIXTRAL GPT-4 good contextual results easy to use & configure fast prototyping LLM

Slide 13

Slide 13 text

Falcon MIXTRAL GPT-4 good contextual results ⚠ transparency easy to use & configure fast prototyping LLM

Slide 14

Slide 14 text

Falcon MIXTRAL GPT-4 good contextual results ⚠ transparency ⚠ e iciency easy to use & configure fast prototyping LLM

Slide 15

Slide 15 text

Falcon MIXTRAL GPT-4 good contextual results ⚠ data privacy ⚠ transparency ⚠ e iciency easy to use & configure fast prototyping LLM

Slide 16

Slide 16 text

P rototype task-specific output 💬 prompt 📖 text LLM GPT-4 API

Slide 17

Slide 17 text

P rototype task-specific output 💬 prompt 📖 text LLM prompt model & transform output to structured data github.com/explosion/spacy-llm GPT-4 API

Slide 18

Slide 18 text

📖 text task-specific output P roduction P rototype task-specific output 💬 prompt 📖 text LLM prompt model & transform output to structured data github.com/explosion/spacy-llm GPT-4 API

Slide 19

Slide 19 text

📖 text task-specific output P roduction P rototype task-specific output 💬 prompt 📖 text LLM distilled task-specific components prompt model & transform output to structured data github.com/explosion/spacy-llm GPT-4 API

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation LLM

Slide 24

Slide 24 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM

Slide 25

Slide 25 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting

Slide 26

Slide 26 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting

Slide 27

Slide 27 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting transfer learning CO M PO N EN T

Slide 28

Slide 28 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting transfer learning CO M PO N EN T distilled model

Slide 29

Slide 29 text

Case Stud y : S&P Global 99% 99% • real-time commodities trading insights by extracting structured attributes 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 30

Slide 30 text

Case Stud y : S&P Global 99% 99% • real-time commodities trading insights by extracting structured attributes • high-security environment 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 31

Slide 31 text

Case Stud y : S&P Global 99% 99% • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 32

Slide 32 text

Case Stud y : S&P Global 99% 99% • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 33

Slide 33 text

Case Stud y : S&P Global 99% 99% • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop • 8 market pipelines in production 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 34

Slide 34 text

Case Stud y : S&P Global 99% 99% • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop • 8 market pipelines in production 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 35

Slide 35 text

Refactor your code and data.

Slide 36

Slide 36 text

Software 1.0 Software 1.0 📄 code 💾 program compiler

Slide 37

Slide 37 text

Software 1.0 Software 1.0 📄 code 💾 program compiler Software 2.0 Software 2.0 📊 data 🔮 model algorithm

Slide 38

Slide 38 text

Software 1.0 Software 1.0 📄 code 💾 program compiler Software 2.0 Software 2.0 📊 data 🔮 model algorithm ✅ tests 📈 evaluation

Slide 39

Slide 39 text

Software 1.0 Software 1.0 📄 code 💾 program compiler Software 2.0 Software 2.0 📊 data 🔮 model algorithm ✅ tests 📈 evaluation refactoring refactoring iteration iteration

Slide 40

Slide 40 text

I lo v e cats. SIMILAR OR NOT? I ha t e cats.

Slide 41

Slide 41 text

I lo v e cats. SIMILAR OR NOT? I ha t e cats. Your application context always matters!

Slide 42

Slide 42 text

Case Stud y : GitLab 1 year 1 year 6× • extract actionable insights from support tickets and usage questions 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 43

Slide 43 text

Case Stud y : GitLab 1 year 1 year 6× • extract actionable insights from support tickets and usage questions • high-security environment 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 44

Slide 44 text

Case Stud y : GitLab 1 year 1 year 6× • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 45

Slide 45 text

Case Stud y : GitLab 1 year 1 year 6× • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions • separated general-purpose features from product-specific logic 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 46

Slide 46 text

Case Stud y : GitLab 1 year 1 year 6× • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions • separated general-purpose features from product-specific logic 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 47

Slide 47 text

Language is just another interface.

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

“knocker-uppers”

Slide 50

Slide 50 text

The Window K nocking Machine Tes t ines.io/blog/window-knocking-machine-test “knocker-uppers”

Slide 51

Slide 51 text

The Window K nocking Machine Tes t ines.io/blog/window-knocking-machine-test Are you designing a window-knocking machine or an alarm clock? “knocker-uppers”

Slide 52

Slide 52 text

Hello, I ’ m Toni ’ s virtual assistant and I help schedule appointments. Do you have time at 1pm on Monday? No, but Tuesday would work for me. Okay, please confirm: Tuesday at 1pm? 1pm is unideal but 3pm would work. Toni doesn ’ t have availability at 3pm but I could offer a slot at 4pm or 5 : 30pm. Which time zone is this by the way? I ’ m in CET. ines.io/blog/window-knocking-machine-test

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Slide 55

Slide 55 text

What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ ines.io/blog/window-knocking-machine-test

Slide 56

Slide 56 text

What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation ines.io/blog/window-knocking-machine-test

Slide 57

Slide 57 text

2023 Year Services Type ACME Inc. FooBar GmbH NLPCorp XKCD Ltd. Python AG 432,032 82,000 1,500 193,000 91,320 $ 2,625,032 Clients (28) Revenue What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation ines.io/blog/window-knocking-machine-test

Slide 58

Slide 58 text

2023 Year Services Type ACME Inc. FooBar GmbH NLPCorp XKCD Ltd. Python AG 432,032 82,000 1,500 193,000 91,320 $ 2,625,032 Clients (28) Revenue A I still needs produc t decisions! Kim Miller Analyst What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation ines.io/blog/window-knocking-machine-test

Slide 59

Slide 59 text

Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI

Slide 60

Slide 60 text

Reason and refactor. The key to success lies in your data and may surprise you! Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI

Slide 61

Slide 61 text

Reason and refactor. The key to success lies in your data and may surprise you! Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI Think beyond chat bots. You don’t want to build a “window-knocking machine”.

Slide 62

Slide 62 text

Reason and refactor. The key to success lies in your data and may surprise you! LLM Stay ambitious. Don’t compromise on best practices, e iciency and privacy. Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI Think beyond chat bots. You don’t want to build a “window-knocking machine”.

Slide 63

Slide 63 text

Explosion spaCy Prodigy Twitter Mastodon Bluesky explosion.ai spacy.io prodigy.ai @_inesmontani @[email protected] @inesmontani.bsky.social LinkedIn