Applied NLP with LLMs: Beyond Black-Box Monoliths

Slide 1

Slide 1 text

Ines Montani Explosion LLM

Slide 2

Slide 2 text

270m+ 270m+ spaC y Open-source library for industrial- strength natural language processing spacy.io downloads

Slide 3

Slide 3 text

270m+ 270m+ spaC y ChatGPT can write spaCy code! Open-source library for industrial- strength natural language processing spacy.io downloads

Slide 4

Slide 4 text

900+ 10k+ Prodi g y Modern scriptable annotation tool for machine learning developers prodigy.ai 900+ companies 10k+ users

Slide 5

Slide 5 text

900+ 10k+ Prodi g y Modern scriptable annotation tool for machine learning developers prodigy.ai Alex Smith Developer Kim Miller Analyst GPT-4 API 900+ companies 10k+ users

Slide 6

Slide 6 text

Falcon MIXTRAL GPT-4 LLM

Slide 7

Slide 7 text

Falcon MIXTRAL GPT-4 good contextual results LLM

Slide 8

Slide 8 text

Falcon MIXTRAL GPT-4 good contextual results easy to use & configure LLM

Slide 9

Slide 9 text

Falcon MIXTRAL GPT-4 good contextual results easy to use & configure fast prototyping LLM

Slide 10

Slide 10 text

Falcon MIXTRAL GPT-4 good contextual results ⚠ transparency easy to use & configure fast prototyping LLM

Slide 11

Slide 11 text

Falcon MIXTRAL GPT-4 good contextual results ⚠ transparency ⚠ e iciency easy to use & configure fast prototyping LLM

Slide 12

Slide 12 text

Falcon MIXTRAL GPT-4 good contextual results ⚠ data privacy ⚠ transparency ⚠ e iciency easy to use & configure fast prototyping LLM

Slide 13

Slide 13 text

Pro t ot y pe & Productio n CLOSE THE GAP BETWEEN CLOSE THE GAP BETWEEN

Slide 14

Slide 14 text

Pro t ot y pe & Productio n CLOSE THE GAP BETWEEN CLOSE THE GAP BETWEEN How to avoid the prototype plateau?

Slide 15

Slide 15 text

Pro t ot y pe & Productio n CLOSE THE GAP BETWEEN CLOSE THE GAP BETWEEN 📝 standardize inputs and outputs How to avoid the prototype plateau?

Slide 16

Slide 16 text

Pro t ot y pe & Productio n CLOSE THE GAP BETWEEN CLOSE THE GAP BETWEEN 📝 standardize inputs and outputs 📈 start with evaluation How to avoid the prototype plateau?

Slide 17

Slide 17 text

Pro t ot y pe & Productio n CLOSE THE GAP BETWEEN CLOSE THE GAP BETWEEN 📝 standardize inputs and outputs 📈 start with evaluation 🔮 assess utility, not just accuracy explosion.ai/blog/applied-nlp-thinking How to avoid the prototype plateau?

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

P rototype task-specific output 💬 prompt 📖 text LLM GPT-4 API

Slide 21

Slide 21 text

P rototype task-specific output 💬 prompt 📖 text LLM prompt model & transform output to structured data github.com/explosion/spacy-llm GPT-4 API

Slide 22

Slide 22 text

📖 text task-specific output P roduction P rototype task-specific output 💬 prompt 📖 text LLM prompt model & transform output to structured data github.com/explosion/spacy-llm GPT-4 API

Slide 23

Slide 23 text

📖 text task-specific output P roduction P rototype task-specific output 💬 prompt 📖 text LLM distilled task-specific components prompt model & transform output to structured data github.com/explosion/spacy-llm GPT-4 API

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation LLM

Slide 28

Slide 28 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM

Slide 29

Slide 29 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting

Slide 30

Slide 30 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting

Slide 31

Slide 31 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting transfer learning CO M PO N EN T

Slide 32

Slide 32 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting transfer learning CO M PO N EN T distilled model

Slide 33

Slide 33 text

Case Stud y : PyData NYC 8hr 400mb 2k+ 8hr 400mb 2k+ • extracting dishes, ingredients and equipment from r/cooking Reddit posts model size words/second data dev time spacy.fyi/pydata-nyc

Slide 34

Slide 34 text

Case Stud y : PyData NYC 8hr 400mb 2k+ 8hr 400mb 2k+ • extracting dishes, ingredients and equipment from r/cooking Reddit posts • used LLM during annotation model size words/second data dev time spacy.fyi/pydata-nyc

Slide 35

Slide 35 text

Case Stud y : PyData NYC 8hr 400mb 2k+ 8hr 400mb 2k+ • extracting dishes, ingredients and equipment from r/cooking Reddit posts • used LLM during annotation • 20× inference time speedup model size words/second data dev time spacy.fyi/pydata-nyc

Slide 36

Slide 36 text

Case Stud y : PyData NYC 8hr 400mb 2k+ 8hr 400mb 2k+ • extracting dishes, ingredients and equipment from r/cooking Reddit posts • used LLM during annotation • 20× inference time speedup • beat few-shot LLM baseline of 0.74 with task-specific model model size words/second data dev time spacy.fyi/pydata-nyc

Slide 37

Slide 37 text

Case Stud y : PyData NYC 8hr 400mb 2k+ 8hr 400mb 2k+ • extracting dishes, ingredients and equipment from r/cooking Reddit posts • used LLM during annotation • 20× inference time speedup • beat few-shot LLM baseline of 0.74 with task-specific model model size words/second data dev time spacy.fyi/pydata-nyc

Slide 38

Slide 38 text

Case Stud y : S&P Global 99% 6mb 16k+ 99% 6mb 16k+ • real-time commodities trading insights by extracting structured attributes model size words/second F-score explosion.ai/blog/sp-global-commodities

Slide 39

Slide 39 text

Case Stud y : S&P Global 99% 6mb 16k+ 99% 6mb 16k+ • real-time commodities trading insights by extracting structured attributes • high-security environment model size words/second F-score explosion.ai/blog/sp-global-commodities

Slide 40

Slide 40 text

Case Stud y : S&P Global 99% 6mb 16k+ 99% 6mb 16k+ • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation model size words/second F-score explosion.ai/blog/sp-global-commodities

Slide 41

Slide 41 text

Case Stud y : S&P Global 99% 6mb 16k+ 99% 6mb 16k+ • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop model size words/second F-score explosion.ai/blog/sp-global-commodities

Slide 42

Slide 42 text

Case Stud y : S&P Global 99% 6mb 16k+ 99% 6mb 16k+ • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop • 8 market pipelines in production model size words/second F-score explosion.ai/blog/sp-global-commodities

Slide 43

Slide 43 text

Case Stud y : S&P Global 99% 6mb 16k+ 99% 6mb 16k+ • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop • 8 market pipelines in production model size words/second F-score explosion.ai/blog/sp-global-commodities

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

break down larger problems

Slide 46

Slide 46 text

break down larger problems make problem easier

Slide 47

Slide 47 text

break down larger problems make problem easier reassess dependencies

Slide 48

Slide 48 text

break down larger problems make problem easier reassess dependencies choose the best techniques

Slide 49

Slide 49 text

break down larger problems make problem easier reassess dependencies choose the best techniques iterate on code and data

Slide 50

Slide 50 text

break down larger problems make problem easier factor out business logic reassess dependencies choose the best techniques iterate on code and data

Slide 51

Slide 51 text

Case Stud y : GitLab 1 year 6× 1 year 6× • extract actionable insights from support tickets and usage questions speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 52

Slide 52 text

Case Stud y : GitLab 1 year 6× 1 year 6× • extract actionable insights from support tickets and usage questions • high-security environment speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 53

Slide 53 text

Case Stud y : GitLab 1 year 6× 1 year 6× • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 54

Slide 54 text

Case Stud y : GitLab 1 year 6× 1 year 6× • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions • separated general-purpose features from product-specific logic speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 55

Slide 55 text

Case Stud y : GitLab 1 year 6× 1 year 6× • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions • separated general-purpose features from product-specific logic speedup of support tickets explosion.ai/blog/gitlab-support-insights

Slide 56

Slide 56 text

Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI

Slide 57

Slide 57 text

Reason and refactor. The key to success lies in your data and may surprise you! Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI

Slide 58

Slide 58 text

Reason and refactor. The key to success lies in your data and may surprise you! Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI Iterate. The right tooling and mindset gets you past the “prototype plateau”.

Slide 59

Slide 59 text

Reason and refactor. The key to success lies in your data and may surprise you! LLM Stay ambitious. Don’t compromise on best practices, e iciency and privacy. Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI Iterate. The right tooling and mindset gets you past the “prototype plateau”.

Slide 60

Slide 60 text

Explosion spaCy Prodigy Twitter Mastodon Bluesky explosion.ai spacy.io prodigy.ai @_inesmontani @[email protected] @inesmontani.bsky.social LinkedIn