$30 off During Our Annual Pro Sale. View Details »

Large Language Models: From Prototype to Produc...

Large Language Models: From Prototype to Production (EuroPython keynote)

Large Language Models (LLMs) have shown some impressive capabilities and their impact is the topic of the moment. What will the future look like? Are we going to only talk to bots? Will prompting replace programming? Or are we just hyping up unreliable parrots and burning money? In this talk, I'll present visions for NLP in the age of LLMs and a pragmatic, practical approach for how to use Large Language Models to ship more successful NLP projects from prototype to production today.

Video: https://www.youtube.com/watch?v=ZjjgMiCU8s4
Twitter: https://twitter.com/_inesmontani/status/1681700743693172738
LinkedIn: https://www.linkedin.com/posts/inesmontani_nlp-llm-llms-activity-7087478372418625536-3VDo

Ines Montani

July 19, 2023
Tweet

Video


Resources

More Decks by Ines Montani

Other Decks in Technology

Transcript

  1. Ines Montani Explosion LARGE LANGUAGE LARGE LANGUAGE MODELS ✨ CHATGPT

    " ARTIFICIAL INTELLIGENCE # MACHINE LEARNING ✨ PROTOTYPE TO PRODUCTION MODELS FROM LLAMA $ NATURAL LANGUAGE PROCESSING % ✨ OPEN SOURCE & PYTHON ' PROMPT ENGINEERING ⚙ ZERO-SHOT LEARNING ) GPT-4 EVALUATION * COPILOT + GENERATIVE AI , Ines Montani - Explosion
  2. SPACY SPACY.IO & @SPACY_IO ✍ SPACY.TV / GITHUB.COM/EXPLOSION/SPACY Open-source library

    for industrial-strength Natural Language Processing 150m+ downloads
  3. SPACY SPACY.IO & @SPACY_IO ✍ SPACY.TV / GITHUB.COM/EXPLOSION/SPACY Open-source library

    for industrial-strength Natural Language Processing 150m+ downloads ChatGPT can write spaCy code!
  4. PRODIGY Modern scriptable annotation tool for machine learning developers PRODIGY.AI

    & GITHUB.COM/EXPLOSION/PRODIGY-RECIPES 8k+ users 700+ companies
  5. PRODIGY Modern scriptable annotation tool for machine learning developers PRODIGY.AI

    & GITHUB.COM/EXPLOSION/PRODIGY-RECIPES 8k+ users 700+ companies
  6. 0 single/multi-doc summarization ✅ problem solving ✍ paraphrasing 2 reasoning

    3 style transfer Generative ❓question answering 5 text classification 6 entity recognition 7 relation extraction 8 grammar & morphology ) semantic parsing 9 coreference resolution % discourse structure Predictive UNDERSTANDING NLP TASKS
  7. 0 single/multi-doc summarization ✅ problem solving ✍ paraphrasing 2 reasoning

    3 style transfer Generative ❓question answering 5 text classification 6 entity recognition 7 relation extraction 8 grammar & morphology ) semantic parsing 9 coreference resolution % discourse structure Predictive UNDERSTANDING NLP TASKS human-readable machine-readable
  8. NLP IN THE AGE OF LLMS SQL is all you

    need dialogue is all you need : %
  9. NLP IN THE AGE OF LLMS SQL is all you

    need dialogue is all you need : % lots of humans is all you need prompting is all you need ; "
  10. COMPANY COMPANY MONEY INVESTOR “Hooli raises $5m to revolutionize search,

    led by ACME Ventures” 5923214 1681056 CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA Database
  11. COMPANY COMPANY MONEY INVESTOR “Hooli raises $5m to revolutionize search,

    led by ACME Ventures” 5923214 1681056 CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA Database named entity recognition
  12. COMPANY COMPANY MONEY INVESTOR “Hooli raises $5m to revolutionize search,

    led by ACME Ventures” 5923214 1681056 CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA Database named entity recognition entity disambiguation
  13. COMPANY COMPANY MONEY INVESTOR “Hooli raises $5m to revolutionize search,

    led by ACME Ventures” 5923214 1681056 CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA Database named entity recognition entity disambiguation custom database lookup
  14. COMPANY COMPANY MONEY INVESTOR “Hooli raises $5m to revolutionize search,

    led by ACME Ventures” 5923214 1681056 CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA Database named entity recognition entity disambiguation custom database lookup currency normalization
  15. COMPANY COMPANY MONEY INVESTOR “Hooli raises $5m to revolutionize search,

    led by ACME Ventures” 5923214 1681056 CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA Database named entity recognition entity disambiguation custom database lookup currency normalization entity relation extraction
  16. VISION #1 dialogue is all you need % < LLM

    = user actions or information natural language input
  17. VISION #1 dialogue is all you need % < LLM

    = user actions or information natural language input LLM is the system and needs to manage the whole interaction
  18. VISION #2 prompting is all you need " < LLM

    0 text % prompt > system = user ? structured data
  19. VISION #2 prompting is all you need " < LLM

    0 text % prompt > system = user LLM replaces the specific ML model ? structured data
  20. VISION #3 modern practical NLP - @ developer 2 code

    < LLM 5 training data > system = user ? structured data ⚙ ML system
  21. VISION #3 modern practical NLP - @ developer 2 code

    < LLM 5 training data > system = user ? structured data ⚙ ML system LLM helps with building the pipeline
  22. VISION #3 modern practical NLP - @ developer 2 code

    < LLM 5 training data > system = user ? structured data ⚙ ML system LLM helps with building the pipeline
  23. VISION #3 modern practical NLP - @ developer 2 code

    < LLM 5 training data > system = user ? structured data ⚙ ML system LLM helps with building the pipeline
  24. 0 single/multi-doc summarization ✅ problem solving ✍ paraphrasing 2 reasoning

    3 style transfer Generative ❓question answering 5 text classification 6 entity recognition 7 relation extraction 8 grammar & morphology ) semantic parsing 9 coreference resolution % discourse structure Predictive UNDERSTANDING NLP TASKS
  25. LLMS VS. TASK- SPECIFIC MODELS Text Classification accuracy on %

    of examples SST2 AG News Banking77 GPT-3 baseline 65 70 75 80 85 90 95 100 1% 5% 10% 20% 50% 100% Explosion (2023), to be released
  26. LLMS VS. TASK- SPECIFIC MODELS Text Classification accuracy on %

    of examples SST2 AG News Banking77 GPT-3 baseline 65 70 75 80 85 90 95 100 1% 5% 10% 20% 50% 100% Explosion (2023), to be released
  27. LLMS VS. TASK- SPECIFIC MODELS Text Classification accuracy on %

    of examples SST2 AG News Banking77 GPT-3 baseline 65 70 75 80 85 90 95 100 1% 5% 10% 20% 50% 100% Explosion (2023), to be released
  28. LLMS VS. TASK- SPECIFIC MODELS Text Classification accuracy on %

    of examples SST2 AG News Banking77 GPT-3 baseline 65 70 75 80 85 90 95 100 1% 5% 10% 20% 50% 100% Explosion (2023), to be released
  29. LLMS VS. TASK- SPECIFIC MODELS F-Score Speed (words/s) GPT-3.5 1

    78.6 < 100 GPT-4 1 83.5 < 100 spaCy 91.6 4,000 Flair 93.1 1,000 SOTA 2023 2 94.6 1,000 SOTA 2003 3 88.8 > 20,000 1. Ashok and Lipton (2023), 2. Wang et al. (2021), 3. Florian et al. (2003) SOTA on few- shot prompting RoBERTa-base CoNLL 2003 NER Text Classification accuracy on % of examples SST2 AG News Banking77 GPT-3 baseline 65 70 75 80 85 90 95 100 1% 5% 10% 20% 50% 100% Explosion (2023), to be released
  30. < Large Language Model in-context learning knows a lot about

    what the text means doesn’t really know what you want it to do
  31. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what

    the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do
  32. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what

    the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer
  33. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what

    the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer prompt engineering
  34. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what

    the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer prompt engineering problem definition
  35. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what

    the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer prompt engineering data annotation problem definition
  36. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what

    the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer prompt engineering data annotation model training problem definition
  37. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what

    the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer prompt engineering data annotation evaluation model training problem definition
  38. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what

    the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer prompt engineering data annotation evaluation model training + production problem definition
  39. NLP IN THE AGE OF LLMS SQL is all you

    need dialogue is all you need : % lots of humans is all you need prompting is all you need ; " modern practical NLP -
  40. NLP IN THE AGE OF LLMS SQL is all you

    need dialogue is all you need : % lots of humans is all you need prompting is all you need ; " modern practical NLP - structured data
  41. NLP IN THE AGE OF LLMS SQL is all you

    need dialogue is all you need : % lots of humans is all you need prompting is all you need ; " modern practical NLP - structured data humans in the loop
  42. NLP IN THE AGE OF LLMS SQL is all you

    need dialogue is all you need : % lots of humans is all you need prompting is all you need ; " modern practical NLP - structured data fast prototyping humans in the loop
  43. NLP IN THE AGE OF LLMS SQL is all you

    need dialogue is all you need : % lots of humans is all you need prompting is all you need ; " modern practical NLP - structured data fast prototyping humans in the loop powered by open source
  44. NLP IN THE AGE OF LLMS SQL is all you

    need dialogue is all you need : % lots of humans is all you need prompting is all you need ; " modern practical NLP - structured data fast prototyping humans in the loop powered by open source conversational and graphical interfaces
  45. LLM-POWERED NLP IN PRACTICE LLM-powered collaborative data development environment @

    Assign labeling tasks to LLMs " Review label decisions, correct errors A
  46. LLM-POWERED NLP IN PRACTICE LLM-powered collaborative data development environment @

    Assign labeling tasks to LLMs " Review label decisions, correct errors A Tune prompts and compare LLMs empirically ?
  47. LLM-POWERED NLP IN PRACTICE LLM-powered collaborative data development environment @

    Assign labeling tasks to LLMs " Review label decisions, correct errors A Tune prompts and compare LLMs empirically ? Build data sets to train and evaluate e icient, production-ready pipelines +
  48. GITHUB.COM/EXPLOSION/SPACY-LLM Named Entity Recognition Text Classification Relation Extraction Lemma- tization

    % unstructured text input ? structured Doc object < LLM ⚙ Supervised Model ✍ Rules mix, match and replace techniques
  49. EASIER ISN'T AMBITIOUS ENOUGH. Let’s not settle for systems that

    are worse than what we’ve been building.
  50. THANK YOU! - Explosion – explosion.ai B spaCy – spacy.io

    ✨ Prodigy – prodigy.ai C Twitter – @_inesmontani D Mastodon – @[email protected] E LinkedIn