Large Language Models: From Prototype to Production (EuroPython keynote)

Large Language Models (LLMs) have shown some impressive capabilities and their impact is the topic of the moment. What will the future look like? Are we going to only talk to bots? Will prompting replace programming? Or are we just hyping up unreliable parrots and burning money? In this talk, I'll present visions for NLP in the age of LLMs and a pragmatic, practical approach for how to use Large Language Models to ship more successful NLP projects from prototype to production today.

Video: https://www.youtube.com/watch?v=ZjjgMiCU8s4
Twitter: https://twitter.com/_inesmontani/status/1681700743693172738
LinkedIn: https://www.linkedin.com/posts/inesmontani_nlp-llm-llms-activity-7087478372418625536-3VDo

Transcript

1. Ines Montani Explosion LARGE LANGUAGE LARGE LANGUAGE MODELS ✨ CHATGPT " ARTIFICIAL INTELLIGENCE # MACHINE LEARNING ✨ PROTOTYPE TO PRODUCTION MODELS FROM LLAMA $ NATURAL LANGUAGE PROCESSING % ✨ OPEN SOURCE & PYTHON ' PROMPT ENGINEERING ⚙ ZERO-SHOT LEARNING ) GPT-4 EVALUATION * COPILOT + GENERATIVE AI , Ines Montani - Explosion
2. SPACY SPACY.IO & @SPACY_IO ✍ SPACY.TV / GITHUB.COM/EXPLOSION/SPACY Open-source library for industrial-strength Natural Language Processing 150m+ downloads
3. SPACY SPACY.IO & @SPACY_IO ✍ SPACY.TV / GITHUB.COM/EXPLOSION/SPACY Open-source library for industrial-strength Natural Language Processing 150m+ downloads ChatGPT can write spaCy code!
4. PRODIGY Modern scriptable annotation tool for machine learning developers PRODIGY.AI & GITHUB.COM/EXPLOSION/PRODIGY-RECIPES 8k+ users 700+ companies
5. PRODIGY Modern scriptable annotation tool for machine learning developers PRODIGY.AI & GITHUB.COM/EXPLOSION/PRODIGY-RECIPES 8k+ users 700+ companies
6. 0 single/multi-doc summarization ✅ problem solving ✍ paraphrasing 2 reasoning 3 style transfer Generative ❓question answering 5 text classification 6 entity recognition 7 relation extraction 8 grammar & morphology ) semantic parsing 9 coreference resolution % discourse structure Predictive UNDERSTANDING NLP TASKS
7. 0 single/multi-doc summarization ✅ problem solving ✍ paraphrasing 2 reasoning 3 style transfer Generative ❓question answering 5 text classification 6 entity recognition 7 relation extraction 8 grammar & morphology ) semantic parsing 9 coreference resolution % discourse structure Predictive UNDERSTANDING NLP TASKS human-readable machine-readable
8. THE HISTORY OF FUTURE TECHNOLOGY
9. THE HISTORY OF FUTURE TECHNOLOGY How people in 1900 imagined the year 2000
10. THE HISTORY OF FUTURE TECHNOLOGY How people in 1900 imagined the year 2000
11. THE HISTORY OF FUTURE TECHNOLOGY
12. THE HISTORY OF FUTURE TECHNOLOGY manual calculation vs. calculator
13. THE HISTORY OF FUTURE TECHNOLOGY manual calculation vs. calculator
14. THE HISTORY OF FUTURE TECHNOLOGY “knocker-uppers” vs. alarm clock manual calculation vs. calculator
15. THE HISTORY OF FUTURE TECHNOLOGY
16. THE HISTORY OF FUTURE TECHNOLOGY human assistant vs. calendar apps Calendly Fantastical
17. THE HISTORY OF FUTURE TECHNOLOGY human assistant vs. calendar apps Calendly Fantastical WHAT’S NEXT?
18. NLP IN THE AGE OF LLMS
19. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need : %
20. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need : % lots of humans is all you need prompting is all you need ; "
21. COMPANY COMPANY MONEY INVESTOR “Hooli raises $5m to revolutionize search, led by ACME Ventures” 5923214 1681056 CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA Database
22. COMPANY COMPANY MONEY INVESTOR “Hooli raises $5m to revolutionize search, led by ACME Ventures” 5923214 1681056 CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA Database named entity recognition
23. COMPANY COMPANY MONEY INVESTOR “Hooli raises $5m to revolutionize search, led by ACME Ventures” 5923214 1681056 CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA Database named entity recognition entity disambiguation
24. COMPANY COMPANY MONEY INVESTOR “Hooli raises $5m to revolutionize search, led by ACME Ventures” 5923214 1681056 CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA Database named entity recognition entity disambiguation custom database lookup
25. COMPANY COMPANY MONEY INVESTOR “Hooli raises $5m to revolutionize search, led by ACME Ventures” 5923214 1681056 CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA Database named entity recognition entity disambiguation custom database lookup currency normalization
26. COMPANY COMPANY MONEY INVESTOR “Hooli raises $5m to revolutionize search, led by ACME Ventures” 5923214 1681056 CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA Database named entity recognition entity disambiguation custom database lookup currency normalization entity relation extraction
27. VISION #1 dialogue is all you need % < LLM = user actions or information natural language input
28. VISION #1 dialogue is all you need % < LLM = user actions or information natural language input LLM is the system and needs to manage the whole interaction
29. VISION #2 prompting is all you need " < LLM 0 text % prompt > system = user ? structured data
30. VISION #2 prompting is all you need " < LLM 0 text % prompt > system = user LLM replaces the specific ML model ? structured data
31. VISION #3 modern practical NLP - @ developer 2 code < LLM 5 training data > system = user ? structured data ⚙ ML system
32. VISION #3 modern practical NLP - @ developer 2 code < LLM 5 training data > system = user ? structured data ⚙ ML system LLM helps with building the pipeline
33. VISION #3 modern practical NLP - @ developer 2 code < LLM 5 training data > system = user ? structured data ⚙ ML system LLM helps with building the pipeline
34. VISION #3 modern practical NLP - @ developer 2 code < LLM 5 training data > system = user ? structured data ⚙ ML system LLM helps with building the pipeline
35. 0 single/multi-doc summarization ✅ problem solving ✍ paraphrasing 2 reasoning 3 style transfer Generative ❓question answering 5 text classification 6 entity recognition 7 relation extraction 8 grammar & morphology ) semantic parsing 9 coreference resolution % discourse structure Predictive UNDERSTANDING NLP TASKS
36. LLMS VS. TASK- SPECIFIC MODELS Text Classification accuracy on % of examples SST2 AG News Banking77 GPT-3 baseline 65 70 75 80 85 90 95 100 1% 5% 10% 20% 50% 100% Explosion (2023), to be released
37. LLMS VS. TASK- SPECIFIC MODELS Text Classification accuracy on % of examples SST2 AG News Banking77 GPT-3 baseline 65 70 75 80 85 90 95 100 1% 5% 10% 20% 50% 100% Explosion (2023), to be released
38. LLMS VS. TASK- SPECIFIC MODELS Text Classification accuracy on % of examples SST2 AG News Banking77 GPT-3 baseline 65 70 75 80 85 90 95 100 1% 5% 10% 20% 50% 100% Explosion (2023), to be released
39. LLMS VS. TASK- SPECIFIC MODELS Text Classification accuracy on % of examples SST2 AG News Banking77 GPT-3 baseline 65 70 75 80 85 90 95 100 1% 5% 10% 20% 50% 100% Explosion (2023), to be released
40. LLMS VS. TASK- SPECIFIC MODELS F-Score Speed (words/s) GPT-3.5 1 78.6 < 100 GPT-4 1 83.5 < 100 spaCy 91.6 4,000 Flair 93.1 1,000 SOTA 2023 2 94.6 1,000 SOTA 2003 3 88.8 > 20,000 1. Ashok and Lipton (2023), 2. Wang et al. (2021), 3. Florian et al. (2003) SOTA on few- shot prompting RoBERTa-base CoNLL 2003 NER Text Classification accuracy on % of examples SST2 AG News Banking77 GPT-3 baseline 65 70 75 80 85 90 95 100 1% 5% 10% 20% 50% 100% Explosion (2023), to be released
41. < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do
42. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do
43. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer
44. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer prompt engineering
45. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer prompt engineering problem definition
46. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer prompt engineering data annotation problem definition
47. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer prompt engineering data annotation model training problem definition
48. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer prompt engineering data annotation evaluation model training problem definition
49. ⚙ Task-Specific Model fine-tuning BERT etc. knows less about what the text means can encode exactly what you want it to do < Large Language Model in-context learning knows a lot about what the text means doesn’t really know what you want it to do @ developer prompt engineering data annotation evaluation model training + production problem definition
50. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need : % lots of humans is all you need prompting is all you need ; " modern practical NLP -
51. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need : % lots of humans is all you need prompting is all you need ; " modern practical NLP - structured data
52. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need : % lots of humans is all you need prompting is all you need ; " modern practical NLP - structured data humans in the loop
53. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need : % lots of humans is all you need prompting is all you need ; " modern practical NLP - structured data fast prototyping humans in the loop
54. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need : % lots of humans is all you need prompting is all you need ; " modern practical NLP - structured data fast prototyping humans in the loop powered by open source
55. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need : % lots of humans is all you need prompting is all you need ; " modern practical NLP - structured data fast prototyping humans in the loop powered by open source conversational and graphical interfaces
56. LLM-POWERED NLP IN PRACTICE LLM-powered collaborative data development environment @
57. LLM-POWERED NLP IN PRACTICE LLM-powered collaborative data development environment @ Assign labeling tasks to LLMs "
58. LLM-POWERED NLP IN PRACTICE LLM-powered collaborative data development environment @ Assign labeling tasks to LLMs " Review label decisions, correct errors A
59. LLM-POWERED NLP IN PRACTICE LLM-powered collaborative data development environment @ Assign labeling tasks to LLMs " Review label decisions, correct errors A Tune prompts and compare LLMs empirically ?
60. LLM-POWERED NLP IN PRACTICE LLM-powered collaborative data development environment @ Assign labeling tasks to LLMs " Review label decisions, correct errors A Tune prompts and compare LLMs empirically ? Build data sets to train and evaluate e icient, production-ready pipelines +
61. 8 PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS
62. 8 correct mistakes PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS
63. PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS correct mistakes
64. add correct answer to prompt to tune it PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS correct mistakes
65. GITHUB.COM/EXPLOSION/SPACY-LLM TOWARDS STRUCTURED DATA Prompt Template < LLM London is bigger than Berlin LOCATION: London, Berlin LOCATION
66. GITHUB.COM/EXPLOSION/SPACY-LLM TOWARDS STRUCTURED DATA Prompt Template < LLM London is bigger than Berlin LOCATION: London, Berlin LOCATION
67. GITHUB.COM/EXPLOSION/SPACY-LLM % unstructured text input ? structured Doc object
68. GITHUB.COM/EXPLOSION/SPACY-LLM Named Entity Recognition Text Classification Relation Extraction Lemma- tization % unstructured text input ? structured Doc object
69. GITHUB.COM/EXPLOSION/SPACY-LLM Named Entity Recognition Text Classification Relation Extraction Lemma- tization % unstructured text input ? structured Doc object < LLM ⚙ Supervised Model ✍ Rules mix, match and replace techniques
70. EASIER ISN'T AMBITIOUS ENOUGH. Let’s not settle for systems that are worse than what we’ve been building.
71. SPECIFIC Task-specific models powered by LLMs IS BETTER.
72. SMALLER & FASTER Task-specific models powered by LLMs IS BETTER.
73. PRIVATE Task-specific models powered by LLMs IS BETTER.
74. BETTER Task-specific models powered by LLMs IS BETTER.
75. THANK YOU! - Explosion – explosion.ai B spaCy – spacy.io ✨ Prodigy – prodigy.ai C Twitter – @_inesmontani D Mastodon – @ines@sigmoid.social E LinkedIn