Large Language Models: From Prototype to Production (PyData London keynote)

Large Language Models (LLMs) have shown some impressive capabilities and their impact is the topic of the moment. What will the future look like? Are we going to only talk to bots? Will prompting replace programming? Or are we just hyping up unreliable parrots and burning money? In this talk, I'll present visions for NLP in the age of LLMs and a pragmatic, practical approach for how to use Large Language Models to ship more successful NLP projects from prototype to production today.

Transcript

1. Ines Montani Explosion LARGE LANGUAGE LARGE LANGUAGE MODELS ✨ CHATGPT " ARTIFICIAL INTELLIGENCE # MACHINE LEARNING ✨ PROTOTYPE TO PRODUCTION MODELS FROM LLAMA $ NATURAL LANGUAGE PROCESSING % ✨ OPEN SOURCE & PYTHON ' PROMPT ENGINEERING ⚙ ZERO-SHOT LEARNING ) GPT-4 EVALUATION * COPILOT + GENERATIVE AI , Ines Montani - Explosion
2. SPACY SPACY.IO & @SPACY_IO ✍ SPACY.TV / GITHUB.COM/EXPLOSION/SPACY Open-source library for industrial-strength Natural Language Processing 140m+ downloads
3. SPACY SPACY.IO & @SPACY_IO ✍ SPACY.TV / GITHUB.COM/EXPLOSION/SPACY Open-source library for industrial-strength Natural Language Processing 140m+ downloads ChatGPT can write spaCy code!
4. PRODIGY Modern scriptable annotation tool for machine learning developers PRODIGY.AI & GITHUB.COM/EXPLOSION/PRODIGY-RECIPES 8k+ users 700+ companies
5. PRODIGY Modern scriptable annotation tool for machine learning developers PRODIGY.AI & GITHUB.COM/EXPLOSION/PRODIGY-RECIPES 8k+ users 700+ companies
6. THE HISTORY OF FUTURE TECHNOLOGY
7. THE HISTORY OF FUTURE TECHNOLOGY How people in 1900 imagined the year 2000
8. THE HISTORY OF FUTURE TECHNOLOGY How people in 1900 imagined the year 2000
9. THE HISTORY OF FUTURE TECHNOLOGY
10. THE HISTORY OF FUTURE TECHNOLOGY manual calculation vs. calculator
11. THE HISTORY OF FUTURE TECHNOLOGY manual calculation vs. calculator
12. THE HISTORY OF FUTURE TECHNOLOGY “knocker-uppers” vs. alarm clock manual calculation vs. calculator
13. THE HISTORY OF FUTURE TECHNOLOGY
14. THE HISTORY OF FUTURE TECHNOLOGY human assistant vs. calendar apps Calendly Fantastical
15. THE HISTORY OF FUTURE TECHNOLOGY human assistant vs. calendar apps Calendly Fantastical WHAT’S NEXT?
16. NLP IN THE AGE OF LLMS
17. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need 0 %
18. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need 0 % lots of humans is all you need prompting is all you need 1 "
19. VISION #1 dialogue is all you need % 2 LLM 3 user actions or information natural language input
20. VISION #1 dialogue is all you need % 2 LLM 3 user actions or information natural language input LLM is the system and needs to manage the whole interaction
21. VISION #2 prompting is all you need " 2 LLM 4 text % prompt 5 system 3 user 6 structured data
22. VISION #2 prompting is all you need " 2 LLM 4 text % prompt 5 system 3 user LLM replaces the specific ML model 6 structured data
23. VISION #3 modern practical NLP - 7 developer 8 code 2 LLM 9 training data 5 system 3 user 6 structured data ⚙ ML system
24. VISION #3 modern practical NLP - 7 developer 8 code 2 LLM 9 training data 5 system 3 user 6 structured data ⚙ ML system LLM helps with building the pipeline
25. VISION #3 modern practical NLP - 7 developer 8 code 2 LLM 9 training data 5 system 3 user 6 structured data ⚙ ML system LLM helps with building the pipeline
26. VISION #3 modern practical NLP - 7 developer 8 code 2 LLM 9 training data 5 system 3 user 6 structured data ⚙ ML system LLM helps with building the pipeline
27. COMPANY COMPANY MONEY INVESTOR “Hooli raises $5m to revolutionize search, led by ACME Ventures” 5923214 1681056 CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA Database
28. Database CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA
29. Database CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA named entity recognition
30. Database CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA named entity recognition entity disambiguation
31. Database CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA named entity recognition entity disambiguation custom database lookup
32. Database CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA named entity recognition entity disambiguation custom database lookup currency normalization
33. Database CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA named entity recognition entity disambiguation custom database lookup currency normalization entity relation extraction
34. Database CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA named entity recognition entity disambiguation custom database lookup currency normalization entity relation extraction 6 structured data 2 LLM 7 developer quick prototype
35. Database CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA named entity recognition entity disambiguation custom database lookup currency normalization entity relation extraction 6 structured data 2 LLM 7 developer quick prototype ⚡ fast to develop, slow to run, hard to improve
36. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need 0 % lots of humans is all you need prompting is all you need 1 " modern practical NLP -
37. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need 0 % lots of humans is all you need prompting is all you need 1 " modern practical NLP - structured data
38. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need 0 % lots of humans is all you need prompting is all you need 1 " modern practical NLP - structured data humans in the loop
39. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need 0 % lots of humans is all you need prompting is all you need 1 " modern practical NLP - structured data fast prototyping humans in the loop
40. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need 0 % lots of humans is all you need prompting is all you need 1 " modern practical NLP - structured data fast prototyping humans in the loop powered by open source
41. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need 0 % lots of humans is all you need prompting is all you need 1 " modern practical NLP - structured data fast prototyping humans in the loop powered by open source conversational and graphical interfaces
42. NLP IN THE AGE OF LLMS SQL is all you need dialogue is all you need 0 % lots of humans is all you need prompting is all you need 1 " modern practical NLP - structured data fast prototyping humans in the loop powered by open source robust evaluation conversational and graphical interfaces
43. A CASE FOR LLM PRAGMATISM EXPLOSION.AI/BLOG/AGAINST-LLM-MAXIMALISM NOOO YOU CAN'T JUST MIX UP ALL THE STEPS OF YOUR TASK AND ASK AN LLM TO DO IT ALL. HOW WILL YOU EVER MAKE A RELIABLE AND EXTENSIBLE SYSTEM THAT WAY? HAHA LLM GO BRRR
44. A CASE FOR LLM PRAGMATISM EXPLOSION.AI/BLOG/AGAINST-LLM-MAXIMALISM NOOO YOU CAN'T JUST MIX UP ALL THE STEPS OF YOUR TASK AND ASK AN LLM TO DO IT ALL. HOW WILL YOU EVER MAKE A RELIABLE AND EXTENSIBLE SYSTEM THAT WAY? HAHA LLM GO BRRR avoid coupling prediction tasks to arbitrary business logic
45. A CASE FOR LLM PRAGMATISM EXPLOSION.AI/BLOG/AGAINST-LLM-MAXIMALISM NOOO YOU CAN'T JUST MIX UP ALL THE STEPS OF YOUR TASK AND ASK AN LLM TO DO IT ALL. HOW WILL YOU EVER MAKE A RELIABLE AND EXTENSIBLE SYSTEM THAT WAY? HAHA LLM GO BRRR avoid coupling prediction tasks to arbitrary business logic design modular solutions
46. A CASE FOR LLM PRAGMATISM EXPLOSION.AI/BLOG/AGAINST-LLM-MAXIMALISM NOOO YOU CAN'T JUST MIX UP ALL THE STEPS OF YOUR TASK AND ASK AN LLM TO DO IT ALL. HOW WILL YOU EVER MAKE A RELIABLE AND EXTENSIBLE SYSTEM THAT WAY? HAHA LLM GO BRRR avoid coupling prediction tasks to arbitrary business logic design modular solutions prototype modules with LLMs
47. A CASE FOR LLM PRAGMATISM EXPLOSION.AI/BLOG/AGAINST-LLM-MAXIMALISM NOOO YOU CAN'T JUST MIX UP ALL THE STEPS OF YOUR TASK AND ASK AN LLM TO DO IT ALL. HOW WILL YOU EVER MAKE A RELIABLE AND EXTENSIBLE SYSTEM THAT WAY? HAHA LLM GO BRRR avoid coupling prediction tasks to arbitrary business logic design modular solutions prototype modules with LLMs evaluate alternatives
48. TRADE-OFFS performance on the bar exam kentlaw.iit.edu
49. TRADE-OFFS performance on the bar exam kentlaw.iit.edu OpenAI API latency promptlayer.com
50. TRADE-OFFS Supervised 1 LLM 2 accuracy words/s accuracy words/s Textcat on SST2 (Stanford Sentiment Treebank) 0.9 4019 0.9 <100 Textcat on Banking77 (intent recognition) 0.9 3234 0.7 <100 NER on AnEm (anatomical entity mentions) 0.7 5146 0.1 <100 1. RoBERTa-base with spaCy, 2. text-davinci-003 zero-shot ongoing experiments comparing LLMS to task-specific models performance on the bar exam kentlaw.iit.edu OpenAI API latency promptlayer.com
51. LLM-POWERED NLP IN PRACTICE LLM-powered collaborative data development environment 7
52. LLM-POWERED NLP IN PRACTICE LLM-powered collaborative data development environment 7 Assign labeling tasks to LLMs "
53. LLM-POWERED NLP IN PRACTICE LLM-powered collaborative data development environment 7 Assign labeling tasks to LLMs " Review label decisions, correct errors ;
54. LLM-POWERED NLP IN PRACTICE LLM-powered collaborative data development environment 7 Assign labeling tasks to LLMs " Review label decisions, correct errors ; Tune prompts and compare LLMs empirically 6
55. LLM-POWERED NLP IN PRACTICE LLM-powered collaborative data development environment 7 Assign labeling tasks to LLMs " Review label decisions, correct errors ; Tune prompts and compare LLMs empirically 6 Build data sets to train and evaluate e icient, production-ready pipelines +
56. 8 PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS
57. 8 correct mistakes PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS
58. PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS correct mistakes
59. add correct answer to prompt to tune it PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS correct mistakes
60. PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS
61. PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS query LLM and parse response
62. PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS query LLM and parse response tune prompt if needed
63. GITHUB.COM/EXPLOSION/SPACY-LLM TOWARDS STRUCTURED DATA Prompt Template 2 LLM London is bigger than Berlin LOCATION: London, Berlin LOCATION
64. GITHUB.COM/EXPLOSION/SPACY-LLM TOWARDS STRUCTURED DATA Prompt Template 2 LLM London is bigger than Berlin LOCATION: London, Berlin LOCATION
65. SPECIFIC Task-specific models powered by LLMS IS BETTER.
66. SMALLER & FASTER Task-specific models powered by LLMS IS BETTER.
67. PRIVATE Task-specific models powered by LLMS IS BETTER.
68. BETTER Task-specific models powered by LLMS IS BETTER.
69. THANK YOU! - Explosion – explosion.ai < spaCy – spacy.io ✨ Prodigy – prodigy.ai = Twitter – @_inesmontani > Mastodon – @ines@sigmoid.social ? LinkedIn