spaCy meets LLMs: Using Generative AI for Structured Data

Large Language Models (LLMs) have enormous potential, but also challenge existing workflows in industry that require modularity, transparency and structured data. In this talk, I'll present pragmatic and practical approaches for how to use the latest generative models beyond just chat bots. I'll dive deeper into spaCy's LLM integration, which lets you plug in open-source and proprietary models and provides a robust framework for extracting structured information from text, distilling large models into smaller task-specific components, and closing the gap between prototype and production.

Transcript

1. meets LLMs Structured Data USING GENERATIVE AI FOR Ines Montani Explosion
2. Open-source library for industrial-strength natural language processing spacy.io 225m+ DOWNLOADS github.com/explosion/spaCy
3. Open-source library for industrial-strength natural language processing spacy.io 225m+ DOWNLOADS github.com/explosion/spaCy ChatGPT can write spaCy code!
4. designed for production
5. extensible and programmable designed for production
6. carefully designed and consistent API extensible and programmable designed for production
7. carefully designed and consistent API extensible and programmable serializable data structures designed for production
8. carefully designed and consistent API extensible and programmable serializable data structures good error handling designed for production
9. carefully designed and consistent API extensible and programmable serializable data structures good error handling pipeline approach to combine techniques and share data designed for production
10. carefully designed and consistent API extensible and programmable serializable data structures good error handling pipeline approach to combine techniques and share data designed for production “just works” pre-configured solutions
11. Prototype HOW TO AVOID THE Plateau
12. Prototype HOW TO AVOID THE Plateau standardize inputs and outputs
13. Prototype HOW TO AVOID THE Plateau standardize inputs and outputs start with evaluation
14. Prototype HOW TO AVOID THE Plateau standardize inputs and outputs start with evaluation assess utility, not just accuracy explosion.ai/blog/applied-nlp-thinking
15. Prototype HOW TO AVOID THE Plateau standardize inputs and outputs start with evaluation assess utility, not just accuracy explosion.ai/blog/applied-nlp-thinking work iteratively
16. Prototype HOW TO AVOID THE Plateau standardize inputs and outputs start with evaluation assess utility, not just accuracy explosion.ai/blog/applied-nlp-thinking consider structure and properties of language work iteratively
17. Doc classic NLP pipeline
18. Doc Entity Recognizer classic NLP pipeline
19. Doc Entity Recognizer Text Categorizer classic NLP pipeline
20. Doc Entity Recognizer Text Categorizer ... classic NLP pipeline
21. Doc Entity Recognizer Text Categorizer ... classic NLP pipeline spacy.io/usage/large-language-models Doc ... llm LLM-powered pipeline
22. Doc Entity Recognizer Text Categorizer ... classic NLP pipeline spacy.io/usage/large-language-models Doc ... llm LLM-powered pipeline GPT-4 Falcon
23. Doc llm spacy.io/usage/large-language-models
24. Doc llm spacy.io/usage/large-language-models Prompt Template
25. Doc llm spacy.io/usage/large-language-models Prompt Template Model
26. Doc llm spacy.io/usage/large-language-models Prompt Template Model Response Parser
27. Doc llm spacy.io/usage/large-language-models Prompt Template Model Response Parser Structured Attributes
28. Doc llm spacy.io/usage/large-language-models Prompt Template Model Response Parser Structured Attributes unified, model- agnostic API
29. Doc llm spacy.io/usage/large-language-models Prompt Template Model Response Parser Structured Attributes unified, model- agnostic API con fi g.cfg
30. Doc llm ... other components spacy.io/usage/large-language-models Prompt Template Model Response Parser Structured Attributes unified, model- agnostic API con fi g.cfg
31. Doc llm ... other components LLM trained model rules spacy.io/usage/large-language-models Prompt Template Model Response Parser Structured Attributes unified, model- agnostic API con fi g.cfg
32. Doc llm ... other components LLM trained model rules spacy.io/usage/large-language-models Prompt Template Model Response Parser Structured Attributes unified, model- agnostic API con fi g.cfg Entity Linking, Summarization, Entity Recognition, Span Categorization, Relation Extraction, Text Categorization, Sentiment Analysis, Translation
33. Doc llm prototype
34. Doc llm prototype continuous evaluation baseline
35. Doc llm human-in- the-loop distillation prototype continuous evaluation baseline
36. Doc llm Entity Recognizer Text Categorizer … human-in- the-loop distillation prototype continuous evaluation baseline
37. production Doc Doc llm Entity Recognizer Text Categorizer … human-in- the-loop distillation prototype continuous evaluation baseline
38. production Doc Doc llm Entity Recognizer Text Categorizer … human-in- the-loop distillation prototype continuous evaluation baseline distilled model
39. Thank you! Explosion spaCy Prodigy Twitter Mastodon Bluesky explosion.ai spacy.io prodigy.ai @_inesmontani @ines@sigmoid.social @inesmontani.bsky.social LinkedIn