10 Years of Open Source: Navigating the Next AI Revolution

Slide 1

Slide 1 text

Ines Montani Explosion

Slide 2

Slide 2 text

Open-source library for industrial-strength natural language processing spacy.io 255m+ downloads

Slide 3

Slide 3 text

Open-source library for industrial-strength natural language processing spacy.io ChatGPT can write spaCy code! 255m+ downloads

Slide 4

Slide 4 text

Modern scriptable annotation tool for machine learning developers prodigy.ai 900+ companies 10k+ users

Slide 5

Slide 5 text

Alex Smith Developer Kim Miller Analyst GPT-4 API Modern scriptable annotation tool for machine learning developers prodigy.ai 900+ companies 10k+ users

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

^ first commit to spaCy

Slide 8

Slide 8 text

^ first commit to spaCy spaCy is first released spacy.io

Slide 9

Slide 9 text

OUR DEVELOPMENT PHILOSOPHY OUR DEVELOPMENT PHILOSOPHY “Let T h e m W rite Code” spacy.fyi/ltwc

Slide 10

Slide 10 text

OUR DEVELOPMENT PHILOSOPHY OUR DEVELOPMENT PHILOSOPHY “Let T h e m W rite Code” spacy.fyi/ltwc Good tools help people do their work. You don’t have to do their work for them.

Slide 11

Slide 11 text

OUR DEVELOPMENT PHILOSOPHY OUR DEVELOPMENT PHILOSOPHY “Let T h e m W rite Code” spacy.fyi/ltwc Good tools help people do their work. You don’t have to do their work for them. ["go", "swim"]

Slide 12

Slide 12 text

OUR DEVELOPMENT PHILOSOPHY OUR DEVELOPMENT PHILOSOPHY “Let T h e m W rite Code” spacy.fyi/ltwc Good tools help people do their work. You don’t have to do their work for them. ["go", "swim"] spaCy

Slide 13

Slide 13 text

OUR DEVELOPMENT PHILOSOPHY OUR DEVELOPMENT PHILOSOPHY “Let T h e m W rite Code” spacy.fyi/ltwc Good tools help people do their work. You don’t have to do their work for them. You can reinvent the wheel, but don’t try to reinvent the road. ["go", "swim"] spaCy

Slide 14

Slide 14 text

^ first commit to spaCy spaCy is first released spacy.io

Slide 15

Slide 15 text

^ first commit to spaCy spaCy is first released spacy.io everyone gets excited about chat bots

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

“knocker-uppers”

Slide 18

Slide 18 text

The Window K nocking Machine Tes t ines.io/blog/window-knocking-machine-test “knocker-uppers”

Slide 19

Slide 19 text

The Window K nocking Machine Tes t ines.io/blog/window-knocking-machine-test Are you designing a window-knocking machine or an alarm clock? “knocker-uppers”

Slide 20

Slide 20 text

Hello, I ’ m Toni ’ s virtual assistant and I help schedule appointments. Do you have time at 1pm on Monday? No, but Tuesday would work for me. Okay, please confirm: Tuesday at 1pm? 1pm is unideal but 3pm would work. Toni doesn ’ t have availability at 3pm but I could offer a slot at 4pm or 5 : 30pm. Which time zone is this by the way? I ’ m in CET.

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

^ first commit to spaCy spaCy is first released spacy.io everyone gets excited about chat bots

Slide 24

Slide 24 text

^ first commit to spaCy spaCy is first released spacy.io deep learning is widely adopted everyone gets excited about chat bots

Slide 25

Slide 25 text

Software 1.0 Software 1.0 📄 code 💾 program compiler

Slide 26

Slide 26 text

Software 1.0 Software 1.0 📄 code 💾 program compiler Software 2.0 Software 2.0 📊 data 🔮 model algorithm

Slide 27

Slide 27 text

Software 1.0 Software 1.0 📄 code 💾 program compiler Software 2.0 Software 2.0 📊 data 🔮 model algorithm ✅ tests 📈 evaluation

Slide 28

Slide 28 text

Software 1.0 Software 1.0 📄 code 💾 program compiler Software 2.0 Software 2.0 📊 data 🔮 model algorithm ✅ tests 📈 evaluation refactoring refactoring

Slide 29

Slide 29 text

Software 1.0 Software 1.0 📄 code 💾 program compiler Software 2.0 Software 2.0 📊 data 🔮 model algorithm ✅ tests 📈 evaluation refactoring refactoring iteration iteration

Slide 30

Slide 30 text

Prodigy is first released prodigy.ai

Slide 31

Slide 31 text

language model pre-training works ^ ^ Prodigy is first released prodigy.ai

Slide 32

Slide 32 text

language model pre-training works ^ ^ Prodigy is first released prodigy.ai few-shot in-context learning works ^ ^

Slide 33

Slide 33 text

language model pre-training works ^ ^ Prodigy is first released prodigy.ai few-shot in-context learning works ^ ^

Slide 34

Slide 34 text

spaCy v3 is first released

Slide 35

Slide 35 text

i U se cases i n industr y generative tasks 📖 single/multi-doc summarization 🧮 reasoning ✅ problem solving ✍ paraphrasing 🖼 style transfer ⁉ question answering predictive tasks 🔖 entity recognition 🔗 relation extraction 👫 coreference resolution 🧬 grammar & morphology 🎯 semantic parsing 💬 discourse structure 📚 text classification

Slide 36

Slide 36 text

Slide 37

Slide 37 text

spaCy v3 is first released

Slide 38

Slide 38 text

spaCy v3 is first released in-context learning gains traction

Slide 39

Slide 39 text

human-facing systems machine-facing models ChatGPT GPT-4 A I products are m ore t h an jus t a model

Slide 40

Slide 40 text

human-facing systems machine-facing models ChatGPT GPT-4 most important di erentiation is product, not just technology A I products are m ore t h an jus t a model

Slide 41

Slide 41 text

human-facing systems machine-facing models ChatGPT GPT-4 UI / UX marketing customization most important di erentiation is product, not just technology A I products are m ore t h an jus t a model

Slide 42

Slide 42 text

human-facing systems machine-facing models ChatGPT GPT-4 UI / UX marketing customization most important di erentiation is product, not just technology swappable components based on research, impacts are quantifiable A I products are m ore t h an jus t a model

Slide 43

Slide 43 text

human-facing systems machine-facing models ChatGPT GPT-4 UI / UX marketing customization speed accuracy latency cost most important di erentiation is product, not just technology swappable components based on research, impacts are quantifiable A I products are m ore t h an jus t a model

Slide 44

Slide 44 text

human-facing systems machine-facing models ChatGPT GPT-4 UI / UX marketing customization speed accuracy latency cost But what about the data? most important di erentiation is product, not just technology swappable components based on research, impacts are quantifiable A I products are m ore t h an jus t a model

Slide 45

Slide 45 text

human-facing systems machine-facing models ChatGPT GPT-4 UI / UX marketing customization speed accuracy latency cost But what about the data? User data is an advantage for product, not the foundation for machine-facing tasks. most important di erentiation is product, not just technology swappable components based on research, impacts are quantifiable A I products are m ore t h an jus t a model

Slide 46

Slide 46 text

human-facing systems machine-facing models ChatGPT GPT-4 UI / UX marketing customization speed accuracy latency cost But what about the data? User data is an advantage for product, not the foundation for machine-facing tasks. You don’t need specific data to gain general knowledge. most important di erentiation is product, not just technology swappable components based on research, impacts are quantifiable A I products are m ore t h an jus t a model

Slide 47

Slide 47 text

spaCy v3 is first released in-context learning gains traction

Slide 48

Slide 48 text

spacy-llm is first released github.com/explosion/spacy-llm spaCy v3 is first released in-context learning gains traction

Slide 49

Slide 49 text

task-specific output 💬 prompt 📖 text LLM spacy.io/usage/large-language-models spac y -llm

Slide 50

Slide 50 text

task-specific output 💬 prompt 📖 text LLM prompt model & transform output to structured data spacy.io/usage/large-language-models spac y -llm

Slide 51

Slide 51 text

task-specific output 💬 prompt 📖 text LLM prompt model & transform output to structured data spacy.io/usage/large-language-models spac y -llm config.cfg Structured Data {} LLM Text

Slide 52

Slide 52 text

task-specific output 💬 prompt 📖 text LLM prompt model & transform output to structured data spacy.io/usage/large-language-models unified, model-agnostic API spac y -llm config.cfg Structured Data {} LLM Text

Slide 53

Slide 53 text

Slide 54

Slide 54 text

spacy-llm is first released github.com/explosion/spacy-llm spaCy v3 is first released in-context learning gains traction

Slide 55

Slide 55 text

spacy-llm is first released github.com/explosion/spacy-llm spaCy v3 is first released in-context learning gains traction LLMs and Generative AI fully hit the mainstream ChatGPT ⏺ ⏺ ⏺

Slide 56

Slide 56 text

E cono m ies of scale of scale output costs

Slide 57

Slide 57 text

E cono m ies of scale of scale output costs OpenAI Google

Slide 58

Slide 58 text

E cono m ies of scale of scale output costs OpenAI Google access to talent, compute etc.

Slide 59

Slide 59 text

E cono m ies of scale of scale output costs OpenAI Google access to talent, compute etc. API request batching

Slide 60

Slide 60 text

E cono m ies of scale of scale output costs OpenAI Google high tra ff ic 💧 💧 💧 💧 💧 💧 💧 💧 low tra ff ic batch 💧 💧 💧 💧 💧 💧 💧 💧 … access to talent, compute etc. API request batching

Slide 61

Slide 61 text

E cono m ies of scale of scale output costs OpenAI Google you 🤠 high tra ff ic 💧 💧 💧 💧 💧 💧 💧 💧 low tra ff ic batch 💧 💧 💧 💧 💧 💧 💧 💧 … access to talent, compute etc. API request batching

Slide 62

Slide 62 text

human-in-the-loop distillation is promising prodigy.fyi/distillation

Slide 63

Slide 63 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation LLM

Slide 64

Slide 64 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM

Slide 65

Slide 65 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting

Slide 66

Slide 66 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting

Slide 67

Slide 67 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting transfer learning CO M PO N EN T

Slide 68

Slide 68 text

in the loop H uma n explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting transfer learning CO M PO N EN T distilled model

Slide 69

Slide 69 text

99% 99% Case Stud y : S&P Global • real-time commodities trading insights by extracting structured attributes 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 70

Slide 70 text

99% 99% Case Stud y : S&P Global • real-time commodities trading insights by extracting structured attributes • high-security environment 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 71

Slide 71 text

99% 99% Case Stud y : S&P Global • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 72

Slide 72 text

99% 99% Case Stud y : S&P Global • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 73

Slide 73 text

99% 99% Case Stud y : S&P Global • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop • 8 market pipelines in production 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 74

Slide 74 text

99% 99% Case Stud y : S&P Global • real-time commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop • 8 market pipelines in production 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities

Slide 75

Slide 75 text

human-in-the-loop distillation is promising prodigy.fyi/distillation

Slide 76

Slide 76 text

human-in-the-loop distillation is promising prodigy.fyi/distillation everyone is excited about chat bots again

Slide 77

Slide 77 text

? ines.io/blog/window-knocking-machine-test

Slide 78

Slide 78 text

What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ ? ines.io/blog/window-knocking-machine-test

Slide 79

Slide 79 text

What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation ? ines.io/blog/window-knocking-machine-test

Slide 80

Slide 80 text

2023 Year Services Type ACME Inc. FooBar GmbH NLPCorp XKCD Ltd. Python AG 432,032 82,000 1,500 193,000 91,320 $ 2,625,032 Clients (28) Revenue What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation ? ines.io/blog/window-knocking-machine-test

Slide 81

Slide 81 text

2023 Year Services Type ACME Inc. FooBar GmbH NLPCorp XKCD Ltd. Python AG 432,032 82,000 1,500 193,000 91,320 $ 2,625,032 Clients (28) Revenue A I still needs produc t decisions! Kim Miller Analyst What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation ? ines.io/blog/window-knocking-machine-test

Slide 82

Slide 82 text

human-in-the-loop distillation is promising prodigy.fyi/distillation everyone is excited about chat bots again

Slide 83

Slide 83 text

Explosion goes back to independent-minded and self-su ff icient explosion.ai/blog/ back-to-our-roots human-in-the-loop distillation is promising prodigy.fyi/distillation everyone is excited about chat bots again

Slide 84

Slide 84 text

Slide 85

Slide 85 text

Cycle A doptio n

Slide 86

Slide 86 text

Cycle A doptio n rules and conditional logic

Slide 87

Slide 87 text

Cycle A doptio n rules and conditional logic applied workflow

Slide 88

Slide 88 text

Cycle A doptio n rules and conditional logic linear models applied workflow

Slide 89

Slide 89 text

Cycle A doptio n rules and conditional logic linear models applied workflow applied workflow

Slide 90

Slide 90 text

Cycle A doptio n rules and conditional logic linear models applied workflow applied workflow combine new techniques with established workflows

Slide 91

Slide 91 text

Cycle A doptio n rules and conditional logic deep learning linear models applied workflow applied workflow combine new techniques with established workflows

Slide 92

Slide 92 text

Cycle A doptio n rules and conditional logic deep learning linear models chat bots applied workflow applied workflow combine new techniques with established workflows

Slide 93

Slide 93 text

Cycle A doptio n rules and conditional logic deep learning linear models chat bots applied workflow applied workflow applied workflow combine new techniques with established workflows

Slide 94

Slide 94 text

Cycle A doptio n rules and conditional logic deep learning transfer learning linear models chat bots applied workflow applied workflow applied workflow combine new techniques with established workflows

Slide 95

Slide 95 text

Cycle A doptio n rules and conditional logic deep learning transfer learning linear models chat bots trans- formers applied workflow applied workflow applied workflow combine new techniques with established workflows

Slide 96

Slide 96 text

Cycle A doptio n rules and conditional logic deep learning transfer learning linear models chat bots trans- formers applied workflow applied workflow applied workflow applied workflow combine new techniques with established workflows

Slide 97

Slide 97 text

Cycle A doptio n rules and conditional logic deep learning transfer learning in-context learning linear models chat bots trans- formers applied workflow applied workflow applied workflow applied workflow combine new techniques with established workflows

Slide 98

Slide 98 text

Cycle A doptio n rules and conditional logic deep learning transfer learning in-context learning linear models chat bots LLMs and GenAI trans- formers applied workflow applied workflow applied workflow applied workflow combine new techniques with established workflows

Slide 99

Slide 99 text

Slide 100

Slide 100 text

Summar y NAVIGATING AI & NLP NAVIGATING AI & NLP

Slide 101

Slide 101 text

Summar y NAVIGATING AI & NLP NAVIGATING AI & NLP Think beyond chat bots or human-shaped tasks. You don’t want to build a “window-knocking machine”.

Slide 102

Slide 102 text

Summar y NAVIGATING AI & NLP NAVIGATING AI & NLP Think beyond chat bots or human-shaped tasks. You don’t want to build a “window-knocking machine”. Structured Data {} Focus on your application. Consider what it really needs and let your data guide you.

Slide 103

Slide 103 text

Summar y NAVIGATING AI & NLP NAVIGATING AI & NLP Think beyond chat bots or human-shaped tasks. You don’t want to build a “window-knocking machine”. Stay ambitious. Don’t compromise on best practices, e iciency and privacy. Structured Data {} Focus on your application. Consider what it really needs and let your data guide you.

Slide 104

Slide 104 text

Summar y NAVIGATING AI & NLP NAVIGATING AI & NLP Think beyond chat bots or human-shaped tasks. You don’t want to build a “window-knocking machine”. Stay ambitious. Don’t compromise on best practices, e iciency and privacy. LLM Keep filling up your toolbox. Know the techniques you have available and apply the best ones to get the job done. Structured Data {} Focus on your application. Consider what it really needs and let your data guide you.

Slide 105

Slide 105 text

Explosion spaCy Prodigy Twitter Mastodon Bluesky explosion.ai spacy.io prodigy.ai @_inesmontani @[email protected] @inesmontani.bsky.social LinkedIn