Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Large Language Models: From Prototype to Production (PyData London keynote)

Large Language Models: From Prototype to Production (PyData London keynote)

Large Language Models (LLMs) have shown some impressive capabilities and their impact is the topic of the moment. What will the future look like? Are we going to only talk to bots? Will prompting replace programming? Or are we just hyping up unreliable parrots and burning money? In this talk, I'll present visions for NLP in the age of LLMs and a pragmatic, practical approach for how to use Large Language Models to ship more successful NLP projects from prototype to production today.

Ines Montani

June 03, 2023
Tweet

More Decks by Ines Montani

Other Decks in Technology

Transcript

  1. Ines Montani
    Explosion
    LARGE LANGUAGE
    LARGE LANGUAGE MODELS ✨ CHATGPT " ARTIFICIAL INTELLIGENCE # MACHINE LEARNING ✨
    PROTOTYPE TO PRODUCTION
    MODELS FROM
    LLAMA $ NATURAL LANGUAGE PROCESSING %
    ✨ OPEN SOURCE & PYTHON ' PROMPT ENGINEERING ⚙ ZERO-SHOT LEARNING ) GPT-4
    EVALUATION * COPILOT + GENERATIVE AI ,
    Ines Montani - Explosion

    View full-size slide

  2. SPACY
    SPACY.IO & @SPACY_IO ✍ SPACY.TV / GITHUB.COM/EXPLOSION/SPACY
    Open-source library for
    industrial-strength Natural
    Language Processing
    140m+
    downloads

    View full-size slide

  3. SPACY
    SPACY.IO & @SPACY_IO ✍ SPACY.TV / GITHUB.COM/EXPLOSION/SPACY
    Open-source library for
    industrial-strength Natural
    Language Processing
    140m+
    downloads
    ChatGPT can write spaCy code!

    View full-size slide

  4. PRODIGY
    Modern scriptable annotation
    tool for machine learning
    developers
    PRODIGY.AI & GITHUB.COM/EXPLOSION/PRODIGY-RECIPES
    8k+
    users
    700+
    companies

    View full-size slide

  5. PRODIGY
    Modern scriptable annotation
    tool for machine learning
    developers
    PRODIGY.AI & GITHUB.COM/EXPLOSION/PRODIGY-RECIPES
    8k+
    users
    700+
    companies

    View full-size slide

  6. THE HISTORY OF FUTURE TECHNOLOGY

    View full-size slide

  7. THE HISTORY OF FUTURE TECHNOLOGY
    How people in 1900
    imagined the year 2000

    View full-size slide

  8. THE HISTORY OF FUTURE TECHNOLOGY
    How people in 1900
    imagined the year 2000

    View full-size slide

  9. THE HISTORY OF FUTURE TECHNOLOGY

    View full-size slide

  10. THE HISTORY OF FUTURE TECHNOLOGY
    manual calculation vs. calculator

    View full-size slide

  11. THE HISTORY OF FUTURE TECHNOLOGY
    manual calculation vs. calculator

    View full-size slide

  12. THE HISTORY OF FUTURE TECHNOLOGY
    “knocker-uppers” vs. alarm clock
    manual calculation vs. calculator

    View full-size slide

  13. THE HISTORY OF FUTURE TECHNOLOGY

    View full-size slide

  14. THE HISTORY OF FUTURE TECHNOLOGY
    human assistant
    vs. calendar apps
    Calendly
    Fantastical

    View full-size slide

  15. THE HISTORY OF FUTURE TECHNOLOGY
    human assistant
    vs. calendar apps
    Calendly
    Fantastical
    WHAT’S NEXT?

    View full-size slide

  16. NLP IN THE AGE OF LLMS

    View full-size slide

  17. NLP IN THE AGE OF LLMS
    SQL
    is all you need
    dialogue
    is all you need
    0 %

    View full-size slide

  18. NLP IN THE AGE OF LLMS
    SQL
    is all you need
    dialogue
    is all you need
    0 %
    lots of humans
    is all you need
    prompting
    is all you need
    1 "

    View full-size slide

  19. VISION #1 dialogue
    is all you need
    %
    2 LLM
    3 user
    actions or information
    natural language input

    View full-size slide

  20. VISION #1 dialogue
    is all you need
    %
    2 LLM
    3 user
    actions or information
    natural language input
    LLM is the system
    and needs to manage
    the whole interaction

    View full-size slide

  21. VISION #2 prompting
    is all you need
    "
    2 LLM
    4 text % prompt
    5 system
    3 user
    6 structured data

    View full-size slide

  22. VISION #2 prompting
    is all you need
    "
    2 LLM
    4 text % prompt
    5 system
    3 user
    LLM replaces the
    specific ML model
    6 structured data

    View full-size slide

  23. VISION #3 modern
    practical NLP
    -
    7 developer 8 code
    2 LLM 9 training data
    5 system
    3 user
    6 structured data
    ⚙ ML system

    View full-size slide

  24. VISION #3 modern
    practical NLP
    -
    7 developer 8 code
    2 LLM 9 training data
    5 system
    3 user
    6 structured data
    ⚙ ML system
    LLM helps with
    building the pipeline

    View full-size slide

  25. VISION #3 modern
    practical NLP
    -
    7 developer 8 code
    2 LLM 9 training data
    5 system
    3 user
    6 structured data
    ⚙ ML system
    LLM helps with
    building the pipeline

    View full-size slide

  26. VISION #3 modern
    practical NLP
    -
    7 developer 8 code
    2 LLM 9 training data
    5 system
    3 user
    6 structured data
    ⚙ ML system
    LLM helps with
    building the pipeline

    View full-size slide

  27. COMPANY
    COMPANY
    MONEY
    INVESTOR
    “Hooli raises $5m to
    revolutionize search,
    led by ACME Ventures”
    5923214
    1681056
    CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA
    Database

    View full-size slide

  28. Database
    CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA

    View full-size slide

  29. Database
    CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA
    named entity recognition

    View full-size slide

  30. Database
    CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA
    named entity recognition
    entity disambiguation

    View full-size slide

  31. Database
    CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA
    named entity recognition
    entity disambiguation
    custom database lookup

    View full-size slide

  32. Database
    CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA
    named entity recognition
    entity disambiguation
    custom database lookup
    currency normalization

    View full-size slide

  33. Database
    CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA
    named entity recognition
    entity disambiguation
    custom database lookup
    currency normalization
    entity relation extraction

    View full-size slide

  34. Database
    CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA
    named entity recognition
    entity disambiguation
    custom database lookup
    currency normalization
    entity relation extraction
    6 structured data
    2 LLM
    7 developer
    quick prototype

    View full-size slide

  35. Database
    CLASSIC NLP PROBLEM: EXTRACT STRUCTURED DATA
    named entity recognition
    entity disambiguation
    custom database lookup
    currency normalization
    entity relation extraction
    6 structured data
    2 LLM
    7 developer
    quick prototype

    fast to develop,
    slow to run, hard
    to improve

    View full-size slide

  36. NLP IN THE AGE OF LLMS
    SQL
    is all you need
    dialogue
    is all you need
    0 %
    lots of humans
    is all you need
    prompting
    is all you need
    1 "
    modern
    practical
    NLP
    -

    View full-size slide

  37. NLP IN THE AGE OF LLMS
    SQL
    is all you need
    dialogue
    is all you need
    0 %
    lots of humans
    is all you need
    prompting
    is all you need
    1 "
    modern
    practical
    NLP
    -
    structured data

    View full-size slide

  38. NLP IN THE AGE OF LLMS
    SQL
    is all you need
    dialogue
    is all you need
    0 %
    lots of humans
    is all you need
    prompting
    is all you need
    1 "
    modern
    practical
    NLP
    -
    structured data
    humans in
    the loop

    View full-size slide

  39. NLP IN THE AGE OF LLMS
    SQL
    is all you need
    dialogue
    is all you need
    0 %
    lots of humans
    is all you need
    prompting
    is all you need
    1 "
    modern
    practical
    NLP
    -
    structured data fast prototyping
    humans in
    the loop

    View full-size slide

  40. NLP IN THE AGE OF LLMS
    SQL
    is all you need
    dialogue
    is all you need
    0 %
    lots of humans
    is all you need
    prompting
    is all you need
    1 "
    modern
    practical
    NLP
    -
    structured data fast prototyping
    humans in
    the loop
    powered by
    open source

    View full-size slide

  41. NLP IN THE AGE OF LLMS
    SQL
    is all you need
    dialogue
    is all you need
    0 %
    lots of humans
    is all you need
    prompting
    is all you need
    1 "
    modern
    practical
    NLP
    -
    structured data fast prototyping
    humans in
    the loop
    powered by
    open source
    conversational
    and graphical
    interfaces

    View full-size slide

  42. NLP IN THE AGE OF LLMS
    SQL
    is all you need
    dialogue
    is all you need
    0 %
    lots of humans
    is all you need
    prompting
    is all you need
    1 "
    modern
    practical
    NLP
    -
    structured data fast prototyping
    humans in
    the loop
    powered by
    open source
    robust
    evaluation
    conversational
    and graphical
    interfaces

    View full-size slide

  43. A CASE FOR LLM PRAGMATISM
    EXPLOSION.AI/BLOG/AGAINST-LLM-MAXIMALISM
    NOOO YOU CAN'T JUST MIX
    UP ALL THE STEPS OF YOUR TASK
    AND ASK AN LLM TO DO IT ALL.
    HOW WILL YOU EVER MAKE A RELIABLE
    AND EXTENSIBLE SYSTEM THAT WAY?
    HAHA LLM GO BRRR

    View full-size slide

  44. A CASE FOR LLM PRAGMATISM
    EXPLOSION.AI/BLOG/AGAINST-LLM-MAXIMALISM
    NOOO YOU CAN'T JUST MIX
    UP ALL THE STEPS OF YOUR TASK
    AND ASK AN LLM TO DO IT ALL.
    HOW WILL YOU EVER MAKE A RELIABLE
    AND EXTENSIBLE SYSTEM THAT WAY?
    HAHA LLM GO BRRR
    avoid coupling prediction tasks
    to arbitrary business logic

    View full-size slide

  45. A CASE FOR LLM PRAGMATISM
    EXPLOSION.AI/BLOG/AGAINST-LLM-MAXIMALISM
    NOOO YOU CAN'T JUST MIX
    UP ALL THE STEPS OF YOUR TASK
    AND ASK AN LLM TO DO IT ALL.
    HOW WILL YOU EVER MAKE A RELIABLE
    AND EXTENSIBLE SYSTEM THAT WAY?
    HAHA LLM GO BRRR
    avoid coupling prediction tasks
    to arbitrary business logic
    design modular solutions

    View full-size slide

  46. A CASE FOR LLM PRAGMATISM
    EXPLOSION.AI/BLOG/AGAINST-LLM-MAXIMALISM
    NOOO YOU CAN'T JUST MIX
    UP ALL THE STEPS OF YOUR TASK
    AND ASK AN LLM TO DO IT ALL.
    HOW WILL YOU EVER MAKE A RELIABLE
    AND EXTENSIBLE SYSTEM THAT WAY?
    HAHA LLM GO BRRR
    avoid coupling prediction tasks
    to arbitrary business logic
    design modular solutions
    prototype modules with LLMs

    View full-size slide

  47. A CASE FOR LLM PRAGMATISM
    EXPLOSION.AI/BLOG/AGAINST-LLM-MAXIMALISM
    NOOO YOU CAN'T JUST MIX
    UP ALL THE STEPS OF YOUR TASK
    AND ASK AN LLM TO DO IT ALL.
    HOW WILL YOU EVER MAKE A RELIABLE
    AND EXTENSIBLE SYSTEM THAT WAY?
    HAHA LLM GO BRRR
    avoid coupling prediction tasks
    to arbitrary business logic
    design modular solutions
    prototype modules with LLMs
    evaluate alternatives

    View full-size slide

  48. TRADE-OFFS
    performance on
    the bar exam
    kentlaw.iit.edu

    View full-size slide

  49. TRADE-OFFS
    performance on
    the bar exam
    kentlaw.iit.edu
    OpenAI API
    latency
    promptlayer.com

    View full-size slide

  50. TRADE-OFFS
    Supervised 1 LLM 2
    accuracy words/s accuracy words/s
    Textcat on SST2
    (Stanford Sentiment Treebank)
    0.9 4019 0.9 <100
    Textcat on Banking77
    (intent recognition)
    0.9 3234 0.7 <100
    NER on AnEm
    (anatomical entity mentions)
    0.7 5146 0.1 <100
    1. RoBERTa-base with spaCy, 2. text-davinci-003 zero-shot
    ongoing experiments
    comparing LLMS to
    task-specific models
    performance on
    the bar exam
    kentlaw.iit.edu
    OpenAI API
    latency
    promptlayer.com

    View full-size slide

  51. LLM-POWERED NLP IN PRACTICE
    LLM-powered collaborative data development environment
    7

    View full-size slide

  52. LLM-POWERED NLP IN PRACTICE
    LLM-powered collaborative data development environment
    7
    Assign labeling tasks to LLMs
    "

    View full-size slide

  53. LLM-POWERED NLP IN PRACTICE
    LLM-powered collaborative data development environment
    7
    Assign labeling tasks to LLMs
    "
    Review label decisions, correct errors
    ;

    View full-size slide

  54. LLM-POWERED NLP IN PRACTICE
    LLM-powered collaborative data development environment
    7
    Assign labeling tasks to LLMs
    "
    Review label decisions, correct errors
    ;
    Tune prompts and compare LLMs empirically
    6

    View full-size slide

  55. LLM-POWERED NLP IN PRACTICE
    LLM-powered collaborative data development environment
    7
    Assign labeling tasks to LLMs
    "
    Review label decisions, correct errors
    ;
    Tune prompts and compare LLMs empirically
    6
    Build data sets to train and evaluate e icient, production-ready pipelines
    +

    View full-size slide

  56. 8
    PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS

    View full-size slide

  57. 8
    correct
    mistakes
    PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS

    View full-size slide

  58. PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS
    correct
    mistakes

    View full-size slide

  59. add correct
    answer to prompt
    to tune it
    PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS
    correct
    mistakes

    View full-size slide

  60. PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS

    View full-size slide

  61. PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS
    query LLM and
    parse response

    View full-size slide

  62. PRODIGY.AI/FEATURES/LARGE-LANGUAGE-MODELS
    query LLM and
    parse response
    tune prompt
    if needed

    View full-size slide

  63. GITHUB.COM/EXPLOSION/SPACY-LLM
    TOWARDS STRUCTURED DATA
    Prompt Template
    2 LLM
    London is bigger
    than Berlin
    LOCATION:
    London, Berlin
    LOCATION

    View full-size slide

  64. GITHUB.COM/EXPLOSION/SPACY-LLM
    TOWARDS STRUCTURED DATA
    Prompt Template
    2 LLM
    London is bigger
    than Berlin
    LOCATION:
    London, Berlin
    LOCATION

    View full-size slide

  65. SPECIFIC
    Task-specific models
    powered by LLMS
    IS BETTER.

    View full-size slide

  66. SMALLER & FASTER
    Task-specific models
    powered by LLMS
    IS BETTER.

    View full-size slide

  67. PRIVATE
    Task-specific models
    powered by LLMS
    IS BETTER.

    View full-size slide

  68. BETTER
    Task-specific models
    powered by LLMS
    IS BETTER.

    View full-size slide

  69. THANK YOU!
    - Explosion – explosion.ai
    < spaCy – spacy.io
    ✨ Prodigy – prodigy.ai
    = Twitter – @_inesmontani
    > Mastodon – @[email protected]
    ? LinkedIn

    View full-size slide