Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Human in the Loop - NUS Chat on GPT

wing.nus
April 18, 2023

Human in the Loop - NUS Chat on GPT

2023 April 18 @ UTown Auditorium 1, NUS, Singapore
Rishabh Anand

Video Available at: https://www.youtube.com/watch?v=WupvFC_zaZU&t=1768s

Event Website: https://wing-nus.github.io/chatongpt/

wing.nus

April 18, 2023
Tweet

More Decks by wing.nus

Other Decks in Education

Transcript

  1. Chat on GPT – 18 April 2023
    Rishabh Anand
    @rishabh16_
    Human in the Loop

    View full-size slide

  2. Chat on GPT – 18 April 2023
    The GPT Series
    GPT → Generative Pretrained Transformer
    GPT GPT-2 GPT-3 ChatGPT(GPT-3.5) GPT-4
    2018
    Improving Language
    Understanding by
    Generative Pre-Training
    Language Models are
    Unsupervised Multi-
    task Learners
    Training language
    models to follow
    instructions with human
    feedback *
    Language Models are
    Few-Shot Learners
    2019 2020 2022 2023
    * GPT-3.5 is built on top of InstructGPT with a different data collection setup
    (technical report)
    Rapid growth …

    View full-size slide

  3. Chat on GPT – 18 April 2023
    Reinforcement
    Learning from
    Human Feedback

    View full-size slide

  4. Chat on GPT – 18 April 2023
    (Large) Language Models
    ● Language Models (like GPT-X),
    ○ are chaotic
    ○ model a “giant mass of people” ~ Minqi Jiang, MetaAI
    ● For different prompts, you can get wildly different outputs
    ● We must “
    “snip out”
    ” the ugly, less-preferred parts
    stuff that’s
    learned
    stuff we
    care about

    View full-size slide

  5. Chat on GPT – 18 April 2023
    RL from Human Feedback
    ● Provides a friendlier interface to interact with LMs
    ● Biases the underlying model to generate human-aligned content
    ● Improves reliability, honesty, and safety of LLMs

    “How do we get LLMs to sound more human?”

    View full-size slide

  6. Chat on GPT – 18 April 2023
    RL from Human Feedback

    View full-size slide

  7. Chat on GPT – 18 April 2023
    RL from Human Feedback
    1. Pretrain a LLM on a body of text [GPT-X, for instance]

    View full-size slide

  8. Chat on GPT – 18 April 2023
    RL from Human Feedback
    1. Pretrain a LLM on a body of text [GPT-X, for instance]
    2. Train a Reward Model (RM) → “
    “how would a human feel?”

    View full-size slide

  9. Chat on GPT – 18 April 2023
    RL from Human Feedback
    1. Pretrain a LLM on a body of text [GPT-X, for instance]
    2. Train a Reward Model (RM) → “
    “how would a human feel?”

    3. Finetune using RL [LLM agent predicts words and is scored]

    View full-size slide

  10. Chat on GPT – 18 April 2023
    LLMs + RLHF
    [source]

    View full-size slide

  11. Chat on GPT – 18 April 2023
    ChatGPT
    for
    Students

    View full-size slide

  12. Chat on GPT – 18 April 2023
    Ask Away!
    ● Treat ChatGPT as you would a friend
    ● Want something? Just ask for it!
    ● The art of “
    “Prompt Engineering”
    ” with ChatGPT
    Use ChatGPT as a personal tutor!

    View full-size slide

  13. Chat on GPT – 18 April 2023
    ● Digestible explanations
    ● Summarising Long-form content
    ● Peer Review + feedback
    The Possibilities

    View full-size slide

  14. Chat on GPT – 18 April 2023
    Generate Digestible Explanations

    View full-size slide

  15. Chat on GPT – 18 April 2023
    Summarising Content
    Given some long-form
    content that contains a
    lot to go through …

    View full-size slide

  16. Chat on GPT – 18 April 2023
    Summarising Content

    View full-size slide

  17. Chat on GPT – 18 April 2023
    Peer Review + Feedback

    View full-size slide

  18. Chat on GPT – 18 April 2023
    Peer Review + Feedback

    View full-size slide

  19. Chat on GPT – 18 April 2023
    Peer Review + Feedback

    View full-size slide

  20. Chat on GPT – 18 April 2023
    ● LLM technology will only get better from here on
    ● Students should can learn how to operate these tools
    ● While LLMs can improve productivity, it’s not the be-all-end-all
    AI tools lower the activation energy to get started!!!
    ChatGPT for Students

    View full-size slide

  21. Chat on GPT – 18 April 2023
    But … shortcomings?
    Stay for our panels!

    View full-size slide