Human in the Loop - NUS Chat on GPT

2023 April 18 @ UTown Auditorium 1, NUS, Singapore
Rishabh Anand

Video Available at: https://www.youtube.com/watch?v=WupvFC_zaZU&t=1768s

Event Website: https://wing-nus.github.io/chatongpt/

Transcript

1. Chat on GPT – 18 April 2023 Rishabh Anand @rishabh16_ Human in the Loop
2. Chat on GPT – 18 April 2023 The GPT Series GPT → Generative Pretrained Transformer GPT GPT-2 GPT-3 ChatGPT(GPT-3.5) GPT-4 2018 Improving Language Understanding by Generative Pre-Training Language Models are Unsupervised Multi- task Learners Training language models to follow instructions with human feedback * Language Models are Few-Shot Learners 2019 2020 2022 2023 * GPT-3.5 is built on top of InstructGPT with a different data collection setup (technical report) Rapid growth …
3. Chat on GPT – 18 April 2023 Reinforcement Learning from Human Feedback
4. Chat on GPT – 18 April 2023 (Large) Language Models • Language Models (like GPT-X), ◦ are chaotic ◦ model a “giant mass of people” ~ Minqi Jiang, MetaAI • For different prompts, you can get wildly different outputs • We must “ “snip out” ” the ugly, less-preferred parts stuff that’s learned stuff we care about
5. Chat on GPT – 18 April 2023 RL from Human Feedback • Provides a friendlier interface to interact with LMs • Biases the underlying model to generate human-aligned content • Improves reliability, honesty, and safety of LLMs “ “How do we get LLMs to sound more human?” ”
6. Chat on GPT – 18 April 2023 RL from Human Feedback
7. Chat on GPT – 18 April 2023 RL from Human Feedback 1. Pretrain a LLM on a body of text [GPT-X, for instance]
8. Chat on GPT – 18 April 2023 RL from Human Feedback 1. Pretrain a LLM on a body of text [GPT-X, for instance] 2. Train a Reward Model (RM) → “ “how would a human feel?” ”
9. Chat on GPT – 18 April 2023 RL from Human Feedback 1. Pretrain a LLM on a body of text [GPT-X, for instance] 2. Train a Reward Model (RM) → “ “how would a human feel?” ” 3. Finetune using RL [LLM agent predicts words and is scored]
10. Chat on GPT – 18 April 2023 LLMs + RLHF [source]
11. Chat on GPT – 18 April 2023 ChatGPT for Students
12. Chat on GPT – 18 April 2023 Ask Away! • Treat ChatGPT as you would a friend • Want something? Just ask for it! • The art of “ “Prompt Engineering” ” with ChatGPT Use ChatGPT as a personal tutor!
13. Chat on GPT – 18 April 2023 • Digestible explanations • Summarising Long-form content • Peer Review + feedback The Possibilities
14. Chat on GPT – 18 April 2023 Generate Digestible Explanations
15. Chat on GPT – 18 April 2023 Summarising Content Given some long-form content that contains a lot to go through …
16. Chat on GPT – 18 April 2023 Summarising Content
17. Chat on GPT – 18 April 2023 Peer Review + Feedback
18. Chat on GPT – 18 April 2023 Peer Review + Feedback
19. Chat on GPT – 18 April 2023 Peer Review + Feedback
20. Chat on GPT – 18 April 2023 • LLM technology will only get better from here on • Students should can learn how to operate these tools • While LLMs can improve productivity, it’s not the be-all-end-all AI tools lower the activation energy to get started!!! ChatGPT for Students
21. Chat on GPT – 18 April 2023 But … shortcomings? Stay for our panels!