Fergal Reid - Building products in the age of Ai

Transcript

1. Building products in the Age of AI @fergal_reid
2. GPT / LLMs • Internet sized change • Change in capability • Change in how we build and use AI
6. Level 1: GPTs are incredible! Level 2: GPTs make things up and aren’t trustworthy. Level 3: GPTs can be incredible when used right 

7. See them as engineering components Separate out aspects accidentally bundled
8. What is GPT?
9. Training objective: token prediction
10. Training objective: token prediction
13. • A sequence model • That uses ‘attention’ • Gradient descent
14. • A sequence model • That uses ‘attention’ • Gradient descent
15. Not a useful model • Human = genes and evolution ? • Distrust: 
 ‘It de fi nitely can’t do X because its just trained to predict the next word’
16. Model: Database + Reasoning Engine • The reasoning engine is key • Often, the database is a liability
17. Reasoning capabilities
20. Model: ‘Interpolative’ vs ‘Extrapolative’ tasks
22. • Less reliable at extrapolation • Favour interpolation • Perform a task, given a context • ‘Retrieval Augmented Generation’
23. Model: Human intuition Ask a human to answer a historical question 
 
 vs Give them a history book and ask them the question
24. Note: Context window limited • Thousands of words • Can’t put a whole KB, or context, in it • Synergizes well with Vector Search
25. How we build with GPTs
30. 30 November 2022: 
 ChatGPT
31. First features we built • Summarisation • Edit tone of voice • Expand from shorthand
34. • 5th Dec: Rolling • 20th Dec: Internal use • ~13th Jan: Customer beta • 31st Jan: Launch with testimonials Timeline
35. Model: Easy vs Hard 
 AI features
36. • ‘Easy’: • Out-of-box accuracy high • Cost of error low • E.g. ‘Draft me a summary’
37. • ‘Hard’: • Out-of-box accuracy low • Cost of error high
38. Development Tactics
39. • Fast customer contact • Assume you can build v1 of most ML with powerful LLM • Make cheap later • “LLMs aren’t all of AI” • How we build software has changed
40. Hard feature: Fin • GPT-powered 
 question answering Bot
41. • An LLM can seem inert • However, can easily be turned into an agent
42. My key points • Internet sized change • Good model: DB+Reasoning • Changes how we build ML • Feature dif fi culty varies
43. Guessing what’s next
44. • V1: text tools, working around clunky interfaces • V2: features reasoning can enhance • V?: End to end problems where intelligence can help • Don’t underestimate the reasoning capability, very sophisticated
45. • Breakneck progress • Smaller models, open? • Exciting but overhyped today • Productisation • Larger models
46. Thank you! @fergal_reid