apidays Paris 2024 - Putting the Engineering into Prompt Engineering - Mike Taylor

Putting the 'Engineering' into Prompt Engineering
Mike Taylor, Prompt Engineer and Author of "Prompt Engineering for Generative AI: Future-Proof Inputs for Reliable AI Outputs", O'Reilly 2024

apidays Paris 2024 - The Future API Stack for Mass Innovation
December 3 - 5, 2024

------

Check out our conferences at https://www.apidays.global/

Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8

Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io

Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/

Transcript

1. Putting the engineering into prompt engineering Putting the engineering into prompt engineering
2. Putting the engineering into prompt engineering Meet your instructor 2 • Mike Taylor co-founded a 50-person marketing agency called Ladder, created marketing & AI courses on LinkedIn, Vexpower, and Udemy taken by 450,000 people, and published a book with O’Reilly on prompt engineering. He builds AI products at Brightpool.dev. • Twitter: @hammer_mt | LinkedIn: in/mjt145/
3. Putting the engineering into prompt engineering • Is prompt engineer a real job? • What is prompt engineering? • What does a prompt engineer do? • The business case for prompting Agenda
4. Putting the engineering into prompt engineering Is Prompt Engineer a real job?
5. Putting the engineering into prompt engineering LinkedIn
6. Putting the engineering into prompt engineering “AI Expert Answers Prompt Engineering Questions From Twitter | Tech Support | WIRED”
7. Putting the engineering into prompt engineering
8. Putting the engineering into prompt engineering
9. Putting the engineering into prompt engineering Prompting vs Prompt Engineering Prompting: infrequent, open-ended tasks ⛵ Prompt Engineering: large-scale automated tasks 🌉
10. Putting the engineering into prompt engineering What is Prompt Engineering?
11. Putting the engineering into prompt engineering Naive Prompt Write a social media post about how {insight}, for {social_network}. - insight: prompt engineering will still be needed with smarter models, as even genius humans need prompting from legal, HR, management to align with business interests - social_network: Twitter
12. Putting the engineering into prompt engineering Prompting GPT-4o • Prompt template • Task context • OpenAI API
13. Putting the engineering into prompt engineering Naive Response 🌟 As AI models get smarter, some wonder: will we still need #PromptEngineering? Absolutely! Even genius minds need guidance from #Legal, #HR, & #Management to align with business goals. Similarly, advanced AI will need careful prompting to ensure it acts in our best interests. 🤖✨ #AI #FutureOfWork #TechInsights 🚀
14. Putting the engineering into prompt engineering Optimized Response Even Einstein needed prompting. Think about it: High IQ alone doesn't ensure alignment with corporate objectives. Just like genius minds need legal and HR to steer them, smart AI models will still need prompt engineers. The art of asking the right questions remains invaluable in business.
15. Putting the engineering into prompt engineering Prompting vs Prompt Engineering Naive 🌟 As AI models get smarter, some wonder: will we still need #PromptEngineering? Absolutely! Even genius minds need guidance from #Legal, #HR, & #Management to align with business goals. Similarly, advanced AI will need careful prompting to ensure it acts in our best interests. 🤖✨ #AI #FutureOfWork #TechInsights 🚀 Optimized Even Einstein needed prompting. Think about it: High IQ alone doesn't ensure alignment with corporate objectives. Just like genius minds need legal and HR to steer them, smart AI models will still need prompt engineers. The art of asking the right questions remains invaluable in business.
16. Putting the engineering into prompt engineering AI Applications “Patterns for Building LLM-based Systems & Products” by Eugene Yan
17. Putting the engineering into prompt engineering Prompt engineering is the process of discovering prompts which reliably yield useful or desired results.
18. Putting the engineering into prompt engineering What does a prompt engineer do?
19. Putting the engineering into prompt engineering Prompt: Write a social media post about how {insight}, for {social_network}, in the style of Malcolm Gladwell. The social post should sound authentically human and colloquial, while being practically useful and brief. Be very creative and make niche references to sound more human. To maximize engagement, the content should be surprising, interesting, or practically useful, or induce high-arousal emotions such as anxiety, anger, awe, or surprise. Prompt Writing
20. Putting the engineering into prompt engineering FewShot Examples / RAG Prompt: Example 1: - insight: overqualified students take service jobs - social_network: Facebook - bait: Use the trope that smart people are so focused they fail to look after themselves. - hook: Particle Physics is a subject that is universally associated with smart people. - reward: Fantasizing about being somewhere remote where you can earn money while focusing on your work. - post_content: Meet Jon. He has been working the night shift at our hotel for over a month already, and he says the best part is the peace and quiet. Particle physics, which is the topic of Jon's Masters Thesis, requires concentration. That's fine by us. So long as our guests can count on Jon, he can count all the particles he wants (or whatever it is that particle physicists do). Example 2: ...
21. Putting the engineering into prompt engineering Eval Metrics Prompt: You are an expert social media analyst. Your task is to evaluate the following social media post and predict its engagement potential. Consider factors such as: - Insight: Does the post use the relevant insight? - Bait: Does it grab attention? - Hook: Does it keep the attention? ... Based on these factors, rate the post's engagement potential on a scale of 1-5, where 1 is very low engagement, and 5 is very high engagement. Provide a brief explanation for your rating. Any posts that contain hallucinations or made up statistics should be ranked 0. Output your response in YAML with the following keys: - Analysis: [Your analysis] - Rating: [Your rating]
22. Putting the engineering into prompt engineering Malcolm Gladwell (4.0 / 5.0) Even Einstein needed prompting. Think about it: High IQ alone doesn't ensure alignment with corporate objectives. Just like genius minds need legal and HR to steer them, smart AI models will still need prompt engineers. The art of asking the right questions remains invaluable in business. A/B Testing Malcolm Tucker (4.43 / 5.0) Even the brightest minds need a kick in the arse sometimes. Think you're too smart for prompt engineering? Well, even Einstein had a bit of a nudge from reality, didn't he? Smarter models won't replace human intuition; they'll need direction like any genius on a caffeine bender. Prompt engineers, gear up. You're not out of a job yet! In the style of...
23. Putting the engineering into prompt engineering Prompt Optimizers Response: Think smarter AI models will make prompt engineering obsolete? Think again, mate! Even Einstein needed a nudge from his boss. Just like your office genius needs a prod from legal, HR, and management to align with business goals, so do our AI models. Smarter doesn't mean self-sufficient. So, prompt engineers, keep your boots on. Your job isn't going anywhere! DSPy
24. Putting the engineering into prompt engineering Fine-Tuning Response: Even Einstein needed a nudge! Think smarter AI means prompt engineers are on their way out? Wrong. Just like even the most brilliant minds need direction from legal, HR, and management, advanced AI models still struggle to align with business goals. As models get smarter, the need for effective prompting becomes even more critical. Stay ahead of the curve, or risk your own algorithmic chaos.
25. Putting the engineering into prompt engineering The Business Case for Prompting
26. Putting the engineering into prompt engineering ~25% Performance Gain From Prompting + 1 Example “Language Models are Few-Shot Learners” Putting 1 example in the prompt gets ~28% accuracy, and better prompting takes it to ~46%
27. Putting the engineering into prompt engineering GPT-4 can cost as much as an employee 27 You can see cost reductions of 10x to 100x on some tasks by using prompt engineering to get a smaller model up to GPT-4 level of quality.
28. Putting the engineering into prompt engineering Prompting Beats Fine-Tuning Under 2,000 Data Points “How Many Data Points is a Prompt Worth” Putting examples in the prompt (In-Context Learning) gets ~85% accuracy vs ~78% even with 1,000 data points.
29. Putting the engineering into prompt engineering The Smiling Curve of AI applications “What works in AI UX” by swyx “Why Vertical LLM Agents Are The New $1 Billion SaaS Opportunities” by YC
30. Putting the engineering into prompt engineering Putting the engineering into prompt engineering Questions?
31. Putting the engineering into prompt engineering Prompt Engineering for Generative AI • 5 principles of prompting • 100s of real-world examples • LLMs with OpenAI and LangChain • Image generation with AUTOMATIC1111 • Build an end-to-end AI application > Prompt Engineering for Generative AI