HCI: The UI of AI DevFestYYC 2024

Prompt Engineer, so hot right now! Do you wonder why we need an entire specialization dedicated to communicating with a computer? Does it feel like "it just doesn't get you" when "chatting" with AI tools?

In this session, we'll cover the history of ways humans have communicated with computers. You'll see why "natural language" doesn't seem to translate as you expected. Finally, you'll learn options for creating an interface with AI that doesn't require a special title. Along the way, you'll learn to create trust and reduce hallucinations.

This session is for anyone building AI products where the audience isn't another computer, or for anyone interested in why it's terrible talking to computers and what we can do about it.

Transcript

1. Human Computer Interaction : The UX of AI Devfest YYC - 2024
2. Buttons +
5. A user interface (UI) is simply how you interact with a machine, device, or software. It's the point of contact where you give instructions and receive information.
6. A Brief (ish) History of UI
8. Punch Cards Yep - How computers think Nope - Time consuming to create - Limited command scope - Immutable
10. Text UI Yep - Uses “human” language - Very fast - Powerful commands through chaining Nope - Difficult to remember - Hard to know what’s “possible” - Linear process/chain of thought
14. Graphical User Interface Yep - Low “memorization” - Powerful tools for complex workflows - Mimics analogue interfaces - Appears simple & “easy” Nope - Pictograms (icons) meaning must be learned - Organization can be complicated - Limited screen space - Slow & distracting
17. Contextual Interface Yep - Low cognitive load (only what matters now) - Combines physical & digital, easily understood Nope - Complex to produce - Needs special hardware - Triggers are hard to define
19. Voice/Ambient UI Yep - Understands language & verbal concepts (intent) - Hands & eyes free Nope - Typically interactions are very slow - Open to misinterpretation - Difficult to know what’s possible
20. What’s your question? Human Computer Interaction 2024 edition
22. Prompt: Adjective done without delay; immediate. "the surfer would have died but for the prompt action of two paramedics"
23. Key components of a good LLM prompt: Task instruction: Clearly state the desired action, like "Summarize this text", "Write a poem about...", or "Explain the concept of...". Contextual information: Provide relevant background details or specific data points to help the LLM understand the situation. Example prompts (few-shot prompting): If necessary, include a couple of example inputs with the desired outputs to show the LLM the format you expect. Constraints or guidelines: Specify any limitations on the response, such as length, tone, or style. Open-ended questions: For more in-depth responses, ask questions that encourage explanation and reasoning.
25. Poor Approach 5 reasons why we shouldn’t rely on prompts
26. Goals over questions 01
27. Asking what we want 02
33. Asking what we want 02
34. What’s not said 03
35. Lacking Detail 05
36. Make me a sandwich Gemini: Create an image of a bacon and egg breakfast sandwich
37. Make me a sandwich Gemini: create an image of: A breakfast sandwich with bacon, fried egg, garlic mayo, arugula, tomato jam, on sourdough bread.
39. Humans talk funny
41. But why???
43. Let’s do better!
45. Fully Hidden Completely remove the prompt interface, create the correct prompts behind the scenes. Get Context Add information that you know about the user, task or goal to provide details to the prompt. Builder Create prompts based on separate blocks of information collected through traditional UI 01 02 03
46. !! Warning: Race Condition There’s a very good chance this Javascript code will result in a race condition! This is likely NOT the intended result. Consider using async.all() to resolve the issue. Should I rewrite this for you?
47. Community Search Guided Real Estate Finder for places you haven’t been
49. → As a real estate agent describe the pros and cons of moving to the Calgary metropolitan area for a family of 4 including two children ages 13 and 9. The family favors the outdoors, cultural activities, and quality restaurants. This should be a summary of no more than a paragraph
50. Event planning Guided inputs for the perfect evening
51. Help me plan a date night with my significant other for this weekend in Calgary. Include two activities that we could do together and suggest a place for dinner.
54. → Generate a list of cuisine types available within 10km of the Calgary AB Canada city center. Maintain this list for later. → As an expert local guide in the city of Calgary AB Canada, plan a date night for two people to take place either Nov 16 or 17 2024. Suggest two activities, the first activity is described as adventurous and outdoorsy while the second activity is described as low-key and romantic. Select these from local events calendars, top 10 lists, and things to do in Calgary. Include restaurant recommendations for a random cuisine type selected from the previous list. Any restaurant selected must include dairy free menu options. Favor restaurants that take reservations.
55. There’s a catch
57. → As an expert local guide in the city of Calgary AB Canada, plan a date night for two people to take place either Nov 16 or 17 2024. Suggest two activities, the first activity is described as adventurous and outdoorsy while the second activity is described as low-key and romantic. Select these from local events calendars, top 10 lists, and things to do in Calgary. Include restaurant recommendations for a random cuisine type selected from the previous list. Any restaurant selected must include dairy free menu options. Favor restaurants that take reservations. Any activity or restaurant recommendation must include a business or event name, a street address and a URL for more information.
58. No more questions!
59. https://www.linkedin.com/in/brycehowitson