Devfest MPLS - HCI: The UX of AI

Prompt Engineer, so hot right now. Ever wonder why talking to a computer requires it's own job description and specialization? Or when "chatting" with AI wondered why the computer just "doesn't get you"?
In this session, you'll learn the history of human and computer interaction from words to interfaces and back again. You'll discover WHY chatting with AI feels anything but natural. Finally, you'll learn techniques to build better interfaces to improve a user's ask and reduce obvious hallucination, in turn creating trust in the system.

Transcript

1. Human Computer Interaction : The UX of AI Devfest TC - 2024
2. Buttons +
5. A user interface (UI) is simply how you interact with a machine, device, or software. It's the point of contact where you give instructions and receive information.
6. A Brief (ish) History of UI
8. Punch Cards Yep - How computers think Nope - Time consuming to create - Limited command scope - Immutable
10. Text UI Yep - Uses “human” language - Very fast - Powerful commands through chaining Nope - Difficult to remember - Hard to know what’s “possible” - Linear process/chain of thought
14. Graphical User Interface Yep - Low “memorization” - Powerful tools for complex workflows - Mimics analogue interfaces - Appears simple & “easy” Nope - Pictograms (icons) meaning must be learned - Organization can be complicated - Limited screen space - Slow & distracting
17. Contextual Interface Yep - Low cognitive load (only what matters now) - Combines physical & digital, easily understood Nope - Complex to produce - Needs special hardware - Triggers are hard to define
19. Voice/Ambient UI Yep - Understands language & verbal concepts (intent) - Hands & eyes free Nope - Typically interactions are very slow - Open to misinterpretation - Difficult to know what’s possible
20. What’s your question? Human Computer Interaction 2024 edition
22. Prompt: Adjective done without delay; immediate. "the surfer would have died but for the prompt action of two paramedics"
23. Key components of a good LLM prompt: Task instruction: Clearly state the desired action, like "Summarize this text", "Write a poem about...", or "Explain the concept of...". Contextual information: Provide relevant background details or specific data points to help the LLM understand the situation. Example prompts (few-shot prompting): If necessary, include a couple of example inputs with the desired outputs to show the LLM the format you expect. Constraints or guidelines: Specify any limitations on the response, such as length, tone, or style. Open-ended questions: For more in-depth responses, ask questions that encourage explanation and reasoning.
25. Poor Approach 5 reasons why we shouldn’t rely on prompts
26. Goals over questions 01
27. Asking what we want 02
33. Asking what we want 02
34. What’s not said 03
35. Lacking Detail 05
36. Make me a sandwich Gemini: Create an image of a bacon and egg breakfast sandwich
37. Make me a sandwich Gemini: create an image of: A breakfast sandwich with bacon, fried egg, garlic mayo, arugula, tomato jam, on sourdough bread.
39. Humans talk funny
41. But why???
43. Let’s do better!
45. Fully Hidden Completely remove the prompt interface, create the correct prompts behind the scenes. Get Context Add information that you know about the user, task or goal to provide details to the prompt. Builder Create prompts based on separate blocks of information collected through traditional UI 01 02 03
46. !! Warning: Race Condition There’s a very good chance this Javascript code will result in a race condition! This is likely NOT the intended result. Consider using async.all() to resolve the issue. Should I rewrite this for you?
47. Community Search Guided Real Estate Finder for places you haven’t been
49. → As a real estate agent describe the pros and cons of moving to the Calgary metropolitan area for a family of 4 including two children ages 13 and 9. The family favors the outdoors, cultural activities, and quality restaurants. This should be a summary of no more than a paragraph
50. Event planning Guided inputs for the perfect evening
51. Help me plan a date night with my significant other for this weekend in Minneapolis. Include two activities that we could do together and suggest a place for dinner.
54. → Generate a list of cuisine types available within 10km of the Calgary AB Canada city center. Maintain this list for later. → As an expert local guide in the city of Calgary AB Canada, plan a date night for two people to take place either Nov 16 or 17 2024. Suggest two activities, the first activity is described as adventurous and outdoorsy while the second activity is described as low-key and romantic. Select these from local events calendars, top 10 lists, and things to do in Calgary. Include restaurant recommendations for a random cuisine type selected from the previous list. Any restaurant selected must include dairy free menu options. Favor restaurants that take reservations.
55. There’s a catch
57. → As an expert local guide in the city of Minneapolis MN, plan a date night for two people to take place either Nov 16 or 17 2024. Suggest two activities, the first activity is described as adventurous and outdoorsy while the second activity is described as low-key and romantic. Select these from local events calendars, top 10 lists, and things to do in Minneapolis. Include restaurant recommendations for a random cuisine type selected from the previous list. Any restaurant selected must include dairy free menu options. Favor restaurants that take reservations. Any activity or restaurant recommendation must include a business or event name, a street address and a URL for more information.
58. No more questions!
59. https://www.linkedin.com/in/brycehowitson