@arfon Collaborations Workshop 2024 Five principles for building generative AI products 

Collaborations Workshop 2014 

Collaborations Workshop 2024 Zenodo & fi gshare archives https://gh.io/doi 

Collaborations Workshop 2024 Zenodo DOIs for releases https://gh.io/doi 

Collaborations Workshop 2024 Citation Files https://gh.io/cite 

Collaborations Workshop 2024 Citation Files https://gh.io/cite 

Collaborations Workshop 2024 Citation Files https://gh.io/cite 

Collaborations Workshop 2024 ORCID https://gh.io/orcid 

Collaborations Workshop 2024 ORCID https://gh.io/orcid 

Collaborations Workshop 2024 A quick recap of (GitHub’s) Copilot* * Other Copilots exist. Millions of users, > 500k students, teachers, open source maintainers Started as ‘code completion’ product Developers writing code ~55%† faster People feel more ful fi lled (~75%†), con fi dent in code quality (~85%†), more focused VS Code, VS, Neovim, JetBrains, GitHub.com Technical Preview June 2021, ‘generally available’ June 2022. https://gh.io/accenture † With the vast majority of it being retained. Maintaining ‘ fl ow’ state. 

Collaborations Workshop 2024 (Generative) AI is here… The people around you are likely using it There is a “there” there Moore’s law will continue to hold People are going try all sorts of crazy sh*t A community of enablers should have opinions Some working assumptions… Whether it’s ChatGPT or GitHub Copilot, these are technologies people are using. Generative AI can be genuinely useful when applied to the right problems. Models will likely become more capable, costs will reduce, more will be possible for less. Just check your favourite tech news site. And that’s what I’m sharing with you today. 

Collaborations Workshop 2024 Some further framing LLMs are remarkably versatile They are changing how people work There will be much more code in the future Lots of tools (and models) to experiment with There’s real work to do to build responsibly Some working assumptions… Answer questions, summarise content, generate code, understand sentiment. Cost of exploring an idea is trending towards zero. Including written by many people who currently aren’t (lower barrier to entry). Open AI, AWS Bedrock, Hugging Face, Replicate. Please learn about responsible AI development. rai.ac.uk is a good place to start. 

Collaborations Workshop 2024 

Collaborations Workshop 2024 

Collaborations Workshop 2024 No decisi on s 

Collaborations Workshop 2024 Copilot in the ‘right seat’ AIs make mistakes – building assistive experiences signi fi cantly safer than ones that take action directly. Copilot is designed to help you make better, faster decisions, not make decisions for you. No decisions 

Collaborations Workshop 2024 Design f or fail ur e 

Collaborations Workshop 2024 What is the cost of getting it wrong? If an error occurs, is the user likely to spot it? What is the cost to the user if they don't? If detected, how easy is it to dismiss an error and move on? Can *you* measure a good or bad outcome? Design for failure 

Collaborations Workshop 2024 

Collaborations Workshop 2024 AI/LLM hallucinations † An inherent characteristic of LLMs Hallucinations are unavoidable Mild hallucinations are very common Lots of techniques and tools exist for mitigating them Did I say they are unavoidable? They are *always* hallucinating, it’s just sometimes those hallucinations are useful. But can be reduced through a variety of techniques including tuning, prompting, context. Design for roughly 50% of the time. Severe ones less common, especially with grounding. Active area of research and development. Responsible development means only building systems where hallucinations aren’t dealbreakers. A quick aside about… 

Collaborations Workshop 2024 Gr ou nded in reality 

Collaborations Workshop 2024 Example of optimising the output of large language models Many, many options to improve model outputs. Retrieval-Augmented Generation is probably the best ‘bang for your buck’. Most use cases *do not* require custom models. Grounded in reality 

Collaborations Workshop 2024 Retrieval-Augmented Generation 🧑💻 LLM Find related information Prompt + user query Prompt + user query + retrieved context 

Collaborations Workshop 2024 Retrieval Augmented Generation † Lots of di ff erent ways to do the ‘retrieval’ Essentially giving the model relevant input context Combined with links/citations can increase utility Allows base models to ‘work’ for specialised areas Typically much cheaper than other optimisations Keyword search, vector search, other ‘similarity-type’ searches. Steers the model outputs towards better/more relevant answers. Allow users to follow relevant links, decide which of them are most relevant. For example if the topic being discussed isn’t ‘knowable’ from public information. e.g., fi ne-tuned (or completely custom) models. 

Collaborations Workshop 2024 

Collaborations Workshop 2024 Invite on ly 

Collaborations Workshop 2024 How can I help you today? LLMs can power remarkable conversational experiences (e.g., natural language — source code). *But remember*, small hallucinations are very common. Only surprise if you can delight. Invite only 

Collaborations Workshop 2024 Explain like it’s me 

Collaborations Workshop 2024 Leveraging the LLM’s ability to personalise responses ELI5: Explain Like I’m Five. ELIKL: Explain Like I know Lots. ELIAPTDKMC: Explain Like I’m A Physicist That Doesn’t Know Much Chemistry. Explain like it’s me 

Collaborations Workshop 2024 No decisi on s • Design f or fail ur e • Gr ou nded in reality • Invite on ly • Explain like it’s me 

Collaborations Workshop 2024 Thanks! [email protected]
 @[email protected]