Joe Cheng / CTO at Posit / posit::conf(2024) Shiny × AI I, for one, welcome our new robot overlords

No content

What is your AI strategy?

Joe’s 5 stages of AI grief Denial: Arti fi cial “Intelligence”? All I see are stochastic parrots. Anger: Enough with the AI hype! Can we talk about literally anything else!? Bargaining: Fine, there’s something here, but let’s wait for the dust to settle. Depression: AI is going to eat everything I’ve ever loved. Acceptance: pip install openai

Two-pronged approach • What does Shiny have to o ff er AI researchers? • “I have an idea for an AI-driven app. Is Shiny a good framework to use?” • What does AI have to o ff er Shiny app authors? • “I have a Shiny app. Are there interesting, useful, and responsible ways to leverage LLMs to enhance my app?”

What can Shiny do for AI? • Honestly, I don’t know yet • But let me share one recent experience...

No content

Hi Curtis, I want to build a video chat assistant using GPT-4o, can Shiny do that? Tina Great, can Winston join my livestream
 in 9 days and live code one? Ahoy! You betcha Curtis @ Posit Fo sho

A few small problems 😬 • GPT-4o doesn’t o ff er a video chat API • Shiny doesn’t have a video input component • Winston had just left for PyCon ✈

No content

One week later... input_video_clip()

One week later... input_video_clip() audio_spinner()

Morning of the livestream...

Demo https://jcheng.shinyapps.io/multimodal/

Morning of the livestream... Awesome! What if it could
 query three di ff erent LLMs
 and let you compare which
 one did the best? And plot
 which model is in the lead? 😳😬😰🤔🧐👍

Three hours later

What did we learn? • Interactivity “in the small” is hard and requires JavaScript expertise • Creating polished controls for input_video_clip and audio_spinner took time and skill • Interactivity “in the large” is easy (with Shiny) • Building apps around these components was blazingly fast to build and iterate

What did we learn? • Sometimes you need a Tina

What can AI do for Shiny? • Honestly, I don’t know yet • But we’re o ff to an amazing start!

Have you thought at all about how you might combine shiny + LLMs to create dashboards that give the viewer some ability to do independent analysis? Hadley How did you get this number Joe

No content

Put a chatbot on it!

Demo https://jcheng.shinyapps.io/sidebot/

What did we learn? The power of Domain Speci fi c Languages • SQL is a stunningly good language for connecting LLMs to data apps • Easy for LLMs to write • Easy for humans to read and reason about • Powerful and fl exible • Safe (relative to executing straight Python or R from an LLM) • Other DSLs might hold similar promise • Vega-lite for grammar of graphics • CSS Grid for dashboard layout

What did we learn? • Being a mere consumer of Copilot and ChatGPT is not enough to have a well- informed opinion of the potential of LLMs (you will miss out on structured output and tool calls). Start coding! • Revise your mental model of LLMs • The “stochastic parrot” mental model is true, reassuring, and unhelpful • The “machines that reason” mental model is false, terrifying, and helpful • Stay skeptical, but also, let yourself be curious

No content

What is your AI strategy?

Thank you • Links to all packages and demos:
 https://tinyurl.com/jcheng-conf2024 or QR code 👉 • For a less con fl icted take on Shiny and AI, stick around for the next talk! • Come talk to us at the Lounge from 12-1PM today, or catch us on Discord