AWS re:Invent '24 - Rogue Agents - Stop AI from misusing APIs

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

©2024 TWILIO INC. ALL RIGHTS RESERVED RESOLVING ISSUES HOW AI AGENTS WILL CHANGE YOUR CUSTOMER JOURNEY I JUST LANDED AND I CAN'T SEE MY BAGS! YES PLEASE! I'M AT THE FOUR SEASONS IN SF BOTH OF YOUR BAGS (ID #12345, #12931) ARE CURRENTLY STUCK IN TRANSIT AT YOUR LAYOVER LOCATION IN FRANKFURT, GERMANY. DO YOU WANT ME TO SCHEDULE A DELIVERY FOR YOU? CHECK CUSTOMER PROFILE baggage_ids: #12345, #12931 current_flight: ABCREF GET BAG STATUS (#12345) status: delayed location: Frankfurt, Germany GET BAG STATUS (#12931) status: delayed location: Frankfurt, Germany GET FLIGHT DETAILS (ABCREF) destination: SFO connections: FRA LOOKUP LOCATION (FOUR SEASONS IN SF) address: 222 Sansome St, San Francisco, CA 94104 SCHEDULE DELIVERY baggage_ids: #12345, #12931 address: 222 Sansome St, San Francisco, CA 94104 YOUR BAGS HAVE BEEN SCHEDULED TO BE DELIVERED TO 222 SANSOME ST, SAN FRANCISCO, CA 94104 AND SHOULD ARRIVE TOMORROW.

Slide 7

Slide 7 text

©2024 TWILIO INC. ALL RIGHTS RESERVED AI system that decides when and which action to perform WHAT'S AN AI AGENT? DEFINITION ACTION-ORIENTED User Input and Agent Output doesn't have to be chat or voice MORE THAN A CHATBOT

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

©2024 TWILIO INC. ALL RIGHTS RESERVED The LLM might not always make the same description LLMs ARE (REALLY SMART) TODDLERS THE PROBLEM WITH AGENTS UNPREDICTABLE Any input can manipulate the behavior of the LLM EASILY IMPRESSIONABLE

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

©2024 TWILIO INC. ALL RIGHTS RESERVED TWILIO DEVELOPER PREVIEW TRY IT TODAY! AN OPINIONATED FRAMEWORK TO BUILD AND HOST AI AGENTS FOR CUSTOMER-FACING USE CASES SOLVES PROBLEMS AUTONOMOUSLY WITH TOOLS & KNOWLEDGE Assistants can perform complex tasks to solve customer problems without the need for ever-growing ﬂow charts. They can interact with other systems and perform tasks using regular HTTP requests as Tools, and use our built-in Retrieval-Augmented-Generation (RAG) pipelines. USE YOUR EXISTING CUSTOMER DATA AI Assistants can leverage existing Segment proﬁles through the CustomerAI Personalization Engine and enrich them further. Leverage the insights from your AI Assistant using Segment or send it into your own data warehouse. ENGAGE WITH CUSTOMERS HOW THEY PREFER AI Assistants can communicate over a variety of channels so your customers can engage with you in their preferred way.

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

©2024 TWILIO INC. ALL RIGHTS RESERVED LLM-PROOF YOUR APIs TAKEAWAYS Treat LLMs as an untrusted client the same way we do with browsers or mobile apps. WHAT DO NEXT? ■ THREAT MODEL! Think what could an attacker do if they had direct access to the same APIs the AI has access to? ■ SECURITY OUTSIDE OF AI Add any security measures outside the control of the AI ■ CHECK OUT THE OWASP LLM TOP 10 Check out the newly released OWASP LLM Top 10 risks to learn more about securely building AI agents ■ TRY OUT AI ASSISTANTS Come by the demo booth or try out Twilio AI Assistants for yourself. SCAN ME LEARN MORE

Slide 30

Slide 30 text