Defensive LLM Calling - Speaker Deck

Defensive LLM Calling

by Ogata Katsuya

Slide 1

Slide 1 text

Kyoto Tech Talk #9 2025/12/05 Defensive LLM Calling Katsuya Ogata

Slide 2

Slide 2 text

ࣗݾ঺հ • ໊લ: ॹํ ࠀ࠸ʢ͓͕ͨ ͔ͭ΍ʣ • ॴଐ: େࡕେֶ ৘ใՊֶݚڀՊ • झຯ: ొࢁɾཱྀߦɾα΢φ • MLLMΛ༻͍ͨಈըղੳͷݚڀ • ϝϧϘϧϯʹߦͬͯ·ͨ͠ • X: @ogata_katsuya • ݸਓ: www.ogatakatsuya.com ϝϧϘϧϯཱྀߦ ޒ৭ϲݪ΁ͷొࢁ

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

AWS 2025/10/20 Google Cloud 2025/06/13 Azure 2025/10/29

Slide 5

Slide 5 text

Ϋϥ΢υ͸յΕΔલఏͰར༻͢Δ

Slide 6

Slide 6 text

2025೥ʹো֐͕ى͖ͯͨͷ͸ʁ

Slide 7

Slide 7 text

શ෦

Slide 8

Slide 8 text

ো֐ͷݪҼͱ಺༰ • ݪҼ • AIΛϗετ͍ͯ͠ΔΫϥ΢υͷো֐ • ίϯύΠϥͷόάʢΠϯϑϥىҼʣ • ಺༰ • ϨΠςϯγͷٸܹͳ૿Ճ • ਫ਼౓ͷ௿Լ • ͦ΋ͦ΋LLMΛݺͼग़͢͜ͱ͕Ͱ͖ͳ͍

Slide 9

Slide 9 text

LLM΋յΕΔલఏͰར༻͢Δ

Slide 10

Slide 10 text

🤔

Slide 11

Slide 11 text

Application Layer Transport Layer Network Layer Network Access Layer TCP/IP

Slide 12

Slide 12 text

τϥϯεϙʔτ૚ • ΞυϨε͸ϙʔτ൪߸ͷΈʢIPΛ஌ͬͯΔͷ͸ωοτϫʔΫ૚ʣ • ͲΜͳωοτϫʔΫͷܦ࿏Λ௨Δ͔͸஌Βͳ͍ • ωοτϫʔΫͦͷ΋ͷΛ৴པ͍ͯ͠ͳ͍લఏͰ࣮֬ʹσʔλΛૹΓಧ͚Δ • ࠶ૹ੍ޚɺ᫔᫓੍ޚɺޡΓݕग़ • ʢ๻ͷߟ͑Δʣτϥϯεϙʔτ૚ͷ໾ׂ • ΞϓϦέʔγϣϯ૚ʹωοτϫʔΫͷ੾ΓޱΛݟͤΔ • ΞϓϦέʔγϣϯ͸ωοτϫʔΫͷҧ͍Λҙࣝ͠ͳͯ͘ྑ͍ • ωοτϫʔΫͦͷ΋ͷͷ৴པੑΛ୲อ͢Δ

Slide 13

Slide 13 text

🤔

Slide 14

Slide 14 text

Application Layer Transport Layer Network Layer Network Access Layer TCP/IP

Slide 15

Slide 15 text

Application Endpoint ????? LLM Calling LLM Calling ͜͜

Slide 16

Slide 16 text

LLM Callingʹ͓͚Δτϥϯεϙʔτ૚ • LLM Providerͷҧ͍Λٵऩͯ͠ɺͦͷ੾Γޱ͚ͩΛݟͤΔ • ந৅Խ • Ӆṭ • LLM ProviderΛ৴པ͍ͯ͠ͳ͍લఏͰ඼࣭Λ୲อ͢Δ • ϦτϥΠॲཧ • ଞͷProvider΁ͷϑΥʔϧόοΫ • ඞͣԿ͔͠ΒͷϨεϙϯε͕ฦͬͯ͘Δ

Slide 17

Slide 17 text

PythonͰͷ࣮૷ྫ abcΛ༻͍ͨந৅Ϋϥεͷ࡞੒ LiteLLMͱ͍͏ศརϥΠϒϥϦ΋͋Γ·͢

Slide 18

Slide 18 text

PythonͰͷ࣮૷ྫ ֤ϓϩόΠμʔͰͷந৅ϝιουͷ࣮૷

Slide 19

Slide 19 text

PythonͰͷ࣮૷ྫ LLM Callingͷτϥϯεϙʔτ૚(͔ͳΓ୺ંͬͯ·͢)

Slide 20

Slide 20 text

LLM Caller໨ઢ • τϥϯεϙʔτ૚ʹඞཁͳύϥϝʔλΛ౉͚ͩ͢ • ͲΜͳϓϩόΠμͰ࣮ߦ͞ΕΔ͔͸஌Βͳ͍ • ΠϯλʔϑΣʔε͚ͩΛҙࣝͯ͠ɺͦΕҎԼΛؾʹ͠ͳ͍ • ͚ͩͲɺҰఆ඼࣭͕อͨΕͯɺ͋Δఔ౓ͷ଎౓Ͱฦ౴͕ฦͬͯ͘Δ

Slide 21

Slide 21 text

ສࣄղܾʁʁʁ • LLMͷো֐͸ૉ௚ʹΤϥʔΛు͍ͯམͪͯ͘Εͳ͍ • Ϩεϙϯε͕500൪୆ͳΒେৎ෉ • ϨΠςϯγ͕૿Ճ͢Δ • λΠϚʔͱϦτϥΠॲཧͰͳΜͱ͔ͳΔʢ͔΋ʣ • ਫ਼౓͕མͪΔ • ো֐Λײ஌͢Δ͜ͱ͕೉͍͠ • ࠷ऴతʹ͸ɺ͜·Ίͳ؂ࢹͱɺϢʔβʔͷ੠ʹৗʹහײͰ͋Δ͜ͱ͕ॏཁ • Langfuse΍LangsmithͳͲͷLLM؂ࢹମ੍ͷ੔උ΋ඞཁ

Slide 22

Slide 22 text

·ͱΊ • ͜ͷLTʹ໌֬ͳ౴͑͸͋Γ·ͤΜ • LLMͷ඼࣭Λ୲อ͢Δͷ͸ϓϩόΠμ͸΋ͪΖΜɺϢʔβʔ΋੹຿Λෛ͍ͬͯ Δͱ͍͏໰୊ఏى • ࠷௿ݶఏڙ͢΂͖αʔϏεͷ඼࣭Λ୲อ͢ΔͨΊͷϑΥʔϧόοΫઓུΛߟ͑ ͍ͯ͘ඞཁ͕͋Δ • օ͞Μ͕࣮ફ͍ͯ͠ΔϑΥʔϧόοΫઓུɾख๏Λڭ͍͑ͯͩ͘͞

Slide 23

Slide 23 text

Reference • https://azure.status.microsoft/en-us/status/history/ • https://www.youtube.com/watch?v=D297rfqSFvY • https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW • https://www.mobihealthnews.com/news/google-cloud-outage-disrupts-ai- services-used-healthcare • https://www.youtube.com/watch?v=Ar_ckPSwe1Y • https://www.anthropic.com/engineering/a-postmortem-of-three-recent-issues