apidays Munich 2025 - Streamline & Secure LLM Traffic with APISIX AI Gateway (API7)

Streamline & Secure LLM Traffic with APISIX AI Gateway
Yilia Lin, Technical Writer at API7

apidays Munich 2025 - Accelerate AI Use Cases with APIs
July 2 & 3, 2025

------

Check out our conferences at https://www.apidays.global/

Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8

Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io

Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/

Transcript

1. Streamline & Secure LLM Traffic Using APISIX AI Gateway Yilia Lin, 3 July, 2025, API Days Munich
2. Agenda 01 02 03 Apache APISIX Overview APISIX AI Gateway Overview Proxy Multi-LLMs and Token -based Rate Limiting 04 Q&A
3. About Speaker  Apache APISIX Committer  Technical Writer at API7.ai  LinkedIn: linkedin.com/in/yilialin/  GitHub: github.com/Yilialinn Yilia Lin
4. Apache APISIX Overview 01
5. Apache APISIX Overview • Donated to Apache Software Foundation by API7.ai in 2019 • Ultra High-Performance: > 23,000 single-core QPS • Low Latency: < 0.6 ms average delay • Lightweight Architecture: Decoupled control plane and data plane • High Scalability: >100 open-source plugins • Open-Source without Vendor Lock-in: Apache License 2.0
6. APISIX AI Gateway Overview 02
7. The Rise of AI and New Challenges AI Application Characteristics • High-concurrency LLM Services • Token-based Pricing Model • Dynamic Scalability • Content Sensitivity New Challenges • Traffic Governance • Cost Optimization • Multi-Version Management • Content Security
8. APISIX AI Gateway Features
9. AI plugins APISIX AI Gateway Architecture More Resources • https://apisix.apache.org/docs/apisix/plugins • https://docs.api7.ai/hub
10. APISIX AI Gateway Characteristics • Open-Source • Out-of-the-box • High Scalability • High Stability • High Security
11. Practical Application of APISIX AI Gateway 03 Proxy Multi-LLMs and Implement Token-Based Rate Limiting
12. Workflow
13. Configure Multi-LLMs and Implement Token-Based Rate Limiting  demo: https://app.storylane.io/share/cjpfweudrq1n  doc: https://docs.api7.ai/hub/ai-proxy-multi#configure-instance- priority-and-rate-limiting
14. Thank You! Yilia Lin yilialin Yilialinn