[CNCF Q1 2024] Agentic Installer LLMs Helm Charts by Chris Gruel @Akeyless

Transcript

1. AI DevOps Workshop: Using LLMs to Deploy Kubernetes Helm Charts Chris Gruel Senior Solution Architect, Akeyless
2. Agenda The Challenge 01 Our Goal 02 The How 03 Tool Selection: Langchain, LangGraph & Chainlit 04 Integrating Human Insights 05 Developing Conversational Agents 06 Demo: Agentic Installer in Action 07 Lessons Learned 08 Future Directions 09 Q&A 10
3. The Challenge: Helm Chart Dependencies and the Akeyless Gateway • What is the Gateway? • Deployment requirements: • Connectivity • Identify cloud provider • Create appropriate auth method • Create the gateway! • Currently, manual process Applications and platforms Gateway CUSTOMER ENVIRONMENT CUSTOMER ENVIRONMENT TLC Gateway
4. Our Goal: Use Function-Calling in LLMs to create Agentic Behavior What is agentic behavior? • Agent = autonomous entity that can perform tasks on its own or with human supervision • Agentic behavior = the agent does the work for us!
5. The How 1. Create the OpenAI agent as an expert-pair programmer 2. Use agent for code structure of the code 3. Use Cursor IDE to fill out the details
6. Tool Selection: Langchain and LangGraph • Langchain: built to abstract LLM complexities, and for connecting to data sources • Langchain Benefits • Abstraction • Flexibility • Monitoring • LangGraph: for cyclical workflows • => Create Human-in-the-Loop for chat interface
7. Choosing Chainlit for UI • Streamlit: difficult to get full interactivity for chat • Chainlit is built for this specific use case • Lesson: It’s a fast-moving industry. If there’s a better solution, switch!
8. Integrating Human Insights Give the agent the ability to ask questions: 1. Gather information 2. Handle exceptions 3. Give us aggregated data based on its questions and the answers it receives
9. Developing Conversational Agents 1. Build asynchronous graphs: list of tools at LLM’s disposal 2. Purpose-built functions – minimize complexities for agent 3. Akeyless SDK is easy to use, but don’t want agent to worry about token 4. Minimize number of inputs for agent 5. We don’t want to send a token to OpenAI: the LLM model shouldn’t have actual credentials
10. Demo: The Agentic Installer in Action! https://github.com/akeyless-community/heimdal
11. Lessons Learned 1. Embrace an AI-powered IDE 2. Build an OpenAI assistant, load the relevant knowledge and use the assistant to output pseudo-code as a start, to be filled out with Cursor IDE. Benefit from the better context information built into the IDE itself! 3. Don’t waste time on a dead end not suited for purpose!
12. Future Directions 1. Deploy K8s Secret Injector 2. Detect if external secrets operator already in cluster >> install integration 3. Detect if Nginx ingress is installed in cluster, and offer to deploy with Nginx ingress 4. Self-service POC: deploy gateway, database, set up secrets… 5. Incorporate ALL public Akeyless documentation and enable RAG for instant support help
13. Thank you.