Bridging Enterprise Data and AI: Lessons Learnt During LLM and AI Agent Integrations

As organizations embrace AI, ensuring secure and efficient data access is critical. This session explores the best practices for integrating large language models (LLMs) via APIs, emphasizing both opportunities and challenges. However, our primary focus will be on securely enabling AI agents to interact with enterprise data while maintaining control and compliance.
Key topics include:
- LLM Integration via APIs (e.g., Langchain4j)
- Secure Data Access for AI Agent Implementing the Principle of Least Privilege
- Secure Function Calling for AI Agents (MCP)
- At Goodbards, we successfully integrated AI agents into both our enterprise data and user interface—ensuring a seamless, user-friendly experience without imposing AI-driven interactions.
Join us to discover how to build a robust and secure bridge between enterprise data and AI, paving the way for the next generation of intelligent business solutions.

Transcript

1. Bridging Enterprise Data and AI Lessons Learnt During LLM and AI Agent Integrations
2. Bridging Enterprise Data and AI: Lessons Learnt During LLM and AI Agent Integrations As organizations embrace AI, ensuring secure and efficient data access is critical. This session explores the best practices for integrating large language models (LLMs) via APIs, emphasizing both opportunities and challenges. However, our primary focus will be on securely enabling AI agents to interact with enterprise data while maintaining control and compliance. Key topics include: - LLM Integration via APIs (e.g., Langchain4j) - Secure Data Access for AI Agent Implementing the Principle of Least Privilege - Secure Function Calling for AI Agents (MCP) - At Goodbards, we successfully integrated AI agents into both our enterprise data and user interface—ensuring a seamless, user-friendly experience without imposing AI-driven interactions. Join us to discover how to build a robust and secure bridge between enterprise data and AI, paving the way for the next generation of intelligent business solutions.
3. Cédrick Lunven @clunven • Co-Founder & CTO Goodbards ◦ Platform, Design • Team Lead AI ◦ AI as a Data API on top of Cassandra • Developer (+Advocate) ◦ Framework ff4j ◦ Langchain4j, spring-ai ◦ Tools (sdk, cli, plugins)
4. CXO executives Marketing Campaigns Objective Duration Budget Audience requirements SME Individuals AI MARKETER Co-PILOT questions Marketing Objective TACTIC RECOMMENDER objective Marketing Tactics Twitter LinkedIn Blog PR Reddit Emails Social Tiktok Professional Marketers Community Managers Content Creators Preferred platform, formatted content required* Marketing Agentic AI Platform
5. Agentic AI
6. ❏ Introduction ❏ Marketing platform ❏ Generative AI ❏ Content Creation ❏ Retrieval Augmented Generation ❏ Use your documents ❏ Adapt your tone of voices ❏ Agentic AI ❏ Assistants to help users ❏ Integration with tools and services ❏ Multi Agent systems ❏ AI Sessions and UI ❏ Agent Collaboration ❏ Agents Orchestration Agenda
7. What is it ? How did we build it ? Risks and Vulnerabilities Lesson Learnt 1 2 3 4
8. ❏ Introduction ❏ Marketing platform ❏ Generative AI ❏ Content Creation ❏ Retrieval Augmented Generation ❏ Use your documents ❏ Adapt your tone of voices ❏ Agentic AI ❏ Assistants to help users ❏ Integration with tools and services ❏ Multi Agent systems ❏ AI Sessions and UI ❏ Agent Collaboration ❏ Agents Orchestration Agenda
9. Marketing Campaign Tactics, Assets, Tracking Customer Data Platform Contacts, company, segments Email Marketing Designs, Templates Marketing Tools QR Code, URL Shortener Portal Cloud-Based, Multi-Tenant Platform Relational DATABASE NoSQL Database Web Rest API
10. Infrastructure Tenants, Users Marketing Campaign Tactics, Assets, Tracking Customer Data Platform Contacts, company, segments Email Marketing Designs, Templates Marketing Tools QR Code, URL Shortener Web Rest API Portal Documents ` PDF Images Bylines …. LLM LLM Connectors Social, CRM Cloud-Based, Multi-Tenant Platform Relational DATABASE NoSQL Database
11. Infrastructure Tenants, Users Marketing Campaign Tactics, Assets, Tracking Customer Data Platform Contacts, company, segments Email Marketing Designs, Templates Marketing Tools QR Code, URL Shortener Web Rest API Portal Documents ` PDF Images Bylines …. LLM LLM Connectors Social, CRM Cloud-Based, Multi-Tenant Platform Relational DATABASE NoSQL Database Generative AI Prompts Images Gen. LLM Connectors LLM Clients Text, Images
12. Generative AI ChatLanguageModel ChatMemory Prompt Template LLM Prompts template Chat Memory
13. user AI System Prompt user AI Prompt & Memory Injection Risks & Mitigations 🛡 Mitigations • Process one user message at a time • Sanitize and validate user inputs • Clear or isolate memory between sessions • Clearly define usage boundaries • Limit scope of capabilities per user/session • Apply authentication or rate-limiting • Throttle token usage per request/session • Set global or user-specific rate limits • Monitor for abnormal consumption patterns • Use system prompts to enforce input restrictions • Never store API keys in frontend code or public repos • Rotate and scope API keys regularly ⚠ Risks Prompt or Memory Injection: manipulate the LLM’s behavior or context through crafted inputs. Abuse of Free Access ("Cool! An AI system for free!") Unrestricted access may lead to misuse or unintended usage. High Volume of Requests or Token Usage: Excessive usage can degrade performance or lead to cost overrun. Data or Intellectual Property Leakage: Sensitive input or generated output could expose private content. Insecure API Integrations: Secrets or tokens may be leaked if exposed improperly.
14. ❏ Introduction ❏ Marketing platform ❏ Generative AI ❏ Content Creation ❏ Retrieval Augmented Generation ❏ Use your documents ❏ Adapt your tone of voices ❏ Agentic AI ❏ Assistants to help users ❏ Integration with tools and services ❏ Multi Agent systems ❏ AI Sessions and UI ❏ Agent Collaboration ❏ Agents Orchestration Agenda
15. There is no free meal
16. Infrastructure Tenants, Users Marketing Campaign Tactics, Assets, Tracking Customer Data Platform Contacts, company, segments Email Marketing Designs, Templates Marketing Tools QR Code, URL Shortener Web Rest API Portal Documents ` PDF Images Bylines …. LLM LLM Connectors Social, CRM Cloud-Based, Multi-Tenant Platform Relational DATABASE NoSQL Database Generative AI Prompts Images Gen. LLM Connectors LLM Clients Text, Images Knowledge Base (vector db) Vector Database RAG
17. Agents ChatLanguageModel ChatMemory Prompt Template LLM Prompts template Chat Memory Knowlegde Base EmbeddingStore Content Retriever
18. user AI System Prompt user AI RAG: Risks & Mitigations Overview ⚠ Risks Data or Intellectual Property Leakage: Sensitive data or trade secrets could be exposed via prompts or responses. Injection in the RAG (Document Corpus): Malicious or biased documents may poison retrieval results and outputs. Regulatory Non-Compliance: Violation of GDPR, HIPAA, or data locality requirements. IP Infringement: Model may generate content that infringes on copyrighted material. Lack of Explainability: AI outputs cannot be justified or audited legally. 🛡 Mitigations • Sanitize inputs/outputs • Guardrails in system prompt • Avoid confidential data in prompts • Use anonymization where possible • validate and sanitize documents • Isolate untrusted content • Monitor and version vector index content • Deploy in compliant regions • Encrypt data, control access • Audit and log data usage • Log prompts, outputs, source • Use RAG with citation tracking • Provide traceable references in responses
19. Documents : Risks & Mitigations Overview ⚠ Risks Documents Confidentiality Level: Documents may contain sensitive or classified content not intended for LLM processing. Documents “Effective Dates”: LLMs may reference outdated or non-applicable documents. Data Quality Issues: Low-quality or inconsistent documents can result in misleading or incorrect responses. Data Lifecycle Management: Stale or irrelevant data may persist and pollute model behavior. Document Parser Document Document Loader Document Transformer Document Splitter Segment EmbeddingStore Embeddin g Segment Segment Embeddin g Embedding Embedding Model Split Already exist ? Split with no meaning ? Split too small ? 🛡 Mitigations • Tag documents with confidentiality levels • Enforce access control based on sensitivity • Exclude restricted documents from ingestion • Filter content based on date metadata • Prioritize recent or currently valid documents • Log version history and usage timeframes • Perform pre-ingestion quality checks • Normalize formats and metadata • Use semantic-aware chunking strategies • Validate chunks with retrieval quality tests • Implement TTL (time-to-live) for documents • Schedule regular cleanup and reindexing • Version and archive old datasets properly
20. ❏ Introduction ❏ Marketing platform ❏ Generative AI ❏ Content Creation ❏ Retrieval Augmented Generation ❏ Use your documents ❏ Adapt your tone of voices ❏ Agentic AI ❏ Assistants to help users ❏ Integration with tools and services ❏ Multi Agent systems ❏ AI Sessions and UI ❏ Agent Collaboration ❏ Agents Orchestration Agenda
21. Infrastructure Tenants, Users Marketing Campaign Tactics, Assets, Tracking Customer Data Platform Contacts, company, segments Email Marketing Designs, Templates Marketing Tools QR Code, URL Shortener Web Rest API Portal Documents ` PDF Images Bylines …. LLM LLM Connectors Social, CRM Cloud-Based, Multi-Tenant Platform Relational DATABASE NoSQL Database Generative AI Prompts Images Gen. LLM Connectors LLM Clients Text, Images Knowledge Base (vector db) Vector Database RAG
22. Infrastructure Tenants, Users Marketing Campaign Tactics, Assets, Tracking Customer Data Platform Contacts, company, segments Email Marketing Designs, Templates Marketing Tools QR Code, URL Shortener Web Rest API Portal Documents ` PDF Images Bylines …. LLM LLM Connectors Social, CRM Cloud-Based, Multi-Tenant Platform Relational DATABASE NoSQL Database AGENTIC AI Prompts Images Gen. LLM Connectors LLM Clients Text, Images Knowledge Base (vector db) Vector Database RAG Agent TOOLS
23. Agents ChatLanguageModel ChatMemory Prompt Template LLM Prompts template Chat Memory Knowlegde Base EmbeddingStore Content Retriever webSearch reranker Tools ScoringModel Content Aggregator QueryTransformer ToolsProvider
24. "AI executes code on your server—and decides when to do it.” → Sounds powerful… what could possibly go wrong? ⚠ Risks • Execution of incorrect or unintended instructions • Unauthorized access via tools with elevated privileges • Infinite or excessive tool invocation loops 🛡 Mitigations • Sanitize both inputs and outputs of the LLM • Human-in-the-loop validation before sensitive executions • Use local models cautiously, especially open-source ones • Throttle token and tool usage (per session/request) • Design small, well-scoped services for tools • Restrict permissions of tools to the minimum necessary Tool Usage in AI Systems AI is part of the information system and accountable to the business. Its responses can carry legal value.
25. Infrastructure Tenants, Users Marketing Campaign Tactics, Assets, Tracking Customer Data Platform Contacts, company, segments Email Marketing Designs, Templates Marketing Tools QR Code, URL Shortener Web Rest API Portal Documents ` PDF Images Bylines …. LLM LLM Connectors Social, CRM Cloud-Based, Multi-Tenant Platform Relational DATABASE NoSQL Database AGENTIC AI Prompts Images Gen. LLM Connectors LLM Clients Text, Images Knowledge Base (vector db) Vector Database RAG Agent TOOLS LLM LLM External Tools MCP Clients Tools
26. MCP Tools "AI executes code —that you did not write on your server - with its own runtime and decides when to do it.” It is the SOA of AI Tools: • Service Discovery • Service Invocations Mitigations: • New technologies like Api Gateways
27. Infrastructure Tenants, Users Marketing Campaign Tactics, Assets, Tracking Customer Data Platform Contacts, company, segments Email Marketing Designs, Templates Marketing Tools QR Code, URL Shortener Web Rest API Portal Documents ` PDF Images Bylines …. LLM LLM Connectors Social, CRM Cloud-Based, Multi-Tenant Platform Relational DATABASE NoSQL Database AGENTIC AI Prompts Images Gen. LLM Connectors LLM Clients Text, Images Knowledge Base (vector db) Vector Database RAG Agent TOOLS LLM LLM External Tools MCP Clients Tools CHATBOT
28. ❏ Introduction ❏ Marketing platform ❏ Generative AI ❏ Content Creation ❏ Retrieval Augmented Generation ❏ Use your documents ❏ Adapt your tone of voices ❏ Agentic AI ❏ Assistants to help users ❏ Integration with tools and services ❏ Multi Agent systems ❏ AI Sessions and UI ❏ Agent Collaboration ❏ Agents Orchestration Agenda
29. Agents with Lanchain4j LLM Prompts template Chat Memory Knowlegde Base webSearch reranker Tools
30. Multi-Agents Systems : Gap Analysis • Single Agent, Single Prompt • No Automation • No UI interaction • Limited user interactions V1 What you want • 1 session, multiple agents • Agents and SubAgents • Agents Collaboration • Agent User interaction • Agent driving UI • Agents Orchestration
31. A2A From Google
32. Agent Hierarchy General Purpose task-based (sub-agent) technical tools internal Internal or external custom custom Internal or external • General Purpose • Task Based • Technical Tool
33. Multi-Agents System AI Session AI Session AI Sessions (id) Start Agent Chat Active Agent Chat Message Chat Message Chat Message author Shared Context (Variables) UI
34. Agent Hierarchy AI Session AI Session AI Sessions (id) Start Agent Chat Active Agent Chat Message Chat Message Chat Message author Variables active_task Event Log UI Decisions Charts websocket REST - active_state active_task
35. Agents ⇔ User Interactions AI Session AI Session AI Sessions (id) Start Agent Chat Active Agent Chat Message Chat Message Chat Message author Shared Context Actions / Notifications UI Decisions sessionid Charts
36. Orchestration : Risks & Mitigations Overview ⚠ Risks Unclear UI Interaction Flow (No Precedence) Users may get confused if there is no clear order or visual guidance. Lack of Basic Rule Enforcement: Without rules, the system may behave inconsistently or accept invalid inputs. Unpredictable Model Behavior: LLMs are probabilistic — retries may yield different results or side effects. Implicit or Hidden Context : Users or developers may not understand what’s influencing the LLM’s output. 🛡 Mitigations • Define a clear interaction precedence in the UI • Guide users through step-by-step inputs • Use tooltips, progress indicators, or highlights • Implement complete interaction flow (front + back) • Connect user actions to agent logic and feedback loops • Simulate or handle fallback paths for incomplete flows • Add simple validation rules (input formats, required fields) • Design retry-safe operations (idempotency, checks) • Log decisions for traceability • Apply consistency guards for repeated inputs • Display active context or memory where relevant • Show current system/user prompt if needed
37. Agent Orchestration AI Session AI Session AI Sessions (id) Start Agent Chat Current Agent Chat Message Chat Message Chat Message author Shared Context Actions / Notifications UI Scheduler
38. GenAi OWASP Top 10 https://genai.owasp.org/llm-top-10
39. Thank you ! “The best way to predict the future ….is to create it” — Peter F. Drucker "Believe nothing you hear, and only one half that you see." — Edgar Allan Poe