Sensitivity: Public Securing Large Language Models: Threats and Mitigations Sena Yakut, Developer Summit 2024 

Sensitivity: Public ® aws sts get-caller-identity Sena Yakut, Cloud Security Architect @CyberWhiz All details, links about me: 

The Rise of the LLM 

LLM01: Prompt Injection How? . Similar to SQL injections . Malicious inputs as legitimate prompts . Override developer instructions Impact . Prompt leaks . Remote code execution . Data theft . Spread misinformation Remediation . Hard to prevent . Fllow least privilege principles . Input validation . Human in the loop 

LLM01: Prompt Injection 

LLM02: Insecure Output Handling How? . Outputs from LLM are not properly managed, validated before using in app Impact . Harmful content . Data leaks . Misinformation / Bias Remediation . Treat the model like a user . Zero trust approach . Validate the inputs / output filtering . Continuous monitoring 

LLM02: Insecure Output Handling 1 2 3 4 

LLM03: Training Data Poisoning How? . Inject false or mislead data . Manipulation of pre-training data . Deleting some parts of the data Impact . Reduce accuracy . Bias & Discrimination . Legal & Ethical Issues Remediation . Get data from trusted resources . Validate data quality / quality filtering . Data processing / validation . Privacy – Remove PII 

LLM03: Training Data Poisoning Tay AI Case: Tay AI was a chatbot developed by Microsoft, launched in 2016. By learning from the tweets it received, effectively adapting its responses based on user interactions. Within 24 hours of its launch, Tay began posting offensive and inappropriate tweets. 

LLM04: Model Denial of Service How? . Consumes exceptionally high amount of resources. . Large texts . Continuous input overflow . High volume generation of tasks Impact . Quality of service is decreasing . High costs . Unavailable services 5xx, 4xx Remediation . Enforce API rate limits. . Limit the number of queued actions. . Limit input sizes. . Monitor, alert, take action! 

LLM05: Supply Chain Vulnerabilities How? . Training data . Vulnerable pre-trained models . 3rd party software Impact . Loss of data integrity . Operational downtime . Unauthorized access Remediation . Up-to-date your software / libraries. . Implement a strong patch policy. . Model / code signing when using external models and suppliers. . Anomaly detection (analyze / alert latest vulnerabilities on LLMs – be aware) 

LLM05: Supply Chain Vulnerabilities 

LLM06: Sensitive Information Disclosure How? . Reveal sensitive info: PII, LLM algorithms, confidential details Impact . Unauthorized access to sensitive data . Privacy violations (GDPR, HIPAA etc) . LLMs interact with another LLMs –> Where is my confidential data? Remediation . Data anonymization . Role based access controls . Review / monitor / alert! 

LLM06: Sensitive Information Disclosure LLM Shield 

LLM07: Insecure Plugin Design How? . Lack of strong security controls . Misconfigured access controls . Untrusted libraries, packages Impact . Data breach . Unauthorized remote access / execution . Privilege escalation Remediation . Enforce parameterized inputs in plugins . Design minimalistic plugins – Less is more . Implement auth methods, API keys . For critical plugins → Manual user auth and approval 

LLM08: Excessive Agency How? . Permission exceeds necessary limits . Unexpected behaviors based on prompts . Unchecked agency poses risks Impact . System overload . Unwanted decisions . Unauthorized operations, interaction errors with another systems Remediation . Limit plugins / tools – Do not use lots of tools and plugins . Limit the functions that LLM can do . Manual checks are still important . Implement rate limiting – blocks if something goes crazy 

LLM09: Overreliance How? . Incorrect, inappropriate or unsafe information . LLMs can generate codes without security Impact . Reputational damage . Critical vulnerabilities and misconfigurations in applications . Fake news Remediation . Cross checks – Trust but verify . Regularly monitor and review the LLM outputs . Break down – Complex tasks → Subtasks → Assign them different agents 

LLM09: Overreliance 

LLM10: Model Theft How? . Unauthorized access and exfiltration of LLM models . With supply chain attacks – could be very complex but possible . Model republishing – Non tech, without your permission . Model extraction – querying model, analyzing results Impact . Brand reputation loss . Unexpected costs in your cloud env Remediation . Strong access controls . Restrict LLM access to network resources . Monitor/audit and alert 

Sensitivity: Public Securing Large Language Models: Threats and Mitigations Sena Yakut, Developer Summit 2024