Security Implications in A.I

Guest lecture at VIT University Chennai.

Transcript

1. FOR INTERNAL USE Security Implications in A.I SANTHOSH KUMAR PRINCIPAL INFORMATION SECURITY ENGINEER - DHL
2. FOR INTERNAL USE About me! • Principal Security Engineer at DHL, previously with IBM and EY India LLP. • Been in this industry for 7 years now J Played both Attacking and Defending sides in the previous roles J • Spoke at Defcon Las vegas, OWASP Appsec USA toorcon san diego, Bsides las vegas etc…. • Now i do a lot of security architecture work. • Holds industry certs such as CISSP, CISM, SABSA SCF OSCP, OSCE, OSWE, Crest CRT , CRTP etc.
3. FOR INTERNAL USE Tl;Dr about this talk ? u This talk will have quick introduction to Artificial intelligence in our case will be Large Language models(LLM). u I will try to keep theory part minimal, and we will do the hands-on demo for most of the scenarios that is to be discussed in this talk. u This talk will have a considerable amount of rant on A.I :) u The talk will revolve around the industry practices and experiences that current adaption of A.I in enterprises and risks that we are facing with these quick adaption. u Incase you have any questions feel free to stop me anytime and I will answer your questions J
4. FOR INTERNAL USE Artificial intelligence?
5. FOR INTERNAL USE What it actually is?
6. FOR INTERNAL USE A.I Boom! u A little bit of History J A.I has been there from 1950s. It started gaining traction from 2012. u Google, apple, facebook, Alibaba, tesla etc all did the work before 2020. u OpenAI started beta testing GPT-3 a model that uses Deep Learning to create code, poetry, and other such language and writing tasks in 2020 u At this time where A.I started seeing the mainstream media focus. u ChatGPT was released on end of 2022 which powered by gpt 3.5
7. FOR INTERNAL USE Large Language models(LLMs) u Large language models (LLMs) are a type of artificial intelligence (AI) program that can recognize and generate text, and perform other natural language processing (NLP) tasks. u LLMs are trained on large sets of data, such as programming languages, and are made up of multiple neural network layers. u LLMs are complex and resource-intensive, and can take months to train and consume lots of resources. u Large language models are trained using unsupervised learning. With unsupervised learning, models can find previously unknown patterns in data using unlabelled datasets. This also eliminates the need for extensive data labeling, which is one of the biggest challenges in building AI models.
8. FOR INTERNAL USE A.I Risks? NCSC Uk survey
9. FOR INTERNAL USE 5 Top Risk of A.I
10. FOR INTERNAL USE Regulations targeting A.I u NIST AI 100-2 E2023 Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations u UK Information Commisioner's Office (ICO) Generative AI: eight questions that developers and users need to ask u UK National Cyber Security Centre (NCSC) Principles for the security of machine learning u EU AI Act EU AI Act: first regulation on artificial intelligence u INDIA -THE DIGITAL PERSONAL DATA PROTECTION ACT
11. FOR INTERNAL USE OWASP Top 10 LLMs
12. FOR INTERNAL USE LLMs Security Risk
13. FOR INTERNAL USE LLM01 – Prompt Injections u We are going to look into a live scenario where we will abuse a application in leveraging in LLMs to perform a malicious function. u The demo application runs a ChatBot which is intergrated with openAI API u The chatbot LLMs interact with the backend Application. u LLMs doesn’t santise any inputs when given by user(in our case it is us) DEMO…………
14. FOR INTERNAL USE LLM07 – InSecure Plugin Design u This is again a similar scenario but we are going to do something little complex u We are going to change LLM01 + LLM07 which is we use the backend API vulnerability u The Backend API suffers from Input validation issue. u We are going to convert this into RCE – Remote code execution DEMO …………
15. FOR INTERNAL USE A.I Rant u Overreliance and quick adaption into the business space does have its own chaotic outputs.
16. FOR INTERNAL USE AI Deception: How Tech Companies Are Fooling Us u This video from Coldfusion an youtube channel focusing on STEM is a good eye opener. https://www.youtube.com/watch?v=vQChW_jgMMM I would suggest to see this video on getting a real idea on how much influence AI really have and how much it can do at this stage . IN SHORT A.I will not replace us J but it can make our life easier J
17. FOR INTERNAL USE References: u https://www.morganlewis.com/blogs/sourcingatmorganlewis/2024/ 01/ai-regulation-in-india-current-state-and-future-perspectives u https://www.europarl.europa.eu/topics/en/article/20230601STO9380 4/eu-ai-act-first-regulation-on-artificial-intelligence u https://www.ncsc.gov.uk/collection/machine-learning u https://owasp.org/www-project-top-10-for-large-language-model- applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-v05.pdf u https://portswigger.net/web-security/llm-attacks u https://github.com/OWASP/www-project-top-10-for-large- language-model-applications/wiki/Educational-Resources
18. FOR INTERNAL USE Thank You J My Linkedin : https://www.linkedin.com/in/santhoshkumar22/ My Email: santhosh@nearsecurity.com