Bypassing the Gatekeepers: LLM Enabled Techniques for Circumventing WAFs at Scale

Research presented at SAINTCON in Provo Utah on Friday Oct 22nd 2024.

Session Name: "Bypassing the Gatekeepers: LLM Enabled Techniques for Circumventing WAFs at Scale"

Web Application Firewalls (WAFs) have become ubiquitous in modern enterprise environments, presenting a significant challenge for security researchers and bug bounty hunters. This talk delves into cutting-edge techniques for bypassing WAFs on large-scale attack surfaces, empowering attendees to uncover critical vulnerabilities even in heavily protected environments.

We'll explore how innovations in Large Language Models (LLMs) can revolutionize penetration testing processes, focusing on their application in WAF circumvention. From crafting sophisticated payloads to automating reconnaissance, we'll demonstrate how AI can augment human creativity in offensive security.

This presentation will showcase real-world case studies, practical tools, and live demonstrations of AI-assisted WAF bypasses. We'll also discuss the ethical implications and potential defensive countermeasures, providing a balanced view of this rapidly evolving field.

Attendees will gain hands-on knowledge of implementing LLM-driven techniques, understanding both their power and limitations. Join us to glimpse the future of AI-driven offensive security and its critical role in shaping the cybersecurity landscape.

Transcript

1. BYPASSING GATEKEEPERS LLM Enabled Techniques for Circumventing WAFs at Scale PRESENTED BY BRYCE KUNZ @TWEEKFAWKES GAMMAXON . COM OCTOBER 2024
2. INTRODUCTION GAMMAXON . COM Bryce Kunz // @TweekFawkes
3. GAMMAXON . COM As organizations digitize their operations, the number of connected devices, systems, and cloud instances increases exponentially, providing cybercriminals with an ever-expanding range of potential vulnerabilities to exploit. #1 - EXPANDING & COMPLEX ATTACK SURFACES As enterprises grow, their applications and network infrastructure must scale accordingly. Maintaining full visibility across all network environments, especially in multi-cloud setups, is a major challenge. #2 - SCALING APPLICATION SECURITY & THREAT MITIGATION PROBLEMS
4. HYPOTHESIS Continuously identifying, monitoring, and managing an organization's entire attack surface will significantly reduce the risk of successful cyberattacks by minimizing vulnerabilities and potential attack vectors. Many organizations are unaware of a significant portion of their digital assets. These hidden assets, AKA Shadow IT, pose a greater risk then known assets. On average, attack surface management tools discover 35% more assets than company leaders were previously aware of. GAMMAXON . COM SPEED AWARENESS
5. GAMMAXON . COM METHODOLOGY Modular approach to application development, allowing teams to build applications by composing individual functions. This enables parallel development, code reusability, and easier maintenance of complex systems. MODULAR Leverage serverless tech to scale operations in an effort to combat the massive scale of Internet facing risks enterprises are exposed to everyday. Automatically scale up or down based on incoming workload without manual intervention. SCALABLE
6. Run each module on a set schedule CRON JOB Each module runs a small script SCRIPT All scripts output to cloud storage (e.g. S3) STORAGE Infrastructure as Code (IaC) framework to make updates to serverless microservice IAC IMPLEMENTATION GAMMAXON . COM
7. Run each module on a set schedule CLOUDWATCH DESIGN GAMMAXON . COM S3 Store Jobs Run each module on a set schedule LAMBDA S3 Store Results
8. GAMMAXON . COM Focusing on one task at a time, you can give it your full attention and complete it more quickly and effectively. Multitasking actually reduces productivity by up to 40% due to the mental effort of constantly switching between tasks. ENHANCED PRODUCTIVITY AND EFFICIENCY When you're focused on one thing, you're less likely to make mistakes or produce subpar results compared to dividing your attention across multiple tasks. FEWER ERRORS AND HIGHER QUALITY WORK FOCUS ON A PROBLEM
9. CONTENT DISCOVERY Through content discovery, testers can often find sensitive files, directories, or resources that were not intended to be publicly accessible. This could include things like: Backup files Configuration files Administrative interfaces Development/staging environments Older versions of pages/files Discovering such sensitive content can reveal vulnerabilities or lead to further exploitation paths. GAMMAXON . COM IDENTIFICATION OF SENSITIVE INFORMATION
10. BLOCKED! GAMMAXON . COM ....
11. WEB APP FIREWALLS A WAF may block or restrict access to certain paths, directories, or file types that the tester is trying to discover. This can significantly limit the tester's ability to identify potentially vulnerable or sensitive areas of the application. #1 - RESTRICTED ACCESS Many WAFs implement rate limiting, which can slow down the content discovery process. This limitation on the number of requests a tester can send in a given time period can significantly extend the duration of testing. #3 - RATE LIMITING The WAF may alter the application's responses, making it challenging for the tester to understand the true behavior of the underlying application. This can hinder the identification of potential vulnerabilities or misconfigurations. #2 - MASKING OF APP BEHAVIOR GAMMAXON . COM
12. HYPOTHESIS Attackers use techniques to disguise their network identity, including but not limited to: Using botnets and/or proxy services to rotate through multiple IP addresses, making it difficult for WAFs to track and block malicious traffic consistently. GAMMAXON . COM ROTATING IPS == BYPASS WAF ... MAYBE? ¯\_(ツ)_/¯
13. GAMMAXON . COM METHODOLOGY Leverage existing techniques and/or open source tools to bypass the WAFs. REUSE PUBLIC TECHNIQUES
14. TYPICAL SETUP Cloud Based Service IP: 200.200.200.200 Open Services/Ports: 80/TCP Open HTTP 443/TCP Open HTTPS GAMMAXON . COM Internet Facing IP: 100.100.100.100 Open Services/Ports: 80/TCP Open HTTP 443/TCP Open HTTPS SERVERS https://www.youtube.com/watch?v=jfjzYpgte-A WAF SaaS WAF Provider ADMIN WEB PORTAL Residential ISP IP: 70.70.70.70 USER IP: 50.50.50.50 RED TEAM REVERSE PROXY Optional
15. REVERSE PROXY Optional Cloud Based Service IP: 200.200.200.200 Open Services/Ports: 80/TCP Open HTTP 443/TCP Open HTTPS GAMMAXON . COM Internet Facing IP: 100.100.100.100 Open Services/Ports: 80/TCP Open HTTP 443/TCP Open HTTPS SERVERS https://www.youtube.com/watch?v=jfjzYpgte-A WAF SaaS WAF Provider ADMIN WEB PORTAL Residential ISP IP: 70.70.70.70 USER IP: 50.50.50.50 RED TEAM Censys, etc. OSINT ORIGIN IP
16. GAMMAXON . COM PROS & CONS Hard to Scale via pure automation e.g. need human to analyze outputs from censys, etc. Doesn’t Work Often CONS Simple to Implement PROS
17. TYPICAL SETUP Cloud Based Service IP: 200.200.200.200 Open Services/Ports: 80/TCP Open HTTP 443/TCP Open HTTPS GAMMAXON . COM Internet Facing IP: 100.100.100.100 Open Services/Ports: 80/TCP Open HTTP 443/TCP Open HTTPS SERVERS https://www.youtube.com/watch?v=jfjzYpgte-A WAF SaaS WAF Provider ADMIN WEB PORTAL Residential ISP IP: 70.70.70.70 USER IP: 50.50.50.50 RED TEAM REVERSE PROXY Optional
18. SIGN UP FOR WAF REVERSE PROXY Optional Cloud Based Service GAMMAXON . COM Internet Facing IP: 100.100.100.100 Open Services/Ports: 80/TCP Open HTTP 443/TCP Open HTTPS SERVERS https://certitude.consulting/blog/en/using-cloudflare-to-bypass-cloudflare/ WAF SaaS WAF Provider ADMIN WEB PORTAL Residential ISP IP: 70.70.70.70 USER IP: 50.50.50.50 RED TEAM
19. BE THE WAF Optional Cloud Based Service IP: 50.50.50.50 RED TEAM REVERSE PROXY GAMMAXON . COM Internet Facing IP: 100.100.100.100 Open Services/Ports: 80/TCP Open HTTP 443/TCP Open HTTPS SERVERS https://certitude.consulting/blog/en/using-cloudflare-to-bypass-cloudflare/ WAF SaaS WAF Provider ADMIN WEB PORTAL Residential ISP IP: 70.70.70.70 USER
20. WAF IS ALLOWED Optional Cloud Based Service IP: 50.50.50.50 RED TEAM REVERSE PROXY GAMMAXON . COM Internet Facing IP: 100.100.100.100 Open Services/Ports: 80/TCP Open HTTP 443/TCP Open HTTPS SERVERS https://certitude.consulting/blog/en/using-cloudflare-to-bypass-cloudflare/ WAF SaaS WAF Provider ADMIN WEB PORTAL Residential ISP IP: 70.70.70.70 USER
21. GAMMAXON . COM PROS & CONS Multiple Steps to Implement Hard to Scale via pure automation e.g. need human to setup WAFs, etc. May not work every time CONS Effective PROS
22. IP ROTATE GAMMAXON . COM https://portswigger.net/bappstore/2eb2b1cb1cf34cc79cda36f0f9019874
23. GAMMAXON . COM PROS & CONS Designed for a Single Target Hence will not out of the box scale to 1000s of targets Requires Burp Suite Mostly Operator Driven (e.g. No APIs) API Gateway Limits per Region 600 Regional APIs 120 Edge-optimized APIs 600 Private APIs Request rate limit is separate and is set at 10,000 requests per second (RPS) across all APIs in an account per region CONS Simple to Implement New Source IP Address with Every Request No Charge $ In AWS for Making New API Gateways PROS
24. FIREPROX GAMMAXON . COM https://github.com/ustayready/fireprox
25. GAMMAXON . COM PROS & CONS Designed for a Single Target Hence will not out of the box scale to 1000s of targets Requires Burp Suite Mostly Operator Driven (e.g. No APIs) API Gateway Limits per Region 600 Regional APIs 120 Edge-optimized APIs 600 Private APIs Request rate limit is separate and is set at 10,000 requests per second (RPS) across all APIs in an account per region CONS Simple to Implement New Source IP Address with Every Request No Charge $ In AWS for Making New API Gateways PROS
26. ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions. SPLITS WORDLISTS Lithops is a Python multi-cloud serverless computing framework. It allows to run unmodified local python code at massive scale in the main serverless computing platforms. SHADOW CLONE GAMMAXON . COM https://github.com/fyoorer/ShadowClone
27. GAMMAXON . COM PROS & CONS Source IPs Are Typically In Chunks So Locked Down WAFs May Still Block Most Requests Mostly Operator Driven (e.g. No APIs) Default Limits (But Can Easily Request Upgrades) function and layer storage is 75 GB per region concurrent executions is 1,000 per region CONS Simple to Implement Flexible Out of the Box Support for Many Common Tools PROS
28. GAMMAXON . COM METHODOLOGY Build new custom tools to rotate IP source addresses and bypass the WAFs, which will work at scale, against thousands of targeted servers. CUSTOM
29. RESIDENTIAL PROXY GAMMAXON . COM Internet Facing SERVERS Each module runs a small script SCRIPT https://smartproxy.com Residential Proxy PROXY Proxy Provider SAAS
30. GAMMAXON . COM PROS & CONS Questionable How Source IPs are Acquired... “Ethically” Sourced IPs CONS Very Simple to Implement Very Flexible Out of the Box Support for Many Common Tools API Driven Source IPs Can Rotate On Every HTTP Request PROS
31. SERVERLESS Run each module on a set schedule Run each module on a set schedule CLOUDWATCH GAMMAXON . COM S3 Store Jobs LAMBDA S3 Store Results Internet Facing SERVERS CONTAINER Custom Tools from Dockerfile
32. GAMMAXON . COM PROS & CONS Source IPs Do NOT Rotate Until the Container Goes Cold CONS Simple to Implement Very Flexible Out of the Box Support for Many Common Tools API Driven NOTE: Do NOT have to create new API Gateways for each target Hence will easily scale to thousands of targets PROS
33. GAMMAXON . COM COLD & WARM After a function executes, the execution environment is frozen and retained for a non-deterministic period. If another request for the same function arrives during this time: The Lambda service may reuse the existing environment. This results in a faster execution since the environment is already set up. There's no need to download the code or run initialization code again. This reuse of an existing environment is called a "warm start". WARM STARTS A cold start occurs when a Lambda function is invoked for the first time or after a period of inactivity. During a cold start: The Lambda service prepares a new execution environment. It downloads the function code from S3 or ECR. The environment is set up with the specified memory, runtime, and configuration. Any initialization code outside the event handler is executed. Finally, the handler code runs. You are not charged for the time it takes Lambda to prepare the function. COLD STARTS
34. GAMMAXON . COM ARE WE WARM? Files in /tmp persist in the warm state... So, Write file to /tmp called “WARM” and check on first boot if the file exists HOW TO DETERMINE IF CNTR IS WARM? Python Requests to https://checkip.amazonaws.com HOW TO DETERMINE SOURCE IP?
35. GAMMAXON . COM If we can force the container to go into a cold state, then we can most likely get a new IP address for each request. #1 - CAN WE FORCE A LAMBDA TO GO COLD? PROBLEMS
36. try throw error except exit(1) Exit Status: It sets the exit status of the program to 1, which conventionally indicates that an error or problem occurred during execution VALUE ERROR THROW ERROR GAMMAXON . COM
37. try throw error except exit(1) Exit Status: It sets the exit status of the program to 1, which conventionally indicates that an error or problem occurred during execution VALUE ERROR THROW ERROR GAMMAXON . COM
38. Create a Segfault CALL OUT TO C BINARY SEGFAULT BINARY GAMMAXON . COM ...
39. Create a Segfault CALL OUT TO C BINARY SEGFAULT BINARY GAMMAXON . COM ...
40. REDDIT GAMMAXON . COM https://www.reddit.com/r/aws/comments/12s72a6/aws_lambda_cold_start_on_demand_making_lambda/
41. Rotate Value on Each Invoke of Lambda Function CREATED A UUID ENV VAR UPDATE ENV VAR GAMMAXON . COM https://www.reddit.com/r/aws/comments/12s72a6/aws_lambda_cold_start_on_demand_making_lambda/
42. Rotate Value on Each Invoke of Lambda Function CREATED A UUID ENV VAR UPDATE ENV VAR GAMMAXON . COM https://www.reddit.com/r/aws/comments/12s72a6/aws_lambda_cold_start_on_demand_making_lambda/
43. throw error THROW AN UNCAUGHT EXCEPTION UNCAUGHT EXCEPT GAMMAXON . COM https://www.reddit.com/r/aws/comments/12s72a6/aws_lambda_cold_start_on_demand_making_lambda/
44. throw error THROW AN UNCAUGHT EXCEPTION UNCAUGHT EXCEPT GAMMAXON . COM https://www.reddit.com/r/aws/comments/12s72a6/aws_lambda_cold_start_on_demand_making_lambda/
45. IDK ¯ \_(ツ)_/¯ GAMMAXON . COM ...
46. GAMMAXON . COM If we can force the container to go into a cold state, then we can most likely get a new IP address for each request. #1 - CAN WE FORCE A LAMBDA TO GO COLD? PROBLEMS
47. DIRB SPRAY V1 Run each module executed via Invoke Invoke Lambda Function Send Job Info LAMBDA SDK BOTO3 GAMMAXON . COM S3 Store Results Internet Facing SERVERS CONTAINER Custom Tools from Dockerfile Residential Proxy PROXY Container Images ECR
48. DIRB SPRAY V2 Run each module on a set schedule LAMBDA GAMMAXON . COM S3 Store Results Internet Facing SERVERS CONTAINER Custom Tools from Dockerfile Container Images ECR Source IP to Job JOBS Run each module on a set schedule CLOUDWATCH
49. ANALYZE OUTPUT Run each module on a set schedule Invoke Lambda Function Send Job Info LAMBDA SDK BOTO3 GAMMAXON . COM S3 Store Results Internet Facing SERVERS CONTAINER Custom Tools from Dockerfile Residential Proxy PROXY Container Images ECR Many Apps Do Not Respond The Same Way e.g. status codes, reply length, etc. WHAT IS NORMAL?
50. HTTP GET / Random URI Content Discovery 200 (OK) 404 (Not Found) 404 -> Miss 200 -> Hit ~4933 to ~4954 ~582 to ~590 No Exact Sizes but Ranges are Pretty Close HTML... HTML... HTML... STATUS CODES GAMMAXON . COM
51. HTTP GET / Random URI Content Discovery 200 (OK) 200 (OK) 200 (OK) ~4933 to ~4954 ~582 to ~590 No Exact Sizes but Ranges are Pretty Close HTML... HTML... HTML... REPLY LENGTH GAMMAXON . COM
52. HTTP GET / Random URI Content Discovery 200 (OK) 200 (OK) 200 (OK) Random Like Random Like Random Like HTML... <title>404 - Page not found</title> Miss -> <title>404 - Page not found</title> CONTENT BODY GAMMAXON . COM
53. HTTP GET / Random URI Content Discovery 200 (OK) 1x -> 200 (OK) 2x -> 404 (Not Found) a few -> 404 (Not Found) mostly -> 401 (Unauthorized) Random Like Random Like Random Like HTML... different HTML messages A few different HTML messages MANY WEIRD APPS GAMMAXON . COM
54. GAMMAXON . COM METHODOLOGY Leverage LLMs to Augment and Scale Capabilites LLMS
55. Gemini reportedly has a much larger context window (up to 2 million tokens) compared to ChatGPT, allowing it to handle more extensive inputs. Gemini 1.5 Pro 2M context window, code execution capabilities, and Gemma 2 are available as of JUN 27, 2024 More Context means More Supporting Documents can be used with the query! GOOGLE GEMINI LARGEST CONTEXT GAMMAXON . COM ...
56. PROMPT - PART 1 GAMMAXON . COM ...
57. PROMPT - PART 2 GAMMAXON . COM ...
58. PROMPT - PART 3 GAMMAXON . COM ...
59. PROMPT - PART 4 GAMMAXON . COM ...
60. PROMPT - PART 5 GAMMAXON . COM ...
61. PROMPT - PART 6 GAMMAXON . COM ...
62. RECOMMENDATION Leverage LLMs to Augment and Scale Capabilities Reduce Time Spent on Tedious Tasks RECOMMENDATION - 2 Good for a limited number of targets (e.g. one target at a time): API Gateways Good for Scale: Residential Proxies (but does cost some $) RECOMMENDATION - 1 GAMMAXON . COM
63. THANK YOU SO MUCH TweekFawkes.com BryceKunz@gmail.com @BryceKunz.99 @TweekFawkes Website : Signal : Social Media : Email Address : GAMMAXON . COM