HTTP Header Heuristics

This is a presentation based on the paper "HTTP Header Heuristics for Malware Detection", by Tobias Lewis. The paper is in the SANS Institute InfoSec Reading Room.

http://www.sans.org/reading-room/whitepapers/detection/http-header-heuristics-malware-detection-34460

1. HTTP Header Heuristics
   for Malware Detection
   SANS Institute InfoSec Reading Room Author: Tobias Lewis link to paper ! Presented by: Clay Wells (Sr. Systems Programmer) Department of Biostatistics & Epidemiology University of Pennsylvania
2. HTTP Header Heuristics for Malware Detection Overview Introduction Analysis of HTTP heuristics Recommendations Conclusion Resources
3. HTTP Header Heuristics for Malware Detection Introduction Methods for malware detection - Signature based “known knowns” - Heuristics, rules and patterns “unknown”
4. HTTP Header Heuristics for Malware Detection Introduction Basic assumptions ! - Malware will “phone-home” (C&C) - Malware masquerades as legitimate web traffic - Uses Hypertext Transfer Protocol, HTTP
5. HTTP Header Heuristics for Malware Detection Introduction HTTP ! - Application layer protocol (client-server) - Transfers information using headers ! “allows the client to pass additional information about the request, and about the client itself, to the server”! - RFC2616!
6. HTTP Header Heuristics for Malware Detection Introduction HTTP request-header fields ! - User-Agent: - Host: *required - Referer:
7. HTTP Header Heuristics for Malware Detection Introduction User-Agent ! "User-Agent" ":" 1*( product | comment )! ! User-Agent: CERN-LineMode/2.15 libwww/2.17b3!
8. HTTP Header Heuristics for Malware Detection Introduction Host ! Host = "Host" ":" host [ ":" port ]! ! Host: www.w3.org!
9. HTTP Header Heuristics for Malware Detection Introduction Referer ! Referer = "Referer" ":" ( absoluteURI | relativeURI )! ! https://encrypted.google.com/#q=w3c
10. HTTP Header Heuristics for Malware Detection Overview Introduction Analysis of HTTP heuristics Recommendations Conclusion Resources
11. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics General heuristics ! - User-Agents - Typographic errors - URL complexity ! Using p0f for generic malware detection ! - Introduction - Demonstration - Deployment options - Considerations
12. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics User-Agents ! Proxy log example ! ! !
13. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics User-Agents ! - fairly static, but.. - a large number exist with BYOD, etc - works best when fine-tuned
14. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics User-Agents, SNORT based signatures ! - doesn’t present User-Agent of a possible standard build - doesn’t contain the correct OS
15. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics Typographic errors ! - Explicitly hardcoded header fields MEDIANA PROTUX QUARIAN
16. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics Typographic errors, MEDIANA ! Surplus white space (ACSII 0x20)
17. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics Typographic errors, MEDIANA ! SNORT rule
18. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics Typographic errors, PROTUX ! Surplus white space in URL
19. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics Typographic errors, QUARIAN ! Other incorrect header fields
20. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics URL complexity ! - Some complexity will exist (valid Referer) - Servers shouldn’t rely on > 255 characters
21. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics URL complexity, IXESHE ! Absent Referer field
22. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics URL complexity, TAIDOOR ! Absent Referer field
23. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics URL complexity, MONGALL ! More extreme example
24. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics URL complexity, NETTRAVELER ! More extreme example
25. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics
26. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics URL complexity ! Long & complex URLs do exist in legitimate traffic ! - Microsoft-CryptoAPI service - Banner ads
27. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics URL complexity ! - Fine tune for your environment - Use with other heuristics
28. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics URL complexity ! Snort rule using “urilen”
29. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics General heuristics ! - User-Agents - Typographic errors - URL complexity ! Using p0f for generic malware detection ! - Introduction - Demonstration - Deployment options - Considerations
30. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics p0f: Introduction ! - Passive fingerprinting tool - Assuming malware uses TCP library of the host OS - Uses structure of OSI model ! TTL, Window Size, Sequence numbers ! References a fingerprint library
31. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics p0f: Demonstration ! - Generated traffic using wget and Firefox (header plug-in) - Conducted from a Windows 7 machine - Used a fixed User-Agent claiming to be IE6 on XP - Captured data using Fakenet
32. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics wget
33. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics Firefox modified
34. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics NETTRAVELER
35. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics Legitimate traffic
36. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics p0f: Deployment options ! - gateway devices - part of a larger suite of network monitoring tools (using API)
37. HTTP Header Heuristics for Malware Detection Analysis of HTTP heuristics p0f: Considerations ! - Examines at the application and transport layers - Struggles to reliably detect the host OS - Relies on the spoof-ability of the malware author - Identifies malicious behavior fairly well - No tool or tool set will ever be 100% accurate!
38. HTTP Header Heuristics for Malware Detection Overview Introduction Analysis of HTTP heuristics Recommendations Conclusion Resources
39. HTTP Header Heuristics for Malware Detection Recommendations - Baseline your network - Lower the noise floor where possible - Heuristics work best when fine-tuned to the environment - Don’t rely on a single method for detection
40. HTTP Header Heuristics for Malware Detection Overview Introduction Analysis of HTTP heuristics Recommendations Conclusion Resources
41. HTTP Header Heuristics for Malware Detection Conclusion - Heuristics can be a useful tool in your arsenal - Compliments signature based IDS or host based
42. HTTP Header Heuristics for Malware Detection Resources Link to paper RFC2616, HTTP/1.1 http://practicalmalwareanalysis.com/fakenet/ p0f www.snort.org
43. HTTP Header Heuristics for Malware Detection 2014-02-06: User-Agent strings! ! Fedora release 19! -----------------! ! Firefox: "Mozilla/5.0 (X11; Linux i686; rv:26.0) Gecko/20100101 Firefox/26.0"! ! Chrome: "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36"! ! Seamonkey: "Mozilla/5.0 (X11; Linux i686; rv:26.0) Gecko/20100101 Firefox/26.0 SeaMonkey/2.23"! ! Midori: "Mozilla/5.0 (X11; Linux) AppleWebKit/537.32 (KHTML, like Gecko) Chrome/18.0.1025.133 Safari/537.32 Midori/ 0.5"! ! ! OS X 10.9.1! -----------! ! Firefox: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:26.0) Gecko/20100101 Firefox/26.0"! ! Chrome: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36"! ! Safari: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.73.11 (KHTML, like Gecko) Version/7.0.1 Safari/537.73.11"! ! ! Windows XP! ----------! ! Firefox: "Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0"! ! IE: "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; InfoPath.1; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 1.1.4322; .NET4.0C; .NET4.0E)"! ! Chrome: "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36"! ! Safari: "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/534.57.2 (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2"
44. HTTP Header Heuristics for Malware Detection Thank You! everyone ! www.sans.org !