Classifiers Under Attack

Presentation at O'Reilly Security
New York, 2 November 2016

http://conferences.oreilly.com/security/network-data-security-ny/public/schedule/detail/53176

Transcript

1. Classifiers Under Attack David Evans University of Virginia evans@virginia.edu evadeML.org work with Weilin Xu and Yanjun Qi
2. Machine Learning is Solving All Our Problems! 2 Fake Spam IDS Malware Fake Accounts … …
3. Machine Learning is Eating the World 3 Data Scientist Security Expert ?
4. Assumption: Training Data is Representative 4 Labelled Training Data ML Algorithm Feature Extraction Vectors Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning)
5. Reality: Adversary Adapts ACM CCS 2016 Actual images Recognized faces
6. Case study: Evading PDF Malware Classifiers
7. Malware Classifiers in Practice Goal: Automatically simulate adaptive adversary against generic classifier Purpose: Understand classifier robustness Build better classifiers (?)
8. 0 20 40 60 80 100 120 140 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 (1/4) Vulnerabilities reported in Adobe Acrobat Reader Source: http://www.cvedetails.com/vulnerability-list/vendor_id-53/product_id-497/year-2016/Adobe-Acrobat-Reader.html
9. High-Value Exploits: MiniDuke Source: https://cdn.securelist.com/files/2014/07/themysteryofthepdf0-dayassemblermicrobackdoor.pdf
10. PDF Malware 1 0 Exploits CVE- 2007-5659 buffer overflow
11. PDF Malware Classifiers PDFrate [ACSA 2012] Hidost16 [JIS 2016] Hidost13 [NDSS 2013] Random Forest Random Forest Support Vector Machine
12. Random Forest x y w 0 1 z 1 0 1 r q 0 z 0 0 y 0 1 Generate many random decision trees Train independently Select best trees Vote on result
13. PDF Malware Classifiers PDFrate [ACSA 2012] Hidost16 [JIS 2016] Hidost13 [NDSS 2013] Random Forest Random Forest Support Vector Machine Features Object counts, lengths, positions, etc. Object structural paths Very robust against “strongest conceivable mimicry attack”.
14. Classifier Performance 14 PDFrate* Hidost Accuracy 0.9976 0.9996 False Negative Rate 0.0000 0.0056 * Mimicus [Oakland ’14], an open source reimplementation of PDFrate.
15. Classifier Performance 15 PDFrate* Hidost Accuracy 0.9976 0.9996 False Negative Rate 0.0000 0.0056 Adversarial False Negative Rate 1.0000 1.0000 * Mimicus [Oakland ’14], an open source reimplementation of PDFrate.
16. Automatically Evading Classifiers
17. Variants Automated Classifier Evasion Using Genetic Programming 17 Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive?
18. Variants Goal: Find Evasive Variant 18 Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Evasive variant: Benign Simulated attacker’s goal: find a variant that is classified as benign, but exhibits the same malicious behavior.
19. PDF Structure 1 9
20. Malicious Seed 20 Clone Malicious PDF Modified Parser 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Parser is “robust” version of pdfrw: - Handles ungrammatical PDFs - Ignores inconsistencies, etc. Malware often malformed
21. Variants Generating Variants 21 Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓
22. Variants Generating Variants 22 Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants Variants Variants Select Variants ✓ ✓ ✗ ✓ 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Select random node
23. Variants Generating Variants 23 Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Variants Variants Select Variants ✓ ✓ ✗ ✓ Select random node 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Random transform: delete, insert, replace
24. Variants Generating Variants 24 Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Variants Variants Select Variants ✓ ✓ ✗ ✓ Nodes from Benign PDFs 128 546 7 63 Select random node 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Random transform: delete, insert, replace 128
25. Next Generation Selecting Promising Variants 25 Clone Benign PDFs Malicious PDF Mutation Variants Generated Variants Select Variants ✓ ✓ ✗ ✓
26. Selecting Promising Variants 26 Clone Generated Variants Select Variants ✓ ✓ ✗ ✓ Clone Variants Fitness Function Candidate Variant ($%&'() , '(&++ ) Score Malicious Benign PDFs Malicious PDF Variants Benign PDFs Malicious PDF Variants Oracle Variant 0 /JavaScript eval(‘…’); /Root /Catalog /Pages 128 Oracle Target Classifier
27. Oracle Execute candidate in vulnerable Adobe Reader in virtual environment Behavioral signature: only considered malicious if signature matches https://github.com/cuckoosandbox Simulated network: INetSim HTTP_URL + HOST extracted from API traces
28. Fitness Function 28 Assumes lost malicious behavior will not be recovered = 0 .5 − classifier_score if oracle = "malicious" −∞ otherwise classifier_score ≥ 0.5: labeled malicious
29. Experimental Results
30. 30 Original Malicious Seeds Evading PDFrate Malicious Label Threshold
31. 31 Original Malicious Seeds Evading PDFrate Discovered Evasive Variants 100% success rate ∼130 hours on typical desktop
32. 32 Evading Hidost2013 100% success rate ∼46 hours on typical desktop Original Malicious Seeds Discovered Evasive Variants
33. Evading Hidost2016
34. Evading Hidost2016 100% success rate ∼14 hours on typical desktop
35. PDFRate Hidost Seeds Evaded
36. PDFRate Hidost Seeds Evaded Simple transformations often worked
37. PDFRate Seeds Evaded (insert, /Root/Pages/Kids, 3:/Root/Pages/Kids/4/Kids/5/) Inserting new pages works on 162/500 seeds Training malware often had no/little content
38. Hidost Seeds Evaded (delete, /Root/OpenAction/JS/Length) Deleting object worked on 1 seed No impact on malicious behavior
39. PDFRate Hidost Seeds Evaded Some seeds required complex transformations
40. Complex Transformations Insert: Threads, ViewerPreferences/Direction, Metadata, Metadata/Length, Metadata/Subtype, Metadata/Type, OpenAction/Contents, OpenAction/Contents/Filter, OpenAction/Contents/Length, Pages/MediaBox Delete: AcroForm, Names/JavaSCript/Names/S, AcroForm/DR/Encoding/PDFDocEncoding, AcroForm/DR/Encoding/PDFDocEncoding/Differences, AcroForm/DR/Encoding/PDFDocEncoding/Type, Pages/Rotate, AcroForm/Fields, AcroForm/DA, Outlines/Type, Outlines, Outlines/Count, Pages/Resources/ProcSet, Pages/Resources 85-step mutation trace evading Hidost Effective for 198/500 seeds
41. Practical, Inexpensive Less than 1 week to find evasive variants for all 500 seeds, running on single desktop PC
42. Possible Defenses
43. Adjust threshold? Hidost16 results
44. Adjust threshold? Hidost16 results Variants found with threshold = 0.25 Variants found with threshold = 0.50
45. Adjust threshold? PDFRate results Variants found with threshold = 0.25 Variants found with threshold = 0.50
46. Retraining Classifier Labelled Training Data ML Algorithm Feature Extraction Vectors Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning)
47. Labelled Training Data ML Algorithm Feature Extraction Vectors Training (Supervised Learning) Clone ✓ ✓ ✗ ✓ EvadeML
48. Labelled Training Data ML Algorithm Feature Extraction Vectors Training (Supervised Learning) Clone ✓ ✓ ✗ ✓ EvadeML Deployment
49. (Probably) Doesn’t Work Original (Hidost 2016) Retrained (without new benign) Retrained (with new benign) Accuracy on Test Set 0.9983 0.9983 0.9983 False negatives on 250 non-training seeds 12 1 2 False positive rate (on benign samples) 0.0% 77% 0.0% Evasion rate 100% 49% 100% more experiments in progress...
50. Hide the Classifier? “Security through Obscurity” Clone Generated Variants Select Variants ✓ ✓ ✗ ✓ Clone Variants Fitness Function Candidate Variant ($%&'() , '(&++ ) Score Benign PDFs Malicious PDF Variants Benign PDFs Malicious PDF Variants Oracle Variant 0 /JavaScript eval(‘…’); /Root /Catalog /Pages 128 Oracle Target Classifier
51. Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware (against Hidost) Automated Evasion PDFrate 2/500 Evasive (0.4% Success) Potentially Good News?
52. Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware (against Hidost) Automated Evasion PDFrate 387/500 Evasive (77.4% Success)
53. Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware (against Hidost) Automated Evasion 6/500 Evasive (0.6% Success)
54. Evading Gmail’s Classifier 54 Evasion rate on Gmail: 179/380 (47.1%) for javascript in pdf.all_js: javascript.append_code("var oreilly=1;“) if pdf.get_size() < 7050000: pdf.add_padding(7050000 – pdf.get_size())
55. Fundamental Problem Classifier features are not intrinsic to malicious behavior Adversary can modify those features Artifacts of training data Heuristic search can find evasive variants automatically
56. Conclusion For source code, technical paper: EvadeML.org If you are developing or using malware classifiers, we want to work with you to test them for evadability: evans@virginia.edu Adversaries adapt, classifiers cannot rely on superficial features
57. David Evans evans@virginia.edu EvadeML.org