Classifiers Under Attack

Talk at USENIX Enigma 2017
1 February 2017
Oakland, CA

https://evadeML.org

Transcript

1. Classifiers Under Attack David Evans work with Weilin Xu and Yanjun Qi University of Virginia evans@virginia.edu evadeML.org 1 February 2017
2. Machine Learning Does Amazing Things 2 FeatureSmith
3. … and can solve all Security Problems! Fake Spam IDS Malware Fake Accounts …
4. Labelled Training Data ML Algorithm Feature Extraction Vectors Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning) Assumption: Training Data is Representative
5. Training Data Deployment Training Adversaries Don’t Cooperate Assumption: Training Data is Representative
6. Training Data Deployment Training Adversaries Don’t Cooperate Assumption: Training Data is Representative Poisoning
7. Adversaries Don’t Cooperate Assumption: Training Data is Representative Training Data Deployment Training Evading
8. Focus: Evasion Attacks Goal: Automatically simulate adaptive adversary against generic classifier Purpose: Understand classifier robustness Build better classifiers (or give up)
9. Case study: Evading PDF Malware Classifiers
11. 0 50 100 150 200 250 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 Vulnerabilities reported in Adobe Acrobat Reader Source: http://www.cvedetails.com/vulnerability-list.php?vendor_id=53&product_id=921 33 already in Jan 2017!
12. PDF Malware Classifiers PDFrate [ACSA 2012] Hidost16 [JIS 2016] Hidost13 [NDSS 2013] Random Forest Random Forest Support Vector Machine Classifier Accuracy 0.9976 0.9996 0.9996 * Mimicus [Oakland 2014], an open source reimplementation of PDFrate.
13. Random Forest x y w 0 1 z 1 0 1 r q 0 z 0 0 y 0 1 Generate many random decision trees Train independently Select best trees Vote on result
14. PDF Malware Classifiers Random Forest Random Forest Support Vector Machine Features Object counts, lengths, positions, … Object structural paths Very robust against “strongest conceivable mimicry attack”. Automated Features Manual Features PDFrate [ACSA 2012] Hidost16 [JIS 2016] Hidost13 [NDSS 2013]
15. Automatically Evading Classifiers
16. Variants Automated Classifier Evasion Using Genetic Programming Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive?
17. Variants Goal: Find Evasive Variant Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Benign Simulated attacker’s goal: find sample classified as benign, that exhibits malicious behavior.
18. Variants Start with Malicious Seed Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Benign
19. PDF Structure
20. Variants Clone Benign PDFs Malicious PDF Mutation Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Modified Parser 0 /JavaScript eval(‘…’); /Root /Catalog /Pages “robust” version of pdfrw
21. Variants Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Generating Variants
22. Variants Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Generating Variants Select random node
23. Variants Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Select random node Random transform: delete, insert, replace Generating Variants
24. Variants Generating Variants Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Nodes from Benign PDFs 128 546 7 63 Random transform: delete, insert, replace 128 Select random node
25. Variants Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Selecting Promising Variants
26. Variants Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Selecting Promising Variants Clone Generated Variants Clone Variants Fitness Function Candidate Variant ($%&'() , '(&++ ) Score Malicious Benign PDFs Malicious PDF Variants Benign PDFs Malicious PDF Variants Oracle Variant 0 /JavaScript eval(‘…’); /Root /Catalog /Pages 128 Oracle Target Classifier
27. Oracle Execute candidate in vulnerable Adobe Reader in virtual environment Behavioral signature: malicious if signature matches https://github.com/cuckoosandbox Simulated network: INetSim Cuckoo HTTP_URL + HOST extracted from API traces Advantage: we know the target malware behavior
28. Variants Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Selecting Promising Variants Clone Generated Variants Clone Variants Fitness Function Candidate Variant ($%&'() , '(&++ ) Score Malicious Benign PDFs Malicious PDF Variants Benign PDFs Malicious PDF Variants Oracle Variant 0 /JavaScript eval(‘…’); /Root /Catalog /Pages 128 Oracle Target Classifier
29. Fitness Function Assumes lost malicious behavior will not be recovered = 0 .5 − classifier_score if oracle = "malicious" −∞ otherwise classifier_score ≥ 0.5: labeled malicious
30. Experimental Results
31. Classifier Performance PDFrate Hidost Accuracy 0.9976 0.9996 False Negative Rate 0.0000 0.0056 Results on non-adversarial samples
32. Classifier Performance PDFrate Hidost Accuracy 0.9976 0.9996 False Negative Rate 0.0000 0.0056 False Negative Rate against Adversary 1.0000 1.0000
33. 0 100 200 300 400 500 0 100 200 300 Seeds Evaded (out of 500) PDFRate Number of Mutations Hidost
34. 0 100 200 300 400 500 0 100 200 300 Seeds Evaded (out of 500) PDFRate Hidost Number of Mutations Simple transformations often worked
35. 0 100 200 300 400 500 0 100 200 300 Seeds Evaded (out of 500) PDFRate Number of Mutations (insert, /Root/Pages/Kids, 3:/Root/Pages/Kids/4/Kids/5/) Works on 162/500 seeds
36. 0 100 200 300 400 500 0 100 200 300 Seeds Evaded (out of 500) PDFRate Number of Mutations Hidost Some seeds required complex transformations
37. Insert: Threads, ViewerPreferences/Direction, Metadata, Metadata/Length, Metadata/Subtype, Metadata/Type, OpenAction/Contents, OpenAction/Contents/Filter, OpenAction/Contents/Length, Pages/MediaBox Delete: AcroForm, Names/JavaSCript/Names/S, AcroForm/DR/Encoding/PDFDocEncoding, AcroForm/DR/Encoding/PDFDocEncoding/Differences, AcroForm/DR/Encoding/PDFDocEncoding/Type, Pages/Rotate, AcroForm/Fields, AcroForm/DA, Outlines/Type, Outlines, Outlines/Count, Pages/Resources/ProcSet, Pages/Resources 85-step mutation trace evading Hidost Effective for 198/500 seeds
38. 0 20 40 60 80 100 120 Hidost PDFrate Oracle Execution Cost Hours to find all 500 variants on one desktop PC Oracle Mutation Classifier
39. Possible Defenses
40. Possible Defense: Adjust Threshold Charles Smutz, Angelos Stavrou. When a Tree Falls: Using Diversity in Ensemble Classifiers to Identify Evasion in Malware Detectors. NDSS 2016.
41. Original Malicious Seeds Evading PDFrate Malicious Label Threshold
42. Discovered Evasive Variants Adjust threshold?
43. Adjust threshold? Variants found with threshold = 0.25 Variants found with threshold = 0.50
44. Hidost16 Variants found with threshold = 0.25 Variants found with threshold = 0.50
45. Possible Defense: Retrain Classifier
46. Labelled Training Data ML Algorithm Feature Extraction Vectors Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning) Retraining Classifier
47. Labelled Training Data ML Algorithm Feature Extraction Vectors Training (Supervised Learning) Clone EvadeML
48. Labelled Training Data ML Algorithm Feature Extraction Vectors Training (Supervised Learning) Deployment Clone EvadeML
49. 0 100 200 300 400 500 0 200 400 600 800 Seeds Evaded (out of 500) Generations Hidost16 Original classifier: Takes 614 generations to evade all seeds
50. 0 100 200 300 400 500 0 200 400 600 800 Seeds Evaded (out of 500) Generations Hidost16 HidostR1
51. 0 100 200 300 400 500 0 200 400 600 800 Seeds Evaded (out of 500) Generations Hidost16 HidostR1 HidostR2
52. 0 100 200 300 400 500 0 200 400 600 800 Seeds Evaded (out of 500) Generations Hidost16 HidostR1 HidostR2
53. 0 100 200 300 400 500 0 200 400 600 800 Seeds Evaded (out of 500) Generations Hidost16 HidostR1 HidostR2 Genome Contagio Benign Hidost16 0.00 0.00 HidostR1 0.78 0.30 HidostR2 0.85 0.53 False Positive Rates
54. 0 100 200 300 400 500 0 200 400 600 800 Seeds Evaded (out of 500) Generations Hidost16 HidostR1 HidostR2 Genome Contagio Benign Hidost16 0.00 0.00 HidostR1 0.78 0.30 HidostR2 0.85 0.53 False Positive Rates
55. 0 100 200 300 400 500 0 500 1000 1500 2000 Retrained using evasive variants and all benign samples available to adversary .11 Evasion Rate .07 False Positive Generations
56. Possible Defense: Hide Classifier
57. Variants Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Hide Classifier “Security Through Obscurity” Clone Generated Variants Clone Variants Fitness Function Candidate Variant ($%&'() , '(&++ ) Score Malicious Benign PDFs Malicious PDF Variants Benign PDFs Malicious PDF Variants Oracle Variant 0 /JavaScript eval(‘…’); /Root /Catalog /Pages 128 Oracle Target Classifier
58. Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware (against PDFrate) Automated Evasion PDFrate 2/500 Evasive (0.4% Success) Potentially Good News?
59. Evasive PDF Malware (against PDFrate) Cross-Evasion Effects PDF Malware Seeds Hidost 13 Automated Evasion PDFrate 2/500 Evasive (0.4% Success) Evasive PDF Malware (against Hidost) 387/500 Evasive (77.4% Success)
60. 387/500 Evasive (77.4% Success) Cross-Evasion Effects PDF Malware Seeds Hidost 13 Automated Evasion PDFrate Evasive PDF Malware (against Hidost)
61. Cross-Evasion Effects PDF Malware Seeds Hidost 13 Automated Evasion Evasive PDF Malware (against Hidost) 6/500 Evasive (0.6% Success)
62. Evading Gmail’s Classifier Evasion rate on Gmail: 179/380 (47.1%) for javascript in pdf.all_js: javascript.append_code("var enigma=1;“) if pdf.get_size() < 7050000: pdf.add_padding(7050000 – pdf.get_size())
63. Conclusion
64. Conclusion Domain Knowledge is Not Dead Trust Demands Understanding
65. David Evans University of Virginia evans@virginia.edu EvadeML.org source code, papers Credits: Weilin Xu, Yanjun Qi