Share
Embed
Classifiers Under Attack
Presentation at O'Reilly Security
New York, 2 November 2016
http://conferences.oreilly.com/security/network-data-security-ny/public/schedule/detail/53176
David Evans
November 03, 2016
Transcript
57 slides
-
Classifiers Under Attack David Evans University of Virginia [email protected] evadeML.org
work with Weilin Xu and Yanjun Qi -
Machine Learning is Solving All Our Problems! 2 Fake Spam
IDS Malware Fake Accounts … … -
Assumption: Training Data is Representative 4 Labelled Training Data ML
Algorithm Feature Extraction Vectors Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning) -
Malware Classifiers in Practice Goal: Automatically simulate adaptive adversary against
generic classifier Purpose: Understand classifier robustness Build better classifiers (?) -
0 20 40 60 80 100 120 140 2006 2007
2008 2009 2010 2011 2012 2013 2014 2015 2016 (1/4) Vulnerabilities reported in Adobe Acrobat Reader Source: http://www.cvedetails.com/vulnerability-list/vendor_id-53/product_id-497/year-2016/Adobe-Acrobat-Reader.html -
PDF Malware Classifiers PDFrate [ACSA 2012] Hidost16 [JIS 2016] Hidost13
[NDSS 2013] Random Forest Random Forest Support Vector Machine -
Random Forest x y w 0 1 z 1 0
1 r q 0 z 0 0 y 0 1 Generate many random decision trees Train independently Select best trees Vote on result -
PDF Malware Classifiers PDFrate [ACSA 2012] Hidost16 [JIS 2016] Hidost13
[NDSS 2013] Random Forest Random Forest Support Vector Machine Features Object counts, lengths, positions, etc. Object structural paths Very robust against “strongest conceivable mimicry attack”. -
Classifier Performance 14 PDFrate* Hidost Accuracy 0.9976 0.9996 False Negative
Rate 0.0000 0.0056 * Mimicus [Oakland ’14], an open source reimplementation of PDFrate. -
Classifier Performance 15 PDFrate* Hidost Accuracy 0.9976 0.9996 False Negative
Rate 0.0000 0.0056 Adversarial False Negative Rate 1.0000 1.0000 * Mimicus [Oakland ’14], an open source reimplementation of PDFrate. -
Variants Automated Classifier Evasion Using Genetic Programming 17 Clone Benign
PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? -
Variants Goal: Find Evasive Variant 18 Clone Benign PDFs Malicious
PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Evasive variant: Benign Simulated attacker’s goal: find a variant that is classified as benign, but exhibits the same malicious behavior. -
Malicious Seed 20 Clone Malicious PDF Modified Parser 0 /JavaScript
eval(‘…’); /Root /Catalog /Pages Parser is “robust” version of pdfrw: - Handles ungrammatical PDFs - Ignores inconsistencies, etc. Malware often malformed -
Variants Generating Variants 21 Clone Benign PDFs Malicious PDF Mutation
Variants Variants Select Variants ✓ ✓ ✗ ✓ -
Variants Generating Variants 22 Clone Benign PDFs Malicious PDF Mutation
Variants Variants Select Variants Variants Variants Select Variants ✓ ✓ ✗ ✓ 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Select random node -
Variants Generating Variants 23 Clone Benign PDFs Malicious PDF Mutation
Variants Variants Select Variants ✓ ✓ ✗ ✓ Variants Variants Select Variants ✓ ✓ ✗ ✓ Select random node 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Random transform: delete, insert, replace -
Variants Generating Variants 24 Clone Benign PDFs Malicious PDF Mutation
Variants Variants Select Variants ✓ ✓ ✗ ✓ Variants Variants Select Variants ✓ ✓ ✗ ✓ Nodes from Benign PDFs 128 546 7 63 Select random node 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Random transform: delete, insert, replace 128 -
Next Generation Selecting Promising Variants 25 Clone Benign PDFs Malicious
PDF Mutation Variants Generated Variants Select Variants ✓ ✓ ✗ ✓ -
Selecting Promising Variants 26 Clone Generated Variants Select Variants ✓
✓ ✗ ✓ Clone Variants Fitness Function Candidate Variant ($%&'() , '(&++ ) Score Malicious Benign PDFs Malicious PDF Variants Benign PDFs Malicious PDF Variants Oracle Variant 0 /JavaScript eval(‘…’); /Root /Catalog /Pages 128 Oracle Target Classifier -
Oracle Execute candidate in vulnerable Adobe Reader in virtual environment
Behavioral signature: only considered malicious if signature matches https://github.com/cuckoosandbox Simulated network: INetSim HTTP_URL + HOST extracted from API traces -
Fitness Function 28 Assumes lost malicious behavior will not be
recovered = 0 .5 − classifier_score if oracle = "malicious" −∞ otherwise classifier_score ≥ 0.5: labeled malicious -
31 Original Malicious Seeds Evading PDFrate Discovered Evasive Variants 100%
success rate ∼130 hours on typical desktop -
32 Evading Hidost2013 100% success rate ∼46 hours on typical
desktop Original Malicious Seeds Discovered Evasive Variants -
PDFRate Seeds Evaded (insert, /Root/Pages/Kids, 3:/Root/Pages/Kids/4/Kids/5/) Inserting new pages works
on 162/500 seeds Training malware often had no/little content -
Hidost Seeds Evaded (delete, /Root/OpenAction/JS/Length) Deleting object worked on 1
seed No impact on malicious behavior -
Complex Transformations Insert: Threads, ViewerPreferences/Direction, Metadata, Metadata/Length, Metadata/Subtype, Metadata/Type, OpenAction/Contents,
OpenAction/Contents/Filter, OpenAction/Contents/Length, Pages/MediaBox Delete: AcroForm, Names/JavaSCript/Names/S, AcroForm/DR/Encoding/PDFDocEncoding, AcroForm/DR/Encoding/PDFDocEncoding/Differences, AcroForm/DR/Encoding/PDFDocEncoding/Type, Pages/Rotate, AcroForm/Fields, AcroForm/DA, Outlines/Type, Outlines, Outlines/Count, Pages/Resources/ProcSet, Pages/Resources 85-step mutation trace evading Hidost Effective for 198/500 seeds -
Practical, Inexpensive Less than 1 week to find evasive variants
for all 500 seeds, running on single desktop PC -
Adjust threshold? Hidost16 results Variants found with threshold = 0.25
Variants found with threshold = 0.50 -
Adjust threshold? PDFRate results Variants found with threshold = 0.25
Variants found with threshold = 0.50 -
Retraining Classifier Labelled Training Data ML Algorithm Feature Extraction Vectors
Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning) -
Labelled Training Data ML Algorithm Feature Extraction Vectors Training (Supervised
Learning) Clone ✓ ✓ ✗ ✓ EvadeML -
Labelled Training Data ML Algorithm Feature Extraction Vectors Training (Supervised
Learning) Clone ✓ ✓ ✗ ✓ EvadeML Deployment -
(Probably) Doesn’t Work Original (Hidost 2016) Retrained (without new benign)
Retrained (with new benign) Accuracy on Test Set 0.9983 0.9983 0.9983 False negatives on 250 non-training seeds 12 1 2 False positive rate (on benign samples) 0.0% 77% 0.0% Evasion rate 100% 49% 100% more experiments in progress... -
Hide the Classifier? “Security through Obscurity” Clone Generated Variants Select
Variants ✓ ✓ ✗ ✓ Clone Variants Fitness Function Candidate Variant ($%&'() , '(&++ ) Score Benign PDFs Malicious PDF Variants Benign PDFs Malicious PDF Variants Oracle Variant 0 /JavaScript eval(‘…’); /Root /Catalog /Pages 128 Oracle Target Classifier -
Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware
(against Hidost) Automated Evasion PDFrate 2/500 Evasive (0.4% Success) Potentially Good News? -
Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware
(against Hidost) Automated Evasion PDFrate 387/500 Evasive (77.4% Success) -
Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware
(against Hidost) Automated Evasion 6/500 Evasive (0.6% Success) -
Evading Gmail’s Classifier 54 Evasion rate on Gmail: 179/380 (47.1%)
for javascript in pdf.all_js: javascript.append_code("var oreilly=1;“) if pdf.get_size() < 7050000: pdf.add_padding(7050000 – pdf.get_size()) -
Fundamental Problem Classifier features are not intrinsic to malicious behavior
Adversary can modify those features Artifacts of training data Heuristic search can find evasive variants automatically -
Conclusion For source code, technical paper: EvadeML.org If you are
developing or using malware classifiers, we want to work with you to test them for evadability: [email protected] Adversaries adapt, classifiers cannot rely on superficial features