Assumption: Training Data is Representative 4 Labelled Training Data ML Algorithm Feature Extraction Vectors Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning)
PDF Malware Classifiers PDFrate [ACSA 2012] Hidost16 [JIS 2016] Hidost13 [NDSS 2013] Random Forest Random Forest Support Vector Machine Features Object counts, lengths, positions, etc. Object structural paths Very robust against “strongest conceivable mimicry attack”.
Variants Goal: Find Evasive Variant 18 Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Evasive variant: Benign Simulated attacker’s goal: find a variant that is classified as benign, but exhibits the same malicious behavior.
Oracle Execute candidate in vulnerable Adobe Reader in virtual environment Behavioral signature: only considered malicious if signature matches https://github.com/cuckoosandbox Simulated network: INetSim HTTP_URL + HOST extracted from API traces
Fitness Function 28 Assumes lost malicious behavior will not be recovered = 0 .5 − classifier_score if oracle = "malicious" −∞ otherwise classifier_score ≥ 0.5: labeled malicious
PDFRate Seeds Evaded (insert, /Root/Pages/Kids, 3:/Root/Pages/Kids/4/Kids/5/) Inserting new pages works on 162/500 seeds Training malware often had no/little content
Retraining Classifier Labelled Training Data ML Algorithm Feature Extraction Vectors Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning)
(Probably) Doesn’t Work Original (Hidost 2016) Retrained (without new benign) Retrained (with new benign) Accuracy on Test Set 0.9983 0.9983 0.9983 False negatives on 250 non-training seeds 12 1 2 False positive rate (on benign samples) 0.0% 77% 0.0% Evasion rate 100% 49% 100% more experiments in progress...
Fundamental Problem Classifier features are not intrinsic to malicious behavior Adversary can modify those features Artifacts of training data Heuristic search can find evasive variants automatically
Conclusion For source code, technical paper: EvadeML.org If you are developing or using malware classifiers, we want to work with you to test them for evadability: [email protected] Adversaries adapt, classifiers cannot rely on superficial features
evansuva
.png){kind=avatar}
noteinc
itakigawa
yumulab
gaelvaroquaux
masakat0
tatsui
osx
tomoshige_n
verypluming
lyh125
ebisawahayata
joisino
jasonvnalue
zakiyama
thoeni
phodgson
addyosmani
roundedbygravity
dotmariusz
brettharned
maltzj
chrisshort
philnash