Classifiers Under Attack

Slide 1

Slide 1 text

Classifiers Under Attack David Evans University of Virginia [email protected] evadeML.org work with Weilin Xu and Yanjun Qi

Slide 2

Slide 2 text

Machine Learning is Solving All Our Problems! 2 Fake Spam IDS Malware Fake Accounts … …

Slide 3

Slide 3 text

Machine Learning is Eating the World 3 Data Scientist Security Expert ?

Slide 4

Slide 4 text

Assumption: Training Data is Representative 4 Labelled Training Data ML Algorithm Feature Extraction Vectors Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning)

Slide 5

Slide 5 text

Reality: Adversary Adapts ACM CCS 2016 Actual images Recognized faces

Slide 6

Slide 6 text

Case study: Evading PDF Malware Classifiers

Slide 7

Slide 7 text

Malware Classifiers in Practice Goal: Automatically simulate adaptive adversary against generic classifier Purpose: Understand classifier robustness Build better classifiers (?)

Slide 8

Slide 8 text

0 20 40 60 80 100 120 140 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 (1/4) Vulnerabilities reported in Adobe Acrobat Reader Source: http://www.cvedetails.com/vulnerability-list/vendor_id-53/product_id-497/year-2016/Adobe-Acrobat-Reader.html

Slide 9

Slide 9 text

High-Value Exploits: MiniDuke Source: https://cdn.securelist.com/files/2014/07/themysteryofthepdf0-dayassemblermicrobackdoor.pdf

Slide 10

Slide 10 text

PDF Malware 1 0 Exploits CVE- 2007-5659 buffer overflow

Slide 11

Slide 11 text

PDF Malware Classifiers PDFrate [ACSA 2012] Hidost16 [JIS 2016] Hidost13 [NDSS 2013] Random Forest Random Forest Support Vector Machine

Slide 12

Slide 12 text

Random Forest x y w 0 1 z 1 0 1 r q 0 z 0 0 y 0 1 Generate many random decision trees Train independently Select best trees Vote on result

Slide 13

Slide 13 text

PDF Malware Classifiers PDFrate [ACSA 2012] Hidost16 [JIS 2016] Hidost13 [NDSS 2013] Random Forest Random Forest Support Vector Machine Features Object counts, lengths, positions, etc. Object structural paths Very robust against “strongest conceivable mimicry attack”.

Slide 14

Slide 14 text

Classifier Performance 14 PDFrate* Hidost Accuracy 0.9976 0.9996 False Negative Rate 0.0000 0.0056 * Mimicus [Oakland ’14], an open source reimplementation of PDFrate.

Slide 15

Slide 15 text

Classifier Performance 15 PDFrate* Hidost Accuracy 0.9976 0.9996 False Negative Rate 0.0000 0.0056 Adversarial False Negative Rate 1.0000 1.0000 * Mimicus [Oakland ’14], an open source reimplementation of PDFrate.

Slide 16

Slide 16 text

Automatically Evading Classifiers

Slide 17

Slide 17 text

Variants Automated Classifier Evasion Using Genetic Programming 17 Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive?

Slide 18

Slide 18 text

Variants Goal: Find Evasive Variant 18 Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Evasive variant: Benign Simulated attacker’s goal: find a variant that is classified as benign, but exhibits the same malicious behavior.

Slide 19

Slide 19 text

PDF Structure 1 9

Slide 20

Slide 20 text

Malicious Seed 20 Clone Malicious PDF Modified Parser 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Parser is “robust” version of pdfrw: - Handles ungrammatical PDFs - Ignores inconsistencies, etc. Malware often malformed

Slide 21

Slide 21 text

Variants Generating Variants 21 Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓

Slide 22

Slide 22 text

Variants Generating Variants 22 Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants Variants Variants Select Variants ✓ ✓ ✗ ✓ 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Select random node

Slide 23

Slide 23 text

Variants Generating Variants 23 Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Variants Variants Select Variants ✓ ✓ ✗ ✓ Select random node 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Random transform: delete, insert, replace

Slide 24

Slide 24 text

Variants Generating Variants 24 Clone Benign PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Variants Variants Select Variants ✓ ✓ ✗ ✓ Nodes from Benign PDFs 128 546 7 63 Select random node 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Random transform: delete, insert, replace 128

Slide 25

Slide 25 text

Next Generation Selecting Promising Variants 25 Clone Benign PDFs Malicious PDF Mutation Variants Generated Variants Select Variants ✓ ✓ ✗ ✓

Slide 26

Slide 26 text

Selecting Promising Variants 26 Clone Generated Variants Select Variants ✓ ✓ ✗ ✓ Clone Variants Fitness Function Candidate Variant ($%&'() , '(&++ ) Score Malicious Benign PDFs Malicious PDF Variants Benign PDFs Malicious PDF Variants Oracle Variant 0 /JavaScript eval(‘…’); /Root /Catalog /Pages 128 Oracle Target Classifier

Slide 27

Slide 27 text

Oracle Execute candidate in vulnerable Adobe Reader in virtual environment Behavioral signature: only considered malicious if signature matches https://github.com/cuckoosandbox Simulated network: INetSim HTTP_URL + HOST extracted from API traces

Slide 28

Slide 28 text

Fitness Function 28 Assumes lost malicious behavior will not be recovered = 0 .5 − classifier_score if oracle = "malicious" −∞ otherwise classifier_score ≥ 0.5: labeled malicious

Slide 29

Slide 29 text

Experimental Results

Slide 30

Slide 30 text

30 Original Malicious Seeds Evading PDFrate Malicious Label Threshold

Slide 31

Slide 31 text

31 Original Malicious Seeds Evading PDFrate Discovered Evasive Variants 100% success rate ∼130 hours on typical desktop

Slide 32

Slide 32 text

32 Evading Hidost2013 100% success rate ∼46 hours on typical desktop Original Malicious Seeds Discovered Evasive Variants

Slide 33

Slide 33 text

Evading Hidost2016

Slide 34

Slide 34 text

Evading Hidost2016 100% success rate ∼14 hours on typical desktop

Slide 35

Slide 35 text

PDFRate Hidost Seeds Evaded

Slide 36

Slide 36 text

PDFRate Hidost Seeds Evaded Simple transformations often worked

Slide 37

Slide 37 text

PDFRate Seeds Evaded (insert, /Root/Pages/Kids, 3:/Root/Pages/Kids/4/Kids/5/) Inserting new pages works on 162/500 seeds Training malware often had no/little content

Slide 38

Slide 38 text

Hidost Seeds Evaded (delete, /Root/OpenAction/JS/Length) Deleting object worked on 1 seed No impact on malicious behavior

Slide 39

Slide 39 text

PDFRate Hidost Seeds Evaded Some seeds required complex transformations

Slide 40

Slide 40 text

Complex Transformations Insert: Threads, ViewerPreferences/Direction, Metadata, Metadata/Length, Metadata/Subtype, Metadata/Type, OpenAction/Contents, OpenAction/Contents/Filter, OpenAction/Contents/Length, Pages/MediaBox Delete: AcroForm, Names/JavaSCript/Names/S, AcroForm/DR/Encoding/PDFDocEncoding, AcroForm/DR/Encoding/PDFDocEncoding/Differences, AcroForm/DR/Encoding/PDFDocEncoding/Type, Pages/Rotate, AcroForm/Fields, AcroForm/DA, Outlines/Type, Outlines, Outlines/Count, Pages/Resources/ProcSet, Pages/Resources 85-step mutation trace evading Hidost Effective for 198/500 seeds

Slide 41

Slide 41 text

Practical, Inexpensive Less than 1 week to find evasive variants for all 500 seeds, running on single desktop PC

Slide 42

Slide 42 text

Possible Defenses

Slide 43

Slide 43 text

Adjust threshold? Hidost16 results

Slide 44

Slide 44 text

Adjust threshold? Hidost16 results Variants found with threshold = 0.25 Variants found with threshold = 0.50

Slide 45

Slide 45 text

Adjust threshold? PDFRate results Variants found with threshold = 0.25 Variants found with threshold = 0.50

Slide 46

Slide 46 text

Retraining Classifier Labelled Training Data ML Algorithm Feature Extraction Vectors Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning)

Slide 47

Slide 47 text

Labelled Training Data ML Algorithm Feature Extraction Vectors Training (Supervised Learning) Clone ✓ ✓ ✗ ✓ EvadeML

Slide 48

Slide 48 text

Labelled Training Data ML Algorithm Feature Extraction Vectors Training (Supervised Learning) Clone ✓ ✓ ✗ ✓ EvadeML Deployment

Slide 49

Slide 49 text

(Probably) Doesn’t Work Original (Hidost 2016) Retrained (without new benign) Retrained (with new benign) Accuracy on Test Set 0.9983 0.9983 0.9983 False negatives on 250 non-training seeds 12 1 2 False positive rate (on benign samples) 0.0% 77% 0.0% Evasion rate 100% 49% 100% more experiments in progress...

Slide 50

Slide 50 text

Hide the Classifier? “Security through Obscurity” Clone Generated Variants Select Variants ✓ ✓ ✗ ✓ Clone Variants Fitness Function Candidate Variant ($%&'() , '(&++ ) Score Benign PDFs Malicious PDF Variants Benign PDFs Malicious PDF Variants Oracle Variant 0 /JavaScript eval(‘…’); /Root /Catalog /Pages 128 Oracle Target Classifier

Slide 51

Slide 51 text

Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware (against Hidost) Automated Evasion PDFrate 2/500 Evasive (0.4% Success) Potentially Good News?

Slide 52

Slide 52 text

Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware (against Hidost) Automated Evasion PDFrate 387/500 Evasive (77.4% Success)

Slide 53

Slide 53 text

Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware (against Hidost) Automated Evasion 6/500 Evasive (0.6% Success)

Slide 54

Slide 54 text

Evading Gmail’s Classifier 54 Evasion rate on Gmail: 179/380 (47.1%) for javascript in pdf.all_js: javascript.append_code("var oreilly=1;“) if pdf.get_size() < 7050000: pdf.add_padding(7050000 – pdf.get_size())

Slide 55

Slide 55 text

Fundamental Problem Classifier features are not intrinsic to malicious behavior Adversary can modify those features Artifacts of training data Heuristic search can find evasive variants automatically

Slide 56

Slide 56 text

Conclusion For source code, technical paper: EvadeML.org If you are developing or using malware classifiers, we want to work with you to test them for evadability: [email protected] Adversaries adapt, classifiers cannot rely on superficial features

Slide 57

Slide 57 text

David Evans [email protected] EvadeML.org