Classifiers Under Attack

Transcript

1. Classifiers Under Attack David Evans University of Virginia evans@virginia.edu evadeML.org

   work with Weilin Xu and Yanjun Qi

2. Machine Learning is Solving All Our Problems! 2 Fake Spam

   IDS Malware Fake Accounts … …

3. Machine Learning is Eating the World 3 Data Scientist Security

   Expert ?

4. Assumption: Training Data is Representative 4 Labelled Training Data ML

   Algorithm Feature Extraction Vectors Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning)

5. Reality: Adversary Adapts ACM CCS 2016 Actual images Recognized faces

6. Case study: Evading PDF Malware Classifiers

7. Malware Classifiers in Practice Goal: Automatically simulate adaptive adversary against

   generic classifier Purpose: Understand classifier robustness Build better classifiers (?)

8. 0 20 40 60 80 100 120 140 2006 2007

   2008 2009 2010 2011 2012 2013 2014 2015 2016 (1/4) Vulnerabilities reported in Adobe Acrobat Reader Source: http://www.cvedetails.com/vulnerability-list/vendor_id-53/product_id-497/year-2016/Adobe-Acrobat-Reader.html

9. High-Value Exploits: MiniDuke Source: https://cdn.securelist.com/files/2014/07/themysteryofthepdf0-dayassemblermicrobackdoor.pdf

10. PDF Malware 1 0 Exploits CVE- 2007-5659 buffer overflow

11. PDF Malware Classifiers PDFrate [ACSA 2012] Hidost16 [JIS 2016] Hidost13

    [NDSS 2013] Random Forest Random Forest Support Vector Machine

12. Random Forest x y w 0 1 z 1 0

    1 r q 0 z 0 0 y 0 1 Generate many random decision trees Train independently Select best trees Vote on result

13. PDF Malware Classifiers PDFrate [ACSA 2012] Hidost16 [JIS 2016] Hidost13

    [NDSS 2013] Random Forest Random Forest Support Vector Machine Features Object counts, lengths, positions, etc. Object structural paths Very robust against “strongest conceivable mimicry attack”.

14. Classifier Performance 14 PDFrate* Hidost Accuracy 0.9976 0.9996 False Negative

    Rate 0.0000 0.0056 * Mimicus [Oakland ’14], an open source reimplementation of PDFrate.

15. Classifier Performance 15 PDFrate* Hidost Accuracy 0.9976 0.9996 False Negative

    Rate 0.0000 0.0056 Adversarial False Negative Rate 1.0000 1.0000 * Mimicus [Oakland ’14], an open source reimplementation of PDFrate.

16. Automatically Evading Classifiers

17. Variants Automated Classifier Evasion Using Genetic Programming 17 Clone Benign

    PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive?

18. Variants Goal: Find Evasive Variant 18 Clone Benign PDFs Malicious

    PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Evasive variant: Benign Simulated attacker’s goal: find a variant that is classified as benign, but exhibits the same malicious behavior.

19. PDF Structure 1 9

20. Malicious Seed 20 Clone Malicious PDF Modified Parser 0 /JavaScript

    eval(‘…’); /Root /Catalog /Pages Parser is “robust” version of pdfrw: - Handles ungrammatical PDFs - Ignores inconsistencies, etc. Malware often malformed

21. Variants Generating Variants 21 Clone Benign PDFs Malicious PDF Mutation

    Variants Variants Select Variants ✓ ✓ ✗ ✓

22. Variants Generating Variants 22 Clone Benign PDFs Malicious PDF Mutation

    Variants Variants Select Variants Variants Variants Select Variants ✓ ✓ ✗ ✓ 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Select random node

23. Variants Generating Variants 23 Clone Benign PDFs Malicious PDF Mutation

    Variants Variants Select Variants ✓ ✓ ✗ ✓ Variants Variants Select Variants ✓ ✓ ✗ ✓ Select random node 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Random transform: delete, insert, replace

24. Variants Generating Variants 24 Clone Benign PDFs Malicious PDF Mutation

    Variants Variants Select Variants ✓ ✓ ✗ ✓ Variants Variants Select Variants ✓ ✓ ✗ ✓ Nodes from Benign PDFs 128 546 7 63 Select random node 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Random transform: delete, insert, replace 128

25. Next Generation Selecting Promising Variants 25 Clone Benign PDFs Malicious

    PDF Mutation Variants Generated Variants Select Variants ✓ ✓ ✗ ✓

26. Selecting Promising Variants 26 Clone Generated Variants Select Variants ✓

    ✓ ✗ ✓ Clone Variants Fitness Function Candidate Variant ($%&'() , '(&++ ) Score Malicious Benign PDFs Malicious PDF Variants Benign PDFs Malicious PDF Variants Oracle Variant 0 /JavaScript eval(‘…’); /Root /Catalog /Pages 128 Oracle Target Classifier

27. Oracle Execute candidate in vulnerable Adobe Reader in virtual environment

    Behavioral signature: only considered malicious if signature matches https://github.com/cuckoosandbox Simulated network: INetSim HTTP_URL + HOST extracted from API traces

28. Fitness Function 28 Assumes lost malicious behavior will not be

    recovered = 0 .5 − classifier_score if oracle = "malicious" −∞ otherwise classifier_score ≥ 0.5: labeled malicious

29. Experimental Results

30. 30 Original Malicious Seeds Evading PDFrate Malicious Label Threshold

31. 31 Original Malicious Seeds Evading PDFrate Discovered Evasive Variants 100%

    success rate ∼130 hours on typical desktop

32. 32 Evading Hidost2013 100% success rate ∼46 hours on typical

    desktop Original Malicious Seeds Discovered Evasive Variants

33. Evading Hidost2016

34. Evading Hidost2016 100% success rate ∼14 hours on typical desktop

35. PDFRate Hidost Seeds Evaded

36. PDFRate Hidost Seeds Evaded Simple transformations often worked

37. PDFRate Seeds Evaded (insert, /Root/Pages/Kids, 3:/Root/Pages/Kids/4/Kids/5/) Inserting new pages works

    on 162/500 seeds Training malware often had no/little content

38. Hidost Seeds Evaded (delete, /Root/OpenAction/JS/Length) Deleting object worked on 1

    seed No impact on malicious behavior

39. PDFRate Hidost Seeds Evaded Some seeds required complex transformations

40. Complex Transformations Insert: Threads, ViewerPreferences/Direction, Metadata, Metadata/Length, Metadata/Subtype, Metadata/Type, OpenAction/Contents,

    OpenAction/Contents/Filter, OpenAction/Contents/Length, Pages/MediaBox Delete: AcroForm, Names/JavaSCript/Names/S, AcroForm/DR/Encoding/PDFDocEncoding, AcroForm/DR/Encoding/PDFDocEncoding/Differences, AcroForm/DR/Encoding/PDFDocEncoding/Type, Pages/Rotate, AcroForm/Fields, AcroForm/DA, Outlines/Type, Outlines, Outlines/Count, Pages/Resources/ProcSet, Pages/Resources 85-step mutation trace evading Hidost Effective for 198/500 seeds

41. Practical, Inexpensive Less than 1 week to find evasive variants

    for all 500 seeds, running on single desktop PC

42. Possible Defenses

43. Adjust threshold? Hidost16 results

44. Adjust threshold? Hidost16 results Variants found with threshold = 0.25

    Variants found with threshold = 0.50

45. Adjust threshold? PDFRate results Variants found with threshold = 0.25

    Variants found with threshold = 0.50

46. Retraining Classifier Labelled Training Data ML Algorithm Feature Extraction Vectors

    Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning)

47. Labelled Training Data ML Algorithm Feature Extraction Vectors Training (Supervised

    Learning) Clone ✓ ✓ ✗ ✓ EvadeML

48. Labelled Training Data ML Algorithm Feature Extraction Vectors Training (Supervised

    Learning) Clone ✓ ✓ ✗ ✓ EvadeML Deployment

49. (Probably) Doesn’t Work Original (Hidost 2016) Retrained (without new benign)

    Retrained (with new benign) Accuracy on Test Set 0.9983 0.9983 0.9983 False negatives on 250 non-training seeds 12 1 2 False positive rate (on benign samples) 0.0% 77% 0.0% Evasion rate 100% 49% 100% more experiments in progress...

50. Hide the Classifier? “Security through Obscurity” Clone Generated Variants Select

    Variants ✓ ✓ ✗ ✓ Clone Variants Fitness Function Candidate Variant ($%&'() , '(&++ ) Score Benign PDFs Malicious PDF Variants Benign PDFs Malicious PDF Variants Oracle Variant 0 /JavaScript eval(‘…’); /Root /Catalog /Pages 128 Oracle Target Classifier

51. Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware

    (against Hidost) Automated Evasion PDFrate 2/500 Evasive (0.4% Success) Potentially Good News?

52. Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware

    (against Hidost) Automated Evasion PDFrate 387/500 Evasive (77.4% Success)

53. Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware

    (against Hidost) Automated Evasion 6/500 Evasive (0.6% Success)

54. Evading Gmail’s Classifier 54 Evasion rate on Gmail: 179/380 (47.1%)

    for javascript in pdf.all_js: javascript.append_code("var oreilly=1;“) if pdf.get_size() < 7050000: pdf.add_padding(7050000 – pdf.get_size())

55. Fundamental Problem Classifier features are not intrinsic to malicious behavior

    Adversary can modify those features Artifacts of training data Heuristic search can find evasive variants automatically

56. Conclusion For source code, technical paper: EvadeML.org If you are

    developing or using malware classifiers, we want to work with you to test them for evadability: evans@virginia.edu Adversaries adapt, classifiers cannot rely on superficial features

57. David Evans evans@virginia.edu EvadeML.org