Classifiers Under Attack

40e37c08199ed4d3866ce6e1ff0be06d?s=47 David Evans
November 03, 2016

Classifiers Under Attack

Presentation at O'Reilly Security
New York, 2 November 2016

http://conferences.oreilly.com/security/network-data-security-ny/public/schedule/detail/53176

40e37c08199ed4d3866ce6e1ff0be06d?s=128

David Evans

November 03, 2016
Tweet

Transcript

  1. Classifiers Under Attack David Evans University of Virginia evans@virginia.edu evadeML.org

    work with Weilin Xu and Yanjun Qi
  2. Machine Learning is Solving All Our Problems! 2 Fake Spam

    IDS Malware Fake Accounts … …
  3. Machine Learning is Eating the World 3 Data Scientist Security

    Expert ?
  4. Assumption: Training Data is Representative 4 Labelled Training Data ML

    Algorithm Feature Extraction Vectors Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning)
  5. Reality: Adversary Adapts ACM CCS 2016 Actual images Recognized faces

  6. Case study: Evading PDF Malware Classifiers

  7. Malware Classifiers in Practice Goal: Automatically simulate adaptive adversary against

    generic classifier Purpose: Understand classifier robustness Build better classifiers (?)
  8. 0 20 40 60 80 100 120 140 2006 2007

    2008 2009 2010 2011 2012 2013 2014 2015 2016 (1/4) Vulnerabilities reported in Adobe Acrobat Reader Source: http://www.cvedetails.com/vulnerability-list/vendor_id-53/product_id-497/year-2016/Adobe-Acrobat-Reader.html
  9. High-Value Exploits: MiniDuke Source: https://cdn.securelist.com/files/2014/07/themysteryofthepdf0-dayassemblermicrobackdoor.pdf

  10. PDF Malware 1 0 Exploits CVE- 2007-5659 buffer overflow

  11. PDF Malware Classifiers PDFrate [ACSA 2012] Hidost16 [JIS 2016] Hidost13

    [NDSS 2013] Random Forest Random Forest Support Vector Machine
  12. Random Forest x y w 0 1 z 1 0

    1 r q 0 z 0 0 y 0 1 Generate many random decision trees Train independently Select best trees Vote on result
  13. PDF Malware Classifiers PDFrate [ACSA 2012] Hidost16 [JIS 2016] Hidost13

    [NDSS 2013] Random Forest Random Forest Support Vector Machine Features Object counts, lengths, positions, etc. Object structural paths Very robust against “strongest conceivable mimicry attack”.
  14. Classifier Performance 14 PDFrate* Hidost Accuracy 0.9976 0.9996 False Negative

    Rate 0.0000 0.0056 * Mimicus [Oakland ’14], an open source reimplementation of PDFrate.
  15. Classifier Performance 15 PDFrate* Hidost Accuracy 0.9976 0.9996 False Negative

    Rate 0.0000 0.0056 Adversarial False Negative Rate 1.0000 1.0000 * Mimicus [Oakland ’14], an open source reimplementation of PDFrate.
  16. Automatically Evading Classifiers

  17. Variants Automated Classifier Evasion Using Genetic Programming 17 Clone Benign

    PDFs Malicious PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive?
  18. Variants Goal: Find Evasive Variant 18 Clone Benign PDFs Malicious

    PDF Mutation Variants Variants Select Variants ✓ ✓ ✗ ✓ Found Evasive? Evasive variant: Benign Simulated attacker’s goal: find a variant that is classified as benign, but exhibits the same malicious behavior.
  19. PDF Structure 1 9

  20. Malicious Seed 20 Clone Malicious PDF Modified Parser 0 /JavaScript

    eval(‘…’); /Root /Catalog /Pages Parser is “robust” version of pdfrw: - Handles ungrammatical PDFs - Ignores inconsistencies, etc. Malware often malformed
  21. Variants Generating Variants 21 Clone Benign PDFs Malicious PDF Mutation

    Variants Variants Select Variants ✓ ✓ ✗ ✓
  22. Variants Generating Variants 22 Clone Benign PDFs Malicious PDF Mutation

    Variants Variants Select Variants Variants Variants Select Variants ✓ ✓ ✗ ✓ 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Select random node
  23. Variants Generating Variants 23 Clone Benign PDFs Malicious PDF Mutation

    Variants Variants Select Variants ✓ ✓ ✗ ✓ Variants Variants Select Variants ✓ ✓ ✗ ✓ Select random node 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Random transform: delete, insert, replace
  24. Variants Generating Variants 24 Clone Benign PDFs Malicious PDF Mutation

    Variants Variants Select Variants ✓ ✓ ✗ ✓ Variants Variants Select Variants ✓ ✓ ✗ ✓ Nodes from Benign PDFs 128 546 7 63 Select random node 0 /JavaScript eval(‘…’); /Root /Catalog /Pages Random transform: delete, insert, replace 128
  25. Next Generation Selecting Promising Variants 25 Clone Benign PDFs Malicious

    PDF Mutation Variants Generated Variants Select Variants ✓ ✓ ✗ ✓
  26. Selecting Promising Variants 26 Clone Generated Variants Select Variants ✓

    ✓ ✗ ✓ Clone Variants Fitness Function Candidate Variant ($%&'() , '(&++ ) Score Malicious Benign PDFs Malicious PDF Variants Benign PDFs Malicious PDF Variants Oracle Variant 0 /JavaScript eval(‘…’); /Root /Catalog /Pages 128 Oracle Target Classifier
  27. Oracle Execute candidate in vulnerable Adobe Reader in virtual environment

    Behavioral signature: only considered malicious if signature matches https://github.com/cuckoosandbox Simulated network: INetSim HTTP_URL + HOST extracted from API traces
  28. Fitness Function 28 Assumes lost malicious behavior will not be

    recovered = 0 .5 − classifier_score if oracle = "malicious" −∞ otherwise classifier_score ≥ 0.5: labeled malicious
  29. Experimental Results

  30. 30 Original Malicious Seeds Evading PDFrate Malicious Label Threshold

  31. 31 Original Malicious Seeds Evading PDFrate Discovered Evasive Variants 100%

    success rate ∼130 hours on typical desktop
  32. 32 Evading Hidost2013 100% success rate ∼46 hours on typical

    desktop Original Malicious Seeds Discovered Evasive Variants
  33. Evading Hidost2016

  34. Evading Hidost2016 100% success rate ∼14 hours on typical desktop

  35. PDFRate Hidost Seeds Evaded

  36. PDFRate Hidost Seeds Evaded Simple transformations often worked

  37. PDFRate Seeds Evaded (insert, /Root/Pages/Kids, 3:/Root/Pages/Kids/4/Kids/5/) Inserting new pages works

    on 162/500 seeds Training malware often had no/little content
  38. Hidost Seeds Evaded (delete, /Root/OpenAction/JS/Length) Deleting object worked on 1

    seed No impact on malicious behavior
  39. PDFRate Hidost Seeds Evaded Some seeds required complex transformations

  40. Complex Transformations Insert: Threads, ViewerPreferences/Direction, Metadata, Metadata/Length, Metadata/Subtype, Metadata/Type, OpenAction/Contents,

    OpenAction/Contents/Filter, OpenAction/Contents/Length, Pages/MediaBox Delete: AcroForm, Names/JavaSCript/Names/S, AcroForm/DR/Encoding/PDFDocEncoding, AcroForm/DR/Encoding/PDFDocEncoding/Differences, AcroForm/DR/Encoding/PDFDocEncoding/Type, Pages/Rotate, AcroForm/Fields, AcroForm/DA, Outlines/Type, Outlines, Outlines/Count, Pages/Resources/ProcSet, Pages/Resources 85-step mutation trace evading Hidost Effective for 198/500 seeds
  41. Practical, Inexpensive Less than 1 week to find evasive variants

    for all 500 seeds, running on single desktop PC
  42. Possible Defenses

  43. Adjust threshold? Hidost16 results

  44. Adjust threshold? Hidost16 results Variants found with threshold = 0.25

    Variants found with threshold = 0.50
  45. Adjust threshold? PDFRate results Variants found with threshold = 0.25

    Variants found with threshold = 0.50
  46. Retraining Classifier Labelled Training Data ML Algorithm Feature Extraction Vectors

    Deployment Malicious / Benign Operational Data Trained Classifier Training (Supervised Learning)
  47. Labelled Training Data ML Algorithm Feature Extraction Vectors Training (Supervised

    Learning) Clone ✓ ✓ ✗ ✓ EvadeML
  48. Labelled Training Data ML Algorithm Feature Extraction Vectors Training (Supervised

    Learning) Clone ✓ ✓ ✗ ✓ EvadeML Deployment
  49. (Probably) Doesn’t Work Original (Hidost 2016) Retrained (without new benign)

    Retrained (with new benign) Accuracy on Test Set 0.9983 0.9983 0.9983 False negatives on 250 non-training seeds 12 1 2 False positive rate (on benign samples) 0.0% 77% 0.0% Evasion rate 100% 49% 100% more experiments in progress...
  50. Hide the Classifier? “Security through Obscurity” Clone Generated Variants Select

    Variants ✓ ✓ ✗ ✓ Clone Variants Fitness Function Candidate Variant ($%&'() , '(&++ ) Score Benign PDFs Malicious PDF Variants Benign PDFs Malicious PDF Variants Oracle Variant 0 /JavaScript eval(‘…’); /Root /Catalog /Pages 128 Oracle Target Classifier
  51. Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware

    (against Hidost) Automated Evasion PDFrate 2/500 Evasive (0.4% Success) Potentially Good News?
  52. Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware

    (against Hidost) Automated Evasion PDFrate 387/500 Evasive (77.4% Success)
  53. Cross-Evasion Effects PDF Malware Seeds Hidost 13 Evasive PDF Malware

    (against Hidost) Automated Evasion 6/500 Evasive (0.6% Success)
  54. Evading Gmail’s Classifier 54 Evasion rate on Gmail: 179/380 (47.1%)

    for javascript in pdf.all_js: javascript.append_code("var oreilly=1;“) if pdf.get_size() < 7050000: pdf.add_padding(7050000 – pdf.get_size())
  55. Fundamental Problem Classifier features are not intrinsic to malicious behavior

    Adversary can modify those features Artifacts of training data Heuristic search can find evasive variants automatically
  56. Conclusion For source code, technical paper: EvadeML.org If you are

    developing or using malware classifiers, we want to work with you to test them for evadability: evans@virginia.edu Adversaries adapt, classifiers cannot rely on superficial features
  57. David Evans evans@virginia.edu EvadeML.org