⿊黑吃⿊黑的藝術 機器學習惡惡意程式偵測之對抗樣本 報告⼈人：林林殿智 指導老師：Birdman、PK、Benson、⼤大Alan

Whoami • 林林殿智 / Tien-Chih Lin • 成功⼤大學電通所碩⼆二（已畢業） • Research • Car Security • AI-based Malware Detection

「⿊黑盒 v.s. ⿊黑盒。」

Machine Learning in Cyber Security https://www.blackhat.com/docs/us-15/materials/us-15-Klein-Defeating-Machine-Learning-What-Your-Security-Vendor-Is-Not-Telling-You.pdf

Adversarial Example in Image ＋ 0.007 x = “panda” 57.7% confidence “nematode” 8.2% confidence “gibbon” 99.3% confidence Goodfellow, Ian J., Jonathon Shlens, and Christian Szegedy. "Explaining and harnessing adversarial examples." arXiv preprint arXiv:1412.6572 (2014).

Adversarial Example in Malware Detection + Malicious Benign

Problem of Perturbation + = Image + = Broken PE Random Modify

Attack Strategy • Modify feature vector • Modify raw PE • Specially crafted by experts • Gradient-based attack • Reinforcement learning

Feature Vector Perturbation Hu, Weiwei, and Ying Tan. "Generating adversarial malware examples for black-box attacks based on GAN." arXiv preprint arXiv:1702.05983 (2017).

Specially Crafted by Experts https://skylightcyber.com/2019/07/18/cylance-i-kill-you/

• AnalyzeFile hashed C: \Users\Administrator\Desktop\mimikatz_with_slight_modification.exe 143020851E35E3234DBCC879759322E8AD4D6D3E89EAE1F662BF8EA9B9898 D05 • LocalAnalyzeItem LocalInfinity.ComputeScore begin • LocalAnalyzeItem, C: \Users\Administrator\Desktop\mimikatz_with_slight_modification.exe score -852 detector execution_control • Detected as 'Unsafe'! path:'C: \Users\Administrator\Desktop\mimikatz_with_slight_modification.exe' hash: 143020851E35E3234DBCC879759322E8AD4D6D3E89EAE1F662BF8EA9B9898 D05

No content

Gradient-based Attack Demetrio, Luca, et al. "Explaining Vulnerabilities of Deep Learning to Adversarial Malware Binaries." arXiv preprint arXiv:1901.03583 (2019).

Reinforcement Learning https://www.blackhat.com/docs/us-17/thursday/us-17-Anderson-Bot-Vs-Bot-Evading-Machine-Learning-Malware-Detection.pdf http://adsabs.harvard.edu/abs/2018arXiv180108917A

Reinforcement Learning Agent Environment Action Reward State Rt Rt+1 St+1 At St

Agent Environment Action At Reward State Rt Rt+1 St+1 St

Action • adding a function to the import address table that is never used • manipulating existing section names • creating new (unused) sections • appending bytes to extra space at the end of sections • creating a new entry point which immediately jumps to the original entry point • removing signer information • manipulating debug info • packing or unpacking the file • modifying (breaking) header checksum • appending bytes to the overlay(end of PE file)

LIEF for Modify PE https://github.com/lief-project/LIEF

State • Static Windows PE file features compressed to 2350 dimensions • General file information (size) • Header info • Section characteristics • Imported/exported functions • Strings • File byte and entropy histograms

Agent Environment Action At Reward State Rt Rt+1 St+1 St

Attack Target in Original Work • Static PE malware classifier • gradient boosted decision tree • trained on 100,000 malicious and benign samples • ROC-AUC score is 0.993

Attack Windows Defender

Why Attack Win Defender? • Windows’ built-in antivirus • 18% of Windows 7 and Windows 8 are running Windows Defender • more than 50% of Windows 10 are running Windows Defender • Get the full score in AV-Test. https://windowsreport.com/windows-defender-enterprise-antivirus/ https://www.av-test.org/en/antivirus/home-windows/

WannaCry -> upx

Porting Win Defender to Linux https://github.com/taviso/loadlibrary

Start Training Agent

Evade Rate • After training 8 hr • < 40 actions • Evade rate : 81.2%

Before Modification

After Modification

No content

No content

No content

No content

https://i.blackhat.com/us-18/Thu-August-9/us-18-Bulazel-Windows-Offender-Reverse-Engineering-Windows-Defenders-Antivirus-Emulator.pdf

Conclusion • There are blind spots / hallucinate in classifier. • Avoid setting detect engine at local. • Restrict the access frequency. • Do not show the full information (score) in the log.

[email protected] Thanks for Listening.