AI for security, security for AI

Transcript

1. AI for security, security for AI Jun Sakuma U. Tsukuba / RIKEN AIP
2. training data test data predi ction model learning alg. Overview of AI/machine learning
3. training data test data predi ction model learning alg. Overview of AI/machine learning Training the model E.g., a model is trained to recognize face images of “Alice” =Bpb =Alice =Bob =Alice
4. training data test data predi ction model learning alg. Overview of AI/machine learning obtain recognition/prediction =??? =Alice
5. training data test data predi ction model learning alg. Deployment of AI to the real world data submission data submission usage of prediction interpretation of models AI always has interfaces to humans
6. training data test data predi ction model learning alg. Risk of AI in the real world data submission data submission usage of prediction interpretation of models AI might be maliciously affected by adversarial entities
7. Adversarial examples (AEs) of image recognition Goodfellowet al. (2015)
8. Adversarial examples (AEs) of image recognition Goodfellowet al. (2015)
9. How AEs work? Panda-Gibbon boundary of humans Panda-Gibbon boundary of a machine Region of AEs
10. Potential risk: Fooling speaker/face authentication
11. Image AEs in the physical world
12. Image AEs in the real world [Etimovet al.] • Paint characteristic pattern on a physical road sign (AE) • Recognition model misrecognizes the “STOP” sign as “speed limit 45 mph”
13. Bugs that work as AEs
14. Adversarial patch that fools YOLO2 Thyset al., Fooling automated surveillance cameras: adversarial patches to attack person detection
15. Audio AEs in the physical world • Audio AE works inside computer but does not work over the air • Over-the-air attack is not easy; never reported with RNN • Can AE be a realistic risk for speech recognition? • If it works, it can affect a large number of devices Audio AE playback broadcast Order 100 pizza! Order 100 pizza! Order 100 pizza!
16. Audio AE over the air [YS IJCAI‘19] • Audio AE: attack on audio response systems (e.g., smartphone, smart speaker) • The world first audio AE that works with deep model over the air • Audio signal has higher diffuseness (e.g., speaker, broadcast) • Audio AE can affect a larger number of devices than image AE Original audio audio AE (recognized as “Hello World”) original audio adversarial noise audio AE deepspeech https://yumetaro.info/projects/audio-ae/
17. Information leakage from deep models
18. Potential risk: Fooling face authentication
19. Model inversion against face recognition models • Deep learning gives an abstract label to images • Given a model, can we extract photorealistic images recognized as a specific label? • If yes, it can be a risk (e.g., face authentication) “Keanu Reeves” model inversion? recognition
20. Basic Idea of GANs: Counterfeiting currency • Minimax game between generator and discriminator • Given a random number, generator tries to generate currency that appears to be real • Given either of fake or real currency, discriminator tries to distinguish them correctly • Update generator and discriminator alternately until generator defeats discriminator Fake sample Generate fake Real sample Generator Distinguish fake and real Discriminator Random number
21. Base technology: Generative adversarial network • Competitive training of two deep neural networks • generator: generate fake images look like “real” • discriminator: discriminate fake images and real images Generator discriminate fake from real Discriminator random number real face images fake face images
22. PreImageGAN Demo
23. Deep fake
24. Style transfer by Cycle GAN [Zhu et al., ICCV’17]
25. Threats of Deep Fake • Spreading disinformation via movies or photos • Fabrication of evidences
26. Differential privacy •Privacy protection with statistical techniques
27. Apple introduced “differential privacy” into iOS 10
28. Neural-attentive malware analysis Yakura+, The 8thACM Conference on Data and Application Security and Privacy (CODASPY18), to appear • What we usually do: • Disassemble samples and obtain instructions • Interpret instructions line-by-line • So time-consuming. • What we want: • Disassemble samples and obtain instructions • Find instructions characterizing the malware automatically Dis- assemble Analyze by human
29. Attention mechanism 2 9 • A network architecture that can be embedded into CNN • Find regions in the image that are expected to characterize the target class Examples of correspondences between the region in images and the underlined word in captions obtained from the feature map [Xu+, 2015]
30. Proposal: Neural-attentive Malware Analysis disassemble Identify instructions with strong attention Manual analysis
31. Evaluation: Worm:Win32/Gaobot 3 1 A function to connect to an IRC server and enter a chat room • A malware family that spread explosively around 2004 to construct a large botnet [Liang+, 2007] • execute commands sent over IRC • intercepts HTTP / FTP communication to steal login information of PAYPAL[Goel+, 2006]. An image of Backdoor.Win32.Agobot.on, a sample belonging to Worm:Win32/Gaobot, and its attention map obtained. A function to redirect packets to designated destinations to perform DDoS attacks A function to ascertain whether the contents of the intercepted HTTP communication include strings like “PAYPAL”
32. AI as attacker, AI as defender System AI as attacker Adv. Ex. Model inv. AI as defender Malware analysis Human attacker
33. Security of AI world AI to be secured AI as attacker System to be secured AI as defender User to be secured