Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AI for security, security for AI

AI for security, security for AI

LINE Developers
PRO

May 29, 2019
Tweet

More Decks by LINE Developers

Other Decks in Technology

Transcript

  1. AI for security, security for AI Jun Sakuma U. Tsukuba

    / RIKEN AIP
  2. training data test data predi ction model learning alg. Overview

    of AI/machine learning
  3. training data test data predi ction model learning alg. Overview

    of AI/machine learning Training the model E.g., a model is trained to recognize face images of “Alice” =Bpb =Alice =Bob =Alice
  4. training data test data predi ction model learning alg. Overview

    of AI/machine learning obtain recognition/prediction =??? =Alice
  5. training data test data predi ction model learning alg. Deployment

    of AI to the real world data submission data submission usage of prediction interpretation of models AI always has interfaces to humans
  6. training data test data predi ction model learning alg. Risk

    of AI in the real world data submission data submission usage of prediction interpretation of models AI might be maliciously affected by adversarial entities
  7. Adversarial examples (AEs) of image recognition Goodfellowet al. (2015)

  8. Adversarial examples (AEs) of image recognition Goodfellowet al. (2015)

  9. How AEs work? Panda-Gibbon boundary of humans Panda-Gibbon boundary of

    a machine Region of AEs
  10. Potential risk: Fooling speaker/face authentication

  11. Image AEs in the physical world

  12. Image AEs in the real world [Etimovet al.] • Paint

    characteristic pattern on a physical road sign (AE) • Recognition model misrecognizes the “STOP” sign as “speed limit 45 mph”
  13. Bugs that work as AEs

  14. Adversarial patch that fools YOLO2 Thyset al., Fooling automated surveillance

    cameras: adversarial patches to attack person detection
  15. Audio AEs in the physical world • Audio AE works

    inside computer but does not work over the air • Over-the-air attack is not easy; never reported with RNN • Can AE be a realistic risk for speech recognition? • If it works, it can affect a large number of devices Audio AE playback broadcast Order 100 pizza! Order 100 pizza! Order 100 pizza!
  16. Audio AE over the air [YS IJCAI‘19] • Audio AE:

    attack on audio response systems (e.g., smartphone, smart speaker) • The world first audio AE that works with deep model over the air • Audio signal has higher diffuseness (e.g., speaker, broadcast) • Audio AE can affect a larger number of devices than image AE Original audio audio AE (recognized as “Hello World”) original audio adversarial noise audio AE deepspeech https://yumetaro.info/projects/audio-ae/
  17. Information leakage from deep models

  18. Potential risk: Fooling face authentication

  19. Model inversion against face recognition models • Deep learning gives

    an abstract label to images • Given a model, can we extract photorealistic images recognized as a specific label? • If yes, it can be a risk (e.g., face authentication) “Keanu Reeves” model inversion? recognition
  20. Basic Idea of GANs: Counterfeiting currency • Minimax game between

    generator and discriminator • Given a random number, generator tries to generate currency that appears to be real • Given either of fake or real currency, discriminator tries to distinguish them correctly • Update generator and discriminator alternately until generator defeats discriminator Fake sample Generate fake Real sample Generator Distinguish fake and real Discriminator Random number
  21. Base technology: Generative adversarial network • Competitive training of two

    deep neural networks • generator: generate fake images look like “real” • discriminator: discriminate fake images and real images Generator discriminate fake from real Discriminator random number real face images fake face images
  22. PreImageGAN Demo

  23. Deep fake

  24. Style transfer by Cycle GAN [Zhu et al., ICCV’17]

  25. Threats of Deep Fake • Spreading disinformation via movies or

    photos • Fabrication of evidences
  26. Differential privacy •Privacy protection with statistical techniques

  27. Apple introduced “differential privacy” into iOS 10

  28. Neural-attentive malware analysis Yakura+, The 8thACM Conference on Data and

    Application Security and Privacy (CODASPY18), to appear • What we usually do: • Disassemble samples and obtain instructions • Interpret instructions line-by-line • So time-consuming. • What we want: • Disassemble samples and obtain instructions • Find instructions characterizing the malware automatically Dis- assemble Analyze by human
  29. Attention mechanism 2 9 • A network architecture that can

    be embedded into CNN • Find regions in the image that are expected to characterize the target class Examples of correspondences between the region in images and the underlined word in captions obtained from the feature map [Xu+, 2015]
  30. Proposal: Neural-attentive Malware Analysis disassemble Identify instructions with strong attention

    Manual analysis
  31. Evaluation: Worm:Win32/Gaobot 3 1 A function to connect to an

    IRC server and enter a chat room • A malware family that spread explosively around 2004 to construct a large botnet [Liang+, 2007] • execute commands sent over IRC • intercepts HTTP / FTP communication to steal login information of PAYPAL[Goel+, 2006]. An image of Backdoor.Win32.Agobot.on, a sample belonging to Worm:Win32/Gaobot, and its attention map obtained. A function to redirect packets to designated destinations to perform DDoS attacks A function to ascertain whether the contents of the intercepted HTTP communication include strings like “PAYPAL”
  32. AI as attacker, AI as defender System AI as attacker

    Adv. Ex. Model inv. AI as defender Malware analysis Human attacker
  33. Security of AI world AI to be secured AI as

    attacker System to be secured AI as defender User to be secured