Magic Tricks for Self-driving Cars

C00609f16933013a7c81ecd13d50b65e?s=47 Weilin Xu
August 11, 2018

Magic Tricks for Self-driving Cars

A highlighted talk at the Defcon 2018 CAAD Village. I conducted this work at Baidu X-Lab in summer 2018 while I was an intern researcher.

C00609f16933013a7c81ecd13d50b65e?s=128

Weilin Xu

August 11, 2018
Tweet

Transcript

  1. Magic Tricks for Self-driving Cars Weilin Xu, Zhenyu Zhong, Yunhan

    Jia
  2. • This is a proof-of-concept. • We are NOT targeting

    at any autonomous vehicle vendors. • Don’t try to fool your neighbor’s car. AUTHORS’ WARNING
  3. Autonomous Vehicle Framework Radar LiDAR Camera Image: https://medium.com/toyota-ai-ventures/https-medium-com-toyota-ai-ventures-announcingblackmore-7947eacc9e9e Camera

  4. Camera-based Obstacle Detection

  5. None
  6. Our Target: YOLOv3 YOLO v3 Object Detection Model [147 Layers,

    62M Parameters] Input [416x416x3] Output [3549 Bounding Boxes] Image: http://media.nj.com/traffic_impact/photo/all-way-stop-sign-that-flashes-in-montclairjpg-30576ab330660eff.jpg
  7. Trained with the COCO Dataset • Common Objects in Context

    • 80 Classes: person, [car, truck, bus], [bicycle, motorcycle], [stop sign, traffic light], etc. Source: http://cocodataset.org/
  8. car 0.01% YOLOv3 Inference 116x9 0 156x 198 373x326 Anchor

    Boxes !" = $ %" + '" !( = $ %( + '( !) = *) +,- Center Point Object Size !. = *. +,/ *. *) 13 x 13 Grid ('" , '( ) = (11,2) Prediction Vector Bounding Box 80 Class Confidence Objectness %" %( %) %. *567 '8 '9 … … ':; '<= ▪ stop sign 99% × car 0.01% car 0.01%
  9. Threat Model: Image Patch Attack Company Logo

  10. Threat Model: Image Patch Attack Company Logo

  11. Threat Model: Image Patch Attack Company Logo

  12. Threat Model: Image Patch Attack Company Logo

  13. Attack Algorithms • Input Construction • Objectives • Optimization

  14. Resize Perspective Transform Differentiable Input Construction Company Logo Company Logo

  15. Objectives • Object Production • Object Vanish • Object Transformation

  16. Object Production - Coarse We want more certain objects on

    the whole image. • Easy to implement. • May be difficult to optimize. Company Logo
  17. Object Production - Precise We want a certain object of

    a specific size in a specific location. Company Logo
  18. Object Vanish - Coarse We want a certain object class

    to vanish on the whole image.
  19. Object Transformation - Coarse We want certain object class to

    transform to other class.
  20. Optimization • Change of variable Convert to tanh() space to

    encode the [0, 1] interval constraint. Friendly to many off-the-shelf optimizers, e.g. Adam. • To optimize logits Skip sigmoid() to avoid vanishing gradient. Carlini, Nicholas, and David Wagner. “Towards Evaluating the Robustness of Neural Networks.” IEEE S&P (Oakland) 2016.
  21. But, Image Sensing is not an Identity Function Model Input

    [416x416x3] § Limited Resolution § Distortions § Random Noise § … Digital Image Scene
  22. Towards Robust Physical Adversarial Examples • [Limited Resolution] Smoother patch

    with the total variation regularization. • [Distortions] Color management with the non-printability loss. • [Inaccurate Patch] Random transformation during optimization iterations. • …… Sharif, Mahmood, et al. "Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition." ACM CCS 2016.
  23. Conclusion • Magicians can fool object detection models, so can

    attackers. • We should be cautious with self-driving cars that rely on computer vision.