Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars

DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars

Liang Gong

May 06, 2018
Tweet

More Decks by Liang Gong

Other Decks in Research

Transcript

  1. Presented by Liang Gong DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous

    Cars Yuchi Tian, Kexin Pei, Suman Jana, Baishakhi Ray Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley.
  2. Recap of DeepXplore Liang Gong, Electric Engineering & Computer Science,

    University of California, Berkeley. • Neuron Coverage • Differential Testing + Gradient Descent • Simulate a few realistic scenarios: • Lightening Conditions, Masks, Dirt • Evaluate on different domains 2
  3. DeepTest Liang Gong, Electric Engineering & Computer Science, University of

    California, Berkeley. • Empirically show relations between Neuron Coverage and DNN output • AFL-like guided testing of DNN • Simulate 9 realistic scenarios • Detect thousands of erroneous behaviors 3
  4. Uber Auto Car Kills Pedestrian Liang Gong, Electric Engineering &

    Computer Science, University of California, Berkeley. 4
  5. Uber Auto Car Kills Pedestrian Wearing a black shirt Liang

    Gong, Electric Engineering & Computer Science, University of California, Berkeley. 5 The car did not slow down.
  6. Uber Auto Car Kills Pedestrian Wearing a black shirt The

    car did not slow down. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 6 Cause: Lightening Condition + Image Contrast
  7. Rain fall https://jalopnik.com/this-is-how-bad-self-driving-cars-suck-in-the-rain-1666268433 Liang Gong, Electric Engineering & Computer Science,

    University of California, Berkeley. 8 Hyundai Competition Crashed while testing The sensors failed to pick up street signs, lane markings, and even pedestrians due to the angle of the car shifting in rain and the direction of the sun. Cause: Rain Fall + Lighting Condition + Angle
  8. https://www.nytimes.com/2017/01/19/business/tesla-model-s-autopilot-fatal-crash.html Liang Gong, Electric Engineering & Computer Science, University of

    California, Berkeley. 9 Tesla Autopilot Mode Kills the Driver The camera failed to recognize the white truck against a bright sky. … was killed when his Tesla Model S, which was operating in Autopilot mode, crashed into a tractor-trailer. Krista Kitchen, via Associated Press Cause: Image Contrast
  9. Liang Gong, Electric Engineering & Computer Science, University of California,

    Berkeley. 10 Google Self-driving Car Accident Hit a bus while shifting lane. https://www.wired.com/2016/02/googles-self-driving-car-may-caused-first-crash/
  10. Liang Gong, Electric Engineering & Computer Science, University of California,

    Berkeley. 11 Google Self-driving Car Accident Hit a bus while shifting lane. The car assumed that the bus would yield when it attempted to merge back into traffic. https://www.wired.com/2016/02/googles-self-driving-car-may-caused-first-crash/
  11. https://www.wired.com/2016/02/googles-self-driving-car-may-caused-first-crash/ Liang Gong, Electric Engineering & Computer Science, University of

    California, Berkeley. 12 Google Self-driving Car Accident Hit a bus while shifting lane. The car assumed that the bus would yield when it attempted to merge back into traffic. Cause: failed to estimate speed
  12. Outline Liang Gong, Electric Engineering & Computer Science, University of

    California, Berkeley. Empirically demonstrate that: 13 Changes in Neuron Coverage Changes in the Actions of Self- driving Cars correlate • Generate tests to increase neuron coverage. • Simulate realistic scenarios: fog, rain, angle, scale etc. • Found thousands of erroneous behaviors
  13. Intuition Liang Gong, Electric Engineering & Computer Science, University of

    California, Berkeley. 14 Change the image a little and observe the DNN’s output.
  14. Simplified Model of Self-driving Car Liang Gong, Electric Engineering &

    Computer Science, University of California, Berkeley. 15 Signal lights etc. CNN / RNN
  15. Simplified Model of Self-driving Car Liang Gong, Electric Engineering &

    Computer Science, University of California, Berkeley. 16 Signal lights etc. CNN / RNN
  16. Guided Testing Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 17 • neuron coverage as a metric to guide the testing • Activation of a neuron: • In a fully connect layer • In a convolutional layer • In a RNN
  17. Liang Gong, Electric Engineering & Computer Science, University of California,

    Berkeley. 18 • Activation of a neuron: • In a fully connect layer • In a convolutional layer Output of a neuron is a scalar value (threshold 0.2) Neuron Activation
  18. Liang Gong, Electric Engineering & Computer Science, University of California,

    Berkeley. 19 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Neuron Activation
  19. Liang Gong, Electric Engineering & Computer Science, University of California,

    Berkeley. 20 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Neuron Activation
  20. Liang Gong, Electric Engineering & Computer Science, University of California,

    Berkeley. 21 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Neuron Activation
  21. Liang Gong, Electric Engineering & Computer Science, University of California,

    Berkeley. 22 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Neuron Activation
  22. Liang Gong, Electric Engineering & Computer Science, University of California,

    Berkeley. 23 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Neuron Activation
  23. Liang Gong, Electric Engineering & Computer Science, University of California,

    Berkeley. 24 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Neuron Activation
  24. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 25 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer
  25. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 26 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer
  26. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 27 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Output: a feature map
  27. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 28 • Activation of a neuron: • In a fully connect layer • In a convolutional layer Output: a feature map Average the map to get a scalar value A neuron in a conv layer
  28. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 29 • Activation of a neuron: • In a RNN Unroll the RNN
  29. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 30 • Activation of a neuron: • In a RNN Unroll the RNN
  30. Neuron Activation Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 31 • Activation of a neuron: • In a RNN Unroll the RNN cosnider as different neurons
  31. Increase Coverage with Transformations Liang Gong, Electric Engineering & Computer

    Science, University of California, Berkeley. 32 • Nine different transformations • Brightness • Contrast • Translation • Scaling • Horizontal Shearing • Rotation • Blurring • Fog Effect • Rain Effect
  32. Increase Coverage with Transformations Liang Gong, Electric Engineering & Computer

    Science, University of California, Berkeley. 33 • Nine different transformations • Brightness • Contrast • Translation • Scaling • Horizontal Shearing • Rotation • Blurring • Fog Effect • Rain Effect
  33. Increase Coverage with Transformations Liang Gong, Electric Engineering & Computer

    Science, University of California, Berkeley. 34 • Transformation of Brightness • For each pixel +/- a constant value • Transformation of Contrast • For each pixel * a constant value
  34. Increase Coverage with Transformations Liang Gong, Electric Engineering & Computer

    Science, University of California, Berkeley. 35 • Nine different transformations • Brightness • Contrast • Translation • Scaling • Horizontal Shearing • Rotation • Blurring • Fog Effect • Rain Effect Affine Transformations
  35. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 36 • Moving pixels to new positions rotations Move along x/y axis Transformation Matrix
  36. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 37 • Moving pixels to new positions Transformation Matrix
  37. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 38 • Moving pixels to new positions Transformation Matrix
  38. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 39 • Moving pixels to new positions coordinates of all pixels in the original image Transformation Matrix
  39. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 40 • Moving pixels to new positions coordinates of all pixels in the original image Transformation Matrix coordinates of all pixels in the new image
  40. Affine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 41 • Moving pixels to new positions coordinates of all pixels in the original image Transformation Matrix coordinates of all pixels in the new image
  41. Increase Coverage with Transformations Liang Gong, Electric Engineering & Computer

    Science, University of California, Berkeley. 53 • Nine different transformations • Brightness • Contrast • Translation • Scaling • Horizontal Shearing • Rotation • Blurring • Fog Effect • Rain Effect Photoshop A combination of built-in convolutional transformations
  42. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 54 • Greedily combine transformations to images • Try to maximize the coverage
  43. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 55 • Greedily combine transformations to images • Try to maximize the coverage Pop a seed image from the stack
  44. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 56 • Greedily combine transformations to images • Try to maximize the coverage For the seed image: • A transformation queue • A fail counter
  45. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 57 • Greedily combine transformations to images • Try to maximize the coverage For the seed image: • A upper bound of transform combos that do not increase coverage
  46. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 58 • Greedily combine transformations to images • Try to maximize the coverage dequeue or randomly pick a transformation with random parameters
  47. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 59 • Greedily combine transformations to images • Try to maximize the coverage randomly pick another transformation with random parameters
  48. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 60 • Greedily combine transformations to images • Try to maximize the coverage if improves coverage of the seed image: • Queue the transformation • Record the new image
  49. Combine Transformations Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 61 • Greedily combine transformations to images • Try to maximize the coverage Otherwise increase the fail counter
  50. Specify Test Oracle (Steering Angle) Liang Gong, Electric Engineering &

    Computer Science, University of California, Berkeley. 62
  51. Specify Test Oracle (Steering Angle) Liang Gong, Electric Engineering &

    Computer Science, University of California, Berkeley. 63 predicted angle of the synthesized image Ground truth of the original image
  52. Specify Test Oracle (Steering Angle) Liang Gong, Electric Engineering &

    Computer Science, University of California, Berkeley. 64 predicted angle of the synthesized image Ground truth of the original image
  53. Specify Test Oracle (Steering Angle) Liang Gong, Electric Engineering &

    Computer Science, University of California, Berkeley. 65 predicted angle of the synthesized image Ground truth of the original image predicted angle of the original image
  54. Specify Test Oracle (Steering Angle) Liang Gong, Electric Engineering &

    Computer Science, University of California, Berkeley. 66 predicted angle of the synthesized image Ground truth of the original image predicted angle of the original image n is the number of original images
  55. Specify Test Oracle (Steering Angle) Liang Gong, Electric Engineering &

    Computer Science, University of California, Berkeley. 67 predicted angle of the synthesized image Ground truth of the original image predicted angle of the original image n is the number of original images Error of a Transformed Image ≤ λ * Avg Error of Original Images
  56. DNN Models Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 68 Udacity self-driving challenge top models Max steering angles
  57. DNN Models Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 69 CNN 100 Images 001011010101010… 100 features of 100 images … LSTM Steering angle
  58. DNN Models Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 70 CNN Images Steering angle CNN CNN …
  59. DNN Models Liang Gong, Electric Engineering & Computer Science, University

    of California, Berkeley. 71 CNN Images Steering angle …
  60. Relation between NC and Test Output Liang Gong, Electric Engineering

    & Computer Science, University of California, Berkeley. 72
  61. Relation between NC and Test Output Liang Gong, Electric Engineering

    & Computer Science, University of California, Berkeley. 73 Negative Correlation Larger NC  Smaller Angle [-1, 1] Positive Correlation Larger NC  Larger Angle [-1, 1]
  62. Relation between NC and Test Output Liang Gong, Electric Engineering

    & Computer Science, University of California, Berkeley. 74
  63. Relation between NC and Test Output Liang Gong, Electric Engineering

    & Computer Science, University of California, Berkeley. 75 Larger NC  More Likely to Turn Right
  64. Diff Transformations Activate diff Neurons? Liang Gong, Electric Engineering &

    Computer Science, University of California, Berkeley. 76 1. Randomly Pick a set of images 2. For each Image I and each possible pair of transformations: <T1 , T2 > 3. Apply T1 on I and get cov N1 4. Apply T2 on I and get cov N2 5. Get the Distance between N1 and N2
  65. Diff Transformations Activate diff Neurons? Liang Gong, Electric Engineering &

    Computer Science, University of California, Berkeley. 77 For most models, different transformations activate different set of neurons.
  66. Average Cumulative Neuron Coverage per Input Image Liang Gong, Electric

    Engineering & Computer Science, University of California, Berkeley. 78
  67. Neuron Coverage Increase Caused by Transformations Liang Gong, Electric Engineering

    & Computer Science, University of California, Berkeley. 79
  68. Is the Guided Approach Useful? Liang Gong, Electric Engineering &

    Computer Science, University of California, Berkeley. 80 Baseline 1: 100 seed images Get coverage Baseline 2: Cumulative Transformation Apply single transform on seed images 7000 synthetic Images Get Coverage
  69. Is the Guided Approach Useful? Liang Gong, Electric Engineering &

    Computer Science, University of California, Berkeley. 81 Neuron Coverage (Higher  Better) Baseline 2
  70. Liang Gong, Electric Engineering & Computer Science, University of California,

    Berkeley. 82 Erroneous Behaviors Detected by DeepTest λ = 5
  71. Liang Gong, Electric Engineering & Computer Science, University of California,

    Berkeley. 84 Improvement of Model after Retraining based on Tests from DeepTest Epoch Model