DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars

Presented by Liang Gong DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous
Cars Yuchi Tian, Kexin Pei, Suman Jana, Baishakhi Ray Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley.

Recap of DeepXplore Liang Gong, Electric Engineering & Computer Science,
University of California, Berkeley. • Neuron Coverage • Differential Testing + Gradient Descent • Simulate a few realistic scenarios: • Lightening Conditions, Masks, Dirt • Evaluate on different domains 2

DeepTest Liang Gong, Electric Engineering & Computer Science, University of
California, Berkeley. • Empirically show relations between Neuron Coverage and DNN output • AFL-like guided testing of DNN • Simulate 9 realistic scenarios • Detect thousands of erroneous behaviors 3

Uber Auto Car Kills Pedestrian Liang Gong, Electric Engineering &
Computer Science, University of California, Berkeley. 4

Uber Auto Car Kills Pedestrian Wearing a black shirt Liang
Gong, Electric Engineering & Computer Science, University of California, Berkeley. 5 The car did not slow down.

Uber Auto Car Kills Pedestrian Wearing a black shirt The
car did not slow down. Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. 6 Cause: Lightening Condition + Image Contrast

https://jalopnik.com/this-is-how-bad-self-driving-cars-suck-in-the-rain-1666268433 Liang Gong, Electric Engineering & Computer Science, University of
California, Berkeley. 7 Hyundai Competition Crashed while testing Rain fall

Rain fall https://jalopnik.com/this-is-how-bad-self-driving-cars-suck-in-the-rain-1666268433 Liang Gong, Electric Engineering & Computer Science,
University of California, Berkeley. 8 Hyundai Competition Crashed while testing The sensors failed to pick up street signs, lane markings, and even pedestrians due to the angle of the car shifting in rain and the direction of the sun. Cause: Rain Fall + Lighting Condition + Angle

https://www.nytimes.com/2017/01/19/business/tesla-model-s-autopilot-fatal-crash.html Liang Gong, Electric Engineering & Computer Science, University of
California, Berkeley. 9 Tesla Autopilot Mode Kills the Driver The camera failed to recognize the white truck against a bright sky. … was killed when his Tesla Model S, which was operating in Autopilot mode, crashed into a tractor-trailer. Krista Kitchen, via Associated Press Cause: Image Contrast

Liang Gong, Electric Engineering & Computer Science, University of California,
Berkeley. 10 Google Self-driving Car Accident Hit a bus while shifting lane. https://www.wired.com/2016/02/googles-self-driving-car-may-caused-first-crash/

Berkeley. 11 Google Self-driving Car Accident Hit a bus while shifting lane. The car assumed that the bus would yield when it attempted to merge back into traffic. https://www.wired.com/2016/02/googles-self-driving-car-may-caused-first-crash/

https://www.wired.com/2016/02/googles-self-driving-car-may-caused-first-crash/ Liang Gong, Electric Engineering & Computer Science, University of
California, Berkeley. 12 Google Self-driving Car Accident Hit a bus while shifting lane. The car assumed that the bus would yield when it attempted to merge back into traffic. Cause: failed to estimate speed

Outline Liang Gong, Electric Engineering & Computer Science, University of
California, Berkeley. Empirically demonstrate that: 13 Changes in Neuron Coverage Changes in the Actions of Self- driving Cars correlate • Generate tests to increase neuron coverage. • Simulate realistic scenarios: fog, rain, angle, scale etc. • Found thousands of erroneous behaviors

Intuition Liang Gong, Electric Engineering & Computer Science, University of
California, Berkeley. 14 Change the image a little and observe the DNN’s output.

Simplified Model of Self-driving Car Liang Gong, Electric Engineering &
Computer Science, University of California, Berkeley. 15 Signal lights etc. CNN / RNN

Simplified Model of Self-driving Car Liang Gong, Electric Engineering &
Computer Science, University of California, Berkeley. 16 Signal lights etc. CNN / RNN

Guided Testing Liang Gong, Electric Engineering & Computer Science, University
of California, Berkeley. 17 • neuron coverage as a metric to guide the testing • Activation of a neuron: • In a fully connect layer • In a convolutional layer • In a RNN

Berkeley. 18 • Activation of a neuron: • In a fully connect layer • In a convolutional layer Output of a neuron is a scalar value (threshold 0.2) Neuron Activation

Berkeley. 19 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Neuron Activation

Neuron Activation Liang Gong, Electric Engineering & Computer Science, University
of California, Berkeley. 25 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer

of California, Berkeley. 26 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer

of California, Berkeley. 27 • Activation of a neuron: • In a fully connect layer • In a convolutional layer A neuron in a conv layer Output: a feature map

of California, Berkeley. 28 • Activation of a neuron: • In a fully connect layer • In a convolutional layer Output: a feature map Average the map to get a scalar value A neuron in a conv layer

of California, Berkeley. 29 • Activation of a neuron: • In a RNN Unroll the RNN

of California, Berkeley. 30 • Activation of a neuron: • In a RNN Unroll the RNN

of California, Berkeley. 31 • Activation of a neuron: • In a RNN Unroll the RNN cosnider as different neurons

Increase Coverage with Transformations Liang Gong, Electric Engineering & Computer
Science, University of California, Berkeley. 32 • Nine different transformations • Brightness • Contrast • Translation • Scaling • Horizontal Shearing • Rotation • Blurring • Fog Effect • Rain Effect

Science, University of California, Berkeley. 33 • Nine different transformations • Brightness • Contrast • Translation • Scaling • Horizontal Shearing • Rotation • Blurring • Fog Effect • Rain Effect

Science, University of California, Berkeley. 34 • Transformation of Brightness • For each pixel +/- a constant value • Transformation of Contrast • For each pixel * a constant value

Science, University of California, Berkeley. 35 • Nine different transformations • Brightness • Contrast • Translation • Scaling • Horizontal Shearing • Rotation • Blurring • Fog Effect • Rain Effect Affine Transformations

Affine Transformations Liang Gong, Electric Engineering & Computer Science, University
of California, Berkeley. 36 • Moving pixels to new positions rotations Move along x/y axis Transformation Matrix

of California, Berkeley. 37 • Moving pixels to new positions Transformation Matrix

of California, Berkeley. 38 • Moving pixels to new positions Transformation Matrix

of California, Berkeley. 39 • Moving pixels to new positions coordinates of all pixels in the original image Transformation Matrix

of California, Berkeley. 40 • Moving pixels to new positions coordinates of all pixels in the original image Transformation Matrix coordinates of all pixels in the new image

of California, Berkeley. 41 • Moving pixels to new positions coordinates of all pixels in the original image Transformation Matrix coordinates of all pixels in the new image

of California, Berkeley. 42

Science, University of California, Berkeley. 53 • Nine different transformations • Brightness • Contrast • Translation • Scaling • Horizontal Shearing • Rotation • Blurring • Fog Effect • Rain Effect Photoshop A combination of built-in convolutional transformations

Combine Transformations Liang Gong, Electric Engineering & Computer Science, University
of California, Berkeley. 54 • Greedily combine transformations to images • Try to maximize the coverage

of California, Berkeley. 55 • Greedily combine transformations to images • Try to maximize the coverage Pop a seed image from the stack

of California, Berkeley. 56 • Greedily combine transformations to images • Try to maximize the coverage For the seed image: • A transformation queue • A fail counter

of California, Berkeley. 57 • Greedily combine transformations to images • Try to maximize the coverage For the seed image: • A upper bound of transform combos that do not increase coverage

of California, Berkeley. 58 • Greedily combine transformations to images • Try to maximize the coverage dequeue or randomly pick a transformation with random parameters

of California, Berkeley. 59 • Greedily combine transformations to images • Try to maximize the coverage randomly pick another transformation with random parameters

of California, Berkeley. 60 • Greedily combine transformations to images • Try to maximize the coverage if improves coverage of the seed image: • Queue the transformation • Record the new image

of California, Berkeley. 61 • Greedily combine transformations to images • Try to maximize the coverage Otherwise increase the fail counter

Specify Test Oracle (Steering Angle) Liang Gong, Electric Engineering &
Computer Science, University of California, Berkeley. 62

Computer Science, University of California, Berkeley. 63 predicted angle of the synthesized image Ground truth of the original image

Computer Science, University of California, Berkeley. 64 predicted angle of the synthesized image Ground truth of the original image

Computer Science, University of California, Berkeley. 65 predicted angle of the synthesized image Ground truth of the original image predicted angle of the original image

Computer Science, University of California, Berkeley. 66 predicted angle of the synthesized image Ground truth of the original image predicted angle of the original image n is the number of original images

Computer Science, University of California, Berkeley. 67 predicted angle of the synthesized image Ground truth of the original image predicted angle of the original image n is the number of original images Error of a Transformed Image ≤ λ * Avg Error of Original Images

DNN Models Liang Gong, Electric Engineering & Computer Science, University
of California, Berkeley. 68 Udacity self-driving challenge top models Max steering angles

of California, Berkeley. 69 CNN 100 Images 001011010101010… 100 features of 100 images … LSTM Steering angle

of California, Berkeley. 70 CNN Images Steering angle CNN CNN …

of California, Berkeley. 71 CNN Images Steering angle …

Relation between NC and Test Output Liang Gong, Electric Engineering
& Computer Science, University of California, Berkeley. 72

& Computer Science, University of California, Berkeley. 73 Negative Correlation Larger NC  Smaller Angle [-1, 1] Positive Correlation Larger NC  Larger Angle [-1, 1]

& Computer Science, University of California, Berkeley. 75 Larger NC  More Likely to Turn Right

Diff Transformations Activate diff Neurons? Liang Gong, Electric Engineering &
Computer Science, University of California, Berkeley. 76 1. Randomly Pick a set of images 2. For each Image I and each possible pair of transformations: <T1 , T2 > 3. Apply T1 on I and get cov N1 4. Apply T2 on I and get cov N2 5. Get the Distance between N1 and N2

Diff Transformations Activate diff Neurons? Liang Gong, Electric Engineering &
Computer Science, University of California, Berkeley. 77 For most models, different transformations activate different set of neurons.

Average Cumulative Neuron Coverage per Input Image Liang Gong, Electric
Engineering & Computer Science, University of California, Berkeley. 78

Neuron Coverage Increase Caused by Transformations Liang Gong, Electric Engineering

Is the Guided Approach Useful? Liang Gong, Electric Engineering &
Computer Science, University of California, Berkeley. 80 Baseline 1: 100 seed images Get coverage Baseline 2: Cumulative Transformation Apply single transform on seed images 7000 synthetic Images Get Coverage

Is the Guided Approach Useful? Liang Gong, Electric Engineering &
Computer Science, University of California, Berkeley. 81 Neuron Coverage (Higher  Better) Baseline 2

Berkeley. 82 Erroneous Behaviors Detected by DeepTest λ = 5

Berkeley. 83 λ = 5 False Positives

Berkeley. 84 Improvement of Model after Retraining based on Tests from DeepTest Epoch Model

DeepTest: Automated Testing of Deep-Neural-Netw...

DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars

More Decks by Liang Gong

Other Decks in Research

Featured

Transcript