Ptolemy: Architecture Support for Robust Deep Learning

Slide 1

Slide 1 text

Ptolemy: Architecture Support for Robust Deep Learning Yiming Gan Department of Computer Science, University of Rochester with Yuxian Qiu, Shanghai Jiao Tong University Jingwen Leng, Shanghai Jiao Tong University Minyi Guo, Shanghai Jiao Tong University Yuhao Zhu University of Rochester https://github.com/Ptolemy-dl/Ptolemy

Slide 2

Slide 2 text

Deep Learning: Not Robust

Slide 3

Slide 3 text

Deep Learning: Not Robust Legitimate Example Adversarial Example Perturbation + =

Slide 4

Slide 4 text

Mission Critical System ADAS Security Cameras

Slide 5

Slide 5 text

Robust Deep Learning Requirements • Accurately detect adversarial examples

Slide 6

Slide 6 text

+ Robust Deep Learning Requirements • Accurately detect adversarial examples • Do not bring large overhead on system performance

Slide 7

Slide 7 text

+ Robust Deep Learning Requirements • Accurately detect adversarial examples • Do not bring large overhead on system performance = Ptolemy

Slide 8

Slide 8 text

Hot Path Traditional Software [1]Thomas Ball, James R. Larus, Using Paths to Measure, Explain, and Enhance Program Behavior

Slide 9

Slide 9 text

Hot Path Traditional Software [1]Thomas Ball, James R. Larus, Using Paths to Measure, Explain, and Enhance Program Behavior • Measure Program Behavior • Optimizing Program • Debugging

Slide 10

Slide 10 text

Hot Path Hot Path Deep Learning Traditional Software Layer 1 Layer 2 Layer 3 Layer 4

Slide 11

Slide 11 text

0.2 0.2 0.3 0.3 0.2 0.4 0.4 0.1 0.2 -0.1 0.09 0.1 -1.0 2.1 0.5 Weights = 0.06 0.46 0.44 Output Feature Map Deﬁning Important Neuron Input Feature Map x 0.3 0.4 0.2 1.0 0.1

Slide 12

Slide 12 text

0.2 0.2 0.3 0.3 0.2 0.4 0.4 0.1 0.2 -0.1 0.09 0.1 -1.0 2.1 0.5 Weights = 0.06 0.46 0.44 Output Feature Map Deﬁning Important Neuron Input Feature Map x 0.3 0.4 0.2 1.0 0.1

Slide 13

Slide 13 text

0.2 0.2 0.3 0.3 0.2 0.4 0.4 0.1 0.2 -0.1 0.09 0.1 -1.0 2.1 0.5 Weights = 0.06 0.46 0.44 Output Feature Map Deﬁning Important Neuron Input Feature Map x 0.3 0.4 0.2 1.0 0.1

Slide 14

Slide 14 text

0.2 0.2 0.3 0.3 0.2 0.4 0.4 0.1 0.2 -0.1 0.09 0.1 -1.0 2.1 0.5 Weights = 0.06 0.46 0.44 Output Feature Map Deﬁning Important Neuron Input Feature Map x 0.3 0.4 0.2 1.0 0.1

Slide 15

Slide 15 text

Deﬁning Important Neuron

Slide 16

Slide 16 text

From Neuron to Path Input Layer Hidden Layer Output Layer

Slide 17

Slide 17 text

From Neuron to Path Input Layer Hidden Layer Output Layer

Slide 18

Slide 18 text

From Neuron to Path Input Layer Hidden Layer Output Layer

Slide 19

Slide 19 text

Class Path

Slide 20

Slide 20 text

Class Path } Union

Slide 21

Slide 21 text

Class Path Similarity AlexNet @ ImageNet

Slide 22

Slide 22 text

Class Path Similarity AlexNet @ ImageNet

Slide 23

Slide 23 text

Ptolemy Overview

Slide 24

Slide 24 text

Ptolemy Overview Neural Networks “Cat”

Slide 25

Slide 25 text

Ptolemy Overview Neural Networks “Cat” Extract

Slide 26

Slide 26 text

Ptolemy Overview Neural Networks “Cat” Extract Compare

Slide 27

Slide 27 text

Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N

Slide 28

Slide 28 text

Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Inference

Slide 29

Slide 29 text

Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Inference

Slide 30

Slide 30 text

Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Inference

Slide 31

Slide 31 text

Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Inference

Slide 32

Slide 32 text

Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Extraction

Slide 33

Slide 33 text

Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Extraction

Slide 34

Slide 34 text

Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Extraction

Slide 35

Slide 35 text

Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Extraction

Slide 36

Slide 36 text

Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Extraction IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det

Slide 37

Slide 37 text

Layer 1 Layer 2 …… Layer N-1 Layer N Inference Algorithmic Variation

Slide 38

Slide 38 text

Layer 1 Layer 2 …… Layer N-1 Layer N Inference Extraction Algorithmic Variation

Slide 39

Slide 39 text

Layer 1 Layer 2 …… Layer N-1 Layer N Inference Extraction Algorithmic Variation

Slide 40

Slide 40 text

Layer 1 Layer 2 …… Layer N-1 Layer N Inference Extraction Algorithmic Variation

Slide 41

Slide 41 text

Layer 1 Layer 2 …… Layer N-1 Layer N Extraction Algorithmic Variation

Slide 42

Slide 42 text

Layer 1 Layer 2 …… Layer N-1 Layer N Extraction IF 1 IF 2 … IF N EX 1 EX 2 EX N Det Algorithmic Variation

Slide 43

Slide 43 text

IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det Sorting IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det Threshold IF: Inference, EX: Extraction, Det: Detection Algorithmic Variation

Slide 44

Slide 44 text

IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det Full Extraction IF 1 IF 2 … IF N EX N EX N-1 Det Partially Extraction IF: Inference, EX: Extraction, Det: Detection Algorithmic Variation

Slide 45

Slide 45 text

Framework Backward Forward Sorting Thresholding Fully Extraction Partially Extraction

Slide 46

Slide 46 text

Framework Backward Forward Sorting Full Extraction Partial Extraction = Backward + Fully Extraction + Sorting Thresholding

Slide 47

Slide 47 text

Interface • High-level: Python-based, user deﬁne input

Slide 48

Slide 48 text

Interface • High-level: Python-based, user deﬁne input Compiler • Low-level: Customized ISA

Slide 49

Slide 49 text

Compiler Optimization: Layer Level for j = 1 to L { inf(j) }

Slide 50

Slide 50 text

Compiler Optimization: Layer Level inf(1) for j = 1 to L { inf (j+1) } for j = 1 to L { inf(j) }

Slide 51

Slide 51 text

Compiler Optimization: Neuron Level for j = 1 to N { sort(i) acum(i) }

Slide 52

Slide 52 text

Compiler Optimization: Neuron Level sort(1) for i = 1 to N-1{ sort(i+1) acum(i) } acum(N) for j = 1 to N { sort(i) acum(i) }

Slide 53

Slide 53 text

Architecture Overview DNN Accelerator SRAM (Weights, Feature Maps, Partial Sums, Masks) Path Costructor Sort & Merge Accumulate Controller SRAM (Code, Paths) DRAM Input/Output Weights Feature Maps Partial Sums Masks Gen Masks SRAM (Partial sums, Partial masks, Masks) Paths

Slide 54

Slide 54 text

Enhanced MAC unit i w x + psum >? thd MUX 0/1 mode to SRAM to SRAM

Slide 55

Slide 55 text

Evaluation Network AlexNet, ResNet Dataset Cifar10, Cifar100,ImageNet Attacks BIM, CWL2, DeepFool, FGSM,JSMA Adaptive Attacks Self constructed Baselines EP[1], CDRP[2] [1]Y. Qiu, J. Leng, C. Guo, et.al, Adversarial Defense Through Network Proﬁling Based Path Extraction  [2]Y. Wang, H. Su, B. Zhang, X. Hu, Interpret neural networks by identifying critical data routing paths.

Slide 56

Slide 56 text

Evaluation Backward Forward Sorting Thresholding Full Extraction Partial Extraction Type 1 Type 2 Type 3

Slide 57

Slide 57 text

Hardware Setup DNN Accelerator 20 x 20 Technology Silvaco 15nm On-chip SRAM 1.5MB

Slide 58

Slide 58 text

Evaluation Accuracy 0 1 1 2 3 Hybrid EP CDRP AlexNet on ImageNet

Slide 59

Slide 59 text

Evaluation Accuracy 0.84 0.88 0.92 0.96 1 1 2 3 Hybrid EP CDRP AlexNet on ImageNet

Slide 60

Slide 60 text

Evaluation Accuracy 0.84 0.88 0.92 0.96 1 1 2 3 Hybrid EP CDRP AlexNet on ImageNet

Slide 61

Slide 61 text

Evaluation Accuracy 0.84 0.88 0.92 0.96 1 1 2 3 Hybrid EP CDRP AlexNet on ImageNet Accuracy decrease

Slide 62

Slide 62 text

Evaluation Latency Overhead 0 4 8 12 16 BwCU BwAb FwAb Hybrid EP AlexNet on ImageNet Energy Overhead 0 2 4 6 8 BwCU BwAb FwAb Hybrid EP

Slide 63

Slide 63 text

Evaluation Latency Overhead 0 4 8 12 16 BwCU BwAb FwAb Hybrid EP AlexNet on ImageNet Energy Overhead 0 2 4 6 8 BwCU BwAb FwAb Hybrid EP

Slide 64

Slide 64 text

Evaluation Latency Overhead 0 4 8 12 16 BwCU BwAb FwAb Hybrid EP AlexNet on ImageNet Energy Overhead 0 2 4 6 8 BwCU BwAb FwAb Hybrid EP

Slide 65

Slide 65 text

Latency Overhead 0 4 8 12 16 BwCU BwAb FwAb Hybrid EP Latency Overhead Decrease Evaluation AlexNet on ImageNet Energy Overhead 0 2 4 6 8 BwCU BwAb FwAb Hybrid EP Energy Overhead Decrease

Slide 66

Slide 66 text

Conclusion Ptolemy: Accurate, low overhead, adversarial attack detection • Algorithm Framework • Compiler Optimization • Architecture Support

Slide 67

Slide 67 text

Collaborators Yuxian Qiu Jingwen Leng Minyi Guo Yuhao Zhu

Slide 68

Slide 68 text

Questions https://github.com/Ptolemy-dl/Ptolemy

Slide 69

Slide 69 text

Evaluation Accuracy 0 1 8 7 6 5 4 3 2 1 Termination Layer Latency Overhead 0 1 2 3 4 8 7 6 5 4 3 2 1 Termination Layer

Slide 70

Slide 70 text

Evaluation Accuracy 0.84 0.91 8 7 6 5 4 3 2 1 Termination Layer Latency Overhead 0 4 8 12 16 8 7 6 5 4 3 2 1 Termination Layer

Slide 71

Slide 71 text

Backup def AdversaryDetection(model, input, θ, φ): output = Inference(model, input) N = model.num_layers // Selective extraction only in the last three layers for L in range(N-3, N): if L != N-1: // Forward extraction using absolute thresholds ImptN[L] = ExtractImptNeurons(1, 1, φ, L) else: // Forward extraction using cumulative thresholds ImptN[L] = ExtractImptNeurons(1, 0, θ, L) dynPath.concat(GenMask(ImptN[L])) classPath = LoadClassPath(argmax(output)) is_adversary = Classify(classPath, dynPath) return is_adversary 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15