Ptolemy: Architecture Support for Robust Deep Learning

MICRO 2020 talk by Yiming Gan

Transcript

1. Ptolemy: Architecture Support for Robust Deep Learning Yiming Gan Department of Computer Science, University of Rochester with Yuxian Qiu, Shanghai Jiao Tong University Jingwen Leng, Shanghai Jiao Tong University Minyi Guo, Shanghai Jiao Tong University Yuhao Zhu University of Rochester https://github.com/Ptolemy-dl/Ptolemy
2. Deep Learning: Not Robust
3. Deep Learning: Not Robust Legitimate Example Adversarial Example Perturbation + =
4. Mission Critical System ADAS Security Cameras
5. Robust Deep Learning Requirements • Accurately detect adversarial examples
6. + Robust Deep Learning Requirements • Accurately detect adversarial examples • Do not bring large overhead on system performance
7. + Robust Deep Learning Requirements • Accurately detect adversarial examples • Do not bring large overhead on system performance = Ptolemy
8. Hot Path Traditional Software [1]Thomas Ball, James R. Larus, Using Paths to Measure, Explain, and Enhance Program Behavior
9. Hot Path Traditional Software [1]Thomas Ball, James R. Larus, Using Paths to Measure, Explain, and Enhance Program Behavior • Measure Program Behavior • Optimizing Program • Debugging
10. Hot Path Hot Path Deep Learning Traditional Software Layer 1 Layer 2 Layer 3 Layer 4
11. 0.2 0.2 0.3 0.3 0.2 0.4 0.4 0.1 0.2 -0.1 0.09 0.1 -1.0 2.1 0.5 Weights = 0.06 0.46 0.44 Output Feature Map Defining Important Neuron Input Feature Map x 0.3 0.4 0.2 1.0 0.1
12. 0.2 0.2 0.3 0.3 0.2 0.4 0.4 0.1 0.2 -0.1 0.09 0.1 -1.0 2.1 0.5 Weights = 0.06 0.46 0.44 Output Feature Map Defining Important Neuron Input Feature Map x 0.3 0.4 0.2 1.0 0.1
13. 0.2 0.2 0.3 0.3 0.2 0.4 0.4 0.1 0.2 -0.1 0.09 0.1 -1.0 2.1 0.5 Weights = 0.06 0.46 0.44 Output Feature Map Defining Important Neuron Input Feature Map x 0.3 0.4 0.2 1.0 0.1
14. 0.2 0.2 0.3 0.3 0.2 0.4 0.4 0.1 0.2 -0.1 0.09 0.1 -1.0 2.1 0.5 Weights = 0.06 0.46 0.44 Output Feature Map Defining Important Neuron Input Feature Map x 0.3 0.4 0.2 1.0 0.1
15. Defining Important Neuron
16. From Neuron to Path Input Layer Hidden Layer Output Layer
17. From Neuron to Path Input Layer Hidden Layer Output Layer
18. From Neuron to Path Input Layer Hidden Layer Output Layer
19. Class Path
20. Class Path } Union
21. Class Path Similarity AlexNet @ ImageNet
22. Class Path Similarity AlexNet @ ImageNet
23. Ptolemy Overview
24. Ptolemy Overview Neural Networks “Cat”
25. Ptolemy Overview Neural Networks “Cat” Extract
26. Ptolemy Overview Neural Networks “Cat” Extract Compare
27. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N
28. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Inference
29. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Inference
30. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Inference
31. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Inference
32. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Extraction
33. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Extraction
34. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Extraction
35. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Extraction
36. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer N Extraction IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det
37. Layer 1 Layer 2 …… Layer N-1 Layer N Inference Algorithmic Variation
38. Layer 1 Layer 2 …… Layer N-1 Layer N Inference Extraction Algorithmic Variation
39. Layer 1 Layer 2 …… Layer N-1 Layer N Inference Extraction Algorithmic Variation
40. Layer 1 Layer 2 …… Layer N-1 Layer N Inference Extraction Algorithmic Variation
41. Layer 1 Layer 2 …… Layer N-1 Layer N Extraction Algorithmic Variation
42. Layer 1 Layer 2 …… Layer N-1 Layer N Extraction IF 1 IF 2 … IF N EX 1 EX 2 EX N Det Algorithmic Variation
43. IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det Sorting IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det Threshold IF: Inference, EX: Extraction, Det: Detection Algorithmic Variation
44. IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det Full Extraction IF 1 IF 2 … IF N EX N EX N-1 Det Partially Extraction IF: Inference, EX: Extraction, Det: Detection Algorithmic Variation
45. Framework Backward Forward Sorting Thresholding Fully Extraction Partially Extraction
46. Framework Backward Forward Sorting Full Extraction Partial Extraction = Backward + Fully Extraction + Sorting Thresholding
47. Interface • High-level: Python-based, user define input
48. Interface • High-level: Python-based, user define input Compiler • Low-level: Customized ISA
49. Compiler Optimization: Layer Level for j = 1 to L { inf(j) <extraction on layer j> }
50. Compiler Optimization: Layer Level inf(1) for j = 1 to L { inf (j+1) <extraction on layer j> } <extraction on layer L> for j = 1 to L { inf(j) <extraction on layer j> }
51. Compiler Optimization: Neuron Level for j = 1 to N { sort(i) acum(i) }
52. Compiler Optimization: Neuron Level sort(1) for i = 1 to N-1{ sort(i+1) acum(i) } acum(N) for j = 1 to N { sort(i) acum(i) }
53. Architecture Overview DNN Accelerator SRAM (Weights, Feature Maps, Partial Sums, Masks) Path Costructor Sort & Merge Accumulate Controller SRAM (Code, Paths) DRAM Input/Output Weights Feature Maps Partial Sums Masks Gen Masks SRAM (Partial sums, Partial masks, Masks) Paths
54. Enhanced MAC unit i w x + psum >? thd MUX 0/1 mode to SRAM to SRAM
55. Evaluation Network AlexNet, ResNet Dataset Cifar10, Cifar100,ImageNet Attacks BIM, CWL2, DeepFool, FGSM,JSMA Adaptive Attacks Self constructed Baselines EP[1], CDRP[2] [1]Y. Qiu, J. Leng, C. Guo, et.al, Adversarial Defense Through Network Profiling Based Path Extraction
 [2]Y. Wang, H. Su, B. Zhang, X. Hu, Interpret neural networks by identifying critical data routing paths.
56. Evaluation Backward Forward Sorting Thresholding Full Extraction Partial Extraction Type 1 Type 2 Type 3
57. Hardware Setup DNN Accelerator 20 x 20 Technology Silvaco 15nm On-chip SRAM 1.5MB
58. Evaluation Accuracy 0 1 1 2 3 Hybrid EP CDRP AlexNet on ImageNet
59. Evaluation Accuracy 0.84 0.88 0.92 0.96 1 1 2 3 Hybrid EP CDRP AlexNet on ImageNet
60. Evaluation Accuracy 0.84 0.88 0.92 0.96 1 1 2 3 Hybrid EP CDRP AlexNet on ImageNet
61. Evaluation Accuracy 0.84 0.88 0.92 0.96 1 1 2 3 Hybrid EP CDRP AlexNet on ImageNet Accuracy decrease
62. Evaluation Latency Overhead 0 4 8 12 16 BwCU BwAb FwAb Hybrid EP AlexNet on ImageNet Energy Overhead 0 2 4 6 8 BwCU BwAb FwAb Hybrid EP
63. Evaluation Latency Overhead 0 4 8 12 16 BwCU BwAb FwAb Hybrid EP AlexNet on ImageNet Energy Overhead 0 2 4 6 8 BwCU BwAb FwAb Hybrid EP
64. Evaluation Latency Overhead 0 4 8 12 16 BwCU BwAb FwAb Hybrid EP AlexNet on ImageNet Energy Overhead 0 2 4 6 8 BwCU BwAb FwAb Hybrid EP
65. Latency Overhead 0 4 8 12 16 BwCU BwAb FwAb Hybrid EP Latency Overhead Decrease Evaluation AlexNet on ImageNet Energy Overhead 0 2 4 6 8 BwCU BwAb FwAb Hybrid EP Energy Overhead Decrease
66. Conclusion Ptolemy: Accurate, low overhead, adversarial attack detection • Algorithm Framework • Compiler Optimization • Architecture Support
67. Collaborators Yuxian Qiu Jingwen Leng Minyi Guo Yuhao Zhu
68. Questions https://github.com/Ptolemy-dl/Ptolemy
69. Evaluation Accuracy 0 1 8 7 6 5 4 3 2 1 Termination Layer Latency Overhead 0 1 2 3 4 8 7 6 5 4 3 2 1 Termination Layer
70. Evaluation Accuracy 0.84 0.91 8 7 6 5 4 3 2 1 Termination Layer Latency Overhead 0 4 8 12 16 8 7 6 5 4 3 2 1 Termination Layer
71. Backup def AdversaryDetection(model, input, θ, φ): output = Inference(model, input) N = model.num_layers // Selective extraction only in the last three layers for L in range(N-3, N): if L != N-1: // Forward extraction using absolute thresholds ImptN[L] = ExtractImptNeurons(1, 1, φ, L) else: // Forward extraction using cumulative thresholds ImptN[L] = ExtractImptNeurons(1, 0, θ, L) dynPath.concat(GenMask(ImptN[L])) classPath = LoadClassPath(argmax(output)) is_adversary = Classify(classPath, dynPath) return is_adversary 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
72. Backup
73. Backup
74. Backup
75. Backup