Ptolemy: Architecture Support for Robust Deep Learning

F0c4b39a71fc7c752d4e6c451f6f678b?s=47 HorizonLab
October 28, 2020

Ptolemy: Architecture Support for Robust Deep Learning

MICRO 2020 talk by Yiming Gan

F0c4b39a71fc7c752d4e6c451f6f678b?s=128

HorizonLab

October 28, 2020
Tweet

Transcript

  1. Ptolemy: Architecture Support for Robust Deep Learning Yiming Gan Department

    of Computer Science, University of Rochester with Yuxian Qiu, Shanghai Jiao Tong University Jingwen Leng, Shanghai Jiao Tong University Minyi Guo, Shanghai Jiao Tong University Yuhao Zhu University of Rochester https://github.com/Ptolemy-dl/Ptolemy
  2. Deep Learning: Not Robust

  3. Deep Learning: Not Robust Legitimate Example Adversarial Example Perturbation +

    =
  4. Mission Critical System ADAS Security Cameras

  5. Robust Deep Learning Requirements • Accurately detect adversarial examples

  6. + Robust Deep Learning Requirements • Accurately detect adversarial examples

    • Do not bring large overhead on system performance
  7. + Robust Deep Learning Requirements • Accurately detect adversarial examples

    • Do not bring large overhead on system performance = Ptolemy
  8. Hot Path Traditional Software [1]Thomas Ball, James R. Larus, Using

    Paths to Measure, Explain, and Enhance Program Behavior
  9. Hot Path Traditional Software [1]Thomas Ball, James R. Larus, Using

    Paths to Measure, Explain, and Enhance Program Behavior • Measure Program Behavior • Optimizing Program • Debugging
  10. Hot Path Hot Path Deep Learning Traditional Software Layer 1

    Layer 2 Layer 3 Layer 4
  11. 0.2 0.2 0.3 0.3 0.2 0.4 0.4 0.1 0.2 -0.1

    0.09 0.1 -1.0 2.1 0.5 Weights = 0.06 0.46 0.44 Output Feature Map Defining Important Neuron Input Feature Map x 0.3 0.4 0.2 1.0 0.1
  12. 0.2 0.2 0.3 0.3 0.2 0.4 0.4 0.1 0.2 -0.1

    0.09 0.1 -1.0 2.1 0.5 Weights = 0.06 0.46 0.44 Output Feature Map Defining Important Neuron Input Feature Map x 0.3 0.4 0.2 1.0 0.1
  13. 0.2 0.2 0.3 0.3 0.2 0.4 0.4 0.1 0.2 -0.1

    0.09 0.1 -1.0 2.1 0.5 Weights = 0.06 0.46 0.44 Output Feature Map Defining Important Neuron Input Feature Map x 0.3 0.4 0.2 1.0 0.1
  14. 0.2 0.2 0.3 0.3 0.2 0.4 0.4 0.1 0.2 -0.1

    0.09 0.1 -1.0 2.1 0.5 Weights = 0.06 0.46 0.44 Output Feature Map Defining Important Neuron Input Feature Map x 0.3 0.4 0.2 1.0 0.1
  15. Defining Important Neuron

  16. From Neuron to Path Input Layer Hidden Layer Output Layer

  17. From Neuron to Path Input Layer Hidden Layer Output Layer

  18. From Neuron to Path Input Layer Hidden Layer Output Layer

  19. Class Path

  20. Class Path } Union

  21. Class Path Similarity AlexNet @ ImageNet

  22. Class Path Similarity AlexNet @ ImageNet

  23. Ptolemy Overview

  24. Ptolemy Overview Neural Networks “Cat”

  25. Ptolemy Overview Neural Networks “Cat” Extract

  26. Ptolemy Overview Neural Networks “Cat” Extract Compare

  27. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer

    N
  28. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer

    N Inference
  29. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer

    N Inference
  30. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer

    N Inference
  31. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer

    N Inference
  32. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer

    N Extraction
  33. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer

    N Extraction
  34. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer

    N Extraction
  35. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer

    N Extraction
  36. Ptolemy Pipeline Layer 1 Layer 2 …… Layer N-1 Layer

    N Extraction IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det
  37. Layer 1 Layer 2 …… Layer N-1 Layer N Inference

    Algorithmic Variation
  38. Layer 1 Layer 2 …… Layer N-1 Layer N Inference

    Extraction Algorithmic Variation
  39. Layer 1 Layer 2 …… Layer N-1 Layer N Inference

    Extraction Algorithmic Variation
  40. Layer 1 Layer 2 …… Layer N-1 Layer N Inference

    Extraction Algorithmic Variation
  41. Layer 1 Layer 2 …… Layer N-1 Layer N Extraction

    Algorithmic Variation
  42. Layer 1 Layer 2 …… Layer N-1 Layer N Extraction

    IF 1 IF 2 … IF N EX 1 EX 2 EX N Det Algorithmic Variation
  43. IF 1 IF 2 … IF N EX N EX

    N-1 … EX 1 Det Sorting IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det Threshold IF: Inference, EX: Extraction, Det: Detection Algorithmic Variation
  44. IF 1 IF 2 … IF N EX N EX

    N-1 … EX 1 Det Full Extraction IF 1 IF 2 … IF N EX N EX N-1 Det Partially Extraction IF: Inference, EX: Extraction, Det: Detection Algorithmic Variation
  45. Framework Backward Forward Sorting Thresholding Fully Extraction Partially Extraction

  46. Framework Backward Forward Sorting Full Extraction Partial Extraction = Backward

    + Fully Extraction + Sorting Thresholding
  47. Interface • High-level: Python-based, user define input

  48. Interface • High-level: Python-based, user define input Compiler • Low-level:

    Customized ISA
  49. Compiler Optimization: Layer Level for j = 1 to L

    { inf(j) <extraction on layer j> }
  50. Compiler Optimization: Layer Level inf(1) for j = 1 to

    L { inf (j+1) <extraction on layer j> } <extraction on layer L> for j = 1 to L { inf(j) <extraction on layer j> }
  51. Compiler Optimization: Neuron Level for j = 1 to N

    { sort(i) acum(i) }
  52. Compiler Optimization: Neuron Level sort(1) for i = 1 to

    N-1{ sort(i+1) acum(i) } acum(N) for j = 1 to N { sort(i) acum(i) }
  53. Architecture Overview DNN Accelerator SRAM (Weights, Feature Maps, Partial Sums,

    Masks) Path Costructor Sort & Merge Accumulate Controller SRAM (Code, Paths) DRAM Input/Output Weights Feature Maps Partial Sums Masks Gen Masks SRAM (Partial sums, Partial masks, Masks) Paths
  54. Enhanced MAC unit i w x + psum >? thd

    MUX 0/1 mode to SRAM to SRAM
  55. Evaluation Network AlexNet, ResNet Dataset Cifar10, Cifar100,ImageNet Attacks BIM, CWL2,

    DeepFool, FGSM,JSMA Adaptive Attacks Self constructed Baselines EP[1], CDRP[2] [1]Y. Qiu, J. Leng, C. Guo, et.al, Adversarial Defense Through Network Profiling Based Path Extraction
 [2]Y. Wang, H. Su, B. Zhang, X. Hu, Interpret neural networks by identifying critical data routing paths.
  56. Evaluation Backward Forward Sorting Thresholding Full Extraction Partial Extraction Type

    1 Type 2 Type 3
  57. Hardware Setup DNN Accelerator 20 x 20 Technology Silvaco 15nm

    On-chip SRAM 1.5MB
  58. Evaluation Accuracy 0 1 1 2 3 Hybrid EP CDRP

    AlexNet on ImageNet
  59. Evaluation Accuracy 0.84 0.88 0.92 0.96 1 1 2 3

    Hybrid EP CDRP AlexNet on ImageNet
  60. Evaluation Accuracy 0.84 0.88 0.92 0.96 1 1 2 3

    Hybrid EP CDRP AlexNet on ImageNet
  61. Evaluation Accuracy 0.84 0.88 0.92 0.96 1 1 2 3

    Hybrid EP CDRP AlexNet on ImageNet Accuracy decrease
  62. Evaluation Latency Overhead 0 4 8 12 16 BwCU BwAb

    FwAb Hybrid EP AlexNet on ImageNet Energy Overhead 0 2 4 6 8 BwCU BwAb FwAb Hybrid EP
  63. Evaluation Latency Overhead 0 4 8 12 16 BwCU BwAb

    FwAb Hybrid EP AlexNet on ImageNet Energy Overhead 0 2 4 6 8 BwCU BwAb FwAb Hybrid EP
  64. Evaluation Latency Overhead 0 4 8 12 16 BwCU BwAb

    FwAb Hybrid EP AlexNet on ImageNet Energy Overhead 0 2 4 6 8 BwCU BwAb FwAb Hybrid EP
  65. Latency Overhead 0 4 8 12 16 BwCU BwAb FwAb

    Hybrid EP Latency Overhead Decrease Evaluation AlexNet on ImageNet Energy Overhead 0 2 4 6 8 BwCU BwAb FwAb Hybrid EP Energy Overhead Decrease
  66. Conclusion Ptolemy: Accurate, low overhead, adversarial attack detection • Algorithm

    Framework • Compiler Optimization • Architecture Support
  67. Collaborators Yuxian Qiu Jingwen Leng Minyi Guo Yuhao Zhu

  68. Questions https://github.com/Ptolemy-dl/Ptolemy

  69. Evaluation Accuracy 0 1 8 7 6 5 4 3

    2 1 Termination Layer Latency Overhead 0 1 2 3 4 8 7 6 5 4 3 2 1 Termination Layer
  70. Evaluation Accuracy 0.84 0.91 8 7 6 5 4 3

    2 1 Termination Layer Latency Overhead 0 4 8 12 16 8 7 6 5 4 3 2 1 Termination Layer
  71. Backup def AdversaryDetection(model, input, θ, φ): output = Inference(model, input)

    N = model.num_layers // Selective extraction only in the last three layers for L in range(N-3, N): if L != N-1: // Forward extraction using absolute thresholds ImptN[L] = ExtractImptNeurons(1, 1, φ, L) else: // Forward extraction using cumulative thresholds ImptN[L] = ExtractImptNeurons(1, 0, θ, L) dynPath.concat(GenMask(ImptN[L])) classPath = LoadClassPath(argmax(output)) is_adversary = Classify(classPath, dynPath) return is_adversary 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
  72. Backup

  73. Backup

  74. Backup

  75. Backup