Ptolemy: Architecture Support for Robust Deep Learning

MICRO 2020 talk by Yiming Gan

Transcript

1. Ptolemy: Architecture Support
   for Robust Deep Learning
   Yiming Gan
   Department of Computer Science,
   University of Rochester
   with
   Yuxian Qiu, Shanghai Jiao Tong University
   Jingwen Leng, Shanghai Jiao Tong University
   Minyi Guo, Shanghai Jiao Tong University
   Yuhao Zhu University of Rochester
   https://github.com/Ptolemy-dl/Ptolemy
   

   View Slide

2. Deep Learning: Not Robust
   

   View Slide

3. Deep Learning: Not Robust
   Legitimate Example Adversarial Example
   Perturbation
   + =
   

   View Slide

4. Mission Critical System
   ADAS Security Cameras
   

   View Slide

5. Robust Deep Learning Requirements
   • Accurately detect adversarial examples
   

   View Slide

6. +
   Robust Deep Learning Requirements
   • Accurately detect adversarial examples
   • Do not bring large overhead on system performance
   

   View Slide

7. +
   Robust Deep Learning Requirements
   • Accurately detect adversarial examples
   • Do not bring large overhead on system performance
   =
   Ptolemy
   

   View Slide

8. Hot Path
   Traditional
   Software
   [1]Thomas Ball, James R. Larus, Using Paths to Measure, Explain, and Enhance Program Behavior
   

   View Slide

9. Hot Path
   Traditional
   Software
   [1]Thomas Ball, James R. Larus, Using Paths to Measure, Explain, and Enhance Program Behavior
   • Measure Program
   Behavior
   • Optimizing
   Program
   • Debugging
   

   View Slide

10. Hot Path
    Hot Path
    Deep Learning
    Traditional
    Software
    Layer 1
    Layer 2
    Layer 3
    Layer 4
    

    View Slide

11. 0.2 0.2 0.3
    0.3 0.2 0.4
    0.4 0.1 0.2
    -0.1 0.09 0.1
    -1.0 2.1 0.5
    Weights
    =
    0.06
    0.46
    0.44
    Output
    Feature Map
    Defining Important Neuron
    Input
    Feature Map
    x
    0.3 0.4 0.2 1.0 0.1
    

    View Slide

12. 0.2 0.2 0.3
    0.3 0.2 0.4
    0.4 0.1 0.2
    -0.1 0.09 0.1
    -1.0 2.1 0.5
    Weights
    =
    0.06
    0.46
    0.44
    Output
    Feature Map
    Defining Important Neuron
    Input
    Feature Map
    x
    0.3 0.4 0.2 1.0 0.1
    

    View Slide

13. 0.2 0.2 0.3
    0.3 0.2 0.4
    0.4 0.1 0.2
    -0.1 0.09 0.1
    -1.0 2.1 0.5
    Weights
    =
    0.06
    0.46
    0.44
    Output
    Feature Map
    Defining Important Neuron
    Input
    Feature Map
    x
    0.3 0.4 0.2 1.0 0.1
    

    View Slide

14. 0.2 0.2 0.3
    0.3 0.2 0.4
    0.4 0.1 0.2
    -0.1 0.09 0.1
    -1.0 2.1 0.5
    Weights
    =
    0.06
    0.46
    0.44
    Output
    Feature Map
    Defining Important Neuron
    Input
    Feature Map
    x
    0.3 0.4 0.2 1.0 0.1
    

    View Slide

15. Defining Important Neuron
    

    View Slide

16. From Neuron to Path
    Input Layer
    Hidden Layer
    Output Layer
    

    View Slide

17. From Neuron to Path
    Input Layer
    Hidden Layer
    Output Layer
    

    View Slide

18. From Neuron to Path
    Input Layer
    Hidden Layer
    Output Layer
    

    View Slide

19. Class Path
    

    View Slide

20. Class Path
    } Union
    

    View Slide

21. Class Path Similarity
    AlexNet @ ImageNet
    

    View Slide

22. Class Path Similarity
    AlexNet @ ImageNet
    

    View Slide

23. Ptolemy Overview
    

    View Slide

24. Ptolemy Overview
    Neural
    Networks
    “Cat”
    

    View Slide

25. Ptolemy Overview
    Neural
    Networks
    “Cat”
    Extract
    

    View Slide

26. Ptolemy Overview
    Neural
    Networks
    “Cat”
    Extract
    Compare
    

    View Slide

27. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    

    View Slide

28. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Inference
    

    View Slide

29. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Inference
    

    View Slide

30. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Inference
    

    View Slide

31. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N Inference
    

    View Slide

32. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N Extraction
    

    View Slide

33. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Extraction
    

    View Slide

34. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Extraction
    

    View Slide

35. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Extraction
    

    View Slide

36. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Extraction
    IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det
    

    View Slide

37. Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Inference
    Algorithmic Variation
    

    View Slide

38. Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Inference
    Extraction
    Algorithmic Variation
    

    View Slide

39. Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Inference
    Extraction
    Algorithmic Variation
    

    View Slide

40. Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N Inference
    Extraction
    Algorithmic Variation
    

    View Slide

41. Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N Extraction
    Algorithmic Variation
    

    View Slide

42. Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N Extraction
    IF 1 IF 2 … IF N
    EX 1 EX 2 EX N Det
    Algorithmic Variation
    

    View Slide

43. IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det
    Sorting
    IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det
    Threshold
    IF: Inference, EX: Extraction, Det: Detection
    Algorithmic Variation
    

    View Slide

44. IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det
    Full Extraction
    IF 1 IF 2 … IF N EX N EX N-1 Det
    Partially Extraction
    IF: Inference, EX: Extraction, Det: Detection
    Algorithmic Variation
    

    View Slide

45. Framework
    Backward
    Forward
    Sorting
    Thresholding
    Fully Extraction
    Partially Extraction
    

    View Slide

46. Framework
    Backward
    Forward
    Sorting
    Full Extraction
    Partial Extraction
    = Backward + Fully Extraction + Sorting
    Thresholding
    

    View Slide

47. Interface
    • High-level: Python-based, user define input
    

    View Slide

48. Interface
    • High-level: Python-based, user define input
    Compiler
    • Low-level: Customized ISA
    

    View Slide

49. Compiler Optimization: Layer Level
    for j = 1 to L {
    inf(j)
    
    }
    

    View Slide

50. Compiler Optimization: Layer Level
    inf(1)
    for j = 1 to L {
    inf (j+1)
    
    }
    
    for j = 1 to L {
    inf(j)
    
    }
    

    View Slide

51. Compiler Optimization: Neuron Level
    for j = 1 to N {
    sort(i)
    acum(i)
    }
    

    View Slide

52. Compiler Optimization: Neuron Level
    sort(1)
    for i = 1 to N-1{
    sort(i+1)
    acum(i)
    }
    acum(N)
    for j = 1 to N {
    sort(i)
    acum(i)
    }
    

    View Slide

53. Architecture Overview
    DNN Accelerator
    SRAM (Weights, Feature
    Maps, Partial Sums, Masks)
    Path Costructor
    Sort & Merge Accumulate
    Controller
    SRAM (Code,
    Paths)
    DRAM
    Input/Output
    Weights
    Feature Maps
    Partial Sums
    Masks
    Gen
    Masks
    SRAM (Partial sums, Partial masks, Masks)
    Paths
    

    View Slide

54. Enhanced MAC unit
    i w
    x
    + psum
    >?
    thd
    MUX
    0/1
    mode
    to SRAM
    to SRAM
    

    View Slide

55. Evaluation
    Network AlexNet, ResNet
    Dataset Cifar10, Cifar100,ImageNet
    Attacks BIM, CWL2, DeepFool,
    FGSM,JSMA
    Adaptive Attacks Self constructed
    Baselines EP[1], CDRP[2]
    [1]Y. Qiu, J. Leng, C. Guo, et.al, Adversarial Defense Through Network Profiling Based Path Extraction

    [2]Y. Wang, H. Su, B. Zhang, X. Hu, Interpret neural networks by identifying critical data routing paths.
    

    View Slide

56. Evaluation
    Backward
    Forward
    Sorting
    Thresholding
    Full Extraction
    Partial Extraction
    Type 1
    Type 2
    Type 3
    

    View Slide

57. Hardware Setup
    DNN Accelerator 20 x 20
    Technology Silvaco 15nm
    On-chip SRAM 1.5MB
    

    View Slide

58. Evaluation
    Accuracy
    0
    1
    1 2 3 Hybrid EP CDRP
    AlexNet on ImageNet
    

    View Slide

59. Evaluation
    Accuracy
    0.84
    0.88
    0.92
    0.96
    1
    1 2 3 Hybrid EP CDRP
    AlexNet on ImageNet
    

    View Slide

60. Evaluation
    Accuracy
    0.84
    0.88
    0.92
    0.96
    1
    1 2 3 Hybrid EP CDRP
    AlexNet on ImageNet
    

    View Slide

61. Evaluation
    Accuracy
    0.84
    0.88
    0.92
    0.96
    1
    1 2 3 Hybrid EP CDRP
    AlexNet on ImageNet
    Accuracy decrease
    

    View Slide

62. Evaluation
    Latency
    Overhead
    0
    4
    8
    12
    16
    BwCU BwAb FwAb Hybrid EP
    AlexNet on ImageNet
    Energy
    Overhead
    0
    2
    4
    6
    8
    BwCU BwAb FwAb Hybrid EP
    

    View Slide

63. Evaluation
    Latency
    Overhead
    0
    4
    8
    12
    16
    BwCU BwAb FwAb Hybrid EP
    AlexNet on ImageNet
    Energy
    Overhead
    0
    2
    4
    6
    8
    BwCU BwAb FwAb Hybrid EP
    

    View Slide

64. Evaluation
    Latency
    Overhead
    0
    4
    8
    12
    16
    BwCU BwAb FwAb Hybrid EP
    AlexNet on ImageNet
    Energy
    Overhead
    0
    2
    4
    6
    8
    BwCU BwAb FwAb Hybrid EP
    

    View Slide

65. Latency
    Overhead
    0
    4
    8
    12
    16
    BwCU BwAb FwAb Hybrid EP
    Latency Overhead
    Decrease
    Evaluation
    AlexNet on ImageNet
    Energy
    Overhead
    0
    2
    4
    6
    8
    BwCU BwAb FwAb Hybrid EP
    Energy Overhead
    Decrease
    

    View Slide

66. Conclusion
    Ptolemy: Accurate, low overhead, adversarial attack
    detection
    • Algorithm Framework
    • Compiler Optimization
    • Architecture Support
    

    View Slide

67. Collaborators
    Yuxian Qiu Jingwen Leng Minyi Guo Yuhao Zhu
    

    View Slide

68. Questions
    https://github.com/Ptolemy-dl/Ptolemy
    

    View Slide

69. Evaluation
    Accuracy
    0
    1
    8 7 6 5 4 3 2 1
    Termination Layer
    Latency Overhead
    0
    1
    2
    3
    4
    8 7 6 5 4 3 2 1
    Termination Layer
    

    View Slide

70. Evaluation
    Accuracy
    0.84
    0.91
    8 7 6 5 4 3 2 1
    Termination Layer
    Latency Overhead
    0
    4
    8
    12
    16
    8 7 6 5 4 3 2 1
    Termination Layer
    

    View Slide

71. Backup
    def AdversaryDetection(model, input, θ, φ):
    output = Inference(model, input)
    N = model.num_layers
    // Selective extraction only in the last three layers
    for L in range(N-3, N):
    if L != N-1:
    // Forward extraction using absolute thresholds
    ImptN[L] = ExtractImptNeurons(1, 1, φ, L)
    else:
    // Forward extraction using cumulative thresholds
    ImptN[L] = ExtractImptNeurons(1, 0, θ, L)
    dynPath.concat(GenMask(ImptN[L]))
    classPath = LoadClassPath(argmax(output))
    is_adversary = Classify(classPath, dynPath)
    return is_adversary
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    

    View Slide

72. Backup
    

    View Slide

73. Backup
    

    View Slide

74. Backup
    

    View Slide

75. Backup
    

    View Slide