Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Ptolemy: Architecture Support for Robust Deep Learning

HorizonLab
October 28, 2020

Ptolemy: Architecture Support for Robust Deep Learning

MICRO 2020 talk by Yiming Gan

HorizonLab

October 28, 2020
Tweet

More Decks by HorizonLab

Other Decks in Research

Transcript

  1. Ptolemy: Architecture Support
    for Robust Deep Learning
    Yiming Gan
    Department of Computer Science,
    University of Rochester
    with
    Yuxian Qiu, Shanghai Jiao Tong University
    Jingwen Leng, Shanghai Jiao Tong University
    Minyi Guo, Shanghai Jiao Tong University
    Yuhao Zhu University of Rochester
    https://github.com/Ptolemy-dl/Ptolemy

    View Slide

  2. Deep Learning: Not Robust

    View Slide

  3. Deep Learning: Not Robust
    Legitimate Example Adversarial Example
    Perturbation
    + =

    View Slide

  4. Mission Critical System
    ADAS Security Cameras

    View Slide

  5. Robust Deep Learning Requirements
    • Accurately detect adversarial examples

    View Slide

  6. +
    Robust Deep Learning Requirements
    • Accurately detect adversarial examples
    • Do not bring large overhead on system performance

    View Slide

  7. +
    Robust Deep Learning Requirements
    • Accurately detect adversarial examples
    • Do not bring large overhead on system performance
    =
    Ptolemy

    View Slide

  8. Hot Path
    Traditional
    Software
    [1]Thomas Ball, James R. Larus, Using Paths to Measure, Explain, and Enhance Program Behavior

    View Slide

  9. Hot Path
    Traditional
    Software
    [1]Thomas Ball, James R. Larus, Using Paths to Measure, Explain, and Enhance Program Behavior
    • Measure Program
    Behavior
    • Optimizing
    Program
    • Debugging

    View Slide

  10. Hot Path
    Hot Path
    Deep Learning
    Traditional
    Software
    Layer 1
    Layer 2
    Layer 3
    Layer 4

    View Slide

  11. 0.2 0.2 0.3
    0.3 0.2 0.4
    0.4 0.1 0.2
    -0.1 0.09 0.1
    -1.0 2.1 0.5
    Weights
    =
    0.06
    0.46
    0.44
    Output
    Feature Map
    Defining Important Neuron
    Input
    Feature Map
    x
    0.3 0.4 0.2 1.0 0.1

    View Slide

  12. 0.2 0.2 0.3
    0.3 0.2 0.4
    0.4 0.1 0.2
    -0.1 0.09 0.1
    -1.0 2.1 0.5
    Weights
    =
    0.06
    0.46
    0.44
    Output
    Feature Map
    Defining Important Neuron
    Input
    Feature Map
    x
    0.3 0.4 0.2 1.0 0.1

    View Slide

  13. 0.2 0.2 0.3
    0.3 0.2 0.4
    0.4 0.1 0.2
    -0.1 0.09 0.1
    -1.0 2.1 0.5
    Weights
    =
    0.06
    0.46
    0.44
    Output
    Feature Map
    Defining Important Neuron
    Input
    Feature Map
    x
    0.3 0.4 0.2 1.0 0.1

    View Slide

  14. 0.2 0.2 0.3
    0.3 0.2 0.4
    0.4 0.1 0.2
    -0.1 0.09 0.1
    -1.0 2.1 0.5
    Weights
    =
    0.06
    0.46
    0.44
    Output
    Feature Map
    Defining Important Neuron
    Input
    Feature Map
    x
    0.3 0.4 0.2 1.0 0.1

    View Slide

  15. Defining Important Neuron

    View Slide

  16. From Neuron to Path
    Input Layer
    Hidden Layer
    Output Layer

    View Slide

  17. From Neuron to Path
    Input Layer
    Hidden Layer
    Output Layer

    View Slide

  18. From Neuron to Path
    Input Layer
    Hidden Layer
    Output Layer

    View Slide

  19. Class Path

    View Slide

  20. Class Path
    } Union

    View Slide

  21. Class Path Similarity
    AlexNet @ ImageNet

    View Slide

  22. Class Path Similarity
    AlexNet @ ImageNet

    View Slide

  23. Ptolemy Overview

    View Slide

  24. Ptolemy Overview
    Neural
    Networks
    “Cat”

    View Slide

  25. Ptolemy Overview
    Neural
    Networks
    “Cat”
    Extract

    View Slide

  26. Ptolemy Overview
    Neural
    Networks
    “Cat”
    Extract
    Compare

    View Slide

  27. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N

    View Slide

  28. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Inference

    View Slide

  29. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Inference

    View Slide

  30. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Inference

    View Slide

  31. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N Inference

    View Slide

  32. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N Extraction

    View Slide

  33. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Extraction

    View Slide

  34. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Extraction

    View Slide

  35. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Extraction

    View Slide

  36. Ptolemy Pipeline
    Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Extraction
    IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det

    View Slide

  37. Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Inference
    Algorithmic Variation

    View Slide

  38. Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Inference
    Extraction
    Algorithmic Variation

    View Slide

  39. Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N
    Inference
    Extraction
    Algorithmic Variation

    View Slide

  40. Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N Inference
    Extraction
    Algorithmic Variation

    View Slide

  41. Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N Extraction
    Algorithmic Variation

    View Slide

  42. Layer 1
    Layer 2
    ……
    Layer N-1
    Layer N Extraction
    IF 1 IF 2 … IF N
    EX 1 EX 2 EX N Det
    Algorithmic Variation

    View Slide

  43. IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det
    Sorting
    IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det
    Threshold
    IF: Inference, EX: Extraction, Det: Detection
    Algorithmic Variation

    View Slide

  44. IF 1 IF 2 … IF N EX N EX N-1 … EX 1 Det
    Full Extraction
    IF 1 IF 2 … IF N EX N EX N-1 Det
    Partially Extraction
    IF: Inference, EX: Extraction, Det: Detection
    Algorithmic Variation

    View Slide

  45. Framework
    Backward
    Forward
    Sorting
    Thresholding
    Fully Extraction
    Partially Extraction

    View Slide

  46. Framework
    Backward
    Forward
    Sorting
    Full Extraction
    Partial Extraction
    = Backward + Fully Extraction + Sorting
    Thresholding

    View Slide

  47. Interface
    • High-level: Python-based, user define input

    View Slide

  48. Interface
    • High-level: Python-based, user define input
    Compiler
    • Low-level: Customized ISA

    View Slide

  49. Compiler Optimization: Layer Level
    for j = 1 to L {
    inf(j)

    }

    View Slide

  50. Compiler Optimization: Layer Level
    inf(1)
    for j = 1 to L {
    inf (j+1)

    }

    for j = 1 to L {
    inf(j)

    }

    View Slide

  51. Compiler Optimization: Neuron Level
    for j = 1 to N {
    sort(i)
    acum(i)
    }

    View Slide

  52. Compiler Optimization: Neuron Level
    sort(1)
    for i = 1 to N-1{
    sort(i+1)
    acum(i)
    }
    acum(N)
    for j = 1 to N {
    sort(i)
    acum(i)
    }

    View Slide

  53. Architecture Overview
    DNN Accelerator
    SRAM (Weights, Feature
    Maps, Partial Sums, Masks)
    Path Costructor
    Sort & Merge Accumulate
    Controller
    SRAM (Code,
    Paths)
    DRAM
    Input/Output
    Weights
    Feature Maps
    Partial Sums
    Masks
    Gen
    Masks
    SRAM (Partial sums, Partial masks, Masks)
    Paths

    View Slide

  54. Enhanced MAC unit
    i w
    x
    + psum
    >?
    thd
    MUX
    0/1
    mode
    to SRAM
    to SRAM

    View Slide

  55. Evaluation
    Network AlexNet, ResNet
    Dataset Cifar10, Cifar100,ImageNet
    Attacks BIM, CWL2, DeepFool,
    FGSM,JSMA
    Adaptive Attacks Self constructed
    Baselines EP[1], CDRP[2]
    [1]Y. Qiu, J. Leng, C. Guo, et.al, Adversarial Defense Through Network Profiling Based Path Extraction

    [2]Y. Wang, H. Su, B. Zhang, X. Hu, Interpret neural networks by identifying critical data routing paths.

    View Slide

  56. Evaluation
    Backward
    Forward
    Sorting
    Thresholding
    Full Extraction
    Partial Extraction
    Type 1
    Type 2
    Type 3

    View Slide

  57. Hardware Setup
    DNN Accelerator 20 x 20
    Technology Silvaco 15nm
    On-chip SRAM 1.5MB

    View Slide

  58. Evaluation
    Accuracy
    0
    1
    1 2 3 Hybrid EP CDRP
    AlexNet on ImageNet

    View Slide

  59. Evaluation
    Accuracy
    0.84
    0.88
    0.92
    0.96
    1
    1 2 3 Hybrid EP CDRP
    AlexNet on ImageNet

    View Slide

  60. Evaluation
    Accuracy
    0.84
    0.88
    0.92
    0.96
    1
    1 2 3 Hybrid EP CDRP
    AlexNet on ImageNet

    View Slide

  61. Evaluation
    Accuracy
    0.84
    0.88
    0.92
    0.96
    1
    1 2 3 Hybrid EP CDRP
    AlexNet on ImageNet
    Accuracy decrease

    View Slide

  62. Evaluation
    Latency
    Overhead
    0
    4
    8
    12
    16
    BwCU BwAb FwAb Hybrid EP
    AlexNet on ImageNet
    Energy
    Overhead
    0
    2
    4
    6
    8
    BwCU BwAb FwAb Hybrid EP

    View Slide

  63. Evaluation
    Latency
    Overhead
    0
    4
    8
    12
    16
    BwCU BwAb FwAb Hybrid EP
    AlexNet on ImageNet
    Energy
    Overhead
    0
    2
    4
    6
    8
    BwCU BwAb FwAb Hybrid EP

    View Slide

  64. Evaluation
    Latency
    Overhead
    0
    4
    8
    12
    16
    BwCU BwAb FwAb Hybrid EP
    AlexNet on ImageNet
    Energy
    Overhead
    0
    2
    4
    6
    8
    BwCU BwAb FwAb Hybrid EP

    View Slide

  65. Latency
    Overhead
    0
    4
    8
    12
    16
    BwCU BwAb FwAb Hybrid EP
    Latency Overhead
    Decrease
    Evaluation
    AlexNet on ImageNet
    Energy
    Overhead
    0
    2
    4
    6
    8
    BwCU BwAb FwAb Hybrid EP
    Energy Overhead
    Decrease

    View Slide

  66. Conclusion
    Ptolemy: Accurate, low overhead, adversarial attack
    detection
    • Algorithm Framework
    • Compiler Optimization
    • Architecture Support

    View Slide

  67. Collaborators
    Yuxian Qiu Jingwen Leng Minyi Guo Yuhao Zhu

    View Slide

  68. Questions
    https://github.com/Ptolemy-dl/Ptolemy

    View Slide

  69. Evaluation
    Accuracy
    0
    1
    8 7 6 5 4 3 2 1
    Termination Layer
    Latency Overhead
    0
    1
    2
    3
    4
    8 7 6 5 4 3 2 1
    Termination Layer

    View Slide

  70. Evaluation
    Accuracy
    0.84
    0.91
    8 7 6 5 4 3 2 1
    Termination Layer
    Latency Overhead
    0
    4
    8
    12
    16
    8 7 6 5 4 3 2 1
    Termination Layer

    View Slide

  71. Backup
    def AdversaryDetection(model, input, θ, φ):
    output = Inference(model, input)
    N = model.num_layers
    // Selective extraction only in the last three layers
    for L in range(N-3, N):
    if L != N-1:
    // Forward extraction using absolute thresholds
    ImptN[L] = ExtractImptNeurons(1, 1, φ, L)
    else:
    // Forward extraction using cumulative thresholds
    ImptN[L] = ExtractImptNeurons(1, 0, θ, L)
    dynPath.concat(GenMask(ImptN[L]))
    classPath = LoadClassPath(argmax(output))
    is_adversary = Classify(classPath, dynPath)
    return is_adversary
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15

    View Slide

  72. Backup

    View Slide

  73. Backup

    View Slide

  74. Backup

    View Slide

  75. Backup

    View Slide