Pro Yearly is on sale from $80 to $50! »

Reading Circle (Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation)

7e2aff680cf5ccd644764bf589dd57e2?s=47 pyman
July 29, 2020

Reading Circle (Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation)

Explanation of Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation [Wang+, CVPR20]

7e2aff680cf5ccd644764bf589dd57e2?s=128

pyman

July 29, 2020
Tweet

Transcript

  1. Reading Circle Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic

    Segmentation [Wang+, CVPR20]
  2. Semantic Segmentation Assign a semantic category to each pixel. 2

    Supervised Dataset : image & pixel-level class label ✗ huge annotation cost ↓ Weakly-supervised Dataset : image & image-level class label
  3. Weakly-Supervised SS Methods Three steps to train on the image-level

    label. 1. predict an initial category-wise response map to localize the object 2. refine the initial response as the pseudo GT 3. train the segmentation network based on pseudo labels 3 1 2 3
  4. Weakly-Supervised SS Methods Three steps to train on the image-level

    label. 1. predict an initial category-wise response map to localize the object 2. refine the initial response as the pseudo GT 3. train the segmentation network based on pseudo labels 4 1
  5. What’s New Introduce a self-supervised equivariant attention mechanism (SEAM). -

    Narrow the supervision gap between fully and weakly supervised semantic segmentation 5
  6. What’s New Introduce a self-supervised equivariant attention mechanism (SEAM). -

    Focus on affine transformation 6 Previous Proposed
  7. What’s New Introduce a self-supervised equivariant attention mechanism (SEAM). -

    Focus on affine transformation 7 Previous Proposed CAM varies depending on the size of the input image
  8. What’s New Introduce a self-supervised equivariant attention mechanism (SEAM). -

    Focus on affine transformation 8 Previous Proposed Consistent CAM regardless of the size of the input image
  9. Network Architecture of SEAM Use Siamese Network and three kinds

    of Losses. 9
  10. Network Architecture of SEAM Use Siamese Network and three kinds

    of Losses. 10
  11. Pixel Correlation Module (PCM) Modify CAM by self attention mechanism.

    11
  12. Pixel Correlation Module (PCM) Modify CAM by self attention mechanism.

    12 “Non-local neural networks” [Wang+, CVPR18]
  13. Self Attention [Wang+, CVPR18] Non-local mean operation x : input

    signal y : output signal g : representation function f : similarity function (scalar) 13 Gaussian Embedded Gaussian Dot product Concatenation
  14. Self Attention [Wang+, CVPR18] Non-local block 14

  15. Pixel Correlation Module (PCM) Modify CAM by self attention mechanism.

    15 “Non-local neural networks” [Wang+, CVPR18]
  16. Pixel Correlation Module (PCM) Modify CAM by self attention mechanism.

    16
  17. Network Architecture of SEAM Use Siamese Network and three kinds

    of Losses. 17
  18. Loss Design of SEAM 1. Class Loss : multi-label soft

    margin loss 18 original CAM
  19. Network Architecture of SEAM Use Siamese Network and three kinds

    of Losses. 19
  20. Loss Design of SEAM 2. Equivariant Regularization (ER) Loss 20

    Consistency between before and after affine transformation
  21. Network Architecture of SEAM Use Siamese Network and three kinds

    of Losses. 21
  22. Loss Design of SEAM 2. Equivariant Cross Regularization (ECR) Loss

    22 to further improve the ability of network for equivariance learning
  23. Network Architecture Use Siamese Network and three kinds of Losses.

    23
  24. Dataset PASCAL VOC 2012 semantic segmentation benchmark 21 categories one

    or multiple object class 1,464 images in training set 1,449 images in validation set 1,456 images in test set 24
  25. Result -CAM- 25 Proposed Baseline GT Original

  26. Result -Semantic Segmentation- 26 Input GT Ours

  27. Quantitative Comparison Evaluation of transformations on equivariant regularization - evaluation

    metric : ↑ mIoU (%) 27
  28. Quantitative Comparison Category performance comparison - evaluation metric : ↑

    mIoU (%) 28
  29. Quantitative Comparison Evaluation of WSSS performance 29 [Chang+, CVPR20] val

    : 66.1 test : 65.9
  30. Conclusion Weakly-supervised Learning 1. generate pseudo GT label 2. apply

    supervised learning Weakly-supervised Semantic Segmentation Self-supervised Equivariant Attention Mechanism ↓ appropriate CAM ↓ better pseudo GT label 30