Pro Yearly is on sale from $80 to $50! »

Reading Circle (Weakly-Supervised Semantic Segmentation via Sub-category Exploration)

7e2aff680cf5ccd644764bf589dd57e2?s=47 pyman
July 01, 2020

Reading Circle (Weakly-Supervised Semantic Segmentation via Sub-category Exploration)

Explanation of Weakly-Supervised Semantic Segmentation
via Sub-category Exploration [Chang+, CVPR20]

7e2aff680cf5ccd644764bf589dd57e2?s=128

pyman

July 01, 2020
Tweet

Transcript

  1. Reading Circle Weakly-Supervised Semantic Segmentation via Sub-category Exploration [Chang+, CVPR20]

  2. Semantic Segmentation Assign a semantic category to each pixel. 2

    Supervised Dataset : image & pixel-level class label ✗ huge annotation cost ↓ Weakly-supervised Dataset : image & image-level class label
  3. Weakly-Supervised SS Methods Three steps to train on the image-level

    label. 1. predict an initial category-wise response map to localize the object 2. refine the initial response as the pseudo GT 3. train the segmentation network based on pseudo labels 3 1 2 3
  4. Weakly-Supervised SS Methods Three steps to train on the image-level

    label. 1. predict an initial category-wise response map to localize the object 2. refine the initial response as the pseudo GT 3. train the segmentation network based on pseudo labels 4 AffinityNet [Ahn+, CVPR18] CRF [Krahenbuhl+, NeurIPS11] 1 2 3
  5. Weakly-Supervised SS Methods AffinityNet [Ahn+, CVPR18] 5

  6. Weakly-Supervised SS Methods AffinityNet [Ahn+, CVPR18] 6 generate affinity labels

    from CAM train to predict affinity matrix
  7. Weakly-Supervised SS Methods AffinityNet [Ahn+, CVPR18] 7 generate segmentation labels

    by random walk from affinity matrix
  8. Weakly-Supervised SS Methods AffinityNet [Ahn+, CVPR18] 8 train on pseudo

    segmentation labels
  9. Weakly-Supervised SS Methods Three steps to train on the image-level

    label. 1. predict an initial category-wise response map to localize the object 2. refine the initial response as the pseudo GT 3. train the segmentation network based on pseudo labels 9 Deeplab-v2 [Chen+, CoRR16] 1 2 3
  10. Weakly-Supervised SS Methods Three steps to train on the image-level

    label. 1. predict an initial category-wise response map to localize the object 2. refine the initial response as the pseudo GT 3. train the segmentation network based on pseudo labels 10 1
  11. What’s New Introduce self-supervised sub-category exploration. 11 Previous Proposed

  12. What’s New Introduce self-supervised sub-category exploration. 12 Previous Proposed only

    focus on a portion of the object
  13. What’s New Introduce self-supervised sub-category exploration. 13 Previous Proposed focus

    on entire object
  14. Method Introduce sub-category exploration. 14

  15. Method Introduce sub-category exploration. 15 optimize feature extractor E and

    extract image feature f
  16. Method Introduce sub-category exploration. 16 K-means clustering for each parent

    class
  17. Method Introduce sub-category exploration. 17 obtain sub-category pseudo labels

  18. Method Introduce sub-category exploration. 18 jointly optimize parent classifier and

    sub-category classifier → multi-label classification loss
  19. Method Introduce sub-category exploration. 19 re-train feature extracter E

  20. Dataset PASCAL VOC 2012 semantic segmentation benchmark 21 categories one

    or multiple object class 10,528 images in training set 1,449 images in validation set 1,456 images in test set 20
  21. Result -CAM- 21 CAM is widely covered

  22. Result -Clustering- 22 the composition is similar

  23. Result -Visualization- 23 t-SNE method [Maaten+, MLR08] Parent and sub-category

    classes are placed close together
  24. Result -Semantic Segmentation- 24 Input GT Ours

  25. Quantitative Comparison Evaluation of activation maps - evaluation metric :

    ↑ mIoU (%) 25
  26. Effect of Sub-Category Number K 26 original CAM (K=1)

  27. Quantitative Comparison Evaluation of semantic segmentation performance - evaluation metric

    : ↑ mIoU (%) 27 red > green > blue
  28. Quantitative Comparison Evaluation of weakly-supervised semantic segmentation performance 28

  29. Conclusion Weakly-supervised Learning 1. generate pseudo GT label 2. apply

    supervised learning Weakly-supervised Semantic Segmentation sub-category exploration ↓ widely covered CAM ↓ better pseudo GT label 29