Upgrade to Pro — share decks privately, control downloads, hide ads and more …

IPTA 2020

IPTA 2020

1509d0ae6901a6cf2c2fd7cc97f02fc0?s=128

Olivier Lézoray

November 10, 2020
Tweet

Transcript

  1. Instance segmentation in sheye images Rémi DUFOUR (PhD Student FCS

    Railenium), Cyril Meurie, Clément Strauss, Olivier Lézoray IPTA 2020
  2. • Context • Related work • Our method : Fisheye

    Data augmentation • Experiments  Impact of MS COCO pretraining  Finding the right balance  Evaluation datasets  Results on the evaluation datasets • Conclusion 2 Outline
  3. • Autonomous train prototype project directed by IRT Railenium •

    camera surveillance is needed to provide services, safety and security without sta+ • Wide angle or Fisheye cameras will be used (barrel distortion) • Objective : a method to adapt computer vision algorithms to deal with both rectilinear and sheye images Context 3
  4. • Instance segmentation : Mask R-CNN  Derived from Faster

    R-CNN  Widely used reference for instance segmentation, state-of-the- art performance  Trained on MS COCO1, on rectilinear images 4 Related work K. He et al., Mask R-CNN, ICCV 2017 1Lin TY. et al., Microsoft COCO: Common Objects in Context, ECCV 2014
  5. • Semantic segmentation of sheye images  Use a spherical

    projection model to apply a sheye e+ect (FE)  A set of 25 sheye transformations for data augmentation  Improves semantic segmentation performance on real custom sheye camera dataset. 5 Related work G. Blott et al., Semantic segmentation of fisheye images, ECCV 2018
  6. • We propose to use the projection model of the

    “Semantic segmentation of sheye images” and modify it to keep a good performance on rectilinear images. 6 Our method: Fisheye Data augmentation
  7. • We propose to use the projection model of the

    “Semantic segmentation of sheye images” and modify it to keep a good performance on rectilinear images. 7 Our method: Fisheye Data augmentation
  8. • We propose to use the projection model of the

    “Semantic segmentation of sheye images” and modify it to keep a good performance on rectilinear images. 8 Our method: Fisheye Data augmentation
  9. • We propose to use the projection model of the

    “Semantic segmentation of sheye images” and modify it to keep a good performance on rectilinear images. 9 Our method: Fisheye Data augmentation • We use Mask R-CNN as the detection algorithm.
  10. 10 Our method: Fisheye Data augmentation

  11. 11 Our method: Fisheye Data augmentation

  12. • Impact of MS COCO pretraining  We compare 2

    di+erent pretraining for Mask R-CNN, with a backbone pretrained on imagenet1, or with MS COCO pretraining.  We train and evaluate on arti cial sheye images.  Results demonstrate that the weights pretrained on MS COCO have good priors for dealing with sheye images. 12 Experiments 1O. Russakovsky et al. ImageNet Large Scale Visual Object Recognition Challenge, IJCV, 2015
  13. • Impact of MS COCO pretraining  We compare 2

    di+erent pretraining for Mask R-CNN, with a backbone pretrained on imagenet1, or with MS COCO pretraining.  We train and evaluate on arti cial sheye images.  Results demonstrate that the weights pretrained on MS COCO have good priors for dealing with sheye images. 13 Experiments Average Precision with 50%/75% threshold Average Precision Average Precision for small, medium and large objects 1O. Russakovsky et al. ImageNet Large Scale Visual Object Recognition Challenge, IJCV, 2015
  14. • Impact of MS COCO pretraining  We compare 2

    di+erent pretraining for Mask R-CNN, with a backbone pretrained on imagenet1, or with MS COCO pretraining.  We train and evaluate on arti cial sheye images.  Results demonstrate that the weights pretrained on MS COCO have good priors for dealing with sheye images. 14 Experiments Average Precision with 50%/75% threshold Average Precision Average Precision for small, medium and large objects 1O. Russakovsky et al. ImageNet Large Scale Visual Object Recognition Challenge, IJCV, 2015
  15. • Finding the right balance  We compare training runs

    using 0%, 25%, 50%, 75% or 100% sheye augmentation ratio. We settle on 50% for the rest of the experiments. Experiments 15
  16. • Custom Evaluation datasets  TrainDoor dataset: custom dataset feature

    scenes meant to resemble pedestrians walking through a door. 121 images annotated for human instance segmentation.  TrainDoorAug dataset : made by augmenting trainDoor with vertical 9ip. 242 images. Experiments 16
  17. Experiments 17

  18. Experiments 18

  19. • Custom Evaluation datasets  ValBOSS dataset : 60 frames

    samples from BOSS dataset, annotated for human instance segmentation. Experiments 19
  20. Experiments 20

  21. Experiments 21

  22. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 22
  23. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 23
  24. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 24
  25. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 25
  26. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 26
  27. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 27
  28. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 28
  29. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 29
  30. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 30
  31. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 31
  32. • Rectilinear pretraining is a good prior for dealing with

    sheye images. • Using a sheye augmentation method for 50% of training examples can result in good performance on both rectilinear and sheye images. • Using only 8 di+erent sheye transformations is enough to get the increased performance. • Not speci c to segmentation tasks. • Doesn’t require additional computation. • We plan to use this method for other algorithms and other tasks related to the safety of train passengers. Conclusion 32
  33. Thank you for your attention 33