IPTA 2020

Transcript

1. Instance segmentation in sheye images Rémi DUFOUR (PhD Student FCS

   Railenium), Cyril Meurie, Clément Strauss, Olivier Lézoray IPTA 2020

2. • Context • Related work • Our method : Fisheye

   Data augmentation • Experiments  Impact of MS COCO pretraining  Finding the right balance  Evaluation datasets  Results on the evaluation datasets • Conclusion 2 Outline

3. • Autonomous train prototype project directed by IRT Railenium •

   camera surveillance is needed to provide services, safety and security without sta+ • Wide angle or Fisheye cameras will be used (barrel distortion) • Objective : a method to adapt computer vision algorithms to deal with both rectilinear and sheye images Context 3

4. • Instance segmentation : Mask R-CNN  Derived from Faster

   R-CNN  Widely used reference for instance segmentation, state-of-the- art performance  Trained on MS COCO1, on rectilinear images 4 Related work K. He et al., Mask R-CNN, ICCV 2017 1Lin TY. et al., Microsoft COCO: Common Objects in Context, ECCV 2014

5. • Semantic segmentation of sheye images  Use a spherical

   projection model to apply a sheye e+ect (FE)  A set of 25 sheye transformations for data augmentation  Improves semantic segmentation performance on real custom sheye camera dataset. 5 Related work G. Blott et al., Semantic segmentation of fisheye images, ECCV 2018

6. • We propose to use the projection model of the

   “Semantic segmentation of sheye images” and modify it to keep a good performance on rectilinear images. 6 Our method: Fisheye Data augmentation

7. • We propose to use the projection model of the

   “Semantic segmentation of sheye images” and modify it to keep a good performance on rectilinear images. 7 Our method: Fisheye Data augmentation

8. • We propose to use the projection model of the

   “Semantic segmentation of sheye images” and modify it to keep a good performance on rectilinear images. 8 Our method: Fisheye Data augmentation

9. • We propose to use the projection model of the

   “Semantic segmentation of sheye images” and modify it to keep a good performance on rectilinear images. 9 Our method: Fisheye Data augmentation • We use Mask R-CNN as the detection algorithm.

10. 10 Our method: Fisheye Data augmentation

11. 11 Our method: Fisheye Data augmentation

12. • Impact of MS COCO pretraining  We compare 2

    di+erent pretraining for Mask R-CNN, with a backbone pretrained on imagenet1, or with MS COCO pretraining.  We train and evaluate on arti cial sheye images.  Results demonstrate that the weights pretrained on MS COCO have good priors for dealing with sheye images. 12 Experiments 1O. Russakovsky et al. ImageNet Large Scale Visual Object Recognition Challenge, IJCV, 2015

13. • Impact of MS COCO pretraining  We compare 2

    di+erent pretraining for Mask R-CNN, with a backbone pretrained on imagenet1, or with MS COCO pretraining.  We train and evaluate on arti cial sheye images.  Results demonstrate that the weights pretrained on MS COCO have good priors for dealing with sheye images. 13 Experiments Average Precision with 50%/75% threshold Average Precision Average Precision for small, medium and large objects 1O. Russakovsky et al. ImageNet Large Scale Visual Object Recognition Challenge, IJCV, 2015

14. • Impact of MS COCO pretraining  We compare 2

    di+erent pretraining for Mask R-CNN, with a backbone pretrained on imagenet1, or with MS COCO pretraining.  We train and evaluate on arti cial sheye images.  Results demonstrate that the weights pretrained on MS COCO have good priors for dealing with sheye images. 14 Experiments Average Precision with 50%/75% threshold Average Precision Average Precision for small, medium and large objects 1O. Russakovsky et al. ImageNet Large Scale Visual Object Recognition Challenge, IJCV, 2015

15. • Finding the right balance  We compare training runs

    using 0%, 25%, 50%, 75% or 100% sheye augmentation ratio. We settle on 50% for the rest of the experiments. Experiments 15

16. • Custom Evaluation datasets  TrainDoor dataset: custom dataset feature

    scenes meant to resemble pedestrians walking through a door. 121 images annotated for human instance segmentation.  TrainDoorAug dataset : made by augmenting trainDoor with vertical 9ip. 242 images. Experiments 16

17. Experiments 17

18. Experiments 18

19. • Custom Evaluation datasets  ValBOSS dataset : 60 frames

    samples from BOSS dataset, annotated for human instance segmentation. Experiments 19

20. Experiments 20

21. Experiments 21

22. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 22

23. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 23

24. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 24

25. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 25

26. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 26

27. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 27

28. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 28

29. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 29

30. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 30

31. • Results Experiments Average Precision Average Precision with 50%/75% threshold

    Average Precision Average Precision for medium and large objects 31

32. • Rectilinear pretraining is a good prior for dealing with

    sheye images. • Using a sheye augmentation method for 50% of training examples can result in good performance on both rectilinear and sheye images. • Using only 8 di+erent sheye transformations is enough to get the increased performance. • Not speci c to segmentation tasks. • Doesn’t require additional computation. • We plan to use this method for other algorithms and other tasks related to the safety of train passengers. Conclusion 32

33. Thank you for your attention 33