IPTA 2020

Transcript

1. Instance segmentation in sheye images Rémi DUFOUR (PhD Student FCS Railenium), Cyril Meurie, Clément Strauss, Olivier Lézoray IPTA 2020
2. • Context • Related work • Our method : Fisheye Data augmentation • Experiments  Impact of MS COCO pretraining  Finding the right balance  Evaluation datasets  Results on the evaluation datasets • Conclusion 2 Outline
3. • Autonomous train prototype project directed by IRT Railenium • camera surveillance is needed to provide services, safety and security without sta+ • Wide angle or Fisheye cameras will be used (barrel distortion) • Objective : a method to adapt computer vision algorithms to deal with both rectilinear and sheye images Context 3
4. • Instance segmentation : Mask R-CNN  Derived from Faster R-CNN  Widely used reference for instance segmentation, state-of-the- art performance  Trained on MS COCO1, on rectilinear images 4 Related work K. He et al., Mask R-CNN, ICCV 2017 1Lin TY. et al., Microsoft COCO: Common Objects in Context, ECCV 2014
5. • Semantic segmentation of sheye images  Use a spherical projection model to apply a sheye e+ect (FE)  A set of 25 sheye transformations for data augmentation  Improves semantic segmentation performance on real custom sheye camera dataset. 5 Related work G. Blott et al., Semantic segmentation of fisheye images, ECCV 2018
6. • We propose to use the projection model of the “Semantic segmentation of sheye images” and modify it to keep a good performance on rectilinear images. 6 Our method: Fisheye Data augmentation
7. • We propose to use the projection model of the “Semantic segmentation of sheye images” and modify it to keep a good performance on rectilinear images. 7 Our method: Fisheye Data augmentation
8. • We propose to use the projection model of the “Semantic segmentation of sheye images” and modify it to keep a good performance on rectilinear images. 8 Our method: Fisheye Data augmentation
9. • We propose to use the projection model of the “Semantic segmentation of sheye images” and modify it to keep a good performance on rectilinear images. 9 Our method: Fisheye Data augmentation • We use Mask R-CNN as the detection algorithm.
10. 10 Our method: Fisheye Data augmentation
11. 11 Our method: Fisheye Data augmentation
12. • Impact of MS COCO pretraining  We compare 2 di+erent pretraining for Mask R-CNN, with a backbone pretrained on imagenet1, or with MS COCO pretraining.  We train and evaluate on arti cial sheye images.  Results demonstrate that the weights pretrained on MS COCO have good priors for dealing with sheye images. 12 Experiments 1O. Russakovsky et al. ImageNet Large Scale Visual Object Recognition Challenge, IJCV, 2015
13. • Impact of MS COCO pretraining  We compare 2 di+erent pretraining for Mask R-CNN, with a backbone pretrained on imagenet1, or with MS COCO pretraining.  We train and evaluate on arti cial sheye images.  Results demonstrate that the weights pretrained on MS COCO have good priors for dealing with sheye images. 13 Experiments Average Precision with 50%/75% threshold Average Precision Average Precision for small, medium and large objects 1O. Russakovsky et al. ImageNet Large Scale Visual Object Recognition Challenge, IJCV, 2015
14. • Impact of MS COCO pretraining  We compare 2 di+erent pretraining for Mask R-CNN, with a backbone pretrained on imagenet1, or with MS COCO pretraining.  We train and evaluate on arti cial sheye images.  Results demonstrate that the weights pretrained on MS COCO have good priors for dealing with sheye images. 14 Experiments Average Precision with 50%/75% threshold Average Precision Average Precision for small, medium and large objects 1O. Russakovsky et al. ImageNet Large Scale Visual Object Recognition Challenge, IJCV, 2015
15. • Finding the right balance  We compare training runs using 0%, 25%, 50%, 75% or 100% sheye augmentation ratio. We settle on 50% for the rest of the experiments. Experiments 15
16. • Custom Evaluation datasets  TrainDoor dataset: custom dataset feature scenes meant to resemble pedestrians walking through a door. 121 images annotated for human instance segmentation.  TrainDoorAug dataset : made by augmenting trainDoor with vertical 9ip. 242 images. Experiments 16
17. Experiments 17
18. Experiments 18
19. • Custom Evaluation datasets  ValBOSS dataset : 60 frames samples from BOSS dataset, annotated for human instance segmentation. Experiments 19
20. Experiments 20
21. Experiments 21
22. • Results Experiments Average Precision Average Precision with 50%/75% threshold Average Precision Average Precision for medium and large objects 22
23. • Results Experiments Average Precision Average Precision with 50%/75% threshold Average Precision Average Precision for medium and large objects 23
24. • Results Experiments Average Precision Average Precision with 50%/75% threshold Average Precision Average Precision for medium and large objects 24
25. • Results Experiments Average Precision Average Precision with 50%/75% threshold Average Precision Average Precision for medium and large objects 25
26. • Results Experiments Average Precision Average Precision with 50%/75% threshold Average Precision Average Precision for medium and large objects 26
27. • Results Experiments Average Precision Average Precision with 50%/75% threshold Average Precision Average Precision for medium and large objects 27
28. • Results Experiments Average Precision Average Precision with 50%/75% threshold Average Precision Average Precision for medium and large objects 28
29. • Results Experiments Average Precision Average Precision with 50%/75% threshold Average Precision Average Precision for medium and large objects 29
30. • Results Experiments Average Precision Average Precision with 50%/75% threshold Average Precision Average Precision for medium and large objects 30
31. • Results Experiments Average Precision Average Precision with 50%/75% threshold Average Precision Average Precision for medium and large objects 31
32. • Rectilinear pretraining is a good prior for dealing with sheye images. • Using a sheye augmentation method for 50% of training examples can result in good performance on both rectilinear and sheye images. • Using only 8 di+erent sheye transformations is enough to get the increased performance. • Not speci c to segmentation tasks. • Doesn’t require additional computation. • We plan to use this method for other algorithms and other tasks related to the safety of train passengers. Conclusion 32
33. Thank you for your attention 33