ACIVS 2015

Transcript

1. Introduction Proposed approach Experimental results Conclusion - ACIVS 2015 - A graph based people silhouette segmentation using combined probabilities extracted from appearance, shape template prior, and color distribution C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSYS, LEOST, F-59650 Villeneuve d'Ascq, France 2 Normandie Univ., UNICAEN, ENSICAEN, GREYC UMR CNRS 6072, Caen, France October 26-29, 2015 1/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSYS - ACIVS 2015 -
2. Introduction Proposed approach Experimental results Conclusion Plan 1 Introduction Context Problem with detection-oriented method 2 Proposed approach Purpose of the application Template shape prior based probabilities Appearance based probabilities Color distribution based probabilities Combination of the probability maps Graph-cut segmentation 3 Experimental results Databases Combination step : coefficient results Image results Score results 4 Conclusion 2/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSYS - ACIVS 2015 -
3. Introduction Proposed approach Experimental results Conclusion Context Problem with detection-oriented method Context In video surveillance application, people extraction is a well-know problem to efficiently perform task as : • Re-identification and tracking of people (optimize passenger flows) • Clothes segmentation and recognition (marketing) • Action recognition (security : falling, fight) Fact All these applications need precise people silhouette extraction 3/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSYS - ACIVS 2015 -
4. Introduction Proposed approach Experimental results Conclusion Context Problem with detection-oriented method Problem with detection-oriented method Case of video sequence  Traditionally people silhouette segmentation is performed with standard motion-based background substation strategies Case of image  Detection-oriented have emerged and make extensive use of machine learning Problem → Result of detection-oriented is a bounding box which does not contain precise information on the position silhouette Our goal → We propose a method designed to extract precisely people silhouettes in images Bounding Box obtained using techniques of person detection *Compare our proposed method with results obtained by C.Migniot (Experimental results) 4/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSYS - ACIVS 2015 -
5. Introduction Proposed approach Experimental results Conclusion Purpose of the application Template shape prior based probabilities Appearance based probabilities Color distribution based probabilities Combination of the probability maps Graph-cut segmentation Purpose of the application Idea Estimate a probability map ( from 6 methods ) → initialize a graph-cut segmentation Main contribution Developing a new method to estimate probability map Optimizing coefficients of combination with a genetic algorithm 5/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSYS - ACIVS 2015 -
6. Introduction Proposed approach Experimental results Conclusion Purpose of the application Template shape prior based probabilities Appearance based probabilities Color distribution based probabilities Combination of the probability maps Graph-cut segmentation Purpose of the application 6 probability map are estimate 2 probability map from methods using appearance of silhouette Standards technique based on template shape prior New method based on local small windows probability appearance 4 probability map from color distribution based method 6/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSYS - ACIVS 2015 -
7. Introduction Proposed approach Experimental results Conclusion Purpose of the application Template shape prior based probabilities Appearance based probabilities Color distribution based probabilities Combination of the probability maps Graph-cut segmentation Purpose of the application 6 probability map are estimate 2 probability map from methods using appearance of silhouette 4 probability map from color distribution based method Color histograms Gaussian Mixture 7/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSYS - ACIVS 2015 -
8. Introduction Proposed approach Experimental results Conclusion Purpose of the application Template shape prior based probabilities Appearance based probabilities Color distribution based probabilities Combination of the probability maps Graph-cut segmentation Template shape prior based probabilities Objective A method based on the people probability position on image Fact People in Bounding Box are centered on the person → people detection process based on machine learning are trained with positive images where people are centered Concept Using ground truth of training set to make a shape template prior by an average 8/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSYS - ACIVS 2015 -
9. Introduction Proposed approach Experimental results Conclusion Purpose of the application Template shape prior based probabilities Appearance based probabilities Color distribution based probabilities Combination of the probability maps Graph-cut segmentation Appearance based probabilities (1/3) Objective Adapt the method Histogram Of Gradient (HOG) + Support Vector Machine classifier (SVM) to produce a probability map on Bounding Box Concept Dividing image into small windows and extracting and classify HOG features Final probability map is obtained by an average of all windows results Each windows has its own learning 9/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSYS - ACIVS 2015 -
10. Introduction Proposed approach Experimental results Conclusion Purpose of the application Template shape prior based probabilities Appearance based probabilities Color distribution based probabilities Combination of the probability maps Graph-cut segmentation Appearance based probabilities (2/3) Define best configuration setting Size of local windows Size of blocks Size of cells Size of block overlapping HOG feature Number of bins HOG feature Normalization SVM Kernel Evaluation strategy Genetic algorithm optimized F-measure score is used as fitness 10/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSY - ACIVS 2015 -
11. Introduction Proposed approach Experimental results Conclusion Purpose of the application Template shape prior based probabilities Appearance based probabilities Color distribution based probabilities Combination of the probability maps Graph-cut segmentation Appearance based probabilities (3/3) Define best configuration setting Size of local windows Size of block Size of cell Size of block overlapping HOG feature Number of bins HOG feature Normalization SVM Kernel 11/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSY - ACIVS 2015 -
12. Introduction Proposed approach Experimental results Conclusion Purpose of the application Template shape prior based probabilities Appearance based probabilities Color distribution based probabilities Combination of the probability maps Graph-cut segmentation Color distribution based probabilities Objective Probability map from color distribution Using color histograms method Using gaussian mixture models method Initialized from two previous probability map 12/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSY - ACIVS 2015 -
13. Introduction Proposed approach Experimental results Conclusion Purpose of the application Template shape prior based probabilities Appearance based probabilities Color distribution based probabilities Combination of the probability maps Graph-cut segmentation Combination of the probability maps Objective Define the best combination between previous probability map → A genetic algorithm to define best coefficient 13/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSY - ACIVS 2015 -
14. Introduction Proposed approach Experimental results Conclusion Purpose of the application Template shape prior based probabilities Appearance based probabilities Color distribution based probabilities Combination of the probability maps Graph-cut segmentation Graph-cut segmentation Objective Segmentation of the image into two classes (silhouette and background) Graph-cut segmentation method → Initialized with the final probability map 14/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSY - ACIVS 2015 -
15. Introduction Proposed approach Experimental results Conclusion Databases Combination step : coefficient results Image results Score results Databases Databases Tested on INRIA Static Person Dataset and BOSS european project databases Training set and test set are clearly separate Training step include : Making of shape template prior Optimization and learning SVM classifier on appearance based probabilities Genetic optimization of combination probabilities maps 15/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSY - ACIVS 2015 -
16. Introduction Proposed approach Experimental results Conclusion Databases Combination step : coefficient results Image results Score results Combination step : coefficient results 16/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSY - ACIVS 2015 -
17. Introduction Proposed approach Experimental results Conclusion Databases Combination step : coefficient results Image results Score results Image results « Boss europeen project (on bord wireless secured video surveillance). » http ://www.multitel.be/image/research- development/research-projects/boss.php 17/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSY - ACIVS 2015 -
18. Introduction Proposed approach Experimental results Conclusion Databases Combination step : coefficient results Image results Score results Score results Rating F-measure score is used to rate segmentation on bounding box • Higher score is better ∼ Migniot, C., Bertolino, P., & Chassery, J. M. (2011, September). Automatic people segmentation with a template-driven graph cut. In Image Processing (ICIP), 2011 18th IEEE International Conference on (pp. 3149-3152). IEEE. ∼ Migniot, C., Bertolino, P., & Chassery, J. M. (2013, November). Iterative Human Segmentation from Detection Windows Using Contour Segment Analysis. In 9th International Conference on Computer Vision Theory and Applications (VISAPP 2013) (p. CD). ∼ Sharma, V., & Davis, J. W. (2007, October). Integrating appearance and motion cues for simultaneous detection and segmentation of pedestrians. In Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on (pp. 1-8). IEEE. 18/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSY - ACIVS 2015 -
19. Introduction Proposed approach Experimental results Conclusion Conclusion • Detection-oriented method does not give information on position of silhouette • We have proposed to use a graph based segmentation to extract silhouette of people from bounding box • We have proposed to combine several probability maps estimate from appearance and colors cues with a genetic optimization combination • We have adapted and optimize HOG+SVM classifier for local small windows • We have tested our proposed method on INRIA Static Person Dataset and BOSS European project databases and evaluate with F-Measure criteria = 0.860 Future works • Improving template driven method • Integrating a superpixel segmentation Re-identification application 19/19 C. Coniglio 1 , C. Meurie 1,O. Lezoray 2, M. Berbineau 1 1 Univ Lille Nord de France, F-59000 Lille, IFSTTAR, COSY - ACIVS 2015 -