OpenScience 2017 - Deep Learning Feedback On Some Sentinel2 Cases

Magellium was present at ESRIN, Frascatti, Italy on monday 10/09 to share its point of view over some use case in segmentation for the sentinel 2 imagery.

This conference was a great opportunity to talk about upcoming challenges in data science, processing and distribution for many sentinel cases.

Our presentation can be watched here around 1:01:00.

Transcript

1. PUTTING KNOWLEDGE ON THE MAP Magellium
2. Domains of activities 2 20/09/2017 propriété Magellium 2003 creation 150+ employees 2 sites Paris Toulouse* 14,2 M€ CA 2016 Earth Observation Geo-information Image & Applications Joint ARTAL group 1st September 2016
3. Offer & Markets 3 propriété Magellium Scientific & technological studies Software development & Systems integration 30 % Energy & Transport 10 % Others 30 % Space Consulting & technical support 30 % Defense & Security 20/09/2017
4. Offer & Markets 4 propriété Magellium Scientific & technological studies Software development & Systems integration Consulting & technical support 20/09/2017 Deep Learning Internal R&D group Since 2015 4 experts Academic partneships
5. PUTTING KNOWLEDGE ON THE MAP Deep Learning Feedback on Some Sentinel-2 Cases 25/09/2017 5 12/01/2018 Property of Magellium
6. Deep Learning activities Opportunity/motivation Use cases Questions ? 6 Property of Magellium Building footprints Cloud detection S2 data 20/09/2017
7. PUTTING KNOWLEDGE ON THE MAP Sentinel Opportunity
8. Sentinel data • Many Open Data: – SAR, visible passive, … S1, S2, S3, S4, S5… • S2 case: – 13 bands – High revisit : 5 days =>temporal series ! • Opportunities for companies: – 10m GSD passive visible Object detection problems different from VHR (land cover) 8 Property of Magellium 12/01/2018
9. PUTTING KNOWLEDGE ON THE MAP Some Deep Learning Use Cases
10. PUTTING KNOWLEDGE ON THE MAP Building Foot Prints
11. Building footprints on VHR images • Why building footprints ? – Administrative collection – Urban growth monitoring – People density/location – Fusion with tax (e.g. pools declarations) 11 Property of Magellium 12/01/2018
12. Building footprints on VHR images • What is required for deep learning ? 12 Property of Magellium 12/01/2018 Data Model Ground truth Model output Metric Optimization Learning strategy
13. Building footprints on VHR images What is required for the learning step in deep learning ? – Data base for object detection problems: • Classification – 1 value for one image • Segmentation – 1 value per image pixel Ground truth is harder to build (precision) but fewer examples are need to train. 13 Property of Magellium 12/01/2018
14. Building footprints on VHR images • Data and Ground Truth – OSM and IGN Ocsge (land cover) – VHR images • Models: Segnet first, now Unet • Metric choices – Cross entropy + IoU, balance, reweighting ? • Learning strategy – Active Learning: add failed examples of previous model in next learning step 14 Property of Magellium 12/01/2018
15. Results 15 Property of Magellium 12/01/2018
16. Results on VHR images (Haiti/Jacmel/2010) 16 Property of Magellium 12/01/2018 @CNES KAL-HAITI project • Robust over various sensors (airborne & PHR)
17. Results on VHR images (Denver/2016) 17 Property of Magellium 12/01/2018 @GAIDDON SOFTWARE • Robust over environment/scenes
18. Building Footprints 18 Property of Magellium 12/01/2018 • FeedBack – Imperfections in ground truth are diluted when many examples are used but... – Need to evaluate the quality of the ground truth: • IGN Ocsge not precise enough for building detection – Design an iterative scheme for building your model • Base model • Add new cases – Segmentation is better than classification for preciseness
19. PUTTING KNOWLEDGE ON THE MAP Cloud Masks
20. Cloud detection on PHR • Magellium’s Experience: • Automatic Cloud Detection – CNES (since 2003) • Specification and development of an operational prototype for cloud cover estimation for Pleiades satellites using new correlation approaches between panchromatic and hyper-spectral bands. • Magellium has developed the A2NCNlib software library which includes a large collection of algorithms based on : – Artificial Intelligence (neural networks, SVM, fuzzy logic …), – Pattern recognition. 20 Property of Magellium 12/01/2018
21. Cloud detection on SPOT • Deep Learning Internal Study (2015): – Simple CNN (convolutional neural netwroks) using sliding windows – RGB album images – 4 classes: cloud, light haze, strong haze, other 21 Property of Magellium 12/01/2018 @SPOTIMAGE
22. Cloud masks for S2 • What exists ? – Masks from different sources : • L2A sen2corr masks • L2A Theia masks (MACCS/MAJA) • Tools from other satellite (Lib-toolbox for Landsat) 22 Property of Magellium 12/01/2018
23. Cloud masks for S2 • L2A (refl. products) sen2cor-2.3.0 Clear sky mask 23 Property of Magellium 12/01/2018
24. Cloud masks for S2 • L2A (refl. products) Theia (MACCS/MAJA): Computed at lower resolutions Cloud, cirrus, shadows 24 Property of Magellium 12/01/2018
25. Cloud masks for S2 Computed from Lib-toolbox: Matlab NN for cloud detection on Landsat MSS/TM and ETM+ Clear, thin, thick 25 Property of Magellium 12/01/2018
26. Cloud masks for S2 • What use ? – Image exploitability – « Best Image »: composition – Science: • Precise segmentation • Need every pixel for temporal series ! 26 Property of Magellium 12/01/2018
27. Cloud detection • Apply same recipes but – No input « by hand » – Rely on existing masks for Ground Truth 27 Property of Magellium 12/01/2018
28. Cloud Mask Results • Encouraging first results 28 Property of Magellium 12/01/2018
29. Cloud Masks • Main feedback: – Although imperfect, existing masks are enough to build first model before active learning steps – Hard examples need to be added with ground truth by hand: existing masks are not enough – Segmentation approach works 29 Property of Magellium 12/01/2018
30. Cloud Masks • Main feedback: – Although imperfect, existing masks are enough to build first model before active learning steps – Hard examples need to be added with ground truth by hand: existing masks are not enough – Segmentation approach works 30 Property of Magellium 12/01/2018
31. Sentinel-2 Land Cover • Ground truth: – OCSGE 2012(IGN free product) • Results: – Visual validation satisfactory BUT – Validation DB necessary to go further 31 12/01/2018 Property of Magellium c) a) b) d)
32. PUTTING KNOWLEDGE ON THE MAP THANKS ! Thomas RISTORCELLI IA Unit thomas.ristorcelli@magellium.fr François De Vieilleville EO Unit françois.devieilleville@magellium.fr Sébastien Bosch GEO Unit sebastien.bosch@magellium.fr