Cloud Computing & Web Processing Services

Toulouse, CNES - 2013.10.01
RTSTechniques de l'Information et de la Communication

Synthesis of R&T projects on Cloud Computing and WPS - i.e. OpenStack cloud based solution for processing SRTM data followed by Land Cover classification through WPS

Transcript

1. RTS Techniques de l'Information et de la Communication CNES - Toulouse, France - October 1st, 2013 https://speakerdeck.com/jjrom/cloud-computing-and-web-processing-services Cloud Computing & Web Processing Services Jerome.Gasperi@cnes.fr Jérôme Gasperi
2. Cloud Computing Introduction What we have done Issues Web Processing Services Introduction What we have done Issues What's next ? WPS on the cloud
3. R&T Cloud Computing (2011) Use a cloud infrastructure to process Earth Observation data
4. ?
5. host build consume IAAS PAAS SAAS
6. host build consume IAAS PAAS SAAS
7. Private or public ?
8. process 1 € store 5 € access 0.5 € to + 1 time per year each time (Very) approximative computations for a 20x20 km PA+XS Pleiades product (4 Go) - based on Amazon EC2 and S3 pricing (2012)
9. process 1 € store 5 € access 0.5 € to + 1 time per year each time (Very) approximative computations for a 20x20 km PA+XS Pleiades product (4 Go) - based on Amazon EC2 and S3 pricing (2012)
10. Data security and user privacy cannot be guaranteed in public clouds Majority of cloud providers are subject to governmental law (e.g. US patriot act)
11. ...so we choose a private solution
12. Web browser What we have done eNovance (Paris)
13. Web browser What we have done eNovance (Paris) SRTM tile Data
14. Web browser Processing (Orfeo Toolbox) What we have done eNovance (Paris) SRTM tile Data
15. Hill Shading Web browser Processing (Orfeo Toolbox) What we have done eNovance (Paris) SRTM tile Data
16. Hill Shading Projecting Web browser Processing (Orfeo Toolbox) What we have done eNovance (Paris) SRTM tile Data
17. Hill Shading Projecting Web browser Processing (Orfeo Toolbox) What we have done VISUALIZE WMS Server Tiling eNovance (Paris) SRTM tile Data
18. Conclusions
19. Private cloud technologies are quite simple to implement. They reduce the cost of operation and maintenance by sharing a common infrastructure across multiple projects
20. However...
21. Transfering large volumes of data is problematic However...
22. Transfering large volumes of data is problematic This is true for upload and download However...
23. ...so
24. Data and processes should be colocated ...so
25. Data and processes should be colocated Processing results should be downloaded only if needed ...so
26. Data and processes should be colocated Processing results should be downloaded only if needed Better to offer a Web service to visualize/manipulate results ...so
27. Data and processes should be colocated Processing results should be downloaded only if needed Better to offer a Web service to visualize/manipulate results Standardize processes inputs/outputs description (e.g. WPS) ...so
28. Web Processing Service (2012-2013) Set WPS on top of Orfeo Toolbox processing chains
29. Web Processing Services (WPS) is an OGC standard which defines interfaces to publish, describe and execute geospatial processes
30. WPS a Processing Web application
31. From pixel... Remote sensing image What we have done
32. From pixel... Remote sensing image ...to information Land cover thematic map What we have done
33. mapshup Web application http://mapshup.info WPS Constellation http://www.constellation-sdi.org Orfeo Toolbox Processing http://orfeo-toolbox.org
34. Orfeo Toolbox Remote sensing image library Open Source Developped by the French Space Agency Processing
35. Orfeo Toolbox More than 70 high level processing chains orthorectification segmentation classification etc. Processing
36. Orfeo Toolbox More than 70 high level processing chains orthorectification segmentation classification etc. Processing Supervised learning (land cover is computed from a set of "well known areas" given by user) Based on SVM (http://en.wikipedia.org/wiki/Support_vector_machine)
37. Constellation WPS Geospatial Networking Services OGC/ISO implementations Multi platform (JAVA) OpenSource WMS WFS CSW WPS
38. mapshup Improves access to geospatial data HTML + javascript Based on OpenLayers and jQuery Web application
39. mapshup Supported data sources WMS WFS CSW WPS OpenSearch etc. Flickr Youtube GeoRSS KML Wikipedia Google maps Bing maps OpenStreetMap MapBox WMTS Web application
40. mapshup Supported data sources See WPS demo https://vimeo.com/67288678 WMS WFS CSW WPS OpenSearch etc. Flickr Youtube GeoRSS KML Wikipedia Google maps Bing maps OpenStreetMap MapBox WMTS Web application
41. https://vimeo.com/67288677 What have we done ?
42. Conclusions
43. WPS 1.0 lacks a true asynchronous process management... There is no way to cancel a launched process !
44. However...
45. WPS 2.0 defines a set of process management operations - GetStatus, Delete, Pause and Resume However...
46. WPS 2.0 defines a set of process management operations - GetStatus, Delete, Pause and Resume This is a must have to deploy asynchronous WPS on the cloud However...
47. WPS 2.0 defines a set of process management operations - GetStatus, Delete, Pause and Resume This is a must have to deploy asynchronous WPS on the cloud Should be an official OGC standard by the end of 2013 However...
48. What's next WPS on the cloud - OGC OWS-10 Testbeb
49. Orthorectifying images hosted by a cloud infrastructure using WPS standard In kind contribution to the «Open Mobility» thread of the OGC OWS-10 Testbeb Final delivery and demonstration April/May 2014
50. Raw images Processing Data (Pléiades samples) Client Interoute (location : UK) Services PROCESS WPS Server Orthorectify (OTB) VISUALIZE WMS Server SEARCH CSW Server Metadata + 1 2 3 Assess Quality Improve Quality Services Processing Astrium GEO Result Quality WPS Server 4 VISUALIZE WMS Server
51. Landsat Landsat Landsat 2. Click on '+' to process a new orthorectification. Process parameters are set by user and sent to an asynchronous WPS orthorectification process located on the 3. Result is displayed within the map trough a WMS Landsat 4. Orthorectified image quality can be checked through "Assess Quality" WPS process located on the Result is displayed within the map as a WMS quality layer stored on the 1. Select raw image to orthorectify within CSW catalog located on the Laat
52. RTS Techniques de l'Information et de la Communication CNES - Toulouse, France - October 1st, 2013 https://speakerdeck.com/jjrom/cloud-computing-and-web-processing-services Cloud Computing & Web Processing Services Jerome.Gasperi@cnes.fr Jérôme Gasperi