Cloud Computing & Web Processing Services

Slide 1

Slide 1 text

RTS Techniques de l'Information et de la Communication CNES - Toulouse, France - October 1st, 2013 https://speakerdeck.com/jjrom/cloud-computing-and-web-processing-services Cloud Computing & Web Processing Services [email protected] Jérôme Gasperi

Slide 2

Slide 2 text

Cloud Computing Introduction What we have done Issues Web Processing Services Introduction What we have done Issues What's next ? WPS on the cloud

Slide 3

Slide 3 text

R&T Cloud Computing (2011) Use a cloud infrastructure to process Earth Observation data

Slide 4

Slide 4 text

Slide 5

Slide 5 text

host build consume IAAS PAAS SAAS

Slide 6

Slide 6 text

host build consume IAAS PAAS SAAS

Slide 7

Slide 7 text

Private or public ?

Slide 8

Slide 8 text

process 1 € store 5 € access 0.5 € to + 1 time per year each time (Very) approximative computations for a 20x20 km PA+XS Pleiades product (4 Go) - based on Amazon EC2 and S3 pricing (2012)

Slide 9

Slide 9 text

process 1 € store 5 € access 0.5 € to + 1 time per year each time (Very) approximative computations for a 20x20 km PA+XS Pleiades product (4 Go) - based on Amazon EC2 and S3 pricing (2012)

Slide 10

Slide 10 text

Data security and user privacy cannot be guaranteed in public clouds Majority of cloud providers are subject to governmental law (e.g. US patriot act)

Slide 11

Slide 11 text

...so we choose a private solution

Slide 12

Slide 12 text

Web browser What we have done eNovance (Paris)

Slide 13

Slide 13 text

Web browser What we have done eNovance (Paris) SRTM tile Data

Slide 14

Slide 14 text

Web browser Processing (Orfeo Toolbox) What we have done eNovance (Paris) SRTM tile Data

Slide 15

Slide 15 text

Hill Shading Web browser Processing (Orfeo Toolbox) What we have done eNovance (Paris) SRTM tile Data

Slide 16

Slide 16 text

Hill Shading Projecting Web browser Processing (Orfeo Toolbox) What we have done eNovance (Paris) SRTM tile Data

Slide 17

Slide 17 text

Hill Shading Projecting Web browser Processing (Orfeo Toolbox) What we have done VISUALIZE WMS Server Tiling eNovance (Paris) SRTM tile Data

Slide 18

Slide 18 text

Conclusions

Slide 19

Slide 19 text

Private cloud technologies are quite simple to implement. They reduce the cost of operation and maintenance by sharing a common infrastructure across multiple projects

Slide 20

Slide 20 text

However...

Slide 21

Slide 21 text

Transfering large volumes of data is problematic However...

Slide 22

Slide 22 text

Transfering large volumes of data is problematic This is true for upload and download However...

Slide 23

Slide 23 text

...so

Slide 24

Slide 24 text

Data and processes should be colocated ...so

Slide 25

Slide 25 text

Data and processes should be colocated Processing results should be downloaded only if needed ...so

Slide 26

Slide 26 text

Data and processes should be colocated Processing results should be downloaded only if needed Better to offer a Web service to visualize/manipulate results ...so

Slide 27

Slide 27 text

Data and processes should be colocated Processing results should be downloaded only if needed Better to offer a Web service to visualize/manipulate results Standardize processes inputs/outputs description (e.g. WPS) ...so

Slide 28

Slide 28 text

Web Processing Service (2012-2013) Set WPS on top of Orfeo Toolbox processing chains

Slide 29

Slide 29 text

Web Processing Services (WPS) is an OGC standard which defines interfaces to publish, describe and execute geospatial processes

Slide 30

Slide 30 text

WPS a Processing Web application

Slide 31

Slide 31 text

From pixel... Remote sensing image What we have done

Slide 32

Slide 32 text

From pixel... Remote sensing image ...to information Land cover thematic map What we have done

Slide 33

Slide 33 text

mapshup Web application http://mapshup.info WPS Constellation http://www.constellation-sdi.org Orfeo Toolbox Processing http://orfeo-toolbox.org

Slide 34

Slide 34 text

Orfeo Toolbox Remote sensing image library Open Source Developped by the French Space Agency Processing

Slide 35

Slide 35 text

Orfeo Toolbox More than 70 high level processing chains orthorectification segmentation classification etc. Processing

Slide 36

Slide 36 text

Orfeo Toolbox More than 70 high level processing chains orthorectification segmentation classification etc. Processing Supervised learning (land cover is computed from a set of "well known areas" given by user) Based on SVM (http://en.wikipedia.org/wiki/Support_vector_machine)

Slide 37

Slide 37 text

Constellation WPS Geospatial Networking Services OGC/ISO implementations Multi platform (JAVA) OpenSource WMS WFS CSW WPS

Slide 38

Slide 38 text

mapshup Improves access to geospatial data HTML + javascript Based on OpenLayers and jQuery Web application

Slide 39

Slide 39 text

mapshup Supported data sources WMS WFS CSW WPS OpenSearch etc. Flickr Youtube GeoRSS KML Wikipedia Google maps Bing maps OpenStreetMap MapBox WMTS Web application

Slide 40

Slide 40 text

mapshup Supported data sources See WPS demo https://vimeo.com/67288678 WMS WFS CSW WPS OpenSearch etc. Flickr Youtube GeoRSS KML Wikipedia Google maps Bing maps OpenStreetMap MapBox WMTS Web application

Slide 41

Slide 41 text

https://vimeo.com/67288677 What have we done ?

Slide 42

Slide 42 text

Conclusions

Slide 43

Slide 43 text

WPS 1.0 lacks a true asynchronous process management... There is no way to cancel a launched process !

Slide 44

Slide 44 text

However...

Slide 45

Slide 45 text

WPS 2.0 defines a set of process management operations - GetStatus, Delete, Pause and Resume However...

Slide 46

Slide 46 text

WPS 2.0 defines a set of process management operations - GetStatus, Delete, Pause and Resume This is a must have to deploy asynchronous WPS on the cloud However...

Slide 47

Slide 47 text

WPS 2.0 defines a set of process management operations - GetStatus, Delete, Pause and Resume This is a must have to deploy asynchronous WPS on the cloud Should be an official OGC standard by the end of 2013 However...

Slide 48

Slide 48 text

What's next WPS on the cloud - OGC OWS-10 Testbeb

Slide 49

Slide 49 text

Orthorectifying images hosted by a cloud infrastructure using WPS standard In kind contribution to the «Open Mobility» thread of the OGC OWS-10 Testbeb Final delivery and demonstration April/May 2014

Slide 50

Slide 50 text

Raw images Processing Data (Pléiades samples) Client Interoute (location : UK) Services PROCESS WPS Server Orthorectify (OTB) VISUALIZE WMS Server SEARCH CSW Server Metadata + 1 2 3 Assess Quality Improve Quality Services Processing Astrium GEO Result Quality WPS Server 4 VISUALIZE WMS Server

Slide 51

Slide 51 text

Landsat Landsat Landsat 2. Click on '+' to process a new orthorectification. Process parameters are set by user and sent to an asynchronous WPS orthorectification process located on the 3. Result is displayed within the map trough a WMS Landsat 4. Orthorectified image quality can be checked through "Assess Quality" WPS process located on the Result is displayed within the map as a WMS quality layer stored on the 1. Select raw image to orthorectify within CSW catalog located on the Laat