Publishing maritime data with GeoServer

This presentation will describe the architecture put in place, and the challenges that GeoSolutions had to overcome to publish maritime data through GeoServer OGC services (WMS, WFS and WPS), finding the correct balance that maximized ingestion performance and visualization performance. We integrated with a streaming processing platform that took care of most of the processing and storing of the data in a storage that allows GeoServer to efficiently query for the latest available features, respecting all the authorization policies that were put in place. A few custom GeoServer extensions were implemented to handle the authorization complexity, the advanced styling needs and data integration needs of maritime data.

Transcript

1. Nuno Oliveira Simone Giannecchini GeoSolutions Publishing Maritime Data with GeoServer
2. GeoSolutions • Offices in Italy & US, Worldwide clients • 30+ collaborators, 25+ Engineers • Our products • Our Offer Enterprise Support Services Deployment Subscription Professional Training Customized Solutions GeoNode
3. Affiliations We strongly support Open Source, it Is in our core We actively participate in OGC working groups and get funded to advance new open standards We support standards critical to GEOINT
4. The Use Case
5. Maritime Data Overview • Maritime Data is produced by a variety of sources: • Ships positions AIS, SAR, VMS, … • Maritime assets ports, navigational aid systems, …
6. Maritime Data Overview • Provide a foundation for informed decision-making applications: • Maritime traffic monitoring • Search and rescue operations • Environmental marine disasters monitoring • … • Several datasets need to be combined: • Fisheries data • Ships registries information • …. Interoperability!
7. Our Use Case in Numbers • The maritime data is provide as: • Streams of data, landing on Kafka topics • Enrichment data files, pushed in the system • In 24 hours: • We receive up to 50 millions ship positions • We handle up to 500K different ships • Peaks of activity during daylight: • Up to 2500 messages per second!
8. Implemented Scenario • We have implemented the following scenarios in GeoServer using WMS, WFS and WPS: • Visualize in real time ships positions • Density maps computation and visualization • Visualize in real time navigation to aid systems • Detected ships positions correlation and visualization • Electronic navigational charts publishing through WMS
9. • Authorization rights need to be respected: • Different authorization rights will result: Authorization Rights In different views of maritime assets! t1 Ships Sensors SAT-AIS T-AIS User 1 can see all vessels positions. User 2 can only see SAT-AIS vessels positions. t1 t0 t0 t1 t1 t1 t0
10. The Technical Solution
11. High Level System Architecture • The maritime data is provide as: • Streams of data, landing on Kafka topics • Enrichment data files, pushed in the system
12. Data Ingestion Quick Overview • Finding the right balance between writing and reading: • Indexes make readings more performant, but make writings slower! • Processing uses user defined functions: • Filtering Creating layers for different views • Computations Create new attributes • Transformations Transform existing attributes • Processing was implemented with Kafka Streams.
13. Generating Geometries • Storing numerical latitudes and longitudes: • Allow us to use efficient numerical indexes • Geometries are generated on the fly GeoServer side • OGC spatial operators are supported
14. Advanced Authorization • Each position is associated with a list of authorized roles: • GeoServer injects the authorization rules when querying the data! • GeoServer SQL views are sometimes used for better control of the final query. • Administrators can configure pre-authorized caches: • GeoServer takes care of the routing!
15. Advanced Styling • Several performance optimizations where implemented. • Highlighting thousands of fast moving objects:
16. Live Demo
17. Traffic Density Maps • Traffic density maps are computed per ship type and different periods of time:
18. Traffic Density Maps
19. Nautical Charts • Nautical charts served with WMS and WMTS:
20. Real Time Ships Positions • Real time maritime picture displayed using a polar projection (EPSG:5041):
21. Real Time Ships Positions • Real time maritime picture displayed using a style that color each vessel according to its type:
22. Real Time Ships Positions • Real time maritime picture displayed using a style that color each vessel according to the age:
23. Real Time Ships Positions • Real time maritime picture displaying only fishing vessels colored based on their fishing gear:
24. Real Time Ships Positions • Real time maritime picture displaying only cargo vessels, some of them highlighted:
25. Real Time Navigation to Aid Systems • Real time aircraft search and rescue operation:
26. Real Time Navigation to Aid Systems • Real time aids to navigation systems positions:
27. Real Time Navigation to Aid Systems • Search and rescue transponder for ships in distress:
28. Detected Ships Positions Correlation • Correlated and uncorrelated detected ships positions:
29. Detected Ships Positions Correlation • Correlated and uncorrelated detected ships positions:
30. Next Steps • Tracks computation several years of data! • Billions of positions to deal with • Authorization rights are still part of the game • More advanced computations: • On the fly zones intersections detection • Vessel metrics • … • Better integration with “big data technologies”: Integrating with historical data!
31. The End Questions? nuno.oliveira@geosolutionsgroup.com simone.giannecchini@geosolutionsgroup.com info@geosolutionsgroup.com