GeoSolutions GeoSolutions is one of the world’s leaders in supporting open-source geospatial software. ~ 30 Staff > 200 Clients - Actively participant in the OGC
UN FAO (CIOK, FIGIS, NRL, FORESTRY, ESTG), UN WFP, World Bank, DLR, EUMETSAT, JRC, ARPAT, NATO CMRE, UNESCO, IGAD, UNEP, ... BAYER, BASF, DigitalGlobe, MDA, TOPCON, SwissRE, e- GEOS, Halliburton, .. More than 200 clients since 2006
GeoSolutions Support • Advice regarding a geospatial challenge • Architect, help configure and deploy clusters of GeoNodes in the cloud • Automate deployments from different environments • Performance monitoring • Develop custom application (server or client slide) • Optimize ingesting and publishing geospatial data • Training • Other technical support
GeoSolutions Contact Information Luis E. Bermudez, Ph.D. CEO GeoSolutions USA https://www.geosolutionsgroup.com/contacts/ +1 (301) 337-7055 [email protected] https://www.linkedin.com/in/bermudez https://www.geosolutionsgroup.com https://twitter.com/geosolutions_it
GeoNode at a Glance A web framework based on Python and Django to allow people to upload, describe, share and use their geospatial data. We usually define GeoNode as a geospatial-CMS. Core Components • Django • PotsgreSQL-PostGIS • GeoServer • GeoWebCache • pycsw • MapStore
Capabilities o Upload geospatial datasets (by default shapefiles and GeoTIFFs) o User with appropriate permissions can edit layer metadata, which are exposed by OGC CSW and REST, to provide search/discovery capability o Create thematic maps accessible to general public o Users with appropriate permissions can edit layer styles and features (for vector layers) o Granular permission systems: viewing, downloading, metadata editing, styles and feature editing for a layer can be restricted to users or groups o GeoNode exposes a number of standards for each layer: OGC (WMS, WMS-C, WFS, WFS-T, WCS, CSW) and mass market search standards (OAI-PMH, SRU, OpenSearch)
Release History o April 2021, GeoNode 3.2.0 (Django 2.2.16, GeoServer 2.18.2) / GeoNode 3.1.1 (last 3.1 release) o May 2020, GeoNode 3.0 (Django 2.2.13, GeoServer 2.16.2) / GeoNode 2.10.4 (last 2.10 release) o February 2020, GeoNode 2.10.2 (Django 1.11.29, GeoServer 2.15.4, pycsw 2.4.2) o June 2019, GeoNode 2.10 (Django 1.11.20, GeoServer 2.14, pycsw 2.2.0) o April 2018, GeoNode 2.8 (Django 1.8.19, GeoServer 2.12.2, pycsw 2.0.2, group moderation and resources publication workflow, SLD upload, metadata wizard) o May 2017, GeoNode 2.6 (Django 1.8.7, GeoServer 2.9, pycsw 2.0.2, React client, QGIS server backend, ansible and docker setup, Ubuntu 16.04 support) o November 2015, GeoNode 2.4 (Django 1.6.11, GeoServer 2.7, pycsw 1.10.5, django- guardian, groups, remote services, responsive template, Ubuntu 14.04 support) o April 2014, GeoNode 2.0 (Django 1.5.5, GeoServer 2.5, pycsw 1.8.6, django-polymorphic, bootstrap, Ubuntu 12.04 support) o October 2012, GeoNode 1.2 (Django 1.4, GeoServer 2.3, South migrations, django-taggit, social features, comments and ratings, find/add layers widget) o May 2012, GeoNode 1.1.1 (Ubuntu 10.04 and 11.04 installer) o December 2010, GeoNode 1.0, with major contributions from OpenGeo, the World Bank, GFDRR, UNISDR, and GEM o August 2010, GeoNode 1.0-beta
June 2019 Releases covered by this presentation 2.10.1 3.1 development stable maintenance Python 3.8+ Django 2.2.16 GeoServer 2.18.2 YOU ARE HERE Sept 2019 May 2020 Feb 2020 2.10.2 2.10.4 3.2.0 April 2021 3.1.1
Are you using a older version? Upgrade! o User/Developer lists and Gitter typically covers only supported versions o Today it means only 3.2.x o Security fixes and installation support on new OSs being added to supported versions only! o Moving data from old versions will be much harder if not impossible o Web interfaces and GIS clients will be much different. No one will be able to backport fixes or improvements to the old ones. o Please upgrade your GeoNode installations!!!
Upgrade? What’s in it for me? o Let’s check what’s new in 3.2.0 upcoming release o Check the bottom of each slide to see who sponsored a certain feature, who implemented it, and what version contains it Sponsor Author Version
Upgrade to GeoServer 2.18.2 Community GeoSolutions GeoSolutions 3.2.0 Improvements and Fixes • Improved GWC seeding scalability (also check GWC 1.18.2 release notes). • Fixed back-links generation during WPS asynchronous requests, when a proxy base URL is used. • Fixed a GeoFence server packaging issue. • A number of dependent libraries have been upgraded, including the PostgreSQL and MySQL JDBC drivers, HTTP components, Guava. • The new support for COGs based on ImageIO Ext landed in a community module, adding it adds support for COG in both the GeoTIFF reader and image mosaic. Support for harvesting COG granules is also added in the REST API. • A deadlock in JDBCConfig has been resolved, along with issues related to high load when GeoServer has just started up. • The WPS download “map” and “animation” processes sport improved legend support. For more information check the 2.18.2 release notes. Community Updates ✓ Backup & Restore ✓ OAuth2 ✓ GeoFence
Upgrade to MapStore Client 2.1.3 Community GeoSolutions GeoSolutions 3.2.0 Improvements and Fixes • Save Search Services to the Map • Save Map Preferences into the adapter model • Advanced Style Editor with graphical UI • Improved Map Save Widget, specific for GeoNode • New MapStore client configuration subsystem • Group Layer as named groups For more information check the 2.1.3 release notes. For technical details check: https://github.com/GeoNode/geonode-mapstore-client#readme
MapStore Advanced Styler AlmavivA GeoSolutions GeoSolutions 3.2.0 o Old Styler was somewhat “poor” ✓ Widget based styler ✓ Vector data classifications ✓ Raster Data Classifications ✓ Well Known Markers ✓ Advanced Styling Options
MapStore WPS Based Async Export AlmavivA GeoSolutions GeoSolutions 3.2.0 o Relies on WMS Download Process to export data asynchronously ✓ Different File Formats ✓ Native or WGS-84 projections ✓ Crop dataset option ✓ Download filtered dataset ✓ Export data results
Breaking Changes o Bump to postgresql-13; do not upgrade the docker image or be prepared to do a dump/restore of the DB o Based on MapStore client 2.1.3; MapStore client is no more compatible with 3.0.x and 3.1.x train o ResourceBase Model BBOX now is a geometry; that is no more compatible with the 3.1.x one o Revised Async Model; tasks and queues have been updated and refreshed. Introduced a non-blocking lock mechanism based on MEMCACHED. o GNIP-81 - GeoNode Core Cleanup (ongoing…): ⚫ Removed GeoNetwork support ⚫ Removed QGIS-Server support ⚫ Removed SPC GeoNode support ⚫ Advanced Resources Publishing Logic has been revised Community GeoSolutions Catalyst CSGIS 3.2.0
Breaking Changes: BBOX Geometries o ResourceBase Model BBOX now is now a Geo-Django geometry field o The migrations should fix automatically the model from old releases o The management command “set_all_layers_metadata -d” refreshes and synchronizes the BBOXes with the GeoServer layers ones Community Catalyst GeoSolutions Catalyst CSGIS 3.2.0
Breaking Changes: Revised Async Model o Enabling Fully Asynchronous Tasks: o https://docs.geonode.org/en/master/install/advanced/core/index.html#enabling-fully-asynchronous-tasks o Enabled and configured by default on the Docker images o Upgrade from old Dockerized instances will require to drop the rabbitmq image in order to let GeoNode recreate the queues and topics correctly Community GeoSolutions 3.2.0
GNIP-76: Add RTL Support CSGIS CSGIS 3.2.0 ✓ This GNIP suggest enabling right-to-left support for named languages https://github.com/GeoNode/geonode/issues/6156
GNIP-77: GetFeatureInfo Templating For GeoNode AlmavivA GeoSolutions 3.2.0 ✓ This proposal aims to improve the way GeoNode injects a “GetFeatureInfo” Template into the maps created with MapStore2 https://github.com/GeoNode/geonode/issues/6182
GNIP-78: GeoNode generic "Apps" model to include pluggable entities into the framework AlmavivA / GeoSolutions GeoSolutions 3.2.0 https://github.com/GeoNode/geonode/issues/6684 o GeoSpatial Storytelling in GeoNode ✓ MapStore Plugin ✓ Reuse GeoNode Layers and Resources to build Story ✓ Images, Video, Maps and so on ✓ Integrated visual editor ✓ Documents now support more media-types
GNIP-79: GeoNode REST APIs (v2) AlmavivA GeoSolutions 3.2.0 ✓ Create a real REST API for GeoNode resources, which is: ✓ Pluggable: new external dependencies and modules should be easily allowed to add their own API endpoints without modifying the GeoNode code base. ✓ Modular: each app has their own API module if needed. No more monolithic modules with a ton of classes ✓ REST compliant: the new API must adhere to the REST paradigm and protocol ✓ Lives together with the standard Tastypie based GeoNode APIs https://github.com/GeoNode/geonode/issues/6685 Allows for search, filtering, ordering, field embedding: https://github.com/AltSchool/dynamic-rest#demo e.g.: ❖ https://master.demo.geonode.org/api/v2/geostories/20?include[]=data ❖ https://master.demo.geonode.org/api/v2/resources?filter{resource_type}=layer
GNIP-83: ResourceBase for metadata-only resources AlmavivA GeoSolutions 3.2.0 ✓ Flag ResourceBase instances in order not to be displayed in the GUI, but only available through CSW. ✓ With this kind of "hidden" ResourceBase, our underlying pyCSW can be decoupled from the data handled in GeoNode/GeoServer, and regain a bit of the functionalities lost when using the internal pyCSW (the only one currently maintained) with external catalogues. ✓ A good use case is the publishing of service metadata documents that details the WMS, WFS and WCS services offered by our GeoNode/GeoServer instance. https://github.com/GeoNode/geonode/issues/7057 A service metadata is an ISO19115 metadata which describes the service in its entirety. The responsible parties are the ones running the service, which are not described in the data. Service metadata also describes the various operation available in the service, which operations are supported, which are the standards it conforms to, which are other qualitative and quantitative constraints it follows. INSPIRE requires that a catalogue should provide service metadata for an organization. If GeoNode can not provide them, then GeoNode can only be used as a slave catalogue by some other high level catalogue that can include such extra metadata. With this GNIP we can remove the use of other catalogues in the architecture.
GNIP-84: Upload Page Enhancements https://github.com/GeoNode/geonode/issues/7154 o Improved the upload of resources both manual as well as programmatic. o A first page of enhancements has landed as per GNIP-84, which provides better control over upload sessions, first of all the ability to terminate sessions after we have left the upload page. GeoSolutions GeoSolutions 3.2.0 ✓ Improved Async Upload of Layers ✓ GeoServer Importer Uploads are now stateful and can be resumed or cancelled
GNIP-86: Metadata parsing and storing AlmavivA GeoSolutions 3.2.0 ✓ Metadata parsing is not easily customizable. ✓ The proposed changes allows to ✓ Define one or more custom parsers, in order to be able to handle metadata documents following specific profiles not handled by OWSLib. ✓ Add an hook to handle custom DB object on layer creation. ✓ Other tasks included: ✓ Fix keyword parsing. https://github.com/GeoNode/geonode/issues/7263 Linked issues: • #7279: Keyword parsing improvements • #7288: Layer saving clean up Docs: https://docs.geonode.org/en/master/basic/settings/index.html#metadata-parsers
GNIP 87: Reduce information returned to users to only what is strictly required and accessible UNESCO GeoSolutions 3.2.0 ✓ Fixes a security leak: ✓ GeoNode's /owners API endpoint returns the full list of users, including their personal information (!!!!). Also it returns users with no resources. https://github.com/GeoNode/geonode/issues/7282
Improved Thumbnails AlmavivA GeoSolutions 3.2.0 ✓ Thumbs creation logic which now relies on WMS GetMap instead of outdated GeoServer Print Plugin ✓ Better quality and ratio ✓ Works on any WMS-based service ✓ Works on most of the Remote Services also ✓ Does not relies on GeoServer plugins and therefore does not consume memory and resources on server side
Improved B/R Machinery ADBPO GeoSolutions 3.2.0 ✓ Refactored B/R code ✓ Now, by default, it performs an hard restore, by removing also the DATA tables on the target
Advanced Upload Workflow Improvements ADBPO GeoSolutions 3.2.0 ✓ When the Advanced Upload Workflow is enabled: ✓ Non admin user cannot change permission ✓ Disable edit permissions globally when read-only mode is active ✓ Unapproved Resources: ✓ The owner won't be able to change to neither "approved" nor "published" state (unless he is a superuser) ✓ If the Resource belongs to a Group somehow, the Managers will be able to change the state to "approved" but NOT to "published". Only a superuser can publish a resource. ✓ By default the uploaded resources will be "unpublished“: ✓ "unpublished" won't be visible to Anonymous users ✓ "unpublished" will be visible to registered users IF they have view permissions ✓ "unpublished" will be always visible to the owner and Group Managers ✓ If the owner assigns "unpublished" resources to a Group, both from Metadata and Permissions, in any case the Group "Managers" will be able to edit the Resource. ✓ The "unapproved" and "unpublished" Resources will be accessible ONLY by owners, superusers and members of the belonging groups.
New Contrib Apps: GeoNode extension Keycloak-Role Django Application Community Community 3.2.0 ✓ The goal of this application is to synchronize Keycloak roles with GeoNode group profiles. ✓ The application creates a connection with Keycloak. Using client credentials the roles are retrieved from Keycloak. Each role is then assigned a group. ✓ The synchronize button can be found in Django Administration page, Home -> Keycloak -> Keycloak roles and right next to add Keycloak role button. https://github.com/GeoNode/geonode-contribs/tree/master/django-geonode-keycloak
Translations Community THUNEN 3.2.0 o Updated translations for all the languages and revised the major ones o Other related issues with frontend behavior ✓ ISO category translation not shown #6984 ✓ makemessages command will set fixture translations to inactive #7015 ✓ multi word iso category strings will not get translated #7126 ✓ html body class translate #7354
Other Fixes and Improvements Community Community 3.2.0 ✓ Improving GeoNode Theme Library: introducing Jumbotron Slides ✓ Implementation of an action to assign bulk permissions on layers to users selected from People and/or Group Django admin forms enhancement #6582 ✓ File system operations do not adhere to Django file storage API enhancement in progress #6414 ✓ Nav Toolbar gets distorted when multiple navbar items are added by the admin enhancement frontend major #6412 ✓ Improve GeoNode OpenID SP Protocol in order to be able to provide access to external clients enhancement security #6273 ✓ Limit "maps using this layer" to maps the user has permission to see enhancement security #6261 ✓ Prevent integrity errors on singleton model save enhancement #6223 ✓ …
Next 3.2.x minors GeoSolutions 3.2.x ✓ Upgrade Geoserver to 2.18.3 ✓ Review and improve Docker configurations ✓ Improve the permissions system, and their integration with the client ✓ Introduce GUI based execution of management commands ✓ Start the refactoring of the monitoring service ✓ Integrate MapStore Dashboards
Dashboard GeoApp - data dashboards in GeoNode GeoSolutions 3.2.x ✓ Dashboard is a space where the user can add many Widgets, such as charts, maps, tables, texts and counters, and can create connections between them in order to: • Provide an overview to better visualize a specific data context • Interact spatially and analytically with the data by creating connections between widgets • Perform analysis on involved data/layers ✓ Add Maps and Layers from the GeoNode catalogue ✓ Integrated visual editor https://geosolutionsgroup.com/blog/mapstore-release-dashboards/ https://mapstore.readthedocs.io/en/latest/user-guide/exploring-dashboards/
Master branch GeoSolutions 3.2.x ✓ New SPA client based on React and MapStore • New home page and user workspace • Refactoring of resource viewers and editors ✓ Introduction of a Resource Service which will provide the API to manage the full lifecycle of a resource ✓ Introduction of a Storage Service to implement proper management of the data storage for the resources ✓ Introduction of a Harvester Service which will implement a pluggable and extendable system for remote resources and data harvesting ✓ General improvements to the resource management and the core asynchronous systems to provide support for more advanced pipelines, which include: • imports from external storage services (AWS S3, Dropbox, etc.) • data pre-processing • interactive workflows • …. The Pacific Community
Home Page and Workspace 3.2.x ✓ React and MapStore based with improved layout ✓ Direct access to the full list of resources ✓ Quick live previews ✓ A home page for casual users, a workspace for contributors ✓ Visual and functional separation between catalogue and dataviz functionalities ✓ Improved organization of tools, layered on multiple visual levels, depending on the user profile (end user, data owner, administrator) GeoSolutions The Pacific Community
Viewers and Editors 3.2.x ✓ Simplified and clear UI/UX both for readers and editors ✓ Remove duplicated functionalities between backend pages and client ✓ Quick editing workflows ✓ Permissions management turned into sharing options GeoSolutions The Pacific Community
User Lists Participation o Answering users' questions relies on a low number of people o We lack “testers”; this kind of project would need also a lot of manual testing for all its functionalities on a regular basis o Developers are very few and Pull Requests often do not respect the contribution policies. In particular: ✓ There’s no GitHub issue describing the problem linked to the Pull Request ✓ GitHub issues are plain requests often without a good description of the use case and how to reproduce it ✓ Lack of test cases and documentation ✓ History of commits is usually messed up
Steps to get in touch with developers 1. If you need for clarification first of all try to describe the issue as well as possible through the official mailing lists; IMPORTANT: always specify ✓ Which versions of GeoNode and GeoServer you are using ✓ Which Operative System and hardware you are using ✓ How you installed the framework 2. If you need to get in touch directly with developers, consider using the official “gitter” chat https://gitter.im/GeoNode/general 2. If the issue has been confirmed and there’s no easy or immediate resolution, open a ticket on GitHub https://github.com/GeoNode/geonode/issues with steps on how to reproduce the problem and labels
In case you stumble into a vulnerability: Responsible Disclosure o Keep exploit details out of issue report. o Mark the issue as a vulnerability. o Be prepared to work with Project Steering Committee (PSC) on a solution. o Keep in mind PSC members are volunteers and an extensive fix may require fundraising / resources. If you are not in position to communicate in public please consider commercial support, contacting a PSC member, or reaching us via the Open Source Geospatial Foundation at [email protected]
simboss
tapompom
okaharuna
lnit
devoc
ogiwarat
yuhta28
aethiopicuschan
skanehira
ari_mharada
tomokusaba
hirodragon112
tagomoris
cherdarchuk
lara
dotmariusz
jonyablonski
jasonvnalue
kneath
colly
cromwellryan
deanohume
denniskardys
geoffreycrofte
malarkey