The New Geoplatform: Tod Dabolt, DOI

Summary: Geoplatform.gov was recently rebuilt from the ground up. Tod talks about new features of the platform and plans for the future.

Transcript

1. GeoPlatform Geospatial Platform Update Tod Dabolt Department of the Interior USGS TWG April, 2016
2. GeoPlatform Release Schedule Recent and Near Term Releases  Release 2: October 2015 (Done)  Release 3: February 2016 (Done)  Release 4: May 2016 (Next)  Release 5: June 2016 (Planning)  Release 6: September 2016 (Planning) 2
3. GeoPlatform  What’s New: Look and feel Additional map viewer functionality Create and import maps from Esri’s ArcGIS On-Line Create and embed Map Galleries into external sites Portfolio management capabilities to measure and manage national geospatial data assets Enriched geospatial marketplace Secure scalable Amazon Web Service infrastructure 3 Current GeoPlatform
4. GeoPlatform Landing Page Map Viewer 2.0 AGOL Map Harvester Map Manager, Phase I NGDA Survey Tool 3DEP Project Tracker Performance Dashboard, Phase I IDM/IDP Replacement Service Desk process defined and implemented O&M/DevOps fully implemented 90% in-the-cloud 4 Release 2 & 3 Accomplishments
5. GeoPlatform Went live Feb 2016 Enhanced user experience and navigation for both first-time and frequent users New Homepage and overall responsive site design Interactive browse / view / mashup of all geospatial web services in Geoplatform.gov catalog with service quality indicators 5 Release 3 Overview
6. GeoPlatform Map Viewer Phase II: Map saving and sharing Map Knowledge Graph prototype Map Manager, Phase II with AGOL Harvester Performance Dashboard, Phase II: Service Checker NGDA Metadata Viewer Batch Selections Registry prototype 6 GeoPlatform R3 Overview (con’t.)
7. GeoPlatform Enhanced navigation to quickly direct users to specific areas of interest  Account registration and management for added access and functionality Tutorials and training opportunities Help tools Dedicated and responsive Service Desk 7 New User Benefits
8. GeoPlatform SSO and seamless access to data.gov and geoplatform.gov resources Access to enhanced service capabilities: Performance Dashboard AGOL Map Harvester Ability to save and import maps in the Web Map Viewer (WMV) Creating/editing Map Galleries with Map Manager 8 Account Management
9. GeoPlatform Multiple data browse, filter, and sort features  Dataset preview Add user content: Digitize features or Upload .KML, GeoJSON, .CSV, or shapefiles (in .ZIP) to extract features Gazetteer and Geolocation support Control layer ordering, visibility, opacity Save and share maps (Web Map Context) Publish composed maps 9 Map Viewer
10. GeoPlatform Browse harvested AGOL maps by search filters or organization Create map galleries with maps from WMV and AGOL Share maps and galleries via email Embed map blocks and curated gallery pages into other websites Customize gallery page with community and keywords Manage gallery pages Permissions-based access and use 10 Map Manager
11. GeoPlatform Harvest maps from ArcGIS online organizations and publish them to the GeoPlatform Search and Discover maps by search filters or organization Unlimited publishing of data and maps to AGOL* Create map galleries with maps from Map Viewer and AGOL 11 ArcGIS Online Integration
12. GeoPlatform Tools to measure, monitor, and report on NGDAs and services NGDA Dataset Lifecycle Maturity Assessment survey tools Service Checker 12 Performance Dashboard
13. GeoPlatform 13 Performance Dashboard - Reliability Reliability of USGS National Structures Service
14. GeoPlatform Community Engagement. New applications, services and tools for integrating with and enhancing content Upgraded Marketplace Service New metadata tools for discovery of resources and linking data to services Measurement, Monitoring, and Reporting. Quantitative performance metrics (data lifecycle maturity, QoS service tracker, with service utilization, 14 SCREENSHOT HERE What’s Coming in Release 4 (May)?
15. GeoPlatform High level Cloud Architecture 15
16. GeoPlatform Development to Production workflow 16
17. GeoPlatform Geoplatform Cloud Hosting Services Platform as a Service (Paas) Geoplatform Managed GIS Infrastructure as a Service (IaaS)  ESRI Arc Server  Boundlass Stack Software as a Service ArcGis on-line for Organizations (AGOL) 17
18. GeoPlatform Map Knowledge Graph and enhanced Registry services Layer recommendation service More NGDAs available via reliable services Additional Marketplace functionality APIs for Developers 18 What are we working on?
19. GeoPlatform Technical Approach Summary Page 19 Key Solution Characteristic Benefits A common reusable approach to GeoPlatform resource management Lowers software lifecycle costs A state-of-the-art technology base built upon extensible framework and standards from the ground up Proven extensible, interoperable standards-technology stack; lowers software lifecycle costs Resources, and all critical operational data about resources, are available through machine-friendly RESTful APIs A common set of machine-friendly APIs for all GeoPlatform resources; Simplifies architecture; Reduces redundancy and development cost Common core metadata model for all resources; Specialized profiles for resource types Harmonizes resource management and interoperability under a common framework Employs semantic techniques to ground Registry concepts for machine-understanding Key to machine-understanding at API level, which is critical for automation Augments resource behaviors through Auxiliary Resource extensions Shields client developer and users from resource state engine, where complex specialized behaviors like resource tradecraft, resource associations, access control, business policy enforcement, reliability, usage, budget, plans, etc. are handled
20. GeoPlatform Registry+: THE Key to Achieving High- Performance GeoPlatform Services Key Feature Benefits Flexible REST API Clear separation of concerns between client-side business logic and server-side resource management; Reduces client-side resource business logic; Less brittle hypermedia-based response actions Rich, common core metadata model Standards-based; Semantic integrity; Common high-performance indexing solution Extended, common metaknowledge model to capture resource tradecraft Key to automating certain operations, like resource recommendation; Extends resource semantics (richer meanings) Auxiliary resources Standard extension mechanism to quickly add any auxiliary resource behaviors (e.g., service reliability) Aggregation pattern Means of aggregating key synoptic resource behaviors as extended metadata or metaknowledge (e.g. a “green” service score) Page 20 Distinguishing Features of Registry+:
21. GeoPlatform Registry+ Component Summary Registry+ Model Core Metadata Dataset Application Profile Service Application Profile Tradecraft (Resource Metaknowledge) Registry+ REST API Registry+ Datastore w/ Multi-modal Index (spatial, temporal, lexical, semantic) Autonomous Harvester Service to populate Registers Scrapes data.gov and mines 19139 docs Issues: • Dataset metadata lacks semantics and consistency • Service & layer metadata is inconsistent and in many cases non-existent Page 21
22. GeoPlatform A Proven Registry+ Technology Stack Java 8 Object Store: MongoDB Search Engine: ElasticSearch Data Access: Spring Data + QueryDSL + Jackson Service Layer: Spring Core Framework Web Layer : Spring MVC and Spring REST HATEOAS: Spring HATEOAS* Unit Testing: Spring MVC Test REST Documentation: Spring RESTDoc Technical Documentation: AsciiDoctor Deployment Management: Spring Boot Key benefits • Robust, stable enterprise-level APIs • Dramatically lowers development and O&M costs; Our code footprint is small … mostly handled by framework • Large community and good documentation and resources • Well-proven technologies • Scalable architecture • Unified approach to accessing heterogeneous datastores (RDBMS, NoSQL, Graph stores) • Accommodates changes of backend RegistryItem states without rewriting code • HATEOAS out of the box • Registry+ resource states must be managed server-side • Added flexibility: "link"-tags in REST APIs provide URLs that are needed for permissible actions, depending on resource state • REST documentation is driven by unit testing • Flexible/portable documentation; integral part of the software lifecycle * HATEOAS (Hypermedia as the Engine of Application State) is a constraint of the REST application architecture. A hypermedia-driven site provides information to navigate the site's REST interfaces dynamically by including hypermedia links with the responses.
23. GeoPlatform Registry+ Technology Standards Framework JSON JSON-Schema Hypermedia-driven HTTP REST protocols Application-Level Profile Semantics (ALPS) Hypermedia Application Language (HAL) ASCIIDoc JSON-LD (Linked Data) Well-proven standards with widespread adoption
24. GeoPlatform Registry+ Service Architecture Layer-based approach • Based on best practices • Each layer can evolve independently without breaking other layers • Agnostic to back-end store Spring Data MongoDB Spring Data Elasticsearch Data Access Layer Repository API Entries Repo Item Repo Register Repo Item Index Repo Registry Service Provenance Repo Indexing Service Harvesting Service Harvester Repo Spring REST API Service Layer Web Layer Persistence Layer MongoDB
25. Page 25 How Auxiliary Resources Come Into Play Client GeoPlatform Service Service Checker Registry+ Registry+ Service Resource Authority Policy Engine Policies & Standards Metered Service Service Reliability Service Utility Access Control Auxiliary resource and associated application state complexities remain behind the scenes. Registry+ aggregates these states for a resource. Registry+ Service is the main entry point for all significant GeoPlatform resources.
26. GeoPlatform 26 Conclusion GeoPlatform is helping advance the NSDI by enabling users to: More easily find existing maps and data: Semantic search / Knowledge graphs will help users surface the right content more quickly and easily Get access to more high quality NSDI services: Cloud hosting offerings have the ability to change the game in terms of making high quality data available Understand (and improve!) the quality of the NSDI services: Service status checker and performance dashboard an continuously monitor the component services of the distributed NSDI More easily share data, tools, maps and collaborate: GeoPlatform offerings bring collaboration and analytics tools at scale to end users in all sectors
27. GeoPlatform Thank you 27