Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deploying and operating GeoServer: a DevOps perspective - FOSS4G 2021 Cut

Deploying and operating GeoServer: a DevOps perspective - FOSS4G 2021 Cut

In this presentation we will share with you the lessons we have learned at GeoSolutions when deploying and operating GeoServer as well as some common patterns for the migration of on premise GeoServer clusters to the cloud. We'll share with you tips on how to:
- best practices to migrate your existing GeoServer cluster to the cloud
- insights on your geoserver cluster using centralized logging and Monitor plugin
- avoid common bottlenecks to best set up a distributed scalable GeoServer cluster
- work containers and container orchestrators like Kubernetes

Simone Giannecchini

October 04, 2021

More Decks by Simone Giannecchini

Other Decks in Technology


  1. GeoSolutions • Offices in Italy & US, Worldwide clients •

    30+ collaborators, 25+ Engineers • Our products • Our Offer Enterprise Support Services Deployment Subscription Professional Training Customized Solutions GeoNode
  2. Affiliations We strongly support Open Source, it Is in our

    core We actively participate in OGC working groups and get funded to advance new open standards We support standards critical to GEOINT
  3. Agenda • Cloud Computing • Two words definition • Why

    is relevant? • Moving to the Cloud • Avoid common bottlenecks • Distributed, Scalable GeoServer cluster • Containers, Orchestrators and Kubernetes • Monitoring and Logging • Debugging, Challenges and Solutions • Gain Insights from your Cluster
  4. What is Cloud Computing • Computing services over the internet

    • Servers, Storage, Databases, Networking, Software, ... • Pros and cons • Elasticity and Scalability • Reduced time to market • Business flexibility • Security and Privacy • Lower Costs?
  5. Deployment Models • Public Cloud • Most widely used •

    Cost Effective • Private Cloud • Dedicated Cloud infrastructure to an organization • Secure env for sensitive tasks • Governments, schools, agencies, … • Costly • Hybrid Cloud • combination of public cloud infrastructure with a private environment
  6. Moving GeoServer to che cloud • Rehost or “Lift &

    Shift” • IaaS approach • Redeploy existing data and applications on the cloud infrastructure • “Quick and Easy” • Not leveraging all services offered by the provider • Refactor or “lift, tinker, and shift” • Tweak and optimize applications for the cloud • PaaS approach • Same core architecture but leveraging services offered by the provider
  7. Moving GeoServer to che cloud • Revise, Rebuild, Replace •

    More changes to the architecture • Code Changes, Rewriting, adoption of third-party and prebuilt applications • Enables applications to take advantage of the services offered • Requires foreplanning and Knowledge
  8. • Set up VMs • Use computing resources of the

    provider • Choose the right kind of VM with good compute performance • Rule of thumb: 4 Cores and 4GB per instance • Redeploy the cluster • Migrate Instances • Configuration and data • Done Rehost your GeoServer Cluster
  9. Refactor your Cluster • PaaS Approaches • Setup your VMs

    or use a platform • Managed DBMS Service • Storage Options • Choose the right kind for each component (datadir, data files, cache, ..) • Object vs File vs Block Storage • GeoServer supports COGs! (Community Module) • You can store Cached Tiles in Object Storage
  10. Choose Storage - File Share • File Share • File

    Storage ~ NFS • Shared (locally) • Scales well • Typical use case is GeoServer datadirs • Spatial Data • Cached tiles?
  11. Choose Storage - Block Storage • Local or Block Storage

    • Local • Non Shared • Not Robust • Low latency • Good fit for temporary storing Logs and Audits • Cached Tiles?
  12. Choose Storage - Blob Storage • Blob Storage • Max

    scalability • Elastic • Cheap • High Latency • Shared • Robust • Cached Tiles?
  13. Cloud Migration Recap • Go for compute oriented instances for

    GeoServer • Choose migration strategy • Lift and Shift vs Refactor • Consider using PaaS services • Choose between managed and non-managed services • Pick the right Storage for the purpose and the needs of your project
  14. Helm • Package Manager for Kubernetes • Cloud Native Computing

    Foundation project • Helps manage Kubernetes applications • deployment, lifecycle, updates and upgrades • Deployment unit is called a Chart • logical grouping of manifests / resources to be deployed • Charts can easily be published, versioned and shared • Provide templating for your Manifests
  15. Available Resources • There are Docker Images for GeoServer available

    • docker pull geosolutionsit/geoserver:latest • https://github.com/geosolutions-it/docker-geoserver • official project docker image coming soon • We’re actively working on a HELM Chart too will be ready soon, check out blog for updates! • Free Webinar on running GeoServer on K8s https://www.geosolutionsgroup.com/blog/devops-k8s/
  16. • Can be tricky in Cloud environments! • Dynamic and

    distributed • Instances are spinning up and down • distributed across multiple nodes • It can be hard to identify and debug problems • What can we do? Monitoring and Logging
  17. • Centralize and Aggregate • Single central location • Easy

    to navigate and filter • New nodes can spawn but also go away • You’ll need “shippers” to collect and send out logs to the central service Centralize and Aggregate
  18. • Audit files • GeoServer Monitor extension • Tracks requests

    made to GeoServer • Export them into audit logs • Collect and ingest them to create pretty Dashboards Auditing
  19. • Set up Alerts • OOM errors, Service or machine

    down, Error Rate • Use different channels depending on the severity of the problem • Page someone if immediate attention is required • Email for less severe problems and notifications • Automate the fix • Use scripts and tools to try fix the problem and not be paged at all :) • Watchdogs, Healthchecks Alerting
  20. Useful Links • GeoSolutions Website https://geosolutionsgroup.com/ • GeoServer on Kubernetes

    https://www.geosolutionsgroup.com/blog/devops-k8s/ • A DevOps perspective on GeoServer https://www.geosolutionsgroup.com/blog/devops-geoserver-moni toring-metering/ • Cloud Optimized GeoTiffs https://www.cogeo.org/ https://docs.geoserver.org/latest/en/user/community/cog/in dex.html • Helm https://helm.sh/