Deploying and operating GeoServer: a DevOps perspective - FOSS4G 2021 Cut

In this presentation we will share with you the lessons we have learned at GeoSolutions when deploying and operating GeoServer as well as some common patterns for the migration of on premise GeoServer clusters to the cloud. We'll share with you tips on how to:
- best practices to migrate your existing GeoServer cluster to the cloud
- insights on your geoserver cluster using centralized logging and Monitor plugin
- avoid common bottlenecks to best set up a distributed scalable GeoServer cluster
- work containers and container orchestrators like Kubernetes

Transcript

1. Alessandro Parma Luca Pasquali GeoSolutions Deploying and operating GeoServer: a DevOps perspective
2. GeoSolutions • Offices in Italy & US, Worldwide clients • 30+ collaborators, 25+ Engineers • Our products • Our Offer Enterprise Support Services Deployment Subscription Professional Training Customized Solutions GeoNode
3. Affiliations We strongly support Open Source, it Is in our core We actively participate in OGC working groups and get funded to advance new open standards We support standards critical to GEOINT
4. Quick Intro
5. Agenda • Cloud Computing • Two words definition • Why is relevant? • Moving to the Cloud • Avoid common bottlenecks • Distributed, Scalable GeoServer cluster • Containers, Orchestrators and Kubernetes • Monitoring and Logging • Debugging, Challenges and Solutions • Gain Insights from your Cluster
6. Cloud Computing
7. What is Cloud Computing • Computing services over the internet • Servers, Storage, Databases, Networking, Software, ... • Pros and cons • Elasticity and Scalability • Reduced time to market • Business flexibility • Security and Privacy • Lower Costs?
8. Deployment Models • Public Cloud • Most widely used • Cost Effective • Private Cloud • Dedicated Cloud infrastructure to an organization • Secure env for sensitive tasks • Governments, schools, agencies, … • Costly • Hybrid Cloud • combination of public cloud infrastructure with a private environment
9. Moving GeoServer to Cloud
10. Moving GeoServer to che cloud • Rehost or “Lift & Shift” • IaaS approach • Redeploy existing data and applications on the cloud infrastructure • “Quick and Easy” • Not leveraging all services offered by the provider • Refactor or “lift, tinker, and shift” • Tweak and optimize applications for the cloud • PaaS approach • Same core architecture but leveraging services offered by the provider
11. Moving GeoServer to che cloud • Revise, Rebuild, Replace • More changes to the architecture • Code Changes, Rewriting, adoption of third-party and prebuilt applications • Enables applications to take advantage of the services offered • Requires foreplanning and Knowledge
12. • Set up VMs • Use computing resources of the provider • Choose the right kind of VM with good compute performance • Rule of thumb: 4 Cores and 4GB per instance • Redeploy the cluster • Migrate Instances • Configuration and data • Done Rehost your GeoServer Cluster
13. Refactor your Cluster • PaaS Approaches • Setup your VMs or use a platform • Managed DBMS Service • Storage Options • Choose the right kind for each component (datadir, data files, cache, ..) • Object vs File vs Block Storage • GeoServer supports COGs! (Community Module) • You can store Cached Tiles in Object Storage
14. K8s Example
15. Choose Storage - File Share • File Share • File Storage ~ NFS • Shared (locally) • Scales well • Typical use case is GeoServer datadirs • Spatial Data • Cached tiles?
16. Choose Storage - Block Storage • Local or Block Storage • Local • Non Shared • Not Robust • Low latency • Good fit for temporary storing Logs and Audits • Cached Tiles?
17. Choose Storage - Blob Storage • Blob Storage • Max scalability • Elastic • Cheap • High Latency • Shared • Robust • Cached Tiles?
18. Cloud Migration Recap • Go for compute oriented instances for GeoServer • Choose migration strategy • Lift and Shift vs Refactor • Consider using PaaS services • Choose between managed and non-managed services • Pick the right Storage for the purpose and the needs of your project
19. Helm • Package Manager for Kubernetes • Cloud Native Computing Foundation project • Helps manage Kubernetes applications • deployment, lifecycle, updates and upgrades • Deployment unit is called a Chart • logical grouping of manifests / resources to be deployed • Charts can easily be published, versioned and shared • Provide templating for your Manifests
20. Available Resources • There are Docker Images for GeoServer available • docker pull geosolutionsit/geoserver:latest • https://github.com/geosolutions-it/docker-geoserver • official project docker image coming soon • We’re actively working on a HELM Chart too will be ready soon, check out blog for updates! • Free Webinar on running GeoServer on K8s https://www.geosolutionsgroup.com/blog/devops-k8s/
21. Monitoring and Logging
22. • Can be tricky in Cloud environments! • Dynamic and distributed • Instances are spinning up and down • distributed across multiple nodes • It can be hard to identify and debug problems • What can we do? Monitoring and Logging
23. • Centralize and Aggregate • Single central location • Easy to navigate and filter • New nodes can spawn but also go away • You’ll need “shippers” to collect and send out logs to the central service Centralize and Aggregate
24. • Collect Metrics • Resp Time, • Throughput, • Uptime, • Error Rate • ... Collect Metrics
25. • Audit files • GeoServer Monitor extension • Tracks requests made to GeoServer • Export them into audit logs • Collect and ingest them to create pretty Dashboards Auditing
26. • Audit Event Auditing
27. Analyze performance of your Cluster
28. Analyze performance of your Cluster
29. • Set up Alerts • OOM errors, Service or machine down, Error Rate • Use different channels depending on the severity of the problem • Page someone if immediate attention is required • Email for less severe problems and notifications • Automate the fix • Use scripts and tools to try fix the problem and not be paged at all :) • Watchdogs, Healthchecks Alerting
30. Useful Links • GeoSolutions Website https://geosolutionsgroup.com/ • GeoServer on Kubernetes https://www.geosolutionsgroup.com/blog/devops-k8s/ • A DevOps perspective on GeoServer https://www.geosolutionsgroup.com/blog/devops-geoserver-moni toring-metering/ • Cloud Optimized GeoTiffs https://www.cogeo.org/ https://docs.geoserver.org/latest/en/user/community/cog/in dex.html • Helm https://helm.sh/
31. The End Questions? alessandro.parma@geosolutionsgroup.com luca.pasquali@geosolutionsgroup.com info@geosolutionsgroup.com