Save 37% off PRO during our Black Friday Sale! »

Deploying and operating GeoServer: a DevOps perspective - FOSS4G 2021 Cut

Deploying and operating GeoServer: a DevOps perspective - FOSS4G 2021 Cut

In this presentation we will share with you the lessons we have learned at GeoSolutions when deploying and operating GeoServer as well as some common patterns for the migration of on premise GeoServer clusters to the cloud. We'll share with you tips on how to:
- best practices to migrate your existing GeoServer cluster to the cloud
- insights on your geoserver cluster using centralized logging and Monitor plugin
- avoid common bottlenecks to best set up a distributed scalable GeoServer cluster
- work containers and container orchestrators like Kubernetes

Bac74c17d65c22d0ae63915251f7750f?s=128

Simone Giannecchini
PRO

October 04, 2021
Tweet

Transcript

  1. Alessandro Parma Luca Pasquali GeoSolutions Deploying and operating GeoServer: a

    DevOps perspective
  2. GeoSolutions • Offices in Italy & US, Worldwide clients •

    30+ collaborators, 25+ Engineers • Our products • Our Offer Enterprise Support Services Deployment Subscription Professional Training Customized Solutions GeoNode
  3. Affiliations We strongly support Open Source, it Is in our

    core We actively participate in OGC working groups and get funded to advance new open standards We support standards critical to GEOINT
  4. Quick Intro

  5. Agenda • Cloud Computing • Two words definition • Why

    is relevant? • Moving to the Cloud • Avoid common bottlenecks • Distributed, Scalable GeoServer cluster • Containers, Orchestrators and Kubernetes • Monitoring and Logging • Debugging, Challenges and Solutions • Gain Insights from your Cluster
  6. Cloud Computing

  7. What is Cloud Computing • Computing services over the internet

    • Servers, Storage, Databases, Networking, Software, ... • Pros and cons • Elasticity and Scalability • Reduced time to market • Business flexibility • Security and Privacy • Lower Costs?
  8. Deployment Models • Public Cloud • Most widely used •

    Cost Effective • Private Cloud • Dedicated Cloud infrastructure to an organization • Secure env for sensitive tasks • Governments, schools, agencies, … • Costly • Hybrid Cloud • combination of public cloud infrastructure with a private environment
  9. Moving GeoServer to Cloud

  10. Moving GeoServer to che cloud • Rehost or “Lift &

    Shift” • IaaS approach • Redeploy existing data and applications on the cloud infrastructure • “Quick and Easy” • Not leveraging all services offered by the provider • Refactor or “lift, tinker, and shift” • Tweak and optimize applications for the cloud • PaaS approach • Same core architecture but leveraging services offered by the provider
  11. Moving GeoServer to che cloud • Revise, Rebuild, Replace •

    More changes to the architecture • Code Changes, Rewriting, adoption of third-party and prebuilt applications • Enables applications to take advantage of the services offered • Requires foreplanning and Knowledge
  12. • Set up VMs • Use computing resources of the

    provider • Choose the right kind of VM with good compute performance • Rule of thumb: 4 Cores and 4GB per instance • Redeploy the cluster • Migrate Instances • Configuration and data • Done Rehost your GeoServer Cluster
  13. Refactor your Cluster • PaaS Approaches • Setup your VMs

    or use a platform • Managed DBMS Service • Storage Options • Choose the right kind for each component (datadir, data files, cache, ..) • Object vs File vs Block Storage • GeoServer supports COGs! (Community Module) • You can store Cached Tiles in Object Storage
  14. K8s Example

  15. Choose Storage - File Share • File Share • File

    Storage ~ NFS • Shared (locally) • Scales well • Typical use case is GeoServer datadirs • Spatial Data • Cached tiles?
  16. Choose Storage - Block Storage • Local or Block Storage

    • Local • Non Shared • Not Robust • Low latency • Good fit for temporary storing Logs and Audits • Cached Tiles?
  17. Choose Storage - Blob Storage • Blob Storage • Max

    scalability • Elastic • Cheap • High Latency • Shared • Robust • Cached Tiles?
  18. Cloud Migration Recap • Go for compute oriented instances for

    GeoServer • Choose migration strategy • Lift and Shift vs Refactor • Consider using PaaS services • Choose between managed and non-managed services • Pick the right Storage for the purpose and the needs of your project
  19. Helm • Package Manager for Kubernetes • Cloud Native Computing

    Foundation project • Helps manage Kubernetes applications • deployment, lifecycle, updates and upgrades • Deployment unit is called a Chart • logical grouping of manifests / resources to be deployed • Charts can easily be published, versioned and shared • Provide templating for your Manifests
  20. Available Resources • There are Docker Images for GeoServer available

    • docker pull geosolutionsit/geoserver:latest • https://github.com/geosolutions-it/docker-geoserver • official project docker image coming soon • We’re actively working on a HELM Chart too will be ready soon, check out blog for updates! • Free Webinar on running GeoServer on K8s https://www.geosolutionsgroup.com/blog/devops-k8s/
  21. Monitoring and Logging

  22. • Can be tricky in Cloud environments! • Dynamic and

    distributed • Instances are spinning up and down • distributed across multiple nodes • It can be hard to identify and debug problems • What can we do? Monitoring and Logging
  23. • Centralize and Aggregate • Single central location • Easy

    to navigate and filter • New nodes can spawn but also go away • You’ll need “shippers” to collect and send out logs to the central service Centralize and Aggregate
  24. • Collect Metrics • Resp Time, • Throughput, • Uptime,

    • Error Rate • ... Collect Metrics
  25. • Audit files • GeoServer Monitor extension • Tracks requests

    made to GeoServer • Export them into audit logs • Collect and ingest them to create pretty Dashboards Auditing
  26. • Audit Event Auditing

  27. Analyze performance of your Cluster

  28. Analyze performance of your Cluster

  29. • Set up Alerts • OOM errors, Service or machine

    down, Error Rate • Use different channels depending on the severity of the problem • Page someone if immediate attention is required • Email for less severe problems and notifications • Automate the fix • Use scripts and tools to try fix the problem and not be paged at all :) • Watchdogs, Healthchecks Alerting
  30. Useful Links • GeoSolutions Website https://geosolutionsgroup.com/ • GeoServer on Kubernetes

    https://www.geosolutionsgroup.com/blog/devops-k8s/ • A DevOps perspective on GeoServer https://www.geosolutionsgroup.com/blog/devops-geoserver-moni toring-metering/ • Cloud Optimized GeoTiffs https://www.cogeo.org/ https://docs.geoserver.org/latest/en/user/community/cog/in dex.html • Helm https://helm.sh/
  31. The End Questions? alessandro.parma@geosolutionsgroup.com luca.pasquali@geosolutionsgroup.com info@geosolutionsgroup.com