Elastic Meetup - Central Logging in OpenShift 3

Slides from my talk on the 11th Elastic Meetup in Zurich. Contains background information about central logging service in OpenShift 3 and therefore APPUiO.

Transcript

1. Elasticsearch in OpenShift 3 How central logging in a containerized world looks like
2. 08/29/16 VSHN AG | http://vshn.ch 2 Agenda • VSHN? Or: How do I spell that? • Short intro to OpenShift • Central Logging in OpenShift • Security of Logs • Cluster Maintenance • Custom Application Logging
3. 08/29/16 VSHN AG | http://vshn.ch 3 /me • Tobias Brunner aka tobru • VSHNeer since the beginning of VSHN • Open Source fan • Blog: https://tobrunet.ch • Twitter: @tobruzh • Interested in OpenShift: – http://sysadvent.blogspot.ch/2015/12/day-13-introduct ion-to-openshift-3.html – https://speakerdeck.com/tobru/opensource-paas
4. 08/29/16 VSHN AG | http://vshn.ch 4 Company Profile | ˈvɪʒn̩ | • Owner-operated Swiss company • 15 employees at the head office in Zurich • Service provider for DevOps, software delivery automation and configuration management • Partner for operations/hosting of web applications • Further specialty fields: Consulting, System Engineering, Continuous Delivery, Monitoring, Backup, 24/7 Support
5. 08/29/2016 VSHN AG | http://vshn.ch 5 Software Delivery Automation
6. 08/29/16 VSHN AG | http://vshn.ch 6 What is OpenShift? • Container Platform • Built on top of Docker and Kubernetes • Completely overhauled in version 3, released in 2015 • Three flavours: – Origin: Open Source, Community supported – Container Platform (formerly Enterprise): Commercial, supported by Red Hat – Online 3: Public Service – Currently in Dev Preview • APPUiO – Swiss Container Platform
7. 08/29/16 VSHN AG | http://vshn.ch 7 What is OpenShift?
8. 08/29/16 VSHN AG | http://vshn.ch 8 Central Logging in OpenShift 3 • EFK Stack: – Elasticsearch (indexing) – Fluentd (shipping) – Kibana (displaying) • Runs as part of the Kubernetes Cluster • Currently runs Elasticsearch 1.5.2 – Upgrade plans to 2.3.x • Integrated in OpenShift Web Console • Completely automated setup and configuration with logging deployer
9. 08/29/16 VSHN AG | http://vshn.ch 9 Central Logging in OpenShift 3
10. 08/29/16 VSHN AG | http://vshn.ch 10 Central Logging in OpenShift 3
11. 08/29/16 VSHN AG | http://vshn.ch 11 Central Logging in OpenShift 3 • Collects logs from applications and optionally from system components into two ES clusters: – Application logs (Main) – System logs (Ops) • Each node runs a fluentd agent – Enriches logs with metadata – Ships logs to Elasticsearch – Fluentd knows Kubernetes Metadata
12. 08/29/16 VSHN AG | http://vshn.ch 12 Central Logging in OpenShift 3 • Elasticsearch Plugins: – Floragunn Search Guard – Fabric8 Elasticsearch OpenShift Plugin • Dynamically update the SearchGuard ACL based on a user's name • Transform kibana index requests to support multitenant deployments • Seed the Searchguard index config, roles, rolesmapping, and actiongroups types – Fabric8 Kubernetes Cloud Plugin • The Kubernetes Cloud plugin allows to use Kubernetes API for the unicast discovery mechanism
13. 08/29/16 VSHN AG | http://vshn.ch 13 Security: Elasticsearch • Everything shipped by TLS secure connection – Authentication with certificate • Per OpenShift project access – Index per project: {project_name}.{project_uuid}.YYYY.MM.DD – Index security: Floragunn Search Guard
14. 08/29/16 VSHN AG | http://vshn.ch 14 Security: Elasticsearch • Floragunn Search Guard – Plugin for Elasticsearch – Adds SSL authentication and TLS enforcement – Controlls access to Elasticsearch cluster – Authorization for accessing indexes
15. 08/29/16 VSHN AG | http://vshn.ch 15 Security: Kibana • Login using a custom OAuth2 proxy • Two containers in the Kibana Pod: – kibana-proxy – kibana • Accessed and secured through the OpenShift router • Access only to logs of pods with access rights
16. 08/29/16 VSHN AG | http://vshn.ch 16 Cluster Maintenance • Curator – Part of the deployment – Retention policy per project • Scaling – One deployment controller per ES instance – Manual intervention needed – Automatic cluster formation
17. 08/29/16 VSHN AG | http://vshn.ch 17 Custom Application Logging • Application logs written to STDOUT are captured by fluentd • Structure your application logs in fields with JSON – Automatically discovered by fluentd • Makes custom fields available in Kibana
18. 08/29/16 VSHN AG | http://vshn.ch 18 Conclusion • Nice integration of existing tools • Well thought • Still open challenges in terms of automatic scaling Central Logging done right – Using Elasticsearch, Fluentd and Kibana Join the Beta under https://appuio.ch
19. 08/29/2016 VSHN AG | http://vshn.ch 19 Questions? Discussion?