Elastic{ON} Tour 2018 Munich : Volkswagen

Elastic{ON} Tour Munich - February 1, 2018

Distributed Logging and Monitoring at Volkswagen using the Elastic Stack

To be an independent cloud provider and enforce additional security guidelines, Volkswagen developed their internal central logging and monitoring solution. The solution is based on Microservices implemented in Java as a log endpoint, Kafka for the processing pipeline, Elasticsearch for log storage and Kibana for visualization. In this talk we’ll explain the rationale around the decision to build our own service, the architecture and describe our experiences with the platform.

Till Gloger | Project Lead Systems Management | Volkswagen

Transcript

1. Till Gloger, Project Lead Systems Management, Volkswagen Distributed Logging and Monitoring at Volkswagen using the Elastic Stack
2. BILD Distributed Logging and Monitoring @ Volkswagen Till Gloger – Volkswagen IT
3. Volkswagen 3 Volkswagen Tiguan - Kraftstoffverbrauch kombiniert in l/100 km: 7,4 – 4,7; CO2-Emissionen kombiniert in g/km: 170 – 123; Effizienzklassen: D - A Stand: 16. Januar 2018
4. Audi – „Vorsprung durch Technik“ 4 Audi A5 Coupé - Kraftstoffverbrauch kombiniert in l/100 km: 6,7 – 4,2; CO2-Emission kombiniert in g/km: 153 – 111; Effizienzklasse: C – A+ Stand: 16. Januar 2018
5. SEAT – „TECHNOLOGY TO ENJOY“ 5 SEAT Ateca – Kraftstoffverbrauch kombiniert in l/100 km 5,4 – 5,1 ; CO₂-Emissionen kombiniert in g/km: 129 – 119; CO₂-Effizienzklasse: B Stand: 16. Januar 2018
6. ŠKODA – „Simply clever” 6 ŠKODA Kodiaq - Kraftstoffverbrauch kombiniert in l/100 km: 7,4 – 5,3; CO2-Emissionen kombiniert in g/km: 170 – 139; Effizienzklasse: C-A Stand: 16. Januar 2018
7. BENTLEY – „Driving Luxury Performance“ 7 Bentley Bentayga - Kraftstoffverbrauch kombiniert in l/100 km: 13,1 – 7,9; CO2-Emissionen kombiniert in g/km: 296 - 210; Effizienzklasse: F - B Stand: 16. Januar 2018
8. BUGATTI – „Die Kunst des Automobils“ 8 Bugatti Chiron – Kraftstoffverbrauch kombiniert in l/100 km: 22,5; CO2-Emissionen kombiniert in g/km: 516 - 210; Effizienzklasse: G Stand: 16. Januar 2018
9. Lamborghini – „Der ultimative Sportwagen“ 9 Lamborghini Aventador LP 750-4 Superveloce - Kraftstoffverbrauch kombiniert in l/100 km: 16,0; CO2-Emissionen kombiniert in g/km: 370; Effizienzklasse: G Stand: 16. Januar 2018
10. Porsche 10 Porsche Panamera Turbo - Kraftstoffverbrauch kombiniert 9,4 – 9,3 l/100 km; CO2-Emissionen 214 – 212 g/km; Effizienzklasse: G Stand: 16. Januar 2018
11. Volkswagen Nutzfahrzeuge – „Souveräne Mobilität “ 11 Volkswagen Nutzfahrzeuge Crafter Stand: 16. Januar 2018
12. Scania 12 Scania R 730 6x2 Highline Stand: 16. Januar 2018
13. MAN – „Engineering the Future - since 1758. “ 13 MAN Neoplan Tourliner Stand: 16. Januar 2018
14. Ducati 14 Ducati SuperSport Ducati SuperSport Stand: 16. Januar 2018
15. Brands The Group comprises twelve brands from seven European countries: Each brand has its own character and operates as an independent entity on the market. 15 Stand: 16. Januar 2018
16. 16 Production 10,405,000 vehicles Key Figures of the Volkswagen Group 2016 Production per working day Ø 43,000 vehicles Finance Sales revenue: €217.3bn Profit after tax: €5.4bn Stand: 16. Januar 2018
17. Key Figures of the Volkswagen Group 2016 Models approx. 330 Production plants 120 world wide 17 Employees 626,715 Stand: 16. Januar 2018
18. Our Motivation – the digitalization in the automotive industry 18 Stand: 16. Januar 2018 § digitalization in cars § mobile online services § rising pressure from software companies § High amount of services and devices § Time-To-Market requirements for Software-development from businesses § massive increase of data § data as business
19. Volkswagen Group IT Cloud – enabling digitalization The VW Group IT Cloud Program aims to be 1. Brand and location independent. 2. Based on Open Source Software and Commodity-Hardware. 3. A highly scalable platform for applications. 4. Global infrastructure extension across locations using hybrid/multi cloud approach. Available within the Volkswagen Group. Developed using agile methodologies. 19 Stand: 16. Januar 2018
20. To support the applications common services will be used and developed based on general principles. 20 Stand: 16. Januar 2018
21. A cornerstone for applications within the cloud is a central monitoring and logging solution Application Hardware Cloud infrastructure layer Linux based OS VM unstruct ured PaaS Structured PaaS Platform as a Service (PaaS) Infrastructure as a Service (IaaS) ‘cloud infrastructure‘ monitoring Data center local solution Monitoring & Logging Solution Cloud Layer Architecture component MLS Alarm Integration in existing solution Alarm Integration in existing solution 21 Stand: 16. Januar 2018
22. Since no available solutions could fit the requirements we decided to build our own solution We needed • a solution with a uniform behavior independent of the used cloud provider (private or public) • to support strict security and data protection requirements • to operate the solution on all major cloud platforms (Openstack, AWS, Azure, …) • To scale the solution horizontally 22 Stand: 16. Januar 2018
23. We first created an architecture vision, at this point it still looks easy to achieve OS-level-metrics App-metrics CF-metrics CF-logs … OS-logs App-logs … metrics pipeline logs pipeline Alerting metrics export logs export metrics logs Existing alerting solution M&L as a Service UI for Diagnostics 23 Stand: 16. Januar 2018 23
24. Today it already looks a little bit more complex, but is running successfully in production 24 Implemented Service (examples): • Endpoint • Authentication • Field Extraction • OSB API • Full self service API for our customers • … Stand: 16. Januar 2018
25. We provide support for shipping of logs and metrics, central processing, analytics and retrieving 25 Stand: 16. Januar 2018 Elasticsearch Custom UI X-pack Kibana X-pack Kibana Instances Master Nodes Ingest Nodes Data Nodes - User Management - Self Service API - User documentation Apache Kafka Datastore Sensors Web APIs … Project VM Log Files Metrics Project VM Log Files Metrics https API Messaging Pipeline Data processing solution: - Message authentication - Content enrichment - Project assignment - … Config DB
26. • Productive since July/2016 • 40 projects are using the solution in over 150 tenants • Average throughput (November): 70.000 entries / 100 MB per min • Maximum Throughput (current sizing): 250.000 entries / min Current Status – key figures 26 Stand: 16. Januar 2018
27. We’ve learned a lot on the way • you need specialist knowledge to use emerging technologies • Implement a regular update of your components • best of breed open source is key • be aware of noisy neighbors • in the cloud everything will fail, be prepared • observability is key – also of your own services • automate everything – from the beginning! • provide an API for self service from the beginning • authenticating every request is hard to implement efficiently • use PixieDust 27 Stand: 16. Januar 2018
28. Thank you for your attention!