From enterprise-ish to serverless — Insights and practices from our connected vehicle journey

The way of building applications and orchestrating them to larger distributed systems and service-oriented solutions has changed continuously. Several abstractions emerged to ease application development and the usage of infrastructure. Originally, the cloud and on-demand computing journey started with basic pay-per-use infrastructure services like virtual servers, storage and network. Today, more powerful and developer-friendly abstractions are available. Examples include managed execution environments (like PaaS, FaaS and container runtimes), storage and communication services (like managed databases, event brokers and API gateways), service meshes and machine learning platforms.

These abstractions hide most of the underlying infrastructure and servers, so building distributed systems becomes increasingly "serverless" and focuses on delivering business value. However, developing and operating serverless systems at scale requires pretty different approaches to succeed. This is especially challenging for organizations with a longer history of enterprise-ish infrastructure management and bureaucratic interactions between development, operations and other parties. Besides the required technical changes, structural and mindset transformations are often hard to get into the organization.

This talk discusses several aspects like moving from enterprise silos (development, test, security, compliance, operations, etc.) to holistic software ownership by empowered teams who truly practice continuous delivery. Another example for transitioning from enterprise-ish to serverless is moving from sophisticated and time-consuming staging to continuous deployment and testing in production.

Takeaways from this talk:
1) Lessons learned from our connected vehicle journey: moving from enterprise-ish development and operations to building cloud native microservices and (mostly) serverless systems
2) When minor cosmetic organizational changes are not enough: essential transformations in mindset and structure to achieve technical excellence, speed, quality and fast feedback in a serverless world
3) How to utilize continuous delivery as the business-critical driver to scale solutions (not only serverless) and organizations both economically and technically

Transcript

1. From enterprise-ish to serverless Insights and practices from our connected vehicle journey Johannes Wettinger // @johannesw7r
2. Johannes Wettinger Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 3 System Architect – former Research Scientist / Tech Lead / Software Engineer Driving architecture, dev + ops excellence and organizational change to deliver VALUE: Software + hardware, UX / DX / APIs and services. Strategy / business / IT translator. Transparency creator. Dismantling ivory towers. @johannesw7r
3. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 4 Bosch Group 60 Countries 402 166 Bosch associates
4. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 5 Connected Mobility Today Vehicle as the central object Future Connected, intermodal transportation services v
5. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 6 Building systems and solutions, simplified by emerging abstractions Usage of IT infrastructure Application development Orchestration to larger distributed systems Providing service-oriented solutions
6. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 7 Cloud & on-demand journey Basic pay-per-use infrastructure: Virtual servers, storage, network Powerful and developer-friendly abstractions: Managed execution environments (like PaaS, FaaS, container runtimes) IoT services Storage, communication, API services (like managed DBs, BaaS, event brokers, API gateways) Machine learning platforms Service meshes …
7. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 8 IT infrastructure & platforms become commodity and standard
8. Infrastructure and servers (mostly) invisible Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 9 Building (distributed) systems becomes increasingly “serverless” Focus on delivery of business value Improved developer experience (not always – just saying “FaaS debugging”) “Failure doesn’t respect abstraction” architectelevator.com/architecture/failure-doesnt-respect-abstraction
9. Less servers – less challenges? Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 10 If no greenfield: Several organizational (and technical) challenges!  Organization with a longer history in non-SaaS business  From enterprise-ish infrastructure management to cloud native & serverless platforms  Bureaucratic interactions between development, operations and other parties  …
10. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 11 Our connected vehicle journey Insights (Anti-)Principles Practices
11. Connected Vehicle SaaS + hardware Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 12 Logbook / Vehicle Health / Driving Score / Accident Notification / Fleet Tracking / Predictive Diagnostics / OTA / … Monolith Portal Solution Servers in Enterprise IT Issues: Performance & Scaling Changes & New Features Mobile Readiness Multi-Tenancy APIs
12. Connected Vehicle SaaS + hardware Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 13 Logbook / Vehicle Health / Driving Score / Accident Notification / Fleet Tracking / Predictive Diagnostics / OTA / … (Mostly) serverless // multi-cloud managed services (DBs, message brokers, …) Cloud native microservices // 12-factor apps (instead of VMs) Continuous integration and delivery Responsive web application RESTful APIs and messaging
13. Architecture sketch Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 14 Device Services Base Services Use Case Services Frontend Single Page App Device Connectivity Device Operations Device Abstraction … … … Shadow Vehicle User Management Billing … Vehicle Health Fleet Tracking Logbook … … E2E Integration & Testing Device Simulators Load Generator … API Gateway AWS + Bosch IoT Cloud Basic Device Premium Device
14. From enterprise-ish to serverless Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 15 Not a greenfield approach  Highly distributed enterprise-ish organization  Starting (partly painful) transformation from there  Migration of existing customers Grown org structures & established key positions  “We always did it this way [in a different context] and were successful” [in a different context!]  Example: IT service management
15. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 16 1. Don’t split the two worlds, but play well together
16. 1. Don’t split the two worlds, but play well together Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 17 On-Premise Enterprise IT: “Layer silos”, no pay-per-use, no self-service, sophisticated firewalling, limited scaling, ... Private / Hybrid / Public Cloud: Decoupled org and infrastructure (master data, billing ...), cloud & serverless benefits, DevOps, …
17. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 18 2. Do your homework: Essential changes in mindset and structure Move from enterprise-ish silos to software ownership by empowered teams.
18. Software Ownership Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 19 Charity Majors
19. 2. Essential changes in mindset and structure Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 20 Relentless explaining to entire org – Ensure management buy-in // 1:1s, blogs, talks, …  Speed + Quality + Cost reduction  Continuous delivery, technical excellence, … People and politics – It’s never only about facts  New skills (not only technical!) – for example, QA needs deep understanding of continuous delivery  Less people in certain areas like QA – central department leads “loose” head count
20. 2. Essential changes in mindset and structure Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 21 Processes and roles  Needed at scale. But keep them slim, consistent and appropriate.  Beware of overengineering Stay open for continuous change – Experiment and measure  Build upon (but don’t blindly copy!) best practices // SRE, devopstopologies.com, …  Don’t make temporary enablers permanent – “Automate yourself out of the job”  Don’t separate DevOps from service management
21. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 22 3. Keep the “right” things you already have
22. 3. Keep the “right” things you already have Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 23 Resist the temptation “we are going serverless now, so everything is totally different”  Incident and change management not obsolete, but different – now driven by continuous delivery principles  System architecture  Interface and format definitions
23. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 24 4. Nice-to-have in the past, essential in the future
24. 4. Nice-to-have in the past, essential in the future Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 25  Automation (test, security, compliance, deployment, …)  Common API guidelines  Strict API evolution  Observability and monitoring  … Business-critical for economic reasons (cost, scale) – not just technical details Don’t have them yet? – LAST CALL!
25. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 26 5. Continuous delivery is the business-critical driver to scale
26. 5. Continuous Delivery (CD) is the business-critical driver to scale Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 27 … to scale solutions and organizations, both economically and technically. Invest in automation to trust the delivery process  Every change must pass the delivery pipeline  Even the smallest config change
27. 5. Continuous Delivery (CD) is the business-critical driver to scale Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 29 Treat your CD machinery & infrastructure as business-critical system  You can’t update or deploy to production otherwise  Separate test & QA teams are gone (or reduced to minimum)  Manual compliance checks and acceptance tests are automated in the pipeline  Don’t overdo “freezing” – underlying (cloud) layers cannot be frozen either  Mindset challenge: test against moving target  Move from time-consuming and expensive staging to continuous deployment and testing in production
28. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 30 6. Establish internal providers for managed base services
29. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 31 Entering the world of devices and hardware …
30. A glimpse at the world of devices Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 32  Limited hardware resources  Different platforms and tools  Chain of devices: connectivity  sensors  vehicle components …  Safety!  Lifecycle services: mount/replace/unmount devices in cars  Logistics, legal, …
31. A glimpse at the world of devices Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 33  Org structure: device + backend feature teams?  SaaS operations != device operations  Continuous deployment and feature toggling for devices in the field?  …  Backend + devices: In-vehicle software as Docker containers // github.com/eclipse/kuksa.invehicle
32. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 34 7. Devices are no excuse to skip test automation
33. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 35 8. Design and build for system observability
34. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 36 9. Distinguish non-functional requirements by system interfaces
35. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 37 Bad Practices Sales-driven architecture Sell on-prem variant of SaaS solution Allow SaaS customers deployment veto Politics-driven architecture and isolated platform development
36. Three more helpers …
37. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 39 1. Technology Radars 2. Code project blueprints and templates for microservices, functions, pipelines, … 3. Enough documentation and automation for seamless onboarding
38. Connected Mobility Solutions | Dr. Johannes Wettinger | @johannesw7r | 2019 © Robert Bosch GmbH 2019. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights. 40 Play well together Johannes Wettinger @johannesw7r CHANGE mindset & structure Keep the “right” things Essential in future Continuous Delivery Internal providers Device test automation System observability end-to-end NFR != NFR Bad Practices