GABC2018: Build Pipeline as a Service on Kubernetes by Sandro Köchli & Thomas Hasler

Do you run a build pipeline on prem? How much time does it take you to manage it? Have you ever wondered if you could get rid of the hassle of lifecycle managing and patching your build pipeline? We at die Mobiliar did! Together with Adfinis SyGroup AG and Linkyard, we deployed the whole build pipeline into the public Azure cloud as a proof of concept. Even more interesting: on a Kubernetes cluster! And as a result, we got a “Build Pipeline as a Service”. How did we achieve this? Come and visit our session! We are glad to explain our setup, the inner workings and the management of the build pipeline!

Transcript

1. Build Pipeline as a Service Sandro Köchli (Adfinis SyGroup AG) Thomas Hasler (die Mobiliar)
2. Abstract Build Pipeline as a Service on Kubernetes Do you run a build pipeline on prem? How much time does it take you to manage it? Have you ever wondered if you could get rid of the hassle of lifecycle managing and patching your build pipeline? We at die Mobiliar did! Together with Adfinis SyGroup AG and Linkyard, we deployed the whole build pipeline into the public Azure cloud as a proof of concept. Even more interesting: on a Kubernetes cluster! And as a result, we got a “Build Pipeline as a Service”. How did we achieve this? Come and visit our session! We are glad to explain our setup, the inner workings and the management of the build pipeline! 12.02.2018 Build Pipeline as a Service 2
3. Thomas Hasler 12.02.2018 Build Pipeline as a Service 3 Thomas Hasler Technology Architect Delivery Platform Team thomas.hasler1@mobiliar.ch https://www.xing.com/profile/Thomas_Hasler9/ https://www.linkedin.com/in/thomas-hasler-5bab0a12b/ @codeheap
4. Sandro Köchli 12.02.2018 Build Pipeline as a Service 4 Sandro Köchli Board Member & Senior Solution Architect sandro.koechli@adfinis-sygroup.ch Phone: +41 61 500 31 32Mobile: +41 76 491 37 01 https://www.xing.com/profile/Sandro_Koechli https://ch.linkedin.com/in/sandro-köchli-a01a792
5. About Adfinis SyGroup AG 12.02.2018 Build Pipeline as a Service 5 Your trusted partner for tailor-made IT services Be Smart. Think open source.
6. About Adfinis SyGroup AG 12.02.2018 Build Pipeline as a Service 6 We maintain strategic partnerships with leading providers of software, cloud and business solutions.
7. About Adfinis SyGroup AG 12.02.2018 Build Pipeline as a Service 7 Engineering Managed Services DevOps Development • SLA – 24/7/365 • Monitoring • Maintenance • Support • Clustering / HA • System Mgmt. • Database • Config Mgmt. • Automation • Container • PaaS • IaaS • 3rd / 4th Support • Customizing • Individual • Optimization
14. Current CI/CD Pipeline@Mobiliar 12.02.2018 Build Pipeline as a Service 14
15. Current Situation • The CI / CD Pipeline for JEE / Spring Boot / Angular Apps consists of a set of tools, which are all installed on prem. • For the deployments we built our own deployment tool called “Liima” (open sourced) • For monitoring the pipelines of all apps we built our own tool called “Mobitor” (not yet OSS) • The deployments are made on our internal K8S clusters (DEV / INT / PROD) • Most apps are deployed as docker containers - some DevOps teams started to use HELM Charts 12.02.2018 15 Build Pipeline as a Service
16. Complications • Lifecycle of all the tools is a full time job for at least two persons • Scaling is really hard, because our internal processes for maintaining infrastructure are “slow” (not lean at all) • The CI part of a build (until all automated tests are done) can go up to 1.5 hours (scaling problem) • The team is stuck into daily business instead of evolving the pipeline to the next level 12.02.2018 16 Build Pipeline as a Service
17. Key Questions • Is it possible to have this CI Pipeline as a managed service out of the cloud? YES • Could it be made on top of azure ? YES • Can we deploy the artifacts back on our internal INT / PROD clusters? YES 12.02.2018 17 Build Pipeline as a Service
18. New Approach: CI/CD Pipeline on Azure • CI/CD Tooling (Teamcity, Nexus, Sonarqube) deployed to the Azure cloud using Helm • Linkyard Atlassian Cloud (Jira, Confluence, BitBucket, KeyCloak, Addons) deployed using Helm on Azure • Azure Services: − ACS (PoC) − AKS (Pilot) − Site-to-Site VPN − Azure AD 12.02.2018 18 Build Pipeline as a Service
19. Why Azure? • The preferred Cloud Vendor of Mobi is Azure • K8s is a well known citizen on Azure • Services like AKS make it easy to deploy Kubernetes Cluster • LifeCycle Management is simplified with the Azure Tooling • o365 already deployed inside Mobiliar. Azure AD as identity provider exists already • Microsoft will have a Swiss Region by 2019 • Microsoft is one of the largest Kubernetes contributors and employs the Lead Kubernetes Engineer (Brendan Burns) • Kubernetes is a first class citizen on Microsoft Azure - fully managed Kubernetes cluster at the tip of your fingers 12.02.2018 19 Build Pipeline as a Service
20. Azure Container Services (AKS) • Layered support of Kubernetes • Docker tooling and API Support • Integrated app management and scaling • Linux and Windows Server Containers • Public and private cloud 12.02.2018 20 Build Pipeline as a Service
21. Atlassian as a Service (linkyard.cloud) • SaaS Solution • Fully managed Atlassian Products (Jira, Confluence, Bitbucket, Bamboo) • Server-variants of Atlassian (all Add-ons) • Great performance • Swiss company, servers in Switzerland or the EU (Azure) • Everything running on Kubernetes with Docker • Mix with on-prem is possible (Build-Agents, Deployment, ...) • Containers deployed using Helm Charts • Lifecycle Management and Integration Support • Technologie Partner Adfinis SyGroup 12.02.2018 21 Build Pipeline as a Service
22. Managed Pipeline Tooling • Managed Teamcity and Agents • Managed Sonarqube • Managed Nexus • Managed Pipelines for Java, node.js, etc 12.02.2018 22 Build Pipeline as a Service
23. Helm! • Helm is the official package manager for Kubernetes • Microsoft is a big contributor of Helm • CI/CD pipeline is managed and deployed with Helm • Deployments can be done with or without Helm (optional) 12.02.2018 23 Build Pipeline as a Service
24. 24 12.02.2018 Build Pipeline as a Service New Approach: CI/CD Pipeline on Azure
25. Challenges • Integration in existing environment • Automated horizontal scaling − Based on Prometheus − Container on Kubernetes (Build Agents Count) − Kubernetes Nodes (Cluster Size) • User Acceptance to accelerate adoption 12.02.2018 25 Build Pipeline as a Service
26. Advantages / Disadvantages compared to other offerings Advantages • The complete pipeline can be consumed as a managed cloud services, no exceptions • The complete toolchain can be delivered as is - no changes needed • Teamcity can be used either from the existing on prem environment or from the new managed build pipeline Disadvantages • We will find out in the future….. 12.02.2018 26 Build Pipeline as a Service