DEVOPS MEETUP LISBON HOW TO MANAGE CLOUD INFRASTRUCTURE Stefan Killian Photo by Alex Paganelli on Unsplash MAN | Digital Hub MAN Truck & Bus´s HQ

< > ŒŽ 34x + 6x + We drive transportation to the next level by creating a startup within our cooperate MAN Truck & Bus | Community Event | April 2019 | The journey of MAN Digital Hub in Lisbon 2 MAN Digital Hub: Our journey so far Jan 2018 Kick-off Nov 2017 Go! 2020 Jul 2018 Start Operations Aug 2018 New Office 100x Office Opening Oct 2018 Nov 2018 Volkswagen Press Event Today Setup, Growing, Recruiting, Scouting, Entering Scene, … We are currently setting up our Cloud Platform and CI/CD Platform Team!

Agenda 1 What do we want to achieve? 2 Team organization and Responsibilities 3 Cloud Platform and Blueprints – What do we provide? 4 Setup and Provisioning – How it is done? 5 Wrap up

WHAT DO WE WANT TO ACHIEVE AT MAN TRUCK & BUS? Photo by Andreas Brücker on Unsplash

< > ŒŽ What do we want to achieve at MAN Truck and Bus? DevOps in a controlled manner § Principles § Architecture § Security Guard Rails § Reduce common efforts - Do not reinvent the wheel (over and over again) DevEx (Developer Experience) § Enhance our Development/SoftwareDelivery Efforts § Enable our Product Teams (Onboarding, Consulting, Training, Education) Build and Run Cloud Native Application/Products in AWS § Apps which are born in the cloud! Developed and Maintained by MAN Truck & Bus. Our goals 5 Intro Note: This applies not to all application and/or IT organization of MAN Truck & Bus MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure

< > ŒŽ Architecture and Principles Principles § Cloud serverless first > Cloud fully managed > Cloud managed (> Custom-Build/Operate on EC2) § Infrastructure as Code (IaC) – every Infrastructure must be defined as IaC and must use AWS CloudFormation § CI/CD – every code artifact (also IaC) should be deployed via pipeline in Gitlab CI § Stateless Application – Apps must be stateless and follow the Twelve-Factor App methodology (Link) § Grant least privileges – give minimal amount of permissions that are required to get job done Cloud native applications only 6 What do we want to achieve at MAN Truck and Bus? MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure

TEAM ORGANIZATION AND RESPONSIBILITIES

< > ŒŽ How are we organized? Teams § Product Teams § Platform Team Shared Responsibility Model – why we need it? § An AWS Account comes with great freedom and power, but this only comes along with also a greater responsibility of operation (like high availability, backup, restore), security and many other topics § Therefore a clear understanding of the responsibility between Cloud Platform Engineering Team and the Product Team is needed MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure 8 Team organization and responsibilities

< > ŒŽ 9 Product Team´s Task and Responsibilities Team organization and responsibilities B A Managed Database Microsservices Managed Database Microsservices Product Team 1 Product Team N C D § Are building their application in cloud native approach with containers or FaaS (AWS Lambda) and using cloud service for API Gateways or managed services for data persistence or/and queuing. § Product Team develops and deploys via our CI/CD Platform § You build it, you run it, (you secure it) - Product Team is fully resposible for their product inclusive their infrastructure (Operation) § Cost Control, Monitoring and Planning of AWS spend MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure

< > ŒŽ 10 Cloud Platform Engineering Team´s Task and Responsibilities Team organization and responsibilities § Providing a Platform as a product § Cloud Infrastructure with AWS Mulit-Account Setup and Provisioning of new Accounts § CI/CD Platform – (Gitlab CI, Jfrog Artifactory, Sonarqube) § Provide and maintain a common set of blueprints, basic templates and examples § Infrastructure as Code (IaC) - Templates for queuing, persistence, etc § The Cloud Platform Engineering Team offers § Onboarding Support & Consulting § Enablement § Training § Architecture Reviews § Organize community events for collaborating and contributing to the same standards MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure Cloud Platform Engineering Team

< > ŒŽ 11 Collaboration between Product Teams and Cloud Platform Engineering Team Team organization and responsibilities B A Managed Database Microsservices Managed Database Microsservices Product Team 1 Product Team N C D Collaboration § Inner-Source with Gitlab CI § Open issues § Request new features via issues § Open merge requests § Encourage Product Team to submit code § Community Events § Microsoft Teams MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure Cloud Platform Engineering Team

CLOUD PLATFORM AND BLUEPRINTS

< > ŒŽ What do we provide? § Accounts for platform management § Multiple AWS accounts per Product Team § One AWS Account per stage § Sandbox Account for experiments § Management (Mgmt) Account for orchestration over all stages like CI/CD § Examples § Gitlab Runner with Example Pipeline for Cross- Account Deployments § Blueprints § Runtime for containerized microservices § Runtime for serverless microservices Cloud Platform and Blueprints Mgmt Prod Int Dev Sandbox deploy deploy deploy MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure Product - Account structure per bounded context 13 Master Audit IAM Shared Services Platform Account

< > ŒŽ Cloud Platform Cloud Platform Engineering´s Responsibility Cloud Platform and Blueprints Hybrid VPC Private Public AWS Customer Account AWS Shared Service Account Corporate Network Private Shared Services Firewall Private Public Private Public Internet Transit Gateway AWS IAM / Audit / Control Accounts GuardDuty Config CloudTrail S3 SES Budgets IAM IAM Roles Route53 Public Hosted Zone and Resolver for MAN internal DNS Domains Route53 MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure 14

< > ŒŽ Cloud Platform Product Team´s Responsibility Cloud Platform and Blueprints Hybrid VPC AWS Customer Account AWS Shared Service Account Corporate Network Private Shared Services Firewall Internet Transit Gateway Product Team´s Responsibility MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure 15

< > ŒŽ Private subnet Blueprint for containerized microservices Cloud Platform and Blueprints VPC AWS Account API Gateway Container Registry (ECR) Elastic Container Service (ECS) Network Load Balancing Fargate App-Container Auto Scaling group App-Container CloudFront Public subnet Simple Storage Service (S3) Database / Persistence FrontEnd (SPA) Application Integration MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure 16

SETUP AND PROVISIONING

< > ŒŽ How we manage Multi-Account AWS Environment § AWS Organizations § Organizational Units § Service Control Policies (SCP) § AWS Cloudtrail (via Organizations) § AWS Config § Account Vending Maschine - Create and Update AWS Accounts § Step Functions § Lambda § CloudFormation Automate everything! MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure 18 Setup and Provisioning { "Sid": "DenyModifyDefaultTemplates", "Effect": "Deny", "Action": [ "cloudformation:Set*", "cloudformation:Cancel*", "cloudformation:Signal*", "cloudformation:Continue*", "cloudformation:Delete*", "cloudformation:Update*", "cloudformation:Stop*", "cloudformation:Execute*", "cloudformation:Create*" ], "Resource": [ "arn:aws:cloudformation:*:*:stack/gov-*/*", "arn:aws:cloudformation:*:*:stackset/gov-*:*" ], "Condition": { "StringNotLike": { "aws:PrincipalARN": [ "arn:aws:iam::*:role/man/Automation", "arn:aws:iam::*:role/man/OrgAdmin", "arn:aws:iam::*:role/AutomationStackSets" ] } } } Example of SCP

< > ŒŽ § IAM § Governance Features § Security § Send Notification Setup and Provisioning MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure 19 Account Vending Maschine - general

< > ŒŽ § IAM § Governance Features § Security § Send Notification Setup and Provisioning MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure 20 Account Vending Maschine - general

< > ŒŽ Setup and Provisioning § Budgets § DNS Delegation § VPC / Networking Note: This setup is work in progress MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure 21 Account Vending Maschine - specific

< > ŒŽ How we use Continuous Integration and Deployment (CI/CD) § We are dogfooding our CI/CD Platform § We are currently focusing on CI and Source Code Quality § Due to Security Restrictions we are not doing Continuous Deployments What next? § Deploying Lambda for Automation as new version without pointing the alias to the new version § Service Control Policies (SCP) should be checked with a nightly job against the saved code in the source code repository 22 Setup and Provisioning MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure

WRAP UP

< > ŒŽ Wrap up § CI/CD for Account Automation needs to be improved - currently only focus on quality, CD is work in progress § Product Teams need experience in Cloud => Training and Enablement is very important § Sharing of Information is not easy, e.g. § Usage of Blueprints and Templates § Feedback Loops – What developers need? § Platform Engineering – treat your platform as product – a great platform can enhance the software delivery § Reduce common efforts § Help onboard new teams to the cloud 24 Wrap up MAN Truck & Bus | DevOps Meetup Lisbon | Stefan Killian | June 2019 | How to Manage Cloud Infrastructure

THANK YOU! We are hiring for our MAN Digital Hub (Jobs)