Apps vs Infra, Where are My Pipelines? | DevOps.js Conference

Automation of a single monolithic app is pretty straight-forward. Split it into a frontend and backend and it's still manageable. Throw in more components or infrastructure and suddenly you're scratching your head at why a build ran - or didn't run. How many pipelines do I need? How many git repos should I have? Let's walkthrough use cases from small teams who own their entire stack to organizations with central IT units that manage shared infrastructure. Learn which scenarios and criteria determine how to slice but not spaghettify your pipelines.

DevOps.js Conference, 29-30 March 2021

Transcript

1. Apps vs Infra Where are my Pipelines? Julie Ng, Engineer, Microsoft DevOps.js Conference, March 2021
2. Hi, I’m Julie • Engineer, FastTrack for Azure • Previously Enterprise Architect Allianz Germany (2016-2019) • Full-Stack Engineer, UX Designer • Likes: Node.js, Ruby, Open Source • Dislikes: Windows 😜 • Opinions are my own Engineering Bootcamp, Dublin, Ireland, February 2020
3. Monorepo – CI Workflow Once Upon a Time /app refs/head/main dev.app.com refs/head/production app.com
4. Monorepo – CD Workflow Once Upon a Time /app npm run e2e Build n+1 refs/head/main Build n push deploy dev.app.com deploy app.com OK? No Fail git commit Yes push refs/head/production
5. Backends for Frontends (BFFs) Along came Mobile Devices and… /frontend refs/head/main refs/head/production /backend What do you want me to deploy? • Backend • Frontend • Both?
6. • Git Push • Branches • Paths • Pull Requests • Schedule • Web Hooks Events and Triggers
7. Monorepos & Triggers Pipeline As Code Example Snippet - Jenkins Example Snippet – Azure Pipelines
8. 🥴
9. Split ‘em up Pipeline as Code backend.dev.Jenkinsfile backend.prod.Jenkinsfile frontend.dev.Jenkinsfile frontend.prod.Jenkinsfile
10. What About Versioning? /frontend /backend v1.8.2 Version? v1.1 Distributed Monolith?
11. Microservices Truly independent calculator subtract multiply divide add • v1.0 • v1.1 • v1.2
12. Which End to End Tests? Independent, but still challenging multiply multiply.dev.app.com npm run e2e deploy calculator Which version of e2e? • /refs/head/production • /refs/tags/v1.3 promote or not promote?
13. So how do we solve this? Talk to each other 😎
14. Ingress Calculator to Kubernetes Management: All the Cool Kids are Doing it Ingress calculator code images charts calculator.com calculator Infra as Code Pipeline as Code 3 Repos X n Triggers So many triggers, events and dependencies 😬
15. PROs • Total Control • More conducive to “Learning by Doing” without others relying on you • Easier overview of possible triggers • Easier to version • High Trust à Less Security configuration required. CONs • More skills required – App Dev, DevOps, Infrastructure, esp. networking • Tightly coupled app and infra Lifecycles Monorepo – App + Infra (one team)
16. Multi-tenant Kubernetes What if it’s many teams? images Ingress Ingress calcula tor calculator add subtract multiply divide add subtract multiply divide calc infra CI/CD pull min. 7 Repos X n Triggers MORE triggers, events and dependencies(?) 🤯
17. CI/CD at Scale - Layers infra dev.app.com foo.app.com DNS TLS Certs dev.app.com foo.app.com Ingress namespace namespace calculator pull TLS Certs team.app.com pull? pull? aks app Layer 0 - Infra Layer 1 – K8s Layer 2 – Apps dns dns fork Pull Request Example - InnerSource Multiple managed clusters
18. InnerSource Code Examples Terraform https://github.com/julie-ng
19. All the Security Configs Don’t Forget End-to-End Governance https://github.com/azure/devops-governance
20. PROs • App Dev Teams – less infra experience required if everything pre-configured • Loose Coupling / Independent teams • InnerSource Possibilities • …you’re doing this because it’s required, right? CONs • Need very experienced central IT team • Many Security Vectors, the more granular… • The smaller the attack surface • The larger the management overhead DevOps at Scale (many teams)
21. Choreography CI/CD is Easier with small teams, but still a dance 😎
22. • Triggers grow exponentially. • Who owns the responsibility? E.g. uptime. Suppliers make the rules, not consumers. • What level of complexity are you most comfortable with? • E2E Tests are critical for automated promotion. • It’s OK to promote manually. • TALK TO EACH OTHER (within and across teams) • In Person • Chat • Video Meetings • Issues, Pull Requests, etc. Takeaways
23. Follow Me GitHub.com/julie-ng julie.io