Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Apps vs Infra, Where are My Pipelines? | DevOps.js Conference

March 30, 2021

Apps vs Infra, Where are My Pipelines? | DevOps.js Conference

Automation of a single monolithic app is pretty straight-forward. Split it into a frontend and backend and it's still manageable. Throw in more components or infrastructure and suddenly you're scratching your head at why a build ran - or didn't run. How many pipelines do I need? How many git repos should I have? Let's walkthrough use cases from small teams who own their entire stack to organizations with central IT units that manage shared infrastructure. Learn which scenarios and criteria determine how to slice but not spaghettify your pipelines.

DevOps.js Conference, 29-30 March 2021


March 30, 2021

More Decks by jng

Other Decks in Programming


  1. Apps vs Infra Where are my Pipelines? Julie Ng, Engineer,

    Microsoft DevOps.js Conference, March 2021
  2. Hi, I’m Julie • Engineer, FastTrack for Azure • Previously

    Enterprise Architect Allianz Germany (2016-2019) • Full-Stack Engineer, UX Designer • Likes: Node.js, Ruby, Open Source • Dislikes: Windows 😜 • Opinions are my own Engineering Bootcamp, Dublin, Ireland, February 2020
  3. Monorepo – CI Workflow Once Upon a Time /app refs/head/main

    dev.app.com refs/head/production app.com
  4. Monorepo – CD Workflow Once Upon a Time /app npm

    run e2e Build n+1 refs/head/main Build n push deploy dev.app.com deploy app.com OK? No Fail git commit Yes push refs/head/production
  5. Backends for Frontends (BFFs) Along came Mobile Devices and… /frontend

    refs/head/main refs/head/production /backend What do you want me to deploy? • Backend • Frontend • Both?
  6. • Git Push • Branches • Paths • Pull Requests

    • Schedule • Web Hooks Events and Triggers
  7. Monorepos & Triggers Pipeline As Code Example Snippet - Jenkins

    Example Snippet – Azure Pipelines
  8. 🥴

  9. Split ‘em up Pipeline as Code backend.dev.Jenkinsfile backend.prod.Jenkinsfile frontend.dev.Jenkinsfile frontend.prod.Jenkinsfile

  10. What About Versioning? /frontend /backend v1.8.2 Version? v1.1 Distributed Monolith?

  11. Microservices Truly independent calculator subtract multiply divide add • v1.0

    • v1.1 • v1.2
  12. Which End to End Tests? Independent, but still challenging multiply

    multiply.dev.app.com npm run e2e deploy calculator Which version of e2e? • /refs/head/production • /refs/tags/v1.3 promote or not promote?
  13. So how do we solve this? Talk to each other

  14. Ingress Calculator to Kubernetes Management: All the Cool Kids are

    Doing it Ingress calculator code images charts calculator.com calculator Infra as Code Pipeline as Code 3 Repos X n Triggers So many triggers, events and dependencies 😬
  15. PROs • Total Control • More conducive to “Learning by

    Doing” without others relying on you • Easier overview of possible triggers • Easier to version • High Trust à Less Security configuration required. CONs • More skills required – App Dev, DevOps, Infrastructure, esp. networking • Tightly coupled app and infra Lifecycles Monorepo – App + Infra (one team)
  16. Multi-tenant Kubernetes What if it’s many teams? images Ingress Ingress

    calcula tor calculator add subtract multiply divide add subtract multiply divide calc infra CI/CD pull min. 7 Repos X n Triggers MORE triggers, events and dependencies(?) 🤯
  17. CI/CD at Scale - Layers infra dev.app.com foo.app.com DNS TLS

    Certs dev.app.com foo.app.com Ingress namespace namespace calculator pull TLS Certs team.app.com pull? pull? aks app Layer 0 - Infra Layer 1 – K8s Layer 2 – Apps dns dns fork Pull Request Example - InnerSource Multiple managed clusters
  18. InnerSource Code Examples Terraform https://github.com/julie-ng

  19. All the Security Configs Don’t Forget End-to-End Governance https://github.com/azure/devops-governance

  20. PROs • App Dev Teams – less infra experience required

    if everything pre-configured • Loose Coupling / Independent teams • InnerSource Possibilities • …you’re doing this because it’s required, right? CONs • Need very experienced central IT team • Many Security Vectors, the more granular… • The smaller the attack surface • The larger the management overhead DevOps at Scale (many teams)
  21. Choreography CI/CD is Easier with small teams, but still a

    dance 😎
  22. • Triggers grow exponentially. • Who owns the responsibility? E.g.

    uptime. Suppliers make the rules, not consumers. • What level of complexity are you most comfortable with? • E2E Tests are critical for automated promotion. • It’s OK to promote manually. • TALK TO EACH OTHER (within and across teams) • In Person • Chat • Video Meetings • Issues, Pull Requests, etc. Takeaways
  23. Follow Me GitHub.com/julie-ng julie.io