The Future of Platforms: The Invisible Glue Between Backstage and Terraform

In the world of building platforms seemingly everyone is investing in a portal. There is clear value to be had from this, but it is just one part of the full platform puzzle.

The goal of enabling developers to access infrastructure and other services through self-service is mission-critical, and a portal is the only interface.

Some invisible (and intelligent) glue is needed between platform tooling like Backstage and Terraform to add the governance and orchestration required to enable self-service.

In our experience with customers, we’ve learned that a portal typically provides an easy-to-understand user experience, but many developers prefer using an API or the CLI.

Infrastructure as code tooling excels in templating and deploying solutions for day one, but what about upgrades and maintenance on day two (and beyond)?

A modern platform needs to support all options.

Join this webinar to learn more about the invisible glue of platforms: platform orchestrators.

Transcript

1. The Future of Platforms: The Invisible Glue Between Backstage and Terraform Daniel Bryant Product Marketing Abby Bangser Principal Engineer
2. • Top-down, application developer-focused ◦ “The Backstage service catalog is fantastic, but the support for day 2 ops… not so much” • Bottom-up, operations/infrastructure-focused ◦ “The Terraform workflow is fantastic, but infrastructure abstractions leak through to developers (HCL, K8s, etc)” • Middle-out, platform engineering-focused ◦ X-as-a-service, process automation, fleet management ◦ “Platform as a product” approach Tl;dr: How do you build a platform?
3. Introductions Abby Bangser Principal Engineer @ Syntasso Daniel Bryant, Product Marketing @ Syntasso
4. The “what” of platforms 🏗
5. “Platform engineering improves developer experience and productivity by providing self-service capabilities with automated infrastructure operations. It is trending because of its promise to optimise the developer experience and accelerate product teams’ delivery of customer value.” https://www.gartner.com/en/articles/what-is-platform-engineering Gartner: What is platform engineering?
6. “Platform engineering improves developer experience and productivity by providing self-service capabilities with automated infrastructure operations . It is trending because of its promise to optimise the developer experience and accelerate product teams’ delivery of customer value.” https://www.gartner.com/en/articles/what-is-platform-engineering Gartner: What is platform engineering?
7. gartner.com/en/newsroom/press-releases/2023-11-28-gartner-hype-cycle-shows-ai-practices-and-platform-engineering-will-reach-mainstream-adoption-in-software-engineering-in-two-to-five-years Platform engineering expectations
8. Platform challenges
9. The “why” of platforms 🤔
10. Why build a platform? “The thing to keep in mind about Backstage is that its purpose is to improve developer onboarding time [...] it lets you more easily encode best practice and the best and fastest way to do things. Organisations adopt Kubernetes without understanding that if you're not building a custom PaaS for your organisation, you're wasting your time.” reddit.com/r/devops/comments/10guddj/backstageio_common_issues_and_pitfalls/#t1_j59lbs3-comment-rtjson-content
11. Why build a platform? “Operations were just drowning . We spent too much time on repetitive requests from developers that asked stuff in Slack or Jira Tickets. [...] We first trained people around Terraform and Helm but they just didn't want to deal with that and they ended up poking senior developers to fix things instead [...] This basically created a shadow operations problem on top.” reddit.com/r/devops/comments/stuep4/weve_spent_months_building_this_platform_devs/
12. • Go faster: Platform teams need to provide “everything as a service” to help rapidly and sustainably deliver value to end-users • Decrease risk: Teams need to automate manual processes in reusable components • Increase efficiency: You need to manage and scale your digital platform and resources as a fleet What are the goals of your platform?
13. The “how” of platforms 󰠼󰲪󰠹
14. “Backstage is my platform. Developers go here to spin up a new application, deploy it, and view metrics” ✅ Fantastic developer experience (and service catalog) ✅ Highly customisable ❌ Often a direct relationship with underlying infrastructure code ❌ Not easy to use a different interface when needed ❌ Day 2 aspects of the portal and plugins can be challenging Top down, app developer-focused rollout
15. Git Repo: tiger/checkout .gitlab-ci.yml src/ pom.xml README.md Git Repo: tiger/checkout .gitlab-ci.yml src/ pom.xml README.md Git Repo: tiger/checkout .gitlab-ci.yml src/ pom.xml README.md Git Repo: tiger/checkout .gitlab-ci.yml src/ pom.xml README.md Git Repo: tiger/checkout .gitlab-ci.yml src/ pom.xml README.md …67 repos… 2020 2018 Git Repo: tiger/checkout .gitlab-ci.yml src/ pom.xml README.md Git Repo: tiger/checkout .gitlab-ci.yml src/ pom.xml README.md Git Repo: tiger/checkout .gitlab-ci.yml src/ pom.xml README.md …12 repos… 2016 $ sb create new service \ --name "checkout" \ --language "java" \ --team "tiger" Our ServiceBuilder wasn't a "service manager"
16. “Terraform is my platform. I can orchestrate all of my infrastructure via HCL and cron jobs, and the GitOps pipelines automatically deploy applications” ✅ Everything-as-code ✅ Highly automatable ❌ Infrastructure abstractions leak outwards towards developers ❌ At scale, the diversity of tech can become challenging to orchestrate Bottom up, operations-focused rollout
17. • Terraform module with conventions meant PR reviews were necessary • Business requirements were managed out of band • Transition from Terraform to Controllers for Kubernetes required training HCL self service was overly optimistic
18. The missing middle? 🏗
19. https://thenewstack.io/crossplane-what-most-people-get-wrong-and-how-to-get-it-right/ https://platformengineering.org/platform-tooling https://cnoe.io/ https://www.kratix.io/
20. syntasso.io/post/platform-engineering-orchestrating-applications-platforms-and-infrastructure
21. syntasso.io/post/platform-engineering-orchestrating-applications-platforms-and-infrastructure
22. Making it happen 💪
23. Tech "stacks" are emerging • The “BACK” stack : Backstage, Argo, Crossplane, Kyverno • CNOE Framework : Cloud Native Operation Excellence • Kubefirst • DIY: All the other CNCF tech ++ • … More opinionated Less opinionated Platform builders need to elevate above Infra as Code
24. Business requirements change, user expectations expand, APIs are at the core of sustainable platforms • Open Application Model (OAM) / Score • Kratix and Promises • KubeBuilder • Crossplane • Massdriver • … More user focused More implementation focused Platform extensibility and usability demands APIs
25. The tech can be overwhelming
26. Build it, and they will come Build it, and they will come Solve problems, and they will come ⚾
27. thoughtworks.com/en-gb/insights/looking-glass/platforms-as-products Don't forget product focus
28. Don't forget product focus thoughtworks.com/en-gb/insights/looking-glass/platforms-as-products
29. Wrapping up 🎉
30. • 🏗 Build your platform intentionally ◦ APIs, abstrations, and automation are key ◦ Middle-out, platform engineering-focused ◦ Watch out for the missing middle: “platform orchestrators” • 👀 Watch for warning signs ◦ Struggling with scaling day 2 operations with your portal? ◦ Infrastructure abstractions leaking to developers? • 🎯 Focus on “platform as a product” Conclusion
31. syntasso.io/post/platform-engineering-orchestrating-applications-platfo rms-and-infrastructure docs.kratix.io/main/quick-start speakerdeck.com/syntasso Thank you!