Platform-powered: Building a Frontend Platform that Scales

As Lyft grew in size, we faced the problem of platform fragmentation. How would we manage a growing frontend microservice explosion, while allowing our services to evolve for the future?

We share about the next-generation frontend platform at Lyft, and the tooling that is built in to manage and migrate all 100+ frontend microservices. A talk relevant to any platform team building a technology platform to accelerate product delivery across all teams, while managing technical debt.

Transcript

1. Platform-Powered 🚀 Build a frontend platform that scales as fast as you do Andrew Hao (@andrewhao)
2. Ever felt growing pains? 2 / 31
3. "...most tools and processes only support about one order of magnitude of growth before becoming ineffective" Will Larson. An Elegant Puzzle: Systems of Engineering Management 3 / 31
4. Monolith to Microservices Python/Angular monolith ...to Node + React isomorphic apps via a templated service generator ...coupled with infrastructure investments in microservices ...led to a microservice explosion! 💥 4 / 31
5. But the problems were starting to catch up to us Long-lived services require maintenance Platform was fragmenting New infrastructure updates were hard to apply 5 / 31
6. Technical leverage 6 / 31
7. But where to start? 7 / 31
8. 8 / 31
9. 9 / 31
10. Generation 3: @lyft/service ✨ 10 / 31
11. Principles for Technical Leverage 👟 ✨ 🤖 Stand on the Shoulders of Giants Simplify to Understand Standardize and Automate 11 / 31
12. Stand on the Shoulders of Giants 👟 We chose Next.js as our platform of choice 12 / 31
13. Stand on the Shoulders of Giants 👟 We chose Next.js as our platform of choice Solved: Build configurations, static site generation, AMP pages, code splitting, dynamic imports 12 / 31
14. Stand on the Shoulders of Giants 👟 We chose Next.js as our platform of choice Solved: Build configurations, static site generation, AMP pages, code splitting, dynamic imports 🎷 Now: We don't need to maintain our internal build system anymore 12 / 31
15. Simplify to Understand ✨ Paradigm shift: convention over configuration 13 / 31
16. Simplify to Understand ✨ Paradigm shift: convention over configuration Next.js: Filesystem router, server-side getInitialProps and getServerSideProps handlers 13 / 31
17. Simplify to Understand ✨ Paradigm shift: convention over configuration Next.js: Filesystem router, server-side getInitialProps and getServerSideProps handlers 🍕 Now: Lower cognitive load working in apps, higher developer productivity 13 / 31
18. Standardize and Automate 🤖 We built a plugin system that standardizes our library integrations 14 / 31
19. Standardize and Automate 🤖 We built a plugin system that standardizes our library integrations We made migrations a first-class part of our new system 14 / 31
20. Standardize and Automate 🤖 We built a plugin system that standardizes our library integrations We made migrations a first-class part of our new system 🎸 Now: we have the tools to reuse code, keep the stack modern and prevent drift 14 / 31
21. Anatomy of a Plugin A set of hooks, bundled up in a library Plugin Hooks: Webpack Express Middleware Next.js Configuration Next.js Application Next.js Document 15 / 31
22. 1. Install the plugin in lyft.plugins.ts import CookieAuthPlugin from "@lyft/service plugin cookie auth"; const plugins = [ new CookieAuthPlugin(), Other plugins ]; 16 / 31
23. 2. Use it! import { useCookieAuth } from "@lyft/service plugin cookie auth"; In React component const Page: React.FC = () { const { userId, isLoggedIn } = useCookieAuth(); That's it! You can now use as you see f t if (!isLoggedIn()) { return <p>Sorry, you must be logged in p>; } }; 17 / 31
24. CookieAuthPlugin: Express.js server hook import cookieParser from "cookie parser"; import { Application } from "express"; const cookieAuthServerHook = (app: Application) { Gives us req.cookies app.use(cookieParser); app.use(function parseUserId(req, res, next) { Assume this decrypts data and returns a user ID from a session const { userId } = parseSessionCookies(req.cookies); Store userId in response for later retrieval res.locals.userId = userId; next(); }); }; 18 / 31
25. CookieAuthPlugin: Next.js App hook function CookieAuthApp({ App: NextApp }) { return class extends App { static getInitialProps = async (appContext) { const originalProps = await App.getInitialProps(appContext); const userId = appContext.ctx.res locals userId; return { originalProps, userId }; }; render() { return ( <CookieAuthContext.Provider value={this.props.userId}> {super.render()} </CookieAuthContext.Provider> ); } }; } 19 / 31
26. Bring it all together into the Plugin export default class CookieAuthPlugin { apply = (service: ServicePluginHost) { service.hooks.server.tap(this.name, cookieAuthServerHook); service.hooks.app.tap(this.name, (App) CookieAuthApp({ App })); }; } 20 / 31
27. And add a nice developer facing convenience hook const useCookieAuth = () ({ userId: React.useContext(CookieAuthContext), isLoggedIn: () { const userId = React.useContext(CookieAuthContext); return userId; }, }); 21 / 31
28. @lyft/service Plugin Ecosystem State management (Redux, MobX, XState) GraphQL Lyft Product Language, styled-components, Material UI authn/authz i18n RUM performance tracking Feature flagging and experimentation MirageJS Logging/metrics/bug reporting 22 / 31
29. Coming Soon Developer Support Tooling Embedded tools to help developers debug or ask for help 23 / 31
30. Flywheel effect Now users are contributing back to these plugins Over 40% of new plugins have been product-engineer contributions 24 / 31
31. Migrations - How we Automate Guardrails to prevent drift jscodeshift scripts 25 / 31
32. Original import { logger } from "@lyft/service plugin logging"; logger.info("test log"); Upgraded import { getLogger } from "@lyft/service plugin logging"; const logger = getLogger(); logger.info("test log"); 26 / 31
33. Migration Versioning We use versioned migrations If you change an interface, you must ship a migration Store migration state per plugin in package.json 27 / 31
34. Release Management: One bold constraint The platform version and the plugin system are pinned to the same version 🎯 Plugins are guaranteed to work with a specific version of the platform This means the entire platform moves together! 28 / 31
35. Organizational process Hands-on migration workshops Migration scripts take services most of the way from Gen 2 to Gen 3 Relentless internal evangelism Technical program management + senior leadership visibility are key 29 / 31
36. Wins Higher developer happiness and productivity Quicker adoption of new service releases, preventing drift 30 / 31
37. Principles for Technical Leverage 👟 ✨ 🤖 Stand on the Shoulders of Giants Simplify to Understand Standardize and Automate 31 / 31