Platform-powered: Building a Frontend Platform that Scales

As Lyft grew in size, we faced the problem of platform fragmentation. How would we manage a growing frontend microservice explosion, while allowing our services to evolve for the future?

We share about the next-generation frontend platform at Lyft, and the tooling that is built in to manage and migrate all 100+ frontend microservices. A talk relevant to any platform team building a technology platform to accelerate product delivery across all teams, while managing technical debt.

Transcript

1. Platform-Powered
   🚀
   Build a frontend platform that scales as fast as
   you do
   Andrew Hao (@andrewhao)
   

   View Slide

2. Ever felt growing pains?
   2 / 31
   

   View Slide

3. "...most tools and processes only
   support about one order of
   magnitude of growth before
   becoming ineffective"
   Will Larson. An Elegant Puzzle: Systems of Engineering Management
   3 / 31
   

   View Slide

4. Monolith to Microservices
   Python/Angular monolith
   ...to Node + React isomorphic apps via a templated
   service generator
   ...coupled with infrastructure investments in
   microservices
   ...led to a microservice explosion!
   💥
   4 / 31
   

   View Slide

5. But the problems were starting to
   catch up to us
   Long-lived services require maintenance
   Platform was fragmenting
   New infrastructure updates were hard to apply
   5 / 31
   

   View Slide

6. Technical leverage
   6 / 31
   

   View Slide

7. But where to start?
   7 / 31
   

   View Slide

8. 8 / 31
   

   View Slide

9. 9 / 31
   

   View Slide

10. Generation 3: @lyft/service
    ✨
    10 / 31
    

    View Slide

11. Principles for Technical Leverage
    👟 ✨ 🤖
    Stand on the
    Shoulders of
    Giants
    Simplify to
    Understand
    Standardize and
    Automate
    11 / 31
    

    View Slide

12. Stand on the Shoulders
    of Giants
    👟
    We chose Next.js as our platform of choice
    12 / 31
    

    View Slide

13. Stand on the Shoulders
    of Giants
    👟
    We chose Next.js as our platform of choice
    Solved: Build configurations, static site
    generation, AMP pages, code splitting, dynamic
    imports
    12 / 31
    

    View Slide

14. Stand on the Shoulders
    of Giants
    👟
    We chose Next.js as our platform of choice
    Solved: Build configurations, static site
    generation, AMP pages, code splitting, dynamic
    imports
    🎷
    Now: We don't need to maintain our internal
    build system anymore
    12 / 31
    

    View Slide

15. Simplify to Understand
    ✨
    Paradigm shift: convention over configuration
    13 / 31
    

    View Slide

16. Simplify to Understand
    ✨
    Paradigm shift: convention over configuration
    Next.js: Filesystem router, server-side
    getInitialProps and getServerSideProps handlers
    13 / 31
    

    View Slide

17. Simplify to Understand
    ✨
    Paradigm shift: convention over configuration
    Next.js: Filesystem router, server-side
    getInitialProps and getServerSideProps handlers
    🍕
    Now: Lower cognitive load working in apps,
    higher developer productivity
    13 / 31
    

    View Slide

18. Standardize and Automate
    🤖
    We built a plugin system that standardizes our
    library integrations
    14 / 31
    

    View Slide

19. Standardize and Automate
    🤖
    We built a plugin system that standardizes our
    library integrations
    We made migrations a first-class part of our new
    system
    14 / 31
    

    View Slide

20. Standardize and Automate
    🤖
    We built a plugin system that standardizes our
    library integrations
    We made migrations a first-class part of our new
    system
    🎸
    Now: we have the tools to reuse code, keep
    the stack modern and prevent drift
    14 / 31
    

    View Slide

21. Anatomy of a Plugin
    A set of hooks, bundled up in a library
    Plugin Hooks:
    Webpack
    Express Middleware
    Next.js Configuration
    Next.js Application
    Next.js Document
    15 / 31
    

    View Slide

22. 1. Install the plugin in lyft.plugins.ts
    import CookieAuthPlugin from "@lyft/service plugin cookie auth";
    const plugins = [
    new CookieAuthPlugin(),
    Other plugins
    ];
    16 / 31
    

    View Slide

23. 2. Use it!
    import { useCookieAuth } from "@lyft/service plugin cookie auth";
    In React component
    const Page: React.FC = () {
    const { userId, isLoggedIn } = useCookieAuth();
    That's it! You can now use as you see f t
    if (!isLoggedIn()) {
    return Sorry, you must be logged in p>;
    }
    };
    17 / 31
    

    View Slide

24. CookieAuthPlugin: Express.js server hook
    import cookieParser from "cookie parser";
    import { Application } from "express";
    const cookieAuthServerHook = (app: Application) {
    Gives us req.cookies
    app.use(cookieParser);
    app.use(function parseUserId(req, res, next) {
    Assume this decrypts data and returns a user ID from a session
    const { userId } = parseSessionCookies(req.cookies);
    Store userId in response for later retrieval
    res.locals.userId = userId;
    next();
    });
    };
    18 / 31
    

    View Slide

25. CookieAuthPlugin: Next.js App hook
    function CookieAuthApp({ App: NextApp }) {
    return class extends App {
    static getInitialProps = async (appContext) {
    const originalProps = await App.getInitialProps(appContext);
    const userId = appContext.ctx.res locals userId;
    return { originalProps, userId };
    };
    render() {
    return (
    
    {super.render()}
    
    );
    }
    };
    }
    19 / 31
    

    View Slide

26. Bring it all together into the Plugin
    export default class CookieAuthPlugin {
    apply = (service: ServicePluginHost) {
    service.hooks.server.tap(this.name, cookieAuthServerHook);
    service.hooks.app.tap(this.name, (App) CookieAuthApp({ App }));
    };
    }
    20 / 31
    

    View Slide

27. And add a nice developer facing convenience hook
    const useCookieAuth = () ({
    userId: React.useContext(CookieAuthContext),
    isLoggedIn: () {
    const userId = React.useContext(CookieAuthContext);
    return userId;
    },
    });
    21 / 31
    

    View Slide

28. @lyft/service Plugin Ecosystem
    State management (Redux, MobX, XState)
    GraphQL
    Lyft Product Language, styled-components, Material UI
    authn/authz
    i18n
    RUM performance tracking
    Feature flagging and experimentation
    MirageJS
    Logging/metrics/bug reporting
    22 / 31
    

    View Slide

29. Coming Soon
    Developer Support Tooling
    Embedded tools to help developers debug or ask for
    help
    23 / 31
    

    View Slide

30. Flywheel effect
    Now users are contributing back to these plugins
    Over 40% of new plugins have been product-engineer
    contributions
    24 / 31
    

    View Slide

31. Migrations - How we Automate
    Guardrails to prevent drift
    jscodeshift scripts
    25 / 31
    

    View Slide

32. Original
    import { logger } from "@lyft/service plugin logging";
    logger.info("test log");
    Upgraded
    import { getLogger } from "@lyft/service plugin logging";
    const logger = getLogger();
    logger.info("test log");
    26 / 31
    

    View Slide

33. Migration Versioning
    We use versioned migrations
    If you change an interface, you must ship a migration
    Store migration state per plugin in package.json
    27 / 31
    

    View Slide

34. Release Management: One bold
    constraint
    The platform version and the plugin system are pinned
    to the same version
    🎯
    Plugins are guaranteed to work with a specific version
    of the platform
    This means the entire platform moves together!
    28 / 31
    

    View Slide

35. Organizational process
    Hands-on migration workshops
    Migration scripts take services most of the way from
    Gen 2 to Gen 3
    Relentless internal evangelism
    Technical program management + senior leadership
    visibility are key
    29 / 31
    

    View Slide

36. Wins
    Higher developer happiness and productivity
    Quicker adoption of new service releases, preventing
    drift
    30 / 31
    

    View Slide

37. Principles for Technical Leverage
    👟 ✨ 🤖
    Stand on the
    Shoulders of
    Giants
    Simplify to
    Understand
    Standardize and
    Automate
    31 / 31
    

    View Slide