Engineering the New Profile Josh Clemm Profile Tech Lead March 2013 | LinkedIn David Fleming Senior Product Manager at Zoomjax San Francisco Bay Area | Software Previous Education Golden Phase, FixDex Silicon Valley Business Academy

Let's compare...

Really a brand new product Refreshed Look & Feel Simplified Design Surfaced New & Interactive Modules New Data Insights

New Features Improved in-line editing experience Modules with In-line searching & pagination

The New Profile - Goals ● Represent your entire professional identity ○ Not just your resume ○ Activity, Groups, Following, Connections, Insights about you and your network

The New Profile - Goals ● Represent your entire professional identity ○ Not just your resume ○ Activity, Groups, Following, Connections, Insights about you and your network ● Needs to be more interactive ○ Keep users engaged on the page ○ Inline pagination, editing, searching

The New Profile - Goals ● Represent your entire professional identity ○ Not just your resume ○ Activity, Groups, Following, Connections, Insights about you and your network ● Needs to be more interactive ○ Keep users engaged on the page ○ Inline pagination, editing, searching ● Needs to be fluid, flexible, fast ○ Progressive rendering ○ Maintain high performance

So how did we achieve all that?

Using a lot of Technologies

And in particular...

Let's look at our High Level Frontend Architecture

Groups Content Service Connections Content Service Profile Content Service Client/Browser CDN Load Balancer SCDS Fizzy Profile Web App Profile Content Service Profile Web App Connections Content Service Groups Content Service Dust/JS/CSS Server Retrieve Data Models from Mid-tier /profile/view?id=32 top_card.tl, background.tl /profile/topcard /profile/background

Groups Content Service Connections Content Service Profile Content Service Client/Browser CDN Load Balancer SCDS Fizzy Profile Web App Profile Content Service Profile Web App Connections Content Service Groups Content Service Dust/JS/CSS Server Retrieve Data Models /profile/view?id=32 top_card.tl, background.tl /profile/topcard /profile/background Our new architecture uses new tech at all layers

Groups Content Service Connections Content Service Profile Content Service Client/Browser CDN Load Balancer SCDS Fizzy Profile Web App Profile Content Service Profile Web App Connections Content Service Groups Content Service Dust/JS/CSS Server Retrieve Data Models /profile/view?id=32 top_card.tl, background.tl /profile/topcard /profile/background Let's start at the bottom with Mappers

Mappers - JSON endpoints ● Convert data models from mid-tier services into JSON ● Each have an unique endpoint URL ( /profile/positions?id=42 ) Positions Mapper JSON "positions": [{ "position": {"id:{}"}, ]} Biz Profile Model Profile Model Rich Media Rest Model References Model Profile Flex Model

Mappers - an example public class PictureMapper extends ProfileParametersAwareMapper { private PictureContentMap pictureContentMap; private static final int ZOOMABLE_DIMENSION = 225; @Override public void doService() { PictureContentModel picCM = getContent(PictureContentModel.class); //declare content needs picCM.criteria().setId(getVieweeId()); //supply any input params assemble(); //invoke framework to retrieve declared content if (isResolvedWithoutErrors(picCM)) { //all went well, create new content map to hold output (JavaBean-like objects) pictureContentMap = ContentMap.proxyNew(PictureContentMap.class); Integer pictureWidth = picCM.getPictureWidth(); pictureContentMap.setIsZoomable(pictureWidth >= ZOOMABLE_DIMENSION); if(pictureWidth != null && pictureWidth > 0) { pictureContentMap.setWidth(pictureWidth); pictureContentMap.setHeight(picCM.getPictureHeight()); } pictureContentMap.setPictureID(picCM.getPictureID()); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Mappers - an example public class PictureMapper extends ProfileParametersAwareMapper { private PictureContentMap pictureContentMap; private static final int ZOOMABLE_DIMENSION = 225; @Override public void doService() { PictureContentModel picCM = getContent(PictureContentModel.class); //declare content needs picCM.criteria().setId(getVieweeId()); //supply any input params assemble(); //invoke framework to retrieve declared content if (isResolvedWithoutErrors(picCM)) { //all went well, create new content map to hold output (JavaBean-like objects) pictureContentMap = ContentMap.proxyNew(PictureContentMap.class); Integer pictureWidth = picCM.getPictureWidth(); pictureContentMap.setIsZoomable(pictureWidth >= ZOOMABLE_DIMENSION); if(pictureWidth != null && pictureWidth > 0) { pictureContentMap.setWidth(pictureWidth); pictureContentMap.setHeight(picCM.getPictureHeight()); } pictureContentMap.setPictureID(picCM.getPictureID()); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 Declare the data you need Set the data you want coming back as JSON

Mappers - features ● Modular ○ A single mapper can retrieve data for a section ■ Positions, Educations, Groups, etc. ● Reusable & Combinable ○ Mappers can be used more than once ■ Positions Mapper is used for Positions section and the positions part of Top Card ○ You can Aggregate Mappers under a common root element and a new URL endpoint

Profile's Many Mappers ● Each section on Profile has either a single Mapper or Aggregated Mapper (like Top Card) for its data

Profile's Many Mappers ● Each section on Profile has either a single Mapper or Aggregated Mapper (like Top Card) for its data Profile Web App Summary Mapper Positions Mapper Educations Mapper Picture Mapper Top Card Mapper "Summary": { "summary": "I'm an experienced..."} JSON JSON "TopCard": { "positions": {}, "educations":{}, "picture":{}} Groups Mapper Connections Mapper Patents Mapper "Connections": { "profiles": [ ...]} JSON

So we have these Mappers that return JSON for each section. Who calls each one?

Groups Content Service Connections Content Service Profile Content Service Client/Browser CDN Load Balancer SCDS Fizzy Profile Web App Profile Content Service Profile Web App Connections Content Service Groups Content Service Dust/JS/CSS Server Retrieve Data Models /profile/view?id=32 top_card.tl, background.tl /profile/topcard /profile/background Fizzy - the UI aggregator

Fizzy ● Fizzy is an UI aggregator in 2 parts: ○ Fizzy Server fetches the content you want ○ Fizzy Client renders it when ready ● Your base page defines its structure and which UI components it needs (called "embeds") *Fizzy Server is an Apache Traffic Server Plugin, Fizzy Client is a JS library

Profile's Embeds Top Card Embed Activity Embed Embed Embed Embed Embed Yet another embed

Profile's Embeds in code ...

Profile's Embeds in code ...

Fizzy fetches data and sends it to the browser. What's rendering the actual markup?

Groups Content Service Connections Content Service Profile Content Service Client/Browser CDN Load Balancer SCDS Fizzy Profile Web App Profile Content Service Profile Web App Connections Content Service Groups Content Service Dust/JS/CSS Server Retrieve Data Models /profile/view?id=32 top_card.tl, background.tl /profile/topcard /profile/background You guessed it... Dust client templates

{Dust} client templates ● LinkedIn's latest and greatest rendering layer ● Logic-less client template language ● Profile page made up of many* templates ● Templates + JSON = full markup! *over 400 actually

{Dust} - what it looks like { "name": "Frank Stallone", "headline": "Actor and Less Famous Brother", "info":{ "location": "Hollywood", "industry": "Entertainment" } }

{Dust} - why it's cool for profile ● Cached markup in CDNs and browsers ○ Members browse many profiles in a row ● Very DRY and reusable ○ Improved development speed ○ Templates within templates within templates... ■ date range, degree badge, formatted summary field ● Super easy to refresh a section on the page ○ Re-render and replace ○ Useful in pagination, inline searching, and inline editing

{Dust} on profile {>"tl/apps/profile/v2/embed/company_logo"/}

{title_highlight|s} {?selfView} {i18n_Edit} {/selfView}

{?companyName} {>"tl/apps/profile/v2/embed/company_link" track_param="prof-exp"/} {/companyName}

{>"tl/apps/profile/v2/partial/daterange"/} {! Location !} {@pre.fmt key="fmt_location" type="geo.region" value="{location}" render="false"/} {?fmt_location}{fmt_location} {:else} {?locationName} {locationName} {/locationName} {/fmt_location} {>"tl/apps/profile/v2/partial/summary_field" _summary=summary/} {>"tl/apps/profile/v2/partial/associated_content" trkCodePrefix="exp"/}

{Dust} on profile {>"tl/apps/profile/v2/embed/company_logo"/}

{title_highlight|s} {?selfView} {i18n_Edit} {/selfView}

{?companyName} {>"tl/apps/profile/v2/embed/company_link" track_param="prof-exp"/} {/companyName}

{>"tl/apps/profile/v2/partial/daterange"/} {! Location !} {@pre.fmt key="fmt_location" type="geo.region" value="{location}" render="false"/} {?fmt_location}{fmt_location} {:else} {?locationName} {locationName} {/locationName} {/fmt_location} {>"tl/apps/profile/v2/partial/summary_field" _summary=summary/} {>"tl/apps/profile/v2/partial/associated_content" trkCodePrefix="exp"/} Partial templates like this are used on almost all the background sections

We have our Mappers, Fizzy, and Dust templates... Let's bring it all together.

Feature: Inline Editing ● Dust + Mappers make this easy ● Dust templates for view and edit ● Mappers return just the data we need to refresh "View" template "Edit" template Summary Section

Feature: Inline Editing ● Dust + Mappers make this easy ● Dust templates for view and edit ● Mappers return just the data we need to refresh "View" template "Edit" template Summary Section When edit link is clicked, render "edit" template

Feature: Inline Editing ● Dust + Mappers make this easy ● Dust templates for view and edit ● Mappers return just the data we need to refresh "View" template "Edit" template Summary Section On submit, our endpoint sends back JSON, and we re-render

Inline editing - Example Either the "view" or "edit" template is shown at a time

Inline editing sounds easy... ● Issue #1: Profile data is highly coupled with different sections. ○ Adding/editing a position needs to be reflected on Top Card... ○ Deleting a project needs to also be removed from any associated position... ● Solution: since we have mappers for each section, we know which to re-fetch upon a save and refresh the respective templates

What about pagination, search? ● Same idea but easier (not coupled with other sections) ● Mappers can take URL offset params Switch out a partial template containing just list of profiles

Let's talk performance

High Performance Page ● Profile is the most trafficked page on LinkedIn ● Profile fetches 48+ different types of content on each page load ○ This content results in 250+ total downstream calls

High Performance Page ● Profile is the most trafficked page on LinkedIn ● Profile fetches 48+ different types of content on each page load ○ This content results in 250+ total downstream calls Bottom Line: We need to be fast, but need to consider downstream fanout

High Performance - Parallel Requests Fizzy Profile App Profile App Profile App Profile App Profile App Profile App Profile App Profile App Profile App Profile App In the beginning, Profile had 15 embeds with 15 different endpoints which is great for speed...

High Performance - Parallel Requests Fizzy Profile App Profile App Profile App Profile App Profile App Profile App Profile App Profile App Profile App Profile App Content Service Content Service Content Service Content Service Content Service Content Service Content Service Content Service Content Service Content Service That's a lot of downstream calls, often requesting the same data too. Content Service Content Service

High Performance - Parallel Requests Fizzy Profile App Profile App Profile App Profile App Profile App Profile App Profile App Profile App Profile App Profile App Content Service Content Service Content Service Content Service Content Service Content Service Content Service Content Service Content Service Content Service Content Service Content Service and those calls call more and more... until...

High Performance - Parallel Requests Fizzy Profile App Profile App Profile App Profile App Profile App Profile App Profile App Profile App Profile App Profile App Content Service Content Service Content Service Content Service Content Service Content Service Content Service Content Service Content Service Content Service Content Service Content Service aaaaand the site's down

High Performance - Parallel Requests ● Tradeoff: Speed vs Scalability ● 15 parallel calls will be fast but with Profile's load will take down site

High Performance - Parallel Requests ● Tradeoff: Speed vs Scalability ● 15 parallel calls will be fast but with Profile's load will take down site Batched Endpoints to the rescue

Batching Calls to Mappers Originally, we had separate endpoints (URIs) for each embed 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ...

Batching Calls to Mappers We now tell framework to batch these two endpoints (Fizzy knows how to deliver the right data to the right embed) ...

Batching Calls in Profile ● Today, we use between 3-5 parallel requests to Profile ● It's a good balance of speed vs. scalability ● Batching requests that need the same data has the added benefit of less downstream calls

Progressive Rendering With parallel fetch, Profile modules render when ready for improved perceived performance

Progressive Rendering With parallel fetch, Profile modules render when ready for improved perceived performance

How else to improve page load times?

Profiles can be long... Embed Embed Embed Embed Embed Embed Embed Profile Page

Optimize for Above the Fold Embed Embed Embed Embed Embed Embed Embed Profile Page The fold No need to render No need to render or fetch

Optimize for Above the Fold ● The New Profile renders above the fold as fast as we can ○ Deferring everything at the bottom ○ Reduces the static content we need to initially download

Optimize for Above the Fold ● The New Profile renders above the fold as fast as we can ○ Deferring everything at the bottom ○ Reduces the static content we need to initially download Sorry guys, we’ll get to you later

Optimize for Above the Fold ● The New Profile defers fetching the modules at the bottom of the page ○ Improves server side assembly times ○ Lowers payload and improves network time ○ Improves client rendering times

Optimize for Above the Fold ● The New Profile defers fetching the modules at the bottom of the page ○ Improves server side assembly times ○ Lowers payload and improves network time ○ Improves client rendering times Go fetch!

So we covered the product, technologies, and some features... let's revisit our goals.

Revisiting the New Profile Goals ● Needs to surface your entire professional identity ○ Easily expose new data endpoints ● Needs to be more interactive ○ Inline editing, searching ● Needs to be ○ Fluid ○ Flexible ○ Fast

Revisiting the New Profile Goals ● Needs to surface your entire professional identity ○ Easily expose new data endpoints (Mappers) ● Needs to be more interactive ○ Inline editing, searching (Dust + Mappers) ● Needs to be ○ Fluid (Fizzy progressive rendering) ○ Flexible (Fizzy deferred rendering) ○ Fast (Fizzy parallel calls + defer fetch, batching Mappers, Dust template caching)

Takeaways ● Mappers for each module makes sense ○ You can combine and reuse with ease ○ A must if refreshing part of page ● When structuring your page, start with many embeds ○ You can control number of requests, when embeds are rendered, and when to fetch data ● Many partial templates are a good thing ○ Allows you finer control over what to re-render ○ Improves developer speed

Takeaways cont. ● Make these technologies work for you ○ Our Mappers are structured to guarantee minimal downstream calls ○ Take advantage of decoupling rendering layer with server side endpoints ■ Engineers can build endpoints ■ Web devs can start the templates with mocked JSON data

Thanks! Josh Clemm linkedin.com/in/joshclemm