Metro's Newsfeed Algorithm

How Metro built an algorithmically driven Homepage.

Transcript

1. Metro’s Newsfeed Algorithm How Metro built an algorithmically driven Homepage
2. ! David Jensen ! Head of Development ! Metro.co.uk ! WordPress VIP » since Dec 2012 Who am I?
3. ! Metro is a very lean operation ! 6 developers ! 20 content producers ! 24/7 mindset ! Constant experimentation ! Trending -> Timeline -> Newsfeed Why algorithms?
4. Collating data from: ! Facebook » Shares » Likes » Comments ! Twitter ! Omniture ! WordPress Started as a dissertation project
5. Views + ((Tweets + Facebook Interactions) * 50) = Score ! Calculated every 30 minutes ! Rate of change = constantly changing Current Score – Previous Score = Trending Initial calculations
6. Trending Designs
7. Trending Stats
8. ! Removed swipe ! Trialled at the bottom of the homepage ! Then rolled out to the bottom of every page ! Native Content ! Native Display Units Timeline “A stream of news”
9. ! Time based stream ! Picture size based on currently popularity ! Native content clearly marked ! Native display units for CPA and CPM based advertising ! Consistency across all platforms Timeline Design
10. ! Scrolls Timeline Statistics ! Clicks
11. (Views + ((Social Interactions) * 10)) * Time Since Publish Coefficient ! Highest clicks on the top of every stream ! Popular content at the top of each stream ! Gives fresh content a chance ! Penalises popular content after a period ! Four hours about the half life of a story Newsfeed “Algorithmic stream of news”
12. Stream based iterations / DAU 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 Timeline New Style Newsfeed Infinite Scrolls/DAU Clicks/DAU
13. ! Based on actions per daily active user (DAU) ! Timeline -> Newsfeed clicks increased 9% ! Allowed us to take over the homepage ! Content density A/B test increased clicks 20% ! Infinite scroll increased clicks 20% ! Native Display -> 10x click through vs sidebar MPU ! Native content traffic drivers on every page NewsFeed Statistics
14. ! Content volume is key to ensure freshness ! Cut the data at the highest level for cache-ability ! Speed of lazy load essential ! Publishing times can affect clustering ! MySQL is simple but limited ! Common understanding helps iterating Lessons learned
15. ! Cache the first page of it using wpcom_vip_file_get_contents ! Copy the public API format to be able to change between sources quickly ! Large options allows you to store data in it ! Post meta can also store information ! CHEEZETEST is great but can add complication 15 WordPress Lessons
16. ! Micro services architecture » Data mining » Newsfeed » Commercial feed ! Backbone used for templates ! Cloud front for caching API / Frontend
17. ! Just top 10 stories on the site at any time ! Gives you more of what you read most ! 600 installs, 120 DAUs, 2 sessions a day, 13 screens/session ! Wouldn't have been possible without the API Metro10 – Android App
18. http://blog.david-jensen.com @elgrom Thanks for listening