Building Multilingual Recommendations Systems for BBC News

Magda Piatkowska, Lead Data Scientist, BBC

Transcript

1. Building Multilingual Recommendations Systems for BBC News Magda Piatkowska, Lead Data Scientist, BBC News
2. Inform, educate, entertain global audience
3. Technology can help open new paths into understanding the diverse interests, opinions and perspectives of our audiences.
4. Technically it is possible “…that around 90% of news will be computer generated by 2026…” Kris Hammond, Chief Scientist Narrative Science
5. NO BLACK BOXES IN THE NEWSROOM
6. 40 languages Algorithmically driven distribution for BBC World Service • We are addressing anonymous, disengaged readers • At a large scale in challenging environments • In 40 different languages • We need the solution to be universal ◦ Work with little information about the user ◦ Work fast (scale and kick start) ◦ Work based on language independent techniques
7. Distributed data community (made of superheroes) • Audience Engagement @BBC News • BBC News Labs (check out the blog!) • BBC Data Labs
8. BBC is competing for attention. The pressure is the same as in a commercial organisation. We need to start now and start simple. But where? • Content to content ◦ Diversity Low ◦ Novelty High ◦ Suprisal High • User Profile to content ◦ Diversity Low ◦ Novelty Low ◦ Suprisal High • Based on mass user (Most viewed) ◦ Diversity Low ◦ Novelty Low ◦ Suprisal Low • User to User ◦ Diversity High ◦ Novelty High ◦ Suprisal High
9. Understanding anonymous users’ needs
10. Design and architecture determines the product iterations
11. ONLINE OFFLINE Segment to Recommender Mapping NEAR-LINE Default Content Similarity User Profile User to User Header Recs
12. Simplified segmentation to enable online processing
13. Prioritisation of recommenders: How do we map the recommender to the audience segment?
14. We need to learn how to love the legacy infrastructure
16. What metadata about the content is universal across languages?
17. Text Based Content Similarity http://bbcnewslabs.co.uk/projects/news-in-space/ OFFLINE NEAR-LINE
18. Topic based content profiles Don’t we just love Latent Dirichlet Allocation? Documents are modelled as a mixture of topics. It’s basically like doing a soft clustering of the articles (it allows documents to exhibit multiple topics) and project them into smaller topics space. Topic 1
19. Semantically independent user profiles
20. This newsroom feels empty without you
21. Come and help us... • deploy machines, reinvent our objective function, build the future of the newsroom that is behind BBC for all. • be a superhero!