Building Data Culture in your workplace

Transcript

1. Building Data Culture in your workplace 羅經凱 副理理, KKStream Data Analytics 2017-06-30
2. Self Introduction • 羅經凱, Jing-Kai Lou • Mining knowledge from data and turning it into decisions. • Data Scientist, KKStream, 2016 - now • Data Scientist, KKBOX Research Center, 2014 - 2016 • PhD. NTU EE, 2014 Ronald Coase: “If you torture the data long enough, it will confess.”
3. Communicate A way to find niche market How our title performs Which kinds are alluring How to measure the satisfaction instead of popularity? How users binge-watch This measure leads us to find fascinating titles Better user preference understanding Observe & Measure Analysis & Forecast Smart content purchase 智慧擴充片庫
4. Communicate In early stage, know which one to promote and which one to abandon Promote which titles? Expose on ones potentially grasp eyes Monitor the watch hours Build trend forecast model (according to first 3 weeks performance) Precise marketing ⾏行行銷時機掌握 Analysis & Forecast Observe & Measure
5. Communicate Be confident to release better version Which rec. sys performs better? CTR of related items Apply A/B testing To tell which one is better Product Optimization 產品優化 Observe & Measure Analysis & Forecast
6. Communicate Making decision Verifying hypothesis, Asking right questions Acquiring the needed data Making hypothesis, Finding patterns Building predictive models Analysis & Forecast Observe & Measure
7. Helping someone see things from your perspective could be the easiest way to win an argument.
8. Building a data culture — O’Reilly Media
9. The problem: what do we do with data? The solution: train your org. to adapt data culture People Process Product Data access Regular meeting Reporting structure Set measurable goal Act on insights Utilize analytics and productivity tools
10. Using Data Effectively
11. 2 Factors for Effectively Using Data Selecting the right data Building the capacity, competence, and confidence of staff to effectively use data
12. Data democratization Data democratization is the ability for information in a digital format to be accessible to the average end user. The goal of data democratization is to allow non-specialists to be able to gather and analyze data without requiring outside help.
13. benefits of giving company-wide access to data outweigh the costs Facebook was one of the first companies to give its employees access to data at scale World bank makes its data open so volunteers can come together to clean and interpret it.
14. Why Data Game?
15. Content-based recommendation
16. U1, U2, U3, U4 U1, U2, U3, U5 U3, U7, U8, U9
17. Seed 相似度 線上版
18. 精準 (Accuracy) / 多元 (Diversity) / 新穎 (Novelty) 精準，泛指根據⽤用⼾戶過去歷史記錄，是否可以預測到⽤用⼾戶下⼀一部看到什什麼？舉例例來來說，假使有六個⽉月 的資訊，我們隱藏後兩兩個⽉月的資訊，單純憑藉前四個⽉月的資訊預測後兩兩個⽉月的發展。 多元，推薦應避免單⼀一⼝口味的推薦，充分展現我們⼿手上的資源(coverage)。我知道這⽤用⼾戶喜歡看超級 英雄，但是不能永遠只推薦超級英雄的電影。 新穎，拋出的新物品（新劇，或者是鮮少被⼈人看過的影劇）能夠使⽤用⼾戶產⽣生正⾯面的情緒反應。這也是 ⽬目前最難的部分，也正是⼤大家正在努⼒力力的議題之⼀一。（學者認為單純仰賴 CTR ，則放⼤大了了推薦系統的 效果，⾒見見⽂文）
19. Helping someone see things from your perspective could be the easiest way to win an argument.
20. Data + Game → Arouse the awareness and interests of data we have…
21. 提供數萬⽤用⼾戶每段觀影體驗的起始時間以及歷時。參參賽者將以前四個⽉月的 資料為分析基⽯石，推測⽤用⼾戶在接下來來⼀一個⽉月花最長時間觀看的是哪部劇
22. Training set Training labels Testing set Testing labels Submission Testing labels Public Score Private Score Evaluation: accuracy (user 1, title A) (user 2, title B) (user 2, title B) Acc. = 0.33 You can make another guess depending on your previous ones
23. Good job, Kaggle...
24. This 7-day game has 32 teams 77 players 102 downloads 409 submissions
26. Why not public?
27. This 14-day game has 63 teams 81 players 334 downloads 835 submissions Internal champion
28. Ronald Coase: “If you torture the data long enough, it will confess.”
30. Gains & Insightful findings from this public game
31. Media Exposure on FB Performance in the past 28 days 22X Reach 24X Page followers
32. Heuristic strategy method 1. the most popular title. (17%) method 2. pick one of the last 4 titles. (23%)
33. Champion KST’s secret sauce https://github.com/kstseng/kkbox-data-game-17.06 http://gmail17958.d.pr/lxOClZ all credits belong to KST (David Tseng)
34. First-step Observation In training dataset, 27% customers’ labels = the last one saw in history views 37% customers’ labels = one appeared in history views 18% customers’ labels = one never appeared in training set
35. Naïve Baseline Just fill in the last title id in view history for each individual You get 27%, namely, rank 20th
36. Comments from a participant
37. Transition Matrix In training data, we observe how users view over titles time 甄嬛傳 甄嬛傳 甄嬛傳 琅琊榜 ⽉月薪嬌妻
38. Benefits from Transition Matrix As Collaborate filtering and Matrix Factorization would not obey our finding (the last one is the answer in most cases). The transition matrix method supports the our finding! So, base on it, we have high confidence to improve the score higher than baseline. 0.27421
39. Next Observation Consider it as a sequential problem, we overlook the spent time on each title. We find individuals spent time differently on titles. For some, they only view no longer than 5 mins, and never watch it again. Longer spent time = Favorite
40. XGBoost One decision model commonly used in competition. It runs fast. So, you can interplay with data quickly. We formulate the questions as a multiple classification problem. According to the time customer spent (feature), we pick top 40 titles and 1 (as the other) as labels to classify.
41. Transition Matrix + XGBoost 0.28562
42. How to select your submissions Offline evaluation DO NOT over-fitting your result! Cross-validation would be your best friend. Training set Training set Validation set
43. Take Away * 企業內部，讓⼈人⼈人對資料有更更多的互動，創造價值。 * 企業外部，點起業界的⼀一把火，推向 democratization，在 有限制前提下，製造雙贏。 * 企業解決問題 * 外部⼈人才建立 reputation * 我們正在蘊釀下⼀一場 Data Game，敬啟期待！