Using Kaggle in the business world

These slides give some insight into Adzuna's experience of using Kaggle to develop a machine learning algorithm for job salary prediction.

Transcript

1. Using in the business world James Neave, Head of Data Science james@adzuna.com April 2015
2. Topics • What is Adzuna? • What is Kaggle? • What was the goal of the project? • How it all worked • The deliverable • Learning outcomes 2
3. What is Adzuna? 3
4. What is Adzuna? • A search engine for jobs • 95% coverage across 11 countries • Powers the No. 10 dashboard • An independent, London-based startup (launched 2011) 4 UK site • Approx. 1m jobs • Hundreds of concurrent users • Over 1m registered jobseekers Adzuna traffic growth >6m user visits per month
5. What is Kaggle? 5
6. What was the goal of the project? • Predict salaries for job adverts 6
7. Data provided 7 Job advert details
8. Data provided 8 Job advert details Location tree
9. How it all worked 1. Proposal 2. Description 3. Evaluation 4. Rules 5. Prizes 6. Timeline 9 • Kaggle StartUp programme • Structure • Evaluation
10. How it all worked 1. Proposal 2. Description 3. Evaluation 4. Rules 5. Prizes 6. Timeline 10 • Overview, commercial relevance • Goal • Data sets
11. How it all worked 1. Proposal 2. Description 3. Evaluation 4. Rules 5. Prizes 6. Timeline 11
12. How it all worked 1. Proposal 2. Description 3. Evaluation 4. Rules 5. Prizes 6. Timeline 12 • Mean Absolute Error • File format for submission
13. How it all worked 1. Proposal 2. Description 3. Evaluation 4. Rules 5. Prizes 6. Timeline 13 • One account per participant • No private sharing • Submission limits • Team mergers • Additional submission data
14. How it all worked 1. Proposal 2. Description 3. Evaluation 4. Rules 5. Prizes 6. Timeline 14 • 1st - $3,000 • 2nd - $2,000 • 3rd - $1,000
15. How it all worked 1. Proposal 2. Description 3. Evaluation 4. Rules 5. Prizes 6. Timeline 15 • Launch date • Submission deadline: validation data • Release of test file • Submission deadline: test file
16. How it all worked – Part 2 • Training and validation data sets released • Benchmark results shared (mean and random forest model) • Teams register and get down to it • Submissions made, results posted on leader board • Forum chat commences… • Test data set released • Teams’ final submissions made • Models evaluated • Winners determined! 16
17. The deliverable 17
18. The winning model • We did a Netflix 18 “Why Netflix Never Implemented The Algorithm That Won” http://goo.gl/jr2z1x Volodymyr Mnih, University of Toronto
19. Learning outcomes • Data quality • Tight problem definition • Risks on implementation 19
20. Should you use Kaggle? Perhaps! • Suitable data & clear problem definition? • Business goals and data can be public? • Do you have someone to manage it? • Do you have someone to understand it? (at least partly) • Can you afford it? • Do you need ongoing model development? • Can you build it yourself? • Can you host the competition yourself? 20
21. Questions? 21