How to Interview a Data Scientist

This O'Reilly Strata 2013 presentation discusses effective strategies for interviewing data scientists, emphasizing the importance of realistic interview practices, avoiding tricky questions, and making definitive hiring decisions. It advocates for evaluating candidates through collaborative tasks, reviewing prior work, and employing simple coding tests to ensure alignment with job requirements. Key principles include maintaining authenticity in interviews, avoiding complexity that might confuse candidates, and being decisive in hiring outcomes to prevent bad hires.

Transcript

1. Recruiting Solutions Recruiting Solutions Recruiting Solutions How to Interview a Data Scientist Daniel Tunkelang Director of Data Science, LinkedIn Daniel 1
2. Drew Conway’s Venn Diagram 2
3. GOAL 3
4. Specification for a Data Scientist 4 analyzes data implements algorithms thinks product
5. What about 5 C ulture ommunication uriosity ? Hold that thought…
6. What can you learn from an interview? 6
7. Interviewing is a last resort. 7 Alternatives?
8. Only hire people you’ve worked with. 8
9. Hire interns. Convert to full-time. Profit! 9
10. Try before you buy: short-term contracts. 10
11. Alternatives are at best a partial solution. §  Only hiring people you’ve worked with doesn’t scale. –  And traps you in a locally optimal monoculture. §  Interns are great! But they are a significant investment. –  Managing interns well is a productivity gamble. –  Most interns have at least a year of school left. –  Not all interns will make your bar. You won’t always make theirs. §  Try before you buy: nice in theory. –  Adverse selection bias when other offers are permanent roles. –  Creates bureaucracy. 11
12. Can we at least make interviews natural? 12
13. Spend a day working together. 13
14. Take-home assignment. 14
15. Review candidate’s previous work. 15
16. High-fructose corn syrup is 100% natural. §  Working sessions are difficult to set up. –  No more natural than a final exam. –  High variance, and very difficult to calibrate performance. §  Take-home assignments are great for the employer. –  But they are a significant investment for the candidate. –  Adverse selection bias if other companies don’t require them. –  Creates incentive to cheat if significant part of hiring process. §  Previous work is like natural experiments. –  Always good to review a candidate’s previous work. –  But not always possible to find work with high predictive value. 16
17. So you gotta do interviews. But how? 17
18. Three Principles 1.  Keep it real. 2.  No gotchas. 3.  Maybe = no. 18
19. Keeping It Real 19
20. Test basic coding with FizzBuzz questions. 20 1, 2, Fizz, 4, Buzz, Fizz, 7, 8, Fizz, Buzz, 11, Fizz, 13, 14, FizzBuzz, 16, … multiple of 3 -> Fizz multiple of 5 -> Buzz multiple of 15 -> FizzBuzz
21. Whiteboards suck for coding. 21 http://ericleads.com/2012/10/how-to-conduct-a-better-coding-interview/
22. Don’t ask pointless algorithm questions. 22 implement
23. Use real-world algorithms questions. 23 bigdatascientist Did you mean: big data scientist
24. Ask candidates to design your products. 24
25. Keeping it real is also a great sell. 25 Similar Profiles People You May Know
26. But no gotchas. 26
27. Gotchas reduce the signal-to-noise ratio. §  Avoid problems where success hinges on a single insight. –  Good interview problems offer lots of room for partial credit. –  Making a key insight often reflects experience, not intelligence. §  Don’t test a candidate’s knowledge of a niche technique. –  Unless that niche technique is critical to job performance. –  And can’t be learned on the job as part of on-boarding. §  Be a hard interviewer, but don’t be an asshole. –  An interview is not a stress-test to see where candidates break. –  Interviews communicate your values to the candidate. 27
28. Maybe = no. 28
29. Commit to binary interview outcomes. §  Forced choice so interviewers don’t take easy way out. –  Just like having 4 choices instead of 5 on a rating scale. –  Encourages interviewers to take their role seriously. §  Each team member is a critical filter. –  Two no’s or one strong no is a no. –  All weak yes’s is a no. §  Short-circuit candidates early in the process. –  Resume and phone screening should be aggressive. –  Onsite interviews should have ~50% chance of leading to offers. 29
30. But what about 30 ulture ommunication uriosity ? All are must-haves. Every interview evaluates all three. C
31. Remember Your Goal 31
32. Three Principles 1.  Keep it real. –  Avoid whiteboard coding. Filter with FizzBuzz. –  Use real-world algorithms questions. –  Ask candidates to design your products. 2.  No gotchas. –  Gotchas reduce the signal-to-noise ratio. 3.  Maybe = no. –  Bad hires suck. Be conservative. –  Trust your team. 32
33. Thank you! 33