How to Ace a Technical Interview

Given at PyCon APAC 2019

Transcript

1. How to Ace 
 a Technical Interview Jacob Kaplan-Moss jacob@jacobian.org Photo by Leone Venter on Unsplash
2. What we’ll cover 1. Orientation and overview: how most companies think about hiring, and where the technical interviews fit into the process. 2. General advice: tips, tools, and techniques that’ll help with all forms of technical interviews. 3. Technical interview techniques: 11 ways that companies test for technical skill, why companies use them, and how to tackle them effectively.
3. Who the heck is this guy? • Co-owner, REVSYS • Director of Technical Services, HackerOne • Formerly: 
 Engineering Supervisor, 18F; 
 Director of Security, Heroku. • I’ve interviewed hundreds of candidates, hired dozens, and helped multiple organizations build or improve their hiring processes.
4. Disclaimers 1. This talk won’t help you land a job if you’re not qualified. 2. Companies mostly suck at hiring. This means you can be qualified, do everything right, and still not get an offer. (I know, right?) ⚠
5. Part 1: 
 Orientation & Overview Photo by Han Chenxu on Unsplash
6. The hiring funnel Source Screen Interview Select Technical Skills Professional Skills {
7. The problem: • We’re trying to predict performance at Task A (the job) by asking people to complete Task B (an interview). • How do we know that performance at Task B correlates with Task A? • Spoiler alert: usually we don’t! • The best interview technique would be to actually ask them to perform the job. Unfortunately, that’s usually unrealistic. So, we find other techniques.
8. People you’ll encounter • Recruiter: in-house or external, responsible for hiring company-wide, usually partner with the Hiring Manager to fill positions. Sometimes technical, often not. • Hiring Manager: the person responsible for hiring; usually your future boss. • Director: often the organization’s executive (Director/VP) is involved; e.g. you might interview with a Director/VP of Engineering for a job reporting to someone further down the chain. • Note: your interactions with these people are part of the interview!
9. Part 2: 
 General Advice Photo by Joshua Davis on Unsplash
10. If you pay attention 
 to only one thing, 
 make it this: Prepare Ask questions Take notes
11. Preparation • Ask about the format, what questions you’ll be asked. • Take notes, and bring them to the interview. • Practice: mock interviews are very helpful.
12. How to find out what you’ll be asked • Ask the recruiter, hiring manager, etc • At each stage, ask what's next • Ask for questions or topics ahead of each interview • Consult Glassdoor, etc • Read the job description carefully
13. Answering questions, in the moment • Listen carefully, take notes • Ask questions, clarify that you understand the question • "Let me repeat what I just heard, to make sure I understand..." • It's OK to mess up! Acknowledge you made a mistake, ask to go back / start over. • Do your best to project confidence. (Great interviewers shouldn’t care, but most interviewers aren’t great.) • This should go without saying, but: be honest.
14. After an interview • Reflect on what you were asked (you took notes, right?), and how you think 
 you did. • If you can take it, ask for feedback (but understand why you might not get it.) • Iterate and improve on your preparation for the next one.
15. Part 3: Technical Interview Techniques Photo by Scott Webb on Unsplash
16. The techniques 1. Behavioral 2. Hypothetical 3. Trivia
 4. Live-coding 5. Take-home exercises 6. Pairing 7. Online tools 8. Whiteboarding
 9. Code Review 10. Lab exercise 11. “Starter Project” RED FLAG RED FLAG RED FLAG YELLOW FLAG ⭐ ⭐ ⭐ Question/Answer Coding challenges Exercises
17. How we’ll examine a technique • Theory: why a company might use this technique. • Example: an example question using this technique. • What use of this technique tells you about the organization. You can infer some things about your potential new employer and coworkers. Poor hiring practices are often the tip of the iceberg on a toxic working environment. • Preparation tips: how best to prepare for this kind of interview. • How to give a strong answer: how interviewers evaluate answers; what strong answers look like
18. Question & Answer interview techniques 1. Behavioral 2. Hypothetical 3. Trivia RED FLAG ⭐
19. 1. Behavioral interviews • “Tell me about a time…” questions — questions about your past behavior. • Theory: past performance predicts future behavior. • Example: “Tell me about a difficult technical problem that you helped solve: what was the problem, and how was it solved?” • Behavioral interviews are a strong positive signal about the organization’s culture. They have the most evidence backing their effectiveness. They take effort to use effectively. So you can infer that the organization has its act together: they’re being evidence-driven, and are making a serious attempt to hire effectively. ⭐
20. Preparing for behavioral interviews • The best organizations will share questions or general topics ahead of time (a strength of this format is that it rewards preparation and doesn’t provide any good ways to “cheat”) • Look at the job description; you can probably guess. • Common themes: problem solving, testing/refactoring, debugging, performance, deadlines/execution, conflict, feedback. • The guide I wrote at 18F has lots of examples: 
 https://eng-hiring.18f.gov
21. Answering Behavioral Interview Questions • A good format: situation, actions, outcome. • The interviewer is looking for a story - a specific instance, or time, you did something. Be specific, not general. • Make sure it’s relevant - summarize quickly, ask if it’s a good example. • Include details: names, dates, specific tools you used, etc. • Focus on behavior: what you did, said, etc.
22. 2. Hypothetical questions • Like behavioral questions, but forward-looking. Theory: candidates need not have experience with a specific thing to be good at it, so we can ask hypothetically. • Example: “imagine you’ve inherited a Django app, and it’s running very slowly — the p95 request time is 5 seconds. How would you diagnose the problem?” • These are very common, so you can’t learn much. There’s no evidence of effectiveness; in fact, the data shows that people say different things when asked about hypotheticals versus past behavior. Orgs using hypotheticals may not have thought carefully about good interviewing practices.
23. Preparing for hypotheticals • Pretty much the same as for behavioral.
24. Answering hypothetical interview questions • Strong answers look very similar to behavioral; see that too. • In fact, if you can turn a hypothetical question into a specific example: you should! • If not: make sure you properly scope out and understand the hypothetical; since it’s imaginary, you need to ask questions to learn what the interviewer has in mind.
25. 3. Trivia • Questions about specific technical details. • Theory: check that candidates know… things… I guess? • Example: “what algorithm does list.sort() use?” • This is a terrible technique. Success at trivia doesn’t correlate with job performance. Companies that rely on this practice are likely to have hired poorly. Proceed with caution. RED FLAG
26. Tips for trivia • Prep: Unless you really, really need the job: don’t waste your time. • If you must: spaced repetition is the most effective technique for memorization of trivia. Several apps exist to help. • Answering: try not to stress out. Either you know the answer or you don’t. • You can try explaining how you’d look up the answer — even ask to use a phone/computer to try — but it’s unlikely that your interviewer will care; they’re likely evaluating pass/fail. RED FLAG
27. Coding Challenges 4. Live-coding 5. Take-home exercises 6. Pairing 7. Online tools 8. Whiteboarding RED FLAG RED FLAG ⭐
28. 4. Live coding • The interviewer will ask you to write some code while they watch (physically or remotely) • Theory: the best way to predict if someone can code is… to ask them to code. • Example: “Here’s a CSV file containing healthcare plan costs. It’s got fields for zip code, plan level (bronze, silver, gold), and cost. Write a command-line script that, given a zipcode, prints the second-lowest bronze-level healthcare plan in that zipcode.” 
 (Source: https://homework.adhoc.team/slcsp/) • The theory is sound, though little evidence of effectiveness exists. If the problem’s well- designed and appropriate for the role, that’s probably a strong positive signal. • Be wary of companies forcing weird environments — spyware, coding in Google Docs, etc. These are RED FLAGS
29. Preparing for live coding • Ask to use your own machine, if possible. If not, make sure you know what the testing machine will be. • If you’re using your machine: make sure everything’s all set up to write/run code and tests. Turn off distractions. • Practice with the tools you’ll have available. Fumbling with your editor or terminal is a bad look! • Practice writing code under time pressure. For exercises, try Project Euler, Advent of Code, etc., and set a clock.
30. Doing live-coding exercises • Talk through the problem with the interviewer; make sure you understand it. Ask questions until you do. Check your assumptions. • You’re being graded on your process as well as the results. So talk out loud as you do everything. Explain what and why you’re about to take each step. If you need to stop and think, say so. If you get stuck, ask for help. • Start outside-in: write the function signature / CLI script / etc., make sure the inputs and outputs are what the interviewer expects. (Sometimes this is done for you.) • Write tests next, if appropriate. Validate that the tests match the interviewer’s assumptions. • Finish the code. Don’t forgot to keep talking!
31. 5. Take-home exercises • Like live-coding, except you’re given the challenge and some time to work on it at your own speed. There’s a deadline, typically around 1-3 days. • Theory: We want people to write code, but coding while someone is watching is unrealistic and stressful. We’ll see more realistic results if we give people more realistic conditions. • Examples: https://homework.adhoc.team/ • Like live-coding, take-homes are on solid theoretical ground but have little evidence of effectiveness. Anecdotally, they seem more effective than live coding. • Good-home exercises shouldn’t take tons of time. Asking you to spend more than half a day is a — a sign that the company doesn’t respect its candidate’s time. Proceed with caution. ⭐ RED FLAG
32. Preparing for take-home exercises • Much the same as live-coding, except asynchronous. Be sure you really understand the problem before diving in. • Some good practice exercises: • https://homework.adhoc.team/ • https://adventofcode.com/ • https://projecteuler.net/
33. Doing take-home challenges • Mostly: just write code normally. • If you get stuck: reach out to the recruiter/hiring manager immediately. • Pay close attention to better practices: tests, documentation, comments, clean code, tests. • Did I mention tests? Submit them even if you’re not asked to. • Double-check that your code runs and works before submitting. Document how to run the code even if you think it’s obvious.
34. 6. Pairing • Like live coding, but you’ll be paired with a company engineer and write the code together, pair-programming style. • Theory: we practice pair programming in real life, so let’s do it in the interview! • Pairing tells you that the organization probably practices pairing! As an interview technique it’s unusual, though it’s on the same theoretical ground as the previous two techniques.
35. Tips for pairing sessions • Prep is mostly the same as live coding. • Read up on, and practice pair programming if you haven’t done it before. It can be uncomfortable if you’ve never done it. • Talking is even more important. Talk about every step you take. • If it’s really pairing, you should expect your pair to write some of the code too. So if you get stuck you can really get help, this is great.
36. 7. Online Tools • Like the previous coding exercises, except you have to use an online 
 platform to write, submit, and grade your code. • Theory: I want to ask candidates to write code, but I’m too lazy to grade it myself. • This is a poor technique. Good coding interviews judge communication and code quality as much as they do success, but online platforms can only measure success. This indicates that the company doesn’t prioritize hiring effectively. Proceed carefully. RED FLAG
37. Tips for platform-based challenges • Prep: mostly, I wouldn’t bother with specific prep for platforms. • Exception, some of these platforms (HackerRank, TripleByte) can be 
 good practice for you, for technical interviewing in general. Just be wary 
 of companies forcing the issue. • If you must use one: try to get on it ahead of time, explore the environment. Some can be very awkward. • Examine your output very carefully: these platforms can be unforgiving. Sometimes even just whitespace can cause a fail, and often you have limited retries. RED FLAG
38. 8. Whiteboarding • Like live coding, but they force you to do it on a whiteboard for some reason. • Theory: Microsoft used to do this, so… we should too? • This is a terrible, horrible, very bad practice. It has no predictive power whatsoever. Companies that force you to whiteboard are wasting your time. Proceed with caution. If you have the power, consider walking out. RED FLAG
39. Tips for whiteboarding • Try to get them to let you use a computer. • Explain you’ll probably make mistakes in syntax without a real editor. • Most tips as for live-coding apply: outside-in, talk a lot, etc. • Take advantage of it being a board: draw diagrams, etc. RED FLAG
40. Exercises 9. Code Review 10. Lab Exercise 11. “Starter Project” ⭐ YELLOW FLAG
41. 9. Code Review • You’re given some example code (or a pull request) and asked to review it. • Theory: you can demonstrate coding competence through reading and understanding other code. And, code review is an important part of an engineer’s job, so it makes a good technical screen. • Like other coding exercises, this is on solid theoretical ground but lacks good data. It indicates that the organization puts a premium on code review, which is generally a good sign.
42. Preparing for code review • Study good code review technique. Some resources:
 https://github.com/ryanmcdermott/code-review-tips
 https://github.com/joho/awesome-code-review • Practice: ask a friend to review their code, or find a project with patches needing review (Django usually has plenty!)
43. Reviewing code in an interview • Talk through the code’s intent with the interviewer, make sure you understand it. Remember to ask questions if things are unclear. • Start with the basics: typos, errors, readability, comments. • Then look for bigger stuff: edge cases, security, performance. • Make concrete suggestions, don’t just flag errors.
44. 10. Lab Exercises • Any sort of hands-on exercise with a simulated environment — a running web application, a virtual machine, etc. You’re given tasks to accomplish in this simulated environment. Sometimes this is live, sometimes take-home. • Theory: direct simulation — if Task X is part of the job, create a repeatable environment for candidates to do Task X in an interview setting. • Example: “there’s a running web app at https://company.example.com/, which has several security vulnerabilities. Find as many as you can!” • Good sign. Technical lab exercises are on solid theoretical ground. Technical exercises haven’t been studied directly, but exercises more generally have (e.g., journalists submitting writing samples), and are highly effective. Lab exercises take serious effort to set up, and indicate that the organization is serious about hiring well. ⭐
45. Lab exercise tips • You’ll typically get some preparation steps up-front, make sure you follow and understand them. Ask questions if anything’s unclear. • Besides that… there’s little you can prep for — one of the reasons exercises are good techniques. • If you’ll be working the lab live: talk out loud as you do everything. Explain what and why you’re about to take each step. If you need to stop and think, say so. If you get stuck, ask for help.
46. 11. “Starter Projects” • Work directly with your potential team to complete a simple but realistic 
 project (build a real feature, fix a real bug, etc.). See also:
 http://www.craigkerstiens.com/2011/12/02/how-heroku-works-hiring/ • Theory: actually trial working together as a team, in a real way. • Example: when I interviewed for the Director of Security position at Heroku, I managed a real security incident (a PostgreSQL vulnerability). YELLOW FLAG
47. 11. “Starter Projects” • Starter Projects are complicated. They’re highly predictive, not just of technical ability but also ability to work as a team. They provide a huge 
 amount of confidence. But they require a major time commitment from the candidate. Companies that use them are serious about de-risking hiring, and are willing to lose good candidates over it. • Projects shouldn’t last more than 2-3 days. The company should be very flexible to your schedule. Your work should not actually be used by the company. That’s not a hiring exercise, that’s unpaid labor. YELLOW FLAG
48. Starter project tips • Like lab exercises (but moreso): specific preparation is hard. • Remember that you’re being judged by your interactions with the team, as well as your technical ability. Be friendly, pleasant, and professional. Treat everyone like they’re your coworkers — they might be! YELLOW FLAG
49. Techniques: Recap Question/Answer 1. Behavioral 2. Hypothetical 3. Trivia
 Coding challenges 4. Live-coding 5. Take-home exercises 6. Pairing 7. Online tools 8. Whiteboarding
 Exercises 9. Code Review 10. Lab exercise 11. “Starter Project” RED FLAG RED FLAG RED FLAG YELLOW FLAG ⭐ ⭐ ⭐
50. Final thoughts • Interviews are weird and stressful. It’s normal to feel nervous. Practice really helps: the more you do it, the easier it gets. • Remember to prepare, ask questions, and take notes! • Most companies are bad at hiring. You might do everything right and still not get an offer. Try not to let it get you down!
51. Good luck! Jacob Kaplan-Moss jacob@jacobian.org Photo by Jeremy Bishop on Unsplash Deck background by Joanna Kosinska on Unsplash