think about hiring, and where the technical interviews fit into the process. 2. General advice: tips, tools, and techniques that’ll help with all forms of technical interviews. 3. Technical interview techniques: 11 ways that companies test for technical skill, why companies use them, and how to tackle them eﬀectively.
Director of Technical Services, HackerOne • Formerly: Engineering Supervisor, 18F; Director of Security, Heroku. • I’ve interviewed hundreds of candidates, hired dozens, and helped multiple organizations build or improve their hiring processes.
A (the job) by asking people to complete Task B (an interview). • How do we know that performance at Task B correlates with Task A? • Spoiler alert: usually we don’t! • The best interview technique would be to actually ask them to perform the job. Unfortunately, that’s usually unrealistic. So, we find other techniques.
hiring company-wide, usually partner with the Hiring Manager to fill positions. Sometimes technical, often not. • Hiring Manager: the person responsible for hiring; usually your future boss. • Director: often the organization’s executive (Director/VP) is involved; e.g. you might interview with a Director/VP of Engineering for a job reporting to someone further down the chain. • Note: your interactions with these people are part of the interview!
• Ask questions, clarify that you understand the question • "Let me repeat what I just heard, to make sure I understand..." • It's OK to mess up! Acknowledge you made a mistake, ask to go back / start over. • Do your best to project confidence. (Great interviewers shouldn’t care, but most interviewers aren’t great.) • This should go without saying, but: be honest.
5. Take-home exercises 6. Pairing 7. Online tools 8. Whiteboarding 9. Code Review 10. Lab exercise 11. “Starter Project” RED FLAG RED FLAG RED FLAG YELLOW FLAG ⭐ ⭐ ⭐ Question/Answer Coding challenges Exercises
might use this technique. • Example: an example question using this technique. • What use of this technique tells you about the organization. You can infer some things about your potential new employer and coworkers. Poor hiring practices are often the tip of the iceberg on a toxic working environment. • Preparation tips: how best to prepare for this kind of interview. • How to give a strong answer: how interviewers evaluate answers; what strong answers look like
— questions about your past behavior. • Theory: past performance predicts future behavior. • Example: “Tell me about a diﬀicult technical problem that you helped solve: what was the problem, and how was it solved?” • Behavioral interviews are a strong positive signal about the organization’s culture. They have the most evidence backing their eﬀectiveness. They take eﬀort to use eﬀectively. So you can infer that the organization has its act together: they’re being evidence-driven, and are making a serious attempt to hire eﬀectively. ⭐
questions or general topics ahead of time (a strength of this format is that it rewards preparation and doesn’t provide any good ways to “cheat”) • Look at the job description; you can probably guess. • Common themes: problem solving, testing/refactoring, debugging, performance, deadlines/execution, conflict, feedback. • The guide I wrote at 18F has lots of examples: https://eng-hiring.18f.gov
outcome. • The interviewer is looking for a story - a specific instance, or time, you did something. Be specific, not general. • Make sure it’s relevant - summarize quickly, ask if it’s a good example. • Include details: names, dates, specific tools you used, etc. • Focus on behavior: what you did, said, etc.
candidates need not have experience with a specific thing to be good at it, so we can ask hypothetically. • Example: “imagine you’ve inherited a Django app, and it’s running very slowly — the p95 request time is 5 seconds. How would you diagnose the problem?” • These are very common, so you can’t learn much. There’s no evidence of eﬀectiveness; in fact, the data shows that people say diﬀerent things when asked about hypotheticals versus past behavior. Orgs using hypotheticals may not have thought carefully about good interviewing practices.
to behavioral; see that too. • In fact, if you can turn a hypothetical question into a specific example: you should! • If not: make sure you properly scope out and understand the hypothetical; since it’s imaginary, you need to ask questions to learn what the interviewer has in mind.
check that candidates know… things… I guess? • Example: “what algorithm does list.sort() use?” • This is a terrible technique. Success at trivia doesn’t correlate with job performance. Companies that rely on this practice are likely to have hired poorly. Proceed with caution. RED FLAG
the job: don’t waste your time. • If you must: spaced repetition is the most eﬀective technique for memorization of trivia. Several apps exist to help. • Answering: try not to stress out. Either you know the answer or you don’t. • You can try explaining how you’d look up the answer — even ask to use a phone/computer to try — but it’s unlikely that your interviewer will care; they’re likely evaluating pass/fail. RED FLAG
write some code while they watch (physically or remotely) • Theory: the best way to predict if someone can code is… to ask them to code. • Example: “Here’s a CSV file containing healthcare plan costs. It’s got fields for zip code, plan level (bronze, silver, gold), and cost. Write a command-line script that, given a zipcode, prints the second-lowest bronze-level healthcare plan in that zipcode.” (Source: https://homework.adhoc.team/slcsp/) • The theory is sound, though little evidence of eﬀectiveness exists. If the problem’s well- designed and appropriate for the role, that’s probably a strong positive signal. • Be wary of companies forcing weird environments — spyware, coding in Google Docs, etc. These are RED FLAGS
machine, if possible. If not, make sure you know what the testing machine will be. • If you’re using your machine: make sure everything’s all set up to write/run code and tests. Turn oﬀ distractions. • Practice with the tools you’ll have available. Fumbling with your editor or terminal is a bad look! • Practice writing code under time pressure. For exercises, try Project Euler, Advent of Code, etc., and set a clock.
interviewer; make sure you understand it. Ask questions until you do. Check your assumptions. • You’re being graded on your process as well as the results. So talk out loud as you do everything. Explain what and why you’re about to take each step. If you need to stop and think, say so. If you get stuck, ask for help. • Start outside-in: write the function signature / CLI script / etc., make sure the inputs and outputs are what the interviewer expects. (Sometimes this is done for you.) • Write tests next, if appropriate. Validate that the tests match the interviewer’s assumptions. • Finish the code. Don’t forgot to keep talking!
challenge and some time to work on it at your own speed. There’s a deadline, typically around 1-3 days. • Theory: We want people to write code, but coding while someone is watching is unrealistic and stressful. We’ll see more realistic results if we give people more realistic conditions. • Examples: https://homework.adhoc.team/ • Like live-coding, take-homes are on solid theoretical ground but have little evidence of eﬀectiveness. Anecdotally, they seem more eﬀective than live coding. • Good-home exercises shouldn’t take tons of time. Asking you to spend more than half a day is a — a sign that the company doesn’t respect its candidate’s time. Proceed with caution. ⭐ RED FLAG
If you get stuck: reach out to the recruiter/hiring manager immediately. • Pay close attention to better practices: tests, documentation, comments, clean code, tests. • Did I mention tests? Submit them even if you’re not asked to. • Double-check that your code runs and works before submitting. Document how to run the code even if you think it’s obvious.
with a company engineer and write the code together, pair-programming style. • Theory: we practice pair programming in real life, so let’s do it in the interview! • Pairing tells you that the organization probably practices pairing! As an interview technique it’s unusual, though it’s on the same theoretical ground as the previous two techniques.
as live coding. • Read up on, and practice pair programming if you haven’t done it before. It can be uncomfortable if you’ve never done it. • Talking is even more important. Talk about every step you take. • If it’s really pairing, you should expect your pair to write some of the code too. So if you get stuck you can really get help, this is great.
you have to use an online platform to write, submit, and grade your code. • Theory: I want to ask candidates to write code, but I’m too lazy to grade it myself. • This is a poor technique. Good coding interviews judge communication and code quality as much as they do success, but online platforms can only measure success. This indicates that the company doesn’t prioritize hiring eﬀectively. Proceed carefully. RED FLAG
with specific prep for platforms. • Exception, some of these platforms (HackerRank, TripleByte) can be good practice for you, for technical interviewing in general. Just be wary of companies forcing the issue. • If you must use one: try to get on it ahead of time, explore the environment. Some can be very awkward. • Examine your output very carefully: these platforms can be unforgiving. Sometimes even just whitespace can cause a fail, and often you have limited retries. RED FLAG
to do it on a whiteboard for some reason. • Theory: Microsoft used to do this, so… we should too? • This is a terrible, horrible, very bad practice. It has no predictive power whatsoever. Companies that force you to whiteboard are wasting your time. Proceed with caution. If you have the power, consider walking out. RED FLAG
you use a computer. • Explain you’ll probably make mistakes in syntax without a real editor. • Most tips as for live-coding apply: outside-in, talk a lot, etc. • Take advantage of it being a board: draw diagrams, etc. RED FLAG
a pull request) and asked to review it. • Theory: you can demonstrate coding competence through reading and understanding other code. And, code review is an important part of an engineer’s job, so it makes a good technical screen. • Like other coding exercises, this is on solid theoretical ground but lacks good data. It indicates that the organization puts a premium on code review, which is generally a good sign.
Some resources: https://github.com/ryanmcdermott/code-review-tips https://github.com/joho/awesome-code-review • Practice: ask a friend to review their code, or find a project with patches needing review (Django usually has plenty!)
intent with the interviewer, make sure you understand it. Remember to ask questions if things are unclear. • Start with the basics: typos, errors, readability, comments. • Then look for bigger stuﬀ: edge cases, security, performance. • Make concrete suggestions, don’t just flag errors.
a simulated environment — a running web application, a virtual machine, etc. You’re given tasks to accomplish in this simulated environment. Sometimes this is live, sometimes take-home. • Theory: direct simulation — if Task X is part of the job, create a repeatable environment for candidates to do Task X in an interview setting. • Example: “there’s a running web app at https://company.example.com/, which has several security vulnerabilities. Find as many as you can!” • Good sign. Technical lab exercises are on solid theoretical ground. Technical exercises haven’t been studied directly, but exercises more generally have (e.g., journalists submitting writing samples), and are highly eﬀective. Lab exercises take serious eﬀort to set up, and indicate that the organization is serious about hiring well. ⭐
up-front, make sure you follow and understand them. Ask questions if anything’s unclear. • Besides that… there’s little you can prep for — one of the reasons exercises are good techniques. • If you’ll be working the lab live: talk out loud as you do everything. Explain what and why you’re about to take each step. If you need to stop and think, say so. If you get stuck, ask for help.
to complete a simple but realistic project (build a real feature, fix a real bug, etc.). See also: http://www.craigkerstiens.com/2011/12/02/how-heroku-works-hiring/ • Theory: actually trial working together as a team, in a real way. • Example: when I interviewed for the Director of Security position at Heroku, I managed a real security incident (a PostgreSQL vulnerability). YELLOW FLAG
predictive, not just of technical ability but also ability to work as a team. They provide a huge amount of confidence. But they require a major time commitment from the candidate. Companies that use them are serious about de-risking hiring, and are willing to lose good candidates over it. • Projects shouldn’t last more than 2-3 days. The company should be very flexible to your schedule. Your work should not actually be used by the company. That’s not a hiring exercise, that’s unpaid labor. YELLOW FLAG
preparation is hard. • Remember that you’re being judged by your interactions with the team, as well as your technical ability. Be friendly, pleasant, and professional. Treat everyone like they’re your coworkers — they might be! YELLOW FLAG
challenges 4. Live-coding 5. Take-home exercises 6. Pairing 7. Online tools 8. Whiteboarding Exercises 9. Code Review 10. Lab exercise 11. “Starter Project” RED FLAG RED FLAG RED FLAG YELLOW FLAG ⭐ ⭐ ⭐
to feel nervous. Practice really helps: the more you do it, the easier it gets. • Remember to prepare, ask questions, and take notes! • Most companies are bad at hiring. You might do everything right and still not get an oﬀer. Try not to let it get you down!