Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Coding interviewers from the other side of the whiteboard

Adam Barr
February 26, 2021

Coding interviewers from the other side of the whiteboard

Talk at ConFoo on February 26, 2021. "Most advice about coding interviews is designed to help the candidate get a job. However, one day you may be the interviewer, expected to administer and evaluate a coding interview. What makes a good coding question? What are you looking for in the candidate's response? How much of a hint should you give? This talk will draw on the speaker's 20+ years of experience at Microsoft, where he interviewed hundreds of programming candidates."

Adam Barr

February 26, 2021
Tweet

Other Decks in Programming

Transcript

  1. Coding interviewers from the other side of the whiteboard Adam

    Barr Crosslake Technologies https://speakerdeck.com/adamba
  2. Introductory Quiz “Why are manhole covers round?” Was this: A.

    A question Microsoft used when evaluating college graduates for full-time employment B. A topic that was discussed in my son’s second-grade class
  3. Introductory Quiz #2 “Write the code for Kanji backspace” Was

    this: A. A question asked of college seniors to determine if they should be flown back to Microsoft’s main campus for further interviews B. The final technical challenge given to a candidate with industry experience to determine if they should be hired at the senior (~10 years experience) or principal level (~15 year experience)
  4. About me • 23+ years at Microsoft  Various development

    jobs • Now work at Crosslake Technologies  Technology due diligence consultant • Wrote 3 books: Working at Microsoft Reading code What is taught in a CS program is not what developers do all day
  5. About that last book… • “What is taught in a

    CS program is not what developers do all day” • The corollary is that having a CS degree doesn’t mean you are a competent coder • Doctors, other engineers, etc don’t have to deal with this • This is why we have to do coding interviews!
  6. What you are not looking for • Some perfect algorithm

    you have in mind  “Premature optimization is the root of all evil” – Donald Knuth  “Optimization is the root of all evil” – Adam Barr • Specific knowledge of the language or runtime that everybody Googles anyway • Perfect syntax
  7. What you are looking for • Can they lay out

    code • Can they talk about the code • Can they debug the code • Can they improve the code • Can they test the code
  8. You can ask algorithm questions! • Just don’t make that

    the coding question; ask it as a theoretical question without needing to code
  9. Setting up the coding problem • Don’t play the “they

    didn’t probe for details” game  This is a trick that favors candidate who know to expect it • Write the problem down!  This was an accommodation Microsoft discussed for blind/deaf candidates, which winds up benefitting everybody • OK to use an online coding tool, even in person  But be able to see what they type • OK to let them Google—why not? (not for the whole solution) • Strongly encourage them to talk out loud  “Commentary coding”
  10. Don’t worry so much about specific knowledge… • Most things

    can be learned on the job  Domain expertise (e.g. SQL)  Language syntax • Admittedly, some basic concepts like arrays, linked lists, loops can reasonably be expected… • But what about recursion? Or O(n) analysis? Or pointers? Or bit manipulation? Or two’s complement representation?
  11. …focus on their ability to work through a problem “Creative

    thinkers make many false starts, and continually waver between unmanageable fantasies and systematic attack.” Harry Hepner, psychologist and personnel researcher “To deal effectively with puzzles (and with the bigger problems for which they may be a model), you must operate on two or more levels simultaneously. One thread of consciousness tackles the problem while another, higher-level thread monitors the progress. You need to keep asking yourself ‘Is this approach working? How much time have I spent on this approach, and how likely is it to produce an answer soon? Is there something else I should be trying?’” William Poundstone, How Would You Move Mount Fuji?
  12. Attributes of a good coding question Solvable Explanation Non-trivial Algorithm

    Testable Expandable (And, it has to be small enough that the explanation, solution, expansion, etc. can be done in the time allotted)
  13. Solvable The solution offers choices, and is something the candidate

    can work out incrementally (not an “Aha!” they get or don’t) You have to be open to multiple solutions
  14. Explanation The program should require some explanation on your part

    (to see if they have the capacity to learn it), but not too much Be careful of questions if the candidate doesn’t know something basic to solving it (such as two’s complement notation)
  15. Non-trivial Question has to present a reasonable challenge to solve

    and explain Wisdom for coding questions is it should require 3 or more variables
  16. Testable Offers an opportunity to talk about what unit tests

    they would write, and what complete tests Ideally could discuss possible limits
  17. Expandable Can follow up by introducing new constraints, discussing optimizations,

    scaling up, etc. This is the ultimate guard against “The candidate already knew the question”
  18. Attributes of a bad coding question It depends on an

    “Aha!” moment or has a single solution Problem is too trivial or too hard to explain It isn’t complicated enough Can’t discuss the algorithm Doesn’t lead to a test discussion No opportunity to expand the problem
  19. Sample coding questions • Sort an array  Pretty good,

    but easy to explain • Strlen(), strcmp()  Not non-trivial; algorithm usually obvious • Remove duplicates from an array  Not too difficult, but no obvious weakness • Linked list manipulation  Algorithm may be too easy • Convert day in year to month and day  Hard to expand beyond basic problem • Draw a line between two points  May need too much explanation • Etc… Solvable More than one! Explanation Some, not too much Non-trivial 3 variables? Algorithm Requires explanation Testable Unit, complete Expandable Constrain, optimize
  20. Evaluating a candidate • First step: Be quiet  Let

    the candidate write code in peace  Watching and listening to what they do is very valuable  If you see a bug, wait to see if they catch it first  An interview is not about showing off how smart you are • Second step: understand their answer  Understand how the code works  Don’t be reading email while they are coding!  A language you aren’t familiar with is fine as long as you can understand their code  If you need more time, ask for it
  21. Evaluating a candidate (part 2) • You have to ask

    follow up questions  This is where the “real” interview happens  Cut off their coding time if needed • Ask candidate to explain it • Ask a question to probe their understanding • Ask if they are confident their solution is correct • If there is a bug they missed, ask increasingly obvious questions about it • Now expand further (improve, constrain, test, limits, etc) • Special note: If they call a library function to do all the work, that’s cute, but it shows knowledge, not capacity/motivation
  22. Responding to complaints that “this is not what programmers do

    all day” • First of all, people who complain during an interview are probably not people you want to work with • For more experienced candidates, still OK to ask a coding question, but do want to focus on higher-level questions as well  Don’t expect them to remember every language they ever used • Also, is there a better way? This is the worst way to interview programmers, except for every other way
  23. Take-home coding questions • Somebody else may write it for

    them (minor problem) • Don’t see their thought process in writing it (major problem) • Too complicated to talk about in the interview (major problem)
  24. Asking them to read a code sample • Sure, why

    not • Programmers spend more time reading code than writing code • Can help validate their claimed experience (e.g. “I know React”) • To see if they can “think like a computer” for small segments, which is the key debugging skill (as I said in my book…)
  25. Two things you should not think “I was hired, so

    the interview system must work” “I was hired, so I must already be a good interviewer ”
  26. Gut instincts • Some evidence that interviewers make judgments in

    the first 15 seconds • Yes, this can happen in coding interviews also  “They just needed a small hint” vs “They got stuck and I had to help them” • Confidence that they can solve the problem is a useful skill
  27. Should you ask everybody the same question? • Allows better

    comparisons • Gets away from gut instinct, reduces “spinning” of results • Remember: Have questions that work even if the candidate knows them beforehand • Related topic: Should you choose a question that is drawn from your current work?  Answer: It depends…  In either case, you will probably be better at this than the candidate
  28. Other advice • Don’t ask estimation (aka Fermi) questions (“How

    many gas stations in the US”) • Don’t ask logic puzzles (“A wolf, a sheep, and a cabbage walk into a bar…”) • Don’t read https://en.wikipedia.org/wiki/Coding_interview • Remember that you have to provide feedback after the interview • Remember to cover the non-coding (“soft”) skills!  Don’t evaluate communication by “they communicated their code well”, and collaboration by “they wanted to work to solve the problem with me”, etc
  29. Call to action • Don’t interview for specific knowledge •

    Think about SENATE when choosing questions  (And feel free to suggest a new mnemonic!) • Follow up after they write the code • Don’t ignore soft skills • Practice