Thank God it's not Leetcode: Interviewing in the age of AI

Transcript

1. Thank God it’s not LeetCode Interviewing in the age of AI
2. 01　About me 02　Recruiting overview 03　What’s Gilded Rose 04　Why Gilded Rose 05　Our results so far 06　Things you can do starting tomorrow 07　Q&A
3. 01 About me
4. Luigi Taira D. Brazilian living in Japan for since 2017 Tech Lead at KATIX Outside of work I’m also a dad to a family of four I enjoy playing the drums, chess, and building hobby compilers Speaker
5. About KATIX A platform that helps users sell their vehicles safely online. USERS PARTNERS
6. 02 Recruiting Overview KATIX,Inc
7. Recruiting at KATIX
8. Recruiting at KATIX Being able to write code
9. Being collaborative and open to feedback Recruiting at KATIX Being able to write code
10. 2021 1Q 2Q 3Q 4Q 2022 1Q 2Q 3Q 4Q 2023 1Q 2Q 3Q 4Q 2024 1Q 2Q 3Q 4Q 2025 1Q 2Q (計 画 ) 3Q (計 画 ) 4Q (計 画 ) 0 1,000,000,000 2,000,000,000 3,000,000,000 4,000,000,000 5,000,000,000 6,000,000,000 180億円 130億円 88億円 51億円 29億円 (計画) Gross Merchandise Value （GMV） KATIX’s growth
11. of all respondents are using or are planning to use AI tools in their development process this year Stack Overflow 2024 Developer Survey 76% Let's talk about AI
12. Only of developers would openly use AI during the recruitment process. 19% Let's talk about AI
13. Only of developers would openly use AI during the recruitment process. 19% of developers actually consider the use of AI during an interview or technical test to be cheating. 1 / 3 Let's talk about AI
14. Let's talk about AI
15. Soft Skills of developers and of recruiters say that soft skills are at least as important as hard skills. 78% 81%
16. Soft Skills TokyoDev 2024 survey lists “Coworker Relations” as the third most voted for Workplace Perk, just behind “Work- life Balance” and “Working conditions”.
17. CV check, casual interview, 日本語 etc. First Interview (EM) About the company and the candidate’s background Technical Interview Technical skills, communication and emotional intelligence CEO Interview About leadership, the future, and mutual expectations
18. Why not Leet Code? We’ve had people who have great Leetcode results but performed poorly on the job. What we’ve seen at KATIX Interview Coder is an AI assisted tool that helps people do well in Leetcode style interviews. It works! Evidence that people don’t like these kinds of interviews.
19. 03 What’s Gilded Rose KATIX,Inc
21. Gilded Rose is the legendary fantasy item shop from the classic coding kata. Every item in the shop ages differently: some grow in value, some decay, and some have special rules specific for them. The shopkeeper needs an automated system to update each item’s quality and sell-by date every night. The code base is a tangled mess, left behind by a mysterious dev who vanished into the mountains. The Item class is off-limits. Touch it and a goblin will one-shot you.
23. Implement a new Item type
24. Implement a new Item type Refactor the code
25. Implement a new Item type Refactor the code 2 hours take home assignment
26. Implement a new Item type Refactor the code 2 hours take home assignment + 1 hour interview
27. 04 Why Gilded Rose KATIX,Inc
28. Why Gilded Rose RailsConf 2014 All the Little Things by Sandi Metz nateberkopec/gilded_rose
29. Or at least the two hour take home assignment is not the skill test The two hour take home assignment is just for the person to get familiar with how the Item shop works. The way they guide us through the changes and talk about possible next steps is way more important than the final code This is one of the reasons we consider this interviewing style to be resilient to people using AI. Gilded Rose is not the skill test Take home assignment In person interview
30. Its very easy to adapt the one hour interview to each candidate Gilded Rose is very flexible
31. Lead engineer Younger, high potential engineer
32. What are the design choices? Is the code object oriented or is it leaning more functional? Are there any design patterns we can try to implement? How can we minimize mutation? High potential engineers
33. Can we make this easier for people to add new items? How can the type system help us avoid certain bugs? We expect more complete answers! Lead engineers
34. You’re there to help them! During the one hour talk, think less “interview” and more “pair programming partner” When somebody is clearly having trouble and are overthinking something I will often help them clarify their thinking. If they’re having trouble deciding between two possible refactor paths, I just ask them to tell me about both and we can talk about them together.
35. What kind of person does well Is this a module inside a monolith or a microservice? Are there any databases or ORMs available? Are we worrying about performance? being on the safe side by making smaller changes clarifying naming conventions Asking lots of questions
36. Be goal oriented ‘I chose to extract this class because it reduces cognitive load and allows future features without touching core logic.’ ‘I thought it looked cleaner.’
37. About people using AI Again, it’s fine to use AI in the code and even in the PR description. We don’t change the interview at all even for people who are clearly using AI. AI code is not perfect though, and people who did hand out code generated by AI will be asked to explain it. If they can explain it it’s fine, it just becomes a problem when they can’t.
38. 05 Our results so far KATIX,Inc
39. Our results so far Again, we’re growing but are still a startup with just 50 people (10 developers) So I don’t have huge impressive numbers to show. What do we have: a very collaborative team that works really well together, where everyone is focused on the same shared goal.
40. We’ve had great feedback It's the first time I had fun while doing a technical interview. I've actually learned something from this interview. I thought it was going to be something like Leetcode. Thank God it wasn't!
41. Less false positives We’ve had higher confidence in hiring decisions. Because the conversation after the take-home reveals how someone reasons and communicates, we’ve had far fewer “false positives” (people who ace the code but struggle on the team). Even with only about a dozen interviews a month, the trend is clear: the engineers we’ve hired through this process have become key contributors and future leads.
42. The team we’ve built at KATIX We’re tackling BIG hard problems. Our Scrum roadmap is packed with challenging engineering problems. Our team of just 10 people are the base for over 3000 bikes sold a month for almost 2 billion yen.
43. 06 Things you can do starting tomorrow KATIX,Inc
44. Things you can do starting tomorrow Embrace AI Explicitly score soft skills
45. Q&A We’re hiring!