Interviewing tactics for a post-LLM world

by Dunya Kirkali

Embed

Start on current slide

Slide 1

Slide 1 text

Dünya Kırkalı Interviewing tactics for a post-LLM world

Slide 2

Slide 2 text

Interviewing like it’s 2022

Slide 3

Slide 3 text

Interviewing like it’s 2022 Five stages of grief • Denial • Anger • Bargaining • Depression • Acceptance

Slide 4

Slide 4 text

Five stages of grief Denial • Ban AI • Remove take-home • Live-only obsession Photo by Saif71.com on Unsplash

Slide 5

Slide 5 text

Five stages of grief Anger • Hostile process • Gotchas • Purity tests Photo by Nsey Benajah on Unsplash

Slide 6

Slide 6 text

Five stages of grief Bargaining • Awkward constraints • Inconsistent rules • Unclear expectations Photo by charlesdeluvio on Unsplash

Slide 7

Slide 7 text

Five stages of grief Depression • Endless process changes • Low confidence • Stage bloat Photo by charlesdeluvio on Unsplash

Slide 8

Slide 8 text

Five stages of grief Acceptance • Clear process • Encouraged LLM usage • Focus on reasoning over output Photo by Darius Bashar on Unsplash

Slide 9

Slide 9 text

🌍 • Team Leader at Remote.com • Author of the • Engineering Manager’s Compass • Blogger on • Incremental Forgetting • ❤ Hiring Dünya Kırkalı

Slide 10

Slide 10 text

Why?

Slide 11

Slide 11 text

Laszlo Bock “Great people are a company's most important asset. Hiring is the most important thing you do”

Slide 12

Slide 12 text

Let’s take a step back

Slide 13

Slide 13 text

ACME Corp. • What will they be doing • Who will they work with • Which KPIs • The level of support • The level of autonomy AI Native Engineer

Slide 14

Slide 14 text

Let’s take a step back What & How • Specific skill • Specific personality • Generalist vs Specialist • Past experiences • Future potential • Gap in team • Take home • Open vs closed questions • N stages • Leetcode • Brain teasers • IQ Tests

Slide 15

Slide 15 text

KSAO

Slide 16

Slide 16 text

Knowledge Skills Abilities Other

Slide 17

Slide 17 text

Knowledge Examples • LLM Capabilities • Strengths: drafting, transformation, summarisation • Weaknesses: hallucinations, non-determinism, stale knowledge • AI Risk/Governance • Privacy, prompt injection, intellectual property

Slide 18

Slide 18 text

Skills Examples • Task Decomposition • Break ambiguous work into steps • Design prompts • Build skills • Verification of AI Output • Fact-check • Design tests

Slide 19

Slide 19 text

Abilities Examples • Curiosity • Experiment • Learning Agility • Adapt to new models/tools • Update way-of-working as the ecosystem changes

Slide 20

Slide 20 text

Other Examples • Intellectual humility • Comfortably saying “I don’t know” • Correcting model/user output • Ownership with judgment • Escalate high-risk decisions • Don’t over automate

Slide 21

Slide 21 text

Let’s revamp

Slide 22

Slide 22 text

Let’s revamp Go deep Choose one thing you’ve built and walk me through it • It’s not the results, it’s how we got there • What did they learn from the experience • What would they do differently • Generalist or Specialist • Critical thinking • Domain understanding

Slide 23

Slide 23 text

Let’s revamp Reading code Help your team mate understand what this large code base does • Ambiguity • Legacy • Documentation • LLM Usage • Context Management

Slide 24

Slide 24 text

Let’s revamp Reviewing Code Review this change request and explain what feedback you leave • Focus • Feedback • Communication • Values

Slide 25

Slide 25 text

Let’s revamp Writing an RFC Write on RFC on this topic and present it to us • Systems Thinking • Structured Reasoning • Trade-Off Analysis • Communication Clarity • Collaboration Mindset

Slide 26

Slide 26 text

Let’s revamp Systems Design Design a [ x ] system and explain to us what you’ve come up with • Have them design a system at home • Invite over for a deep-dive • Tradeoffs • Chaos • Communication • Feedback

Slide 27

Slide 27 text

Tips

Slide 28

Slide 28 text

Tips 3 Good vs 10 Mediocre • Quality over quantity • Depth over Breadth Photo by Brett Jordan on Unsplash

Slide 29

Slide 29 text

Tips Not Live • Every second is precious • What if the systems are down • Ask async - review sync Photo by Lucas Alexander on Unsplash

Slide 30

Slide 30 text

Tips Encourage LLM usage • Take home • Share sessions • Process > Output • Tradeoffs Photo by Harrison Broadbent on Unsplash

Slide 31

Slide 31 text

Tips A day at the races • Have them work along your engineers • Have them contribute to OSS • Pay them • Have them join lunch • Collaboration is essential Photo by Gene Devine on Unsplash

Slide 32

Slide 32 text

Tips Extremists • Dangerous • Present dilemma • Tradeoffs • Beliefs Photo by Web Donut on Unsplash

Slide 33

Slide 33 text

Tips Scorecards • Standardize • Close • Disclose at the end • Don’t discuss, rather update Photo by Rong on Unsplash

Slide 34

Slide 34 text

Tips Test your stages • Measure • Time • Accuracy • Try out with engineers • Try out with non-engineers • Using an LLM • A/B Test Photo by Akram Huseyn on Unsplash

Slide 35

Slide 35 text

Tips 10x • Whatever you think you’ll ask, 10x it • Issues • Users • LoC • Pull Requests • Code Complexity • Number of Services, Photo by Miguel Ángel Padriñán Alba on Unsplash

Slide 36

Slide 36 text

Recap

Slide 37

Slide 37 text

Recap • Write down the job description • Define KSAO • Prepare questions • Test internally • Run experiments • Measure success

Slide 38

Slide 38 text

Closing

Slide 39

Slide 39 text

Closing Acknowledgements • 🔥 Idea by LeadDev • 💡 Inspiration by Christopher Redmond • 💡 Inspiration by Dan Cook • 🫶 Support by Maxim Schepelin

Slide 40

Slide 40 text

https://schepelin.com/go/book

Slide 41

Slide 41 text

Closing Resources • Scalar Interviews by Dunya Kirkali • How to combat bias in the hiring process by Maxim Schepelin • Software engineering at the tipping point by Adam Bender • Work Rules! by Laszlo Bock • Make Something Wonderful by Steve Jobs • Who: The A Method for Hiring by Geoff Smart and Randy Street

Slide 42

Slide 42 text

@dunyakirkali Questions