I run Vibeladder, a competition where people get an unseen task, 90 minutes and any AI tool they like, and the finished apps are compared head to head. After four rounds of reading every session prompt by prompt, I keep coming back to a question that has little to do with the competition: how do you hire a developer now?
Most technical interviews were built to answer one question: can this person write correct code under some pressure. Whiteboard algorithms, timed coding tests and take-home projects all look at the code. That made sense when the code was the hard part.
It no longer is. An agent writes a working first version of most small apps in minutes. That leaves the interview with two bad options. Ban AI, and you are testing a skill the job no longer uses day to day. Allow it, and the take-home comes back looking much like every other candidate's, because a capable model did most of the work.
Neither tells you what you actually want to know: how this person will get good work out of the tools you hand them on their first day.
In Season 1 the code itself was rarely what separated entries. In Round 3, a spending dashboard checked against an answer key, both entries had every figure right to the cent. In Round 4, a spreadsheet, all three formula engines gave identical, correct results on every probe we tried. Not one of the four winning sessions has a line of code typed by the person.
What decided rounds was visible in the transcripts:
None of these is a coding skill in the old sense. All of them show up in a transcript, and none of them shows up in the finished file alone.
If the finished code no longer separates candidates, the work that produced it has to. In practice that means the session transcript becomes the main thing you assess. What I would look for in it:
In the competition we read the top transcripts by hand every round, and the most useful part of each write-up turned out to be the winner's session, annotated prompt by prompt. An interviewer can get the same from a candidate's session.
There is a catch, and Season 1 has an example of it too. One finalist in the spreadsheet round tested hard as a user, found real problems, fixed them, and still submitted an app where a value committed in the formula bar was written again into the next cell clicked. Testing through the interface finds symptoms. It does not always find causes.
That is acceptable in a 90-minute build. It is not acceptable in production at two in the morning, when a payment is being written to the wrong account. Somebody has to open the code, understand it, and decide what is actually wrong.
So the skill has moved rather than disappeared. It used to be writing the code. Now it is knowing when to stop trusting the output and look underneath, and being able to when you do.
If I were hiring for a team that builds with AI, this is roughly what I would do:
If you can, do it more than once. One session is noisy. In Season 1 the top two traded wins round by round, and it took four rounds and a rating across all of them to separate them.
A 90-minute build is one slice of the job. It says little about working in a team, designing a system that has to live for years, or maintaining code someone else wrote. And our sample is small: three people entered Season 1, so everything above is a set of examples, not data.
That is also why it runs in public. Every task, entry, verdict and winning session from Season 1 is published, so you can decide for yourself whether the session tells you more than the code does. The Season 1 summary is the place to start.
Should candidates be allowed to use AI in a coding interview?
If they will use AI on the job, yes. Banning it tests a skill they will rarely use. The useful question is how well they direct the tools, and that only shows when they are allowed to use them.
What should you look at in an AI-assisted coding interview?
The session more than the result: how the candidate breaks the problem down, what they ask for beyond the brief, whether they check the AI's claims, what they do when it is wrong, and how they use the time.
Do developers still need to read code?
Yes, but for a different reason. The AI writes most first drafts. Reading code matters when something is wrong and the interface only shows the symptom, which is when it matters most.