What AI changed about how we evaluate engineers

When people ask how AI changed the way we hire, they expect me to talk about speed. And speed did change. But the speed is the least interesting part. What actually changed is what we look for in an engineer, and why.

Five years ago, evaluating a candidate was slow and manual on purpose. The first interview was about attitude, communication, and whether the person fit how we work, a soft-skills filter. The second interview was the technical one, and that is where most of the time went. For a mobile role, a senior engineer had to design the test by hand: first step, second step, third step, scoped so it could be done in about an hour. Designing one good test could take days. Then reviewing the candidate's submission took another couple of days, because a person had to read the code and judge it line by line. One or two engineers, pulled off their own work, reading someone else's code. The whole process ran two to three weeks.

That model rested on an assumption we never questioned: that writing and reviewing code was expensive, so the ability to do it was the thing worth measuring.

linkThe signal that the old way had stopped working

The first real sign didn't come from a tool or a trend piece. It came from losing a client. They went with a competitor because we were too slow. In the time it took us to source, test, and send an engineer to an interview, the other firm already had a developer working and shipping. We were still at the interview stage while the work had already started somewhere else.

Around the same time, we were using AI in our own projects and seeing the same gap from the inside. A simple module, a form, a basic CRUD screen, that used to take three or four weeks, was coming together in two or three days. Once you see that, you can't keep hiring as if code were scarce. We realized we could not put anyone on a team who didn't work this way, not because it's fashionable, but because a slower team loses the client. It was never really about the tooling. It was about value: how much value you can give a client, in the least time. That is who wins.

linkWhat we evaluate now

The technical test changed first, because it had to. In the same time a senior engineer used to spend building one test, they can now build many more, and review many more submissions, because the manual parts are faster. What makes that work is that we already know the red flags. After enough projects, we have them documented, and the job is to define clearly what we are looking for and let the review surface it.

But the deeper change is in the quality we are actually hiring for. Five years ago, if you sent an engineer a twenty-page PDF, or the documentation for integrating one system with another, they might spend half a day or a full day reading it, processing it, and turning it into a work plan. Our engineers now do that in twenty or thirty minutes. The difference between a strong engineer today and a strong engineer five years ago is the speed at which they take in a large amount of information, make sense of it, and turn it into a plan they can execute.

That is the thing worth measuring now. Not how much code someone can produce, because production is no longer the scarce part. Judgment under a pile of information is. This doesn't mean technical skill stopped mattering. It means technical skill is table stakes, and the differentiator sits one layer up, in how someone reasons about what they are building.

You can see the pace change in the day-to-day too, though volume is exactly the wrong thing to hire on. An engineer who used to land a commit every two or three days can now make several a day. That tells you the work moves faster. It doesn't tell you the work is right. The thing we learned to evaluate is the judgment that decides whether the fast work is the right work.

linkWhat this means if you're the one hiring

Here is the part that matters for anyone leading a team right now. When you stop evaluating engineers by how much they produce and start evaluating them by judgment, the bottleneck in delivery moves.

For years, the bottleneck was us. We were the slow part, the ones taking weeks to build a feature. That specific bottleneck moved. Because a bottleneck doesn't disappear, it relocates, first inside the team, to review and QA, which is why the way we hire and the way we run quality became the same conversation. Hiring for judgment is the first move in building a delivery system that can keep up with its own speed.

So when I hire an engineer today, I'm not really hiring production capacity. I'm hiring the first piece of a system that only works if judgment runs through all of it. The speed is real. But the speed is downstream of getting the judgment right.