The Most Expensive Input in AI Right Now Is Human Judgment

Share

For the better part of a year the AI conversation has pointed in one direction, toward fewer people. Every new model release seemed to carry the same implication, that the routine work a business used to hire for could increasingly be done by a machine, and the events of this week, voluntary buyouts offered to tens of thousands of workers at one of the largest software companies, only sharpened that reading. It is easy to feel that the arc of this technology bends toward reducing the number of people a company needs. And yet, in the same few days, a piece of news arrived that cuts the other way, and I keep turning it over because it complicates the whole picture. The companies building the most advanced AI in the world are spending money on humans at a scale they did not need to a year ago, and a company whose entire product is organizing people just became worth twenty billion dollars.

The company is Mercor. This week it raised seven hundred fifty million dollars at a twenty billion dollar valuation, with Nvidia, Salesforce, and Zoom joining the round. Mercor runs a network of more than five million experts, paid by the task, who train and evaluate frontier AI models using their own domain knowledge. By the company's own account the network is paid more than four million dollars a day to do this work, and nearly all of that demand so far has come from the frontier labs themselves. Think about what that means. The most sophisticated model builders on earth are not just bolting human review onto the edge of their process. They are buying, in aggregate, millions of hours of paid human judgment every single day, at a time when anyone would tell you the whole point of the technology is to need less of it.

That is the strangeness worth sitting with. We have spent a year debating whether machines will make people redundant, and here are the people who make the machines quietly paying a small fortune for human judgment they cannot get any other way. What separates a good model from a great one, more and more, is not raw compute. It is careful, verified, paid judgment about what a right answer actually looks like, in a particular domain, for a particular customer. The frontier of the technology has come to depend on the least automatable thing there is, which is a knowledgeable person deciding that an output is correct. The machines have not removed the need for judgment. They have moved it higher, to the place where it grades them.

There is a familiar shape to this if you look back. Every wave of automation has eventually produced a marketplace to organize the very labor it was supposed to make obsolete. The internet did not remove the need for freelance work, it produced the platforms that organize it. And the pattern has its own logic here. Technology gold rushes are rarely won by the prospectors. The durable fortunes are made selling the picks and shovels, and for the modern gold rush the picks and shovels are the ground truth, the human data that every model is trained and measured against. Mercor is selling that ground truth at industrial scale. What looked like a labor side effect of the buildout has become one of its essential inputs, and the market has started to value it accordingly.

I want to hold up the obvious objection before I go further, because it is strong. It may be that this is a temporary phase, an awkward middle in which models still need a human hand before they learn to grade themselves. It is entirely possible that within a few years the frontier models will be good enough to generate and assess much of their own training signal, and that the market for paid human evaluation will thin out the way the market for manual review did in earlier waves. And twenty billion is a serious number for a business whose economic core is brokering labor, a category that has historically been full of thin margins and unhappy participants. I do not think we can rule any of that out.

Even granting all of it, the direction of the moment is what matters, and it inverts the usual AI story. The cost of compute keeps falling, and the value of verified human expertise in the right domain keeps rising, at the same time. That is not a small thing. For every task a model absorbs, there is a residue of judgment left behind that becomes more valuable, not less. Somebody still has to say whether the answer is good enough to trust. Somebody has to know the domain well enough to catch the confident mistake. Somebody has to be accountable for the result when it goes in front of a customer. Automation concentrates that residue rather than eliminating it. The cheaper and faster the machines get, the more the bottleneck moves to the people who can tell whether they are right.

And this is the part a founder should actually notice, because it decides where the money lands. The individual experts in Mercor's network may well be treated as replaceable gig participants, and the history of such marketplaces suggests many of them will be paid less than their judgment is worth. The company organizing them, by contrast, captured a twenty billion dollar valuation. So value is not moving simply to human expertise. It is moving to the platform that verifies, routes, and brokers that expertise, the layer that can certify to an enterprise that a model gets its specific domain right. Once again the durable asset is distribution and trust, the ability to connect the right judgment to the right machine and stand behind that connection, rather than any single model or any single person.

Which is probably why Mercor is turning from the labs toward the rest of the economy. It is now selling custom evaluations built on a company's actual workflows, and it points out that the United States alone has more than eight hundred distinct occupations whose workflows no model has been evaluated against yet. That number is the real story. Most of what remains for AI in business is not building the next great model. It is figuring out, industry by industry and job by job, whether a model can be trusted to do a specific kind of work, and then training it until it can. The scarce capacity is deciding what good looks like in every corner of the economy, and that is a market that runs on people.

So which story is right, the one that says people are becoming optional, or the one that says human judgment is becoming priceless? I think both are true, and it is less contradictory than it sounds. The machines really do absorb the routine and the evaluable, the layers of work that resemble what a well trained assistant can absorb, and that part of the picture should not be ignored. But the value that is left concentrates in the judgment that grades the machine and in the trust layer that surrounds it. The people exposed are the broad middle of the organization. The people newly valuable are the genuine experts, and even more, the platforms that organize and certify them.

We assumed AI would make the marginal person cheaper to replace. What the past few days suggest instead is that it has made the right person's judgment the most scarce and contested input in the economy. The machines did not stop needing people. They got specific about which people, and they moved the price to the judgment rather than the job. That is a stranger and more human ending than the story we were told.