Theo says Jev cannot validate because it cannot run tools
The argument is about using Jev as a judge for reasoning model output, not about Jev as a model.
Theo, posting as @theo, pushed back on the idea of using Jev to validate the output of other models. His argument has two parts. The first is a coverage problem. Anything simple enough for Jev to check, he says, is simple enough that 99% of modern models will get it right anyway, which leaves the validator catching a class of errors that mostly is not happening.
Ranking outputs from expensive reasoning models might be the worst case I’ve seen for it thus far.
Jev can’t do much validation because it isn’t able to run tools and modify its own context.
The second part is structural. Jev cannot run tools and cannot modify its own context, so what it can do when asked to validate something is bounded by what fits in a single pass. That is a ceiling, not a tuning problem, and it does not move with a better prompt.
What he was replying to
The thread started with Malte Ubl, @cramforce, making the standard case for cheap verification. His analogy was formal proofs, that it is more difficult to write the lean proof than it is to validate the lean proof, and that having the new tool in the toolbox is not worthless even if there is nuance. Theo's reply does not dispute that checking is generally easier than generating. It disputes that Jev, specifically, is doing the kind of checking that would catch the errors an expensive reasoning model actually makes.
He was blunt about the specific pattern. Ranking outputs from expensive reasoning models, he wrote, might be the worst case he has seen for Jev so far.
The line he did not cross
Theo still calls Jev a very cool model with a lot of real world use cases. So this is not a takedown of the model. It is a takedown of one deployment shape, the cheap-judge-in-front-of-an-expensive-model pattern that tends to be the first thing people reach for when they want to cut spend on a reasoning tier.
That distinction matters because it is not the same as the confidence based routing people have been shipping. Hassan's setup, where Jev's low confidence calls got handed to Kimi K3, uses Jev's own uncertainty as the signal rather than asking Jev to grade somebody else's answer. Theo's objection is aimed at the grading case.
For people building on it, this is an argument, not a measurement. Neither Theo nor Malte Ubl posted an eval, a hit rate or a cost number in the exchange, so what you have is a structural claim about tool access that is easy to verify against the model's capabilities, and a claim about 99% of modern models that is not backed by a benchmark here. If you are already running Jev as a judge, the thing to go measure is what fraction of the errors it flags are errors the upstream model would have made anyway.