Almeida says Jev clones copy the interface, not the reliability
A position posted on X with no test behind it, from someone who says his own side is ahead.
Asked by Ash what he makes of everyone trying to copy Jev, Diogo Almeida said the copies take the interface and miss the part that matters. His phrasing was blunt. They copy the interface, but he does not think they understand the importance of reliability.
they copy the interface, but I don't think they understand the importance of reliability
The second half of his answer is the more interesting one. His stated concern is not that competitors win share, it is that someone tries a weaker clone, gets a bad result, and concludes that decision models as a category are not useful. That is a reputational argument about the whole product class, not a claim that any specific competitor scored worse on any specific task.
No numbers attached
There is no eval here. Almeida names no competing model, no workload, no accuracy or latency figure, and no failure case. He does not say what he means by reliability, whether that is output format stability, consistency across repeated calls, or calibration. So this reads as a position from an interested party, not a measured gap, and it should be weighed accordingly.
How interested is clear from a separate post the same day, where he said he is anti public benchmark even when his side is ahead, and would not recommend believing benchmarks blindly. He argued that the people most hurt by benchmarks are their target audience, people too busy to play with the models themselves and get a feel for their intelligence.
That is a consistent stance rather than a convenient one, and it is worth sitting with for a second. It also means the one person here making a reliability claim has pre-committed to not producing the kind of public number that would let anyone check it.
For people building on it
If you are picking between Jev and something with a compatible interface, this gives you nothing to test against. The useful read is narrower. If a drop-in replacement is cheap to swap in, the thing worth measuring yourself is not headline accuracy on a leaderboard but variance across repeated runs on your own inputs, because that is the axis Almeida is pointing at and the one nobody has published on. Run it on your traffic for a day. That costs less than reading another argument about it.
The biggest victim of benchmarks is exactly their target audience: people too busy to play with the models themselves and get a feel for their intelligence
Know what Jev is good at before you bet on it
One email a day. Under three minutes. What builders found out about Jev yesterday.