Zachi says Cognition cropped the Devin Fusion price chart

Cognition posted that Devin Fusion took a top score of 68 on the Artificial Analysis Coding Agent Index, running an Opus 5.5 lead with a GPT-6 Luna sidekick. The claim attached to the number is a price-performance one. Cognition says Devin Fusion beats Claude Code with Opus 5.5 on both performance and price, at a 31% lower cost per task.

Zachi says the chart behind that claim is not showing you the field. By his count there are over 30 models priced below $5 that were cut from the plot, and cutting them moves where the attractive quadrant sits.

What is actually being disputed

Not the score. Zachi is explicit that the result survives the correction. In his words the true gap is "still 68 points and still cheaper, but not as claimed." The objection is to the framing, that removing the cheap end of the price axis makes the distance between Devin Fusion and the competition look larger than it is, because the comparison set shrinks to the expensive models.

Zachi

@iam_zachi

X

still 68 points and still cheaper, but not as claimed

That is a different kind of criticism than a disputed benchmark run, and it is worth separating. A contested score means somebody thinks the eval was run wrong. A contested axis means the eval is fine and the picture drawn on top of it is doing extra work. Zachi is in the second camp, and he opened by saying he likes the Cognition folks before calling the chart a crime.

For people building on it

If you are deciding what to put in front of a coding agent, the useful part of Cognition's post is the 68 and the 31% lower cost per task against one named competitor, Claude Code with Opus 5.5. That is a pairwise comparison the vendor chose. It is not a ranking of the price-performance frontier, and on Zachi's reading of the chart it was not drawn against the full set of models that compete on price.

Neither side has published a redrawn version with the sub-$5 models restored that we can check here, so what you have is Cognition's number and one critic's count of what is missing from the plot. The number came from Artificial Analysis. The quadrant came from Cognition.