Haiku 5.5 takes first on the WebMCP benchmark, ahead of Jev

Idan Levin added six models to the WebMCP benchmark and says Haiku 5.5 using WebMCP now takes first place on the combined success, speed and cost score, ahead of GPT-6 Luna in second and previous winner Jev.

The six additions are Haiku 5.5, Sonnet 5.5, GPT-6 Luna, GPT-6.1 Sol, Gemini 3.8 Flash and Muse Wonder. Levin credits @alexatallah and OpenRouter for sponsoring the tokens.

The numbers he published

The headline comparison is not Haiku against Jev. It is Haiku with WebMCP against the highest-ranked computer-use configuration in the benchmark, which Levin says is GPT-6 Luna. On that pairing he reports 49 of 49 tasks solved versus 41 of 49, 3.7 seconds per task versus 19.4 seconds, and a cost of one fortieth of a cent per task versus 0.21 cents. He puts those as 5.3 times faster and 8.4 times lower cost.

That is a protocol comparison as much as a model comparison. WebMCP driving a page directly is doing less work than a model clicking through a screen, and the gap in the table reflects that.

Why Jev lost the top slot

The interesting part is Levin's follow-up, which takes most of the capability story off the table. Asked whether the models all succeed, he said success is near total across the board on WebMCP, with Luna 6 showing slight degradation at 47 of 48 solved. If everything solves almost everything, the combined score collapses into speed and cost, and that is where Haiku separates.

Levin puts Haiku at roughly 3.5 seconds per task, which he says is more than twice as fast as the other models running WebMCP, and notes the effect compounds on long horizon tasks where many steps stack up.

For people building on it, read this as a ranking change on one benchmark with a saturated success metric, not as evidence that Jev got worse at anything. Levin did not publish a per-model table of Jev's own seconds per task or cost in these posts, so the only concrete figures available are the Haiku against Luna computer-use pairing and the approximate 3.5 seconds for Haiku. If your workload is a handful of decisions rather than dozens of chained browser steps, a two second per task difference is not what will decide your stack. If it is long horizon browser automation, this is the number to go measure yourself.

idan levin

@0xidanlevin

X

haiku 5.5 is ~3.5 seconds per task, which is more than 2x faster than other models with webmcp