Jev Daily

idan levin's WebMCP run splits Jev and Mercury 2.5

The 112x cost figure comes from a two model harness, not from Jev driving a browser by itself.

The WebMCP benchmark that earlier put Jev at 49 of 49 tasks belongs to idan levin, and his write up spells out the harness. Jev plus Mercury 2.5 using WebMCP solved every task at roughly 112x lower model cost than GPT-6 Astra using computer use with code execution, and 245x lower than Astra using screenshot based computer use.

idan levin
@0xidanlevin
X
Jev’s browser-control accuracy on its own was not amazing
Sep 18, 2026 · View on X
idan levin
@0xidanlevin
X
Jev picks the tool and Mercury 2.5 generates the arguments if needed
Sep 18, 2026 · View on X

The split matters more than the multiplier. Jev takes text plus a set of discrete options and picks one, which means it can choose the search_products tool but cannot write the search query. So levin has Jev select the tool and Mercury 2.5, which he describes as outputting 1,000 plus tokens per second, fill in the arguments when arguments are needed. His reasoning is that most of the cognitive load sits in choosing the action, and argument generation is simple enough to hand to a small fast model.

What happens without the tools

On the same harness, Browser Use's open source Ultrafast with modifications, Jev operating the browser without WebMCP solved 25 of 49. Adding WebMCP nearly doubled that and cut model cost a further 18 percent. levin's explanation is that choosing a valid button is not the same as choosing the right next step, and WebMCP compresses a sequence of clicks into a single tool call, which is the kind of multi step reasoning he notes TypeSafe AI itself documents as weaker.

He is direct about the limits of the lower number. He calls 25 of 49 a result for his particular implementation and benchmark rather than a universal limit on Jev or Browser Use, and the benchmark is open for contributions.

A separate number from Vercel

Guillermo Rauch posted that fx runs a safety reviewer over every command in auto mode, that the reviewer runs on GPT Luna today, and that Jev is up to 18x faster at p95 and more accurate. He said it is coming to Vercel AI Gateway and is likely the new default. Pranit, who ran the benchmark, gave the range as roughly 5 to 18x faster and more accurate than gpt-5.6-luna.

Guillermo Rauch
@rauchg
X
Jev is up to 18x faster (p95) *and* more accurate
Sep 16, 2026 · View on X

For people building on it, the two results describe two shapes of work. The fx number is a classifier picking from a fixed set of labels. The WebMCP number is Jev doing tool selection with a second model behind it doing everything that requires generating text. Neither is Jev driving a browser on its own, and the one measurement of that in these sources is 25 of 49.

Get the next one by email

Jev, read daily so you do not have to. The builds, the benchmarks, the criteria that worked and the cases where it lost, from the people shipping on TypeSafe AI's System One model.