Jev Daily

Theo and elvis got opposite results from Jev Router

One coding benchmark says the router costs more and runs longer, one small support agent says it halves the bill.

OpenRouter shipped typesafe/jev-router, a cache-aware router that asks Jev to score each turn on difficulty and precision and then pick the model and reasoning effort. Two people benchmarked it within a day and came out on opposite sides.

Theo spent $1,000 benchmarking it overnight, and as we covered earlier, found performance on DeepSWE roughly the same as GPT-6 Astra on low, at slightly higher cost and almost 5x the runtime. His objection is structural rather than numerical. Jev categorizes, it does not reason, and it reads the prompt without the codebase, so it cannot tell whether "port this to Rust" means a 100-line TypeScript file or a million-line application. He also argues that dropping to a cheaper model mid task saves little because cache writes are a large share of cost anyway.

Theo - t3.gg
@theo
X
It costs slightly more, and it took almost 5x longer to run.
Sep 26, 2026 · View on X

The other run

elvis ran Jev Router against a small support agent built on a custom harness with the Pi SDK, 8 cases against a fixed GPT-6 Sol baseline, 32 real calls in total. Both setups got every case right. The router cost less than half as much, $0.008 versus $0.018, with a lower median response time of 1.5s versus 1.9s. He called the sample small and named caching as his biggest remaining concern with Jev as a router.

OpenRouter's own numbers sit behind both. On four agent benchmarks it says Jev Router solved 82% more tasks than its Auto Router, 237 versus 130 of 423, and that on five agent benchmarks it had a faster median time to first token than every other router tested. That is the vendor reporting on its own product. The design detail worth knowing is that it holds a model for the rest of the session and switches only when the expected gain beats the cost of the cached chat it would lose. If the Jev call times out or returns invalid output the request fails rather than falling back to another router.

For people building on it, these are not a head-to-head. DeepSWE is a coding agent workload where the hard information lives in the repository that Jev never sees, and elvis was routing short support turns where the prompt is most of the task. The split in results tracks that difference, and neither run is a trend.

Theo - t3.gg
@theo
X
In order to know how much thinking a task requires, you have to think. Jev does not "think".
Sep 25, 2026 · View on X

Know what Jev is good at before you bet on it

One email a day. Under three minutes. What builders found out about Jev yesterday.

Get the next one by email

Jev, read daily so you do not have to. The builds, the benchmarks, the criteria that worked and the cases where it lost, from the people shipping on TypeSafe AI's System One model.