Archive, page 3
- Theo's $1,000 Jev Router bench vs elvis's half-cost run
Theo spent $1,000 benchmarking Jev Router and got roughly GPT-6 Astra quality on DeepSWE for slightly more money and almost 5x the time. elvis ran the same router on a support agent and paid less than half. Same product, opposite verdicts, and the difference is the workload.
- Theo and Diogo Almeida agree: Jev demos oversell it
Theo says he is tired of calling out bad Jev ideas, and Diogo Almeida agreed that outlier demos lead to over-promise and under-deliver. Meanwhile elvis got Jev Router to half the cost of a GPT-6 Sol baseline on eight support cases, the opposite of what Theo's code benchmark found.
- Theo spent $1,000 benchmarking Jev Router
Theo says Jev Router matched GPT-6 Astra on low on DeepSWE, cost slightly more and took almost 5x longer to run, a day after OpenRouter launched it. He also says he loves Jev.
- Deel's Jev numbers, posted by TypeSafe
TypeSafe AI published Deel's measurements on six classification tasks, including repeat-question matching going from 70% to 97% and expense categorization from 50% to 86%. Every number here comes from the vendor's post, not an independent run.
- Hassan trained a Jev-like classifier for $17
Hassan says he fine-tuned Qwen3.5 4B into a Jev-like classifier for $17 and released the weights, data recipe and tutorial. If it holds up, the classification layer is now something you can clone.
- Metaview put Jev in every agent, claims 10x faster
Shahriar Tajbakhsh says Metaview's sourcing searches went from minutes to seconds at the same accuracy after shipping Jev into every agent over the weekend. No per-search numbers yet, so the interesting part is the build pattern, not the multiple.
- MotherDuck ships prompt_jev; TypeSafe pauses signups
MotherDuck says 100k rows classified in 40 seconds for $0.50, against 32 minutes and $37 for the LLM it replaced. Hours later TypeSafe suspended new Jev signups, so the people who can act on that number are the ones already in.
- Jev drops the waitlist, Theo trashes instant compaction
idan levin's WebMCP benchmark went from 25 of 49 tasks to 49 of 49 once he handed Jev website tools instead of raw page controls. That is the week in one line: Jev is strong at picking from a good option list and weak at building one.
- OpenRouter: Jev 5x faster, same accuracy on 30 labels
OpenRouter ran Jev and four LLMs over the same 200 classification cases and says Jev's slowest requests beat every other model's median, with accuracy inside a handful of cases. Theo, meanwhile, calls ranking reasoning model outputs the worst use he has seen for it.
- Jev aced one browser benchmark, scored 1/20 on another
Gregor Zunic shipped a Jev browser agent that found flights in 7 seconds for $0.0039, then benchmarked the same approach at 1 of 20 on long horizon tasks against 17 of 20 for BrowserCode plus Luna. The speed is settled. The reasoning is not.