Theo's $1,000 Jev Router bench vs elvis's half-cost run
Theo spent $1,000 benchmarking Jev Router and got roughly GPT-6 Astra quality on DeepSWE for slightly more money and almost 5x the time. elvis ran the same router on a support agent and paid less than half. Same product, opposite verdicts, and the difference is the workload.
Built with it
Hassan open sourced Tev1 0.8B, runs on a Mac at about 50ms
Hassan released the weights for Tev1 0.8B, the tiny Jev-like classifier he filmed running locally through Ollama at roughly 50ms end to end. He is blunt about the ceiling: not as good as Tev1 4B or Jev, and suited to very simple classification. It follows tev1-4B-experimental, which he fine-tuned on Qwen3.5 4B for $17 and also released with the data recipe.
Definitely not as good as Tev1 4B or Jev, but it runs entirely locally on a mac.
Someone measured it
OpenRouter says Jev took 27% of its classification requests
OpenRouter posted that Jev took 27% of weekly request volume in the classification category, nearly twice the share of DeepSeek V4 Flash, which previously held the top spot. Earlier in the week TypeSafe's Diogo Almeida said Jev is the top model at 1k to 10k context on OpenRouter.
Writing criteria
Kieran Klaassen builds readable embeddings out of Jev answers
Instead of an embedding model returning 1,536 meaningless numbers, Kieran Klaassen asks Jev a fixed set of questions per document and uses the answers as the vector, then runs plain cosine similarity. An email reading "I got charged twice this month, pls fix asap" scores [is_customer, urgent, about_billing, needs_reply] at [1.0, 0.9, 1.0, 1.0], so you can see why a match happened and add a dimension by adding a question.
some of our biggest prod successes are in this regime - tune-able RAG
The case against
Theo's $1,000 Jev Router benchmark: almost 5x slower on DeepSWE
Theo spent $1,000 benchmarking Jev Router and reported performance on DeepSWE roughly the same as GPT-6 Astra on low, at slightly higher cost and almost 5x the runtime. His objection is structural: Jev categorizes rather than reasons, and cannot know from a prompt whether "port this to Rust" is a 100-line file or a million-line app.
It costs slightly more, and it took almost 5x longer to run.
Theo says the Jev demos are pushing it at things it is bad for
Theo said he is tired of seeing people push Jev at problems it is not suited to, and worries that not calling out the bad ideas makes people think they work. His definition of the job: classification of fixed data with mostly known quantities. Diogo Almeida replied agreeing with him.
Posting outlier demos leads to over-promise under-deliver
Also on the timeline
Jev signups reopen, free credits off for new users
TypeSafe says capacity has increased and anyone can sign up again. Diogo Almeida said free credits are temporarily disabled for new users after abuse.
DSPy 3.4.0 adds native Jev support and a calibration optimizer
Isaac Miller announced DSPy 3.4.0 with native support for Jev and System One models via compatible signatures. It also ships ReAnchor, an optimizer for calibrating outputs with confidence.
Frequently asked questions
Can I run Jev locally on my computer?
Yes, Hassan released Tev1 0.8B, a smaller version that runs on a Mac in about 50ms through Ollama. It's suited for simple classification tasks but won't perform as well as the larger 4B or full Jev models.
What is Jev actually good for?
Jev is best for classification of fixed data with mostly known quantities, according to critics in the community. It's not designed for reasoning tasks like coding agents that require understanding complex context.
How do I make embeddings (vector representations) more understandable?
Instead of using standard embedding models that return meaningless numbers, you can ask Jev a fixed set of questions about each document and use the answers as your vector. This lets you see exactly why matches happened and adjust dimensions by adding questions.
Can new users get free credits for Jev?
Signups are open again now that capacity has increased, but free credits are temporarily disabled for new users after abuse.