OpenRouter clocks Luna fastest, Jev second
OpenRouter's latency board puts GPT-6 Luna on top at 180ms on global requests, with Jev behind it. Separately, Shrivu Shankar says Jev was the best calibrated of 41 decision models he tested on email classification.
Someone measured it
OpenRouter says Luna is the fastest decisions model, Jev second
OpenRouter published a latency board for decisions models and put GPT-6 Luna first at 180ms on global requests, with Jev and Perplexity Decider behind it. OpenRouter did not post Jev's own number in the text, only the ordering.
GPT-6 Luna is the fastest Decisions model today, at 180ms on global requests
Shrivu Shankar says Jev beat 40 other models on calibration
Shrivu Shankar says Jev was more calibrated than the 40 other decision models his team tested on email classification. He describes the result as expected. TypeSafe AI reposted it with the line "Stay calibrated out there!".
as expected, @typesafeai 's jev was more calibrated than the 40 other decision models we tested on email classification
Writing criteria
Harrison Chase splits trajectory labeling into typed answers per dimension
Harrison Chase says Jev as a judge in LangSmith evals gives difficulty and correctness each their own typed answer, in one pass, on every trace, rather than a single pass or fail. He was picking up Vaibhav Tulsyan's point that every agent trajectory needs assessing across several dimensions, including whether the agent considered diverse design choices.
trajectory labeling is really several questions, not one pass/fail