Gemini Decisions hits HotdogBench, Jev still ahead
Hamilton Ulmer put Google's day-old Decisions API on HotdogBench and says he was surprised how expensive it is next to Perplexity Decider, with Jev still ahead. Claude Fable 5.2 made the fewest mistakes, at the highest cost.
Built with it
ILIAS ISM says a 100-agent swarm researched 6,823 gyms for $200
ILIAS ISM rebuilt a site into a global CrossFit and HYROX atlas off one prompt, with a swarm of more than 100 agents researching 6,823 gyms across 97 countries for about $200. The output: 4,279 city guides, 2,864 real drop-in prices, 10,759 website screenshots, and every claim linked to its source.
Sydney Runkle's Slack agent uses Jev to decide when to speak
Sydney Runkle says their software factory Slack app uses Jev to decide whether the agent should respond at all. The case she calls out is multiplayer threads, where humans are talking to each other and the bot should stay quiet until it is looped in. No accuracy, latency or cost numbers were posted.
people are often just talking to each other and the agent should only jump in when someone actually loops it in
Someone measured it
Hamilton Ulmer adds Gemini Decisions to HotdogBench, says Jev still beats it
Hamilton Ulmer added the Gemini Decisions API to HotdogBench and says it came out expensive next to Perplexity Decider, with Jev still ahead of it. Claude Fable 5.2 had the fewest mistakes at the highest cost.
surprised at how expensive it is vs Perplexity Decider, and still ~ Jev beats it.

