SubQ Snap says it beats Jev on Decision Index
Alexander Whedon launched SubQ Snap claiming 61.7 on Decision Index v0.2.1 against Jev's 57.9, with 384K tokens of context. Every number so far comes from the vendor and two people it gave early access to.
Someone measured it
Contested
SubQ Snap claims 61.7 on Decision Index against Jev's 57.9
Alexander Whedon launched SubQ Snap, a long-context decision model he says scores 61.7 on Decision Index v0.2.1 against Jev's 57.9, with 384K tokens of multimodal context and $0.05 per million input tokens, output free.
61.7 on Decision Index v0.2.1 vs Jev’s 57.9, outperforming on 13 of 16 enterprise benchmarks.
The case against
Measured
Dan Shipper's rewrite benchmark puts Muse Wonder at 52%, GPT-Sol at 71%
Dan Shipper says Muse Wonder scored 52% on his senior engineer benchmark, which asks a model to rewrite a vibe coded slop codebase from scratch, against GPT-Sol 6.1 at 71%. He pulled his first post after finding a problem in the run, restarted it, and got the same verdict.
and it's back up! still bad