Shreya Shankar says per-row Jev calls are wrong for batch work
Her argument is about throughput on bulk rows, not whether the answers are any good.
Shreya Shankar, who works on the open-source AI-SQL engine Quail, says that calling a decision model like Jev once per row across thousands of rows is the wrong shape for the job. Her reason is not accuracy. It is that an API call per row "gets no benefit from query planning" and sits, in her words, extremely far from optimal performance.
The number she anchors it with is a speed-of-light estimate, the fastest the hardware could possibly do the work. With Qwen3-4B on one H100, she puts the floor for running a single AI filter over 5,000 movie reviews at about 6.6 seconds. Her point is what follows from that figure rather than the figure itself. No system today comes close to it, and she includes Quail in that.
That is the part worth sitting with. This is not a vendor comparison or a benchmark of Jev against anything. It is a person who builds batch AI query engines saying the whole category, hers included, is leaving most of the available throughput on the table, and that a per-row API loop is the furthest thing from the ceiling. She points to a new blog post on how Quail costs AI-powered filters using those speed-of-light estimates.
What it does not say
Nothing here measures whether Jev returns the right answer. The claim is about shape and throughput on bulk rows, so it tells you nothing about Jev on one request in a hot path, which is where most people have been putting it. If you are classifying a single inbound email or scoring one record behind a user action, query planning has nothing to plan.
It does land on the batch patterns people have been shipping, including running Jev from inside SQL, as with prompt_jev() in MotherDuck. Shankar does not name those, and she has not yet spelled out the mechanism. Kush asked her directly to explain what she means by getting no benefit from query planning, and the answer is not in the thread.
For people building on it, treat this as a throughput argument from someone who measures throughput for a living, not a verdict on the model. If your workload is thousands of rows and you are looping, the ceiling she describes is the thing to check your runtime against.
No system today comes close, including Quail, our open-source AI-SQL engine
Calling an API once per row is a terrible idea for batch work