Merge says Clef beat Jev on 3 of 4 workflow evals

Merge has put Cloudflare's Clef decision model family on Merge Gateway, and announced it with a number aimed squarely at Jev. Merge says Clef-flash answers in a median 39ms, and that in Cloudflare's tests Clef came out ahead of Jev on 3 of 4 workflow evals. Cloudflare Developers confirmed the availability, posting that the Clef models are now on Merge.

Merge

@merge_api

X

Clef-flash answers in a median 39ms, and in Cloudflare's tests Clef came out ahead of Jev on 3 of 4 workflow evals.

That is the whole of the public evidence. The post names the score line and nothing underneath it.

What is missing

The four workflow evals are not named. The tasks are not described. The Jev version tested is not stated, and neither is which Clef model carried the comparison, given that Clef-flash is called out separately on latency. There is no link to a methodology page in the post, no per-eval breakdown, and no indication of whether the one Jev won was close or lopsided. Three of four is a ratio that reads well and carries very little information when the denominator is four.

The 39ms figure has the same problem. A median response time means little without the prompt size, the concurrency, the region the client sat in, and whether the measurement came from Cloudflare, from Merge's gateway, or from somewhere in between. A gateway adds a hop. Whether the 39ms includes that hop is not said.

How to read a vendor scoreboard

This is a model vendor's own result, relayed by the gateway that just started selling it. That is not a reason to ignore it. Cloudflare has an interest in the number being defensible, and a wrong claim about a model people can call in a minute does not survive long. It is a reason to treat it as a prompt to measure rather than a measurement.

For people building on it, the practical move is the same one that applies to every decision model comparison. Take the four or five routing calls your product actually makes, the ticket triage, the upload flag, the next-step pick that Merge names as the use cases, and run both models against your own labeled set with your own prompts. A workflow eval someone else designed is a proxy for your workload at best. On a four-task spread, one task either way flips the headline.

Until Cloudflare publishes the eval set, three of four is a claim with a source attached, not a result you can check.