Berman ran 100 AI shoppers over 3,092 Meta ads for $1.59

Matthew Berman pointed 100 AI shoppers at 3,092 real US Meta ads from 669 brands and says Jev got through the set in three minutes. Of the top 100 ads the simulated shoppers picked, he reports only 2 carried 50% off. In a follow-up post he puts the total cost of the whole experiment at $1.59 and says it can be rerun in under 10 minutes.

Matthew Berman

@TheMattBerman

X

in 3 minutes it ran 100 AI shoppers on 3,092 real US meta ads from 669 brands.

The headline claim he draws from it is that the standard holiday playbook is backwards, and that cutting prices by half "actually turns off 81% of your buyers". That 81% figure is asserted in the post, not shown. The sources do not include the scoring rubric, the shopper personas, or how a shopper registers as turned off, so the number stands on Berman's description of his own run and nothing more.

What the $1.59 actually bought

The interesting part for anyone costing out a pipeline is the shape of the job rather than the marketing conclusion. Three thousand ad creatives, 100 raters each applying preferences, finished in three minutes and under two dollars. That is the class of work people normally do not run at all because a frontier model pass over the same corpus would be a budget line somebody has to approve. At this price the decision is whether the output means anything, not whether to spend the money.

The ads came from StealAds, which Berman credits alongside TypeSafe AI, and he says the top 100 list is available there.

For people building on it

Simulated shoppers ranking ad creative is a preference panel, not a measurement of buyer behaviour. Nothing here is tied to a click, a conversion or a dollar of revenue. What the run produces is an ordering of 3,092 creatives according to what 100 model-generated personas said they preferred, which is a hypothesis you would then test against real spend, not a result you would act on directly. The $1.59 is cheap precisely because no humans were involved at any point.

That is still a reasonable use of the money. Screening thousands of creatives down to a hundred worth a human look is the kind of filtering job that survives a noisy judge, because the cost of a false positive is one person spending ten seconds. Treating the 81% as a finding about actual buyers is the version that breaks.

This is the second Berman workflow on Jev we have covered, after his Meta ad loop that gates spend decisions before a more expensive model runs. Same pattern, cheap model doing high-volume triage, judgement about what to do with the output still sitting with the operator.

Matthew Berman

@TheMattBerman

X

this whole experiment cost $1.59 and you can run it in less than 10 minutes.