Lightpanda's judge() call clocks 0.2s per page on cookie walls

Lightpanda has compiled Jev into its binary and built a judge() call on top of it, so a crawler can decide whether a page is worth anything before it spends a full LLM call finding out it fetched a cookie banner. The company now puts a time on it. One judge() call takes about 0.2 seconds, and a demo video shows 30 sites handled in roughly 14 seconds total.

Lightpanda

@lightpanda_io

X

No more sending a whole page to an LLM just to find out it's a cookie banner.

The problem it targets is the one every crawler operator knows. Lightpanda lists blocked pages, CAPTCHAs, cookie walls, logins, paywalls, error pages, empty searches and pages still loading as things that all happily return an HTTP 200. Status codes do not tell you whether there is content behind them, so something has to read the page. Doing that with a frontier model on every fetch is the expensive version.

What judge() actually returns

It asks a set of predefined questions and reports whether the page has real content, and what kind, from a fixed set of article, listing, product or docs. That is the whole surface. It is a classifier, not an extractor, and the output is designed to drive a keep or skip decision in the crawl loop rather than to feed a downstream summarizer.

In the demo, Lightpanda says the crawler backs off when it is rate limited, accepts cookie walls, and keeps or skips each page on the result. That is the behaviour the embedded model is there to enable, the crawler making a routing call locally instead of shipping the DOM somewhere.

What is not in the numbers

The 0.2 seconds and the 14 seconds come from Lightpanda's own demo run, on sites Lightpanda chose. There is no independent benchmark here and no accuracy figure at all, which matters most on exactly the pages the feature exists for, the paywall that leaves two paragraphs visible, the half loaded listing, the error page styled like an article. Lightpanda first showed the integration without those timings, and it still has not published a hit rate.

For people building on it, treat the speed claim and the classification quality as separate questions. The first one is cheap to verify on your own corpus. The second one is the one that will decide whether you trust a skip.