Milind S drives a Mac with Jev at about 90ms per decision
A local CoreML segmenter and on-device OCR feed text to Jev, which returns a probability across the elements on screen.
Milind S built a computer use loop on a Mac where Jev picks which UI element to click, with no screenshots and no LLM in the path. A local CoreML model segments every button and UI element on screen, on-device OCR reads the labels, and that text is all Jev gets. Jev returns a probability across those elements and names the best one to click. The loop clicks, re-runs detection, and decides again until the goal is done. He measured roughly 90ms per decision.
~90ms per decision. Faster than any LLM computer use I've tried.
A local CoreML model segments every button and UI element on screen.
Notably he is not reading the DOM either. The whole input is the list of OCR strings pulled off the segmented regions, which is why no pixels leave the machine. That is the part worth sitting with, because the usual vision-language approach to computer use ships a screenshot to a hosted model on every step and pays for it in both latency and privacy posture.
What the 90ms actually covers
For people building on it, the number is the decision only. The CoreML segmentation pass and the OCR pass run before Jev sees anything, and Milind did not post timings for either, so your end to end click latency is 90ms plus whatever local detection costs on your hardware. His comparison is also against his own prior attempts rather than a published benchmark, so treat it as one builder's result on one setup.
There is a second constraint hiding in the design. If OCR misses a label or the segmenter does not produce a region for a control, that element simply does not exist as far as Jev is concerned. The decision layer is only as good as the candidate list handed to it, and nothing in the loop can recover an element that detection dropped.
ThePrimeagen's version
ThePrimeagen described a similar probabilistic setup built on Cloudflare's decision API rather than Jev. He said he took probabilities to certain power levels and then did probabilistic centering, and that the result clicks the correct icon, with "Click the monitor icon in the menu bar" as his example prompt. Milind replied that he had basically done the same thing.
Click the monitor icon in the menu bar" results in what you see.
These are two people posting demos of the same shape of idea, not a comparison. Neither posted a success rate across a task set, and neither published the prompts or the element lists, so the open question stays open. Fast is established. Reliable over a long loop, where one bad click puts the screen somewhere the next decision was never scoped for, is not.