Moritz Kremb open sources Eyes, a video preprocessor for agents

Moritz Kremb released Eyes, a free and open source tool that preprocesses a video once so an agent can work from the output instead of the raw file. You hand it a video file or a link. It returns sheets of labelled frames to look through, every spoken word with a timestamp, text shown on screen with when and where it appears, and details about the sound and editing style. It runs locally, and the repo is linked from the post.

Moritz Kremb

@moritzkremb

X

Claude can't actually watch videos. When it tries, it may only check a few frames and guess what happens between them.

The argument behind it is about waste rather than capability. Kremb's claim is that Claude cannot actually watch videos, and that when it tries it may only check a few frames and guess what happens in between. Follow-up questions, he says, can make it start over, using more time and tokens. The second failure he points at is information that is shown but never said out loud, such as commands typed in a coding tutorial, which a transcript-only approach drops entirely.

Eyes does that work once up front. After that the agent can search the video, find the right moment, and zoom in to read small text or code, which is the part that matters if your use case is tutorials, screen recordings or anything where the pixels carry the payload.

What is not in the post

No numbers. There is no latency figure for the preprocessing pass, no token count for a before and after comparison, no accuracy measurement against any baseline, and no description of the hardware it was run on. The claim that repeated questions burn time and tokens is Kremb's reasoning, not a measurement he published. If you are deciding whether this is cheaper than letting a model sample frames itself, the post does not answer it and you will have to run it.

Nothing in the write-up says Jev is involved. The tool is framed around Claude and other agents generally, and the output is a set of artifacts any agent could read.

This is Kremb's second agent video release in the set we have covered. He earlier shipped FrameJam Playbooks for video editing inside Claude and ChatGPT. Eyes points the other direction, at reading video rather than producing it.

For people building on it, the useful distinction is that Eyes is an indexing step, not a vision model. It changes what your agent can retrieve and re-query cheaply, which is a different problem from whether the model understands what it is looking at once you put a frame in front of it.