We Got Tired of Wrestling ffmpeg, So We Taught the Machine to Edit Video

If you've ever tried to make a computer cut a video from the command line, you know the feeling: you're not editing, you're negotiating. A wall of flags, a syntax that punishes a missing colon, and a result you can't quite predict until it renders. Do it once, fine. Do it for the fortieth clip and you start to resent the tool.

We make a lot of video — or want to. Demos of what JARVIS can do, clips for the things we write here, pieces of a bigger streaming idea we'll talk about another day. So the resentment added up, and we did the thing we keep doing in this lab: we stopped fighting the tool and built the small one we actually wanted.

A handful of verbs that always behave

The engine isn't clever. That's the point. It's a short list of primitives — trim a clip, crossfade between two, slow-pan across a still image, drop a title on, duck the music under a voice, change the speed. Each one does exactly one thing, and does it the same way every single time. No surprises, no "it depends."

What you get from a handful of reliable verbs is composition. Trim, then crossfade, then title, then duck the audio — and you've described an edit the way a person would, in steps, out loud. The machine just performs the steps. Because each verb is boring and deterministic, the whole chain is predictable: the same description produces the same video, today and next month.

The model decides; the software does

Here's the principle underneath, and it's the one I'd actually stand on a stage and defend: the AI should decide intent, and ordinary software should do the work.

It is tempting — fashionable, even — to ask a model to be the video editor. Hand it the footage, ask for a cut, marvel when it sort of works. But "sort of" is the problem. The moment the doing and the deciding live in the same fuzzy place, you lose the thing that makes software worth trusting: that it does the same thing twice.

So we split them. The model is the intent layer — it reads what you want and chooses which verbs, in what order, with what timing. The engine is the work layer — it executes those verbs with the reliability of a calculator. The AI never touches a pixel directly. It writes the recipe; the kitchen is deterministic.

The payoff is that the AI got better at editing by being given less to do. It doesn't have to hallucinate frame math or get lucky with a render. It composes trustworthy parts. And when something looks wrong, you can see exactly which verb to blame, because each one is small enough to hold in your head.

Why we keep building the small tool

There's a quiet cost to renting your capabilities. Every SaaS editor, every API you lean on, is a verb you don't own and can't change. For a one-off, who cares. For something you'll do a thousand times, owning the verb means you can shape it, automate it, and let your own AI reach it — which is exactly what we did.

It's not a flashy thing to report. We didn't invent video. We made eight small, dependable verbs and put the machine in charge of arranging them, not performing them. But that division of labor — model decides, software does — is most of what we believe about building with AI, compressed into something you can watch render.