The encoder had ignored tune=still since the first build

The symptom was a number that didn't add up. I had a battery of encoders running in parallel: nineteen configurations across AV1, AVIF, x264, x265, WebP and lossy JPEG XL. Even VVC got in, more out of curiosity than hope. Each one calibrated to the same visual quality, to find out whether Nest's AV1 was the most compact option or just the one closest to home. One of the agents reproduced the Nest build by hand, with the same parameters, and the file came out a different size. Same card, same CRF, same preset, different bytes. I read that in the morning and my first reaction was that the agent had gotten the command wrong.

The probe failed and nobody said a word

It hadn't. The forge asks the encoder for tune=still, the still-picture mode, which is the right one for a stream where every frame is an independent photo. Before asking, it runs a probe to check whether the encoder accepts that mode. The probe ran without a parameter the mode requires, failed, and the code fell back to a generic mode without telling anyone. The manifest recorded the spec, which said "still", not what the encoder had actually done. Since the first build I had been reading a manifest that didn't match the binary, and the only place that recorded the mode was the wrong place.

The difference measured eleven points of visual quality for ten percent more bytes, at the same CRF. The old build was the worst point on the curve: smaller, but ugly in a way that isn't worth it. And nobody would have caught it by watching the video, because what a frame looks like in the player doesn't tell you which tune was used. Only bytes against the bytes of an independent encode.

And it was by chance. The encoder battery existed to answer a different question, and the manual baseline encode was just its reference point. If I had calibrated the battery against the manifest instead of against an encode done from scratch, the bug would have stayed invisible, and every number in the series would have come out ten percent optimistic on bytes and eleven points pessimistic on quality. Now the manifest records the mode the encoder actually used, and the probe passes the right parameter.

Even after the fix there was a second layer. SVT-AV1 has two modes you could call "still": tune=2, optimized for a structural similarity metric, which is what the fixed probe started picking, and tune=3, called image quality, which the documentation itself marks as still images only. The second one gives one more point of quality for one percent more bytes. Not much, but it's the right mode for still pictures, and the recommended build of the full corpus went out with it. The manifest now says which of the two, by number, because "still" on its own already fooled me once.

The AVIF row looks like the winner and it isn't. It sits at a much lower point on the quality curve, so you can't compare it on the same number. Calibrated to the same quality as AV1, it comes out about 12% smaller, and that left an open question about which backend to use for stills. And inter, my favorite bet, came out bigger than tune=still. I explain that in the next post, because the configuration was wrong and the problem was something else.

I won't table the rest of the nineteen, but three findings stayed with me. x264, the encoder for a 2003 standard, was the fastest inside the quality window by an enormous margin, and VVC, the newest codec on the list, cost a hundred times x264's time to land in the middle of the table.

Two different SVT-AV1 presets spat out byte-for-byte identical streams, because the version I use maps both to the same internal mode when the stream is all keyframes. One more case of a parameter I was passing and the encoder was ignoring, this time documented. And lossy JPEG XL, which I expected to compete, only competes at high quality, well above the window that matters for a deck tile. It's a good codec for a target this corpus doesn't ask for.

CapCut was throwing cards away

I had written in the bets that speeding up the video cut bytes, because I had done it in CapCut and the file came out half the size. I reproduced it programmatically: speed it up sixty times, export at thirty frames per second in H.264, the way the editor does. The file came out with half the bytes. And half the cards.

The export samples the sped-up timeline and drops whatever falls between one frame and the next. Of the 2,048 cards, 993 disappeared. What was left got re-encoded to H.264 on top of AV1 that was already lossy. CapCut didn't compress anything, it threw half the dataset away and called it optimization, and I looked at the file size and believed it. Changing only the fps, without speeding up, doesn't change a single byte. It's a timestamp annotation in the container, as ffprobe had already told me on day one and I hadn't properly written down.

The wrong bet yielded a real bug. With an fps other than one in the manifest, the code that fetches a frame by index asked for an instant that didn't exist, because it assumed ordinal and timestamp were the same thing. Fixed, and tested with the fps I never should have changed. The real gain CapCut was pointing at, redundancy between neighboring frames, does exist. It just has to be measured with inter, without losing cards, and that's the subject of the next late night.

What else is the manifest getting wrong

I closed the day with two things validated by hash and a bigger doubt than when I started. The encoder mode was a constant I thought was on, and it had been off since the beginning. CapCut was a number I thought was compression, and it was data loss. Both lived in a manifest or an export I had read and accepted without checking against anything.

If the encoder wasn't doing what the manifest said, the question left over was what else I was reading as a summary instead of validating on disk. I didn't know yet that the answer would come in the middle of the night, with a meeting on the calendar.