Nest has the potential to be a Magic card archiver

Felipe Navarro, a friend of mine, is building pasta.cards, an open source app for managing your collection and Commander decks. It runs locally, keeps everything in SQLite, and to work offline it downloads the art of every Magic card ever printed, in two versions: the full card and just the illustration. On my Mac that came to 83 thousand JPGs and 7 GB. Every card that exists, twice, on my disk, and Finder dragging its feet to open the folder.

What caught my attention in that number wasn't the size, disk is cheap. It was the structure. Cards share the frame, the text box, the typography, the mana symbols. And the database next to it shows that almost half of all printings reuse art that already appeared on another card. A loose JPG knows nothing about that. A good catalog points instead of copying, and a good codec doesn't spend bits on a pixel it can predict from its neighbor. I like both for the same reason.

I had Nest at hand, the single-file vector database I've been building this year, and since August it stores images too. The question I asked that morning was far more naive than everything that came after: how does it work, does it turn this into a video? I have 38 thousand images and a database, what does it do with them?

Does it turn this into a video?

It really does turn it into a video. Each image becomes a frame of an AV1 stream inside the file, and search runs on vectors taken from the decoded frames, not from the source JPEG. That's on purpose: the index has to describe what the file serves, and what the file serves is the frame after the codec. Video codecs are the most optimized image compression technology there is, so what looks like a hack is just borrowing decades of other people's work.

I had already measured this on skin. Dermoscopy, biopsy slides, pages of old books, the datasets I had on hand from my work at Hoff. What those numbers taught me is that how much something compresses is a property of the domain, not the codec. Same encoder, same settings: dermoscopy compressed 151 times and the book page compressed 4. Smooth skin compresses well because there's little detail to keep. Small print is the opposite, every edge costs bits.

A Magic card has small print. It has the same frame on every card, which the codec may or may not exploit. It has reprints of the same art, something dermoscopy doesn't have. And it has a product constraint that no search metric measures: the deck tile has to look good on the app's screen. I thought the question was how much it compresses. It was, but not for long.

Before running anything, I took a calmer look at the raw folder, and it had surprises of its own. Half the weight was the cropped version of the art, and my first idea was to throw it away and crop from the full card when needed. An agent read the app's code and came back with bad news: the database doesn't store any crop coordinates, and the app uses the crop directly in the deck tiles. The cropped art is a class of its own, not a derivative. And next to the image folder there was a directory called _redundant, holding another seven gigs of copies of things that were already there. That one I deleted without mercy.

The first build

The pilot with a few hundred cards died with a segfault before doing anything. Nest's Python binding had been compiled without the flag that says "use the process's Python, don't bring your own", and the Python I use is static, so two interpreters were fighting over the same memory. One flag and done. A small bug, and the kind that makes me suspicious: how many other environment assumptions does the build carry without declaring them?

With the flag, the full build ran in the background and I opened one of the videos in the player. Cards going by, one per second, forty minutes of cards. My first hypothesis was that the encoder was repeating frames to hold each card on screen, and that this could be cut. One line of ffprobe killed the hypothesis: one frame per card, and the duration is just a timestamp annotation in the container, it costs no bytes.

7 GB became 1.9 GB, with image search working on top. It ran on the first try, which in a project of mine is rare enough to make me suspicious. Then I opened the output folder.

An archiver with 38 MP4s next to it isn't an archiver

38 video files, a .nest with the indexes, a JSON manifest and a cache directory. Four kinds of file to serve one thing. Nest exists to be the database in one file, and what came out was a database with forty things next to it. It worked, the way a piece of furniture assembled with leftover screws works. The point of Nest was getting lost there, and compression wasn't even the problem. Nest has the potential to be an archiver for Magic cards, for maps, for any image dataset with a database beside it. But an archiver is a file, not a folder.

The second question came from the model. The script used CLIP because that was what the forge had, and I didn't really know what CLIP did with a card. I looked into it: it looks at the art and ignores the printed text. For a Magic card that means looking at the less useful half, because what identifies the card is the name written at the top. I asked for a search on Hugging Face and GitHub of the open multimodal models available that month, and the answer was a Tencent family called WeMM, which reads text inside images, plus SigLIP and Jina. I already had the two-billion-parameter WeMM downloaded for another reason.

In a comparison with 1,500 cards and sixty queries like "the art of such-and-such card", CLIP found the right card one time in four. WeMM got nine in ten, SigLIP came close, Jina landed in the middle. WeMM's price is being about thirty times slower than CLIP, and running it over the whole corpus would take almost a day on my machine. So the comparison stayed on the sample, and the full corpus kept CLIP for now.

What came out of this wasn't "switch to WeMM". It was that the model can't be a constant at the top of a script. It has to be a list of presets inside the engine, and whoever builds picks which one, or which ones, at build time. And the video has to fit inside the file, not next to it. Before either of those two things, I wanted to know how far I could compress without losing anything. Even if we had to invent it.