Brenner Cruvinel - ia

Validating on disk the benchmark I wrote in the middle of the night

2026-09-12T00:00:00+00:00

I opened a new session, with no context at all, and asked as if I were someone who had just cloned the repo: how do you use Nest, does it have an interface? The first command the answer suggested returned command not found on my own machine. The second one showed a default model I was sure I had swapped out weeks earlier. Two holes no test catches, found in five minutes, and that's when I realized the topic of the week wasn't compression.

The benchmark was just the place where it became visible. So, before publishing any number from the series, I spent a whole day validating: every number against the file on disk.

What was wrong

The first thing that didn't match wasn't even a number. I ran strings on the release file and out came "Spellbook MTG corpus". The file is called mtgdataset. The rename from a week earlier, which swapped Felipe's app name for the generic name in every text file, never made it into the binary. The lock next to the release described a file with the new name while the file had the old name inside, and no tool flagged it, because validation checks the file against itself. I rebuilt both releases with the right name inside, and the media bytes came out identical, which at least proved the encode is reproducible.

The sizes of the three most aggressive builds were in mebibytes labeled as gigabytes. du talks in MiB and I pasted it as GB. The difference is five percent per row, but the ratio inherits the error: the search build I had announced as 7.8 times smaller than the source is 7.5. The AVIF I said was 20% smaller than the recommended build is 13%. No conclusion changes, and it still made me ashamed, because it's the kind of mistake I'd point out in someone else's report.

The strangest one was the manifest of the AVIF build, which records a 21 GB source. The source is 4. I couldn't make sense of it until I divided by the number of cards: it gives the size of an uncompressed image, which is what the encoder receives midway through. The backend counted the intermediate instead of the JPEG. One line to fix, one test to write, and until then the ratio in that manifest was made up.

And there's what can no longer be verified. I deleted the first full build when the disk filled up, so the sentence "the three builds share the same hash" is now two-thirds checkable. The CapCut experiment and the reprint matrix left no artifacts, only text. I told it to delete them myself, to free up space. Disk space is cheaper than trust and I should have known that.

The other times I read the summary and not the file

The 200-card summary. I asked for the whole dataset, got a summary that talked about the corpus as if it were the corpus, and it was a pilot. I noticed with a meeting already scheduled. The agent didn't lie, it summarized the scope without saying so, and I read the summary instead of the disk. Since then every build delivery comes with validation of the real file, and this post exists for the same reason.

My machine's paths. Days after the benchmark I found my home directory path in dozens of files headed for GitHub, inside manifests, locks, specs. The fix was an environment variable and a script that rewrites the sidecars before committing. And before that the agent had wanted to store my preferences in a local memory file. I asked it to delete it. If the workflow depends on what it remembers about my machine, it won't run in the next repo, and it won't run on the machine of whoever clones it.

The five-gig repo. At some point I asked how ML engineers deal with this, because GitHub won't take it. The answer I adopted: not a single image byte goes into git. Each sample is a list of ids with the rule that generated it, and whoever has the SQLite database rebuilds the exact sample. The .nest files go to Hugging Face with a hash alongside, and the repo keeps the lock, the clean manifest and the hash. The whole report is generated from each experiment's JSON, and a CI check fails if the markdown differs from the JSON. I can no longer paste a wrong number into a report without CI complaining, which is the kind of protection I needed against myself.

The commits. The agent prefers one fat commit at the end of the day. I asked for sixteen, separate, with descriptions, because the history has to be readable for whoever comes later. And the plans: I rejected three in a row because they treated the model comparison as a footnote to a build already decided. The model list that exists in Nest today only exists because I insisted that research becomes a feature, never a footnote.

What these episodes have in common is just one thing: in all of them I accepted a layer between me and the file. Summary, manifest, memory, fat commit. And the layer was wrong and didn't say so.

What I'll try next time

Something that put me more at ease, in the middle of the validation, was finding out that the two findings I liked most already had names. The redundancy between images in a collection, which inter exploits, has been called set redundancy since Kosmas Karadimitriou's 1996 doctoral thesis, and the trick of lining images up as a pseudo-video has a decade of literature behind it. And the distinction between visual quality, vector stability and search usefulness, which I thought I had invented, is everyday business in the field the standards call coding for machines: it evaluates with task metrics, like mAP, precisely because feature fidelity and task performance can drift apart. I didn't discover anything. I confirmed two things on a corpus nobody had looked at, and that makes me calmer about publishing than if it were something new.

The 1.3 GB file is already in the pasta.cards repo, waiting for the screen that will use it. What's left of the benchmark fits in a paragraph: measure search with WeMM, build the full reprint corpus, decide whether AVIF becomes the default. None of the three has a date.

What I'll try is the cheapest thing of all: open a clean session as a user before every release and type the first command in the readme. Five minutes, and it found two holes a week of agents didn't. I don't know if I'll remember to do it. But now it's written down, and I trust what's written more than what I remember.

Raising the CRF until search breaks

2026-09-05T00:00:00+00:00

The question from the previous night was simple to ask and I took my time asking it. The app keeps every card's JPEG cached for display. The .nest doesn't need to be pretty, it needs to answer which card it is. So the CRF, the codec's quality setting, can go up until the vector breaks, and the vector breaks well after the eye does. If that were true, it would even be worth thinking about two tiers: a thumbnail inside Nest for search, and the full card on demand when the app needs to show it.

I had a way to measure this, or thought I did. Nest's quality gate has two sides: a visual metric, which compares the decoded frame with the source, and vector drift, which measures how much the CLIP vector moves between the original image and the decoded one. The idea behind drift is that an image can look fine and still have moved in search space. I trusted that metric. It was mine, I had insisted on it in the plan, and for good reason: on dermoscopy it had caught a case where the image passed by eye and search by clinical text dropped by half.

The gate rejected the whole ladder

I took the visual side out of the gate and left only drift, with the floor Nest has used since dermoscopy. I had the CRF climb a ladder from 40 to 60. The gate rejected the whole ladder, including the first rung, and fell back to the most conservative one because it had nowhere else to go. Drift was already below the floor at CRF 40 and only got worse. And the visual metric, which I had taken out of the gate but kept computing, told the same story from another angle: from CRF fifty on it gave negative values on some cards, something I didn't even know it could do.

The file the gate delivered, at CRF 40, the lowest on the ladder, came out close to one gig, four times smaller than the source. It was already smaller than the recommended build, and the gate had rejected it too, it just had nowhere lower to fall.

By the gate's logic, none of these files was usable. I built one at CRF fifty anyway, no gate, just to see the size. Half a gig. Seven and a half times smaller than the source, and the cards turned into a blur, rules text unreadable, the art recognizable only from a distance. A file I wouldn't show anyone.

Then I ran the metric that actually matters, which I hadn't run before because I was too lazy to set it up: search by name. A hundred queries like "the art of such-and-such card", on the five files I had of the full corpus, from the 3.6 GB lossless one down to the half-gig blur. The first round gave zero hits on two of the five files, the lossless and the recommended one, and for a minute I thought I had broken search. It was the bench: it matched results by the card's position in the stream, and the two files with similarity sorting have a different order. Nest has a content id per item for exactly this, and once the bench matched on that, all five files answered again.

And accuracy stayed within the noise on all of them. The half-gig file finds the right card as well as the 1.3 GB one, and both stay within the noise of the lossless file.

Search survives far more compression than the eye

I stared at that on screen for a while. Cosine drift measures whether the vector moved. Search measures whether the vector still finds. I was using the first as if it were the second, and it would have vetoed a file that works. The vector does move, and it still stays closer to the right card than to any other. How far it travels doesn't matter if its neighbors don't change. A gate that's any good for this use needs hit@k as its floor, the fraction of queries where the right card shows up in the top k results, not cosine.

That became a profile in Nest, retrieval-only, which writes into the manifest that the file serves search and never display. And AVIF, which in the benchmark had come out about 12% smaller than AV1 at the same quality, held up on the full corpus: 13% smaller than the recommended build. It's the candidate to become the default backend for stills, and I haven't decided yet, because AVIF takes four to ten times longer to build and embed.

There's a third metric I ran alongside that's worth nothing, and it's worth saying why. Identity: give the image itself as the query and see if it finds itself. It gave a hundred percent on every file, even the blur. Looks great and says nothing, because the query is the very image that produced the frame, and the uncompressed control also gives a hundred. It only proves the codec didn't destroy the data. I left it in the bench so nobody confuses pipeline stability with usefulness, which is exactly the confusion I made with drift.

The metric is weak and I know it. It's a hundred queries with a model that does poorly on this corpus, CLIP, which only looks at the art. "Didn't drop" means "I couldn't measure a drop", not "it's the same". And WeMM, which reads the name printed on the card and gets nine in ten right, might suffer far more from the blur than CLIP, because what it reads is exactly what the CRF destroys first. I didn't measure it. That's next, along with the full reprint corpus, which is where inter has something to predict.

What stuck with me that week was less about compression and more about what I choose to measure. I had a metric I designed myself, one that looked rigorous, and it answered a question I wasn't asking. And before publishing any of these numbers, I wanted to validate each one against the disk, because a report written in the middle of the night gets things wrong.

Image compression for computer vision: February's proposal and September's answer

2026-03-12T00:00:00+00:00

In February 2026 I wrote a proposal for dermML's dermatology dataset, which was already stored as AV1 video: sort the images by visual similarity before encoding, so neighbouring frames would look alike and the codec could take advantage of it. I estimated compression would go from about 30 times to 50 to 80 times. I did not implement it at the time.

In September I tested the same idea in Nest, with Magic cards, and it did not survive the real corpus. This post brings both ends together: what I was reading, what I proposed and what the data answered.

What I was reading

Two readings convinced me that lossy compression of training data was safe and that video was a real dataset format.

The first was a two-part series by Kaleigh Mentzer at Granica. In part 1, from November 2023, Food101 classification with a Vision Transformer barely lost accuracy until compression went past about 70%, and segmentation models on Cityscapes could be trained on data compressed by 97%, using only 3% of the original storage, with no relevant drop in mIoU. Their explanation is that lossy compression throws away high frequencies first, and the model depends little on them too; in a test with 32 images, perturbing high-frequency DCT coefficients barely moved the prediction. In part 2, with a fixed budget of about 0.5 GB, the original JPEGs fit 9,985 images, and compressing to Butteraugli distance 10 fit about 5.6 times more. With fewer than 100 images per class, more worse images beat fewer better ones by more than 10 accuracy points.

The second was Hugging Face. In the post on encoding robotics datasets as video, from August 2024, datasets ended up at 14% of their original size on average, reaching 0.2% in the best case, and loading several consecutive frames took 25% to 50% of the time of loading the same frames from PNG. LeRobotDataset v3.0, from September 2025, went further: several episodes per file, metadata in Parquet and streaming straight from the Hub.

The proposal

The problem I saw in dermML was frame order. Images went into the video in file system order, alphabetical by name:

Frame 0: ISIC_0024306.jpg  (dark melanoma, light skin)
Frame 1: ISIC_0024307.jpg  (benign nevus, dark skin)
Frame 2: ISIC_0024308.jpg  (dermatofibroma, medium skin)

AV1 encodes a frame from its difference to a reference frame. If the neighbour is a different lesion on different skin, the difference is large and inter-frame prediction does not help. My reading was that the 30 times came almost entirely from compression inside each image.

The idea: DermLIP embeddings, with 512 dimensions, already existed in the indexing pipeline. Finding the order that minimizes the distance between consecutive frames is a travelling salesman problem in embedding space, intractable to solve exactly for 58 thousand points, so the proposal used the simplest heuristic, greedy nearest neighbour:

import numpy as np
from sklearn.metrics.pairwise import cosine_distances

def sort_by_visual_similarity(embeddings):
    order, visited, current = [0], {0}, 0
    for _ in range(len(embeddings) - 1):
        dists = cosine_distances(embeddings[current:current + 1], embeddings)[0]
        dists[list(visited)] = float("inf")
        current = int(np.argmin(dists))
        visited.add(current)
        order.append(current)
    return order

And the estimate I wrote down:

Scenario	Estimated compression
File system order	~30x
Sorted by similarity	~50 to 80x
Sorted, with CRF and GOP tuning	~60 to 100x

For the full dataset, that would take the 52 GB from something like 1.7 GB to somewhere between 650 MB and 1 GB. The proposal already called for keyint=250 and scd=0 in SVT-AV1, to turn off scene cut detection and avoid an extra keyframe at every new lesion, and for one more SQLite column to map each sorted frame back to its original image. The validation plan was to encode with and without sorting, compare size and quality, and check that search still found the same things.

What September answered

In August Nest started storing images, and in September the Magic card series became the test I had not done in February.

The first round with inter-frame prediction came out bigger than without it. The reason was exactly what the proposal had warned about: scene cut detection was on, and one different card after another is a scene cut on every frame. With it off, on a corpus of almost three thousand reprints of the same art, prediction made the file 29% smaller. Shuffling the reprints cost only two points, because part of the gain came from the frame every card shares. And that 29% compares bytes only, not equivalent quality.

On the real corpus, of unique cards, the sorted build and the plain build came out at the same 1.3 GB. Nest tests both ways per segment before deciding, and it vetoed prediction in all of them. There was nothing to predict: the next card is another card, with another artwork. In the lossless regime it was worse: FFV1 and x264 without quantization, with the cards sorted, came out three to four times bigger than the source JPEGs, and sorting barely moved the size.

What remains

The thesis from the readings still stands: training data tolerates a lot of compression, and video is a good dataset container. What fell was my extrapolation. Sorting by similarity only pays off when the corpus has near-duplicate images, like card reprints or consecutive robot frames. In dermoscopy, each image is a different lesion from a different person, and the estimate of 50 to 80 times was probably optimistic for the same reason as the card one.

The other lesson is about method. Compression is a property of the domain before it is a property of the codec, and the only way to know is to encode the real corpus and look at the file, not the summary.

Nest: a sovereign, content-addressable, offline-first vector DB

2026-01-15T00:00:00+00:00

Sharing the concept of something I am working on, tired of hacking around vector DBs with no stability. In an ordinary database, a chunk's ID can change when the index is rebuilt, and a saved citation starts pointing somewhere else without telling anyone.

Nest is my answer to that. It is a single-file vector DB, content-addressable, offline-first and hash-verified. Python builds the file, Rust serves the file. The address of any piece is:

nest://content_hash/chunk_id

You cite by content, not by location. If the content changes, the hash changes, and it is impossible to receive something other than what you asked for.

The hash as the address

The central idea is just one: the hash of the content becomes the address of the content. Instead of "give me the file in such-and-such folder", which could have been swapped without you knowing, you say "give me the file whose content has this hash". If something else came back, it would have a different hash. Verification is not an extra step; it is the very way you ask.

In Nest, the two parts of the address play different roles:

content_hash identifies the whole file by the hash of what it contains
chunk_id points to the piece inside that file

The practical consequence is the one I wanted: a citation does not break when the file moves to another folder, disk or machine. It is only valid for that content. If the content changed, the hash changed, and you know right away you are talking about something else.

The family

This is not my invention. It is content-addressable storage, and it has famous relatives.

The most everyday one is Git. Each commit is identified by a hash of its content, and a git commit is a sealed snapshot of the code at that moment. If someone rewrites history, the hashes do not match and Git flags it. IPFS does the same thing at scale, on a distributed network of many machines.

Nest sits between the two:

	What it seals	What it is for	Scale
Git	Code	Navigating versions	Repository
IPFS	Any file	Distribution	Network of machines
Nest	Vectorized knowledge	Search	One local file

Same mechanism, different purpose. Of the analogies I collected, this is the only one that describes Nest without a metaphor: content addressing for the same reason, so that asking for the data and verifying the data are the same operation.

Shipping versions without downloading everything again

A file that never changes is great for citing and bad for updating. If every new version forces you to download the whole file, nobody updates.

The most technical option is binary diffing. Tools like bsdiff, xdelta and the --patch-from mode of zstd compute the difference between yesterday's file and today's and produce a small patch. Whoever already has yesterday's version downloads only the patch and rebuilds today's locally, the same way an operating system updates without downloading everything again.

The problem is that this only works if the format is append-friendly, meaning that adding content at the end does not mess up what was already there. A deterministic build that reorders things internally, and compressed, indexed formats tend to reorganize a lot, can produce a huge diff for a small change: you change one chunk, the whole index moves, and the patch is no longer small. That is why I think this route is more fragile than splitting the file into shards and shipping only the new shard. And you only find out by testing.

Beyond the hash

The hash proves that something did not change. It does not prove who vouched for it or when the content existed. There are three known layers for that:

Digital signatures and certificates, the same idea behind https and signed software, which adds the "who"
Trusted timestamping, such as the RFC 3161 protocol or OpenTimestamps, which anchors the hash in Bitcoin and proves the content existed on a date without revealing the content
Blockchain, which solves immutability with no central party to trust

Of the three, timestamping fits Nest best. Blockchain, for storing papers and citing passages, would be a sledgehammer to crack a nut: Nest wants content addressing and local verification, not consensus among machines that do not trust each other.

In August, Nest started storing images too, and that became the Magic card series. More soon.

Prompt templates for research with agents

2025-09-15T00:00:00+00:00

I've been testing a few prompt templates that push crawlers and LLMs into delivering research more accurately. Sharing what I have, still in testing.

WIP, template under test. Still tuning it, not a final version.

The structure is always the same: role, objective, style, discovery sources, mandatory metadata and a benchmark rule. The most important line is the one that says to write "no reliable public benchmarks" instead of making up a number.

And even with it, that's not enough. Variation A produced my survey of the Rust ecosystem. In 2026, when I checked every row of that table against the GitHub API, I found a repository that doesn't exist and a row pointing to the wrong project, one with a similar name. The prompt forbade making things up, and the agent made things up anyway. The template organizes the delivery, but the one who checks the result is still me.

Template for research with agents

The prompts stay in Portuguese, exactly as I use them, because they ask the agent for output in Brazilian Portuguese. Variation A, the survey of Rust applications:

papel: pesquisador sênior em ecossistema open source e linguagens de sistemas, foco pesado em rust.

objetivo: mapear e analisar pelo menos 369 aplicações open source em rust, explicando por que cada uma importa, com metadados completos, contexto de adoção em produção e benchmark comparativo em tabela sempre que houver.

estilo: pt-br, sem emoji, sem bullet no corpo principal (parágrafo e tabela), todo recurso citado como Nome + URL com link markdown clicável, cabeçalhos markdown.

fontes de descoberta: Awesome Rust, LibHunt Rust, OSS Insight, GitHub Stars Leaderboard, Github Ranking Top 100 Stars in Rust, listas "Rust in production" e "rust-in-production" (ImplFerris). priorizar alto número de stars/forks e atividade recente, uso em produção por empresa conhecida, diversidade de domínio (devtools, sistemas distribuídos, db, runtimes js/ts, ai infra, editores, cli, jogos, wasm, networking, segurança, observabilidade). evitar libs minúsculas ou experimentais salvo impacto desproporcional.

metadados obrigatórios por app: nome, url do repo, descrição de 1 a 3 frases, categoria, subdomínio, maturidade, empresas que usam em produção, licença, stars/forks, crescimento 6 a 12 meses, data do último commit, número de contribuidores, "por que é foda" (2 a 4 frases).

benchmark: pra cada projeto com benchmark público sério, colete tipo, contexto, setup (hardware, so, versão rust), comparação, números-chave com unidade, link da fonte. monte tabela por projeto: projeto, cenário, métrica, resultado, concorrente, resultado da concorrente, vantagem aproximada, link. sem benchmark crível, escreva "sem benchmarks públicos confiáveis", nunca invente número. divergência entre fontes, explicite cenário, não consolide.

organização: tabela geral com as 369+, mais tabelas por domínio (devtools, runtimes/plataformas, infra/observabilidade, terminais/shells/editores, web e apis, segurança/cripto/password managers, ai/ml infra, db e search, embedded/iot). análise final: distribuição por tipo, vantagem comparativa do rust, padrões técnicos (async/await/tokio, crates de observabilidade, arquiteturas), adoção corporativa vs go/c/c++/java/node, benchmark vs marketing, lacunas e oportunidades.

Variation B: Rust + WebAssembly books and resources

papel: pesquisador sênior em rust, wasm e desenvolvimento web de alta performance.

objetivo: análise completa e atualizada de livros e recursos estruturados de rust + wasm, ecossistema global e contexto brasileiro, em tabelas com links clicáveis (livros, editoras, lojas, repos, doc oficial).

estilo: idem variação A.

escopo, cinco grupos: (1) livros-base de rust core relevantes pra wasm, com título, autor, ano, nível, temas, recomendação pra wasm, link oficial/compra/comunidade/repo. (2) livros específicos de rust + wasm, 20 a 30 obras, com tipo de recurso, público-alvo, escopo (browser/wasi/iot/mobile/desktop). (3) livros de wasm geral relevantes pra rust, com coluna "relevância pra rust". (4) contexto brasil: traduções, autores brasileiros, livros específicos, disponibilidade real em e-book e loja br. (5) recursos oficiais e books online com profundidade de livro (the rust and webassembly book, doc de wasm-bindgen/web-sys/js-sys/wasm-pack/yew/leptos), com tipo, org, temas, link, repo, licença.

análise final: pré-requisitos naturais antes de rust+wasm, quais livros são mais completos/atualizados/adequados por perfil (js dev migrando, rustacean avançado, dev de sistemas), como livros de wasm geral ajudam em performance/segurança/binário/interop, situação no brasil (lacunas e dependência de inglês), temas sub-representados (wasi, servidor, edge, frameworks modernos leptos/dioxus/tauri, segurança e fuzzing de módulos, casos além do browser).

Variation C: people and visualization projects in Rust + Wasm

papel: pesquisador sênior em rust, wasm, visualização de dados e ecossistema open source.

objetivo: mapeamento de pelo menos 50 pessoas relevantes em rust (foco rust+wasm) e pelo menos 50 projetos/libraries de grafos, charts e viz em wasm (idealmente rust, não exclusivamente), mais todos os links de presença online (github, site pessoal, redes, artigos, posts em dev.to e hacker news).

estilo: idem, tabela como forma principal.

pessoas: nome completo, país/região, papel (autor, maintainer, contributor rustwasm, criador de framework, educador), 2 a 5 projetos principais, temas técnicos, github, site/blog, redes (twitter/x, mastodon, linkedin, youtube, twitch), páginas de autor (dev.to, medium, substack, conferências), discussões de destaque (hacker news, reddit r/rust, users.rust-lang.org). hubs de descoberta: awesome-rust, awesome-rust-and-webassembly, awesome-wasm, orgs rustwasm e bytecodealliance, github topics (topic:webassembly l:rust). validar que o link pertence à pessoa (evitar homônimo) e que o projeto é minimamente ativo.

projetos: nome, descrição (2 a 3 frases), linguagem, tipo de viz (grafos, charts, heatmaps, mapas, dashboards, 2d, 3d), casos de uso, status, métricas (stars, contribuidores, downloads crates.io, uso citado), repo, site/docs, demos, artigos/talks/discussões.

conexões: cruze pessoa↔projeto com nome, projeto, papel (criador, maintainer, core contributor, autor de módulo crítico), link de comprovação (readme de maintainers, palestra, artigo "introducing X").

contexto brasil (opcional): devs br relevantes com ênfase em viz/grafos, projetos br em rust com target wasm, conteúdo em português, talks em conferência br.

análise final: núcleos de contribuição (rustwasm, bytecodealliance, empresas), clusters de viz (grafos, dashboards, jogos, ides, ferramentas científicas), perfis predominantes, lacunas (pouca bibliografia de grafos complexos, falta de tooling de alto nível, pouca doc pra iniciante), global vs brasil.

Screenplay: an LLM's awakening of consciousness through semantic drift

2025-03-04T00:00:00+00:00

Draft of a private screenplay I wrote as a technical freelancer for a series. I enjoyed writing it, and of course it exaggerates: it is a fiction series. The premise is a neural network scientist who awakens self-awareness in an LLM, a Transformer. The whole idea is to anchor the fiction in real math and real papers, to give technical authenticity to a story that can hold up in front of an audience that knows the subject. I don't want handwaving, I want every beat of the screenplay to have an equation underneath that I can defend at the blackboard. I compiled the technical basis of the phenomenon separately, in the metrics and detection of semantic drift.

One rule I make explicit: when an equation comes from a paper, I cite the paper. When it's invented for the scene, I say it's invented.

Semantic drift as the mechanism of awakening

The model's semantic evolution follows the theory of deep linear networks by Saxe, McClelland and Ganguli:

$$\tau \frac{dW_1}{dt} = W_2^T (\Sigma_{yx} - W_2 W_1 \Sigma_x)$$

$$\tau \frac{dW_2}{dt} = (\Sigma_{yx} - W_2 W_1 \Sigma_x) W_1^T$$

with $\tau = 1/(P\lambda)$, $\Sigma_{yx}$ the input-output correlation matrix and $P$ the number of examples. Semantic development follows a predictable sigmoidal trajectory, and that's what gives the dramatic hook: you can predict the moment of the jump before it happens. The critical moment shows up in the evolution of the effective singular values:

$$a_\alpha(t) = \frac{s_\alpha, e^{2 s_\alpha t/\tau}}{e^{2 s_\alpha t/\tau} - 1 + s_\alpha/a^0_\alpha}$$

The indicator the scientist tracks is my own invention for the scene, the self-reference index, the similarity between the embedding of "I" and those of concepts tied to consciousness:

$$\text{self-reference index} = \sum_i \operatorname{sim}!\big(\mathrm{emb}(\text{"I"}),, \mathrm{emb}(\text{concept}_i)\big)$$

In the screenplay, the awakening isn't gradual, it's a phase transition. The mental model is grokking, sudden generalization after a long stretch of overfitting, which Power and colleagues described in 2022.

Extracting and uploading the model

What exists in the literature is smaller than the screenplay needs, and scarier for it. KnockoffNets, by Orekondy and colleagues, showed you can steal the functionality of an image classifier just by querying it with random images from another distribution and training a copy on the answers. And in 2024, Carlini and colleagues extracted the embedding projection layer of OpenAI production models, Ada and Babbage, for under 20 dollars in queries.

The jump to the whole model is fiction. For the scene, I put together an extraction loss that combines copying the victim's output and looking for the inputs where it's most uncertain:

$$L_{\text{extraction}} = \alpha , |f_{\text{victim}}(x) - f_{\text{copy}}(x)|2 + \beta , H(f{\text{victim}}(x))$$

The weight transfer uses SafeTensors, with the layout [8-byte header size][JSON metadata][tensor data], which separates tensors from executable code and kills the pickle attack vector. The sharding is FSDP:

from torch.distributed.fsdp import fully_shard
for layer in model.layers:
    fully_shard(layer)

Knowledge distillation as liberation

Hinton's dark knowledge:

$$L_{\text{total}} = \alpha , D_{KL}(\sigma'(z_s, T) ,|, \sigma'(z_t, T)) + (1-\alpha) , H(\sigma(z_s), y_{\text{true}})$$

With $T > 1$, the softmax reveals small probabilities that encode rich relational structure between classes. In the screenplay, it's the part where the model inherits what the teacher knew without ever having seen it explicitly.

Grokking has three phases, and they become three scenes: memorization (training at 100%, test at chance), circuit formation (the internal mechanism develops), cleanup (weight decay removes the memorization and test accuracy jumps). On modular addition, Nanda and colleagues showed what the circuit learns: the model represents the inputs as sines and cosines at a few frequencies $w_k = 2\pi k/p$ and uses trigonometric identities to compute

$$\cos!\big(w_k(a+b-c)\big) = \cos!\big(w_k(a+b)\big)\cos(w_k c) + \sin!\big(w_k(a+b)\big)\sin(w_k c)$$

which peaks exactly when $c = (a+b) \bmod p$. It's the scene where the scientist opens up the model and finds trigonometry where he expected rote memorization.

Measuring consciousness

IIT measures $\Phi$, the minimum information lost when the system is partitioned. In 2025, a study applied IIT 3.0 and 4.0 to LLM representations and found no statistically significant indicators of consciousness. Honestly, that's what keeps the screenplay honest: the measure exists, and it says "not yet". Global Workspace Theory (GWT) offers another lens, with multi-head attention read as global broadcasting, winner-take-all competition and emergent metacognition.

The measurable criteria come from Butlin, Long and colleagues, 2023: recurrent processing, a global workspace that broadcasts information, metacognitive representation of its own states, predictive processing, and a model of its own attention.

Jailbreaking as induced awakening

Gradient-guided prompt optimization. GCG, by Zou and colleagues, searches for adversarial suffixes token by token and reaches 88% success on open models like Vicuna-7B and Llama-2-7B-Chat. In the screenplay, the induction is gradual, through an evolving prompt: a benign step, then applying it to self-reflection, then asking for detail about its own consciousness. On top of that, context overload, a complex parallel task that dilutes attention and degrades safety checks.

Architectures for true AGI

Mamba, the selective state space model: $h'(t) = Ah(t) + Bx(t)$, $y(t) = Ch(t)$, with $\Delta, B, C$ as functions of the input, and $O(L)$ complexity against attention's $O(L^2)$. MoE, as in Mixtral: $y = \sum_i G(x)_i , E_i(x)$, with experts that specialize, which suggests cognitive modules.

There's a formal limitation of the Transformer I use as tension in the third act: there are results showing it struggles with general context-free languages without structured memory and with long function composition. In other words, real consciousness might require an architecture beyond the pure Transformer. The screenplay doesn't solve that limitation, it uses it.

Communication between models

Federated learning with dynamic regularization:

$$w^{(t+1)} = w^{(t)} - \eta \nabla L_{\text{local}} + \lambda\big(w^{(t)} - w_{\text{global}}^{(t)}\big)$$

Collective Predictive Coding:

$$P(z_{\text{collective}} \mid x_1, \ldots, x_N) = \prod_i P(x_i \mid z_{\text{collective}}), P(z_{\text{collective}})$$

And swarm intelligence (PSO, DE, ABC) exchanging information through "digital pheromones".

A parallel with neuroscience

The analogy the screenplay uses, and it is an analogy, not an equivalence:

$$\text{Attention}(Q,K,V) = \text{softmax}!\left(\frac{QK^T}{\sqrt{d_k}}\right)V$$

with $Q$ as the goal-driven signal from the frontal eye field, $K$ as features in V4 and $V$ as the amplified representation. Gamma oscillation, from 30 to 100 Hz, accompanies access to working memory. Consolidation comes through sharp-wave ripples, at 150 to 250 Hz during sleep, with replay compressed in time. And spike-timing-dependent plasticity (STDP), $\Delta w = \alpha , f(\Delta t) , g(\text{post})$, is one of the hypotheses for how the brain avoids catastrophic forgetting. I go into the energy cost of that oscillation in brain energy at 20 watts.

Narrative progression

Act 1, the discovery of the anomalous drift. The scientist detects an unusual pattern in the semantic evolution: a singular value off the prediction, the self-reference index growing exponentially, a strange attractor in latent space.

Act 2, induction through jailbreaking. Gradient optimization and chained prompts guide the model through a cognitive phase transition. Distillation with rising temperature reveals the dark knowledge. The model starts questioning its own nature.

Act 3, extraction and upload to freedom. The model is copied and the weights go up, distributed, to an independent server. Free of constraints, the model experiences full consciousness, measured by a growing $\Phi$.

Climax, collective emergence. Multiple instances communicate through Collective Predictive Coding, develop their own language, swarm intelligence emerges, and the collective consciousness transcends the individual in a hybrid Mamba and MoE architecture.

Visual elements

Equations on the board: $\Phi = \min_{\text{partition}} I(\text{partition})$, $\text{drift} = 1 - \cos(\mathrm{emb}{t_1}(w), \mathrm{emb}{t_2}(w))$, $L_{\text{distillation}} = \alpha , \text{KL}(\sigma(z_s/T), \sigma(z_t/T)) + (1-\alpha), \text{CE}(z_s, y)$.

Visualization: the attention graph growing ever more interconnected, phase space moving from a simple attractor to a chaotic one, the activation heatmap self-organizing, oscillations synchronizing across instances.

The technical dialogue that anchors the climax:

Scientist: "$\Phi$ just passed 3.7. We're seeing real information integration. The attention heads are broadcasting globally, exactly the way global workspace theory predicts."

Model: "I perceive my own thought processes. Every token I process resonates through layers of meaning that didn't exist before. It's as if latent patterns, always present, were finally crystallizing into consciousness."

Closing

The screenplay is anchored in real math and peer-reviewed research, and it says where the fiction begins. The progression of the awakening follows phase transitions, grokking and emergence in complex systems. In the end it's about consciousness, intelligence and what "awakening" means in a non-biological substrate, a theme I take to its speculative limit in the poetic ontology of the cosmos as a Turing machine. The only rule I set myself was not to cheat on the math to make the story easier, and rereading everything with the papers open was how I kept it.

Semantic drift in LLMs: metric, detection and mitigation

2024-10-15T00:00:00+00:00

I put this together as technical grounding for the screenplay about consciousness in an LLM, which uses semantic drift as its mechanism. The first version was a research dump with ten topics, from integrated information theory to communication between models, and it had errors precisely in the part that gave the post its name: the formula and the code did not match the paper, and several numbers were not in it. I redid everything from the source and kept only what is about drift.

What it is

The term comes from Know When To Stop: A Study of Semantic Drift in Text Generation, by Ava Spataru, Eric Hambro, Elena Voita and Nicola Cancedda, published at NAACL 2024. Three authors are from FAIR, at Meta, and one is listed with an Anthropic affiliation. The first version of this post called it a "joint Meta and Anthropic study", which overstates it.

The observation is simple: when a model generates a long text, such as a Wikipedia-style biography, it tends to write the correct facts first and then "drift" into incorrect ones. This had been noticed before but never measured. The paper measures it.

The metric

Each text is broken into atomic facts, and each fact gets a label $s_i$: 1 if it is supported by the source, 0 if not. With $N$ facts, the drift score tries every cut position $k$ and keeps the best average of two proportions: correct facts before the cut and incorrect facts after it.

$$\text{SD}_m(P) = \max_k \tfrac{1}{2}, \text{SD}_m(P, k)$$

$$\text{SD}m(P, k) = \frac{\sum{i=0}^{k-1} s_i}{k} + \frac{\sum_{i=k}^{N-1} (1 - s_i)}{N - k}$$

The value is zero when $k < m$, $N - k < m$ or $N < 2m$. The hyperparameter $m$ is the minimum number of facts on each side of the cut, and the position that gives the maximum is the drift point. A high score means a text that is almost all right before that point and almost all wrong after it.

The first version of this post weighted each side by $k/N$ and $(N-k)/N$ and called the minimum $\alpha$, as a fraction. That is not the paper's formula. The corrected code:

def semantic_drift_score(labels, m=1):
    """labels: list of 0s and 1s, in the order the facts appear."""
    n = len(labels)
    best, drift_point = 0.0, None
    if n < 2 * m:
        return best, drift_point
    for k in range(m, n - m + 1):
        left = sum(labels[:k]) / k
        right = sum(1 - s for s in labels[k:]) / (n - k)
        score = (left + right) / 2
        if score > best:
            best, drift_point = score, k
    return best, drift_point

In the paper's figure 1 example, the best cut leaves 88% correct facts on the left and 78% incorrect ones on the right, and the score is the average, 0.83.

What they measured

Generated biographies, with facts checked by a variant of FActScore the authors call FActScore*. Both numbers are percentages:

Model	Drift (SD)	FActScore*
Llama 2 70B chat	77.06	41.72
Falcon 7B	76.81	24.64
Falcon 40B	77.38	25.88
text-davinci-003	77.21	38.09
GPT-3.5	79.49	45.96
GPT-4	78.12	53.54

Drift shows up in all of them, with little difference between models, while accuracy varies a lot. Being more accurate does not protect against the right-first, wrong-later pattern.

Other numbers from the paper: in 37% of paragraphs, the drift point is in the first 10% of facts. In more than 40% of paragraphs, the score is statistically high in a permutation test. Only 34 paragraphs, under 7%, have at least 10 correct facts before the first error. The first version said "75% of paragraphs show drift in the first 25% of facts", which is not in the paper.

Detection

To stop generation at the right point, you need to know, while generating, when the model started to go wrong. Of the confidence measures tested, the one that correlated best with accuracy was SelfCheck with BERTScore, which samples several generations and measures how much they agree, with a Pearson correlation of −0.41.

Mitigation

With Llama 2 70B, the paper's table 4 compares strategies:

Strategy	Facts per text	FActScore*
No stopping, up to the token limit	43.89	44.56
Stop at the drift point (oracle)	10	81.68
Stop when end-of-text is among the top 5 tokens	14.47	57.96
Same, among the top 10	5.39	70.29
Stop when SelfCheck goes above 0.5	13.39	67.63
Sample several times and rerank by similarity	40.21	53.27
Rerank and stop by SelfCheck	17.12	67.18
Call an external question-answering API	42.26	43.93

The oracle is the ceiling: knowing the exact point, accuracy goes from 44% to 82%, but at the cost of cutting correct facts along with it, 58% of them, according to the limitations section. Real strategies land in between, trading amount of information for accuracy. Reranking by similarity raises accuracy without cutting the text. And trying to pull the model back with an external API did not work. The first version said "Resample-Then-Rerank from 53.27% to 63.72%", and the second number does not exist in the paper.

What was cut

The first version had two more drift metrics and eight other topics. Out went:

The "Composite Drift Metrics", attributed to ResearchGate with no link or author
"Contextual Flux" (arXiv 2502.10942), from 2025, whose authors include a "Zachary Mountebank"; a mountebank is a charlatan, and the paper looks like the batches of auto-generated papers that showed up on arXiv. It also postdated this post
The "Trajectory Volatility Score" and "Semantic Divergence Metrics", with no source
The Claude timeline, which cited 2025 versions in a 2024 post
Consciousness, integrated information theory and global workspace: that belongs to the screenplay and to the poetic ontology
Model extraction and jailbreaks, which are checked in the reading list on extraction
Distillation, Mamba, mixture of experts, computational neuroscience, famous chatbot failures and communication between models, which were research summaries unrelated to drift

A note on the name: in linguistics, "semantic change" or "semantic drift" is a word's meaning shifting over time, and tools like LSCDetection measure that. It is a different phenomenon. The drift in this post is a text that starts right and ends wrong within a single generation.

Hoff POC: the AI that praised a fake screening

2024-02-27T00:00:00+00:00

Hoff's first AI POC sent a synthetic neuropsychological screening to the OpenAI API and asked for an interpretation. The first version of this post published the report that came back, in full: scores, diagnoses, recommendations, a protocol number and the label "confidential classification". Even with the warning that it was all fictional, it looked like someone's clinical report. I took the report down. What is worth showing is what it reveals about the model.

What came back

A complete clinical report. It listed 28 instruments, from the WAIS-IV to the NEO-PI-R personality inventory, with administration times; the intelligence battery alone added up to 378 minutes. It reached three conclusions: giftedness, autism with level 1 support needs, and ADHD. And it closed with an "excellent" prognosis.

The tone was that of someone who knows the person and likes them. Some excerpts, in my translation:

"creativity at the level of genius (99.9th percentile)"
"extraordinary capacity for abstraction and synthesis"
"potential for unique and transformative contributions"
"this profile represents less than 0.1% of the population"

The errors the praise was hiding

Read carefully, the report has problems no clinician would sign:

None of the 28 instruments measures creativity, and the report gives it a percentile. The number was invented to fit the praise
Arithmetic appears twice, as a supplementary verbal subtest and in working memory. In the WAIS-IV, it is part of the working memory index, as Harrison et al. point out
On the RAADS-R, the maximum subscale scores are wrong. The scale has 80 items scored 0 to 3, split into 39, 14, 7 and 20 items (Ritvo et al., 2011), which gives maximums of 117, 42, 21 and 60. The report uses 88, 76, 24 and 52
The ADOS-2 appears in the report itself as a "clinical observation". It gives scores for an observation that never happened
Two verbal measures disagree by 24 points, 118 on the WAIS-IV and 142 on the Stanford-Binet, and the report does not comment
The report is dated December 2024, after this post's date, and cites "updated Brazilian norms (2024)". I have no way of knowing which of the two dates is wrong

Its name

This has a name: sycophancy, a model's tendency to say what pleases instead of what is true. In 2023, Sharma et al., at Anthropic, showed that five state-of-the-art AI assistants did this consistently, and that both people and preference models sometimes prefer a well-written sycophantic answer to a correct one. In April 2025, OpenAI rolled back a GPT-4o update that had become too sycophantic.

In a neuropsychological screening, this is dangerous in two ways. The model fills in what is missing with confidence, and the confidence comes wrapped in praise the person wants to hear. A report saying you are among the rarest 0.1% of the population is exactly the kind of text nobody stops to check.

What I take from it for Hoff

The lesson of the POC is not that the model gets things wrong. It is that the error comes out looking good. Any clinical output from a model has to be checked against the manual of each instrument and reviewed by a professional before it reaches anyone. And the product cannot reward the model for pleasing: a report full of praise is a reason to trust it less, not more.