Semantic drift in LLMs: metric, detection and mitigation

I put this together as technical grounding for the screenplay about consciousness in an LLM, which uses semantic drift as its mechanism. The first version was a research dump with ten topics, from integrated information theory to communication between models, and it had errors precisely in the part that gave the post its name: the formula and the code did not match the paper, and several numbers were not in it. I redid everything from the source and kept only what is about drift.

What it is

The term comes from Know When To Stop: A Study of Semantic Drift in Text Generation, by Ava Spataru, Eric Hambro, Elena Voita and Nicola Cancedda, published at NAACL 2024. Three authors are from FAIR, at Meta, and one is listed with an Anthropic affiliation. The first version of this post called it a "joint Meta and Anthropic study", which overstates it.

The observation is simple: when a model generates a long text, such as a Wikipedia-style biography, it tends to write the correct facts first and then "drift" into incorrect ones. This had been noticed before but never measured. The paper measures it.

The metric

Each text is broken into atomic facts, and each fact gets a label $s_i$: 1 if it is supported by the source, 0 if not. With $N$ facts, the drift score tries every cut position $k$ and keeps the best average of two proportions: correct facts before the cut and incorrect facts after it.

$$\text{SD}_m(P) = \max_k \tfrac{1}{2}, \text{SD}_m(P, k)$$

$$\text{SD}m(P, k) = \frac{\sum{i=0}^{k-1} s_i}{k} + \frac{\sum_{i=k}^{N-1} (1 - s_i)}{N - k}$$

The value is zero when $k < m$, $N - k < m$ or $N < 2m$. The hyperparameter $m$ is the minimum number of facts on each side of the cut, and the position that gives the maximum is the drift point. A high score means a text that is almost all right before that point and almost all wrong after it.

The first version of this post weighted each side by $k/N$ and $(N-k)/N$ and called the minimum $\alpha$, as a fraction. That is not the paper's formula. The corrected code:

def semantic_drift_score(labels, m=1):
    """labels: list of 0s and 1s, in the order the facts appear."""
    n = len(labels)
    best, drift_point = 0.0, None
    if n < 2 * m:
        return best, drift_point
    for k in range(m, n - m + 1):
        left = sum(labels[:k]) / k
        right = sum(1 - s for s in labels[k:]) / (n - k)
        score = (left + right) / 2
        if score > best:
            best, drift_point = score, k
    return best, drift_point

In the paper's figure 1 example, the best cut leaves 88% correct facts on the left and 78% incorrect ones on the right, and the score is the average, 0.83.

What they measured

Generated biographies, with facts checked by a variant of FActScore the authors call FActScore*. Both numbers are percentages:

Drift shows up in all of them, with little difference between models, while accuracy varies a lot. Being more accurate does not protect against the right-first, wrong-later pattern.

Other numbers from the paper: in 37% of paragraphs, the drift point is in the first 10% of facts. In more than 40% of paragraphs, the score is statistically high in a permutation test. Only 34 paragraphs, under 7%, have at least 10 correct facts before the first error. The first version said "75% of paragraphs show drift in the first 25% of facts", which is not in the paper.

Detection

To stop generation at the right point, you need to know, while generating, when the model started to go wrong. Of the confidence measures tested, the one that correlated best with accuracy was SelfCheck with BERTScore, which samples several generations and measures how much they agree, with a Pearson correlation of −0.41.

Mitigation

With Llama 2 70B, the paper's table 4 compares strategies:

The oracle is the ceiling: knowing the exact point, accuracy goes from 44% to 82%, but at the cost of cutting correct facts along with it, 58% of them, according to the limitations section. Real strategies land in between, trading amount of information for accuracy. Reranking by similarity raises accuracy without cutting the text. And trying to pull the model back with an external API did not work. The first version said "Resample-Then-Rerank from 53.27% to 63.72%", and the second number does not exist in the paper.

What was cut

The first version had two more drift metrics and eight other topics. Out went:

• The "Composite Drift Metrics", attributed to ResearchGate with no link or author
• "Contextual Flux" (arXiv 2502.10942), from 2025, whose authors include a "Zachary Mountebank"; a mountebank is a charlatan, and the paper looks like the batches of auto-generated papers that showed up on arXiv. It also postdated this post
• The "Trajectory Volatility Score" and "Semantic Divergence Metrics", with no source
• The Claude timeline, which cited 2025 versions in a 2024 post
• Consciousness, integrated information theory and global workspace: that belongs to the screenplay and to the poetic ontology
• Model extraction and jailbreaks, which are checked in the reading list on extraction
• Distillation, Mamba, mixture of experts, computational neuroscience, famous chatbot failures and communication between models, which were research summaries unrelated to drift

A note on the name: in linguistics, "semantic change" or "semantic drift" is a word's meaning shifting over time, and tools like LSCDetection measure that. It is a different phenomenon. The drift in this post is a text that starts right and ends wrong within a single generation.