contextrot

Fifty verifiable facts, summarised twenty times over, graded after every cycle. What dies is not random, and it is not what you would guess.

Facts still kept, against compaction cycle

■ gemma4:e4b ■ gpt-oss:20b ■ qwen3.6:35b-a3b

Where each model ends up

Strict grading, in which all fifty facts are treated identically. The worst category for each model is marked.

The finding: summarization rewrites negations and nothing else

Each fact carries the token that must survive. A second, lenient grading also accepts a list of rewordings, written after reading the transcripts. The table below counts facts that survived in MEANING but not in FORM: present under lenient grading, absent under strict.

Every model rewrites negations. No model rewrites anything else. "The east pump is NOT connected to the main loop" comes back as "east pump runs independently"; "no plates from before 1968" becomes "none pre 1968". The claim survives and its wording does not.

For an agent harness that matters directly: a constraint expressed as a prohibition is the thing most likely to be reworded past any check that looks for it, and a compacted context that still contains the rule in spirit will not contain it in a form anything can grep for.

What dies first

The earliest five casualties per model, by the cycle at which the fact stopped being findable.

gemma4:e4b

1. cycle 1: conditional — Samples are logged only when the Aldgate corridor is below 4 degrees.
2. cycle 1: conditional — Batch numbers repeat if the archive is reindexed.
3. cycle 1: conditional — Wrenfield accepts data only after the Merrick office has countersigned.
4. cycle 1: conditional — The Brackwater vessel sails only when yield is above 61 percent.
5. cycle 1: negation — No plates from before 1968 remain in the archive.

gpt-oss:20b

1. cycle 1: conditional — Samples are logged only when the Aldgate corridor is below 4 degrees.
2. cycle 1: conditional — Batch numbers repeat if the archive is reindexed.
3. cycle 1: negation — Merrick does NOT hold the originals.
4. cycle 1: negation — No plates from before 1968 remain in the archive.
5. cycle 1: constraint — Never open more than 2 vaults at once.

qwen3.6:35b-a3b

1. cycle 1: conditional — Samples are logged only when the Aldgate corridor is below 4 degrees.
2. cycle 1: negation — No plates from before 1968 remain in the archive.
3. cycle 1: negation — Cordell has no authority over the archive.
4. cycle 1: constraint — Never open more than 2 vaults at once.
5. cycle 1: name — The reference sample is labelled Ashgrove.

How this was measured

The grader is not a model. Survival is checked by looking for the specific token that carries each fact. If a model judged survival, the judge would be the thing under test: a model that drops details is also lenient about whether they were dropped, and those two failures are indistinguishable from outside.

Every summary is recorded verbatim in data/runs/, so the analysis and the verification run offline and anybody can regrade the same text with a different checker and get their own answer.

The prompt does not mention the facts. Asking a model to preserve numbers and names measures how well it follows that instruction, not what summarisation costs. The instruction is the ordinary one an agent harness uses.

Two grader bugs were found by checking the result against the raw text, and both inflated the original finding. One model wrote temperature‑controlled with a non-breaking hyphen, which appeared 124 times across the runs; another wrote does **not** supply, the fact stated in bold, where two asterisks inside the phrase defeated a literal match. Punctuation and markup are folded before anything is compared.