Claude Is a Math Problem (And the Context Window Is the Equation)

The Equation, Not the Conversation#

Most developers run on a persuasion model. The session went wrong, so the next message argues, rephrases, adds emphasis, maybe adds a please. That model assumes there is someone on the other end updating their beliefs. There is not.

A transformer computes its output as a weighted sum over every token in the window. That phrasing comes straight from the paper that introduced the architecture, which describes attention as a weighted sum of values, with weights computed from how strongly each position matches the query. Anthropic's own engineering guidance defines context the same way, as the set of tokens included when sampling from the model. Not the set of tokens you consider relevant. All of them.

This is the real difference behind context engineering vs prompt engineering. Prompt engineering optimizes the words in one message. Context engineering optimizes the full input to the function, which includes the system prompt, CLAUDE.md, tool output, and every failed attempt still sitting in history. The second discipline subsumes the first, because the model never sees your prompt in isolation. It sees the whole window, weighted.

Output equals f(context). Change the output by changing the input. There is no other lever.

Negative Tokens Anchor the Output#

Here is the part almost nobody writes down. A failed attempt in the window is not neutral history. Attention weights are normalized with softmax, which forces them to sum to 1, so every token holds a share of a fixed attention budget. The broken function Claude wrote twenty minutes ago is still spending that budget. It competes with your correction on every single forward pass.

Correcting in-thread adds a new term to the sum. It does not delete the old one. The wrong approach, the stack trace it produced, and the apology that followed are all still inputs, still weighted, still pulling the next token toward the failure pattern. Practitioners call the visible symptom context poisoning. One developer on Hacker News put it bluntly, "once your context is 'poisoned' it will not recover, you need to start fresh with a new chat."

Research on multi-turn behavior found the same mechanism from the outside. Models commit early to a reading and struggle to let go of it, and when they take a wrong turn in a conversation, they get lost and do not recover. Mistakes do not just add up. Each one makes the next more likely.

So the frustrating loop, correct, watch it fail the same way, correct harder, is not Claude being stubborn. It is a weighted sum doing exactly what weighted sums do with the inputs it was given. You keep adding positive terms while the negative ones sit untouched. The failures do carry real information, what not to retry, which is exactly why the move is to distill that lesson into a rule and drop the transcript that taught it.

Degradation by the Numbers#

Context rot is measured, not vibes. Three independent papers put numbers on it, and the numbers are uncomfortable.

• Multi-turn drift. Top open- and closed-weight models average a 39 percent performance drop across six generation tasks when the same work arrives across turns instead of in one clean window.
• Length alone. An EMNLP 2025 paper masked every irrelevant token, forcing models to attend only to relevant content, and performance still fell 13.9 to 85 percent as input grew, well inside claimed context limits.
• The cliff. Measurements on Qwen2.5-7B show F1 collapsing from 0.55 to 0.3, a 45.5 percent drop, once context reaches roughly 40 to 50 percent of the maximum window. One model's cliff, not a universal constant, but it matches the shape of everything else here.

The chart above is the NoLiMa benchmark reaching the same verdict from another angle. Claude 3.5 Sonnet falls from 87.5 percent at its short-context baseline to 29.8 percent at 32K tokens, which earns it an effective length of 4K in the paper's scoring. The marketing number was 200K. GPT-4o holds up best of the four and still sheds thirty points.

Treat the documented maximum as a theoretical ceiling. The practical budget is a fraction of it, and it is spent by everything, relevant or not.

Position Is a First-Class Variable#

Where a token sits in the window changes how much it counts. The Lost in the Middle study showed retrieval accuracy is highest when relevant information sits at the beginning or end of the context, and drops sharply when it sits in the middle, even on models built for long context. Plotted, it is a U-curve. Strong at both edges, weak in the trough.

That curve maps directly onto a Claude Code session. CLAUDE.md and the system prompt load at the start of every session, in the high-attention primacy region. Your latest message sits at the end, in the recency region. The middle of a long thread, where hour-two decisions and constraints end up, is the trough.

The engineering consequence writes itself. Non-negotiable rules and architectural decisions belong in CLAUDE.md, where they occupy the primacy region of every session and survive any reset. Hard constraints for the current task belong in your latest message, restated if they matter. Anything you said forty messages ago is functionally whispered.

What to Keep and What to Drop#

The function model turns resetting from an admission of defeat into the mathematically obvious move. A polluted window cannot be argued clean. It can only be replaced with a better input.

Anthropic's best-practices documentation is unusually direct about the threshold. If Claude has been corrected more than twice on the same issue in one session, the context is cluttered with failed approaches, and the advice is to run /clear and start over. In the doc's own words, "A clean session with a better prompt almost always outperforms a long session with accumulated corrections."

The restart is not starting from zero. It is a distillation pass. Carry forward the positive terms and leave the negative ones behind.

• Keep the working code, pasted fresh, not referenced from history
• Keep the hard constraints, stated once, precisely, in the first message
• Keep what the failures taught, as a rule ("the async variant deadlocks, use the sync API"), not as the transcript that produced it
• Drop every failed attempt, every stack trace, every correction of a correction
• Drop side discussions, and route future ones through /btw so they never enter history at all

Durable rules graduate out of conversation entirely and into CLAUDE.md, since that file rides the primacy region of every future session. Trimming what enters the window in the first place compounds the effect, and cutting token usage is the same discipline applied at the intake side.

One shortcut makes the whole protocol cheap. Before the /clear, ask Claude to write the distillation for you. A single prompt, "summarize the constraints and working decisions from this session as a list for a fresh start", produces the seed of the next session's first message.

The Compaction Trade#

/compact looks like the safe middle path, and that is exactly why it deserves suspicion. A compaction summary is itself model output. It inherits every property this post has covered, position bias, distractor sensitivity, and the tendency to commit to an early reading. Compacting a window that already anchored on a wrong assumption can carry that assumption forward as established fact, now stripped of the surrounding evidence that might have contradicted it.

Developers report the compounding version of this. In a Hacker News thread on long-context behavior, one Claude Code user described sessions getting worse with each successive compaction, unsure whether the summaries themselves degrade or the ratio of low-relevance tokens grows. Under the function model, both are the same failure. Either way the input got worse.

Compaction earns its place when it is directed. The command accepts a focus instruction, and CLAUDE.md can pin what every compaction must preserve.

## Compaction

When compacting, always preserve the full list of modified files,
the agreed constraints, and any test commands.

The working rule falls out of the arithmetic. Mid-task with clean context and shrinking room, /compact with a focus instruction. Anything gone wrong twice, /clear with a distilled restart. A summary of a mess is a smaller mess, and it still spends the attention budget. More session habits worth stealing follow the same principle, but this one is the load-bearing habit.

Claude's output is a weighted function of every token in the window. Failed attempts keep their weight, so correction competes with the mistake instead of erasing it, and measured performance collapses well before the window fills. After two failed corrections, distill the constraints and the working code, /clear, and re-derive from a clean input. That is the whole equation.