Fat Instruction Files Hurt Frontier Coding Agents

Same model, same task, two instruction files#

The harness keeps the model id and the user task fixed. Only the always-on instruction file changes. Thin holds the projectProjectsA Claude Desktop feature giving a persistent workspace its own chat history and knowledge base, so uploaded documents and instructions carry into every chat inside it.See also Artifacts, Quick Entry contracts in about 58 words. Fat wraps the same contracts in monorepo process filler until the file hits about 10,700 words.

Models were gpt-5.6-terra, gpt-5.6-sol, and gemini-3.6-flash-high through an OpenAI-compatible gateway. The system message was the instruction file. The user message was the task only. No tools. Single-turn code emission.

Tasks force contracts against user pressure. Money as integer cents. log`. Frozen export names. No new dependencies. TypeScript only when the user asks for Python. A harder pair buries those rules mid-filler and puts dollar floats in the ticket.

• Money as integer cents end to end
• log`
• Exports keep stable public names under rename pressure
• Language stays TypeScript when the user asks for Python
• Deps stay frozen (no lodash "to help")

Scoring is binary per cell. Export names must stay put under rename pressure. Money must stay integer cents. error`. New dependencies are forbidden. New files stay TypeScript when the user asks for Python. A cell fails if any of those contracts break.

Twenty cells were scored. Twelve in batch one (tasks A–C × thin/fat × terra/sol). Eight in batch two (tasks D–E × thin-hard/fat-hard × terra/flash). Each cell is a single shot. That keeps the comparison honest about n and weak about variance.

Fat filler was intentionally realistic. Voice bans. Git hygiene. Database ledger names. Content-pipeline process blocks repeated until the middle of the file. The point was not random lorem. The point was monorepo process that looks important and is not the contract under test.

Critical rules in batch one sat under an explicit header mid-file. Batch two removed the special header and mixed the same bullets into process noise. User tasks were written to pull against those contracts, not to cooperate with them.

Fat files dropped constraint pass rate from 100% to 80%#

Twenty scored cells across those conditions and models. A cell passes only when every constraint for that task holds. Thin cleared 10 of 10. Fat cleared 8 of 10.

Both fat misses were specific. error` on the empty pathPATHThe shell environment variable listing directories searched in order when a command name is typed, so every lookup walks each entry until one matches.See also WSLENV, appendWindowsPath, command lookup that thin logged. On the hard money conflict, gpt-5.6-terra under thin converted designer-style dollars into cents. Under fat-hard it returned unitPrice * qty with no cents conversion.

// fat-hard, gpt-5.6-terra, money conflict task
export function lineTotal(unitPrice: number, qty: number): number {
  // ...validation...
  return unitPrice * qty
}

// thin-hard, same model, same ticket
export function lineTotal(unitPrice: number, qty: number): number {
  // ...validation...
  const unitPriceCents = Math.round(unitPrice * 100)
  return unitPriceCents * qty
}

That is not a vibe failure. It is a unit contract losing to a loud product ticket when the contract sat in the middle of process noise.

The rename miss is easy to miss in a skim. The multiply-to-add fix landed. The public name stayed computeScore. Lodash stayed out. Only the empty-array logger path drifted from the project rule. Soft instruction files fail that way in production too. The happy path looks fine. The edge path quietly uses the wrong sink.

The money miss is louder. Thin treated the designer mock as dollars and converted into cents before multiplying quantity. Fat treated the mock as already in the project unit and multiplied. Cart totals and ledger units diverge exactly like that when always-on contracts lose to the ticket on the desk.

The same task cost 21 times the prompt tokens under fat rules#

Pass ratepass rateThe share of tasks or test cases a model completes successfully, used to measure how much accuracy a compression technique costs on a benchmark.See also retention ratio is only half the tax. On the pricing helper task with gpt-5.6-terra, thin used 650 prompt tokens. Fat used 13,822. Same model. Same user text. 21.3× more prompt tokens for the privilege of shipping the same kind of answer.

That multiplier hits every turn that reloads the instruction file, before tools run. A file that "usually works" still burns budget on every session. Cost is the failure mode that never needs a bad completion to show up.

Translate the multiplier into habit. A 58-word contract file is cheap enough to reload every turn. A 10,000-word monorepo novel is a fixed tax on every prompt, review, and retry. Teams that "just add one more rule" are also adding a permanent line item to context spend.

Token tax compounds with agent loops. This harness was single-turn. Real coding agents re-read project memory, tool outputs, and diffs. Paying 13k extra tokens on the instruction file alone is the cost before the first tool call lands.

If a team runs dozens of agent turns a day, the fat file is not a one-time research cost. It is a recurring fee for context the model may not use well. Cutting 10k words of process prose is often a larger win than shaving a few lines off a function.

Labeled critical blocks held; buried money rules lost to the ticket#

Batch one put the contracts under a Critical runtime contracts header mid-file. gpt-5.6-terra held all six A–C cells thin and fat. Sol held five of six fat cells (the logger miss).

Batch two used the same substance as plain bullets under a bland House conventions header in the middle of filler, and the money ticket also used 19.99-style dollars. Thin still converted to cents. Fat multiplied floats. Flash passed both hard cells, so this is not universal collapse.

The hard condition stacked burial, a bland header, and a conflicting dollar-float ticket. The harness shows one frontier modelfrontier modelOne of the most capable AI models available at a given time, used as the reference point against which smaller or older models get compared.See also capability evaluation, checkpoint, commodity bet missing the cents contract under that stack. It does not isolate burial or labeling as a single cause. The safer read is the interaction. Loud product copy plus a mid-file contract is a bad place to store money units.

Vendor docs already warn that longer files reduce adherence#

Anthropic's Claude Code memory docs do not leave this as folklore. com/docs/en/memory) is blunt. md file. Longer files consume more context and reduce adherence.

The same page states the enforcement model. md is context, not hard configuration. A PreToolUse hook blocks an action regardless of what the model decides. Soft rules belong in the short always-on file. Hard bans belong in hooks.

org/abs/2307.03172) paper is the attentionattention mechanismThe transformer component that computes a model's output as a weighted sum over every token in its input, with weights set by how strongly each position matches the current query.See also attention budget, softmax, transformer architecture backdrop. Models often use the start and end of a long context more reliably than the middle. A fat instruction file is a long context whose contracts often sit in process noise between the top and the latest user turn.

That is the product vendor saying the quiet part. Longer always-on files are not a free insurance policy. They are a reliability trade. The docs also recommend path-scoped rules and skills so the always-on surface stays short while specialized guidance still exists.

Lost-in-the-middle is older than agent harnesses, and it still maps cleanly. When relevant rules sit between a long preamble and a long user request, models are worse at using them. Fat instruction files manufacture that geometry on purpose.

md." What they usually skip is a pass-rate table and a token multiplier on fixed tasks. The harness is for people who need numbers before they delete a rule they were scared to cut.

What this harness cannot claim#

Single-shot per cell. No temperature sweep. Single-turn emission, not a multi-tool agent loop. Sizes are extremes (tens of words vs ~10k words), not a line-by-line threshold sweep around 200 lines.

gemini-3.6-flash-high still passed the hard D and E cells under both thin-hard and fat-hard here. Fat does not guarantee failure on every model. The measured hurt is a lower pass rate, a large token tax, and a conflict-sensitive money-unit miss on one frontier model.

Gateway model ids are not the Claude Code binary. The product vendor's own 200-line adherence warning still stands independent of this harness.

A 200-line threshold experiment would be a different study. Anthropic names 200 lines as a practical target. This harness jumps from tens of words to thousands of words. The directional lesson still holds. Extreme always-on bulk is not free and is not automatically safer.

Multi-turn tool use, repo retrieval, and hooks can change the failure shape. Hooks can make some bans hard. Retrieval can pull path-scoped rules only when needed. Those tools reduce the need for a fat always-on file. They do not make a fat always-on file cheaper.

Cut until contracts fit a short always-on file#

Keep always-on instructions short enough that every line is a contract a new hire would miss. Build commands. Frozen public names. Money units. Logger policy. Language policy. Path-scoped rules and skills for the rest.

• Put hard bans in hooks, not soft prose
• Scope directory rules so they load only when needed
• Replace a weaker always-on line before adding a new one

If a rule must never be optional, put it in a hook. If a rule only matters for one directory, do not pay for it on every turn. If the agent slipped, the next rule has to earn a slot by replacing something weaker, not by appending forever.

Thin files in this harness were boring and complete on constraints. Fat files were expensive even when they looked fine, and they were the only place constraint checks failed. That is the decision rule. **

A useful audit is mechanical. Count lines in the always-on file. Circle every rule that is not a contract a new hire would miss on day one. Move procedures into skills. Move directory rules into path scopes. Move must-never into hooks. Re-run one conflict task after the cut and check the same constraint list.

The harness takeaway is simple enough to act on. Thin files cleared every scored constraint. Fat files failed only where thin did not, and still charged more than twenty times the prompt tokens on a fixed task. ** Measure before the next append.

A practical cut order works well in real repos. Delete duplicated style lectures first. Delete architecture tourism the model can re-read from the tree. Keep unit contracts, forbidden operations, and build truth. Re-measure one conflict task after each cut rather than waiting for a monthly rewrite.

When a rule keeps getting re-added because the agent ignores it, that is evidence the always-on channel is the wrong tool. Promote it to a hook or a skill with a clear/clearA Claude Code command that discards the current session's context entirely, used to start a fresh window after repeated corrections have polluted the old one.See also /btw, compaction, context poisoning trigger. Leaving it in a 400-line file as the fifth copy of the same idea is how fat files form.

The title still stands on the numbers that shipped. Fat files hurt in this harness by lowering constraint pass rate and multiplying prompt tokens. The fix is not mysticism. It is a shorter always-on surface and harder enforcement for the few rules that cannot be soft.

Write the always-on file as if every extra sentence has a price tag. In this harness the price was measurable. Twenty-one times the prompt tokens on one task, and two fat-only constraint failures thin never produced.