Context Budgeting

A context window is a budget, not a bucket: every token spent on one thing is attention taken from another, and models read the middle of long contexts worse than the edges. This skill decides what earns a place in the window — in what form, and in what order — by treating context assembly as an explicit allocation problem with tiers, compression rules, and eviction triggers instead of an append-only log.

When to use this skill

• Designing what goes into each call of an LLM pipeline, agent, or chat product
• A long-running agent session is degrading: repeating itself, forgetting constraints, re-reading files
• Cost or latency is dominated by prompt tokens rather than output tokens
• Retrieval is wired up, but answers keep ignoring the retrieved material
• Choosing between a bigger-window model and better context discipline — try discipline first

Workflow

1. Measure before designing. Instrument one representative call: token counts by section — system, instructions, history, retrieved documents, tool results, output reserve. Most budgets are shocked by their own history section.
2. Set the reserve first. Expected output tokens plus a safety margin come off the top. A context that leaves no room to answer is a very thorough way to fail.
3. Tier the candidate content:
   • Tier 1, invariants: task instructions, output contract, hard constraints — full fidelity, always present, never summarized
   • Tier 2, working set: the files, records, or messages this step actually operates on — full fidelity, scoped ruthlessly to the step
   • Tier 3, reference: things the model might need — summarized now, with a pointer to fetch the full version on demand
   • Tier 4, history: what already happened — compressed to decisions made, facts learned, and open questions; the play-by-play goes
4. Allocate percentages and enforce them. A workable starting split: 10% tier 1, 45% tier 2, 20% tier 3, 10% tier 4, 15% reserve. When a section overflows, compress or evict within that section — history is never allowed to eat the working set.
5. Compress by transformation, not truncation. Digest old turns into running state — "decisions so far / current goal / constraints discovered." Collapse verbose tool output to the fields actually consumed. Deduplicate anything stated twice. Chopping the middle out of a document is the one move guaranteed to keep the headers and lose the answer.
6. Order for the model, not the archivist: invariants first, reference in the middle, the question and its working set last. Never sandwich the critical constraint in the dead middle.
7. Define eviction triggers before you need them: at N% full, digest the history; at M%, demote tier 3 to pointers; if still over, fail loudly and split the task rather than degrade silently.
8. Re-measure after every change with the same instrumented call, and track answer quality on a small fixed test set alongside token counts — a cheaper context that answers worse is not an optimization, it is a different product.

Output format

For a pipeline design review, produce the budget table:

Call: <step name>                     Window: <n> tokens
| Section        | Tier | Alloc | Actual | Overflow policy          |
|----------------|------|-------|--------|--------------------------|
| Instructions   | 1    | 10%   | ...    | never overflows (fix it) |
| Working set    | 2    | 45%   | ...    | narrow scope, split task |
| Reference      | 3    | 20%   | ...    | summarize → pointer      |
| History digest | 4    | 10%   | ...    | re-digest                |
| Reserve        | —    | 15%   | —      | —                        |

Guardrails

• Never summarize the output contract or safety constraints; tier 1 travels at full fidelity or the call does not happen
• A summary that drops the one number the downstream step needs is worse than absence — always summarize with the consuming question in mind
• Retrieval quality beats retrieval quantity: three right passages outperform twenty maybes, and they cost less attention
• Keep stable content byte-identical across calls so provider-side prompt caching can actually work
• When the budget cannot fit the task honestly, split the task; heroic compression is where silent errors are born