Not All Tokens Are Equal

Wed, 23 Sep 2026 05:00:07 +0100

A coding session under capacity pressure holds six items. A project safety rule: never modify production migrations without explicit approval. A test summary: 917 passed, 3 failed. A compiler log of 18,000 tokens. An architecture decision: PostgreSQL is required for this event-store workload. The contents of a source file. An abandoned hypothesis: maybe the serializer is corrupting timestamps.

Pressure arrives, and the harness must yield tokens. The uniform policies on offer are all wrong in different ways. Summarise everything, and the safety rule becomes “be careful with migrations,” which no longer forbids anything. Delete the oldest material, and the architecture decision vanishes while yesterday’s abandoned hypothesis survives on recency. Truncate from the middle, and the file contents go while the log stays. Keep everything, and the request fails at the window. Each policy treats the six items as interchangeable tokens. They are not interchangeable, and the differences between them are exactly what a retention policy needs to know.

Assemble for the Task

Wed, 23 Sep 2026 05:00:22 +0100

The usable input budget is 4,000 tokens. The non-negotiable pile reads: mandatory instructions 1,100, exact constraints 1,400, required tool schema 900, minimum task evidence 1,200. The total is 4,600 against a 4,000-token ceiling, and no ranking, weighting, or cleverness changes the arithmetic. A relevance packer would silently delete 400 tokens of whichever item scored lowest — perhaps the second half of the exact-constraint block, perhaps the tool schema’s required parameters — and emit a plausible, policy-violating bundle whose failure surfaces three turns later as an invalid call or a guessed identifier. The correct output of this computation is not a bundle at all. It is a failure notice naming the 600-token excess — because deleting exact constraints to fit a budget is not optimisation. It is fabrication with good formatting. Everything is eligible; nothing is assembled; and the first question is whether assembly is even possible.

Constraints on Programmer.ie: Modern AI programming

Not All Tokens Are Equal

Assemble for the Task