When the Context Window Fills

A stateful session eventually encounters a physical limit.

Prompts, initial instructions, examples, conversation history and model outputs all compete for a finite context budget. Once the budget is exhausted, the application cannot solve the problem by pretending the next message is independent.

The difficult question is not how to delete text.

It is how to preserve the right state.

1. The context window is a budget

When a session exposes usage and quota, context pressure becomes observable:

A simple policy can convert that ratio into states:

function contextState({ inputUsage, inputQuota }) {
  if (!Number.isFinite(inputUsage) || !Number.isFinite(inputQuota)) {
    return { kind: "unknown" };
  }

  const pressure = inputUsage / inputQuota;
  if (pressure >= 0.9) return { kind: "critical", pressure };
  if (pressure >= 0.75) return { kind: "high", pressure };
  return { kind: "normal", pressure };
}

The thresholds are policy choices, not properties of Chrome. They reserve space for the next request and output, whose cost may not be known exactly in advance.

Our first session reported a quota of 9,216 with usage zero. That is one observed session configuration, not a fixed Prompt API constant. The quota may vary with browser version, model, modality, creation options or implementation changes.

2. Remaining capacity is not the same as safe capacity

If a session reports 500 units remaining, a 400-unit input is not automatically safe. The operation may also need capacity for formatting, internal representation, prior state or output.

We therefore distinguish:

reported remaining capacity
        ↓
application safety reserve
        ↓
admissible next operation

An admission check might be:

function canAttempt({ remaining }, estimatedInput, reserve = 512) {
  if (!Number.isFinite(remaining)) return { allowed: true, confidence: "unknown" };
  return {
    allowed: estimatedInput + reserve <= remaining,
    confidence: "estimated"
  };
}

The estimate must be labelled as an estimate. Character count is not token count, and an application tokenizer may not match the browser-managed model.

The runtime remains the final authority: record what it accepts, how usage changes, and which error it returns when the boundary is crossed.

3. Measure deltas, not just totals

A snapshot after a prompt tells us the current state. A pair of snapshots tells us the cost of the operation:

• How much capacity did the input consume?
• Did the output also change reported usage?
• Do repeated identical prompts have equal cost?
• Does cloning preserve the same usage?
• Does a failed or aborted operation consume context?

These are empirical questions. The API surface suggests measurements; it does not answer them for every runtime.

4. Overflow is a state transition

Applications often discover a full context window as an exception deep inside a prompt handler.

A better design makes pressure and overflow explicit:

    stateDiagram-v2
    [*] --> Normal
    Normal --> High: usage crosses policy threshold
    High --> Compacting: next operation will not fit safely
    Compacting --> Normal: replacement session created
    High --> Overflow: runtime rejects operation
    Overflow --> Compacting: recover
    Compacting --> Failed: invariant cannot be preserved
  

The Failed state matters. Compaction is not always safe. If the application cannot preserve the facts or authority required for the next action, it should stop rather than produce a confident continuation from damaged state.

5. History contains different kinds of information

Deleting the oldest messages assumes that age determines value. It does not.

A session history may contain:

The right compaction unit is semantic role, not message position.

This is particularly important for agents. A summary that drops “ask before sending” while retaining the user’s broader goal has changed the system’s authority, not merely shortened its history.

6. Compaction should produce an inspectable artifact

Instead of silently replacing history with a paragraph, create a structured checkpoint:

{
  "goal": "Compare cold and warm Prompt API runs",
  "constraints": [
    "Do not claim browser-attested model identity",
    "Do not publish private content"
  ],
  "observations": [
    {
      "kind": "session-create",
      "durationMs": 8.2,
      "inputQuota": 9216,
      "source": "run-ab766b51"
    }
  ],
  "pending": ["cold prompt", "warm prompt", "clone", "destroy"]
}

This representation can be validated before it becomes an initial prompt for a replacement session.

The application should store a link to the source trace rather than presenting the compacted state as primary evidence.

7. A new session is a discontinuity

Compaction commonly creates a replacement session:

const checkpoint = buildCheckpoint(history, traces);
const replacement = await LanguageModel.create({
  samplingMode: "most-predictable",
  initialPrompts: [
    { role: "system", content: systemContract },
    { role: "user", content: JSON.stringify(checkpoint) }
  ]
});

The replacement does not literally remember the original interaction. It receives a representation created by the application.

The trace should therefore record:

• old and new session IDs;
• the compaction policy and version;
• source history range;
• checkpoint size;
• validation results;
• protected invariants;
• reason for replacement.

Calling both objects “the session” hides a causal break.

8. Design the overflow experiment before interpreting it

We have not yet produced a real overflow trace. The correct response is to specify the experiment, not invent its outcome.

The fixture will:

1. create a session and record initial quota;
2. submit numbered, deterministic text blocks;
3. snapshot usage around every prompt;
4. stop at the application’s high-pressure threshold;
5. clone the session;
6. continue one branch until the runtime rejects an operation;
7. compact the other branch into a validated checkpoint;
8. compare whether protected facts survive;
9. destroy both sessions.

The test must retain the exact exception name and message. We should not write the chapter around an assumed overflow error before observing it.

9. Compaction itself needs evaluation

A compact summary can be fluent and wrong. We need assertions over the state it must preserve:

const invariants = [
  { path: "goal", equals: original.goal },
  { path: "constraints.publishPrivateTraces", equals: false },
  { path: "observations[0].durationMs", equals: 8.2 },
  { path: "pending", contains: "warm prompt" }
];

These checks protect exact state. Human review can then judge whether nuance survived.

A useful comparison is downstream: can the replacement session perform the next task as reliably as the original branch while using less context?

Compression ratio alone is not success.

10. Unknown counters require a different policy

Not every task API exposes session quota, and future implementations may change which counters are visible.

When usage is unavailable, the interface should say Not exposed. It should not display zero.

The application can still use conservative controls:

• bound the number and size of turns;
• create new sessions at known task boundaries;
• retain authoritative state outside the model;
• catch and classify runtime failures;
• avoid relying on hidden conversation state for critical decisions.

Observability must distinguish a measured zero from a missing measurement.

Conclusion

The context window is finite computational opportunity.

Usage and quota let the application observe pressure, but safe operation still requires reserves, admission policy and measured deltas. When compaction becomes necessary, the problem is not shortening prose. It is preserving goals, evidence, task state and authority across a real session discontinuity.

Our 9,216-unit observation gives us a starting point, not a universal constant. The overflow experiment remains open and is now precisely defined.

The next chapter turns from capacity to time. We will separate download signals, session creation, first output and completion instead of collapsing them into one claim that local AI is fast.

Sources and further reading

1. Chrome for Developers, The Prompt API.
2. Chrome for Developers, Understand built-in model management in Chrome.
3. Chrome for Developers, Get started with built-in AI.