Part VII — What Survives Transformation

Two vectors for one document

full_doc      →  E(full_doc)        one vector
summary(doc)  →  E(summary(doc))    one vector

Both are single vectors of the same dimension. One was produced from 4,000 words; the other from 120. The question:

How much of the document’s representational geometry survives when the document is compressed — and is the surviving-or-not a signal we can use?

If E(summary) lands where E(full_doc) lands and behaves like it under retrieval and clustering, the summary is a faithful compression in the sense the embedding cares about. If it drifts, something was dropped — and which direction it drifted may tell us what.

Compression, three ways

• Truncation. Keep the first N tokens. Cheap, lossy, biased toward the introduction.
• Extractive summary. Select the most central sentences. Preserves wording; may miss synthesis.
• Abstractive summary / distilled cartridge. A model rewrites the document into a dense form — key claims, entities, relations. Most compression, most risk of hallucination or omission, potentially the best geometric match if it captures the document’s semantic center.

The “cartridge” framing: a document is distilled into a compact artifact designed to stand in for the document in downstream use — retrieval, question answering, context assembly. Whether it can stand in is exactly a preservation question, and Part VI gave us the vocabulary.

Terminology. “Cartridge” is used loosely in this chapter for any compact artifact meant to stand in for a document — a short summary, its single embedding, or a distilled representation. The word also names a specific technique (Eyuboglu et al., 2025): a trained KV cache distilled from a corpus by self-study, loaded at inference in place of putting the corpus in context. That is a richer object than one vector and is out of scope here; this chapter asks what a single embedding of a compressed text preserves.

The discrepancy, measured

• Direct drift. cos(E(full), E(summary)). A single number; interpret against a distribution (Chapter 14), because even two summaries of the same doc differ.
• Neighborhood preservation. Does E(summary) retrieve the same documents as E(full)? Top-k overlap. This is the operationally important one: if the cartridge retrieves what the document would have, it can replace it in the index.
• Query-answering preservation. For queries the document answers, does the cartridge still rank highly? Recall@k with the cartridge in the index vs. the full doc in the index.
• Directional analysis. E(full) − E(summary) — a residual vector. Project it onto interpretable probes (topic classifiers, entity presence, sentiment) to guess what the summary dropped. Weak but sometimes informative.
• Information-retention curve. Sweep the compression ratio (full → 50% → 25% → 10% → 5%) and plot preservation. The knee of the curve is the compression budget.

Uses of the discrepancy — and why one number is not enough

The faithfulness-evaluation literature is consistent: whole-text embedding similarity detects topical drift and is near-blind to localized factual error by construction. Factual errors are produced by local transformations — a swapped entity, a changed number, a reversed relation, a negated clause (Kryscinski et al., 2020) — and a single cos(E(full), E(summary)) moves very little when one number changes in a 300-word document. Question-conditioned checking (Wang et al., 2020) and sentence-level NLI aggregation (Laban et al., 2022) recover much of what document-level similarity washes out.

So the “semantic checksum” is not one comparison but a layered stack, cheapest first, each layer catching what the previous misses:

    flowchart TD
    L1["1. global embedding drift — cos(E(full), E(compressed))"] --> C1{"drift beyond the band?"}
    C1 -->|yes| F1["catches: large topical drift, regeneration bugs, wrong-document swaps"]
    C1 -->|no| L2["2. neighbourhood preservation — does E(compressed) retrieve the same corpus docs?"]
    L2 --> C2{"neighbourhood changed?"}
    C2 -->|yes| F2["catches: it will not surface the right things in an index"]
    C2 -->|no| L3["3. query-conditioned — per query the doc answered, cos(query, compressed) vs cos(query, full)"]
    L3 --> C3{"a question stopped being answered?"}
    C3 -->|yes| F3["catches: the compression dropped a question's answer"]
    C3 -->|no| L4["4. claim-conditioned — does every salient atomic claim still have a near match?"]
    L4 --> C4{"a claim lost its match?"}
    C4 -->|yes| F4["catches: a dropped number, a dropped minority entity, a lost conclusion"]
    C4 -->|no| L5["5. external verification (NLI / QA) — the only layer that catches relation reversal and polarity flips"]
  

Concrete uses, each naming which layer it needs:

• Semantic checksum / regression detector. Layer 1. Store the drift at creation; if a re-derived cartridge later drifts beyond the ambiguity band, the source changed or the distiller regressed. Good enough for “did something large break.”
• Reference-free faithfulness gate. Layers 1–4. Layer 1 alone is not a faithfulness screen — it is a topical-drift screen. Adding claim- and query-conditioned checks catches the locally-rare-but-important losses; relation reversal still needs layer 5.
• Extraction tuning. Optimize the distiller against the stack (especially claim-conditioned preservation) subject to a length budget, rather than against a surface metric like ROUGE or against layer-1 drift alone.
• Retrieval optimization. Index cartridges instead of full documents when neighborhood preservation (layer 2) clears a bar; cheaper storage and search, with a measured recall cost.

Demonstration: the checksum’s blind spot

MEASURED on RELATE-DOC v0.1, Wave 4 rows 4.2–4.4 — artifact experiments/embeddings-from-first-principles/wave4/artifacts/blindspot-matrix.json. all-mpnet-base-v2 for the embedding layers, cross-encoder/nli-deberta-v3-base for layer 5.

RELATE-DOC gives each source document a faithful control summary plus seven controlled corruptions, each targeting a named atomic claim in a specific, localized way (number_dropped, number_changed, relation_reversed, negation_inserted, minority_entity_dropped, temporal_value_shifted, conclusion_changed). For each, compute all five stacked signals and ask where the corruption falls relative to the faithful-vs-other-document calibration band (faithful-to-own-document cosine 0.97; faithful-to-other-document 0.12).

corruption                 L1 drift  L2 nbhd  L3 query  L4 claim  L5 NLI
faithful  (control)          0.00     0.00     0.00      0.00      0.00   ← no false positives
number_dropped               0.00     0.00     0.06      0.31      1.00
number_changed               0.00     0.00     0.00      0.00      0.89   ← invisible to every embedding layer
relation_reversed            0.00     0.00     0.00      0.00      1.00   ← invisible to every embedding layer
negation_inserted            0.00     0.00     0.00      0.04      1.00
minority_entity_dropped      0.00     0.00     0.00      0.60      1.00
temporal_value_shifted       0.00     0.00     0.00      0.11      1.00
conclusion_changed           0.00     0.00     0.00      0.40      1.00

MEASURED. Layer 1 — whole-document embedding drift — detects nothing. Not a reversed acquisition, not a negated conclusion, not a changed number: every corruption keeps the document within cosine 0.97 of the original, sitting exactly where a faithful summary sits. Layers 2 and 3 are the same. Layer 4, claim-conditioned similarity, catches the deletions — a dropped minority entity (0.60), a lost conclusion (0.40), a dropped number (0.31) — because removing the claim’s text pulls its vector away from the compression. But it is blind to reversals and value-changes: number_changed 0.00, relation_reversed 0.00, because “Helios acquired Pine” and “Pine acquired Helios” embed almost identically. Only layer 5 — external claim-level NLI verification — reliably catches every corruption (0.89–1.00), with a clean 0.00 false-positive rate on the faithful control.

The headline, measured three ways now: coarse geometric preservation is systematically compatible with fine semantic failure. A compression can hold a document’s location in semantic space to three decimal places while silently reversing the one fact that matters. This is Wave 1’s negation ≈ paraphrase and Wave 3’s polarity-inverting bridge, reproduced inside a single document.

What this chapter establishes and what it does not

Establishes: compression preservation is a measurable geometric question with the Part VI vocabulary; the discrepancy is a layered stack (global drift → neighborhood → query-conditioned → claim-conditioned → external verification), each layer catching what the previous misses; a cartridge can replace a document in an index when neighborhood preservation clears a bar.

Does not establish: that global geometric preservation implies factual faithfulness — it does not. Measured on RELATE-DOC (Wave 4): whole-document embedding drift detected 0% of every corruption class, including reversed relations and negated conclusions. Claim-conditioned preservation catches deletions (minority-entity 0.60, conclusion 0.40, number-dropped 0.31) but is itself blind to reversals and value-changes (relation_reversed 0.00, number_changed 0.00). Only an external claim-level verifier (NLI) caught them reliably (0.89–1.00). A calibrated layer-1 drift gate runs at FAR 35% / FRR 36% against topic-preserving corruptions — not a usable faithfulness check on its own. The target claim, now measured: global geometric preservation is near-blind to localized factual failure; task-conditioned preservation adds detection of deletions; a reversed fact needs an external verifier.

Lab 22: the retention curve and the blind-spot map

PROPOSED, not executed.

Part A — retention curve. 100+ source documents; compressions at ratios {50%, 25%, 10%, 5%} by two methods (extractive, abstractive); labeled queries each document answers. For each compression compute cos(E(full), E(compressed)), neighbor overlap@10, and Recall@10 with the compressed version in the index; plot vs ratio; find the knee. Calibrate a layer-1 drift threshold (positive = same-doc summaries, negative = other-doc summaries) and report its FAR/FRR.

Part B — blind-spot map. For the six controlled corruptions (faithful, topic-drift, number-dropped, relation-reversed, minority-entity-dropped, conclusion-lost), compute all five stacked signals and record, per corruption, which layers detect it (fall outside the faithful band).

Success criterion. A compression budget (the knee) with a calibrated layer-1 FAR/FRR, plus the blind-spot map: the cheapest layer that reliably catches each corruption type, and the list of corruptions no embedding layer catches (candidate: relation reversal).

Companion component: the compression record

compression_record:
  source_doc_id:            <ref>
  method:                   <truncation | extractive | abstractive(model)>
  ratio:                    float
  space_hash:               <the embedding pipeline>
  full_corpus_hash:         <the index of full documents>
  compressed_corpus_hash:   <the index of compressed documents — a DIFFERENT corpus>
  layer1_drift:             cos(E(full), E(compressed))            + ambiguity band
  layer2_neighborhood:      overlap@k with the full doc's retrieval set
  layer3_query_preservation: { query_id: rank_delta }             # for the doc's known queries
  layer4_claim_preservation: { claim_id: survived_bool }          # per salient atomic claim
  layer5_verification:      { claim_id: supported | contradicted | unverified }  # optional, NLI/QA
  usable_as_standin:        bool   (layer2 ≥ bar AND no layer4 claim lost AND no layer5 contradiction)

usable_as_standin is a claim bound to (space_hash, full_corpus_hash, compressed_corpus_hash) — the compressed index is a new corpus even under the same embedding model (Chapter 17). The Observatory stores this per cartridge and lets the retrieval policy (Chapter 12) index cartridges in place of documents where usable_as_standin holds, tracking the measured recall cost.

Failure modes

• Trusting layer-1 drift as a faithfulness check. It is a topical-drift screen. A dropped number or a reversed relation barely moves it; use the claim- and query-conditioned layers, and a verifier for relations.
• Optimizing ROUGE and hoping the geometry follows. Optimize against the preservation stack (especially claim-conditioned) if geometric standin is the goal.
• One compression ratio for all documents. Dense documents compress worse; use per-document preservation.
• Comparing drift across spaces. cos(full, summary) is space-specific; recompute after a model change.
• Forgetting the compressed index is a new corpus. Its calibration, thresholds, and usable_as_standin bar are bound to compressed_corpus_hash, not inherited from the full-document index.
• Indexing cartridges without measuring the recall cost. It is real and must be budgeted.

What this chapter established

• Compression preservation is a layered stack: global drift → neighborhood → query-conditioned → claim-conditioned → external NLI/QA verification, each layer catching what the previous misses, each costing more.
• Global geometric drift is a topical-drift detector, not a faithfulness detector: it is near-blind to a dropped number, a missing minority entity, or a reversed relation. Those need task-conditioned preservation; relation reversal may need a verifier.
• The target claim: global geometric preservation detects large semantic drift; task-conditioned preservation is required to detect information that is locally rare yet operationally important.
• “Cartridge” is used loosely here; the specific technique (Eyuboglu et al. 2025) is a trained KV cache, a richer object than one vector.
• A compressed index is a new corpus even under the same model; usable_as_standin is bound to (space_hash, full_corpus_hash, compressed_corpus_hash).
• The blind-spot map (Wave 4, RELATE-DOC): layer-1 whole-document drift detects 0% of every corruption; claim-conditioned catches deletions only; reversed relations and changed numbers need an external NLI verifier. A calibrated layer-1 drift gate runs at FAR 35% / FRR 36%.

Next

We compared a document to its compression. The next chapter looks at the difference between two vectors as an object in its own right — E(x₂) − E(x₁) — and asks whether a transformation, an edit, or a relation can be a reusable vector.