Part IV — Measuring the Representation

What is 0.81?

A pipeline decides two documents are “duplicates” if their cosine exceeds 0.8. Someone picked 0.8 because it looked reasonable. Then:

distribution of cosine for KNOWN duplicate pairs:      mean 0.79, sd 0.09
distribution of cosine for KNOWN non-duplicate pairs:  mean 0.71, sd 0.11

The threshold sits inside both distributions. A true duplicate at 0.78 is rejected; an unrelated pair at 0.82 is accepted. The number 0.81 does not carry its own meaning — it is a point on two overlapping bell curves, and without those curves it is noise.

A similarity score is only interpretable relative to the distributions of scores for known-related and known-unrelated pairs. What are those distributions, and where do they cross?

Build the distributions

1. Positive set. Pairs you know are related in the sense you care about (from labels, from Chapter 1’s RELATE relations, from user feedback).
2. Negative set — and it must be hard. Random negatives give a flattering, wide gap (Chapter 11). Use topically-matched and structured negatives so the negative distribution sits where it will actually sit in production.
3. Plot both. Overlaid histograms of the similarity score.
4. Read the overlap. The region where both distributions have mass is the ambiguity band — scores in it cannot be classified by threshold alone.

Operating points, not thresholds

From the two distributions:

• False acceptance rate (FAR) at threshold t = fraction of negatives scoring ≥ t.
• False rejection rate (FRR) at t = fraction of positives scoring < t.
• ROC curve = FAR vs. (1 − FRR) across all t. AUC summarizes separability in one number, threshold-free.
• Choose t from a target, not a vibe. “FAR ≤ 1%” for a dedup pipeline that must not merge distinct records. “FRR ≤ 5%” for a recall-critical retrieval filter. The target comes from the cost of each error in your system.
• Ambiguity band [t_low, t_high]. Below t_low: reject. Above t_high: accept. Between: escalate — to a reranker, a cross-encoder, a second signal (Chapter 15), or a human.

    flowchart TD
    P["positive set — pairs related in the sense you care about"] --> H["overlaid score histograms"]
    N["hard negative set — topically-matched + structured, NOT random"] --> H
    H --> OV["read the overlap = the ambiguity band [t_low, t_high]"]
    OV --> T["choose the operating point from the COST of each error (target FAR or target FRR), not a vibe"]
    T --> D{"score vs the band"}
    D -->|"below t_low"| REJ[reject]
    D -->|"above t_high"| ACC[accept]
    D -->|"inside the band"| ESC["escalate — reranker / cross-encoder / second signal / human"]
    REJ --> RC["bind the record to (model version, corpus snapshot, query type, metric); re-derive when any of them changes"]
    ACC --> RC
    ESC --> RC
  

Why a threshold does not transfer

The positive and negative distributions move when:

• The corpus changes. New domain, new document lengths, new languages → anisotropy and score scale shift.
• The model changes. Even a minor version bump reshapes the score distribution (Chapter 17). A threshold is model-version-specific.
• The query mix changes. Short keyword queries and long natural-language queries produce different score ranges.
• Normalization or metric changes. Obvious in hindsight, common in practice.

So a calibrated threshold is an artifact tied to (model version, corpus snapshot, query type, metric) — and it needs re-derivation, on a fresh labeled sample, whenever any of those change.

Deterministic calibration

The calibration procedure should be reproducible: same labeled sample, same code, same threshold, every time. Fix the random seed for any sampling, version the labeled calibration set, and record the derived t, FAR, FRR, and ambiguity band as data. Two runs of calibration on the same inputs must produce the same operating point — otherwise the pipeline’s behavior is not reproducible.

Demonstration: calibrating a RELATE duplicate filter

MEASURED on RELATE v0.1, Wave 1 rows 1.10 and 1.11 — artifacts experiments/embeddings-from-first-principles/wave1/artifacts/calibration.json and threshold-drift.json. Model: all-mpnet-base-v2.

Positive = equivalent + paraphrase pairs (same claim). Negative = negation + contradiction + temporal-mismatch + relation-swap pairs (about the same thing, opposite or altered claim).

positive cosine:   mean 0.87       (350 pairs)
negative cosine:   mean 0.76       (343 pairs)
AUC:               0.75
equal-error rate:  24%   at threshold 0.843
escalate band:     between the 5%-FRR and 5%-FAR thresholds — 86% of all pairs

And the threshold does not hold still across domains (row 1.11, same positive/negative definition, equal-error threshold per domain):

everyday-statements   0.763       corporate-events   0.813
product-support       0.826       geo-civics         0.858
biomed-claims         0.859       → spread 0.10

MEASURED: with genuinely hard negatives the same/different-claim decision from raw cosine is a weak classifier (AUC 0.75), no single threshold gives both low FAR and low FRR, and 86% of pairs fall in the region where you cannot safely decide either way. The honest design returns accept / reject / escalate. And a threshold calibrated on one domain is off by ~0.10 cosine on another — wider than the whole operating margin from Chapter 11. “Similarity > 0.8” is not a specification.

What this chapter establishes and what it does not

Establishes: a score is meaningless without its positive/negative distributions; FAR, FRR, ROC/AUC, and the ambiguity band; operating points chosen from error costs; thresholds are tied to model version, corpus, query type, and metric and must be re-derived; calibration must be deterministic and versioned.

Does not establish: a universal threshold (there is none), or that AUC is a sufficient summary (it hides where the errors fall). It establishes that “similarity > 0.8” is not a specification until the distributions behind it are recorded.

Lab 14: calibrate an operating point

PROPOSED, not executed.

Setup. 200+ labeled positive pairs, 200+ hard negative pairs, one model version, one query type.

Task.

1. Compute similarity for all pairs. Plot overlaid histograms.
2. Compute the ROC curve and AUC.
3. Derive t for target FAR = 1% and separately for target FRR = 5%. Record the other error rate at each.
4. Define the ambiguity band; report the fraction of pairs inside it.
5. Re-run on a second corpus / query type. Did t move?

Success criterion. An operating point with stated FAR/FRR, an ambiguity band with an escalation route, and evidence of how far t moves across corpora — the number you must budget for re-calibration.

Companion component: the calibration record

calibration_record:
  space_version:     <model + version>
  corpus_snapshot:   <id, date>
  query_type:        <...>
  metric:            <...>
  positive_set:      {source, n, mean, sd}
  negative_set:      {descriptor from Ch11, mean, sd}
  auc:               float
  operating_point:   {t_low, t_high, target, FAR, FRR}
  ambiguity_fraction: float
  escalation:        <reranker | cross-encoder | second-signal | human>
  seed / code_hash:  <for determinism>

The Observatory attaches a calibration record to every threshold and warns when the bound (space_version, corpus_snapshot, query_type, metric) no longer matches the live system.

Failure modes

• A bare threshold. No distributions, no FAR/FRR, no provenance — an uninterpretable magic number.
• Calibrating on easy negatives. Flatteringly wide gap; production negatives are harder and the threshold under-rejects.
• Reusing a threshold after a model bump. The distribution moved; the threshold did not.
• Forcing binary where the band is wide. Silently choosing which error to make.
• Non-deterministic calibration. The operating point changes run to run; behavior is unreproducible.

What this chapter established

• A similarity score is a point on two overlapping distributions; without them it means nothing.
• FAR, FRR, ROC/AUC, and the ambiguity band; operating points chosen from the cost of each error, not intuition.
• Thresholds are tied to model version, corpus, query type, and metric — re-derive on change.
• Calibration must be deterministic and versioned.
• With hard negatives, no single threshold is both low-FAR and low-FRR; the honest output is accept / reject / escalate.
• The calibration record, bound to the exact conditions it holds for.

Next

Calibration squeezed one scalar as hard as it can go and still left a wide ambiguity band. The next chapter asks whether one scalar was ever the right object — and pulls several independent geometric signals out of the same pair.