Part VI — Crossing Embedding Spaces

The question, stated carefully

We have two encoders, E_A and E_B, and a set of objects x. Each object has two representations: E_A(x) in space A, E_B(x) in space B. Chapter 16 established these live in unrelated coordinate systems.

The question:

Does there exist a map T such that T(E_A(x)) ≈ E_B(x) for objects x the map was never trained on — and by what standard do we judge “≈”?

If yes, we can take a vector produced by model A and place it, usably, in model B’s space — without re-running model B. That is what would make legacy vectors searchable after an upgrade (Chapter 17), or let two systems with different encoders share a memory.

Why a simple map might work at all

Both models were trained on overlapping data to capture overlapping notions of semantic structure. Their spaces are different bases over related underlying structure. If the relationship between that structure and each model’s coordinates is roughly linear — a big “if,” tested here — then a single linear map could carry one to the other.

There is prior evidence in the neighborhood. Cross-lingual word embeddings can be aligned with a linear map fit on a bilingual dictionary (Mikolov, Le & Sutskever, 2013), and constraining that map to be orthogonal — a pure rotation — generalizes better and has a closed-form solution (Smith et al., 2017). Independently trained models often differ by close to a linear transform (Chapter 16’s CKA is “similarity up to linear map”). None of this guarantees it works for your two models on your corpus. It motivates trying the simplest thing first.

The simplest map: least-squares linear projection

Collect anchor pairs: objects embedded in both spaces.

X_A  =  [ E_A(x₁); E_A(x₂); ... ; E_A(xₙ) ]     (n × d_A)
X_B  =  [ E_B(x₁); E_B(x₂); ... ; E_B(xₙ) ]     (n × d_B)

Solve for the matrix W (d_A × d_B) minimizing ‖X_A W − X_B‖²:

W = (X_Aᵀ X_A + λI)⁻¹ X_Aᵀ X_B          # ridge-regularized least squares

Then T(v) = v W. This handles dimension mismatch for free (W is d_A × d_B). Add a bias term, or center both spaces first.

The test that matters: held-out anchors

Fitting W on anchors and evaluating on the same anchors measures memorization. The real question is generalization:

1. Split anchors into train / test.
2. Fit W on train.
3. For each test object, compute T(E_A(x)) and ask: where does it land in space B?
   • Is E_B(x) its nearest neighbor? (top-1 hit)
   • Is E_B(x) in its top-10?
   • What is cos(T(E_A(x)), E_B(x))?
4. And the downstream question: if you retrieve in space B using T(E_A(query)) as the query vector, do you get the same results as using E_B(query)?

    flowchart TD
    A["anchor pairs: objects embedded in BOTH spaces"] --> SP["split anchors train / test"]
    SP --> FIT["fit W on train — minimize ‖X_A W − X_B‖² (ridge)"]
    FIT --> EV["evaluate on HELD-OUT test objects only"]
    EV --> M1["coordinate reconstruction — cos(T(E_A x), E_B x)"]
    EV --> M2["neighbours stay neighbours — 10-NN overlap vs native B"]
    EV --> M3["rankings stay rankings — retrieval agreement@10"]
    EV --> M4["fine distinctions survive? — hard-negative margin ratio"]
    M1 --> STD["define '≈' from downstream need BEFORE fitting: neighbours / rankings / clusters / thresholds"]
    M2 --> STD
    M3 --> STD
    M4 --> STD
  

Demonstration: linear translation on RELATE

MEASURED on RELATE v0.1, Wave 3 row 3.4 — artifact experiments/embeddings-from-first-principles/wave3/artifacts/ladder-8property-matrix.json. Ridge (linear) bridge, all-MiniLM-L6-v2 (384-d) → all-mpnet-base-v2 (768-d), fitted on ≤712 split_entity:train anchors, evaluated on held-out split_entity:test entities.

property (on unseen test entities)              linear ridge bridge
coordinate reconstruction cos(T(x), y)               0.59
10-NN neighborhood overlap vs native B               0.71
retrieval nDCG@10 ratio vs native B                  ~0.9
relation-profile correlation (relation ordering)     0.87
calibration threshold transfer                       0.79
hard-negative margin ratio vs native B               0.23   ← the fine distinction does not survive

MEASURED: a plain linear map recovers most coarse structure on unseen entities — 71% neighborhood overlap, retrieval within ~10% of native, and the ordering of relation types by similarity is 87% correlated. It is much weaker on exact reconstruction (cosine 0.59) and it keeps only 23% of the hard-negative margin. The split the chapter predicted — “roughly where things go” survives, “fine distinctions” do not — is real and measured. Chapter 21 runs the full ladder and shows a nonlinear map does not close that gap.

What “≈” should mean

Exact coordinate reconstruction (T(E_A(x)) = E_B(x)) is the wrong target. What downstream systems need is usually one of:

neighbors stay neighbors        (retrieval, dedup, clustering)
rankings stay rankings          (search result order)
clusters stay clusters          (topic organization)
thresholds stay meaningful      (calibrated decisions)

A map can succeed at some of these and fail at others. Chapter 21 turns each into a metric. Here, the point is: define the success standard before fitting the map, from what the vectors are for.

Translating without anchors — and the universal-geometry conjecture

Anchor pairs are the easy case: you paid model B to embed the same texts model A already embedded. What if you cannot? You have a database of vectors from an unknown or retired encoder and no way to re-embed the source text.

It is still sometimes possible. The cross-lingual community solved a version of this: start from a rough guess of the mapping (adversarial training, or a small automatically-mined seed), take the pairs that are mutual nearest neighbors under the current guess as pseudo-anchors, fit Procrustes to those, and iterate (Conneau et al., 2018; Artetxe et al., 2018). Recent work carries this to sentence encoders: vec2vec (Jha, Zhang, Shmatikov & Morris, 2025) learns a translator between two embedding spaces with no paired data at all, using a shared latent backbone trained with adversarial, cycle-consistency, and pairwise-distance-preservation losses; mini-vec2vec (Dar, 2025) recovers most of the same result with a plain linear map plus iterative refinement, far more cheaply and stably.

These results motivate a strong conjecture. vec2vec states the Strong Platonic Representation Hypothesis (extending Huh et al., 2024):

neural networks trained with the same objective and modality, with different data and model architectures, converge to a universal latent space such that a translation between their respective representations can be learned without any pairwise correspondence.

Two things to hold at once:

• The results are real and consequential. vec2vec reaches cosine 0.74–0.92 to the true target across model pairs, and near-perfect top-1 matching on thousands of shuffled embeddings for pairs where the naïve “do nothing” map scores zero. Unpaired cross-encoder translation works.
• Every published number is a coarse metric — mean cosine to target, top-1 retrieval accuracy, mean rank, topic inference. Note the scope in the hypothesis itself: “the same objective and modality.” vec2vec’s own cross-modal results show cosine partly surviving while top-1 collapses. And no one has measured whether translation keeps negation distinct from paraphrase, keeps A acquired B distinct from B acquired A, keeps entailment direction, or keeps a calibrated threshold’s false-accept rate.

So the question for Part VI is not “is embedding geometry universal — yes or no.” It is: when two spaces are aligned well on coarse metrics, which finer properties came along, and which did not? Chapters 19–21 build the tools to answer that, and Chapter 21 runs the attack on RELATE’s typed relations.

A successful global alignment is evidence about coarse geometry. It is not, by itself, evidence that fine relational structure or operating points transferred — those are separate measurements.

What this chapter establishes and what it does not

Establishes: the translation question, precisely stated; why a linear map is a reasonable first attempt; the least-squares construction with dimension mismatch handled; the held-out-anchor evaluation; that unpaired translation is possible (pseudo-anchors + iteration; vec2vec; mini-vec2vec) and motivates a universal-geometry conjecture; that “≈” should be defined by downstream need, not coordinate identity.

Does not establish: that linear maps are sufficient (Chapter 19 tries more); that translation preserves what you need (Chapter 21 measures); that embedding geometry is universal (the conjecture is scoped to “same objective and modality” and tested only on coarse metrics); or that any two models can be bridged (some cannot). It establishes the method and the honest evaluation protocol.

Lab 18: fit a linear bridge

PROPOSED, not executed.

Setup. Two models. 1,000+ objects embedded in both. Train/test split of anchors (e.g. 80/20). Labeled queries for downstream test.

Task.

1. Fit ridge least-squares W on train anchors. Sweep λ.
2. On held-out anchors: mean cos(T(E_A), E_B), top-1 hit rate, top-10 hit rate.
3. Downstream: retrieve in space B with T(E_A(query)); measure agreement@10 vs native E_B(query), overall and on hard negatives.
4. Vary anchor count n ∈ {100, 500, 2000}; plot held-out top-10 hit rate vs n.

Success criterion. A held-out generalization number and the anchor count at which it plateaus — plus a statement of which downstream property (neighbors / rankings / clusters) you are willing to say the linear bridge preserves.

Companion component: the bridge — first fields

bridge (v0):
  source_space_hash:   <from Ch17>
  target_space_hash:   <from Ch17>
  method:              linear_least_squares
  lambda:              float
  anchor_set:          {n_train, n_test, selection: "how anchors were chosen"}
  heldout_metrics:     {cos, top1, top10, retrieval_agreement, hard_neg_agreement}
  status:              <fitted | evaluated | rejected>

The Observatory stores a bridge keyed by the exact (source_space_hash, target_space_hash) pair. A bridge fitted for one pair is not valid for any other pair, including a different version of either model.

Failure modes

• Evaluating on training anchors. Measures memorization; always report held-out.
• Anchors that don’t cover the space. A bridge fitted on news anchors will not translate legal vectors.
• Ignoring dimension mismatch handling. W is d_A × d_B; don’t force equal dimensions with padding.
• Claiming the bridge “works” from cosine alone. Cosine to the target is not the same as preserving retrieval or clustering (Chapter 21).
• Reusing a bridge across model versions. New space_hash on either side = new bridge required.

What this chapter established

• The translation question: does T(E_A(x)) ≈ E_B(x) hold on unseen x, and by what standard?
• A linear least-squares map, ridge-regularized, handling dimension mismatch directly.
• The held-out-anchor protocol and the downstream retrieval-agreement test.
• Translation without anchors is possible — pseudo-anchor iteration (cross-lingual), vec2vec (unpaired, nonlinear), mini-vec2vec (unpaired, linear) — and motivates the Strong Platonic Representation Hypothesis, scoped to “same objective and modality” and demonstrated on coarse metrics only.
• A successful global alignment is evidence about coarse geometry, not about fine relational structure or operating points — those are separate measurements (Chapter 21).
• “≈” is defined by downstream need — neighbors, rankings, clusters, thresholds — not coordinate identity.
• The bridge object v0, keyed to the exact source/target space_hash pair.

Next

The linear map was the simplest thing. The next chapter widens the toolkit — orthogonal Procrustes, CCA, learned nonlinear maps — and sharpens the distinction the linear demo already exposed: coordinate reconstruction versus semantic preservation.