Similarity Is Not Discovery

The sidecar of Chapter 7 has an embarrassing regular guest. A researcher reading a paper on retrieval evaluation opens the side panel and finds, ranked first, a paper with nearly identical vocabulary — same benchmarks named, same metrics discussed, same dataset family. She opens it. It contributes nothing: same conclusions, weaker experiments, no new evidence, no disagreement, no extension. Topically it is the closest thing in the corpus to what she is reading. Informationally it is empty. The sidecar did its retrieval job perfectly and its discovery job not at all.

That failure — similar but useless — is this chapter’s starting point, and the chapter refuses to blur what fixes it. The fix is not better similarity. It is a different question. Similarity asks how close a candidate is. Discovery asks how a candidate relates: does it support, contradict, extend, update, duplicate, or implement something the reader cares about? Those are different inquiries, they need different machinery, and this chapter keeps them in separate layers even when one system performs several.

The argument climbs four steps, and each step is labelled with exactly what it licenses — because the inheritance this chapter draws on is unusually honest about its own limits, and the chapter treats that honesty as load-bearing rather than decorative.

Step 1: the failure similarity cannot see

Return to the embarrassing guest. Cosine similarity over embeddings ranked the duplicate first because the duplicate is close — shared vocabulary, shared entities, shared topic. Closeness was measured correctly. The problem is that closeness was the wrong question: the researcher needed to know the candidate duplicates rather than extends, and no distance score carries that information. A contradiction and a paraphrase can sit at identical cosine distances from a claim; a temporal update (“this result no longer holds under the new protocol”) can be nearer than the paper that actually matters.

The author’s Embeddings From First Principles work established this cleanly as a retrieval result, and the RELATE corpus preserves the failure classes as frozen, inspectable machinery. The ontology (frozen 2026-09-08, spec RELATE-v0.1) defines eleven relations with decision procedures and an adjudication order — equivalent, negation, relation-swap, contradiction, temporal-mismatch, entailment, partial-support, paraphrase, topic-related, entity-related, unrelated — and a hard-negative role for candidates highly similar under lexical, topical, or entity signals that are not correct answers (underlying relations: negation, relation-swap, temporal-mismatch, topic-related, entity-related, partial-support). The 937-case hard-negative benchmark exists precisely to ask: when two items are deceptively close, does the method preserve the distinction that matters? Two Embeddings-book distinctions travel with it into this book: similarity is not equivalence, and retrieval is not verification. Both are now load-bearing here. The sidecar retrieves; nothing retrieved is thereby verified, and nothing close is thereby useful.

Step 2: what RELATE actually proved — and refused to claim

With the failure named, the chapter inherits its machinery from RELATE (github.com/ernanhughes/relate), inspected directly for this draft including its scientific-state document, runtime README, hard-negative benchmark, and frozen ontology. The core is small: RelationProjection learns a ridge projection from frozen embeddings into measurable relation coordinates and ranks targets in that relation space — relation-specific readout rather than default geometry — with cosine optionally retained as a cheap candidate generator feeding a relation-defined final order.

The bounded evidence, cited at its original scope:

In repository-separated real Python code, independently predicted AST primitive coordinates exposed a frozen three-way structural relation materially better than raw CodeBERT cosine or Euclidean geometry on the preregistered hard-negative test.

The setting, exactly as preserved: frozen microsoft/codebert-base (768 dimensions), CodeSearchNet Python, 20,000 train / 4,000 validation / 4,000 test rows; external AST-derived coordinates (cyclomatic complexity, maximum control nesting depth, distinct call-site count); Chebyshev distance over robust-scaled coordinates. Results: predicted-coordinate executor 0.733 hard-negative ordering accuracy versus 0.532 raw cosine and 0.533 raw Euclidean, against a preregistered 0.10 continuation threshold — decision REAL_PREMISE_SUPPORTED, independently recomputed by a separate implementation. The benchmark directory preserves these as external evidence with provenance and reproduces the observation structure (readout far above raw geometry) through a deterministic synthetic mirror.

And now the part that makes this inheritance trustworthy rather than convenient — what RELATE’s own scientific-state document says was not established: no superiority over a directly trained compound model; no superiority over supervised metric learning; no general relation-composition algorithm; no semantic understanding; no production-quality retrieval; no transfer across models, languages, or domains; no calibrated support or refusal. The follow-up experiment meant to answer the supervised comparison (RELATE-E01) failed its shuffled control and was declared invalid rather than reinterpreted — development orderings from that attempt are explicitly diagnostic, not results. The repository’s interpretation rule forbids silently broadening the result, treating development performance as confirmation, or letting infrastructure substitute for the unanswered comparison.

This book obeys that rule. RELATE gives us, legitimately: the retrieval-failure framing with typed failure classes, and the narrow but real demonstration that a frozen embedding can contain an externally measurable relation that its default cosine/Euclidean geometry materially underexposes. Everything beyond that — arbitrary papers, SUPPORTS/CONTRADICTS judgements, discovery policy — is the Language book’s new problem, tested here rather than assumed.

Step 3: the Language hypothesis and its four layers

The hypothesis, stated as a hypothesis:

Perhaps discovery should ask how a candidate relates to the current information — separating candidate closeness from the relationship a candidate bears to it.

The machinery separates into four layers, and RELATE belongs principally at exactly one of them:

CANDIDATE GENERATION
"What might be related?"
broad retrieval: keyword, embedding, topical rankers

        ↓

RELATION READOUT
"How is this related?"
relation-specific measurement over candidates

        ↓

DISCOVERY POLICY
"Does that relationship make it useful?"
relation + novelty + task relevance → worth surfacing

        ↓

ATTENTION POLICY
"Should the human see it now?"
timing, intrusiveness, budget — Chapter 19's problem

RELATE lives at relation readout. It does not decide the last two questions — whether a contradiction deserves surfacing given the reader’s task and knowledge, or when it should interrupt. Those decisions need task context (Chapter 14), novelty estimates (Chapter 15), and attention policy (Chapter 19), none of which a relation score supplies. Keeping the layers separate is what lets each improve independently: better candidates without better readout just retrieve more duplicates faster; better readout without discovery policy surfaces well-typed irrelevancies; either without attention policy interrupts dinner with a contradiction. The separation serves the rest of the book, not just this chapter.

Step 4: the discovery relation vocabulary — proposed, not proven

The chapter now needs relation words for research information, and it introduces them with a fence around them. The RELATE ontology’s labels (negation, relation-swap, temporal-mismatch, topic/entity-related) demonstrate that relation type matters for retrieval evaluation. The runtime’s coordinates (complexity, nesting depth, call-site count) demonstrate that relation-specific readout is implementable. Neither demonstrates that the runtime can take arbitrary papers and correctly emit the judgements discovery needs. So the chapter names the latter explicitly as new construction:

ENCODED INFORMATION     what a frozen representation contains
      ↓ (no automatic licence)
DECODABLE INFORMATION   what a supervised probe can recover
      ↓ (no automatic licence)
RELATION CLASSIFIER     what typed readout achieves on held-out cases
      ↓ (no automatic licence)
RETRIEVAL EXPLOITATION  what ranking actually uses
      ↓ (no automatic licence)
RANKING IMPROVEMENT     measured gain on frozen candidates
      ↓ (no automatic licence)
DOWNSTREAM TASK GAIN    better decisions, finds, or completions

Evidence at one rung licenses nothing about the next. RELATE’s retained result lives on the first rungs (information present, recoverable under supervised readout against raw-geometry baselines); every rung above is this book’s untested hypothesis. EXP-08 must therefore beat serious alternatives, not strawmen: where technically applicable the later run adds a direct supervised pair classifier, a strong cross-encoder or reranker, a metric-learning or supervised retrieval baseline, an ontology/schema ablation (raw relevance vs generic categories vs book ontology vs oracle labels), and an oracle-relation condition. The rationale the chapter states plainly: beating cosine would establish only that supervised labels beat unsupervised geometry — the experiment must determine whether typed relational structure adds value beyond ordinary supervised relevance modelling, and how much of any gain comes from the human-designed ontology itself (a positive ontology result earns structured ontology engineering improves discovery, not embeddings understand relations). Relationship classification further permits UNKNOWN / INSUFFICIENT EVIDENCE, scored separately from forced assignment — uncertainty the profile of Chapter 9 can then audit rather than inherit as false confidence.

DISCOVERY RELATION VOCABULARY (proposed application)
SUPPORTS · CONTRADICTS · EXTENDS · UPDATES
DUPLICATES · IMPLEMENTS · QUALIFIES · ADJACENT-BUT-VALUABLE
TOPICALLY-RELATED-BUT-USELESS · UNRELATED

This vocabulary is a proposed application built from the RELATE principle — measure the relation, don’t trust the distance — not an already validated RELATE capability. Every use in this book carries that status until EXP-08 reports. The chapter would rather understate a working method than overstate an unbuilt one, because the very next chapter exists to audit exactly this kind of claim.

The personal-novelty hook arrives here as a pointer, not a mechanism: readout says DUPLICATES; whether duplication matters depends on what the reader already knows — Chapter 15’s ledger. Relation vectors replace the single relevance scalar from this chapter forward: each candidate carries its typed relationships as structured evidence, and policy layers decide. A scalar says “0.87 relevant” and hides everything; a vector says “contradicts claim 3, duplicates claim 7, topically related to the rest” and lets policy reason.

The experiment: same pool, typed adjudication, differential wins

EXP-08 gives every method the same retrieved candidate pool for a frozen active project (its claims, questions, and concept set) and compares four rankings: A keyword, B embedding cosine, C generic LLM relevance (“is this relevant?”), D relationship-aware (retrieve → classify/read out relation → rank using relation plus novelty/usefulness). Human adjudication uses the discovery vocabulary above — never a bare relevant/irrelevant call — and the scoring looks for differential outcomes, not one leaderboard number: useful discoveries, duplicates surfaced, contradictions found, implementation opportunities, irrelevant items, genuinely novel items, and useful items found only by one method. The interesting result may be that cosine finds more broadly related material while relation-aware discovery disproportionately finds contradictions and extensions — a division of labour, not a coronation.

One design requirement is non-negotiable, inherited from RELATE’s unanswered comparison: no straw baselines. If condition D is essentially an LLM multi-class relation judge, it faces a fair opponent — a generic relevance judge plus an appropriate trained or structured relation baseline — so the chapter cannot merely prove that asking a richer question produces a richer label. The supervised-comparison gap RELATE left open must not be re-imported here as a weak control.

The handoff: who checks the contradiction?

Suppose the relation system reports Paper B CONTRADICTS current claim 3. Before any policy acts on that enrichment, a question the chapter cannot answer: did the transformation preserve the actual contradiction — its scope, its qualifier, its evidence — or did the machinery manufacture a stronger relationship than the sources support? A negation dropped, a subgroup qualifier lost, a temporal index stripped, and a partial-support becomes a contradiction; the sidecar then amplifies an artefact. The failure runs in the other direction too: explicit negation in the text is not sufficient evidence of contradiction — scope, subject, time, and qualification still determine the relation, and a classifier that reads surface polarity as contradiction manufactures conflicts where the sources state constraints. Chapter 9 exists for exactly this audit, and the relation vocabulary gives it precise objects to check.

Transforming and augmenting information creates an obligation to measure what survived.

References

• RELATE repository (github.com/ernanhughes/relate): runtime (RelationProjection ridge projection, cosine candidate generation + relation rerank), frozen ontology (11 relations + hard-negative role, frozen 2026-09-08), 937-case hard-negative benchmark, Observatory layers (identity/compatibility/usability). Inspected directly for this chapter.
• RELATE docs/scientific-state.md: Option B result (CodeBERT/CodeSearchNet, 20k/4k/4k, 0.733 vs 0.532/0.533, threshold 0.10, independently recomputed) with explicit non-establishments; RELATE-E01 SHUFFLED_CONTROL_FAIL/EXPERIMENT_INVALID; interpretation rule. Used for the honest-inheritance structure.
• Embeddings From First Principles: similarity ≠ equivalence; retrieval ≠ verification; preservation-profile principle (“every transformation creates an obligation to measure what survived”). Conceptual inheritance; specific chapter verification deferred to Part I coherence pass.
• SOLAR (Findings EMNLP 2025, DOI 10.18653/v1/2025.findings-emnlp.538) and PersonaX (Findings ACL 2025): cited as leads for serendipity-aware and multi-interest recommendation; full reads pending, no claims taken.

Proposed experiment EXP-08: discovery paradigms on a frozen project

Status: PROPOSED. Setup: frozen project concept/claim/question set + frozen candidate research stream; identical candidate pool for all methods. Conditions: A keyword ranking; B embedding nearest-neighbour; C generic LLM relevance judge (“is this relevant?”); D relationship-aware (retrieve → relation classify/readout → rank on relation + novelty/usefulness), with D’s classifier faced by a fair supervised/structured relation baseline, not only the generic judge. Blind human adjudication per candidate in the discovery vocabulary (duplicate / supports / contradicts / extends / updates / implementation / qualifies / adjacent-but-valuable / topically-related-but-useless / unrelated). Measures: useful discoveries, duplicates surfaced, contradictions found, implementations found, irrelevant items, novel items, items found only by one method — reported per class, not collapsed. Failure criteria: D ties B/C across classes (relation readout adds nothing); D wins overall recall via more topically-related-but-useless items (richer labels, no discovery gain); generic judge ties multi-class judge (vocabulary is decoration). Artifacts expected: project pack, candidate pool freeze, adjudication keys, per-class tables. What a positive result would not justify: truth, value, or interruption priority — readout is evidence for policy (Ch 14/15/19), not policy itself; and relation preservation under transformation belongs to EXP-09.