What Does a Preference Know About the Future?

We Trained a Model on Editorial Choices to See Whether It Learned What Happened Next

Most preference-learning systems use a choice to change the future.

A model produces two responses. A human selects one. The chosen response becomes positive evidence, the rejected response becomes negative evidence, and training makes outputs resembling the chosen response more likely.

The preference acts as an instruction:

Produce more things like this.

I wanted to know whether the same choice could also function as evidence.

When an editor replaces one sentence with another, the decision may reflect a factual uncertainty, a structural problem, an institutional constraint or an expectation about how the article will develop. The preference record preserves only the compressed result:

A > B

Preference learning normally reads that record as:

Make A more likely than B.

This experiment asked a stronger question:

After learning that the editor selected A over B, does the model become better at predicting what happens to A next?

The final answer was no.

That answer took considerably more work than the first negative result suggested. We tested conventional encoder fine-tuning, a compact antisymmetric value model, dimension-matched controls, stronger regularisation, shuffled-label controls and finally a shallow XGBoost model over all combined generic and preference-derived features.

Across 12,056 sentence-revision episodes from 3,386 article lineages, authentic editorial preference was weakly learnable. A compact MR.Q model could identify the selected candidate slightly better than chance:

accuracy: 52.33%
95% CI:   [51.44%, 53.22%]
log loss: 0.691868

But the information it learned did not improve prediction of the selected sentence’s later revision beyond properly controlled generic semantic representations.

The final nonlinear test was almost perfectly null:

generic XGBoost log loss:                    0.549565
generic + authentic MR.Q log loss:           0.549557
difference:                                 -0.000007
95% lineage-bootstrap CI:       [-0.001081, +0.001091]

Authentic MR.Q augmentation was also indistinguishable from adding identically shaped MR.Q features trained on shuffled preference labels:

authentic − mean shuffled log loss:          +0.000045
95% lineage-bootstrap CI:       [-0.000942, +0.001011]
authentic favourable against shuffles:       2/5

So the final result is more precise than “preferences contain no information.”

The preference itself was weakly predictable. The text and revision context contained useful information about future revision. MR.Q produced a compact representation that preserved the weak source-task signal. But authentic preference supervision did not add future-predictive information beyond the semantics already present in the text.

The experiment therefore separated two claims that are often treated as one:

a choice can be learned
≠
learning the choice improves prediction of what happens next

Preference as Instruction and Preference as Evidence

Conventional preference learning estimates something like:

where:

• \(H\) is the current history or state;
• \(C\) is the set of available candidates;
• \(Y\) is the observed preference.

The preference is the target.

Preference-conditioned forecasting asks a different question:

where \(F\) is a specified later outcome.

Here the preference is no longer the target.

It becomes evidence.

The distinction matters because predicting what a person chooses is not the same as predicting what follows from that choice.

A model can become very good at identifying the preferred candidate by exploiting regularities such as:

• sentence length;
• lexical style;
• newer dates;
• stronger attribution;
• fewer hedges;
• resolved facts;
• common editorial formulations.

Those features may distinguish selected from rejected sentences without representing the hidden condition that caused the edit or shaped the later trajectory.

From:

we cannot automatically recover:

The two tasks may share information, but one is not the inverse of the other.

The experiment tested whether the representation learned for the first task transferred to the second.

The complete implementation—including the frozen experimental contracts, data-processing pipeline, training scripts, controls, verification procedures and result aggregation—is available in the preference-futures GitHub repository.

The First Experiment Found Future Information in the Rejected Sentence

This was not my first attempt to connect preferences to later outcomes.

The earlier experiment used three successive states from the same NewsEdits sentence lineage:

V0: earlier sentence
        ↓ replaced by
V1: selected replacement
        ↓ survives or changes
V2: next observed state

The preference was already known: the editor had removed V0 and retained V1.

The question was whether the removed sentence contained information about the later fate of the selected replacement.

A baseline model estimated:

The preference-informed model estimated:

where \(R\) represented measured relationships between the selected and rejected sentences.

That experiment found a modest but consistent forecasting improvement from authentic rejected text.

The removed sentence contained information about what happened to the replacement later.

But the future predictor had been trained directly on future outcomes. It was shown many examples in which the later fate of the selected sentence was known.

It learned:

current context
+
selected sentence
+
rejected sentence
+
future labels
→
future forecast

That demonstrated that preference evidence could contain information about a decision-linked future.

It did not show that a preference-learning objective would discover or preserve that information.

The information could exist in the record while remaining invisible to the objective.

That became the stronger test.

The Stronger Test

The new experiment trained a model only on the source decision.

It saw:

article context
candidate A
candidate B

The authentic-preference version was trained to identify which candidate the editor had retained.

It was never shown whether the selected sentence survived the next revision.

After source training, I removed the source-task classifier and froze the encoder.

I then extracted one fixed representation for every episode and trained identical linear probes to predict:

Will the selected sentence change in the next observed state?

Let:

be the representation produced by the untouched language model, and:

be the representation produced after authentic-preference training.

The primary hypothesis was:

where \(g\) was the same linear future probe and \(\mathcal{L}\) was the held-out probabilistic forecast loss.

If the inequality held, preference training would have made future-relevant information more accessible.

It did not.

Why Generic Was Not Enough as a Control

A preference-trained encoder receives more than authentic preference supervision.

It also receives:

• additional optimisation;
• additional exposure to the domain;
• repeated exposure to sentence revisions;
• paired inputs;
• information about temporal changes;
• a binary training objective.

A preference-trained model beating an untouched model would therefore not be enough.

The experiment used seven arms.

Generic

The untouched pretrained encoder.

Language adaptation

The same encoder received additional masked-language-model training on NewsEdits text.

This controlled for domain adaptation.

Pair exposure

The encoder learned whether two sentences formed a genuine revision pair, without learning which candidate the editor retained.

This controlled for learning the structure of revisions.

Temporal direction

The encoder learned which sentence in an independent revision pair was newer.

This tested whether apparent preference transfer was really temporal discrimination.

Random label

The preference pipeline was run with random targets.

This controlled for additional optimisation without meaningful supervision.

Shuffled preference

Authentic-looking preference labels were detached from their actual decision lineages.

This controlled for preference-shaped supervision without the real choice.

Authentic preference

The encoder learned which candidate the editor actually retained.

All six trained encoders used the same architecture, data budget, maximum sequence length, batch size and number of updates.

The seventh arm was the untouched generic encoder.

The future stage then applied the same representation extraction and the same linear-probe procedure to every arm.

    flowchart LR
    subgraph arms["🔬 Seven Source‑Training Arms (same base model, data budget)"]
        direction LR
        A_RAW["🧊 Generic<br/><i>(untouched)</i>"]
        A_LANG["📚 Language Adaptation<br/><i>masked LM on NewsEdits</i>"]
        A_PAIR["🔗 Pair Exposure<br/><i>real pair? yes/no</i>"]
        A_TEMP["⏳ Temporal Direction<br/><i>which is newer?</i>"]
        A_RAND["🎲 Random Label<br/><i>noise supervision</i>"]
        A_SHUFF["🔀 Shuffled Preference<br/><i>labels detached</i>"]
        A_AUTH["🎯 Authentic Preference<br/><i>editor’s actual choice</i>"]:::highlight
    end

    ARROW["⏬ Freeze encoder → extract fixed representation"]:::process
    PROBE["📏 Identical Linear Future Probe<br/><i>Will the retained sentence be revised again?</i>"]:::process
    COMPARE["⚖️ Primary comparison: Authentic Preference vs Generic"]:::decision
    RESULT["❌ No reliable improvement<br/>CI crosses zero, 3/10 folds"]:::fail

    arms --> ARROW --> PROBE --> COMPARE --> RESULT

    classDef highlight fill:#fff2cc,stroke:#b47a00,stroke-width:3px,color:#111
    classDef process fill:#dae8fc,stroke:#3b73b9,stroke-width:2px,color:#111
    classDef decision fill:#e1d5e7,stroke:#73578f,stroke-width:2px,color:#111
    classDef fail fill:#f5b7b7,stroke:#a83232,stroke-width:2px,color:#111
  

The Experiment Was Frozen Before the Result

The confirmatory experiment used:

12,056 episodes
3,386 article lineages
10 article-grouped outer folds
7 representation arms
70 future-probe jobs

All 70 probe jobs passed independent verification.

For each fold and arm:

1. feature standardisation was fitted on the training partition only;
2. five fixed regularisation values were trained;
3. regularisation was selected using validation log loss only;
4. the selected model was evaluated once on the untouched test partition;
5. every prediction was persisted and independently reconstructed.

The primary comparison was fixed in advance:

generic pooled out-of-fold log loss
-
authentic-preference pooled out-of-fold log loss

A positive value would favour authentic preference.

The uncertainty interval used 10,000 paired bootstrap samples at the article-lineage level.

No representation, probe architecture, pooling rule or regularisation grid was changed after inspecting the result.

    flowchart LR
    subgraph legend[" "]
        direction TD
        L_DONE["✅ Established"]:::done
        L_FAIL["❌ Not Supported"]:::fail
        L_UNTESTED["⬜ Untested"]:::untested
    end

    S0["🔬 Stage 0<br/>Measurement<br/><i>The pipeline separates signal from noise</i>"]:::done
    S1["🔍 Stage 1<br/>Existence<br/><i>Rejected text contains future info</i>"]:::done
    S2["🔮 Stage 2<br/>Representation Transfer<br/><i>Preference training → better future rep</i>"]:::fail
    S3["🧠 Stage 3<br/>Prior / Data Efficiency<br/><i>Preference pretraining helps with few labels</i>"]:::untested
    S4["🌐 Stage 4<br/>Cross‑domain Transfer"]:::untested
    S5["🧭 Stage 5<br/>World Model / Counterfactuals"]:::untested

    S0 --> S1 --> S2 --> S3 --> S4 --> S5

    classDef done fill:#b8e6b8,stroke:#2d6a2d,stroke-width:2px,color:#111
    classDef fail fill:#f5b7b7,stroke:#a83232,stroke-width:2px,color:#111
    classDef untested fill:#e0e0e0,stroke:#888,stroke-dasharray: 5 5,color:#444
  

The Primary Hypothesis Was Not Supported

The authentic-preference representation produced:

log loss: 0.540832
ROC AUC:  0.628365

The untouched generic representation produced:

log loss: 0.540113
ROC AUC:  0.630119

The resulting improvement was:

generic loss − authentic-preference loss
= −0.000719

The 95% confidence interval was:

[−0.003425, +0.002020]

Authentic preference beat generic in only three of the ten folds.

The correct conclusion is:

Under this frozen experimental contract, authentic-preference training did not improve linear prediction of the selected sentence’s later revision.

The point estimate was slightly negative, but the interval crossed zero. The result does not establish that authentic-preference training reliably damaged the representation.

It establishes that the predicted advantage was not detected.

The Seven-Arm Result

The complete out-of-fold results were:

Only authentic preference versus generic was the frozen primary inferential comparison.

The other differences are descriptive and should not be promoted into new confirmatory findings after seeing the results.

But the pattern is difficult to ignore.

The future target was not wholly unpredictable. The generic representation achieved a ROC AUC of approximately 0.63.

Additional source training did not uniformly fail either. Pair exposure and language adaptation improved the descriptive metrics, while random, shuffled-preference and temporal-direction training made them worse.

The strongest representation was not trained to predict which candidate won.

It was trained to recognise that the two candidates belonged together.

The Pair-Exposure Result Was Suggestive, but Not Yet Clean

The first seven-arm experiment produced one striking descriptive result.

The representation trained to recognise genuine revision pairs achieved:

log loss: 0.536768
improvement versus generic: +0.003345
ROC AUC: 0.633184

That was better than the untouched generic encoder and better than the authentic-preference encoder.

The initial interpretation was appealing. A real revision pair contains a structured transformation:

what the sentence was
→
what the sentence became

Learning that two sentences participate in the same transformation might encourage a model to represent semantic continuity, edit magnitude, retained information, factual movement and structural repair. Authentic-preference training, by contrast, compresses the relation into a winner:

A or B?

However, a later source-task audit showed that the pair-exposure task itself was nearly perfectly learnable:

accuracy: 99.30%
log loss: 0.033325

The negative examples had been created using cross-lineage donor sentences. That made the task vulnerable to topic, entity and coherence shortcuts. The model may have learned that two sentences belonged to the same article rather than learning the geometry of revision.

The future-prediction result remains an observed descriptive result. What no longer survives is the confident mechanism claim that pair exposure learned revision structure.

A proper test would require hard negatives from the same article, topic and local context, with entity overlap and chronology controlled. Until that benchmark exists, the pair-exposure result should be treated as a lead rather than a finding.

The broader lesson still survives:

A relation between alternatives and the direction of a choice are different supervision signals, and they should not be assumed to teach the same representation.

Preference Prediction May Discard the Information We Care About

The result exposes an important distinction:

information present in a preference record
≠
information learned by a preference objective

The earlier forecasting experiment found incremental future information in the rejected sentence.

The new experiment found no evidence that authentic-preference training made that information more linearly accessible.

Both results can be true.

The decision record can contain useful information while the preference-prediction objective learns a shortcut that ignores it.

To classify the winner, the model does not need to reconstruct the latent editorial state. It only needs features that separate retained from rejected candidates often enough to reduce its training loss.

Those features may be:

• local;
• stylistic;
• publication-specific;
• weakly temporal;
• unrelated to later stability.

Pair exposure may impose less pressure to collapse the relationship into a single preferred direction.

It may preserve more of the transformation because it has to recognise the pair rather than select the winner.

This suggests a broader lesson for preference learning.

A supervision signal can contain rich information without an objective being forced to encode that richness.

The objective decides what part of the signal survives training.

Why the Temporal Control Performed Worst

The temporal-direction arm produced the weakest future representation:

log loss: 0.548960
improvement vs generic: −0.008847
ROC AUC: 0.613058

That matters because one obvious explanation for preference transfer was temporal discrimination.

Perhaps the authentic-preference model would merely learn:

This sentence looks newer.

If newer-looking sentences were also more stable, preference training could appear to predict the future without learning anything about editorial judgement.

But explicit temporal-direction training did not help. It produced the worst descriptive result of the seven arms.

Again, this comparison was not the frozen primary hypothesis test. It should be treated cautiously.

Still, the result does not support the idea that simple newer-versus-older discrimination was the useful transferable structure.

Random Training Was Not Harmless

The random-label and shuffled-preference representations also performed worse than generic:

random label:          −0.005887
shuffled preference:   −0.005931

Additional training was therefore not automatically beneficial.

Meaningless supervision appears to have overwritten some useful structure already present in the generic encoder.

Authentic preference avoided most of that damage, but it did not create an advantage.

Language adaptation and pair exposure moved the encoder in more useful directions. Random and shuffled supervision moved it in less useful ones.

The result is not:

Fine-tuning does nothing.

It is:

What the model is asked to learn changes which parts of its pretrained representation survive.

What the Full Follow-Up Established

The original frozen experiment left several possible escape routes.

Perhaps authentic preference was real but the encoder could not learn it. Perhaps the future probe was too simple. Perhaps the learned representation was useful only after nonlinear combination with the original semantic geometry. Perhaps the first null was caused by label noise.

We tested each of those possibilities.

The labels were not the explanation

A future-label integrity audit examined 125,822 episodes.

Only 192 labels differed under the two normalization procedures:

all episodes:              0.1526%
currently revised cases:   0.6007%

The inconsistency was real and was corrected for future replications, but it was far too small to explain either the source-task failure or the downstream null.

A strong reasoner saw only a weak, order-sensitive preference signal

A blinded 300-example oracle initially achieved:

accuracy: 59.67%
95% CI:   [54.03%, 65.06%]

But after reversing candidate order in a fresh run:

swapped accuracy:       50.00%
order consistency:      57.00%
consistent-set accuracy: 58.48%

The signal was weak and heavily affected by candidate position.

That result changed the architecture of the next experiment. The preference model had to be mathematically antisymmetric: swapping A and B had to reverse the logit exactly.

Conventional encoder fine-tuning could memorize but not generalize

The authentic DistilBERT preference model could perfectly memorize tiny training sets of 256 and 512 examples. On held-out lineages, however, it remained effectively at chance.

Its best validation budget produced:

accuracy: 51.62%
log loss: 0.692733
95% CI:   [48.80%, 54.43%]

This ruled out a broken optimizer or impossible labels. The model had enough capacity to memorize the task but did not discover a generalizable decision rule.

MR.Q recovered a weak authentic source signal

When full encoder fine-tuning memorised small samples but failed to generalise across article lineages, we turned to MR.Q. That is usually where I go when a weak comparison problem is not yielding to conventional fine-tuning. It is one of my preferred models for this kind of task, and I have used it successfully before. Its compact antisymmetric design also gave us a direct way to remove the candidate-order instability exposed by the oracle audit.

The next model therefore imposed the two constraints suggested by those failures: preserve the pretrained semantic geometry rather than rewriting it, and make the comparison mathematically antisymmetric so that swapping A and B reverses the logit exactly.

We froze the semantic encoder and trained only a small comparative value model.

The pooled out-of-fold MR.Q source result was:

episodes: 12,056
accuracy: 52.33%
log loss: 0.691868
95% CI:   [51.44%, 53.22%]
swap-logit error: 0

The source gate passed.

A linear antisymmetric model had similar threshold accuracy but severe overconfidence:

linear accuracy: 52.46%
linear log loss: 0.728408
fold gates passed: 0/10

This was a real architectural finding. Weak editorial preference was learnable when the pretrained geometry was frozen and the decision was represented as a small antisymmetric value comparison.

The first MR.Q transfer result looked positive

Using identical linear future probes, the compact MR.Q state initially beat the corresponding generic representation:

choice-aware MR.Q − generic log loss: -0.009401
95% CI: [-0.015635, -0.003066]

The blind comparison also passed:

blind MR.Q − generic log loss: -0.007408
95% CI: [-0.013178, -0.001707]

Under the frozen Step 8.4 rule, future transfer was supported.

That result did not survive the next control.

The generic baseline had been under-regularised

The generic representations contained 2,304 to 3,072 dimensions. The MR.Q states contained only 129 to 193.

Every generic fold selected the strongest available L2 penalty, and the generic train-to-test loss gap was roughly ten times the MR.Q gap:

MR.Q beat dimension-matched PCA controls, so simple compression was not the whole explanation.

But after extending the generic regularisation grid, the advantage disappeared:

MR.Q choice-aware log loss:              0.556076
extended generic choice-aware log loss:  0.554721
difference:                              +0.001354
95% CI:                     [-0.001838, +0.004455]

The initial positive result had primarily exposed an under-regularised baseline.

Authentic preference did not beat shuffled preference

Five identically shaped MR.Q models were trained on independently shuffled preference labels.

The shuffled source models stayed at chance, confirming that the control worked:

source accuracy: approximately 49%–50%
source log loss: approximately 0.693

Downstream, however, authentic MR.Q was indistinguishable from the shuffled controls:

Authentic preference supervision had not established compact-state transfer specificity.

XGBoost found nonlinear future signal, but not preference signal

The final test combined every generic feature with either authentic MR.Q features or one of the five shuffled-MRQ controls.

The generic representation alone was predictive:

accuracy: 75.25%
log loss: 0.549565
ROC AUC:  0.614048

Adding authentic MR.Q changed almost nothing:

generic + authentic MR.Q log loss: 0.549557
difference versus generic:        -0.000007
95% CI:               [-0.001081, +0.001091]

Authentic augmentation also failed against the shuffled controls:

authentic − mean shuffled:         +0.000045
95% CI:               [-0.000942, +0.001011]
favourable replicas:               2/5

This closed the final nonlinear-interaction route.

The text and revision context contain information about later revision. Authentic editorial preference did not add detectable incremental predictive utility beyond them under either the linear probes or the shallow tree-based nonlinear interaction test.

What the Controls Rule Out — and What They Do Not

Had the investigation stopped after the first frozen experiment, the strongest defensible statement would have been narrow:

Authentic-preference training did not make future-revision information more linearly accessible.

That would not have ruled out an entangled or nonlinear signal.

The later experiments were designed specifically to test that escape route. The final XGBoost model received all 5,376 generic dimensions and either the 322 authentic MR.Q dimensions or an identically shaped shuffled-MRQ control. It could model nonlinear interactions between the semantic geometry and the learned preference state.

The result remained null:

authentic augmentation − generic:       -0.000007
95% CI:                    [-0.001081, +0.001091]

authentic augmentation − mean shuffled: +0.000045
95% CI:                    [-0.000942, +0.001011]

The interval around the authentic-versus-generic comparison is narrow enough to exclude anything resembling the original apparent MR.Q advantage of roughly -0.0094 log loss. Under this dataset and protocol, any average benefit consistent with the final result is on the order of one-thousandth of a log-loss point.

The conclusion is therefore stronger than a linear-probe null:

Authentic preference supervision did not provide detectable incremental future-revision utility under the tested linear and shallow tree-based nonlinear decoders, and it did not outperform identically shaped shuffled-preference features.

It is still not a metaphysical claim that the representation contains literally no information of any kind. No finite collection of probes can establish that.

A geometric analysis such as CKA could show that two representations differ, or that an objective reorganized their geometry. That would be useful for studying representation change. It would not by itself demonstrate that the changed geometry contains usable information about the future target. This investigation was ultimately about predictive utility, not merely representational difference.

Preference Learning May Have Learned Something Else

The future target in this experiment was specific:

Will the selected sentence change in the next observed state?

A preference-trained representation might encode other properties:

• institutional or publication style;
• formality and hedging;
• attribution conventions;
• local notions of acceptability;
• editorial voice;
• patterns associated with selection but unrelated to later survival.

Those possibilities were not tested.

That limitation should not be converted into either a success or a failure. Auxiliary probes for those properties would constitute a new study with new hypotheses. They could show that preference learning transfers editorial alignment structure without transferring revision-stability information.

The result here is target-specific:

The authentic preference signal that MR.Q learned did not provide incremental utility for predicting later revision.

There Is Utility Here — Just Not the Utility I Expected

The original hoped-for utility was a transferable preference representation:

learn what a person chooses
→
obtain a better prior over what happens next

That mechanism was not supported.

But the experiment still suggests several useful principles for the design of AI systems that learn from human decisions.

Preserve decision episodes, not only preference labels

The strongest systems lesson is that:

A > B

is an aggressively lossy summary.

A robust learning system should preserve the complete decision episode:

context before the decision
candidate A
candidate B
selected candidate
rejected candidate
relationship between the candidates
reason or evidence available at the time
later outcomes
lineage identifiers

The choice, the relationship between the alternatives and the eventual consequence should remain distinct fields. They should not be collapsed prematurely into a single preference label or universal embedding.

The durable asset is not merely a collection of wins and losses. It is a historical record of decisions, alternatives, contexts and outcomes.

That richer record can support several different learning problems without assuming that one objective captures all of them.

Use preference models for the task they demonstrably learn

The compact pairwise value model learned the authentic preference task weakly but reproducibly. Conventional encoder fine-tuning did not.

That makes this kind of model useful for:

• ranking alternatives;
• pairwise quality comparison;
• prioritising cases for human review;
• learning from accepted and rejected proposals;
• calibrated local selection.

It does not follow that the model’s hidden state should be treated as a general-purpose representation of the underlying decision process.

A model that can predict which alternative will be selected has demonstrated competence at ranking. It has not necessarily learned the causes of the decision, the structure of the transformation or the consequences that follow.

AI systems should let each model perform the task it has actually demonstrated it can perform.

Train outcome prediction directly on outcomes

Later revision was predictable from general semantic features, but the learned preference state added no reliable information.

The general lesson is that forecasting systems should be trained directly on the outcomes they are expected to predict. An outcome model might use:

• generic semantic representations;
• surrounding context;
• the magnitude and type of change;
• metadata;
• historical trajectory;
• explicit observed outcomes.

A preference model and an outcome model may consume the same decision record, but they should remain different systems with different targets.

Shared input does not imply that the same learned representation is optimal for every task.

Make pairwise models antisymmetric by construction

The candidate-reversal audit exposed a major source of false confidence.

A model can appear to understand a preference while responding partly to whether an answer appears first or second. For a genuine A-versus-B judgement, the scorer should ideally satisfy:

score(A, B) = -score(B, A)

When that constraint cannot be imposed directly, the system should at least be tested under candidate reversal. A judgement that changes when the candidates are swapped should trigger lower confidence, abstention or human review.

This applies broadly to response ranking, reward modelling, recommendation, retrieval reranking and comparative evaluation.

Treat calibration as part of model quality

One source model achieved slightly higher classification accuracy than the compact value model but much worse log loss.

This illustrates why AI systems should not be selected on win rate or accuracy alone.

A useful evaluation should include:

• log loss;
• calibration;
• Brier score;
• prediction balance;
• candidate-reversal error;
• grouped confidence intervals;
• performance across independent data lineages.

A model that is correct 52% of the time and accurately represents its uncertainty can be more useful than one that is correct at the same rate but makes highly confident errors.

Use hard negatives, not convenient negatives

A relationship-learning task can become artificially easy when negative examples differ from positive examples in obvious ways.

For example, unrelated pairs drawn from different documents may differ in:

• topic;
• named entities;
• style;
• chronology;
• vocabulary;
• local coherence.

A model may then appear to have learned the intended relationship while actually learning only corpus membership or topical similarity.

Relationship benchmarks should therefore use hard negatives:

• examples from the same document or project;
• nearby passages;
• overlapping entities;
• comparable lengths;
• similar edit distances;
• plausible but incorrect relationships.

The negative example should be wrong for the reason the benchmark intends to test—not because it belongs to an obviously different part of the corpus.

Install falsification gates in learning pipelines

The experimental sequence itself provides a reusable evaluation pattern:

tiny-set memorisation
→
held-out lineage generalisation
→
label-integrity audit
→
candidate-reversal audit
→
dimension and regularisation audit
→
shuffled-label control
→
nonlinear interaction control

Each gate answers a different question and rules out a different class of false positive.

An AI system should not claim to have learned concepts such as preference, quality, causality, risk, intent or future relevance merely because one downstream metric improves.

The strength of the claim should be limited by the strongest controls the representation has survived.

Do not assume one universal latent representation

The negative result argues against building a single universal “decision intelligence” vector and routing every task through it.

A better architecture is modular:

decision episode store
        ├── preference ranker
        ├── relationship or transformation model
        ├── future-outcome forecaster
        ├── explanation and reporting layer
        └── human decision layer

The same historical record can support all of these components.

But each component should have its own objective, evaluation procedure and failure conditions.

The ranker learns which alternative is preferred.

The relationship model learns how alternatives differ.

The forecaster learns what happens next.

The explanation layer records why a judgement was made.

The human layer retains authority over consequential application.

Shared evidence does not require a shared latent objective.

That may be the most useful general result of the experiment: human decisions should be stored as rich, structured episodes and interpreted through multiple specialised models rather than compressed into a single preference signal expected to explain everything.

Final Conclusion

I began with a proposed transformation:

preference as target
→
preference as evidence
→
preference as latent-state update

The earlier forecasting experiment supported the middle step. The rejected alternative contained information about the future of the selected branch.

The complete follow-up did not support the final step.

Authentic editorial preference was weakly learnable. A small antisymmetric MR.Q architecture recovered that signal where conventional encoder fine-tuning failed. But its hidden state did not improve future prediction beyond a properly regularised generic semantic representation. It did not beat identically shaped shuffled-preference controls, and it did not become useful when combined nonlinearly with all generic features in XGBoost.

The final distinction is:

information present in a decision record
≠
information selected by a preference objective
≠
information useful for predicting a later outcome

The model could learn who won.

Learning who won did not teach it what happened next.

For my future applications, the practical consequence is not to abandon preferences. It is to stop asking one compressed preference label to do every job.

Store the whole decision. Train the ranker on the choice. Train the forecaster on the consequence. Preserve the relationship between alternatives. Test every claimed transfer rather than assuming it.

    flowchart LR
    D["Complete decision episode"] --> P["Preference model"]
    D --> F["Outcome model"]

    P --> P1["Learns which candidate wins"]
    F --> F1["Learns what happens next"]

    P1 --> N["No incremental future utility<br/>beyond generic semantics"]
    F1 --> Y["Future revision remains predictable"]

    D --> L["System-design lesson"]
    L --> L1["Preserve context, alternatives,<br/>choice, relationship and outcome"]
    L --> L2["Use separate objectives<br/>for ranking and forecasting"]

    classDef source fill:#f3f3f3,stroke:#555,color:#111;
    classDef learned fill:#e8f5e9,stroke:#2e7d32,color:#111;
    classDef null fill:#fdecea,stroke:#c62828,color:#111;
    classDef lesson fill:#eaf2ff,stroke:#1565c0,color:#111;

    class D source;
    class P,F,P1,F1 learned;
    class N null;
    class Y,L,L1,L2 lesson;
  

The preference did not know the future.

The decision episode still has a great deal to teach us.

Appendix: Experimental Sequence and Results

This appendix records the complete path from the original frozen test to the final nonlinear control.

The code, frozen contracts, scripts and machine-readable reports for every stage below are available in the preference-futures GitHub repository.

A. Original Seven-Arm Transfer Experiment

Data

12,056 episodes
3,386 article lineages
10 grouped outer folds
7 representation arms
70 future-probe jobs

Frozen primary comparison

generic pooled out-of-fold log loss
-
authentic-preference pooled out-of-fold log loss

Primary result

generic log loss:              0.540113
authentic-preference log loss: 0.540832
improvement:                  -0.000719
95% CI:          [-0.003425, +0.002020]

Authentic preference beat generic in 3/10 folds. The primary hypothesis was not supported.

Descriptive arms

The pair-exposure result was exploratory and was later weakened by evidence that its source task admitted cross-lineage shortcuts.

B. Step 7A — Preference Learnability Audit

Question

Could the authentic preference task be learned at all?

Result

• Tiny sets of 256 and 512 examples were memorised perfectly.
• Held-out validation remained null-like.
• Best validation budget: 600 updates.

accuracy: 0.516156
log loss: 0.692733
95% CI: [0.487956, 0.544253]

Conclusion

The original DistilBERT source model had enough capacity to memorize but did not generalize authentic editorial selection.

C. Step 7B — Future-Label Integrity Audit

episodes audited:                  125,822
currently revised labels:           31,964
revised under alignment normalizer: 31,772
inconsistent labels:                   192
all-episode mismatch:              0.1526%
revised-case mismatch:             0.6007%

Conclusion

The normalization inconsistency was real but scientifically immaterial to the null result.

D. Blinded Oracle and Candidate-Swap Audit

Original oracle

examples: 300
correct: 179
accuracy: 59.67%
95% CI: [54.03%, 65.06%]

Candidate-swapped rerun

swapped accuracy:        50.00%
order consistency:       57.00%
consistent-set accuracy: 58.48%

Conclusion

A weak preference signal was visible to a strong reasoner, but unrestricted performance was not order-robust. This motivated a mathematically antisymmetric ranker.

E. Step 8 — Editorial MR.Q Source Task

MR.Q result

episodes: 12,056
accuracy: 0.523308
log loss: 0.691868
95% accuracy interval: [0.514386, 0.532215]
individual fold gates: 6/10
maximum swap-logit error: 0
pooled source gate: passed

Linear antisymmetric control

accuracy: 0.524635
log loss: 0.728408
individual fold gates: 0/10

Conclusion

Authentic editorial preference was weakly but reproducibly learnable through a compact antisymmetric nonlinear value model.

F. Step 8.4 — Initial MR.Q Future Transfer

Primary comparison

MR.Q choice-aware − generic choice-aware: -0.009401
95% CI: [-0.015635, -0.003066]

Secondary comparison

MR.Q blind − generic unoriented: -0.007408
95% CI: [-0.013178, -0.001707]

Conclusion under the frozen rule

The initial future-transfer gate passed.

G. Step 8.5 — Specificity Audit

Conclusion

The generic controls were 16–18 times larger, selected the maximum available regularisation in every fold and overfit substantially. Dimension- and regularisation-matched controls were required.

H. Step 8.6 — Matched Controls

Dimension-matched PCA comparison

MR.Q choice-aware − PCA generic choice-aware: -0.003991
95% CI: [-0.007606, -0.000435]

MR.Q beat naïve dimension-matched PCA.

Extended-regularisation comparison

MR.Q choice-aware − extended generic choice-aware: +0.001354
95% CI: [-0.001838, +0.004455]

The properly regularised generic model matched or slightly outperformed MR.Q.

Conclusion

The initial MR.Q transfer advantage did not survive a properly regularised generic control.

I. Step 8.7 — Shuffled-Preference MR.Q Control

Five shuffled-label MR.Q replicas were trained with identical architecture and budgets.

Choice-aware

authentic − mean shuffled log loss: -0.000412
95% CI: [-0.002654, +0.001841]
favourable replicas: 3/5

Blind

authentic − mean shuffled log loss: -0.000394
95% CI: [-0.002577, +0.001731]
favourable replicas: 3/5

Conclusion

Authentic preference supervision did not establish compact-state transfer specificity.

J. Step 8.8 — Combined XGBoost Nonlinear Test

Arms

all generic geometry
authentic MR.Q only
all generic + authentic MR.Q
all generic + each of five shuffled MR.Q controls

Results

Authentic versus generic

difference: -0.000007
95% CI: [-0.001081, +0.001091]

Authentic versus mean shuffled

difference: +0.000045
95% CI: [-0.000942, +0.001011]
favourable replicas: 2/5

Conclusion

Combined XGBoost found no authentic preference-specific nonlinear interaction.

K. Final Status of the Claims

The final result is a controlled distinction between source-task learnability and downstream transfer.