Models From First Principles 08: Preference Rankers — Learning Which Answer Is Better

Sat, 22 Aug 2026 01:18:00 +0100

Preference Rankers — Learning Which Answer Is Better

So far in Models From First Principles, we have mostly trained models by telling them what the answer should be.

MR.Q took a context and a response and produced a number:

context + response
        ↓
      model
        ↓
      0.82

That number might mean quality.

Or usefulness.

Or reward.

But there is an awkward question hiding underneath that architecture:

Where did the 0.82 come from?