Models From First Principles 03: SICQL — Building a Model From Q, V and Policy Networks

Sat, 08 Aug 2026 14:44:00 +0100

SICQL — Building a Model From Q, V and Policy Networks

In the previous post we took the MR.Q idea and expanded it into something richer.

Instead of asking one question of a shared representation, EBT asked several:

How good is this state-action pair?      -> Q
How good is the state more generally?    -> V
What action should be preferred?         -> Policy
How much better is Q than V?              -> Advantage

That already gave us a more expressive system.

Q Learning on Programmer.ie: Modern AI programming

Models From First Principles 03: SICQL — Building a Model From Q, V and Policy Networks

SICQL — Building a Model From Q, V and Policy Networks