Optimizers on Programmer.ie: Modern AI programming http://programmer.ie/tags/optimizers/ Recent content in Optimizers on Programmer.ie: Modern AI programming Hugo en-us Sat, 08 Aug 2026 15:05:00 +0100 Models From First Principles 07: PACS — Building an Optimizer From Gradient Statistics http://programmer.ie/post/models-from-first-principles-07/ Sat, 08 Aug 2026 15:05:00 +0100 http://programmer.ie/post/models-from-first-principles-07/ <h1 id="pacs--building-an-optimizer-from-gradient-statistics">PACS — Building an Optimizer From Gradient Statistics</h1> <p>So far in <strong>Models From First Principles</strong>, every post has asked some version of the same question:</p> <blockquote> <p>What should the model compute?</p></blockquote> <p>MR.Q gave us a scalar quality estimate.</p> <p>EBT split one shared representation into Q, V and Policy.</p> <p>SICQL made those heads explicit, replaceable components.</p> <p>HRM introduced repeated computation over fast and slow latent states.</p> <p>Tiny compressed iterative refinement into one recursive latent state.</p>