http://programmer.ie/tags/evaluation/ Recent content in Evaluation on Programmer.ie: Modern AI programming Hugo en-us Sat, 08 Aug 2026 17:31:00 +0100 http://programmer.ie/post/agents-from-first-principles-09/ Sat, 08 Aug 2026 17:31:00 +0100 http://programmer.ie/post/agents-from-first-principles-09/ <p>An AI agent says:</p> <blockquote> <p>Done. The task is complete.</p></blockquote> <p>That sentence is almost worthless.</p> <p>The agent may have:</p> <ul> <li>edited the wrong file,</li> <li>changed the right file incorrectly,</li> <li>skipped part of the request,</li> <li>broken another subsystem,</li> <li>failed to save its work,</li> <li>misread a tool result,</li> <li>passed a stale test,</li> <li>inspected the wrong environment,</li> <li>or simply decided that its own answer looked convincing.</li> </ul> <p>The central problem is simple:</p> <blockquote> <p><strong>The system that produced the answer should not be the only system deciding whether the answer is correct.</strong></p>