http://programmer.ie/tags/counterexamples/ Recent content in Counterexamples on Programmer.ie: Modern AI programming Hugo en-us Sun, 30 Aug 2026 00:17:00 +0100 http://programmer.ie/books/hallucination-from-first-principles/07-chapter/ Sun, 30 Aug 2026 00:17:00 +0100 http://programmer.ie/books/hallucination-from-first-principles/07-chapter/ <p>Chapter 6 changed the standard for evaluation.</p> <p>A detector is not trustworthy because it has a good AUC.</p> <p>It becomes trustworthy only after we understand how its errors behave under the conditions in which policy will use it.</p> <p>That immediately suggests a more aggressive question.</p> <p>Instead of waiting for those conditions to appear naturally, can we <strong>search for them deliberately</strong>?</p> <p>Suppose a containment detector works well on ordinary unsupported claims.</p>