← Adversarial Math From First Principles

Counterexamples

A universal claim is refuted by one validated witness: how the engine produces, re-evaluates, and records counterexamples, starting from the false shear.

You can test a false universal claim many times and watch it pass. Take the shear

$$ t' = t, \qquad x' = x + vt, $$
and the claim that it preserves $t^2-x^2$. At $t=0$ it does, since $x'=x$. At $v=0$ it does. At $x=-vt/2$ it does too, because then $x' = vt/2$ and $x'^2 = x^2$. A reader who tries a few convenient inputs may conclude that the claim holds. The residual explains why it seems to:
$$ \bigl(t^2 - (x+vt)^2\bigr) - \bigl(t^2 - x^2\bigr) = -(2xvt + v^2t^2) = -vt(2x+vt). $$
It vanishes exactly when $v=0$, or $t=0$, or $2x+vt=0$: a thin set. Off that set, the claim fails. Passing some tests says almost nothing about a universal claim. Failing one says everything.

This chapter shows how the engine produces a witness, checks it again, records it, and is careful about what it refutes. It builds on claims (Chapter 1) and the status ladder (Chapter 3).

Two refuted claims

Claim Statement Status Witness
incorrect-transform-preserves-interval@1.0.0 For all real $t, x$ and $\lvert v\rvert<1$, the shear preserves $t^2-x^2$ refuted_by_valid_counterexample SymPy: $t=1, x=0, v=1/2$. Z3: $t=-1, x=0, v=1/2$. Both residual $-1/4$.
pythagorean-all-triangles-2d@1.0.0 For all real points $A, B, C$, $d^2(B,C) = d^2(A,B) + d^2(A,C)$ refuted_by_valid_counterexample $A=(0,0)$, $B=(1,0)$, $C=(1,1)$, residual $-2$

The second claim and its repairs return in Chapter 25 and Chapter 27.

Checking the witness by hand

SymPy’s witness is $t=1$, $x=0$, $v=1/2$:

$$ t' = 1,\quad x' = \tfrac12,\quad t'^2 - x'^2 = 1 - \tfrac14 = \tfrac34,\quad t^2 - x^2 = 1,\quad \text{residual} = \tfrac34 - 1 = -\tfrac14 . $$
The formula agrees: $-vt(2x+vt) = -\tfrac12 \cdot 1 \cdot \tfrac12 = -\tfrac14$.

Z3’s witness is $t=-1$, $x=0$, $v=1/2$:

$$ x' = -\tfrac12,\quad t'^2 - x'^2 = \tfrac34,\quad t^2-x^2 = 1,\quad \text{residual} = -\tfrac14 . $$
Again the formula agrees: $-\tfrac12\cdot(-1)\cdot\left(-\tfrac12\right) = -\tfrac14$.

The two tools chose different witnesses and landed on the same residual. That is a coincidence of the search constraints, not a theorem. The residual depends on $v$ and $t$, and this pair happens to share $\lvert vt\rvert = 1/2$ with $x=0$.

Validating a witness versus finding one

The two tools get their witnesses differently, and the difference matters.

SymPy validates a chosen witness (refute_incorrect_transform). The assignment $t=1$, $x=0$, $v=1/2$ is supplied by the adapter, not discovered. The adapter substitutes it into the exact rational expressions and accepts it only if residual != 0 and the two intervals differ.

Z3 searches for one (discover_incorrect_transform_counterexample). It is given $-1< v<1$, $v\neq 0$, $t\neq 0$ and “transformed $\neq$ original”. If it returns sat, the adapter takes the model, substitutes it back into the encoded residual with z3.simplify, and accepts the witness only if the re-evaluated residual is non-zero. If it returns unsat, the status is inconclusive, not “the claim holds”. If it returns unknown, the status is also inconclusive, with the solver’s stated reason.

Notice the extra constraints $v\neq0$ and $t\neq0$. They steer the search away from the trivial points where the residual is zero. They are search constraints, not claim assumptions, and they do not weaken the refutation. A witness that satisfies the claim’s hypotheses and violates its conclusion refutes the claim, however it was found.

The model code enforces the rule in one place: an EvidenceResult will not construct with status refuted_by_valid_counterexample and no counterexample attached. The cross-record validator repeats the check.

What a counterexample refutes

Exactly the claim whose fingerprint it was run against, in its encoding, and nothing more. The evidence policy says that a refutation “does not imply a broader theory is false.” The shear failing the interval test refutes that universal claim. It does not refute Galilean kinematics, and it says nothing about whether nature conserves the interval.

There is a trust gap too. The witness is validated in the encoded expression. If the SymPy or Z3 encoding of the transform were wrong, the witness would refute the wrong claim and still validate perfectly. Re-evaluating guards against a bad solver output; it cannot guard against a bad encoding. Cross-checking the SymPy and Z3 encodings of the same claim helps, though both were written by the same project.

When proof and counterexample collide

What if a kernel-verified result and a validated counterexample both attach to the same claim fingerprint? Then assess_claim returns inconclusive, with a rationale saying that the “encodings, assumptions, claim fingerprints, and certificates require audit.” It does not break the tie. The report-validation suite also checks that altering a stored counterexample’s assignment is detected. No MATH01 claim triggers this conflict path; it is tested, not exercised on real data.

What this establishes

A counterexample is the cheapest strong result in mathematics, and the one this engine trusts most. The human-checkable part is the residual computation above, which a reviewer can redo in a minute; no review is recorded.

SymPy and Z3 agree on the refutation, but their agreement is not the argument. The argument is the substitution, which anyone can repeat. Chapter 6 explains why the engine does not count tool agreement as evidence.

The chapter also sets the discipline for the rest of the book: state the claim exactly, search for failure honestly, and keep the failure on file after a repair (Chapter 27).

Try it yourself

uv run python scripts/ci_negative_controls.py

Seeded error. In the SymPy adapter, change the assignment to $t=0$, $x=0$, $v=1/2$. This was executed on 2026-11-09 with SymPy 1.14.0 using ch05_10_18_22_24_checks.py:

ch05 seeded residual: 0
ch05 seeded witness valid: False

The residual is zero, so the validity predicate is false. By the adapter’s rule, that outcome is inconclusive with execution failed, not a refutation.

Exercise. Find a witness with $v = 3/5$ and $x = 0$. Which values of $t$ work, and what is the residual at $t = 5$? (Answer: any $t\neq0$ works; at $t=5$ the residual is $-vt(2x+vt) = -3\cdot3 = -9$.)

Open questions.

  1. SymPy’s witness is hand-chosen. Is a validator of a chosen witness weaker evidence than a discovered one? Both pass the same substitution.
  2. How far do $v\neq0$ and $t\neq0$ bias Z3’s choice of witness, and does that matter for claims whose interesting failures are at special points?
  3. Should the engine ever accept a refutation whose witness lies outside the claim’s own hypotheses? Today the Z3 search enforces $\lvert v\rvert<1$; the SymPy adapter relies on the supplied values satisfying it.

Evidence record

From the accepted report (R2 baseline, 2026-10-23):

math01-sympy-incorrect-transform-counterexample-v1   t=1,  x=0, v=1/2   residual -1/4
math01-z3-incorrect-transform-counterexample-v1      t=-1, x=0, v=1/2   residual -1/4

Both are refuted_by_valid_counterexample, succeeded, actual. Their validation strings read “Exact SymPy rational substitution; residual is non-zero,” and “Z3 model substituted back into the exact encoded residual.”

Later phases reuse the pattern with new residuals: $-2$ for the Pythagorean negative control (P6), and $-4$ for the second witness that defeats an overfitted patch (P7: $A=(0,0)$, $B=(1,0)$, $C=(2,1)$, dot product 2, residual $-2\cdot2=-4$).

Sources

No literature is cited; the method uses the project’s own adapters, src/math_engine/adapters/sympy_adapter.py and src/math_engine/adapters/z3_adapter.py.