Correctness, Determinism and Reproducibility

Floating-point nondeterminism, batching, seeds.

Inference From First Principles Chapter 39 of 45

01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

Chapter 38 planned capacity. This chapter examines correctness and reproducibility.

Floating-point nondeterminism

Floating-point operations are not associative. Parallel reductions can produce different results. GPU kernels may vary by version and hardware.

Batching effects

Batch size can affect numerical results due to reduction order. Batch invariance is not automatic.

Seed limitations

Seeds help reproducibility but do not guarantee identical results across hardware or library versions.

Research context

Separate seeded repeatability, deterministic algorithms, batch invariance, numerical tolerance, and semantic equivalence. Record model/tokenizer revisions, runtime and driver versions, dtype, kernels, hardware, request ordering, and sampler state. Cross-hardware bitwise identity is a stronger and often unavailable guarantee. PyTorch documents that deterministic algorithms can reduce performance and still not promise cross-version identity.

Experiment

The harness ran a function twice with the same seed and found exact equality. It also tested batch invariance. Manifest records Python, Torch, platform, and seed. Tests verify exact equality, manifest, batch invariance, and serialisation.

Limitations

CPU only.
No GPU determinism.
No library version sweep.
No statistical analysis.

Key terms

Determinism: same inputs produce same outputs.
Floating-point: finite precision arithmetic.
Batch invariance: output independent of batch size.
Manifest: environment record.
Tolerance: acceptable difference.

Reader exercises

Run on GPU and compare.
Vary batch size and measure differences.
Record full environment.
Explain why seeds are insufficient.

References

PyTorch reproducibility notes - official framework documentation.
PyTorch numerical accuracy notes - official framework documentation.
deepseek-ai/DeepInfer at 774ce244 - pinned first-party repository.

Implementation status

Manuscript: content/books/inference/39-chapter.md
Runnable example: examples/ch39/
Tests: tests/test_ch39_repro.py
Evidence: research/ch39/sources.md and research/ch39/results.json
Verification command: python -m unittest discover -s tests -v

The chapter is a draft. It does not claim cross-hardware determinism. The next chapter will examine benchmarks.