What We Learned: Inference From First Principles
Synthesis and open questions.
The final chapter synthesizes what was built and what remains open.
What we learned
Inference is a system, not just a model. It spans tokenisation, computation, memory, caching, scheduling, serving, protocols, and economics.
Evidence audit
Every headline claim must map to evidence. The manifest records 45 chapters with tests and results. Failures and negative results are preserved. Audit each claim as established background, book hypothesis, local result, or first-party system report. Require an evidence chain from claim to source or raw result, environment, command, and code revision. State whether artefacts are merely available, locally functional, reproduced, or independently replicated.
Research context
ACM artifact badging and MLPerf provide frameworks for qualifying claims. PyTorch documents reproducibility limits. Negative and unavailable results must be published, not hidden.
Open questions
Many areas remain: real hardware performance, production reliability, security, and model quality. These require further work.
Experiment
The manifest reports 45 chapters, all with tests and results. Tests verify count, audit, and serialisation.
Limitations
- Simulations and toy implementations.
- No production validation.
- No real hardware benchmarks.
- No independent review.
Key terms
- Manifest: inventory of evidence.
- Audit: claim verification.
- Negative result: valid failure.
- Open question: unresolved issue.
- Synthesis: combining findings.
Reader exercises
- Audit one claim against its evidence.
- List three open questions.
- Propose the next experiment.
- Explain why negative results matter.
References
- ACM Artifact Review and Badging - Current - research artefact policy.
- MLPerf Inference: Datacenter - benchmark specification and results portal.
- PyTorch reproducibility notes - official framework documentation.
Implementation status
- Manuscript:
content/books/inference/45-chapter.md - Runnable example:
examples/ch45/ - Tests:
tests/test_ch45_manifest.py - Evidence:
research/ch45/sources.mdandresearch/ch45/results.json - Verification command:
python -m unittest discover -s tests -v
The book is a draft. It does not claim verified or production-ready status.