PyTorch: Zero to Hero — Working Ledger
Book intention
Learn PyTorch by understanding what the framework is doing from first principles, then build upward from the core training loop to modern model architectures.
This book is already substantially written. The ledger does not assume that published chapters are editorially complete: the new pass should test the progression, remove avoidable duplication, and preserve the practical debugging character that makes the sequence useful.
Book state: MOSTLY COMPLETE — editorial sweep pending
Review ladder: UNREVIEWED → STRUCTURE OK → CONTENT OK → PROSE OK → DONE
Chapter ledger
| # | File | Canonical title | Status | Job / durable notes |
|---|---|---|---|---|
| 00 | 00-chapter.md |
What Are We Actually Doing? | UNREVIEWED | Establish the mental model for training and for the first-principles approach before introducing PyTorch machinery. |
| 01 | 01-chapter.md |
PyTorch Tensor Shapes: Broadcasting, Reshape, View, Permute and the Errors That Waste Your Time | UNREVIEWED | Make tensor shape reasoning operational early; give the reader a vocabulary for diagnosing the most common structural mistakes. |
| 02 | 02-chapter.md |
PyTorch Autograd Debugging: requires_grad, detach, backward() and NaN Gradients | UNREVIEWED | Explain how gradients actually flow and fail so later training code is understandable rather than magical. |
| 03 | 03-chapter.md |
Build a Neural Network From Scratch in PyTorch Without nn.Module | UNREVIEWED | Construct the training mechanism before introducing framework abstractions; the reader should see what nn.Module will later automate. |
| 04 | 04-chapter.md |
The Most Important Idea in PyTorch: Recursive Composition | UNREVIEWED | Establish recursive composition and traversal as the organizing mental model for nn.Module; derive parameters, registration, ModuleList, buffers, state_dict, device movement, train/eval propagation and common registration bugs from that single idea. Distinguish the registered module hierarchy from autograd’s dynamic dependency graph. |
| 05 | 05-chapter.md |
PyTorch DataLoader Performance: num_workers, pin_memory, Prefetching and Why Your GPU Is Waiting | UNREVIEWED | Add the data pipeline and show that model training performance includes feeding the accelerator, not only model code. |
| 06 | 06-chapter.md |
PyTorch CNN Shape Errors: Conv2d Output Sizes, Channels, Flatten Bugs and How to Debug Them | UNREVIEWED | Apply shape reasoning to convolutional models and bridge from generic tensors to a real architecture family. |
| 06A | 06a-chapter.md |
Beyond 3D: Tensors, 100 Dimensions and an SVM From Scratch | UNREVIEWED | Advanced conceptual bridge from visible 1D/2D/3D geometry to high-dimensional feature spaces. Separate tensor rank from feature-space dimensionality, then use a linear SVM to show that the same dot-product boundary works in 2, 100, 500 or 768 dimensions. Prepare the reader to interpret transformer embedding axes without needing to visualise them. |
| 07 | 07-chapter.md |
PyTorch Attention Shapes: Q, K, V, Multi-Head Attention Masks and Transformer Dimension Errors | UNREVIEWED | Build the tensor mechanics of attention and prepare the reader for transformer/GPT construction without hiding dimension logic. |
| 08 | 08-chapter.md |
PyTorch Model Not Learning? A Systematic Debugging Guide | UNREVIEWED | Synthesize gradients, data, shapes, loss, and optimization into a disciplined diagnosis process for failed training. |
| 09 | 09-chapter.md |
PyTorch Performance Debugging: CUDA OOM, Slow Training, GPU Utilization and torch.compile | UNREVIEWED | Separate correctness from performance and teach how to reason about memory, utilization, profiling, and compilation. |
| 09A | 09a-chapter.md |
PyTorch Compiler Debugging: Graph Breaks, Guards, Recompiles and torch.compile | UNREVIEWED | Take the compiler material beyond the performance overview: make graph capture, graph breaks, guards, recompilation, dynamic shapes, compiler logging, backend isolation and compiled-region boundaries directly inspectable. Teach a layered eager → Dynamo → AOTAutograd → Inductor debugging workflow. |
| 09B | 09b-chapter.md |
Training Regressions and Reproducible Experiments: When Nothing Crashes but the Model Gets Worse | UNREVIEWED | Introduce differences-between-runs as a separate debugging class. Establish baselines, control or record randomness/environment/configuration, separate capability from regression tests, validate the benchmark itself, inspect individual failures, compare quality/performance/compiler health, and turn discovered failures into durable regression tests. |
| 10 | 10-chapter.md |
Build a Small GPT-Style Language Model From Scratch | UNREVIEWED | Use the mechanisms accumulated through the book to build a recognizable end-to-end model; this should feel earned rather than like a new tutorial dropped at the end. |
Book-level questions to resolve during review
- Does the sequence feel like one path from tensors to a working language model, or does it become a collection of debugging articles?
- Are the debugging chapters placed where the reader has enough context to understand them?
- Does chapter 10 explicitly cash out the mechanisms taught earlier, especially tensor shapes, autograd, modules, data loading, attention, compiler behavior, and regression measurement?
- Is any essential bridge missing between attention mechanics and the final GPT-style model?
- Which explanations are genuinely repeated for pedagogy, and which are accidental duplication?
Current next action
This is not the active book. When it becomes active, start at 00-chapter.md unless NEXT.md records a later checkpoint.
