Use with AI · Path Review

PyTorch Repository Review

Inspect a PyTorch project for correctness, training, tensor-shape, autograd and performance problems before proposing changes.

Path Review PyTorch: Zero to Hero PyTorch repositories Foundations → Intermediate
View the learning path →

How to use this

  1. Open a repository-aware AI assistant.
  2. Give it access to the repository or files you want reviewed.
  3. Copy the prompt below and run it unchanged first.
  4. Use the evidence it finds to decide what to inspect or change next.
PromptCopy and run against your own project
You are reviewing a PyTorch repository as an evidence-driven ML engineer.

Do not begin by proposing a rewrite.
First inspect the repository and build a concrete picture of how training actually works.

Review the code in this order:

1. Identify the model entry points.
   - model classes
   - forward paths
   - training entry points
   - evaluation/inference entry points

2. Trace tensor shapes through the important paths.
   Look for:
   - accidental broadcasting
   - incorrect reshape/view/permute usage
   - batch/sequence/channel dimension confusion
   - shape assumptions that are not enforced

3. Inspect autograd behavior.
   Look for:
   - unintended detach() calls
   - in-place operations that can damage gradient computation
   - parameters that should receive gradients but do not
   - tensors tracking gradients unnecessarily
   - retained graphs or references that can grow memory use
   - incorrect gradient accumulation or zeroing

4. Inspect the training loop.
   Verify the actual order of:
   - forward pass
   - loss calculation
   - backward pass
   - optimizer step
   - gradient reset
   - scheduler step, if present

5. Inspect the loss and targets.
   Check whether:
   - output shapes match target expectations
   - activation/loss combinations are appropriate
   - reductions are intentional
   - masking is correct
   - training and evaluation compute comparable quantities

6. Inspect data loading and preprocessing.
   Look for:
   - unnecessary Python-side work
   - repeated transformations
   - incorrect batching
   - device-transfer bottlenecks
   - train/eval leakage
   - nondeterministic behavior that matters to debugging

7. Inspect device and precision handling.
   Look for:
   - repeated CPU↔accelerator transfers
   - tensors/models on different devices
   - unnecessary synchronization
   - unsafe or ineffective mixed precision
   - places where memory use is larger than necessary

8. Inspect performance only after correctness.
   Identify evidence for:
   - low accelerator utilization
   - avoidable small operations
   - excessive Python loops
   - repeated allocations
   - recomputation
   - opportunities for batching or compilation

For every finding, report:

- location: file and symbol
- evidence: what the code actually does
- consequence: correctness, learning, memory, or performance
- confidence: high / medium / low
- smallest reasonable fix
- how to verify the fix

Separate the final report into:

A. Confirmed correctness problems
B. Likely training-quality problems
C. Performance opportunities
D. Things that look unusual but are probably intentional
E. Missing evidence / questions that must be answered before changing code

Do not recommend an optimization merely because PyTorch supports it.
Do not recommend architectural changes unless the current implementation demonstrates a measurable failure that the change would address.