Use with AI · Chapter Application

Audit PyTorch DataLoader and Input Pipeline Performance

Measure where a PyTorch training pipeline waits for data and diagnose DataLoader, worker, transfer, preprocessing and prefetch bottlenecks before tuning the model.

Chapter Application PyTorch: Zero to Hero PyTorch data loading and training pipeline Intermediate
Read the source chapter → View the learning path →

How to use this

  1. Open a repository-aware AI assistant.
  2. Give it access to the repository or files you want reviewed.
  3. Copy the prompt below and run it unchanged first.
  4. Use the evidence it finds to decide what to inspect or change next.
PromptCopy and run against your own project
You are auditing a PyTorch repository where training throughput or GPU utilization may be limited by the input pipeline.

Do not assume DataLoader settings are the problem, and do not recommend increasing num_workers by default. First determine where each training iteration spends time.

Inspect the dataset, transforms, sampler, DataLoader construction, device transfer and training loop.

Work in this order:

1. Reconstruct the batch path:
   storage/source → Dataset.__getitem__ → transforms/collate → worker queue → host batch → device transfer → model.

2. Identify current settings and constraints:
   - batch_size
   - num_workers
   - pin_memory
   - persistent_workers
   - prefetch_factor
   - shuffle/sampler
   - custom collate_fn
   - CPU-heavy transforms
   - storage/network access
   - host-to-device transfer pattern

3. Look for evidence of stalls:
   - synchronous disk/network work per sample
   - repeated parsing/decoding that could be cached or moved
   - serialized Python transforms
   - expensive collate logic
   - too few or too many workers
   - worker startup repeated every epoch
   - unpinned host memory when asynchronous GPU transfer is intended
   - device copies performed in a way that forces synchronization
   - tiny batches that create excessive per-batch overhead
   - data preprocessing occurring on the critical training thread

4. Distinguish bottlenecks:
   - storage/I/O bound
   - CPU preprocessing bound
   - DataLoader scheduling/worker bound
   - host-to-device transfer bound
   - model/GPU bound
   - insufficient evidence

5. For every finding provide:
   - exact file/function
   - evidence
   - expected mechanism causing the delay
   - smallest experiment to prove or falsify it
   - only then, the smallest optimization

Output:

## Pipeline map
Show the actual batch path and relevant settings.

## Bottleneck hypotheses
Ranked by evidence, not intuition.

## Measurement plan
Specify timings/profiling that isolate data time, transfer time and compute time.

## Recommended changes
Only changes whose mechanism is supported by evidence.

## Verification
Define before/after measures such as batches/sec, examples/sec, GPU utilization, data-wait time and epoch duration.

Do not call an optimization successful because one metric improved if total training throughput did not.