DataLoader: Where Is the Training Loop Actually Waiting?

Sat, 08 Aug 2026 13:21:00 +0100

Here are two controlled training pipelines. They use the same batch size, the same machine, and the same synthetic post-batch workload. Each sample also carries the same nominal two-millisecond cost: in one pipeline that cost is waiting, while in the other it is fixed CPU work.

The tensor construction around that controlled cost is the same in both cases. What changes is the resource those two milliseconds consume.

Both are given the same treatment — raise num_workers from 0 to 8 — and measured the same way.

Dataset on Programmer.ie

DataLoader: Where Is the Training Loop Actually Waiting?