Transforms: What Does the Model Actually See?

Here are two training runs. Same model, same data, same optimizer, same seed, same number of epochs. The preprocessing pipelines differ by one keyword argument, and neither one raises.

scale=True  input range=[   -2.12,    2.01] first_epoch_loss=   0.0001 last=   0.0000 val_acc=1.000
scale=False input range=[   -2.12,  975.64] first_epoch_loss=  36.7257 last=   1.5337 val_acc=0.620

The second pipeline produced tensors with the right shape, the right dtype, the right device, and no non-finite values. Every check from Chapter 2 passes. Every check from Chapter 5 passes. The batch arrived on time, which is everything Chapter 6 taught us to verify. And the model went from perfect to barely better than guessing.

The argument was scale, on ToDtype. Its default is False:

ToDtype(dtype, scale: bool = False)

So the broken pipeline is the one you get by writing the obvious thing. v2.ToDtype(torch.float32) converts 8-bit image values to floating point and leaves them as 0.0 ... 255.0, because converting a dtype and reinterpreting a value scale are two different operations and PyTorch will not guess which one you meant.

This chapter is about the gap that failure lives in.

Where we are

Chapter 6 turned input delivery into an observable system. Something produces samples, something collates them, worker processes may prepare future batches, and the training loop consumes them. You can measure the wait at every boundary and name the stage that cannot keep up.

But Chapter 6 deliberately treated each batch as an object with a size, an arrival time and a production cost. The Producer in that chapter returned torch.randn(128) and a random label, because the contents were irrelevant to the question of when the batch showed up.

Real samples are not random numbers. They are files that were decoded, integers that were converted to floats, arrays that were resized, tensors that were normalized against statistics computed from somewhere, and images that may have been augmented differently on this epoch than the last. Each of those steps is a decision about what the numbers mean.

So the question for this chapter:

What exactly is the model seeing, and where did that representation first stop meaning what we intended?

The underlying fact is simple and easy to forget:

The model never trains on your abstract notion of the raw sample. It trains on whatever the preprocessing pipeline produced.

That makes preprocessing part of the model’s definition rather than scaffolding around it. Change the pipeline and you have changed the learning problem, even when every line of model code is identical.

Chapter 6 owned delivery. Chapter 7 owns representation.

The environment

Every numerical result in this chapter was produced by an executed experiment in the environment below. The listings show the mechanisms under investigation; repetitive training harness code is omitted where it would obscure the transform being tested.

PyTorch      2.13.0
TorchVision  0.28.0
Python       3.12.3
OS           Linux, CPU only

TorchVision’s transform API has changed more than most of PyTorch, and some behavior below is version-sensitive. Where that matters, the chapter says so and shows how to check rather than asking you to trust a constant.

A transform is a function with a contract

Strip away the library and a transform is nothing exotic:

def scale_to_unit_interval(x):
    return x.float() / 255.0

Input, output, done. What makes transforms worth a chapter is not their machinery but the number of independent things they can change, only one of which shows up in a shape.

A model-facing sample has at least five kinds of property, and a transform can alter any of them:

Chapter 2 gave us tools for the first row. Nothing so far has given us tools for the other four, and the failures worth a chapter are almost entirely in the other four — precisely because structure is the one property that error messages check for you.

The path a sample travels looks like this:

raw source
    ↓  decode
representation conversion
    ↓  deterministic preprocessing
resize / scale / tokenize
    ↓  stochastic augmentation
crop / flip / jitter
    ↓  normalization
model-facing tensor

Every arrow is a boundary where the sample can stop satisfying the contract you had in mind. The rest of this chapter is about finding the first one.

The technique: trace one sample, find the first divergence

Each chapter of this book has added an investigation method. Chapter 2 asked for the first wrong tensor rather than the first illegal one. Chapter 3 asked for the first broken edge between a parameter and the loss. Chapter 5 asked which of four structures disagreed about ownership. Chapter 6 asked where useful work stopped flowing.

Chapter 7 adds:

Find the first transform boundary where the sample stops satisfying the intended input contract.

The question is never only “which transform raised?” — a raising transform has already told you where it is. The question is where the data first became semantically wrong while remaining perfectly legal.

Making that executable takes about twenty lines.

import torch
from dataclasses import dataclass

@dataclass(frozen=True)
class Stage:
    name: str; kind: str; shape: tuple; dtype: str
    minimum: float; maximum: float; mean: float; std: float; finite: bool

def describe(name, x):
    if not torch.is_tensor(x):
        raise TypeError(
            f"{name}: stage reporter expects a Tensor or TVTensor, got {type(x).__name__}"
        )

    xf = x.float()
    return Stage(
        name,
        type(x).__name__,
        tuple(x.shape),
        str(x.dtype),
        float(xf.min()),
        float(xf.max()),
        float(xf.mean()),
        float(xf.std()),
        bool(torch.isfinite(xf).all()),
    )

def trace(sample, stages):
    """Run a pipeline one stage at a time, describing the sample after each."""
    x, rows = sample, [describe("raw", sample)]
    for name, t in stages:
        x = t(x)
        rows.append(describe(name, x))
    return x, rows

def report(rows):
    print(f"{'stage':<22}{'kind':<8}{'shape':<16}{'dtype':<10}"
          f"{'min':>9}{'max':>10}{'mean':>9}{'std':>8}")
    for r in rows:
        print(f"{r.name:<22}{r.kind:<8}{str(r.shape):<16}"
              f"{r.dtype.replace('torch.',''):<10}"
              f"{r.minimum:9.3f}{r.maximum:10.3f}{r.mean:9.3f}{r.std:8.3f}")

The key move is that trace takes the pipeline apart. Calling a Compose normally gives you the input and final output without exposing the intermediate values. The transforms themselves are inspectable, but their intermediate tensors are not automatically reported. Passing the stages separately makes each boundary observable.

Now run the two pipelines from the opening on the same sample.

from torchvision.transforms import v2

torch.manual_seed(0)
raw = torch.randint(0, 256, (3, 96, 96), dtype=torch.uint8)
MEAN, STD = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]

good = [("ToImage",            v2.ToImage()),
        ("Resize(64)",         v2.Resize((64, 64), antialias=True)),
        ("ToDtype(f32,scale)",  v2.ToDtype(torch.float32, scale=True)),
        ("Normalize",          v2.Normalize(MEAN, STD))]

broken = [("ToImage",     v2.ToImage()),
          ("Resize(64)",  v2.Resize((64, 64), antialias=True)),
          ("ToDtype(f32)", v2.ToDtype(torch.float32, scale=False)),
          ("Normalize",   v2.Normalize(MEAN, STD))]

report(trace(raw, good)[1])
report(trace(raw, broken)[1])

=== intended pipeline ===
stage                 kind    shape           dtype           min       max     mean     std
raw                   Tensor  (3, 96, 96)     uint8         0.000   255.000  127.747  73.759
ToImage               Image   (3, 96, 96)     uint8         0.000   255.000  127.747  73.759
Resize(64)            Image   (3, 64, 64)     uint8        23.000   225.000  127.750  32.216
ToDtype(f32,scale)    Image   (3, 64, 64)     float32       0.090     0.882    0.501   0.126
Normalize             Image   (3, 64, 64)     float32      -1.724     2.117    0.231   0.580

=== same pipeline, scale=False ===
stage                 kind    shape           dtype           min       max     mean     std
raw                   Tensor  (3, 96, 96)     uint8         0.000   255.000  127.747  73.759
ToImage               Image   (3, 96, 96)     uint8         0.000   255.000  127.747  73.759
Resize(64)            Image   (3, 64, 64)     uint8        23.000   225.000  127.750  32.216
ToDtype(f32)          Image   (3, 64, 64)     float32      23.000   225.000  127.750  32.216
Normalize             Image   (3, 64, 64)     float32      98.319   998.196  563.338 142.736

The two runs are identical for three rows and diverge at the fourth. That row is the diagnosis. Everything after it — the enormous normalized range, the loss of 36.7 in the opening, the 0.620 accuracy — is downstream symptom.

Note what the divergence looks like. The dtype column is correct in both. float32 is exactly what was requested and exactly what the model wants. The only column that shows the problem is the value range, and only if you know what range that stage was supposed to produce.

A stage report is evidence about representation. It becomes a diagnosis only when you have written down what each stage was supposed to produce.

Representation and numeric interpretation are separate questions

The reason ToImage and ToDtype are two transforms rather than one is that they answer different questions.

img = torch.randint(0, 256, (3, 8, 8), dtype=torch.uint8)

a = v2.ToImage()(img)
b = v2.ToDtype(torch.float32, scale=True)(a)
c = v2.ToDtype(torch.float32, scale=False)(a)

raw:                  Tensor torch.uint8   0 255
ToImage:              Image  torch.uint8   0 255 (3, 8, 8)
ToDtype(scale=True):  Image  torch.float32 0.0 1.0
ToDtype(scale=False): Image  torch.float32 0.0 255.0

ToImage changed the type and nothing else — same dtype, same values, same shape. What it produced is a tv_tensors.Image, a tensor subclass that carries the information “these numbers are an image.” That tag is what lets later transforms behave differently for images, masks and bounding boxes, which becomes essential later in this chapter.

ToDtype changed the numbers. With scale=True the mapping is exactly what you would write by hand:

uint8 : [0, 1, 127, 128, 254, 255]
scaled: [0.0, 0.00392157, 0.49803925, 0.50196081, 0.99607849, 1.0]
/255  : [0.0, 0.00392157, 0.49803922, 0.50196081, 0.99607843, 1.0]
equal to /255: True

Do not memorize “scaling means divide by 255.” TorchVision defines an expected numeric range for image dtypes and scale=True converts between those conventions.

For the conversion used here, uint8 → float32, that maps [0,255] to [0,1], which is effectively division by 255.

Other dtype pairs follow their own conversion rules. In particular, float-to-integer conversion does involve scaling, while float-to-float conversion does not. When the source or destination dtype changes, inspect the documented conversion rather than extending the /255 rule by analogy.

There is one asymmetry worth knowing, because it explains why this particular bug survives to production. Getting the order wrong is loud:

v2.Normalize(MEAN, STD)(img)     # img is still uint8

TypeError: Input tensor should be a float tensor. Got torch.uint8.

Getting the scale wrong is silent. Normalize checks that its input is floating point, which is a structural property it can verify. It cannot check that the floats mean what the statistics assume, because nothing in a float32 tensor records which convention produced it.

PyTorch enforces the properties it can see. Value-scale conventions are not among them.

Normalization is a change of coordinates

Normalization has a reputation as a vaguely beneficial step, which is the wrong mental model. It computes exactly this, per channel:

x' = (x - mean) / std

and nothing more. Verified against the transform:

manual = (x - torch.tensor(MEAN)[:, None, None]) / torch.tensor(STD)[:, None, None]
torch.allclose(v2.Normalize(MEAN, STD)(x), manual, atol=1e-6)

True

Normalization does not make an image better. It presents the image to the model in a different coordinate system. And the two numbers that define that coordinate system belong to a specific representation of the data.

That is the whole of the opening failure, restated. 0.485 is a plausible red-channel mean for images whose values live in [0, 1]. Applied to values living in [0, 255] it subtracts essentially nothing and then divides by 0.229, multiplying everything by about 4.4:

normalize(scaled)     min=  -2.118  max=    2.640  mean=   0.227  std=  1.289
normalize(unscaled)   min=  -2.118  max= 1136.357  mean= 562.257  std=326.439

Both operations are legal. Both produce finite float32. One of them presents the intended coordinate system and the other produces normalized values hundreds of times larger than intended.

One clarification that trips people up. After normalizing with dataset statistics, a single image will not have zero mean and unit variance. In the trace above, the normalized image has mean 0.231 and std 0.580, not 0 and 1. That is correct behavior: Normalize applies fixed reference statistics, not statistics recomputed per image. Per-image standardization is a different transform that answers a different question, and confusing the two leads people to “fix” a pipeline that was never broken.

Fitted preprocessing is experiment state

Where do those numbers come from? Sometimes from a pretrained model’s contract, which the next section covers. Sometimes you compute them from your own data — and the moment you do, the transform has acquired state derived from data, which changes what it is.

def channel_stats(u8_batch):
    """Population channel mean/std over a [N,C,H,W] uint8 stack, in [0,1] units."""
    x = v2.ToDtype(torch.float32, scale=True)(v2.ToImage()(u8_batch))
    return (
        x.mean(dim=(0, 2, 3)).tolist(),
        x.std(dim=(0, 2, 3), correction=0).tolist(),
    )

Here training data was collected under one lighting condition and validation later under a brighter one — an entirely ordinary situation:

fitted on train only      mean= [0.3511, 0.351, 0.3513]  std= [0.1362, 0.1354, 0.1357]
fitted on train+val       mean= [0.4314, 0.4317, 0.4315]  std= [0.1962, 0.1964, 0.196]
shift in channel-0 mean  : 0.0803

Fitting on everything, because all the files were sitting there, moved the statistics by eight percentage points. The training procedure now depends on a summary of the validation distribution. The validation set is therefore no longer independent of preprocessing selection: its metric measures a pipeline that was allowed to use information derived from that same evaluation split.

That may be a legitimate transductive design in a deliberately defined experiment. It is not a clean held-out validation protocol. Nothing raised, and the leak is invisible in every tensor the model receives.

The rule is narrow and worth stating precisely:

If preprocessing state is estimated from data, fit it using only data the training procedure is allowed to know.

The same shape appears wherever preprocessing carries fitted state: tabular scalers, categorical vocabularies, token vocabularies, quantization calibration. All of them are estimated from a split, and all of them leak if estimated from the wrong one.

The second half of the principle matters as much as the first. A fitted transform is part of the experiment, so it has to survive:

state = {"scale": "unit_interval", "mean": m_tr, "std": s_tr, "fit_split": "train"}

{
  "scale": "unit_interval",
  "mean": [
    0.35109102725982666,
    ...

Save it with the checkpoint, or record how to reconstruct it exactly. A model trained under one preprocessing state and served under another is not the same system, however cleanly the weights load.

Pretrained weights and preprocessing are one contract

There is an important case where you should not casually replace preprocessing with statistics computed from your own data: when you begin from pretrained weights.

Those weights were optimized under a particular input representation, so the preprocessing shipped with them is the safest baseline and the correct contract when you want to reproduce the published model behavior.

Fine-tuning can intentionally change that representation, and the model may adapt. But that is now an experimental change to the transfer-learning setup, not an interchangeable preprocessing detail. Measure it rather than assuming dataset-specific statistics are automatically better.

The habit worth building is to stop copying constants out of blog posts. TorchVision ships the preprocessing alongside the weights:

from torchvision.models import ResNet50_Weights, ViT_B_16_Weights, Inception_V3_Weights

print(ResNet50_Weights.IMAGENET1K_V1.transforms())

ResNet50_Weights.IMAGENET1K_V1
   ImageClassification(
    crop_size=[224]
    resize_size=[256]
    mean=[0.485, 0.456, 0.406]
    std=[0.229, 0.224, 0.225]
    interpolation=InterpolationMode.BILINEAR
)

ResNet50_Weights.IMAGENET1K_V2
   ImageClassification(
    crop_size=[224]
    resize_size=[232]
    mean=[0.485, 0.456, 0.406]
    std=[0.229, 0.224, 0.225]
    interpolation=InterpolationMode.BILINEAR
)

Inception_V3_Weights.IMAGENET1K_V1
   ImageClassification(
    crop_size=[299]
    resize_size=[342]
    mean=[0.485, 0.456, 0.406]
    std=[0.229, 0.224, 0.225]
    interpolation=InterpolationMode.BILINEAR
)

Read the first two rows together. Same architecture. Same normalization constants. Different resize_size: 256 for V1, 232 for V2. Two sets of weights for one model, with different preprocessing contracts.

Anyone who memorized “ResNet50 uses mean [0.485, 0.456, 0.406], resize 256, crop 224” gets the constants right and the resize wrong for half the available checkpoints. The consequence is measurable:

img = torch.randint(0, 256, (3, 400, 500), dtype=torch.uint8)
a = ResNet50_Weights.IMAGENET1K_V1.transforms()(img)
b = ResNet50_Weights.IMAGENET1K_V2.transforms()(img)

V1 preset: (3, 224, 224) torch.float32 mean=0.2264
V2 preset: (3, 224, 224) torch.float32 mean=0.2254
identical: False  max abs diff: 3.1337532997131348

Identical shape, identical dtype, nearly identical mean — and individual values differing by more than three units in normalized space, because the two presets resampled from different source scales.

The Inception row makes a second point: 299 and 342, not 224 and 256. There is no universal image size, just as there is no universal normalization constant.

Model weights and model preprocessing are one compatibility contract. A checkpoint can load with every key matched while inference is quietly broken.

Order is a program, not a style choice

Because each transform assumes something about what arrived from the stage before it, a pipeline is an ordered program. The usual way this is taught — “transforms are not commutative” — is true and does not convey how quietly the difference can hide.

Two pipelines, same operations, opposite order, applied to an image with a small object left of center:

img = tv_tensors.Image(torch.zeros(3, 64, 128, dtype=torch.uint8))
img[:, 20:28, 24:32] = 255                     # small object, left of centre

A = v2.Compose([v2.Resize((48, 48)),      v2.CenterCrop((32, 32))])
B = v2.Compose([v2.CenterCrop((48, 48)),  v2.Resize((32, 32))])

resize -> centercrop   shape=(3, 32, 32) dtype=torch.uint8 max=255 lit_pixels=120
centercrop -> resize   shape=(3, 32, 32) dtype=torch.uint8 max=  0 lit_pixels=0
outputs identical: False

Same shape. Same dtype. One of them contains the object and the other is a blank image. Cropping first discards the region that resizing would have brought inside the crop window, and the object was in exactly that region.

A stage report cannot flag this, because a blank image is a perfectly legal image. What flags it is asking, at each boundary, what the stage assumes it receives and what it must preserve. CenterCrop assumes the interesting content is near the center. On a wide image where it is not, that assumption is the bug.

Read a pipeline top to bottom asking what representation each stage assumes it is receiving. That is what “order matters” actually means.

Augmentation is a claim about invariance

The opening failure was numerical: the numbers stopped meaning what the next stage assumed. The crop-order example already showed that legal tensors can also lose task-relevant content.

Now add a third possibility: the transformed input remains perfectly meaningful on its own, but its relationship to the target becomes false.

Consider classifying arrows by the direction they point.

def arrow(direction):
    """A 1x32x32 uint8 arrow. direction: 'left' (class 0) or 'right' (class 1)."""
    a = torch.zeros(1, 32, 32, dtype=torch.uint8)
    a[:, 15:17, 6:26] = 255                       # shaft
    for k in range(8):                            # head: widens toward the tip
        if direction == "right":
            a[:, 16 - k: 17 + k, 25 - k] = 255
        else:
            a[:, 16 - k: 17 + k, 6 + k] = 255
    return a

def arrow_direction(x):
    """Semantic probe: the head is the end with the greater vertical extent."""
    height = (x[0] > 0).float().sum(0)
    lit = height.nonzero().flatten()
    left_mass  = height[lit[:len(lit) // 3]].mean()
    right_mass = height[lit[-(len(lit) // 3):]].mean()
    return "right" if right_mass > left_mass else "left"

Now apply the most standard augmentation in computer vision to a left-pointing arrow labelled class 0:

img = tv_tensors.Image(arrow("left"))
out = v2.RandomHorizontalFlip(p=1.0)(img)

after RandomHorizontalFlip(p=1.0):
  type  : Image
  shape : (1, 32, 32)  dtype: torch.uint8
  min/max: 0 255
  finite : True
  label  : 0 (still 'left')
  image now depicts: right

Type, shape, dtype, range, finiteness: all correct. The transform succeeded. The training example is now a picture of a right-pointing arrow labelled “left.”

This is not a hypothetical harm. Training the same small classifier three ways:

no flip                     val_acc = 1.000
RandomHorizontalFlip(p=0.5) val_acc = 0.520
RandomHorizontalFlip(p=1.0) val_acc = 0.000

The p=0.5 row is the one people expect: half the labels are now wrong, the two classes become indistinguishable, and the model lands on chance.

The p=1.0 row is more interesting, and worth sitting with. Zero percent. Not chance — perfectly wrong. The model learned the task flawlessly; it learned it from a training set in which every image was mirrored, so it learned the exact inverse of the mapping we wanted, and validation (which is not flipped) scores every single prediction wrong. Successful learning of a consistently mislabelled problem looks exactly like this, and a training-loss curve would have looked wonderful throughout.

The lesson is not “do not use horizontal flips.” Horizontal flipping is genuinely valid for a great deal of object photography. The lesson is what applying it asserts:

Applying augmentation T claims that meaning(x) ≈ meaning(T(x)) for every aspect the target depends on. That claim is about your task, and no transform library can check it for you.

The claim is false whenever the label depends on the property being changed — direction, handedness, text, anatomical orientation, anything where a mirror image is a different thing. It is also false in less obvious ways: an augmentation that removes small objects breaks the claim for a detection task while leaving it true for a scene-classification task on the same images.

Notice also that arrow_direction exists. Some semantic properties can be checked in code, and when they can, they belong in your test suite. A probe that reads the direction out of the pixels turns “does this augmentation preserve the label?” from a judgement call into an assertion.

The target has a contract too

The arrow failure was a relationship bug: the input changed and the target did not. The mirror image of that is treating the target as though its representation were obvious.

Datasets commonly expose two hooks, and the split is exactly what it looks like:

dataset = SomeDataset(..., transform=input_transform, target_transform=target_transform)

The useful discipline is that the target representation is dictated by the objective, not by convention or by whichever example you copied. It is easy to reach for one-hot encoding because a target_transform slot exists and one-hot looks like the tidy thing to do. Check what the loss actually wants first.

For cross_entropy, what it wants is more flexible than the folklore suggests:

import torch.nn.functional as F

torch.manual_seed(0)

logits = torch.randn(4, 3)
idx = torch.tensor([0, 2, 1, 0])

one_hot = F.one_hot(idx, num_classes=3).float()

# A simple softened version of the class-index targets:
soft_targets = 0.8 * one_hot + 0.2 / 3

print("class indices:", F.cross_entropy(logits, idx).item())
print("class probs  :", F.cross_entropy(logits, one_hot).item())
print("soft targets :", F.cross_entropy(logits, soft_targets).item())

try:
    F.cross_entropy(logits, idx.float())
except RuntimeError as e:
    print("float 1-D target: raises RuntimeError")

class indices   : 1.082031011581421 torch.int64
class probs     : 1.082031011581421 torch.float32
soft targets    : 1.3044164180755615
float 1-D target: RuntimeError expected target dtype to be Long or Byte, but got Float

So “cross entropy requires integer class indices” is not true. It accepts class indices or per-class probabilities, and one-hot probabilities give the same loss as the corresponding indices to within floating-point error. That second form is what makes label smoothing, distillation and mixup expressible at all.

There is a very Chapter-7 catch in the probability-target form: PyTorch does not verify that those floating-point targets are actually probability distributions. A same-shaped float tensor can contain negative values or rows that do not sum to one and still produce a loss.

That creates another legal-but-wrong boundary: correct shape, correct dtype, loss executes, invalid probability semantics. If you choose the probability-target form, validating the distribution is your responsibility.

What is true is that for ordinary single-label classification, class indices are the simpler and cheaper representation, and that the two forms are distinguished by shape and dtype rather than by an argument. Hand it a 1-D float tensor — one-hot encoding that you forgot to reshape, or indices you cast by accident — and it raises, because a 1-D float target matches neither contract.

The target transform must produce the representation the chosen objective requires. Read the loss function’s contract; do not infer it from an example.

For the ordinary class-index target representation used by most classification examples in this book, the machine-checkable contract can include:

assert labels.dtype == torch.int64
assert bool(((0 <= labels) & (labels < num_classes)).all())

If the loss deliberately uses an ignore_index, exclude those positions from the range check. If it uses class-probability targets instead, the contract is different: target shape must match the logits and each row should represent a valid probability distribution.

The second line catches an entire genre of bug — labels that are 1-indexed, a class map with a gap, or a filtered subset whose IDs were never remapped — before it becomes an indexing error thirty layers away.

Stochastic transforms must be inspected as a distribution

One execution of a random transform tells you almost nothing. The transform is not a value; it is a distribution over values, and the properties you care about are properties of the distribution.

Take the small object from the order example and ask how often a random crop keeps it:

img = tv_tensors.Image(torch.zeros(3, 64, 128, dtype=torch.uint8))
img[:, 8:14, 10:16] = 255

torch.manual_seed(0)

for scale in [(0.08, 1.0), (0.5, 1.0), (0.9, 1.0)]:
    rrc = v2.RandomResizedCrop(size=(32, 32), scale=scale, antialias=True)
    kept = sum(float((rrc(img) > 0).any()) for _ in range(500))
    print(f"scale={scale} object retained in {kept/500:.1%} of crops")

  scale=(0.08, 1.0)  object retained in  16.8% of crops
  scale=(0.5, 1.0)   object retained in  16.6% of crops
  scale=(0.9, 1.0)   object retained in   0.0% of crops

Two findings, and the second is not the one anybody predicts.

The first: with default settings, more than eighty percent of training views of this image contain no object at all. If the label says “object present,” four out of five augmented examples contradict it.

The second: the gentlest setting retained the object zero percent of the time. That is backwards, so it needs an explanation rather than a shrug. RandomResizedCrop samples an area fraction and an aspect ratio, retries a bounded number of times if the sampled box does not fit, and falls back to a deterministic center crop when sampling fails. Asking for 90% of the area of a 64×128 image with a near-square aspect ratio does not fit inside 64 pixels of height, so every attempt fails:

torch.manual_seed(0)

for scale in [(0.08, 1.0), (0.5, 1.0), (0.9, 1.0)]:
    rrc = v2.RandomResizedCrop(size=(32, 32), scale=scale)
    boxes = [tuple(rrc.make_params([img]).values()) for _ in range(300)]
    print(scale, "distinct crops:", len(set(boxes)))

(0.08, 1.0) distinct crops: 300
(0.5, 1.0)  distinct crops: 186
(0.9, 1.0)  distinct crops: 1

At scale=(0.9, 1.0) the “random” transform produced one crop, three hundred times. It is not augmenting anything, and it always excludes the corner where the object lives.

You cannot reason your way to that from the parameter names. You get it by sampling the transform and looking at the distribution — of outputs, and where the API exposes it, of the parameters themselves.

Inspect a stochastic transform by running it many times on a known sample and asking what fraction of the distribution still satisfies the contract. “It ran and looked fine once” is not evidence.

For images, looking at the samples is legitimate evidence and you should do it — but look at known samples chosen because you know what must survive: the smallest object, the rare class, the example near the crop boundary, the image whose label depends on text. Random samples from a large dataset will almost never surface the case that breaks.

Objects that share geometry must share the geometric decision

Classification often has a scalar target, so geometric transforms do not usually require changing target coordinates. But geometry can still invalidate the meaning of that scalar target — the arrow example just proved it.

Detection, segmentation and keypoint tasks add another requirement: their targets contain geometry themselves. A crop, resize or flip must therefore update both the input and the geometric target using the same sampled transformation.

The failure mode is calling the transform twice:

f = v2.RandomHorizontalFlip(p=0.5)
a, b = f(img), f(mask)        # two independent random draws

independent calls: image and mask disagreed in 99/200 trials
structured call  : image and mask disagreed in   0/200 trials

Half the samples have a mask describing the mirror image of the picture it is attached to. Both tensors are legal. Both have the right shape. The dataset is now half noise.

The repair is to make the transform aware that these objects belong together, which is what the tv_tensors types are for:

from torchvision import tv_tensors

img = tv_tensors.Image(torch.zeros(3, 32, 32, dtype=torch.uint8))
img[:, 4:12, 20:28] = 255

mask = tv_tensors.Mask(torch.zeros(32, 32, dtype=torch.uint8))
mask[4:12, 20:28] = 1

boxes = tv_tensors.BoundingBoxes(
    torch.tensor([[20, 4, 28, 12]]), format="XYXY", canvas_size=(32, 32))

flipped_img, flipped_mask, flipped_boxes = v2.RandomHorizontalFlip(p=1.0)(img, mask, boxes)

image bright cols: [4, 5, 6, 7, 8, 9, 10, 11]
mask  cols       : [4, 5, 6, 7, 8, 9, 10, 11]
boxes            : [[4, 4, 12, 12]]

One call, one sampled decision, three consistently updated objects. The box moved from x-range [20, 28] to [4, 12], which is the exact mirror in a 32-wide canvas, and the mask columns match the image columns.

The dispatch is by type, not by position, and it is worth seeing that the transforms treat the types differently rather than uniformly:

pipe = v2.Compose([v2.RandomHorizontalFlip(p=1.0),
                   v2.ToDtype(torch.float32, scale=True),
                   v2.Normalize([0.5]*3, [0.25]*3)])
i2, m2 = pipe(img, mask)

image: Image torch.float32 range=[-2.00,2.00]
mask : Mask  torch.uint8   unique=[0, 1]

The image was flipped, converted and normalized. The mask was flipped and left otherwise alone — it is still uint8 with values {0, 1}, because normalizing a class-index mask would be meaningless. That is why ToImage exists and why the tag it attaches matters: a bare tensor has no way to say “I am a segmentation mask, do not rescale me.”

Anything that shares geometry must share the geometric decision. Express the relationship in the type so the transform can honor it, rather than hoping two random calls agree.

The principle outlives the API. Whatever library you use, the question is the same: is this one sampled transformation applied consistently, or several independent ones applied hopefully?

Metadata catches structure; it cannot prove meaning

By now the pattern should be visible, and it is worth making explicit because it determines what your tests can and cannot do.

Here is a model that trains to perfect accuracy on a task where the class is determined by which color channel is bright. Serving code then does what serving code does — reads images with a library that returns channels in BGR order:

load_state_dict: <All keys matched successfully>
  serving uses RGB (as trained)  shape=(300, 3, 16, 16) dtype=torch.float32 range=[-1.43,1.49] val_acc=1.000
  serving uses BGR               shape=(300, 3, 16, 16) dtype=torch.float32 range=[-1.43,1.49] val_acc=0.000

Identical shape. Identical dtype. Identical range, to two decimals. Identical every field the stage report prints. Accuracy 1.000 and 0.000.

No amount of metadata inspection separates those two tensors, because the difference is which axis means what, and axis meaning is a convention held in your head rather than a property stored in the tensor. This is Chapter 2’s lesson — legal is not the same as correct — arriving with a much larger blast radius.

So contracts come in two levels, and it is worth being honest about which is which:

Write the machine-checkable half as code, because it is free and it catches an enormous class of errors before the first optimizer step:

def require_model_image(
    x,
    size=(3, 224, 224),
    value_bounds=(-5.0, 5.0),
):
    if tuple(x.shape) != size:
        raise ValueError(f"expected {size}, got {tuple(x.shape)}")
    if x.dtype != torch.float32:
        raise TypeError(f"expected float32, got {x.dtype}")
    if not torch.isfinite(x).all():
        raise ValueError("non-finite values in image")

    low, high = value_bounds
    if not (low < float(x.min()) and float(x.max()) < high):
        raise ValueError(
            f"values outside expected range {value_bounds}: "
            f"[{x.min():.2f}, {x.max():.2f}]"
        )

That last check is the one that would have caught the opening failure, and it is the one people leave out. It encodes an expectation about numeric interpretation rather than structure. It is a heuristic, not a proof — but it fires on [98, 998] and stays quiet on [-2.1, 2.6], which is exactly the discrimination we could not otherwise make automatically.

The bounds here belong to this specific ImageNet-style normalization contract. They are not universal bounds for normalized tensors.

For the semantic half, write probes where the property is computable (arrow_direction is one), and otherwise keep a small gallery of known samples and look at them. Looking at your data is not an admission of defeat.

If the pipeline normalizes, build a display copy rather than inspecting the training tensor, and invert the normalization to do it:

def denormalize(x, mean=MEAN, std=STD):
    m = torch.tensor(mean, device=x.device)[:, None, None]
    s = torch.tensor(std, device=x.device)[:, None, None]
    return (x * s + m).clamp(0, 1)

denormalized range: 0.0902 0.8824
round-trip max error vs pre-normalized: 5.960464477539063e-08

The round trip recovers the pre-normalization image to floating-point precision, so what you are looking at is the actual sample rather than an approximation of it. Note the clamp: it exists for display only, and applying it to a training tensor would silently discard values the model is supposed to see.

Testing a pipeline

The two-level distinction determines what a test can assert, and the mistake is testing stochastic transforms for the property deterministic ones have.

A deterministic pipeline should be reproducible, and that is worth asserting because it is the property that breaks when someone adds an augmentation to the wrong list:

a, b = val_tf(raw), val_tf(raw)
torch.testing.assert_close(a, b)

val_tf deterministic: True

A stochastic pipeline must not be tested that way. Its outputs are supposed to differ; what should not differ are the invariants:

outs = [train_tf(raw) for _ in range(20)]

train_tf identical across draws: False
train_tf shape invariant: {(3, 64, 64)}
train_tf dtype invariant: {torch.float32}
train_tf all finite: True

Variation in the values, none in the contract. That is the shape of a good augmentation test: assert over a sample of draws that structure, dtype, finiteness, target validity and paired-geometry alignment all hold, and assert nothing about the values themselves.

The third test is the one people never write: run both pipelines on the same sample and compare their contracts directly.

def contract(x):
    return (tuple(x.shape), x.dtype, round(float(x.min()), 1), round(float(x.max()), 1))

t, v = train_tf(raw), val_tf(raw)
assert t.shape == v.shape and t.dtype == v.dtype        # structural agreement
assert t.abs().max() / v.abs().max() < 5                # comparable magnitudes

train_tf (scale=True)  train=((3, 64, 64), torch.float32, -1.7, 2.3)
                       val  =((3, 64, 64), torch.float32, -1.5, 2.0)
                       shape/dtype agree: True   magnitudes comparable: True

train_tf (scale=False) train=((3, 64, 64), torch.float32, 73.9, 1087.2)
                       val  =((3, 64, 64), torch.float32, -1.5, 2.0)
                       shape/dtype agree: True   magnitudes comparable: False

The first assertion passes in both cases. Structure was never the problem, and a test that checks only shape and dtype would have signed off on the opening failure without complaint.

The second assertion catches drift between these two pipelines. It does not establish that either pipeline is correct.

If both training and validation accidentally used scale=False, their magnitudes would agree and this comparison would pass while both pipelines violated the model contract.

So use cross-pipeline comparison for consistency, and compare each pipeline independently against the explicit representation the model expects. Agreement is necessary; it is not sufficient.

Three pipelines, one contract

A project usually has three preprocessing paths, and the discipline is knowing exactly where they may differ.

                       may differ          must agree
training      stochastic augmentation   final model-facing representation
validation    deterministic             final model-facing representation
inference     deterministic             final model-facing representation

Training may inject randomness because variation is part of what it is learning from. Validation is usually deterministic because its job is to be a stable measuring instrument — if the benchmark changes between passes, a metric movement no longer distinguishes a better model from a different measurement. That is a default rather than a law: test-time augmentation is a legitimate technique. The principle is that evaluation randomness must be deliberate and interpreted as part of the evaluation procedure, rather than inherited by accident because someone passed train_transform to both datasets.

What must agree across all three is the final representation: channel count and order, dtype, value scale, normalization state, and the spatial contract the model was built for.

Begin with the cheap structural comparison:

train_x, _ = next(iter(train_loader))
val_x, _ = next(iter(val_loader))

assert train_x.shape[1:] == val_x.shape[1:]
assert train_x.dtype == val_x.dtype

but do not stop there. The opening failure passes both lines.

Also verify each pipeline against the intended numeric contract — value scale and normalization convention — and verify semantic conventions such as channel order from known samples or explicit preprocessing configuration.

The inference path is the one that rots, because it usually lives in a different file, often a different repository, and sometimes a different language. Chapter 5 showed a checkpoint that loaded with every key matched and returned a worthless model. This is the same failure with the state moved outside the checkpoint: the weights are perfect and the input convention is not.

matching preprocessing    val_acc = 1.000
forgot to scale           val_acc = 0.513

The remedy is to treat the deterministic preprocessing definition as part of the model artifact — serialized beside the weights, or reconstructed from a recorded configuration — rather than as code that happens to exist in the training script. TorchVision’s weights.transforms() is exactly this idea applied to published models, and it is a good pattern to copy for your own.

One caution against over-correcting. Not every preprocessing mismatch is fatal. In this synthetic experiment, changing the normalization statistics while keeping the scale correct left accuracy at 1.000. The task remained easily separable under that particular affine change in the input coordinates.

Do not generalize that result into “normalization mismatch is harmless.” A fixed trained model does not automatically adapt to a new preprocessing convention, and the effect depends on the model, data and size of the shift.

The experiments here establish only that the unscaled and channel-permuted variants were catastrophic for these controlled tasks. Other preprocessing drift must be measured rather than ranked in advance.

Where transforms run, briefly

A transform applied inside Dataset.__getitem__ is part of sample production. With num_workers > 0 it executes inside worker processes, which means transform cost is producer cost in exactly the sense Chapter 6 measured.

That is the whole of the connection, and it is deliberately brief because Chapter 6 owns the measurement. If your transforms are expensive, go back and use the harness there: split the step, sweep worker counts, and find out whether the producer or the consumer is limiting throughput. The decomposition Chapter 6 recommended — read → decode → transform → collate — is the one that tells you whether transforms are the expensive stage.

Two connections back are worth making explicit here.

Deterministic and stochastic work have different placement economics. Deterministic preprocessing whose output depends only on stable input and stable configuration is a candidate for precomputation or caching.

Whether caching is worthwhile depends on storage cost, cache size, invalidation and I/O behavior.

If fresh stochastic variation is part of the training design, keep that stochastic stage after the cache boundary. You can cache pre-generated augmented views, but doing so changes the training distribution from fresh draws to a fixed finite set. Splitting a pipeline at that boundary is what makes caching possible:

raw → expensive deterministic preprocessing → CACHE → random augmentation → model

Cache invalidation then becomes a correctness problem: change the transform definition and a stale cache silently preserves the old behavior, which is a representation bug that survives a code fix. Version the cache or rebuild it deliberately.

Debug one sample before debugging a pipeline. When something looks wrong, do not start with eight workers, prefetching, random transforms, collation and device transfer all interacting. Start with:

raw = dataset_without_transform[index]
x = transform(raw)

One sample, one process, no batching. Chapter 6 made the same argument for a different reason — num_workers=0 to expose a worker traceback — and it applies here for a stronger one: a stochastic transform inside a worker process is harder to isolate and reproduce, while calling it directly on one chosen sample removes several unrelated sources of complexity.

Which raises randomness. When investigating a stochastic transform, control enough of it to reproduce the failure: call the transform directly, seed a generator, or temporarily replace the random choice with a fixed one (p=1.0 rather than p=0.5, as the arrow experiment did). Chapter 6 covered how worker RNG state actually works; the only rule that belongs here is that a transform you cannot reproduce is a transform you cannot debug.

The same model, beyond images

Images dominate this chapter because they make semantic failure visible — you can see that the arrow points the wrong way. The structure is not specific to images.

Every row has the same four columns, and every failure in this chapter has an analogue in every row. A padding token treated as a real token is the text version of a mask that no longer aligns. A vocabulary fitted on train-plus-test is the text version of leaked normalization statistics. A tokenizer that differs between training and serving is the text version of BGR at inference: the token IDs can have perfectly valid shape and dtype while representing a different mapping from text to model input.

Every model receives a representation produced by an ordered preprocessing program. The domain changes the operations; it does not change the questions.

Using AI on a transform pipeline

Ask an assistant whether a transform pipeline is correct and you may get a fluent answer based only on the code. But code alone is often insufficient to establish correctness here, because the missing facts may include what the raw data represents, what the target means and which preprocessing contract the model was trained under.

The move is the one this book keeps using — make it characterize before it rewrites.

Here is a PyTorch transform pipeline and a description of one known raw sample.
Do not rewrite the pipeline and do not propose fixes yet.

First, build a table with one row per transform:
  - the input representation it assumes (type, shape, dtype, value scale);
  - the output representation it should produce;
  - which properties it intentionally changes;
  - which properties must remain invariant;
  - whether it introduces randomness;
  - whether it changes geometry;
  - whether the target must change with it.

I will then give you the observed type, shape, dtype, min, max, mean and std
after every stage, plus the label and what the sample depicts.

Identify the FIRST stage at which the observed representation becomes
inconsistent with the contract in your table.

Then answer separately:
  1. Is the failure structural, numerical, semantic or relational?
  2. What is the smallest experiment that would confirm that diagnosis rather
     than the two most likely alternatives?
  3. What measurement should change after the repair?
  4. What measurement must NOT change after the repair?

Only after I report the result of that experiment should you propose a change.

Question 4 is the one to keep. It is what stops a repair from being a rewrite: if fixing a value-scale bug also changes the output spatial size, something was replaced that should not have been.

There is a specific weakness worth exploiting. Assistants are good at the transform catalogue and much weaker at your task semantics, because the semantics are not in the code. So do not ask “is RandomHorizontalFlip appropriate here?” Ask:

My labels are: 0 = arrow points left, 1 = arrow points right.
For each transform in this pipeline, state whether it can change the property
that determines the label, and give me a probe I can run on one sample to
check whether it did.

That question has a checkable answer, and the probe is worth more than the opinion.

Ask AI to characterize the pipeline before asking it to rewrite the pipeline.

The transform debugging sequence

When a pipeline is suspect, this is the order that finds the problem fastest. It is deliberately front-loaded with the cheap steps.

1.  Choose ONE known raw sample — one where you know what the answer should be.
    Prefer an edge case: the small object, the rare class, the directional one.

2.  Bypass concurrency. No DataLoader, no workers, no batching.
    raw = dataset_without_transform[i]; x = transform(raw)

3.  Write down the intended final contract BEFORE looking at anything:
    shape, dtype, channel order and meaning, value scale, normalization state.

4.  Run the pipeline one stage at a time and describe each output:
    type, shape, dtype, min, max, mean, std, finite.

5.  Compare each stage against what that stage was supposed to produce.
    The FIRST divergence is the diagnosis; everything after it is symptom.

6.  Classify the divergence:
    structural  → shape, type, axis count
    numerical   → dtype, scale, normalization state
    semantic    → the values no longer depict what the label claims
    relational  → target, mask, boxes or modalities no longer agree

7.  If the stage is stochastic, run it many times on the same sample and
    inspect the DISTRIBUTION: how often does the contract still hold?
    Check the sampled parameters too, not only the outputs.

8.  Verify the target still describes the transformed input.
    Use a probe if the property is computable; look at the sample if not.

9.  Compare the final contracts of train, validation and inference.
    They may differ in randomness. They must agree in representation.

10. Check the provenance of any fitted state: which split produced it,
    and is it saved with the model?

11. Only now reintroduce batching, workers and performance measurement,
    and hand any remaining slowness back to Chapter 6.

Steps 1 through 5 are deliberately cheap. Run them before escalating to a full training experiment.

What you should now be able to answer

Here is a pipeline of the kind you will be handed. It runs. Work through it before reading on.

MEAN, STD = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]

train_tf = v2.Compose([
    v2.ToImage(),
    v2.RandomResizedCrop((224, 224), scale=(0.8, 1.0)),
    v2.RandomHorizontalFlip(p=0.5),
    v2.ToDtype(torch.float32),
    v2.Normalize(MEAN, STD),
])

val_tf = v2.Compose([
    v2.ToImage(),
    v2.Resize((256, 256)),
    v2.CenterCrop((224, 224)),
    v2.ToDtype(torch.float32, scale=True),
    v2.Normalize(MEAN, STD),
])

from torch import nn
from torchvision.models import resnet50, ResNet50_Weights

weights = ResNet50_Weights.IMAGENET1K_V2

# Assume ArrowDataset returns three-channel RGB images and labels:
# 0 = left, 1 = right.
train_ds = ArrowDataset(train_paths, transform=train_tf)
val_ds   = ArrowDataset(val_paths,   transform=val_tf)

model = resnet50(weights=weights)
model.fc = nn.Linear(model.fc.in_features, 2)

Where is the first divergence in train_tf? At ToDtype(torch.float32). scale defaults to False, so this produces float32 values in [0, 255] and the subsequent Normalize is applied under the wrong coordinate system. The stage report shows it as a value range of roughly [98, 998] where [-2.1, 2.6] was intended. The dtype column is correct throughout, which is why nobody notices.

Is that the only problem? No, and this is the point of the technique. Fixing the first divergence does not end the investigation; it lets you see the next one. There are three more.

What is wrong with the flip? The labels encode direction, so meaning(x) and meaning(flip(x)) are different classes. At p=0.5 roughly half the training examples become contradictory. The measured consequence in this chapter was validation accuracy falling from 1.000 to 0.520. Nothing raises, and the training loss curve looks unremarkable.

How does the validation preprocessing differ from the pretrained weights’ published inference contract? ResNet50_Weights.IMAGENET1K_V2 uses resize_size=232, while val_tf uses 256.

If the intention is to reproduce the pretrained model’s published evaluation behavior, use weights.transforms() rather than reconstructing that contract by hand.

If this model has been fine-tuned under a deliberately modified preprocessing regime, the difference is not automatically a bug. It is an experimental choice whose validation and inference pipelines must agree and whose effect should be measured.

Do the two pipelines end at a compatible representation? After the scale bug is fixed, yes: both produce [3, 224, 224] float32 normalized with the same statistics. Before the fix, no — training sees values around [98, 998] and validation sees [-2.1, 2.6], so the validation metric is measuring the model on inputs it was never trained on. That is checkable with a two-line assertion on one batch from each loader.

Is scale=(0.8, 1.0) a safe augmentation here? Unknown from the code, and it needs a distribution check rather than an opinion. Run the crop several hundred times on a known sample and count how often the arrow head survives. This chapter found a case where scale=(0.9, 1.0) collapsed to a single deterministic crop, so also inspect the sampled parameters, not just the outputs.

If the images were masks or boxes rather than class labels, what else breaks? The geometric transforms would have to be applied to the target with the same sampled parameters. Calling the transform separately on image and target produces disagreement about half the time under p=0.5. The repair is to wrap the target in the appropriate tv_tensors type and pass both objects to one transform call.

The model trains to 99% and production predictions are nonsense. Where do you look first? Not the model. Compare the inference preprocessing against the training contract, field by field: channel order, resize policy, value scale, normalization statistics. The chapter’s BGR experiment produced identical shape, dtype and value range with accuracy going from 1.000 to 0.000 — so the comparison has to be of the contract, not of the tensor metadata.

What evidence proves a preprocessing repair worked? Not that the loss fell. A stage report whose every row matches the intended contract, a validation batch and a training batch agreeing on the final representation, a probe confirming the target still describes the transformed input, and end-to-end accuracy on a held-out set that was never used to fit any preprocessing state.

Exercises

These are written to be run, and they map onto the notebook that accompanies this chapter.

1. Trace and diverge. Build the stage reporter and run it on a four-stage pipeline. Then introduce each of these one at a time and record which row first diverges: dropping scale=True, normalizing twice, resizing after normalizing, and converting to float before resizing. For each, say whether the failure is structural, numerical, semantic or relational.

2. Verify the scaling. Confirm that ToDtype(torch.float32, scale=True) on uint8 is exactly division by 255, then find out what it does to int16 and to a float32 input that is already in [0, 1]. Predict each before running.

3. Two orders, one shape. Construct a pair of pipelines that produce identical shape and dtype but different content, and a second pair that produce identical content. Explain what distinguishes the two cases.

4. Build a semantic probe. Write arrow_direction, verify it on known left and right arrows, then use it as a test: assert that a proposed augmentation preserves direction. Find one augmentation it passes and one it fails.

5. Distribution, not sample. For a stochastic transform of your choice, run 500 draws on a known sample and report the fraction that satisfies your contract. Then inspect the sampled parameters and find a setting where the transform is effectively deterministic.

6. Break the pairing. Transform an image and its mask with two independent calls and measure the disagreement rate. Repair with tv_tensors and confirm it goes to zero. Then do the same with bounding boxes and verify the coordinates against hand-computed mirrors.

7. Fit on the wrong split. Compute normalization statistics from train-only and from train-plus-validation on data where the two splits differ. Report the shift, then argue what the validation metric now means.

8. Read the weights’ contract. For three pretrained checkpoints, print the preprocessing associated with the weights. Find two that share an architecture and differ in preprocessing. Apply both to one image and measure the maximum absolute difference.

9. Drift a checkpoint. Train a small model on a color-dependent task, save it, and evaluate it under four preprocessing variants: correct, channel-swapped, unscaled, and differently normalized. Record shape, dtype, range and accuracy for each, and identify which mismatches the metadata could have caught.

10. Audit a pipeline you did not write. Take a preprocessing pipeline from a repository. Without changing it, produce a stage report for one sample, state the intended contract for every stage, and identify anything that has no evidence behind it.

Next: inside the model

The representation is now something you can interrogate rather than assume. A transform is a function with a contract covering structure, numeric interpretation, semantics, relationships and provenance. A pipeline is an ordered program in which each stage assumes something about what the previous one produced. And when a pipeline misbehaves, the move is to trace one known sample stage by stage and find the first boundary whose output stops satisfying the contract — then classify what kind of divergence it is, because structural, numerical, semantic and relational failures need different repairs and only the first kind announces itself.

The two failures at the center of this chapter are worth carrying forward as a pair. Dropping scale=True made the numbers mean something else. Flipping the arrow made the numbers mean something else about the label. The first is a representation bug and the second is a relationship bug, and no amount of shape checking finds either.

That completes the path from stored bytes to the model boundary:

raw source → decode → representation → augmentation → normalization → batch → model

Chapter 6 verified that the batch arrives on time. Chapter 7 verified that the tensor inside it means what we intended. Both stop at the same place — the moment model(x) is called.

So we now have much stronger evidence for the tensor at the model boundary: [B, 3, 224, 224], float32, normalized under a contract we can state and test, with axis meaning made explicit and semantic checks applied where metadata alone was insufficient. Chapter 4 built a network that consumed a flat vector of features, and Chapter 5 organized it. Neither of them did anything with the fact that two of those axes are spatial — that pixel (h, w) is adjacent to (h, w+1) in a way that pixel 300 is not adjacent to pixel 301 in a flattened vector.

A convolutional network is the architecture that takes that adjacency seriously, and it does so by transforming the channel and spatial axes in ways that are entirely mechanical and entirely easy to get wrong. The next chapter asks what happens to [B, C, H, W] once it crosses the model boundary: what convolution does to channels and geometry, how output sizes are determined rather than guessed, and why a channel-count error can actually originate in a mistaken spatial or axis interpretation.

Into the network.