Detection is classification plus regression, run at every position of a feature map, then cleaned up. Divide the image into a grid, let every cell predict boxes, objectness and class scores, then delete the duplicates with non-maximum suppression. This lesson is every number in that tensor — the IoU arithmetic, the three-part loss, the anchor pyramid, and the metric row that tells you which knob to turn.
The image becomes a grid; every cell carries B prior boxes. Under each prior the head writes 4 box numbers, 1 objectness score and C class scores, so one feature map turns into thousands of candidate detections. Nothing in the network loops over objects — density is the trick that makes it real time.
per anchor 5 + C = 25 · per cell B·(5 + C) = 75 · map 13×13×75 = 12,675 numbersIoU is the only similarity metric detection needs: intersection area over union area. Predictions above the threshold are true positives; predictions overlapping a better-scoring prediction are duplicates. NMS keeps the top box, deletes the rest of its crowd, and repeats until nothing overlaps.
IoU(A,B) = 81/119 = 0.681 → suppressed · IoU(A,C) = 64/136 = 0.471 survives at 0.50Only the cells that own an object contribute box regression and classification loss; every empty cell contributes objectness loss so the model learns to stay quiet. The weights keep the small MSE box term and the large majority of empty cells from drowning the signal.
λ_coord 5 · λ_obj 1 · λ_noobj 0.5 · λ_cls 1 (YOLOv1's ratios)Detection is classification plus regression at every position: the grid decides who is responsible, the anchor supplies the prior shape, the loss teaches objectness and class together, and NMS turns a crowd of guesses into one box per object.
By the end you will be able to read a head tensor and say what every number means (S × S × B × (5 + C)); compute IoU by hand and explain why containment scores badly; run NMS from scratch and explain the 0.45 threshold; write the three-part loss with its weights and say what each one teaches; place anchors on a pyramid of 13×13, 26×26 and 52×52 grids; and read precision@0.5, recall, mAP@0.5 and mAP@0.5:0.95 to name the single most useful next experiment.
A classifier says “this image is a dog.” A detector says “there is a dog at pixels (244, 32, 364, 272), a person at (188, 222, 228, 322), and nothing else in the frame.” That one structural change — a variable number of labelled boxes instead of one label per image — is what every autonomous system, surveillance product, document layout parser and factory vision line is built on.
The jump is bigger than it sounds. Classification has a fixed output: C numbers, one per class, for every image. Detection has to answer four questions at once, for an unknown number of objects:
YOLO’s answer (Redmon et al., 2016) was to make all four questions part of one dense prediction: run a single forward pass over the image, and let every position in the final feature map predict boxes. The output is a tensor:
Why a grid? Plain regression — predict (x, y, w, h) for every object as absolute coordinates — asks the network to translate the entire output whenever the image shifts. The grid fixes that: each ground-truth box is assigned to the cell whose patch contains its centre, and only that cell is responsible for that object. The network never has to move a prediction across the whole image, only to nudge it inside its own 32-pixel cell.
Under every anchor in every cell, the five base numbers are exactly the four geometry targets plus objectness, and the C numbers are class logits. That is the whole tensor. The lab below lets you click a cell, read its 75 numbers, and watch the decode arithmetic produce a box.
import torch
from torchvision.models import resnet18
backbone = resnet18(weights="DEFAULT")
feature = torch.randn(1, 512, 13, 13) # what the trunk hands the head
head = torch.nn.Conv2d(512, 3 * (5 + 20), kernel_size=1)
raw = head(feature) # (1, 75, 13, 13)
raw = raw.view(1, 3, 5 + 20, 13, 13) # (N, anchors, 5+C, H, W)
raw = raw.permute(0, 3, 4, 1, 2) # (N, H, W, anchors, 5+C)
print(raw.shape) # torch.Size([1, 13, 13, 3, 25])
# per anchor: tx, ty, tw, th, obj, then 20 class logits
print(raw[0, 4, 9, 1, :5]) # cell (gx=9, gy=4), anchor 1This is a whole detection head’s output: 3 anchors per cell, and under each anchor 4 box targets, 1 objectness, and 20 class logits. Click a grid cell (or use the sliders) and read its 75 numbers.
The three groups in the strip are the three pieces of every YOLO head: geometry (tx, ty, tw, th), “is anything here?” (obj), and “what is it?” (the 20 class logits). The reportable score is the product of the last two.
From v2 onward each anchor owns its class scores: 4 box targets + 1 objectness + C logits = 5 + C, so a cell with B anchors holds B × (5 + C) numbers. YOLOv1 grouped differently: class probabilities were predicted once per cell and shared by both box slots, so depth was B × 5 + C = 2 × 5 + 20 = 30. That is where the famous 7 × 7 × 30 = 1,470 comes from. Modern detectors moved the classes under each anchor because a cell can contain two objects of different classes; the arithmetic in this lesson’s labs uses the modern grouping, and the metric section never depends on either.
A head outputs (N, 13, 13, 3, 25). A friend says the 25 is '20 classes plus a 5-pixel margin'. What are the 25 numbers really?
Before anything is graded, detection needs a way to say how much two boxes agree. There is exactly one metric the whole field uses, and it is one division: the area of the overlap over the area of the union.
Two boxes in corner format (x1, y1, x2, y2). The intersection rectangle is easy to compute with min and max: its left edge is the larger of the two left edges, its right edge is the smaller of the two right edges (and a negative width or height means the boxes miss, so clamp to zero). Then
That is the whole metric. It has two jobs in the pipeline. First, it is the grading rule: a prediction counts as a true positive only if it reaches an IoU threshold with a ground-truth box — 0.50 for the classic PASCAL VOC numbers, 0.75 when you want to be strict. Second, it is the deduplication rule inside NMS: two predictions of the same object overlap heavily, so one absorbs the other.
Notice what IoU is not: it is not a measure of being correct in any semantic sense. It cannot tell a well-placed wrong class from a right class, and it punishes containment brutally. A 80×80 box completely inside a 360×340 box is 100% covered and still scores 6,400 / 122,400 = 0.052 — because the union keeps the big box’s whole area in the denominator. That is precisely why detectors are trained to be tight, not just overlapping.
import numpy as np
def box_iou(boxes_a, boxes_b):
"""(N_a, 4) x (N_b, 4) in (x1, y1, x2, y2) -> (N_a, N_b) IoU matrix."""
ax1, ay1, ax2, ay2 = boxes_a[:, 0], boxes_a[:, 1], boxes_a[:, 2], boxes_a[:, 3]
bx1, by1, bx2, by2 = boxes_b[:, 0], boxes_b[:, 1], boxes_b[:, 2], boxes_b[:, 3]
inter_x1 = np.maximum(ax1[:, None], bx1[None, :]) # right-most left edge
inter_y1 = np.maximum(ay1[:, None], by1[None, :])
inter_x2 = np.minimum(ax2[:, None], bx2[None, :]) # left-most right edge
inter_y2 = np.minimum(ay2[:, None], by2[None, :])
inter_w = np.clip(inter_x2 - inter_x1, 0, None) # no overlap -> 0
inter_h = np.clip(inter_y2 - inter_y1, 0, None)
inter = inter_w * inter_h
area_a = (ax2 - ax1) * (ay2 - ay1)
area_b = (bx2 - bx1) * (by2 - by1)
union = area_a[:, None] + area_b[None, :] - inter
return inter / np.clip(union, 1e-8, None)
# checks the lab reproduces live
assert box_iou(np.array([[10, 10, 50, 50]]), np.array([[10, 10, 50, 50]]))[0, 0] == 1.0
assert abs(box_iou(np.array([[0, 0, 10, 10]]), np.array([[5, 0, 15, 10]]))[0, 0] - 1/3) < 1e-9Drag either box on the canvas, or select one and use the sliders and nudge buttons. The shaded region is the intersection; the readout divides it by the union — the one similarity metric detection runs on.
The trap to remember: a small box completely inside a big one still scores low, because the union keeps the big box’s whole area. IoU measures agreement about coverage, not correctness.
One last habit from the source: implement box_iou once, vectorized, and use it everywhere — training (anchor assignment), inference (NMS), and evaluation (TP/FP decisions). The code is small enough to hold in your head, and every later debugging session will ask it the same question: “how much do these two boxes agree?”
A convolutional detector answers at every position, so an object usually attracts several boxes: the anchor it was assigned to, plus neighbours that saw it too. NMS keeps the best one and deletes the rest — a dozen lines of greedy logic that every real-time detector still runs.
Two boxes are “the same detection” when their IoU is high, so the algorithm is as simple as that rule allows:
It is deterministic, O(N log N) from the sort, and it matches torchvision.ops.nms exactly on identical inputs. Now the arithmetic on the source’s own five boxes, in the 10-pixel world of main.py:
| # | box | score | IoU vs the pick | verdict at 0.40 |
|---|---|---|---|---|
| 0 | (0, 0, 10, 10) | 0.90 | — | ✓ keep (first pick) |
| 1 | (1, 1, 11, 11) | 0.80 | 81 / 119 = 0.681 | ✕ suppressed |
| 2 | (2, 2, 12, 12) | 0.70 | 64 / 136 = 0.471 | ✕ suppressed |
| 3 | (20, 20, 30, 30) | 0.85 | 0 | ✓ keep (second pick) |
| 4 | (21, 21, 31, 31) | 0.60 | 81 / 119 = 0.681 | ✕ suppressed by box 3 |
Sorted by score the order is 0, 3, 1, 2, 4. Box 0 (0.90) absorbs boxes 1 and 2, then box 3 (0.85) absorbs box 4. The survivors are [0, 3] — exactly what the source’s main.py prints. Now move the threshold to 0.50 and the third box comes back: its IoU with the winner is 0.471, which the strict gate refuses to call a duplicate. This is why the production band is narrow — 0.45 to 0.50 — and why the threshold is a tuning knob, not a constant.
def nms(boxes, scores, iou_threshold=0.45):
order = np.argsort(-scores) # highest score first
keep = []
while len(order) > 0:
i = order[0]
keep.append(int(i))
if len(order) == 1:
break
rest = order[1:]
ious = box_iou(boxes[[i]], boxes[rest])[0]
order = rest[ious <= iou_threshold] # drop the ones that overlap too much
return np.array(keep, dtype=np.int64)
# the source's five-box check:
boxes = np.array([[0,0,10,10],[1,1,11,11],[2,2,12,12],[20,20,30,30],[21,21,31,31]], float)
scores = np.array([0.9, 0.8, 0.7, 0.85, 0.6])
print(nms(boxes, scores, 0.40)) # [0 3]
print(nms(boxes, scores, 0.50)) # [0 3 2] <- box 2 survives 0.471 <= 0.50The detector predicted three overlapping dog boxes, two cat boxes and one small bird. Step through the greedy algorithm one pick at a time and watch each suppressed box get its real IoU printed on it.
Try the threshold at 0.80: the duplicate dog boxes survive (IoU 0.757), and you report the same dog three times. Try 0.20: only the strongest box per group survives. NMS trades duplicates against missed objects — 0.45 is the industry’s compromise.
The head is almost insultingly small: a single 1×1 convolution that writes B × (5 + C) numbers at every position of the feature map. The engineering is in the target assignment and in keeping the three loss terms — box, objectness, class — balanced while they train at once.
A 1×1 convolution is a per-pixel linear layer: it mixes the backbone’s 512 feature channels at one position into 75 numbers, and the same weights are applied at all 169 positions of the 13×13 map. That weight sharing is the reason dense prediction is affordable, and the reshape is the only other thing to get right. In practice nobody trains the trunk from scratch either: you start from an ImageNet-pretrained backbone, attach this head, and fine-tune both with a small detection learning rate — the transfer-learning move from the previous lesson, pointed at boxes instead of labels.
class YOLOHead(nn.Module):
def __init__(self, in_c, num_anchors, num_classes):
super().__init__()
self.num_anchors = num_anchors
self.num_classes = num_classes
# out channels = B * (5 + C): for B=3, C=20 that is 75
self.conv = nn.Conv2d(in_c, num_anchors * (5 + num_classes), kernel_size=1)
def forward(self, x):
n, _, h, w = x.shape
y = self.conv(x) # (N, 75, 13, 13)
y = y.view(n, self.num_anchors, 5 + self.num_classes, h, w)
y = y.permute(0, 3, 4, 1, 2).contiguous() # (N, 13, 13, 3, 25)
return yDecoding. The raw numbers are not pixels. Offsets pass through a sigmoid so the centre lands inside its own cell; sizes pass through exp so the anchor can scale up or down without a sign flip; stride converts grid units back to pixels:
def sigmoid(x):
return 1.0 / (1.0 + np.exp(-x))
def decode(tx, ty, tw, th, cell_x, cell_y, stride, anchor_wh):
cx = (sigmoid(tx) + cell_x) * stride # centre, inside the cell
cy = (sigmoid(ty) + cell_y) * stride
w = anchor_wh[0] * np.exp(tw) # size, scaled from the anchor
h = anchor_wh[1] * np.exp(th)
return np.array([cx - w/2, cy - h/2, cx + w/2, cy + h/2])
# the tensor-anatomy lab's dog, decoded by hand:
# tx 0.000 -> sigmoid 0.500 -> cx = (0.500 + 9) * 32 = 304.0 px
# ty 1.099 -> sigmoid 0.750 -> cy = (0.750 + 4) * 32 = 152.0 px
# tw 0.470 -> 75 * exp(0.470) = 120.0 px
# th 0.345 -> 170 * exp(0.345) = 240.0 px -> box (244, 32, 364, 272)
def encode(box_xyxy, cell_x, cell_y, stride, anchor_wh):
# store the offset as a logit so decode(sigmoid(.)) round-trips exactly
x1, y1, x2, y2 = box_xyxy
cx, cy = 0.5 * (x1 + x2), 0.5 * (y1 + y2)
off_x = cx / stride - cell_x # 304 / 32 - 9 = 0.5
off_y = cy / stride - cell_y # 152 / 32 - 4 = 0.75
tx = np.log(off_x / (1 - off_x)) # logit(0.5) = 0.000
ty = np.log(off_y / (1 - off_y)) # logit(0.75) = 1.099
tw = np.log((x2 - x1) / anchor_wh[0]) # ln(120 / 75) = 0.470
th = np.log((y2 - y1) / anchor_wh[1]) # ln(240 / 170) = 0.345
return np.array([tx, ty, tw, th])Assignment. Training needs to know which of the thousands of output slots is responsible for each ground-truth object. The rule is cheap and geometric: the owning cell is the one containing the box centre, and the owning anchor is the one whose shape best matches the box (highest min/max area overlap — not the coordinate IoU, since the anchor has no position). Everything else in the tensor is trained toward zero objectness. YOLOv1, which had no anchors, made the box with the highest IoU against the ground truth responsible instead; same idea, different tie-breaker.
The loss. Three terms, added with weights. Only the assigned slots contribute box regression and classification; every other slot contributes objectness — teaching the model to say “nothing here”:
def yolo_loss(pred, target, has_obj,
lambda_coord=5.0, lambda_obj=1.0, lambda_noobj=0.5, lambda_cls=1.0):
o = has_obj.bool()
# 1. box regression — ONLY where an object was assigned
loss_box = F.mse_loss(pred[..., :4][o], target[..., :4][o], reduction="sum")
# 2. objectness — positives and negatives weighted differently
loss_obj_pos = F.binary_cross_entropy_with_logits(
pred[..., 4][o], target[..., 4][o], reduction="sum")
loss_obj_neg = F.binary_cross_entropy_with_logits(
pred[..., 4][~o], target[..., 4][~o], reduction="sum")
# 3. classification — ONLY where an object was assigned
loss_cls = F.binary_cross_entropy_with_logits(
pred[..., 5:][o], target[..., 5:][o], reduction="sum")
total = (lambda_coord * loss_box + lambda_obj * loss_obj_pos
+ lambda_noobj * loss_obj_neg + lambda_cls * loss_cls)
return total, {"box": loss_box.item(), "obj_pos": loss_obj_pos.item(),
"obj_neg": loss_obj_neg.item(), "cls": loss_cls.item()}Why those weights? Arithmetic, not taste. Take YOLOv1’s 7×7 grid with B = 2: that is 98 slot predictions, and a typical image has one to five objects — call it 97 empty slots against 1 object. An empty slot sitting at 50% objectness contributes −ln 0.5 = 0.693 of cross-entropy; summed over 97 slots that is 67.2 of pure “predict nothing” pressure. λ_noobj = 0.5 halves it to 33.6 so the one positive slot’s signal can still be heard. Go the other way: a box-coordinate squared error of 0.05 per coordinate over four coordinates is 0.2, next to a class term of 0.693 — so the box gradient would be 3× weaker if λ_coord were 1. At λ_coord = 5 it contributes 1.0 and stays comparable. Both ratios are gradient-scale matching.
| Term | Weight | Applied to | What breaks without it |
|---|---|---|---|
| Box regression | λ_coord = 5 | assigned slots only | boxes drift, mAP@0.5:0.95 collapses, AP@0.5 survives |
| Objectness (positive) | λ_obj = 1 | assigned slots only | the model stops trusting its own hits; recall falls |
| Objectness (negative) | λ_noobj = 0.5 | every empty slot | hallucinated boxes everywhere: the majority class wins |
| Classification | λ_cls = 1 | assigned slots only | boxes are found but every label is wrong or ambiguous |
import torch
from torchvision.models.detection import fasterrcnn_resnet50_fpn_v2
model = fasterrcnn_resnet50_fpn_v2(weights="DEFAULT").eval()
with torch.no_grad():
prediction = model([torch.randn(3, 400, 600)])[0]
print(prediction["boxes"].shape) # (N, 4) in (x1, y1, x2, y2) pixels
print(prediction["labels"].shape) # (N,) class index per box
print(prediction["scores"].shape) # (N,) confidence per box
# The production standard is ultralytics: decoding and NMS live inside model()
# from ultralytics import YOLO
# results = YOLO("yolov8n.pt")(image) # -> boxes, scores, labels againThe structure is unchanged in every YOLO since: assign, regress, classify, suppress. The pieces get upgraded. MSE box loss treats a 10-pixel error identically on a 20-pixel and a 200-pixel object, so modern heads use CIoU/DIoU, which optimize the IoU itself (and add centre-distance and aspect-ratio terms). Class imbalance is handled with focal loss or BCE with quality focal loss; objectness is sometimes merged with the class score into a single quality score (YOLOv5/YOLOv8 use separate objectness and class outputs with a task-aligned assignment instead). What a beginner should keep: three signals, different populations of cells, different weights.
The training target stores the centre offset as logit(offset) — e.g. logit(0.75) = 1.099 for a centre three-quarters of the way across its cell. Why not store 0.75 directly?
Grids solve “where”, not “how big”. Asking one 32-pixel cell to invent a 500-pixel box from nothing is the second hard problem in detection — and the answer is to stop predicting sizes from scratch and start predicting small corrections to prior shapes.
Anchors (also called priors or default boxes) are pre-defined box shapes attached to every cell. Instead of outputting a width, the head outputs tw, which is a log ratio to the anchor’s width: w = anchor_w · exp(tw). A model that has learned nothing sits at tw = 0, which means “the anchor, unchanged” — a far better starting point than a random number on an arbitrary coordinate scale.
The arithmetic of why the prior matters, concretely. A 120×240 dog against a 75×170 anchor is tw = ln(120/75) = 0.470, th = ln(240/170) = 0.345 — small numbers near zero. The same dog against a 30×60 anchor (the small-object prior) needs tw = ln(4) = 1.386: the same correction, expressed as a ratio, is 2.95× larger and therefore a 2.95× larger gradient on a network that is trying to learn everything at once. Give the object a prior with the right shape and the regression problem becomes small.
Modern detectors do this at several resolutions at once. YOLOv3 is the cleanest example: the same 416×416 input is predicted at three grid sizes, each with its own three anchors.
Those are the real anchors YOLOv3 derived from COCO with k-means — nine shapes, three per level. The source’s simpler teaching set, (30, 60), (75, 170), (200, 380), is the one the tensor-anatomy lab writes into its single-level head: same idea, one scale instead of three.
Only one (cell, anchor) slot per object is trained as positive: the cell whose patch contains the object’s centre, and within that cell the anchor with the best shape overlap. A small bird goes to the fine map because that is where the 30×61 anchor lives; a bus goes to the coarse map. The lab below makes that assignment visible and lets you move a bird, a dog, a person and a wide truck through all three levels.
Every cell carries the same three prior boxes. The cell whose patch contains the object’s centre owns it; the anchor whose shape best matches the object is the one the head nudges. Switch levels and watch small objects move to the dense map.
This is why scales matter: a 36×56 bird is 0.38 shape-IoU against its best P3 anchor and 0.78 against the medium map’s (30×61). Each level only ever nudges the shapes it was given, so the anchor set decides what the head can learn to see.
At stride 32, one cell covers 32 × 32 = 1,024 px². A 500 × 300 object covers 150,000 px² — 146 cell areas. Its centre can sit in one cell, but the box reaches about 15 cell widths to the left and right of that cell. The head’s 1×1 convolution at that position sees a single 32-pixel patch of the picture; everything it knows about the object’s extent has to come from the prior it was given plus the numbers it writes. This is why small objects are hard in the other direction too: at 52×52 a 36 × 56 bird is 4–7 cells wide, so it is representable — but at 13×13 it is barely one cell, and the objectness signal competes with everything else in that cell. The pyramid exists to give both ends somewhere to live.
A detector can be 90% accurate in the classification sense and useless, because the things it gets wrong are the only ones that matter: a missed object, a duplicate, or a box that is in the right place but three times too large. Detection replaces accuracy with a small vocabulary built on the IoU gate.
The first move is to define “correct”. A prediction is a true positive if it matches a ground-truth object of the same class with IoU ≥ τ, and each ground-truth object can absorb only one prediction — everything else is a false positive. Objects that absorb nothing are misses. From that one rule:
The area under the curve part is worth doing by hand once. The lab below runs a worked evaluation set: 6 objects, 8 predictions, one of which is a background box scoring higher than three real hits. At IoU 0.50 the five hits give recall steps of 1/6 each, and the precision envelope — the best precision achievable at that recall or beyond — is 1.000 for the first two steps and 0.833 for the last three, because the background box drags raw precision down to 0.667 but a later hit lifts it back:
Read the two headline numbers together and the diagnosis writes itself: 0.750 at 0.50 but 0.402 over the sweep means the detector finds objects and localises them loosely. If instead precision is 0.85 and recall is 0.40, the detector is careful and timid; if precision is 0.30 and recall is 0.90, it is firing at everything. The lab’s readout names the next experiment from those two shapes.
A worked evaluation set. Move the score threshold to choose the operating point; move the IoU threshold to make the true-positive rule stricter. AP is the area under the precision-recall curve — it does not move when you slide the score gate, only when you change what counts as a hit.
| rank | score | matched | best IoU | at τ | reported |
|---|---|---|---|---|---|
| 1 | 0.96 | object 1 | 0.91 | TP | yes |
| 2 | 0.91 | object 2 | 0.82 | TP | yes |
| 3 | 0.87 | — | 0.21 | FP | yes |
| 4 | 0.84 | object 3 | 0.71 | TP | yes |
| 5 | 0.77 | object 4 | 0.63 | TP | yes |
| 6 | 0.69 | object 5 | 0.55 | TP | yes |
| 7 | 0.61 | object 6 | 0.44 | FP | yes |
| 8 | 0.52 | — | 0.47 | FP | yes |
Note what the two sliders do. The score threshold only selects which predictions you report — AP itself does not move when you slide it. The IoU threshold changes what counts as a hit, so it moves every AP row: that is why mAP@0.5:0.95 is the strict metric.
A detector scores mAP@0.5 = 0.83 and mAP@0.5:0.95 = 0.46. Which single change is most likely to help the strict number, with almost no effect on the lenient one?
A detection run costs hours. The difference between a wasted night and a useful one is usually which single thing you changed — and the metric row, read as four symptoms instead of one score, tells you which that is.
The source’s guidance is a lookup table, and it is worth memorizing because it covers most real debugging sessions:
| What you see | What it means | The one change to try |
|---|---|---|
| mAP@0.5 high, mAP@0.5:0.95 low | objects found, boxes loose | box loss (MSE → CIoU/DIoU), finer head resolution, anchors tuned to the dataset’s shapes |
| mAP@0.5:0.95 stuck near the 0.50 row | localisation barely good enough for the lenient gate | same as above — this is the same symptom at its worst |
| precision high, recall low | conservative: real objects are being missed or gated out | lower the confidence threshold, raise λ_obj, check small-object coverage in the anchors |
| precision low, recall high | promiscuous: background and duplicates get through | raise the confidence threshold, tighten NMS (lower the IoU threshold), raise λ_noobj |
| everything low, both splits | underfitting or broken pipeline | more data / more epochs / verify labels and input normalization before touching the architecture |
| training great, held-out bad | memorization | more data, augmentation, weight decay — and audit for duplicate frames across splits |
The four physical knobs, in the order of how much they usually buy:
The lesson ships two reusable artifacts for exactly this loop. The first is a prompt that turns a metric row into a one-line diagnosis and the single most useful next experiment — the table above, encoded. The second is a skill that, given a dataset of ground-truth boxes, runs k-means on (w, h) and returns anchors per pyramid level with the coverage statistics you need to choose the number of anchors. Both are small; both pay for themselves on the next project. And boxes are where detection stops: the next lesson (Phase 4, Lesson 07 — U-Net) asks the same convolutional trunk for a class at every single pixel instead of four numbers per object.
Your detector reports precision 0.86, recall 0.38, mAP@0.5 = 0.62. Products are being missed on the line. Which change follows from the diagnosis?
Answer before you look. The tensor-depth question and the loss-weight question are the two that separate “I watched a video about YOLO” from “I could implement this tomorrow.”
0 / 6 answered · 0 correct
01A YOLO detector with grid 13×13, 3 anchors per cell, and 20 classes produces a head output of shape (N, 13, 13, 3, 25). Why is the last dimension 25?
02Why does YOLO push tx and ty through a sigmoid before adding the grid cell?
03A detector predicts 15 boxes around one object. NMS with iou_threshold = 0.45 returns a single box. What did NMS do?
04Your model has mAP@0.5 = 0.75 but mAP@0.5:0.95 = 0.40. What is the most accurate interpretation?
05YOLO loss weights λ_coord = 5.0 and λ_noobj = 0.5 mirror the original paper. What do the ratios encode?
06YOLOv1's output tensor for S = 7, B = 2 boxes and C = 20 classes is 7×7×30. Where does the 30 come from?
Click a card to swap the lazy description for what it actually means — each one carries the arithmetic from the labs.
Three problems with exact numbers: verify box_iou against torchvision, swap MSE for CIoU with a worked derivation, and build multi-scale inference with its latency bill. Try first; a worked answer is one click away.
The vectorized version is one line of broadcasting: inter_x1 = np.maximum(ax1[:, None], bx1[None, :]), inter_y1 = np.maximum(ay1[:, None], by1[None, :]), inter_x2 = np.minimum(ax2[:, None], bx2[None, :]), inter_y2 = np.minimum(ay2[:, None], by2[None, :]), then inter = np.clip(inter_x2 − inter_x1, 0, None) × np.clip(inter_y2 − inter_y1, 0, None) and IoU = inter / (area_a[:, None] + area_b[None, :] − inter). Edge case 1: non-overlapping boxes produce negative widths and heights, so the clip to 0 is mandatory — without it the 'intersection' is a positive product of two negatives (e.g. a box left of another gives (x2−x1) < 0 in both axes and a spurious positive area). Edge case 2: identical or zero-area boxes make the union 0; clip the denominator to 1e-8. Edge case 3: coordinate format — both torchvision and this function use (x1, y1, x2, y2) corners, not (x, y, w, h), and mixing them silently produces plausible but wrong numbers. Verification: sample boxes from a 640×480 frame, compare with torchvision.ops.box_iou, and report max|Δ| — expect ~1e-7 from float32 (torchvision) versus float64 (NumPy) rounding, comfortably under 1e-6. Three hand checks worth keeping: identical boxes → 1.000; the source's half-overlap pair (0,0,10,10) and (5,0,15,10) → 50/150 = 0.333; corner-touching boxes → 0.000 exactly.
CIoU = IoU − (ρ²(b_pred, b_gt) / c²) − αv, where ρ² is the squared distance between centres, c² is the squared diagonal of the smallest enclosing box, and v = (4/π²)(arctan(w_gt/h_gt) − arctan(w_pred/h_pred))² with α = v / (1 − IoU + v). Worked numbers with A = (0, 0, 100, 100) and B = (10, 10, 110, 110): intersection 90 × 90 = 8,100; union 10,000 + 10,000 − 8,100 = 11,900; IoU = 0.681. Centre distance² = 10² + 10² = 200; enclosing box (0, 0, 110, 110) → c² = 110² + 110² = 24,200 → distance penalty 0.0083. Aspect ratios are equal (v = 0), so CIoU = 0.681 − 0.008 = 0.672 and the loss is 1 − 0.672 = 0.328, versus an MSE that never sees the 10-pixel offset as a fraction of box scale. Why it should win on tight localisation: MSE on raw (tx, ty, tw, th) weights a 10% size error the same whether the box is 20 px or 200 px wide, while CIoU normalises distance by the enclosing box and adds an aspect-ratio term, so the gradient keeps pushing after the coarse IoU is already satisfied. The experiment: 100 synthetic images of 2–4 rectangles per class, train two identical heads from the same seed for the same 40 epochs, one with loss_box = MSE (λ_coord = 5) and one with loss_box = 1 − CIoU (λ_coord = 1), and report mAP@0.5:0.95 every 5 epochs. Expect the MSE run to plateau first: same mAP@0.5, visibly worse at 0.75+. Report both curves and the final numbers; a claim without the paired run is not a result.
Pipeline: for each scale s ∈ {320, 640, 960} resize the image, run the forward pass, decode the boxes back into original-image pixels (multiply coordinates by original_width / s), and collect all (box, score, class) triples with the scale tagged. Union everything, then run a single class-wise NMS at IoU 0.45 over the combined list — not one NMS per scale, or the duplicates across scales survive. Worked illustration of the accounting: if the three scales produce 80 + 120 + 160 = 360 candidates and NMS keeps 52, the extra 2 candidates over the best single scale are the small-object wins that single-scale misses — the standard outcome, because a 32 px object is 3 cells wide at 960 but a single cell at 320. Latency is where honesty is required: three forward passes means roughly the sum of the three per-image times (illustratively 18 ms + 42 ms + 96 ms = 156 ms versus 42 ms at 640 alone), plus one NMS over 360 boxes (sub-millisecond at this size), so multi-scale buys recall at ~3.7× the inference cost. Measure it, do not assume the lift: report mAP@0.5 and mAP@0.5:0.95 for each single scale, for the union, and the per-stage milliseconds — on a dataset with almost no small objects the union may add nothing but latency.
You do not need to master these here. Each one gets a proper treatment in its own lesson; the one-line meaning is enough to keep reading. Orange dotted underlines in the prose point back to this list.
This lesson is a port of an open course. Everything here traces back to it — and the next step is running the code yourself.
Lesson text adapted from AI Engineering from Scratch (Phase 04, Lesson 06) and the Math Foundations Notebook reference build. The five labs — the IoU playground, the NMS stepper, the anchor-pyramid visualizer, the output-tensor anatomy, and the metric reader — are original to this page, as are the depth arithmetic for both conventions (YOLOv1's 7×7×30 = B·5 + C versus the modern 13×13×75 = B·(5 + C)), the worked NMS trace on the source's five boxes (81/119 = 0.681, 64/136 = 0.471, [0, 3] at 0.40 versus [0, 3, 2] at 0.50), the containment trap, the decode round-trip numbers (sigmoid(0) = 0.500 → cx = (0.500 + 9) × 32 = 304 px), the loss-weight gradient arithmetic (97 empty slots to 1 object at 7×7×2), the real YOLOv3 COCO anchors per pyramid level, the metric-lab evaluation set with its precision-envelope AP arithmetic (2 × ⅙ × 1.000 + 3 × ⅙ × 0.833 = 0.750 versus mAP@0.5:0.95 = 0.402), the three-scores memory hook, and the diagnosis table and decision tree. Every number shown is computed live by the labs or verified by hand in the prose.