6.15 Object detection and segmentation

What this is and why it exists

Detection and segmentation move the question from "what is in this image" to "where, and how many". The models are interesting; the evaluation machinery is what people underestimate. A detection score has more moving parts than accuracy, and a wrong overlap threshold or a broken duplicate-removal step invalidates a comparison without producing any visible error. So this topic gives the evaluation stack the same weight as the architectures.

The vocabulary

• Bounding box — a rectangle around an object, with a class and a confidence.
• Intersection over union — the overlap of two boxes divided by their combined area.
• Non-maximum suppression — removing duplicate boxes that describe the same object.
• Mean average precision — the standard summary score for detection.
• Anchor — a prior box shape a detector adjusts rather than predicting from nothing.
• Semantic segmentation — assigning a class to every pixel.
• Instance segmentation — separating individual objects, each with its own mask.
• Promptable segmentation — segmenting whatever is indicated, without category training.

The mental model

Start with the measure, because everything else is defined in terms of it. Intersection over union compares a predicted box with a true one: the area they share, divided by the area they cover together. It is one when they coincide and zero when they do not touch, and a threshold on it decides whether a prediction counts as a hit. That threshold is a choice, and it is the first thing to check when two results disagree — a detector evaluated at a lenient threshold and one evaluated at a strict threshold are not comparable, and the strict setting is much harder because it demands accurate box edges rather than roughly the right region.

Non-maximum suppression is the step that makes raw output usable. A detector proposes many overlapping boxes for the same object. Suppression sorts them by confidence, keeps the best, discards everything overlapping it beyond a threshold, and repeats. Its own threshold is a real trade: too strict and genuinely distinct nearby objects are merged into one; too lenient and every object is reported several times. Crowded scenes are where this hurts, and a detector that seems to miss objects in crowds is frequently suppressing them rather than failing to see them.

Mean average precision assembles those pieces. For one class, sweep the confidence threshold from high to low, computing precision and recall at each point, and take the area under that curve — the average precision. Average it over classes to get the mean. It rewards both finding objects and ranking confident correct detections above uncertain ones, which is the right thing to measure. It also has three settings that must match before any comparison means anything: the overlap threshold or range of thresholds, the suppression threshold, and how many detections per image are permitted. State them alongside the number, always.

The two architectural families are a trade that has narrowed. Two-stage detectors first propose regions that might contain something, then classify and refine each proposal — accurate, and slower for the obvious reason. One-stage detectors predict class and box directly across the image in a single pass, which is far faster and was initially less accurate. The gap closed substantially once the class-imbalance problem was addressed: in a single pass, the overwhelming majority of predicted locations are background, and their combined loss drowns the few that matter, which is exactly the situation the focal loss from the loss topic was designed for. For most applied work a modern one-stage detector is the right starting point, and the choice should be driven by the latency you need rather than by a leaderboard.

Segmentation splits into two tasks that are frequently confused. Semantic segmentation labels every pixel with a class and does not distinguish between two adjacent objects of the same class — three people become one region of "person". Instance segmentation separates them, giving each its own mask. Decide which your problem needs before choosing an architecture, because counting objects requires instances and measuring coverage does not.

The architectures follow from the shapes. Semantic segmentation is usually an encoder that reduces spatial size while accumulating meaning, followed by a decoder that restores resolution, with skip connections carrying fine detail across from the encoder to the decoder — because the deep layers know what things are and the shallow layers know exactly where the edges were, and you need both. That design came from biomedical imaging and remains the standard shape. Instance segmentation is commonly detection plus a mask prediction per detected object, which is why the two-stage family adapts to it naturally.

Promptable segmentation changed what a project starts from. Models trained on very large and diverse mask data can segment whatever you indicate — a point, a box, a rough region — without ever having been trained on your categories. The consequence for practice is significant: the annotation step, historically the expensive part of a segmentation project, can become an assisted step where a person indicates objects and corrects the masks rather than drawing them. Reach for one of these before deciding to label from nothing, and remember that they segment what you point at without naming it — the classification remains yours.

Two practical notes to close. Detection and segmentation annotation is expensive and error-prone, and label quality dominates model choice at small scale; a modest detector on carefully checked boxes beats a strong one on sloppy ones. And always look at the failure cases rather than only the score — the pattern in what a detector misses, whether it is small objects, crowds, or one particular class, tells you what to do next in a way the summary number never will.

What you should now be able to explain or do

Define intersection over union and say what the threshold decides. Explain non-maximum suppression and the trade in its threshold. Describe how mean average precision is built and name the three settings that must match for a comparison to mean anything. State the trade between two-stage and one-stage detectors and why the gap narrowed. Distinguish semantic from instance segmentation and choose by what the problem needs. Explain why segmentation architectures use skip connections between encoder and decoder. Say what promptable segmentation changes about starting a project.

Check yourself