A confidence head learns the target you give it

Many models report a confidence next to each prediction: a detector's objectness, a per-point score in a localisation head, a probability attached to a regression. Downstream code often treats that number as "how likely is this prediction to be right" and filters on it. I wanted to know when it actually means that, because the training code for these heads rarely says so. It says what the target is: present or absent, visible or hidden, and sometimes the prediction's own accuracy.

What a cross-entropy head converges to

A confidence head usually outputs a logit $z$, with $c = \sigma(z)$, trained with binary cross-entropy against a target $t \in [0, 1]$:

For a given input $x$, the expected loss over whatever targets that input can carry is minimised where its derivative in $c$ vanishes:

Cross-entropy is a proper scoring rule: a head with enough capacity and data learns the conditional expectation of exactly the target it is given, nothing more and nothing different. So the meaning of the confidence is decided when the target is chosen:

• if $t = 1$ whenever the object exists, the head estimates the probability that it exists;
• if $t = 1$ when it can be seen, the head estimates the probability that it is visible;
• if $t$ is a measure of how accurate the prediction is, the head estimates the expected accuracy.

Only the last is "how likely is this prediction to be right". The other two answer different questions, and the derivation says they will answer them faithfully.

The experiment

I wanted a case where those three questions have different answers. Each 32×32 image holds one Gaussian dot of random brightness, from faint to bright, in noise. A small convolutional network regresses the dot's position and outputs a confidence logit. In 40% of images the dot has been removed, but the label still records where it was, as it would for a point that exists but is hidden. The location loss uses every image. I trained the confidence head three ways, three seeds each, for 4,000 steps:

1. present: target 1 for every image, since every image's point exists;
2. visible: target 1 if the dot was drawn, 0 if it was removed;
3. quality: target $\exp(-e^2 / 2s^2)$, where $e$ is the network's current location error and $s = 1.5$ pixels, with no gradient flowing through the target.

I counted a prediction as good when its error was under 2 pixels. I measured how well each confidence separates good predictions from bad, as the area under the ROC curve, and how well it works as a probability of being good, as the expected calibration error over ten bins (Guo et al., 2017¹).

The location errors are the same in all three rows. The network places a visible dot to within a pixel, and for a removed one it guesses near the middle of the image and misses by almost 10 pixels. Only the confidence changes, and each row does what the derivation said.

Confidently wrong, by design

The present target is the trap, and it isn't a bug. Every image's point exists, so the expected target is 1 everywhere, and the head learnt 0.9999 everywhere: on the visible dots and on the removed ones, where the location is a guess. A filter at any threshold keeps all of them. Its calibration error of 0.53 is close to the worst possible here, because about 47% of all predictions are good and the head claims all of them are.

This is how a model can be "confidently wrong" without anything being broken. The head is calibrated for the question it was asked, whether the point exists, and it answers that one perfectly. Consumers who read it as "is the location right" are reading a different question into it.

Visible is close, quality is the right question

The visibility target separates the two populations sharply, 0.83 against 0.25, and ranks good against bad predictions nearly as well as the quality target. It even ranks within the visible dots, at 0.97, which I didn't expect. The reason is that a faint visible dot looks a little like a removed one, so its visibility confidence is lower, and faint dots are also the ones located with larger errors. Visibility and accuracy are correlated here, so a visibility head carries a good deal of accuracy information.

It is still answering the wrong question, and that shows in the calibration. Visibility is a different probability from "the error is under 2 pixels", so the visibility confidence is off by 0.13 on average. The quality target is the only one whose confidence can be read as an estimate of being right. Its calibration error is 0.04, and its confidence on removed dots, 0.05, says plainly that those predictions are guesses.

Detection research has moved the same way. Wu et al. (2019)² add a branch that predicts each box's overlap with the ground truth and multiply it into the classification score, because classification confidence alone correlated poorly with localisation accuracy. Li et al. (2020)³ go further and train a single score whose target is the localisation quality itself, so that the number used for ranking is the one that measures accuracy.

What I would check in a model's confidence

What I take from this, as questions to ask of any confidence output before trusting it:

• What exactly is its training target? That target's conditional expectation is what the head reports.
• On cases known to be wrong, such as removed, hidden or out-of-range inputs, what confidence does it give? A head trained on presence will give them full confidence.
• How well does it rank errors, and how well is it calibrated? These are separate properties, and a target can be good at one and poor at the other.

The same expectation explains what happens when labels are missing: a score trained on incomplete labels learns the labelling rate, in unlabelled is not negative. A head that predicts a full distribution instead of one confidence has its own trade-offs, in a model that learns its own uncertainty chooses what to fit.

What I still don't know

The quality target depends on the network's own current error, so it moves during training, and early in training it teaches the head that everything is bad. That worked smoothly here; I don't know how it behaves when the location head itself is unstable, or when the quality measure is noisy. I also chose the 2-pixel threshold and the 1.5-pixel scale by hand. The calibration error depends on that pairing, and a quality target built for one threshold will look less calibrated against another. And all three heads were trained on the same images they are evaluated on in kind: with removed points in the training data. A confidence head that never saw a hidden case in training has no reason to lower its confidence on one, and that may be the more common situation in practice.

References

1. On Calibration of Modern Neural Networks — Chuan Guo, Geoff Pleiss, Yu Sun and Kilian Q. Weinberger, ICML 2017
2. IoU-aware Single-stage Object Detector for Accurate Localization — Shengkai Wu, Xiaoping Li and Xinggang Wang, arXiv 2019
3. Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection — Xiang Li, Wenhai Wang, Lijun Wu, Shuo Chen, Xiaolin Hu, Jun Li, Jinhui Tang and Jian Yang, NeurIPS 2020