A loss cannot fix what the training data never shows

When a model fails in a recognisable way, the natural reflex is to add a loss term that penalises that way of failing. If predicted shapes come out distorted, penalise distortion; if outputs that should be consistent aren't, penalise inconsistency. What I couldn't say was what such a term does when the training data never contains the failure, so I set up the smallest case I could think of and measured it.

Why the term might do nothing, on paper

A loss term contributes gradient only where it is nonzero. If the training data never produces the failure, then on the training data the model already satisfies the constraint as well as the labels allow, and the term sits at a floor set by label noise and the model's ordinary errors. Its gradient pushes the model to be slightly more consistent on inputs it already handles, which is not the same as teaching it what to do on inputs it has never seen. Raising the term's weight scales a gradient that is small and pointed at the wrong problem.

A constraint applied at test time works differently. It doesn't change the model; it corrects the output, using knowledge about the world that holds whether or not the input looks like the training data. If that knowledge is right, it repairs the failure directly.

The experiment

A small convolutional network sees a 48×48 image of a bright bar, with noise, and predicts the positions of both ends. Bars vary in length from 12 to 26 pixels and in position and angle. Training images are always clean. I trained three versions, three seeds each, for 5,000 steps: the plain end-point loss, and the same loss plus a length-consistency term, the absolute difference between predicted and true bar length, at weights 0.1 and 0.5.

The test set is 100 sequences of 30 images. Within a sequence the bar keeps one length while it turns and drifts slowly, as a rigid object would. In 15% of the images, between 45% and 70% of the bar is covered from one end, so the visible part is much shorter than the bar. That never happens in training.

On clean images every version finds the ends to within a quarter of a pixel. On covered images every version places the far end at the end of the visible part, about 10 pixels short, and reports a bar less than half as long as it is. The length term changed none of that. Its own value at the end of training was the same, about 0.33, at every weight: it reached its floor on the clean training data and had nothing more to do there. A five-fold increase in its weight bought nothing on the failure it was meant to address, and cost a little on clean images.

A constraint at test time

The fact the training term was trying to express is that a bar's length doesn't change within a sequence. At test time that can be used directly. For each sequence I took the median of the predicted lengths as a robust estimate of the true one. Any image whose predicted length fell below 80% of that median was treated as collapsed, and its far end was moved along the predicted direction to the median length.

The error on covered images fell from 10.3 pixels to 0.65. The repair caught every covered image and touched 0.7% of the clean ones, without changing their error. The median works because collapsed images are a minority in every sequence; the direction works because the visible part of the bar still points the right way.

When this is the right tool

It depends on knowledge that is true about the world and that the model is not guaranteed to have learnt. Here, rigid bars keep their length. The repair only acts on a sharp drop below the sequence's typical length, so it leaves alone the ordinary variation it can't distinguish from noise. A rule that forced every image to the median length would also have "fixed" any real change in length, and for objects that do change length that would be wrong.

The training-side lesson is broader. Hendrycks and Dietterich (2019)¹ showed how much image classifiers degrade under common corruptions absent from their training data. The usual remedy is to change the training distribution, by adding augmentations that resemble the corruption, so that the failure appears where a loss can see it. I didn't try that here, and it is the obvious next experiment: it attacks the cause, where the test-time constraint attacks the symptom.

What I would check before adding a loss term

These are the questions I would now ask first:

• Does the failure appear in the training data at all? If the term's value is already at its floor on training batches, more weight won't help.
• Is the failure measured on the data where it actually occurs? An in-distribution validation metric can't see an out-of-distribution failure.
• Is there knowledge about the outputs, such as constancy, smoothness or physical limits, that holds at test time regardless of the input, and can be applied there?

A length term can only pull against errors it can see, which is the general point of a shape term cannot constrain what it is invariant to. The robust median the repair relies on is examined in cleaning noisy labels with a local median.

What I still don't know

My corruption was a single, clean kind of occlusion, and my constraint was tailored to it. Real failures are messier, and a test-time repair that is too eager could do more harm than the failure it fixes. I measured the repair's false-alarm rate only on clean images from the same distribution. I also haven't run the augmentation experiment, which I expect would close most of the gap inside the model. Comparing the two on the same failure would show how much of the fix belongs in training and how much at inference.

References

1. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations — Dan Hendrycks and Thomas Dietterich, ICLR 2019