Multiclass semantic segmentation with some classes possibly not present in some of the images

Let's assume we have a large annotated dataset with 4 classes. In this dataset, there might be annotated images with less than 4 classes, where the remaining classes might or might not be present. As an example, say we want to detect pedestrians/cars/bicycles/roads in images. In our dataset, there are some annotated images with only 3 classes: pedestrians/cars/bicycles, but this does not mean there are no roads in these images. That is, there might be roads in these images that were ignored by the annotator for some reason, or there might no be roads at all. My question is, how do we take this uncertainty into the loss term?

One option is to work with independent networks for each class. But what if we want to train a single network? how do we add something like a don't care term for objects not annotated in an image, but might still be present in the image?

Topic semantic-segmentation machine-learning-model loss-function deep-learning

Category Data Science

1
answered at

That problem is commonly called weak labels and is associated with weak supervision.

Weak labels are often not handled by the loss function. Most conventional loss functions assume perfectly labeled data points and have no mechanism to handle label uncertainty.

An alternative framework for weak labels is Confident Learning which

characterizing and identifying label errors in datasets, based on the principles of pruning noisy data, counting with probabilistic thresholds to estimate noise, and ranking examples to train with confidence.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.