What are some state of art computer vision models for anomaly detection that can learn continuously and build classes for detected anomalies?

I'm looking forward to build a model that:

  • Detect anomalies
  • Improve over user feedback
  • Build classes for the anomalies based on user feedback

Since a schema is worth a thousand words:

Do you know some state of art models that have this behavior (at least partially), that I can used or benchmark?

Topic uncertainty computer-vision anomaly-detection deep-learning classification

Category Data Science

1
answered at

The system you described will likely need to be made of a combination of several "models".

  1. An anomaly detection model, that all samples go through. Your top diamond in the flow diagram.
  2. A classification model (for the flagged anomalies). The middle and lower diamonds in your diagram.

Since there are not (to my knowledge) well defined tasks and datasets for this, I doubt that you will find ready SOTA architectures for the overall solution.

The two models can either be completely separate, or they can share some parts. One architecture would would be to use a Convolutional Neural Network on the images to obtain a compact vector representation ("image embedding"). This can be either a pretrained model (say from ImageNet), or you can use self-supervised learning on your own unlabeled data to make it. Then on top of this representation you can build simpler models for 1 and 2. A good baseline would be k-Nearest Neighbours, it is very simple and can be used both for anomaly detection and classification. The distance from existing neigbours can be used as an uncertainty estimation.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.