unsupervised anomaly detection for univariate fast frequency time series data?

I have a univariate time series (there is a value for each time sampling) (sampling time: 66.66 micro second, number of samples/sampling time=151) coming from a scala customer

This time series contains some time frame which each of them are 8K (frequencies)*151 (time samples) in 0.5 sec [overall 1.2288 millions samples per half a second)

  • I need to find anomalous based on different rows (frequencies) Report the rows (frequencies) which are anomalous? (an unsupervised learning method)

Do you have an idea to which statistical parameter is more useful for it? or is it possible to evaluate without time sampling?

Do have an idea about suitable fast method (with lowest time delay: therefore some algorithm like kmeans does not work)

I should produce online automated ML

Topic pipelines unsupervised-learning anomaly-detection scala time-series

Category Data Science

1
answered at

To start simple, we can keep a stat of the mean and std of each frequency band from past observations, and compute the likelihood of observing the new incoming frequencies (assuming some distribution).

If the anomaly pattern is more complicated, you may have to resort to other standard unsupervised learning techniques, K-means for example.

Any prior knowledge of anomaly pattern helps simplify the problem, so make sure you look for it.

1
answered at

Is it possible to apply an offline anomaly pattern recognition? Or a "normal" pattern recognition that gives an alert when it is out of its boundaries?

If yes, it could be the best option as you only need to detect specific behavior with some error margin and without heavy calculations.

Low level processes with high sampling speed generally have simple dynamics that don't require complex algorithms.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.