How can calculate Efficiency for predictive models based on accuracy or error over time?

Question

How can calculate Efficiency for predictive models based on accuracy or error over time?

Mario

2020年1月27日 14:00

I was wondering if I could express the efficiency of prognostic models according to their accuracy(error, e.g. MAPE or MSE) over time [sec]. So let's imagine I have the following results for different predictive models:

models MSE   MAE     MAPE       predicting Time[sec]
LSTM   0.12  0.13   15.67%          456789
GRU    0.06  0.05   5.89%           688741
RNN    0.45  0.51   25.33%          55555

What is the best way to illustrate the efficiency of predictive models over predicting time? Is the following equation right? how about its unit when we use MSE or MAE instead of MAPE which is expressed by %?

\begin{equation} \text{Efficiency} = \frac{Accuracy}{predictingtime }= \frac{error (e.g.MAPE)}{predictingtime }= \frac{percentage}{sec} \end{equation} which graph demosntraete efficiency scientifically for prognestic models?

I read this article and the introduce following criteria under name of prediction efficiency PE. is it possible to use such that?

Topic deep-learning accuracy predictive-modeling efficiency

Category Data Science

Noah Weber · Accepted Answer · 2019年12月13日 22:09

Given that computation time is system dependent maybe you could consider MAE/number of parameters (in your case it would be weights since they are all NNs)

If you want to use time, your are right cant compare apples with oranges, so just drop the percantage and make sure they are normalized values (all future metrics)

How can calculate Efficiency for predictive models based on accuracy or error over time?

About