Add timestamp as a feature to model

Question

Add timestamp as a feature to model

Wenuka

2022年2月3日 15:43

I am working with time-series data and am interested in adding time-stamp data (as a feature) into the (DNN) model. From the things I have read online so far, my only option is to come up with my own set of features (like Weekday, day, month, time of the day etc...) and feed it to the model.

So my first question is, is it the only way to do it? Any leads to a different path? I am worried about this approach as there are many features I can think of.

The second question is, if I am following the above path, given a cyclic feature (assume the weekday), what options do I have? I found some people are using one-hot-encoding, but it will further increase the number of input features. I am thinking of using sin and cos as described in this link. Any other better alternative for it?

Thanks

Topic time feature-engineering time-series machine-learning

Category Data Science

Warlax56 · Accepted Answer · 2022年2月3日 15:43

This has a lot to do with your data, needs, and how time relates to other aspects of your problem. From where I'm sitting, you have a few approaches:

You can encode cyclical time using a modulus operator to create a sawtooth wave within the region of time you're interested in

time_feature = (time % interval)/interval

would create

  /|  /|  /| <time_feature=1
 / | / | / | 
/  |/  |/  | <time_feature=0
i1 i2  i3  i4 <intervals

You could do this for multiple time intervals in parallel, if you feel inclined.

You can use one hot encoding

These are some things you should look out for:

using a sin wave, to me, seems like your asking the model to make inferences it shouldn't have to make. The issue with a sine wave is it's hard to know where you are within that wave (which is why, often, encoders have two parallel sin waves offset by 90 degrees). A sawtooth wave seems more appropriate. Using sin/cos to encode cyclical time seems like a common approach, though, so take my words with a grain of salt.
you should be apprehensive in how accurately you encode time. If you encode time as the actual date/time (in some sort of normalized form of a UNIX timestamp, for instance), you are effectively giving an ID to the data, meaning your model can simply memorize what time it should preict certain values, causing over fitting.

hal9000 · Accepted Answer · 2022年2月2日 19:20

A different approach would be to use neural ode's (arxiv:1806.07366) and neural stochastic ode's (arxiv:2001.01328). They don't need timestamps (as features) and work well with irregularly sampled data points as you don't have to work with discrete steps.
(image below is from Neural Ordinary Differential Equations arxiv:1806.07366)

About the timestamps I believe sin cos positional encodings will work fine, but if the timestamps repeat when the data has it's natural cycle it might work even better. (as it constrains the model more than just positional encodings in a way that fits well with your data)

Brian Spiering · Accepted Answer · 2022年1月31日 21:14

There are several possible ways to encode time as a feature:

Timestamp.
Discrete components (e.g., year, quarter, month, day, hour).
Time elapsed since an event. Unix time is the number of seconds that have elapsed since the Unix epoch. Time elapsed could also be age.

After the creation of direct features, derived features could be created. It is common to add facets to model reoccurring patterns.

You can also directly model time-based cycles by pick a neural network that can capture those patterns. Long Short Term Memory (LSTM) is the one of the most popular options.