Multivariate RNN/LSTM architecture for optimizing one input variable

Let $x = [x_1, x_2, x_3]$ and $y = y$ where all variables in $x$ correlates highly with $y$ and there could also be some crosscorrelation within the set of variables in $x$.

The data takes the form of a sequence/timeseries. i.e for each timestep we have three independent variables and one dependent or one independent variable and three dependent depending on how we structure it. So the whole timeseries would be X = [x1, x2, x3, ...xn] and Y = [y1, y2, y3, ....yn]

$x_1$ for timestep h1 could be highly influential for the output y in timestep h2-hn. I want to construct an LSTM that captures all variables and feed it to a blackbox optimization which configures ONE of the variables in $x$ lets say $x_2$ for maximizing the expected output y in the future timesteps(one timestep in the future is sufficient).

Note that for an future timestep I cannot see or configure the two other variables I do not want to optimize. So maybe the notation $x=x$ and $y=[y_1, y_2, y_3]$ would capture this better, if we flip it like this I want to optimize $x$ in the blackbox optimization to maximize e.g $y_1$.

My first idea is to construct some sort of many-to-many network where the variable I want to optimize in the blackbox optimization would be the sole input in each timestep and then output the second two variables in $x$ and $y$, then this output has to be passed on to the cells in the second timestep.

My goal is to feed the network to a black box optimizer in which i configure one of the variables with the goal to maximize one of the other variables.

How would one go about this architecture? Does anynone see any issues about this? is there a smarter way to tackle this?

As for the baseline I just construct a feed forward neural network which does not adhere to the sequence but solely rely on the current state and predict an output. But since it is a timeseries with seasons, trends etc i should be able to outperform it, I just have to wrap my head around the multiple output/an appropriate architecture. Does anyone have some input on this architecture or does anyone have some idea on how to go on about this problem?

Topic recurrent-neural-network deep-learning neural-network machine-learning

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.