I'm a complete n00b to both this stackexchange and ML so please don't flame me too bad. I am trying to make a prediction from Time Series data. I have about 10 years worth of 1-minute resolution price data for the S&P500. What I'd like to do is treat each DAY in the data as it's own series to predict what the price movement will be for the last 15 minutes of market hours. I've looked through several books, some …

I have a collection of time series data with data points of around 2 years of daily data. I am thinking of a way to increase the number of data points in it so that the neural network gets a better understanding of the fluctuations in the data. I am suggesting a hypothesis where I try to cluster similar time-series data following similar distribution, in order to increase the number of data points fed into the neural network. Is this …

I’m employing a denoising RNN autoencoder for a project relating to motion capture data. This is my first time using auto encoder architectures and I was just wondering what non-linearities should be placed in these models and where they should go. This is my model as it stands: class EncoderRNN(nn.Module): def __init__(self, input_size, hidden_size, num_layers): super(EncoderRNN, self).__init__() self.input_size = input_size self.hidden_size = hidden_size self.num_layers = num_layers self.rnn_enc = nn.RNN(input_size=input_size, hidden_size=hidden_size, num_layers=num_layers, batch_first=True) self.relu_enc = nn.ReLU() def forward(self, x): pred, hidden …

I want to create a spell checker that corrects the spelling mistakes contextually. For example, Erroneous sentence: I want to apply for credit cart Corrected sentence: I want to apply for credit card Here, the respective spellings of cart and card are correct but the cart is contextually incorrect. So what methods we can apply for contextual errors like this?

I have a DNA sequence dataset each mapped to a certain class. e,g TCAGCCGAGAGCTCATCGATCGTACGT 2 ATGCAGTGCATCGATCGATCGTAGAAC 3 Where the number after the sequence specifies the type of protein this sequence belongs to. So my question can I use KMers and 1-HOT-Coding to classify these sequences through biLSTM. Or this is not a possible concept and I would appreciate your feedback and suggestions on this task as I am new to Deep Learning. Thank you.

I am running an LSTM neural network in R using the keras package, in an attempt to do time series prediction of Bitcoin. The issue I'm running into is that while my predicted values seem to be reasonable, for some reason, they are "lagging" or "behind" the true values. Right below is some of my code, and farther down I have some graphs to show you what I mean. My model code: batch_size = 2 model <- keras_model_sequential() model%>% layer_lstm(units=22, …

How does one train a neural network model that does regression over real values, using a gaussian output layer? ie estimating the mean and std parameters of the prediction. Since during training there will only be one real valued target. Would one sample from the output gaussian distribution and then perform loss with the target? Or would there somehow be two targets, one for the mean or std. The Deep AR forecasting model by Amazon does something like this for …

I'm using Python and I have a training set of sequences of this shape: (None, 9, 25), where: 9 are rows representing years from 2012 to 2020 25 are features. So each of this 25 features has a value for each year going from 2012 to 2020. As target (labels) I have, for each sequence of (9,25), a list of 9 elements representing a number (either 0 or 1) associated to each year, so a possible label for a certain …

I'm trying to visualise a neural network schematic and found a great tool for building schematics here http://alexlenail.me/NN-SVG/index.html. I've edited the SVG file to change one of the dense layers into a LSTM layer, and the input to time series instead of singular neurons. At the bottom of the image there is some set notation detailing how many neurons is in each layer. I'm not too familiar with set notation. I'm not quite sure how to represent the LSTM layers …

I have one single, very long time series. I want to train an LSTM to distinguish between two behaviours (A or B) at every timestep (sequence-to-sequence). Because the time series is very long, I plan to extract shorter, partially-overlapping subsequences and use each of them as one training input for the LSTM. In my train/validation/test split, do I have to use older subsequences for training and newer for validation and test? Or can I treat them as if they were …

I just read paper about cnn + rnn for text recognition. The labels of dataset is tensor of char index (e.g [0, 1, 2 ] for image with label "abc"). Since the label of each input has different length do I need to convert the label to sparse tensor value ? since the paper does not mention about it.

I am new to this field and I was reading a paper "Predicting citation counts based on deep neural network learning techniques". There the authors describe the code that they implemented if someone wants to reproduce the results. I tried to do this but I am not sure if I succeeded. Here is their description: -RNN module - SimpleRNN -Output dimension of the encoder - 512 -The output layer - Dense layer -Activation function - ReLU -Overfitting prevention technique - …

There are some easy and comprehensive textbooks covering many deep learning concepts using PyTorch in detail. But, I am a little dissatisfied with the weightage given to RNNs compared to CNNs. I mean, some textbooks did not cover RNN at all. I am in need of learning NLP using PyTorch to great detail. So, I want the list of books or a single book that covers RNN, LSTM, etc., using PyTorch in great detail i.e., from beginner to advanced. I …

Nowadays there's a trend towards using architectures of "deep" RNNs i.e. vertically stacked RNNs. RNN chapter from Bengio's bookThese networks seem to work well in practice. What's the intuition around using vertically stacked layers of RNNs (beyond the obvious fact that they increase the capacity by increasing the # parameters)?

I am a machine learning newbie and I am working on a project where I'm given a sequence of integers all of which are in the range 0 to 70. My goal is to predict the next integer in the sequence given the previous 5 integers in the same sequence. There isn't much more information on the sequence of integers itself (for example, how was the sequence obtained, etc). The following are the things I tried. The first thing that …

What's the intuition behind the hidden states of RNN/LSTM? Are they similar to the hidden states of HMM (Hidden Markov Model)?

I am trying to explore models for predicting whether a team will win or lose based on features about the team and their opponent. My training data is 15k samples with 760 numerical features. Each sample represents a game between two teams and the features are long and short term statistics about each team at the time of the game (i.e. avg points over last 10 games). My thought was to use a binary classifier as a multi layered perceptron. …

The problem that I am dealing with is predicting time series values. I am looking at one time series at a time and based on for example 15% of the input data, I would like to predict its future values. So far I have come across two models: LSTM (long short term memory; a class of recurrent neural networks) ARIMA I have tried both and read some articles on them. Now I am trying to get a better sense on …

I am trying to understand RNN. I got a good sense of how it works on theory. But then on PyTorch you have two extra dimensions to your input data: batch size (number of batches) and sequence length. The model I am working on is a simple one to one model: it takes in a letter than estimates the following letter. The model is provided here. First please correct me if I am wrong about the following: Batch size is …

I am working on training RNN model on caption generation with REINFORCE algorithm. I adopt self-critic strategy (see paper Self-critical Sequence Training for Image Captioning) to reduce the variance. I initialize the model with a pre-trained RNN model (a.k.a. warm start). This pre-trained model (trained with log-likelihood objective) got 0.6 F1 score in my task. When I use adam optimizer to train this policy gradient objective, the performance of my model drops to 0 after a few epochs. However, if …