Why are predictions from my LSTM Neural Network lagging behind true values?

I am running an LSTM neural network in R using the keras package, in an attempt to do time series prediction of Bitcoin. The issue I'm running into is that while my predicted values seem to be reasonable, for some reason, they are lagging or behind the true values. Right below is some of my code, and farther down I have some graphs to show you what I mean. My model code:

batch_size = 2              


model - keras_model_sequential()

model%%
  layer_lstm(units=22, 
             batch_input_shape = c(batch_size, 1, 22), use_bias = TRUE, stateful = TRUE,
              return_sequences = TRUE) %%
  layer_lstm(units=16, batch_input_shape = c(batch_size, 1, 22), stateful = TRUE, return_sequences = TRUE) %%
  layer_dense(units=1)
model %% compile(
  loss = 'mean_absolute_error',
  optimizer = optimizer_adam(lr= 0.00004, decay = 0.000004),  
  metrics = c('mean_absolute_error')
)
summary(model)

Epochs - 50
for (i in 1:Epochs){
  print(i)
  model %% fit(x_train, y_train, epochs=1, batch_size=batch_size, verbose=1, shuffle=FALSE)
  model %% reset_states()
}

So in case that's not clear, I have a neural network with 1 middle layer - I have 22 units in the input layer (equal to my number of variables), 16 units in middle layer and my one output layer.

Here is a graph of the training data fit (blue is fit, red are true values):

I am predicting the Bitcoin price 24 hours ahead. I have hourly data, so I'm doing this prediction by simply shifting the Bitcoin price column of my data 24 steps back, so I'm matching past predictor conditions with future outcome.

From the picture above, you can see that the training fit is very strong. However, take a look at my out-of-sample predictions vs. true values (again, blue line is model prediction, red line is true value):

At first glance, it's really not shabby. However, if you take a closer look (and it becomes VERY obvious when I zoom in to smaller time scales), the predicted blue line often lags behind the true red line:

The odd thing to me is that this is not a consistent problem. If you look at some of the movements towards the right of the graph, the model gets it on-target (no lag). Additionally, from zooming in and really looking carefully, I have found that the apparent lag itself is not consistent in magnitude, ranging from around 14 hours to sometimes 22 hours (meaning the prediction is barely usable, since it's predicting 24 hours ahead, but lagging the true value by 22 hours, so I'm really only getting 2 hours ahead of real prediction).

I have tried increasing my batch size (to 5, 10, 30), which doesn't make the problem any better (might even make it worse). I tried increasing the size of my middle neuron layer (to 20, 30, 44), which also didn't fix the issue. Having the loss function as Mean Absolute Error SEEMS to work better than Mean Squared Error, but what you're looking at is already the MAE version, so the problem obviously persists.

Around half of my inputs into the neural network model are lagged values of the Bitcoin price (BTC price 24 hours ago, 25 hours ago etc.), so I thought maybe the problem was that my model was simply grabbing those past values and replicating them because the model couldn't find any other meaningful connections to my predictors. However,

1. You can see that the problem doesn't exist in the training dataset fit, so I don't think this is an issue of my model only using past price values as its best guess.
2. I tried changing the past lags that were used (for example, instead of using the 24 hour ago value, I used the 30 hour ago value). However, this didn't make a difference, so I'm pretty confident now that the issue is not that my model is relying solely on past price values..

As a result, I really have no idea where this gap is coming from.

Any advice, suggestions or tips would be appreciated on how I could deal with this odd gap. Thank you very much!

EDIT (please read entirely, important): In order to once-and-for-all test the idea that it's the lagged time series inputs causing the issue, I just ran the neural network with all past values of the price removed. As in, ALL of the inputs were exogenous variables, no time-series lagged values, and while it's a bit hard to tell (because the predictions are messier), the problem appears to persist. Take a look:

I think this pretty much definitively proves that the lag is not coming from past price values being replicated. HOWEVER, I looked at the training data fit for the model with no time-series inputs, and it is obvious that it ALSO has an offset/lag. Example:

One more thing I need to mention. When I run this neural network on the same data but without offsetting the predictors from outcomes, there is no problem. That's to say, when I run the data without shifting the Bitcoin price column back, meaning my network is matching current conditions to current price, this prediction offset does not exist. In fact, I've been playing around with this offset (so, trying to predict 12 hours ahead, 24 hours ahead, 48 and 72 hours ahead) and it seems like changing this changes the lag in prediction. I have no idea why. When I change it to predicting 72 hours ahead, the prediction lag isn't exactly 72 hours (just like it isn't exactly 24 hours when I'm predicting that far ahead). However, the prediction lag noticeably increases/decreases when I increase/decrease how far ahead I'm trying to predict..

EDIT 2: I am now quite certain that I'm making some mistake in my data processing. Since I noticed that the prediction offset increases/decreases with how far ahead I'm trying to predict, I tried making the value for how many hours ahead I want to predict negative. (-20, to be exact). And here's what I saw now:

Sure enough, the predictions are now significantly ahead of the actual values. As a result, I think I'm making some kind of basic data processing error. As of right now, I haven't found the error yet though.