Keras model prediction always has unwanted offset

I am trying to predict next 10 days by looking into the last 60 days. So tried to implement an LSTM layer. Before jumping into the question, I want to clarify a few points. Firstly, this is a Multiple Parallel Input and Multi-Step Output problem as it is described in the link.

I collected the data of the last 5 years of all funds available in my country from this address. I refined my data as much as possible. Of course some of the values were missing, so I replaced all missing ones with '0.0' (zeros). I shaped the data in a way that it contained all of the funds in time series.

My database has a 'Date' Column, and fund's volume, number of shareholders, distribution of its assets (i.e. portfolio breakdown) (Stock Share / VIOP / Treasury Bill etc.) and of course its Price. It has 38012 columns and 1115 rows ( for 1115 work days). I tried to correct all the anomalies in the database and I think I did a good job.

I am trying to guess the price of each fund within the next 10 days.

from selenium import webdriver
from bs4 import BeautifulSoup
import pandas as pd
import requests
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.by import By
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions

from db.fonDb import fonDb
import time
import re
from datetime import date
from datetime import datetime, timedelta
import csv
from tabulate import tabulate
from itertools import islice
import numpy as np
import shelve
import os
import sys
import math
from tensorflow.keras.models import Sequential 
from tensorflow.keras.layers import LSTM 
from tensorflow.keras.layers import Dense, Dropout, Activation
from tensorflow.keras.layers import RepeatVector
from tensorflow.keras.layers import TimeDistributed
from tensorflow.keras.optimizers import SGD, Adam, RMSprop
from tensorflow.keras.callbacks import TerminateOnNaN
from tensorflow.keras.regularizers import *
from matplotlib import pyplot as plt
from sklearn.preprocessing import StandardScaler
import seaborn as sns
sys.path.insert(0, ..)

term = TerminateOnNaN()

daily_mem_path = 'mem_2\\dailyFonDb'
data_dict = None
with shelve.open(daily_mem_path) as fonDb:
    data_dict = dict(fonDb)
fonDb.close()

df = pd.DataFrame(data_dict)
cols = list(df)[1:len(df.columns)]
df_for_training = df[cols].astype(float)
print(df_for_training.isnull().values.any())
df_for_training.replace([np.inf, -np.inf], 0,inplace=True)
count = np.isinf(df_for_training).values.sum()
print(count)
scaler = StandardScaler()
scaler = scaler.fit(df_for_training)
df_for_training_scaled = scaler.transform(df_for_training)

trainX = []
trainY = []
n_future = 10   # Number of days we want to look into the future based on the past days.
n_past = 60  # Number of past days we want to use to predict the future.
from tensorflow.keras.layers import Dropout
for i in range(n_past, len(df_for_training_scaled) - n_future +1):
    trainX.append(df_for_training_scaled[i - n_past:i, 0:df_for_training_scaled.shape[1]])
    trainY.append(df_for_training_scaled[i :i + n_future, 0:df_for_training_scaled.shape[1]])

trainX, trainY = np.array(trainX), np.array(trainY)

print('trainX shape == {}.'.format(trainX.shape))
print('trainY shape == {}.'.format(trainY.shape))


opt = Adam(learning_rate=0.001)
opt_rms = RMSprop(learning_rate=0.006)

model = Sequential()
model.add(LSTM(2, activation='relu', input_shape=(n_past, trainX.shape[2])))
model.add(RepeatVector(n_future))
model.add(LSTM(2, activation='relu', return_sequences=True))
model.add(Dropout(0.1))
model.add(Dense(2, activation='softmax', use_bias=True, kernel_regularizer=l2(), bias_regularizer = l1()))
model.add(Dropout(0.1))
model.add(TimeDistributed(Dense(trainX.shape[2], activation='relu', use_bias=True, kernel_regularizer=l1(), bias_regularizer = l2())))
model.compile(loss='mse', optimizer= opt_rms , metrics=['mse'])

model.summary()

history = model.fit(trainX, trainY, epochs=10, validation_split=0.5, verbose=1, callbacks=[term])

plt.plot(history.history['loss'], label='Training loss')
plt.plot(history.history['val_loss'], label='Validation loss')
plt.legend()
plt.show()

last_data_scaled = df_for_training_scaled[-n_past :, 0:df_for_training.shape[1]]
last_data_scaled = last_data_scaled.reshape((1, n_past, df_for_training.shape[1]))
new_predict = model.predict(last_data_scaled)
y_new_pred_future = scaler.inverse_transform(new_predict)
y_new_pred_future_x_trs = np.transpose(y_new_pred_future[0])

for j in range(len(df_for_training.columns)):
    if '_Fiyat' in df_for_training.columns[j]:
        temp_arr = list(df_for_training[df_for_training.columns[j]][-10:])
        
        temp_arr.extend(list(y_new_pred_future_x_trs[j]))
        print(temp_arr)
        plt.plot(temp_arr)
        plt.show()
        print(df_for_training.columns[j])

This is the code. What I did is to separate training data as usual, each chunk of X contained 60 days and y contained the folloing 10 days. I trained the data and made a prediction on the last 60 days (it is this years october to november). However, no matter what I do, the output did not show any mercy. You will see the last output from my many trials:

You are seing the prediction for the first three funds' prices. The rest is much less the same. It is clear that the prediction has an offset. In some of my trials, I see that the prediction has some trends (not a straight line) but that is not the issue of now. First I need to know why I am seeing this disturbed offset, like all the weights are vanished but this bias. Secondly, how can I fix this?

I tried adding dropout layers, changing the validation rate, changing the learning rate of the optimizer, changing the optimizer itself (I saw that RMSprop may be a good choice for this kinds of applications), adding regularizors, adding bias regularizors, changing the activation types (softmax, linear, relu, sigmoid vs.), increasing the epoch count. After many weeks of research, today I gave up. It appears to me I am doing something tremendously wrong.

Any ideas?