Unable to train neural network for prediction

My data consists of a time series of values $\pm1$ and I am trying to apply a RBF NN as a function approximator. Essentially, the NN will take as input one data sample and predict the next sample (one step ahead prediction). However, my network is not getting trained. If I use floating point vales as in the data then the same code works. However for $\pm$ data, I am not able to figure out how to train the network Can somebody please help?

x = rand(3000,1);
Data = 2*(x=0.5)-1;

 

% Training and Test datasets
time_steps=1;  % prediction of #time_steps forward value (for this simple architechture time_steps=3)
% Training
start_of_series_tr=100;     
end_of_series_tr=2500;      
% Test
start_of_series_ts=2500;     
end_of_series_ts=3000;

P_train=Data(start_of_series_tr:end_of_series_tr-time_steps,1);   % Input Data
f_train=Data(start_of_series_tr+time_steps:end_of_series_tr,1);   % Label Data (desired output values)
indt=Data(start_of_series_tr+time_steps:end_of_series_tr,1);% Time index

SNR = 30; % signal to noise ratio
f_train=awgn(f_train,SNR);  % Adding white Gaussian noise


%% Simulation parameters
% Defining architechture of the RBF-NN
[m n] = size(P_train);% Dimensions of input data [m]-length of signal, [n]-number of elements in each input
order=1;        % Number of past values used for the prediction of future value
n1 = 20;        % Number of hidden layer neurons

% Tuning parameters for training
epoch=10;  % simulation rounds (number of times the same data pass through the NN for training)
eta=1e-2;  % Gradient Descent step-size (learning rate) 
runs=10;   % Number of Monte Carlo simulations
Iti=[];    % Initial mean square error (MSE)

% Graphics/Plot parameters
fsize=13;   % Fontsize
lw=2;       % line width size

%% Training Phase
for run=1:runs % Monte Carlos simulations loop
    
    % spread and centers of the Gaussian kernel    
    [temp, c, beeta] = kmeans(P_train,n1); % K-means clustering
    beeta=4*beeta;                   % Increasing spread of Gaussian kernel
    
    % Initialization of weights and bias
    w=randn(1,n1); % weight
    b=randn();     % bias
    
    for k=1:epoch % simulation rounds loop
        
        I(k)=0;             % reset MSE
        U=zeros(1,order);   % reset input vector
        
        for i1=1:m % Iteration loop
            % sliding window (updating input vector)
            U(1:end-1)=U(2:end);
            U(end)=P_train(i1); % current value of time-series
            
            % Gaussian Kernel
            for i2=1:n1
                phi(i1,i2)=exp((-(norm(U-c(i2,:))^2))/beeta(i2,:).^2);
            end
            
            % Calculate output of the RBF
            y_train(i1)=w*phi(i1,:)'+b;
            
            e(i1)=f_train(i1)-y_train(i1); % instantaneous error in the prediction
            
            % Gradient descent-based weight-update rule
            w=w+eta*e(i1)*phi(i1,:);
            b=b+eta*e(i1);
            
            % Mean square error 
            I(i1)=mse(e(1:i1));      % Objective Function
           
        end
        Itti(epoch,:)=I; % MSE for all iterations
    end
    Iti(run,:)=mean(Itti,1); % Mean MSE for all epochs
end
It=mean(Iti,1); % Mean MSE for all independent runs (Monte Carlo simulations)

%% Test Phase
P_test=Data(start_of_series_ts:end_of_series_ts-time_steps,1);
f_test=Data(start_of_series_ts+time_steps:end_of_series_ts,1);
indts=Data(start_of_series_ts+time_steps:end_of_series_ts,1);

[m n] = size(P_test);
for i1=1:m % Iteration loop
    % sliding window (updating input vector)   
    U(1:end-1)=U(2:end);
    U(end)=P_test(i1);
    for i2=1:n1
        phi(i1,i2)=exp((-(norm(U-c(i2,:))^2))/beeta(i2,:).^2);
    end
    y_test(i1)=w*phi(i1,:)'+b;
    
    e_test(i1)=real(f_test(i1)-y_test(i1));
    I(2400+i1)=mse(e_test(1:i1));
end

Topic rbf loss-function matlab neural-network

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.