Problem in convergence of hebbian learning approach for Fuzzy Cognitive Map

I was trying to learn Fuzzy Cognitive Map by Active Hebbian Learning approach from here. What I have understand is that the model learns iteratively, at each step a new concept values enters and tune the weighs until the MSE score in output neurone is very small. I thaught that it is similar to stochastic gradient descent. But I don't see any convergence in output MSE value when a new input comes.

import numpy as np
import matplotlib.pyplot as plt


mean1 = [0, 0]
cov1 = [[1, 0], [0, 1]] 

x1, x2 = np.random.multivariate_normal(mean1, cov1, 10000).T
x1 = x1.reshape([len(x1),1])
x2 = x2.reshape([len(x2),1])


def multivariate_normal(x, d, mean, covariance):
    pdf of the multivariate normal distribution.
    x_m = x - mean
    return (1. / (np.sqrt((2 * np.pi)**d * np.linalg.det(covariance))) * 
            np.exp(-(np.linalg.solve(covariance, x_m).T.dot(x_m)) / 2))




def sigmoid(x):
    return 1/(1+np.exp(-x))




def FCM(W,X1,X2,y,lr,gamma):
    y1 = 0
    y2 = 0
    loss = []
    Loss = []
    for i in range(0,2000):
        
        A1 = np.dot(W[0],np.transpose(X1[i]))
        A1 = A1.astype('float')
        A2 = np.dot(W[1],np.transpose(X2[i]))
        A2 = A2.astype('float')  
        y1 = sigmoid(A1 + y[i][0])
        y2 = sigmoid(A2 + y[i][1])  
        temp = ((y1 - y[i][0])**2 + (y2 - y[i][1])**2)/2
        print(i)
        Loss.append(temp) 
        count = 0
        while (count  1):  
            count += 1
            #print(temp)
            A1 = np.dot(W[0],np.transpose(X1[i]))
            A1 = A1.astype('float')
            A2 = np.dot(W[1],np.transpose(X2[i]))
            A2 = A2.astype('float')  
            y1 = sigmoid(A1 + y[i][0])
            y2 = sigmoid(A2 + y[i][1]) 
            temp1 = (1 - gamma)*W[0] + lr*X1[i]*y[i][0]           
            temp2 = (1 - gamma)*W[1] + lr*X2[i]*y[i][1]
            #print(temp1,temp2)
            W[0] = temp1/(np.sqrt(np.dot(temp1,np.transpose(temp1))+np.dot(temp2,np.transpose(temp2))))
            W[1] = temp2/(np.sqrt(np.dot(temp2,np.transpose(temp2))+np.dot(temp1,np.transpose(temp1))))  
            A1 = np.dot(W[0],np.transpose(X1[i]))
            A1 = A1.astype('float')
            A2 = np.dot(W[1],np.transpose(X2[i]))
            A2 = A2.astype('float')  
            y1 = sigmoid(A1 + y[i][0])
            y2 = sigmoid(A2 + y[i][1]) 
            temp = ((y1 - y[i][0])**2 + (y2 - y[i][1])**2)/2   
            loss.append(temp)     
            if count  1:
                if loss[len(loss) - 1]  loss[len(loss) - 2]:
                    break
       # Loss.append(temp)        
    return [W,Loss]        



X = np.concatenate((x1,x2),axis = 1)  



from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_temp = X
X = sc.fit_transform(X)



y = []   
for i in range(len(X)):
    if multivariate_normal(X[i],2,mean1,cov1)  0.5:
        y.append([0,1])
    else:
        y.append([1,0])
        
y = np.array(y).astype('float')        

y1 = y[:,1].reshape([len(y[:,1]),1])
y2 = y[:,0].reshape([len(y[:,1]),1])
X1 = np.concatenate((X,y1),axis = 1)
X2 = np.concatenate((X,y2),axis = 1)



W = np.array([[0.2,-0.3,-0.4],[-0.1,0.7,-0.6]])




L = FCM(W,X1,X2,y,0.1,0.1)
W2 = L[0]
loss = L[1]

To check the method, I have used Bi-normal data, data points close to then mean and far from the mean are classified to distinct class. The two output nodes are either [1,0] or [0,1].The two attributes of Bi-normal and the second of output are taken as input states for first node. Similarly, The two attributes of Bi-normal and the another first of output are taken as input states for scond node. The loss when a new data comes have ploted for 2000 iteration, there is no indication of convergence of the weights of the map. I expected that the loss will grow doen eventually like stochastic gradient descent.