Can we optimize heterogeneous parameters of RBF Network using Gradient Descent?

There're three parameters in the Radial Basis Function Networks (RBFN). Centers of RBFs Width of RBFs Weights of RBFs It's a fact that Weights can be easily updated using a simple Gradient Descent. My question is: Can we optimize Centers and Widths of RBFs using Gradient Descent such that approximation will tend to be better. Any suggestion is welcome.
Category: Data Science

Unable to train neural network for prediction

My data consists of a time series of values $\pm1$ and I am trying to apply a RBF NN as a function approximator. Essentially, the NN will take as input one data sample and predict the next sample (one step ahead prediction). However, my network is not getting trained. If I use floating point vales as in the data then the same code works. However for $\pm$ data, I am not able to figure out how to train the network …
Category: Data Science

Estimating a rbf kernel SVM, followed by Stochastic Gradient Descent

I wanna estimate a rbf SVM to predict property prices. My data set has 11 features and roughly 57,000 rows. When I set C=10, R^2 is about 0.88 while MSE and RMSE are 0.1191 and 0.3451. The results are pretty good. Afterward, I estimate a SGD, using linear_model.SGDRegressor and loss='squared_epsilon_insensitive'. When I use adaptive learning rate, R^2 is reduced to 0.75 while MSE and RMSE are 0.2441 and 0.4940, respectively. When I use optimal learning rate, the results are even …
Topic: sgd rbf svm
Category: Data Science

Same training and test data is fed to SVM RBF kernel in python and matlab giving different results

I have used 60 % data as training data and 40% data as test data. Exactly same instances of data are fed to SVM RBF kernel in Python and SVM Gaussian in MatLab. But the results of prediction in MatLab are horrible, all the data is belonging to the 5th class where I am getting 99% accuracy with good class-specific accuracies in Python. Please tell me where is the problem.
Category: Data Science

Is it efficient to use kernel trick in primal form of SVM?

I know we can use Kernel trick in the primal form of SVM. So the hypothesis will be - and optimization objective - We can optimize the above equation using gradient descent, but in this equation suppose we use RBF kernel (which projects training data into infinite dimensions), then if the number of features are infinite, then dimension of 'w' will also be infinite and the optimization equation will learn 'w' using gradient descent, then how its supposed to learn …
Topic: rbf kernel svm
Category: Data Science

How to use a RBF kernel to create a "Kernel Space" using the similarity of each pair of point?

I am trying to use Semi-Unsupervised clustering using reinforcement learning following this paper. Assume I have n data-points each of which has d dimensions. I also have c pairwise constraints of whether two elements are supposed to be in the same cluster or not. The paper states that "the original input dimension of the dataset is appended to a kernel space with a similarity metric to each pairwise point in the set of constraints" creating a d + 2c dimensional …
Category: Data Science

Why spectral clustering results in disjointed cluster?

I'm working on a project where I have to dynamically cluster the position of objects with respect to one coordinate. So I'm essentially dealing with subsequent frames and each frame represents a one-dimensional dataset. The intuition behind clustering is to form clusters out of points that are in similar distance to other points within the cluster and can be naturally connected. I use spectral clustering due to its ability to cluster points by their connectedness and not the absolute location …
Category: Data Science

Notation unclear - Radial Basis Funtion

In Chapter 6 (Deep Forward Networks) on Page 193 of Deep Learning they talk about the design of Hidden Units. The Radial Basis Function is introduced as follows: $$ h_i = exp\big{(}-\frac{1}{\sigma_i^2}||W_{:,i} − x||^2\big{)} $$ What does the colon as Index for $W$ mean?
Category: Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.