I started to learn reinforcement learning, the first example is handling bandit program using epsilon-greedy method, In this example, there are three bandit machines used, the output is the mean value for all bandit machines and cumulative average with respect to the epsilon value The code - class Bandit: def __init__(self, m): self.m = m self.mean = 0 self.N = 0 def pull(self): return np.random.randn() + self.m def update(self, x): self.N += 1 self.mean = (1 - 1.0/self.N)*self.mean + 1.0/self.N*x …
I am currently trying to write a simple multivariate gaussian mixture model using tensorflow probability. Specifically, I have some 2-dimensional input and 2-dimensional output data and am looking to produce a probabilistic model that best fits this data by using a neural network. I am utilizing tfp.layers.MixtureSameFamily to generate this model, which is working perfectly well as expected. Here is my code for doing so: Notes: I am using tensorflow 2.4.1 and tensorflow-probability 0.12.1 x = some 2-d input data …
I am trying to self-implement a logistic regression algorithm to do some self-learning but I am having a bit of trouble with achieving similar accuracy to the logistic regression of sklearn. Here is the code I am using (the dataset I am using is the titanic 'training.csv' dataset from kaggle which you can download here if you want to test this out yourself.) import numpy as np import random import matplotlib.pyplot as plt #%matplotlib inline def cost(X, Y, W): """ …
Firstly, I am a beginner in this field of Data Science and have tried to implement some time series models for wind speed forecasting. Also, I am aware of the fact that some regression models might give better results, but still, my aim is to crack the same with the help of VAR I tried to implement multivariate time series forecasting - VAR in python. To start with I followed the code in this article- https://towardsdatascience.com/simple-multivariate-time-series-forecasting-7fa0e05579b2 However, the forecasted value …
I have composed a customized loss function (kl_loss): def tensor_pValue(pnls,pnl): vec=tf.contrib.framework.sort(pnls,axis=-1,direction='ASCENDING') rank_p=tf.divide(tf.range(0,264.5,1),264.0) return tf.gather(rank_p, tf.searchsorted(vec,pnl,side='left')) def kl_divergence(p, q): epsilon = 0.00001 p=p+epsilon q=q+epsilon return tf.reduce_sum(p * tf.log(p/q)) def kl_loss(predicted_pnL,actual_pnl_tensor): p_dist=tf.squeeze(tf.map_fn(lambda inp:tensor_pValue(inp[0],inp[1]),(predicted_pnL,actual_pnl_tensor),dtype=tf.float32)) u_dist=tf.random.uniform([264],0,1,dtype=tf.float32) return kl_divergence(p_dist,u_dist) And then i constructed a simple net work using Keras: optimizer = tf.train.AdamOptimizer(0.001) input_dim = X_train.shape[1] model = keras.Sequential([ keras.layers.Dense(UNITS, activation=tf.nn.relu, input_dim=input_dim), keras.layers.Dense(UNITS, activation=tf.nn.relu), keras.layers.Dense(264) ]) model.compile(loss=lambda y, f: kl_loss(f,y), optimizer=optimizer) model.fit(X_train, train_y, epochs=EPOCHS, batch_size=BATCH_SIZE,verbose=0) And got following errors: ValueError: No gradients provided for any variable: …
I am working on implementing a Naive Bayes Classification algorithm. I have a method def prob_continous_value which is supposed to return the probability density function for an attribute given a class attribute. The problem requires classifying the following datasets: Venue,color,Model,Category,Location,weight,Veriety,Material,Volume 1,6,4,4,4,1,1,1,6 2,5,4,4,4,2,6,1,1 1,6,2,1,4,1,4,2,4 1,6,2,1,4,1,2,1,2 2,6,5,5,5,2,2,1,2 1,5,4,4,4,1,6,2,2 1,3,3,3,3,1,6,2,2 1,5,2,1,1,1,2,1,2 1,4,4,4,1,1,5,3,6 1,4,4,4,4,1,6,4,6 2,5,4,4,4,2,4,4,1 2,4,3,3,3,2,1,1,1 Venue,color,Model,Category,Location,weight,Veriety,Material,Volume 2,6,4,4,4,2,2,1,1 1,2,4,4,4,1,6,2,6 1,5,4,4,4,1,2,1,6 2,4,4,4,4,2,6,1,4 1,4,4,4,4,1,2,2,2 2,4,3,3,3,2,1,1,1 1,5,2,1,4,1,6,2,6 1,2,3,3,3,1,2,1,6 2,6,4,4,4,2,3,1,1 1,4,4,4,4,1,2,1,6 1,5,4,4,4,1,2,1,4 1,4,5,5,5,1,6,2,4 2,5,4,4,4,2,3,1,1 The code for this is written like so: from numpy.core.defchararray import count, index import …
I am trying to implement the core-means algorithm, which is basically k-means using coreset. I have searched up and down but could not find any libraries or modules which could help me with this. The paper I am following talks about building a coreset using grids, something like a quadtree where you keep dividing the point space in 4 equal parts to find heavy points which makes a coreset. Any help would be greatly appreciated in getting me started as …
I was trying to write reinforcement learning agent using stable-baselines3 library. The agent(abservations) method should return action. I went through different models API (like PPO) and they do not really allow us to specify action space. Instead action space is specified in environment. This notebook says: The type of action to use (discrete/continuous) will be automatically deduced from the environment action space. So, it seems that "model" deduce action space from environment. Q1. But exactly how? Q2. Also how my …
The KNN algorithm is very handy and particularly suited to some of my problems, but I can't find any resources on how to implement it in production. As a comparative example, when I use a neural network, I already have at my disposal high-level tools allowing me to apply the neural network to examples (either library allowing me to smartly exploit the hardware of my devices when I want to do embedded, or infrastructures allowing me to use my neural …
I am trying to replicate (a way smaller version) the AlphaGo Zero system. However, in the network model, I am having a problem. The loss function I am supposed to implement is the following: $$l = (z - v)^2 - \pi^T log(p) + c ||\theta||^2$$ Where: $z$ is the label (a real value between -1 and 1) of one of the two heads of the network and $v$ is this value predicted by the network. $\pi$ is the label of …
I try to use linear regression for insurance data . But had error on the when try to call a function with features parameter. Here is my code: def h(x): global w return np.sum(np.transpose(w)*x) raise NotImplementedError() when try with a simple data it works fine, w, x = [1,2,3], [2,3,4] h(x) the output is : 20 but when try to use the dataset, it errors: features = dataset.drop(["charges"], axis=1).values h(features ) it returns error: ValueError: operands could not be broadcast …
I am trying to perform a simple linear regression using Pytorch lightning (a network with only one neuron). The network is supposed to learn a simple function: y=-4x. The size of my dataset is 1000 and contains points from the line y=-4x with a small amount of gaussian noise. The dataset looks like this: I am facing a strange problem where the model only converges when the batch size is small enough and when I don't shuffle random data in …
On a recent Hackerrank interview I was faced with the following problem: Given a set of timestamps (format 2019-11-26 11:00) and their corresponding stock prices (single float value), approximate the missing stock prices for a set of timestamps. I tried solving it with an SVR model at first, but I had to realize that none of the usual data science libraries were available for this test. How would you solve this problem without data science libraries in a limited amount …
I am learning Hidden Markov Model and its implementation for Stock Price Prediction. I am trying to implement the Forward Algorithm according to this paper. Here I found an implementation of the Forward Algorithm in Python. import pandas as pd import numpy as np V = np.array([0, 1, 1, 2, 0, 1, 2, 0, 1, 0, 2]) # Transition Probabilities a = np.array(((0.54, 0.46), (0.49, 0.51))) # Emission Probabilities b = np.array(((0.16, 0.26, 0.58), (0.25, 0.28, 0.47))) # # Equal …
I was experimenting with my policy gradient reinforcement learning algorithm, and I was wondering if I could use a similar method to the supervised cross-entropy. So, instead of using existing labels, I would generate a label for every step in the trajectory. Depending on the value of the action, I would shift the stochastic policy (Neural Network) output to a more efficient output and train it as a label to a cross entropy loss function. Example of an action: Real …
In this paper, Stock price prediction using kernel adaptive filtering within a stock market interdependence approach, the authors propose a method for predicting stock prices by combining the predictions of Kernel Adaptive Filter (KAF) models trained on different stocks in different international stock markets. In their results, they compare this model against individual KAF models trained and inferred on individual stocks. The refer to these individual models as 'KAF-based methods' and refer to their model as 'Proposal'. I am quite …
I'm attempting to implement the policy gradient taken from the "Hands-On Machine Learning" book by Geron, which can be found here. The notebook uses Tensorflow and I'm attempting to do it with PyTorch. My models look as follows: model = nn.Sequential( nn.Linear(4, 128), nn.ELU(), nn.Linear(128, 2), ) Criterion and optimisers: criterion = nn.BCEWithLogitsLoss() optim = torch.optim.Adam(model.parameters(), lr=0.01) Training: env = gym.make("CartPole-v0") n_games_per_update = 10 n_max_steps = 1000 n_iterations = 250 save_iterations = 10 discount_rate = 0.95 for iteration in range(n_iterations): …
I have been trying to write a naive bayes classifier from scratch that is supposed to predict the class label of the nominal car.arff dataset. However the classifier always predicts the most common one. I have tried log probabilities and laplace correction, both to no avail. Also I have noticed that the conditional probabilities for any attribute is always the greatest for the most common label. Is this because of my dataset? What can be done about it? Here is …