Poisson model with overdisperssion

I'm working with a dataset $X$ (of length $N$) of count data, which looks like: I developed a statistical model which can be improved, so I'm asking for any suggestions, for instance, differnet likelihoods or prior selection, different approach, anything... My model I'm trying to get the parameters of the likelihood of the data, so thaht I can get a posterior predictive density function, credible intervals and so on. Let's say, I want to model the generative process of the …
Category: Data Science

Poisson distribution, Standard deviation, fitting line

Let's say I have system A, B, C, and D. Each system contains 10,000 numbers generated by Poisson distribution. The difference is the mean is different for different systems. I calculated the std dev for each distribution corresponding to each system. I plot the standard deviation with respect to mean numbers. (I observe std deviation increases with mean). I want to fit the plot with some line, which gives me the general estimate of std dev with respect to the …
Category: Data Science

Is there R functions that allow to test for overdispersion when fitting a model with survey design?

I realized I need to use the package survey to be able to include sample weights in my regression analysis. Initially, I wanted to use a negative binomial regression on each one of my outcomes as count data is more often than not overdispersed, so I tried using svyglm.nb. However, for one of the outcomes which has small values, svyglm.nb makes my program crash, so I think there might be some convergence issue. I thought using a Poisson regression might …
Category: Data Science

MLE for Poisson conditioned on multivariate Gaussian?

I am writing some Python code to fit 2D Gaussians to fluorescent emitters on a dark background to determine the subpixel-resolution (x, y) position of the fluorescent emitter. The crude, pixel-resolution (x, y) locations of the pixels are stored in a list xy. The height of the Gaussian represents the predicted pixel intensity at that location. Each 2D Gaussian has 5 parameters, and my end goal is to find the optimal value of those 5 parameters for each peak using …
Category: Data Science

Difficulty understanding the difference between Poisson, Quasi-Poisson, and Negative Binomial models

I will try to keep this short. As an assignment for my GLM course, we were given a dataset on the # of homicide victims a person knows, as well as the race of the person. The main idea is to answer the scientific question "Does race help explain how many homicide victims a person knows?". This same dataset, and actually nearly all the sub-problems are solved here: https://data.library.virginia.edu/getting-started-with-negative-binomial-regression-modeling/. My issue is, I am struggling to understand the difference between …
Category: Data Science

regression by grouping the dependent variable

I have a large dataset exploring the effects of the independent variables on the dependent variable using Poisson regression since the dependent variable is a count variable. However, the range of the dependent variable is too large. Hence, I was thinking of grouping the dependent variable, like low, medium and high values, and then use the Poisson regression for each group. My question is, does this makes sense? I mean, grouping the dependent variable and then running the tests separately …
Category: Data Science

How to implement large-scale Poisson Regression in Python

I am trying to implement a Poisson Regression in Python to predict rates. I am dealing with a ton of data (too much to store in a DataFrame), which means that using the standard statsmodels.api GLM Poisson Regression won't work. I know that sklearn has a partial_fit() method with the SGDRegressor and SGDClassifier classes for Minibatch learning, but I cannot figure out how to implement a Poisson Regression with these classes. Does anyone know how to implement a Poisson Regression …
Category: Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.