In Gradient Descent, we start with a random value of the co-efficient & improve upon it with each iteration. Do we move on to the next co-efficient after finding the first one, or do we work on all the coefficients simultaneously?

A closed-form solution of MLE exists when the parameter space is convex & the likelihood function is concave. I did not understand this statement very well. I know what 'Convex' is- a single global minimum & 'Concave' is the opposite. What does it mean when a parameter space is 'Convex'? Parameter space refers to the range of possible values a set of parameters can take. Do Convex & Concave refer to the minimum & maximum values of the parameter space?

Suppose I'm training a linear regression model using k-fold cross-validation. I'm training K times each time with a different training and test data set. So each time I train, I get different parameters (feature coefficients in the linear regression case). So I will have K parameters at the end of cross-validation. How do I arrive at the final parameters for my model? If I'm using it to tune hyperparameters as well, do I have to do another cross-validation after fixing …

Here is my understanding of the relation between MLE & Gradient Descent in Logistic Regression. Please correct me if I'm wrong: 1) MLE estimates optimal parameters by taking the partial derivative of the log-likelihood function wrt. each parameter & equating it to 0. Gradient Descent just like MLE gives us the optimal parameters by taking the partial derivative of the loss function wrt. each parameter. GD also uses hyperparameters like learning rate & step size in the process of obtaining …

If MLE (Maximum Likelihood Estimation) cannot give a proper closed-form solution for the parameters in Logistic Regression, why is this method discussed so much? Why not just stick to Gradient Descent for estimating parameters?

Let's say I have a top-down picture of an arrow, and I want to predict the angle this arrow makes. This would be between $0$ and $360$ degrees, or between $0$ and $2\pi$. The problem is that this target is circular, $0$ and $360$ degrees are exactly the same which is an invariance I would like to incorporate in my target, which should help generalization significantly (this is my assumption). The problem is that I don't see a clean way …

I'm working on image signal from a sensor where the incoming signal consist of high degree of constant noise. The noise patterns are multiple, both with very low frequency and very high frequency but not as high as gaussian or uniform noise. I want to retrieve the original signal from a set of images with as much of this noise removed. I'm thinking about trying to formulate a method which is modified Independent Component Analysis (ICA). Standard noise removal procedures …

Let's say our data is discrete-valued and belongs to one of $K$ classes. The underlying probability distribution is assumed to be a categorical/multinoulli distribution given as $p(\textbf{x}) = \prod_{k = 1}^{K}\mu_{k}^{x_{k}}$ where x is a one-hot vector given as $\textbf{x} = [x_{1} x_{2} ... x_{K}]^{T}$ and $\boldsymbol{\mu} = [\mu_{1} ... \mu_{K} ]^{T}$ are the parameters. Suppose $D = \{\mathbf{x}_{1}, \text{ } \mathbf{x}_{2}, \text{ } ... ,\text{ }\mathbf{x}_{N}\}$ is our data. The log likelihood is: $\log p(D|\boldsymbol{\mu}) = \sum_{k = 1}^{K} …

My questions seems very trivial, but I can't quite grasp it. I am also aware this post asks for opinions and knowhow, but do not know were else to ask. I do have quite a lot of experience solving even somewhat difficult machine learning problems, but never faced a situation were the target is still a parameter in an other function. I have a function like: t = ax + by + cz + bias. And a database with t, …

I've been thinking about the difference between ML modelling and statistical modelling. I would to ask, on a philosophical level, is my thinking correct: modelling is basically a process of fitting a data-generating function to a set of data. Is this the case that in statistical modelling, we are explicitly finding a function that's expressible in parameters (in a manual way), but in ML modelling, we just automate this process, at the expense we can never write down explicitly a …

I want to use a neural network to predict the number of new cases of COVID-19. For the same, I have decided to use an SIRD (Susceptible-Infected-Recovered-Deceased) Model, which is parameterized by the transmission rate $\beta$, the recovery rate $\gamma_{r}$, and the fatality rate, $\gamma_{d}$. I want to make these parameters time-dependent, such that they can be changing everyday. I have a dataset containing S,I,R,D values for successive days and want to use these values to predict the SIRD Model …

I'm trying to automatize my autocorrelation study in Python. My question is: is it possible? Let me explain. I have a time series and I just learnt how to interpret the autocorrelation plot. My question is: given that I need to examine a hundred time series, is it possible to get a result from the data (and not look at plots at all)? Here's my whole python code, which returns some graphs (for just one time series). What do you …

Given a dataset $X$ consisted with $w|X|$ samples drawn from a mixture of multivariate Gaussian distributions (say in two dimensions) and $(1-w)|X|$ samples of noise, is there any Python, Julia, Matlab, Mathematica library that can perform a robust parameter estimation for the Gaussians? Above, $|X|$ refers to the total number of elements of the two dimensional $(x,y)$ dataset $X$ and $0\ll w <1$. The idea is to readily use such a library such as to estimate means and covariances in …

The likelihood function is defined as --> P(Data|Parameter) - This means, "The probability that the parameter would generate the observed data". Here, data refers to the independent variables. This makes no sense to me because we generate parameters from the data, not the other way round. Data remains constant. Can somebody explain clearly what P(Data|Parameter) exactly is?

First of all I profusely apologise for the lack of suitable ways to express myself, I lack the formal data science background and am trying to learn as I go along, so finding the right terminology is a bit tricky for me.. I'm trying to run MLE across a great number of data sets (basically a matrix of pixels), with the idea to predict the optimum parameter of a known model in each pixel. According to the literature in my …

I have three different histograms (Impact parameter distributions) corresponding to three groups of the same particle with different properties. However, the three distribution have more or less the same shape. Now I want to predict a fourth distribution (with another property), which should have more or less the same shape as the three other ones. I have only a part of this latter distribution in a certain range. A first attempt was to predict the rest of the distribution using …

Problem: I had planned to use a linear regression model to model time series data in retrospect (i.e., no forecasting). However, I am wondering if this is the best option having come across a few posts - (https://www.quora.com/Is-regression-analysis-legitimate-for-time-series-data) - that regression analysis might not be legitimate for time series data. Preliminary plotting also shows a concave shape in the data, but this would still be a regression model, I think. Question: Would anyone have any good sources to link to …

The standard survival analysis model - for example the model which forms the basis for the proportional hazards model - assumes the hazard rate is constant. In many applications this would be the exception rather than the rule. What parametric model would be appropriate for data such as this: % retention 70% 80% 85% 90% 90%

I am writing some Python code to fit 2D Gaussians to fluorescent emitters on a dark background to determine the subpixel-resolution (x, y) position of the fluorescent emitter. The crude, pixel-resolution (x, y) locations of the pixels are stored in a list xy. The height of the Gaussian represents the predicted pixel intensity at that location. Each 2D Gaussian has 5 parameters, and my end goal is to find the optimal value of those 5 parameters for each peak using …

I have a differential equation: def func(Y, t, r, p, K, alpha): return r * (Y ** p) * (1 - (Y / K) ** alpha) and I want to find the best parameters that fit (r,p,K,alpha). I tried to use curve fit but it was too bad, this is my code # I chose the value of maxfev randomly popt, pcov = curve_fit(func, df.index, df.Value,method='lm',maxfev = 8000) t = np.linspace(0, len(df), len(df)) y0 = popt[0] params = (popt[1], popt[2], …