Linear Discriminant Analysis + bayesian theorem = LDA classifier??

Question

Linear Discriminant Analysis + bayesian theorem = LDA classifier??

Kid_Learning_C

2018年10月21日 06:42

I am new to machine learning and as I learn about Linear Discriminant Analysis, I can't see how it is used as a classifier.

I can understand the difference between LDA and PCA and I can see how LDA is used as dimension reduction method. I've read some articles about LDA classification but I'm still not exactly sure how LDA is used as classifier.

From what I understand, we consider the features vector x as multivariate gaussian distribution and use the bayesian rule to calculate which category gives the max probability for x, then it belongs to that category.

Is this understanding generally correct?

Topic lda-classifier bayesian classification

Category Data Science

Kid_Learning_C · Accepted Answer · 2018年10月21日 06:42

I found the answer in: https://machinelearningmastery.com/linear-discriminant-analysis-for-machine-learning/

Making Predictions with LDA

LDA makes predictions by estimating the probability that a new set of inputs belongs to each class. The class that gets the highest probability is the output class and a prediction is made.

The model uses Bayes Theorem to estimate the probabilities. Briefly Bayes’ Theorem can be used to estimate the probability of the output class (k) given the input (x) using the probability of each class and the probability of the data belonging to each class:

P(Y=x|X=x) = (PIk * fk(x)) / sum(PIl * fl(x))

Where PIk refers to the base probability of each class (k) observed in your training data (e.g. 0.5 for a 50-50 split in a two class problem). In Bayes’ Theorem this is called the prior probability.

PIk = nk/n

The f(x) above is the estimated probability of x belonging to the class. A Gaussian distribution function is used for f(x). Plugging the Gaussian into the above equation and simplifying we end up with the equation below. This is called a discriminate function and the class is calculated as having the largest value will be the output classification (y):

Dk(x) = x * (muk/siga^2) – (muk^2/(2*sigma^2)) + ln(PIk)

Dk(x) is the discriminate function for class k given input x, the muk, sigma^2 and PIk are all estimated from your data.

So yes, it uses Bayes Theorem to estimate the probabilities.

Linear Discriminant Analysis + bayesian theorem = LDA classifier??

About