Naive Bayes Classifier - Discriminant Function

Question

Naive Bayes Classifier - Discriminant Function

Goktug

2019年3月28日 19:01

To classify my samples, I decided to use Naive Bayes classifier, but I coded it, not used built-in library functions.

If I use this equality, I obtain nice classification accuracy: p1(x) p2(x) = x belongs to C1

However, I could not understand why discriminant functions produce negative values. If they are probability functions, I think they must generate a value between 0 and 1.

Is there anyone who can explain the reason ?

Topic naive-bayes-classifier discriminant-analysis classification

Category Data Science

Esmailian · Accepted Answer · 2019年3月28日 19:01

In Naive Bayes, for the case of two classes, a discriminant function could be $$D(\boldsymbol{x}) = \frac{P(\boldsymbol{x}, c=1)}{P(\boldsymbol{x}, c=0)}$$ which can be anywhere in $[0, +\infty)$, and decides $c=1$ if $D(\boldsymbol{x})>1$, $c=0$ otherwise, or it could be the logarithm of that value

$$d(\boldsymbol{x}) = \text{log}\frac{P(\boldsymbol{x}, c=1)}{P(\boldsymbol{x}, c=0)}=\text{log}P(\boldsymbol{x}, c=1)-\text{log}P(\boldsymbol{x}, c=0)$$ which can be anywhere in $(-\infty, +\infty)$ (handling zero probability as a special case), and decides $c=1$ if $d(\boldsymbol{x})>0$, $c=0$ otherwise.

As a side note, $P(\boldsymbol{x}, c=k)$ in Naive Bayes is calculated as $$P(\boldsymbol{x}, c=k)=P(c=k)\prod_{i=1}^{d}P(x_i|c=k)$$ or equivalently for log probabilities as $$\text{log}P(\boldsymbol{x}, c=k)=\text{log}P(c=k) + \sum_{i=1}^{d}\text{log}P(x_i|c=k)$$

Naive Bayes Classifier - Discriminant Function

About