How to determine the function is linear in linear regression problem?

Question

How to determine the function is linear in linear regression problem?

Animesh Kumar Paul

2019年5月5日 14:01

I know that the first degree of the polynomial equation is considered as a linear function.

But, I found some things confusing in linear regression.

1. f(x)= w1 x1+ w2 x2 + W3 x3 -- linear function
2. f(x)= w1 x1+ w2 x2 + W3 x1 x3 -- is it linear? if not, then why?
3. f(x)= w1 x1+ w2 x2 + W3 W4 x3 -- is it linear? if not, then why?
4. f(x)= w1 x1+ w2 x2 + W3 x3 x3 -- it is not linear.

Please help.

Topic objective-function linear-regression

Category Data Science

Anshul G. · Accepted Answer · 2018年11月3日 18:27

Your third example is certainly linear as we can simply imagine $W3.W4$ to be some $W5$.

Regarding your other 2 examples, you need to clarify what your input is. If your input is $f(x1, x2, x3)$ then they are not linear. However, say in the second example if the input to your function is $f(x1,x2,x1x3)$ then it is still a linear function!

You can refer to @Miguel's answer to test it out.

Miguel · Accepted Answer · 2018年11月3日 18:11

A function $x = (x_1, \dots, x_n) \mapsto f(x_1, \dots, x_n) \in \mathbb{R}$ is linear if and only if for every $a,b \in \mathbb{R}$ it holds that $f(a (x_1, \dots, x_n) + b (\xi_1, \dots, \xi_n)) = a f(x_1, \dots, x_n) + b f(\xi_1, \dots,\xi_n)$.

In your second example try computing $f(a x + b \xi)$ to see that the last term $w_3 x_1 x_3$ will make $f(a x + b \xi) \neq a f(x) + b f(\xi)$.

The third example has a coefficient which is expressed as a product, but the function remains linear (just rename $w_3 w_4$ to $w$).

How to determine the function is linear in linear regression problem?

About