How do standardization and normalization impact the coefficients of linear models?

I believe with scaling, the coeff. are scaled by the same level i.e. Std. Deviation times with Standardization and (Max-Min) times with Normalization

If we look at all the features individually, we are basically shifting it and then scaling it down by a constant but $y$ is unchanged.

So, if we imaging a line in a 2-D space, we are keeping the $y$ same and squeeing the $x$ by a constant(Let's assume it = $C$).

This implies(Assuming Coeff. = Slope = $tan{\theta}$ = dy/dx) ,
the slope will also increase by the same amount i.e. $C$ times. (Since, dx has been divided by a constant($C$) but dy is same, so $tan{\theta}$ i.e. slope = $C$ * old_slope(i.e. the slope prior to scaling)

We can observe in this snippet that both the coef are in the ratio of the Standard deviation and (Max - Min) respectively w.r.t the unscaled coeff

import sys;import os;import pandas as pd, numpy as np
os.environ['KAGGLE_USERNAME'] = "10xAI" 
os.environ['KAGGLE_KEY'] = "<<Your Key>>" 

import kaggle
!kaggle datasets download -d camnugent/california-housing-prices

dataset = pd.read_csv("/content/california-housing-prices.zip")
y = dataset.pop('median_house_value')
x = dataset.iloc[:,:4]
from sklearn.linear_model import LinearRegression
model = LinearRegression()
model.fit(x,y)
old_coef = model.coef_  

x_s = (x-x.mean())/x.std()
model.fit(x_s,y)
std_coef = model.coef_  

print("###Ratio of Scaled Coeff and Std. Deviation times Standardized Coeff")
print(std_coef/(old_coef*x.std()))

x_n = (x-x.min())/(x.max()-x.min())
model.fit(x_n,y)
nor_coef = model.coef_  

print("###Ratio of Scaled Coeff and (Max - Min) times Normalized Coeff")
print(nor_coef/(old_coef*(x.max()-x.min())))

So, you can calculate the unscaled Coeff from Standardized and Normalized coeff.

On Importance

The order(Since it's sorted values) might change because the standard deviation will not be equal to (Max - Min).

But this should not impact the importance. Importance should be measured in original data space Or unit should be of standard deviation(as explained by Peter) Or (Max - Min) but that might be not very intuitive for every user.

When you have a linear regression (without any scaling, just plain numbers) and you have a model with one explanatory variable $x$ and coefficients $\beta_0=0$ and $\beta_1=1$, then you essentially have a (estimated) function:

$$y = 0 + 1x .$$

This tells you that when $x$ goes up (down) by one unit, $y$ goes up (down) by one unit. In this case it is just a linear function with slope 1.

Now when you scale $x$ (the plain numbers) like:

scale(c(1,2,3,4,5))
           [,1]
[1,] -1.2649111
[2,] -0.6324555
[3,]  0.0000000
[4,]  0.6324555
[5,]  1.2649111

you essentially have different units or a different scale (with mean=0, sd=1).

However, the way OLS works will be the same, it still tells you "if $x$ goes up (down) by one unit, $y$ will change by $\beta_1$ units. So in this case (given a different scale of $x$), $\beta_1$ will be different.

The interpretation here would be "if $x$ changes by one standard deviation...". This is very handy when you have several $x$ with different units. When you standardise all the different units, you make them comparable to some extent. I.e. the $\beta$ coefficients of your regression will be compareable in terms of how strong the variables impact on $y$ is. This is sometimes calles Beta-Coefficients or Standardised Coefficients.

A very similar thing happens when you normalise. In this case you will also change the scale of $x$, so the way how $x$ is measured.

Also see this handout.