Distribution Shift vs Transfer Learning

Question

Distribution Shift vs Transfer Learning

Carlos Mougan

2022年5月6日 13:34

Transfer learning (TL) is a research problem in machine learning (ML) that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem [1]

Distribution Shift The conditions under which the system was developed will differ from those in which we use the system. [2]

I consider there is no difference between distribution shift and dataset shift. But between transfer learning and distribution shift? What are the differences?

Can we say that transfer learning is an intended distribution shift?

Topic transfer-learning distribution statistics machine-learning

Category Data Science

Christos Karatsalos · Accepted Answer · 2022年5月6日 13:34

Actually, Transfer Learning has a broader definition. It includes distribution shift (covariate shift, sample bias etc..) as well. You could apply a TL method using a model which has been trained on a dataset having different distribution from the test set. Transfer learning includes also shift in the distribution of the labels or the fine tuning of a model trained in a different task.

Please check the following survey https://ieeexplore.ieee.org/document/5288526

Brian Spiering · Accepted Answer · 2021年11月6日 12:50

Yes - One difference is between transfer learning and distribution shift is intention and knowledge of a different dataset.

There are many types of transfer learning. Sometimes models are trained on one dataset and applied to another dataset without additional training. This has to be the case when there are no labels on the second dataset. Other times models are trained on one dataset and then fined tuned on another dataset. This can be the case when the second data has labels.

Distribution Shift vs Transfer Learning

About