What kind of "vector" is a feature vector in machine learning?

Question

What kind of "vector" is a feature vector in machine learning?

Serge Desmedt

2018年11月14日 19:34

I'm having trouble understanding the use of Vector in machine learning to represent a group of features.

If one looks up the definition of a Vector, then, according to wikipedia, a Vector is an entity with a magnitude and direction.

This can be understood when applying Vectors to for example physics to represent force, velocity, acceleration, etc...: the components of the Vector represent the components of the physical property along the axes in space. For example, the components of a velocity vector represent the velocity along the x, y and z axes

However, when applying Vectors to machine learning to represent features, then those features can be totally unrelated entities. They can have totally different units: one feature can be the length in meters of a person and another can be the age in years of the person.

But then what is the meaning of the Magnitude of such a Vector, which would then be formed by a summation of meters and years? And the Direction?

I do know about normalization of features to make them have similar ranges, but my question is more fundamental.

Topic featurization machine-learning

Category Data Science

thanatoz · Accepted Answer · 2018年11月14日 19:34

Vectors have perspective form the point of view of Mathematics, Physics and Computer science.

I suggest you to go through Grant Sanderson's video on vectors on his channel 3BLUE1BROWN or rather go through his entire series on ESSENCE OF LINEAR ALGEBRA, for a better visual understanding of Linear Algebra.

Talking about feature vectors, they are nothing but a collection of all the features (Individial property or characteristic of a phenomenon being observed) arranged in a specific way. It is an n-dimensional vector of numerical features that represent some object that is required by the machine learning algorithms. Just go through this Wikipedia article from where I have written about feature vectors.

Atani · Accepted Answer · 2018年11月14日 15:04

I'm having trouble understanding the use of Vector in machine learning to represent a group of features.

In short, I would say that "Features Vector" is just a convenient way to speak about a set of features.

Indeed, for each label 'y' (to be predicted), you need a set of values 'X'. And a very convenient way of representing this is to put the values in a vector, such that when you consider multiple labels, you end up with a matrix, containing one row per label and one column per feature.

In an abstract way, you can definitely think of those vectors belonging to a multiple dimensions space, but (usually) not an Euclidean one. Hence all the math apply, only the interpretation differs !

Hope that helps you.

Andreas Look · Accepted Answer · 2018年11月14日 08:06

First lets talk about how to organize your data.Lets assume you organize your in a spreadsheet, where columns represent your features and rows your different samples. Imagine you asked 3 people about their sex and age, then you get a spreadsheet with 3 rows (3 people) and 2 columns (sex, age).

Now you can interpret every row as a single feature vector. In our example case the feature vector would have 2 dimensions (sex, age). Instead of physics the (euclid.) magnitude of the feature vector might have no direct usage for us, since the dimensions come from different domains (in contrast compare a velocity vector). Nevertheless we could compute the magnitude (after normalization). On the other hand the direction of the feature vector is important, since it represents the feature values itself.

All in all feature vectors should not directly be interpreted as you do in physics.

What kind of "vector" is a feature vector in machine learning?

About