Learning to rank: how is the label calculated?
I am studying learning to rank and not sure I understand how the train sample and final label (relevance score) is constructed.
Lets assume we sell furniture online. We have logged customer's query, product customer bought, clicked.
Example:
User A searched for "red sofa", clicked on Pa(r=1), Pb(r=5), Pc(r=10) and bought Pc.
User B searched for "red sofa", clicked on Pa(r=1), Pd(r=2), Pe(r=6) and bought Pd.
User C searched for "blue chair", clicked on Pu(r=1), Po(r=2), Ps(r=6) and bought Pu.
(r stands for absolute position from top to bottom of a product). (lets say for clicking we would add 1, for buying we would add 2)
Case 1: Without any personalization, I just want to rank the products only based on textual data from the product and query. How do I construct my training data?
Two users searched for "red sofa", do I treat those separate, e.g per user per query?
user query product relevance some_features
A "red sofa" Pc 2
A "red sofa" Pa 1
A "red sofa" Pb 1
A "red sofa" other products 0
B "red sofa" Pd 2
B "red sofa" Pa 1
B "red sofa" Pe 1
B "red sofa" other products 0
C "blue chair" Pu 2
C "blue chair" Po 1
C "blue chair" Ps 1
C "blue chair" other products 0
This looks to me a bit weird because how should an algorithm understand how products Pc or Pd (bought by user A and user B) should be ranked compared to each other?
Or do I treat logs for the same query together, e.g aggregating per query? (Because user A and B clicked both on Pa, it got a relevance of 2=1+1)
query product relevance some_features
"red sofa" Pc 2
"red sofa" Pa 2
"red sofa" Pb 1
"red sofa" Pd 2
"red sofa" Pe 1
"red sofa" other products 0
"blue chair" Pu 2
"blue chair" Po 1
"blue chair" Ps 1
"blue chair" other products 0
This also looks weird to me because if users click the same products but buy different ones, the relevance of clicked products will be higher.
Question 1: How should I represent a label and a training sample in this case?
Case 2: With personalization. If I want to provide user's features like "clicked before", "put into favorites", I would indeed construct a training record per user, query.
Question 2: Is it correct to do so?
Topic learning-to-rank learning
Category Data Science