Machine learning with constraints on features

I am working on a learning to rank problem. I have queries and documents related to every query which I have to rank. I used lightgbm ranker to fit the model. Some of features are very important and if they are changed the fitted model predicts a better score for that document and thus a better rank.

Lets say, for a single query id, I have a group of documents d1....d5 each having features f1...fn. I change the features f1,f2,f3 for document d4 and it pushes the rank of d4 from 4 to 3.

  1. What should be the values of these features f1,f2,f3 that can push the rank for document d4 from 4 to 1 subject to the condition that f1+f2+f3=45 is always satisfied ? (i.e. the sum of new modifications for f1,f2,f3 should be always equal to 45 )

As far as I understand this problem is simply a constrained optimization problem. In such a problem we have an objective function and constraints. In my case I do not have a mathematical equation that relates rank to my features r=f(f1....fn) but I do have a learned model which I can use as an objective function.

Follow up question

All this made me think of another question. In the case of OLS, which is a constrained optimization problem, the constraints present in this kind of optimization are always on the coefficients.

  1. If I would like to have constraints on the predictors does it really make sense ? How does it relate to my problem ?

I think the objective function of a regression problem and the objective function of constrained optimization problem i mentioned above are different. I would really appreciate some thoughts on this, thanks !

Topic regression optimization machine-learning

Category Data Science


To be honest I didn't understand the problem very well, but it looks like genetic learning could be a good direction to study.

Essentially a genetic algorithm works by iteratively trying many combinations of values for some features, evaluating "something" based on the features, selecting the best combinations and starting again with a combination of the best features so far. The goal is to eventually converge to an optimal combination of values.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.