7

Traditionally ML algorithms for ranking take the features as input and then output a "ranking score" which do not have a natural probabilistic interpretation.

For example, suppose we have three laptops: "macbookAir", "macbookPro", "msSurface", and a bunch of customers preference as data and pass it to ML algorithms like XGBoostRanker, then it outputs a score for each of the three laptops and the laptop with the highest score is the one that the customer is most likely to buy.

I am trying to identify a ML technique that outputs a ranking probability instead. i.e. it outputs

p_123 = P("macbookAir" < "macbookPro" < "msSurface")

p_132 = P("macbookAir" < "msSurface" < "macbookPro")

p_213 = P("macbookPro" < "macbookAir" < "msSurface")

p_231 = P("macbookPro" < "msSurface" < "macbookAir")

p_312 = P("msSurface" < "macbookAir" < "macbookPro")

p_321 = P("msSurface" < "macbookPro" < "macbookAir")

such that p_123+p_132+p_213+p_231+p_312+p_321=1

I would like to ask is there any ML algorithm that does that? If not, is there any way to convert the score from ranking algorithms such as XGboost ranker to the ranking probabilities as above?

Edit: In my original problem, there are many different classes. I understand it becomes exponentially difficult as the number of classes n increases there will be n! possible rankings. But in my problem I only care about the top few classses. So I would like to ask is there at least a "partial ranking probability" where we can predict the probabilities P(A>B>others) so as to reduce the level of difficulty.

Thank you so much in advance.

Ishigami
  • 173
  • 5

1 Answers1

3

I would use the items being ranked as features, along with whatever other features you have, and then the outcome is if the user prefers the first of those “item” features or not. This has the advantage of fitting into a huge number of model types, just regular supervised learning. You can predict probabilities using random forests, neural networks, or logistic regression, to name a few possibilities. If you have to calibrate the probabilities in a post-processing step, that would be reasonable (keeping in mind that models do not automatically return calibrated probabilities).

When you want to make predictions, just pick the categories you want to compare, keeping in mind which feature is the “preferred to” category.

I am not convinced that this will result in a “rational” preference relation the way economics defines preferences, even if you have good calibration. You may wind up with A preferred to B, B preferred to C, and C preferred to A, for instance.

Dave
  • 4,542
  • 1
  • 10
  • 35