Questions tagged [ranking]

Ranking is ordering data from highest to lowest or vice versa. For questions about constructing scores to use in ranking, please use the "valuation" tag, too.

Ranking means ordering data from highest to lowest or vice versa. Statistical "ranks" are normally given the values 1, 2, 3, ..., up to the sample size, assigned (respectively) to values or attributes which have the highest (or lowest) rank, the second highest (/lowest) rank, and so on.

A standard notation is to bracket subscripts when denoting ranks: that is, for a batch of numbers $(x_1, x_2, \ldots, x_n)$ (written in any order), the sorted values are written $(x_{[1]}, x_{[2]}, \ldots, x_{[n]})$.

Since the original values being ranked can sometimes be tied (have a weak order), tied ranks can occur where more than one rank is to be assigned to indistinguishable values. This issue is resolved in various ways, but commonly in statistics the average rank allocated to the tied (indistinguishable) observations is assigned to each of them. Allocating average ranks is sometimes called fractional ranking.

Reference: Wikipedia: Ranking in statistics

117 questions
26
votes
2 answers

How fit pairwise ranking models in XGBoost?

As far as I know, to train learning to rank models, you need to have three things in the dataset: label or relevance group or query id feature vector For example, the Microsoft Learning to Rank dataset uses this format (label, group id, and…
tokestermw
  • 418
  • 1
  • 4
  • 8
11
votes
1 answer

What is query id ("qid") in XGBoost

In XGBoost documentation it's said that for ranking applications we can specify query group ID's qid in the training dataset as in the following snippet: 1 qid:1 101:1.2 102:0.03 0 qid:1 1:2.1 10001:300 10002:400 0 qid:2 0:1.3 1:0.3 1 qid:2 0:0.01…
Konstantin
  • 153
  • 1
  • 9
7
votes
4 answers

From pairwise comparisons to ranking - python

I have to solve a ranking ML issue. To start with, I have successfully applied the pointwise ranking approach. Now, I'm playing around with pairwise ranking algorithms. I've created the pairwise probabilities (i.e. probability of item i being above…
Stergios
  • 320
  • 1
  • 3
  • 11
7
votes
1 answer

Machine learning model for ranking that outputs probabilities

Traditionally ML algorithms for ranking take the features as input and then output a "ranking score" which do not have a natural probabilistic interpretation. For example, suppose we have three laptops: "macbookAir", "macbookPro", "msSurface", and a…
6
votes
2 answers

precision@k and recall@k

Normally, I am familiar with precision and recall evaluation metrics but as you know recall@k and precision@k are different things and used in ranking evaluations especially recommendation systems. I checked many sources, I understood everything I…
drorhun
  • 161
  • 1
  • 1
  • 6
6
votes
2 answers

Preparing for a Machine Learning Design Interview

I am not sure if this is a relevant post here but: I made it to the final round for the Machine Learing Engineer position at Facebook. The final round interview is virtual (thanks to Corona) and will consist of: 2 - General Algorithmic Coding…
Wolfy
  • 237
  • 2
  • 9
6
votes
1 answer

How to use ndcg metric for binary relevance

I am working on a ranking problem to predict the right single document based on the user query and use the NDCG metric to measure the model. Given the details : Queries ( Q ), Result Document ( D ), Relevance score. But the relevance score is a…
5
votes
1 answer

Methods for ensembling ranked lists?

I was wondering if there's a good way to use ensembling when I have two or more algoritims producing ranked lists. That is, suppose I have the following datasets consisting of ordered lists (higher to the top means more relevant): Method1_Rankings …
user3490622
  • 151
  • 2
5
votes
2 answers

Converting non-numeric data values into equivalent rank scores

Consider a data-frame similar to the one shown (the actual data-frame is much larger) ID EDUCATION OCCUPATION BINARY_VAR 1 Undergrad Student 1 2 Grad Business Owner 1 3 Undergrad Unemployed 0 4 PhD…
neural-nut
  • 1,803
  • 3
  • 18
  • 28
4
votes
1 answer

How is "relevance" defined in information retrieval outside the context of systems with user feedback?

I've seen information retrieval systems that return some results from a query, and then the user rates these results as either "relevant" or "not relevant". What can you do if you do not have user feedback? E.g. suppose your system returns some…
4
votes
1 answer

Rank players of any given sport

I've recently become interested in possibly of developing some sort of method for ranking athletes of sports such as American football and determining which players are better than others in terms of specific statistics. My thoughts are that there…
4
votes
1 answer

Can feature importance change a lot between models?

I have a random forest classifier and Multinomial Naive Bayes. For feature importance, I used gini index for random forest and for Multinomial Naive Bayes I used the coefficients of each feature. Then normalized to compare the two lists but there is…
4
votes
5 answers

Machine learning algorithm for ranking

I am working on a ranking question, recommending k out of m items to the users. The evaluation metric is average precision at K. Both R and Python have xgboost can be used for pairwise comparison and can be adapted for ranking problems. Are there…
user7117436
  • 308
  • 4
  • 12
4
votes
0 answers

How Does the Reward Model in ChatGPT Calculate Losses?

Reading the InstructGPT paper(which seems to be what ChatGPT was built off of), I found this equation for the reward function. However, I'm struggling to understand how this equation is used to translate rankings between different response to a…
3
votes
2 answers

How could PageRank be used to rank paragraphs in relevance to keywords?

I have a data set with keywords describing paragraphs in a car manual and the actual paragraphs. I want to rank those paragraphs by that keyword using PageRank algorithm. How would I rank these paragraphs in relevance to the keywords? The keywords…
yogino
  • 31
  • 2
1
2 3 4 5 6 7 8