Questions tagged [precision-recall-curve]
13 questions
2
votes
2 answers
Plotting a no-skill model in a precision-recall curve
I am following this tutorial to apply threshold tuning using precision-recall curve for an imbalanced dataset
Within the tutorial, a no-skill model is defined as:
A no-skill model is represented by a horizontal line with a precision
that is the…
sums22
- 447
- 5
- 15
1
vote
3 answers
ROC vs PR-score and imbalanced datasets
I can see everywhere that when the dataset is imbalanced PR-AUC is a better performance indicator than ROC. From my experience, if the positive class is the most important, and there is higher percentage of positive class than the negative class in…
Vicky
- 21
- 3
1
vote
0 answers
Threshold strategies in determining precision and recall
I have been reading Jurafsky's Chapter 14: Question Answering, Information Retrieval, and RAG section on precision and recall and followed up with this video to understand the 2 metrics, interpolation and average precision.
This videoEvaluation 9:…
heretoinfinity
- 135
- 5
1
vote
0 answers
How to improve precision/recall on multiclass classifier
I am working on an image recognition algorithm to classify images of starch granules to their source plant species. My model right now has 10 classes (plant species). Each class is trained with 600 images, then validated with a different 400 images…
Steven Andrew
- 11
- 1
1
vote
1 answer
Why is Precision-Recall AUC different from Average Precision score?
I have been calculating the area under the Precision-Recall curve (AUPRC) using the code snippet below:
from sklearn import metrics
precision, recall, threshold = metrics.precision_recall_curve(y_true, y_pred)
prec_rec_auc_score =…
chilipepper
- 11
- 4
1
vote
0 answers
PR AUC curve with drop in precision
I have this PR AUC plot, with both PCA and autoencoder related curves having a huge drop of precision in the beginning and then increasing again, with PCA hitting 0 as you can see in the zoomed in image.
Is this correct? What could be the cause for…
GabrielPast
- 11
- 2
1
vote
1 answer
Precision vs probability
Say I have a model which predicts a class $C_i$ from an input $X$, with a probability of 0.95 i.e $P(C_i| X)=0.95$. That would mean that if we do this over and over, then 95/100 times we would be correct.
Having a model with a precision of 0.95 (for…
CutePoison
- 520
- 3
- 10
0
votes
0 answers
Evaluate metric over all batches in tensorflow
I have written a metric for tensorflow which is the area under the Precision-Recall curve to the left of the recall=0.3 threshold. The implementation is (notice I only care about the zeroth prediction of the last timestep in which window):
def…
GaloisFan
- 111
- 3
0
votes
1 answer
Is there a way to focus mainly on high precision when fitting a tree model?
I have a dataset with 95% false and 5% true labels, some 200000 samples overall, I'm fitting a LightGBM model. I mainly need to focus on high precision and have low number of false positives, I don't care for accuracy much.
I have tried playing…
Fireant
- 1
- 1
0
votes
2 answers
How to improve accuracy on a single class out of 3 classes in model
I am training a classification model with 3 classes using a deep neural network.
The classes have been resampled and balanced.
I have around 600000 samples... equally distributed.
The dataset is also divided equitably in the train/test/validation…
Th3Nic3Guy
- 125
- 5
0
votes
1 answer
Can I use macro recall to check if my RF model is overfitting?
I have a dataset with 837377 observations (51% to train, 25% to validation and 24% to test) and 19 features.
I calculated the recall score using average macro for train, validation and test and obtained:
Train: 0.9981845060159042
Val:…
-2
votes
1 answer
How to increase , precision-recall value in your Deep learning model
I am getting good accuracy metrics around 80 with precision =66, recall =37, F1 =47. How can I improve precision, and recall metrics in anomaly detection scenarios.. any suggestions?
user12
- 171
- 1
- 8