Questions tagged [auc]
69 questions
8
votes
2 answers
AUC-ROC for Multi-Label Classification
Hey guys I'm currently reading about AUC-ROC and I have understood the binary case and I think that I understand the multi-classification case. Now I'm a bit confused on how to generalize it to the multi-label case, and I can't find any intuitive…
NotoriousFunk
- 81
- 1
- 1
- 3
7
votes
1 answer
Micro Average vs Macro Average for Class Imbalance
I have a dataset consisting of around 30'000 data points and 3 classes. The classes are imbalanced (around 5'000 in class 1, 10'000 in class 2 and 15'000 in class 3). I'm building a convolutional neural network model for classification of the data.…
machinery
- 246
- 2
- 9
7
votes
1 answer
How to choose between different models with similar results? RF, GLM and XGBoost
I am a medical doctor trying to make prediction models based on a database of approximately 1500 patients with 60+ parameters each.
I am dealing with a classification problem (mortality at 1, 3, 6 and 12 months) and have made stratified splits (70…
user145725
- 71
- 1
5
votes
2 answers
Confused AUC ROC score
I am working on binary classification problem, I try to evaluate the performance of some classification algorithms (LR,Decission Tree , Random forest ...).
I am using a cross validation technique (to avoid over-fitting) with AUC ROC as scoring…
Ak.tech
- 103
- 1
- 6
4
votes
2 answers
Is the PR AUC invariant under label flip?
The ROC-AUC curve is invariant under a flip of the labels. I don't know if its a famous result so I will give the proof below. My question is if the PR-AUC curve also has this property. I have not been able to prove or disprove it yet.
The reason…
Borun Chowdhury
- 197
- 1
- 8
4
votes
0 answers
Fast PR / ROC curves and corespondings AUPR / AUROC
I find myself in a position of calculating numerous PR / ROC curves and their associated area under the PR curves (AUPR) / area under the ROC curve (AUROC).
Its is quite easy to perform those calculations with standards implementations (I am using…
Lucas Morin
- 2,775
- 5
- 25
- 47
3
votes
1 answer
At what stage are ROC curves used when building machine learning model?
When developing a machine learning model, at what stage are ROC curve with AUC used?
Typically I have three data sets
train - validation - final test
I do K-Fold cross validation using the combined train + validation set
During that phase we can…
erotavlas
- 131
- 4
3
votes
3 answers
What is AUC - ROC Curve?
AUC - ROC curve is a performance measurement for classification problem at various thresholds settings. ROC is a probability curve and AUC represents degree or measure of separability.
Is Roc the same as AUC?
user87246
3
votes
1 answer
Area Under the Precision Recall Curve
I have got the following Precision Recall Curve for a classifier I built using AutoML. Most of the Precisio-Recall curves tend to start from (0, 1) go towards (1,0). But mine is the opposite. But I feel like, similar to the ROC curve it is actually…
user77005
- 151
- 4
3
votes
1 answer
Xgboost Multiclass evaluation Metrics
Im training an Xgb Multiclass problem, but im having doubts about my evaluation metrics,
heres my code + output
import matplotlib.pylab as plt
from sklearn import metrics
from matplotlib import pyplot
from sklearn.model_selection import GridSearchCV…
Chichostyle
- 31
- 1
- 2
2
votes
0 answers
AUC on ROC Curve near 1.0 for Multi-Class CNN but Precision/Recall are not perfect?
I am building a ROC Curve and calculating AUC for multi-class classification on the CIFAR-10 dataset using a CNN. My overall Accuracy is ~ 90% and my precision and recall are as follows:
precision recall f1-score support
…
Coldchain9
- 159
- 5
2
votes
2 answers
Don't understand why I get an inverse ROC curve for SVM (Python)
I build an SVM classifier but get an inverse ROC curve. The AUC is only 0.08. I've used the same datasets to build a Logistic Regression classifier and a Decision Tree classifier, and the ROC curves for them look good.
Here are my codes for…
MMMMMay
- 31
- 1
- 3
2
votes
2 answers
Main options on how to deal with imbalanced data
As far as I can tell, broadly speaking, there are three ways of dealing with binary imbalanced datasets:
Option 1:
Create k-fold Cross-Validation samples randomly (or even better create k-fold samples using Stratified k-fold:…
Newbie
- 21
- 4
2
votes
0 answers
Connection between prob output LogisticReg/SVM and ROC
I have the following ROC generated using LPOCV and Logistic regression or SVM (l2 norm). Now, let's say I have a test set containing 10 patients and I get that the probabilities of those patients to be sick range from 0-100% (of course). What I want…
Luis Pinto
- 31
- 3
2
votes
2 answers
How much can the AUC improve comparing the raw dataset and the feature engineered dataset?
Let's say I put the following two datasets in the best possible model (same model for both):
A raw dataset, the variables as they came just from the query.
A feature-engineered dataset, with hundreds of created variables, which came from the same…
Juan Esteban de la Calle
- 2,318
- 9
- 28