Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [early-stopping]

19 questions
19
votes
3 answers

What is the proper way to use early stopping with cross-validation?

I am not sure what is the proper way to use early stopping with cross-validation for a gradient boosting algorithm. For a simple train/valid split, we can use the valid dataset as the evaluation dataset for the early stopping and when refitting we…
amine456
  • 191
  • 1
  • 1
  • 4
8
votes
1 answer

Keras EarlyStopping callback: Why would I ever set restore_best_weights=False?

The point of EarlyStopping is to stop training at a point where validation loss (or some other metric) does not improve. If I have set EarlyStopping(patience=10, restore_best_weights=False), Keras will return the model trained for 10 extra epochs…
codeananda
  • 318
  • 5
  • 12
8
votes
1 answer

Keras Early Stopping: Monitor 'loss' or 'val_loss'?

I often use "early stopping" when I train neural nets, e.g. in Keras: from keras.callbacks import EarlyStopping # Define early stopping as callback early_stopping = EarlyStopping(monitor='loss', patience=5, mode='auto',…
Peter
  • 7,896
  • 5
  • 23
  • 50
3
votes
1 answer

Early stopping with class weights / sample weights

I'm performing a classification of imbalanced multiclass data using a Neural Network in the TensorFlow framework. Therefore, I'm applying class weights. I would like to apply early stopping to reduce overfitting. My concern is that the cost of the…
Igal L
  • 31
  • 2
3
votes
1 answer

What is the purpose of EarlyStopping returning last epoch's weights by default?

I recently realized that keras callback for early stopping returns the last epoch's weights by default. If you want to do otherwise you can use the argument restore_best_weights=True, as stated for example in this answer or documentation. I'm quite…
Luca Clissa
  • 135
  • 7
2
votes
1 answer

When to stop the final model training?

Let's say I'm participating in a Kaggle image recognition competition. Firstly, I create a train/validation split and find the good hyperparameters for my model. Here the stopping criterion is when the validation loss stops decreasing and starts…
SpaceCossack
  • 21
  • 1
1
vote
0 answers

Can we used both cross validation/nested cross validation technique and early stopping with patient at the same time?

Can we use both cross validation/nested cross validation technique and early stopping with patient at the same time? Using early stopping for each (training, validation) fold and get best result of each (training, validation) fold and finally get…
Hoang Le
  • 11
  • 1
1
vote
1 answer

NGBoost and overfit - which model is used?

While training an NGBoost model I got: [iter 0] loss=-2.2911 val_loss=-2.3309 scale=2.0000 norm=1.0976 [iter 100] loss=-3.3288 val_loss=-2.8532 scale=2.0000 norm=0.7841 [iter 200] loss=-4.0889 val_loss=-1.5779 scale=2.0000 norm=0.7544 [iter 300]…
user2182857
  • 167
  • 5
1
vote
1 answer

EarlyStopping based on the loss

When training my CNN model, based on the random initialization of weights, i get the prediction results. In other words, with the same training and test data i get different results every time when i run the code. When tracking the loss, i can know…
phillipe cauchett
  • 45
  • 5
1
vote
1 answer

Daily new data for my neural network, and I want transfer(?) learning

I made my neural network, it is pre-trained for 180 days of data. It filters the fraud data of credit cards everyday and 1-days new data is comming in. And I also want after the filtering, I want to re-train my AI-model but I just want to use new…
INNO TECH
  • 139
  • 4
1
vote
1 answer

Keras CNN early stopping not working as expected with patience parameter in imbalanced datasets

So I want to stop the cnn when a custom (not implemented in keras) logged metric is not improving with a patience of 5 (I chose macro f1 score) and here's what I did: Created a callback to log the macro f1 score on epoch end and an early…
giza2001s
  • 11
  • 1
1
vote
0 answers

Avoid overfitting to noise by a noise penalty approach instead of early stopping?

I came across this article on deep learning for computational MRI and found an interesting sentence "However, early stopping has to be performed to not overfit to the noisy measurements." which made me think, in section III "Physics-driven ML…
Shihao ZENG
  • 11
  • 1
1
vote
2 answers

Strategy to choose maximum value from an unknown array of n numbers

Suppose you have an array of n normally distributed numbers whose values are initially unknown(and the probability parameters are unknown too). You must choose one number and you want it to have maximum value. You examine the numbers one number at a…
AutisticRat
  • 123
  • 1
  • 6
1
vote
0 answers

Is Callback / early stopping and validation set is not mandatory

I just noticed that in mostly github repositry of research papers they didnt implemented early stopping criteria and they didnt use validation set but whats the reason behind this?
user12
  • 171
  • 1
  • 8
1
vote
1 answer

Keras: How to restore initial weights when using EarlyStopping

Using Keras, I setup EarlyStoping like this: EarlyStopping(monitor='val_loss', min_delta=0, patience=100, verbose=0, mode='min', restore_best_weights=True) When I train it behaves almost as advertised. However, I am initializing my model weights …
ruminator
  • 113
  • 3
1
2