Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [weighted-data]

63 questions
267
votes
10 answers

How to set class weights for imbalanced classes in Keras?

I know that there is a possibility in Keras with the class_weights parameter dictionary at fitting, but I couldn't find any example. Would somebody so kind to provide one? By the way, in this case the appropriate praxis is simply to weight up the…
Hendrik
  • 8,767
  • 17
  • 43
  • 55
36
votes
3 answers

xgboost: give more importance to recent samples

Is there a way to add more importance to points which are more recent when analyzing data with xgboost?
kilojoules
  • 463
  • 1
  • 4
  • 6
20
votes
2 answers

Sample Importance (Training Weights) in Keras

How do you add more importance to some samples than others (sample weights) in Keras? I'm not looking for class_weightwhich is a fix for unbalanced datasets. What I currently have is: trainingWeights which is the desired importance I want to give…
wacax
  • 3,500
  • 4
  • 26
  • 48
14
votes
1 answer

Why doesn't class weight resolve the imbalanced classification problem?

I know that in imbalanced classification, the classifier tends to predict all the test labels as larger class label, but if we use class weight in loss function, it would be reasonable to expect the problem to be solved. So why we need some…
user137927
  • 389
  • 1
  • 3
  • 11
12
votes
7 answers

How to apply class weight to a multi-output model?

I have a model with 2 categorical outputs. The first output layer can predict 2 classes: [0, 1] and the second output layer can predict 3 classes: [0, 1, 2]. How can I apply different class weight dictionaries for each of the outputs? For example,…
Gal Avineri
  • 366
  • 1
  • 2
  • 12
11
votes
1 answer

CNN - imbalanced classes, class weights vs data augmentation

I have a dataset with a few strongly imbalanced classes, eg. the smallest class is about 54 times smaller than the largest. Therefore, data augmentation in order to equalize the size of classes seems like a bad idea to me (in the example above each…
I.D.M
  • 175
  • 1
  • 10
8
votes
4 answers

Unbalanced class: class_weight for ML algorithms in Spark MLLib

In python sklearn, there are multiple algorithms (e.g. regression, random forest ... etc.) that have the class_weight parameter to handle unbalanced data. However, I do not find such parameter for the MLLib algorithms. Is there a plan of…
Edamame
  • 2,785
  • 5
  • 25
  • 34
7
votes
1 answer

Machine learning technique to calculate weighted average weights?

I'm just starting to investigate machine learning concepts, so I'm sorry if this question is very naive, but I'm hoping that it will be an easy one to answer! I have a document matching algorithm that individually calculates a match for each field…
SilentMiles
  • 173
  • 1
  • 1
  • 3
6
votes
3 answers

using sklearn class weight to increase number of positive guesses in extremely unbalanced data set?

Hi I have a poorly correlated and unbalanced data set I have to work with. The set is 2 classes, 0 has 96,000 values and 1 has about 200. When I run random forest or other methods I get an output like: precision recall f1-score support …
user3107977
  • 61
  • 1
  • 2
5
votes
3 answers

Understanding Weighted learning in Ensemble Classifiers

I'm currently studying Boosting techniques in Machine Learning and I happened to understand that in Algorithms like Adaboost, each of the training samples is given a weight depending on whether it was misclassified or not by the previous model in…
AnonymousMe
  • 235
  • 3
  • 8
5
votes
2 answers

How to use class_weight parameter for validation set?

I am using Keras' class_weight parameter to deal with an imbalanced class problem. I am doing this to define the weights : weights = class_weight.compute_class_weight('balanced',np.unique(trainY),trainY) then, in my network: model.add(LSTM(...,…
MysteryGuy
  • 190
  • 7
5
votes
1 answer

Purpose of weights in neural networks

I'm beginner at Neural Networks. After reading multiple articles on wikipedia, i've seen the term "weight" being used a lot, although it is a little confusing. I know, that before the inputs are summed and passed to activation functions, they are…
ShellRox
  • 409
  • 5
  • 12
5
votes
1 answer

Should class-weights take validation-set into account?

I need to calculate class-weights to train my deep learning model. In order to simulate real-world producing scenario as possible as I can, I have excluded the testing/infering dataset from which calculate the class-weights, because when my model is…
EvilRoach
  • 153
  • 4
4
votes
1 answer

Weighted loss functions vs weighted sampling?

For image classification tasks, is there a practical difference between using weighted loss functions vs. using weighted sampling? (I would appreciate theoretical arguments, experience or published papers, anything really.) Some details: By…
Jean-Pierre Coffe
  • 41
  • 1
  • 3
4
votes
1 answer

Range to define emotions

We are capturing emotions as survey responses. We need to assign values for the responses(emotions) for analysis purposes. Is there an optimum range that can be assigned to achieve this? (like from -100 to 100). An example of a question and a set of…
user33293
1
2 3 4 5