Questions tagged [entropy]
5 questions
113
votes
9 answers
When should I use Gini Impurity as opposed to Information Gain (Entropy)?
Can someone practically explain the rationale behind Gini impurity vs Information gain (based on Entropy)?
Which metric is better to use in different scenarios while using decision trees?
Krish Mahajan
- 1,231
- 2
- 9
- 4
1
vote
1 answer
How to use differential entropy as pre-processing?
I am currently working on implementing the model EEG_DMNet. For pre-processing it calls for using differential entropy like
$$
h(X) = -\int_{-\infty}^{\infty} p(x) \log p(x) \, dx
$$
Assuming the Data I am using is gaussian-distributed the formula…
Sebastian Krafft
- 11
- 1
1
vote
0 answers
Model uncertainty quantification
I'm reading a paper about model uncertainty quantification. Specifically, it says epistemic uncertainty is a kind of uncertainty due to lack of knowledge about a particular region in the input space.
Also, it makes a mathematical characterization of…
piero
- 61
- 5
1
vote
1 answer
How to use labels to fit several thresholds in a simple decision rule?
I have a binary labelled dataset with numeric features. I want to create a "business rule" of the type y = x1 > t1 and x2 > t2 and x3 > t3. X are my features and I just have to find which are the thresholds t that fit the data better.
I have tried…
hipoglucido
- 1,200
- 1
- 10
- 19
-1
votes
1 answer
Basic understanding of entropy
I calculated the entropy of a Gaussian having say SIGMA2=0.01 ? The formula 1/2 log(2π σ^2) +1/2. It gives a negative number. But I don't think entropy can be negative.
Am I doing something wrong?
