Difference between Probability and Probability Density

Question

This question is from DeGroot's "Probability and Statistics" :

Unbounded p.d.f.’s. Since a value of a p.d.f.(probability density function) is a probability density, rather than a probability, such a value can be larger than $1$. In fact, the values of the following p.d.f. are unbounded in the neighborhood of $x = 0$:$$f(x) = \begin{cases} \frac{2}{3}x^{-\frac{1}{3}} & \text{for 0<$x$<1,} \\ 0 & \text{otherwise.} \\ \end{cases}$$

Now, I don't know how the p.d.f. can take value larger than $1$.Please let me know the difference between the probability and probability density.

Answer 1

Simply put:

$\rho(x) \delta x$ is the probability of measuring $X$ in $[x,x+\delta x]$. With

$\rho(x):=$ probability density.

$\delta x:=$ interval length.

A probability will be obtained by computing the integral of $ \rho(x) $ over a given interval (i.e. the probability of getting $X\in [a,b] $ is $\int_a^b \rho(x) dx$. While $\rho(x)$ can diverge, the integral itself will not, and this is due to the fact that we ask that $\int_\mathbb{R}\rho(x) dx=1$, which means that the probability of measuring any outcome is 1 (we are sure that we will observe something). If the integral over the whole range gives 1, the integral over a smaller portion will give less than 1, because p.d.f. can't be negative (a negative probability is meaningless).

Answer 2

Probability density is a "density" FUNCTION f(X). While probability is a specific value realized over the range of [0, 1]. The density determines what the probabilities will be over a given range. What does it mean to have a probability density?

The probability density function for a given value of random variable X represents the density of probability (probability per unit random variable) within a particular range of that random variable X.

Now, I don't know how the p.d.f. can take value larger than 1

It is in this sense that probability density can take values larger than 1.

Answer 3

There is an excellent explanation about this in the chapter 2 of the book The Scientist and Engineer's Guide to Digital Signal Processing.pdf.

Quoting from the book -

The probability density function (pdf), also called the probability distribution function, is to continuous signals what the probability mass function is to discrete signals.

The vertical axis of the pdf is in units of probability density, rather than just probability. For example, a pdf of 0.03 at 120.5 does not mean that the a voltage of 120.5 millivolts will occur 3% of the time. In fact, the probability of the continuous signal being exactly 120.5 millivolts is infinitesimally small. This is because there are an infinite number of possible values that the signal needs to divide its time between: 120.49997, 120.49998, 120.49999, etc. The chance that the signal happens to be exactly 120.50000 is very remote indeed!

To calculate a probability, the probability density is multiplied by a range of values. For example, the probability that the signal, at any given instant, will be between the values of 120 and 121 is: (121 - 120) × 0.03 = 0.03. The probability that the signal will be between 120.4 and 120.5 is: (120.5 - 120.4) × 0.03 = 0.003 , etc. If the pdf is not constant over the range of interest, the multiplication becomes the integral of the pdf over that range. In other words, the area under the pdf bounded by the specified values.

Answer 4

If you have a continuous random variable X with a value between 0 and 3 and the probability (is always between 0 and 1) that X will occur between 2 and 2.1 is say 0.2, the probability density (probability rate) will be 0.2/0.1 = 2. when you multiply the probability density by the interval of the event (2*0.1 = 0.2), you will get the probability.

Answer 5

The specific values $f(x)$ of the density function $f$ are the probability densities, and they express "relative probabilities", and the main point is that for a (measurable) subset $A$ of possible values (now $A\subseteq\Bbb R$), we have $$\int_Af\ =\ P(X\in A)$$ if the random variable $X$ has distribution described by $f$. In particular, $\int_{\Bbb R}f=1$, though its specific values, as shown by the given unlimited example, can be greater than $1$.