$PDF$ taking values $>1$: what are the "units"?

Question

So there are many questions asking about PDFs having value $>1$ and asking for explanations. A lot of the explanations say to think about PDFs as density.

I.E. a 1 m^3 steel cube can be 1KG but have all that weight concentrated in an inch. (that is, one inch can say have a density of 61 023.7441 kg/m^3.) If we instead looked at the inches density in in^3 I believe it would have density of 1kg/inch^3

That is, I see answers that seem to be saying that probability densities can be $>1$ because the units they are measured in. My question is,

What units are PDFs in (if they even have units). That is, for a distribution uniform over [0,.5], the PDF at $x=.5$ is $=2$. What is this $2$ measured as? 2 probability per what (a length of 1?)

EDIT: To be more concrete (and seek a more advanced answer, at the risk of my not understanding it). Suppose we have some continuous random variable $X$ with PDF $f_X$. How can I think about probability on some "abstract space" (i.e. there is some probability triple $(\Omega, \mathcal{F}, P)$ in the background. Now notions like area/length/volume/time don't necessarily make sense.

Given that the "EDIT" question is harder and perhaps not concise enough, if it does not get answered after enough time I will accept the most satisfactory answer to the original (less abstract) question

---

Also, how does this generalize to more dimensions. If we have a dist. uniform over half the unit square a line has probability 0, so the units would have to be area? But in the 1 dimensional case the units are lines?

My best guess is that the unit is some reference measure (maybe have to get into Radon-Nikodym derivatives to understand this??)

Answer 1

The value of a PDF $f(x)$ at $x$ means that the probability of a small interval around $x$ of length $\delta$ is approximately $f(x) \delta$. So yes, if you must think of this in terms of units, it is "probability per unit [interval] length." In two dimensions, it would be "probability per unit area" and so on. From here it is clearer that PDFs can take values more than $1$, since its "units" are not probability, but "probability per length/area/volume."

Formally, I would advise you to think of the PDF (in one dimension) as the derivative of the CDF, or as "the function $f$ such that $P(a \le X \le b) = \int_a^b f(x) \, dx$ for any interval $[a,b]$."

---

Response to the additions in your edited post:

If $X : \Omega \to \mathbb{R}$ is a real-valued random variable, then one typically considers the density with respect to Lebesgue measure (when it exists). So everything I stated above still holds: for a tiny interval $I$ around $x$, the probability $P(X \in I)$ (with probability given by the underlying probability space $(\Omega, \mathcal{F}, P)$) is approximately $f(x) |I|$ where $|I|$ is the length (or Lebesgue measure) of the interval $I$. Formally, you should think of the density as a function such that $P(X \in A) = \int_A f(x) \, dx$ for any measurable set $A$. And yes, this is the Radon-Nikodym derivative of the probability measure (pushforward measure from $\Omega$ to $\mathbb{R}$ via $X$) with respect to Lebesgue measure.

If you don't want to consider Lebesgue measure (or $\mathbb{R}$ at all), the above can easily be generalized to get a notion of density / Radon-Nikodym derivative with respect to other measures on other spaces.

Answer 2

The CDF is a continuous distribution of the variable and CDF($\infty$)=$1$.

The PDF is more like a relative probability? But it's better to think of it as a density, because as you said, a uniform distribution over $[0,0.5]$ is twice as dense as one over $[0,1]$. A PDF only gives point probabilites for discrete functions, so no, there's no "probability of 2" for getting $X=0.3$ in a continuous distribution. Rather, you would turn to the CDF.

Similarly, the PDF could be greater than $1$ in multivariate distributions. If you imagine the normal distribution in two variables, which forms a bell shape and flattens out in every dimension, the PDF could easily be greater than $1$ in the middle. However, the volume under the surface is still $1$, as the total probability of the variable being in its range is $1$.