1

A comment was made to me saying the following in relation to Kolmogorov complexity:-

You're not the first to think non-computability = impractical or even useless. But it can be useful. In particular to random, "non-algorithmic" data.

I now have a physical box on my bench that outputs perfectly random bytes with the following distribution:-

histogram

I'm trying to estimate the entropy rate of this box, and the problem is that these bytes are highly correlated. Very highly. I can't stress that enough. They are not independent nor identically distributed and arrive in blocks of 20,000ish bytes but that length varies randomly too.

How can Mr. Kolmogorov be useful to me with measuring this "non-algorithmic" data?

How to practically measure entropy of a file? has not yielded a practical answer so far.

Community
  • 1
Paul Uszak
  • 1,602
  • 1
  • 13
  • 21

1 Answers1

1

How can Kolmogorov complexity be useful to you in measuring the complexity of this data? It can't. The Kolmogorov complexity of a sequence of bytes is not computable.

Don't read too much into that comment. I'd take it as hinting that the idea of Kolmogorov complexity might be a relevant or interesting concept to know about, not necessarily that you can directly compute that value in a particular real-life situation. Anyway, just because someone wrote something doesn't mean it's true. If you're not sure what they meant by it, or you are wondering what justification they had for saying that, you might need to ask them.

I notice that Yuval Filmus's answer to the question you linked to provided a similar summary of Kolmogorov complexity, so I'm a bit confused about what is unclear here.

D.W.
  • 167,959
  • 22
  • 232
  • 500