Can a universal worst case problem instance exist?

Question

Given a specific type of problem like sorting a list and a particular algorithm like insertion sort, I am aware that a particular instance of the problem is worst case complexity for the algorithm (i.e a list where all elements are already sorted). However, are there instances of a problem that are difficult for all algorithms?

Intuitively, I feel like this could not be the case since you could design an algorithm to exploit any specific arrangement. But maybe this is not the case in all problem settings.

My application here is to create a problem instance against an unknown set of algorithms such that it has maximal runtime over the set of algorithms. The problem is decidable and the algorithms in question are guaranteed to provide the correct answer in finite steps. Is a large random instance really the best option here? Is this provable?

More generally, what is it about a specific problem instance that makes it difficult to exploit algorithmically? Is this difficulty always domain/problem-specific or is there a more unifying theory?

Answer 1

TL;DR: No, I don't think there's much hope of picking a problem instance that is maximally hard for all algorithms.

The question is tricky. It's tricky even to identify a well-formed statement of the problem that is meaningful.

One thing I can say: there is no single instance that is hard for all algorithms. See https://cs.stackexchange.com/a/153273/755. If you pick a specific instance, it is easy to devise an algorithm that performs well on that one instance (e.g., its first step is to check if the input is that one instance, and if so, output a hardcoded answer).

This might or might not fully answer your question.

There is an additional nuance. Normally, when we talk about the complexity of a problem, we are referring to asymptotic running time. Therefore, it doesn't make sense to talk about the complexity of a single instance. Instead, we can talk about the running time of an infinite sequence of problem instances: e.g., for each $n=1,2,3,\dots,$, you might give a problem instance of length $n$, and then you can talk about the asymptotic running time for this set of problem instances (which provides a lower bound for the worst-case asymptotic running time of the algorithm).

No, a large random instance does not achieve what you want. For some algorithms, random instances are much easier to solve than worst-case instances. For instance, the simplex algorithm is known to run in polynomial time on "random" instances (for some suitable notion of random), but in exponential time on worst-case instances.

Answer 2

As mentioned in @D.W.'s answer, I think a meaningful question would not consider a worst case instance, but a worst case family of instances in an attempt to provide an asymptotic lower-bound for the time or space or any other complexity-metric you wish to study.

I think there is no universal answer, and the answer is problem-specific -- we are aware of different problems that have different asymptotic lower-bounds, and clearly, each is obtained by observations that are problem-specific. For example, NFA determinization has an exponential lower-bound on the runtime because we can design a family $A_1, A_2, \ldots$ of NFAs where $A_i$ is of size linear in $i$, yet a DFA for $L(A_i)$ is of size at least exponential in $i$. Note that this lower-bound applies to all possible determinization algorithms no matter how "differently" they operate, and is obtained by an analysis of the structure of a DFA for the language $L(A_i)$. If you are interested in a different specific example, you can see here how we can obtain an exponential lower-bound on NFA-complementation. So yes, lower-bounds that apply for all possible algorithms exist.

Your question is general however and depends on the setting of the problem: we can perhaps consider a set of problems that share common properties, and we want to know if there is a "universal" lower-bound that applies to all these problems. An example of the latter is common to see in complexity theory were we are interested in studying behaviours of a complexity class (a bunch of algorithms that asymptotically use the same amount of resources). This is more tough as here we abstracted many features of a specific algorithm and focused only on the resources it uses. For example, Savitch's theorem states that every nondeterministic algorithm with space-complexity $f(n)\geq logn$, has an equivalent deterministic algorithm with space complexity at most $f^2(n)$, yet, to the best of my knowledge, no known lower-bound is known. So we don't care if this is a sorting algorithm, or a graph traversal algorithm, we care only about the amount of space it uses. Note that here, suggesting a lower-bound is more challenging as we need to refute an existence of a better algorithm/simulation: we need to suggest a family of nondeterministic algorithms $A_1, A_2, \ldots, ...$ where $A_i$ uses at most $f_i(n)\geq logn$ space, yet there is no generic deterministic algorithm that simulates $A_i$ in space strictly less than $f^2_i(n)$ for all $i$.

So the answer to your question depends on what problem you are interested in, the level of abstraction you are willing to sacrifice, and what resources you want to study. The more specific and restricted your question is the easier it is to use problem-specific properties that might help.