Is this an example of a natural, strictly NP-intermediate language (assuming EXP ≠ NEXP)?

Question

In the wikipedia page for the NP-intermediate complexity class, the following observation is made:

Under the assumption that P ≠ NP, Ladner explicitly constructs a problem in NPI, although this problem is artificial and otherwise uninteresting. It is an open question whether any "natural" problem has the same property [...]

In the '80s, a work by Galperin and Widgerson was dedicated to problems on graphs with succinct circuit representations. Intuitively, a graph has a succint circuit representation if there is a polylogarithmically sized boolean circuit which takes two (padded) integers representing node numbers as input, and outputs 1 if there is an edge between those two nodes and 0 otherwise. Many NP-complete predicates of graphs were shown to be NEXP-complete when the graph is provided in its succinct form.

Now consider any particular NP-complete problem among these. The above fact can be because, aside from the input having become exponentially smaller, the following informal statement must be true:

Call $\sigma$ the subset of instances of the original NP-complete problem that are representable by a succinct circuit. $\sigma$ turns out to not be dramatically easier to decide than its parent set; that is, $\sigma$ is not susceptible to some specialized, polynomial method leveraging some particular structural peculiarity common to all its members.

In fact, if there was a polynomial time algorithm A that, when applied to members of $\sigma$ (it doesn't even matter what A does with the other instances!), would correctly decide them, you could simply "decompress" the graph and then use A to decide the instance. Both of these steps take deterministic time exponential in the succinct input length. But the succinct problem is NEXP complete, so this could happen only if EXP=NEXP. This provides evidence that $\sigma$ itself is not in P unless EXP=NEXP. To decide $\sigma$, you would need to establish that the instance has a small circuit, then solve it. The first part is the MCSP problem, which for all we know could be in P. But the second part would require A, that as mentioned exists only if EXP=NEXP.

I wish to observe that $\sigma$ is sparse, by the simple fact that, asymptotically, as graph size increases linearly only logarithmically many small circuits can be proposed, and therefore only logarithmically many instances selected.

In Papadimitriou and Yannakakis this topic is tackled. They observe:

[...] Also, our result can be interpreted as an elaboration on the work of Hartmanis, Sewelson, and Immerman (1983). They showed that there exist sparse sets in NP-P iff NEXP ≠ EXP. We have exhibited, essentially for any known NP-complete set, a sparse subset that is the most likely to be in NP - P exactly if NEXP ≠ EXP (namely, the members of the language that are succinctly representable by the scheme suggested in the proof).

NP-P includes both NP-complete and NP-intermediate. But, assuming P ≠ NP, $\sigma$ cannot be NP-complete by virtue of Mahaney's theorem, and therefore it must be NP-intermediate. Thus, $\sigma$ is an arguably "natural" example of a problem strictly in NPI (provided naturally that EXP≠NEXP), in contrast with the wikipedia article.

Did I make some mistake in my reasoning? Why do Papadimitriou and Yannakakis use NP-P, and why do they say that if EXP≠NEXP $\sigma$ is only "most likely" to be in NP-P, and not certainly in NP-P?

If $\sigma$ is, in fact, in NPI if NEXP ≠ EXP, why isn't this mentioned in NP-intermediate discussions? Could this really just have been overlooked?

Answer 1

Great observation and question! There is no contradiction, but you have to get into the details. I'll explain why below. Also, as it happens, something very similar to your example has been noted before.

Details

I interpret Wikipedia as saying: it is an open question whether there is any natural problem $L$ with the property that, if $P \ne NP$, we can prove that $L$ is NP-intermediate.

You have shown a natural problem $\sigma$ with the property that, if $P \ne NP$ and $EXP \ne NEXP$, we can prove that $L$ is NP-intermediate.

These two statements are consistent with each other and can both be true. I don't see any contradiction between the two. As far as I am aware, we don't know of any proof that $EXP \ne NEXP$, or any proof that $P \ne NP$ implies $EXP \ne NEXP$.

Noted before

Check out the last paragraph of Joshua Grochow's answer here. That is a very similar construction.

On Wikipedia

As a reminder, Wikipedia is not a primary source. You shouldn't necessarily trust technical claims made on Wikipedia to be correct, unless you can verify them yourself or Wikipedia cites a primary source and you consult the primary source. My experience is that Wikipedia is generally quite reliable for statements about computer science, but is not perfect, and the statements are sometimes stated in a somewhat imprecise manner. In this case, the precise details of the statement matter. In those cases it is often worth finding a better source that gives a more precise treatment of exactly what is being claimed.

Unfortunately, the Wikipedia article doesn't give any citation for the claim it makes. However, I can offer a better source for that statement:

"We do not know of a natural decision problem that, assuming $NP \ne P$, is proven to be in $NP \setminus P$ but not $NP$-complete, and there are remarkably few candidates for such languages."

Sanjeev Arora and Boaz Barak, Computational Complexity: A Modern Approach, Princeton University, 2007.

I think this supports my interpretation above.

On natural problems

One trickiness is that what counts as "natural" is inherently subjective. So, another way to understand these claims is as an empirical observation that it's rare to run across problems in everyday life that are NP-intermediate: most problems that we run across in everyday life that are in NP are either in P or are NP-complete. It is a bit of a mystery why this happens, but you can find some discussion on this point at https://cstheory.stackexchange.com/q/20930/5038.