Why use languages in Complexity theory

Question

I'm just starting to get into the theory of computation, which studies what can be computed, how quickly, using how much memory and with which computational model.

I have a pretty basic question, but am really hoping some of you guys can help me understand the concept behind it:

Why is everything centered around the notion and definition of LANGUAGES (i.e. regular languages and context free languages)? And how do these relate and describe the complexity of an algorithm and the possible computational models for solving them?

I read these sort of related questions:

• https://cstheory.stackexchange.com/questions/14811/what-is-the-enlightenment-im-supposed-to-attain-after-studying-finite-automata
• https://cstheory.stackexchange.com/questions/8539/how-practical-is-automata-theory

but still don't have an answer to my doubts, since they provide a practical justification of why they are important (which I do understand) but don't help me understand why complexity theory is based upon them.

Answer 1

It's because languages are the best (only?) way we have of formalizing the concept of a "problem."

An algorithm (Turing Machine) has performance, which we express via big-O complexity. A problem (language) belongs to a complexity class. These are usually defined by existence: if there exists a machine accepting a language $L$ which runs in given performance (space or time), then the language belongs to the corresponding complexity class.

There's a few reasons for this. First is that languages are platform independent. You're not worrying about whether an integer is 32 or 64 bits, or whether floating point operations run in parallel with other operations. These things give performance speedup at the micro-level, but complexity analysis is interested in the macro level. As you scale from 100 to $10^6$ to $10^9$ to $10^{12}$ inputs, how does the algorithm performance change? Does it go from using 1 million tape cells to 1 billion, or from 1 million to more cells than there are atoms in the universe?

The second is that languages are just a nice abstraction for data. You need something you can do proofs about, something you can model formally. Encoding your input and output as a string mean that you're now dealing not with bits in memory, but with mathematical objects with specific properties. You can reason about them and prove proofs about them in a formal, and very simple sense.

Complexity theory tends to be focused on decision problems because they end up being difficult. When the decision version of travelling salesman is NP-complete (i.e. is there a tour shorter than length $k$), then obviously finding the shortest path is harder. There isn't as much focus on function/optimization problems because there's still a lot of open questions and unsolved problems about the simpler decision problems.

I guess here's my challenge to you: find way to mathematically describe problems that isn't languages. I don't know if languages are special, but I think they're the simplest tool we've got, the easiest one to deal with.

Answer 2

There are two basic answers to your question:

1. There is more to complexity theory than languages, for example function classes, arithmetic complexity, and the subareas of approximation algorithms and inapproximability.

2. Historical reasons: one of the basic papers in computability theory was discussing Hilbert's Entscheidungsproblem (a form of the halting problem).

Unfortunately I don't know much about the latter, but let me expand on the former.

Complexity beyond languages

Every computational complexity class comes with an associated function class. For example, the class P of all problems decidable in polynomial time is associated with FP, the class of all functions computable in polynomial time. FP is important since it is used to define NP-hardness: a language $L$ is NP-hard if for every language $M$ in NP there is a function $f_M$ in FP such that $x \in M$ iff $f_M(x) \in L$. Another complexity class of functions, #P, is related to the so-called polynomial hierarchy via Toda's theorem.

Arithmetic circuit complexity (or algebraic complexity theory) deals with the complexity of computing various polynomials. Important complexity classes here are VP and VNP, and geometric complexity theory is an important project attempting to separate VP and VNP (and later P and NP) using algebraic geometry and representation theory.

Another important example of algebraic complexity is fast matrix multiplication. Here the basic question is how fast can we multiply two matrices? Similar questions ask how fast we can multiply integers, how fast can we test integers for primality (this is a decision problem!) and how fast can we factor integers.

Convex optimization deals with optimization problems that can be solved (or almost solved) efficiently. Examples are linear programming and semidefinite programming, both of which have efficient algorithms. Here we are interested both in the optimum and in the optimal solution itself. Since there is often more than one optimal solution, computing an optimal solution is not well represented as a decision problem.

Approximability is the area that studies how good an approximation we can get for an optimization problem in polynomial time. Consider for example the classical problem of Set Cover: given a collection of sets, how many of them do we need to cover the entire universe? Finding the optimal number is NP-hard, but perhaps it is possible to compute an approximation? Approximation algorithms is the subarea studying algorithms for computing approximations, while inapproximability studies limits of approximation algorithms. In the particular case of Set Cover, we have an algorithm giving a $\ln n$ approximation (the greedy algorithm), and it is NP-hard to do any better.

Answer 3

Let's look at this question from the perspective of category theory. The decision problems (or languages) would then correspond to the objects of a category, and the allowed reductions between two problems would correspond to the morphisms (arrows) of a category.

Talking about languages has the advantage that equivalence of languages is well defined (namely by extensional equality). Two unrelated problems might lead to the same language, and then we are allowed to consider them as equivalent. If we would want to talk about isomorphic problems instead, we would have to define the allowed morphisms between two problems. But the allowed morphisms depend on the actual complexity class under consideration, which makes this approach less suitable for comparing different complexity classes.

The notion of isomorphic problems will normally be coarser than the notion of equivalent languages, i.e. two problems can be isomorphic, even if their associated languages are not equivalent. What is worse is that there are often different reasonable notions for the allowed morphisms, which only agree with respect to the allowed isomorphisms. Focusing on languages allows to postpone such problems until we feel like talking about some different reasonable notions of reduction (like Karp reduction vs. Cook reduction).