Why there are no approximation algorithms for SAT and other decision problems?

Question

I have an NP-complete decision problem. Given an instance of the problem, I would like to design an algorithm that outputs YES, if the problem is feasible, and, NO, otherwise. (Of course, if the algorithm is not optimal, it will make errors.)

I cannot find any approximation algorithms for such problems. I was looking specifically for SAT and I found in Wikipedia page about Approximation Algorithm the following: Another limitation of the approach is that it applies only to optimization problems and not to "pure" decision problems like satisfiability, although it is often possible to ...

Why we do not, for example, define the approximation ratio to be something proportional to the number of mistakes that the algorithm makes? How do we actually solve decision problems in greedy and sub-optimal manner?

Answer 1

Approximation algorithms are only for optimization problems, not for decision problems.

Why don't we define the approximation ratio to be the fraction of mistakes an algorithm makes, when trying to solve some decision problem? Because "the approximation ratio" is a term with a well-defined, standard meaning, one that means something else, and it would be confusing to use the same term for two different things.

OK, could we define some other ratio (let's call it something else -- e.g., "the det-ratio") that quantifies the number of mistakes an algorithm makes, for some decision problem? Well, it's not clear how to do that. What would be the denominator for that fraction? Or, to put it another way: there are going to be an infinite number of problem instances, and for some of them the algorithm will give the right answer and others it will give the wrong answer, so you end up with a ratio that is "something divided by infinity", and that ends up being meaningless or not defined.

Alternatively, we could define $r_n$ to be the fraction of mistakes the algorithm mistakes, on problem instances of size $n$. Then, we could compute the limit of $r_n$ as $n \to \infty$, if such a limit exists. This would be well-defined (if the limit exists). However, in most cases, this might not be terribly useful. In particular, it implicitly assumes a uniform distribution on problem instances. However, in the real world, the actual distribution on problem instances may not be uniform -- it is often very far from uniform. Consequently, the number you get in this way is often not as useful as you might hope: it often gives a misleading impression of how good the algorithm is.

To learn more about how people deal with intractability (NP-hardness), take a look at Dealing with intractability: NP-complete problems.

Answer 2

The reason you don't see things like approximation ratios in decision making problems is that they generally do not make sense in the context of the questions one typically asks about decision making problems. In an optimization setting, it makes sense because it's useful to be "close." In many environments, it doesn't make sense. It doesn't make sense to see how often you are "close" in a discrete logarithm problem. It doesn't make sense to see how often you are "close" to finding a graph isomer. And likewise, in most decision making problems, it doesn't make sense to be "close" to the right decision.

Now, in practical implementations, there are many cases where it's helpful to know what portion of the problems can be decided "quickly" and what portion cannot. However, unlike optimization, there's no one-size-fits-all way to quantify this. You can do it statistically, as you suggest, but only if you know the statistical distribution of your inputs. Most of the time, people who are interested in decision problems are not so lucky to have such distributions.

As a case study, consider the halting problem. The halting problem is known to be undecidable. It's a shame, because its a really useful problem to be able to solve if you're making a compiler. In practice, however, we find that most programs are actually very easy to analyze from a halting problem perspective. Compilers take advantage of this to generate optimal code in these circumstances. However, a compiler must recognize that there is a possibility that a particular block of code is not decidable. Any program which relies on code being "likely decidable" can get in trouble.

However, the metric used by compilers to determine how well they do at solving these particular cases of the halting problem is very different from a metric used by a cryptography program to test whether a particular pair of primes is acceptably hardened against attacks. There is no one size fits all solution. If you want such a metric, you will want to tailor it to fit your particular problems space and business logic.

Answer 3

In addition to the existing answers, let me point out that there are situations where it makes sense to have an approximate solution for a decision problem, but it works different than you might think.

With these algorithms, only one of the two outcomes is determined with certainty, while the other might be incorrect. Take the Miller-Rabin test for prime numbers, for instance: If the test determines that a number is not prime, that result is for certain. But in the other case it only means that the number is probably prime. Depending on how much compute time you are willing to invest, you can increase your confidence in the result, but it won't be 100% as it is for the not-prime case.

This is especially powerful when tackling undecidable problems: You could write a tool that tries to solve the halting problem for a specific piece of code. If it can find a proof that the program will not loop endlessly, you can claim so with 100% certainty. If you cannot find such a proof, it might just be that the program control flow is too convoluted for your tool to analyze, but it is not a proof that it will loop forever. By simplifying the control structures, you might be able to create an equivalent program that is simple enough for the tool to proof that it will halt for certain.