10

I've recently read a really interesting blog entry from Google Research Blog talking about neural network. Basically they use this neural networks for solving various problems like image recognition. They use genetic algorithms to "evolve" the weights of the axons.

So basically my idea is the following. If I was supposed to write a program that recognizes numbers I would not know how to start (I could have some vague idea but my point is: It is not trivial, nor easy.) but by using neural network I do not have to. By creating the right context in order for the neural network to evolve, my neural network will "find the correct algorithm". Down below I quoted a really interesting part of the article where they explain how each layer have different role in the process of image recognition.

One of the challenges of neural networks is understanding what exactly goes on at each layer. We know that after training, each layer progressively extracts higher and higher-level features of the image, until the final layer essentially makes a decision on what the image shows. For example, the first layer maybe looks for edges or corners. Intermediate layers interpret the basic features to look for overall shapes or components, like a door or a leaf. The final few layers assemble those into complete interpretations—these neurons activate in response to very complex things such as entire buildings or trees.

So basically my question is the following: Couldn't we use genetic algorithms + neural networks in order to solve every NP problem? We just create the right evolutionary context and leave "nature" find a solution.

Inceptionism: Going Deeper into Neural Networks

EDIT: I know we can use Brute-Force or find a not-efficient solution in many cases. That is why I try to highlight Evolving artificial neural networks. As I said in a comment: Given sufficient time and an appropriate mutation rate we could find the optimal solution (Or at least that is what I think).

Concept

nmomn
  • 377
  • 3
  • 9

6 Answers6

22

No. This direction is unlikely to be useful, for two reasons:

  1. Most computer scientists believe that P $\ne$ NP. Assuming P $\ne$ NP, this means there does not exist any polynomial-time algorithm to solve any NP-complete problem. If you want your neural network to solve the problem in a reasonable amount of time, then it can't be too large, and thus the neural network will itself be a polynomial-time algorithm. It follows that if P $\ne$ NP, neural networks cannot efficiently solve any NP-complete problem.

  2. Neural networks aren't "magic". They are a way of trying to find patterns. For some problems where there are strong enough patterns to be found, and the patterns can be learned from a reasonable number of examples, they might be effective. But they're not magic fairy dust. Just because you can set up a neural network doesn't mean that backpropagation will necessarily find a good way to solve your problem. It might be that there are no patterns to be found, that the patterns can only be discovered with an unfeasible number of examples, or that patterns exist but the neural network training procedure isn't able to find them.

Neural networks are just another form of machine learning. We could make the same remarks about SVMs or random forests or linear regression any other form of machine learning. Neural networks aren't some kind of magical silver bullet that solve all machine learning problems. They're about as effective as other machine learning methods, or for some kinds of problems, maybe a little bit more effective, but they're not magic.

Sometimes I run across people who have heard only a little bit about neural networks, and they walk away thinking that neural networks are the answer to everything -- maybe because they heard that "your brain uses neural networks too", or they saw some very cool application (voice recognition or something). But don't be fooled. Don't believe the hype. Neural networks are a useful technique, but they're not going to enable computers to solve NP-complete problems, or to beat the Turing test, take away all our jobs and replace humans with computers. Not anytime soon, anyway. That's just science fiction.

D.W.
  • 167,959
  • 22
  • 232
  • 500
11

It seems other answers while informative/ helpful are not actually understanding your question exactly and are reading a little too much into it. You didn't ask if neural networks would outperform other methods, you only asked if they could be applied to NP complete problems. The answer is yes, with some success and this has been known for decades and there is a wide variety of research on this, and it continues. This has to do with the flexibility of machine learning. Note that even if they dont find exact or optimal solutions, the solutions they have may have other desirable properties. some example papers:

peterh
  • 468
  • 4
  • 19
vzn
  • 11,162
  • 1
  • 28
  • 52
4

Neural networks do not actually solve NP-complete problems. What they do do is solve problems which are remarkably close to NP-complete problems.

One big feature of neural networks is that they are not obliged to find the "right" answer every time. They are allowed to be "wrong." For example, you might be solving a bin-packing problem, and come to a solution which is 1% off of the ideal solution and be totally content with that answer.

If you remove the requirement of being 100% right every time, other problem solving approaches work very well. For example, many route planning algorithms (a la Google Maps) have to be NP-complete, but it is quite trivial to find an algorithm which finds a path within 1% of the optimum 99.9% of the time. It's trying to pin down results in that last 0.1% of the cases which drive NP-complete efforts to be so mindnumbingly expensive.

As it happens, when we try to use NP-complete equations in real life, we don't often need the actual answer. We're often very comfortable with a "close" answer, though we often don't have a wording to explain what "close" metric we are using. These are the situations where a neural network can answer the actual question you wanted to ask, rather than having to actually solve the NP-complete problem you asked for instead.

Cort Ammon
  • 3,522
  • 14
  • 16
1

Neural networks are known to be capable of universal function approximation, but this requires training them on the problem (optimization) which is a NP-complete problem in and of itself, which is why you have evolutionary training and SGD with backpropagation and so on.

So while they are not likely to solve NP-complete problems they can be trained to approximate to an arbitrary degree of accuracy a function that models the problem. Also even if you happen to solve an NP-complete problem optimally using a neural network you still have no way to prove that the solution it found is actually the global optimum without brute forcing the solution (this is of course not feasible for almost any practical use case of neural networks).

Your visualization is accurate in the sense that evolutionary algorithms (see how the neat algorithm prevents a single species from taking over the population with an initially highly performant structure by using shared fitness) are less apt than SGD and other machine learning techniques to be trapped in local optimums but they provide no proof that the solution they find is in fact the global optimum solution.

Yuval Filmus
  • 280,205
  • 27
  • 317
  • 514
nickw
  • 111
  • 3
0

Even if P $\neq$ NP, that says nothing about average case complexity. That for any algorithm for solving a NP complete problem must have at least some instances where it takes more than polynomial time. However it is certainly possible that we find an AI that can solve sat in linear time for 80% of instances better then any conventional algorithm. I do not know enough about AI to give the likelihood of that

Colonizor48
  • 239
  • 1
  • 6
-1

Given that neural nets are good at interpolation (but poor at extrapolation), for a given use case it suffices to solve a few billion large problems and feed them to a sufficiently large net. In some cases this would work.