Kernel Perceptron vs Polynomial Perceptron

Question

I was looking at Support Vector machines (SVM) kernels. Looking at Polynomial Kernel and Kernel Perceptron I was curious how they differ?

Work Done

Polynomial Kernel:

$d_{k+1}(x)=d_{k}(\bar{x})+\rho k(\bar{x}_{k}, \bar{x})\; \mbox{if}\; \bar{x}_k\epsilon\,C_1$

$d_{k+1}(x)=d_{k}(\bar{x})-\rho k(\bar{x}_{k}, \bar{x})\; \mbox{if}\; \bar{x}_k\epsilon\,C_2$

where $k(\bar{x}_{k}, \bar{x}) = (\bar{x}\cdot \bar{x}_{k}+1)^{q}$

Kernel Perceptron:

This is given by

$g(x)=\sum_{j=1}^{N}a_{j} K(\bar{x},\bar{x}_j)$

So as per my understanding a bias constant is added in former case when compared to later. So how does that impact and what difference does it make? Or am I missing something?

Any insights are appreciated.

Answer 1

A SVM with a polynomial kernel is a SVM classifier.

A kernel perceptron is a perceptron classifier, or in other words, a neural net.

A SVM is quite different from a neural net. So, that's one way that they differ.

However, Wikipedia says that SVMs are in some respects a generalization of a kernel perceptron, generalized with regularization. Regularization is basically a form of Occam's razor: it says that, all else being equal, simpler models are preferred over more complex models. Regularization helps avoid overfitting and thus is very useful in practice.

(Related: See also Support Vector Machines as Neural Nets? -- a SVM with a linear kernel is similar to a single-layer perceptron classifier, in case that's what you were thinking of.)