I am new in the ML. I know that overfitting is memorizing the data while training. Like in Neural Network, if we make lots of layers and lots of hidden nodes, we can memorize all the data, but it can be bad because train data would not cover the whole space.
Like this, is there any way to overfit in SVM and Logistic Regression? Since they are linear algorithms, they cannot be something curvy, I guess, so I am guessing the answer would be no. But I am not sure.
Any help is appreciated.
Yes, they can overfit too. Overfitting is especially a risk when the number of features is much larger than the number of samples in the training set.
As D.W. points out, in principle every machine learning algorithm can overfit a finite data sample provided you give it enough flexibility and degrees of freedom, e.g., by adding layers or additional features.
However, different methods will be more or less prone to overfitting, and their tendency to overfit is typically studied by theoretical notions such as Rademacher complexity or the Vapnik-Chervonenkis dimension, that roughly speaking characterizes the maximal number of points that an algorithm can perfectly fit for sure.
