0

So I wrote Linear regression from scratch using y = mx+b and ran the algorithm for 50 epochs (times) to minimize the cost and get the best parameters.

When I use Scikit Learn, I just call the Linear Regression method and fit the data-set to it then start predicting. How many epochs does fit method run ? Not only for Linear Regression but also other ML methods in general.

John Constantine
  • 707
  • 2
  • 8
  • 10

1 Answers1

5

In scikit-learn's linear regression, the parameters that minimise the squared error loss aren't estimated using gradient descent, they are computed exactly.

The minimisation problem for linear least squares is

$$ \hat{\beta} = \underset{\beta}\arg \min || \mathbf{y} - \beta \mathbf{X} ||^2 $$

which has a unique solution (assuming the columns of $\mathbf{X}$ are linearly independent):

$$ \hat{\beta} = (\mathbf{X}^\text{T}\mathbf{X})^{-1}\mathbf{X}^\text{T}\mathbf{y} $$

For classifiers that are fitted with an iterative optimisation process like gradient descent, e.g., MLPClassifier, there is a parameter called max_iter which sets the maximum number of epochs. If tol is set to 0, the optimisation will run for max_iter epochs.

timleathart
  • 3,960
  • 22
  • 35