1

In linear Regression we have the following loss function :-

$$L(x)=\frac1{n}\left(\sum_{i=1}^n ((a+bx_i)-y_i)\right)^2$$

Hence, we can observe that L(x) is having a convex graph but when we use the same cost for the logistic regression the equation turns out to be the following :-

$$L_1(x)=\frac1{n}\left(\sum_{i=1}^n (\frac1{1+e^{-(a+bx_i)}}-y_i)\right)^2$$

Here, $y_i\epsilon (0,1)$ as they are the actual outputs in our data set while predicting the class. Now, when we try to plot $L_1(x)$ using $y_i=1$, I am getting the following loss function plot

which is not same as the loss function plot of logistic used in Linear regression's which is shown in the following link here Logistic regression - Prove That the Cost Function Is Convex.

If we are getting the same graph as the hypothesis function of logistic, how can we say that is a non-convex one ?

Seewoo Lee
  • 15,670
noob_python
  • 13
  • You might try deriving the Hessian and showing that it's indefinite. Or you might look for two models for which the average of their losses on a data point is less than the loss of the average of those models on the same data point. – ted May 14 '21 at 00:53
  • @ted can you please post the derivation here I am not able to get an idea of how to do it – noob_python May 14 '21 at 07:04

0 Answers0