I'd like to ask what is the main reason why we find the roots in logistic regression (i.e. why we use Newton Raphson method on logistic regression ). I understand the basics of Newton Raphson method, but I just can't understand what is the importance of finding the roots or using second derivatives.
P.S. I know the idea of Newton Raphson, I am wondering why do we need to use this method and find the zero's or roots of function? What does it want to tell us for logistic regression for instance?
This blogpost gives a broad answer to your question. In short, Newton's method is not used to find a root of the loss, but a root of the gradient. If you find a root of the gradient, then you are either in a maximum, a minimum, or a saddle point (the three of them are critical points). When using the cross-entropy loss function in logistic regression, it can be proved that this loss function has only one critical point, and this critical point corresponds to the global minimum. For this reason, finding a root of the gradient is equivalent to finding the only critical point, therefore the global minimum of the cross-entropy loss function.