7

It is well known that the maximum likelihood estimator of logistic regression does not admit a closed form solution, at least in the general case where the predictors are not binary or categorical. Whereas, ordinary least squares regression does have a closed form solution in terms of matrix inverses.

Is there a proof that logistic regression can't have a closed form solution or it is simply that none have been found after an extensive search? Proofs of the non-existence of things are presumably difficult in general. Is there at least an agreed upon definition of closed form in this context?

RobPratt
  • 50,938
Jamie Ballingall
  • 528
  • 3
  • 9
  • B/c convergence of iterative methods is not guaranteed (at least according to the wikipedia article), then it seems like the corrsponding optimization is non-convex. This is by no means a proof, but generally speaking non-convex optimization problems usually don't have "easy exact solutions" which usually rules out the existence of a "closed form solution", if for no other reason than the existence of a "closed form solution" would imply an "easy" algorithm for the solution. https://math.stackexchange.com/questions/1985008/show-that-logistic-regression-with-squared-loss-function-is-non-convex – hasManyStupidQuestions Feb 15 '22 at 01:49
  • Cf. also https://arxiv.org/abs/1712.07897 Although anyway apparently I am wrong and the log-likelihood for logistic regression is convex https://stats.stackexchange.com/questions/295920/logistic-regression-is-a-convex-problem-but-my-results-show-otherwise -- admittedly it is still fairly common that convex optimization problems lack closed form solutions, but that being said because most are not NP-hard it follows that NP-hardness can not be used as a "proof" of "non-existence of a closed form". Anyway I agree with you that this is a good question and evidently I don't know the answer. Sorry – hasManyStupidQuestions Feb 15 '22 at 01:53

0 Answers0