State of the art GD and SGD convergence analysis for nonconvex problems

Asked Mar 02 '20 at 04:21

Active Mar 02 '20 at 04:21

Viewed 246 times

I am working on the convergence analysis of gradient descent as well as stochastic gradient descent. I have come across the following papers which are great papers.

1- Lee et al. proving GD converges to local minimizer.

2- Jin et al. accelerated gradient descent for escaping saddle points

3- Jin et al. perturbed gradient descent converges to stationary points

4- Allen-Zhu et al. convergence of stochastic gradient descent

Question: Please discuss the recent improvements in this area after the aforementioned papers and cite the relevant papers that are new. Also, provide more materials on SGD since I have only been able to find few papers regarding convergence of SGD.

asked Mar 02 '20 at 04:21

See this post – ProAmateur Mar 02 '20 at 23:20
@madnessweasley: pretty much is what I wanted. Do you mind if I ask you to shed lights on this topic if you come across new things? – Mar 03 '20 at 03:42
I no longer keep up with that line of research, so I'm afraid I won't be able to keep the answer up-to-date. You can keep tabs on the stat.ML and math.OC arXiv threads to stay updated (of course, I ignored many lines of research in my previous answer) – ProAmateur Mar 03 '20 at 14:46

State of the art GD and SGD convergence analysis for nonconvex problems

0 Answers0