Revisiting the Polyak step size

Citation:

E. Hazan and S. Kakade, Revisiting the Polyak step size. ArXiv Report, 2019.

Abstract:

This paper revisits the Polyak step size schedule for convex optimization problems, proving that a simple variant of it simultaneously attains near optimal convergence rates for the gradient descent algorithm, for all ranges of strong convexity, smoothness, and Lipschitz parameters, without a-priory knowledge of these parameters.

Publisher's Version

See also: 2019
Last updated on 12/11/2023