1
$\begingroup$

The author of the SIAM News article "Optimization Theory and Perspectives on the Field of Machine Learning" mentions:

... For unconstrained convex optimization, GD (gradient descent) convergence rates do not depend on the space’s dimensionality. In other words, if we measure a notion of how close we are to the global minimum, convergence rates depend on the function’s properties and the number of descent steps; they are independent of whether a problem has 10 variables or 10 million. Readers might find this dimension-independence surprising, though perhaps there is a “catch” that practical problems may have a correlation between dimensionality and their Lipschitz constant.

I was wondering if there are any research papers/reports/studies that support the claim (in bold) that practical optimization problems have a correlation between dimensionality and their Lipschitz constant.

This may be a general study that performs an analysis on a large set of various problems or a narrow application with a dimensionality-Lipschitz constant study. It would be particularly helpful if the study is related to computational electromagnetics, but I am also interested in other domains. A more general optimization theory reference that would justify this [quite intuitive] claim (as optimization is not my main area of expertise) would be also quite welcome.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.