1
$\begingroup$

I have a working knowledge of calculus and have been able to understand the application of Newton-Raphson technique for unconstrained optimization. Please point me to some of the easy to understand tutorials for advanced techniques like BFGS and LBFGS. I prefer video lectures to books and books to papers.

My aim is to understand how these methods work and code some of them (python/scala/Theano) for better understanding. My area of interest is application of these techniques rather than proofs.

[this was originally posted in SO where I was suggested to post in this site]

$\endgroup$
2
$\begingroup$

Boyd and Vandenberghe's text on Convex Optimization is free, and lectures have been posted online. It's also not incredibly theory heavy.

Nocedal and Wright is very good at discussing practical aspects of optimization, particularly implementation of algorithms. It too is not theory heavy.

For people who care about theory, Bertsekas' book on Nonlinear Programming is a great read. Nocedal's book is better about discussing practical aspects of algorithms, but you could certainly use Bertsekas' book to implement classical optimization algorithms. To understand how the algorithms work at a deep level, understanding the convergence proofs is helpful; however, it requires an understanding of real analysis. I don't think a deep understanding is required to use these algorithms in practice.

$\endgroup$
  • $\begingroup$ Thanks for your reply. As I mentioned, my interest is more in practical aspect than proofs, hence I'm going to go with the Nocedal and Wright book. $\endgroup$ – BiGYaN Jul 8 '14 at 5:46
1
$\begingroup$

I don't know whether there are video lectures, but I find the book by Nocedal and Wright on "Numerical Optimization" to be a fantastic reference to everything related to continuous optimization. It is easy to read and understand.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.