New answers tagged

1 vote

Why aren't Krylov subspace methods popular in the Machine Learning community compared to Gradient Descent?

Posts 3 in the following series on "regret minimization" talks about gradient descent, and the later posts come back to that topic from time to time. Online Optimization Post 1: ...
user avatar
5 votes

Why aren't Krylov subspace methods popular in the Machine Learning community compared to Gradient Descent?

They aren't popular because they don't work. Nicol N. Schraudolph spent a few years on Krylov-like methods for Machine Learning. I first learned of his work at 2004 Machine Learning Summer School in ...
user avatar
17 votes
Accepted

Why aren't Krylov subspace methods popular in the Machine Learning community compared to Gradient Descent?

On a basic level, I don't buy the argument that you have to "solve a linear system for many machine learning algorithms". Much more, you usually have to optimize a non-linear equation which ...
user avatar
  • 2,557
2 votes

Largest singular value without using the adjoint

You could use the characterization $$ \sigma_{\max} = \max_{\dim S = 1} \min_{x \in S} \frac{||Ax||_2}{||x||_2} $$ Creating random vectors $x$ and computing the norm of $||Ax||_2$ will give an ...
user avatar
  • 547
0 votes

Solving numerically the 1D Kuramoto-Sivashinsky Equation using spectral methods

Thanks for the code but when I try the code with T = 1000, it is not stable. When I add u_hat[:,j+1] = (1 / nx) * np.fft.fftshift(np.fft.fft(u[:,j+1])) which ...
user avatar

Top 50 recent answers are included