# Stochastic power iteration for generalized eigenvalue problems?

Suppose $$\mathbf{x}$$ is a random variable in $$n$$ dimensions, and $$u$$ is a vector. How can I estimate the following quantity in an online fashion?

$$f(x)=\max_{\|u\|=1} \frac{ E\left[\langle u\cdot x\rangle ^4\right]}{E\left[\langle u\cdot x\rangle ^2\right]}$$

For a simpler problem of finding $$\operatorname{argmax}_{\|u\|=1}E[\langle x \cdot u \rangle^2]$$ the following method is known to quickly converge:

$$u_{t+1}\leftarrow (I+\nu_t xx')u_t, u_t \leftarrow u_t/\|u_t\|$$

Is there a modification that works for the problem above?

Toy example, take $$\mathbf{x}$$ sampled IID from the following data matrix $$\text{X=}\left( \begin{array}{ccc} 1 & 2 \\ 3 & 5\\ \end{array} \right)$$

We have $$f(x)\approx 30.2987$$ satisfied for $$u\approx (0.520138, 0.854082)$$

notebook

• Is thre any assumption on the distribution of $\mathbf{x}$? Nov 8 at 1:03
• I figured out an efficient way to do this for Gaussian $x$ with known diagonal covariance, so I'm interested in update that works for Gaussian with unknown covariance, but ideally, without any assumptions at all Nov 8 at 1:17
• Well, surely if $x$ is distributed based on some complicated, perhaps multimodal, distribution, the only way you can solve this is by sampling. Nov 9 at 0:22
• Right, it needs to be stochastic. A similar problem is to estimate $\|E[xx^T]\|$, and it's easy to do by iterating $u=x x^Tu; u=u/\|u\|$ which works well (stochastic Power Method), I'm wondering what a similar procedure would be in this case Nov 9 at 3:31