Suppose $\mathbf{x}$ is a random variable in $n$ dimensions, and $u$ is a vector. How can I estimate the following quantity in an online fashion?

$$f(x)=\max_{\|u\|=1} \frac{ E\left[\langle u\cdot x\rangle ^4\right]}{E\left[\langle u\cdot x\rangle ^2\right]}$$

For a simpler problem of finding $\operatorname{argmax}_{\|u\|=1}E[\langle x \cdot u \rangle^2]$ the following method is known to quickly converge:

$$u_{t+1}\leftarrow (I+\nu_t xx')u_t, u_t \leftarrow u_t/\|u_t\|$$

Is there a modification that works for the problem above?

Toy example, take $\mathbf{x}$ sampled IID from the following data matrix $$\text{X=}\left( \begin{array}{ccc} 1 & 2 \\ 3 & 5\\ \end{array} \right)$$

We have $f(x)\approx 30.2987$ satisfied for $u\approx (0.520138, 0.854082)$


  • 1
    $\begingroup$ Is thre any assumption on the distribution of $\mathbf{x}$? $\endgroup$
    – whpowell96
    Nov 8 at 1:03
  • $\begingroup$ I figured out an efficient way to do this for Gaussian $x$ with known diagonal covariance, so I'm interested in update that works for Gaussian with unknown covariance, but ideally, without any assumptions at all $\endgroup$ Nov 8 at 1:17
  • $\begingroup$ Well, surely if $x$ is distributed based on some complicated, perhaps multimodal, distribution, the only way you can solve this is by sampling. $\endgroup$ Nov 9 at 0:22
  • $\begingroup$ Right, it needs to be stochastic. A similar problem is to estimate $\|E[xx^T]\|$, and it's easy to do by iterating $u=x x^Tu; u=u/\|u\|$ which works well (stochastic Power Method), I'm wondering what a similar procedure would be in this case $\endgroup$ Nov 9 at 3:31


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.