I'm working on trying to extract the eigenvectors from a series of observations from a random variable, by using the PAST algorithm, see e.g. 6.2.3 in this book: Large pdf. I don't understand the cost function as described here. The citation refers to

B.Yang.Projection Approximation Subspace Tracking. IEEE Transactions on Signal Processing, 43:95–107, Jan 1995

Why should any other matrix than the identity matrix minimize the cost function, regardless of the value of the input x?

Edit: I've been told to use this method to find all the eigenvectors, not for compression. Later in the section, the algorithm is presented step by step, and it says to initialise the weight matrix as the n x n unit matrix. The term seems ambiguous; if I supply the identity matrix, it never gets updated. If I on the other hand supply a matrix with only ones, the resulting matrix is always of the type

\begin{matrix} a & b & c \\ a & b & c \\ a & b & c \end{matrix} I.e. the columns are clearly not independent.

  • $\begingroup$ Try Stats stackexchange, or Computational Science $\endgroup$ – An old man in the sea. Oct 21 '17 at 23:30

If the identity matrix is a feasible solution, then clearly it is an optimal solution.

However, most of the time, the identity matrix is not a feasible solution.

$$W\in \mathbb{R}^{m \times d}$$

where usually, $m$ is chosen to be less than $d$ to perform compression/ dimension reduction.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.