3
$\begingroup$

Suppose a matrix $X\in\mathbb{R}^{n\times 3}$ is given as a Principal Component Analysis (PCA) projection from some high dimensional space. The 2D PCA solution on X, say $Y\in\mathbb{R}^{n\times 2}$ would simply correspond to the first two columns of $X$.

Now, suppose the configuration is shifted such that the origin corresponds to an arbitrary point. I want to mathematically state (via PCA) that by changing the origin of $X$ (3D data), the new 2D PCA projection $Y'\in\mathbb{R}^{n\times 2}$ simply corresponds to two first columns on $X$ subject to rigid transformation (ie. rotation, reflection, shifting). The reason for the perhaps unnecessary complication is the PCA assumption on the configuration centered at the origin. (In other words, I'm not sure if by getting rid of it, one might loosen the connection with PCA)

To remind you, PCA would be obtained as $$Y=XU_S,$$where $U_S\in\mathbb{R}^{3\times 2}$ would contain eigenvectors of the correlation matrix $S=\frac{1}{n}X^TX$, where $X$ is supposed to be centered at the origin. The origin change would correspond to $X'=PX$, where $P=(I-1_np^T)$ denotes the projector, and $1_n=[1, \dots, 1]\in\mathbb{R}^n$, and $p^T1_n=1$. The new correlation matrix $$S'=\frac{1}{n}((PX)^T(PX))=\frac{1}{n}X^TPX=\frac{1}{n}X^T(J-1_np_1^T)X,$$ where $P$ might be expressed as $P=J-1_np_1^T$, where $J$ is a projector with $p=\frac{1}{n}1_n$, ie. $J=(I-\frac{1}{n}1_n1_n^T)$. So, I would like to state that $Y'=(PX)U_{S'}$ corresponds to a different 2D viewpoint on the primary $X$. The difficulty, in my interpretation, lies in the effects of spectral decomposition of $S'$, and its possible effects on rigid transformation on primary $X$. Again, I apologize for the perhaps unnecessary complication.

Improve this question
$\endgroup$
  • $\begingroup$ Please notify me if the question is not clear (and which part exactly). $\endgroup$ – usero Apr 2 '12 at 13:03
  • $\begingroup$ I don't really understand your question, and PCA was the basis of my master's thesis. If $X$ is given as a Galerkin projection (and it's really important to say "Galerkin projection", because "projection" and "Galerkin projection" are different) of some high dimensional data, and $Y$ is just the first two columns of $X$, then why are you stating that $Y = XU_{S}$? $\endgroup$ – Geoff Oxberry Apr 2 '12 at 15:55
  • $\begingroup$ @GeoffOxberry: See the comment below the answer below. I hope you considered non-centering of high-D data in you thesis. $\endgroup$ – usero Apr 6 '12 at 13:32
3
$\begingroup$

In general, the principal components of $X$ and $X'$ with rows shifted by a fixed vector will not have any simple relationship.

In applications it may therefore be meaningful (or even important) to shift all rows of $X$ by subtraction their mean before doing the prinicpal component analysis. This makes the result independent of the placement of the origin.

It may even be useful (or necessary) to scale the columns after shifting to have norm 1. This makes the result also independent of the scaling of the variables.

Improve this answer
$\endgroup$
  • $\begingroup$ Understood. But, as you might have noticed, I'm interested in the implications of changed origin. Given the original 3D data stored in $X$, any projection of the form $Y'=(PX)U_{S'}$, with $P$ denoting the projector the effect of which is the configuration origin adjustment (?), and $U_{S'}\in\mathbb{R}^{3\times 2}$, should only correspond to a "view-point" change (if one looks at 3D, but from different view point). Is my intuition wrong? $\endgroup$ – usero Apr 2 '12 at 15:03
  • $\begingroup$ I don't think there is any useful relationship between the 2D projection of the shifted data and the 3D projection of the unshifted data. You can compute an approximate $U$ by regression, but it will just be an approximate transformation of the kind you are looking for, and you can check that numerically by calculating the residual. $\endgroup$ – Arnold Neumaier Apr 2 '12 at 17:05
  • $\begingroup$ Suppose certain 3D data is given, $X\in\mathbb{R}^{n\times 3}$, and its PCA solution $Y\in\mathbb{R}^{n\times 2}$. Since PCA first centers $X$, and then projects in the direction of 2 highest eigenvectors of the covariance matrix, I wonder if such PCA solution is identical (up to rigid transformation) to the 2D solution $Y'\in\mathbb{R}^{n\times 2}$ obtained by projecting arbitrarily translated configuration $X\rightarrow X'$ and projecting it in the direction of 2 highest eigenvectors of $\frac{1}{n}(X')^TX'$. $\endgroup$ – usero Apr 12 '12 at 11:51
  • 1
    $\begingroup$ If you center X before doing the SVD, you get of course identical result Y'=Y when you center X' before doing the SVD, as the result of the centering is the same matrix. But if you don't center X' and then take the SVD you generally get something completely different. $\endgroup$ – Arnold Neumaier Apr 12 '12 at 13:43
  • $\begingroup$ I do not center $X'$, but subject it to SVD as it is. I wonder about the possible advantages of such an approach. If you know of some literature in that direction, I would appreciate if you post it here. Thanks. $\endgroup$ – usero Apr 12 '12 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.