1
$\begingroup$

Before I know how to do tucker decomposition, I mistakenly thought the core tensor is only from combining the singular value matrices of the flattening matrices. Yes I know it is not now. For the tucker decomposition method for a 3 way data's tensor decomposition, a core tensor is calculated using the original tensor(e.g. 3D tensor) and its 3 flattening matrices' left singular vectors. I am wondering if there is any relation existing between the singular values of the flattening matrices and the core tensor somehow? Just out of intuition.

enter image description here

Here are the materials about Tucker decomposition for tensors:

  1. http://en.wikipedia.org/wiki/Tucker_decomposition
  2. Tensor Decompositions and Applications
$\endgroup$
  • $\begingroup$ It would help non-specialist readers of your question if you defined or linked to the Tucker decomposition and flattening matrices. $\endgroup$ – Bill Barth Apr 22 '14 at 11:38
2
$\begingroup$

First, an important distinction must be done. A Tucker decomposition of a third-order tensor $\mathcal{T}$ is any decomposition of the form $$ \mathcal{T} = \mathcal{G} \times_1 \mathbf{A} \times_2 \mathbf{B} \times_3 \mathbf{C}, $$ where $\mathcal{G}$ is also a third-order tensor called core and $\mathbf{A}$, $\mathbf{B}$ and $\mathbf{C}$ are the matrix factors of the decomposition. In scalar form, $$ t_{i_1,i_2,i_3} = \sum_{r_1=1}^{I_1} \sum_{r_2=1}^{I_2} \sum_{r_3=1}^{I_3} g_{r_1,r_2,r_3} a_{i_1,r_1} b_{i_2,r_2} c_{i_3,r_3}. $$ where $I_1, I_2, I_3$ are the dimensions of $\mathcal{T}$. The flat (horizontal) matrix unfoldings of $\mathcal{T}$ are then written as $$ \mathbf{T}_1 = \mathbf{A} \mathbf{G}_1 (\mathbf{C} \otimes \mathbf{B})^T \\ \mathbf{T}_2 = \mathbf{B} \mathbf{G}_2 (\mathbf{A} \otimes \mathbf{C})^T \\ \mathbf{T}_3 = \mathbf{C} \mathbf{G}_3 (\mathbf{B} \otimes \mathbf{A})^T, $$ where $\otimes$ denotes the Kronecker product.

When the factor matrices are the matrices of left singular vectors of the singular value decompositions of each matrix unfolding, then we have a particular Tucker decomposition which is usually called the higher-order singular value decomposition (HOSVD) (see the seminal paper by De Lathauwer et al., ftp://ftp.esat.kuleuven.be/pub/SISTA/delathauwer/reports/ldl-94-31.pdf). Mathematically, expressing the SVDs of $\mathbf{T}_1$, $\mathbf{T}_2$ and $\mathbf{T}_3$ as $$ \mathbf{T}_n = \mathbf{U}^{(n)} \mathbf{\Sigma}^{(n)} {\mathbf{V}^{(n)}}^H, \quad n = 1,2,3, $$ the HOSVD of $\mathcal{T}$ is given by $$ \mathcal{T} = \mathcal{S} \times_1 \mathbf{U}^{(1)} \times_2 \mathbf{U}^{(2)} \times_3 \mathbf{U}^{(3)}, $$ where the core satisfies $$ \mathcal{S} = \mathcal{T} \times_1 {\mathbf{U}^{(1)}}^H \times_2 {\mathbf{U}^{(2)} }^H \times_3 {\mathbf{U}^{(3)}}^H. $$ It is easy to verify the above formula for the core, as $$ \mathbf{S}_1 = {\mathbf{U}^{(1)}}^H \mathbf{T}_1 ({\mathbf{U}^{(3)}}^H \otimes {\mathbf{U}^{(2)}}^H)^T \\ = {\mathbf{U}^{(1)}}^H \mathbf{U}^{(1)} \mathbf{S}_1 ({\mathbf{U}^{(3)}} \otimes {\mathbf{U}^{(2)}})^T ({\mathbf{U}^{(3)}}^H \otimes {\mathbf{U}^{(2)}}^H)^T = \mathbf{S}_1 $$ because each $\mathbf{U}^{(n)}$ is unitary by definition.

Now, the answer to your question can be immediately taken from the above formulas, since substituting the SVD of $\mathbf{T}_1$ in the expression for $\mathbf{S}_1$ we have $$ \mathbf{S}_1 = {\mathbf{U}^{(1)}}^H \mathbf{T}_1 ({\mathbf{U}^{(3)}}^H \otimes {\mathbf{U}^{(2)}}^H)^T = {\mathbf{U}^{(1)}}^H \mathbf{U}^{(1)} \mathbf{\Sigma}^{(1)} {\mathbf{V}^{(1)}}^H ({\mathbf{U}^{(3)}}^H \otimes {\mathbf{U}^{(2)}}^H)^T = \mathbf{\Sigma}^{(1)} {\mathbf{W}^{(1)}}^H $$ where we have defined $$ {\mathbf{W}^{(1)}} = ({\mathbf{U}^{(3)}} \otimes {\mathbf{U}^{(2)}})^T {\mathbf{V}^{(1)}}. $$ Note that $$ {\mathbf{W}^{(1)}}^H {\mathbf{W}^{(1)}} = {\mathbf{V}^{(1)}}^H ({\mathbf{U}^{(3)}}^H \otimes {\mathbf{U}^{(2)}}^H)^T ({\mathbf{U}^{(3)}} \otimes {\mathbf{U}^{(2)}})^T {\mathbf{V}^{(1)}} \\ = {\mathbf{V}^{(1)}}^H [({\mathbf{U}^{(3)}} \otimes {\mathbf{U}^{(2)}}) ({\mathbf{U}^{(3)}}^H \otimes {\mathbf{U}^{(2)}}^H)]^T {\mathbf{V}^{(1)}} \\ = {\mathbf{V}^{(1)}}^H {\mathbf{V}^{(1)}} = \mathbf{I}, $$ and hence $\mathbf{W}^{(1)}$ is unitary. This tells us that the mode-1 unfolding of the core $\mathcal{S}$ is a product of the diagonal matrix of singular values of $\mathbf{T}_1$ by a unitary matrix. Therefore, the norm of the $i$th row of $\mathbf{S}_1$ is precisely the $i$th singular value $\sigma_i^{(1)}$ of $\mathbf{T}_1$. An analogous reasoning applies of course to the other modes.

Finally, observe also that $$ \|\mathcal{S}\|_F^2 = \|\mathbf{S}_n\|_F^2 = \text{Trace}(\mathbf{S}_n\mathbf{S}_n^H) = \sum_i (\sigma_i^{(n)})^2 = \|\mathbf{T}_n\|_F^2 = \|\mathcal{T}\|_F^2,$$ which comes as no surprise, since $\mathcal{S}$ is given by a unitary multilinear transformation of $\mathcal{T}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.