So in ${\bf R}^{n\times p}$ we have the Frobenius inner product given by $$\langle A, B\rangle=\text{tr}(A^TB)$$

which can be interpreted as the Euclidean inner product on ${\bf R}^{np}$. My understanding is that all inner products on ${\bf R}^{np}$ can be written as $$a^TPb$$ for $P$ positive-definite. The best I could do in attempting to extend the Frobenius inner product on ${\bf R}^{n\times p}$ is something of the form $$\langle A, B\rangle=\sum_{i=1}^N\text{tr}((X_iAY_i)^T(X_iBY_i))$$ for $X_i\in{\bf R}^{m_i\times n}$ and $Y_i\in{\bf R}^{p\times q_i}$ all full rank. However I would like to know if this covers all inner products on ${\bf R}^{np}$, or if maybe it's more complex than necessary due to redundancies.

I can find the corresponding $P$ matrix for any specific matrix inner product by taking the standard basis for ${\bf R}^{n\times p}$ and forming the matrix

\begin{bmatrix} \langle E_1,E_1\rangle & \langle E_1,E_2\rangle & \dots & \langle E_1,E_{np}\rangle \\ \langle E_2,E_1\rangle & \langle E_2,E_2\rangle & & \vdots \\ \vdots & & \ddots \\ \langle E_{np},E_1\rangle & \dots & \dots & \langle E_{np},E_{np}\rangle \end{bmatrix}

but I don't know if the general form for a matrix inner product I gave above covers all positive-definite matrices $P$.


newer version of this question on MathOverflow: https://mathoverflow.net/questions/229675/extending-the-trace-inner-product-to-all-matrix-real-inner-products

  • $\begingroup$ Welcome to SciComp.SE! This is an interesting question, but seems much more appropriate for math.stackexchange.com. (Unless there's a connection to a computational science problem I'm missing, in which case it'd be great if you could add that.) $\endgroup$ Commented Jan 28, 2016 at 23:38
  • 1
    $\begingroup$ @ChristianClason, it's related to optimization on matrix manifolds with Riemannian metrics, since Riemannian metrics are inner products on the tangent space. It's almost certainly too advanced for Math.SE, the only other appropriate place would be MathOverflow. I actually may have found what I think is a solution which I may post as an answer once I do the messy work of proving it is a solution, but if you'd like to migrate this to MathOverflow I'm ok with that. I'll add the optimization context when I get a chance. $\endgroup$
    – Set
    Commented Jan 29, 2016 at 0:01
  • $\begingroup$ The matrix $P$ also has to be symmetric, not just positive definite. $\endgroup$ Commented Jan 30, 2016 at 0:27
  • $\begingroup$ @WolfgangBangerth, positive-definite is understood to imply symmetric. $\endgroup$
    – Set
    Commented Jan 30, 2016 at 0:39
  • 1
    $\begingroup$ Not for all authors positive-definiteness implies symmetry. $\endgroup$
    – nicoguaro
    Commented Jan 30, 2016 at 1:44

1 Answer 1


You can see an inner product as an operation $f(a,b)=\left<a,b\right>$, i.e., it is a bilinear function that (i) returns a non-negative number, (ii) satisfies the relationship $f(a,b)=f(b,a)$.

For vectors $a,b\in\mathbb R^n$, all bilinear functions that satisfy these properties can be written as $$ f(a,b) = \sum_{i,j=1}^n a_i P_{ij} b_j $$ where $P$ is symmetric and positive definite. For matrices $a,b\in\mathbb R^{n\times p}$, all such functions can be written as $$ f(a,b) = \sum_{i,k=1}^n \sum_{j,l=1}^p a_{ij} P_{ijkl} b_{kl} $$ where now $P$ is a tensor of rank 4 that is symmetric in the sense that $P_{ijkl}=P_{klij}$ and positive definite in the sense that $f(a,a)> 0$ for all $a\neq 0$.

Your question boils down to whether every $P$ that satisfies such conditions can be written a form that results from the vectors $X_i,Y_i$. I believe the answer to this is no. This is simply so because the (for simplicity assuming $n=p$) symmetric $P$ has (asymptotically) $n^4/2$ degrees of freedom, whereas the $n$ vectors $X_i,Y_i$ only have $2n^2$ degrees of freedom. In other words, I don't think that for sufficiently large $n$, your approach has sufficiently many degrees of freedom.

  • $\begingroup$ I actually believe the answer is yes, I'm going to repost this question on math overflow with my updated results. $\endgroup$
    – Set
    Commented Jan 30, 2016 at 0:41
  • $\begingroup$ Yes your argument that the # of parameters grows quarticly in the vector inner product space while only quadratically in the matrix inner product space is compelling, however since the space is ultimately finite we should be able to overcome this by increasing $N$ appropriately. $\endgroup$
    – Set
    Commented Jan 30, 2016 at 1:46
  • $\begingroup$ My apologies I posted a newer version of this question on MathOverflow, however it's sufficiently updated I thought that appropriate, here is the link in case you want to transfer your answer over there or update your answer based on the newer version. mathoverflow.net/questions/229675/… $\endgroup$
    – Set
    Commented Jan 30, 2016 at 1:46
  • $\begingroup$ @Thoth Note that @ ChristianClason advised you to post your question on math.stackexchange.com, not on mathoverflow.net. Those are two different sites with different purposes and audiences. $\endgroup$ Commented Jan 30, 2016 at 12:01
  • $\begingroup$ @FedericoPoloni yes I know, and if you read what I wrote I told him I thought it was too advanced for Math.SE and would be unlikely to get an answer there. $\endgroup$
    – Set
    Commented Jan 30, 2016 at 19:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.