# Extending the Frobenius inner product to all matrix inner products

So in ${\bf R}^{n\times p}$ we have the Frobenius inner product given by $$\langle A, B\rangle=\text{tr}(A^TB)$$

which can be interpreted as the Euclidean inner product on ${\bf R}^{np}$. My understanding is that all inner products on ${\bf R}^{np}$ can be written as $$a^TPb$$ for $P$ positive-definite. The best I could do in attempting to extend the Frobenius inner product on ${\bf R}^{n\times p}$ is something of the form $$\langle A, B\rangle=\sum_{i=1}^N\text{tr}((X_iAY_i)^T(X_iBY_i))$$ for $X_i\in{\bf R}^{m_i\times n}$ and $Y_i\in{\bf R}^{p\times q_i}$ all full rank. However I would like to know if this covers all inner products on ${\bf R}^{np}$, or if maybe it's more complex than necessary due to redundancies.

I can find the corresponding $P$ matrix for any specific matrix inner product by taking the standard basis for ${\bf R}^{n\times p}$ and forming the matrix

\begin{bmatrix} \langle E_1,E_1\rangle & \langle E_1,E_2\rangle & \dots & \langle E_1,E_{np}\rangle \\ \langle E_2,E_1\rangle & \langle E_2,E_2\rangle & & \vdots \\ \vdots & & \ddots \\ \langle E_{np},E_1\rangle & \dots & \dots & \langle E_{np},E_{np}\rangle \end{bmatrix}

but I don't know if the general form for a matrix inner product I gave above covers all positive-definite matrices $P$.

Update:

newer version of this question on MathOverflow: https://mathoverflow.net/questions/229675/extending-the-trace-inner-product-to-all-matrix-real-inner-products

• Welcome to SciComp.SE! This is an interesting question, but seems much more appropriate for math.stackexchange.com. (Unless there's a connection to a computational science problem I'm missing, in which case it'd be great if you could add that.) – Christian Clason Jan 28 '16 at 23:38
• @ChristianClason, it's related to optimization on matrix manifolds with Riemannian metrics, since Riemannian metrics are inner products on the tangent space. It's almost certainly too advanced for Math.SE, the only other appropriate place would be MathOverflow. I actually may have found what I think is a solution which I may post as an answer once I do the messy work of proving it is a solution, but if you'd like to migrate this to MathOverflow I'm ok with that. I'll add the optimization context when I get a chance. – Thoth Jan 29 '16 at 0:01
• The matrix $P$ also has to be symmetric, not just positive definite. – Wolfgang Bangerth Jan 30 '16 at 0:27
• @WolfgangBangerth, positive-definite is understood to imply symmetric. – Thoth Jan 30 '16 at 0:39
• Not for all authors positive-definiteness implies symmetry. – nicoguaro Jan 30 '16 at 1:44

You can see an inner product as an operation $f(a,b)=\left<a,b\right>$, i.e., it is a bilinear function that (i) returns a non-negative number, (ii) satisfies the relationship $f(a,b)=f(b,a)$.
For vectors $a,b\in\mathbb R^n$, all bilinear functions that satisfy these properties can be written as $$f(a,b) = \sum_{i,j=1}^n a_i P_{ij} b_j$$ where $P$ is symmetric and positive definite. For matrices $a,b\in\mathbb R^{n\times p}$, all such functions can be written as $$f(a,b) = \sum_{i,k=1}^n \sum_{j,l=1}^p a_{ij} P_{ijkl} b_{kl}$$ where now $P$ is a tensor of rank 4 that is symmetric in the sense that $P_{ijkl}=P_{klij}$ and positive definite in the sense that $f(a,a)> 0$ for all $a\neq 0$.
Your question boils down to whether every $P$ that satisfies such conditions can be written a form that results from the vectors $X_i,Y_i$. I believe the answer to this is no. This is simply so because the (for simplicity assuming $n=p$) symmetric $P$ has (asymptotically) $n^4/2$ degrees of freedom, whereas the $n$ vectors $X_i,Y_i$ only have $2n^2$ degrees of freedom. In other words, I don't think that for sufficiently large $n$, your approach has sufficiently many degrees of freedom.
• Yes your argument that the # of parameters grows quarticly in the vector inner product space while only quadratically in the matrix inner product space is compelling, however since the space is ultimately finite we should be able to overcome this by increasing $N$ appropriately. – Thoth Jan 30 '16 at 1:46