Imposing special structure on Positive Semi-Definite matrix

Question

I am trying to implement the algorithm described in reference 1 using cvxpy. However I am struggling to constrain the matrix $Z_j$ as described in equations (33-35). Citing the paper ($\bar{\mu_j}$ is just a vector):

Remaining constraint on $R_j$ is:

$$Z_j= \left[ \begin{matrix} 1 && (\bar{\mu_j})^{'} \\ \bar{\mu_j} && U \end{matrix} \right]$$

where $U$ must be $\bar{\mu_j}(\bar{\mu_j})^{'}$. since the constraint is not linear, we relax it such that:

$$U-\bar{\mu_j}(\bar{\mu_j})^{'}\succeq 0$$

by the property that:

$$Z_j\succeq0 \Leftrightarrow U-\bar{\mu_j}(\bar{\mu_j})^{'}\succeq0$$

where $Z_j \succeq 0$ denotes that $Z_j$ is positive semi-definite matrix.

I tried to force this structure by doing the following:

u = cvx.Variable(D)
U = u*u.T
Zj = cvx.bmat([[1, u.T], [u, U]])

However cvxpy complains that u*u.T is incorrect:

UserWarning: Forming a nonconvex expression (affine)*(affine).

And solver fails:

cvxpy.error.DCPError: Problem does not follow DCP rules

It appears that outer product of two vectors cannot be implemented as such. Any ideas on how to enforce this constraint?

References

1. Yun, Sungrack, and Chang D. Yoo. "Loss-scaled large-margin Gaussian mixture models for speech emotion classification." IEEE Transactions on Audio, Speech, and Language Processing 20.2 (2012): 585-59

Answer 1

The whole point is that you do NOT include the constraint $U = uu^T$ . That constraint is non-convex.

Instead you include the constraint that $Z_j$ is (positive) semi-definite. This constraint is a convex relaxation of $U = uu^T$, obtained by Schur complement. I.e., the relaxed constraint, which is convex, is used instead of the original non-convex constraint.