I am looking for a very good optimizer to the following problem:

$$\min_{P,\Theta}\lVert APD(\Theta)P^{-1} -B \rVert_F$$ where $A,B \in \mathbb{R}^{n\times m}$, $P \in \mathbb{R}^{m\times m}$, $D\in \mathbb{R}^{m\times m}$.

For my problem, typically $n = O(10^3)$, $m = O(10^2)$.

where $D(\Theta)$ is a matrix of quadratic function of some variable $\Theta$. The dimension of $\Theta $ is exactly $m$.

I have some heuristic, the best I can find is

  • Adam optimizer from tensorflow for first 3000 epoches
  • then use L-BFGS to fine tune local minimum.

I am looking for any suggestions!

  • 1
    $\begingroup$ What are the dimensions of $P$ and $\Theta$? $\endgroup$ Jun 11 '19 at 22:12
  • $\begingroup$ @MarkL.Stone Yes. I just updated. $\endgroup$ Jun 11 '19 at 22:34
  • 1
    $\begingroup$ How are you getting gradient information to perform L-BFGS? Do you have an analytic expression for how this objective varies as the entries of $P$ vary, or are you using some sort of derivative approximation? $\endgroup$ Jun 12 '19 at 0:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.