# Best optimizer for unconnstrained non-convex nonlinear least-square optimization problem?

I am looking for a very good optimizer to the following problem:

$$\min_{P,\Theta}\lVert APD(\Theta)P^{-1} -B \rVert_F$$ where $$A,B \in \mathbb{R}^{n\times m}$$, $$P \in \mathbb{R}^{m\times m}$$, $$D\in \mathbb{R}^{m\times m}$$.

For my problem, typically $$n = O(10^3)$$, $$m = O(10^2)$$.

where $$D(\Theta)$$ is a matrix of quadratic function of some variable $$\Theta$$. The dimension of $$\Theta$$ is exactly $$m$$.

I have some heuristic, the best I can find is

• Adam optimizer from tensorflow for first 3000 epoches
• then use L-BFGS to fine tune local minimum.

I am looking for any suggestions!

• What are the dimensions of $P$ and $\Theta$? Jun 11 '19 at 22:12
• @MarkL.Stone Yes. I just updated. Jun 11 '19 at 22:34
• How are you getting gradient information to perform L-BFGS? Do you have an analytic expression for how this objective varies as the entries of $P$ vary, or are you using some sort of derivative approximation? Jun 12 '19 at 0:18