# Minimization of least square function together with a nonlinear function

I have the linear underdetermined system $$Ax=b$$ and I need to find $x$ constrained by the maximization of a score function $g(x)$.

I could find the minimum of a function like $$|Ax-b|^2+1/g(x)$$ but I wanted to know if there exists a canonical way to solve this problem, exploiting the linear part to have a more regular convergence.

I'm using python with scikit-learn, but I can also try to convert MATLAB code to solve it.

EDIT. The $g(x)$ function is not convex and has many maximums not necessarily where $Ax=b$.

• How does $g(x)$ look like? What you are wanint to do, of course, is maximize $g(x)$ over all of the solutions of $Ax=b$; whether this is simple or not depends on the structure of $g(x)$. I'm going to note, however, that that's not the formulation you have -- in your minimization problem, the solution does not necessarily solve $Ax=b$. – Wolfgang Bangerth Oct 30 '17 at 23:24
• What assumptions can be mde about the function $1/g(x)$? Is it convex? differentiable? – Brian Borchers Oct 30 '17 at 23:49
• @wolfgangbangerth you are right. You can use $Ax=b$ as a sort of equality constraint for the maximization problem, but I can't find an optimization function that has the constraints expressed as a linear combination of the solution vector. I would need to give $|Ax-B|^2$ as an equality constraint function and manually evaluate the Jacobian of this function to speed up the convergence. I'd like to know if there is an easier way to express this problem. – N74 Oct 31 '17 at 3:14
• Posing $Ax=b$ as a (linear) constraint does not make the problem any more complicated and seems like the way to go. In general, linear constraints are trivial to deal with and don't add much complexity. Can you state the actual form of $g(x)$ and why you chose that one? – Wolfgang Bangerth Oct 31 '17 at 4:02
• @wolfgangbangerth I don't have the analytical expression of $g(x)$ as it is evaluated by a piece of code I import. I know it expresses the likelihood that the solution vector belongs to a class. I suppose it is something like a gaussian mixture but I can't be sure. – N74 Oct 31 '17 at 6:18

• I tried minimize... but it is annoying that I must create a lambda function for each row of the $Ax=b$ constraint. I looked into the SLSQP solver source code and it seems that the Fortran program called by minimize actually needs $A_{eq}$ and $b_{eq}$, but I did not find a way to pass them directly. – N74 Nov 2 '17 at 12:53