Matlab fmincon with zero user-supplied hessian

Question

I have to solve the problem

$$\min_x 1^Tx+\frac{\lambda}{2}\|\Omega\mu-x\|_2^2+\frac{\beta}{2}\|x-\bar{\gamma}\|_2^2\quad\text{w.r.t.}\quad Px-c=0,\ \ x\geq0$$

and in order to do that with Matlab I am currently using fmincon like that:

FMinValue = @(x) sum(x)            + lambda * sum((Omega*mu - x).^2) / 2 + beta * sum((x - gamma_bar).^2) / 2;
GradFMin  = @(x) ones(length(x),1) + lambda * (Omega*mu - x)             + beta * (x - gamma_bar);
Fmin = @(x) FMin(x, FMinValue, GradFMin); % merges FMinValue and GradFMin so that fmincon can handle it
HessianFPlusLagrangianPart = @(x, lagrange_multiplier) spdiags((beta-lambda)*ones(length(x),1),0,length(x),length(x));

options = optimoptions('fmincon', 'Algorithm', 'interior-point' , 'GradObj', 'on', 'Hessian', 'user-supplied', 'HessFcn', HessianFPlusLagrangianPart);
x = fmincon(Fmin, ones(length(gamma_bar), 1), [], [], P_bar, c_bar, zeros(length(gamma_bar), 1), [], [], options);

Here $\lambda$ and $\beta$ are fixed parameters which could be equal. If that is the case then $\lambda-\beta=0$ which means my hessian matrix is zero as well. Of course I get the Matlab warning Warning: Matrix is close to singular or badly scaled, but a zero hessian is correct in this case.

Any suggestions what to do in this situation? Unfortunately I cannot change to lsqnonlin because I have got the linear term sum(x) in my objectiv function. And I cannot use another algorithm instead of interior-point either, since I have bound and linear constraints at the same time.

How to overcome this problem and make it work for all choices of $\lambda$ and $\beta$?

Answer 1

The problem is (likely) that your gradient and Hessian are incorrect: $$\nabla \left(\frac\lambda 2 \|\Omega\mu-x\|_2^2 \right)= \lambda(\Omega\mu-x)(-1) = \lambda(x-\Omega\mu)$$ and hence the corresponding part of the Hessian is $$\nabla^2 \left(\frac\lambda 2 \|\Omega\mu-x\|_2^2 \right) = \lambda I.$$

Answer 2

One way you could overcome this situation is to correct the Hessian of your objective function, which should be $(\lambda + \beta)I$, and then see if you still have scaling issues.