# Minimize cost with Levenberg-Marquart method

I want to minimize a cost function of the form,

$$\min_{q,t}\left(q^T\left(\mathcal A + \mathcal B\right)q + t^T\mathcal C t+\delta t+\varepsilon Q(q)^TW(q)t+\lambda\left(1-q^Tq\right)^2\right)$$

with the Levenberg-Marquart method using scipy.optimize.least_squares function. But I do not see how to formulate it in terms of residuals so that I can use such a method. Otherwise, I get the error message:

Method lm doesn't work when the number of residuals is less than the number of variables.

My cost function is defined as follows:

def canonical_cost(qv, t, A, B, C, delta, epsilon, lam):
assert(type(qv) is np.ndarray and len(qv) == 4)
# assert(type(t) is np.ndarray and len(t) == 3)

q = Quaternion(*qv)
qv, tv = qv.reshape(-1, 1), np.vstack((, t.reshape(-1, 1)))

f1 = qv.T @ (A + B) @ qv
f2 = tv.T @ C @ tv + delta @ tv + epsilon @ (q.Q.T @ q.W) @ tv
qnorm = (1 - qv.T @ qv)**2
return np.squeeze(f1 + f2 + lam*qnorm)


And I try to optimize with,

def cost(x):
qv, t = x[:4], x[4:]
return canonical_cost(qv, t, A, B, C, delta, epsilon, lam)

result = opt.least_squares(cost, initial_conditions, method='lm',
**kwargs)

• Usually the Least Squares method is used to estimate parameters of a model given a measurements of samples supposedly taken from this model. Could you explain in your question what is the model and what it the data given?
– Royi
Jan 4, 2019 at 21:31

Perhaps you could augment your output to include sufficient fake residuals of value 0 such that lm accepts your function.