I want to minimize a cost function of the form,

$$ \min_{q,t}\left(q^T\left(\mathcal A + \mathcal B\right)q + t^T\mathcal C t+\delta t+\varepsilon Q(q)^TW(q)t+\lambda\left(1-q^Tq\right)^2\right) $$

with the Levenberg-Marquart method using scipy.optimize.least_squares function. But I do not see how to formulate it in terms of residuals so that I can use such a method. Otherwise, I get the error message:

Method lm doesn't work when the number of residuals is less than the number of variables.

My cost function is defined as follows:

def canonical_cost(qv, t, A, B, C, delta, epsilon, lam):
    assert(type(qv) is np.ndarray and len(qv) == 4)
    # assert(type(t) is np.ndarray and len(t) == 3)

    q = Quaternion(*qv)
    qv, tv = qv.reshape(-1, 1), np.vstack(([0], t.reshape(-1, 1)))

    f1 = qv.T @ (A + B) @ qv
    f2 = tv.T @ C @ tv + delta @ tv + epsilon @ (q.Q.T @ q.W) @ tv
    qnorm = (1 - qv.T @ qv)**2
    return np.squeeze(f1 + f2 + lam*qnorm)

And I try to optimize with,

def cost(x):
    qv, t = x[:4], x[4:]
    return canonical_cost(qv, t, A, B, C, delta, epsilon, lam)

result = opt.least_squares(cost, initial_conditions, method='lm',
  • $\begingroup$ Usually the Least Squares method is used to estimate parameters of a model given a measurements of samples supposedly taken from this model. Could you explain in your question what is the model and what it the data given? $\endgroup$
    – Royi
    Jan 4, 2019 at 21:31

1 Answer 1


Perhaps you could augment your output to include sufficient fake residuals of value 0 such that lm accepts your function.

  • $\begingroup$ Good idea, but the solution may not be unique. scipy's least_squares method has some regularization options that could help with this. $\endgroup$ Sep 27, 2019 at 21:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.