I want to minimize a cost function of the form,

$$ \min_{q,t}\left(q^T\left(\mathcal A + \mathcal B\right)q + t^T\mathcal C t+\delta t+\varepsilon Q(q)^TW(q)t+\lambda\left(1-q^Tq\right)^2\right) $$

with the Levenberg-Marquart method using scipy.optimize.least_squares function. But I do not see how to formulate it in terms of residuals so that I can use such a method. Otherwise, I get the error message:

Method lm doesn't work when the number of residuals is less than the number of variables.

My cost function is defined as follows:

def canonical_cost(qv, t, A, B, C, delta, epsilon, lam):
    assert(type(qv) is np.ndarray and len(qv) == 4)
    # assert(type(t) is np.ndarray and len(t) == 3)

    q = Quaternion(*qv)
    qv, tv = qv.reshape(-1, 1), np.vstack(([0], t.reshape(-1, 1)))

    f1 = qv.T @ (A + B) @ qv
    f2 = tv.T @ C @ tv + delta @ tv + epsilon @ (q.Q.T @ q.W) @ tv
    qnorm = (1 - qv.T @ qv)**2
    return np.squeeze(f1 + f2 + lam*qnorm)

And I try to optimize with,

def cost(x):
    qv, t = x[:4], x[4:]
    return canonical_cost(qv, t, A, B, C, delta, epsilon, lam)

result = opt.least_squares(cost, initial_conditions, method='lm',
  • $\begingroup$ Usually the Least Squares method is used to estimate parameters of a model given a measurements of samples supposedly taken from this model. Could you explain in your question what is the model and what it the data given? $\endgroup$ – Royi Jan 4 '19 at 21:31

Perhaps you could augment your output to include sufficient fake residuals of value 0 such that lm accepts your function.

  • $\begingroup$ Good idea, but the solution may not be unique. scipy's least_squares method has some regularization options that could help with this. $\endgroup$ – mathew gunther Sep 27 '19 at 21:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.