I am working on a project that I need to add a regularization into the NNLS algorithm. Is there a way to add the Tikhonov regularization into the NNLS implementation of scipy [1]?

[2] talks about it, but does not show any implementation. Sklearn has an implementation, but it is not applied to nnls.

[1] http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.nnls.html

[2] http://icses2012.pwr.wroc.pl/article/34.pdf


Still use

scipy.optimize.nnls(A1, b1), where

A1 = np.concatenate((A, lambda*np.matlib.identity(n)))
b1 = np.concatenate((b, numpy.zeros(shape=(n,1))))
| cite | improve this answer | |
  • $\begingroup$ Thanks lennon310. One more question: how do I define the value of lambda? I am using 1, but I am not sure if it is the best. $\endgroup$ – user3259573 Feb 4 '14 at 1:11
  • $\begingroup$ In theory it depends on the condition number of A. The more ill-posedness A is, the larger lambda value should be chosen. If A is well-posed, lambda should be very close to zero. L curve may be helpful in the parameter selection, check this paper for more details: math.kent.edu/~reichel/publications/tikcrvL.pdf $\endgroup$ – lennon310 Feb 4 '14 at 1:42

Not the answer you're looking for? Browse other questions tagged or ask your own question.