I am working on a project that I need to add a regularization into the NNLS algorithm. Is there a way to add the Tikhonov regularization into the NNLS implementation of scipy [1]?
[2] talks about it, but does not show any implementation. Sklearn has an implementation, but it is not applied to nnls.
[1] http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.nnls.html
If what you want is to solve
$\min \| Ax - b \|_{2}^{2} + \lambda^{2} \| x \|_{2}^{2}$
subject to
$x \geq 0$,
then this is easily implemented. Construct a matrix
$C=\left[ \begin{array}{c} A \\ \lambda I \end{array} \right]$
and a vector
$d=\left[ \begin{array}{c} b \\ 0 \end{array} \right]$.
Then use your nonnegative least squares solver on
$\min \| Cx - d \|_{2}^{2}$
subject to
$x \geq 0$.
You can easily extend this to problems of the form
$\min \| Ax-b \|_{2}^{2} + \lambda^{2} \| L (x-x_{0}) \|_{2}^{2}$
by letting
$C=\left[ \begin{array}{c} A \\ \lambda L \end{array} \right]$
and
$d=\left[ \begin{array}{c} b \\ \lambda Lx_{0} \end{array} \right]$.
If the fortran code https://github.com/scipy/scipy/blob/master/scipy/optimize/nnls/nnls.f doesn't have a Tikhonov regularization option, then it would probably take some work to add this feature to scipy.
Edit: I looked at the .pdf. I guess you want to find the smallest $\lambda$ so that $\text{argmin}_x ||Ax-b||^2 + \lambda^2 ||x||^2$ is positive. I think the $\text{argmin}_x$ for any given $\lambda$ has a more or less closed form solution, and the .pdf mentions bisection search on $\lambda$ until you find the smallest one so that the best $x$ is positive. This seems pretty straightforward, in the sense that it shouldn't need any weird decompositions or messy fortran code.
Edit 2: I noticed that this question is tagged convex-optimization. If you want to take that approach literally, then you could try cvx (http://cvxr.com/cvx/) or cvxpy (https://github.com/cvxgrp/cvxpy).
