# Minimize squared error of linear function

Let $$M$$ be a $$m \times n$$ matrix, $$x$$ a $$n$$-vector, $$y$$ a $$m$$-vector, and $$\|\cdot\|_2$$ represent the $$L_2$$ norm (i.e., Euclidean norm). Given $$M,y$$, the goal is to find $$x$$ that minimizes the expression

$$\Psi(x) = \|x\|^2 + \|Mx-y\|^2.$$

Is there a closed-form expression for the optimal $$x$$? Without the $$\|x\|^2$$ term, this would be an instance of ordinary least squares linear regression and there would be a nice solution, but I'm not sure happens with this additional term.

Your problem is still a linear least squares problem. You can write $$\Psi(x)$$ as

$$\Psi(x)=\| Hx - g \|_{2}^{2}$$

where

$$H=\left[ \begin{array}{c} I \\ M \end{array} \right]$$

and

$$g=\left[ \begin{array}{c} 0 \\ y \end{array} \right]$$.

Using the normal equations,

$$x=(H^{T}H)^{-1}H^{T}g=(M^{T}M+I)^{-1} M^{T}y$$.