I have just read yet another introduction to image processing, which describes Gaussian smoothing from a convolution perspective and least-squares smoothing from an optimization perspective.

It would help me knit this knowledge together if I could imagine how I could also use optimization to obtain a Gaussian-smoothed signal as well, but I never see this.

Given that a least squares smoothing is obtained from

$$ argmin_{x} \| Ax-b \|^{2}_2 + \lambda \| \nabla x \|_2 $$

What corresponding penalty function would produce a Gaussian smoothing?

  • $\begingroup$ p.s. someone should create a "regularization" tag for this exchange. $\endgroup$ – barnhillec Jan 26 '17 at 13:09
  • 1
    $\begingroup$ I don't think you can express Gaussian smoothing via regularization. Do you have any evidence that you can? $\endgroup$ – Wolfgang Bangerth Jan 26 '17 at 20:57
  • $\begingroup$ Very interesting question. First of all I assume that any technique that can be expressed as Ax = b is also solvable by variational means and therefore solvable using regularization. Second I assume that any direct forward technique such as Gaussian smoothing,which can produce a smoothed y in the case of Ax = y where A is a Gaussian smoothing matrix, can also produced a smoothed x in the case of Bx = y, where B is the inverse of A, hence is solved by x = inv(B)y =inv(inv(A))y = Ay . Is this not correct? $\endgroup$ – barnhillec Jan 27 '17 at 9:15
  • $\begingroup$ I agree that a linear equation can be rewritten as a variational problem, but that doesn't necessarily mean that there will be a regularisation problem attached to it. Since Gaussian smoothing can work point wise through superposition there's no over- or under-fitting needing to be sorted out. $\endgroup$ – origimbo Jan 27 '17 at 14:22

The least squares optimization problem in your question has a penalty term that only penalizes 1st derivatives. However, for a Gaussian kernel, the corresponding penalty term would have to penalize all derivatives because Gaussians are infinitely smooth.

A mathematically precise way to link various penalty terms to smoothing kernel functions would require the theory of reproducing kernel Hilbert spaces (RKHS). The penalty term for a kernel $K$ is the corresponding RKHS norm $|| \cdot ||_K$. For a Gaussian kernel this is given by: $$ ||f||_K = \sum_{m=0}^\infty \frac{\sigma^{2m}}{2^m m!} \int_{-\infty}^{\infty} [f^{(m)}(t)]^2 dt $$ where $f^{(m)}$ is the $m^{th}$ derivative and $\sigma$ is the standard deviation of the Gaussian kernel.

Page 212 here: https://alex.smola.org/papers/2004/SmoSch04.pdf

Other references: http://www.mit.edu/~9.520/spring07/Classes/class03_rkhs.pdf https://people.eecs.berkeley.edu/~jordan/courses/281B-spring01/lectures/regularization.ps

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.