7
$\begingroup$

I was wondering what differences are between the terminology: "least square (LS)" "mean square (MS)" and "least mean square (LMS)"?

I get confused when reading in Spall's Introduction to Stochastic Search and Optimization, section 3.1.2 Mean-Squared and Least-Squares Estimation and section 3.2.1 Introduction and section 3.2.2 Basic LMS Algorithm. Is his terminology standard?

Thanks and regards!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
6
$\begingroup$

The term "mean square" is usually used when one wants to minimize a quantity that can be either positive or negative. Consider a series of values $x_i$ for $i = 1, \ldots, N$. If the $x_i$ are all large positive or large negative numbers, then the average value $\left< x \right>$ of the $x_i$ could still be nearly zero, even though none of the individual values are. However, if you want to consider the mean square of the $x_i$, then you know that $\left< x^2 \right>$ will be small only if the individual $x_i$ are small.

The term "least squares" just refers to the form of regression in which you try to control (minimize) the square of the deviations between the predicted and observed values, while "least mean square" combines these ideas.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.