# Differences between “least square”, “mean square” and “least mean square”?

I was wondering what differences are between the terminology: "least square (LS)" "mean square (MS)" and "least mean square (LMS)"?

I get confused when reading in Spall's Introduction to Stochastic Search and Optimization, section 3.1.2 Mean-Squared and Least-Squares Estimation and section 3.2.1 Introduction and section 3.2.2 Basic LMS Algorithm. Is his terminology standard?

Thanks and regards!

## 1 Answer

The term "mean square" is usually used when one wants to minimize a quantity that can be either positive or negative. Consider a series of values $x_i$ for $i = 1, \ldots, N$. If the $x_i$ are all large positive or large negative numbers, then the average value $\left< x \right>$ of the $x_i$ could still be nearly zero, even though none of the individual values are. However, if you want to consider the mean square of the $x_i$, then you know that $\left< x^2 \right>$ will be small only if the individual $x_i$ are small.

The term "least squares" just refers to the form of regression in which you try to control (minimize) the square of the deviations between the predicted and observed values, while "least mean square" combines these ideas.