This question is not very deep. Suppose I have a small rectangular matrix $A$, with number of rows and columns between $50$-$100$, respectively.
Given a right-hand side $b$, I want to solve the least-squares problem $Ax = b$, i.e., I want to find $x$ with norm-minimal residual $r = b - Ax$ and $x$ having minimum norm among these minimizers.
Since the system is very small, and $A$ has a good generalized condition number, I consider solving the normal equations $A^t A x = A^t b$ instead, say, by Cholesky decomposition.
At this stage, efficiency is less important than getting accurate solutions. Would you argue, from your experience, that this is a good choice, rather than, say, computing the SVD?