I am solving a system using singular value decomposition. The singular values (before scaling) are:
1.82277e+29
1.95011e+27
1.15033e+23
1.45291e+21
4.79336e+17
7.48116e+15
8.31087e+12
1.71838e+11
5.63232e+08
2.17863e+08
9.02783e+07
1.72345e+07
1.73889e+05
8.09382e+02
2.16644e+00
I have found that accepting all the singular values and their associated contribution to my solution vector yields poor results. I scale them all by the largest number, yielding singular values of:
1.0
1.06986e-02
6.31091e-07
7.97089e-09
2.62971e-12
4.10428e-14
4.55948e-17
9.42732e-19
3.08998e-21
1.19523e-21
4.95281e-22
9.45510e-23
9.53980e-25
4.44040e-27
1.18854e-29
The best solution only starts to become bad if I include the last two, and only become good around the $10^{-19}$ term.
There is a sharp drop in accuracy when I include the last 2 terms. Why is that? What are the criteria for including/not including singular values?
My matrix equation comes from a linear least squares fitting where I am using a polynomial basis set to fit some noisy data I created. I am solving the standard overdetermined system ($m \times n$ matrix where $m \gg n$) by multiplying each side ($A\cdot X = B$) by the transpose of $A$ ($A^\top A X = A^\top X$) and performing SVD on that.
I am judging the answers to my solutions by how well it approximates my noisy data.
I have also noticed that, even on the 'good' fits, I am not fitting very well near zero (my data ranges from $-10$ to $10$). Why is that?