I'm interested in how much 'signal' is retained from including
k singular values in a Singular Value Decomposition, but I'm having trouble conceptualizing (or creating) a metric for 'signal retention'.
I can create $A_i$, a matrix which is created from the first $i$ singular values of $A$, but how do I compare $A_i$ with $A$ and get a reliably meaningful metric? I am currently evaluating
norm(A_i-A)/norm(A), but I'm getting incredibly good results and it doesn't "feel" like the result should be good.
What I mean by that is that I plot the image of a reconstruction with only the first singular value and it looks nothing like the image I started with, but my analysis shows I have only 10% signal loss. After maybe the first 4 or 5 singular values are included can I start to recognize features of my face in the image, but if I can't even recognize my own face, I have a hard time accepting that it is 90% accurate.
Is there a canonical way for doing this? Surely someone else has performed this kind of analysis with SVD, but I'm having trouble finding how (if?) it is canonically done.