Sometimes it is meaningful to remove the influence of some old data from a SVD-based model, so as to reflect the most updated trends and provide more accurate results. I've seen there're incremental SVD approaches adding new data, e.g, a simple one is called folding-in. I am wondering is there any inverse operations which remove the old data, or some selected data specified by me, from a common SVD model?
Yes. Take a look at the 2002 incremental SVD paper by Matthew Brand. In it, he discusses how to re-weight the SVD to dilute the influence of old data. He also has a 2006 paper on rank-1 updates of SVDs, and I believe downdates are also discussed. The best review-type paper on incremental SVDs I've seen was by CG Baker, Van Dooren, and Gallivan in 2012, which might have some other good references, but doesn't directly discuss SVD downdates.