Are there any tools specifically designed for compressing floating point scientific data?
If a function is smooth, there's obviously a lot of correlation between the numbers representing that function, so the data should compress well. Zipping/gzipping binary floating point data doesn't compress it that well though. I am wondering if there are methods specifically developed for compressing floating point data.
Either lossless compression or the possibility to specify a minimum number of digits to retain (for some applications
doublemight be more than what we need while
floatmight not have enough precision).
Well tested working tool (i.e. not just a paper describing a theoretical method).
Suitable for compressing 1D numerical data (such as a time series)
Cross platform (must work on Windows)
It must be fast---preferably not much slower than gzip. I found that if I have the numbers stored as ASCII, gzipping the file can speed up reading and processing it (as the operation might be I/O bound).
I'd especially like to hear from people who have actually used such a tool.