I have a simulation that can generate quite a bit of data when it runs, for example $650\cdot 400 \cdot 400$ floating point numbers. Without compression, that's a few gigabytes worth if I want to save the whole simulation and not just some data extracted from it. However, with some methods, like Crank-Nicolson, we know the maximum error that can occur, (it has second order accuracy). Suppose I calculate it, then round or truncate all the values in my $650 \cdot 400 \cdot 400$ matrix to the same number of digits (or the same number of digits plus 1): $$tolerence = .00004$$ $$example = .45435354354 \rightarrow example_{rounded} = .454354$$

If I know a maximum value I can now used fixed point math and potentially get some very good compression using algorithms for integers (as storing compressed floating points can be difficult, and fixed-point numbers work similarly to integers "under the hood"). This would also allow me to use binary storage without much difficulty. In any way is this inadvisable? Might I loose important information, even if it is below tolerance? Will this affect the analysis I do on the data (even if tolerance [error] is preserved [is it?])?

  • 1
    $\begingroup$ Well, you lose information. The question is what you want to do with the data and if that information is important. We cannot tell you that without knowing what you want to do with the data. $\endgroup$ Commented Mar 14, 2023 at 23:28
  • $\begingroup$ Another approach is to run the analysis of interest together with your code. Often one is only interestet in dervied measures anyway + 1 pretty plot. When you calculate the averages/Energies/fluxes/variances on the go, you may not have to store the full time series data at all. (If your runs are reproducible from the initial conditions!) $\endgroup$
    – MPIchael
    Commented Mar 23, 2023 at 7:25

1 Answer 1


I would recommend keeping the computation in your original precision and reducing the accuracy just when you write, unless the computation is too slow. If you reduce the precision of the computation, you may need to worry about the accuracy of the linear solve.

First, you should consider whether you need to write all of the time steps. I'd guess that Crank-Nicolson needs a smaller time step than your analysis/visualization. Casting the data to a lower-accuracy floating- or fixed-point when you write it to file is similar. Both of these are used by major scientific codes (e.g., NekRS).

If you need more compression, I'd recommend a floating-point compressor meant for scientific data. This is a hot topic since some of the big simulations can generate petabytes per run. Off the top of my head, there's ZFP and SZ. Both are available as libraries, so they're probably even easier to use than fixed-point.

I doubt using integer compression on fixed-point numbers will be effective. Most integer compression schemes assume the data has particular structure. For example, that the data only contains a few distinct values or that most of the values are small. Neither fixed-point and floating-point satisfy these types of assumptions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.