6
$\begingroup$

I'm working in laboratories where molecular dynamics data are almost always analysed usign block average as stated in the famous Allen and Tildesley book.

We divide the datas in blocks of size $M$ on which we compute our interesting quantities. We consider that for large enough blocks, the fluctuations converge so: $$ s=\lim_{M \rightarrow \infty}\frac{M\sigma^2(<X>_M)}{\sigma^2(X)}$$ Where $s$ is called the statisical inefficiency and, when converged for M large enough, is an estimation of the spacing between uncorrelated records. You can then evaluate your statistical error: $$\sigma^2(<X>_{run})=\frac{s}{N}\sigma^2(X)$$ for a run of N records.

Then I stumbled across this set of lecture notes that starts its chapter saying explicitly how block averaging is a naive and mediocre method to evaluate statistical error. Resampling methods seem to be more accurate and easier to implement.

I then have two questions:

1) How are resampling methods an "upgrade" of block averaging? The author states that it's due to the potential large variation of value between blocks but on simple systems which converge really fast to equilibrium, I still have an order of magnitude of difference between bootstrapping and block average. Can the starting values be sufficient to impact the statistical error evaluation?

2) Why is block average so widely used in the molecular dynamics community? I suspect that this reference books did propagate this methods but it was published at a time where resampling became of common use.

I'm not sure that any of my colleagues ever asked this question and my knowledge of statistical analysis is to low for me to really defend my view.

$\endgroup$
  • 3
    $\begingroup$ Super interesting question. I I would maybe cross-post to Cross-Validated, as this is arguably more a statistics than a Comp. Sci question $\endgroup$ – LKlevin Apr 5 '16 at 7:09
  • $\begingroup$ That's a nice idea. Thanks. As I am more in comp. science field I instinctivly asked here. $\endgroup$ – G.Clavier Apr 5 '16 at 11:29
1
$\begingroup$

I guess that you are not actually interested in the variance, but in a confidence interval for your observable $\theta$. It should be noted that computing the confidence interval from the variance (i.e. $\hat{\theta}\pm 2\sigma(\theta)$) is only guaranteed to work when you estimate $\theta$ with a Maximum-Likelihood estimator.

The problem with your "block" approach is, that it uses the data in an inefficient way: you only use $M$ data points in each estimate and only have $(n/M)<<n$ sampled variance estimators. More efficient estimates for the variance are the Jackknife or the Bootstrap, which are both resampling methods leading to $n$ sampled varaince estimators. The Jackknife is very simple to implement and would be an improvement over your "block" method. References:

  1. Efron, Gong: "A leisurely look at thebootstrap, the jackknife, and crossvalidation." The American Statistician 37, pp. 36–48 (1983)
  2. Dalitz: "Construction of confidence intervals." Technical Report No. 2017-01, pp. 15-28, Hochschule Niederrhein, Fachbereich Elektrotechnik und Informatik (2017)
$\endgroup$
  • $\begingroup$ I asked this question a few years back during my PhD and came to the same conclusion. I actually tried to implement bootstrapping methods but did not really convince my colleagues and my PhD adviser. I moved on to other scientific topics now, but that still my approach in building confidence intervals. $\endgroup$ – G.Clavier Apr 14 at 8:43
  • $\begingroup$ @G.Clavier: Thanks for coming back to this post and for sharing your experience. Some feel uneasy with the bootstrap, because the confidence interval is not deterministically determined from the data. This means that two researchers starting with the same data obtain different results (the Jackknife does not have this drawback). OTOH, this should not have bothered your collegues, because in the "block" approach the confidence interval depends on the partitioning of the data, which is random, too. $\endgroup$ – cdalitz Apr 15 at 11:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.