# How to optimize sampling for parameter estimation

I have a computer model with a number of parameters that need to be calibrated based on experimental results. It's also important to understand the sensitivity of the results to each parameter individually. The intuitive approach to this problem is to sample each parameter (say m realization from each parameter) and conduct a factorial experiment to see the main effects and interactions between parameters. As it's repeatedly reported, the problem with this approach is its computational costs as for a model with 10 parameters, even with only two-level experiment, it would cost $$2^{10}$$ runs, which is not feasible for my model.

I am therefore searching for an alternative sampling approach that rescues me from this headache. Any guidance is appreciated.

• I am not sure if I'm properly understanding what you're trying to do, but it seems like you want to avoid doing a factorial sampling of your space and want an alternative. A simple alternative is Monte Carlo sampling and another alternative is Latin Hypercube sampling. Monte Carlo sampling should make the error bound of integrals (assuming that's what you're approximately computing with this Bayesian stuff) $O\left(\frac{1}{\sqrt{n}}\right)$ regardless of the dimensionality of the integral, where $n$ is the number of samples. So depending on the error you want, this might be sufficient. – spektr Jul 9 '19 at 17:07
• Just doing a quick google, it appears that a random Latin Hypercube approach can have error bounded by $O\left(\frac{1}{n}\right)$ in some lower dimensional settings, so this may actually be better depending. Here is a link I just found and started to read that you might find interesting: lumina.com/blog/latin-hypercube-vs.-monte-carlo-sampling – spektr Jul 9 '19 at 17:11
• LHS? NUTS! the trouble with LHS is that it leads to clustered sample points in high dimensional spaces, so you may completely miss very relevant regions of your probability space (=> biased estimates). if you can build a reasonable, differentiable approximation to the joint PDF of the parameters, I'd suggest you try a state-of-the art Hamiltonian Monte Carlo sampler (for continuous parameters). – GoHokies Jul 10 '19 at 5:58
• also, LHS is completely oblivious to things like concentration of measure, i.e. the unintuitive behavior of volume in high-dimensions. If all you know a priori are (1) the parameter correlations and (2) their upper and lower bounds, you could approximate their joint PDF by a Gaussian copula with uniform marginals and use HMC (NUTS) to sample that efficiently. – GoHokies Jul 10 '19 at 9:57
• Thanks you both for your inputs. Since I am new to the concepts that you are introducing, let me ask you a question before me diving into them. In my simulation, I am also interested to know the sensitivity of the results to each parameter. In Factorial experiments, two factors of "main effect" and "interaction" can be used to interpret the sensitivity. Would it be also possible to get this type of information from the methods you suggested? – Jalil Nourisa Jul 10 '19 at 11:40