1
$\begingroup$

I have heard that one should avoid varying all parameters in a GSA by the same amount (e.g., 10%, 25% etc.) because it suggests that uncertainty is the same for all parameters. This makes sense to me if we have decent estimates of uncertainty for all the input parameters. However, what happens if there are many parameters and many or most of them are very poorly estimated? Perhaps many or all of these parameters don't have associated confidence intervals at all, but we would still like to know their relative effects on output variance. 1) In this situation, wouldn't varying all parameters by the same percentage provide some useful information on the relative importance of each parameter? Think, for example, of a theoretical model in which the parameter values are arbitrarily chosen, but we want to know how each parameter affects model output.

And more generally, if parameters are varied according to their degree of uncertainty, 2) wouldn't highly uncertain parameters be more likely to be identified as highly influential during a sensitivity analysis simply because they are varied across a wider range of values?

$\endgroup$

1 Answer 1

2
$\begingroup$

I do not think it is always bad; such a sensitivity analysis can show you how the model behaves for the given range of input parameters, which may be informative. This can reveal qualitative information such as which parameters may have interactive effects and what is the range of model behavior for the range of inputs. This may lead further investigation or make one aware of under what conditions certain parameters may or may not be influential.

Of course the exact quantitative measures of sensitivity will be dependent on the exact ranges, so interpreting these results as precise measures of importance for an application is limited, and more informed ranges are preferred, but if limited information is available one could assess how different assumptions of uncertainty affect the results. This is one aspect of sensitivity auditing, which is focused on assessing how reliable a model's conclusions are under various assumptions.

Whether or not parameters with higher relative uncertainty will be more influential is very model dependent, and this is essentially why sensitivity analysis is informative. Uncertainty about a parameter that has a saturating effect (e.g. as the argument of a sigmoid), or that just has a relatively smaller "coefficient" can be perturbed a large amount with only a small change in the model output. For example, consider a linear model $y=100 x_1 + 0.1 x_2$ with two inputs $x_1$ and $x_2$. Even if the uncertainty on $x_2$ is 100 times larger than $x_1$, $x_1$ is still more influential.

I have used "coefficient" here for simplicity. In more complex models it will not be so simple, but the qualitative principle is the same, and this "coefficient" is generally what sensitivity analyses (local or global) are trying to quantify and then weight by input uncertainty. Local sensitivity analysis approaches this directly by a linear approximation, $\frac{\partial y}{\partial x_i}$, while global methods account for the varying effect of $x_i$ on model output over different points in the input space.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.