Log-transformation of decision variables in parameter estimation

I am trying to find the diffusion coefficient ($D$) and the partition coefficient ($KLP$) using experimental data of desorption of a pollutant from a film into a liquid. This process can be modelled, under certain assumptions, following Fick's law (related question here for more details on the model)

I am using Matlab's command lsqnonlin to find both parameters from experimental data. In general, $D$ has a range between $10^{-16}$ and $10^{-12} m^2/s$ whereas $KLP$ is comprehended between $10^{-3}$ and $10^{3}$

Sometimes I have been advised to multiply the parameters so that the lowest bounds that lsqnonlin has to deal with are around 1, so that the absolute tolerances are acceptable. Hence, the parameters are to be found are transformed as:

$D^* = D·10^{16}$

$KLP^* = KLP·10^{3}$

My intuition is that using a log-transformation such as

$D^* = log(D)$

$KLP^* = log(KLP)$

is a better solution. But I am not able to prove it or to see the disadvantages of the log-transformation (lower accuracy?) What transformation would be more advisable? And what would be the pros and cons?

1 Answer

Transformations are usually a good idea if they are done to impose domain constraints. Using the exponential of a parameter (so storing the log, which I think is what you mean here) is a good way to constrain to a positive domain, and using the sigmoid of a parameter can put it in a bounded domain (and you can dictate the sharpness between the ends).

Usually the libm (math library) with whatever language you have is able to do log/exp/sigmoid/etc. to 1 ulp so there really isn't much of a numerical issue. In fact, usually it's the opposite: if your parameters can range between 4 orders of magnitude, then taking a log transform can decrease that range and make the gradients stay numerically closer to each other which helps with controlling round-off error.

For this reason, many libraries are actually doing transformations behind the scenes even if users haven't explicitly stated it. For example, if you declare a Stan variable to be positive, it is holding the log and exponentiating as necessary so that the HMC can utilize $p \in \mathbb{R}$ which but still acts strictly positive (in this case it's quite necessary since small numerical errors could overshoot the parameter into the negative regime if this wasn't done!)

• It's a great answer. Can you just clarify what you mean by "Stan variable" and "HMC"? – Toulousain Aug 28 '18 at 8:11
• Stan is a popular Bayesian estimation framework and the variables you define in it are given domains, and they implement those domains via transformations in the background for the user. HMC is Hamiltonian Monte Carlo which is the general MCMC method modern Bayesian estimation tools utilize to sample posteriors. – Chris Rackauckas Aug 28 '18 at 14:07