Log-transformation of decision variables in parameter estimation

Question

I am trying to find the diffusion coefficient ($D$) and the partition coefficient ($KLP$) using experimental data of desorption of a pollutant from a film into a liquid. This process can be modelled, under certain assumptions, following Fick's law (related question here for more details on the model)

I am using Matlab's command lsqnonlin to find both parameters from experimental data. In general, $D$ has a range between $10^{-16}$ and $10^{-12} m^2/s$ whereas $KLP$ is comprehended between $10^{-3}$ and $10^{3}$

Sometimes I have been advised to multiply the parameters so that the lowest bounds that lsqnonlin has to deal with are around 1, so that the absolute tolerances are acceptable. Hence, the parameters are to be found are transformed as:

$D^* = D·10^{16}$

$KLP^* = KLP·10^{3}$

My intuition is that using a log-transformation such as

$D^* = log(D)$

$KLP^* = log(KLP)$

is a better solution. But I am not able to prove it or to see the disadvantages of the log-transformation (lower accuracy?) What transformation would be more advisable? And what would be the pros and cons?

Answer 1

Transformations are usually a good idea if they are done to impose domain constraints. Using the exponential of a parameter (so storing the log, which I think is what you mean here) is a good way to constrain to a positive domain, and using the sigmoid of a parameter can put it in a bounded domain (and you can dictate the sharpness between the ends).

Usually the libm (math library) with whatever language you have is able to do log/exp/sigmoid/etc. to 1 ulp so there really isn't much of a numerical issue. In fact, usually it's the opposite: if your parameters can range between 4 orders of magnitude, then taking a log transform can decrease that range and make the gradients stay numerically closer to each other which helps with controlling round-off error.

For this reason, many libraries are actually doing transformations behind the scenes even if users haven't explicitly stated it. For example, if you declare a Stan variable to be positive, it is holding the log and exponentiating as necessary so that the HMC can utilize $p \in \mathbb{R}$ which but still acts strictly positive (in this case it's quite necessary since small numerical errors could overshoot the parameter into the negative regime if this wasn't done!)