# Parameters estimation with fewer variables than parameters

I am trying to estimate parameters, 4 of them, by fitting a system of 3 ordinary differential equations. I am using a model published that was using 3 parameters and gave a good fit to the data, and I am trying to add a parameter to this (without changing the system of ODE, there is just a rate that is used twice, and I decided to make it two parameters instead).

More precisely, the system in question is: $$\left\{\begin{array} yy_1' = -\alpha(k_1+k_2)y_1 + \alpha k_2y_2\\ y_2' = \beta k_1y_1 - (\beta k_2 + k_4)y_3\\ y_3' = \gamma k_4y_2 - \delta y_3 \end{array} \right.$$ In the original model, $$k_1=k_2$$, but I wanted to try with different values.

Since with this change, I have now more parameters to fit (the $$k_i, \ i=1,...,4$$) than I have dependent variables ($$y_i,\ i = 1,...,3$$), I was wondering if it could have been the reason why I could not get a proper fit using nonlinear least squares, in the same way the authors had done.

I was trying to minimize the sum of the squared errors, namely: $$SSE(\theta) = \frac{1}{3N}\sum_{i=1}^3\sum_{j=0}^{N} (y_i(t_j,\theta)-\bar y_i(t_j))^2$$ with $$\theta=(k_1,k_2,k_3,k_4)$$ the parameters and $$\bar y_i(t_j)$$ the observations.

Turned out it was just a problem of the nonlinear least square problem being to hard for the algorithms and that using the log of the model and observations made it better conditioned. Namely, the problem of minimizing: $$SSE_{log}(\theta) = \frac{1}{3N}\sum_{i=1}^3\sum_{j=0}^{N} \Bigl(\ln\bigl(y_i(t_j,\theta)\bigr)-\ln\bigl(\bar y_i(t_j)\bigr)\Bigr)^2$$ was a much simpler task and I obtained a satisfying fit to my data.

• I think that a system of 3 linear ODE (of first order) can have more than 4 free parameters. Aug 28 at 17:37
• Could you write the system of equations that you're fitting? Aug 28 at 18:11
• The number of parameters should match the number of constraints which is very different from the number of ODEs in this problem. For example, if those three ODEs define a trajectory in 3D space - x(t), y(t), z(t) - and this trajectory should pass close to a given point r0=(x0,y0,z0) that is one constraint only. If the trajectory should also pass close to another point r1=(x1,y1,z1) that's your second constraint. Or it can be a cumulative constraint, like passing through a cloud of points $r_k$. But either way, the number of those ODEs is unrelated to the number of constraints. Aug 28 at 22:32
• The question is not so much how many ODEs you have, but how many observations you have. If you have observed the solution of the ODE at 1000 points in time, then you have 3000 data points and that should be enough to identify 4 parameters, assuming the problem is well posed. Aug 28 at 22:46
• Nonlinear least squares is often a very nonlinear optimization problem and it might be difficult to find a solution. I would start with the three parameter problem and fit those parameters. Then you make that fourth parameter independent from its "twin" and you start the optimization routine again with four parameters, but using the three-parameter solution as a starting guess. Aug 28 at 22:48