I have a system of 10 ordinary differential equations of the form,

$$\frac{dy_1}{dt} = f1(V1,k1,y1,y2)\\ \vdots \\ \frac{dy_{10}}{dt} = f_{10}(V_{10},k_{10},y_{9},y_{10}) $$

I want to estimate the parameters $V_1 ,V_6,V_7,V_{10}$ using a global fit.

From the approaches suggested in the literature, I understand least-squares error minimization is commonly used. However, I'm not able to understand how optimization problem is actually formulated.

Cost function = $\Sigma_{i=1}^{10} (y_i^\text{experiment} - y_i^\text{model})^2$

Where, $y_i^\text{experiment}$ is the steady-state value obtained from experiments and not the time series data of $y_i$. Could someone explain how $y_i^\text{model}$ is expressed in terms of the parameters that are to be estimated?

Is is differential equation,(say) $\frac{dy_1}{dt} = f1(V1,k1,y1,y2)$

expanded using Taylor polynomial to find $y_1^\text{model}$?

Could someone provide an example?

  • 1
    $\begingroup$ Typically it's necessary to use a numerical method (e.g. a Runge-Kutta method) to solve the ODE initial value problem. Derivatives of the solution with respect to the parameters are then computed by finite difference approximations. $\endgroup$ Apr 7, 2019 at 17:41
  • $\begingroup$ In what form do you have the system of equations? Do you have dy_n/dt at several times for some initial conditions? Do you have y_n(t)? $\endgroup$ Apr 7, 2019 at 20:57
  • $\begingroup$ @BrianBorchers Thanks a lot for your response. I'm using stiff- equation solver.I'm facing challenge in understanding how the optimization problem is solved to estimate the unknown parameters $\endgroup$
    – Natasha
    Apr 8, 2019 at 3:10
  • $\begingroup$ @JacobPanikulam I have multiple steady- state values of $y_1$ to $y_{10}$ obtained from experiments $\endgroup$
    – Natasha
    Apr 8, 2019 at 3:13
  • 1
    $\begingroup$ You might find following link useful. The way you state the problem is a classical optimization problem subject to a ODE as constraint. There are two ways of solving this, the forward approach Brian Borchers mentioned or the adjoint method linked here. $\endgroup$
    – Bort
    Apr 10, 2019 at 11:15

1 Answer 1


You can think of the ODE as a constraint between the parameters $V$ and the observed variable $y$. To enforce constraints in optimization problems, you can introduce a Lagrange multiplier, which we'll call $\lambda$. In that case, the Lagrangian for your problem is

$$L(y, \lambda, V) = \sum_i|y(t_i) - y_i^\text{experiment}|^2 + \int_0^T\lambda\cdot (\dot y - f(y, V))dt.$$

You'll then seek an extremum of the Lagrangian. A good understanding of variational calculus is going to be really essential from here on out. If that subject is new to you, Weinstock's book is a great introduction. The next thing you'll want to know about is the adjoint method. Unfortunately a lot of books and references that describe the adjoint method are very confusing, but it's been discussed on this forum a handful of times, for example here. Another term you might want to google is "system identification", but the wikipedia article for it isn't very informative so you may have to do some more digging.

There's a really nice Python library for numerical fitting from symbolic definitions of the models, called symfit. They have an example of ODE models here.

  • $\begingroup$ Excuse me for the naive question, this subject is new to me.I'll definitely read up the references suggested. Is the adjoint method applicable for ODE's too? I saw the descriptions were based on PDE's. $\endgroup$
    – Natasha
    Apr 24, 2019 at 6:27
  • 2
    $\begingroup$ No worries, everyone's naive at first! The adjoint method is applicable to ODEs as well as PDEs. It's really even more general than that -- differential operators and their inverses are just mappings between Banach spaces -- but this is a deeper rabbit hole. $\endgroup$ Apr 24, 2019 at 15:44
  • 1
    $\begingroup$ as described by Daniel Shapero the right approach for these kind of problems is the "adjoint method", some gentle intriductions are: math.mit.edu/~stevenj/18.336/adjoint.pdf and cs.stanford.edu/~ambrad/adjoint_tutorial.pdf $\endgroup$ Apr 24, 2019 at 18:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.