I was looking at the Cheney and Kincaid book (6th edition) on numerical methods, with respect to collocation method for differential equations. Now for linear systems of ODES, collocation is just a linear solve. But for nonlinear systems of equations there is a residual function that we minimize.

The residual function they show in the Cheney and Kincaid book is p. 619 and shows.

$$ \mathcal{L} = \sum_{j=0}^{J}c_jv_j - b $$

This seems a bit odd, since I was expect something like the usual squared loss instead of just the sum of the differences. With this formulation, the positive errors can cancel out the negative errors.

Could someone just confirm that this is indeed the correct loss or residual function to use, or let me know if there a different residual function that is more appropriate. I checked a few other books. Most of them talk about using collocation on a linear system, which like I said is just a linear solve.

  • $\begingroup$ There are several editions of different books by Cheney and Kincaid. It would help to identify which book and edition you're referring to. $\endgroup$ Aug 18, 2021 at 4:24
  • $\begingroup$ @BrianBorchers sorry about that. I am looking at the 6th version of the book. I am also looking at Toby Driscoll's book Fundamentals of Numerical Analysis, and he has Julia code on Github github.com/fncbook/fnc/blob/master/src/chapter10.jl . If you look at line 165, it shows this same function basically just the sum of the differences. If that is correct that is fine, but I did not really see an explanation of why this loss is used versus squared loss, etc. $\endgroup$
    – krishnab
    Aug 18, 2021 at 4:55
  • $\begingroup$ There is a big difference between practice and theory. In practice to solve a system of nonlinear ODEs we mostly used colocation with Newton's iterative method. But sometimes the minimization is also appropriate, and in this case we use some norm defined for a system of algebraic equations. In some cases we can use minimization as first step for Newton's iterative method. We can also compose colocation with wavelets to optimize number of colocation points. $\endgroup$ Aug 21, 2021 at 0:59


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.