Interpolation vs. Neural network

Question

I am seeking knowledge from the community. I am solving a transport PDE (conservation of solute mass) using COMSOL. At each Newton-Raphson iteration, I need to update a constant called $Kd$ for some of the chemical species that are being transported (maybe I will update the $Kd$ at each time step depends on the convergence).

This Kd is obtained from 5 parameters. Hence, $Kd = f(X_1, X_2, X_3, X_4, X_5)$. I have a multidimensional array (or table, or whatever you prefer to called it), where from different combinations of $X_1$, $X_2$, $X_3$, $X_4$, and $X_5$ I obtain a $Kd$. The number of combinations is $61440$, hence the size of $X_1$, $X_2$, $X_3$, $X_4$, $X_5$ and $Kd$ is also $61440$ (Not sure if that is considered a big amount or no, for me it is).

At some point in the simulation, I will have to interpolate, I have tried in Matlab by using the griddatan function, but it took too much time and I got NaN as an answer. So, I am looking for alternatives. I have read that it should be possible to do a "prediction" (what for me would be an interpolation) using neuronal networks, but I wonder:

1) What are the algorithms in multivariate interpolation and neural networks field? (I am not too familiar with these fields)

2) Which one could be better or faster?

3) Where would be the difference in my case of using interpolation algorithms or neural network algorithms? (I do not have noise, although I have a big data).

Note: The grids of the combination $X_1$, $X_2$, $X_3$, $X_4$, $X_5$ are supposed to be equidistant.

Answer 1

I have limited experience in machine learning; however, simplifying, you can think of it as a "trained black box". I would say, you have to know a lot about your problem, the behaviour of your functions in order to successfully and reliably apply any form of machine learning. You would have to decide on how to get the training data, size of the training set, training methodology, your neural network structure (if you choose neural networks), and many other things. So, I would refrain from going via machine-learning route for numerical application when you have just started. It might be a way, but other numerical techniques should be tried first.

Regarding interpolation. You mentioned that your function might not be smooth (I guess, there is no analytic form we can look at). It would be worth a try to use interpolation on a much coarser grid that is tangible to fit into the memory and use, say, bicubic interpolation to start. Then, sample your error to understand what is going on. Choosing different interpolation techniques might also help.

Another choice would be rational function fitting. This might capture the behaviour of a more complicated function better.

I would say, you need to learn the behaviour of your function ($Kd$) better to understand what to expect and then, maybe, try machine learning. However, machine learning in numerical applications rarely gives you high accuracy (there are always exceptions and specialized problems), so if you are going for a lot of digits in your $Kd$, I doubt machine-learning is feasible.

Answer 2

So reading over your problem, I will share some comments and then my perspective with some overlap with the good answer written by Anton.

First, your question is phrased in a confusing manner. You initially make it sound like there's $61440$ combinations of $(X_1, X_2, X_3, X_4, X_5)$ possible in your grid discretization, but then your code issue mentioned in the comments with the ngrid function implies each variable $X_i$ has $61440$ values, making the total combinations of $(X_1, X_2, X_3, X_4, X_5)$ equal to $61440^5$, which is way too much to fit into RAM. Which is the correct situation?

If you do indeed have $61440^5$ discretized combinations, then I would first suggest you reduce the number of samples along each dimension so you can fit this model into RAM. Assuming this coarsened model is accurate enough, this model will be fast to compute.

You can always use some neural network to build a model based on samples of your data, as a neural network is a universal approximator, but training such a model on even a subset of the $61440^5$ dataset would ultimately be a challenge. The approximation errors would likely be greater than that of the table-based interpolation model.

Anton mentions a variety of ways you might try to come up with a better model. I would add that you really need to think about what you are trying to model. Ultimately, you want the technique you use to be able to adequately approximate the function space that the true function for $Kd$ lives. If you know $Kd$ is very un-smooth, you're likely going to struggle building a good model using smooth function approximation/interpolation techniques. If you have more details about $Kd$, you should describe them.