# Interpolation vs. Neural network

I am seeking knowledge from the community. I am solving a transport PDE (conservation of solute mass) using COMSOL. At each Newton-Raphson iteration, I need to update a constant called $$Kd$$ for some of the chemical species that are being transported (maybe I will update the $$Kd$$ at each time step depends on the convergence).

This Kd is obtained from 5 parameters. Hence, $$Kd = f(X_1, X_2, X_3, X_4, X_5)$$. I have a multidimensional array (or table, or whatever you prefer to called it), where from different combinations of $$X_1$$, $$X_2$$, $$X_3$$, $$X_4$$, and $$X_5$$ I obtain a $$Kd$$. The number of combinations is $$61440$$, hence the size of $$X_1$$, $$X_2$$, $$X_3$$, $$X_4$$, $$X_5$$ and $$Kd$$ is also $$61440$$ (Not sure if that is considered a big amount or no, for me it is).

At some point in the simulation, I will have to interpolate, I have tried in Matlab by using the griddatan function, but it took too much time and I got NaN as an answer. So, I am looking for alternatives. I have read that it should be possible to do a "prediction" (what for me would be an interpolation) using neuronal networks, but I wonder:

1) What are the algorithms in multivariate interpolation and neural networks field? (I am not too familiar with these fields)

2) Which one could be better or faster?

3) Where would be the difference in my case of using interpolation algorithms or neural network algorithms? (I do not have noise, although I have a big data).

Note: The grids of the combination $$X_1$$, $$X_2$$, $$X_3$$, $$X_4$$, $$X_5$$ are supposed to be equidistant.

• 61440 does not seem like a lot. How long does one evaluation of $f$ take? – Anton Menshov Jun 26 '19 at 20:48
• So, if I try using the built-in function of Matlab "griddatan" more than 5 min. If I tried to make a grid for Matlab (since the points are supposed to be equidistant) using the ndgrid function, I got a comment that says: "Requested 61440x61440x61440x61440x61440 (17179869184.0GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information." So I have ended up using the class for neural network, and that is ok, it is fast. – Daniel Jun 28 '19 at 11:07
• Oh, you have 61440 in every dimension! Somehow I read it differently. – Anton Menshov Jun 28 '19 at 15:35
• Maybe I do not explain myself good enough. Anyway, if you know what is better to use, interpolation or neural network, advantages and disadvantages. I would be grateful. Thanks. – Daniel Jul 1 '19 at 7:25
• Does the function behave smoothly along each $X_i$'s? (or do we expect it to behave smoothly) – Anton Menshov Jul 1 '19 at 8:25

I have limited experience in machine learning; however, simplifying, you can think of it as a "trained black box". I would say, you have to know a lot about your problem, the behaviour of your functions in order to successfully and reliably apply any form of machine learning. You would have to decide on how to get the training data, size of the training set, training methodology, your neural network structure (if you choose neural networks), and many other things. So, I would refrain from going via machine-learning route for numerical application when you have just started. It might be a way, but other numerical techniques should be tried first.

Regarding interpolation. You mentioned that your function might not be smooth (I guess, there is no analytic form we can look at). It would be worth a try to use interpolation on a much coarser grid that is tangible to fit into the memory and use, say, bicubic interpolation to start. Then, sample your error to understand what is going on. Choosing different interpolation techniques might also help.

Another choice would be rational function fitting. This might capture the behaviour of a more complicated function better.

I would say, you need to learn the behaviour of your function ($$Kd$$) better to understand what to expect and then, maybe, try machine learning. However, machine learning in numerical applications rarely gives you high accuracy (there are always exceptions and specialized problems), so if you are going for a lot of digits in your $$Kd$$, I doubt machine-learning is feasible.

• I think this is a pretty good first answer, even with your lack of a ML background. I may add a little more in separate answer to this. – spektr Jul 3 '19 at 18:25

So reading over your problem, I will share some comments and then my perspective with some overlap with the good answer written by Anton.

First, your question is phrased in a confusing manner. You initially make it sound like there's $$61440$$ combinations of $$(X_1, X_2, X_3, X_4, X_5)$$ possible in your grid discretization, but then your code issue mentioned in the comments with the ngrid function implies each variable $$X_i$$ has $$61440$$ values, making the total combinations of $$(X_1, X_2, X_3, X_4, X_5)$$ equal to $$61440^5$$, which is way too much to fit into RAM. Which is the correct situation?

If you do indeed have $$61440^5$$ discretized combinations, then I would first suggest you reduce the number of samples along each dimension so you can fit this model into RAM. Assuming this coarsened model is accurate enough, this model will be fast to compute.

You can always use some neural network to build a model based on samples of your data, as a neural network is a universal approximator, but training such a model on even a subset of the $$61440^5$$ dataset would ultimately be a challenge. The approximation errors would likely be greater than that of the table-based interpolation model.

Anton mentions a variety of ways you might try to come up with a better model. I would add that you really need to think about what you are trying to model. Ultimately, you want the technique you use to be able to adequately approximate the function space that the true function for $$Kd$$ lives. If you know $$Kd$$ is very un-smooth, you're likely going to struggle building a good model using smooth function approximation/interpolation techniques. If you have more details about $$Kd$$, you should describe them.