You can. This is called an ensemble model. For example, a linear regression between the solutions of different predictive models is a way to take a weighted average of different models. Normally, the winners of all of the machine learning contests use ensemble models since they eek out a bit more accuracy. However, they are a lot more costly to train.
I think you experience the implications of overtraining. From this question and this paper:
Since a neural network with a sufficient number of neurons in the hidden layer can exactly implement an arbitrary training set, it can learn both investigated dependencies and a noise that will lower the predictive ability of the network.
So, in your case, you ...
AlphaNa = 0.1 * f(0.1*(VoltageDelta-10.0))
The limit for $x\to 0$ can actually be interpreted as difference quotient, you might also appeal to l'Hopital, to find that $f(x)\to 1$. So if you want to play it safe, you can employ a 3-tier or 4-tier implementation of this function
if abs(x)<1e-12: ...
I'll start with a disclaimer, my PhD is in the fast computation of eigenvalues, my specialty is not in machine learning at all. This is just some stuff I remember from some master level courses. I have two ideas that might work.
Traditional convolutional neural nets are very good at classifying. For example, "does this image contain a dog", ...
NN activation functions don't need to be between 0 and 1. That's only done for classification problems. Many times you want them continuously differentiable and monotonic, though that isn't even required. RELU activation functions which are $\sigma(x)=max(0,x)$ are quite common for these kinds of scenarios.
A very flexible Python library you could use to build your network and perform the various Machine Learning operations is Tensorflow. The premise of this library is you construct graphs that you can use to represent models, perform computation, and many other things. Using this framework, you can build arbitrary neural network models, amongst other things.
I also found that book written by Akira Hirose in the library. I have the same question as well, and I found this paper: https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2011-42.pdf
Anyway, we are not even in the "real" world. The input spaces of any neural networks implemented on modern computers are not just countable, but finite. Also the ...
I don't really see how a complex valued Neural Network would provide anything particularly useful over a real valued Neural Network.
The whole idea of having a Neural Network that operates on complex numbers, uses complex weights, and outputs complex numbers doesn't seem any different than having a real valued Neural Network that has two times as many ...