Skip to main content
Share Your Experience: Take the 2024 Developer Survey
5 votes
Accepted

Adaptive computation in neural ODEs

Neural networks generally have a fixed amount of computation which is generally referred to as the number of layers. A recurrent neural network applies some neural network ...
Chris Rackauckas's user avatar
4 votes
Accepted

How to implement the following operation in pytorch (tensor by equating indices)

In your case, it might be convenient to perform the entire calculation with Einstein summation: torch.einsum('ijl,il->ij', x, d) This then directly computes the ...
Julian Roth's user avatar
3 votes
Accepted

Problems solving 2D heat equation using physics-informed neural networks

The one thing that i can notice immediately where it might be going wrong is the use of ReLU Activation function for PINNs. In this problem,we have a double derivative in our loss function (Laplacian)....
ThivinAnandh's user avatar
3 votes

Automatic Differentiation using foward mode on matrices

When you train a neural network, you are optimizing the function whose inputs are the weights $W$ in each layer, and whose output is the total approximation error $$ \sum_{(x,y) \in \text{training ...
Federico Poloni's user avatar
3 votes

Adaptive computation in neural ODEs

I'm not an expert on neural networks, so don't take what I say here uncritically. My impression from skimming the paper is that they're replacing a set of discrete layers with a continuous pseudo-time ...
Daniel Shapero's user avatar
3 votes

Why are weight changes in Oja's rule and BCM so different?

No, both look exactly like they should. BCM is to be competitive between the input signals, thus finds that the vertical axis is a better separator for the two points than the horizontal axis. A very ...
Lutz Lehmann's user avatar
  • 6,109
2 votes
Accepted

Why can't we just use machine learning to select which model to use for a given model?

You can. This is called an ensemble model. For example, a linear regression between the solutions of different predictive models is a way to take a weighted average of different models. Normally, the ...
Chris Rackauckas's user avatar
2 votes
Accepted

Training accuracy improves but test set accuracy remains the same

I think you experience the implications of overtraining. From this question and this paper: Since a neural network with a sufficient number of neurons in the hidden layer can exactly implement an ...
Anton Menshov's user avatar
  • 8,672
2 votes
Accepted

Can someone explain the equivalence between Oja's rule and PCA in a simple way?

The wikipedia article on Hebbsian learning with one node is good. You should be able to grasp the idea of "burning in" the weights at connections of the same polarity and "erasing" ...
Lutz Lehmann's user avatar
  • 6,109
2 votes

How to implement the following operation in pytorch (tensor by equating indices)

I don't have PyTorch installed to check, but from the documentation it seems that torch.diagonal(z, dim1=0, dim2=2) should do the trick.
Vladimir Lysikov's user avatar
2 votes

How to find armijo step length for a neural network?

When computing the step length, you don't "update" the weights of the network. You just "test" whether certain updated weights would yield a certain result (that is, you are ...
Wolfgang Bangerth's user avatar
1 vote

Physics informed neural network package for hyperparameter tuning

Here are some PINN implementations/packages that might be useful for you: NVIDIA Modulus (previously: SimNet) DeepXDE from Lu Lu, George Em Karniadakis et al. JAX-PI from Sifan Wang, Paris Perdikaris ...
Julian Roth's user avatar
1 vote

How to implement the following operation in pytorch (tensor by equating indices)

What I did at the end was something like z = torch.tensordot(a,b,dim=[[2],[1]]) z = z[1:M,:,1:M] where M is the common ...
user8469759's user avatar
1 vote

Neural Network for Couette Flow

From your description, your problem simplifies to a 1D Poisson problem which you are trying to solve with PINNs and imho this should also be possible with PINNs. For $\frac{\partial p}{\partial y} = 1$...
Julian Roth's user avatar
1 vote

"Don't take the derivative of the approximation but approximate the derivative"..or something like this

It is heuristics. There are 2 or 3 ways to look at it & we will get the necessary Conclusion. (1) Differentiation is Exact & "undoable" ( within known limits ) & it is not losing ...
Prem's user avatar
  • 141
1 vote

how to overcome division by zero in the Hodgkin Huxley neuron model

You have AlphaNa = 0.1 * f(0.1*(VoltageDelta-10.0)) where $$ f(x)=\frac{x}{1-\exp(-x)} $$ The limit for $x\to 0$ can actually be interpreted as difference quotient,...
Lutz Lehmann's user avatar
  • 6,109
1 vote
Accepted

Is there a way to return a substring of a string using Convolutional Neural Networks?

I'll start with a disclaimer, my PhD is in the fast computation of eigenvalues, my specialty is not in machine learning at all. This is just some stuff I remember from some master level courses. I ...
Thijs Steel's user avatar
  • 1,723
1 vote
Accepted

Artificial neural networks for Temperature prediction

NN activation functions don't need to be between 0 and 1. That's only done for classification problems. Many times you want them continuously differentiable and monotonic, though that isn't even ...
Chris Rackauckas's user avatar
1 vote

Anyone who knows fine neural network code or module for python?

A very flexible Python library you could use to build your network and perform the various Machine Learning operations is Tensorflow. The premise of this library is you construct graphs that you can ...
spektr's user avatar
  • 4,248
1 vote

Power of complex-valued neural network

I also found that book written by Akira Hirose in the library. I have the same question as well, and I found this paper: https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2011-42.pdf Anyway, we ...
Atri Robot's user avatar
1 vote

Power of complex-valued neural network

I don't really see how a complex valued Neural Network would provide anything particularly useful over a real valued Neural Network. The whole idea of having a Neural Network that operates on ...
spektr's user avatar
  • 4,248

Only top scored, non community-wiki answers of a minimum length are eligible