0
$\begingroup$

I'm trying to implement a simple Neural Network for Couette Flow. I'm working with a Fully Connected Neural Network. I'm finding that the convergence of the Neural Network is highly dependent on the problem parameters and the initialization of the weights. I'm minimizing the physics based residual, which which the derivatives are approximated by central finite differences.

$$ L = \Big(-\frac{\partial p}{\partial y} + \mu\frac{\partial^2{u_\theta(x)}}{\partial{x^2}}\Big)^2 $$

where $u_\theta(x)$ is the output of the fully connected Neural Network. $\theta$ are the weights of the neural network. $\frac{\partial p}{\partial y}$ is a user specified (constant) pressure gradient. The walls are fixed, so $u_\theta(0)=u_\theta(L)=0$. The loss function is implemented via MSE (Mean Squared Error).

The fully connected Neural Network seems to drive the $\frac{\partial^2{u_\theta(x)}}{\partial{x^2}}$ term to zero, leading to the loss being stuck in a local minimum at $\Big(\frac{\partial p}{\partial y}\Big)^2$. I've tried different batch sizes, and different initialization strategies and different activations - but I don't escape this local minimum.

Imposing boundary conditions is also tricky - the weight for the boundary condition loss has to be just perfect. I'm aware of the work of Wang, Teng and Perdikaris, in which they show how to weight losses and discuss a new architecture and I will investigate it. I've tried imposing hard boundary conditions as in Lu Lu et. al. but I don't escape the local minimum.

My question is: How do I escape from the local minimum - if indeed it is a local minimum? Are there any other architectures that I should investigate for this problem?

$\endgroup$
2
  • 2
    $\begingroup$ What are your boundary conditions? What is (the expression for) $p$ in your problem? $\endgroup$ Commented Feb 9 at 10:47
  • $\begingroup$ @JulianRoth added the information you requested. $\endgroup$
    – NNN
    Commented Feb 9 at 11:01

1 Answer 1

1
$\begingroup$

From your description, your problem simplifies to a 1D Poisson problem which you are trying to solve with PINNs and imho this should also be possible with PINNs. For $\frac{\partial p}{\partial y} = 1$ and soft boundary conditions you can find a PINN implementation in our lecture notes on neural networks on slides 353 to 358. I think this should work for your problem as well.

If not, you can enforce the boundary conditions by $$ u_\theta(x) = x(L-x) \cdot \text{NN}_\theta(x) $$ which automatically satisfies the Dirichlet boundary conditions. Then, you only need the MSE of the PDE residual as your loss.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.