I'm using PETSc's SNES linesearch (NEWTONLS) to solve a system of nonlinear PDEs. SNES+KSP successfully solved a test problem of the same type before without any trouble.
However, with my current problem, if I use the KSP to invert the Jacobian, the intermediate solution develops a "ridge" at the boundary between two MPI processes. The image below shows the solution after one SNES step running with -ksp_gmres_modifiedgramschmidt -pc_type asm -sub_pc_type lu -sub_pc_factor_shift_type NONZERO.
The problem seems to go away if I execute the exact same binary, but using the MUMPS direct solver with -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package mumps, as shown below.
In both cases above SNES decided to start by taking a full Newton step. However, in the KSP run the ridge continues to grow and the step size gets significantly suppressed over the next few steps before failing completely. In the MUMPS run, SNES converged straightforwardly after taking 5 full steps.
Does anyone have any idea what is happening here?
