What is the difference (in terms of e.g robustness and speed) between proving a gradient obtained by an AD package (like PyAutoDiff) and let the solver (e.g BSGS) calculate the gradient ? It seems so tempting to do something like:
from autodiff import gradient def my_costfun(x): return f(x) @gradient def g(x): return f(x)
Is that really that simple ? What about functions with, lets say 10 parameters ?