I have an optimization problem where i need to find an image x, that is very close to x' such that:
monitor(x') is valid but monitor(x) is invalid. (output is valid when the neural network output is inside bounds of specific box).
x is same classified as x' by a neural network(their prediction class is the same).
Note that x is an image with (1,28,28,1) dimension and its transformed to an array with dim 784 in order to pass into a scipy solver.
The optimization problem is represented as follows:
Obj function: minimize L2 norm of (x'- x) with constraints:
prediction( transformed image of x) = prediction( transformed image of x')
monitor(x) = 0 (0 for invalid)
x is the optimization variable to be found and x' is the valid input passed
So I am using scipy to solve my problem, I tried local optimization (COBYLA and SLSQP) and global optimization(Sgho and differential evolution). I think my problem is non linear and non convex according to my constraints. But some resources pointed out that non convex constraints don't necessarily mean non convex problem.
"Depending on the problem, you may also try to solve it with a solver which give you just a local optimum if the problem is non convex (as ipopt, or bonmin if you have integer variables) and then compare the solution with that of a global MINLP solver as Couenne/BARON/SCIP. If the results are very different the problem is probably non convex. It could also be that some solver gives you as output the information about the convexity of the problem, but I am not sure about that."
Actually, local and global optimization gave me different results, and sgho was performing the best out of all. However all aren't always converging to successful solutions, but sgho performs best. I think this is because of my non linear and non convex constraints.
Is my choice for Sgho and differential evolution good? and so my problem is considered non convex right?