I would like to implement a Numerical Optimizer in my favorite language Go. It shall find solution(s) to this problem: maximize a function f(x)
where f
is non-linear and x
is a real vector of dimension 10 or 20. f
is real-valued.
What is the best method in terms of:
- simplicity to implement
- opportunities to paralellize
The algorithm is supposed to run on a x64 CPU with multiple cores. Maybe one needs to look at different cases, so it would be okay for me to implement 2 or 3 algorithms.
Interesting cases:
f
is rational orf
is a composition of rational functions,exp
and maybelog
f
has multiple maximums
This is not intended as an open question. So I am not looking for the answer that lists as many algorithms as possible. Or an endless discussion of what algorithm might be the nicest. Choosing an algorithm can be somehow subjective, so I want to emphasize on a decent solution, rather than a perfect solution.
If have heard that simulated annealing is considered state-of-the-art when it comes to multiple maximums and a fixed domain. Although this sounds to me like: if I do not fix the domain, although I could fix it a priori, simulated annealing might be inferior to some other algorithm.
Update: suboptimal solutions are fine, however I want to make sure the algorithm doesn't concentrate to easily on a single maximum.
About the function: I want to do maximum likelihood fits on functions that estimate time series. Something like y_{n+1} = a_1 * y_n + a_2 + y_{n-1} + ... + b_1 * epsilon_n + ... + epsilon_n
in a simple case. (epsilon
is i.i.d. noise, y
some time series, a_i
and b_i
real parameters .) Currently I'm doing this by assuming epsilon
to be normal distributed but I want to change to the Cauchy-distribution. Moreover I want to add some non-linear terms and extend this from y real value to y being a real vector. Unfortunately I don't know yet exactly what I want, so the optimizer should not be too scoped.