I am trying to optimise a function, but the function can be noisy and give varying results for the same parameters. Furthermore, it needs to be online, as the data from each new iteration happens slowly (and begins with no data).
I have used gradient descent, which works sometimes, but even then not very well due to noise. I thought there must be a more intelligent algorithm that uses the history of previous iterations in order to better estimate the new parameters being optimised, rather than relying only on the changes to the previous one.
Could someone please tell me a simple algorithm that I can use to do this? I've been trying to find robust online optimisation algorithms but haven't had much success. Thanks in advance.