Computational Science Stack Exchange is a question and answer site for scientists using computers to solve scientific problems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Hill climbing seems to be a very powerful tool for optimization. However, how to generate the "neighbors" of a solution always puzzles me.

For example, I am optimizing a solution $(x_1, x_2, x_3)$. Here $x_1$ is in range $(0, 0.1)$, $x_2$ is in range $(0, 100)$, $x_3$ is in range $(0, 1000000)$. What's the best way to generate "neighbors"? I can't really pick a "step size" here, because a step size of 1 is huge to $x_1$, but very minor to $x_3$.

What is the best generic way to generate "neighbors" in hill climbing algorithms?

share|improve this question

migrated from May 3 '12 at 13:55

This question came from our site for people interested in statistics, machine learning, data analysis, data mining, and data visualization.

Vincent's solution will get you over the first hurdle. You might still be wondering how to select the step size even if all the dimensions have the same range. In that case it helps to understand the behavior of your function so you can select an appropriate value. Another way is to adaptively set the step size based on gradient (tread lightly if the function is changing a lot).

This paper might have some interesting ideas too: On a Hill-Climbing Algorithm with Adaptive Step Size: Towards a Control Parameter-Less Black-Box Optimisation Algorithm

share|improve this answer

The easiest (and safest) is to reparametrize your problem so that all parameters have a comparable range, say [0,1].

Alternatively, you can use a different step size for each parameter.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.