If we want to solve nonlinear minimization problem
$$\min_{x} f(x),$$
making least-squares assumption and using Gauss-Newton method so that at k$th$ iteration we have:
$$J_k^T J_k p_k = - J_k^T r_k,$$
with vector $r$ being residual and matrix $J$ Jacobian.
We then update $x$ in the following way:
$$x_{k+1} = x_k + \alpha p_k,$$
where $\alpha \in (0, 1]$
The question is how to find $\alpha$ such that $f(x_{k+1}) = min$
In other words, what step length calculating strategy would be good enough taking into account $f$ and $f'$ are expensive to compute and highly nonlinear?
I'm aware of methods that approximate this problem with polynomial, e.g. in case of quadratic approximation:
$$p_0 + p_1 \alpha + p_2 \alpha^2 = min$$
where $p_0 = f(x_k), p_1 = f'(x_k), p_2 = f(x_k + \alpha p_k)$
But I'm wondering what are the other options to try? Can somebody point me to a good overview or shortly write down different techniques.
