I have to solve a set of nonlinear optimization problems in the subspace defined as the orthogonal space to a given vector.
More precisely, $$ \arg\min f(\vec x) \qquad \text{with} \qquad \vec x \cdot \vec n =0 $$
I am thinking of applying the nonlinear conjugate gradient method projecting the direction of descent but I am wondering in which step I should do this projection operation.
One idea is that the projection should take place only in the update of the position like $$ \vec d_k = - \vec \nabla f(\vec x_{k-1}) + \beta \vec d_{k-1} \\ \alpha_k = \arg \min f(\vec x_{k-1} + \alpha \vec d_k) \\ \vec x_{k} = \vec x_{k-1} + (\alpha_k \vec d_k) - [(\alpha_k \vec d_k)\cdot \vec n ]\vec n $$
The other idea is to apply projection directly to the gradient itself such that even the past directions are subjected to the constraint of living in the orthogonal subspace to $\vec n$
$$ \vec g_k = \vec \nabla f(\vec x_{k-1}) - [\vec \nabla f(\vec x_{k-1})\cdot \vec n]\vec n \\ \vec d_k = - \vec g_k + \beta \vec d_{k-1} \\ \alpha_k = \arg \min f(\vec x_{k-1} + \alpha \vec d_k) \\ \vec x_{k} = \vec x_{k-1} + (\alpha_k \vec d_k) $$
What is the best way to solve this problem?