# Conjugacy in Non-linear Conjugate Gradient Descent

In linear conjugate gradient method, our goal is to solve the system of linear equations $$Ax = b$$ where A is a symmetric positive definite matrix, and that is equivalent to finding the minimizer of the convex quadratic form : $$φ(x) = \frac12 x^T A x - x^T b$$

In the non linear CG method, our goal is finding the minimum of $$f(x)$$, a general continuous function for which the gradient $$f'$$ can be computed.

My question is, in linear CG, we say that the directions $$p_{k}$$ are conjugate with recpect to the matrix $$A$$ if $$p_{i}^TAp_{j} = 0$$ for all $$i\neq j$$, where $$A$$ is the matrix that appears in the system we would like to solve.

However, in nonlinear, such a matrix doesn't exist, does it? In that case, how do we define the conjugacy of our search directions or what do we mean by keep saying that the directions are conjugate?

Thanks in advance.

• welcome to scicomp stackexchange! the answer to your question is in sections 14.1 - 14.2 of Shewchuk's "An Introduction to the Conjugate Gradient Method Without the Agonizing Pain", available here – GoHokies Jan 25 '20 at 18:41
• Thanks, will check it out. – thenac Jan 25 '20 at 18:47