# Derivative chain rule

Define a sequence $(\mathbf{y})_{i=0}^N$ in $\mathbb{R}^n$ such that: $$\mathbf{y}_{k+1} = \mathbf{y}_{k} + \lambda \nabla_\mathbf{y} E(\mathbf{y}_k,\mathbf{w}), \quad k=0,1,\ldots,N-1,$$ where $\lambda$ is a constant, $\mathbf{w}\in\mathbb{R}^m$, and $E:\mathbb{R}^{n+m}\to \mathbb{R}$ is some differentiable function.

Let $Q:\mathbb{R}^{n}\to \mathbb{R}$ be a differentiable function and $L=Q(\mathbf{y} _N)$.

Applying the chain rule we have: $$\frac{dL}{d\mathbf{w}} = \sum_{k=1}^N\frac{\partial \mathbf{y}_k}{\partial \mathbf{w}} \frac{dQ}{d\mathbf{y}_k}\qquad (1)$$ and $$\frac{dQ}{d\mathbf{y}_k} = \frac{\partial \mathbf{y}_{k+1}}{\partial \mathbf{y}_{k}} \frac{dQ}{d\mathbf{y}_{k+1}}.\qquad (2)$$

(Source: this paper, equation (12) and the one between (12) and (13).)

My questions: how to obtain $(1)$ and $(2)$?

I can show that if $(2)$ holds then $(1)$ holds. But I cannot see why $(2)$ holds. Consider $k=N-1$ for example:

Applying the chain rule we have: \begin{align}\frac{dQ}{d\mathbf{y}_{N-1}} &= \frac{dQ(\mathbf{y}_{N}(\mathbf{w},\mathbf{y}_{N-1}))}{d\mathbf{y}_{N-1}} \\ &= \frac{d\mathbf{y}_{N}(\mathbf{w},\mathbf{y}_{N-1})}{d\mathbf{y}_{N-1}}\frac{dQ}{d\mathbf{y}_{N}} \\ &= \begin{bmatrix}\frac{d\mathbf{w}}{d\mathbf{y}_{N-1}} & \mathbf{I}\end{bmatrix}\begin{bmatrix}\frac{\partial \mathbf{y}_{N}}{\partial \mathbf{w}} \\ \frac{\partial \mathbf{y}_{N}}{\partial \mathbf{y}_{N-1}}\end{bmatrix}\frac{dQ}{d\mathbf{y}_{N}} \\ &= \frac{d\mathbf{w}}{d\mathbf{y}_{N-1}}\frac{\partial \mathbf{y}_{N}}{\partial \mathbf{w}}\frac{dQ}{d\mathbf{y}_{N}} + \frac{\partial \mathbf{y}_{N}}{\partial \mathbf{y}_{N-1}}\frac{dQ}{d\mathbf{y}_{N}}, \end{align} which has an extra term $\frac{d\mathbf{w}}{d\mathbf{y}_{N-1}}\frac{\partial \mathbf{y}_{N}}{\partial \mathbf{w}}\frac{dQ}{d\mathbf{y}_{N}}$ compared to $(2)$.

• I think the notation is a little confusing here, because some of these derivatives are partial derivatives, keeping $w$ constant, and others are total derivatives. I suspect that's where you went wrong. For example, $w$ doesn't explicitly depend on $y_{N-1}$, so why do you get the term $dw/dy_{N-1}$? Doesn't that look strange? Try a scalar case first, or maybe apply the chain rule in the forward direction instead. This is really a pure mathematics question, since there isn't anything specific to computational science, so it might be a little off-topic here. Commented Jun 25, 2017 at 23:38
• @Kirill Thanks. $dw/dy_{N-1}$ comes from applying the chain rule as I showed above (and yes in the above equations, $d$ is for total derivative and $\partial$ is for partial derivatives). This is an active community and thus I posted this question here with the hope of having someone good at maths to help me :D
– f10w
Commented Jun 26, 2017 at 9:07

The idea is that each $\vec{y}_k$ depends on $\vec{w}$. Therefore for the first equation, simply apply the chain rule for each $\vec{y}_k$ and sum them up. Componentwise one has: $$\left(\frac{dL}{d\vec{w}}\right)_{i}=\sum_{k,\alpha}{\left(\frac{dQ}{d\vec{y}_k}\right)_{\alpha}\left(\frac{\partial \vec{y}_k}{\partial \vec{w}}\right)_{i\alpha}} \tag{*}$$ Maybe the order in which they appear multiplying each other has confused you.

The second is obtained supposing that the function $\vec{y}_{k}$ can be put as a function of $\vec{y}_{k+1}$ (your first equation suggets this). Therefore the derivative in $(*)$ can be expressed as: $$\frac{dQ}{d\vec{y}_k}=\frac{\partial \vec{y}_{k+1}}{\partial \vec{y}_k}\frac{dQ}{d\vec{y}_{k+1}}$$

• I think you have reversed the positions of $d$ and $\partial$. Let me recall the chain rule for a vector-valued function $\mathbf{f}$ of the form $\mathbf{f}(\mathbf{x}_1(\mathbf{t}),\mathbf{x}_2(\mathbf{t}),\ldots,\mathbf{x}_n(\mathbf{t}))$: $$\boxed{\frac{d\mathbf{f}}{d\mathbf{t}} = \sum_{i=1}^n \frac{d\mathbf{x}_i}{d\mathbf{t}}\frac{\partial \mathbf{f}}{\partial \mathbf{x}_i}}.$$ Applying this we have $$\frac{dL}{dw} = \sum_{k=1}^N \frac{dy_k}{dw}\frac{\partial Q}{\partial y_k}.$$ However,... (continued next)...
– f10w
Commented Jun 26, 2017 at 22:28
• However, this is trivial because the left-hand side is equal to $\frac{dy_N}{dw}\frac{\partial Q}{\partial y_N}$ (because $\frac{\partial Q}{\partial y_k} = 0 \forall k\neq N$), which is $\frac{dy_N}{dw}\frac{d Q}{d y_N} = \frac{dQ}{dw}$.
– f10w
Commented Jun 26, 2017 at 22:29
• As far as I see: $Q$ is a function only of $\vec{y}_k$ for an arbitrary $k$ i.e. $Q=Q(\vec{y}_k)$. The vector $\vec{y}_{k}$ is a function of $\vec{y}_{k-1}$ and $\vec{w}$. Therefore applying the chain rule we have: $$\frac{dQ(\vec{y}_k(\vec{y}_{k-1},\vec{w}))}{d\vec{w}}=\left(\frac{\partial \vec{y}_k}{\partial \vec{w}}\right)^{T}\frac{dQ}{d\vec{y}_k}$$
– HBR
Commented Jun 27, 2017 at 18:38
• More precisely $Q$ is a function of $y_N$: $Q=Q(y_N)= Q(y_N(w,y_{N-1}))$. Which chain rule did you apply to get $\frac{dQ(y_k(w,y_{k-1}))}{dw} = \frac{\partial y_k}{\partial w} \frac{dQ}{dy_k}$? (convention: vectors are transposed accordingly) Applying the chain rule, it should be $\frac{dQ(y_k(w,y_{k-1}))}{dw} = \frac{\partial y_k}{\partial w} \frac{dQ}{dy_k} + \frac{dy_{k-1}}{dw}\frac{\partial y_k}{\partial y_{k-1}}\frac{dQ}{dy_k}$. I think the authors made some mistakes in applying the chain rule.
– f10w
Commented Jun 28, 2017 at 12:24