0
$\begingroup$

Define a sequence $(\mathbf{y})_{i=0}^N$ in $\mathbb{R}^n$ such that: $$\mathbf{y}_{k+1} = \mathbf{y}_{k} + \lambda \nabla_\mathbf{y} E(\mathbf{y}_k,\mathbf{w}), \quad k=0,1,\ldots,N-1,$$ where $\lambda$ is a constant, $\mathbf{w}\in\mathbb{R}^m$, and $E:\mathbb{R}^{n+m}\to \mathbb{R}$ is some differentiable function.

Let $Q:\mathbb{R}^{n}\to \mathbb{R}$ be a differentiable function and $L=Q(\mathbf{y} _N)$.

Applying the chain rule we have: $$\frac{dL}{d\mathbf{w}} = \sum_{k=1}^N\frac{\partial \mathbf{y}_k}{\partial \mathbf{w}} \frac{dQ}{d\mathbf{y}_k}\qquad (1)$$ and $$\frac{dQ}{d\mathbf{y}_k} = \frac{\partial \mathbf{y}_{k+1}}{\partial \mathbf{y}_{k}} \frac{dQ}{d\mathbf{y}_{k+1}}.\qquad (2)$$

(Source: this paper, equation (12) and the one between (12) and (13).)

My questions: how to obtain $(1)$ and $(2)$?

I can show that if $(2)$ holds then $(1)$ holds. But I cannot see why $(2)$ holds. Consider $k=N-1$ for example:

Applying the chain rule we have: \begin{align}\frac{dQ}{d\mathbf{y}_{N-1}} &= \frac{dQ(\mathbf{y}_{N}(\mathbf{w},\mathbf{y}_{N-1}))}{d\mathbf{y}_{N-1}} \\ &= \frac{d\mathbf{y}_{N}(\mathbf{w},\mathbf{y}_{N-1})}{d\mathbf{y}_{N-1}}\frac{dQ}{d\mathbf{y}_{N}} \\ &= \begin{bmatrix}\frac{d\mathbf{w}}{d\mathbf{y}_{N-1}} & \mathbf{I}\end{bmatrix}\begin{bmatrix}\frac{\partial \mathbf{y}_{N}}{\partial \mathbf{w}} \\ \frac{\partial \mathbf{y}_{N}}{\partial \mathbf{y}_{N-1}}\end{bmatrix}\frac{dQ}{d\mathbf{y}_{N}} \\ &= \frac{d\mathbf{w}}{d\mathbf{y}_{N-1}}\frac{\partial \mathbf{y}_{N}}{\partial \mathbf{w}}\frac{dQ}{d\mathbf{y}_{N}} + \frac{\partial \mathbf{y}_{N}}{\partial \mathbf{y}_{N-1}}\frac{dQ}{d\mathbf{y}_{N}}, \end{align} which has an extra term $\frac{d\mathbf{w}}{d\mathbf{y}_{N-1}}\frac{\partial \mathbf{y}_{N}}{\partial \mathbf{w}}\frac{dQ}{d\mathbf{y}_{N}}$ compared to $(2)$.

Thank you in advance for your help.

$\endgroup$
  • $\begingroup$ I think the notation is a little confusing here, because some of these derivatives are partial derivatives, keeping $w$ constant, and others are total derivatives. I suspect that's where you went wrong. For example, $w$ doesn't explicitly depend on $y_{N-1}$, so why do you get the term $dw/dy_{N-1}$? Doesn't that look strange? Try a scalar case first, or maybe apply the chain rule in the forward direction instead. This is really a pure mathematics question, since there isn't anything specific to computational science, so it might be a little off-topic here. $\endgroup$ – Kirill Jun 25 '17 at 23:38
  • $\begingroup$ @Kirill Thanks. $dw/dy_{N-1}$ comes from applying the chain rule as I showed above (and yes in the above equations, $d$ is for total derivative and $\partial$ is for partial derivatives). This is an active community and thus I posted this question here with the hope of having someone good at maths to help me :D $\endgroup$ – Khue Jun 26 '17 at 9:07
1
$\begingroup$

The idea is that each $\vec{y}_k$ depends on $\vec{w}$. Therefore for the first equation, simply apply the chain rule for each $\vec{y}_k$ and sum them up. Componentwise one has: $$\left(\frac{dL}{d\vec{w}}\right)_{i}=\sum_{k,\alpha}{\left(\frac{dQ}{d\vec{y}_k}\right)_{\alpha}\left(\frac{\partial \vec{y}_k}{\partial \vec{w}}\right)_{i\alpha}} \tag{*}$$ Maybe the order in which they appear multiplying each other has confused you.

The second is obtained supposing that the function $\vec{y}_{k}$ can be put as a function of $\vec{y}_{k+1}$ (your first equation suggets this). Therefore the derivative in $(*)$ can be expressed as: $$\frac{dQ}{d\vec{y}_k}=\frac{\partial \vec{y}_{k+1}}{\partial \vec{y}_k}\frac{dQ}{d\vec{y}_{k+1}}$$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I think you have reversed the positions of $d$ and $\partial$. Let me recall the chain rule for a vector-valued function $\mathbf{f}$ of the form $\mathbf{f}(\mathbf{x}_1(\mathbf{t}),\mathbf{x}_2(\mathbf{t}),\ldots,\mathbf{x}_n(\mathbf{t}))$: \begin{equation} \boxed{\frac{d\mathbf{f}}{d\mathbf{t}} = \sum_{i=1}^n \frac{d\mathbf{x}_i}{d\mathbf{t}}\frac{\partial \mathbf{f}}{\partial \mathbf{x}_i}}. \end{equation} Applying this we have $$\frac{dL}{dw} = \sum_{k=1}^N \frac{dy_k}{dw}\frac{\partial Q}{\partial y_k}.$$ However,... (continued next)... $\endgroup$ – Khue Jun 26 '17 at 22:28
  • $\begingroup$ However, this is trivial because the left-hand side is equal to $\frac{dy_N}{dw}\frac{\partial Q}{\partial y_N}$ (because $\frac{\partial Q}{\partial y_k} = 0 \forall k\neq N$), which is $\frac{dy_N}{dw}\frac{d Q}{d y_N} = \frac{dQ}{dw}$. $\endgroup$ – Khue Jun 26 '17 at 22:29
  • $\begingroup$ As far as I see: $Q$ is a function only of $\vec{y}_k$ for an arbitrary $k$ i.e. $Q=Q(\vec{y}_k)$. The vector $\vec{y}_{k}$ is a function of $\vec{y}_{k-1}$ and $\vec{w}$. Therefore applying the chain rule we have: $$\frac{dQ(\vec{y}_k(\vec{y}_{k-1},\vec{w}))}{d\vec{w}}=\left(\frac{\partial \vec{y}_k}{\partial \vec{w}}\right)^{T}\frac{dQ}{d\vec{y}_k}$$ $\endgroup$ – HBR Jun 27 '17 at 18:38
  • $\begingroup$ More precisely $Q$ is a function of $y_N$: $Q=Q(y_N)= Q(y_N(w,y_{N-1}))$. Which chain rule did you apply to get $\frac{dQ(y_k(w,y_{k-1}))}{dw} = \frac{\partial y_k}{\partial w} \frac{dQ}{dy_k}$? (convention: vectors are transposed accordingly) Applying the chain rule, it should be $\frac{dQ(y_k(w,y_{k-1}))}{dw} = \frac{\partial y_k}{\partial w} \frac{dQ}{dy_k} + \frac{dy_{k-1}}{dw}\frac{\partial y_k}{\partial y_{k-1}}\frac{dQ}{dy_k}$. I think the authors made some mistakes in applying the chain rule. $\endgroup$ – Khue Jun 28 '17 at 12:24
  • $\begingroup$ See also my answer here: math.stackexchange.com/questions/2335740/… $\endgroup$ – Khue Jun 28 '17 at 12:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.