0
$\begingroup$

Define a sequence $(\mathbf{y})_{i=0}^N$ in $\mathbb{R}^n$ such that: $$\mathbf{y}_{k+1} = \mathbf{y}_{k} + \lambda \nabla_\mathbf{y} E(\mathbf{y}_k,\mathbf{w}), \quad k=0,1,\ldots,N-1,$$ where $\lambda$ is a constant, $\mathbf{w}\in\mathbb{R}^m$, and $E:\mathbb{R}^{n+m}\to \mathbb{R}$ is some differentiable function.

Let $Q:\mathbb{R}^{n}\to \mathbb{R}$ be a differentiable function and $L=Q(\mathbf{y} _N)$.

Applying the chain rule we have: $$\frac{dL}{d\mathbf{w}} = \sum_{k=1}^N\frac{\partial \mathbf{y}_k}{\partial \mathbf{w}} \frac{dQ}{d\mathbf{y}_k}\qquad (1)$$ and $$\frac{dQ}{d\mathbf{y}_k} = \frac{\partial \mathbf{y}_{k+1}}{\partial \mathbf{y}_{k}} \frac{dQ}{d\mathbf{y}_{k+1}}.\qquad (2)$$

(Source: this paper, equation (12) and the one between (12) and (13).)

My questions: how to obtain $(1)$ and $(2)$?

I can show that if $(2)$ holds then $(1)$ holds. But I cannot see why $(2)$ holds. Consider $k=N-1$ for example:

Applying the chain rule we have: \begin{align}\frac{dQ}{d\mathbf{y}_{N-1}} &= \frac{dQ(\mathbf{y}_{N}(\mathbf{w},\mathbf{y}_{N-1}))}{d\mathbf{y}_{N-1}} \\ &= \frac{d\mathbf{y}_{N}(\mathbf{w},\mathbf{y}_{N-1})}{d\mathbf{y}_{N-1}}\frac{dQ}{d\mathbf{y}_{N}} \\ &= \begin{bmatrix}\frac{d\mathbf{w}}{d\mathbf{y}_{N-1}} & \mathbf{I}\end{bmatrix}\begin{bmatrix}\frac{\partial \mathbf{y}_{N}}{\partial \mathbf{w}} \\ \frac{\partial \mathbf{y}_{N}}{\partial \mathbf{y}_{N-1}}\end{bmatrix}\frac{dQ}{d\mathbf{y}_{N}} \\ &= \frac{d\mathbf{w}}{d\mathbf{y}_{N-1}}\frac{\partial \mathbf{y}_{N}}{\partial \mathbf{w}}\frac{dQ}{d\mathbf{y}_{N}} + \frac{\partial \mathbf{y}_{N}}{\partial \mathbf{y}_{N-1}}\frac{dQ}{d\mathbf{y}_{N}}, \end{align} which has an extra term $\frac{d\mathbf{w}}{d\mathbf{y}_{N-1}}\frac{\partial \mathbf{y}_{N}}{\partial \mathbf{w}}\frac{dQ}{d\mathbf{y}_{N}}$ compared to $(2)$.

Thank you in advance for your help.

Improve this question
$\endgroup$
2
  • $\begingroup$ I think the notation is a little confusing here, because some of these derivatives are partial derivatives, keeping $w$ constant, and others are total derivatives. I suspect that's where you went wrong. For example, $w$ doesn't explicitly depend on $y_{N-1}$, so why do you get the term $dw/dy_{N-1}$? Doesn't that look strange? Try a scalar case first, or maybe apply the chain rule in the forward direction instead. This is really a pure mathematics question, since there isn't anything specific to computational science, so it might be a little off-topic here. $\endgroup$
    – Kirill
    Commented Jun 25, 2017 at 23:38
  • $\begingroup$ @Kirill Thanks. $dw/dy_{N-1}$ comes from applying the chain rule as I showed above (and yes in the above equations, $d$ is for total derivative and $\partial$ is for partial derivatives). This is an active community and thus I posted this question here with the hope of having someone good at maths to help me :D $\endgroup$
    – f10w
    Commented Jun 26, 2017 at 9:07

1 Answer 1

Reset to default
1
$\begingroup$

The idea is that each $\vec{y}_k$ depends on $\vec{w}$. Therefore for the first equation, simply apply the chain rule for each $\vec{y}_k$ and sum them up. Componentwise one has: $$\left(\frac{dL}{d\vec{w}}\right)_{i}=\sum_{k,\alpha}{\left(\frac{dQ}{d\vec{y}_k}\right)_{\alpha}\left(\frac{\partial \vec{y}_k}{\partial \vec{w}}\right)_{i\alpha}} \tag{*}$$ Maybe the order in which they appear multiplying each other has confused you.

The second is obtained supposing that the function $\vec{y}_{k}$ can be put as a function of $\vec{y}_{k+1}$ (your first equation suggets this). Therefore the derivative in $(*)$ can be expressed as: $$\frac{dQ}{d\vec{y}_k}=\frac{\partial \vec{y}_{k+1}}{\partial \vec{y}_k}\frac{dQ}{d\vec{y}_{k+1}}$$

Improve this answer
$\endgroup$
9
  • $\begingroup$ I think you have reversed the positions of $d$ and $\partial$. Let me recall the chain rule for a vector-valued function $\mathbf{f}$ of the form $\mathbf{f}(\mathbf{x}_1(\mathbf{t}),\mathbf{x}_2(\mathbf{t}),\ldots,\mathbf{x}_n(\mathbf{t}))$: \begin{equation} \boxed{\frac{d\mathbf{f}}{d\mathbf{t}} = \sum_{i=1}^n \frac{d\mathbf{x}_i}{d\mathbf{t}}\frac{\partial \mathbf{f}}{\partial \mathbf{x}_i}}. \end{equation} Applying this we have $$\frac{dL}{dw} = \sum_{k=1}^N \frac{dy_k}{dw}\frac{\partial Q}{\partial y_k}.$$ However,... (continued next)... $\endgroup$
    – f10w
    Commented Jun 26, 2017 at 22:28
  • $\begingroup$ However, this is trivial because the left-hand side is equal to $\frac{dy_N}{dw}\frac{\partial Q}{\partial y_N}$ (because $\frac{\partial Q}{\partial y_k} = 0 \forall k\neq N$), which is $\frac{dy_N}{dw}\frac{d Q}{d y_N} = \frac{dQ}{dw}$. $\endgroup$
    – f10w
    Commented Jun 26, 2017 at 22:29
  • $\begingroup$ As far as I see: $Q$ is a function only of $\vec{y}_k$ for an arbitrary $k$ i.e. $Q=Q(\vec{y}_k)$. The vector $\vec{y}_{k}$ is a function of $\vec{y}_{k-1}$ and $\vec{w}$. Therefore applying the chain rule we have: $$\frac{dQ(\vec{y}_k(\vec{y}_{k-1},\vec{w}))}{d\vec{w}}=\left(\frac{\partial \vec{y}_k}{\partial \vec{w}}\right)^{T}\frac{dQ}{d\vec{y}_k}$$ $\endgroup$
    – HBR
    Commented Jun 27, 2017 at 18:38
  • $\begingroup$ More precisely $Q$ is a function of $y_N$: $Q=Q(y_N)= Q(y_N(w,y_{N-1}))$. Which chain rule did you apply to get $\frac{dQ(y_k(w,y_{k-1}))}{dw} = \frac{\partial y_k}{\partial w} \frac{dQ}{dy_k}$? (convention: vectors are transposed accordingly) Applying the chain rule, it should be $\frac{dQ(y_k(w,y_{k-1}))}{dw} = \frac{\partial y_k}{\partial w} \frac{dQ}{dy_k} + \frac{dy_{k-1}}{dw}\frac{\partial y_k}{\partial y_{k-1}}\frac{dQ}{dy_k}$. I think the authors made some mistakes in applying the chain rule. $\endgroup$
    – f10w
    Commented Jun 28, 2017 at 12:24
  • $\begingroup$ See also my answer here: math.stackexchange.com/questions/2335740/… $\endgroup$
    – f10w
    Commented Jun 28, 2017 at 12:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.