链式法则

1 向量对向量求导法则

$$\frac{\partial \mathbf{z}}{\partial \mathbf{x}}=\frac{\partial \mathbf{y}}{\partial \mathbf{x}}\frac{\partial \mathbf{z}}{\partial \mathbf{y}}$$

证明： $$d\mathbf{y} = \frac{\partial \mathbf{y}}{\partial \mathbf{x}}^Td\mathbf{x} \\d\mathbf{z} = \frac{\partial \mathbf{z}}{\partial \mathbf{y}}^Td\mathbf{y} \\ = \frac{\partial \mathbf{z}}{\partial \mathbf{y}}^T\frac{\partial \mathbf{y}}{\partial \mathbf{x}}^Tdy \\ = (\frac{\partial \mathbf{y}}{\partial \mathbf{x}}\frac{\partial \mathbf{z}}{\partial \mathbf{y}})^Tdy$$

2 矩阵对矩阵求导法则

$$\frac{\partial F}{\partial X} = \frac{\partial Y}{\partial X}\frac{\partial F}{\partial Y}$$

证明：

$$Vec(dF)=\frac{\partial F}{\partial Y}^TVec(dY) \\ = \frac{\partial F}{\partial Y}^T \frac{\partial Y}{\partial X}^TVec(dX) \\ = (\frac{\partial Y}{\partial X}\frac{\partial F}{\partial Y})^TVec(dX) \\ \longrightarrow \frac{\partial F}{\partial X} = \frac{\partial Y}{\partial X}\frac{\partial F}{\partial Y}$$

3 标量对多个向量求导法则

$$\frac{\partial \mathbf{z}}{\partial \mathbf{x}}=\frac{\partial \mathbf{y}}{\partial \mathbf{x}}\frac{\partial \mathbf{z}}{\partial \mathbf{y}}$$

证明： $$d\mathbf{y} = \frac{\partial \mathbf{y}}{\partial \mathbf{x}}^Td\mathbf{x} \\dz = \frac{\partial z}{\partial \mathbf{y}}^Td\mathbf y \\ = \frac{\partial z}{\partial \mathbf{y}}^T\frac{\partial \mathbf{y}}{\partial \mathbf{x}}^Tdy \\ = (\frac{\partial \mathbf{y}}{\partial \mathbf{x}}\frac{\partial z}{\partial \mathbf{y}})^Tdy$$

4 标量对多个矩阵求导法则

1. 不能随意沿用标量的链式法则, 因为矩阵对矩阵的导数未定义。所以应该从微分入手，写写出$df = tr(\frac{\partial f}{\partial Y}^TdY)$， 然后把$dY$用$dX​$表示出来

2. 对于$Y=AXB$, 有 $$df = tr(\frac{\partial f}{\partial Y}^T dY) \\= tr(\frac{\partial f}{\partial Y}^T AdXB) \\=tr(B\frac{\partial f}{\partial Y}^T AdX) \\=tr((A^T\frac{\partial f}{\partial Y}B^T)dX) \\ \longrightarrow \frac{\partial f}{\partial X} = A^T\frac{\partial f}{\partial Y}B^T$$