This commit is contained in:
Morten Hjorth-Jensen
2023-08-27 21:47:32 +02:00
parent 782156697d
commit c23ad32c2c
10 changed files with 516 additions and 512 deletions
+2 -2
View File
@@ -405,11 +405,11 @@ $$
y_i = \sum_{j=0}^{n-1}a_{ij}x_j,
$$
<p>with \( \all i=0,1,2,\dots,m-1 \). The individual matrix elements of \( \boldsymbol{A} \) are given by the symbol \( a_{ij} \).
<p>with \( \forall i=0,1,2,\dots,m-1 \). The individual matrix elements of \( \boldsymbol{A} \) are given by the symbol \( a_{ij} \).
It follows that the partial derivatives of \( y_i \) with respect to \( x_k \)
</p>
$$
\frac{\partial y_i }{\partial x_k}= a_{ik} \all i=0,1,2,\dots,m-1.
\frac{\partial y_i }{\partial x_k}= a_{ik} \forall i=0,1,2,\dots,m-1.
$$
<p>From this we have, using the definition of the Jacobian</p>
+2 -2
View File
@@ -405,7 +405,7 @@ multiplications
</p>
$$
\alpha = \boldsymbol{y}^T\boldsymbol{A}\boldsymbol{x}$,
\alpha = \boldsymbol{y}^T\boldsymbol{A}\boldsymbol{x},
$$
<p>with \( \boldsymbol{y} \) a vector of length \( m \), \( \boldsymbol{A} \) an \( m\times n \) matrix and \( \boldsymbol{x} \) a vector of length \( n \). We assume also that \( \boldsymbol{A} \) does not depend on any of the two vectors.
@@ -413,7 +413,7 @@ In order to find the derivative of \( \alpha \) with respect to the two vectors,
\( \boldsymbol{z}^T=\boldsymbol{y}^T\boldsymbol{A} \), a vector of length \( n \). We have then
</p>
$$
\alpha = \boldsymbol{z}^T\boldsymbol{x}$,
\alpha = \boldsymbol{z}^T\boldsymbol{x},
$$
<p>which means that (using our previous example) we have</p>
+3 -3
View File
@@ -405,14 +405,14 @@ matrix with dimension \( n\times n \).
</p>
$$
\alpha = \boldsymbol{x}^T\boldsymbol{A}\boldsymbol{x}$,
\alpha = \boldsymbol{x}^T\boldsymbol{A}\boldsymbol{x},
$$
<p>with \( \boldsymbol{x} \) a vector of length \( n \).</p>
<p>We write out the specific sums involved in the calculation of \( \alpha \)</p>
$$
\alpha = \sum_{i=0}^{n-1}\sum_{j=0}^({n-1}x_i a_{ij}x_j,
\alpha = \sum_{i=0}^{n-1}\sum_{j=0}^{n-1}x_i a_{ij}x_j,
$$
<p>taking the derivative of \( \alpha \) with respect to a given component \( x_k \) we get the two sums</p>
@@ -420,7 +420,7 @@ $$
\frac{\partial \alpha}{\partial x_k} = \sum_{i=0}^{n-1}a_{ik}x_i+\sum_{j=0}^{n-1}a_{kj}x_j,
$$
<p>for \( \all k =0,1,2,\dots,n-1 \). We identify these sums as</p>
<p>for \( \forall k =0,1,2,\dots,n-1 \). We identify these sums as</p>
$$
\frac{\partial \alpha}{\partial \boldsymbol{x}} = \boldsymbol{x}^T\left(\boldsymbol{A}^T+\boldsymbol{A}\right).
$$
+1 -1
View File
@@ -421,7 +421,7 @@ $$
\frac{\partial \alpha}{\partial z_k} = \sum_{i=0}^{n-1}\left(x_i\frac{\partial y_i}{\partial z_k}+y_i\frac{\partial x_i}{\partial z_k}\right),
$$
<p>for \( \all k =0,1,2,\dots,n-1 \). We can rewrite the partial derivative in a more compact form as</p>
<p>for \( \forall k =0,1,2,\dots,n-1 \). We can rewrite the partial derivative in a more compact form as</p>
$$
\frac{\partial \alpha}{\partial \boldsymbol{z}} = \boldsymbol{x}^T\frac{\partial \boldsymbol{y}}{\partial \boldsymbol{z}}+\boldsymbol{y}^T\frac{\partial \boldsymbol{x}}{\partial \boldsymbol{z}},
$$
+26 -22
View File
@@ -825,12 +825,12 @@ y_i = \sum_{j=0}^{n-1}a_{ij}x_j,
$$
<p>&nbsp;<br>
<p>with \( \all i=0,1,2,\dots,m-1 \). The individual matrix elements of \( \boldsymbol{A} \) are given by the symbol \( a_{ij} \).
<p>with \( \forall i=0,1,2,\dots,m-1 \). The individual matrix elements of \( \boldsymbol{A} \) are given by the symbol \( a_{ij} \).
It follows that the partial derivatives of \( y_i \) with respect to \( x_k \)
</p>
<p>&nbsp;<br>
$$
\frac{\partial y_i }{\partial x_k}= a_{ik} \all i=0,1,2,\dots,m-1.
\frac{\partial y_i }{\partial x_k}= a_{ik} \forall i=0,1,2,\dots,m-1.
$$
<p>&nbsp;<br>
@@ -851,33 +851,35 @@ just think of the mean squared error) as the result of some matrix vector
multiplications
</p>
$$
\alpha = \boldsymbol{y}^T\boldsymbol{A}\boldsymbol{x}$,
<p>&nbsp;<br>
$$
\alpha = \boldsymbol{y}^T\boldsymbol{A}\boldsymbol{x},
$$
<p>&nbsp;<br>
<p>with \( \boldsymbol{y} \) a vector of length \( m \), \( \boldsymbol{A} \) an \( m\times n \) matrix and \( \boldsymbol{x} \) a vector of length \( n \). We assume also that \( \boldsymbol{A} \) does not depend on any of the two vectors.
In order to find the derivative of \( \alpha \) with respect to the two vectors, we define an intermediate vector \( \boldsymbol{z} \). We define first
\( \boldsymbol{z}^T=\boldsymbol{y}^T\boldsymbol{A} \), a vector of length \( n \). We have then
</p>
$$
<p>&nbsp;<br>
\alpha = \boldsymbol{z}^T\boldsymbol{x}$,
<p>&nbsp;<br>
$$
\alpha = \boldsymbol{z}^T\boldsymbol{x},
$$
<p>&nbsp;<br>
<p>which means that (using our previous example) we have</p>
$$
<p>&nbsp;<br>
$$
\frac{\partial \alpha}{\partial \boldsymbol{x}} = \boldsymbol{z}^T=\boldsymbol{y}^T\boldsymbol{A}.
<p>&nbsp;<br>
$$
<p>&nbsp;<br>
<p>Since \( \alpha \) is a scalar we have \( \alpha =\alpha^T=\boldsymbol{x}^T\boldsymbol{A}^T\boldsymbol{y} \). Defining now \( \boldsymbol{z}=\boldsymbol{x}^T\boldsymbol{A}^T \) we find that</p>
$$
<p>&nbsp;<br>
$$
\frac{\partial \alpha}{\partial \boldsymbol{y}} = \boldsymbol{z}^T=\boldsymbol{x}^T\boldsymbol{A}^T..
$$
<p>&nbsp;<br>
</section>
<section>
@@ -888,39 +890,41 @@ replaced by a vector \( \boldsymbol{x} \) and the matrix \( \boldsymbol{A} \) is
matrix with dimension \( n\times n \).
</p>
$$
\alpha = \boldsymbol{x}^T\boldsymbol{A}\boldsymbol{x}$,
<p>&nbsp;<br>
$$
\alpha = \boldsymbol{x}^T\boldsymbol{A}\boldsymbol{x},
$$
<p>&nbsp;<br>
<p>with \( \boldsymbol{x} \) a vector of length \( n \).</p>
<p>We write out the specific sums involved in the calculation of \( \alpha \)</p>
$$
<p>&nbsp;<br>
\alpha = \sum_{i=0}^{n-1}\sum_{j=0}^({n-1}x_i a_{ij}x_j,
<p>&nbsp;<br>
$$
\alpha = \sum_{i=0}^{n-1}\sum_{j=0}^{n-1}x_i a_{ij}x_j,
$$
<p>&nbsp;<br>
<p>taking the derivative of \( \alpha \) with respect to a given component \( x_k \) we get the two sums</p>
$$
<p>&nbsp;<br>
$$
\frac{\partial \alpha}{\partial x_k} = \sum_{i=0}^{n-1}a_{ik}x_i+\sum_{j=0}^{n-1}a_{kj}x_j,
<p>&nbsp;<br>
$$
<p>&nbsp;<br>
<p>for \( \all k =0,1,2,\dots,n-1 \). We identify these sums as</p>
$$
<p>for \( \forall k =0,1,2,\dots,n-1 \). We identify these sums as</p>
<p>&nbsp;<br>
$$
\frac{\partial \alpha}{\partial \boldsymbol{x}} = \boldsymbol{x}^T\left(\boldsymbol{A}^T+\boldsymbol{A}\right).
<p>&nbsp;<br>
$$
<p>&nbsp;<br>
<p>If the matrix \( \boldsymbol{A} \) is symmetric, that is \( \boldsymbol{A}=\boldsymbol{A}^T \), we have </p>
$$
<p>&nbsp;<br>
$$
\frac{\partial \alpha}{\partial \boldsymbol{x}} = 2\boldsymbol{x}^T\boldsymbol{A}.
$$
<p>&nbsp;<br>
</section>
<section>
@@ -954,7 +958,7 @@ $$
$$
<p>&nbsp;<br>
<p>for \( \all k =0,1,2,\dots,n-1 \). We can rewrite the partial derivative in a more compact form as</p>
<p>for \( \forall k =0,1,2,\dots,n-1 \). We can rewrite the partial derivative in a more compact form as</p>
<p>&nbsp;<br>
$$
\frac{\partial \alpha}{\partial \boldsymbol{z}} = \boldsymbol{x}^T\frac{\partial \boldsymbol{y}}{\partial \boldsymbol{z}}+\boldsymbol{y}^T\frac{\partial \boldsymbol{x}}{\partial \boldsymbol{z}},
+8 -8
View File
@@ -906,11 +906,11 @@ $$
y_i = \sum_{j=0}^{n-1}a_{ij}x_j,
$$
<p>with \( \all i=0,1,2,\dots,m-1 \). The individual matrix elements of \( \boldsymbol{A} \) are given by the symbol \( a_{ij} \).
<p>with \( \forall i=0,1,2,\dots,m-1 \). The individual matrix elements of \( \boldsymbol{A} \) are given by the symbol \( a_{ij} \).
It follows that the partial derivatives of \( y_i \) with respect to \( x_k \)
</p>
$$
\frac{\partial y_i }{\partial x_k}= a_{ik} \all i=0,1,2,\dots,m-1.
\frac{\partial y_i }{\partial x_k}= a_{ik} \forall i=0,1,2,\dots,m-1.
$$
<p>From this we have, using the definition of the Jacobian</p>
@@ -929,7 +929,7 @@ multiplications
</p>
$$
\alpha = \boldsymbol{y}^T\boldsymbol{A}\boldsymbol{x}$,
\alpha = \boldsymbol{y}^T\boldsymbol{A}\boldsymbol{x},
$$
<p>with \( \boldsymbol{y} \) a vector of length \( m \), \( \boldsymbol{A} \) an \( m\times n \) matrix and \( \boldsymbol{x} \) a vector of length \( n \). We assume also that \( \boldsymbol{A} \) does not depend on any of the two vectors.
@@ -937,7 +937,7 @@ In order to find the derivative of \( \alpha \) with respect to the two vectors,
\( \boldsymbol{z}^T=\boldsymbol{y}^T\boldsymbol{A} \), a vector of length \( n \). We have then
</p>
$$
\alpha = \boldsymbol{z}^T\boldsymbol{x}$,
\alpha = \boldsymbol{z}^T\boldsymbol{x},
$$
<p>which means that (using our previous example) we have</p>
@@ -960,14 +960,14 @@ matrix with dimension \( n\times n \).
</p>
$$
\alpha = \boldsymbol{x}^T\boldsymbol{A}\boldsymbol{x}$,
\alpha = \boldsymbol{x}^T\boldsymbol{A}\boldsymbol{x},
$$
<p>with \( \boldsymbol{x} \) a vector of length \( n \).</p>
<p>We write out the specific sums involved in the calculation of \( \alpha \)</p>
$$
\alpha = \sum_{i=0}^{n-1}\sum_{j=0}^({n-1}x_i a_{ij}x_j,
\alpha = \sum_{i=0}^{n-1}\sum_{j=0}^{n-1}x_i a_{ij}x_j,
$$
<p>taking the derivative of \( \alpha \) with respect to a given component \( x_k \) we get the two sums</p>
@@ -975,7 +975,7 @@ $$
\frac{\partial \alpha}{\partial x_k} = \sum_{i=0}^{n-1}a_{ik}x_i+\sum_{j=0}^{n-1}a_{kj}x_j,
$$
<p>for \( \all k =0,1,2,\dots,n-1 \). We identify these sums as</p>
<p>for \( \forall k =0,1,2,\dots,n-1 \). We identify these sums as</p>
$$
\frac{\partial \alpha}{\partial \boldsymbol{x}} = \boldsymbol{x}^T\left(\boldsymbol{A}^T+\boldsymbol{A}\right).
$$
@@ -1011,7 +1011,7 @@ $$
\frac{\partial \alpha}{\partial z_k} = \sum_{i=0}^{n-1}\left(x_i\frac{\partial y_i}{\partial z_k}+y_i\frac{\partial x_i}{\partial z_k}\right),
$$
<p>for \( \all k =0,1,2,\dots,n-1 \). We can rewrite the partial derivative in a more compact form as</p>
<p>for \( \forall k =0,1,2,\dots,n-1 \). We can rewrite the partial derivative in a more compact form as</p>
$$
\frac{\partial \alpha}{\partial \boldsymbol{z}} = \boldsymbol{x}^T\frac{\partial \boldsymbol{y}}{\partial \boldsymbol{z}}+\boldsymbol{y}^T\frac{\partial \boldsymbol{x}}{\partial \boldsymbol{z}},
$$
+8 -8
View File
@@ -983,11 +983,11 @@ $$
y_i = \sum_{j=0}^{n-1}a_{ij}x_j,
$$
<p>with \( \all i=0,1,2,\dots,m-1 \). The individual matrix elements of \( \boldsymbol{A} \) are given by the symbol \( a_{ij} \).
<p>with \( \forall i=0,1,2,\dots,m-1 \). The individual matrix elements of \( \boldsymbol{A} \) are given by the symbol \( a_{ij} \).
It follows that the partial derivatives of \( y_i \) with respect to \( x_k \)
</p>
$$
\frac{\partial y_i }{\partial x_k}= a_{ik} \all i=0,1,2,\dots,m-1.
\frac{\partial y_i }{\partial x_k}= a_{ik} \forall i=0,1,2,\dots,m-1.
$$
<p>From this we have, using the definition of the Jacobian</p>
@@ -1006,7 +1006,7 @@ multiplications
</p>
$$
\alpha = \boldsymbol{y}^T\boldsymbol{A}\boldsymbol{x}$,
\alpha = \boldsymbol{y}^T\boldsymbol{A}\boldsymbol{x},
$$
<p>with \( \boldsymbol{y} \) a vector of length \( m \), \( \boldsymbol{A} \) an \( m\times n \) matrix and \( \boldsymbol{x} \) a vector of length \( n \). We assume also that \( \boldsymbol{A} \) does not depend on any of the two vectors.
@@ -1014,7 +1014,7 @@ In order to find the derivative of \( \alpha \) with respect to the two vectors,
\( \boldsymbol{z}^T=\boldsymbol{y}^T\boldsymbol{A} \), a vector of length \( n \). We have then
</p>
$$
\alpha = \boldsymbol{z}^T\boldsymbol{x}$,
\alpha = \boldsymbol{z}^T\boldsymbol{x},
$$
<p>which means that (using our previous example) we have</p>
@@ -1037,14 +1037,14 @@ matrix with dimension \( n\times n \).
</p>
$$
\alpha = \boldsymbol{x}^T\boldsymbol{A}\boldsymbol{x}$,
\alpha = \boldsymbol{x}^T\boldsymbol{A}\boldsymbol{x},
$$
<p>with \( \boldsymbol{x} \) a vector of length \( n \).</p>
<p>We write out the specific sums involved in the calculation of \( \alpha \)</p>
$$
\alpha = \sum_{i=0}^{n-1}\sum_{j=0}^({n-1}x_i a_{ij}x_j,
\alpha = \sum_{i=0}^{n-1}\sum_{j=0}^{n-1}x_i a_{ij}x_j,
$$
<p>taking the derivative of \( \alpha \) with respect to a given component \( x_k \) we get the two sums</p>
@@ -1052,7 +1052,7 @@ $$
\frac{\partial \alpha}{\partial x_k} = \sum_{i=0}^{n-1}a_{ik}x_i+\sum_{j=0}^{n-1}a_{kj}x_j,
$$
<p>for \( \all k =0,1,2,\dots,n-1 \). We identify these sums as</p>
<p>for \( \forall k =0,1,2,\dots,n-1 \). We identify these sums as</p>
$$
\frac{\partial \alpha}{\partial \boldsymbol{x}} = \boldsymbol{x}^T\left(\boldsymbol{A}^T+\boldsymbol{A}\right).
$$
@@ -1088,7 +1088,7 @@ $$
\frac{\partial \alpha}{\partial z_k} = \sum_{i=0}^{n-1}\left(x_i\frac{\partial y_i}{\partial z_k}+y_i\frac{\partial x_i}{\partial z_k}\right),
$$
<p>for \( \all k =0,1,2,\dots,n-1 \). We can rewrite the partial derivative in a more compact form as</p>
<p>for \( \forall k =0,1,2,\dots,n-1 \). We can rewrite the partial derivative in a more compact form as</p>
$$
\frac{\partial \alpha}{\partial \boldsymbol{z}} = \boldsymbol{x}^T\frac{\partial \boldsymbol{y}}{\partial \boldsymbol{z}}+\boldsymbol{y}^T\frac{\partial \boldsymbol{x}}{\partial \boldsymbol{z}},
$$
Binary file not shown.
File diff suppressed because it is too large Load Diff
+8 -8
View File
@@ -551,11 +551,11 @@ Let now $\bm{y}=\bm{A}\bm{x}$, where $\bm{A}$ is an $m\times n$ matrix and the m
y_i = \sum_{j=0}^{n-1}a_{ij}x_j,
\]
!et
with $\all i=0,1,2,\dots,m-1$. The individual matrix elements of $\bm{A}$ are given by the symbol $a_{ij}$.
with $\forall i=0,1,2,\dots,m-1$. The individual matrix elements of $\bm{A}$ are given by the symbol $a_{ij}$.
It follows that the partial derivatives of $y_i$ with respect to $x_k$
!bt
\[
\frac{\partial y_i }{\partial x_k}= a_{ik} \all i=0,1,2,\dots,m-1.
\frac{\partial y_i }{\partial x_k}= a_{ik} \forall i=0,1,2,\dots,m-1.
\]
!et
@@ -576,7 +576,7 @@ multiplications
!bt
\[
\alpha = \bm{y}^T\bm{A}\bm{x}$,
\alpha = \bm{y}^T\bm{A}\bm{x},
\]
!et
with $\bm{y}$ a vector of length $m$, $\bm{A}$ an $m\times n$ matrix and $\bm{x}$ a vector of length $n$. We assume also that $\bm{A}$ does not depend on any of the two vectors.
@@ -584,7 +584,7 @@ In order to find the derivative of $\alpha$ with respect to the two vectors, we
$\bm{z}^T=\bm{y}^T\bm{A}$, a vector of length $n$. We have then
!bt
\[
\alpha = \bm{z}^T\bm{x}$,
\alpha = \bm{z}^T\bm{x},
\]
!et
which means that (using our previous example) we have
@@ -612,7 +612,7 @@ matrix with dimension $n\times n$.
!bt
\[
\alpha = \bm{x}^T\bm{A}\bm{x}$,
\alpha = \bm{x}^T\bm{A}\bm{x},
\]
!et
with $\bm{x}$ a vector of length $n$.
@@ -620,7 +620,7 @@ with $\bm{x}$ a vector of length $n$.
We write out the specific sums involved in the calculation of $\alpha$
!bt
\[
\alpha = \sum_{i=0}^{n-1}\sum_{j=0}^({n-1}x_i a_{ij}x_j,
\alpha = \sum_{i=0}^{n-1}\sum_{j=0}^{n-1}x_i a_{ij}x_j,
\]
!et
taking the derivative of $\alpha$ with respect to a given component $x_k$ we get the two sums
@@ -629,7 +629,7 @@ taking the derivative of $\alpha$ with respect to a given component $x_k$ we get
\frac{\partial \alpha}{\partial x_k} = \sum_{i=0}^{n-1}a_{ik}x_i+\sum_{j=0}^{n-1}a_{kj}x_j,
\]
!et
for $\all k =0,1,2,\dots,n-1$. We identify these sums as
for $\forall k =0,1,2,\dots,n-1$. We identify these sums as
!bt
\[
\frac{\partial \alpha}{\partial \bm{x}} = \bm{x}^T\left(\bm{A}^T+\bm{A}\right).
@@ -670,7 +670,7 @@ and the partial derivative
\frac{\partial \alpha}{\partial z_k} = \sum_{i=0}^{n-1}\left(x_i\frac{\partial y_i}{\partial z_k}+y_i\frac{\partial x_i}{\partial z_k}\right),
\]
!et
for $\all k =0,1,2,\dots,n-1$. We can rewrite the partial derivative in a more compact form as
for $\forall k =0,1,2,\dots,n-1$. We can rewrite the partial derivative in a more compact form as
!bt
\[
\frac{\partial \alpha}{\partial \bm{z}} = \bm{x}^T\frac{\partial \bm{y}}{\partial \bm{z}}+\bm{y}^T\frac{\partial \bm{x}}{\partial \bm{z}},