From 092fdecd9b9bf49014bf60ff1637096a746125f4 Mon Sep 17 00:00:00 2001 From: mhjensen Date: Fri, 19 Oct 2018 06:07:18 +0200 Subject: [PATCH] more boring typos --- doc/pub/NeuralNet/html/._NeuralNet-bs069.html | 2 +- doc/pub/NeuralNet/html/._NeuralNet-bs089.html | 14 +- doc/pub/NeuralNet/html/._NeuralNet-bs092.html | 6 +- doc/pub/NeuralNet/html/._NeuralNet-bs093.html | 4 +- doc/pub/NeuralNet/html/._NeuralNet-bs094.html | 16 +-- doc/pub/NeuralNet/html/._NeuralNet-bs095.html | 14 +- doc/pub/NeuralNet/html/._NeuralNet-bs096.html | 26 ++-- doc/pub/NeuralNet/html/._NeuralNet-bs099.html | 4 +- doc/pub/NeuralNet/html/._NeuralNet-bs100.html | 8 +- doc/pub/NeuralNet/html/._NeuralNet-bs101.html | 10 +- doc/pub/NeuralNet/html/._NeuralNet-bs102.html | 24 ++-- doc/pub/NeuralNet/html/NeuralNet-reveal.html | 128 +++++++++--------- .../NeuralNet/html/NeuralNet-solarized.html | 128 +++++++++--------- doc/pub/NeuralNet/html/NeuralNet.html | 128 +++++++++--------- doc/pub/NeuralNet/ipynb/NeuralNet.ipynb | 128 +++++++++--------- .../ipynb/ipynb-NeuralNet-src.tar.gz | Bin 88051 -> 88051 bytes doc/pub/NeuralNet/pdf/NeuralNet-minted.pdf | Bin 664646 -> 664860 bytes doc/src/NeuralNet/NeuralNet.do.txt | 128 +++++++++--------- 18 files changed, 384 insertions(+), 384 deletions(-) diff --git a/doc/pub/NeuralNet/html/._NeuralNet-bs069.html b/doc/pub/NeuralNet/html/._NeuralNet-bs069.html index ededfecf4..c07eae397 100644 --- a/doc/pub/NeuralNet/html/._NeuralNet-bs069.html +++ b/doc/pub/NeuralNet/html/._NeuralNet-bs069.html @@ -387,7 +387,7 @@ depth respectively). The neurons in a layer will only be connected to a small region of the layer before it, instead of all of the neurons in a fully-connected manner. Moreover, the final -output layer could for this specific image have dimensions \( 1\times 1 times 10 \), +output layer could for this specific image have dimensions \( 1\times 1 \times 10 \), because by the end of the CNN architecture we will reduce the full image into a single vector of class scores, arranged along the depth diff --git a/doc/pub/NeuralNet/html/._NeuralNet-bs089.html b/doc/pub/NeuralNet/html/._NeuralNet-bs089.html index edce247f1..c2e377031 100644 --- a/doc/pub/NeuralNet/html/._NeuralNet-bs089.html +++ b/doc/pub/NeuralNet/html/._NeuralNet-bs089.html @@ -381,17 +381,17 @@ and biases for each layer.

It is assumed that there are no weights and -bias at the input layer, so \( P = \{ P_{\text{hidden}}, -P_{\text{output}} \} \). If there are \( N_{\text{hidden} } \) neurons in -the hidden layer, then \( P_{\text{hidden}} \) is an \( N_{\text{hidden} } +bias at the input layer, so \( P = \{ P_{\mathrm{hidden}}, +P_{\mathrm{output}} \} \). If there are \( N_{\mathrm{hidden} } \) neurons in +the hidden layer, then \( P_{\mathrm{hidden}} \) is an \( N_{\mathrm{hidden} } \times 2 \) matrix.

-The first column in \( P_{\text{hidden} } \) represents +The first column in \( P_{\mathrm{hidden} } \) represents the bias for each neuron in the hidden layer and the second column -represents the weigths for each neuron. If there are \( N_{\text{output} -} \) neurons in the output layer, then \( P_{\text{output}} \) is a -\( N_{\text{output} } \times (1 + N_{\text{hidden} }) \) matrix. Its first +represents the weigths for each neuron. If there are \( N_{\mathrm{output} +} \) neurons in the output layer, then \( P_{\mathrm{output}} \) is a +\( N_{\mathrm{output} } \times (1 + N_{\mathrm{hidden} }) \) matrix. Its first column represents the bias of each neuron and the remaining columns represents the weights to each neuron. diff --git a/doc/pub/NeuralNet/html/._NeuralNet-bs092.html b/doc/pub/NeuralNet/html/._NeuralNet-bs092.html index 8aecae5db..39d3ae459 100644 --- a/doc/pub/NeuralNet/html/._NeuralNet-bs092.html +++ b/doc/pub/NeuralNet/html/._NeuralNet-bs092.html @@ -381,7 +381,7 @@ $$ or, in terms of weights and biases for each layer: $$ -\min_{P_{\text{hidden} }, \ P_{\text{output} }}\Big\{ \big(g_t'(x, \{ P_{\text{hidden} }, P_{\text{output} }\}) - ( -\gamma g_t(x, \{ P_{\text{hidden} }, P_{\text{output} }\}) \big)^2 \Big\} +\min_{P_{\mathrm{hidden} }, \ P_{\mathrm{output} }}\Big\{ \big(g_t'(x, \{ P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) - ( -\gamma g_t(x, \{ P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) \big)^2 \Big\} $$

@@ -405,10 +405,10 @@ $$ $$

-or in terms of \( P_{\text{hidden} } \) and \( P_{\text{output} } \) +or in terms of \( P_{\mathrm{hidden} } \) and \( P_{\mathrm{output} } \) $$ -\min_{P_{\text{hidden} }, \ P_{\text{output} }} c(x, \{P_{\text{hidden} }, P_{\text{output} }\}) +\min_{P_{\mathrm{hidden} }, \ P_{\mathrm{output} }} c(x, \{P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) $$

diff --git a/doc/pub/NeuralNet/html/._NeuralNet-bs093.html b/doc/pub/NeuralNet/html/._NeuralNet-bs093.html index 5ae4d269a..c54af6cb3 100644 --- a/doc/pub/NeuralNet/html/._NeuralNet-bs093.html +++ b/doc/pub/NeuralNet/html/._NeuralNet-bs093.html @@ -380,8 +380,8 @@ extension of it into a deep neural network would (hopefully) be painless.

-For simplicity, we assume that the input is an array \( \vec x = -(x_1, \dots, x_N) \) with \( N \) elements. It is at these points the neural +For simplicity, we assume that the input is an array +\( \hat{x}= (x_1, \dots, x_N) \) with \( N \) elements. It is at these points the neural network should find \( P \) such that it fulfills (19).

diff --git a/doc/pub/NeuralNet/html/._NeuralNet-bs094.html b/doc/pub/NeuralNet/html/._NeuralNet-bs094.html index feabd678d..99244eaac 100644 --- a/doc/pub/NeuralNet/html/._NeuralNet-bs094.html +++ b/doc/pub/NeuralNet/html/._NeuralNet-bs094.html @@ -366,24 +366,24 @@ MathJax.Hub.Config({

Feedforward

-First, a feedforward of the inputs must be done. This means that \( \vec - x \) must be passed through an input layer, a hidden layer and a output +First, a feedforward of the inputs must be done. This means that \( \hat{x} \) +must be passed through an input layer, a hidden layer and a output layer. The input layer in this case, does not need to process the - data any further. The input layer will consist of \( N_{\text{input} } \) + data any further. The input layer will consist of \( N_{\mathrm{input} } \) neurons, passing its element to each neuron in the hidden layer. The - number of neurons in the hidden layer will be \( N_{\text{hidden} } \). + number of neurons in the hidden layer will be \( N_{\mathrm{hidden} } \).

-For the \( i \)-th in the hidden layer with weight \( w_i^{\text{hidden} } \) -and bias \( b_i^{\text{hidden} } \), the weighting from the \( j \)-th neuron +For the \( i \)-th in the hidden layer with weight \( w_i^{\mathrm{hidden} } \) +and bias \( b_i^{\mathrm{hidden} } \), the weighting from the \( j \)-th neuron at the input layer is: $$ \begin{aligned} -z_{i,j}^{\text{hidden}} &= b_i^{\text{hidden}} + w_i^{\text{hidden}}x_j \\ +z_{i,j}^{\mathrm{hidden}} &= b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}}x_j \\ &= \begin{pmatrix} -b_i^{\text{hidden}} & w_i^{\text{hidden}} +b_i^{\mathrm{hidden}} & w_i^{\mathrm{hidden}} \end{pmatrix} \begin{pmatrix} 1 \\ diff --git a/doc/pub/NeuralNet/html/._NeuralNet-bs095.html b/doc/pub/NeuralNet/html/._NeuralNet-bs095.html index 3d26b73dd..7a2b19125 100644 --- a/doc/pub/NeuralNet/html/._NeuralNet-bs095.html +++ b/doc/pub/NeuralNet/html/._NeuralNet-bs095.html @@ -369,28 +369,28 @@ MathJax.Hub.Config({ The result after weighting the input at the \( i \)-th hidden neuron can be written as a vector: $$ \begin{aligned} -\vec{z}_{i}^{\text{hidden}} &= \Big( b_i^{\text{hidden}} + w_i^{\text{hidden}}x_1 , \ b_i^{\text{hidden}} + w_i^{\text{hidden}} x_2, \ \dots \, , \ b_i^{\text{hidden}} + w_i^{\text{hidden}} x_N\Big) \\ +\hat{z}_{i}^{\mathrm{hidden}} &= \Big( b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}}x_1 , \ b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}} x_2, \ \dots \, , \ b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}} x_N\Big) \\ &= \begin{pmatrix} - b_i^{\text{hidden}} & w_i^{\text{hidden}} + b_i^{\mathrm{hidden}} & w_i^{\mathrm{hidden}} \end{pmatrix} \begin{pmatrix} 1 & 1 & \dots & 1 \\ x_1 & x_2 & \dots & x_N \end{pmatrix} \\ -&= \vec{p}_{i, \text{hidden}}^T X +&= \hat{p}_{i, \mathrm{hidden}}^T X \end{aligned} $$

-It is the vector \( \vec{p}_{i, \text{hidden}}^T \) that defines each row -in \( P_{\text{hidden} } \), which contains the weights for the neural +It is the vector \( \hat{p}_{i, \mathrm{hidden}}^T \) that defines each row +in \( P_{\mathrm{hidden} } \), which contains the weights for the neural network to minimize according to (19).

-After having found \( \vec{z}_{i}^{\text{hidden}} \) for every neuron \( i \) +After having found \( \hat{z}_{i}^{\mathrm{hidden}} \) for every neuron \( i \) in the hidden layer, the vector will be sent to an activation function -\( a_i(\vec{z}) \). In this example, the sigmoid function has been used: +\( a_i(\hat{z}) \). In this example, the sigmoid function has been used: $$ f(z) = \frac{1}{1 + \exp{(-z)}}. diff --git a/doc/pub/NeuralNet/html/._NeuralNet-bs096.html b/doc/pub/NeuralNet/html/._NeuralNet-bs096.html index ebd8365fc..81744050f 100644 --- a/doc/pub/NeuralNet/html/._NeuralNet-bs096.html +++ b/doc/pub/NeuralNet/html/._NeuralNet-bs096.html @@ -366,20 +366,20 @@ MathJax.Hub.Config({

Output

-The output $\vec{x}_i^{\text{hidden} }$from each \( i \)-th hidden neuron is: +The output $\hat{x}_i^{\mathrm{hidden}}$from each \( i \)-th hidden neuron is: $$ -\vec{x}_i^{\text{hidden} } = f\big( \vec{z}_{i}^{\text{hidden}} \big). +\hat{x}_i^{\mathrm{hidden} } = f\big( \hat{z}_{i}^{\mathrm{hidden}} \big). $$

-The outputs \( \vec{x}_i^{\text{hidden} } \) are then sent to the output layer. +The outputs \( \hat{x}_i^{\mathrm{hidden} } \) are then sent to the output layer.

The output layer consist of one neuron in this case, and combines the output from each of the neurons in the hidden layers. The output layer combines the results from the hidden layer using some weights \( -w_i^{\text{output}} \) and biases \( b_i^{\text{output}} \). In this case, +w_i^{\mathrm{output}} \) and biases \( b_i^{\mathrm{output}} \). In this case, it is assumes that the number of neurons in the output layer is one.

@@ -389,36 +389,36 @@ layer described previously. $$ \begin{aligned} -z_{1,j}^{\text{output}} & = +z_{1,j}^{\mathrm{output}} & = \begin{pmatrix} -b_1^{\text{output}} & \vec{w}_1^{\text{output}} +b_1^{\mathrm{output}} & \hat{w}_1^{\mathrm{output}} \end{pmatrix} \begin{pmatrix} 1 \\ -\vec{x}_j^{\text{hidden}} +\hat{x}_j^{\mathrm{hidden}} \end{pmatrix} \end{aligned} $$

-Expressing \( z_{1,j}^{\text{output}} \) as a vector gives the following procedure of weighting the inputs from the hidden layer: +Expressing \( z_{1,j}^{\mathrm{output}} \) as a vector gives the following procedure of weighting the inputs from the hidden layer: $$ -\vec{z}_{1}^{\text{output}} = +\hat{z}_{1}^{\mathrm{output}} = \begin{pmatrix} -b_1^{\text{output}} & \vec{w}_1^{\text{output}} +b_1^{\mathrm{output}} & \hat{w}_1^{\mathrm{output}} \end{pmatrix} \begin{pmatrix} 1 & 1 & \dots & 1 \\ -\vec{x}_1^{\text{hidden}} & \vec{x}_2^{\text{hidden}} & \dots & \vec{x}_N^{\text{hidden}} +\hat{x}_1^{\mathrm{hidden}} & \hat{x}_2^{\mathrm{hidden}} & \dots & \hat{x}_N^{\mathrm{hidden}} \end{pmatrix} $$

In this case we seek a continous range of values since we are approximating a function. This means that after computing -\( \vec{z}_{1}^{\text{output}} \) the neural network has finished its -feedforward step, and \( \vec{z}_{1}^{\text{output}} \) is the final +\( \hat{z}_{1}^{\mathrm{output}} \) the neural network has finished its +feedforward step, and \( \hat{z}_{1}^{\mathrm{output}} \) is the final output of the network.

diff --git a/doc/pub/NeuralNet/html/._NeuralNet-bs099.html b/doc/pub/NeuralNet/html/._NeuralNet-bs099.html index 1ae4e033a..3d60f368d 100644 --- a/doc/pub/NeuralNet/html/._NeuralNet-bs099.html +++ b/doc/pub/NeuralNet/html/._NeuralNet-bs099.html @@ -375,12 +375,12 @@ function defined by some weights \( \vec \omega \), \( c(x, \vec \omega) \), goes as follows: $$ -\vec \omega_{\text{new} } = \vec \omega - \lambda \nabla_{\vec \omega} c(x, \vec \omega), +\vec \omega_{\mathrm{new} } = \vec \omega - \lambda \nabla_{\vec \omega} c(x, \vec \omega), $$

for a number of iterations or until $ \big|\big| \vec -\omega_{\text{new} } - \vec \omega \big|\big|$ is smaller than some +\omega_{\mathrm{new} } - \vec \omega \big|\big|$ is smaller than some given tolerance.

diff --git a/doc/pub/NeuralNet/html/._NeuralNet-bs100.html b/doc/pub/NeuralNet/html/._NeuralNet-bs100.html index 9a5050094..4d5df7946 100644 --- a/doc/pub/NeuralNet/html/._NeuralNet-bs100.html +++ b/doc/pub/NeuralNet/html/._NeuralNet-bs100.html @@ -368,17 +368,17 @@ MathJax.Hub.Config({

In our case, we have to minimize the cost function \( c(x, P) \) with respect to the two sets of weights and bisases, that is for the hidden -layer \( P_{\text{hidden} } \) and for the ouput layer \( P_{\text{output} +layer \( P_{\mathrm{hidden} } \) and for the ouput layer \( P_{\mathrm{output} } \) .

-This means that \( P_{\text{hidden} } \) and \( P_{\text{output} } \) is +This means that \( P_{\mathrm{hidden} } \) and \( P_{\mathrm{output} } \) is updated by $$ \begin{aligned} -P_{\text{hidden},\text{new}} &= P_{\text{hidden}} - \lambda \nabla_{P_{\text{hidden}}} c(x, P) \\ -P_{\text{output},\text{new}} &= P_{\text{output}} - \lambda \nabla_{P_{\text{output}}} c(x, P) +P_{\mathrm{hidden},\mathrm{new}} &= P_{\mathrm{hidden}} - \lambda \nabla_{P_{\mathrm{hidden}}} c(x, P) \\ +P_{\mathrm{output},\mathrm{new}} &= P_{\mathrm{output}} - \lambda \nabla_{P_{\mathrm{output}}} c(x, P) \end{aligned} $$ diff --git a/doc/pub/NeuralNet/html/._NeuralNet-bs101.html b/doc/pub/NeuralNet/html/._NeuralNet-bs101.html index d64565b28..39d3c2082 100644 --- a/doc/pub/NeuralNet/html/._NeuralNet-bs101.html +++ b/doc/pub/NeuralNet/html/._NeuralNet-bs101.html @@ -368,13 +368,13 @@ MathJax.Hub.Config({

As previously stated, a Deep Neural Network (DNN) follows the same concept of a neural network, but having more than one hidden -layer. Suppose that the network has \( N_{\text{hidden}} \) hidden layers -where the \( l \)-th layer has \( N_{\text{hidden}}^{(l)} \) neurons. The +layer. Suppose that the network has \( N_{\mathrm{hidden}} \) hidden layers +where the \( l \)-th layer has \( N_{\mathrm{hidden}}^{(l)} \) neurons. The input is still assumed to be an array of size \( 1 \times N \). The network must now try to optimalize its output with respect to the -collection of weigths and biases \( P = \big\{P_{\text{input} }, \ -P_{\text{hidden} }^{(1)}, \ P_{\text{hidden} }^{(2)}, \ \dots , \ -P_{\text{hidden} }^{(N_{\text{hidden}})}, \ P_{\text{output} }\big\} \). +collection of weigths and biases \( P = \big\{P_{\mathrm{input} }, \ +P_{\mathrm{hidden} }^{(1)}, \ P_{\mathrm{hidden} }^{(2)}, \ \dots , \ +P_{\mathrm{hidden} }^{(N_{\mathrm{hidden}})}, \ P_{\mathrm{output} }\big\} \).

diff --git a/doc/pub/NeuralNet/html/._NeuralNet-bs102.html b/doc/pub/NeuralNet/html/._NeuralNet-bs102.html index 8c0d86876..68b801209 100644 --- a/doc/pub/NeuralNet/html/._NeuralNet-bs102.html +++ b/doc/pub/NeuralNet/html/._NeuralNet-bs102.html @@ -370,40 +370,40 @@ The feedforward step is similar to as for the neural netowork, but now consideri

The \( i \)-th neuron at layer \( l \) recieves the result -\( \vec{x}_j^{(l-1),\text{hidden} } \) from the \( j \)-th neuron at layer +\( \hat{x}_j^{(l-1),\mathrm{hidden} } \) from the \( j \)-th neuron at layer \( l-1 \). The \( i \)-th neuron at layer \( l \) weights all of the elements in -\( \vec{x}_j^{(l-1),\text{hidden} } \) with a weight vector \( \vec -w_{i,j}^{(l), \ \text{hidden} } \) with as many weigths as there are -elements in$\vec{x}_j^{(l-1),\text{hidden} }$, and adds a bias -\( b_i^{(l), \ \text{hidden} } \): +\( \hat{x}_j^{(l-1),\mathrm{hidden} } \) with a weight vector \( \vec +w_{i,j}^{(l), \ \mathrm{hidden} } \) with as many weigths as there are +elements in$\hat{x}_j^{(l-1),\mathrm{hidden} }$, and adds a bias +\( b_i^{(l), \ \mathrm{hidden} } \): $$ \begin{aligned} -z_{i,j}^{(l),\ \text{hidden}} &= b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_j^{(l-1),\text{hidden} } \\ +z_{i,j}^{(l),\ \mathrm{hidden}} &= b_i^{(l), \ \mathrm{hidden}} + \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T\hat{x}_j^{(l-1),\mathrm{hidden} } \\ &= \begin{pmatrix} -b_i^{(l), \ \text{hidden}} & \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T +b_i^{(l), \ \mathrm{hidden}} & \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T \end{pmatrix} \begin{pmatrix} 1 \\ -\vec{x}_j^{(l-1),\text{hidden} } +\hat{x}_j^{(l-1),\mathrm{hidden} } \end{pmatrix} \end{aligned} $$

-The output from the \( i \)-th neuron at the hidden layer \( l \) becomes a vector \( \vec{z}_{i}^{(l),\ \text{hidden}} \): +The output from the \( i \)-th neuron at the hidden layer \( l \) becomes a vector \( \hat{z}_{i}^{(l),\ \mathrm{hidden}} \): $$ \begin{aligned} -\vec{z}_{i}^{(l),\ \text{hidden}} &= \Big( b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_1^{(l-1),\text{hidden} }, \ \dots \ , \ b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_{N_{hidden}^{(l-1)}}^{(l-1),\text{hidden} } \Big) \\ +\hat{z}_{i}^{(l),\ \mathrm{hidden}} &= \Big( b_i^{(l), \ \mathrm{hidden}} + \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T\hat{x}_1^{(l-1),\mathrm{hidden} }, \ \dots \ , \ b_i^{(l), \ \mathrm{hidden}} + \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T\hat{x}_{N_{hidden}^{(l-1)}}^{(l-1),\mathrm{hidden} } \Big) \\ &= \begin{pmatrix} -b_i^{(l), \ \text{hidden}} & \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T +b_i^{(l), \ \mathrm{hidden}} & \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T \end{pmatrix} \begin{pmatrix} 1 & 1 & \dots & 1 \\ -\vec{x}_{1}^{(l-1),\text{hidden} } & \vec{x}_{2}^{(l-1),\text{hidden} } & \dots & \vec{x}_{N_{hidden}^{(l-1)}}^{(l-1),\text{hidden} } +\hat{x}_{1}^{(l-1),\mathrm{hidden} } & \hat{x}_{2}^{(l-1),\mathrm{hidden} } & \dots & \hat{x}_{N_{hidden}^{(l-1)}}^{(l-1),\mathrm{hidden} } \end{pmatrix} \end{aligned} $$ diff --git a/doc/pub/NeuralNet/html/NeuralNet-reveal.html b/doc/pub/NeuralNet/html/NeuralNet-reveal.html index 6e4a8b736..da0e17bf7 100644 --- a/doc/pub/NeuralNet/html/NeuralNet-reveal.html +++ b/doc/pub/NeuralNet/html/NeuralNet-reveal.html @@ -3172,7 +3172,7 @@ depth respectively). The neurons in a layer will only be connected to a small region of the layer before it, instead of all of the neurons in a fully-connected manner. Moreover, the final -output layer could for this specific image have dimensions \( 1\times 1 times 10 \), +output layer could for this specific image have dimensions \( 1\times 1 \times 10 \), because by the end of the CNN architecture we will reduce the full image into a single vector of class scores, arranged along the depth @@ -3821,17 +3821,17 @@ and biases for each layer.

It is assumed that there are no weights and -bias at the input layer, so \( P = \{ P_{\text{hidden}}, -P_{\text{output}} \} \). If there are \( N_{\text{hidden} } \) neurons in -the hidden layer, then \( P_{\text{hidden}} \) is an \( N_{\text{hidden} } +bias at the input layer, so \( P = \{ P_{\mathrm{hidden}}, +P_{\mathrm{output}} \} \). If there are \( N_{\mathrm{hidden} } \) neurons in +the hidden layer, then \( P_{\mathrm{hidden}} \) is an \( N_{\mathrm{hidden} } \times 2 \) matrix.

-The first column in \( P_{\text{hidden} } \) represents +The first column in \( P_{\mathrm{hidden} } \) represents the bias for each neuron in the hidden layer and the second column -represents the weigths for each neuron. If there are \( N_{\text{output} -} \) neurons in the output layer, then \( P_{\text{output}} \) is a -\( N_{\text{output} } \times (1 + N_{\text{hidden} }) \) matrix. Its first +represents the weigths for each neuron. If there are \( N_{\mathrm{output} +} \) neurons in the output layer, then \( P_{\mathrm{output}} \) is a +\( N_{\mathrm{output} } \times (1 + N_{\mathrm{hidden} }) \) matrix. Its first column represents the bias of each neuron and the remaining columns represents the weights to each neuron. @@ -3914,7 +3914,7 @@ or, in terms of weights and biases for each layer:

 
$$ -\min_{P_{\text{hidden} }, \ P_{\text{output} }}\Big\{ \big(g_t'(x, \{ P_{\text{hidden} }, P_{\text{output} }\}) - ( -\gamma g_t(x, \{ P_{\text{hidden} }, P_{\text{output} }\}) \big)^2 \Big\} +\min_{P_{\mathrm{hidden} }, \ P_{\mathrm{output} }}\Big\{ \big(g_t'(x, \{ P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) - ( -\gamma g_t(x, \{ P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) \big)^2 \Big\} $$

 
@@ -3943,11 +3943,11 @@ $$

 

-or in terms of \( P_{\text{hidden} } \) and \( P_{\text{output} } \) +or in terms of \( P_{\mathrm{hidden} } \) and \( P_{\mathrm{output} } \)

 
$$ -\min_{P_{\text{hidden} }, \ P_{\text{output} }} c(x, \{P_{\text{hidden} }, P_{\text{output} }\}) +\min_{P_{\mathrm{hidden} }, \ P_{\mathrm{output} }} c(x, \{P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) $$

 
@@ -3971,8 +3971,8 @@ extension of it into a deep neural network would (hopefully) be painless.

-For simplicity, we assume that the input is an array \( \vec x = -(x_1, \dots, x_N) \) with \( N \) elements. It is at these points the neural +For simplicity, we assume that the input is an array +\( \hat{x}= (x_1, \dots, x_N) \) with \( N \) elements. It is at these points the neural network should find \( P \) such that it fulfills (19). @@ -3981,25 +3981,25 @@ network should find \( P \) such that it fulfills (19)

Feedforward

-First, a feedforward of the inputs must be done. This means that \( \vec - x \) must be passed through an input layer, a hidden layer and a output +First, a feedforward of the inputs must be done. This means that \( \hat{x} \) +must be passed through an input layer, a hidden layer and a output layer. The input layer in this case, does not need to process the - data any further. The input layer will consist of \( N_{\text{input} } \) + data any further. The input layer will consist of \( N_{\mathrm{input} } \) neurons, passing its element to each neuron in the hidden layer. The - number of neurons in the hidden layer will be \( N_{\text{hidden} } \). + number of neurons in the hidden layer will be \( N_{\mathrm{hidden} } \).

-For the \( i \)-th in the hidden layer with weight \( w_i^{\text{hidden} } \) -and bias \( b_i^{\text{hidden} } \), the weighting from the \( j \)-th neuron +For the \( i \)-th in the hidden layer with weight \( w_i^{\mathrm{hidden} } \) +and bias \( b_i^{\mathrm{hidden} } \), the weighting from the \( j \)-th neuron at the input layer is:

 
$$ \begin{aligned} -z_{i,j}^{\text{hidden}} &= b_i^{\text{hidden}} + w_i^{\text{hidden}}x_j \\ +z_{i,j}^{\mathrm{hidden}} &= b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}}x_j \\ &= \begin{pmatrix} -b_i^{\text{hidden}} & w_i^{\text{hidden}} +b_i^{\mathrm{hidden}} & w_i^{\mathrm{hidden}} \end{pmatrix} \begin{pmatrix} 1 \\ @@ -4019,29 +4019,29 @@ The result after weighting the input at the \( i \)-th hidden neuron can be writ

 
$$ \begin{aligned} -\vec{z}_{i}^{\text{hidden}} &= \Big( b_i^{\text{hidden}} + w_i^{\text{hidden}}x_1 , \ b_i^{\text{hidden}} + w_i^{\text{hidden}} x_2, \ \dots \, , \ b_i^{\text{hidden}} + w_i^{\text{hidden}} x_N\Big) \\ +\hat{z}_{i}^{\mathrm{hidden}} &= \Big( b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}}x_1 , \ b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}} x_2, \ \dots \, , \ b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}} x_N\Big) \\ &= \begin{pmatrix} - b_i^{\text{hidden}} & w_i^{\text{hidden}} + b_i^{\mathrm{hidden}} & w_i^{\mathrm{hidden}} \end{pmatrix} \begin{pmatrix} 1 & 1 & \dots & 1 \\ x_1 & x_2 & \dots & x_N \end{pmatrix} \\ -&= \vec{p}_{i, \text{hidden}}^T X +&= \hat{p}_{i, \mathrm{hidden}}^T X \end{aligned} $$

 

-It is the vector \( \vec{p}_{i, \text{hidden}}^T \) that defines each row -in \( P_{\text{hidden} } \), which contains the weights for the neural +It is the vector \( \hat{p}_{i, \mathrm{hidden}}^T \) that defines each row +in \( P_{\mathrm{hidden} } \), which contains the weights for the neural network to minimize according to (19).

-After having found \( \vec{z}_{i}^{\text{hidden}} \) for every neuron \( i \) +After having found \( \hat{z}_{i}^{\mathrm{hidden}} \) for every neuron \( i \) in the hidden layer, the vector will be sent to an activation function -\( a_i(\vec{z}) \). In this example, the sigmoid function has been used: +\( a_i(\hat{z}) \). In this example, the sigmoid function has been used:

 
$$ @@ -4055,22 +4055,22 @@ $$

Output

-The output $\vec{x}_i^{\text{hidden} }$from each \( i \)-th hidden neuron is: +The output $\hat{x}_i^{\mathrm{hidden}}$from each \( i \)-th hidden neuron is:

 
$$ -\vec{x}_i^{\text{hidden} } = f\big( \vec{z}_{i}^{\text{hidden}} \big). +\hat{x}_i^{\mathrm{hidden} } = f\big( \hat{z}_{i}^{\mathrm{hidden}} \big). $$

 

-The outputs \( \vec{x}_i^{\text{hidden} } \) are then sent to the output layer. +The outputs \( \hat{x}_i^{\mathrm{hidden} } \) are then sent to the output layer.

The output layer consist of one neuron in this case, and combines the output from each of the neurons in the hidden layers. The output layer combines the results from the hidden layer using some weights \( -w_i^{\text{output}} \) and biases \( b_i^{\text{output}} \). In this case, +w_i^{\mathrm{output}} \) and biases \( b_i^{\mathrm{output}} \). In this case, it is assumes that the number of neurons in the output layer is one.

@@ -4081,30 +4081,30 @@ layer described previously.

 
$$ \begin{aligned} -z_{1,j}^{\text{output}} & = +z_{1,j}^{\mathrm{output}} & = \begin{pmatrix} -b_1^{\text{output}} & \vec{w}_1^{\text{output}} +b_1^{\mathrm{output}} & \hat{w}_1^{\mathrm{output}} \end{pmatrix} \begin{pmatrix} 1 \\ -\vec{x}_j^{\text{hidden}} +\hat{x}_j^{\mathrm{hidden}} \end{pmatrix} \end{aligned} $$

 

-Expressing \( z_{1,j}^{\text{output}} \) as a vector gives the following procedure of weighting the inputs from the hidden layer: +Expressing \( z_{1,j}^{\mathrm{output}} \) as a vector gives the following procedure of weighting the inputs from the hidden layer:

 
$$ -\vec{z}_{1}^{\text{output}} = +\hat{z}_{1}^{\mathrm{output}} = \begin{pmatrix} -b_1^{\text{output}} & \vec{w}_1^{\text{output}} +b_1^{\mathrm{output}} & \hat{w}_1^{\mathrm{output}} \end{pmatrix} \begin{pmatrix} 1 & 1 & \dots & 1 \\ -\vec{x}_1^{\text{hidden}} & \vec{x}_2^{\text{hidden}} & \dots & \vec{x}_N^{\text{hidden}} +\hat{x}_1^{\mathrm{hidden}} & \hat{x}_2^{\mathrm{hidden}} & \dots & \hat{x}_N^{\mathrm{hidden}} \end{pmatrix} $$

 
@@ -4112,8 +4112,8 @@ $$

In this case we seek a continous range of values since we are approximating a function. This means that after computing -\( \vec{z}_{1}^{\text{output}} \) the neural network has finished its -feedforward step, and \( \vec{z}_{1}^{\text{output}} \) is the final +\( \hat{z}_{1}^{\mathrm{output}} \) the neural network has finished its +feedforward step, and \( \hat{z}_{1}^{\mathrm{output}} \) is the final output of the network. @@ -4244,13 +4244,13 @@ goes as follows:

 
$$ -\vec \omega_{\text{new} } = \vec \omega - \lambda \nabla_{\vec \omega} c(x, \vec \omega), +\vec \omega_{\mathrm{new} } = \vec \omega - \lambda \nabla_{\vec \omega} c(x, \vec \omega), $$

 

for a number of iterations or until $ \big|\big| \vec -\omega_{\text{new} } - \vec \omega \big|\big|$ is smaller than some +\omega_{\mathrm{new} } - \vec \omega \big|\big|$ is smaller than some given tolerance.

@@ -4267,18 +4267,18 @@ the elements in \( \vec \omega \).

In our case, we have to minimize the cost function \( c(x, P) \) with respect to the two sets of weights and bisases, that is for the hidden -layer \( P_{\text{hidden} } \) and for the ouput layer \( P_{\text{output} +layer \( P_{\mathrm{hidden} } \) and for the ouput layer \( P_{\mathrm{output} } \) .

-This means that \( P_{\text{hidden} } \) and \( P_{\text{output} } \) is +This means that \( P_{\mathrm{hidden} } \) and \( P_{\mathrm{output} } \) is updated by

 
$$ \begin{aligned} -P_{\text{hidden},\text{new}} &= P_{\text{hidden}} - \lambda \nabla_{P_{\text{hidden}}} c(x, P) \\ -P_{\text{output},\text{new}} &= P_{\text{output}} - \lambda \nabla_{P_{\text{output}}} c(x, P) +P_{\mathrm{hidden},\mathrm{new}} &= P_{\mathrm{hidden}} - \lambda \nabla_{P_{\mathrm{hidden}}} c(x, P) \\ +P_{\mathrm{output},\mathrm{new}} &= P_{\mathrm{output}} - \lambda \nabla_{P_{\mathrm{output}}} c(x, P) \end{aligned} $$

 
@@ -4333,13 +4333,13 @@ for finding the gradients. Luckily, Autograd comes to the rescue.

As previously stated, a Deep Neural Network (DNN) follows the same concept of a neural network, but having more than one hidden -layer. Suppose that the network has \( N_{\text{hidden}} \) hidden layers -where the \( l \)-th layer has \( N_{\text{hidden}}^{(l)} \) neurons. The +layer. Suppose that the network has \( N_{\mathrm{hidden}} \) hidden layers +where the \( l \)-th layer has \( N_{\mathrm{hidden}}^{(l)} \) neurons. The input is still assumed to be an array of size \( 1 \times N \). The network must now try to optimalize its output with respect to the -collection of weigths and biases \( P = \big\{P_{\text{input} }, \ -P_{\text{hidden} }^{(1)}, \ P_{\text{hidden} }^{(2)}, \ \dots , \ -P_{\text{hidden} }^{(N_{\text{hidden}})}, \ P_{\text{output} }\big\} \). +collection of weigths and biases \( P = \big\{P_{\mathrm{input} }, \ +P_{\mathrm{hidden} }^{(1)}, \ P_{\mathrm{hidden} }^{(2)}, \ \dots , \ +P_{\mathrm{hidden} }^{(N_{\mathrm{hidden}})}, \ P_{\mathrm{output} }\big\} \). @@ -4351,43 +4351,43 @@ The feedforward step is similar to as for the neural netowork, but now consideri

The \( i \)-th neuron at layer \( l \) recieves the result -\( \vec{x}_j^{(l-1),\text{hidden} } \) from the \( j \)-th neuron at layer +\( \hat{x}_j^{(l-1),\mathrm{hidden} } \) from the \( j \)-th neuron at layer \( l-1 \). The \( i \)-th neuron at layer \( l \) weights all of the elements in -\( \vec{x}_j^{(l-1),\text{hidden} } \) with a weight vector \( \vec -w_{i,j}^{(l), \ \text{hidden} } \) with as many weigths as there are -elements in$\vec{x}_j^{(l-1),\text{hidden} }$, and adds a bias -\( b_i^{(l), \ \text{hidden} } \): +\( \hat{x}_j^{(l-1),\mathrm{hidden} } \) with a weight vector \( \vec +w_{i,j}^{(l), \ \mathrm{hidden} } \) with as many weigths as there are +elements in$\hat{x}_j^{(l-1),\mathrm{hidden} }$, and adds a bias +\( b_i^{(l), \ \mathrm{hidden} } \):

 
$$ \begin{aligned} -z_{i,j}^{(l),\ \text{hidden}} &= b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_j^{(l-1),\text{hidden} } \\ +z_{i,j}^{(l),\ \mathrm{hidden}} &= b_i^{(l), \ \mathrm{hidden}} + \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T\hat{x}_j^{(l-1),\mathrm{hidden} } \\ &= \begin{pmatrix} -b_i^{(l), \ \text{hidden}} & \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T +b_i^{(l), \ \mathrm{hidden}} & \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T \end{pmatrix} \begin{pmatrix} 1 \\ -\vec{x}_j^{(l-1),\text{hidden} } +\hat{x}_j^{(l-1),\mathrm{hidden} } \end{pmatrix} \end{aligned} $$

 

-The output from the \( i \)-th neuron at the hidden layer \( l \) becomes a vector \( \vec{z}_{i}^{(l),\ \text{hidden}} \): +The output from the \( i \)-th neuron at the hidden layer \( l \) becomes a vector \( \hat{z}_{i}^{(l),\ \mathrm{hidden}} \):

 
$$ \begin{aligned} -\vec{z}_{i}^{(l),\ \text{hidden}} &= \Big( b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_1^{(l-1),\text{hidden} }, \ \dots \ , \ b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_{N_{hidden}^{(l-1)}}^{(l-1),\text{hidden} } \Big) \\ +\hat{z}_{i}^{(l),\ \mathrm{hidden}} &= \Big( b_i^{(l), \ \mathrm{hidden}} + \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T\hat{x}_1^{(l-1),\mathrm{hidden} }, \ \dots \ , \ b_i^{(l), \ \mathrm{hidden}} + \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T\hat{x}_{N_{hidden}^{(l-1)}}^{(l-1),\mathrm{hidden} } \Big) \\ &= \begin{pmatrix} -b_i^{(l), \ \text{hidden}} & \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T +b_i^{(l), \ \mathrm{hidden}} & \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T \end{pmatrix} \begin{pmatrix} 1 & 1 & \dots & 1 \\ -\vec{x}_{1}^{(l-1),\text{hidden} } & \vec{x}_{2}^{(l-1),\text{hidden} } & \dots & \vec{x}_{N_{hidden}^{(l-1)}}^{(l-1),\text{hidden} } +\hat{x}_{1}^{(l-1),\mathrm{hidden} } & \hat{x}_{2}^{(l-1),\mathrm{hidden} } & \dots & \hat{x}_{N_{hidden}^{(l-1)}}^{(l-1),\mathrm{hidden} } \end{pmatrix} \end{aligned} $$ diff --git a/doc/pub/NeuralNet/html/NeuralNet-solarized.html b/doc/pub/NeuralNet/html/NeuralNet-solarized.html index ca1df75cf..bfbe755ff 100644 --- a/doc/pub/NeuralNet/html/NeuralNet-solarized.html +++ b/doc/pub/NeuralNet/html/NeuralNet-solarized.html @@ -3072,7 +3072,7 @@ depth respectively). The neurons in a layer will only be connected to a small region of the layer before it, instead of all of the neurons in a fully-connected manner. Moreover, the final -output layer could for this specific image have dimensions \( 1\times 1 times 10 \), +output layer could for this specific image have dimensions \( 1\times 1 \times 10 \), because by the end of the CNN architecture we will reduce the full image into a single vector of class scores, arranged along the depth @@ -3703,17 +3703,17 @@ and biases for each layer.

It is assumed that there are no weights and -bias at the input layer, so \( P = \{ P_{\text{hidden}}, -P_{\text{output}} \} \). If there are \( N_{\text{hidden} } \) neurons in -the hidden layer, then \( P_{\text{hidden}} \) is an \( N_{\text{hidden} } +bias at the input layer, so \( P = \{ P_{\mathrm{hidden}}, +P_{\mathrm{output}} \} \). If there are \( N_{\mathrm{hidden} } \) neurons in +the hidden layer, then \( P_{\mathrm{hidden}} \) is an \( N_{\mathrm{hidden} } \times 2 \) matrix.

-The first column in \( P_{\text{hidden} } \) represents +The first column in \( P_{\mathrm{hidden} } \) represents the bias for each neuron in the hidden layer and the second column -represents the weigths for each neuron. If there are \( N_{\text{output} -} \) neurons in the output layer, then \( P_{\text{output}} \) is a -\( N_{\text{output} } \times (1 + N_{\text{hidden} }) \) matrix. Its first +represents the weigths for each neuron. If there are \( N_{\mathrm{output} +} \) neurons in the output layer, then \( P_{\mathrm{output}} \) is a +\( N_{\mathrm{output} } \times (1 + N_{\mathrm{hidden} }) \) matrix. Its first column represents the bias of each neuron and the remaining columns represents the weights to each neuron. @@ -3789,7 +3789,7 @@ $$ or, in terms of weights and biases for each layer: $$ -\min_{P_{\text{hidden} }, \ P_{\text{output} }}\Big\{ \big(g_t'(x, \{ P_{\text{hidden} }, P_{\text{output} }\}) - ( -\gamma g_t(x, \{ P_{\text{hidden} }, P_{\text{output} }\}) \big)^2 \Big\} +\min_{P_{\mathrm{hidden} }, \ P_{\mathrm{output} }}\Big\{ \big(g_t'(x, \{ P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) - ( -\gamma g_t(x, \{ P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) \big)^2 \Big\} $$

@@ -3813,10 +3813,10 @@ $$ $$

-or in terms of \( P_{\text{hidden} } \) and \( P_{\text{output} } \) +or in terms of \( P_{\mathrm{hidden} } \) and \( P_{\mathrm{output} } \) $$ -\min_{P_{\text{hidden} }, \ P_{\text{output} }} c(x, \{P_{\text{hidden} }, P_{\text{output} }\}) +\min_{P_{\mathrm{hidden} }, \ P_{\mathrm{output} }} c(x, \{P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) $$

@@ -3839,8 +3839,8 @@ extension of it into a deep neural network would (hopefully) be painless.

-For simplicity, we assume that the input is an array \( \vec x = -(x_1, \dots, x_N) \) with \( N \) elements. It is at these points the neural +For simplicity, we assume that the input is an array +\( \hat{x}= (x_1, \dots, x_N) \) with \( N \) elements. It is at these points the neural network should find \( P \) such that it fulfills \eqref{eq:min}.

@@ -3849,24 +3849,24 @@ network should find \( P \) such that it fulfills \eqref{eq:min}.

Feedforward

-First, a feedforward of the inputs must be done. This means that \( \vec - x \) must be passed through an input layer, a hidden layer and a output +First, a feedforward of the inputs must be done. This means that \( \hat{x} \) +must be passed through an input layer, a hidden layer and a output layer. The input layer in this case, does not need to process the - data any further. The input layer will consist of \( N_{\text{input} } \) + data any further. The input layer will consist of \( N_{\mathrm{input} } \) neurons, passing its element to each neuron in the hidden layer. The - number of neurons in the hidden layer will be \( N_{\text{hidden} } \). + number of neurons in the hidden layer will be \( N_{\mathrm{hidden} } \).

-For the \( i \)-th in the hidden layer with weight \( w_i^{\text{hidden} } \) -and bias \( b_i^{\text{hidden} } \), the weighting from the \( j \)-th neuron +For the \( i \)-th in the hidden layer with weight \( w_i^{\mathrm{hidden} } \) +and bias \( b_i^{\mathrm{hidden} } \), the weighting from the \( j \)-th neuron at the input layer is: $$ \begin{aligned} -z_{i,j}^{\text{hidden}} &= b_i^{\text{hidden}} + w_i^{\text{hidden}}x_j \\ +z_{i,j}^{\mathrm{hidden}} &= b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}}x_j \\ &= \begin{pmatrix} -b_i^{\text{hidden}} & w_i^{\text{hidden}} +b_i^{\mathrm{hidden}} & w_i^{\mathrm{hidden}} \end{pmatrix} \begin{pmatrix} 1 \\ @@ -3884,28 +3884,28 @@ $$ The result after weighting the input at the \( i \)-th hidden neuron can be written as a vector: $$ \begin{aligned} -\vec{z}_{i}^{\text{hidden}} &= \Big( b_i^{\text{hidden}} + w_i^{\text{hidden}}x_1 , \ b_i^{\text{hidden}} + w_i^{\text{hidden}} x_2, \ \dots \, , \ b_i^{\text{hidden}} + w_i^{\text{hidden}} x_N\Big) \\ +\hat{z}_{i}^{\mathrm{hidden}} &= \Big( b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}}x_1 , \ b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}} x_2, \ \dots \, , \ b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}} x_N\Big) \\ &= \begin{pmatrix} - b_i^{\text{hidden}} & w_i^{\text{hidden}} + b_i^{\mathrm{hidden}} & w_i^{\mathrm{hidden}} \end{pmatrix} \begin{pmatrix} 1 & 1 & \dots & 1 \\ x_1 & x_2 & \dots & x_N \end{pmatrix} \\ -&= \vec{p}_{i, \text{hidden}}^T X +&= \hat{p}_{i, \mathrm{hidden}}^T X \end{aligned} $$

-It is the vector \( \vec{p}_{i, \text{hidden}}^T \) that defines each row -in \( P_{\text{hidden} } \), which contains the weights for the neural +It is the vector \( \hat{p}_{i, \mathrm{hidden}}^T \) that defines each row +in \( P_{\mathrm{hidden} } \), which contains the weights for the neural network to minimize according to \eqref{eq:min}.

-After having found \( \vec{z}_{i}^{\text{hidden}} \) for every neuron \( i \) +After having found \( \hat{z}_{i}^{\mathrm{hidden}} \) for every neuron \( i \) in the hidden layer, the vector will be sent to an activation function -\( a_i(\vec{z}) \). In this example, the sigmoid function has been used: +\( a_i(\hat{z}) \). In this example, the sigmoid function has been used: $$ f(z) = \frac{1}{1 + \exp{(-z)}}. @@ -3917,20 +3917,20 @@ $$

Output

-The output $\vec{x}_i^{\text{hidden} }$from each \( i \)-th hidden neuron is: +The output $\hat{x}_i^{\mathrm{hidden}}$from each \( i \)-th hidden neuron is: $$ -\vec{x}_i^{\text{hidden} } = f\big( \vec{z}_{i}^{\text{hidden}} \big). +\hat{x}_i^{\mathrm{hidden} } = f\big( \hat{z}_{i}^{\mathrm{hidden}} \big). $$

-The outputs \( \vec{x}_i^{\text{hidden} } \) are then sent to the output layer. +The outputs \( \hat{x}_i^{\mathrm{hidden} } \) are then sent to the output layer.

The output layer consist of one neuron in this case, and combines the output from each of the neurons in the hidden layers. The output layer combines the results from the hidden layer using some weights \( -w_i^{\text{output}} \) and biases \( b_i^{\text{output}} \). In this case, +w_i^{\mathrm{output}} \) and biases \( b_i^{\mathrm{output}} \). In this case, it is assumes that the number of neurons in the output layer is one.

@@ -3940,36 +3940,36 @@ layer described previously. $$ \begin{aligned} -z_{1,j}^{\text{output}} & = +z_{1,j}^{\mathrm{output}} & = \begin{pmatrix} -b_1^{\text{output}} & \vec{w}_1^{\text{output}} +b_1^{\mathrm{output}} & \hat{w}_1^{\mathrm{output}} \end{pmatrix} \begin{pmatrix} 1 \\ -\vec{x}_j^{\text{hidden}} +\hat{x}_j^{\mathrm{hidden}} \end{pmatrix} \end{aligned} $$

-Expressing \( z_{1,j}^{\text{output}} \) as a vector gives the following procedure of weighting the inputs from the hidden layer: +Expressing \( z_{1,j}^{\mathrm{output}} \) as a vector gives the following procedure of weighting the inputs from the hidden layer: $$ -\vec{z}_{1}^{\text{output}} = +\hat{z}_{1}^{\mathrm{output}} = \begin{pmatrix} -b_1^{\text{output}} & \vec{w}_1^{\text{output}} +b_1^{\mathrm{output}} & \hat{w}_1^{\mathrm{output}} \end{pmatrix} \begin{pmatrix} 1 & 1 & \dots & 1 \\ -\vec{x}_1^{\text{hidden}} & \vec{x}_2^{\text{hidden}} & \dots & \vec{x}_N^{\text{hidden}} +\hat{x}_1^{\mathrm{hidden}} & \hat{x}_2^{\mathrm{hidden}} & \dots & \hat{x}_N^{\mathrm{hidden}} \end{pmatrix} $$

In this case we seek a continous range of values since we are approximating a function. This means that after computing -\( \vec{z}_{1}^{\text{output}} \) the neural network has finished its -feedforward step, and \( \vec{z}_{1}^{\text{output}} \) is the final +\( \hat{z}_{1}^{\mathrm{output}} \) the neural network has finished its +feedforward step, and \( \hat{z}_{1}^{\mathrm{output}} \) is the final output of the network.

@@ -4095,12 +4095,12 @@ function defined by some weights \( \vec \omega \), \( c(x, \vec \omega) \), goes as follows: $$ -\vec \omega_{\text{new} } = \vec \omega - \lambda \nabla_{\vec \omega} c(x, \vec \omega), +\vec \omega_{\mathrm{new} } = \vec \omega - \lambda \nabla_{\vec \omega} c(x, \vec \omega), $$

for a number of iterations or until $ \big|\big| \vec -\omega_{\text{new} } - \vec \omega \big|\big|$ is smaller than some +\omega_{\mathrm{new} } - \vec \omega \big|\big|$ is smaller than some given tolerance.

@@ -4117,17 +4117,17 @@ the elements in \( \vec \omega \).

In our case, we have to minimize the cost function \( c(x, P) \) with respect to the two sets of weights and bisases, that is for the hidden -layer \( P_{\text{hidden} } \) and for the ouput layer \( P_{\text{output} +layer \( P_{\mathrm{hidden} } \) and for the ouput layer \( P_{\mathrm{output} } \) .

-This means that \( P_{\text{hidden} } \) and \( P_{\text{output} } \) is +This means that \( P_{\mathrm{hidden} } \) and \( P_{\mathrm{output} } \) is updated by $$ \begin{aligned} -P_{\text{hidden},\text{new}} &= P_{\text{hidden}} - \lambda \nabla_{P_{\text{hidden}}} c(x, P) \\ -P_{\text{output},\text{new}} &= P_{\text{output}} - \lambda \nabla_{P_{\text{output}}} c(x, P) +P_{\mathrm{hidden},\mathrm{new}} &= P_{\mathrm{hidden}} - \lambda \nabla_{P_{\mathrm{hidden}}} c(x, P) \\ +P_{\mathrm{output},\mathrm{new}} &= P_{\mathrm{output}} - \lambda \nabla_{P_{\mathrm{output}}} c(x, P) \end{aligned} $$ @@ -4180,13 +4180,13 @@ for finding the gradients. Luckily, Autograd comes to the rescue.

As previously stated, a Deep Neural Network (DNN) follows the same concept of a neural network, but having more than one hidden -layer. Suppose that the network has \( N_{\text{hidden}} \) hidden layers -where the \( l \)-th layer has \( N_{\text{hidden}}^{(l)} \) neurons. The +layer. Suppose that the network has \( N_{\mathrm{hidden}} \) hidden layers +where the \( l \)-th layer has \( N_{\mathrm{hidden}}^{(l)} \) neurons. The input is still assumed to be an array of size \( 1 \times N \). The network must now try to optimalize its output with respect to the -collection of weigths and biases \( P = \big\{P_{\text{input} }, \ -P_{\text{hidden} }^{(1)}, \ P_{\text{hidden} }^{(2)}, \ \dots , \ -P_{\text{hidden} }^{(N_{\text{hidden}})}, \ P_{\text{output} }\big\} \). +collection of weigths and biases \( P = \big\{P_{\mathrm{input} }, \ +P_{\mathrm{hidden} }^{(1)}, \ P_{\mathrm{hidden} }^{(2)}, \ \dots , \ +P_{\mathrm{hidden} }^{(N_{\mathrm{hidden}})}, \ P_{\mathrm{output} }\big\} \).











@@ -4198,40 +4198,40 @@ The feedforward step is similar to as for the neural netowork, but now consideri

The \( i \)-th neuron at layer \( l \) recieves the result -\( \vec{x}_j^{(l-1),\text{hidden} } \) from the \( j \)-th neuron at layer +\( \hat{x}_j^{(l-1),\mathrm{hidden} } \) from the \( j \)-th neuron at layer \( l-1 \). The \( i \)-th neuron at layer \( l \) weights all of the elements in -\( \vec{x}_j^{(l-1),\text{hidden} } \) with a weight vector \( \vec -w_{i,j}^{(l), \ \text{hidden} } \) with as many weigths as there are -elements in$\vec{x}_j^{(l-1),\text{hidden} }$, and adds a bias -\( b_i^{(l), \ \text{hidden} } \): +\( \hat{x}_j^{(l-1),\mathrm{hidden} } \) with a weight vector \( \vec +w_{i,j}^{(l), \ \mathrm{hidden} } \) with as many weigths as there are +elements in$\hat{x}_j^{(l-1),\mathrm{hidden} }$, and adds a bias +\( b_i^{(l), \ \mathrm{hidden} } \): $$ \begin{aligned} -z_{i,j}^{(l),\ \text{hidden}} &= b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_j^{(l-1),\text{hidden} } \\ +z_{i,j}^{(l),\ \mathrm{hidden}} &= b_i^{(l), \ \mathrm{hidden}} + \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T\hat{x}_j^{(l-1),\mathrm{hidden} } \\ &= \begin{pmatrix} -b_i^{(l), \ \text{hidden}} & \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T +b_i^{(l), \ \mathrm{hidden}} & \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T \end{pmatrix} \begin{pmatrix} 1 \\ -\vec{x}_j^{(l-1),\text{hidden} } +\hat{x}_j^{(l-1),\mathrm{hidden} } \end{pmatrix} \end{aligned} $$

-The output from the \( i \)-th neuron at the hidden layer \( l \) becomes a vector \( \vec{z}_{i}^{(l),\ \text{hidden}} \): +The output from the \( i \)-th neuron at the hidden layer \( l \) becomes a vector \( \hat{z}_{i}^{(l),\ \mathrm{hidden}} \): $$ \begin{aligned} -\vec{z}_{i}^{(l),\ \text{hidden}} &= \Big( b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_1^{(l-1),\text{hidden} }, \ \dots \ , \ b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_{N_{hidden}^{(l-1)}}^{(l-1),\text{hidden} } \Big) \\ +\hat{z}_{i}^{(l),\ \mathrm{hidden}} &= \Big( b_i^{(l), \ \mathrm{hidden}} + \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T\hat{x}_1^{(l-1),\mathrm{hidden} }, \ \dots \ , \ b_i^{(l), \ \mathrm{hidden}} + \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T\hat{x}_{N_{hidden}^{(l-1)}}^{(l-1),\mathrm{hidden} } \Big) \\ &= \begin{pmatrix} -b_i^{(l), \ \text{hidden}} & \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T +b_i^{(l), \ \mathrm{hidden}} & \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T \end{pmatrix} \begin{pmatrix} 1 & 1 & \dots & 1 \\ -\vec{x}_{1}^{(l-1),\text{hidden} } & \vec{x}_{2}^{(l-1),\text{hidden} } & \dots & \vec{x}_{N_{hidden}^{(l-1)}}^{(l-1),\text{hidden} } +\hat{x}_{1}^{(l-1),\mathrm{hidden} } & \hat{x}_{2}^{(l-1),\mathrm{hidden} } & \dots & \hat{x}_{N_{hidden}^{(l-1)}}^{(l-1),\mathrm{hidden} } \end{pmatrix} \end{aligned} $$ diff --git a/doc/pub/NeuralNet/html/NeuralNet.html b/doc/pub/NeuralNet/html/NeuralNet.html index 680938c01..e4b6df910 100644 --- a/doc/pub/NeuralNet/html/NeuralNet.html +++ b/doc/pub/NeuralNet/html/NeuralNet.html @@ -3077,7 +3077,7 @@ depth respectively). The neurons in a layer will only be connected to a small region of the layer before it, instead of all of the neurons in a fully-connected manner. Moreover, the final -output layer could for this specific image have dimensions \( 1\times 1 times 10 \), +output layer could for this specific image have dimensions \( 1\times 1 \times 10 \), because by the end of the CNN architecture we will reduce the full image into a single vector of class scores, arranged along the depth @@ -3708,17 +3708,17 @@ and biases for each layer.

It is assumed that there are no weights and -bias at the input layer, so \( P = \{ P_{\text{hidden}}, -P_{\text{output}} \} \). If there are \( N_{\text{hidden} } \) neurons in -the hidden layer, then \( P_{\text{hidden}} \) is an \( N_{\text{hidden} } +bias at the input layer, so \( P = \{ P_{\mathrm{hidden}}, +P_{\mathrm{output}} \} \). If there are \( N_{\mathrm{hidden} } \) neurons in +the hidden layer, then \( P_{\mathrm{hidden}} \) is an \( N_{\mathrm{hidden} } \times 2 \) matrix.

-The first column in \( P_{\text{hidden} } \) represents +The first column in \( P_{\mathrm{hidden} } \) represents the bias for each neuron in the hidden layer and the second column -represents the weigths for each neuron. If there are \( N_{\text{output} -} \) neurons in the output layer, then \( P_{\text{output}} \) is a -\( N_{\text{output} } \times (1 + N_{\text{hidden} }) \) matrix. Its first +represents the weigths for each neuron. If there are \( N_{\mathrm{output} +} \) neurons in the output layer, then \( P_{\mathrm{output}} \) is a +\( N_{\mathrm{output} } \times (1 + N_{\mathrm{hidden} }) \) matrix. Its first column represents the bias of each neuron and the remaining columns represents the weights to each neuron. @@ -3794,7 +3794,7 @@ $$ or, in terms of weights and biases for each layer: $$ -\min_{P_{\text{hidden} }, \ P_{\text{output} }}\Big\{ \big(g_t'(x, \{ P_{\text{hidden} }, P_{\text{output} }\}) - ( -\gamma g_t(x, \{ P_{\text{hidden} }, P_{\text{output} }\}) \big)^2 \Big\} +\min_{P_{\mathrm{hidden} }, \ P_{\mathrm{output} }}\Big\{ \big(g_t'(x, \{ P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) - ( -\gamma g_t(x, \{ P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) \big)^2 \Big\} $$

@@ -3818,10 +3818,10 @@ $$ $$

-or in terms of \( P_{\text{hidden} } \) and \( P_{\text{output} } \) +or in terms of \( P_{\mathrm{hidden} } \) and \( P_{\mathrm{output} } \) $$ -\min_{P_{\text{hidden} }, \ P_{\text{output} }} c(x, \{P_{\text{hidden} }, P_{\text{output} }\}) +\min_{P_{\mathrm{hidden} }, \ P_{\mathrm{output} }} c(x, \{P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) $$

@@ -3844,8 +3844,8 @@ extension of it into a deep neural network would (hopefully) be painless.

-For simplicity, we assume that the input is an array \( \vec x = -(x_1, \dots, x_N) \) with \( N \) elements. It is at these points the neural +For simplicity, we assume that the input is an array +\( \hat{x}= (x_1, \dots, x_N) \) with \( N \) elements. It is at these points the neural network should find \( P \) such that it fulfills \eqref{eq:min}.

@@ -3854,24 +3854,24 @@ network should find \( P \) such that it fulfills \eqref{eq:min}.

Feedforward

-First, a feedforward of the inputs must be done. This means that \( \vec - x \) must be passed through an input layer, a hidden layer and a output +First, a feedforward of the inputs must be done. This means that \( \hat{x} \) +must be passed through an input layer, a hidden layer and a output layer. The input layer in this case, does not need to process the - data any further. The input layer will consist of \( N_{\text{input} } \) + data any further. The input layer will consist of \( N_{\mathrm{input} } \) neurons, passing its element to each neuron in the hidden layer. The - number of neurons in the hidden layer will be \( N_{\text{hidden} } \). + number of neurons in the hidden layer will be \( N_{\mathrm{hidden} } \).

-For the \( i \)-th in the hidden layer with weight \( w_i^{\text{hidden} } \) -and bias \( b_i^{\text{hidden} } \), the weighting from the \( j \)-th neuron +For the \( i \)-th in the hidden layer with weight \( w_i^{\mathrm{hidden} } \) +and bias \( b_i^{\mathrm{hidden} } \), the weighting from the \( j \)-th neuron at the input layer is: $$ \begin{aligned} -z_{i,j}^{\text{hidden}} &= b_i^{\text{hidden}} + w_i^{\text{hidden}}x_j \\ +z_{i,j}^{\mathrm{hidden}} &= b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}}x_j \\ &= \begin{pmatrix} -b_i^{\text{hidden}} & w_i^{\text{hidden}} +b_i^{\mathrm{hidden}} & w_i^{\mathrm{hidden}} \end{pmatrix} \begin{pmatrix} 1 \\ @@ -3889,28 +3889,28 @@ $$ The result after weighting the input at the \( i \)-th hidden neuron can be written as a vector: $$ \begin{aligned} -\vec{z}_{i}^{\text{hidden}} &= \Big( b_i^{\text{hidden}} + w_i^{\text{hidden}}x_1 , \ b_i^{\text{hidden}} + w_i^{\text{hidden}} x_2, \ \dots \, , \ b_i^{\text{hidden}} + w_i^{\text{hidden}} x_N\Big) \\ +\hat{z}_{i}^{\mathrm{hidden}} &= \Big( b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}}x_1 , \ b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}} x_2, \ \dots \, , \ b_i^{\mathrm{hidden}} + w_i^{\mathrm{hidden}} x_N\Big) \\ &= \begin{pmatrix} - b_i^{\text{hidden}} & w_i^{\text{hidden}} + b_i^{\mathrm{hidden}} & w_i^{\mathrm{hidden}} \end{pmatrix} \begin{pmatrix} 1 & 1 & \dots & 1 \\ x_1 & x_2 & \dots & x_N \end{pmatrix} \\ -&= \vec{p}_{i, \text{hidden}}^T X +&= \hat{p}_{i, \mathrm{hidden}}^T X \end{aligned} $$

-It is the vector \( \vec{p}_{i, \text{hidden}}^T \) that defines each row -in \( P_{\text{hidden} } \), which contains the weights for the neural +It is the vector \( \hat{p}_{i, \mathrm{hidden}}^T \) that defines each row +in \( P_{\mathrm{hidden} } \), which contains the weights for the neural network to minimize according to \eqref{eq:min}.

-After having found \( \vec{z}_{i}^{\text{hidden}} \) for every neuron \( i \) +After having found \( \hat{z}_{i}^{\mathrm{hidden}} \) for every neuron \( i \) in the hidden layer, the vector will be sent to an activation function -\( a_i(\vec{z}) \). In this example, the sigmoid function has been used: +\( a_i(\hat{z}) \). In this example, the sigmoid function has been used: $$ f(z) = \frac{1}{1 + \exp{(-z)}}. @@ -3922,20 +3922,20 @@ $$

Output

-The output $\vec{x}_i^{\text{hidden} }$from each \( i \)-th hidden neuron is: +The output $\hat{x}_i^{\mathrm{hidden}}$from each \( i \)-th hidden neuron is: $$ -\vec{x}_i^{\text{hidden} } = f\big( \vec{z}_{i}^{\text{hidden}} \big). +\hat{x}_i^{\mathrm{hidden} } = f\big( \hat{z}_{i}^{\mathrm{hidden}} \big). $$

-The outputs \( \vec{x}_i^{\text{hidden} } \) are then sent to the output layer. +The outputs \( \hat{x}_i^{\mathrm{hidden} } \) are then sent to the output layer.

The output layer consist of one neuron in this case, and combines the output from each of the neurons in the hidden layers. The output layer combines the results from the hidden layer using some weights \( -w_i^{\text{output}} \) and biases \( b_i^{\text{output}} \). In this case, +w_i^{\mathrm{output}} \) and biases \( b_i^{\mathrm{output}} \). In this case, it is assumes that the number of neurons in the output layer is one.

@@ -3945,36 +3945,36 @@ layer described previously. $$ \begin{aligned} -z_{1,j}^{\text{output}} & = +z_{1,j}^{\mathrm{output}} & = \begin{pmatrix} -b_1^{\text{output}} & \vec{w}_1^{\text{output}} +b_1^{\mathrm{output}} & \hat{w}_1^{\mathrm{output}} \end{pmatrix} \begin{pmatrix} 1 \\ -\vec{x}_j^{\text{hidden}} +\hat{x}_j^{\mathrm{hidden}} \end{pmatrix} \end{aligned} $$

-Expressing \( z_{1,j}^{\text{output}} \) as a vector gives the following procedure of weighting the inputs from the hidden layer: +Expressing \( z_{1,j}^{\mathrm{output}} \) as a vector gives the following procedure of weighting the inputs from the hidden layer: $$ -\vec{z}_{1}^{\text{output}} = +\hat{z}_{1}^{\mathrm{output}} = \begin{pmatrix} -b_1^{\text{output}} & \vec{w}_1^{\text{output}} +b_1^{\mathrm{output}} & \hat{w}_1^{\mathrm{output}} \end{pmatrix} \begin{pmatrix} 1 & 1 & \dots & 1 \\ -\vec{x}_1^{\text{hidden}} & \vec{x}_2^{\text{hidden}} & \dots & \vec{x}_N^{\text{hidden}} +\hat{x}_1^{\mathrm{hidden}} & \hat{x}_2^{\mathrm{hidden}} & \dots & \hat{x}_N^{\mathrm{hidden}} \end{pmatrix} $$

In this case we seek a continous range of values since we are approximating a function. This means that after computing -\( \vec{z}_{1}^{\text{output}} \) the neural network has finished its -feedforward step, and \( \vec{z}_{1}^{\text{output}} \) is the final +\( \hat{z}_{1}^{\mathrm{output}} \) the neural network has finished its +feedforward step, and \( \hat{z}_{1}^{\mathrm{output}} \) is the final output of the network.

@@ -4100,12 +4100,12 @@ function defined by some weights \( \vec \omega \), \( c(x, \vec \omega) \), goes as follows: $$ -\vec \omega_{\text{new} } = \vec \omega - \lambda \nabla_{\vec \omega} c(x, \vec \omega), +\vec \omega_{\mathrm{new} } = \vec \omega - \lambda \nabla_{\vec \omega} c(x, \vec \omega), $$

for a number of iterations or until $ \big|\big| \vec -\omega_{\text{new} } - \vec \omega \big|\big|$ is smaller than some +\omega_{\mathrm{new} } - \vec \omega \big|\big|$ is smaller than some given tolerance.

@@ -4122,17 +4122,17 @@ the elements in \( \vec \omega \).

In our case, we have to minimize the cost function \( c(x, P) \) with respect to the two sets of weights and bisases, that is for the hidden -layer \( P_{\text{hidden} } \) and for the ouput layer \( P_{\text{output} +layer \( P_{\mathrm{hidden} } \) and for the ouput layer \( P_{\mathrm{output} } \) .

-This means that \( P_{\text{hidden} } \) and \( P_{\text{output} } \) is +This means that \( P_{\mathrm{hidden} } \) and \( P_{\mathrm{output} } \) is updated by $$ \begin{aligned} -P_{\text{hidden},\text{new}} &= P_{\text{hidden}} - \lambda \nabla_{P_{\text{hidden}}} c(x, P) \\ -P_{\text{output},\text{new}} &= P_{\text{output}} - \lambda \nabla_{P_{\text{output}}} c(x, P) +P_{\mathrm{hidden},\mathrm{new}} &= P_{\mathrm{hidden}} - \lambda \nabla_{P_{\mathrm{hidden}}} c(x, P) \\ +P_{\mathrm{output},\mathrm{new}} &= P_{\mathrm{output}} - \lambda \nabla_{P_{\mathrm{output}}} c(x, P) \end{aligned} $$ @@ -4185,13 +4185,13 @@ for finding the gradients. Luckily, Autograd comes to the rescue.

As previously stated, a Deep Neural Network (DNN) follows the same concept of a neural network, but having more than one hidden -layer. Suppose that the network has \( N_{\text{hidden}} \) hidden layers -where the \( l \)-th layer has \( N_{\text{hidden}}^{(l)} \) neurons. The +layer. Suppose that the network has \( N_{\mathrm{hidden}} \) hidden layers +where the \( l \)-th layer has \( N_{\mathrm{hidden}}^{(l)} \) neurons. The input is still assumed to be an array of size \( 1 \times N \). The network must now try to optimalize its output with respect to the -collection of weigths and biases \( P = \big\{P_{\text{input} }, \ -P_{\text{hidden} }^{(1)}, \ P_{\text{hidden} }^{(2)}, \ \dots , \ -P_{\text{hidden} }^{(N_{\text{hidden}})}, \ P_{\text{output} }\big\} \). +collection of weigths and biases \( P = \big\{P_{\mathrm{input} }, \ +P_{\mathrm{hidden} }^{(1)}, \ P_{\mathrm{hidden} }^{(2)}, \ \dots , \ +P_{\mathrm{hidden} }^{(N_{\mathrm{hidden}})}, \ P_{\mathrm{output} }\big\} \).











@@ -4203,40 +4203,40 @@ The feedforward step is similar to as for the neural netowork, but now consideri

The \( i \)-th neuron at layer \( l \) recieves the result -\( \vec{x}_j^{(l-1),\text{hidden} } \) from the \( j \)-th neuron at layer +\( \hat{x}_j^{(l-1),\mathrm{hidden} } \) from the \( j \)-th neuron at layer \( l-1 \). The \( i \)-th neuron at layer \( l \) weights all of the elements in -\( \vec{x}_j^{(l-1),\text{hidden} } \) with a weight vector \( \vec -w_{i,j}^{(l), \ \text{hidden} } \) with as many weigths as there are -elements in$\vec{x}_j^{(l-1),\text{hidden} }$, and adds a bias -\( b_i^{(l), \ \text{hidden} } \): +\( \hat{x}_j^{(l-1),\mathrm{hidden} } \) with a weight vector \( \vec +w_{i,j}^{(l), \ \mathrm{hidden} } \) with as many weigths as there are +elements in$\hat{x}_j^{(l-1),\mathrm{hidden} }$, and adds a bias +\( b_i^{(l), \ \mathrm{hidden} } \): $$ \begin{aligned} -z_{i,j}^{(l),\ \text{hidden}} &= b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_j^{(l-1),\text{hidden} } \\ +z_{i,j}^{(l),\ \mathrm{hidden}} &= b_i^{(l), \ \mathrm{hidden}} + \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T\hat{x}_j^{(l-1),\mathrm{hidden} } \\ &= \begin{pmatrix} -b_i^{(l), \ \text{hidden}} & \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T +b_i^{(l), \ \mathrm{hidden}} & \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T \end{pmatrix} \begin{pmatrix} 1 \\ -\vec{x}_j^{(l-1),\text{hidden} } +\hat{x}_j^{(l-1),\mathrm{hidden} } \end{pmatrix} \end{aligned} $$

-The output from the \( i \)-th neuron at the hidden layer \( l \) becomes a vector \( \vec{z}_{i}^{(l),\ \text{hidden}} \): +The output from the \( i \)-th neuron at the hidden layer \( l \) becomes a vector \( \hat{z}_{i}^{(l),\ \mathrm{hidden}} \): $$ \begin{aligned} -\vec{z}_{i}^{(l),\ \text{hidden}} &= \Big( b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_1^{(l-1),\text{hidden} }, \ \dots \ , \ b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_{N_{hidden}^{(l-1)}}^{(l-1),\text{hidden} } \Big) \\ +\hat{z}_{i}^{(l),\ \mathrm{hidden}} &= \Big( b_i^{(l), \ \mathrm{hidden}} + \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T\hat{x}_1^{(l-1),\mathrm{hidden} }, \ \dots \ , \ b_i^{(l), \ \mathrm{hidden}} + \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T\hat{x}_{N_{hidden}^{(l-1)}}^{(l-1),\mathrm{hidden} } \Big) \\ &= \begin{pmatrix} -b_i^{(l), \ \text{hidden}} & \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T +b_i^{(l), \ \mathrm{hidden}} & \big(\hat{w}_{i}^{(l), \ \mathrm{hidden}}\big)^T \end{pmatrix} \begin{pmatrix} 1 & 1 & \dots & 1 \\ -\vec{x}_{1}^{(l-1),\text{hidden} } & \vec{x}_{2}^{(l-1),\text{hidden} } & \dots & \vec{x}_{N_{hidden}^{(l-1)}}^{(l-1),\text{hidden} } +\hat{x}_{1}^{(l-1),\mathrm{hidden} } & \hat{x}_{2}^{(l-1),\mathrm{hidden} } & \dots & \hat{x}_{N_{hidden}^{(l-1)}}^{(l-1),\mathrm{hidden} } \end{pmatrix} \end{aligned} $$ diff --git a/doc/pub/NeuralNet/ipynb/NeuralNet.ipynb b/doc/pub/NeuralNet/ipynb/NeuralNet.ipynb index 61d8d5658..07dc89694 100644 --- a/doc/pub/NeuralNet/ipynb/NeuralNet.ipynb +++ b/doc/pub/NeuralNet/ipynb/NeuralNet.ipynb @@ -3378,7 +3378,7 @@ "The neurons in a layer will\n", "only be connected to a small region of the layer before it, instead of\n", "all of the neurons in a fully-connected manner. Moreover, the final\n", - "output layer could for this specific image have dimensions $1\\times 1 times 10$, \n", + "output layer could for this specific image have dimensions $1\\times 1 \\times 10$, \n", "because by the\n", "end of the CNN architecture we will reduce the full image into a\n", "single vector of class scores, arranged along the depth\n", @@ -4071,16 +4071,16 @@ "and biases for each layer. \n", "\n", "It is assumed that there are no weights and\n", - "bias at the input layer, so $P = \\{ P_{\\text{hidden}},\n", - "P_{\\text{output}} \\}$. If there are $N_{\\text{hidden} }$ neurons in\n", - "the hidden layer, then $P_{\\text{hidden}}$ is an $N_{\\text{hidden} }\n", + "bias at the input layer, so $P = \\{ P_{\\mathrm{hidden}},\n", + "P_{\\mathrm{output}} \\}$. If there are $N_{\\mathrm{hidden} }$ neurons in\n", + "the hidden layer, then $P_{\\mathrm{hidden}}$ is an $N_{\\mathrm{hidden} }\n", "\\times 2$ matrix. \n", "\n", - "The first column in $P_{\\text{hidden} }$ represents\n", + "The first column in $P_{\\mathrm{hidden} }$ represents\n", "the bias for each neuron in the hidden layer and the second column\n", - "represents the weigths for each neuron. If there are $N_{\\text{output}\n", - "}$ neurons in the output layer, then $P_{\\text{output}} $ is a\n", - "$N_{\\text{output} } \\times (1 + N_{\\text{hidden} })$ matrix. Its first\n", + "represents the weigths for each neuron. If there are $N_{\\mathrm{output}\n", + "}$ neurons in the output layer, then $P_{\\mathrm{output}} $ is a\n", + "$N_{\\mathrm{output} } \\times (1 + N_{\\mathrm{hidden} })$ matrix. Its first\n", "column represents the bias of each neuron and the remaining columns\n", "represents the weights to each neuron.\n", "\n", @@ -4183,7 +4183,7 @@ "metadata": {}, "source": [ "$$\n", - "\\min_{P_{\\text{hidden} }, \\ P_{\\text{output} }}\\Big\\{ \\big(g_t'(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) - ( -\\gamma g_t(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) \\big)^2 \\Big\\}\n", + "\\min_{P_{\\mathrm{hidden} }, \\ P_{\\mathrm{output} }}\\Big\\{ \\big(g_t'(x, \\{ P_{\\mathrm{hidden} }, P_{\\mathrm{output} }\\}) - ( -\\gamma g_t(x, \\{ P_{\\mathrm{hidden} }, P_{\\mathrm{output} }\\}) \\big)^2 \\Big\\}\n", "$$" ] }, @@ -4232,7 +4232,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "or in terms of $P_{\\text{hidden} }$ and $P_{\\text{output} }$" + "or in terms of $P_{\\mathrm{hidden} }$ and $P_{\\mathrm{output} }$" ] }, { @@ -4240,7 +4240,7 @@ "metadata": {}, "source": [ "$$\n", - "\\min_{P_{\\text{hidden} }, \\ P_{\\text{output} }} c(x, \\{P_{\\text{hidden} }, P_{\\text{output} }\\})\n", + "\\min_{P_{\\mathrm{hidden} }, \\ P_{\\mathrm{output} }} c(x, \\{P_{\\mathrm{hidden} }, P_{\\mathrm{output} }\\})\n", "$$" ] }, @@ -4263,21 +4263,21 @@ "extension of it into a deep neural network would (hopefully) be\n", "painless.\n", "\n", - "For simplicity, we assume that the input is an array $\\vec x =\n", - "(x_1, \\dots, x_N)$ with $N$ elements. It is at these points the neural\n", + "For simplicity, we assume that the input is an array \n", + "$\\hat{x}= (x_1, \\dots, x_N)$ with $N$ elements. It is at these points the neural\n", "network should find $P$ such that it fulfills ([eq:min](#eq:min)).\n", "\n", "## Feedforward\n", "\n", - "First, a feedforward of the inputs must be done. This means that $\\vec\n", - " x$ must be passed through an input layer, a hidden layer and a output\n", + "First, a feedforward of the inputs must be done. This means that $\\hat{x}$ \n", + "must be passed through an input layer, a hidden layer and a output\n", " layer. The input layer in this case, does not need to process the\n", - " data any further. The input layer will consist of $N_{\\text{input} }$\n", + " data any further. The input layer will consist of $N_{\\mathrm{input} }$\n", " neurons, passing its element to each neuron in the hidden layer. The\n", - " number of neurons in the hidden layer will be $N_{\\text{hidden} }$.\n", + " number of neurons in the hidden layer will be $N_{\\mathrm{hidden} }$.\n", "\n", - "For the $i$-th in the hidden layer with weight $w_i^{\\text{hidden} }$\n", - "and bias $b_i^{\\text{hidden} }$, the weighting from the $j$-th neuron\n", + "For the $i$-th in the hidden layer with weight $w_i^{\\mathrm{hidden} }$\n", + "and bias $b_i^{\\mathrm{hidden} }$, the weighting from the $j$-th neuron\n", "at the input layer is:" ] }, @@ -4287,10 +4287,10 @@ "source": [ "$$\n", "\\begin{aligned}\n", - "z_{i,j}^{\\text{hidden}} &= b_i^{\\text{hidden}} + w_i^{\\text{hidden}}x_j \\\\\n", + "z_{i,j}^{\\mathrm{hidden}} &= b_i^{\\mathrm{hidden}} + w_i^{\\mathrm{hidden}}x_j \\\\\n", "&= \n", "\\begin{pmatrix}\n", - "b_i^{\\text{hidden}} & w_i^{\\text{hidden}}\n", + "b_i^{\\mathrm{hidden}} & w_i^{\\mathrm{hidden}}\n", "\\end{pmatrix}\n", "\\begin{pmatrix}\n", "1 \\\\\n", @@ -4316,16 +4316,16 @@ "source": [ "$$\n", "\\begin{aligned}\n", - "\\vec{z}_{i}^{\\text{hidden}} &= \\Big( b_i^{\\text{hidden}} + w_i^{\\text{hidden}}x_1 , \\ b_i^{\\text{hidden}} + w_i^{\\text{hidden}} x_2, \\ \\dots \\, , \\ b_i^{\\text{hidden}} + w_i^{\\text{hidden}} x_N\\Big) \\\\\n", + "\\hat{z}_{i}^{\\mathrm{hidden}} &= \\Big( b_i^{\\mathrm{hidden}} + w_i^{\\mathrm{hidden}}x_1 , \\ b_i^{\\mathrm{hidden}} + w_i^{\\mathrm{hidden}} x_2, \\ \\dots \\, , \\ b_i^{\\mathrm{hidden}} + w_i^{\\mathrm{hidden}} x_N\\Big) \\\\\n", "&= \n", "\\begin{pmatrix}\n", - " b_i^{\\text{hidden}} & w_i^{\\text{hidden}}\n", + " b_i^{\\mathrm{hidden}} & w_i^{\\mathrm{hidden}}\n", "\\end{pmatrix}\n", "\\begin{pmatrix}\n", "1 & 1 & \\dots & 1 \\\\\n", "x_1 & x_2 & \\dots & x_N\n", "\\end{pmatrix} \\\\\n", - "&= \\vec{p}_{i, \\text{hidden}}^T X\n", + "&= \\hat{p}_{i, \\mathrm{hidden}}^T X\n", "\\end{aligned}\n", "$$" ] @@ -4334,13 +4334,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "It is the vector $\\vec{p}_{i, \\text{hidden}}^T$ that defines each row\n", - "in $P_{\\text{hidden} }$, which contains the weights for the neural\n", + "It is the vector $\\hat{p}_{i, \\mathrm{hidden}}^T$ that defines each row\n", + "in $P_{\\mathrm{hidden} }$, which contains the weights for the neural\n", "network to minimize according to ([eq:min](#eq:min)).\n", "\n", - "After having found $\\vec{z}_{i}^{\\text{hidden}} $ for every neuron $i$\n", + "After having found $\\hat{z}_{i}^{\\mathrm{hidden}} $ for every neuron $i$\n", "in the hidden layer, the vector will be sent to an activation function\n", - "$a_i(\\vec{z})$. In this example, the sigmoid function has been used:\n", + "$a_i(\\hat{z})$. In this example, the sigmoid function has been used:\n", "\n", "$$\n", "f(z) = \\frac{1}{1 + \\exp{(-z)}}.\n", @@ -4349,18 +4349,18 @@ "\n", "## Output\n", "\n", - "The output $\\vec{x}_i^{\\text{hidden} }$from each $i$-th hidden neuron is:\n", + "The output $\\hat{x}_i^{\\mathrm{hidden}}$from each $i$-th hidden neuron is:\n", "\n", "$$\n", - "\\vec{x}_i^{\\text{hidden} } = f\\big( \\vec{z}_{i}^{\\text{hidden}} \\big).\n", + "\\hat{x}_i^{\\mathrm{hidden} } = f\\big( \\hat{z}_{i}^{\\mathrm{hidden}} \\big).\n", "$$\n", "\n", - "The outputs $\\vec{x}_i^{\\text{hidden} } $ are then sent to the output layer. \n", + "The outputs $\\hat{x}_i^{\\mathrm{hidden} } $ are then sent to the output layer. \n", "\n", "The output layer consist of one neuron in this case, and combines the\n", "output from each of the neurons in the hidden layers. The output layer\n", "combines the results from the hidden layer using some weights $\n", - "w_i^{\\text{output}}$ and biases $b_i^{\\text{output}}$. In this case,\n", + "w_i^{\\mathrm{output}}$ and biases $b_i^{\\mathrm{output}}$. In this case,\n", "it is assumes that the number of neurons in the output layer is one.\n", "\n", "The procedure of weigthing the output neuron $j$ in the hidden layer\n", @@ -4374,13 +4374,13 @@ "source": [ "$$\n", "\\begin{aligned}\n", - "z_{1,j}^{\\text{output}} & = \n", + "z_{1,j}^{\\mathrm{output}} & = \n", "\\begin{pmatrix}\n", - "b_1^{\\text{output}} & \\vec{w}_1^{\\text{output}}\n", + "b_1^{\\mathrm{output}} & \\hat{w}_1^{\\mathrm{output}}\n", "\\end{pmatrix}\n", "\\begin{pmatrix}\n", "1 \\\\\n", - "\\vec{x}_j^{\\text{hidden}}\n", + "\\hat{x}_j^{\\mathrm{hidden}}\n", "\\end{pmatrix}\n", "\\end{aligned}\n", "$$" @@ -4390,23 +4390,23 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Expressing $z_{1,j}^{\\text{output}}$ as a vector gives the following procedure of weighting the inputs from the hidden layer:\n", + "Expressing $z_{1,j}^{\\mathrm{output}}$ as a vector gives the following procedure of weighting the inputs from the hidden layer:\n", "\n", "$$\n", - "\\vec{z}_{1}^{\\text{output}} = \n", + "\\hat{z}_{1}^{\\mathrm{output}} = \n", "\\begin{pmatrix}\n", - "b_1^{\\text{output}} & \\vec{w}_1^{\\text{output}}\n", + "b_1^{\\mathrm{output}} & \\hat{w}_1^{\\mathrm{output}}\n", "\\end{pmatrix}\n", "\\begin{pmatrix}\n", "1 & 1 & \\dots & 1 \\\\\n", - "\\vec{x}_1^{\\text{hidden}} & \\vec{x}_2^{\\text{hidden}} & \\dots & \\vec{x}_N^{\\text{hidden}}\n", + "\\hat{x}_1^{\\mathrm{hidden}} & \\hat{x}_2^{\\mathrm{hidden}} & \\dots & \\hat{x}_N^{\\mathrm{hidden}}\n", "\\end{pmatrix}\n", "$$\n", "\n", "In this case we seek a continous range of values since we are\n", "approximating a function. This means that after computing\n", - "$\\vec{z}_{1}^{\\text{output}}$ the neural network has finished its\n", - "feedforward step, and $\\vec{z}_{1}^{\\text{output}}$ is the final\n", + "$\\hat{z}_{1}^{\\mathrm{output}}$ the neural network has finished its\n", + "feedforward step, and $\\hat{z}_{1}^{\\mathrm{output}}$ is the final\n", "output of the network.\n", "\n", "\n", @@ -4555,7 +4555,7 @@ "metadata": {}, "source": [ "$$\n", - "\\vec \\omega_{\\text{new} } = \\vec \\omega - \\lambda \\nabla_{\\vec \\omega} c(x, \\vec \\omega),\n", + "\\vec \\omega_{\\mathrm{new} } = \\vec \\omega - \\lambda \\nabla_{\\vec \\omega} c(x, \\vec \\omega),\n", "$$" ] }, @@ -4564,7 +4564,7 @@ "metadata": {}, "source": [ "for a number of iterations or until $ \\big|\\big| \\vec\n", - "\\omega_{\\text{new} } - \\vec \\omega \\big|\\big|$ is smaller than some\n", + "\\omega_{\\mathrm{new} } - \\vec \\omega \\big|\\big|$ is smaller than some\n", "given tolerance.\n", "\n", "The value of $\\lambda$ decides how large steps the algorithm must take\n", @@ -4577,10 +4577,10 @@ "\n", "In our case, we have to minimize the cost function $c(x, P)$ with\n", "respect to the two sets of weights and bisases, that is for the hidden\n", - "layer $P_{\\text{hidden} }$ and for the ouput layer $P_{\\text{output}\n", + "layer $P_{\\mathrm{hidden} }$ and for the ouput layer $P_{\\mathrm{output}\n", "}$ .\n", "\n", - "This means that $P_{\\text{hidden} }$ and $P_{\\text{output} }$ is\n", + "This means that $P_{\\mathrm{hidden} }$ and $P_{\\mathrm{output} }$ is\n", "updated by" ] }, @@ -4590,8 +4590,8 @@ "source": [ "$$\n", "\\begin{aligned}\n", - "P_{\\text{hidden},\\text{new}} &= P_{\\text{hidden}} - \\lambda \\nabla_{P_{\\text{hidden}}} c(x, P) \\\\\n", - "P_{\\text{output},\\text{new}} &= P_{\\text{output}} - \\lambda \\nabla_{P_{\\text{output}}} c(x, P) \n", + "P_{\\mathrm{hidden},\\mathrm{new}} &= P_{\\mathrm{hidden}} - \\lambda \\nabla_{P_{\\mathrm{hidden}}} c(x, P) \\\\\n", + "P_{\\mathrm{output},\\mathrm{new}} &= P_{\\mathrm{output}} - \\lambda \\nabla_{P_{\\mathrm{output}}} c(x, P) \n", "\\end{aligned}\n", "$$" ] @@ -4655,13 +4655,13 @@ "\n", "As previously stated, a Deep Neural Network (DNN) follows the same\n", "concept of a neural network, but having more than one hidden\n", - "layer. Suppose that the network has $N_{\\text{hidden}}$ hidden layers\n", - "where the $l$-th layer has $N_{\\text{hidden}}^{(l)}$ neurons. The\n", + "layer. Suppose that the network has $N_{\\mathrm{hidden}}$ hidden layers\n", + "where the $l$-th layer has $N_{\\mathrm{hidden}}^{(l)}$ neurons. The\n", "input is still assumed to be an array of size $1 \\times N$. The\n", "network must now try to optimalize its output with respect to the\n", - "collection of weigths and biases $P = \\big\\{P_{\\text{input} }, \\\n", - "P_{\\text{hidden} }^{(1)}, \\ P_{\\text{hidden} }^{(2)}, \\ \\dots , \\\n", - "P_{\\text{hidden} }^{(N_{\\text{hidden}})}, \\ P_{\\text{output} }\\big\\}$.\n", + "collection of weigths and biases $P = \\big\\{P_{\\mathrm{input} }, \\\n", + "P_{\\mathrm{hidden} }^{(1)}, \\ P_{\\mathrm{hidden} }^{(2)}, \\ \\dots , \\\n", + "P_{\\mathrm{hidden} }^{(N_{\\mathrm{hidden}})}, \\ P_{\\mathrm{output} }\\big\\}$.\n", "\n", "\n", "## Feed forward again\n", @@ -4669,12 +4669,12 @@ "The feedforward step is similar to as for the neural netowork, but now considering more than one hidden layer. \n", "\n", "The $i$-th neuron at layer $l$ recieves the result\n", - "$\\vec{x}_j^{(l-1),\\text{hidden} }$ from the $j$-th neuron at layer\n", + "$\\hat{x}_j^{(l-1),\\mathrm{hidden} }$ from the $j$-th neuron at layer\n", "$l-1$. The $i$-th neuron at layer $l$ weights all of the elements in\n", - "$\\vec{x}_j^{(l-1),\\text{hidden} }$ with a weight vector $\\vec\n", - "w_{i,j}^{(l), \\ \\text{hidden} }$ with as many weigths as there are\n", - "elements in$\\vec{x}_j^{(l-1),\\text{hidden} }$, and adds a bias\n", - "$b_i^{(l), \\ \\text{hidden} }$:" + "$\\hat{x}_j^{(l-1),\\mathrm{hidden} }$ with a weight vector $\\vec\n", + "w_{i,j}^{(l), \\ \\mathrm{hidden} }$ with as many weigths as there are\n", + "elements in$\\hat{x}_j^{(l-1),\\mathrm{hidden} }$, and adds a bias\n", + "$b_i^{(l), \\ \\mathrm{hidden} }$:" ] }, { @@ -4683,14 +4683,14 @@ "source": [ "$$\n", "\\begin{aligned}\n", - "z_{i,j}^{(l),\\ \\text{hidden}} &= b_i^{(l), \\ \\text{hidden}} + \\big(\\vec{w}_{i}^{(l), \\ \\text{hidden}}\\big)^T\\vec{x}_j^{(l-1),\\text{hidden} } \\\\\n", + "z_{i,j}^{(l),\\ \\mathrm{hidden}} &= b_i^{(l), \\ \\mathrm{hidden}} + \\big(\\hat{w}_{i}^{(l), \\ \\mathrm{hidden}}\\big)^T\\hat{x}_j^{(l-1),\\mathrm{hidden} } \\\\\n", "&= \n", "\\begin{pmatrix}\n", - "b_i^{(l), \\ \\text{hidden}} & \\big(\\vec{w}_{i}^{(l), \\ \\text{hidden}}\\big)^T\n", + "b_i^{(l), \\ \\mathrm{hidden}} & \\big(\\hat{w}_{i}^{(l), \\ \\mathrm{hidden}}\\big)^T\n", "\\end{pmatrix}\n", "\\begin{pmatrix}\n", "1 \\\\\n", - "\\vec{x}_j^{(l-1),\\text{hidden} }\n", + "\\hat{x}_j^{(l-1),\\mathrm{hidden} }\n", "\\end{pmatrix} \n", "\\end{aligned}\n", "$$" @@ -4700,7 +4700,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The output from the $i$-th neuron at the hidden layer $l$ becomes a vector $\\vec{z}_{i}^{(l),\\ \\text{hidden}}$:" + "The output from the $i$-th neuron at the hidden layer $l$ becomes a vector $\\hat{z}_{i}^{(l),\\ \\mathrm{hidden}}$:" ] }, { @@ -4709,14 +4709,14 @@ "source": [ "$$\n", "\\begin{aligned}\n", - "\\vec{z}_{i}^{(l),\\ \\text{hidden}} &= \\Big( b_i^{(l), \\ \\text{hidden}} + \\big(\\vec{w}_{i}^{(l), \\ \\text{hidden}}\\big)^T\\vec{x}_1^{(l-1),\\text{hidden} }, \\ \\dots \\ , \\ b_i^{(l), \\ \\text{hidden}} + \\big(\\vec{w}_{i}^{(l), \\ \\text{hidden}}\\big)^T\\vec{x}_{N_{hidden}^{(l-1)}}^{(l-1),\\text{hidden} } \\Big) \\\\\n", + "\\hat{z}_{i}^{(l),\\ \\mathrm{hidden}} &= \\Big( b_i^{(l), \\ \\mathrm{hidden}} + \\big(\\hat{w}_{i}^{(l), \\ \\mathrm{hidden}}\\big)^T\\hat{x}_1^{(l-1),\\mathrm{hidden} }, \\ \\dots \\ , \\ b_i^{(l), \\ \\mathrm{hidden}} + \\big(\\hat{w}_{i}^{(l), \\ \\mathrm{hidden}}\\big)^T\\hat{x}_{N_{hidden}^{(l-1)}}^{(l-1),\\mathrm{hidden} } \\Big) \\\\\n", "&= \n", "\\begin{pmatrix}\n", - "b_i^{(l), \\ \\text{hidden}} & \\big(\\vec{w}_{i}^{(l), \\ \\text{hidden}}\\big)^T\n", + "b_i^{(l), \\ \\mathrm{hidden}} & \\big(\\hat{w}_{i}^{(l), \\ \\mathrm{hidden}}\\big)^T\n", "\\end{pmatrix}\n", "\\begin{pmatrix}\n", "1 & 1 & \\dots & 1 \\\\\n", - "\\vec{x}_{1}^{(l-1),\\text{hidden} } & \\vec{x}_{2}^{(l-1),\\text{hidden} } & \\dots & \\vec{x}_{N_{hidden}^{(l-1)}}^{(l-1),\\text{hidden} }\n", + "\\hat{x}_{1}^{(l-1),\\mathrm{hidden} } & \\hat{x}_{2}^{(l-1),\\mathrm{hidden} } & \\dots & \\hat{x}_{N_{hidden}^{(l-1)}}^{(l-1),\\mathrm{hidden} }\n", "\\end{pmatrix}\n", "\\end{aligned}\n", "$$" diff --git a/doc/pub/NeuralNet/ipynb/ipynb-NeuralNet-src.tar.gz b/doc/pub/NeuralNet/ipynb/ipynb-NeuralNet-src.tar.gz index a3461374c3f44d0e9d62191e423feca78ca7b4e1..67b12625b2de244be7b042aa6abdd06bd5cd1667 100644 GIT binary patch delta 20 bcmeyop7ry3RyO%=4hF`EMz*bNjBmpMP|*hG delta 20 ccmeyop7ry3RyO%=4u)ePjci-l7~h5i09T#|r~m)} diff --git a/doc/pub/NeuralNet/pdf/NeuralNet-minted.pdf b/doc/pub/NeuralNet/pdf/NeuralNet-minted.pdf index e6e3cc0f6a457fed58b1cca7bc332dcb0d6b751f..95ce8247487b5f10bbd324755a5e0aa3adadf8d7 100644 GIT binary patch delta 32387 zcmV)RK(oKb$S9o3D1d|kv;yU|0yi_0@v;t=>J0qz!%(zRD`m4VReF`gX{b%Sy56sT3$E8$9JK2zjnARkmR&y- z&6eBMe?Z?P_(Gqc9EP%c4l+-IVOR2$g2r+Pz`w6+n#KL)w_<-+OQXGEIYlFK_rT$lWdjPIMf-RgjO-e1jS~kzOG|P z9ICeIxpP+tv90!H)4Kr(eDPMB$z>@ML0pa{f3CNXXf)3;YC1=UDoy>A|3r~KA&4mR zV=>8-#PEcfcC*k#`Y0rh?yK#vdoE`#tJ@u2;ov%Kuq|dO8O_nZc$yD-vS z{Qwy%Qx_!oVNJ_)+;v~oHB)U;O? z8V#i)nH%mDFD7zV9Jk3mm)&e~1Zz|nr)qu#b;@BzhAhnVeEu2tnXN-642S#Q*NKxb zf~`M!p(IAgT^=Yp;k+}h6ssk6&czCkoVdBd<2(&*ax_@n`=&Vbbon+yk5jLsf7u#9 zloZv8aE9teU~Mw^`ojkzR8Vx}c*MV}2H57IlNtA3xkOi(GGJ z5ZSn3)N-ph8PMY_4&%u3$@;3ftxI-&Hm=R!j1$qzNBd1(^b*?p4IHMI;8%2A(cE$X zL91??TV|B#fJ{(ID4Zk(7y8C*fBNS7ncts|bQnWTIU;VJuQvN?&A~=J6SqBnTv&0xNCY_-8wbdC<*jW>no+Q zGq?PFH$y>*UDKJORN&@pUe+9e{>9*0Z10!&D$tUl;}Yq8u?3>$)%yn0s6ohl+8IJ?;}pmiICp9pggm_e0T6X3Z8g{2_<zz$ADF-)Thn ztNL&nw?a{28cCSva9aQi)cuIR#vp`L13nezajYGnN3@6#EK&x2?~AojP!0aU2l&pB z2t2eOgePucEQFFw*_E8{s^Uv6OWYiq&EV3^ zmBj-8rrN7yK1tq6g6}D#fpG$pi$3-RBW(lUA>R~)CaI7wf6Fc|UFNqu=4}F9a+#Dh z=efT}I&ylYG`|+mjC+=`_e*2p9EJF{8MwemILYAK!#SC%j!xkDDW8sD ztLdEWbVctOaQPW3jm1JI3y0Z48hi-5PguzyK->7~D_QVG*6KY`%~;K z-qslzaNn4eyMjH*_1;zAN^voSS-2*W}9y=&&QenLcgBzy&F80zgwsA-^e{ zk7<5)f1>YLu?m*dM^pmR=;u<+PEpBpfTNM)@=OAiiQKd`SI!A#^xS83?)m)u-Sb!B zCHq&4w-(eBA@Y^K=O=i>>yID)btc1)cq&f-$R8~dN1N6qt*>-|X`n}9B3+NsDi3@T z6vczuw0j`Nc`2xl4-bA*R?L&qVOHgTB43Z>mOPD z1L~MhN>^0T(3x)9k+49Ci^} zq;iLIm;k)~bod&#WDve0qvu=tZG9DAj+Q1-VP+R6IUg;L%cbCp(mCJdl`?~s^wU?$ z!57&%pVGgdac0kEqZ}r;I9Mfcd3?74f4wa?WLw)(KsQWb24}={IXDO^J9wa?G-rJR z1XK_WuH?QD(5^TrT`9`D@n6X5pWnRy_4KpZF^t(z>3pS-lj2-XJ{GG22LZH^d2nw3 zXcfc8?&u}lfkSFFJguxtE*{-$2K~FQ-0`UxnEk52eICwJdPw4+KbkQy^ekMSf7w{` z*{OiQfG$D^lNXb{h5Tol2hOG{;`dtIC{{SB9sq`{BY%hND6>7(`3 zWew>lM!~+=CMC_DV1kRVRv}&u9EQJ_T}$scSt=ogI-OQL(BvA2y8Zhl(S7w@D1I=B z{L_q@#T#lLnyb@R4}6nk`Me|_e;*Lf07Z+GWR*AA3o*z7N!kCyk4C4X2jrw?^0Se; z?8u7N^`4yk1yBC;@#Fi`b`p(m4v2j@3`8Nzr{Emz`Hss-mvCJ%=`p8y{Ns|DJWZp) ztYx>Tqfn%~pH=Rgo?gH>&C~Bke*UD~86S?hvN@CN6_R}xNd?&7fu9hFe@?W#EK}5V zIE1>U@25J2nN1VEyo*wQNBxvP-3E!%rEGk?NxNXhUB}o%t+OfAXAEt^1g+V=o%O2Z z;!iJ$s~cu)7xLQp(@P95(Q~%T3&nAm=IY4{5&G8)b)3(ilb>GFInUf3j@wEQ10g2M zL&STYYVGnTgZ~0uc&ZW#mlB=<6Stqh0!5gYvDO4Bf7M!DbKAHTeb=wRWA|Y};9h)o zJUf%k)@^3mZ6@BG**25OC^F-ykx30j$LYWCxs>e{Eh(WW#cdxPf#3yjaqhVv07KC# zqbRb<9zRZ5&iM7pCWqgkh~(LH?S77y(3Wbv9adI%%EPZK6Q1dndQe_y~>m@rRtG74J8B$J_$osn6UGqMEq zcHI&{**R>Pnb1g(VOGwQpI|{WWM7n&>-NQHBvA@;xhy+y7@j?bV?A$x7IQ?yNe>$2 zu!LjfImsxepfXPwyY(?Z*ke@2djT|vy%<;r8=$YZDv&^exdM6V@hUh00J{wyO0kbB ze?-8~*hz>EXh1H+EJ$FU1brFHgp{O*RLI>ZhlZFkIAIgi2th>>okK(eB;t@%WQINf zD{?Xu1`yGU-6N?ev|&sUF|~lLBBTX=G@|ri!Ysq_kPKr&Fg+OvGFX+YXb7DWh&y38 zKoa<|3>pzY!Dh)56vON(2pv*JLl_0+e^La|SX>}_>46VqArVNMjj#n&&oIH{>}*o} z^6zD-ikH)?a#B36tG2A#75s_djISoet8!J}G_!Kmxn9@xxAJm6{jvV6UP}jzf}PNt z38rW;7EBMF+z&r=)BmJiQ|H;QRPozC{wXe|qkw0;hNo0Fi^ZG$vfRnSt91Yge})MM zr!zP|OxQ)uz;RPp0ft50Tr8)vQt^7s%W2z|O{Hl4_@~e9pD)^JTdMV(lj0o6sk1Xx zoLeg>ujTN7a7GUgfc;L1bCURT|3bn`*M5~{|nfh7LL3PM;r#nTBRNwX}D+iW!L@} zqSZ*Kr^K%3So?A85$f(isa|*Whv*{Zhx|TTx|5p}Ki17<*>rQ7 zH^pCy7sYctVDE24iy71>wmKs_1@<~iRy0<7B;VkIhK(S1ar3UloW)<~f7Qq0+1XjQ zLh)?Y&g-hUD1LqQLjL~np>3C|KNdw<>D&3o`BFBd>*lJEr{dXixtPzUGUn~Fsjr&p z_4T~EdOHU;&3oAJJE%Wy2R0{dVN7pH6Lx~Eq$BdAC)UP~*ulrLnXdG_E;e)$Y9b#F zrg2~PX=wbw&*B`k;&2GPe@9YUs>(-{ma26mrSm$aNNs!+rLEHjl?_w)7_M%und%Si_2LKu3ztwzSq}mysl&6* za)t;XaP&{0Fc>RGl%ec?B)ev}gWF})`h$em6;(cMHtaEbgY)~Ge+1>9rQ_J2x7j~7 zOUfZ{xB+%#NssX2gBvD{$sh&4dj3^s+fL{l4|28*RQZ5}-jhv6dn0$Ga4Y_r@SV9h8!a~x-^yJ3 z%(7{^O_P4Xe_W_=zZMe*n#-jvbGa}!Ho7%Ja=pC%y^EEej*6A{)ji>0ZE(WD-Vrka zr?onDD*f*VO9x$@9Ify6;2an#A6v%kVO+30G`uJH895L*=)k|HC<~wR*&eKMRo*ny z#l2$F>hiKImy2@RR5)hoS$$p1K1`S4R42jUadLGMf7Y}?B`muz6rM_Ji7_q?4_ZR2bnCQLZbxkST+ z<<7=Gf2zm6(UGHBtKToE##$q8AIn1?YmIoou{`cLX?xVH@|}E;u;Z+Wp6FZHetR5!F%sjJ z1)?8jf%VtZ9#G-wzZo8lK-#|kPi8Cv>3|#PNVMxxIXi9qw}We6Al{(cXdJhhltwct zg#&O5JABzQsxdoEn7q;QAB_IreL3@%1+pD$fqU(TSfDR^7a%=k&4va314z@J{+Dre z0~43nU;`JIar_A>w-{jqg9v{<&R{h#0HtYsK(;G4n__P4Lg%fGT8x%XGGolwo#~2* zalnPAyK;=PG2@3Dhjbh%WN~P`ed2#={mF)}%=*CK?ttamsVy@=b~&--hnX#DS2Qk} z>XNRUA;T>SjoDia^i>z?xp7#kkZ^br*t+{^{PE1iY4w~4WN2^D?nF`%c)|DL#)gr|hUa3<-xEv+gPWl^4@2 zrppBkk1`j8Nggfv%j(sC0PNB^f0uD}0}}x;li@)W12;1@lku`Bm%(ZS8h`Wxc>3()mLwX zT{-4TS+4FLR-CyCUh$P!30*NO$ z-`Xkbjj*Gb5^$?7Og_T$RjORpEk?DOax6f@!B$clR*sslC5% z0_~T$mSx4b6)E|?dG4D5mP%KQ1MxXVNj|i77<%Y>x@qbSPJN=MeRJr$DBXUs`pDBt zFfQZSDaC9FLTba{6g(5#`bOH~6aF-CM!1y`MZLeL8=`1wc%Cq_FW#6ZUR;R5|6*aG8!B%v2!kWmaBdjm$h0E@;s} zs|&O!KFDa)p8BqC_wD|Xg3gk2?wA%qUA5`@iGQqOx-u!9$BfA&d)tZ`FtTJ3nltx?19I8on{%0sYgXF)o^y-V%)(jC zEt+3A6)6-1R*{lMi_|J+WLbcf)?2zAK<{HqISb4ZHFuy~EsKZ47SxI6yA89)Cm(d+ z4iirs7s{Z2S$qK5E7AZM0*yWQ-7^jMSvxdFFi}Zjb$`x}BD;qx9xbkYg)2GJtNw8n zZ-4y$6|56wcu#O8xG+0ORRO6O<(QP=c)mr2se+j7hN1niZD`2bdW{MG6Ahr8cy0E> zp%0VBr+Pf6Yin4=UAkpdwp%D$kg^RadpsYxBxE^`2RNtft?(Q>PBIJ8G+FdDMB&m- z3G0RZ6n|K)Gx}sdGg<3$WZAME2v85r_l2Iz177HYG~{dJB$!xUi4IBqKj>A7_{-7k2l{b(+=?MGzXFX!<&B zYxHcV7F%baLEXe{7(v-rWESI&ABaj1{0Q z88X}eC<2tRyHLi4SfP!D>QYmz`0K;i?0;_wVFje`ioV;1J1P|%fiI}y0l@(T_Z|YH zdq{!kJ^0i0KhvOhI_!6JblpMCuxoz}1luvZey;m^*Njacoayj6RC*2 zmVr-e$V=dI0q*T^8_g$=N93mkVnqQ`_4KsqTiCa)F-{m0;{>dVFK%fbod=G09)HBb z5Ru@;7Jwa%JRJ5L4>3g%mbKwcO;dMGP3R6E;5xOo`0+wg+r?*m~H~|~cHfTw0ATVT05DlfUEQ2PeWD|YVKh(z#UraX{73pGA z@26x$I$)=V9;y%pF-Gw!>6iMl9yz96Y;G5uoE3E#eDSYh_W*7Nti?yVCw~RmsdlH{ zAFQn7GVC=p!#SM9D^>}U^879zza~L7Ko|$Z$)dT+kY8*Wj7R2Q=#Q$(FlC~OLe2;O zgkuH*v1WvUHe3f!w;k|snh-9&ypCK0){81~i~Xm9)ddHIVKg^eG)-WJB<_azJUau; zC$q(1o3&466;m#H=eBR^`+rXhD@%LTYF40UbSkxPLoN_R3JL|(th40D&tnosly)rm!l3sNW4^yPIKC z4^f8e5UGS{EN!TG|9?cC)=-g2&>x~hpRWQgY44k1j&o#D2V~Ox<4_;li1lzB-KIlH zckY%#0RS~baZ`~vWs>j^LSg*@Ct;ip*sAMemJwzt7)Dwdn~}fgQj8j22&X%i8&mvihp05J^=GNkQ#RNcI!Vg zxZ>@!Wjnm|VNRBUsWND51Bw1NVN;*TW>9oI%^#7RNdK(qM5;c@RR*DNmgB#WXnq#=ei{OFMnm6MMo4rBcCQ$wP?v?x_$|r zgsU@2+FhTd)$gJ!c4}Kh?PwUY9#o1yy#Yza%V2>rL(F$34P^EB6+9CH))IcAX3L6iXx~rf6|ERQ%Q~rNBv#AUBEYq-1$!q^L zf`k(UpDR0cXc%4!k<}F;Qe6=u_3t@E{@*Sx+9<^r<^g+69>8y>i$pN>g_hq6Ac#Sb z1Ux8sNC!qI3%lf_~kUmX-( z+%M?jr&lySe|PulyEn`%9C4cK#ocZ}iL>x8x?l??EisHQv@(PXwYYm+{C&A=yVY7y zy{r!SYz|Mye*uS|R>~}^{jpyCY%OX+E4Usv?FF5@Y8(auN-E0k6t)=A4 zw>#WAIHOr+vVpKcgwc>=wF&;Ju+Dhb$^9VVI+}O<-6X;1#Y2 zk{HDoYeppSc(;XFL!2eb=Mk0@BWILYGW zc3td}F~a>x(`8lE^kdsafmdF{lj4I!C$1T2&S<;PQV}@TWIu&-!UXAHQ4nCsAQaIc z-#Fn0_7-B=W#jbQ0zJlw1lx`?-~jq`fH^Tov^F#5G8;hVQX6OrTWX?( zStOHQepahtKmw;2K4R3#Np5m;F9N*K}k|ga;>{AK#M2Z;`6aNG>^?M-qM6zHhn=9YbFh`%q`YH z%`qee>xxR^S})d`NWaw2i;g2w)){~eC<#Eye?~0@S_=yX72I5n+WT8^4`X2u4l)Ct zQU&IlP~b%^m?M@eoc1%NB><2_f$6QSXDZB#5F~iEE29pkGA3{mEwHwxL@4XeuZ+G7 zRDvjK7vLOPjEohLNf2w)3w=W6rks-zOfQm2S1=7+mea|cPY5Ph<{Q)|b0(ptg#1V( ze{^Vpjz6C)&`Fs$faC5ohpE*XV5AfzK;d4XsWdJNKbiD@Abfeo*U4BxZX+(~EMzM0 ziJVX!H|7Om5Vf>}lTZ~cu`OVTF+>2Ut|3@au}YGOY5Y(l9@31x0wKA~ibWL7D1NH7+5o|0UN`q!+<5J&lWtj<)tW)M4>HJGiA zKs~15%>iXL)ZJqrMZGKVq&Z~z6XWj+2B(Q|I;)Kynl0E*sUUz#lVufBe|6ZFMCynz z{M$BdkAv@16eod0KSt^oU#Fg}sl6{EbfFV&VR2y9l^TX!vI z)p^vgJO}DjWZ5wS;S8?te~TgmZJaw6oS%ivfTNi}{cE5?Uke@z@G$%LU0sDdb1jTo zR`}?f$ESUb;~(q#2}ieM6Dy2|GCnhzz^Y_Wa|8IpX#=X>1ytn@VDZaAPq*5%yXmLuMD0s)u{e-S2I^IWV}K!DEn zc_vjp?Wb+P_W6eF8A%JH2?NN_*7XnValb{`y!XABWgVK6bousxZ-<8pt zN@7@)AO#MQKM9MDpG5Ci65~vaaWEm!hUa4ybO_USUHgdd`b}328$bO4Uj&uIl;dGr zcm3euHN20E4*>W0f5O{=33dRptBjS%erA7xf1vc-U1_^{GY&^x8EU(v+6JvXoh+-9J@M_#Z!5cuM?}r)4-44hNNI<@4ISS z%ul569;#ReZK{}!co9MFfIp6Xy-j;n5>^|!vjhKrXA|h5U82SPe2+fWt3k;@_;A%%B$gP$icdhgfZN-#&MK2 z1^$KXFl}SYe|;N_v;BxaF*{@_Qn9E7nu{C%RpH|#Gn&XOm}78Pw)`WuvSTYCnE2c> zgoTjmqK2wyyQe)(RB<%i9qv|@F?5!!QtYjJV+H%2EE)d=AYLE(oN{o4cxpUvo9!3O zrDWa^RfrOH^wUu+VkQguO!LO|&_1C%f9#I?{nydXW=`@1?!B-3o=ozb zTfESMC>!PZW@S0%pFqO96L7F^HqC%W1OL3i|4k+XkE-vFk74W!5fq`fQ&<~kjQ3M8 z0b$ri%^ARtE?Q(EUL?0QYR>9E3y}#DGcCltDK`&6(eMSgU(U72LC%F=bPwSvYgNh|v7x3yz zZ*8=GNE|fe)oEffzXA)FD`=+>@FA7Y7S$(&=>v4A<=fzhF&2a62Rp!RuEPJ^^m&i1=!f6gjFfuYfsF6xhIu^k=Ehqa*$Ey58E zl^Ndj9rvEo{k}iphF|r{uK~p`y8FdA{L4?Tym`9#f4rQI#z0CrHiax349>>n`pwG8 zWnFJ~ZTETQLBHEx?`n)3a|f~B#oNs{4FnwzG8z+fWtO{0?z6|)Q#?A)}0><<> ze}42+vF8B4Vq)cgKm_+ulli-A1wy^Uw9qf9t3yAMtcELSB%z{kP}5~>zZi`!2?L^E zXbxsn%jdX|yK)O|FJkWc1fU$!Bc;<*O)ngxDKK*{VR}sKVT5Tt zgcJrU#t0byb7%)#bf|+w{IV9GPaRI(K!n6OXio62Z8cOlst&k)_^~^7Fr|x_gNH$W z*b>hTzdXZk6zF(FXYyK+1Ar>-M&R7HH?6&Qf!X>t2X|lIsuuCq~h8G&WKnn zlitG_zWKslnYn4$ZUH*#KOvFvwg}&QaPLwXOVkJ?1M(FopP>1 zvjo7CX*c@};u-y1?#`V$BylRae{{d*KiSdD&GR~&d@pd}qD>KLY`(sE2Fa~2=JO@% zP`cRI#iADr9IQe2l4{ zOe+2u+@jms#K6Lx!}q0VZDeRj`CjyR+x<&1qo?rRI_tJ2|4!6=o9e*UeH7k&0o#g)5sbGjb5q$d6EyKsIt009B2FsP56 zxT=L;33{10q7J-NtjoVL0b-Cifc%}AM8$dW`yT^qy{OFPx_Aw0F$TS?vFP}gPi;OF zKZ#(FXmTGaCe}n&?acAfUy@R9N0bK5hE+;sYyNQe>R)bDK);u9bpsQZL4O4k1Rygo zHEm-p8{-u&&;zcN!f;xyMd z9wpAgT(n|-Xvr!qEU^+EE@0%Fhwi%Jme!j%1-9c9*nGU^E=<8~n8M%Qrq}xAdY9hX z?w@%%xShzsq7Z^9!Z^>R=uZnSm?cWd{GDd|5axU6XR~e@+RJXYS;h8k8yj!%Qb&Ow zH~oKho4U6{yyEf@&whU_`nY*{m=Y_>Feeq|;~DjiR(@TgD0H zFKXONY%9*#_Qfr$o^M(H49mX0ZkRUVqVXbxJY{tuiGc|DgIWDK9_>ED?(U@BzbSuY z6a(p7qpvC@#)QHOwrB(u1T5PKNG)2O%~2LIWEzmt;)*Ls-;E*8>2n!M5Y5yj%gL!d zZa1hLR)g7;6Q^V3lIiv_ayVyZjks256zfKNcmu;=96EtH&1fmk3#Nw_BoefgNvWw} ziqm3>5{7_TqdOi(Pr=0%F`T9Tr2&5%jS_NSOaKafYzk!%>4Hxq{lk-nAB#=LdXg>} zIA1Y5py0w-mxTw1UVL04QHl;`AUdG3H9`}Sg7%iI6ciG|0-iW#}j^%7LqHkl$KE1WWY~fBlv$6y6%T? z_4pGTdrWo$l5u7Tr8*z-2qZ&%dB$F18oCpE31>gVUPmp#6@XC4OPSc)hc?<`*8M<* zTz1hUU3-hRcnJSCEBx~qt+qS`$TcN`9?bHOf&!?Z4!$g(p@ndWb4@<}t036pLq8Sz z7N9%QjJ97+)*0=~atZ@niBNy2VScPf?*fUU454`6t&k%Lkvv6sYS7M8dmGHdO}*Ou z9DA<9bTMgL2Dt-ZZnZDyYM>^n(_Q02?b70LzRyV~4Wsrl_!SD2Pud>*Ktk7~up z$TaMsze^gT5Ow+@k~{&nX;IX-Q33;+?Uu7s&JN&20Tjr_JIX)oIsSjaF0E5i6zkLz zPM7rN8gLIdR3>L!HdxOP*qir3>B#F&crlE^#eo&ii6A6s00lFDX&EGlj)Tt0>PRv1 z@7%A)`yk${tgkspM+Ne&xFt*_&+X@PB%p3eQEfeRAq>JJ^fxrp zuj0Ks5OKG>^*4^&h&z9bU6ISs7Ly#;I;&bG+&U%bl!B}1eB=}&`!%9aNukDPy)I|Q zN9VT0N%drIc5Y9B6C)3VA=izBtn+m=>-M1^g7Ld|?P5LeV7a+-Y4^j`Lw~+o~W!ez*`#}H(a$^GsuO#9R1FmA^1bkh6d2{vOtG%0XPgW#c4fxgV;_CY!X*Gj~ z7!g<{AAQ#a1Rl{8S7cs&clFP>9hL#dG?s2tMGr!Dy7{Jil(tbB$ycbWczy!2R!M(yBh(ZC~ffleYeGZw5J{mp&%3@ zNJ~V5_BW}dttM~f(xTIixbH}aX)zMYv?hyq7abhlY8f)XQcDUdgG_Ysbj z1{NOX#}0l+5Qlsi)3q3P1hHsCG_xW<0#MDqS z5O7}iM+z$7mT+SdRPCl4L2q<_k0ngK%^~jgAcOem$bb!bf?py`6&7@0)6)ZiAUcf$ zZ5+p?al(Hs0QN&a#OJF!yy|+jj4zKN3hh2#L+SPJwivaYM_Lq6&@2tPz-iv}fdKhe za$$TK2D+vXO0fVEowoF28bVR`^a(COM!O#~$T)<%LCDF|G2!rl4^2{k@QA`BL)nR; zCP>mmL)oB}-D_p{3S48r?@ZX0smHh|LvXmKOfJ=_tx98~cV~61wEbluu+{NMnA%2I%mOZ#B6tmG^QLcy z?vxTxPCV4gUO%>Fs4r52K1+TH6Ek+Ow&+DtATBLoOEkPuX?viL*b?F<1dXWs?O6W; zryYNW-J%;(E@-z|R*xSD+A3bvx?yWc7 z+kUkh=I^hqs>un{B^TpP``tCK-zgwdi0~1*Oahw~LSOhj?T80H`>6!-z-t85aFc)b zT8|!f=*tc}4B{V=c*UvX*pgm8S4k;zm8JpLl1Mop$Bt-*!)x`f$T5M+A@$Z27ek$pVL<%q;Cqh3tD|iz4 z^3dfCuu~R1{>yg|GQ%)_G*XTghrWN5z4u#2%29>50ITy{u&}4%>NZ>^J8}O1eIM?` zUa?qV0-DdmUU`qD0r8KdPbv-e48_kSM1-9uU&W+BO9h&1Nvs-cg69{$1nybzSbNbq zutY5jpu9v~h4tH54X^>p#Y+&kqpN7oCI|n037aN|7dX4UAf4%tOK0}~C!K%U)9Flr zF@!(Fl_}UhDW2ad5SU+nfQtj$G0S}tb0k0>l=X#jq$uaekQHe%Y;h5#-`D0u8u2S- z>H{hGk%NEbrczvRFwTL)+uw9gK`3|pBzj+8blUnqQePlDf=}S;1Dq5lcK&*qC}$s; zDAJIS;zHgGcZaU~Ikei?EgpaU^-BP8q5q~`-pB4C^>Rq+O4sdt*Tuo1eBBmCkNY6d zSg*rD=(~Q=rVk{MZt=qmjo5CmXyGT5khVAB zAJBM+{(Wr~MUnvo4{lAT{+MIB=O*q$20EVC8mz^+T%kIFV&fsIW#NAi)l3gjp#vUm zlu96Tl(0lP{JC{MJb3W#`Dv<$T{ic8SZ3}BP&-*wvv zOpK~`>u?R@LgN-U0+D}Zy$b))@UaESs>UVoaTIT^+s!tdW)rRei76U>@X$NI^X*qx zU=k*wx;xyx4j9`?C`wHEBI%|3({-b`s=rx5)c972By2s#JG)_g)ZrI4iq^~BV*Nh$ z+CXe0A;dqPL}KHM-F9`qX=mXd9tc6AnT&!BeBO7u^+ns>}B`>cWB}izm`)F zsFt&`jel+5*yJ6r?oFH>)B%EYfDs#HZs{{OrL$MRyt(=xZRCzAmvMCi69F`nVSE#} zQH}#V2LU#dkq8r)p_KzKf8#h3zW1-tVlVO_=7=wen!y}q(GTok7TXx)w!H`xTXBTg zQdW;RJ%|16TV#>8C{sz?oyN{wB9bPH#j3B0UolCviIV8Uo5XzylTkusf`3{V^+$gD z@aAuqZ@zy|sfZNG(^N#4HxZSbq?AUfP=uvX?^5KpCnqE4l1%PZ$c8`5A`Lw5i608 zPpF7uIV?94H=aBg_>{=x@d8hW4851e$%4NqMs)EzIY)E zC`PS4zub-)p1Ljjf1$0Z_)HHP~j~3btD3E2nMJ?U(~92Jb>9EdEQE z9hn0i43uzDmuu`^FIXObK9@$B149AFmx`GKCw~tiU?@TrgCxO84u?6l8B9w_o^gdH zp;-<&i-=s{+`8O{kD+oIjpqFn+8c5sS3yrfemTyAaFHWBCm}nV)^QG#0XdQ=dYIE% zA^GTdig0O;C)GixoF6QqJXslF^)PV`9_$715NE=Jp5kL(=8pjYL8Zv1xd<+a1hz8K zc7M~f03Sh3Jj18PM5aM>jY-e`ZjumuE66&94ss1Uk;HC2M8A&GYHOV#LK!LHL<-1A zTmo8bUip+z;dqh+_b7K@JIo1(6cqX;yx_z_TyVfVYOQeVlcoSks)QFZ#lf z&l&r3I6H*^_)7cFDhp|mNSRImQh(M>kJP|XN@!?W4qA_yxv~##y498)a!$aOUD*U< znq@h-#>U8jOXIVGk!-n0Rfjm1&NXLpH4?6N0N6n8y=vzljp6uE<(S}r?}xYeE%XzP zZdk6G`L8ginSq{!psvKAyc)6hJZ^>-iTer)g$o?p+t}9a+d_99boAR1Q-3$Wm?;>l z-`%Lr6}Iizw3=ub=J&vU6JWodcAHqw+14g7f7?7 za6m2z)9Gxi`^O-I;d$zDcDr5B)(6`NNP8-u*yeCX?+n4LkmTSv>!N13A9_6ebH|em z!uJsBFFQ{&^!G#P^#XDMgMrahq#F8ZL`n%lOaj55E^nQKc4bj_?tg8d=mT12aFy`q zl*~NZIBWEgl6I=La=`I91Sxx=W#r2(W2d^qk6s$qIC701JJkqc9R$F&wPX;F?xH#$ zgWkmlg(%PSN%`UlV{kPd5c0M4Cp{BuW4*l_o3GX;T&cp9J66o8pF!OTzThY( z$;_;T2dq9;is3^aBY%>KSFdPiE`*Ha345lNh1L3~+6QnfAzl=5fwXW6G4+Rq*jVpB z3&b>hvVhrx7&B%5{}p3%kDRXXq#`4!W|6zY#lSNCs;jU3T$5$p-Reg-AboeXTixj& zOY9Z+Sqz`+x1OtC-e<|fnH$4WZulaN1L@x+ZrCDJ7${OJl+EE%HI$QonsBX(fQv!^+}(?YRNka_mv8&6j&Kfgji5JYOC4fjGWd5k=?NKVkAFpS1;C#ME2t!%auDUg(X+=u@5 z&g_NauBoe))1ZCvUbs6uJC|=}R*pswqiFQ=yU4X9ibfHM@elOshfy%|x1Znr>Gs|C zA1IASDan#VjBXbr8gr6RI!a`G{^U=rge_SmlpKkvW(uP0Or|o8dHWKE2!YN5v z2HQ2ordmF!yK1iPO0}u;B{c1GRsYjF2G;!=#nI>QCexV3<8rgotE;lPdo1R2RrWp< zt*Y&Cxw?mK^u3`)SnMtxTUNap%c|(H8jYrUU|9X?D?|1d)tb&`ZQ7zN+9F??!LuqJ zf7-{!4DxbrI`>81s3wrzVTOJ}T2OVzAfFUsUZK6{^6$*E+HS z5_qutqxtb>ke`qGgdYO!7~!w$6X=XdsNoufRic`Q)O#X>Y=}X$g}0c(QHrQG?RwLm zsR;V}Qp71#Xh>zJubz6xZv(G6dLzBKe-W^V5kV7&UDgP`rKW9)a;CQKZn43& zn=w}P&7b?wI;)!YeP9iGlNLeTC*x)Yt{`K=X~)sm4?_fny#;I-!O|vZW@ct)W@ct) zjF}n7%*+%s#%#yzn3?UEDUK;-W?a90|L#erlTIbIs!H88nwg$y>HDg?9&a#>Si#eW z6Sldnbb!;R(>b53a<5oLUb|SiuHE>hEvuh@xYw@Iq-afLv39a!9UP4G@ANPZx@m~T z{Po+0ZCREbavbl)zt6&J>WR56O=*}=Q9}wJnmhgtu^nBUmVeb*JNj38n6aM`t5<8c z1VXIXux8b&2T`Dk33}kh6atsDEtoH>xW3b1y$%TRbsr7L7zx{J5r+P@@opyTM*VTn zxRo5DbK-Xs={TxymTlGVMY9)Mh=V2uT#Xf^5XF@QPu(oo^c!ndu1@>rw=waodK#rq z%<7+)o88lC3cy$So#<%1rbo*1df{m4xCHgvPHh}MY)|zS!Gsr+tLJ;bCNhfBm?w}> zPyxucl`PITe)_G#OO15bZh<8!1^G7ztljl>t=0svp0ZMk!USz#Gu9WdGim0FWn&KVv7> zeX4$gUoQ?VGwXAUrueV@ppBNrx`x+Q-0jui{9ZJKkcIm3{+ZOD zR8?|K-$0JqZ-Ow}etdt<${o1@`IP{@Po6^9(-TVI=3??ANEecRKzr0V+F2O&AJ~6_ zO=0B=Wcj0|SxcbrW0P@=0aoE20cI!9Zg42`2x(K9qfjn(b7cywm@e#>HcjP$Fk4X9Fgw)bTL`BKcC`u%wXewoQ5=epSDg}Gy||+m9Th7 z2VRx1v*Z(HM{ts>W=rLlz0({Il0pdZ*dO02f06`y_?;8hxMiR)RZ9DJ6cEp_UHzHb zD2}xeK+GN|U>_Yzs9=qT6sGBllfWe$4x3now$;hJUv>#;Fi|z(6^TmbIS&3MyI&5w z3R(`W(Xhu!j$>kx2YFaxYy#8@ZtLZWGp7{ImpNPGM$vCJLOEv zO-~orD)=EsAeB35r?rO5feG#oBgDIb)$SV4zofWQxo zC*vw~IP=i!*al4`q-I?y+(TUkD=VKHz_;N}Ct9DshO6L0GzkmOS^z}8V;a2gruB;% z$`k=~zol(Jtri6PijMBu`*PHPhI-Cbfa&M4c)iy2dZop;US2UMvm$ozE6s;#1#~?f zGQ6QPcw(iaADH|b#?79liCv;04Pw)XjuGyNs@kPW(84;yF(#0TC}k;A&(QHuGYPVC zw7&9fSKs>PL3G%Ti-7N}H^E&&ZsJ?3`aY0r5*UanWkJ}5RUoiO%jvX#t5%OExBHV` z%5=e!=F)4aiKk|Sb&v?&0@=!L*_2jfh+sthc(Jg78endng%E_`r|IDu3$6lg zwn-ScXLjB^(>u%fkdBQ=#*59KPjTal(mt|}3yFK-hS@4)iCHDconE|yg47;l(y|0- z7l&$|yJ6YI3%^6^eUxQ`5!G&&k;4#CieF&QEH?+ z{$+!ur>@Y5IRGVqOi3p%$FEyy7lOj2uafPZ$>*5brAld6%o__uM^upml61kIzFg3o zrietYvbPK;foF>~=mW!{vsb!;L#&j9#)L4m6W5H7Hf%9ZJ~QybG>Q_oXvdc5`XDeO z3xXXBRJpd@BcOH#C$Ydf`I4zq&szk(v+_IpxKo9AS)3Nc}dc+w)JGq0XvCDYyx%lp$Htki1jMHNUwJu?DCZ|-8pwiUa|j@M0`aOb+T0} zkEmf61`u7XXd{q8GLH%t*F$WYNl=AqB>s(+$U>Mg;E9egqr>i<>mGtbC<2WYx-@TM z<#N=HvxopXwqZd>l(eH?Izie##7c;9s^r-os97?JkeIh) z1kC79eV5gP=L%2z6C^yDbN;>JeF`TmgWcE#hi9a<&N(Yrbq+{{PyYjnP@LUrf39d+ z*@fzhZ>|gLAvQ5Wp-TO8l77;5OHA5vDGZ*nPy08YEpgh=Op44mJ_R?TverCKuzy zU;K{00GHwEjB&nufY~^0vywLcRb*3|l%gSJm5KV7Rt_eU(uQvgd-RA-CzdubdeC9riMmS z&O%m$A#)Zw5RdBNP>Vw@?d+lnH*je+Cs3hEhKPPrx_AWHWCbL%_+OpCk*K1CNtgZc zYS5L$b^K_WFl2E+pEGXwNyH;u8Vo&UWMc<(Q5+?Ub;#m|q1ss$*1=nl_^VbkqHa-l z4<5PQN@wT$#8>ES83CBgRIMm ziYS!MD937P)v&nTNE9J{pGI%(DR zSdC_Sv~VcvKk@ZN*GM&hp6&FEm^EW+hAdU{5w`6xrDUV(MsFS;8wY!`;BGTFK zjEp@CqZ%c?(>HQzNTD6mM8w^4ET-9nsP)m5_c@iW>&OeyGqi2& z5J>4mg)nav4YrCiFCNN1imT|9ub~JE-!#r9Sj%v$NkLmUK*W^b%ft*#qrt}&ag9n) z9g^Kk3jh2QZVR3o)xWTY+ztHXxq^)a&u!ol@C3BLHV;zihheD;Bt1;f2{IL$+1DQ0 zAqGNK#iCIItq#!objo%TvXRC>#c!DCW`GE2R~2~;zJ~%IFoni125lT-EDYr8+d^14 zTa~%IrthC=7d_ex2?>w0KO3Nlv^4UxyGg~Ya}04ylO1!-0@%J)iOOy)5LLN%MRzIhc)=vIY@cES8^jnAPu>R0$%o;aV{aa8OaJExx0cX}ZND zq#3Sm-+TM}C6Q?%3+ffdJCxVGCWr6MsWnatx}@%i%%R$+f=5I?Eg{ z^hgX*0=XSkzarS^yQWfK_ZAR#=o9E>bXd)&-&gjl_C)v>u6-dt>lL>?SAAi%3gFVc ztjgQ64O+U4ZCZPkuS5UT2UoKO>Cd9|HhSdmb%9iPUs24uf}|dxUYCLG<8Z;b^TJ2a zFebF`v6n6ExvgZU?RGqOWi1#`S)Jj*J{~4&lAKbmv&w;CwZP%(3 zr$&bIaq*Gs^(UTU3z_BD{>+DMkGVh`?3}n)vHAO3xsBEqGPdDK#=C)?TgO=J&4jP% zdF_NiKK3L5NM&cvLV)hdS|p(w&hpUnt9|1{?!UAlyUflKkDmOYv3pUjWx&Vn`__J4 zfJ3!aK{%(L#?_9d@wojWl%0P)*}5eeyhXA6XAL1^n5bip$K2KoeP);ClF?(wiI$P| zoo_CON3bwAcm=|zhIc^QM1WOhwWR(g0}%r&SJYfw>;jzS;f+vr?Abtk&Fy6O%%m>L z^!IAz26Lq9oa+358po5j06?`iC-}UAKP}_FLyJXmto!Pcm`B^S_EC5fe%dCA=v<;p ziq}tJ&%xtL#$q1AyV(XK}VD52D$ z*+i0wERE%jj;R{ME0{i_H}B; zW~`UyggE^iqGDv!S%6m1I;PrEbE*ZA82a9A_x`Onm)GL%-QGs4`INPT{`NjVj@u1C zUNhBc6~PdR?rug??E_JxKX#F5daTugSRMXkcF%C<*hqo_rLOHc%lh=fed|MTOSbdE zf}GdG%lGB`{Nnl5sEZSSfQ_T_#->nkE7#wX?1oz0>Q*~DbPivZkiDH+_3v%foty0^ zIFfQnA6D42L0zPS!VPM&rsm`&S+X<;SWw52Z?DZIy&%bWef0lDcevR9_e$Q@-2e{z z#mQuv2!LX=t+#;(LxEnm8T5eTgF-MGjx7DRGdI_NJD2o=PeX!pva)9O$-rQ=wJw6E z5@b3LgVQ6j!!avZI$F6~lW?%HXa02sr2`D~6ZJ=v1OL?@N#ny4KfI&CfKt$VhQ){W zxiWQy5RI7E2}B8$5Y7kRT;2v~P7fX?X^dqzuE0N;#!O$F=xEg6Rl;L6@zD^MKAEEv|n#+uhsNAKnS%`jSP)0lNs>;f?9 za@4r=n6XEGF9xS;b)iM2#?Qp(av5$%Yk-e1ta3p$9BdTu6)0apoA0|jj6$CqJBuFa zieT*K7TO7#x7srwqV*RoF@rabSMD#FpQEuvb%8BlC5Pg$9r6ib@3EX32ncN^n6Ms- zlM)S0gCj{qJ_C*4=R@UCXg7#E8V9%-+g-*`sfQHB9C=7#2=J<8Vpc^gNC_jo(T%93 z(7;H>i$#=pi_ z2}saUy+Svi!r4hA*|sf=D6hb~b_#iuMKQrSo2hfmNHrS8`Us*U^f8s0f&fZPt{=4V z6_Bt$m_?xE>9QgtxLdB5v(^O0;}rEtjY$q?^yD-{y>(HgWICF|e}?BZFNx5@U`cQ? zi3t5{V0s}jM; zj2E~tQa&lXwQbO2P&-Y3cJ~I~{PykM)Pd`tZs6^#7d_G6Y1`lu!+PzN(?=AWwXaaV z_aBF&8A__tb4I})JkxznMf=coe}=4rHn^_M-y4dvy3|@w9cJLo zGafS&Xz%=Zy*W;$Zu+=6K6p7f^nG@H{#S2&b(wm(a_V&nG#eOg2DZP9y1Tu8^SM5M zEt}+0YK6!Y9zG`W7P%4>!4`v6cV3L^?Kn6cw7(WF5RCzm<71;8*aZf z5GcXSmkbk2%*GyJ=2CUh`#?KuDwrx&xYN~5qQ`H@j?Mk0HQC|VPpbei6o%zZ<9YTb zYhw27blV?Y92ks=3_R;Tj!Ydl{f*4sk`1q{c~ed-Wjzu+9*wMK618Orxx92))aU*z zAb<8)Iw)KRj2Vu^!#ta|xZGH3<>r(d45uo_5fR+w5;3*VQkw{37WZ;YGCTEju$tUb zSAV`=htRR>!t7NXMh)Z#pcI^IfP?3-VM3uoPVD1m?ND%j)<{SD(5XzQY-z9;7iv&v z`9ct@-(Pg6sxWtQe77L?bm`h|<6y<$L0lWH((u9q*vx{W8+T;KK!0rRlqics;Wzsk z3=0?kDY>{fx+^>_`M!GcSmz)zhtB1Odi1kY@dE|vi!A`?g2#9yn?%?l5cW_^rt4Fb`1D*{!R*2W?}fSN*)s( zkfId;s7y;Ri0m{Hbv(RvD^N#k>C|ER^i@qgIO(@Z(-Ab~B z{g!maZi}^WQHgiPV`CF~Lh{0EqW8HuR*m1M1)WbY{oZgtz-{GuB?@CtF<&NRue>GHf1ax~o4Kk!iK?vIxvIwq zQf1(D*l^CLHe*86Pu!2~27;oLMCIB1oBYLP$jIsofm@_&udbZmlW{773p1cY?}f$1y;nq-&@@R}J{nAt8(wa?avv0Rqzu zmFtQ25jy-V~@D&x8Dw<<5XIu$RoN9iHc zNS3Rm{Z1kukS6-9xV8MZ)Rp+!#Cna}GLIP`@__P@{;pFx^mvaE>-6Y!|Tg^ zz#lwmZaX(NnG9YbT(k;io^iSTHrurUaR15Cz75Wp)&6YnGzfSAG?lfb9O1KYKNZ(JCNcfo>}R4?p@ zedfr^YBAvPW!j<7L~w+w(F8w?+0L8SpU+iLLX4!nn)5%4{4Jrbo5i|BW*(jf+;VVI zDD@2-MK4^^y}+BKa}^-#<(HU&CB*%rBbII=K)OTalb>vRHVGg*;29^^51iekXk?nK zNb8Q?D?-y?S8t>iV)P`qC&8YX@#!9{H@kO>y{gxEN^klss8Xo@_pCcnT94ZI4R*f^ z+e^Pf1V@x!eJ0m)pcoA4(eYsect9B2xLhMOIldzK!CYk?1E}cOg(GT>!H`p~n})}b zYEx2jJM)xYXz5`pr^+f9%rn8{oBaM|!gsA2Usosf5=6mr%9ribJ;%z&oKQO;Z5o{6 zlRaXtK@Dwb3NoXd5DDolP_Pd})S$T)laCmTV{+}%v#-jN#31FO=I1O9ypvF<9a+uG zvj1IM_`vgyNvUZi3DP7mgozK-{%)iQy-^_oaqE%qd9k0GaX~yPJ@U&-+Oz!P0h8Vz z^(k2vG*EcP%{7+}@tcKAA&Yi(KSDHNZDvDsN|gg)Ak3K^QiD^5d5-C&En5jBloR9l z5(61;qBr5&tQZMBSW%WgkPTjBvM2d;gPD!JFQg<<9iM4e+_w%vngWx4vGAyQ#zG`% z;vsPf5pC16_yO4x3GZc@uU|7#ZrVMYvQ0+)2PM)8?RW@R7nRGty7;HPIl`HHYYIgp zxWr+X)w$C4aZSVG+v{o^fJ4|ECu2z)48?JK zFCO5Ogtn8dSxT}7B@v5YCWP8g)-xOl(W^y;FPXKPx$}ye@-h%dhH1)#I`p&)rZK1> zC~szOM=4xA=7tswt5xi4KbvX7O{JzUiEsfG+C{UNy=N^mw9=X&1o6NL={cL;cJA_s zmR*WD`4+Jt&s&}e{8(I};7GBMy#}){7{MF+lx0^YtaQ+kHH@3OCskQd)j_o%BFw4*O04C&K%gNMjdTp6O3pGY zc-s0f3yG`DBTj}DU{z7{gFv%OYUdbfm5g<@jvOjQ4`cg&>NUy#(I&Ak**Gp7h#)k( zYjxFh_LQN7uLkfpL^T_x|{`pOx5wOQ~TnQFcZQb z^;@B`Qz-*Lqj1#YsY{vdcd=_cIBE7ORrAQf9C8d~_BUSoM!ip(T|>--=B6-=*9JM2 z4&?76ZtvpZ6@!k2pNifc*rou1=UdN1_b5uKt@UvpU1f?P%tlw_X{n+gD|OC>f!uLhK_@qz&~Sk3o?W|30qG*p`Jo&vsEL z;p?qm1x;@A6vgE}=`n}Wie0qNWq%`|D6JTp@?bK@COU4mjpMiA2OF8n>gtf{s^9lk zkPo-OgBwUEk%DlQtxw!-Oq*!@_A}#J2+l-xG^WUQqc6t zxDviGw`lc%MyXO?sK8BS5jTZB?}lA2LPDGYs(1Mz9+gcgA(+_qlus7xB7aS-W42Yw zHWj!HY_ZN$`M79gYblnhz2z~BsIIMBb{p=Mk`seD_D}@wjWyY7POLuX!_Z4%D%WoX z7JM~0!}Whu)|&?i%2Vd_2Bu(^38040`v2patPe~}6g|&!BScnP`LQ@vt#SUGN_ag3 zaLM$O%4=>m3O5gJWB!0))|@s#qjez@+f*vqu*d#K zDEU;0yUCW-+gJHnp=H-=`vBjjw*h|_J^ry!qZQf29_le}Emr!?T0i;@rSg=9K~Zf9 zRH};jKHrQ|9BmTi8KY9;$qje%MkNjzs8Ji;JC%@SAh7x!<2~=zt1JC4o}Wq0@&hL9-%kZMhhiBXj z`32R}?v*E-xVkS+)i=B$W=`ccj2xMU4Xib!KN4GpPjC%oIOHNbyPG@1yT4(YeccT~pGAnzB^61z~*1~=nKfg27L zry14O-P~!DvkL7nzLQ$dv7ljR0Ff=SQ+jIOAJkT1Z_YR>0W-23d9~mi^<4taf3{>y zvJ4XGPhmcpi{3KcFUa>YV5t(cm_3Z%wO$fc(-A&rqz(1FA{S{PLeq6Akm(;U%PXY$nFfl1(mE zkKHsij3%;qSjlBQ2GZWSfXuZLKI?1m>CU1aCStv4Y{1y9jgAA_2&M)S#6%{n5dTrn zUpQQ(HpTIBQy-6m_S0`R;v&cKv6Wd3Cbh!Mve}^`QtGPp?s6hl$Z2HyhTL_Ve*&Ij zBKtf`Et>I76)XiuB8&@dTSlr##Yo{aP)N@2Y{fGj|Kt_lG9soCAcS|5mnH&xjREUI zm{P0Iq<<#TPo&UEL^WXTllcYHxH~dn_L?nDjsSSr#EDX}{4(qcmhUEF#p>iXUnbPZkqc9`bM_9rH`N zgY5R30b!zS3ulxaz|3zhN9y1;CSdvsM)Zfvug4eg@w;~naI1DIc)#9-j(fYWXa2Z- zCjQvFt?%)@p3;r${^fnUEg|Vc#k?DEnDRWu4tJP%d(bg+bNnjblaPteBHZi!_`0-| zd%5zK2o+2D_;=MRA(VL;+o_(Zz^K=!OAMLMs7_n>Eu6@ms_{*~AfgYW+8=P4^ja6v zHDVC<|M3dAgzL)wN!Z(P_ypr_MnB5z&w)Y*V`XnsxCZ~{l3BP2j@c$-4ngmhDR&Ky z)fT-BF@vAUVGe=WX8*NGmnYMG83LotksfjlHdFiNOV#Iu1TZso=^-)OF6AJ}XfmZZ zA+g%1-5~K1GB@PDyxIWdX?!M{+kaFS6B z*@pW9rDC1Almms)7GMS~^(_vcyg$;%qJ`;rq z2D8mT5XKxn^Nke-yKPwp<{NCLjNlhKqoM|rYn9n61B2Px3A5&w$*%^3)pp4U>*tYq z((yG4D1mL!$y{fI#b}dxgGGTwX6N~TKpdX`o^S{J(zjE`=Ux7|FfK|Q&rkmhyh@mr&a3aooy!-J{9gu3wvxf16ibnot^&Cb1CX ztlM8tim5HZCP#yf#>7*U&f-i8%Zcf3iD??Q8&0aQ3t17Snb!|Vork-XYB1moN8I0@ zUhWK-M$xp8sHu>(2hW6{7twi#{Z7N6!UPGTN5W<)7!pDwvcoZwQ_`sNR+f{+(4NDc zqtlc-lcA&INRCO7>kkvlEM0IWolNQ5K-1%y*DQ->p=2%7aPE&bO((%{jN31k#$}ab z6{BUv+Xj`2Q_W#HQdfJc-U-o+aPCGiKmi9R($zvOFdHJGOyS7_f!hCTBoeO<2WdDsnF#z_B~2rM zScVpq<*dSh$-@9mR=ML$$s?h*c5rKnGgq8JNP{5>d13>;DIFHI+!V6@ZN zf)J!Gg`~P_R6aWaVTeqkEa z`%Goi<1FwHlPFANr$(SOavSRF$xbLRL;XAd7qCLypE?_fjg4K=o1d4jEcFhsGI~8y z4eB4Uoc)=Nau$4ip!zd@*#$;&$wpw9Zk?F-IZtmi?RavX2QvT5toPGf1NnmZjZrCT z))UmOo+tk?GBj6v@6}Xs!pMcSUb?^(#rtNsmjYt*O!%$>`#$eYt()#4c4gSqMS}pl z7n6F}=DNqQ0pRk;?i;82>Wtpo)F}m}?4XY+`zOg@S!_ zF%!i^5+@?xx8q=ZGJP;a3|Hb{r|vd-pDCYSH}vFoRnrd85S<>Ixa#}Umk-xuVJi|$2oG)}Z48;C znLp6g>$!?Pj*fo!uQ`dl+W;R|FR$H1vnEOnx3@1JHme1#cAz2LI($(2q>xo~tzU<@ zh!ds-Luu%GT8o6k?Qj4ZefK{Bb#h+6(AOiX_Aa6!?vEL@R`2O;PQlNbf324S-POI3 zUbmx%GrxEf9Ww*Y&jselW^lG4yjDV=sJe5R>vQL^z$Co>p5K#n#5#(H4zO0FZdBwx zpPM}JoPE-KQ#L8LHqix_;b#8$F^}VD1+JlKZ{mv%&fhK|;O7S%BTm@>-#tYCHcwha zbAnnp=th`wTs!wn40^Hp`oh!mGfs7Qqs!t$3xkjLF!W$(4{JMLd|X|F8oqVrKh_rT zBlp%uIwO-}eryNF{&{SC57c|?H|_t6uN>*^7qGz`ac_V4E0FESPmtgN zXsx^JwH+d;9K=_k8T#~+2lUA=0DQldPmrE_e*9x}aAnlk?0W+zUs1dgNwkwRb<@Tv z`rlQmfnPpeJ|3ns3b9J#j`n6EzeNUdLSa`a@FwZGk{{ofa;tCO zWwvcwnD8~l(pxEm>6d9)_^>;d7aE1IijbF<-H1|ZR-ZOS=x7l5-seB54NvqR>dMr~OIPtJ{S||W)W$-bTFrbK7rQsEnVFsk z5%hlLf$P@RReZJO(dm0jL*`*)x~nvH2EShYpAY1L!4SPcu=FFo_Fc?nkA3Clp!t$e z2n+&uBKS7dm?z)P&*R-nR7XY6zS{4GU81Is1T^Tvj0TokwyPawx1#PM71vfz_umezg zUE3Kz(2my%{~*75u%x;9HiEr5f_**HTW2M;PQ#za-&9jN z&CIxCR{W$3gnseer*94?&fi<`=y%zfj@4}eKMOIISv32YFg%&74wcuP)|D^hzI z=^rK0oja`j6D4a{J?QXq3TD@Aaw%BQ_NbMML)U6YP0Gk=l=tsP8-IBD;@t6{tu7q##jYhZFZU-` zwx4!#E3c=2L9`gZDpDQm4k|tsm^iEmgtq$W!!gljV?}$|W2`YTGbaf3-t5*@tM>i& z;&rGA{qb*NTt{1%dbutk*F1s)8=8I>mkG-XkSbrym)EMvERno9w}4?_Sed8;@Q_cg zUrpqj?Mwv%@eD%^kuFA=^~odU_ncYnGy51P{|q7WX$`oR)foZNj}VoJBerZ4{3 zlSFA+_kcP;9I4*2L0ek1H})>m18*+>hUxgeu~vjgK0TA!23K6!HcJHtaCX^9j8oWl zx?PzFCtGJY(xQo*Lr05Q@6ip7^_@jky~$PDHNUV5BS7ODLRd23g!a9SHviOb&BNo0 z^p|MM3t%Er!z$Q9{h^0Mz1KYaB;e|jC6g6{tOaZ3V3iwNBFl33l#~IpmI@~ zI_)x2ulD*b`s_oL7|E8+O=Ld?jzsA+^L84)*SWWPU0;)AKjf^~7p z2v%Ca(jxu05fo#>osbtRW22cp#B1iQ2G4+`P}3uGLa4hqePUC^tr#M{U z(Os~uZ53vC(g@H3_Rtx+68{k8y>$Y z|G7Hz10uG3%iPt6moE|Y;iPIf3eqe`U=a-QryG@j0?9g98|p6e)l4*@7!D&DgXdl8 z`#P1;(CzceSFNdA)$GCbA20mjrc39pHtHxjAl9?e*J6Z#F7AGG>yWw#tn8dksGk?{ zM;LgBp8G5=%i2Y=bl)L5{gVJRPk|$C_XU%>7b9 z*;)`SzS|vRWb1}9X#CiQYO?ab&KmiHV2)eG>AtK{`!xJEg0d38p1EW0gHVndNe42< zj`Vy8vJxNyWPnWxtemunPrBm6toTFz$HeA#;8P1DvvSFrb~5as9Pb3yR`0LA%k&S_ z%I)L}p|4efgDTs2+enU9`(E^vF2*HjkESRt|IUXnf0Z^xJdDaidBFy^w@2usUSx9h zIs!x|Jz&5*LfUKNW@oI5mmFO0$UB{|m*y{jr1MdCO}e54hqsDv2?A{|;l{VSyjlN3 zScS1}&E5W79GEk0{IW+n9(9(c3-fW*IoO~Vu9#e({T?dq=ADqW!s}(p{PrSD1Qc(v@|l%RF(P?ujnXmgPWi7#)cT4+4(E*B^*Wb@b*QCdpdp?3M&|9QTU0lH9 z0n&%vN<+9AJhDDcYg9wHd2B@&4{jXPt;K?5FG98-i6@L3{bG4_7#5bO=G-oojUJu7 zs%h*ts9*dGj=d`F&*7R_qw_uy4WcH{keeiUoZv_WnSE}1l~ok59}Pu>$Q!zMHqs@@ zhe_z6h(wlDyqLekCn69l|~|((cP$q!a zZ|1?{b=C(Jtc0k>3e{S>{?)zIXs5>G=?PC|>g%(6`@ zEq`F*%yb?tfg>vq5+DaXALs41Y7*w%#G4xtfTEzKKwP|KG#gIyJHD49er{CxZ0OZo_7hIFxi@dc6cQ-zq$X^7zj$%nl~qp^F@#5HsXXtXq} z#kLGC@J||@H-ssA7VQs|cUgZST19K{CXtgWIG|_Fn4qEKK1)alR)FE%2}s151}T0J zS&kanSA6BK#us{xwFr>23psxXcVqHx8BD1xkpWdR8;R0miN2omq$p=b9bt}+rf2hy z&aN1|a#@zF_cUR#vRL>r^jEBm)WO3={F?Fpt+KXN}5DwbjZwR^bbvaS(_E5_f1s3IuQ+H^w5=>yo*>uL1k;&oHh8c zI*5PMgu}PP$$6$-X4S~tYADem(+WqcVX9Pm9_ft8K_FwZQ)4s<5Lcw*S>|DoH_`dm z&~yU9RIIb0b;&?%_DkV6tbIUotLauAFa5G`Tp5`efHhp;Rf=LPMg4NEMW)|}FwP+y?aCUwzMfbG<)Hc25Zmd zZtf(koE-lbR^#O5CHYU`2;jTsq~it)O7|(fcMbgFZcM!)$=A`BZ9-~yE_ALaXrwGj ztBz#^FOOU4%aRfqQpvwr_NHf?+cCf;%@s*Ff`XIdVy%lUnRujF#wAB)RG{N#f*@(H zpg_==LxPb6n2SN7=Ol6wypOW}mYW(S1=GMcbjvh>fYa)dGZdymdI3^g=pEQ7rH8x4 zHp2IbgVurFc##`6UaP&^R9&nImbK4j`HdtDhxViP|6G@0Pu-T;PdoDv2WDEh6<mTtbNJW`s?1kC)}UDv7~qv=MTjo^(2-S(rPbP_AX+ zOUQB^>i?&qj!3Zw7`VtuDYZG79JQ9L;^doXg};_*u`Im;G(n?hJ%?(M#qGYIDUwy9l616T=B6o1UEPfR2WgTdg8-JL#!98yT=%* zGL~k^3)s<9`=4D#*2u+r{!*a(~^7+oN75C(m!1IvGHaXM6p%s*z)pU)p%SiVY!oDd^DD!yh% zy6od~U%uZ#CRzD?s0Y4#x8tv0?m546d{qM(T>ZH+YWnH9%>J?Xf((~UL;e3XQMpK1 zNSwZ=tbhOY?O`^|A!cM~aZ z|6LHjF7!V+gRC5!slm>)0Ohi*y{08b-}Pnp>&Q%*d*NIL7N7FZ079DY@Qm(-d-M*p zgp?ASH2sA53Itrlwg{s*U)XC@Cn?p4B(~Wu%6w8gYZi`qluHTxUKlLw`lspbq?cch zSI6Vl=bqrdyjSM^p~OBK0%TrnmtCXx!A7*GXty;_oDhkuuZ+(O(lpp?-hbTA2n`ziH%At7Jib_!_ zH2oM+=}_rvZKB(m&%UMEF9}(0)zBzR4sZSX_8X2wuuP{G<1ru}@ksgjOMJXMDtZel z900lhi0L}TMH@61i6EsQC6fc@qw*#cM2+m1$#fpBgz7WXx?0i@OTm8rH^)IzZJj#h zzVp0u2DfwM%lZ86`~%ZzPkXMd?JUBl9!UE6wTqj{>NI) zDrhGy$=#?_&{REV5t3&({YG{kU~iyUXtvddQG)%Fl|EZNOx#XWeQQMr>CK4@K{?A&58cpcC3M3{ObM5*a2muf#R~c!Wyn@D6 z8?09uDrJqVz`V%!A5L^xrC zXlK2axvz!9P<`9grZ}Obu(rlO1El}rT%Bjmwi$o8@PMkbFp0WOaw+#wV4~+imH;=AdPNB$b5Ow4FCdeo#tZz1yLK z=t=eO%pfT@5__=}-VfLGG*jwX6~e{?=eINhvOW{nmS_ea^IX91f_jkMo1p`nF>P7r zvcXmjaF=JQ%G8~cTdqRkKOjF3X{Wg5o!=F6(K4x-{=lW}rJ`-Vjg(Uy$WEZAT&C>7p_FnN-_H zw{5j)CJepi+NaZBH{!RhQ(iyA=^x1Yzv}CsXGq*S=pV@Mj>Of%QCuHg|HAzGA%qH6 znzMkFI1?xZ3AVXp1Vg@X8{H9%CKQU#(%3=B!d1hSK$nig4!e@;U>ZQ1Eo!*(5{rVPj55F@&zuR b;1)d63z>uza76Izteos{l$4UnQgHtZJAcDd delta 32213 zcmZsiLzpH?v}Mz_ZCjPLZQHi}r)}GIrES}`QE9txzy5mmW-&R_6S3FYkuym(I!+Z3 z0^AI8(gMtxPW%U$IvueU1RNla(tD+GC#jyvbQ-AP+=#z*LSKLFM%@)o2AL5eswyZ5 zEI9R0zgJ)GBM3xtp~j$7h2l-H!@9H-Umt#O=#I>=s~`H{=jHeO6)TOQl`z7sue(P# zNF>9I#;`2B!6uoaKdgjeiArDh8oigcR?Vi(UfnW!rsxpediCf%bq#1Ncj*2S&No|l z75juU#bEr@8&Gk^nIZ!E?X9!Kc*Wd%_|_wjJqYh!UWc;QJ@eJpA0<$2mZ z0b^g0rbqlVeja-kaD?KDVirOvk6srKlz%5{j$$^?boh9ayD4b z#SQ2`b7;nv#9o0Aq>t>(?eQw zZ0aCv1TcGRVTP-t)jIio&Ah9eki8j+O*L=38K|SH$m#JhkpnPBW93xVnwR7ZY&*&w zj;%X3u3Z?5=Ju$@AArezyqMa@DBPJs4|}nvg0z@hFum+kMZ3)5fyf4=kew8K4(>+0v&)e( zvI*Qg`a0QT20}>S%Z_uKVi!$(;)*Sq8m0 z#cWC&GKmdlJdXVdrHOZnxDc8)zm~>oo7%wcTv^LSZXij(dq7|1gM5)sTi~24%I`z# z%#}_YF~Cwy&TlFPg~+fVRCFk@w)32WY!ymxCEEHlhiR9X3`**|O+YLQKQ|F2z zMv?<EYwNz3X2n)*;$VA*WdGH=vGQl)x(N5QV4l3daJcnl#@EWs(iW zgwT$oRj6b{KX;dU1*uQCvET}{=1to@yql)4i)YG{#Ai{Oa z5gJtt}@-e&32#K}>QpoC< z(TYY|&iKKCgam+bnN5`?;UU*@?dfXh*30nFDPHo#GL;FJj9`M|1V+s=Mjg<&&;feR z7I7q1M}hdl4s0y**J_-%E?Q)SX^h}C62Lpb5smYcAmYo=DsR~B z3%ge-_BM6th|_sk6hxN7GGfq+hDUB0>K^!DD*naER%L1bj z47dYcz=@LCDFB+7|*kG$4h3G2M9_q zxJ30Ki1j3D)qFplM}(MjN|)<@o}*+uIvPKy%Fo(N9#(qRx@hHTU{WrbqaBJ?U^Qws zMx$ZPu24NY>8tMcB$+@Pfrl&_?({2a+Y`yNhi~P^+zrzoP#BV_G=raYvHU7epm2nH zFLkjswQBAQsQRN%Q%>ke-SL^R0hr|bojc5=SUa`<2j3`2>ZkFl5joyqQ=S${tjeqf&Bq?1Oc9685D_)oh)Q!&n z#ZDj;14UZ@J#8OIJPfrpyW3Ah8~r|oy5MUxZolOf@-Ec`;y4)l(kvR^|=(>H~43!(yD?ql`Qkk$miG)D7E_zep7TvLB z5x%)X>ByDYmPdQf0iBdr+*rWoRx8B8THO)<<2ND?2C^$IuQwgQENe>jFA#v=Paf1b z`%w8uzq^W!?YNhavX#R(@w}V52jj#XqSa23bWElaj}@Q!aozyh55UG8LUt4akpr@{ zt~}xBcx0YYPcI!# z6KvDvt&nSiG|ov20$jemF-iXwY5}Iv6S>if~bOw3`7$% zI$(A?5 zEEJ>h;0%ZE=XvctY%@jb6)rB<`LJt=V~~(RttM(|O4jw{z)6$o$D~l{ouVy;puH%s zPYGmfR^c!G1#lis=gov+%i6m)DDOYq*@JrP*p9chh6dPY41&!@TsS)-sk+|@idoHm zfI(FdUFuEv5f{{QQPrZK?3_E8KF`(f{dymsAI&Fh2wvD0CL|Zh%JZXDgFpf|Cwe*B z|5~CCjNY6}Ed`2LukhAZz%UT(*g$-DG49+~Vr6_a0Q&t1W>vf+ip@OOOpMPp+j*OE z0Ty`6aogxRR|ns1DgsI|ad{vB@I6N*ks)Z8u5E{b6lVgyf6`Mv2Qt zQd;dbALlO)-uI=k_*;c7Db8*#q8BqJq)@v20GEa{9tKjqLAQ~Tv7FQUTN(2wwpfTJ zjO&^>;lvw%t&VygZ{Q;~1p(ss_hM%YPtMaxz^QDbhUAMC6}q7Zq`xo`_v%BVtkT*6 z(j`t`{w0_>yZqY&gY-B3v&xSvE}49K1L4o9aS&GRL~rhM)!QN? zBZa%$P{=jl?PNs!4u#LGt6LEbT&gUu8>tY$*A@5p?((!G!*Y_@Ipn0nY)mvXmY`)hhcKT^!yTrd-ywoR%XDf_lnN|5`q#I2tC&Wstt_h)wZ+eggD^Lp^FHgrz#0=PZfQgs z`S1QHq%g*SQP%kleo)(gb%zJH$A-?EDco0@gmT=B zJ_ayKlpV`7YFvu8uwSUTnh<*qJ;*u7M<(ltl}n7;l)Ii&fN;+{&uB!jm?7gaI6o`s zv*;u3Ueh2{5 z%AfD2T|{E@BnqbtUQ1apVP5=XBaRN=UjW$MV>X~SW>9hyfYX~CPRx(XKh_Ld>4SK8 z*+Uon!)ui_U64#5Mw!(S7znbw>$OFx+rlhN)TEw1VY%%$@_fOHd+vtN7Da|13^E!V zn1Gu_M0Hm}aok432r$y;z>L1AL2AZ%blz+gQ-SXC{Rf^#4!GICYlS%<>XGW z+`%04AdP?sW%2R@+^|ROWY6_eW8mm{u{kf;V|%$Fwh*+BX{Q;+l)7fca#Xu8p5d9zz3`HR#*E|=Do8*SOc3*`1NUcA#Y#QTc15ubaA@u zu)Mbd`;3RU{)xM44Dn2lotc;m;+4roYfZ|R-?t_j(Tc{dp*~!<_1)QB!?r{nBLc6lt`tLlda9h#u+Ak@K3~5>qkwvvhUU z*y*hUl6}f&`v$*E*an)M%*Is4p|1T8OiN1qSjs`lTEG~RFxtE`#jKXy)!3SZ5TVS; z;Kj<)cTT|#L0gG4WEVh6J)?F4D($kuowVqcs@%@bAGJ6t6)O#)*3n$h7%jz}qD(QU z?utby2s!VjZpgT@qO3oM+hG}XqU<+$2@_VWQ(XQa;=?Bp)IUw0qC6VhA9;00$C^(J z0)_XDN(DFifNVz9ML1r!e!cIG-SR_ppeB~8kPYuMy}SR#PYDouxA^DuZ*J=!VK(`D z0>?HGPh*LM=luXY>gc_uc8;;URe+r3EioZ%?Y+)C4i0Rs(@^2{)UKF)2?{C%OqcAU zwa+3GV(<vbXb>`b{l+LqQTL+V0Gzd*|>?E(X*qyRz)MhLQz=>NUe_1lQ;wTZbV zei!=>GZ#+zuKFyVd&^z8+%uucu0R@+>?zN7anYuiG*oB7WYFWZsD^w`)R*3c&pHj# z{NIYYp9`RH(2tG+fh}D|=hWHA*4C9a|eW$WlXIuWal4(Xc- zwx&1lB>+a@cjnF#Z1Lq8uxNrT3Ars;E~b}0xgmAkN1*!&au2a1e$+lw!j>gPjTd0D zelV4%7@KJ{xN0-;J8(XCGnLs&moTFVThl^l9N7C$FW@O*1sCH@K*}e*%@C9>{lFQR zx(&_<1x{n+2hL_)T3BCf2=87LTb zfh64jvxw_N?7%6+^kpo;(p2(n6UJxEr_jYh)m@)}B2*+U@D=TMIXV%8^c18_q84_L zEI#ea?De%kUGJ_DVP{9) zk$zl12N#|qmTB|ZN+=oFtBE3pV8Ln7hn-nZpyB&`jZ07m==OOW;W5&P>rhUs?g~e@ zs0-5YC07^Ln1<<~i8WKsT-mXIZaciG?9ieujYibH@W;9^TCzjz;@fHCC|ck9`o5Vn z=uTdVX@oKblrBz3(2#%f(Ibi1NT1eJIf_lB&}!HWG%ryvE#t)LDYUx=K2O7>!Zs0LYXhf1uSZ$xkqW! zZ#+u@fAvWdXvmjJ=9F zk3jqFhec<%G^pK&vkF_%XD+>{ow6W;Gd`FP=(@TMW{*>mpLHoSnOUvW5q!Q_IO5h$ z9`?NQYt04`abk7Sw70yfd-8hQi0ww{U(KMXVuU4titK>KPIsd*SS|}z!!}cik6W(g zebm{%UjHP5%q6+pWMBvx!VvQhR;tt)+QOLSciS|_D@<%Lgoy5+jgxjqB!;L0Wy7Av zB0lprsLUxVgZ(-VBYS_lta%{FcfwZe=j=tFk@?sJ6Wi6$6_YPe72&nnHo)z zH(6i9K%F-Zt6eT)ZPYiv@=m6U{xmtNyF{O2%8S8(52%%tEReKBS!rPymOiK1L(YG$ zEAN6diO%3t8BHA?G=RGmz1t7{yPpWhZy!2d8VzMRH^3kidI3bS0fh;g%=x{_4*9Jj z#3IY8CTZqA7LVT?L`g*3?Cnd>K2-s9N45xi{Xnlov3F6;p&6#-7u5o2#knZVIP>b2 zjH-wzx{NFZ>~V1(TJ=qmenTM?<-6HG59Y(Aj#T*amRkPQ95U>%&}`AC&`WNFrSRp& z>!P6GkAdc>ya3caf{wd^5rH``genlZW;a^ zF-?rmZH-}5Oor}!`{tJY5k(Feqs;1+-@@VBsvKFxrUNJmqtP2lg&>oNs2Y{PsW9Lr zvm-qTOHyRK71F)YG`*{&tF5 zgj4m`m<6Ds^5B_mgdc?fy4wIjb{QfG?r5Q6&j3RUk*X9GSkL*zst!(h)&7NONjw5S zkw;ET`v;Q*a_C;3ewy;$$aGu(&S)GrmG^8|QIP9iE9}e5!YgodD<+oPk-_MU7n-Ly zRZuHr2Y5!E?Kd1k`f3NhpkSz#U_ujkGl_TzJ^(->h}{L7Y8!#RsY%83XC$hRKW649 z4M|<_qwc)RuDw`!0QoIQng9t;(D9a-sWTicVmDYo?-v!`yL>25LShYn8LsPPO|Suf z=<2Mq-eYF{>EK<3hd*>Do0zD3hBGm0+y?utIjkFu0ds#Bp>4tU!Bf;Q!ptn+U-fpFcDmrT?|CLf2Zs!;ZwtJ>Fg|3|caNlkW~7C?yUgacwj6SE z2RzH_YS*qW4WdCsD49eFBLn8;;R#sZzeSB@R7hfGdkc`0E&}uLJt>A#MKDKcNG&a=KaVqC8v|Ur z){ko{hW4WY=v|PGtcLXbi+t4M@+~~izlLV`=|=U1*s}*@`hTS}QqFE4?`=?2K|`FF za_F+E;oe8m20TrFLJ2CX;8ZB8sE&Q_6gbU^kZi=UEQj3gT(^J`%b^brpWTje8uV2Y zb^ZyU*D!EW!4-FG6j_0=k~whsA#pwE#ZQe#R=A z-x~84L9fbB&;g8XO=jwIr%*0U>}D7aus1$iYe4os{Kt)8O`>uKC4 zf59B-DMRwEo9jYBEMclqA^;yVOs!qm?9WJM?Ffm|5u=M?QLhTiR`BSVVBf^tPwzDQ zNiV(Fr<@WPZh{mu0tnB4V>J=S7VNlFT+i&O(SQ_1&^XW|RVbhliP2UGolr(E0%|+? z&9RNrf{aR1|6)ViB8>;t-s&ipgD5ucftlhN`^wHSZ=2}w9zIZU#Q+6l?qjfJ;C8l8Jd5V@qo=pv6g=0UEv5QN3dPf z{yo=e1I7qX0#k(}hX%|`6gw5t5a;-oF>8ZB$i9C53`!hFfts&gWPvAOvJO+FY0jpz z;u(Pp?QE575ub`3LX#!zFX9*@+IgR+CoGFMRG>6qA%kLFF$bF$k)9<48Fz&of~SOL zzQH>IEg4L()#a3`|3P#q%!hJ9>1%njQjJcjSq^|9%cy|ZSKz~&tOiTpmvazJ zcw*Bjnn%3PmIT%i2n1UQ_iGFPwKXIFV=NPuEY~B%r}=p=zVHzFz^8q@Rvp8U9l@bv z;|`YiROnUvN(M+h+=jGupn`J3SV|uhR=D_grEPbaj=F#sK>S-TL@iWsx^m)4XLwZe zlSh@bE>~}=G%!9;J%uK1WkJ|k=wNv)5AM(|0h_PvaZS}6MRcSEJuVI;k$mJajH!tAivx9w> z127cy%rThS*n^=oHw4jr^)GT=yHHgcC&>BF_ZQ26qq`;TOf@e~wJX5(_eB@Sw)}ac zWQvQ3sq_P^eBUSpW{@I>j>sx5_50;DfyWIY4IW;#iVi`TQ{Sokv)9Z{m6gCpy15#> zY?|K+{T+1-*XjR8@lQujI&N#Ix){`BP$U7|pILk<>K@fWL_Yb0)U zWD+Yt&DX1Em&s23HtjVTO`c?A#NSryUlQ}+w^aGmWW~i;Su6-+PlUt>WMrH!>wL~a z>%1~D=c=VZrpqAMg`pCX;Vco_0?{7l0mKW2@k;1SpZU3y5SZD}(-s`xXe8KH9P2*N zhi?EEvqUpH>nRGZ;At7VqDAYnD6EPsMUN-|!%e&+i-|-2&8X*Kv`#ZG3GM?9wYbIi zmJo-~+_QT$ol6Ls36vR?w|&bA?*NB$$qW8`D>8AY_hNvYkvcf31m{G%#V!Ctp?ToU z#IJ_%%AmUxx}&NqhKgMl$2gOgU2uk(4E(B5`cNAaSFXh1@=Rk&2XXyP-xdD10TMD3SV0B6EL(A341F4T3SIs)a}f_nvZ()weU%mcR16A* zcOz8^QUx28GtyjhOOYhy`AkFH0Fp)stp%Jj90Sxu%M@i8=UF&kv68yn6l`h4bP=R1 zaLSoG>k)eZOvX2Ob0SBnLPgl0jKURQ85SpyKRutx;s7+ZY~pAQ>>&BUrxJ z&hRx+@(X$}1=~A23ge9HC`C!E;-zpLa@o#2hZI~_4AreCjBIpJ>5EbMW zFKl9Oh&(ItPFJiHhfMKDyhK35&w%h2vxu*2C&W#8=Yw9L1|Uf4Kh1H61|<;5ZX=tDM(yv-iwR?el8-3>a>?^2NQB zAGOwJAIY`iDUJ$(OeLVy@l*Pg8iYIKZJvG?dJq4r-#D6@LwsJf_(psKG;xjOFr)d5 zV*0RS_~e!Sy20oIN7ixqJ}_2gjF_WJ=PTe?8}q;03RgNqn_27e7n+WksxvrCGGVx9 zCeew0liRQiEBRwrqoO-t4a=Lun^%J>B2s($iDo{leXffs-LmppA!PoHdzyw(vm+-{ z3|^FQdRKTD3U)*RNz0`HQY25D(vvGm2k#uVTezNkRT3+S1y>5(JwLJ(87qBi>-Q0pp$rs0tR6y*g!`Dxka$4Y$y z!8FGaIkR9?2Out=V)qN-5S#v^VP@m(9D@~Vk@0o@zTB#uKYy$899?;u8wcCxirJD; z9aG_G9H4+5Hv?)tfOR@R9lsWjDy~4iuvP|YsWGz6aO+U_1dEW`okE z0)PAZ`xJqwm1xq1u;T=nc6at~KO5snqV&WVNydQ<<@VSYc0ownYS@%L57f;46jpw%bk^vP&iRL{$IcY^E_&-jf8ufdFzktvRkW zrJjzxf1kGjcte(#-=BX6SwrE%L-_o<+;;6={WLC20{33)pPg_m*NOZ+{~-|?vTcfD zA=ch7fBqft^6&166(i~vOph-K5~$yxSrkkcf0niMt^0)h@=!(Sz9Lk_VY=3Y4-=e` z^#@5(qAva-NrB|$O3{Zm5TRHNQ;j(&Lys%2p}u1Tm_|?F>fmP?=FCWZMRJj3lb}ay z7gV<28<6wQvJ|6cg1f-oOsWHh!`yA5z)dTZBKwaq{ zlR!>H)+4w0Vo}L726#E2dtJ{*4i5X?Cz&!pMHii<7CYo4e9Kgg)rB~b#*c8Cy_j~) z9zQ8qrKRM5*qLs4^O3t^w18b|6YLjR^&Ej_kNEFq{6ORYIEdhW{&I zT8o^Dsd>39x^t{gqV3 z>kc~MOdXmwxaPoZ#v_zt0~tnzq3t&@$y=}CtcYMqDK;&X52P!SRg8AG&0g-~KuY5e z1jGnw8LELYOtZ1C2jF5Hf>0y7yNSw|vXUJAo#CB^IYw5bFU@`L5MG zRLu)WON<%P%N!7gToUm{AeEgXW@Az$6>ogYH(5F=GC?OhO#k!dYwdK2L4rRklpF}S zp6ne~@*2J#ivO+*rM4SJ=Y@M1QunlfJHd7j77QolPG0rJ6I6EM2sk(Jh8o3;R8?u`9S>IK4M)n|H>kw{;B6@VN9)iAMB%GO4_B{#WTb_*0K$%^B*>Hr&D&!IKf2j7XN!`%RIDlo7xJ;Tbz z)A8Hq>{J%P#;#`|5WKdUesYl{MHS@q&NXVMnQP&Wk~PUOWHAn{|Imbirh<2M~!!Q&O(@z&S3sTn`Tq(ht63^RtVGohIGVBfhTn9kcD@ zj`)6Dk0v3%7w~=u6#IV@CwqoJ#VPb1yb)W_{jhj^eV7k$>kk5O9bNCm(a;zmw+-!)_X z+KhBAJy5#=MM7e%xHU|@C$L$F8;tl^ra9XZF^PW?Q`lV+o;>oGbx=x$+Kf__C$a># zdpTQ6XAyWRfod=_b$Ll@*&#cVJ|KiB#HO`jKcJO07|333qie;t6z^I24e407oS z-x=3WtXZFb=1uO{qf_o}S|j&*sE`hKN)kc&>^lJW>@p5N-^2ItBIRYZEayR2-h2AV zf}X89_WC)d|KzxKmb8mhNt6sATaxw?F;NaYT`&h3V~x7qTtjLw11cT=9h$OzVVas8 zrA+>hl6ZKBlA~@6)Ik{2B&~c&9FIS;IW!kVItus$#7yyIn3aH%JZud0{ERhs-dovl zI2mBOqWY~MU2DDp9=GW(9mL2Vtm_?GfjkQ?F-4D72SHtf#MYN=n|jGvcD;*J#zLbi z7c3F^q6oNYP``j+Q>|DkU!g2&vC$QqGNrG=vp%t|oooj)z1F+e_MO%A)0+kS6Am#J zk@#4v16B$yW8yb^`?n> ztN`ghd-Ya^IAwTS-xk<)!Mk-kX2m);SP&+9a6YPIK~{0f7zyV=A%``BpaYY2reQ;# z2?WvhXs>947tk_39D=aegi{9{!{qb4X2W!b`c^Q%&Gta0UQ#; zJJB~eKYI_BUILEzU&)DFloNY@u?-O67FyTE&&#(xJU|eXD`c1xh(7K+Cc}cAc_v9Q zxDlT;A!XkIq6^JLU2ONcF6wV_t)Mz?hPz>(&2#+9{IywgH>Q#@{eU;RngZqRI4K~F zM-(+nx=z_I_(e=*b#<+npC)I1lR=wT`092D zbpqLFWQWth_x+wixBEBeAlEb%#@%mw-asW44B`mikf`H~+r!~3Y z#7Ys91#kFcxI}(cKRY1y$1} zkx)1T*3%iaAz+H!3m_V~Ps83^ZbE%tW()Zf@k3K5#^w|a7^KqH;oqHb>jhj$)k`#4>`5#Uvi+1SJ)Mgc9ehYHYPajZ7d z!0H#iXQq(l_Hd5bg^)HZX1sn`%x2VRiP1Yct+HSK( zymtq|UCxvX2Uq-F8#LXTn9lsk9;XqFPQ;Af3P|uC*PjNYhNBTC$u9NwW~0=A`uXJt z@$ZnUdKe={ELPY?egQ@O#A!I0r@mA-Lh!hDAXss@xJbg}$UInz2&59@7pK)**}vQW z(4#+XMFy7wD0b-W{l7BRS>Gbq0}gH{6!Zcrk>&2iPgf8*eq⩔hsu+=+(Tf>tRTlsew0GYkwr*V&{*F2QDE zi;le-jKE1?N3=%TS8wN7IiOBss@=1J3d`djTe#3*t01w3 z;7YNdxT*OBB;IT3zsfh`MC83uqGJ8&(L5CqvE0ht`JaSkzbue=!Q!1;Y|DQ`TY|^u zSNnvXf|eA2_dcN*6)@R=%x~*J6^eR4`Kkh(QOrEL>_9Hh@v_=UEV$$_&$f-xVtr zELaK6X1DZYeHFB7P{P<;$YumQS+XOP-G6q7ZevdcTWnTeGl#nCx&g5ul z{852)mHR31kb0n+&EP*&SurRM!Gc1*03!T4Ey<3uA_lQsC;-srIFZOKmIuM27YY1~d9?CT7(pO)(*MQ0~PFPs#I$LEfqC{e+=;*P$k z{kr4oi$^wt^(kGqj>Bf#j`iByoX|aw zzb|Ed_zALrkW!d1buYfgfZ(zx&=<$^-urijhbgC*@aMM3mKiAeUU>}mKWFBW1Q21M zTKEc>SNw8xLnSCC$^8AE59M%Dk1;w=KYdiJxnVLUVoG|V0y>yRLe&eB=Ja~Xr?6U5 z-af?7>(Np z|0p`2SRU!6ayHT*O?H|K@Cu`31prDYo6h($ft;ZBh*Bp;e|&?~q~4!I+qFCFW8d>kDVHOjo@rkU4M_NDK)U7>Qn(w`BcKX*Eb(pn zWz1Km_>;&6$OO~6)no;2Ayuw_T(giT=uOiymqPhdb9UXeI&2vXRYWcs1Nv%oK!{); zIc6S+l%D&tosdn5Uz(jc8a8asUs>;1zfAqx*R9Ihm5Df zt+~jQvy{iFcxCR47TBjaV!3wJEFEIG2A0A0uwhHV9CH5tW)&X_m^!VFzM3&jrkhGpX`t*il0KOosN zbYQeK`?g!p)pym@ZO~?K^|&A5Ayq@tS?bn!Rh>UXA0&MlEurGI(=6L5sPWvLQZ*PO z(ot?+=G27&qfCNZ!xM_rg8mpk0NL%mGxLl5}o;`M_LDQ;#PJv)`%;kgXrB0eh4FXrb)R#cFT20pPtMWGVAh(3OXY?w{ zo=A@ZcBi|z$nZUu*QYD+J3Zb<;HK(N`ARtDA@t&@W$dbzkfizNY2&UzLx^wFs|)Da z^YY_m&j*KJ7{Ba3LZK85%=oZ|?nt2)H{16@e^`qV&M*&@>YP2ahK5VLv?5W97XzAIhpp1ay|8WtD3DMpMr%{ZNnp zCi~Z`}9^lRDes0P@f`&X&h3GGh7imG#szlt=e@OPCfaL0d zf(-db(Vz(TFUtE{NEB7tkTa>%(L`_{W27tr;~iLbb#HA{?G<{LUf@#Xe0IBlxsJmz z2mK`7hgs2WG-hLf@$%;5v-|YjvfTzEz|ODLT1CZK+<790I$b4p?vLizP$ZaZKYttKR%dN&%_ddo=V~rQHEu-HikD^NQuM{98z-)VYN3O^I)fKb$C%dfh|Bm+gR>_GgFwD8pd-20_(NdJkB^2@SDg_c3;*;xZ{)Vw^PK%ftqe%~AhL7ta7K1YSr# z&j6)?m=mU;`w>aU$^^xSOLA07xA~{fs7V0eOBc!n6#+g=56uLn1Sk+2QVF96lMP5A zghQ{(3d<=x#LzZF&p1nJiynheuAT7vBBV}PvfZZ7pC;gSz;wc(Ogj+_N8@30Lt}3? z$GK#{jVFpO4CSoCOh}BWgvhw?Rw8hpeC<=8OiTjRgh#r(88Y^VbD|PHp%Fc@01$|v zq{Z0f#2sbG8d9O#0NG{WUr^Q{IQ)OiSz?Q9roCT0l0)}(gf^*NQ*5>)$=t7qzuYp^ zb*|u$rKO-Ll?9~_QRa)5vs$8u+ZrWp7P}$C-?N`YN|#$&DkhpUA^76j?Er0JpP~^p5dev z84!L7S&<{TfWO!;Y}rmz_#Tp!?xtMis@gmRR*0s4ZVlmx3Y3BaJ!y^x{jv1k8DyNQun4Nly!Z4>s-N>x zZR@`TH;~qUlN4A>bW;O=*u@cSaR@PDOsB-FxQ82E>$s9@6CHa%`ysq&7b!3=P7~&TP@XwVMCd~MKq9+V6bk2azldPb1P~Cfd z8w4oJQ22T|dkEbyQLQ{j$=rJ0nc}1zdP@ShOeW5Z2))W`|2uK6a`G=luPKtqN1r;x z;HH?*HHpgd-*#H$!D);(IKT+Y@Ikhgq|i|(;L^3wGmx*H%*R$OEswO1MId#(y>`_` zMNG)YhUmd9koO82@;HepcXqTBfS`hvS59El_6Z*A#}pUPUH%a1gb@COfGi6ug^<_8 zZyokoIJP9md5=*n>Dqa-N-EK= z5Rk5(33oHEr@KAh<@!RnjAj0%;MrJEBnwZpUFpt{WsD;1U9RyD%B7H=U zh$m-I_B+mqef8@X!W>6ftN%eLCo}685Y^N{sh|;}oS3M}I=OQYWti!Iw6X@Mi)^~F ze@4Ya>gU<2y?Kf!e}Va`fq10yjRh;c{wQlj{eb8b{L^5>;%IJAKC@Y72VcGbz@Kp; zxD71yD7#z|Iptr4e;n+ryOg_AFT<<)GfQde_a(k}u_!+NRzD>DdRuk97K-w=NOdTX z*;dfYJQD23zSrR);R}u_(NS<x6C{7?NApsf>s)QaxEXJAAGWz4}$s!t}}H7*iu8jLn#JU_i<&OE>V z6r-)hLgr;&j`)4e7mlfQJl=+Ffs5vct%TzbASlRb+{hi1jO2 z-H{E(2l#!TF{s<~{monSOiH4Y8b0>^Q^-sSU7Adm7IN?D+>_%A$YEHqe!2LlFSLCV z@!P0_G%YsdN6Se{F=Nt`vAuc_+F6!yQ>nB|PQN=RxH`?>WnsE$ql)7PaAc3s<*9GA z5a?-K&-ne-t+Ku=JXbX_n0ls{Ep~{d8#HdaSQmcwrc-4Oz>z!s7!mxwq$PdPSjil9 zH^~!Ia~0FiN|#UrY`eYIdQDW9T5V%^Rc{JeO6gOS_6wL#Saht^&HUhwQD7=Bc$zY{UUFaDBq3Cp2|rr!=|q+xr^@R|b~9|bC6 z>>fpzXA$kWX5YFA4J!c9iR6%mhAF>{UUv9=u}5SjQ!YjWcz~m^klCEJ&2sm3tn=)Y z=$9F`={edhTBzD&T6oV9T-pN8Y&0t|I zt!($zl7>i@e->nCmS5PhsC2+*|DJTidpIYTLGL z+vd0KIkj!u?y2qe)V6K6ulK&+yf<^-A8)cJv$B$%?3rXwRu(=ng=MD5divenjf?7u zq|r76CyT-3wf(mjpi^oP-WSZ%rDazsrm?_{R$Fz+U-;cU9)8R!4K33UFr#aKemC5( zTv3)H=7xE<>1WrNJD-7@ zpo12@###Z6XUDh-sZ$#?V;LcMC&L~D5x2!pASKdaSZ}!n*yQOxnU~z!uv_NqedXQI zLcW5Ua^AkA7OHvxeH0PcXKSCl-@#qD71M+RH{$i&o1sQ5nUgg0I&Ii-_{DnR-Z;sV zig)3nr7>Qsb6I5kS*7ZWtb#aMUxA2=80mb)Sygui6uy|%T6x)(6E7AIBgRnTJx9T- z8KgLzB}gR;xLR20>pi`8Tl~^Jt{AL+jh6EF9-Xjr=;d384;MGE%`Ex$_Eyx7hh>_4 zj2n=JTU)N6!>d$#H0kw?ZQjqO09bE#VMW#UZ>?&~SeqL}gX;h7fc83Nj_|_9kO{YN z-VrN{4N8nvJZl^WIu6ZpeY{nBcsV8c`&leBIU<+@IA3=;{IHUhcUXqC{pFZ`{~-De zHC*r=IlR_cevf*)aj?YTV;@BF+E7|TMxiyx;A@TYU92(>8)r?FrhXK5cnQQyp zQ1!^!nKwPMVtLOpL!$G){BE`K6NQ##)8iQL8>eBBO+{KMra{BfNnd7`Xjh;TVKZgN3{n7W+6%I@=~hnK zx_G+aE?NbcTGbIl@KaV3yXUb~4pV`aRlAkBTB=Y|fCCaHQSbZS1Z63q$H)|%gki=3 zqB9gEjmsaMksZAT5LDPEq9y#&8r8Q{o75!WVofF2xB6CcAnu*ld@EcsP?!s-`UJxR zS`fflUMde(I2t01>7XtVcb+(jnm0^`eZARN%qV8AD5$?oa?6c$*|A2T?}Ca0okfvI zotO=MxXsJFK>Py_LC`N;MW7+Fuq^@sJg2x?=ef&EkW3Y(Cn40d8N=ooT}eBFzr<{E z>@<>!t&Z$E^(u?KPOb0%f{%nN){*{t4@m?UA5HXP>w{^C0C1J=085Ppnn-56)>Sod z;}w705muCXlf8Sdm@gJU$Ug zb9Jy(=~t=eXyLDmovedagxa< zc%F$BO1%9@n4<6$M${(1*G-$sy(N1s4TA!Wf(g1R(mCur!=s4uBWL~kZ5)D!n{ox+ zqhxbY$ZbT~sDkHnQ;Dt{aN<)}Y<=SaE7wk}BbHUi$@cE44+cX80$j#?+5Lo#5m-(S zBEpRs_q$J)6`cH^ZD|SU2lcLCvLZm5DJTEUj7UfE;dU2K4;eZ>r~rr?1NaDiLW#IY zoL=wM8wg1TML7K-G&LNU3EG|(L@-(#?M$e483$^uW+gzinXXd=Af}Cij4-(i{qZbY z z_qTl^T#oOBFx-xqNR%c0twM+G(c2>U(>vT6MHQR#rXCzM|IbxA$J)*;4j}|#a$V+N zr3P_ZUc@pbWy+;4fQ@gj0-3p*p%$+k%VgBo#2j9Fj({rI8^{J*ff0YX-yS8a=2mxC z?w^{8YY2!YoV`SF`cVC$F(Zp9wnG9JdLV>Bp&9}-zfa1K&xvm9NWJrsYYXX4^rr8**olC!y z&v(3^>N%6tKh!3KFV;pH@J!?;MMoQMdUkwA?KDR+E zxNO=NicxYCfQfO4v}g$=Rvbgc5mTky-mhHGXNaT#Z_OPkDQO>OM~%`quxNEH`oQHi zAw{Qq1=E>kgbi$qPvlp3QOF?1^bh}3qR%4|0xuv?zDn}0WC!ox8g%L=BFX2QYvGP0;PJUR)q`b6QLB~K69LSNzoq>ai-I4umAw@8dVkVg+ zA_>lt8+l_uP3YG!9&URR

fn9EFY95rG(*lTi;O{s9F(Lo?9sV7)k5eRA8V1&fyF zLkO8LH7$LV(s((>w857NnHFN$Zz2NNI=UEmh}x(mPz>=b*Wp}m3`}Ae>l^}UrAgf+ zU#z~f#Io2qMC}MMzVat}Nh0m4TP&Zv11Qx{oBD460=OK71_2ou2~sK|q!lYlk&~!j z9JgUT7&I2JQ?jVdU*8ll!@p3%5p|4=cs`2YRV0o@f>CJ#JpQ=cUzzql_Ou+gdgIZV*mz2gOuZOl=Oq1; zgHAmgM6yQOcyVleDpV#YcnnF~3hZ)TzS;rDt1b^<>!3rodXhjUh`g+}@bc z#WaW3AZNHfq75b^mwXHQ1LAbr%7$6OMsqvy_g)qhgjvNhl&pu3 zD*;dfq>@;hviY>6fN-?EXTfX7$?f#9dm`N5#|x+V_DhHaFPZp3ooP>d2lmJN_W zU#57bzDS%b)apXm8U#gF&VJbgJ5tz;DNW?-(*!P_z5OT=Bv6A$ZAinX_rMcBj|QL@ z1Q^GI7|-Y%SO(+~3=UIP(;dlqrNvuIIHL(2bzLr12A=m*xw^2BX{!(yD+0iI^z7t* z|6BU0MpW*ewE>%<#{V6KP62gW)n{GO8eVe&5rf>XswRFFEpHu;D4MwMpM-}p=NTeu zC*zOH3)EON9I=;YSlrbxk|j*Lq3=s}dRiGluCrR5kX1;sI#p;^Ka`H^Vmomp*UCfz zN#;3Xs@CGdTMO^}@-U3NOdB9!tcvf6w^xu2WC!&O{;J~M@aEui9rT89w@ybg^tbjG zaLF$S8P140r%BlyE59cW*J|*s-<8Y!KS){zwH2FsaMMM9d8rQ?#*ZAWYR1nDEFktz z)Zvyg>CP}`F{yH^k~Cr_&w!L0G3=`xbFgR%a?myFfnM^eU-Zv?J&XY}W{ahNmzFV9 z+*D{WOB?<@!#KKKBBoj7nyK*|A!0g|u*tP%D%6rFAFZ{zqfaM3!$i9mXn_9yG0XXs zrRNckyM2iv#I-o}#9HkV@w*d9LV!MdP$N@Eh-!RADTox*y6MYC6F#0jd^XXZ{k7LE zZ2@UenqELwJhNfXMHB|~?|(ggJ$7<{O1*cAQno*Pqf5-uz(FJN`a~Eny>`^3UV8cC zc8GB-OB=|W>1SoCVfjo5hc)^V0|zuzQ(va4CHrlWh7eQA^PAllR}rcp)!LpQ-1)La z#8>jHYL2~;bSAa&V7Bv>>g3@t<+Z7q(hGrg-#sn1Mnq$YnPrA~BMyw4F7o4FOBMeQ?>e71UI(Y2dD z_UOAz#m5p{igf}Yn4Hzsb%<4L$2VE<%eIS!_;scCdX0bT4Q~)~*`fH2v5Y*!F8p{kL=% zS}Ji?=XqITlddOk{%$S2wjTE`roq)qT=Gu}#OOU*u#^JO)DDv)nP;Pcvq8k!4W>;I zTl`?&EQfYuvOuv8SZz#!8PNZoxk*IBP*++LiY<3`tsgH=T;;I(Uiw`-wTxen?muDu z=0ndXwfls6zx&-H6ajp#TW z#+g?hxz24UxS+IfVN1kpW)b)su)1@6*sZL^oms9Jy|GU=uCM_ek10pDy}Tw5Sb!@zyt>hwb6gSzU%-H&^#DYXInwh z5_g4ATUYBqVIYAy5{-OOTc2A&gCKzCTD`kLae=Y%YKG_k=kEUF>czqNzpXZXJ)l$I zshoSDGzg3^46di9Br6)h5}a~@ZdbZ-r0-Hed!*w}8{)6G}ydhHv3d)AU zY3!T*1aRdF@3d*OPOQ<9~qH?*|RYCg-$_a;y zE|IgE!P=8WrLInU9fy^I{Y_cDl5GG&j+C8lun`Lc4H6}hA-Pd5xN#ODPSj+i=IQyZ zDfkPoDbMFH&(Od+&YJdQi<2rt#iojy!p9l?etIEAldc(TzHHj`Oc_9n9h!#eDL+dS zD{#YkgIK7F#{8<4uO1y)HcOM)mkcxH9Licr4a|))*G2XZY+unb5-OW6SvJ_VBAmg$f*QgSjNTEEE2NLdn9;7`;hO1p1lrz90$N$*XL2z5^9UujL#0T^rypOPieakv(@v?wEm<2FH~9=+sx!$ zE#RGpD&!Bup-eijc+c0(=Q9IluD`TtW7+2Z;E$I7DdUYf6))MmE+9|IZrnITc^Hs4;b8Be&u36Q1snA+ zlxqHZc~;uII%)BG3D~OU4CSZI?EG0b!yU3+}%)^2xwe>!`)z0a=`mS(PVOjKTPG3d28H7Nf3fHr^bg%-Xd zo_7hM78F} z?qhZ;GAJITT%7cBut3a-M=(kQ!n{VQ6@W1WV7ZbwUpY}%F?lxA?@Y9Jod!hs#J3g( zJPzy15uuziy*YdSmWP|H?e)SB!OKC@JP93L+y!#ce8Nt0JuNc7oCDfMsyc^pm0tJ7 z4-SlML;-Aa*vMpJSf~%cUckkv%9+j}ODf2HE{z{&N9(wt`QP@B*N7Udd1JzC0!);X z&li7)n$vzYK!iM$GcHmw(m}YzrZ0R{%99_J_lkpg6J>ptfk zvKoFe#>2H)=*gJYLSm*fh154l1B8!Yg~kui#&4dlz7dH>NNgwINoe4BNdLZMZm)3` zlCt64#ZPyQXTO}<_xH962`-W{6HFFZ>{UA9%FJ*)-=Mpy580@VwJbWxpy>K*Yuu+hB$r(8lysIx15p~kN8f(^W&H%DTYGiPE zh}S6cD0#crXiYDajHfJmb|U{}>T7H@lDRH|v3G>X{`RBsb2c7B_dD zz}u@dQmie1u$a05!E$ES&N6<>bo_}FpgkMPedqd6zCAtOk9h8ldGZf2D+#x|wfWx98dJ$0NnTC)zv#cZ zbpKNQ@rJ=q_{)*5dzs{gKFU*C{p}{BO33F(BdG)!M(aB9Hl8E90y;iUX28}n5_82 zePvcDfeY30n{e6 zFHj7v8;RS&)Bmy^zC1}10nTXdV*C~e37(!)=Xy1RPRh_<6!SVp);}^BuL?sN&$|;m zq21wf%S5d1%joL&c^gRW@vFgjk%NA(1@r+8iHD2}QC40u{jpo7kZSMk+LUK!2C&z2 z6|(BClB$97NYYDQ-4YC8Rh*TI^uquQx=4z&y!{TN6n-@;bP{zUAeWy-os=|0;W_A( zOq?yO0|y&4p|{GlZf-I;aLxwO6Uy8zKt+M4CYha?O)0*t(6~q7B~wEwGeP6SO*f$ehoxdMoh}F z;DzE@%X)jZMf?>2P^vRr|E%-WakY$2OrsxX2PIC)A*$3Ek*&REtc~>q*cFIM7btz=&6->`TQk{f253 zJDb>G`(1AWjOJ6Kn~3F6jplD>oMi0WCynuE_ch^cn5z#5M5^N1_*U8zw!v&q6{r+2~mnzH%p{5fW|dAVxAf{R6h*cw7Wy5mb~!6 z3EbG^;^~)Volz`ySCuk)7N0t_(POCzmn0{?6L(`X|2zaw;;+;KPSWl@QA}!^Ww5X* zs)m_L9ZY=k1CO%qPap~>8JLkbqTSYGCTlmC!%@NA6ACt&Q^!0mVJ>|v|7mIa*>CAY z6No?)KrJot1(LdyF+%?QNB(#Do$Ctz!%PtiB=+WK-M+`#I#XDYqzd-&f?)Dx|BC+M zg6p+k!@qo~=~A+!GwPFqJa_sFM2{H*o_{hZhU6ZajioY$2yl)&H1nKt7p(Vo2?G_m za_#~g{8S#*SkC88L)^!C<(F|c4jedA`~o|_02g19cNIPY$Sx?7Vbw9h-M6&^l8$&O zv*Gx6YpfUOMRsXh^?$P8UW2ugrgU3quP=ztSs510{`g#vWkLvXJ&@#-kXmU{x&xK8 zS3@<)&wwU(;Iwsc7x%Ck_Tc;bytSFE*pKa#n0g7@E>P1U{!3irOMkGQEUSbCKMA;=|?E6A6ru5ID-B&_RN&?-IzjLNP%s5;b=NTzxgY##39x~ zEZ6dFo?G=ew%sP4YUC{$u4!;(_)^Scx}%<-dN?z4qTA|R zh~&g<1M?}d;wcraTuTLLl(YVvO++$&f@j#MPPj7iOflUAIUIwe^2k0ej}$MS&wQk3 zy-QDi;pZ@NqAlWVt%?S3pyVGiz$acs0}|ot?8GnGzEu;Hp2`JK4(d5ANX+o1X?;pU zNaJ5`x~=nTM%Fw6*Ty%$KPYT|iA2&1PJm$hyGxNEYGyyo((z@fvP2V(vCY(>Wmwj& zeq-m(V-+tCv7x|=I6f0OC}tRD)0{>U+?fA7oaF% zkCNlpE+pW!-%Um#bo`c}{&Z~`w<|O^{;Np)`H#*LN9(1$L^nPnqD;<*YX#a_1Cy2C zi8Mn-IdskbkIMp|2CAFX9)S3)$lzZJkp-_Q?|!&AS)lijHuCzLn{Bi+23Ct5Vqo?W z5L1GwQm39)Vs>eNh@Yb2k{)%KUUtd@+%L|YnnBG2Va*2etGuulqFwn3+sya#F7RJn z$P{CS-qFreYD#KcVFOX=z*aYTlNrA|m_}xMNpd>0#`Xt2?q`7jqMGF-&Lt8xsg=6> z_Utviaib-mJf)pd*`b1bo9or>sjMrNwBA!;*Bx>fW(Eg9684DW<}1^PQ@nPkGruCY z93m&4sE*B*=-1SkJJ}CWWL~;-KQJo>$b3?Tj3H1{am~QcTZ@K3lkiftSHLh^CGSD! zol^N9z%W{w%)oxRqzci2qqjP)fX(2hGH`&SwW>UTU2~=m$o`P(bl|Jdsc)`74MaJ> z{n1mY1Hmy`S7gD7sZyyCAuw7I-N13-Qq_zh&|3Ke!7rjyyRsnATLpfqR8lYBf4El0 z5GvfMil&h0t#errxCW^eI- zJQ36qYHEoz6jtjy3luACDyb^e&o>A_nZTybcm9xnrJ*dLQ~Md9(ObDxp)xE|Rf?g} zT8}%S)?89?zkaw&4A8!AAneSk@k5}ftro@5&6>ch|5ZS5W&VOjfJS6x`ah}wRxTEf z{|*N0($P`D<-+iPuASYlQ<<&qRBdin0~R&}fd)nqE;;HG1l_s~T`{LvHRw5(JZo9N z#0~bSL^)BfEWY+V{#72Y8kZ$m@dxc?9fgiIJ_<$J1spR=(TGe}vfr6zM;Mi&DiBsC zEew;aG(_tUW|kv`or1Ge8zY)FD{^~4UpqM<=y+B_k%TO~9E`HR7gxFW1TBO72^Ee| zNT51TOSyo`%(3s!UeHrflBg7@UlGN4n*>n0@<|odoGys)&>waTh+twk4wxgOLCLnb zb|Q8S5N+0astBO?sxdQ#URc%{CH?Z92&*;=M`CAa-QLioFvSb7dE`KRmqJ6Z3{nk% z;|!Rtq8)T7Fx7F8<6orVC)&GmFxe_6R`?GstSk~cj4g}V+?}|nzPM3-(2gC-ptzeB zXAYERV9XR_jer4{DIXm3UAQtZ2t(~SD6Bj@DbQ-A8X<(FFx)AqV5|1fS9D#d2vT9f zG&r8#&;TJUIH(@88RN)lCz42aAPgY@JUR1;>HvJ`{{o-L602#qON z73@RrZTTRLS=_PiV^O~cx}{{$fvPnt?(>4>0(rMhAVhf}aPqziyzwuH2kE%*UkG?2L(E zrV9r|m8^{6fjN%gE?5YxQjYM}ArP4yF}TYRE?9Q;gp)>9eYb`PkuAhZn<=CqtT3<` zh#HLHi&W&y_H-oRYz@34O0*pW^Ci!n3D%GcaN0xPDd*XDrK+}&*)f+y?Nv)xweW(+ zXxE{gm`brjEc$w{l(nze;;sUKXmpP@D>m+l*OP9~07g=25X zjb*6i^#WfkS3#lSgByqwZ^+T9X zRmC;R4ujySD>Z~86&YC(VDxtYDhO2VJ3QQs0o8%v+tyr$Pjvb;S6vl=S^z!SG|2|8 z@!SO8J>2X04a>~;$>)7;*TvqkiBLoEm}j@aqEg#l;=?r}{V<)MCCE5f#Q*X*159iQH zK7r}{N4|Z_)Ms`5f{14_reV8b|AOp2&Tmq_vGIbpbClo0K=2N5enHWh4o7M7!iDqL zxL!3HLC7)iMb$ZRbp?U_(K$oV5S|;0IK+u-`aS(M8MaD=Yms=tR4GpS4a-b`1iUdXG8r5PW!7FJRy)6OW1C>)c(V1v@}{lJ%ON^6COz+t zUT(f=P?V|TDUf5Fv@jf`>B!j$n{9&uC4pf~5U7q)tjQkL2d}qhn~SUZB(?fA6UxbT zhO2SC8E!C9)1fuZj#ZJgPx<{`(;{>)Jrd&>mUh$eIBmFG%u7!tL6dWKq@Wbha)n>t|b$Q^7MAh5Mi-SfSlwq*y z1(s+MLZCo2AawiVT1xlB?3=nIalv{?8X^~wKBmW?zS`WYY-6}9$u7rbW3YFVyhPuQ zwXWu>DT7%(CKWXnxnQhcTWkMpAa$bFKn7z024e$d2}ko3`k4D}krr!93Iqm$TLBQN zLnG$Pv-xp0@2yYW$>{$`XkvXVF+9id1_w_8_I>9MtZ6 z+}xV`K_pO;e%k4j9Sezyl7OjYDsCHBmyu!FSi4DYOApT^TWpP7+N7;0u1Q*H-g~^- zJ)Uup7bf@ka&hIuo>sro)@-$L&V1 zC#^L%428!MDm=@pSZmbe(Y}DAx9PXKg8rqi*Pa>JcV(1rB|p$+`kG(84Nv@39aWp4 zkbG0$>q|Fn8|#6-Kb4Ovh6lJtbgFy$ySO>MTymm&h#3zY1}1xY8j*QMZicjNY7e3K zYi`IA0OXI$vuuXe87!`61lqv2OnJF-7`AJ}^M#}twW!jprb(eU)y>F^2(C?-E0!7d z_}k1Fs*DYH_a{nWlKUM3m>U|Ek_h=*8mV|p1K!L-#q#7 zdMk=6c_%?_+MvH-mRc@ZEIU1lcY+eE)Ei@(N!AY!Uw2bEgf`UJU_9U0ON-qUA9cB~ z&wrf{jxRcSO)Aa|%|&>IMGmdz{-JI@P9Q%&G>!M=!3KeiY;LF>F7*E3U;B5VjTM_K zzHFEp{C+#JGb~poua{SS(RwL89Av*bGhBA& z&D42mnK9Ng(fjpj%)iO*AMMg#FI@qH%ZZ0g<-N~ZG~G*g*9#Y3l{p);0fl$WRG({q zKCW%PQ|_X$!PaO47$yW8^oasLfqYR5@-4UC&~d`O1GHMbkUqLQWy;no`x=H9>LnH5_Gz zj6#?CiDf0u?<7=<*l8?mrH;qjTL^ks)E?bIYDi+ieksUh8*-%5O9dqYkg-pwe~cdK z#Eh-Zh@_zC^g;oe*y;H7+*KhR?3IL>;k$-~YKp@_OfZuGUYSm>iE<;aQ3=_JRg1^C zhKGw7#YJb%^c=d94*5~E*yn&er{*>>s@iH0Yq#B|6d`?@Yp-D}Qk8T?h*UGCXy zVZmAjqW79?e2?d>cq#L z0TG7FFGavoK*r8_#s;*ZiM)xSGqD3T%c^>;c6ebx>(5P<2-BkT(6Fsv1WKHE7-WDk;x*(PBr&#<$#TN@1TI^LANdG!y@2G z_gt$waAfdydQd9*J2#aLpGNtEW`LXN?YpA_`OlEVIfM;+VAXqO8+U_v8B)vU3i%XA z;01bRbqEVO!^NsgT=vs-q!z<^ChsnSGU8R~)uaTtj;BCdcKr4x{Oz_A>UTB4?RG2D zwWlbLA>gm6V~-&@|HLg8%^W5m0hlfK7#H;`MfOZ8zX>4ya`OIm+e^gfRKJ z5A%;7bRxLCJx;hB z&yL{*l47_{e=;ZB@M@-QzY}kKUoJ-;?F8ZyOeVjZ(e3sbw!^rbXiVm--2i?EY=`*t z$OCUbT>e;MW{5uwB9V4MxsXYs@DGsrnf3{LYchg8rjgr_a$sM&{vmq8t&ojBddl1% z_YKKfbF zQXu>ip<`sb3I@wOaRf!=B&p~`Gg`U8{Ni4@^wYJW6q^)yj4CQQobKE6=FXRu_7CE;U6~G96AS}?hKYUjA&-FECx4*G`%KqpuK85rXH*wxeZwU{e@Y)L{ z-fhzk?^I}2>f`s${!t_OfqFEwU6N{k18TIeD^{C-r~hW+uVGzh!9E4lm7xm z2Zt{JG0xdi);1KnR)iR~7Gf5bpHi-F$Z4}!35qprmLqFaM-E5_Suy&Pz}ey9n`;uA zW)kBhcjvB|>~%~3zBd{pT_Wg?!scu$L0K1vj|w{wkB@pkt+az6wpXQAi(oh!BWI|| znv4apgTe$L8JWa~CDo;mRio2=GQO;vTq@Auq1{c;qiuIQrB7Eima%;TS=nXA5C`JX zCGRzKu2~POP>gdj2b+mnI67sJ_KJ91uzt*9{2|+fUgT5(%blra%;{KF(gML+Vovy0c!bx zHU?!A(1wjoy-92>&V$r27xi=RPS+fXHkoi^9K5@Vtkt&}NS;(O0Sa58UOB{NZ-y;9 zccB$rV78c*o>C(VVLh1tI6wy;n@_AjE|~i)$T*lgjhhy{P#jB$4Grr0JS<00Zp|G4 z>LF&b;R9{2XopJREf*|qIKV24-(+%@R{v$yWQ0E#r2`#Pp8nsi~o;t--v6 zk+hUP{1(JcYz#hphBL?=%)AP{v54bzP2O8i0$cf1hF;0M1r#$Ewu~OTS~C|mdDmfF zhmc>e-lj)~@_ZE*y$EbK2va3fRt5_g(9_ddhF_voWhEkMET{pmQYpJQ%A{lpWq_sD z6qpqk?du3-2Z@-7d}KFOUo;E}A0^)|F3c}L%9uSa6I|H!JsS5CxOJp96*3eE z+Elen1brz9Bq=r+L;`1Zw#asOQMeV0su>uU(u-VT?$=xjbhbx&MKBImLZl|X_fo?# zRLx*76K7CA6E_fp02ySyIe{tU=?$i902jhZWU%u~BbwYGsIwq6evn6B=_sfpu=1Xe zBQYKTVlLDZiwrX%Rmh###RTjCI1D(Q$Z$tr(JRi9lM7otcd2!M`qQ%+m*F|{a`W7* z02~vMx0>N9Hv^NNEhBZ(Qq7PpKrOgzs3LyRv&@I7ir?0r<2}qQ#g8c~=Zlbsl3J_< z*d<)GI--ERJ#;avN?UA79X%1LV@H@};wl)xBGW)nz!|yZfMzi#4>6?GlqlrXvND#J zChBLaJNXlb6Is7d=NNdBStF)uL)=i0e8DY*vN)XuTeMI*9jRrX+u>PmuVDnNbg2vT zjuZUJOdervp}De*g?mP8ovUMMzADi>v|VtXu-% zf3i(>bSLbLXtV^b(zlHQD&SLN-lMQ=|pz&d%1EkWjXyl@Pqekr*#hu}Vy~&iwM9 zJ|=|Wp7v44Ry~V^^d=j>**yV0?R%3|BgXXlq?T|vg~R5FId%-&zMHjx%l3hndm}o! z6#qYRkqOOU*0ea4;$me}tT{=^48A?x1gZ6`@DvbXrVWA(s@V`$9MYe`P2ofuCidgO$tI}2d}e{|;jtEhjqZC}*O@=kzo z$f4(2qSd=~tJb0A=k(3OFq}Oxpa1Czvg3-W(Br?MA6bDTxy4@G^644`iLePVv2$>-a)@&>u?jOWGc&TW z{v;tWexm<>6NI0E{^N7U#QLLQ{jY1&$d|M%b=B}jZtrt>5+{pRpXseq*x&!L%4HNw ztruCPP*|my|6{dFY!Z^blqr8xcLYT~pt6@htCrMz+X;puQI3GAVE|XMhUI@DDj+HV zN3SS*{d&9G-FVIAX>Whdoa8*-05DnjgNYl_9_jX{`@FT7n8jI5AD%KoC`}Fl3arb5 z_$|Zr%Yt#9hKPZ&|1QC7&Vn@0g8Qr_KuWocl|BS3$ARM#;W`tgvYcrp3wO5{%(#gI zqX{FUi9p+iGP~$}FLpbxKxG=T*apH_hDf_aZ(F&btBSx2ctBnR5uBLT!MBLeEdMAkErI@vW}1v1PGavd{Umt7y=5HF;w0D+rU|>jL#3D>@ z!RPMvNrE{H6nH;D^ZZOXa4=p|Fy9gjG2VOl?INu#YQnqq1>Ko~gZc=B10d_Pm;mzf=L z+fPu&PCrapiCJSlM^oq$NOkDrQWY8Sk2Ywtp82;Bm{TOi_7V{^wz*6;_hbls9J5Y) z#d1R)&>hkGMb0wYDDdYuuBjRnd7Oj$mA@TzkTF#-ouak-u8bTUvx>!vrfuDtvn}xA6*I>8mfYqL)x0~NZunf=6`wkJo zct1gx^U0exzSdhd!n_ExKMCfWMeM-s!cfzjKkBMOd>R143lL#6RoZu1E(kP|voJ!4=~a>h=(y&|&4c@M==C{iHd#Jg&2vM@ z)WJ~FW>n1YkkJmn z^aRT62fpbNapoc(?7Pr*JQH`pa(M^cqa9ICCgw04!jJZ&rcx3b*w2B+p0R4*Hyu&W zY1E|y*026d%*WL>o$MMkr=$46pM1KiEgYQFdN&y8oCQw$;<9Qze4yee7B~P`{Ji9$ z>3Yg0H#V7=ehq|kmyMzKDy_12H+U#D{;c^hor&`h)V-4U%J zM7r5wwg9oK1r8W-3$anmBe;#1{0%zIG%ph%qMOskd-{rlJ ztA$FL