Morten Hjorth-Jensen
3694 lines
440 KiB
Plaintext
3694 lines
440 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7660a7d5",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
|
|
"doconce format html chapter11.do.txt -->"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7174820b",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"# Solving Differential Equations with Deep Learning\n",
|
|
"\n",
|
|
"The Universal Approximation Theorem states that a neural network can\n",
|
|
"approximate any function at a single hidden layer along with one input\n",
|
|
"and output layer to any given precision. \n",
|
|
"\n",
|
|
"An ordinary differential equation (ODE) is an equation involving functions having one variable.\n",
|
|
"\n",
|
|
"In general, an ordinary differential equation looks like"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ac838a12",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"ode\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation} \\label{ode} \\tag{1}\n",
|
|
"f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right) = 0\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ccee4286",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"where $g(x)$ is the function to find, and $g^{(n)}(x)$ is the $n$-th derivative of $g(x)$.\n",
|
|
"\n",
|
|
"The $f\\left(x, g(x), g'(x), g''(x), \\, \\dots \\, , g^{(n)}(x)\\right)$ is just a way to write that there is an expression involving $x$ and $g(x), \\ g'(x), \\ g''(x), \\, \\dots \\, , \\text{ and } g^{(n)}(x)$ on the left side of the equality sign in ([1](#ode)).\n",
|
|
"The highest order of derivative, that is the value of $n$, determines to the order of the equation.\n",
|
|
"The equation is referred to as a $n$-th order ODE.\n",
|
|
"Along with ([1](#ode)), some additional conditions of the function $g(x)$ are typically given\n",
|
|
"for the solution to be unique.\n",
|
|
"\n",
|
|
"Let the trial solution $g_t(x)$ be"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "39d7dabd",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"_auto1\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation}\n",
|
|
"\tg_t(x) = h_1(x) + h_2(x,N(x,P))\n",
|
|
"\\label{_auto1} \\tag{2}\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "74f33170",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"where $h_1(x)$ is a function that makes $g_t(x)$ satisfy a given set\n",
|
|
"of conditions, $N(x,P)$ a neural network with weights and biases\n",
|
|
"described by $P$ and $h_2(x, N(x,P))$ some expression involving the\n",
|
|
"neural network. The role of the function $h_2(x, N(x,P))$, is to\n",
|
|
"ensure that the output from $N(x,P)$ is zero when $g_t(x)$ is\n",
|
|
"evaluated at the values of $x$ where the given conditions must be\n",
|
|
"satisfied. The function $h_1(x)$ should alone make $g_t(x)$ satisfy\n",
|
|
"the conditions.\n",
|
|
"\n",
|
|
"But what about the network $N(x,P)$?\n",
|
|
"\n",
|
|
"As described previously, an optimization method could be used to minimize the parameters of a neural network, that being its weights and biases, through backward propagation.\n",
|
|
"\n",
|
|
"For the minimization to be defined, we need to have a cost function at hand to minimize.\n",
|
|
"\n",
|
|
"It is given that $f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right)$ should be equal to zero in ([1](#ode)).\n",
|
|
"We can choose to consider the mean squared error as the cost function for an input $x$.\n",
|
|
"Since we are looking at one input, the cost function is just $f$ squared.\n",
|
|
"The cost function $c\\left(x, P \\right)$ can therefore be expressed as"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2565ab16",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"C\\left(x, P\\right) = \\big(f\\left(x, \\, g(x), \\, g'(x), \\, g''(x), \\, \\dots \\, , \\, g^{(n)}(x)\\right)\\big)^2\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "83d4ee25",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"If $N$ inputs are given as a vector $\\boldsymbol{x}$ with elements $x_i$ for $i = 1,\\dots,N$,\n",
|
|
"the cost function becomes"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a7a3a709",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"cost\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation} \\label{cost} \\tag{3}\n",
|
|
"\tC\\left(\\boldsymbol{x}, P\\right) = \\frac{1}{N} \\sum_{i=1}^N \\big(f\\left(x_i, \\, g(x_i), \\, g'(x_i), \\, g''(x_i), \\, \\dots \\, , \\, g^{(n)}(x_i)\\right)\\big)^2\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d3ca8026",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"The neural net should then find the parameters $P$ that minimizes the cost function in\n",
|
|
"([3](#cost)) for a set of $N$ training samples $x_i$.\n",
|
|
"\n",
|
|
"To perform the minimization using gradient descent, the gradient of $C\\left(\\boldsymbol{x}, P\\right)$ is needed.\n",
|
|
"It might happen so that finding an analytical expression of the gradient of $C(\\boldsymbol{x}, P)$ from ([3](#cost)) gets too messy, depending on which cost function one desires to use.\n",
|
|
"\n",
|
|
"Luckily, there exists libraries that makes the job for us through automatic differentiation.\n",
|
|
"Automatic differentiation is a method of finding the derivatives numerically with very high precision."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9bde4bd4",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"### Example: Exponential decay\n",
|
|
"\n",
|
|
"An exponential decay of a quantity $g(x)$ is described by the equation"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e74337c3",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"solve_expdec\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation} \\label{solve_expdec} \\tag{4}\n",
|
|
" g'(x) = -\\gamma g(x)\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "273549a4",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"with $g(0) = g_0$ for some chosen initial value $g_0$.\n",
|
|
"\n",
|
|
"The analytical solution of ([4](#solve_expdec)) is"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "db3c6623",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"_auto2\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation}\n",
|
|
" g(x) = g_0 \\exp\\left(-\\gamma x\\right)\n",
|
|
"\\label{_auto2} \\tag{5}\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8547e3c8",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"Having an analytical solution at hand, it is possible to use it to compare how well a neural network finds a solution of ([4](#solve_expdec)).\n",
|
|
"\n",
|
|
"The program will use a neural network to solve"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "48341fc6",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"solveode\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation} \\label{solveode} \\tag{6}\n",
|
|
"g'(x) = -\\gamma g(x)\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d25b1e02",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"where $g(0) = g_0$ with $\\gamma$ and $g_0$ being some chosen values.\n",
|
|
"\n",
|
|
"In this example, $\\gamma = 2$ and $g_0 = 10$.\n",
|
|
"\n",
|
|
"To begin with, a trial solution $g_t(t)$ must be chosen. A general trial solution for ordinary differential equations could be"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7c5dd91e",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"g_t(x, P) = h_1(x) + h_2(x, N(x, P))\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "24223062",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"with $h_1(x)$ ensuring that $g_t(x)$ satisfies some conditions and $h_2(x,N(x, P))$ an expression involving $x$ and the output from the neural network $N(x,P)$ with $P $ being the collection of the weights and biases for each layer. For now, it is assumed that the network consists of one input layer, one hidden layer, and one output layer.\n",
|
|
"\n",
|
|
"In this network, there are no weights and bias at the input layer, so $P = \\{ P_{\\text{hidden}}, P_{\\text{output}} \\}$.\n",
|
|
"If there are $N_{\\text{hidden} }$ neurons in the hidden layer, then $P_{\\text{hidden}}$ is a $N_{\\text{hidden} } \\times (1 + N_{\\text{input}})$ matrix, given that there are $N_{\\text{input}}$ neurons in the input layer.\n",
|
|
"\n",
|
|
"The first column in $P_{\\text{hidden} }$ represents the bias for each neuron in the hidden layer and the second column represents the weights for each neuron in the hidden layer from the input layer.\n",
|
|
"If there are $N_{\\text{output} }$ neurons in the output layer, then $P_{\\text{output}} $ is a $N_{\\text{output} } \\times (1 + N_{\\text{hidden} })$ matrix.\n",
|
|
"\n",
|
|
"Its first column represents the bias of each neuron and the remaining columns represents the weights to each neuron.\n",
|
|
"\n",
|
|
"It is given that $g(0) = g_0$. The trial solution must fulfill this condition to be a proper solution of ([6](#solveode)). A possible way to ensure that $g_t(0, P) = g_0$, is to let $F(N(x,P)) = x \\cdot N(x,P)$ and $A(x) = g_0$. This gives the following trial solution:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ebf04383",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"trial\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation} \\label{trial} \\tag{7}\n",
|
|
"g_t(x, P) = g_0 + x \\cdot N(x, P)\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ea4a0013",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"### Reformulating the problem\n",
|
|
"\n",
|
|
"We wish that our neural network manages to minimize a given cost function.\n",
|
|
"\n",
|
|
"A reformulation of out equation, ([6](#solveode)), must therefore be done,\n",
|
|
"such that it describes the problem a neural network can solve for.\n",
|
|
"\n",
|
|
"The neural network must find the set of weights and biases $P$ such that the trial solution in ([7](#trial)) satisfies ([6](#solveode)).\n",
|
|
"\n",
|
|
"The trial solution"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2351b84f",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"g_t(x, P) = g_0 + x \\cdot N(x, P)\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f6cc00e4",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"has been chosen such that it already solves the condition $g(0) = g_0$. What remains, is to find $P$ such that"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "84e066a1",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"nnmin\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation} \\label{nnmin} \\tag{8}\n",
|
|
"g_t'(x, P) = - \\gamma g_t(x, P)\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "feff9ab3",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"is fulfilled as *best as possible*.\n",
|
|
"\n",
|
|
"The left hand side and right hand side of ([8](#nnmin)) must be computed separately, and then the neural network must choose weights and biases, contained in $P$, such that the sides are equal as best as possible.\n",
|
|
"This means that the absolute or squared difference between the sides must be as close to zero, ideally equal to zero.\n",
|
|
"In this case, the difference squared shows to be an appropriate measurement of how erroneous the trial solution is with respect to $P$ of the neural network.\n",
|
|
"\n",
|
|
"This gives the following cost function our neural network must solve for:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "0c6f0e79",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\min_{P}\\Big\\{ \\big(g_t'(x, P) - ( -\\gamma g_t(x, P) \\big)^2 \\Big\\}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9302a1dd",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"(the notation $\\min_{P}\\{ f(x, P) \\}$ means that we desire to find $P$ that yields the minimum of $f(x, P)$)\n",
|
|
"\n",
|
|
"or, in terms of weights and biases for the hidden and output layer in our network:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f7f204bb",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\min_{P_{\\text{hidden} }, \\ P_{\\text{output} }}\\Big\\{ \\big(g_t'(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) - ( -\\gamma g_t(x, \\{ P_{\\text{hidden} }, P_{\\text{output} }\\}) \\big)^2 \\Big\\}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8d61d75f",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"for an input value $x$.\n",
|
|
"\n",
|
|
"If the neural network evaluates $g_t(x, P)$ at more values for $x$, say $N$ values $x_i$ for $i = 1, \\dots, N$, then the *total* error to minimize becomes"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "10d3aec9",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"min\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation} \\label{min} \\tag{9}\n",
|
|
"\\min_{P}\\Big\\{\\frac{1}{N} \\sum_{i=1}^N \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2 \\Big\\}\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5e296388",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"Letting $\\boldsymbol{x}$ be a vector with elements $x_i$ and $C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2$ denote the cost function, the minimization problem that our network must solve, becomes"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fe010d79",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\min_{P} C(\\boldsymbol{x}, P)\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "add715f9",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"In terms of $P_{\\text{hidden} }$ and $P_{\\text{output} }$, this could also be expressed as\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\min_{P_{\\text{hidden} }, \\ P_{\\text{output} }} C(\\boldsymbol{x}, \\{P_{\\text{hidden} }, P_{\\text{output} }\\})\n",
|
|
"$$\n",
|
|
"\n",
|
|
"For simplicity, it is assumed that the input is an array $\\boldsymbol{x} = (x_1, \\dots, x_N)$ with $N$ elements. It is at these points the neural network should find $P$ such that it fulfills ([9](#min)).\n",
|
|
"\n",
|
|
"First, the neural network must feed forward the inputs.\n",
|
|
"This means that $\\boldsymbol{x}s$ must be passed through an input layer, a hidden layer and a output layer. The input layer in this case, does not need to process the data any further.\n",
|
|
"The input layer will consist of $N_{\\text{input} }$ neurons, passing its element to each neuron in the hidden layer. The number of neurons in the hidden layer will be $N_{\\text{hidden} }$.\n",
|
|
"\n",
|
|
"For the $i$-th in the hidden layer with weight $w_i^{\\text{hidden} }$ and bias $b_i^{\\text{hidden} }$, the weighting from the $j$-th neuron at the input layer is:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cb9b22eb",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{aligned}\n",
|
|
"z_{i,j}^{\\text{hidden}} &= b_i^{\\text{hidden}} + w_i^{\\text{hidden}}x_j \\\\\n",
|
|
"&=\n",
|
|
"\\begin{pmatrix}\n",
|
|
"b_i^{\\text{hidden}} & w_i^{\\text{hidden}}\n",
|
|
"\\end{pmatrix}\n",
|
|
"\\begin{pmatrix}\n",
|
|
"1 \\\\\n",
|
|
"x_j\n",
|
|
"\\end{pmatrix}\n",
|
|
"\\end{aligned}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "49f3c493",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"The result after weighting the inputs at the $i$-th hidden neuron can be written as a vector:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a352bd61",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{aligned}\n",
|
|
"\\boldsymbol{z}_{i}^{\\text{hidden}} &= \\Big( b_i^{\\text{hidden}} + w_i^{\\text{hidden}}x_1 , \\ b_i^{\\text{hidden}} + w_i^{\\text{hidden}} x_2, \\ \\dots \\, , \\ b_i^{\\text{hidden}} + w_i^{\\text{hidden}} x_N\\Big) \\\\\n",
|
|
"&=\n",
|
|
"\\begin{pmatrix}\n",
|
|
" b_i^{\\text{hidden}} & w_i^{\\text{hidden}}\n",
|
|
"\\end{pmatrix}\n",
|
|
"\\begin{pmatrix}\n",
|
|
"1 & 1 & \\dots & 1 \\\\\n",
|
|
"x_1 & x_2 & \\dots & x_N\n",
|
|
"\\end{pmatrix} \\\\\n",
|
|
"&= \\boldsymbol{p}_{i, \\text{hidden}}^T X\n",
|
|
"\\end{aligned}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e5636ad7",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"The vector $\\boldsymbol{p}_{i, \\text{hidden}}^T$ constitutes each row in $P_{\\text{hidden} }$, which contains the weights for the neural network to minimize according to ([9](#min)).\n",
|
|
"\n",
|
|
"After having found $\\boldsymbol{z}_{i}^{\\text{hidden}} $ for every $i$-th neuron within the hidden layer, the vector will be sent to an activation function $a_i(\\boldsymbol{z})$.\n",
|
|
"\n",
|
|
"In this example, the sigmoid function has been chosen to be the activation function for each hidden neuron:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "68bb8b9c",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"f(z) = \\frac{1}{1 + \\exp{(-z)}}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ff6e6536",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"It is possible to use other activations functions for the hidden layer also.\n",
|
|
"\n",
|
|
"The output $\\boldsymbol{x}_i^{\\text{hidden}}$ from each $i$-th hidden neuron is:\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\boldsymbol{x}_i^{\\text{hidden} } = f\\big( \\boldsymbol{z}_{i}^{\\text{hidden}} \\big)\n",
|
|
"$$\n",
|
|
"\n",
|
|
"The outputs $\\boldsymbol{x}_i^{\\text{hidden} } $ are then sent to the output layer.\n",
|
|
"\n",
|
|
"The output layer consists of one neuron in this case, and combines the\n",
|
|
"output from each of the neurons in the hidden layers. The output layer\n",
|
|
"combines the results from the hidden layer using some weights $w_i^{\\text{output}}$\n",
|
|
"and biases $b_i^{\\text{output}}$. In this case,\n",
|
|
"it is assumes that the number of neurons in the output layer is one.\n",
|
|
"\n",
|
|
"The procedure of weighting the output neuron $j$ in the hidden layer to the $i$-th neuron in the output layer is similar as for the hidden layer described previously."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6fb4c55b",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{aligned}\n",
|
|
"z_{1,j}^{\\text{output}} & =\n",
|
|
"\\begin{pmatrix}\n",
|
|
"b_1^{\\text{output}} & \\boldsymbol{w}_1^{\\text{output}}\n",
|
|
"\\end{pmatrix}\n",
|
|
"\\begin{pmatrix}\n",
|
|
"1 \\\\\n",
|
|
"\\boldsymbol{x}_j^{\\text{hidden}}\n",
|
|
"\\end{pmatrix}\n",
|
|
"\\end{aligned}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "09c31d8d",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"Expressing $z_{1,j}^{\\text{output}}$ as a vector gives the following way of weighting the inputs from the hidden layer:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f81fe361",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\boldsymbol{z}_{1}^{\\text{output}} =\n",
|
|
"\\begin{pmatrix}\n",
|
|
"b_1^{\\text{output}} & \\boldsymbol{w}_1^{\\text{output}}\n",
|
|
"\\end{pmatrix}\n",
|
|
"\\begin{pmatrix}\n",
|
|
"1 & 1 & \\dots & 1 \\\\\n",
|
|
"\\boldsymbol{x}_1^{\\text{hidden}} & \\boldsymbol{x}_2^{\\text{hidden}} & \\dots & \\boldsymbol{x}_N^{\\text{hidden}}\n",
|
|
"\\end{pmatrix}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c309a4ce",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"In this case we seek a continuous range of values since we are approximating a function. This means that after computing $\\boldsymbol{z}_{1}^{\\text{output}}$ the neural network has finished its feed forward step, and $\\boldsymbol{z}_{1}^{\\text{output}}$ is the final output of the network.\n",
|
|
"\n",
|
|
"The next step is to decide how the parameters should be changed such that they minimize the cost function.\n",
|
|
"\n",
|
|
"The chosen cost function for this problem is"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ea47ae29",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"C(\\boldsymbol{x}, P) = \\frac{1}{N} \\sum_i \\big(g_t'(x_i, P) - ( -\\gamma g_t(x_i, P) \\big)^2\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9dd83767",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"In order to minimize the cost function, an optimization method must be chosen.\n",
|
|
"\n",
|
|
"Here, gradient descent with a constant step size has been chosen."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "531a7b4f",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"### Gradient descent\n",
|
|
"\n",
|
|
"The idea of the gradient descent algorithm is to update parameters in\n",
|
|
"a direction where the cost function decreases goes to a minimum.\n",
|
|
"\n",
|
|
"In general, the update of some parameters $\\boldsymbol{\\omega}$ given a cost\n",
|
|
"function defined by some weights $\\boldsymbol{\\omega}$, $C(\\boldsymbol{x},\n",
|
|
"\\boldsymbol{\\omega})$, goes as follows:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e10c204a",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\boldsymbol{\\omega}_{\\text{new} } = \\boldsymbol{\\omega} - \\lambda \\nabla_{\\boldsymbol{\\omega}} C(\\boldsymbol{x}, \\boldsymbol{\\omega})\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9f4fa48e",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"for a number of iterations or until $ \\big|\\big| \\boldsymbol{\\omega}_{\\text{new} } - \\boldsymbol{\\omega} \\big|\\big|$ becomes smaller than some given tolerance.\n",
|
|
"\n",
|
|
"The value of $\\lambda$ decides how large steps the algorithm must take\n",
|
|
"in the direction of $ \\nabla_{\\boldsymbol{\\omega}} C(\\boldsymbol{x}, \\boldsymbol{\\omega})$.\n",
|
|
"The notation $\\nabla_{\\boldsymbol{\\omega}}$ express the gradient with respect\n",
|
|
"to the elements in $\\boldsymbol{\\omega}$.\n",
|
|
"\n",
|
|
"In our case, we have to minimize the cost function $C(\\boldsymbol{x}, P)$ with\n",
|
|
"respect to the two sets of weights and biases, that is for the hidden\n",
|
|
"layer $P_{\\text{hidden} }$ and for the output layer $P_{\\text{output}\n",
|
|
"}$ .\n",
|
|
"\n",
|
|
"This means that $P_{\\text{hidden} }$ and $P_{\\text{output} }$ is updated by"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4652fc79",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{aligned}\n",
|
|
"P_{\\text{hidden},\\text{new}} &= P_{\\text{hidden}} - \\lambda \\nabla_{P_{\\text{hidden}}} C(\\boldsymbol{x}, P) \\\\\n",
|
|
"P_{\\text{output},\\text{new}} &= P_{\\text{output}} - \\lambda \\nabla_{P_{\\text{output}}} C(\\boldsymbol{x}, P)\n",
|
|
"\\end{aligned}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "41ae0626",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"### The code for solving the ODE"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "af7c165f",
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"editable": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Initial cost: 367.01\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Final cost: 0.0666807\n",
|
|
"Max absolute difference: 0.0437499\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAJcCAYAAAAo6aqNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABazklEQVR4nO3ddbhUdeLH8ff3Bh3SXXagKCAiFtbagt3drrXqmuuu7rq2u3YgNhZ2YSAKigoIiDRigDSXbrhxfn/M4O96l7jEnTNz7/v1PPeBmTPxOTFnPvOdMzMhiiIkSZJU9rLiDiBJklRRWLwkSZJSxOIlSZKUIhYvSZKkFLF4SZIkpYjFS5IkKUUsXikSQmgUQvgyhLA4hHB/3HniFkKoGkJ4P4SwMITwetx51iWEcHYIYWDcOTZFCKFrCGFqGd9H/xDC+aW43D4hhAllmUXprbTbyma8v0khhINSdX8bY1P2MyGEliGEJSGE7E3MsNnXy6Ys+801X+nG4rUOyQ1meXLFzwohPBtCqLGRN3chMAeoFUXRNZsxZqY6HmgE1Iui6IS4w8QhhNA6hBCFEHLizpJKURR9FUXRdnHnKI0QwpEhhCEhhKUhhLkhhJdCCM2LTT87hFCY3EcsCSH8mtxPbFvsMqvX85ISfyfFM1fpLYRwawih1yZc/7kQwu2bM1O6KVlmoij6LYqiGlEUFcaZa1OV1/kqyeK1fkdFUVQDaA/sDvxtQ64cErKAVsDYaCO+sbacPjG3An6MoqggzhDldNn+QUWYx7IQQjgeeBl4EKgP7ASsBAaGEOoUu+i3yX1EbeAgYDkwLITQtsRNbpF8Eln991rZz0XZc/uSNlAURf6t5Q+YBBxU7PS9wAfJ/3cGvgEWAD8AXYtdrj/wb+BrEjvhXkA+sApYQmLnXBl4AJie/HsAqJy8fldgKnA9MBN4EbgVeD15W4uBUcC2wI3AbGAK8KdiGc4BxiUv+wtwUbFpq2//muR1ZwDnFJteFbgfmAwsBAYCVdc332tYfjskl8UCYAxwdPL825LLIj+5PM5bw3VvBXoDLyTnYQzQsdj0psCbQB7wK3BFsWnPAbeXnN8S6/V6YCSJJ9Ic4Abg5+R9jQWOKXb5s4GBa5nH1kAEnAX8RmJU8+Zi07OK3fbc5DzVTU77LXndJcm/PZPLvENy+unJ6TsmT58PvJP8/4ZuPyWXwRXJ+Wy+hnnaGhiQXPdzgNeKTesCfJec9h3QpcR2f34y2wKgbbFpDUg8FhquZX1cm1wfC4HXgCrFpl9HYhudnrz9CNh6Letjo7f7ErcTkuviuhLnZwGjgX+ua9sAPgDeKLGN5JRyv1MXeDY5v/NXr/PktAuAn4B5wHtA02LTIuBSYGJy/v8FbAV8Cywise1VKrEsbkqu40nAacVu6wjg++T1pgC3rmGbP4/ENvxl8vxzk8t+PvAJ0KrYdQ4GxifX7yMktq/z1zDvh/LHfcMPxR7v7yXn+yfggrUsuwv54772/VJuY0cCI0hst98Au6xj/TyYXCaLgGHAPhuw3yrVfgZ4FLi/xP2+D1xF4vFcROLxtITE42P1OslZ1zYE1CGxbeYlz/+AYvsAko/htcx3J2Bocr5nAf8pNu3o5LwuSN7GDiUe3wetb99cyvla63awvmWfTn+xB0jnvxIbTIvkivwX0IzEk+jhJHbEBydPNyi28f5G4hVyDpC7hg3un8AgEk9EDUg82P9VbGMsAO4m8SRWNblRrQAOSd7mCyQKx83J278A+LXY7R9BYqcbgP2AZUD7Erf/z+R1D09Or5Oc/mhyHpoB2SSebCuvb75LLLvc5APjJqAScEDywbBdsQdJr3Us+9Xze3gyw53AoOS0LBI7vL8nb3tLEk+yhySnl1zWXfnfJ/oRyXW6ulCeQOJBnQWcBCwFmiSnnc36i9dTyfXUjkSZ2yE5/arkem6eXIZPAq+UuG5Osdt7Abgm+f8eJHbSlxSb9peN3H5+XwbALcDwNa235PRXSGxXWUAVYO/k+XVJ7KzPILENnpI8Xa/Ydn9+8v/PAP8udpt/Bj5ex/oYklz+dUk8eV+cnHYoifK4E1CNxM55XcVro7f7ErezffJ+2qxh2m0kRrnWum2QKCGz1rae17Pf+ZBEMaiTzLlf8vwDSJSk9sn1+jDJ0pOcHpF4UqrF/4/O9SPx+KhN4on+rBLL4j/J29qPxDa/XbHpOye3gV1IPNF2LzE/LwDVSWxf3Uk83ndIbht/A75JXr4+iSfr45Pz85fkfa/tCf5WSuwbSBS1x0hsj7uSKA4HruX6z1Hs8V+Kbaw9iSK+B4l9zVnJy1dey+2fDtRLzuc1JLbPKuvbb23IfoZEyZkOZBVbhsuARiWfm9a0jbH2bagecByJx1JNEi/m3yl2O/3XsV6+Bc5I/r8G0Dn5/22T83Fw8r6uS24LlUpmLbluWPO+YF3ztdbtYH3LPp3+Yg+Qzn/JjWAJiRY/ObnCq5IYSXixxGU/4f93av1JviIuNr3kBvczcHix04cAk4ptjKv44yuyW4G+xU4flcyWnTxdM7mBbrGWeXkHuLLY7S/nj0/4s0mMZmUlp7Vbw22sc75LnL8PiR1SVrHzXiH5ypnSFa/Pip3eEVie/P8ewG8lLn8j8OxalvWaHtznrmfdjwC6Jf9/NusvXsVfNQ4BTk7+fxzFniCAJiRekeew5uJ1HvBeseueD7yaPD2Z/y8RG7r9dAWmkXiiHQjUXse8v0Ci9DUvcf4ZwJAS530LnF1su19dvA4Cfil2ua+BM9exPk4vdvoe4Ink/58B7iw2bWvWUbw2drtfw/X2Tt5PlTVMuxiYuK5tg0RhzC+xjSwo8bfDGq7XhMSr/jWVwaeBe4qdrpHcllonT0fAXsWmDwOuL3b6fuCBYsuiAKhebHpv4Ja1LMcHgP+WmJ8ti03/iGIj1yT2I8tIHFJwJn8sH4HEaFupiheJF0iFQM1i590JPLeW6z/HmovX2raxx0m+aCk2fQLJslKKbWw+yf0l69hvreW6I1jLfobE4//g5P8vA/qUmJ81FpR1bUNruP9dgfnFTvdfx3r5ksSLjvolzr8F6F1i3U8j+W4Im6l4rW872NBlH+efx3itX/coiraIoqhVFEWXRlG0nMTO5IQQwoLVfyR21E2KXW/Kem63KYkn0tUmJ89bLS+KohUlrjOr2P+XA3Oi/z/ocHny3xoAIYTDQgiDQgjzkvkOJ/GqabW50R+Pr1qWvG59Eq8mfl5D5tLMd/H5mxJFUVGJeWy2hsuuzcwS+aokjydpBTQtkeMmEgfrl9Yf1k8I4cwQwohit9eWPy6vDc26+kMYrYC3i93uOBI7j7VlHQDsE0JoTOJV22vAXiGE1iRGLUYkL7cx288WJN6KuTOKooXrmJfrSDw5DgkhjAkhnLuW+1x9v2tap58DVUMIe4QQWpHYwb+9jvtc2/Jryh/X1TofV5uw3Zc0J/nvmrbtJsWmr00zEm+HFFc/uS9Z/TduDddrAcyLomj+Gqb9YflHUbSExIhz8eVfch9R8nTxeZ0fRdHSYqd/34aS6+2LEEJeCGEhibJZ8vFQfF20Ah4stp3PI7ENNaPEOowSz4rr2z8W15TEMllcIuuG7Etg3Y/Ra0rsT1rwx8fT70II14QQxiU/kb2AxOOy+LJZ235rQ/czz5MYXSP574ulm821b0MhhGohhCdDCJNDCItIlKktSvmpwfNIjG6NDyF8F0I4Mnl+ye2yiMT63dD1sz6l2Q7WuuzTicVr40whMfJTfCdaPYqiu4pdJlrPbUwn8YBfrWXyvNJef61CCJVJHP90H4mh6S2APiR2hOszh8Rw7VZrmFaa+V5tOtAi+cGC1VqSeCW0qaaQeFu1eI6aURQdnpy+lMRQ+mqN13Abvy/fZDF4isSrynrJ5TWa0i2v0mQ9rETWKlEUTWMN6ziKop9I7DCuIPE20mISO5MLSbwaXl1kN2b7mU/iWJZnQwh7rS1wFEUzoyi6IIqipsBFwGMhhK3XcJ+r7/d/1mkyZ28Sb0eeSuLYyMUlL1cKM0i8Tbtai7VdcBO3+5ImkBiV+cMnbpPb83Ek3sJbl2OArzbifqcAdUMIW6xh2h+WfwihOom3jjb2MVUneRurFd+GXibxtmWLKIpqA0/wv8ux+DY2hcTxdMW386pRFH1DYh3+vt5CCIF1rEf+d9udTmKZ1CyRdW3zvaH7zikk3hYvnr1aFEWvlLxgCGEfEiP/J5IYUdqCxDFj693GNmI/0wvoFkJoR+It3HeKTVvXPK5rG7oG2A7YI4qiWsC+q+OtL38URROjKDqFxOENdwNvJLefktvl6vW7pvWzvn3zuuZrQ7eDtGXx2ji9gKNCCIeEELJDCFVC4nuSmq/3mv/vFeBvIYQGIYT6JI5X2uiPUJdQicRxG3lAQQjhMOBPpbli8gnzGeA/IYSmyfnbM/mktiHzPZjEg+y6EEJuCKEribdHX93kuUu8lbcohHB9SHwfWHYIoW0IYffk9BHA4SGEusmRo6vWc3vVSTzg8wBCCOeQeCW6OTwB/Du50yW5vrslp+WReEtgyxLXGUBi5zwgebp/idOwkdtPFEX9gdNIjMLtsabLhBBOKLZO55NYNoUkSsy2IYRTQwg5IfF1CDuSOEB3TV4mcRzLacn/b4zewDkhhB1CCNVIzOfabPR2X1JyVOZaEsv41OR21hjoSeIYqv+WvE5yO2wTQniYxFsot23E/c4g8bbdYyGEOsnHzuonx5dJLItdk4/HO4DBURRN2ohZXO22EEKlZKE4ksQxP5A4dGFeFEUrQgidSJTndXkCuDGEsBNACKF2CGF1af0Q2CmEcGxy9OEK1vxiaLVZQOvVL9qiKJpC4hjGO5P7nF1IjL68tI7rl3xMrctTwMXJUb4QQqgeQjiixBP8ajVJvEWbB+SEEP5OYnsojQ3az0RRNJXEB1heBN5Mvtuy2lrncT3bUE0SI58LQgh1gX+UMjshhNNDCA2SzxELkmcXkniMHhFCODCEkEui3K0ksc5KGsG6983rmq8N3Q7SlsVrIyQ3gG4k3t7KI/EK469s2PK8ncQnREaS+ITi8OR5myPfYhI7t94knjhPJfHqtbSuTWb6jsRbBneTOFar1PMdRdEqEp90OYzEKNpjJI7xGb9xc/WH2y4kUeJ2JfEBgzkknhBrJy/yIolPXE4CPiXxdt26bm8sieNfviXxwN+ZxDFJm8ODJJb9pyGExSQOiN8jeb/LSH76NfnWQ+fkdQaQ2EF+uZbTsAnbTxRFfUl8+u+9EEKHNVxkd2BwCGFJMvuVURT9GkXRXBJPzteQeIvrOuDIKIrW+LZbFEWry3dTEk8EGyyKoo+Ah4AvSByw+21y0so1XHZTt/uSt/caiePa/kJiGxtL4hjPvZLLYrU9k8tqEYmSXAvYPYqiUSVuckH44/d4Xb2Wuz6DxLFb40kcg3ZVMk8/EsfTvEliFGkr4OSNnT8SI6nzSYwkvETiYPPVj89LgX8mt9m/k1imaxVF0dsk9hOvhsRbWKNJPPZJbh8nAHeR2G62Yd2Pr9Xlb24IYXjy/6eQON5nOom3rP+R3I7X5Glgx+Rj6p115U7mG0riw0mPkFgeP5E43mpNPiGxLf9I4m2uFZTybdON3M88n7xcybcZ7yTxomBBCOHaNVxvjdsQiWP1qpLYngcBH5cme9KhwJjktv4gieNYV0RRNIHEW6EPJ2/3KBJfw7RqDbexvn3z+uZrQ7aDtBUSL+wkKf2FEHYg8aReOYr5O+AyWXIEulcURRsySq8US45U9SLxAYqi9V1emcERL0lpLYRwTPLtsDokRlXet3SpvEu+bXcl0NPSVb5YvCSlu4tIvLX9M4ljSi6JN45UtpIjuwtIfIL2gVjDaLPzrUZJkqQUccRLkiQpRdLui8XWpH79+lHr1q3jjiFJkrRew4YNmxNFUYM1TcuI4tW6dWuGDh0adwxJkqT1CiGU/JWP3/lWoyRJUopYvCRJklLE4iVJkpQiGXGMlyRJ2nT5+flMnTqVFStWxB2lXKhSpQrNmzcnNze31NexeEmSVEFMnTqVmjVr0rp1a0IIccfJaFEUMXfuXKZOnUqbNm1KfT3fapQkqYJYsWIF9erVs3RtBiEE6tWrt8GjhxYvSZIqEEvX5rMxy9LiJUmSlCIWL0mSlHGee+45LrvssvVeZvr06b+fPv/88xk7duwG31f//v058sgjN/h6a2LxkiRJ5VLJ4tWzZ0923HHHGBNZvCRJUop1796dDh06sNNOO9GjRw8AatSowc0330y7du3o3Lkzs2bNAuD9999njz32YLfdduOggw76/fzVFi9eTJs2bcjPzwdg0aJFtG7dmtdff52hQ4dy2mmnseuuu7J8+XK6du36+08Qfvzxx7Rv35527dpx4IEHAjBkyBC6dOnCbrvtRpcuXZgwYcJmn3e/TkKSpArotvfHMHb6os16mzs2rcU/jtppvZd75plnqFu3LsuXL2f33XfnuOOOY+nSpXTu3Jl///vfXHfddTz11FP87W9/Y++992bQoEGEEOjZsyf33HMP999//++3VbNmTbp27cqHH35I9+7defXVVznuuOM44YQTePTRR7nvvvvo2LHjH+4/Ly+PCy64gC+//JI2bdowb948ALbffnu+/PJLcnJy+Oyzz7jpppt48803N+sysnhJkqSUeuihh3j77bcBmDJlChMnTqRSpUq/H0fVoUMH+vbtCyS+e+ykk05ixowZrFq1ao3fmXX++edzzz330L17d5599lmeeuqpdd7/oEGD2HfffX+/rbp16wKwcOFCzjrrLCZOnEgI4fdRtM3J4iVJUgVUmpGpstC/f38+++wzvv32W6pVq0bXrl1ZsWIFubm5v389Q3Z2NgUFBQBcfvnlXH311Rx99NH079+fW2+99X9uc6+99mLSpEkMGDCAwsJC2rZtu84MURSt8asgbrnlFvbff3/efvttJk2aRNeuXTd5fkvyGC9JkpQyCxcupE6dOlSrVo3x48czaNCg9V6+WbNmADz//PNrvdyZZ57JKaecwjnnnPP7eTVr1mTx4sX/c9k999yTAQMG8OuvvwL8/lZj8ft67rnnNmi+SsviJUmSUubQQw+loKCAXXbZhVtuuYXOnTuv8/K33norJ5xwAvvssw/169df6+VOO+005s+fzymnnPL7eWeffTYXX3zx7wfXr9agQQN69OjBscceS7t27TjppJMAuO6667jxxhvZa6+9KCws3MQ5XbMQRVGZ3PDm1LFjx2j1pxAkSdLGGTduHDvssEPcMcrEG2+8wbvvvsuLL76Y0vtd0zINIQyLoqjjmi7vMV6SJCmjXX755Xz00Uf06dMn7ijrZfGSJEkZ7eGHH447Qql5jJckSVKKlFnxCiE8E0KYHUIYXey8uiGEviGEicl/65TV/UuSJKWbshzxeg44tMR5NwD9oijaBuiXPJ0WCvJXxR1BkiSVc2VWvKIo+hKYV+LsbsDqL+F4HuheVve/IYa8/TAz79iZ5Uv/97s+JEmSNpdUH+PVKIqiGQDJfxuu7YIhhAtDCENDCEPz8vLKNFTNptvSPJrJD+/8p0zvR5IkVWxpe3B9FEU9oijqGEVRxwYNGpTpfe2wxyGMrrwr20x8mhXLHPWSJEllI9XFa1YIoQlA8t/ZKb7/tcrqegP1WMgPb/837iiSJJVbkyZNYocdduCCCy5gp5124k9/+hPLly+na9euXH/99XTq1Iltt92Wr776Ku6oZSLV3+P1HnAWcFfy33dTfP9rteOehzHqi13ZeuLTrFj2F6pUqxl3JEmSys5HN8DMUZv3NhvvDIfdtd6LTZw4kVdeeYWnnnqKE088kTfffBOAgoIChgwZQp8+fbjtttv47LPPNm++NFCWXyfxCvAtsF0IYWoI4TwShevgEMJE4ODk6bQRul5PPRYw8p0H4o4iSVK51aZNG3bddVcAOnTowKRJkwA49thj/+e88qbMRryiKDplLZMOLKv73FQ77XkYo/u3Y8sfeyZHvWrEHUmSpLJRipGpslK5cuXf/5+dnf37D1ivPj87O5uCgoJYspW1tD24Pg4hBKL9rqc+Cxj13oNxx5EkSeWMxauEtl0OZ3TuLrQZ34OVy5fEHUeSJJUj/kh2CSEECve9jvr9Tue7dx9i95NvijuSJEnlRuvWrRk9+vdfE+Taa6/9n8vUr1+/3B7j5YjXGuyy95GMzt2ZNuN7sGrFsrjjSJKkcsLitQYhBAr2uY76zGekx3pJkqTNxOK1Fu32PpLRuW1pNbYH+Ssd9ZIklQ9RFMUdodzYmGVp8VqLkJVF/t5/pQHzGPneQ3HHkSRpk1WpUoW5c+davjaDKIqYO3cuVapU2aDreXD9Ouy6z9GM+WonWoztQf7KK8itXC3uSJIkbbTmzZszdepU8vLy4o5SLlSpUoXmzZtv0HUsXusQsrJYsddfaTjgbIa//wjtj78u7kiSJG203Nxc2rRpE3eMCs23Gtej/X7dGJOzI83HPEHByuVxx5EkSRnM4rUeISuL5V3+SsNoLqM+eCTuOJIkKYNZvEqhQ9fujMnegWajH3fUS5IkbTSLVymErCyWdbmWhtFcRn/4aNxxJElShrJ4lVKHrscyNnt7mox6nMJVK+KOI0mSMpDFq5SysrNYsue1NIrmMOrDx+KOI0mSMpDFawN03P84xmZvR5ORj1GYvzLuOJIkKcNYvDZAVnYWi/a4lkZRHqM/fDzuOJIkKcNYvDbQ7gcez7jsbWn8wyMUOeolSZI2gMVrA2VnZ7Gg09U0ivIY89ETcceRJEkZxOK1EToddBLjsrah4feOekmSpNKzeG2E7Ows5ne6mkbRbMZ+/GTccSRJUoaweG2k1aNe9b9/hKL8VXHHkSRJGcDitZFycrKZ1/EvNC6axbhPHPWSJEnrZ/HaBHv86WTGZ21NveEPExU46iVJktbN4rUJcnKymdPhqsSo18dPxR1HkiSlOYvXJtrjkFMZn7UVdYY/5KiXJElaJ4vXJsrNySav/VU0KZrJ+E97xh1HkiSlMYvXZtD50NOYELZki6EPERXmxx1HkiSlKYvXZpCbk82s9lfRpGgGExz1kiRJa2Hx2kw6H3o6E8KW1B76oKNekiRpjSxem0ml3Gxm7HolTQpn8GPfZ+KOI0mS0pDFazPa8/DT+TG0odZ3DzjqJUmS/ofFazOqnJvD9HZX0KRwOhP7PRt3HEmSlGYsXpvZnkecyY+hNTUGO+olSZL+yOK1mVXOzWHqLlfQtHAaEz9/Pu44kiQpjVi8ykCXI85kIq2oMeg/UFQYdxxJkpQmLF5loEqlXH7b+fLkqNdzcceRJElpwuJVRvY66mwm0pLqjnpJkqQki1cZqVIpl8ltL6NpwVR+7v9C3HEkSVIasHiVob2OOpefaEnVb+5z1EuSJFm8ylLVyrn82vbPNC2Yyi8DXow7jiRJipnFq4wlRr1aUOVrR70kSaroLF5lrFrlSvy6059pWjCFX7/sFXccSZIUI4tXCnQ56jx+pjmVvr7fUS9Jkiowi1cKVK9SiZ92+DPN8icz6auX444jSZJiYvFKkb2OTox65Q68F4qK4o4jSZJiYPFKkRpVK/PT9pfQLH8yvw101EuSpIrI4pVCXY4+n19oRvZXjnpJklQRWbxSqGa1KkzY7hKa5U/it69fiTuOJElKMYtXinU5+oLEqNeXjnpJklTRWLxSrHb1Kozf9mKa5f/KlG9eizuOJElKIYtXDPY6+kJ+jZoSvrzHUS9JkioQi1cMateowrhtL6b5ql+YNqh33HEkSVKKWLxi0qXbhfwaNSHqf7ejXpIkVRAWr5hsUaMqY7e5iOarfmH64DfijiNJklLA4hWjLt0uYlLUhKL+dznqJUlSBWDxilGdmtUYvfWFNF/5MzOGvBl3HEmSVMYsXjHbs9tFTI4aU/jFXRBFcceRJEllyOIVs3q1qjNqywtovvInZgx5K+44kiSpDFm80kDnYy5hctSIws/vdNRLkqRyzOKVBurXqs7ILS+g+cqJzBr6dtxxJElSGbF4pYnO3S9lctSI/H53OOolSVI5ZfFKEw1qV2dE6/NpvmIis4e9E3ccSZJUBixeaWTP7pfwW9SQVZ856iVJUnlk8UojDevU5PtW59N8xY/kDX8v7jiSJGkzs3ilmc7HXMqUqCErHPWSJKncsXilmUZ1ajK81bm0WD6evOHvxx1HkiRtRhavNNSp+5+ZEjVw1EuSpHLG4pWGmtStxdCW59Ji+TjmjPgg7jiSJGkzsXilqU7d/8zUqD7L+/7bUS9JksoJi1eaalavNkNanEuLZeOY98OHcceRJEmbgcUrje3e7c9Mi+qz9FNHvSRJKg8sXmmsRYMtGNz8HFosG8u8kR/FHUeSJG0ii1ea2737ZclRr9sd9ZIkKcNZvNJciwZbMLjZWbRYOoYFoz6JO44kSdoEFq8M0KH7ZUyP6rH4k3856iVJUgazeGWAVg3r8k2Ts2ixdDTzR38adxxJkrSRLF4ZosMxlzM9qsuSjx31kiQpU1m8MkSbRnX5pvGZtFg6ioVj+8YdR5IkbQSLVwZpf8wVzIjqsugjR70kScpEFq8MsmXjenzd+AxaLBnJwrGfxR1HkiRtIItXhtmt2+WJUS+P9ZIkKeNYvDLMVk0bMLDR6bRY/AOLxn0edxxJkrQBLF4ZaLduVzAzqsPCj/7pqJckSRnE4pWBtm7WgC8bnk6LxSNYMv6LuONIkqRSsnhlqHbdrmBWtAXzP/pX3FEkSVIpWbwy1HbNGzKgwem0WDTcUS9JkjKExSuD7dLtCmZHWzC/j6NekiRlAotXBtu+RSO+qH8qLRYNY8mE/nHHkSRJ62HxynA7d7vKUS9JkjKExSvD7diyEZ/XO5UWC4ey9McBcceRJEnrYPEqB9oefRV5UW3mfeiolyRJ6cziVQ60bd2IfnVPocXC71j201dxx5EkSWth8SonduqWGPWa+8E/444iSZLWwuJVTuzcugn96p5EiwVDWP7TwLjjSJKkNbB4lSM7HHkVeVEt5nzoqJckSenI4lWOtNuqGZ/VOZkW8wez/Oev444jSZJKsHiVM9sfeRVzoloe6yVJUhqyeJUzu23djL5bnEjz+YNY+cs3cceRJEnFWLzKoW2P/Atzo5rkOeolSVJasXiVQx22ac4ntU+k+bxvWfnLt3HHkSRJSRavcmqbI65KjHr5CUdJktKGxauc2n27lnxS6wSaz/2GlZMGxR1HkiRh8SrXtjriL8yLapD3vqNekiSlA4tXOdZpuxZ8VPMEms/9mpWThsQdR5KkCi+W4hVC+EsIYUwIYXQI4ZUQQpU4cpR3IQS2OuIq5kc1/ISjJElpIOXFK4TQDLgC6BhFUVsgGzg51Tkqij22b0WfmsfTfM5XrJo8OO44kiRVaHG91ZgDVA0h5ADVgOkx5Sj3Qgi0OTzxG44L3rwGiorijiRJUoWV8uIVRdE04D7gN2AGsDCKok9LXi6EcGEIYWgIYWheXl6qY5Yre+7QmrfrXUTDRaNY9O0zcceRJKnCiuOtxjpAN6AN0BSoHkI4veTloijqEUVRxyiKOjZo0CDVMcuVEAKHnHoVQ6PtyOp3GyybF3ckSZIqpDjeajwI+DWKorwoivKBt4AuMeSoUFrVr8GPHW6jSuESpr15Q9xxJEmqkOIoXr8BnUMI1UIIATgQGBdDjgrnuMP/xDuVjqTJz739eglJkmIQxzFeg4E3gOHAqGSGHqnOURFVzsmm5bH/Ii+qzfzXL4OiwrgjSZJUocTyqcYoiv4RRdH2URS1jaLojCiKVsaRoyLqtENrPm1+OY2XTmD2F4/HHUeSpArFb66vgA4/+TKGsBPVBt5JtGR23HEkSaowLF4VUL2aVZiz7x1UKlrOpFf/GnccSZIqDItXBXVo1/34sPqxtJn6DosmfBV3HEmSKgSLVwWVlRXY8ZTbmR7VY8nbV0JhQdyRJEkq9yxeFdh2LRozaNtraLriZyZ//EDccSRJKvcsXhXcocdfyKCs3aj/3X2smj8t7jiSJJVrFq8KrlrlXIoOvZucKJ9Jr1wTdxxJkso1i5fo0mkP+tY9mW1nf8TsH/rGHUeSpHLL4iUAOpz+L6ZEDcn/4GqiAr/PVpKksmDxEgBN6tVl7K430yz/N358956440iSVC5ZvPS7A48+k0G5nWg56mGW5k2OO44kSeWOxUu/y8nOomb3+wlREZNfujLuOJIklTsWL/3BTjvtwsAmZ7Hjgi+YNPj9uONIklSuWLz0P3Y/7TYm04TcT66jcNWKuONIklRuWLz0P2rXrMG0PW+jWdF0RvX+Z9xxJEkqNyxeWqM9/3QiQ6ruzfYTezBnyoS440iSVC5YvLRGIQQan/hfCsli+mt/iTuOJEnlgsVLa9WyzbZ83+ZCdlnyNaO/eC3uOJIkZTyLl9Zp91NuZlJoTt0vb2HFsiVxx5EkKaNZvLROlStXZcmBd9M0msWIl/8edxxJkjKaxUvr1XbvIxlW6yDaT3meyT+OjDuOJEkZy+KlUml9yn9ZFXKZ/+ZVREVFcceRJCkjWbxUKvWatGT89pex68phDO7zfNxxJEnKSBYvlVr746/n15w2tB56OwsWzI87jiRJGcfipVLLysmFw++nMXMY8dLNcceRJCnjWLy0Qdq0P5Af6h/BXrNfZcwPQ+KOI0lSRrF4aYNtfer9LA9VWPX+NeQXFMYdR5KkjGHx0garXrcJ03a7ht0KRjLgrSfjjiNJUsaweGmj7HDUVUyqtC27jLmHaTNnxR1HkqSMYPHSxsnKptqxD1CfBYx5+UaiKIo7kSRJac/ipY3WcPu9mNDsGA5Y+DbffPtl3HEkSUp7Fi9tkq1PuZelWdWp3vd6lq7IjzuOJElpzeKlTZJbsz7zu9zMrtE4Pnv1wbjjSJKU1ixe2mStD7yI36rtxF6/Psi4X3+LO44kSWnL4qVNl5VFnRMeok5Ywi+v3UhRkQfaS5K0JhYvbRY123RkUpuTOXT5h3zc9+O440iSlJYsXtpstjzxDpZk16bZt7eQt2h53HEkSUo7Fi9tNqFqHVbsfyvtmEi/l++LO44kSWnH4qXNqtHeZzO11m78acYTDBo9Me44kiSlFYuXNq8QaHDSw9QKy5jzzk2syPdHtCVJWs3ipc2ucrOdmbn92Rye35d3Png37jiSJKUNi5fKRPNj/smi3Hq0HfFPfpm1MO44kiSlBYuXykblmoRD/k3b8CsDXrnHH9GWJAmLl8pQ7Y4nMaNuJ46d/wwfDxkVdxxJkmJn8VLZCYGGJz9CtbCKgo9vYeEyf0RbklSxWbxUprIbbseCdhdyVNSf1958Le44kiTFyuKlMtfgiL+xoFJj9p14F8Mn5cUdR5Kk2Fi8VPYqVafykXezfdYUvnvtbgoKi+JOJElSLCxeSomqO3cjr9E+nLqsF72/GBJ3HEmSYmHxUmqEQP0TH6RKKKDWV/9k2gJ/RFuSVPFYvJQyod5WLOt0OUeGr3n1tV5xx5EkKeUsXkqp2gdfx6Iqzeg27T98NmpK3HEkSUopi5dSK7cq1brfz9ZZ0/nx3btZtqog7kSSJKWMxUspl7P9YcxvcRBn5/fmmQ8Hxh1HkqSUsXgpFnWO/Q85WbDV9/9m/MxFcceRJCklLF6KR51WFO51NYdlDaH3K89RVOSPaEuSyj+Ll2JTtetfWFy9NWfMf4TXB/8UdxxJksqcxUvxyalMjWP+S5usWcz55F7mLFkZdyJJksqUxUuxClsfwOKtjuS86C2eePvzuONIklSmLF6KXc2j7yErO4c9fryHb36eE3ccSZLKjMVL8avdjND1eg7OHs6HbzzDyoLCuBNJklQmLF5KC7l7XcbSWltz8dIePP352LjjSJJUJixeSg/ZuVQ/5gFaZOURDbyfSXOWxp1IkqTNzuKl9NFmH5ZvfxwXhPd55I1PiCK/20uSVL5YvJRWqh5xB+RW4ahp/+X9H6bHHUeSpM3K4qX0UrMxOQfezH7ZIxn4/jMsXJ4fdyJJkjYbi5fSTlanC1led0euKniGh/p8H3ccSZI2G4uX0k92DlW7P0DTMI8G3z/EiCkL4k4kSdJmYfFSemq5B/k7n8r5OX144o0+FBQWxZ1IkqRNZvFS2so99F9EudU5c97DPP/NpLjjSJK0ySxeSl/V65Nz8D/okj2W8X2fZcbC5XEnkiRpk1i8lNZCx3NY2bAdfw0vcs+738UdR5KkTWLxUnrLyqZyt//SICyg7Y+P8fn4WXEnkiRpo1m8lP6adaCo/VmcnfMJz731IctX+SPakqTMZPFSRsg+6B8UVa7N5Sse56F+P8YdR5KkjWLxUmaoVpfcQ/7J7lk/Mufr55gwc3HciSRJ2mAWL2WOXU8nv+nu3JjzMne+9Q1FRf6ItiQps1i8lDmyssg96j9swRIOmP4UbwybGnciSZI2iMVLmaXJLoRO53N6zme83ecD5i5ZGXciSZJKzeKljBMO+BtFVetzfeFT3NVnbNxxJEkqNYuXMk+V2uQc+m92zfqZ7B96MeiXuXEnkiSpVCxeyky7nEhhiy7cmPsq97z1DasK/BFtSVL6s3gpM4VA9pH3UzMs54QFT/PUV7/EnUiSpPWyeClzNdqRrM6XcErOFwzo14fJc5fGnUiSpHWyeCmzdb2BwuqNuTX7aW5+awSFfreXJCmNWbyU2SrXJPuwO9kxTGLLSa/xUL+JcSeSJGmtLF7KfDsdQ7Tl/txc6TX6fPEFX0yYHXciSZLWyOKlzBcC4ZgnqFS1Jj2rPMxNr37L1PnL4k4lSdL/sHipfKjZmHD807SMpnFzUQ8u7TWMlQWFcaeSJOkPLF4qP7bcj9D1Jo4MA2k7823++b7fai9JSi8WL5Uv+1wDWx/EPyu9wIghA3jTH9KWJKURi5fKl6wsOKYH2TUb8Ey1h7nrnUGMn7ko7lSSJAEWL5VH1esRTniehtEc7s15kkteHMaiFflxp5IkyeKlcqpFJ8LB/6JrNISDF77OX1//gSjyy1UlSfGyeKn86nwJ7HA0N+S+ytyxA/w9R0lS7CxeKr9CgG6PEOq0ome1R+n58RAG/zI37lSSpArM4qXyrUptwgnPU5slPFb1ca54eSizF62IO5UkqYKyeKn8a7IL4fB76Vj4A2es6s1lL39PfmFR3KkkSRWQxUsVQ/szod0p/DnrTSr/1p97P5kQdyJJUgVk8VLFEAIccT+hwfY8UfUJ3vvyOz4ePSPuVJKkCsbipYqjUnU48QWqZRfwbI3HuOH14fyStyTuVJKkCsTipYqlwbaEox5kh4JxXB1e4ZJew1m2qiDuVJKkCsLipYpn5+Nh9/M5k/dpNedz/vb2aL9cVZKUEhYvVUyH3AFNd+PhKk/x3YjhvDzkt7gTSZIqAIuXKqacynDCc1TKzebFmo9y53sj+GHKgrhTSZLKuViKVwhhixDCGyGE8SGEcSGEPePIoQquTmvCMU/SetVP3F6lF5e+NJz5S1fFnUqSVI7FNeL1IPBxFEXbA+2AcTHlUEW33WGw15V0L/yUzkv6cdVrIygq8ngvSVLZSHnxCiHUAvYFngaIomhVFEULUp1D+t0Bf4dWe3F35Z5Mm/g9D3/+U9yJJEnlVBwjXlsCecCzIYTvQwg9QwjVS14ohHBhCGFoCGFoXl5e6lOq4sjOgeOfIbtKTXrVfJQe/UYy4Ee3OUnS5hdH8coB2gOPR1G0G7AUuKHkhaIo6hFFUccoijo2aNAg1RlV0dRsTDjuaRrlT+HhGi9w5SvDmbZgedypJEnlTBzFayowNYqiwcnTb5AoYlK8ttyP0PUmDsjvz7FFfbn0peGsLCiMO5UkqRxJefGKomgmMCWEsF3yrAOBsanOIa3RPtfA1gfxt+znKJj6Pbd/4Oc+JEmbT1yfarwceCmEMBLYFbgjphzSH2VlwTE9yKrRkF61HuXdQWN5+/upcaeSJJUTsRSvKIpGJI/f2iWKou5RFM2PI4e0RtXrwQnPsUX+bHrWfpYb3xrJhJmL404lSSoH/OZ6aU1adCIc/C86rfyGi3M/5pJew1i8Ij/uVJKkDGfxktam8yWww9FcGfWi/vzvue6Nkf6YtiRpk1i8pLUJAbo9QqjTimdqPMbg0T/y9MBf404lScpgFi9pXarUhhOep3rBQnrVeYq7PxrLkF/nxZ1KkpShLF7S+jTZhXD4vey4fBg31fiAy14ezuzFK+JOJUnKQBYvqTTanwntTuHsVa/SdsUwLn/5ewoKi+JOJUnKMKUuXiGEOiGEnUIIW4YQLGyqWEKAI+4nNNiex6s+waRff+LeTyfEnUqSlGHWWaBCCLVDCDeFEEYBg4Angd7A5BDC6yGE/VMRUkoLlarDiS9QmVW8VvdJnh7wI5+MmRl3KklSBlnfyNUbwBRgnyiKtouiaO/kF5+2AO4CuoUQzivzlFK6aLAtHPUgrZeN4p4673Bt7x+YNGdp3KkkSRkiZ10Toyg6eB3ThgHDNnsiKd3tfDz89i3HfteTL7O25uJeVXn70r2oWik77mSSpDRXqmO1So5qhRCyQwj/KJtIUgY45A5ouhv35TzOslk/8bd3RvvlqpKk9SrtQfIHhhD6hBCahBDakjjeq2YZ5pLSW05lOOE5crKzeL3uk3ww/Bde/W5K3KkkSWmuVMUriqJTgeeBUUAf4Kooiq4ty2BS2qvTGo55kkZLx/NY/Tf4x7tjGDl1QdypJElprLRvNW4DXAm8CUwCzgghVCvDXFJm2O4w2OtKDlzyAadWG8wlvYazYNmquFNJktJUad9qfB+4JYqii4D9gInAd2WWSsokB/wdWnbhluhJai7+mateG0FRkcd7SZL+V2mLV6coivoBRAn3A93LLJWUSbJz4PhnyK5cnVfrPM7gCVN49Iuf4k4lSUpD6/sC1b0BoihaVHJaFEUTQwi1kgfbSxVbrSZwXE9qL/mFFxu+wn8+m8BXE/PiTiVJSjPrG/E6LoTwTQjh7yGEI0IInUII+4YQzg0hvAh8AFRNQU4p/W3ZlbD/TXRc1Jeran/NFa98z7QFy+NOJUlKI+v7AtW/hBDqAMcDJwCNgeXAOOCJKIq+LvuIUgbZ51r4bRBX/NqTgYUt+fNL1el90Z5UyvHnTSVJpTjGK4qi+UAtYCTQFxgIzAG2DyHsWqbppEyTlQXHPkWo0YDnqz/CL1Om8e8Px8adSpKUJkr7MrwDcDHQBGgKXAh0BZ4KIVxXNtGkDFW9HpzwHNVWzOT1xi/y/LeTeHfEtLhTSZLSQGmLVz2gfRRF10ZRdA3QEWgA7AucXUbZpMzVohMc/E+2W/Al/2zQnxveHMWPsxbHnUqSFLPSFq+WQPFvhcwHWkVRtBxYudlTSeVB50thh6M4Y8kzdKn0Exf3GsaSlQVxp5Ikxai0xetlYFAI4R/JH8f+GnglhFAd8AAWaU1CgG6PErZoyWOVH2bx3Jlc/8ZIf0xbkiqw0v5W47+AC4AFwELg4iiK/hlF0dIoik4rw3xSZqtSG058gcor5/NWo2f5aNQ0nvl6UtypJEkxWefXSRQXRdEwYFgZZpHKpya7wOH30OL9K3mgyXZc3Sebds1r07F13biTSZJSzC8XklKh/Vmwy8kcNf8FutWawJ9fHk7eYg+PlKSKxuIlpUIIcOR/CA22524epvLyWVzxyvcUFBbFnUySlEIWLylVKlWHE58np3AFb9V/iu9+mcX9fX+MO5UkKYUsXlIqNdgOjn6I+vO/55kWH/F4/5/pO3ZW3KkkSSli8ZJSbefjYffz2TfvFc5vMJare49g8tylcaeSJKWAxUuKwyF3QNPduGnlQ7RgNhf3Gs6K/MK4U0mSypjFS4pDTmU44TmysgKv1XmcX2bO4W/vjPbLVSWpnLN4SXGp0xqOeZKa88fQu9X7vDFsKq99NyXuVJKkMmTxkuK03WGw15W0m/km1zUdyd/fG8PoaQvjTiVJKiMWLyluB/wdWnbhksUP0aHqbC7uNYwFy1at/3qSpIxj8ZLilp0Dxz9DqFSdZ6o/zKJFC7i69w8UFXm8lySVNxYvKR3UagLH9aTqgp94p+UbfD5+Fo/1/ynuVJKkzcziJaWLLbvC/jex5YwPuav1cP7T90cGTpwTdypJ0mZk8ZLSyT7XwlYHclLeIxxabxZXvPo9MxYujzuVJGkzsXhJ6SQrC459ilC9Pg9mPUCl/EVc+tJwVhX4Y9qSVB5YvKR0U70enPAcuUum8U7zl/n+t/nc0Wdc3KkkSZuBxUtKRy06wcH/pPH0z+ix9WCe+2YS7/0wPe5UkqRNZPGS0lXnS2GHozh4+mOc1mQGN7w5kh+mLIg7lSRpE1i8pHQVAnR7lFC7Bf/Mv4+tqi/nzGeGMHb6oriTSZI2ksVLSmdVasOJL5C9fB69GzxLjVw4/enBTJy1OO5kkqSNYPGS0l2TXeDwe6n62wA+bvMauaGIU3sO5tc5S+NOJknaQBYvKRN0OAsO+Bs1J7zBp1u/RVFhIac9NYgp85bFnUyStAEsXlKm2PevsO911B7/Kn23f58lK/M5tecgv2BVkjKIxUvKJPvfBHtdSd2xL9Jvp0+Yv3QVpz01mNmLV8SdTJJUChYvKZOEAAfdBp0vpcGYZ+i78+fMWLic03sOZt7SVXGnkySth8VLyjQhwCF3wO7n02T0k3y660Amz13GGU8PZuGy/LjTSZLWweIlZaIQ4LB7of2ZtBj1CB/vNogfZy3mrGeHsGRlQdzpJElrYfGSMlVWFhz5ILQ7hTajHuCDDsMZNW0h5z77HctWWb4kKR1ZvKRMlpUF3R6Ftsex3ch7ebfDSIZOnseFLwxjRX5h3OkkSSVYvKRMl5UNxzwJOxxN21F38kbHcQz8aQ6XvjScVQVFcaeTJBVj8ZLKg+xcOO5p2PYw2o/6F692/JHPx8/mile+p6DQ8iVJ6cLiJZUXOZXgxOdh64PoPPo2nu/wCx+Pmck1r/9AYVEUdzpJEhYvqXzJqQwn9YI2+7Lf2L/To/1k3h0xnRvfGkmR5UuSYmfxksqb3KpwyqvQck/+NO5vPLTrFHoPnco/3htDFFm+JClOFi+pPKpUDU59DZp14Kgfb+benafx4qDJ3NFnnOVLkmJk8ZLKq8o14fQ3CI134fhfbub2nWby1Fe/8p++P8adTJIqLIuXVJ5VqQ1nvEVosD2nTb6Jm3eYxcOf/8SjX/wUdzJJqpAsXlJ5V7UOnPEOoe5WnD/lJq7eNo97P5lAz69+iTuZJFU4Fi+pIqheD858l7BFSy6feTN/3nout384jhcHTY47mSRVKBYvqaKo0QDOeo9QoxHX5t3E+VvO45Z3RtN76JS4k0lShWHxkiqSmo3hrPcJ1epx87ybOb3VAq5/cyTvjpgWdzJJqhAsXlJFU7tZonxVrsW/Fv2N45ot5OreP/Dx6BlxJ5Okcs/iJVVEW7RMvO2YU4V7lv2dwxsv5PJXvueL8bPjTiZJ5ZrFS6qo6m4JZ71HVlYWD676B/s3WMRFvYYxcOKcuJNJUrll8ZIqsvrbwJnvkVVUyBOFt9GlzmLOf+E7hvw6L+5kklQuWbykiq7h9nDmu2QVLOfp8E92q7WYc54dwve/zY87mSSVOxYvSdC4LZzxDtmrFvFizu1sV20JZz0zhNHTFsadTJLKFYuXpISmu8Lpb5OzfB6vVbmDVpWXcMbTg/lx1uK4k0lSuWHxkvT/mneA098gd+lM3qx+Fw2yFnPqU4P5JW9J3MkkqVyweEn6o5ad4bTeVFo0hfdr30utokWc1nMwU+YtizuZJGU8i5ek/9V6bzjlFSov+IUP6/6HnFWLOOWpQUxfsDzuZJKU0SxektZsq/3h5JeoOn8Cn9R/gIJlCzmt52BmL1oRdzJJylgWL0lrt83BcMLzVJs7ms8aPcLiRQs4redg5i5ZGXcyScpIFi9J67b94XDc09TI+55+TR5n9rz5nPH0EBYuy487mSRlHIuXpPXbqTsc8yS1Zw/hi+Y9+G32PM58dgiLV1i+JGlDWLwklc4uJ0C3R6k78xu+aPk0E6fN4dznvmPZqoK4k0lSxrB4SSq9XU+Fox6gwYwBfN7qeX6YnMf5zw9lRX5h3MkkKSNYvCRtmA5nw+H30XhGPz5v3Yshv8zm4l7DWFlg+ZKk9bF4SdpwnS6AQ+6g+YxP+azNK3w5YRaXv/w9+YVFcSeTpLRm8ZK0cfb8Mxx0K62n9+GTLXvTd+wMru79A4VFUdzJJClt5cQdQFIG2/svULCKbfrfwYdtsjnih+OolJ3FvcfvQlZWiDudJKUdi5ekTbPfdVC4kh2/up+32+TQfXh3quRmcXv3toRg+ZKk4ixekjZNCHDALVC4il2/eZjerXM5cfARVM7J5pYjd7B8SVIxFi9Jmy4EOPhfUJhPp8FP8GKrbM74+lCqVsrir4dsH3c6SUobFi9Jm0cIcOhdULCSfYY9S8+WOZz/RaBKTjaXH7hN3OkkKS1YvCRtPiHAEf+BwnwOGvEsj7bM4c99oUpuNhfsu2Xc6SQpdhYvSZtXVhYc/RAU5XPEyKdY2TyHq/tA5dwsztyzddzpJClWFi9Jm19WNnR7DApWcuzYx1nZLJsb34XKOVmctHvLuNNJUmwsXpLKRnYOHNcTigo4ZfwjrGiSzQ1vQeWcbLrv1izudJIUC7+5XlLZyc6F45+FbQ7hnPkPcn2joVzz+g98NGpG3MkkKRYWL0llK6cSnPgCbHUAFy34L1fUH87lr3xPv3Gz4k4mSSln8ZJU9nKrwMkvE1rvzRWL7+eCej9wSa/hfDUxL+5kkpRSFi9JqZFbFU59jdBiD65bci+nbzGKC14YyqBf5sadTJJSxuIlKXUqVYfTXic0a88ty+/huBpjOO+57xg2eX7cySQpJSxeklKrck047Q1Co524fdU9HFp1LGc/O4TR0xbGnUySypzFS1LqVd0CznibUH9b7iu4m6654zn96cGMmW75klS+WbwkxaNaXTjzHULdNjwY3Unn7Amc8MS3fDpmZtzJJKnMWLwkxad6fTjrPbJqN+cx7qRbnUlc1GsYj/X/iSiK4k4nSZudxUtSvGo0hLPeJ6tWE+5Y/DfuaDmcez6ewNW9f2BFfmHc6SRps7J4SYpfrSZwfj/Clvtxyqz7eL/NW3zw/WRO7jGI2YtXxJ1Okjab2IpXCCE7hPB9COGDuDJISiNVt4BTe8NeV7HzjDcY3Pwh8mZOpdsjX/uJR0nlRpwjXlcC42K8f0npJisbDr4NjnuaugvG8EWtW9m+6GdOeOJbf99RUrkQS/EKITQHjgB6xnH/ktLczsfDeZ9QKSebZ6JbuKjOUC55aTgP9ZvoQfeSMlpcI14PANcBRWu7QAjhwhDC0BDC0Lw8f89NqnCatIML+xOadeCqRffybLP3eKDveC5/5XsPupeUsVJevEIIRwKzoygatq7LRVHUI4qijlEUdWzQoEGK0klKK9Xrw5nvwu4XsP/cVxnQ5BEGjprIiU9+y8yFHnQvKfPEMeK1F3B0CGES8CpwQAihVww5JGWC7Fw44j44+mFaLBzGN/Vuh9njOPqRgfwwZUHc6SRpg6S8eEVRdGMURc2jKGoNnAx8HkXR6anOISnDtD8TzulDNVbyduV/cGD4jhOf/Jb3fpgedzJJKjW/x0tS5mjRCS7sT3bD7blz1V3cXvt9rnxlGPd/OoGiIg+6l5T+Yi1eURT1j6LoyDgzSMowtZrC2X1g19M4YelLvN/gSZ75fBR/fnk4y1YVxJ1OktbJES9JmSe3CnR7FA69m52WfMPAev9m/JgRHP/4t0xfsDzudJK0VhYvSZkpBOh8MeGMt6lTtIBPa9xKi3nfcPQjXzP8t/lxp5OkNbJ4ScpsW+4HF35Bbt2WPJF1F+eF9zi5x7e8NXxq3Mkk6X9YvCRlvjqt4bxPCTsczSX5z/NMzR7c1HsId3003oPuJaUVi5ek8qFSdTjhOTjw7+y1vD/96tzJewMGc+GLw1iy0oPuJaUHi5ek8iME2Ocawqmv0bRoJv1q/oMlPw7g+Me/Ycq8ZXGnkySLl6RyaNtDCBd8TtVa9Xm50h3sveBduj8ykO8mzYs7maQKzuIlqXyqvw1c0I+srQ/kb/TktqynOPupr+g9dErcySRVYBYvSeVXldpwyiuwzzUcWfApb1e/i3vf+JLbPxhLoQfdS4qBxUtS+ZaVDQf+HU54jm2iSfSr+Q+GfP0Z5z//HYtW5MedTlIFY/GSVDHsdAzhvE+pVb0qb1f9F/V+fotjH/uGyXOXxp1MUgVi8ZJUcTTeGS7oT3arztyX8zhnL3qSYx/5km9/nht3MkkVhMVLUsVSvR6c/jbscQmn8yFPZd3J5U9/xsuDf4s7maQKwOIlqeLJzoHD7oJuj7Eb4+lT7e+88M6H3PreGAoKi+JOJ6kcs3hJqrh2O41wzkc0qBp4r+qtzBr0Guc89x0Ll3nQvaSyYfGSVLE170C4sD+Vmu7C45UeZM9Jj3Hso1/xS96SuJNJKocsXpJUszGc/QHsdgaXZr/DP5bezumP9mXgxDlxJ5NUzli8JAkgpzIc/TAcfh/7ZP3Aa1k3c+uz7/DCt5PiTiapHLF4SdJqIUCnCwhnvkvzyit4r/ItfP5+L/72zijyPehe0mZg8ZKkklrvTbioP1Ubbc2zle6j1ncPc9bTg1mwbFXcySRlOIuXJK3JFi0J535CaHss1+W+xulTb+WkR/rx0+zFcSeTlMEsXpK0NpWqwXFPw0G3cVjWYB5adgN/fvQd+k+YHXcySRnK4iVJ6xIC7H0V4bQ32KbSPHpn3USP55/j6YG/EkVR3OkkZRiLlySVxjYHkXVRf2rWbcyLle7kt4/+y41vjmRVgQfdSyo9i5cklVa9rci6oB9Z2x7CbbnP037ELZzz1FfMXbIy7mSSMoTFS5I2RJVahJNfhv2u58ScAfx15tWc+8j7TJjpQfeS1s/iJUkbKisL9r8JTnyRnXOn03PFtdz62LP0Gzcr7mSS0pzFS5I21o5Hk33BZ9SpXYsXsm7jk1738eSAnz3oXtJaWbwkaVM02omci/qT1Xov7sntQeW+N3Bd7+GsLCiMO5mkNGTxkqRNVa0u2We8RdT5z5yd8ynHjr6Mi574hLzFHnQv6Y8sXpK0OWTnEA69A455kk65P/HvvMv568O9GDt9UdzJJKURi5ckbU7tTib7vE9oUCOXJ1bdQM8n7uPj0TPjTiUpTVi8JGlza9aeSpd8SVbTXflP1oP8+uq1PPb5BA+6l2TxkqQyUaMhlc79kILdzuaSnPfZ4YsL+Guvr1iwbFXcySTFyOIlSWUlpxI53R4kOuK/7Jszmit+Oo8b73+UvmP9vi+porJ4SVIZC7ufS/Y5H9GodjUeL7yVvJcv4oaXHP2SKiKLlySlQss9qHz5IAr3vIKTcwZw1Y9n8M/77ueTMR54L1UkFi9JSpXcqmQf8i+yLujHFvUa8Z+iu1j16lnc9OLnzF/q6JdUEVi8JCnVmrWnyqVfUdj1Zg7LGca1P53B/ff/i49HzYg7maQyZvGSpDjkVCK763XkXDKQKo225faih8jtfQq3vPAJ8xz9ksoti5ckxanh9lS7+DMK/3Qn++aO4/qfz+LJ+2/i41HT4k4mqQxYvCQpblnZZHe5lNzLB0PzjtxY9BR1Xj+W2557j7lL/L1HqTyxeElSuqjTmhrnv0/BUQ+za+5Urv/1XHrdfzUf/TAl7mSSNhOLlySlkxDI6XAmla8cSn6bA7gy6kWzN4/ijmd6M8fRLynjWbwkKR3VakLNs16j8Lhn2bryQv46+WLevv8S+nw/Ke5kkjaBxUuS0lUIZO98LNX+Moxl2x3DBdGbbPP24dzb8wVHv6QMZfGSpHRXrS61T32awlNep3HVQq6ZcgWf3Hc2fYZNJIqiuNNJ2gAWL0nKENnb/YmaVw9l0c5nchp92Pndw3ioRw/yFjv6JWUKi5ckZZLKNdni+IcoOOtDalavxpUzruPr+0+iz5Cxjn5JGcDiJUkZKKfN3mxx9RDm7/ZnjmIAHT88jMefeJDZi1fEHU3SOli8JClT5VahTrc74ILPyarZiEtn/YMR93fn40E/OPolpSmLlyRluOxmu1H/L18zd48b6MpQOn90GM8+diezFy6PO5qkEixeklQeZOdS77Abyb70a1ZssTXn5t3NxP8ewsdfD3H0S0ojFi9JKkeyG25H4yv7k7fP7bQPE9j706N4+ZFbmL1wWdzRJGHxkqTyJyuLBgdeTqXLB7Og3m6cNvdhpv53fz798itHv6SYWbwkqZzKrtua5pd/xOwD/8s2YSr79TuGNx+8hlnzF8cdTaqwLF6SVJ6FQMN9zqXaX4YxvdF+HL/gaeY9uA+ffd7X0S8pBhYvSaoAsms1ps2lbzLr0KdokrWArgNO5MP/XszMuQvijiZVKBYvSapAGnU+kVrXfM8vzY7iyEWvsvzhPfn803cd/ZJSxOIlSRVMVvU6bHvhC8w8+mWqZxfQ9euz6Pefs5iZlxd3NKncs3hJUgXVuP0R1P/rcMa3PJkDFr1H0SOdGdDnVUe/pDJk8ZKkCiyrSk12PO8JZh//NlFOVfYbchED7zuBGTOnxx1NKpcsXpIkGu+8P02u/46RW15A5yWfU+nxzgx8t6ejX9JmZvGSJAGQVakqu5x5H3NO+ZiFlRqw9/fXMPSeI5kxbVLc0aRyw+IlSfqDJtt3ovX1gxi+7VXssmww1Xp0YdCbDxIVFcUdTcp4Fi9J0v/Iysml/am3Me/ML5heuQ2dR/2d0XcfyIzJE+KOJmU0i5ckaa2abLUz213/JUN2vIktV4yl9jP7MPS1O4gKC+KOJmUki5ckaZ2ysrPpdOL1LDz3KyZW2ZmO4+5m4t17M+OnEXFHkzKOxUuSVCpNW23LLtf35Ztd7qDhyinUe/FARrx0M0X5q+KOJmUMi5ckqdRCVhZdjv0zyy78luHV9mLXiY8w5e5OzBz/bdzRpIxg8ZIkbbCmzVqyx3Xv8mX7B6iav4D6rxzO6OeupHDF4rijSWnN4iVJ2ighBPY9+hwKLhnEVzUOoe2k51h8905MePdeovwVcceT0pLFS5K0SZo2bkzXa19hYNfX+Dm0ZLvvbyfvzp355bOnoKgw7nhSWrF4SZI2WQiBvbseyi43f0m/3XswN6rJlgOvZdqduzHlm97gTw9JgMVLkrQZ5WZnceARJ9H6hiF8stO9rMrPp8WnFzDp7i7M/qFv3PGk2Fm8JEmbXdXKORxywoXUvXY4fdrcTOXlM2n49vH8fP9BzJ84OO54UmwsXpKkMlO7RlUOP+s6whXf82HTy6izaBx1XvoTPz58DEumjY07npRyFi9JUplrXG8Ljrjw3yy8cCgf1j2LpnO+ocpTezGhx1msmDs57nhSyli8JEkp06ZZE4644iF+O/1bPqvRjdbTPiA83IEJL1xBweK8uONJZc7iJUlKuR232ZJDr32O0cd8zleVu7L1zy+w8v6d+en1vxGtWBR3PKnMWLwkSbHpsGs7Drzhdb455H2G57Rj6zEPs/Dutvz6wX1QsDLueNJmZ/GSJMUqhMA+XfZhzxs/om+Xl/iJlrQZ+i/m3NmWqV/09EtYVa5YvCRJaSEnO4uD/3QkbW8awIe7Ps6swpo0H3ANM+5qz+zBb/glrCoXLF6SpLRSJTebI7qfSovrB/HutnexfOUqGn50HlPu3ZP5o/0SVmU2i5ckKS3VqlqJbqdeQo2rv+PtljeSs3QWdd44nl//ezBLfhkSdzxpo1i8JElprWHtGhxz7g2sunQobzf8M7UXjKXGCwfz86PHsXLGuLjjSRvE4iVJygitGtXjmEvvYNY5g3mn9hk0mj2QnCe78HPPcyiY91vc8aRSsXhJkjLKDq2b0/0vjzDhpK/oU/Vomk95j6KH2vNLryuJls6JO560ThYvSVJG6rDjthx53XMMObIvn+fuS6uJz7P8vrZMfusfsHJx3PGkNbJ4SZIyVgiBfXZvz8E3vkm//d9hSGhHq5EPsPDunZj20X/8ElalHYuXJCnjZWcF/tS1K3ve1If3dn+RCUUtaDb4Nube1ZbZXz7tl7AqbVi8JEnlRuWcbI4+4mh2vHEAb7Z9hBn5NWj4+dXMuns35g170y9hVewsXpKkcqdG5RyOO/4MGl/7Lb23/DdLVuRT9/1zmX5fF5aM6xd3PFVgFi9JUrlVv2YVTjzzMipdPpjXmlxPtGQWNV47likP/IkVk7+LO54qIIuXJKnca1G/FidddBNLLxjMa3Uvodr8sVR59iAmP34c+bPGxx1PFYjFS5JUYWzbvAEnXXEXv53+Na9XP416MweS9fieTH7mHIrm+yWsKnsWL0lShbPbNq04/tpHGXFMf96tfCSNJ79HwYPtmfLKVX4Jq8pUiDLgEx4dO3aMhg4dGncMSVI5VFQU8dmg78jvdxeHFnzOyqyqLGh3IU0PuxYq14w7njJQCGFYFEUd1zTNES9JUoWWlRX4U5dOHHzjG3yw95sMYmeajniAxXfvxOy+//VLWLVZWbwkSQIq5WTR7eAD2eOGPvTe9TnGFLak4de3Mv/unZk38BkoLIg7osoBi5ckScVUr5zDid2PYbvrPuelbR9k6qrq1P3sL+Td24FFQ16GglVxR1QG8xgvSZLWYfr8ZfR7qyedJz/BNlnTWJhTj2W7nEWTAy6FGg3ijqc0tK5jvCxekiSVwq95i/nm41dp/dOL7BV+YBW5zGhxBE0P+Qu5zXeNO57SiMVLkqTNZPGKfD4b8CVZ3z3FwfmfUy2sZGqt3ai572XU3q07ZOfEHVExs3hJkrSZFRVFfDP2F6b1e5Iu896iRchjXk4jlu16Ds0OuIhQrW7cERWTtCpeIYQWwAtAY6AI6BFF0YPruo7FS5KUzibNXsSQT16i9c8v0okxrKQy01oeTbNDr6Jy07Zxx1OKpVvxagI0iaJoeAihJjAM6B5F0di1XcfiJUnKBEtXFvDFl1+QPeRJ9l/Vnyohn8m1d6fmfpdTd9cjISs77ohKgbQqXv8TIIR3gUeiKOq7tstYvCRJmSSKIoaMmci0fk+w57y3aRLmkZfbhGXtzqPlgRcQqm4Rd0SVobQtXiGE1sCXQNsoihaVmHYhcCFAy5YtO0yePDn1ASVJ2kRT8hYy9OMXaP3zi+zGBJZRlWmtutPysL9QufF2ccdTGUjL4hVCqAEMAP4dRdFb67qsI16SpEy3fFUhX/b/lOzvnmTfVV9SKRTyc+09qbX/5TTY5TDI8jvNy4u0K14hhFzgA+CTKIr+s77LW7wkSeVFFEUMHzuBGf0eY4+579AgLGRGbkuW73oebQ46j+APc2e8tCpeIYQAPA/Mi6LoqtJcx+IlSSqPps9ZwPCPnqXNzy+yEz+zhOr81vo42hx+FVUbbhV3PG2kdCteewNfAaNIfJ0EwE1RFPVZ23UsXpKk8mzFqgK+HvARud89SZeVX5MVIiZusQ+197+cxrscDCHEHVEbIK2K18aweEmSKoIoihg1biwzPnuM3ee+Q92whCm5bVi+2/lsfeC5ZFWuFndElYLFS5KkDDNr7nxG9OlJm59fZFsmszDUZHLrE9nq8Cup3qBV3PG0DhYvSZIy1Mr8Agb3f5/c73rQaeW3RATG19mfOgdcTrOdu/o2ZBqyeEmSVA6MHTuKmZ89Qoe571M7LOXXStuwfLfz2f7As8mqVCXueEqyeEmSVI7kzZvH6A+fpPXPL9KGacwNWzC59UlsffgV1GrQPO54FZ7FS5KkcmhVfiHDvniL3KE96LhqCPlRNmPqHkTdAy6n5c77xB2vwrJ4SZJUzv04dgSzPnuI3eZ+SI2wgh8r7cDy9hfS9sDTyc6tFHe8CsXiJUlSBTFv7hzG9HmMNj+/RHNmMpt6TGpzMtsdcRm16zeNO16FYPGSJKmCKSgoYMTnvckd+hTtVg1nZZTLyLp/ot6BV7Jl2z3ijleuWbwkSarAfh47lLzPHqLd3I+oGlYxptLOrGx/IbsceAo5ublxxyt3LF6SJImFc2cxrs+jtP75ZRqTxwwa8MuWp7LjEZdRp17DuOOVGxYvSZL0u8KCfEb1e4XKw3qww6pRLIsqM6LuodTd/zK223l3gl/KukksXpIkaY0mjxnEnM8epO28vlQO+fyU1ZrpzY+g2T5nsNU2O8QdLyNZvCRJ0jotmjuDiZ89S+2f32XrVeMBGJO9PXmtj6LNvqfRqlWbmBNmDouXJEkqtblTJjD5y17U+/U9WhVMojAKjMxtx4KtjmabrqfSvEmTuCOmNYuXJEnaKHk/f8/Ur3rR5LcPaFw0k5VRDiMqd2TpNt3Ycf8TaVy/ftwR047FS5IkbZooYtb4b5n5dS+aTfuI+tE8lkaV+b7qnqza4Rh23u84GmxRM+6UacHiJUmSNp+iQqaP7MfcQS/TatZn1IoWszCqxvDq+xK1PY7d9jmKOjWrxp0yNhYvSZJUNgrzmTr0QxYNfZXWeV9QjRXMjrZgRK2u5LQ7kY57HUytqhXrtyItXpIkqcxFq5YyZfA7LBvemzbzv6Yy+UyJGjCqzkFU3e1EOu2xD9WrlP9vyrd4SZKklIqWL2DyN69T8MMbtF40hByKmBg1Z3y9g6nd6WQ6ddidKrnZcccsExYvSZIUm6LFeUwe+DJh9Ju0XvoDAKOjLfmp0aHU3+NkOrXbmUo5WTGn3HwsXpIkKS0UzPuNKQNfJnfcWzRfPoGiKDA8bM9vTQ+nceeT6LTTNuRkZ3YJs3hJkqS0s2rWj0z76kWq/fgOjVb9Rn6UzeCwCzNaHE6rLifSYbtWZGdl3u9GWrwkSVL6iiJWThvJtK9epPbP71GvYBYroly+zurAnNZHsc3ex7JrmyZkZUgJs3hJkqTMEEWs+PVbZgzsRb3JH1KrcAGLo6oMzO7Ewq27sePeR7Nzi/qEkL4lzOIlSZIyT2EByyb2Z/Y3L9FwyidUi5YyN6rJwNy9WLZdd3bd6zC2b1I77UqYxUuSJGW2gpUsHfMxcwe/QqMZn1M5WsmMqC4DK+9D/g7H0qnLgWzdKD1+ssjiJUmSyo+VS1g88n0WDnmFxnlfk0MBvxY14ttq+0Pb49l7z71oWa9abPEsXpIkqXxaNo+F37/N0mGv0Wjed2RTxLiilnxXY39y253Ifnt0oOkWqf3dSIuXJEkq/xbPYsHQ11jx/es0XjQSgGFF2zCi9oHU2O149t99ZxrWrFLmMSxekiSpYpk/mflDXqFw5BvUXzqRwigwKNqRov1uZJ8DjyrTu15X8cop03uWJEmKQ51W1DnkBjjkBpg9noVDXmbH0W+R36h6rLEc8ZIkSRXD6s5Txl8/4YiXJElSGnzfV2b/CqUkSVIGsXhJkiSliMVLkiQpRSxekiRJKWLxkiRJShGLlyRJUopYvCRJklLE4iVJkpQiFi9JkqQUsXhJkiSliMVLkiQpRSxekiRJKWLxkiRJShGLlyRJUopYvCRJklLE4iVJkpQiFi9JkqQUsXhJkiSliMVLkiQpRSxekiRJKWLxkiRJShGLlyRJUoqEKIrizrBeIYQ8YHIZ3019YE4Z34c2nOsl/bhO0pPrJf24TtJTKtZLqyiKGqxpQkYUr1QIIQyNoqhj3Dn0R66X9OM6SU+ul/TjOklPca8X32qUJElKEYuXJElSili8/l+PuANojVwv6cd1kp5cL+nHdZKeYl0vHuMlSZKUIo54SZIkpYjFS5IkKUUqXPEKIRwaQpgQQvgphHDDGqaHEMJDyekjQwjt48hZkZRinZyWXBcjQwjfhBDaxZGzolnfeil2ud1DCIUhhONTma8iKs06CSF0DSGMCCGMCSEMSHXGiqgU+7DaIYT3Qwg/JNfLOXHkrEhCCM+EEGaHEEavZXp8z/VRFFWYPyAb+BnYEqgE/ADsWOIyhwMfAQHoDAyOO3d5/ivlOukC1En+/zDXSXqsl2KX+xzoAxwfd+7y/FfKx8oWwFigZfJ0w7hzl/e/Uq6Xm4C7k/9vAMwDKsWdvTz/AfsC7YHRa5ke23N9RRvx6gT8FEXRL1EUrQJeBbqVuEw34IUoYRCwRQihSaqDViDrXSdRFH0TRdH85MlBQPMUZ6yISvNYAbgceBOYncpwFVRp1smpwFtRFP0GEEWR66XslWa9REDNEEIAapAoXgWpjVmxRFH0JYnlvDaxPddXtOLVDJhS7PTU5HkbehltPhu6vM8j8SpFZWu96yWE0Aw4BngihbkqstI8VrYF6oQQ+ocQhoUQzkxZuoqrNOvlEWAHYDowCrgyiqKi1MTTWsT2XJ+TijtJI2EN55X8Po3SXEabT6mXdwhhfxLFa+8yTSQo3Xp5ALg+iqLCxAt5lbHSrJMcoANwIFAV+DaEMCiKoh/LOlwFVpr1cggwAjgA2AroG0L4KoqiRWWcTWsX23N9RSteU4EWxU43J/EKZEMvo82nVMs7hLAL0BM4LIqiuSnKVpGVZr10BF5Nlq76wOEhhIIoit5JScKKp7T7rzlRFC0FloYQvgTaARavslOa9XIOcFeUOLjopxDCr8D2wJDURNQaxPZcX9HeavwO2CaE0CaEUAk4GXivxGXeA85MfuKhM7AwiqIZqQ5agax3nYQQWgJvAWf4yj1l1rteoihqE0VR6yiKWgNvAJdauspUafZf7wL7hBByQgjVgD2AcSnOWdGUZr38RmIUkhBCI2A74JeUplRJsT3XV6gRryiKCkIIlwGfkPgkyjNRFI0JIVycnP4EiU9nHQ78BCwj8UpFZaSU6+TvQD3gseToSkEU4y/LVwSlXC9KodKskyiKxoUQPgZGAkVAzyiK1vhxem0epXys/At4LoQwisRbXNdHUTQnttAVQAjhFaArUD+EMBX4B5AL8T/X+5NBkiRJKVLR3mqUJEmKjcVLkiQpRSxekiRJKWLxkiRJShGLlyRJUopYvCRJklLE4iVJkpQiFi9JFUoIYfcQwsgQQpUQQvUQwpgQQtu4c0mqGPwCVUkVTgjhdqAKiR+SnhpF0Z0xR5JUQVi8JFU4yd/U+w5YAXSJoqgw5kiSKgjfapRUEdUFagA1SYx8SVJKOOIlqcIJIbwHvAq0AZpEUXRZzJEkVRA5cQeQpFQKIZwJFERR9HIIIRv4JoRwQBRFn8edTVL554iXJElSiniMlyRJUopYvCRJklLE4iVJkpQiFi9JkqQUsXhJkiSliMVLkiQpRSxekiRJKfJ/JPwvQ3IlC3wAAAAASUVORK5CYII=\n",
|
|
"text/plain": [
|
|
"<Figure size 720x720 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"filenames": {
|
|
"image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter11_50_2.png"
|
|
},
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"%matplotlib inline\n",
|
|
"\n",
|
|
"import autograd.numpy as np\n",
|
|
"from autograd import grad, elementwise_grad\n",
|
|
"import autograd.numpy.random as npr\n",
|
|
"from matplotlib import pyplot as plt\n",
|
|
"\n",
|
|
"def sigmoid(z):\n",
|
|
" return 1/(1 + np.exp(-z))\n",
|
|
"\n",
|
|
"# Assuming one input, hidden, and output layer\n",
|
|
"def neural_network(params, x):\n",
|
|
"\n",
|
|
" # Find the weights (including and biases) for the hidden and output layer.\n",
|
|
" # Assume that params is a list of parameters for each layer.\n",
|
|
" # The biases are the first element for each array in params,\n",
|
|
" # and the weights are the remaning elements in each array in params.\n",
|
|
"\n",
|
|
" w_hidden = params[0]\n",
|
|
" w_output = params[1]\n",
|
|
"\n",
|
|
" # Assumes input x being an one-dimensional array\n",
|
|
" num_values = np.size(x)\n",
|
|
" x = x.reshape(-1, num_values)\n",
|
|
"\n",
|
|
" # Assume that the input layer does nothing to the input x\n",
|
|
" x_input = x\n",
|
|
"\n",
|
|
" ## Hidden layer:\n",
|
|
"\n",
|
|
" # Add a row of ones to include bias\n",
|
|
" x_input = np.concatenate((np.ones((1,num_values)), x_input ), axis = 0)\n",
|
|
"\n",
|
|
" z_hidden = np.matmul(w_hidden, x_input)\n",
|
|
" x_hidden = sigmoid(z_hidden)\n",
|
|
"\n",
|
|
" ## Output layer:\n",
|
|
"\n",
|
|
" # Include bias:\n",
|
|
" x_hidden = np.concatenate((np.ones((1,num_values)), x_hidden ), axis = 0)\n",
|
|
"\n",
|
|
" z_output = np.matmul(w_output, x_hidden)\n",
|
|
" x_output = z_output\n",
|
|
"\n",
|
|
" return x_output\n",
|
|
"\n",
|
|
"# The trial solution using the deep neural network:\n",
|
|
"def g_trial(x,params, g0 = 10):\n",
|
|
" return g0 + x*neural_network(params,x)\n",
|
|
"\n",
|
|
"# The right side of the ODE:\n",
|
|
"def g(x, g_trial, gamma = 2):\n",
|
|
" return -gamma*g_trial\n",
|
|
"\n",
|
|
"# The cost function:\n",
|
|
"def cost_function(P, x):\n",
|
|
"\n",
|
|
" # Evaluate the trial function with the current parameters P\n",
|
|
" g_t = g_trial(x,P)\n",
|
|
"\n",
|
|
" # Find the derivative w.r.t x of the neural network\n",
|
|
" d_net_out = elementwise_grad(neural_network,1)(P,x)\n",
|
|
"\n",
|
|
" # Find the derivative w.r.t x of the trial function\n",
|
|
" d_g_t = elementwise_grad(g_trial,0)(x,P)\n",
|
|
"\n",
|
|
" # The right side of the ODE\n",
|
|
" func = g(x, g_t)\n",
|
|
"\n",
|
|
" err_sqr = (d_g_t - func)**2\n",
|
|
" cost_sum = np.sum(err_sqr)\n",
|
|
"\n",
|
|
" return cost_sum / np.size(err_sqr)\n",
|
|
"\n",
|
|
"# Solve the exponential decay ODE using neural network with one input, hidden, and output layer\n",
|
|
"def solve_ode_neural_network(x, num_neurons_hidden, num_iter, lmb):\n",
|
|
" ## Set up initial weights and biases\n",
|
|
"\n",
|
|
" # For the hidden layer\n",
|
|
" p0 = npr.randn(num_neurons_hidden, 2 )\n",
|
|
"\n",
|
|
" # For the output layer\n",
|
|
" p1 = npr.randn(1, num_neurons_hidden + 1 ) # +1 since bias is included\n",
|
|
"\n",
|
|
" P = [p0, p1]\n",
|
|
"\n",
|
|
" print('Initial cost: %g'%cost_function(P, x))\n",
|
|
"\n",
|
|
" ## Start finding the optimal weights using gradient descent\n",
|
|
"\n",
|
|
" # Find the Python function that represents the gradient of the cost function\n",
|
|
" # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
|
|
" cost_function_grad = grad(cost_function,0)\n",
|
|
"\n",
|
|
" # Let the update be done num_iter times\n",
|
|
" for i in range(num_iter):\n",
|
|
" # Evaluate the gradient at the current weights and biases in P.\n",
|
|
" # The cost_grad consist now of two arrays;\n",
|
|
" # one for the gradient w.r.t P_hidden and\n",
|
|
" # one for the gradient w.r.t P_output\n",
|
|
" cost_grad = cost_function_grad(P, x)\n",
|
|
"\n",
|
|
" P[0] = P[0] - lmb * cost_grad[0]\n",
|
|
" P[1] = P[1] - lmb * cost_grad[1]\n",
|
|
"\n",
|
|
" print('Final cost: %g'%cost_function(P, x))\n",
|
|
"\n",
|
|
" return P\n",
|
|
"\n",
|
|
"def g_analytic(x, gamma = 2, g0 = 10):\n",
|
|
" return g0*np.exp(-gamma*x)\n",
|
|
"\n",
|
|
"# Solve the given problem\n",
|
|
"if __name__ == '__main__':\n",
|
|
" # Set seed such that the weight are initialized\n",
|
|
" # with same weights and biases for every run.\n",
|
|
" npr.seed(15)\n",
|
|
"\n",
|
|
" ## Decide the vales of arguments to the function to solve\n",
|
|
" N = 10\n",
|
|
" x = np.linspace(0, 1, N)\n",
|
|
"\n",
|
|
" ## Set up the initial parameters\n",
|
|
" num_hidden_neurons = 10\n",
|
|
" num_iter = 10000\n",
|
|
" lmb = 0.001\n",
|
|
"\n",
|
|
" # Use the network\n",
|
|
" P = solve_ode_neural_network(x, num_hidden_neurons, num_iter, lmb)\n",
|
|
"\n",
|
|
" # Print the deviation from the trial solution and true solution\n",
|
|
" res = g_trial(x,P)\n",
|
|
" res_analytical = g_analytic(x)\n",
|
|
"\n",
|
|
" print('Max absolute difference: %g'%np.max(np.abs(res - res_analytical)))\n",
|
|
"\n",
|
|
" # Plot the results\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
"\n",
|
|
" plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
|
|
" plt.plot(x, res_analytical)\n",
|
|
" plt.plot(x, res[0,:])\n",
|
|
" plt.legend(['analytical','nn'])\n",
|
|
" plt.xlabel('x')\n",
|
|
" plt.ylabel('g(x)')\n",
|
|
" plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "875a0c0b",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"## The network with one input layer, specified number of hidden layers, and one output layer\n",
|
|
"\n",
|
|
"It is also possible to extend the construction of our network into a more general one, allowing the network to contain more than one hidden layers.\n",
|
|
"\n",
|
|
"The number of neurons within each hidden layer are given as a list of integers in the program below."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"id": "00684827",
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"editable": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Initial cost: 324.246\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/numpy/core/fromnumeric.py:3208: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\n",
|
|
" return asarray(a).size\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Final cost: 0.119936\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAJOCAYAAABm9wkdAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABcl0lEQVR4nO3dd3gUZcPF4d+zqYQeeu/SixQLIqJgQ1AsiIiCBTuKr7231/Ziw15AAUFAQRGpCkrvvXcIvddAEtKe749Z/GJMSIBkZ8u5rysXbD87Mztz9pnZXWOtRURERETyn8ftACIiIiKhQsVLRERExEdUvERERER8RMVLRERExEdUvERERER8RMVLRERExEdUvDIwxpQxxkw3xsQbYz5wO4/bjDEFjDFjjDFHjTEj8uH+44wx7fL6fgOFMaaqMcYaY8LdznIuvM+hZj7e/2vGmCG5vO5xY0z1/Moi/u1MlpU8eryBxpg3ffV4Z+Nc1zPGmFXGmDbnmCHP58u5Tvu8eF5nK+CLl3fjnehd4e41xgwwxhQ6y7u7HzgAFLHWPpmHMQPVLUAZoIS1trPbYST3QrXUWmsLWWs3u50jJ8aYesaY37xvauKNMVOMMS0zXH5qY3k8w7ptrDHmykz3k3H9d+rvM98/I/9njGljjNlxDre/yxgzMy8z+Zusyoy1tr61dqpLkfKEvz2vgC9eXh2ttYWApkAL4KUzubFxeIAqwGp7Ft8qG+ijFtmoAqy31qa6HSQQGWPC3M6Qn4J0mc93xpgawCxgBVANKA+MAv4wxlyc6erFvOu2xsAkYJQx5q5M1+noLZyn/nrl7zPwDS1fErSstQH9B8QB7TKcfg8Y6/3/RcBs4AiwDGiT4XpTgbdwVoCJwBAgBUgGjgPtgCigL7DL+9cXiPLevg2wA3gW2AMMBl4DRnjvKx5nxXoe8DywD9gOXJUhw93AGu91NwMPZLjs1P0/6b3tbuDuDJcXAD4AtgJHgZlAgZyedxbTr653WhwBVgHXe89/3TstUrzT494sbnsBMMd7293AZ0DkaR7rTm/eg8CLGecdzpuA54BN3st/AmIz3DanefkOMN87LUZnvG2mDDlN1yjgfWAbsBf4KsN0vQuYmen+LFDT+/+BwJfAeOAEzjJ0HbAEOOad/69luG1V7+3DT7NsPwUs9z6vH4HoDJd3AJZ6p8lsoJH3/MFAOs5yfRx4BhgEPOm9vIL3cR/2nq4JHAKM9/R9wEbveb8B5TM930eADcCWLKZBK+/zvDyL5xON89o46M28ACjjvay897EOeR/7vgy3ew0Y4v3/RKBXpvtdBtyUzfz4HBiH8xqbB9TIcLurgHXeafsFMA3omc28OO2y7n3cB73T5bD3cU029zUYGJ/F+V8C00+3bHiXh72AJ6v1Xw7ryjDgBZzXWDywCKjkvayld34c9f7bMtPr602cZew4MAYoAfyAs1wvAKpmmhaP4azTDuCsk0/lrQH85V0GDnjvo1imZf5ZnGX+JBDO6V/71bzzLR6nmH52alnJ9NwL4rwe0r3P4TjOMpftOj6L9WQSkOa97ZFcLmN1vLkO4Sxrt55m/pzL9iBX6xmgM7Ao0+M+CfyKs8cn4zZwTOZljNMvQx97H/uY9/xLs3oNZ/G8SwJjvfP3EDAjw/KS5fYpw7R/M6d1cy6fV2629VlO+7P5O+sb+stfpolXyTtz/ouzcTkItMfZqF/pPV0qw8pkG1Dfu0BGZJyR3uu8AcwFSgOlcF78/80wM1KB/3lnWgHvwpUEXO29z++BLTglIwJng7Yl04ulBmCAy4AEoGmm+3/De9v23suLey//3PscKuC8GFp6c5z2eWeadhE4G7kXgEjgCpwXU+2cXizey5vhrBTDcV7ca4DHs7luPe9C39qb80Pv8zs17x73TuuK3su/BoZ5L8vNvNwJNMBZwf6cXe5cTNe+OAUgFiiMs5F5J6cXd4YVwVHgEm/OaO/jNfSeboSz0eyUeYV4mmV7Ps4GItY7fR/0XtYUZyVwoXf+9/BePyrDbTO+IbmH/1/h3I6z4vwxw2Wjvf+/AmeD2NQ7Hz7FWwYyPN9J3jwFMk4DnOV+O3BBNs/nAe/0jPFmboazWx+cjecX3mnWBNgPtM28HALdgVmZlqsjGZ535vlxCKc0heNs5Id7LyuJs4G4yXtZb5yVc3bF67TLuvdxxwLFgMre/Ndkc197yGLFDVyOs2GPyW7ZAKp7z6+b1XzOYV35NM6bwdo465zGOAUqFqcs3ul9fl29p0tkeH1txFlXFQVWA+tx3licWs8NyDQtpnjvt7L3uj29l9XEef1G4axTpwN9My3zS3HW5QXI+bU/B2ddEoWzbonn9K/9HZnOy3Ydn8Xt7+Lfr//TLWMFcV4Pd3sva4rz2qqfzf2fy/agDblYz3in06FTy4/38iXAzRmez5uZcsXx/+vpLJch72V34CxP4TglZQ/eN4qcvni9g/MGN8L7d6n3vnPaPv2dNZt5k3ldcLrnlZttfZbT/mz+fF6U8vrPO/GO46x8t+KsvAvgvGsanOm6vwM9MqxM3sjiRZSxeG0C2mc4fTUQl2FmJPPPEYjXgEkZTnf0Zgvzni7sXRiKZfNcfgV6Z7j/RDKseHE2tBfhvLgSgcZZ3Mdpn3em8y/1vjg8Gc4bhvfd0uleLNnkfxwYlc1lr+BdIXlPF/ROv1ML/hq8G1rv6XI4G8LwXM7LdzNcVs9732FZ5DjddDU4I1UZ37FezP+P7NxFzi/u73OYRn2Bj7z/r0rOxeuODKf7AF95//8lmTYQOO+oL8tw24zFqwbOa8SDs5J7AO9GCGc07Anv/78F+mS4XSHvfKia4flekcU0eB7n9dfwNM/9HjKMzGU4vxJO4Sic4bx3gIGZl0Oc19AJoIr39FvAd6eZH/0zXNYeWOv9f3dgTobLDM5GMsvildOy7n3cVhlO/wQ8l81tU8milOGMjlicspHlsoFTTC1wSYb5fGr9d+rvvmwedx1wQxbn3wnMz3TeHOCuDK+vFzNc9gEwIcPpjsDSTNPimgynHwb+zCZTJ2BJpmX+ngyns33t45S6VKBghsuGcmbFK9t1fBa3v4usi1d2y1gXYEam638NvJrLZexXcrE9yOa2fclmPYOz7njL+//6OCU7KsPzOV1ByXIZyibDYbzbKE5fvN7A2UtRM9P5OW2f/s6azbw5k+KV07Y+19M+N3/BcoxXJ2ttMWttFWvtw9baRJzjkzobY46c+sPZDVIuw+2253C/5XE2Jqds9Z53yn5rbVKm2+zN8P9E4IC1Ni3DaXA2ZhhjrjXGzDXGHPLma4/zTvyUg/afx1cleG9bEmcFvCmLzLl53hmf33ZrbXqm51ghi+v+izHmPO8Bv3uMMceAtzPl/9djnTphrT2B8841Y+5RGTKvwdkYl8nlc8o4L7fivDPJLkt207UUzmjDogyPM9F7fm79Y5kyxlzoPXB6vzHmKM7uqOxyZWVPFjnBmSZPZpomlfjn8vk3a+0mnA10E5wV2lhglzGmNs6762neq/5jmbfWHseZTxmXiaxeN48DP1lrV5zmuQzG2WgON8bsMsb0McZEeB/zkLU2PsN1s1wOvdcZB9zmPes2nFGG7GQ3/TIvjxZnd0KWcrmsZ/dYmR0g69djOZxdYYezy8H/T5NDGc47tf479dcvm9tWIut1Rub1HPx7+mder2U+nfm5Zn49lgcwxpQ2xgw3xuz0Tsch/Hs6Zrzt6V775YHD3nVJxsc6Ezmt43PjdK/RCzNl7waUzepOzmF7cKbrmUHA7cYYg1O6f7LWnszlc81uGcIY86QxZo33AyNHcEZHc7Ouew9nZOsPY8xmY8xz3vPPaft0hnJaDrKd9mcjWIpXVrbjvFPKuEIqaK19N8N1bA73sQvnxXNKZe95ub19towxUTi7xN7HOc6lGM6xQSYXNz+As0uzRhaX5eZ5n7ILqOT9YMEplXF22+XGl8BaoJa1tgjOkHB2+XfjvGgBMMbE4AxLZ8x9babc0dbanbl8TpUy/L8yzijNgVw+j1MO4GxE6md4nKLWObgZnJGWmAzPIasVaOZlYijOrstK1tqiOKNNuZnHOdmO86414zSJsdYOyyYHOOXqFpxjk3Z6T3cHiuPs3oFMy7wxpiDOfMq4TGR1352BTsaYx7MLbK1Nsda+bq2th7NrvIP38XcBscaYwhmufrrlcBjQ1XsgegGc3VpnajfObm3A+YBNxtNZOJNlPSeTcaZXZrfijMIlnOa2N+K82153Fo+7nazXGZnXc3Bm64GsZH49nlpvvoOz/DTyTsc7+Pd0zLh8ne61vxso7l1GMz5WdrJabnNax+d0+9PZDkzLlL2QtfahzFc8x+0BnMF6xlo7F2ePwKU4hx0MznhxLp7Tv5YhY8ylOKOTt+LsgiuGc9hFjvmttfHW2iettdVxRk+fMMa05cy2Tzmtm891W5+ngrl4DQE6GmOuNsaEGWOivR8nPt3KNbNhwEvGmFLGmJI4u8vy6rtIInH2t+8HUo0x1+Ic7Jsj7zuA74APjTHlvc/vYu+L90ye9zycBfYZY0yEcb7TpCMwPJfPoTDOcTLHjTF1gH+tUDIYCXQwxrQyxkTiDC9nXP6+At4yxlQB8E7zG7yX5eY53WGcj+jHeO97ZIaRxlzxTtd+wEfGmNLeHBWMMVd7r7IMqG+MaWKMicYZPs9JYZzRnCRjzAU4K7q80A940PtO1xhjChpjrstQXvbiHA+U0TSgF85xNeDsQnoUZ4j+1LQaCtztfY5ROCM786y1cTnk2QW0BR4zxjyc1RWMMZcbYxoa59Oex3DKcZq1djvOLsh3vPO2EXAv2Y9kjcdZSb6Bc5xaejbXO51xQENjTCfjfHruEbIZifA6k2U9J68DLY0xbxljYo0xhY0xj+KU0GezuoFxvmOwF/Aq8PxZPuf+wH+NMbW8y0wjY0wJnOl5njHmdmNMuDGmC87u+rFn9ewcTxtjihtjKuEcP/ej9/zCeHeNGmMq4BwzdDrZvvattVuBhcDrxphIY0wrnPVXdvYCJYwxRTOcdybr+L1ARe/6KzfG4kzXO73r1whjTAtjTN0srnvW2wOvM13PfI/zQYRUa23Gr8jIar2RUXbLUGGc3b77gXBjzCtAkdwEN8Z0MMbU9L75OYazpyONM9s+5bRuzul55ee2/l+Ctnh5V+Y34Lwz3Y/T1J/mzJ7zmzgv7OU4BxQu9p6XF/nicT758xPOroXbcd6x5NZT3kwLcHY7/A9nX3iun7e1Nhm4HrgWZ7TnC6C7tXbtGWS4HeeAx378/8r1X6y1q3A2bkNx3qke5p+7dj7Gef5/GGPicQ50vNB729w8p8E4+/H34OyGfSyXzyGzZ3GGvecaZ1fIZJwDSbHWrsfZ2E/G+fRabr7T52HgDe9zegVnfp8za+1CnA9rfIYzLTfiHOdwyjs4K5IjxpinvOdNw1lBnipeM3HeJZ46jbX2T+BlnHffu3He3Z7arZdTpm045etZY0zPLK5SFqeAH8PZlTyN/1+5dcU5FmUXzlcrvGqtnZTN45wEfsE5uHtobrJlcR8HcEad+uDsSq2H81rPbpdLrpf1XDz2BpzdZY1xjjPZDdwMXG2tnZXp6keMMSdwXuvtgc7W2u8yXWeM+ef3eI3K5qE/xFn+/sCZB9/ifEDiIM7o45M40+IZoIN3Gp2t0TifbFuKU3K/9Z7/Os5B5ke95/9yujvJxWv/dpz1xCGcUvr9ae5rLc4GdrP3dVGeM1vH/4Xz4a09xpgcp413HX8VzutnF8666dSHsbK67rlsD850PTMY58NIgzOd/y1Qzzt9fs3idlkuQziHEEzA+SDFVpw9MjkdynNKLZx16nGcYwu/sNZOPZPtUy7WzTk9r3zb1mfl1MfHRQKWMWYqzoGb/d3OIoHJOLszdgDdrLVns+tSvIwxFmeX7Ea3s0jWjDEFcHZZN/W+ERAfCtoRLxGR0/Huvirm3aV66pituS7HEvGFh4AFKl3u0DcDi0iouhhnV2UkzndTdfJ+IlokaBlj4nDeZHRyN0no0q5GERERER/RrkYRERERHwmIXY0lS5a0VatWdTuGiIiISI4WLVp0wFqb5ZdvB0Txqlq1KgsXLnQ7hoiIiEiOjDHZ/oqCdjWKiIiI+IiKl4iIiIiPqHiJiIiI+EhAHOMlIiIi5y4lJYUdO3aQlJTkdpSgEB0dTcWKFYmIiMj1bVS8REREQsSOHTsoXLgwVatWxfldajlb1loOHjzIjh07qFatWq5vp12NIiIiISIpKYkSJUqodOUBYwwlSpQ449FDFS8REZEQotKVd85mWqp4iYiIiPiIipeIiIgEnIEDB9KrV68cr7Nr166/T/fs2ZPVq1ef8WNNnTqVDh06nPHtsqLiJSIiIkEpc/Hq378/9erVczGRipeIiIj4WKdOnWjWrBn169fnm2++AaBQoUK8+OKLNG7cmIsuuoi9e/cCMGbMGC688ELOP/982rVr9/f5p8THx1OtWjVSUlIAOHbsGFWrVmXEiBEsXLiQbt260aRJExITE2nTps3fP0E4ceJEmjZtSuPGjWnbti0A8+fPp2XLlpx//vm0bNmSdevW5flz19dJiIiIhKDXx6xi9a5jeXqf9coX4dWO9XO83nfffUdsbCyJiYm0aNGCm2++mRMnTnDRRRfx1ltv8cwzz9CvXz9eeuklWrVqxdy5czHG0L9/f/r06cMHH3zw930VLlyYNm3aMG7cODp16sTw4cO5+eab6dy5M59//jnvv/8+zZs3/8fj79+/n/vuu4/p06dTrVo1Dh06BECdOnWYPn064eHhTJ48mRdeeIGff/45T6eRipeIiIj41CeffMKoUaMA2L59Oxs2bCAyMvLv46iaNWvGpEmTAOe7x7p06cLu3btJTk7O8juzevbsSZ8+fejUqRMDBgygX79+p338uXPn0rp167/vKzY2FoCjR4/So0cPNmzYgDHm71G0vKTiJSIiEoJyMzKVH6ZOncrkyZOZM2cOMTExtGnThqSkJCIiIv7+eoawsDBSU1MBePTRR3niiSe4/vrrmTp1Kq+99tq/7vOSSy4hLi6OadOmkZaWRoMGDU6bwVqb5VdBvPzyy1x++eWMGjWKuLg42rRpc87PNzMd4yUiIiI+c/ToUYoXL05MTAxr165l7ty5OV6/QoUKAAwaNCjb63Xv3p2uXbty9913/31e4cKFiY+P/9d1L774YqZNm8aWLVsA/t7VmPGxBg4ceEbPK7dUvERERMRnrrnmGlJTU2nUqBEvv/wyF1100Wmv/9prr9G5c2cuvfRSSpYsme31unXrxuHDh+natevf59111108+OCDfx9cf0qpUqX45ptvuOmmm2jcuDFdunQB4JlnnuH555/nkksuIS0t7RyfadaMtTZf7jgvNW/e3J76FIKIiIicnTVr1lC3bl23Y+SLkSNHMnr0aAYPHuzTx81qmhpjFllrm2d1fR3jJSIiIgHt0UcfZcKECYwfP97tKDlS8RIREZGA9umnn7odIdd0jJeIiIiIj+Rb8TLGfGeM2WeMWZnhvFhjzCRjzAbvv8Xz6/FFRERE/E1+jngNBK7JdN5zwJ/W2lrAn97TfiE1JdntCCIiIhLk8q14WWunA4cynX0DcOpLOAYBnfLr8c/E/FGfsefthiSe+Pd3fYiIiIjkFV8f41XGWrsbwPtv6eyuaIy53xiz0BizcP/+/fkaqnD5WlS0e1j264f5+jgiIiLyT6+99hrvv/++2zF8xm8PrrfWfmOtbW6tbV6qVKl8fay6F17Nyqgm1NrwLUkJGvUSERGR/OHr4rXXGFMOwPvvPh8/frZMm+cowVGWjfrI7SgiIiJB7a233qJ27dq0a9eOdevWAdCmTRueffZZLrjgAs477zxmzJgBOD/dc9NNN3HNNddQq1YtnnnmGTejnzNff4/Xb0AP4F3vv6N9/PjZqn/xtayY0oSaG74lKeE/RMcUdjuSiIhI/pnwHOxZkbf3WbYhXPvuaa+yaNEihg8fzpIlS0hNTaVp06Y0a9YMgNTUVObPn8/48eN5/fXXmTx5MgBLly5lyZIlREVFUbt2bR599FEqVaqUt9l9JD+/TmIYMAeobYzZYYy5F6dwXWmM2QBc6T3tP9o8RwmOsGJ0X7eTiIiIBKUZM2Zw4403EhMTQ5EiRbj++uv/vuymm24CoFmzZsTFxf19ftu2bSlatCjR0dHUq1ePrVu3+jp2nsm3ES9rbddsLmqbX495rhpcfA3Lpzah+rr+nEx8nKgCGvUSEZEglcPIVH4yxmR5flRUFABhYWGkpqb+6/ysLgs0fntwvRuMMXCZM+q1fPTHbscREREJOq1bt2bUqFEkJiYSHx/PmDFj3I7kUypemTRseQ3LIxtTbW0/TiYedzuOiIhIUGnatCldunShSZMm3HzzzVx66aVuR/IpY611O0OOmjdvbhcuXOizx1s2cxyNJ9/OwjpP0/y2l3z2uCIiIvlpzZo11K1b1+0YQSWraWqMWWStbZ7V9TXilYVGl7RneURjqq7tT3LiCbfjiIiISJBQ8cqCMYbU1s9QksOs/K2v23FEREQkSKh4ZeP8VtexPKIRldf0IyVJo14iIhIcAuEQo0BxNtNSxSsbxhhSWp0a9dInHEVEJPBFR0dz8OBBla88YK3l4MGDREdHn9HtfP3N9QGlaesOLJvRiMqrvyE1qTfh0QXdjiQiInLWKlasyI4dO9i/f7/bUYJCdHQ0FStWPKPbqHidxqlRrxJT72DpmE9o0vl5tyOJiIictYiICKpVq+Z2jJCmXY05aHZZB5aFN6TS6q9J1bFeIiIicg5UvHJgjOFky6cpYQ+zauynbscRERGRAKbilQvN21zPsvCGVFj1FWknE9yOIyIiIgFKxSsXPB5DUsunKGkPs3rsJ27HERERkQCl4pVLLdrcwLKwhpRb+TVpyYluxxEREZEApOKVSx6PIaHlU5S0h1itY71ERETkLKh4nYELL7+BZWENKL/iS9I16iUiIiJnSMXrDHg8hviLnqKEPcSacRr1EhERkTOj4nWGLm7biaVhDSi7XKNeIiIicmZUvM5QmMdw/MInKWEPsXa8Rr1EREQk91S8zsJFV9zAMk99Si/TqJeIiIjknorXWQgPD+PohU9S0h5i3YTP3Y4jIiIiAULF6yy1bNuJpZ76lFr6OTZFo14iIiKSMxWvsxQeHsaRC7yjXuM16iUiIiI5U/E6B5e068RSTz1KLftCo14iIiKSIxWvcxARHsbhFk9QIv0gGyZ+6XYcERER8XMqXueo1ZU3sdTUI3bJZxr1EhERkdNS8TpHEeFhHGzxBCXTD7Lxd416iYiISPZUvPLApVfexDJTl9jFn2FTktyOIyIiIn5KxSsPREaEsb/5fyiRfpBNf2jUS0RERLKm4pVHLr3qZpaauhRf9KlGvURERCRLKl55JCoinH3NnFGvLX985XYcERER8UMqXnmotXfUq+iiTyH1pNtxRERExM+oeOWh6Mhw9p7fmxLpB9isY71EREQkExWvPHbZNZ1ZaupQdKFGvUREROSfVLzyWHRkOLubPE6J9APETfra7TgiIiLiR1S88kGbazqzlDoUXvixRr1ERETkbype+aBAVDi7mvSmRNoBtk7WqJeIiIg4VLzySZtrO7OU2hRc8IlGvURERARQ8co3MVER7Gjcm5Jp+9n21zduxxERERE/oOKVjy6/9lZn1GuejvUSERERFa98VTA6gm2NHqNE2n62/9XP7TgiIiLiMhWvfHZF+y4s4zxi5vXVqJeIiEiIU/HKZ4WiI9ja0Bn12jGlv9txRERExEUqXj5wefsuLOU8CsztC6nJbscRERERl6h4+UDhApHENXiMEmn72DlVx3qJiIiEKhUvH7niui4soxYF5vTVqJeIiEiIUvHykSIFItlU71Fi0/axa6qO9RIREQlFKl4+1Pa6riyztYia+5FGvUREREKQipcPFS0YycZ6vSiRuo/d0791O46IiIj4mIqXj7Xt0JWlthZRszXqJSIiEmpUvHysWMEoNtR9hNjUveydoVEvERGRUKLi5YJ2HW5nma1J+CyNeomIiIQSFS8XFC8Uxbq6vSiRupd9M79zO46IiIj4iIqXS5xPONYkfOaHGvUSEREJESpeLilROJq1tR8mNnUv+2cOcDuOiIiI+ICKl4vaduzGMluDsFka9RIREQkFKl4uKlk4mtXnPUJsyh4OzBrodhwRERHJZypeLmvb8XaW2Rp4Zn6gUS8REZEgp+LlstJFCrCqljPqdXD2QLfjiIiISD5S8fIDbTveznJbAzNDx3qJiIgEMxUvP1CmaAFW1HyI2JTdHJo9yO04IiIikk9UvPxE2453sDy9Bsz4ANJS3I4jIiIi+UDFy0+ULVaApTUf1KiXiIhIEFPx8iPtOt7BsvQaMON9jXqJiIgEIRUvP1K+eAzLajxAbPJujsz53u04IiIiksdUvPxM2+vvZHl6ddKna9RLREQk2Kh4+ZkKxWNYXP0BYpN3cXTuYLfjiIiISB5S8fJDbTt6R72mvadRLxERkSCi4uWHKpUoyKKq91M8eRdH52nUS0REJFioePmpttd3Z0V6NdKnatRLREQkWKh4+anKJQsyv+oDFE/exbH5Q9yOIyIiInlAxcuPte14JyvSq5GmUS8REZGgoOLlx6qWKsS8yvdT/ORO4uf/4HYcEREROUcqXn7uiuudUa/UqX006iUiIhLgVLz8XPXShZlb6T6Kn9zJcY16iYiIBDQVrwBw+fXdWZFelZSp70FaqttxRERE5CypeAWAmmUKM7vS/RQ/uYPjCzXqJSIiEqhUvALEFR3vdEa9/uqjUS8REZEApeIVIGqVLcKsij0pfnIHCRr1EhERCUgqXgGkTcfurEyvSvIUjXqJiIgEIhWvAFKnXFFmVLiXYkk7SFg0zO04IiIicoZUvALMZR16OKNef72rUS8REZEAo+IVYOpVKMq0cvdQLGkHiYs16iUiIhJIVLwC0GUde7AqvQon/9Sol4iISCBR8QpADSoWY0rZe72jXsPdjiMiIiK5pOIVoP4e9dKxXiIiIgFDxStANaxUjL/K3EOxxO0kLdGol4iISCBQ8Qpgrb2jXkk61ktERCQgqHgFsMaVi/Nnmbsplridk0t+dDuOiIiI5EDFK8C16uCMeiVq1EtERMTvqXgFuKZVYplU6i6KJW7j5FKNeomIiPgzFa8gcGnHHqxOr0LSZI16iYiI+DMVryDQrGoJfi/Vg6KJ20he+pPbcURERCQbKl5B4pLrnFGvBB3rJSIi4rdUvILEBdVLMqFED4olbCV52Qi344iIiEgWVLyCyCUdurMmvTKJk9/RqJeIiIgfUvEKIhfVKMW42O4U1aiXiIiIX1LxCjItr+vx/6Ne6WluxxEREZEMVLyCzMU1SzHWO+qVolEvERERv6LiFWSMMVzU3nus16S3NeolIiLiR1S8glCrWqUZU+wOiiRsJVWjXiIiIn5DxSsIGWO48Lq7WJNeiRM61ktERMRvqHgFqdbnlWZ0sTspeiKO1OUj3Y4jIiIiqHgFLWMMF17bgzXplUjQsV4iIiJ+QcUriLWpU4bRRe+giEa9RERE/IIrxcsY8x9jzCpjzEpjzDBjTLQbOYKdMYYW1/ZgbXolEibpWC8RERG3+bx4GWMqAI8Bza21DYAw4DZf5wgVV9Qty6gi3ShyYgtpS390O46IiEhIc2tXYzhQwBgTDsQAu1zKEfSMMTS7pgdL06uT/PvLkHTU7UgiIiIhy+fFy1q7E3gf2AbsBo5aa//IfD1jzP3GmIXGmIX79+/3dcygcmX9cgwr2ZuokwdJmvSm23FERERClhu7GosDNwDVgPJAQWPMHZmvZ639xlrb3FrbvFSpUr6OGVSMMdx9600MS2tL5KL+sGeF25FERERCkhu7GtsBW6y1+621KcAvQEsXcoSUOmWLsO+CpzlsCxL/y+OQnu52JBERkZDjRvHaBlxkjIkxxhigLbDGhRwh54Grm/N1RHcK71tI6pKhbscREREJOW4c4zUPGAksBlZ4M3zj6xyhKCYynAtvepRF6bVImfgSJB52O5KIiEhIceVTjdbaV621day1Day1d1prT7qRIxS1rVeO8ZWfIjL5CPHjX3M7joiISEjRN9eHoHtvuYFhXE3BFYOwO5e4HUdERCRkqHiFoPLFCpDW5gUO2iIcHfmYDrQXERHxERWvENWtdUMGFOxJscPLSVow0O04IiIiIUHFK0SFh3m4sksv5qXXIf2PV+HEQbcjiYiIBD0VrxB2fpVY5tZ5gcjU4xwa86LbcURERIKeileIu6tTe4Z5rqPY2uGkbVvgdhwREZGgpuIV4orGRBDb/hX22WIcGfkopKe5HUlERCRoqXgJ7ZvXYkSJBylxbA3xs/RdtiIiIvlFxUswxnDdbY8wO70BYVPehOP73Y4kIiISlFS8BIDqpQuzvtmrhKclsveXZ9yOIyIiEpRUvORvt7Vvy4jITpTZ/AsnN89yO46IiEjQUfGSv0VHhFHtptfYaUtwbGRvSEt1O5KIiEhQUfGSf2hZtzITK/amVMIGDkz51O04IiIiQUXFS/6lY5f7mUkTCs7qgz222+04IiIiQUPFS/6ldJECHLj0v3jSk9nx05NuxxEREQkaKl6Spesvv5RfC3am0o5xHF87xe04IiIiQUHFS7Lk8Rga3vY6220pTox6HNJS3I4kIiIS8FS8JFv1KpdhVq2nKXMyjh0TP3A7joiISMBT8ZLT6tD5XmZ4mlNiwUekHt7udhwREZGApuIlp1UoKpz0q9/F2DS2DfuP23FEREQCmoqX5Kj1Bc2ZULwb1fdN4uCyCW7HERERCVgqXpIjYwzNb3+VOFuWlLFPQepJtyOJiIgEJBUvyZVKpWNZ2ehFyqbsYOOv77gdR0REJCCpeEmuXd3pDmaEX0zFlZ+TuG+z23FEREQCjoqX5FpEmIcind4j3Rq2DXvc7TgiIiIBR8VLzkjjBg2ZVvYuah+exvZ5v7odR0REJKCoeMkZu6jbK2ymAhF/PEv6yQS344iIiAQMFS85Y8WLFGL7Ra9TNm0Pq0b+1+04IiIiAUPFS85K66tvYXb0ZZy3oR+Hd6x1O46IiEhAUPGSs2KModyt75Niw9g9vDdY63YkERERv6fiJWetWvXzWFD1Aeodn8vaaT+6HUdERMTvqXjJObnothfYbCpRbNrLnEyMdzuOiIiIX1PxknNSoEA0R654l7J2H8uHvuJ2HBEREb+m4iXnrOmlHZhX+Eoab/uenRuXux1HRETEb6l4SZ6o1vVDThLB4ZG9senpbscRERHxSypekidKl6/MqjqP0SBpMYt/H+R2HBEREb+k4iV5pkXnp9kUVp2K897g2LHDbscRERHxOypekmfCwiOw7d+nDIdYMeRFt+OIiIj4HRUvyVM1m7VlcWwHLtg7nHUr5rsdR0RExK+oeEmeq3XH+ySYApwc/QRpaTrQXkRE5BQVL8lzhWPLsbXJkzRKXcHMUV+6HUdERMRvqHhJvmjY8TE2R55HvRV92Ld/v9txRERE/IKKl+QLExZOgU59KcFRVv7wnNtxRERE/IKKl+SbcvUuYVW5G2l9+BcWzJ3udhwRERHXqXhJvjqv23uc8BQk6o+nSUpOdTuOiIiIq1S8JF9FFS7J/gtfoFH6Wqb8+LHbcURERFyl4iX5ruZVD7KlQH0u2NiXzdt3uB1HRETENSpekv88Hord8gnFTDzrhz2HtdbtRCIiIq5Q8RKfKF6jOZuq3MaVJ8by51+T3I4jIiLiChUv8ZmaXd4hPqwoZWa8wOHjSW7HERER8TkVL/EZT0xxEi57lYZs4K9hH7gdR0RExOdUvMSnyre+m22Fm3D5ji9YvHaT23FERER8SsVLfMsYSnX5lCImgd2/PE+KfkRbRERCiIqX+FyBio3YeV53rj35B2PGj3E7joiIiM+oeIkrqtz0X46Fx1J74atsPxDvdhwRERGfUPESd0QXIf2qt6hvtjB1aB99t5eIiIQEFS9xTewFt7GrWAuuP9ifKYtWux1HREQk36l4iXuModRtnxJjkkkY/yLHT+pHtEVEJLipeImrIsrW5WDDnnRIn8LIX0a4HUdERCRfqXiJ68p2eJkjEaW5cM3brNpx0O04IiIi+UbFS9wXVYiI9v+jrmcbs4f9j7R0HWgvIiLBScVL/ELBJjeyt/QldDn+Pb9OX+R2HBERkXyh4iX+wRhK3/oJBUwqUVNfZX/8SbcTiYiI5DkVL/EbpmRN4ps/TAdmMnzEULfjiIiI5DkVL/ErsVc9x9GoclwV9z6z1u12O46IiEieUvES/xIZQ4Hr36e2ZwfLf36XpJQ0txOJiIjkGRUv8TuR9TtwsPzldD85jCF/zHY7joiISJ5R8RK/VOKWj4jwWMrPe5MtB064HUdERCRPqHiJf4qtRvLFj9PeM5dhwwfpR7RFRCQoqHiJ3yp0+ZMcK1CJLvs+YeziOLfjiIiInDMVL/FfEdEU7PQhNTy72TquD0cTUtxOJCIick5UvMSvhdW+iqNVr+HetJH0GzvV7TgiIiLnRMVL/F7RTu8T7jE0WPE/lmw77HYcERGRs6biJf6vWCXSWz/NNWELGPXTQFLT0t1OJCIiclZUvCQgRF3am+OFqnHPsS8ZPGOd23FERETOioqXBIbwSAre+BFVPXs5/tcH7DqS6HYiERGRM6biJQHD1LichFrXc5/5lS9+mex2HBERkTOm4iUBJabj/wgLC6fNlg/4c81et+OIiIicERUvCSxFymMuf552YUv445cBJCSnup1IREQk11S8JOCEt3yYhGLn8WhyP774Y6XbcURERHJNxUsCT1gEMZ0+oqI5QPS8vqzbE+92IhERkVxR8ZLAVLUVJ+vewv1hY/h0xETS0/Uj2iIi4v9UvCRgRbV/G8KjuXXfx4xYuM3tOCIiIjlS8ZLAVbgM4W1fonXYCuZPGMTB4yfdTiQiInJaKl4S0DwX3EdSiXo8lT6A//220O04IiIip6XiJYEtLJzoG/pSzhyixurP+XGBdjmKiIj/UvGSwFf5QtKb9uC+8PFM/G0oK3cedTuRiIhIllS8JCh4rnmH9BK1+TDsc14e/AdHE1LcjiQiIvIvKl4SHCILEn7bYIqEp/Fiwns8/eNCfcWEiIj4HRUvCR6lziPshk9o7llHs02f8uW0TW4nEhER+QcVLwkuDW/BNr+XB8LHsWzyD8zaeMDtRCIiIn9T8ZKgY655h7Syjfkg4mveHTqRPUeT3I4kIiICqHhJMAqPIuzWQcREhvFO2gf0/mEuKWnpbqcSERFR8ZIgFVuNsJu+ooHZzHW7PuOd8WvdTiQiIqLiJUGsznXQ8lG6h09i/5wfGLt8l9uJREQkxKl4SXBr+yrpFS+kT1R/vho5gY37jrudSEREQpiKlwS3sAg8nQcQGR3DR56+PD54NidOprqdSkREQpSKlwS/ohUIu7k/NdnOXUc+5fmfl2OtvlxVRER8T8VLQkPNtpjLnuGWsOlErRrG4Llb3U4kIiIhSMVLQsdlz2KrXcZbkQMZMXYCi7cddjuRiIiEGBUvCR2eMMzN/QmPKc4XkR/z9JCZHDx+0u1UIiISQlS8JLQUKo3n1oFUZB9PJn3G48OXkKYf0xYRER9R8ZLQU6Ulpu0rtPfMpfqWoXw8eb3biUREJESoeEloavkY9ryreTnyB6ZNmciUtfvcTiQiIiFAxUtCk8eD6fQVniLl+KbAZ7wyfAbbDyW4nUpERIKcipeErphYPJ0HUZrDvGk/45EhC0lKSXM7lYiIBDFXipcxppgxZqQxZq0xZo0x5mI3cohQsRnm6re5zCym5d4feGPsarcTiYhIEHNrxOtjYKK1tg7QGFjjUg4RuOA+qH8jz0SMYOP83/l50Q63E4mISJDyefEyxhQBWgPfAlhrk621R3ydQ+RvxkDHTzCxVfmqwOd8+OsM1uw+5nYqEREJQm6MeFUH9gMDjDFLjDH9jTEFM1/JGHO/MWahMWbh/v37fZ9SQkt0Ecyt31Pck0Df8M95ZPB8jiWluJ1KRESCjBvFKxxoCnxprT0fOAE8l/lK1tpvrLXNrbXNS5Uq5euMEorKNsBc9wEt7ApujP+Bp35aph/TFhGRPOVG8doB7LDWzvOeHolTxETcd/4d0KQbvcJGkbT2D/rN2Ox2IhERCSI+L17W2j3AdmNMbe9ZbQF9lEz8R/v3oXRdPo/+ikETZzNv80G3E4mISJBw61ONjwI/GGOWA02At13KIfJvkTGYW7+nUHgqX0d/Ru+hC9h3LMntVCIiEgRcKV7W2qXe47caWWs7WWsPu5FDJFsla2Gu/5QG6Wu5P3kwvYYuISUt3e1UIiIS4PTN9SLZaXAztLiPezxjKbbtd977fZ3biUREJMCpeImcztVvQfnz+Ti6HxNmzGHiyt1uJxIRkQCm4iVyOuFR0Hkg0ZFhDIj5jBdHLGTLgRNupxIRkQCl4iWSk+JVMZ2+ombaJp4zg3hoyCISk/Vj2iIicuZUvERyo057aPkYnZlE7f0TeXHUCn25qoiInLFcFy9jTHFjTH1jTHVjjAqbhJ62r0Dli3kv6juWLZ3P0Pnb3E4kIiIB5rQFyhhT1BjzgjFmBTAX+Br4CdhqjBlhjLncFyFF/EJYBNzyHRHRBRlU6DP6/LaY5TuOuJ1KREQCSE4jVyOB7cCl1tra1tpW3u/fqgS8C9xgjLk331OK+Isi5TE396dCyjbejRrIQ4MXcfhEstupREQkQJy2eFlrr7TWDrbWHsniskXW2settd/mWzoRf1Tjckyb57g2fSptTkzkPz8tJT1dx3uJiEjOcnWsVuZRLWNMmDHm1fyJJBIAWj8N1dvwRuRA9q1fwGdTNrqdSEREAkBuD5Jva4wZb4wpZ4xpgHO8V+F8zCXi3zxhcFN/PAVLMKjQ5/SfvJTp6/e7nUpERPxcroqXtfZ2YBCwAhgPPG6tfSo/g4n4vUKlMLcMoGTqHj4v+B29hy1m55FEt1OJiIgfy+2uxlpAb+BnIA640xgTk4+5RAJDlYsx7V7l0tTZ3Jo+nkd+WExyqn5MW0REspbbXY1jgJettQ8AlwEbgAX5lkokkLR8DM67lmc9P8COhbw1brXbiURExE/ltnhdYK39E8A6PgA65VsqkUBiDNz4JZ6i5RhU+HNGz1nJ6KU73U4lIiJ+KKcvUG0FYK09lvkya+0GY0wR78H2IqGtQHHoPIgiaYf5rmh/nv95Gev3xrudSkRE/ExOI143G2NmG2NeMcZcZ4y5wBjT2hhzjzFmMDAWKOCDnCL+r0JTzNVv0/TkAh6OGMuDQxZx/GSq26lERMSPhJ/uQmvtf4wxxYFbgM5AWSARWAN8Za2dlf8RRQJIi56wbQ6PrBrGzIPVeXZkET67/XyMMW4nExERP5DjMV7W2sNAEWA5MAmYCRwA6hhjmuRrOpFAYwx0/BgTW51vC33J/BVrGDArzu1UIiLiJ3J7cH0z4EGgHFAeuB9oA/QzxjyTP9FEAlRUYbj1e2LSTzC42De8O34VC+MOuZ1KRET8QG6LVwmgqbX2KWvtk0BzoBTQGrgrn7KJBK4y9THXfUCdpKW8WPBXHhm6mAPHT7qdSkREXJbb4lUZSM5wOgWoYq1NBLQ1EcnK+d3g/DvokTKChokLeHToElLT9OWqIiKhLLfFaygw1xjzqvfHsWcBw4wxBQF9W6RIdq59D0rX5/Por4jbvJ4PJ613O5GIiLgot7/V+F/gPuAIcBR40Fr7hrX2hLW2Wz7mEwlskTFw6/dEmVR+jP2KflPXMXn1XrdTiYiIS3I74oW1dpG19mNrbV9r7cL8DCUSVErWhOs/pXLCKt4r9jP/+Wkp2w4muJ1KRERckOviJSLnoMFNcMH9dEoaTTvm89APi0hKSXM7lYiI+JiKl4ivXPUmlG/KexFfEb97A6+OXuV2IhER8TEVLxFfCY+CzgMJDwtnROxX/LpwEz8t2O52KhER8SEVLxFfKl4FbvyaMifW8UWJn3h59EpW7jzqdioREfERFS8RX6t9DVzSm7YnxnNb9Bwe/mExRxNT3E4lIiI+oOIl4oYrXoHKLXmFb4g5soEnf1pKerp1O5WIiOQzFS8RN4SFwy3fERZZkOHFv2TWmm18NX2T26lERCSfqXiJuKVIObi5P0VPbGFQ6WG8//taZm864HYqERHJRypeIm6qcTmmzfNccGwSvYrO5rFhS9hzNMntVCIikk9UvETc1vppqHEFj6f0p0ryRnoNXUyKfkxbRCQoqXiJuM3jgZv64YkpwfeFv2Dd1p28O2Gt26lERCQfqHiJ+IOCJaHzAAom7OTHskP4duZmxi3f7XYqERHJYypeIv6i8kXQ7jXqHZnKKyWn8czIZWzcd9ztVCIikodUvET8SctHofZ13J3wHS3CN/LQkEWcOJnqdioREckjKl4i/sQY6PQ5pkgFvo7+lEP7d/HCqBVYqy9XFREJBipeIv6mQHG4dRBRJw/xc9nv+W3pDobM3ep2KhERyQMqXiL+qPz5cM07VD08m/fL/sUbY1ezZNtht1OJiMg5UvES8VfN74UGN3PT0YFcU3ADj/ywmL3H9OWqIiKBTMVLxF8ZAx0/xsTW4MOwT4lI3Mft/eZy4PhJt5OJiMhZUvES8WdRheHW74lIOc6YcgPYeySeO/rP40hCstvJRETkLKh4ifi7MvWgw0cU2TuPv6oPJ+5APN2/m8+xpBS3k4mIyBlS8RIJBE26QrvXKb11LH/VHMGaXUe4e8ACfceXiEiAUfESCRStHofLnqN83Cgm1x7Dkm2H6DloIUkpaW4nExGRXFLxEgkkbZ6DS3pTZctw/qj7O3O3HOD+wYs4maryJSISCFS8RAKJMdDudbjgAWpu/p5x9aYyff1+eg1dQkpautvpREQkBypeIoHGGLjmXWjanXqb+jGqwSwmrd7L4z8uJS1dPy0kIuLPwt0OICJnweOBDn0h9STnL/+cYQ0i6bq8BVHhHt6/pTEej3E7oYiIZEHFSyRQecLghi8g9SQXr/6IAfWf5e7FEB0RxludGmCMypeIiL9R8RIJZGHhcFM/SD3J5ev/xxf1XuTheRAV7uGVDvVUvkRE/IyO8RIJdOGR0HkgVL+caze/zQd11zNgVhx9fl+HtTrmS0TEn6h4iQSDiGi4bSimyiXcFPdf3qq9hS+nbuLTvza6nUxERDJQ8RIJFpExcPtwTIWm3L79NV6stZ0PJ63nm+mb3E4mIiJeKl4iwSSqMHQbiSlTj567XuGJmrt4e/xaBs2OczuZiIig4iUSfAoUgzt/xZSowaN7X+Hhant59bdVDJ+/ze1kIiIhT8VLJBjFxEL30Zgi5Xn64MvcXeUgz49awa9LdrqdTEQkpKl4iQSrQqWh+2+YmBK8cvQlulQ8wpMjljFhxW63k4mIhCwVL5FgVrQC9BiDiSzMO8dfpmO5ozw6bAl/rtnrdjIRkZCk4iUS7IpXgR6/YcLC+TDpFdqWieehIYuZsWG/28lEREKOipdIKChRA7r/hsem8WXqa1wce5z7vl/I3M0H3U4mIhJSVLxEQkXpOnDnr3hSTvCd5w2aFE3g3oELWLztsNvJRERChoqXSCgp1wjuHEVY4mGGRLzFeQUT6PHdfFbuPOp2MhGRkKDiJRJqKjSDbiMIP76bn2L+R6WoRO78dh7r9sS7nUxEJOipeImEoioXQ9dhRBzZwqgi7xEblki3/nPZtP+428lERIKaipdIqKreBm77gaiD6xgX25cYm0i3fvPYdjDB7WQiIkFLxUsklNW6EjoPIHrfMiaU/gxSEujaby67jiS6nUxEJCipeImEurod4aZvKLh7PpPKf01S4glu7zeXfceS3E4mIhJ0VLxEBBreAjd8TuGdM/iz8nccjj9Bt/7zOHj8pNvJRESCioqXiDjO7wbXfUCx7X/xV9Uh7DwUzx3fzudIQrLbyUREgoaKl4j8vxY94aq3KLFtAlNq/sSWfcfo8d18jiWluJ1MRCQoqHiJyD+17AVXvESZuNH8ed4oVu86wj0DFpCQnOp2MhGRgKfiJSL/1vppuPQpKmwZwR91J7B42yF6DlpIUkqa28lERAKaipeIZO2Kl+DiXlTbNIQJ9f9kzuYDPDhkESdTVb5ERM6WipeIZM0YuOpNaH4vtTd+x2/1ZzJ13X4eHbqElLR0t9OJiAQkFS8RyZ4x0P59aHIHDTd+ycgG8/hj9V6e+GkZaenW7XQiIgEn3O0AIuLnPB64/hNITaT5yo8Z0uBp7lgGUeEe+tzcCI/HuJ1QRCRgqHiJSM48YXDj15B6klZr36N//efpuQiiIzz894YGGKPyJSKSG9rVKCK5ExYBt3wHNa+k7aZ3+bTeOobM3cab49ZgrXY7iojkhoqXiOReeBR0GYypdikdtvyXPnU38+3MLbz/xzq3k4mIBAQVLxE5MxEFoOtwTMUL6Lz1Nd6ovZXPp2zis782uJ1MRMTvqXiJyJmLLAjdRmDKNuLOHa/yXK1dvP/HevrP2Ox2MhERv6biJSJnJ7oI3PEzpmRtHtj9Mr1r7OXNcWsYPCfO7WQiIn5LxUtEzl5MLHT/FVOsMo/ve4n7qx3g5dGr+GnBdreTiYj4JRUvETk3BUtCj98whcvw/KEXubPKIZ79ZTmjl+50O5mIiN9R8RKRc1e4LHT/DRNdnDeOvczNFY7yxE/LmLhyt9vJRET8ioqXiOSNYpWcka/waPokvEL7ssd4dNgSpqzd53YyERG/oeIlInknthp0/w2PgY+TX+WyUsd5YMgiZm444HYyERG/oOIlInmr1HnQfTSetJN8nf4GF8aeoOf3C5i/5ZDbyUREXKfiJSJ5r0x9uPNXwk4eY4DnTRoWSeDuAfNZsu2w28lERFyl4iUi+aN8E7hjJOEn9jEs6l2qF0yix3fzWbnzqNvJRERco+IlIvmn0gXQ7SfCj23n54J9KBeZyJ3fzmP93ni3k4mIuELFS0TyV9VWcNsPRB7ewG/FPqKYJ5Hb+81j8/7jbicTEfE5FS8RyX8128Kt3xN1YCXjS35KVHoi3frPY/uhBLeTiYj4lIqXiPhG7Wvh5v4U2LuIP8p+SdrJBLr2m8vuo4luJxMR8RkVLxHxnfo3QqevKLhrNpMr9ichIYFu/eaxLz7J7WQiIj6h4iUivtW4C3TsS5EdU/mzyvccOHacO/rP49CJZLeTiYjkOxUvEfG9ZnfBtX0ovu13/qw+jO0HnfJ1NCHF7WQiIvlKxUtE3HHhA9DudUrFjeWvWiPZtO8YN381m7gDJ9xOJiKSb1S8RMQ9rR6HNs9TbssvTKs/jgPxSXT6YhazN+m3HUUkOKl4iYi7LnsWLulN2fU/MKPuKMoW9ND92/n8MG+r28lERPKcipeIuMsYaPc6tH6GwquHMbboe1xbPYwXR63ktd9WkZqW7nZCEZE841rxMsaEGWOWGGPGupVBRPyEMXDFi3DLd4TvWcYn8U/wQtMUBs6O4+6BC3TQvYgEDTdHvHoDa1x8fBHxNw1uhnsmYqzl/g0P8UPLXczdfJAbv5ilnxgSkaDgSvEyxlQErgP6u/H4IuLHyjeB+6dC2YZcsvgppjSbzdGEk3T6fBYzN+igexEJbG6NePUFngGyPXjDGHO/MWahMWbh/v37fRZMRPxAodLQYwycfwcVl3/KjGoDqFrE0mPAfL6fE+d2OhGRs+bz4mWM6QDss9YuOt31rLXfWGubW2ublypVykfpRMRvhEfB9Z/BNf8jZvPvjIp6nVuqpfLK6FW89OsKUnTQvYgEIDdGvC4BrjfGxAHDgSuMMUNcyCEi/s4YuOhBuONnwuJ38e6h3rzV5DBD5m6jx3fzOZKgnxkSkcDi8+JlrX3eWlvRWlsVuA34y1p7h69ziEgAqXEF3PcXpmApuq3vzS8t1rAw7jCdPp/Fxn3xbqcTEck1fY+XiASGEjWg5ySo0ZamK/7LrAZjSExK4sbPZzN13T6304mI5IqrxctaO9Va28HNDCISQKKLQtdhcMnjlFr3A9PLfky9osncM3AB383cgrXW7YQiIqelES8RCSyeMLjydbipP1F7lzDMvECP6sd5Y+xqnv9lBcmpOuheRPyXipeIBKZGneHu8XjSU3hl33/o22g7wxds585v53HohA66FxH/pOIlIoGrQjO4fyqmdF06rX+WCU1ms3T7IW74fCbr9+qgexHxPypeIhLYCpeFu8ZB467UXfsZc2sOgeQEbvpiNn+t3et2OhGRf1DxEpHAFxENnb6Eq96i+NaJ/FXsbZoXO869gxbyzfRNOuheRPyGipeIBAdjoGUvuH0EEfE7GJDyDL2q7+Pt8Wt5ZuRyTqamuZ1QRETFS0SCTK12cN+fmALFeGL30/Srv5IRi3ZwR/95HDh+0u10IhLiVLxEJPiUrAU9/8RUa82Vm95mSr1xrN5xkBs+m8XaPcfcTiciIUzFS0SCU4Fi0G0EXNyLapt/YG7lL4hJO8rNX8xm0moddC8i7lDxEpHg5QmDq9+CTl9ReO9CJsS8SpvYg9w/eCFfTN2og+5FxOdUvEQk+DXpCneNJzwtic8SnuG5apvpM3EdT/60jKQUHXQvIr6j4iUioaFSC+fLVkvW4v5dLzO09kx+WbKDrv3msi8+ye10IhIiVLxEJHQUKQ93T8A0vIWWW79gds0fiNt9gE6fzWLVrqNupxOREKDiJSKhJaIA3NQP2r1G+R0TmF2mD6XsAW75cg4TV+52O52IBDkVLxEJPcZAq/9A1+EUOLaVX8Jf5PoS23lwyGI+/XODDroXkXyj4iUioav2NXDfn4RFF+bd+Bd4p+pSPpi0nt7Dl+qgexHJFypeIhLaStV2vmy1Sku67unDrzXGMm7Zdrp8PYe9x3TQvYjkLRUvEZGYWOj2M1z0ME12DmVela/Yu28PN3w2ixU7dNC9iOQdFS8REYCwcLjmHbj+M0run8+0Yv+lOjvo/PVsxi3XQfcikjdUvEREMmp6J9w1lqi0EwzhBe4ssY5Hhi7mo0nrSU/XQfcicm5UvEREMqt8Edw/FU+JGrxw5DU+rTyNj/9cz6PDlpCYrIPuReTsqXiJiGSlaEW4eyKmfic67vuaSVWG8OfKrXT+eja7jya6nU5EApSKl4hIdiJj4JYBcMXL1No7gfll3+fE/u3c8Nkslm4/4nY6EQlAKl4iIqdjDLR+Cm4bSpETcUwq9BpNPBvp8vUcRi/d6XY6EQkwKl4iIrlR5zq4dxLhkdF8nfoKj5ZYSO/hS3n/93U66F5Eck3FS0Qkt8rUg/unYipdQK+j7zOo4m98MWU9D/2wiBMnU91OJyIBQMVLRORMxMTCnaPggvu57MBwplX4krmrN3PLV3PYeUQH3YvI6al4iYicqbAIaP8edOhLpcPzmFPybcIPbeSGz2axaOtht9OJiB9T8RIROVvN74buvxGTdoxfo16hTdhyun4zl18W73A7mYj4KRUvEZFzUfUSuH8qYcUr817yf3mpxJ888dNS3p2wljQddC8imah4iYicq2KV4Z7fMXU60P1YP34uN4QB09bwwOCFHNdB9yKSgYqXiEheiCoEnQdBm+dpdngCM8t8wMp167nly9lsP5TgdjoR8RMqXiIiecXjgTbPwa2DKZWwiWlFX6fYkRV0+nwWC+IOuZ1ORPyAipeISF6rdz3c+wdRkVEMDXuDG8Nnc3u/uQybvw1rddyXSChT8RIRyQ9lG8L9U/BUbMZLJz/kw9hfefGXZfQctJC9x5LcTiciLlHxEhHJLwVLwp2/QrO76Rj/I9PKf8GGjeu48sNp/Lxoh0a/REKQipeISH4Kj4SOfaHDR1Q6toSpBZ6lV+FpPDViCfcOWsieoxr9EgklKl4iIr7Q/B54eA6eSs25P/5z5pT9gN2blnHlR9MYsXC7Rr9EQoSKl4iIr8RWc3Y93vAFZZO2MD7ieV4oNJbnRy7mnoELNPolEgJUvEREfMkYOL8bPDIfU6c9XY9/z/ySbxK/eb5Gv0RCgIqXiIgbCpeBWwfBbUOJNfGMCH+Zt2N+5JWR87l74AJ2H010O6GI5AMVLxERN9W5Dh6Zh2nag44JPzO/+Mt4Nk/lqg+n89MCjX6JBBsVLxERt0UXdT75eNc4CheI5ruwt/gkpj9v/jybuwYsYNcRjX6JBAsVLxERf1G1FTw0C1o9QZukP5lX5HmKbRnP1R9N48cF+tZ7kWCg4iUi4k8iCkC7VzH3T6VAiYp8HPYR30X35cOfp9H9u/ns1OiXSEBT8RIR8UflGkHPv+DKN2ietoQZhZ6j+tYRXPPRVIbrNx9FApaKl4iIvwoLh0t6Yx6aTWTF83nd048fo97i61F/aPRLJECpeImI+LsSNaDHGLj+U+qarUwu8Dznbx3AdR/9xTCNfokEFBUvEZFAYAw07Y7ptYCw2tfwhGcYv0a8zJBRv9H9u/nsOJzgdkIRyQUVLxGRQFK4LHQZDF2GUCX6OGOiX6HNtk/p1HcyQ+dp9EvE36l4iYgEorodMY/Mx3N+N+41Yxgb/gxjRg/nzm81+iXiz1S8REQCVYFicP2n0GMMZYpEMyzyLTptf5dbPprAD/O2avRLxA+peImIBLpqrTEPz4FLenOzZxoTwp9i+ujvuOPbeWw/pNEvEX+i4iUiEgwiCsCVb2Du+4tipSvwdWRf7tr+Mnf0Hc2QuVtJT9fol4g/UPESEQkm5Ztg7psC7V6jXfhyxoc9xfIxn3JH/7ka/RLxAypeIiLBJiwCWv0H8/BsYio3pk9EP3rvfJL7+v7E4DlxGv0ScZGKl4hIsCpRA9NjLHToS4uobfwW9jTbx77Lnf1ma/RLxCUqXiIiwczjgeZ34+k1n4jz2vFCxDBe2NWL3n2/53uNfon4nIqXiEgoKFIec9tQ6DyIOgXjGeF5nuPjXqZHv2lsO6jRLxFfUfESEQkVxkD9ToT1mo+nye08HP4b/931IC/1/YpBszX6JeILKl4iIqEmJhbT6XPoPpqKxSL5Pux1wsc/wT1f/8nWgyfcTicS1FS8RERCVfU2hD8yB3txL7qGT+F/e3rSp++HDJy1RaNfIvlExUtEJJRFFsRc/Rae+yYTW6ocn4e9T8mJD/DAVxOIO6DRL5G8puIlIiJQoRkRD03HXvEy10Ys4f19Pfn6kzcYMHOzRr9E8pCKl4iIOMIiMK2fIuzh2RQo34B3PF9R8/c7eeyLURr9EskjKl4iIvJPJWsR2XMitv0HXBS1hT4HHmT4J88yYMZGjX6JnCMVLxER+TePB3NBTyIeXYCn+mU85xnM+ZM68/TnQ9mi0S+Rs6biJSIi2Stageg7f8Le/B11oo/w7sFHmfjJIwyctpY0jX6JnDEVLxEROT1jMA1vJvrxRaTWu5mHPKNo9eeNvPppPzbvP+52OpGAouIlIiK5ExNLgVv7Ye/4hXKFDG8efpo5n97NoCnLNfolkksqXiIickZMzbYU7D2fhKb309UziSun3sD/Pumr0S+RXFDxEhGRMxdViJjr38PcO4mCRUrwwpHXWP1pZ774dQpHE1LcTifit1S8RETkrJlKLSjaezbHWz7LNWEL6LnkZn7v05XvJ84iMTnN7XgifsdY6//75Zs3b24XLlzodgwRETmdozs49Pu7FFk9jDQLo8OuJOKyp+jQqikRYXqfL6HDGLPIWts8y8tUvEREJE8d2ca+cW8Ru2EEadbDmMhrKHrl07Rt3giPx7idTiTfna546S2IiIjkrWKVKd3ta8IeW8SB6jdwY8o4Wo1rx+g+dzN3xVq304m4SsVLRETyhYmtRoUe30KvheyrdA3XJ42m0cjW/PbBfazcsNnteCKuUPESEZF8FVayBlV6DibtoTnsLteWDvEjqDrkYiZ8/DBbtm93O56IT6l4iYiIT0SWqUONB4eReN8sdpZqzdWHhlKyfwv+/OIx9uzd7XY8EZ9Q8RIREZ8qWKE+tXuN4OhdU9lW/CLa7htEzBdNmd7vKY4cPuB2PJF8peIlIiKuKF6tCfUf/5U9XSeztUgzWu/sh/m4EXMHPk9C/GG344nkCxUvERFxVdnaLWj45Fjibp7AlphGXBT3BckfNGTx0FdJSTzmdjyRPKXiJSIifqFqw5Y0eWYiazr8yubIOjRd35fj/2vAqpFvkn7yhNvxRPKEipeIiPiVus0v5/znJ7Go3U9sDq9O/ZXvceTdemwc/T9scoLb8UTOiYqXiIj4HWMMzVpdzfkvTGF6qyFsNpWpueRtDr9bn+0TP4KUJLcjipwVFS8REfFbHo+hdbuONHphGhObf8uW9LJUmvsah9+tz74/P4fUk25HFDkjKl4iIuL3IsM9XNPhFuo8N51RDb9kS2oJSs94gcP/a8jhGd9AarLbEUVyRcVLREQCRsHoCG68+XaqPj2DH2p9TFxyEYr/+TRH+jTi+NwBkJbidkSR01LxEhGRgBNbKIpu3e6i9OPT6V+5D1uTClBo4uMcfa8xJxcMhrRUtyOKZEnFS0REAlaF4jH0vOcBYh6exmdl32R7QgRR43px9IPzSV0yHNLT3I4o8g8qXiIiEvBqlS1CrwcfJfneKbxf/BV2HYfw0Q8Q/2Fz0pePhPR0tyOKACpeIiISRJpWieXJx55gX7fJvFXoeXYdS8bzy70c//gC7KpfVcDEdcZa63aGHDVv3twuXLjQ7RgiIhJA0tMtY5fvZPGEAXRLHEYtz04Sitch5qqXoE4HMMbtiBKkjDGLrLXNs7xMxUtERIJZSlo6P82PY83kQdyd8iM1PLtJKtmA6HYvQu1rVcAkz6l4iYhIyEtITmXgzI1snz6YB+wIqpq9JJduTGS7l6DWlSpgkmdUvERERLyOJCTz1ZR1HJ07hIc9P1PJ7CelXDMi2r4ANdqqgMk5U/ESERHJZPfRRD6dtIb0JUN5LHwU5c0B0ipcQFjbF6HaZSpgctZUvERERLKxcd9xPv5jJUXW/MijEaMpy0HSK7fEc8WLULWV2/EkAKl4iYiI5GDZ9iN8OGE5VbaO5NGIMZTiELbqpZjLX4QqF7sdTwKIipeIiEguzdxwgI8mLKPR3lE8GjmGWHsEW/1yzOUvQKUL3I4nAeB0xSvc12FERET8WataJWlZ4womrKzH7b935JIjo3lsy1iKbr4Sql4KLe51vgcsLMLtqBKAVLxEREQy8XgM1zUqx1X1yzByUV2un9SeqxLGcd/2Pykddxe2UFlMsx7QtAcUreB2XAkg2tUoIiKSg8TkNAbPjWPwrM3UjJ/HfdF/cXH6YjAeTJ320KKnPgkpf9MxXiIiInkgNS2dyWv2MnB2HDu2rKV7xF90i5hGwbSjUKKWsxuycVcoUMztqOIiFS8REZE8tnbPMQbN3sq4JZu5Im0uDxWcQu2UNdjwAphGnaH5vVC+idsxxQV+VbyMMZWA74GyQDrwjbX249PdRsVLRET81ZGEZH5auJ3v52yl6JE13FdgCtcxg4j0JKjYwtkNWa8TRES7HVV8xN+KVzmgnLV2sTGmMLAI6GStXZ3dbVS8RETE36WlW/5au49Bs+NYvnErnSNm8kCBKZRO3gYFYqHpndDsboit5nZUyWd+Vbz+FcCY0cBn1tpJ2V1HxUtERALJhr3xDJoTxy+Ld9A4dQWPFZ7KhclzMTYdU7MdXHAf1GwHnjC3o0o+8NviZYypCkwHGlhrj2W67H7gfoDKlSs327p1q+8DioiInIOjiSmMXLSD7+fEkXRwBz1jptM1/C8KJR+AopWh+d3QtDsULOl2VMlDflm8jDGFgGnAW9baX053XY14iYhIIEtPt0xbv5+Bs+OYtX4314Yv5rEi06iVsATCIp1jwFr0dL4ZX19JEfD8rngZYyKAscDv1toPc7q+ipeIiASLTfuPM3jOVkYu2kGZ5K38p9gMrk75i4jU41CmofOVFA07Q1Qht6PKWfKr4mWMMcAg4JC19vHc3EbFS0REgk18Ugq/LN7JoDlx7Nl/kG4x87ivwF+UPrEBooo43wfW4l4oVdvtqHKG/K14tQJmACtwvk4C4AVr7fjsbqPiJSIiwSo93TJj4wEGzY5jyrq9NPds5JnYGTQ7MR1PerJ+HzIA+VXxOhsqXiIiEgq2HjzB93O28tPC7UQkHeKx2Ll0tn9QMHEXFCoLzXpAs7ugSHm3o8ppqHiJiIgEkBMnUxm1ZCeDZsexad8xOsSsoneRaVQ/MgdjPKDfh/RrKl4iIiIByFrL7E0HGTg7jslr9lLF7OOFMnO5POF3Ik4e1u9D+ikVLxERkQC3/VACQ+ZuZfiC7SQlnqBn7HLujpxMySPLISIGGt7ijIKVa+x21JCn4iUiIhIkEpPT+HWpsxty7Z54LiqwnedKzqLR4T/wpOr3If2BipeIiEiQsdYyb8shBs2O4/dVeyjMCZ4vv4QbUiZQ4NgW/T6ki1S8REREgtjOI4nObsj52zickEyXElt4pNAUKu2birHpUOtKZxRMvw/pEypeIiIiISApJY3flu1i0Ow4Vu06Rq3oo7xcbgEtj40j/MReKFYZmt8D59+p34fMRypeIiIiIcRay8Kthxk4O46JK/fgsSk8UWkjXc0fFNs7V78Pmc9UvERERELUnqNJ/DBvK0PnbePgiWSuiD3I0yVnU2fvWMzJeP0+ZD5Q8RIREQlxJ1PTGLd8NwNnx7F8x1FKR6XycpWVXHViLFEHVzu/D9moCzTpCuWbahTsHKh4iYiICODshlyy/QiDZscxfsVuUtLS6VllPz2j/6TMjj8waSeh5HlOCWvUBYpVcjtywFHxEhERkX/ZdyyJofO38cO8beyPP0m94un0LreKSxP/JGb3fOdKVS91Cli9GyC6iLuBA4SKl4iIiGQrOTWdCSt3M2LhDmZvOkC6hVYlj/NI7GKaHZ1I5NE4CI+GOtc5P09U/XIIC3c7tt9S8RIREZFc2R9/kokrdzNm+W4WxB3CWsuNpXZzT+F51Ds0mbCkw1CwtHMwfuMuULaRjgfLRMVLREREztieo0mMW7Gbsct3sWTbESJI5e7S67k9ajZVDs7ApKdA6Xre48FuhSLl3Y7sF1S8RERE5JxsP5TAuBW7GbNsF6t2HaMY8TxSejk3eGZS+sgywED1y5xdkXU6hPRXU6h4iYiISJ7ZvP8445bvZszyXazfe5zqnt30KrGYq1KnUihxJ0TEQN2O0Pg2qHZZyP1MkYqXiIiI5It1e+IZu3wXY5fvZsuB41wYtp4Hiy/kkqTpRKbGQ+Fy3uPBboMy9d2O6xMqXiIiIpKvrLWs2nWMMct3MXbZbg4cOcpV4Uu5t8g8GiUuwGNToWxDaHSbU8QKl3E7cr5R8RIRERGfOfUlrWOX7Wbcil2kHNvPjZFzubPAXKqeXIs1HkyNK5zjwWq3h8gYtyPnKRUvERERcUV6umVB3CHGLt/N+BW7KZawhVsjZ9M5YjaxqXuxkYUx9a53dkVWaQUej9uRz5mKl4iIiLguNS2duZsPMWbZLn5fuYs6ySvoEjmLaz3ziE5PwBapgGnUxSlhpWq7HfesqXiJiIiIX0lOTWfmxv2MXbab6au3cXHKPLpEzuZilhFGGrb8+ZhGt0HDW6BgSbfjnhEVLxEREfFbSSlpTF23n7HLd7F0zXquSp/JrREzqcMWrAmHWm0xjbvCeddCRLTbcXOk4iUiIiIBISE5lT/X7GPMsl3sXL+YjkznpvDZlOYgaZFF8DS4EdP4Nqh0kd8eD6biJSIiIgEnPimFSav3MnbpdlI3TecGzwzahy2gAEkkF65E5PldnePBStRwO+o/qHiJiIhIQDt8IpnfV+1h0tLNFNn6Ozd6ZnBJ2CrCSCepTFOim90ODW6GmFi3o6p4iYiISPDYH3+SiSt3M3PJSqrsGseNnhnU9WwnzYRzslo7YlrcAbWugvAoV/KpeImIiEhQ2nM0iXErdrNi0Uzq7Z9Ap7BZlDZHSAovQnq9m4hp0Q0qtgBjfJZJxUtERESC3vZDCYxbup2diyfQ7OjvXO1ZSAGTzLGYyoQ1uY2CzW+H2Gr5nkPFS0RERELK5v3H+X3xRo4v/YWWxydzsWc1HmPZfOkHVG/bM18f+3TFKzxfH1lERETEBdVLFeKhq5vA1U1Yt+dp+i9YAst/okWZlq7mUvESERGRoFa7bGFqd2yN7XCp21FUvERERCQ0GB8eYJ8d//zKVxEREZEgpOIlIiIi4iMqXiIiIiI+ouIlIiIi4iMqXiIiIiI+ouIlIiIi4iMqXiIiIiI+ouIlIiIi4iMqXiIiIiI+ouIlIiIi4iMqXiIiIiI+ouIlIiIi4iMqXiIiIiI+ouIlIiIi4iMqXiIiIiI+ouIlIiIi4iMqXiIiIiI+ouIlIiIi4iMqXiIiIiI+ouIlIiIi4iMqXiIiIiI+ouIlIiIi4iPGWut2hhwZY/YDW/P5YUoCB/L5MeTMab74H80T/6T54n80T/yTL+ZLFWttqawuCIji5QvGmIXW2uZu55B/0nzxP5on/knzxf9onvgnt+eLdjWKiIiI+IiKl4iIiIiPqHj9v2/cDiBZ0nzxP5on/knzxf9onvgnV+eLjvESERER8RGNeImIiIj4iIqXiIiIiI+EXPEyxlxjjFlnjNlojHkui8uNMeYT7+XLjTFN3cgZSnIxT7p558VyY8xsY0xjN3KGmpzmS4brtTDGpBljbvFlvlCUm3lijGljjFlqjFlljJnm64yhKBfrsKLGmDHGmGXe+XK3GzlDiTHmO2PMPmPMymwud29bb60NmT8gDNgEVAcigWVAvUzXaQ9MAAxwETDP7dzB/JfLedISKO79/7WaJ/4xXzJc7y9gPHCL27mD+S+Xr5ViwGqgsvd0abdzB/tfLufLC8D/vP8vBRwCIt3OHsx/QGugKbAym8td29aH2ojXBcBGa+1ma20yMBy4IdN1bgC+t465QDFjTDlfBw0hOc4Ta+1sa+1h78m5QEUfZwxFuXmtADwK/Azs82W4EJWbeXI78Iu1dhuAtVbzJf/lZr5YoLAxxgCFcIpXqm9jhhZr7XSc6Zwd17b1oVa8KgDbM5ze4T3vTK8jeedMp/e9OO9SJH/lOF+MMRWAG4GvfJgrlOXmtXIeUNwYM9UYs8gY091n6UJXbubLZ0BdYBewAuhtrU33TTzJhmvb+nBfPIgfMVmcl/n7NHJzHck7uZ7expjLcYpXq3xNJJC7+dIXeNZam+a8kZd8lpt5Eg40A9oCBYA5xpi51tr1+R0uhOVmvlwNLAWuAGoAk4wxM6y1x/I5m2TPtW19qBWvHUClDKcr4rwDOdPrSN7J1fQ2xjQC+gPXWmsP+ihbKMvNfGkODPeWrpJAe2NMqrX2V58kDD25XX8dsNaeAE4YY6YDjQEVr/yTm/lyN/CudQ4u2miM2QLUAeb7JqJkwbVtfajtalwA1DLGVDPGRAK3Ab9lus5vQHfvJx4uAo5aa3f7OmgIyXGeGGMqA78Ad+qdu8/kOF+stdWstVWttVWBkcDDKl35Kjfrr9HApcaYcGNMDHAhsMbHOUNNbubLNpxRSIwxZYDawGafppTMXNvWh9SIl7U21RjTC/gd55Mo31lrVxljHvRe/hXOp7PaAxuBBJx3KpJPcjlPXgFKAF94R1dSrYu/LB8KcjlfxIdyM0+stWuMMROB5UA60N9am+XH6SVv5PK18l9goDFmBc4urmettQdcCx0CjDHDgDZASWPMDuBVIALc39brJ4NEREREfCTUdjWKiIiIuEbFS0RERMRHVLxEREREfETFS0RERMRHVLxEREREfETFS0RERMRHVLxEREREfOT/ACp4NFCgp3PtAAAAAElFTkSuQmCC\n",
|
|
"text/plain": [
|
|
"<Figure size 720x720 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"filenames": {
|
|
"image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter11_52_3.png"
|
|
},
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"import autograd.numpy as np\n",
|
|
"from autograd import grad, elementwise_grad\n",
|
|
"import autograd.numpy.random as npr\n",
|
|
"from matplotlib import pyplot as plt\n",
|
|
"\n",
|
|
"def sigmoid(z):\n",
|
|
" return 1/(1 + np.exp(-z))\n",
|
|
"\n",
|
|
"# The neural network with one input layer and one output layer,\n",
|
|
"# but with number of hidden layers specified by the user.\n",
|
|
"def deep_neural_network(deep_params, x):\n",
|
|
" # N_hidden is the number of hidden layers\n",
|
|
"\n",
|
|
" N_hidden = np.size(deep_params) - 1 # -1 since params consists of\n",
|
|
" # parameters to all the hidden\n",
|
|
" # layers AND the output layer.\n",
|
|
"\n",
|
|
" # Assumes input x being an one-dimensional array\n",
|
|
" num_values = np.size(x)\n",
|
|
" x = x.reshape(-1, num_values)\n",
|
|
"\n",
|
|
" # Assume that the input layer does nothing to the input x\n",
|
|
" x_input = x\n",
|
|
"\n",
|
|
" # Due to multiple hidden layers, define a variable referencing to the\n",
|
|
" # output of the previous layer:\n",
|
|
" x_prev = x_input\n",
|
|
"\n",
|
|
" ## Hidden layers:\n",
|
|
"\n",
|
|
" for l in range(N_hidden):\n",
|
|
" # From the list of parameters P; find the correct weigths and bias for this layer\n",
|
|
" w_hidden = deep_params[l]\n",
|
|
"\n",
|
|
" # Add a row of ones to include bias\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n",
|
|
"\n",
|
|
" z_hidden = np.matmul(w_hidden, x_prev)\n",
|
|
" x_hidden = sigmoid(z_hidden)\n",
|
|
"\n",
|
|
" # Update x_prev such that next layer can use the output from this layer\n",
|
|
" x_prev = x_hidden\n",
|
|
"\n",
|
|
" ## Output layer:\n",
|
|
"\n",
|
|
" # Get the weights and bias for this layer\n",
|
|
" w_output = deep_params[-1]\n",
|
|
"\n",
|
|
" # Include bias:\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n",
|
|
"\n",
|
|
" z_output = np.matmul(w_output, x_prev)\n",
|
|
" x_output = z_output\n",
|
|
"\n",
|
|
" return x_output\n",
|
|
"\n",
|
|
"# The trial solution using the deep neural network:\n",
|
|
"def g_trial_deep(x,params, g0 = 10):\n",
|
|
" return g0 + x*deep_neural_network(params, x)\n",
|
|
"\n",
|
|
"# The right side of the ODE:\n",
|
|
"def g(x, g_trial, gamma = 2):\n",
|
|
" return -gamma*g_trial\n",
|
|
"\n",
|
|
"# The same cost function as before, but calls deep_neural_network instead.\n",
|
|
"def cost_function_deep(P, x):\n",
|
|
"\n",
|
|
" # Evaluate the trial function with the current parameters P\n",
|
|
" g_t = g_trial_deep(x,P)\n",
|
|
"\n",
|
|
" # Find the derivative w.r.t x of the neural network\n",
|
|
" d_net_out = elementwise_grad(deep_neural_network,1)(P,x)\n",
|
|
"\n",
|
|
" # Find the derivative w.r.t x of the trial function\n",
|
|
" d_g_t = elementwise_grad(g_trial_deep,0)(x,P)\n",
|
|
"\n",
|
|
" # The right side of the ODE\n",
|
|
" func = g(x, g_t)\n",
|
|
"\n",
|
|
" err_sqr = (d_g_t - func)**2\n",
|
|
" cost_sum = np.sum(err_sqr)\n",
|
|
"\n",
|
|
" return cost_sum / np.size(err_sqr)\n",
|
|
"\n",
|
|
"# Solve the exponential decay ODE using neural network with one input and one output layer,\n",
|
|
"# but with specified number of hidden layers from the user.\n",
|
|
"def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n",
|
|
" # num_hidden_neurons is now a list of number of neurons within each hidden layer\n",
|
|
"\n",
|
|
" # The number of elements in the list num_hidden_neurons thus represents\n",
|
|
" # the number of hidden layers.\n",
|
|
"\n",
|
|
" # Find the number of hidden layers:\n",
|
|
" N_hidden = np.size(num_neurons)\n",
|
|
"\n",
|
|
" ## Set up initial weights and biases\n",
|
|
"\n",
|
|
" # Initialize the list of parameters:\n",
|
|
" P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
|
|
"\n",
|
|
" P[0] = npr.randn(num_neurons[0], 2 )\n",
|
|
" for l in range(1,N_hidden):\n",
|
|
" P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
|
|
"\n",
|
|
" # For the output layer\n",
|
|
" P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
|
|
"\n",
|
|
" print('Initial cost: %g'%cost_function_deep(P, x))\n",
|
|
"\n",
|
|
" ## Start finding the optimal weights using gradient descent\n",
|
|
"\n",
|
|
" # Find the Python function that represents the gradient of the cost function\n",
|
|
" # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
|
|
" cost_function_deep_grad = grad(cost_function_deep,0)\n",
|
|
"\n",
|
|
" # Let the update be done num_iter times\n",
|
|
" for i in range(num_iter):\n",
|
|
" # Evaluate the gradient at the current weights and biases in P.\n",
|
|
" # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n",
|
|
" # in the hidden layers and output layers evaluated at x.\n",
|
|
" cost_deep_grad = cost_function_deep_grad(P, x)\n",
|
|
"\n",
|
|
" for l in range(N_hidden+1):\n",
|
|
" P[l] = P[l] - lmb * cost_deep_grad[l]\n",
|
|
"\n",
|
|
" print('Final cost: %g'%cost_function_deep(P, x))\n",
|
|
"\n",
|
|
" return P\n",
|
|
"\n",
|
|
"def g_analytic(x, gamma = 2, g0 = 10):\n",
|
|
" return g0*np.exp(-gamma*x)\n",
|
|
"\n",
|
|
"# Solve the given problem\n",
|
|
"if __name__ == '__main__':\n",
|
|
" npr.seed(15)\n",
|
|
"\n",
|
|
" ## Decide the vales of arguments to the function to solve\n",
|
|
" N = 10\n",
|
|
" x = np.linspace(0, 1, N)\n",
|
|
"\n",
|
|
" ## Set up the initial parameters\n",
|
|
" num_hidden_neurons = np.array([10,10])\n",
|
|
" num_iter = 10000\n",
|
|
" lmb = 0.001\n",
|
|
"\n",
|
|
" P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)\n",
|
|
"\n",
|
|
" res = g_trial_deep(x,P)\n",
|
|
" res_analytical = g_analytic(x)\n",
|
|
"\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
"\n",
|
|
" plt.title('Performance of a deep neural network solving an ODE compared to the analytical solution')\n",
|
|
" plt.plot(x, res_analytical)\n",
|
|
" plt.plot(x, res[0,:])\n",
|
|
" plt.legend(['analytical','dnn'])\n",
|
|
" plt.ylabel('g(x)')\n",
|
|
" plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5420c490",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"### Example: Population growth\n",
|
|
"\n",
|
|
"A logistic model of population growth assumes that a population converges toward an equilibrium.\n",
|
|
"The population growth can be modeled by"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1436ed3c",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"log\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation} \\label{log} \\tag{10}\n",
|
|
"\tg'(t) = \\alpha g(t)(A - g(t))\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a590bdb6",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"where $g(t)$ is the population density at time $t$, $\\alpha > 0$ the growth rate and $A > 0$ is the maximum population number in the environment.\n",
|
|
"Also, at $t = 0$ the population has the size $g(0) = g_0$, where $g_0$ is some chosen constant.\n",
|
|
"\n",
|
|
"In this example, similar network as for the exponential decay using Autograd has been used to solve the equation. However, as the implementation might suffer from e.g numerical instability\n",
|
|
"and high execution time (this might be more apparent in the examples solving PDEs),\n",
|
|
"using a library like TensorFlow is recommended.\n",
|
|
"Here, we stay with a more simple approach and implement for comparison, the simple forward Euler method.\n",
|
|
"\n",
|
|
"Here, we will model a population $g(t)$ in an environment having carrying capacity $A$.\n",
|
|
"The population follows the model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "48d788d6",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"solveode_population\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation} \\label{solveode_population} \\tag{11}\n",
|
|
"g'(t) = \\alpha g(t)(A - g(t))\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "169c6c25",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"where $g(0) = g_0$.\n",
|
|
"\n",
|
|
"In this example, we let $\\alpha = 2$, $A = 1$, and $g_0 = 1.2$.\n",
|
|
"\n",
|
|
"We will get a slightly different trial solution, as the boundary conditions are different\n",
|
|
"compared to the case for exponential decay.\n",
|
|
"\n",
|
|
"A possible trial solution satisfying the condition $g(0) = g_0$ could be\n",
|
|
"\n",
|
|
"$$\n",
|
|
"h_1(t) = g_0 + t \\cdot N(t,P)\n",
|
|
"$$\n",
|
|
"\n",
|
|
"with $N(t,P)$ being the output from the neural network with weights and biases for each layer collected in the set $P$.\n",
|
|
"\n",
|
|
"The analytical solution is\n",
|
|
"\n",
|
|
"$$\n",
|
|
"g(t) = \\frac{Ag_0}{g_0 + (A - g_0)\\exp(-\\alpha A t)}\n",
|
|
"$$\n",
|
|
"\n",
|
|
"The network will be the similar as for the exponential decay example, but with some small modifications for our problem."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"id": "6d2e33bf",
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"editable": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Initial cost: 0.221805\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/numpy/core/fromnumeric.py:3208: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\n",
|
|
" return asarray(a).size\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Final cost: 0.000417932\n",
|
|
"The max absolute difference between the solutions is: 0.00424909\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAm4AAAJcCAYAAABAGii1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABxo0lEQVR4nO3dd3gU5frG8e+TRiihBEINvfcqIFIVFBUBUawo2BB7L0c91uPR48/exQY2FAVUFLsCovRepNfQEnoNpLy/P2aREBMIkGR2k/tzXbmSnXd29pmd2dk7U94x5xwiIiIiEvzC/C5ARERERHJGwU1EREQkRCi4iYiIiIQIBTcRERGREKHgJiIiIhIiFNxEREREQoSCW4gwswpmNtHMdpvZc37X4zczK2pmY81sp5l97nc9R2Nmg8xskt91nAwz62pmCXn8GuPN7NocjNfJzJbkZS0S3HK6ruTi6602s+759Xon4mS2M2ZWzcz2mFn4SdaQ68vlZN773JqvYKPglocCK9z+wIqz2czeN7MSJzi5wcAWoKRz7q5cLDNUXQhUAMo65/r7XYwfzKyGmTkzi/C7lvzknPvdOVff7zpywsx6mdk0M9trZlvN7GMzi8/QPsjM0gLbiD1mtiqwnaiXYZxDy3lPpp+L/Zmr4GZmj5rZRyfx/GFm9p/crCnYZA5Dzrm1zrkSzrk0P+s6WQV1vjJTcMt75znnSgCtgFOAh47nyeYJA6oDi9wJ9JhcQL/YqwNLnXOpfhZRQN/bIxSGecwLZnYh8AnwElAOaAwcACaZWZkMo04ObCNKAd2B/cBMM2uSaZKlA19Ch34+y/u5yHtav0SOk3NOP3n0A6wGumd4/H/AN4G/2wN/AjuAuUDXDOONB54E/sDbiH8EpAAHgT14G/ciwIvAhsDPi0CRwPO7AgnAfcAm4EPgUeDzwLR2A/OBesC/gERgHXBmhhquAv4KjLsSuD5D26Hp3xV47kbgqgztRYHngDXATmASUPRY853F+9cw8F7sABYCvQPDHwu8FymB9+OaLJ77KDAS+CAwDwuBNhnaKwOjgCRgFXBrhrZhwH8yz2+m5XofMA/vizgCuB9YEXitRcD5GcYfBEzKZh5rAA4YCKzF26v6YIb2sAzT3hqYp9hA29rAc/cEfk4NvOetA+0DAu2NAo+vBb4M/H2860/m9+DWwHzGZzFPdYAJgWW/BfgsQ1sHYHqgbTrQIdN6f22gth1AkwxtcXifhfLZLI+7A8tjJ/AZEJ2h/V68dXRDYPoOqJPN8jjh9T7TdCywLO7NNDwMWAA8frR1A/gG+CLTOhKRw+1OLPB+YH63H1rmgbbrgOXANuBroHKGNgfcCCwLzP8TQG1gMrALb92LyvRePBBYxquByzNM61xgduB564BHs1jnr8FbhycGhl8deO+3Az8A1TM8pwewOLB8X8Vbv67NYt57cuS2YW6Gz/vXgfleDlyXzXs3mCO3tWNzuI71Aubgrbd/As2OsnxeCrwnu4CZQKfj2G7laDsDvAY8l+l1xwK3432e0/E+T3vwPh+HlknE0dYhoAzeupkUGP4NGbYBBD7D2cx3W2BGYL43A89naOsdmNcdgWk0zPT57n6sbXMO5yvb9eBY730w/fheQEH+ybTCVQ2sCE8AVfC+hM/B25D3CDyOC4w7Hm+D1hgvFERmscI+DkzB+yKLw9tYPBFo6wqkAv/D+xIsGlgpk4GzAtP8AC+wPBiY/nXAqgzTPxdvo21AF2Af0CrT9B8PPPecQHuZQPtrgXmoAoTjfVkXOdZ8Z3rvIgMfrAeAKOD0wIepfqD9UeCjo7z3h+b3nEANTwFTAm1heBvMhwPTroX3JX1WoD3ze92VfwaFOYFleiiQ9sfbKIQBFwN7gUqBtkEcO7i9HVhOzfHCYMNA++2B5RwfeA/fAkZkem5Ehul9ANwV+Hso3kb+hgxtd5zg+vP3ewD8G5iV1XILtI/AW6/CgGigY2B4LN7G/gq8dfDSwOOyGdb7awN/vwc8mWGaNwHfH2V5TAu8/7F4X/5DAm098cJnY6AY3sb9aMHthNf7TNNpEHidmlm0PYa3ly3bdQMvxGzObjkfY7vzLV6wKBOos0tg+Ol4IatVYLm+QiA0Bdod3pdaSQ7vHfwF7/NRCi8oDMz0XjwfmFYXvHW+fob2poF1oBneF3XfTPPzAVAcb/3qi/d5bxhYNx4C/gyMXw7vy/7CwPzcEXjt7ALCo2TaNuAFvdfx1scWeMHjjGyeP4wMn/8crGOt8IJ8O7xtzcDA+EWymf4AoGxgPu/CWz+jj7XdOp7tDF5I2gCEZXgP9wEVMn83ZbWOkf06VBa4AO+zFIO3M+DLDNMZf5TlMhm4IvB3CaB94O96gfnoEXitewPrQlTmWjMvG7LeFhxtvrJdD4713gfTj+8FFOSfwEq0B++/iDWBFaYo3p6MDzON+wOHN4rjCfxHnqE98wq7Ajgnw+OzgNWBv7vi/ceY8T/CR4GfMjw+L1BbeOBxTGAFL53NvHwJ3JZh+vs5MjAk4u1NCwu0Nc9iGked70zDO+Ft0MIyDBtB4D93chbcfs7wuBGwP/B3O2BtpvH/BbyfzXud1cbh6mMs+zlAn8Dfgzh2cMv4X+s04JLA33+R4QsGqIS3RyCCrIPbNcDXGZ57LfBp4PEaDoeQ411/ugLr8b6oJwGljjLvH+CFxvhMw68ApmUaNhkYlGG9PxTcugMrM4z3B3DlUZbHgAyPnwHeDPz9HvBUhrY6HCW4neh6n8XzOgZeJzqLtiHAsqOtG3iBMyXTOrIj00/DLJ5XCW+vQ1Zh8l3gmQyPSwTWpRqBxw44LUP7TOC+DI+fA17M8F6kAsUztI8E/p3N+/gi8EKm+amVof07Muw5x9uO7MM7JeJKjgwvhre3L0fBDe8frDQgJsOwp4Bh2Tx/GFkHt+zWsTcI/NOToX0JgbCTg3VsO4HtJUfZbmXz3Dlks53B+/z3CPx9MzAu0/xkGXCOtg5l8fotgO0ZHo8/ynKZiPdPS7lMw/8NjMy07NcTOBpDLgW3Y60Hx/ve+/mjc9zyXl/nXGnnXHXn3I3Ouf14G6P+Zrbj0A/ehr5ShuetO8Z0K+N9ER+yJjDskCTnXHKm52zO8Pd+YIs7fNLm/sDvEgBmdraZTTGzbYH6zsH7r+2Qre7I88v2BZ5bDu+/mRVZ1JyT+c44f+ucc+mZ5rFKFuNmZ1Om+qID59NUBypnquMBvIsdcuqI5WNmV5rZnAzTa8KR79fx1nroIpbqwJgM0/0Lb+OTXa0TgE5mVhHvv8bPgNPMrAbeXpM5gfFOZP0pjXco6Snn3M6jzMu9eF+u08xsoZldnc1rHnrdrJbpr0BRM2tnZtXxviDGHOU1s3v/KnPksjrq5+ok1vvMtgR+Z7VuV8rQnp0qeIdzMioX2JYc+vkri+dVBbY557Zn0XbE+++c24O3xzvj+595G5H5ccZ53e6c25vh8d/rUGC5/WZmSWa2Ey+sZv48ZFwW1YGXMqzn2/DWoSpkWobO+1Y91vYxo8p478nuTLUez7YEjv4ZvSvT9qQqR36e/mZmd5nZX4Er4nfgfS4zvjfZbbeOdzszHG/vHoHfH+ZsNrNfh8ysmJm9ZWZrzGwXXhgrncOrNq/B27u22Mymm1mvwPDM62U63vI93uVzLDlZD7J974OJgps/1uHtecq4ES7unHs6wzjuGNPYgLfBOKRaYFhOn58tMyuCd/7Xs3i71ksD4/A2pMeyBW93c+0s2nIy34dsAKoGLsw4pBref2Inax3eYeGMdcQ4584JtO/FOxRwSMUspvH3+xsIFm/j/VdbNvB+LSBn71dOaj07U63Rzrn1ZLGMnXPL8TY4t+IdBtuNtzEajPff+KEgfCLrz3a8c3neN7PTsivYObfJOXedc64ycD3wupnVyeI1D73uP5ZpoM6ReIdTL8M7N3R35vFyYCPeYeZDqmY34kmu95ktwdsrdMQVz4H1+QK8Q5BHcz7w+wm87jog1sxKZ9F2xPtvZsXxDn2d6GeqTGAah2Rchz7BO+xa1TlXCniTf76PGdexdXjnE2Zcz4s65/7EW4Z/LzczM46yHPnnursB7z2JyVRrdvN9vNvOdXiH9TPWXsw5NyLziGbWCe/Iw0V4e7RK450zd8x17AS2Mx8BfcysOd4h6C8ztB1tHo+2Dt0F1AfaOedKAp0PlXes+p1zy5xzl+KdnvE/4IvA+pN5vTy0fLNaPsfaNh9tvo53PQhaCm7++Ag4z8zOMrNwM4s2r5+s+GM+87ARwENmFmdm5fDO1zrhS+AzicI7byUJSDWzs4Ezc/LEwBfue8DzZlY5MH+nBr4Uj2e+p+J9SO81s0gz64p3ePfTk54771DkLjO7z7z+4MLNrImZnRJonwOcY2axgT1Xtx9jesXxNhhJAGZ2Fd5/wrnhTeDJwEabwPLuE2hLwjukUSvTcybgbdwnBB6Pz/QYTnD9cc6NBy7H2wvYLqtxzKx/hmW6He+9ScMLQfXM7DIzizCvO4tGeCc4Z+UTvPN4Lg/8fSJGAleZWUMzK4Y3n9k54fU+s8Beobvx3uPLAutZReAdvHPIXsj8nMB6WNPMXsE7BPTYCbzuRrzDjq+bWZnAZ+fQl+sneO9Fi8Dn8b/AVOfc6hOYxUMeM7OoQCDphXfOE3inXmxzziWbWVu88H00bwL/MrPGAGZWyswOhd5vgcZm1i+w9+NWsv5n6pDNQI1D//Q559bhncP5VGCb0wxv78/HR3l+5s/U0bwNDAnsZTQzK25m52YKCIfE4B1iTgIizOxhvPUhJ45rO+OcS8C7AOhDYFTgaM8h2c7jMdahGLw9rzvMLBZ4JIe1Y2YDzCwu8B2xIzA4De8zeq6ZnWFmkXjh8ADeMstsDkffNh9tvo53PQhaCm4+CKxAffAOzyXh/YdzD8e3PP6Dd4XOPLwrRGcFhuVGfbvxNo4j8b54L8P77zmn7g7UNB3vkMf/8M5Vy/F8O+cO4l1pdDbeXrzX8c5xWnxic3XEtNPwQmALvAs0tuB9oZYKjPIh3hWvq4Ef8Q43Hm16i/DO/5mMt+FoindOVm54Ce+9/9HMduNdUNAu8Lr7CFx9HDh00j7wnAl4G9iJ2TyGk1h/nHM/4V19+bWZtc5ilFOAqWa2J1D7bc65Vc65rXhf7nfhHaK7F+jlnMvysKFz7lB4r4z3RXLcnHPfAS8Dv+Gd8Dw50HQgi3FPdr3PPL3P8M7ruwNvHVuEd47raYH34pBTA+/VLryQXRI4xTk3P9Mkd9iR/bjdmc1LX4F37tpivHPwbg/U8wve+USj8PZi1QYuOdH5w9uTux1vT8bHeCfrH/p83gg8HlhnH8Z7T7PlnBuDt5341LxDcAvwPvsE1o/+wNN4601djv75OhQet5rZrMDfl+Kd77QB75D7I4H1OCvvAo0Cn6kvj1Z3oL4ZeBd3vYr3fizHO98sKz/grctL8Q7TJZPDw74nuJ0ZHhgv82HSp/D+qdhhZndn8bws1yG8cxWL4q3PU4Dvc1J7QE9gYWBdfwnvPN5k59wSvEO5rwSmex5eN1oHs5jGsbbNx5qv41kPgpZ5/xiKiBR8ZtYQLxQUcT73ARjKAnvAP3LOHc9RAslngT1lH+FdgJJ+rPElNGiPm4gUaGZ2fuBwXhm8vTpjFdqkoAscdrwNeEehrWBRcBORgu56vEPzK/DOqbnB33JE8lZgz/IOvCuYX/S1GMl1OlQqIiIiEiK0x01EREQkRARdx3J5oVy5cq5GjRp+lyEiIiJyTDNnztzinIvLqq1QBLcaNWowY8YMv8sQEREROSYzy3yXmb/pUKmIiIhIiFBwExEREQkRCm4iIiIiIaJQnOMmIiIiJy8lJYWEhASSk5P9LqVAiI6OJj4+nsjIyBw/R8FNREREciQhIYGYmBhq1KiBmfldTkhzzrF161YSEhKoWbNmjp+nQ6UiIiKSI8nJyZQtW1ahLReYGWXLlj3uvZcKbiIiIpJjCm2550TeSwU3ERERkRCh4CYiIiKFzrBhw7j55puPOc6GDRv+fnzttdeyaNGi436t8ePH06tXr+N+XlYU3ERERESykDm4vfPOOzRq1MjHihTcREREJMT07duX1q1b07hxY4YOHQpAiRIlePDBB2nevDnt27dn8+bNAIwdO5Z27drRsmVLunfv/vfwQ3bv3k3NmjVJSUkBYNeuXdSoUYPPP/+cGTNmcPnll9OiRQv2799P165d/76F5vfff0+rVq1o3rw5Z5xxBgDTpk2jQ4cOtGzZkg4dOrBkyZJcn3d1ByIiIiLH7bGxC1m0YVeuTrNR5ZI8cl7jY4733nvvERsby/79+znllFO44IIL2Lt3L+3bt+fJJ5/k3nvv5e233+ahhx6iY8eOTJkyBTPjnXfe4ZlnnuG55577e1oxMTF07dqVb7/9lr59+/Lpp59ywQUX0L9/f1577TWeffZZ2rRpc8TrJyUlcd111zFx4kRq1qzJtm3bAGjQoAETJ04kIiKCn3/+mQceeIBRo0bl6nuk4CYiIiIh5eWXX2bMmDEArFu3jmXLlhEVFfX3eWStW7fmp59+Ary+5y6++GI2btzIwYMHs+wz7dprr+WZZ56hb9++vP/++7z99ttHff0pU6bQuXPnv6cVGxsLwM6dOxk4cCDLli3DzP7ei5ebFNxERETkuOVkz1heGD9+PD///DOTJ0+mWLFidO3aleTkZCIjI//uXiM8PJzU1FQAbrnlFu6880569+7N+PHjefTRR/8xzdNOO43Vq1czYcIE0tLSaNKkyVFrcM5l2ZXHv//9b7p168aYMWNYvXo1Xbt2Pen5zUznuImIiEjI2LlzJ2XKlKFYsWIsXryYKVOmHHP8KlWqADB8+PBsx7vyyiu59NJLueqqq/4eFhMTw+7du/8x7qmnnsqECRNYtWoVwN+HSjO+1rBhw45rvnJKwU1ERERCRs+ePUlNTaVZs2b8+9//pn379kcd/9FHH6V///506tSJcuXKZTve5Zdfzvbt27n00kv/HjZo0CCGDBny98UJh8TFxTF06FD69etH8+bNufjiiwG49957+de//sVpp51GWlraSc5p1sw5lycTDiZt2rRxh64CERERkRPz119/0bBhQ7/LyBNffPEFX331FR9++GG+vm5W76mZzXTOtclqfJ3jJiIiIoXaLbfcwnfffce4ceP8LuWYFNxERESkUHvllVf8LiHHdI6biIiISIjIs+BmZu+ZWaKZLcim/XIzmxf4+dPMmmdo62lmS8xsuZndn2F4rJn9ZGbLAr/L5FX9IiIiIsEmL/e4DQN6HqV9FdDFOdcMeAIYCmBm4cBrwNlAI+BSMzt0Y7D7gV+cc3WBXwKPg0JaoL8YERERkbySZ8HNOTcR2HaU9j+dc9sDD6cA8YG/2wLLnXMrnXMHgU+BPoG2PsChTliGA31zu+4TMW3MK2x8sjHJ+/b4XYqIiIgUYMFyjts1wHeBv6sA6zK0JQSGAVRwzm0ECPwun90EzWywmc0wsxlJSUl5UPJhxSvUJt5tYt64t/L0dURERKRw8z24mVk3vOB236FBWYx23J3NOeeGOufaOOfaxMXFnUyJx9SofU+WhdehwqL3cOl50+GeiIiIiK/BzcyaAe8AfZxzWwODE4CqGUaLBzYE/t5sZpUCz60EJOZXrUdjYWFsb34d1dMTmD/+C7/LERERKbBWr15Nw4YNue6662jcuDFnnnkm+/fvp2vXrtx33320bduWevXq8fvvv/tdap7wrR83M6sGjAaucM4tzdA0HahrZjWB9cAlwGWBtq+BgcDTgd9f5V/FR9firKvYPOsZwqe+Dqdf7Hc5IiIieeu7+2HT/NydZsWmcPbTxxxt2bJljBgxgrfffpuLLrqIUaNGAZCamsq0adMYN24cjz32GD///HPu1hcE8rI7kBHAZKC+mSWY2TVmNsTMhgRGeRgoC7xuZnPMbAaAcy4VuBn4AfgLGOmcWxh4ztNADzNbBvQIPA4KUUWKsKLWABofmMPqBUe/4a2IiIicuJo1a9KiRQsAWrduzerVqwHo16/fP4YVNHm2x805d+kx2q8Frs2mbRzwj/tOBA6nnpErBeaBRufewt6X32Lrz89To8lIv8sRERHJOznYM5ZXihQp8vff4eHhf98A/tDw8PBwUgtoN12+X5xQkJQuG8ecuPNotv1ntm1c7Xc5IiIiUsAouOWyKj3vJIx0Vnz7gt+liIiISAGjm8znshp1GjG9eEfqJ3xO8t7HiS5eyu+SRERECowaNWqwYMHhu2nefffd/xinXLlyBfYcN+1xywNRnW6jJHtZNO5Nv0sRERGRAkTBLQ80a9+dReENqPjXe7i0gnlypIiIiOQ/Bbc8YGZsb3E9ldM3sXjCZ36XIyIikmucO+6bGUk2TuS9VHDLI63PHEAC5Qmf+prfpYiIiOSK6Ohotm7dqvCWC5xzbN26lejo6ON6ni5OyCPRRaJYUfMKuqx6jnXzJ1K1aWe/SxIRETkp8fHxJCQkkJSU5HcpBUJ0dDTx8fHH9RwFtzzU5Lyb2PXSG2z/+QUFNxERCXmRkZHUrFnT7zIKNR0qzUNlY8syK64PjXf8xs4Ny/0uR0REREKcglseq3b2HaQTxqpvn/e7FBEREQlxCm55rFbt+kwv3pk660dzcM92v8sRERGREKbglg+iOt1KCfazeJyuMBUREZETp+CWD1q378bc8CZUXDwMl5bidzkiIiISohTc8oGZsaPF9ZRPT2LZbx/7XY6IiIiEKAW3fNLurEtZTWUip70O6rhQREREToCCWz6Jjopkea0rqHlwCRvm/+Z3OSIiIhKCFNzyUbNeQ9juYtjxywt+lyIiIiIhSMEtH5WPjWVG3Pk02PE7uxMW+12OiIiIhBgFt3xWredtpBDOmu+e87sUERERCTEKbvmsfp06TC5+OrXXf0XKnq1+lyMiIiIhRMHNB9GdbqEoB1g27mW/SxEREZEQouDmg7btOjE9vAUVFn+ASz3gdzkiIiISIhTcfBAWZuxscT1l07exavwHfpcjIiIiIULBzScdzuzPcqoSpQ55RUREJIcU3HxSrEgky2pdSfzBlWye+6Pf5YiIiEgIUHDzUctzryfJlWLnr+qQV0RERI5Nwc1HFcuWYnrcBdTbNZk9CQv9LkdERESCnIKbz2r0vJVkF0nCuGf9LkVERESCnIKbzxrVqcnvxXtQc8NYUndt9rscERERCWIKbkEguvMtFCGFld+pQ14RERHJnoJbEOjQ9lT+DG9DhcUfQsp+v8sRERGRIKXgFgTCw4xdLa6nlNvJmvHD/C5HREREgpSCW5Do1ON8FlGTqOlvQHq63+WIiIhIEFJwCxLFoyNZXmsglQ6uYcvccX6XIyIiIkFIwS2ItDn3Gja6WHb/pg55RURE5J8U3IJI5bIlmRp3ITV3zWDf2jl+lyMiIiJBRsEtyNTueTN7XRE2fPd/fpciIiIiQUbBLcg0rVOd8cV6Un3j96Tt3OB3OSIiIhJEFNyCULHONxPu0lg97kW/SxEREZEgouAWhDq1bcPE8HZUWPoxHNzrdzkiIiISJBTcglBEeBi7Wl5PCbeH9ePf8bscERERCRIKbkGqW/dezHV1iZr+FqSn+V2OiIiIBAEFtyAVUzSKZbUHEpeynu2zv/K7HBEREQkCCm5BrN05A0lw5dgz/iW/SxEREZEgoOAWxKqWK8nkuIuounsOyaun+V2OiIiI+EzBLcjVOWsIu1xRNn7/nN+liIiIiM8U3IJcizrV+KXYOVTb9CPp29f6XY6IiIj4SMEtyJkZJTrfiHOw7jvdfF5ERKQwU3ALAV3btuLX8NOIW/YpJO/yuxwRERHxiYJbCIgMD2N3y+sp5vaxafzbfpcjIiIiPlFwCxHdu/dkumtI1MyhkJbqdzkiIiLiAwW3EFGqaCTLag0kNmUTO2eN8rscERER8YGCWwjpcM4AVqVXYN+El8A5v8sRERGRfKbgFkJqxMXwZ9zFVNqzkAOr/vS7HBEREclnCm4hpu6Zg9nuSpD4gzrkFRERKWwU3ELMKfXi+bHYOVTZ/CvpW1b6XY6IiIjkIwW3EGNmxHS+kVQXxoYfnve7HBEREclHCm4hqPspzfkxrBNxy0fC/u1+lyMiIiL5RMEtBEVFhLGn1RCKuAMkjX/T73JEREQkn+RZcDOz98ws0cwWZNPewMwmm9kBM7s7w/D6ZjYnw88uM7s90Paoma3P0HZOXtUf7HqecQZ/uKZEzXwHUg/6XY6IiIjkg7zc4zYM6HmU9m3ArcCzGQc655Y451o451oArYF9wJgMo7xwqN05Ny53Sw4dpYtFsazWQEqlbmH3zM/8LkdERETyQZ4FN+fcRLxwll17onNuOpBylMmcAaxwzq3J7foKgk5nX8LS9Crsn/iyOuQVEREpBIL9HLdLgBGZht1sZvMCh2LLZPdEMxtsZjPMbEZSUlLeVumT2uVj+CPuEsrvXcrB5eP9LkdERETyWNAGNzOLAnoDn2cY/AZQG2gBbASy7YXWOTfUOdfGOdcmLi4uL0v1Vf0zryHJlWTLT+oaREREpKAL2uAGnA3Mcs5tPjTAObfZOZfmnEsH3gba+lZdkDi1XmW+K9qLyokTcYmL/S5HRERE8lAwB7dLyXSY1MwqZXh4PpDlFauFiZlRqtMQkl0km358we9yREREJA/lZXcgI4DJQH0zSzCza8xsiJkNCbRXNLME4E7gocA4JQNtxYAewOhMk33GzOab2TygG3BHXtUfSnq2a8K4sK6UXTEa9m7xuxwRERHJIxF5NWHn3KXHaN8ExGfTtg8om8XwK3KnuoKlSEQ4e1tdT9TMn9g6/nXKnvuw3yWJiIhIHgjmQ6VyHM7p1pnf0ltSZPZ7kJLsdzkiIiKSBxTcCoiyJYqwvPZASqRuZ8+MT/wuR0RERPKAglsB0uWsC1iYXp0Dv7+iDnlFREQKIAW3AqRexZL8HncxZfet5ODSH/0uR0RERHKZglsB06j7IDa5Mmz/6UW/SxEREZFcpuBWwHRqUJlvos+jwpY/cZvm+12OiIiI5CIFtwLGzIjtNJh9rghJP6lDXhERkYJEwa0AOqddI74OO53YFV/B7k1+lyMiIiK5RMGtAIqODGdfy+sIc2nsmPC63+WIiIhILlFwK6DO69aRn10boma/Dwf3+l2OiIiI5AIFtwIqLqYIS2sNpFjaLvZN/8jvckRERCQXKLgVYGec2Zs56bU5OOlVSE/3uxwRERE5SQpuBVjDyqX4vdxFlN6/ltTF4/wuR0RERE6SglsB16T7FSS4cuz4RV2DiIiIhDoFtwKuS4PKfB3dm3JbZ+DWz/K7HBERETkJCm4FXFiYUbbTtex2Rdn2s/a6iYiIhDIFt0Kgd9sGjLEzKL3qG9iZ4Hc5IiIicoIU3AqBolHh7G91Lc7BrvGv+l2OiIiInCAFt0Kib9dT+d61I2ruB3Bgt9/liIiIyAlQcCskKpSMZlmtgUSn7yV52jC/yxEREZEToOBWiPTocQ7T0uuT8sfrkJbqdzkiIiJynBTcCpEmVUoxoezFxCRvIG3R136XIyIiIsdJwa2QaX7GJaxOr8Cu3170uxQRERE5TgpuhcwZjSrzZXQfymybC2un+l2OiIiIHAcFt0ImPMyI63gVO1xxtus2WCIiIiFFwa0Q6tuuHl9YD0qt+QG2rfK7HBEREckhBbdCqHiRCPa3vIZUZ+ye+Irf5YiIiEgOKbgVUhd0OYVv0jtQZN4nsH+73+WIiIhIDii4FVKVSxdlaa2BRKXv58C09/0uR0RERHJAwa0Q69m9B5PSGpP65xuQetDvckREROQYFNwKsRZVSzOh7EUUP5BI+sIxfpcjIiIix6DgVsi1Or0/y9KrsPu3F8E5v8sRERGRo1BwK+TObFKZMdG9KbVjEaye5Hc5IiIichQKboVceJhR/rSBbHEl2fWrOuQVEREJZgpuwgXt6jCSMym57hfYsszvckRERCQbCm5CTHQkyS2u4oCLZO+El/0uR0RERLKh4CYA9O/SijHpHYla8Bns3ep3OSIiIpIFBTcBoGpsMZbWvIJId4CDU972uxwRERHJgoKb/O3cM07nt7TmpE19C1KS/S5HREREMlFwk7+1rl6G32IvpujBbaTPG+l3OSIiIpKJgpsc4ZRuffkrvRr7JrysDnlFRESCjIKbHOHsppUYVaQPJXYtgxW/+F2OiIiIZKDgJkeICA+j4mkD2OxKs2f8S36XIyIiIhkouMk/9G9XmxGuJyUSJsLmRX6XIyIiIgEKbvIPpYpGcqDFIPa5IuyfqL1uIiIiwULBTbJ0SZdmjErvTOSiUbB7s9/liIiICApuko3qZYuzpMYAwlwqKVPe8rscERERQcFNjqL36Z35Oa0VadPehYP7/C5HRESk0FNwk2ydUqMMv5S5iOiUHaTPGeF3OSIiIoWegptky8w4tWsv5qbXIvn3VyA93e+SRERECjUFNzmqc5pV5vPIvhTbvQqWfOt3OSIiIoWagpscVVREGPGnXcSy9CqkfHs/HNzrd0kiIiKFloKbHNMl7WvzuF1P5J4EGP+U3+WIiIgUWgpuckyli0XRpft5fJLajfTJr8PGeX6XJCIiUigpuEmODOxQg5FlrmOHK0H617dCeprfJYmIiBQ6Cm6SI5HhYdzTpx2PHryCsI2zYfo7fpckIiJS6Ci4SY6dVqccqY3O53fXnPSfH4Od6/0uSUREpFBRcJPj8mCvxjyafg2pqanw3b1+lyMiIlKoKLjJcalSuih9u3bguYP9YPE38Nc3fpckIiJSaCi4yXG7rnMtfix5ASvCauDG3QPJu/wuSUREpFDIs+BmZu+ZWaKZLcimvYGZTTazA2Z2d6a21WY238zmmNmMDMNjzewnM1sW+F0mr+qX7EVHhvNQ72bctf9q2L0Rfv2P3yWJiIgUCnm5x20Y0PMo7duAW4Fns2nv5pxr4Zxrk2HY/cAvzrm6wC+Bx+KDMxpWoEy9UxnhzsRNGwoJM/0uSUREpMDLs+DmnJuIF86ya090zk0HUo5jsn2A4YG/hwN9T7hAOWkPn9eY/0u9mF0RZWHsbZCW6ndJIiIiBVqwnuPmgB/NbKaZDc4wvIJzbiNA4Hf57CZgZoPNbIaZzUhKSsrjcgunmuWKc1nnxty7bwBsng9TXve7JBERkQItWIPbac65VsDZwE1m1vl4J+CcG+qca+OcaxMXF5f7FQoAN3Wrw7wSnZgc2Q43/inYvsbvkkRERAqsoAxuzrkNgd+JwBigbaBps5lVAgj8TvSnQjmkWFQED/ZqxJ27B5CaDnx7Fzjnd1kiIiIFUtAFNzMrbmYxh/4GzgQOXZn6NTAw8PdA4Kv8r1AyO7dpJWrUqscL6RfB8p9g4Wi/SxIRESmQ8rI7kBHAZKC+mSWY2TVmNsTMhgTaK5pZAnAn8FBgnJJABWCSmc0FpgHfOue+D0z2aaCHmS0DegQei8/MjMf6NObtAz1IKFofvrsf9m/3uywREZECx1whOKzVpk0bN2PGjGOPKCfl8bGLmDb5V8YW+TfWaiCc96LfJYmIiIQcM5uZqTu0vwXdoVIJXbf3qMumYvUZG90HZr4Pa6f4XZKIiEiBouAmuaZkdCT39WzA/dvPY2/RSl7fbqkH/S5LRESkwFBwk1x1Qat46leryAMHBkHSYvjzJb9LEhERKTAU3CRXhYUZj/duwtf7m7KozOkw4f9g6wq/yxIRESkQFNwk1zWNL8Wlbatx9eYLSQuPgm9uV99uIiIiuUDBTfLEPWfWZ3+ROIYVHQirJsLcT/0uSUREJOQpuEmeKFM8irvPqs9/Nrdne2wL+OEB2LvV77JERERCmoKb5JnL2lajYaXS3Lx7IO7ALvjp336XJCIiEtIU3CTPhIcZj/dpzB+7KzC10uUw52PvsKmIiIicEAU3yVNtasTSr2UVrlt9BimlasDY2yEl2e+yREREQpKCm+S5+89ugIuI5qXoG2DbCvj9Ob9LEhERCUkKbpLnypeM5rYz6vLqmqpsrN4bJr0AiYv9LktERCTkKLhJvhh0Wg3qlC/BkKR+uCIlvL7d0tP9LktERCSkKLhJvogMD+PR8xozd1sUv1a9GdZOhtkf+l2WiIhISFFwk3zTsW45zm5SkZv+asSBKqd63YPsSfS7LBERkZCh4Cb56sFzGwLGUxHXQ8p++P5ffpckIiISMhTcJF/FlynGTV3rMGxJFGsb3wALvoBlP/tdloiISEhQcJN8d13nWlSLLcb1qzrjytaDb++Eg/v8LktERCToKbhJvouODOfhXo34K+kA31a/F3asgQlP+12WiIhI0FNwE1+c0bA83erHcf/Mkuxvcin8+Spsmu93WSIiIkFNwU18YWY8fF5jDqam82TKZVC0jHc7rPQ0v0sTEREJWgpu4pua5YpzbaeafDR3N6vaPAjrZ8CM9/wuS0REJGgpuImvbj69DpVKRXPz/Dq4WqfDz4/Brg1+lyUiIhKUFNzEV8WiInjw3IYs3Libr+LvgvQU+O5ev8sSEREJSgpu4rtzm1bi1FpleXTSPvZ1uBv+GguLx/ldloiISNBRcBPfmRmP9m7M7uRUntrRHco3gnF3w4HdfpcmIiISVBTcJCjUrxjDwFNr8NH0Daxo/6R3ntuvT/pdloiISFBRcJOgcXuPupQtHsU9U4rg2lwN096C9bP8LktERCRoKLhJ0CgZHcl9PRswa+0Ovi53HRQvD2Nvg7RUv0sTEREJCgpuElQuaBVPy2qleeLn9ezr/l/YNA+mvul3WSIiIkFBwU2CSliY8XjvJmzde4Dn1jWAej3htydhx1q/SxMREfGdgpsEnabxpbjklGoMm7yGlW0fAwy+vRuc87s0ERERXym4SVC656z6lCgSwUO/7cB1+xcs+wEWfel3WSIiIr5ScJOgFFs8irvPrMefK7byXbG+UKk5fHcf7N/hd2kiIiK+UXCToHVZu+o0qlSSJ75byv6ez8PeJPjlcb/LEhER8Y2CmwSt8DDj8T6N2bgzmdcWx0C7ITDjPVg3ze/SREREfKHgJkGtTY1Yzm9ZhaETV7Km2R1Qskqgb7cUv0sTERHJdwpuEvT+dXYDIsONx35cA+c+C4mL4M+X/S5LREQk3ym4SdArXzKa27vX49fFifyS3goa9oYJz8C2lX6XJiIikq8U3CQkDOxQg9pxxXn8m0Ukd/8vhEXCN3eobzcRESlUFNwkJERFhPFo78as2bqPd+YmQ/dHYOV4mDfS79JERETyjYKbhIxOdeM4u0lFXv1tOevrXApV2sAP/4J92/wuTUREJF8ouElIefDchgD897ulcN5LkLwTfvq3z1WJiIjkDwU3CSnxZYpxY9c6fDt/I3/sqQin3gyzP4LVk/wuTUREJM8puEnIGdy5FtVii/Ho1wtJ6XQPlK4OY2+H1AN+lyYiIpKnFNwk5ERHhvNwr0YsS9zD8OmJ0Ot52LoMfn/e79JERETylIKbhKQzGpana/04Xvx5GYkVOkKTC2HS85C01O/SRERE8oyCm4QkM+OR8xpzMDWd/323BHo+BZFF4ZvbIT3d7/JERETyhIKbhKya5YpzbaeajJqVwMytEdDjCVjzB8z52O/SRERE8oSCm4S0m7rVoWLJaB7+aiFpLQZAtQ7w40OwJ8nv0kRERHKdgpuEtOJFInjw3IYs3LCLT2ckwHkvwsG98MMDfpcmIiKS6xTcJOT1alaJ9rVi+b8flrC9WE3odCfMHwkrfvW7NBERkVyl4CYhz8x4rHcTdien8uyPS6DjnVC2jncT+oP7/C5PREQk1yi4SYFQv2IMV55anU+mrWVB4gHo9QJsXw0Tn/G7NBERkVyj4CYFxu3d61G2eBQPf7WA9OqdoMXl8OcrsHmh36WJiIjkCgU3KTBKFY3k3p4NmLV2B2Nmr4cz/wPRpWDsberbTURECgQFNylQLmwVT4uqpXnqu8XsCouBs/4LCdNh5nt+lyYiInLSFNykQAkLMx7v05itew/w8s/LoNnFULML/PwY7Nrod3kiIiInRcFNCpxm8aW55JRqvP/napYm7vEuVEg7CN/f53dpIiIiJ0XBTQqke86qT4kiETz69UJcbC3ofA8s+gqWfO93aSIiIidMwU0KpNjiUdx9Zj3+XLGV7xZsgg63QlxDGHc3HNjjd3kiIiInJM+Cm5m9Z2aJZrYgm/YGZjbZzA6Y2d0Zhlc1s9/M7C8zW2hmt2Voe9TM1pvZnMDPOXlVv4S+y9pVp1Glkvznm0XsSw/zboe1cx389l+/SxMRETkhebnHbRjQ8yjt24BbgWczDU8F7nLONQTaAzeZWaMM7S8451oEfsblZsFSsISHGY/1acyGncm8/tsKqNYeWl8FU9+ADXP8Lk9EROS45Vlwc85NxAtn2bUnOuemAymZhm90zs0K/L0b+Auokld1SsF2So1Yzm9ZhaETV7J6y17o/igUj/P6dktL9bs8ERGR4xLU57iZWQ2gJTA1w+CbzWxe4FBsmaM8d7CZzTCzGUlJSXldqgSxf53dgMhw44lvFkHR0tDzadg4B6YN9bs0ERGR4xK0wc3MSgCjgNudc7sCg98AagMtgI3Ac9k93zk31DnXxjnXJi4uLq/LlSBWvmQ0t3Wvyy+LE/l18WZofD7UPRN+/Q/sWOd3eSIiIjkWlMHNzCLxQtvHzrnRh4Y75zY759Kcc+nA20Bbv2qU0DKoQ01qxxXnsbGLSE5Nh3OeBRyMuwec87s8ERGRHAm64GZmBrwL/OWcez5TW6UMD88HsrxiVSSzqIgwHu3dmDVb9/HupFVQpjp0/Rcs/Q7++trv8kRERHIkL7sDGQFMBuqbWYKZXWNmQ8xsSKC9opklAHcCDwXGKQmcBlwBnJ5Ftx/PmNl8M5sHdAPuyKv6peDpVDeOno0r8uqvy9mwYz+0vxEqNoVx90LyTr/LExEROSZzheAwUZs2bdyMGTP8LkOCQML2fZzx3AS6N6rAa5e1gvUz4Z3u0OYaODdzzzQiIiL5z8xmOufaZNUWdIdKRfJSfJli3NStDt/O28ify7dAldbQdjBMfwfWTfe7PBERkaNScJNCZ3DnWlSNLcojXy8kJS0dTn8ISlaGr26E5F3HnoCIiIhPFNyk0ImODOfhXo1ZlriHDyavgSIxcP5bsHUFjLke0tP9LlFERCRLCm5SKHVvWJ6u9eN48aelJO5OhpqdoOdTsGQcTPif3+WJiIhkScFNCiUz4+FejUhOTeN/3y3xBrYdDC0uhwlPw19j/S1QREQkCwpuUmjViivBtZ1qMWpWAjPXbAMzOPd574KFMUMg8S+/SxQRETmCgpsUajd3q0PFktE88vVC0tIdREbDxR9BZDH49DLYv93vEkVERP6m4CaFWvEiETx4bkMWrN/FiGlrvYElK3vhbcc6+OIaSE/zt0gREZEABTcp9Ho1q8Sptcry1Li/WJ64xxtYrZ3XIe+KX+CXx/0tUEREJEDBTQo9M+P5i5sTHRnOkI9msudAqtfQehC0uRr+eBHmf+FniSIiIoCCmwgAlUoV5ZXLWrIyaQ/3fTGPv28F1/N/UO1U+Opm2DjP3yJFRKTQU3ATCehQuxz39WzAt/M38u6kVd7AiCi46AMoFgufXg57t/hbpIiIFGoKbiIZDO5ci56NK/LUd4uZunKrN7BEee9ihT2b4fNBkJbia40iIlJ4KbiJZGBm/F//ZlQvW4ybPpnN5l3JXkOVVtD7ZVj9O/z4kL9FiohIoaXgJpJJTHQkbw1ozb6Dqdz48SwOpgbuXdr8Emh/E0x9E2Z/7G+RIiJSKCm4iWShboUYnrmwGTPXbOe/4zLcQaHH41CzC3xzByTM9K9AEREplBTcRLLRq1llrulYk2F/ruarOeu9geER0H8YxFSAzy6H3Zt9rVFERAoXBTeRo7j/7Aa0rRHL/aPms3jTLm9gsVi4ZAQk74SRV0DqAX+LFBGRQkPBTeQoIsPDePXylsRERzDkw5nsSg5cUVqxCfR9HdZNhe/u9bdIEREpNBTcRI6hfEw0r13eioTt+7lr5FzS0wOd8zY+HzreCTOHwfR3fa1RREQKBwU3kRw4pUYsD57bkJ8WbeaNCSsON5z+ENTp4e11W/OnfwWKiEihoOAmkkODOtSgd/PKPPfjEiYtC9xBISwcLngHSleHkVfCzvX+FikiIgWagptIDpkZT1/QlLrlY7j109ms37HfayhaGi4dASnJ3pWmKft9rVNERAouBTeR41AsKoI3BrQiJTWdGz+ayYHUNK8hrj70GwobZnt9vB26Sb2IiEguUnATOU614krw7EXNmZuwk8fGLjrc0OAc6PoAzB0BU97wr0ARESmwFNxETsBZjStyQ9fafDJ1LSNnrDvc0PkeaNDLu5/pyvG+1SciIgWTgpvICbqrRz1Oq1OWh75cwIL1O72BYWFw/ptQrh58Pgi2r/azRBERKWByHNzMrIyZNTazWmamwCeFXkR4GC9f0pKyxaMY8tFMduw76DUUiYFLPgaXDp9eDgf3+luoiIgUGEcNYGZWysweMLP5wBTgLWAksMbMPjezbvlRpEiwKluiCG8MaE3irgPc9umcw53zlq0NF74HiYvgyxt1sYKIiOSKY+05+wJYB3RyztV3znV0zrVxzlUF/gf0MbNr8rxKkSDWomppHundiAlLk3jpl2WHG+p0h+6PwqIvYdILfpUnIiIFSMTRGp1zPY7SNgOYkesViYSgy9pWY/baHbz0yzKaVy3F6Q0qeA0dboWN8+CXx6FCE6h3pr+FiohISMvRuWpm9ktOhokUVmbGf/o2oVGlktz+6RzWbt13qAF6v+LdlH7UtbBlub+FiohISDvWOW7RZhYLlAtcnBAb+KkBVM6XCkVCRHRkOG8OaI2ZMeSjmSSnBDrnjSoGl3wC4RHw6WWQvMvfQkVEJGQda4/b9cBMoEHg96Gfr4DX8rY0kdBTrWwxXrykBX9t2sWDYxbgDl2UULoa9B8OW5fDmOshPd3fQkVEJCQdNbg5515yztUE7nbO1XLO1Qz8NHfOvZpPNYqElG71y3Pr6XUZNSuBT6atPdxQsxP0fAqWjIMJ//OvQBERCVnHOlTaEcA590o27SXNrEleFCYSym47oy5d68fx6NcLmb12++GGtoOhxeUw4Wn4a6x/BYqISEg61qHSC8zsTzN72MzONbO2ZtbZzK42sw+Bb4Ci+VCnSEgJCzNevLgFFUpGc+PHs9i654DXYAbnPg9VWsOYIZD4l7+FiohISDnWodI7gHOBjUB/4HHgDqAO8KZzrrNzbnqeVykSgkoXi+LNAa3Ztvcgt4yYTWpa4Ly2yGi4+COIKu5drLB/+9EnJCIiEnDM7kCcc9uBksA84CdgErAFaGBmLfK0OpEQ16RKKf7Ttwl/rtjKcz8tPdxQsjJc9CHsWAdfXAPpaf4VKSIiISOn9xxtDQwBKuF1AzIY6Aq8bWb35k1pIgVD/zZVuaxdNd4Yv4LvF2w63FCtHZz7LKz4BX55zL8CRUQkZOQ0uJUFWjnn7nbO3QW0AeKAzsCgPKpNpMB45LxGNI8vxd2fz2Vl0p7DDa0HQZtr4I+XYP4XvtUnIiKhIafBrRpwMMPjFKC6c24/cCDXqxIpYIpEhPP6gNZERYQx5KOZ7DuYerix59NQ7VT46mbYONe/IkVEJOjlNLh9Akwxs0fM7BHgD2CEmRUHFuVZdSIFSJXSRXnl0pYsT9zD/aPmH+6cNyIKLvoAisXCp5fD3i3+FioiIkErR8HNOfcEcB2wA9gJDHHOPe6c2+ucuzwP6xMpUE6rU467zqzP13M3MOzP1YcbSpT3rjTdmwSfD4K0FL9KFBGRIJbTPW4452YG7qTwonNuRl4WJVKQ3dClNj0aVeDJb/9i+upthxuqtILzXoLVv8OPD/lXoIiIBK0cBzcRyR1hYcZzFzUnvkxRbvx4Fom7kg83Nr8E2t8EU9+E2R/5V6SIiAQlBTcRH5SMjuTNK1qzJzmVmz+ZTUpahpvO93gcanaBb+6ABO3cFhGRwxTcRHzSoGJJnr6gKdNWb+Pp7xYfbgiPgP7DIKYifDYAdm/2rUYREQkuCm4iPurTogqDOtTg3UmrGDt3w+GGYrFwyQhI3gkjr4BU9bojIiIKbiK+e+CchrSpXob7Rs1j6ebdhxsqNoG+r8O6qfCdblAiIiIKbiK+i4oI47XLW1EsKoIhH85kd3KGrkAanw8d74SZw2D6u77VKCIiwUHBTSQIVCgZzWuXtWTNtn3c8/m8w53zApz+ENQ909vrtuZP/4oUERHfKbiJBIl2tcryr7Mb8P3CTQyduPJwQ1g49HsbytSAkVfCzgTfahQREX8puIkEkWs61uTcppX43/eL+XN5hltfFS0Nl3wCKcnebbFS9vtWo4iI+EfBTSSImBn/u7AZteJKcMuI2WzcmSGgxdWHfkNh4xwYeztkPJwqIiKFgoKbSJApUSSCNwe0JjkljRs+msWB1LTDjQ3OgW4PwrxPYcob/hUpIiK+UHATCUJ1ypfg2f7NmbNuB//55q8jGzvdDQ16efczXTnel/pERMQfCm4iQersppW4vnMtPpyyhlEzM1yQEBYG578J5erB54Ng+2q/ShQRkXym4CYSxO45qz7ta8XywJj5LNyw83BDkRi45GNw6d7FCgf3+lekiIjkGwU3kSAWER7GK5e2okyxKG74aBY792XonLdsbbjwPUhcBF/eqIsVREQKgTwLbmb2npklmtmCbNobmNlkMztgZndnautpZkvMbLmZ3Z9heKyZ/WRmywK/y+RV/SLBIi6mCK9d3oqNO/dzx8g5pKdnCGh1ukP3R2HRlzDpBb9KFBGRfJKXe9yGAT2P0r4NuBV4NuNAMwsHXgPOBhoBl5pZo0Dz/cAvzrm6wC+BxyIFXuvqZXi4VyN+XZzIq78tP7Kxw63Q5EL45XFY+qM/BYqISL7Is+DmnJuIF86ya090zk0HUjI1tQWWO+dWOucOAp8CfQJtfYDhgb+HA31ztWiRIDagfXXOb1mFF35eyoSlSYcbzKD3K95N6UddC1uWZz8REREJacF4jlsVYF2GxwmBYQAVnHMbAQK/y2c3ETMbbGYzzGxGUlJSdqOJhAwz47/nN6V+hRhu+3Q267btO9wYVcy7s0J4BHx6GSTv8q9QERHJM8EY3CyLYcd91rVzbqhzro1zrk1cXFwulCXiv6JR4bw5oDVp6Y4bPp5JckqGznlLV4P+w2HrchhzPaSn+1eoiIjkiWAMbglA1QyP44ENgb83m1klgMDvxHyuTcR3NcoV54WLWrBg/S4e+WrhkY01O0HPp2DJOJjwtD8FiohIngnG4DYdqGtmNc0sCrgE+DrQ9jUwMPD3QOArH+oT8V33RhW45fQ6fDZjHZ9OW3tkY9vB0GIATPgf/DXWnwJFRCRPROTVhM1sBNAVKGdmCcAjQCSAc+5NM6sIzABKAulmdjvQyDm3y8xuBn4AwoH3nHOHdis8DYw0s2uAtUD/vKpfJNjd3r0ec9bt4OGvFtKwUkmaVy3tNZjBuc9B0l8wZgiUrQPlG/paq4iI5A5zhaDTzjZt2rgZM2b4XYZIrtu+9yC9XpkEwNhbOhJbPOpw464NMLQrRBaDwb9BUXV7KCISCsxspnOuTVZtwXioVERyqEzxKN4Y0IqkPQe47dPZpGXsnLdkZbjoQ9iZAF9cDelp2U9IRERCgoKbSIhrFl+aJ/o05vdlW3jhp6VHNlZr5x02XfEr/PKYPwWKiEiuybNz3EQk/1x8SjVmrdnBq78tp3nV0vRoVOFwY+uBsHEu/PESxFSG9kP8K1RERE6K9riJFBCP9WlM0yqluHPkHFZv2Xtk49n/gwa94Pv7YPo7/hQoIiInTcFNpICIjgzn9ctbER5mDPloJvsPZjinLTwSLnwf6p0N394FM4f5VqeIiJw4BTeRAqRqbDFeuqQlSzbv5oEx8zniqvGIKLhoONTpAWNvh9kf+1aniIicGAU3kQKmS7047uxejzGz1/PhlDVHNkYUgYs/glpd4aubYO5nvtQoIiInRsFNpAC6qVsdzmhQnsfHLmLmmm1HNkZGezekr9ERvhwCC0b5U6SIiBw3BTeRAigszHj+4hZUKVOUGz+eRdLuA0eOEFUMLvsMqp0Ko66DRbp7nIhIKFBwEymgShWN5I3LW7Nzfwo3fzKL1LT0I0eIKu6Ft/g2Xge9i7/1p1AREckxBTeRAqxR5ZI81a8pU1dt45kflvxzhCIxcPkXUKkFjBwIS3/I9xpFRCTnFNxECrjzW8Zz5anVGTpxJe/8vvKfI0SXhAGjoGIT+GwALP85/4sUEZEcUXATKQT+3asRZzepyH++/YuhE1f8c4SipWHAaIirD59eDivH53eJIiKSAwpuIoVAZHgYL1/akl7NKvHfcYt5ffzyf45ULBau+Apia8Mnl8Cq3/O/UBEROSoFN5FCIjI8jBcvbkGfFpV55vslvPzLsn+OVLwsXPkVlKkOn1wMaybnf6EiIpItBTeRQiQiPIznL2pBv1ZVeP6npTz/09Ij764AUCIOrvwaSlaGjy+EddP8KVZERP5BwU2kkAkPM/7vwuZc1Cael39ZxrM/LvlneIupAAPHQony8NEFkDDTn2JFROQICm4ihVB4mPF0v2Zc2rYqr/22gqe/X/zP8FaykhfeipaBj86HDXN8qVVERA5TcBMppMLCjCf7NmVA+2q8NWElT3771z/DW6l4GPQNFCkFH/SBTfP9KVZERAAFN5FCLSzMeKJPEwZ1qME7k1bx2NhF/wxvpavBwK+9Oy180Ac2L/KnWBERUXATKezMjEfOa8Q1HWsy7M/V/PurBaSnZwpvsTW9w6bhUfBBb0jK4i4MIiKS5xTcRAQz46FzG3J9l1p8NGUtD36ZRXgrW9sLbxYGw8+DLVl0JyIiInlKwU1EAC+83d+zATd1q82IaWu5f/Q80jKHt3J1va5C0tO88LY1i7swiIhInlFwE5G/mRl3n1mf286oy8gZCdzzxdx/hrfyDbw9b6kHYHhv2L7al1pFRAojBTcROYKZcUePetzZox6jZ63nzpFzSE1LP3KkCo28Oywc3APDzoMda/0pVkSkkFFwE5Es3XpGXe45qz5fzdnAbZ/NISVzeKvUDK78EpJ3eodNd673pU4RkcJEwU1EsnVTtzo8cE4Dvp23kVtHzP5neKvcEq4YA/u2wfBesGujP4WKiBQSCm4iclSDO9fm370a8d2CTdz08SwOpmYKb/GtYcAo2JPo7XnbvdmfQkVECgEFNxE5pms61uSx3o35cdFmbvhoJgdS044coWpbuPxz2LXe6+dtT5I/hYqIFHAKbiKSIwM71OA/fZvwy+JErv9wJskpmcJb9Q5w2UjYvsa7w8Lerf4UKiJSgCm4iUiODWhfnaf7NWXC0iSu+2DGP8NbzU5w6QjYtgI+7OOd+yYiIrlGwU1EjsslbavxzAXNmLR8C1cPm87+g5nCW+1ucMnH3m2xPjwf9u/wpU4RkYJIwU1Ejlv/NlV5/qLmTFm5lUHvT2PvgdQjR6jTHS7+CDYvhI8ugORd/hQqIlLAKLiJyAk5v2U8L1zcgumrtzHo/WnsyRze6p0FFw2HjXPg4wvhwG5f6hQRKUgU3ETkhPVpUYVXLm3FrLU7uPLdqexOTjlyhAbnwoXvQcIM+PgiOLjXn0JFRAoIBTcROSnnNqvEa5e1ZF7CTga8O42d+zOFt0Z94IK3Yd0U+ORiOLjPn0JFRAoABTcROWk9m1Ti9ctbsWjDTga8M5Ud+w4eOUKTC6Dvm7B6Enx6GaQk+1OoiEiIU3ATkVxxZuOKvDmgNUs27ebyd6ayfW+m8Nb8YujzGqwcD59dDqkHfKlTRCSUKbiJSK45o2EFhl7ZmmWJe7j07Sls3ZMpnLW8HM57CZb/DCOvhNSDWU9IRESypOAmIrmqa/3yvDuwDau27OXSt6eQtDtTeGs9EM59DpZ+D19cBWkpWU9IRET+QcFNRHJdp7pxvD/oFNZt28+lb08hcXemc9pOuRbOfgYWfwOjroW01KwnJCIiR1BwE5E80aFOOd6/6hQ27NjPJUOnsHlXpvDW7no480lY9CV8OQTS07KcjoiIHKbgJiJ5pn2tsgy/ui2bdyZz8VuT2bhz/5EjdLgZuj8K8z+Hr25SeBMROQYFNxHJU6fUiOWDa9qxdc9BLn5rCgnbM/Xj1vEO6PYQzB0BY2+F9HR/ChURCQEKbiKS51pXL8OH17Zj+z4vvK3blim8dbkHutwHsz+Cb+8E5/wpVEQkyCm4iUi+aFG1NJ9c2549B1K5ZOgU1mzNdPurrv+CjnfCzPdh3D0KbyIiWVBwE5F80zS+FJ9c1459B1O5+K0prNqSIbyZwRkPQ4dbYPrb8MMDCm8iIpkouIlIvmpcuRSfXNeeg2npXPzWZFYk7TncaAY9noB2N8CU1+GnhxXeREQyUHATkXzXsFJJRlzXnnTnuPitKSzbvPtwoxn0fApOuQ7+fBl+fULhTUQkQMFNRHxRv2IMnw5ujxlcMnQKSzZlCm9nPwOtB8Hvz8H4p32rU0QkmCi4iYhv6pT3wltEuHHp21NYtGHX4cawMDj3BWg5ACY8DRP/z79CRUSChIKbiPiqdlwJPht8KkUiwrjsnSksWL/zcGNYGJz3MjS7BH79D0x60bc6RUSCgYKbiPiuRrnifDb4VIpHRXDZ21OYl7DjcGNYOPR9HZpcAD8/ApNf861OERG/KbiJSFCoVrYYnw5uT6likVz+zlRmr91+uDEsHM4fCo36eN2ETH3Lv0JFRHyk4CYiQaNqbDE+HXwqscWjuOLdacxcs+1wY3gEXPAuNOgF390L09/1r1AREZ8ouIlIUKlSuiifDm5PXEwRrnx3GtNWZQxvkXDh+1Cvp3drrJnD/StURMQHCm4iEnQqlfLCW8VS0Qx8bxqTV2w93BgRBRd9AHW6w9jbYM4n/hUqIpLPFNxEJChVKBnNiMHtiS9TlKuGTeOP5VsON0YUgYs/glpd4MsbYd5I/woVEclHCm4iErTKx3jhrXpsca4eNp2JS5MON0YWhUtGQI2OMOZ6WDDKv0JFRPKJgpuIBLVyJYowYnB7asWV4NoPZvDb4sTDjVHF4LLPoGp7GHUdzPvcv0JFRPJBngU3M3vPzBLNbEE27WZmL5vZcjObZ2atAsPrm9mcDD+7zOz2QNujZrY+Q9s5eVW/iASP2OJRjLiuHfUqlOD6D2fy86LNhxujisPlI6Faexh9LYz/n+5tKiIFVl7ucRsG9DxK+9lA3cDPYOANAOfcEudcC+dcC6A1sA8Yk+F5Lxxqd86Ny4vCRST4lC4WxcfXtKdhpRhu+HgmPyzcdLixSAxcMQaaXwrj/wujB0NKsn/FiojkkTwLbs65icC2o4zSB/jAeaYApc2sUqZxzgBWOOfW5FWdIhI6ShWL5MNr29GkSilu+ngW4+ZvPNwYUQT6vgGn/xvmj4QPesPeLdlPTEQkBPl5jlsVYF2GxwmBYRldAozINOzmwKHV98ysTHYTN7PBZjbDzGYkJSVlN5qIhJiS0ZF8cHVbmlctzS0jZjN27obDjWbQ+W7oPww2zoW3T4fExb7VKiKS2/wMbpbFsL9PTDGzKKA3kPFs4zeA2kALYCPwXHYTd84Ndc61cc61iYuLy5WCRSQ4xERHMvzqtrSuVobbPp3NV3PWHzlC4/Nh0DhI2Q/v9oAVv/pTqIhILvMzuCUAVTM8jgcy/OvM2cAs59zfZyE75zY759Kcc+nA20DbfKlURIJOiSIRDLv6FNrVLMsdn81h1MyEI0eIbw3X/QqlqsJHF+oWWSJSIPgZ3L4GrgxcXdoe2Omcy3DCCpeS6TBppnPgzgeyvGJVRAqHYlERvDfoFDrULsfdX8zl3UmrcBmvKC1dFa75wbvLwrd3wvf/gvQ0/woWETlJedkdyAhgMlDfzBLM7BozG2JmQwKjjANWAsvx9p7dmOG5xYAewOhMk33GzOab2TygG3BHXtUvIqGhaFQ47wxsQ/eGFXjim0Xc/MlsdienHB6hSAxcOgLa3QBTXodPL4MDu/0rWETkJJgrBP0dtWnTxs2YMcPvMkQkD6WnO4b+vpL/+2EJ1WKL8frlrWhYqeSRI01/B8bdC+Ubeh33lor3p1gRkaMws5nOuTZZtenOCSJSIISFGUO61OaTa9ux90AqfV/7g5HT1x050inXep317ljrXXG6fqY/xYqInCAFNxEpUNrVKsu3t3aidfUy3DtqHnd/Ppf9BzOc11anO1zzE0REw/vnwsIvfatVROR4KbiJSIETF1OED69px62n12HUrATOf/0PViTtOTxC+QbeFaeVmsHnA+H353SbLBEJCQpuIlIghYcZd55Zn/cHncLmXcn0fmUS38zL0ONQ8XJw5dfQtD/88jh8eSOkHvSvYBGRHFBwE5ECrWv98nx7ayfqV4zh5k9m88hXCziQGjh0GhkN/d6Grg/A3E/gw76w72h36hMR8ZeCm4gUeJVLF+Wz60/lmo41GT55DRe9OZmE7fu8RjPoeh9c8C4kzIB3zoAty/wtWEQkGwpuIlIoRIaH8e9ejXhzQCtWJu3l3Jcn8evizYdHaHohDPoGknd54W3lBP+KFRHJhoKbiBQqPZtUYuwtHalSuihXD5vB/75fTGpautdYta130UJMZfioH8wc7m+xIiKZKLiJSKFTo1xxRt/YgUvbVuWN8Su4/J2pJO5K9hrLVPduk1WzC4y9FX58SLfJEpGgoeAmIoVSdGQ4T/VrxvMXNWdewk7OeXkSf67YEmgsBZeNhFOugz9fgc+ugIN7/S1YRAQFNxEp5Pq1iuerm0+jVNEIBrwzlVd/XUZ6uoPwCDj3WTj7GVj6HbzXE3ZtOPYERUTykIKbiBR69SrE8PXNHenVrDLP/riUq4dPZ/veQJ9u7a6HSz+DbSu922RtmONrrSJSuCm4iYgAxYtE8NIlLXiibxP+XL6Vc1/+nVlrt3uN9c6Ea36EsAh4/2z46xt/ixWRQkvBTUQkwMy4on11vrjhVMLCjIvfmsx7k1bhnIMKjeHaX6B8Q/hsAPzxkm6TJSL5TsFNRCSTZvGl+faWTnSpV57Hv1nETZ/MYndyCsRUgEHfQuO+8NPD8PUtuk2WiOQrBTcRkSyUKhbJ21e25l9nN+CHhZs575VJLNqwCyKLwgXvQed7YPaHXn9v+7f7Xa6IFBIKbiIi2TAzru9SmxHXtWd/Shrnv/4HI6evg7AwOP0hOP8tWDcV3ukOW1f4Xa6IFAIKbiIix9C2Zizf3tqJNjXKcO+oedz9+Vz2H0yD5pfAlV95N6Z/5wxY/YffpYpIAafgJiKSA+VKFOGDq9tx6xl1GTUrgb6v/cGKpD1QvQNc9wsUj4MP+sCcT/wuVUQKMAU3EZEcCg8z7uxRj+FXtSVpzwF6vzKJsXM3QGwtuOYnL8R9eQP8/Bikp/tdrogUQApuIiLHqXO9OL69tSMNKpXklhGzefirBRyIjIEBo6D1IJj0PHwxCA7u87tUESlgFNxERE5ApVJF+XRwe67tWJMPJq/hojcns25nCvR6Ec58EhZ9DcPOhd2b/C5VRAoQBTcRkRMUGR7GQ70a8eaA1qxM2kuvVybxy+JE6HAzXPIJJC2Bt8+ATfP9LlVECggFNxGRk9SzSUW+ubUj8WWKcs3wGTz93WJS6/aEq78Hl+7doH7J936XKSIFgIKbiEguqF62OKNu6MClbavx5oQVXPbOVBKL14PrfoWydWDEJTD5Nd0mS0ROioKbiEguiY4M56l+TXnh4ubMT9jJOS//zp+JkXDVOGjYC354AL65A9JS/C5VREKUgpuISC47v2U8X918GqWKRjLg3am88vsG0i8cDqfdDjPfh4/7w/4dfpcpIiFIwU1EJA/UqxDD1zd35LzmlXnup6VcNXwm2zo8CH1eg9W/w7tnwrZVfpcpIiFGwU1EJI8ULxLBixe34D99mzB5xVZ6vfw7s8qeC1d8CXs2e7fJWjvF7zJFJIQouImI5CEzY0D76oy6oQPh4cZFb07mvfXxuGt/hujSMPw8mDfS7zJFJEQouImI5IOm8aX45uZOdGtQnse/WcSN3+9i1xXfQ3xbGH0d/PqkrjgVkWNScBMRySelikUy9IrWPHBOA35ctJne7yxkUffh0GIATHwGvrgaUvb7XaaIBDEFNxGRfGRmDO5cm08Ht2d/ShrnvzWdzyrfizvjUVg42jt0uifR7zJFJEgpuImI+OCUGrF8e2snTqkRy32jF3D3xtM50G8YbFrg3SZr8yK/SxSRIKTgJiLik3IlijD86rbcdkZdRs9O4LxfYlnXdxSkHfS6C1n2s98likiQUXATEfFReJhxR496DL+qLVv2HKTnyN382HEExNaAT/rD1KF+lygiQUTBTUQkCHSuF8e3t3akQaWSDP5yI0/EPU9anTPhu3tg3D2Qlup3iSISBBTcRESCRKVSRfl0cHuu61STd6cnccG2G9nVcghMG+rdpD55l98liojPFNxERIJIZHgYD57biLeuaM2Krcl0nH06i1o/ASt/8+60sHGe3yWKiI8U3EREgtBZjSvyzS0dqRpbjHP+qM3H9V7CJe+Ct0+HP16C9DS/SxQRHyi4iYgEqeplizPqhg5c1q4aD84pw8DoF9lVrTv89DB80Ad2rPO7RBHJZwpuIiJBLDoynP+e35SXLmnBgu0RtFgygDHVHsCtnwVvnAbzv/C7RBHJRwpuIiIhoE+LKvx2V1euaF+Du5Y1oXfaM2wpVhNGXQOjroX9O/wuUUTygYKbiEiIKFUsksf6NOGbWzoRXb4W7TbexUfFrsAtGO3tfVv1u98likgeU3ATEQkxjSqXZOT1p/Lsxa14KaUv5x94hC3J4Iaf553/lnrA7xJFJI8ouImIhCAz4/yW8fx6VxdOOa0H3fY8wRd0hz9ewr1zBiQu9rtEEckDCm4iIiEsJjqSB89txOjbevBl/N1cc/Audm5eS/pbnWHqW+Cc3yWKSC5ScBMRKQDqVojho2vaccGl13F55Av8drARfHcvB4afD7s3+V2eiOQSBTcRkQLCzDinaSU+v7sPs057g4fTriF91R8kv9yO1IVf+12eiOQCBTcRkQKmWFQE9/RsyFW3PcFjVd5k6YEyRHx+BZs/vBYO7Pa7PBE5CQpuIiIFVM1yxXnqun4k9R/L8IgLKbf8C5KebUvSInUbIhKqFNxERAowM+OMplW5+L6hjG7+NgcOphD72XlMf/9uDhxI9rs8ETlOCm4iIoVAdGQ4/fv1hyGTmBbTnVPWvM2K/3Vi6oxpfpcmIsdBwU1EpBCJr1SRU+/+gkWnvUR8+gaajO3F8FcfY93WvX6XJiI5oOAmIlIINeoxiOhbp7A9tjkDtzzPkpd68+a4qSSnpPldmogchYKbiEghFRVblfhbfmBXl8foGj6XC6b256H/e54fFm7CqeNekaCk4CYiUpiFhVGy2+1EDJlAsTIVefbgf9g04mYGv/s7K5P2+F2diGSi4CYiIlChMcVvmkha+5sYGPET/1o3hDteHM7/vl/M3gOpflcnIgEKbiIi4omMJrznf+HKr6gek87oyH/D7y/Q49lfGTt3gw6figQBBTcRETlSra6E3/gn4Y3O477ITxnqHuV/n/7IpW9PYckm3XlBxE95FtzM7D0zSzSzBdm0m5m9bGbLzWyembXK0LbazOab2Rwzm5FheKyZ/WRmywK/y+RV/SIihVqxWOg/DM5/i8Zha/m1+IPU2vgt57w8kcfHLmJXcorfFYoUSnm5x20Y0PMo7WcDdQM/g4E3MrV3c861cM61yTDsfuAX51xd4JfAYxERyQtm0PwS7IY/iKrclP+6VxhT/l1G/Tmf05+dwKiZCaSn6/CpSH7Ks+DmnJsIbDvKKH2AD5xnClDazCodY7J9gOGBv4cDfU+6UBEROboy1WHQt3DGwzTbNYHpsY9wTokl3PX5XPq/NZkF63f6XaFIoeHnOW5VgHUZHicEhgE44Eczm2lmgzOMU8E5txEg8Lt8dhM3s8FmNsPMZiQlJeVy6SIihUxYOHS6C679majoEjy+4wF+aPQDG7bs4LxXJ/HQl/PZse+g31WKFHh+BjfLYtihfe6nOeda4R1OvcnMOh/vxJ1zQ51zbZxzbeLi4k6mThEROaRyS7h+IpxyLfVXDmdS7H+4p2UaI6ato9uz4/lk6lrSdPhUJM/4GdwSgKoZHscDGwCcc4d+JwJjgLaBcTYfOpwa+J2Yb9WKiIgnqhic+xxcNpLwfYncuORa/uiymHrli/PAmPn0fe0PZq3d7neVIgWSn8Hta+DKwNWl7YGdzrmNZlbczGIAzKw4cCawIMNzBgb+Hgh8ld9Fi4hIQL2z4IbJUPt0Kk5+jE+LPcNbfSqRuDuZfq//yb1fzGXLngN+VylSoFhedahoZiOArkA5YDPwCBAJ4Jx708wMeBXvytN9wFXOuRlmVgtvLxtABPCJc+7JwDTLAiOBasBaoL9z7mgXQADQpk0bN2PGjGONJiIiJ8I5mDkMfngAwqPYf/YLvLihIe9NWkV0ZDh39ajHgPbViQhX16EiOWFmMzP1qnG4rTD0hK3gJiKSD7Ysh9HXwYZZ0OJyVrT5N4/+sJbfl22hQcUYHuvdmHa1yvpdpUjQO1pw078/IiKSO8rVgWt+hM73wtwR1P7iLD7ons6bA1qzOzmVi4dO4bZPZ7N5V7LflYqELAU3ERHJPeGRcPqDcNX3ANiwc+iZ+A4/39aBW0+vw3cLNnH6s+N5a8IKDqam+1ysSOhRcBMRkdxXrR0MmQTNL4OJ/0fRD3tyZ6twfrqjM6fWLstT3y2m50sT+X2Z+tkUOR4KbiIikjeiS0Lf16D/cNi+Gt7qRPVVn/HOlW14b1Ab0tIdV7w7jRs+mknC9n1+VysSEnRxgoiI5L1dG+DLG2Hlb1CvJ/R+leQisbw7aRWv/LoMgBu71uGajjUpXiTC52JF/KWrShXcRET8l54O096Cnx7x9sb1fhXq92T9jv08+e0ixs3fRKmikQw8tToDO9SgbIkiflcs4gsFNwU3EZHgsXmR123I5gXQ5mo48z8QVZxZa7fz5vgV/LhoM9GRYVzUpirXdapF1dhiflcskq8U3BTcRESCS+oB+PUJ+PNVKFsb+r0NVVoBsDxxD0MnrmDM7PWkO+jVrBJDutSmYaWSPhctkj8U3BTcRESC08oJ8OUNsGczdLkfOt7udSkCbNy5n/cmreKTqWvZezCNrvXjGNKlNu1qxuLdfEekYFJwU3ATEQle+7fDN3fCwtFQrh6c+STU7QGBcLZzXwofTV3De5NWsXXvQVpULc2QLrU5s1EFwsIU4KTgUXBTcBMRCW7OwdLv4ceHYOtyqH0GnPUklG/49yjJKWl8PjOBoRNXsG7bfmrHFef6zrXp27IKURHq3UoKDgU3BTcRkdCQehCmvwMTnoYDe6DNVdD1ASh++B6nqWnpjFuwiTfHr2DRxl1ULBnNNR1rcmm7apRQVyJSACi4KbiJiISWvVth/FMw4z2IKgFd74NTroOIqL9Hcc4xcdkW3hy/gskrt1IyOoIrT63BoNNqUE5diUgIU3BTcBMRCU2Ji+GHB2DFLxBb2zt8Wq/n3+e/HTJn3Q7eHL+CHxZtIio8jP5t4hncqTbVyqorEQk9Cm4KbiIioW3ZT16A27IUanaBs/4LFZv8Y7QVSXt4e+JKRs9aT2p6Ouc2q8yQLrVoXLmUD0WLnBgFNwU3EZHQl5YCM96H8f+F5J3Q6kro9hCUiPvHqJt3JfPepFV8PHUtew6k0rleHEO61OLUWmXVlYgEPQU3BTcRkYJj3zaY8AxMfxsii0Hnu6HdEIj453ltO/en8NGUNbz/x2q27DlA8/hS3NC1Nj0aVSRcXYlIkFJwU3ATESl4tizzug9Z+j2UqeHdOqtBr3+c/wZeVyKjZiUwdOJK1mzdR61yxRncuRbnt6pCkYjw/K9d5CgU3BTcREQKruW/wA8PQtJfUL0j9PwvVGqe5ahp6Y7vFmzkzQkrWLB+F+VjinBNx5pc1q4aMdGR+Vy4SNYU3BTcREQKtrRUmDUcfnvSO5TacgCc/m+IqZDl6M45/li+lTcmLOeP5VuJiY7givbVueq0msTFqCsR8ZeCm4KbiEjhsH8HTPw/mPqWd85bpzuh/U0QGZ3tU+Yl7OCtCSsZt2AjkeFh9G8dz+DOtahetnj+1S2SgYKbgpuISOGydQX89DAs/gZKV4Mej0Ojvlme/3bIqi17GTpxJaNmJpCans7ZTStxQ5faNKmirkQkfym4KbiJiBROKyd4/b9tXgDVTvX6f6vS6qhPSdyVzHt/rObjKWvYfSCVTnXLMaRLbTrUVlcikj8U3BTcREQKr/Q0mP0h/Pof2JsEzS+DMx6GkpWO+rRdySl8MnUt705aRdLuAzSLL8WQLrU5q7G6EpG8peCm4CYiIsm74PfnYMrrEBYBHe+AU2+GqKPfFis5JY0xs9czdOJKVm3ZS81yxbmuUy36tapCdKS6EpHcp+Cm4CYiIodsW+Wd//bX11AyHno8Bk0uOOr5b+B1JfLDwk28OWEF8xJ2EhdThKtPq8nl7atRUl2JSC5ScFNwExGRzFZPgu//BZvmQXxb6PkUxGf5XXkE5xyTV2zljQkr+H3ZFmKKRHB5++pcfVoNypfM/upVkZxScFNwExGRrKSnwdwR8MvjsGczNL0Iuj8CpeJz9PQF63fy5oQVjJu/kYiwMC4IdCVSs5y6EpETp+Cm4CYiIkdzYDdMegH+fBUsDE67DU67FaJyFsDWbPW6Evl8ZgIpaemc3aQiQ7rUpll86bytWwokBTcFNxERyYnta+DnR2HhaIip7O19a3oRhIXl6OlJuw/w/h+r+HDKGnYnp3JanbIM6VKbjnXKqSsRyTEFNwU3ERE5HmunwPf3w4bZULkV9HwaqrXL8dN3J6cwYtpa3vl9FYm7D9CkSkmGdKnN2U0qqSsROSYFNwU3ERE5XunpMO8z+OUx2L3Ru/K0+6PenRhy6EBqGl/OXs9bE1aycsteKpQsQt+WVbigVTz1KsTkXe0S0hTcFNxEROREHdwLf7wEf7wMOK/vt453QJESOZ5EWrrj57828/mMBMYvSSQ13dGkSkn6tYynd4vKlCuhG9vLYQpuCm4iInKydibAz4/B/JFQogKc8Qg0vzTH578dsnXPAcbO3cDo2euZl7CT8DCja704+rWK54yG5dWpryi4KbiJiEiuWTfdO/9t/Qyo1BzOegpqnHZCk1q2eTejZ69nzKz1bNqVTEx0BL2aVeaCVlVoXb2MLmgopBTcFNxERCQ3OQfzv/CuQN2VAI36QI/HoUyNE5pcWrpjysqtjJqVwPcLNrHvYBrVYovRr1UV+rWMp1rZo9+WSwoWBTcFNxERyQsH98HkV70+4NJTof2N0OkuiC55wpPceyCVHxZuYvSs9fyxYgvOwSk1ytCvVTznNK1EqaK6vVZBp+Cm4CYiInlp1wbv7gtzR0DxODj939ByAISd3PlqG3fu58vZGxg1K4HliXuIigijR6MKXNCqCp3qxhEZfnzn10loUHBTcBMRkfywfiZ8/wCsmwIVmkLP/0LNzic9WeccC9bvYtSsBL6eu4Ftew9SrkQUvZtXoV+rKjSuXFLnwxUgCm4KbiIikl+cg4Vj4KdHYOdaaNDLO/+tbO1cmXxKWjoTliQxenYCPy9K5GBaOvUrxNCvVRX6tqxCBd3oPuQpuCm4iYhIfkvZD1Neh9+fh9QD0O567/y3YrG59hI796XwzfwNjJ61nplrthNmcFqdclzQKp4zG1egWFRErr2W5B8FNwU3ERHxy+5N8OsTMPtjiCgCjc+H1ldB1baQi4c3V23Zy5hZCYyevZ6E7fspHhXO2U0r0a9VFdrXLEuYbrUVMhTcFNxERMRvmxfCtLdh/udwcA+UbwStB0Gzi6Fo6Vx7mfR0x/TV2xgzez3fztvI7gOpVC4VzfmtqnB+y3jqlM/5HR/EHwpuCm4iIhIsDuyBBV/AzGHeTewjinp74dpcBfGn5OpeuOSUNH5atJnRsxKYuGwLaemO5lVLc0GrKvRqVpnY4lG59lqSexTcFNxERCQYbZgDM9/3OvM9uAfKNw7shbsoV/fCASTuTubrOd75cIs27iIy3OhWvzz9WsXTrUEcRSJ0q61goeCm4CYiIsHswG4vvM18HzbO9fbCNennnQsX3yZX98IB/LVxF2Nmr2fM7PUk7T5A6WKRnNesMv1aVaFF1dLqWsRnCm4KbiIiEio2zIYZgb1wKXuhQpPDe+GiS+XqS6WmpTNp+RZGz1rPDws3cSA1nVrliv/dtUh8Gd1qyw8KbgpuIiISag7s9i5kmPE+bJoHkcUO74Wr0jrX98LtTk7hu/mbGDUrgamrtgHQvlYs/VrFc3aTisRE61Zb+UXBTcFNRERClXPeXriZ78P8UYG9cE2hzSBo2j/X98IBrNu2jy9nr2f07PWs2rKX6MgwzmpckX6t4jmtdlkidKutPKXgpuAmIiIFQfIuby/czPdh0/zAXrgLvCtSK7fK9b1wzjlmr9vB6FkJjJ27kZ37UygfU4S+Lb1bbTWoWDJXX088Cm4KbiIiUpA4BxtmeYdRF4yClH1Qsal3GLVpf4jO/UB1IDWN3xYnMmrWen5bnEhquqNRpZL0a1WF3i0qUz5Gt9rKLQpuCm4iIlJQJe8MnAs3DDbPh8ji0PSCwLlwrfLkJbftPcjYuRsYPSuBuQk7CQ8zOtctR9+WVehSL47SxdQ/3MlQcFNwExGRgs45WD8LZr4HC0Z7e+EqNfeuSG3aH4rE5MnLLk/czehZXtciG3cmE2bQqloZujUoT7f65WlYKUbdixwnBTcFNxERKUySd8K8kd7dGTYvCOyFuzBwLlzLPHnJ9HTHnIQdjF+cyG9Lkpi/ficAFUtG061BHF3rl+e0OuUoUUQ3vj8WBTcFNxERKYycg/UzD58Ll7ofKrUI7IW7MM/2wgEk7kpm/NIkxi9J5PelW9h9IJXIcKNtzVi61S9PtwblqVWuuPbGZUHBTcFNREQKu/07DvcLl7gQokp44a31VVC5RZ6+dEpaOjNWb2f8kkR+W5LI0s17AKgWW4zTG5Sna/042tcqS3SkbrsFCm4KbiIiIoc4BwkzvC5FFoz29sJVbunthWtyIRQpkeclrNu2z9sbtziRP1ZsITklnejIMDrULhc4Ny6uUN+1QcFNwU1EROSf9u8InAv3PiQugqgYaNbfC3GVmudLCckpaUxZuZXxS5L4dXEia7ftA6Bu+RJ/X+DQpkYZIgtRp78KbgpuIiIi2XMOEqZ7h1EXjobUZK9D3zZXQeN++bIXzivDsXLLXn5bnMj4JUlMXbWVlDRHTJEIOtb19sZ1rRdH+ZIFu884BTcFNxERkZzZv93bCzfjfUj6K7AX7iIvxFVsmq+l7DmQyh/Lt3jnxi1OYtOuZACaVCnJ6fXL07VBeZrHlyY8rGBd4OBLcDOz94BeQKJzrkkW7Qa8BJwD7AMGOedmmVlV4AOgIpAODHXOvRR4zqPAdUBSYDIPOOfGHasWBTcREZHj5Bysm+p1KbJwjLcXrkpr72KGJv0gqng+l+P4a+NufluSyG+LE5m1djvpDmKLR9GlXhxd68cVmM5//QpunYE9wAfZBLdzgFvwgls74CXnXDszqwRUCoS4GGAm0Nc5tygQ3PY45549nloU3ERERE7Cvm2Hz4VLWgxFSnp74VpfBRX/8RWfL3bsO8jEZVsYvziR8UuT2Lb3IGEGLauV+ftK1UaVSoZkdyO+HSo1sxrAN9kEt7eA8c65EYHHS4CuzrmNmcb7CnjVOfeTgpuIiIiPnIO1Uw7vhUs7APGneBczNO4HUf5cCZqW7piXsIPfliTx2+LEvzv/rVCyCN3ql6dr/fJ0rBs6nf8Ga3D7BnjaOTcp8PgX4D7n3IxMz58INHHO7QoEt0HALmAGcJdzbns2rz0YGAxQrVq11mvWrMm9GRMRESns9m2DuZ96IW7LEihSChqeBxUaQ1w9KFcfSlaBsPy/GjRxdzITliTxW6bOf0+pERvYG1ee2nHB2/lvsAa3b4GnMgW3e51zMwOPSwATgCedc6MDwyoAWwAHPIF3SPXqY9WhPW4iIiJ5xDlYO9m7mGH5T97FDYdEFoNydb0QF1cPygUCXWwtiMifc9FS0tKZuWb73+fGHer8t2ps0b/v4HBqkHX+G6zBLdtDpWYWCXwD/OCce/54p52ZgpuIiEg+cA72bvH2wG1ZCklLA38vg53rDo9n4V54K1fv8N65cvW8kBddMk9LTNi+j/FLvFtx/bF8K/tT0igSEUaH2mX/3htXNdbfzn+DNbidC9zM4YsTXnbOtQ1cbToc2Oacuz3TcyodOgfOzO4A2jnnLjlWHQpuIiIiPjuwB7YuC4S5QKBLWgrbVkB66uHxYipn2DtXD+Lqe8GuRHnI5UObySlpTF21jd8We7fiWrPV6/y3TvkSf1/g0KZ6LFER+Xu416+rSkcAXYFywGbgESASwDn3ZiCgvQr0xOsO5Crn3Awz6wj8DszH6w4EAt1+mNmHQAu8Q6WrgeszX8yQFQU3ERGRIJWWAttXQ9KSw2FuS+Dn4J7D40WXOnyoNWOwK1MDwk7+MKdzjlVb9v59gcOhzn9LFImgU91ygYsc8qfzX3XAq+AmIiISWpyDXRsOh7ikJYf/3rP58HjhRaBsHe8wa1z9w3vpytaByKIn/PJ7A53/Hgpyhzr/feHi5pzfMv5k5+6oFNwU3ERERAqO/du98+YyhrmkJbBjDbhDB+sMSlc7HOb+PuxaD4rFHtfLOedYvMnr/Ld388rEl8nbc+COFtxCo0MTERERkUOKloGqbb2fjFKSvXPmjgh0S2HVRO/OD4cUK3fk3rlDV76Wis/yPDozo2GlkjSslLcXTuSEgpuIiIgUDJHRXj9yFRofOTw9DXas9fbSbVlyONgtHAPJOzI8v3ggxGW42jWuPpSpmW/dlxyLgpuIiIgUbGHhEFvT+6l35uHh2XVfsuZPmD8yw/MjvPAWVx863ALV2uf/PAQouImIiEjhZAYl4ryfGh2PbMuu+5KUff7UGqDgJiIiIpJZkRJQuaX3E0Ty/wZiIiIiInJCFNxEREREQoSCm4iIiEiIUHATERERCREKbiIiIiIhQsFNREREJEQouImIiIiECAU3ERERkRCh4CYiIiISIhTcREREREKEgpuIiIhIiFBwExEREQkRCm4iIiIiIULBTURERCREKLiJiIiIhAgFNxEREZEQoeAmIiIiEiIU3ERERERChIKbiIiISIhQcBMREREJEQpuIiIiIiFCwU1EREQkRCi4iYiIiIQIc875XUOeM7MkYE0ev0w5YEsev4YcPy2X4KNlEpy0XIKPlklwyo/lUt05F5dVQ6EIbvnBzGY459r4XYccScsl+GiZBCctl+CjZRKc/F4uOlQqIiIiEiIU3ERERERChIJb7hnqdwGSJS2X4KNlEpy0XIKPlklw8nW56Bw3ERERkRChPW4iIiIiIULBTURERCREKLgdJzPraWZLzGy5md2fRbuZ2cuB9nlm1sqPOguTHCyTywPLYp6Z/Wlmzf2os7A51nLJMN4pZpZmZhfmZ32FUU6WiZl1NbM5ZrbQzCbkd42FUQ62YaXMbKyZzQ0sl6v8qLMwMbP3zCzRzBZk0+7bd72C23Ews3DgNeBsoBFwqZk1yjTa2UDdwM9g4I18LbKQyeEyWQV0cc41A55AJ/zmuRwul0Pj/Q/4IX8rLHxyskzMrDTwOtDbOdcY6J/fdRY2Ofys3AQscs41B7oCz5lZVL4WWvgMA3oepd2373oFt+PTFljunFvpnDsIfAr0yTROH+AD55kClDazSvldaCFyzGXinPvTObc98HAKEJ/PNRZGOfmsANwCjAIS87O4Qiony+QyYLRzbi2Ac07LJe/lZLk4IMbMDCgBbANS87fMwsU5NxHvfc6Ob9/1Cm7HpwqwLsPjhMCw4x1Hcs/xvt/XAN/laUUCOVguZlYFOB94Mx/rKsxy8lmpB5Qxs/FmNtPMrsy36gqvnCyXV4GGwAZgPnCbcy49f8qTbPj2XR+RHy9SgFgWwzL3p5KTcST35Pj9NrNueMGtY55WJJCz5fIicJ9zLs3bkSB5LCfLJAJoDZwBFAUmm9kU59zSvC6uEMvJcjkLmAOcDtQGfjKz351zu/K4Nsmeb9/1Cm7HJwGomuFxPN5/QMc7juSeHL3fZtYMeAc42zm3NZ9qK8xyslzaAJ8GQls54BwzS3XOfZkvFRY+Od1+bXHO7QX2mtlEoDmg4JZ3crJcrgKedl7Hq8vNbBXQAJiWPyVKFnz7rteh0uMzHahrZjUDJ4ZeAnydaZyvgSsDV5y0B3Y65zbmd6GFyDGXiZlVA0YDV2jPQb455nJxztV0ztVwztUAvgBuVGjLUznZfn0FdDKzCDMrBrQD/srnOgubnCyXtXh7QTGzCkB9YGW+VimZ+fZdrz1ux8E5l2pmN+NdARcOvOecW2hmQwLtbwLjgHOA5cA+vP+UJI/kcJk8DJQFXg/s3Ul1zrXxq+bCIIfLRfJRTpaJc+4vM/semAekA+8457LsDkFyRw4/K08Aw8xsPt4huvucc1t8K7oQMLMReFfwljOzBOARIBL8/67XLa9EREREQoQOlYqIiIiECAU3ERERkRCh4CYiIiISIhTcREREREKEgpuIiIhIiFBwExE5TmZW2sxu9LsOESl8FNxERI5faUDBTUTynYKbiMjxexqobWZzzOz//C5GRAoPdcArInKczKwG8I1zronftYhI4aI9biIiIiIhQsFNREREJEQouImIHL/dQIzfRYhI4aPgJiJynJxzW4E/zGyBLk4QkfykixNEREREQoT2uImIiIiECAU3ERERkRCh4CYiIiISIhTcREREREKEgpuIiIhIiFBwExEREQkRCm4iIiIiIeL/Aelm2l1vRCaZAAAAAElFTkSuQmCC\n",
|
|
"text/plain": [
|
|
"<Figure size 720x720 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"filenames": {
|
|
"image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter11_58_3.png"
|
|
},
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"import autograd.numpy as np\n",
|
|
"from autograd import grad, elementwise_grad\n",
|
|
"import autograd.numpy.random as npr\n",
|
|
"from matplotlib import pyplot as plt\n",
|
|
"\n",
|
|
"def sigmoid(z):\n",
|
|
" return 1/(1 + np.exp(-z))\n",
|
|
"\n",
|
|
"# Function to get the parameters.\n",
|
|
"# Done such that one can easily change the paramaters after one's liking.\n",
|
|
"def get_parameters():\n",
|
|
" alpha = 2\n",
|
|
" A = 1\n",
|
|
" g0 = 1.2\n",
|
|
" return alpha, A, g0\n",
|
|
"\n",
|
|
"def deep_neural_network(P, x):\n",
|
|
" # N_hidden is the number of hidden layers\n",
|
|
" N_hidden = np.size(P) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
|
|
"\n",
|
|
" # Assumes input x being an one-dimensional array\n",
|
|
" num_values = np.size(x)\n",
|
|
" x = x.reshape(-1, num_values)\n",
|
|
"\n",
|
|
" # Assume that the input layer does nothing to the input x\n",
|
|
" x_input = x\n",
|
|
"\n",
|
|
" # Due to multiple hidden layers, define a variable referencing to the\n",
|
|
" # output of the previous layer:\n",
|
|
" x_prev = x_input\n",
|
|
"\n",
|
|
" ## Hidden layers:\n",
|
|
"\n",
|
|
" for l in range(N_hidden):\n",
|
|
" # From the list of parameters P; find the correct weigths and bias for this layer\n",
|
|
" w_hidden = P[l]\n",
|
|
"\n",
|
|
" # Add a row of ones to include bias\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n",
|
|
"\n",
|
|
" z_hidden = np.matmul(w_hidden, x_prev)\n",
|
|
" x_hidden = sigmoid(z_hidden)\n",
|
|
"\n",
|
|
" # Update x_prev such that next layer can use the output from this layer\n",
|
|
" x_prev = x_hidden\n",
|
|
"\n",
|
|
" ## Output layer:\n",
|
|
"\n",
|
|
" # Get the weights and bias for this layer\n",
|
|
" w_output = P[-1]\n",
|
|
"\n",
|
|
" # Include bias:\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n",
|
|
"\n",
|
|
" z_output = np.matmul(w_output, x_prev)\n",
|
|
" x_output = z_output\n",
|
|
"\n",
|
|
" return x_output\n",
|
|
"\n",
|
|
"\n",
|
|
"def cost_function_deep(P, x):\n",
|
|
"\n",
|
|
" # Evaluate the trial function with the current parameters P\n",
|
|
" g_t = g_trial_deep(x,P)\n",
|
|
"\n",
|
|
" # Find the derivative w.r.t x of the trial function\n",
|
|
" d_g_t = elementwise_grad(g_trial_deep,0)(x,P)\n",
|
|
"\n",
|
|
" # The right side of the ODE\n",
|
|
" func = f(x, g_t)\n",
|
|
"\n",
|
|
" err_sqr = (d_g_t - func)**2\n",
|
|
" cost_sum = np.sum(err_sqr)\n",
|
|
"\n",
|
|
" return cost_sum / np.size(err_sqr)\n",
|
|
"\n",
|
|
"# The right side of the ODE:\n",
|
|
"def f(x, g_trial):\n",
|
|
" alpha,A, g0 = get_parameters()\n",
|
|
" return alpha*g_trial*(A - g_trial)\n",
|
|
"\n",
|
|
"# The trial solution using the deep neural network:\n",
|
|
"def g_trial_deep(x, params):\n",
|
|
" alpha,A, g0 = get_parameters()\n",
|
|
" return g0 + x*deep_neural_network(params,x)\n",
|
|
"\n",
|
|
"# The analytical solution:\n",
|
|
"def g_analytic(t):\n",
|
|
" alpha,A, g0 = get_parameters()\n",
|
|
" return A*g0/(g0 + (A - g0)*np.exp(-alpha*A*t))\n",
|
|
"\n",
|
|
"def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n",
|
|
" # num_hidden_neurons is now a list of number of neurons within each hidden layer\n",
|
|
"\n",
|
|
" # Find the number of hidden layers:\n",
|
|
" N_hidden = np.size(num_neurons)\n",
|
|
"\n",
|
|
" ## Set up initial weigths and biases\n",
|
|
"\n",
|
|
" # Initialize the list of parameters:\n",
|
|
" P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
|
|
"\n",
|
|
" P[0] = npr.randn(num_neurons[0], 2 )\n",
|
|
" for l in range(1,N_hidden):\n",
|
|
" P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
|
|
"\n",
|
|
" # For the output layer\n",
|
|
" P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
|
|
"\n",
|
|
" print('Initial cost: %g'%cost_function_deep(P, x))\n",
|
|
"\n",
|
|
" ## Start finding the optimal weigths using gradient descent\n",
|
|
"\n",
|
|
" # Find the Python function that represents the gradient of the cost function\n",
|
|
" # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
|
|
" cost_function_deep_grad = grad(cost_function_deep,0)\n",
|
|
"\n",
|
|
" # Let the update be done num_iter times\n",
|
|
" for i in range(num_iter):\n",
|
|
" # Evaluate the gradient at the current weights and biases in P.\n",
|
|
" # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n",
|
|
" # in the hidden layers and output layers evaluated at x.\n",
|
|
" cost_deep_grad = cost_function_deep_grad(P, x)\n",
|
|
"\n",
|
|
" for l in range(N_hidden+1):\n",
|
|
" P[l] = P[l] - lmb * cost_deep_grad[l]\n",
|
|
"\n",
|
|
" print('Final cost: %g'%cost_function_deep(P, x))\n",
|
|
"\n",
|
|
" return P\n",
|
|
"\n",
|
|
"if __name__ == '__main__':\n",
|
|
" npr.seed(4155)\n",
|
|
"\n",
|
|
" ## Decide the vales of arguments to the function to solve\n",
|
|
" Nt = 10\n",
|
|
" T = 1\n",
|
|
" t = np.linspace(0,T, Nt)\n",
|
|
"\n",
|
|
" ## Set up the initial parameters\n",
|
|
" num_hidden_neurons = [100, 50, 25]\n",
|
|
" num_iter = 1000\n",
|
|
" lmb = 1e-3\n",
|
|
"\n",
|
|
" P = solve_ode_deep_neural_network(t, num_hidden_neurons, num_iter, lmb)\n",
|
|
"\n",
|
|
" g_dnn_ag = g_trial_deep(t,P)\n",
|
|
" g_analytical = g_analytic(t)\n",
|
|
"\n",
|
|
" # Find the maximum absolute difference between the solutons:\n",
|
|
" diff_ag = np.max(np.abs(g_dnn_ag - g_analytical))\n",
|
|
" print(\"The max absolute difference between the solutions is: %g\"%diff_ag)\n",
|
|
"\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
"\n",
|
|
" plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
|
|
" plt.plot(t, g_analytical)\n",
|
|
" plt.plot(t, g_dnn_ag[0,:])\n",
|
|
" plt.legend(['analytical','nn'])\n",
|
|
" plt.xlabel('t')\n",
|
|
" plt.ylabel('g(t)')\n",
|
|
"\n",
|
|
" plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e31ec549",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"## Using forward Euler to solve the ODE\n",
|
|
"\n",
|
|
"A straightforward way of solving an ODE numerically, is to use Euler's method.\n",
|
|
"\n",
|
|
"Euler's method uses Taylor series to approximate the value at a function $f$ at a step $\\Delta x$ from $x$:\n",
|
|
"\n",
|
|
"$$\n",
|
|
"f(x + \\Delta x) \\approx f(x) + \\Delta x f'(x)\n",
|
|
"$$\n",
|
|
"\n",
|
|
"In our case, using Euler's method to approximate the value of $g$ at a step $\\Delta t$ from $t$ yields"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2e9ee105",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{aligned}\n",
|
|
" g(t + \\Delta t) &\\approx g(t) + \\Delta t g'(t) \\\\\n",
|
|
" &= g(t) + \\Delta t \\big(\\alpha g(t)(A - g(t))\\big)\n",
|
|
"\\end{aligned}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e876da16",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"along with the condition that $g(0) = g_0$.\n",
|
|
"\n",
|
|
"Let $t_i = i \\cdot \\Delta t$ where $\\Delta t = \\frac{T}{N_t-1}$ where $T$ is the final time our solver must solve for and $N_t$ the number of values for $t \\in [0, T]$ for $i = 0, \\dots, N_t-1$.\n",
|
|
"\n",
|
|
"For $i \\geq 1$, we have that"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b79a3def",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{aligned}\n",
|
|
"t_i &= i\\Delta t \\\\\n",
|
|
"&= (i - 1)\\Delta t + \\Delta t \\\\\n",
|
|
"&= t_{i-1} + \\Delta t\n",
|
|
"\\end{aligned}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9d99cb4e",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"Now, if $g_i = g(t_i)$ then"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "3b9dcc24",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"odenum\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation}\n",
|
|
" \\begin{aligned}\n",
|
|
" g_i &= g(t_i) \\\\\n",
|
|
" &= g(t_{i-1} + \\Delta t) \\\\\n",
|
|
" &\\approx g(t_{i-1}) + \\Delta t \\big(\\alpha g(t_{i-1})(A - g(t_{i-1}))\\big) \\\\\n",
|
|
" &= g_{i-1} + \\Delta t \\big(\\alpha g_{i-1}(A - g_{i-1})\\big)\n",
|
|
" \\end{aligned}\n",
|
|
"\\end{equation} \\label{odenum} \\tag{12}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "65ce688e",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"for $i \\geq 1$ and $g_0 = g(t_0) = g(0) = g_0$.\n",
|
|
"\n",
|
|
"Equation ([12](#odenum)) could be implemented in the following way,\n",
|
|
"extending the program that uses the network using Autograd:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"id": "d5497948",
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"editable": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Initial cost: 0.221805\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/numpy/core/fromnumeric.py:3208: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\n",
|
|
" return asarray(a).size\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Final cost: 0.000417932\n",
|
|
"The max absolute difference between the solutions is: 0.00424909\n",
|
|
"Max absolute difference between Euler method and analytical: 0.011225\n",
|
|
"Max absolute difference between deep neural network and analytical: 0.00424909\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAm4AAAJcCAYAAABAGii1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABxo0lEQVR4nO3dd3gU5frG8e+TRiihBEINvfcqIFIVFBUBUawo2BB7L0c91uPR48/exQY2FAVUFLsCovRepNfQEnoNpLy/P2aREBMIkGR2k/tzXbmSnXd29pmd2dk7U94x5xwiIiIiEvzC/C5ARERERHJGwU1EREQkRCi4iYiIiIQIBTcRERGREKHgJiIiIhIiFNxEREREQoSCW4gwswpmNtHMdpvZc37X4zczK2pmY81sp5l97nc9R2Nmg8xskt91nAwz62pmCXn8GuPN7NocjNfJzJbkZS0S3HK6ruTi6602s+759Xon4mS2M2ZWzcz2mFn4SdaQ68vlZN773JqvYKPglocCK9z+wIqz2czeN7MSJzi5wcAWoKRz7q5cLDNUXQhUAMo65/r7XYwfzKyGmTkzi/C7lvzknPvdOVff7zpywsx6mdk0M9trZlvN7GMzi8/QPsjM0gLbiD1mtiqwnaiXYZxDy3lPpp+L/Zmr4GZmj5rZRyfx/GFm9p/crCnYZA5Dzrm1zrkSzrk0P+s6WQV1vjJTcMt75znnSgCtgFOAh47nyeYJA6oDi9wJ9JhcQL/YqwNLnXOpfhZRQN/bIxSGecwLZnYh8AnwElAOaAwcACaZWZkMo04ObCNKAd2B/cBMM2uSaZKlA19Ch34+y/u5yHtav0SOk3NOP3n0A6wGumd4/H/AN4G/2wN/AjuAuUDXDOONB54E/sDbiH8EpAAHgT14G/ciwIvAhsDPi0CRwPO7AgnAfcAm4EPgUeDzwLR2A/OBesC/gERgHXBmhhquAv4KjLsSuD5D26Hp3xV47kbgqgztRYHngDXATmASUPRY853F+9cw8F7sABYCvQPDHwu8FymB9+OaLJ77KDAS+CAwDwuBNhnaKwOjgCRgFXBrhrZhwH8yz2+m5XofMA/vizgCuB9YEXitRcD5GcYfBEzKZh5rAA4YCKzF26v6YIb2sAzT3hqYp9hA29rAc/cEfk4NvOetA+0DAu2NAo+vBb4M/H2860/m9+DWwHzGZzFPdYAJgWW/BfgsQ1sHYHqgbTrQIdN6f22gth1AkwxtcXifhfLZLI+7A8tjJ/AZEJ2h/V68dXRDYPoOqJPN8jjh9T7TdCywLO7NNDwMWAA8frR1A/gG+CLTOhKRw+1OLPB+YH63H1rmgbbrgOXANuBroHKGNgfcCCwLzP8TQG1gMrALb92LyvRePBBYxquByzNM61xgduB564BHs1jnr8FbhycGhl8deO+3Az8A1TM8pwewOLB8X8Vbv67NYt57cuS2YW6Gz/vXgfleDlyXzXs3mCO3tWNzuI71Aubgrbd/As2OsnxeCrwnu4CZQKfj2G7laDsDvAY8l+l1xwK3432e0/E+T3vwPh+HlknE0dYhoAzeupkUGP4NGbYBBD7D2cx3W2BGYL43A89naOsdmNcdgWk0zPT57n6sbXMO5yvb9eBY730w/fheQEH+ybTCVQ2sCE8AVfC+hM/B25D3CDyOC4w7Hm+D1hgvFERmscI+DkzB+yKLw9tYPBFo6wqkAv/D+xIsGlgpk4GzAtP8AC+wPBiY/nXAqgzTPxdvo21AF2Af0CrT9B8PPPecQHuZQPtrgXmoAoTjfVkXOdZ8Z3rvIgMfrAeAKOD0wIepfqD9UeCjo7z3h+b3nEANTwFTAm1heBvMhwPTroX3JX1WoD3ze92VfwaFOYFleiiQ9sfbKIQBFwN7gUqBtkEcO7i9HVhOzfHCYMNA++2B5RwfeA/fAkZkem5Ehul9ANwV+Hso3kb+hgxtd5zg+vP3ewD8G5iV1XILtI/AW6/CgGigY2B4LN7G/gq8dfDSwOOyGdb7awN/vwc8mWGaNwHfH2V5TAu8/7F4X/5DAm098cJnY6AY3sb9aMHthNf7TNNpEHidmlm0PYa3ly3bdQMvxGzObjkfY7vzLV6wKBOos0tg+Ol4IatVYLm+QiA0Bdod3pdaSQ7vHfwF7/NRCi8oDMz0XjwfmFYXvHW+fob2poF1oBneF3XfTPPzAVAcb/3qi/d5bxhYNx4C/gyMXw7vy/7CwPzcEXjt7ALCo2TaNuAFvdfx1scWeMHjjGyeP4wMn/8crGOt8IJ8O7xtzcDA+EWymf4AoGxgPu/CWz+jj7XdOp7tDF5I2gCEZXgP9wEVMn83ZbWOkf06VBa4AO+zFIO3M+DLDNMZf5TlMhm4IvB3CaB94O96gfnoEXitewPrQlTmWjMvG7LeFhxtvrJdD4713gfTj+8FFOSfwEq0B++/iDWBFaYo3p6MDzON+wOHN4rjCfxHnqE98wq7Ajgnw+OzgNWBv7vi/ceY8T/CR4GfMjw+L1BbeOBxTGAFL53NvHwJ3JZh+vs5MjAk4u1NCwu0Nc9iGked70zDO+Ft0MIyDBtB4D93chbcfs7wuBGwP/B3O2BtpvH/BbyfzXud1cbh6mMs+zlAn8Dfgzh2cMv4X+s04JLA33+R4QsGqIS3RyCCrIPbNcDXGZ57LfBp4PEaDoeQ411/ugLr8b6oJwGljjLvH+CFxvhMw68ApmUaNhkYlGG9PxTcugMrM4z3B3DlUZbHgAyPnwHeDPz9HvBUhrY6HCW4neh6n8XzOgZeJzqLtiHAsqOtG3iBMyXTOrIj00/DLJ5XCW+vQ1Zh8l3gmQyPSwTWpRqBxw44LUP7TOC+DI+fA17M8F6kAsUztI8E/p3N+/gi8EKm+amVof07Muw5x9uO7MM7JeJKjgwvhre3L0fBDe8frDQgJsOwp4Bh2Tx/GFkHt+zWsTcI/NOToX0JgbCTg3VsO4HtJUfZbmXz3Dlks53B+/z3CPx9MzAu0/xkGXCOtg5l8fotgO0ZHo8/ynKZiPdPS7lMw/8NjMy07NcTOBpDLgW3Y60Hx/ve+/mjc9zyXl/nXGnnXHXn3I3Ouf14G6P+Zrbj0A/ehr5ShuetO8Z0K+N9ER+yJjDskCTnXHKm52zO8Pd+YIs7fNLm/sDvEgBmdraZTTGzbYH6zsH7r+2Qre7I88v2BZ5bDu+/mRVZ1JyT+c44f+ucc+mZ5rFKFuNmZ1Om+qID59NUBypnquMBvIsdcuqI5WNmV5rZnAzTa8KR79fx1nroIpbqwJgM0/0Lb+OTXa0TgE5mVhHvv8bPgNPMrAbeXpM5gfFOZP0pjXco6Snn3M6jzMu9eF+u08xsoZldnc1rHnrdrJbpr0BRM2tnZtXxviDGHOU1s3v/KnPksjrq5+ok1vvMtgR+Z7VuV8rQnp0qeIdzMioX2JYc+vkri+dVBbY557Zn0XbE+++c24O3xzvj+595G5H5ccZ53e6c25vh8d/rUGC5/WZmSWa2Ey+sZv48ZFwW1YGXMqzn2/DWoSpkWobO+1Y91vYxo8p478nuTLUez7YEjv4ZvSvT9qQqR36e/mZmd5nZX4Er4nfgfS4zvjfZbbeOdzszHG/vHoHfH+ZsNrNfh8ysmJm9ZWZrzGwXXhgrncOrNq/B27u22Mymm1mvwPDM62U63vI93uVzLDlZD7J974OJgps/1uHtecq4ES7unHs6wzjuGNPYgLfBOKRaYFhOn58tMyuCd/7Xs3i71ksD4/A2pMeyBW93c+0s2nIy34dsAKoGLsw4pBref2Inax3eYeGMdcQ4584JtO/FOxRwSMUspvH3+xsIFm/j/VdbNvB+LSBn71dOaj07U63Rzrn1ZLGMnXPL8TY4t+IdBtuNtzEajPff+KEgfCLrz3a8c3neN7PTsivYObfJOXedc64ycD3wupnVyeI1D73uP5ZpoM6ReIdTL8M7N3R35vFyYCPeYeZDqmY34kmu95ktwdsrdMQVz4H1+QK8Q5BHcz7w+wm87jog1sxKZ9F2xPtvZsXxDn2d6GeqTGAah2Rchz7BO+xa1TlXCniTf76PGdexdXjnE2Zcz4s65/7EW4Z/LzczM46yHPnnursB7z2JyVRrdvN9vNvOdXiH9TPWXsw5NyLziGbWCe/Iw0V4e7RK450zd8x17AS2Mx8BfcysOd4h6C8ztB1tHo+2Dt0F1AfaOedKAp0PlXes+p1zy5xzl+KdnvE/4IvA+pN5vTy0fLNaPsfaNh9tvo53PQhaCm7++Ag4z8zOMrNwM4s2r5+s+GM+87ARwENmFmdm5fDO1zrhS+AzicI7byUJSDWzs4Ezc/LEwBfue8DzZlY5MH+nBr4Uj2e+p+J9SO81s0gz64p3ePfTk54771DkLjO7z7z+4MLNrImZnRJonwOcY2axgT1Xtx9jesXxNhhJAGZ2Fd5/wrnhTeDJwEabwPLuE2hLwjukUSvTcybgbdwnBB6Pz/QYTnD9cc6NBy7H2wvYLqtxzKx/hmW6He+9ScMLQfXM7DIzizCvO4tGeCc4Z+UTvPN4Lg/8fSJGAleZWUMzK4Y3n9k54fU+s8Beobvx3uPLAutZReAdvHPIXsj8nMB6WNPMXsE7BPTYCbzuRrzDjq+bWZnAZ+fQl+sneO9Fi8Dn8b/AVOfc6hOYxUMeM7OoQCDphXfOE3inXmxzziWbWVu88H00bwL/MrPGAGZWyswOhd5vgcZm1i+w9+NWsv5n6pDNQI1D//Q559bhncP5VGCb0wxv78/HR3l+5s/U0bwNDAnsZTQzK25m52YKCIfE4B1iTgIizOxhvPUhJ45rO+OcS8C7AOhDYFTgaM8h2c7jMdahGLw9rzvMLBZ4JIe1Y2YDzCwu8B2xIzA4De8zeq6ZnWFmkXjh8ADeMstsDkffNh9tvo53PQhaCm4+CKxAffAOzyXh/YdzD8e3PP6Dd4XOPLwrRGcFhuVGfbvxNo4j8b54L8P77zmn7g7UNB3vkMf/8M5Vy/F8O+cO4l1pdDbeXrzX8c5xWnxic3XEtNPwQmALvAs0tuB9oZYKjPIh3hWvq4Ef8Q43Hm16i/DO/5mMt+FoindOVm54Ce+9/9HMduNdUNAu8Lr7CFx9HDh00j7wnAl4G9iJ2TyGk1h/nHM/4V19+bWZtc5ilFOAqWa2J1D7bc65Vc65rXhf7nfhHaK7F+jlnMvysKFz7lB4r4z3RXLcnHPfAS8Dv+Gd8Dw50HQgi3FPdr3PPL3P8M7ruwNvHVuEd47raYH34pBTA+/VLryQXRI4xTk3P9Mkd9iR/bjdmc1LX4F37tpivHPwbg/U8wve+USj8PZi1QYuOdH5w9uTux1vT8bHeCfrH/p83gg8HlhnH8Z7T7PlnBuDt5341LxDcAvwPvsE1o/+wNN4601djv75OhQet5rZrMDfl+Kd77QB75D7I4H1OCvvAo0Cn6kvj1Z3oL4ZeBd3vYr3fizHO98sKz/grctL8Q7TJZPDw74nuJ0ZHhgv82HSp/D+qdhhZndn8bws1yG8cxWL4q3PU4Dvc1J7QE9gYWBdfwnvPN5k59wSvEO5rwSmex5eN1oHs5jGsbbNx5qv41kPgpZ5/xiKiBR8ZtYQLxQUcT73ARjKAnvAP3LOHc9RAslngT1lH+FdgJJ+rPElNGiPm4gUaGZ2fuBwXhm8vTpjFdqkoAscdrwNeEehrWBRcBORgu56vEPzK/DOqbnB33JE8lZgz/IOvCuYX/S1GMl1OlQqIiIiEiK0x01EREQkRARdx3J5oVy5cq5GjRp+lyEiIiJyTDNnztzinIvLqq1QBLcaNWowY8YMv8sQEREROSYzy3yXmb/pUKmIiIhIiFBwExEREQkRCm4iIiIiIaJQnOMmIiIiJy8lJYWEhASSk5P9LqVAiI6OJj4+nsjIyBw/R8FNREREciQhIYGYmBhq1KiBmfldTkhzzrF161YSEhKoWbNmjp+nQ6UiIiKSI8nJyZQtW1ahLReYGWXLlj3uvZcKbiIiIpJjCm2550TeSwU3ERERkRCh4CYiIiKFzrBhw7j55puPOc6GDRv+fnzttdeyaNGi436t8ePH06tXr+N+XlYU3ERERESykDm4vfPOOzRq1MjHihTcREREJMT07duX1q1b07hxY4YOHQpAiRIlePDBB2nevDnt27dn8+bNAIwdO5Z27drRsmVLunfv/vfwQ3bv3k3NmjVJSUkBYNeuXdSoUYPPP/+cGTNmcPnll9OiRQv2799P165d/76F5vfff0+rVq1o3rw5Z5xxBgDTpk2jQ4cOtGzZkg4dOrBkyZJcn3d1ByIiIiLH7bGxC1m0YVeuTrNR5ZI8cl7jY4733nvvERsby/79+znllFO44IIL2Lt3L+3bt+fJJ5/k3nvv5e233+ahhx6iY8eOTJkyBTPjnXfe4ZlnnuG55577e1oxMTF07dqVb7/9lr59+/Lpp59ywQUX0L9/f1577TWeffZZ2rRpc8TrJyUlcd111zFx4kRq1qzJtm3bAGjQoAETJ04kIiKCn3/+mQceeIBRo0bl6nuk4CYiIiIh5eWXX2bMmDEArFu3jmXLlhEVFfX3eWStW7fmp59+Ary+5y6++GI2btzIwYMHs+wz7dprr+WZZ56hb9++vP/++7z99ttHff0pU6bQuXPnv6cVGxsLwM6dOxk4cCDLli3DzP7ei5ebFNxERETkuOVkz1heGD9+PD///DOTJ0+mWLFidO3aleTkZCIjI//uXiM8PJzU1FQAbrnlFu6880569+7N+PHjefTRR/8xzdNOO43Vq1czYcIE0tLSaNKkyVFrcM5l2ZXHv//9b7p168aYMWNYvXo1Xbt2Pen5zUznuImIiEjI2LlzJ2XKlKFYsWIsXryYKVOmHHP8KlWqADB8+PBsx7vyyiu59NJLueqqq/4eFhMTw+7du/8x7qmnnsqECRNYtWoVwN+HSjO+1rBhw45rvnJKwU1ERERCRs+ePUlNTaVZs2b8+9//pn379kcd/9FHH6V///506tSJcuXKZTve5Zdfzvbt27n00kv/HjZo0CCGDBny98UJh8TFxTF06FD69etH8+bNufjiiwG49957+de//sVpp51GWlraSc5p1sw5lycTDiZt2rRxh64CERERkRPz119/0bBhQ7/LyBNffPEFX331FR9++GG+vm5W76mZzXTOtclqfJ3jJiIiIoXaLbfcwnfffce4ceP8LuWYFNxERESkUHvllVf8LiHHdI6biIiISIjIs+BmZu+ZWaKZLcim/XIzmxf4+dPMmmdo62lmS8xsuZndn2F4rJn9ZGbLAr/L5FX9IiIiIsEmL/e4DQN6HqV9FdDFOdcMeAIYCmBm4cBrwNlAI+BSMzt0Y7D7gV+cc3WBXwKPg0JaoL8YERERkbySZ8HNOTcR2HaU9j+dc9sDD6cA8YG/2wLLnXMrnXMHgU+BPoG2PsChTliGA31zu+4TMW3MK2x8sjHJ+/b4XYqIiIgUYMFyjts1wHeBv6sA6zK0JQSGAVRwzm0ECPwun90EzWywmc0wsxlJSUl5UPJhxSvUJt5tYt64t/L0dURERKRw8z24mVk3vOB236FBWYx23J3NOeeGOufaOOfaxMXFnUyJx9SofU+WhdehwqL3cOl50+GeiIiIiK/BzcyaAe8AfZxzWwODE4CqGUaLBzYE/t5sZpUCz60EJOZXrUdjYWFsb34d1dMTmD/+C7/LERERKbBWr15Nw4YNue6662jcuDFnnnkm+/fvp2vXrtx33320bduWevXq8fvvv/tdap7wrR83M6sGjAaucM4tzdA0HahrZjWB9cAlwGWBtq+BgcDTgd9f5V/FR9firKvYPOsZwqe+Dqdf7Hc5IiIieeu7+2HT/NydZsWmcPbTxxxt2bJljBgxgrfffpuLLrqIUaNGAZCamsq0adMYN24cjz32GD///HPu1hcE8rI7kBHAZKC+mSWY2TVmNsTMhgRGeRgoC7xuZnPMbAaAcy4VuBn4AfgLGOmcWxh4ztNADzNbBvQIPA4KUUWKsKLWABofmMPqBUe/4a2IiIicuJo1a9KiRQsAWrduzerVqwHo16/fP4YVNHm2x805d+kx2q8Frs2mbRzwj/tOBA6nnpErBeaBRufewt6X32Lrz89To8lIv8sRERHJOznYM5ZXihQp8vff4eHhf98A/tDw8PBwUgtoN12+X5xQkJQuG8ecuPNotv1ntm1c7Xc5IiIiUsAouOWyKj3vJIx0Vnz7gt+liIiISAGjm8znshp1GjG9eEfqJ3xO8t7HiS5eyu+SRERECowaNWqwYMHhu2nefffd/xinXLlyBfYcN+1xywNRnW6jJHtZNO5Nv0sRERGRAkTBLQ80a9+dReENqPjXe7i0gnlypIiIiOQ/Bbc8YGZsb3E9ldM3sXjCZ36XIyIikmucO+6bGUk2TuS9VHDLI63PHEAC5Qmf+prfpYiIiOSK6Ohotm7dqvCWC5xzbN26lejo6ON6ni5OyCPRRaJYUfMKuqx6jnXzJ1K1aWe/SxIRETkp8fHxJCQkkJSU5HcpBUJ0dDTx8fHH9RwFtzzU5Lyb2PXSG2z/+QUFNxERCXmRkZHUrFnT7zIKNR0qzUNlY8syK64PjXf8xs4Ny/0uR0REREKcglseq3b2HaQTxqpvn/e7FBEREQlxCm55rFbt+kwv3pk660dzcM92v8sRERGREKbglg+iOt1KCfazeJyuMBUREZETp+CWD1q378bc8CZUXDwMl5bidzkiIiISohTc8oGZsaPF9ZRPT2LZbx/7XY6IiIiEKAW3fNLurEtZTWUip70O6rhQREREToCCWz6Jjopkea0rqHlwCRvm/+Z3OSIiIhKCFNzyUbNeQ9juYtjxywt+lyIiIiIhSMEtH5WPjWVG3Pk02PE7uxMW+12OiIiIhBgFt3xWredtpBDOmu+e87sUERERCTEKbvmsfp06TC5+OrXXf0XKnq1+lyMiIiIhRMHNB9GdbqEoB1g27mW/SxEREZEQouDmg7btOjE9vAUVFn+ASz3gdzkiIiISIhTcfBAWZuxscT1l07exavwHfpcjIiIiIULBzScdzuzPcqoSpQ55RUREJIcU3HxSrEgky2pdSfzBlWye+6Pf5YiIiEgIUHDzUctzryfJlWLnr+qQV0RERI5Nwc1HFcuWYnrcBdTbNZk9CQv9LkdERESCnIKbz2r0vJVkF0nCuGf9LkVERESCnIKbzxrVqcnvxXtQc8NYUndt9rscERERCWIKbkEguvMtFCGFld+pQ14RERHJnoJbEOjQ9lT+DG9DhcUfQsp+v8sRERGRIKXgFgTCw4xdLa6nlNvJmvHD/C5HREREgpSCW5Do1ON8FlGTqOlvQHq63+WIiIhIEFJwCxLFoyNZXmsglQ6uYcvccX6XIyIiIkFIwS2ItDn3Gja6WHb/pg55RURE5J8U3IJI5bIlmRp3ITV3zWDf2jl+lyMiIiJBRsEtyNTueTN7XRE2fPd/fpciIiIiQUbBLcg0rVOd8cV6Un3j96Tt3OB3OSIiIhJEFNyCULHONxPu0lg97kW/SxEREZEgouAWhDq1bcPE8HZUWPoxHNzrdzkiIiISJBTcglBEeBi7Wl5PCbeH9ePf8bscERERCRIKbkGqW/dezHV1iZr+FqSn+V2OiIiIBAEFtyAVUzSKZbUHEpeynu2zv/K7HBEREQkCCm5BrN05A0lw5dgz/iW/SxEREZEgoOAWxKqWK8nkuIuounsOyaun+V2OiIiI+EzBLcjVOWsIu1xRNn7/nN+liIiIiM8U3IJcizrV+KXYOVTb9CPp29f6XY6IiIj4SMEtyJkZJTrfiHOw7jvdfF5ERKQwU3ALAV3btuLX8NOIW/YpJO/yuxwRERHxiYJbCIgMD2N3y+sp5vaxafzbfpcjIiIiPlFwCxHdu/dkumtI1MyhkJbqdzkiIiLiAwW3EFGqaCTLag0kNmUTO2eN8rscERER8YGCWwjpcM4AVqVXYN+El8A5v8sRERGRfKbgFkJqxMXwZ9zFVNqzkAOr/vS7HBEREclnCm4hpu6Zg9nuSpD4gzrkFRERKWwU3ELMKfXi+bHYOVTZ/CvpW1b6XY6IiIjkIwW3EGNmxHS+kVQXxoYfnve7HBEREclHCm4hqPspzfkxrBNxy0fC/u1+lyMiIiL5RMEtBEVFhLGn1RCKuAMkjX/T73JEREQkn+RZcDOz98ws0cwWZNPewMwmm9kBM7s7w/D6ZjYnw88uM7s90Paoma3P0HZOXtUf7HqecQZ/uKZEzXwHUg/6XY6IiIjkg7zc4zYM6HmU9m3ArcCzGQc655Y451o451oArYF9wJgMo7xwqN05Ny53Sw4dpYtFsazWQEqlbmH3zM/8LkdERETyQZ4FN+fcRLxwll17onNuOpBylMmcAaxwzq3J7foKgk5nX8LS9Crsn/iyOuQVEREpBIL9HLdLgBGZht1sZvMCh2LLZPdEMxtsZjPMbEZSUlLeVumT2uVj+CPuEsrvXcrB5eP9LkdERETyWNAGNzOLAnoDn2cY/AZQG2gBbASy7YXWOTfUOdfGOdcmLi4uL0v1Vf0zryHJlWTLT+oaREREpKAL2uAGnA3Mcs5tPjTAObfZOZfmnEsH3gba+lZdkDi1XmW+K9qLyokTcYmL/S5HRERE8lAwB7dLyXSY1MwqZXh4PpDlFauFiZlRqtMQkl0km358we9yREREJA/lZXcgI4DJQH0zSzCza8xsiJkNCbRXNLME4E7gocA4JQNtxYAewOhMk33GzOab2TygG3BHXtUfSnq2a8K4sK6UXTEa9m7xuxwRERHJIxF5NWHn3KXHaN8ExGfTtg8om8XwK3KnuoKlSEQ4e1tdT9TMn9g6/nXKnvuw3yWJiIhIHgjmQ6VyHM7p1pnf0ltSZPZ7kJLsdzkiIiKSBxTcCoiyJYqwvPZASqRuZ8+MT/wuR0RERPKAglsB0uWsC1iYXp0Dv7+iDnlFREQKIAW3AqRexZL8HncxZfet5ODSH/0uR0RERHKZglsB06j7IDa5Mmz/6UW/SxEREZFcpuBWwHRqUJlvos+jwpY/cZvm+12OiIiI5CIFtwLGzIjtNJh9rghJP6lDXhERkYJEwa0AOqddI74OO53YFV/B7k1+lyMiIiK5RMGtAIqODGdfy+sIc2nsmPC63+WIiIhILlFwK6DO69aRn10boma/Dwf3+l2OiIiI5AIFtwIqLqYIS2sNpFjaLvZN/8jvckRERCQXKLgVYGec2Zs56bU5OOlVSE/3uxwRERE5SQpuBVjDyqX4vdxFlN6/ltTF4/wuR0RERE6SglsB16T7FSS4cuz4RV2DiIiIhDoFtwKuS4PKfB3dm3JbZ+DWz/K7HBERETkJCm4FXFiYUbbTtex2Rdn2s/a6iYiIhDIFt0Kgd9sGjLEzKL3qG9iZ4Hc5IiIicoIU3AqBolHh7G91Lc7BrvGv+l2OiIiInCAFt0Kib9dT+d61I2ruB3Bgt9/liIiIyAlQcCskKpSMZlmtgUSn7yV52jC/yxEREZEToOBWiPTocQ7T0uuT8sfrkJbqdzkiIiJynBTcCpEmVUoxoezFxCRvIG3R136XIyIiIsdJwa2QaX7GJaxOr8Cu3170uxQRERE5TgpuhcwZjSrzZXQfymybC2un+l2OiIiIHAcFt0ImPMyI63gVO1xxtus2WCIiIiFFwa0Q6tuuHl9YD0qt+QG2rfK7HBEREckhBbdCqHiRCPa3vIZUZ+ye+Irf5YiIiEgOKbgVUhd0OYVv0jtQZN4nsH+73+WIiIhIDii4FVKVSxdlaa2BRKXv58C09/0uR0RERHJAwa0Q69m9B5PSGpP65xuQetDvckREROQYFNwKsRZVSzOh7EUUP5BI+sIxfpcjIiIix6DgVsi1Or0/y9KrsPu3F8E5v8sRERGRo1BwK+TObFKZMdG9KbVjEaye5Hc5IiIichQKboVceJhR/rSBbHEl2fWrOuQVEREJZgpuwgXt6jCSMym57hfYsszvckRERCQbCm5CTHQkyS2u4oCLZO+El/0uR0RERLKh4CYA9O/SijHpHYla8Bns3ep3OSIiIpIFBTcBoGpsMZbWvIJId4CDU972uxwRERHJgoKb/O3cM07nt7TmpE19C1KS/S5HREREMlFwk7+1rl6G32IvpujBbaTPG+l3OSIiIpKJgpsc4ZRuffkrvRr7JrysDnlFRESCjIKbHOHsppUYVaQPJXYtgxW/+F2OiIiIZKDgJkeICA+j4mkD2OxKs2f8S36XIyIiIhkouMk/9G9XmxGuJyUSJsLmRX6XIyIiIgEKbvIPpYpGcqDFIPa5IuyfqL1uIiIiwULBTbJ0SZdmjErvTOSiUbB7s9/liIiICApuko3qZYuzpMYAwlwqKVPe8rscERERQcFNjqL36Z35Oa0VadPehYP7/C5HRESk0FNwk2ydUqMMv5S5iOiUHaTPGeF3OSIiIoWegptky8w4tWsv5qbXIvn3VyA93e+SRERECjUFNzmqc5pV5vPIvhTbvQqWfOt3OSIiIoWagpscVVREGPGnXcSy9CqkfHs/HNzrd0kiIiKFloKbHNMl7WvzuF1P5J4EGP+U3+WIiIgUWgpuckyli0XRpft5fJLajfTJr8PGeX6XJCIiUigpuEmODOxQg5FlrmOHK0H617dCeprfJYmIiBQ6Cm6SI5HhYdzTpx2PHryCsI2zYfo7fpckIiJS6Ci4SY6dVqccqY3O53fXnPSfH4Od6/0uSUREpFBRcJPj8mCvxjyafg2pqanw3b1+lyMiIlKoKLjJcalSuih9u3bguYP9YPE38Nc3fpckIiJSaCi4yXG7rnMtfix5ASvCauDG3QPJu/wuSUREpFDIs+BmZu+ZWaKZLcimvYGZTTazA2Z2d6a21WY238zmmNmMDMNjzewnM1sW+F0mr+qX7EVHhvNQ72bctf9q2L0Rfv2P3yWJiIgUCnm5x20Y0PMo7duAW4Fns2nv5pxr4Zxrk2HY/cAvzrm6wC+Bx+KDMxpWoEy9UxnhzsRNGwoJM/0uSUREpMDLs+DmnJuIF86ya090zk0HUo5jsn2A4YG/hwN9T7hAOWkPn9eY/0u9mF0RZWHsbZCW6ndJIiIiBVqwnuPmgB/NbKaZDc4wvIJzbiNA4Hf57CZgZoPNbIaZzUhKSsrjcgunmuWKc1nnxty7bwBsng9TXve7JBERkQItWIPbac65VsDZwE1m1vl4J+CcG+qca+OcaxMXF5f7FQoAN3Wrw7wSnZgc2Q43/inYvsbvkkRERAqsoAxuzrkNgd+JwBigbaBps5lVAgj8TvSnQjmkWFQED/ZqxJ27B5CaDnx7Fzjnd1kiIiIFUtAFNzMrbmYxh/4GzgQOXZn6NTAw8PdA4Kv8r1AyO7dpJWrUqscL6RfB8p9g4Wi/SxIRESmQ8rI7kBHAZKC+mSWY2TVmNsTMhgTaK5pZAnAn8FBgnJJABWCSmc0FpgHfOue+D0z2aaCHmS0DegQei8/MjMf6NObtAz1IKFofvrsf9m/3uywREZECx1whOKzVpk0bN2PGjGOPKCfl8bGLmDb5V8YW+TfWaiCc96LfJYmIiIQcM5uZqTu0vwXdoVIJXbf3qMumYvUZG90HZr4Pa6f4XZKIiEiBouAmuaZkdCT39WzA/dvPY2/RSl7fbqkH/S5LRESkwFBwk1x1Qat46leryAMHBkHSYvjzJb9LEhERKTAU3CRXhYUZj/duwtf7m7KozOkw4f9g6wq/yxIRESkQFNwk1zWNL8Wlbatx9eYLSQuPgm9uV99uIiIiuUDBTfLEPWfWZ3+ROIYVHQirJsLcT/0uSUREJOQpuEmeKFM8irvPqs9/Nrdne2wL+OEB2LvV77JERERCmoKb5JnL2lajYaXS3Lx7IO7ALvjp336XJCIiEtIU3CTPhIcZj/dpzB+7KzC10uUw52PvsKmIiIicEAU3yVNtasTSr2UVrlt9BimlasDY2yEl2e+yREREQpKCm+S5+89ugIuI5qXoG2DbCvj9Ob9LEhERCUkKbpLnypeM5rYz6vLqmqpsrN4bJr0AiYv9LktERCTkKLhJvhh0Wg3qlC/BkKR+uCIlvL7d0tP9LktERCSkKLhJvogMD+PR8xozd1sUv1a9GdZOhtkf+l2WiIhISFFwk3zTsW45zm5SkZv+asSBKqd63YPsSfS7LBERkZCh4Cb56sFzGwLGUxHXQ8p++P5ffpckIiISMhTcJF/FlynGTV3rMGxJFGsb3wALvoBlP/tdloiISEhQcJN8d13nWlSLLcb1qzrjytaDb++Eg/v8LktERCToKbhJvouODOfhXo34K+kA31a/F3asgQlP+12WiIhI0FNwE1+c0bA83erHcf/Mkuxvcin8+Spsmu93WSIiIkFNwU18YWY8fF5jDqam82TKZVC0jHc7rPQ0v0sTEREJWgpu4pua5YpzbaeafDR3N6vaPAjrZ8CM9/wuS0REJGgpuImvbj69DpVKRXPz/Dq4WqfDz4/Brg1+lyUiIhKUFNzEV8WiInjw3IYs3Libr+LvgvQU+O5ev8sSEREJSgpu4rtzm1bi1FpleXTSPvZ1uBv+GguLx/ldloiISNBRcBPfmRmP9m7M7uRUntrRHco3gnF3w4HdfpcmIiISVBTcJCjUrxjDwFNr8NH0Daxo/6R3ntuvT/pdloiISFBRcJOgcXuPupQtHsU9U4rg2lwN096C9bP8LktERCRoKLhJ0CgZHcl9PRswa+0Ovi53HRQvD2Nvg7RUv0sTEREJCgpuElQuaBVPy2qleeLn9ezr/l/YNA+mvul3WSIiIkFBwU2CSliY8XjvJmzde4Dn1jWAej3htydhx1q/SxMREfGdgpsEnabxpbjklGoMm7yGlW0fAwy+vRuc87s0ERERXym4SVC656z6lCgSwUO/7cB1+xcs+wEWfel3WSIiIr5ScJOgFFs8irvPrMefK7byXbG+UKk5fHcf7N/hd2kiIiK+UXCToHVZu+o0qlSSJ75byv6ez8PeJPjlcb/LEhER8Y2CmwSt8DDj8T6N2bgzmdcWx0C7ITDjPVg3ze/SREREfKHgJkGtTY1Yzm9ZhaETV7Km2R1Qskqgb7cUv0sTERHJdwpuEvT+dXYDIsONx35cA+c+C4mL4M+X/S5LREQk3ym4SdArXzKa27vX49fFifyS3goa9oYJz8C2lX6XJiIikq8U3CQkDOxQg9pxxXn8m0Ukd/8vhEXCN3eobzcRESlUFNwkJERFhPFo78as2bqPd+YmQ/dHYOV4mDfS79JERETyjYKbhIxOdeM4u0lFXv1tOevrXApV2sAP/4J92/wuTUREJF8ouElIefDchgD897ulcN5LkLwTfvq3z1WJiIjkDwU3CSnxZYpxY9c6fDt/I3/sqQin3gyzP4LVk/wuTUREJM8puEnIGdy5FtVii/Ho1wtJ6XQPlK4OY2+H1AN+lyYiIpKnFNwk5ERHhvNwr0YsS9zD8OmJ0Ot52LoMfn/e79JERETylIKbhKQzGpana/04Xvx5GYkVOkKTC2HS85C01O/SRERE8oyCm4QkM+OR8xpzMDWd/323BHo+BZFF4ZvbIT3d7/JERETyhIKbhKya5YpzbaeajJqVwMytEdDjCVjzB8z52O/SRERE8oSCm4S0m7rVoWLJaB7+aiFpLQZAtQ7w40OwJ8nv0kRERHKdgpuEtOJFInjw3IYs3LCLT2ckwHkvwsG98MMDfpcmIiKS6xTcJOT1alaJ9rVi+b8flrC9WE3odCfMHwkrfvW7NBERkVyl4CYhz8x4rHcTdien8uyPS6DjnVC2jncT+oP7/C5PREQk1yi4SYFQv2IMV55anU+mrWVB4gHo9QJsXw0Tn/G7NBERkVyj4CYFxu3d61G2eBQPf7WA9OqdoMXl8OcrsHmh36WJiIjkCgU3KTBKFY3k3p4NmLV2B2Nmr4cz/wPRpWDsberbTURECgQFNylQLmwVT4uqpXnqu8XsCouBs/4LCdNh5nt+lyYiInLSFNykQAkLMx7v05itew/w8s/LoNnFULML/PwY7Nrod3kiIiInRcFNCpxm8aW55JRqvP/napYm7vEuVEg7CN/f53dpIiIiJ0XBTQqke86qT4kiETz69UJcbC3ofA8s+gqWfO93aSIiIidMwU0KpNjiUdx9Zj3+XLGV7xZsgg63QlxDGHc3HNjjd3kiIiInJM+Cm5m9Z2aJZrYgm/YGZjbZzA6Y2d0Zhlc1s9/M7C8zW2hmt2Voe9TM1pvZnMDPOXlVv4S+y9pVp1Glkvznm0XsSw/zboe1cx389l+/SxMRETkhebnHbRjQ8yjt24BbgWczDU8F7nLONQTaAzeZWaMM7S8451oEfsblZsFSsISHGY/1acyGncm8/tsKqNYeWl8FU9+ADXP8Lk9EROS45Vlwc85NxAtn2bUnOuemAymZhm90zs0K/L0b+Auokld1SsF2So1Yzm9ZhaETV7J6y17o/igUj/P6dktL9bs8ERGR4xLU57iZWQ2gJTA1w+CbzWxe4FBsmaM8d7CZzTCzGUlJSXldqgSxf53dgMhw44lvFkHR0tDzadg4B6YN9bs0ERGR4xK0wc3MSgCjgNudc7sCg98AagMtgI3Ac9k93zk31DnXxjnXJi4uLq/LlSBWvmQ0t3Wvyy+LE/l18WZofD7UPRN+/Q/sWOd3eSIiIjkWlMHNzCLxQtvHzrnRh4Y75zY759Kcc+nA20Bbv2qU0DKoQ01qxxXnsbGLSE5Nh3OeBRyMuwec87s8ERGRHAm64GZmBrwL/OWcez5TW6UMD88HsrxiVSSzqIgwHu3dmDVb9/HupFVQpjp0/Rcs/Q7++trv8kRERHIkL7sDGQFMBuqbWYKZXWNmQ8xsSKC9opklAHcCDwXGKQmcBlwBnJ5Ftx/PmNl8M5sHdAPuyKv6peDpVDeOno0r8uqvy9mwYz+0vxEqNoVx90LyTr/LExEROSZzheAwUZs2bdyMGTP8LkOCQML2fZzx3AS6N6rAa5e1gvUz4Z3u0OYaODdzzzQiIiL5z8xmOufaZNUWdIdKRfJSfJli3NStDt/O28ify7dAldbQdjBMfwfWTfe7PBERkaNScJNCZ3DnWlSNLcojXy8kJS0dTn8ISlaGr26E5F3HnoCIiIhPFNyk0ImODOfhXo1ZlriHDyavgSIxcP5bsHUFjLke0tP9LlFERCRLCm5SKHVvWJ6u9eN48aelJO5OhpqdoOdTsGQcTPif3+WJiIhkScFNCiUz4+FejUhOTeN/3y3xBrYdDC0uhwlPw19j/S1QREQkCwpuUmjViivBtZ1qMWpWAjPXbAMzOPd574KFMUMg8S+/SxQRETmCgpsUajd3q0PFktE88vVC0tIdREbDxR9BZDH49DLYv93vEkVERP6m4CaFWvEiETx4bkMWrN/FiGlrvYElK3vhbcc6+OIaSE/zt0gREZEABTcp9Ho1q8Sptcry1Li/WJ64xxtYrZ3XIe+KX+CXx/0tUEREJEDBTQo9M+P5i5sTHRnOkI9msudAqtfQehC0uRr+eBHmf+FniSIiIoCCmwgAlUoV5ZXLWrIyaQ/3fTGPv28F1/N/UO1U+Opm2DjP3yJFRKTQU3ATCehQuxz39WzAt/M38u6kVd7AiCi46AMoFgufXg57t/hbpIiIFGoKbiIZDO5ci56NK/LUd4uZunKrN7BEee9ihT2b4fNBkJbia40iIlJ4KbiJZGBm/F//ZlQvW4ybPpnN5l3JXkOVVtD7ZVj9O/z4kL9FiohIoaXgJpJJTHQkbw1ozb6Dqdz48SwOpgbuXdr8Emh/E0x9E2Z/7G+RIiJSKCm4iWShboUYnrmwGTPXbOe/4zLcQaHH41CzC3xzByTM9K9AEREplBTcRLLRq1llrulYk2F/ruarOeu9geER0H8YxFSAzy6H3Zt9rVFERAoXBTeRo7j/7Aa0rRHL/aPms3jTLm9gsVi4ZAQk74SRV0DqAX+LFBGRQkPBTeQoIsPDePXylsRERzDkw5nsSg5cUVqxCfR9HdZNhe/u9bdIEREpNBTcRI6hfEw0r13eioTt+7lr5FzS0wOd8zY+HzreCTOHwfR3fa1RREQKBwU3kRw4pUYsD57bkJ8WbeaNCSsON5z+ENTp4e11W/OnfwWKiEihoOAmkkODOtSgd/PKPPfjEiYtC9xBISwcLngHSleHkVfCzvX+FikiIgWagptIDpkZT1/QlLrlY7j109ms37HfayhaGi4dASnJ3pWmKft9rVNERAouBTeR41AsKoI3BrQiJTWdGz+ayYHUNK8hrj70GwobZnt9vB26Sb2IiEguUnATOU614krw7EXNmZuwk8fGLjrc0OAc6PoAzB0BU97wr0ARESmwFNxETsBZjStyQ9fafDJ1LSNnrDvc0PkeaNDLu5/pyvG+1SciIgWTgpvICbqrRz1Oq1OWh75cwIL1O72BYWFw/ptQrh58Pgi2r/azRBERKWByHNzMrIyZNTazWmamwCeFXkR4GC9f0pKyxaMY8tFMduw76DUUiYFLPgaXDp9eDgf3+luoiIgUGEcNYGZWysweMLP5wBTgLWAksMbMPjezbvlRpEiwKluiCG8MaE3irgPc9umcw53zlq0NF74HiYvgyxt1sYKIiOSKY+05+wJYB3RyztV3znV0zrVxzlUF/gf0MbNr8rxKkSDWomppHundiAlLk3jpl2WHG+p0h+6PwqIvYdILfpUnIiIFSMTRGp1zPY7SNgOYkesViYSgy9pWY/baHbz0yzKaVy3F6Q0qeA0dboWN8+CXx6FCE6h3pr+FiohISMvRuWpm9ktOhokUVmbGf/o2oVGlktz+6RzWbt13qAF6v+LdlH7UtbBlub+FiohISDvWOW7RZhYLlAtcnBAb+KkBVM6XCkVCRHRkOG8OaI2ZMeSjmSSnBDrnjSoGl3wC4RHw6WWQvMvfQkVEJGQda4/b9cBMoEHg96Gfr4DX8rY0kdBTrWwxXrykBX9t2sWDYxbgDl2UULoa9B8OW5fDmOshPd3fQkVEJCQdNbg5515yztUE7nbO1XLO1Qz8NHfOvZpPNYqElG71y3Pr6XUZNSuBT6atPdxQsxP0fAqWjIMJ//OvQBERCVnHOlTaEcA590o27SXNrEleFCYSym47oy5d68fx6NcLmb12++GGtoOhxeUw4Wn4a6x/BYqISEg61qHSC8zsTzN72MzONbO2ZtbZzK42sw+Bb4Ci+VCnSEgJCzNevLgFFUpGc+PHs9i654DXYAbnPg9VWsOYIZD4l7+FiohISDnWodI7gHOBjUB/4HHgDqAO8KZzrrNzbnqeVykSgkoXi+LNAa3Ztvcgt4yYTWpa4Ly2yGi4+COIKu5drLB/+9EnJCIiEnDM7kCcc9uBksA84CdgErAFaGBmLfK0OpEQ16RKKf7Ttwl/rtjKcz8tPdxQsjJc9CHsWAdfXAPpaf4VKSIiISOn9xxtDQwBKuF1AzIY6Aq8bWb35k1pIgVD/zZVuaxdNd4Yv4LvF2w63FCtHZz7LKz4BX55zL8CRUQkZOQ0uJUFWjnn7nbO3QW0AeKAzsCgPKpNpMB45LxGNI8vxd2fz2Vl0p7DDa0HQZtr4I+XYP4XvtUnIiKhIafBrRpwMMPjFKC6c24/cCDXqxIpYIpEhPP6gNZERYQx5KOZ7DuYerix59NQ7VT46mbYONe/IkVEJOjlNLh9Akwxs0fM7BHgD2CEmRUHFuVZdSIFSJXSRXnl0pYsT9zD/aPmH+6cNyIKLvoAisXCp5fD3i3+FioiIkErR8HNOfcEcB2wA9gJDHHOPe6c2+ucuzwP6xMpUE6rU467zqzP13M3MOzP1YcbSpT3rjTdmwSfD4K0FL9KFBGRIJbTPW4452YG7qTwonNuRl4WJVKQ3dClNj0aVeDJb/9i+upthxuqtILzXoLVv8OPD/lXoIiIBK0cBzcRyR1hYcZzFzUnvkxRbvx4Fom7kg83Nr8E2t8EU9+E2R/5V6SIiAQlBTcRH5SMjuTNK1qzJzmVmz+ZTUpahpvO93gcanaBb+6ABO3cFhGRwxTcRHzSoGJJnr6gKdNWb+Pp7xYfbgiPgP7DIKYifDYAdm/2rUYREQkuCm4iPurTogqDOtTg3UmrGDt3w+GGYrFwyQhI3gkjr4BU9bojIiIKbiK+e+CchrSpXob7Rs1j6ebdhxsqNoG+r8O6qfCdblAiIiIKbiK+i4oI47XLW1EsKoIhH85kd3KGrkAanw8d74SZw2D6u77VKCIiwUHBTSQIVCgZzWuXtWTNtn3c8/m8w53zApz+ENQ909vrtuZP/4oUERHfKbiJBIl2tcryr7Mb8P3CTQyduPJwQ1g49HsbytSAkVfCzgTfahQREX8puIkEkWs61uTcppX43/eL+XN5hltfFS0Nl3wCKcnebbFS9vtWo4iI+EfBTSSImBn/u7AZteJKcMuI2WzcmSGgxdWHfkNh4xwYeztkPJwqIiKFgoKbSJApUSSCNwe0JjkljRs+msWB1LTDjQ3OgW4PwrxPYcob/hUpIiK+UHATCUJ1ypfg2f7NmbNuB//55q8jGzvdDQ16efczXTnel/pERMQfCm4iQersppW4vnMtPpyyhlEzM1yQEBYG578J5erB54Ng+2q/ShQRkXym4CYSxO45qz7ta8XywJj5LNyw83BDkRi45GNw6d7FCgf3+lekiIjkGwU3kSAWER7GK5e2okyxKG74aBY792XonLdsbbjwPUhcBF/eqIsVREQKgTwLbmb2npklmtmCbNobmNlkMztgZndnautpZkvMbLmZ3Z9heKyZ/WRmywK/y+RV/SLBIi6mCK9d3oqNO/dzx8g5pKdnCGh1ukP3R2HRlzDpBb9KFBGRfJKXe9yGAT2P0r4NuBV4NuNAMwsHXgPOBhoBl5pZo0Dz/cAvzrm6wC+BxyIFXuvqZXi4VyN+XZzIq78tP7Kxw63Q5EL45XFY+qM/BYqISL7Is+DmnJuIF86ya090zk0HUjI1tQWWO+dWOucOAp8CfQJtfYDhgb+HA31ztWiRIDagfXXOb1mFF35eyoSlSYcbzKD3K95N6UddC1uWZz8REREJacF4jlsVYF2GxwmBYQAVnHMbAQK/y2c3ETMbbGYzzGxGUlJSdqOJhAwz47/nN6V+hRhu+3Q267btO9wYVcy7s0J4BHx6GSTv8q9QERHJM8EY3CyLYcd91rVzbqhzro1zrk1cXFwulCXiv6JR4bw5oDVp6Y4bPp5JckqGznlLV4P+w2HrchhzPaSn+1eoiIjkiWAMbglA1QyP44ENgb83m1klgMDvxHyuTcR3NcoV54WLWrBg/S4e+WrhkY01O0HPp2DJOJjwtD8FiohIngnG4DYdqGtmNc0sCrgE+DrQ9jUwMPD3QOArH+oT8V33RhW45fQ6fDZjHZ9OW3tkY9vB0GIATPgf/DXWnwJFRCRPROTVhM1sBNAVKGdmCcAjQCSAc+5NM6sIzABKAulmdjvQyDm3y8xuBn4AwoH3nHOHdis8DYw0s2uAtUD/vKpfJNjd3r0ec9bt4OGvFtKwUkmaVy3tNZjBuc9B0l8wZgiUrQPlG/paq4iI5A5zhaDTzjZt2rgZM2b4XYZIrtu+9yC9XpkEwNhbOhJbPOpw464NMLQrRBaDwb9BUXV7KCISCsxspnOuTVZtwXioVERyqEzxKN4Y0IqkPQe47dPZpGXsnLdkZbjoQ9iZAF9cDelp2U9IRERCgoKbSIhrFl+aJ/o05vdlW3jhp6VHNlZr5x02XfEr/PKYPwWKiEiuybNz3EQk/1x8SjVmrdnBq78tp3nV0vRoVOFwY+uBsHEu/PESxFSG9kP8K1RERE6K9riJFBCP9WlM0yqluHPkHFZv2Xtk49n/gwa94Pv7YPo7/hQoIiInTcFNpICIjgzn9ctbER5mDPloJvsPZjinLTwSLnwf6p0N394FM4f5VqeIiJw4BTeRAqRqbDFeuqQlSzbv5oEx8zniqvGIKLhoONTpAWNvh9kf+1aniIicGAU3kQKmS7047uxejzGz1/PhlDVHNkYUgYs/glpd4aubYO5nvtQoIiInRsFNpAC6qVsdzmhQnsfHLmLmmm1HNkZGezekr9ERvhwCC0b5U6SIiBw3BTeRAigszHj+4hZUKVOUGz+eRdLuA0eOEFUMLvsMqp0Ko66DRbp7nIhIKFBwEymgShWN5I3LW7Nzfwo3fzKL1LT0I0eIKu6Ft/g2Xge9i7/1p1AREckxBTeRAqxR5ZI81a8pU1dt45kflvxzhCIxcPkXUKkFjBwIS3/I9xpFRCTnFNxECrjzW8Zz5anVGTpxJe/8vvKfI0SXhAGjoGIT+GwALP85/4sUEZEcUXATKQT+3asRZzepyH++/YuhE1f8c4SipWHAaIirD59eDivH53eJIiKSAwpuIoVAZHgYL1/akl7NKvHfcYt5ffzyf45ULBau+Apia8Mnl8Cq3/O/UBEROSoFN5FCIjI8jBcvbkGfFpV55vslvPzLsn+OVLwsXPkVlKkOn1wMaybnf6EiIpItBTeRQiQiPIznL2pBv1ZVeP6npTz/09Ij764AUCIOrvwaSlaGjy+EddP8KVZERP5BwU2kkAkPM/7vwuZc1Cael39ZxrM/LvlneIupAAPHQony8NEFkDDTn2JFROQICm4ihVB4mPF0v2Zc2rYqr/22gqe/X/zP8FaykhfeipaBj86HDXN8qVVERA5TcBMppMLCjCf7NmVA+2q8NWElT3771z/DW6l4GPQNFCkFH/SBTfP9KVZERAAFN5FCLSzMeKJPEwZ1qME7k1bx2NhF/wxvpavBwK+9Oy180Ac2L/KnWBERUXATKezMjEfOa8Q1HWsy7M/V/PurBaSnZwpvsTW9w6bhUfBBb0jK4i4MIiKS5xTcRAQz46FzG3J9l1p8NGUtD36ZRXgrW9sLbxYGw8+DLVl0JyIiInlKwU1EAC+83d+zATd1q82IaWu5f/Q80jKHt3J1va5C0tO88LY1i7swiIhInlFwE5G/mRl3n1mf286oy8gZCdzzxdx/hrfyDbw9b6kHYHhv2L7al1pFRAojBTcROYKZcUePetzZox6jZ63nzpFzSE1LP3KkCo28Oywc3APDzoMda/0pVkSkkFFwE5Es3XpGXe45qz5fzdnAbZ/NISVzeKvUDK78EpJ3eodNd673pU4RkcJEwU1EsnVTtzo8cE4Dvp23kVtHzP5neKvcEq4YA/u2wfBesGujP4WKiBQSCm4iclSDO9fm370a8d2CTdz08SwOpmYKb/GtYcAo2JPo7XnbvdmfQkVECgEFNxE5pms61uSx3o35cdFmbvhoJgdS044coWpbuPxz2LXe6+dtT5I/hYqIFHAKbiKSIwM71OA/fZvwy+JErv9wJskpmcJb9Q5w2UjYvsa7w8Lerf4UKiJSgCm4iUiODWhfnaf7NWXC0iSu+2DGP8NbzU5w6QjYtgI+7OOd+yYiIrlGwU1EjsslbavxzAXNmLR8C1cPm87+g5nCW+1ucMnH3m2xPjwf9u/wpU4RkYJIwU1Ejlv/NlV5/qLmTFm5lUHvT2PvgdQjR6jTHS7+CDYvhI8ugORd/hQqIlLAKLiJyAk5v2U8L1zcgumrtzHo/WnsyRze6p0FFw2HjXPg4wvhwG5f6hQRKUgU3ETkhPVpUYVXLm3FrLU7uPLdqexOTjlyhAbnwoXvQcIM+PgiOLjXn0JFRAoIBTcROSnnNqvEa5e1ZF7CTga8O42d+zOFt0Z94IK3Yd0U+ORiOLjPn0JFRAoABTcROWk9m1Ti9ctbsWjDTga8M5Ud+w4eOUKTC6Dvm7B6Enx6GaQk+1OoiEiIU3ATkVxxZuOKvDmgNUs27ebyd6ayfW+m8Nb8YujzGqwcD59dDqkHfKlTRCSUKbiJSK45o2EFhl7ZmmWJe7j07Sls3ZMpnLW8HM57CZb/DCOvhNSDWU9IRESypOAmIrmqa/3yvDuwDau27OXSt6eQtDtTeGs9EM59DpZ+D19cBWkpWU9IRET+QcFNRHJdp7pxvD/oFNZt28+lb08hcXemc9pOuRbOfgYWfwOjroW01KwnJCIiR1BwE5E80aFOOd6/6hQ27NjPJUOnsHlXpvDW7no480lY9CV8OQTS07KcjoiIHKbgJiJ5pn2tsgy/ui2bdyZz8VuT2bhz/5EjdLgZuj8K8z+Hr25SeBMROQYFNxHJU6fUiOWDa9qxdc9BLn5rCgnbM/Xj1vEO6PYQzB0BY2+F9HR/ChURCQEKbiKS51pXL8OH17Zj+z4vvK3blim8dbkHutwHsz+Cb+8E5/wpVEQkyCm4iUi+aFG1NJ9c2549B1K5ZOgU1mzNdPurrv+CjnfCzPdh3D0KbyIiWVBwE5F80zS+FJ9c1459B1O5+K0prNqSIbyZwRkPQ4dbYPrb8MMDCm8iIpkouIlIvmpcuRSfXNeeg2npXPzWZFYk7TncaAY9noB2N8CU1+GnhxXeREQyUHATkXzXsFJJRlzXnnTnuPitKSzbvPtwoxn0fApOuQ7+fBl+fULhTUQkQMFNRHxRv2IMnw5ujxlcMnQKSzZlCm9nPwOtB8Hvz8H4p32rU0QkmCi4iYhv6pT3wltEuHHp21NYtGHX4cawMDj3BWg5ACY8DRP/z79CRUSChIKbiPiqdlwJPht8KkUiwrjsnSksWL/zcGNYGJz3MjS7BH79D0x60bc6RUSCgYKbiPiuRrnifDb4VIpHRXDZ21OYl7DjcGNYOPR9HZpcAD8/ApNf861OERG/KbiJSFCoVrYYnw5uT6likVz+zlRmr91+uDEsHM4fCo36eN2ETH3Lv0JFRHyk4CYiQaNqbDE+HXwqscWjuOLdacxcs+1wY3gEXPAuNOgF390L09/1r1AREZ8ouIlIUKlSuiifDm5PXEwRrnx3GtNWZQxvkXDh+1Cvp3drrJnD/StURMQHCm4iEnQqlfLCW8VS0Qx8bxqTV2w93BgRBRd9AHW6w9jbYM4n/hUqIpLPFNxEJChVKBnNiMHtiS9TlKuGTeOP5VsON0YUgYs/glpd4MsbYd5I/woVEclHCm4iErTKx3jhrXpsca4eNp2JS5MON0YWhUtGQI2OMOZ6WDDKv0JFRPKJgpuIBLVyJYowYnB7asWV4NoPZvDb4sTDjVHF4LLPoGp7GHUdzPvcv0JFRPJBngU3M3vPzBLNbEE27WZmL5vZcjObZ2atAsPrm9mcDD+7zOz2QNujZrY+Q9s5eVW/iASP2OJRjLiuHfUqlOD6D2fy86LNhxujisPlI6Faexh9LYz/n+5tKiIFVl7ucRsG9DxK+9lA3cDPYOANAOfcEudcC+dcC6A1sA8Yk+F5Lxxqd86Ny4vCRST4lC4WxcfXtKdhpRhu+HgmPyzcdLixSAxcMQaaXwrj/wujB0NKsn/FiojkkTwLbs65icC2o4zSB/jAeaYApc2sUqZxzgBWOOfW5FWdIhI6ShWL5MNr29GkSilu+ngW4+ZvPNwYUQT6vgGn/xvmj4QPesPeLdlPTEQkBPl5jlsVYF2GxwmBYRldAozINOzmwKHV98ysTHYTN7PBZjbDzGYkJSVlN5qIhJiS0ZF8cHVbmlctzS0jZjN27obDjWbQ+W7oPww2zoW3T4fExb7VKiKS2/wMbpbFsL9PTDGzKKA3kPFs4zeA2kALYCPwXHYTd84Ndc61cc61iYuLy5WCRSQ4xERHMvzqtrSuVobbPp3NV3PWHzlC4/Nh0DhI2Q/v9oAVv/pTqIhILvMzuCUAVTM8jgcy/OvM2cAs59zfZyE75zY759Kcc+nA20DbfKlURIJOiSIRDLv6FNrVLMsdn81h1MyEI0eIbw3X/QqlqsJHF+oWWSJSIPgZ3L4GrgxcXdoe2Omcy3DCCpeS6TBppnPgzgeyvGJVRAqHYlERvDfoFDrULsfdX8zl3UmrcBmvKC1dFa75wbvLwrd3wvf/gvQ0/woWETlJedkdyAhgMlDfzBLM7BozG2JmQwKjjANWAsvx9p7dmOG5xYAewOhMk33GzOab2TygG3BHXtUvIqGhaFQ47wxsQ/eGFXjim0Xc/MlsdienHB6hSAxcOgLa3QBTXodPL4MDu/0rWETkJJgrBP0dtWnTxs2YMcPvMkQkD6WnO4b+vpL/+2EJ1WKL8frlrWhYqeSRI01/B8bdC+Ubeh33lor3p1gRkaMws5nOuTZZtenOCSJSIISFGUO61OaTa9ux90AqfV/7g5HT1x050inXep317ljrXXG6fqY/xYqInCAFNxEpUNrVKsu3t3aidfUy3DtqHnd/Ppf9BzOc11anO1zzE0REw/vnwsIvfatVROR4KbiJSIETF1OED69px62n12HUrATOf/0PViTtOTxC+QbeFaeVmsHnA+H353SbLBEJCQpuIlIghYcZd55Zn/cHncLmXcn0fmUS38zL0ONQ8XJw5dfQtD/88jh8eSOkHvSvYBGRHFBwE5ECrWv98nx7ayfqV4zh5k9m88hXCziQGjh0GhkN/d6Grg/A3E/gw76w72h36hMR8ZeCm4gUeJVLF+Wz60/lmo41GT55DRe9OZmE7fu8RjPoeh9c8C4kzIB3zoAty/wtWEQkGwpuIlIoRIaH8e9ejXhzQCtWJu3l3Jcn8evizYdHaHohDPoGknd54W3lBP+KFRHJhoKbiBQqPZtUYuwtHalSuihXD5vB/75fTGpautdYta130UJMZfioH8wc7m+xIiKZKLiJSKFTo1xxRt/YgUvbVuWN8Su4/J2pJO5K9hrLVPduk1WzC4y9FX58SLfJEpGgoeAmIoVSdGQ4T/VrxvMXNWdewk7OeXkSf67YEmgsBZeNhFOugz9fgc+ugIN7/S1YRAQFNxEp5Pq1iuerm0+jVNEIBrwzlVd/XUZ6uoPwCDj3WTj7GVj6HbzXE3ZtOPYERUTykIKbiBR69SrE8PXNHenVrDLP/riUq4dPZ/veQJ9u7a6HSz+DbSu922RtmONrrSJSuCm4iYgAxYtE8NIlLXiibxP+XL6Vc1/+nVlrt3uN9c6Ea36EsAh4/2z46xt/ixWRQkvBTUQkwMy4on11vrjhVMLCjIvfmsx7k1bhnIMKjeHaX6B8Q/hsAPzxkm6TJSL5TsFNRCSTZvGl+faWTnSpV57Hv1nETZ/MYndyCsRUgEHfQuO+8NPD8PUtuk2WiOQrBTcRkSyUKhbJ21e25l9nN+CHhZs575VJLNqwCyKLwgXvQed7YPaHXn9v+7f7Xa6IFBIKbiIi2TAzru9SmxHXtWd/Shrnv/4HI6evg7AwOP0hOP8tWDcV3ukOW1f4Xa6IFAIKbiIix9C2Zizf3tqJNjXKcO+oedz9+Vz2H0yD5pfAlV95N6Z/5wxY/YffpYpIAafgJiKSA+VKFOGDq9tx6xl1GTUrgb6v/cGKpD1QvQNc9wsUj4MP+sCcT/wuVUQKMAU3EZEcCg8z7uxRj+FXtSVpzwF6vzKJsXM3QGwtuOYnL8R9eQP8/Bikp/tdrogUQApuIiLHqXO9OL69tSMNKpXklhGzefirBRyIjIEBo6D1IJj0PHwxCA7u87tUESlgFNxERE5ApVJF+XRwe67tWJMPJq/hojcns25nCvR6Ec58EhZ9DcPOhd2b/C5VRAoQBTcRkRMUGR7GQ70a8eaA1qxM2kuvVybxy+JE6HAzXPIJJC2Bt8+ATfP9LlVECggFNxGRk9SzSUW+ubUj8WWKcs3wGTz93WJS6/aEq78Hl+7doH7J936XKSIFgIKbiEguqF62OKNu6MClbavx5oQVXPbOVBKL14PrfoWydWDEJTD5Nd0mS0ROioKbiEguiY4M56l+TXnh4ubMT9jJOS//zp+JkXDVOGjYC354AL65A9JS/C5VREKUgpuISC47v2U8X918GqWKRjLg3am88vsG0i8cDqfdDjPfh4/7w/4dfpcpIiFIwU1EJA/UqxDD1zd35LzmlXnup6VcNXwm2zo8CH1eg9W/w7tnwrZVfpcpIiFGwU1EJI8ULxLBixe34D99mzB5xVZ6vfw7s8qeC1d8CXs2e7fJWjvF7zJFJIQouImI5CEzY0D76oy6oQPh4cZFb07mvfXxuGt/hujSMPw8mDfS7zJFJEQouImI5IOm8aX45uZOdGtQnse/WcSN3+9i1xXfQ3xbGH0d/PqkrjgVkWNScBMRySelikUy9IrWPHBOA35ctJne7yxkUffh0GIATHwGvrgaUvb7XaaIBDEFNxGRfGRmDO5cm08Ht2d/ShrnvzWdzyrfizvjUVg42jt0uifR7zJFJEgpuImI+OCUGrF8e2snTqkRy32jF3D3xtM50G8YbFrg3SZr8yK/SxSRIKTgJiLik3IlijD86rbcdkZdRs9O4LxfYlnXdxSkHfS6C1n2s98likiQUXATEfFReJhxR496DL+qLVv2HKTnyN382HEExNaAT/rD1KF+lygiQUTBTUQkCHSuF8e3t3akQaWSDP5yI0/EPU9anTPhu3tg3D2Qlup3iSISBBTcRESCRKVSRfl0cHuu61STd6cnccG2G9nVcghMG+rdpD55l98liojPFNxERIJIZHgYD57biLeuaM2Krcl0nH06i1o/ASt/8+60sHGe3yWKiI8U3EREgtBZjSvyzS0dqRpbjHP+qM3H9V7CJe+Ct0+HP16C9DS/SxQRHyi4iYgEqeplizPqhg5c1q4aD84pw8DoF9lVrTv89DB80Ad2rPO7RBHJZwpuIiJBLDoynP+e35SXLmnBgu0RtFgygDHVHsCtnwVvnAbzv/C7RBHJRwpuIiIhoE+LKvx2V1euaF+Du5Y1oXfaM2wpVhNGXQOjroX9O/wuUUTygYKbiEiIKFUsksf6NOGbWzoRXb4W7TbexUfFrsAtGO3tfVv1u98likgeU3ATEQkxjSqXZOT1p/Lsxa14KaUv5x94hC3J4Iaf553/lnrA7xJFJI8ouImIhCAz4/yW8fx6VxdOOa0H3fY8wRd0hz9ewr1zBiQu9rtEEckDCm4iIiEsJjqSB89txOjbevBl/N1cc/Audm5eS/pbnWHqW+Cc3yWKSC5ScBMRKQDqVojho2vaccGl13F55Av8drARfHcvB4afD7s3+V2eiOQSBTcRkQLCzDinaSU+v7sPs057g4fTriF91R8kv9yO1IVf+12eiOQCBTcRkQKmWFQE9/RsyFW3PcFjVd5k6YEyRHx+BZs/vBYO7Pa7PBE5CQpuIiIFVM1yxXnqun4k9R/L8IgLKbf8C5KebUvSInUbIhKqFNxERAowM+OMplW5+L6hjG7+NgcOphD72XlMf/9uDhxI9rs8ETlOCm4iIoVAdGQ4/fv1hyGTmBbTnVPWvM2K/3Vi6oxpfpcmIsdBwU1EpBCJr1SRU+/+gkWnvUR8+gaajO3F8FcfY93WvX6XJiI5oOAmIlIINeoxiOhbp7A9tjkDtzzPkpd68+a4qSSnpPldmogchYKbiEghFRVblfhbfmBXl8foGj6XC6b256H/e54fFm7CqeNekaCk4CYiUpiFhVGy2+1EDJlAsTIVefbgf9g04mYGv/s7K5P2+F2diGSi4CYiIlChMcVvmkha+5sYGPET/1o3hDteHM7/vl/M3gOpflcnIgEKbiIi4omMJrznf+HKr6gek87oyH/D7y/Q49lfGTt3gw6figQBBTcRETlSra6E3/gn4Y3O477ITxnqHuV/n/7IpW9PYckm3XlBxE95FtzM7D0zSzSzBdm0m5m9bGbLzWyembXK0LbazOab2Rwzm5FheKyZ/WRmywK/y+RV/SIihVqxWOg/DM5/i8Zha/m1+IPU2vgt57w8kcfHLmJXcorfFYoUSnm5x20Y0PMo7WcDdQM/g4E3MrV3c861cM61yTDsfuAX51xd4JfAYxERyQtm0PwS7IY/iKrclP+6VxhT/l1G/Tmf05+dwKiZCaSn6/CpSH7Ks+DmnJsIbDvKKH2AD5xnClDazCodY7J9gOGBv4cDfU+6UBEROboy1WHQt3DGwzTbNYHpsY9wTokl3PX5XPq/NZkF63f6XaFIoeHnOW5VgHUZHicEhgE44Eczm2lmgzOMU8E5txEg8Lt8dhM3s8FmNsPMZiQlJeVy6SIihUxYOHS6C679majoEjy+4wF+aPQDG7bs4LxXJ/HQl/PZse+g31WKFHh+BjfLYtihfe6nOeda4R1OvcnMOh/vxJ1zQ51zbZxzbeLi4k6mThEROaRyS7h+IpxyLfVXDmdS7H+4p2UaI6ato9uz4/lk6lrSdPhUJM/4GdwSgKoZHscDGwCcc4d+JwJjgLaBcTYfOpwa+J2Yb9WKiIgnqhic+xxcNpLwfYncuORa/uiymHrli/PAmPn0fe0PZq3d7neVIgWSn8Hta+DKwNWl7YGdzrmNZlbczGIAzKw4cCawIMNzBgb+Hgh8ld9Fi4hIQL2z4IbJUPt0Kk5+jE+LPcNbfSqRuDuZfq//yb1fzGXLngN+VylSoFhedahoZiOArkA5YDPwCBAJ4Jx708wMeBXvytN9wFXOuRlmVgtvLxtABPCJc+7JwDTLAiOBasBaoL9z7mgXQADQpk0bN2PGjGONJiIiJ8I5mDkMfngAwqPYf/YLvLihIe9NWkV0ZDh39ajHgPbViQhX16EiOWFmMzP1qnG4rTD0hK3gJiKSD7Ysh9HXwYZZ0OJyVrT5N4/+sJbfl22hQcUYHuvdmHa1yvpdpUjQO1pw078/IiKSO8rVgWt+hM73wtwR1P7iLD7ons6bA1qzOzmVi4dO4bZPZ7N5V7LflYqELAU3ERHJPeGRcPqDcNX3ANiwc+iZ+A4/39aBW0+vw3cLNnH6s+N5a8IKDqam+1ysSOhRcBMRkdxXrR0MmQTNL4OJ/0fRD3tyZ6twfrqjM6fWLstT3y2m50sT+X2Z+tkUOR4KbiIikjeiS0Lf16D/cNi+Gt7qRPVVn/HOlW14b1Ab0tIdV7w7jRs+mknC9n1+VysSEnRxgoiI5L1dG+DLG2Hlb1CvJ/R+leQisbw7aRWv/LoMgBu71uGajjUpXiTC52JF/KWrShXcRET8l54O096Cnx7x9sb1fhXq92T9jv08+e0ixs3fRKmikQw8tToDO9SgbIkiflcs4gsFNwU3EZHgsXmR123I5gXQ5mo48z8QVZxZa7fz5vgV/LhoM9GRYVzUpirXdapF1dhiflcskq8U3BTcRESCS+oB+PUJ+PNVKFsb+r0NVVoBsDxxD0MnrmDM7PWkO+jVrBJDutSmYaWSPhctkj8U3BTcRESC08oJ8OUNsGczdLkfOt7udSkCbNy5n/cmreKTqWvZezCNrvXjGNKlNu1qxuLdfEekYFJwU3ATEQle+7fDN3fCwtFQrh6c+STU7QGBcLZzXwofTV3De5NWsXXvQVpULc2QLrU5s1EFwsIU4KTgUXBTcBMRCW7OwdLv4ceHYOtyqH0GnPUklG/49yjJKWl8PjOBoRNXsG7bfmrHFef6zrXp27IKURHq3UoKDgU3BTcRkdCQehCmvwMTnoYDe6DNVdD1ASh++B6nqWnpjFuwiTfHr2DRxl1ULBnNNR1rcmm7apRQVyJSACi4KbiJiISWvVth/FMw4z2IKgFd74NTroOIqL9Hcc4xcdkW3hy/gskrt1IyOoIrT63BoNNqUE5diUgIU3BTcBMRCU2Ji+GHB2DFLxBb2zt8Wq/n3+e/HTJn3Q7eHL+CHxZtIio8jP5t4hncqTbVyqorEQk9Cm4KbiIioW3ZT16A27IUanaBs/4LFZv8Y7QVSXt4e+JKRs9aT2p6Ouc2q8yQLrVoXLmUD0WLnBgFNwU3EZHQl5YCM96H8f+F5J3Q6kro9hCUiPvHqJt3JfPepFV8PHUtew6k0rleHEO61OLUWmXVlYgEPQU3BTcRkYJj3zaY8AxMfxsii0Hnu6HdEIj453ltO/en8NGUNbz/x2q27DlA8/hS3NC1Nj0aVSRcXYlIkFJwU3ATESl4tizzug9Z+j2UqeHdOqtBr3+c/wZeVyKjZiUwdOJK1mzdR61yxRncuRbnt6pCkYjw/K9d5CgU3BTcREQKruW/wA8PQtJfUL0j9PwvVGqe5ahp6Y7vFmzkzQkrWLB+F+VjinBNx5pc1q4aMdGR+Vy4SNYU3BTcREQKtrRUmDUcfnvSO5TacgCc/m+IqZDl6M45/li+lTcmLOeP5VuJiY7givbVueq0msTFqCsR8ZeCm4KbiEjhsH8HTPw/mPqWd85bpzuh/U0QGZ3tU+Yl7OCtCSsZt2AjkeFh9G8dz+DOtahetnj+1S2SgYKbgpuISOGydQX89DAs/gZKV4Mej0Ojvlme/3bIqi17GTpxJaNmJpCans7ZTStxQ5faNKmirkQkfym4KbiJiBROKyd4/b9tXgDVTvX6f6vS6qhPSdyVzHt/rObjKWvYfSCVTnXLMaRLbTrUVlcikj8U3BTcREQKr/Q0mP0h/Pof2JsEzS+DMx6GkpWO+rRdySl8MnUt705aRdLuAzSLL8WQLrU5q7G6EpG8peCm4CYiIsm74PfnYMrrEBYBHe+AU2+GqKPfFis5JY0xs9czdOJKVm3ZS81yxbmuUy36tapCdKS6EpHcp+Cm4CYiIodsW+Wd//bX11AyHno8Bk0uOOr5b+B1JfLDwk28OWEF8xJ2EhdThKtPq8nl7atRUl2JSC5ScFNwExGRzFZPgu//BZvmQXxb6PkUxGf5XXkE5xyTV2zljQkr+H3ZFmKKRHB5++pcfVoNypfM/upVkZxScFNwExGRrKSnwdwR8MvjsGczNL0Iuj8CpeJz9PQF63fy5oQVjJu/kYiwMC4IdCVSs5y6EpETp+Cm4CYiIkdzYDdMegH+fBUsDE67DU67FaJyFsDWbPW6Evl8ZgIpaemc3aQiQ7rUpll86bytWwokBTcFNxERyYnta+DnR2HhaIip7O19a3oRhIXl6OlJuw/w/h+r+HDKGnYnp3JanbIM6VKbjnXKqSsRyTEFNwU3ERE5HmunwPf3w4bZULkV9HwaqrXL8dN3J6cwYtpa3vl9FYm7D9CkSkmGdKnN2U0qqSsROSYFNwU3ERE5XunpMO8z+OUx2L3Ru/K0+6PenRhy6EBqGl/OXs9bE1aycsteKpQsQt+WVbigVTz1KsTkXe0S0hTcFNxEROREHdwLf7wEf7wMOK/vt453QJESOZ5EWrrj57828/mMBMYvSSQ13dGkSkn6tYynd4vKlCuhG9vLYQpuCm4iInKydibAz4/B/JFQogKc8Qg0vzTH578dsnXPAcbO3cDo2euZl7CT8DCja704+rWK54yG5dWpryi4KbiJiEiuWTfdO/9t/Qyo1BzOegpqnHZCk1q2eTejZ69nzKz1bNqVTEx0BL2aVeaCVlVoXb2MLmgopBTcFNxERCQ3OQfzv/CuQN2VAI36QI/HoUyNE5pcWrpjysqtjJqVwPcLNrHvYBrVYovRr1UV+rWMp1rZo9+WSwoWBTcFNxERyQsH98HkV70+4NJTof2N0OkuiC55wpPceyCVHxZuYvSs9fyxYgvOwSk1ytCvVTznNK1EqaK6vVZBp+Cm4CYiInlp1wbv7gtzR0DxODj939ByAISd3PlqG3fu58vZGxg1K4HliXuIigijR6MKXNCqCp3qxhEZfnzn10loUHBTcBMRkfywfiZ8/wCsmwIVmkLP/0LNzic9WeccC9bvYtSsBL6eu4Ftew9SrkQUvZtXoV+rKjSuXFLnwxUgCm4KbiIikl+cg4Vj4KdHYOdaaNDLO/+tbO1cmXxKWjoTliQxenYCPy9K5GBaOvUrxNCvVRX6tqxCBd3oPuQpuCm4iYhIfkvZD1Neh9+fh9QD0O567/y3YrG59hI796XwzfwNjJ61nplrthNmcFqdclzQKp4zG1egWFRErr2W5B8FNwU3ERHxy+5N8OsTMPtjiCgCjc+H1ldB1baQi4c3V23Zy5hZCYyevZ6E7fspHhXO2U0r0a9VFdrXLEuYbrUVMhTcFNxERMRvmxfCtLdh/udwcA+UbwStB0Gzi6Fo6Vx7mfR0x/TV2xgzez3fztvI7gOpVC4VzfmtqnB+y3jqlM/5HR/EHwpuCm4iIhIsDuyBBV/AzGHeTewjinp74dpcBfGn5OpeuOSUNH5atJnRsxKYuGwLaemO5lVLc0GrKvRqVpnY4lG59lqSexTcFNxERCQYbZgDM9/3OvM9uAfKNw7shbsoV/fCASTuTubrOd75cIs27iIy3OhWvzz9WsXTrUEcRSJ0q61goeCm4CYiIsHswG4vvM18HzbO9fbCNennnQsX3yZX98IB/LVxF2Nmr2fM7PUk7T5A6WKRnNesMv1aVaFF1dLqWsRnCm4KbiIiEio2zIYZgb1wKXuhQpPDe+GiS+XqS6WmpTNp+RZGz1rPDws3cSA1nVrliv/dtUh8Gd1qyw8KbgpuIiISag7s9i5kmPE+bJoHkcUO74Wr0jrX98LtTk7hu/mbGDUrgamrtgHQvlYs/VrFc3aTisRE61Zb+UXBTcFNRERClXPeXriZ78P8UYG9cE2hzSBo2j/X98IBrNu2jy9nr2f07PWs2rKX6MgwzmpckX6t4jmtdlkidKutPKXgpuAmIiIFQfIuby/czPdh0/zAXrgLvCtSK7fK9b1wzjlmr9vB6FkJjJ27kZ37UygfU4S+Lb1bbTWoWDJXX088Cm4KbiIiUpA4BxtmeYdRF4yClH1Qsal3GLVpf4jO/UB1IDWN3xYnMmrWen5bnEhquqNRpZL0a1WF3i0qUz5Gt9rKLQpuCm4iIlJQJe8MnAs3DDbPh8ji0PSCwLlwrfLkJbftPcjYuRsYPSuBuQk7CQ8zOtctR9+WVehSL47SxdQ/3MlQcFNwExGRgs45WD8LZr4HC0Z7e+EqNfeuSG3aH4rE5MnLLk/czehZXtciG3cmE2bQqloZujUoT7f65WlYKUbdixwnBTcFNxERKUySd8K8kd7dGTYvCOyFuzBwLlzLPHnJ9HTHnIQdjF+cyG9Lkpi/ficAFUtG061BHF3rl+e0OuUoUUQ3vj8WBTcFNxERKYycg/UzD58Ll7ofKrUI7IW7MM/2wgEk7kpm/NIkxi9J5PelW9h9IJXIcKNtzVi61S9PtwblqVWuuPbGZUHBTcFNREQKu/07DvcLl7gQokp44a31VVC5RZ6+dEpaOjNWb2f8kkR+W5LI0s17AKgWW4zTG5Sna/042tcqS3SkbrsFCm4KbiIiIoc4BwkzvC5FFoz29sJVbunthWtyIRQpkeclrNu2z9sbtziRP1ZsITklnejIMDrULhc4Ny6uUN+1QcFNwU1EROSf9u8InAv3PiQugqgYaNbfC3GVmudLCckpaUxZuZXxS5L4dXEia7ftA6Bu+RJ/X+DQpkYZIgtRp78KbgpuIiIi2XMOEqZ7h1EXjobUZK9D3zZXQeN++bIXzivDsXLLXn5bnMj4JUlMXbWVlDRHTJEIOtb19sZ1rRdH+ZIFu884BTcFNxERkZzZv93bCzfjfUj6K7AX7iIvxFVsmq+l7DmQyh/Lt3jnxi1OYtOuZACaVCnJ6fXL07VBeZrHlyY8rGBd4OBLcDOz94BeQKJzrkkW7Qa8BJwD7AMGOedmmVlV4AOgIpAODHXOvRR4zqPAdUBSYDIPOOfGHasWBTcREZHj5Bysm+p1KbJwjLcXrkpr72KGJv0gqng+l+P4a+NufluSyG+LE5m1djvpDmKLR9GlXhxd68cVmM5//QpunYE9wAfZBLdzgFvwgls74CXnXDszqwRUCoS4GGAm0Nc5tygQ3PY45549nloU3ERERE7Cvm2Hz4VLWgxFSnp74VpfBRX/8RWfL3bsO8jEZVsYvziR8UuT2Lb3IGEGLauV+ftK1UaVSoZkdyO+HSo1sxrAN9kEt7eA8c65EYHHS4CuzrmNmcb7CnjVOfeTgpuIiIiPnIO1Uw7vhUs7APGneBczNO4HUf5cCZqW7piXsIPfliTx2+LEvzv/rVCyCN3ql6dr/fJ0rBs6nf8Ga3D7BnjaOTcp8PgX4D7n3IxMz58INHHO7QoEt0HALmAGcJdzbns2rz0YGAxQrVq11mvWrMm9GRMRESns9m2DuZ96IW7LEihSChqeBxUaQ1w9KFcfSlaBsPy/GjRxdzITliTxW6bOf0+pERvYG1ee2nHB2/lvsAa3b4GnMgW3e51zMwOPSwATgCedc6MDwyoAWwAHPIF3SPXqY9WhPW4iIiJ5xDlYO9m7mGH5T97FDYdEFoNydb0QF1cPygUCXWwtiMifc9FS0tKZuWb73+fGHer8t2ps0b/v4HBqkHX+G6zBLdtDpWYWCXwD/OCce/54p52ZgpuIiEg+cA72bvH2wG1ZCklLA38vg53rDo9n4V54K1fv8N65cvW8kBddMk9LTNi+j/FLvFtx/bF8K/tT0igSEUaH2mX/3htXNdbfzn+DNbidC9zM4YsTXnbOtQ1cbToc2Oacuz3TcyodOgfOzO4A2jnnLjlWHQpuIiIiPjuwB7YuC4S5QKBLWgrbVkB66uHxYipn2DtXD+Lqe8GuRHnI5UObySlpTF21jd8We7fiWrPV6/y3TvkSf1/g0KZ6LFER+Xu416+rSkcAXYFywGbgESASwDn3ZiCgvQr0xOsO5Crn3Awz6wj8DszH6w4EAt1+mNmHQAu8Q6WrgeszX8yQFQU3ERGRIJWWAttXQ9KSw2FuS+Dn4J7D40WXOnyoNWOwK1MDwk7+MKdzjlVb9v59gcOhzn9LFImgU91ygYsc8qfzX3XAq+AmIiISWpyDXRsOh7ikJYf/3rP58HjhRaBsHe8wa1z9w3vpytaByKIn/PJ7A53/Hgpyhzr/feHi5pzfMv5k5+6oFNwU3ERERAqO/du98+YyhrmkJbBjDbhDB+sMSlc7HOb+PuxaD4rFHtfLOedYvMnr/Ld388rEl8nbc+COFtxCo0MTERERkUOKloGqbb2fjFKSvXPmjgh0S2HVRO/OD4cUK3fk3rlDV76Wis/yPDozo2GlkjSslLcXTuSEgpuIiIgUDJHRXj9yFRofOTw9DXas9fbSbVlyONgtHAPJOzI8v3ggxGW42jWuPpSpmW/dlxyLgpuIiIgUbGHhEFvT+6l35uHh2XVfsuZPmD8yw/MjvPAWVx863ALV2uf/PAQouImIiEjhZAYl4ryfGh2PbMuu+5KUff7UGqDgJiIiIpJZkRJQuaX3E0Ty/wZiIiIiInJCFNxEREREQoSCm4iIiEiIUHATERERCREKbiIiIiIhQsFNREREJEQouImIiIiECAU3ERERkRCh4CYiIiISIhTcREREREKEgpuIiIhIiFBwExEREQkRCm4iIiIiIULBTURERCREKLiJiIiIhAgFNxEREZEQoeAmIiIiEiIU3ERERERChIKbiIiISIhQcBMREREJEQpuIiIiIiFCwU1EREQkRCi4iYiIiIQIc875XUOeM7MkYE0ev0w5YEsev4YcPy2X4KNlEpy0XIKPlklwyo/lUt05F5dVQ6EIbvnBzGY459r4XYccScsl+GiZBCctl+CjZRKc/F4uOlQqIiIiEiIU3ERERERChIJb7hnqdwGSJS2X4KNlEpy0XIKPlklw8nW56Bw3ERERkRChPW4iIiIiIULBTURERCREKLgdJzPraWZLzGy5md2fRbuZ2cuB9nlm1sqPOguTHCyTywPLYp6Z/Wlmzf2os7A51nLJMN4pZpZmZhfmZ32FUU6WiZl1NbM5ZrbQzCbkd42FUQ62YaXMbKyZzQ0sl6v8qLMwMbP3zCzRzBZk0+7bd72C23Ews3DgNeBsoBFwqZk1yjTa2UDdwM9g4I18LbKQyeEyWQV0cc41A55AJ/zmuRwul0Pj/Q/4IX8rLHxyskzMrDTwOtDbOdcY6J/fdRY2Ofys3AQscs41B7oCz5lZVL4WWvgMA3oepd2373oFt+PTFljunFvpnDsIfAr0yTROH+AD55kClDazSvldaCFyzGXinPvTObc98HAKEJ/PNRZGOfmsANwCjAIS87O4Qiony+QyYLRzbi2Ac07LJe/lZLk4IMbMDCgBbANS87fMwsU5NxHvfc6Ob9/1Cm7HpwqwLsPjhMCw4x1Hcs/xvt/XAN/laUUCOVguZlYFOB94Mx/rKsxy8lmpB5Qxs/FmNtPMrsy36gqvnCyXV4GGwAZgPnCbcy49f8qTbPj2XR+RHy9SgFgWwzL3p5KTcST35Pj9NrNueMGtY55WJJCz5fIicJ9zLs3bkSB5LCfLJAJoDZwBFAUmm9kU59zSvC6uEMvJcjkLmAOcDtQGfjKz351zu/K4Nsmeb9/1Cm7HJwGomuFxPN5/QMc7juSeHL3fZtYMeAc42zm3NZ9qK8xyslzaAJ8GQls54BwzS3XOfZkvFRY+Od1+bXHO7QX2mtlEoDmg4JZ3crJcrgKedl7Hq8vNbBXQAJiWPyVKFnz7rteh0uMzHahrZjUDJ4ZeAnydaZyvgSsDV5y0B3Y65zbmd6GFyDGXiZlVA0YDV2jPQb455nJxztV0ztVwztUAvgBuVGjLUznZfn0FdDKzCDMrBrQD/srnOgubnCyXtXh7QTGzCkB9YGW+VimZ+fZdrz1ux8E5l2pmN+NdARcOvOecW2hmQwLtbwLjgHOA5cA+vP+UJI/kcJk8DJQFXg/s3Ul1zrXxq+bCIIfLRfJRTpaJc+4vM/semAekA+8457LsDkFyRw4/K08Aw8xsPt4huvucc1t8K7oQMLMReFfwljOzBOARIBL8/67XLa9EREREQoQOlYqIiIiECAU3ERERkRCh4CYiIiISIhTcREREREKEgpuIiIhIiFBwExE5TmZW2sxu9LsOESl8FNxERI5faUDBTUTynYKbiMjxexqobWZzzOz//C5GRAoPdcArInKczKwG8I1zronftYhI4aI9biIiIiIhQsFNREREJEQouImIHL/dQIzfRYhI4aPgJiJynJxzW4E/zGyBLk4QkfykixNEREREQoT2uImIiIiECAU3ERERkRCh4CYiIiISIhTcREREREKEgpuIiIhIiFBwExEREQkRCm4iIiIiIeL/Aelm2l1vRCaZAAAAAElFTkSuQmCC\n",
|
|
"text/plain": [
|
|
"<Figure size 720x720 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"filenames": {
|
|
"image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter11_66_3.png"
|
|
},
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAm4AAAJNCAYAAACIkPmLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAACHBElEQVR4nOzdZ3RV1cKF4XelJ5CEFmoCoYTeCb33piIgTVRAFLBg7+UiKlbAgr0BKgIqIChIExGkh957Cy0hgSSkl31/nIigVDkn5ySZzxgMkl2njvt9d96991rLWJaFiIiIiLg+N2cHEBEREZFro+ImIiIikkuouImIiIjkEipuIiIiIrmEipuIiIhILqHiJiIiIpJLeDg7QE4oVqyYFRoa6uwYIiIiIle1fv3605ZlBV1qX74obqGhoURERDg7hoiIiMhVGWMOX26fXpWKiIiI5BIqbiIiIiK5hIqbiIiISC6RL75xExERkRuXnp5OZGQkKSkpzo6SJ/j4+BAcHIynp+c1n6PiJiIiItckMjISf39/QkNDMcY4O06uZlkWMTExREZGUr58+Ws+T69KRURE5JqkpKRQtGhRlTY7MMZQtGjR6356qeImIiIi10ylzX7+y79LFTcRERHJs1566SXGjh3r7Bh2o+ImIiIiki0jI8PZEa5IxU1ERERylW+//ZZGjRpRt25dhg8fTmZmJgULFjy//8cff2Tw4MH/Om///v106dKFBg0a0LJlS3bt2gXA4MGDeeyxx2jbti1PP/10Tv1j/CcaVSoiIiK5xs6dO5k+fTorVqzA09OT+++/nylTplzTucOGDeOTTz4hLCyMNWvWcP/997NkyRIA9uzZw+LFi3F3d3dk/Bum4iYiIiLXbfTP29lxPN6u16xeOoBRN9e44jG//fYb69evp2HDhgAkJydTvHjxq1773LlzrFy5kj59+pzflpqaev7nPn36uHxpAxU3ERERyUUsy2LQoEG8/vrrF20fN27c+Z8vNcVGVlYWhQoVYtOmTZe8boECBeya01FU3EREROS6Xe3JmKO0b9+eHj168Oijj1K8eHFiY2NJSEigRIkS7Ny5kypVqjBr1iz8/f0vOi8gIIDy5cvzww8/0KdPHyzLYsuWLdSpU8cp/xz/lQYniIiISK5RvXp1Xn31VTp16kTt2rXp2LEjJ06c4I033uCmm26iXbt2lCpV6pLnTpkyhS+//JI6depQo0YNZs+encPpb5yxLMvZGRwuPDzcioiIcHYMERGRXG3nzp1Uq1bN2THylEv9OzXGrLcsK/xSx+uJm4iIiEguoeImIiIikkuouImIiIjkEg4rbsaYr4wxUcaYbZfZP9AYsyX7z0pjTJ0L9nUxxuw2xuwzxjxzwfYixphFxpi92X8XdlR+EREREVfjyCduk4AuV9h/EGhtWVZt4BXgMwBjjDvwIdAVqA4MMMZUzz7nGeA3y7LCgN+yfxcRERHJFxxW3CzLWgbEXmH/SsuyzmT/uhoIzv65EbDPsqwDlmWlAdOAHtn7egCTs3+eDNxq79z/VXpa6tUPEhEREbkBrvKN21Dg1+yfywBHL9gXmb0NoIRlWScAsv+++hoXOeCd6SPp9nUDzsafdnYUERERuQaTJk3iwQcfvOoxx48fP//7Pffcw44dO677XkuXLuWmm2667vMuxenFzRjTFltxe/qvTZc47LonmzPGDDPGRBhjIqKjo28k4lWVLlKFk56GL3991qH3ERERkZzzz+L2xRdfUL169Suc4XhOLW7GmNrAF0APy7JisjdHAiEXHBYM/PVv7ZQxplT2uaWAqMtd27KszyzLCrcsKzwoKMj+4S9wW9v7qJRqWJiwisyMNIfeS0REJL+79dZbadCgATVq1OCzzz4DoGDBgjz//PPUqVOHJk2acOrUKQB+/vlnGjduTL169ejQocP57X9JSEigfPnypKenAxAfH09oaCg//PADERERDBw4kLp165KcnEybNm34a0L/+fPnU79+ferUqUP79u0BWLt2Lc2aNaNevXo0a9aM3bt32/2f3WnFzRhTFpgJ3GlZ1p4Ldq0Dwowx5Y0xXkB/YE72vjnAoOyfBwEusVaFu4c7TQt25LinYebSN5wdR0REJE/76quvWL9+PREREbz//vvExMSQmJhIkyZN2Lx5M61ateLzzz8HoEWLFqxevZqNGzfSv39/3nrrrYuu5e/vT5s2bZg7dy4A06ZNo3fv3vTp04fw8HCmTJnCpk2b8PX1PX9OdHQ09957LzNmzGDz5s388MMPAFStWpVly5axceNGXn75ZZ577jm7/7M7bJF5Y8xUoA1QzBgTCYwCPAEsy/oE+B9QFPjIGAOQkf2ELMMY8yCwAHAHvrIsa3v2Zd8AvjfGDAWOAH0clf96DezyP36bOZ9ph2Zym/Ui2f9MIiIiedOvz8DJrfa9Zsla0PXqD0Def/99Zs2aBcDRo0fZu3cvXl5e578ja9CgAYsWLQIgMjKSfv36ceLECdLS0ihfvvy/rnfPPffw1ltvceuttzJx4sTzpe9yVq9eTatWrc5fq0iRIgDExcUxaNAg9u7dizHm/FM8e3JYcbMsa8BV9t8D3HOZffOAeZfYHgO0t0tAOytTJJBaGXVY4LuVNVu+pkmdQVc/SURERK7L0qVLWbx4MatWrcLPz482bdqQkpKCp6fn+Ycm7u7uZGRkADBy5Egee+wxbrnlFpYuXcpLL730r2s2b96cQ4cO8ccff5CZmUnNmjWvmMGyrEs+oHnxxRdp27Yts2bN4tChQ7Rp0+aG/3n/yWHFLT/q0PhF1my8jS82fKziJiIieds1PBlzhLi4OAoXLoyfnx+7du1i9erVVz2+TBnb5BSTJ0++7HF33XUXAwYM4MUXXzy/zd/fn4SEhH8d27RpUx544AEOHjxI+fLliY2NpUiRIhfda9KkSf/hn+7qnD6qNC/pULsK1ROCWeOWyL5Dvzs7joiISJ7TpUsXMjIyqF27Ni+++CJNmjS54vEvvfQSffr0oWXLlhQrVuyyxw0cOJAzZ84wYMDfLwwHDx7MiBEjzg9O+EtQUBCfffYZvXr1ok6dOvTr1w+Ap556imeffZbmzZuTmZl5g/+kl2Ys67pn2sh1wsPDrb9GgTjauJ8WMO3MY7T3Ks0bAxflyD1FRERyws6dO6lWrZqzYzjEjz/+yOzZs/nmm29y9L6X+ndqjFlvWVb4pY7Xq1I7u61lC7Z9W4gFgSd49PROShTLm/8BFxERyStGjhzJr7/+yrx5//q83uXoVamdlStaAD+vQVjAt8tGOTuOiIiIXMWECRPYt28flStXdnaUq1Jxc4BOTbtT+5w3P8TtICHRsas2iIiISP6h4uYAnWuUJC2hC4luhh+X66mbiIiI2IeKmwP4eLpTrcYAaiQZvjmxnPT0VGdHEhERkTxAxc1B+jUqi1tME6LdYO5qLYMlIiIiN07FzUGqlgwgudBAKqRlMWn/T2RZWc6OJCIikue89NJLjB071tkxcoyKmwP1b1yeojHV2W8y+HPTl86OIyIiIrmcipsD3VS7NFuS+lMiI4uvtqq4iYiI2MOYMWOoUqUKHTp0YPfu3QC0adOGp59+mkaNGlG5cmWWL18O2Jae6tWrF126dCEsLIynnnrKmdFvmIqbAxXw9qBDnTAqnS3LeiuRLft+dXYkERGRXG39+vVMmzaNjRs3MnPmTNatW3d+X0ZGBmvXruXdd99l9OjR57dv2rSJ6dOns3XrVqZPn87Ro0edEd0utHKCg/VrGMK962/Dv/B4Jq0by/hKXZ0dSURE5Ia9ufZNdsXusus1qxapytONnr7iMcuXL6dnz574+fkBcMstt5zf16tXLwAaNGjAoUOHzm9v3749gYGBAFSvXp3Dhw8TEhJi1+w5RU/cHKxuSCGKFQ+jUVIxFqee4sipzc6OJCIikqsZYy653dvbGwB3d3cyMjL+tf1S+3IbPXFzMGMM/RqG8PWvPfCo9BWTV7zMi71mODuWiIjIDbnakzFHadWqFYMHD+aZZ54hIyODn3/+meHDhzslizPoiVsO6FmvDMeoRqvUgsyO301MwjFnRxIREcmV6tevT79+/ahbty69e/emZcuWzo6Uo4xlWc7O4HDh4eFWRESEUzM8PG0jUft/YnuZGQwr3oQHu33h1DwiIiLXa+fOnVSrVs3ZMfKUS/07Ncastywr/FLH64lbDunXMIRVCQ1pnu7JtFNrSEo95+xIIiIiksuouOWQJuWLUq5oAQqltCPODWates3ZkURERCSXUXHLIW5uhr7hIXx/rDV10i2+OTSPjKzcO6pFREREcp6KWw7q0yAY3Dxo4N6MYyaTRRs+cXYkERGR65Ifvo3PKf/l36WKWw4qHuBDu6rFmRHZjdCMTCbu+Fr/ByAiIrmGj48PMTEx+u8uO7Asi5iYGHx8fK7rPM3jlsP6Nwxh6I5T3OFdhw8zt7Fmz080qdLT2bFERESuKjg4mMjISKKjo50dJU/w8fEhODj4us5RccthrSsHUTLAh+3JAyjq/gyT1r+n4iYiIrmCp6cn5cuXd3aMfE2vSnOYh7sbfcKDmXvAnb7e5VmRHsPu42udHUtERERyARU3J+gbHkKWBZ5+Q/HNymLSyledHUlERERyARU3Jwgp4keLSsWYtLsQt7kXZf65A5w4e8jZsURERMTFqbg5Sf9GIRw7m0yDkHuwgG/+fMnZkURERMTFqbg5ScfqJSjs58mMY9XpmuXNjNPriUs+6+xYIiIi4sJU3JzE28OdXvWDWbQzitsq30WSgR9WjnF2LBEREXFhKm5O1L9hCOmZFhszutI8DaYcXUhqZqqzY4mIiIiLUnFzorAS/jQoV5jv1h9nUIWbOW2y+CXiA2fHEhERERel4uZk/RqGcCA6Ec+Q4VRLz2TS7qlkWVnOjiUiIiIuSMXNyW6qXYqC3h5M3XSWIcWbcshKZenO6c6OJSIiIi5Ixc3J/Lw8uKVuaeZuPU7jRi9QJiODSRs/cnYsERERcUEqbi6gf8MQUtKzmHvIjTsLVmFjxlk2HVnm7FgiIiLiYlTcXECtMoFUKxXA9HVH6NlyFIGZmUxc84azY4mIiIiLUXFzAcYYBjQKYduxeA5YFejvUZzfE49wMGaXs6OJiIiIC1FxcxE96pTB28ONaeuOMKDxk3hZFpNXvOLsWCIiIuJCVNxcRKCfJ91qlWL2xuP4hXamR5YPc2K3cDoxytnRRERExEWouLmQ/g1DSEjNYN62k9xVexgZWHynZbBEREQkm4qbC2lUvgjlixVg+rqjlKs/lA7pMO3Y7ySmJzo7moiIiLgAFTcXYoyhX8MQ1h6KZV9MCoMr9ibBWMxc956zo4mIiIgLUHFzMb3rB+PhZvg+4ii1mz9Bg9QMvt73I+lZ6c6OJiIiIk6m4uZigvy96VCtBDPWR5LmXoAhpVpy0kpnwbZvnR1NREREnEzFzQX1axRCTGIai3eeomWrUVRMS2fils+xLMvZ0URERMSJVNxcUKuwIEoH+jBt3VHcAsswKLA6ezITWHVwobOjiYiIiBOpuLkgdzdDn/AQlu+NJvJMEt1bjqJ4RgZfrRvr7GgiIiLiRCpuLqpPeDAAP0RE4lWqDgM9S7Im5SQ7Tm1ybjARERFxGhU3FxVc2I+WYUH8EHGUzCyLPk2epkBWFpNWaUJeERGR/ErFzYX1bxjC8bgUlu2Nxj+sM30yfVl4dheRcUecHU1EREScQMXNhXWoVoKiBbyYvvYoGMPAevdjsPhm9WvOjiYiIiJOoOLmwrw83OjdIJjFO08RnZBKybp30i0NZp1YydmUs86OJyIiIjlMxc3F9Q0PISPLYsaGSHD3ZHDlfiQbi+nr3nF2NBEREclhKm4urlLxgjQKLcL0dUexLIuwpo/QMiWd7w7MISUjxdnxREREJAepuOUC/RqGcPB0ImsPxoK3P0PKtCOWDOZsnejsaCIiIpKDVNxygW61SuHv7cG0dUcBCG/5PDVT05i8fRKZWZlOTiciIiI5xWHFzRjzlTEmyhiz7TL7qxpjVhljUo0xT1ywvYoxZtMFf+KNMY9k73vJGHPsgn3dHJXflfh6udOjXmnmbT1BXFI6JrAMQwrV5khmEr/v+9nZ8URERCSHOPKJ2ySgyxX2xwIPARet42RZ1m7LsupallUXaAAkAbMuOOSdv/ZbljXPvpFdV/+GZUnNyGL25mMAtG81ipD0dCZueE+Lz4uIiOQTDitulmUtw1bOLrc/yrKsdUD6FS7THthvWdZhe+fLbWqWCaRmmQCmrrUNUnAvWZNBXmXYknqaDSfWODueiIiI5ABX/8atPzD1H9seNMZsyX4VW9gZoZylX8Oy7DwRz9ZjcQD0aPo0hTMzmbj6DScnExERkZzgssXNGOMF3AL8cMHmj4GKQF3gBDDuCucPM8ZEGGMioqOjHRk1x/SoWxofT7fzgxR8KnVkQJYffyTsZ3/sXienExEREUdz2eIGdAU2WJZ16q8NlmWdsiwr07KsLOBzoNHlTrYs6zPLssItywoPCgrKgbiOF+DjSfdapZmz6ThJaRlgDP3rj8QnK4tJq193djwRERFxMFcubgP4x2tSY0ypC37tCVxyxGpe1r9RCOdSM/hlywkACte5nZ6phl+i1hGVFOXkdCIiIuJIjpwOZCqwCqhijIk0xgw1xowwxozI3l/SGBMJPAa8kH1MQPY+P6AjMPMfl33LGLPVGLMFaAs86qj8riq8XGEqBhVgevbrUtw9uava7WRh8e3ay745FhERkTzAw1EXtixrwFX2nwSCL7MvCSh6ie132idd7mWMoX/DsoyZt5O9pxIIK+FPcJORdNo5mR8OL2BY2osU9Cro7JgiIiLiAK78qlQuo2f9Mni6m7+funn7MzikE+fI5MfNnzs3nIiIiDiMilsuVKygNx2rl2DGhkhSM2xLXtVo+SyNk1P5Ztd3pGdeaWo8ERERya1U3HKp/g3LciYpnUU7sgfdBpRmcNH6RGWlMG/3D1c+WURERHIlFbdcqkWlYpQp5Pv361KgecsXCEtLY9Kmj7UMloiISB6k4pZLubkZ+oaHsHzvaY7GJgFgStZkiHcI+9LPsvzIEicnFBEREXtTccvF+oQH42bg+4i/n7p1afoMJTMymLROU4OIiIjkNSpuuVjpQr60rhzEDxGRZGRmAeBZqT13WAVZl3iUbVFbnJxQRERE7EnFLZfr17AsJ+NTWLY3ez1WY7itwUP4Z2Yxce1bzg0nIiIidqXilsu1r1acYgW9mbr279elBWr3p2+aYfHpzRyNP3qFs0VERCQ3UXHL5Tzd3bitQTBLdkURFZ9i2+juycDqd+KOxWR96yYiIpJnqLjlAf0ahpCZZfHjhsjz24Ia3cfNSWn8FLmE2JRYJ6YTERERe1FxywPKFytA4/JFmL7uKFlZ2fO3+QQwqFw3UrGYtvET5wYUERERu1BxyyP6NwrhcEwSqw/GnN9WoeVTtElKZureH0nOSHZiOhEREbEHFbc8omvNUgT4eFy0kgIBpRlSrBFnrXR+2vGd88KJiIiIXai45RE+nu70rFeGX7ed5GxS2vnt9Vo8S52UVL7e+iUZWRlOTCgiIiI3SsUtD+nXsCxpGVnM2njs/DZTqhZDfMsRmZHA4oPznZhOREREbpSKWx5SvXQAdYIDmbb26EWLzLdp9gyhaelMXP+eFp8XERHJxVTc8ph+Dcuy+1QCmyPjzm9zr9iOu6wAdiSfZN3JtU5MJyIiIjdCxS2PublOKXw93Zm29sjfG43hloYPUyQzk4lrxzovnIiIiNwQFbc8xt/Hk5vrlGLO5uOcS/17MIJ37b4MTDX8eXYXe87scWJCERER+a9U3PKgfg3LkpSWydwtx//e6O5Jv5qD8c3KYvK68c4LJyIiIv+ZilseVL9sIcKKF2TauosXmA9sOJzeSenMO7GSk4knnZRORERE/isVtzzIGEO/hiFsPHKW3ScT/t7hE8Cd5W/GsrL4duNHzgsoIiIi/4mKWx7Vq34wXu5uTFt35KLtpZs/RuekZH44MIf4tHgnpRMREZH/QsUtjypSwItONUowa+MxUtIz/94RWIYhQU1JsjL5YdvXzgsoIiIi103FLQ/r37AsZ5PSWbjj1EXbq7Z8mqbJyXy742vSMtMuc7aIiIi4GhW3PKxZxaKEFPG9eE43gJK1GOJXkdOZyczd95NTsomIiMj1U3HLw9zcDP3CQ1i5P4bDMYkX7WvS7CmqpqYxceNHZFlZTkooIiIi10PFLY+7rUEIbga+j7h4ahBTsR1DCORgagzLjv7hpHQiIiJyPVTc8riSgT60rVKcHyIiyci84MmaMXRq9Cil0zOYGPGO8wKKiIjINVNxywf6NypLVEIqv++Ovmi7R63buCvVsCHhIJuiNjknnIiIiFwzFbd8oG2VIIr7ezP9H3O64e5Jz9pDCcjMZNL695wTTkRERK6Zils+4OHuxm0NglmyK4qTcSkX7fNreA/9E9NYEhXBobhDzgkoIiIi10TFLZ/o1zCELAt+XH/xIAV8AhlQ6VY8LYvJGz90TjgRERG5Jipu+US5ogVoVrEo0yOOkpVlXbSvWLNH6HEuiTmHF3I6+bSTEoqIiMjVqLjlI/0ahnA0NplVB2Iu3hEYzKCSzUm3Mvlu60TnhBMREZGrUnHLRzrXKEkhP0+m/nMlBaBci6don5TM9N3TSEpPckI6ERERuRoVt3zEx9OdnvXKsHD7KWIT/7FGaclaDC4QRnxWGjN3f++cgCIiInJFKm75TL+GIaRlZjFr47F/7avT/Enqp6Tw9ZbPSc9Kd0I6ERERuRIVt3ymaskA6oYUYvq6I1jWxYMUqNiOIaYwJ9LjWXhwgXMCioiIyGWpuOVDAxqFsOfUOTYcOXvxDmNo1ehRKqSlM2njB/8udiIiIuJUKm750E21S1PAy/3fKykAbjV7MzjVjV2Jx1h9YrUT0omIiMjlqLjlQwW8Pbi5Tml+3nyChJR/fMvm4UX3uvcQlJHBxPXvOyegiIiIXJKKWz7Vr2EIyemZ/Lz5xL/2eYUPZWBiGqtit7ErdpcT0omIiMilqLjlU3VDClG1pP8lX5fiE0ifsN74ZWUxUctgiYiIuAwVt3zKGEO/hiFsjoxjx/H4f+0PaPYQfRKSWBD5B8fPHXdCQhEREfknFbd8rGe9Mnh5uPF9xNF/7wwM5o7SrTBWFt9s+SLnw4mIiMi/qLjlY4X8vOhasyQzN0SSkp75r/0lmz9Ot3OJzNg3i7jUOCckFBERkQupuOVz/RqGEJ+SwfxtJ/+9s1RtBvlXJdnKYPrO73I+nIiIiFxExS2fa1K+KOWK+jHtUoMUgMrNn6BFUjJTtk8iNTM1h9OJiIjIhVTc8jk3N0Pf8BBWH4jl4OnEfx9QsR1DTBFiM5L4ftf0nA8oIiIi56m4CX0aBOPuZpi+7hKDFIyhYeNHaJ6UzIQN72mEqYiIiBOpuAnFA3xoV7U4P66PJD0z61/7Ta3b+F96AchM4+WVo7WGqYiIiJOouAkA/RuGcPpcKkt2Rf17p4cXpbuO5eHYWFacWMkvB37J+YAiIiKi4iY2rSsHUTLAh2lrLz1IgbCO9C/dltqp6by19g1iU2JzNqCIiIiouImNh7sbfcKD+WNPNMfPJl/yGPeub/DymUQS0xJ4Y+0bOZxQREREVNzkvL7hIWRZ8OP6yEsfEBhMxRZPcO+ZM/x68FeWRS7L2YAiIiL5nIqbnBdSxI+WYcWYvu4oWVmXGYDQ5H7u8SxFpUx4edVozqWdy9mQIiIi+ZiKm1ykX8MQjp1N5s99py99gLsnnt3HM/rUSaKSonh3w7s5mk9ERCQ/c1hxM8Z8ZYyJMsZsu8z+qsaYVcaYVGPME//Yd8gYs9UYs8kYE3HB9iLGmEXGmL3Zfxd2VP78qmP1EhT287z0nG5/CW1B7Sq9GBifyPTd09lwakPOBRQREcnHHPnEbRLQ5Qr7Y4GHgLGX2d/Wsqy6lmWFX7DtGeA3y7LCgN+yfxc78vZwp1f9YBbuOEnMuSsscdXpFUYmZlAGD0atHKXlsERERHKAw4qbZVnLsJWzy+2PsixrHZB+HZftAUzO/nkycOt/DiiX1b9hCOmZFjM3HLv8QQWL49fuRf538jiH4g/x6eZPcy6giIhIPuWq37hZwEJjzHpjzLALtpewLOsEQPbfxZ2SLo8LK+FPg3KFmbbuyJVXSQi/m2aFqnBLSiYTt33F7tjdORdSREQkH3LV4tbcsqz6QFfgAWNMq+u9gDFmmDEmwhgTER0dbf+EeVy/hiHsj05k/eEzlz/IzR26v8NTp04SYNz538r/kZGVkXMhRURE8hmXLG6WZR3P/jsKmAU0yt51yhhTCiD770usz3T+Gp9ZlhVuWVZ4UFCQoyPnOTfVLkVBbw+mXWmQAkBwAwLr3cWzJ0+yI2YH3+74NmcCioiI5EMuV9yMMQWMMf5//Qx0Av4amToHGJT98yBgds4nzB/8vDy4pW5pftlynPiUq3yG2P5/dLa8aZPlzYebPuRo/FXKnoiIiPwnjpwOZCqwCqhijIk0xgw1xowwxozI3l/SGBMJPAa8kH1MAFAC+NMYsxlYC8y1LGt+9mXfADoaY/YCHbN/Fwfp3zCElPQs5mw6fuUD/YpgOr7CC5EH8LCyGL1q9JW/jRMREZH/xMNRF7Ysa8BV9p8Egi+xKx6oc5lzYoD2N55OrkWtMoFUKxXA9HVHuaNJuSsfXGcAJTZ8zaNnD/JK1hpm7ZtFr7BeORNUREQkn3C5V6XiOowxDGgUwtZjcWw7Fnflg93coPs4bos9TQOPQoxdN5boJA0KERERsScVN7miHnXK4O3hduWVFP5SsiZuTe7jpYM7Sc1M5rU1rzk+oIiISD6i4iZXFOjnSbdapfhp0zGS0zKvfkKbZwj1Lcb9qZ4sPrKYxYcXOz6kiIhIPqHiJlfVv2EICSkZzN50hZUU/uLtD51fY1Dkbqp5BzFmzRjiUq/ymlVERESuiYqbXFWj8kWoE1KIdxfvvbanbjV64lGhDaOPHuBMSizj1493fEgREZF8QMVNrsoYw4vdq3EyPoXPlx+4lhOg2ziqpSQyyLMkM/fOZM2JNY4PKiIiksepuMk1CQ8tQteaJfnkj/1Exadc/YRilaD5w9y3Zw3lfIvz0sqXSM5IdnxQERGRPEzFTa7ZM12rkp6ZxfhFe67thBaP4RMYwqiYOCLPRfLRpo8cG1BERCSPU3GTa1auaAHuahrK9xFH2Xki/uonePlB17dpeGovtwVU4esdX7Pt9LarnyciIiKXpOIm12Vku0r4+3jy2ryd17asVZUuUKU7j+1cSTHvwoxaOYr0rKusfSoiIiKXpOIm16WQnxcPtQ9j+d7TLN1zjSsjdH0D/6wsns/0Z8+ZPUzcNtGxIUVERPIoFTe5bnc2KUdoUT9em7uTjMysq59QqCy0fpJ2e5bRqWgdPtn8CQfirmF0qoiIiFxExU2um5eHG890rcbeqHNMj7iGpbAAmo6EYpV59sAWfD18Gb1yNFnWNZQ+EREROU/FTf6TzjVK0Ci0CO8s2kNCyjV8s+bhBd3GUiz2ME8WrM6GqA18v/t7xwcVERHJQ1Tc5D8xxvDCTdU4fS6NT/7Yf20nVWgNNW+jx6bZNC1Wh3fWv8PJxJOODSoiIpKHqLjJf1Y7uBC31i3NF8sPcvzsNU6u23kMxt2L/8XEYZHFK6tfubbRqSIiIqLiJjfmyS5VAXh7we5rO8G/JLR7nuADy3iwVFuWRS7j14O/OjChiIhI3qHiJjekTCFfhrYoz6yNx9gSefbaTmp4L5SoxcCNP1OrSHXeWPsGZ1LOODSniIhIXqDiJjfsvjYVKVbQi1d/ucZJed094KbxuCccZ7RbSRLSE3hr3VuODyoiIpLLqbjJDfP38eSRDpVZeyiWBdtPXdtJIY2g3p2Erf+Weyrcyi8HfmF55HLHBhUREcnlVNzELvo3DCGseEHe+HUnaRnXOD9bh9Hg7c+9e1ZTIbACL69+mcT0RMcGFRERycVU3MQuPNzdeK57NQ7FJPHt6sPXdlKBotBhNF6HVzK6eEtOJZ7ivQ3vOTaoiIhILqbiJnbTpnIQLcOK8f6SvcQlXeNC8vXuhOCG1F3xCQMq9WLarmlsitrk0JwiIiK5lYqb2I0xhue6VSMuOZ0JS/Ze20lubtB9HCTF8PCZs5QsUJJRK0eRlpnm2LAiIiK5kIqb2FW1UgH0bRDC5FWHOBxzjd+rlaoDjYbhFzGJ/1UeyIG4A3y25TPHBhUREcmFVNzE7h7vVBlPdzfenL/r2k9q+xwULE6L1ZO4qXx3vtz6JXvO7HFcSBERkVxIxU3srniAD8NbVWTe1pNEHIq9tpN8AqHTGDi+gae8QvD38uellS+RmZXp2LAiIiK5iIqbOMS9rcpTIsCbV+buJCvrGtcirXUbhLak8NK3eKbO/Ww9vZUpO6c4NqiIiEguouImDuHn5cETnaqw+ehZft5y/NpOMsY2UCEtka67/qBVcCs+2PQBkQmRjg0rIiKSS6i4icP0rh9M9VIBvDV/Nynp1/jKM6gKNHsQs/k7XgzuhptxY/Sq0de2lJaIiEgep+ImDuPmZnihezWOnU1m4opD135iqychMISSv73KI3VHsvrEambvn+2wnCIiIrmFips4VLNKxehQrTgf/b6PmHOp13aSVwHo8gZE7aDvmVjqF6/P2+ve5nTyaceGFRERcXEqbuJwz3StRlJ6Ju8uvsZJeQGqdoewzrj98Qajat9HckYyr6953XEhRUREcgEVN3G4SsULMrBxWb5be4R9UQnXdpIx0PVNyMqgwspPuK/OfSw8vJDfjvzm2LAiIiIuTMVNcsTD7cPw83Tn9XnXMSlvkfLQ8nHYPovBPuWoUrgKY1aPIT4t3nFBRUREXJiKm+SIogW9eaBdJX7bFcWKfdfxrVqzh6BIRTznP8Poxs8TkxLD+IjxjgsqIiLiwlTcJMcMbhZKmUK+vDp3J5nXOimvpw90exti91Nj12Luqn4XM/bOYN3JdY4NKyIi4oJU3CTH+Hi683TXquw8Ec+MDdcxqW6l9lD9Vlg+lvvLdSfEP4SXVr5ESkaKw7KKiIi4IhU3yVE31y5F3ZBCjF2wm6S0jGs/scvr4OaB78L/MarJ/ziScISPNn/kuKAiIiIuSMVNcpQxhhdvqkZUQiqfLTtw7ScGlIY2z8DeBTSOj6FXWC++3v41O2J2OC6siIiIi1FxkxzXoFwRutcqxad/HOBU/HW87mw8AopXh1+f5rHaIyjsU5hRK0eRnpXuuLAiIiIuRMVNnOLpLlXJzLIYt3D3tZ/k7gndx0PcUQJXf8bzjZ9nV+wuJm+f7LigIiIiLkTFTZyibFE/BjUrxw/rI9lx/DrmZSvXFOrcDisn0MEvhA5lO/Dxpo85FHfIYVlFRERchYqbOM2DbcMI9PVkzLwdWNY1Tg8C0PFl8PKDuY/zXKNn8fbw5qVVL5FlZTkurIiIiAtQcROnCfTz5OH2YazYF8Pvu6Ou/cSCQdB+FBxaTtCBZTwR/gTrT63nxz0/Oi6siIiIC1BxE6ca2Lgc5YsV4LV5u8jIvI4nZg0GQ+l6sOA5ega3o3HJxryz/h1OJp50WFYRERFnU3ETp/LycOOZrlXZF3WOqeuOXvuJbu62gQrnojBLX2dU01FkZGUwZvWY63vtKiIikououInTdapegkbli/Duoj0kpFzH1B5l6kPDobD2M0ISz/BA3QdYGrmUBYcWOC6siIiIE6m4idMZY3ixe3ViEtP4aOn+6zu53QvgWwTmPsYdVW+nRtEavL72dc6mnHVIVhEREWdScROXUCs4kF71yvDlnweJPJN07Sf6FoZOr0LkOjw2T2V0s9HEp8bzdsTbjgsrIiLiJCpu4jKe6FwFA7y94Dom5QWo0x/KNoPFo6jiXYwhNYcwZ/8cVhxb4ZCcIiIizqLiJi6jdCFf7m1ZgdmbjrPp6NlrP9EY6D4OUuLht5cYXmc4oQGhvLzqZZLSr+PpnYiIiItTcROXMqJNRYoV9GbM3OuclLdEdWh6P2z4Gu/jmxndbDTHE48zYeMEx4UVERHJYSpu4lIKenvwWMfKrDt0hvnbrnNOttbPgH9pmPsY9YvVpl+VfkzZOYXN0ZsdE1ZERCSHqbiJy+kbHkzlEgV5Y/4u0jKuY1Je74LQ5XU4uRXWfcEj9R+huF9xXlr5EumZ1zHNiIiIiItScROX4+HuxnPdqnE4JomvVx26vpOr94CK7WHJqxRMPcf/mv6PfWf38cXWLxySVUREJCepuIlLalOlOC3DijFhyT7OJqVd+4nGQLe3ITMVFr5Aq+BWdC3flc+2fsa+M/scF1hERCQHqLiJy3q+ezUSUtJ5/7frLFxFK0KLR2HrD3DgD55p9AwFPQsyatUoMrMyHRNWREQkBzisuBljvjLGRBljtl1mf1VjzCpjTKox5okLtocYY343xuw0xmw3xjx8wb6XjDHHjDGbsv90c1R+cb6qJQPo1zCEb1Yf4tDpxOs7ucWjUDgU5j1BEY+CPNXwKbZEb2Ha7mkOySoiIpITHPnEbRLQ5Qr7Y4GHgLH/2J4BPG5ZVjWgCfCAMab6BfvfsSyrbvafefYMLK7n0Y6V8XJ3441fd13fiZ6+0PVtOL0HVn3ATRVuokWZFry34T2OnTvmmLAiIiIO5rDiZlnWMmzl7HL7oyzLWgek/2P7CcuyNmT/nADsBMo4Kqe4tuL+PoxoXZH520+y9uBl/+N0aZU7QdWb4I+3MHFHebHJiwC8vOrl65sjTkRExEW49DduxphQoB6w5oLNDxpjtmS/ii3snGSSk+5pWYGSAT68OncHWVnXWbi6vGEbsPDrM5QuWJqH6z/MyuMr+fnAz44JKyIi4kAuW9yMMQWBGcAjlmXFZ2/+GKgI1AVOAOOucP4wY0yEMSYiOjra0XHFgXy93HmycxW2RMYxZ/Px6zu5UAi0fgp2z4Xd8+lfpT91gurw1rq3iEmOcUxgERERB3HJ4maM8cRW2qZYljXzr+2WZZ2yLCvTsqws4HOg0eWuYVnWZ5ZlhVuWFR4UFOT40OJQPeuVoWaZAN6av4uU9OscGdrkAShWBX59EveMVEY3G01SehJvrH3DMWFFREQcxOWKmzHGAF8COy3LGv+PfaUu+LUncMkRq5L3uLkZnu9WneNxKXz558HrO9nDy7YI/dkj8Od4KhaqyLDaw5h/aD5Ljy51RFwRERGHcOR0IFOBVUAVY0ykMWaoMWaEMWZE9v6SxphI4DHghexjAoDmwJ1Au0tM+/GWMWarMWYL0BZ41FH5xfU0rViUDtVK8PHS/Zw+l3p9J5dvCbX6wor34PQ+htYcSqVClXhl9SskpCU4JrCIiIidmfwwui48PNyKiIhwdgyxg/3R5+j8zjL6NQxhTM9a13dywin4IBzK1Ic7f2Lr6W3c8esd3BZ2Gy82fdExgUVERK6TMWa9ZVnhl9rncq9KRa6kYlBB7mhSjqlrj7D31HU+KfMvAe1ehANLYfssagXVYmC1gXy/53siTqrYi4iI61Nxk1znofZhFPD24LV5O6//5IZDoWRtWPAcpCbwYN0HKVOwDKNXjSY18zpfv4qIiOQwFTfJdYoU8GJku0r8vjua5Xuvc6oXN3e46R1IOAlL38DP049RTUdxKP4QH2760DGBRURE7ETFTXKlQc1CCSniy5i5O8m83kl5g8OhwSBY/TGc2k7T0k3pHdabidsmsujwIscEFhERsQMVN8mVvD3cebpLVXadTODH9Uev/wLtR4FPIPzyGGRl8WzjZ6kdVJvn/3yenTH/4RWsiIhIDlBxk1yre61S1C9biLEL95CYmnF9J/sVgY4vw9HVsHkq3u7evNf2PQK8Anjo94c4nXzaMaFFRERugIqb5FrGGJ7vXp3ohFQ+XXbg+i9QdyCENIZFL0JSLMV8izGh3QTiUuN45PdHSMtMs39oERGRG6DiJrlag3KF6V67FJ8t28/JuJTrO9nNzbaiQvIZWPIKANWKVuPV5q+yOXozo1eNJj/McygiIrmHipvkes90qUpWFoxduPv6Ty5ZCxqPgIiJELkegE6hnbi/7v3M2T+Hydsn2zmtiIjIf6fiJrleSBE/hjQPZcaGSLYfj7v+C7R5FgqWgDkPQloiACNqj6BzaGfGrx/Psshldk4sIiLy36i4SZ5wf9tKFPL1ZMzcndf/etMnAG79CKJ3wewHwbIwxvBK81eoWqQqTy17in1n9jkmuIiIyHVQcZM8IdDXk0c6VGbl/hiW7Iq6/gtUag/t/wfbZ9oWogd8PXx5v937+Hr4MnLJSM6knLFzahERkeuj4iZ5xu2Ny1IhqABj5u0kPTPr+i/Q/BGo0RN+Gw37fgOgZIGSvNf2PaKSonhs6WOkZ6bbN7SIiMh1UHGTPMPT3Y1nu1bjQHQiU9ceuf4LGAM9PoSgavDj3RBrm2KkdlBtXmr2EhGnInh97esaaSoiIk6j4iZ5SodqxWlSoQjvLt5LfMp/eDrmVQD6T7H9PO2O84MVbq54M0NrDuWHPT8wbfc0OyYWERG5dipukqcYY3ihe3XOJKXx4e//cUBBkfLQZyJE74TZD0D2E7aH6j9Em+A2vLn2TVYdX2XH1CIiItdGxU3ynJplAulZrwwT/zzE0dik/3aRiu2gw0uwfRaseBcAN+PGG63eoHxgeR7/43EOxx+2W2YREZFroeImedKTnavg5gZvLfgPk/L+pdlDULM3LB4NexcDUMCzABPaTcDDePDgbw8SnxZvp8QiIiJXp+ImeVKpQF+GtazAz5uPs/HIf5zGwxi4ZQKUqAEz7oaY/QAE+wczvs14IhMieeqPp8jIus4F7kVERP4jFTfJs4a3rkiQvzev/pdJef/y12AF4wbTBkLqOQDCS4bzQpMXWHF8BeMixtkxtYiIyOWpuEmeVcDbg8c7Vmb94TP8uu3kf79Q4VC4bSKc3g2z7z8/WKF35d7cUe0Ovt35LTP3zrRPaBERkStQcZM8rU94CFVL+vP6rztJzcj87xeq2BY6vgw7ZsOf489vfjz8cZqVbsYrq19h/an1dkgsIiJyeSpukqe5uxme61aNo7HJfL3yBkeBNn0QavWB316BvYsA8HDz4O3WbxNcMJhHf3+UY+eO2SG1iIjIpam4SZ7XqnIQrSsHMWHJXs4kpv33CxkDN78PJWvCjKHnBysEeAUwod0EMqwMRi4ZSWJ6op2Si4iIXEzFTfKF57tX41xqBu/9tvfGLuTlB/2mgHHPHqyQAEBoYChjW4/lwNkDPLP8GbKs/7BWqoiIyFWouEm+ULmEP/0aluXb1Yc5EH3uxi5WuJxtZYXTu+Gn+84PVmhWuhlPNnySpUeXMmHjhBsPLSIi8g8qbpJvPNaxMt4ebrzx664bv1iFNtDxFdj5Mywfe37z7VVv57bKt/HF1i/45cAvN34fERGRC6i4Sb4R5O/N/W0rsXDHKVYfiLnxCzZ9AGr1hSVjYM8CwLZW6nONniO8RDijVoxiS/SWG7+PiIhINhU3yVeGtihP6UAfxszdSVbWf5yU9y/GwM3vQclaMOPe84MVPN09Gd9mPEF+QTz8+8OcTLyBOeREREQuoOIm+YqPpztPdqnC1mNx/LTJDlN3ePnZVlZw94Bpt58frFDYpzAftPuApPQkHv79YZIzkm/8XiIiku+puEm+06NOGWoHB/L2gt0kp93ApLx/KVQW+kyC03th1gjIso0orVS4Em+1eoudMTt5ccWL/33ZLRERkWwqbpLvuLkZnu9WjRNxKXz55wH7XLR8K+j0Kuz6BZb/vXZp65DWPNrgURYcWsCnWz61z71ERCTfUnGTfKlxhaJ0ql6Cj5fuJyohxT4XbXIf1O4Hv4+B3fPPbx5cYzA3V7iZDzd9yKLDi+xzLxERyZeuubgZYwobY2oYYyoYY1T4JNd7pmtVUjOyeGfRDU7K+5e/BiuUqg0z77W9OsU20nRUs1HUDqrN838+z86Ynfa5n4iI5DtXLGDGmEBjzHPGmK3AauBT4HvgsDHmB2NM25wIKeIIFYIKckeTckxfd4TdJxPsc1FPX9vKCu6etsEKKfEAeLt7817b9wjwCuCh3x/idPJp+9xPRETylas9OfsROAq0tCyrimVZLSzLCrcsKwR4E+hhjBnq8JQiDvJw+zAKenvw2jw7PgUrFAJ9JtumB7lgsEIx32JMaDeBuNQ4Hvn9EdIyb2DdVBERyZeuWNwsy+poWdY3lmWdvcS+CMuyHrEs60uHpRNxsMIFvHiofRh/7Ilm2Z5o+124fEvo/BrsngvL3j6/uVrRarza/FU2R29m9KrRGmkqIiLX5Zq+VTPG/HYt20RyozublqNsET9GzdlOQkq6/S7ceDjUGQBLX4Pdv57f3Cm0E/fXvZ85++cweftk+91PRETyvKt94+ZjjCkCFMsenFAk+08oUDpHEoo4mLeHO2/fVpsjsUk89eMW+z0FMwZuegdK1YWZwyB6z/ldI2qPoHNoZ8avH8+yyGX2uZ+IiOR5V3viNhxYD1TN/vuvP7OBDx0bTSTnNK5QlGe6VOXXbSf5YvlB+13Y0zd7ZQWv7MEKcYBtpOkrzV+hapGqPLXsKfad2We/e4qISJ51tW/c3rMsqzzwhGVZFSzLKp/9p45lWR/kUEaRHHFPy/J0rVmSN+bvYo09FqH/S2Aw9J0MZw7CzOHnByv4evjyfrv38fXwZeSSkZxJOWO/e4qISJ50tVelLQAsy5pwmf0BxpiajggmktOMMbx1W23KFfHjwakbiYq308S8AKEtbIMV9vwKf7x5fnPJAiV5r+17RCVF8djSx0jPtOM3diIikudc7VVpb2PMSmPM/4wx3Y0xjYwxrYwxdxtjvgF+AXxzIKdIjvD38eSTOxtwLiWDB77bQHpmlv0u3mgY1Lkd/ngDds09v7l2UG1eavYSEacieG3taxppKiIil3W1V6WPAt2BE0Af4GXgUaAS8IllWa0sy1rn8JQiOahyCX/e6F2LdYfO8Oavu+x34b8GK5Sub3tlesFghZsr3szQmkP5cc+PTN011X73FBGRPOWq04FYlnUGCAC2AIuAP4HTQFVjTF2HphNxkh51yzCoaTm++PMg87aesN+FPX2g37e2v6cNOD9YAeCh+g/RJrgNb617i1XHV9nvniIikmdc65qjDYARQCls04AMA9oAnxtjnnJMNBHner57deqVLcSTP2xmX9Q5+104sAz0/RrOHLJNE5I9WMHNuPFGqzcoH1iex/94nMPxh+13TxERyROutbgVBepblvWEZVmPA+FAENAKGOygbCJO5eXhxkcD6+Pj6c59364nMTXDfhcv1wy6vAF75sPS189vLuBZgAntJuBhPHjwtweJT4u33z1FRCTXu9biVha4cGHFdKCcZVnJQKrdU4m4iFKBvrw/oB77o8/xzMyt9h040PAeqHsHLHsLdv58fnOwfzDj24wnMiGSp/54iowsOxZGERHJ1a61uH0HrDbGjDLGjAJWAFONMQWAHQ5LJ+ICmlcqxuOdqvDz5uNMXnnIfhc2BrqPgzINbIvRR/09ECK8ZDgvNHmBFcdXMC5inP3uKSIiudo1FTfLsl4B7gXOAnHACMuyXrYsK9GyrIEOzCfiEu5rXZEO1Urw6tydrD8ca78Le/pA329sKyxMux2Sz57f1btyb+6odgff7vyWGXtm2O+eIiKSa13rEzcsy1qfvZLCu5ZlRTgylIircXMzjOtbhzKFfbl/ygZOn7PjFwJ/DVY4e/iiwQoAj4c/TrPSzXh1zausP7XefvcUEZFc6ZqLm0h+F+jryccDG3A2KZ2R320kw56T85ZrBl3fhL0LYOlr5zd7uHnwduu3CS4YzKO/P8qxc8fsd08REcl1VNxErkP10gGM6VmLVQdiGLdoz9VPuB7hQ6HenbDsbdgx5/zmAK8AJrSbQIaVwcglI0lMT7TvfUVEJNdQcRO5Trc1COb2xmX5eOl+Fm4/ab8Lnx+sEJ49WGHn+V2hgaGMbT2WA2cP8MzyZ8iy7Pi0T0REcg0VN5H/4H83Vad2cCCPf7+ZQ6ft+ATMwxv6fQPeBbMHK5w5v6tZ6WY82fBJlh5dyoSNE+x3TxERyTVU3ET+Ax9Pdz4aWB93d8OIb9eTnJZpv4sHlM4erHAUZtwLWX9f+/aqt3Nb5dv4YusX/HLgF/vdU0REcgUVN5H/KLiwH+/1r8fuUwk8P8vOk/OWbWIbrLBvEfw+5vxmYwzPNXqO8BLhjFoxii3RW+x3TxERcXkqbiI3oHXlIB5pX5mZG48xZc0R+148/G6ofxcsHwc7Zp/f7Onuyfg24wnyC+Lh3x/mZKIdv7MTERGXpuImcoNGtqtEmypBvPzzDjYdPWu/CxsD3cZCcEOYdR+c+nuRksI+hfmg3QckpSfx0JKHSM5Itt99RUTEZTmsuBljvjLGRBljtl1mf1VjzCpjTKox5ol/7OtijNltjNlnjHnmgu1FjDGLjDF7s/8u7Kj8ItfKzc3wbr+6BPl7c/+364lNTLv6SdfKw9u2ssIlBitUKlyJt1q9xa7YXby44kX7vqoVERGX5MgnbpOALlfYHws8BIy9cKMxxh34EOgKVAcGGGOqZ+9+BvjNsqww4Lfs30WcrpCfF5/c0YDTiWk8PG0jmVl2LFEBpWzlLS4Sfhx60WCF1iGtebTBoyw4tIBPt3xqv3uKiIhLclhxsyxrGbZydrn9UZZlrQPS/7GrEbDPsqwDlmWlAdOAHtn7egCTs3+eDNxq19AiN6BWcCAv31KD5XtP895iO0/OW7YxdHsb9v8GS165aNfgGoO5ucLNfLjpQxYdXmTf+4qIiEtxxW/cygBHL/g9MnsbQAnLsk4AZP9dPIeziVxRv4Yh9GkQzPtL9rFk1yn7Xjx8CDQYDH++A9tnnd9sjGFUs1HUDqrN838+z86YnZe/hoiI5GquWNzMJbZd93snY8wwY0yEMSYiOjraDrFErs4Ywyu31qR6qQAembaJo7FJ9r1B17cguBH8dD+c2n5+s7e7N++1fY8ArwAe+v0hTieftu99RUTEJbhicYsEQi74PRg4nv3zKWNMKYDsv6MudxHLsj6zLCvcsqzwoKAgh4UV+ScfT3c+uaMBACO+XU9Kuh0n5z2/skKAbbBC0t9fIxTzLcaEdhOIS43j4d8fJjUz1X73FRERl+CKxW0dEGaMKW+M8QL6A3+tuD0HGJT98yBg9iXOF3G6skX9eKdfXbYfj2fU7O1XP+F6+Je0lbe4YzDj4sEK1YpW49Xmr7Ilegsvr3pZI01FRPIYR04HMhVYBVQxxkQaY4YaY0YYY0Zk7y9pjIkEHgNeyD4mwLKsDOBBYAGwE/jesqy//pvvDaCjMWYv0DH7dxGX1L5aCR5sW4npEUeZvs7Ok/OGNLItSL9/Cfw2+qJdnUI7cX/d+5mzfw6Tt0++zAVERCQ38nDUhS3LGnCV/SexvQa91L55wLxLbI8B2tsloEgOeLRjZTZHnuXF2dupUTqQmmUC7XfxBoPgxCZY8R6UqgM1e5/fNaL2CPaf3c/49eOpUKgCrYJb2e++IiLiNK74qlQkz3B3M7zXvx7FCngx4tv1nE2y4+S8AF3ehJAmMPtBOLn1/GZjDK80f4WqRary1LKn2Hdmn33vKyIiTqHiJuJgRQp48eHA+pyKT+HR6ZvIsufkvB5e0Pdr8AmEaQMvGqzg6+HL++3ex9fDl5FLRnIm5cwVLiQiIrmBiptIDqhXtjD/u7kGv++O5oPf7fz0y78E9PsWEk7Aj3dDZsb5XSULlOS9tu8RlRTFY0sfIy3Tzk/8REQkR6m4ieSQOxqXpWe9MryzeA/L9th5bsHgcNtghQO//2uwQu2g2oxuPpqIUxE8uvRRlTcRkVxMxU0khxhjeK1nLaqU8OfhaRuJPGPnyXnr3wXhQ2Hl+7D1x4t23VThJl5s8iLLIpfxyO+PaI43EZFcSsVNJAf5ernz8R0NyMi0eGDKBlIz7Dg5L0CXN6BsU9tghRNbLtrVt0pf/tf0fyw/tlwT9IqI5FIqbiI5rHyxAoztW4fNkXG8/PMO+178r8EKvoVh+sWDFQD6VO7DqKajWHFsBQ8vUXkTEcltVNxEnKBzjZIMb12BKWuOMGN9pH0vXrB49mCFU/DD4IsGKwDcVvk2RjcbzcrjK3loyUOkZKTY9/4iIuIwKm4iTvJkpyo0qVCE53/ays4T8fa9eHADuGk8HPwDFo/61+5eYb0Y3Ww0q46vUnkTEclFVNxEnMTD3Y0JA+oT6OvJiG/XE5ecbt8b1LsDGt4Lqz6ALT/8a3fPsJ6Mbjaa1SdWM3LJSJIzku17fxERsTsVNxEnCvL35sPb63PsTDJP/LDZvpPzAnR5Hco2gzkj4cTmf+3uGdaTV5q/wpoTaxj5m8qbiIirU3ETcbLw0CI8160ai3ac4tNlB+x7cXdP6DsZ/IrAtDsg7ti/DulRqQevtniVtSfX8uBvD5KUbudpSkRExG5U3ERcwJDmodxUuxRvL9jFyn2n7XvxvwYrJJ+BiV3hzKF/HXJLxVsY02IM606u48ElKm8iIq5KxU3EBRhjeLN3bSoEFWTk1I2cjLPzYIEy9eGu2ZASBxO7wel/L7t1c8WbGdNiDOtPreeB3x5QeRMRcUEqbiIuooC3B5/c0YCU9Ezun7KetIws+94guAEMngsZqbYnb6f+PYfczRVv5rUWr7EhagP3/3a/ypuIiItRcRNxIZWKF+St2+qw4chZXpu30/43KFkThvwKbu4wqRsc3/ivQ7pX6M7rLV5nY9RG7lt8n8qbiIgLUXETcTHda5diaIvyTFp5iNmb/j2Y4IYFVYYh88DLHybfAkdW/+uQbhW68WbLN9kcvZn7Ft9HYnqi/XOIiMh1U3ETcUHPdK1Kw9DCPDNjK3tOJdj/BkUqwN2/QoEg+KYnHPjjX4d0Kd+FN1q9ofImIuJCVNxEXJCnuxsf3F6fAt4ejPh2PQkpdp6cFyAw2PbatFA5mNIH9iz81yFdQrvwZqs32RK9hRGLRnAu7Zz9c4iIyDVTcRNxUSUCfPjg9nocjkni6RlbsCw7T84L4F/CNmCheFWYdjvsmP2vQzqHduatVm+x9fRWRixWeRMRcSYVNxEX1qRCUZ7uUoV5W0/y5Z8HHXOTAkXhrjlQuh78MAQ2T//XIZ1CO/F267fZfno7wxcPJyHNAa9vRUTkqlTcRFzcvS0r0KVGSV7/dRdrDsQ45ia+heDOWVCuGcwaDhET/3VIx3IdGdt6LDtO72DEohEqbyIiTqDiJuLijDG83ac2ZYv48eDUjUTF23ly3r94F4SBP0ClDvDLI7D6438d0r5ce8a2GcuOmB0MXzSc+LR4x2QREZFLUnETyQX8fTz55I4GnEvJ4MHvNpKeaefJef/i6Qv9p0DVm2D+M7Bs7L8OaV+2PePajGNn7E6GL1R5ExHJSSpuIrlElZL+vN6rFmsPxfLW/F2Ou5GHN/SZDLX6wpJX4LdX4B8DI9qVbcf41uPZdWYXwxYOIy41znF5RETkPBU3kVzk1npluKtpOT5ffpB5W0847kbuHtDzE6h/FywfCwue+1d5a1u2Le+0eYfdZ3YzbJHKm4hITlBxE8llXuhenXplC/HkD5vZH+3AqTnc3OHm96HxCFj9ke27t6yLX9G2CWnDu23eZe+Zvdy78F6VNxERB1NxE8llvDzc+Ghgfbw93RnxzXoSUzMcdzNjoMsb0OIxWD8JfhoBmRffr3VIa95t+y77zu5TeRMRcTAVN5FcqFSgLxMG1GN/9DmenbnVMZPz/sUY6DAK2r0AW6bDj0MgI+2iQ1oFt+K9tu+x/+x+7ll4D2dTzjouj4hIPqbiJpJLNa9UjMc7VWHO5uNMXnnI8Tds9SR0fg12zoHpd0D6xdOStAxuyXvt3uPA2QPcs/AezqSccXwmEZF8RsVNJBe7r3VFOlQrzqtzd7L+cA4UpaYPwE3vwN6F8F1fSLt44fkWZVrwfrv3ORh3UOVNRMQBVNxEcjE3N8O4vnUpXciXB6Zs4PS5VMffNPxu24jTQ8vhm16QcvE3bc3LNGdCuwkcjj/M0IVDiU2JdXwmEZF8QsVNJJcL9PXk4zvqcyYpjZHfbSTDUZPzXqhOf7jtKzgWAZNvgaSLy1mzMs14v937HIk/wj0L71F5ExGxExU3kTygRulAxvSsxaoDMYxbtCeHbtoT+k2BqJ0wqTuci7pod7PSzZjQbgJH4o8wdMFQYpIdtM6qiEg+ouImkkfc1iCYAY3K8vHS/SzcfjJnblqlC9w+Hc4cgoldIe7YRbublm7KB+0/IDIhknsW3qPyJiJyg1TcRPKQUTdXp3ZwII9/v5lDpxOvfoI9VGwLd8yEhFO28nbm0EW7m5Rqcr68DV0wlNPJp3Mml4hIHqTiJpKH+Hi689HA+ri7G0Z8u57ktMycuXG5pjBotm2gwldd4fTei3Y3LtWYjzp8xPHE4ypvIiI3QMVNJI8JLuzHu/3qsvtUAs//5ODJeS9UpgEMngtZ6bYnbye3XbS7YcmGfNj+Q04knuDuBXcTnRSdM7lERPIQFTeRPKhNleI83D6MmRuO8d3aIzl345I1YfA8cPOwDVg4tuGi3Q1LNuSj9h9xMvGkypuIyH+g4iaSRz3ULow2VYIYPWcHm4+ezbkbB1WGIb+CTwB83QOOrL5od3jJcD7u8DGnkk5x94K7iUqKusyFRETkn1TcRPIoNzfDO33rEuTvzf1TNhCbmHb1k+ylSHlbeStYHL7pCQeWXrS7QYkGfNLhE6KSorh7wd2cSjyVc9lERHIxFTeRPKxwAS8+uaMB0QmpPDwthybn/UtgsK28FQ6FKX1hz4KLdtcvUZ9POn5CdFI0QxcOVXkTEbkGKm4ieVyt4EBe7lGD5XtP8+B3G0nNyKGRpmB74jZ4LhSvBtMGwvafLtpdr3g9Pu34KaeTT3P3grs5mZhD88+JiORSKm4i+UD/RmX5303Vmb/9JPd+nYPThAD4FYFBc6BMffhxCGyedtHuusXr8kmHT4hJiVF5ExG5ChU3kXzi7hbleat3bZbvjWbQV2tJSEnPuZv7BNom6Q1tAbNGQMTEi3bXLV6XTzt+ypmUMwyZP4QT507kXDYRkVxExU0kH+nbMIT3+9djw5EzDPxiDWdycsCCd0G4/XsI6wi/PAKrPrpod52gOnza8VPOpp5lyAKVNxGRS1FxE8lnbq5Tmk/vbMCukwn0+2wVUfEpOXdzT1/bwvTVboEFz8Kyty/aXTuoNp91/Iz41HiGLBjC8XPHcy6biEguoOImkg+1r1aCSYMbEnkmmT6friLyTFLO3dzDC26bCLX7wZJX4beX4YLVHWoF1eKzTrbydveCuzl27tgVLiYikr+ouInkU80qFePbexpzJjGNPp+s4kD0uZy7ubsH3PoJ1B8Ey8fB/GcvKm81i9Xk806fE58Wz93zVd5ERP6i4iaSj9UvW5ipw5qQlpFF309XsfNEfM7d3M0Nbn4PGt8Haz6Gnx+GrL9Hu9YoVoPPO33OufRzDJk/hMiEyJzLJiLiolTcRPK5GqUDmT68KR5ubvT7dBUbj5zJuZsbA11eh5aPw4bJthGnmRl/ZytqK2+J6YncveBujiYczblsIiIuSMVNRKhUvCA/jGhKIT8v7vhiDav2x+TczY2B9v+Ddi/C1u9tc71l/D3atXrR6nzR6QuSMpJs5S1e5U1E8i8VNxEBIKSIHz+MaErpQr4MnriW33fl8OLvrZ6Azq/DzjkwfSCkJ5/fVa1oNb7o9AXJGckMWTBE5U1E8i0VNxE5r0SAD9OHNyWsREGGfRPB3C05PJda0/vhpndh7yL4ri+k/j1gomqRqnzZ6UtSM1MZvGAwR+KP5Gw2EREXoOImIhcpUsCL7+5tQp3gQoycuoEfInL46Vb4EOj5KRz6E77tBSlx53dVKVKFLzp9QVpmGkMWDOFw/OGczSYi4mQqbiLyLwE+nnw9tBHNKhbjyR+3MHnloZwNUKefba63Y+th8i2QFHt+11/lLT0znbvn382huBzOJiLiRCpuInJJfl4efDEonI7VSzBqznY+/H1fzgaocSv0/w6idsKk7pBw6vyuKkWq8EXnL0jPSufuBXdzMO5gzmYTEXESFTcRuSwfT3c+GlifHnVL8/aC3bw5fxfWBRPlOlzlzjDwezhzCCZ1g7i/J+KtXLgyX3b+kkwrk6ELhqq8iUi+4LDiZoz5yhgTZYzZdpn9xhjzvjFmnzFmizGmfvb2KsaYTRf8iTfGPJK97yVjzLEL9nVzVH4RsfF0d2N837oMaFSWj5fu56U528nKysHyVqEN3DkLzkXBxC4Q+3dBCyscxpedbOXt7gV3s+9MDj8VFBHJYY584jYJ6HKF/V2BsOw/w4CPASzL2m1ZVl3LsuoCDYAkYNYF573z137LsuY5IriIXMzdzfBaz5rc27I8k1cd5qkZW8jIzMq5AGWbwF2zITUBJnaD6D3nd1UqXImvOn+FZVnc+eudLI9cnnO5RERymMOKm2VZy4DYKxzSA/jaslkNFDLGlPrHMe2B/ZZlaeiYiJMZY3iuWzUe6RDGj+sjeWjaRtIycrC8lakPg+dCVjpM7Aon/36YX7FQRaZ2n0qZgmV4cMmDTN4+OWdf6YqI5BBnfuNWBrhwnoHI7G0X6g9M/ce2B7NfrX5ljCnsyIAicjFjDI90qMwL3asxb+tJhn0TQUp65tVPtJcSNWDIr+DuZRuwcGz9+V2lCpbi665f0y6kHWMjxvLiihdJy0y7wsVERHIfZxY3c4lt5/8nsjHGC7gF+OGC/R8DFYG6wAlg3GUvbswwY0yEMSYiOjraLoFFxOaelhV4rWct/tgTzaCv1nIuNePqJ9lLsTC4+1fwCYTJPeDwqvO7/Dz9GNdmHCPqjGD2/tkMXTCU08mncy6biIiDObO4RQIhF/weDBy/4PeuwAbLss7PAWBZ1inLsjIty8oCPgcaXe7ilmV9ZllWuGVZ4UFBQXaOLiK3Ny7Lu/3qEnH4DAO/WMPZpBx8ulU41Pbkzb+EbZLe/b+f3+Vm3Hig7gO83fptdsXuYsDcAeyM2Zlz2UREHMiZxW0OcFf26NImQJxlWReurzOAf7wm/cc3cD2BS45YFZGc0aNuGT4eWJ+dx+Pp/9lqohNSc+7mgWVs5a1wefiuH+xZcNHuLqFdmNR1EllWFoPmD2LR4UU5l01ExEEcOR3IVGAVUMUYE2mMGWqMGWGMGZF9yDzgALAP29Oz+y841w/oCMz8x2XfMsZsNcZsAdoCjzoqv4hcm041SvLl4HAOxyTR99NVHDubfPWT7KVgcRj8C5SoDtNuh41TLtpdo2gNpnWfRlihMB5b+hgfb/5YgxZEJFcz+eH/iYWHh1sRERHOjiGSp0UcimXIxHUE+Hoy5Z7GhBYrkHM3T4mDaQPh0HKoczt0exu8C57fnZqZyuiVo/n5wM90KteJV1u8iq+Hb87lExG5DsaY9ZZlhV9qn1ZOEBG7CA8twtRhTUhOz6TPp6vYfTIh527uE2ib5631M7B5Knze9qLpQrzdvRnTYgyPNniURYcXMejXQZxMPJlz+URE7ETFTUTspmaZQKYPa4KbgX6frWJL5Nmcu7mbO7R91lbgUuLgi/YQMRGy3yoYY7i75t1MaDeBw/GH6f9LfzZHb865fCIidqDiJiJ2FVbCnx+GN6Ogtwe3f76GtQevNA+3A1RoDSNWQLlm8Msj8OPdkBJ/fnfrkNZM6TYFXw9fhswfwpz9c3I2n4jIDVBxExG7K1vUjx9HNKNEgDd3fbWGP/bk8FyKBYNg4AxoPwp2zIZPW8Hxjed3Vypcie+6f0fd4nV5/s/nGR8xnsysHJxIWETkP1JxExGHKBnow/ThTalQrCD3TF7H/G0nrn6SPbm5QcvHYMg8yEyDLzrC6k/Ovzot7FOYTzt+St/KfZm4fSIP/f4Q59LO5WxGEZHrpOImIg5TrKA3U4c1oVaZQB74biMzN0TmfIiyTWDEn1CpA8x/2jb6NMn2+tbTzZMXm77I842fZ8WxFdwx7w6Oxh+9ygVFRJxHxU1EHCrQ15NvhjamcfkiPPb9Zr5ZfTjnQ/gVgQFTofPrsHeh7dXp0bXnd/ev2p9POn5CdHI0A+YNYO2JtVe4mIiI86i4iYjDFfD24KvBDelQrTgv/rSNT/7Yn/MhjIGm98PQBWDc4Ksu8Oe7kJUFQJNSTZjafSpFfIowfNFwvt/9fc5nFBG5ChU3EckRPp7ufHxHA26qXYo3ft3F2AW7nbOKQZkGMGI5VLsJFo+C7/pAom0h+rIBZZnSbQpNSjfhldWv8OrqV0nPSs/5jCIil6HiJiI5xtPdjff616NfeAgf/L6P0T/vICvLCeXNJxD6TIbu4+DgcvikBRz6EwB/L38+aPcBg2sMZvru6dy36D7iUuNyPqOIyCWouIlIjnJ3M7zRuxZ3Ny/PpJWHeGbmFjKdUd6MgYb3wD2LwasATL4Zlr4JWZm4u7nzePjjvNr8VTZEbWDA3AEcOHsg5zOKiPyDipuI5DhjDC/eVI2H2lXi+4hIHpq2kbSMLOeEKVUbhi2FmrfB0tfg6x6QYFsOq0elHnzV+SsS0xMZOG8gyyKXOSejiEg2FTcRcQpjDI91qsKzXasyd8sJRny7npR0J02C6+0PvT6DHh9CZAR83Bz2/QZA3eJ1mdZ9GsH+wTz424NM2jbJOd/miYig4iYiTja8dUVevbUmv++OYsjEdSSmZjgniDFQ7w7b07cCQfBtb/jtZcjMoFTBUkzuMpkO5Towbv04XljxAmmZac7JKSL5moqbiDjdHU3KMb5vHdYeiuWOL9cQl+TEkZzFq8K9S6D+nbB8HEzqDnGR+Hn6Mbb1WO6rcx9z9s/h7gV3czr5tPNyiki+pOImIi6hZ71gPry9PtuOxdH/89WcPpfqvDBefnDLBOj1BZzaZht1uvtX3Iwb99e9n7Gtx7I7djcD5g5gZ8xO5+UUkXxHxU1EXEaXmiX5YlBDDp4+R99PV3EiLtm5gWr3geHLIDAYpvaH+c9BRhqdQzszuetkLMti0PxBLDy00Lk5RSTfUHETEZfSunIQX9/dmKj4VPp8sorDMYnODVS0IgxdDI2GweoP4avOEHuQ6kWrM+2maYQVDuPxPx7n400fk2U5aWSsiOQbKm4i4nIalS/Cd/c25lxqBn0+WcXeUwnODeTpA93ehr7fQMx+21qn23+imG8xvur8FbdUvIWPNn/EE388QVJ6knOzikiepuImIi6pdnAhpg9rigX0/XQV2465wOoF1W+BEcugWBj8MAh+eQzvLItXm7/K4w0eZ/HhxQyeP5iTiSednVRE8igVNxFxWVVK+vPD8Kb4eXkw4LPVRByKdXYkKBwKQ+ZDs5EQ8SV80QETs5/BNQfzQfsPOJJwhP6/9GdT1CZnJxWRPEjFTURcWmixAvwwoilB/t7c+eValu+NdnYk8PCCTq/C7d9D/DHbq9Mt39MquBVTuk3Bz9OPuxfczex9s52dVETyGBU3EXF5pQv5Mn14U8oV9WPopAgWbneRV5GVO8OIP6FUHZh5L8x+gIp+Jfmu23fUK16PF1a8wLiIcWRmOWlFCBHJc1TcRCRXCPL3ZtqwJlQrHcB9UzYwe9MxZ0eyCSwDg36GVk/CxinweTsKxZ/gk46f0K9KPyZtn8TIJSM5l3bO2UlFJA9QcRORXKOQnxdT7mlMw9DCPDJ9E9+tOeLsSDbuHtDuBbhzFiTFwmdt8dw0lRcaP88LjV9g5fGVDJw3kCPxLpJXRHItFTcRyVUKenswaUgjWlcO4rlZW/l82QFnR/pbxba2V6chjWDOSJh5L/3Kd+Ozjp8RkxLD7fNuZ82JNc5OKSK5mIqbiOQ6Pp7ufHZnON1qlWTMvJ28s2gPlmU5O5aNfwnbk7d2L8C2GfBpKxrhzdRuUynqU5Thi4Yzbdc0Z6cUkVxKxU1EciUvDzfe71+P2xoE895vexkzd6frlDc3d9s3b4N+gfQU+KIDIbvmM6XrtzQv05wxa8bw6upXSc9Kd3ZSEcllVNxEJNfycHfjrd61GdS0HF/8eZD7p2zgbFKas2P9LbS57dVphTYw7wkK/nQf7zd5mSE1hjB993RGLBrB2ZSzzk4pIrmIipuI5GpuboaXbqnBM12rsmjHKbq8u5yV+047O9bfChSFAdNt877t/hX3z9vwWMlWjGkxho1RGxkwdwD7z+53dkoRySVU3EQk1zPGMKJ1RWbd3xw/b3du/2INY+buIDXDReZPc3OzrbRw9wKwgK86c8upw3zV6UuSM5IZOG8gyyKXOTuliOQCKm4ikmfUCg7kl5EtGNi4LJ8vP8itH65kj7MXqL9QcLhtrdPKXWDhC9Rd/DrT2n1MWf+yPPjbg0zcNtF1vtMTEZek4iYieYqflwdjetbii7vCiYpP4eYJfzJpxUHXKUS+haHft9D1bTjwOyW/7sWkGvfRsVxHxq8fz/N/Pk9qZqqzU4qIi1JxE5E8qUP1Esx/pBXNKhblpZ93MHjiOqISUpwdy8YYaDwMhi4CTx/8vunFWFOC++vcx88HfubuBXdzOtmFvtMTEZeh4iYieVaQvzdfDW7Iyz1qsPpADF3eXc6iHaecHetvpevCsD+gRk/M72O4b8sCxjcZxd4ze+n/S392xOxwdkIRcTEqbiKSpxljuKtpKL+MbEHJAB/u/TqCZ2duJSktw9nRbHwCoPeXcPP7cGQ1Hec8y9e1HsIYw6BfB7Hg0AJnJxQRF6LiJiL5QlgJf2Y90IzhrSowbd0Rbnr/T7ZEnnV2LBtjoMEguPd38C1M1ZkPMjWwCVUKV+aJP57gw00fkmVlOTuliLgAFTcRyTe8Pdx5tls1ptzTmOT0THp9tJIPf99HZpaLDFwoUR2G/Q51B1Jsxft8FRVLj7Id+WTzJzzxxxMkpSc5O6GIOJmKm4jkO80qFmP+w63oXLMkby/YzYDPVnM01kVKkVcBuPVD6PkpXsc388raWTxR7mZ+O/Ibd/x6B7tjdzs7oYg4kYqbiORLgX6efDCgHuP71mHHiXi6vbecnzYec3asv9XpD8P/wASUZtDSD/mocGNik2PpP7c/X2z9gswsF5lcWERylIqbiORbxhh61Q/m14dbUqWkP49M38RDUzcSl+wii78XC4N7FkP4UJqvn86ss5m0LVqX9za8x+D5gzkSf8TZCUUkh6m4iUi+F1LEj2nDmvB4x8rM3XqCbu8tZ/WBGGfHsvH0hZvGQ9+vKZwQxbi1M3nDJ4z9Z/dx28+3MW3XNNeZXFhEHE7FTUQE8HB3Y2T7MH4c0RRPd8OAz1fz5vxdpGW4yGjO6j1g5HpMy8fpvmc5sw4fpZ5nYcasGcOIxSM4mXjS2QlFJAeY/PC/1MLDw62IiAhnxxCRXCIxNYOXf97B9Iij1CwTwLv96lGpeEFnx/pb7EFY+ALWrl/4vnhZxvl74uHuw3NNnqN7+e4YY5ydUERugDFmvWVZ4ZfapyduIiL/UMDbgzdvq80ndzQg8kwyN01YzrerD7vOK8ki5aH/FMxds+lHQX48fJiK6Wk8u/xZHv/jcWJTYp2dUEQcRMVNROQyutQsyYJHWtEwtAgv/LSNeyZHcPqcCy0AX6ENDF9O2U5vMOnkaR6JjWPp4cX0/OlWfj/yu7PTiYgD6FWpiMhVZGVZTFp5iDfm7yLAx4O3b6tD26rFnR3rYkmxsPR1dm+azHPFi7HH051bK/bg6UbPUNDLhV7zishV6VWpiMgNcHMz3N2iPHMebE6xgt4MmbSOF3/aRnKaC82l5lcEur1NlaFLmeZZkXvPxjFn32x6z+zOupPrnJ1OROxExU1E5BpVLRnATw80Z2iL8nyz+jA3f/An247FOTvWxUpUx/OuOTzU6SMmnzN4njvF3Qvu5s1lz5GSkeLsdCJyg1TcRESug4+nOy/eVJ1vhjYiPjmdnh+t4JM/9pPlKuudgm3R+mo3UXf4Wr4PG0L/cyl8e/Bn+k5vx7bja5ydTkRugL5xExH5j84kpvHszK3M336SJhWKML5vXUoX8nV2rH+LP8HKBY/yYvxmYtzdubdkS4Z1eBdPD29nJxORS9A3biIiDlC4gBcf31Gft3rXZktkHF3eXcbPm487O9a/BZSiWZ9pzGr3Kd2yfPjk1J8M/KYJ+3fNdnYyEblOKm4iIjfAGEPfhiHMe6glFYIKMnLqRh6bvomEFBdZ7/QCAeVb8drgtbwT2puTpNN31fNMnnYTmXGRzo4mItdIr0pFROwkPTOLCUv28cGSvZQu5Mu7/eoSHlrE2bEu6XTcYV6eP4zfU47TIDWdVyv1J7jlM+Dp4+xoIvmeXpWKiOQAT3c3HutYmR9GNMUY6PvpKsYt3E16pousd3qBYoHleK/vfF6p+zC7vb3pffgHZnzeEGvHHMgH/4NeJLfSEzcREQdISEnnpTk7mLEhkjohhXi3X13KFyvg7FiXdOLcCV5c/CBr4vbQKimZl/wqE9RlLJSo7uxoIvmSnriJiOQwfx9PxvWtw4e31+fQ6US6v7+caWuPuM56pxcoVbAUn/X4gWfCn2RNgYL0zDrM/K87wNwnbCsyiIjLUHETEXGg7rVLMf+RltQNKcQzM7cy/Jv1xCamOTvWv7gZNwbWuIvve8wkpGh1nixelKcOzSLug/qw9nPIzHB2RBFBr0pFRHJEVpbFF38e4O0Fuyns58XYPnVoVTnI2bEuKSMrgy+3fsknmz+msAUvnzxBi4LloesbtoXtRcShnPKq1BjzlTEmyhiz7TL7jTHmfWPMPmPMFmNM/Qv2HTLGbDXGbDLGRFywvYgxZpExZm/234UdlV9ExJ7c3AzDWlXkpweaE+jryV1frWX0z9tJSXeh9U6zebh5MLzOcKZ0/47AQqHcV7I4L3smkvTNrTBtIMQedHZEkXzLka9KJwFdrrC/KxCW/WcY8PE/9re1LKvuPxrnM8BvlmWFAb9l/y4ikmvUKB3IzyNbMLhZKBNXHKLHByvYeSLe2bEuqXrR6ky7aTqDawzmR29D70rV2XB0OXzYCBaPhtRzzo4oku84rLhZlrUMuNJXrT2Ary2b1UAhY0ypq1y2BzA5++fJwK03HFREJIf5eLrz0i01mDikITGJafT4YAVfLD/gWuudZvN29+bx8MeZ2GUilk8Ag4sXYnzF+qSuGA8TGsDmaZDletOdiORVzhycUAY4esHvkdnbACxgoTFmvTFm2AXHlLAs6wRA9t/FcySpiIgDtK1SnAWPtKRV5SBenbuTu75ay6n4FGfHuqQGJRow45YZ9K7cm4npx+lfoym7CpWAWcPhy44Qqe+IRXKCM4ubucS2v/7nZnPLsupje536gDGm1XVf3JhhxpgIY0xEdHT0jeQUEXGYogW9+fyuBozpWZOIw7F0fncZ87edcHasSyrgWYBRTUfxYfsPictKY4BXAp81vZOMuKPwRXuYdR8knHR2TJE8zZnFLRIIueD3YOA4gGVZf/0dBcwCGmUfc+qv16nZf0dd7uKWZX1mWVa4ZVnhQUGuOXJLRARs650ObFyOuQ+1JKSwHyO+3cBTP24mMdU1p+BoFdyKmbfMpEO5Dkw4+QeDqtbnYON7YNuPtteny8dDums+ORTJ7ZxZ3OYAd2WPLm0CxFmWdcIYU8AY4w9gjCkAdAK2XXDOoOyfBwGzczq0iIijVAwqyIz7mnF/m4r8sD6Sbu8vZ+ORM86OdUmFfArxduu3eavVWxxKOErfmGVM6fYiWeVbw2+j4aPGsPMXLZ8lYmcOm8fNGDMVaAMUA04BowBPAMuyPjHGGOADbCNPk4AhlmVFGGMqYHvKBuABfGdZ1pjsaxYFvgfKAkeAPpZlXXVab83jJiK5zZoDMTz2/WZOxqfwULswHmhbEQ9315wzPSopilErR/HnsT9pXLIxrwR3pdTStyF6J5RvDV3e0PJZItfhSvO4aQJeEREXFZeczqjZ2/hp03Hqly3EO/3qUq6oa653alkWM/bO4K11b+Fu3Hkm/EluiY3GLH0NUhOg4VBo8yz4FXF2VBGXp+Km4iYiudjsTcd44adtpGZkMbhZKPe3qUghPy9nx7qkowlHeeHPF9gQtYF2Ie34X92HKLrqY4j4EnwCoe3z0GAIuHs4O6qIy1JxU3ETkVzu+Nlkxi3cw8yNkRT09mBE64oMaR6Kn5frFaDMrEy+2fEN7298H38vf/7X5H+09ykJvz4Nh5ZD8eq216cVWjs7qohLUnFTcRORPGL3yQTGLtzNoh2nCPL35qH2YfRvGIKnC37/tvfMXp7/83l2xu7kloq38HTDpwg4sAwWPA9nD0PVm6DTq1CkvLOjirgUFTcVNxHJY9YfjuXNX3ez9lAs5Yr68VjHytxcuzRubpeaItN50jPT+XTLp3yx9QuK+Rbjleav0DSoHqz6wDZtSFYGNHsQWjwG3gWdHVfEJai4qbiJSB5kWRZLd0fz5vxd7DqZQPVSATzVpQqtKwdhG7jvOrZGb+W5P5/jUPwhBlQdwKMNHsU36YxtzdMt06BgSejwEtTuC27uzo4r4lQqbipuIpKHZWVZ/LzlOOMW7uFIbBKNyxfh6a5VqV+2sLOjXSQ5I5n3N7zPtzu/JTQglDEtxlA7qDYcXWv7/u34BihSAZo+AHVuBy8/Z0cWcQoVNxU3EckH0jKymLbuCO//to/T51LpVL0ET3auQlgJf2dHu8iaE2t4ccWLnEo6xdCaQ7mvzn14GnfYORtWvG8rcL5FoNG90PBeKKjVbyR/UXFTcRORfCQxNYOJKw7y6R8HSEzLoFf9YB7tWJkyhXydHe28hLQE3lz7JrP3z6ZqkaqMaTGGyoUr21ZaOLLKVuD2/AoePlBnADR9EIpVcnZskRyh4qbiJiL50JnEND5auo/Jqw6DBXc2LccDbStRpIDrzAG35MgSRq8aTUJaAg/We5C7qt+Fh1v2FCfRe2yDGDZPg8w0qNINmj8EIY3Bxb7hE7EnFTcVNxHJx46fTebdxXv4cX0kfl4e3NuyAve0LE8Bb9eYAy42JZZXVr3C4iOLKR9YnscaPEbr4NZ/D7A4FwVrP4N1X0DyGQhuCM1G2qYT0UAGyYNU3FTcRETYF5XA2AV7mL/9JEULeDGyXSVub1wOLw/nzwFnWRZLjy5l/PrxHIo/RMOSDXk8/HFqFK3x90FpibDpO9tTuDOHoHB520CGugM1kEHyFBU3FTcRkfM2HjnDW/N3s+pADMGFfXm8U2VuqVMGdxeYAy49K50Ze2bw0aaPOJN6hpsq3MRD9R6iVMFSfx+UlQk7f4aV78Ox9baBDA3vgUbDNJBB8gQVNxU3EZGLWJbF8r2neXP+LrYfj6dqSX+e7FyFdlWLu8QccAlpCXy17Su+2fENAHdWv5OhNYdS0OuCSXotC46sthW43fPA3Rvq/jWQIcxJyUVunIqbipuIyCVlZVnM3XqCcQt3cygmiYahhXmqS1UahhZxdjQATpw7wYSNE/j5wM8U8SnCfXXuo3fl3ni6eV584Om9tleom6ZCZqptIEOzh6BsEw1kkFxHxU3FTUTkitIzs/g+4ijvLd5LVEIq7asW58kuVahaMsDZ0QDYHrOdcRHjWHdyHaEBoTzW4DHahLT599PBc9Gw7nNY+zkkx0KZcNtAhmo3ayCD5BoqbipuIiLXJDktk4krD/Lx0v2cS82gZ90yPNqxMiFFnP/xv2VZLItcxrj14zgYd5DwEuE8Ef4ENYrV+PfBaUmwaQqs+hDOHITCobZXqHVvB68COZ5d5HqouKm4iYhcl7NJaXzyxwEmrjhIlmUxsHE5HmxXiWIFvZ0djYysDNsAhs0fEZsSS/cK3Xmo3kOULlj63wdnZcKuX2wT+h6LAN/CttUYGt0LBYvnfHiRa6DipuImIvKfnIxL4b3f9vJ9xFG8Pdy4p2UF7m1ZHn8fz6uf7GDn0s7x1bav+HrH11iWxR3V7+CeWvfg73WJJb4sC46ugZUTYNdccPeCOv1tT+GCKud8eJErUHFTcRMRuSH7o88xfuEe5m49QZECXjzQthIDG5fFx9P5342dTDxpG8Cw/2cKeRdiRJ0R9KnS598DGP5yeq/tFeqm7y4YyDASyjbVQAZxCSpuKm4iInaxJfIsby/YzfK9pylTyJdHOoTRq36wS8wBtyNmB+MixrH25FrKBZTj0QaP0i6k3eWnN/nXQIYG2QMZbtFABnEqFTcVNxERu1qxzzYH3JbIOMKKF+TJzlXoWL2E0+eAsyyL5ceWMy5iHAfiDlC/eH2eCH+CWkG1Ln9SWhJs/s72FC72ABQqZ3uFWm+gBjKIU6i4qbiJiNidZVn8uu0kYxfs5sDpROqXLcTTXarSuEJRZ0cjIyuDmXtn8uGmD4lNiaVr+a48XP9hyhQsc/mTsjJt37+tfB8i12UPZPhrRQYNZJCco+Km4iYi4jAZmVn8uD6Sdxfv5WR8Cm2qBPFk5yrUKB3o7GgkpifaBjBs/5pMK5M7qt3BPbXvIcDrKvPTHVljK3DnBzL0g6YjNZBBcoSKm4qbiIjDpaRnMnnlIT5aup+45HRuqVOaxztVplxR579uPJl4kg82fsCc/XMI8A7gvjr30bdyXzzdrzI69vQ+WJ09kCEjBSp3tX0HV66ZBjKIw6i4qbiJiOSYuOR0Plu2ny//PEhGpsWARmUZ2b4Sxf19nB2NXbG7GBsxljUn1lDWvyyPNniU9mXbX/3bvMTTsO4LWPsZJMVA6frQ/CGoejO4e+RMeMk3VNxU3EREclxUfArvL9nLtLVH8XR3Y2iL8gxrXYEAJ88BZ1kWfx77k3ER49gft596xevxRPgT1A6qffWT05Jg81TbuqjnBzI8APXu0EAGsRsVNxU3ERGnOXQ6kXGL9vDz5uMU8vPk/jYVuatpqNPngMvIyuCnfT/xwcYPiEmJoUtoFx6u/zDB/sFXPzkrE3b/avsO7uga8Cn090AG/xIOzy55m4qbipuIiNNtOxbH2wt288eeaEoF+vBIhzB61w/Gw93NqbkS0xOZtH0Sk7ZNItPK5Paqt3Nv7XsJ9L7GwRVH1sCqCbDzF3D3hNr9bN/BBVVxbHDJs1TcVNxERFzGqv0xvDl/F5uOnqVCUAGe7FSFLjVLOn0OuFOJp/hw04f8tO8n/L38GVFnBP2r9L/6AIa/xOzPXpFhSvZAhi7ZAxmaayCDXBcVNxU3ERGXYlkWC3ec4u0Fu9kXdY46wYE81aUqzSoWdXqB2x27m3ER41h1YhUh/iE82uBROpTtcO25LjWQ4a8VGTSQQa6BipuKm4iIS8rMspixIZJ3F+3heFwK1UoFcHvjstxat7RTF7K3LIsVx1cwLmIc+87uo25QXZ5o+AR1gupc+0XSk20DGVZ+ALH7oVBZCL8bavWBwGv4jk7yLRU3FTcREZeWkp7JjA2RTFl9hB0n4vHzcqdH3dLc3qgctYKdN5FvZlambQDDpg84nXyazqGdebj+w4T4h1z7RbKyYM+vtgJ3ZCVgILSFrcBV7wG+hRwVX3IpFTcVNxGRXMGyLDZHxjFl9WF+3nKclPQsagcHcnujstxStzR+Xs551ZiUnmQbwLB9EulZ6dxe9XaG1R527QMY/hJ7ALb+CFumQ8w+26oMlTvbBjSEdQIPb8f8A0iuouKm4iYikuvEJacza0Mk3609wp5T5/D39uDWemW4vXFZqpW6ypJVDhKVFMWHmz5k1t5Z+Hv5M7z2cPpX7Y+Xu9f1Xciy4PhG2PI9bJsBiVHgEwjVb7WVuLJNwc25o23FeVTcVNxERHIty7KIOHyG79YcYe7WE6RlZFG/bCEGNi5H99qlnDIf3O7Y3byz/h1WHF9BcMFgHmnwCJ3KdfpvAysyM+DgUluJ2/kLpCdCYAjUus1W4opXs3t+cW0qbipuIiJ5wpnENGZsiOS7NUc4cDqRQF9PetcP5vbGZalUvGCO51lxbAXj1o9j75m91AmqwxPhT1C3eN3/fsG0RNg1z/Yqdf8SsDKhRC2o3ddW5AJK2y27uC4VNxU3EZE8xbIsVh2I4bs1R1iw/STpmRaNyxfh9sZl6VKzJN4eOfcULjMrkzn75zBh4wSik6PpWK4jj9Z/lJCA6xjAcCnnomH7TFuJO7YeMFC+pe0pXLWbba9WJU9ScVNxExHJs06fS+WHiEi+W3uYo7HJFCngRZ8GwQxoVJbQYjm3fmhSehKTd0xm4raJpGel079Kf4bXHk4hn0I3fvGY/bZXqVu/tw1wcPeGKl1tJa5SB/C4zm/sxKWpuKm4iYjkeVlZFn/uO82UNYdZvDOKzCyLFpWKcXvjsnSsXgLPHFpaKzop2jaAYd8sCngWYHjt4QyoOuD6BzBcimXZnr79Nagh6TT4FoYaPW0lLqSxVmnIA1TcVNxERPKVU/EpTF93lGlrj3A8LoUgf2/6hgfTv2FZQor45UiGvWf2Mn79eP489idlCpbhkfqP0Dm0s/1WhshMh/2/257C7fwFMpJtk/zW6mv7Jk5rpeZaKm4qbiIi+VJmlsUfe6KYsvoIv++OwgJaVw5iYONytK0SlCML3K88vpJxEePYc2YPFQIr0CusFzdXvJkiPkXsd5PUBNg11/Yk7sDvYGVBqTq2p3A1e4N/SfvdSxxOxU3FTUQk3zt2Npnpa48wbd1RohJSKRngQ/9GIfRrGEKpQF+H3jszK5N5B+cxffd0NkdvxsPNg3Yh7egd1psmpZvgZuxYIBNO2V6jbv3eNleccYPyrbMHNdwE3v72u5c4hIqbipuIiGRLz8zit51RfLf2CMv2RONmoH21EtzeuCytwoJwd3PsN2L7zuxj5r6Z/Lz/Z86mnqV0gdLcGnYrPSv1pGQBOz8ZO73X9hRuy3Q4exg8fKFqN9vr1Ertwd1568HK5am4qbiJiMglHIlJYuq6I/wQcZTT59IILuzLgEZl6RMeTHF/H4feOy0zjSVHlzBzz0xWnViFwdC8THN6h/WmdXBrPO1ZqiwLjq61PYXbNhOSY8Gv6N+DGoIbalCDC1FxU3ETEZErSMvIYuGOk3y35ggr98fg4WboVKMEAxuXo2mForg5+ClcZEIkP+37iVn7ZhGVFEURnyL0qNiDnmE9KR9Y3r43y0iD/b/ZnsTtngcZKVA49O9BDcXC7Hs/uW4qbipuIiJyjQ5En2Pq2iP8sD6Ss0nphBb1Y0CjstzWIJiiBR27CHxmViYrjq9g5t6Z/HH0DzKsDOoXr0/vyr3pWK4jvh52/hYvJR52/mx7EnfgD8CC0vVtBa5mbyhY3L73k2ui4qbiJiIi1yklPZP5204yZc1h1h06g5e7G11qlmRg47I0Kl/EftN6XMbp5NPM2T+HmXtncjj+MAU9C9K9Qnd6hfWietHq9r9h/AnboIYt0+HkFjDuUKGN7VVq1e7gnfNLiuVXKm4qbiIicgP2nErguzVHmLEhkoSUDCoVL8jtjcrSu34wgX6O/cDfsizWn1rPzL0zWXh4IamZqVQrUo1eYb3oVqEbAV4B9r9p1C7bU7gtP0DcEfD0s5W32v2gQltw97D/PeU8FTcVNxERsYPktEx+2XKcKWuOsOnoWbw93Lipdmlub1yW+mULOfwpXHxaPPMOzGPG3hnsit2Ft7s3ncp1oldYLxqUaGD/+2dlwdHVtu/hts+ClLNQIAhq9LKVuDL1NajBAVTcVNxERMTOth+P47s1R/hp4zES0zKpWtKfgY3Lcmu9Mvj7OH6ajR0xO5i5dyZzD8zlXPo5QgNC6RnWk1sq3kIx32L2v2FGKuxbbHuVuns+ZKZCkQq2AlerDxStaP975lMqbipuIiLiIOdSM5iz6ThT1hxm+/F4/LzcuaWO7Slc7eBCDr9/ckYyCw8tZObemWyI2oCH8aB1SGt6hfWieenmuLu52/+mKXGwY46txB36E7CgTDjUug3COqnE3SAVNxU3ERFxMMuy2BJpewo3Z/NxktMzqVUmkNsbl+WWOqUp4O3478IOxB3gp70/MXv/bGJTYinhV4JbK91Kz7CelClYxjE3jTsG2360vU49tc22rXB5qNTB9qd8S/Aq4Jh751EqbipuIiKSg+JT0vlp4zG+W3OEXScTKOjtwa31StOrfjB1gws5fF649Mx0/oj8gxl7Z7Di2AoAmpRqQq/KvWgX0g4vdy/H3Dj2AOz7zfZK9eAySE8Cdy8o1wwqdbQVuaAq+i7uKlTcVNxERMQJLMtiw5EzTFlzhF+2nCAtI4sgf286VCtOx+olaFaxGD6eDniVeYET506cn9z3ROIJCnkX4uaKN9M7rDcVCznwlWZ6ChxZZStx+xZD9C7b9sAQ23JblTrY1lD1ccCo2FxOxU3FTUREnCwuOZ2lu6NYuOMUS3dFkZiWiZ+XO63CguhYvQTtqhancAEHPQnDNrnv6hOrmbl3JkuOLiEjK4M6QXXoHdabzqGd8fP0c9i9ATh79O8Sd+APSEsANw8IaWIrcmEdoURNPY1DxU3FTUREXEpqRiar9seweOcpFu+I4mR8Cu5uhvByhelYvQSdqpekbFHHFanYlFh+3v8zM/bO4GDcQQp4FqBr+a70DutNjaI1HD6tCRlpELnWVuL2LoZTW23bC5bM/jauPVRsC76FHZvDRam4qbiJiIiLsiyLrcfiWLTjFIt2nGLXyQQAqpTwp2P1EnSsXoJaZQId8l2cZVlsit7EjD0zWHh4IckZyVQuXJleYb24qcJNBHoH2v2elxR/wrZ+6r7FsH+JbdSqcYPghn8PcihVF9zcciaPk6m4qbiJiEgucSQmiUU7T7Fox0nWHTpDZpZFiQBvOlSzlbimFYvi7WH/7+IS0hL49eCvzNw7k+0x2/Fy86JDuQ70DutNeMlw3EwOlabMDDi2Pvu16iI4vtG23a8oVMx+pVqxHRRwwFx1LsIpxc0Y8xVwExBlWVbNS+w3wHtANyAJGGxZ1gZjTAjwNVASyAI+syzrvexzXgLuBaKzL/OcZVnzrpZFxU1ERHKjM4lp/L47ikU7TvHHnmiS0jIp6O1B68q27+LaVinukCW3dsXuYubemfxy4BcS0hII8Q+hV1gvbql4C8X9cnjh+XPRcOB32LvI9lQuKQYwULre30/jgsPBEfPVOYmzilsr4Bzw9WWKWzdgJLbi1hh4z7KsxsaYUkCp7BLnD6wHbrUsa0d2cTtnWdbY68mi4iYiIrldSrrtu7iFO06xeOcpohNScXczNC5fhI7VS9ChWglCitj3u7iUjBQWH1nMzL0zWXdyHe7GnZbBLekd1psWZVrg4ZbDa5ZmZcGJjX9PORK5Dqws8Clk+yauUkfb93H+JXM2l5057VWpMSYU+OUyxe1TYKllWVOzf98NtLEs68Q/jpsNfGBZ1iIVNxEREcjKstgcefb8d3F7o84BULWkP52ql6Bj9ZLULBNg10EGh+MPM2vvLGbvn83p5NME+QbZJvet1JOQgBC73ee6JMXCgaV/j1Y9d8q2vUQtCMt+GhfSGNwdvwSZPblqcfsFeMOyrD+zf/8NeNqyrIh/nL8MqGlZVnx2cRsMxAMRwOOWZZ25Wg4VNxERycsOnU48X+IiDseSZUGpQJ/z38U1qVAULw/7fKOWnpXO8sjlzNw7k+XHlpNlZdG4ZGN6hfWifbn2eLt72+U+182ybCs37F1keyJ3dDVkZYCXP1Ro/fdr1UJOKpnXwVWL21zg9X8Ut6csy1qf/XtB4A9gjGVZM7O3lQBOAxbwCrZXqndf5t7DgGEAZcuWbXD48GE7/9OJiIi4nphzqSzZZfsubvne0ySnZ+Lv7UHrKrbv4tpUKU6gr32eQJ1KPMXs/bOZuXcmx84dw8vNi7IBZSkfWJ7ygeUJDQilQmAFQgNDKeCZw8tepcTDwT/+nnIkPtK2Pajq3yWuXDPwcFLRvAJXLW6XfVVqjPEEfgEWWJY1/nqv/U964iYiIvlRSnomf+49zaIdp/ht1ylOn0vDw83QpELR81ONlC7ke8P3ybKyWHtyLSuOreBQ3CEOxh8kMiGSTCvz/DHF/YpTPqA8oYGhtmIXYCt3JQqUcPyIVcuC6N22Uar7FsPhlZCZBp5+UL7V30WuSHnH5rhGrlrcugMP8vfghPcty2qUPdp0MhBrWdYj/zin1F/fwBljHgUaW5bV/2o5VNxERCS/y8yy2HT0DAuzX6keiE4EoEbpgPMlrnop+30Xl56ZztGEoxyMO8jB+IMcjDtoK3VxB0lITzh/nK+HL6EBoYQGhP79pC4wlHIB5fD1uPFSeUlpiXBw+d9Tjpw5ZNtepKKtwIV1hHLNwcvBq0lchrNGlU4F2gDFgFPAKMATwLKsT7IL2gdAF2zTgQyxLCvCGNMCWA5sxTYdCGRP+2GM+Qaoi+1V6SFg+D8HM1yKipuIiMjF9kefO/9d3IYjZ7AsKFPI93yJa1S+CJ7u9n8SZlkWMSkxtkIXd5BD8YfO/3z83HEs/u4lpQuU/tcTutDAUIJ8g+w38MKyIPZA9ivVRXBoOWSkgIePrbz99TSuWFiOLcelCXhV3ERERC4rOiGVJbtOnf8uLjUjiwAfD9pWLU7H6iVoXTkIfx/Hj8xMyUjhSMKRS5a65Izk88cV8Cxw8WvX7GJXNqAsXu43uN5rejIcXmEb4LB3EcT8v737j427vu84/nzHdn41juNLIIQ48VkZZQkQymJnC9oPtgkNWqnsB5v2Q+tUVZr2U/ur6v7aKlWT1rFJ036waqoQnSaVP7pupdVKh/aDMjGWHyMkBUpJsQ1pmibhjOPEJI7t9/64i0mNIWfw3fmbez4kS/bdR7637627e/nz/X4/n5eqt2/YXg1wgx+DG656ltZ7YnAzuEmSVJfJqWmerJ0X9x/fPEXl/BRdHcG+HZuqs3E7N3NDz+qm1pSZnJo89ZZDrsNnhzl5/uTcuBWxgq3rts5dGHHlRRKl1aV3N0s3NvLmunEvPwG/9LnqodQGMrgZ3CRJWrSZ2eTQ6BiPP3+Sx5//HiOvTQKwu6+Hu3du5u5bNnPz5u7Gb0r/DiYvTTJ6dnQuyF0OdSNnR7g4c3Fu3PqV698S6AZ6Bujr7qNrRZ2zidNT1cOlDV4XzuBmcJMk6T3JTI6dOjd3ccPhV18HYFtpDXfvvIG7d21mqNxLZwPOi3s3ZnOWk+dPzh1qvfLQ6+k3Ts+N64xO+rr75s6fu3wu3UDPAD2relpSu8HN4CZJ0pI6NXGBf3+hul7cfx87w9T0LN2rOtlT7mWoXGKoXGJ3Xw+ru5bfHqITUxNvztJdEepGz45yafbS3LjS6tJbZuhu23Qbvat7G1qfwc3gJklSw5y/OM2TL53miW+d4eBIZW4LrpUdK9jd18PQQImhci97+ktLtvhvI0zPTnPi3InvuyjicqirXKgA8MCPP8A9A/c0tA6Dm8FNkqSmqZyf4tDoGAdGKhwYqXD0+DjTs0kE3Ly5m6FyicFyL3sHSmzpadBabUts/OI4w+PD9K/vd8at0QxukiS1zhtTMxx+9fW5IPd/o2Ocn6ruqtDXu2bu0OpQuZcfuH5dSy92WA7eKbh1NrsYSZLUXtas7GDfjo3s27ERgOmZWb55coL9w9Ug9+RLZ/jnZ74DQO/aLvb0l9g70MtgucStN/awsnN5XPCwHDjjJkmSWiozGXltsjojN1zh4OgYw2eqW3Kt7lrBB7ZtYG+5xNBAiTu297Ju1bU97+ShUoObJEmFcmriAodGxtg/UuHgyBjPnRhnNqFjRbBry/rqOXLlEoPlEtd1r2p1uUvK4GZwkySp0M5dnOaZV8Y4MFxh/0iFw6++zoVL1S3NBza9j6Fy9dDq3nKJ/o1rC32enMHN4CZJ0jVlanqWb5wY5+BIhf3DYxwcrfD6ZHUNtuu6V1WDXH+JvQMldm5ZT8eK4gQ5g5vBTZKka9rsbPLt0+c4MPLmMiTHx6ob069b1ckd2zfMHVq9Y/uGZbkw8GUGN4ObJElt57vjb7B/uHqO3IGRCi9+b4JM6OoIbt3aMxfkBvt76X3fylaXO8fgZnCTJKntjU9e4tArleqs3HCFI8fHmZqpnid30/XrGBoo1cJcL329a1tWp8HN4CZJkua5cGmGI8fH5w6tHhoZY+LiNAA39qxmsLYEyVC5l/df382KJp0n5wK8kiRJ86zu6mDvQPUCBoCZ2eTFkxNzQe5/h1/j0WdPANCzpovB/l5++64dDJZLLavZ4CZJkkRtjbgb17PrxvX8xp1lMpPjY7Xz5EYr7B+uMDU929IaDW6SJEkLiAi2ldayrbSWX9jT1+pyAHDzL0mSpIIwuEmSJBWEwU2SJKkgDG6SJEkFYXCTJEkqCIObJElSQRjcJEmSCsLgJkmSVBAGN0mSpIIwuEmSJBWEwU2SJKkgDG6SJEkFYXCTJEkqCIObJElSQRjcJEmSCsLgJkmSVBAGN0mSpIIwuEmSJBWEwU2SJKkgDG6SJEkFYXCTJEkqCIObJElSQRjcJEmSCiIys9U1NFxEnAZGG/wwm4AzDX4MLZ59WX7syfJkX5Yfe7I8NaMv/Zl53UJ3tEVwa4aIOJiZg62uQ9/Pviw/9mR5si/Ljz1ZnlrdFw+VSpIkFYTBTZIkqSAMbkvn71tdgBZkX5Yfe7I82Zflx54sTy3ti+e4SZIkFYQzbpIkSQVhcFukiLgnIl6MiGMR8YcL3B8R8Ve1+49ExA+1os52UkdPfq3WiyMR8VRE3N6KOtvN1fpyxbihiJiJiPubWV87qqcnEXFXRByOiOci4olm19iO6ngP64mIL0fEs7W+fLQVdbaTiHgoIk5FxDfe5v6WfdYb3BYhIjqAvwXuBXYBvxIRu+YNuxe4qfb1m8DfNbXINlNnT4aBn8jM3cCn8LyRhquzL5fHfRr4WnMrbD/19CQiNgAPAh/OzFuAX2x2ne2mztfK7wLPZ+btwF3AX0TEyqYW2n4eBu55h/tb9llvcFucvcCxzHw5M6eAR4D75o25D/iHrHoa2BARW5pdaBu5ak8y86nMHKv9+DTQ1+Qa21E9rxWA3wf+CTjVzOLaVD09+VXgi5n5CkBm2pfGq6cvCXRHRADrgAow3dwy20tmfp3q8/x2WvZZb3BbnK3Aq1f8fLx222LHaOks9vn+GPDVhlYkqKMvEbEV+DngM02sq53V81p5P9AbEf8VEYci4iNNq6591dOXvwF2AieAo8AfZOZsc8rT22jZZ31nMx7kGhIL3Db/stx6xmjp1P18R8RPUg1uP9rQigT19eUvgU9k5kx1IkENVk9POoE9wE8Da4D/iYinM/NbjS6ujdXTl58BDgM/BewAHo+IJzPzbINr09tr2We9wW1xjgPbrvi5j+p/QIsdo6VT1/MdEbuBzwL3ZuZrTaqtndXTl0HgkVpo2wR8MCKmM/NfmlJh+6n3/etMZp4HzkfE14HbAYNb49TTl48Cf5rV9buORcQw8IPA/uaUqAW07LPeQ6WLcwC4KSIGaieG/jLw6LwxjwIfqV1x8iPAeGZ+t9mFtpGr9iQitgNfBH7dmYOmuWpfMnMgM8uZWQa+APyOoa2h6nn/+hLwYxHRGRFrgR8GXmhyne2mnr68QnUWlIjYDNwMvNzUKjVfyz7rnXFbhMycjojfo3oFXAfwUGY+FxG/Vbv/M8C/Ah8EjgGTVP9TUoPU2ZM/AjYCD9Zmd6bduLmx6uyLmqienmTmCxHxGHAEmAU+m5kLLoegpVHna+VTwMMRcZTqIbpPZOaZlhXdBiLi81Sv4N0UEceBPwa6oPWf9e6cIEmSVBAeKpUkSSoIg5skSVJBGNwkSZIKwuAmSZJUEAY3SZKkgjC4SWpbEbExIg7Xvk5GxHdq35+LiAcb8Hg/u8AG4pJUN5cDkSQgIj4JnMvMP2/gYzwMfCUzv9Cox5B0bXPGTZLmiYi7IuIrte8/GRGfi4h/i4iRiPj5iPiziDgaEY9FRFdt3J6IeKK2OfvXImLLvN95J/Bh4IHarN6O5v9lkorO4CZJV7cD+BBwH/CPwH9m5m3AG8CHauHtr4H7M3MP8BDwJ1f+gsx8iuo2OR/PzA9k5reb+QdIuja45ZUkXd1XM/NSbcuhDuCx2u1HgTLVvSNvBR6vbavWAbhHsaQlZ3CTpKu7CJCZsxFxKd88OXiW6vtoAM9l5r5WFSipPXioVJLeuxeB6yJiH0BEdEXELQuMmwC6m1qZpGuKwU2S3qPMnALuBz4dEc8Ch4E7Fxj6CPDxiHjGixMkvRsuByJJklQQzrhJkiQVhMFNkiSpIAxukiRJBWFwkyRJKgiDmyRJUkEY3CRJkgrC4CZJklQQBjdJkqSC+H9Z3tvHnNFfMAAAAABJRU5ErkJggg==\n",
|
|
"text/plain": [
|
|
"<Figure size 720x720 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"filenames": {
|
|
"image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter11_66_4.png"
|
|
},
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Assume that all function definitions from the example program using Autograd\n",
|
|
"# are located here.\n",
|
|
"\n",
|
|
"if __name__ == '__main__':\n",
|
|
" npr.seed(4155)\n",
|
|
"\n",
|
|
" ## Decide the vales of arguments to the function to solve\n",
|
|
" Nt = 10\n",
|
|
" T = 1\n",
|
|
" t = np.linspace(0,T, Nt)\n",
|
|
"\n",
|
|
" ## Set up the initial parameters\n",
|
|
" num_hidden_neurons = [100,50,25]\n",
|
|
" num_iter = 1000\n",
|
|
" lmb = 1e-3\n",
|
|
"\n",
|
|
" P = solve_ode_deep_neural_network(t, num_hidden_neurons, num_iter, lmb)\n",
|
|
"\n",
|
|
" g_dnn_ag = g_trial_deep(t,P)\n",
|
|
" g_analytical = g_analytic(t)\n",
|
|
"\n",
|
|
" # Find the maximum absolute difference between the solutons:\n",
|
|
" diff_ag = np.max(np.abs(g_dnn_ag - g_analytical))\n",
|
|
" print(\"The max absolute difference between the solutions is: %g\"%diff_ag)\n",
|
|
"\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
"\n",
|
|
" plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
|
|
" plt.plot(t, g_analytical)\n",
|
|
" plt.plot(t, g_dnn_ag[0,:])\n",
|
|
" plt.legend(['analytical','nn'])\n",
|
|
" plt.xlabel('t')\n",
|
|
" plt.ylabel('g(t)')\n",
|
|
"\n",
|
|
" ## Find an approximation to the funtion using forward Euler\n",
|
|
"\n",
|
|
" alpha, A, g0 = get_parameters()\n",
|
|
" dt = T/(Nt - 1)\n",
|
|
"\n",
|
|
" # Perform forward Euler to solve the ODE\n",
|
|
" g_euler = np.zeros(Nt)\n",
|
|
" g_euler[0] = g0\n",
|
|
"\n",
|
|
" for i in range(1,Nt):\n",
|
|
" g_euler[i] = g_euler[i-1] + dt*(alpha*g_euler[i-1]*(A - g_euler[i-1]))\n",
|
|
"\n",
|
|
" # Print the errors done by each method\n",
|
|
" diff1 = np.max(np.abs(g_euler - g_analytical))\n",
|
|
" diff2 = np.max(np.abs(g_dnn_ag[0,:] - g_analytical))\n",
|
|
"\n",
|
|
" print('Max absolute difference between Euler method and analytical: %g'%diff1)\n",
|
|
" print('Max absolute difference between deep neural network and analytical: %g'%diff2)\n",
|
|
"\n",
|
|
" # Plot results\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
"\n",
|
|
" plt.plot(t,g_euler)\n",
|
|
" plt.plot(t,g_analytical)\n",
|
|
" plt.plot(t,g_dnn_ag[0,:])\n",
|
|
"\n",
|
|
" plt.legend(['euler','analytical','dnn'])\n",
|
|
" plt.xlabel('Time t')\n",
|
|
" plt.ylabel('g(t)')\n",
|
|
"\n",
|
|
" plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fa8f0bb4",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"## Solving the one dimensional Poisson equation\n",
|
|
"\n",
|
|
"The Poisson equation for $g(x)$ in one dimension is"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f765b0ba",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"poisson\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation} \\label{poisson} \\tag{13}\n",
|
|
" -g''(x) = f(x)\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "bd63b92e",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"where $f(x)$ is a given function for $x \\in (0,1)$.\n",
|
|
"\n",
|
|
"The conditions that $g(x)$ is chosen to fulfill, are"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c0a7face",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{align*}\n",
|
|
" g(0) &= 0 \\\\\n",
|
|
" g(1) &= 0\n",
|
|
"\\end{align*}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f71c3cb9",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"This equation can be solved numerically using programs where e.g Autograd and TensorFlow are used.\n",
|
|
"The results from the networks can then be compared to the analytical solution.\n",
|
|
"In addition, it could be interesting to see how a typical method for numerically solving second order ODEs compares to the neural networks.\n",
|
|
"\n",
|
|
"Here, the function $g(x)$ to solve for follows the equation"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "35aa6a37",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"-g''(x) = f(x),\\qquad x \\in (0,1)\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f8b35111",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"where $f(x)$ is a given function, along with the chosen conditions"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "33813514",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"cond\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{aligned}\n",
|
|
"g(0) = g(1) = 0\n",
|
|
"\\end{aligned}\\label{cond} \\tag{14}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c70aadd4",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"In this example, we consider the case when $f(x) = (3x + x^2)\\exp(x)$.\n",
|
|
"\n",
|
|
"For this case, a possible trial solution satisfying the conditions could be"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d5719dee",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"g_t(x) = x \\cdot (1-x) \\cdot N(P,x)\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a683041b",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"The analytical solution for this problem is"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "672cdaff",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"g(x) = x(1 - x)\\exp(x)\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"id": "66e6bebb",
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"editable": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Initial cost: 457.256\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/numpy/core/fromnumeric.py:3208: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\n",
|
|
" return asarray(a).size\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Final cost: 0.00310113\n",
|
|
"The max absolute difference between the solutions is: 0.000464088\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAJcCAYAAACxEXM4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABsJ0lEQVR4nO3dd3hUVeLG8e9Jr5AEAoEESOiE3quK2MAG6tr76lrWsrrruuW3xe2ru+q6q66iq65d166grg0sgBTpvSchlBAIIb2d3x8zuDEGSEImZ8r7eR4eMnPvTN6Ze+fOmzt37jHWWkRERESkbYW5DiAiIiISilTCRERERBxQCRMRERFxQCVMRERExAGVMBEREREHVMJEREREHFAJc8AY09kY86kx5qAx5l7XeVwzxsQaY942xhwwxvzHdZ4jMcZcZYz53HWOY2GMmWyMyfPx75hjjLm2CfMdZ4xZ78ss4t+auq604u/bZow5ua1+X0scy3bGGNPdGFNijAk/xgytvlyO5blvrcflb1TCmsi78pR7V4LdxpgnjTEJLby764C9QDtr7Y9aMWag+g7QGehgrT3fdRgXjDGZxhhrjIlwnaUtWWs/s9b2c52jKYwxZxpjFhpjSo0xhcaY54wxGfWmX2WMqfVuI0qMMVu924m+9eY5tJxLGvy70M2j8m/GmLuMMc8ew+2fMsb8vjUz+ZuGxcZam2OtTbDW1rrMdayC9XE1pBLWPGdZaxOAEcBo4BfNubHxCAN6AGtsC86UG6Rv0j2ADdbaGpchgvS5/YZQeIy+YIz5DvA88ADQERgIVAKfG2OS680637uNaA+cDJQDS4wxgxrcZZL3DeXQv5d8/yh8T+uXSDNZa/WvCf+AbcDJ9S7/BXjH+/M4YB5QBCwHJtebbw7wB+ALPBvkZ4FqoAoowbOhjgb+BuR7//0NiPbefjKQB/wE2AU8A9wF/Md7XweBlUBf4GfAHiAXOLVehquBtd55twDX15t26P5/5L3tTuDqetNjgXuB7cAB4HMg9miPu5Hnb4D3uSgCVgNne6//jfe5qPY+H9c0ctu7gJeBp72PYTUwqt70rsCrQAGwFbi13rSngN83fLwNlutPgBV43lQjgJ8Cm72/aw1wTr35rwI+P8xjzAQscCWQg2dv5//Vmx5W774LvY8pxTstx3vbEu+/8d7nfKR3+mXe6dney9cCb3h/bu760/A5uNX7ODMaeUy9gbneZb8XeKnetAnAIu+0RcCEBuv9td5sRcCgetNS8bwWOh1medzhXR4HgJeAmHrT78SzjuZ7798CvQ+zPFq83je4H+NdFnc2uD4MWAX89kjrBvAO8EqDdSSiidudFOBJ7+Pdf2iZe6d9D9gE7APeArrWm2aB7wMbvY//d0AvYD5QjGfdi2rwXPzcu4y3AZfWu68zgKXe2+UCdzWyzl+DZx3+1Hv9d73P/X7gfaBHvducAqzzLt8H8axf1zby2KfyzW3D8nqv97e8j3sT8L3DPHfX8c1t7dtNXMfOBJbhWW/nAUOOsHwe8D4nxcAS4LhmbLeatJ0BHgLubfB73wZuw/N6rsPzeirB8/o4tEwijrQOAcl41s0C7/XvUG8bgPc1fJjHPQZY7H3cu4H76k072/tYi7z3MaDB6/vko22bm/i4DrseHO2596d/zgMEyr8GK08370L9HZCO5w31dDwb5VO8l1O9887Bs3EaiOcNPrKRle+3wAI8b0qpeF74v6u3YtYAd+N5Q4v1rmAVwGne+3waT/n4P+/9fw/YWu/+z8CzATbACUAZMKLB/f/We9vTvdOTvdMf8j6GdCAczxtv9NEed4PnLtL7Ivk5EAVM8b4w+tV7wTx7hOf+0OM93ZvhT8AC77QwPBu/X3nvuyeeN9zTvNMbPteT+fab/jLvMj1ULs/H8wIPAy4ESoEu3mlXcfQS9ph3OQ3FU+wGeKff5l3OGd7n8FHghQa3jah3f08DP/L+PBPPBvvGetNub+H68/VzAPwS+Kqx5ead/gKe9SoMiAEmea9PwbPhvhzPOnix93KHeuv9td6fnwD+UO8+bwLeO8LyWOh9/lPwvJHf4J02FU+RHAjE4dlQH6mEtXi9b3A//b2/J6uRab/Bs/frsOsGnkKy+3DL+SjbnVl4SkKyN+cJ3uun4ClMI7zL9R94C5B3usXzBtWO/+21+wjP66M9njf9Kxs8F/d57+sEPOt8v3rTB3vXgSF43nRnNHg8TwPxeNavGXhe7wO868YvgHne+TvieeP+jvfx3O793Yd7s7+LBtsGPKXtYTzr4zA8JeKkw9z+Keq9/puwjo3AU8rH4tnWXOmdP/ow938Z0MH7OH+EZ/2MOdp2qznbGTyFJx8Iq/cclgGdG743NbaOcfh1qANwHp7XUiKeP+zfqHc/c46wXOYDl3t/TgDGeX/u630cp3h/153edSGqYdaGy4bGtwVHelyHXQ+O9tz70z/nAQLln3eFKMHT7rd7F34snj0MzzSY933+t4Gbg/cv5XrTG658m4HT610+Ddjm/Xkynr/k6v+ldhfwQb3LZ3mzhXsvJ3pX1qTDPJY3gB/Uu/9yvvnmvwfPXq4w77ShjdzHER93g+uPw7NxCqt33Qt4/6KmaSXsw3qXs4Fy789jgZwG8/8MePIwz3VjL/TvHmXZLwOme3++iqOXsPp/TS4ELvL+vJZ6bxZAFzx/qUfQeAm7Bnir3m2vBV70Xt7O/wpFc9efycAOPG+6nwPtj/DYn8ZTADMaXH85sLDBdfOBq+qt94dK2MnAlnrzfQFccYTlcVm9y/cAj3h/fgL4U71pvTlCCWvpet/I7SZ5f09MI9NuADYead3AUx6rG6wjRQ3+DWjkdl3w7A1orBj+C7in3uUE77qU6b1sgYn1pi8BflLv8r3A3+o9FzVAfL3pLwO/PMzz+Dfg/gaPp2e96e9Sb482nu1IGZ7DDq7gm0XE4NkL16QShuePpVogsd51fwKeOsztn6LxEna4deyfeP+AqTd9Pd7i0oR1bD/e7SVH2G4d5rbLOMx2Bs/r/xTvzzcDsxs8nkbLypHWoUZ+/zBgf73Lc46wXD7F8wdIxwbX/xJ4ucGy34H3UxJaqYQdbT1o7nPv8p+OCWueGdbaJGttD2vt96215Xg2LOcbY4oO/cOz0e5S73a5R7nfrnjeVA/Z7r3ukAJrbUWD2+yu93M5sNf+74DFcu//CQDGmGnGmAXGmH3efKfj+WvqkEL7zeOxyry37Yjnr4zNjWRuyuOu//hyrbV1DR5jeiPzHs6uBvlivMef9AC6NsjxczwH+jfVN5aPMeYKY8yyevc3iG8+X83NeugLHD2A1+vd71o8G5LDZZ0LHGeMScPz19xLwERjTCaevRnLvPO1ZP1JwvNxzZ+stQeO8FjuxPNGudAYs9oY893D/M5Dv7exZfoxEGuMGWuM6YFnY//6EX7n4Z6/rnxzWR3xdXUM631De73/N7Zud6k3/XDS8XxkUl9H77bk0L+1jdyuG7DPWru/kWnfeP6ttSV49kTXf/4bbiMaXq7/WPdba0vrXf56HfIut0+MMQXGmAN4imfD10P9ZdEDeKDeer4PzzqUToNlaD3vkEfbPtbXFc9zcrBB1uZsS+DIr9EfNdiedOObr6evGWN+ZIxZ6/1mdxGe12X95+Zw263mbmf+jWevG97/n2nawzz8OmSMiTPGPGqM2W6MKcZTrJKa+O3Da/Ds9VpnjFlkjDnTe33D9bIOz/Jt7vI5mqasB4d97v2JStixy8WzR6j+BjXeWvvnevPYo9xHPp4X/yHdvdc19faHZYyJxnO81F/x7L5OAmbj2SgezV48u3R7NTKtKY/7kHygm/dLCYd0x/MX0rHKxfPRa/0cidba073TS/Hsbj8krZH7+Pr59ZaEx/D8tdnB+3ytomnPV1OyTmuQNcZau4NGlrG1dhOejceteD5qOohnw3Idnr+SD5Xalqw/+/Ec+/KkMWbi4QJba3dZa79nre0KXA88bIzp3cjvPPR7v7VMvTlfxvOR5SV4jqU82HC+JtiJ56PcQ7odbsZjXO8bWo9nb803vrnrXZ/Pw/Mx35GcA3zWgt+bC6QYY5IamfaN598YE4/n46WWvqaSvfdxSP116Hk8H212s9a2Bx7h289j/XUsF8/xd/XX81hr7Tw8y/Dr5WaMMRxhOfLtdTcfz3OS2CDr4R53c7eduXg+Oq+fPc5a+0LDGY0xx+H5ROACPHuakvAcY3bUdawF25lngenGmKF4PuZ9o960Iz3GI61DPwL6AWOtte2A4w/FO1p+a+1Ga+3FeA6BuBt4xbv+NFwvDy3fxpbP0bbNR3pczV0P/JZK2LF7FjjLGHOaMSbcGBNjPOdhyjjqLf/nBeAXxphUY0xHPMc3tfhr2Q1E4TnOowCoMcZMA05tyg29b55PAPcZY7p6H9947xtccx73l3hecHcaYyKNMZPxfIT64jE/Os/HfcXGmJ8Yz/nGwo0xg4wxo73TlwGnG2NSvHuUbjvK/cXjefEXABhjrsbzF2preAT4g3cDjHd5T/dOK8DzsUHPBreZi2dDPdd7eU6Dy9DC9cdaOwe4FM/eubGNzWOMOb/eMt2P57mpxVNo+hpjLjHGRBjPKRay8Rzc25jn8Rz3cqn355Z4GbjaGDPAGBOH53EeTovX+4a8e2vuwPMcX+Jdz9KAx/Ecc3V/w9t418MsY8w/8HzM8psW/N6deD7ae9gYk+x97Rx6o3wez3MxzPt6/CPwpbV2Wwse4iG/McZEecvFmXiOEQLP4Q37rLUVxpgxeIr0kTwC/MwYMxDAGNPeGHOowM4CBhpjzvXulbiVxv8wOmQ3kHnoDzhrbS6eYx7/5N3mDMGzV+a5I9y+4WvqSB4DbvDu/TPGmHhjzBkN3uwPScTzMW4BEGGM+RWe9aEpmrWdsdbm4fnyyzPAq95PYQ457GM8yjqUiGePaJExJgX4dROzY4y5zBiT6n2PKPJeXYvnNXqGMeYkY0wknqJXiWeZNbSMI2+bj/S4mrse+C2VsGPkXRmm4/kIrADPXx4/pnnP7e/xfNNkBZ5vOn7lva418h3Es6F7Gc+b6CV4/qptqju8mRbh+VjhbjzHdjX5cVtrq/B8Y2Yanr1rD+M5Jmhdyx7VN+67Fk+hG4bnywl78bw5tvfO8gyeb25uA/6L5yO9I93fGjzHy8zHsxEYjOcYptbwAJ7n/r/GmIN4DqYf6/29ZXi/Rev9eGKc9zZz8WwsPz3MZTiG9cda+wGebxG+ZYwZ2cgso4EvjTEl3uw/sNZutdYW4nmj/hGej8HuBM601jb60Zy19lAR74rnTaHZrLXvAn8HPsFzsO9876TKRuY91vW+4f29hOc4uNvxrGNr8BwTOtH7XBwy3vtcFeMpzO2A0dbalQ3ussh88zxhPzzMr74cz7Fe6/Acs3abN89HeI6/eRXP3qVewEUtfXx49rDux7OH4Tk8B6ofen1+H/itd539FZ7n9LCsta/j2U68aDwfc63C89rHu36cD/wZz3rThyO/vg4VwUJjzFfeny/Gc3xQPp6PtX/tXY8b8y8g2/uaeuNIub35FuP5YtODeJ6PTXiOz2rM+3jW5Q14PgqroIkfrbZwO/Nv73wNP4r8E54/EIqMMXc0crtG1yE8x/bF4lmfFwDvNSW711RgtXddfwDPca8V1tr1eD4u/Yf3fs/Cc2qnqkbu42jb5qM9ruasB37LeP7IExEJLMaYAXje4KOt43PMBTLvnulnrbXN2Xsvbcy7B+tZPF++qDva/BIYtCdMRAKGMeYc70dmyXj2trytAibBzvvR3g+Ax1XAgotKmIgEkuvxfPy9Gc8xKDe6jSPiW949vkV4von7N6dhpNXp40gRERERB7QnTERERMQBvztx2dF07NjRZmZmuo4hIiIiclRLlizZa61NbWxawJWwzMxMFi9e7DqGiIiIyFEZYxqOLvI1fRwpIiIi4oBKmIiIiIgDKmEiIiIiDgTcMWEiIiLSOqqrq8nLy6OiosJ1lIAXExNDRkYGkZGRTb6NSpiIiEiIysvLIzExkczMTIwxruMELGsthYWF5OXlkZWV1eTb6eNIERGREFVRUUGHDh1UwI6RMYYOHTo0e4+iSpiIiEgIUwFrHS15HlXCRERERBxQCRMREZGA9tRTT3HzzTcfdZ78/PyvL1977bWsWbOm2b9rzpw5nHnmmc2+XWNUwkRERCToNSxhjz/+ONnZ2Q4TqYSJiIiIQzNmzGDkyJEMHDiQmTNnApCQkMD//d//MXToUMaNG8fu3bsBePvttxk7dizDhw/n5JNP/vr6Qw4ePEhWVhbV1dUAFBcXk5mZyX/+8x8WL17MpZdeyrBhwygvL2fy5MlfD4P43nvvMWLECIYOHcpJJ50EwMKFC5kwYQLDhw9nwoQJrF+/vtUfu05RISIiIvzm7dWsyS9u1fvM7tqOX5818IjzPPHEE6SkpFBeXs7o0aM577zzKC0tZdy4cfzhD3/gzjvv5LHHHuMXv/gFkyZNYsGCBRhjePzxx7nnnnu49957v76vxMREJk+ezKxZs5gxYwYvvvgi5513Hueffz4PPfQQf/3rXxk1atQ3fn9BQQHf+973+PTTT8nKymLfvn0A9O/fn08//ZSIiAg+/PBDfv7zn/Pqq6+26vOjEiYiIiLO/P3vf+f1118HIDc3l40bNxIVFfX1cVcjR47kgw8+ADznNbvwwgvZuXMnVVVVjZ6T69prr+Wee+5hxowZPPnkkzz22GNH/P0LFizg+OOP//q+UlJSADhw4ABXXnklGzduxBjz9d611qQSJiIiIkfdY+ULc+bM4cMPP2T+/PnExcUxefJkKioqiIyM/PqUD+Hh4dTU1ABwyy238MMf/pCzzz6bOXPmcNddd33rPidOnMi2bduYO3cutbW1DBo06IgZrLWNnl7il7/8JSeeeCKvv/4627ZtY/Lkycf8eBvSMWEiIiLixIEDB0hOTiYuLo5169axYMGCo86fnp4OwL///e/DznfFFVdw8cUXc/XVV399XWJiIgcPHvzWvOPHj2fu3Lls3boV4OuPI+v/rqeeeqpZj6upVMJERETEialTp1JTU8OQIUP45S9/ybhx4444/1133cX555/PcccdR8eOHQ8736WXXsr+/fu5+OKLv77uqquu4oYbbvj6wPxDUlNTmTlzJueeey5Dhw7lwgsvBODOO+/kZz/7GRMnTqS2tvYYH2njjLXWJ3fsK6NGjbKHvs0gIiIiLbd27VoGDBjgOkare+WVV3jzzTd55pln2vT3NvZ8GmOWWGtHNTa/jgkTERGRoHHLLbfw7rvvMnv2bNdRjkolTERERILGP/7xD9cRmkzHhImIiIg4oBImIiIi4oBKmIiIiIgDOiZMREQ86uqgphxbVUZVRSnlZSVUlB2koqyEqvISqspLqa4oo7ayhNrKUuqqyqirKofqMkx1OaamnPDacsJrKwirq6Zi1A2Mn3qJ60cl4rdUwkREAkFtDVSXQXW59//6P5dTXVFKVXkJleWl1FSWUF1RSm1lGXXesnRoPlNTTlhNBeG15UTUVhBRV0FUXSVRtoIYqgAwQLT331FjWUO5iaGSaKrCoqk2MVSHx5BQu4/282/lq3YdGTHhVF8+MyIBSyVMRKStHdhB7cLHqSnbT01FGbVVZdiqMmxVqbdYlRNWb69SZF0F4bbmiHcZ6f0XX++6ShtBBVGUE025jaKcGCqIoioshpqwJGrCY6iNjKEuIhbr/Wei4iAqjvDoeMKj4oiIiScqJp7ImHii4xKIjk0gNi6BuPh2xCUkEhUVTYIxJDTIU7JvJ/senEyP969hY8os+vQf0spPokjgUwkTEWlDtmA9pY+fRUxFASXEU040FTaKcm9Z8vzcjnI6Um6jqTDR1IbHUBce6ylLkbGYyDhMVCxhUfGERcURHpNAVEwckbEJRMXGExObQExsPPGxMcRHR5AQHUH76Ai6RocTHRHeJo8zIaULZVe8SsRTpxH14oXsuvEj0jp3bZPfLYFj27ZtTJs2jUmTJjFv3jzS09N58803mTZtGmPHjuWTTz6hqKiIf/3rXxx33HGu47Y6lTARkTZSl7uEin+fQ3k13JfxTzIGjCU+Opz46Iivy1LHKM//h66PjghrdHDhQNApcxDbzn6Krm9exPqZ3yH+9v+SmNBwn5n4jXd/CrtWtu59pg2GaX8+4iwbN27khRde4LHHHuOCCy7g1VdfBaCmpoaFCxcye/ZsfvOb3/Dhhx+2bjY/oBImItIGajZ9Qs3zF1NYk8Drgx/iF+edSlhYYJar5sgccQpr9/6FwfNuZ97DlzH6h68QGaG3HvmfrKwshg0bBsDIkSPZtm0bAOeee+63rgs2eiWIiPhY5Yo3CHvtWrbVpTF//ExumTo+YPdutcSAU7/Lin3bmLDuAT569DamfP8fIfX4A8ZR9lj5SnT0/74CEh4e/vXg2oeuDw8Pp6bmyMdEBiqdJ0xExIfK5j9BxGtXs6Iuk5WnPM/V0yaEZAEZcuFvWNlpOicVPMNHz//VdRwRv6ASJiLiIwc//Atx79/OF3WD2XvOS5x/XAh/Q9AYBl33OOviRzN5wx/5/L2XXCcScc5Ya11naJZRo0bZxYsXu44hInJ41lL01k9JWvoIs+xEUi55nPH99M1AgKrSInbdP5mU6l1sOvNVho2e6DpSSFu7di0DBgxwHSNoNPZ8GmOWWGtHNTa/9oSJiLSm2hr2v3AdSUsf4SUzlW7XPqsCVk9UfBJJ33uTirBY0mZdzuYtG11HEnFGJUxEpLVUV7DvyYtI3vAy/wq/kNE3Ps6QbimuU/mddp17UHvxSyRSSu0z51Owd6/rSCJOqISJiLSGimIKZ55FSt4HPBRzHWfc+nd6dkp0ncpvde47hoKpj9KzbjvbHr2Q0vIK15FCVqAdluSvWvI8qoSJiByrkgL2PXwq7fYs4W/tfsylt/6BtPYxrlP5vcxxM9g0+jeMrl7Mlw9dQ01NretIIScmJobCwkIVsWNkraWwsJCYmOa97nWeMBGRY2D3b6do5lnElu3kH51/ww3X3khclDatTdX/zFtZVbiVKVuf4N1//YKp1/0xJE/h4UpGRgZ5eXkUFBS4jhLwYmJiyMjIaNZttKUQEWmhut1rKXn8LMKqSnm0x73ccuVlRIbrA4bmGnT5vax9MIdpOx/mv6/04NTzb3AdKWRERkaSlZXlOkbI0tZCRKQFanIWUj7zVCqqqnl+wCPcetXlKmAtFRZGvxueZUvMIE5Y9Qs+/3iW60QibUJbDBGRZqpc/yE1T57F3uoY3hvzFDdceHZIjAPpS2FRsXS98XX2RaSSPfd6li//ynUkEZ9TCRMRaYbSr14h7IUL2VrbicUnvcAVZ5yoY5haSUz7TsRd/RrhBpJev4StuTmuI4n4lEqYiEgTFX8+k9i3rmVFXS9yp7/CeSc0ehJsOQbtMwZQft6zpLGX4icvYG/RAdeRRHxGJUxE5GisZd97f6Ldhz/mMzuMqkte5dSR/VynClppgyeTP/l+htatZc0/L6e8stp1JBGfUAkTETmSujr2vnoHKQv+zCxzPMnX/Ifx/bu5ThX0siZfzobBd3B85Vw++ect1NbpPFYSfFTCREQOp7aaPc9eQ8dVj/Ny+BkMuPE5hnRPdZ0qZPQ99xesS/8Opxe9wKyn/uQ6jkirUwkTEWlMdTl7Hj+fTlte46noSzj+ln/Rs1M716lCizH0/+6jbGo3jtO3/4X33nzOdSKRVqUSJiLSUMUB9jx8Oh3z5/BowveZ8YMHSEuKdZ0qNIVHkHXjf9gZncXEr37EF59/4jqRSKtRCRMRqcce3E3Bg6eQvG85j3T8GZff+juS4qJcxwpp4bHt6Hj9m1SGx9Hrg++ycu1a15FEWoVKmIiIV92+bex/cArxB7fxeLc/8r3v/1jjQPqJ2A7diLj8FRJNOVEvXcT2/N2uI4kcM5UwERGgeucqDj58EmEV+3m+39+5/rvXaxgiP5OUNYLisx6nFzns+tdF7DtY5jqSyDHRFkZEQl7llvlUPjaViuoaZo38F9dcfKGGIfJTXUaeSd6E3zO29isWP/xdKqpqXEcSaTGVMBEJaaWr38c+PZ2CmjjmT36eS8+epmGI/FzmqTexqe91nFr+Lu8++jPqdA4xCVAqYSISsooWvkDUfy5hq01j81mvMuPEia4jSRP1vuhuNnU6jXMKZ/Lmcw+6jiPSIiphIhKS9n7yMO1m38gy24cDF77JyaMHu44kzREWRq/v/Ztt8UM4fdNveG/2664TiTSbSpiIhBZr2f32b+k492d8xkhirn6DcdlZrlNJC5jIWDJueJ39kZ0Y8+XNfPHll64jiTSLSpiIhI66Ona+dBudl9zLu2GTybjhFQZnprlOJccgIrEj7a99g/CwMDJmX8HqTVtcRxJpMpUwEQkNtdXkP3UFXdY9xStR0xl+6wv0Skt2nUpaQWxaX2ovep40s4+aZy8id88+15FEmsSnJcwYM9UYs94Ys8kY89MjzDfaGFNrjPmOL/OISIiqKiP/0XPpmvM2z8Zfyck/eIy0pDjXqaQVpfQ7jn2n/oPBbGDzzMs5UFrpOpLIUfmshBljwoGHgGlANnCxMSb7MPPdDbzvqywiErps+X7yH5pG2u7PeDL5B5z7g/tIio92HUt8oMuEi8gd+TMm13zOJw/fTGVNretIIkfkyz1hY4BN1tot1toq4EVgeiPz3QK8CuzxYRYRCUF1xbvY8/eT6Vi0kqfSf82lN92lYYiCXI8z72RL5sXMKH2ZNx/7PdbqHGLiv3xZwtKB3HqX87zXfc0Ykw6cAzxypDsyxlxnjFlsjFlcUFDQ6kFFJPhU793Cvn+cSEJZLi/2uZerrr2NqAgdBhv0jKHn5Q+yLWUS5+76G6+99JTrRCKH5cstUmOnnG74J8nfgJ9Ya4+4z9haO9NaO8paOyo1NbW18olIkKrIW0HpP08ivKqYWcMe5fJLr9IwRKEkPIIe17/I7thenLb2p7z34X9dJxJplC9LWB7Qrd7lDCC/wTyjgBeNMduA7wAPG2Nm+DCTiAS5kg2fU/OvaZTXwOfHPcMF55yjYYhCkIlOpNMNb1IZkciwz65j/rIVriOJfIsvS9gioI8xJssYEwVcBLxVfwZrbZa1NtNamwm8AnzfWvuGDzOJSBDbv+wdIp4/hz11iayd9h/OOnmK60jiUGRSOrFXvUaiqSTl9UtZuy3PdSSRb/BZCbPW1gA34/nW41rgZWvtamPMDcaYG3z1e0UkNO354hkS37iCzTadgu+8wZRxo1xHEj8Q120Ilec+RS+zgwP/voT8wmLXkUS+ZgLtmyOjRo2yixcvdh1DRPxI/vt/o+v8X7OYbGKueJlBPbsd/UYSUvI/mUnXuT/m3ahTmXj7c7SLjXIdSUKEMWaJtbbRvwr1VSERCVzWkvvaL+k6/9fMDRtLyvVvq4BJo7qeeB05g25iWtV/ee+RO6murXMdSUQlTEQCVF0dOc/eRLcVf+e9yJPoe8ur9OzS0XUq8WPdz/sD27uewQUHnuTlJ+/XOcTEOZUwEQk8NVVsf/wSum9+jjfizmPsD16gS3Ki61Ti74yhx3efJK/dcL6T+0deee0/rhNJiFMJE5GAYitL2P7wdHrkv8tLSddy6m0zSU7QMETSRBHRpN/wGkXRXTl5xe28P/cz14kkhKmEiUjAqCvdR97fTyOjcD4vdL6Dc27+i4YhkmYzcSkkf+9NwsLDGfDxd1m4ar3rSBKiVMJEJCBUF+1g99+n0KlkHa/0/D0XXv8LDUMkLRaV2pPwS1+msyki+pVL2bhDwxdL29MWTET8XsWujRQ9OIXEip3MHvx3LrjiJg1DJMcsodc4Ss78J4PZRN6/Lmf3gTLXkSTEqISJiF87uG0p5TNPIby6hM8mPME537lUwxBJq+kw6jvsHvdLTqxbwOf/vInSyhrXkSSEqISJiN/at2YO5qkzqKg1rDj1ZaaddobrSBKEupz2Q/L6XM55Fa/x6qN3UaNziEkbUQkTEb+0a9EbxL98Pntse/LOeZPJEye6jiTByhgyLn6AvE4ncGnhgzz37GM6h5i0CZUwEfE7OZ8+Q8dZV7OJbpRd+g6jhw1xHUmCXVg4Gde+wJ6Efnxnyy955Z1ZrhNJCFAJExG/cnDXJlI//hGrTD9ivzebQX17uY4koSIqns7XvUFlZBLHL76JD+ZrnGLxLZUwEfEf1rLr2eupsWFEXvAEPdPTXCeSEBPWvgsJ17xGYlgV3d+7isXrtrqOJEFMJUxE/MbG9/5Jn5LFfJF1CwMHZLuOIyEqqssg6i54hl4mn+oXL2fTzv2uI0mQUgkTEb9QtjeHLl/+jqXhg5h8yZ2u40iISxhwMgdOvo/xrGTd49dQUFzhOpIEIZUwEXHPWnKevoFwW0PE9AeJiYp0nUiEDpOuYtfw2ziz9iPef+QOyqp0DjFpXSphIuLcpo+fon/xF3yacT2Dhwx3HUfka2ln30V+j+lcVvYMzz32V2rrdOoKaT0qYSLiVPn+3XT87FesDuvLcZf/0nUckW8yhq6XP87O5NFcsecvPP3CczqHmLQalTARcWrLMzcRZ0upOuMB4mKiXccR+baIKLpc9x+KYzM4Z8OdvPL+x64TSZBQCRMRZzZ9+hID933Ap12uZvjICa7jiBxebDIdrnuTsIgoxs6/no8WrXKdSIKASpiIOFFxcB9Jn/yEjSaTcVf83nUckaMKS8kk+or/0NkU0+GdK/lqc77rSBLgVMJExIkNT99KUt0Bik/7Gwlxsa7jiDRJdI/RVM6YyRCzmYLnr6dOB+rLMVAJE5E2t2XB2wwpeJtPUy9m5LgTXccRaZZ2w2awuc81nFLzGUvXrHEdRwKYSpiItKmqsmLi3v8h2+jKyCvvdh1HpEW6nXQDYcay67NnXEeRAKYSJiJtavUzd9CproC9U+6lfWKi6zgiLRKT1odtsQPpu+sdKnQSV2khlTARaTNbv/qQofkv83nKOYw6/nTXcUSOSe3gC+ljclm4YK7rKBKgVMJEpE1UV5QS+c6t7DSpDL7iXtdxRI5Z5vGXUU0E5Yufdx1FApRKmIi0iRXP/YyMuh3kH/dnkpNTXMcROWbhCR3YkjyR4Qc+pLC41HUcCUAqYSLic9tXfcHQnGf4ot3pjD7pPNdxRFpNwpjL6GSKWPLJG66jSABSCRMRn6qpqqDu9ZvYZ5Lof8UDruOItKr00dM5aBKIWvOy6ygSgFTCRMSnlr5wF1m1W9k67nd06NjJdRyR1hURTV7XqYypmM+WHbtcp5EAoxImIj6Tu24JQ7fMZGH8iYw57VLXcUR8ovNxVxFnKln/8XOuo0iAUQkTEZ+oramh4pUbKTFxZF3+IMYY15FEfCKl3yR2R3Sl49Y3NIyRNItKmIj4xJKX/kCfmvVsHPErUtMyXMcR8R1j2N9rBiNrV7J89WrXaSSAqISJSKvbsXkVQzb8g69ixzPmzGtdxxHxuR4nXk2Ysez8QsMYSdOphIlIq6qrreXASzdSRSTpl/0TE6bNjAS/2LS+bI0dSJ9dszSMkTSZto4i0qoWvXof2VUrWDvkJ3ROz3IdR6TN1A66iD7ksmjBp66jSIBQCRORVrMzZyMDV9/LyujhjDnnVtdxRNpU1uTLqCKC8sX6lqQ0jUqYiLQKW1fHnudvJIw6Olz0iD6GlJATHp/C1uSJDNMwRtJE2kqKSKtY+ObDDK1YxKoBt9E1q7/rOCJOxI+5lE6miK/mvOE6igQAlTAROWZ78nPov/yPrIvMZtT5P3EdR8SZjNEzPMMYrdYwRnJ0KmEickysteQ8+31ibBUJFzxCWHi460gi7kREk9t1KmMq5rFVwxjJUaiEicgxWTj7SUaVfcby3jeQ0Weo6zgizqUddxWxpor1n+gAfTkylTARabGCPTvpteguNkf0ZuTFv3YdR8QvfD2M0ZY3sFbDGMnhqYSJSItYa9n09K20tyVEnvsw4RGRriOJ+AdjKOx1DiNqV7J8zRrXacSPqYSJSIss+uAlxpf8l+WZV9M9e6zrOCJ+JfPQMEafaRgjOTyVMBFptn2Fe+k+7+fkhHdn2CW/dx1HxO/EpfVha+wg+ux6R8MYyWGphIlIs61++nZS7T7s2Q8SER3rOo6IX6oZdAG9yWXJl3NdRxE/pRImIs2y8JM3Oe7AWyzPuIQeQ09wHUfEb/WcfDlVRFCmYYzkMFTCRKTJig4U0WXuneSHdWHw5fe4jiPi18LjU9iSNJFhRR+y/2CZ6zjih1TCRKTJlv37Trqxi8ppfyMyJsF1HBG/Fz/6UlLNAb765HXXUcQPqYSJSJMs+eK/HFf4Mss7n0PW6Kmu44gEhG5jZ1BsEonQMEbSCJUwETmq4pISkj/8IfvCOtD/ivtdxxEJHBHR5HbxDGO0LX+36zTiZ1TCROSoFj39c3raXIpP+SvR8cmu44gElLTjriTWVLFOwxhJAyphInJESxd+xvG7n2Vlh2n0mnCO6zgiAadD/0nsiuhKx82vaxgj+QaVMBE5rNLyCmLf/QEHwxLoc8U/XMcRCUzGsK+nZxijFRrGSOpRCRORw/rimbvobzez/4Q/EtM+1XUckYCVOcU7jNHnT7uOIn5EJUxEGrVs6SJO2PE4a5Im02vyZa7jiAS0uLQ+bIkdRO+ds6is1jBG4qESJiLfUl5ZTdjbt1Bposm84mHXcUSCwqFhjBYv+NR1FPETKmEi8i1zn/0jQ+rWsnvCXcSlpLuOIxIUep5wGVVEUL74WddRxE+ohInIN6xctYLjch5iQ+JY+pxyres4IkEjIqEDm5MmMbToQ4pKNIyRqISJSD0VVTVUvX4zxhjSL58JxriOJBJU4rzDGC3RMEaCSpiI1PPxC/cxsnY5O0b/jPhOma7jiASd7mOnU2wSidIwRoJKmIh4rV63nklb7mdL3FD6TLvVdRyRoGQiosnpMpVR5fPI2alhjEKdSpiIUFldQ/ErtxBlaki97DEI06ZBxFe+HsboYx2gH+q0pRURPvzPI4yv+ZLcYbeT2LWf6zgiQa1j/0nsCu9KBw1jFPJUwkRC3LrNWxi//s9sjxlAn7N/4jqOSPAzhsJeMxheu4qVa9e6TiMOqYSJhLDq2jp2vXgbiaac5EtmQli460giISFzyjWeYYw++7frKOKQSphICHv/tSeYXD2Xbdk30q77ENdxREJGfFpvNscOotfOWVRV17qOI46ohImEqE05eYxe9Xt2RPWkz7m/ch1HJOR8PYzRl3NdRxFHVMJEQlBNbR1bn7udjuYA8Rc8AhFRriOJhJxeJ1zuGcZokb4lGapUwkRC0Htvvcgplf9lS5/vktR7rOs4IiEpIiGFzUmTGFL0IQdKyl3HEQdUwkRCzJYduxm67Ffsisyg9/m/cx1HJKQdGsboqzmvuo4iDqiEiYSQ2jrL2md/TLrZS/S5D2Oi4lxHEglph4Yxilj1H9dRxAGVMJEQ8u7s15lW9hZbMi8mecAJruOIhDzPMEanMap8Hrk797iOI21MJUwkRGzfXUj2ov9jX0Qnel18j+s4IuLV+biriTVVrNUwRiFHJUwkBNTVWZY+81N6mnzM9Acw0YmuI4mIV2r/iewMT6fjFg1jFGpUwkRCwLsfvMeZB19hS8Y5dBgyzXUcEanPO4zRsJqVrF67xnUaaUMqYSJBLm/vAXrN+wkHw5PIuvR+13FEpBGZU672DmP0tOso0oZUwkSCmLWWBU//kv5mOzXT7sXEJruOJCKNSEjrw+aYQfTc+Y6GMQohKmEiQez9T+Zw1oHn2JZ2Gqmjz3UdR0SOoHrQBfQij68WahijUKESJhKkdu0vpcunP6YyLI7ulz7kOo6IHEWvyZ5hjEo1jFHIUAkTCULWWuY8/VuGspHKk/9IWGKq60gichSRCSlsSprEkP0fcqBUwxiFApUwkSD0wefzmL7vCbZ3PIHUCZe5jiMiTXRoGKOln7zmOoq0AZUwkSCzp7iM5I9+TF1YBBmX/ROMcR1JRJqox9jpHDCJRKx62XUUaQMqYSJBxFrLf5++m9GspvSE3xCelO46kog0g2cYo6mMKv+CvJ27XccRH1MJEwkiH335FdMLHiUveQydTvie6zgi0gKdJ11JjKlmnYYxCnoqYSJBovBgBTHv/YhIU0fapTP1MaRIgOo0YBL54el00DBGQU8lTCRIzHruASaxlAMTfkZExyzXcUSkpYyhsNd0htasYs26ta7TiA+phIkEgU+WrOKsnX8nv90QOp98q+s4InKMMqd8lzBjyf/0366jiA+phIkEuKKyKmrfuYN4U0nqpY9BWLjrSCJyjBLT+rApdjA9d75DdY2GMQpWKmEiAe715x/hZDuf/aNvJ7Jzf9dxRKSVVA883zOM0ZcaxihYqYSJBLBPl6/nzNy/sju+H52n3uk6joi0ot5fD2P0nOso4iMqYSIBqriimoNv3kmyKSH54pkQHuk6koi0ov8NY/SBhjEKUiphIgHqlRef5Iy6ORQOvZGojGGu44iID8SOupSO5gDL57zqOor4gEqYSACat2YLp239M3tjMul81q9cxxERH8kc5xnGKHylhjEKRiphIgGmvKqWXa/+jC5mH4kXPQoR0a4jiYiPmIhotqdNZWT5PHbs0jBGwUYlTCTAzPrvu5xb+x67B1xFdOY413FExMe+HsboEx2gH2xUwkQCSHFFNWmL/0JJWCJdpv/GdRwRaQOdsz3DGKVsek3DGAUZlTCRAPLu268wiWUcHH0rxLR3HUdE2oIx7O05g6E1q1i3XsMYBROVMJEAsfdgBb1X3U9ReEe6nHyL6zgi0oYODWO0Q8MYBRWVMJEA8cGbzzDSrKdq0h0QGes6joi0oXZderMpZjA98zWMUTBRCRMJAPn7Sxm28R/sjUqn0/HXuo4jIg5UDTyfnuSxbOEc11GklaiEiQSAua8/ygCzHXPiz3VmfJEQdWgYo5KF+pZksFAJE/FzW3cXMX77o+yO6UWHsZe4jiMijkQlprAxaRKD939IcZmGMQoGKmEifm7Ba/8g0+wiZupdEKaXrEgoix15CR3NAZZ98prrKNIKfLpFN8ZMNcasN8ZsMsb8tJHp040xK4wxy4wxi40xk3yZRyTQrM3ZzeRdT7AjYTDth57lOo6IOJY1fgYHSCR81Uuuo0gr8FkJM8aEAw8B04Bs4GJjTHaD2T4ChlprhwHfBR73VR6RQLTi9fvoYvbR/szfgzGu44iIYyYimm1dpjKybB75u/e4jiPHyJd7wsYAm6y1W6y1VcCLwPT6M1hrS+z/Tv8bD+hUwCJeSzdu55R9z5KTPI6E/pNdxxERP5F23FWeYYw+esZ1FDlGvixh6UBuvct53uu+wRhzjjFmHTALz96wbzHGXOf9uHJxQUGBT8KK+BNrLZvevJsUU0LqjD+4jiMifqTzgImeYYw2v65hjAKcL0tYY5+dfGttsda+bq3tD8wAftfYHVlrZ1prR1lrR6WmprZuShE/tGDVBqYdfJVtnU4mtsco13FExJ94hzEaVruS9RrGKKD5soTlAd3qXc4A8g83s7X2U6CXMaajDzOJ+D1rLbtn/YlYU0XXcxv9u0REQlzmiZ4PjjSMUWDzZQlbBPQxxmQZY6KAi4C36s9gjOltjOdoY2PMCCAKKPRhJhG/N2fRcqaVv0NOxtlEpTX8LouICLTr6hnGKCv/HWo0jFHA8lkJs9bWADcD7wNrgZettauNMTcYY27wznYesMoYswzPNykvtPqAW0JYbZ2l7IM/YIyl2zm/cR1HRPxYpXcYo6WL5rqOIi0U4cs7t9bOBmY3uO6Rej/fDdztywwigeTDz+dxWtWH5Pa+lKwOma7jiIgf6zP5ciqX/I7Shc/C+Cmu40gL6PTbIn6iqqaO8Ll/pNpE0WP6L13HERE/F5WYwsb2kxi070NKNIxRQFIJE/ET//34A06u/ZzdA68mrF1n13FEJADEjPIMY7T0k1ddR5EWUAkT8QPlVbUkzb+bEpNAjzN/4jqOiASIXuOnc4BEIla97DqKtIBKmIgfeP/dN5hkl7B/+E2Y2GTXcUQkQJiIaLZ2mcrwsnns2rPbdRxpJpUwEceKy6votvQvFIWl0G3qba7jiEiA6TzpSmJMNWs/etZ1FGkmlTARxz58+3lGspbScT+EqDjXcUQkwHTJnkR+eDrJmzSMUaBRCRNxqPBgOf1X38/eiC6kT7nedRwRCUTGsKfnOQyrXcmGDRrGKJCohIk4NOf1x8k226g94ScQEeU6jogEqJ5TrgY0jFGgUQkTcWTn/oMM3/wwu6Kz6DzxCtdxRCSAtevSm40xg8ncoWGMAolKmIgjX7z6ID1NPuEn/wrCwl3HEZEAV5l9AT3JY/nCOa6jSBOphIk4sH1XIRNyHyM3LpvUUee4jiMiQaDPlMuoJJKSRc+5jiJNpBIm4sCS1+6jqykk4YzfgjGu44hIEIhOSGFD+4kM2veBhjEKECphIm1s/fZ8jt/9NNvajSZ54Cmu44hIEIkddQkdTDHL5rzmOoo0gUqYSBtb+/rddDTFdJj+e9dRRCTI9Bo/gyISCV/5kuso0gQqYSJtaPmGLUzZ/xKbO0wmsdc413FEJMh4hjGapmGMAoRKmEgbynnrjySYCrqc8wfXUUQkSH09jNHHGsbI36mEibSRRStWc8rBN9jc5QziMga5jiMiQapr9kR2hKfTYaOOC/N3KmEibcBay97ZvyfcWLqf+zvXcUQkmBlDQc9zGFK7io3r17hOI0egEibSBj5fuIiTy99nW4/vEJ3a03UcEQlyWSd6hjHK+/Qpt0HkiFTCRHysts5S9eEfqDXhZJ3za9dxRCQEtO/amw3eYYxqa+tcx5HDUAkT8bG5n83hxKq55Pa9koikrq7jiEiIqMw+nyx2aBgjP6YSJuJD1bV1RH36R8rC4ug1/f9cxxGRENJ3yuWeYYwW6luS/kolTMSHPv7gHSbVLmT3oOsIi092HUdEQkh0Qgrr209i4L4PKNUwRn5JJUzERyqqauj45Z8pCkui55l3uI4jIiEoxjuM0fK5Ol2FP1IJE/GRj2a/xEi7mqKRP8BEJ7iOIyIhqM/4GRTRjrAVGsbIH6mEifjAwfIqMpfdS0F4ZzJPu8l1HBEJUSYiiq1ppzG8bB57Cva4jiMNqISJ+MCcN59gIJupmHgnRES7jiMiIazTcVcRbapZ+9EzrqNIAyphIq1s38Fystf+nZ2RPeg2+WrXcUQkxKV7hzFK1jBGfkclTKSVff7ag/QyO7BT/g/Cwl3HEZFQZwx7smZ4hjHaoGGM/IlKmEgr2llYxIgtj5AT05+u4y5wHUdEBICsKd8FYMen/3acROpTCRNpRYtfvZ8Ms5eYqXeBMa7jiIgAkNS1N+tjhtAj720NY+RHVMJEWsn2nQWM3/EEWxOG02noVNdxRES+4dAwRisWzXEdRbxUwkRaycpX/0xHU0z7M3+vvWAi4nf6nniZZxijL/UtSX+hEibSCjZsz+H4gufZmHQcKf0nuY4jIvItMYmeYYyy931IWbmGMfIHKmEirWDT638kgXI6n/N711FERA4rZqR3GKM5Ol2FP1AJEzlGK9et58T9r7Kh01Ta9RjmOo6IyGH1Hj+D/bQjbKWGMfIHKmEix2jn278j0tTS7bzfuY4iInJEYZFRbEs7jWGl8yjQMEbOqYSJHIPFy5YyuWQ2G9PPJT6tj+s4IiJHlTrJO4zRxzpA3zWVMJEWstZS/O7vsCacrPPuch1HRKRJMgZOJC88gyQNY+ScSphIC81f8AWTKz5mS89LiUnJcB1HRKRpjGFP5nSG1Kxis4YxckolTKQF6uosdR/9njITS+9zfuE6johIs2RNuRqAvE+fchskxKmEibTAZ3PfY1LNfPIGXENkYkfXcUREmiU5vQ/ro4fQI+8dDWPkkEqYSDNV19YR/9kfKTLt6Xv2na7jiIi0SEX2d8hkB6s0jJEzKmEizfTpe68wqm4Fe4bdTFhsO9dxRERapN+Uy6m0kRQvfNZ1lJClEibSDBVVNaQtvoeCsFT6nH6L6zgiIi329TBGhR9QXl7hOk5IUgkTaYZP3/k3A+1GDoz9ESYy1nUcEZFjEjXKO4zR3FddRwlJKmEiTXSwrIKeK+4nPyKD3id/z3UcEZFj1tc7jJFZoWGMXFAJE2mieW88Qm9yqTr+5xAe4TqOiMgxC4uMYkvaVO8wRrtdxwk5KmEiTbC/uITs9Q+RE9WbzEkXu44jItJqOk28kmhTzbqPNIxRW1MJE2mCha/eTzezh7BTfg1hetmISPDoNsg7jNGm111HCTl6NxE5it2F+xi+7XE2xw0lY9RZruOIiLQuY9idOZ3BNavYslHDGLUllTCRo1j2yt10MkXET/stGOM6johIqzs0jFHu3H87ThJaVMJEjiA3fydj859hXbsJpA2e7DqOiIhPpHiHMcrc8TZ1GsaozaiEiRzBulf/QJIpJfXs37mOIiLiU+XZ59PD7mDV4jmuo4QMlTCRw9i0ZQsT9r7Mmg6n0KH3KNdxRER8qv+Uy6mwkRR/qWGM2opKmMhhbH/jt0RTTfo52gsmIsEvJjHZO4zRfzWMURtRCRNpxOo1q5h04G3WdZlO+4wBruOIiLSJqJGXkGIOskLDGLUJlTCRRux9x/NNyKzzfuM6iohIm+k34dAwRi+6jhISVMJEGli6ZAGTSv/Lhu4XEp/aw3UcEZE2ExYZxea0qQwtnc/egj2u4wQ9lTCReqy1lP/3t1SYaPqc92vXcURE2lyniVd4hjH6WMMY+ZpKmEg9C7/4iAmVX7Cl99XEtO/kOo6ISJvrPmgSuWEZJG18zXWUoKcSJuJVV2eJmPN7DpBI/3N/6jqOiIgbxrA7awaDalaxfct612mCmkqYiNe8j95gZM1ScgfdSGRckus4IiLOdB49A4Cdyz9yGyTIqYSJANU1tSTN/xMFpgPZZ93uOo6IiFPpvYdRZqOp2/GV6yhBTSVMBJj37vMMqltPwYjbCIuOcx1HRMSpsIhIcqJ7k1y0ynWUoKYSJiGvoqqarl/9hfzwrgyYdoPrOCIifqE4ZTCZ1ZuprKp0HSVoqYRJyJv/5kz62O0cHH8nJiLKdRwREb8Q1W0ksaaK7Wv1kaSvqIRJSCspK6f36r+zPbIn/aZc6TqOiIjf6Nx/PACFGxc4ThK8VMIkpC187QG6sYvayb+AML0cREQOScvK5iBxkL/UdZSgpXcdCVn7iw4wcOMjbIoeRM8J57qOIyLiV0xYODkx/ehwYLXrKEFLJUxC1lev/YXOZj9Rp90FxriOIyLid0o7DCazZivlZWWuowQllTAJSXsK9jBy+5OsjR9D9xGnuI4jIuKXYnqMJsrUsnXNl66jBCWVMAlJq1/5I0mmhKQzf+s6ioiI3+qaPQGAoo0qYb6gEiYhJy8vh9G7XmBl+xPpMmC86zgiIn6rY3ov9tGO8F06ON8XVMIk5Gx67bfEUknajN+7jiIi4t+MYUfsAFIPrnWdJCiphElI2bJ5PeML32Bl6pmkZg1yHUdExO+Vpw6hR20OxcVFrqMEHZUwCSn5b9yFwZJ5no4FExFpiris0YQby/ZVOmlra1MJk5CxdtVXjCt+j9Xp3yGpS0/XcUREAkKG9+D84s06OL+1qYRJyDgw6y6qTCR9zvu16ygiIgEjqXM39pgORO1e5jpK0FEJk5CwfNGnjCufy7rMK0jo0NV1HBGRgJIfn03nUh2c39pUwiToWWup/eA3HCCB7PP+z3UcEZGAU9VpKN3tTvbt3eM6SlBRCZOg99VnsxhRtZjN/b5HTGKy6zgiIgEnsedoAHJWf+E4SXBpcgkzxiQbYwYaY3oaY1TeJCDU1dYR/ekf2GuSGXzOHa7jiIgEpG6DJgFQumWx4yTBJeJIE40x7YGbgIuBKKAAiAE6G2MWAA9baz/xeUqRFlr04UuMrVnDsiG/omNMgus4IiIBKSGpIztMGjEFy1xHCSpHLGHAK8DTwHHW2qL6E4wxI4HLjTE9rbX/8lE+kRarqakh5cu7yQ9LY/BZt7iOIyIS0HYnZtO1eAXWWowxruMEhSOWMGvtKUeYtgRY0uqJRFrJl+88wcS6rawY8xe6Rka5jiMiEtBq0obTpfhjdu/KpXOX7q7jBIUmHdtljLmmweVwY4xOtiR+q6Kykm7L72NbeCaDp15z9BuIiMgRJfUeA0De6nmOkwSPph5gf5IxZrYxposxZhCwAEj0YS6RY7LojQfpbndSPulnmLBw13FERAJe94HjqbWGim06OL+1HO2YMACstZcYYy4EVgJlwMXWWn1PVfxSSWkJfdY+xIaoAQyYfKHrOCIiQSEmvj3bwrsRt3eF6yhBo6kfR/YBfgC8CmzDc0B+nA9zibTY0tfuJ41CzEm/Ah08KiLSagrbZ9O9Yh22rs51lKDQ1I8j3wZ+aa29HjgB2Ags8lkqkRaqqCinz+anWBc9mD5jT3cdR0QkqNR1GUEHDrAjZ5PrKEGhqSVsjLX2IwDrcS8ww2epRFpoyazHSGMvduIPXUcREQk6KX3GAbBzjQ7Obw1HLGHGmEkA1trihtOstRuNMe28B+qLOFdTU0P6qkfZGt6T/pNmuI4jIhJ0umePptqGU5WrM1S1hqMdmH+eMeYe4D085wQ7dMb83sCJQA/gRz5NKNJEX33wPGNsHstH3YcJ08haIiKtLTI6js2RWSQWrnQdJSgc7WSttxtjkoHvAOcDaUA5sBZ4RN+QFH9h6+pot/gf5Js0Bp9yhes4IiJBa3/7gfTZ+yG1tXWEh+sP3mNx1GfPWrsfaAesAD4APgf2Av2NMcN8mk6kiZZ/Pov+tRvIH3QdYRGRruOIiASv9BG0N6Xkbl7lOknAa2qFHQncAHQBugLXAZOBx4wxd/ommkgzfHE/hSQx5IwbXScREQlqqf3GA7B73XzHSQJfU0tYB2CEtfYOa+2PgFFAKnA8cJWPsok0ydqlnzGscgmbe11BVIxOXyci4ksZfYdTYSOp1cH5x6ypJaw7UFXvcjXQw1pbDlS2eiqRZij58K8cJJaBZ9/uOoqISNALj4xie1Rv2hfp48hj1aRhi4DngQXGmDe9l88CXjDGxANrfJJMpAm2b1zBiJK5LMm4gjHtU1zHEREJCQeSBzNw95tUV1cTGanjcFuqSXvCrLW/A74HFAEHgBustb+11pZaay/1YT6RI8qffQ81RND77DtcRxERCRkR3UYQbyrZtm6p6ygBral7wrDWLsFzrjARv7BrxzZG7nuX5alnMbpzd9dxRERCRuf+42EJFG5cQJ/BY1zHCVg6wYcErM1v/YVwask4Q1/QFRFpS117DaaEWOyOr1xHCWgqYRKQigoLGLrrVZa3n0KXrGzXcUREQooJCycnui8pRatdRwloKmESkFa/dT8JppyUU3/iOoqISEgq6TCYrJotVFSUu44SsHxawowxU40x640xm4wxP21k+qXGmBXef/OMMUN9mUeCQ1npQfpvf5YVsWPIHDTWdRwRkZAU1W0kUaaGrWsWuY4SsHxWwowx4cBDwDQgG7jYGNPwc6OtwAnW2iHA74CZvsojwWPZ2w/TgQNEnfBD11FEREJW14ETACja+KXjJIHLl3vCxgCbrLVbrLVVwIvA9PozWGvnecemBFgAZPgwjwSB6uoqeqz/F+sjB9B/7FTXcUREQlZqRl+KSMTs1GkqWsqXJSwdyK13Oc973eFcA7zb2ARjzHXGmMXGmMUFBQWtGFECzdJ3nyDd7qZy3K1gjOs4IiIhy4SFkRvTj9RiHZzfUr4sYY29Q9pGZzTmRDwlrNGjrK21M621o6y1o1JTU1sxogSSuto6Oi57mO1h3Rh84oWu44iIhLyy1KH0qM2hpKTYdZSA5MsSlgd0q3c5A8hvOJMxZgjwODDdWlvowzwS4JbP+Q8967ZTMPRGTFi46zgiIiEvrscoIkwd21YtcB0lIPmyhC0C+hhjsowxUcBFwFv1ZzDGdAdeAy631m7wYRYJcNZaohY8wC5SGTrtWtdxREQESB80EYDizQsdJwlMTR62qLmstTXGmJuB94Fw4Alr7WpjzA3e6Y8AvwI6AA8bz/E9NdbaUb7KJIFr7cIPGFi9mi/7/4S0qGjXcUREBEhJ60EBKUTsWuY6SkDyWQkDsNbOBmY3uO6Rej9fC2i3hhxV5Zz72E87hpx1i+soIiJST358fzqXrHEdIyDpjPni97asXsjw8vms73EJsfGJruOIiEg9FanD6GF3ULRfh3U3l0qY+L3C9++hzEaTffaPXEcREZEGEnqOBmD7qi8cJwk8KmHi1/K3rWf4gY9YkXYu7Tp0ch1HREQa6OY9c37pFg1f1FwqYeLXct65mzoMWWf92HUUERFpRLsOaeSbzkTtWe46SsBRCRO/Vbg7j2EFb7Es+TQ6Z/RyHUdERA5jd8IAupaudR0j4KiEid/a8Pa9RFFD52mNDqQgIiJ+orrzMLqyh4LdO1xHCSgqYeKXSor3MzDvJZYlHEePfsNcxxERkSNo12ssAHmr5jlOElhUwsQvrXrrAdpRSvxJd7iOIiIiR9F90DjqrKFsuw7Obw6VMPE7lRVl9Nz0b1ZGDaPfiBNcxxERkaOIS0whLzyd2IIVrqMEFJUw8TsrZj1KJ/ZhJ93uOoqIiDRRQeJAupWvw1rrOkrAUAkTv1JbU0PaqplsDO/N4Elnu44jIiJNVNdlGKnsZ2feFtdRAoZKmPiVFR8+QzebT/GomzFhWj1FRAJFUm/Pwfn5a+Y7ThI49C4nfsPW1ZG4+EFyTFeGnXK56zgiItIM3QeOocaGUbl9sesoAUMlTPzGmi/epnfNJvIHXkd4RITrOCIi0gzRsYnkRPQgoVAH5zeVSpj4Dfv5/ewhhWFnXO86ioiItEBh+0H0qFhPXW2d6ygBQSVM/MLmZZ8yqHIpm3pdSUxsnOs4IiLSAqbrcJJMCXlbNYRRU6iEiV8o/vAeim08g86+1XUUERFpoQ59xwGwa50Ozm8KlTBxLm/jMoYe/JxVGRfQrn2K6zgiItJC3fqPpNJGUpO7xHWUgKASJs7tnH0PlUTS52wNUSQiEsgiomLIiexJ+30rXUcJCCph4tTeHVsYuu89lnU8i9TOGa7jiIjIMdqfPIjMqo3U1NS4juL3VMLEqc1v30MYlowzfuI6ioiItIKIjBHEmwpyNi53HcXvqYSJM8X79zB452ssaXcS3Xr2cx1HRERaQcd+4wEoWLfAcRL/pxImzqx9817iTCUpp97pOoqIiLSSjN5DKbPR1O34ynUUv6cSJk5UlBbTb9tzfBUzjj6Dx7iOIyIirSQsIoLt0X1ILlrlOorfUwkTJ1a+/SBJHCTyhB+6jiIiIq2sOGUwmdWbqayscB3Fr6mESZurqaqk+/p/sTpyIIPGneo6joiItLKo7qOIMdVsX6fzhR2JSpi0uZXv/4vOdi/lY36AMcZ1HBERaWVpAzwH5xdu0MH5R6ISJm3K1tWSsvSfbA7LZMSU813HERERH0jrMYBi4jH5S11H8WsqYdKmVs95mR51OewZ+n3CwrX6iYgEIxMWRk5MPzoWr3Ydxa/pXVDajrVEzf8bO+jMyGlXu04jIiI+VNphCD1qtlNWVuI6it9SCZM2s3Hhe/StXse2ft8lKirKdRwREfGhmB6jiDS1bF/9pesofkslTNpM5Zx7KaQ9Q8+6yXUUERHxsfSBEwDYv2mh4yT+SyVM2kTOmgUMKl/Emu6XkpCQ6DqOiIj4WMeuPSkkiYidOjj/cFTCpE0UvncPJTaWgWff7jqKiIi0BWPIi+tPp5I1rpP4LZUw8bnd29Yy5MDHLOt8LikdO7mOIyIibaQidQjda/M4ULTPdRS/pBImPpc7625qiCDrrB+7jiIiIm0oLnMMYcaSu1onbW2MSpj41IE9uQze8w5LkqeR3i3LdRwREWlD3Qd5Ds4v3qKD8xujEiY+tfGtvxJBDZ2n3eE6ioiItLH2qensMqlE7V7mOopfUgkTnykr3ke/vJdYEn8CvfoNdR1HREQc2Bk/gLTSta5j+CWVMPGZtW/dTyLlxJ+kvWAiIqGqqtNQMuwu9hXsch3F76iEiU9UV5SSuelplkaNZODI41zHERERRxJ7jQEgZ/U8x0n8j0qY+MSq2Y/QgSLqJt7mOoqIiDjUfdBEAMq2LnKcxP+ohEmrq6uppvOqmawN78fwSWe6jiMiIg4ltO9AXlhXYgqWu47id1TCpNWt+ehputbt4sDImwgL1yomIhLqdidkk162Dmut6yh+Re+Q0rqsJX7Rg2w1GYw89VLXaURExA/Upg2jM4Xs2ZnjOopfUQmTVrXhi9fJqtlCXvZ1REZEuI4jIiJ+oH3vsQDsWP2F4yT+RSVMWpX97H520pFRZ17nOoqIiPiJHgPHUWsNFdsXu47iV1TCpNVsX/Yx/SpXsKHnlcTGxrqOIyIifiImvh254d2J37vCdRS/ohImrab4w79SZBMYevYtrqOIiIif2dt+IN0q1mPr6lxH8RsqYdIqdm5cyuCSL1iefhFJScmu44iIiJ+p6zKCFIrZsX2j6yh+QyVMWsXud/9MmY2m39k/dB1FRET8UIe+noPzd63VmfMPUQmTY7Z/xyYGFn7A4g5nk5aW7jqOiIj4oe4DRlNlw6nKWeI6it9QCZNjtvWde7BAtzN/7DqKiIj4qcjoWLZH9iRx30rXUfyGSpgck5L9uxiw83UWtTuFrJ79XMcRERE/VpQ0kMzK9dTW1rqO4hdUwuSYbHjzr0TbapJPucN1FBER8XMmfSSJppzcTatcR/ELKmHSYpWlRfTa9gKLYyeQPWS06zgiIuLnUvt5Ds7fs26+4yT+QSVMWmztOw/SnhIijr/ddRQREQkAGX1HUG6jqM3TwfmgEiYtVFdVQca6f7E8YgjDx5/sOo6IiASA8IhItkf1pv1+fRwJKmHSQqvff5yOdh/lY27FGOM6joiIBIjilEFkVW+iurrKdRTnVMKk2WxtDcnL/smGsJ6MmnKu6zgiIhJAwruNItZUsW3dUtdRnFMJk2ZbP/dFMmrz2D3k+0REhLuOIyIiASSt/3gA9m1Y4DiJeyph0jzWEjX/AXJIY/S0K12nERGRANO150AOEovdoYPzVcKkWbYumk3P6g1s7nsNMdFRruOIiEiAMWHh5ET3I+XAGtdRnFMJk2apnHMve2wyI8/+vusoIiISoEo6DCazZgsV5WWuozilEiZNlr96Hv3LlrCq+6W0S0hwHUdERAJUdI9RRJlatq1d5DqKUyph0mSF/72bYhvH4LNvcx1FREQCWNfsCQAUbfzScRK3VMKkSQq3r2Zg0VyWdD6P1NRU13FERCSApab3Zj/tMDu/ch3FKZUwaZLcd+6migh6nflj11FERCTAmbAwcmP7k1oc2gfnq4TJUR0syGHgnnf4MukMunfv4TqOiIgEgfKOQ+lRm0PJwQOuozijEiZHtfnNezBY0qbe4TqKiIgEidiskYQby/ZV811HcUYlTI6oongvffL+w5fxk+k3YLDrOCIiEiQysicCULwldL8hqRImR7T+7fuJp4L4KdoLJiIirSclrTt76EDErtAdQ1IlTA6rtrKUHhufZlHkaIaOnOA6joiIBJn8+P50LlnrOoYzKmFyWGtnP0wSxdROvA1jjOs4IiISZCo7DaO7zefAvgLXUZxQCZNG2ZoqUlfOZGX4AMYcf4brOCIiEoQSskYDsH3VPMdJ3FAJk0at/+gpOtftoWj4TYSFaS+YiIi0vm6DPAfnl2xd6DiJGyph8m11dcQvepBNpjtjTrvYdRoREQlS7VI6scOkEb1nuesoTqiEybdsmf8a3Wq2kzPgeqIjI1zHERGRILY7YQBdS0Pz4HyVMPkma7Gf3ssOUhlz1rWu04iISJCrThtOF/ZSsCvXdZQ2pxIm35C3/GN6Va5hXdZVJMTGuI4jIiJBrn0vz8H5eatD7+B8lTD5hoMf3kOhbcews29yHUVEREJA94HjqbOG8m2LXUdpcyph8rWCTUsYULKAZV0vokNysus4IiISAuISk8kNzyBu7wrXUdqcSph8bffsP1NiY+h31u2uo4iISAgpSMwmo3wdtq7OdZQ2pRImABTnb2BA4Qcs7DCdjK5dXccREZEQUtd1BB0pYlfeFtdR2pRKmACw7e27qSGc7mdooG4REWlbyX3GApC/JrQOzlcJE8r37aTfzjdZkHgqvXv1dR1HRERCTPfsMVTbcCpzQuvgfJUwYePbfyHS1pB8qvaCiYhI24uOiScnIpOEwpWuo7QplbAQV126n55bX2BBzCSGDBnpOo6IiISofe2z6VGxnrra0Dk4XyUsxK2f9XcSKCP8OH0jUkRE3DHpI2lvSsnbusZ1lDajEhbCbHU5XdY+yZKIYYyeMMV1HBERCWEd+o0DYPfa+Y6TtB2VsBC2/v2ZdLD7KRt9C2FhxnUcEREJYd36jqDCRlKTGzoH56uEharaGpKWPsxq04dxJ53jOo2IiIS4iKhotkf1ot3+Va6jtBmVsBC1+dPnSavdxe4hNxIZEe46joiICEVJg8is2khNdbXrKG1CJSwUWUvk/L+xla6MP/0K12lEREQAiMgYSbypJGfjMtdR2oRKWAjKXfwO3as2s6nPtcRGR7qOIyIiAkCn/uMBKFj/peMkbUMlLARVfvJXdtkURp11vesoIiIiX0vvNZhSG4PNW+I6SptQCQsxBRu+pHfZMlZkXEJyuwTXcURERL4WFhHB9ui+JB8IjYPzVcJCzO737+WgjSX7zFtcRxEREfmW4g6DyazeSmVluesoPqcSFkLKCrbTr/BDFiafQUaXNNdxREREviW620iiTTU5a4P/fGEqYSFk86z7CLN1pJ3yA9dRREREGpWWPQGAwo0LHSfxPZWwEFFbcZDMbS/zZcwkBg4c4jqOiIhIo9K696OIBMyOr1xH8TmVsBCx/t1/kkgZZsJNrqOIiIgclgkLIzemHx2LV7uO4nMqYaGgrpaUVf9iZVh/xhw31XUaERGRIyrtOJQetdspKz3oOopPqYSFgC2fv0xa7S72DrqWcA3ULSIifi62xygiTB3bVwf3SVtVwkKAnfcgeXRizDQNUSQiIv4v3XtwftEmlbAWM8ZMNcasN8ZsMsb8tJHp/Y0x840xlcaYO3yZJVTtXvM5vSpWsbbHZcTHRruOIyIiclQd07MoIJnwXctcR/GpCF/dsTEmHHgIOAXIAxYZY96y1q6pN9s+4FZghq9yhLq9H9xHrI1j8Jnfdx1FRESkyfLj+tP54JqjzxjAfLknbAywyVq7xVpbBbwITK8/g7V2j7V2EVDtwxwhq3jXZvrv+5jFHc4mLTXVdRwREZEmq+g0lG51OzhQtM91FJ/xZQlLB3LrXc7zXtdsxpjrjDGLjTGLCwoKWiVcKNg26z7qCCP9tNtcRxEREWmWuMzRhBlLzqp5rqP4jC9LWGNfw7MtuSNr7Uxr7Shr7ahU7dFpkuqyInrlvsqXccfTr98A13FERESapfugiQCUbAneM+f7soTlAd3qXc4A8n34+6Se9bMfIp5yoibd7DqKiIhIs7Xv2IWdphORe5a7juIzvixhi4A+xpgsY0wUcBHwlg9/n3jZ2mo6rXmS5eEDGTX+JNdxREREWmRn/AC6lK51HcNnfFbCrLU1wM3A+8Ba4GVr7WpjzA3GmBsAjDFpxpg84IfAL4wxecaYdr7KFCo2zX2BTnUF7B96HWE6OauIiASo6s7DSLe72bcnOD9I89kpKgCstbOB2Q2ue6Tez7vwfEwprcVawr98iBzSGDf1UtdpREREWiyx1xjYDLmr55PS6TzXcVqdzpgfZHasnEPPynVszLqCmKhI13FERERarPtAz5nzS7cG58H5Pt0TJm2v6KP7ibfxDDnrRtdRREREjklC+xRywtKJLVjhOopPaE9YEDmQt54BRZ+ypNO5pKakuI4jIiJyzPYkZpNevg5rW3SWK7+mEhZEts2+lxrCyJz2A9dRREREWkVd2nA6sY89+dtcR2l1KmFBovJgIX3y32RhwhR69ezjOo6IiEiraN97LAB5q4PvzPkqYUFi/awHiaOCuBNudR1FRESk1fQYOJYaG0bl9kWuo7Q6lbAgYGuq6LL+3yyNGMrw0ce5jiMiItJqYuISyYnoTkLhStdRWp1KWBBY//HTpNpCSoZfjzE6OauIiASXwnYD6VaxHltX5zpKq1IJC3TWErPoEbaSzphTL3CdRkREpPV1HUEyB8nftt51klalEhbgcpd9QGb1Rrb2uYroSJ2cVUREgk9yH8/B+fnr5jtO0rpUwgLcwY//xj6byPAzrncdRURExCd6ZI+mykZQk7PYdZRWpRIWwPZtX0P2wS9YlvYdkpPau44jIiLiE5FRMWyL7EnivuA6OF8lLIDlvPtXKm0kvU7XyVlFRCS4FSUNIrNyI7W1ta6jtBqVsABVcaCAfrveZmHiyfTokeU6joiIiE+FZYwgwZSTuzF4xpFUCQtQ6995gFiqaDdFe8FERCT4pfYdB8CeIDo4XyUsANVVVZCx6VmWRI5gyPBxruOIiIj4XEbf4ZTZaOp2fOU6SqtRCQtA6z56kg52P5Wjv6+Ts4qISEgIj4hge1Rvkvavch2l1aiEBRprSVjyKJtNd0ZPOdd1GhERkTZTnDKYzOpNVFdXuY7SKlTCAszWRbPoXrOV3H5XExkR7jqOiIhIm4noNooYU832tUtcR2kVKmEBpnzu39lr2zP8jOtcRxEREWlTaQM8x0EXbljgOEnrUAkLIAVblpNd+iUr0i+gfWKC6zgiIiJtqmvWQIqJh/ylrqO0CpWwALLj3XupsJH0O0OnpRARkdBjwsLIie5LyoHgODhfJSxAlO7byYCC2SxqP5X09G6u44iIiDhR0mEwmTXbqCgvdR3lmKmEBYgN7/yNaKpJOeU211FEREScie4xmkhTy7Y1C11HOWYqYQGgtrKMzC0vsChqDAMHj3IdR0RExJmu2eMBKNoY+Afnq4QFgLX/fZxkDlA39vuuo4iIiDjVKb0XhbQnbOcy11GOmUqYv7OW9stmssH0ZNTk6a7TiIiIOGXCwsiL7U/qwTWuoxwzlTA/t2n+G3SrzWXnwGsID9fiEhERKe84hO61uZQcLHId5ZjoXd3PVX/+D3aTzMjTv+s6ioiIiF+IyxpNuLFsXzXfdZRjohLmx3ZtWMyAsiWs6XYxCXFxruOIiIj4hYyBEwAo3hzY35BUCfNju96/jzIbzYAzb3UdRURExG+kdO7GLjoSuXuZ6yjHRCXMTx3cm8vAve+xOPl00jp3cR1HRETEr+yMH0BayVrXMY6JSpif2vT23winjrRTb3cdRURExO9UdhpKht1JUeEe11FaTCXMD1VXlNBz+4ssjh1P3+yhruOIiIj4ncSeYwDIWfWF4yQtpxLmh9a8N5P2lGDG3+w6ioiIiF/KGOQ5OL906yLHSVpOJczP2LpaOqx8nHVhfRg5aZrrOCIiIn6pfXIqeaYL0XuWu47SYiphfmbj56+SUbuDgsHXEqaTs4qIiBzW7sRsupatcx2jxfQu72fq5j3ITjoyatrVrqOIiIj4terOw0hjLwU7c1xHaRGVMD+yY818+lcsZ32PS4mNiXYdR0RExK8l9fYcnL9j9TzHSVpGJcyP7P3gPkpsDAPP0gH5IiIiR9N94DhqraF8W2AenK8S5icO7N5G9r6P+KrDWaR27OQ6joiIiN+LS0giN7wbcYUrXEdpEZUwP7HpnfsIo470qT90HUVERCRgFLQbSLfyddi6OtdRmk0lzA9Ulh2gT+4rLI47jl59s13HERERCRh1XYeTQjE7cze5jtJsKmF+YM3sR2hHKVHH3eI6ioiISEBJ6T0WgF1rA+/gfJUwx2xtDZ3XPMHq8AEMG3+K6zgiIiIBpXv2GKpsOBXbl7iO0mwqYY6tm/sSXet2UTTsOowxruOIiIgElOiYOLZHZtFuX+AdnK8S5lj4lw+RRydGnXaZ6ygiIiIBaX/7gXSv2EBdba3rKM2iEuZQzoq59K1czZaelxMdFeU6joiISEAyXUfQzpSRt2W16yjNohLm0P6P/kaxjWOQTs4qIiLSYh37jQNgz7r5jpM0j0qYI4U7NjGwaA7LOs0gJTnFdRwREZGA1a3fCMptFDW5gXVwvkqYI9tm3YvF0H3aba6jiIiIBLSIyCi2R/Wm/f5VrqM0i0qYAxUl++mX/zpLEk4gs2c/13FEREQC3oHkQfSo2kRNdZXrKE2mEubAmlkPkUA5cSfc6jqKiIhIUAjPGEGcqSRnwzLXUZpMJayN1dVU03Xdv1kZMYjBoye7jiMiIhIUOvcfD8De9QscJ2k6lbA2tubj50izeygdcb1OzioiItJK0nsN5qCNxe4InIPzVcLakrXELHqYXLow8tRLXKcREREJGmHh4eRE9yX5wBrXUZpMJawNbVn6Eb2r17Ot75VERkS4jiMiIhJUDnYYQmb1Fiory11HaRKVsDZU8skDHLDxDDnjRtdRREREgk5U95FEmRq2r17kOkqTqIS1kYLt6xhU/BnL086jffsk13FERESCTpfsCQDs2/Sl4yRNoxLWRra/ey81hNHr9NtdRxEREQlKad36sJ9EwvK/ch2lSVTC2kBp0V6yd73JV+1OIr1HT9dxREREgpIJCyM3pj8diwPj4HyVsDawdtY/iKOSdlNucx1FREQkqJV1HEL32hzKSotdRzkqlTAfq62uovumZ1geOYzs4RNdxxEREQlqMT1GEWHq2L7K/0/aqhLmY6s+eIpOtpCq0fpGpIiIiK9lDPTs8DiweaHjJEenEuZL1pL41aNsM+kMn/Id12lERESCXseuPdhDChG7lrqOclQqYT60cdH79KzZRF7/7xKhk7OKiIi0ify4/nQ66P8H56uE+VD53AfYRyLDzrjedRQREZGQUdFpGN1tPgeKCl1HOSKVMB/ZtWUlg0rms6brBSQkJLqOIyIiEjLiM0cBkLvqC8dJjkwlzEfy3r2PaiLofcZtrqOIiIiElO6DPQfnH9zi38MXqYT5wMH9uxm45x2+SjqFtPTuruOIiIiElPYd0thhOhO1e5nrKEekEuYD6975O7Gmig4na4giERERF3bFD6Br6VrXMY5IJayV1VSWk7X5OZZFj6Tv4DGu44iIiISk6rThdKGAwj07XEc5LJWwVrby/SfpyH5qx97sOoqIiEjISuzp2RGSt2qe4ySHpxLWimxdHUnLZ7IlrAfDT5jhOo6IiEjI6jFoPHXWULbNfw/OVwlrRRsWvENW7VZ2ZV9DWLieWhEREVcS2iWTG55BbMEK11EOS02hFVV99g/20p7hp3/PdRQREZGQtydxABnla7F1da6jNEolrJXs2LCUweULWdftImLj4lzHERERCXl1XYbTkSL27NzmOkqjVMJayc7376PCRtLvzB+4jiIiIiJA+95jAdjhpwfnq4S1ggMFOxi8912WpkwjtXO66zgiIiICZGaPpdqGU5njnwfnq4S1gvXvPEC0qabzaT90HUVERES8YuISyInoQXzhStdRGqUSdoyqKsrovf0FlsaMpWf/4a7jiIiISD2F7QfSvWK9Xx6crxJ2jFbOnkkKxYRN1MlZRURE/E7X4SRRQv42/xvCSCXsGNi6OlJXPc6msCyGTDzTdRwRERFpIKXPOAB2rp3vOMm3qYQdgzWfv073ulz2Dr4OE6anUkRExN/0GDCKShtJTc5i11G+Rc3hGNh5D7KHFIaffrXrKCIiItKIyKhotkX2pN3+Va6jfItKWAvlrPmSQRVfsTHzEqKjY13HERERkcMoShpEj8qN1NbUuI7yDSphLbTng/sps9Fkn6WTs4qIiPizsIwRxJsK8jYudx3lG1TCWqBwVw5D9v2X5R3PJLlDJ9dxRERE5Ag69RsPwO71Cxwn+SaVsBbYNOt+Iqij69TbXUcRERGRo8joM5RSG4PNW+I6yjeohDVTRdlB+uW+zLL4CfToM9h1HBERETmK8IgItkf3IanIvw7OVwlrppWzHiWJEqKOu9V1FBEREWmi4uRBZFZvoaqy0nWUr6mENUNdbS1pa59gfXgfBo491XUcERERaaLI7iOJNtXkrPOf84WphDXDqrkv061uB8XDrtfJWUVERAJI2oAJABRu8J+D89UkmiHiy4fZSUeGTb3SdRQRERFphq6ZAzhAPGbnUtdRvqYS1kRbVnxBduUKtva6nMjIKNdxREREpBlMWBg50f3ocGC16yhfUwlrov0f3U+pjWHgmbe4jiIiIiItUNJxCN1rtlNRVuI6CqAS1iQFO7YwpOhjVnSeTvvkDq7jiIiISAvE9BhNpKll25ovXUcBVMKaZPOs+wmjju7TfuQ6ioiIiLRQ12zPmfOLNqqEBYSykiKy819lWeLxpGf1cx1HREREWqhT1yz2kkT4rmWuowAqYUe16p2HaUcp8SdooG4REZFAZsLC2BHbn04H17iOAqiEHVFtTQ3p659ibeQA+o8+yXUcEREROUZlqUPpVptHSfF+11FUwo5kxccvkG53UzbiBtdRREREpBXEZY0mzFi2r5rvOopK2JHELnqYfNOZoSdf6jqKiIiItIJuAz1nzj+4xf3B+Sphh7FxySf0r15DTp8riIiMdB1HREREWkFKp3R2kkrE7uWuo6iEHU7xnAcotnEMOvMm11FERESkFe1MGECXEvcH5/u0hBljphpj1htjNhljftrIdGOM+bt3+gpjzAhf5mmqXTkbGFo8l9VdziGhXbLrOCIiItKKKjsNI93upmjvLqc5fFbCjDHhwEPANCAbuNgYk91gtmlAH++/64B/+ipPc2ybdR8WQ9YZP3QdRURERFpZYs/RAOSunuc0hy/3hI0BNllrt1hrq4AXgekN5pkOPG09FgBJxpguPsx0VAcP7GPQrjdY3u5E0rr1dhlFREREfKD7IM/B+SVbFzrN4csSlg7k1ruc572uufNgjLnOGLPYGLO4oKCg1YPWt3vbWorDEmk/RSdnFRERCUbtkjqyLjIb6mqd5ojw4X2bRq6zLZgHa+1MYCbAqFGjvjW9NfUeOpHagWsJj/DlUyMiIiIu9f+/4D5PWB7Qrd7lDCC/BfO0ORUwERER8TVflrBFQB9jTJYxJgq4CHirwTxvAVd4vyU5Djhgrd3pw0wiIiIifsFnu3ystTXGmJuB94Fw4Alr7WpjzA3e6Y8As4HTgU1AGXC1r/KIiIiI+BOffu5mrZ2Np2jVv+6Rej9bQGdDFRERkZCjM+aLiIiIOKASJiIiIuKASpiIiIiIAyphIiIiIg6ohImIiIg4oBImIiIi4oBKmIiIiIgDKmEiIiIiDqiEiYiIiDigEiYiIiLigEqYiIiIiAMqYSIiIiIOqISJiIiIOKASJiIiIuKASpiIiIiIAyphIiIiIg6ohImIiIg4oBImIiIi4oBKmIiIiIgDxlrrOkOzGGMKgO1t8Ks6Anvb4PdI02mZ+B8tE/+k5eJ/tEz8U1sslx7W2tTGJgRcCWsrxpjF1tpRrnPI/2iZ+B8tE/+k5eJ/tEz8k+vloo8jRURERBxQCRMRERFxQCXs8Ga6DiDfomXif7RM/JOWi//RMvFPTpeLjgkTERERcUB7wkREREQcUAkTERERcSCkS5gxZqoxZr0xZpMx5qeNTDfGmL97p68wxoxwkTPUNGG5XOpdHiuMMfOMMUNd5AwlR1sm9eYbbYypNcZ8py3zhaqmLBdjzGRjzDJjzGpjzNy2zhhqmrD9am+MedsYs9y7TK52kTOUGGOeMMbsMcasOsx0Z+/1IVvCjDHhwEPANCAbuNgYk91gtmlAH++/64B/tmnIENTE5bIVOMFaOwT4HTrg1aeauEwOzXc38H7bJgxNTVkuxpgk4GHgbGvtQOD8ts4ZSpr4WrkJWGOtHQpMBu41xkS1adDQ8xQw9QjTnb3Xh2wJA8YAm6y1W6y1VcCLwPQG80wHnrYeC4AkY0yXtg4aYo66XKy186y1+70XFwAZbZwx1DTltQJwC/AqsKctw4WwpiyXS4DXrLU5ANZaLRvfasoysUCiMcYACcA+oKZtY4YWa+2neJ7nw3H2Xh/KJSwdyK13Oc97XXPnkdbV3Of8GuBdnyaSoy4TY0w6cA7wSBvmCnVNea30BZKNMXOMMUuMMVe0WbrQ1JRl8iAwAMgHVgI/sNbWtU08OQxn7/URbfFL/JRp5LqG5+toyjzSupr8nBtjTsRTwib5NJE0ZZn8DfiJtbbW8we+tIGmLJcIYCRwEhALzDfGLLDWbvB1uBDVlGVyGrAMmAL0Aj4wxnxmrS32cTY5PGfv9aFcwvKAbvUuZ+D5y6S580jratJzbowZAjwOTLPWFrZRtlDVlGUyCnjRW8A6AqcbY2qstW+0ScLQ1NRt2F5rbSlQaoz5FBgKqIT5RlOWydXAn63nJJ2bjDFbgf7AwraJKI1w9l4fyh9HLgL6GGOyvAdFXgS81WCet4ArvN+cGAccsNbubOugIeaoy8UY0x14Dbhcf9G3iaMuE2ttlrU201qbCbwCfF8FzOeasg17EzjOGBNhjIkDxgJr2zhnKGnKMsnBs2cSY0xnoB+wpU1TSkPO3utDdk+YtbbGGHMznm9yhQNPWGtXG2Nu8E5/BJgNnA5sAsrw/AUjPtTE5fIroAPwsHfPS421dpSrzMGuictE2lhTlou1dq0x5j1gBVAHPG6tbfRr+nLsmvha+R3wlDFmJZ6PwX5ird3rLHQIMMa8gOebqB2NMXnAr4FIcP9er2GLRERERBwI5Y8jRURERJxRCRMRERFxQCVMRERExAGVMBEREREHVMJEREREHFAJExEREXFAJUxERETEAZUwEQlpxpjRxpgVxpgYY0y8MWa1MWaQ61wiEvx0slYRCXnGmN8DMXgGuc6z1v7JcSQRCQEqYSIS8rzj/C0CKoAJ1tpax5FEJATo40gREUgBEoBEPHvERER8TnvCRCTkGWPeAl4EsoAu1tqbHUcSkRAQ4TqAiIhLxpgrgBpr7fPGmHBgnjFmirX2Y9fZRCS4aU+YiIiIiAM6JkxERETEAZUwEREREQdUwkREREQcUAkTERERcUAlTERERMQBlTARERERB1TCRERERBz4fyU26442cDFiAAAAAElFTkSuQmCC\n",
|
|
"text/plain": [
|
|
"<Figure size 720x720 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"filenames": {
|
|
"image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter11_79_3.png"
|
|
},
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"import autograd.numpy as np\n",
|
|
"from autograd import grad, elementwise_grad\n",
|
|
"import autograd.numpy.random as npr\n",
|
|
"from matplotlib import pyplot as plt\n",
|
|
"\n",
|
|
"def sigmoid(z):\n",
|
|
" return 1/(1 + np.exp(-z))\n",
|
|
"\n",
|
|
"def deep_neural_network(deep_params, x):\n",
|
|
" # N_hidden is the number of hidden layers\n",
|
|
" N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
|
|
"\n",
|
|
" # Assumes input x being an one-dimensional array\n",
|
|
" num_values = np.size(x)\n",
|
|
" x = x.reshape(-1, num_values)\n",
|
|
"\n",
|
|
" # Assume that the input layer does nothing to the input x\n",
|
|
" x_input = x\n",
|
|
"\n",
|
|
" # Due to multiple hidden layers, define a variable referencing to the\n",
|
|
" # output of the previous layer:\n",
|
|
" x_prev = x_input\n",
|
|
"\n",
|
|
" ## Hidden layers:\n",
|
|
"\n",
|
|
" for l in range(N_hidden):\n",
|
|
" # From the list of parameters P; find the correct weigths and bias for this layer\n",
|
|
" w_hidden = deep_params[l]\n",
|
|
"\n",
|
|
" # Add a row of ones to include bias\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n",
|
|
"\n",
|
|
" z_hidden = np.matmul(w_hidden, x_prev)\n",
|
|
" x_hidden = sigmoid(z_hidden)\n",
|
|
"\n",
|
|
" # Update x_prev such that next layer can use the output from this layer\n",
|
|
" x_prev = x_hidden\n",
|
|
"\n",
|
|
" ## Output layer:\n",
|
|
"\n",
|
|
" # Get the weights and bias for this layer\n",
|
|
" w_output = deep_params[-1]\n",
|
|
"\n",
|
|
" # Include bias:\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n",
|
|
"\n",
|
|
" z_output = np.matmul(w_output, x_prev)\n",
|
|
" x_output = z_output\n",
|
|
"\n",
|
|
" return x_output\n",
|
|
"\n",
|
|
"def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n",
|
|
" # num_hidden_neurons is now a list of number of neurons within each hidden layer\n",
|
|
"\n",
|
|
" # Find the number of hidden layers:\n",
|
|
" N_hidden = np.size(num_neurons)\n",
|
|
"\n",
|
|
" ## Set up initial weigths and biases\n",
|
|
"\n",
|
|
" # Initialize the list of parameters:\n",
|
|
" P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
|
|
"\n",
|
|
" P[0] = npr.randn(num_neurons[0], 2 )\n",
|
|
" for l in range(1,N_hidden):\n",
|
|
" P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
|
|
"\n",
|
|
" # For the output layer\n",
|
|
" P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
|
|
"\n",
|
|
" print('Initial cost: %g'%cost_function_deep(P, x))\n",
|
|
"\n",
|
|
" ## Start finding the optimal weigths using gradient descent\n",
|
|
"\n",
|
|
" # Find the Python function that represents the gradient of the cost function\n",
|
|
" # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
|
|
" cost_function_deep_grad = grad(cost_function_deep,0)\n",
|
|
"\n",
|
|
" # Let the update be done num_iter times\n",
|
|
" for i in range(num_iter):\n",
|
|
" # Evaluate the gradient at the current weights and biases in P.\n",
|
|
" # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n",
|
|
" # in the hidden layers and output layers evaluated at x.\n",
|
|
" cost_deep_grad = cost_function_deep_grad(P, x)\n",
|
|
"\n",
|
|
" for l in range(N_hidden+1):\n",
|
|
" P[l] = P[l] - lmb * cost_deep_grad[l]\n",
|
|
"\n",
|
|
" print('Final cost: %g'%cost_function_deep(P, x))\n",
|
|
"\n",
|
|
" return P\n",
|
|
"\n",
|
|
"## Set up the cost function specified for this Poisson equation:\n",
|
|
"\n",
|
|
"# The right side of the ODE\n",
|
|
"def f(x):\n",
|
|
" return (3*x + x**2)*np.exp(x)\n",
|
|
"\n",
|
|
"def cost_function_deep(P, x):\n",
|
|
"\n",
|
|
" # Evaluate the trial function with the current parameters P\n",
|
|
" g_t = g_trial_deep(x,P)\n",
|
|
"\n",
|
|
" # Find the derivative w.r.t x of the trial function\n",
|
|
" d2_g_t = elementwise_grad(elementwise_grad(g_trial_deep,0))(x,P)\n",
|
|
"\n",
|
|
" right_side = f(x)\n",
|
|
"\n",
|
|
" err_sqr = (-d2_g_t - right_side)**2\n",
|
|
" cost_sum = np.sum(err_sqr)\n",
|
|
"\n",
|
|
" return cost_sum/np.size(err_sqr)\n",
|
|
"\n",
|
|
"# The trial solution:\n",
|
|
"def g_trial_deep(x,P):\n",
|
|
" return x*(1-x)*deep_neural_network(P,x)\n",
|
|
"\n",
|
|
"# The analytic solution;\n",
|
|
"def g_analytic(x):\n",
|
|
" return x*(1-x)*np.exp(x)\n",
|
|
"\n",
|
|
"if __name__ == '__main__':\n",
|
|
" npr.seed(4155)\n",
|
|
"\n",
|
|
" ## Decide the vales of arguments to the function to solve\n",
|
|
" Nx = 10\n",
|
|
" x = np.linspace(0,1, Nx)\n",
|
|
"\n",
|
|
" ## Set up the initial parameters\n",
|
|
" num_hidden_neurons = [200,100]\n",
|
|
" num_iter = 1000\n",
|
|
" lmb = 1e-3\n",
|
|
"\n",
|
|
" P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)\n",
|
|
"\n",
|
|
" g_dnn_ag = g_trial_deep(x,P)\n",
|
|
" g_analytical = g_analytic(x)\n",
|
|
"\n",
|
|
" # Find the maximum absolute difference between the solutons:\n",
|
|
" max_diff = np.max(np.abs(g_dnn_ag - g_analytical))\n",
|
|
" print(\"The max absolute difference between the solutions is: %g\"%max_diff)\n",
|
|
"\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
"\n",
|
|
" plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
|
|
" plt.plot(x, g_analytical)\n",
|
|
" plt.plot(x, g_dnn_ag[0,:])\n",
|
|
" plt.legend(['analytical','nn'])\n",
|
|
" plt.xlabel('x')\n",
|
|
" plt.ylabel('g(x)')\n",
|
|
" plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d425cba5",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"### Comparing with a numerical scheme\n",
|
|
"\n",
|
|
"The Poisson equation is possible to solve using Taylor series to approximate the second derivative.\n",
|
|
"\n",
|
|
"Using Taylor series, the second derivative can be expressed as\n",
|
|
"\n",
|
|
"$$\n",
|
|
"g''(x) = \\frac{g(x + \\Delta x) - 2g(x) + g(x-\\Delta x)}{\\Delta x^2} + E_{\\Delta x}(x)\n",
|
|
"$$\n",
|
|
"\n",
|
|
"where $\\Delta x$ is a small step size and $E_{\\Delta x}(x)$ being the error term.\n",
|
|
"\n",
|
|
"Looking away from the error terms gives an approximation to the second derivative:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a2efa110",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"approx\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation} \\label{approx} \\tag{15}\n",
|
|
"g''(x) \\approx \\frac{g(x + \\Delta x) - 2g(x) + g(x-\\Delta x)}{\\Delta x^2}\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5011ee25",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"If $x_i = i \\Delta x = x_{i-1} + \\Delta x$ and $g_i = g(x_i)$ for $i = 1,\\dots N_x - 2$ with $N_x$ being the number of values for $x$, ([15](#approx)) becomes"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "705ee300",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{aligned}\n",
|
|
"g''(x_i) &\\approx \\frac{g(x_i + \\Delta x) - 2g(x_i) + g(x_i -\\Delta x)}{\\Delta x^2} \\\\\n",
|
|
"&= \\frac{g_{i+1} - 2g_i + g_{i-1}}{\\Delta x^2}\n",
|
|
"\\end{aligned}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b390796d",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"Since we know from our problem that"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "19c9ece4",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{aligned}\n",
|
|
"-g''(x) &= f(x) \\\\\n",
|
|
"&= (3x + x^2)\\exp(x)\n",
|
|
"\\end{aligned}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6ade1a7a",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"along with the conditions $g(0) = g(1) = 0$,\n",
|
|
"the following scheme can be used to find an approximate solution for $g(x)$ numerically:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "78b16d02",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"odesys\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation}\n",
|
|
" \\begin{aligned}\n",
|
|
" -\\Big( \\frac{g_{i+1} - 2g_i + g_{i-1}}{\\Delta x^2} \\Big) &= f(x_i) \\\\\n",
|
|
" -g_{i+1} + 2g_i - g_{i-1} &= \\Delta x^2 f(x_i)\n",
|
|
" \\end{aligned}\n",
|
|
"\\end{equation} \\label{odesys} \\tag{16}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f2bfcca4",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"for $i = 1, \\dots, N_x - 2$ where $g_0 = g_{N_x - 1} = 0$ and $f(x_i) = (3x_i + x_i^2)\\exp(x_i)$, which is given for our specific problem.\n",
|
|
"\n",
|
|
"The equation can be rewritten into a matrix equation:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a6191528",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{aligned}\n",
|
|
"\\begin{pmatrix}\n",
|
|
"2 & -1 & 0 & \\dots & 0 \\\\\n",
|
|
"-1 & 2 & -1 & \\dots & 0 \\\\\n",
|
|
"\\vdots & & \\ddots & & \\vdots \\\\\n",
|
|
"0 & \\dots & -1 & 2 & -1 \\\\\n",
|
|
"0 & \\dots & 0 & -1 & 2\\\\\n",
|
|
"\\end{pmatrix}\n",
|
|
"\\begin{pmatrix}\n",
|
|
"g_1 \\\\\n",
|
|
"g_2 \\\\\n",
|
|
"\\vdots \\\\\n",
|
|
"g_{N_x - 3} \\\\\n",
|
|
"g_{N_x - 2}\n",
|
|
"\\end{pmatrix}\n",
|
|
"&=\n",
|
|
"\\Delta x^2\n",
|
|
"\\begin{pmatrix}\n",
|
|
"f(x_1) \\\\\n",
|
|
"f(x_2) \\\\\n",
|
|
"\\vdots \\\\\n",
|
|
"f(x_{N_x - 3}) \\\\\n",
|
|
"f(x_{N_x - 2})\n",
|
|
"\\end{pmatrix} \\\\\n",
|
|
"\\boldsymbol{A}\\boldsymbol{g} &= \\boldsymbol{f},\n",
|
|
"\\end{aligned}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "aefef707",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"which makes it possible to solve for the vector $\\boldsymbol{g}$.\n",
|
|
"\n",
|
|
"We can then compare the result from this numerical scheme with the output from our network using Autograd:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"id": "471664cd",
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"editable": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Initial cost: 457.256\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/numpy/core/fromnumeric.py:3208: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\n",
|
|
" return asarray(a).size\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Final cost: 0.00310113\n",
|
|
"The max absolute difference between the analytical solution and DNN Autograd: 0.000464088\n",
|
|
"The max absolute difference between the analytical solution and numerical scheme: 0.00266858\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAJcCAYAAACxEXM4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABsJ0lEQVR4nO3dd3hUVeLG8e9Jr5AEAoEESOiE3quK2MAG6tr76lrWsrrruuW3xe2ru+q6q66iq65d166grg0sgBTpvSchlBAIIb2d3x8zuDEGSEImZ8r7eR4eMnPvTN6Ze+fOmzt37jHWWkRERESkbYW5DiAiIiISilTCRERERBxQCRMRERFxQCVMRERExAGVMBEREREHVMJEREREHFAJc8AY09kY86kx5qAx5l7XeVwzxsQaY942xhwwxvzHdZ4jMcZcZYz53HWOY2GMmWyMyfPx75hjjLm2CfMdZ4xZ78ss4t+auq604u/bZow5ua1+X0scy3bGGNPdGFNijAk/xgytvlyO5blvrcflb1TCmsi78pR7V4LdxpgnjTEJLby764C9QDtr7Y9aMWag+g7QGehgrT3fdRgXjDGZxhhrjIlwnaUtWWs/s9b2c52jKYwxZxpjFhpjSo0xhcaY54wxGfWmX2WMqfVuI0qMMVu924m+9eY5tJxLGvy70M2j8m/GmLuMMc8ew+2fMsb8vjUz+ZuGxcZam2OtTbDW1rrMdayC9XE1pBLWPGdZaxOAEcBo4BfNubHxCAN6AGtsC86UG6Rv0j2ADdbaGpchgvS5/YZQeIy+YIz5DvA88ADQERgIVAKfG2OS680637uNaA+cDJQDS4wxgxrcZZL3DeXQv5d8/yh8T+uXSDNZa/WvCf+AbcDJ9S7/BXjH+/M4YB5QBCwHJtebbw7wB+ALPBvkZ4FqoAoowbOhjgb+BuR7//0NiPbefjKQB/wE2AU8A9wF/Md7XweBlUBf4GfAHiAXOLVehquBtd55twDX15t26P5/5L3tTuDqetNjgXuB7cAB4HMg9miPu5Hnb4D3uSgCVgNne6//jfe5qPY+H9c0ctu7gJeBp72PYTUwqt70rsCrQAGwFbi13rSngN83fLwNlutPgBV43lQjgJ8Cm72/aw1wTr35rwI+P8xjzAQscCWQg2dv5//Vmx5W774LvY8pxTstx3vbEu+/8d7nfKR3+mXe6dney9cCb3h/bu760/A5uNX7ODMaeUy9gbneZb8XeKnetAnAIu+0RcCEBuv9td5sRcCgetNS8bwWOh1medzhXR4HgJeAmHrT78SzjuZ7798CvQ+zPFq83je4H+NdFnc2uD4MWAX89kjrBvAO8EqDdSSiidudFOBJ7+Pdf2iZe6d9D9gE7APeArrWm2aB7wMbvY//d0AvYD5QjGfdi2rwXPzcu4y3AZfWu68zgKXe2+UCdzWyzl+DZx3+1Hv9d73P/X7gfaBHvducAqzzLt8H8axf1zby2KfyzW3D8nqv97e8j3sT8L3DPHfX8c1t7dtNXMfOBJbhWW/nAUOOsHwe8D4nxcAS4LhmbLeatJ0BHgLubfB73wZuw/N6rsPzeirB8/o4tEwijrQOAcl41s0C7/XvUG8bgPc1fJjHPQZY7H3cu4H76k072/tYi7z3MaDB6/vko22bm/i4DrseHO2596d/zgMEyr8GK08370L9HZCO5w31dDwb5VO8l1O9887Bs3EaiOcNPrKRle+3wAI8b0qpeF74v6u3YtYAd+N5Q4v1rmAVwGne+3waT/n4P+/9fw/YWu/+z8CzATbACUAZMKLB/f/We9vTvdOTvdMf8j6GdCAczxtv9NEed4PnLtL7Ivk5EAVM8b4w+tV7wTx7hOf+0OM93ZvhT8AC77QwPBu/X3nvuyeeN9zTvNMbPteT+fab/jLvMj1ULs/H8wIPAy4ESoEu3mlXcfQS9ph3OQ3FU+wGeKff5l3OGd7n8FHghQa3jah3f08DP/L+PBPPBvvGetNub+H68/VzAPwS+Kqx5ead/gKe9SoMiAEmea9PwbPhvhzPOnix93KHeuv9td6fnwD+UO8+bwLeO8LyWOh9/lPwvJHf4J02FU+RHAjE4dlQH6mEtXi9b3A//b2/J6uRab/Bs/frsOsGnkKy+3DL+SjbnVl4SkKyN+cJ3uun4ClMI7zL9R94C5B3usXzBtWO/+21+wjP66M9njf9Kxs8F/d57+sEPOt8v3rTB3vXgSF43nRnNHg8TwPxeNavGXhe7wO868YvgHne+TvieeP+jvfx3O793Yd7s7+LBtsGPKXtYTzr4zA8JeKkw9z+Keq9/puwjo3AU8rH4tnWXOmdP/ow938Z0MH7OH+EZ/2MOdp2qznbGTyFJx8Iq/cclgGdG743NbaOcfh1qANwHp7XUiKeP+zfqHc/c46wXOYDl3t/TgDGeX/u630cp3h/153edSGqYdaGy4bGtwVHelyHXQ+O9tz70z/nAQLln3eFKMHT7rd7F34snj0MzzSY933+t4Gbg/cv5XrTG658m4HT610+Ddjm/Xkynr/k6v+ldhfwQb3LZ3mzhXsvJ3pX1qTDPJY3gB/Uu/9yvvnmvwfPXq4w77ShjdzHER93g+uPw7NxCqt33Qt4/6KmaSXsw3qXs4Fy789jgZwG8/8MePIwz3VjL/TvHmXZLwOme3++iqOXsPp/TS4ELvL+vJZ6bxZAFzx/qUfQeAm7Bnir3m2vBV70Xt7O/wpFc9efycAOPG+6nwPtj/DYn8ZTADMaXH85sLDBdfOBq+qt94dK2MnAlnrzfQFccYTlcVm9y/cAj3h/fgL4U71pvTlCCWvpet/I7SZ5f09MI9NuADYead3AUx6rG6wjRQ3+DWjkdl3w7A1orBj+C7in3uUE77qU6b1sgYn1pi8BflLv8r3A3+o9FzVAfL3pLwO/PMzz+Dfg/gaPp2e96e9Sb482nu1IGZ7DDq7gm0XE4NkL16QShuePpVogsd51fwKeOsztn6LxEna4deyfeP+AqTd9Pd7i0oR1bD/e7SVH2G4d5rbLOMx2Bs/r/xTvzzcDsxs8nkbLypHWoUZ+/zBgf73Lc46wXD7F8wdIxwbX/xJ4ucGy34H3UxJaqYQdbT1o7nPv8p+OCWueGdbaJGttD2vt96215Xg2LOcbY4oO/cOz0e5S73a5R7nfrnjeVA/Z7r3ukAJrbUWD2+yu93M5sNf+74DFcu//CQDGmGnGmAXGmH3efKfj+WvqkEL7zeOxyry37Yjnr4zNjWRuyuOu//hyrbV1DR5jeiPzHs6uBvlivMef9AC6NsjxczwH+jfVN5aPMeYKY8yyevc3iG8+X83NeugLHD2A1+vd71o8G5LDZZ0LHGeMScPz19xLwERjTCaevRnLvPO1ZP1JwvNxzZ+stQeO8FjuxPNGudAYs9oY893D/M5Dv7exZfoxEGuMGWuM6YFnY//6EX7n4Z6/rnxzWR3xdXUM631De73/N7Zud6k3/XDS8XxkUl9H77bk0L+1jdyuG7DPWru/kWnfeP6ttSV49kTXf/4bbiMaXq7/WPdba0vrXf56HfIut0+MMQXGmAN4imfD10P9ZdEDeKDeer4PzzqUToNlaD3vkEfbPtbXFc9zcrBB1uZsS+DIr9EfNdiedOObr6evGWN+ZIxZ6/1mdxGe12X95+Zw263mbmf+jWevG97/n2nawzz8OmSMiTPGPGqM2W6MKcZTrJKa+O3Da/Ds9VpnjFlkjDnTe33D9bIOz/Jt7vI5mqasB4d97v2JStixy8WzR6j+BjXeWvvnevPYo9xHPp4X/yHdvdc19faHZYyJxnO81F/x7L5OAmbj2SgezV48u3R7NTKtKY/7kHygm/dLCYd0x/MX0rHKxfPRa/0cidba073TS/Hsbj8krZH7+Pr59ZaEx/D8tdnB+3ytomnPV1OyTmuQNcZau4NGlrG1dhOejceteD5qOohnw3Idnr+SD5Xalqw/+/Ec+/KkMWbi4QJba3dZa79nre0KXA88bIzp3cjvPPR7v7VMvTlfxvOR5SV4jqU82HC+JtiJ56PcQ7odbsZjXO8bWo9nb803vrnrXZ/Pw/Mx35GcA3zWgt+bC6QYY5IamfaN598YE4/n46WWvqaSvfdxSP116Hk8H212s9a2Bx7h289j/XUsF8/xd/XX81hr7Tw8y/Dr5WaMMRxhOfLtdTcfz3OS2CDr4R53c7eduXg+Oq+fPc5a+0LDGY0xx+H5ROACPHuakvAcY3bUdawF25lngenGmKF4PuZ9o960Iz3GI61DPwL6AWOtte2A4w/FO1p+a+1Ga+3FeA6BuBt4xbv+NFwvDy3fxpbP0bbNR3pczV0P/JZK2LF7FjjLGHOaMSbcGBNjPOdhyjjqLf/nBeAXxphUY0xHPMc3tfhr2Q1E4TnOowCoMcZMA05tyg29b55PAPcZY7p6H9947xtccx73l3hecHcaYyKNMZPxfIT64jE/Os/HfcXGmJ8Yz/nGwo0xg4wxo73TlwGnG2NSvHuUbjvK/cXjefEXABhjrsbzF2preAT4g3cDjHd5T/dOK8DzsUHPBreZi2dDPdd7eU6Dy9DC9cdaOwe4FM/eubGNzWOMOb/eMt2P57mpxVNo+hpjLjHGRBjPKRay8Rzc25jn8Rz3cqn355Z4GbjaGDPAGBOH53EeTovX+4a8e2vuwPMcX+Jdz9KAx/Ecc3V/w9t418MsY8w/8HzM8psW/N6deD7ae9gYk+x97Rx6o3wez3MxzPt6/CPwpbV2Wwse4iG/McZEecvFmXiOEQLP4Q37rLUVxpgxeIr0kTwC/MwYMxDAGNPeGHOowM4CBhpjzvXulbiVxv8wOmQ3kHnoDzhrbS6eYx7/5N3mDMGzV+a5I9y+4WvqSB4DbvDu/TPGmHhjzBkN3uwPScTzMW4BEGGM+RWe9aEpmrWdsdbm4fnyyzPAq95PYQ457GM8yjqUiGePaJExJgX4dROzY4y5zBiT6n2PKPJeXYvnNXqGMeYkY0wknqJXiWeZNbSMI2+bj/S4mrse+C2VsGPkXRmm4/kIrADPXx4/pnnP7e/xfNNkBZ5vOn7lva418h3Es6F7Gc+b6CV4/qptqju8mRbh+VjhbjzHdjX5cVtrq/B8Y2Yanr1rD+M5Jmhdyx7VN+67Fk+hG4bnywl78bw5tvfO8gyeb25uA/6L5yO9I93fGjzHy8zHsxEYjOcYptbwAJ7n/r/GmIN4DqYf6/29ZXi/Rev9eGKc9zZz8WwsPz3MZTiG9cda+wGebxG+ZYwZ2cgso4EvjTEl3uw/sNZutdYW4nmj/hGej8HuBM601jb60Zy19lAR74rnTaHZrLXvAn8HPsFzsO9876TKRuY91vW+4f29hOc4uNvxrGNr8BwTOtH7XBwy3vtcFeMpzO2A0dbalQ3ussh88zxhPzzMr74cz7Fe6/Acs3abN89HeI6/eRXP3qVewEUtfXx49rDux7OH4Tk8B6ofen1+H/itd539FZ7n9LCsta/j2U68aDwfc63C89rHu36cD/wZz3rThyO/vg4VwUJjzFfeny/Gc3xQPp6PtX/tXY8b8y8g2/uaeuNIub35FuP5YtODeJ6PTXiOz2rM+3jW5Q14PgqroIkfrbZwO/Nv73wNP4r8E54/EIqMMXc0crtG1yE8x/bF4lmfFwDvNSW711RgtXddfwDPca8V1tr1eD4u/Yf3fs/Cc2qnqkbu42jb5qM9ruasB37LeP7IExEJLMaYAXje4KOt43PMBTLvnulnrbXN2Xsvbcy7B+tZPF++qDva/BIYtCdMRAKGMeYc70dmyXj2trytAibBzvvR3g+Ax1XAgotKmIgEkuvxfPy9Gc8xKDe6jSPiW949vkV4von7N6dhpNXp40gRERERB7QnTERERMQBvztx2dF07NjRZmZmuo4hIiIiclRLlizZa61NbWxawJWwzMxMFi9e7DqGiIiIyFEZYxqOLvI1fRwpIiIi4oBKmIiIiIgDKmEiIiIiDgTcMWEiIiLSOqqrq8nLy6OiosJ1lIAXExNDRkYGkZGRTb6NSpiIiEiIysvLIzExkczMTIwxruMELGsthYWF5OXlkZWV1eTb6eNIERGREFVRUUGHDh1UwI6RMYYOHTo0e4+iSpiIiEgIUwFrHS15HlXCRERERBxQCRMREZGA9tRTT3HzzTcfdZ78/PyvL1977bWsWbOm2b9rzpw5nHnmmc2+XWNUwkRERCToNSxhjz/+ONnZ2Q4TqYSJiIiIQzNmzGDkyJEMHDiQmTNnApCQkMD//d//MXToUMaNG8fu3bsBePvttxk7dizDhw/n5JNP/vr6Qw4ePEhWVhbV1dUAFBcXk5mZyX/+8x8WL17MpZdeyrBhwygvL2fy5MlfD4P43nvvMWLECIYOHcpJJ50EwMKFC5kwYQLDhw9nwoQJrF+/vtUfu05RISIiIvzm7dWsyS9u1fvM7tqOX5818IjzPPHEE6SkpFBeXs7o0aM577zzKC0tZdy4cfzhD3/gzjvv5LHHHuMXv/gFkyZNYsGCBRhjePzxx7nnnnu49957v76vxMREJk+ezKxZs5gxYwYvvvgi5513Hueffz4PPfQQf/3rXxk1atQ3fn9BQQHf+973+PTTT8nKymLfvn0A9O/fn08//ZSIiAg+/PBDfv7zn/Pqq6+26vOjEiYiIiLO/P3vf+f1118HIDc3l40bNxIVFfX1cVcjR47kgw8+ADznNbvwwgvZuXMnVVVVjZ6T69prr+Wee+5hxowZPPnkkzz22GNH/P0LFizg+OOP//q+UlJSADhw4ABXXnklGzduxBjz9d611qQSJiIiIkfdY+ULc+bM4cMPP2T+/PnExcUxefJkKioqiIyM/PqUD+Hh4dTU1ABwyy238MMf/pCzzz6bOXPmcNddd33rPidOnMi2bduYO3cutbW1DBo06IgZrLWNnl7il7/8JSeeeCKvv/4627ZtY/Lkycf8eBvSMWEiIiLixIEDB0hOTiYuLo5169axYMGCo86fnp4OwL///e/DznfFFVdw8cUXc/XVV399XWJiIgcPHvzWvOPHj2fu3Lls3boV4OuPI+v/rqeeeqpZj6upVMJERETEialTp1JTU8OQIUP45S9/ybhx4444/1133cX555/PcccdR8eOHQ8736WXXsr+/fu5+OKLv77uqquu4oYbbvj6wPxDUlNTmTlzJueeey5Dhw7lwgsvBODOO+/kZz/7GRMnTqS2tvYYH2njjLXWJ3fsK6NGjbKHvs0gIiIiLbd27VoGDBjgOkare+WVV3jzzTd55pln2vT3NvZ8GmOWWGtHNTa/jgkTERGRoHHLLbfw7rvvMnv2bNdRjkolTERERILGP/7xD9cRmkzHhImIiIg4oBImIiIi4oBKmIiIiIgDOiZMREQ86uqgphxbVUZVRSnlZSVUlB2koqyEqvISqspLqa4oo7ayhNrKUuqqyqirKofqMkx1OaamnPDacsJrKwirq6Zi1A2Mn3qJ60cl4rdUwkREAkFtDVSXQXW59//6P5dTXVFKVXkJleWl1FSWUF1RSm1lGXXesnRoPlNTTlhNBeG15UTUVhBRV0FUXSVRtoIYqgAwQLT331FjWUO5iaGSaKrCoqk2MVSHx5BQu4/282/lq3YdGTHhVF8+MyIBSyVMRKStHdhB7cLHqSnbT01FGbVVZdiqMmxVqbdYlRNWb69SZF0F4bbmiHcZ6f0XX++6ShtBBVGUE025jaKcGCqIoioshpqwJGrCY6iNjKEuIhbr/Wei4iAqjvDoeMKj4oiIiScqJp7ImHii4xKIjk0gNi6BuPh2xCUkEhUVTYIxJDTIU7JvJ/senEyP969hY8os+vQf0spPokjgUwkTEWlDtmA9pY+fRUxFASXEU040FTaKcm9Z8vzcjnI6Um6jqTDR1IbHUBce6ylLkbGYyDhMVCxhUfGERcURHpNAVEwckbEJRMXGExObQExsPPGxMcRHR5AQHUH76Ai6RocTHRHeJo8zIaULZVe8SsRTpxH14oXsuvEj0jp3bZPfLYFj27ZtTJs2jUmTJjFv3jzS09N58803mTZtGmPHjuWTTz6hqKiIf/3rXxx33HGu47Y6lTARkTZSl7uEin+fQ3k13JfxTzIGjCU+Opz46Iivy1LHKM//h66PjghrdHDhQNApcxDbzn6Krm9exPqZ3yH+9v+SmNBwn5n4jXd/CrtWtu59pg2GaX8+4iwbN27khRde4LHHHuOCCy7g1VdfBaCmpoaFCxcye/ZsfvOb3/Dhhx+2bjY/oBImItIGajZ9Qs3zF1NYk8Drgx/iF+edSlhYYJar5sgccQpr9/6FwfNuZ97DlzH6h68QGaG3HvmfrKwshg0bBsDIkSPZtm0bAOeee+63rgs2eiWIiPhY5Yo3CHvtWrbVpTF//ExumTo+YPdutcSAU7/Lin3bmLDuAT569DamfP8fIfX4A8ZR9lj5SnT0/74CEh4e/vXg2oeuDw8Pp6bmyMdEBiqdJ0xExIfK5j9BxGtXs6Iuk5WnPM/V0yaEZAEZcuFvWNlpOicVPMNHz//VdRwRv6ASJiLiIwc//Atx79/OF3WD2XvOS5x/XAh/Q9AYBl33OOviRzN5wx/5/L2XXCcScc5Ya11naJZRo0bZxYsXu44hInJ41lL01k9JWvoIs+xEUi55nPH99M1AgKrSInbdP5mU6l1sOvNVho2e6DpSSFu7di0DBgxwHSNoNPZ8GmOWWGtHNTa/9oSJiLSm2hr2v3AdSUsf4SUzlW7XPqsCVk9UfBJJ33uTirBY0mZdzuYtG11HEnFGJUxEpLVUV7DvyYtI3vAy/wq/kNE3Ps6QbimuU/mddp17UHvxSyRSSu0z51Owd6/rSCJOqISJiLSGimIKZ55FSt4HPBRzHWfc+nd6dkp0ncpvde47hoKpj9KzbjvbHr2Q0vIK15FCVqAdluSvWvI8qoSJiByrkgL2PXwq7fYs4W/tfsylt/6BtPYxrlP5vcxxM9g0+jeMrl7Mlw9dQ01NretIIScmJobCwkIVsWNkraWwsJCYmOa97nWeMBGRY2D3b6do5lnElu3kH51/ww3X3khclDatTdX/zFtZVbiVKVuf4N1//YKp1/0xJE/h4UpGRgZ5eXkUFBS4jhLwYmJiyMjIaNZttKUQEWmhut1rKXn8LMKqSnm0x73ccuVlRIbrA4bmGnT5vax9MIdpOx/mv6/04NTzb3AdKWRERkaSlZXlOkbI0tZCRKQFanIWUj7zVCqqqnl+wCPcetXlKmAtFRZGvxueZUvMIE5Y9Qs+/3iW60QibUJbDBGRZqpc/yE1T57F3uoY3hvzFDdceHZIjAPpS2FRsXS98XX2RaSSPfd6li//ynUkEZ9TCRMRaYbSr14h7IUL2VrbicUnvcAVZ5yoY5haSUz7TsRd/RrhBpJev4StuTmuI4n4lEqYiEgTFX8+k9i3rmVFXS9yp7/CeSc0ehJsOQbtMwZQft6zpLGX4icvYG/RAdeRRHxGJUxE5GisZd97f6Ldhz/mMzuMqkte5dSR/VynClppgyeTP/l+htatZc0/L6e8stp1JBGfUAkTETmSujr2vnoHKQv+zCxzPMnX/Ifx/bu5ThX0siZfzobBd3B85Vw++ect1NbpPFYSfFTCREQOp7aaPc9eQ8dVj/Ny+BkMuPE5hnRPdZ0qZPQ99xesS/8Opxe9wKyn/uQ6jkirUwkTEWlMdTl7Hj+fTlte46noSzj+ln/Rs1M716lCizH0/+6jbGo3jtO3/4X33nzOdSKRVqUSJiLSUMUB9jx8Oh3z5/BowveZ8YMHSEuKdZ0qNIVHkHXjf9gZncXEr37EF59/4jqRSKtRCRMRqcce3E3Bg6eQvG85j3T8GZff+juS4qJcxwpp4bHt6Hj9m1SGx9Hrg++ycu1a15FEWoVKmIiIV92+bex/cArxB7fxeLc/8r3v/1jjQPqJ2A7diLj8FRJNOVEvXcT2/N2uI4kcM5UwERGgeucqDj58EmEV+3m+39+5/rvXaxgiP5OUNYLisx6nFzns+tdF7DtY5jqSyDHRFkZEQl7llvlUPjaViuoaZo38F9dcfKGGIfJTXUaeSd6E3zO29isWP/xdKqpqXEcSaTGVMBEJaaWr38c+PZ2CmjjmT36eS8+epmGI/FzmqTexqe91nFr+Lu8++jPqdA4xCVAqYSISsooWvkDUfy5hq01j81mvMuPEia4jSRP1vuhuNnU6jXMKZ/Lmcw+6jiPSIiphIhKS9n7yMO1m38gy24cDF77JyaMHu44kzREWRq/v/Ztt8UM4fdNveG/2664TiTSbSpiIhBZr2f32b+k492d8xkhirn6DcdlZrlNJC5jIWDJueJ39kZ0Y8+XNfPHll64jiTSLSpiIhI66Ona+dBudl9zLu2GTybjhFQZnprlOJccgIrEj7a99g/CwMDJmX8HqTVtcRxJpMpUwEQkNtdXkP3UFXdY9xStR0xl+6wv0Skt2nUpaQWxaX2ovep40s4+aZy8id88+15FEmsSnJcwYM9UYs94Ys8kY89MjzDfaGFNrjPmOL/OISIiqKiP/0XPpmvM2z8Zfyck/eIy0pDjXqaQVpfQ7jn2n/oPBbGDzzMs5UFrpOpLIUfmshBljwoGHgGlANnCxMSb7MPPdDbzvqywiErps+X7yH5pG2u7PeDL5B5z7g/tIio92HUt8oMuEi8gd+TMm13zOJw/fTGVNretIIkfkyz1hY4BN1tot1toq4EVgeiPz3QK8CuzxYRYRCUF1xbvY8/eT6Vi0kqfSf82lN92lYYiCXI8z72RL5sXMKH2ZNx/7PdbqHGLiv3xZwtKB3HqX87zXfc0Ykw6cAzxypDsyxlxnjFlsjFlcUFDQ6kFFJPhU793Cvn+cSEJZLi/2uZerrr2NqAgdBhv0jKHn5Q+yLWUS5+76G6+99JTrRCKH5cstUmOnnG74J8nfgJ9Ya4+4z9haO9NaO8paOyo1NbW18olIkKrIW0HpP08ivKqYWcMe5fJLr9IwRKEkPIIe17/I7thenLb2p7z34X9dJxJplC9LWB7Qrd7lDCC/wTyjgBeNMduA7wAPG2Nm+DCTiAS5kg2fU/OvaZTXwOfHPcMF55yjYYhCkIlOpNMNb1IZkciwz65j/rIVriOJfIsvS9gioI8xJssYEwVcBLxVfwZrbZa1NtNamwm8AnzfWvuGDzOJSBDbv+wdIp4/hz11iayd9h/OOnmK60jiUGRSOrFXvUaiqSTl9UtZuy3PdSSRb/BZCbPW1gA34/nW41rgZWvtamPMDcaYG3z1e0UkNO354hkS37iCzTadgu+8wZRxo1xHEj8Q120Ilec+RS+zgwP/voT8wmLXkUS+ZgLtmyOjRo2yixcvdh1DRPxI/vt/o+v8X7OYbGKueJlBPbsd/UYSUvI/mUnXuT/m3ahTmXj7c7SLjXIdSUKEMWaJtbbRvwr1VSERCVzWkvvaL+k6/9fMDRtLyvVvq4BJo7qeeB05g25iWtV/ee+RO6murXMdSUQlTEQCVF0dOc/eRLcVf+e9yJPoe8ur9OzS0XUq8WPdz/sD27uewQUHnuTlJ+/XOcTEOZUwEQk8NVVsf/wSum9+jjfizmPsD16gS3Ki61Ti74yhx3efJK/dcL6T+0deee0/rhNJiFMJE5GAYitL2P7wdHrkv8tLSddy6m0zSU7QMETSRBHRpN/wGkXRXTl5xe28P/cz14kkhKmEiUjAqCvdR97fTyOjcD4vdL6Dc27+i4YhkmYzcSkkf+9NwsLDGfDxd1m4ar3rSBKiVMJEJCBUF+1g99+n0KlkHa/0/D0XXv8LDUMkLRaV2pPwS1+msyki+pVL2bhDwxdL29MWTET8XsWujRQ9OIXEip3MHvx3LrjiJg1DJMcsodc4Ss78J4PZRN6/Lmf3gTLXkSTEqISJiF87uG0p5TNPIby6hM8mPME537lUwxBJq+kw6jvsHvdLTqxbwOf/vInSyhrXkSSEqISJiN/at2YO5qkzqKg1rDj1ZaaddobrSBKEupz2Q/L6XM55Fa/x6qN3UaNziEkbUQkTEb+0a9EbxL98Pntse/LOeZPJEye6jiTByhgyLn6AvE4ncGnhgzz37GM6h5i0CZUwEfE7OZ8+Q8dZV7OJbpRd+g6jhw1xHUmCXVg4Gde+wJ6Efnxnyy955Z1ZrhNJCFAJExG/cnDXJlI//hGrTD9ivzebQX17uY4koSIqns7XvUFlZBLHL76JD+ZrnGLxLZUwEfEf1rLr2eupsWFEXvAEPdPTXCeSEBPWvgsJ17xGYlgV3d+7isXrtrqOJEFMJUxE/MbG9/5Jn5LFfJF1CwMHZLuOIyEqqssg6i54hl4mn+oXL2fTzv2uI0mQUgkTEb9QtjeHLl/+jqXhg5h8yZ2u40iISxhwMgdOvo/xrGTd49dQUFzhOpIEIZUwEXHPWnKevoFwW0PE9AeJiYp0nUiEDpOuYtfw2ziz9iPef+QOyqp0DjFpXSphIuLcpo+fon/xF3yacT2Dhwx3HUfka2ln30V+j+lcVvYMzz32V2rrdOoKaT0qYSLiVPn+3XT87FesDuvLcZf/0nUckW8yhq6XP87O5NFcsecvPP3CczqHmLQalTARcWrLMzcRZ0upOuMB4mKiXccR+baIKLpc9x+KYzM4Z8OdvPL+x64TSZBQCRMRZzZ9+hID933Ap12uZvjICa7jiBxebDIdrnuTsIgoxs6/no8WrXKdSIKASpiIOFFxcB9Jn/yEjSaTcVf83nUckaMKS8kk+or/0NkU0+GdK/lqc77rSBLgVMJExIkNT99KUt0Bik/7Gwlxsa7jiDRJdI/RVM6YyRCzmYLnr6dOB+rLMVAJE5E2t2XB2wwpeJtPUy9m5LgTXccRaZZ2w2awuc81nFLzGUvXrHEdRwKYSpiItKmqsmLi3v8h2+jKyCvvdh1HpEW6nXQDYcay67NnXEeRAKYSJiJtavUzd9CproC9U+6lfWKi6zgiLRKT1odtsQPpu+sdKnQSV2khlTARaTNbv/qQofkv83nKOYw6/nTXcUSOSe3gC+ljclm4YK7rKBKgVMJEpE1UV5QS+c6t7DSpDL7iXtdxRI5Z5vGXUU0E5Yufdx1FApRKmIi0iRXP/YyMuh3kH/dnkpNTXMcROWbhCR3YkjyR4Qc+pLC41HUcCUAqYSLic9tXfcHQnGf4ot3pjD7pPNdxRFpNwpjL6GSKWPLJG66jSABSCRMRn6qpqqDu9ZvYZ5Lof8UDruOItKr00dM5aBKIWvOy6ygSgFTCRMSnlr5wF1m1W9k67nd06NjJdRyR1hURTV7XqYypmM+WHbtcp5EAoxImIj6Tu24JQ7fMZGH8iYw57VLXcUR8ovNxVxFnKln/8XOuo0iAUQkTEZ+oramh4pUbKTFxZF3+IMYY15FEfCKl3yR2R3Sl49Y3NIyRNItKmIj4xJKX/kCfmvVsHPErUtMyXMcR8R1j2N9rBiNrV7J89WrXaSSAqISJSKvbsXkVQzb8g69ixzPmzGtdxxHxuR4nXk2Ysez8QsMYSdOphIlIq6qrreXASzdSRSTpl/0TE6bNjAS/2LS+bI0dSJ9dszSMkTSZto4i0qoWvXof2VUrWDvkJ3ROz3IdR6TN1A66iD7ksmjBp66jSIBQCRORVrMzZyMDV9/LyujhjDnnVtdxRNpU1uTLqCKC8sX6lqQ0jUqYiLQKW1fHnudvJIw6Olz0iD6GlJATHp/C1uSJDNMwRtJE2kqKSKtY+ObDDK1YxKoBt9E1q7/rOCJOxI+5lE6miK/mvOE6igQAlTAROWZ78nPov/yPrIvMZtT5P3EdR8SZjNEzPMMYrdYwRnJ0KmEickysteQ8+31ibBUJFzxCWHi460gi7kREk9t1KmMq5rFVwxjJUaiEicgxWTj7SUaVfcby3jeQ0Weo6zgizqUddxWxpor1n+gAfTkylTARabGCPTvpteguNkf0ZuTFv3YdR8QvfD2M0ZY3sFbDGMnhqYSJSItYa9n09K20tyVEnvsw4RGRriOJ+AdjKOx1DiNqV7J8zRrXacSPqYSJSIss+uAlxpf8l+WZV9M9e6zrOCJ+JfPQMEafaRgjOTyVMBFptn2Fe+k+7+fkhHdn2CW/dx1HxO/EpfVha+wg+ux6R8MYyWGphIlIs61++nZS7T7s2Q8SER3rOo6IX6oZdAG9yWXJl3NdRxE/pRImIs2y8JM3Oe7AWyzPuIQeQ09wHUfEb/WcfDlVRFCmYYzkMFTCRKTJig4U0WXuneSHdWHw5fe4jiPi18LjU9iSNJFhRR+y/2CZ6zjih1TCRKTJlv37Trqxi8ppfyMyJsF1HBG/Fz/6UlLNAb765HXXUcQPqYSJSJMs+eK/HFf4Mss7n0PW6Kmu44gEhG5jZ1BsEonQMEbSCJUwETmq4pISkj/8IfvCOtD/ivtdxxEJHBHR5HbxDGO0LX+36zTiZ1TCROSoFj39c3raXIpP+SvR8cmu44gElLTjriTWVLFOwxhJAyphInJESxd+xvG7n2Vlh2n0mnCO6zgiAadD/0nsiuhKx82vaxgj+QaVMBE5rNLyCmLf/QEHwxLoc8U/XMcRCUzGsK+nZxijFRrGSOpRCRORw/rimbvobzez/4Q/EtM+1XUckYCVOcU7jNHnT7uOIn5EJUxEGrVs6SJO2PE4a5Im02vyZa7jiAS0uLQ+bIkdRO+ds6is1jBG4qESJiLfUl5ZTdjbt1Bposm84mHXcUSCwqFhjBYv+NR1FPETKmEi8i1zn/0jQ+rWsnvCXcSlpLuOIxIUep5wGVVEUL74WddRxE+ohInIN6xctYLjch5iQ+JY+pxyres4IkEjIqEDm5MmMbToQ4pKNIyRqISJSD0VVTVUvX4zxhjSL58JxriOJBJU4rzDGC3RMEaCSpiI1PPxC/cxsnY5O0b/jPhOma7jiASd7mOnU2wSidIwRoJKmIh4rV63nklb7mdL3FD6TLvVdRyRoGQiosnpMpVR5fPI2alhjEKdSpiIUFldQ/ErtxBlaki97DEI06ZBxFe+HsboYx2gH+q0pRURPvzPI4yv+ZLcYbeT2LWf6zgiQa1j/0nsCu9KBw1jFPJUwkRC3LrNWxi//s9sjxlAn7N/4jqOSPAzhsJeMxheu4qVa9e6TiMOqYSJhLDq2jp2vXgbiaac5EtmQli460giISFzyjWeYYw++7frKOKQSphICHv/tSeYXD2Xbdk30q77ENdxREJGfFpvNscOotfOWVRV17qOI46ohImEqE05eYxe9Xt2RPWkz7m/ch1HJOR8PYzRl3NdRxFHVMJEQlBNbR1bn7udjuYA8Rc8AhFRriOJhJxeJ1zuGcZokb4lGapUwkRC0Htvvcgplf9lS5/vktR7rOs4IiEpIiGFzUmTGFL0IQdKyl3HEQdUwkRCzJYduxm67Ffsisyg9/m/cx1HJKQdGsboqzmvuo4iDqiEiYSQ2jrL2md/TLrZS/S5D2Oi4lxHEglph4Yxilj1H9dRxAGVMJEQ8u7s15lW9hZbMi8mecAJruOIhDzPMEanMap8Hrk797iOI21MJUwkRGzfXUj2ov9jX0Qnel18j+s4IuLV+biriTVVrNUwRiFHJUwkBNTVWZY+81N6mnzM9Acw0YmuI4mIV2r/iewMT6fjFg1jFGpUwkRCwLsfvMeZB19hS8Y5dBgyzXUcEanPO4zRsJqVrF67xnUaaUMqYSJBLm/vAXrN+wkHw5PIuvR+13FEpBGZU672DmP0tOso0oZUwkSCmLWWBU//kv5mOzXT7sXEJruOJCKNSEjrw+aYQfTc+Y6GMQohKmEiQez9T+Zw1oHn2JZ2Gqmjz3UdR0SOoHrQBfQij68WahijUKESJhKkdu0vpcunP6YyLI7ulz7kOo6IHEWvyZ5hjEo1jFHIUAkTCULWWuY8/VuGspHKk/9IWGKq60gichSRCSlsSprEkP0fcqBUwxiFApUwkSD0wefzmL7vCbZ3PIHUCZe5jiMiTXRoGKOln7zmOoq0AZUwkSCzp7iM5I9+TF1YBBmX/ROMcR1JRJqox9jpHDCJRKx62XUUaQMqYSJBxFrLf5++m9GspvSE3xCelO46kog0g2cYo6mMKv+CvJ27XccRH1MJEwkiH335FdMLHiUveQydTvie6zgi0gKdJ11JjKlmnYYxCnoqYSJBovBgBTHv/YhIU0fapTP1MaRIgOo0YBL54el00DBGQU8lTCRIzHruASaxlAMTfkZExyzXcUSkpYyhsNd0htasYs26ta7TiA+phIkEgU+WrOKsnX8nv90QOp98q+s4InKMMqd8lzBjyf/0366jiA+phIkEuKKyKmrfuYN4U0nqpY9BWLjrSCJyjBLT+rApdjA9d75DdY2GMQpWKmEiAe715x/hZDuf/aNvJ7Jzf9dxRKSVVA883zOM0ZcaxihYqYSJBLBPl6/nzNy/sju+H52n3uk6joi0ot5fD2P0nOso4iMqYSIBqriimoNv3kmyKSH54pkQHuk6koi0ov8NY/SBhjEKUiphIgHqlRef5Iy6ORQOvZGojGGu44iID8SOupSO5gDL57zqOor4gEqYSACat2YLp239M3tjMul81q9cxxERH8kc5xnGKHylhjEKRiphIgGmvKqWXa/+jC5mH4kXPQoR0a4jiYiPmIhotqdNZWT5PHbs0jBGwUYlTCTAzPrvu5xb+x67B1xFdOY413FExMe+HsboEx2gH2xUwkQCSHFFNWmL/0JJWCJdpv/GdRwRaQOdsz3DGKVsek3DGAUZlTCRAPLu268wiWUcHH0rxLR3HUdE2oIx7O05g6E1q1i3XsMYBROVMJEAsfdgBb1X3U9ReEe6nHyL6zgi0oYODWO0Q8MYBRWVMJEA8cGbzzDSrKdq0h0QGes6joi0oXZderMpZjA98zWMUTBRCRMJAPn7Sxm28R/sjUqn0/HXuo4jIg5UDTyfnuSxbOEc11GklaiEiQSAua8/ygCzHXPiz3VmfJEQdWgYo5KF+pZksFAJE/FzW3cXMX77o+yO6UWHsZe4jiMijkQlprAxaRKD939IcZmGMQoGKmEifm7Ba/8g0+wiZupdEKaXrEgoix15CR3NAZZ98prrKNIKfLpFN8ZMNcasN8ZsMsb8tJHp040xK4wxy4wxi40xk3yZRyTQrM3ZzeRdT7AjYTDth57lOo6IOJY1fgYHSCR81Uuuo0gr8FkJM8aEAw8B04Bs4GJjTHaD2T4ChlprhwHfBR73VR6RQLTi9fvoYvbR/szfgzGu44iIYyYimm1dpjKybB75u/e4jiPHyJd7wsYAm6y1W6y1VcCLwPT6M1hrS+z/Tv8bD+hUwCJeSzdu55R9z5KTPI6E/pNdxxERP5F23FWeYYw+esZ1FDlGvixh6UBuvct53uu+wRhzjjFmHTALz96wbzHGXOf9uHJxQUGBT8KK+BNrLZvevJsUU0LqjD+4jiMifqTzgImeYYw2v65hjAKcL0tYY5+dfGttsda+bq3tD8wAftfYHVlrZ1prR1lrR6WmprZuShE/tGDVBqYdfJVtnU4mtsco13FExJ94hzEaVruS9RrGKKD5soTlAd3qXc4A8g83s7X2U6CXMaajDzOJ+D1rLbtn/YlYU0XXcxv9u0REQlzmiZ4PjjSMUWDzZQlbBPQxxmQZY6KAi4C36s9gjOltjOdoY2PMCCAKKPRhJhG/N2fRcqaVv0NOxtlEpTX8LouICLTr6hnGKCv/HWo0jFHA8lkJs9bWADcD7wNrgZettauNMTcYY27wznYesMoYswzPNykvtPqAW0JYbZ2l7IM/YIyl2zm/cR1HRPxYpXcYo6WL5rqOIi0U4cs7t9bOBmY3uO6Rej/fDdztywwigeTDz+dxWtWH5Pa+lKwOma7jiIgf6zP5ciqX/I7Shc/C+Cmu40gL6PTbIn6iqqaO8Ll/pNpE0WP6L13HERE/F5WYwsb2kxi070NKNIxRQFIJE/ET//34A06u/ZzdA68mrF1n13FEJADEjPIMY7T0k1ddR5EWUAkT8QPlVbUkzb+bEpNAjzN/4jqOiASIXuOnc4BEIla97DqKtIBKmIgfeP/dN5hkl7B/+E2Y2GTXcUQkQJiIaLZ2mcrwsnns2rPbdRxpJpUwEceKy6votvQvFIWl0G3qba7jiEiA6TzpSmJMNWs/etZ1FGkmlTARxz58+3lGspbScT+EqDjXcUQkwHTJnkR+eDrJmzSMUaBRCRNxqPBgOf1X38/eiC6kT7nedRwRCUTGsKfnOQyrXcmGDRrGKJCohIk4NOf1x8k226g94ScQEeU6jogEqJ5TrgY0jFGgUQkTcWTn/oMM3/wwu6Kz6DzxCtdxRCSAtevSm40xg8ncoWGMAolKmIgjX7z6ID1NPuEn/wrCwl3HEZEAV5l9AT3JY/nCOa6jSBOphIk4sH1XIRNyHyM3LpvUUee4jiMiQaDPlMuoJJKSRc+5jiJNpBIm4sCS1+6jqykk4YzfgjGu44hIEIhOSGFD+4kM2veBhjEKECphIm1s/fZ8jt/9NNvajSZ54Cmu44hIEIkddQkdTDHL5rzmOoo0gUqYSBtb+/rddDTFdJj+e9dRRCTI9Bo/gyISCV/5kuso0gQqYSJtaPmGLUzZ/xKbO0wmsdc413FEJMh4hjGapmGMAoRKmEgbynnrjySYCrqc8wfXUUQkSH09jNHHGsbI36mEibSRRStWc8rBN9jc5QziMga5jiMiQapr9kR2hKfTYaOOC/N3KmEibcBay97ZvyfcWLqf+zvXcUQkmBlDQc9zGFK7io3r17hOI0egEibSBj5fuIiTy99nW4/vEJ3a03UcEQlyWSd6hjHK+/Qpt0HkiFTCRHysts5S9eEfqDXhZJ3za9dxRCQEtO/amw3eYYxqa+tcx5HDUAkT8bG5n83hxKq55Pa9koikrq7jiEiIqMw+nyx2aBgjP6YSJuJD1bV1RH36R8rC4ug1/f9cxxGRENJ3yuWeYYwW6luS/kolTMSHPv7gHSbVLmT3oOsIi092HUdEQkh0Qgrr209i4L4PKNUwRn5JJUzERyqqauj45Z8pCkui55l3uI4jIiEoxjuM0fK5Ol2FP1IJE/GRj2a/xEi7mqKRP8BEJ7iOIyIhqM/4GRTRjrAVGsbIH6mEifjAwfIqMpfdS0F4ZzJPu8l1HBEJUSYiiq1ppzG8bB57Cva4jiMNqISJ+MCcN59gIJupmHgnRES7jiMiIazTcVcRbapZ+9EzrqNIAyphIq1s38Fystf+nZ2RPeg2+WrXcUQkxKV7hzFK1jBGfkclTKSVff7ag/QyO7BT/g/Cwl3HEZFQZwx7smZ4hjHaoGGM/IlKmEgr2llYxIgtj5AT05+u4y5wHUdEBICsKd8FYMen/3acROpTCRNpRYtfvZ8Ms5eYqXeBMa7jiIgAkNS1N+tjhtAj720NY+RHVMJEWsn2nQWM3/EEWxOG02noVNdxRES+4dAwRisWzXEdRbxUwkRaycpX/0xHU0z7M3+vvWAi4nf6nniZZxijL/UtSX+hEibSCjZsz+H4gufZmHQcKf0nuY4jIvItMYmeYYyy931IWbmGMfIHKmEirWDT638kgXI6n/N711FERA4rZqR3GKM5Ol2FP1AJEzlGK9et58T9r7Kh01Ta9RjmOo6IyGH1Hj+D/bQjbKWGMfIHKmEix2jn278j0tTS7bzfuY4iInJEYZFRbEs7jWGl8yjQMEbOqYSJHIPFy5YyuWQ2G9PPJT6tj+s4IiJHlTrJO4zRxzpA3zWVMJEWstZS/O7vsCacrPPuch1HRKRJMgZOJC88gyQNY+ScSphIC81f8AWTKz5mS89LiUnJcB1HRKRpjGFP5nSG1Kxis4YxckolTKQF6uosdR/9njITS+9zfuE6johIs2RNuRqAvE+fchskxKmEibTAZ3PfY1LNfPIGXENkYkfXcUREmiU5vQ/ro4fQI+8dDWPkkEqYSDNV19YR/9kfKTLt6Xv2na7jiIi0SEX2d8hkB6s0jJEzKmEizfTpe68wqm4Fe4bdTFhsO9dxRERapN+Uy6m0kRQvfNZ1lJClEibSDBVVNaQtvoeCsFT6nH6L6zgiIi329TBGhR9QXl7hOk5IUgkTaYZP3/k3A+1GDoz9ESYy1nUcEZFjEjXKO4zR3FddRwlJKmEiTXSwrIKeK+4nPyKD3id/z3UcEZFj1tc7jJFZoWGMXFAJE2mieW88Qm9yqTr+5xAe4TqOiMgxC4uMYkvaVO8wRrtdxwk5KmEiTbC/uITs9Q+RE9WbzEkXu44jItJqOk28kmhTzbqPNIxRW1MJE2mCha/eTzezh7BTfg1hetmISPDoNsg7jNGm111HCTl6NxE5it2F+xi+7XE2xw0lY9RZruOIiLQuY9idOZ3BNavYslHDGLUllTCRo1j2yt10MkXET/stGOM6johIqzs0jFHu3H87ThJaVMJEjiA3fydj859hXbsJpA2e7DqOiIhPpHiHMcrc8TZ1GsaozaiEiRzBulf/QJIpJfXs37mOIiLiU+XZ59PD7mDV4jmuo4QMlTCRw9i0ZQsT9r7Mmg6n0KH3KNdxRER8qv+Uy6mwkRR/qWGM2opKmMhhbH/jt0RTTfo52gsmIsEvJjHZO4zRfzWMURtRCRNpxOo1q5h04G3WdZlO+4wBruOIiLSJqJGXkGIOskLDGLUJlTCRRux9x/NNyKzzfuM6iohIm+k34dAwRi+6jhISVMJEGli6ZAGTSv/Lhu4XEp/aw3UcEZE2ExYZxea0qQwtnc/egj2u4wQ9lTCReqy1lP/3t1SYaPqc92vXcURE2lyniVd4hjH6WMMY+ZpKmEg9C7/4iAmVX7Cl99XEtO/kOo6ISJvrPmgSuWEZJG18zXWUoKcSJuJVV2eJmPN7DpBI/3N/6jqOiIgbxrA7awaDalaxfct612mCmkqYiNe8j95gZM1ScgfdSGRckus4IiLOdB49A4Cdyz9yGyTIqYSJANU1tSTN/xMFpgPZZ93uOo6IiFPpvYdRZqOp2/GV6yhBTSVMBJj37vMMqltPwYjbCIuOcx1HRMSpsIhIcqJ7k1y0ynWUoKYSJiGvoqqarl/9hfzwrgyYdoPrOCIifqE4ZTCZ1ZuprKp0HSVoqYRJyJv/5kz62O0cHH8nJiLKdRwREb8Q1W0ksaaK7Wv1kaSvqIRJSCspK6f36r+zPbIn/aZc6TqOiIjf6Nx/PACFGxc4ThK8VMIkpC187QG6sYvayb+AML0cREQOScvK5iBxkL/UdZSgpXcdCVn7iw4wcOMjbIoeRM8J57qOIyLiV0xYODkx/ehwYLXrKEFLJUxC1lev/YXOZj9Rp90FxriOIyLid0o7DCazZivlZWWuowQllTAJSXsK9jBy+5OsjR9D9xGnuI4jIuKXYnqMJsrUsnXNl66jBCWVMAlJq1/5I0mmhKQzf+s6ioiI3+qaPQGAoo0qYb6gEiYhJy8vh9G7XmBl+xPpMmC86zgiIn6rY3ov9tGO8F06ON8XVMIk5Gx67bfEUknajN+7jiIi4t+MYUfsAFIPrnWdJCiphElI2bJ5PeML32Bl6pmkZg1yHUdExO+Vpw6hR20OxcVFrqMEHZUwCSn5b9yFwZJ5no4FExFpiris0YQby/ZVOmlra1MJk5CxdtVXjCt+j9Xp3yGpS0/XcUREAkKG9+D84s06OL+1qYRJyDgw6y6qTCR9zvu16ygiIgEjqXM39pgORO1e5jpK0FEJk5CwfNGnjCufy7rMK0jo0NV1HBGRgJIfn03nUh2c39pUwiToWWup/eA3HCCB7PP+z3UcEZGAU9VpKN3tTvbt3eM6SlBRCZOg99VnsxhRtZjN/b5HTGKy6zgiIgEnsedoAHJWf+E4SXBpcgkzxiQbYwYaY3oaY1TeJCDU1dYR/ekf2GuSGXzOHa7jiIgEpG6DJgFQumWx4yTBJeJIE40x7YGbgIuBKKAAiAE6G2MWAA9baz/xeUqRFlr04UuMrVnDsiG/omNMgus4IiIBKSGpIztMGjEFy1xHCSpHLGHAK8DTwHHW2qL6E4wxI4HLjTE9rbX/8lE+kRarqakh5cu7yQ9LY/BZt7iOIyIS0HYnZtO1eAXWWowxruMEhSOWMGvtKUeYtgRY0uqJRFrJl+88wcS6rawY8xe6Rka5jiMiEtBq0obTpfhjdu/KpXOX7q7jBIUmHdtljLmmweVwY4xOtiR+q6Kykm7L72NbeCaDp15z9BuIiMgRJfUeA0De6nmOkwSPph5gf5IxZrYxposxZhCwAEj0YS6RY7LojQfpbndSPulnmLBw13FERAJe94HjqbWGim06OL+1HO2YMACstZcYYy4EVgJlwMXWWn1PVfxSSWkJfdY+xIaoAQyYfKHrOCIiQSEmvj3bwrsRt3eF6yhBo6kfR/YBfgC8CmzDc0B+nA9zibTY0tfuJ41CzEm/Ah08KiLSagrbZ9O9Yh22rs51lKDQ1I8j3wZ+aa29HjgB2Ags8lkqkRaqqCinz+anWBc9mD5jT3cdR0QkqNR1GUEHDrAjZ5PrKEGhqSVsjLX2IwDrcS8ww2epRFpoyazHSGMvduIPXUcREQk6KX3GAbBzjQ7Obw1HLGHGmEkA1trihtOstRuNMe28B+qLOFdTU0P6qkfZGt6T/pNmuI4jIhJ0umePptqGU5WrM1S1hqMdmH+eMeYe4D085wQ7dMb83sCJQA/gRz5NKNJEX33wPGNsHstH3YcJ08haIiKtLTI6js2RWSQWrnQdJSgc7WSttxtjkoHvAOcDaUA5sBZ4RN+QFH9h6+pot/gf5Js0Bp9yhes4IiJBa3/7gfTZ+yG1tXWEh+sP3mNx1GfPWrsfaAesAD4APgf2Av2NMcN8mk6kiZZ/Pov+tRvIH3QdYRGRruOIiASv9BG0N6Xkbl7lOknAa2qFHQncAHQBugLXAZOBx4wxd/ommkgzfHE/hSQx5IwbXScREQlqqf3GA7B73XzHSQJfU0tYB2CEtfYOa+2PgFFAKnA8cJWPsok0ydqlnzGscgmbe11BVIxOXyci4ksZfYdTYSOp1cH5x6ypJaw7UFXvcjXQw1pbDlS2eiqRZij58K8cJJaBZ9/uOoqISNALj4xie1Rv2hfp48hj1aRhi4DngQXGmDe9l88CXjDGxANrfJJMpAm2b1zBiJK5LMm4gjHtU1zHEREJCQeSBzNw95tUV1cTGanjcFuqSXvCrLW/A74HFAEHgBustb+11pZaay/1YT6RI8qffQ81RND77DtcRxERCRkR3UYQbyrZtm6p6ygBral7wrDWLsFzrjARv7BrxzZG7nuX5alnMbpzd9dxRERCRuf+42EJFG5cQJ/BY1zHCVg6wYcErM1v/YVwask4Q1/QFRFpS117DaaEWOyOr1xHCWgqYRKQigoLGLrrVZa3n0KXrGzXcUREQooJCycnui8pRatdRwloKmESkFa/dT8JppyUU3/iOoqISEgq6TCYrJotVFSUu44SsHxawowxU40x640xm4wxP21k+qXGmBXef/OMMUN9mUeCQ1npQfpvf5YVsWPIHDTWdRwRkZAU1W0kUaaGrWsWuY4SsHxWwowx4cBDwDQgG7jYGNPwc6OtwAnW2iHA74CZvsojwWPZ2w/TgQNEnfBD11FEREJW14ETACja+KXjJIHLl3vCxgCbrLVbrLVVwIvA9PozWGvnecemBFgAZPgwjwSB6uoqeqz/F+sjB9B/7FTXcUREQlZqRl+KSMTs1GkqWsqXJSwdyK13Oc973eFcA7zb2ARjzHXGmMXGmMUFBQWtGFECzdJ3nyDd7qZy3K1gjOs4IiIhy4SFkRvTj9RiHZzfUr4sYY29Q9pGZzTmRDwlrNGjrK21M621o6y1o1JTU1sxogSSuto6Oi57mO1h3Rh84oWu44iIhLyy1KH0qM2hpKTYdZSA5MsSlgd0q3c5A8hvOJMxZgjwODDdWlvowzwS4JbP+Q8967ZTMPRGTFi46zgiIiEvrscoIkwd21YtcB0lIPmyhC0C+hhjsowxUcBFwFv1ZzDGdAdeAy631m7wYRYJcNZaohY8wC5SGTrtWtdxREQESB80EYDizQsdJwlMTR62qLmstTXGmJuB94Fw4Alr7WpjzA3e6Y8AvwI6AA8bz/E9NdbaUb7KJIFr7cIPGFi9mi/7/4S0qGjXcUREBEhJ60EBKUTsWuY6SkDyWQkDsNbOBmY3uO6Rej9fC2i3hhxV5Zz72E87hpx1i+soIiJST358fzqXrHEdIyDpjPni97asXsjw8vms73EJsfGJruOIiEg9FanD6GF3ULRfh3U3l0qY+L3C9++hzEaTffaPXEcREZEGEnqOBmD7qi8cJwk8KmHi1/K3rWf4gY9YkXYu7Tp0ch1HREQa6OY9c37pFg1f1FwqYeLXct65mzoMWWf92HUUERFpRLsOaeSbzkTtWe46SsBRCRO/Vbg7j2EFb7Es+TQ6Z/RyHUdERA5jd8IAupaudR0j4KiEid/a8Pa9RFFD52mNDqQgIiJ+orrzMLqyh4LdO1xHCSgqYeKXSor3MzDvJZYlHEePfsNcxxERkSNo12ssAHmr5jlOElhUwsQvrXrrAdpRSvxJd7iOIiIiR9F90DjqrKFsuw7Obw6VMPE7lRVl9Nz0b1ZGDaPfiBNcxxERkaOIS0whLzyd2IIVrqMEFJUw8TsrZj1KJ/ZhJ93uOoqIiDRRQeJAupWvw1rrOkrAUAkTv1JbU0PaqplsDO/N4Elnu44jIiJNVNdlGKnsZ2feFtdRAoZKmPiVFR8+QzebT/GomzFhWj1FRAJFUm/Pwfn5a+Y7ThI49C4nfsPW1ZG4+EFyTFeGnXK56zgiItIM3QeOocaGUbl9sesoAUMlTPzGmi/epnfNJvIHXkd4RITrOCIi0gzRsYnkRPQgoVAH5zeVSpj4Dfv5/ewhhWFnXO86ioiItEBh+0H0qFhPXW2d6ygBQSVM/MLmZZ8yqHIpm3pdSUxsnOs4IiLSAqbrcJJMCXlbNYRRU6iEiV8o/vAeim08g86+1XUUERFpoQ59xwGwa50Ozm8KlTBxLm/jMoYe/JxVGRfQrn2K6zgiItJC3fqPpNJGUpO7xHWUgKASJs7tnH0PlUTS52wNUSQiEsgiomLIiexJ+30rXUcJCCph4tTeHVsYuu89lnU8i9TOGa7jiIjIMdqfPIjMqo3U1NS4juL3VMLEqc1v30MYlowzfuI6ioiItIKIjBHEmwpyNi53HcXvqYSJM8X79zB452ssaXcS3Xr2cx1HRERaQcd+4wEoWLfAcRL/pxImzqx9817iTCUpp97pOoqIiLSSjN5DKbPR1O34ynUUv6cSJk5UlBbTb9tzfBUzjj6Dx7iOIyIirSQsIoLt0X1ILlrlOorfUwkTJ1a+/SBJHCTyhB+6jiIiIq2sOGUwmdWbqayscB3Fr6mESZurqaqk+/p/sTpyIIPGneo6joiItLKo7qOIMdVsX6fzhR2JSpi0uZXv/4vOdi/lY36AMcZ1HBERaWVpAzwH5xdu0MH5R6ISJm3K1tWSsvSfbA7LZMSU813HERERH0jrMYBi4jH5S11H8WsqYdKmVs95mR51OewZ+n3CwrX6iYgEIxMWRk5MPzoWr3Ydxa/pXVDajrVEzf8bO+jMyGlXu04jIiI+VNphCD1qtlNWVuI6it9SCZM2s3Hhe/StXse2ft8lKirKdRwREfGhmB6jiDS1bF/9pesofkslTNpM5Zx7KaQ9Q8+6yXUUERHxsfSBEwDYv2mh4yT+SyVM2kTOmgUMKl/Emu6XkpCQ6DqOiIj4WMeuPSkkiYidOjj/cFTCpE0UvncPJTaWgWff7jqKiIi0BWPIi+tPp5I1rpP4LZUw8bnd29Yy5MDHLOt8LikdO7mOIyIibaQidQjda/M4ULTPdRS/pBImPpc7625qiCDrrB+7jiIiIm0oLnMMYcaSu1onbW2MSpj41IE9uQze8w5LkqeR3i3LdRwREWlD3Qd5Ds4v3qKD8xujEiY+tfGtvxJBDZ2n3eE6ioiItLH2qensMqlE7V7mOopfUgkTnykr3ke/vJdYEn8CvfoNdR1HREQc2Bk/gLTSta5j+CWVMPGZtW/dTyLlxJ+kvWAiIqGqqtNQMuwu9hXsch3F76iEiU9UV5SSuelplkaNZODI41zHERERRxJ7jQEgZ/U8x0n8j0qY+MSq2Y/QgSLqJt7mOoqIiDjUfdBEAMq2LnKcxP+ohEmrq6uppvOqmawN78fwSWe6jiMiIg4ltO9AXlhXYgqWu47id1TCpNWt+ehputbt4sDImwgL1yomIhLqdidkk162Dmut6yh+Re+Q0rqsJX7Rg2w1GYw89VLXaURExA/Upg2jM4Xs2ZnjOopfUQmTVrXhi9fJqtlCXvZ1REZEuI4jIiJ+oH3vsQDsWP2F4yT+RSVMWpX97H520pFRZ17nOoqIiPiJHgPHUWsNFdsXu47iV1TCpNVsX/Yx/SpXsKHnlcTGxrqOIyIifiImvh254d2J37vCdRS/ohImrab4w79SZBMYevYtrqOIiIif2dt+IN0q1mPr6lxH8RsqYdIqdm5cyuCSL1iefhFJScmu44iIiJ+p6zKCFIrZsX2j6yh+QyVMWsXud/9MmY2m39k/dB1FRET8UIe+noPzd63VmfMPUQmTY7Z/xyYGFn7A4g5nk5aW7jqOiIj4oe4DRlNlw6nKWeI6it9QCZNjtvWde7BAtzN/7DqKiIj4qcjoWLZH9iRx30rXUfyGSpgck5L9uxiw83UWtTuFrJ79XMcRERE/VpQ0kMzK9dTW1rqO4hdUwuSYbHjzr0TbapJPucN1FBER8XMmfSSJppzcTatcR/ELKmHSYpWlRfTa9gKLYyeQPWS06zgiIuLnUvt5Ds7fs26+4yT+QSVMWmztOw/SnhIijr/ddRQREQkAGX1HUG6jqM3TwfmgEiYtVFdVQca6f7E8YgjDx5/sOo6IiASA8IhItkf1pv1+fRwJKmHSQqvff5yOdh/lY27FGOM6joiIBIjilEFkVW+iurrKdRTnVMKk2WxtDcnL/smGsJ6MmnKu6zgiIhJAwruNItZUsW3dUtdRnFMJk2ZbP/dFMmrz2D3k+0REhLuOIyIiASSt/3gA9m1Y4DiJeyph0jzWEjX/AXJIY/S0K12nERGRANO150AOEovdoYPzVcKkWbYumk3P6g1s7nsNMdFRruOIiEiAMWHh5ET3I+XAGtdRnFMJk2apnHMve2wyI8/+vusoIiISoEo6DCazZgsV5WWuozilEiZNlr96Hv3LlrCq+6W0S0hwHUdERAJUdI9RRJlatq1d5DqKUyph0mSF/72bYhvH4LNvcx1FREQCWNfsCQAUbfzScRK3VMKkSQq3r2Zg0VyWdD6P1NRU13FERCSApab3Zj/tMDu/ch3FKZUwaZLcd+6migh6nflj11FERCTAmbAwcmP7k1oc2gfnq4TJUR0syGHgnnf4MukMunfv4TqOiIgEgfKOQ+lRm0PJwQOuozijEiZHtfnNezBY0qbe4TqKiIgEidiskYQby/ZV811HcUYlTI6oongvffL+w5fxk+k3YLDrOCIiEiQysicCULwldL8hqRImR7T+7fuJp4L4KdoLJiIirSclrTt76EDErtAdQ1IlTA6rtrKUHhufZlHkaIaOnOA6joiIBJn8+P50LlnrOoYzKmFyWGtnP0wSxdROvA1jjOs4IiISZCo7DaO7zefAvgLXUZxQCZNG2ZoqUlfOZGX4AMYcf4brOCIiEoQSskYDsH3VPMdJ3FAJk0at/+gpOtftoWj4TYSFaS+YiIi0vm6DPAfnl2xd6DiJGyph8m11dcQvepBNpjtjTrvYdRoREQlS7VI6scOkEb1nuesoTqiEybdsmf8a3Wq2kzPgeqIjI1zHERGRILY7YQBdS0Pz4HyVMPkma7Gf3ssOUhlz1rWu04iISJCrThtOF/ZSsCvXdZQ2pxIm35C3/GN6Va5hXdZVJMTGuI4jIiJBrn0vz8H5eatD7+B8lTD5hoMf3kOhbcews29yHUVEREJA94HjqbOG8m2LXUdpcyph8rWCTUsYULKAZV0vokNysus4IiISAuISk8kNzyBu7wrXUdqcSph8bffsP1NiY+h31u2uo4iISAgpSMwmo3wdtq7OdZQ2pRImABTnb2BA4Qcs7DCdjK5dXccREZEQUtd1BB0pYlfeFtdR2pRKmACw7e27qSGc7mdooG4REWlbyX3GApC/JrQOzlcJE8r37aTfzjdZkHgqvXv1dR1HRERCTPfsMVTbcCpzQuvgfJUwYePbfyHS1pB8qvaCiYhI24uOiScnIpOEwpWuo7QplbAQV126n55bX2BBzCSGDBnpOo6IiISofe2z6VGxnrra0Dk4XyUsxK2f9XcSKCP8OH0jUkRE3DHpI2lvSsnbusZ1lDajEhbCbHU5XdY+yZKIYYyeMMV1HBERCWEd+o0DYPfa+Y6TtB2VsBC2/v2ZdLD7KRt9C2FhxnUcEREJYd36jqDCRlKTGzoH56uEharaGpKWPsxq04dxJ53jOo2IiIS4iKhotkf1ot3+Va6jtBmVsBC1+dPnSavdxe4hNxIZEe46joiICEVJg8is2khNdbXrKG1CJSwUWUvk/L+xla6MP/0K12lEREQAiMgYSbypJGfjMtdR2oRKWAjKXfwO3as2s6nPtcRGR7qOIyIiAkCn/uMBKFj/peMkbUMlLARVfvJXdtkURp11vesoIiIiX0vvNZhSG4PNW+I6SptQCQsxBRu+pHfZMlZkXEJyuwTXcURERL4WFhHB9ui+JB8IjYPzVcJCzO737+WgjSX7zFtcRxEREfmW4g6DyazeSmVluesoPqcSFkLKCrbTr/BDFiafQUaXNNdxREREviW620iiTTU5a4P/fGEqYSFk86z7CLN1pJ3yA9dRREREGpWWPQGAwo0LHSfxPZWwEFFbcZDMbS/zZcwkBg4c4jqOiIhIo9K696OIBMyOr1xH8TmVsBCx/t1/kkgZZsJNrqOIiIgclgkLIzemHx2LV7uO4nMqYaGgrpaUVf9iZVh/xhw31XUaERGRIyrtOJQetdspKz3oOopPqYSFgC2fv0xa7S72DrqWcA3ULSIifi62xygiTB3bVwf3SVtVwkKAnfcgeXRizDQNUSQiIv4v3XtwftEmlbAWM8ZMNcasN8ZsMsb8tJHp/Y0x840xlcaYO3yZJVTtXvM5vSpWsbbHZcTHRruOIyIiclQd07MoIJnwXctcR/GpCF/dsTEmHHgIOAXIAxYZY96y1q6pN9s+4FZghq9yhLq9H9xHrI1j8Jnfdx1FRESkyfLj+tP54JqjzxjAfLknbAywyVq7xVpbBbwITK8/g7V2j7V2EVDtwxwhq3jXZvrv+5jFHc4mLTXVdRwREZEmq+g0lG51OzhQtM91FJ/xZQlLB3LrXc7zXtdsxpjrjDGLjTGLCwoKWiVcKNg26z7qCCP9tNtcRxEREWmWuMzRhBlLzqp5rqP4jC9LWGNfw7MtuSNr7Uxr7Shr7ahU7dFpkuqyInrlvsqXccfTr98A13FERESapfugiQCUbAneM+f7soTlAd3qXc4A8n34+6Se9bMfIp5yoibd7DqKiIhIs7Xv2IWdphORe5a7juIzvixhi4A+xpgsY0wUcBHwlg9/n3jZ2mo6rXmS5eEDGTX+JNdxREREWmRn/AC6lK51HcNnfFbCrLU1wM3A+8Ba4GVr7WpjzA3GmBsAjDFpxpg84IfAL4wxecaYdr7KFCo2zX2BTnUF7B96HWE6OauIiASo6s7DSLe72bcnOD9I89kpKgCstbOB2Q2ue6Tez7vwfEwprcVawr98iBzSGDf1UtdpREREWiyx1xjYDLmr55PS6TzXcVqdzpgfZHasnEPPynVszLqCmKhI13FERERarPtAz5nzS7cG58H5Pt0TJm2v6KP7ibfxDDnrRtdRREREjklC+xRywtKJLVjhOopPaE9YEDmQt54BRZ+ypNO5pKakuI4jIiJyzPYkZpNevg5rW3SWK7+mEhZEts2+lxrCyJz2A9dRREREWkVd2nA6sY89+dtcR2l1KmFBovJgIX3y32RhwhR69ezjOo6IiEiraN97LAB5q4PvzPkqYUFi/awHiaOCuBNudR1FRESk1fQYOJYaG0bl9kWuo7Q6lbAgYGuq6LL+3yyNGMrw0ce5jiMiItJqYuISyYnoTkLhStdRWp1KWBBY//HTpNpCSoZfjzE6OauIiASXwnYD6VaxHltX5zpKq1IJC3TWErPoEbaSzphTL3CdRkREpPV1HUEyB8nftt51klalEhbgcpd9QGb1Rrb2uYroSJ2cVUREgk9yH8/B+fnr5jtO0rpUwgLcwY//xj6byPAzrncdRURExCd6ZI+mykZQk7PYdZRWpRIWwPZtX0P2wS9YlvYdkpPau44jIiLiE5FRMWyL7EnivuA6OF8lLIDlvPtXKm0kvU7XyVlFRCS4FSUNIrNyI7W1ta6jtBqVsABVcaCAfrveZmHiyfTokeU6joiIiE+FZYwgwZSTuzF4xpFUCQtQ6995gFiqaDdFe8FERCT4pfYdB8CeIDo4XyUsANVVVZCx6VmWRI5gyPBxruOIiIj4XEbf4ZTZaOp2fOU6SqtRCQtA6z56kg52P5Wjv6+Ts4qISEgIj4hge1Rvkvavch2l1aiEBRprSVjyKJtNd0ZPOdd1GhERkTZTnDKYzOpNVFdXuY7SKlTCAszWRbPoXrOV3H5XExkR7jqOiIhIm4noNooYU832tUtcR2kVKmEBpnzu39lr2zP8jOtcRxEREWlTaQM8x0EXbljgOEnrUAkLIAVblpNd+iUr0i+gfWKC6zgiIiJtqmvWQIqJh/ylrqO0CpWwALLj3XupsJH0O0OnpRARkdBjwsLIie5LyoHgODhfJSxAlO7byYCC2SxqP5X09G6u44iIiDhR0mEwmTXbqCgvdR3lmKmEBYgN7/yNaKpJOeU211FEREScie4xmkhTy7Y1C11HOWYqYQGgtrKMzC0vsChqDAMHj3IdR0RExJmu2eMBKNoY+Afnq4QFgLX/fZxkDlA39vuuo4iIiDjVKb0XhbQnbOcy11GOmUqYv7OW9stmssH0ZNTk6a7TiIiIOGXCwsiL7U/qwTWuoxwzlTA/t2n+G3SrzWXnwGsID9fiEhERKe84hO61uZQcLHId5ZjoXd3PVX/+D3aTzMjTv+s6ioiIiF+IyxpNuLFsXzXfdZRjohLmx3ZtWMyAsiWs6XYxCXFxruOIiIj4hYyBEwAo3hzY35BUCfNju96/jzIbzYAzb3UdRURExG+kdO7GLjoSuXuZ6yjHRCXMTx3cm8vAve+xOPl00jp3cR1HRETEr+yMH0BayVrXMY6JSpif2vT23winjrRTb3cdRURExO9UdhpKht1JUeEe11FaTCXMD1VXlNBz+4ssjh1P3+yhruOIiIj4ncSeYwDIWfWF4yQtpxLmh9a8N5P2lGDG3+w6ioiIiF/KGOQ5OL906yLHSVpOJczP2LpaOqx8nHVhfRg5aZrrOCIiIn6pfXIqeaYL0XuWu47SYiphfmbj56+SUbuDgsHXEqaTs4qIiBzW7sRsupatcx2jxfQu72fq5j3ITjoyatrVrqOIiIj4terOw0hjLwU7c1xHaRGVMD+yY818+lcsZ32PS4mNiXYdR0RExK8l9fYcnL9j9TzHSVpGJcyP7P3gPkpsDAPP0gH5IiIiR9N94DhqraF8W2AenK8S5icO7N5G9r6P+KrDWaR27OQ6joiIiN+LS0giN7wbcYUrXEdpEZUwP7HpnfsIo470qT90HUVERCRgFLQbSLfyddi6OtdRmk0lzA9Ulh2gT+4rLI47jl59s13HERERCRh1XYeTQjE7cze5jtJsKmF+YM3sR2hHKVHH3eI6ioiISEBJ6T0WgF1rA+/gfJUwx2xtDZ3XPMHq8AEMG3+K6zgiIiIBpXv2GKpsOBXbl7iO0mwqYY6tm/sSXet2UTTsOowxruOIiIgElOiYOLZHZtFuX+AdnK8S5lj4lw+RRydGnXaZ6ygiIiIBaX/7gXSv2EBdba3rKM2iEuZQzoq59K1czZaelxMdFeU6joiISEAyXUfQzpSRt2W16yjNohLm0P6P/kaxjWOQTs4qIiLSYh37jQNgz7r5jpM0j0qYI4U7NjGwaA7LOs0gJTnFdRwREZGA1a3fCMptFDW5gXVwvkqYI9tm3YvF0H3aba6jiIiIBLSIyCi2R/Wm/f5VrqM0i0qYAxUl++mX/zpLEk4gs2c/13FEREQC3oHkQfSo2kRNdZXrKE2mEubAmlkPkUA5cSfc6jqKiIhIUAjPGEGcqSRnwzLXUZpMJayN1dVU03Xdv1kZMYjBoye7jiMiIhIUOvcfD8De9QscJ2k6lbA2tubj50izeygdcb1OzioiItJK0nsN5qCNxe4InIPzVcLakrXELHqYXLow8tRLXKcREREJGmHh4eRE9yX5wBrXUZpMJawNbVn6Eb2r17Ot75VERkS4jiMiIhJUDnYYQmb1Fiory11HaRKVsDZU8skDHLDxDDnjRtdRREREgk5U95FEmRq2r17kOkqTqIS1kYLt6xhU/BnL086jffsk13FERESCTpfsCQDs2/Sl4yRNoxLWRra/ey81hNHr9NtdRxEREQlKad36sJ9EwvK/ch2lSVTC2kBp0V6yd73JV+1OIr1HT9dxREREgpIJCyM3pj8diwPj4HyVsDawdtY/iKOSdlNucx1FREQkqJV1HEL32hzKSotdRzkqlTAfq62uovumZ1geOYzs4RNdxxEREQlqMT1GEWHq2L7K/0/aqhLmY6s+eIpOtpCq0fpGpIiIiK9lDPTs8DiweaHjJEenEuZL1pL41aNsM+kMn/Id12lERESCXseuPdhDChG7lrqOclQqYT60cdH79KzZRF7/7xKhk7OKiIi0ify4/nQ66P8H56uE+VD53AfYRyLDzrjedRQREZGQUdFpGN1tPgeKCl1HOSKVMB/ZtWUlg0rms6brBSQkJLqOIyIiEjLiM0cBkLvqC8dJjkwlzEfy3r2PaiLofcZtrqOIiIiElO6DPQfnH9zi38MXqYT5wMH9uxm45x2+SjqFtPTuruOIiIiElPYd0thhOhO1e5nrKEekEuYD6975O7Gmig4na4giERERF3bFD6Br6VrXMY5IJayV1VSWk7X5OZZFj6Tv4DGu44iIiISk6rThdKGAwj07XEc5LJWwVrby/SfpyH5qx97sOoqIiEjISuzp2RGSt2qe4ySHpxLWimxdHUnLZ7IlrAfDT5jhOo6IiEjI6jFoPHXWULbNfw/OVwlrRRsWvENW7VZ2ZV9DWLieWhEREVcS2iWTG55BbMEK11EOS02hFVV99g/20p7hp3/PdRQREZGQtydxABnla7F1da6jNEolrJXs2LCUweULWdftImLj4lzHERERCXl1XYbTkSL27NzmOkqjVMJayc7376PCRtLvzB+4jiIiIiJA+95jAdjhpwfnq4S1ggMFOxi8912WpkwjtXO66zgiIiICZGaPpdqGU5njnwfnq4S1gvXvPEC0qabzaT90HUVERES8YuISyInoQXzhStdRGqUSdoyqKsrovf0FlsaMpWf/4a7jiIiISD2F7QfSvWK9Xx6crxJ2jFbOnkkKxYRN1MlZRURE/E7X4SRRQv42/xvCSCXsGNi6OlJXPc6msCyGTDzTdRwRERFpIKXPOAB2rp3vOMm3qYQdgzWfv073ulz2Dr4OE6anUkRExN/0GDCKShtJTc5i11G+Rc3hGNh5D7KHFIaffrXrKCIiItKIyKhotkX2pN3+Va6jfItKWAvlrPmSQRVfsTHzEqKjY13HERERkcMoShpEj8qN1NbUuI7yDSphLbTng/sps9Fkn6WTs4qIiPizsIwRxJsK8jYudx3lG1TCWqBwVw5D9v2X5R3PJLlDJ9dxRERE5Ag69RsPwO71Cxwn+SaVsBbYNOt+Iqij69TbXUcRERGRo8joM5RSG4PNW+I6yjeohDVTRdlB+uW+zLL4CfToM9h1HBERETmK8IgItkf3IanIvw7OVwlrppWzHiWJEqKOu9V1FBEREWmi4uRBZFZvoaqy0nWUr6mENUNdbS1pa59gfXgfBo491XUcERERaaLI7iOJNtXkrPOf84WphDXDqrkv061uB8XDrtfJWUVERAJI2oAJABRu8J+D89UkmiHiy4fZSUeGTb3SdRQRERFphq6ZAzhAPGbnUtdRvqYS1kRbVnxBduUKtva6nMjIKNdxREREpBlMWBg50f3ocGC16yhfUwlrov0f3U+pjWHgmbe4jiIiIiItUNJxCN1rtlNRVuI6CqAS1iQFO7YwpOhjVnSeTvvkDq7jiIiISAvE9BhNpKll25ovXUcBVMKaZPOs+wmjju7TfuQ6ioiIiLRQ12zPmfOLNqqEBYSykiKy819lWeLxpGf1cx1HREREWqhT1yz2kkT4rmWuowAqYUe16p2HaUcp8SdooG4REZFAZsLC2BHbn04H17iOAqiEHVFtTQ3p659ibeQA+o8+yXUcEREROUZlqUPpVptHSfF+11FUwo5kxccvkG53UzbiBtdRREREpBXEZY0mzFi2r5rvOopK2JHELnqYfNOZoSdf6jqKiIiItIJuAz1nzj+4xf3B+Sphh7FxySf0r15DTp8riIiMdB1HREREWkFKp3R2kkrE7uWuo6iEHU7xnAcotnEMOvMm11FERESkFe1MGECXEvcH5/u0hBljphpj1htjNhljftrIdGOM+bt3+gpjzAhf5mmqXTkbGFo8l9VdziGhXbLrOCIiItKKKjsNI93upmjvLqc5fFbCjDHhwEPANCAbuNgYk91gtmlAH++/64B/+ipPc2ybdR8WQ9YZP3QdRURERFpZYs/RAOSunuc0hy/3hI0BNllrt1hrq4AXgekN5pkOPG09FgBJxpguPsx0VAcP7GPQrjdY3u5E0rr1dhlFREREfKD7IM/B+SVbFzrN4csSlg7k1ruc572uufNgjLnOGLPYGLO4oKCg1YPWt3vbWorDEmk/RSdnFRERCUbtkjqyLjIb6mqd5ojw4X2bRq6zLZgHa+1MYCbAqFGjvjW9NfUeOpHagWsJj/DlUyMiIiIu9f+/4D5PWB7Qrd7lDCC/BfO0ORUwERER8TVflrBFQB9jTJYxJgq4CHirwTxvAVd4vyU5Djhgrd3pw0wiIiIifsFnu3ystTXGmJuB94Fw4Alr7WpjzA3e6Y8As4HTgU1AGXC1r/KIiIiI+BOffu5mrZ2Np2jVv+6Rej9bQGdDFRERkZCjM+aLiIiIOKASJiIiIuKASpiIiIiIAyphIiIiIg6ohImIiIg4oBImIiIi4oBKmIiIiIgDKmEiIiIiDqiEiYiIiDigEiYiIiLigEqYiIiIiAMqYSIiIiIOqISJiIiIOKASJiIiIuKASpiIiIiIAyphIiIiIg6ohImIiIg4oBImIiIi4oBKmIiIiIgDxlrrOkOzGGMKgO1t8Ks6Anvb4PdI02mZ+B8tE/+k5eJ/tEz8U1sslx7W2tTGJgRcCWsrxpjF1tpRrnPI/2iZ+B8tE/+k5eJ/tEz8k+vloo8jRURERBxQCRMRERFxQCXs8Ga6DiDfomXif7RM/JOWi//RMvFPTpeLjgkTERERcUB7wkREREQcUAkTERERcSCkS5gxZqoxZr0xZpMx5qeNTDfGmL97p68wxoxwkTPUNGG5XOpdHiuMMfOMMUNd5AwlR1sm9eYbbYypNcZ8py3zhaqmLBdjzGRjzDJjzGpjzNy2zhhqmrD9am+MedsYs9y7TK52kTOUGGOeMMbsMcasOsx0Z+/1IVvCjDHhwEPANCAbuNgYk91gtmlAH++/64B/tmnIENTE5bIVOMFaOwT4HTrg1aeauEwOzXc38H7bJgxNTVkuxpgk4GHgbGvtQOD8ts4ZSpr4WrkJWGOtHQpMBu41xkS1adDQ8xQw9QjTnb3Xh2wJA8YAm6y1W6y1VcCLwPQG80wHnrYeC4AkY0yXtg4aYo66XKy186y1+70XFwAZbZwx1DTltQJwC/AqsKctw4WwpiyXS4DXrLU5ANZaLRvfasoysUCiMcYACcA+oKZtY4YWa+2neJ7nw3H2Xh/KJSwdyK13Oc97XXPnkdbV3Of8GuBdnyaSoy4TY0w6cA7wSBvmCnVNea30BZKNMXOMMUuMMVe0WbrQ1JRl8iAwAMgHVgI/sNbWtU08OQxn7/URbfFL/JRp5LqG5+toyjzSupr8nBtjTsRTwib5NJE0ZZn8DfiJtbbW8we+tIGmLJcIYCRwEhALzDfGLLDWbvB1uBDVlGVyGrAMmAL0Aj4wxnxmrS32cTY5PGfv9aFcwvKAbvUuZ+D5y6S580jratJzbowZAjwOTLPWFrZRtlDVlGUyCnjRW8A6AqcbY2qstW+0ScLQ1NRt2F5rbSlQaoz5FBgKqIT5RlOWydXAn63nJJ2bjDFbgf7AwraJKI1w9l4fyh9HLgL6GGOyvAdFXgS81WCet4ArvN+cGAccsNbubOugIeaoy8UY0x14Dbhcf9G3iaMuE2ttlrU201qbCbwCfF8FzOeasg17EzjOGBNhjIkDxgJr2zhnKGnKMsnBs2cSY0xnoB+wpU1TSkPO3utDdk+YtbbGGHMznm9yhQNPWGtXG2Nu8E5/BJgNnA5sAsrw/AUjPtTE5fIroAPwsHfPS421dpSrzMGuictE2lhTlou1dq0x5j1gBVAHPG6tbfRr+nLsmvha+R3wlDFmJZ6PwX5ird3rLHQIMMa8gOebqB2NMXnAr4FIcP9er2GLRERERBwI5Y8jRURERJxRCRMRERFxQCVMRERExAGVMBEREREHVMJEREREHFAJExEREXFAJUxERETEAZUwEQlpxpjRxpgVxpgYY0y8MWa1MWaQ61wiEvx0slYRCXnGmN8DMXgGuc6z1v7JcSQRCQEqYSIS8rzj/C0CKoAJ1tpax5FEJATo40gREUgBEoBEPHvERER8TnvCRCTkGWPeAl4EsoAu1tqbHUcSkRAQ4TqAiIhLxpgrgBpr7fPGmHBgnjFmirX2Y9fZRCS4aU+YiIiIiAM6JkxERETEAZUwEREREQdUwkREREQcUAkTERERcUAlTERERMQBlTARERERB1TCRERERBz4fyU26442cDFiAAAAAElFTkSuQmCC\n",
|
|
"text/plain": [
|
|
"<Figure size 720x720 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"filenames": {
|
|
"image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter11_91_3.png"
|
|
},
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAI/CAYAAABTd1zJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABq7ElEQVR4nO3dd3jV5eH+8fdzzskCEkbYSSBhzzACCIgIsh2ADLfiHtWqtVatX7W21Q61raNWxb3qnjjrQlBAhih7773JzlnP7w+QHypCIOM5435dF1clOUnecGpy+znJeYy1FhERERE5Nh7XASIiIiLRTGNKREREpAI0pkREREQqQGNKREREpAI0pkREREQqQGNKREREpAJ8rj5w/fr1bXZ2tqsPLyIiIlJuc+bM2WGtbXCo1zkbU9nZ2cyePdvVhxcREREpN2PM2l96nR7mExEREakAjSkRERGRCtCYEhEREakAZ98zJSIiEqkCgQAbNmygtLTUdYpUs+TkZDIzM0lISCj322hMiYiI/MSGDRtITU0lOzsbY4zrHKkm1lp27tzJhg0byMnJKffb6WE+ERGRnygtLSU9PV1DKs4YY0hPTz/qK5IaUyIiIoegIRWfjuV+15gSERGRn3n33Xf529/+dkxvm52dzY4dO475Y0+ePJlTTz31mN++uul7pkRERORHgsEgI0eOZOTIka5TooKuTImIiESYNWvW0L59ey677DI6duzI0KFDKSkpAWDAgAEHThDZsWMHPxzN9swzzzB69GhOO+00cnJy+Pe//80///lPunXrRu/evdm1axcAK1euZPjw4eTl5XHCCSewZMkSAC688EJuuOEGBg4cyM0338wzzzzDNddcA8DWrVs5/fTT6dKlC126dGHatGkAjB49mry8PDp27MjEiRMP+2cKhUJceOGFdOrUic6dO/Ovf/0LgBUrVjB48GC6dOlC9+7dWblyJQCFhYWMGzeOdu3ace6552KtBWDOnDmceOKJ5OXlMWzYMDZv3nzg7+U3v/kN/fv3p3379syaNYsxY8bQunVrbrvttgMdL7zwAr169aJr165cccUVhEKhit1ZsO871138ysvLsyIiIpFo0aJFTj/+6tWrrdfrtXPnzrXWWjt+/Hj7/PPPW2utPfHEE+2sWbOstdZu377dNm/e3Fpr7dNPP21btmxp8/Pz7bZt22xaWpp95JFHrLXWXn/99fZf//qXtdbak046yS5btsxaa+2MGTPswIEDrbXWTpgwwZ5yyik2GAweeH9XX321tdbaM84448DbB4NBu2fPHmuttTt37rTWWltcXGw7duxod+zYYa21tnnz5nb79u0/+jPNnj3bDh48+MDvd+/eba21tlevXvbNN9+01lpbUlJii4qK7BdffGHT0tLs+vXrbSgUsr1797ZTp061fr/f9unTx27bts1aa+3LL79sL7roogN/LzfddJO11tr777/fNmnSxG7atMmWlpbajIwMu2PHDrto0SJ76qmnWr/fb6219qqrrrLPPvvsz/7+D3X/A7PtL2waPcwnIiJyGH+ctJBFm/Ir9X12aJrGH07reNjb5OTk0LVrVwDy8vJYs2bNEd/vwIEDSU1NJTU1ldq1a3PaaacB0LlzZ+bNm0dhYSHTpk1j/PjxB96mrKzswD+PHz8er9f7s/f7+eef89xzzwHg9XqpXbs2AA8++CBvvfUWAOvXr2f58uWkp6cfsq1FixasWrWKX//615xyyikMHTqUgoICNm7cyOmnnw7se46nH/Tq1YvMzEwAunbtypo1a6hTpw4LFixgyJAhwL6rXU2aNDnwNj88LNm5c2c6dux44HUtWrRg/fr1fPXVV8yZM4eePXsCUFJSQsOGDY/493okGlMiIiIRKCkp6cA/e73eAw/z+Xw+wuEwwM9+hP/gt/F4PAd+7/F4CAaDhMNh6tSpw3fffXfIj1mzZs1y902ePJlPP/2U6dOnU6NGDQYMGHDYpxSoW7cu33//PR9//DEPP/wwr776Kvfff/8v3v6nf/5gMIi1lo4dOzJ9+vTDvs3Bf/Yffv/D20+YMIG//vWv5f5zlofGlIiIyGEc6QpSdcvOzmbOnDn06tWL119//ajeNi0tjZycHF577TXGjx+PtZZ58+bRpUuXw77doEGDeOSRR7j++usJhUIUFRWxd+9e6tatS40aNViyZAkzZsw47PvYsWMHiYmJjB07lpYtW3LhhReSlpZGZmYmb7/9NqNHj6asrOyw38PUtm1btm/fzvTp0+nTpw+BQIBly5bRsWP57qNBgwYxatQofvOb39CwYUN27dpFQUEBzZs3L9fb/xJ9A7qIiEgUufHGG3nkkUfo27fvMT39wIsvvsiTTz5Jly5d6NixI++8884R3+aBBx7giy++oHPnzuTl5bFw4UKGDx9OMBgkNzeX22+/nd69ex/2fWzcuJEBAwbQtWtXLrzwwgNXh55//nkefPBBcnNz6du3L1u2bPnF95GYmMjrr7/OzTffTJcuXejateuBb4Yvjw4dOnDXXXcxdOhQcnNzGTJkyIFvYK8IY/d/d3x169Gjh/3hpxFEREQiyeLFi2nfvr3rDHHkUPe/MWaOtbbHoW6vK1MiIiIiFaAxJSIiIlIBGlMiIiIiFaAxJSIiIlIBGlMiIiIiFaAxJSIiIlIBetJOEZEYE7ZhSoOllAZLKC0roLRkFyWluygt3bPvV1k+Jf58SvwFlPqLKQ0WURIo3vc2oVJKQ2WUhv2UhIMEbIgLWp9B/xNudf3HkmP0zDPPMHv2bP79738f9jZDhw6ladOmAFx66aXccMMNdOjQ4ag+1uTJk7nvvvt47733KtQcbTSmRESqUTAcPDBaSoIllARL9g+f/S8rK6C0bC+lpXspPTB4CikJFFEaKKYktP/2oTJKQ35KwwFKbJBSG9r3C0upOfouj7WkWEjGkIwhxXhJ9njZGQ5zw4r/8mSthnTpdmnl/4VIRHjmmWfo1KnTgTH1xBNPOC6KLhpTIiLHaEvRFl5d8hL5Jbso9RdS+sPgCe67ylMSKt0/ePyUhoOU2CBBjv6JkhOsJTlsSbFhkq0lJWxJtpBsPNQ2XpKNj2RvAsmeGqR4k0j2JpOckEKKrwbJCTVISUwlObEWyUlpJCemkZJcl+Tk2iQn1yWlRn1SUurjS6yBMT9fYbv2rOXct07j2rn/4oXa2WS1GFwZf3VSTqNHj2b9+vWUlpZy3XXXcfnll1OrVi2uu+463nvvPVJSUnjnnXdo1KgRkyZN4q677sLv95Oens6LL75Io0aNDryvgoICcnNzWbZsGQkJCeTn55Obm8u9997L7NmzOffcc0lJSWH69OmMGDGC++67jx49evDRRx9x6623EgqFqF+/Pp999hkzZ87k+uuvp6SkhJSUFJ5++mnatm3r8G/KLY0pEZFjsGrPKi5/7xx2BAtJC+8bOclhu2/s2DC1wpb69off77/q4/GR7Ekk2ZtEijeZZF8yyb4UkhNqkpJQk5QDg6c2ycm1SUmuQ1JyHXzJtSGxFiTVgsTUff/rSzpioz8YpqgsSOH+X0VlQXaXBdlQFqRwxw8v81NYtpbCstCB2xSW/v+3KQuGuLD7n3hx82386ovreSH1dWo3aFcNf8MC8NRTT1GvXj1KSkro2bMnY8eOpaioiN69e3P33Xdz00038fjjj3PbbbfRr18/ZsyYgTGGJ554gnvuuYd//OMfB95XamoqAwYM4P3332f06NG8/PLLjB07lvHjx/Pwww8fGE8H2759O5dddhlTpkwhJyeHXbt2AdCuXTumTJmCz+fj008/5dZbb+WNN96o1r+bSKIxJSJylBZsm8dVH12EN1DCyyntaNdiCCTW3D94Ug8aPj/8vib4kuEQV35+6uABtP2HIVQSpHD3/qFTtpfCsp0HjaQQhaUBispCFPwwhvb/8gfD5frzpCR4qZXso1aSj5pJXmol+WhaJ5laST427inh75PLuO2k6/nPpvu5dtJZPH7mpyTWrF/Rv8bo8eEtsGV+5b7Pxp1hxN+OeLMHH3yQt956C4D169ezfPlyEhMTOfXUUwHIy8vjk08+AWDDhg2ceeaZbN68Gb/fT05Ozs/e36WXXso999zD6NGjefrpp3n88ccP+/FnzJhB//79D7yvevXqAbB3714mTJjA8uXLMcYQCATK/2ePQRpTIiJHYcaGr7jus6upG/AzsdFgmp3yAP4wB67qFJQGKfIHKdzzw5WfYgrL8vcNnP2vKygN/ngMle0bQ8c6gH4YQRl1UqiVtO/lNZN8pCbt+9+D//nHb+OjZqIXn/eXf7C7NBDigqdmcs+XHn7X7wwe3PYat71+Gn87dzKeclwdk2M3efJkPv30U6ZPn06NGjUYMGAApaWlJCQkHHhI1uv1EgwGAfj1r3/NDTfcwMiRI5k8eTJ33nnnz97n8ccfz5o1a/jyyy8JhUJ06tTpsA3W2kM+/Hv77bczcOBA3nrrLdasWcOAAQMq/OeNZhpTIiLl9OnK97npq9/T3F/GYy3P4cOkC7jnzv9R5A+V6+1rJHp/PGySfGTUSaRWUq2fDaAfBs/PXp7so2aiD6/nGL7L/BgkJ3iZeH4eYx+ZxkPfHMfF3Tby1J5pZL5+Otee+X65rrZFvXJcQaoKe/fupW7dutSoUYMlS5YwY8aMI94+IyMDgGefffYXb3fBBRdw9tlnc/vttx94WWpqKgUFBT+7bZ8+fbj66qtZvXr1gYf56tWr96OP9cwzzxzDny62aEyJiJTDGwuf50+z7qFzWRn/7nI9T+YP4qH/LeKE1vU5LqfegSs9qcmHvhpUnQOostWpkcgzF/VizCPTeG3xGYzM2czjpavJfP9yxpx6+IeJ5NgNHz6cRx99lNzcXNq2bUvv3r0Pe/s777yT8ePHk5GRQe/evVm9evUhb3fuuedy2223cfbZZx942YUXXsiVV1554BvQf9CgQQMmTpzImDFjCIfDNGzYkE8++YSbbrqJCRMm8M9//pOTTjqpcv7AUcxYe/Q/WVIZevToYWfPnu3kY4uIHI0nZ9/P/Quf5PiSMu7rcxd3r2zPSzPXcVbPLO4a3emwD5PFkgUb93LGY9PJqZ9E03q3Myu4h/+0PIe+MfgcVIsXL6Z9+/auM6rE66+/zjvvvMPzzz/vOiViHer+N8bMsdb2ONTt4+MzgIjIMbDW8o+pt3P/wicZUVzGPwY+zI3zWvLSzHVcPbAlfx3TOW6GFECnjNo8fG53lmwpIeC/kxYkcsOKF1k2/7+u06Scfv3rX3PLLbf86CE+qbj4+SwgInIUguEgd3x6Dc+sepuzigPcPuw5LvmyFh8t3MIdp3bgd8PaHfIbc2PdwLYNuWt0JyYvK6NVyp+oiZdfzbqbbeu+dp0m5fDQQw+xYsUK2rRp4zolpmhMiYj8RFmojBs+mMDbm6ZwVYnlyhFvcNZ7AWav2c39Z3bl4n4//5HzeHJ2r2ZcM7AVr841DKt3MwXGcPUnV1C0a5XrNBEnNKZERA5S6C/kqnfG88XOedxSlsioEe8w5rVtrNpexBMTejC6W4brxIjw26FtGNMtg/9Mr8N5DS5juRd+9/ZYgiV7XKeJVDuNKRGR/XaW7OTit0YzN38Vfw3WJm/wm4x+YTV7SwK8eNlxDGjb0HVixDDG8LexuRzfKp0HprXmovRTmeoN8tfXTsMG4/sJHCX+aEyJiACbCjZy4VsjWV28mQe8mTTq9xLjn12Cz2N47Yo+dG9W13VixEn0eXjkvDxaNazF47MHcmZqV161e3j2rTPB0U+Ki7igMSUicW/lruWc//ZodpbuYWLNTpR1fZxzn19Aw7QkXr+qL60bpbpOjFhpyQk8fVFPaiX5mLTkAgYlNuUfxcv5+OPrXafFnDvvvJP77rvPdYYcgsaUiMS1eVu/ZcJ7ZxD2F/F0g/4sz76HK/47j/ZN0njtyr5k1ElxnRjxmtRO4emLelJcFmbx5hvpYmpy6+bP+G7Gv1yniVQLjSkRiVvT1n7BpR9dSKq/hGebj+OL1N9y05sLOL5Vff576XHUq5noOjFqtG+SxqPn57FyRwDK7qCx8XHtoidYt/ht12lR7e6776Zt27YMHjyYpUuXAjBgwABuvvlmevXqRZs2bZg6dSqw71iXMWPGMHz4cFq3bs1NN93kMj2uaEyJSFz6eNmbXP3FtWSVlfFMh1/xXOlZ/O2jpZzWpSlPTuhJzSSdtnW0jm9Vn7+PzeWrlZbWvv/DGg+/mnYrezZ96zotKs2ZM4eXX36ZuXPn8uabbzJr1qwDrwsGg8ycOZP777+fP/7xjwde/t133/HKK68wf/58XnnlFdavX+8iPe7os4WIxJ1Xv3+cu+Y+SNcyPw/0+gN/XtaBN+euZkKf5vzhtI54ovQMvUgwNi+TTXtK+Mcny7io53W8W3A/130wgYnjPyKpdnQ+rcTfZ/6dJbuWVOr7bFevHTf3uvmwt5k6dSqnn346NWrUAGDkyJEHXjdmzBgA8vLyWLNmzYGXDxo0iNq1awPQoUMH1q5dS1ZWVqW2y8/pypSIxA1rLY/P+Bt//u5B+pX5eeiEB7jhu1a8OXcjvx3ShjtHakhVhmtOasVZPbN4elYTzqhzDt8mwO1vjCJcVug6Ler80rPsJyUlAeD1egkGgz97+aFeJ1VHV6ZEJC6EbZj7vryZ59d+xKklQW4a9DQXfwpz123nL6d35pzjmrlOjBnGGO4a3Ykt+aVMnNmVS3pu4aX8L8h4dSTXnfsJeLyuE4/Kka4gVZX+/ftz4YUXcssttxAMBpk0aRJXXHGFkxY5PF2ZEpGYFwgHuP2jy3l+7UecW2q5btgrnPlegPkb9vLwOd01pKqAz+vh4XO6075JKi/MHcHJKW15Iryd198+z3Va1OjevTtnnnkmXbt2ZezYsZxwwgmuk+QXGOvoidV69OhhZ8+e7eRji0j8KA2W8rv3z2fyniVcXZbAsMEvc96rG9hbEmDiBXn0bVnfdWJM21ZQyukPT6M0GKB7zt+YFdjOw5mncvzgv7lOO6zFixfTvn171xniyKHuf2PMHGttj0PdXlemRCRmFfgLuOKt0/ly92L+L5RKv5PeZOyLaykNhHj58t4aUtWgYWoyz17ck2DIsGrLjbQ0Kfx2/SSWzn7MdZpIpdGYEpGYtKN4Bxe/cSrzitbzd08Tsvq8xBnPLaFGopfXr+pLp4zarhPjRquGqTx+QQ/W7/LgK7mVmsbHr75/gK0r/uc6TaRSaEyJSMzZmL+eCW+eytrSHTyU0o5w7hNc+MJCmtWrwZtX9SWnfk3XiXGnV049/nFGF75Zk0gbcyNFXi9XT76eoq0LXKeJVJjGlIjElOU7F3PB26PZ489nYvrxrMq6h2teWUC3rLq8ckUfGqYlu06MW6d1acrvR7Tjw8X1OCnlMlb4PPx20jkEC7a6TjskV99TLG4dy/2uMSUiMeO7Td9w4XtnYQPFPJ01is+Tf8sd7y5mULuGPHdJL2qnJLhOjHuX92/BBX2a89+5LRmVOoqvEyx3vz4S6y9xnfYjycnJ7Ny5U4Mqzlhr2blzJ8nJR/cfXXqeKRGJCV+v/pjffPk7GgT9PNL+CiZuH8wLM1YwLi+Tv43pjM+r/3aMBMYY/nBaRzbvLeW52X05u/sWXi+eSdZro7n47A/BExn3U2ZmJhs2bGD79u2uU6SaJScnk5mZeVRvo6dGEJGo9+Gi/3LrzL/SMhDg33m38ufFnXh//mauOLEFtwxv94vPIi3ulPhDnP34DBZv3sOQjo/wpX8t99bpyfBRT7lOEzkkPTWCiMSsV+b8m5tn/pUu/gCP9L2P337bhvfnb+b/Tm7P70e015CKUCmJXp6c0IPGtWvw9fKryPXU5v92zWTu5D8e+Y1FIozGlIhEJWstj351J3cteIwT/UH+fuLjXPx5HWas2sU/xnfhsv4tXCfKEaTXSuKZi3phTCIbt/yOxp5Erl31Cmu/e851mshR0ZgSkagTtmH+/um1PLzyDUaWWW4a+CJnTQqxfFsBj1+Qx9i8o/t+B3Enp35NnpjQg817E0ksvAnj8fGr2X9j9+ovXaeJlFu5xpQxZrgxZqkxZoUx5pbD3K6nMSZkjBlXeYkiIv9fIBzg1vcn8OKmyZzv9zFh4GuMfXUXOwvLeOGS4zipXSPXiXKUujerywNndeO7Dam0Dl/DVp+Haz/9FWU7lrtOEymXI44pY4wXeBgYAXQAzjbGdPiF2/0d+LiyI0VEAEqCJVz/9jje3/kd14ZqMfiE1xn34jqMgdeu7EuP7HquE+UYDe/UmD+c2oHPlmXQP+Fcvk8w/N/b4wkX73SdJnJE5bky1QtYYa1dZa31Ay8Dow5xu18DbwDbKrFPRASA/LK9XPnGaUzNX8ntpiGtev6Xs19YRv1aSbx+ZV/aNk51nSgVdOHxOVx2Qg5vzs9leMoQPk4I8cCrIyFY5jpN5LDKM6YygPUH/X7D/pcdYIzJAE4HHq28NBGRfXYUbeOi109mXskW7k1ujbf941z60mLaNk7ltSv7kFWvhutEqSS/H9GeU3Kb8OrcQQxK6sxTJp9XXx8PevJMiWDlGVOH+rnin/6/+n7gZmtt6LDvyJjLjTGzjTGz9URoIlIe6/es5vw3TmG9fw8P1+nF2ib38Ns3FtO7RT3+e1lv0msluU6USuTxGP4xvgu9stP5YP459PQ14S+lq5j6/tWu00R+UXnG1AYg66DfZwKbfnKbHsDLxpg1wDjgP8aY0T99R9baidbaHtbaHg0aNDi2YhGJG0u3fs8F75xOQbCIxzNO5suEG/nLh8s4pXMTnrqwJ7WSdIhDLEpO8DLxgjyy6tVi9opraOmpxY3bv2TJV/e6ThM5pPKMqVlAa2NMjjEmETgLePfgG1hrc6y12dbabOB14FfW2rcrO1ZE4sfc9VO46MPz8QTLeKr1BJ7fex6PTV3Neb2b8eDZ3UjyeV0nShWqUyORZy7qRaK3Bls3/5ZUk8jVS59my8LXXaeJ/MwRx5S1Nghcw76f0lsMvGqtXWiMudIYc2VVB4pI/Jmy7B0u/+xq0oMBnsy9kXtWncTrczZw/eDW/HlUJ7wePat5PMiqV4OnL+zJzsJUUgqvo9jr5erpt1O4fobrNJEf0dl8IhJR3p/3FLd9+y9aB4Lc1+sv/HZ6I2av3c2fRnbk/D7ZrvPEgc+XbOXSZ2fTr+UK5vsep1fA8u9x75NQN9t1msQRnc0nIlHhxW/u5Za5/6JrIMR9x/+Hy79I57v1e3jo7G4aUnHspHaNuGt0Z6asaEUf72imJXq4+40x2JI9rtNEAI0pEYkA1lr+88XN/G3Jc5zkt9ze/1nOmWRZt6uYpy/sxam5TV0nimPnHNeMqwe25IOFfRiQ0Ic3EgI8+eooCAVcp4loTImIW2Eb5i8fXc4j6z5gdMDLZSe8wvhX91DsD/HSZb3p17q+60SJEDcObcvp3TKYNG8kvRNa8QC7+OCNc/QcVOKcxpSIOBMIBbjlnbN4edsMLgzV4NTer3Lmf9eTnODltSv70CWrjutEiSDGGP4+Npe+LevzxcKL6ORJ57bixcz5+EbXaRLnNKZExIlifxG/fuM0Pty7mN+Y+nTq9l/Of2klGXVTeOOqvrRsUMt1okSgRJ+HR8/Po2X9OixYdS1NTArXbfqQ1TP+7TpN4pjGlIhUu70lu7n89RFML97AnYnZJLd6nCtfXULnzNq8ekUfGtdOdp0oESwtOYGnL+pJzYTa7N58PR5PAr9a8DC7lrznOk3ilMaUiFSrbQWbuPD14Szy7+Le2t3Y2OA+bn13KQPbNuSFS46jTo1E14kSBZrWSeHpi3qSX1KfWvlXsd3n49dTf0fp5u9dp0kc0pgSkWqzbudSLnjzVDYFC3m40RC+Mjfxz09XMKZbBo+dn0dKop7VXMqvfZM0HjmvO8u2NKO1/xzmJ3i5ddK5hPdudJ0mcUZjSkSqxdLNs7hg0hkUhUqZmHM2L+4+n+dmrOOyE3K4b3wXErz6dCRH74TWDfjb2Fymr8qlhxnKJ0mGf702CsoKXadJHNFnLxGpcnNW/Y+LPr4YXyjA4x2u5b6VQ3hv3mZ+P6Id/3dKBzw6HkYqYFxeJjcMacPni0/iOG9Xnkko4+VXR0Mo6DpN4oTGlIhUqS8XvcwVU35LejDEf7rdyc2zO/H1ih3cMy6XK05s6TpPYsSvT2rFWT2b8emCM+jqzeSvoS1MeftCPQeVVAuNKRGpMh/NfYzrZt5Fq2CIfxz3L678oiFLthTw2Pk9OKNHlus8iSHGGP48uhMntmnEtMVX0MJTmxvz57Lo89tcp0kc0JgSkSqxYcdi7vj+IboE4fY+T3LBJB/bC8p47uJeDOnQyHWexKAEr4eHz+1Ou0bpLFv9a9I8SVyz5k02z3nSdZrEOI0pEal01lr++PEVeKzl0o5/4pzXCwhZy6tX9OG4Fumu8ySG1Ury8fSFPamT1JCCzb+m2JvAr769l4IVn7pOkximMSUile6taX9lRnA3FyZ24rIPUqhbI4E3r+pL+yZprtMkDjRMS+aZi3pSUtqU2vkXsyYhgRs+/zWBrYtcp0mM0pgSkUq1dfcq7l3+Ej0ChieWn0N2ek1eu7IvWfVquE6TONK6USoTL+jB6q1taVl2OjOSfPz5nTOxBVtdp0kM0pgSkUpjreWujy4laMM0DV9MQTCR/5zbnQapSa7TJA71bpHOfWd0Yfbq48gNH89bSfD4qyPBX+Q6TWKMxpSIVJoPZz3AZP92zva04sVVrblxaFta6MBicWhkl6bcMqIdXy89lVzTlod8xbz36jgIh1ynSQzRmBKRSrErfyN/W/gknYPwypqL6NasDhf3y3GdJcIV/Vtwfu9svl50Pu1MQ+4IrmfWu5frOaik0mhMiUil+NuHl1BgLC3D57E7kMS943Lx6pnNJQIYY7hzZEcGt89gztIraWxqcv2u6ayacrfrNIkRGlMiUmGff/sYH5Zu5EzTjOdXduI3g9vQqmGq6yyRA7wew0Nnd6Nz06asXnMNXk8iv1r+Iju/f9F1msQAjSkRqZD8ou3c9f2/aRO0TFp7GV0ya3PZCXp4TyJPSqKXJyf0oEFKJqVbrmCHz8e13/yZkjVTXadJlNOYEpEKue+Di9llLJ3sGWwvS+be8V3wefWpRSJT/VpJPHtxLwL+ltTZew7zExP44/9+BeGw6zSJYvqMJyLHbNq853ireA1jaMKzK/K4bnBr2jTSw3sS2XLq1+SJCT1Zv70rnUrb80GiZevyD1xnSRTTmBKRY1Jcsoc/zrmP7KDls/WX0ykjjcv7t3CdJVIuec3r8sBZ3Zi7aQjWGD747nHXSRLFNKZE5Jg88MElbDZh8uxoNpXW5N5xXUjQw3sSRYZ3aswFffqRWZLEuwXLsf5i10kSpfSZT0SO2reLX+OlgqWMtA14ZkUfrhnYWufuSVQ6p1czwvldWJHgZel3T7vOkSilMSUiR6W0rIA/zLiLJmHL1xuvoEOTNH41sKXrLJFj0jy9JjVrjcNnLZOWvuo6R6KUxpSIHJVHPryMNZ4wfcKnsr44jXvH5+rhPYlqY/M6kFVUmw/82wnqIGQ5BvoMKCLltnD5+zy7ZwEjwnV4ZkV/fjWgJR2b1nadJVIhp3ZuSkF+P3b4vHwz8wHXORKFNKZEpFwC/hLu+Or/qBe2fLf5Cto1TuWak1q7zhKpsNo1EmiTeTq1QjBp7f9c50gU0pgSkXJ58uOrWOYJcYIdwsrCetw7rguJPn0Kkdgwtns2jQoa87kpoWjLPNc5EmX0mVBEjmjF6s95bOdsBoZSeXb5YK48sQWdM/XwnsSOAW0bsrtkGCUeD5/qoT45ShpTInJYoaCfO768kdSwZdnWy2ndMJVrB+nhPYktiT4P/dsNo0HAw6St3+h4GTkqGlMiclgv/O/XzDcBBtj+LMtvwL3ju5Dk87rOEql0Y/OyqLm3JTMTYIuOl5GjoDElIr9o3fqveWjb1xwfqsFzy0/msv4t6JpVx3WWSJXoklmbUjNm3/Ey3+t4GSk/jSkROaRwKMgfPruORGvZsP1yWjZI5TeD27jOEqkyxhhGdT2OzJIkJuUvx/pLXCdJlNCYEpFDev3T3zLblHFS+DgW7mnMPeO6kJygh/ckto3qmoHdm7vveJnvdbyMlI/GlIj8zJbNc/jnps/oEUriheWnc2m/HPKa13WdJVLlsurVoFbtM0mwlklLXnGdI1FCY0pEfsSGw/zxf78iBOzccSk59VP57dC2rrNEqs34vPY6XkaOisaUiPzIpC9+z1cUM8R2Y/7uLO4Zl6uH9ySujOjchIL849nh8zJj5oOucyQKaEyJyAE7ti3g7+veIzeUwEvLxnFh32x6ZtdznSVSrdKSE2jXbOz+42U+dp0jUUBjSkT2sZa/fHQ5pcZQsvMimqWn8bthenhP4tP4vJyDjpeZ7zpHIpzGlIgA8MmUP/KJLWBwuCPf7mzB38fmUiPR5zpLxIkTWtdnb+kwSj0ePp2l42Xk8DSmRIS9O5dz98rXaBvy8vrys5nQpzm9W6S7zhJxxuf1MKD9iP9/vIy1rpMkgmlMicQ7a7nng0vY6zGw5wKa1k3jpuHtXFeJODc2L5PUvS2Z6bNsWfah6xyJYBpTInFuyrR7eDe8m5PCbZi9rS1/H5NLzSQ9vCfSsWkafu8Px8tMdJ0jEUxjSiSOFe5Zy5+WPkdOyMM7K87j3OOa0bdVfddZIhHBGMPIrn3IKknk3fxlOl5GfpHGlEgc+9f7F7HdY0jeczaN0urw+5Pbu04SiSijuzWFvV1YmeBlyffPuM6RCKUxJRKnZn3zIK8GtzMgnMPMbZ3529jO1NLDeyI/0qR2Cmn1ztLxMnJYGlMicagkfxN/WDiRzJDhwxUXcFbPLE5o3cB1lkhEGt+9w77jZQLbCBZuc50jEUhjSiQOPfz+xaz3GurkjyU9tR63nqKH90R+yfBOjSkqOJ6dXi8zZuo5p+TnNKZE4sy8b5/g+bIN9A9lMH1LD/46pjNpyQmus0QiVs0kHx2zx+87XmaNjpeRn9OYEokj/qLt3DH3fupbw+erLmJ8XiYD2jZ0nSUS8cZ2z6bxD8fLbF3gOkcijMaUSByZ+N5FrPQZGuePIq1GOred2sF1kkhUOL5VfQrKhu87Xmbm/a5zJMJoTInEiaXfv8CTJWvoE2rI15v78Ncxnamdoof3RMrD6zGc1HEEDfw6XkZ+TmNKJA4ES/Zwx+y/k2YNX626hDHdMjipXSPXWSJRZUxeJqn5+4+XWa7jZeT/05gSiQPPvncxi3yQWTicmikNueM0PbwncrTaNU4jlDAOawzvf6fjZeT/05gSiXGrF73OfwqX0SOUztcbB3D36E7UqZHoOkskKo3u1puskkQm5S/DBkpd50iE0JgSiWHhskL+MP2PJGOYvfoiRnVtytCOjV1niUStkV2bwt7c/cfLPOs6RyKExpRIDHvp/cuY64MWRQNJSmrKnad1dJ0kEtUapiaTXv/c/cfLvOw6RyKExpRIjNq47AMe2DuPLqHaTN0wlLtGd6JuTT28J1JR4/La06yoNh/4t+p4GQE0pkRikvWXcOfU32MwzF97EafkNmV4pyaus0RiwtAOjSkp3H+8zKyHXOdIBNCYEolBb394JTN8YVoX98Pra8afRurhPZHKkpLopVP2GdQKWd5d85HrHIkAGlMiMWbbqk+5d9ds2odq8dX6U/jTqI6k10pynSUSU8b2yKZJQWO+oJiibTpeJt5pTInEEBso5c9f3EjAGFasu4jhHZtySmc9vCdS2XrnpFMUGEGpx8MnMx9wnSOOaUyJxJCPPr6Oyb4QbYt7EfZk8+fRnTDGuM4SiTkej+GkjifT0G+YtGWGjpeJcxpTIjFi19qv+Ou2qbQOp/DVutP548iONEjVw3siVWXs/uNlZvksW1Z87DpHHNKYEokFoQB/+/RaCjwe1q2fwJAOTRjZpanrKpGY1qphKibxjP3HyzzmOkcc0pgSiQFf/O+3fOgL0KGkKwHbmrv18J5ItRjVvTfNShKZtHcpNlDmOkcc0ZgSiXL5G2fz502f0jycxNdrx/OH0zrQMC3ZdZZIXDitS1NMfhcdLxPnNKZEolkoyD8+vpKdXg/bNp7LSe2acnq3DNdVInEjvVYSDRvuO17m3SUvuc4RRzSmRKLY9M9v5U1vGR1LO1IS7MBfTu+sh/dEqtnY7h1oXpTGh/6tBIu2u84RBzSmRKJU8ZZ5/HHde2SEE5i29ixuP7UDjWvr4T2R6jaofUNKi/YdLzN95oOuc8QBjSmRaBQO8eCHl7HJ6yF/01mc2DqD8XmZrqtE4lJygpfcFmfpeJk4pjElEoXmTv4j/zVFdCxrTb6/C38do4f3RFwa1yOHpgWNmUwxRdsWus6RaqYxJRJlyrYv4Y5Vr9HA+pi+5jxuO6U9TeukuM4SiWs9mtelJHiyjpeJUxpTItEkHOaR9y9hTYKP0i3jOKFVJmf2zHJdJRL3jDEM6nyKjpeJUxpTIlFk0Vd/4xn20qGsOXtKeujhPZEIMqZ7Jmn5LZnlC7Nl5f9c50g10pgSiRKBXau4Y+nz1LZevlkzgd+f3J7MujVcZ4nIfjn1a+JNPhNrDO/N1fEy8URjSiQaWMtTky5iaaIPu20UfXOyOKdXM9dVIvITo/L60LwkkUl7l+h4mTiiMSUSBVZMv5/H7E7a+jPYUdSXv4/NxePRw3sikea03CZ48juzKsHL4nnPuc6RaqIxJRLhQnvX84eFE0mxHuasmcDNw9uRVU8P74lEojo1EmnU6HwSrGXSYh0vEy80pkQimbW88O6FzEv04dtxMj2bZXN+7+auq0TkMMb16Eh2YRof+rcQLNrhOkeqgcaUSARbN+tR/h3cQqtAQ7bm9+cePbwnEvEGtm2Iv7jfvuNlZj3kOkeqgcaUSIQK52/mzu8exGu8zFtzETcNa092/Zqus0TkCBJ9Hrq0OpvUkOXd1R+6zpFqoDElEoms5fVJFzEryUfKjkF0y2jBhX2zXVeJSDmN7ZFNk4LGfEERhdsWu86RKqYxJRKBtsx9hn/615MTqMemPUO4Z5we3hOJJt2y6hAIn0yZx8MnM+93nSNVTGNKJMLYwu38cfY9hIyHRWsv4sah7WjRoJbrLBE5CsYYBnc+lYZ+w7tbput4mRinMSUSYd6bdAlfJflI3dWfzk1ac3G/HNdJInIMTu+eSVp+C+b4wmxZ+YnrHKlCGlMiEWTHvJf4e8kKmgVrs2HXydw7rgtePbwnEpWy6tUgqeZZ+4+XedR1jlQhjSmRSFG8i7/M+DMlHi/L1l7IDUPa0aqhHt4TiWaju/cl+4fjZYJ+1zlSRTSmRCLEp+9dwSdJXurs6U2Hhu24VA/viUS9k3Ob4CnI3X+8zLOuc6SKaEyJRIC9C9/iroKFNA3VYv32Udw7vgs+r/71FIl2ackJNGlywb7jZRa97DpHqog+W4u4VrqXe766jT1eL2vWXsB1g9vRplGq6yoRqSRn9OhAdmEaH/g363iZGKUxJeLY1Pev4t1kD/Xz82hTvzNX9G/hOklEKtEJrRsQLOnHLq+XabP+7TpHqoDGlIhDhUs/4E975tIolMLabWO4d3yuHt4TiTEJXg/d2pyz/3iZD1znSBXQZ20RV/zF3D/5ZrZ6fWxYdx6/Htiedo3TXFeJSBUYl5dD04LGTKaIwu1LXOdIJdOYEnFk0dS/8EoyNC7oRIu63bhqQEvXSSJSRTplpBGyPxwv8y/XOVLJNKZEXCjdy4Mr3qBW2LBq6zjuHZ9Lgh7eE4lZxhgGdzmNRn7Du5t1vEys0WdvEQdmf34HXyf78O04jl/170THprVdJ4lIFTu9Wya1fzheZtWnrnOkEmlMiVQzW7CNBzd8TJ2Ql1L/6Vw1oJXrJBGpBk3rpJCSeraOl4lBGlMi1Wzq579nblICoe39uXZQR1ISva6TRKSajM47nuySBN7do+NlYonGlEg1Cu9Zx4PbviY9mIDXjOTMHlmuk0SkGg3v1BhfQS6rEzwsmve86xypJBpTItXo409/x9LEBIq3DuG3Q9qT6NO/giLxpFaSj6yMCSRYy7uL/us6RyqJPpOLVJPA9qX8e888GvmTqVtjBCO7NHWdJCIOjO3RiezCVD7ybyZYvNN1jlSCco0pY8xwY8xSY8wKY8wth3j9KGPMPGPMd8aY2caYfpWfKhLd3vn0RtYl+Ni59VR+N7Q9Ho9xnSQiDhzfMp1w6f7jZWY+5DpHKsERx5Qxxgs8DIwAOgBnG2M6/ORmnwFdrLVdgYuBJyq5UySqlW6cwyPFK2hSWpPseoMY1L6h6yQRccTn9dC97Xn7j5f50HWOVILyXJnqBayw1q6y1vqBl4FRB9/AWlto7YFnIKsJ6NnIRA7yyuc3s83nY/PWMdw0rD3G6KqUSDwb3yOHjIJGTKaQwu1LXedIBZVnTGUA6w/6/Yb9L/sRY8zpxpglwPvsuzolIkDhqsk8EdhEZkkdumacSJ+W6a6TRMSx9k3SsJyq42ViRHnG1KH+E/pnV56stW9Za9sBo4E/H/IdGXP5/u+pmr19+/ajChWJStby3JT/Y4/Xy5otZ3DTsHaui0QkQgztOpJGfsM7m6fpeJkoV54xtQE4+MlwMoFNv3Rja+0UoKUxpv4hXjfRWtvDWtujQYMGRx0rEm12L53Es3YPmYUNGNi6L50zdWyMiOwzulsGdfJbMCfBslnHy0S18oypWUBrY0yOMSYROAt49+AbGGNamf3fBGKM6Q4kAvp5T4lv1vLE13+m1BhWbzuLG4a0dV0kIhGkYVoytWqfA6DjZaKc70g3sNYGjTHXAB8DXuApa+1CY8yV+1//KDAWuMAYEwBKgDMP+oZ0kbi0Zd6LvOwpoWlBJp06HUerhrVcJ4lIhBmddzw7v0lgUmAJlwYDGF+C6yQ5BuV6nilr7QfW2jbW2pbW2rv3v+zR/UMKa+3frbUdrbVdrbV9rLVfVWW0SMQLh3h01n1YY1iz41yuH9LGdZGIRKChHRuR8MPxMvN1vEy00jOgi1SBtbMf421fkAZ7WnBWj+5k1ElxnSQiEahGoo/mWReRGNbxMtFMY0qksgX9PPz9IyRYw4a953D1wFaui0Qkgo3v0ZHsolQ+KttMoGS36xw5BhpTIpVsyfR/8GEi1NndgUv6dKV+rSTXSSISwXq3SIeyfuzyepg+80HXOXIMNKZEKpO/mIcWv0CtMGwrOptL+7dwXSQiEc7jMeS1v4DUkOWdVR+4zpFjoDElUonmTvkzU5I8pOzI4+oTO5OWrJ/MEZEjG5+XTWZBI76kkIIdy1znyFHSmBKpJLZkD/evfpvaIQ+FwTO4oE+26yQRiRKtG6Xi8ew7XuZTHS8TdTSmRCrJ11/cxreJPrzb+3DdSR1JTvC6ThKRKDK026h9x8ts0vEy0UZjSqQShAu38cDGz0gPesEzljN6ZB35jUREDjKyawZ183OYkxBm8+rPXefIUdCYEqkEn3x2M0sSffi3ncQNQzqQ4NW/WiJydOrXSqJOnXMBeO9bHS8TTfQZX6SCgrvX8e/tM2gUSCStxihOy23qOklEotSonv3IKUlg0p5F2GDAdY6Uk8aUSAVN+uy3rEnwUbB1BDcNa4fHY1wniUiUGty+EYlFOl4m2mhMiVRA2bbF/GfvAjLKUsisN4KBbRu6ThKRKJac4KVF1sUkhi3v6HiZqKExJVIBr332O7b4fGzbMoqbhrXDGF2VEpGKGdezEzk6XiaqaEyJHKOiDbOYWLKKrJJUOmcO4bgW6a6TRCQG9GheF+Pvx26vh+mzHnKdI+WgMSVyjJ6ffAu7vV42bBnH74a1dZ0jIjHC4zH06rDveJm3V77vOkfKQWNK5BjsWfUFzwa20LywLie0GUCnjNquk0Qkhozr0YKsgkZM0fEyUUFjSuQYPDX1NoqMYe32c/jtkDauc0QkxuTUr0mC77T9x8vc7zpHjkBjSuQobVv8Dv8N76F5QWNO6dyXFg1quU4SkRg0tNsoGvsNb2/+2nWKHIHGlMjRsJbHpt9FyBhW7zyXawe1dl0kIjHqtC4Z1CvI4VtfmE06XiaiaUyJHIX13z/Pm54SMvY248wex9G0TorrJBGJUXVrJlK37g/HyzziuEYOR2NKpLzCIR6e8y+8FtbuOZ9fDWjpukhEYtzonv1p8cPxMqGg6xz5BRpTIuW0bNZ/+MAboOHuNlzcN4/0Wkmuk0Qkxg1s14CkolzW+DwsmqfjZSKVxpRIeYQCPDR/IjWsYXPReVx6Qo7rIhGJA0k+Ly2zfzhe5kXXOfILNKZEyuG7r//O5ASos7MzVw/oSmpyguskEYkT43p0psWB42X2uM6RQ9CYEjkC6y/mwaUvUTtk2BM4l/N6N3edJCJxpHuzOvgCOl4mkmlMiRzB9C/vZFaih+QdPbluUGeSE7yuk0Qkjhhj6NXxItJClrd0vExE0pgSOQxbspcH175HetBD0HMW4/MyXSeJSBwam5dNZkHDfcfL7FzhOkd+QmNK5DA++/z3LEzwwrb+/HZIR3xe/SsjItWvWXoNkhNG4vcY/jfzn65z5Cf0lUHkF4QKt/HQ5sk0DvhIqTGOUzo3cZ0kInFsWN7pNPbDu5t0vEyk0ZgS+QXvffpbViV4Kdk6hJuGt8fjMa6TRCSOnZLblPSCFvuPl/nCdY4cRGNK5BD8u1bzn51zyChLokn6aAa0aeA6SUTiXO2UBOqnnw/ApG//47hGDqYxJXIIr312I5t8XnZvPZWbhrfDGF2VEhH3Rvc8gRYlCbyn42UiisaUyE8Ub13IxILFNC+pQfvMEfTMruc6SUQEgBPbNiCleP/xMvNfcJ0j+2lMifzEi5//jl1eL5u3jOF3w9q5zhEROSDB66FNziUkhi1vLdTxMpFCY0rkIHs3fMPTJWvJKapN37ZD6di0tuskEZEfGdezMy2KavG/sk0ESve6zhE0pkR+5Okvfk+hx7B+25ncMKSN6xwRkZ/pnFGbxOAJ7PZ6mKbjZSKCxpTIfjtWfsaLwa20KKzP8NwB5NSv6TpJRORnjDEc13nf8TJvr3jPdY6gMSVywGNf3U7AGNbsPI/rBrV2nSMi8ovG5uWQ9cPxMrtWus6JexpTIsCGRa/zus0ne29TxvfoS+Paya6TRER+UUadFGom7T9e5hsdL+OaxpSItTwy4+94LKzdO4GrBrRyXSQickTD8sbQxA/vbvrKdUrc05iSuLdi7tNM8pSQuSeHC/v2ol7NRNdJIiJHdHJuU+rvP15m4+rJrnPimsaUxLdwmH/PfZAUC5uLJnDJCTmui0REyqVWko+GDXS8TCTQmJK4Nv+bB/nMF6LhrnZcMSCPWkk+10kiIuU2uld/Wpb4eG/PQh0v45DGlMSvUIAHFj5J7RDs9E/g3OOauS4SETkq/VrVp2ZxF9b6PCxcoGdEd0VjSuLWjKl/4ZsESNvRlesHdyU5wes6SUTkqPi8Htq22He8zNsLdFafKxpTEpesv5gHVrxGetBQ4jmfMd0zXCeJiByTcb260PLA8TL5rnPiksaUxKUvJt/BggRD0vbe3Di0Mz6v/lUQkejUoWkaKeH+Ol7GIX0FkbgTKt7Ng+s+oHHAg6/mOYzo1Nh1kohIhfTOvYjaIcvbKya5TolLGlMSdz74/BZWJngJbxvITcM64vEY10kiIhUypvv/P14mf6eOl6luGlMSVwKFW3l461Qyy3ykp4+nf+v6rpNERCqsUVoyqcn7j5eZqeNlqpvGlMSVNz65gY0+L0XbRnDz8PYYo6tSIhIbhvccu/94ma9dp8QdjSmJGyW7VvPY7rlklyTRKnM0ec3ruU4SEak0wzs1oUFBC+b6Qmxc86XrnLiiMSVx47+f3cAOr5ed20bxu2HtXOeIiFSqGok+Gjfaf7zMHB0vU500piQu5G+dz1MFy2hZVJOebU6lfZM010kiIpVuzHEDDjpeJuQ6J25oTElceObzm8j3eti8fTw3DGnjOkdEpEr0bpFOanEX1voMCxf+13VO3NCYkpi3Y900XihdR+uCOgzOHUrz9Jquk0REqoTXY2jf+vJ9x8vMf951TtzQmJKY98SUW/Ebw4Zd53DtoNauc0REqtS4Xp1pVVSLj8s26niZaqIxJTFt0/KPeTW4g5b5DTm95wAapSW7ThIRqVJtGqVS0/Znj9fD1zpeplpoTElMe2TanRhg3Z4JXNm/pescEZFq0bvLJfuPl3nPdUpc0JiSmLVqwSu8awvI3p3JBcf3pW7NRNdJIiLV4vRu2TQraMBUCsjftcp1TszTmJLYZC0PzbqXJAsbiy/m4n45rotERKpNg9Qk6qSM0vEy1URjSmLSwm8f51NPGZm7W3LFgJ7UTPK5ThIRqVbDe42niR/e2fiV65SYpzElsScc5sHv/kNayLLdfwnnHNfMdZGISLUb2rExjQpz+M4XYvP66a5zYprGlMScWTP+yTRfiAY7O3Ht4G4k+byuk0REql1ygpesBiMB+GbJ225jYpzGlMQUG/TzwOJnSQ9CgbmQ07tluE4SEXFmWM9TSA5b5mz+znVKTNOYkpgyZeqf+d4HqTvyuHFYF3xe/V9cROJXXnYDMst8LPdvcZ0S0/SVRmJG2F/Mg6veonHAQI0LGd6pseskERGnaiX5qBdKZ6U3SNBf5DonZmlMScz48ItbWeYz+LYfz03DO2KMcZ0kIuJc/eQOlHo8rFz1ieuUmKUxJTEhULKbhzd8QpbfQ+308+nXqr7rJBGRiNAiYyAAs1Z85rgkdmlMSUx467Pfsd7nIbB1MDcNb6+rUiIi+/Vo15+aoTDzdyx0nRKz9EyGEvVKCzbz2Lbp5ASSaJh1Ft2b1XWdJCISMTpm1CWjLJEVCTtcp8QsXZmSqPfypzewzeshf9up/G5YO9c5IiIRJTnBS3q4Eat8Yfwle1znxCSNKYlqBTtX8MTuebQuTqZbmzG0bZzqOklEJOI0qNmJoDEsXfGh65SYpDElUe25z37LXq+H7dvG8pvBbVzniIhEpNbNhgIwc+UXjktik8aURK1dm7/juaIVtCuoxYAup9IsvYbrJBGRiNSz7XHUDoVZsHuJ65SYpDElUeuJyTdTagybdp3Nr09q5TpHRCRitW2cStOyZFaGdrtOiUkaUxKVtqydyiulG2mXX4+RvYbSMC3ZdZKISMTyeT3Us01Y67MUF+pomcqmMSVR6dGpt2ENbNh7AVf2b+k6R0Qk4jVK7UrYGBYte991SszRmJKos2bZ+7wd3EnrPY05p98AatdIcJ0kIhLx2rcYAcDM1VMcl8QejSmJOv+efheJFjYVXcxFx2e7zhERiQq92nShftCyaM8K1ykxR8+ALlFl8fz/8jGFdNjdjJMH9qVGov4vLCJSHtnpNWlclsLKxL2uU2KOrkxJ9LCWB2f/k7SQZWvgUs7u1cx1kYhI1PB4DOlksiHBkL97jeucmKIxJVFjzpxH+cpTRsauNvx6cC8Sffq/r4jI0WhcpwcA85a967gktuirkUQFGwrxwLzHqBe07DGXMbpbhuskEZGo06n1yQDMWjvdcUls0ZiSqDB1xj3M9YZosCOXG4Z1w+sxrpNERKJOz9btaBywLClY5TolpmhMScQLB/08uPS/NApAsOZlDOvYyHWSiEhUalo7mYZltVhJIVjrOidmaExJxPvflD+w1Aup23vxu+GdMUZXpUREjoUxhvre5mz1edi5fZHrnJihMSURLVhWyENrJpHlN6TUv5jjW9V3nSQiEtUy6vUCYK6eCb3SaExJRHvni1tY5zWYbSdy0/COrnNERKJe17an4rGWOeu/cZ0SMzSmJGKVFe/gkU1f0LLUS0bWBXTNquM6SUQk6nVvmU3TgGFZ0VrXKTFDY0oi1uuf38xWr4eSbcP43bB2rnNERGJC/VpJNPCnsdxTgg2HXefEBI0piUiB0nye2TaDViU+OrY9m9aNUl0niYjEjPoJLdjt9bB18yzXKTFBY0oi0gdT72SL10No1wB+M7iN6xwRkZjSrMHxAMxe+pHjktigMSURJxz089T6T2hWZmjb6gKy6tVwnSQiElO6tx+B11rmbprtOiUmaExJxPli+j2s8kLSrp5cObC16xwRkZjTNbspWX4Py0s3uk6JCRpTElFsOMwTy1+jccDSqOlltGxQy3WSiEjMSUtOoH6gLis8ZdhQ0HVO1NOYkogy+9uJLPCGqberI1edpJ/gExGpKg2SWlPg9bB+3VTXKVFPY0oiyhMLnqRuMExi3SvJzazjOkdEJGZlNz4BgJnL/+e4JPppTEnEWLzodaaZUjJ3t+SqgZ1d54iIxLS89sNIClu+3zLXdUrU87kOEPnBk3Pup2Y4jD/5Mvq0THedIyIS03KzGpDp97LCs8V1StTTlSmJCOvWTOaT0B5a7WnKpQN6YoxxnSQiEtNSEr3UD9ZnpS9AKFDqOieqaUxJRHh6+t34LOSHJzC0QyPXOSIicaFBSltKPB5Wrv7EdUpU05gS57Zvncc7ZZtpl1+X8088EY9HV6VERKpDq4xBAMxc9rnjkuimMSXOPT/lDkLA3tKzGdU1w3WOiEjc6NlhIDXCYebvmOc6JarpG9DFqfw9a3m1cDkdimoxtN8IEn3a9yIi1aV90zpkliWw0rfddUpUK9dXLmPMcGPMUmPMCmPMLYd4/bnGmHn7f00zxnSp/FSJRa9+eRtFHg8F+aM5s2eW6xwRkbiS4PVQP9yIVb4wgdIC1zlR64hjyhjjBR4GRgAdgLONMR1+crPVwInW2lzgz8DEyg6V2FNavJPnd35L+6IEhh43jhqJulAqIlLdGtXsSMAYlqz80HVK1CrPlalewApr7SprrR94GRh18A2stdOstbv3/3YGkFm5mRKL3vnyD+zyevDvGcKEPtmuc0RE4lLb5kMAmLniC8cl0as8YyoDWH/Q7zfsf9kvuQTQvJXDCgZKeHrzZFqWeuiRewG1ayS4ThIRiUu92vUlLRRm4a5FrlOiVnnG1KF+Tt0e8obGDGTfmLr5F15/uTFmtjFm9vbt+ma3ePbx13ez0Wvw7Tqey/q3dJ0jIhK3WjZMI6MsiZWhXa5TolZ5xtQG4ODvDM4ENv30RsaYXOAJYJS1dueh3pG1dqK1toe1tkeDBg2OpVdigA2FeHLVu2T6La1aX0ajtGTXSSIiccvjMaTThLU+S0mRLnQci/KMqVlAa2NMjjEmETgLePfgGxhjmgFvAudba5dVfqbEkqmzHmC515K2sxtXDGjjOkdEJO41Te1CyBgWLnvfdUpUOuKYstYGgWuAj4HFwKvW2oXGmCuNMVfuv9kdQDrwH2PMd8aY2VVWLNHNWp5Y/AINA2HqZ15FTv2arotEROJe+xYjAJi58kvHJdGpXD+Lbq39APjgJy979KB/vhS4tHLTJBbNnf88cz0Bcne05apzOrrOERER4Lh2eaQvDrN473LXKVFJTzct1erJuY9QOxQmse6v6JRR23WOiIgAmXVTaFJWg5V2j+uUqKQxJdVm+fIP+ZJCcnY344qTurnOERGR/Ywx1PdksD7BULh3/ZHfQH5EY0qqzZPf/J2UcJhA0mUcl1PPdY6IiBykae3uAHy3ZJLjkuijMSXVYuPGb/gouIN2extwyUnHY8yhnr5MRERc6dz6FABmrf3acUn00ZiSavHsV3/CACX2Aga1a+g6R0REfqJX2440CliW5q90nRJ1dLKsVLmdO5bxZvFaOhakMebEoXg8uiolIhJpGqYm08hfk1VJBa5Too6uTEmV+++U2/AbKC49g9O6NHWdIyIiv6CBtzmbfR5271jqOiWqaExJlSoq3MJLexfSqSiZUf1Gk+DV/+VERCJVVnpPAOYsec9xSXTRVzapUq9/eTsFHg9lBadyRo+sI7+BiIg407XdaRhrmb1+uuuUqKIxJVXGX1bAs1un067Yy0nHnUdKotd1koiIHEaPli1pGoAVRWtdp0QVjSmpMpOm/pHtXoPZO5Dzejd3nSMiIkdQOyWBRoE0VppisNZ1TtTQmJIqEQr6eWrdx+SUQtfOl1I7JcF1koiIlEMDXwt2+Dxs3TLXdUrU0JiSKvHZjPtY54Wau3tx6QktXeeIiEg5NWvYB4BZSz5wXBI9NKak0tlwmCeWvUqGP0xO66tpmJbsOklERMopr8PJeK1l7sZZrlOihsaUVLoZcx9nsTdE/V2duGJAW9c5IiJyFLplZ5HhN6wo2eA6JWpoTEmle2L+k9QPhqmfcQ3N02u6zhERkaNQI9FHo2AdVnhLsaGQ65yooDEllWrB4jeZaUpotjuHy0/q4jpHRESOQYOk1uR7PWzYOM11SlTQmJJK9fjsf5EaCpNU92o6NE1znSMiIsegReMTAPhm8UeOS6KDxpRUmlVrv+SL0G7a7GnCpScd5zpHRESOUa+OI0gMW77fqqdHKA+f6wCJHU9Pu5tECzbpUnrl1HOdIyIix6hjZkMy/R5Weja5TokKujIllWLL1vm8V7aJjvl1mDBooOscERGpgESfhwahdFb6AoSDftc5EU9jSirF81PvwAKB8PkMbNvQdY6IiFRQo5S2FHs8rFr9heuUiKcxJRW2d+96XitYTm5BDc4ceCrGGNdJIiJSQa0yBgAwY9nHbkOigMaUVNhLX/4fJR6Dv2wMp3Ru4jpHREQqQe/OQ0kJWxZsn+86JeLpG9ClQopLdvHCzm/pXJLAiH5n4/Nqn4uIxII2jeqSWeZjpXeb65SIp698UiFvfXkHez0GWzCM8XmZrnNERKSSeD2GhrYBq30hAv4i1zkRTWNKjlkgUMLTm76kbYmhf69LSU7wuk4SEZFK1LhmR8o8hqXL9X1Th6MxJcfsw2l/YasXkvb049w+zV3niIhIJWvbfDAAM5Z/5rgksmlMyTEJh0M8ufJdssssublXk5ac4DpJREQqWd8OA6gVCrN41yLXKRFNY0qOyZezHmKVN0zd3d24+IRWrnNERKQKNKtfk0x/EqvCO1ynRDSNKTlq1loeX/Q8jQNhsltdR4PUJNdJIiJSBYwxNLCNWeOzlJXscp0TsTSm5KjNXvAi8z1+Mna34bKBHVzniIhIFcpIyyVoDAuWfuA6JWJpTMlRe2Luw9QNhqnf5Fqy6tVwnSMiIlWoQ4thAHyzUsfK/BKNKTkqS1Z+xDRbSMs9mVw2uIfrHBERqWJ9OvSmbjDMkj3LXKdELI0pOSpPzPg7NcNhatS9mraNU13niIhIFWtcO4UMfwqr7G7XKRFLY0rKbf2mmXwS2E6HPfW5eFA/1zkiIlJNGpgM1vmgKH+T65SIpDEl5fb0V3/CZ8GTdAl5zeu5zhERkWqSWacb1hjmLnnPdUpE0piSctmxaznvFK8htyCVcwcNd50jIiLVKLfNKQDMWj3VcUlk0piScnn+y9sJAoTPZkCbBq5zRESkGh3XtgsNApZlBStcp0QkjSk5ooLCrbyyZwFdCpMYM3AcxhjXSSIiUo3q1kykaaAGqyhwnRKRNKbkiF758jaKPAZTOpIRnRq7zhEREQcaeJuxKcGwd/cq1ykRR2NKDqu0rIDnt06nU7GHYSdchM+r/8uIiMSjZvV6AjBr4buOSyKPvjLKYb371Z/Y5TUkFZzE2LwM1zkiIuJI9/anAjBn3TTHJZFHY0p+UTDo56m1H9O61NK751Uk+byuk0RExJEerdrSxG9ZXrTGdUrE0ZiSX/TJN/9ko9dSe28vzuvbwnWOiIg4VDPJR5NgKqs9RWCt65yIojElh2St5fFlr5DlD9O582+oleRznSQiIo419OWwzedh+7b5rlMiisaUHNJXcx9nuSdIkz0dmXBCG9c5IiISAXIa9gZgxqL3HZdEFo0pOaTH5z9Bw2CY5q1+S3qtJNc5IiISAfI6nobHWr7bMNN1SkTRmJKf+W7Jm8ylhJzdOVw8oLPrHBERiRBds5uRETCsLF3nOiWiaEzJzzw+61+khcLUb3odmXVruM4REZEIkeTz0jhYm1XeUmw47DonYmhMyY+sWDuFKeE9tN/TmEsG9XadIyIiEaZxUit2ez1s1EN9B2hMyY88Me0uksNhatW5itaNUl3niIhIhGnRuB8A0xbrm9B/oDElB2zatoCPyzbRZW8dJgwe5DpHREQi0HGdT8FnLQu2fOs6JWLoyYPkgGem3AFAUuJFdGtW13GNiIhEovZNG5FV5mGlZ5PrlIihK1MCwK6963izYBndClI4c/Ao1zkiIhKhfF4PjcL1WOnzEw4FXOdEBI0pAeDFL2/Hb8AXHs8Jreu7zhERkQjWJKUNRR4Pq9d86TolImhMCUUlu3h55xy6FPkYfdL5GGNcJ4mISARrnTkAgGmLP3YbEiE0poTXp/yBfI8hpXQEwzo2dp0jIiIRrk+n4SSFLYt2fO86JSLoG9DjnD9QyrObJtOxDAadcCVej65KiYjI4bVsVJdmfi+rPFtdp0QEXZmKc+9P/yvbPZBa0I/R3TNd54iISBQwxtAw3ICVvhDBQInrHOc0puJYKBTkyRVv06IszHE9ryfJ53WdJCIiUaJpzQ6UeQxLV3ziOsU5jak49vmch1nrDdNoT3fO7tPSdY6IiESRdtmDAZi2TGNKYypOWWt5fNFzZARCtM+9kZpJ+vY5EREpv+M7n0TNcJglOxe6TnFOYypOfbPgvyw2fprvbsOFJ7R3nSMiIlGmaZ2aZJUlsjq0w3WKcxpTcerxuQ9TPxiieavfUbdmouscERGJMsYYGtGI1Qlh/KV7Xec4pTEVhxau/JiZtoA2e7K4cGB31zkiIhKlmqblEjSG+Us/cJ3ilMZUHHp8xt+pFQrTsPG1NK2T4jpHRESiVOeWwwCYsfwLxyVuaUzFmTWbZvN5YBud96YzYciJrnNERCSK9e1wPLVDYZbuXeI6xSmNqTjz5Fd3kmgtdWpfSauGtVzniIhIFEtPTSbTn8wau9t1ilMaU3Fk664VvFe8hm75tTh3yMmuc0REJAY0Mk1Z67MUF21zneKMxlQcefbL27BAzcTz6JJVx3WOiIjEgKy63Qgbw+yF77pOcUZjKk7sLdzKG3sW0L0wkfFDznadIyIiMaJL61MAmLN6quMSdzSm4sRLU26n2GOoFRpN35bprnNERCRG9OmQR3owzLKC5a5TnNGYigMl/kJe3DqdrkWGk0+6FGOM6yQREYkRtZJ8ZPprsIZ81ynOaEzFgbe++jN7PJBWMoihHZu4zhERkRjTyNuMDQmGvXvWuk5xQmMqxgVCfp5e+yHtS8IM6H89Ho+uSomISOVqVj8PgG8WvuO4xA2NqRj30Tf/ZIvHUr+wN6O6ZbnOERGRGNSj3WkAzF07zXGJGxpTMSxswzy+9BWy/SF69vwdiT7d3SIiUvnyWnegccCyomi16xQn9NU1hk357ilWe4I029uJM/u0cp0jIiIxKjnBS9NALVZ7Cl2nOKExFaOstTw2byJNAiHad7yZGok+10kiIhLDGidks9XnYcf2Ra5Tqp3GVIz6dtk7LKCEVntzOO/Ezq5zREQkxuU07A3A1/MnOS6pfhpTMWrizH9SNxQiO+dG6tRIdJ0jIiIxrlfHkRhrmb9ppuuUaqcxFYOWrpvKtPBuOu5pxAWD+rjOERGROJCbnU1GwLCyZJ3rlGqnMRWDHv/6LmqEwzRpfC2Naye7zhERkTjg83poEkxjlbcYGw67zqlWGlMxZv22BXxStpFue2tz3pAhrnNERCSONE5qyS6vh02bvnWdUq00pmLMk1PuwAvUT7uMFg1quc4REZE40rpJPwC+Xvie45LqpTEVQ3bsXc+kwmX0yE/mrGFjXOeIiEic6Z17Kl5rWbBljuuUaqUnH4ohz315GwGgTsJZdMqo7TpHRETiTNsmTcjyG1aZja5TqpWuTMWIgpJdvLpzDnlFXk4fepHrHBERiUMej6FxuC6rvWXYUMh1TrXRmIoRr0y5kyKPoV7wFHq3qOc6R0RE4lTTlLbkez2sWveV65RqozEVA8qCpbyw6Qtyiy3DBl2DMcZ1koiIxKm2mScC8PXCDxyXVB+NqRjwzrS/stMDDUv6M7hDE9c5IiISx47vcgpJYcvi7d+7Tqk2+gb0KBcMB3ly5du09Yfo1+9GPB5dlRIREXeapdchy+9ltWeL65RqoytTUe7TOf9hkydMRmF3RuZlu84REZE4Z4yhsU1nlS9AKFDmOqdaaExFMWstExc+S5Y/RI+evyfBq7tTRETcy6jZgRKPh8UrPnOdUi301TeKTVv4EsuNn9b5bRjXu53rHBEREQA6ZA8CYMbSjx2XVA+NqSj22Lf/pkEwRIeON5OS6HWdIyIiAsDxXYaQEg6zZNcC1ynVQmMqSi1aO5m5toCOe5tyVv8ernNEREQOaJRWi2b+BFaHtrtOqRYaU1HqyWl/o2Y4THbz66ldI8F1joiIyI80phFrfGECZYWuU6qcxlQU2rJ7FZ+VbaBbfipnDxrkOkdERORnslI74/cYvlvyoeuUKqcxFYWe+/IPWKBJ2vk0rZPiOkdERORnclsOA2BmHPxEn8ZUlCkuK+Ct3XPJK/IyZuh5rnNEREQOqU+n/qSFwizbs9h1SpXTmIoyb3x1N4UeQyOG0ymjtuscERGRQ6pTM4ksfxJrwrtcp1Q5jakoEgqHeGHth7QvCTN04PWuc0RERA6rsacpaxMspcU7XadUKY2pKPL53Ils8obJLO3Oie0au84RERE5rOZ1uhIyhpkLJrlOqVLlGlPGmOHGmKXGmBXGmFsO8fp2xpjpxpgyY8yNlZ8pAE/Nf4YmgRB9et2iA41FRCTidW07AoDZqya7DaliRxxTxhgv8DAwAugAnG2M6fCTm+0CrgXuq/RCAWDeqo9ZYIppl9+c03rq6BgREYl8vdv3pl4wzIrCFa5TqlR5rkz1AlZYa1dZa/3Ay8Cog29grd1mrZ0FBKqgUYAnpt9LrVCYDm1+S3KCjo4REZHIl5LoJTOQwhr2uE6pUuUZUxnA+oN+v2H/y6SabNyxmC8DW+iaX5szB/Z3nSMiIlJujb1ZbPBBQf4G1ylVpjxj6lDfnGOP5YMZYy43xsw2xszevj0+zuupDM98+Uc8QE6Dy6hbM9F1joiISLm1SO+BNYZp895xnVJlyjOmNgBZB/0+E9h0LB/MWjvRWtvDWtujQYMGx/Iu4k5hyW7ezV9A98IEzhh2huscERGRo9Kjw2kAzF33teOSqlOeMTULaG2MyTHGJAJnAe9WbZb84JWpf6LYY8hIOI3s+jVd54iIiByV7q070SgQZlXRKtcpVcZ3pBtYa4PGmGuAjwEv8JS1dqEx5sr9r3/UGNMYmA2kAWFjzPVAB2ttftWlx75gKMBLGz6joz/MyGHXus4RERE5agleDxnBWqxJKHCdUmWOOKYArLUfAB/85GWPHvTPW9j38J9Uov/N+Q9bvZZu/t7kZae7zhERETkmTRKa861vMTt2rKB+/VaucyqdngE9QllreXrh82QEQgw84VaM0ZN0iohIdGrZ4DgAvp73luOSqqExFaG+W/4eSzxltCtsybAuLVzniIiIHLPjcvc9PeX3G79xXFI1yvUwn1S/x7/5J2mhMN063YJXR8eIiEgU69SsJRl+yxrWuU6pEroyFYHWb5vPV6HtdCmox7gTjnOdIyIiUiEej6FpKI3VniKwx/RUlRFNYyoCPTn5TrxA24yrqZmki4ciIhL9mia1ZIfPw8bN37tOqXQaUxFmb9F2PihaSl5hImcNHe06R0REpFK0btIHgK/nT3JcUvk0piLMy1/eSYnHkFNjPI3Skl3niIiIVIrju4zCay3zt8x2nVLp9BhSBAmE/Ly8ZQqdyyxjRv3KdY6IiEiladm4KZl+wxpi78BjXZmKIO/P+Bc7vJBt+9O+aW3XOSIiIpXGGEPTcF1W+0qx4bDrnEqlMRUhrLU8u/QVmvlDDBtwi+scERGRSpdRozV7vR5Wrp3mOqVSaUxFiNlL3mCFN0Db4nb0b5/lOkdERKTStcvoD8C0hR8c4ZbRRWMqQkyc+QB1QmGO73Gbjo4REZGY1K/baSRYy6Ltc12nVCqNqQiwZvMcZrCH3IKGnHZcV9c5IiIiVSKjXj2a+T2sDWxxnVKpNKYiwONf/pHEsKVLy+tI9OkuERGR2NXEprMqwU84FHCdUmn0lduxPYWb+bhkFd0LUzjzpFNc54iIiFSprJrtKfZ4WLj8c9cplUZjyrHnPr+DMo+hTd1zqF0jwXWOiIhIleqYPRCA6Us+clxSeTSmHPIHSnljxwxyi+HM4Ze7zhEREalyJ3Q5meSwZdnOBa5TKo3GlEPvTLuXXV5o5RlEs/o1XeeIiIhUuXqpNWnm97EmvM11SqXRmHLEWsvzK98k2x9i1NDfu84RERGpNk1owBpfiGCgxHVKpdCYcuTr+S+y2hukXWlnuuc0cp0jIiJSbZrX7kyZx/Dtoth48k6NKUeenPMf6gVDDOpzh+sUERGRatWlxRAAZi771HFJ5dCYcmDF+mnM9hTQpagpQ7p3cJ0jIiJSrY7PHUStUJjle5e4TqkUGlMOTJxyF0nhML3a3YjXo6NjREQkvtRMTqR5IJG1dofrlEqhMVXNdu5dx2eBdXQvrMXYAUNc54iIiDjR2DRhbYKltGS365QK05iqZs98fjt+Y+jc+CJSEr2uc0RERJzIrtuVoDF8M+891ykVpjFVjUr9Rby9ew5dizycOfxC1zkiIiLO5LUdAcCcNZPdhlQCjalq9MaUv7DHa2ibPIKGqcmuc0RERJzp1a4PdYJhVuYvc51SYRpT1cRay3/XvkfLsjBnDL/ZdY6IiIhTSYk+mgVSWGv2uE6pMI2pajJ57lOs84VpH+xOm6Z1XeeIiIg418SXwXqfpTB/i+uUCtGYqiZPffc49YMhTjlRT9IpIiIC0CI9j7AxfDXvbdcpFaIxVQ2WrPmC77xF5BY35/gOrVzniIiIRITjOp4GwHdrv3JcUjEaU9Xgsal/ITkc5sSuv8cYPUmniIgIQNdWXWkQDLOqeKXrlArRmKpi23evZHJoM3mFtTmtbz/XOSIiIhHD6zFkBmqy1hS4TqkQjakq9vintxMCuje7nASv/rpFREQO1jShOZsSDDt3rnadcsz01b0KlZTl837BPLoWJ3DWsHNd54iIiESc1o16ATDl+7cclxw7jakq9PIXfybfa+icOpK05ATXOSIiIhGnd+eRACzYOMNxybHzuQ6IVeFwiFc3fkzrYJizxt3oOkdERCQidWjWhiYBy+rwGtcpx0xXpqrIJ7MeZYPP0tH0Jis91XWOiIhIRDLGkBlMZa23yHXKMdOYqiLPLniGRoEQY4f80XWKiIhIRMtIbsE2n4cNmxe4TjkmGlNVYN7yj5jvK6VLWUu65mS6zhEREYlobZv0AWDq92+7DTlGGlNVYOK0e6gRDjO05+2uU0RERCJev66j8FjL4q2zXaccE42pSrZ5xxK+stvoXlSPIT16us4RERGJeNmNssgIGNaUbXCdckw0pirZxE9vwwJ9W/0aj0dHx4iIiJRHRrg2q30l2HDYdcpR05iqRMUlu/moZAndihIZP2is6xwREZGokZXSmj1eDyvWznSdctQ0pirRc5/eQaHH0C39DJITvK5zREREokb7zP4ATFvwnuOSo6cxVUlCoSBvbptM21LLeadc7zpHREQkqvTvOhKftSzZMdd1ylHTmKok709/gM0+yPX1Jz012XWOiIhIVGlUL51mfg9rA5tdpxw1jalK8sLS/9IkEOask+90nSIiIhKVmtp6rPaVEQ4FXaccFY2pSjB70dss9vnpGmhLmyYNXeeIiIhEpWa12lHo9bBg+WTXKUdFY6oSPPHNP6kVCjOy352uU0RERKJWp+YDAZix+CPHJUdHY6qCNmyZxwyzi7yShhzfqbPrHBERkah1QrdTSApblu2a5zrlqPhcB0S7Rz67HQMM6HgDxuhJOkVERI5VnZq1aB7wso5trlOOiq5MVUBh8XY+9a+kW3EKo084xXWOiIhI1GtKA1YnBAgESlynlJvGVAU8+fHtFHsMxzU+F59Xf5UiIiIV1TytI6UeD3MW/s91SrlpARyjYNDPpF1f074Ezj35atc5IiIiMaFbq6EAzFrxieOS8tOYOkZvTbmXrT7onjKIWskJrnNERERiQt/cwdQMh1mxZ5HrlHLTmDoG1lpeXvU6TQNhzj/1Ttc5IiIiMSMlMYnm/kTW2x2uU8pNY+oYTJv3CssSgvQIdyIjvY7rHBERkZjS1NOYNQlhykoKXKeUi8bUMXh69kOkhcKMO+nPrlNERERiTou6uQSMYdq8d12nlIvG1FFauX4mM7176VHahG6t2rjOERERiTnd24wA4NvVk92GlJPG1FGa+MUf8QLDu9/sOkVERCQm9Wrfj9qhMKsKlrpOKReNqaOwt2AjX4TWkldck+HHDXadIyIiEpMSEnw0CySzjt2uU8pFY+ooTPzoNko8hv7NL9HRMSIiIlUow5fB+gRLQcF21ylHpDFVToFgKR/kz6JjiYczh13iOkdERCSmtUzvRsgYpnz3luuUI9KYKqeXP7ubHT5Dr9QRJPm8rnNERERiWu8OpwEwf91XjkuOTGOqHKy1vLF+Eln+MBNOu811joiISMzr0jqP+sEwq4tWuE45Io2pcvhi9rOsTAjR09Od9LRarnNERERinjGGrGAN1nryXacckcZUOTz//aPUCYU5d+jdrlNERETiRmZCMzYmGHbsWuc65bA0po5gyeqvmOMrpKc/izZZzVzniIiIxI3WDXsAMGVuZH8TusbUEUz88s8kWBjdW98rJSIiUp365p4OwMJN0x2XHJ7PdUAk27l7LVPsRvJK0jihy/Guc0REROJKm6y2NA5Y1oRXu045LF2ZOozHPrqVMo9hcJur9CSdIiIi1cwYQ1aoFmu9ha5TDktj6hf4/cV8XPw9nUu8jD3pPNc5IiIicSkzKYetPg8bNi92nfKLNKZ+wXMf38kun6Ff+mi8Hl2VEhERcaF9094ATPn+bbchh6ExdQg2HOadrR/R3G+54NTfu84RERGJWyd0G4uxlsVbZrlO+UUaU4fw4fTHWJNg6Z3Ym1opSa5zRERE4lZmg0wyA7DOv951yi/SmDqElxc9Tb1gmAkj7nKdIiIiEvcywrVZ4y3GhsOuUw5JY+onvl/6CXMTSzgu1IKsho1d54iIiMS9rBqt2OXzsHzdXNcph6Qx9RNPfv03ksKWM0/8o+sUERERATpm9gPg6/nvOC45NI2pg2zevpyvzVZ6lNUlr2131zkiIiICnNj9dHzWsnRHZF6Z0jOgH+TRj2/F7zGc2vE61ykiIiKyX/3a9ckKGNbbja5TDklXpvYrLcvnc/8iupQkcEq/sa5zRERE5CCZth5rfGWEQyHXKT+jMbXfk+/fzh6vh4FNztDRMSIiIhGmWc225Hs9zFs+xXXKz2hMse9JOt/f9Tk5ZZbzRtzoOkdERER+IjdnAAAzFn/otONQNKaAt758gPUJcHyN/iQl6tvIREREIk3/riNJDFuW75rnOuVntByAN1a8SH1PmEtG3e06RURERA6hVo1aNA94WM8W1yk/E/dXpmbMf495iWX0tm2pX6eu6xwRERH5BRmmAWsTAgQDZa5TfiTux9Rz39xLcjjMhME6OkZERCSSZad1oNjjYebCT1yn/Ehcj6m1mxYw3beT48oa0C67g+scEREROYzuLQcBMHv5/xyX/Fhcj6nHPrmNEDAm73euU0REROQI+nQeTo1wmJV7F7pO+ZG4/Qb0gsKdfBlaTjd/Cif1PMV1joiIiBxBclIyzQIJrGe765QfidsrU49/8H/kez0MbX6+6xQREREppyxPI9YmhCkpLXSdckBcjqlQKMD/8r+iVRmcOeQa1zkiIiJSTjl1c/F7DF/Pe891ygFxOaZe/uQ+NiYY+tcejM/ndZ0jIiIi5dSzzXAAvlv1heOS/y8ux9S7616jYTDMpaf+2XWKiIiIHIWe7QeQGgqzumCJ65QD4m5MTZ79KouSAhzv6URqzVquc0REROQoeH1emgeSWM8u1ykHxN2Y+u/cB6kRDnPxsL+4ThEREZFjkOlryvoEy96CHa5TgDgbU0vWzGFmwh56B5qQ3bSl6xwRERE5Bq3TuxM0hqnfves6BYizMfXU53/AAmf3+T/XKSIiInKMenc4GYD566Y4LtknbsbUrr1bmMpqupfVpHfnga5zRERE5Bh1atmTesEwa4pWuE4B4ugZ0Ce+/3sKvR5Oy77EdYqIiIhUgMfrpVkwhfWePa5TgDi5MhXwl/FZ8Szalnk4feBlrnNERESkgrISstiQANt2bnSdEh9j6rmP72ZLguGk9BEYY1zniIiISAW1bdQTawxT5r7lOiU+xtSHm9+lcSDMxaf+wXWKiIiIVILjc0cBsHDTNMclcTCmPvz6WZYmheif1J3kpBTXOSIiIlIJWmV1pFEgzPrS1a5TyjemjDHDjTFLjTErjDG3HOL1xhjz4P7XzzPGdK/81GPz+sJHqRUKc+mIv7pOERERkUqUFarFOk+B64wjjyljjBd4GBgBdADONsZ0+MnNRgCt9/+6HHikkjuPyffLvmZOYgF9w1k0qZ/pOkdEREQqUbPkbDYnGNZtXua0ozxXpnoBK6y1q6y1fuBlYNRPbjMKeM7uMwOoY4xpUsmtR+3ZKX/CABNOvNN1ioiIiFSy9k16AzD1+7eddpRnTGUA6w/6/Yb9Lzva21SrzTvW87V3Az3K0sht3dtlioiIiFSBft1OB2Dxlm+cdpRnTB3quQTsMdwGY8zlxpjZxpjZ27dvL0/fMZu//CtSQzC201VV+nFERETEjcwG2bQt8xC2Yacd5XkG9A1A1kG/zwQ2HcNtsNZOBCYC9OjR42djqzIN7XM2A/LGkJiYVJUfRkRERBx6/fLvXSeU68rULKC1MSbHGJMInAX89Jjmd4EL9v9UX29gr7V2cyW3HjUNKREREalqR7wyZa0NGmOuAT4GvMBT1tqFxpgr97/+UeAD4GRgBVAMXFR1ySIiIiKRo1wHHVtrP2DfYDr4ZY8e9M8WuLpy00REREQiX8w/A7qIiIhIVdKYEhEREakAjSkRERGRCtCYEhEREakAjSkRERGRCtCYEhEREakAjSkRERGRCtCYEhEREakAjSkRERGRCtCYEhEREakAjSkRERGRCtCYEhEREakAjSkRERGRCtCYEhEREakAjSkRERGRCtCYEhEREakAjSkRERGRCtCYEhEREakAjSkRERGRCtCYEhEREakAY61184GN2Q6srYYPVR/YUQ0fR8pP90nk0X0SmXS/RB7dJ5GpOu6X5tbaBod6hbMxVV2MMbOttT1cd8j/p/sk8ug+iUy6XyKP7pPI5Pp+0cN8IiIiIhWgMSUiIiJSAfEwpia6DpCf0X0SeXSfRCbdL5FH90lkcnq/xPz3TImIiIhUpXi4MiUiIiJSZWJiTBljhhtjlhpjVhhjbjnE640x5sH9r59njOnuojPelON+OXf//THPGDPNGNPFRWc8OdJ9ctDtehpjQsaYcdXZF6/Kc78YYwYYY74zxiw0xnxZ3Y3xphyfv2obYyYZY77ff59c5KIznhhjnjLGbDPGLPiF17v7Wm+tjepfgBdYCbQAEoHvgQ4/uc3JwIeAAXoD37jujvVf5bxf+gJ19//zCN0v7u+Tg273OfABMM51d6z/Kue/K3WARUCz/b9v6Lo7ln+V8z65Ffj7/n9uAOwCEl23x/IvoD/QHVjwC6939rU+Fq5M9QJWWGtXWWv9wMvAqJ/cZhTwnN1nBlDHGNOkukPjzBHvF2vtNGvt7v2/nQFkVnNjvCnPvysAvwbeALZVZ1wcK8/9cg7wprV2HYC1VvdN1SrPfWKBVGOMAWqxb0wFqzczvlhrp7Dv7/mXOPtaHwtjKgNYf9DvN+x/2dHeRirX0f6dX8K+/6KQqnPE+8QYkwGcDjxajV3xrjz/rrQB6hpjJhtj5hhjLqi2uvhUnvvk30B7YBMwH7jOWhuunjz5Bc6+1vuq44NUMXOIl/30RxTLcxupXOX+OzfGDGTfmOpXpUVSnvvkfuBma21o339wSzUoz/3iA/KAQUAKMN0YM8Nau6yq4+JUee6TYcB3wElAS+ATY8xUa21+FbfJL3P2tT4WxtQGIOug32ey778UjvY2UrnK9XdujMkFngBGWGt3VlNbvCrPfdIDeHn/kKoPnGyMCVpr366WwvhU3s9hO6y1RUCRMWYK0AXQmKoa5blPLgL+Zvd9s84KY8xqoB0ws3oS5RCcfa2PhYf5ZgGtjTE5xphE4Czg3Z/c5l3ggv3f6d8b2Gut3VzdoXHmiPeLMaYZ8CZwvv4Lu1oc8T6x1uZYa7OttdnA68CvNKSqXHk+h70DnGCM8RljagDHAYuruTOelOc+Wce+K4UYYxoBbYFV1VopP+Xsa33UX5my1gaNMdcAH7PvJzCestYuNMZcuf/1j7Lvp5JOBlYAxez7LwqpQuW8X+4A0oH/7L8SErQ6QLTKlPM+kWpWnvvFWrvYGPMRMA8IA09Yaw/54+FSceX8d+XPwDPGmPnse3jpZmvtDmfRccAY8xIwAKhvjNkA/AFIAPdf6/UM6CIiIiIVEAsP84mIiIg4ozElIiIiUgEaUyIiIiIVoDElIiIiUgEaUyIiIiIVoDElIiIiUgEaUyIiIiIVoDElIiIiUgH/D5XhzWadMbqFAAAAAElFTkSuQmCC\n",
|
|
"text/plain": [
|
|
"<Figure size 720x720 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"filenames": {
|
|
"image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter11_91_4.png"
|
|
},
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"import autograd.numpy as np\n",
|
|
"from autograd import grad, elementwise_grad\n",
|
|
"import autograd.numpy.random as npr\n",
|
|
"from matplotlib import pyplot as plt\n",
|
|
"\n",
|
|
"def sigmoid(z):\n",
|
|
" return 1/(1 + np.exp(-z))\n",
|
|
"\n",
|
|
"def deep_neural_network(deep_params, x):\n",
|
|
" # N_hidden is the number of hidden layers\n",
|
|
" N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
|
|
"\n",
|
|
" # Assumes input x being an one-dimensional array\n",
|
|
" num_values = np.size(x)\n",
|
|
" x = x.reshape(-1, num_values)\n",
|
|
"\n",
|
|
" # Assume that the input layer does nothing to the input x\n",
|
|
" x_input = x\n",
|
|
"\n",
|
|
" # Due to multiple hidden layers, define a variable referencing to the\n",
|
|
" # output of the previous layer:\n",
|
|
" x_prev = x_input\n",
|
|
"\n",
|
|
" ## Hidden layers:\n",
|
|
"\n",
|
|
" for l in range(N_hidden):\n",
|
|
" # From the list of parameters P; find the correct weigths and bias for this layer\n",
|
|
" w_hidden = deep_params[l]\n",
|
|
"\n",
|
|
" # Add a row of ones to include bias\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_values)), x_prev ), axis = 0)\n",
|
|
"\n",
|
|
" z_hidden = np.matmul(w_hidden, x_prev)\n",
|
|
" x_hidden = sigmoid(z_hidden)\n",
|
|
"\n",
|
|
" # Update x_prev such that next layer can use the output from this layer\n",
|
|
" x_prev = x_hidden\n",
|
|
"\n",
|
|
" ## Output layer:\n",
|
|
"\n",
|
|
" # Get the weights and bias for this layer\n",
|
|
" w_output = deep_params[-1]\n",
|
|
"\n",
|
|
" # Include bias:\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_values)), x_prev), axis = 0)\n",
|
|
"\n",
|
|
" z_output = np.matmul(w_output, x_prev)\n",
|
|
" x_output = z_output\n",
|
|
"\n",
|
|
" return x_output\n",
|
|
"\n",
|
|
"def solve_ode_deep_neural_network(x, num_neurons, num_iter, lmb):\n",
|
|
" # num_hidden_neurons is now a list of number of neurons within each hidden layer\n",
|
|
"\n",
|
|
" # Find the number of hidden layers:\n",
|
|
" N_hidden = np.size(num_neurons)\n",
|
|
"\n",
|
|
" ## Set up initial weigths and biases\n",
|
|
"\n",
|
|
" # Initialize the list of parameters:\n",
|
|
" P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
|
|
"\n",
|
|
" P[0] = npr.randn(num_neurons[0], 2 )\n",
|
|
" for l in range(1,N_hidden):\n",
|
|
" P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
|
|
"\n",
|
|
" # For the output layer\n",
|
|
" P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
|
|
"\n",
|
|
" print('Initial cost: %g'%cost_function_deep(P, x))\n",
|
|
"\n",
|
|
" ## Start finding the optimal weigths using gradient descent\n",
|
|
"\n",
|
|
" # Find the Python function that represents the gradient of the cost function\n",
|
|
" # w.r.t the 0-th input argument -- that is the weights and biases in the hidden and output layer\n",
|
|
" cost_function_deep_grad = grad(cost_function_deep,0)\n",
|
|
"\n",
|
|
" # Let the update be done num_iter times\n",
|
|
" for i in range(num_iter):\n",
|
|
" # Evaluate the gradient at the current weights and biases in P.\n",
|
|
" # The cost_grad consist now of N_hidden + 1 arrays; the gradient w.r.t the weights and biases\n",
|
|
" # in the hidden layers and output layers evaluated at x.\n",
|
|
" cost_deep_grad = cost_function_deep_grad(P, x)\n",
|
|
"\n",
|
|
" for l in range(N_hidden+1):\n",
|
|
" P[l] = P[l] - lmb * cost_deep_grad[l]\n",
|
|
"\n",
|
|
" print('Final cost: %g'%cost_function_deep(P, x))\n",
|
|
"\n",
|
|
" return P\n",
|
|
"\n",
|
|
"## Set up the cost function specified for this Poisson equation:\n",
|
|
"\n",
|
|
"# The right side of the ODE\n",
|
|
"def f(x):\n",
|
|
" return (3*x + x**2)*np.exp(x)\n",
|
|
"\n",
|
|
"def cost_function_deep(P, x):\n",
|
|
"\n",
|
|
" # Evaluate the trial function with the current parameters P\n",
|
|
" g_t = g_trial_deep(x,P)\n",
|
|
"\n",
|
|
" # Find the derivative w.r.t x of the trial function\n",
|
|
" d2_g_t = elementwise_grad(elementwise_grad(g_trial_deep,0))(x,P)\n",
|
|
"\n",
|
|
" right_side = f(x)\n",
|
|
"\n",
|
|
" err_sqr = (-d2_g_t - right_side)**2\n",
|
|
" cost_sum = np.sum(err_sqr)\n",
|
|
"\n",
|
|
" return cost_sum/np.size(err_sqr)\n",
|
|
"\n",
|
|
"# The trial solution:\n",
|
|
"def g_trial_deep(x,P):\n",
|
|
" return x*(1-x)*deep_neural_network(P,x)\n",
|
|
"\n",
|
|
"# The analytic solution;\n",
|
|
"def g_analytic(x):\n",
|
|
" return x*(1-x)*np.exp(x)\n",
|
|
"\n",
|
|
"if __name__ == '__main__':\n",
|
|
" npr.seed(4155)\n",
|
|
"\n",
|
|
" ## Decide the vales of arguments to the function to solve\n",
|
|
" Nx = 10\n",
|
|
" x = np.linspace(0,1, Nx)\n",
|
|
"\n",
|
|
" ## Set up the initial parameters\n",
|
|
" num_hidden_neurons = [200,100]\n",
|
|
" num_iter = 1000\n",
|
|
" lmb = 1e-3\n",
|
|
"\n",
|
|
" P = solve_ode_deep_neural_network(x, num_hidden_neurons, num_iter, lmb)\n",
|
|
"\n",
|
|
" g_dnn_ag = g_trial_deep(x,P)\n",
|
|
" g_analytical = g_analytic(x)\n",
|
|
"\n",
|
|
" # Find the maximum absolute difference between the solutons:\n",
|
|
"\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
"\n",
|
|
" plt.title('Performance of neural network solving an ODE compared to the analytical solution')\n",
|
|
" plt.plot(x, g_analytical)\n",
|
|
" plt.plot(x, g_dnn_ag[0,:])\n",
|
|
" plt.legend(['analytical','nn'])\n",
|
|
" plt.xlabel('x')\n",
|
|
" plt.ylabel('g(x)')\n",
|
|
"\n",
|
|
" ## Perform the computation using the numerical scheme\n",
|
|
"\n",
|
|
" dx = 1/(Nx - 1)\n",
|
|
"\n",
|
|
" # Set up the matrix A\n",
|
|
" A = np.zeros((Nx-2,Nx-2))\n",
|
|
"\n",
|
|
" A[0,0] = 2\n",
|
|
" A[0,1] = -1\n",
|
|
"\n",
|
|
" for i in range(1,Nx-3):\n",
|
|
" A[i,i-1] = -1\n",
|
|
" A[i,i] = 2\n",
|
|
" A[i,i+1] = -1\n",
|
|
"\n",
|
|
" A[Nx - 3, Nx - 4] = -1\n",
|
|
" A[Nx - 3, Nx - 3] = 2\n",
|
|
"\n",
|
|
" # Set up the vector f\n",
|
|
" f_vec = dx**2 * f(x[1:-1])\n",
|
|
"\n",
|
|
" # Solve the equation\n",
|
|
" g_res = np.linalg.solve(A,f_vec)\n",
|
|
"\n",
|
|
" g_vec = np.zeros(Nx)\n",
|
|
" g_vec[1:-1] = g_res\n",
|
|
"\n",
|
|
" # Print the differences between each method\n",
|
|
" max_diff1 = np.max(np.abs(g_dnn_ag - g_analytical))\n",
|
|
" max_diff2 = np.max(np.abs(g_vec - g_analytical))\n",
|
|
" print(\"The max absolute difference between the analytical solution and DNN Autograd: %g\"%max_diff1)\n",
|
|
" print(\"The max absolute difference between the analytical solution and numerical scheme: %g\"%max_diff2)\n",
|
|
"\n",
|
|
" # Plot the results\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
"\n",
|
|
" plt.plot(x,g_vec)\n",
|
|
" plt.plot(x,g_analytical)\n",
|
|
" plt.plot(x,g_dnn_ag[0,:])\n",
|
|
"\n",
|
|
" plt.legend(['numerical scheme','analytical','dnn'])\n",
|
|
" plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6f02e86d",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"## Partial Differential Equations\n",
|
|
"\n",
|
|
"A partial differential equation (PDE) has a solution here the function\n",
|
|
"is defined by multiple variables. The equation may involve all kinds\n",
|
|
"of combinations of which variables the function is differentiated with\n",
|
|
"respect to.\n",
|
|
"\n",
|
|
"In general, a partial differential equation for a function $g(x_1,\\dots,x_N)$ with $N$ variables may be expressed as"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ad5c63e8",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"PDE\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation} \\label{PDE} \\tag{17}\n",
|
|
" f\\left(x_1, \\, \\dots \\, , x_N, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1}, \\dots , \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_N}, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(x_1,\\dots,x_N) }{\\partial x_N^n} \\right) = 0\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c7b98797",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"where $f$ is an expression involving all kinds of possible mixed derivatives of $g(x_1,\\dots,x_N)$ up to an order $n$. In order for the solution to be unique, some additional conditions must also be given."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "52f9394e",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"### Type of problem\n",
|
|
"\n",
|
|
"The problem our network must solve for, is similar to the ODE case.\n",
|
|
"We must have a trial solution $g_t$ at hand.\n",
|
|
"\n",
|
|
"For instance, the trial solution could be expressed as"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4d489854",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{align*}\n",
|
|
" g_t(x_1,\\dots,x_N) = h_1(x_1,\\dots,x_N) + h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))\n",
|
|
"\\end{align*}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "82311538",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"where $h_1(x_1,\\dots,x_N)$ is a function that ensures $g_t(x_1,\\dots,x_N)$ satisfies some given conditions.\n",
|
|
"The neural network $N(x_1,\\dots,x_N,P)$ has weights and biases described by $P$ and $h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))$ is an expression using the output from the neural network in some way.\n",
|
|
"\n",
|
|
"The role of the function $h_2(x_1,\\dots,x_N,N(x_1,\\dots,x_N,P))$, is to ensure that the output of $N(x_1,\\dots,x_N,P)$ is zero when $g_t(x_1,\\dots,x_N)$ is evaluated at the values of $x_1,\\dots,x_N$ where the given conditions must be satisfied. The function $h_1(x_1,\\dots,x_N)$ should alone make $g_t(x_1,\\dots,x_N)$ satisfy the conditions."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9d134db4",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"### Network requirements\n",
|
|
"\n",
|
|
"The network tries then the minimize the cost function following the\n",
|
|
"same ideas as described for the ODE case, but now with more than one\n",
|
|
"variables to consider. The concept still remains the same; find a set\n",
|
|
"of parameters $P$ such that the expression $f$ in ([17](#PDE)) is as\n",
|
|
"close to zero as possible.\n",
|
|
"\n",
|
|
"As for the ODE case, the cost function is the mean squared error that\n",
|
|
"the network must try to minimize. The cost function for the network to\n",
|
|
"minimize is"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5a463498",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"C\\left(x_1, \\dots, x_N, P\\right) = \\left( f\\left(x_1, \\, \\dots \\, , x_N, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1}, \\dots , \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_N}, \\frac{\\partial g(x_1,\\dots,x_N) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(x_1,\\dots,x_N) }{\\partial x_N^n} \\right) \\right)^2\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c40d8997",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"If we let $\\boldsymbol{x} = \\big( x_1, \\dots, x_N \\big)$ be an array containing the values for $x_1, \\dots, x_N$ respectively, the cost function can be reformulated into the following:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cc033de6",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"C\\left(\\boldsymbol{x}, P\\right) = f\\left( \\left( \\boldsymbol{x}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}) }{\\partial x_N^n} \\right) \\right)^2\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ae18b4f4",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"If we also have $M$ different sets of values for $x_1, \\dots, x_N$, that is $\\boldsymbol{x}_i = \\big(x_1^{(i)}, \\dots, x_N^{(i)}\\big)$ for $i = 1,\\dots,M$ being the rows in matrix $X$, the cost function can be generalized into"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "44ca21bd",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"C\\left(X, P \\right) = \\sum_{i=1}^M f\\left( \\left( \\boldsymbol{x}_i, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1}, \\dots , \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_N}, \\frac{\\partial g(\\boldsymbol{x}_i) }{\\partial x_1\\partial x_2}, \\, \\dots \\, , \\frac{\\partial^n g(\\boldsymbol{x}_i) }{\\partial x_N^n} \\right) \\right)^2.\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fe94451e",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"## Example: The diffusion equation\n",
|
|
"\n",
|
|
"In one spatial dimension, the equation reads"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "30a42273",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\frac{\\partial g(x,t)}{\\partial t} = \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "fd4077e7",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"where a possible choice of conditions are"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "dbbf2e4b",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{align*}\n",
|
|
"g(0,t) &= 0 ,\\qquad t \\geq 0 \\\\\n",
|
|
"g(1,t) &= 0, \\qquad t \\geq 0 \\\\\n",
|
|
"g(x,0) &= u(x),\\qquad x\\in [0,1]\n",
|
|
"\\end{align*}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "83653db9",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"with $u(x)$ being some given function.\n",
|
|
"\n",
|
|
"For this case, we want to find $g(x,t)$ such that"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d2c2ef4f",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"diffonedim\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation}\n",
|
|
" \\frac{\\partial g(x,t)}{\\partial t} = \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n",
|
|
"\\end{equation} \\label{diffonedim} \\tag{18}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8fb27bdb",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"and"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "dc9297ab",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{align*}\n",
|
|
"g(0,t) &= 0 ,\\qquad t \\geq 0 \\\\\n",
|
|
"g(1,t) &= 0, \\qquad t \\geq 0 \\\\\n",
|
|
"g(x,0) &= u(x),\\qquad x\\in [0,1]\n",
|
|
"\\end{align*}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ab90784f",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"with $u(x) = \\sin(\\pi x)$.\n",
|
|
"\n",
|
|
"First, let us set up the deep neural network.\n",
|
|
"The deep neural network will follow the same structure as discussed in the examples solving the ODEs.\n",
|
|
"First, we will look into how Autograd could be used in a network tailored to solve for bivariate functions.\n",
|
|
"\n",
|
|
"The only change to do here, is to extend our network such that\n",
|
|
"functions of multiple parameters are correctly handled. In this case\n",
|
|
"we have two variables in our function to solve for, that is time $t$\n",
|
|
"and position $x$. The variables will be represented by a\n",
|
|
"one-dimensional array in the program. The program will evaluate the\n",
|
|
"network at each possible pair $(x,t)$, given an array for the desired\n",
|
|
"$x$-values and $t$-values to approximate the solution at."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"id": "b07e858c",
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"editable": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def sigmoid(z):\n",
|
|
" return 1/(1 + np.exp(-z))\n",
|
|
"\n",
|
|
"def deep_neural_network(deep_params, x):\n",
|
|
" # x is now a point and a 1D numpy array; make it a column vector\n",
|
|
" num_coordinates = np.size(x,0)\n",
|
|
" x = x.reshape(num_coordinates,-1)\n",
|
|
"\n",
|
|
" num_points = np.size(x,1)\n",
|
|
"\n",
|
|
" # N_hidden is the number of hidden layers\n",
|
|
" N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
|
|
"\n",
|
|
" # Assume that the input layer does nothing to the input x\n",
|
|
" x_input = x\n",
|
|
" x_prev = x_input\n",
|
|
"\n",
|
|
" ## Hidden layers:\n",
|
|
"\n",
|
|
" for l in range(N_hidden):\n",
|
|
" # From the list of parameters P; find the correct weigths and bias for this layer\n",
|
|
" w_hidden = deep_params[l]\n",
|
|
"\n",
|
|
" # Add a row of ones to include bias\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)\n",
|
|
"\n",
|
|
" z_hidden = np.matmul(w_hidden, x_prev)\n",
|
|
" x_hidden = sigmoid(z_hidden)\n",
|
|
"\n",
|
|
" # Update x_prev such that next layer can use the output from this layer\n",
|
|
" x_prev = x_hidden\n",
|
|
"\n",
|
|
" ## Output layer:\n",
|
|
"\n",
|
|
" # Get the weights and bias for this layer\n",
|
|
" w_output = deep_params[-1]\n",
|
|
"\n",
|
|
" # Include bias:\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)\n",
|
|
"\n",
|
|
" z_output = np.matmul(w_output, x_prev)\n",
|
|
" x_output = z_output\n",
|
|
"\n",
|
|
" return x_output[0][0]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9a167ec1",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"The cost function must then iterate through the given arrays\n",
|
|
"containing values for $x$ and $t$, defines a point $(x,t)$ the deep\n",
|
|
"neural network and the trial solution is evaluated at, and then finds\n",
|
|
"the Jacobian of the trial solution.\n",
|
|
"\n",
|
|
"A possible trial solution for this PDE is\n",
|
|
"\n",
|
|
"$$\n",
|
|
"g_t(x,t) = h_1(x,t) + x(1-x)tN(x,t,P)\n",
|
|
"$$\n",
|
|
"\n",
|
|
"with $A(x,t)$ being a function ensuring that $g_t(x,t)$ satisfies our given conditions, and $N(x,t,P)$ being the output from the deep neural network using weights and biases for each layer from $P$.\n",
|
|
"\n",
|
|
"To fulfill the conditions, $A(x,t)$ could be:\n",
|
|
"\n",
|
|
"$$\n",
|
|
"h_1(x,t) = (1-t)\\Big(u(x) - \\big((1-x)u(0) + x u(1)\\big)\\Big) = (1-t)u(x) = (1-t)\\sin(\\pi x)\n",
|
|
"$$\n",
|
|
"since $(0) = u(1) = 0$ and $u(x) = \\sin(\\pi x)$.\n",
|
|
"\n",
|
|
"The Jacobian is used because the program must find the derivative of\n",
|
|
"the trial solution with respect to $x$ and $t$.\n",
|
|
"\n",
|
|
"This gives the necessity of computing the Jacobian matrix, as we want\n",
|
|
"to evaluate the gradient with respect to $x$ and $t$ (note that the\n",
|
|
"Jacobian of a scalar-valued multivariate function is simply its\n",
|
|
"gradient).\n",
|
|
"\n",
|
|
"In Autograd, the differentiation is by default done with respect to\n",
|
|
"the first input argument of your Python function. Since the points is\n",
|
|
"an array representing $x$ and $t$, the Jacobian is calculated using\n",
|
|
"the values of $x$ and $t$.\n",
|
|
"\n",
|
|
"To find the second derivative with respect to $x$ and $t$, the\n",
|
|
"Jacobian can be found for the second time. The result is a Hessian\n",
|
|
"matrix, which is the matrix containing all the possible second order\n",
|
|
"mixed derivatives of $g(x,t)$."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"id": "921e5969",
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"editable": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Set up the trial function:\n",
|
|
"def u(x):\n",
|
|
" return np.sin(np.pi*x)\n",
|
|
"\n",
|
|
"def g_trial(point,P):\n",
|
|
" x,t = point\n",
|
|
" return (1-t)*u(x) + x*(1-x)*t*deep_neural_network(P,point)\n",
|
|
"\n",
|
|
"# The right side of the ODE:\n",
|
|
"def f(point):\n",
|
|
" return 0.\n",
|
|
"\n",
|
|
"# The cost function:\n",
|
|
"def cost_function(P, x, t):\n",
|
|
" cost_sum = 0\n",
|
|
"\n",
|
|
" g_t_jacobian_func = jacobian(g_trial)\n",
|
|
" g_t_hessian_func = hessian(g_trial)\n",
|
|
"\n",
|
|
" for x_ in x:\n",
|
|
" for t_ in t:\n",
|
|
" point = np.array([x_,t_])\n",
|
|
"\n",
|
|
" g_t = g_trial(point,P)\n",
|
|
" g_t_jacobian = g_t_jacobian_func(point,P)\n",
|
|
" g_t_hessian = g_t_hessian_func(point,P)\n",
|
|
"\n",
|
|
" g_t_dt = g_t_jacobian[1]\n",
|
|
" g_t_d2x = g_t_hessian[0][0]\n",
|
|
"\n",
|
|
" func = f(point)\n",
|
|
"\n",
|
|
" err_sqr = ( (g_t_dt - g_t_d2x) - func)**2\n",
|
|
" cost_sum += err_sqr\n",
|
|
"\n",
|
|
" return cost_sum"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "0bc00b69",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"### Setting up the network using Autograd; The full program\n",
|
|
"\n",
|
|
"Having set up the network, along with the trial solution and cost function, we can now see how the deep neural network performs by comparing the results to the analytical solution.\n",
|
|
"\n",
|
|
"The analytical solution of our problem is\n",
|
|
"\n",
|
|
"$$\n",
|
|
"g(x,t) = \\exp(-\\pi^2 t)\\sin(\\pi x)\n",
|
|
"$$\n",
|
|
"\n",
|
|
"A possible way to implement a neural network solving the PDE, is given below.\n",
|
|
"Be aware, though, that it is fairly slow for the parameters used.\n",
|
|
"A better result is possible, but requires more iterations, and thus longer time to complete.\n",
|
|
"\n",
|
|
"Indeed, the program below is not optimal in its implementation, but rather serves as an example on how to implement and use a neural network to solve a PDE.\n",
|
|
"Using TensorFlow results in a much better execution time. Try it!"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"id": "20734418",
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"editable": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/numpy/core/fromnumeric.py:3208: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\n",
|
|
" return asarray(a).size\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Initial cost: 41.05505310046363\n"
|
|
]
|
|
},
|
|
{
|
|
"ename": "KeyboardInterrupt",
|
|
"evalue": "",
|
|
"output_type": "error",
|
|
"traceback": [
|
|
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
|
"\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)",
|
|
"\u001b[0;32m<ipython-input-9-fdf8a5d7c717>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 141\u001b[0m \u001b[0mlmb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0.01\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 142\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 143\u001b[0;31m \u001b[0mP\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msolve_pde_deep_neural_network\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mt\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_hidden_neurons\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_iter\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlmb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 144\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 145\u001b[0m \u001b[0;31m## Store the results\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m<ipython-input-9-fdf8a5d7c717>\u001b[0m in \u001b[0;36msolve_pde_deep_neural_network\u001b[0;34m(x, t, num_neurons, num_iter, lmb)\u001b[0m\n\u001b[1;32m 118\u001b[0m \u001b[0;31m# Let the update be done num_iter times\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 119\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnum_iter\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 120\u001b[0;31m \u001b[0mcost_grad\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcost_function_grad\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m,\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 121\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 122\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mN_hidden\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/wrap_util.py\u001b[0m in \u001b[0;36mnary_f\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 18\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 19\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtuple\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0margnum\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 20\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0munary_operator\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0munary_f\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0mnary_op_args\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mnary_op_kwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 21\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mnary_f\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 22\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mnary_operator\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/differential_operators.py\u001b[0m in \u001b[0;36mgrad\u001b[0;34m(fun, x)\u001b[0m\n\u001b[1;32m 23\u001b[0m \u001b[0marguments\u001b[0m \u001b[0;32mas\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m`\u001b[0m\u001b[0mfun\u001b[0m\u001b[0;31m`\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbut\u001b[0m \u001b[0mreturns\u001b[0m \u001b[0mthe\u001b[0m \u001b[0mgradient\u001b[0m \u001b[0minstead\u001b[0m\u001b[0;34m.\u001b[0m \u001b[0mThe\u001b[0m \u001b[0mfunction\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m`\u001b[0m\u001b[0mfun\u001b[0m\u001b[0;31m`\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 24\u001b[0m should be scalar-valued. The gradient has the same type as the argument.\"\"\"\n\u001b[0;32m---> 25\u001b[0;31m \u001b[0mvjp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mans\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_make_vjp\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfun\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 26\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mvspace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mans\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msize\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 27\u001b[0m raise TypeError(\"Grad only applies to real scalar-output functions. \"\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py\u001b[0m in \u001b[0;36mmake_vjp\u001b[0;34m(fun, x)\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mmake_vjp\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfun\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 9\u001b[0m \u001b[0mstart_node\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mVJPNode\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnew_root\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 10\u001b[0;31m \u001b[0mend_value\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend_node\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtrace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstart_node\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfun\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 11\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mend_node\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 12\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mvjp\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mvspace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mzeros\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/tracer.py\u001b[0m in \u001b[0;36mtrace\u001b[0;34m(start_node, fun, x)\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[0;32mwith\u001b[0m \u001b[0mtrace_stack\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnew_trace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 9\u001b[0m \u001b[0mstart_box\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnew_box\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstart_node\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 10\u001b[0;31m \u001b[0mend_box\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstart_box\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 11\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0misbox\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mend_box\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mend_box\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_trace\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0mstart_box\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_trace\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 12\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mend_box\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_value\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend_box\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_node\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/wrap_util.py\u001b[0m in \u001b[0;36munary_f\u001b[0;34m(x)\u001b[0m\n\u001b[1;32m 13\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 14\u001b[0m \u001b[0msubargs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msubvals\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mzip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margnum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 15\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mfun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0msubargs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 16\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margnum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 17\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0margnum\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m<ipython-input-9-fdf8a5d7c717>\u001b[0m in \u001b[0;36mcost_function\u001b[0;34m(P, x, t)\u001b[0m\n\u001b[1;32m 78\u001b[0m \u001b[0mg_t\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mg_trial\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpoint\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 79\u001b[0m \u001b[0mg_t_jacobian\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mg_t_jacobian_func\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpoint\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 80\u001b[0;31m \u001b[0mg_t_hessian\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mg_t_hessian_func\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpoint\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mP\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 81\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 82\u001b[0m \u001b[0mg_t_dt\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mg_t_jacobian\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/wrap_util.py\u001b[0m in \u001b[0;36mnary_f\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 18\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 19\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtuple\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0margnum\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 20\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0munary_operator\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0munary_f\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0mnary_op_args\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mnary_op_kwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 21\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mnary_f\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 22\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mnary_operator\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/differential_operators.py\u001b[0m in \u001b[0;36mhessian\u001b[0;34m(fun, x)\u001b[0m\n\u001b[1;32m 76\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mhessian\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfun\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 77\u001b[0m \u001b[0;34m\"Returns a function that computes the exact Hessian.\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 78\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mjacobian\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mjacobian\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfun\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 79\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 80\u001b[0m \u001b[0;34m@\u001b[0m\u001b[0munary_to_nary\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/wrap_util.py\u001b[0m in \u001b[0;36mnary_f\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 18\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 19\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtuple\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0margnum\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 20\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0munary_operator\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0munary_f\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0mnary_op_args\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mnary_op_kwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 21\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mnary_f\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 22\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mnary_operator\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/differential_operators.py\u001b[0m in \u001b[0;36mjacobian\u001b[0;34m(fun, x)\u001b[0m\n\u001b[1;32m 59\u001b[0m \u001b[0mjacobian_shape\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mans_vspace\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mvspace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 60\u001b[0m \u001b[0mgrads\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmap\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvjp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mans_vspace\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstandard_basis\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 61\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstack\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgrads\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mjacobian_shape\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 62\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 63\u001b[0m \u001b[0;34m@\u001b[0m\u001b[0munary_to_nary\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_wrapper.py\u001b[0m in \u001b[0;36mstack\u001b[0;34m(arrays, axis)\u001b[0m\n\u001b[1;32m 86\u001b[0m \u001b[0;31m# primitives defined in this file\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 87\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 88\u001b[0;31m \u001b[0marrays\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0marray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marr\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0marr\u001b[0m \u001b[0;32min\u001b[0m \u001b[0marrays\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 89\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0marrays\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 90\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'need at least one array to stack'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_wrapper.py\u001b[0m in \u001b[0;36m<listcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m 86\u001b[0m \u001b[0;31m# primitives defined in this file\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 87\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 88\u001b[0;31m \u001b[0marrays\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0marray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marr\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0marr\u001b[0m \u001b[0;32min\u001b[0m \u001b[0marrays\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 89\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0marrays\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 90\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'need at least one array to stack'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py\u001b[0m in \u001b[0;36mvjp\u001b[0;34m(g)\u001b[0m\n\u001b[1;32m 12\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mvjp\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mvspace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mzeros\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 13\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 14\u001b[0;31m \u001b[0;32mdef\u001b[0m \u001b[0mvjp\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mbackward_pass\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mg\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend_node\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 15\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mvjp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend_value\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 16\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py\u001b[0m in \u001b[0;36mbackward_pass\u001b[0;34m(g, end_node)\u001b[0m\n\u001b[1;32m 19\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mnode\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtoposort\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mend_node\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 20\u001b[0m \u001b[0moutgrad\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0moutgrads\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnode\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 21\u001b[0;31m \u001b[0mingrads\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnode\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvjp\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutgrad\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 22\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mparent\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mingrad\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mzip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnode\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mparents\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mingrads\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 23\u001b[0m \u001b[0moutgrads\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mparent\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0madd_outgrads\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0moutgrads\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mparent\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mingrad\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/core.py\u001b[0m in \u001b[0;36m<lambda>\u001b[0;34m(g)\u001b[0m\n\u001b[1;32m 65\u001b[0m \"VJP of {} wrt argnum 0 not defined\".format(fun.__name__))\n\u001b[1;32m 66\u001b[0m \u001b[0mvjp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mvjpfun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mans\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 67\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mg\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mvjp\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 68\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mL\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 69\u001b[0m \u001b[0margnum_0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margnum_1\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0margnums\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py\u001b[0m in \u001b[0;36m<lambda>\u001b[0;34m(g)\u001b[0m\n\u001b[1;32m 421\u001b[0m \u001b[0mA_ndim\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0manp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mndim\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mA\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 422\u001b[0m \u001b[0mB_meta\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0manp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmetadata\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mB\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 423\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mg\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mmatmul_adjoint_1\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mA\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mg\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mA_ndim\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mB_meta\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 424\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 425\u001b[0m \u001b[0mdefvjp\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0manp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmatmul\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmatmul_vjp_0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmatmul_vjp_1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/numpy/numpy_vjps.py\u001b[0m in \u001b[0;36mmatmul_adjoint_1\u001b[0;34m(A, G, A_ndim, B_meta)\u001b[0m\n\u001b[1;32m 408\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;31m# We need to swap the last two axes of A\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 409\u001b[0m \u001b[0mA\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0manp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mswapaxes\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mA\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mA_ndim\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mA_ndim\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 410\u001b[0;31m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0manp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmatmul\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mA\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mG\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 411\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mB_is_vec\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 412\u001b[0m \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0manp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msqueeze\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0manp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mndim\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mG\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/tracer.py\u001b[0m in \u001b[0;36mf_wrapped\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 42\u001b[0m \u001b[0mparents\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtuple\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbox\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_node\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0m_\u001b[0m \u001b[0;34m,\u001b[0m \u001b[0mbox\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mboxed_args\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 43\u001b[0m \u001b[0margnums\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtuple\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margnum\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0margnum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0m_\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mboxed_args\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 44\u001b[0;31m \u001b[0mans\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mf_wrapped\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margvals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 45\u001b[0m \u001b[0mnode\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnode_constructor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mans\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mf_wrapped\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margvals\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margnums\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparents\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 46\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mnew_box\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mans\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtrace\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnode\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/tracer.py\u001b[0m in \u001b[0;36mf_wrapped\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 35\u001b[0m \u001b[0;34m@\u001b[0m\u001b[0mwraps\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mf_raw\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 36\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mf_wrapped\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 37\u001b[0;31m \u001b[0mboxed_args\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtrace\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnode_constructor\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfind_top_boxed_args\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 38\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mboxed_args\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 39\u001b[0m \u001b[0margvals\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msubvals\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margnum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbox\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_value\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0margnum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbox\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mboxed_args\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/miniforge3/envs/myenv/lib/python3.9/site-packages/autograd/tracer.py\u001b[0m in \u001b[0;36mfind_top_boxed_args\u001b[0;34m(args)\u001b[0m\n\u001b[1;32m 68\u001b[0m \u001b[0mtop_node_type\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 69\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0margnum\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0marg\u001b[0m \u001b[0;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 70\u001b[0;31m \u001b[0;32mif\u001b[0m \u001b[0misbox\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 71\u001b[0m \u001b[0mtrace\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marg\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_trace\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 72\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mtrace\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0mtop_trace\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;31mKeyboardInterrupt\u001b[0m: "
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import autograd.numpy as np\n",
|
|
"from autograd import jacobian,hessian,grad\n",
|
|
"import autograd.numpy.random as npr\n",
|
|
"from matplotlib import cm\n",
|
|
"from matplotlib import pyplot as plt\n",
|
|
"from mpl_toolkits.mplot3d import axes3d\n",
|
|
"\n",
|
|
"## Set up the network\n",
|
|
"\n",
|
|
"def sigmoid(z):\n",
|
|
" return 1/(1 + np.exp(-z))\n",
|
|
"\n",
|
|
"def deep_neural_network(deep_params, x):\n",
|
|
" # x is now a point and a 1D numpy array; make it a column vector\n",
|
|
" num_coordinates = np.size(x,0)\n",
|
|
" x = x.reshape(num_coordinates,-1)\n",
|
|
"\n",
|
|
" num_points = np.size(x,1)\n",
|
|
"\n",
|
|
" # N_hidden is the number of hidden layers\n",
|
|
" N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
|
|
"\n",
|
|
" # Assume that the input layer does nothing to the input x\n",
|
|
" x_input = x\n",
|
|
" x_prev = x_input\n",
|
|
"\n",
|
|
" ## Hidden layers:\n",
|
|
"\n",
|
|
" for l in range(N_hidden):\n",
|
|
" # From the list of parameters P; find the correct weigths and bias for this layer\n",
|
|
" w_hidden = deep_params[l]\n",
|
|
"\n",
|
|
" # Add a row of ones to include bias\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)\n",
|
|
"\n",
|
|
" z_hidden = np.matmul(w_hidden, x_prev)\n",
|
|
" x_hidden = sigmoid(z_hidden)\n",
|
|
"\n",
|
|
" # Update x_prev such that next layer can use the output from this layer\n",
|
|
" x_prev = x_hidden\n",
|
|
"\n",
|
|
" ## Output layer:\n",
|
|
"\n",
|
|
" # Get the weights and bias for this layer\n",
|
|
" w_output = deep_params[-1]\n",
|
|
"\n",
|
|
" # Include bias:\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)\n",
|
|
"\n",
|
|
" z_output = np.matmul(w_output, x_prev)\n",
|
|
" x_output = z_output\n",
|
|
"\n",
|
|
" return x_output[0][0]\n",
|
|
"\n",
|
|
"## Define the trial solution and cost function\n",
|
|
"def u(x):\n",
|
|
" return np.sin(np.pi*x)\n",
|
|
"\n",
|
|
"def g_trial(point,P):\n",
|
|
" x,t = point\n",
|
|
" return (1-t)*u(x) + x*(1-x)*t*deep_neural_network(P,point)\n",
|
|
"\n",
|
|
"# The right side of the ODE:\n",
|
|
"def f(point):\n",
|
|
" return 0.\n",
|
|
"\n",
|
|
"# The cost function:\n",
|
|
"def cost_function(P, x, t):\n",
|
|
" cost_sum = 0\n",
|
|
"\n",
|
|
" g_t_jacobian_func = jacobian(g_trial)\n",
|
|
" g_t_hessian_func = hessian(g_trial)\n",
|
|
"\n",
|
|
" for x_ in x:\n",
|
|
" for t_ in t:\n",
|
|
" point = np.array([x_,t_])\n",
|
|
"\n",
|
|
" g_t = g_trial(point,P)\n",
|
|
" g_t_jacobian = g_t_jacobian_func(point,P)\n",
|
|
" g_t_hessian = g_t_hessian_func(point,P)\n",
|
|
"\n",
|
|
" g_t_dt = g_t_jacobian[1]\n",
|
|
" g_t_d2x = g_t_hessian[0][0]\n",
|
|
"\n",
|
|
" func = f(point)\n",
|
|
"\n",
|
|
" err_sqr = ( (g_t_dt - g_t_d2x) - func)**2\n",
|
|
" cost_sum += err_sqr\n",
|
|
"\n",
|
|
" return cost_sum /( np.size(x)*np.size(t) )\n",
|
|
"\n",
|
|
"## For comparison, define the analytical solution\n",
|
|
"def g_analytic(point):\n",
|
|
" x,t = point\n",
|
|
" return np.exp(-np.pi**2*t)*np.sin(np.pi*x)\n",
|
|
"\n",
|
|
"## Set up a function for training the network to solve for the equation\n",
|
|
"def solve_pde_deep_neural_network(x,t, num_neurons, num_iter, lmb):\n",
|
|
" ## Set up initial weigths and biases\n",
|
|
" N_hidden = np.size(num_neurons)\n",
|
|
"\n",
|
|
" ## Set up initial weigths and biases\n",
|
|
"\n",
|
|
" # Initialize the list of parameters:\n",
|
|
" P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
|
|
"\n",
|
|
" P[0] = npr.randn(num_neurons[0], 2 + 1 ) # 2 since we have two points, +1 to include bias\n",
|
|
" for l in range(1,N_hidden):\n",
|
|
" P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
|
|
"\n",
|
|
" # For the output layer\n",
|
|
" P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
|
|
"\n",
|
|
" print('Initial cost: ',cost_function(P, x, t))\n",
|
|
"\n",
|
|
" cost_function_grad = grad(cost_function,0)\n",
|
|
"\n",
|
|
" # Let the update be done num_iter times\n",
|
|
" for i in range(num_iter):\n",
|
|
" cost_grad = cost_function_grad(P, x , t)\n",
|
|
"\n",
|
|
" for l in range(N_hidden+1):\n",
|
|
" P[l] = P[l] - lmb * cost_grad[l]\n",
|
|
"\n",
|
|
" print('Final cost: ',cost_function(P, x, t))\n",
|
|
"\n",
|
|
" return P\n",
|
|
"\n",
|
|
"if __name__ == '__main__':\n",
|
|
" ### Use the neural network:\n",
|
|
" npr.seed(15)\n",
|
|
"\n",
|
|
" ## Decide the vales of arguments to the function to solve\n",
|
|
" Nx = 10; Nt = 10\n",
|
|
" x = np.linspace(0, 1, Nx)\n",
|
|
" t = np.linspace(0,1,Nt)\n",
|
|
"\n",
|
|
" ## Set up the parameters for the network\n",
|
|
" num_hidden_neurons = [100, 25]\n",
|
|
" num_iter = 250\n",
|
|
" lmb = 0.01\n",
|
|
"\n",
|
|
" P = solve_pde_deep_neural_network(x,t, num_hidden_neurons, num_iter, lmb)\n",
|
|
"\n",
|
|
" ## Store the results\n",
|
|
" g_dnn_ag = np.zeros((Nx, Nt))\n",
|
|
" G_analytical = np.zeros((Nx, Nt))\n",
|
|
" for i,x_ in enumerate(x):\n",
|
|
" for j, t_ in enumerate(t):\n",
|
|
" point = np.array([x_, t_])\n",
|
|
" g_dnn_ag[i,j] = g_trial(point,P)\n",
|
|
"\n",
|
|
" G_analytical[i,j] = g_analytic(point)\n",
|
|
"\n",
|
|
" # Find the map difference between the analytical and the computed solution\n",
|
|
" diff_ag = np.abs(g_dnn_ag - G_analytical)\n",
|
|
" print('Max absolute difference between the analytical solution and the network: %g'%np.max(diff_ag))\n",
|
|
"\n",
|
|
" ## Plot the solutions in two dimensions, that being in position and time\n",
|
|
"\n",
|
|
" T,X = np.meshgrid(t,x)\n",
|
|
"\n",
|
|
" fig = plt.figure(figsize=(10,10))\n",
|
|
" ax = fig.gca(projection='3d')\n",
|
|
" ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n",
|
|
" s = ax.plot_surface(T,X,g_dnn_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
|
|
" ax.set_xlabel('Time $t$')\n",
|
|
" ax.set_ylabel('Position $x$');\n",
|
|
"\n",
|
|
"\n",
|
|
" fig = plt.figure(figsize=(10,10))\n",
|
|
" ax = fig.gca(projection='3d')\n",
|
|
" ax.set_title('Analytical solution')\n",
|
|
" s = ax.plot_surface(T,X,G_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
|
|
" ax.set_xlabel('Time $t$')\n",
|
|
" ax.set_ylabel('Position $x$');\n",
|
|
"\n",
|
|
" fig = plt.figure(figsize=(10,10))\n",
|
|
" ax = fig.gca(projection='3d')\n",
|
|
" ax.set_title('Difference')\n",
|
|
" s = ax.plot_surface(T,X,diff_ag,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
|
|
" ax.set_xlabel('Time $t$')\n",
|
|
" ax.set_ylabel('Position $x$');\n",
|
|
"\n",
|
|
" ## Take some slices of the 3D plots just to see the solutions at particular times\n",
|
|
" indx1 = 0\n",
|
|
" indx2 = int(Nt/2)\n",
|
|
" indx3 = Nt-1\n",
|
|
"\n",
|
|
" t1 = t[indx1]\n",
|
|
" t2 = t[indx2]\n",
|
|
" t3 = t[indx3]\n",
|
|
"\n",
|
|
" # Slice the results from the DNN\n",
|
|
" res1 = g_dnn_ag[:,indx1]\n",
|
|
" res2 = g_dnn_ag[:,indx2]\n",
|
|
" res3 = g_dnn_ag[:,indx3]\n",
|
|
"\n",
|
|
" # Slice the analytical results\n",
|
|
" res_analytical1 = G_analytical[:,indx1]\n",
|
|
" res_analytical2 = G_analytical[:,indx2]\n",
|
|
" res_analytical3 = G_analytical[:,indx3]\n",
|
|
"\n",
|
|
" # Plot the slices\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
" plt.title(\"Computed solutions at time = %g\"%t1)\n",
|
|
" plt.plot(x, res1)\n",
|
|
" plt.plot(x,res_analytical1)\n",
|
|
" plt.legend(['dnn','analytical'])\n",
|
|
"\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
" plt.title(\"Computed solutions at time = %g\"%t2)\n",
|
|
" plt.plot(x, res2)\n",
|
|
" plt.plot(x,res_analytical2)\n",
|
|
" plt.legend(['dnn','analytical'])\n",
|
|
"\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
" plt.title(\"Computed solutions at time = %g\"%t3)\n",
|
|
" plt.plot(x, res3)\n",
|
|
" plt.plot(x,res_analytical3)\n",
|
|
" plt.legend(['dnn','analytical'])\n",
|
|
"\n",
|
|
" plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "143f1c80",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"## Solving the wave equation with Neural Networks\n",
|
|
"\n",
|
|
"The wave equation is"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "190bd4f2",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\frac{\\partial^2 g(x,t)}{\\partial t^2} = c^2\\frac{\\partial^2 g(x,t)}{\\partial x^2}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9719cfc5",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"with $c$ being the specified wave speed.\n",
|
|
"\n",
|
|
"Here, the chosen conditions are"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f69ec7fd",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"$$\n",
|
|
"\\begin{align*}\n",
|
|
"\tg(0,t) &= 0 \\\\\n",
|
|
"\tg(1,t) &= 0 \\\\\n",
|
|
"\tg(x,0) &= u(x) \\\\\n",
|
|
"\t\\frac{\\partial g(x,t)}{\\partial t} \\Big |_{t = 0} &= v(x)\n",
|
|
"\\end{align*}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6ce8c5a8",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"where $\\frac{\\partial g(x,t)}{\\partial t} \\Big |_{t = 0}$ means the derivative of $g(x,t)$ with respect to $t$ is evaluated at $t = 0$, and $u(x)$ and $v(x)$ being given functions.\n",
|
|
"\n",
|
|
"The wave equation to solve for, is"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4be700d7",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"wave\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{equation} \\label{wave} \\tag{19}\n",
|
|
"\\frac{\\partial^2 g(x,t)}{\\partial t^2} = c^2 \\frac{\\partial^2 g(x,t)}{\\partial x^2}\n",
|
|
"\\end{equation}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "993f93ba",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"where $c$ is the given wave speed.\n",
|
|
"The chosen conditions for this equation are"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2cb2a80f",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"<!-- Equation labels as ordinary links -->\n",
|
|
"<div id=\"condwave\"></div>\n",
|
|
"\n",
|
|
"$$\n",
|
|
"\\begin{aligned}\n",
|
|
"g(0,t) &= 0, &t \\geq 0 \\\\\n",
|
|
"g(1,t) &= 0, &t \\geq 0 \\\\\n",
|
|
"g(x,0) &= u(x), &x\\in[0,1] \\\\\n",
|
|
"\\frac{\\partial g(x,t)}{\\partial t}\\Big |_{t = 0} &= v(x), &x \\in [0,1]\n",
|
|
"\\end{aligned} \\label{condwave} \\tag{20}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "3f1dcfd4",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"In this example, let $c = 1$ and $u(x) = \\sin(\\pi x)$ and $v(x) = -\\pi\\sin(\\pi x)$.\n",
|
|
"\n",
|
|
"Setting up the network is done in similar matter as for the example of solving the diffusion equation.\n",
|
|
"The only things we have to change, is the trial solution such that it satisfies the conditions from ([20](#condwave)) and the cost function.\n",
|
|
"\n",
|
|
"The trial solution becomes slightly different since we have other conditions than in the example of solving the diffusion equation. Here, a possible trial solution $g_t(x,t)$ is\n",
|
|
"\n",
|
|
"$$\n",
|
|
"g_t(x,t) = h_1(x,t) + x(1-x)t^2N(x,t,P)\n",
|
|
"$$\n",
|
|
"\n",
|
|
"where\n",
|
|
"\n",
|
|
"$$\n",
|
|
"h_1(x,t) = (1-t^2)u(x) + tv(x)\n",
|
|
"$$\n",
|
|
"\n",
|
|
"Note that this trial solution satisfies the conditions only if $u(0) = v(0) = u(1) = v(1) = 0$, which is the case in this example.\n",
|
|
"\n",
|
|
"The analytical solution for our specific problem, is\n",
|
|
"\n",
|
|
"$$\n",
|
|
"g(x,t) = \\sin(\\pi x)\\cos(\\pi t) - \\sin(\\pi x)\\sin(\\pi t)\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"id": "230a9aef",
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"editable": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import autograd.numpy as np\n",
|
|
"from autograd import hessian,grad\n",
|
|
"import autograd.numpy.random as npr\n",
|
|
"from matplotlib import cm\n",
|
|
"from matplotlib import pyplot as plt\n",
|
|
"from mpl_toolkits.mplot3d import axes3d\n",
|
|
"\n",
|
|
"## Set up the trial function:\n",
|
|
"def u(x):\n",
|
|
" return np.sin(np.pi*x)\n",
|
|
"\n",
|
|
"def v(x):\n",
|
|
" return -np.pi*np.sin(np.pi*x)\n",
|
|
"\n",
|
|
"def h1(point):\n",
|
|
" x,t = point\n",
|
|
" return (1 - t**2)*u(x) + t*v(x)\n",
|
|
"\n",
|
|
"def g_trial(point,P):\n",
|
|
" x,t = point\n",
|
|
" return h1(point) + x*(1-x)*t**2*deep_neural_network(P,point)\n",
|
|
"\n",
|
|
"## Define the cost function\n",
|
|
"def cost_function(P, x, t):\n",
|
|
" cost_sum = 0\n",
|
|
"\n",
|
|
" g_t_hessian_func = hessian(g_trial)\n",
|
|
"\n",
|
|
" for x_ in x:\n",
|
|
" for t_ in t:\n",
|
|
" point = np.array([x_,t_])\n",
|
|
"\n",
|
|
" g_t_hessian = g_t_hessian_func(point,P)\n",
|
|
"\n",
|
|
" g_t_d2x = g_t_hessian[0][0]\n",
|
|
" g_t_d2t = g_t_hessian[1][1]\n",
|
|
"\n",
|
|
" err_sqr = ( (g_t_d2t - g_t_d2x) )**2\n",
|
|
" cost_sum += err_sqr\n",
|
|
"\n",
|
|
" return cost_sum / (np.size(t) * np.size(x))\n",
|
|
"\n",
|
|
"## The neural network\n",
|
|
"def sigmoid(z):\n",
|
|
" return 1/(1 + np.exp(-z))\n",
|
|
"\n",
|
|
"def deep_neural_network(deep_params, x):\n",
|
|
" # x is now a point and a 1D numpy array; make it a column vector\n",
|
|
" num_coordinates = np.size(x,0)\n",
|
|
" x = x.reshape(num_coordinates,-1)\n",
|
|
"\n",
|
|
" num_points = np.size(x,1)\n",
|
|
"\n",
|
|
" # N_hidden is the number of hidden layers\n",
|
|
" N_hidden = np.size(deep_params) - 1 # -1 since params consist of parameters to all the hidden layers AND the output layer\n",
|
|
"\n",
|
|
" # Assume that the input layer does nothing to the input x\n",
|
|
" x_input = x\n",
|
|
" x_prev = x_input\n",
|
|
"\n",
|
|
" ## Hidden layers:\n",
|
|
"\n",
|
|
" for l in range(N_hidden):\n",
|
|
" # From the list of parameters P; find the correct weigths and bias for this layer\n",
|
|
" w_hidden = deep_params[l]\n",
|
|
"\n",
|
|
" # Add a row of ones to include bias\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_points)), x_prev ), axis = 0)\n",
|
|
"\n",
|
|
" z_hidden = np.matmul(w_hidden, x_prev)\n",
|
|
" x_hidden = sigmoid(z_hidden)\n",
|
|
"\n",
|
|
" # Update x_prev such that next layer can use the output from this layer\n",
|
|
" x_prev = x_hidden\n",
|
|
"\n",
|
|
" ## Output layer:\n",
|
|
"\n",
|
|
" # Get the weights and bias for this layer\n",
|
|
" w_output = deep_params[-1]\n",
|
|
"\n",
|
|
" # Include bias:\n",
|
|
" x_prev = np.concatenate((np.ones((1,num_points)), x_prev), axis = 0)\n",
|
|
"\n",
|
|
" z_output = np.matmul(w_output, x_prev)\n",
|
|
" x_output = z_output\n",
|
|
"\n",
|
|
" return x_output[0][0]\n",
|
|
"\n",
|
|
"## The analytical solution\n",
|
|
"def g_analytic(point):\n",
|
|
" x,t = point\n",
|
|
" return np.sin(np.pi*x)*np.cos(np.pi*t) - np.sin(np.pi*x)*np.sin(np.pi*t)\n",
|
|
"\n",
|
|
"def solve_pde_deep_neural_network(x,t, num_neurons, num_iter, lmb):\n",
|
|
" ## Set up initial weigths and biases\n",
|
|
" N_hidden = np.size(num_neurons)\n",
|
|
"\n",
|
|
" ## Set up initial weigths and biases\n",
|
|
"\n",
|
|
" # Initialize the list of parameters:\n",
|
|
" P = [None]*(N_hidden + 1) # + 1 to include the output layer\n",
|
|
"\n",
|
|
" P[0] = npr.randn(num_neurons[0], 2 + 1 ) # 2 since we have two points, +1 to include bias\n",
|
|
" for l in range(1,N_hidden):\n",
|
|
" P[l] = npr.randn(num_neurons[l], num_neurons[l-1] + 1) # +1 to include bias\n",
|
|
"\n",
|
|
" # For the output layer\n",
|
|
" P[-1] = npr.randn(1, num_neurons[-1] + 1 ) # +1 since bias is included\n",
|
|
"\n",
|
|
" print('Initial cost: ',cost_function(P, x, t))\n",
|
|
"\n",
|
|
" cost_function_grad = grad(cost_function,0)\n",
|
|
"\n",
|
|
" # Let the update be done num_iter times\n",
|
|
" for i in range(num_iter):\n",
|
|
" cost_grad = cost_function_grad(P, x , t)\n",
|
|
"\n",
|
|
" for l in range(N_hidden+1):\n",
|
|
" P[l] = P[l] - lmb * cost_grad[l]\n",
|
|
"\n",
|
|
"\n",
|
|
" print('Final cost: ',cost_function(P, x, t))\n",
|
|
"\n",
|
|
" return P\n",
|
|
"\n",
|
|
"if __name__ == '__main__':\n",
|
|
" ### Use the neural network:\n",
|
|
" npr.seed(15)\n",
|
|
"\n",
|
|
" ## Decide the vales of arguments to the function to solve\n",
|
|
" Nx = 10; Nt = 10\n",
|
|
" x = np.linspace(0, 1, Nx)\n",
|
|
" t = np.linspace(0,1,Nt)\n",
|
|
"\n",
|
|
" ## Set up the parameters for the network\n",
|
|
" num_hidden_neurons = [50,20]\n",
|
|
" num_iter = 1000\n",
|
|
" lmb = 0.01\n",
|
|
"\n",
|
|
" P = solve_pde_deep_neural_network(x,t, num_hidden_neurons, num_iter, lmb)\n",
|
|
"\n",
|
|
" ## Store the results\n",
|
|
" res = np.zeros((Nx, Nt))\n",
|
|
" res_analytical = np.zeros((Nx, Nt))\n",
|
|
" for i,x_ in enumerate(x):\n",
|
|
" for j, t_ in enumerate(t):\n",
|
|
" point = np.array([x_, t_])\n",
|
|
" res[i,j] = g_trial(point,P)\n",
|
|
"\n",
|
|
" res_analytical[i,j] = g_analytic(point)\n",
|
|
"\n",
|
|
" diff = np.abs(res - res_analytical)\n",
|
|
" print(\"Max difference between analytical and solution from nn: %g\"%np.max(diff))\n",
|
|
"\n",
|
|
" ## Plot the solutions in two dimensions, that being in position and time\n",
|
|
"\n",
|
|
" T,X = np.meshgrid(t,x)\n",
|
|
"\n",
|
|
" fig = plt.figure(figsize=(10,10))\n",
|
|
" ax = fig.gca(projection='3d')\n",
|
|
" ax.set_title('Solution from the deep neural network w/ %d layer'%len(num_hidden_neurons))\n",
|
|
" s = ax.plot_surface(T,X,res,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
|
|
" ax.set_xlabel('Time $t$')\n",
|
|
" ax.set_ylabel('Position $x$');\n",
|
|
"\n",
|
|
"\n",
|
|
" fig = plt.figure(figsize=(10,10))\n",
|
|
" ax = fig.gca(projection='3d')\n",
|
|
" ax.set_title('Analytical solution')\n",
|
|
" s = ax.plot_surface(T,X,res_analytical,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
|
|
" ax.set_xlabel('Time $t$')\n",
|
|
" ax.set_ylabel('Position $x$');\n",
|
|
"\n",
|
|
"\n",
|
|
" fig = plt.figure(figsize=(10,10))\n",
|
|
" ax = fig.gca(projection='3d')\n",
|
|
" ax.set_title('Difference')\n",
|
|
" s = ax.plot_surface(T,X,diff,linewidth=0,antialiased=False,cmap=cm.viridis)\n",
|
|
" ax.set_xlabel('Time $t$')\n",
|
|
" ax.set_ylabel('Position $x$');\n",
|
|
"\n",
|
|
" ## Take some slices of the 3D plots just to see the solutions at particular times\n",
|
|
" indx1 = 0\n",
|
|
" indx2 = int(Nt/2)\n",
|
|
" indx3 = Nt-1\n",
|
|
"\n",
|
|
" t1 = t[indx1]\n",
|
|
" t2 = t[indx2]\n",
|
|
" t3 = t[indx3]\n",
|
|
"\n",
|
|
" # Slice the results from the DNN\n",
|
|
" res1 = res[:,indx1]\n",
|
|
" res2 = res[:,indx2]\n",
|
|
" res3 = res[:,indx3]\n",
|
|
"\n",
|
|
" # Slice the analytical results\n",
|
|
" res_analytical1 = res_analytical[:,indx1]\n",
|
|
" res_analytical2 = res_analytical[:,indx2]\n",
|
|
" res_analytical3 = res_analytical[:,indx3]\n",
|
|
"\n",
|
|
" # Plot the slices\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
" plt.title(\"Computed solutions at time = %g\"%t1)\n",
|
|
" plt.plot(x, res1)\n",
|
|
" plt.plot(x,res_analytical1)\n",
|
|
" plt.legend(['dnn','analytical'])\n",
|
|
"\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
" plt.title(\"Computed solutions at time = %g\"%t2)\n",
|
|
" plt.plot(x, res2)\n",
|
|
" plt.plot(x,res_analytical2)\n",
|
|
" plt.legend(['dnn','analytical'])\n",
|
|
"\n",
|
|
" plt.figure(figsize=(10,10))\n",
|
|
" plt.title(\"Computed solutions at time = %g\"%t3)\n",
|
|
" plt.plot(x, res3)\n",
|
|
" plt.plot(x,res_analytical3)\n",
|
|
" plt.legend(['dnn','analytical'])\n",
|
|
"\n",
|
|
" plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a23bd19a",
|
|
"metadata": {
|
|
"editable": true
|
|
},
|
|
"source": [
|
|
"## Resources on differential equations and deep learning\n",
|
|
"\n",
|
|
"1. [Artificial neural networks for solving ordinary and partial differential equations by I.E. Lagaris et al](https://pdfs.semanticscholar.org/d061/df393e0e8fbfd0ea24976458b7d42419040d.pdf)\n",
|
|
"\n",
|
|
"2. [Neural networks for solving differential equations by A. Honchar](https://becominghuman.ai/neural-networks-for-solving-differential-equations-fa230ac5e04c)\n",
|
|
"\n",
|
|
"3. [Solving differential equations using neural networks by M.M Chiaramonte and M. Kiener](http://cs229.stanford.edu/proj2013/ChiaramonteKiener-SolvingDifferentialEquationsUsingNeuralNetworks.pdf)\n",
|
|
"\n",
|
|
"4. [Introduction to Partial Differential Equations by A. Tveito, R. Winther](https://www.springer.com/us/book/9783540225515)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.9.10"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
} |