FYS-STK4155/doc/LectureNotes/_build/jupyter_execute/exercisesweek41.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e3354581",
   "metadata": {
    "editable": true
   },
   "source": [
    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
    "doconce format html exercisesweek41.do.txt  -->\n",
    "<!-- dom:TITLE: Exercises week 41 -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d5fc499",
   "metadata": {
    "editable": true
   },
   "source": [
    "# Exercises week 41\n",
    "**October 9-13, 2023**\n",
    "\n",
    "Date: **Deadline is Sunday October 15 at midnight**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3ef3596",
   "metadata": {
    "editable": true
   },
   "source": [
    "# Overarching aims of the exercises this week\n",
    "\n",
    "The aim of the exercises this week is to get started with implementing\n",
    "gradient methods of relevance for project 2. This exercise will also\n",
    "be continued next week with the addition of automatic differentation.\n",
    "Everything you develop here will be used in project 2. \n",
    "\n",
    "In order to get started, we will now replace in our standard ordinary\n",
    "least squares (OLS) and Ridge regression codes (from project 1) the\n",
    "matrix inversion algorithm with our own gradient descent (GD) and SGD\n",
    "codes.  You can use the Franke function or the terrain data from\n",
    "project 1. **However, we recommend using a simpler function like**\n",
    "$f(x)=a_0+a_1x+a_2x^2$ or higher-order one-dimensional polynomials.\n",
    "You can obviously test your final codes against for example the Franke\n",
    "function. Automatic differentiation will be discussed next week.\n",
    "\n",
    "You should include in your analysis of the GD and SGD codes the following elements\n",
    "1. A plain gradient descent with a fixed learning rate (you will need to tune it) using the analytical expression of the gradients\n",
    "\n",
    "2. Add momentum to the plain GD code and compare convergence with a fixed learning rate (you may need to tune the learning rate), again using the analytical expression of the gradients.\n",
    "\n",
    "3. Repeat these steps for stochastic gradient descent with mini batches and a given number of epochs. Use a tunable learning rate as discussed in the lectures from week 39. Discuss the results as functions of the various parameters (size of batches, number of epochs etc)\n",
    "\n",
    "4. Implement the Adagrad method in order to tune the learning rate. Do this with and without momentum for plain gradient descent and SGD.\n",
    "\n",
    "5. Add RMSprop and Adam to your library of methods for tuning the learning rate.\n",
    "\n",
    "The lecture notes from weeks 39 and 40 contain more information and code examples. Feel free to use these examples.\n",
    "\n",
    "In summary, you should \n",
    "perform an analysis of the results for OLS and Ridge regression as\n",
    "function of the chosen learning rates, the number of mini-batches and\n",
    "epochs as well as algorithm for scaling the learning rate. You can\n",
    "also compare your own results with those that can be obtained using\n",
    "for example **Scikit-Learn**'s various SGD options.  Discuss your\n",
    "results. For Ridge regression you need now to study the results as functions of  the hyper-parameter $\\lambda$ and \n",
    "the learning rate $\\eta$.  Discuss your results.\n",
    "\n",
    "You will need your SGD code for the setup of the Neural Network and\n",
    "Logistic Regression codes. You will find the Python [Seaborn\n",
    "package](https://seaborn.pydata.org/generated/seaborn.heatmap.html)\n",
    "useful when plotting the results as function of the learning rate\n",
    "$\\eta$ and the hyper-parameter $\\lambda$ when you use Ridge\n",
    "regression.\n",
    "\n",
    "We recommend reading chapter 8 on optimization from the textbook of [Goodfellow, Bengio and Courville](https://www.deeplearningbook.org/). This chapter contains many useful insights and discussions on the optimization part of machine learning."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9940e21b",
   "metadata": {
    "editable": true
   },
   "source": [
    "# Code examples from week 39 and 40"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6559d6d2",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Code with a Number of Minibatches which varies, analytical gradient\n",
    "\n",
    "In the code here we vary the number of mini-batches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "021b66d2",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.97739698]\n",
      " [2.9189726 ]]\n",
      "Eigenvalues of Hessian Matrix:[0.27987128 4.51549827]\n",
      "theta from own gd\n",
      "[[3.97739698]\n",
      " [2.9189726 ]]\n",
      "theta from own sdg\n",
      "[[3.9577228 ]\n",
      " [2.91473433]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHFCAYAAADxOP3DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABVcElEQVR4nO3deVxU9f4/8NcBBMQABRTZFDRT0CzX3JdyHfRqZJbeyPzWbXG/drvptVK7mbd+3dIy87bpLUNbRK0YNUtQM/VqLuW+oSKiuIKiIAyf3x/TjAzMziznzHk9H4952JzzmTOfMzOn8+bzeX8+H0kIIUBERESkUn7ergARERGRNzEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIpKhJUuWQJIk4yMgIAAxMTF49NFHcfToUa/Va9asWZAkyWvvrwQnT56EJEl46623vF0VIrJTgLcrQESWLV68GK1atUJpaSm2bNmCOXPmIDs7G4cOHUKDBg28XT0iIp/AYIhIxtq0aYOOHTsCAPr06QOdToeZM2di1apVGDt2rJdrR95UXl5ubDUkotphNxmRghgCo/Pnzxu3lZaW4vnnn8e9996L8PBwREREoGvXrli9enWN10uShAkTJuDzzz9HcnIyQkJCcM899+D777+vUTYrKwv33nsvgoKCkJSUZLHbp7S0FNOnT0dSUhICAwMRFxeH8ePH4+rVqyblEhMTMWTIEHz//fdo164d6tati+TkZON7L1myBMnJyahXrx46d+6MnTt32vw8DN2J2dnZeO655xAVFYXIyEikpaXh7NmzNc591qxZNY6RmJiIJ554osYxN2zYgL/85S+IjIxEWFgYHn/8cZSUlODcuXMYOXIk6tevj5iYGPztb39DeXl5jeNWVlZizpw5aNKkCYKDg9GxY0f89NNPNcodPXoUo0ePRqNGjRAUFITk5GS8//77JmVycnIgSRI+//xzPP/884iLi0NQUBCOHTuGGzdu4G9/+xuSkpIQHByMiIgIdOzYEcuWLbP5+RGRHv+kIFKQ3NxcAMBdd91l3FZWVobLly/jb3/7G+Li4nDr1i38+OOPSEtLw+LFi/H444+bHCMrKws7duzAq6++ijvuuANvvvkmHnzwQRw+fBjNmjUDAPz0008YNmwYunbtiuXLl0On0+HNN980CcIAQAiB4cOH46effsL06dPRs2dP/Pbbb5g5cya2bt2KrVu3IigoyFh+7969mD59OmbMmIHw8HDMnj0baWlpmD59On766Se8/vrrkCQJL774IoYMGYLc3FzUrVvX5ufy1FNPITU1FRkZGcjLy8MLL7yAxx57DBs2bHD6s37qqaeQlpaG5cuXY/fu3fjHP/6BiooKHD58GGlpaXj66afx448/4o033kBsbCymTp1q8voFCxagadOmmDdvHiorK/Hmm29i8ODB2LhxI7p27QoAOHDgALp164YmTZrg3//+Nxo3box169Zh0qRJuHjxImbOnGlyzOnTp6Nr165YtGgR/Pz80KhRI0ydOhWff/45XnvtNbRr1w4lJSXYt28fLl265PS5E6mOICLZWbx4sQAgtm3bJsrLy8W1a9fE2rVrRePGjUWvXr1EeXm5xddWVFSI8vJy8eSTT4p27dqZ7AMgoqOjRXFxsXHbuXPnhJ+fn5g7d65x23333SdiY2PFzZs3jduKi4tFRESEqPq/jbVr1woA4s033zR5ny+//FIAEB9++KFxW9OmTUXdunXFmTNnjNv27NkjAIiYmBhRUlJi3L5q1SoBQHz77bd2fU7jxo0z2f7mm28KAKKgoMDk3GfOnFnjGE2bNhVjxoypccyJEyealBs+fLgAIN5++22T7ffee69o37698Xlubq4AYPHz69evn3HbwIEDRXx8vCgqKjI55oQJE0RwcLC4fPmyEEKI7OxsAUD06tWrRv3btGkjhg8fXmM7EdmP3WREMtalSxfUqVMHoaGhGDRoEBo0aIDVq1fXyBP5+uuv0b17d9xxxx0ICAhAnTp18Mknn+DgwYM1jtm3b1+EhoYan0dHR6NRo0Y4deoUAKCkpAQ7duxAWloagoODjeVCQ0MxdOhQk2MZWl6qdjMBwMMPP4x69erV6Ba69957ERcXZ3yenJwMQJ8PFRISUmO7oU62/OlPfzJ53rZtW4deb86QIUNMnhvqlJqaWmO7ufex9Plt2rQJOp0OpaWl+Omnn/Dggw8iJCQEFRUVxodGo0FpaSm2bdtmcsyHHnqoxvt07twZa9aswbRp05CTk4ObN286fc5EasVgiEjGPvvsM+zYsQMbNmzAM888g4MHD2LUqFEmZTIzMzFy5EjExcVh6dKl2Lp1K3bs2IH/+7//Q2lpaY1jRkZG1tgWFBRkvIleuXIFlZWVaNy4cY1y1bddunQJAQEBaNiwocl2SZLQuHHjGl01ERERJs8DAwOtbjdXf3Oqn5Oha642gYEjdTVXT0uf361bt3D9+nVcunQJFRUVeO+991CnTh2Th0ajAQBcvHjR5PUxMTE1jvnuu+/ixRdfxKpVq9C3b19ERERg+PDhXp2CgUhpmDNEJGPJycnGpOm+fftCp9Ph448/xjfffIMRI0YAAJYuXYqkpCR8+eWXJnMAlZWVOfWeDRo0gCRJOHfuXI191bdFRkaioqICFy5cMAmIhBA4d+4cOnXq5FQd3CEoKMjsZ+Ku3BpLn19gYCDuuOMO1KlTB/7+/khPT8f48ePNHiMpKcnkubk5nurVq4fZs2dj9uzZOH/+vLGVaOjQoTh06JBrTobIx7FliEhB3nzzTTRo0ACvvPIKKisrAehvkIGBgSY3ynPnzpkdTWYPw2iuzMxMkxaPa9eu4bvvvjMp+8ADDwDQB2RVrVixAiUlJcb9cpCYmIjffvvNZNuGDRtw/fp1t7yfpc+vZ8+e8Pf3R0hICPr27Yvdu3ejbdu26NixY42HuVY8a6Kjo/HEE09g1KhROHz4MG7cuOHq0yLySWwZIlKQBg0aYPr06fj73/+OjIwMPPbYYxgyZAgyMzMxbtw4jBgxAnl5efjnP/+JmJgYp7tK/vnPf2LQoEHo378/nn/+eeh0OrzxxhuoV68eLl++bCzXv39/DBw4EC+++CKKi4vRvXt342iydu3aIT093VWnXmvp6el4+eWX8corr6B37944cOAAFixYgPDwcLe8n7+/P/r374+pU6eisrISb7zxBoqLizF79mxjmfnz56NHjx7o2bMnnnvuOSQmJuLatWs4duwYvvvuO7tGw913330YMmQI2rZtiwYNGuDgwYP4/PPP0bVrV5M8LCKyjMEQkcJMnDgRCxYswKuvvopRo0Zh7NixKCwsxKJFi/Dpp5+iWbNmmDZtGs6cOWNy43VE//79sWrVKrz00kt45JFH0LhxY4wbNw43b940OaYkSVi1ahVmzZqFxYsXY86cOYiKikJ6ejpef/11k2H13vbCCy+guLgYS5YswVtvvYXOnTvjq6++wrBhw9zyfhMmTEBpaSkmTZqEwsJCtG7dGllZWejevbuxTEpKCnbt2oV//vOfeOmll1BYWIj69eujRYsWxrwhW+6//358++23eOedd3Djxg3ExcXh8ccfx4wZM9xyXkS+SBJCCG9XgoiIiMhbmDNEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1VQxz1BlZSXOnj2L0NBQs9PZExERkfwIIXDt2jXExsbCz8997TeqCIbOnj2LhIQEb1eDiIiInJCXl4f4+Hi3HV8VwVBoaCgA/YcZFhbm5doQERGRPYqLi5GQkGC8j7uLKoIhQ9dYWFgYgyEiIiKFcXeKCxOoiYiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNW8Hgxt2rQJQ4cORWxsLCRJwqpVqyyWfeaZZyBJEubNm+ex+hEREZFv83owVFJSgnvuuQcLFiywWm7VqlXYvn07YmNjPVQzIiIiUoMAb1dg8ODBGDx4sNUy+fn5mDBhAtatW4fU1FQP1YyIiIjUwOstQ7ZUVlYiPT0dL7zwAlq3bu3t6hAREZGP8XrLkC1vvPEGAgICMGnSJLtfU1ZWhrKyMuPz4uJid1SNiIiIfICsW4Z+/fVXzJ8/H0uWLIEkSXa/bu7cuQgPDzc+EhIS3FhLIiIiUjJZB0ObN29GYWEhmjRpgoCAAAQEBODUqVN4/vnnkZiYaPF106dPR1FRkfGRl5fnuUoTERGRosi6myw9PR39+vUz2TZw4ECkp6dj7NixFl8XFBSEoKAgd1ePiIiIfIDXg6Hr16/j2LFjxue5ubnYs2cPIiIi0KRJE0RGRpqUr1OnDho3boyWLVt6uqpERETkg7weDO3cuRN9+/Y1Pp86dSoAYMyYMViyZImXakVERERq4fVgqE+fPhBC2F3+5MmT7qsMERERqY6sE6iJiIiI3I3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoB3q4AERERqZhOB2zeDBQUADExQM+egL+/R6vAYIiIiIi8IzMTmDwZOHPm9rb4eGD+fCAtzWPVYDcZEREReV5mJjBihGkgBAD5+frtmZkeqwqDISIiIvIsnU7fIiREzX2GbVOm6Mt5AIMhIiIi8qzNm2u2CFUlBJCXB/zyi0eqw2CIiIiIPKugwL5y5865tx5/YDBEREREnhUTY1+5xo3dW48/MBgiIiIiz+rZUz9qTJLM75ckICEB6NbNI9VhMERERESe5e+vHz4P1AyIDM/nzfPYfEMMhoiIiMjz0tKAb74B4uJMt8fH67d7cJ4hTrpIRERE3pGWBgwb5vUZqL3eMrRp0yYMHToUsbGxkCQJq1atMu4rLy/Hiy++iLvvvhv16tVDbGwsHn/8cZw9e9Z7FSYiIiLX8fcH+vQBRo3S/+vhQAiQQTBUUlKCe+65BwsWLKix78aNG9i1axdefvll7Nq1C5mZmThy5Aj+9Kc/eaGmRERE5IskIcxN/+gdkiRh5cqVGD58uMUyO3bsQOfOnXHq1Ck0adLEruMWFxcjPDwcRUVFCAsLc1FtiYiIyJ08df9WXM5QUVERJElC/fr1LZYpKytDWVmZ8XlxcbEHakZERERK5PVuMkeUlpZi2rRpGD16tNUIce7cuQgPDzc+EhISPFhLIiIiUhLFBEPl5eV49NFHUVlZiYULF1otO336dBQVFRkfeXl5HqolERERKY0iusnKy8sxcuRI5ObmYsOGDTb7DYOCghAUFOSh2hEREZGSyT4YMgRCR48eRXZ2NiIjI71dJSIiIvIhXg+Grl+/jmPHjhmf5+bmYs+ePYiIiEBsbCxGjBiBXbt24fvvv4dOp8O5P1awjYiIQGBgoLeqTURERD7C60Prc3Jy0Ldv3xrbx4wZg1mzZiEpKcns67Kzs9GnTx+73oND64mIiJRHNUPr+/TpA2vxmIymQSIiIiIfpJjRZERERETuwGCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlUL8HYFiIiIZEenAzZvBgoKgJgYoGdPwN/f27UiN2EwREREVFVmJjB5MnDmzO1t8fHA/PlAWpr36kVuw24yIiIig8xMYMQI00AIAPLz9dszM71TL6XT6YCcHGDZMv2/Op23a2SCwRARERGgv0FPngwIUXOfYduUKbK7kcteZiaQmAj07QuMHq3/NzFRVoElgyEiIiJAnyNUvUWoKiGAvDx9ObKPQlraGAwREREB+mRpV5ZTOwW1tDEYIiIiAvSjxlxZTu0U1NLGYIiIiAjQD5+Pjwckyfx+SQISEvTlyDYFtbQxGCIiIgL08wjNn6//7+oBkeH5vHmcb8heCmppYzBERERkkJYGfPMNEBdnuj0+Xr+d8wzZT0EtbZx0kYiIqKq0NGDYMM5AbYm9s3MbWtpGjNAHPlUTqWXW0sZgiIiIfE9tl9Pw9wf69HFb9RTL0dm5DS1t5l4zb55sWtokIcyNefMtxcXFCA8PR1FREcLCwrxdHSIicicup3GbK9dYM8wZVD1sMLTyWOtGdLIenrp/MxgiIiLfUZsbtq9xZVCo0+lnjbY0VF6S9MfOzXVpt5en7t9eT6DetGkThg4ditjYWEiShFWrVpnsF0Jg1qxZiI2NRd26ddGnTx/s37/fO5UlIiLHeHJNKgVN8ud2rp75WUFzBjnD68FQSUkJ7rnnHixYsMDs/jfffBNvv/02FixYgB07dqBx48bo378/rl275uGaEhGRQ8GNp9ek8vEbtt3cERQqaM4gZ3g9GBo8eDBee+01pJlpshNCYN68eZgxYwbS0tLQpk0b/Pe//8WNGzeQkZHhhdoSEamYI8GNN9ak8vEbtt3cERQqaM4gZ3g9GLImNzcX586dw4ABA4zbgoKC0Lt3b/zyyy8WX1dWVobi4mKTBxER1YIjwY23uqt8/IZtN3cEhQqaM8gZsg6Gzp07BwCIjo422R4dHW3cZ87cuXMRHh5ufCQkJLi1nkREPs3R4MZb3VWuumF7Ms/JHdwRFPr47NyyDoYMpGofvBCixraqpk+fjqKiIuMjLy/P3VUkIvJdjgY33uqucsUN29N5Tu7grlYcH56dW9bBUOPGjQGgRitQYWFhjdaiqoKCghAWFmbyICIiJzka3Hizu6o2N2xv5Dm5gzNBob2tYWlpwMmTQHY2kJGh/zc3V9GBECDzYCgpKQmNGzfG+vXrjdtu3bqFjRs3olu3bl6sGRGRijga3Hg7v8SZG7YchuW7snvOkaDQ0dYww+zco0bp/3Vj19jl45fdduyqvL4cx/Xr13Hs2DHj89zcXOzZswcRERFo0qQJpkyZgtdffx0tWrRAixYt8PrrryMkJASjR4/2Yq2JiFTEENzk55sPFgwT7hmCGzmsSeXochqOdAW6Y5kOd8yabc8aa5YmqTS0hnm4+0tUCuz58jC0n56DdnsEtl7zUM6v8LLs7GwBoMZjzJgxQgghKisrxcyZM0Xjxo1FUFCQ6NWrl/j9998deo+ioiIBQBQVFbnhDIiIVGDFCiEkSf/Q3zr1D8O2FSvMvyY+3rR8QoL5st6WkWFaT0uPjAzXv7fhs63+XtY+W1eoqKj5/VR//4QEfTk3KsorEite2Cr+r8UmEeNXUK0anrl/czkOIiKyj7nWi4QE6wtuunJtLHfKydF3D9mSne3aliEvLXMBwGvnLCoFDmlPQPufPGT9HI7NV9ugAnWM++vhOvo13g/NA2Xo8X/RaP1AK7ffv73eTUZERAphT7dLdUpZ/d3RrkBX8Wb3nAdH/d24eAPZC/ZDu+IGtIea4WRFcwDNjftbBRzD+MTv0btDCVqmd0LgoAcAf3+PzRPIYIiIiOynlODGUd7Kc/LmrNluHvWXuykPWe+dgHZjCLIvtEEpOhn3BaEUfSL3IbVPCUa0OYSYT14Djp0BjgH4Erfzpfr1c+q9HcVgiIiICLg9AstcIrO1rsDa8OY0BC5uDbt1/RY2f7AP2i+Lof09AYduNQdwOwG6if8ZaFqeQOpDweg7oTXqNer4RwL3c5YTuD/7rBYnaD/mDBEREVXlyTwnQ86QrYDEHTlDwO3RZID51jAbo8nydxZgzXvHoF1fB+sLWuM6Qo37/FGBHuH7kNr9KjRPxyNlaHNIflWmW7AjX6o4Nhbh+fluv38zGCIiIvWRU2J3LQMSl7y/nYnxFaUV2L74ALRLLyNrVwz2lrY02R/tVwhN88PQDA1A/8kpCG8Sbvl97UjgLgYQDjCBmoiIyKXcMadPbXije676+1tJjL9w8CLWvXsY2jXA2tMpuCLaGl8qoRL33bEfms6XoBkbjXaPtoRfQCP73tcdeVBOYssQERGph6VJBj3VCmONTFqrKisqsXv5YWR9eh7a/0XhfyUpEFUWrGggXcGgJgegGQwMmtIKUS0jnXsjGbUMMRgiIiJ18OacPjJXdLoIP8w7AO13FVhzoiXOV5q27txb9xA07c4h9fFIdB6TjIDgWnYsGZYeGTkSuGxhyQ0P5gyxm4yIiNTB20tuyIioFNi/+hi0H+VD+0t9/FzUBjp0Ne6/A9fQP2Y/UgeUY9CEOxHXsRWAVq55c3PdlNUZWur+9S8gPd0172sFgyEiIlIHb87pIwMlhSXY8N5+aDNLoT3cDKd1LQC0MO5vFXgcqXfnQTMqHD2eaY3AO7q4vhKWuimrM+RLcZ4hIiIiF/LmnD5ecuynU9AuPAntxnrIudQGZehs3BeMm7i/4T5o+tzA4HFJaNbHdFZol9Pp9C1C1gKhyEjgyy/1LXP+/gBnoCYiItVzZVKxt5bc8KCy4jJsWrgP2q+uI2tfExwtTwLQ1Lg/MSAPqcknoHkoBH3Gt0ZIVCfLB3M1W92UAHDpkv779XDOFoMhIiKSJ1cPgffWkhtulrf9LNa8dwxZPwXhp3OtUYIOxn0BKEevBr9D070YmmcS0ErTDJJfgpWjuZGMuykZDBERVSWT4c2qZym3xLBMg7ND4D0xp4+bf0MVpRXY+vF+ZC29Au3eWOwvbY6eOIIYnEBH3MQxqQUGtTgBzbA66DcpBWHx7V323rUi425KDq0nIjKQ22R8auWJIfDuCljc9Bs6v+8C1s4/DO06P6zLa40i6Gd2fhCZmI/JSMDt9xPx8ZDk+Jt1YukRT92/GQwREQHynoxPbeyYjA8AkJ0tryHwLvwNVVZUYufnB6H97wVodzbEjpLWJvsjpUt4ueEiTCp8Sf8WtXw/j3Fw6REGQy7EYIiIrHJFSwS711xn2TJg9Gjb5TIygFGjHDu2u74nF/yGruRe1U98+H0l1uS2xAXR0GR/+7oHkdrxPDSPR6HTn++C/13NlTmBpANroXnq/s2cISKi2k7Gx+4113JXbok7vycnfkOiUuD3zKPQfnwWWVsbYGtxa+jQzfiSMBRhQNwBaAZUYNCkuxBzbzKAZP3OnBzlTiBpYy00b2AwRERUm1Eu7kr0VTN3DIF39/dk52+o9GAu1v1QF1kry6A90gL5lXcBuMu4v3XQUWja5kMzuj66P90adUK6mj+Qvb/Z1avlFwwB+sBHRvViMERE5GxLhLVJ5ITQ37SnTNH/FSy3rgo5c/UQeE98T3b+hoaOi8OPuM/4vC5u4IFG+6C5vxSaCc3QtLvprNC1fT/Mm6cPGhmQW8WcISIiJ0a5AFBuom9teDI3yoHcEqs88T3Z+A1VQsIZxCMJuUgMOIPUlJPQPFwPfSa0QXD9YOffz9YkhnLOHbKDp+7ffm47MhGRUhhaIoDbLQ8G1loiZDyJnFtkZupvwH376hOc+/bVP8/MdM/7paUBJ0/qg5SMDP2/ubmOt3J44Hs6ta0Aa5s+AyEEKk3HdqESEiQI7O38FxxcexrHyprg3b29Meiljs4FQoDpb9aaqrlDZBGDISIi4PZkfHFxptvj4y3nk8h4EjmXM+TcVG+JMOTcuCsgMuSWjBp1e70qR7nheyq/UY6ceXvw9845aB18DIk94jF4y0t4CCuQj2q/odhYSCtWYOj2l3HXwCRIfpL5gzoqLU3fvWcPXwnI3YTdZEREVTnSDeRs95rSeGISRHdy0fdUsOc81r57BNofAvBDfgqK/5j4EAD8UYFuYfuh6XoFmrHRuLvhOUjnz7m/K9EdXYAymiaCQ+uJiLzBkVEuPrrWVQ21nXrA25z8nnS3dNjx2UFoP7uIrJ3R2HUzGUC0cX9D6QIGJx2GZogfBkxJQYOke6q8Otltp2PC1SPvVDpNBIMhIqLa8MRaV97mC7lRdn5Pl45exg/vHkLW9wJrT7XCJdHG5DCd6u2HpuMFaMY0RMf0ZPgF9PDgSZjhyoBcxdNEsJuMiMgVZNS14HK+NGqu2vckuvfA3szjyPq4ANrtEdh2rTUqcft7C0cRBibsh2ZgJQZNbonoNg2tHNyLajvyTqZdoVyOw4UYDBER1YKP5UYVnynGj+8egHZ1ObTHWqCgsrHJ/ruDj0Bzz1mkPtYAXZ9qjYBghXSi1CYgl2nAy5whIiKSB4XnRolKgUPaE9D+Jw/aLWHYfKUNytHFuL8erqNf4/3QPFCGwRPvRMJ9dwG65vrAYuXXymnpq82szr7QFVoLDIaIiOzhy91g9lBYbtSNizeQ8/5+aFfcQNbBZjhZ0RxAc+P+u+rkQtPmNDQj70CvcW0QFHZ7VmhVJhGraZoIM9hNRkRkixpvjpbIOCjM3ZQH7YITyMoJQfaFNihFXeO+IJSiT+Q+aHqXQDMuEXc+0NT8QSwlERtawHw1iVimXaHMGXIhBkNEMiLjm6lZar05KsCt67fw83/2I2tZEbS/J+DQreYm+xP885Ha8jg0acG4f2Jr1GtUz/oBZZpE7DGG3zpgvivUC791BkMuxGCISCastbAMGya/IEkJN0elBZe1dHbXOWjnH4V2fR2sL2iN6wg17vNHBXqE74Om21WkPhOPlKHNHZvtWaZJxB7lqvXgXIQJ1ETkW6zNYfLQQ0BkJHDp0u3tcuiGkvtkg0rvvrMjkNPd0mHbJ/uhXXoZ2t2NsedmKwC3R39F+xVicLPDSP1TAPpNSkH9pvc6Xx+VJxED0P9u5PiHiZsxGCIi99Pp9Ddtcw3Rhm1VAyFAHhO9yfnmqPQJ8qwEcheSe+GHeQdwYvVenD4fjKNogc3oiUr4Q0IlOtc7gNT7LkIzNhrtHm0Jv4BGrqmTUpOIXd06WJtRaQrFbjIicj97ux+q83Y3lFy7TZTQfWeNhUBO/0zC/8PzGIXlSMDt87vk3xD7HpiClHlPo2FylHvqJdMkYquU3jpog6fu31y1nojcz9mWk6rdUN5gWPdJspB3Ikn6fAp7131yFUe67+Tmj1ZCc3+HSwAEBF7AW4iD6flFVl5E7/UvoeHBTe6rm2E+JaDmdy7H+ZQMQWX134KhdTAz0zv1UiAGQ0TkfrXtVvBWjoZcb45y7r6zQFQK7F99DMu7vwecOQNLac1+0AdFNW5OhuBpyhR9QOUuhvmU4uJMt8fHy6vr0Z6uZ3d/Vj5E9sFQRUUFXnrpJSQlJaFu3bpo1qwZXn31VVRWVnq7akRkL1stLLZ4M0dDjjdHheS2lBSW4LuX/4fnWm9CYmA+2gy/E6u3R9t+oSWeavFKSwNOntR3f2Zk6P/NzZVPIAQou3VQhmSfQP3GG29g0aJF+O9//4vWrVtj586dGDt2LMLDwzF58mRvV4+I7GFtOQdrDDkanu6Gqk5uI2wMwaWt3BYvfG7HN5xC1vsnod1YDzmX2qAMnY37gnETTcKLgKJavoknWrzknkSswNZBOZN9MLR161YMGzYMqampAIDExEQsW7YMO3fu9HLNiMghlpZzMAypl/uaV3K6OcporbCy4jJsWrgP2q+uQ7uvCY6UJwG4PbtzU/8zSE0+jtQRIegzvjVCGvwFSJxjOZCzh9xGc3mDQloHlUL23WQ9evTATz/9hCNHjgAA9u7di59//hkajcbLNSMih5nrfjh/HlixQl7dUErgxe67vO1n8eFjmzA8Zjsiw8sxYHoHzNvdG0fKkxCAcvStvxtvDcnBge+OI/dWHN7/vTc0MzshJCrENA/LGQ0bAt26ue5klEquyf0KJfuh9UII/OMf/8Abb7wBf39/6HQ6zJkzB9OnT7f4mrKyMpSVlRmfFxcXIyEhgUPriWrD3TMdq2wmZZfxwOdWUVqBrR/vh/aLK8jaE4vfS+8y2R/jdw6aO49CM6wO+k1KQVi8Hf+fzcwEnnkGuHjR8Qq5Y+i4En9/Mlw+w9U8NjWOkLlly5aJ+Ph4sWzZMvHbb7+Jzz77TERERIglS5ZYfM3MmTMF9FNWmDyKioo8WHMiH7JihRDx8ULo/5erf8TH67eTTzq/r1D89y+bxciELaK+dMXkq/dDhegWule81i9b7Mo4KCp1lc69ydKlpr8pex+SpH9Y+/1VVAiRnS1ERob+34oKy2Xl/vu2di7m6p6QIJ+611JRUZFH7t+ybxlKSEjAtGnTMH78eOO21157DUuXLsWhQ4fMvoYtQ0QuxIVKVaGyohK/fnEIWYsLod3ZEDtLkiGqZFJESpcwqOkhaDTAwCnJiGwRUfs3dXYyTsD6BIiOTEQo99+3PediaNXKzwcuXNB3JcbFWW/dUkhLGFuG/hARESEWLlxosu31118XLVq0sPsYnoosiXxORUXNvzqr/4WekGD9r26Srcsnrojlk7aIx5ttFg2lwhpfb/u6B8RLPbLF1o9+FxVlbviODb8vSXKuhQjQt5RUtWKF+eOZa02S++/bkXNxpHVL7i1hVXjq/i37YGjMmDEiLi5OfP/99yI3N1dkZmaKqKgo8fe//93uYzAYInJSdrZzNySSpUpdpdj79WExd2C26Bm2R/ij3ORrDEWReCjuF/Hp2E3i7O5znqmU4YbvbECUkXH7WI4GN3L+fTtyLo4GTfaWlQFP3b9lP7T+vffew8svv4xx48ahsLAQsbGxeOaZZ/DKK694u2pEvo9zmSje9XPX8dO7+6FdWQbt0TtxRncXgNsJ0ClBx5Da9gw0o+uj+9OtUSekq2craGnKhYYN9V0+tlQdOu7IRIR9+sj7923vueTkWJ+JWpL0M1EPG6bfZm9ZGXaZuZPsg6HQ0FDMmzcP8+bN83ZViNSHc5kojqgUOLr+JLSLTkO76Q5svNwGt3CfcX9d3MADjfZBc38pBo9vhsQedwK403sVBsxPatmtG9C8uWMTSzoa3Mj5923vueTkODYTtSPBoorIPhgiIi+S8UzHdFvp1VJsfH8fsr4qgfZAIo5XJAFIMu5vFnAKqSknoXm4HnqPa426EZ0tH8xbzE1q6ejEko4GNxcu6F9vaf0ub/6+XR2AOdK6pcKWXgZDpBwKGf3gU2Q00zGZOrXlDLQLTkC7IRg/FbbBTXQ07quDW+gd8Ts0Pa8h9bmmaNE/EZJfUy/W1kmWutDi4/W/u+qjvBwJ3jMzgUceMV+uKm/9vu09lz59gNdes308R4IrFbb0OjS0Pi8vDwkJCe6sj1t4bGgeuY8jQ2XJ9cx9/gkJ5m9IjmCAa7fyG+XY8uF+aDOuQvtbHPaXtTDZH+dXAM1dR5H6YBDun5CC0NhQL9XUDRz5ndgzEeGwYUBiovUuI39/YPny28fylKrnevQoMGuWfrutc7EVNOXm6p/bW1Ym16Esh9aHhISIl156SVy/ft096dxuwtFkCqew0Q8+y5FJ7OyhoOG9DnHh51Sw97z4dOwmMSLuFxGGqyYflR8qRI+wPeL1Adli79eHnZ/40BfZmohQrqPIzNU7MlL/sHQuhteZG5FnbTSZPWU9ycJ1I8uh9Vu2bBGdO3cWMTEx4tNPP3VXnVyOwZCCyX0eEHKOrwa4tQzwKsoqxNaPfhcv98wWHUL21/h4GkqF4vFmm8XySVvE5RNX3HsuSmctKM3IcHzYvrtZuyYAIWbPth5gOzITtdxmrbZy3ch6BurPPvsMM2bMQFRUFN555x30kXnWObvJFMzeGWqzs1U3+kGxdDrrXRQybKq3i5MzGV8+fgXr5h2ENqsSa0+2wkURZbK/Y8gBpHYqhGZMQ3RMT4ZfgOzX15Y/uf1/xVXXhCPdiXLporZx3RR/9hnC09Pl1U1W1Y0bN8TLL78sQkJCxPDhw8XRo0ddF6K5GFuGFEyOf8FR7ci1i6I2HGjBrNRVit3LD4k5/bNFt9C9wg8VJkXDcVWMTNgiljy1WZz7vdDbZ+abbM187ekWZ1+8Juxhx3VTFBfnkfu306PJhBAYMGAArl27hnfffRdr1qzB+PHjMWvWLISG+lDiHnmXN+YBkctfTL5KzhPdOcvOCfL+X+L7mHf2YZytbAmgpXH33cFHoLnnLDR/boCuT6agTkg399dZzeQ2StIXrwl72HPd5Od7pCoOBUOLFi3Cjh07sGPHDhw8eBD+/v5o27Ytxo8fj3vvvRdffPEFUlJSsHLlSnTs2NH2AYls8fQ8Nxy15n5ynujOWXbepHadaYiziEEIStCv8T5o7i+DZtKdSLjPdFZo8gBHh+27ky9eE/aQUXDnUM5QQkICunTpYnx07NgRQUFBJmVef/11ZGRkYN++fS6vrLOYM6Rw9gyVdcX/uOS+erWvMORHKGh4r0125qAsaP4OWj7VE73GtUFQWJDN8uQBcmgJ9sVrwh52XDfFAMIBt9+/nUqgtub8+fOIjY2FztKMnl7AYMgHuGueGwNfTeqVK08FuG6WuykP2gUnsDY7CB9cHIFYnIUfav4vVUCClMDfD1nhI9eEQ+wIAotjYxGen6+8YEgIgU2bNqF3796uPGytMBjyEe78C05uo0t8jbnvbvVq9wa4tt7fid/Oreu38PN/9kO7rAhZvyfg0K3mxn0PIhPfYAQkCEhVX2TpZiaHFgmSF3f/0SdHNoJA2Y8mUxKOJiObOGrNfazNvePqiRwdfX875P9aID4es0mkxW4VoSgyOYw/ykXv8N3ijcHZ4vfMI6Ly62/sm7/FVyecpNrzxDUhN1bmPZL1PENKw5YhsoktQ+7h7TwsJ95fd0uH7YsPIOuzS9Dubow9N1uZ7I/2K8TgZoehGRqA/pNTUL9puOmxq7b4NGqk31ZYaNoixtw092Krm/JY+M48df9mMEQEqDeB0Z28nYflwPtfPHYV6+YfQpYWWHc6GZdFxO1iqETnegeg6XwRmicaof3oVvZNfGhpZOLNm8ClSzbrxN+Zkzgi1Kd46v7NVeuJAPnNO+IL7Jx7B5s3u6e1zc73H3fHf7Go9AkIdDfuaiBdwcCEg0jVCAyc1BINk9s49t6WWqSs1adKndz2mfg6S597fr5+O1vdyALO605kYJh3JC7OdHt8PP8n6gxvTyRn53GvlNaFgB/uCT6Mf3TLwc8Lf0PhjVAsO9UNj33QHQ2To2wfpCqdTt8yUZtGdxnNv6IY1j53w7YpU/TliKphyxBRVWlpwLBhzDdwBS9PJCeiG5uO6rLgsQEX8NacAsR1NJ0V2mm2WqTsERPDvBdHebslkhSNwRBRdf7+/J+lM6onDut0QEQEcPmy+fKunj0cQElhCbIX7Ic2sxRrDzbDRsQjDvnm5/6RJEjx8UjVjndtkFGbVh3DZ3LxYs18J+a9WOftlkhSNAZDRFR75pJWrbEnD8vOlpHjG05Bu/AksnLqIedSG5Shs3HfC3gTyzAaAqgx949k6/2d5WxLl+EzefRRYORI5r04Sq1LWpBLcDQZEdWOpaRVa2xNJGdlRFBZv1RsXrQfWcuvQbuvCY6UJ5m8tKn/GaQmH4fmoRD0ndAaIZvWenYiO1sjEy1JSAD+/W9g6lTOhO4Mjgj1SRxa70IMhojMcEVOiq3h69VFRABffaXvhrT0XhaCK8OzP2MpluHPxu0BKEfP+vug6V6E1GcT0ErTDJJftWwhT+ffWJpV15LZs4EZM/R15HxXzlPjkhY+zlP3b44mI1KjzEx9ENO3LzB6tP7fxET9dkc4mix8+bI+CLHWNTZ5Msz9jSZBv8bXvzAdsdJZ/F+LzVjxwjZcyruJDVfa4W/f90HykOY1AyHgdh7YqFHWAzFXsTQy0RxJAj7+WP/fzHupHY4IJScxZ4hIbVw5F4szN2ULryncfwG7X1yOgWfOWBwF5geBJshD3vpD8Hvgfsff25MMIxPfew/4618tl6s6yol5L7XHEaHkBAZDRGpiay4WSdLPxTJsmH03D2duyn+8prKiEr9+cQjaJYXI2tEQO0uS8QiiMNCOQ/gVnnf8fb3B3x+IjravbEGBPnE6Pt523osLR+DJjiu6NDkilBzEYIhITtyd2+LquVh69rR+865KklDZOBYrVvrj+yd/xprclrggUgCkGIuEBt4CbtlxHkpqGXGktUftM6FzKQ3yEuYMEcmFq/J4rHF1Torh5g3cvlmbIaC/r48seAcj3+2Jz070wAXREKEoxkNxW/HJE5txdvd5fHjjMf3Nz9KxJEk/6kpJLSOGgNHec1Jr3ouh+7Z6sG7ovnXldUBUDUeTEcmBp1Z3z8lxz2glG/MMnUYCpmAeViINKUHHoLn7DDSjwtH96dYIvCOw5rF8bUSQM+ekphmovb2oL8kWh9a7EIMhkjVP3ghcPRfLHzfsgrV7sWWzDv/7rS4KrtdDPvStGtEoxGU0QFDDcAx+oByDxzdDYo9428c1F1y5c24gT/DFc3IVdwXppHhctZ7IHHv/WlbSX9WeXFPJRTkppVdLcXDcu0j85i00KL+AGAAjAPRFBOZhMrb698TglNNIHdkCvce1Rt2Iuo7V0xdHBPniObkKpxQgL2MwRMphb3Kl0pIwPXEjqB4cfvWVfrh39c/ISivF6a350L53HNoNQah3/ji+wDSg2ppfkbiMf2ImXq3/LqRZHwJpvZ2vsy+OCPLFc3IFTilAXsZuMlIGe3NqPJV740ru7iKwFBy+/TbQsKHFVoryG+X45aP90GZcRdbeOOwvawEA8IMOJ5GIOJyxPgJDkuT5eZP8cCkNsoA5Qy7EYEjh7M2pOXYMaN5ceUmY7rwROBgcnvutEGvfPYKstf74IT8FxQg37vODDt3C9uHZu7Lx551WJhGsKiFBfp83yZMvJs5TrXE5DiIDe3NqFi60P/dGTqwNT6/N3DK2JlgEICZPxvaP9uKVXjnoWO8AYu5phLGf9MA3+V1RjHBESReR3uxnLJv4Cy4cK8bmonvw56l2TiIIyPPzNken07fQLVum/1en83aN1EetUwqQLDBniOTP3lyZo0ddezx3qp7DM2yY/n/45rqznB1tZEcQKZ05gxefvoKN6GPc3DHkADQdC5H6REN0+HMr+Af2MH2do3kbcvi8rVFajpkvY5I5eQmDIZI/e2++S5e69njuYu3me/Kk624EdgYhzXEcjeKDkDpIh0GTWyK6jems0DUYJhG0d4FWb3/e1jiyTpuSRigqGZPMyQuYM0TyZyunxl5yyBnyUIL3tbPXsOf5z9Fz+XibZSvWrEfAoH6OvUFmJvDQQ9bLyOHztsaR+Z1Wr2brEZEXMGeIyMDOJR9MuDL3xlXsyOHBlClO5auISoFD2hN4e1gO+kXsQmRcEPosfwZ5iEelpTXg/1gGIqC/HSPZqktLA1asACIjLR8bkPc6Wvbmos2Zw2UiiHwcgyFSBkNyZVSUfeWrl6ttEqYrEmwdmVzRjrrc+ui/2D52ESbevQHNg/KQnNoMz3/bBz9daY9yBKJ5ndNYn/g0JAgIdwSHaWnA+fPA7NlARITpPiUkvdqbyzR/vlsCWCKSD0XkDOXn5+PFF1/EmjVrcPPmTdx111345JNP0KFDB29XjTwpLQ24eRN47DHbZUeNAh580DX5Ha5KsHXB5IqFcz5C3TkzEHrzAgIB3AcgFvHIx3zkIxp9In+HplcJNM81RYv+SQBeBjJbuzYxuyp/f+CVV4AZM5SXT2NvLtPly5b3uXJ2cCLyGtkHQ1euXEH37t3Rt29frFmzBo0aNcLx48dRv359b1eNvKH6sFtL3n0X6N1bHxTVhiMJtrbYe/M9f17f0uDvj1vXb+Hn/+yHdlkRbu09gHkVE1B91ud4nMEKPISyT79A8NjRNY/niRE6Skx6NSSCW5vfqUED68GQgdxHzBGRVbJPoJ42bRq2bNmCzbWYq4QJ1D7EkWTq2k745+oFVB2o+/WQhlgYNBWvXRmHawizPeuz3JOV5crWRH+zZgEzZ9o+DhcQJXILJlD/4dtvv0XHjh3x8MMPo1GjRmjXrh0++ugjq68pKytDcXGxyYN8hCGZ2p4YvrYT/rkqx8fASiJ49bMJuXERf7vyD/TDj2gkXcCc2PeRYG35C7lOKCl3tib6mzFD/9+WEvf/SEJHz57urysRuY3sg6ETJ07ggw8+QIsWLbBu3To8++yzmDRpEj777DOLr5k7dy7Cw8ONj4SEBA/WmNwuLU2ftGqP2nRfuGMB1T9uvrpo0y6z6rdaPwhIEFgWMR4FN+tj2lsNXV8X0ktL08/vlJ0NZGTo/83N1W931+zgRCQrsu8mCwwMRMeOHfHLL78Yt02aNAk7duzA1q1bzb6mrKwMZWVlxufFxcVISEhgN5lcOTOZnbsXN3Xxe1RWVGLPl4ehXXweWdsjseN6K0zA+5gHO9b4ys7W/+vu8yXLzCXRJyS4JgmdiCzyVDeZ7BOoY2JikJJiOhtucnIyVqxYYfE1QUFBCAoKcnfVyBWcHallT/JrfHztui9q+R5Fp4uwfv4BaL+rwJrjd+FcZTKAZOP+4Do6oNyOehQUACNHuv98yTIuE0Hk02QfDHXv3h2HDx822XbkyBE0bdrUSzUil6nNSC1D98WIEfpAwFzya227Lxx8D1EpcPD748j6zxlofwnHz1fboAJdjS+ph+voH7MPmn63MHhiC8SXdADsme8wJsYz50vWKXHEHBHZR8jc//73PxEQECDmzJkjjh49Kr744gsREhIili5davcxioqKBABRVFTkxpqSQyoqhIiPF0J/W6/5kCQhEhL05axZsUKIuDjT18bH67e7yooVNeuakCDEihWi5EKJ+O7l7eK51htFU/+8GqfRMvC4+Gv7bPHjm7+K0qJS85+BJNn/GZirS3y8ELNnC5GRIUR2tu3PjIhIITx1/5Z9zhAAfP/995g+fTqOHj2KpKQkTJ06FX/5y1/sfj2H1suQq/JxPLXieJW8pvwCCSu3NELWplBkX7wbZQg2FgtCKfpG/Y7UPjcw+LlENL/fRgumraHd5lrHquZYHT0KfPQR18wiIp/kqfu3IoKh2mIwJEPLlgGjzUwQWF1GhuWJEz206GlZcRk2L9oP7ZfFyPq9KY6UJ5nsb+p/BqnJx6F5KAR9J7RGSFSIY2/gbHKuh86fiMhbGAy5kMeDIWdGR6lNbVuGXD0hYjVndhRgzXtHof0xED8WtMZ1hBr3BaAcPevvg6Z7ETRPxyN5SHNIfnYuIGuJo78ZN58/EZEccDSZUnmq20bpajsaLCfH/gkR7Uh6rSitwLZPDyDr88vQ7onBb6UtAdyeC6ix33lomh+BZlgA+k1MQXiTdjaP6RBHk3MdmRCSSb9ERFYxGHIlV65j5etqMzoqMxOwN2fM2qKn+y9g7fzD0K71w7ozKbgq2t6uAirR5Y790Nx3CalPNsY9D98Fv4Bo+97TE9wxIaSvY4stEVnAYMhVdDp9i5C5Vg4h9Df4KVP0c5Xwf8B6hqUQHFlR3VLAaUmVxVErKyrx6xeHoF1SCO2OKOwoSYFAD+P+COkyBjU5iNRUYMCkVohqebeTJ+YB9i76am85X8cWWyKygjlDruKJGZF9lb1/sdvKk6nqj262qzl78MN7h6D9rhJrTrREoTBd1qJd3YNI7XAemvRIdH4iBf6BCglUbS36ypyh25hoTqRYzBlSGnZbOM/efBlbeTJ/EAAgBF6+NBn/ah4GHboZ94WiGAPi9kPTvwKDJrZAbHvTWaEVg5Mw2octtkRkBwZDrsJuC/ezM5C8hAg8jY+w8ob+r/2UoGPQ3H0GmlHh6P50awTe0dXGERTCmW5GtWGiORHZgcGQq3hirSy1szOQTMfn8G8Uhff7bsTgcc2Q1OtOAHe6t27ewjWzrGOLLRHZgcGQq7Dbwq1Kr5Zi05YQdKrTEOHlF+GHmgGnAFAaFo3MY71Qt+Ednq+kt3DNLMvYYktEdvDzdgV8iqHbIi7OdHt8vDySNHU6faL3smX6f3U679bHhtNb87Fo9Cb8qfF2RDbQYeBLnfFk+SIAQCVMJzkUkgRJklB38ULvB0IK+5x9mqHFVrIwKaYk6Wf7ZostkaqxZcjV5NptIfehxTodKtZn40jGDmzeURcLTwzCb7daAbgdWMb5FSDqrijsSH4dnba9BxScNe6T5JInI/fPWW3YYktEduDQejVw19BiF0xid+63QhyY+hHa5ryLKF2hcXse4vFXvINzoXchtetlaJ6MQdsRd91e9kKOE+hxCLd8Obv+GxF5FdcmcyFVB0PuWsPKyRYQ3S0ddn5+ENrPLiJrZyM0uXEI32AEAGHSZyv+qJuklACCa4XJnxwDaCKyisGQC6k6GHLHZJAOtoBcPn4FP8w/iKzvK7H2ZCtcFFEAAD/ocBKJiMMZ88lrSgogOOkmEZHLcdJFcg1XDy22YxI7MWUKfrvVClmfFkK7LQJbr7VGZZWJD8NQhIHxB/B/bXciQesjc8BwCDcRkWIxGJIzVzTru3posR2T2El5eZg8qhAb0ce4uU3QUWjuyYdmdH10+0tr1AnpCiw7CWjteE8lBBD2fn7nz+u/V7m3dBERqQiDIbly1agkV08GaWdgkohchEXXReoDZRg8oTmadG0BoIVpIWcDNTnmftj6nA3++lfg3//m6DIiIhnhPENyZMjJqd4Ck5+v356Zaf+xDEOLgZpzrTg4tPjm5ZvY/kORXW/74Xdx+PbcfXjmi15o0jXOfCFn5oDJzNQnKvftC4werf83MdGxz8QdrH3O1TnzPRIRkdswGJIbWzk5gH5hSUcm8qvFZJAnfz6DhY9uxJDo/yEyUqDbkr8gD/E1Jj00+iOACRz8gO16GQIISy0pQpgGaq4MEt3B0udcnbPfIxERuQWDIblxZGFJR6SlASdP6kczZWTo/83NrREI3bp+C9lv78YLnXKQEnQcST3jMf7L3sgq7IybCEGs/3lkxT0DCQKili1NDnFHkOgOhs/5nXesl3P2eyQiIpdjzpA7OZPb4s5RSRbWsCrYcx5r5h9B1g91sP5sCq6h3e2XoALdw/dB0/UqNE/Fos2DLSD5vQRkptR+tXRDgGOJJOkDHMOM3kpZfdzfH4iOtq+sEpLDiYh8HIMhd3E2AdoDC0vqbunwvyUHkPXZJWh3RWP3zWQAt2/ejaQLGNzsMDRD/TBgSmvUb3pvzYO4YtkRRwIcpQ1d5wKhRESKwWDIHSxNSmjIbbGWp+Pq0V9/uHj4EtbNPwStFlh7OhmXxd23D4lKdKp3AJpOF5E6thHaj24Fv4Aetg9a29XS7Q1cVq/WB172kEtw4abvkYiIXI/BkKvZMSmhsevHXCuKixaWFJUCu5cdgnbxeWi3R2Lb9dYQ6G7cX1+6ikEJB6AZXImBE1uiUes2jp9rbdkbuMybB3TrpqzggguEEhEpBpfjcDVXLcvgxMKSRaeL8ON7B6BdXYE1x1ugoLKxyf62wYeR2q4Amsci0OX/UhAQ7OVY2LCel625eQD9ub/9NjBypP65ueBCjuuYcYFQIiKncTkOpXJVbosdOTmiUuDg98eh/fAMsraE4+erbVCBrsb99XAd/WP2QdPvFgZPbIH4Ti0BtHTipNzE0Hry0EO2y+blAVFR+oCntonbnuSK3CoiInIrBkOu5srEWTM5OTcu3kD2gv3I+uYmtIea4ZTuTgB3Gve3DDwBTZvT0DwShp7PtkZQWBf76+4NaWn6bsN582yXLSgARo3SBxc5OfoHoP+MvD2CzJra5lYREZFbsZvM1Wx1/TixEvuJnNPIWpAL7cYQZF+8G2UINu4LQin6Rv0OTe8b0IxLRPP7m7roRDzI0a5FVy1V4k1yXFKEiEhmPHX/ZjDkDobRZIBTuS1lxWXYvGg/tF8WQ7uvCQ7famayv4n/GaS2OoHUEXXRd0JrhESFuPoMPMue3KHISP0ip6tXmx+pJ+e8oep8IZgjIvIABkMu5PFgCHA4cfbMjgKsee8otD8G4seC1riOUOO+AJSjR/19SO1eBM3T8Uge0hySn431rzzJFa0cmZm2c4e++gqYOtXy3EROtLp5nKVpF5QUzBEReQiDIRdyyYfpzA3fymsqSiuw7dMD0C69DO3uGOwtNU1sbux3HprmR6AZFoB+E1MQ3iTcuXq7m6taOXQ6/azNly6Z3y9J+gTqCxdsH8vWSD1vMbSAKTmYIyLyII4mkxNnb/jVEmcvHLyItfMPQbvGD+vyknFFtDXuk1CJLnfsh+a+S9CMjca9j7SEX4CdSzp4S20ml6xu82bLgRCgfw97AiFAPrNQV6ekJUWIiFSEwZAttbjhV1ZUYlfGIWiXFCLrf1HYUZICgdszO0dIlzGoyUFoNMDAya0Q1fJus8eRpdpOLlmdKwMYucxCXZ3SlhQhIlIJBkPWOHHDv3qqCD/M2w/td5VYc6IlCkUKgBTjy9rVPQhN+/PQpEfivrEp8A/sXvPYSuDqVg57A5iGDYGLF5UxC3V1XK+MiEiWGAxZY+cNP/f1Zfjql3hot9bHlqI20KGbsUgoitE/9gBSB5Rj0MQWiG2fDCC59nXz9tBse1sv8vPtK2fvWl6GWaiVuMQF1ysjIpIlBkPW2HnD/8cr/liOPsbnyYHHobk7D6mjw9H96dYIvMPFEx/KYWi2va0XU6YAdevarpe1tbwA/fO3377dNamkWagNuF4ZEZEscTSZNXZOBjgAa1GnYQRS77+BweOaIalXgvOVtUUuQ7MdWVdMkuyvl7lAz6BqwOftlrHa4HplRER24dB6F3Lmwyy9WopN7/+GTrOHILz8IvxQ82MSAErDooFjx1C34R0urrUZchuabWlyydrW6+uvby/IWv04gG/MxaPkYI6IyEM8FQz5ue3IbjJ37lxIkoQpU6a4/Nint+bjP3/ehGEx2xHZQIeBL3XGk+WLAACVMJ3kUEgSJElC3cULPRMIAY4lLXtCWpo+MImKsl7OkXrpdMBf/2r5OIC+602nc6iqsmOYdmHUKP2/DISIiLxGUTlDO3bswIcffoi2bdvaLmyH8hvl2PrJAWQtvQLt3jjsK2sBIM64P9avAJEtorAj+XV02v4eUHDWuE/yRo6KHIdmp6UBN28Cjz1mu6w99Zozx3rSNefiISIiF1NMMHT9+nX8+c9/xkcffYTXXnvN6eOc33cBa+YdhnadP344k4Ii3GPc5wcduobuh6bLZaQ+FYO2I+6C5BcDoBege8H73RpyHZodF2e7DGC7XpmZwMyZ9h2Lc/EQEZGLKCYYGj9+PFJTU9GvXz+bwVBZWRnKysqMz4uLiwEAfRofwu6bnQE0NO6Lki5iUOIhpA7xw4DJyYhobqHVqdps0l4h16HZrqiXYU4ne3EuHiIichFFBEPLly/Hrl27sGPHDrvKz507F7Nnz66xfffNVgCADiEHkNqxEJrHo9AxPRn+gT1qlJUluQ7NdkW9bOVDVX+/ixedri4REVFVsk+gzsvLw+TJk7F06VIEBwfb9Zrp06ejqKjI+MjLywMALEzfioK9hdhZkoLZG/vgvifbwD9QYYmrhqTl6l1T8fHeHWVV23o50u2l0+lHm2VmOl5PIiKiamQ/tH7VqlV48MEH4V+lVUGn00GSJPj5+aGsrMxknzmeGprnUXIdmu1sveyc08mIK7wTEfk8zjP0h2vXruHUqVMm28aOHYtWrVrhxRdfRJs2bWwewyeDIV/jyCSOVWVnez+Xi4iI3MJT92/Z5wyFhobWCHjq1auHyMhIuwIhUghby3FYwlFlRERUS7LPGSIVsZR3ZA1HlRERUS3JvpvMFdhNpjA6nT6HaORI4PJl82WYM0RE5PO4HAepl78/8MADwEcf6YMeyXQpFK7wTkRErsRgiORLrtMIEBGRT5F9AjWpXFoaMGyYPKcRICIin8BgiORPDkuhEBGRz2I3GREREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI2TLrqLTsdZk4mIiBSAwZA7ZGYCkycDZ87c3hYfD8yfz/W0iIiIZIbdZK6WmQmMGGEaCAFAfr5+e2amd+pFREREZjEYciWdTt8iJETNfYZtU6boyxEREZEsMBhypc2ba7YIVSUEkJenL0dERESywGDIlQoKXFuOiIiI3I7BkCvFxLi2HBEREbkdgyFX6tlTP2pMkszvlyQgIUFfjoiIiGSBwZAr+fvrh88DNQMiw/N58zjfEBERkYwwGHK1tDTgm2+AuDjT7fHx+u2cZ4iIiEhWOOmiO6SlAcOGcQZqIiIiBWAwZODq5TP8/YE+fVxWPSIiInIPBkMAl88gIiJSMeYMcfkMIiIiVVN3MMTlM4iIiFRP3cEQl88gIiJSPXUHQ1w+g4iISPXUHQxx+QwiIiLVU3cwxOUziIiIVE/dwRCXzyAiIlI9dQdDAJfPICIiUjlOughw+QwiIiIVYzBkwOUziIiIVIndZERERKRqDIaIiIhI1RgMERERkarJPhiaO3cuOnXqhNDQUDRq1AjDhw/H4cOHvV0tIiIi8hGyD4Y2btyI8ePHY9u2bVi/fj0qKiowYMAAlJSUeLtqRERE5AMkIcwt2S5fFy5cQKNGjbBx40b06tXLrtcUFxcjPDwcRUVFCAsLc3MNiYiIyBU8df9W3ND6oqIiAEBERITFMmVlZSgrKzM+Ly4udnu9iIiISJlk301WlRACU6dORY8ePdCmTRuL5ebOnYvw8HDjIyEhwYO1JCIiIiVRVDfZ+PHjkZWVhZ9//hnx8fEWy5lrGUpISGA3GRERkYKwm6yaiRMn4ttvv8WmTZusBkIAEBQUhKCgIA/VjIiIiJRM9sGQEAITJ07EypUrkZOTg6SkJG9XiYiIiHyI7IOh8ePHIyMjA6tXr0ZoaCjOnTsHAAgPD0fdunW9XDsiIiJSOtnnDEmSZHb74sWL8cQTT9h1DA6tJyIiUh7mDP1B5rEaERERKZyihtYTERERuRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVVNMMLRw4UIkJSUhODgYHTp0wObNm71dJSIiIvIBigiGvvzyS0yZMgUzZszA7t270bNnTwwePBinT5/2dtWIiIhI4SQhhPB2JWy577770L59e3zwwQfGbcnJyRg+fDjmzp1r8/XFxcUIDw9HUVERwsLC3FlVIiIichFP3b9l3zJ069Yt/PrrrxgwYIDJ9gEDBuCXX37xUq2IiIjIVwR4uwK2XLx4ETqdDtHR0Sbbo6Ojce7cObOvKSsrQ1lZmfF5UVERAH2ESURERMpguG+7uxNL9sGQgSRJJs+FEDW2GcydOxezZ8+usT0hIcEtdSMiIiL3uXTpEsLDw912fNkHQ1FRUfD396/RClRYWFijtchg+vTpmDp1qvH51atX0bRpU5w+fdqtH6bcFBcXIyEhAXl5earKleJ587zVgOfN81aDoqIiNGnSBBEREW59H9kHQ4GBgejQoQPWr1+PBx980Lh9/fr1GDZsmNnXBAUFISgoqMb28PBwVf2IDMLCwnjeKsLzVheet7qo9bz9/Nyb4iz7YAgApk6divT0dHTs2BFdu3bFhx9+iNOnT+PZZ5/1dtWIiIhI4RQRDD3yyCO4dOkSXn31VRQUFKBNmzbQarVo2rSpt6tGRERECqeIYAgAxo0bh3Hjxjn12qCgIMycOdNs15kv43nzvNWA583zVgOet3vPWxGTLhIRERG5i+wnXSQiIiJyJwZDREREpGoMhoiIiEjVGAwRERGRqikyGFq4cCGSkpIQHByMDh06YPPmzVbLb9y4ER06dEBwcDCaNWuGRYsW1SizYsUKpKSkICgoCCkpKVi5cqW7qu80R847MzMT/fv3R8OGDREWFoauXbti3bp1JmWWLFkCSZJqPEpLS919Kg5x5LxzcnLMntOhQ4dMyvna9/3EE0+YPe/WrVsbyyjh+960aROGDh2K2NhYSJKEVatW2XyNL1zfjp63r1zfjp63r1zfjp63r1zfc+fORadOnRAaGopGjRph+PDhOHz4sM3XeeIaV1ww9OWXX2LKlCmYMWMGdu/ejZ49e2Lw4ME4ffq02fK5ubnQaDTo2bMndu/ejX/84x+YNGkSVqxYYSyzdetWPPLII0hPT8fevXuRnp6OkSNHYvv27Z46LZscPe9Nmzahf//+0Gq1+PXXX9G3b18MHToUu3fvNikXFhaGgoICk0dwcLAnTskujp63weHDh03OqUWLFsZ9vvh9z58/3+R88/LyEBERgYcfftiknNy/75KSEtxzzz1YsGCBXeV95fp29Lx95fp29LwNlH59O3revnJ9b9y4EePHj8e2bduwfv16VFRUYMCAASgpKbH4Go9d40JhOnfuLJ599lmTba1atRLTpk0zW/7vf/+7aNWqlcm2Z555RnTp0sX4fOTIkWLQoEEmZQYOHCgeffRRF9W69hw9b3NSUlLE7Nmzjc8XL14swsPDXVVFt3D0vLOzswUAceXKFYvHVMP3vXLlSiFJkjh58qRxmxK+76oAiJUrV1ot4yvXd1X2nLc5Sry+q7LnvH3l+q7Kme/bF65vIYQoLCwUAMTGjRstlvHUNa6olqFbt27h119/xYABA0y2DxgwAL/88ovZ12zdurVG+YEDB2Lnzp0oLy+3WsbSMT3NmfOurrKyEteuXaux2N3169fRtGlTxMfHY8iQITX+svSm2px3u3btEBMTgwceeADZ2dkm+9TwfX/yySfo169fjVna5fx9O8MXrm9XUOL1XRtKvr5dwVeu76KiIgCwugirp65xRQVDFy9ehE6nq7FafXR0dI1V7Q3OnTtntnxFRQUuXrxotYylY3qaM+dd3b///W+UlJRg5MiRxm2tWrXCkiVL8O2332LZsmUIDg5G9+7dcfToUZfW31nOnHdMTAw+/PBDrFixApmZmWjZsiUeeOABbNq0yVjG17/vgoICrFmzBk899ZTJdrl/387whevbFZR4fTvDF67v2vKV61sIgalTp6JHjx5o06aNxXKeusYVsxxHVZIkmTwXQtTYZqt89e2OHtMbnK3jsmXLMGvWLKxevRqNGjUybu/SpQu6dOlifN69e3e0b98e7733Ht59913XVbyWHDnvli1bomXLlsbnXbt2RV5eHt566y306tXLqWN6i7N1XLJkCerXr4/hw4ebbFfK9+0oX7m+naX069sRvnR9O8tXru8JEybgt99+w88//2yzrCeucUW1DEVFRcHf379GtFdYWFgjKjRo3Lix2fIBAQGIjIy0WsbSMT3NmfM2+PLLL/Hkk0/iq6++Qr9+/ayW9fPzQ6dOnWTzl0RtzruqLl26mJyTL3/fQgh8+umnSE9PR2BgoNWycvu+neEL13dtKPn6dhWlXd+14SvX98SJE/Htt98iOzsb8fHxVst66hpXVDAUGBiIDh06YP369Sbb169fj27dupl9TdeuXWuU/+GHH9CxY0fUqVPHahlLx/Q0Z84b0P/F+MQTTyAjIwOpqak230cIgT179iAmJqbWdXYFZ8+7ut27d5uck69+34B+tMaxY8fw5JNP2nwfuX3fzvCF69tZSr++XUVp13dtKP36FkJgwoQJyMzMxIYNG5CUlGTzNR67xu1OtZaJ5cuXizp16ohPPvlEHDhwQEyZMkXUq1fPmFU/bdo0kZ6ebix/4sQJERISIv7617+KAwcOiE8++UTUqVNHfPPNN8YyW7ZsEf7+/uJf//qXOHjwoPjXv/4lAgICxLZt2zx+fpY4et4ZGRkiICBAvP/++6KgoMD4uHr1qrHMrFmzxNq1a8Xx48fF7t27xdixY0VAQIDYvn27x8/PEkfP+5133hErV64UR44cEfv27RPTpk0TAMSKFSuMZXzx+zZ47LHHxH333Wf2mEr4vq9duyZ2794tdu/eLQCIt99+W+zevVucOnVKCOG717ej5+0r17ej5+0r17ej522g9Ov7ueeeE+Hh4SInJ8fkd3vjxg1jGW9d44oLhoQQ4v333xdNmzYVgYGBon379ibD8saMGSN69+5tUj4nJ0e0a9dOBAYGisTERPHBBx/UOObXX38tWrZsKerUqSNatWplcnHJhSPn3bt3bwGgxmPMmDHGMlOmTBFNmjQRgYGBomHDhmLAgAHil19+8eAZ2ceR837jjTdE8+bNRXBwsGjQoIHo0aOHyMrKqnFMX/u+hRDi6tWrom7duuLDDz80ezwlfN+GodOWfre+en07et6+cn07et6+cn078zv3hevb3DkDEIsXLzaW8dY1Lv1RQSIiIiJVUlTOEBEREZGrMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRESKtGzZMgQHByM/P9+47amnnkLbtm1RVFTkxZoRkdJwbTIiUiQhBO6991707NkTCxYswOzZs/Hxxx9j27ZtiIuL83b1iEhBArxdASIiZ0iShDlz5mDEiBGIjY3F/PnzsXnzZgZCROQwtgwRkaK1b98e+/fvxw8//IDevXt7uzpEpEDMGSIixVq3bh0OHToEnU6H6Ohob1eHiBSKLUNEpEi7du1Cnz598P7772P58uUICQnB119/7e1qEZECMWeIiBTn5MmTSE1NxbRp05Ceno6UlBR06tQJv/76Kzp06ODt6hGRwrBliIgU5fLly+jevTt69eqF//znP8btw4YNQ1lZGdauXevF2hGREjEYIiIiIlVjAjURERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1f4/bwxn9/OjnzsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_5_1.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "# Importing various packages\n",
    "from math import exp, sqrt\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.inv(X.T @ X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = 2.0/n*X.T @ ((X @ theta)-y)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "# Can you figure out a better way of setting up the contributions to each batch?\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (2.0/M)* xi.T @ ((xi @ theta)-yi)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        theta = theta - eta*gradients\n",
    "print(\"theta from own sdg\")\n",
    "print(theta)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8e7110b",
   "metadata": {
    "editable": true
   },
   "source": [
    "In the above code, we have use replacement in setting up the\n",
    "mini-batches. The discussion\n",
    "[here](https://sebastianraschka.com/faq/docs/sgd-methods.html) may be\n",
    "useful."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74a7859a",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Momentum based GD\n",
    "\n",
    "The stochastic gradient descent (SGD) is almost always used with a\n",
    "*momentum* or inertia term that serves as a memory of the direction we\n",
    "are moving in parameter space.  This is typically implemented as\n",
    "follows"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "619df8e8",
   "metadata": {
    "editable": true
   },
   "source": [
    "$$\n",
    "\\mathbf{v}_{t}=\\gamma \\mathbf{v}_{t-1}+\\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t) \\nonumber\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c759cfc4",
   "metadata": {
    "editable": true
   },
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"_auto1\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation} \n",
    "\\boldsymbol{\\theta}_{t+1}= \\boldsymbol{\\theta}_t -\\mathbf{v}_{t},\n",
    "\\label{_auto1} \\tag{1}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a4571f9",
   "metadata": {
    "editable": true
   },
   "source": [
    "where we have introduced a momentum parameter $\\gamma$, with\n",
    "$0\\le\\gamma\\le 1$, and for brevity we dropped the explicit notation to\n",
    "indicate the gradient is to be taken over a different mini-batch at\n",
    "each step. We call this algorithm gradient descent with momentum\n",
    "(GDM). From these equations, it is clear that $\\mathbf{v}_t$ is a\n",
    "running average of recently encountered gradients and\n",
    "$(1-\\gamma)^{-1}$ sets the characteristic time scale for the memory\n",
    "used in the averaging procedure. Consistent with this, when\n",
    "$\\gamma=0$, this just reduces down to ordinary SGD as discussed\n",
    "earlier. An equivalent way of writing the updates is"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bebfdd9",
   "metadata": {
    "editable": true
   },
   "source": [
    "$$\n",
    "\\Delta \\boldsymbol{\\theta}_{t+1} = \\gamma \\Delta \\boldsymbol{\\theta}_t -\\ \\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t),\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9e635e8",
   "metadata": {
    "editable": true
   },
   "source": [
    "where we have defined $\\Delta \\boldsymbol{\\theta}_{t}= \\boldsymbol{\\theta}_t-\\boldsymbol{\\theta}_{t-1}$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb544f8b",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Algorithms and codes for Adagrad, RMSprop and Adam\n",
    "\n",
    "The algorithms we have implemented are well described in the text by [Goodfellow, Bengio and Courville, chapter 8](https://www.deeplearningbook.org/contents/optimization.html).\n",
    "\n",
    "The codes which implement these algorithms are discussed after our presentation of automatic differentiation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d088ad85",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Practical tips\n",
    "\n",
    "* **Randomize the data when making mini-batches**. It is always important to randomly shuffle the data when forming mini-batches. Otherwise, the gradient descent method can fit spurious correlations resulting from the order in which data is presented.\n",
    "\n",
    "* **Transform your inputs**. Learning becomes difficult when our landscape has a mixture of steep and flat directions. One simple trick for minimizing these situations is to standardize the data by subtracting the mean and normalizing the variance of input variables. Whenever possible, also decorrelate the inputs. To understand why this is helpful, consider the case of linear regression. It is easy to show that for the squared error cost function, the Hessian of the cost function is just the correlation matrix between the inputs. Thus, by standardizing the inputs, we are ensuring that the landscape looks homogeneous in all directions in parameter space. Since most deep networks can be viewed as linear transformations followed by a non-linearity at each layer, we expect this intuition to hold beyond the linear case.\n",
    "\n",
    "* **Monitor the out-of-sample performance.** Always monitor the performance of your model on a validation set (a small portion of the training data that is held out of the training process to serve as a proxy for the test set. If the validation error starts increasing, then the model is beginning to overfit. Terminate the learning process. This *early stopping* significantly improves performance in many settings.\n",
    "\n",
    "* **Adaptive optimization methods don't always have good generalization.** Recent studies have shown that adaptive methods such as ADAM, RMSPorp, and AdaGrad tend to have poor generalization compared to SGD or SGD with momentum, particularly in the high-dimensional limit (i.e. the number of parameters exceeds the number of data points). Although it is not clear at this stage why these methods perform so well in training deep neural networks, simpler procedures like properly-tuned SGD may work as well or better in these applications.\n",
    "\n",
    "Geron's text, see chapter 11, has several interesting discussions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "937063d3",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Using Automatic differentation with OLS\n",
    "\n",
    "We conclude the part on optmization by showing how we can make codes\n",
    "for linear regression and logistic regression using **autograd**. The\n",
    "first example shows results with ordinary leats squares."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e08215a6",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.32133765]\n",
      " [2.59905073]]\n",
      "Eigenvalues of Hessian Matrix:[0.29426584 4.28858038]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own gd\n",
      "[[4.32133765]\n",
      " [2.59905073]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHFCAYAAADxOP3DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABSoUlEQVR4nO3deXxTVfo/8E+6pSxtoUD3UgoCLSD7VrDQBsGVAQuiMBZBUWfABVEHGJ0BZgYZ+Dmu6NdBERQFNwruC5IWkEVBQEcpZStQllLWthQa2uT8/ogNpE2aPffe3M/79eoLcnNye26W3ifnPuc5GiGEABEREZFKBUndASIiIiIpMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgikqHly5dDo9FYfkJCQhAfH4+7774b+/fvl6xfc+fOhUajkez3K8Hhw4eh0Wjw3HPPSd0VInJSiNQdICL7li1bhrS0NFRXV2Pz5s2YP38+8vPzsXfvXrRs2VLq7hERBQQGQ0Qy1q1bN/Tt2xcAkJWVBaPRiDlz5mDt2rWYPHmyxL0jKdXU1FhGDYnIM7xMRqQgdYHRqVOnLNuqq6vxxBNPoGfPnoiKikJ0dDQyMjLwySefNHi8RqPBww8/jBUrViA9PR1NmzZFjx498Pnnnzdo+8UXX6Bnz57QarVITU21e9mnuroas2fPRmpqKsLCwpCYmIhp06bhwoULVu3atWuH22+/HZ9//jl69eqFJk2aID093fK7ly9fjvT0dDRr1gz9+/fHjh07HD4fdZcT8/Pz8ec//xmtW7dGq1atkJOTgxMnTjQ49rlz5zbYR7t27TBp0qQG+9Tr9XjggQfQqlUrREZGYuLEiaiqqkJpaSnGjRuHFi1aID4+Hk8++SRqamoa7NdkMmH+/Plo27YtwsPD0bdvX6xfv75Bu/3792PChAmIiYmBVqtFeno6Xn31Vas2BQUF0Gg0WLFiBZ544gkkJiZCq9XiwIEDuHTpEp588kmkpqYiPDwc0dHR6Nu3L1atWuXw+SMiM36lIFKQ4uJiAECnTp0s2wwGA86dO4cnn3wSiYmJuHLlCr777jvk5ORg2bJlmDhxotU+vvjiC2zfvh3/+Mc/0Lx5cyxatAh33HEHioqK0L59ewDA+vXrMWrUKGRkZOD999+H0WjEokWLrIIwABBCYPTo0Vi/fj1mz56NzMxM/PLLL5gzZw62bt2KrVu3QqvVWtr//PPPmD17Np5++mlERUVh3rx5yMnJwezZs7F+/Xo8++yz0Gg0mDlzJm6//XYUFxejSZMmDp+XKVOm4LbbbsPKlStRUlKCp556Cvfccw/0er3bz/WUKVOQk5OD999/H7t27cJf//pX1NbWoqioCDk5OXjwwQfx3XffYeHChUhISMCMGTOsHr948WKkpKTgxRdfhMlkwqJFi3DLLbdgw4YNyMjIAADs2bMHgwYNQtu2bfGf//wHcXFx+Oabb/Doo4/izJkzmDNnjtU+Z8+ejYyMDLz++usICgpCTEwMZsyYgRUrVuBf//oXevXqhaqqKvz66684e/as28dOpDqCiGRn2bJlAoDYtm2bqKmpEZWVleLrr78WcXFxYsiQIaKmpsbuY2tra0VNTY24//77Ra9evazuAyBiY2NFRUWFZVtpaakICgoSCxYssGwbMGCASEhIEJcvX7Zsq6ioENHR0eLaPxtff/21ACAWLVpk9Xs++OADAUAsWbLEsi0lJUU0adJEHDt2zLJt9+7dAoCIj48XVVVVlu1r164VAMSnn37q1PM0depUq+2LFi0SAMTJkyetjn3OnDkN9pGSkiLuvffeBvt85JFHrNqNHj1aABDPP/+81faePXuK3r17W24XFxcLAHafvxtvvNGy7aabbhJJSUmivLzcap8PP/ywCA8PF+fOnRNCCJGfny8AiCFDhjTof7du3cTo0aMbbCci5/EyGZGMDRw4EKGhoYiIiMDNN9+Mli1b4pNPPmmQJ/LRRx9h8ODBaN68OUJCQhAaGoqlS5eisLCwwT6zs7MRERFhuR0bG4uYmBgcOXIEAFBVVYXt27cjJycH4eHhlnYREREYOXKk1b7qRl6uvcwEAHfeeSeaNWvW4LJQz549kZiYaLmdnp4OwJwP1bRp0wbb6/rkyB/+8Aer2927d3fp8bbcfvvtVrfr+nTbbbc12G7r99h7/jZu3Aij0Yjq6mqsX78ed9xxB5o2bYra2lrLz6233orq6mps27bNap9jxoxp8Hv69++Pr776CrNmzUJBQQEuX77s9jETqRWDISIZe+edd7B9+3bo9Xo89NBDKCwsxPjx463a5OXlYdy4cUhMTMS7776LrVu3Yvv27bjvvvtQXV3dYJ+tWrVqsE2r1VpOoufPn4fJZEJcXFyDdvW3nT17FiEhIWjTpo3Vdo1Gg7i4uAaXaqKjo61uh4WFNbrdVv9tqX9MdZfmPAkMXOmrrX7ae/6uXLmCixcv4uzZs6itrcUrr7yC0NBQq59bb70VAHDmzBmrx8fHxzfY58svv4yZM2di7dq1yM7ORnR0NEaPHi1pCQYipWHOEJGMpaenW5Kms7OzYTQa8eabb+Ljjz/G2LFjAQDvvvsuUlNT8cEHH1jVADIYDG79zpYtW0Kj0aC0tLTBffW3tWrVCrW1tTh9+rRVQCSEQGlpKfr16+dWH3xBq9XafE58lVtj7/kLCwtD8+bNERoaiuDgYOTm5mLatGk295Gammp121aNp2bNmmHevHmYN28eTp06ZRklGjlyJPbu3eudgyEKcBwZIlKQRYsWoWXLlvj73/8Ok8kEwHyCDAsLszpRlpaW2pxN5oy62Vx5eXlWIx6VlZX47LPPrNoOGzYMgDkgu9bq1atRVVVluV8O2rVrh19++cVqm16vx8WLF33y++w9f5mZmQgODkbTpk2RnZ2NXbt2oXv37ujbt2+DH1ujeI2JjY3FpEmTMH78eBQVFeHSpUvePiyigMSRISIFadmyJWbPno2//OUvWLlyJe655x7cfvvtyMvLw9SpUzF27FiUlJTgn//8J+Lj492+VPLPf/4TN998M4YPH44nnngCRqMRCxcuRLNmzXDu3DlLu+HDh+Omm27CzJkzUVFRgcGDB1tmk/Xq1Qu5ubneOnSP5ebm4m9/+xv+/ve/Y+jQodizZw8WL16MqKgon/y+4OBgDB8+HDNmzIDJZMLChQtRUVGBefPmWdq89NJLuOGGG5CZmYk///nPaNeuHSorK3HgwAF89tlnTs2GGzBgAG6//XZ0794dLVu2RGFhIVasWIGMjAyrPCwiso/BEJHCPPLII1i8eDH+8Y9/YPz48Zg8eTLKysrw+uuv46233kL79u0xa9YsHDt2zOrE64rhw4dj7dq1eOaZZ3DXXXchLi4OU6dOxeXLl632qdFosHbtWsydOxfLli3D/Pnz0bp1a+Tm5uLZZ5+1mlYvtaeeegoVFRVYvnw5nnvuOfTv3x8ffvghRo0a5ZPf9/DDD6O6uhqPPvooysrK0LVrV3zxxRcYPHiwpU2XLl2wc+dO/POf/8QzzzyDsrIytGjRAh07drTkDTmi0+nw6aef4oUXXsClS5eQmJiIiRMn4umnn/bJcREFIo0QQkjdCSIiIiKpMGeIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqqmizpDJZMKJEycQERFhs5w9ERERyY8QApWVlUhISEBQkO/Gb1QRDJ04cQLJyclSd4OIiIjcUFJSgqSkJJ/tXxXBUEREBADzkxkZGSlxb4iIiMgZFRUVSE5OtpzHfUUVwVDdpbHIyEgGQ0RERArj6xQXJlATERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqkkeDG3cuBEjR45EQkICNBoN1q5da7ftQw89BI1GgxdffNFv/SMiIqLAJnkwVFVVhR49emDx4sWNtlu7di1++OEHJCQk+KlnREREpAYhUnfglltuwS233NJom+PHj+Phhx/GN998g9tuu81PPSMiIiI1kHxkyBGTyYTc3Fw89dRT6Nq1q9TdISIiogAj+ciQIwsXLkRISAgeffRRpx9jMBhgMBgstysqKnzRNSIiIgoAsh4Z+umnn/DSSy9h+fLl0Gg0Tj9uwYIFiIqKsvwkJyf7sJdERESkZLIOhjZt2oSysjK0bdsWISEhCAkJwZEjR/DEE0+gXbt2dh83e/ZslJeXW35KSkr812kiIiJSFFlfJsvNzcWNN95ote2mm25Cbm4uJk+ebPdxWq0WWq3W190jIiKiACB5MHTx4kUcOHDAcru4uBi7d+9GdHQ02rZti1atWlm1Dw0NRVxcHDp37uzvrhIREVEAkjwY2rFjB7Kzsy23Z8yYAQC49957sXz5col6RURERGoheTCUlZUFIYTT7Q8fPuy7zhAREZHqyDqBmoiIiMjXGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNcmDoY0bN2LkyJFISEiARqPB2rVrLffV1NRg5syZuP7669GsWTMkJCRg4sSJOHHihHQdJiIiooAieTBUVVWFHj16YPHixQ3uu3TpEnbu3Im//e1v2LlzJ/Ly8rBv3z784Q9/kKCnREREFIg0QgghdSfqaDQarFmzBqNHj7bbZvv27ejfvz+OHDmCtm3bOrXfiooKREVFoby8HJGRkV7qLREREfmSv87fIT7bs4+Ul5dDo9GgRYsWdtsYDAYYDAbL7YqKCj/0jIiIiJRI8stkrqiursasWbMwYcKERiPEBQsWICoqyvKTnJzsx14SERGRkigmGKqpqcHdd98Nk8mE1157rdG2s2fPRnl5ueWnpKTET70kIiIipVHEZbKamhqMGzcOxcXF0Ov1Dq8barVaaLVaP/WOiIiIlEz2wVBdILR//37k5+ejVatWUneJiIiIAojkwdDFixdx4MABy+3i4mLs3r0b0dHRSEhIwNixY7Fz5058/vnnMBqNKC0tBQBER0cjLCxMqm4TERFRgJB8an1BQQGys7MbbL/33nsxd+5cpKam2nxcfn4+srKynPodnFpPRESkPKqZWp+VlYXG4jEZlUEiIiKiAKSY2WREREREvsBgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjXJF2olIiIiLzAagU2bgJMngfh4IDMTCA6WuleKwGCIiIhI6fLygMceA44du7otKQl46SUgJ0e6fikEL5MREREpWV4eMHasdSAEAMePm7fn5UnTLwVhMERERKRURqN5REiIhvfVbZs+3dyO7GIwREREpFSbNjUcEbqWEEBJibkd2cVgiIiISKlOnvRuO5ViMERERKRU8fHebadSDIaIiIiUKjPTPGtMo7F9v0YDJCeb25FdDIaIiIiUKjjYPH0eaBgQ1d1+8UXWG3KAwRAREZGS5eQAH38MJCZab09KMm9nnSGHWHSRiIhI6XJygFGjWIHaTQyGiIiI/MlXy2YEBwNZWZ7vR4UYDBEREfkLl82QJeYMERER+QOXzZAtBkNERES+xmUzZI3BEBERka9x2QxZYzBERETka1w2Q9YYDBEREfkal82QNQZDREREvsZlM2SNwRAREZGvNbZsBmDOGeKyGZJhMEREROQPdctmREc3vK9VK//3hywYDBEREfnTuXO2t7HWkGQYDBEREfkDaw3JFoMhIiIif2CtIdliMEREROQPrDUkWwyGiIiI/IG1hmRL8mBo48aNGDlyJBISEqDRaLB27Vqr+4UQmDt3LhISEtCkSRNkZWXht99+k6azRERE7mKtIdmSPBiqqqpCjx49sHjxYpv3L1q0CM8//zwWL16M7du3Iy4uDsOHD0dlZaWfe0pEROSBxmoN1d1mrSFJaISwldYuDY1GgzVr1mD06NEAzKNCCQkJmD59OmbOnAkAMBgMiI2NxcKFC/HQQw85td+KigpERUWhvLwckZGRvuo+ERGRY3l55lll1yZTJyebA6GcHMm6JUf+On+H+GzPXlBcXIzS0lKMGDHCsk2r1WLo0KHYsmWL3WDIYDDAYDBYbldUVPi8r0RERE7JyQFGjTLPGjt50pwjlJnJESEJyToYKi0tBQDExsZabY+NjcWRI0fsPm7BggWYN2+eT/tGRETktuBgICtL6l7Q7yTPGXKGpt61VSFEg23Xmj17NsrLyy0/JSUlvu4iERGRuhmNQEEBsGqV+V8FFY+U9chQXFwcAPMIUfw1Uw3LysoajBZdS6vVQqvV+rx/REREBNt5UElJ5oRxBeRByXpkKDU1FXFxcVi3bp1l25UrV7BhwwYMGjRIwp4RERHJlL9HaPLyzOuq1a+uffy4YtZbk3xk6OLFizhw4IDldnFxMXbv3o3o6Gi0bdsW06dPx7PPPouOHTuiY8eOePbZZ9G0aVNMmDBBwl4TEZEiGI3qSlT29wiNo/XWNBrzemujRsn6eZc8GNqxYweys7Mtt2fMmAEAuPfee7F8+XL85S9/weXLlzF16lScP38eAwYMwLfffouIiAipukxEREqg8Es3LqsboakfmNSN0Hz8sfeP25X11mScMC6rOkO+wjpDREQqYy8wqJt844vAQEpGI9Cunf3ARKMxB4LFxd4doVm1CnDmSs3KlcD48S7v3l/nb1nnDBEREbnM0aUbwHzpRkGznRxyZYTGmwJkvTUGQ0REFFikCgyk5OxK9862c1aArLfGYIiIiFwj93oyUgUGUpJqhCZA1ltjMERERM4HOHl55tyU7Gxzrkh2tvm2nKZPB8ilG5dIOUKTk2POwUpMtN6elKSY3CwmUBMRORLo07OdnXWllKTkumTi48dt5w35KplYCte+N/fvB+bONW+/9rj99fp46XNy+dxlbFlaCP3aCnz7sxY7qgb5/PzNYIiIqDGBPj3b2QBHqtlK7qo7LkCawMAfbL03W7Uy/3v27NVtycnmS1XeOF4ffDGouVSDH98uhP7jc9DvjMKWC11wBXWrSFQA8P35m8EQEZE9ShkJcZcrAc6mTeZLYo7k58unnoytYMGbgYGUGntvCgHMmwd07OjdkUwvfTEwXjFi94f7oF91Cvofm2HTmXRUoblVm8Sgk9C1O4iMwVWYuuJmBkPewGCIiFymtJEQdxQUOB/gnDzp03oyPhOIlzileG968MVAmAQKPz8I/Yrj0G/WoqC0M86LllZtWmvOIDtxP3SZNdDdm4yOw9tBE6Tx2/lb8grURESyFCCVdRvlyqwrpSYlBwcr9/Wxx9/vTTeW3DhUcBT6ZUeg3xAEfUlHnDJdB+A6y8MiUY6hsXuhy6iG7o/x6Db6OgSFZHjeVzcxGCIiskUN07OdDVxOnQLGjDGPNjhKSpZ5PRnZ8GTEyt/vTSeDr3U5r+H9wp7QF7fD4dq2ANpamoTjMm6I3gNd30roxrVGnz+mISR8gHf65wUMhoiIbFHqSIgr6qZj2wtw6jz+OPCf/5gvfz333NW8lDoKqicjC57m3vj7velkUPW/Tw/hLTwCAAhBDQZG7oGu53noclpg4OR0aCP7eKc/PsCcISIiW9QyPdverKv66gKeJ5801yIKxKRkf/BGUr6/35tO5padQ0ss7PsRskdF4YYpaWge19zhYxzx1/mbwRARkT1qmJ4N2B6psKXuJHvgALBlS2AlJfuDNxOfffzevHzuMrYu2wv9mnIU7G6BvKrhiMEZxw/08mxCLtRKRCS1AKis65ScHODwYeCFFxpvV5eYu2WL+YQ3frz5XwZCzvHmmmlefm/WXKrBlv/+D/+6sQC6lrvQspUGw57shfmbs7C5qifexT3O7UihOXTMGSIiakxOjnmWTKBNz64vOBiIjXWurUJPeJLzduKzB+9NU60JP3+0D/qVpdD/0AwbT6fhIq63apMQdBK6lIPQZQnc0msQ8OiLjvuk0Bw6BkNERI4E4vRsW9SQNC4lXzy/Tr43hUlg75eHoH/nmLnWz8lOOCfSAKRZ2rTSnEV2wj5zrZ+JSeh0Uyo0Qb/3xWgEFiU5vsSn0NmEDIaIiMjM0ewyd094gVj40B2nT5uP294iuF4OKA5/fwz6pcVYnx8Efcl1KDV1ANDBcn9zVGJoTCF0Ay9BNyEe3cd0tF/rp251+sbylBQ8m5DBEBERmV17wvPW9PlAX9vNWXl5wF13NT5jD/AooDi5+xTy3zgA/Xcm6A+loLi2LYAky/3huIzBLQuh61thrvUzoTNCm/Z3/hfU5SnZej0VPpuQs8mIiMiat9b0CvS13ZzlaBYZYA6A3n//6siLE84dPI+C/xZB/7UB+qIkFF7pYHV/CGrQP6IQuh7nMGxMCwyclIbwFuFuHsQ1/DjSx6n1XsRgiIjIRZ6e8NSwtpuzXFkDrpH8n4ulF7FpSSH0n1VB/1ssdl3uDHHNpHANTOjVpAi6rqegG9kMN0xJQ0RChOf9lxDXJiMiIul4mjTuyjTyzMzAzilycxZZ9YVqbH2rEPo15dD/HI0fK9NRi35WbbpoD0DX6Th0t2gx9MHOiO6QDiDdSx1XDwZDRESBQG5Jys4GAJ98AuTmBnZOkZOzw4ytYrD9zV+h//AM1u+IxObzXWBAL6s2qSFHoWt/BLobg5D9wHWI72m9ACq5h5fJiIiUTo5Jys5eGrIl0HKKHCyfIQCcCYpFR1MhytHS6r74oFLo2h6ELsuE7MntkDok2T99lgnmDHkRgyEiClhSJik3NhrlzPpZQUGOp5kHSk5RXh7E2LGAADS4+nyYYH6dxuJjrEEOWmrOIzu+CLrBBgyblIzON6dCE6SRps/Xvr4xMeZtZWV+HXlkMORFDIaIFEBul3mUQMokZWdGoxpbP8vZU4+X17rytyObzbV+9AUaaA8XYY6Yi2Rcfc5KkIh3Ih9F+JD+0E2IQ487OyEoRAYrZTlar85PI48MhryIwRCRzMnxMo8SeGmWkstcGY2yN01/zBjzVH1HVq40r4GmEKW/lJlr/ayrhf5gCg7Vpljd3wRV+FPz95DZrgSdRqSi09wJCI3wwnR3b7L3+l6r7rX+4AOgTRuffYlhMORFDIaIZIy1aNy3ahUwYYLjdu4GFLZG6wDXR6Ns7WfTJmkCOS87X3wBG5YUYf0X1dDvS8Qeg3UyczBq0b95IXQ9zkJ3RxQyJqehSXQTiXrrBGdqIl2rfkVtL3+J4dR6Igp8RqN51MDWdzIhzCfW6dPNi1HykllDvlxLzN5o3QMPOD9lvi6IsTVN31dLf/jYxdKL+P7NvdB/ehH632Kw81IaBAZY7tfAhJ5NiqDrYq71k/lAGiISrm9kj3ZIddnYUUmE+urnfB0/bv5yo7AvMQyGiEg6rtSikfHogGR8FVDYG607fhyYM8e5fTiaWu+LpT98wFBhwLZlhVi/+gL0u6PxQ2U6atHXqk1a2EHoOh3DsFu0GPpAJ7Tq6GGtHykvGztbEsEehX6JYTBERNJxsxgd/c4XAYWj0TpnOTMaJcO1rmqra/HTe3uh//AM9Dsi8P25LqhGT6s2KcHHMKx9sbnWz5QOSOhtvQCqRxoLRP0x4uLOKGJ9CvwSw2CIiKTjy8s8auHtgMLVyyT1uToalZNjHkGQaCahqdaE/+Xth37lSei3NsWGsjRUoptVm7igU9AlH4AuywTdfXW1fpJs79ATcrhs7Gi00RUK+hLDYIiIpKPQvBHZ8WZA4ckJzN3RKFeW/vAwl0aYBPavO4z8ZYdxcv0enDgTin3ohE3IhAnm/bTQXEB23F7oBhugy01E+u0doAmKdf543CWHy8aNjTa6SkFfYhgMEZF0FJI3ogieriVWx5MTmK8vb7mZS3N063FzrZ98QH+4A/qbduElPGZV7+d0UAy29JyG5Bl3osednRAcNtA3x9AYKS8b1w8yP/wQePxx+8FZ/Vlk11LglxgGQ0QkLRnmjaiaq5dJ2rQBXngBSEz07eUtF3Jpyn47jfwl+6H/thbrD6TgYG0KgEQAwB3Iw8cYa1UFGgDaiNMYtWsu0KQbECbRQqdSXTa2F2Q+//zVGkL1K1CfOQOMG2feFgBfYlhniIjkgRWo5cOZonvX8nUtIAe1b4RGg8vNY/DX1JVYX5SMXw0dre4PRi36NS/EsOvL8MyeCdCWl8HmAhdSLwHizBIm3u6fJ3W+7BXU9OKXGBZd9CIGQ0RELsrLM9cUOnfOcVtfV4l2stJ2FvKxAVkAgB7hRdB1OQndbU0x5ME0RCZFSlex2xWNLWECeHc2mTeWc/HxlxgWXSQiIunk5ABRUcCNNzpu6+NE2ZpDRxHqRLuJSXo8PE6LrAc7oXXnzgA6WzdQQikHf1429kbCtrdy1STGYIhILniZiOQmK0uS2X611bXYuaoI+g9OQ789AppzMfjGicfdt0IHZGXYb6CUUg7+KjeghODQT2QfDNXW1mLu3Ll47733UFpaivj4eEyaNAnPPPMMgoJksLIvkTdwoVKSIz/N9jPVmvDbJwegf+8k1m8Jx4ZTaahAV8v9QTDiOBIQjxOw+Vff2aBMSaUc/DHiopTg0A9kHwwtXLgQr7/+Ot5++2107doVO3bswOTJkxEVFYXHHntM6u4ReU7qirOkDFKNHPrgso0wCRxYfwT65Ueh3xSC/GMdcVp0AtDJ0qaF5gKy4vZCN8hc6yeh5mVoxt35+w7cDMpYysGakoJDH5N9AvXtt9+O2NhYLF261LJtzJgxaNq0KVasWOHUPphATbLljQTGQMJLhbbJYeTQw9em5IcT0L95CHo9oD/SHseMCVb3N0UVMlsVQtf/IoZNiEXPcZ0QHFZv/96aveSHWVCK4c+EbTf47fwtZG7BggUiJSVFFBUVCSGE2L17t4iJiRErV650eh/l5eUCgCgvL/dVN4nck58vhPlPUOM/+flS99T3Vq8WIinJ+riTkszb1Wz1aiE0mobvCY3G/CPT5+fUr2Xig+mbxUPpG0TH0EMNuh+GajE0apeYl50vNr36szBUGpzbcW2t+fOwcqX539pa9zro6X681Q85sPXZS06WxXvLX+dv2V8mmzlzJsrLy5GWlobg4GAYjUbMnz8f4xuZxmkwGGAwGCy3Kyoq/NFVItcxgdGMlwpt88daVV4ajbtwpBwbl+yF/stq6PfG43/VnQC0sdwfBCP6NivEsO5noBsdiUH3paFp656u99dbuTSe7EcOI3XeJPH6cLLg01DLC1atWiWSkpLEqlWrxC+//CLeeecdER0dLZYvX273MXPmzBEAGvxwZIhkhyND5m/U9b+V1h8BSU5W9jdvd/n6/eHBaFzV6Srx7YIdYtbAfNGv2a8iCLUNutU9fK+Y3qtAfPrMD+LCkQvu9VFuFDpSp1T+GhmSfc5QcnIyZs2ahWnTplm2/etf/8K7776LvXv32nyMrZGh5ORk5gyR/EhRcVZulFAITyqrVgETJjhu507RQxcrD1+5eAU/LC+EfvV56He1wNbyLqhBmNVDO4UWQ9exBLqbQpH1QEe0SW/tWp/kjjl+fseii7+7dOlSgyn0wcHBMJlMdh+j1Wqh1Wp93TUiz3F2Cy8VNsaTqc+NXf5y4vKbeOwx/FTREes/OAv99ub4/mw6LqGHVdOk4BMY1u4QdDpA90AHJPVLBZDq2jEqiRxWlSefkH0wNHLkSMyfPx9t27ZF165dsWvXLjz//PO47777pO4aeYKzhq5S+0KlrHVin7tTnx3ltDhxUtccO4YnJ5+1LG8BAG00p6FL2g/dkFroJqegQ3ZbaIIS7O8n0DBwD1iyD4ZeeeUV/O1vf8PUqVNRVlaGhIQEPPTQQ/j73/8uddfIXYGWfOgNak5gZK0T+9wZOXQmGf2aNILGtMdBRMU1gS6jGsNyE9B11HXQBLVx/MBAxcA9YMk+Z8gbWGdIRjxZIZkCl8xrnUjO2bo4jlZ3hwaXmrXB4qjZmHnicYe/1vjNdwgeMcyzvgcS5vj5HVet9yIGQxKpfyls0CCgQwcmH5JtUhfCk/ulW2f652Qyug7f4W1MQiKOIwg8qbvEUeD+wQdAmzbyfR8pDBOoSdlsndhatwbOnLH/GCYfqpuUlwqVcOnWibo4l349hKZO7OqBDgU4O+BRJK2aCUClifvuaizH7+67gRkz5P0+IptcGhkqKSlBcnKyL/vjExwZcoI3vxXbuxTmLHemCRO5S8GXbi+duYQtb+2Ffm0F9L+0RrOqU1iPGx0/sK5MgdSjce6Swyhe/T6cOQOMG6fI95Es2HlNZbkcR9OmTcUzzzwjLl686O16Rz7F5Tgc8OYyCI4K6Km9wCDJi8IKPhoqDeL7134W87LzxdCoXSIM1VbdDUKtOIE4YXLleJS2rIQcl21R2PtIdhp5Tf11/nYpGNq8ebPo37+/iI+PF2+99Zav+uR1DIYa4e1qqs5WzOUfDJIDmVcArzXUih0r9ohFt+aLm1v/KJqhskHXEoNOiIntN4nlUzaJI1uOXf1M1/9cB0KFZLlWf5b5+0jWHLym5StWyC8YqvP222+LpKQk0bNnT5GvgBeXwZAdvvg2s3Kl+4GQ0v9Qk/I4+351YWFoT5iMJvHr2v3i5TEFYnT8VtFCc75BV1prTos7k7aI1ydsEPu+LRYmo6nhjmS88Kbb5Dz6IrP3kWI48ZqWJyb65fztVgL1xIkTceedd2LBggW47bbbMGLECPy///f/cN1113nv+h35ni+qqTpbX6NNG+D06au31VJgkORF4roxwiRQvLEE+mVHoN8QBH1JR5wyXQfg6t/SSJRjaOxe6DKqoftjPLqNvg5BIRmN7zgQ61bJufqzkusPSZl/5cxrevy4X7ri9mwyIQRGjBiByspKvPzyy/jqq68wbdo0zJ07FxEREd7sI/mKL6qpOltA78ABYMuWwPlDTcrk7Pt10CDztHUvvF+P7ziJ/KWHoF9vgv5QKo4Y2wJoa7m/CS7hhuhC6PpVQndXG/Qe3xkh4QNc/0XeWt1dLuRc/VmphUOlnkUpo0rdLgVDr7/+OrZv347t27ejsLAQwcHB6N69O6ZNm4aePXvivffeQ5cuXbBmzRr07dvXV30mb/HFtxlnK+aGhQXWH+rGyGHmSyDx5vPpzPv17rsb1sdy4YRxdv85FCwpwvqvaqDfn4SiK+0BXP1MhaAGAyP3QNfzPHQ5LTBwcjq0kX3cO55AJufRFyWuMehMpXJfB0RyGilz5ZpaUlKSGDt2rHjuuefE999/L6qrqxu0mT9/vujatau3LuN5BXOG7Ki7Xmsrec3Ta/CBmLPgDjnOfPEVf8xK8tXzae/9+tRTLifslpeUi8/n/Chm9MkXPZsUNnwojKJv09/EX/rni6//tV1Unqz0rO9q4cu/V96ilL97csm/cuI19VfOkFsJ1I0pLS0VQUFB3t6tRxgMNcKXM0+UNmXX2+Q688UX/BH0+fr5rP9+NRicOmFcKqsU3y36Sfx1UL4Y2PwXEYyaBk27aveJR7oXiLWzt4lzh8574clQKSXMlFPC3z05zX5z8JrKejZZY0wmkygoKPD2bj3CYMgBpXybURJvfvOS+x9XfwR9UnyTdfKEMRzfNNjcIeSweCBtg1j1yGZR+r8y7/WJ+PfKG+Q2+62R19Rf52+uTUZmzGvxLifXiLJUArZH6gRHRxwsDOq1Na689Xy6YtUqYMIEh83GYyU2BmVhWLuD0GUD2ZPbIWVwknf6QLbx75VnpPg8OSJxBWquTUZmgTbzRGremPkihwRHR/w13dmPM4mESWDvl4ew9/VTuMOJ9gv/LZD8VBw0QTJKBg10/HvlGTnOfpP4NQ2S7DcTBTJPZ74YjeYRIVt/qOq2TZ9ubiclfwUpPp5JVLyxBEsnbcIf221GQmgZuozsgLEbH0EJkmCCxvaDNBogORltn7wLmiA7bYjkqG72G3B1tlsduc5+8zEGQ0S+UPfNq/4fmjq/n0jtfvNyZcRFSv6a7uzp81nPiZ2leG/qZtzfaRNSQ0vQfmgyprydiZVHBqPUFItwXEZ2y5/xY48HoAEgeMKgQJOTYx5dTky03p6UJI9RZz/jZTIiX/C07ogcCsw5k5fhr+F2D5/PcwfPo+C/RdB/bYC+KAmFVzoAiLPcH4IaDIgohK7nOXOtn0lpCG/RG0BvIK+b7bwtuVVMZx4NuSoQK5W7y6fp2TLB2WQkGXdnvkg99dWVqfL+nO7s5PNZcbxCfDH3R/FEn3zRq8keoYHRumswij5NfxNP9csXX/3TiVo/SpjRV/95adNGiI8+krpnRB7hbDIv4mwykpQ739jrZmk5GnHxdJaWLfYSt+tGYGwNodua9Zac7JvRExvPZ3VlDba+VYj1eeXQ/9wKP15Mh7HewHcX7QHoOh2H7hYthj7YGdEdWnq3X1Kx93rVeeopYNEi//aJyEv8df5mMEQkV3UnOcD2ZSFfXNf3ZKq8Hy/T1FyqwY5390L/0Vnof4rE5vNdYEC4VZv2IUeg63AEuuEhyH7gOsR1j/FJXyTl6PWq8+GHwJ13+qVLRN7EYMiLGAyRYvlzxAWQZ/0RAKZaE37+aB/0K0uh/6EZNp5Ow0VYLwgdH1QKXduDGJZtQvZ9qWh3gwpq/Tj7erVuDZSWqjMXhBSNdYaIyP8Jjp4kbntxZEiYBIq+Lob+nRLovw9D/onOOCfSAKRZ2kRrziE7oQi6G65ANzEZnW9OhSYozv5OA5Gzr9eZM8D8+cDf/+7b/hApFIMhIrnzZzEyd6fKe6FS9uHvj0G/tBj6giDoj3bASVN7AO0t9zdHJYbGFEI38BJ0E+LRfUxHBIVkONffQOVKyYI5c4Bu3eQ1A45IJniZjIiucidx252EawClv5Qh/40D0K+rhf5gCg7Vpljdr0U1BrfcA12fCujubIW+96QhtGmoFw7SBqVOSzcazf09fdq59snJvkm6J/IR5gx5EYMhBVDqySgQuZK47ULC9bnDFdiwpAj6rwwoKIpDqysnEY+TOIl4bEImNBAYELEHuh7noLsjChn3pSO8Rbjt/XqT3Nd/c+Tjj11LjvZzvheRJ/x2/vbpxH2ZYJ0hmXOlpg35h7P1kZysh/RA2HJLrZ87sFochfW+L0XGiEtvvCvNcdavj+SrGkm+9NRTztWl8udK5ERe4K/zN5fjIGnVjULUH1moW4w0L0+afqldTg5w+LB5FGHlSvO/xcUNR0qcTOCtvBIGgSA8EvIaVmMMkmD9ejepPI0mD+b69/VWyvpvzli0yJwT5AxPl0YhCkAMhkg6gXQyCkR1idvjx5v/rXfZsra6Fnt2Vju1q/tHncGJHSfwctwCaICGS59K8XorZf03Z/3tb+bLe/a4uH4bkZowGCLpBNrJyFlGo7k+zKpV5n+9efJvbN8e/l5TrQk/f1iEF0YXYGTsj4hucgnXPzfRqZXdb1w9FfGV++T1en/yiXPtfLn+mzfVrd+m0XAlciIXcWo9SUcOi5H6my+TdRvbN+Dy7xUmgX3fFEP/zjHoN4Ui/0QnnBWdAXS2tGmpOY9VLabhqfOzIaCBBo0soCqn19toBN57z7m2SrqsVLcSuRIWliWSEQZD5D/1Z4zFOLk8gpJORo2xNwW9Lj/Kk+U1Gtv3mDG2H2Pj9x7dehz6pcVYrwf0RzrgRL1aP81wEUPaFELXvwq68bHocWcnBIfNAvI6OT4Bu1vDyBc2bXJuOnqbNsq7rMSVyIlcxqn15B+2Ri0SE4HqauDcOf8vRupvnqz55em+GyE0Glxq1gYzEt/H+oPtcdBGrZ9BLQqh61MO3Z2t0C+3kVo/jsojSLn4bH2rVgETJjhuN3068MILvu0LEdnF5TgocNgbtThx4uo2jcZ2TRtnchyUUKPIlfwoV2vAONp3IzRCoNnFMhQVaXAQKQhGLfo1L4Su+1kMy4lCxuQ0NInu5dzOHFXKrstpGTvWs9fbG5wdfRo1yrf9ICJZYDBEvuVoxphGA0RHA02auJfjoJSCeb7Ml/FCjs2U1PV4amIzZE7pjMik6z3en11yyWnJzDT/TkejVP68RKaEoJ4oQDEYIt9yZkTk7Fngu++uJtk6eyLwZQ6Ot/kyX8YLOTb3vDUMyOrn8X6cIoecFjmNUgHKCeqJAhRzhsg5rn5rrWu/ejWweLHj/a9caa5n40p/fJWD4ws+ypepra7Fzvf2IG3ajWhuOO16rQxbv1dNIxS2gpDkZP+OUrm5thuRGnA5Di/ichwecnW5DFvtHf3k57vWJyeXgXB5v75Ut/RD/eUfXFj6wVhjFD9/VCReGF0gRsZuE5G4YFniwgiNMMJ636bGnhtbv1eNS6PU1prfJytXmv+trfXf7zIYGv+saDTmZVBqa/3bTyKZ8Nf5m8EQNc7VtZvstXfmj70rVq5U5jpMzq759TuT0ST2fVssXp+wQdyZtEW01pxucIgtNOfF6Pit4ouB/xBXWsU5/9zXD3ICZZ0uubL12rdu7dxrNW+e+oJUIuG/8zcvk5F9rl6KcnWKtyeXAQoKgOxsx+3kuEK3g8tQJT+cgP7NQ9DrAf2R9jhmTLB6eFNUYUjrPZZaPz3HdUJw2DWXuF55BXj8ccf9+O47YNiwq49T0mVHpbF3KcwTvIxGKsCp9dc4fvw4Zs6cia+++gqXL19Gp06dsHTpUvTp00fqrgU2V6eDuzrF25MZRHKcDeSselPQy347jfwl+6H/thb6g21xoKYdgKsBUBgMyIgqxLA+F6AbG41+uWkIa24n2Tk4GIiNda4fZWVX/+/Lqf9q19iMSk/UzcacPt2ckM4glchtsg+Gzp8/j8GDByM7OxtfffUVYmJicPDgQbRo0ULqrgU+V6eDO9v+4YfNVZE9ScyV22wgF1w4Uo6NS/ZC/2U11hcm4FdDRwBtLPcHwYh+zQqh634GutGRGHRfGpq27un8L3Bn5pqclsoINB7UgXKIQSqRV8g+GFq4cCGSk5OxbNkyy7Z27dpJ1yE1cfWk6mz7MWO884dbLjVrHKgqq8LmpXuh/6QS+l/b4KeqNJgwwKpNj/Ai6LqchO62psic0hlRbbu5/wsdjZoB5iDxzJmrt+W0VEagcTeArB/k++J3EBEABUyt79KlC2666SYcO3YMGzZsQGJiIqZOnYoHHnjA7mMMBgMMBoPldkVFBZKTk5kz5CpXp4NLtdyCzKaCX7l4BT8sL4R+9Xms39kS2yrSUYMwqzadQouh61gC3U2hyHqgI9qkt/ZuJ5zJUdForuabOJvv9eGHwJ13erWrLpHZa+0UZ/Pb2rSxXi8tORmYMgWYM8fxY+WYG0fkBZxa/zutViu0Wq2YPXu22Llzp3j99ddFeHi4ePvtt+0+Zs6cOQJAgx/OJnODq9PBvTB9XGlqDbXix+W/iX/fnC9GtNoumqCqwWSg5OBjYtJ1G8U7D20SJT+e8E/HPvxQiOBg52fyffSR41lN7sz88xalTvuvrTX3094sy7rXwWBoOHXe2cdymj0FKE6t/11oaKjIyMiw2vbII4+IgQMH2n1MdXW1KC8vt/yUlJQwGPKEi9PBXW6vMCajSfwvb5948Y4C8Ye4bSLq91o/1/600ZSJu5I3iyW5G8SB9YeFyWjyf0ddrcUk59pNSp/278mXBBV+wSCq469gSPY5Q/Hx8ejSpYvVtvT0dKxevdruY7RaLbRara+7ph6uLp8gh+UWvEiYBA7mH4V+2RHoN4ZAf6wjTouOADpa2kShHFlxe6EbVA3dPQnoOuo6aILa2N+pP/gqAd7f+SnOrG8n9xlVnuS3KSQ3jkjJZB8MDR48GEVFRVbb9u3bh5SUFIl6JCEp8yUcrUjuaXuZObb9JPRvHDTX+jmcihJjCoCr77kmuITMVnswrP9F6O6OQa+7OyM4bID9HUrBVwnw/k6iDpRp/558SQiwLxhEciP7YOjxxx/HoEGD8Oyzz2LcuHH48ccfsWTJEixZskTqrvlXoC3kKLNE2NOFZ1Dwxn7ov6nB+v3J2F+TCuDqST8UV5ARtQe6XhegG9MSAyalI6x5X8n66xRXazHJtXaTXEes3OHJlwSFf8EgkjWfXoTzks8++0x069ZNaLVakZaWJpYsWeLS4xW/HIfS8yXqk0Ei7IUjF8Snz/wgpvcqEN3D9zZ4aoNQK/o3+5+YNTBffLtgh6g6XeW3vnlVICTAyzmXiYh8istxeJGil+MItGUSJFqh+9KZS9j8ZqG51s//WmNHVTpMsH6+rg/fB13aSehuDcfQh9IQ1TbK6/2QhKsrs8thJfdrSVWygYgk56/zN4MhuVPyGlz1uRLYAR5dRrty8Qp+fGcv9B+fg35nC2wtT8cVWCfVdwwthu66EuhuCkHWlI6I6SpxwrMvuXpZUmaXMS1BNGC72jjX5yIKSFybjMwCKV/C2UTY+fOBN95wKT/KeMWIXe8XQf9+GfQ/Nsems+m4hO5WbZKCT0CXcgjDhgHZ97dH8oBUAKleODAJORu0KD0BnjOqiMiH1BkMye1bb2PkOsPHHc4GbLYq7h4/bh4Z+H0EQJgE9nx2EPoVx6HfokVBaRouiC4ArpZhaK05A13SPugya6Gb1BbXDUuBJiih4b6vpaT3RqAl1TvCGVVE5CPqu0z23XfKOoEEUr6Es5f87BDQoKppGzzY6mOsP5aGMmF9WSuyrtZPRjV0f4xH11HXISgkyPlfoKTgQqLcKyIif2LOkBdZnswVKxA5caLyTiCBki/h7PpXDmQhHxuQhSa4hBuiC6HrVwndXW3Qe3xnhIS7OdippOAi0JLqiYjsYDDkRZYnMyEBkSdO2G4k9xOI3Gb4uOvjjz1e6PPj9GcQ8+exGHBvGrSRXqg0rrTgIpCS6omIGsEEal+wFwgB8q9iGyj5Eq09X5197GvDgKweXujM75RW4TiQkuqJiGRAXcGQM1w5gfg72VZuM3xcdPncZRxcsgXd3N2BryogKy24CKSkeiIiGWAwVJ+zJxAlJdtKpOZSDX58u/D3Wj9R2HKhCzIwCAXu7Kwud+fFF70fcCotuJDrshlERArFnKE6ruSFKCnZ1o+MV4z4+aN90K86Bf2PzbDxdDqq0NyqTZLmGHYH9Ua08TQ0tnai0QDR0UCTJv7Lj1LijD25JNUrqRQBESkOE6i9yPJkPvooIl9+2XYjjca5E4jSkm19SJgECj//vdbPZi0KSjvjvGhp1aaV5ix0ifugy6yB7t5kdBzeDpq1axyfyP2dHyWX4MIVUifVc3SUiHyMwZAXWZ5MAHafyqeeAhYtcrwzlc/kOVRwFPplR6DfEAR9SUecMsVY3R+BCgyN2QtdxiXoJsTj+pyOtmv9SH0it0WOfXJEqpEZpYyOcuSKSNEYDHmRw2DIldGcVauACRMc/9KVK4Hx493orbyc2FmK/DcPQv+dCfridjhcm2x1fzgu44boPdD1rYRuXGv0+WOa87V+5HiikmOf5EYpo6McuSJSPE6t9ydXpk4rLdnWRWf3n0PBkiLov74C/b4k7L3SAUCc5f4Q1GBg5B7oep6HLqcFBk5Ohzayj3u/TI6z4xrrEwMlMyWUIrA3clVvWRciIoDBkDVnpk4H2EyeyhOV2PTGXug/q4J+Tyx2X+4MgQzL/RqY0LvpXui6lkH3h+a4YUoamsd5scaPUnCU4Sq5lyIwGs2vla3PpxDmz+j06ea8NDUGs0TUAIOhazkzmhMcbD4Bjh1r/qNqK9nWF9O/veTyucvYumwv9GvKof+5FX68mA4j+lm16ardD13nE9DdGo6hD3ZGy1TrBVBVh6MM1uQ+OurLkSuODhIFJAZDgOujOTk55hOgrZECmSXb1lyqwY5392L9h2eh/ykKWy6kw4BeVm3ahxyBrsMR6IaHIPuB6xDXvSOAjtJ0WG44ytCQ3EdHfTVyxdFBooClvmDIW6M5Ml0ew1RrMtf6WVkK/Q/NsPF0Gi7ieqs2iZrj+HNsHgZ3PoO0Md0QNzUHCE6RqMcyp4T8GH+T++ioL0auODpIFNDUNZtsxQpEzp6trKnTDgiTwN4vD0H/zjFzrZ+TnXBORFu1idacQ3ZCEYZl1uAPHX5DwtvPQsNvt85xdvbg9OnACy/4vDuyItdSBN4uoqmU2XNEAYhT673I6sls1kx2ozmuOvz9MeiXFkNfEAT90Q44aYqzur85KjE0phC6geZaP93H/F7rRym1YeTE2bpSALB6tfqeP7nm0HiziKbKa4sRSYnBkBf568n0lZO7TyH/jQPmWj+HUlBc29bq/nBcxuCWhdD1rTDX+pnQGaFNQ6134stvt3I9IXqDo+etDkcH5MdbI1cqqy1GJCesM6Ri5w6ex4YlRVj/lQH6oiQUXukAINZyfwhq0D+iELoe56C7IwoZ96UjvEXvxnfqq9yXQE8qrcuPGTOm8XZyzB0K5CDVGd7K65P77Dki8hiDIRm4WHoRm5YUmmv9/BaLXZc7Q2Cg5X4NTOjVpAi6rqegG9kMN0xJQ0RCd9d+iS9m2DSWVDpmzNVZVko/CefkmI/lxRcdt5Wqtk59gR6kOssbhT3lPnuOiDzGYEgC1ReqsfWtwt9r/UTjx8p01Nar9ZMedhC6zscw7BYthj7YGdEd0gGku/9Lvf3t1tGUc8AcPLz4YmCchEeNci4YksPoAGc+eZfcZ88RkceYM+QHtdW12PHuXug/PAP9jkhsPp+OajSxapMachS69keguzEI2Q9ch/iesXb25iZvz7BxJbE4EBK0vf38+QpnPvmOXGfPEQUw5gwpmKnWhF9W74d+5UnotzXFxrI0VKKbVZu4oFMY1vYAdFkmZE9uh9QhbQG0vZrnsUrv3TwPb3+7deVyUCAUJ1TK6ADrIvmOTGuLEZHnGAx5gTAJ7PumGPp3jmH9plAUnOiEs6IzgM6WNi0155EdXwTdYAN0E5OQdmt7aILqjf74Os/Dm5WzXb0cFAgnYSVUHpf7umFKJ8fFhYnIYwyG3HRkc12tHw30RzrghKk9gPaW+5ujEkPa7IVuQBV0E+LQ485OCAoZaH+H/srz8Na3W0dJpfYo/SQs99EBznwiInIZc4acdOrX09D/dz/062qhP5iCQ7XWy1doUY1BLQqh61OOYeNaoe89aQ1r/dij1DwPe4XtGsPCdL6llNwmIiInMGdIYueLL2DDkiLov6yGvigBvxk6AmhjuT8YtejfvBC6HmfNtX4mp6FJdC/7O2yMv/M8vFV/xt5lI1s4/dg/lJLbREQkIwyGfnex9CK+f3Mv9J9ehP63GOy8lAaBAVZtejbZi2FdSqEb2QyZD6QhIuF6O3tzkT/zPLydl3TtZaNPPrE9/dzTk7Daiwe6Sgm5TUREMqLay2SGCgO2LSuEPu8C1u+Kxg+V6aiF9WWttLCD0HU6Bt3NYch6sDNadYy2tXvP+WvtI3+sTebt6ccsHug+BpFEpHBcm8yL6p7M9a9sxQ+fVUO/IwLfn+vSoNZPSvAxDGtfbK71M6UDEnrH2dmjl3kzz8PeCdCfeUneOglzYVkiIlVjMORFdU8mUA7g6pMZG1QGXfJ+6IaaoJucgtQhydAEaaTppDdW2W5sFCU6Wlkrbys1qZyIiLyGCdQ+EIUL0MXvMdf6yU1E+u0doAmKkbpbZp7meTiamv/YY871Qy5T31k8kIiI/ERVwVDx6Qi0bN1IrR+puVvDxtE6YRoN8N57zvVBLvVnWDyQiIj8RFXBUHCYAi6nuFPh1plRlNOngdatgbNnlbHyNosHEhGRnwRJ3QHyAmdHR+65x/yvpl5elBzrz9RVuK7f1zoajXmWmlyCNyIiUizFBUMLFiyARqPB9OnTpe6KfDg7OjJqlDkvKTHRentSkvxmZtUVDwSUEbwREZFiKeoy2fbt27FkyRJ0795d6q7Ii6N1wq69BBYcLO+1ta7F4oFEROQHigmGLl68iD/+8Y9444038K9//Uvq7njOmwXxXF2CQUkrb8t9YVQiIlI8xVwmmzZtGm677TbceOONDtsaDAZUVFRY/chKXp65hk52NjBhgvnfdu3M291VN4qihEtgrqoL3saPN//LQIiIiLxIESND77//Pnbu3Int27c71X7BggWYN2+ej3vlJkf1gDwJXDiKQkRE5DLZV6AuKSlB37598e2336JHjx4AgKysLPTs2RMv2loUFOaRIYPBYLldUVGB5ORkn1ewdIhVlYmIiJzG5Th+t3btWtxxxx0IviY4MBqN0Gg0CAoKgsFgsLrPFn89mQ75a0FWIiKiAMDlOH43bNgw/O9//7PaNnnyZKSlpWHmzJkOAyGn+Gt1b29XVeaq5ERERB6TfTAUERGBbt26WW1r1qwZWrVq1WC7Wxpb3NTbScferKrsz34TEREFMMXMJvOJumTm+jk8dcnMnszussVbVZX93W8iIqIAJvucIW+wec1RqmTmukAGsF0PyNFsMiZhExGRSvgrZ0i9I0POLG5aUmJu502e1gOSqt9KYDSak9RXrTL/azRK3SMiIlIA2ecM+Yy3k5ld4Uk9ICn7LWfMoSIiIjepNxjyZjKzO9xdEkPqfsuRLwtZEhFRwGPOkKPFTeWWe6PUfvsKc6iIiAIWc4Z8rW5xU6Dh7C5bi5vKhVL77SvMoSIiIg+pNxgClLu4qVL77QvMoSIiIg+pN2eojlIXN1Vqv72NOVREROQh9eYMUWBgDhURUcBizhCRM5hDRUREHmIwRMrHHCoiIvIAc4YoMDCHioiI3MRgiAKHu4UsiYhI1XiZjIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNFajJM0Yjl8AgIiJFYzBE7svLAx57DDh27Oq2pCTzKvJcHJWIiBSCl8nIPXl5wNix1oEQABw/bt6elydNv4iIiFzEYIhcZzSaR4SEaHhf3bbp083tiIiIZI7BELlu06aGI0LXEgIoKTG3IyIikjkGQ+S6kye9246IiEhC6kqgNhqBggLOfPJUfLx32xEREUlIXcFQt27AiRNXb3Pmk3syM83P3fHjtvOGNBrz/ZmZ/u8bERGRi9R1mezaQAjgzCd3BQebg0jAHPhcq+72iy9y1I2IiBRBXcFQfZz55L6cHODjj4HEROvtSUnm7RxtIyIihdAIYes6R2CpqKhAVFQUygFE2muUnw9kZfmvU4GCFaiJiMhHLOfv8nJERto9g3tMXTlDjeHMJ/cEBzOIJCIiRVP3ZbJrceYTERGRKnFkiDOfiIiIVE3dI0Oc+URERKR66gqGEhKsb3PmExERkeqp6zLZr78CP//MmU9ERERkIfuRoQULFqBfv36IiIhATEwMRo8ejaKiIvd2Vjfzafx4878MhIiIiFRP9sHQhg0bMG3aNGzbtg3r1q1DbW0tRowYgaqqKqm7RkRERAFAcUUXT58+jZiYGGzYsAFDhgxx6jH+KtpERERE3sOii3aUl5cDAKKjo+22MRgMMBgMltsVFRU+7xcREREpk+wvk11LCIEZM2bghhtuQLdu3ey2W7BgAaKioiw/ycnJfuwlERERKYmiLpNNmzYNX3zxBb7//nskJSXZbWdrZCg5OZmXyYiIiBSEl8nqeeSRR/Dpp59i48aNjQZCAKDVaqHVav3UMyIiIlIy2QdDQgg88sgjWLNmDQoKCpCamip1l4iIiCiAyD4YmjZtGlauXIlPPvkEERERKC0tBQBERUWhSZMmEveOiIiIlE72OUOauvXD6lm2bBkmTZrk1D44tZ6IiEh5mDP0O5nHakRERKRwippaT0RERORtDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFRNMcHQa6+9htTUVISHh6NPnz7YtGmT1F0iIiKiAKCIYOiDDz7A9OnT8fTTT2PXrl3IzMzELbfcgqNHj0rdNSIiIlI4jRBCSN0JRwYMGIDevXvj//7v/yzb0tPTMXr0aCxYsMDh4ysqKhAVFYXy8nJERkb6sqtERETkJf46f8t+ZOjKlSv46aefMGLECKvtI0aMwJYtWyTqFREREQWKEKk74MiZM2dgNBoRGxtrtT02NhalpaU2H2MwGGAwGCy3y8vLAZgjTCIiIlKGuvO2ry9iyT4YqqPRaKxuCyEabKuzYMECzJs3r8H25ORkn/SNiIiIfOfs2bOIiory2f5lHwy1bt0awcHBDUaBysrKGowW1Zk9ezZmzJhhuX3hwgWkpKTg6NGjPn0y5aaiogLJyckoKSlRVa4Uj5vHrQY8bh63GpSXl6Nt27aIjo726e+RfTAUFhaGPn36YN26dbjjjjss29etW4dRo0bZfIxWq4VWq22wPSoqSlVvojqRkZE8bhXhcasLj1td1HrcQUG+TXGWfTAEADNmzEBubi769u2LjIwMLFmyBEePHsWf/vQnqbtGRERECqeIYOiuu+7C2bNn8Y9//AMnT55Et27d8OWXXyIlJUXqrhEREZHCKSIYAoCpU6di6tSpbj1Wq9Vizpw5Ni+dBTIeN49bDXjcPG414HH79rgVUXSRiIiIyFdkX3SRiIiIyJcYDBEREZGqMRgiIiIiVWMwRERERKqmyGDotddeQ2pqKsLDw9GnTx9s2rSp0fYbNmxAnz59EB4ejvbt2+P1119v0Gb16tXo0qULtFotunTpgjVr1viq+25z5bjz8vIwfPhwtGnTBpGRkcjIyMA333xj1Wb58uXQaDQNfqqrq319KC5x5bgLCgpsHtPevXut2gXa6z1p0iSbx921a1dLGyW83hs3bsTIkSORkJAAjUaDtWvXOnxMIHy+XT3uQPl8u3rcgfL5dvW4A+XzvWDBAvTr1w8RERGIiYnB6NGjUVRU5PBx/viMKy4Y+uCDDzB9+nQ8/fTT2LVrFzIzM3HLLbfg6NGjNtsXFxfj1ltvRWZmJnbt2oW//vWvePTRR7F69WpLm61bt+Kuu+5Cbm4ufv75Z+Tm5mLcuHH44Ycf/HVYDrl63Bs3bsTw4cPx5Zdf4qeffkJ2djZGjhyJXbt2WbWLjIzEyZMnrX7Cw8P9cUhOcfW46xQVFVkdU8eOHS33BeLr/dJLL1kdb0lJCaKjo3HnnXdatZP7611VVYUePXpg8eLFTrUPlM+3q8cdKJ9vV4+7jtI/364ed6B8vjds2IBp06Zh27ZtWLduHWprazFixAhUVVXZfYzfPuNCYfr37y/+9Kc/WW1LS0sTs2bNstn+L3/5i0hLS7Pa9tBDD4mBAwdabo8bN07cfPPNVm1uuukmcffdd3up155z9bht6dKli5g3b57l9rJly0RUVJS3uugTrh53fn6+ACDOnz9vd59qeL3XrFkjNBqNOHz4sGWbEl7vawEQa9asabRNoHy+r+XMcduixM/3tZw57kD5fF/Lndc7ED7fQghRVlYmAIgNGzbYbeOvz7iiRoauXLmCn376CSNGjLDaPmLECGzZssXmY7Zu3dqg/U033YQdO3agpqam0Tb29ulv7hx3fSaTCZWVlQ0Wu7t48SJSUlKQlJSE22+/vcE3Syl5cty9evVCfHw8hg0bhvz8fKv71PB6L126FDfeeGODKu1yfr3dEQifb29Q4ufbE0r+fHtDoHy+y8vLAaDRRVj99RlXVDB05swZGI3GBqvVx8bGNljVvk5paanN9rW1tThz5kyjbezt09/cOe76/vOf/6Cqqgrjxo2zbEtLS8Py5cvx6aefYtWqVQgPD8fgwYOxf/9+r/bfXe4cd3x8PJYsWYLVq1cjLy8PnTt3xrBhw7Bx40ZLm0B/vU+ePImvvvoKU6ZMsdou99fbHYHw+fYGJX6+3REIn29PBcrnWwiBGTNm4IYbbkC3bt3stvPXZ1wxy3FcS6PRWN0WQjTY5qh9/e2u7lMK7vZx1apVmDt3Lj755BPExMRYtg8cOBADBw603B48eDB69+6NV155BS+//LL3Ou4hV467c+fO6Ny5s+V2RkYGSkpK8Nxzz2HIkCFu7VMq7vZx+fLlaNGiBUaPHm21XSmvt6sC5fPtLqV/vl0RSJ9vdwXK5/vhhx/GL7/8gu+//95hW398xhU1MtS6dWsEBwc3iPbKysoaRIV14uLibLYPCQlBq1atGm1jb5/+5s5x1/nggw9w//3348MPP8SNN97YaNugoCD069dPNt8kPDnuaw0cONDqmAL59RZC4K233kJubi7CwsIabSu319sdgfD59oSSP9/eorTPtycC5fP9yCOP4NNPP0V+fj6SkpIabeuvz7iigqGwsDD06dMH69ats9q+bt06DBo0yOZjMjIyGrT/9ttv0bdvX4SGhjbaxt4+/c2d4wbM3xgnTZqElStX4rbbbnP4e4QQ2L17N+Lj4z3usze4e9z17dq1y+qYAvX1BsyzNQ4cOID777/f4e+R2+vtjkD4fLtL6Z9vb1Ha59sTSv98CyHw8MMPIy8vD3q9HqmpqQ4f47fPuNOp1jLx/vvvi9DQULF06VKxZ88eMX36dNGsWTNLVv2sWbNEbm6upf2hQ4dE06ZNxeOPPy727Nkjli5dKkJDQ8XHH39sabN582YRHBws/v3vf4vCwkLx73//W4SEhIht27b5/fjscfW4V65cKUJCQsSrr74qTp48afm5cOGCpc3cuXPF119/LQ4ePCh27dolJk+eLEJCQsQPP/zg9+Ozx9XjfuGFF8SaNWvEvn37xK+//ipmzZolAIjVq1db2gTi613nnnvuEQMGDLC5TyW83pWVlWLXrl1i165dAoB4/vnnxa5du8SRI0eEEIH7+Xb1uAPl8+3qcQfK59vV466j9M/3n//8ZxEVFSUKCgqs3reXLl2ytJHqM664YEgIIV599VWRkpIiwsLCRO/eva2m5d17771i6NChVu0LCgpEr169RFhYmGjXrp34v//7vwb7/Oijj0Tnzp1FaGioSEtLs/pwyYUrxz106FABoMHPvffea2kzffp00bZtWxEWFibatGkjRowYIbZs2eLHI3KOK8e9cOFC0aFDBxEeHi5atmwpbrjhBvHFF1802Gegvd5CCHHhwgXRpEkTsWTJEpv7U8LrXTd12t77NlA/364ed6B8vl097kD5fLvzPg+Ez7etYwYgli1bZmkj1Wdc83sHiYiIiFRJUTlDRERERN7GYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQESnSqlWrEB4ejuPHj1u2TZkyBd27d0d5ebmEPSMipeHaZESkSEII9OzZE5mZmVi8eDHmzZuHN998E9u2bUNiYqLU3SMiBQmRugNERO7QaDSYP38+xo4di4SEBLz00kvYtGkTAyEichlHhohI0Xr37o3ffvsN3377LYYOHSp1d4hIgZgzRESK9c0332Dv3r0wGo2IjY2VujtEpFAcGSIiRdq5cyeysrLw6quv4v3330fTpk3x0UcfSd0tIlIg5gwRkeIcPnwYt912G2bNmoXc3Fx06dIF/fr1w08//YQ+ffpI3T0iUhiODBGRopw7dw6DBw/GkCFD8N///teyfdSoUTAYDPj6668l7B0RKRGDISIiIlI1JlATERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVO3/A9xCDlESmSzTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_16_2.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients for OLS\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "623ad923",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Same code but now with momentum gradient descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b7fe53c1",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.]\n",
      " [3.]]\n",
      "Eigenvalues of Hessian Matrix:[0.30311767 4.03556032]\n",
      "0 [-14.28624958] [-15.54071847]\n",
      "1 [-0.04900086] [0.04473913]\n",
      "2 [-0.04532032] [0.0413787]\n",
      "3 [-0.04191624] [0.03827068]\n",
      "4 [-0.03876784] [0.0353961]\n",
      "5 [-0.03585592] [0.03273744]\n",
      "6 [-0.03316272] [0.03027848]\n",
      "7 [-0.03067182] [0.02800421]\n",
      "8 [-0.02836801] [0.02590077]\n",
      "9 [-0.02623724] [0.02395532]\n",
      "10 [-0.02426651] [0.022156]\n",
      "11 [-0.02244382] [0.02049182]\n",
      "12 [-0.02075802] [0.01895265]\n",
      "13 [-0.01919885] [0.01752908]\n",
      "14 [-0.0177568] [0.01621244]\n",
      "15 [-0.01642305] [0.0149947]\n",
      "16 [-0.01518949] [0.01386842]\n",
      "17 [-0.01404858] [0.01282674]\n",
      "18 [-0.01299337] [0.0118633]\n",
      "19 [-0.01201742] [0.01097223]\n",
      "20 [-0.01111477] [0.01014809]\n",
      "21 [-0.01027992] [0.00938585]\n",
      "22 [-0.00950778] [0.00868086]\n",
      "23 [-0.00879363] [0.00802883]\n",
      "24 [-0.00813313] [0.00742577]\n",
      "25 [-0.00752224] [0.00686801]\n",
      "26 [-0.00695723] [0.00635214]\n",
      "27 [-0.00643466] [0.00587502]\n",
      "28 [-0.00595134] [0.00543374]\n",
      "29 [-0.00550433] [0.0050256]\n",
      "theta from own gd\n",
      "[[3.98320492]\n",
      " [3.01533437]]\n",
      "0 [-0.00509089] [0.00464812]\n",
      "1 [-0.0047085] [0.00429899]\n",
      "2 [-0.00424012] [0.00387135]\n",
      "3 [-0.00378113] [0.00345227]\n",
      "4 [-0.00335942] [0.00306724]\n",
      "5 [-0.00298058] [0.00272135]\n",
      "6 [-0.00264305] [0.00241318]\n",
      "7 [-0.00234327] [0.00213947]\n",
      "8 [-0.00207732] [0.00189665]\n",
      "9 [-0.00184151] [0.00168135]\n",
      "10 [-0.00163245] [0.00149047]\n",
      "11 [-0.00144711] [0.00132125]\n",
      "12 [-0.00128282] [0.00117125]\n",
      "13 [-0.00113717] [0.00103827]\n",
      "14 [-0.00100807] [0.00092039]\n",
      "15 [-0.00089362] [0.0008159]\n",
      "16 [-0.00079216] [0.00072326]\n",
      "17 [-0.00070222] [0.00064115]\n",
      "18 [-0.0006225] [0.00056836]\n",
      "19 [-0.00055182] [0.00050383]\n",
      "20 [-0.00048917] [0.00044663]\n",
      "21 [-0.00043363] [0.00039592]\n",
      "22 [-0.0003844] [0.00035097]\n",
      "23 [-0.00034076] [0.00031112]\n",
      "24 [-0.00030207] [0.0002758]\n",
      "25 [-0.00026778] [0.00024449]\n",
      "26 [-0.00023737] [0.00021673]\n",
      "27 [-0.00021042] [0.00019212]\n",
      "28 [-0.00018653] [0.00017031]\n",
      "29 [-0.00016536] [0.00015097]\n",
      "theta from own gd wth momentum\n",
      "[[3.99951642]\n",
      " [3.00044152]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients for OLS\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x#+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 30\n",
    "\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(theta)\n",
    "    theta -= eta*gradients\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "# Now improve with momentum gradient descent\n",
    "change = 0.0\n",
    "delta_momentum = 0.3\n",
    "for iter in range(Niterations):\n",
    "    # calculate gradient\n",
    "    gradients = training_gradient(theta)\n",
    "    # calculate update\n",
    "    new_change = eta*gradients+delta_momentum*change\n",
    "    # take a step\n",
    "    theta -= new_change\n",
    "    # save the change\n",
    "    change = new_change\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"theta from own gd wth momentum\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "749000b3",
   "metadata": {
    "editable": true
   },
   "source": [
    "## But noen of these can compete with Newton's method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e7b4abe1",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.66959644]\n",
      " [3.26513904]]\n",
      "Eigenvalues of Hessian Matrix:[0.33285444 4.11450263]\n",
      "0 [-12.48534921] [-14.7906583]\n",
      "1 [-1.09712586e-14] [-5.19623863e-15]\n",
      "2 [-1.27068495e-16] [-2.9424428e-16]\n",
      "3 [5.91540705e-16] [7.2837521e-16]\n",
      "4 [-1.27068495e-16] [-2.9424428e-16]\n",
      "beta from own Newton code\n",
      "[[3.66959644]\n",
      " [3.26513904]]\n"
     ]
    }
   ],
   "source": [
    "# Using Newton's method\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "beta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(beta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "# Note that here the Hessian does not depend on the parameters beta\n",
    "invH = np.linalg.pinv(H)\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "beta = np.random.randn(2,1)\n",
    "Niterations = 5\n",
    "\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(beta)\n",
    "    beta -= invH @ gradients\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"beta from own Newton code\")\n",
    "print(beta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d47c2b1c",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Including Stochastic Gradient Descent with Autograd\n",
    "In this code we include the stochastic gradient descent approach discussed above. Note here that we specify which argument we are taking the derivative with respect to when using **autograd**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e2b04ba9",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.68184997]\n",
      " [3.32507975]]\n",
      "Eigenvalues of Hessian Matrix:[0.26370919 4.62518501]\n",
      "theta from own gd\n",
      "[[3.68184997]\n",
      " [3.32507975]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHFCAYAAADxOP3DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABWwklEQVR4nO3deXgT1foH8O90Zy0UaOkGFGQvZd9EloqACgiUFhVF9LqDaMWrgHqv9CpWuC7gRfSnF8Grt4gtBVFUZGkBBb3s+yoFSqHstFDolpzfHzGhaZNmkmaZyXw/z9MHO5lMzkwS5+0573mPJIQQICIiItIoH083gIiIiMiTGAwRERGRpjEYIiIiIk1jMERERESaxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTGAwRKdDixYshSZLpx8/PD+Hh4XjggQdw9OhRj7Vr5syZkCTJY6+vBidOnIAkSXj33Xc93RQiksnP0w0gIusWLVqEdu3aobi4GL/++itmzZqFrKwsHDp0CA0bNvR084iIvAKDISIFi42NRY8ePQAAgwYNgk6nwxtvvIEVK1bgscce83DryJPKyspMvYZEVDMcJiNSEWNgdO7cOdO24uJivPTSS+jSpQuCg4MREhKCvn374ttvv63yfEmS8Nxzz+HLL79E+/btUbt2bXTu3Bnff/99lX1XrVqFLl26IDAwEDExMVaHfYqLizFjxgzExMQgICAAkZGRmDx5Mq5evWq2X4sWLTBixAh8//336Nq1K2rVqoX27dubXnvx4sVo37496tSpg169emHbtm02r4dxODErKwvPPvssGjdujEaNGiEhIQFnzpypcu4zZ86scowWLVrg0UcfrXLM9evX48knn0SjRo1Qv359PPLIIygqKkJ+fj7GjRuHBg0aIDw8HH/9619RVlZW5bh6vR6zZs1Cs2bNEBQUhB49emDdunVV9jt69CjGjx+P0NBQBAYGon379vjoo4/M9snOzoYkSfjyyy/x0ksvITIyEoGBgTh27Bhu3LiBv/71r4iJiUFQUBBCQkLQo0cPLFmyxOb1IyID/klBpCI5OTkAgDZt2pi2lZSU4PLly/jrX/+KyMhIlJaWYu3atUhISMCiRYvwyCOPmB1j1apV2Lp1K/7xj3+gbt26mDNnDsaMGYPDhw+jZcuWAIB169Zh1KhR6Nu3L77++mvodDrMmTPHLAgDACEERo8ejXXr1mHGjBno378/9uzZgzfeeANbtmzBli1bEBgYaNp/9+7dmDFjBl577TUEBwcjJSUFCQkJmDFjBtatW4e3334bkiRh2rRpGDFiBHJyclCrVi2b1+WJJ57A8OHDkZaWhtzcXLz88st4+OGHsX79eoev9RNPPIGEhAR8/fXX2LlzJ1599VWUl5fj8OHDSEhIwFNPPYW1a9di9uzZiIiIwNSpU82eP3/+fDRv3hxz586FXq/HnDlzcM8992DDhg3o27cvAODAgQO4/fbb0axZM7z33nto2rQpVq9ejeeffx4XL17EG2+8YXbMGTNmoG/fvvjkk0/g4+OD0NBQTJ06FV9++SXeeustdO3aFUVFRdi3bx8uXbrk8LkTaY4gIsVZtGiRACB+++03UVZWJq5duyZ++ukn0bRpUzFgwABRVlZm9bnl5eWirKxMPP7446Jr165mjwEQYWFhorCw0LQtPz9f+Pj4iNTUVNO23r17i4iICHHz5k3TtsLCQhESEiIq/m/jp59+EgDEnDlzzF5n6dKlAoD49NNPTduaN28uatWqJU6fPm3atmvXLgFAhIeHi6KiItP2FStWCABi5cqVsq7TpEmTzLbPmTNHABBnz541O/c33nijyjGaN28uJk6cWOWYU6ZMMdtv9OjRAoB4//33zbZ36dJFdOvWzfR7Tk6OAGD1+t11112mbcOGDRNRUVGioKDA7JjPPfecCAoKEpcvXxZCCJGVlSUAiAEDBlRpf2xsrBg9enSV7UQkH4fJiBSsT58+8Pf3R7169XD33XejYcOG+Pbbb6vkiaSnp6Nfv36oW7cu/Pz84O/vj4ULF+LgwYNVjhkfH4969eqZfg8LC0NoaChOnjwJACgqKsLWrVuRkJCAoKAg03716tXDyJEjzY5l7HmpOMwEAElJSahTp06VYaEuXbogMjLS9Hv79u0BGPKhateuXWW7sU223HfffWa/x8XF2fV8S0aMGGH2u7FNw4cPr7Ld0utYu34bN26ETqdDcXEx1q1bhzFjxqB27dooLy83/dx7770oLi7Gb7/9ZnbMsWPHVnmdXr164ccff8T06dORnZ2NmzdvOnzORFrFYIhIwf7zn/9g69atWL9+PZ5++mkcPHgQDz74oNk+mZmZGDduHCIjI/HVV19hy5Yt2Lp1K/7yl7+guLi4yjEbNWpUZVtgYKDpJnrlyhXo9Xo0bdq0yn6Vt126dAl+fn5o0qSJ2XZJktC0adMqQzUhISFmvwcEBFS73VL7Lal8TsahuZoEBva01VI7rV2/0tJSXL9+HZcuXUJ5eTn+9a9/wd/f3+zn3nvvBQBcvHjR7Pnh4eFVjvnhhx9i2rRpWLFiBeLj4xESEoLRo0d7tAQDkdowZ4hIwdq3b29Kmo6Pj4dOp8O///1vZGRkIDExEQDw1VdfISYmBkuXLjWrAVRSUuLQazZs2BCSJCE/P7/KY5W3NWrUCOXl5bhw4YJZQCSEQH5+Pnr27OlQG1whMDDQ4jVxVW6NtesXEBCAunXrwt/fH76+vpgwYQImT55s8RgxMTFmv1uq8VSnTh2kpKQgJSUF586dM/USjRw5EocOHXLOyRB5OfYMEanInDlz0LBhQ/z973+HXq8HYLhBBgQEmN0o8/PzLc4mk8M4myszM9Osx+PatWv47rvvzPYdPHgwAENAVtGyZctQVFRkelwJWrRogT179phtW79+Pa5fv+6S17N2/fr37w9fX1/Url0b8fHx2LlzJ+Li4tCjR48qP5Z68aoTFhaGRx99FA8++CAOHz6MGzduOPu0iLwSe4aIVKRhw4aYMWMGXnnlFaSlpeHhhx/GiBEjkJmZiUmTJiExMRG5ubl48803ER4e7vBQyZtvvom7774bQ4YMwUsvvQSdTofZs2ejTp06uHz5smm/IUOGYNiwYZg2bRoKCwvRr18/02yyrl27YsKECc469RqbMGEC/va3v+Hvf/87Bg4ciAMHDmD+/PkIDg52yev5+vpiyJAhmDp1KvR6PWbPno3CwkKkpKSY9pk3bx7uuOMO9O/fH88++yxatGiBa9eu4dixY/juu+9kzYbr3bs3RowYgbi4ODRs2BAHDx7El19+ib59+5rlYRGRdQyGiFRmypQpmD9/Pv7xj3/gwQcfxGOPPYbz58/jk08+weeff46WLVti+vTpOH36tNmN1x5DhgzBihUr8Prrr+P+++9H06ZNMWnSJNy8edPsmJIkYcWKFZg5cyYWLVqEWbNmoXHjxpgwYQLefvtts2n1nvbyyy+jsLAQixcvxrvvvotevXrhm2++wahRo1zyes899xyKi4vx/PPP4/z58+jYsSNWrVqFfv36mfbp0KEDduzYgTfffBOvv/46zp8/jwYNGqB169amvCFb7rzzTqxcuRIffPABbty4gcjISDzyyCN47bXXXHJeRN5IEkIITzeCiIiIyFOYM0RERESaxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTNFFnSK/X48yZM6hXr57FcvZERESkPEIIXLt2DREREfDxcV3/jSaCoTNnziA6OtrTzSAiIiIH5ObmIioqymXH10QwVK9ePQCGi1m/fn0Pt4aIiIjkKCwsRHR0tOk+7iqaCIaMQ2P169dnMERERKQyrk5xYQI1ERERaRqDISIiItI0BkNERESkaQyGiIiISNMYDBEREZGmMRgiIiIiTWMwRERERJrGYIiIiIg0jcEQERERaRqDISIiItI0BkNERESkaQyGiIiISNMYDBEREZGmMRgiIiIiTWMwRERERJrGYIiIiIg0jcEQERERaZrHg6GNGzdi5MiRiIiIgCRJWLFihdV9n376aUiShLlz57qtfUREROTdPB4MFRUVoXPnzpg/f361+61YsQK///47IiIi3NQyIiIi0gI/TzfgnnvuwT333FPtPnl5eXjuueewevVqDB8+3E0tIyIiIi3weM+QLXq9HhMmTMDLL7+Mjh07ero5RERE5GU83jNky+zZs+Hn54fnn39e9nNKSkpQUlJi+r2wsNAVTSMiIiIvoOieoe3bt2PevHlYvHgxJEmS/bzU1FQEBwebfqKjo13YSiIiIlIzRQdDmzZtwvnz59GsWTP4+fnBz88PJ0+exEsvvYQWLVpYfd6MGTNQUFBg+snNzXVfo4mIiEhVFD1MNmHCBNx1111m24YNG4YJEybgscces/q8wMBABAYGurp5RERE5AU8Hgxdv34dx44dM/2ek5ODXbt2ISQkBM2aNUOjRo3M9vf390fTpk3Rtm1bdzeViIiIvJDHg6Ft27YhPj7e9PvUqVMBABMnTsTixYs91CoiIiLSCo8HQ4MGDYIQQvb+J06ccF1jiIiISHMUnUBNRERE5GoMhoiIiEjTGAwRERGRpjEYIiIiIk1jMERERESaxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTGAwRERGRpjEYIiIiIk1jMERERESaxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTGAwRERGRpjEYIiIiIk1jMERERESaxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTGAwRERGRpjEYIiIiIk1jMERERESaxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTGAwRERGRpjEYIiIiIk1jMERERESaxmCIiIiINI3BEBEREWmax4OhjRs3YuTIkYiIiIAkSVixYoXpsbKyMkybNg2dOnVCnTp1EBERgUceeQRnzpzxXIOJiIjIq3g8GCoqKkLnzp0xf/78Ko/duHEDO3bswN/+9jfs2LEDmZmZOHLkCO677z4PtJSIiIi8kSSEEJ5uhJEkSVi+fDlGjx5tdZ+tW7eiV69eOHnyJJo1aybruIWFhQgODkZBQQHq16/vpNYSERGRK7nr/u3nsiO7SEFBASRJQoMGDazuU1JSgpKSEtPvhYWFbmgZERERqZHHh8nsUVxcjOnTp2P8+PHVRoipqakIDg42/URHR7uxlURERKQmqgmGysrK8MADD0Cv12PBggXV7jtjxgwUFBSYfnJzc93USiIiIi+h0wHZ2cCSJYZ/dTpPt8hlVDFMVlZWhnHjxiEnJwfr16+3OW4YGBiIwMBAN7WOiIjIy2RmAi+8AJw+fWtbVBQwbx6QkOC5drmI4nuGjIHQ0aNHsXbtWjRq1MjTTSIiIvJemZlAYqJ5IAQAeXmG7ZmZnmmXC3m8Z+j69es4duyY6fecnBzs2rULISEhiIiIQGJiInbs2IHvv/8eOp0O+fn5AICQkBAEBAR4qtlERETeR6cz9AhZmmguBCBJQHIyMGoU4Ovr9ua5isen1mdnZyM+Pr7K9okTJ2LmzJmIiYmx+LysrCwMGjRI1mtwaj0REZEM2dmAhXtyFVlZgMx7cE1oZmr9oEGDUF08pqAySERERN7t7Fnn7qcSis8ZIiIiIjcJD3fufirBYIiIiIgM+vc3zBqTJMuPSxIQHW3Yz4swGCIiIiIDX1/D9HmgakBk/H3uXPuTpxVes4jBEBEREd2SkABkZACRkebbo6IM2+2tM5SZCbRoYUjMHj/e8G+LFoqaou/x2WTuwNlkREREdtLpgE2bDMnS4eGGoTF7e4SMNYsqhxrGXiYbwZW77t8MhoiIiMj5dDpDD1Dl4o1GkmTobcrJsRpkuev+zWEyIiIicr5Nm6wHQoChtyg317CfhzEYIiIiIudTUc0iBkNERETkfCqqWcRgiIiIiJxPRTWLGAwRERGR87mqZpELMBgiIiIi13B2zSIX8fhCrUREROTFEhKAUaNqXrPIhRgMERERkWv5+gKDBnm6FVZxmIyIiIg0jcEQERERaRqHyYiIiEgeZ6xXpkAMhoiIiMi2zEzghRfMl9iIijJMn1fIrDBHcZiMiIiIqmdcfb7yWmN5eYbtmZmeaZeTMBgiIiIi63Q6Q4+QEFUfM25LTjbsp1IMhoiIiMg6Fa0+7ygGQ0RERGSdilafdxSDISIiIrJORavPO4rBEBEREVmnotXnHcVgiIiIiKxT0erzjmIwRERE6qXTAdnZwJIlhn9VPKNJ0Wqy+rwK3iMWXSQiInXy4iKAiuTI6vPp6cCkScDFi7e2KfA9koSwVDjAuxQWFiI4OBgFBQWoX7++p5tDREQ1ZSwCWPkWZhy2sdVb4U5euoSFTa+8Avzzn5YfkyRZ75G77t8MhoiISF10OqBFC+u1byTJ0PuQk+P5oEOrvVcZGUBSUvX7REfbfI/cdf9mzhAREamLWooAevkSFlbpdIahMVuU8B79iTlDRESkLmooAihnCYtnngFu3jQkJXvT0NmmTcCFC/L2VUihRvYMERGRuqihCKCt3ivAEDA8/DAQH28Y9vOWniJ7AhyFFGpkMEREROqihiKA9vZ4eNPQmdwAp0kTxRRqZDBERETqooYigPb2eHjJ6u8AbgWrtnz0kdX36MQvp/HuiGzcGX7AyY2zjMEQERGpT02KALpD//5V22aLUhK/a8oYrFrruQOAl1+uMtvsePYpzLk3Gz3rHEBM/yi8vGoQtt/o4OLGGjAYIiIidUpIAE6cALKygLQ0w785OZ4PhABDQPDUU449VyFJxTViDFYr9xA1aWIoxDhnDgDg2LqTSB2Wje61D6JVfDNM+3EQtt3oAB/ocGfDHXhv7K9uaS5nkxERkXr5+gKDBnm6FZa1bu3Y8xSSVFxjVipWH1l7CulDspH+Szh2F7cF0BwA4ItyxIfsRuLdRRjzanuEduyGwsJCvLTM9U1lMEREROQK9gY1xmKRCkkqdoo/g9WD3/+BjDdzkb75D+wtbgMgxvAwyjG40S4kDb+J0a92QOO23T3STI8Pk23cuBEjR45EREQEJEnCihUrzB4XQmDmzJmIiIhArVq1MGjQIOzfv98zjSUiIpLL1qy3ipSS+G2NA4ut7v/2GGYOykZs0FF0GNkKf18/CHuL28APZbi78VYsfHQTzh0pxOqLPfDEF/3RuG0jl5+GNR4PhoqKitC5c2fMnz/f4uNz5szB+++/j/nz52Pr1q1o2rQphgwZgmvXrrm5pURERHaobtZbZUpJ/LYkM9NQByk+Hhg/3mpdJKEX2LvsCP4+IBsdAv9A7OjbkLJhEPaXtIY/SjE89H9Y9PgvOH+8CD9e6Im/LOqPRq1DPHFGVShqbTJJkrB8+XKMHj0agKFXKCIiAsnJyZg2bRoAoKSkBGFhYZg9ezaefvppWcfl2mREROQx1tYne/JJQ16RkhdvtbEgrvgmHXv0sUj/8CwytjbD4dKWpl0CUIJhYbuQeF8Z7nutExo0D7b75d11/1Z0zlBOTg7y8/MxdOhQ07bAwEAMHDgQmzdvthoMlZSUoKSkxPR7YWGhy9tKRERkkZVEYkUGPxXZWFJEADibNAXdkAs92gIAAlGMu5vuRtKYcoyYHovgZr3d22YHKToYys/PBwCEhYWZbQ8LC8PJkyetPi81NRUpKSkubRsREZFsxllvOp0hKPrmG+UHRTaWFJEAROAsBmMt6oYHI2msHiNmdEK9CHUEQBUpOhgykiqNtQohqmyraMaMGZg6darp98LCQkRHR7usfURERDZZGy6bN0+RuUIi7wxkpH5j5YIzCHp2mMvb40qKDoaaNm0KwNBDFF5hiuL58+er9BZVFBgYiMDAQJe3j4iISBZruTfGNckUkjytL9fj98/3I+PTSzi90x9LZTwnqH2My9vlah6fTVadmJgYNG3aFGvWrDFtKy0txYYNG3D77bd7sGVEREQy2ci9AeDRNcn05Xr8+vEevNhtA5oH5eP2pzvh/e2DkKFPwGlEQG/tiUpYENdJPN4zdP36dRw7dsz0e05ODnbt2oWQkBA0a9YMycnJePvtt9G6dWu0bt0ab7/9NmrXro3x48d7sNVEREQy2ci9MVuTzE3VtHWlOmz+dB/SP7uKZfva4Iw+zvRYPRRiZPO9SHrQD03avQufxx661U4jpddFspPHg6Ft27YhPj7e9Lsx12fixIlYvHgxXnnlFdy8eROTJk3ClStX0Lt3b/z888+oV6+ep5pMREQkn9y1xly8JpmuVIdNC/Yi4/MCLNvfDvn6zqbH6qMAo1ruReID/hj6cmcENej35yO9gXqBlnOd5s5VxNCeMyiqzpCrsM4QERF5THa2oVChLVlZhiEnJ07BLy8ux8aP9iJj0TVkHmyHc/pQ02MNpKsY1XIfkh4OxF1T4xBYv5pcW+MsODeXBmCdISIiIm9gXJYjL89y3pBxTbKLFw2VnWs426y8uBzZH+5B+qLrWH64PS6IrqbHGkpXMPq2fUh6OAiDp3ZGQN075B1UyQviOgGDISIiIlcyLsuRmGgIfCzl3jzwADBunMOzzcpulCFr3h6kf1GE5Uc64pLoZnqskXQJY9rsR+JDgRjc8xr8rlww9O7UUn+uj7NwmIyIiKim5AwjWaozFB0NvPceMHWq9SRrY89RTo7ZMUuvl2L93D1I/89NrDjWEZfFrXW+GksXkdD2AJIeq4uBz3WC/0/fqarGkZG77t8MhoiIiBxhDIC+/Rb473+BCxduPWYt0LAUNG3aJDunqLTH7Vjz7m5k/LcYK/7ohKuigenhUOkCEtofRNJf6mHA5E7wC/pz8MfG+mJKqXFkCYMhJ2IwRERETmWpl6ciewKNJUsMq8HbsCD0Dbx6/kUU4NaCp019zmFsx0NI/Esw+k/qBN+ASr1ROl3VPKTK7bTQ66QUTKAmIiLyJGtDX9Z6WioSwhBoJCcbFmmtLtCosMJCdb45PwgFCEaEz1mMjT2CxCcaoN/TsfANGGj9SQqscaREDIaIiLyBh6Y+ey1r64i9/74hv0fOoIrcQOPP2WYiLw+ShePqIeEswtG1sx6znt6Dvk/GwsevmgCq4mfhwAHb7QRcXuNI6RgMERGpncoWAFW86tYRGzfO/uNVE2jcuHgDP6Tuxsmbz+JF8ToEJPjg1usKGFaHD/9mHj5IulNe26sbvrNGZu+Ut2IwRESkZipZAFQ15KwjZq9Kgcb1/Ov4YfYepKdL+CEvDjfQF0Bf/Ip2mI/nEIFbwZMUHQ3MnQtJznsoZ/iuMmPOkBesL1YTTKAmIlIrlSfHKpLcatFyVLj+187dwPepe5GR6YMfz8ThJmqbdovxO4WkbseROCkUPca3gfTrL/YPd9r6LFhrH6DogJkJ1EREVD2tJce6Iy/KWbkzkgQBYFOHZ/B+1Db8lN8ZJbjd9HArv5NI6pGDpOfC0PXBdpB8mt16riPvla3PgiVetr5YTTAYIiJSK4UsAOoW7sqLclLuzAUpFJP185C++n7Tttb+OUjqeRJJz4ejc1IbSD7NnfJaAOS/x6+/DnTowCT7ShgMERGpldwbt9qTY92ZF2VrHbFqXJUa4gsxAcsxBpv0/aGHL9oF/IGk3rlIfD4CnRJaQ/KJcU47K5P7Hg8e7B29hE7GnCEiIrUy5onYWgBUzTlDnsiLsjMRWf/nv+PwDZYhCR0CjyGpz2kkvRiFDiNbQfKRnNOu6njpZ8Fd928flx2ZiIhcy7gAKHArGdbI+Pvcuaq6+VVhT16UsyQkGHqbGjeWtbvhRiphYeBk7M88jP3Ft2Fm9iB0HHWbewIhQBufBRdiMEREpGbGG3dkpPn2qChFzxKSzVN5UQkJQF4e9A0bQU7/kA8EgksuoENDD+ZneftnwYWYM0REpHYJCYYlH7yxArUH8qLO7buAzFkHkbG6Lhpe+QTfYBwEhLzeA08nq3vzZ8GFGAwREXkDX1/vTIy1ldBcXdFAO6bin911DplvH0LGmvrYeDUOegz485FumBbwAWaKN1CnrMB2e5WQrO6tnwUXYjBERETKZcyFSUw0BD4VA6LqcmFkTMXP23YWme8cQfqaBvilsBMEwky79qqzD4kDLyJxWivEDHgBKH3WMPx08aLldrKSs6pxNhkRESmfpeDmz6UqquTCWJsNJkkQAljVKwXvHByFX6/FmT3cp+5eJN15CWNfuQ3N+0VZbkNiouG/LQVlzMtxOnfdvxkMEZEycRV2qkzOZ8LGVHw9JJxGFGKQAz18cXu9PUgafAVjp7dGdO8I222wJyijGmMw5EQMhohUhquwk6Nkri22rP8H6PP+/Yjs4UCODwN1t+HaZESkTd62CjtvnG5zPPsU9ievw0gZ+459NgxwJBAC7EtQ5vuvCqwzRETKodMZeoQsdVgbtyUnG/ZTg8xMw5BNfDwwfrzh3xYtDNvJKY6tO4nUYdnoXvsgWsU3w3u7B8t7ojtmffH9Vw32DBGRcqhtFfbq/ur3th4uBTmyOgfp755Exq9NsetmOwCGBU99UY7AhrVxvaQJ6ty8CMneqfjOxPdfVdgzRETKoaZV2Kv7q9/berjsodMZ8naWLDH866RzPPTDcbw5OBtxtY6g7d0xeH3tIOy62Q6+KMfQRtvw6YSNOHvgKlZf7oW6X34CCfDcshRafv9Vij1DRKQcNak27M7cDFt/9c+cqa4erurYc12dnPi+/9tjyJh7GulbIrG/pDWAlgAAP5Thrsa7kDi8GKNf64hGrXuYP9G4LIWltrhj1pfaejiJwRARKYij1YbdOfvM1l/9knRrwUxblNDDVR17rqsThoWEXmDf8qNIn3cGGb9H42DpbQBuAwD4oxRDmuxG0n0luG9GR4S06ll92z25LIWaejgJAIMhIlISR6oNuzs3Q85f/ZcvyzuWEpZusMae6yonQExONgQnlYIRoRfYk3EE6R+eReb/IhFadgbhOItQADkIx5CwvUi8rwz3vdYJDZrbCIAq89SyFB5YT80mzmqrntCAgoICAUAUFBR4uilEJMeyZUJERQlhuJUafqKjDdsrKi+vul/FH0kyPK+83HltS0uz/noVf0JCDK/vrnY5kz3XtbxciA8+kHdNsrKEEELodXqx/asDYkbfLHGbf44AhBiDZeIUzF9TFx5R9T1XA+P1U8r7b+n7FBWlimvrrvs3E6iJSHkSEoATJ4CsLCAtzfBvTk7VHh57cjOcRe5f8y+8YPjXU0m8NSH3us6aZUgaf/FFWYc9vnQrpvXOxm2Bp9D94fZI3TIIx8pa4H4swTKMRRTMX9Mn/6yhF0ptU9GNPZyA599/Yw9f5ffT2MOntmvrIqxATUTqtWSJYSaXLWlpwIMPVr+P3GEE43IPtvKajh0D3nnHcFOsOGymhqUb5F5XOw1CFjZgEACgFm7g3sjdGDemHInLHoTP2TzLTzJez5wc5QaP1nh66Q4bS5Oo4dqyAjURkS3Oys2wdNNq3BhYsABISjLfV05e0wMPAK1amR8vJMTwGq+9ptgbj4mTc1mM64FtRzeMi96MxEQJ906PQ53Qvobp9/OtBEJA9TOvlJ4H48kkboCz2uzAYIiI1MvR2WcVWUsUvngRGDcOePllYM4c88eqm7r9wAPAu+9WPd6VK4Yp97Gxyu4VAmxfVzvoIUGCwJlRk3Du336o3fh28x0cnXmllvXrPJXEDXBWmx2YM0RE6lXT3IzqZkEZ/fOfhsCnMkt5TceOGYaY1F5sT851lSsyEtKyZeizYjpqN65d9XFHeveYByOPEme1KRRzhohI/RzNzZC5wjmaNDH89WxreEPu8bKy1DEsYeG6FjcIw9qAezHi/CLbz//gA2DKlOqvm9wcLGNei04HNG9u2N8SFeTBuI2911aB3HX/Zs8QEamf3NlnlckdHrhwQd6MNG8blkhIQPmhY9jxzKf4POpvSJAyUedqHkad/wy5iIIeVnqJJMkQjNoKhAD7e/dmzbIeCAGumUGoVkqa1aZwzBkiIu/gSG6GPcMDcgIYLxmWKC8uR/aHe5Cx+DoyD7XHBfGk6bGG0hWMvm0fLnSbgqhvpgOQWRyzOnKXz8jMBN54Q94x1RJwupqnlyZRCcUPk5WXl2PmzJn473//i/z8fISHh+PRRx/F66+/Dh8feR1bHCYjr6D0mTNqpNMBTZsakqVtkTO0peJhibIbZciatwfpXxRh+ZGOuCQamR4LkS5jTOv9SHqkFu58sTP8a/sbHnD21PHqPuO2polXppahSHdR6f8/3Hb/dmlJRyd46623RKNGjcT3338vcnJyRHp6uqhbt66YO3eu7GOwAjWpnooryCreN9/YrpxsT7XgZcsMFYYrVx82blPQe1ZyrUT8+OZW8ZfWG0WIdMmsuY2lC+KpdhvEz6nbRGlRqfWDlJcbKkunpRn+dUVVZXuqXNv7fpGiuev+rfieoREjRiAsLAwLFy40bRs7dixq166NL7/8UtYx2DNEqmZt6rdxOMLZa29p0SuvGGaNWSJJ9l9jTxfbq0bp9VKseXc3Mv5bjBV/dMJV0cD0WKh0AQntDyLx0boYOCUOfkEKyKSwdC1tWbbM49eZnINFF/90xx134JNPPsGRI0fQpk0b7N69G7/88gvmzp3r6aYRuV4NFsAkO8yZA/TqBUyaZEiWNnI0gPF0sb1Kiq8WY817e5CeVoqVxzuhALcWPA3zOY+xHQ4i6fFg9J/UCb4BA2490dNDK9b+EKhOSgoDIbKb4nuGhBB49dVXMXv2bPj6+kKn02HWrFmYMWOG1eeUlJSgpKTE9HthYSGio6PZM0Tq421TtZXO0zd/Jyq+WozVc3YjfUk5Vp7ohGu49f++cJ98JMYeRuITDdDv6Vj4Blg4R08VNTS+B3l5hkBfTj6XUVSUYVahSt8zl1LpZ5s9Q39aunQpvvrqK6SlpaFjx47YtWsXkpOTERERgYkTJ1p8TmpqKlJSUtzcUiIX8Lap2krnyWrBTnDz8k38+M5uZHyjw3cn43AdvU2PRfqcRWLcESQ+2RC3PxULH7+m1g9krUfGWNTQVUOzjgyJAbeGjOfNU8UN3u3UUq3bgxTfMxQdHY3p06dj8uTJpm1vvfUWvvrqKxw6dMjic9gzRF6DPUNkw42LN/BD6m6kpwusyo1DEeqaHov2zUNi56NIfKoR+jzeET5+MmbgempxT0eGxIwUko+lSCrPOWTP0J9u3LhRZQq9r68v9Hq91ecEBgYiMDDQ1U0jcj1nrL0lh0q70LWq6HwRVqXuRnq6hB/y4nADfU2PNfc9jcQux5D0bGP0nNgBPn6R9h3c1Yt7WvqsAbaXRbFGTpVrrWLOoWyKD4ZGjhyJWbNmoVmzZujYsSN27tyJ999/H3/5y1883TQi15OzQnpNK8iyC10Vrp25hlXv7EX6Mh/8eCYON3FrwdMYv1NI7HocSZND0WNCe0g+UY6/kCuHZq191p580rGhsagoZQRCSv1jgqvWy+fSiftOUFhYKF544QXRrFkzERQUJFq2bClee+01UVJSIvsYrDNEqmepzlB0dM1r1hhr4lSu06LAmjhO5Y7aOE5QkFsgvnr2FzE6fIsIwg2zt6iV3wkxvU+W2PblAaHX6R1/kcrXYu1aebV8srLse53qPmty6wcp8fOp5BpgaWnyrmdamqdbahXrDDkR6wyRV3D2X5+eyg3xNIX3hBWcKsDKt/cifYU/Vp/rglLcGvJv7Z+DpJ4nkfR8ODontYHkY+cK8pVZuhaNGwN6PXDlivOqaNtbPdoWOTlC7uitUXo+jhfkHLICtROxZ4jIgqws1/QAuFJNe3QU2hN2+fgVsfiJTWJ46O/CHyVmTWsb8Id4/Y4ssTv9cM16gCqzdi2q67Vx9DrJ/axV144mTYT46it577s7emvKy6u+RuU2e7oStrGN1t5nJbTRBnfdvxkMEWmV2rrQa3qDU9jN69Kxy2LhoxvFPU3+VyUA6hB4VLwxMEvszTzi3ADIyNa1MP6EhJj/7ujQrNzPmjMCMHcFvGr5Y0JFy8NY4q77t7yVTv+Um5vris4pIvIENa2wbhyOqDzMYqx7k5lp+xj2JJNao9MZhh6WLDH8q9PJab3JxcOX8O+Jm3B3420Iu60uHl/cHz9e6IkyBCA28ChS4rOx/9tj2F98G2ZmD0LsmNY1HwqzxNa1MPLxAdauBdLSDEMpOTmODfvI/QylpACRlWa/RUXJH26yNXsKMMyesvN9s0gtNcCMq9bX5LpqgT2RU+3atcXrr78url+/7qrgzCXYM0RkgVq60J3Vo1PTnrDqeqaqGb47f+CC+L+HNoi7QrYJX5SZPT0u6JB4c3CWOLjqD6ddLlns6alxRs+GPZ+1mgyFurO3Ru5rpaTU/LWcQSWTBipT5DDZr7/+Knr16iXCw8PF559/7qo2OR2DISIr1NCF7qwbXE2OY2smVKNGZtvLwyLEj/3eFHc23C58UG72lK61DohZQ7LE4Z+OO/9ayWVPDo+zhknd8Vlz59CvrQCv4o8SvkcqpchgyOiLL74QUVFRokuXLiLL0+OhMjAYIqqGq6btO4uzbnCO9oTJza+p8KODJHSQxBgsE4AQ3WvvF+/cnSWOrj0h75xd/Vd8ebkhIdldvShGrv6suTuPR24SuhJ6WFVK0cGQEELcuHFD/O1vfxO1a9cWo0ePFkePHnVmu5yKwRCRDUruQnfmDc6R3gkHZ0LpIImCoCbi+Poc+87XXXVr0tNtn4crbuKu/Kx5Yug3JcX9QaWGKDKBulKuEYYOHYqnnnoKK1euRGxsLF566SVcu3bNGalMROROxgVKH3zQ8K+S6goZlySRrCQSS5Kh7oycJUkcSSZ1MAHWBwL1iy8gRjoh/0nOSBSvzFrSd2Ii8PLL1p8nSTWvbm6JKz9rxortQNXPi7MqtlfWurW8/TydSE3VsisY+uSTT/D4448jLi4OwcHBuOuuu/Drr79i8uTJWLBgAXbt2oUOHTpg27ZtrmovEWmNs29wCQnAiROG2VFyZknVdDad3JugK2ZCZWYaih3GxwPjxxv+bdHiVlA1Zw7wzTeGQosVRUerd6aRu2dPqWlWJlllVwXq6Oho9OnTx/TTo0ePKguivv3220hLS8O+ffuc3lhHsQI1kQc4uwKwpWrJLlyt/MQvp5HxzjFkZjXE0hsjEIk8+ED2/y5vkVvd19nVgu2pjqzUtbVqwl3nZKyubWsxZW+r5O4m7rp/O305jnPnziEiIgI6Z9RxcBIGQ0Ru5qolLxy5wdnxnOPZp5Ax5zjSN4Ri240Opu0JyEAGxgEQkF31x96b4JIlht4bW5KTDSu1V0erS614ijHwBMwDIqUsy6Fiql2OQ6/Xi+zsbGcftkaYQE3kRkpa8kJGMvLRtSdE6rAs0a3WAbPdfFAu4hvsEAse2CDO7j5n+VjGKfXOmC5uT6K2reOqpTqyN1H6rEyV4kKtTsSeISI3UVKPRDXDREIAy+JmYtbRcdh1s53pIR/oEN9wN5LuuY7RM9ojLLaJ+XMt9TJ9+61zhu/kLmYq5xrK7WVKSzMkMpNzeONwo4epdphMiRgMEbmJUlbJthFY6CHhNKIQgxxIELgzZDeShhdh9IwOaNK+scXn2Hw9Z9wEMzOBsWPl7VvdNVTK+0BUQ+66f/u57MhEpD1KWa/JxtpbPhBohlysHPov3DGhJYJ9iwxBTJuGjr2ecbp4TSUkGHKC5s61vW9119BYjsBWUq+ccgREGuBwnSEioio8PM1Y6AX2LjuC9GfWydp/+LY3ETxhlOVp554yapS8/aq7hp6ot0OkYhwmI1IjpeYmeGCasdAL7Mk4gvQPzyJjazMcLm2JgchGNmQME1lqH+DZ2T/OvIY1LUeg1M8ZaQaHyYjIMldNW3cGY49EYqLhpm1pmrETeiSEXmDnkkPI+Ogc0re1wLGytgDaAgACUYyGYQEout4EtW9chGTP33tCGNqZnGzoofHEjd+Z1zAhwXAejgQ0Sv6cETmbS+eqKQSn1pPXUNK09eq4YJqxXqcXW7/YL6b1zhIt/U6YHToIN8SY8C3iv5N+EQW5BbfaYGkdMrk/np527smp2mr5nJHX49R6J+IwGXkFJU1bl8MJQyxCL7D1iwNIX3ABGbta4UR5tOmxWriBeyN3I2mswL3TOqFeRL2qB7DUuxESAly+bPvFlTDt3BPDVGr7nJFX49R6J2IwRF5BI9Ol9eV6/O/zvdjxXhYOHvXDXhGLTegPPXxRG0UYHrUHSUnAvdPjUCe0ju0DVg4odDrgrrtsP0/l19FhGvmckTowZ4iIzCll2roL6Mv12PLZPmR8dgXXdx/F3/Up6INbPROXfJvgyPAX0XnhC6jduK99B6887V2n47Tz6njx54zIGk6tJ1ILL1sdW1+ux6b5u/FClw1oFngOd0yKw8mdl/B/+qcQCfMhmkb6i+j73WuovfGnmr8wp51Xz8s+Z0RycJiMSC28YHVsXakOv3y8F+kLC5C5vy3O6puaHmuAyzjm2w4huguWF0N19vnVdNq5t/KCzxl5Dw6TkbaxvklVbpq27mzlxeXY+NFeZCy6hsyD7XBO38X0WDAKMKrlXiQ9FIChva8iYMQF6wcSAsjNNXwunFXt2dFp595MpZ8zoppgMETKw/om1iUkGAoCWro+CurRKC8uR/aHe5Cx+DoyD7XHBdHV9FhD6QpG37YPiQ8F4a6XOiOg7h2GB5YskXdwZ+aqOGsZDW+jks8ZkbNwmIyUpZqVxgF4tjKwkiiw56zsRhmy5u1B+hdFWH6kIy6JRqbHQqTLGNN6P5IeqYX4F+IQUDeg6gHcPYtJgddQcXiNyMM4td6JGAypBOubqE7p9VKsn7sH6f+5iRXHOuKyCDE91li6iDFtDyBpYh0Mej4O/rX9qz+YO3NV2PtIpArMGSLtsbHSuMWcEf7lKo8Tr1Pp9VKseXc3Mv5bjBV/dMJV0cP0WKh0AQntDyLx0boYOCUOfkEDbr1+dnb1r++uXBVrvY95eYbt7H0k0h6X1rdWCM0sx1FeblhCIC3N8G95uadbZJ+0NHnLJKSlGfa3tFxBVBSXCqjMCdepuKBYrHz9dzGh5SYRjKtmhwrzOScmxWaL9e/tEOUlFj5z9r6+K5ehKC+veuzKy01ER6vvu0PkpbgchxNpYpjMG7r97ckZuXyZuUVy1CAHq/hqMVbP2Y30JeX47kQsChFseizcJx9jOx5G0pMN0O/pWPgGWOmtcfT1XdXjx+rKRKrCnCEn8vpgyFuSjuXmjBw7BrRqxdwiWxzIwbp5+SZ+mr0b6Ut1+O5kHK7j1npfkT5nMbbTESQ91RC3PxULHz8bNVuVmAO2ZAkwfrzt/ZSwLhkRMWeIZNLpDD1CloIHIQw3nORkQz0VpQcGcnNGNm+2P7dIi2TmYBV/vwbfbwxGRrrA97lxKEIf0y7RvnlI7HwUiU81Qp/HO8LHz46qw47kgLkaqysTkQUMhtROiTecmpBT38QT9WjUSOb5PzP6LL7A3abfm/ueRmKXY0h8pjF6PdoBPn6RLn19t75P/ftzXTIiqoLBkNop8YZTU7YqA/Ove3lknv8JxKCFXy6Suv6BpMmh6DGhPSSfKLe9vlvfJ1ZXJiILmDOkdlpMCOXaSfLodNBHN4N09ozFtb70kFAY0Bh/fLYe3R7uCMnH4opgNXp9xb5PXJeMSBWYQO1EXh0MueOGo8RaPsakccDyX/eeSBpXyHUqOFWAlW/vRfoKfwSdO4GvYUgE9sGt6yQgARIgufo6KfF9MlLI+0VE1rnt/u3SifsK4fV1hpYtM9RHkaSqNVMkqWb1WZRcy8eV9Wic0RY3XqfLx6+IxU9sEsNDfxf+KDFrxnN+C8TVgCY1v06O1rFS0vtERKrCOkNO5NU9Q0au6PZXw5R9Jfx178zrZMf5XP7jCla8tQ8Zq4Kw9kJnlOHWel8dAo8hsfdpJCVHouOo2yAJfc2uU03rWCnhfSIi1eEwWQV5eXmYNm0afvzxR9y8eRNt2rTBwoUL0b17d1nP10QwBDj3hqPEGjFK5MzrJCPguHj4Ela8fQAZq2ph3aXOKMet9b5iA48isW8ekl6MQof7bqvhiVVql9KDYiLySgyG/nTlyhV07doV8fHxePbZZxEaGoo//vgDLVq0QKtWrWQdQzPBkDNpMTHbEc66TtUEHALAmjv+gX/uvwdZlztDV2ESaFzQYST1O4vEqc3Q7t6WjpxB9bwlKGbPFJEqsejin2bPno3o6GgsWrTItK1Fixaea5BWeOOUfVdwxnWyUThTQEK7TZ9iPWZAD190qXUISXfkI/Gl5mgzrC2Atg41XRZ76lj176/MgMMblqohIpeyUU/f81auXIkePXogKSkJoaGh6Nq1Kz777LNqn1NSUoLCwkKzH7KTEmvEKJEzrpONgMMHAs2Qiy97/gtH157Ezhvt8OrPg9BmWIydjXWA3GDv228NPUjx8YblLuLjDb9nZrqydbYZe9wqX1/jCvWebh8RKYLig6Hjx4/j448/RuvWrbF69Wo888wzeP755/Gf//zH6nNSU1MRHBxs+omOjnZji72EsVKvZKX2jCQZErS1Xqm3htfpzI58/DRtvayXGv9iGG4b3NzRljpGbrA3d67yAg5bS9UAhqVqdDq3NouIlEfxOUMBAQHo0aMHNm/ebNr2/PPPY+vWrdiyZYvF55SUlKCkpMT0e2FhIaKjo5kzZC8l14ipjrvzQ+y8Tqe3nsWy1CNIX9sQm6/FYgA2IhsKzc+yVccKMFxbawGFJ3OKmPdGpHruyhlSfM9QeHg4OnToYLatffv2OHXqlNXnBAYGon79+mY/5ADjOmGRldamiopSbiCUmen+4RoZ1+nUljx8MDobt9fbi+he4UhePhC/XouDgA90dYNxLaiJoRCiJZ7shTMuX2FsR+V2AdX3rFTMKXI35r0RkUyKT6Du168fDh8+bLbtyJEjaN7czcMFWmVrnTAlsTYjyzhc48oAzsJ1OiHFIOOfOch4ZB9+L4oFYAiWJOjRr/5eJN11FQnT2yCqZ1cg85M/e5cUuF5WdYvnjh1raJstngg4mPdGRDIpfphs69atuP3225GSkoJx48bhf//7H5588kl8+umneOihh2Qdg1PrnUipU5QVMgX8ePYpZMw5jvQNodh241aPpgQ9+gfvQdKQAiTMaIuIbk2rPlnp62VZeu83bVLuUJSS10YjIllYZ6iC77//HjNmzMDRo0cRExODqVOn4sknn5T9fAZDTqLkKcoezA85tu4kMv6Zg/SNYdhxs71puw90GNBgD5KGFSLh1fZoGhdq+2BKDTatqUnA4Y5zVWveGxEB4NpkTuX1a5O5g3H9s4rrSzlr/TNnSEur2jZLP2lpTnm5wz8dF2/dlSW61DpodngflIvBDbeLjx/cIPL3nnfKaymeI2vjyV3LzdH10Gy9FtdGI1IFrk3mROwZqiGFDEFVyw09Q4d+OI70904hfXME9ha3MW33RTnuDNmNxHuKMOa1DmjSvrFDx1c1e4b45C7v4cyeSLX1uBERAA6TORWDIQcZbyDr1gFvvWV7f09OUXZRfsiBlceQ/sFppG+JxP6S1qbtfijD4Ea7kTTiJkbN6IDGbRtZb5dWbsJyzlVuYP3ee8D993M9NCKN43Ic5FmW/iq3xZNTlI1TwBMTDTdMB2dkCb3AvuVHkfHhGaT/Fo2DpbcBMCx66o9SDGmyG4kjijHqtViEtOpRfZuUnGPlCr6+toNhuct7TJpkvViiJBmKJY4a5b2BJRG5FYMhqsraMIYtnp6iXN0U8GpmZAm9wJ6MI0j/8CwytjbD4dI2AAzDYAEowdDQ3Ui8rxT3vRqLhjE95bXFk9P8lUxuwHzxovXHKtYuYrFEInICDpOROVvDGJYoIWeoIhnDNUIvsGvpYaT/Kx8Z25rjaNmtdb4CUYxhTXcjaXQ5Rs6IRXCzYPtfX+k5Vp4iN7dLjrQ04MEHnXMsIlIkDpORZ9gaxqhMCUUBK7MyXCP0Atu/OoiMBeeRsT0Gf5S3A9AOgCEAuid8F5LG6DBiRifUj+rt+Ovbs9K71no2jGu5VZfb1bgxcOGC7WN5uieSiLwGgyEyZ2/ej40hKE8TeoGtXxxA+oILyNjVCifKOwAwFEOshRu4N3I3EhMEhk/vhHoRfex/AUu9UHKvYV6eoadEC8nVRnJyuxYsAF580XYyvNYXCSYip2EwRObk/rX9+uvA4MGKvIHry/X43+IDSP/kIjJ23YZTuo6mx2qjCMOj9iAxUeDeaXGo27Sv4y9kLUFabkHQF1807wHx5uTqiuTkdvn41DgZnohILuYMkTmVLmGgL9djy2f7kPHZFWTsaY3TugjTY3VwHSOi9yDpfgl3vxyHOqF1av6CtmrlhIQAly/bl4SutWnjtnK7lL48CRG5HOsMORGDITupZAkDfbkev36yFxn/vople9sgT3+rV6surmFk8z1Iut8Xd0/rjFohtZz3wnISpENCgEuXqvZs2KLQYNNjtFSniYiqYDDkRAyGHKDQv8p1pTr88vFepC8sQOb+tjirv7XgaX0U4L6YvUh8wB/DXumMoAZBFg7ghJur3BlRKSnAZ5+ZX8MmTeQlB3uygKU9GKwQkQtxNhl5VkKCoaidAm505cXl2LRgL9I/v4bMg+1wTt/F9FgwCjCq5V4kPRSAIX/tjMD6d1g/kLOKIMpNkG7dGjhxwvwa5uUBDz/svNfwJK0VlSQir8VgiKyTU1HYRcqLy5H94R5kLL6OzEPtcUF0NT3WQLqK0a32IunhIAx+Ma76AMjImUUQ5SaZh4dXvYbZ2c59DU9hUUki8iIcJiPFKLtRhqx5e5DxnyIsP9wBF8WtBU9DpMsY03o/EifUwp3JcQioGyD/wM4ugliTJPPSUqB2bcMxrPH1BW7cAALsOEd3YlFJInITDpORJpReL8X6uXuQ/p+bWHGsIy6L7qbHGksXMabtASRNrINBz8fBv7aDdWWcXQSxJuugbd5cfSAEGB7fvFm5OUMsKklEXobBELld6fVSrH1vN9K/Ksa3f8Tiiri14GkT6QIS2h1E0mN1MXBKHPyCBtT8BeXm39iTp+PgOmguaYu7ecM5EBFVwGDIHpw547CSwhL8/M/dyEgrxbfHO6EAtxY8DfM5j4T2h5D0l3oY8FwcfAOcEABVZE+Ojz0cSTJ3VVvcyRvOgYioAuYMycWZM3YrvlqM1XN2I31JOb47EYtC3FrwNNwnH2M7Hkbi48G449lO8A1wYVCppEKSSmqLo7zhHIhIFZgzpCTWZs6cPs2ZM5XcvHwTP83ejfSlOnx3Mg7XcWvB0wifs0jsdASJTzRAv2c6wcevaTVHcqKa5Ph4c1sc5Q3nQERUAXuGbLE1cwYwFCPU8F/BNy7ewA+pu5GRLvB9bhyKUNf0WJTvGSTGHUXS0yHo83hH+Pj5eK6hSiokqaS2OMobzoGIFI0VqJ2oRhdTbrVhtVQMdpKi80VYlbob6ekSfsiLww3cWu+rme9pJHU5hsRnGqPXox08GwBVpqS8LyW1xVHecA5EpFgcJlOKvDzn7leZim4m185cw6p39iJ9mQ9+PBOHm7jd9FgLv1wkdf0Dic82Qc+JHSD5RHmwpdXwYCHJKpTUFkd5wzkQkeYxGLJFzjpS9uxXkaeSsu0IwApPF+L71L1IX+6Ln852RnGFAKil30kkdc9B0nNh6Da+HSSfaNe1mYiIyEUYDNnSpIlz9zPy1HIGMgKwglMFWPn2XmR864/V+Z1Rgn6mXW/zP4GkHieQNKUputzfFpJPc+e3kYiIyI0YDNkSGenc/QBDz8wLL1ieliyEYUZOcrKhho0zh8yqCcBEYiKy73wT7+8dgp/Pd0Ypbq331TbgOJJ6nULilHDEJbaB5NPCeW0iIiLyMCZQ2+KK2WSeSMq2cR56SDiNKMQgB3r4on3AH0jqk4uk5Eh0HHUbJB/JOe0gIiKSiQnUSlGxpgrgnJoqnljOwMZ6Uj4QaIZcLOz8L/T6xwh0uO82AK2c9/pEREQKpaA5zwpmXIeq8lBYVJRjuT1uXs7gwsGLWPf6eln7Pjot7M9AiIiISBvYMySXI+tQWdO/vyGQsrWcQX8HV2kHcG7fBSx/+yDSf6qL7Cud0R93YjDetP1EridFREQaw2DIHs6qqeKi5Qzy95xH5tsHkb66PjZejYMetxY8LQpqjELRBPVKLkKCawIwIiIiNeIwmac4aejtzI58/CtxAwY22IWIzo0xeelAZF/tCj180bPOfsy+Jxt/ZJ3C1puxqJ/2iSHWkiolQ3M9KSIi0jDOJvM0BypQn956FstSjyB9bUNsvhYLUSGm7V1nH5LiL2LstNvQ4g4LVaC5nhQREakE1yZzIkUHQzKd2pKHZbOPIn1dI2y53snssb519yJp8CWMndYazfrKqHfkzCVAVLScCBERqQun1hNO/HIaGe8cQ0Z2Y/xeFAvAEOhI0KNf/b1IHHwVY2e0QVTPTtUfqDJn5T55ajkRIiIiJ2IwpDDHs08hY85xZG4IQdCNywjHWQQB8EUZ+gXvR9KQAiTMaIuIbp0921BPLSdCRETkZAyGFODYupPI+GcO0jeGYcfN9hiDbUjHBETjVo+LLiwcvgvmKyPA8NRyIkRERC7AYMhDjqzOQcZ7J5H+S1PsutkOgGHB07HIQDqSquzvez5fOT0uNqpZQwggN9ewn7OWEyEiInIRBkNudOiH40h/7xQyNodjT3FbADEAAF+U486Q3UgaVoi/rH8B0jkLT1ZSj4snlhMhIiJyEdXVGUpNTYUkSUhOTvZ0U2Q5sPIYUuKzERt0FO2Ht8Tf1w/CnuK28EMZhjXahs8e2YT8QwX4+VJ3PPmUBN9zZ6wfrGKPiye5eTkRIiIiV1JVz9DWrVvx6aefIi4uztNNsUroBfYtP4qMD88g/bdoHCy9DYBhrS9/lOKuJruRNKIYo16LRUirHuZPVkuPixuWEyEiInIX1QRD169fx0MPPYTPPvsMb731lvsbYKmeDgBs2gSRdwZ/HCnHF2sjkb4tBodL2wBoAwAIQAmGhu5G4n2luO/VWDSM6Wn9NeT2pBw9WrNzqciROkEuWk6EiIjIE1RTdHHixIkICQnBBx98gEGDBqFLly6YO3euxX1LSkpQUlJi+r2wsBDR0dGOF22yUE9HNGoEXYkOftevmrblIgovYB5WYTjubroLiaPKcN+rnRDcLFje6+h0QIsW1ScnA4ZelxMnah5s1LROEKtZKx+LYhKRirHoYgVff/01duzYga1bt8raPzU1FSkpKc55cWv1dC5dQuVbSiROYxnG4uanX6L2kw/b/1q+vsCTTwJvvFH9fqdP13ymljPqBCUkGJK5ebNVJhbFJCKSRfE9Q7m5uejRowd+/vlndO5sKDTotp4hnQ7iz54ayebOfzLmy+TkOBYULFkCjB9ve7+0NODBB+0/PmC7B6qm50CeZy3YNQ5jKqFEAxGRDe7qGVL8bLLt27fj/Pnz6N69O/z8/ODn54cNGzbgww8/hJ+fH3Q6XZXnBAYGon79+mY/9hB6gd/+vQ8LOvwLkj2BEFDzGV/umKllT50gUh9bRTEBQ4kGC98dIiItUnwwNHjwYOzduxe7du0y/fTo0QMPPfQQdu3aBV8n9Vzoy/XY/H978WK3DWgecAZ9n4zFpiNhjh/Q0RlfxplakpUQTJIMeTk1mamllllr5BgGu0REdlF8zlC9evUQGxtrtq1OnTpo1KhRle320pfr8esne5Hx76tYtrcN8vS3Fjyti2voEHYRsFQAUQ5He27cMVOLdYK8G4NdIiK7KL5nyNl0pTpsmLcLU+I2ICrwPAZM6YwPdw9Enj4c9VCIh1r8ihUzfseFK/74W96k6ntpLHFGz01CgiGnIzLSfHtUlHNyPdzR+2SJTgdkZxvyorKzOUzjKgx2iYjsovgEamcwJmA93u4HfH+kO87pQ02PBaMAo1ruReL4AAx9uTMC6weaP9mYiApYzsGoyNnJqa6cFm3tvFyVYMuZTe5jTJC3VRSTCfJEpHDuSqDWVDAEFACojwbSVYxutRdJDwdh8ItxVQOgyizdyBs1Mvx76dKtbWqrseOuOkGc2eR+7g52iYhcgMGQExkv5sMtf8RDjzXGnclxCKgbYN9BqqlAreoaO64uysdp/J7DophEpHIMhpzIXReTLMjOBuLjbe+XlVWzIpJkGStQE5GKsQI1eQfObPIsX18GmURENmhuNhm5GWc2ERGRwjEYItfy1DR+IiIimRgMkWsZi0gCVQMiZxWRJCIiqgEGQ+R6ri4iSUREVANMoCb3SEgARo3izCYiIlIcBkPkPpzZRERECsRhMiIiItI0BkNERESkaQyGiIiISNOYM2QJlzAgIiLSDAZDlVla3DIqylArh1PAiYiIvA6HySrKzAQSE6uusJ6XZ9iememZdhEREZHLMBgy0ukMPUJCVH3MuC052bAfEREReQ0GQ0abNlXtEapICCA317AfYAiKsrOBJUsM/zJIIiIiUiXmDBmdPSt/P+YVEREReQ32DBmFh8vb7+hR5hURERF5EQZDRv37G3p3Kq+sbiRJhsc/+4x5RURERF6EwZCRr69hmAuoGhAZf3/ySfvyioiIiEjxGAxVlJAAZGQAkZHm26OiDNtbt5Z3HLn5R0RERORxTKCuLCEBGDXKcgXq7Gx5x5Cbf0REREQex2DIEl9fYNCgqtuNeUV5eZbzhox5Rf37u7yJRERE5BwcJrOHnLyiuXO5jhkREZGKMBiyl628ItYZIiIiUhUOkzmiurwiIiIiUhUGQ46ylldEREREqsJhMiIiItI0BkNERESkaRwmUwudjjlKRERELsBgSA0yM4EXXjBfCiQqyjDNn7PXiIiIaoTDZEqXmQkkJlZdEy0vz7A9M9Mz7SIiIvISDIaUTKcz9AhZqnZt3JacbNiPiIiIHMJgSMk2baraI1SREEBurmE/IiIicgiDISU7e9a5+xEREVEVDIaULDzcufsRERFRFYoPhlJTU9GzZ0/Uq1cPoaGhGD16NA4fPuzpZrlH//6GWWOVF4U1kiQgOtqwHxERETlE8cHQhg0bMHnyZPz2229Ys2YNysvLMXToUBQVFXm6aa7n62uYPg9UDYiMv8+dy3pDRERENSAJYWmqknJduHABoaGh2LBhAwYMGCDrOYWFhQgODkZBQQHq16/v4ha6gKU6Q9HRhkCIdYaIiMhLuev+rbqiiwUFBQCAkJAQq/uUlJSgpKTE9HthYaHL2+VSCQnAqFGsQE1EROQCquoZEkJg1KhRuHLlCjZVM5185syZSElJqbJdtT1DREREGuSuniFVBUOTJ0/GqlWr8MsvvyAqKsrqfpZ6hqKjoxkMERERqQiHySqZMmUKVq5ciY0bN1YbCAFAYGAgAgMD3dQyIiIiUjPFB0NCCEyZMgXLly9HdnY2YmJiPN0kIiIi8iKKD4YmT56MtLQ0fPvtt6hXrx7y8/MBAMHBwahVq5aHW0dERERqp/icIclKwcFFixbh0UcflXUM1U+tJyIi0iDmDP1J4bEaERERqZziK1ATERERuRKDISIiItI0BkNERESkaQyGiIiISNMYDBEREZGmMRgiIiIiTWMwRERERJrGYIiIiIg0jcEQERERaRqDISIiItI0BkNERESkaQyGiIiISNMYDBEREZGmMRgiIiIiTWMwRERERJrGYIiIiIg0jcEQERERaRqDISIiItI0BkNERESkaQyGiIiISNMYDBEREZGmMRgiIiIiTWMwRERERJrGYIiIiIg0jcEQERERaRqDISIiItI0BkNERESkaQyGiIiISNMYDBEREZGmMRgiIiIiTWMwRERERJrGYIiIiIg0jcEQERERaRqDISIiItI0BkNERESkaQyGiIiISNMYDBEREZGmMRgiIiIiTVNNMLRgwQLExMQgKCgI3bt3x6ZNmzzdJCIiIvICqgiGli5diuTkZLz22mvYuXMn+vfvj3vuuQenTp3ydNOIiIhI5SQhhPB0I2zp3bs3unXrho8//ti0rX379hg9ejRSU1NtPr+wsBDBwcEoKChA/fr1XdlUIiIichJ33b8V3zNUWlqK7du3Y+jQoWbbhw4dis2bN3uoVUREROQt/DzdAFsuXrwInU6HsLAws+1hYWHIz8+3+JySkhKUlJSYfi8oKABgiDCJiIhIHYz3bVcPYik+GDKSJMnsdyFElW1GqampSElJqbI9OjraJW0jIiIi17l06RKCg4NddnzFB0ONGzeGr69vlV6g8+fPV+ktMpoxYwamTp1q+v3q1ato3rw5Tp065dKLqTSFhYWIjo5Gbm6upnKleN48by3gefO8taCgoADNmjVDSEiIS19H8cFQQEAAunfvjjVr1mDMmDGm7WvWrMGoUaMsPicwMBCBgYFVtgcHB2vqQ2RUv359nreG8Ly1heetLVo9bx8f16Y4Kz4YAoCpU6diwoQJ6NGjB/r27YtPP/0Up06dwjPPPOPpphEREZHKqSIYuv/++3Hp0iX84x//wNmzZxEbG4sffvgBzZs393TTiIiISOVUEQwBwKRJkzBp0iSHnhsYGIg33njD4tCZN+N587y1gOfN89YCnrdrz1sVRReJiIiIXEXxRReJiIiIXInBEBEREWkagyEiIiLSNAZDREREpGmqDIYWLFiAmJgYBAUFoXv37ti0aVO1+2/YsAHdu3dHUFAQWrZsiU8++aTKPsuWLUOHDh0QGBiIDh06YPny5a5qvsPsOe/MzEwMGTIETZo0Qf369dG3b1+sXr3abJ/FixdDkqQqP8XFxa4+FbvYc97Z2dkWz+nQoUNm+3nb+/3oo49aPO+OHTua9lHD+71x40aMHDkSERERkCQJK1assPkcb/h+23ve3vL9tve8veX7be95e8v3OzU1FT179kS9evUQGhqK0aNH4/Dhwzaf547vuOqCoaVLlyI5ORmvvfYadu7cif79++Oee+7BqVOnLO6fk5ODe++9F/3798fOnTvx6quv4vnnn8eyZctM+2zZsgX3338/JkyYgN27d2PChAkYN24cfv/9d3edlk32nvfGjRsxZMgQ/PDDD9i+fTvi4+MxcuRI7Ny502y/+vXr4+zZs2Y/QUFB7jglWew9b6PDhw+bnVPr1q1Nj3nj+z1v3jyz883NzUVISAiSkpLM9lP6+11UVITOnTtj/vz5svb3lu+3veftLd9ve8/bSO3fb3vP21u+3xs2bMDkyZPx22+/Yc2aNSgvL8fQoUNRVFRk9Tlu+44LlenVq5d45plnzLa1a9dOTJ8+3eL+r7zyimjXrp3Ztqefflr06dPH9Pu4cePE3XffbbbPsGHDxAMPPOCkVtecvedtSYcOHURKSorp90WLFong4GBnNdEl7D3vrKwsAUBcuXLF6jG18H4vX75cSJIkTpw4Ydqmhve7IgBi+fLl1e7jLd/viuSctyVq/H5XJOe8veX7XZEj77c3fL+FEOL8+fMCgNiwYYPVfdz1HVdVz1BpaSm2b9+OoUOHmm0fOnQoNm/ebPE5W7ZsqbL/sGHDsG3bNpSVlVW7j7Vjupsj512ZXq/HtWvXqix2d/36dTRv3hxRUVEYMWJElb8sPakm5921a1eEh4dj8ODByMrKMntMC+/3woULcdddd1Wp0q7k99sR3vD9dgY1fr9rQs3fb2fwlu93QUEBAFS7CKu7vuOqCoYuXrwInU5XZbX6sLCwKqvaG+Xn51vcv7y8HBcvXqx2H2vHdDdHzruy9957D0VFRRg3bpxpW7t27bB48WKsXLkSS5YsQVBQEPr164ejR486tf2OcuS8w8PD8emnn2LZsmXIzMxE27ZtMXjwYGzcuNG0j7e/32fPnsWPP/6IJ554wmy70t9vR3jD99sZ1Pj9doQ3fL9rylu+30IITJ06FXfccQdiY2Ot7ueu77hqluOoSJIks9+FEFW22dq/8nZ7j+kJjrZxyZIlmDlzJr799luEhoaatvfp0wd9+vQx/d6vXz9069YN//rXv/Dhhx86r+E1ZM95t23bFm3btjX93rdvX+Tm5uLdd9/FgAEDHDqmpzjaxsWLF6NBgwYYPXq02Xa1vN/28pbvt6PU/v22hzd9vx3lLd/v5557Dnv27MEvv/xic193fMdV1TPUuHFj+Pr6Von2zp8/XyUqNGratKnF/f38/NCoUaNq97F2THdz5LyNli5discffxzffPMN7rrrrmr39fHxQc+ePRXzl0RNzruiPn36mJ2TN7/fQgh8/vnnmDBhAgICAqrdV2nvtyO84ftdE2r+fjuL2r7fNeEt3+8pU6Zg5cqVyMrKQlRUVLX7uus7rqpgKCAgAN27d8eaNWvMtq9Zswa33367xef07du3yv4///wzevToAX9//2r3sXZMd3PkvAHDX4yPPvoo0tLSMHz4cJuvI4TArl27EB4eXuM2O4Oj513Zzp07zc7JW99vwDBb49ixY3j88cdtvo7S3m9HeMP321Fq/347i9q+3zWh9u+3EALPPfccMjMzsX79esTExNh8jtu+47JTrRXi66+/Fv7+/mLhwoXiwIEDIjk5WdSpU8eUVT99+nQxYcIE0/7Hjx8XtWvXFi+++KI4cOCAWLhwofD39xcZGRmmfX799Vfh6+sr3nnnHXHw4EHxzjvvCD8/P/Hbb7+5/fyssfe809LShJ+fn/joo4/E2bNnTT9Xr1417TNz5kzx008/iT/++EPs3LlTPPbYY8LPz0/8/vvvbj8/a+w97w8++EAsX75cHDlyROzbt09Mnz5dABDLli0z7eON77fRww8/LHr37m3xmGp4v69duyZ27twpdu7cKQCI999/X+zcuVOcPHlSCOG93297z9tbvt/2nre3fL/tPW8jtX+/n332WREcHCyys7PNPrc3btww7eOp77jqgiEhhPjoo49E8+bNRUBAgOjWrZvZtLyJEyeKgQMHmu2fnZ0tunbtKgICAkSLFi3Exx9/XOWY6enpom3btsLf31+0a9fO7MulFPac98CBAwWAKj8TJ0407ZOcnCyaNWsmAgICRJMmTcTQoUPF5s2b3XhG8thz3rNnzxatWrUSQUFBomHDhuKOO+4Qq1atqnJMb3u/hRDi6tWrolatWuLTTz+1eDw1vN/GqdPWPrfe+v2297y95ftt73l7y/fbkc+5N3y/LZ0zALFo0SLTPp76jkt/NpCIiIhIk1SVM0RERETkbAyGiIiISNMYDBEREZGmMRgiIiIiTWMwRERERJrGYIiIiIg0jcEQERERaRqDISIiItI0BkNERESkaQyGiIiISNMYDBGRKi1ZsgRBQUHIy8szbXviiScQFxeHgoICD7aMiNSGa5MRkSoJIdClSxf0798f8+fPR0pKCv7973/jt99+Q2RkpKebR0Qq4ufpBhAROUKSJMyaNQuJiYmIiIjAvHnzsGnTJgZCRGQ39gwRkap169YN+/fvx88//4yBAwd6ujlEpELMGSIi1Vq9ejUOHToEnU6HsLAwTzeHiFSKPUNEpEo7duzAoEGD8NFHH+Hrr79G7dq1kZ6e7ulmEZEKMWeIiFTnxIkTGD58OKZPn44JEyagQ4cO6NmzJ7Zv347u3bt7unlEpDLsGSIiVbl8+TL69euHAQMG4P/+7/9M20eNGoWSkhL89NNPHmwdEakRgyEiIiLSNCZQExERkaYxGCIiIiJNYzBEREREmsZgiIiIiDSNwRARERFpGoMhIiIi0jQGQ0RERKRpDIaIiIhI0xgMERERkaYxGCIiIiJNYzBEREREmsZgiIiIiDTt/wEgrLp9sOh8BgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_22_1.png"
      }
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own sdg\n",
      "[[3.68809785]\n",
      " [3.32032017]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using SGD\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = (1.0/n)*training_gradient(y, X, theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "# Can you figure out a better way of setting up the contributions to each batch?\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        theta = theta - eta*gradients\n",
    "print(\"theta from own sdg\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6d7455a",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Same code but now with momentum gradient descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "40a68e4f",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.55555773]\n",
      " [3.41891092]]\n",
      "Eigenvalues of Hessian Matrix:[0.30326262 4.34133193]\n",
      "theta from own gd\n",
      "[[3.55511609]\n",
      " [3.41928689]]\n",
      "theta from own sdg with momentum\n",
      "[[3.58207928]\n",
      " [3.37895549]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using SGD\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 100\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = (1.0/n)*training_gradient(y, X, theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "change = 0.0\n",
    "delta_momentum = 0.3\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        # calculate update\n",
    "        new_change = eta*gradients+delta_momentum*change\n",
    "        # take a step\n",
    "        theta -= new_change\n",
    "        # save the change\n",
    "        change = new_change\n",
    "print(\"theta from own sdg with momentum\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1f7f38d",
   "metadata": {
    "editable": true
   },
   "source": [
    "## AdaGrad algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/adagrad.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/adagrad.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b85cc96c",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Similar (second order function now) problem but now with AdaGrad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "3ced9e13",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own AdaGrad\n",
      "[[2.00039962]\n",
      " [2.99772199]\n",
      " [4.00233436]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using AdaGrad and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-8\n",
    "for epoch in range(n_epochs):\n",
    "    Giter = 0.0\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        Giter += gradients*gradients\n",
    "        update = gradients*eta/(delta+np.sqrt(Giter))\n",
    "        theta -= update\n",
    "print(\"theta from own AdaGrad\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed1fa9e4",
   "metadata": {
    "editable": true
   },
   "source": [
    "Running this code we note an almost perfect agreement with the results from matrix inversion."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "787b2b5e",
   "metadata": {
    "editable": true
   },
   "source": [
    "## RMSProp algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/rmsprop.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/rmsprop.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac61b6e0",
   "metadata": {
    "editable": true
   },
   "source": [
    "## RMSprop for adaptive learning rate with Stochastic Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "5febac4f",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own RMSprop\n",
      "[[1.99858411]\n",
      " [2.9981377 ]\n",
      " [3.99861427]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Value for parameter rho\n",
    "rho = 0.99\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-8\n",
    "for epoch in range(n_epochs):\n",
    "    Giter = 0.0\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "\t# Accumulated gradient\n",
    "\t# Scaling with rho the new and the previous results\n",
    "        Giter = (rho*Giter+(1-rho)*gradients*gradients)\n",
    "\t# Taking the diagonal only and inverting\n",
    "        update = gradients*eta/(delta+np.sqrt(Giter))\n",
    "\t# Hadamard product\n",
    "        theta -= update\n",
    "print(\"theta from own RMSprop\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee8a6301",
   "metadata": {
    "editable": true
   },
   "source": [
    "## ADAM algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/adam.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/adam.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2d0f16b",
   "metadata": {
    "editable": true
   },
   "source": [
    "## And finally [ADAM](https://arxiv.org/pdf/1412.6980.pdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "59349e6f",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own ADAM\n",
      "[[2.00002678]\n",
      " [2.99985103]\n",
      " [4.00014662]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Value for parameters beta1 and beta2, see https://arxiv.org/abs/1412.6980\n",
    "beta1 = 0.9\n",
    "beta2 = 0.999\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-7\n",
    "iter = 0\n",
    "for epoch in range(n_epochs):\n",
    "    first_moment = 0.0\n",
    "    second_moment = 0.0\n",
    "    iter += 1\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        # Computing moments first\n",
    "        first_moment = beta1*first_moment + (1-beta1)*gradients\n",
    "        second_moment = beta2*second_moment+(1-beta2)*gradients*gradients\n",
    "        first_term = first_moment/(1.0-beta1**iter)\n",
    "        second_term = second_moment/(1.0-beta2**iter)\n",
    "\t# Scaling with rho the new and the previous results\n",
    "        update = eta*first_term/(np.sqrt(second_term)+delta)\n",
    "        theta -= update\n",
    "print(\"theta from own ADAM\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d490115",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Introducing [JAX](https://jax.readthedocs.io/en/latest/)\n",
    "\n",
    "Presently, instead of using **autograd**, we recommend using [JAX](https://jax.readthedocs.io/en/latest/)\n",
    "\n",
    "**JAX** is Autograd and [XLA (Accelerated Linear Algebra))](https://www.tensorflow.org/xla),\n",
    "brought together for high-performance numerical computing and machine learning research.\n",
    "It provides composable transformations of Python+NumPy programs: differentiate, vectorize, parallelize, Just-In-Time compile to GPU/TPU, and more."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bc7785a",
   "metadata": {
    "editable": true
   },
   "source": [
    "### Getting started with Jax, note the way we import numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "f1090d5e",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import jax\n",
    "import jax.numpy as jnp\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from jax import grad as jax_grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac6bf976",
   "metadata": {
    "editable": true
   },
   "source": [
    "### A warm-up example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ed55121d",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/jax/_src/numpy/lax_numpy.py:173: UserWarning: Explicitly requested dtype float64 requested in asarray is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more.\n",
      "  return asarray(x, dtype=self.dtype)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1262b9a90>]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABQCUlEQVR4nO3deXxU1eH+8c+dmaxkhZCFECDsS9hBFNkURami4K6tim1tUyOVUmu/WlutbeXXRUstYLRWXLHugDu4sCj7JoussgWSAAnJTNZJMnN/fwxEEQQCydyZyfN+veYFTG7Iw4iZh3POPccwTdNERERExE9sVgcQERGR5kXlQ0RERPxK5UNERET8SuVDRERE/ErlQ0RERPxK5UNERET8SuVDRERE/ErlQ0RERPzKYXWA7/J6veTn5xMbG4thGFbHERERkTNgmiZlZWW0adMGm+3UYxsBVz7y8/PJyMiwOoaIiIichby8PNq2bXvKawKufMTGxgK+8HFxcRanERERkTPhcrnIyMiofx8/lYArH8emWuLi4lQ+REREgsyZLJnQglMRERHxK5UPERER8SuVDxEREfErlQ8RERHxK5UPERER8SuVDxEREfErlQ8RERHxK5UPERER8auA22RMREREmojXA3uXQvlBiEmB9kPBZvd7jAaNfEydOpXBgwcTGxtLcnIy48ePZ9u2bcddM3HiRAzDOO5x/vnnN2poEREROXMz188k9+PJMC0Lnr8S3vyJ78dpWeR+PJmZ62f6NU+DyseiRYvIyclh+fLlLFiwgLq6OsaMGUNFRcVx111++eUUFBTUP95///1GDS0iIiJnzla0nRkHPiHXVn7c87m2CmYc+ARb0Xa/5mnQtMuHH3543K9nzZpFcnIya9asYcSIEfXPR0REkJqa2jgJRURE5Ox5PWRv+Ahs5cxITAAgu9RFbkIcMxLjySlxkl08Hy72+G0K5pzWfDidTgBatmx53PMLFy4kOTmZhIQERo4cyV/+8heSk5NP+nu43W7cbnf9r10u17lE+v6slbW8unofOw+V87fr+jbJ1xAREQk4e5eCK5/so7+ckZhAbkI8HsMgp6SU7FIX4PRdlzncL5HO+m4X0zSZMmUKw4YNIysrq/75sWPH8vLLL/Ppp5/y2GOPsWrVKi6++OLjCsa3TZ06lfj4+PpHRkbG2UY6pRqPl//3wVZeW72fvCOVTfI1REREAk75wfqfjqisAsBjGISZ5tHiceJ1Te2sy8fdd9/Nhg0beOWVV457/sYbb+SKK64gKyuLcePG8cEHH7B9+3bee++9k/4+999/P06ns/6Rl5d3tpFOqXVsBOd3bAXAexsLmuRriIiIBJyYlPqf/qWVb6bCME1qDYPchLiTXtfUzmraZdKkScybN4/FixfTtm3bU16blpZG+/bt2bFjx0k/HhERQURExNnEaLAr+qSx9Oti3t2QT/bITn75miIiIpZqPxTi2jDDVsGGSN/77YyDh9kcEX50DYhBtjfGd52fNGjkwzRN7r77bt566y0+/fRTMjMzT/s5xcXF5OXlkZaWdtYhG8vYrDTsNoNNB1zsKao4/SeIiIgEO5ud3D6XkZsYD0DrujouqKomu9RFTomTGYnx5PYZ49f9PhpUPnJycnjppZeYPXs2sbGxFBYWUlhYSFWVbw6pvLyce++9l2XLlrFnzx4WLlzIuHHjSEpKYsKECU3yB2iIli3CGdrJN/Xy7oZ8i9OIiIj4hzepKxm1BgBXllfWT3tke2PISR+NN6mrX/MYpmmaZ3yxYZz0+VmzZjFx4kSqqqoYP34869ato7S0lLS0NC666CL+9Kc/nfFCUpfLRXx8PE6nk7i4uNN/QgO9tiqP+97cQPfUWD6cPOL0nyAiIhLkiot3cMk7E6gzDP4adwM/6Nyj0Xc4bcj7d4PWfJyup0RFRfHRRx815Lf0uzG9UnjgbYOthWXsPFRO5+QYqyOJiIg0qVcW/pU6w6BjNVx8ywMQ5v8t1b+t2R0slxAdzvAuSYCmXkREpHn4pGQlAN1sfYm0uHhAMywfAFf2aQPAuxsKTjuaIyIiEsy2bn+PnWEmDtNkRN9fWh0HaKbl49JeKYTbbew8VM62g2VWxxEREWky/1vlOzSuR2Uklw4YaHEan2ZZPuIiwxjRtTUA723QhmMiIhKaamsr+cy9B4AOkRcR4bB+ygWaafkAGNfXt++Ipl5ERCRUfb76SY7YbSR4vFx8Xo7Vceo12/IxukcKEQ4bu4sq2JzfNIfZiYiIWOnNbW8B0Lm8NSO6t7M4zTeabfmIiXBwcXffSbvvaupFRERCjLN0D0vxnT6fnnA94Y7AecsPnCQWOHbXyztf5mvqRUREQsp7y/9BrWHQwQ1jzr/B6jjHadbl4+LuybQIt3OgtIq1+0qtjiMiItJo5uZ/DkDrii71R4sEimZdPqLC7Vza03eE8DtfasMxEREJDbt2f8JXdg8O06Rd+k9x2APr7T6w0ljgqn6+qZf3Nhbg8WrqRUREgt/ba2YA0K0igivOO9/iNCdq9uVjWOfWxEeFcbjMzYpdxVbHEREROSeeuhrec20HIKJ6MAPbJVqc6ETNvnyEO2z8oHcqAPM09SIiIkFu+bqnOWw3iPd46dr1Z9hsJz+R3krNvnwAjOvrm3r5YFMhNXVei9OIiIicvTnbXgcgs6wlVw/oZHGak1P5AIZktiI5NgJnVS1Ldhy2Oo6IiMhZKXMd4LNa3xICb91l9GoTZ3Gik1P5AOw2gyv6+LZb110vIiISrD5a8Rhum0FGjcmArOswjMCbcgGVj3rHpl7mf3WQqhqPxWlEREQabs7+hQC0cnbg6v7p1oY5BZWPo/pnJJDRMorKGg+fbD1odRwREZEG2bt3CV/aarGZJtWOm+icHGt1pO+l8nGUYRiM+9Z26yIiIsFk3trpAHSvDOPifhdYnObUVD6+5djUy2fbDuOqrrU4jYiIyJnxeup4p2QzAKazP1ceXccYqFQ+vqV7aixdkmOoqfPy4aZCq+OIiIickVVfPkuB3SDG48WMu4mMltFWRzollY9vMQyDq49utz5vvaZeREQkOMzd8j8AOpcncGX/rhanOT2Vj++4up9vdfDSr4s45Kq2OI2IiMipVZQXsqDmEADFzlH1W0cEMpWP78hoGc2Adgl4TXhnQ4HVcURERE5p/orHqbYZtKkxSWozjqSYCKsjnZbKx0mMP3pv9Nz1ByxOIiIicmrz8j4FINnZngkD2lqc5syofJzED3qnYbcZbNjvZNfhcqvjiIiInNT+/ctZbbgxTJM95Vdwac9UqyOdEZWPk0iKiWBY5yQA5mrhqYiIBKh3Vj8BQLeqMPp1u4CYCIfFic6Mysf3GN//6F0vX+ZjmqbFaURERI7n9dQx98hGAGylWVx9dK+qYKDy8T0u7ZlKZJiN3UUVbNjvtDqOiIjIcdZufIkDdoj2etlTO54RXVtbHemMqXx8j5gIR/3cmaZeREQk0Mzb8hIAXcriGd27K+GO4HlLD56kFjg2hPXOhnw8Xk29iIhIYKisLOKjat9O3CWlwxjfL3BPsD0ZlY9TGNG1NQnRYRwuc7Ps62Kr44iIiADwyYppVNoMUmu9HHFcyqD2iVZHahCVj1MId9j4QW/fTnFztOeHiIgEiHn75gOQ5szg6v4Z2GyGxYkaRuXjNI4NZX24qZCqGo/FaUREpLkrLFjHCrMSgJ3OsfVnkgUTlY/TGNQ+kfSEKMrddXy85aDVcUREpJl7Z9U/MQ2DrpV24lr2p0danNWRGkzl4zRsNqN+z4856zT1IiIi1jG9XuYWrQMg3NWTCf2Da6HpMSofZ+DYf9xF2w9TXO62OI2IiDRXX371P/baIcrrZYPzyvqT2IONyscZ6JwcS+/0eOq8Ju/qpFsREbHIvE0vANC1PJZ+HTuSGh9pcaKzo/Jxho6NfrytqRcREbFAdVUJH1btB8BZOjTo9vb4NpWPMzSubxvsNoP1eaU66VZERPxu4aonKLMZJNd62VlzEZdnBccJtiej8nGGWsdGMLyL76TbOdpuXURE/GzO7vcBSHe14ZKebYmNDLM40dlT+WiAY1Mvc9Yd0Em3IiLiN4cObmKZWQHArtLLmdA/+Pb2+DaVjwa4tGcK0eF29h2pZO2+EqvjiIhIM/HuysfxGgadqwyqwvowvEvwnGB7MiofDRAd7qifY9PCUxER8QfT62Xe4dUARDl7Mq5vG8Lswf32HdzpLXBs6uXdDQXU1HktTiMiIqHuq21v87XdJMJrstF5BeODdGOxb1P5aKChnZJIiYugtLKWz7YdsjqOiIiEuDkb/gtAt4poWrfMoG/beIsTnTuVjway24z6HeXeWrvf4jQiIhLKatxlfFC5D4Dy0vOZ0D8dwwiuE2xPRuXjLFw7oC0An249RElFjcVpREQkVC1a9W+cNoOkOi8by0czYUDwT7mAysdZ6ZYaS682cdR6TN7ZoD0/RESkaczb9S4AGa5UzuuYTNvEaIsTNQ6Vj7N0bPTjzTWaehERkcZXVLSVJV4XAHtLL61/3wkFKh9n6ap+bXDYDL7c72TnoTKr44iISIh5f+XjeAyDTtUGh7z9GNs7zepIjUbl4ywlxUQwqptvk5e31mrPDxERaVzzDq4AoIWzK5dnpRIT4bA4UeNR+TgH1xwdAnt73QE8Xm23LiIijWPrtnlss3kJM002ll7JNSGy0PQYlY9zMLpHMnGRDgqc1SzfVWx1HBERCRFzNzwDQPfyKKJjMhjaKcniRI1L5eMcRDjsjOvrO9xHC09FRKQx1NZW8n75LgCqnIMZ3z8duy349/b4NpWPc3TtQN/UywebCqlw11mcRkREgt3nq2dyxGaQWOdlQ9mlXBtiUy6g8nHO+mckkJnUgqpaDx9sKrQ6joiIBLm5O98GoENZMr3atqZLSqzFiRpfg8rH1KlTGTx4MLGxsSQnJzN+/Hi2bdt23DWmafLwww/Tpk0boqKiGDVqFJs3b27U0IHEMIz6VqqpFxERORelJbtZ5HECsL90dEjt7fFtDSofixYtIicnh+XLl7NgwQLq6uoYM2YMFRUV9df87W9/4/HHH2f69OmsWrWK1NRULr30UsrKQncvjAkD2mIYsGxXMXlHKq2OIyIiQer9Ff+gzjDo4Ia82gH16wpDTYPKx4cffsjEiRPp1asXffv2ZdasWezbt481a9YAvlGPadOm8bvf/Y5rrrmGrKwsnn/+eSorK5k9e3aT/AECQXpCFEM7tQK054eIiJy9uQVfABDv7Mzo7im0bBFucaKmcU5rPpxO39BQy5YtAdi9ezeFhYWMGTOm/pqIiAhGjhzJ0qVLT/p7uN1uXC7XcY9gdP3ADADeWJuHV3t+iIhIA+3Y+SFf2Tw4TJNNpVdy/aDQnHKBcygfpmkyZcoUhg0bRlZWFgCFhb4FlykpKcddm5KSUv+x75o6dSrx8fH1j4yMjLONZKnLeqUSG+Eg70gVK3YfsTqOiIgEmXnrnwKge0UEjqj2jOza2uJETeesy8fdd9/Nhg0beOWVV074mGEcfz+yaZonPHfM/fffj9PprH/k5eWdbSRLRYXbubKvb9/9N7TwVEREGqCutpp3XTsAqHEO4poB6TjsoXtD6ln9ySZNmsS8efP47LPPaNv2m2Gh1NRUgBNGOQ4dOnTCaMgxERERxMXFHfcIVtcd3fPj/Y0FlGvPDxEROUPL1j1Nkd0g3uNlg2tM/ftJqGpQ+TBNk7vvvpu33nqLTz/9lMzMzOM+npmZSWpqKgsWLKh/rqamhkWLFjF06NDGSRzABrRLpOPRPT/e31BgdRwREQkSc7e/AUBHVxI926bQNQT39vi2BpWPnJwcXnrpJWbPnk1sbCyFhYUUFhZSVVUF+KZbJk+ezKOPPsrbb7/Npk2bmDhxItHR0dxyyy1N8gcIJIZh1O94qqkXERE5E07nPj6r860VLCi9mOsGBefax4ZoUPl48skncTqdjBo1irS0tPrHq6++Wn/Nfffdx+TJk7nrrrsYNGgQBw4cYP78+cTGhnaLO+baAW2xGbByzxH2FFWc/hNERKRZ+2j5Y9QYBu3cJnvrBnFVn9Dc2+PbGjztcrLHxIkT668xDIOHH36YgoICqqurWbRoUf3dMM1Banwkw7v4Vihr9ENERE5nbv4SABKdHRnTM4346DCLEzW90F1Ka6FjC4XeXLsfj/b8EBGR77F7z0I22GqxmyZbSsdxfTOYcgGVjyZxac8U4iIdFDirWfp1kdVxREQkQM1bMx2AbpXhhLXoyLDOSRYn8g+VjyYQGWbn6n6+w+ZeXRWc+5aIiEjT8tTV8I5zKwDe0v5cMyAdu+3ke2KFGpWPJnLD0aGz+ZsPUlJRY3EaEREJNCvWP8NBu0Gsx8uXZWPr3zeaA5WPJpKVHkePtDhqPF7mrtdhcyIicrx5214HoFNZS/p3SKdDUguLE/mPykcTMQyDG48eCvTq6v2YphaeioiIT3lZAZ/UHAbgUOkIbhzcfEY9QOWjSY3vn064w8aWAhebDgTnab0iItL45q94jGqbQdsak3zvMMZmpVkdya9UPppQQnQ4l/XynXfz6up9FqcREZFAMXf/ZwAkOdszrl9bosLtFifyL5WPJnbj0QVEc9fnU13rsTiNiIhYLS/vC9YaNdhMk62lV9S/TzQnKh9NbGinVrRNjKKsuo4PNumwORGR5m7e6mN7e4TRKimLPm3jLU7kfyofTcxmM7h+oK/VvrZK262LiDRnXk8d80o2AWA4+3DDoAwMo3ns7fFtKh9+cN2gthgGLNtVzN5iHTYnItJcrdnwPPl2aOH18lXFOCb0T7c6kiVUPvwgPSGqfsvc11dr9ENEpLmas2U2AF3K4hnRsyOJLcItTmQNlQ8/OXYP9+tr8qjzeC1OIyIi/lZZfogF7oMAFJWOaFY7mn6XyoefXNozhZYtwjnocrNo+2Gr44iIiJ99vHIaVTaDNjUmzrBLms0hciej8uEnEQ471xyd23tlpfb8EBFpbubumw9AsqsdNw5u32wOkTsZlQ8/uuk83xDbp1sPUeistjiNiIj4S37+alYabgzT5OvSsdwwuK3VkSyl8uFHnZNjOa9DS7wmvL46z+o4IiLiJ/NWTQOga5WDHp3OJy0+ytpAFlP58LNjox+vrs7D69VhcyIioc70eplX/CUADmcWN53XzuJE1lP58LMf9E4jLtLB/pIqPt9ZZHUcERFpYus2vkSeHaK9XvK913BRt9ZWR7KcyoefRYbZ6zeV0cJTEZHQN++rlwDoUhbHVQN74bDrrVevgAVuHuIbclvw1UEOl7ktTiMiIk2lqvIIH1XnA+B0Xli/51Nzp/Jhge6pcfTLSKDOa/LmWu14KiISqj5dOY1ym0FKrZeEtAlktIy2OlJAUPmwyM3HFp6uysM0tfBURCQUzd37IQBtXOncPKSjxWkCh8qHRa7s04YW4XZ2F1WwbFex1XFERKSRFRauZ7lZCcDh6qu4pEeKxYkCh8qHRVpEOLi6fuGp9vwQEQk1766ahmkYdK2yMaLfaMIdess9Rq+EhW45eq/3h5sKKCrXwlMRkVBher3MPbwGgHBnT27W3h7HUfmwUFZ6PH0zEqj1mLyxRgtPRURCxcYtr7PHDpFek6iWt9EhqYXVkQKKyofFfni0Dc9esU87noqIhIg5G58DoGt5C64/v5+lWQKRyofFruybRmykg31HKvnia+14KiIS7NzVTj6s8m0iaVYN45KeWmj6XSofFosOd3DtAN/phi8v146nIiLBbuGqJyiz2Whd52VAnzsI046mJ9ArEgBuObbj6ZaDHHRVW5xGRETOxVtfvwtAW1caN2lvj5NS+QgAXVNiGdwhEY/X5LVVuu1WRCRYHT60mRVUABAfcyNtE7Wj6cmofASIHw5pD/gOm/No4amISFCat+JxPIZB52qD8cOusjpOwFL5CBCXZ6WSGB1GvrOahdsOWR1HREQayPR6mXdoFQCtKnsxqluyxYkCl8pHgIgMs3PdQN/C09krtPBURCTYbNk+j10Ok3CvSe/ud2G3GVZHClgqHwHklqNTL59uO0TekUqL04iISEO8tuYpALpXRHPThYMtThPYVD4CSGZSC4Z1TsI0fWs/REQkONS6K/jY7fu+nR4xmpS4SIsTBTaVjwDzo/N9ox+vrsrDXeexOI2IiJyJBSv+hdNuo1Wdl6tGTLI6TsBT+Qgwl/RIJi0+kuKKGj7YWGh1HBEROQNv75gLQNeqNlzYNc3iNIFP5SPAOOy2+tNuX1y+1+I0IiJyOsXFO1ht9+3tMTDzdgxDC01PR+UjAN14XgYOm8GavSVszndaHUdERE7G64HdS3j93V9SZxh0rIabR99odaqgoPIRgJJjI7k8KxWAl3Tei4hIQJm5fia5H0+GaVnw/JV86t4FwLUVFcz+/DfMXD/T2oBBQOUjQN12QQcA5qw7gLOq1towIiJSz1a0nRkHPiHXVs62sDC2RITjME2O2D3MOPAJtqLtVkcMeCofAWpwh0S6pcRSVevhrbX7rY4jIiIAXg/ZGz4ip6SUGYkJ/DkpEYD2tbX8NyGOnBIn2Rvm+6Zk5HupfAQowzD40QW+225fXL4X09R5LyIiltu7FFz5ZJe6+EVJKesjfft5fB0eTk5JKdmlTnAd8F0n30vlI4BN6J9OTISDXYcrWPZ1sdVxRESk/GD9TzvW1tX/PMw0yS51nfQ6OZHKRwCLiXBwzYB0AJ5ftsfaMCIiAjEpAJjAX1smAGAzTWoNg9yEuBOuk5NT+Qhwtx7d8XTBVwfZX6LzXkRELNV+KMS14cGklhQ5HDi8XhbtO1C/BiQ3IR7i0n3XyfdS+QhwXVJiubBzK7ymbrsVEbGczU5un8uYFxsDwE1l5SR4vWSXusgpcTIjMZ7cPmPAZrc4aGBT+QgCtx+97fZ/q/ZRXasV1CIiVsqr821/4DBNbnOW1T+f7Y0hJ3003qSuVkULGg6rA8jpje6RQtvEKPaXVDFvfT43DM6wOpKISLNVdGAtOKBfZSypP5oD5Yd8azzaDyVbIx5nRCMfQcBuM+rXfjy3dI9uuxURsciu3YtYZveNdgzPvBsjcwT0vg4yh2uqpQFUPoLEjYMziAyz8VWBi9V7S6yOIyLSLOUufgTTMMiqDOMmneNy1lQ+gkRCdDjj+/luu31u6R5rw4iINEOHD23mY9O3f0dW/E1Eh2vlwtlS+Qgitw/tAMCHmwopdFZbG0ZEpJl55tMHqTUMOlcZ3HrZ3VbHCWoNLh+LFy9m3LhxtGnTBsMwmDNnznEfnzhxIoZhHPc4//zzGytvs9YjLY7zMlvi8Zq8vGKv1XFERJqNMtcB5lb5DozrZLuUdq2iLU4U3BpcPioqKujbty/Tp0//3msuv/xyCgoK6h/vv//+OYWUb0w8Ovoxe8U+3HW67VZExB9mf/Y7Kmw22taYjBt5r9Vxgl6DJ6zGjh3L2LFjT3lNREQEqampZx1Kvt+YnimkxUdS4KzmnS8LuG5gW6sjiYiENHe1k1eKV4PdIKNyICO66v3tXDXJmo+FCxeSnJxM165dufPOOzl06ND3Xut2u3G5XMc95Ps57DZuO7rp2Kwvduu2WxGRJjZv8R8pthu0rvMyfND/YRiG1ZGCXqOXj7Fjx/Lyyy/z6aef8thjj7Fq1Souvvhi3G73Sa+fOnUq8fHx9Y+MDG2gdTo3n+e77XZzvouVu49YHUdEJGR56mp4Nm8BAO2d3bh2cBeLE4WGRi8fN954I1dccQVZWVmMGzeODz74gO3bt/Pee++d9Pr7778fp9NZ/8jLy2vsSCEnITqcawf4plue/WK3xWlERELXJ8v+xn4HxHq8dOtyr26vbSRNfqttWloa7du3Z8eOHSf9eEREBHFxccc95PTuuLADAPO/Osi+Yp12KyLS2Eyvl2d2vAFAF2c6tw3vb3Gi0NHk5aO4uJi8vDzS0tKa+ks1K52TYxnZtTWmCc8v22N1HBGRkLNi3X/YYvcQ4TVp2WoSbRKirI4UMhpcPsrLy1m/fj3r168HYPfu3axfv559+/ZRXl7Ovffey7Jly9izZw8LFy5k3LhxJCUlMWHChMbO3uwdG/14dVUeZdW11oYREQkxz2x8BoAerkR+NHKYxWlCS4PLx+rVq+nfvz/9+/uGn6ZMmUL//v35wx/+gN1uZ+PGjVx99dV07dqV22+/na5du7Js2TJiY2MbPXxzN6JLazq1bkG5u4431uy3Oo6ISMj4auscVhjV2E0TbBMZ0C7B6kghpcErZ0aNGnXK2zs/+uijcwokZ85mM7jjwkwenLOJ55bu4bYLOmC36RYwEZFz9ezqxwHoVd6C64ZfrttrG5nOdgly1wxIJz4qjL3FlXy29fv3UxERkTOTl/cFC+p82xhUVYznB721ZrGxqXwEuehwBzef1w6AZz7fZXEaEZHgN+uLP+E1DHpWOBh93vWE2fVW2dj0ioaA24e2x2EzWL7rCJsOOK2OIyIStIqKtjK32reGrqbkEm45+o87aVwqHyEgLT6KK/r4hgWfWaLRDxGRszV78e+pMQw6VRv07PEjEluEWx0pJKl8hIg7h3cE4N0NBRQ4qyxOIyISfMrLCnjFuQUAR/EQfjqis8WJQpfKR4jISo/n/I4tqfOaPPfFHqvjiIgEnTcW/Z5ym0F6jUli+k/ITGphdaSQpfIRQo6NfsxeuY9yd53FaUREgkeNu4wXDi0HIPFIb+4c0dXiRKFN5SOEXNQtmY6tW1BWXcerq3RAn4jImXp3yR85bDdIqvPijv4xgzskWh0ppKl8hBCbzeCnw3yjH89+vps6j9fiRCIigc/rqWPWvvkApJd04o4RvbSpWBNT+Qgx1wxIp1WLcA6UVvHh5kKr44iIBLzPlv+DPXaTGI+XgrqJjM1KtTpSyFP5CDGRYXZ+dH57AP6zZPcpt8IXEWnuTK+XZ7a/CkAXZxtuvrAfDm0q1uT0CoegWy9oT7jDxpd5pazaU2J1HBGRgLX6y1lsstUR4TXZ7bqFGwdnWB2pWVD5CEFJMRFcO6AtAE8v/triNCIigeu/G54GoIcrgbGDzyc2MsziRM2DykeIunN4JoYBH285xI6DZVbHEREJONu2v8sXVGIzTfaVXMuPL8y0OlKzofIRojq2juGynr5FU08v1pbrIiLf9eyqfwCQVR7Neb0uJjU+0uJEzYfKRwj7+Ujfbbdz1h+g0FltcRoRkcCxf/9yPqwtAuBQ0RX8bERHixM1LyofIax/u0TOy2xJrcdk1he7rY4jIhIwnv/iEbyGQY9KB+0zf0CXlFirIzUrKh8hLvvo6MfLK/bhqq61OI2IiPWKi7bzduU+AKqKLuLnIztZnKj5UfkIcaO6JtM1JYZydx2zV+yzOo6IiOVmL/k9bptBx2pwtJzAoPbaSt3fVD5CnM1m8LMRvlb/7Oe7cdd5LE4kImKdyvJD/K90MwBhxUPIHtVFW6lbQOWjGbiqbxtS4yI5VOZm7rp8q+OIiFjmjcUP4rIZtKkxKQm7kUt7pFgdqVlS+WgGwh02fjLMd/967uKv8Xi15bqIND+17gpeKFwKQKsjWdw5shs2m0Y9rKDy0UzcPKQdcZEOdh2uYL4OnBORZui9zx/hoN2gVZ2XPbU/ZHz/dKsjNVsqH81ETISDiUM7ADBj4U4dOCcizYrXU8esvR8AkFGSyR0jehEZZrc4VfOl8tGMTLwwk6gwO5sOuFiyo8jqOCIifrNo5TR22U1aeL18XfFDbh7SzupIzZrKRzPSskU4N5/n+x9uxmc7LU4jIuIfptfLf7fOBqBraSrXDx1ITITD4lTNm8pHM3PniEzC7AYrdh9hzd4jVscREWlyaze+yJe2WsK9JrtcN9dPQYt1VD6ambT4KCYcXWQ187OvLU4jItL0nv0yF4CeZfFcNnAoiS3CLU4kKh/NUPbIThgGfLL1EFsKXFbHERFpMtt3fsBisxybabLvyLXcOVwHyAUClY9mqGPrGH6QlQbAkws1+iEioWvWir8B0Ks8igt6X0JqfKTFiQRUPpqtX4zybbn+7oZ89hZXWJxGRKTx5eev5oPawwAUF4/VAXIBROWjmcpKj2dk19Z4TY1+iEhoeuHzP+IxDLpV2unSZTyZSS2sjiRHqXw0Y5Mu7gzAm2v3c6C0yuI0IiKNp+TI17xZsRuAmuKR3H30+50EBpWPZmxQh5Zc0LEVtR6TXI1+iEgIeWXRg1TbDDq4ISXjh3RPjbM6knyLykczN2m0718Dr67O46Cr2uI0IiLnrrKyiNmlGwGIKh7MPZd0sziRfJfKRzN3QcdWDGqfSE2dl6cW7bI6jojIOXt70e9x2gzSak1iUn9KVnq81ZHkO1Q+mjnDMJg0ugsAs1fupajcbXEiEZGzV1tbyXMFSwBIOtKTSaN7WJxITkblQxjRJYm+beOprvXynyUa/RCR4PXh53+h0G6QWOclvGU2/dslWh1JTkLlQ3yjHxf7Rj9eXLaXkooaixOJiDSc6fXy7O53AGhf0oG7RvezNpB8L5UPAWB0j2R6psVRWePh2S92Wx1HRKTBlqz6FzvtJtFeL7T4OedltrQ6knwPlQ8Bjo1++O58ee6LPTgray1OJCLSMP/56kUAupWmkH3pUIvTyKmofEi9y3ql0j01ljJ3Hf/9XGs/RCR4rN/4MutttYSZJkb4TxjaqZXVkeQUVD6kns1mcM/RO1+e/WIPpZVa+yEiweE/a6cD0NMVx88uuwzDMCxOJKei8iHHOTb6Ue6u45klWvshIoHv668XsJhyDNMk0v4jjXoEAZUPOY7NZjD5kq4AzPpit+58EZGA959ljwLQqyKSH192g0Y9goDKh5xgTM8UeqTFUVHj0b4fIhLQCgvW8VHdYQDiuZYLNOoRFFQ+5AS+0Q/f2o/nl+7hiEY/RCRA/XfRH6gzDLpW2rn98p9p1CNIqHzISY3pmUKvNhr9EJHA5Szdw9wq39q0NuYYjXoEEZUPOSnD+Gbtx/NL91CsM19EJMA898n9VNkMOrjhljG/sTqONIDKh3yvS3ok0zs9nsoaD08t1uiHiASOqsojvOHcAEDH2qFc0Lm1xYmkIVQ+5HsZhsGvLvWt/Xhh2R4OuaotTiQi4vPyxw9QareRUuvl5ssftjqONJDKh5zSRd2SGdAugepaL9M/22l1HBER6mqrefXw5wBk1fbj/E5pFieShlL5kFMyDIN7L+sGwCsr95F3pNLiRCLSbHk9sHsJb71xO4UOgwSPl9vGTLU6lZwFlQ85raGdkriwcytqPSZPfLLD6jgi0szMXD+T3I8nw7QszOev5LWy9QDc6Kpi5e7Hmbl+pqX5pOFUPuSM/HqMb/TjzbX72XW43OI0ItKc2Iq2M+PAJ+TayvkiKpJtEeFEe714jDpmHPgEW9F2qyNKA6l8yBkZ0C6R0d2T8Zrwz481+iEifuL1kL3hI3JKSpmRmMCfWiUC0LWmhmcS4skpcZK9Yb5vSkaCRoPLx+LFixk3bhxt2rTBMAzmzJlz3MdN0+Thhx+mTZs2REVFMWrUKDZv3txYecVCU8b49v1458t8thS4LE4jIs3C3qXgyie71MU1rnLyw8LANFkfGUlOSSnZpU5wHfBdJ0GjweWjoqKCvn37Mn369JN+/G9/+xuPP/4406dPZ9WqVaSmpnLppZdSVlZ2zmHFWr3axHNFH9+q8sfma5hTRPyg/CAAHmBLRLjvOcMgzDTJLnWdcJ0EhwaXj7Fjx/LnP/+Za6655oSPmabJtGnT+N3vfsc111xDVlYWzz//PJWVlcyePbtRAou1fnVJV2wGfLzlIGv2llgdR0RCXUwKAHNiWtSXjzDTpNYwyE2IO+E6CQ6NuuZj9+7dFBYWMmbMmPrnIiIiGDlyJEuXnnxIzO1243K5jntI4OqcHMN1A9sC8NcPt2KapsWJRCSktR+KMy6N/3d0rceFlVWs3ZNXvwYkNyEe4tKh/VCLg0pDNGr5KCwsBCAl5fgGmpKSUv+x75o6dSrx8fH1j4yMjMaMJE1g8iVdCXfYWLn7CAu3HbY6joiEMpudOxNiqLbZSKzz8O+Dvu852aUuckqczEiMJ7fPGLDZLQ4qDdEkd7t890hj0zS/95jj+++/H6fTWf/Iy8trikjSiNokRDFxaAfAN/rh9Wr0Q0SaxradC9hq8x3t8Mcj1YR962PZ3hhy0kfjTepqTTg5a47G/M1SU1MB3whIWto3290eOnTohNGQYyIiIoiIiGjMGOIHd43qxCsr97G1sIy5Xx5gQv+2VkcSkRBjer08uuh+TIdB78oIhkxaDQUrfYtLY1Kg/VCyNeIRlBp15CMzM5PU1FQWLFhQ/1xNTQ2LFi1i6FDNx4WShOhwfjGqE+C788Vdp3vsRaRxzV/6D9Y63ISZJqM7/Y7oyAjIHA69r/P9qOIRtBpcPsrLy1m/fj3r168HfItM169fz759+zAMg8mTJ/Poo4/y9ttvs2nTJiZOnEh0dDS33HJLY2cXi90xNJOUuAj2l1Qxe8U+q+OISAipcZfxz20vADCwLI3bRl9tcSJpTA0uH6tXr6Z///70798fgClTptC/f3/+8Ic/AHDfffcxefJk7rrrLgYNGsSBAweYP38+sbGxjZtcLBcVbuee0b651n9/upOy6lqLE4lIqPjv+zkccBi0qvPyg/MeI8yuDblDiWEG2L2SLpeL+Ph4nE4ncXFxp/8EsVSdx8uYfy5mV1EFv7y4M1OOngEjInK2Dh3cxJXv30SVzWC4azDTc/6LzXbymxYkcDTk/VtVUs6Jw27jN5f5Csd/luzmoKva4kQiEuz+Pv9uqmwGnaoNbhrzqIpHCFL5kHN2eVYqA9snUlXr4XFtuy4i52D9ptl86C0GoB0/YkS3VIsTSVNQ+ZBzZhgGD/ygBwCvrcnToXMicla8njoeXfk3APq5Ysm5+hcWJ5KmovIhjWJg+0Su6J2GacLUD7ZaHUdEgtCcz+5ni91DC6+X9kn/R7dU3agQqlQ+pNHcd3k3wuwGi7cfZvF2bbsuImeuzHWAf+37AIBuRzrzyyvGnOYzJJipfEijad+qBbee3wGAR9/fgkfbrovIGXryo7s4YjdIrzHp2+sRkuMirY4kTUjlQxrVpIs7ExfpYGthGW+u3W91HBEJArv3LOSViq8BSCodzc8v6mVxImlqKh/SqBJbhDPp4i4APDZ/G5U1dRYnEpFA99dFv6XOMOhZEcaVI39Ni4hGPXZMApDKhzS624a2J6NlFAddbnIX7bI6jogEsMUr/skXVOIwTQz3T7h+UIbVkcQPVD6k0UU47Nw/1nfr7VOLvuZAaZXFiUQkENW4y/jr5mcB6F2STM6VN2DXhmLNgsqHNImxWamcl9kSd52Xv+rWWxE5iZfm38M+OyTWeYlO+A3Du7S2OpL4icqHNAnDMPjDlT0xDJj3ZT5r9h6xOpKIBJDDhzbzVNFKANoW9eO+K4dbnEj8SeVDmkxWejw3DPTN3z7yzld4deutiBw1bcEvqbQZdKw26NHjt3RsHWN1JPEjlQ9pUvde1o2YCAdf7nfy9roDVscRkQCwYfOrzKs7BICjZDy/vLSHxYnE31Q+pEm1jo0g56LOAPzto61UuHXrrUhz5vXUMXXl/wOgr6sFE0b9hPioMItTib+pfEiT+/GwDrRrGc1Bl5snF35tdRwRsdA7ix5kk62OaK8Xd10ONw3WrbXNkcqHNLkIh73+1Nunl+xib3GFxYlExAoV5YVM2/MuAF2LM5k8biwOu96GmiP9Vxe/uKxXCsO7JFFT5+VP735ldRwRscBTH95Fkd0grcZLfPJ9XNg5yepIYhGVD/ELwzB4aFwvHDaDj7cc4tOtB62OJCJ+tHfvEl4s3w5AQtFI7r9qsMWJxEoqH+I3nZNj+PGwTAD++M5XVNd6LE4kIv7yt4X3UWcY9KhwMGzQPaQnRFkdSSyk8iF+NeniziTHRrC3uJL/fr7b6jgi4gdLVj7BYspxmCaeion8bFRnqyOJxVQ+xK9iI8PqF59O/3Qn+Tr3RSSk1bor+Oum/wDQu7Q1d115E5FhdotTidVUPsTvru7XhsEdEqmq9fCX97dYHUdEmtDLC+5h79HzW6LifsPoHilWR5IAoPIhfmcYBn+8KgubAe9tKODzHUVWRxKRJlBUtJWnDi8DIKO4D/dfNdLiRBIoVD7EEj3bxHHr+e0B+P3cTVp8KhKCps2/m3Kbjcxq6Nf7QToktbA6kgQIlQ+xzK8v60br2Ah2F1Xw1KJdVscRkUa0ecubzKspBCCmbAKTLulucSIJJCofYpm4yDB+f2VPAGYs3MmeIu18KhIKTK+Xvyz7M6Zh0McVzZ1X3q1FpnIclQ+x1Lg+aQzr7Nv59PdzN2GaptWRROQcvbPo92y01xHl9dI69tdc1D3Z6kgSYFQ+xFKGYfDI1b0It9tYsqOI9zYWWB1JRM5BRXkh/9w9F4CeJR347YRxFieSQKTyIZbr2DqGX4zqBMAj73xFWXWtxYlE5Gw9+f7R81tqTS7s/yhp8drJVE6k8iEB4RejOtGhVTSHytw8Nn+71XFE5Czs2/c5syt9//92qLiYO0b0tjiRBCqVDwkIkWF2Hrk6C4Dnl+1h3b4SixOJSEM9+sm91BoGPSod/Oyqh3DY9RYjJ6e/GRIwRnRtzYT+6Zgm3P/WRmo9XqsjicgZWrR8Ol/YKnCYJr0SJjEos5XVkSSAqXxIQHnwih4kRoextbCMpxdr7w+RYFBbW8nfNz0FQP+y1kyZcKvFiSTQqXxIQGkVE8Efxvn2/vjXJzvYdbjc4kQiclJeD+xeAhvf4L+v3cLeMEjweLl26D+JjQyzOp0EOIfVAUS+a3y/dN5el8/i7Ye5/62NvHLn+dhshtWxRASYuX4mtqLtZG/4CFz5HLHZeKFtG7DbyKhLYK99KdDP6pgS4DTyIQHHMAz+Mj6LqDA7K3Yf4dXVeVZHEpGjbEXbmXHgE3JtvlHJJxITKLPbSKqrY2OEC1uR7laT01P5kICU0TKaX4/pCsCj72/hkKva4kQigtdD9oaPyCkpZUZiAn9qlchbsb7D4oocDnJKnGRvmO+bkhE5BZUPCVh3XJhJ37bxlFXX8cDb2npdxHJ7l4Irn+xSF3eVlPJaXCym4ZsSzSkpJbvUCa4DvutETkHlQwKW3Wbw1+v6EGY3+HjLQeZ9mW91JJHmrfxg/U+jvd/8YyDMNMkudZ30OpGTUfmQgNY9NY5fXtwFgIfmbeZQmaZfRCwTkwLA5vAwHm+ZAIDdNKk1DHIT4k64TuT7qHxIwMse1YlebeIorazlQU2/iFin/VDK49L4eWoyXsOgY00N6/bk1a8ByU2Ih7h0aD/U6qQS4FQ+JOCF2W384/q+OGwG8786yDsbdPKtiCVsdn4UH4XTbifG4+GFgkMYQHapi5wSJzMS48ntMwZsdquTSoBT+ZCg0CMtjknHpl/mbuJwmdviRCLNz/8++i1fO+owTJPHit3Ee785AiHbG0NO+mi8SV0tTCjBQpuMSdC466JOfLi5kC0FLn4/ZxNP/mgAhqHNx0T8Ye/eL/hn/rtgszG8sgPn/2oO5C3zLS6NSYH2Q8nWiIecIY18SNDwTb/0wWEz+HBzIW+vO2B1JJFmodZdwa8/votKm42uVTZ+de2L2BwOyBwOva/z/ajiIQ2g8iFBpVebeO4ZfWz6ZTMHSqssTiQS+v7+9i1sc3iJ9XgZ23EqnVMSrY4kQU7lQ4LOL0Z1on+7BMrcddz72pd4vbr7RaSpLF75b15x+06YPs99CT++ZKzFiSQUqHxI0HHYbTx+Qz+iwuws21XMrKV7rI4kEpKKirby4KZcAAa6Evm/W/6qQx6lUah8SFDKTGrB767oAcBfP9zK9oNlFicSCS1eTx3/985tlNhttHObXH3BU6TGR1odS0KEyocErR8Oaceobq2pqfPyq1fXU1PnPf0nicgZmfX+z1lhqyLCa9IrfBITBvWwOpKEEJUPCVqGYfC3a/uQEB3G5nwX//xYR3mLNIZNX73O9OIVAPQtyeKBG+6wOJGEGpUPCWrJcZFMndAbgNxFX7N0Z5HFiUSCW0V5Ib9Z9gh1hkFWeQS3X/EvEqLDrY4lIUblQ4Le2N5p3DQ4A9OEX722niMVNVZHEglaf3z7JvY7ILnWS5/0vzGimw6Jk8an8iEh4Q/jetKpdQsOutzc98YGHT4nchbmfvoAH3iLsZsmmZXX8+srR1kdSUKUyoeEhOhwB0/c3J9wu42PtxzkpeV7rY4kElT27l3Co3vnAtDvSAYP3Hwv4Q69RUjTaPS/WQ8//DCGYRz3SE1NbewvI3KCXm3i+e3Y7gD8+b0tbCvU7bciZ6LWXcG9n0yq3z597LDpdGwdY3UsCWFNUmt79epFQUFB/WPjxo1N8WVETvDjCzswqltr3HVeJr2ylqoaj9WRRALe43NvYavdQ5zHS4eoB7hhcEerI0mIa5Ly4XA4SE1NrX+0bt26Kb6MyAkMw+Af1/clKSaC7QfLeWjeJqsjiQS0xSv+xUtVvu3Tu5RcxB9uuEanRUuTa5LysWPHDtq0aUNmZiY33XQTu3bt+t5r3W43LpfruIfIuUiKieCJm/phM+C11ft5c81+qyOJBKSiw1t4cPPTAPQvTWDyDX8mPirM4lTSHDR6+RgyZAgvvPACH330Ef/5z38oLCxk6NChFBcXn/T6qVOnEh8fX//IyMho7EjSDA3tnMQ9o7sC8OCcTezQ9usix/F66rjvXd/26e3dMLLPDPplJFgdS5oJw2ziexIrKiro1KkT9913H1OmTDnh4263G7fbXf9rl8tFRkYGTqeTuLi4powmIc7jNbn92ZV8vrOIzskxzLv7QqLDHVbHEgkIT82dyPTSNUR4TUZ4J/HYj3+m6RY5Jy6Xi/j4+DN6/27y+6hatGhB79692bFjx0k/HhERQVxc3HEPkcZgtxlMu6kfybER7DxUzoNvb9L+HyLAhs2vkVuyGoB+zr48dMuPVTzEr5q8fLjdbrZs2UJaWlpTfymREyTFRPDvm/tjM+CtdQd4dVWe1ZFELFVeVsBvlvu2T+9dHsk9107XOg/xu0YvH/feey+LFi1i9+7drFixguuuuw6Xy8Xtt9/e2F9K5IwM6diKX4/pBsAf5m5mfV6ptYFELGJ6vTz45o3kOwySa72M6fkEvTMSrY4lzVCjl4/9+/dz8803061bN6655hrCw8NZvnw57du3b+wvJXLGfjGyE5f2TKHG4+UXL62hqNx9+k8SCTGvzP8tnxgl2E2TIfbbuH3U+VZHkmaqyRecNlRDFqyINERZdS1XT/+CXUUVnN+xJS/9ZAgOu7aPlhDm9cDepVB+kB1lBfxwWy5VNhvDXJn8M/ttIsPsVieUENKQ928t/ZdmIzYyjKduHcj4GV+wfNcR/vrhVn53RU+rY4k0upnrZ2Ir2k72ho/AlU8N8Ls2qVRFhJNca5IxYJSKh1hK/+yTZqVLSiz/uL4vAP9Zspt3vsy3OJFI47MVbWfGgU/ItZUDMK1lAlsiwon0ejkUZtCyap/FCaW5U/mQZmds7zR+MaoTAPe9sYHN+U6LE4k0Iq+H7A0fkVNSyozEBO5PasmL8b4h8GqbjZwSJ9kb5vumZEQsovIhzdK9Y7oxvEsSVbUe7nx+NYfLtABVQsTepeDKJ7vUxW1OF+/GfnM6bU5JKdmlTnAd8F0nYhGVD2mW7DaD6TcPoGNSC/Kd1fz8xdVU1+pfghICyg8CcMRmY3FUVP3TYaZJdqnrhOtErKDyIc1WfHQYz9w+iLhIB2v3lfLAWxu1A6oEv5gUKg2DnNTW7An3bR4WZprUGga5CXHHXSdiFZUPadY6to5h5g8HYrcZvLXuAE8t/v4TmEWCQW1qX+5JSWZTRAQAtzhdrN2TV78GJDchHuLSof1Qi5NKc6byIc3esC5JPDTOd8vtXz/cyoKvNBwtwcnrqeNXr/yA5VG+4nGdq4z7j5QCkF3qIqfEyYzEeHL7jAGbbrUV66h8iAC3XdCBH53fDtOEX76yjg37S62OJNIgptfLQ6+MY5HdiWGaDPGk8VBt1HHXZHtjyEkfjTepq0UpRXy0yZjIUQ+N68W+I1Us3n6YHz+3irfvupCMltFWxxI5I0+8dStzPPsBuKxmGH/96ZOAt36HU2JSoP1QsjXiIQFAIx8iR4XZbcz84QB6psVRVF7DxFkrKa2ssTqWyGm99P5knqnYAMDI8u786Y4Z2GyGb2olczj0vs73o4qHBAiVD5FviYlwMOuOwaTFR/L14Qp+9uIa3HW6BVcC14dLpvL3Qx8DcL4rlUcmvqyt0yXgqXyIfEdKXCSz7hhMbISDlbuPcO/rG/B6dQuuBJ4V657ldztfxmsYDCiL5YEb36Jli3CrY4mclsqHyEl0T40j99aBOGwG73yZz5/e+0p7gEhA2bbjPSavf4wam0FWRTh3X/EWmcmxVscSOSMqHyLf48LOSfWH0M36Yg/TP91pcSIRn7z9y/j5kvsot9noUmXj1qGzGZyZanUskTOm8iFyCuP7p9fvAfLYgu28uHyvxYmkuSsu2s7PPrqTYruNDLfJdb2e4gf9ulkdS6RBVD5ETuOOCzP55cWdAfjD3E2882W+xYmkuaooL+Rnc69nv8MgudbLVe3+wS3Dz7c6lkiDqXyInIFfXdq1fhOyKa+tZ9H2w1ZHkmamxl3GXa9eyXaHl3iPlzEJD5B9+eVWxxI5KyofImfAMAz+eFUWV/ZJo9Zj8rMXVrP06yKrY0kz4amrYcrsy1nrcBPl9XKx46fcd+0tVscSOWsqHyJnyG4zePyGflzcPRl3nZefPLealbuPWB1LQo3XA7uXwMY3YPcSzLpafj97HItsLhymyUWeCTz0o8kYhmF1UpGzZpgBdv+gy+UiPj4ep9NJXFzc6T9BxM+qaz3c+cJqluwookW4nRd/OoQB7RKtjiVBbub6mdiKtpO94SNwfbOu6MmWScyMjwbT5HL3CB796XTC7Pp3owSehrx/62+wSANFhtn5z22DGNqpFRU1Hm5/dqUOopNzZivazowDn5BrK69/7vXYFr7iAfSqieUvP1HxkNCgv8UiZyEyzM4ztw/ivA4tKauu49b/rmTjfqfVsSRYeT1kb/iInJJSZiQmkJsQx8fRUfypVUsABlZV80q5k3BbQA1Ui5w1lQ+RsxQd7uDZOwYzoF0CzqpabnlmOWv2llgdS4LR3qXgyie71FVfQH6VnIRpGPSsdjOr8BCG64DvOpEQoPIhcg5iIhw8/+PzvjUCsoJlXxdbHUuCTfnB+p+2r60D0wTDwDBNXi44iHGS60SCmcqHyDmKjQzjuR8PZljnJCprPEyctVL7gEjDxKRgArPiY7kvOam+eJiGwTMJccddJxIKVD5EGkF0uINnbh9Ufxvunc+vZsFX+leqnBlP+iD+ktSax1v67prqU13Nuj1531oDEg9x6dB+qMVJRRqHyodII4kMs5P7o4GMzUqlxuMl+6U1vLY6z+pYEuCqKo+Q/eIoXo2NAuDCyipeLjiEHY6uAXEyIzGe3D5jwGa3NqxII1H5EGlE4Q4b/765P9cOaIvHa3LfGxuYuXAnAbadjgSI4uId3Db7IpY7KrGZJgNrW/FkVdhx12R7Y8hJH403qatFKUUan8PqACKhxmG38Y/r+9A6NoLcRV/ztw+3ccjl5g9X9sRm066U4rNrzxLu+uQXHAgziPN4uS7ux0y+dgqG6fXd1VJ+0LfGo/1QsjXiISFG5UOkCRiGwf+N7U7r2Aj+9O5XPLd0D0Xlbh67oS8RDr2RNHervnyZKWsepdRhI6XWy63tHuL2MTf4PmjYIXO4tQFFmpjKh0gT+smwTFrHRvDr19bz7oYCDrnc5N46kJYtwq2OJhaZ99lfeGTPK7jtNjLdcGefpxh3/jCrY4n4ldZ8iDSxq/q2YdbE84iNcLByzxHGz/iCnYfKrI4lTe07B8Th9fDk2z/jwb2v4LYZ9KoM4/6Rb6p4SLOkg+VE/GTHwTJ+/Pwq8o5UERvpYOYPBzC8S2urY0kjO9kBcV7gsVZJvBDnO6flvPIEHrxuLpkpLS1MKtK4dLCcSADqkhLLnLsuZHCHRMqq65g4axUvLtujO2FCzHcPiHMb8NvWreqLR3d3FI/f8bGKhzRrWvMh4ketYiJ46adDuP+tjby19gC/n7uZDfud/Gl8FpFhWoga9I4eEIetnBmJCbgNg3UREayJigTgkvIKHnd7MCL1rVeaN418iPhZhMPOY9f35f/GdsdmwOtr9nNd7lLyjlRaHU3O1bcOiPuh08UzCfH1xePqsnL+ebhYB8SJoPIhYgnDMMge2YkXfzKEli3C2XTAxbjpn7Nw2yGro8m5KD+ICbwe24K3YmPqn3aYJn8uOnLcdSLNmcqHiIUu7JzEO5OG0bdtPKWVtdzx3Coen7+NOo/X6mhyFgrr3PwyOYlHklpRZfN9ew0zTeoMg1wdECdST+VDxGLpCVG8ln0BN5/XDtOEJz7dyY1PL9c0TJCZ8+lfuWHdoyxsEY3t6CLiu0pKWasD4kROoPIhEgAiHHamXtObJ27uT2yEgzV7S/jBE0t458t8q6PJaVRWFvObWZfw+7yXKLHbSKjz4jUMckqc/KLUBeiAOJHv0pJrkQByVd829M9I4J7/rWPtvlImvbKORdsP84dxPYmLDDv9byBNx+s54cyVlRvf4uFVfyQvzHdmz7DKFNr1GUNiVT7ZxR8BzvpPz/bGgA6IEwG0yZhIQKrzeHnikx1M/2wnXhPS4iN59JreXNQt2epozc7JNg3zAM8kJjEzPgqvYdCqzstVsbdwz3UPYD92eOBJyopGPCSUNeT9WyMfIgHIYbcxZUw3hndtzW9e/5I9xZXcMWsV1w1sy++v6El8tEZB/OXYpmHYyskGDjjsPNC6FWsjfbfQtq0xeGjY/zi/R+/vfKIOiBP5Phr5EAlwVTUe/jF/G89+sRvThOTYCB65OovLeqVgGIbV8UKb1wPTssg9umnY6IpKlkdFUnH0TpbRFZU8Xh2BbfJGjWpIs6ft1UVCSFS4nd9f2ZPXf34BHZNacKjMTfZLa7jjuVXsKaqwOl5oO7pp2LDKatJqa/mkRXR98bjV6WLaoSJs2jRMpMFUPkSCxKAOLXn/nuHkXNSJMLvBwm2HGfPPxTw+fxvVtR6r4wWvk5w+e8yevcu4v3Urbk5PpSAsDI4OFIeZJvcdKf3m99CmYSINomkXkSC063A5D83bzJIdRQC0TYziN5d1Y1yfNthsmoo5EydbSApAXBue6DGKJYc2s9tbjPvoSEc3dw3bIsIJM01qDYOcklKyj95Ky+3van2HNHuadhEJcR1bx/DCj8/jyR8OIC0+kv0lVdzzv/VcNeNzPj9aSOTUvnv6LPjuYvllZA3/OfQ5WynBbbPRv9rN9a4ytkWEk6NNw0Qahe52EQlShmEwtncaI7u15tnPd5O7aBebDrj40X9XMLxLEvdd1p3ebeOtjhkYvnvba8aQ406fBehX7eb/Wrei2OH7tphRW8foFuOITK0mt/Azckqc9SMdvh8NZiTGQ/posrXYVKRBNO0iEiKKy91M/2wnLy3fS63H97/1yK6tufvizgzu0NLidNb43qmV6CRyw2vwYLDfYefdbx0CF+718ssSJze7ygi//V1mOjd+z/RMOrl9xuBN6spd/e7y459KJDA15P1b5UMkxOwrrmTax9uZ+2U+Hq/vf+/zMlty90WdGd4lqVndnpv78WRmHPjk+PUZwPSEOJ5KTCCxro4SxzcDwDbTZOG+AyR6jx7sd+1/ofd1vp9r0zCRU1L5EBH2FVfy5KKveWNNXv1ISJfkGG67oD0TBrQlJiIEZ12/XRBatIa3f06uvYIZiQnklJRyk6uce5NbsSIqqv5Twrxeam02LSQVOUcqHyJSr8BZxdOLd/Hqqjwqa3y3kcZEOLhuYFtuGdKOrimxFic8d987vQLkJsTxWXQUX0VE+G6VPTryk1JXR6eaWpZGR9UXjtyEuKNFxek7i0Wbh4mcMW2vLiL10uKjeGhcL351aVfeWrOfF5btZVdRBc8t3cNzS/eQlR7H+H7pXNWvDcmxkVbHPSvf3QIdIM9h5+Gklqz81igHhoFhmjx6uJh9YQ6ePDoiooWkIv6lkQ+RZsbrNfni6yJeWLaXz7Yeou7ouhCbAcO6tOayXimM7p5CanyQFJHvbIE+sqISl93Gushv8ju8XupsNhymSd3RqRUvBjZMsmvCoLL4m99PC0lFzkpATLvMnDmTv//97xQUFNCrVy+mTZvG8OGnnztV+RDxnyMVNby3IZ+31h1g3b7S4z6WlR7H6O4pjOrWmqz0eMLsgbctUE2dl92rPqTbRzcDcGdKa5ZHfzPSkVFbS9ca37bo3zu18sv1kLdCC0lFzpHl5ePVV1/l1ltvZebMmVx44YU89dRTPPPMM3z11Ve0a9fulJ+r8iFijT1FFby3sYCPtxxkfV4p3/7OEB1uZ0C7RM7LbMngDi3plR5HXKT/T9Z1Vtay8YCTlXuOsGr3EdbllTDG8zlPhE8HfCfOjm3bBtMwCDNNflbqrF9s+u27XXIT4pmRGE9O+miyL5nm9z+HSCiyvHwMGTKEAQMG8OSTT9Y/16NHD8aPH8/UqVNP+bkqHyLWKyp38+nWQ3yy5SDLdx3BWVV7wjVtE6PonhpHj7RYOifH0DYxivSEaFrHRmA/hy3ePV6Tg65qDpRWcaCkiu0Hy9haWMbWAhf5zuoTrr80ejv/8T4MUD+qcezOlcFV1ZxXXX1c8QA0tSLSBCwtHzU1NURHR/P6668zYcKE+ufvuece1q9fz6JFi075+SofIoHF6zXZcaiclbuLWbH7CGv3lpy0BBwTZjdIjY8kMTqcuMgwYiMdxEWGERVu59tbjJgmVNbUUVZdh6u6FldVHUcqaih0VdfvT3IybROjGNQ+kfMyW3FeZiKdWkVh/Ks3ubYK32jG902vXD0TKos0tSLSRCy926WoqAiPx0NKSspxz6ekpFBYWHjC9W63G7fbXf9rl8t1wjUiYh2bzaBbaizdUmO59YIOgG/6Y2uhiy0FLrYUlLG7uIIDJVUUuqqp9ZjkHaki70jVWX9Nh80gLSGS9IQoMpNi6JkWS/e0OLqlxp50uie3z2VHNxM7xRbonUaddR4RaVxNdqvtd3dRNE3zpDsrTp06lT/+8Y9NFUNEmkB8dBhDOrZiSMdWxz1f5/FysMxNQWkVzqpaXNW1vpGNqlqqaj0n/D7R4Q7iIh3EHh0hSYgOo01CFMmxkQ2auvEmdSUHyC7+CHDWP5/tjYH00XiTup7tH1VEmoDl0y4nG/nIyMjQtIuINJy2QBexjKXTLuHh4QwcOJAFCxYcVz4WLFjA1VdffcL1ERERRERENHYMEWmObHZthy4SBJpk2mXKlCnceuutDBo0iAsuuICnn36affv2kZ2dffpPFhERkZDWJOXjxhtvpLi4mEceeYSCggKysrJ4//33ad++fVN8OREREQki2l5dREREzllD3r8Db79kERERCWkqHyIiIuJXKh8iIiLiVyofIiIi4lcqHyIiIuJXKh8iIiLiVyofIiIi4lcqHyIiIuJXKh8iIiLiV02yvfq5OLbhqsvlsjiJiIiInKlj79tnsnF6wJWPsrIyADIyMixOIiIiIg1VVlZGfHz8Ka8JuLNdvF4v+fn5xMbGYhiG1XEs53K5yMjIIC8vT2fdNCG9zv6h19l/9Fr7h17nb5imSVlZGW3atMFmO/WqjoAb+bDZbLRt29bqGAEnLi6u2f/F9ge9zv6h19l/9Fr7h15nn9ONeByjBaciIiLiVyofIiIi4lcqHwEuIiKChx56iIiICKujhDS9zv6h19l/9Fr7h17nsxNwC05FREQktGnkQ0RERPxK5UNERET8SuVDRERE/ErlQ0RERPxK5SMIud1u+vXrh2EYrF+/3uo4IWfPnj385Cc/ITMzk6ioKDp16sRDDz1ETU2N1dGC3syZM8nMzCQyMpKBAweyZMkSqyOFlKlTpzJ48GBiY2NJTk5m/PjxbNu2zepYIW/q1KkYhsHkyZOtjhI0VD6C0H333UebNm2sjhGytm7ditfr5amnnmLz5s3885//JDc3lwceeMDqaEHt1VdfZfLkyfzud79j3bp1DB8+nLFjx7Jv3z6ro4WMRYsWkZOTw/Lly1mwYAF1dXWMGTOGiooKq6OFrFWrVvH000/Tp08fq6MEFd1qG2Q++OADpkyZwptvvkmvXr1Yt24d/fr1szpWyPv73//Ok08+ya5du6yOErSGDBnCgAEDePLJJ+uf69GjB+PHj2fq1KkWJgtdhw8fJjk5mUWLFjFixAir44Sc8vJyBgwYwMyZM/nzn/9Mv379mDZtmtWxgoJGPoLIwYMHufPOO3nxxReJjo62Ok6z4nQ6admypdUxglZNTQ1r1qxhzJgxxz0/ZswYli5dalGq0Od0OgH0d7eJ5OTkcMUVV3DJJZdYHSXoBNzBcnJypmkyceJEsrOzGTRoEHv27LE6UrPx9ddf8+9//5vHHnvM6ihBq6ioCI/HQ0pKynHPp6SkUFhYaFGq0GaaJlOmTGHYsGFkZWVZHSfk/O9//2Pt2rWsWrXK6ihBSSMfFnv44YcxDOOUj9WrV/Pvf/8bl8vF/fffb3XkoHWmr/W35efnc/nll3P99dfz05/+1KLkocMwjON+bZrmCc9J47j77rvZsGEDr7zyitVRQk5eXh733HMPL730EpGRkVbHCUpa82GxoqIiioqKTnlNhw4duOmmm3jnnXeO+0bt8Xiw2+388Ic/5Pnnn2/qqEHvTF/rY99M8vPzueiiixgyZAjPPfccNpu6+tmqqakhOjqa119/nQkTJtQ/f88997B+/XoWLVpkYbrQM2nSJObMmcPixYvJzMy0Ok7ImTNnDhMmTMBut9c/5/F4MAwDm82G2+0+7mNyIpWPILFv3z5cLlf9r/Pz87nssst44403GDJkCG3btrUwXeg5cOAAF110EQMHDuSll17SN5JGMGTIEAYOHMjMmTPrn+vZsydXX321Fpw2EtM0mTRpEm+//TYLFy6kS5cuVkcKSWVlZezdu/e45+644w66d+/Ob3/7W01znQGt+QgS7dq1O+7XMTExAHTq1EnFo5Hl5+czatQo2rVrxz/+8Q8OHz5c/7HU1FQLkwW3KVOmcOuttzJo0CAuuOACnn76afbt20d2drbV0UJGTk4Os2fPZu7cucTGxtavp4mPjycqKsridKEjNjb2hILRokULWrVqpeJxhlQ+RL5j/vz57Ny5k507d55Q7DRQePZuvPFGiouLeeSRRygoKCArK4v333+f9u3bWx0tZBy7jXnUqFHHPT9r1iwmTpzo/0Ai30PTLiIiIuJXWkEnIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+9f8Bor1Qczd146QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_39_2.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def function(x):\n",
    "    return x**2\n",
    "\n",
    "def analytical_gradient(x):\n",
    "    return 2*x\n",
    "\n",
    "def gradient_descent(starting_point, learning_rate, num_iterations, solver=\"analytical\"):\n",
    "    x = starting_point\n",
    "    trajectory_x = [x]\n",
    "    trajectory_y = [function(x)]\n",
    "\n",
    "    if solver == \"analytical\":\n",
    "        grad = analytical_gradient    \n",
    "    elif solver == \"jax\":\n",
    "        grad = jax_grad(function)\n",
    "        x = jnp.float64(x)\n",
    "        learning_rate = jnp.float64(learning_rate)\n",
    "\n",
    "    for _ in range(num_iterations):\n",
    "        \n",
    "        x = x - learning_rate * grad(x)\n",
    "        trajectory_x.append(x)\n",
    "        trajectory_y.append(function(x))\n",
    "\n",
    "    return trajectory_x, trajectory_y\n",
    "\n",
    "x = np.linspace(-5, 5, 100)\n",
    "plt.plot(x, function(x), label=\"f(x)\")\n",
    "\n",
    "descent_x, descent_y = gradient_descent(5, 0.1, 10, solver=\"analytical\")\n",
    "jax_descend_x, jax_descend_y = gradient_descent(5, 0.1, 10, solver=\"jax\")\n",
    "\n",
    "plt.plot(descent_x, descent_y, label=\"Gradient descent\", marker=\"o\")\n",
    "plt.plot(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a994430",
   "metadata": {
    "editable": true
   },
   "source": [
    "### A more advanced example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "863db98d",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/jax/_src/numpy/lax_numpy.py:173: UserWarning: Explicitly requested dtype float64 requested in asarray is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more.\n",
      "  return asarray(x, dtype=self.dtype)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x126323b50>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGdCAYAAAAvwBgXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAAB8SklEQVR4nO29eZhcZZn3/z21dFUvVb2mO0t3VhAStkgCCQjIJsuIijqMDMgMDvq+/AQGZGZwAiqo+ObngAsygjLziigKuKKOiGFU0BEiISxhSyBJJ+mkt/Ra3dVd+3n/qHrOOd2pqj7L85zzVNf9ua5cF+l0dz0k1efc53t/7++tqKqqgiAIgiAIwgN8Xh+AIAiCIIjqhQoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8I+D1AcqRy+XQ29uLSCQCRVG8Pg5BEARBECZQVRUTExNYvHgxfL7ymofUhUhvby+6urq8PgZBEARBEDbo6elBZ2dn2c+RuhCJRCIA8v8j0WjU49MQBEEQBGGGWCyGrq4u7T5eDqkLEdaOiUajVIgQBEEQRIVhxlZBZlWCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCkIQ9hyfxrWf2IJHOen0U15B6+y5BEARBVBNf3fIWfv1qH9oaQvjrdZ1eH8cVSBEpQzqbQyab8/oYBEEQRJVweCIJAOgZmfL4JO5BhUgJ4skMTtv8e3zkga1IZagYIQiCIMQTS6QBAIOFgqQaoEKkBPuG4xiaTGL7/lF88w+7vT4OQRAEUQVMJDIAgMFYwuOTuAcVIiVIGlSQb/5hN97si3l4GoIgCKIamCBFhGAk03ohksmpuOUnO8gvQhAEQQhDVVVMJvOKyAApIkQikx+dWtJUi2g4gFcPjeM//6fb41MRBEEQ85V4Koucmv/vockksuw38xwqRErAFJGFjWF85pI1AICvPvUW9hye9PJYBEEQxDwlNp3W/junAsOT1dGeoUKkBMmCIhIK+HDZuk6ceXQbUpkc/vWnO5CrkiqVIAiCcA9mVGUMxKgQqWqYWTUU8EFRFGz+0Amor/Fj275R/NerfR6fjiAIgphvMKMqY3CiOnwirhUimzdvhqIouOmmm9x6SUckC/G64aAfANDZXIdLTlwMAOg+HPfsXARBEMT8hBQRgWzbtg0PPPAATjzxRDdejgtGRYRRF8oXJczIShAEQRC8iM1SRKplckZ4ITI5OYkrr7wS//Ef/4Hm5mbRL8cNvRDxax9j6kg1LSMiCIIg3GG2IlItWSLCC5HrrrsO733ve3H++eeLfimusGIjFNT/isKFoiRJke8EQRAEZ1ghoij531dLuqrQ7buPPvooXnzxRWzbts3U5yeTSSSTegUYi3mXZsqKDaaCAHpRQooIQRAEwRtmVu1qrsOBkSlSRJzS09ODG2+8EQ8//DDC4bCpr9m8eTMaGxu1X11dXaKONyfMrGr0iIQL/02KCEEQBMEbpoisWlAPgDwijtm+fTsGBwexbt06BAIBBAIBPPPMM/jGN76BQCCAbPZIVWHTpk0YHx/XfvX09Ig63pwk0keaVUMFdSRJighBEATBGaaIHNXeAKB60lWFtWbOO+88vPrqqzM+9rGPfQzHHnssPv3pT8Pv9x/xNaFQCKFQSNSRLMECzYytmbDWmiFFhCAIguALU0SWt9XDp+jpqu1Rc12FSkVYIRKJRHD88cfP+Fh9fT1aW1uP+LiMFBvf1c2qpIgQBEEQfGGFSFNtDdoaQhicSGIgNv8LEUpWLUGx8d0QKSIEQRCEIFiOSCQcQHs03x2ohnRVoVMzs3n66afdfDlHlB/fJUWEIAiC4AtTRCLhADoiYbyGmNB01b7xabzRG8OCSAgndjYJe525IEWkBKSIEARBEG4yoSkiQa0dI3JyZuveYVzz0Au467e7hL2GGagQKYG2fdegiLCihHJECIIgCJ6oqorJZF4RiYYDaI+w1ow4RSRVeOCu8XtbClAhUoJi47tsgsarHJHx6TRe7x335LUJgiAIccRTWbBJ3Ug4iI6CIiIyXVUrRALelgKuekQqiWLju6wocVsRSaSzePDP+3D/07sRS2Tww49vwOlHtbl6BoIgCEIcrC0T8CkIB32uKCJJKkTkJjmHIqKqKhS2EEAQmWwOP95+EF//77dmGJb2DU/h9KOEvjRBEAThIkajqqIomiIi0iOSyubvc0GPWzNUiJSg+PZd34w/N6olvBmNp/CRB57DWwOTAIAlTbVoCAWwa2ACk8n0HF9NEARBVBJGoyoAdBTGd1m6qt/H/8E3ncn3grxWRMgjUgLWfgkXMasCumIiihcPjOKtgUnU1/jx2UvW4Pf//G6cuqIFADA5a1U0QRAEUdnEDIoIALQ2hGakq4ogVVi1QmZVCVFVtagiEvQrYEWp6CwR5p4+qasJ15yxAqGAHw2FN+hEkgoRgiCI+cTErELE71PQ1pBXRURliaSKJIh7ARUiRWB9M2Dm+K6iKIYRXrGKyFQqX+jU1ejds4ZQ/r9JESEIgphfzG7NABCerirL1AwVIkUwFhmzK0Vt8Z1gRSReUD3qQ7oiwyrlSVJECIIg5hWzFREA6Igww6qo1owcZlUqRIrA2i6KcmTvTJuc8VIRoUKEIAhiXsEUkegMRUTs5EyKzKryYhzdnT2iq2WJiFZEUgVFpEZXRFghMkGtGYIgiHlFbPpIRUR0lghTRMisKiHFjKoM1xSRZEERCRkUEWrNEARBzEuKKSKi01VThQdqUkQkpNjoLsOtdNViikgklH+DklmVIAiCD8OTSSke7op5RIQrImRWlZdyikiooIiIbs2QIkIQBCGWiUQaZ9/1NM7/yjM4NDbt8VlYIXKkIiLMI0KtGXnRNu8WqRLdas2U84hMJjPIse1IBEEQhC0OjExhIplBfyyBf3hwG2IJ71KrY9r4rmFqZla6Km8oWVViWJFRLMLdLbNqsakZ4xuUFSoEQRCEPcam9MJj18AEPvnwi0hnvdmuXqw1IzpdNUmKiLyYUUREB5oVyxEJBXwIFKJdqT1DEAThjJF4CgDQ2VyLuho//mf3ED7z89egqu4rzsUCzUSnq5JHRGI0j0gRs2q48A8mOuK9mCKiKIruEyHDKkEQhCNGp/KFyAlLGnHv374TPgV47IUe3Pf0HlfPoaqq9nAZDc/cRatNzghIV6WpGYlhEzHFzapsakZ0oNmRighgyBIhRYQgCMIRo/G8CtFcX4PzVnfg8+8/DgBw12934c+7h1w7RzyVBbOAGBURQJ+cEaGIpLP5F6VkVQlhikix8d1woTgRrYjEC1Mz9TUzq2PaN0MQBMEHpog01+Vv/ledthzvO2kxAOCPbx927RysLRPwKUfcd0Smq9LSO4nRk1VLKyIip2ayORXTadaamXkG2jdDEATBB+YRaa6r0T521IIGAO4mWBuNqrPTvEVmiWjju1SIyEdZs6q2fVecIjJt+N71IVJECIIgRMAUkZZ6vRBhD3uxafdGeYsZVRki01U1syq1ZuQjUWZ8V8sRyYhTRKYKaodPObIYaii8UckjQhAE4Qy9NXNkIeKmIhIrMrrLYFkiQhSRwn0sSIqIfJRTRHSzqjhFJJ7S/SGzZTpSRAiCIPhgNKsyorWFhz0Xw82KZYgw2iN5RaSfsyKiqiolq8pMsoyBRzerilNEWIZIXehIRUb3iHiXAMg4PJHEuV95Gvf899teH4UgCMIys82qgKE146pHpHRrpqlOTGHEJmYA8ohIiTa+WyxZ1QVFZCpVfGIGmBnz7jXPvHUYew/H8X//Z6+Q+GGCIAhRJNJZ7Vo7QxEJy6WIaAMSmRzXoLWUIUGWpmYkpJwiEnLBrMri24spIlqOiAStmX1DcQD5J4c3emMen4YgCMI8LN494FMQMQwF6IWI+4pItIgiwu45qjpTxXBKyqDqU46IhGjju0XNqnp1Kgpt824xRUSi8d3u4bj238/ucS/8hyAIwilsdLeprmaGF4+pElOprGt7Z8oqIoYHYp75VawQ8fsU+H3KHJ8tFipEilDWrOqiIlJfU8QjIpFZdf+MQmTYw5MQBEFYQx/dnalCNBiKAbeus+YLEX6FUVoSoypAhUhRyo/vio94n9LMqvIqIqqqYt/QlPb757tHZkh9BEEQMsMKkSbD6C6Qb1PUFq79brVnyplVFUXRzKQ8C5GkJAvvACpEimJm+67QqRnNrCqvR2RoMoXJZAaKknd1T6ezeOXgmKdnIgiCMMtooTXTMqsQAYBoLZucccewWi5HBNDvRUmOSrwsm3cBKkSKUt6syv8NMRu28K6YR0SWiHfWllncWIt3HdUGAHh2N7VnCIKoDEaKZIgwmDLhViGit2aOVEQA3RLA8wFYlgwRgAqRojD/h1fJqtrCu6JTM/k36mQyw3WUyyrdhYmZFW31OH1VKwAyrBIEUTkUyxBhuJ2uqrdm5lBEeBYipIjIjRlFJJXNCcvOKKeIMI9INqcK9anMxb6CIrKstQ7vWpVXRF46MIbplNitxARBEDwotmeGwcZo3do3wwqeaKlCJMhfiSezquTohUhpRST/eWJuuuU8InVBP9ik2YSH6ar7hvNG1RVt9VjWWofFjWGksjls3z/q2ZkIgiDMMlrIEWku4hFxUxFRVbWsWRUQ1JohRURuklqyamlFJP95YhSJclMzPp+ChhrvR3hZmNny1nooioLTCqoItWcIgqgEmFm1ub5Ya8a9ULN4KgsmrrvZmqGpGclJZEqP7wb8PgQK4S8J4YpI8Tel1yO8+dHdQiHSVgcABp8IGVYJgpAfFmhWTBFxc2qGqSF+n6KNDc9GL0Q4Ts1Qa0ZeVFXVJKtS+fuaYVWUIlIm4h3wfgPv4ckk4qksfArQ1ZIvRE4rFCI7Do655jQnCIKwy9hUmULExX0zxjCz2dvWGSEB9xx2nwuSIiIfRumrdCFSCDUTpIiwiPe5FJEJjxSR/QV/yOKmWq13ubipFiva6pFTgef3jnhyLoIgCDMk0llNeS4+vuueR2SuiRlATGuGzKoSY6w4i7VmAGPMuxhFRFt6V8SsCniviBhHd42cRu0ZgiAqALbwzu9Tik6qRF3MEdHCzELFjaqAoNbMHMq/m3h/Aslg/9A+BZoXZDYiRqmMaIpIEbMq4H2oGfOHLGutm/FxyhMhCKISMGaIFGuHuKuIlE9VBWhqpuowju6W7NcxRURAqJmqqmWX3gEGRcTj1szy1pmKyMaV+UJkZ/8EhieTrp+LIAjCDKNljKqAu1MzrDUTrS2jiGgPv5SsWhUkyozuMvTFd/wVkWQmp41yFRvfBfR0Va/2zZRqzbQ1hHDswggAYNs+yhMhCEJORsoYVQHD1IwLgWbmFBH+rZmkZlYt/sDtJlSIzIL944SLhJkxwgJkMkbcoHKUGuXSx3fdn05RVdWQqlp/xJ+z4mQglnD1XARBEGbRwsyKZIgAHikiJcLMADGtGd2sWvpe5xZUiMxC27xbRhEJCVREpgpO7tqgH/4SHpWIh2bVwxNJTBVGd5e21B3x5ywuebggfRIEQciGtnm3yMQMoKsTqWxOyHXeiFeKCHlEJIb14Mo5iYUqIswfUiJDBPA20IxFuy9pri36Bm5tCAEAeUQIgpAWZlZtKtGaaagJaKs0RE/OmCpERHhEqBCRF5YNUmp0FxA7NcM27xZbeMdgZlUvPCLGaPditDUUFJFJUkQIgpATTREpUYj4fIpr19m59swAYqdmZBjfLX23q1KsKCJiWjPlM0QAbxWR7uHyhUhrfUERiZMiQhDVSiqTw7Z9I/jz7iGc1NWEC49b6PWRZjBS8Ig01ZW++UfDQUwkMsILkZhHrRnmEQn6vTerUiEyi3KbdxlsakaMWbV8hgige0TiHhQi+1kh0laiECFFhCCqkqlUBr94uRd/2DmIP+8e0pJLw0EfXrvjQgQkGBNlsHj3Uh4RQC8MRE/O6K0ZM4FmHJfeSTS+S4XILJjKES5rVq1iRWQo7xFZ0XakURUAWsmsShBVyRd+9QYe3daj/b6tIYTx6RQS6RwOjExh5YIGD083E23hXZlCJOrS5IypiHeBu2Zqyjx0u4X3pZBkmFJEBFSnjLk27wLeeURUVdUUkWKju4BuVh2fTmtvdIIg5j/b9+ezgz66cSl+df0ZeP7W8/COjnyu0O7BSS+PdgRzBZoBxnRVdxSRYlHzDJqaqTK08d0y/zhCFZFk+c27gL6TIJnJuXqzHzSM7nY1F1dEmmqDYFPHzJlOEMT8JpnJYm/ByH7dOUfhhM5G+HwKjmrPqyC7D8tTiCQz+sK7UmZVQE86FTk1o6qqpmy73ZrRtu9K4BGhQmQWmlm1XGsmwHJEvFFEjKO9bvpE2MRMZ3NdySra51PQUjCsDtEIL0FUBXsG48jmVETDASyMhrWPH1Vox8ikiLCFdz6lfDvEjX0zU6kssoUobbd3zTCzqgxTM96fQDISmiJSzqzK3hTeKCIBv09LXXXTJ7JvDqMqg0Z4CaK62NkfAwAcuzA6Y0cXU0T2SFSIjBjaMr4SoZGAO4UI+95+n1IySRswREbwbM1kqTUjLZWgiAC6YdVNnwgzqi5vLd6WYbDJmREyrBJEVbCrfwIAcOyiyIyPa4XI4ThUVXX9XMXQNu+WMaoCullV5NSM0ahaaskqYGjNiDCrUsS7fJgb3xWoiJiYmgEMMe8uKiL758gQYVBrhiCqi52FQuSYhTMLkWWt9fD7FEwmM+iXZP/UaLywZ6ZMhgigezZiAh/2zGSIAGIDzUgRkRAz47thzazqTY4I4M3iOxbvvrzE6C6DRngJorrQFJFZhUhNwIdlBQV1z2Dc9XMVY3SOzbsMN6Zm2IPnXAq40O27ZFaVDzOKiN6a8U4R8WKEl428tUfCZT9P94iQIkIQ852xqZSmdrBxXSO6YXXC1XOVYq6Fdwx9akbcNVa3ApS/3ocMIZq8Wlxp8ojIi5nxXaaIiBidNe0R8aA1EzMRvAMYF9+RIkIQ8x3Wlulsri06girbCO/IHAvvGG4oItpuszmKAfZgrKpAOsunEEnR1Iy8JE0sAmJtG69yRABDa8YlRSSdzWGqUCRFy8y7A9SaIYhqolRbhrFKshFeNr7bUl/+OhZ1YSAgYVYRMdyPeLVnyKwqMbpHpFxrpuAREaCITJlURNw2qxoLnrkVEVaIUGuGIOY7pYyqDE0RkcQjwqb55lJE9Ij3tLCJn6RpRcRYiPC575BZVWKsKCJJAYpInJmXTCoibnlEWFumrsY/5/IqbQMvtWYIYt7DMkSOWRgt+uerCoXI0GQS41PumetLwcyq5VJVAX1qJqfqLXPemFVEFEXRCgYehUgupyJTCFIjs6qEmDEPCVVEClMzdXN6RPI/JG4pIrFptg+hvJwJ6IrIVCqrmW8Jgph/5HIq3iooIqtLKCINoQAWNeYN7rsPe29YNZsjEg76ECgEnonyiWgKvAlVQs8ScV4UMX8IQIqIlJgxDzFFJJtTkcnynetmbxCzgWZueUTYD2K0du6FzQ2hgPbmJlWEIOYvh8amEU9lUeP3lU1c1tsz3vtEzOaIKIqiT85Mi7nOMnWjnBWAwTNLhAoRyTGjiBjfNDxVkWmD/FcrWaCZPjEztyKiKAra6ildlSDmO8wfsqq9AcEyLVtZDKupTE67Zs41vguIn5xh6oaZyRWei++ME581c7Ta3UDoCTZv3oxTTjkFkUgE7e3tuPTSS7Fr1y6RL+kYMx4R4z8cz8kZ5g+p8fvmrFK1HBHXCpG5V1UbaSHDKkHMe3b2sR0zxdsyDFkUkbFCW8anmGszi943Y2Y4ghHi6E3UJ2Z8ZaPl3UJoIfLMM8/guuuuw9atW/HUU08hk8ngggsuQDwuh3u6GJqLucwbw+fjaxxiaGFmcxhVAWNrxh3zF9u3YEYRAXTD6hC1Zghi3rJzoPzoLkOWLBFjhki5hXcMbd+MKEVEa82YUUQ4tmYkSlUFAHOPtzZ58sknZ/z+wQcfRHt7O7Zv346zzjpL5EvbRmvNmBinSmVyfBWRpLnRXcD9QDNNETHhEQEMI7xUiBDEvGXXHKO7DFaIHBydRiKdNaUAiMCsP4TBFBFR6aoJrTVjxiPC7+FXplRVwGWPyPj4OACgpaXFzZc1TS6nmk6b0xbfcdw3EzcZ7w7oPyCum1VNKiJtWroqtWYIYj6SSGfRPZRXt48tMbrLaK2vQVNdEKoK7D3snSJuds8MI2LIEhEBG981p4jw2zeTlChDBHCxEFFVFTfffDPOOOMMHH/88UU/J5lMIhaLzfjlJkYn8Vxz3Vq6KsclRNro7hwL7wBdEYmnssjmxK/XZq5x860ZMqsSxHxm9+AksjkVjbVBdERDZT9XURR954yH7Rl2PZprdJehKSLCpmYKiogpjwi/h99UtSoi119/PXbs2IFHHnmk5Ods3rwZjY2N2q+uri63jgdgpvHUbPa/CLNqvQlFpMFgGo27kNURszC+C+iO9KEKLUSyORVv9MaEJSoSRKVjjHY3Y3iUwbA6piki5h6ooi4pIl5NzcgwMQO4VIjccMMN+OUvf4k//OEP6OzsLPl5mzZtwvj4uParp6fHjeNpsH9gv0+ZMz00HBRhVjUXZgbkCyH2JnKjPTNhYXwXqPzWzM0/ehl/9Y0/4eYfveKK4kQQlcYuk0ZVBitE9nhYiIwwj4hFRUTY1IyJ4QgGz9YM84iUG7l2E6FmVVVVccMNN+DnP/85nn76aaxYsaLs54dCIYRC5SU+kZg1qgJAmDmYuZpVzcW7MxrCAYzEU64YVvVk1flvVt26dxi/eLkXAPDzlw5BAXDXZSfBb8JlTxDVwpt95aPdZ7NKAkXEbLw7Q/jUjOYRcTnQzERMhZsIPcV1112Hhx9+GD/84Q8RiUTQ39+P/v5+TE9Pi3xZ21iqTrUNvN4oIoAhS8QFRURvzZj0iDBFJJ6sqPZGNqfiC796AwCwblkz/D4FP3vpEP7lJ6SMEIQRsxMzDOYR6R6Kc02ktsJ4IYag0eR1jLWiRSsiplozWo4Ix9ZMNRQi999/P8bHx3H22Wdj0aJF2q/HHntM5MvaxpYiwtGsasUjArg7wjthMdCMmVXTWdW10DUe/OiFHrzRF0M0HMADV63DvX/7znwx8uIh3PKTHVSMEASA0XgKgxP5tqvZQmRJUy1qg36ksjn0jHrzMMra2A0mr2Oip2asKSL8WjOymVWFt2YqiaSN6pSrImJhagZwb9+MqqqWx3fDQT/qa/yIp7IYnkyZ/jovGZ9O4+7f5pN/bzr/HWhtCOGvTlgEVQX+8dGX8NMXD6Im4MPmD53g8UkJwlveLrRXOptrtQeiufD5FKxcUI/Xe2PYPTiJFWV204hiUmt/my1E3JmacTvQLJmRyyMixykkQY93n7s6lUER0ffNiE1XjaeyYEKA2dYMYGjPVIhh9d7fvY3heAqrFtTjqtOWaR9/74mLcM/la6EowCPPH0DPyJSHpyQI7+kdyysaXc11lr7O650z7Bprtnhyb2rGG7NqVU3NVAp67r8ZRYSN73qviIj2iLB496BfsWRuYobVSoh533N4Et99dh8A4LOXrDniSeGSExfjtJWtAIBfv9rn9vEIQioOFQqRRU1hS1/X2VwLABiIJbifyQxsIMBsIcIUkXgqK8TXYu2eQx6RqsCKIsJuyF7liADueURihraMlQVJbN9MJSy++9Kv30Qmp+LcY9tx9jHtRT/nvScuAgD8egcVIkR10zeeL0SWNNVa+jrdxO7Nw8mk5UJEV4B5X2cz2RwyBak5bOqew39qhgoRCdFT7kyYVYP83hQMy1MzLnlEmOISMWnwYrQVFJERyRWRXf0T+P3OQQR8Cm577+qSn3fRcQvhU4BXD41j/7C8ixsJQjS9Y3lFY1GjxUJES1x2/+Ekk81pCrZZj0hNwKepFbyVZ+O9w8w9h6tZtZrGdysNO/06vkvvrOWIRNxSRKatje4ytCwRydNVt+8fBQBsWNmi9bCL0doQwumr2gBQe4aobphHZLHF1oyX+UJsqShg/hoL6KoI7ywRYyFiThHhmKwqWaCZHKeQBBZOVjGKCMsRcbE1Y4WWQmtmSHKz6o6DYwCAEzub5vxcas8QhLEQsaaIsNUPXjycTBZa33mvm5VCRMzkDHuIrfH74DMRlihk1wwVIvKRtCBXaUvvOCoiUylrikhDoTCQvTUje7rqKwfzW6FP6myc83MvPG4h/D4Fr/fGsG+I2jNE9RFPZhArXBMWNVpTRNjqh9F4CjmXM3msGlUZoiZn2L3DbHtERGuGPCISwgoRK3G7PKdmmHRYb1ERca01Y1ERqQSz6nQqi7cKOzPMKCIt9TU4fRVNzxDVCzOqRsIB07unGM2FaPVMThUWm14KqxkiDFH7ZrQHXxP3G0DQ0jsqROTDSoWqL73jo4hkcyqm06w1Y9Ij4rJZ1ezmXQbrB49I7BF5o28c2ZyKtoaQ6ae7S6g9Q1QxhwpGVasTM0D+xseuW263Z7RUVZuKCO/CycroLkBTM1WDpUAzjv06AFoRApiv2N0e37X69GMsRGSNRn+lR2/LmB1NvmDNQgR8Ct7oi2HvYe8WeBGEF/SxDBGLbRmGvpnb3UIkblMREbVvJmFhpQhgyBHhGfFOHhH5sBTxzqZmOCkiU4UfEp9i/o3JvCTiWzPW9swwmAybU4GxKTlVEStGVUZzfQ3edVR+euYJas8QVYZdoyqjxaMRXvutGTEekaSFJauAoTXD4eE3LdmuGTlOIQkJCwuIeCsi8ZTuDzH7ZF5b8JJMczTMFsOuIhL0+9BUl/8aWUd4dxSMqid2zW1UNcKmZ/6L2jNEldE7nm/N2C1EWJaI24nLTBGJWC1EQqKmZszfbwBBrRlSROTDytQMb0WE/ZDUWZhvry28gVOZnNDWR0zziFhfXKdfdOQzrI5Pp7G3MPlykgVFBAAuXLMQQb+Cnf0Tnu3NIAgvsJshwvDKO6Y97Fm4xgL6dW+C804vKwq88fN4tGaS5BGRF1s5IpwUkamUtYkZYKapVaQqMqFNzVhf1tzqUT/YDK8dyqshnc21mlxslsa6IM4otGd+Q+0ZooroG7eXqspg03RuFyLM4yHN1IxVRSSoT8043WxPZlWJ0cZ3TZlVOSsiKeuKiLGSnk6JK0RiWo6IdUWkTeLJmVcK/hCragjj/DUdAIBn9wxzOhFByI2qqroiYrMQafFIJbWbI6Ilq05znprJ2JuaUVUgnXVYiFCyqrwkLCgieo4IL7OqtVRVAFAURWvPiC1EWMS7dUVES1KUsDWzozAxc6KJILNibFiRzxN58cAotzFugpCZkXgKyUwOigJ0NIZsfQ/PWjN2p2YEKSJ6XIQ1syrgvD1DZlWJsbR9l6NMBljfvMtg7RlRrZlEOqvJePY8IoWYdwkVETYxc1JXk62vX7WgHm0NISQzOc30ShDzGbbsrq0hZCkm3YgWdOhyu9bq5l2GvmtGVGvGmkcEcG5Y1ZbekSIiH9Yi3nWZjMlcTpjSzKrWfkjYOVg8PG/YU4CiAA0W1BqGHvMulyJyeCKJ3vEEFAU4fok9RURRFGxY0QIA2ErtGaIK6B13NroLeLdvhj3sWS9EClMzvAPNMtYUEUVRNAWDVyFCioiEJNPm57p5VqeAcXxXLkWE/fA1hAKmFjPNRlazKlNDjlrQYPnCZGTDynwh8pfuER7HIgip0f0h9iZmAP3hZHTK3X0zkzbNqkwJTmVyXFuwWqCZSUUEMGaJODsHFSISY0URqfH7wOI+ePhEmKJhxSMCALU1Yj0iWry7DaMqoI/vymZWZYvurASZFYP5RLbvH9X6rgQxX+lzmCEC5AMBgfxai3HOBtBy6IFm1h72jA8qPNdpaIFmFlpcvLJEyKwqMdpct4kKVVEUrkl32sI7iz8kmllVlCIyzcLM7KkGXq79LofuD7HXlmEc3d6A5rogptNZ8okQ855DDuPdgfzNr7HW/aBDdo21qoD6fXpLhOd11mqgGcBv8R0pIhKjvTFMVqhalggHuc6pIjIlSBHRJ2bsKSKNdfrCKLfXfpdCVVU9UdWhIuLzKTh1BWvPkE+EmN+wPTN2Ft4ZafVgms7u+C6gP/DxmpI0fi+zgWaAYUjCaWuGds3IixVFBNALlgQPRcSmR0TED4gRvTVjTxFhTz6qyn/8zS4HR6cxEk8h6FewelHE8fdj7Zm/7CWfCDG/YVMzixwWIi0ut2xVVcWkTbMqYPDipfi1X7XcKkuKCKfWjAUbghvIcQoJyOZULSTG9Fw3x22IdqdmRHtEYlqqqj1FJBTwaz/EY9NytGdYkNmxC6O2RxCNMMPqC/tGkCGfCDFPSWdzGJxgHhH7rRlAzxJxa6x/KpUFS1mwalYF9Ac+ntOJCW04woZZ1UEhksnmwMRp8ohIRsrwD2v2jSFGEbFYiATlbs0AQFPha8em3DOmlUNvyzjzhzCOXRhFNBxAPJXF670xLt+TIGRjIJZATgWCfgVt9fbCzBgtLObdpWk61pZRlJmrMcxSK2A6kXkLrTwM8dg3Y0xlJY+IZBhbG2b7ZkwR4To1Y9Gsyn6oRLdm7JpVAb2IcdMhX47Xe/kWIn7yiRBVgHHHjJ1RfiNavlDcHY+INjFjYbu5EREJ1kmLEe8AEOKw48z40E2FiGQwqSvgUxAwWYiEOa5l1iLeLfQLARcUEYetGQBoKhhWxyQpRPYNTQEAjmpv4PY9ySdCzHd6OUzMMNyeprM7McMQoYh4NTWTzOb/HxQlf7+TASpEClhdyQzwVUQSFsLUjNQWWjmixnd5KCJNtfmLzviU9x6RdDaHvkI6ZFdzHbfvy3wiz+8bQVaS6SCC4AkzqjqdmAH0oEO3WjN2M0QYIh74EnbuORxaM9rort9nSx0SARUiBexVp/w8InYc1ABQWyiGhJlVeXhE6uTxiPSOTSOn5n+gF0Sc9bmNrFkURUMogIlEBm/2kU+EmH9oiohDoypgGN91uTXjVBHh2QJPOrjnOFFEjIWILMhzEo+xo4iEOU7N2JkpB8RIhkZi086SVQE9S0SG1syBkXxbZmlLHdengYDfh/XLmwFQ3DsxP+njsGeG4fb4rt3Nu4w6AdOJthQRLUfEfiHCzKqy+EMAKkQ0tHh3C9VpOMhPEUnYVURYa0ZYxLuzZFVAzxKRwaxqLER4o/tEyLBKzD8OFVozixt5tGb0QsSNoMNJh4WItlxUQKCZPY8Ih9YMFSLyoY9SuduvA/Jz3cxXYMVBDRh6l8KW3hUUEUfju/mLjgytmZ6Rgj9ERCFi8InIkiJLELzgqYg01+WvCTnVHaWUKSIRSRQRVVUND79W7jkcWjMFsyoVIhKitUY8UEQShjeV1YAtbXxXgCKSzanak4TdZFVA94iMSxBo1lNQREQUIicsaURt0I+xqTTeHpzk/v0JD9m2DVi0CFiwQP+1aFH+41XAVCqjPUjw8IgE/T7tuuBGzLvT1gzv8d1UNqcFrLmdI5Ikj4i8WNm8y2Cf69TAZPx6qx4RXTLkH59u3DQZcTK+K1GgmcjWTNDvw9quJgDAC/vJJ1LxbNsGtLbm5xxPPRVT/f0IDQ3hgqEhhIaGMNXfD3zwg16f0hXYxEwkFHDkFzPi5gjvhNNChPN0ovHh1VqOiHOPCGvNyJKqClAhomHPrMonRyRp6NlZDQoSsQOBwSZmwkGfIxlPRrNqV4tzebkYzLC6fd+okO9PuEB3N3DqqcCGDZgaGUEIgOLzo2XJaviWnYQ/LTsJvuVr0br8nehJeH1Yd+A5McNg6azDLozw6gvv5BjfZfcbRbGmTPBozchoVrWvt88z7IzvhjltQrQ7MQMYJUP+ikhMM6o6ewIymlVVVfVsdn18Oq0ZZnlmiBhZtyxfiLywnwqRiiMeB771LeDzn0dqYgKrAByqqUXDiRdgwSkfQCDafsSXnK0A1zzxJm46/2jLm7MrCZ7+EIY+OeNGa8ZpoBm/zChgpifRyvVwvppV5+9PjkVsBZpxSla1M0/OqDOM7/K+yeuju87eJk0FY1oqk0MindNGjt2G+UPaGmpsS7RzcfKyZihKXnkZnEigPcLvCZIQSHc38P73I/faa/gsgC/XNSGy7hIsOfkS+MP5BN7s1Diy8VFozX2fHzVtS/HAH/fiiVf78KUPnoB3v2OBd/8PAmETM4s4TMwwWhrca804nZqpDfJuzdgLsOSRrMrMqrJs3gWoENHQPSLWFRHHHhEbOwe0MxRu6jk1b4DisU2WwSPMDADqa/wI+BRkcirGplOorRHTFpkLVoh0ClJDgHzeyjEdEezsn8D2faO4+IRFwl6L4EA8Djz4IHDnnUgNDGABgMwx78Lii2+EL5R/n6SHDyL2/M8w+fofgOzM9uLv3uzHZx9/HQdHp/H333keH1i7GHe87zg0F5725wt9hdbMEq6tmUIh4mprxlmgGb/WTOHh0+L1mueuGTKrSoiuSlhXRBIOx3f11oyNrZCGipp3loge7+6sEFEURYp01Z5RcUZVI8wnQu2ZCuDXv0buhhvwmYEB1PsD8J9/LRZcugm+UB2S/bsx+LMvoff/fhKTO7YcUYQMDAzg3GM7sOVTZ+GaM1bApwC/eLkX//v72+ddzH/vONszI6I1I78iwnu5qD6lae0WzLM1Q2ZVCdFT7ux4RPiYVe0oIkG/D0F/vh3DO11VX3jnXDhrlGByRuTEjJH1y/J5IlSISEw8Dvz7vwO33ILPAfhy00K0ffRuRNddAgAYf+5H6P/ezZh++zlAzSEIYHrVKqgNDVAVBerAANrb856R+lAAn71kDX72yXehvsaP5/eN4D/+tNe7/zcB6Jt3+SkibN/MkAvju85bM0wR4ePF0zyJVhURLq0Z+cyq8pzEYzTzkI2VzE4VEWZ2tfqmZIQFbeDl1ZoB5EhXPVAIMxNdiDDD6uuHxoUl3hIOiMeB227LKyH79+Peleux6Op7EFp4FLJT4xj48e0Y++P3ADWHOgC3LliARDyO8O7dwMQEkMsB7UcaV9d2NeH29x0HAPjKll14o3d+7BxSVRX9rBDhaFZtdVER0QLNbD5U1XIONEvabMdz3TVDhYh8aG8MG+EyjgPNbBRBRkTsQQD4bN5lMMOql6FmmkdE0Oguo7O5Fh3REDI5Fa8cHBP6WoRFWBFyzz14N4Cvn3A+mj78WfhC9UgcfAN9370Rib3bEQRw69KlmNiyBV/q74evzlzxetn6TrxnTQfSWRU3PfYS1yVpXhFLZLSHnIVR/oqIG2ZVNjXjONCMc46I1XZ8iMN+MypEJMZOMaDniDhURGwUQUZ4/5Aw9NaMc0XE61CzbE7FoVF3FBFFUbT2zHZqz8jD4CDwoQ8hd889OAvAjo2Xoe2vboLi82Py1acw8MgmZCeGEFUUTD35JL7U3Q3fe94D+MxfExRFweYPnYC2hhq8NTCJu3+7S9z/j0swNaSpLsh14o15REanUkI9NclMFqls/vreYHPEmj3spbMq0lkOu8WcekScmFVZxDt5ROTDVqAZM6tyUkTsjO8C4hbfaa0ZHh4Rj0PNBmIJpLI5BHwKV8NdKbQ8kX2UsCoFg4PAJZcgs2ULlkPBG+d9As3v/nsAwPjWH2P4iXuAXBZRnw/DU1MIXHihpQLESFtDCF/+8IkAgP/8n248u3uI2/+GF7AMEZ5qCAA0F64JqgqMTYlTRZgaAgD1NgPNjNdmHipX0uaSU2rNzHPsjO+GJAg0A4Dawjl4KyITHBbeMbxefMeMqkuaa+G3mF5rBy1hdf8oLcDzGlaEbNuGVl8A0+/7J0TXfwAAMPK7BzD2zEMAgK5gEMPxOAJh5zfc81Z34G9PXQoA+Kcfv6J5FCqRfgFGVQAI+H1aMSKyPcP+7kMBHwI2VYB88Fj+v3k88Nm95vOYmtGSVUkRkQ87u2a0pXecIt6tLNwzwttIxdAVER5m1byqEvNIEelxaWKGsXpRFLVBP2KJDHYfpgV4nmEsQvxBhD54K+rXnA01m8HhX96FiRd+CQA4o60N+yYnuRQhjM+8dzU6m2vRN57AT188yO37ug2bmFkoQElscSFLZNKhURXIt9zqOLbAbSsimkckB1W194CTJEVEXpiqYeUfhxUtqUzO0VOvnrJnVxHhm/rHEGFWHfPIrCpy624xZizAo70z3mAsQoIhhP/6c6g76lTk0kkM/uyLmHrzGQDAGUuW4Jm+Pvhq+IaQ1YcC+MSZKwEA3312X8UqY6IUEcBoWBU3wut08y6DZ6iZ3Ws+U+xVVVc2rEKtGYlhBiQr/zjGatZJz86ug5rBO/WPoZlVeYzvehxopi27E5iqOhs92Ix8Iq5jLEJqalF72edRu/ydyKWmMfjjO5DYux31AG7buBHP7NsHX0BMyPSH13WiIRTA3sNx/E+FekX6Y0wREVCIuDDCq2WIONwFVFvDTxGxk1uV/3z9/mS3PcOMuxRoJiEpG4WI8U2RclCI2J0pZzDJkOeooKqqiDGPyDyYmnErzMwIM6zS5IzLGIqQtnAD6j5yJ8JdxyOXjGPgsc8i2fMqoj4fxrZswZ1//rOwIgTIR4r/9bpOAMBDz+4T9joiEauI5AuRIRdaM3bj3Rn6glEOrRkbSd7A7ELE3j0nTYqIvLBCImShSgz4FDDfoxPzkPOpGb6pf0C+6mcjdXxzRDzyiLg0umuELcDbPzyFwxPi0yMJzCxC6hpRd/n/QWjxMchOxzDwyK1I9e7MT8bE4whYHM21y9+dtgwA8PtdgzgwPCX89XjTp8W78y9EWurzrRmRG3j11oyz0WOe04l2IxsURdEKCLuFCHvotnKvE408J/EYO30zRVG4jFMlbIwOG9HNqs7n2xls867fp2gz9E5gyaqTyQyXOXwrTKeyWiHgZiHCFuABwHZqz4jHWIRE2lB/xZdR07ES2clRDPxwE1IDe/QihKMpdS5WLmjAWe9YAFUFvvfcPtdelwfxZEZTRkWYVd1pzeSvrw0OlV02nTjFozXjIMRSzxKx2ZohRURe7C4C4pF0l3SqiGhubn6KCJuYiYQDUBTn467GLBK3J2fYsrtIOKB5VdxCzxOh9oxQDEXIgqZFaLjy3xBs7URmfBD9P/w00kP7PSlCGB87fTkA4LEXeipqlJf5QyKhgOPWRjHcaM3om3cdKiKsBc5TEbFxzXf68EuFiMTYXQTEI+bdqUeEZ++SMcFxdBfIZwawFo/boWZuj+4aOXVFPmH1eQo2EwNbXrdhQ74IaVuG+iu/jEBjO9LDB9H/g1uQGe31tAgBgHe/YwGWt9ZhIpHBz1865MkZ7NA/Ls6oCrizgTfOyaxaV8NvOtHu0jvA+eK7JJlV5SWVsT6+C/BJutPDbRzmiHA0q7LWTLSW31NQk0eTM14YVRkbVrQCAF47NK6pTAQnjMvr9u1D6+Jj0HDFZgQaWpAa2Iv+H/5rPrLd4yIEAHw+BVedthxAvj1jNwPCbfoEFyJtbHxX4AbeCU7juzyXi9qNeDd+jd3WDJlVJUabmrHamuGQdKeH2zhTRHiO72qtmRC/VgZLV3V78d0BlzNEjCxsDGNFWz1yKsW9c2XW8rqvrjoVjZd/Cf7aKJKHdmLgkU3ITY1JUYQwLlvfiboaP94amMRze4a9Po4p+gUaVQFdERmbTiMjyDsW5zQ1U8fxgc9OkjfDcWvG5r1OJPKcxGNSNpJVgZlJd3bRwm1sKiLsB4Tn+G6MY5gZgxlW3Z6ccTvMbDYbV+bbM5Vy85GeeBy45RatCHn5pAux4EO3wRcMY2rPNgw8dhtyybhURQiQb3N+6OQlAIDvb93v8WnMITJVFQCaC9N0qiquZasVIg6vZfpQgHOPj5MQS6etGfKISEommwMLPbRsVmXVqQOPiBMHNQCEBQSa8frhNeJVqFnPiPuju0Y2rsy3Z7buJUXEMWyD7n334SwAr77rCrRedEN+g+6OLTj80y9CTSelK0IYl5+S3z/zh12D3FcyiEBkhgiQn8pjRnZRDyiTvJJVOUa8O2nHO1Xh7T50i0Sek3iIMSrXrlnVWWvGmUeE5w4EBi8504gXoWaqqhpSVcVv3S0GK0Re7x33LEdlXmDcoOsL4M2LbkDTGVcAAMaefRTDv/kGoOa4Lq/jzXGLo1jSVItEOoc/V0DSqmiPCGBY/yDousC27zqemuH4wOekHc92ktl9+KVkVUkxpqLaL0ScKyJOA834Ts3weYowwsyqbt6MhyZTmE5noSj5zbte0BENYyX5RJzR3Q1s3JjPCKmNIvWRLyJy0oVQc1kM//abGP/TwwDELK/jiaIoOH91OwDgqTcGPD7N3Gjx7lGRhQi7LojxjnGLeOeYYO3kmu/0nkNmVUlJZvNvLEXJp6Vage/UjM2IdwFTM2IUEfbk455Zlakhi6Jh24oTDzYUVBHyiViEjeeedhpS3d1obFuG+r/7KsJLT0AuOYXBn34Rky//BgDQFYkIWV7Hm/esWQgA+N3OAS29WEYS6aw2ViuqNQPo3jFRigi31gzHB76kg2u+UxU+aWOdiWjkOYmHGMPMrIZ3OR2lUlXV9kpoRlhAjkg8xb8Q8cKsemgs7w/pdHHZXTGYYXVrNxUipmF+kBtuwGcGBtC66hS0fvQuBJsWIj3ah/7v/xMSe18AAHSFQtg7NCR0bwwvTl3RgkgogKHJFF7uGfP6OCUZKKgh4aBP+9kVgejrAnuocmq85zmd6OSa7+ThV1VV3axKrRm5YB4RO9n7jsNlDF/ndHw3mclxe8Jiscg8WzOaWdXFQkQbP2zyVqrXfSIx8omYYXAQuPjivB8ECu497W/Q8uHPwheqQ2L/DvR/72akh3vyG3TXrMG+WAwByZUQRk3Ah7OPlb8906cZVWu5pCuXQmS+UC6naoWD02sZr+nEbE7VfBq2ChHt4df6PceJH1Ik8pzEQ5yMMzltzRjfTLbNqobeJ68RXl6xyEaYWXXcRbOqZrYT2OM2A/OJqCqwrZt8IiWJx4GvfAU47jhkXnwRbaF6JD78WTSf9XdQFB8mXv4NBn70OeQSE6gB8ht0X31V+nbMbN6zpgMA8NQb/R6fpDT9Lv3s6PlC/K8LccOoLa/tu04VEWNLxe3WTMqQ1UKKiGQ4K0Qc9usKX+dTgKDf3lOH8c3Ma4R3UohZVQ8vcgsmL4t0/Ztl4yo2xkvtmaKwVsw//zNuGxpCtGMVGq6+B3VHnYpcOomhJ+7ByG+/CeQyqAEw4eIGXd6cfcwCBHwK9hyOY+/hSa+PU5Q+waO7DF0R4e8dY/4Qv09xPK7KK8Ha+PDpdmsm7WAwQyTynMRDUll78e6AM5kMMGSIBPy25U+fT+Hq6Ab4GbyMGKdm3Iq4dutiagbWnnmOCpGZMEPqKacUWjHAN066EAs+ehcCzA/y8L8g/upTAIAuvx/xeBw1dd76fpwQDQe198N/vylne8atIl4zq4pQRLSJGfvXVwYvsyrbth70K/BbHI4A+Cgifp+91xYFFSLQK0s7c9VOWzMJhwvvGDxn3AFd0owIMKtmc6pW6IimX3AypBU2FhbgvdEXc7U9JTUsJfWGG/CZAwfQVFOH6fffkg8pC9Rg6u2/oO+hm5Ae3AugMJ47NYVABRchDL09I2ch0ic43p0h0qzKvG4RDss764J8lt4ZHz7t4OThV0ajKuBSIXLfffdhxYoVCIfDWLduHf70pz+58bKmYQYeO/84jlszDjNEGDxT/wDDkwTHQiQc9Gt/X26EmmWyOQxO5JdpyaCItEfDWLkg7xOhbbzQskFy992HdwO4a9E70Pixb6B+9VlQsxmM/uFBHP7ZnVCTcQQB3LZxY0WM55rl/EIhsn3/qNClb3Zxq4hnLVsRxbl+HXPudQvX5K9d0+msI0XX6bZ1Jw+/SQkzRAAXCpHHHnsMN910E2677Ta89NJLOPPMM3HxxRfjwIEDol/aNHw8Is4UEW79Sy6jZVmtOONZiADuhpoNTaaQzanw+xRty6fX6HHvVdyeYYbUtWuReu01NEPBjlM/hIVX/huCTQuRGetH/w8/jdjzPwWgIgJg8sknceef/1wR47lmWdJUi+MWR5FTgd/vHPT6OEfgukdEiCLC74GKPeypKp8AS9uKiJPWjAP1XyTCT/PVr34V11xzDT7+8Y9j9erV+PrXv46uri7cf//9ol/aNI4KEYdxu/ryI16KiPOWBzOqAvneKk/0UDPxhQiTljsiIWn6oadVe7CZ0ZAai6Guvhnhv/k8ms/5Byj+AOI7/4Te796IVO8uAMAZ0SjGpqdRc+GFFWlKnYvzV8vZnklnczhcUGlEe0T01Q8p5DgHvLFrGY88pFrDNdrJA58WYGlbEbH/8JvOyrdnBhBciKRSKWzfvh0XXHDBjI9fcMEFePbZZ4/4/GQyiVgsNuOXGzCzqicpd9rCOz57EKZTzldps90MtUE/ApwrZz1LRHy6ar8LezKssqEQbPZmf8zVhFkp6O4GTj1VM6R+beV6LPqHf0ftipORSycw/OS/Y+gXX4aajKMOhVbM8DB8ksa184D5RP709hDX7dlOGZxIQlXz7eqWOrGtsGihEMmpwCSHzbZGmNfNabw7AAT8Pq19P+Xg30oLM7PtEbH/8JuSMFUVEFyIDA0NIZvNoqOjY8bHOzo60N9/5Pz85s2b0djYqP3q6uoSeTwNJ3IVL7Oq49aMNuPOQRER4A9huJmuagxkkoX2SBhHtTdAVfM3n6rAGNO+fz/q/UFMnve/0H7ZHfDXNSI12I3+hz6FyVeeBAB0BYMY37Jl3rViisGW4E2ns1KpZCwIsKMxBJ9gNTEc9Gt+Cd4+kUnOW8R5tMB1FdypImK/NVOVZtXZY1OqqhYdpdq0aRPGx8e1Xz09PW4cDykuZlVn47tOWzO8Uv8AY7w7/90sbm7g7ZcoQ8TIeYWlZ7KObXJlcBB4//u1mPbG1k60/t1XEV3/fgBAbNvj6CukpPoA3Lp6dX5pXYXmg1hFURSceXQbALnGurUiPupOES8q1Iz3zqxaDus09N1iTj0i9qdmggE5WtUMoY8bbW1t8Pv9R6gfg4ODR6gkABAKhRAKuW8qdOYRcbZrRnNQc1NEnBciIhURN82qMmWIGHnP6g58+5m9+MPOQaSzOemMY9zo7gbOOw+Z7m6sBDB64nuw4Pz/DV8wjGx8FENPfB2JvdsBABEAQ088MW+9IOXYuLIVj27rkcrA7HZbs6kuiP5YgvsDSlxbVcHnoYrHglF9z4z7UzNaa0aya47Q09TU1GDdunV46qmnZnz8qaeewumnny7ypS3BI+I95VAR4eYR4aCIiEhVZWjpqi74I5i8LJsi8s6lzWitr0EskcHz8zHufXAQuOoq4KSTkOruRgPLBrn4RviCYUx3v4jeB2/QihDNkHrxxVVXhAD6JNVrh8YRS8iRL+N2Ea+HmvG9LkxwvpaFOcQkJJ0qIkEOrZlq8ogAwM0334z//M//xHe+8x28+eab+NSnPoUDBw7g2muvFf3SpvF0fDfNVxHhMb6rbasU6BFxZ2pGTkXE71O09oxs0xKOYcvqHn4YZ05MoHHhUWidlQ0y+KPbkYuPIQjg1g0b5r0hdS4WNoaxoq0eOYn2EHmhiAD8rwvcWzOaR8S+F09vxzv0iDgJNLNZBIlCuBPsIx/5CIaHh/GFL3wBfX19OP744/HEE09g2bJlol/aNFrEux2PiIPqNP91nALNeCoi88CsmsuphohqecyqjPNXd+BHLxzEU28M4Pb3rRG63dQ1DK2YVgC5tRej/bz/BSUQRGasH4d/dZc2lhtVFAz/5jdV4wWZi40rW9A9FMfWvcM4b/WRbWu3cStVlSHquqD73fhcy/i0ZpxFNszH1owrlvRPfvKT+OQnP+nGS9lCS1Z1sn3XZo6ILtPJE2im91Ur1yMyHE8hnVWhKEB7RI4wMyNnHr0A4aAPh8am8WbfBNYsjnp9JPvE48CDDwJ33onUwAAigRAiF16HhuPPBQBMvfUchp74OtRkHH4AG+vr8fTQEAJVrILMZuPKVjzyfA+27pVNEXHJrFonxqzK+6EqzMGLpweaeTg1I5lZVa6yyCOcjDQ5bc1wU0TYD4j0UzPuBJqxC+mChpCUZtDaGj/OOGoBgApvzxh3xQwMINq8GK1X3Y2G48+Fmsti9A/fweGffwlqMo4aAFNf/jL+Z2yMipBZbFiR94m83jvuipG7HNmcigGXVyM01orZwMu7NVPHdXzX3vWVfV0yk7McNZ+WVBGR6zQe4SR/nxUiqWzOViqg05lyhja+y0ER4W3wMtLkUqAZG92VzR9i5AK29OzNIzN1KgKWklrYFfOVlevQ/vdfQ037CmQnRzHw6G2IPf8zAEAn25h7yy3APM8GsYPRJ/KCx3uIhiaTrq9GEOUR4W285+HFY/cbuwMKzA6gqrqab/W1q86sWgnwiHgH9P6bFZzOlDN4SIYM3k8RRliyaiKdE5okKevEjJFzjm2HogCvHYppPfmKYXAQuOQSLSX1lVMuRfuHPwdfqB6JntfR991/RLLntfyyutWrsX+ebMwVycZC6q7XwWbM5O3magRNKRXUmuGl7vLw4iUctuONX2e1PVO1UzOVACsg7CWrGt4UNnwiTmfKGXU1fFZUA2ILkYaaANi1LSZQgpYxVXU2CyIhnLy0GQDw35XUnilszc1s24ZWfwBTF9+IlnM/DsXnx8Qrv8XAo7chGx9FDQrL6l57bd5szBWJthCx29tCRE9Vda+IZ60ZntcEVVURLzyYNYSCXL4nj7ymhMN2vLGtYtUSoJtV5ZqaoUIEQNpBlRjwKdqN1Y55SF+AxGnpneSBZj6fYsgMEFeIMI9IR1ReRQTQd41sqZRCZHAQuOgipLq7EamNovYjX0LDie+Bmsti5L8fwMiT9wK5DKKKkm/FVGFAmV1YIfJ6b8xTn0jPSL4Q6Wx2T8ES0ZpJZnLI5tgWcU6KSNB5gnXSYTteURTb3kRZk1XpCgG9SgzZUETybwr741ROHdSM2pr813NRRDiPvM1GDzVzQxGpjEJk695hacKsisL2xWzYgMxbb6GpaRFarvoKwl3HIZeYxOBPPo+J7b8EkN8VM0ytGMt0RMNY2VYP1eM8kZ7RKQDA0hb31EQRgWbsgQrgs/QO4NSaYR4RB+14PUvE2jnSDu51IpHrNB7htG/mJEvE6Uw5ozbIrzWjrc7mtChqNqIc8kZk3TMzm1ULGrCyrR7prIo/vnXY6+MUJx4HbrstPxmzbx9aF70DrR+9C8HmRUiP9aPv+/+MRPeLqAdw25o1+V0xNBVjiw2sPeNh3PuBkXwh0uWBIsLTO8auY3U1fm6L+1gh4qQ141QRAQwbeG0qIuQRkRAn23cBvTpN2PCI8Fp6xzNHZJLliHB6ipiNPjkjRgFQVdX1QCYnaO2Z1yVsz7Ai5J57CpMx69F4+f+Bv74Jyf7d6P/+PyMzchA1AMa2bMGdr75KfhAHaIZVCQqRpS3uFSINoYBmjOXVlhLRYuaxXFTziPBQRKgQmT9oBh7bLmYHrZkMn0AzY+Kf1dny2Yg0qwKGFEVBrZnx6bRW4MnuEQGAi09YBAB48vV+jMTF7+Axzawi5KUT3oP2D38Wvpowpvdux8Ajm5CbGkMNgIl4nFJSOXBaQRF5oy8m7OejHLmcioOj+SK+y8VCRFEU7umqIq5jPMyqWoilE0XEZmsmSTki8uK4NeMg6S7JSRFhX5/NqbbGiBnZnKq1d3gZvGbTUp9/Yh4WdNNl/pCW+hrHf69ucFJnI05Y0ohUJodHnj/g9XHyzCpCdpz6YbT91Y1QfH5Mvvo7DP70C1BT07oplfwgXGiPhrFyQd4n8rwHeSKDE0mkMjn4fYrramIT5z1ULA8pyrHFXMumEznkiDi5Ntl9+NXNqnLd+uU6jUc4zd/XPSJ2xnf5BpoBQCJlvxCJG5Y5iZiaAYDWQiEyEk8K+f5aPHUFqCFA/mnw709fDgD4wdb9yDgoJLlgKELOAvDamR9F8zkfAwCMb/0xhp/4GpDLkilVEBs99ImwtsySploEXH5qbqzj6x1j5u9oLZ/RXcAwncghR8RRa8bmPYeSVSXGuSJif9+MPjXj7Mk96PchUOixTqXtb4ZkBq+gX3HcLipFayGtcXhSrCJSCf4QxiUnLkJLfQ16xxPeRr7PKEIUvHHe/0Lj6ZcDAEaf/i7GnnkIAHBGWxuZUgUhQyHipj+E0cR5rJ9lkkQ4KiI8I965tGYo0Gz+4GTXDOCwNcNJEQH4GFbjBoOXqI2wolszlZCqOptw0I+/PbULAPDdZ/d5cwgW2X7PPThL8WHnxf+I6Pr3AwCGt9yH2F9+AgA4Y8kSPNPXR6ZUQWxYkTesvtkXmzGC6gY9bGLGxdFdhrb4jntrhp8iEuagiCS5mFXtPfymtNFhuW79cp3GI9KOzar2ZLJsTtV2BTh5UzJ4yIaa01zQxAwAtDWwQkRMa6YSFREA+OjGZfD7FPylewRv9sXcfXFjZLsvgF3v/xctqGzov76KyZeeAFAoQvbtg4/2xQijIxpGZ3Mtcirw0oFRV19bL0TcV0R4m1WFtGYcPuypqsplv5jtqRmH9zpRyHUaj+DWmrH4pjCOgDmR6Rh8FBEWiSzuRtNSn2/NjAhqzegZIvLGuxdjUWMtLjpuIQDgITdVEVaEbNuGVn8QiQ/eivpjz4SaTePwL/5/xF//PQAqQtxk/bJ89P8L+9wtRLxszfAONYtN5x+qIjzHdwsPe6lszpaXK51VwXajOgo003JE7LVmZNtILtdpPCLpVBFhxiGro1SGwkU6RUTQxAygt2biqayQxXeVqogA0Eyrj798SGjgm4axCAmGEP7rz6HuqFORSycx+NMvYvqt5wBQEeI265bn2zPb97usiIy6H2bG4B3zPpEUp4gA9q6zxsKBj0fE7q4ZuW79cp3GA1RV5egRsaeI1Ph9XJL/eKT+adsqOfZVZxMNBxD05/9/RfhEBsYrI1W1GKcsb8bqRVEk0jk8tq1H7IsZi5CaWtRe9nnULn8ncqlpDP74DiS6XwRARYgXMEXkpQOjrk1RJdJZDMTy7VJPzKp1nFszBUUkWsvvfRsK+MCsc3YKEWPopROfhh6iSWbVeQHzaABOChFnrRlexiEeqX9xzmuzi6EoiqaK8G7PTCTSmCj8P1TK+K4RRVFw9enLAADf37pfW9rFHUMR0hZuQN1H7kS463jkEpMYeOyzSPa8mo9s37iRihAPeEdHBJFQAPFUFjv7J1x5zYMFNSQSCmhFgZs01fLdQTVR8IhEOG3eBfI/n04WjBqv+U6GAZzmiFAhIhlpw9OG24Fm7E3kdPMug0fqnxtmVQBoLfhEhjgbVgcK/pBoOCAsB0U0H1i7BE11QRwcncZPtgtQRYxFSKQVdVd+GaHFxyA7NY6BR29Dqncnoj5fPrL9z3+mIsQD/D4F79R8Iu4Em7Gtu10tdcIm5soR5W5WZYoI36KqzsHiOx5hZoDe1rGsiFBrRk5SGQ6FiOYRsaeI8BjdBfik/sUF7GcoRmuDGEVE94dUllHVSDjox7XvXgUA+OJ/valNMjjGuEF32zYsaO1Ew0fvQk3bMmQmhjDwyCakBvYg6vNhmCLbPUczrLrkEzng4eguYPSI8DKrMo8I32tZmIMi4vSaH7apiPBK8uZN1V9lWIXo9yna0iWr2G/N8J3pri28uZ2YVVkhwjMEqBit9WJGePsq2B9i5BNnrsT6Zc2YTGbwTz96xXmLZvYG3cXHoOHKf0Mg2o70cA/6v/8vSA8d0IsQCirzHFaIuGVY9XJiBtADzWKJjOP3u6qqWo5IhLPfzUlrJqntFnO60sO6IqKqquH15br1y3UaD3BqVAWctGZYdcy3NeNEEZlwSRFhI7y8zar9FTwxY8TvU/DVv1mL+ho/nt83ggf+uNf+NxscBN7//lkbdL8Ef20Uyd6d6P/Bp5GdOExFiGSsXdoEv09B33gCh8amhb+e14VIo6GFEnPYnklmctpDJs9dM4DD1oymSDi79doZkJgxOkyKiFwktblqJ8Yhu1MzfGUyrTXDQRGp9NZMpSsiALC0tQ63v+84AMBXn9qF13vHrX+T7m5g40Zkfv97LFd8ePVdV6D9rz8HXzCM6T0vYODR25CbjlERIiF1NQGsWRQF4I5PhLUAOz0qRAJ+n5b54TTmnRUyisLf7xZ24MVLcHr4ZF9vJTJixugwKSJyoaeqcgiXsegR4S2T8TCr6oFmYivmVkEx71q8ewVOzBTjsvWduGBNB9JZFZ967GXzUuzgIPC3fwusWYNUdzci9U1I/s0X0XTGFVAUHyZefhKDP/si1HSSihCJWedSe0ZVVa0Q8UoRAfgZVplRNRIKcIlGMOJEEeHVjtfNqubvOcYHZSpEJINH9r7t1gxnRYTH+K5bUzOi9s3MJ0UEyI8Lbv7QCWhrqMFbA5P4zOOvlf/3LSgg6OpC5tFHcUYigejSE9B69TdQu/wk5FIJDP3qboz89t/1DbpUhEjL+uXuJKyOxFOIp7JQlPzmXa/gZVgVEe/OcJJgzasdr5tVrSgi+r3Oi6moclT9XB6P7H3bZlWOC+8AIKwFmtlflKXliIg2q2obeMWYVSt5amY2rQ0hfPnDJ+Kah17AT7YfxPb9o/jSB4/H6avagMWLgb6+GZ+fArAKwGDbUkRO/SA6jjsXis+P1OH9OPyLzcgMHwQAdNXWYt/YGC2vk5j1y/IJqzv78wvwRK1e6BnNK4kdkbCnExW8Qs1EGVUBoDZovwXOa9s6+zeyoojwzq3iiXwnchm+ZlW7gWacFBEt4t1+EqOWrCraI8ICzTgqIvFkRruALW6aX0/4563uwLevWof2SAjdQ3Fc8R9/wb/8+BWMjk7ihwAUw69o53FIfvhzWHTNfWg44T1QfH5M7ngK/d+7GZnhg/ADeFdjI/ZSESI9CxvdWYDntVGVwSvUTBvdFfBAVVuTv97b8oiwaz43s6oFRYQVQZIZVQFSRPQlQAEHZlW7u2Y4OagZumRoXxGZdNmsOpXKYjqVnbHDwS59BX9IJBwQ8iTkNRcetxCnrWrFvz25Ew9vPYAfbz+IJ//xexgZG8JiRYECAP4gApFWAICq5jD11nOI/eWnSPW9BSD/5DH15S+j5uabAQoqqwjWL2vGwdFpvLBvFGcevUDIa3i5dddII6d9M0JbM0H7LXAt0Mzhw2fIhiIi6+guQIUIl6Q5pmikbLZmeCkiTpfeqapqiHgX+9ZoCAVQ4/chlc1hOJ5EZ43zC+ChsXxbZvE8asvMJhoO4s5LT8Cla5dg089exduDkwg2L5rxOWomhcnXfofY8z9HZrRX+3gEwNDEBGoaGlw+NeGEdctb8PjLvUINqz0eh5kxGjmZVfXWjAhFxH5wJLdAMxs5IrynNHlChQiH7H2n47tOZTqGExMVOw+bMxetiLB9M/2xBEbiKXRy2PbZV8hamG9tmWKsX96CJ248E2/2xXDKhtMAqICa/8fLjPYhl9D3k9QC+NTSpfjirl3wkSm14pi9AC8gIJ5bntZMQRGZdmhW1Voz4hQRe60ZPh4RO75EmRUR+U7kMnoh4mR812GgGW9FxGYhwtZmK4ruNxEJa88Mc8oS6S0UIos8dP27SdDvw4mdTXjwrs8i1bsLqb63kOp7a0YR8q9LlmByyxZ8qbubipAKxY0FeNIUIsysyqs1I0ARcTKdmOQ0oGBHEUlymBAVhXwnchmerRnru2b4KiJO5tsBPUOkvob/7H0xeI/w9hYmZrwcP/SCK664AuoPfgAVyP/q6IC6cSPUJ5/E5gMH4KOdMRWN6AV46WxOK+K9LkQamVmVU2tGpEfEznQir/aIURFRVXNx+LwW7omg6q9OeqCZF8mqfBURJ4l/gDFV1Z03ahvnEV5NEZknGSKWuOKKfGtGVYH+fuC554ALL6QCZJ6wYUV+jPcv3fwLkb6xBHJq/jq2IBLi/v2twC1HRGRrxlHEO5/2iFFRMXvfofFdieE5vpvK5pCzsKyJd4XKFJFkxto5GG5NzDBaOI/wsgyRxVWmiBDzH1aIPN89YvoJ2CwHDBMzXgdd6WZV+5N/gGCzqqOld3yu+cavN6vE660ZUkSkI8nDrGp4U7BWjxl4V6jGEVg71bpbEzMM5hEZ4uARUVVVU0Tm89QMUZ2c2NmEcNCH4XgKuwcnuX5vWfwhgDHQLOWo4BI5vuss4p2PRyTgU8C652a9iUlOry0C+U7kMjynZgBrPhHeEe/GFo+dHxK3wswYeqiZ89bMSDyFZCYHRQE6Gr2VlwmCNzUBH05emveJbOXcnukZLSgizd4X8CzQLJ1VHe3MihUUFRGtmbCTQoRTZIOiKJbTVUkRkRimYAQdtGbsVKfGz+VVofp8iva97MiG7rdm8gUDj9YMa8u0NYSk/EEjCKdsWJEPqvvL3mGu3/eAJGFmQP5ayB4KnRhWJwqKiHStGY4hluwBOGFREeE1HMET+U7kMmkOioiiKLbmunnNlBtxEmpWya2ZQ1qGiPdPdQQhgg0rdcMqT5+IDFt3GYqi6FkiNg2rmWwO8UKRILQ1YydHhGOIZdji1nca35UYpoiEHIYE2ckS4b30DgDqHKT+TbLxXZemZnjum9HCzKpxYoaoCtZ2NaEm4MPhiSS6h+Jcvmcup6L7cP57LWut5/I9neI0XZUZVQGxishUOmu5IGTtpjoOKy201ozJe47uT5FPMaZChIMiAhhkMg89IvnvZX8hk66IuLOnhW3gnU5nHW0MBvQMEVJEiPlKOOjH2q4mAPzGePcOxTGRzCAc9GHVAjkKEaehZqwQqQ36HbXcS8GGAlTVemTDlPaw57xA0mIjSBGpfPgVIjZaMwIid5kiYif1bzLBChF3Kub6Gr/29+40XbWqM0SIqmEjyxPh5BPZcXAMAHD84kYh0fF2cBpqpk/MiGkx1xoeHK0qz/HCAxcPRSRkcfkemVUlJsnBrArYW8ssQipzsgdhMuWuWVVRFLRxSldlhUi1paoS1cWGlQXDKiefyI6D4wDy48Gy0ORwA29MM6qKUXYDfp+WO2XFi6eq+iQQT0XEtFlVgBWAF/KdyGV4mFUBo0fEnCKiqqpeoXJ8YzhJ/Yu7PDUDAC0NfEZ42dRMteyZIaqTk5c2I+hX0DeeQM/ItOPv90pBETmpq9Hx9+KF08V3+uiuuOuYNp1ocddLthA0ydMjYrY1I2I4ghdVX4jw2DUDWN83k8rm2LJUMVMzNjwXbk/NAPoIr5PWTCabw0CMeUSoNUPMX2pr/Jp6sbXbWXsmnc3hjd4YALkUEc2s6lARETExw7Cz6dyoUrMWuhPCNhURGt+VEN5mVdMpdwblhKdUxn5AbLVmCkYqNwsRHq2ZgYkkcioQ9Ctoq6cwM2J+o+2d2evMsLqrfwLJTA7RcADLW70f3WXoKqm9a4Ie7y6uENGmEy0oIuxBLxz0wc9hqWjI6vhumsyq0pLi5CS2uviO+UMUxbkaY4QVEexNb4XJZP5JwtXWDIcRXuYPWdgYdmVrMEF4ie4TcaaIsLbMiZ1Nnu+YMdLq8OFEX3gnsjVj/YFP84dwUEMA64qINhxB47vywSNZFbA+NWOsTnleBNjcfCxhpzXjviLCRniHHGzgpR0zRDWxblkz/D4FB0entSA/O+zoYUZVefwhgH5NsLuV243WjJ1QM21ihtNUouWId1JE5IVba4aZVU2PUokJl2FyJPthtIIe8e5excwj1Kx3jDJEiOqhIRTA8UvyxYOTMV6jIiITmiJi0zcmcvMuQ0+wNv/Ap2WIcFJE7NoByKwqIfzMqlZbM4UwM85vCjY7P2FREUlnc1pR5q4i4uyiAwB94yzenYyqRHWw0aFPZDqVxduFLb4yTcwAuiIykcxYikNg6K0ZN8yq5nOjeGaIAHYi3ml8V1q8CjRLCFpAxBSRCYuKiNFTUqkekUXUmiGqBLZ3xu7kzOu948jmVCyIhLAwKlcBHw0HEPTn29V2rgvuKiJWPCJ84xGsKiI0visxXk/N8FZENI/ItDVFhP3whgI+IbHIpWhl47vxpO2AJtaaoTAzolo4ZXkLgn4F+4en8NbAhOWvf6UQZHZSZ6NURlUgH3TY6mCs35XxXRsxCcyDx1sRMe0REZDkzQv5TuQy3FozmkfEmiLCWyZjcuRE0qIiknI/QwTQWzOJdM7WyDEA9BZaM4uoNUNUCZFwEGcdvQAA8OsdfZa/foek/hAGU0rtmNi1QsSN1owdRYTX1EyQ7Teb+wyiAjR5Id+JXIZbsqrl1owYmYyNrFn1iHiRqgrknw5YhW5Hhp1KZbQoaDKrEtXEe09cBAD49at9ltVEPdpdLn8Iw4l3jF37RI7v2slr0hQRTsMAVu45xgBN2r4rIZoi4nprRrRHJGPp4uRFmBlQ2DfjYISXtWUaQgGhT0AEIRvnr+lAjd+H3YOTeGtg0vTXjU+n0T0UByCvIsKuCcMWVz+oqqoXIiLHd4PWx3d5KyIhC4qIsVih1oxk5HIq0tn8zdqzqRnO1SmbmsnmVIvVujetGcCZYZUmZohqJRoO4qx3sPZMr+mve7WghnS11Go/e7JhN9RsKpXV9rmIfDCpY8GRlnJEmEeEl1nV/PZdZhngHaDJC/lO5CJMDQF45IhYXUAkxjhUG/Rr8cFW2jOTCfczRBhOZFiamCGqmUsK7Zn/stCekTU/xIgeambtmsD8IQGfInRMNVIoRCYtTCdOcc5pCltYtGq838hmTgaqvBBJGwoR58mqNqdmOCsiiqIY0lXN/5BMeuQRAXRFxE6kM4WZEdXMeavbURPwYe/hOHb2m5ueYUbVtTIXIlqombXWjLZ5tzYo9IbbULjGTlpYpcFbEdGnZsy3ZmQc3QWqvBBJGSpJbtt3LeaIiKjaozayRLxszdi96ADGeHdqzRDVRyQcxNnvsDY984qk0e5GNJXU4sMJu+aJzBAB9OukFdVZzxHhZVY1r4jIPLoLVHshou2ZURwvS7PsEdHeGPwrVDv7ZiY9Gt8FdBnWnkeEFBGiurEyPTMYS6A/loBPgRYTLyNOWzOijeu2FJGkKEXETCEiRoHnRXUXIhk+GSKAjV0zmlmV/z+BHmpmXRGpvNYMZYgQ1c15qzsQCvjQPRTHG32xsp/LgsyOam/w5GfdLLpZ1VrQod6aEfv/pnlELBQi+tQM74j3ue85ojyJvJDzVC7BCpEgh38cpmykzG7fZbn/AhSRqGGE1yzMrOqFItLWoF90rKCqqhZmRqmqRLXSEArgnGPaAczdnvnBX/YDyG/wlRm7QYdaaybkkiJiISZBzxHhHfFuXhGRMcwMqPZChFOqKmB/fFfEGyNipxBhmyE9UUQKrRmLMuzoVFr7e1xIHhGiijHTnvnDzkE8veswgn4FnzhzpZvHs0xdTUCLUbfSnokl3FFE2ANbJqeavuaLUkRS2Zw2slyKpMR7ZoBqL0Q4paoChtaMxUAzET07O1Mzcc6jZVZgMuxQPGVJhmVtmbaGkLQ/YAThBuce245w0If9w1N4vffI9kwqk8MXf/0GAOBj71qBlQsa3D6iZZgqMmRBKXXLI2IMJTP7wKdNzXBWRIC5lXiZN+8CVIgA4FSIBKzmiIhZegfoiYKWpmZS4jdWlqI9GoJPyf97HLYwOaNNzJA/hKhy6kMBnHtsvj1zxy9fP8K78L3n9mHv4TjaGmpw/blHeXFEy9gxrDKPSERwIeLzKZoqYsYnks7mtPsNb0UEmHuElxQRifG2NSMm4h2wt29GyxHh5Oi2QijgR1dLHQBg96D5qOqeUTa6S/4Qgrjh3KMRCQfwwv5RfOzB5zWVc2gyiXt+9zYA4F8uPKZiViG02Rjr1zfvir+OaYWIieus0efCa2rG71MQ9OenPRNzKPE0visxfBWR/PdIZXPIzdGvA8QGzNiZmtGTVb1x0h9VkIr3WChE3ioEOL2jQ36ZmSBEs3pRFN+/ZgMioQC27RvFxx7chngyg69seQsTiQyOXxLFX6/r8vqYprGTJaIvvBNfbDHDqplN58wfEvQrXO43DLNKfNWO7+7btw/XXHMNVqxYgdraWqxatQq33347UinrI5qiSPNURAz/wMbo+FKIVUSsm1W9DDQD8uOEgDVFZGd/vhd+zMKokDMRRKWxtqsJ3/94vhh5ft8IPvLAc3h02wEAwO3vO05b/1AJMBO7tdaMO4FmgDVFhHeGCIN5PuZSRKp2fHfnzp3I5XL49re/jddffx1f+9rX8K1vfQu33nqrqJe0TFKAIgKY84loyapCFBFrhUgqk9OMVE113si2q1ghcthcIZLLqdrG0WMWRoSdiyAqDWMx8tqhGFQ1v5PmlOUtXh/NEnbG+vXWjPjrWMRCqBnviRmGVUVE1kJEWNl40UUX4aKLLtJ+v3LlSuzatQv3338/7r77blEvawmerZmAT4FPAXIq68eV/0HQpTKBgWYmzapjU/knDp/ijqRZDKuKyIGRKUynswgFfFjeWifyaARRcaztasL3rjkVf/ed56EA2PRXq70+kmXsLMNkD1+uKiImChHeGSIMpqjPaVbVckTkbM24qsOPj4+jpaV0VZ5MJpFM6tVvLFY+JdApPM2qiqIgFPBjOp01uQ1RnEdEn5oxp4iMTuULlqa6GsdR93ZhhchALIlYIj1nQcQWfB3d0YCAhGutCcJr3rm0Gc/+67lIZnJoK0ygVBKthdbMkBWz6rQ747uAtX0zohQRpqgn5rjn6Aq8nNdK1061Z88e3Hvvvbj22mtLfs7mzZvR2Nio/erqEmus4pmsCljLEkkKXHpnlAznCroB9B0vzR61ZYD8haM9kr/wmDGsav6QDvKHEEQpIuFgRRYhgK6ImN1BlczoD4FutGas7JvhvXmXYXa1iDa+K6kiYvkueMcdd0BRlLK/XnjhhRlf09vbi4suugiXXXYZPv7xj5f83ps2bcL4+Lj2q6enx/r/kQWYWTXE6Yma9d+8XkJklCXNGKlGC60ZtvPFK6y0Z3YVFJHVi8gfQhDzEaaIjMRTpiYRjcqEG6b7iJXxXUGBkWYVEdnHdy3/a11//fW4/PLLy37O8uXLtf/u7e3FOeecg9NOOw0PPPBA2a8LhUIIhdyr3nl6RACDcWiON0U2p2ptIRFvjFDAj1DAh2Qmh1gijcY5lA5WiDTVeV+IPLtn2JRhlRUiZFQliPkJezDK5FTEEuk5r0/axEwo4Mp0kAyKSNikRyQhuSJi+W+lra0NbW1tpj730KFDOOecc7Bu3To8+OCD8Pnkqsb4FyLmWjPGPxc11x0JB5GcTJrqX44WpM8WCQoRYO7WzHQqi33DcQBUiBDEfKUm4EM0HEAskcHQZGrOQkTLEHGhLQMADSHzXjxRiojZh1/ZFRFhp+rt7cXZZ5+Nrq4u3H333Th8+DD6+/vR398v6iUtk+RoVgWMHpE53hSG1o2oN0bUwuTMSDz/Oc1et2YWmGvNvD04gZyaf2JaUKH9b4Ig5qZNi3mf27DKrnVuranQFZG5r7GiFZE5PSLVOr67ZcsW7N69G7t370ZnZ+eMP7Oy2Ewk3M2qJme6WfhMwKcIm/iIWJicYeO7XppVAT1L5MDIFBLpbEm1iE3MHLswAkWpnIAmgiCs0dpQg71DcVOGVbZnxq0IgoiF8V3hOSJzKiJVumvm6quvhqqqRX/JAs9kVcB8a0ZbeCewX6fvmzGhiLBCxGNFpD0SQiQUQE6F1nopBvlDCKI6aDFs5p6LCa8UESvJqpxNtOY9IrR9V1rEeUS8X8lsZd/MqDa+620hoiiKnrBapj2zy6CIEAQxf2m10ZpxzyNiJdBMUI5I4WHWdKBZtSkilUCKc9/MrEwmMsyMYWXfDAs0a6n3fiunblgtrYjQjhmCqA70DbxmFBHWmnFJEbEQaBYvtGa454hYfPgVsduMB3KeyiW0ZFXegWYmZTKRb4qIthnS/NSM14oIYMgSKTHCOzSZxNBkCopCW3cJYr6jKSIm9s3oC+9c8ogUrrHJTE57qC3FVMGsyn1qxqQiotkBSBGRj1Qm71cJcveImFzJLPBNoS++K9+aSWdzWrEiRSEyx+QMa8ssa6nj/nRBEIRcWNk3E9PGd925LtQb/B7xOR742J97poi48PDrBDlP5RI8d80AVloz4t8U2vjudPkfkFHjwjuXeqvlYIrI3sOTRePpd5JRlSCqBmZWHbZgVnVraibo92k+v7l8IqIUEeseETlv+XKeyiVShb6Z24Fm+gIi8YrIXDkio3F94Z0baYRz0dVSh5pCKuyh0ekj/nxnH/lDCKJasJQjMs0277r3QGU21GxKkEdEL0RKP/yqqkpmVZnhPjWjeURMtmbc8IjM8QOix7t7r4YAgN+nYGVbPQBg9+GJI/5810BhxwwpIgQx72ktKCKjU2lksuWvq4MTCQDu7syKmIx5Z+O79cJaM6Uffo0KPY3vSohXrRlR/UIjrM0ytyIiR7y7kVIjvNmcircGqDVDENVCU10NmFDL8o6Kkc7m0FNQUJe31blxNADGEd7S19lsTsV0muWIiGrNlL7nGO9HpIhISLpgVnW7NcNUCpHBO2YVkRFJFt4ZKWVYzSeu5hAO+rCstd6LoxEE4SJ+n6L7RMoYVg+NTiObUxEO+tARCbt1PFMjvNMG/4Y4RaRcIZJ/fUUBgn7v2+/FqOpChPv4rkkHsxsJgFGTUzNjEmWIMI4qoYgwf8jR7REp/CwEQYiHFSLlYt67C0nMy1vr4XPx2mBmAy9beKco/FsjTBEpFxmRNIzuyroSo7oLkQzn1oz2ppirEBFvqmKFSCJdfsad/XB7He9uxFiIGFcC7KREVYKoOlrr84bVoTKG1X1D+UJkWat7bRnAsG+mjCLCFt7V1wS4FwJmIt5lDzMDqrwQSQqLePe+NdNg+N7lVJHRKXnCzBgr2urhU/K5AIcNFx/aMUMQ1YeZLBFWiCxvc7dla0YR0T2B/P0ZZnyJepK3vLd7eU/mAmx8l1+gmTmzqr6uWpwi4vcp2l6Dcv1LGc2q4aAfXS35J5vdg5M4NDaNh7fux/P7RgAAx9LoLkFUDW0m0lX3DU8BAFa47B0z4xHRM0T4P3haUURELll1SlVHU6azedmf364Zsx4R8YoIkJ+ciaeyZSdnRqZYjog8HhEgb1jdPzyF637worYLB8j/HR+/hAoRgqgWWk2YVdm2bikVkZQ4RUSbmilnVq0ARaSqCxGvds24lQAYCQfQN16+Wh8rtGbcnL03w7GLIvjdzkGMTqXhU4B1y5px9jHt+KsTFkk14UMQhFhaGsqnq6azORxko7suKyJmPCJTgjJEAL24yOZUZLI5BIqo+7KHmQFVXIhkc6oWIc47R2SuBUhuKSJm9s3IaFYFgI+fsRI1fj9WLKjHWUe3UfFBEFUKM6uWSlc9WBjdrQ360RENuXk0a4oI5wwRYGa7JZHJoaFoIVIwq5IiIh/GYsH98V22rlqsIjLXvpl0NqedRSazKpAvjG48/2ivj0EQhMe0zaGIGCdm3B5P1SLeTYzvilBEjA/RyXRW86wY0TbvSuwRkbdEEoyxEOFmVg3OPTWTzua0gBu3FJFSHhGWIaIoQKMEC+8IgiBm06rtmyleiHQXCpEVLvtDAEOyahnVmY3vivCI+HyK9iBdyidSCYqIvCcTTCprLET4VNHa1EyZHBFjL7FBeCFS3tGt7ZmpDVJAGEEQUsLGdyeTmaLTIcyo6kXaspldM2zhnYipGQAIB8pPzmgeEcoRkQ+jUZWXnGemNcOKgtqgn5sSU4q59s2w0V3Z2jIEQRCMSCigPSwWS1fVRndd3DHDaDATaMbMqgI8IsDcQZpubHt3SvUWIqxK5FgMsEIklc0hl1OLfk7MhXh3hllFRDajKkEQBENRFCxszO+Pmb32ATCEmXmgiDBVO57KasMPs5nSxncFKSIsS6SEJUAb3yVFRD5SnFNVAb0yBWa2foy4NTGTf43yUzMj8fzHmyXLECEIgjBy6vJWAMBze4dnfDyVyeHgKFNEvPOIAPp0zGz0iHdBisgcloBKGN+t+kKEZ3vEaAYq9aaYcCFVlTHX1IyM8e4EQRCzOX1VvhB5ds/MQuTg6BRyat4IuiDi7ugukL/ms7ZRqfYMm5qpE+URmUMRYa0ZMqtKCO8wMwAI+BQwz2epyRk3FRFtA2+yvEdEtjAzgiAII6cVCpFXD47N8LwZjapebJZVFEX3iZQwrBqX3okgHCi/gVc3q5IiIh0iWjOKosy5bybmUqoqMLdHZIRNzZAiQhCExCxuqsWKtnrkVOD5vSPax7uHvDOqMuYKNZsSGGgGGGMjaHy34tAUEc6TK3NlibiqiLCpmenyOSIt9eQRIQhCbpgq8uc9Q9rHvDSqMlioWenWjDuKyJzju1SIyIcIRQTQ/7ETc3pE3J2aUdUjHd0jNL5LEESFwHwizxl8ItqyOw8LkcicrRlxS+8APTG1lCKije9Sa0Y+tEKEtyKitWbmUkTcaM3kXyOTU4sWRmM0vksQRIVw2sp8IbKzfwJDhb0zXm3dNaK1ZuZSRASZVUNmA81IEZGPtACzKmAINSupiLjXmqmv8Wvm2WKhZqSIEARRKbQ2hHDswggAYOveYaQyORxiW3e99IgUCoxi+2ZUVdUUEWHju0HWminhEUmTWVVahLVm5jAOxVwc31UUpWSWSCabQ0xbeEceEYIg5Of0VW0A8mO8PYXR3foaPxY0uD+6yyiniCQzObCcM1Hju3qid4nxXTKryktSlFnVdGvGncXH7HVis35IxqZp4R1BEJWF0Seib931ZnSXoXtEjlSd4waVpFaQIhE2qYiQR0RCRJtVSykibppVAX1MePbkDMsQaawNIiB45w1BEAQPTl3ZAp+S37jLws28SFQ1Ui5HZKqQIVIb9AtbLKoFmpX0iJAiIi3MI8J78ZxZj4gbOSJA6SwR8ocQBFFpRMNBnNDZBAD46YsHAXjrDwH01kyxvCbNHyIoQwTAnNlVZFaVGHGKiGytGeYRmflDMjpFe2YIgqg8WHuG5SAt83B0FyiviLDNu6IW3gFzKyIJas3IS0pQlVjOrJrO5jBdeLO4YVYFgGgt84jMas3QnhmCICoQVogwvG7NRMqYVacEZ4gAZhQRas1Ii4hdM0B5j4jxjeq2R2T21MwoZYgQBFGBrF/WMmPIwMswM8CQrFpGERGVIQKY8YjQ+K606Nt3+RqIQmUWELH2SDjo4+5NKUUpjwgtvCMIohKprfHjnUubAOTbIm0N3l7DynlE3FBEtGTVIr7EXE7V7nVhUkTkQ981w/cNUk4RcTNDhFFqamYknv99E3lECIKoMFieyLLWOk9Hd4E5PCKCN+8C5XNE2H0OIEVESkQHmhWTyfSJGXfaMkBpRYTFu7eQR4QgiArjw+uWYPWiKK7YsNTro+gekeSRO72mkmI37wLlc0SMKonMHhH37oiSIaoQYeFgY0U23k54oIiUmpoZIY8IQRAVSmdzHX5z45leHwOArohkCzu9ag1tGDcUkXCZje8sVdWnAAFBOSY8kLdEEoyoQqQjGgYADMaSR/yZ26O7QJmpGcoRIQiCcExdjR+sOzQxK13VDUWE+RLLKSLhoN/zFlY5qrcQ0TwifP9xFkTyOw8GJhJH/BlTRNwKMwPmzhFpqSePCEEQhF0URdF9IrOus24qIokiikgljO4CVVyIiNq+K5siou+a0Sv1TDaH8WlmViVFhCAIwgmREoZVV3NEiikiWl6WvEZVoIoLEfYPxHtqpr2giEwmMzMWHgH6mmhXWzNhfca9Z2QKALQiBACaaOEdQRCEI0pt4HUjRyRkUERmm2XZ0ARTTWRF7tMJRJRHpCEU0KrfwYmZqogXZtW2hhqc1NkIVQU+9t1tGJ9Oa2FmtPCOIAjCOaw1M+GBIsKmZlQVSGdnFiKkiEiOqEJEURRDe2amTyQ27b4ioigKvn3VeiyMhrF7cBL/38PbtbYR7ZkhCIJwTgNTnj3wiBj9H7N9IppHhBQROUllxSSrAkbD6kxFxItAMwBY2BjGd64+BfU1fjy7Zxifefw1ADS6SxAEwYOSHhEXpmZq/D5tamd2fhWbpCGzqqQws6qIf6BSiogXZlXGmsVRfPPKk+H3Kdg7FAdAYWYEQRA8KJWuOuWCIqIoip6uOsuwyhQRmTfvAlVciKQEmVUB3bBa2iPiTY7c2ce04wsfOE77PU3MEARBOKfUvpl4wSNSL1ARAQz7Zma3ZkgRkRtRHhEA6IgWWjMlFBE3c0Rmc+WGZfjk2asAAGu7Gj07B0EQxHxBV0RmB5rlC4M6gYoIAIRLhJpVilmVIt6FtmZmKyLetWaM3HLRsfiHM1agrSHk6TkIgiDmA5Ei47upTE7zIopszQC6GXW2IsI8I2RWlZSkK2ZVXRFJZ3OYLrwp3DarFoOKEIIgCD4U84hMp/SioFbg+C5Q+YpIVRYiqqoKS1YFdEXksEERMVbKXisiBEEQBD+KeUSYP6TG7xNynzFSavEdRbxLTCanggXQhQSaVScM6arsDRoO+hCkEDGCIIh5QzFFRAszE2xUBUovvtPGd6k1Ix/MHwKIUUSKpat6lSFCEARBiEXziBgKES3eXbA/BDDEvKeLKyJhas3Ih+hCpFi6qixGVYIgCIIvDaEjk1XjLsS7M7TFd5lZHhFSROSF+UN8CuD38TerAkemq3qxZ4YgCIIQj+YRMbZm2OiuwIV3jHAJRSRRIWbVqnw8Twoc3WWUUkSipIgQBEHMK5hHJJXJIZnJ4sX9Y7jnd28DAOpdUET0QLPZikhlbN+tyrsim+2uEWganZ2u6nWqKkEQBCGGBoPq8Xf/93n8pXsEQL4tc/mpS4W/PpuKOdIjQoqItOhhZuL+cWanq2oekRC1ZgiCIOYTfp+Cuho/plJZ/KV7BAGfgis2LMUN5x6ttelFwhSRI3NEKmN8t7oLEQFhZozZ6aqsd0iKCEEQxPzjqPYG7Dg4jvedtBj/9J53YHlbvWuvrS29OyJZNX+vk33pXVXeFUWGmTFmp6uSWZUgCGL+8v1rNiA2nUZXS53rr11aEamMpXdVWYiI3DPDmJ2uGmNm1dqq/CsnCIKY1zTWBtFY682DJiWrViBJFxQRVohMJDOYSmUMOSKkiBAEQRD80HJEZisiWo4ItWak451dTfjR/z5NaJXI0lWnUlkMxpI0NUMQBEEIoVSOiJasSuO78tFUV4NTV7QIf52OaBjdQ3EMxBKITVMhQhAEQfBnzmRVycd3XSmTkskk1q5dC0VR8PLLL7vxklJgTFfVA82oNUMQBEHwo7QiUhlmVVdOd8stt2Dx4sVuvJRUGNNVadcMQRAEIYJQkWTVbE7VwjtlH98VXoj85je/wZYtW3D33XeLfinp6CgoIr1jCUwXKlUyqxIEQRA8KZasalzuKrsiIvTxfGBgAJ/4xCfw+OOPo65u7tnqZDKJZDKp/T4Wi4k8nnDaC+mqew5Pah8jRYQgCILgiZYjYhjfNY7yyl6ICDudqqq4+uqrce2112L9+vWmvmbz5s1obGzUfnV1dYk6niuw1szuwXwhEg76EBS434YgCIKoPsJFxndZuJnfpyAg+X3H8unuuOMOKIpS9tcLL7yAe++9F7FYDJs2bTL9vTdt2oTx8XHtV09Pj9XjSQUzqx4amwZAbRmCIAiCP6EiZlVtdFdyNQSw0Zq5/vrrcfnll5f9nOXLl+POO+/E1q1bEQrNXPizfv16XHnllXjooYeO+LpQKHTE51cyTBFhUFuGIAiC4I3emtEVEW1iRnKjKmCjEGlra0NbW9ucn/eNb3wDd955p/b73t5eXHjhhXjsscewYcMGqy9bkRxZiJAiQhAEQfCFeUBSmRxUVYWiKJo6Irs/BBBoVl26dOmM3zc0NAAAVq1ahc7OTlEvKxXGdFUAiJIiQhAEQXDGOJ47kcwgGg5qiojso7tAle6acROjKkKtGYIgCII3dUE/ljTVAgC++fvdAIypqvLf5l074fLly6GqKtauXevWS0oBM6wCQCRErRmCIAiCLz6fgi984DgAwH/8aS9e6RmrmM27ACkiwiFFhCAIghDNeas78IG1i5FTgVt+skNL85Z9zwxAhYhwOoyKCJlVCYIgCEHc/r7j0Fpfg10DE/jmH/ItmpDkm3cBKkSEw9JVAVJECIIgCHG01Nfg84UWzduFIE1SRAhqzRAEQRCu8d4TFuGCNR3a70kRIWaaVak1QxAEQQhEURTceenxWlxEmBQRwqiIUI4IQRAEIZr2aBj/50MnoL7Gj9NWtXp9nDmhO6NgZrZmSBEhCIIgxHPJiYtx8fGL4PcpXh9lTkgREUxDKIDW+hoAM9s0BEEQBCGSSihCAFJEXOFbV63DYCyJhY3huT+ZIAiCIKoIKkRc4JTlLV4fgSAIgiCkhFozBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4htTbd1VVBQDEYjGPT0IQBEEQhFnYfZvdx8shdSEyMTEBAOjq6vL4JARBEARBWGViYgKNjY1lP0dRzZQrHpHL5dDb24tIJAJFUbw+jufEYjF0dXWhp6cH0WjU6+PMW+jv2R3o79kd6O/ZPejvWkdVVUxMTGDx4sXw+cq7QKRWRHw+Hzo7O70+hnREo9Gqf5O7Af09uwP9PbsD/T27B/1d55lLCWGQWZUgCIIgCM+gQoQgCIIgCM+gQqSCCIVCuP322xEKhbw+yryG/p7dgf6e3YH+nt2D/q7tIbVZlSAIgiCI+Q0pIgRBEARBeAYVIgRBEARBeAYVIgRBEARBeAYVIgRBEARBeAYVIhVOMpnE2rVroSgKXn75Za+PM6/Yt28frrnmGqxYsQK1tbVYtWoVbr/9dqRSKa+PNi+47777sGLFCoTDYaxbtw5/+tOfvD7SvGLz5s045ZRTEIlE0N7ejksvvRS7du3y+ljzns2bN0NRFNx0001eH6VioEKkwrnllluwePFir48xL9m5cydyuRy+/e1v4/XXX8fXvvY1fOtb38Ktt97q9dEqnsceeww33XQTbrvtNrz00ks488wzcfHFF+PAgQNeH23e8Mwzz+C6667D1q1b8dRTTyGTyeCCCy5APB73+mjzlm3btuGBBx7AiSee6PVRKgoa361gfvOb3+Dmm2/GT3/6Uxx33HF46aWXsHbtWq+PNa+56667cP/992Pv3r1eH6Wi2bBhA04++WTcf//92sdWr16NSy+9FJs3b/bwZPOXw4cPo729Hc888wzOOussr48z75icnMTJJ5+M++67D3feeSfWrl2Lr3/9614fqyIgRaRCGRgYwCc+8Ql8//vfR11dndfHqRrGx8fR0tLi9TEqmlQqhe3bt+OCCy6Y8fELLrgAzz77rEenmv+Mj48DAL1/BXHdddfhve99L84//3yvj1JxSL30jiiOqqq4+uqrce2112L9+vXYt2+f10eqCvbs2YN7770XX/nKV7w+SkUzNDSEbDaLjo6OGR/v6OhAf3+/R6ea36iqiptvvhlnnHEGjj/+eK+PM+949NFH8eKLL2Lbtm1eH6UiIUVEIu644w4oilL21wsvvIB7770XsVgMmzZt8vrIFYnZv2cjvb29uOiii3DZZZfh4x//uEcnn18oijLj96qqHvExgg/XX389duzYgUceecTro8w7enp6cOONN+Lhhx9GOBz2+jgVCXlEJGJoaAhDQ0NlP2f58uW4/PLL8atf/WrGRTubzcLv9+PKK6/EQw89JPqoFY3Zv2d2Uent7cU555yDDRs24Lvf/S58PqrfnZBKpVBXV4cf//jH+OAHP6h9/MYbb8TLL7+MZ555xsPTzT9uuOEGPP744/jjH/+IFStWeH2cecfjjz+OD37wg/D7/drHstksFEWBz+dDMpmc8WfEkVAhUoEcOHAAsVhM+31vby8uvPBC/OQnP8GGDRvQ2dnp4enmF4cOHcI555yDdevW4eGHH6YLCic2bNiAdevW4b777tM+tmbNGnzgAx8gsyonVFXFDTfcgJ///Od4+umncfTRR3t9pHnJxMQE9u/fP+NjH/vYx3Dsscfi05/+NLXCTEAekQpk6dKlM37f0NAAAFi1ahUVIRzp7e3F2WefjaVLl+Luu+/G4cOHtT9buHChhyerfG6++WZcddVVWL9+PU477TQ88MADOHDgAK699lqvjzZvuO666/DDH/4Qv/jFLxCJRDT/TWNjI2praz0+3fwhEokcUWzU19ejtbWVihCTUCFCECXYsmULdu/ejd27dx9R4JGQ6IyPfOQjGB4exhe+8AX09fXh+OOPxxNPPIFly5Z5fbR5AxuNPvvss2d8/MEHH8TVV1/t/oEIogTUmiEIgiAIwjPIdUcQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGf8P6ZyOHzGZunmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_41_2.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "backend = np\n",
    "\n",
    "def function(x):\n",
    "    return x*backend.sin(x**2 + 1)\n",
    "\n",
    "def analytical_gradient(x):\n",
    "    return backend.sin(x**2 + 1) + 2*x**2*backend.cos(x**2 + 1)\n",
    "\n",
    "\n",
    "x = np.linspace(-5, 5, 100)\n",
    "plt.plot(x, function(x), label=\"f(x)\")\n",
    "\n",
    "descent_x, descent_y = gradient_descent(1, 0.01, 300, solver=\"analytical\")\n",
    "\n",
    "# Change the backend to JAX\n",
    "backend = jnp\n",
    "jax_descend_x, jax_descend_y = gradient_descent(1, 0.01, 300, solver=\"jax\")\n",
    "\n",
    "plt.scatter(descent_x, descent_y, label=\"Gradient descent\", marker=\"v\", s=10, color=\"red\") \n",
    "plt.scatter(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\", s=5, color=\"black\")"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}