FYS-STK4155/doc/LectureNotes/_build/jupyter_execute/exercisesweek41.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e3354581",
   "metadata": {
    "editable": true
   },
   "source": [
    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
    "doconce format html exercisesweek41.do.txt  -->\n",
    "<!-- dom:TITLE: Exercises week 41 -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d5fc499",
   "metadata": {
    "editable": true
   },
   "source": [
    "# Exercises week 41\n",
    "**October 9-13, 2023**\n",
    "\n",
    "Date: **Deadline is Sunday October 15 at midnight**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3ef3596",
   "metadata": {
    "editable": true
   },
   "source": [
    "# Overarching aims of the exercises this week\n",
    "\n",
    "The aim of the exercises this week is to get started with implementing\n",
    "gradient methods of relevance for project 2. This exercise will also\n",
    "be continued next week with the addition of automatic differentation.\n",
    "Everything you develop here will be used in project 2. \n",
    "\n",
    "In order to get started, we will now replace in our standard ordinary\n",
    "least squares (OLS) and Ridge regression codes (from project 1) the\n",
    "matrix inversion algorithm with our own gradient descent (GD) and SGD\n",
    "codes.  You can use the Franke function or the terrain data from\n",
    "project 1. **However, we recommend using a simpler function like**\n",
    "$f(x)=a_0+a_1x+a_2x^2$ or higher-order one-dimensional polynomials.\n",
    "You can obviously test your final codes against for example the Franke\n",
    "function. Automatic differentiation will be discussed next week.\n",
    "\n",
    "You should include in your analysis of the GD and SGD codes the following elements\n",
    "1. A plain gradient descent with a fixed learning rate (you will need to tune it) using the analytical expression of the gradients\n",
    "\n",
    "2. Add momentum to the plain GD code and compare convergence with a fixed learning rate (you may need to tune the learning rate), again using the analytical expression of the gradients.\n",
    "\n",
    "3. Repeat these steps for stochastic gradient descent with mini batches and a given number of epochs. Use a tunable learning rate as discussed in the lectures from week 39. Discuss the results as functions of the various parameters (size of batches, number of epochs etc)\n",
    "\n",
    "4. Implement the Adagrad method in order to tune the learning rate. Do this with and without momentum for plain gradient descent and SGD.\n",
    "\n",
    "5. Add RMSprop and Adam to your library of methods for tuning the learning rate.\n",
    "\n",
    "The lecture notes from weeks 39 and 40 contain more information and code examples. Feel free to use these examples.\n",
    "\n",
    "In summary, you should \n",
    "perform an analysis of the results for OLS and Ridge regression as\n",
    "function of the chosen learning rates, the number of mini-batches and\n",
    "epochs as well as algorithm for scaling the learning rate. You can\n",
    "also compare your own results with those that can be obtained using\n",
    "for example **Scikit-Learn**'s various SGD options.  Discuss your\n",
    "results. For Ridge regression you need now to study the results as functions of  the hyper-parameter $\\lambda$ and \n",
    "the learning rate $\\eta$.  Discuss your results.\n",
    "\n",
    "You will need your SGD code for the setup of the Neural Network and\n",
    "Logistic Regression codes. You will find the Python [Seaborn\n",
    "package](https://seaborn.pydata.org/generated/seaborn.heatmap.html)\n",
    "useful when plotting the results as function of the learning rate\n",
    "$\\eta$ and the hyper-parameter $\\lambda$ when you use Ridge\n",
    "regression.\n",
    "\n",
    "We recommend reading chapter 8 on optimization from the textbook of [Goodfellow, Bengio and Courville](https://www.deeplearningbook.org/). This chapter contains many useful insights and discussions on the optimization part of machine learning."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9940e21b",
   "metadata": {
    "editable": true
   },
   "source": [
    "# Code examples from week 39 and 40"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6559d6d2",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Code with a Number of Minibatches which varies, analytical gradient\n",
    "\n",
    "In the code here we vary the number of mini-batches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "021b66d2",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.69887085]\n",
      " [3.34977681]]\n",
      "Eigenvalues of Hessian Matrix:[0.25923926 4.67193435]\n",
      "theta from own gd\n",
      "[[3.69887085]\n",
      " [3.34977681]]\n",
      "theta from own sdg\n",
      "[[3.63685221]\n",
      " [3.37369014]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHFCAYAAADxOP3DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABR6ElEQVR4nO3deXwTZeIG8Gfa0haQFlpo6QUtu8gph1wCFqgKioCFSquwIrqyXihWXAVkXcAL4ectXuwq7KpUKZRjvRFaDgHlRgU5tEA5KncDhZY2eX9/hISmSZprkpnJPN/Pp5+ayWTmnaRxHt5TEkIIEBEREelUiNIFICIiIlISwxARERHpGsMQERER6RrDEBEREekawxARERHpGsMQERER6RrDEBEREekawxARERHpGsMQERER6RrDEJEKzZ8/H5IkWX/CwsKQkJCAO++8E/v27VOsXNOnT4ckSYqdXwsOHDgASZLw8ssvK10UInJTmNIFICLn5s2bh7Zt26KiogLff/89XnjhBRQWFuLXX39FkyZNlC4eEVFQYBgiUrGOHTuie/fuAIABAwbAaDRi2rRpWLp0Ke69916FS0dKqqqqstYaEpFv2ExGpCGWYPTHH39Yt1VUVOCJJ55Aly5dEB0djZiYGPTu3RvLli2ze70kSXjkkUfw0UcfoV27dmjQoAE6d+6Mzz//3G7fL774Al26dEFERATS0tKcNvtUVFRgypQpSEtLQ3h4OJKSkjB+/HicPXvWZr/U1FQMHToUn3/+Obp27Yr69eujXbt21nPPnz8f7dq1Q8OGDdGzZ09s3rzZ5fthaU4sLCzEQw89hKZNmyI2NhZZWVk4evSo3bVPnz7d7hipqam455577I65atUq/O1vf0NsbCyioqJw9913o7y8HKWlpcjJyUHjxo2RkJCAv//976iqqrI7rslkwgsvvIAWLVogMjIS3bt3x8qVK+3227dvH0aPHo24uDhERESgXbt2ePvtt232KSoqgiRJ+Oijj/DEE08gKSkJERER2L9/Py5cuIC///3vSEtLQ2RkJGJiYtC9e3fk5eW5fP+IyIz/pCDSkOLiYgDA1Vdfbd1WWVmJ06dP4+9//zuSkpJw6dIlfPfdd8jKysK8efNw99132xzjiy++wKZNm/Dss8/iqquuwuzZszFixAjs2bMHrVq1AgCsXLkSmZmZ6N27Nz799FMYjUbMnj3bJoQBgBACw4cPx8qVKzFlyhSkp6dj586dmDZtGjZs2IANGzYgIiLCuv+OHTswZcoUTJ06FdHR0ZgxYwaysrIwZcoUrFy5Ei+++CIkScKkSZMwdOhQFBcXo379+i7fl3HjxmHIkCFYsGABSkpK8OSTT+Kuu+7CqlWrvH6vx40bh6ysLHz66afYtm0bnn76aVRXV2PPnj3IysrC/fffj++++w6zZs1CYmIiJk6caPP6OXPmoGXLlnj99ddhMpkwe/ZsDB48GKtXr0bv3r0BALt27UKfPn3QokULvPLKK2jevDm++eYbTJgwASdPnsS0adNsjjllyhT07t0b7733HkJCQhAXF4eJEyfio48+wvPPP4+uXbuivLwcP//8M06dOuX1tRPpjiAi1Zk3b54AIDZu3CiqqqrEuXPnxNdffy2aN28u+vXrJ6qqqpy+trq6WlRVVYn77rtPdO3a1eY5ACI+Pl4YDAbrttLSUhESEiJmzpxp3darVy+RmJgoLl68aN1mMBhETEyMqPm/ja+//loAELNnz7Y5z2effSYAiLlz51q3tWzZUtSvX18cPnzYum379u0CgEhISBDl5eXW7UuXLhUAxPLly916nx5++GGb7bNnzxYAxLFjx2yufdq0aXbHaNmypRg7dqzdMR999FGb/YYPHy4AiFdffdVme5cuXcS1115rfVxcXCwAOH3/brrpJuu2m2++WSQnJ4uysjKbYz7yyCMiMjJSnD59WgghRGFhoQAg+vXrZ1f+jh07iuHDh9ttJyL3sZmMSMWuu+461KtXD40aNcItt9yCJk2aYNmyZXb9RPLz89G3b19cddVVCAsLQ7169fDBBx9g9+7ddsfMyMhAo0aNrI/j4+MRFxeHgwcPAgDKy8uxadMmZGVlITIy0rpfo0aNMGzYMJtjWWpeajYzAUB2djYaNmxo1yzUpUsXJCUlWR+3a9cOgLk/VIMGDey2W8rkym233WbzuFOnTh693pGhQ4faPLaUaciQIXbbHZ3H2fu3Zs0aGI1GVFRUYOXKlRgxYgQaNGiA6upq68+tt96KiooKbNy40eaYt99+u915evbsia+++gqTJ09GUVERLl686PU1E+kVwxCRiv33v//Fpk2bsGrVKjzwwAPYvXs3Ro0aZbNPQUEBcnJykJSUhI8//hgbNmzApk2b8Ne//hUVFRV2x4yNjbXbFhERYb2JnjlzBiaTCc2bN7fbr/a2U6dOISwsDM2aNbPZLkkSmjdvbtdUExMTY/M4PDy8zu2Oyu9I7WuyNM35Egw8Kaujcjp7/y5duoTz58/j1KlTqK6uxltvvYV69erZ/Nx6660AgJMnT9q8PiEhwe6Yb775JiZNmoSlS5ciIyMDMTExGD58uKJTMBBpDfsMEalYu3btrJ2mMzIyYDQa8e9//xuLFi3CyJEjAQAff/wx0tLS8Nlnn9nMAVRZWenVOZs0aQJJklBaWmr3XO1tsbGxqK6uxokTJ2wCkRACpaWl6NGjh1dl8IeIiAiH74m/+tY4e//Cw8Nx1VVXoV69eggNDcWYMWMwfvx4h8dIS0uzeexojqeGDRtixowZmDFjBv744w9rLdGwYcPw66+/ynMxREGONUNEGjJ79mw0adIE//znP2EymQCYb5Dh4eE2N8rS0lKHo8ncYRnNVVBQYFPjce7cOfzvf/+z2ffGG28EYA5kNS1evBjl5eXW59UgNTUVO3futNm2atUqnD9/3i/nc/b+paenIzQ0FA0aNEBGRga2bduGTp06oXv37nY/jmrx6hIfH4977rkHo0aNwp49e3DhwgW5L4soKLFmiEhDmjRpgilTpuCpp57CggULcNddd2Ho0KEoKCjAww8/jJEjR6KkpATPPfccEhISvG4qee6553DLLbdg4MCBeOKJJ2A0GjFr1iw0bNgQp0+ftu43cOBA3HzzzZg0aRIMBgP69u1rHU3WtWtXjBkzRq5L99mYMWPwzDPP4J///Cf69++PXbt2Yc6cOYiOjvbL+UJDQzFw4EBMnDgRJpMJs2bNgsFgwIwZM6z7vPHGG7j++uuRnp6Ohx56CKmpqTh37hz279+P//3vf26NhuvVqxeGDh2KTp06oUmTJti9ezc++ugj9O7d26YfFhE5xzBEpDGPPvoo5syZg2effRajRo3Cvffei+PHj+O9997Dhx9+iFatWmHy5Mk4fPiwzY3XEwMHDsTSpUvxj3/8A3fccQeaN2+Ohx9+GBcvXrQ5piRJWLp0KaZPn4558+bhhRdeQNOmTTFmzBi8+OKLNsPqlfbkk0/CYDBg/vz5ePnll9GzZ08sXLgQmZmZfjnfI488goqKCkyYMAHHjx9Hhw4d8MUXX6Bv377Wfdq3b4+tW7fiueeewz/+8Q8cP34cjRs3RuvWra39hly54YYbsHz5crz22mu4cOECkpKScPfdd2Pq1Kl+uS6iYCQJIYTShSAiIiJSCvsMERERka4xDBEREZGuMQwRERGRrjEMERERka4xDBEREZGuMQwRERGRruliniGTyYSjR4+iUaNGDqezJyIiIvURQuDcuXNITExESIj/6m90EYaOHj2KlJQUpYtBREREXigpKUFycrLfjq+LMNSoUSMA5jczKipK4dIQERGROwwGA1JSUqz3cX/RRRiyNI1FRUUxDBEREWmMv7u4sAM1ERER6RrDEBEREekawxARERHpGsMQERER6RrDEBEREekawxARERHpGsMQERER6RrDEBEREekawxARERHpGsMQERER6RrDEBEREekawxARERHpGsMQERER6RrDEBEREekawxARERHpGsMQERER6RrDEBEREema4mFozZo1GDZsGBITEyFJEpYuXep03wceeACSJOH1118PWPmIiIgouCkehsrLy9G5c2fMmTOnzv2WLl2KH374AYmJiQEqGREREelBmNIFGDx4MAYPHlznPkeOHMEjjzyCb775BkOGDAlQyYiIiEgPFK8ZcsVkMmHMmDF48skn0aFDB6WLQ0REREFG8ZohV2bNmoWwsDBMmDDB7ddUVlaisrLS+thgMPijaERERBQEVF0ztGXLFrzxxhuYP38+JEly+3UzZ85EdHS09SclJcWPpSQiIiItU3UYWrt2LY4fP44WLVogLCwMYWFhOHjwIJ544gmkpqY6fd2UKVNQVlZm/SkpKQlcoYmIiEhTVN1MNmbMGNx00002226++WaMGTMG9957r9PXRUREICIiwt/FIyIioiCgeBg6f/489u/fb31cXFyM7du3IyYmBi1atEBsbKzN/vXq1UPz5s3Rpk2bQBeViIiIgpDiYWjz5s3IyMiwPp44cSIAYOzYsZg/f75CpSIiIiK9UDwMDRgwAEIIt/c/cOCA/wpDREREuqPqDtRERERE/sYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6pngYWrNmDYYNG4bExERIkoSlS5dan6uqqsKkSZNwzTXXoGHDhkhMTMTdd9+No0ePKldgIiIiCiqKh6Hy8nJ07twZc+bMsXvuwoUL2Lp1K5555hls3boVBQUF2Lt3L2677TYFSkpERETBSBJCCKULYSFJEpYsWYLhw4c73WfTpk3o2bMnDh48iBYtWrh1XIPBgOjoaJSVlSEqKkqm0hIREZE/Ber+Hea3I/tJWVkZJElC48aNne5TWVmJyspK62ODwRCAkhEREZEWKd5M5omKigpMnjwZo0ePrjMhzpw5E9HR0daflJSUAJaSiIiItEQzYaiqqgp33nknTCYT3nnnnTr3nTJlCsrKyqw/JSUlASolERERaY0mmsmqqqqQk5OD4uJirFq1ymW7YUREBCIiIgJUOiIiItIy1YchSxDat28fCgsLERsbq3SRiIiIgpvRCKxdCxw7BiQkAOnpQGio0qXyG8XD0Pnz57F//37r4+LiYmzfvh0xMTFITEzEyJEjsXXrVnz++ecwGo0oLS0FAMTExCA8PFypYhMRESnHn2GloAB47DHg8OEr25KTgTfeALKy5DmHyig+tL6oqAgZGRl228eOHYvp06cjLS3N4esKCwsxYMAAt87BofVERBQ0/BlWCgqAkSOB2tFAksy/Fy0KaCAK1P1b8TAUCAxDREQUFPwZVoxGIDXVNmTVPkdyMlBcHLAms0DdvzUzmoyIiEjXjEZzjZCjOgzLttxc837eWLvWeRCynKOkxLxfkGEYIiIi0gJ/h5Vjx+TdT0MYhoiIiLTA32ElIUHe/TSEYYiIiEgL/B1W0tPNfYIs/Y9qkyQgJcW8X5BhGCIiItICf4eV0FDziDTLsWofGwBefz0o5xtiGCIiItKCQISVrCzziLSkJNvtyckBH1YfSBxaT0REpCWO5hlKSTEHIbnCikpmoOY8QzJiGCIioqASqLCicCgK1P1b8eU4iIiIyEOhoYCbqzB4TUfLcrDPEBEREdmyzHRde16jI0fM2wsKlCmXnzAMERER0RX+nulahRiGiIiI6AodLsvBMERERERX6HBZDoYhIiIiukKHy3IwDBEREdEVOlyWg2GIiIiIrtDhshwMQ0RERGRLZ8tycNJFIiIispeVBWRmqmJZDn9jGCIiIiLHAjHTtQqwmYyIiIh0jTVDRETkmEpWLqcA0+HnzjBERET2dLRIJ9Wg08+dzWRERGRLZ4t00mU6/twlIRytxBZcDAYDoqOjUVZWhqioKKWLQ0SkXkYjkJrqfG0qSTLXFBQXB33Tia6o9HMP1P2bNUNERHSFDhfpJOj+c2cYIiKiK3S4SCdB9587wxAREV2hw0U6Cbr/3DmajIiIrrAs0nnkiLlppDZL35EgWqTTJ8EyDN3Xz13j7wNrhoiI6AodLtLptYICc6fjjAxg9Gjz79TUwI26MhqBoiIgL8/822j0/li+fO5Kvw8yYBgiIiJbOluk0yu+DEOXI8T4I4B487kHyXB8Dq0nIiLHNN704Te+DEOXY1JDSwCpffu21OD4Gljd/dwDMBw/UPdvhiEiIiJPFBWZa2JcKSy0XeRUjhCjpvmAvH0f3FS68zg+nrYZTy4dwnmGiIiIVMXd4eVHjlz5b6PRXCPkqP7Bsi0313WTmZrmA/LDcPxj2//AnOzV6N94OxI7N8WTS6/3snCe4WgyIiIiT7g7vPzxx4H69c21PZ6EmLpqUQI9H1BdTWYyDcc/tv0PLH7hV+SviMbask4QiLc+163BLmy54G3h3ccwRERE5AlXw9AtTp40N4stWgRUVrp3bFchJlDzARmNwAsvmPsynT59ZXvN/k0+DMc/urUUi1/cg/wVjbHOcI1NALruqp+QfcMpjJzcGo07tEd0tG+X4g42kxEREXmi5jD0utRs/oqLc+/YrkKMJYDUHv5uIUlASopv80AVFADx8cC0abZBCLAdJebhcPyjW0vx1sjV6Be9A8nd4jBhcX+sNXSGQAiuu+onvJpZhIPrj2DDuWswcdkAtOhda1SbHzEMERERecoyDL1p07r3szR/AfKEGH/PA2Xp5H3qlOPna/dvcjEc/2hqH7w1cjXSHQSg3pcD0KGNR7Hh3DV4fGlgA1BNHE1GRETkrU8+Ae66y/V+CxYAERHmoAHYNit5MyTe0RD9lBRzEPJ2WL2rkWq11RwlVqNv0QlDOD79pgkWrozF94ZrIGrUu/S+6ifk3HQat09ujZReiS5PoZtV69esWYNhw4YhMTERkiRh6dKlNs8LITB9+nQkJiaifv36GDBgAH755RdlCktERFRT7RoRZxIS5J3MMisLOHDAHEgWLDD/Li72bX4hV528a6vRv+nItuN48y0J6Q+2R/yDIzBhyQ1Yd7kGqE+jnXht+Goc2ngU689dg9wl/d0KQoGkeAfq8vJydO7cGffeey9uv/12u+dnz56NV199FfPnz8fVV1+N559/HgMHDsSePXvQqFEjBUpMRER0maediLOygMxMeSazDA31av4epzwcgXbCEI4FWauR/10TfH+uE4Ar/Z36NNqJ7BvPXK4B6iRfGf1EVc1kkiRhyZIlGD58OABzrVBiYiJyc3MxadIkAEBlZSXi4+Mxa9YsPPDAA24dl81kRETkN5Z+NoDvzV9KcnMSRQHguBSPRHEEJlwJcX0a7UTOTWdw+5SrkdxDntXtddNMVpfi4mKUlpZi0KBB1m0RERHo378/1q9f7/R1lZWVMBgMNj9ERER+ESxrubkaqQZzEBIAHhLvwIRQ9G20E6+PWI2SH4/he0MnPFbQX7YgFEiKN5PVpbS0FAAQHx9vsz0+Ph4HDx50+rqZM2dixowZfi0bERHJTMtrocnZ/KUUy0i1kSMhIEGCfcPRScTitcinMeDWWLw15RiSuqu/Ccwdqg5DFlKtlCqEsNtW05QpUzBx4kTrY4PBgJSUFL+Vj4iIfCTHAqZKk7sPT4Ad3nQMiz6KxfGIl/FQxWtIwZXPogzR2NUuCy3mPYsXeyUrWEr/UHUYat68OQBzDVFCjYmojh8/bldbVFNERAQiIiL8Xj4iIpKBswVMLRP8aampSWNKfjiKRTP3In9lLDacvwbmTtD9MRuP4qEG/8GQDofQ7b7OaDZuOHprqZbLQ6oOQ2lpaWjevDlWrFiBrl27AgAuXbqE1atXY9asWQqXjoiIfOZqAVNJMk/wl5mprSYnFbMPQOZh7hJM6Bv1E3IGnsXtT7dB4rXjlC1oACkehs6fP4/9+/dbHxcXF2P79u2IiYlBixYtkJubixdffBGtW7dG69at8eKLL6JBgwYYPXq0gqUmIiJZyLWAKdXJEoAWrozFxloB6Pqon5BtDUCdlS2oQhQPQ5s3b0ZGjaF8lr4+Y8eOxfz58/HUU0/h4sWLePjhh3HmzBn06tUL3377LecYIiIKBoFehV1HDm04gkUv7UP+KscBKGfQWWRN0W8AqklV8wz5C+cZIiJSKTfntrFZ+oGcsg9AZhJMSI/eieyBZZcDUHMFS+m+QN2/Fa8ZIiIiHfN0Bmeyc/D7w1j00n7kFzbFD+UdAZjnO6oZgG6f2hYJXbooWk41YxgiIiLl1JjbBpLkeAZnX1Zhd4cG5zeyD0Dm4e6WAJQzqAxZTzMAuYthiIiIlGWZwdnRPEO+rMLuDg3Nb2QJQAsLm+LHWgGoX/ROZHsSgDQYAP2JfYaIiEgdAn2Ddja/kYrWFDuw7nINUJElAJlZAlDOLWXIerodmneKc/+gGgqAgbp/MwwREZEy6go//g5GRiOQmup8WL+lr1JxccBrTCwBaGFRM2wq73ClSDChf+MdyL7Z4HkAstBAAKyJYUhGDENERCpTV+0E4P+aC5WNYjuw7jDyZ+5H/mrbABQCI/o13ulbALJQcQB0hqPJiIi0iH0xXKtr+Y3bb3f8GrmX5lDB/Eb2AcjcB8gSgHJuOYesqe0Q37GrPCfkBJdOMQwREclFQ30xFONq+Q1n5F6ao8Z6l7Ls56biNSVYNOs3LCyKw+YL7VEzAPVvvBPZcgegmlQQANWKYYiISA5cbNQ9rmon6iJnzUUA5zcqXlOC/Jd+Q/5qSwBKAXAlAOUMPocRT/spANWkUADUAoYhIiJfcbFR98lR6yDHMfw8v5ElAC1cHYcttQLQgCY7kH3L+cAEoJo4waVTDENERL5iXwz3yVHrIFfNhczzG/1edAj5s35H/hrnAShrajvEdbhWnvJ7Sg0TXKoUwxARka/YF8N9rmon6uKPmousLHONnZed3u0DUAsA5gCU0WQHsgeba4AUC0C1KTnBpYoxDBER+Yp9MdznqnbC8jiQNRehoR7V2FkC0MLV8dh6sR1UH4Bq8zEABiPOM0RE5CvL/C2u+mKoaP4WxTkaeZeSYg47gPPnFKq5+G3VQeTPLkb+GksAMrMEoJxbz2PE1PZo1q6pIuULVpx0UUYMQ0Tkd5bRZIDjGg2OJrOn5AzUbrAEoIVr4rGtVgC6IWY7sgeXMwD5GcOQjBiGiCgg6qrtcCcIBTIAqCBsqNH+lZdrgNbaBqBQVCMjZgcDUIAxDMmIYYiIAsbbkBHICRudneu114CmTdUdkPwQ4uoKQDfE7ED2kHIMn8IApASGIRkxDBGRqgVy8Uxn53JEbbNnyxgYLQFo4drm2H6xrXV7zQA0YmoHNG0TK1fpg5O/ahgvH9fw22+IHjeOYUgODENE5Fe+3BACuXimq3M5Ojegjv5OMgTGfSsOIP//DiB/neMAlDP0AoY/3Z4ByF3+qs2scVwDgGiAYUgODENE5De+3hACuXq6u+eqSQ0j4XwIjJYAtHBtAnZUtLFuD0U1bozdjuwhFxmAvOGv2sxaxw1UGOI8Q0SkbUp2BJZjPbJATtjozTHUMHu2hzN87/2mGPkvH0T+OksASgXgKAB1tz8WO5a75q/lZ+o6rp8xDBGRdim5SrxcN4RATtjoyzGUnD3bzXMveXQlZuy3BKA0AOYAdFPsdmQPvYjhUzsgtrWDAGSh5N+Tlvhr+ZmiIu8X8fVRiCJnJSLylaVWpvb/PC21MgUF/j2/JzeEuliWp7A0L9QmSebh+XIsQeHqXHVRcvZsN8/9xs83YkdFG4ShCjfHbsa/x67FH3sN+PqPrrjvHiNiN39jvuEajfYvVvrvSUv8UZtZUADk5HhXHhkwDBGR9riqlQHMtTKObnpykeuGYFmeArAPKXIvQVHXuZyRM4x5y0WIM0FCCZLRIKY+PrhnLUr3nsPXJ7vjvvnpiP2pyNzfKCMDGD3a/Ds11TbcqOHvSUvkrs20BNHTp70vk48YhohIe+SqlfGFnDcEy+KZSUm225OT5R/J5excjii9krnRCBQV4ejTb6Go3k0wCQETbAORCYAEgcZvv4AvT/XCX+elI7Z1jPlJd2t71PD3pCVy1mYq2E+oJvYZIiLtUcMq8a5WX/d0hfVALp7p6FwnTgATJ6pmJfPDU99Fo1emIbryBBIBJAI4iVgAAk1xpQYh5PIM341ql9GTPl1q+HvSEleL7QLuB2hXQTRAGIaISHvUsEq8nDeEmscM1IgtR+fKylJ0JNWvX/6O/FcO4fTan/FK1QQAtkEmFqfM7+30GUDr1nWX0ZPaHjX8PWmNpYbRUYdzTwK0SgIm5xkiIu3x9yrxngyv9nU9smDkwftnCUALv0/Cz5WtEQIjDiAVSTjsuB+Hu59tXp65j5ArCxaYO+768+8pmPk6FYGLua84zxARkTP+qJWx8HR4dSCbt3wViDl03Hj/dn/+G/JfLUH+enMAAloBAMJQhYnRHyClTIZh257U9vjz7ynY+Vqb6aq5OVCEDpSVlQkAoqysTOmiEJGcFi8WIjlZCPP/Rs0/KSnm7d4eT5JsjweYt0mS+8etrhaisFCIBQvMv6urvSuPnBy9V8nJ3r9Xzs7h5P0zQRKfdZwhOkTstXmqHirFrc1+FPPuWytO/37G/J7Vfr2jnwUL6i5LdbX5+hyVx/KZpqTYfjZy/z2Reyx/N7U/K0kSZea2Ur/fv9lMRkTaJldth1xrhKlx4r5ALATr4v0zQcJhJCMNxQiFEQOb7UD2bZXInNoRTdIaX9lRzuVJLNcNOK7tcXTddf09cXZq/3HS3Gx48UVEjxnj//u3X6OWSrBmiIhcKix0r0aisND5MeSqWZKTpYbE2fU4qiHxhpvv31e3vmmuAXJVXk9qdOoiV21PIGrW9M5BjWqg7t/sM0REBPg+vNpf6zX5yl9LJ9Swa/l+7J68Ere7se8tdzUFatYE1eZt/x1ntTZy9OmSYw06ci2Qoylr4aSLRESA78Or1Tpxn5/m0Pll2X5MH1CEDpH70SHzz3hr943uvdAfk1AWFNQ9y7TlJjtqlPm3XIuHWuqIHnwQuHTJ/WOS6rBmiIgI8H0SRbVO3CfjHDq/LNuP/NcOY+GGFOy+9GcAfwYA1MMlRDeLQPmFZmhw4SQkb96/2jU7mZnu1ej4u9bGnUkBT5wwB7f332cNkUYxDBERAVeaZ2530NjjzvBqtU7c527I69PH3Hm5VvD4Zdl+LHz1MPI32gagcFRiUNwOZGdewm1Tr0Hjlr2Bgve8G57ubafzQDRNuhteT55kk5mW+bVHkkqwAzURuWXxYiFiY+077cbGuu4oK3fHXznVMXRZSJIQTz5p1zn4bHgz8WjY2za7h6NCDIvfKP77wFpx9uBZ5+fypMOyL53O5ej07oq751D6M1YDP0wpEaj7N8MQEZEQzm/KlpucO6OGXIUOJUceOQspTz4pTJIkTLWu2QhJGCGJbHzqOgDV5u5N0deRbnLNSeTqWuoKuXKHL63y02g7hqHLqqqqxNSpU0VqaqqIjIwUaWlpYsaMGcJoNLp9DIYhIqqTnMPP1TxxX42QYlq5SuzM+0mcDW8mjE6u2wQIY0KS/2o6fK3ZCUTNkBB1B2W5w5cW+XFKiUDdv1U/mmzWrFl47733MGfOHOzevRuzZ8/G//3f/+Gtt95SumhEFCzkHAmWlQUcOGCeFHDBAvPv4mJV9CMRUgh+OpWIf76bgHaDW+LRUScRfemE02HFEoCQY0f8NwLO107nlv5Qlj5JtUmSeZ04Z5223WUZ3dasmXv762lBV1f9tgBzvy2jMaDF8pTqO1Bv2LABmZmZGDJkCAAgNTUVeXl52Lx5s8IlI6KgIfdIMAXnS6lNmAR+XrIPC18/ivwfW2DPpasBXA0A6IF17h1k5Ur/zLbsa6fzQK4plpUFDB1qHjV28qTjfVyNmAtGAZjHKhBUXzN0/fXXY+XKldi7dy8AYMeOHVi3bh1uvfVWhUtGREFDrSPBvCRMAjsX7cUz6UVoV78YnUZejefXDcCeS60QgQpkNv8BHz/0Pd5bGOveAZ9/3nbeHrnIUbPj6ZxEvggPNw+flyT7Mut1QVe1TinhIdXXDE2aNAllZWVo27YtQkNDYTQa8cILL2DUqFFOX1NZWYnKykrrY4PBEIiiElGgyL1GlK9zDKmAMAn8VGCpAWqJvVVXaoAiUIFbmu9A9ohqDHv6GkQl9zK/yGh0f8Vwf8y2LFfNjhyzTLvLEr4cTQXw+uuqaA4NqGD5h4RfeyTJIC8vTyQnJ4u8vDyxc+dO8d///lfExMSI+fPnO33NtGnTBAC7H3agJlIZb4bi+muNKDWPBHPCZDSJ7Z/9Kqb2LRRX1/vdptgRuCgym28Unzy8TpSV1PH/PmfX7WtHck+oudO5M34YRq5Jfp5SgqPJLktOThZz5syx2fbcc8+JNm3aOH1NRUWFKCsrs/6UlJQwDBGpjTehRo5RK3XdxDRwU64ZgFo7CEDDEza4DkC1ObruQA8dZ7jQLj/+Q4ILtV524cIFhITYdm0KDQ2FyWRy+pqIiAhERET4u2hE5C1vllCQY7ZhVzMd+6u5xcdmPUsfoIVvHEP+ppbYV9UGQBsA5iawwQnbkT3CiKFTrkFU8nWel8ey9MX06eb+Qa74o/+Hijqdy0bu5ly1CoamQ79GLRmMHTtWJCUlic8//1wUFxeLgoIC0bRpU/HUU0+5fQzOM0SkApZ/+X/8sRDNmnneFOPrnDJ+nAulTl4265mMJrHt01/F030c1wCNSNggFjzyvTAcMchXnkDN26MH/mrOVTPOQO0/BoNBPPbYY6JFixYiMjJStGrVSkydOlVUVla6fQyGISKFedoM4+iG68tsw3JOqujpdXsQwCwBaErvQvHnesU2L4nEBe8DkLvlWbhQvUuKaIlSwTsIBer+LQnhqM45uBgMBkRHR6OsrAxRUVFKF4dIX5w1ibmyYAFQc9RoURGQkeH6dYWF9s0tvrzWW0ajeTi6szlYLo9QE7/9ju2L9iP/rVLkb07F/qpU6y6RuIjBCTuQM9KEIZOvQaPERn4vD159FcjJMW+r+ZlZRndxIdK6ufs+FxcHZ5OZzAJ1/1Z9nyEi0rC6+vm4UnsorifD32v31ThyxL1zytkXxs3J6O6qvwifGrORjlJ0xwakYQ+iE67C7SOlywHIjT5AMpYHTZtqv/+HkoJkEkK9YRgiIv9xdWNwxNmcPu7OSbNsmf2NXIllFNwMVt2MP2AW/o5k1AhsocnAgDeAxD62O/vSIdeTyfFGjQrcvD3BJkgmIdQbj2agLikp8Vc5iCgYefo/fFcT7bmabRgwh6XaAczZ8gk1zyvHGlY1uRmsHsfrSEKtmivLqLqaMz4XFJibXzIygNGjzb89mRXa08nxLKO7Ro0y/2YQck+wTEKoMx6FobZt2+KZZ55BeXm5v8pDRMHE0//hu7OEgrOFUDMzXS8Y6YwQwLhxnpXV2aFMAls/2Y0pk4w4hgSY4HipCSFJQGgoJMB+D0t5LQtcWvpd1Q55jkKTM4Fa1FTvtPg+G43mfnV5eebfKl9U1S886W39/fffi549e4qEhATx4Ycf+qdLtx9wNBmRQlzNTguYh9l//LHvQ3HdHRbetKnz57wc+mwymsTmj3aJSb0KRauwA9bDjcBiYYQkjI5GFbk7qu677+QbCafBWbY1SUvvs8qnAFD10Pr//Oc/Ijk5WXTp0kUUamC+CYYhIgUF6sbg7tD7jz8WYsYM58HCzTI5C0CAEPVRLkYmrRef5X4vLvz7Y8ezWufmulfef/zDvf3c/X+xBmbZDgpaeJ81MAWAqsOQEEJcuHBBPPPMM6JBgwZi+PDhYt++fXKWS1YMQ0QKC8SNwd2aIR9qWiwB6KmedQeg83+ct32ho8no3C2vu2HI0fxKznDpi8BQ8/us1NxbHlL9PEMXLlzA1q1bsXjxYrz55puoV68exo8fj+nTp6NRIx/mwvADzjNEpAL+XprAMr+Lq6H38+YBN93k+niX5xwSJoEtH+9G/tvHkb+1FYqrW1h3qY8LGJq8A9nZwK2TO6FhXEPFy0sO6GVZDE8oMfeWF1Q5z9B7772HTZs2YdOmTdi9ezdCQ0PRqVMnjB8/Hl26dMEnn3yC9u3bY8mSJejevbu/ykxEWuTvtafcHXp//Lhbhyt96UO8+hSwaFsrFFe3B9AeANAA5RiSvLNGAOrt3/IOGOD+/Epkz9V6dHrFKQBseVKNlJycLEaOHClefvllsW7dOlFRUWG3zwsvvCA6dOggU8WVPNhMRqQjrprk3GyeKkUzEYJqAQjRAOdFdvJ6kT9xvX0TmL/La9lHKx1y1UQDfWIUo5F16FTfTObMH3/8gcTERBhVNDSPzWREOlNXs4jRaN524oTLw8xo9hY6jOmGwZM8bAKTs7wWjmo4UlKCb1ZouZq0uCxG3dxtplX4/QnU/Vv2MCSEwJo1a9C/f385D+sThiEishAmgT8Gj0Xzbz9yvXPt9dGU5iooaL1vjJxNWhrpE6Moy/xVgONmWhWsQxeo+7dHky66Q5IkVQUhIiJhEvhx3i94skcRWkUcxp3f/tW9F6ptluC6ZoX2dYZqpckxsWRN7BPjmqsZ3YOpxtEFrlpPREFJmAQ2/WcXFr59Aou2/xkHjcnW5xqhDMWhrRFjPOF4fmiVNBG4zRIkav/vXEX/wq+TP5q0WDPkPhXXKGq2mUyNGIaI9MFSA5T/7km7ANQQ5zE0ZSdy7gzBLU91QoM1X6u+icAtwdA3xh/BRSN9YqhuqhxaT6RJKv5Xj0fUfB0Klq1mAMrf/mccMna0PtcQ5zGsxU5k33E5ADWtsQq8pYnAUR8VLXVKXrvWeRACzEGgpMS8n1prQPzRpOXu1AVq+Q6RohiGKLgFyxwjar4OBcpmCUALL9cAOQtAgyd3Rv2YPs4PlJVlXuBVrSHTHcHQN8ZfK70HS+Alv2MzGQUvrfejsFDzdQSwbMIk8MOHvyD/PUsAsm0Cu63lDmTfEYpbJnVG/Zj6spxTE4Khb4y/m7TUXKtKdWKfIRkxDOlQMPSjANR9HZ6WzYsbkiUALXz3JBbtaI0S45VRL1fhHIa13KnPAFRTsPSN0cAwbwo89hki8kUw9KMA1H0dnpTt9Gm3m9JM1SZzDdD7py4HoCtNYJYAlDMqDDc/2Qn1Y/rKfVXaEyx9Y9ikRQpiGKLgFAz9KAB1X4e751y2zHyzrl1rYZk/ZtEimG4bbg1A+TuuxmHjNdbdrsI53NZyJ7IZgJwLliARDH24SJMYhig4+atDZqCp4TqcNW+5e85PPnHcfCMEBCScvfMhdDH2wCGTlwGI/UHMagaJI0fMy400awbExJjfI628J/5e0JfIAYYhCk7p6cqu9C3XDVrp66hrpFhmpuuyNW1a5xpgEgSaVB1HGn7DGUTjttSfzAHoqc6IbOxGDZCaR9kpITTU3CQ5eTLfEyJP+HUZWJXgqvU6pdRK345WIU9O9v58Sl6HqxW/XZUtN9etlbE33fasuHjmovzl0xu+JxRkAnX/ln1tMiLVUGLdHbnXVwKUuQ6j0Vzj4qR5CwCQm2uuHXJQNpGUjN3jXsac/7V063TdH09HZONI+ctnNLp/TK3je0LkNQ6tp+AXqD4l/h4GH8i+MZ7OXWM0wlS0Gvs+2YQVayIw+/eRKBHJCIERB5CKJBxBCGQc9h0Mc+vIje8JBSEOrSeSS6A6ZPp7GHwgO5a6OVLMdPgINry7EwvfP4PFP7XDEdMN1ueiUIbb0n5CadfxSF7yNACZhn0bjcDKle7tq/bRgnJS88hDIpVjGCKSSzDdjNwcKXbH3ZFYJDpZH0ehDJmtfkL26HAMerIzIqKuB3A9UHC1PMO+HXWYluE6goIaRh4SaRSbyYjkEkzNFC5mNTZBwmEkIw3FuArnawWgCOfH9KWZz9nSH45oZdZlOQXLTNRENbCZjMhf/NX3Rulh8HIKDYXp1dcg5eQAAKQaT5kuP1oa/wCW3b8FA/9uqQFyfUyvQ2BdnYNr09Ksy3IKlpmoiRTA0WSkLwUF5n89Z2QAo0ebf6emejfKqzbLzQi4cvOx0MjNyFRtwto5O/BYp1XIviMEr+ExnEBTm30qo+Jg/CgPE0qnYuizPZ3XBDliNJpr0PLyzL/dHdnkqj9WTf4cZad2Sow8JAoCbCYj/QjUCuuO+rWkpKh2WQRTtQnfv/cTFs49i8W/tMF1pvV4A48hBVfKX9mgMULvGYOw7Czva9J8mSAxL88cXl35xz+A6dNVHTgDgrNyU5DgqvUyYhiigK/+rvKbkfGSEd+//zPy/2UOQMdMzQEAI1CARRgJCcKmacznwOhrEA2m/lhE5DaGIRkxDBFvplcC0MJ/ncXiX9qi1BRvfS4aZRjRajveOZmDSMNxSI4O4G1glCOIuuocDJiPceCAqkInEfkmUPdv9hkifVDLsHdv+8x4e7pLRqx+Yzse6bQayfVPov+Eznj7p/4oNcUjGmUY+6d1+GL6Jhw/Vx/zPhCo7ywIAbbzJHnCk/mXnKmrP5bFxYvAsmWelY2ICBxNRnqhhjlYArSoqPGSEeve/Qn5H5RdrgHqYn2usXQWI1ptx/3X/4ru/RsiLC0FSO9tDhv+CoxyHdfSOfj++4FTp+yfP33a3BTHjsJE5CGGIQpeNfvtxMWZR9gcParc6u+O+sxY1izz8QZuCUAL/12Ggl32AWj4n35CzphI3PSnYtSb/ATwn8PAfy7vYAlk/gqMch43MxOYMMHxc0KYP0fLmmlsLiMiN7HPEAUnR7UwsbHmGgVnc7D4c9FTP3TeNl4yYu07l2uAdrXDH6Y463ONpbMY8eefkH1XJG6c2BnhV4W77sT82WfAxInyT9on52SA7PtFpCucdJHIW85u+qdPm3/HxNg2s3izLIQnZFyzzBKAFv7bgILdbfFHjRqgJtIZDP/zzzUCUI1aLlcrmksS8MQTwGuvATk58k7aJ+dkgGrp+0VEQUUTHaiPHDmCu+66C7GxsWjQoAG6dOmCLVu2KF0sUiN3bvr16wPffQcsWGCuQSgu9m8fEx9v4MZLRhS+ug0Pd1yDpPqnkPF4F7z7Sz/8YYpDE+kM7m29Fl89txmlhob4cG86Bv+zh7kmqCZ3A1nTpv6ZtE+uyQDV0PeLiIKO6muGzpw5g759+yIjIwNfffUV4uLi8Ntvv6Fx48ZKF43UyJ2b/uHD5lqIUaMCUyYvbuDGS0asmbMT+R+ew+Jd7XBcdLU+10Q6gxF//hnZd9fHDbmdbGuAnPEkkI0aZe5zI/c8SVlZvh83mJY8ISLVUH0YmjVrFlJSUjBv3jzrttTUVOUKROqmxmYUN2/gxl59sObVbVj4wTkU7HYcgHLG1scNj3dGvQYe3uw9DWS+rCNWF1+Py/W3iMgPVN9Mtnz5cnTv3h3Z2dmIi4tD165d8a9//avO11RWVsJgMNj8kE6osRmljjlyhCRBCOB9cT8SG5bhhie64r1d/XBcNEOMdBr3Xb0WXz+/GX+cvwof7E3HzVO7o16Dep6XwRLInM3RI0nmJUO0UKPC9beISGaqH00WGRkJAJg4cSKys7Px448/Ijc3F++//z7uvvtuh6+ZPn06ZsyYYbedo8l0QM6RS3JzMMLtMBIxAW9hCcw38BjpNEa0/sXcBPZ4Z++CT13nHznS/N+BHE3nLypf8oSIfMflOC4LDw9H9+7dsX79euu2CRMmYNOmTdiwYYPD11RWVqKystL62GAwICUlhWFIL1R406+uqMbqt3Zi0YdlOPZrGerjIo4hAWuRjsZSGUa0/gU59zRAxmOd5A1AtWlsEVki0jcOrb8sISEB7du3t9nWrl07LF682OlrIiIiEBER4e+ikVpZmlEczfYcwJu+JQDlzzuPgl/b4YS41vpcrHQKI67+BU+P3X45AAWoeUqOTsxEREFG9WGob9++2LNnj822vXv3omXLlgqViDRBoZu+JQAt/PA8luxxHIBy7mmIARM6oV6Dfn4ti1P+6hxNRKRRqg9Djz/+OPr06YMXX3wROTk5+PHHHzF37lzMnTtX6aKR2gXopl9dUY2iNy/XAO1pj5O1AlBWm1+QPVbhAERERE6pvs8QAHz++eeYMmUK9u3bh7S0NEycOBF/+9vf3H49l+Mgp7zshGsJQAvnnceSPe1xUjS1PmcfgPzYByhYsXM0EYEdqGXFMEQOebiKfHVFNQpf34H8/5Q7DUA5916FARM6ISxS9ZWu6uXh50JEwYthSEYMQ2TH1aKll0ec1QxABXs64JSIte7aVDqJrDa7kM0AJB83Pxci0geGIRkxDJENF6vIC0lCRVQcHovPQ8G+Tg4DUM5fr0L/hzsgbNMGNuXIxcXnougcUUSkCA6tJ/IXF+uXSUKgftkf2Fsm4RRi0VQ6idvbmmuA+j/aCWGR/cw1GG0z2ZQjJ3cXk127lqPhiEhW+gpDa9cCBgP/Fa93bq5LdlfiKjyTG3UlAFk4a8o5csS8nU053lHjunJEpAv6CkNDh175b/4rXpeqLlRh23cG9HRj33Gf3AAMuNZ2o9Fo7tzrqHVZCHNTTm6ueY4jhm3PqHFdOSLSBX31GQJgbXFkh0z18mVYtYPXVlWasOq1Hcj/70Us2dcBZ0U0DiAVSTiCEHi4fllREZCR4bochYXB1ZQTiKHual5XjogUwT5D/sZ/xddNqXlefBlW7eC1p8Oa4fHq/8N/Mda6LU46gS+THsD9h/8JIUmQHK1f9vrrjq9Xj005gRrqHhpqPubIkebPwZPPhYjIF0IHysrKBABRZv7fq/1PYaHSRVSXxYuFSE62fY+Sk83b/X1eSbL/fCTJ/FPX+RcvFiZJEqZarzVCEkZI4h58KB7qsFqsemWrqK6sdn6dKSl1n6ew0PHfULD+TfnymfhyTk8/FyIKStb7d1mZX8+j32aymhYsAEaNCnCpVEqpeV68HFZddaEKK1/eip7PDUPj6hMIcfBSAQlIToJ04IB9rYKnNWB6asrx9DORszaRM1ATETjPkKxchqFg69/hLSXnefGgL86l7n2w8tUdyP+4Akv3d0QnsQNFCGA/HktgBBw35QRLPzRP+kedPs1Zo4lIdoEKQ47+Ia0fkgSkpJj/1UmezfMiNzf72Lw/qhDNo8px67QemLcvHWdEE7SVfvX9HEaj+eafl2f+bTQ63zcryxx4kpJstycnB08QAtzv97RsmTkc1v7bsUw1UFAgf9mIiGSk3w7USnfIVGMzgJKdg90cLp1XmoEzaIL4kOO4vf1u5IyLxvXtWwODfDiHNx2Es7LMHe/V9hnKyd0h7J98wqkGiEjT9BuGkpPNQUiJf8WrdSFKJed5SU8HkpMhjhyxHd11mQkSjiEB13Q0Yca47bj+oWsQGt7f/KTRaH7/XPXjcVQD6MsEiqGhwd28evkzqfN9bdoUOHHC+TE4azQRaYC+msk+/9zcWbqw0NzvRakgpNYmBcvNz1JrVpufmhUvnb+EL5/fireqHoQQ5uBTkwAgAWj+2Rt466cb0P+xLggNr1HLYBmSbSlj7TIDjmsAXU2gCJhrNepqMvOkeU1r3Hlf//IX944VTFMNEFHw8etYNZUI1NA8l6qr7YcM1x6unJJi3k8plqHUtYdTyzyUuvJcpfh82o9i7J/WisbSGetpRmCxOIIE74ZVezok29dh8kpNQRBodb2veptqgIgCikPrZaSaVeu1Mnuxo2a8lBSfmxUvnb+EFS/vQP7HlVj62zUoQ7T1uYSQUtzeYQ+y/9YYfce1Q+gP62Wbgdrpa/PygNGjXR/T0dQLSk1BoBRn76uephogooDjDNTBSCuzF8vYObjSUGkOQJ9cwrLfr0EZelifswSgnPsbo8/9HREa3vzKC70Ng5704/G2j5Qe1ydz9r5y1mgiCgIMQ4GkpYUofegcbB+AriyLmhBSipEdL9cAPXgNQsKa13EkP3Ong7CjjteeTEGgh07DlqkGHA0KUGqQAhGRBxiGAsnbm68GWALQwk8uYbmaA1BN3tZqaKWGL5D0MNUAEQUthqFACrImhUpDJb79vx3IX2CuATLUCECJIcdwe8e9yHmgCfrc31E9Aag2b2o1tFTDF0jBPtUAEQUtdqBWgp86KAeCfQC60gk6MeQYRl6zF9n3WwKQhmZu8KTjNTsNExEFBNcmk5HqwhCgzhmonag4W2EOQHlVWF7sOADlPNAEvf+msQDkC72sT0ZEpCCGIRmpMgypXM0AtKy4E87VWOI2MeQYsjuZa4B0FYBq03ANHxGRFjAMyYhhyD2WALRwQTWWH7jGJgAlhRzDSAYgexqq4SMi0hrOM0QBUXG2At/M3oH8PEsA6mV9zhKAch6MwXX3dUBImM46BFvUFXjYaZiISPMYhnSorgCUHHoUIzvtQ/YDOg9AFmpdVJeIiGTDMBQM3GiqqThbga9nmQPQ/w46DkA5D8ag1187ICQsMdBXoE6+rGhPRESawT5DWldHzUXFDbdaA9Dyg51wHo2u7BJ6FNmd9yL7gdjLAYh9gGxYhs87m2maw+eJiPyOfYbINSc1F+LwYeD223EfPsYC/MW6PSX0CEZ23lcjALEGyCkuuUFEpBsMQ2rh6aikOhYLlQCYIGEmpuD7kHRkdfkdOQ81Rc972iMkLMl/1xBMuOQGEZFuMAypgReddCu/+A4RddRchECgBUrw+7f7EXLjDXKXOPhxyQ0iIt1gRxGlWZq6agcbSyfdggLrpounL6LgqY0Y1XI9Hsw84tbhQ47/IWdp5WM0AkVFQF6e+bfRqHSJbFkW1bXMKF2bJJknWNTgorpERGSLNUNKqqOpC0IAkgTThMewdH088vMl/O9QJ5TjOgBAf1xy7xxqrLlQy3B1V/MHBdGiukRE5BxrhpTkRifdkCOH8eYrVfj0UB+U4yq0CD2MJ7oVYdbcJhBarLnwoCbM7+VITQUyMoDRo82/U1Ntz29Z0T6pVj+r5GQOqyciCiIcWq+kvDzzjdiFCdJbCL+2I3LGN0OPse0hhVwOQFpbLFQtw9WdzR/k7H3jkhtERIrg2mQyUmsYqlj2NSKHD3a5n1i5CtINGY6f1NJioUVF5hoYVwoL/TdcXS2BjIiIXOI8Q0HqwskL+HLmDuTnC3xV0hu/IBlJOIIQOMmksbGQ+vdzfsCsLCAzUxs1F2oYrs75g4iIqBaGoQCoGYA+L+mMC+htfe75kH/iPdP9EDDPD2Tn1Clg2bK6a3m0slioGoarqyGQERGRqmiuA/XMmTMhSRJyc3MDd1IvhoFfOHkBi57YgJyUDWjWTCD71d5YWNIHF9AQqWEleLJHEX6cvwvvVdwLKTbWcRACzM02ubnqG3ruDTUMV1dDICMiIlXRVM3Qpk2bMHfuXHTq1ClwJ/VgGHj58XJ8+dJO5OcDXxzuZFMDlBpWguyuvyHnkTh0u6sdpJAU8xNFRebaH2eCqdlGDcPVLYHsyBHHUxpY+gypbRQeERH5jWZqhs6fP4+//OUv+Ne//oUmTZoE5qTOhoFfXvsLzz6L8mMG5E801wDFxQM5r/VG/uHeuICGSAs7hKd6FmHTf3bh98pkzP5xALrfXWM0GKDdZhtvJ01Ueri6JZAB9jVUnD+IiEiXNBOGxo8fjyFDhuCmm25yuW9lZSUMBoPNj0eMRmDlSuBvf3Nce2AxbRrOJLZD3mvH7ALQ5o9247fKFMz6wUEAqkmLzTbuzNFTl6ws4MAB86ixBQvMv4uLAzf6TelARkREqqKJZrJPP/0UW7duxaZNm9zaf+bMmZgxY4Z3J3PULFaHRBzFIozEJ62no/0/s3Ht6LaQQlq4fz6tNds4m6PHMmmiu2FC6U7fWhqFR0REfqX6eYZKSkrQvXt3fPvtt+jcuTMAYMCAAejSpQtef/11h6+prKxEZWWl9bHBYEBKSorreQqc3ehdEJIEyZe5abQyeSLn6CEiogAK1DxDqm8m27JlC44fP45u3bohLCwMYWFhWL16Nd58802EhYXB6KCvSkREBKKiomx+XLq8Tpg32VCq2cnZG1pptvFkjh4iIiKNUH0z2Y033oiffvrJZtu9996Ltm3bYtKkSQiVoQbifOl5/Dh+Pm44fNj5EHd3+NLJWQvNNlrt7E1ERFQH1YehRo0aoWPHjjbbGjZsiNjYWLvtnjhfeh6fv7gT+YtD8NXRTshELG7wtbC+dnJWuh+NK1rs7E1EROSC6sOQnM7/cR5f/uNn5C8OwZdHO6MCfazPhYRIgMnLA6utk7O/aK2zNxERkRs0GYaKioq8el2rq0NRWSMA/bneAWR3P4DsR5ujy8iRQKsnnd/ondHT3DRqmDSRiIhIZqrvQC2nStTHn+sdwJTeRdj26R7srWiJF9cPQNdRbSHVC6t7Mj5JAp580lzzUZPaOjn7m1Y6exMREblJ9UPr5WAZmrf2w83oO/Za5xMgAo7nGUpJMdd4ZGWZR52puZNzoPB9ICIiPwvU0HpdhSG330ze6ImIiBQXqDCkyT5Dfqf2UV011RXcGOqIiIhcYhjSIkvIWbYM+Phj4OTJK88lJ1/p+1S7uc/yHPv1EBERWbGZTGtcrZ1We5RX7ecAdnQmIiJN4HIcZM+yhpmrJTFcPZeba65dIiIiIoYhzbi8dpqni8ja4fphRERENhiGtMLVIqme4vphREREABiGtEPu8ML1w4iIiABwNJl2yBVeuH4YERGRDdYMaYVlkdTaS4XUxdGyIgDXDyMiIqqBYUgrLIukAnUHopQUYPFi8w/XDyMiInKJzWRaYlkktfY8Q82aAX/5C5CZaTvLdGYmZ6AmIiJygZMuahGX2SAiIh3g2mTknJbWTiMiIlI59hkiIiIiXWMYIiIiIl1jGCIiIiJdYxgiIiIiXWMHan/iqC8iIiLVYxjyl4IC+/mAkpPNEydy0kMiIiLVYDOZPxQUACNH2q8yf+SIeXtBgTLlIiIiIjsMQ3IzGs01Qo7msrRsy80170dERESKYxiS29q19jVCNQkBlJSY9yMiIiLFMQzJ7dgxefcjIiIiv2IYkltCgrz7ERERkV8xDMktPd08akySHD8vSUBKink/IiIiUhzDkNxCQ83D5wH7QGR5/PrrnG+IiIhIJRiG/CErC1i0CEhKst2enGzeznmGiIiIVIOTLvpLVhaQmckZqImIiFSOYcifQkOBAQOULgURERHVgc1kREREpGsMQ0RERKRrDENERESkawxDREREpGsMQ0RERKRrDENERESka6oPQzNnzkSPHj3QqFEjxMXFYfjw4dizZ4/SxSIiIqIgofowtHr1aowfPx4bN27EihUrUF1djUGDBqG8vFzpohEREVEQkIQQQulCeOLEiROIi4vD6tWr0a9fP7deYzAYEB0djbKyMkRFRfm5hERERCSHQN2/NTcDdVlZGQAgJibG6T6VlZWorKy0PjYYDH4vFxEREWmT6pvJahJCYOLEibj++uvRsWNHp/vNnDkT0dHR1p+UlJQAlpKIiIi0RFPNZOPHj8cXX3yBdevWITk52el+jmqGUlJS2ExGRESkIWwmq+XRRx/F8uXLsWbNmjqDEABEREQgIiIiQCUjIiIiLVN9GBJC4NFHH8WSJUtQVFSEtLQ0pYtEREREQUT1YWj8+PFYsGABli1bhkaNGqG0tBQAEB0djfr16ytcOiIiItI61fcZkiTJ4fZ58+bhnnvucesYHFpPRESkPewzdJnKsxoRERFpnKaG1hMRERHJjWGIiIiIdI1hiIiIiHSNYYiIiIh0jWGIiIiIdI1hiIiIiHSNYYiIiIh0jWGIiIiIdI1hiIiIiHSNYYiIiIh0jWGIiIiIdI1hiIiIiHSNYYiIiIh0jWGIiIiIdI1hiIiIiHSNYYiIiIh0jWGIiIiIdI1hiIiIiHSNYYiIiIh0jWGIiIiIdI1hiIiIiHSNYYiIiIh0jWGIiIiIdI1hiIiIiHSNYYiIiIh0jWGIiIiIdI1hiIiIiHSNYYiIiIh0jWGIiIiIdI1hiIiIiHSNYYiIiIh0jWGIiIiIdI1hiIiIiHSNYYiIiIh0jWGIiIiIdI1hiIiIiHSNYYiIiIh0jWGIiIiIdE0zYeidd95BWloaIiMj0a1bN6xdu1bpIhEREVEQ0EQY+uyzz5Cbm4upU6di27ZtSE9Px+DBg3Ho0CGli0ZEREQaJwkhhNKFcKVXr1649tpr8e6771q3tWvXDsOHD8fMmTNdvt5gMCA6OhplZWWIioryZ1GJiIhIJoG6f6u+ZujSpUvYsmULBg0aZLN90KBBWL9+vUKlIiIiomARpnQBXDl58iSMRiPi4+NttsfHx6O0tNThayorK1FZWWl9XFZWBsCcMImIiEgbLPdtfzdiqT4MWUiSZPNYCGG3zWLmzJmYMWOG3faUlBS/lI2IiIj859SpU4iOjvbb8VUfhpo2bYrQ0FC7WqDjx4/b1RZZTJkyBRMnTrQ+Pnv2LFq2bIlDhw759c1UG4PBgJSUFJSUlOiqrxSvm9etB7xuXrcelJWVoUWLFoiJifHreVQfhsLDw9GtWzesWLECI0aMsG5fsWIFMjMzHb4mIiICERERdtujo6N19UdkERUVxevWEV63vvC69UWv1x0S4t8uzqoPQwAwceJEjBkzBt27d0fv3r0xd+5cHDp0CA8++KDSRSMiIiKN00QYuuOOO3Dq1Ck8++yzOHbsGDp27Igvv/wSLVu2VLpoREREpHGaCEMA8PDDD+Phhx/26rURERGYNm2aw6azYMbr5nXrAa+b160HvG7/XrcmJl0kIiIi8hfVT7pIRERE5E8MQ0RERKRrDENERESkawxDREREpGuaDEPvvPMO0tLSEBkZiW7dumHt2rV17r969Wp069YNkZGRaNWqFd577z27fRYvXoz27dsjIiIC7du3x5IlS/xVfK95ct0FBQUYOHAgmjVrhqioKPTu3RvffPONzT7z58+HJEl2PxUVFf6+FI94ct1FRUUOr+nXX3+12S/YPu977rnH4XV36NDBuo8WPu81a9Zg2LBhSExMhCRJWLp0qcvXBMP329PrDpbvt6fXHSzfb0+vO1i+3zNnzkSPHj3QqFEjxMXFYfjw4dizZ4/L1wXiO665MPTZZ58hNzcXU6dOxbZt25Ceno7Bgwfj0KFDDvcvLi7GrbfeivT0dGzbtg1PP/00JkyYgMWLF1v32bBhA+644w6MGTMGO3bswJgxY5CTk4MffvghUJflkqfXvWbNGgwcOBBffvkltmzZgoyMDAwbNgzbtm2z2S8qKgrHjh2z+YmMjAzEJbnF0+u22LNnj801tW7d2vpcMH7eb7zxhs31lpSUICYmBtnZ2Tb7qf3zLi8vR+fOnTFnzhy39g+W77en1x0s329Pr9tC699vT687WL7fq1evxvjx47Fx40asWLEC1dXVGDRoEMrLy52+JmDfcaExPXv2FA8++KDNtrZt24rJkyc73P+pp54Sbdu2tdn2wAMPiOuuu876OCcnR9xyyy02+9x8883izjvvlKnUvvP0uh1p3769mDFjhvXxvHnzRHR0tFxF9AtPr7uwsFAAEGfOnHF6TD183kuWLBGSJIkDBw5Yt2nh864JgFiyZEmd+wTL97smd67bES1+v2ty57qD5ftdkzefdzB8v4UQ4vjx4wKAWL16tdN9AvUd11TN0KVLl7BlyxYMGjTIZvugQYOwfv16h6/ZsGGD3f4333wzNm/ejKqqqjr3cXbMQPPmumszmUw4d+6c3WJ358+fR8uWLZGcnIyhQ4fa/ctSSb5cd9euXZGQkIAbb7wRhYWFNs/p4fP+4IMPcNNNN9nN0q7mz9sbwfD9loMWv9++0PL3Ww7B8v0uKysDgDoXYQ3Ud1xTYejkyZMwGo12q9XHx8fbrWpvUVpa6nD/6upqnDx5ss59nB0z0Ly57tpeeeUVlJeXIycnx7qtbdu2mD9/PpYvX468vDxERkaib9++2Ldvn6zl95Y3152QkIC5c+di8eLFKCgoQJs2bXDjjTdizZo11n2C/fM+duwYvvrqK4wbN85mu9o/b28Ew/dbDlr8fnsjGL7fvgqW77cQAhMnTsT111+Pjh07Ot0vUN9xzSzHUZMkSTaPhRB221ztX3u7p8dUgrdlzMvLw/Tp07Fs2TLExcVZt1933XW47rrrrI/79u2La6+9Fm+99RbefPNN+QruI0+uu02bNmjTpo31ce/evVFSUoKXX34Z/fr18+qYSvG2jPPnz0fjxo0xfPhwm+1a+bw9FSzfb29p/fvtiWD6fnsrWL7fjzzyCHbu3Il169a53DcQ33FN1Qw1bdoUoaGhdmnv+PHjdqnQonnz5g73DwsLQ2xsbJ37ODtmoHlz3RafffYZ7rvvPixcuBA33XRTnfuGhISgR48eqvmXhC/XXdN1111nc03B/HkLIfDhhx9izJgxCA8Pr3NftX3e3giG77cvtPz9lovWvt++CJbv96OPPorly5ejsLAQycnJde4bqO+4psJQeHg4unXrhhUrVthsX7FiBfr06ePwNb1797bb/9tvv0X37t1Rr169OvdxdsxA8+a6AfO/GO+55x4sWLAAQ4YMcXkeIQS2b9+OhIQEn8ssB2+vu7Zt27bZXFOwft6AebTG/v37cd9997k8j9o+b28Ew/fbW1r/fstFa99vX2j9+y2EwCOPPIKCggKsWrUKaWlpLl8TsO+4212tVeLTTz8V9erVEx988IHYtWuXyM3NFQ0bNrT2qp88ebIYM2aMdf/ff/9dNGjQQDz++ONi165d4oMPPhD16tUTixYtsu7z/fffi9DQUPHSSy+J3bt3i5deekmEhYWJjRs3Bvz6nPH0uhcsWCDCwsLE22+/LY4dO2b9OXv2rHWf6dOni6+//lr89ttvYtu2beLee+8VYWFh4ocffgj49Tnj6XW/9tprYsmSJWLv3r3i559/FpMnTxYAxOLFi637BOPnbXHXXXeJXr16OTymFj7vc+fOiW3btolt27YJAOLVV18V27ZtEwcPHhRCBO/329PrDpbvt6fXHSzfb0+v20Lr3++HHnpIREdHi6KiIpu/2wsXLlj3Ueo7rrkwJIQQb7/9tmjZsqUIDw8X1157rc2wvLFjx4r+/fvb7F9UVCS6du0qwsPDRWpqqnj33Xftjpmfny/atGkj6tWrJ9q2bWvz5VILT667f//+AoDdz9ixY6375ObmihYtWojw8HDRrFkzMWjQILF+/foAXpF7PLnuWbNmiT/96U8iMjJSNGnSRFx//fXiiy++sDtmsH3eQghx9uxZUb9+fTF37lyHx9PC520ZOu3s7zZYv9+eXnewfL89ve5g+X5783ceDN9vR9cMQMybN8+6j1LfcelyAYmIiIh0SVN9hoiIiIjkxjBEREREusYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6xjBEREREusYwRERERLrGMERERES6xjBERJqUl5eHyMhIHDlyxLpt3Lhx6NSpE8rKyhQsGRFpDdcmIyJNEkKgS5cuSE9Px5w5czBjxgz8+9//xsaNG5GUlKR08YhIQ8KULgARkTckScILL7yAkSNHIjExEW+88QbWrl3LIEREHmPNEBFp2rXXXotffvkF3377Lfr37690cYhIg9hniIg065tvvsGvv/4Ko9GI+Ph4pYtDRBrFmiEi0qStW7diwIABePvtt/Hpp5+iQYMGyM/PV7pYRKRB7DNERJpz4MABDBkyBJMnT8aYMWPQvn179OjRA1u2bEG3bt2ULh4RaQxrhohIU06fPo2+ffuiX79+eP/9963bMzMzUVlZia+//lrB0hGRFjEMERERka6xAzURERHpGsMQERER6RrDEBEREekawxARERHpGsMQERER6RrDEBEREekawxARERHpGsMQERER6RrDEBEREekawxARERHpGsMQERER6RrDEBEREena/wMescmozDSvawAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_5_1.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "# Importing various packages\n",
    "from math import exp, sqrt\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.inv(X.T @ X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = 2.0/n*X.T @ ((X @ theta)-y)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "# Can you figure out a better way of setting up the contributions to each batch?\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (2.0/M)* xi.T @ ((xi @ theta)-yi)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        theta = theta - eta*gradients\n",
    "print(\"theta from own sdg\")\n",
    "print(theta)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8e7110b",
   "metadata": {
    "editable": true
   },
   "source": [
    "In the above code, we have use replacement in setting up the\n",
    "mini-batches. The discussion\n",
    "[here](https://sebastianraschka.com/faq/docs/sgd-methods.html) may be\n",
    "useful."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74a7859a",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Momentum based GD\n",
    "\n",
    "The stochastic gradient descent (SGD) is almost always used with a\n",
    "*momentum* or inertia term that serves as a memory of the direction we\n",
    "are moving in parameter space.  This is typically implemented as\n",
    "follows"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "619df8e8",
   "metadata": {
    "editable": true
   },
   "source": [
    "$$\n",
    "\\mathbf{v}_{t}=\\gamma \\mathbf{v}_{t-1}+\\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t) \\nonumber\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c759cfc4",
   "metadata": {
    "editable": true
   },
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"_auto1\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation} \n",
    "\\boldsymbol{\\theta}_{t+1}= \\boldsymbol{\\theta}_t -\\mathbf{v}_{t},\n",
    "\\label{_auto1} \\tag{1}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a4571f9",
   "metadata": {
    "editable": true
   },
   "source": [
    "where we have introduced a momentum parameter $\\gamma$, with\n",
    "$0\\le\\gamma\\le 1$, and for brevity we dropped the explicit notation to\n",
    "indicate the gradient is to be taken over a different mini-batch at\n",
    "each step. We call this algorithm gradient descent with momentum\n",
    "(GDM). From these equations, it is clear that $\\mathbf{v}_t$ is a\n",
    "running average of recently encountered gradients and\n",
    "$(1-\\gamma)^{-1}$ sets the characteristic time scale for the memory\n",
    "used in the averaging procedure. Consistent with this, when\n",
    "$\\gamma=0$, this just reduces down to ordinary SGD as discussed\n",
    "earlier. An equivalent way of writing the updates is"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bebfdd9",
   "metadata": {
    "editable": true
   },
   "source": [
    "$$\n",
    "\\Delta \\boldsymbol{\\theta}_{t+1} = \\gamma \\Delta \\boldsymbol{\\theta}_t -\\ \\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t),\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9e635e8",
   "metadata": {
    "editable": true
   },
   "source": [
    "where we have defined $\\Delta \\boldsymbol{\\theta}_{t}= \\boldsymbol{\\theta}_t-\\boldsymbol{\\theta}_{t-1}$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb544f8b",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Algorithms and codes for Adagrad, RMSprop and Adam\n",
    "\n",
    "The algorithms we have implemented are well described in the text by [Goodfellow, Bengio and Courville, chapter 8](https://www.deeplearningbook.org/contents/optimization.html).\n",
    "\n",
    "The codes which implement these algorithms are discussed after our presentation of automatic differentiation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d088ad85",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Practical tips\n",
    "\n",
    "* **Randomize the data when making mini-batches**. It is always important to randomly shuffle the data when forming mini-batches. Otherwise, the gradient descent method can fit spurious correlations resulting from the order in which data is presented.\n",
    "\n",
    "* **Transform your inputs**. Learning becomes difficult when our landscape has a mixture of steep and flat directions. One simple trick for minimizing these situations is to standardize the data by subtracting the mean and normalizing the variance of input variables. Whenever possible, also decorrelate the inputs. To understand why this is helpful, consider the case of linear regression. It is easy to show that for the squared error cost function, the Hessian of the cost function is just the correlation matrix between the inputs. Thus, by standardizing the inputs, we are ensuring that the landscape looks homogeneous in all directions in parameter space. Since most deep networks can be viewed as linear transformations followed by a non-linearity at each layer, we expect this intuition to hold beyond the linear case.\n",
    "\n",
    "* **Monitor the out-of-sample performance.** Always monitor the performance of your model on a validation set (a small portion of the training data that is held out of the training process to serve as a proxy for the test set. If the validation error starts increasing, then the model is beginning to overfit. Terminate the learning process. This *early stopping* significantly improves performance in many settings.\n",
    "\n",
    "* **Adaptive optimization methods don't always have good generalization.** Recent studies have shown that adaptive methods such as ADAM, RMSPorp, and AdaGrad tend to have poor generalization compared to SGD or SGD with momentum, particularly in the high-dimensional limit (i.e. the number of parameters exceeds the number of data points). Although it is not clear at this stage why these methods perform so well in training deep neural networks, simpler procedures like properly-tuned SGD may work as well or better in these applications.\n",
    "\n",
    "Geron's text, see chapter 11, has several interesting discussions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "937063d3",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Using Automatic differentation with OLS\n",
    "\n",
    "We conclude the part on optmization by showing how we can make codes\n",
    "for linear regression and logistic regression using **autograd**. The\n",
    "first example shows results with ordinary leats squares."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e08215a6",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.64342603]\n",
      " [3.30485583]]\n",
      "Eigenvalues of Hessian Matrix:[0.29030069 4.6608358 ]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own gd\n",
      "[[3.64342603]\n",
      " [3.30485583]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHFCAYAAADxOP3DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABXZElEQVR4nO3deXxTVfo/8M/tlrIWCpSuQJGtgGVHtrIoggoIlICAovjVGRVEsMpA3QAdRFARRnT44TjgqBQkLYvihtICCih7EZHNAgVadlJobaHJ+f0REhqaNHtyk/t5v14ZJnc9N8n1Pj3nOedIQggBIiIiIoUK8nUBiIiIiHyJwRAREREpGoMhIiIiUjQGQ0RERKRoDIaIiIhI0RgMERERkaIxGCIiIiJFYzBEREREisZgiIiIiBSNwRCRDC1btgySJJleISEhiImJwejRo3HkyBGflWvmzJmQJMln5/cHx48fhyRJeOedd3xdFCKyU4ivC0BE1i1duhStWrVCaWkpfv75Z8yePRvZ2dn4448/ULduXV8Xj4goIDAYIpKxtm3bonPnzgCAvn37QqfTYcaMGVizZg0ef/xxH5eOfOnGjRumWkMicg2byYj8iDEwOnv2rGlZaWkpXnjhBbRv3x4RERGIjIxE9+7dsXbt2kr7S5KEZ599Fp9++imSkpJQvXp1tGvXDl999VWlbdevX4/27dtDpVIhMTHRarNPaWkp0tPTkZiYiLCwMMTFxWHixIm4cuWK2XZNmjTB4MGD8dVXX6FDhw6oVq0akpKSTOdetmwZkpKSUKNGDXTt2hU7d+60+XkYmxOzs7PxzDPPoH79+qhXrx5SU1Nx5syZStc+c+bMSsdo0qQJxo8fX+mYGzduxN/+9jfUq1cPtWvXxqOPPori4mIUFhZi1KhRqFOnDmJiYvDiiy/ixo0blY6r1+sxe/ZsNGrUCOHh4ejcuTN+/PHHStsdOXIEY8eORVRUFFQqFZKSkvDBBx+YbZOTkwNJkvDpp5/ihRdeQFxcHFQqFY4ePYqSkhK8+OKLSExMRHh4OCIjI9G5c2dkZGTY/PyIyIB/UhD5kby8PABAixYtTMvKyspw6dIlvPjii4iLi8P169fxww8/IDU1FUuXLsWjjz5qdoz169djx44deP3111GzZk3MmzcPw4cPx6FDh9C0aVMAwI8//oihQ4eie/fuWLFiBXQ6HebNm2cWhAGAEALDhg3Djz/+iPT0dKSkpCA3NxczZszAtm3bsG3bNqhUKtP2+/btQ3p6Ol5++WVERERg1qxZSE1NRXp6On788Ue8+eabkCQJ06ZNw+DBg5GXl4dq1arZ/FyefPJJDBo0CMuXL0d+fj6mTp2KRx55BBs3bnT6s37yySeRmpqKFStWYM+ePXjppZdQXl6OQ4cOITU1FX//+9/xww8/YO7cuYiNjUVaWprZ/osWLULjxo2xYMEC6PV6zJs3D/fffz82bdqE7t27AwB+//139OjRA40aNcK7776L6OhofPfdd3juuedw4cIFzJgxw+yY6enp6N69OxYvXoygoCBERUUhLS0Nn376Kf75z3+iQ4cOKC4uxm+//YaLFy86fe1EiiOISHaWLl0qAIjt27eLGzduiKtXr4pvv/1WREdHi969e4sbN25Y3be8vFzcuHFDPPHEE6JDhw5m6wCIhg0biqKiItOywsJCERQUJObMmWNadtddd4nY2Fjx119/mZYVFRWJyMhIUfE/G99++60AIObNm2d2npUrVwoAYsmSJaZljRs3FtWqVROnTp0yLdu7d68AIGJiYkRxcbFp+Zo1awQAsW7dOrs+pwkTJpgtnzdvngAgCgoKzK59xowZlY7RuHFj8dhjj1U65qRJk8y2GzZsmAAg5s+fb7a8ffv2omPHjqb3eXl5AoDVz69///6mZQMHDhTx8fFCq9WaHfPZZ58V4eHh4tKlS0IIIbKzswUA0bt370rlb9u2rRg2bFil5URkPzaTEclYt27dEBoailq1auG+++5D3bp1sXbt2kp5IqtWrULPnj1Rs2ZNhISEIDQ0FB9//DEOHjxY6Zj9+vVDrVq1TO8bNmyIqKgonDhxAgBQXFyMHTt2IDU1FeHh4abtatWqhSFDhpgdy1jzUrGZCQBGjhyJGjVqVGoWat++PeLi4kzvk5KSABjyoapXr15pubFMtjz44INm75OTkx3a35LBgwebvTeWadCgQZWWWzqPtc9v8+bN0Ol0KC0txY8//ojhw4ejevXqKC8vN70eeOABlJaWYvv27WbHHDFiRKXzdO3aFd988w2mT5+OnJwc/PXXX05fM5FSMRgikrH//e9/2LFjBzZu3IinnnoKBw8exJgxY8y2ycrKwqhRoxAXF4fPPvsM27Ztw44dO/B///d/KC0trXTMevXqVVqmUqlMD9HLly9Dr9cjOjq60na3L7t48SJCQkLQoEEDs+WSJCE6OrpSU01kZKTZ+7CwsCqXWyq/Jbdfk7FpzpXAwJGyWiqntc/v+vXruHbtGi5evIjy8nK8//77CA0NNXs98MADAIALFy6Y7R8TE1PpmP/6178wbdo0rFmzBv369UNkZCSGDRvm0yEYiPwNc4aIZCwpKcmUNN2vXz/odDr85z//gUajgVqtBgB89tlnSExMxMqVK83GACorK3PqnHXr1oUkSSgsLKy07vZl9erVQ3l5Oc6fP28WEAkhUFhYiC5dujhVBk9QqVQWPxNP5dZY+/zCwsJQs2ZNhIaGIjg4GOPGjcPEiRMtHiMxMdHsvaUxnmrUqIFZs2Zh1qxZOHv2rKmWaMiQIfjjjz/cczFEAY41Q0R+ZN68eahbty5ee+016PV6AIYHZFhYmNmDsrCw0GJvMnsYe3NlZWWZ1XhcvXoVX375pdm299xzDwBDQFZRZmYmiouLTevloEmTJsjNzTVbtnHjRly7ds0j57P2+aWkpCA4OBjVq1dHv379sGfPHiQnJ6Nz586VXpZq8arSsGFDjB8/HmPGjMGhQ4dQUlLi7ssiCkisGSLyI3Xr1kV6ejr+8Y9/YPny5XjkkUcwePBgZGVlYcKECVCr1cjPz8cbb7yBmJgYp5tK3njjDdx3332499578cILL0Cn02Hu3LmoUaMGLl26ZNru3nvvxcCBAzFt2jQUFRWhZ8+ept5kHTp0wLhx49x16S4bN24cXn31Vbz22mvo06cPfv/9dyxatAgREREeOV9wcDDuvfdepKWlQa/XY+7cuSgqKsKsWbNM2yxcuBC9evVCSkoKnnnmGTRp0gRXr17F0aNH8eWXX9rVG+6uu+7C4MGDkZycjLp16+LgwYP49NNP0b17d7M8LCKyjsEQkZ+ZNGkSFi1ahNdffx1jxozB448/jnPnzmHx4sX473//i6ZNm2L69Ok4deqU2YPXEffeey/WrFmDV155BQ899BCio6MxYcIE/PXXX2bHlCQJa9aswcyZM7F06VLMnj0b9evXx7hx4/Dmm2+adav3talTp6KoqAjLli3DO++8g65du+KLL77A0KFDPXK+Z599FqWlpXjuuedw7tw5tGnTBuvXr0fPnj1N27Ru3Rq7d+/GG2+8gVdeeQXnzp1DnTp10Lx5c1PekC1333031q1bh/feew8lJSWIi4vDo48+ipdfftkj10UUiCQhhPB1IYiIiIh8hTlDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFE0R4wzp9XqcOXMGtWrVsjicPREREcmPEAJXr15FbGwsgoI8V3+jiGDozJkzSEhI8HUxiIiIyAn5+fmIj4/32PEVEQzVqlULgOHDrF27to9LQ0RERPYoKipCQkKC6TnuKYoIhoxNY7Vr12YwRERE5Gc8neLCBGoiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNJ8HQ5s3b8aQIUMQGxsLSZKwZs0aq9s+9dRTkCQJCxYs8Fr5iIiIKLD5PBgqLi5Gu3btsGjRoiq3W7NmDX755RfExsZ6qWRERESkBCG+LsD999+P+++/v8ptTp8+jWeffRbfffcdBg0a5KWSERERkRL4vGbIFr1ej3HjxmHq1Klo06aNr4tDREREAcbnNUO2zJ07FyEhIXjuuefs3qesrAxlZWWm90VFRZ4oGhEREQUAWdcM7dq1CwsXLsSyZcsgSZLd+82ZMwcRERGmV0JCggdLSURERP5M1sHQli1bcO7cOTRq1AghISEICQnBiRMn8MILL6BJkyZW90tPT4dWqzW98vPzvVdoIiIi8iuybiYbN24c+vfvb7Zs4MCBGDduHB5//HGr+6lUKqhUKk8Xj4iIiAKAz4Oha9eu4ejRo6b3eXl52Lt3LyIjI9GoUSPUq1fPbPvQ0FBER0ejZcuW3i4qERERBSCfB0M7d+5Ev379TO/T0tIAAI899hiWLVvmo1IRERGRUvg8GOrbty+EEHZvf/z4cc8VhoiIiBRH1gnURERERJ7GYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNFCfF0AIiIiUjidDtiyBSgoAGJigJQUIDjYa6f3ec3Q5s2bMWTIEMTGxkKSJKxZs8a07saNG5g2bRruvPNO1KhRA7GxsXj00Udx5swZ3xWYiIiI3CcrC2jSBOjXDxg71vBvkyaG5V7i82CouLgY7dq1w6JFiyqtKykpwe7du/Hqq69i9+7dyMrKwuHDh/Hggw/6oKRERETkVllZgFoNnDplvvz0acPydeu8UgxJCCG8ciY7SJKE1atXY9iwYVa32bFjB7p27YoTJ06gUaNGdh23qKgIERER0Gq1qF27tptKS0RERE7T6Qw1QLcHQkaShKLYWEScPu3x57ff5QxptVpIkoQ6depY3aasrAxlZWWm90VFRV4oGREREdltyxbrgRAACGGoIfICnzeTOaK0tBTTp0/H2LFjq4wQ58yZg4iICNMrISHBi6UkIiIimwoKfF0CE78Jhm7cuIHRo0dDr9fjww8/rHLb9PR0aLVa0ys/P99LpSQiIiK7xMT4ugQmftFMduPGDYwaNQp5eXnYuHGjzXZDlUoFlUrlpdIRERGRw1JSgPh4Q1OYpfRlSQJiY73SVCb7miFjIHTkyBH88MMPqFevnq+LRERERK4KDgYWLjT8f0kyX2d8/9ZbXimKz4Oha9euYe/evdi7dy8AIC8vD3v37sXJkydRXl4OtVqNnTt34vPPP4dOp0NhYSEKCwtx/fp13xaciIiIXJOaCmg0QFyc+fL4eMNyLw2l4/Ou9Tk5OejXr1+l5Y899hhmzpyJxMREi/tlZ2ejb9++dp2DXeuJiIhkzMoI1N56fvs8Z6hv376oKh6T0TBIRERE5AnBwYCdFRye4PNmMiIiIiJf8nnNEBEREfmIjydIlQsGQ0REREqUlQVMnmw+CnR8vKGHV2qq78rlA2wmIyIiUhpbE6R6ccZ4OWAwREREpCQ6naFGyFIHJeOyKVMM2ykEgyEiIiIlsWeC1Px8w3YKwWCIiIhISeydIFVGE6l6GoMhIiIiJbF3glQZTaTqaQyGiIiIlMQ4Qert84EZSRKQkGDYTiEYDBERESmJPROkLligqPGGGAwREREpja0JUhU2zhAHXSQiIlKi1FRg6FCOQA0GQ0RERMrl4wlS5YLNZERERKRoDIaIiIhI0RgMERERkaIxGCIiIiJFYzBEREREisZgiIiIiBSNwRAREREpGoMhIiIiUjQGQ0RERKRoDIaIiIhIfnQ6w1QhXsBgiIiIiOQlKwto0gQYPNgrp+PcZERERCQfWVmAWg0I4bVTsmaIiIiI5EGnAyZP9mogBDAYIiIiIrnYsgU4dcrrp2UzGRERWWZMYC0oAGJigJQUIDjY16WiQFZQ4JPTMhgiIqLKsrIMzRUV/0qPjwcWLgRSU31XLgpsMTE+OS2byYiIyJwxgfX25orTpw3Ls7J8Uy4KfCkphqBbkrx6WgZDRER0S1UJrMZlU6YYtiOyRqcDcnKAjAzDv/b+XoKDDbWPgFcDIgZDRER0i60EViGA/HyvDYZHfsg4RlC/fsDYsYZ/mzSxr0ZRpwMiIw0Beb16ni6pCXOGiIjoFnsTWH2U6EoyZ22MIGMTq0ZjPefMUp5avXrAxYueK+9NrBkiIqJb7E1g9VGiK8mYK02s1vLUvBAIAQyGiIioIlsJrJIEJCQYtiOqyNkmVh8NtFgRgyEiIrqlqgRW4/sFCzjeEFW2dq19293exOqjgRYrYjBERETmUlMNuR1xcebL4+Orzvkg33G295a7ZGUZgmR73N7EKoP8MyZQExFRZampwNChHIH6dnIcldvXA2Revw489ZTt7STJUK6bTaxCL/Dr0gPYPe8snvFwEW3xec3Q5s2bMWTIEMTGxkKSJKxZs8ZsvRACM2fORGxsLKpVq4a+ffviwIEDviksEZGSBAcDffsCY8YY/vX1Qx/wbQ2IK13GPVkmXw6QmZVlCHAuXLC9rRDQvzsfPy85gOc7bkLjsDPo9mRbPHt4EvIRDz28O9BiRT4PhoqLi9GuXTssWrTI4vp58+Zh/vz5WLRoEXbs2IHo6Gjce++9uHr1qpdLSkREPuXLYKSqoGPECOD5570fnPl6gEzjZ3L+vF2bb6ynRvzoXug1IRkL9vRBvi4ONXEVoxr9glMPTjCkpHl55GkTISMAxOrVq03v9Xq9iI6OFm+99ZZpWWlpqYiIiBCLFy+2+7harVYAEFqt1p3FJSIib8nMFEKShDA85m+9JMnwysz03LnLy4WIj698bkuv+HjPlyU7W4jly4V47z37ypSd7Zly2PuZ3Hz1QbYAhKiNK+KRxC1iTfp2UXKx5NYxMzMrHVMbF+eV57esc4by8vJQWFiIAQMGmJapVCr06dMHW7duxVNW2ijLyspQVlZmel9UVOTxshIRkYfYqgGRJEMNyNCh7mvKq5gbdPas/b2d7Blc0FmWcoPs4YkEZQd6gOkh4Qxi0fQOYOojO9A/LRmq2r0qb2gpT61dO8OI1B4m62CosLAQANCwYUOz5Q0bNsSJEyes7jdnzhzMmjXLo2UjIiI3sCch2ZHxa/r2db1MzgYdxrJ4IjizNrKzPTwxQKadAZYegASBhp/Nx38f7mt7B2OempGXKjN8njNkD+m2NkQhRKVlFaWnp0Or1Zpe+fn5ni4iERE5yt4cIG9OEWItN8gR7p6/zdlBCT00QGbplVL8vP6yXdsGNWgAKTMToQ+PcmsZ3E3WNUPR0dEADDVEMRUi23PnzlWqLapIpVJBpVJ5vHxEROQkR+aw8tYUIe4eCdldzVPODEro5gEySy6U4Nt5udB8ocOXJ5JRgqdwHHMQh9MIgpXPq0EDQ7nDwlw+v6fJumYoMTER0dHR2LBhg2nZ9evXsWnTJvTo0cOHJSMiIqc52gvKW1OEuHsk5LNn3TMEgDNBlRsGyLxWeA1fPL8VoxK2oUEDgRFvd0PGiZ64hlqICTqHDU3+DgmAsDRSuSQBixf7RSAEyKBm6Nq1azh69KjpfV5eHvbu3YvIyEg0atQIU6ZMwZtvvonmzZujefPmePPNN1G9enWMHTvWh6UmIiKnOZoDZJwiRK02PGQrBlHurAFxZ6JxcLChu72RK4Mg2lvj9d57QMOGLg0GWXSqCF/N2Q/N6mB8U9AOpbhV8dA4+BTU7Y9C/XR9dB3fGkEhrwJZbSwP+LhggX+NVO7Rvmp2yM7OFgAqvR577DEhhKF7/YwZM0R0dLRQqVSid+/eYv/+/Q6dg13riYhkZPly+7pjL19uvp+FrtciIcF9Xdmzs+0r13vvCTFlikPdyl0aAsDYjd3S0ALGYyckGLZzwuXjV8T/ntoiHozeLlT4y+zQTUOOi2l3ZYsdnxwQep3eevmM3f2zs50uhyXeen5LQvhwmlgvKSoqQkREBLRaLWrXru3r4hARKVtOjiFZ2pbs7Mq9w4wjUOfkGN737eu+0bF1OkMC9+nTlpvwjNNJ5OUZzmep11lwsPUmsdv3d4QxxwqwXDPmYJPYpWOXsXb2b9B8FY4N59vhBm41Z7UIzcPIriegnhSDdiNbQAry3cjQ3np+MxgiIiLvcjToqMjT83A5GnTcPh5RxaYxaywFefaW7fZrT0iwu0nq/MELWDPnd2i+ro6NF9uhHKGmda1VRzGy2ymoJ8ehzdBm3gmA7BhWwWvPb4/WO8kEm8mIiGTGOKL07U0/VTUneWsUameb45xt/nOEg01SBfvOig9HbxJ3190lglBuVozk8D/E63dniwNrjzhfHmdZ+owtjN7NZjI3Ys0QEZEMOVLTYaxNspZ47UoTlCXOzE7vSPNfSorjx7fT6Z0FyHrrMDQ/RGCLNhmiQsfxjtUOQp1yFiPSGqPFwES3nM9h1oZVsFD7xmYyN2IwREQkU/YGHa7kGXmLvc1/8+cbmtPc2NR3cttpZM49As3GSGy9mmy2rmuN36DucwEjpjZF076NnDq+2zgY1Hrr+e3zrvVERKRgt0+/YI03R6F2lj1DAIweDYwaZd9gkzbkbc5H5rxj0OTUxy/FbQHEmdb1qJUL9d2XkDq1GRr3bOvihbmRt6dWsRODISIikj9vjULtqtRUQ0BjKcn73XeBtDSXJpw9suE4Mt89Ds2WKOwqaQ0gAQAgQY+UiFyo+2sxfFoLxHdJtri/z8k0qGUwRERE8mcchdpWE5Sb5+FyiqXZ1405Qk7Uivzx9Z/QzD8Jzc8x2FfaEkATAEAQdOhTJxfqgUUYPr0VYtq39+RVuYdMg1oGQ0REJH/eGoXaXSw1/9lZ2yFOn8GB1UegWXgamu1xOFDWHEBTw2FRjrsj90F9fzGGpSchqk0H95bb02Qa1DIYIiIi/1BVE5Q/TP9gZ23HY+OBT8ubA2gOAAjFdfRvsA/qQaUY+lIb1GveyXNl9DSZBrXsTUZERP7FmW7vciiLjd5mekg4hXgkIg8hKMfAhnuhfvAGhqS3Rd3EOu4pg1zYOawCu9a7EYMhIpIlf3yI0S1OjIYtNJnAyJEABCqO8ay/+e7tOm8iYWwKBqffidrxdjyvPD0ityfJaARqBkNERL7gzw8xcmjgQH25Hts++g2ajy4jM7cZOut+wUJMRgJuffclNaMgLZiPak887JEy+CsGQ27EYIiIZEUBD7GAZsfAgSIuDpvT1kCz9BoyD7REgT7atLomrmJIwl481T0Xd/WrjvBWiY7VChonqx01Crh0yWoZ3Doit48wGHIjBkNEJBvenlaC3M/O0bD7Ihub0BcAUBtaPJi4H+rRoRjwYjKqRVZz7tyWahSr4ssRud2AI1ATEQUimY7AG7A8kZdlZxf5ZjiKxGbBUD8cjv5pyVDV7uXaea3VKFbFlyNy+xEGQ0RE3nT6tH3b8SHmOg/lZV2vXQ9hdmz3768bI/R+N42Xo9MZrsXRxhxfj8jtJ4Jsb0JERG6RlWWYbsEefIi5xliLcnstnHEOsKwshw5XeqUU617+BY/e8ROiB3dGPuJNPcAqkSQgIQGhA+52svAW2KpRtFIGWYzI7QdYM0RE5A32NnHIaVoJR8llqICqalHsnAMMAEoulODbebnQfKHDlyeScQ13mdbNlGbhP+IJCEiQ4IWBAx2pKZTjiNwyx5ohIiJPs7eJw58fYllZhsTwfv2AsWMN/zZp4nANjFs4kpd1m2uF1/DF81sxKmEbGjQQGPF2N2Sc6IlrqIW4oAJMbr8JWz7IxUfXx0PKzIQUH2d+gPh4z/QGdKSm0FNlCGCsGSIi8jR7mzjq1wcWL/a/h5i1Wi9jk5S3H8wOzoxedKoIX83ZD83qYHxT0A6l6GHapHHwKajbH4X66froOr41gkIqBCXWJmT1RCBra04vAKhXD1i50pB472/BtI8xGCIi8jR7H87vved/gZCbmqTcys5alO8+v4APnv8F351tj+voaVreNOQERnbKg3pCFDo9kgQpKN76QSxNyOoJ9szptWQJcM89ni9LAGIwRETkafY2ccTF2d5GbuQ4VICNWhQ9gNOIxwPrJ0APQ4DWIjQPI7uegHpSDNqNbAEpqLF3yuoIf5+oVsYYDBEReZqtJg5/Tpp2sEnKKyrUoghJglThMzf2AJuMhWilysPIbqegnhyHNkObQQpK9F4ZneXNpjkFYTBERORp9jRxvPuufz7g7K318uJQAYW557B6VX0UVp+HJ4sXms0BdhYN8VPbZ/DP2clo/WAzAM28Vi638VbTnIJwOg4iIm+xNAhgQgIwejSQkeGfk7YapxexVevl4elFTu8sQNZbh6H5IQJbtMkQNztLB0GHJ8I+w6CWR9HxkdZIeGGUfwSZBIBzk7kVgyEiko3bx+K5cMEw4aY/T9pq7E0GWK718tA1nNx2Gplzj0CzMRJbryabreta4zeo+1zAiKlN0bRvI7efm7yDwZAbMRgiIlny9KSt3hwE0Vqtl5sTe/M25yNz3jFocurjl+K2Zut61MqF+u5LSJ3aDI17VtEDzNPkMvhkAOBErUREgc6TPbHcMS+XIw91Dyb2HtlwHJnvHodmSxR2lbQGkAAAkKBHSkQu1P21SJ3eAnGdk6s+kDd4aD408iwGQ0RErnK2JsBTPbGsDYJ46hQwYgSQmWn7wWzpod6gAfDhh7eaxG7nxsTeP77+E5r5J6H5OQb7SlsCaALAkAPUp04u1AOLMHx6K8S0b++W87mF3AafJLuxmYyIyBWu1ATk5BimrbAlO9v+IMNW0xtgGKn47FnrAZutedSmTgXmzbOvPHYSeoEDa49Cs/A0NNvjcKCsuWldMMpxd+Q+qO8vxrD0JES1aeDWc7uFp5s8FYo5Q27EYIgogMgpH8Na0GBv4rAnemLZG2DNmgW89pr1MtmaPuSLL4CRIyvv68B3I/QC+1Ydhub9Amh2NMKh601N60JxHf0b7IN6UCmGvtQG9ZpH2r4mX/JEYEvee34LBdBqtQKA0Gq1vi4KEbkiM1OI+HghDKGD4RUfb1jubeXllctS8SVJQiQkGLarSmamYVtJqry/JDl+bcuXWy9TxVe9epbLlp1t3/4NGpjvb+d3o9fpxY5PDohpd2WLO0KOm20ehlIxpOF28cnftohLf1527Lp9zd7PfflyX5fUr3jr+c1Z64nIPxhrYW6vsTDmY3h7dnQXZkY3Y5xi4fapOJydedzewQ0vXgRmzjTUaOh0t5bbm590/vyta7Px3QhNJn75+DdM7ZKDpqpT6PJYa8z9pS+OlTdGOP7C8Jjt+HzCzzifX4Z1hXfh0SW9UDexjn3lkAsZDj5J9mMzGRHJnxzzMTIygLFjbW+3fDkwZozt7dzV/KfTAVFRwKVL9u9TMcfJ3uYewHBto0ZV+d0IAGcQi0Y4aZoHrDqK8UBcLkaOFHhgWjJqRte0v6xyJZPBJwMNu9YTERl5ezJQewITd9cEuKsnVnCwIaF7xgz796nY22noUEOvsfPnbe8XE2Pzu5EAxOEMBuA71GlUB+pREu6bmowaUd3tL58/CA4G5s83BIe3M+aQLVjAQEim2ExGRPLnzclAs7IMf+H362eo+enXz/D+9mY44+Srxgfd7STJMOigLyZfffllQ48xexlrMqZMMfz74Ye297l5bboTNhKtb1q75DwyTvTAiLe7o0ZUDfvL5i+ysoC0NMvrnG3yJK9hMERE8uetfAxH8pKMk68ClQMiX9cEBAcDS5ZYD9QsqVi7plYbus9b2xTAn4374m9tfkbq+Fp2HT6seWP7y+JvrP1ujObPZyAkcwyGiEj+vFELo9MZmpcs5XtUrDmpmGzs7uRndzKWLd7BaSmMtWvz5gEvvGB1s8SfPkX0oRx8jftxGrHQW9vQlzVk3lDV7wYwXH9amvnvhmRH9sFQeXk5XnnlFSQmJqJatWpo2rQpXn/9dej1Vm89Igo03qiFcbZ3WGoqcPy4YfyY5csN/+blyaMmoGLZXnnFvn2MtWs6HUTGClh6xEs3X29gBq7Wjkf082MQJEnyqyHzBnf1KiSfkn0C9dy5c7F48WJ88sknaNOmDXbu3InHH38cERERmDx5sq+LR0TeYqzpsDTaszsmA3UlL8mN01C4nbFsKSnAsmU2ezuVJHXCt//YjgP/24FXz562efjwovPAgvnAiy8aeth54rsB5DXYZkXezGcjj5F9MLRt2zYMHToUgwYNAgA0adIEGRkZ2Llzp49LRkQeZenh58HJQAN+nBhj7ZpabQh8KgRE4ub7d0uewYzoIJSgG0Yjz7Hjr1gBHDsGbN3q/u9GzpOfBvrvRiFkHwz16tULixcvxuHDh9GiRQvs27cPP/30ExYsWODrohGRp9h6+HmiFsaYl2RrnBh/zn2xUrt2SsTheczHhYsN8CDWQQSFoM8dBcARO49rbArautX9343cJz9Vwu9GAWQfDE2bNg1arRatWrVCcHAwdDodZs+ejTFVDGJWVlaGsrIy0/uioiJvFJWI3MFXD78qak4CJfflygkt1n0bhazrK1GMa6iPiyhADOrjAt7Hc4jBzaYcPYC/4g3d8y9dsp4cfDt3NwXZSmqXJENS+9Ch3vlerDXVBfjvRhE8OtmHG2RkZIj4+HiRkZEhcnNzxf/+9z8RGRkpli1bZnWfGTNmCBh6f5q9ODcZkcy5a74vV1iaYyshwTfzn7nBxaOXxH8f3yweaPCrCEWZ2WW1CP1TfN5qltBDEnpLn7U9c21VfGVnu7fw9s6T5u7zWmJr7rUA+93IhbfmJpP9dBwJCQmYPn06Jk6caFr2z3/+E5999hn++OMPi/tYqhlKSEjgdBxEcieXmb/lmqxrp/MHL2DNnN+h+bo6Nl5sh3KEmta1Vh3FyG6noJ4chzaDEyE1Tax6mpPISCA83FAzZ42npppw95QnzrJWW2ms+THWVvr570aOOB3HTSUlJQgKMh8BIDg4uMqu9SqVCiqVytNFIyJ3k0vPHDn3DrOiMPccVs/5A5rvaiLncjvo0du0Ljn8ENQ9CjBicjxaP9gMQDPDipwc293CL14EfvgB+Plny1N8uKspyFIgIYfkZEeb6vzsd0MGsg+GhgwZgtmzZ6NRo0Zo06YN9uzZg/nz5+P//u//fF00InI3OTz8/MjpnQXIeuswND9EYIs2GaJCANSx2kGoU85iRFpjtBjYEkDLygewN6g8dw547TWgbVvPDG1gLWH+vffsS07u0cMQ2HmiRsbb8+KRT8g+GHr//ffx6quvYsKECTh37hxiY2Px1FNP4bXXXvN10YjI3dgzx6aT204jc+4RaDZGYuvVZAC3AsOuNX6Dus8FjJjaFE37JgFIqvpgjgafFYc2OH3aMJlrgwaGpjSdzrkApKqE+VGjDOMXvfOO9eTk0aOBO+7wXLf7tWvt247jCPk3j2YkyYS3ErCIyA0yM6tOllVgQuqfm06KeQ9ki6419lf6OHrU2ifmD80Wx3/Kd/zAxoR1a8nS1hLWbSUTO3p+WwnzX3xhOTl56lTLZZckw8vV34qt36K3k7gVyFvPb9nXDBGRlzEJ1HVu+AyPbDiOzHePQ7MlCrtKWgNIAABI0CMlIhfq/lqkTm+BuM7JzpfTmW7h7hz6wN4mqAYNDNOKVPxMe/Qw1Ah5qtu9MVfIHg0aGMpD/sujoZZMsGaIyE7u+ovfWXLoWu8qFz7Dg+uPiTfuyRbtwv8w2z0I5aJfnd3ig4dyxJk9hd4ps6Vu4e7+fpYvt6/WZfnyyvt6utu9vcf3xX2iILKsGcrPz0dCQoJnojIi8i05jPTr78mqDn6GQi9wYO1RaBaehmZ7HA6UNQfQFAAQjHLcHbkP6vuLMSw9CVFtOniu3PZOc+Lu78eVhHlP9zx0dD+5jIhNTnEoGGrVqhXS0tIwffp01KhRw1NlIiJvk8tIv3LpWu8MOz9DMeRB7Ms6Bs37BdDsaIRD15sDaA4ACMV19G+wD+pBpRj6UhvUa97Je+W3p1u4u78fVxLmPd3z0NH9vHmfkNsF2d7klg0bNuD7779H8+bNsXTpUk+ViYi8zZG/+D3Jn7vW2/kZPhKuQYfRLTH75744dL0pwlCGIQ1/wSd/+wln/yzB1+e64P+WpqBe80jvld1e7v5+jDlLwK0cJSNb4xcZA6nb96u4f0KC8z0PbR3fEm/dJ+R2DgVDPXr0wC+//IK33noLr732Gjp06ICcnBwPFY2IbNLpDOOrZGQY/tXpnDuOXGpkPP2A8yQ7Pxu9XiAcf2F4zHZ8PuFnnM8vw7rCu/Dokl6om1jHs2V0lSe+H+PksXFx5svj46tucnIlkLJHVce3RY41l1Qlh4Iho0cffRSHDx/GkCFDMGjQIAwfPhxHjx51d9mIqCpZWUCTJobpK8aONfzbpIlhuaPkUiPj6QecJ9n52fx9xEWcL9Ah60w3jP2gJ2rH+9EUQZ76flJTDb3FsrMNU2tkZxum9rCVe+NsIOVIuSwd3xY51lxS1ZzNvC4uLhZbtmwRU6ZMEUFBQUKlUom0tDRRVFTkzgRvt2BvMgo4mZnuHV/F2fFmbj9Gdrah5092tms9vvxo0svysnKRs2CPmNT2R3EaMUIHy5+h3h96wtlLbt+PO397VR3/s8+EqF/fv3s7+hlZTtS6ePFi7NixAzt27MDBgwcRHByM5ORkdOvWDe3bt8fnn3+Ow4cPY/Xq1ejcubPnIjgHeWuiNyKv0OkMNUBVTa7pzKSZxp5QgOXxZqr6S9vadAqujAIs4/GOykvLsen9XGiWXUPWwSScEw0AAMORBQ3UAIR5tbs9n6G/kfH341Gu3CfkMK89vx2JnOLj44VarRbvvPOO+Omnn0RpaWmlbWbPni3atGnjpljNPVgzRAHFk+OrOPMXv7trqWTqevF18e0/d4gnW24S9aQLZpdaR7osxjfbLL6a8au4/ukKedWakPvJrWYsgMmyZsgeZ8+eRWxsLHTOJnJ6AGuGKKBkZBhyhGxZvhwYM8bx4zvyF7+naqncVT4XlRWV4Yf5udB8Xoq1x9risqhrWldPuojhLQ5APa46+k1ORljNMJ+U0YxSa2t8gZ+1V3jr+e326TiioqKwceNGdx+WiIw8nexsz3gzRt4eJNETzXG3Kb1Siu/f3gfNihtY9+ed0KKLaV2UdB6pSQehHl8TfSYlIyS8t+WDOPIZuosXPhuqwBffMXmM24MhSZLQp08fdx+WiIzkNLO7N7vke3CE7JILJfh2Xi40X+jw5YlkXMNdpnUxQYUY0eYQ1E9EoNczdyI4zEoA5EtyGD1czliLQzZwolYif+PM5Jqe4q0u+R4YIfta4TV8PTcXGo2E9aeSUYJupnXxwWcw4s4jUP+tLnr8vS2CQqJdK78nyWX0cLlijRnZwe05Q3LEnCEKSJb+I5+QYAiEvPUfeWPOkK1aKldzhnJyDOMo2ZKdXWXTRdGpInw1Zz80q4PxTUE7lKKaaV3j4FNQtz8K9dP10XV8awSFODUMm/e56bMJSNZqzNjzy2/4bc4QEXmJvZNrepK3aqlcaI67ckKLdbP3Q7MuFN+dbY/r6Gla1zTkBEZ2yoN6QhQ6PZIEKSjetXL6glxGD5cb1piRAxgMEfkzOSRxGkfptdQU4a5aKgeb4y4du4w1//wNmvXh+OF8O9xAL9MmLULzMLLrCagnxaDdyBaQghq7Xj53cDavRS6jh8uNt5P7ya8xGCIi13m6lsqOpHFdw1gs/Y/AKvVObLzYDuW4lUDeWnUUI7udgnpyHNoMbQYpKNE95XIXV/Ja5JRQLyesMSMHMBgiIvfwZC1VFc1x4ub/PFS4AJmf38qdSQ4/BHWPAoyYHI/WDzYD0MwzZXOVqz3B5JRQLyesMSMHMIGaiPxHVhbKn3kWIedu/TV/EgmYggVYjVR0rHYQ6pSzGJHWGC0Gyqz2xxJ3Dloph4R6OfFWcj95FBOoiYhuOrntNDLnHoFmYzNsv5qPFGxBDApQgBiUVa+L1L6X8c7Uk2jaNwlAkq+Laz935rXIIaFeTlhjRg5gMEREsvRnzklkvv0nNJvq49fitgDiTOtu1IpE17uB1KnN0LinH/YAM3J3XoscEurlxBvJ/RQQGAwRke/c1oPqSGkCNO/lQ7OlIXb/lQSgEQBAgh4pEblQ99cidXoLxHVO9m253YV5LZ7HGjOyA4Mh8l8cYt+/WchxCUc8dmAhdqMvgqBDnzq5GHnfVQxPb4Xo5Pa+K6unsCeYd7DGjGxgMET+iUPs+y2hF8hP/wAJ8yZBAKg4znMcTiMTI7Ah5Q20//dTiGrTwVfF9A7mtRDJgp+MN09UgbEr8u2Jp8auyFlZvikXWSX0AntXHsIrvXLQJvwopHlzKwVCABAEAUmSMOD4EkS1ivRFUb3PmNcSF2e+PD6e00UQeQm71itBIDUnubMrMnmU0Avs+uwgNB+eg2ZXIo6VG0Z67oMc5IBzaVUSSPcpkZuwaz25R6A1J3GIfVkTeoFflx7Aqn9fgGZvM5zQtQbQGgAQjr9wf8w+vNA2B9hgx8GUNjIw81qIfIbBUCBzdWRbOeIQ+7KjL9dj20e/QfPRZWTmNkO+rq1pXXUUY1B8LtRqgQemJaNmdDcgpxTYMMv2gdmDKjCxBoxkiMFQoArUGZvZFVkWdNd1+Onf+6H5WIvMAy1RoL/V1b0mrmJwo/0Y+VAQ7vtHMqrX726+M3tQKVeg1VRTwGDOUKDKyQH6BWBeBofY99lf1uWl5dj0fi40y64h62ASzokGpnW1ocWDifuhHh2KAS8mo1pktaoPZqy1BCz3oPLHWktHKa2GxFpNtZK+c3IYc4bINYHanKT0rshe/sv6RskNbHxvHzSflmD14Ta4KDqa1tWRrmDYHfuhfjgc/dOSoardy/4DK31kYKXVkARqTTUFDNYMBapArRkyUuKklF76y7qsqAw/zM+F5vNSrD3WFpdFXdO6etJFDG9xAOpx1dFvcjLCaoa5djKl1Y4AyqwhCfT/HpHHeOv5zWAoUCmhOUlJD1IPDylQeqUU37+9D6sybmBd3p0oQoRpXZR0HqlJB6EeXxN9JiUjJJwVyk5T6tAQGRnA2LG2t1u+HBgzxvPlIb/BZjJyTSA3J90eBI0a5Z/X4QgPDClQcqEE387LheYLHb48kYxruMu0LiaoECPaHIL6iQj0euZOBIf1dvECCIByh4ZgxweSOQZDgSwQ8zKUlmth5KYcsGuF1/D13FxoNBLWn0pGCbqZ1sUHn8GIO49A/be66PH3tggKiXalxGRJoOby2cIehCRzDIYCXSDN2Ozv4ya50qznwl/WRaeK8NWc/dCsDsY3Be1Qih6mdY2DT0Hd/ijUT9dH1/GtERQSa995yDlKrSEJ5JpqCgjMGSL/4O+5Fq7WaDmYA3blhBbrZu+HZl0ovjvbHtehMm3aNOQERnbKg3pCFDo9kgQpSHL9+sg+csnl81W+nRI7PpBLmEDtRgyGAoA/90ZxV+8hG2PzXP3XUqza1RSa9eH44Xw73MCtnl4tQvMwsusJqCfFoN3IFgyAfMnXYyz5uqlZSR0fyGVee34LP3Dq1Cnx8MMPi8jISFGtWjXRrl07sXPnTrv312q1AoDQarUeLCV51PLlQhgeHVW/li/3dUnNlZcLER9vvbySJERCgmE7e2RmVjre1epRYmbNeSIYN8wO3UZ1WMzoky32Zx0Wep2+crmysw2fV3a2/ed3la/OKzcWvkeRkGBY7unzSpLl36Ekef78RA7y1vNb9jlDly9fRs+ePdGvXz988803iIqKwrFjx1CnTh1fF428yV9zLdzdeyg1FYWJ3fHLC19g/6/F2Fh8FzaV9IUehr+sk8MPYWTPAoyYkoCkwc0BNK98DF/VDPi6RkJOfJHLx4EPiaySfTPZ9OnT8fPPP2PLli1OH4PNZAFALrkWjnLT+CqndxYg663D0PwQgS3aZAgEmdZ1rHYQ6pSzGJHWGC0GJlZ9Hl8N+KfEgQblxp+bmkmxvPX8DrK9iW+tW7cOnTt3xsiRIxEVFYUOHTrgo48+qnKfsrIyFBUVmb3Ig3Q6w39oMzIM/+p07j+HsTcKcOsBaiTn3igu1Gid3HYa7w3LQc/auYjvEoPnMvtgs7Y9BILQtcZvmPdADo5ln8SukiSkf9fXdiBkq2YAMNQMuPv789V5yZxSu/UT2cOjjXBuoFKphEqlEunp6WL37t1i8eLFIjw8XHzyySdW95kxY4YAUOnFnCEPsJT7EB/vudwDX+VaOMuYM2QpT8NCztCx7BNi3gPZomuN/ZU27VFrn5g/NFsc/ynfubJkZ9uXd5Wd7bbL9+l5yRy/B/JD3soZkn0zWVhYGDp37oytW7ealj333HPYsWMHtm3bZnGfsrIylJWVmd4XFRUhISGBzWTu5qumD3/rjWKj99CZVz/EJ9tbQbOlIXb/lXRrNfRIiciFur8WqdNbIK6zi/lQvpoSgVMxyIOt4SkAQzd3uTU1k6JxOo6bYmJi0Lp1a7NlSUlJyMzMtLqPSqWCSqWyup7cwJfJmMHB/pXTYGUkcG1YfczSv4r3Xn/atCwIOvStuw/qgdcwPL0VopPbu68cvkpC99fkd3eQU+AeHGwINt9+2/o2o0czECJFkn0w1LNnTxw6dMhs2eHDh9G4cWMflYgAKHeOJSeJYcNxQN8aO//5LQ4cENhR3gFbylKgRzCCUY67I/dh5KBiDEtvjQZJHT1TCF9NiaDUqRjk1ntOpzPU0lVlxQpgzhwGRKQ4sg+Gnn/+efTo0QNvvvkmRo0ahV9//RVLlizBkiVLfF00ZWMypk1CL7Bv1WFo3i+AZkcjHLreCkArAEAormNgg91QDyrF0JfaoF7zTp4vkK+mRFDiVAxynDrG1h8wAP+AIeXyaEaSm3z55Zeibdu2QqVSiVatWoklS5Y4tD8HXfQAJmNapNfpxY5PDohpd2WLO0KOm30UYSgVQxpuF5/8bYu49Odl3xXSlwP++VPyu7PcPdCmu/jrwKWkaEygdiOOM+QB/jrujwfoy/X4ddnv0Cy+AM3eZjihizetC8dfuD9mH9TDdRicfidqxzvw+/NkvomvclnklEPjKXIdz0eu5SKqAhOoSd6U2PRRgb5cj20f/YZVSy4jc39znNK1Na2rjmIMis+FWi3wwLRk1Izu5vgJPJ1v4qskdH9LfneGXJuQlZq7RWQHBkPkPCu9pBAfH5CzUOuu6/DTv/dD87EWmQdaokCfbFpXE1cxuNF+jHwoCPf9IxnV63d3/kRyzDch+8m195zC/4AhqgqbySxRQlW+O3n78/Li+cpLy7Hp/Vxoll1D1sEknBMNTOtqQ4sHE/dDPToUA15MRrXIaq6f0NZYMApqfvRbcm9CtlTrmJAQkH/AkP/zVjMZg6Hbya07LJnzwvdzo+QGNr63D5pPS7D6cBtcFPVM6+pIVzDsjv1QPxyO/mnJUNV283hWzOsIDDYG2vR57R7/4CM/wZwhX1Bi84Q//UfRg99PWVEZfpifC83npVh7rC0ui86mdfWkixje4gDU46qj3+RkhNX0YE6FXPNNyDFyb0JWQu4WkQNYM2SkxOYJf6oF88D3U3qlFN/N2wfNihtYl3cnihBhWhclnUdq0kGox9dEn0nJCAn30t8NrBkKLNevAx9+CBw7BtxxBzBhAhAW5utSEfkNNpO5kV0fptIeQrbmFVu5EmjQQD41Rm76fkoulODbeblYtVKPr07eiWuoZVoXE1SIEW0OQf1EBHo9cyeCw3xwvXLPNyH7+dMfG0QyxWYyb1NS84StecUAwxxGOt2t5b7+j7gL38+1wmv4em4uNBoJ608lowS3urrHB5/BiDuPYOTf66L739oiKCTaXSV2Dnv8BAYlNrkT+TEGQ0Zy7Q7rCfYMy18xEAJ8/x9xB7+folNFWD97L37/4jecvFQDJ9AYW2CYC6xx8Cmo2x+F+un66Dq+NYJCYj1YcCfIPd+EqubLSYyJyClsJjNSUvNERgYwdqzj+/nyM7Dj+9FHx+LzBz7Hqq/CEX72ON7Fi0jArWDiSlgDXHjqZdyx4DlIQZL3yu4sf0pup1uU1uRO5EHeaiYL8tiR/Y2xeQK41RxhFGjNE87WblWcid7bqvh+BAAhBEYXzMejH/dByNnTWIExiIN57VedGxfQbNHzkNas9lKhXWTs8TNmjOHfQPjtKYGSmtyJAgSDoYqMzRNxcebL4+MDq43fOCz/7UGfvXz1H/Gb348uyjyYy0cCRiATqzAKd4b9gaVhT0OCqPzjNtYoTZlSuRnQEp3O8Fd+RobhX3v2IVJSkztRgGAzmSVKaJ6wNiicPXxQvV+Yew6r5/yBVd/WwpYrbdATWxGDAhQgBkWqKKT2OocRUxKQVDPfPU0U7AlEzlJSkzuRh7E3mS8pYUAya0m6wcHWa0C8PJHj6Z0FyHrrMDQ/RGCLNhkCvU3rrlZriPt6A6+/0ATN720CoLVhRcav9h28qtot9gQiV7BHIJHfYc1QIHKkZuv2bS9cAEaNMqzzwTQCJ7edRubcI9BsjMTWq8lm67rW+A3qPhcwYmpTNO3byPIBXE1eVeLgm+QZnAOMyGUcdNGNPPZhyrE5zR3NO17+j/ifOSeR+faf0Gyqj1+L25qt61ErF+q7LyF1ajM07hlv+2CuNlGwJ5CBHH/b/oifI5FL2Ewmd3LMKXFX805qqmEMFA/+R/zIhuPQvHMcmi0NsfuvJACGmh4JeqRE5ELdX4vU6S0Q1zm56gPdztUmCkd6AgXqg06Ov21/pYQmd6JAIBRAq9UKAEKr1brngJmZQkiSEIZH7a2XJBlemZnuOY8jysuFiI+vXKaKZUtIMGznIwfXHxNv3JMtksP/MCtaEMrF3XV3iQ9HbxIF+86652SZmZU/j4QE299Ndrb1z7Dia9asysePj/fNd+9OcvxtE5Fiuf35bQWbyRwl15wSGTbvCL3AgbVHoVl4GprtcThQ1ty0LhjluDtyH0YOKsaw9NZokFTf/QVwpubGnma2yEjg0iXr87r5a4K1XH/bRKRYbCaTK1tTWVQcmNCb1eMyGehN6AX2rToMzfsF0OxohEPXmwMwBEGhuI7+DfZBPagUQ19qg3rNO3m0LE41UdhqZjO+D8SpFuT62yYi8jAGQ46SSdBRiQ8HehN6gV2fHYTmw3PQ7ErEsfKWAFoCAMJQhoEN90L94A0MSW+Luold3H5+t6tqbrAnnwRmzLC+rz8HDHL9bRMReRiDIUfJdXRZ46jStnpRuWmMIH25Hr8u+x2axReg2dsMJ3StYRzrJxx/4f6YfVAP12Fw+p2oHX+XW87pVdaSyL/4wr79/TFgkOtvm4jIwxgMOcrLQYfdvDDQm75cj61LfoPmo8vI3N8cp3S3usFXRzEGxedCrRZ4YFoyakZ3c/o8smGpmS2QAwa5/raJiDyMc5M5quKEodb4anRZD8ytpruuw6aFezEpeRPiVeeQMjEZC/f2wSldLGriKsY0/hmZU7fj/HkJX+R3x6j3eqBmdE03XZAM2ZrXTZIMYzL5Y8CgpMmKiYgqYG8yZ/3jH8D8+eZTVwQHA2lpwLx57jmHs1wc/6a8tByb3s+FZtk1ZB1MwjnRwLSuNrR4MHE/Ro4JxYCp7RBeJ9wTVyBv1uZ18/feZEYcOZmIZIIjULuR2z9Ma4Mb3v4w9KNB+W6U3MDG9/ZB82kJVh9ug4uinmldHekKht2xH+qHw9E/LRmq2iofllQmAj1g8KPfLhEFLgZDbuSTcYbmzweef17Wo/iWFZXhh/m50HxeirXH2uKyqGtaV0+6iOEtDkA9rjr6TU5GWM0wH5ZUphgwEBF5FIMhN3Lrh2nv4IaWeKIZxcEHcumVUnw3bx80K25gXd6dKEKEaV2UdB6pSQehHl8TfSYlIySc+fVEROQ7HHRRrlzpMu3uQfnsnEOq5EIJvpm7D5ovBL46eSeu4VZX95igQoxocwjqJyLQ65k7ERzW27Uy+RJraoiIyAkMhhzlapdpdw3KZ2NS1r8++hRf/pYIjUbC+lPJKEF30ybxwWegTj4C9d/qovvf2iIoJNr5csgFJxclIiInsZnMUbbmrrLX8uXAmDGulcFK3pIewGnEowmOQw9DzUjj4FNQtz+Kkc/UR5fHWiMoJIBGVbA3oZ2IiPwKm8nkyt65q2xxpYbJxhxSQQAScAqjg1YhoUs01BOi0OmRJEhB8c6fU650OkONUCDOFUZERF4RQNUDXlTV4IZffOHxQfmu7jli13af/U+Pt7b3RedHW0MKslIef+fI5KJEREQWsGbIWdbmrgoONrzcPC3GuQPnsWbOQWi+qQHdpab40Y59pLhYh84ha9aSozm5KBERuYjBkCsszV0FVD3ruQOD8hXmnsPqOX9g1be1sOlKMvQw9PQKgg6FiEZDFMJifU+gzSFVVXJ0IM8VRkREXsEEak9yoqv36Z0FyHrrMDQ/RGCLNhmiQktmx2oHMbL3WYx4oQmaX90d2FNCGNlKjl650jAFiq3JRfPymDNERORnOOiiG/ksGLLTyW2nkTn3CDQbI7H1arLZuq41foO6zwWMmNoUTfs2Mt9RCVNC2Dva96hRhmWBHBgSESkMe5N5ikwG5vsz5yQy3/4Tmk318WtxWwC3krF71MqF+u5LSJ3aDI17trV8AJ0OiIwE3noLOH8eaNDAkNAdSAMN2pscXb++W5oliYhImfwuGJozZw5eeuklTJ48GQsWLHBs53XrgPR07w7MVyH4OnVawmffN8Cqn2Kx+68kAIaaHgl6pETkQt1fi9TpLRDXObnqY1aVQxMogRDgWHL0mDHWE9qJiIiq4FfB0I4dO7BkyRIkJ9sIFqwZN67yspsjNnukKSUrCzf+PhGhFwsBAPEAHkY8fsVC7EUL9K27D+qB1zA8vRWik9vbfcyqRp4OqCYhR5OjrSW0ExERVcFvxhm6du0aHn74YXz00UeoW7eu7R3sZQwqpkwx1OK4eji9wG+rj2Blm9ehH6FG8M1AyCgOp5CJEbi08FP8eKkjnsnojejkKPsObmuAQcBt1yELKSkeH7OJiIjIb4KhiRMnYtCgQejfv7/NbcvKylBUVGT2qpKLA/MJvcDelYfwSq8cJFXLQ7vUpujx+0eAWV8wgyAAkiQh4p3XHA9alDbAoHG0b6ByQOTCmE1EREQV+UUwtGLFCuzevRtz5syxa/s5c+YgIiLC9EpISLDvRA4MzCf0Ajv/9zumd8tBc9VJdBjdErN/7otD15vibvyIBJyy/uE6G7QocYDBqkb7DqQmQSIi8hnZ5wzl5+dj8uTJ+P777xEeHm7XPunp6UhLSzO9Lyoqsi8gspGjoi/X49dlv0Oz+AI0e5vhhK41gNYAgHD8hftj9kE9XIdhSaeASXYU1NGgRakDDFY12jcREZGLZD/O0Jo1azB8+HAEV3jw6XQ6SJKEoKAglJWVma2zxDROAQCLoxRUMTCfvlyPrUt+g+ajy8jc3xyndLemuKiOYgyKz4VaLfDAtGTUjK5pWJGTA/TrZ/visrMdS/g1jrvjiwEGZTIkgVVyLx8RETmM4wzddM8992D//v1myx5//HG0atUK06ZNsxkIVWJpZnkhDAP33TyW7roOWz7cD81/tcg60BIF+lu912riKoY0zoV6VDDu+0cyqtfvXvkcxsRfW0GLo4m/xhwaN897ZlNVXfnl0Ewl9/IREZG8CT/Up08fMXnyZLu312q1AoDQfvqpEPHxQhjCCLOXPi5e7Bv3tni69SYRJZ0zW10bV8S4plvE2pe2i78u/2XfSTMzhZAkw6viwYzLMjOdu3jjsW+/joQE145Z1bluvwZ3XYcSykdERE4zPb+1Wo+eR/bNZJb07dsX7du3t3vQRbNqtu++uzV1QwX6m1OeqqHBaqSijnQFw+7Yj5GPhOOe55Ohqq1yvKCenC7DG81C9k6H4at5v+RePiIicgnnJnMj04d56RJqtb0TOHPa4mzveki4EtIAO15bh37Pd0BYzTDXT+7PuSyeyn1yF7mXj4iIXMKcIQ94u9VHeOPcaavrgyAQWX4OA1P+AoyBkKvBjD+Piiz3rvxyLx8REfkFRQVDB8/Vs29D48NT6Ym5cu/KL/fyERGRX/CLQRfd5a6m5+3bMCbm1hxgt+ejGOcAy8pyfwHlRu7TYci9fERE5BcUFQw9tfMp+x6ePXr45xxgOp0hjyYjw/Cvq+WT+3QYci8fERH5BUUFQ3Y/PLdu9b85wLKyDD2r+vUDxo41/Nukies1WHKfDkPu5SMiItlTVm8yYza6rS7vGRmGgMKW5cuBMWM8Vm67GZv0bv8qjQGeO4ICufeKk3v5iIjIYexa70YWP8yqHp7+1GWbY+0QEVGAYtd6T6uqy7unptPwhC1b7G/S83XgRkREJEPKyhmylz8l5gbqWDvuTgYnIiKygsGQNf6SmBuIY+14KhmciIjIAuXmDNlL7om5xpwhW016/pIz5I1kcCIi8gtMoHYjb32YPmMMIADzIMLfAggmgxMRUQXeen6zmSwQ+EuTni2OJIMTERG5iXJ7kwWa1FRg6FB5N+nZEqjJ4EREJGsMhgJJVcMF+AN3JoPLPdeLiIhkg81kJB/umniVvdGIiMgBDIZIPtwxvpMxmfz23KPTpw3LGRAREdFtGAyRvLiSDK7TGeacs9RB0rhsyhQO4EhERGaYM0Ty42wyOKcmISIiJzAYInlyJhmcvdGIiMgJbCajwBGIU5MQEZHHMRiiwOGu3mhERKQoDIYocLijNxoRESkOgyEKLIEyNQkREXkNE6gp8ATC1CREROQ1DIYoMPn71CREROQ1bCYjIiIiRWMwRERERIrGZjLObk5ERKRoyg6GsrIMc1lVnMIhPt7QPZu9joiIiBRBuc1knN2ciIiIoNRgiLObExER0U3KDIYcmd2ciIiIApoygyHObk5EREQ3KTMY4uzmREREdJMygyHObk5EREQ3KTMY4uzmREREdJMygyGAs5sTERERAD8IhubMmYMuXbqgVq1aiIqKwrBhw3Do0CH3HDw1FTh+HMjOBpYvN/ybl8dAiIiISEFkPwL1pk2bMHHiRHTp0gXl5eV4+eWXMWDAAPz++++oUaOG6yfg7OZERESKJglhaeRB+Tp//jyioqKwadMm9O7d2659ioqKEBERAa1Wi9q1a3u4hEREROQO3np+y75m6HZarRYAEBkZaXWbsrIylJWVmd4XFRV5vFxERETkn2SfM1SREAJpaWno1asX2rZta3W7OXPmICIiwvRKSEjwYimJiIjIn/hVM9nEiROxfv16/PTTT4iPj7e6naWaoYSEBDaTERER+RE2k91m0qRJWLduHTZv3lxlIAQAKpUKKpXKSyUjIiIifyb7YEgIgUmTJmH16tXIyclBYmKir4tEREREAUT2wdDEiROxfPlyrF27FrVq1UJhYSEAICIiAtWqVfNx6YiIiMjfyT5nSLIyf9jSpUsxfvx4u47BrvVERET+hzlDN8k8ViMiIiI/51dd64mIiIjcjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGi+U0w9OGHHyIxMRHh4eHo1KkTtmzZ4usiERERUQDwi2Bo5cqVmDJlCl5++WXs2bMHKSkpuP/++3Hy5ElfF42IiIj8nCSEEL4uhC133XUXOnbsiH//+9+mZUlJSRg2bBjmzJljc/+ioiJERERAq9Widu3aniwqERERuYm3nt+yrxm6fv06du3ahQEDBpgtHzBgALZu3eqjUhEREVGgCPF1AWy5cOECdDodGjZsaLa8YcOGKCwstLhPWVkZysrKTO+1Wi0AQ4RJRERE/sH43PZ0I5bsgyEjSZLM3gshKi0zmjNnDmbNmlVpeUJCgkfKRkRERJ5z8eJFREREeOz4sg+G6tevj+Dg4Eq1QOfOnatUW2SUnp6OtLQ00/srV66gcePGOHnypEc/TLkpKipCQkIC8vPzFZUrxevmdSsBr5vXrQRarRaNGjVCZGSkR88j+2AoLCwMnTp1woYNGzB8+HDT8g0bNmDo0KEW91GpVFCpVJWWR0REKOpHZFS7dm1et4LwupWF160sSr3uoCDPpjjLPhgCgLS0NIwbNw6dO3dG9+7dsWTJEpw8eRJPP/20r4tGREREfs4vgqGHHnoIFy9exOuvv46CggK0bdsWX3/9NRo3buzrohEREZGf84tgCAAmTJiACRMmOLWvSqXCjBkzLDadBTJeN69bCXjdvG4l4HV79rr9YtBFIiIiIk+R/aCLRERERJ7EYIiIiIgUjcEQERERKRqDISIiIlI0vwyGPvzwQyQmJiI8PBydOnXCli1bqtx+06ZN6NSpE8LDw9G0aVMsXry40jaZmZlo3bo1VCoVWrdujdWrV3uq+E5z5LqzsrJw7733okGDBqhduza6d++O7777zmybZcuWQZKkSq/S0lJPX4pDHLnunJwci9f0xx9/mG0XaN/3+PHjLV53mzZtTNv4w/e9efNmDBkyBLGxsZAkCWvWrLG5TyDc345ed6Dc345ed6Dc345ed6Dc33PmzEGXLl1Qq1YtREVFYdiwYTh06JDN/bxxj/tdMLRy5UpMmTIFL7/8Mvbs2YOUlBTcf//9OHnypMXt8/Ly8MADDyAlJQV79uzBSy+9hOeeew6ZmZmmbbZt24aHHnoI48aNw759+zBu3DiMGjUKv/zyi7cuyyZHr3vz5s2499578fXXX2PXrl3o168fhgwZgj179phtV7t2bRQUFJi9wsPDvXFJdnH0uo0OHTpkdk3Nmzc3rQvE73vhwoVm15ufn4/IyEiMHDnSbDu5f9/FxcVo164dFi1aZNf2gXJ/O3rdgXJ/O3rdRv5+fzt63YFyf2/atAkTJ07E9u3bsWHDBpSXl2PAgAEoLi62uo/X7nHhZ7p27Sqefvpps2WtWrUS06dPt7j9P/7xD9GqVSuzZU899ZTo1q2b6f2oUaPEfffdZ7bNwIEDxejRo91Uatc5et2WtG7dWsyaNcv0funSpSIiIsJdRfQIR687OztbABCXL1+2ekwlfN+rV68WkiSJ48ePm5b5w/ddEQCxevXqKrcJlPu7Inuu2xJ/vL8rsue6A+X+rsiZ7zsQ7m8hhDh37pwAIDZt2mR1G2/d435VM3T9+nXs2rULAwYMMFs+YMAAbN261eI+27Ztq7T9wIEDsXPnTty4caPKbawd09ucue7b6fV6XL16tdJkd9euXUPjxo0RHx+PwYMHV/rL0pdcue4OHTogJiYG99xzD7Kzs83WKeH7/vjjj9G/f/9Ko7TL+ft2RiDc3+7gj/e3K/z5/naHQLm/tVotAFQ5Cau37nG/CoYuXLgAnU5Xabb6hg0bVprV3qiwsNDi9uXl5bhw4UKV21g7prc5c923e/fdd1FcXIxRo0aZlrVq1QrLli3DunXrkJGRgfDwcPTs2RNHjhxxa/md5cx1x8TEYMmSJcjMzERWVhZatmyJe+65B5s3bzZtE+jfd0FBAb755hs8+eSTZsvl/n07IxDub3fwx/vbGYFwf7sqUO5vIQTS0tLQq1cvtG3b1up23rrH/WY6jookSTJ7L4SotMzW9rcvd/SYvuBsGTMyMjBz5kysXbsWUVFRpuXdunVDt27dTO979uyJjh074v3338e//vUv9xXcRY5cd8uWLdGyZUvT++7duyM/Px/vvPMOevfu7dQxfcXZMi5btgx16tTBsGHDzJb7y/ftqEC5v53l7/e3IwLp/nZWoNzfzz77LHJzc/HTTz/Z3NYb97hf1QzVr18fwcHBlaK9c+fOVYoKjaKjoy1uHxISgnr16lW5jbVjepsz1220cuVKPPHEE/jiiy/Qv3//KrcNCgpCly5dZPOXhCvXXVG3bt3MrimQv28hBP773/9i3LhxCAsLq3JbuX3fzgiE+9sV/nx/u4u/3d+uCJT7e9KkSVi3bh2ys7MRHx9f5bbeusf9KhgKCwtDp06dsGHDBrPlGzZsQI8ePSzu071790rbf//99+jcuTNCQ0Or3MbaMb3NmesGDH8xjh8/HsuXL8egQYNsnkcIgb179yImJsblMruDs9d9uz179phdU6B+34Cht8bRo0fxxBNP2DyP3L5vZwTC/e0sf7+/3cXf7m9X+Pv9LYTAs88+i6ysLGzcuBGJiYk29/HaPW53qrVMrFixQoSGhoqPP/5Y/P7772LKlCmiRo0apqz66dOni3Hjxpm2//PPP0X16tXF888/L37//Xfx8ccfi9DQUKHRaEzb/PzzzyI4OFi89dZb4uDBg+Ktt94SISEhYvv27V6/Pmscve7ly5eLkJAQ8cEHH4iCggLT68qVK6ZtZs6cKb799ltx7NgxsWfPHvH444+LkJAQ8csvv3j9+qxx9Lrfe+89sXr1anH48GHx22+/ienTpwsAIjMz07RNIH7fRo888oi46667LB7TH77vq1evij179og9e/YIAGL+/Pliz5494sSJE0KIwL2/Hb3uQLm/Hb3uQLm/Hb1uI3+/v5955hkREREhcnJyzH63JSUlpm18dY/7XTAkhBAffPCBaNy4sQgLCxMdO3Y065b32GOPiT59+phtn5OTIzp06CDCwsJEkyZNxL///e9Kx1y1apVo2bKlCA0NFa1atTK7ueTCkevu06ePAFDp9dhjj5m2mTJlimjUqJEICwsTDRo0EAMGDBBbt2714hXZx5Hrnjt3rrjjjjtEeHi4qFu3rujVq5dYv359pWMG2vcthBBXrlwR1apVE0uWLLF4PH/4vo1dp639bgP1/nb0ugPl/nb0ugPl/nbmdx4I97elawYgli5datrGV/e4dLOARERERIrkVzlDRERERO7GYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQEfmljIwMhIeH4/Tp06ZlTz75JJKTk6HVan1YMiLyN5ybjIj8khAC7du3R0pKChYtWoRZs2bhP//5D7Zv3464uDhfF4+I/EiIrwtAROQMSZIwe/ZsqNVqxMbGYuHChdiyZQsDISJyGGuGiMivdezYEQcOHMD333+PPn36+Lo4ROSHmDNERH7ru+++wx9//AGdToeGDRv6ujhE5KdYM0REfmn37t3o27cvPvjgA6xYsQLVq1fHqlWrfF0sIvJDzBkiIr9z/PhxDBo0CNOnT8e4cePQunVrdOnSBbt27UKnTp18XTwi8jOsGSIiv3Lp0iX07NkTvXv3xv/7f//PtHzo0KEoKyvDt99+68PSEZE/YjBEREREisYEaiIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGi/X9p9Axbjf6uLAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_16_2.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients for OLS\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "623ad923",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Same code but now with momentum gradient descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b7fe53c1",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.]\n",
      " [3.]]\n",
      "Eigenvalues of Hessian Matrix:[0.27335131 4.46000649]\n",
      "0 [-13.27099835] [-15.86570776]\n",
      "1 [0.01163425] [-0.00974702]\n",
      "2 [0.01092119] [-0.00914964]\n",
      "3 [0.01025184] [-0.00858886]\n",
      "4 [0.00962351] [-0.00806245]\n",
      "5 [0.00903369] [-0.00756831]\n",
      "6 [0.00848002] [-0.00710445]\n",
      "7 [0.00796028] [-0.00666902]\n",
      "8 [0.0074724] [-0.00626028]\n",
      "9 [0.00701442] [-0.00587659]\n",
      "10 [0.00658451] [-0.00551642]\n",
      "11 [0.00618095] [-0.00517832]\n",
      "12 [0.00580212] [-0.00486095]\n",
      "13 [0.00544651] [-0.00456302]\n",
      "14 [0.0051127] [-0.00428336]\n",
      "15 [0.00479935] [-0.00402083]\n",
      "16 [0.0045052] [-0.0037744]\n",
      "17 [0.00422908] [-0.00354307]\n",
      "18 [0.00396988] [-0.00332591]\n",
      "19 [0.00372657] [-0.00312207]\n",
      "20 [0.00349817] [-0.00293072]\n",
      "21 [0.00328377] [-0.0027511]\n",
      "22 [0.00308251] [-0.00258249]\n",
      "23 [0.00289358] [-0.00242421]\n",
      "24 [0.00271624] [-0.00227563]\n",
      "25 [0.00254976] [-0.00213616]\n",
      "26 [0.00239349] [-0.00200523]\n",
      "27 [0.00224679] [-0.00188233]\n",
      "28 [0.00210909] [-0.00176697]\n",
      "29 [0.00197982] [-0.00165867]\n",
      "theta from own gd\n",
      "[[4.00679887]\n",
      " [2.994304  ]]\n",
      "0 [0.00185848] [-0.00155701]\n",
      "1 [0.00174457] [-0.00146158]\n",
      "2 [0.00160348] [-0.00134337]\n",
      "3 [0.00146287] [-0.00122558]\n",
      "4 [0.00133103] [-0.00111512]\n",
      "5 [0.0012099] [-0.00101364]\n",
      "6 [0.00109941] [-0.00092107]\n",
      "7 [0.00099888] [-0.00083685]\n",
      "8 [0.0009075] [-0.00076029]\n",
      "9 [0.00082447] [-0.00069073]\n",
      "10 [0.00074902] [-0.00062752]\n",
      "11 [0.00068048] [-0.0005701]\n",
      "12 [0.00061822] [-0.00051793]\n",
      "13 [0.00056165] [-0.00047054]\n",
      "14 [0.00051025] [-0.00042748]\n",
      "15 [0.00046356] [-0.00038836]\n",
      "16 [0.00042114] [-0.00035283]\n",
      "17 [0.0003826] [-0.00032054]\n",
      "18 [0.00034759] [-0.00029121]\n",
      "19 [0.00031579] [-0.00026456]\n",
      "20 [0.00028689] [-0.00024035]\n",
      "21 [0.00026064] [-0.00021836]\n",
      "22 [0.00023679] [-0.00019838]\n",
      "23 [0.00021512] [-0.00018022]\n",
      "24 [0.00019544] [-0.00016373]\n",
      "25 [0.00017755] [-0.00014875]\n",
      "26 [0.0001613] [-0.00013514]\n",
      "27 [0.00014654] [-0.00012277]\n",
      "28 [0.00013313] [-0.00011154]\n",
      "29 [0.00012095] [-0.00010133]\n",
      "theta from own gd wth momentum\n",
      "[[4.00040199]\n",
      " [2.99966322]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients for OLS\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x#+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 30\n",
    "\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(theta)\n",
    "    theta -= eta*gradients\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "# Now improve with momentum gradient descent\n",
    "change = 0.0\n",
    "delta_momentum = 0.3\n",
    "for iter in range(Niterations):\n",
    "    # calculate gradient\n",
    "    gradients = training_gradient(theta)\n",
    "    # calculate update\n",
    "    new_change = eta*gradients+delta_momentum*change\n",
    "    # take a step\n",
    "    theta -= new_change\n",
    "    # save the change\n",
    "    change = new_change\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"theta from own gd wth momentum\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "749000b3",
   "metadata": {
    "editable": true
   },
   "source": [
    "## But noen of these can compete with Newton's method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e7b4abe1",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.71369789]\n",
      " [3.2314999 ]]\n",
      "Eigenvalues of Hessian Matrix:[0.30237154 4.4642383 ]\n",
      "0 [-17.75091492] [-21.33108943]\n",
      "1 [-4.60742555e-15] [5.64228618e-16]\n",
      "2 [-5.34294831e-16] [-5.79981535e-16]\n",
      "3 [-5.34294831e-16] [-5.79981535e-16]\n",
      "4 [-5.34294831e-16] [-5.79981535e-16]\n",
      "beta from own Newton code\n",
      "[[3.71369789]\n",
      " [3.2314999 ]]\n"
     ]
    }
   ],
   "source": [
    "# Using Newton's method\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "beta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(beta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "# Note that here the Hessian does not depend on the parameters beta\n",
    "invH = np.linalg.pinv(H)\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "beta = np.random.randn(2,1)\n",
    "Niterations = 5\n",
    "\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(beta)\n",
    "    beta -= invH @ gradients\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"beta from own Newton code\")\n",
    "print(beta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d47c2b1c",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Including Stochastic Gradient Descent with Autograd\n",
    "In this code we include the stochastic gradient descent approach discussed above. Note here that we specify which argument we are taking the derivative with respect to when using **autograd**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e2b04ba9",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.39917327]\n",
      " [2.69542733]]\n",
      "Eigenvalues of Hessian Matrix:[0.29765192 4.0375827 ]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own gd\n",
      "[[4.39917327]\n",
      " [2.69542733]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHFCAYAAADxOP3DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABUgklEQVR4nO3deXhTVf4/8He6s7VQli60tGUrrci+Q4FCAyIgWBEFRWVcUFCHQR1hnO8AOsjozwUVFVEHXAbcWpDBbRJoWQSU1a1ltUCBlrKmUGhpk/P7IzY0bdIsTXLvzX2/nicP5N6bm3Nvcns/OedzztEIIQSIiIiIVCpA6gIQERERSYnBEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEJEMrVy5EhqNxvIICgpCTEwM7rzzThw6dEiyci1YsAAajUay91eCo0ePQqPR4KWXXpK6KETkpCCpC0BE9q1YsQJdunRBeXk5vv/+eyxatAg5OTnYv38/WrRoIXXxiIj8AoMhIhnr2rUr+vTpAwAYPnw4jEYj5s+fj7Vr12L69OkSl46kVFlZaak1JKKGYTMZkYJUB0anT5+2LCsvL8cTTzyBHj16ICIiApGRkRg4cCC+/PLLOq/XaDR49NFH8dFHHyElJQWNGzdG9+7dsX79+jrbfvXVV+jRowdCQ0ORlJRkt9mnvLwc8+bNQ1JSEkJCQtC2bVvMmjULFy9etNouMTER48aNw/r169GzZ080atQIKSkplvdeuXIlUlJS0KRJE/Tr1w+7du1yeD6qmxNzcnLwyCOPoFWrVmjZsiUyMzNx6tSpOse+YMGCOvtITEzEfffdV2efGzduxIMPPoiWLVsiPDwc99xzD8rKylBcXIzJkyejefPmiImJwZNPPonKyso6+zWZTFi0aBHatWuHsLAw9OnTBxs2bKiz3aFDhzB16lS0adMGoaGhSElJwZtvvmm1TW5uLjQaDT766CM88cQTaNu2LUJDQ3H48GFcuXIFTz75JJKSkhAWFobIyEj06dMHq1evdnj+iMiMPymIFKSgoAAA0LlzZ8uyiooKnD9/Hk8++STatm2La9euQa/XIzMzEytWrMA999xjtY+vvvoKO3fuxLPPPoumTZvixRdfxK233ooDBw6gffv2AIANGzZgwoQJGDhwID755BMYjUa8+OKLVkEYAAghMHHiRGzYsAHz5s1DWloafv75Z8yfPx/bt2/H9u3bERoaatn+p59+wrx58/DMM88gIiICCxcuRGZmJubNm4cNGzbg+eefh0ajwdNPP41x48ahoKAAjRo1cnheHnjgAYwdOxarVq1CYWEhnnrqKdx9993YuHGj2+f6gQceQGZmJj755BPs3bsXf/vb31BVVYUDBw4gMzMTDz30EPR6PV544QXExsZizpw5Vq9funQpEhISsGTJEphMJrz44osYM2YMNm3ahIEDBwIA8vLyMGjQILRr1w4vv/wyoqOj8d133+Hxxx/H2bNnMX/+fKt9zps3DwMHDsSyZcsQEBCANm3aYM6cOfjoo4/wz3/+Ez179kRZWRl+/fVXnDt3zu1jJ1IdQUSys2LFCgFA7NixQ1RWVopLly6Jb7/9VkRHR4uhQ4eKyspKu6+tqqoSlZWV4v777xc9e/a0WgdAREVFidLSUsuy4uJiERAQIBYvXmxZ1r9/fxEbGyuuXr1qWVZaWioiIyNFzT8b3377rQAgXnzxRav3+fTTTwUAsXz5csuyhIQE0ahRI3HixAnLsn379gkAIiYmRpSVlVmWr127VgAQ69atc+o8zZw502r5iy++KACIoqIiq2OfP39+nX0kJCSIe++9t84+H3vsMavtJk6cKACIV155xWp5jx49RK9evSzPCwoKBAC75y8jI8OybPTo0SIuLk4YDAarfT766KMiLCxMnD9/XgghRE5OjgAghg4dWqf8Xbt2FRMnTqyznIicx2YyIhkbMGAAgoOD0axZM9x0001o0aIFvvzyyzp5Ip9//jkGDx6Mpk2bIigoCMHBwXj//feRn59fZ5/p6elo1qyZ5XlUVBTatGmDY8eOAQDKysqwc+dOZGZmIiwszLJds2bNMH78eKt9Vde81GxmAoDbb78dTZo0qdMs1KNHD7Rt29byPCUlBYA5H6px48Z1lleXyZFbbrnF6nm3bt1cer0t48aNs3peXaaxY8fWWW7rfeydv82bN8NoNKK8vBwbNmzArbfeisaNG6OqqsryuPnmm1FeXo4dO3ZY7fO2226r8z79+vXDN998g7lz5yI3NxdXr151+5iJ1IrBEJGMffjhh9i5cyc2btyIGTNmID8/H1OmTLHaJjs7G5MnT0bbtm3x8ccfY/v27di5cyf+9Kc/oby8vM4+W7ZsWWdZaGio5SZ64cIFmEwmREdH19mu9rJz584hKCgIrVu3tlqu0WgQHR1dp6kmMjLS6nlISEi9y22V35bax1TdNNeQwMCVstoqp73zd+3aNVy+fBnnzp1DVVUV3njjDQQHB1s9br75ZgDA2bNnrV4fExNTZ5+vv/46nn76aaxduxbp6emIjIzExIkTJR2CgUhpmDNEJGMpKSmWpOn09HQYjUa89957+OKLLzBp0iQAwMcff4ykpCR8+umnVmMAVVRUuPWeLVq0gEajQXFxcZ11tZe1bNkSVVVVOHPmjFVAJIRAcXEx+vbt61YZvCE0NNTmOfFWbo298xcSEoKmTZsiODgYgYGBmDZtGmbNmmVzH0lJSVbPbY3x1KRJEyxcuBALFy7E6dOnLbVE48ePx/79+z1zMER+jjVDRAry4osvokWLFvjHP/4Bk8kEwHyDDAkJsbpRFhcX2+xN5ozq3lzZ2dlWNR6XLl3Cf//7X6ttR44cCcAckNWUlZWFsrIyy3o5SExMxM8//2y1bOPGjbh8+bJX3s/e+UtLS0NgYCAaN26M9PR07N27F926dUOfPn3qPGzV4tUnKioK9913H6ZMmYIDBw7gypUrnj4sIr/EmiEiBWnRogXmzZuHv/71r1i1ahXuvvtujBs3DtnZ2Zg5cyYmTZqEwsJCPPfcc4iJiXG7qeS5557DTTfdBK1WiyeeeAJGoxEvvPACmjRpgvPnz1u202q1GD16NJ5++mmUlpZi8ODBlt5kPXv2xLRp0zx16A02bdo0/N///R/+8Y9/YNiwYcjLy8PSpUsRERHhlfcLDAyEVqvFnDlzYDKZ8MILL6C0tBQLFy60bPPaa69hyJAhSEtLwyOPPILExERcunQJhw8fxn//+1+nesP1798f48aNQ7du3dCiRQvk5+fjo48+wsCBA63ysIjIPgZDRArz2GOPYenSpXj22WcxZcoUTJ8+HSUlJVi2bBn+/e9/o3379pg7dy5OnDhhdeN1hVarxdq1a/H3v/8dd9xxB6KjozFz5kxcvXrVap8ajQZr167FggULsGLFCixatAitWrXCtGnT8Pzzz1t1q5faU089hdLSUqxcuRIvvfQS+vXrh88++wwTJkzwyvs9+uijKC8vx+OPP46SkhLccMMN+OqrrzB48GDLNqmpqdizZw+ee+45/P3vf0dJSQmaN2+OTp06WfKGHBkxYgTWrVuHV199FVeuXEHbtm1xzz334JlnnvHKcRH5I40QQkhdCCIiIiKpMGeIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqqlinCGTyYRTp06hWbNmNoezJyIiIvkRQuDSpUuIjY1FQID36m9UEQydOnUK8fHxUheDiIiI3FBYWIi4uDiv7V8VwVCzZs0AmE9meHi4xKUhIiIiZ5SWliI+Pt5yH/cWVQRD1U1j4eHhDIaIiIgUxtspLkygJiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlWTPBjavHkzxo8fj9jYWGg0Gqxdu9butjNmzIBGo8GSJUt8Vj4iIiLyb5IHQ2VlZejevTuWLl1a73Zr167FDz/8gNjYWB+VjIiIiNQgSOoCjBkzBmPGjKl3m5MnT+LRRx/Fd999h7Fjx/qoZERERKQGktcMOWIymTBt2jQ89dRTuOGGG6QuDhEREfkZyWuGHHnhhRcQFBSExx9/3OnXVFRUoKKiwvK8tLTUG0UjIiIiPyDrmqHdu3fjtddew8qVK6HRaJx+3eLFixEREWF5xMfHe7GUREREpGSyDoa2bNmCkpIStGvXDkFBQQgKCsKxY8fwxBNPIDEx0e7r5s2bB4PBYHkUFhb6rtBERESkKLJuJps2bRoyMjKslo0ePRrTpk3D9OnT7b4uNDQUoaGh3i4eERER+QHJg6HLly/j8OHDlucFBQXYt28fIiMj0a5dO7Rs2dJq++DgYERHRyM5OdnXRSUiIiI/JHkwtGvXLqSnp1uez5kzBwBw7733YuXKlRKVioiIiNRC8mBo+PDhEEI4vf3Ro0e9VxgiIiJSHVknUBMRERF5G4MhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqiZ5MLR582aMHz8esbGx0Gg0WLt2rWVdZWUlnn76adx4441o0qQJYmNjcc899+DUqVPSFZiIiIj8iuTBUFlZGbp3746lS5fWWXflyhXs2bMH//d//4c9e/YgOzsbBw8exC233CJBSYmIiMgfaYQQQupCVNNoNFizZg0mTpxod5udO3eiX79+OHbsGNq1a+fUfktLSxEREQGDwYDw8HAPlZaIiIi8yVf37yCv7dlLDAYDNBoNmjdvbnebiooKVFRUWJ6Xlpb6oGRERESkRJI3k7mivLwcc+fOxdSpU+uNEBcvXoyIiAjLIz4+3oelJCIiIiVRTDBUWVmJO++8EyaTCW+99Va9286bNw8Gg8HyKCws9FEpiYiISGkU0UxWWVmJyZMno6CgABs3bnTYbhgaGorQ0FAflY6IiIiUTPbBUHUgdOjQIeTk5KBly5ZSF4mIiEhaRiOwZQtQVATExABpaUBgoNSlUizJg6HLly/j8OHDlucFBQXYt28fIiMjERsbi0mTJmHPnj1Yv349jEYjiouLAQCRkZEICQmRqthERETSyM4G/vxn4MSJ68vi4oDXXgMyM6Url4JJ3rU+NzcX6enpdZbfe++9WLBgAZKSkmy+LicnB8OHD3fqPdi1noiI/EJ2NjBpElD71q3RmP/94gu/Coh8df+WPBjyBQZDRESkeEYjkJhoXSNUk0ZjriEqKPCbJjNf3b8V05uMiIhI1bZssR8IAebaosJC83bkEgZDRERESlBU5NntyILBEBERkRLExHh2O7JgMERERKQEaWnmnKDqZOnaNBogPt68HbmEwRAREZESBAaau88DdQOi6udLlvhN8rQvMRgiIiJSisxMc/f5tm2tl8fF+V23el+SfNBFIiIickFmJjBhAkeg9iAGQ0REREoTGAg4OfAwOcZmMiIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWNvMiIikh+jkV3HyWcYDBERkbxkZwN//rP1DO1xcebRlzmoIHkBm8mIiEg+srOBSZOsAyEAOHnSvDw7W5pykV9jMERERPJgNJprhISou6562ezZ5u2IPIjBEBERycOWLXVrhGoSAigsNG9H5EEMhoiISB6Kijy7HZGTmEBNRETyEBPj2e3IGnvo2cWaISIikoe0NHOvMY3G9nqNBoiPN29HrsnOBhITgfR0YOpU87+JiUxI/wODISIikofAQHP3eaBuQFT9fMkS1ma4ij30HGIwRERE8pGZCXzxBdC2rfXyuDjzco4z5Br20HMKc4aIiEheMjOBCROY3+IJrvTQGz7cZ8WSGwZDREQkP4GBqr45ewx76DmFzWRERET+ij30nMJgiIiIyF+xh55TGAwRERH5K/bQcwqDISIiIn/GHnoOMYGaiIjI37GHXr0YDBEREakBe+jZxWCIiIiIfE9Gc6UxGCIiIiLfys42j4xdc0DIuDhzsrcEOUxMoCYiIiLfkeFcaQyGiIiIyDdkOlcagyEiIiI1MhqB3Fxg9Wrzv74IQFyZK82HmDNERESuk1HyqySUfvxS5ezIdK401gwREXmbFL/AvSk7G0hMBNLTgalTzf8mJkqS6yEJpR+/lDk7Mp0rTfJgaPPmzRg/fjxiY2Oh0Wiwdu1aq/VCCCxYsACxsbFo1KgRhg8fjt9++02awhIRuUrpN87aZJj86lW1A9nPP1f28UudsyPTudIkD4bKysrQvXt3LF261Ob6F198Ea+88gqWLl2KnTt3Ijo6GlqtFpcuXfJxSYmIXORvgYPUN1JfsxXITpmi7OOXOmdHpnOlSR4MjRkzBv/85z+RaaONUgiBJUuW4JlnnkFmZia6du2KDz74AFeuXMGqVaskKC0RkZP8MXCQ+kbqS/YC2fo+LyUcvxxydmQ4V5rkwVB9CgoKUFxcjFGjRlmWhYaGYtiwYdi2bZvd11VUVKC0tNTqQUTkU/4YOMjhRuoL9QWyzpDz8cslZyczEzh6FMjJAVatMv9bUCDZpLGy7k1WXFwMAIiKirJaHhUVhWPHjtl93eLFi7Fw4UKvlo2IqF7+GDjI5UbqbY4CWUfkfPzVOTsnT9oO9jQa83pf5OzIaK40WdcMVdPUalcUQtRZVtO8efNgMBgsj8LCQm8XkYjImj8GDjJNfvU4dwNUJRy/THN2pCbrYCg6OhrA9RqiaiUlJXVqi2oKDQ1FeHi41YOIyKf8MXBQy43UnQBVSccvw5wdqck6GEpKSkJ0dDR0Op1l2bVr17Bp0yYMGjRIwpIRETngr4GDGm6kjgJZoO7nprTjl1nOjtQkzxm6fPkyDh8+bHleUFCAffv2ITIyEu3atcPs2bPx/PPPo1OnTujUqROef/55NG7cGFOnTpWw1ERETqgOHGyN9LtkiXJvPJmZwIQJyh6BuT7VgeykSeaAqGZuTXWA9MknQKtWyj5+GeXs1CZMAr+uOYT1/z7kk/fTCOFuurxn5ObmIj09vc7ye++9FytXroQQAgsXLsQ777yDCxcuoH///njzzTfRtWtXp9+jtLQUERERMBgMbDIjIt9T+tQNamVryor4eGUHsjJ2clcR9O8cgU6vgf54J5w2tQFQCsD792/JgyFfYDBERERuYSDrNaUnSrHpnf3QrbsK/f445F/rYLW+Ea5gcOQP0J8f4fX7t+TNZERERLJVuympenoOBkcuq7xSiR8/yIf+s/PQ7Y7ED5dSUIV+lvUBMKJPk3xkdD8L7e3NMfBPKahAb0REeL9sDIaIiIicIdVM7wolTAL7v/4d+pUnoNsahtzTKbiEblbbdAw+ioxOx6EdG4L0R7qgRZJ1CkxFaYVPyspgiIiIyJHq6TlqZ5ZUzzOnpJ5kXnT61zPQv3UQuv8J6As64KSpA4DrzV8tNecwMu4AtOlGZDyYhMQhiQASJSrtdcwZIiIiqo/RaJ6w1d6o1NWjNhcUqK7JrKykDJuX5UG/rgy632LxS3lnq/WhKEda5G/I6HcJ2ruj0OOOZAQEOT+qj6/u36wZIiIiqo8r88zJtKu6pxivGbHro3zoPzkL3c7m2GZIRSX6Wm3Tq1E+Mm48De1t4Rj8QAoaRfaWqLTOYzBERERUH3+cZ85JwiRwJOc4dO8dw8ZNgbhadAHNcAlFiMEW3AgTApEQeALajr8jY3QQRs5MRqvkFAApUhfdJQyGiIjIMTV3MffHeebqcfbAOWx46wD031VBd7g9jhkTcCt24zX8GfG4XkN2uVFrXPrrc4j+x0PQBMRJWOKGY84QERHVT+29qKpzhhzN9K7QnKGr569i6/I86Ndcgu6XKOy9al2rczs+w6e4E4CA1QQl1aNxezF53Ff3bwZDRERkn71eVD64EcpK9XkAbE/PoaDzYKoyYd+nB6D7+DT0PzbDlvM3oAJhVtvcGHYQ2htOIWNcGEYvn4SAopO2d+blQJDBkAcxGCIicgN7UVlT8PQcR7eegG7579DnBmHDiWScEy2t1scGFEGbdATaURqMfLgToru1Ma/IzQVsTJlVR06OV5LH2ZuMiIikxV5U1hQ0Qe2FgovIeXs/dF9dg+5gAo5UJQC4ntfTDKUYHpUP7ZByZNwXhy43t4cmwEbOk0qSxxkMEREpkS8SmlVyI3SJTGd6ryitwPZ/50P3+UXof2qFXWUpMGGAZX0gqjCgWR4yep2H9o5I9Ls3BcGN+zvesUqSxxkMEREpja8SmlVyI1QiYRL4JfsQdB+cgn5HE2w+m4or6GG1TUrIEWR0OQHtLY0wbEYXhMd1s72z+qSlmb9bjpLH09LcOxCZYM4QEZGS+DKh2c97USnNiZ1F0L9zBDq9BvrjnVEiWlutjwooQUa7Q9BmCIx8qAPi+nooSJUweZwJ1B7EYIiI/IIUCc1+1ItKaUpPlCL37Xzo15dDtz8O+691sFrfGGUY1joPGQPLoL0nFl1v7QRNgMbO3hpIouRxBkMexGCIiPyCVD17FNyLSkkqr1Tih5X50H92Hro9kfjhUiqMNbJZAmBE3yb5yOhxFtrbm2PA9BSEhof6roASDLzJ3mRERGRNqoRmZ3tRqXmUajcIk8D+r3+HbsUJ6LY2Qm5JCi7DOq+nY/BRaDsfh3ZsCIY/3AUtkrpKVFrINnncExgMEVH9fHWD443UMSkTmh3dCNU+SrWTin8ugf7tQ9DrBPQFHXDS1AHA9eavlppzyIg7gIx0IzIeTELikEQAiRKVVj3YTEZE9vnqBscbqXPkmtDMUartKispw6a386BfVwbdb23xa0Unq/WhKEda5G/Q9r+EjLui0OOOZAQEBUhUWvlhzpAHMRgicoOvbnC8kbpGbgnNvkjqVlCtYVV5FXb/Zz90q89Ct6s5thtSUYkQy3oNTOjZ6AC03U4jIzMcgx9IQaPIRhKWWN4YDHkQgyEiF/mq1xKne3CPnBKanU3q1uvNn6GrAY3Maw2FSeDwhmPQvX8M+s2h2FiUAgMirLZJCDwBbcffoR0ThBEPJ6NVcks7e6PaGAx5EIMhIhf5qteSxPMeKZpcaktWrwamTnW8XWQkcP789efOBDQyrTU8k38WG94+CP13RuiOJOG4Mc5qfXPNRYyI2Q/tsGvIuD8BHdLbea/Lu59jbzIiko6vei1xugf3yaVnj7PJ2jUDIcCc9zRpkv2Axmg01wjZ+r0uhDkgmj3b3MvNy0Hg1fNXsXV5HnTZl6D/NQp7r6YAaGVZH4xrGNz8N2j7GpAxpTV639UFgSED7O+QZIfBEBHV5ateS5zuQfkcTddgj6OARsJJYk1VJuz95AB0H5+G/sdwbL2Qigr0ttqmW9gBZNxQBO3Epkh7KAVN2vT0aBnItxgMEVFdvpqPSCXzHvm1wEBzc9ekSebPy9WAyF5A4+Naw4LNhdC/VwBdTjA2nEzGeZECIMWyvm1AEbTtD0M7KgAjH+mMqK7JAJI98t6qJ4MmXwZDRFRXfTe46nyNJUsa/gfLW+8jgz+uqpKZaW7uqp3o3LIlcO6c49fbCmi8XGt4/sgF5CzbD91XldAfSsCRqgQA8Zb1zVCK9Oh8aIeUI2N6PJJvSoImQKY1lEr+vsslQV6ogMFgEACEwWCQuihEypKVJURcnBDmMMX8iI83L5fr+9jaV1yc58tMdVVVCZGTI8SqVeZ/9Xrrz8HeIyfH9r7i4oTQaGy/RqMxf0eqqpwqWrmhXGx8eY+YNzBH9G3yq9DAaLW7IFwTQ8L3iQXDc8T3y34W18quefLMeI+Sv+9ZWbY/X43G/MjK8tn9m73JiKh+ShqBWsreR0r+de4tDR0ksgFjKpmqTPgl+xD0HxVBt70pNp9LRQVCkYYtiEERihCDcyExGJFSDO0tjTDs4RQ0i23WsOP1NZn2tnOKk8NqlP70EyIiI71///ZqqCUTrBkiUoHqmgR7tQ8u1iS4RIm/zmvX4njjvAhx/dd/7RqAGr/+Hb7eyVrD4ztOin9P3yymJGwVbTQlVi+5FVniJGI8+xn56hzae2+pvu+ekJPjVK2hYf16n9y/GQwRkX9w8o+rzSaZhnCiql92fB28NbQZ1E7QcfHYRbF23g4x68ZckRxypM5H0BiXxZjWP4p1fRcKEzTC5MnPSOoAWKrvu6esWuVcMPTeez65fzOBmoj8gxRjFjk7Fs64ccC2bfJoQrPXtOJo3J+GcHbWe3v+GFOp8koldvw7D/r5W6Db0xI/Xk6BEf0tmwXAiL5N8qHteRYZk5pj4P2pCGnUC0jMBODgM3JlvCIpzmFtSh+jy9nE9+ho75bjDwyGiMg/SDFmkbNj4bRtC5w9e325VNNJSDmQoRuDRAqTQP76I9CtPAnd942wqSQFl9HdaptOwQXQJh9Hxs2hSJ+ZguYJXa13kpvr2fGK5DIYpNLH6HJ2WI1Bg3xSHAZDROQfpBizyNlf3TUDIcC3NQg1STiQobOK9p2Gftkh6HWA/mgHnDJ1BNDRsr6V5ixGxh2EdoQRGQ8mIWFwEoCkenbo4RoUuZxDpY/R5avhO5zEYIiI/IMUf1zd/dXtyxqEmmTYtHK5+DI2v5MP3ZdXoMuLxW8VnQBEWdaH4SrSIvOgHXAJGXdHo/vtnREQ5EJtgadrUORyDmUWTLjF3vhUcXHXJx0uLfVJURgMEZH/cOaPqye5OxUFIE0tjAyaVqrKq7Dr4/3QrT4L/e7m2G5IRSX6WtZrYEKvxvuRcWMJtLeFY/CDqQhr3ruePTrgzGfkSg2KDM6hha+/797Q0HwyD+E4Q0Tkf3w55o+9sXCctWoVMGWKZ8tkT0PH/XGDMAkc0h2F7v3j0G8NRU5RCgyIsNomMagQ2o4F0I4JRvqMzmiV3NIj723h6DNq2RJYvty54EGCc+hUmfx0jCtf3b8ZDBERNZStKQVatwbOnHH82pwc3+bnNGAgQ6cYjbj48Xrkrd6Hrb+2wNtFE3DUlGC1SQvNBYyI3Q/tsEpk3J+ADiMS7OzMg7KzgYcesj09iKvH7u1zSBYMhjyIwRAReV3tX+eDBgEdOsirBqGareAtPt7tppUrZ69g67v5KHr/K4w+8jaiUWxZV4g4PIGXcLZ5Z2T0NUB7Vxv0mpKMwBAfH7PRCCQkmD8PW1z9PDx8Dsk2BkN/qKqqwoIFC/Cf//wHxcXFiImJwX333Ye///3vCAgIcGofDIaISBJyrkFoQNOK8ZoRez85AP2qEuh+DMf2C8l4Ai/jWcwHAGhqbCsAQKOBRuraktxcID3d8Xau1NT5cfOUXPjq/i37BOoXXngBy5YtwwcffIAbbrgBu3btwvTp0xEREYE///nPUhePiMg+OSe4ujjuz++5x6F/7yh0ucHYeCoZ50UqgFTcimwcwC2Ih+0aF0tg5Ouec7V5oxeYG2MnkTzJPhjavn07JkyYgLFjxwIAEhMTsXr1auzatUvikhEROUEmvWVcdf7IBWx8az90X1dCfzgBv1clAGhnWR8OA+Y1fxtPX5zneGcyGL9IVr3ASHZkHwwNGTIEy5Ytw8GDB9G5c2f89NNP2Lp1K5YsWSJ10YiInOOJGgRXm2Rc3L6itALfv5sHfZYBup/aYPeVLhAYaFkfhEoMCM+DtvcFaO9sib53dUZQlzeBiy4cg5RTQ3hqkEI2jfkl2QdDTz/9NAwGA7p06YLAwEAYjUYsWrQIU+rpilpRUYGKigrL81IfDdpERCrl6AbZ0BuorWTd+qb0cGJ7U5UJP2cdgv6jIuh2NMWWc6m4ip5Wu0kNPQxtyklkjG+EYQ+noFlsjakwHE1zYYuUtS6eGKTQ1c+BlMOr08B6wOrVq0VcXJxYvXq1+Pnnn8WHH34oIiMjxcqVK+2+Zv78+QLmvD2rB2etJ1IIO7OUy5Kj2csbOrt5VpZ5ZnV7M3svXGh9fuxtr9EIEzRCn/6cuLPd96K1pqTOJtEBxeLupC3igwe3iJO7i+ovl5Ozjle/t4iPl8fnaOvziI93/HnUc16FRuO72epVxmAw+OT+LftgKC4uTixdutRq2XPPPSeSk5Ptvqa8vFwYDAbLo7CwkMEQkVI0NHjwJUc3yKeeatgNtKqq7rmw9ag+Pw62N0IjjiFeBKBKAEI0wSVxc+sfxasTc8Uv2QeFyWhy/thzclwLhuT0+bkabDv6HOQU7PkZXwVDsm8mu3LlSp0u9IGBgTCZTHZfExoaitDQUG8XjYg8rboreu2cDqkmNq2Po9nLAeCVVxo2u7mjSUGrnTgBTJqEqr/9H4Lq2T4AAu1QiHdvfAMdH0zHgOkpCGna1+729XJ2KhI5NiO5msMll8lZyWucG6hHQuPHj8eiRYvw1Vdf4ejRo1izZg1eeeUV3HrrrVIXzb8ZjeacgNWrzf8ajVKXiPydM8HF7Nny+S46E6jUV9aaN1B7XEg4NgkBw6KlTm37p3lRGPpYd4Q0DXF6/3VU5+AA13Nualu4EDh6VF6BkDvkMjkreY3sg6E33ngDkyZNwsyZM5GSkoInn3wSM2bMwHPPPSd10fxXdrZ57p30dGDqVPO/iYnm5UTe4sqvbznw1I2vvv24kHAcAKAlzju3sacSmavHUWrb1np5fDyQlQX84x/+0dOK3fL9nuxHoPYEjkDtAnvNFHIYMZe8T8puw6tXm4NvR3w5sWl9nB3R2JH6Rjw2GmGKbwdN0SnYqXupQ0RGQnPhgm+nAPH37uZynJxVJXx1/5Z9zRD5kNKaKcizpK4RVNqv7+qcGXtNRID5xmhvvUZjrkGpNa5NVXkVti//Bc+OyMXQyF9xR9GrENDAfpZkrd1Wj8xf+32d7T7ujuocnClTzP/6W0BQX5OgN88r+Y5X07NlwlfZ6IrnbO+QnBypS0qeJoduw9U9dux1I5djj53q81a7zLV7k9lbn5UlTEaT2P/1EbH09lwxIXqHCMfFOof+cMAycS6wlfPd193tPk7143n1OV/dv11qJissLER8fLz3IjMvYTOZk5TWTEGeUd0EYC9fx5dNAHKe2NQeR7OX21hvjI7Ftp6zsOJwGvS/t0eh0TrnpoXmAkbG7kfGsEpoH0xE++HtzJ/TokXA/Pl1y2Dr/Ph705VUeF59Spaz1jdp0gRz5szB3Llz0aRJE68VytMYDDnJG7M6k/zJ7XN3FFzIkYMb5JXTl/Dr31bhUM4J5JzohBWVd8GE6+tDUIHBzfOg7WeA9q426HlnMgJD7NxglXh+iNwky2Bo27Zt+Mtf/oLCwkIsWrQI06dP91rBPInBkJOYJKhOcqwRVPivb+M1I/asPgD9qhLofozA9xdTcQ3WY591DzsAbdciZExsirQZqWjcqrELb6Ds8+N3+Hl4jSyDoWoffvghnnnmGbRq1Qqvvvoqhsu8loDBkAuU2ExBDSO3miGFOrLxGPTvH4NuUzA2nuqCC6KF1fq4wFPQtj8C7ehAjHy4E9rc0FqikpJHcb4yr5J1MAQAV69exeLFi/Hyyy9j1KhR+H//7/+hY8eOni6fRzAYchGr4dWFNYJuOXfoPDa+fQD6byqhO5yIgqp2VuvDYUB69H5o08qRMT0enUcnQRPgbAd5UgQOReJ1sg+Grly5gj179iArKwuvv/46goODMWvWLCxYsADNmjXzdDkbhMGQG1jtqy6sEXSo/GI5vn83D/rsUuh+boM9V7pA1BidJAiVGBieB22fC8i4oyX63pOCoDAZzHjEa9k75NTxwI/JMhhatmwZdu7ciZ07dyI/Px+BgYHo1q0bBgwYgB49euA///kPDh48iDVr1qBPnz5eK7SrGAwROYE1glZMVSb8nHUIug+LoNvRDFvOp6Icjay2uSH0EDJSTkE7oTGGPZyCptFNJSqtHWzC8R65Ni/7WfAry2AoPj4eAwYMsDz69OlTZ0LU559/HqtWrcKvv/7q8cK6i8EQ+T1P/QH0sz+krjq+/SR07/wOfU4gNhR2whlhndcTE1CMjMTD0GqBkQ91RGyvaIlK6gQ24XiXHDse+GHwK8tgyBmnT59GbGwsjDIapZjBEPk1T/8BVFFAdPGYATlv5UP/dQV0B9rhUGWS1fomuIzhbfKgHXwVGfe2Rer4Dg3L+/HVuWUTjvfJrWbIT4NfxQZDQghs3rwZw4YN8+RuG4TBEPktT/8B9MNfljVdu3wNO1bkQ/fZBej3tcSPl1OtxvsJRBX6Nc1HRs9z0E5ugf73pTRsZveafHlu5Xaj9kdy6njgx8GvYoMhOWIwRH7J038A/fCXpTAJ/PblYeg+OAn9tibYdCYFZbDO60kO+R0ZyYXQjgvD8Ie7IKJdhO2dNaRWx9fnVo5NOP5ILh0P/Dj49dX9WwZdHYjILVu22A+EAPMf58JC83aO/gA6mqRXozFP0jthgux/WZ7cVYQNy49Apwf0xzqh2NQJQCfL+taaM8hodxAZ6SZkPNQe7Qa2B9C+/p02pFZHinOrtElvlSoz0xzw2Ppu+LLjQVGRZ7dTIQZDRErlyT+AngysfOzSqUvYtCwfunVXod/fFnkVHQFcv8k3whUMbZmHjAGXob0nBjdmdkJA0GDn38Berc7Jk+bljn79S3Fu09LMN2RHTThpaZ55PzXLzDQHslLm2TH4bTAGQ0RK5ck/gFL+snSx+anySiV2frQfuk/OQb+nBXaUpqIK/SzrNTChd+P90HYvgXZSBAb+KQVhzd0c6sMTtTpSnNvAQHOt1aRJ5jLaasJZskT2tXyKERgo7Y8EBr8NxmCISKk8+QdQql+WTjQ/CZPAgW8LoF9RCN3WMOQUp+ASbrTaTfugY9B2OoaMMcEYMbMLIjukAkhtePk8Uasj1bmVSxMOeR+D3wZjAjWRknkqgVOKnjH1JBULAFtG/RMrf0+D7vcOOGGMtdqkheYCRsbuh3Z4JTIeSET74dZTYXiMJxKRpe51pKKhElTPDwdOZW8yD2IwRH7NU38As7OB226zvz4ry3N/UB30hDNBgxOIQxIKYEIgQlCBIS1+g7ZfKTKmtkHPO5MRGOKDG7qneul4u9cRAx6q5mffBfYmIyLnyCGB00XGjbkIrKf5KQAC7VCIN5PfQPt70zDkwRQ0btXLhyX8g6eaIr3ZZOXnY0ORi6TOX1Io1gwRkdcHbRMmgd9zj0P33jHoNwej+cnf8B4edPxCOYyD48laHU//avfDsaGIamLNEBH5jhe6f587dB4b3toP/bdV0B1OwtGqBAAJAIBhqHCuXHLoCuzJWh1P/mr3o7GhnOZnTUAkHwyGiMi17t92bkjlF8vx/bt50GWVQvdzFPZeTYbAIMtLg3ENAyPykNH7IrR3tIB4Ng6aUwrpCizHpkgFjw3lFjYHkhcxGCIi52tgDh2q05xWGtYarwY9hX9dfhTlsM7r6Rp6CBmpp6Cd0BhDZ6SgaXSP6ytbKawrsNxyMfx11GFbwfaXXzZs4EsiB5gzROrBKnb7nOn+HRkJce48AIGa87ab/ng2CV9gR8AgaBMPQztKg5EzOiKmR1T97+uHXYF9xlvzUUl5ndir/bl6FTh3zvZrFDwJKTnGrvUexGCIZFXFLtegzE6icPX/LqI5InARATZeKgBUtYpB0Knj0AS7WOEs1/Mhd66OX+TMeZbyOrGXDO4sBU5CSo757P4tVMBgMAgAwmAwSF0UkkJWlhAajRDmP7PXHxqN+ZGV5duyxMVZlyMuzrdlqMe1jz8VV5tHWZXvGOLF37Gw7vmz9cjJkfoQ1KX6u137+137u+3M907K66Sqqm75XH2sWuW98pFkfHX/tvUjj/yB0WiuRl+92vyv0Sh1iaThqMcNYO5x44vzU/3Lt3bSa3XeQ3a298tQizAJ/JJ1EK9MyMXNbXai+d1j0eTiSQxHDqZgFYYjBzcFb0CH+Crndqi0/BSlq+7p1rat9fK4uOt5NM5876S+ThwlgztDDj0PSbGYQO2P5NQkJDW59LiRUTfok7uKoH/nCHR6DfTHO+G0qTOAzpb1rTVnENsuGNoRcciY0QHx/WOB3BFA+nOOd84bku/V19PN2e9dRIS010lDgmi59TwkRWIw5G/stbvLvdeFt/JG5NLjRsKgrPREKTa9sx+6dVeh3x+H/GsdAFwPWhrhCoa2zIN24GVkTIvBjZmdEBA02HonnBVb3uz1dHP2e5eb69z7eOs6cTeIlmvPQ1IcBkP+REa1Dy7xZk2WVDOG1+bDoKzySiV+/CAf+s/OQ7c7Ej9cSkEV+lnWa2BCnyb50HY/g4zbIjDogVSEhvepf6ecFdsxOSaCezp48dZ14kywHRkJhIWZt6nmielMiAAmUPuVnBzlJbl6O2mzOjHT1ntUv098vHk7b/LiZ2MymkTefw+L12/LFeOjdohmMNTZbYego2JGyibxxZPbxLnD510vf1WVuWyzZwvRurX1zuPjZZMALhm5JsY7+73T66W/TpxJBq/+Hq5aZf7X29ctSc5X928GQ/5k1Spl9bpw1IPEU3+Ane1x400eDsqKfykRHz+yVdzbYYtoG3Cqzu4iNefE7XHbxPJpm8Tvm443rOy2bvStWpkDI96Q5NVbsTZXvndyuE5sfdcYbKsagyEPUk0wpLSaIV+WVw5/ZBtws7l8+rL4euGPYk7vHHFj2IE6pygUV8XIFrvFv27KEbs+yhPGSqNnyyzHG70c+CqgbwhXvndyuE5Y+0M1+Or+zUEX/Ymrg7BJbfVqYOpUx9t5auZyOeR0ODnisvGaEbv/sx+6VWeg29kc2wypqESI1a56NspHRtfT0GY2w5CHUtEospFny+rlmez9grdGgfY0V0b6lsN1QvQHzlpPrlNakquvk5vlMLeUnW7QQhOAIxuOQffeMeg3h2BjURdcFDdYvbRd4AloOxRAe1MgRjzcGa1TUgCkeK+schmWoCG8fWOXS29FR1yZaFYO1wmRjzEYkqOG/AGvHoTNVu8sufW6UGt37T9uNmcPnMPGZQege/h76A63xzFjAoAEy2YRMGBETD4yhlZAe38COo5MgCYgznflVMqN3h5fjLflbKCel2euRZKyloVBDpFdbCaTG0/9AVdKVbed+bAsNVlyHRfJDVfPX8X37+VDl1UK/S9R2Hs1GaLGTF/BuIZBEXnI6HMR2imt0PuuLggKC5Lus1RKE5At9sbb8vT3ylHTdG1qHfyUyE2cqNWDFBMM+eoPuNz46czlpioT9n16ALqPT0P/YzNsPZ+Kcljn9XQNPQTtDSeRcUsTDJ2RgqbRTa13IuVo4krLQavm61wnewG9vfcG/PdaJvIwBkM1nDx5Ek8//TS++eYbXL16FZ07d8b777+P3r17O/V6RQRDak9WVUpNlgNHt56A/t0C6HICseFEMs6JllbrYwOKoE06ggytBiNndERMjyj7O5NDcKzEmruG1Gi5+z20FbTa4+/XMpEHMYH6DxcuXMDgwYORnp6Ob775Bm3atMGRI0fQvHlzqYvmWf6QrNoQCs1nuFBwETlv74fuq2vQH2qHw5WJAK7n9TTFJQxvkw/tkKvIuLctUsZ1gCbAiTwTuYwmrqQctGru5jo1pBauZoLyhg3AP/9pf1t/v5aJFEj2wdALL7yA+Ph4rFixwrIsMTFRugJ5i9KTVVWiorQC2/+dD93nF6H/qRV2laXAhAGW9YGoQv9medD2Oo+MyZHof18Kghv3q2ePdvg6OK6vRsSVnkhy4E4vRU/M6Vcd0PNaJlIc2QdD69atw+jRo3H77bdj06ZNaNu2LWbOnIkHH3zQ7msqKipQUVFheV5aWuqLojaMXObQIivCJPBL9iHoPzoF3bYm2Hw2FVfQw2qbLiFHkJF8AtrxYRj+SArC47o1/I19eUN1pkZESTV3rvZS9HQtHK9lIuXx6pCOHhAaGipCQ0PFvHnzxJ49e8SyZctEWFiY+OCDD+y+Zv78+QJAnYesR6CWyxxaJAp/PCVW3L9FTE3YKqICTtf5KNpoSsTUhK1ixf1bxPEdJ71TCF+Nzu2vI0y7Muqyp881r2Uij+F0HH8IDg4WAwcOtFr22GOPiQEDBth9TXl5uTAYDJZHYWGh/IMhIeQxN5AKGQoN4su/7RCPdcsVXUIO17l3NUKZuKnVj+LlW3LET58fECajyfuF8sUNVQlTSTSEs1NLeGNOP17LRB7hq2BI9s1kMTExSE1NtVqWkpKCrKwsu68JDQ1FaGiot4vmeUpMVlWgyiuV+GFlPvSfnYduTyR+uJQKI/pb1gfAiD5N8pHR/Sy0tzfHwD+lIDS8r28L6YvRxP09ad/ZXCdvNGvxWiZSFNkHQ4MHD8aBAweslh08eBAJCQl2XqFwSktWVQBhEtj/9e/QrTgB3dZGyC1JwWVY5/V0DD6KjE7HoR0bgvRHuqBFUteGv3FDhwvw9g1VDYm+zuQ6eWskdF7LRIoh+2DoL3/5CwYNGoTnn38ekydPxo8//ojly5dj+fLlUhfNe5SUrCpTxXuL8Mv8L3D4h3PYeOZGZIuJMKEDAHPNzy34EiMi96Fjnxa44W8TkDgsEUCi5wrgbjft2gHUhAneu6Ey0dfMm7VwvJaJFEERgy6uX78e8+bNw6FDh5CUlIQ5c+bU25usNkUMuiglPxjwsKykDJvezoN+XRlMP/+KJ6peQDyuByIn0BbvN/0zunW8irHH30LI+dPXX+zpEZ3dHSzR16NNy3GEaSm/i+6MhO4H1w6RnPns/u3VjCSZ8FUCliLZSjKNi5N9gmfl1Uqx471fxHMjc8TQiL0iGBUCEOJWZAkjNMJYK/nVZC8R2dNJre4mJUvVq0tOib5y+C5WVZl7ja1aZf63vuRxOZSXyM/56v6tiJqhhrIZWfIXnTyme3CSMAkc3nAMuvePQb85FBuLUmBAhNU2SQHHsCugL1pUnYHG1TfwVC2IO1NBSD0VixzmhlPQdxGA8spLpFCcm8yD6pxMKSe/lAupb8BOOJN/FhuXHYTuWyN0R5Jw3BhntT4CBoyIyYd22DVoH0hAB83v0Iwc0bA3begM7KtXA1OnOt5u1SpgyhTz/z0xO3xDg3spfxwo4LtoRWnlJVIwzk3mLZ4Ydt8fONut+o03gKgon9wgr56/iq3L86DLvoSNv7RC0/KziEERihCDE4hBMK5hUEQetH0vIuPOVuh9VxcEhV2fCgOrtzW8EA3tOeVOUnJDe3V5IriXMtFXaV38lVZeInJIXcGQHCa/lEvznLM34L/85fr/PVx7ZqoyYe8nB6D/z2nofgjH1gupqEBv3IpsrME0qwToq+FtoFnyKsKm11Pr4oleTw3dhzvdtBvSq8sfgnuldfFXWnmJyKEAqQvgU9u2Of+Lzhuys83V6+np5qaU9HTz8+xsz7+X0Whuflm92vyv0Wi93p2bfvUNtgHlLdhciHfv2YzJ8dvRJuQC+kxLwdxvh2PDhV6oQBju17yPLNyGOFh/To0unUHY/XfX/97VgYjG5Ywh82vi410fS6a26m7a1fus/R7A9W7a1Z/RyZNA69b2y22vbI6Ce8Ac3Nf+7OVGaV38lVZeInLMq+nZMmHJRn/vPc8Pu1/NUS8UX/YWcqaXi6PpHjw0RcP53y+IL57cJmakbBIdgo7W2V0zGMT4qB3i9dtyRd7aA8LU0Okh6usdZev/vvwMak4FYWu9qz3dfDV/mbcpbS4vpZWXSME4N5kHWU7m+vXeuXk4Cj58OQeUK0GXvcChAeeo3FAuNr68R/xtUI7o2+RXEYAqq5cFolIMbvaTmD8sR2x96ydxreza9Rd76uZeXyDy2WdCtGplP0jxJHsBsr3PyNajvrJ5Y04tqcipi78zlFZeIoViMORBlpN5/rznf9E5E3z46he8O0GXszUUdm6wxkqj2PfpfvHSuBwxuuVO0QhldTZNCTksHuuWK9b9/QdhKKznC+3Jm7utQMTWsbZuLcTnnzfotLvE0WdUXaaPP3Y8zo2/1AxVc3ZiVblQWnmJFIjjDHmQVdc8vd6c9wKY/3xVszc+SH0Jz0YjkJBgzvmwpTpZdvFi4O67HRe0Zndrd7jbRbvmMZ4+bZ00bcc3N72Gj/N7Q3+8M0pEa6t1UQElyGh3CNoMgZEPdUBcXydzJzzRxdweuYwL48ljlOMI0g0llw4GzlJaeYkUhiNQe1CdyNLZX3SOmr8WLnTul/mrr/rmF7wnalYc5EMYoRHHEG/V/NUYl8WY1j+Kl2/JET9/cUCYjCb3yu+tXAxvNFO6MlJxTZ5u2mJzDRH5MTaTeZDNk9nQhOennnK+Senjj32TcOmhZpPKVZ8KEzTCZCMQMkIjbsPnYmDjfeK9rq+K36Y9LyrWf+e5ZFFv3Nw93ZzUkGkYvNG0xeYaIvJTDIY8yOWT6UxNQmCg88FQTo5rN3l3ax3crFkxGU3ity8PiSW35opxbX4QTVEqbkWWOA7rc3AuqI3YPmGxuPTWh96dk8nTN3dP1sY0tFegN2u/3PnOEBHJGIMhD3L5ZDr7692ZR80bmzM3+YZO/uhk0HVqb7H4cMYWcU/7LSI24FSdYrfUnBV3xm0W60e9Jkrmv2GdhOyLIQI8eXP3VG2Mp5rb2LRFROQUJlB7kMsJWM7OL+WMrCznE7I9leRrY3oGU2xb7B3yOD4+1B+6vFj8VtHJ6iWhKEda5G/Q9r+EjLui0OOOZAQE1RqTU6lzMnkq0diTyc9ymBzVFUwUJiIJcKJWD3L5ZDp703MkPBw4f965m4aHA42qsgocXLAK+7/+HZuOJeCdsrtQgUbXdwcTejY6AG2308jIDMfgB1LQKLJRPXuEd3t7eVt1oAlYB0SuBJruTMJaH6UEGJzYmIgkwolapeTM/FIaDWAy1b+f0lLnJ2ts4OSPwiRwSHcUuvePQ781FDlFKTBgutU2CYEnoO34O7RjgjDi4WS0Sk4BkOK4bNW+/NK57eQ4J1NmpjngsXVTd7Y2xtPTMEg5Oaqz/GHuMyIiBxgM2VI9v9SkSeagx1ZNws03A+vXO96Xpyd1rLHdmfyz2PD2Qei+NUL/exKOG5MAJFnWN9dcxIiY/dAOu4aM+xPQIb0dNAFxzr1PbUYj8PHHzm0r1zmZMjPNk/C6WxvjziSsSiaHiY2JiHyAwZA9jmoSIiOdC4Y8XJuwc2MpPnslF/rforHvahcArSzrgnENg5v/Bm1fAzKmtEbvu7ogMGSAc+/vyJYtwNmzjrdr3VrewUBDamOcCZKrJ2H1Bw2srSQiUgoGQ/WprybBaPRsLYGDWgcTgJOIw4D3HoAJ12+23cIOIOOGImgnNkXaQylo0qanmwfrgLM1V3fd5T/BgC2eaG5TCk/XahIRyRSDIUfs1SR4upbgj/2J28xJvpoaq0x/PPszXkNMQAm07Q9DOyoAIx/pjKiuyQCSXT0q1zlbwzVhgnfLIQcNbW5TCk/nSBERyRR7kzWUvZ42Dz4IdOrk8EZ5/sgFbHxrP/TfVEJ3KAHdq3bjNfwZ8bi+vzMBUdjZfxba//0uJN+UBE2Axua+vMpR93TA3DVcbt3qyX3+OPcZESkKu9Z7kNdPZs0u0ocOAcuXW0/eWqMbckVpBb5/Nw/6LAN0P7XB7itdIHB9PJ8gVGJQs18wvf1mDOpTgfaTeiNImy6Pm40nuqeTsvAzJyIJMRjyIJ/NemunG7KABoDAP5u+gMWXH8VVNLZanxp6GBldTkJ7SyMMezgFzWKbea+MDaW0wQKp4fiZE5FEGAx5kE9OpoNBE03Q4ATikIQCtAk4i4yEQ8gYCWTM6IC2fRSWc1GzJqxNG/OykhLbTYJKGViQ6sfPkYgkwEEXFcRw3IBf5v4HQ+rphhwAgXYoxOFnVyPxmbugCYjyYQk9rDqpPDsbuO8++yMTc+Ri/6GEASKJiNwU4HgTqu3a5WvYsvQn/GNoLgY1+wUtE5rgzdUtnHptUsdAaRKgPa26SbB2AFg9MvFf/1r/+uxs35WViIioHqwZcoIwCeT99wh0K09Cv60RcktSUYbuVtsEBgIwOrEzf+iG7MzIxK+84vmRi9lUQ0REXsBgyI5Te4qxYflh6HSA/mhHFJk6AuhoWd9KcxYj4w5CO8KIjAeTkDBgMpD4V3VM1eDMyMTGeiJDd0YuZpMbERF5CYOhP1w6dQmbl++H7ssr0OfH4reKTgCiLevDcBVpkXnQDriEjLuj0f32zggIGmS9E7VM1eCpEYed3Q8nCyUiIi9SbTBUVV6FnR/mQ/fJOeh3t8D20lRUoa9lvQYm9Gq8Hxk3lkB7WzgGP5iKsOa969+pWqZq8FRTnzP74WShRETkZarqWr/ri33Y8elF6LaEIae4C0oRYbVdYlAhtB0LoB0TjPQZndEquaV7b+jvuS3OjEwcEGC/qcyVkYtzc4H0dMdl0uuBkSMdb0dERIrBrvVe0GdSEoDrJ7OF5gJGxO6HdlglMu5PQIcRCQDiG/5G/t4N2Zl52ebMAV56yfz/hjQZOtuUNnky8O67/lP7RkREPqOqrvXBqEB6871YpM3FjyvzcKY8HF+cGIgZ/xn6RyAkY0ajuZZk9Wrzv/UlKPtCdZNg27bWy+PizMtffLH+9c4GLc42yZ0/zy77RETkFlU1k506fAoxHRTYtV3OPakcNQk2tMnQmQliq3HiUCIiv8LpODzIZ3OTeYO9nlRqmijT3jmwJyfHv5spiYhUwlf3b1U1k1mRW7OTLY56UgHmnlRyLLsnVTfJRUY6t72nuv4TEZEqqCqB2sJWs1Pr1sBdd5m7aMul95czgxvaG7xQaT3aHJU3MxOIiAAyMhzvyx9G+SYiIp9RX82QvTm1zpwx93BKTzfnqMghEdfZGo7a22Vnm48hPR2YOlVex2SLs+UdPtycE6SxM7ebRgPEx/vHKN9EROQziguGFi9eDI1Gg9mzZ7v+4vqanWo6cUIePZOcreGouZ2jCVSlPqbaXClvdZd+oG5A5G+jfBMRkc8oKhjauXMnli9fjm7durm3g23b6m92qs2ZfBxv5h6lpblWE6K0HCN3yuuoS7+/J5MTEZHHKSYYunz5Mu666y68++67aNGihXs7KS52ftua+Tj2eLs5ytWaEFdyjOTA3fJmZgJHj5p7ja1aZf63oICBEBERuUUxwdCsWbMwduxYZDiRQFtRUYHS0lKrBwAgOrr+F9piL2/HV81RrtSEuJtjJJWGlLd6lO8pU8z/smmMiIjcpIjeZJ988gn27NmDnTt3OrX94sWLsXDhwrorBg0yBxHODOBXzVbejq8nD83MNO/LUe8wd3KMpKS08hIRkV+S/aCLhYWF6NOnD/73v/+he/fuAIDhw4ejR48eWLJkic3XVFRUoKKiwvK8tLQU8fHx5kGb9HpzzQ1Qf0BU32jGzk4e6uvB/5yZQFVOIzQrrbxERORTHHTxD7t370ZJSQl69+6NoKAgBAUFYdOmTXj99dcRFBQEo41k4NDQUISHh1s9LOw1O9XkqGeSXJujlNbbSmnlJSIivyT7YGjkyJH45ZdfsG/fPsujT58+uOuuu7Bv3z4EunOjrJmAO3s20KqV9XpHPZPk3LyjtN5WSisvERH5Hdk3k9niqJmsNofVbK6O1qyE5h1/G4GaiIhUx1fNZIpIoPa66p5Jrmz/2mvm3CONxjogkkvzjqvHJDWllZeIiPyGImuGXOW1yNLWHGfx8eZAiM07REREDcKaISVwtss7ERERyRaDoYZi8w4REZGiyb43GREREZE3MRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUTfbB0OLFi9G3b180a9YMbdq0wcSJE3HgwAGpi0VERER+QvbB0KZNmzBr1izs2LEDOp0OVVVVGDVqFMrKyqQuGhEREfkBjRBCSF0IV5w5cwZt2rTBpk2bMHToUKdeU1paioiICBgMBoSHh3u5hEREROQJvrp/B3ltz15iMBgAAJGRkXa3qaioQEVFheV5aWmp18tFREREyiT7ZrKahBCYM2cOhgwZgq5du9rdbvHixYiIiLA84uPjfVhKIiIiUhJFNZPNmjULX331FbZu3Yq4uDi729mqGYqPj2czGRERkYKwmayWxx57DOvWrcPmzZvrDYQAIDQ0FKGhoT4qGRERESmZ7IMhIQQee+wxrFmzBrm5uUhKSpK6SERERORHZB8MzZo1C6tWrcKXX36JZs2aobi4GAAQERGBRo0aSVw6IiIiUjrZ5wxpNBqby1esWIH77rvPqX2waz0REZHyMGfoDzKP1YiIiEjhFNW1noiIiMjTGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREamaYoKht956C0lJSQgLC0Pv3r2xZcsWqYtEREREfkARwdCnn36K2bNn45lnnsHevXuRlpaGMWPG4Pjx41IXjYiIiBROI4QQUhfCkf79+6NXr154++23LctSUlIwceJELF682OHrS0tLERERAYPBgPDwcG8WlYiIiDzEV/dv2dcMXbt2Dbt378aoUaOslo8aNQrbtm2TqFRERETkL4KkLoAjZ8+ehdFoRFRUlNXyqKgoFBcX23xNRUUFKioqLM8NBgMAc4RJREREylB93/Z2I5bsg6FqGo3G6rkQos6yaosXL8bChQvrLI+Pj/dK2YiIiMh7zp07h4iICK/tX/bBUKtWrRAYGFinFqikpKRObVG1efPmYc6cOZbnFy9eREJCAo4fP+7Vkyk3paWliI+PR2FhoapypXjcPG414HHzuNXAYDCgXbt2iIyM9Or7yD4YCgkJQe/evaHT6XDrrbdalut0OkyYMMHma0JDQxEaGlpneUREhKq+RNXCw8N53CrC41YXHre6qPW4AwK8m+Is+2AIAObMmYNp06ahT58+GDhwIJYvX47jx4/j4YcflrpoREREpHCKCIbuuOMOnDt3Ds8++yyKiorQtWtXfP3110hISJC6aERERKRwigiGAGDmzJmYOXOmW68NDQ3F/PnzbTad+TMeN49bDXjcPG414HF797gVMegiERERkbfIftBFIiIiIm9iMERERESqxmCIiIiIVI3BEBEREamaIoOht956C0lJSQgLC0Pv3r2xZcuWerfftGkTevfujbCwMLRv3x7Lli2rs01WVhZSU1MRGhqK1NRUrFmzxlvFd5srx52dnQ2tVovWrVsjPDwcAwcOxHfffWe1zcqVK6HRaOo8ysvLvX0oLnHluHNzc20e0/79+62287fP+7777rN53DfccINlGyV83ps3b8b48eMRGxsLjUaDtWvXOnyNP1zfrh63v1zfrh63v1zfrh63v1zfixcvRt++fdGsWTO0adMGEydOxIEDBxy+zhfXuOKCoU8//RSzZ8/GM888g7179yItLQ1jxozB8ePHbW5fUFCAm2++GWlpadi7dy/+9re/4fHHH0dWVpZlm+3bt+OOO+7AtGnT8NNPP2HatGmYPHkyfvjhB18dlkOuHvfmzZuh1Wrx9ddfY/fu3UhPT8f48eOxd+9eq+3Cw8NRVFRk9QgLC/PFITnF1eOuduDAAatj6tSpk2WdP37er732mtXxFhYWIjIyErfffrvVdnL/vMvKytC9e3csXbrUqe395fp29bj95fp29birKf36dvW4/eX63rRpE2bNmoUdO3ZAp9OhqqoKo0aNQllZmd3X+OwaFwrTr18/8fDDD1st69Kli5g7d67N7f/617+KLl26WC2bMWOGGDBggOX55MmTxU033WS1zejRo8Wdd97poVI3nKvHbUtqaqpYuHCh5fmKFStERESEp4roFa4ed05OjgAgLly4YHefavi816xZIzQajTh69KhlmRI+75oAiDVr1tS7jb9c3zU5c9y2KPH6rsmZ4/aX67smdz5vf7i+hRCipKREABCbNm2yu42vrnFF1Qxdu3YNu3fvxqhRo6yWjxo1Ctu2bbP5mu3bt9fZfvTo0di1axcqKyvr3cbePn3NneOuzWQy4dKlS3Umu7t8+TISEhIQFxeHcePG1fllKaWGHHfPnj0RExODkSNHIicnx2qdGj7v999/HxkZGXVGaZfz5+0Of7i+PUGJ13dDKPn69gR/ub4NBgMA1DsJq6+ucUUFQ2fPnoXRaKwzW31UVFSdWe2rFRcX29y+qqoKZ8+erXcbe/v0NXeOu7aXX34ZZWVlmDx5smVZly5dsHLlSqxbtw6rV69GWFgYBg8ejEOHDnm0/O5y57hjYmKwfPlyZGVlITs7G8nJyRg5ciQ2b95s2cbfP++ioiJ88803eOCBB6yWy/3zdoc/XN+eoMTr2x3+cH03lL9c30IIzJkzB0OGDEHXrl3tbuera1wx03HUpNForJ4LIeosc7R97eWu7lMK7pZx9erVWLBgAb788ku0adPGsnzAgAEYMGCA5fngwYPRq1cvvPHGG3j99dc9V/AGcuW4k5OTkZycbHk+cOBAFBYW4qWXXsLQoUPd2qdU3C3jypUr0bx5c0ycONFquVI+b1f5y/XtLqVf367wp+vbXf5yfT/66KP4+eefsXXrVofb+uIaV1TNUKtWrRAYGFgn2ispKakTFVaLjo62uX1QUBBatmxZ7zb29ulr7hx3tU8//RT3338/PvvsM2RkZNS7bUBAAPr27SubXxINOe6aBgwYYHVM/vx5CyHw73//G9OmTUNISEi928rt83aHP1zfDaHk69tTlHZ9N4S/XN+PPfYY1q1bh5ycHMTFxdW7ra+ucUUFQyEhIejduzd0Op3Vcp1Oh0GDBtl8zcCBA+ts/7///Q99+vRBcHBwvdvY26evuXPcgPkX43333YdVq1Zh7NixDt9HCIF9+/YhJiamwWX2BHePu7a9e/daHZO/ft6AubfG4cOHcf/99zt8H7l93u7wh+vbXUq/vj1Fadd3Qyj9+hZC4NFHH0V2djY2btyIpKQkh6/x2TXudKq1THzyySciODhYvP/++yIvL0/Mnj1bNGnSxJJVP3fuXDFt2jTL9r///rto3Lix+Mtf/iLy8vLE+++/L4KDg8UXX3xh2eb7778XgYGB4l//+pfIz88X//rXv0RQUJDYsWOHz4/PHlePe9WqVSIoKEi8+eaboqioyPK4ePGiZZsFCxaIb7/9Vhw5ckTs3btXTJ8+XQQFBYkffvjB58dnj6vH/eqrr4o1a9aIgwcPil9//VXMnTtXABBZWVmWbfzx86529913i/79+9vcpxI+70uXLom9e/eKvXv3CgDilVdeEXv37hXHjh0TQvjv9e3qcfvL9e3qcfvL9e3qcVdT+vX9yCOPiIiICJGbm2v1vb1y5YplG6muccUFQ0II8eabb4qEhAQREhIievXqZdUt79577xXDhg2z2j43N1f07NlThISEiMTERPH222/X2efnn38ukpOTRXBwsOjSpYvVxSUXrhz3sGHDBIA6j3vvvdeyzezZs0W7du1ESEiIaN26tRg1apTYtm2bD4/IOa4c9wsvvCA6dOggwsLCRIsWLcSQIUPEV199VWef/vZ5CyHExYsXRaNGjcTy5ctt7k8Jn3d112l731t/vb5dPW5/ub5dPW5/ub7d+Z77w/Vt65gBiBUrVli2keoa1/xRQCIiIiJVUlTOEBEREZGnMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRESKtHr1aoSFheHkyZOWZQ888AC6desGg8EgYcmISGk4NxkRKZIQAj169EBaWhqWLl2KhQsX4r333sOOHTvQtm1bqYtHRAoSJHUBiIjcodFosGjRIkyaNAmxsbF47bXXsGXLFgZCROQy1gwRkaL16tULv/32G/73v/9h2LBhUheHiBSIOUNEpFjfffcd9u/fD6PRiKioKKmLQ0QKxZohIlKkPXv2YPjw4XjzzTfxySefoHHjxvj888+lLhYRKRBzhohIcY4ePYqxY8di7ty5mDZtGlJTU9G3b1/s3r0bvXv3lrp4RKQwrBkiIkU5f/48Bg8ejKFDh+Kdd96xLJ8wYQIqKirw7bffSlg6IlIiBkNERESkakygJiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREana/wdOOhhsAQsSkgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_22_2.png"
      }
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own sdg\n",
      "[[4.32234998]\n",
      " [2.64530585]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using SGD\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = (1.0/n)*training_gradient(y, X, theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "# Can you figure out a better way of setting up the contributions to each batch?\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        theta = theta - eta*gradients\n",
    "print(\"theta from own sdg\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6d7455a",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Same code but now with momentum gradient descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "40a68e4f",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.7635689 ]\n",
      " [3.10080981]]\n",
      "Eigenvalues of Hessian Matrix:[0.31633433 3.9824638 ]\n",
      "theta from own gd\n",
      "[[3.76366462]\n",
      " [3.1007216 ]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own sdg with momentum\n",
      "[[3.75707243]\n",
      " [3.12422141]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using SGD\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 100\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = (1.0/n)*training_gradient(y, X, theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "change = 0.0\n",
    "delta_momentum = 0.3\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        # calculate update\n",
    "        new_change = eta*gradients+delta_momentum*change\n",
    "        # take a step\n",
    "        theta -= new_change\n",
    "        # save the change\n",
    "        change = new_change\n",
    "print(\"theta from own sdg with momentum\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1f7f38d",
   "metadata": {
    "editable": true
   },
   "source": [
    "## AdaGrad algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/adagrad.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/adagrad.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b85cc96c",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Similar (second order function now) problem but now with AdaGrad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "3ced9e13",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own AdaGrad\n",
      "[[1.99999956]\n",
      " [3.00000215]\n",
      " [3.99999797]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using AdaGrad and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-8\n",
    "for epoch in range(n_epochs):\n",
    "    Giter = 0.0\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        Giter += gradients*gradients\n",
    "        update = gradients*eta/(delta+np.sqrt(Giter))\n",
    "        theta -= update\n",
    "print(\"theta from own AdaGrad\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed1fa9e4",
   "metadata": {
    "editable": true
   },
   "source": [
    "Running this code we note an almost perfect agreement with the results from matrix inversion."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "787b2b5e",
   "metadata": {
    "editable": true
   },
   "source": [
    "## RMSProp algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/rmsprop.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/rmsprop.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac61b6e0",
   "metadata": {
    "editable": true
   },
   "source": [
    "## RMSprop for adaptive learning rate with Stochastic Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "5febac4f",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own RMSprop\n",
      "[[1.99907985]\n",
      " [2.99897733]\n",
      " [3.99754609]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Value for parameter rho\n",
    "rho = 0.99\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-8\n",
    "for epoch in range(n_epochs):\n",
    "    Giter = 0.0\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "\t# Accumulated gradient\n",
    "\t# Scaling with rho the new and the previous results\n",
    "        Giter = (rho*Giter+(1-rho)*gradients*gradients)\n",
    "\t# Taking the diagonal only and inverting\n",
    "        update = gradients*eta/(delta+np.sqrt(Giter))\n",
    "\t# Hadamard product\n",
    "        theta -= update\n",
    "print(\"theta from own RMSprop\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee8a6301",
   "metadata": {
    "editable": true
   },
   "source": [
    "## ADAM algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/adam.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/adam.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2d0f16b",
   "metadata": {
    "editable": true
   },
   "source": [
    "## And finally [ADAM](https://arxiv.org/pdf/1412.6980.pdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "59349e6f",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own ADAM\n",
      "[[2.00003617]\n",
      " [2.99986253]\n",
      " [4.00012569]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Value for parameters beta1 and beta2, see https://arxiv.org/abs/1412.6980\n",
    "beta1 = 0.9\n",
    "beta2 = 0.999\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-7\n",
    "iter = 0\n",
    "for epoch in range(n_epochs):\n",
    "    first_moment = 0.0\n",
    "    second_moment = 0.0\n",
    "    iter += 1\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        # Computing moments first\n",
    "        first_moment = beta1*first_moment + (1-beta1)*gradients\n",
    "        second_moment = beta2*second_moment+(1-beta2)*gradients*gradients\n",
    "        first_term = first_moment/(1.0-beta1**iter)\n",
    "        second_term = second_moment/(1.0-beta2**iter)\n",
    "\t# Scaling with rho the new and the previous results\n",
    "        update = eta*first_term/(np.sqrt(second_term)+delta)\n",
    "        theta -= update\n",
    "print(\"theta from own ADAM\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d490115",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Introducing [JAX](https://jax.readthedocs.io/en/latest/)\n",
    "\n",
    "Presently, instead of using **autograd**, we recommend using [JAX](https://jax.readthedocs.io/en/latest/)\n",
    "\n",
    "**JAX** is Autograd and [XLA (Accelerated Linear Algebra))](https://www.tensorflow.org/xla),\n",
    "brought together for high-performance numerical computing and machine learning research.\n",
    "It provides composable transformations of Python+NumPy programs: differentiate, vectorize, parallelize, Just-In-Time compile to GPU/TPU, and more."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bc7785a",
   "metadata": {
    "editable": true
   },
   "source": [
    "### Getting started with Jax, note the way we import numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "f1090d5e",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import jax\n",
    "import jax.numpy as jnp\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from jax import grad as jax_grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac6bf976",
   "metadata": {
    "editable": true
   },
   "source": [
    "### A warm-up example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ed55121d",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/jax/_src/numpy/lax_numpy.py:173: UserWarning: Explicitly requested dtype float64 requested in asarray is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more.\n",
      "  return asarray(x, dtype=self.dtype)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x11cb23a60>]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABQCUlEQVR4nO3deXxU1eH+8c+dmaxkhZCFECDsS9hBFNkURami4K6tim1tUyOVUmu/WlutbeXXRUstYLRWXLHugDu4sCj7JoussgWSAAnJTNZJMnN/fwxEEQQCydyZyfN+veYFTG7Iw4iZh3POPccwTdNERERExE9sVgcQERGR5kXlQ0RERPxK5UNERET8SuVDRERE/ErlQ0RERPxK5UNERET8SuVDRERE/ErlQ0RERPzKYXWA7/J6veTn5xMbG4thGFbHERERkTNgmiZlZWW0adMGm+3UYxsBVz7y8/PJyMiwOoaIiIichby8PNq2bXvKawKufMTGxgK+8HFxcRanERERkTPhcrnIyMiofx8/lYArH8emWuLi4lQ+REREgsyZLJnQglMRERHxK5UPERER8SuVDxEREfErlQ8RERHxK5UPERER8SuVDxEREfErlQ8RERHxK5UPERER8auA22RMREREmojXA3uXQvlBiEmB9kPBZvd7jAaNfEydOpXBgwcTGxtLcnIy48ePZ9u2bcddM3HiRAzDOO5x/vnnN2poEREROXMz188k9+PJMC0Lnr8S3vyJ78dpWeR+PJmZ62f6NU+DyseiRYvIyclh+fLlLFiwgLq6OsaMGUNFRcVx111++eUUFBTUP95///1GDS0iIiJnzla0nRkHPiHXVn7c87m2CmYc+ARb0Xa/5mnQtMuHH3543K9nzZpFcnIya9asYcSIEfXPR0REkJqa2jgJRURE5Ox5PWRv+Ahs5cxITAAgu9RFbkIcMxLjySlxkl08Hy72+G0K5pzWfDidTgBatmx53PMLFy4kOTmZhIQERo4cyV/+8heSk5NP+nu43W7cbnf9r10u17lE+v6slbW8unofOw+V87fr+jbJ1xAREQk4e5eCK5/so7+ckZhAbkI8HsMgp6SU7FIX4PRdlzncL5HO+m4X0zSZMmUKw4YNIysrq/75sWPH8vLLL/Ppp5/y2GOPsWrVKi6++OLjCsa3TZ06lfj4+PpHRkbG2UY6pRqPl//3wVZeW72fvCOVTfI1REREAk75wfqfjqisAsBjGISZ5tHiceJ1Te2sy8fdd9/Nhg0beOWVV457/sYbb+SKK64gKyuLcePG8cEHH7B9+3bee++9k/4+999/P06ns/6Rl5d3tpFOqXVsBOd3bAXAexsLmuRriIiIBJyYlPqf/qWVb6bCME1qDYPchLiTXtfUzmraZdKkScybN4/FixfTtm3bU16blpZG+/bt2bFjx0k/HhERQURExNnEaLAr+qSx9Oti3t2QT/bITn75miIiIpZqPxTi2jDDVsGGSN/77YyDh9kcEX50DYhBtjfGd52fNGjkwzRN7r77bt566y0+/fRTMjMzT/s5xcXF5OXlkZaWdtYhG8vYrDTsNoNNB1zsKao4/SeIiIgEO5ud3D6XkZsYD0DrujouqKomu9RFTomTGYnx5PYZ49f9PhpUPnJycnjppZeYPXs2sbGxFBYWUlhYSFWVbw6pvLyce++9l2XLlrFnzx4WLlzIuHHjSEpKYsKECU3yB2iIli3CGdrJN/Xy7oZ8i9OIiIj4hzepKxm1BgBXllfWT3tke2PISR+NN6mrX/MYpmmaZ3yxYZz0+VmzZjFx4kSqqqoYP34869ato7S0lLS0NC666CL+9Kc/nfFCUpfLRXx8PE6nk7i4uNN/QgO9tiqP+97cQPfUWD6cPOL0nyAiIhLkiot3cMk7E6gzDP4adwM/6Nyj0Xc4bcj7d4PWfJyup0RFRfHRRx815Lf0uzG9UnjgbYOthWXsPFRO5+QYqyOJiIg0qVcW/pU6w6BjNVx8ywMQ5v8t1b+t2R0slxAdzvAuSYCmXkREpHn4pGQlAN1sfYm0uHhAMywfAFf2aQPAuxsKTjuaIyIiEsy2bn+PnWEmDtNkRN9fWh0HaKbl49JeKYTbbew8VM62g2VWxxEREWky/1vlOzSuR2Uklw4YaHEan2ZZPuIiwxjRtTUA723QhmMiIhKaamsr+cy9B4AOkRcR4bB+ygWaafkAGNfXt++Ipl5ERCRUfb76SY7YbSR4vFx8Xo7Vceo12/IxukcKEQ4bu4sq2JzfNIfZiYiIWOnNbW8B0Lm8NSO6t7M4zTeabfmIiXBwcXffSbvvaupFRERCjLN0D0vxnT6fnnA94Y7AecsPnCQWOHbXyztf5mvqRUREQsp7y/9BrWHQwQ1jzr/B6jjHadbl4+LuybQIt3OgtIq1+0qtjiMiItJo5uZ/DkDrii71R4sEimZdPqLC7Vza03eE8DtfasMxEREJDbt2f8JXdg8O06Rd+k9x2APr7T6w0ljgqn6+qZf3Nhbg8WrqRUREgt/ba2YA0K0igivOO9/iNCdq9uVjWOfWxEeFcbjMzYpdxVbHEREROSeeuhrec20HIKJ6MAPbJVqc6ETNvnyEO2z8oHcqAPM09SIiIkFu+bqnOWw3iPd46dr1Z9hsJz+R3krNvnwAjOvrm3r5YFMhNXVei9OIiIicvTnbXgcgs6wlVw/oZHGak1P5AIZktiI5NgJnVS1Ldhy2Oo6IiMhZKXMd4LNa3xICb91l9GoTZ3Gik1P5AOw2gyv6+LZb110vIiISrD5a8Rhum0FGjcmArOswjMCbcgGVj3rHpl7mf3WQqhqPxWlEREQabs7+hQC0cnbg6v7p1oY5BZWPo/pnJJDRMorKGg+fbD1odRwREZEG2bt3CV/aarGZJtWOm+icHGt1pO+l8nGUYRiM+9Z26yIiIsFk3trpAHSvDOPifhdYnObUVD6+5djUy2fbDuOqrrU4jYiIyJnxeup4p2QzAKazP1ceXccYqFQ+vqV7aixdkmOoqfPy4aZCq+OIiIickVVfPkuB3SDG48WMu4mMltFWRzollY9vMQyDq49utz5vvaZeREQkOMzd8j8AOpcncGX/rhanOT2Vj++4up9vdfDSr4s45Kq2OI2IiMipVZQXsqDmEADFzlH1W0cEMpWP78hoGc2Adgl4TXhnQ4HVcURERE5p/orHqbYZtKkxSWozjqSYCKsjnZbKx0mMP3pv9Nz1ByxOIiIicmrz8j4FINnZngkD2lqc5syofJzED3qnYbcZbNjvZNfhcqvjiIiInNT+/ctZbbgxTJM95Vdwac9UqyOdEZWPk0iKiWBY5yQA5mrhqYiIBKh3Vj8BQLeqMPp1u4CYCIfFic6Mysf3GN//6F0vX+ZjmqbFaURERI7n9dQx98hGAGylWVx9dK+qYKDy8T0u7ZlKZJiN3UUVbNjvtDqOiIjIcdZufIkDdoj2etlTO54RXVtbHemMqXx8j5gIR/3cmaZeREQk0Mzb8hIAXcriGd27K+GO4HlLD56kFjg2hPXOhnw8Xk29iIhIYKisLOKjat9O3CWlwxjfL3BPsD0ZlY9TGNG1NQnRYRwuc7Ps62Kr44iIiADwyYppVNoMUmu9HHFcyqD2iVZHahCVj1MId9j4QW/fTnFztOeHiIgEiHn75gOQ5szg6v4Z2GyGxYkaRuXjNI4NZX24qZCqGo/FaUREpLkrLFjHCrMSgJ3OsfVnkgUTlY/TGNQ+kfSEKMrddXy85aDVcUREpJl7Z9U/MQ2DrpV24lr2p0danNWRGkzl4zRsNqN+z4856zT1IiIi1jG9XuYWrQMg3NWTCf2Da6HpMSofZ+DYf9xF2w9TXO62OI2IiDRXX371P/baIcrrZYPzyvqT2IONyscZ6JwcS+/0eOq8Ju/qpFsREbHIvE0vANC1PJZ+HTuSGh9pcaKzo/Jxho6NfrytqRcREbFAdVUJH1btB8BZOjTo9vb4NpWPMzSubxvsNoP1eaU66VZERPxu4aonKLMZJNd62VlzEZdnBccJtiej8nGGWsdGMLyL76TbOdpuXURE/GzO7vcBSHe14ZKebYmNDLM40dlT+WiAY1Mvc9Yd0Em3IiLiN4cObmKZWQHArtLLmdA/+Pb2+DaVjwa4tGcK0eF29h2pZO2+EqvjiIhIM/HuysfxGgadqwyqwvowvEvwnGB7MiofDRAd7qifY9PCUxER8QfT62Xe4dUARDl7Mq5vG8Lswf32HdzpLXBs6uXdDQXU1HktTiMiIqHuq21v87XdJMJrstF5BeODdGOxb1P5aKChnZJIiYugtLKWz7YdsjqOiIiEuDkb/gtAt4poWrfMoG/beIsTnTuVjway24z6HeXeWrvf4jQiIhLKatxlfFC5D4Dy0vOZ0D8dwwiuE2xPRuXjLFw7oC0An249RElFjcVpREQkVC1a9W+cNoOkOi8by0czYUDwT7mAysdZ6ZYaS682cdR6TN7ZoD0/RESkaczb9S4AGa5UzuuYTNvEaIsTNQ6Vj7N0bPTjzTWaehERkcZXVLSVJV4XAHtLL61/3wkFKh9n6ap+bXDYDL7c72TnoTKr44iISIh5f+XjeAyDTtUGh7z9GNs7zepIjUbl4ywlxUQwqptvk5e31mrPDxERaVzzDq4AoIWzK5dnpRIT4bA4UeNR+TgH1xwdAnt73QE8Xm23LiIijWPrtnlss3kJM002ll7JNSGy0PQYlY9zMLpHMnGRDgqc1SzfVWx1HBERCRFzNzwDQPfyKKJjMhjaKcniRI1L5eMcRDjsjOvrO9xHC09FRKQx1NZW8n75LgCqnIMZ3z8duy349/b4NpWPc3TtQN/UywebCqlw11mcRkREgt3nq2dyxGaQWOdlQ9mlXBtiUy6g8nHO+mckkJnUgqpaDx9sKrQ6joiIBLm5O98GoENZMr3atqZLSqzFiRpfg8rH1KlTGTx4MLGxsSQnJzN+/Hi2bdt23DWmafLwww/Tpk0boqKiGDVqFJs3b27U0IHEMIz6VqqpFxERORelJbtZ5HECsL90dEjt7fFtDSofixYtIicnh+XLl7NgwQLq6uoYM2YMFRUV9df87W9/4/HHH2f69OmsWrWK1NRULr30UsrKQncvjAkD2mIYsGxXMXlHKq2OIyIiQer9Ff+gzjDo4Ia82gH16wpDTYPKx4cffsjEiRPp1asXffv2ZdasWezbt481a9YAvlGPadOm8bvf/Y5rrrmGrKwsnn/+eSorK5k9e3aT/AECQXpCFEM7tQK054eIiJy9uQVfABDv7Mzo7im0bBFucaKmcU5rPpxO39BQy5YtAdi9ezeFhYWMGTOm/pqIiAhGjhzJ0qVLT/p7uN1uXC7XcY9gdP3ADADeWJuHV3t+iIhIA+3Y+SFf2Tw4TJNNpVdy/aDQnHKBcygfpmkyZcoUhg0bRlZWFgCFhb4FlykpKcddm5KSUv+x75o6dSrx8fH1j4yMjLONZKnLeqUSG+Eg70gVK3YfsTqOiIgEmXnrnwKge0UEjqj2jOza2uJETeesy8fdd9/Nhg0beOWVV074mGEcfz+yaZonPHfM/fffj9PprH/k5eWdbSRLRYXbubKvb9/9N7TwVEREGqCutpp3XTsAqHEO4poB6TjsoXtD6ln9ySZNmsS8efP47LPPaNv2m2Gh1NRUgBNGOQ4dOnTCaMgxERERxMXFHfcIVtcd3fPj/Y0FlGvPDxEROUPL1j1Nkd0g3uNlg2tM/ftJqGpQ+TBNk7vvvpu33nqLTz/9lMzMzOM+npmZSWpqKgsWLKh/rqamhkWLFjF06NDGSRzABrRLpOPRPT/e31BgdRwREQkSc7e/AUBHVxI926bQNQT39vi2BpWPnJwcXnrpJWbPnk1sbCyFhYUUFhZSVVUF+KZbJk+ezKOPPsrbb7/Npk2bmDhxItHR0dxyyy1N8gcIJIZh1O94qqkXERE5E07nPj6r860VLCi9mOsGBefax4ZoUPl48skncTqdjBo1irS0tPrHq6++Wn/Nfffdx+TJk7nrrrsYNGgQBw4cYP78+cTGhnaLO+baAW2xGbByzxH2FFWc/hNERKRZ+2j5Y9QYBu3cJnvrBnFVn9Dc2+PbGjztcrLHxIkT668xDIOHH36YgoICqqurWbRoUf3dMM1Banwkw7v4Vihr9ENERE5nbv4SABKdHRnTM4346DCLEzW90F1Ka6FjC4XeXLsfj/b8EBGR77F7z0I22GqxmyZbSsdxfTOYcgGVjyZxac8U4iIdFDirWfp1kdVxREQkQM1bMx2AbpXhhLXoyLDOSRYn8g+VjyYQGWbn6n6+w+ZeXRWc+5aIiEjT8tTV8I5zKwDe0v5cMyAdu+3ke2KFGpWPJnLD0aGz+ZsPUlJRY3EaEREJNCvWP8NBu0Gsx8uXZWPr3zeaA5WPJpKVHkePtDhqPF7mrtdhcyIicrx5214HoFNZS/p3SKdDUguLE/mPykcTMQyDG48eCvTq6v2YphaeioiIT3lZAZ/UHAbgUOkIbhzcfEY9QOWjSY3vn064w8aWAhebDgTnab0iItL45q94jGqbQdsak3zvMMZmpVkdya9UPppQQnQ4l/XynXfz6up9FqcREZFAMXf/ZwAkOdszrl9bosLtFifyL5WPJnbj0QVEc9fnU13rsTiNiIhYLS/vC9YaNdhMk62lV9S/TzQnKh9NbGinVrRNjKKsuo4PNumwORGR5m7e6mN7e4TRKimLPm3jLU7kfyofTcxmM7h+oK/VvrZK262LiDRnXk8d80o2AWA4+3DDoAwMo3ns7fFtKh9+cN2gthgGLNtVzN5iHTYnItJcrdnwPPl2aOH18lXFOCb0T7c6kiVUPvwgPSGqfsvc11dr9ENEpLmas2U2AF3K4hnRsyOJLcItTmQNlQ8/OXYP9+tr8qjzeC1OIyIi/lZZfogF7oMAFJWOaFY7mn6XyoefXNozhZYtwjnocrNo+2Gr44iIiJ99vHIaVTaDNjUmzrBLms0hciej8uEnEQ471xyd23tlpfb8EBFpbubumw9AsqsdNw5u32wOkTsZlQ8/uuk83xDbp1sPUeistjiNiIj4S37+alYabgzT5OvSsdwwuK3VkSyl8uFHnZNjOa9DS7wmvL46z+o4IiLiJ/NWTQOga5WDHp3OJy0+ytpAFlP58LNjox+vrs7D69VhcyIioc70eplX/CUADmcWN53XzuJE1lP58LMf9E4jLtLB/pIqPt9ZZHUcERFpYus2vkSeHaK9XvK913BRt9ZWR7KcyoefRYbZ6zeV0cJTEZHQN++rlwDoUhbHVQN74bDrrVevgAVuHuIbclvw1UEOl7ktTiMiIk2lqvIIH1XnA+B0Xli/51Nzp/Jhge6pcfTLSKDOa/LmWu14KiISqj5dOY1ym0FKrZeEtAlktIy2OlJAUPmwyM3HFp6uysM0tfBURCQUzd37IQBtXOncPKSjxWkCh8qHRa7s04YW4XZ2F1WwbFex1XFERKSRFRauZ7lZCcDh6qu4pEeKxYkCh8qHRVpEOLi6fuGp9vwQEQk1766ahmkYdK2yMaLfaMIdess9Rq+EhW45eq/3h5sKKCrXwlMRkVBher3MPbwGgHBnT27W3h7HUfmwUFZ6PH0zEqj1mLyxRgtPRURCxcYtr7PHDpFek6iWt9EhqYXVkQKKyofFfni0Dc9esU87noqIhIg5G58DoGt5C64/v5+lWQKRyofFruybRmykg31HKvnia+14KiIS7NzVTj6s8m0iaVYN45KeWmj6XSofFosOd3DtAN/phi8v146nIiLBbuGqJyiz2Whd52VAnzsI046mJ9ArEgBuObbj6ZaDHHRVW5xGRETOxVtfvwtAW1caN2lvj5NS+QgAXVNiGdwhEY/X5LVVuu1WRCRYHT60mRVUABAfcyNtE7Wj6cmofASIHw5pD/gOm/No4amISFCat+JxPIZB52qD8cOusjpOwFL5CBCXZ6WSGB1GvrOahdsOWR1HREQayPR6mXdoFQCtKnsxqluyxYkCl8pHgIgMs3PdQN/C09krtPBURCTYbNk+j10Ok3CvSe/ud2G3GVZHClgqHwHklqNTL59uO0TekUqL04iISEO8tuYpALpXRHPThYMtThPYVD4CSGZSC4Z1TsI0fWs/REQkONS6K/jY7fu+nR4xmpS4SIsTBTaVjwDzo/N9ox+vrsrDXeexOI2IiJyJBSv+hdNuo1Wdl6tGTLI6TsBT+Qgwl/RIJi0+kuKKGj7YWGh1HBEROQNv75gLQNeqNlzYNc3iNIFP5SPAOOy2+tNuX1y+1+I0IiJyOsXFO1ht9+3tMTDzdgxDC01PR+UjAN14XgYOm8GavSVszndaHUdERE7G64HdS3j93V9SZxh0rIabR99odaqgoPIRgJJjI7k8KxWAl3Tei4hIQJm5fia5H0+GaVnw/JV86t4FwLUVFcz+/DfMXD/T2oBBQOUjQN12QQcA5qw7gLOq1towIiJSz1a0nRkHPiHXVs62sDC2RITjME2O2D3MOPAJtqLtVkcMeCofAWpwh0S6pcRSVevhrbX7rY4jIiIAXg/ZGz4ip6SUGYkJ/DkpEYD2tbX8NyGOnBIn2Rvm+6Zk5HupfAQowzD40QW+225fXL4X09R5LyIiltu7FFz5ZJe6+EVJKesjfft5fB0eTk5JKdmlTnAd8F0n30vlI4BN6J9OTISDXYcrWPZ1sdVxRESk/GD9TzvW1tX/PMw0yS51nfQ6OZHKRwCLiXBwzYB0AJ5ftsfaMCIiAjEpAJjAX1smAGAzTWoNg9yEuBOuk5NT+Qhwtx7d8XTBVwfZX6LzXkRELNV+KMS14cGklhQ5HDi8XhbtO1C/BiQ3IR7i0n3XyfdS+QhwXVJiubBzK7ymbrsVEbGczU5un8uYFxsDwE1l5SR4vWSXusgpcTIjMZ7cPmPAZrc4aGBT+QgCtx+97fZ/q/ZRXasV1CIiVsqr821/4DBNbnOW1T+f7Y0hJ3003qSuVkULGg6rA8jpje6RQtvEKPaXVDFvfT43DM6wOpKISLNVdGAtOKBfZSypP5oD5Yd8azzaDyVbIx5nRCMfQcBuM+rXfjy3dI9uuxURsciu3YtYZveNdgzPvBsjcwT0vg4yh2uqpQFUPoLEjYMziAyz8VWBi9V7S6yOIyLSLOUufgTTMMiqDOMmneNy1lQ+gkRCdDjj+/luu31u6R5rw4iINEOHD23mY9O3f0dW/E1Eh2vlwtlS+Qgitw/tAMCHmwopdFZbG0ZEpJl55tMHqTUMOlcZ3HrZ3VbHCWoNLh+LFy9m3LhxtGnTBsMwmDNnznEfnzhxIoZhHPc4//zzGytvs9YjLY7zMlvi8Zq8vGKv1XFERJqNMtcB5lb5DozrZLuUdq2iLU4U3BpcPioqKujbty/Tp0//3msuv/xyCgoK6h/vv//+OYWUb0w8Ovoxe8U+3HW67VZExB9mf/Y7Kmw22taYjBt5r9Vxgl6DJ6zGjh3L2LFjT3lNREQEqampZx1Kvt+YnimkxUdS4KzmnS8LuG5gW6sjiYiENHe1k1eKV4PdIKNyICO66v3tXDXJmo+FCxeSnJxM165dufPOOzl06ND3Xut2u3G5XMc95Ps57DZuO7rp2Kwvduu2WxGRJjZv8R8pthu0rvMyfND/YRiG1ZGCXqOXj7Fjx/Lyyy/z6aef8thjj7Fq1Souvvhi3G73Sa+fOnUq8fHx9Y+MDG2gdTo3n+e77XZzvouVu49YHUdEJGR56mp4Nm8BAO2d3bh2cBeLE4WGRi8fN954I1dccQVZWVmMGzeODz74gO3bt/Pee++d9Pr7778fp9NZ/8jLy2vsSCEnITqcawf4plue/WK3xWlERELXJ8v+xn4HxHq8dOtyr26vbSRNfqttWloa7du3Z8eOHSf9eEREBHFxccc95PTuuLADAPO/Osi+Yp12KyLS2Eyvl2d2vAFAF2c6tw3vb3Gi0NHk5aO4uJi8vDzS0tKa+ks1K52TYxnZtTWmCc8v22N1HBGRkLNi3X/YYvcQ4TVp2WoSbRKirI4UMhpcPsrLy1m/fj3r168HYPfu3axfv559+/ZRXl7Ovffey7Jly9izZw8LFy5k3LhxJCUlMWHChMbO3uwdG/14dVUeZdW11oYREQkxz2x8BoAerkR+NHKYxWlCS4PLx+rVq+nfvz/9+/uGn6ZMmUL//v35wx/+gN1uZ+PGjVx99dV07dqV22+/na5du7Js2TJiY2MbPXxzN6JLazq1bkG5u4431uy3Oo6ISMj4auscVhjV2E0TbBMZ0C7B6kghpcErZ0aNGnXK2zs/+uijcwokZ85mM7jjwkwenLOJ55bu4bYLOmC36RYwEZFz9ezqxwHoVd6C64ZfrttrG5nOdgly1wxIJz4qjL3FlXy29fv3UxERkTOTl/cFC+p82xhUVYznB721ZrGxqXwEuehwBzef1w6AZz7fZXEaEZHgN+uLP+E1DHpWOBh93vWE2fVW2dj0ioaA24e2x2EzWL7rCJsOOK2OIyIStIqKtjK32reGrqbkEm45+o87aVwqHyEgLT6KK/r4hgWfWaLRDxGRszV78e+pMQw6VRv07PEjEluEWx0pJKl8hIg7h3cE4N0NBRQ4qyxOIyISfMrLCnjFuQUAR/EQfjqis8WJQpfKR4jISo/n/I4tqfOaPPfFHqvjiIgEnTcW/Z5ym0F6jUli+k/ITGphdaSQpfIRQo6NfsxeuY9yd53FaUREgkeNu4wXDi0HIPFIb+4c0dXiRKFN5SOEXNQtmY6tW1BWXcerq3RAn4jImXp3yR85bDdIqvPijv4xgzskWh0ppKl8hBCbzeCnw3yjH89+vps6j9fiRCIigc/rqWPWvvkApJd04o4RvbSpWBNT+Qgx1wxIp1WLcA6UVvHh5kKr44iIBLzPlv+DPXaTGI+XgrqJjM1KtTpSyFP5CDGRYXZ+dH57AP6zZPcpt8IXEWnuTK+XZ7a/CkAXZxtuvrAfDm0q1uT0CoegWy9oT7jDxpd5pazaU2J1HBGRgLX6y1lsstUR4TXZ7bqFGwdnWB2pWVD5CEFJMRFcO6AtAE8v/triNCIigeu/G54GoIcrgbGDzyc2MsziRM2DykeIunN4JoYBH285xI6DZVbHEREJONu2v8sXVGIzTfaVXMuPL8y0OlKzofIRojq2juGynr5FU08v1pbrIiLf9eyqfwCQVR7Neb0uJjU+0uJEzYfKRwj7+Ujfbbdz1h+g0FltcRoRkcCxf/9yPqwtAuBQ0RX8bERHixM1LyofIax/u0TOy2xJrcdk1he7rY4jIhIwnv/iEbyGQY9KB+0zf0CXlFirIzUrKh8hLvvo6MfLK/bhqq61OI2IiPWKi7bzduU+AKqKLuLnIztZnKj5UfkIcaO6JtM1JYZydx2zV+yzOo6IiOVmL/k9bptBx2pwtJzAoPbaSt3fVD5CnM1m8LMRvlb/7Oe7cdd5LE4kImKdyvJD/K90MwBhxUPIHtVFW6lbQOWjGbiqbxtS4yI5VOZm7rp8q+OIiFjmjcUP4rIZtKkxKQm7kUt7pFgdqVlS+WgGwh02fjLMd/967uKv8Xi15bqIND+17gpeKFwKQKsjWdw5shs2m0Y9rKDy0UzcPKQdcZEOdh2uYL4OnBORZui9zx/hoN2gVZ2XPbU/ZHz/dKsjNVsqH81ETISDiUM7ADBj4U4dOCcizYrXU8esvR8AkFGSyR0jehEZZrc4VfOl8tGMTLwwk6gwO5sOuFiyo8jqOCIifrNo5TR22U1aeL18XfFDbh7SzupIzZrKRzPSskU4N5/n+x9uxmc7LU4jIuIfptfLf7fOBqBraSrXDx1ITITD4lTNm8pHM3PniEzC7AYrdh9hzd4jVscREWlyaze+yJe2WsK9JrtcN9dPQYt1VD6ambT4KCYcXWQ187OvLU4jItL0nv0yF4CeZfFcNnAoiS3CLU4kKh/NUPbIThgGfLL1EFsKXFbHERFpMtt3fsBisxybabLvyLXcOVwHyAUClY9mqGPrGH6QlQbAkws1+iEioWvWir8B0Ks8igt6X0JqfKTFiQRUPpqtX4zybbn+7oZ89hZXWJxGRKTx5eev5oPawwAUF4/VAXIBROWjmcpKj2dk19Z4TY1+iEhoeuHzP+IxDLpV2unSZTyZSS2sjiRHqXw0Y5Mu7gzAm2v3c6C0yuI0IiKNp+TI17xZsRuAmuKR3H30+50EBpWPZmxQh5Zc0LEVtR6TXI1+iEgIeWXRg1TbDDq4ISXjh3RPjbM6knyLykczN2m0718Dr67O46Cr2uI0IiLnrrKyiNmlGwGIKh7MPZd0sziRfJfKRzN3QcdWDGqfSE2dl6cW7bI6jojIOXt70e9x2gzSak1iUn9KVnq81ZHkO1Q+mjnDMJg0ugsAs1fupajcbXEiEZGzV1tbyXMFSwBIOtKTSaN7WJxITkblQxjRJYm+beOprvXynyUa/RCR4PXh53+h0G6QWOclvGU2/dslWh1JTkLlQ3yjHxf7Rj9eXLaXkooaixOJiDSc6fXy7O53AGhf0oG7RvezNpB8L5UPAWB0j2R6psVRWePh2S92Wx1HRKTBlqz6FzvtJtFeL7T4OedltrQ6knwPlQ8Bjo1++O58ee6LPTgray1OJCLSMP/56kUAupWmkH3pUIvTyKmofEi9y3ql0j01ljJ3Hf/9XGs/RCR4rN/4MutttYSZJkb4TxjaqZXVkeQUVD6kns1mcM/RO1+e/WIPpZVa+yEiweE/a6cD0NMVx88uuwzDMCxOJKei8iHHOTb6Ue6u45klWvshIoHv668XsJhyDNMk0v4jjXoEAZUPOY7NZjD5kq4AzPpit+58EZGA959ljwLQqyKSH192g0Y9goDKh5xgTM8UeqTFUVHj0b4fIhLQCgvW8VHdYQDiuZYLNOoRFFQ+5AS+0Q/f2o/nl+7hiEY/RCRA/XfRH6gzDLpW2rn98p9p1CNIqHzISY3pmUKvNhr9EJHA5Szdw9wq39q0NuYYjXoEEZUPOSnD+Gbtx/NL91CsM19EJMA898n9VNkMOrjhljG/sTqONIDKh3yvS3ok0zs9nsoaD08t1uiHiASOqsojvOHcAEDH2qFc0Lm1xYmkIVQ+5HsZhsGvLvWt/Xhh2R4OuaotTiQi4vPyxw9QareRUuvl5ssftjqONJDKh5zSRd2SGdAugepaL9M/22l1HBER6mqrefXw5wBk1fbj/E5pFieShlL5kFMyDIN7L+sGwCsr95F3pNLiRCLSbHk9sHsJb71xO4UOgwSPl9vGTLU6lZwFlQ85raGdkriwcytqPSZPfLLD6jgi0szMXD+T3I8nw7QszOev5LWy9QDc6Kpi5e7Hmbl+pqX5pOFUPuSM/HqMb/TjzbX72XW43OI0ItKc2Iq2M+PAJ+TayvkiKpJtEeFEe714jDpmHPgEW9F2qyNKA6l8yBkZ0C6R0d2T8Zrwz481+iEifuL1kL3hI3JKSpmRmMCfWiUC0LWmhmcS4skpcZK9Yb5vSkaCRoPLx+LFixk3bhxt2rTBMAzmzJlz3MdN0+Thhx+mTZs2REVFMWrUKDZv3txYecVCU8b49v1458t8thS4LE4jIs3C3qXgyie71MU1rnLyw8LANFkfGUlOSSnZpU5wHfBdJ0GjweWjoqKCvn37Mn369JN+/G9/+xuPP/4406dPZ9WqVaSmpnLppZdSVlZ2zmHFWr3axHNFH9+q8sfma5hTRPyg/CAAHmBLRLjvOcMgzDTJLnWdcJ0EhwaXj7Fjx/LnP/+Za6655oSPmabJtGnT+N3vfsc111xDVlYWzz//PJWVlcyePbtRAou1fnVJV2wGfLzlIGv2llgdR0RCXUwKAHNiWtSXjzDTpNYwyE2IO+E6CQ6NuuZj9+7dFBYWMmbMmPrnIiIiGDlyJEuXnnxIzO1243K5jntI4OqcHMN1A9sC8NcPt2KapsWJRCSktR+KMy6N/3d0rceFlVWs3ZNXvwYkNyEe4tKh/VCLg0pDNGr5KCwsBCAl5fgGmpKSUv+x75o6dSrx8fH1j4yMjMaMJE1g8iVdCXfYWLn7CAu3HbY6joiEMpudOxNiqLbZSKzz8O+Dvu852aUuckqczEiMJ7fPGLDZLQ4qDdEkd7t890hj0zS/95jj+++/H6fTWf/Iy8trikjSiNokRDFxaAfAN/rh9Wr0Q0SaxradC9hq8x3t8Mcj1YR962PZ3hhy0kfjTepqTTg5a47G/M1SU1MB3whIWto3290eOnTohNGQYyIiIoiIiGjMGOIHd43qxCsr97G1sIy5Xx5gQv+2VkcSkRBjer08uuh+TIdB78oIhkxaDQUrfYtLY1Kg/VCyNeIRlBp15CMzM5PU1FQWLFhQ/1xNTQ2LFi1i6FDNx4WShOhwfjGqE+C788Vdp3vsRaRxzV/6D9Y63ISZJqM7/Y7oyAjIHA69r/P9qOIRtBpcPsrLy1m/fj3r168HfItM169fz759+zAMg8mTJ/Poo4/y9ttvs2nTJiZOnEh0dDS33HJLY2cXi90xNJOUuAj2l1Qxe8U+q+OISAipcZfxz20vADCwLI3bRl9tcSJpTA0uH6tXr6Z///70798fgClTptC/f3/+8Ic/AHDfffcxefJk7rrrLgYNGsSBAweYP38+sbGxjZtcLBcVbuee0b651n9/upOy6lqLE4lIqPjv+zkccBi0qvPyg/MeI8yuDblDiWEG2L2SLpeL+Ph4nE4ncXFxp/8EsVSdx8uYfy5mV1EFv7y4M1OOngEjInK2Dh3cxJXv30SVzWC4azDTc/6LzXbymxYkcDTk/VtVUs6Jw27jN5f5Csd/luzmoKva4kQiEuz+Pv9uqmwGnaoNbhrzqIpHCFL5kHN2eVYqA9snUlXr4XFtuy4i52D9ptl86C0GoB0/YkS3VIsTSVNQ+ZBzZhgGD/ygBwCvrcnToXMicla8njoeXfk3APq5Ysm5+hcWJ5KmovIhjWJg+0Su6J2GacLUD7ZaHUdEgtCcz+5ni91DC6+X9kn/R7dU3agQqlQ+pNHcd3k3wuwGi7cfZvF2bbsuImeuzHWAf+37AIBuRzrzyyvGnOYzJJipfEijad+qBbee3wGAR9/fgkfbrovIGXryo7s4YjdIrzHp2+sRkuMirY4kTUjlQxrVpIs7ExfpYGthGW+u3W91HBEJArv3LOSViq8BSCodzc8v6mVxImlqKh/SqBJbhDPp4i4APDZ/G5U1dRYnEpFA99dFv6XOMOhZEcaVI39Ni4hGPXZMApDKhzS624a2J6NlFAddbnIX7bI6jogEsMUr/skXVOIwTQz3T7h+UIbVkcQPVD6k0UU47Nw/1nfr7VOLvuZAaZXFiUQkENW4y/jr5mcB6F2STM6VN2DXhmLNgsqHNImxWamcl9kSd52Xv+rWWxE5iZfm38M+OyTWeYlO+A3Du7S2OpL4icqHNAnDMPjDlT0xDJj3ZT5r9h6xOpKIBJDDhzbzVNFKANoW9eO+K4dbnEj8SeVDmkxWejw3DPTN3z7yzld4deutiBw1bcEvqbQZdKw26NHjt3RsHWN1JPEjlQ9pUvde1o2YCAdf7nfy9roDVscRkQCwYfOrzKs7BICjZDy/vLSHxYnE31Q+pEm1jo0g56LOAPzto61UuHXrrUhz5vXUMXXl/wOgr6sFE0b9hPioMItTib+pfEiT+/GwDrRrGc1Bl5snF35tdRwRsdA7ix5kk62OaK8Xd10ONw3WrbXNkcqHNLkIh73+1Nunl+xib3GFxYlExAoV5YVM2/MuAF2LM5k8biwOu96GmiP9Vxe/uKxXCsO7JFFT5+VP735ldRwRscBTH95Fkd0grcZLfPJ9XNg5yepIYhGVD/ELwzB4aFwvHDaDj7cc4tOtB62OJCJ+tHfvEl4s3w5AQtFI7r9qsMWJxEoqH+I3nZNj+PGwTAD++M5XVNd6LE4kIv7yt4X3UWcY9KhwMGzQPaQnRFkdSSyk8iF+NeniziTHRrC3uJL/fr7b6jgi4gdLVj7BYspxmCaeion8bFRnqyOJxVQ+xK9iI8PqF59O/3Qn+Tr3RSSk1bor+Oum/wDQu7Q1d115E5FhdotTidVUPsTvru7XhsEdEqmq9fCX97dYHUdEmtDLC+5h79HzW6LifsPoHilWR5IAoPIhfmcYBn+8KgubAe9tKODzHUVWRxKRJlBUtJWnDi8DIKO4D/dfNdLiRBIoVD7EEj3bxHHr+e0B+P3cTVp8KhKCps2/m3Kbjcxq6Nf7QToktbA6kgQIlQ+xzK8v60br2Ah2F1Xw1KJdVscRkUa0ecubzKspBCCmbAKTLulucSIJJCofYpm4yDB+f2VPAGYs3MmeIu18KhIKTK+Xvyz7M6Zh0McVzZ1X3q1FpnIclQ+x1Lg+aQzr7Nv59PdzN2GaptWRROQcvbPo92y01xHl9dI69tdc1D3Z6kgSYFQ+xFKGYfDI1b0It9tYsqOI9zYWWB1JRM5BRXkh/9w9F4CeJR347YRxFieSQKTyIZbr2DqGX4zqBMAj73xFWXWtxYlE5Gw9+f7R81tqTS7s/yhp8drJVE6k8iEB4RejOtGhVTSHytw8Nn+71XFE5Czs2/c5syt9//92qLiYO0b0tjiRBCqVDwkIkWF2Hrk6C4Dnl+1h3b4SixOJSEM9+sm91BoGPSod/Oyqh3DY9RYjJ6e/GRIwRnRtzYT+6Zgm3P/WRmo9XqsjicgZWrR8Ol/YKnCYJr0SJjEos5XVkSSAqXxIQHnwih4kRoextbCMpxdr7w+RYFBbW8nfNz0FQP+y1kyZcKvFiSTQqXxIQGkVE8Efxvn2/vjXJzvYdbjc4kQiclJeD+xeAhvf4L+v3cLeMEjweLl26D+JjQyzOp0EOIfVAUS+a3y/dN5el8/i7Ye5/62NvHLn+dhshtWxRASYuX4mtqLtZG/4CFz5HLHZeKFtG7DbyKhLYK99KdDP6pgS4DTyIQHHMAz+Mj6LqDA7K3Yf4dXVeVZHEpGjbEXbmXHgE3JtvlHJJxITKLPbSKqrY2OEC1uR7laT01P5kICU0TKaX4/pCsCj72/hkKva4kQigtdD9oaPyCkpZUZiAn9qlchbsb7D4oocDnJKnGRvmO+bkhE5BZUPCVh3XJhJ37bxlFXX8cDb2npdxHJ7l4Irn+xSF3eVlPJaXCym4ZsSzSkpJbvUCa4DvutETkHlQwKW3Wbw1+v6EGY3+HjLQeZ9mW91JJHmrfxg/U+jvd/8YyDMNMkudZ30OpGTUfmQgNY9NY5fXtwFgIfmbeZQmaZfRCwTkwLA5vAwHm+ZAIDdNKk1DHIT4k64TuT7qHxIwMse1YlebeIorazlQU2/iFin/VDK49L4eWoyXsOgY00N6/bk1a8ByU2Ih7h0aD/U6qQS4FQ+JOCF2W384/q+OGwG8786yDsbdPKtiCVsdn4UH4XTbifG4+GFgkMYQHapi5wSJzMS48ntMwZsdquTSoBT+ZCg0CMtjknHpl/mbuJwmdviRCLNz/8++i1fO+owTJPHit3Ee785AiHbG0NO+mi8SV0tTCjBQpuMSdC466JOfLi5kC0FLn4/ZxNP/mgAhqHNx0T8Ye/eL/hn/rtgszG8sgPn/2oO5C3zLS6NSYH2Q8nWiIecIY18SNDwTb/0wWEz+HBzIW+vO2B1JJFmodZdwa8/votKm42uVTZ+de2L2BwOyBwOva/z/ajiIQ2g8iFBpVebeO4ZfWz6ZTMHSqssTiQS+v7+9i1sc3iJ9XgZ23EqnVMSrY4kQU7lQ4LOL0Z1on+7BMrcddz72pd4vbr7RaSpLF75b15x+06YPs99CT++ZKzFiSQUqHxI0HHYbTx+Qz+iwuws21XMrKV7rI4kEpKKirby4KZcAAa6Evm/W/6qQx6lUah8SFDKTGrB767oAcBfP9zK9oNlFicSCS1eTx3/985tlNhttHObXH3BU6TGR1odS0KEyocErR8Oaceobq2pqfPyq1fXU1PnPf0nicgZmfX+z1lhqyLCa9IrfBITBvWwOpKEEJUPCVqGYfC3a/uQEB3G5nwX//xYR3mLNIZNX73O9OIVAPQtyeKBG+6wOJGEGpUPCWrJcZFMndAbgNxFX7N0Z5HFiUSCW0V5Ib9Z9gh1hkFWeQS3X/EvEqLDrY4lIUblQ4Le2N5p3DQ4A9OEX722niMVNVZHEglaf3z7JvY7ILnWS5/0vzGimw6Jk8an8iEh4Q/jetKpdQsOutzc98YGHT4nchbmfvoAH3iLsZsmmZXX8+srR1kdSUKUyoeEhOhwB0/c3J9wu42PtxzkpeV7rY4kElT27l3Co3vnAtDvSAYP3Hwv4Q69RUjTaPS/WQ8//DCGYRz3SE1NbewvI3KCXm3i+e3Y7gD8+b0tbCvU7bciZ6LWXcG9n0yq3z597LDpdGwdY3UsCWFNUmt79epFQUFB/WPjxo1N8WVETvDjCzswqltr3HVeJr2ylqoaj9WRRALe43NvYavdQ5zHS4eoB7hhcEerI0mIa5Ly4XA4SE1NrX+0bt26Kb6MyAkMw+Af1/clKSaC7QfLeWjeJqsjiQS0xSv+xUtVvu3Tu5RcxB9uuEanRUuTa5LysWPHDtq0aUNmZiY33XQTu3bt+t5r3W43LpfruIfIuUiKieCJm/phM+C11ft5c81+qyOJBKSiw1t4cPPTAPQvTWDyDX8mPirM4lTSHDR6+RgyZAgvvPACH330Ef/5z38oLCxk6NChFBcXn/T6qVOnEh8fX//IyMho7EjSDA3tnMQ9o7sC8OCcTezQ9usix/F66rjvXd/26e3dMLLPDPplJFgdS5oJw2ziexIrKiro1KkT9913H1OmTDnh4263G7fbXf9rl8tFRkYGTqeTuLi4powmIc7jNbn92ZV8vrOIzskxzLv7QqLDHVbHEgkIT82dyPTSNUR4TUZ4J/HYj3+m6RY5Jy6Xi/j4+DN6/27y+6hatGhB79692bFjx0k/HhERQVxc3HEPkcZgtxlMu6kfybER7DxUzoNvb9L+HyLAhs2vkVuyGoB+zr48dMuPVTzEr5q8fLjdbrZs2UJaWlpTfymREyTFRPDvm/tjM+CtdQd4dVWe1ZFELFVeVsBvlvu2T+9dHsk9107XOg/xu0YvH/feey+LFi1i9+7drFixguuuuw6Xy8Xtt9/e2F9K5IwM6diKX4/pBsAf5m5mfV6ptYFELGJ6vTz45o3kOwySa72M6fkEvTMSrY4lzVCjl4/9+/dz8803061bN6655hrCw8NZvnw57du3b+wvJXLGfjGyE5f2TKHG4+UXL62hqNx9+k8SCTGvzP8tnxgl2E2TIfbbuH3U+VZHkmaqyRecNlRDFqyINERZdS1XT/+CXUUVnN+xJS/9ZAgOu7aPlhDm9cDepVB+kB1lBfxwWy5VNhvDXJn8M/ttIsPsVieUENKQ928t/ZdmIzYyjKduHcj4GV+wfNcR/vrhVn53RU+rY4k0upnrZ2Ir2k72ho/AlU8N8Ls2qVRFhJNca5IxYJSKh1hK/+yTZqVLSiz/uL4vAP9Zspt3vsy3OJFI47MVbWfGgU/ItZUDMK1lAlsiwon0ejkUZtCyap/FCaW5U/mQZmds7zR+MaoTAPe9sYHN+U6LE4k0Iq+H7A0fkVNSyozEBO5PasmL8b4h8GqbjZwSJ9kb5vumZEQsovIhzdK9Y7oxvEsSVbUe7nx+NYfLtABVQsTepeDKJ7vUxW1OF+/GfnM6bU5JKdmlTnAd8F0nYhGVD2mW7DaD6TcPoGNSC/Kd1fz8xdVU1+pfghICyg8CcMRmY3FUVP3TYaZJdqnrhOtErKDyIc1WfHQYz9w+iLhIB2v3lfLAWxu1A6oEv5gUKg2DnNTW7An3bR4WZprUGga5CXHHXSdiFZUPadY6to5h5g8HYrcZvLXuAE8t/v4TmEWCQW1qX+5JSWZTRAQAtzhdrN2TV78GJDchHuLSof1Qi5NKc6byIc3esC5JPDTOd8vtXz/cyoKvNBwtwcnrqeNXr/yA5VG+4nGdq4z7j5QCkF3qIqfEyYzEeHL7jAGbbrUV66h8iAC3XdCBH53fDtOEX76yjg37S62OJNIgptfLQ6+MY5HdiWGaDPGk8VBt1HHXZHtjyEkfjTepq0UpRXy0yZjIUQ+N68W+I1Us3n6YHz+3irfvupCMltFWxxI5I0+8dStzPPsBuKxmGH/96ZOAt36HU2JSoP1QsjXiIQFAIx8iR4XZbcz84QB6psVRVF7DxFkrKa2ssTqWyGm99P5knqnYAMDI8u786Y4Z2GyGb2olczj0vs73o4qHBAiVD5FviYlwMOuOwaTFR/L14Qp+9uIa3HW6BVcC14dLpvL3Qx8DcL4rlUcmvqyt0yXgqXyIfEdKXCSz7hhMbISDlbuPcO/rG/B6dQuuBJ4V657ldztfxmsYDCiL5YEb36Jli3CrY4mclsqHyEl0T40j99aBOGwG73yZz5/e+0p7gEhA2bbjPSavf4wam0FWRTh3X/EWmcmxVscSOSMqHyLf48LOSfWH0M36Yg/TP91pcSIRn7z9y/j5kvsot9noUmXj1qGzGZyZanUskTOm8iFyCuP7p9fvAfLYgu28uHyvxYmkuSsu2s7PPrqTYruNDLfJdb2e4gf9ulkdS6RBVD5ETuOOCzP55cWdAfjD3E2882W+xYmkuaooL+Rnc69nv8MgudbLVe3+wS3Dz7c6lkiDqXyInIFfXdq1fhOyKa+tZ9H2w1ZHkmamxl3GXa9eyXaHl3iPlzEJD5B9+eVWxxI5KyofImfAMAz+eFUWV/ZJo9Zj8rMXVrP06yKrY0kz4amrYcrsy1nrcBPl9XKx46fcd+0tVscSOWsqHyJnyG4zePyGflzcPRl3nZefPLealbuPWB1LQo3XA7uXwMY3YPcSzLpafj97HItsLhymyUWeCTz0o8kYhmF1UpGzZpgBdv+gy+UiPj4ep9NJXFzc6T9BxM+qaz3c+cJqluwookW4nRd/OoQB7RKtjiVBbub6mdiKtpO94SNwfbOu6MmWScyMjwbT5HL3CB796XTC7Pp3owSehrx/62+wSANFhtn5z22DGNqpFRU1Hm5/dqUOopNzZivazowDn5BrK69/7vXYFr7iAfSqieUvP1HxkNCgv8UiZyEyzM4ztw/ivA4tKauu49b/rmTjfqfVsSRYeT1kb/iInJJSZiQmkJsQx8fRUfypVUsABlZV80q5k3BbQA1Ui5w1lQ+RsxQd7uDZOwYzoF0CzqpabnlmOWv2llgdS4LR3qXgyie71FVfQH6VnIRpGPSsdjOr8BCG64DvOpEQoPIhcg5iIhw8/+PzvjUCsoJlXxdbHUuCTfnB+p+2r60D0wTDwDBNXi44iHGS60SCmcqHyDmKjQzjuR8PZljnJCprPEyctVL7gEjDxKRgArPiY7kvOam+eJiGwTMJccddJxIKVD5EGkF0uINnbh9Ufxvunc+vZsFX+leqnBlP+iD+ktSax1v67prqU13Nuj1531oDEg9x6dB+qMVJRRqHyodII4kMs5P7o4GMzUqlxuMl+6U1vLY6z+pYEuCqKo+Q/eIoXo2NAuDCyipeLjiEHY6uAXEyIzGe3D5jwGa3NqxII1H5EGlE4Q4b/765P9cOaIvHa3LfGxuYuXAnAbadjgSI4uId3Db7IpY7KrGZJgNrW/FkVdhx12R7Y8hJH403qatFKUUan8PqACKhxmG38Y/r+9A6NoLcRV/ztw+3ccjl5g9X9sRm066U4rNrzxLu+uQXHAgziPN4uS7ux0y+dgqG6fXd1VJ+0LfGo/1QsjXiISFG5UOkCRiGwf+N7U7r2Aj+9O5XPLd0D0Xlbh67oS8RDr2RNHervnyZKWsepdRhI6XWy63tHuL2MTf4PmjYIXO4tQFFmpjKh0gT+smwTFrHRvDr19bz7oYCDrnc5N46kJYtwq2OJhaZ99lfeGTPK7jtNjLdcGefpxh3/jCrY4n4ldZ8iDSxq/q2YdbE84iNcLByzxHGz/iCnYfKrI4lTe07B8Th9fDk2z/jwb2v4LYZ9KoM4/6Rb6p4SLOkg+VE/GTHwTJ+/Pwq8o5UERvpYOYPBzC8S2urY0kjO9kBcV7gsVZJvBDnO6flvPIEHrxuLpkpLS1MKtK4dLCcSADqkhLLnLsuZHCHRMqq65g4axUvLtujO2FCzHcPiHMb8NvWreqLR3d3FI/f8bGKhzRrWvMh4ketYiJ46adDuP+tjby19gC/n7uZDfud/Gl8FpFhWoga9I4eEIetnBmJCbgNg3UREayJigTgkvIKHnd7MCL1rVeaN418iPhZhMPOY9f35f/GdsdmwOtr9nNd7lLyjlRaHU3O1bcOiPuh08UzCfH1xePqsnL+ebhYB8SJoPIhYgnDMMge2YkXfzKEli3C2XTAxbjpn7Nw2yGro8m5KD+ICbwe24K3YmPqn3aYJn8uOnLcdSLNmcqHiIUu7JzEO5OG0bdtPKWVtdzx3Coen7+NOo/X6mhyFgrr3PwyOYlHklpRZfN9ew0zTeoMg1wdECdST+VDxGLpCVG8ln0BN5/XDtOEJz7dyY1PL9c0TJCZ8+lfuWHdoyxsEY3t6CLiu0pKWasD4kROoPIhEgAiHHamXtObJ27uT2yEgzV7S/jBE0t458t8q6PJaVRWFvObWZfw+7yXKLHbSKjz4jUMckqc/KLUBeiAOJHv0pJrkQByVd829M9I4J7/rWPtvlImvbKORdsP84dxPYmLDDv9byBNx+s54cyVlRvf4uFVfyQvzHdmz7DKFNr1GUNiVT7ZxR8BzvpPz/bGgA6IEwG0yZhIQKrzeHnikx1M/2wnXhPS4iN59JreXNQt2epozc7JNg3zAM8kJjEzPgqvYdCqzstVsbdwz3UPYD92eOBJyopGPCSUNeT9WyMfIgHIYbcxZUw3hndtzW9e/5I9xZXcMWsV1w1sy++v6El8tEZB/OXYpmHYyskGDjjsPNC6FWsjfbfQtq0xeGjY/zi/R+/vfKIOiBP5Phr5EAlwVTUe/jF/G89+sRvThOTYCB65OovLeqVgGIbV8UKb1wPTssg9umnY6IpKlkdFUnH0TpbRFZU8Xh2BbfJGjWpIs6ft1UVCSFS4nd9f2ZPXf34BHZNacKjMTfZLa7jjuVXsKaqwOl5oO7pp2LDKatJqa/mkRXR98bjV6WLaoSJs2jRMpMFUPkSCxKAOLXn/nuHkXNSJMLvBwm2HGfPPxTw+fxvVtR6r4wWvk5w+e8yevcu4v3Urbk5PpSAsDI4OFIeZJvcdKf3m99CmYSINomkXkSC063A5D83bzJIdRQC0TYziN5d1Y1yfNthsmoo5EydbSApAXBue6DGKJYc2s9tbjPvoSEc3dw3bIsIJM01qDYOcklKyj95Ky+3van2HNHuadhEJcR1bx/DCj8/jyR8OIC0+kv0lVdzzv/VcNeNzPj9aSOTUvnv6LPjuYvllZA3/OfQ5WynBbbPRv9rN9a4ytkWEk6NNw0Qahe52EQlShmEwtncaI7u15tnPd5O7aBebDrj40X9XMLxLEvdd1p3ebeOtjhkYvnvba8aQ406fBehX7eb/Wrei2OH7tphRW8foFuOITK0mt/Azckqc9SMdvh8NZiTGQ/posrXYVKRBNO0iEiKKy91M/2wnLy3fS63H97/1yK6tufvizgzu0NLidNb43qmV6CRyw2vwYLDfYefdbx0CF+718ssSJze7ygi//V1mOjd+z/RMOrl9xuBN6spd/e7y459KJDA15P1b5UMkxOwrrmTax9uZ+2U+Hq/vf+/zMlty90WdGd4lqVndnpv78WRmHPjk+PUZwPSEOJ5KTCCxro4SxzcDwDbTZOG+AyR6jx7sd+1/ofd1vp9r0zCRU1L5EBH2FVfy5KKveWNNXv1ISJfkGG67oD0TBrQlJiIEZ12/XRBatIa3f06uvYIZiQnklJRyk6uce5NbsSIqqv5Twrxeam02LSQVOUcqHyJSr8BZxdOLd/Hqqjwqa3y3kcZEOLhuYFtuGdKOrimxFic8d987vQLkJsTxWXQUX0VE+G6VPTryk1JXR6eaWpZGR9UXjtyEuKNFxek7i0Wbh4mcMW2vLiL10uKjeGhcL351aVfeWrOfF5btZVdRBc8t3cNzS/eQlR7H+H7pXNWvDcmxkVbHPSvf3QIdIM9h5+Gklqz81igHhoFhmjx6uJh9YQ6ePDoiooWkIv6lkQ+RZsbrNfni6yJeWLaXz7Yeou7ouhCbAcO6tOayXimM7p5CanyQFJHvbIE+sqISl93Gushv8ju8XupsNhymSd3RqRUvBjZMsmvCoLL4m99PC0lFzkpATLvMnDmTv//97xQUFNCrVy+mTZvG8OGnnztV+RDxnyMVNby3IZ+31h1g3b7S4z6WlR7H6O4pjOrWmqz0eMLsgbctUE2dl92rPqTbRzcDcGdKa5ZHfzPSkVFbS9ca37bo3zu18sv1kLdCC0lFzpHl5ePVV1/l1ltvZebMmVx44YU89dRTPPPMM3z11Ve0a9fulJ+r8iFijT1FFby3sYCPtxxkfV4p3/7OEB1uZ0C7RM7LbMngDi3plR5HXKT/T9Z1Vtay8YCTlXuOsGr3EdbllTDG8zlPhE8HfCfOjm3bBtMwCDNNflbqrF9s+u27XXIT4pmRGE9O+miyL5nm9z+HSCiyvHwMGTKEAQMG8OSTT9Y/16NHD8aPH8/UqVNP+bkqHyLWKyp38+nWQ3yy5SDLdx3BWVV7wjVtE6PonhpHj7RYOifH0DYxivSEaFrHRmA/hy3ePV6Tg65qDpRWcaCkiu0Hy9haWMbWAhf5zuoTrr80ejv/8T4MUD+qcezOlcFV1ZxXXX1c8QA0tSLSBCwtHzU1NURHR/P6668zYcKE+ufvuece1q9fz6JFi075+SofIoHF6zXZcaiclbuLWbH7CGv3lpy0BBwTZjdIjY8kMTqcuMgwYiMdxEWGERVu59tbjJgmVNbUUVZdh6u6FldVHUcqaih0VdfvT3IybROjGNQ+kfMyW3FeZiKdWkVh/Ks3ubYK32jG902vXD0TKos0tSLSRCy926WoqAiPx0NKSspxz6ekpFBYWHjC9W63G7fbXf9rl8t1wjUiYh2bzaBbaizdUmO59YIOgG/6Y2uhiy0FLrYUlLG7uIIDJVUUuqqp9ZjkHaki70jVWX9Nh80gLSGS9IQoMpNi6JkWS/e0OLqlxp50uie3z2VHNxM7xRbonUaddR4RaVxNdqvtd3dRNE3zpDsrTp06lT/+8Y9NFUNEmkB8dBhDOrZiSMdWxz1f5/FysMxNQWkVzqpaXNW1vpGNqlqqaj0n/D7R4Q7iIh3EHh0hSYgOo01CFMmxkQ2auvEmdSUHyC7+CHDWP5/tjYH00XiTup7tH1VEmoDl0y4nG/nIyMjQtIuINJy2QBexjKXTLuHh4QwcOJAFCxYcVz4WLFjA1VdffcL1ERERRERENHYMEWmObHZthy4SBJpk2mXKlCnceuutDBo0iAsuuICnn36affv2kZ2dffpPFhERkZDWJOXjxhtvpLi4mEceeYSCggKysrJ4//33ad++fVN8OREREQki2l5dREREzllD3r8Db79kERERCWkqHyIiIuJXKh8iIiLiVyofIiIi4lcqHyIiIuJXKh8iIiLiVyofIiIi4lcqHyIiIuJXKh8iIiLiV02yvfq5OLbhqsvlsjiJiIiInKlj79tnsnF6wJWPsrIyADIyMixOIiIiIg1VVlZGfHz8Ka8JuLNdvF4v+fn5xMbGYhiG1XEs53K5yMjIIC8vT2fdNCG9zv6h19l/9Fr7h17nb5imSVlZGW3atMFmO/WqjoAb+bDZbLRt29bqGAEnLi6u2f/F9ge9zv6h19l/9Fr7h15nn9ONeByjBaciIiLiVyofIiIi4lcqHwEuIiKChx56iIiICKujhDS9zv6h19l/9Fr7h17nsxNwC05FREQktGnkQ0RERPxK5UNERET8SuVDRERE/ErlQ0RERPxK5SMIud1u+vXrh2EYrF+/3uo4IWfPnj385Cc/ITMzk6ioKDp16sRDDz1ETU2N1dGC3syZM8nMzCQyMpKBAweyZMkSqyOFlKlTpzJ48GBiY2NJTk5m/PjxbNu2zepYIW/q1KkYhsHkyZOtjhI0VD6C0H333UebNm2sjhGytm7ditfr5amnnmLz5s3885//JDc3lwceeMDqaEHt1VdfZfLkyfzud79j3bp1DB8+nLFjx7Jv3z6ro4WMRYsWkZOTw/Lly1mwYAF1dXWMGTOGiooKq6OFrFWrVvH000/Tp08fq6MEFd1qG2Q++OADpkyZwptvvkmvXr1Yt24d/fr1szpWyPv73//Ok08+ya5du6yOErSGDBnCgAEDePLJJ+uf69GjB+PHj2fq1KkWJgtdhw8fJjk5mUWLFjFixAir44Sc8vJyBgwYwMyZM/nzn/9Mv379mDZtmtWxgoJGPoLIwYMHufPOO3nxxReJjo62Ok6z4nQ6admypdUxglZNTQ1r1qxhzJgxxz0/ZswYli5dalGq0Od0OgH0d7eJ5OTkcMUVV3DJJZdYHSXoBNzBcnJypmkyceJEsrOzGTRoEHv27LE6UrPx9ddf8+9//5vHHnvM6ihBq6ioCI/HQ0pKynHPp6SkUFhYaFGq0GaaJlOmTGHYsGFkZWVZHSfk/O9//2Pt2rWsWrXK6ihBSSMfFnv44YcxDOOUj9WrV/Pvf/8bl8vF/fffb3XkoHWmr/W35efnc/nll3P99dfz05/+1KLkocMwjON+bZrmCc9J47j77rvZsGEDr7zyitVRQk5eXh733HMPL730EpGRkVbHCUpa82GxoqIiioqKTnlNhw4duOmmm3jnnXeO+0bt8Xiw2+388Ic/5Pnnn2/qqEHvTF/rY99M8vPzueiiixgyZAjPPfccNpu6+tmqqakhOjqa119/nQkTJtQ/f88997B+/XoWLVpkYbrQM2nSJObMmcPixYvJzMy0Ok7ImTNnDhMmTMBut9c/5/F4MAwDm82G2+0+7mNyIpWPILFv3z5cLlf9r/Pz87nssst44403GDJkCG3btrUwXeg5cOAAF110EQMHDuSll17SN5JGMGTIEAYOHMjMmTPrn+vZsydXX321Fpw2EtM0mTRpEm+//TYLFy6kS5cuVkcKSWVlZezdu/e45+644w66d+/Ob3/7W01znQGt+QgS7dq1O+7XMTExAHTq1EnFo5Hl5+czatQo2rVrxz/+8Q8OHz5c/7HU1FQLkwW3KVOmcOuttzJo0CAuuOACnn76afbt20d2drbV0UJGTk4Os2fPZu7cucTGxtavp4mPjycqKsridKEjNjb2hILRokULWrVqpeJxhlQ+RL5j/vz57Ny5k507d55Q7DRQePZuvPFGiouLeeSRRygoKCArK4v333+f9u3bWx0tZBy7jXnUqFHHPT9r1iwmTpzo/0Ai30PTLiIiIuJXWkEnIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+9f8Bor1Qczd146QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_39_2.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def function(x):\n",
    "    return x**2\n",
    "\n",
    "def analytical_gradient(x):\n",
    "    return 2*x\n",
    "\n",
    "def gradient_descent(starting_point, learning_rate, num_iterations, solver=\"analytical\"):\n",
    "    x = starting_point\n",
    "    trajectory_x = [x]\n",
    "    trajectory_y = [function(x)]\n",
    "\n",
    "    if solver == \"analytical\":\n",
    "        grad = analytical_gradient    \n",
    "    elif solver == \"jax\":\n",
    "        grad = jax_grad(function)\n",
    "        x = jnp.float64(x)\n",
    "        learning_rate = jnp.float64(learning_rate)\n",
    "\n",
    "    for _ in range(num_iterations):\n",
    "        \n",
    "        x = x - learning_rate * grad(x)\n",
    "        trajectory_x.append(x)\n",
    "        trajectory_y.append(function(x))\n",
    "\n",
    "    return trajectory_x, trajectory_y\n",
    "\n",
    "x = np.linspace(-5, 5, 100)\n",
    "plt.plot(x, function(x), label=\"f(x)\")\n",
    "\n",
    "descent_x, descent_y = gradient_descent(5, 0.1, 10, solver=\"analytical\")\n",
    "jax_descend_x, jax_descend_y = gradient_descent(5, 0.1, 10, solver=\"jax\")\n",
    "\n",
    "plt.plot(descent_x, descent_y, label=\"Gradient descent\", marker=\"o\")\n",
    "plt.plot(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a994430",
   "metadata": {
    "editable": true
   },
   "source": [
    "### A more advanced example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "863db98d",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/jax/_src/numpy/lax_numpy.py:173: UserWarning: Explicitly requested dtype float64 requested in asarray is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more.\n",
      "  return asarray(x, dtype=self.dtype)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x11cb23fd0>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGdCAYAAAAvwBgXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAAB8SklEQVR4nO29eZhcZZn3/z21dFUvVb2mO0t3VhAStkgCCQjIJsuIijqMDMgMDvq+/AQGZGZwAiqo+ObngAsygjLziigKuKKOiGFU0BEiISxhSyBJJ+mkt/Ra3dVd+3n/qHrOOd2pqj7L85zzVNf9ua5cF+l0dz0k1efc53t/7++tqKqqgiAIgiAIwgN8Xh+AIAiCIIjqhQoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8I+D1AcqRy+XQ29uLSCQCRVG8Pg5BEARBECZQVRUTExNYvHgxfL7ymofUhUhvby+6urq8PgZBEARBEDbo6elBZ2dn2c+RuhCJRCIA8v8j0WjU49MQBEEQBGGGWCyGrq4u7T5eDqkLEdaOiUajVIgQBEEQRIVhxlZBZlWCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCkIQ9hyfxrWf2IJHOen0U15B6+y5BEARBVBNf3fIWfv1qH9oaQvjrdZ1eH8cVSBEpQzqbQyab8/oYBEEQRJVweCIJAOgZmfL4JO5BhUgJ4skMTtv8e3zkga1IZagYIQiCIMQTS6QBAIOFgqQaoEKkBPuG4xiaTGL7/lF88w+7vT4OQRAEUQVMJDIAgMFYwuOTuAcVIiVIGlSQb/5hN97si3l4GoIgCKIamCBFhGAk03ohksmpuOUnO8gvQhAEQQhDVVVMJvOKyAApIkQikx+dWtJUi2g4gFcPjeM//6fb41MRBEEQ85V4Koucmv/vockksuw38xwqRErAFJGFjWF85pI1AICvPvUW9hye9PJYBEEQxDwlNp3W/junAsOT1dGeoUKkBMmCIhIK+HDZuk6ceXQbUpkc/vWnO5CrkiqVIAiCcA9mVGUMxKgQqWqYWTUU8EFRFGz+0Amor/Fj275R/NerfR6fjiAIgphvMKMqY3CiOnwirhUimzdvhqIouOmmm9x6SUckC/G64aAfANDZXIdLTlwMAOg+HPfsXARBEMT8hBQRgWzbtg0PPPAATjzxRDdejgtGRYRRF8oXJczIShAEQRC8iM1SRKplckZ4ITI5OYkrr7wS//Ef/4Hm5mbRL8cNvRDxax9j6kg1LSMiCIIg3GG2IlItWSLCC5HrrrsO733ve3H++eeLfimusGIjFNT/isKFoiRJke8EQRAEZ1ghoij531dLuqrQ7buPPvooXnzxRWzbts3U5yeTSSSTegUYi3mXZsqKDaaCAHpRQooIQRAEwRtmVu1qrsOBkSlSRJzS09ODG2+8EQ8//DDC4bCpr9m8eTMaGxu1X11dXaKONyfMrGr0iIQL/02KCEEQBMEbpoisWlAPgDwijtm+fTsGBwexbt06BAIBBAIBPPPMM/jGN76BQCCAbPZIVWHTpk0YHx/XfvX09Ig63pwk0keaVUMFdSRJighBEATBGaaIHNXeAKB60lWFtWbOO+88vPrqqzM+9rGPfQzHHnssPv3pT8Pv9x/xNaFQCKFQSNSRLMECzYytmbDWmiFFhCAIguALU0SWt9XDp+jpqu1Rc12FSkVYIRKJRHD88cfP+Fh9fT1aW1uP+LiMFBvf1c2qpIgQBEEQfGGFSFNtDdoaQhicSGIgNv8LEUpWLUGx8d0QKSIEQRCEIFiOSCQcQHs03x2ohnRVoVMzs3n66afdfDlHlB/fJUWEIAiC4AtTRCLhADoiYbyGmNB01b7xabzRG8OCSAgndjYJe525IEWkBKSIEARBEG4yoSkiQa0dI3JyZuveYVzz0Au467e7hL2GGagQKYG2fdegiLCihHJECIIgCJ6oqorJZF4RiYYDaI+w1ow4RSRVeOCu8XtbClAhUoJi47tsgsarHJHx6TRe7x335LUJgiAIccRTWbBJ3Ug4iI6CIiIyXVUrRALelgKuekQqiWLju6wocVsRSaSzePDP+3D/07sRS2Tww49vwOlHtbl6BoIgCEIcrC0T8CkIB32uKCJJKkTkJjmHIqKqKhS2EEAQmWwOP95+EF//77dmGJb2DU/h9KOEvjRBEAThIkajqqIomiIi0iOSyubvc0GPWzNUiJSg+PZd34w/N6olvBmNp/CRB57DWwOTAIAlTbVoCAWwa2ACk8n0HF9NEARBVBJGoyoAdBTGd1m6qt/H/8E3ncn3grxWRMgjUgLWfgkXMasCumIiihcPjOKtgUnU1/jx2UvW4Pf//G6cuqIFADA5a1U0QRAEUdnEDIoIALQ2hGakq4ogVVi1QmZVCVFVtagiEvQrYEWp6CwR5p4+qasJ15yxAqGAHw2FN+hEkgoRgiCI+cTErELE71PQ1pBXRURliaSKJIh7ARUiRWB9M2Dm+K6iKIYRXrGKyFQqX+jU1ejds4ZQ/r9JESEIgphfzG7NABCerirL1AwVIkUwFhmzK0Vt8Z1gRSReUD3qQ7oiwyrlSVJECIIg5hWzFREA6Igww6qo1owcZlUqRIrA2i6KcmTvTJuc8VIRoUKEIAhiXsEUkegMRUTs5EyKzKryYhzdnT2iq2WJiFZEUgVFpEZXRFghMkGtGYIgiHlFbPpIRUR0lghTRMisKiHFjKoM1xSRZEERCRkUEWrNEARBzEuKKSKi01VThQdqUkQkpNjoLsOtdNViikgklH+DklmVIAiCD8OTSSke7op5RIQrImRWlZdyikiooIiIbs2QIkIQBCGWiUQaZ9/1NM7/yjM4NDbt8VlYIXKkIiLMI0KtGXnRNu8WqRLdas2U84hMJjPIse1IBEEQhC0OjExhIplBfyyBf3hwG2IJ71KrY9r4rmFqZla6Km8oWVViWJFRLMLdLbNqsakZ4xuUFSoEQRCEPcam9MJj18AEPvnwi0hnvdmuXqw1IzpdNUmKiLyYUUREB5oVyxEJBXwIFKJdqT1DEAThjJF4CgDQ2VyLuho//mf3ED7z89egqu4rzsUCzUSnq5JHRGI0j0gRs2q48A8mOuK9mCKiKIruEyHDKkEQhCNGp/KFyAlLGnHv374TPgV47IUe3Pf0HlfPoaqq9nAZDc/cRatNzghIV6WpGYlhEzHFzapsakZ0oNmRighgyBIhRYQgCMIRo/G8CtFcX4PzVnfg8+8/DgBw12934c+7h1w7RzyVBbOAGBURQJ+cEaGIpLP5F6VkVQlhikix8d1woTgRrYjEC1Mz9TUzq2PaN0MQBMEHpog01+Vv/ledthzvO2kxAOCPbx927RysLRPwKUfcd0Smq9LSO4nRk1VLKyIip2ayORXTadaamXkG2jdDEATBB+YRaa6r0T521IIGAO4mWBuNqrPTvEVmiWjju1SIyEdZs6q2fVecIjJt+N71IVJECIIgRMAUkZZ6vRBhD3uxafdGeYsZVRki01U1syq1ZuQjUWZ8V8sRyYhTRKYKaodPObIYaii8UckjQhAE4Qy9NXNkIeKmIhIrMrrLYFkiQhSRwn0sSIqIfJRTRHSzqjhFJJ7S/SGzZTpSRAiCIPhgNKsyorWFhz0Xw82KZYgw2iN5RaSfsyKiqiolq8pMsoyBRzerilNEWIZIXehIRUb3iHiXAMg4PJHEuV95Gvf899teH4UgCMIys82qgKE146pHpHRrpqlOTGHEJmYA8ohIiTa+WyxZ1QVFZCpVfGIGmBnz7jXPvHUYew/H8X//Z6+Q+GGCIAhRJNJZ7Vo7QxEJy6WIaAMSmRzXoLWUIUGWpmYkpJwiEnLBrMri24spIlqOiAStmX1DcQD5J4c3emMen4YgCMI8LN494FMQMQwF6IWI+4pItIgiwu45qjpTxXBKyqDqU46IhGjju0XNqnp1Kgpt824xRUSi8d3u4bj238/ucS/8hyAIwilsdLeprmaGF4+pElOprGt7Z8oqIoYHYp75VawQ8fsU+H3KHJ8tFipEilDWrOqiIlJfU8QjIpFZdf+MQmTYw5MQBEFYQx/dnalCNBiKAbeus+YLEX6FUVoSoypAhUhRyo/vio94n9LMqvIqIqqqYt/QlPb757tHZkh9BEEQMsMKkSbD6C6Qb1PUFq79brVnyplVFUXRzKQ8C5GkJAvvACpEimJm+67QqRnNrCqvR2RoMoXJZAaKknd1T6ezeOXgmKdnIgiCMMtooTXTMqsQAYBoLZucccewWi5HBNDvRUmOSrwsm3cBKkSKUt6syv8NMRu28K6YR0SWiHfWllncWIt3HdUGAHh2N7VnCIKoDEaKZIgwmDLhViGit2aOVEQA3RLA8wFYlgwRgAqRojD/h1fJqtrCu6JTM/k36mQyw3WUyyrdhYmZFW31OH1VKwAyrBIEUTkUyxBhuJ2uqrdm5lBEeBYipIjIjRlFJJXNCcvOKKeIMI9INqcK9anMxb6CIrKstQ7vWpVXRF46MIbplNitxARBEDwotmeGwcZo3do3wwqeaKlCJMhfiSezquTohUhpRST/eWJuuuU8InVBP9ik2YSH6ar7hvNG1RVt9VjWWofFjWGksjls3z/q2ZkIgiDMMlrIEWku4hFxUxFRVbWsWRUQ1JohRURuklqyamlFJP95YhSJclMzPp+ChhrvR3hZmNny1nooioLTCqoItWcIgqgEmFm1ub5Ya8a9ULN4KgsmrrvZmqGpGclJZEqP7wb8PgQK4S8J4YpI8Tel1yO8+dHdQiHSVgcABp8IGVYJgpAfFmhWTBFxc2qGqSF+n6KNDc9GL0Q4Ts1Qa0ZeVFXVJKtS+fuaYVWUIlIm4h3wfgPv4ckk4qksfArQ1ZIvRE4rFCI7Do655jQnCIKwy9hUmULExX0zxjCz2dvWGSEB9xx2nwuSIiIfRumrdCFSCDUTpIiwiPe5FJEJjxSR/QV/yOKmWq13ubipFiva6pFTgef3jnhyLoIgCDMk0llNeS4+vuueR2SuiRlATGuGzKoSY6w4i7VmAGPMuxhFRFt6V8SsCniviBhHd42cRu0ZgiAqALbwzu9Tik6qRF3MEdHCzELFjaqAoNbMHMq/m3h/Aslg/9A+BZoXZDYiRqmMaIpIEbMq4H2oGfOHLGutm/FxyhMhCKISMGaIFGuHuKuIlE9VBWhqpuowju6W7NcxRURAqJmqqmWX3gEGRcTj1szy1pmKyMaV+UJkZ/8EhieTrp+LIAjCDKNljKqAu1MzrDUTrS2jiGgPv5SsWhUkyozuMvTFd/wVkWQmp41yFRvfBfR0Va/2zZRqzbQ1hHDswggAYNs+yhMhCEJORsoYVQHD1IwLgWbmFBH+rZmkZlYt/sDtJlSIzIL944SLhJkxwgJkMkbcoHKUGuXSx3fdn05RVdWQqlp/xJ+z4mQglnD1XARBEGbRwsyKZIgAHikiJcLMADGtGd2sWvpe5xZUiMxC27xbRhEJCVREpgpO7tqgH/4SHpWIh2bVwxNJTBVGd5e21B3x5ywuebggfRIEQciGtnm3yMQMoKsTqWxOyHXeiFeKCHlEJIb14Mo5iYUqIswfUiJDBPA20IxFuy9pri36Bm5tCAEAeUQIgpAWZlZtKtGaaagJaKs0RE/OmCpERHhEqBCRF5YNUmp0FxA7NcM27xZbeMdgZlUvPCLGaPditDUUFJFJUkQIgpATTREpUYj4fIpr19m59swAYqdmZBjfLX23q1KsKCJiWjPlM0QAbxWR7uHyhUhrfUERiZMiQhDVSiqTw7Z9I/jz7iGc1NWEC49b6PWRZjBS8Ig01ZW++UfDQUwkMsILkZhHrRnmEQn6vTerUiEyi3KbdxlsakaMWbV8hgige0TiHhQi+1kh0laiECFFhCCqkqlUBr94uRd/2DmIP+8e0pJLw0EfXrvjQgQkGBNlsHj3Uh4RQC8MRE/O6K0ZM4FmHJfeSTS+S4XILJjKES5rVq1iRWQo7xFZ0XakURUAWsmsShBVyRd+9QYe3daj/b6tIYTx6RQS6RwOjExh5YIGD083E23hXZlCJOrS5IypiHeBu2Zqyjx0u4X3pZBkmFJEBFSnjLk27wLeeURUVdUUkWKju4BuVh2fTmtvdIIg5j/b9+ezgz66cSl+df0ZeP7W8/COjnyu0O7BSS+PdgRzBZoBxnRVdxSRYlHzDJqaqTK08d0y/zhCFZFk+c27gL6TIJnJuXqzHzSM7nY1F1dEmmqDYFPHzJlOEMT8JpnJYm/ByH7dOUfhhM5G+HwKjmrPqyC7D8tTiCQz+sK7UmZVQE86FTk1o6qqpmy73ZrRtu9K4BGhQmQWmlm1XGsmwHJEvFFEjKO9bvpE2MRMZ3NdySra51PQUjCsDtEIL0FUBXsG48jmVETDASyMhrWPH1Vox8ikiLCFdz6lfDvEjX0zU6kssoUobbd3zTCzqgxTM96fQDISmiJSzqzK3hTeKCIBv09LXXXTJ7JvDqMqg0Z4CaK62NkfAwAcuzA6Y0cXU0T2SFSIjBjaMr4SoZGAO4UI+95+n1IySRswREbwbM1kqTUjLZWgiAC6YdVNnwgzqi5vLd6WYbDJmREyrBJEVbCrfwIAcOyiyIyPa4XI4ThUVXX9XMXQNu+WMaoCullV5NSM0ahaaskqYGjNiDCrUsS7fJgb3xWoiJiYmgEMMe8uKiL758gQYVBrhiCqi52FQuSYhTMLkWWt9fD7FEwmM+iXZP/UaLywZ6ZMhgigezZiAh/2zGSIAGIDzUgRkRAz47thzazqTY4I4M3iOxbvvrzE6C6DRngJorrQFJFZhUhNwIdlBQV1z2Dc9XMVY3SOzbsMN6Zm2IPnXAq40O27ZFaVDzOKiN6a8U4R8WKEl428tUfCZT9P94iQIkIQ852xqZSmdrBxXSO6YXXC1XOVYq6Fdwx9akbcNVa3ApS/3ocMIZq8Wlxp8ojIi5nxXaaIiBidNe0R8aA1EzMRvAMYF9+RIkIQ8x3Wlulsri06girbCO/IHAvvGG4oItpuszmKAfZgrKpAOsunEEnR1Iy8JE0sAmJtG69yRABDa8YlRSSdzWGqUCRFy8y7A9SaIYhqolRbhrFKshFeNr7bUl/+OhZ1YSAgYVYRMdyPeLVnyKwqMbpHpFxrpuAREaCITJlURNw2qxoLnrkVEVaIUGuGIOY7pYyqDE0RkcQjwqb55lJE9Ij3tLCJn6RpRcRYiPC575BZVWKsKCJJAYpInJmXTCoibnlEWFumrsY/5/IqbQMvtWYIYt7DMkSOWRgt+uerCoXI0GQS41PumetLwcyq5VJVAX1qJqfqLXPemFVEFEXRCgYehUgupyJTCFIjs6qEmDEPCVVEClMzdXN6RPI/JG4pIrFptg+hvJwJ6IrIVCqrmW8Jgph/5HIq3iooIqtLKCINoQAWNeYN7rsPe29YNZsjEg76ECgEnonyiWgKvAlVQs8ScV4UMX8IQIqIlJgxDzFFJJtTkcnynetmbxCzgWZueUTYD2K0du6FzQ2hgPbmJlWEIOYvh8amEU9lUeP3lU1c1tsz3vtEzOaIKIqiT85Mi7nOMnWjnBWAwTNLhAoRyTGjiBjfNDxVkWmD/FcrWaCZPjEztyKiKAra6ildlSDmO8wfsqq9AcEyLVtZDKupTE67Zs41vguIn5xh6oaZyRWei++ME581c7Ta3UDoCTZv3oxTTjkFkUgE7e3tuPTSS7Fr1y6RL+kYMx4R4z8cz8kZ5g+p8fvmrFK1HBHXCpG5V1UbaSHDKkHMe3b2sR0zxdsyDFkUkbFCW8anmGszi943Y2Y4ghHi6E3UJ2Z8ZaPl3UJoIfLMM8/guuuuw9atW/HUU08hk8ngggsuQDwuh3u6GJqLucwbw+fjaxxiaGFmcxhVAWNrxh3zF9u3YEYRAXTD6hC1Zghi3rJzoPzoLkOWLBFjhki5hXcMbd+MKEVEa82YUUQ4tmYkSlUFAHOPtzZ58sknZ/z+wQcfRHt7O7Zv346zzjpL5EvbRmvNmBinSmVyfBWRpLnRXcD9QDNNETHhEQEMI7xUiBDEvGXXHKO7DFaIHBydRiKdNaUAiMCsP4TBFBFR6aoJrTVjxiPC7+FXplRVwGWPyPj4OACgpaXFzZc1TS6nmk6b0xbfcdw3EzcZ7w7oPyCum1VNKiJtWroqtWYIYj6SSGfRPZRXt48tMbrLaK2vQVNdEKoK7D3snSJuds8MI2LIEhEBG981p4jw2zeTlChDBHCxEFFVFTfffDPOOOMMHH/88UU/J5lMIhaLzfjlJkYn8Vxz3Vq6KsclRNro7hwL7wBdEYmnssjmxK/XZq5x860ZMqsSxHxm9+AksjkVjbVBdERDZT9XURR954yH7Rl2PZprdJehKSLCpmYKiogpjwi/h99UtSoi119/PXbs2IFHHnmk5Ods3rwZjY2N2q+uri63jgdgpvHUbPa/CLNqvQlFpMFgGo27kNURszC+C+iO9KEKLUSyORVv9MaEJSoSRKVjjHY3Y3iUwbA6piki5h6ooi4pIl5NzcgwMQO4VIjccMMN+OUvf4k//OEP6OzsLPl5mzZtwvj4uParp6fHjeNpsH9gv0+ZMz00HBRhVjUXZgbkCyH2JnKjPTNhYXwXqPzWzM0/ehl/9Y0/4eYfveKK4kQQlcYuk0ZVBitE9nhYiIwwj4hFRUTY1IyJ4QgGz9YM84iUG7l2E6FmVVVVccMNN+DnP/85nn76aaxYsaLs54dCIYRC5SU+kZg1qgJAmDmYuZpVzcW7MxrCAYzEU64YVvVk1flvVt26dxi/eLkXAPDzlw5BAXDXZSfBb8JlTxDVwpt95aPdZ7NKAkXEbLw7Q/jUjOYRcTnQzERMhZsIPcV1112Hhx9+GD/84Q8RiUTQ39+P/v5+TE9Pi3xZ21iqTrUNvN4oIoAhS8QFRURvzZj0iDBFJJ6sqPZGNqfiC796AwCwblkz/D4FP3vpEP7lJ6SMEIQRsxMzDOYR6R6Kc02ktsJ4IYag0eR1jLWiRSsiplozWo4Ix9ZMNRQi999/P8bHx3H22Wdj0aJF2q/HHntM5MvaxpYiwtGsasUjArg7wjthMdCMmVXTWdW10DUe/OiFHrzRF0M0HMADV63DvX/7znwx8uIh3PKTHVSMEASA0XgKgxP5tqvZQmRJUy1qg36ksjn0jHrzMMra2A0mr2Oip2asKSL8WjOymVWFt2YqiaSN6pSrImJhagZwb9+MqqqWx3fDQT/qa/yIp7IYnkyZ/jovGZ9O4+7f5pN/bzr/HWhtCOGvTlgEVQX+8dGX8NMXD6Im4MPmD53g8UkJwlveLrRXOptrtQeiufD5FKxcUI/Xe2PYPTiJFWV204hiUmt/my1E3JmacTvQLJmRyyMixykkQY93n7s6lUER0ffNiE1XjaeyYEKA2dYMYGjPVIhh9d7fvY3heAqrFtTjqtOWaR9/74mLcM/la6EowCPPH0DPyJSHpyQI7+kdyysaXc11lr7O650z7Bprtnhyb2rGG7NqVU3NVAp67r8ZRYSN73qviIj2iLB496BfsWRuYobVSoh533N4Et99dh8A4LOXrDniSeGSExfjtJWtAIBfv9rn9vEIQioOFQqRRU1hS1/X2VwLABiIJbifyQxsIMBsIcIUkXgqK8TXYu2eQx6RqsCKIsJuyF7liADueURihraMlQVJbN9MJSy++9Kv30Qmp+LcY9tx9jHtRT/nvScuAgD8egcVIkR10zeeL0SWNNVa+jrdxO7Nw8mk5UJEV4B5X2cz2RwyBak5bOqew39qhgoRCdFT7kyYVYP83hQMy1MzLnlEmOISMWnwYrQVFJERyRWRXf0T+P3OQQR8Cm577+qSn3fRcQvhU4BXD41j/7C8ixsJQjS9Y3lFY1GjxUJES1x2/+Ekk81pCrZZj0hNwKepFbyVZ+O9w8w9h6tZtZrGdysNO/06vkvvrOWIRNxSRKatje4ytCwRydNVt+8fBQBsWNmi9bCL0doQwumr2gBQe4aobphHZLHF1oyX+UJsqShg/hoL6KoI7ywRYyFiThHhmKwqWaCZHKeQBBZOVjGKCMsRcbE1Y4WWQmtmSHKz6o6DYwCAEzub5vxcas8QhLEQsaaIsNUPXjycTBZa33mvm5VCRMzkDHuIrfH74DMRlihk1wwVIvKRtCBXaUvvOCoiUylrikhDoTCQvTUje7rqKwfzW6FP6myc83MvPG4h/D4Fr/fGsG+I2jNE9RFPZhArXBMWNVpTRNjqh9F4CjmXM3msGlUZoiZn2L3DbHtERGuGPCISwgoRK3G7PKdmmHRYb1ERca01Y1ERqQSz6nQqi7cKOzPMKCIt9TU4fRVNzxDVCzOqRsIB07unGM2FaPVMThUWm14KqxkiDFH7ZrQHXxP3G0DQ0jsqROTDSoWqL73jo4hkcyqm06w1Y9Ij4rJZ1ezmXQbrB49I7BF5o28c2ZyKtoaQ6ae7S6g9Q1QxhwpGVasTM0D+xseuW263Z7RUVZuKCO/CycroLkBTM1WDpUAzjv06AFoRApiv2N0e37X69GMsRGSNRn+lR2/LmB1NvmDNQgR8Ct7oi2HvYe8WeBGEF/SxDBGLbRmGvpnb3UIkblMREbVvJmFhpQhgyBHhGfFOHhH5sBTxzqZmOCkiU4UfEp9i/o3JvCTiWzPW9swwmAybU4GxKTlVEStGVUZzfQ3edVR+euYJas8QVYZdoyqjxaMRXvutGTEekaSFJauAoTXD4eE3LdmuGTlOIQkJCwuIeCsi8ZTuDzH7ZF5b8JJMczTMFsOuIhL0+9BUl/8aWUd4dxSMqid2zW1UNcKmZ/6L2jNEldE7nm/N2C1EWJaI24nLTBGJWC1EQqKmZszfbwBBrRlSROTDytQMb0WE/ZDUWZhvry28gVOZnNDWR0zziFhfXKdfdOQzrI5Pp7G3MPlykgVFBAAuXLMQQb+Cnf0Tnu3NIAgvsJshwvDKO6Y97Fm4xgL6dW+C804vKwq88fN4tGaS5BGRF1s5IpwUkamUtYkZYKapVaQqMqFNzVhf1tzqUT/YDK8dyqshnc21mlxslsa6IM4otGd+Q+0ZooroG7eXqspg03RuFyLM4yHN1IxVRSSoT8043WxPZlWJ0cZ3TZlVOSsiKeuKiLGSnk6JK0RiWo6IdUWkTeLJmVcK/hCragjj/DUdAIBn9wxzOhFByI2qqroiYrMQafFIJbWbI6Ilq05znprJ2JuaUVUgnXVYiFCyqrwkLCgieo4IL7OqtVRVAFAURWvPiC1EWMS7dUVES1KUsDWzozAxc6KJILNibFiRzxN58cAotzFugpCZkXgKyUwOigJ0NIZsfQ/PWjN2p2YEKSJ6XIQ1syrgvD1DZlWJsbR9l6NMBljfvMtg7RlRrZlEOqvJePY8IoWYdwkVETYxc1JXk62vX7WgHm0NISQzOc30ShDzGbbsrq0hZCkm3YgWdOhyu9bq5l2GvmtGVGvGmkcEcG5Y1ZbekSIiH9Yi3nWZjMlcTpjSzKrWfkjYOVg8PG/YU4CiAA0W1BqGHvMulyJyeCKJ3vEEFAU4fok9RURRFGxY0QIA2ErtGaIK6B13NroLeLdvhj3sWS9EClMzvAPNMtYUEUVRNAWDVyFCioiEJNPm57p5VqeAcXxXLkWE/fA1hAKmFjPNRlazKlNDjlrQYPnCZGTDynwh8pfuER7HIgip0f0h9iZmAP3hZHTK3X0zkzbNqkwJTmVyXFuwWqCZSUUEMGaJODsHFSISY0URqfH7wOI+ePhEmKJhxSMCALU1Yj0iWry7DaMqoI/vymZWZYvurASZFYP5RLbvH9X6rgQxX+lzmCEC5AMBgfxai3HOBtBy6IFm1h72jA8qPNdpaIFmFlpcvLJEyKwqMdpct4kKVVEUrkl32sI7iz8kmllVlCIyzcLM7KkGXq79LofuD7HXlmEc3d6A5rogptNZ8okQ855DDuPdgfzNr7HW/aBDdo21qoD6fXpLhOd11mqgGcBv8R0pIhKjvTFMVqhalggHuc6pIjIlSBHRJ2bsKSKNdfrCKLfXfpdCVVU9UdWhIuLzKTh1BWvPkE+EmN+wPTN2Ft4ZafVgms7u+C6gP/DxmpI0fi+zgWaAYUjCaWuGds3IixVFBNALlgQPRcSmR0TED4gRvTVjTxFhTz6qyn/8zS4HR6cxEk8h6FewelHE8fdj7Zm/7CWfCDG/YVMzixwWIi0ut2xVVcWkTbMqYPDipfi1X7XcKkuKCKfWjAUbghvIcQoJyOZULSTG9Fw3x22IdqdmRHtEYlqqqj1FJBTwaz/EY9NytGdYkNmxC6O2RxCNMMPqC/tGkCGfCDFPSWdzGJxgHhH7rRlAzxJxa6x/KpUFS1mwalYF9Ac+ntOJCW04woZZ1UEhksnmwMRp8ohIRsrwD2v2jSFGEbFYiATlbs0AQFPha8em3DOmlUNvyzjzhzCOXRhFNBxAPJXF670xLt+TIGRjIJZATgWCfgVt9fbCzBgtLObdpWk61pZRlJmrMcxSK2A6kXkLrTwM8dg3Y0xlJY+IZBhbG2b7ZkwR4To1Y9Gsyn6oRLdm7JpVAb2IcdMhX47Xe/kWIn7yiRBVgHHHjJ1RfiNavlDcHY+INjFjYbu5EREJ1kmLEe8AEOKw48z40E2FiGQwqSvgUxAwWYiEOa5l1iLeLfQLARcUEYetGQBoKhhWxyQpRPYNTQEAjmpv4PY9ySdCzHd6OUzMMNyeprM7McMQoYh4NTWTzOb/HxQlf7+TASpEClhdyQzwVUQSFsLUjNQWWjmixnd5KCJNtfmLzviU9x6RdDaHvkI6ZFdzHbfvy3wiz+8bQVaS6SCC4AkzqjqdmAH0oEO3WjN2M0QYIh74EnbuORxaM9rort9nSx0SARUiBexVp/w8InYc1ABQWyiGhJlVeXhE6uTxiPSOTSOn5n+gF0Sc9bmNrFkURUMogIlEBm/2kU+EmH9oiohDoypgGN91uTXjVBHh2QJPOrjnOFFEjIWILMhzEo+xo4iEOU7N2JkpB8RIhkZi086SVQE9S0SG1syBkXxbZmlLHdengYDfh/XLmwFQ3DsxP+njsGeG4fb4rt3Nu4w6AdOJthQRLUfEfiHCzKqy+EMAKkQ0tHh3C9VpOMhPEUnYVURYa0ZYxLuzZFVAzxKRwaxqLER4o/tEyLBKzD8OFVozixt5tGb0QsSNoMNJh4WItlxUQKCZPY8Ih9YMFSLyoY9SuduvA/Jz3cxXYMVBDRh6l8KW3hUUEUfju/mLjgytmZ6Rgj9ERCFi8InIkiJLELzgqYg01+WvCTnVHaWUKSIRSRQRVVUND79W7jkcWjMFsyoVIhKitUY8UEQShjeV1YAtbXxXgCKSzanak4TdZFVA94iMSxBo1lNQREQUIicsaURt0I+xqTTeHpzk/v0JD9m2DVi0CFiwQP+1aFH+41XAVCqjPUjw8IgE/T7tuuBGzLvT1gzv8d1UNqcFrLmdI5Ikj4i8WNm8y2Cf69TAZPx6qx4RXTLkH59u3DQZcTK+K1GgmcjWTNDvw9quJgDAC/vJJ1LxbNsGtLbm5xxPPRVT/f0IDQ3hgqEhhIaGMNXfD3zwg16f0hXYxEwkFHDkFzPi5gjvhNNChPN0ovHh1VqOiHOPCGvNyJKqClAhomHPrMonRyRp6NlZDQoSsQOBwSZmwkGfIxlPRrNqV4tzebkYzLC6fd+okO9PuEB3N3DqqcCGDZgaGUEIgOLzo2XJaviWnYQ/LTsJvuVr0br8nehJeH1Yd+A5McNg6azDLozw6gvv5BjfZfcbRbGmTPBozchoVrWvt88z7IzvhjltQrQ7MQMYJUP+ikhMM6o6ewIymlVVVfVsdn18Oq0ZZnlmiBhZtyxfiLywnwqRiiMeB771LeDzn0dqYgKrAByqqUXDiRdgwSkfQCDafsSXnK0A1zzxJm46/2jLm7MrCZ7+EIY+OeNGa8ZpoBm/zChgpifRyvVwvppV5+9PjkVsBZpxSla1M0/OqDOM7/K+yeuju87eJk0FY1oqk0MindNGjt2G+UPaGmpsS7RzcfKyZihKXnkZnEigPcLvCZIQSHc38P73I/faa/gsgC/XNSGy7hIsOfkS+MP5BN7s1Diy8VFozX2fHzVtS/HAH/fiiVf78KUPnoB3v2OBd/8PAmETM4s4TMwwWhrca804nZqpDfJuzdgLsOSRrMrMqrJs3gWoENHQPSLWFRHHHhEbOwe0MxRu6jk1b4DisU2WwSPMDADqa/wI+BRkcirGplOorRHTFpkLVoh0ClJDgHzeyjEdEezsn8D2faO4+IRFwl6L4EA8Djz4IHDnnUgNDGABgMwx78Lii2+EL5R/n6SHDyL2/M8w+fofgOzM9uLv3uzHZx9/HQdHp/H333keH1i7GHe87zg0F5725wt9hdbMEq6tmUIh4mprxlmgGb/WTOHh0+L1mueuGTKrSoiuSlhXRBIOx3f11oyNrZCGipp3loge7+6sEFEURYp01Z5RcUZVI8wnQu2ZCuDXv0buhhvwmYEB1PsD8J9/LRZcugm+UB2S/bsx+LMvoff/fhKTO7YcUYQMDAzg3GM7sOVTZ+GaM1bApwC/eLkX//v72+ddzH/vONszI6I1I78iwnu5qD6lae0WzLM1Q2ZVCdFT7ux4RPiYVe0oIkG/D0F/vh3DO11VX3jnXDhrlGByRuTEjJH1y/J5IlSISEw8Dvz7vwO33ILPAfhy00K0ffRuRNddAgAYf+5H6P/ezZh++zlAzSEIYHrVKqgNDVAVBerAANrb856R+lAAn71kDX72yXehvsaP5/eN4D/+tNe7/zcB6Jt3+SkibN/MkAvju85bM0wR4ePF0zyJVhURLq0Z+cyq8pzEYzTzkI2VzE4VEWZ2tfqmZIQFbeDl1ZoB5EhXPVAIMxNdiDDD6uuHxoUl3hIOiMeB227LKyH79+Peleux6Op7EFp4FLJT4xj48e0Y++P3ADWHOgC3LliARDyO8O7dwMQEkMsB7UcaV9d2NeH29x0HAPjKll14o3d+7BxSVRX9rBDhaFZtdVER0QLNbD5U1XIONEvabMdz3TVDhYh8aG8MG+EyjgPNbBRBRkTsQQD4bN5lMMOql6FmmkdE0Oguo7O5Fh3REDI5Fa8cHBP6WoRFWBFyzz14N4Cvn3A+mj78WfhC9UgcfAN9370Rib3bEQRw69KlmNiyBV/q74evzlzxetn6TrxnTQfSWRU3PfYS1yVpXhFLZLSHnIVR/oqIG2ZVNjXjONCMc46I1XZ8iMN+MypEJMZOMaDniDhURGwUQUZ4/5Aw9NaMc0XE61CzbE7FoVF3FBFFUbT2zHZqz8jD4CDwoQ8hd889OAvAjo2Xoe2vboLi82Py1acw8MgmZCeGEFUUTD35JL7U3Q3fe94D+MxfExRFweYPnYC2hhq8NTCJu3+7S9z/j0swNaSpLsh14o15REanUkI9NclMFqls/vreYHPEmj3spbMq0lkOu8WcekScmFVZxDt5ROTDVqAZM6tyUkTsjO8C4hbfaa0ZHh4Rj0PNBmIJpLI5BHwKV8NdKbQ8kX2UsCoFg4PAJZcgs2ULlkPBG+d9As3v/nsAwPjWH2P4iXuAXBZRnw/DU1MIXHihpQLESFtDCF/+8IkAgP/8n248u3uI2/+GF7AMEZ5qCAA0F64JqgqMTYlTRZgaAgD1NgPNjNdmHipX0uaSU2rNzHPsjO+GJAg0A4Dawjl4KyITHBbeMbxefMeMqkuaa+G3mF5rBy1hdf8oLcDzGlaEbNuGVl8A0+/7J0TXfwAAMPK7BzD2zEMAgK5gEMPxOAJh5zfc81Z34G9PXQoA+Kcfv6J5FCqRfgFGVQAI+H1aMSKyPcP+7kMBHwI2VYB88Fj+v3k88Nm95vOYmtGSVUkRkQ87u2a0pXecIt6tLNwzwttIxdAVER5m1byqEvNIEelxaWKGsXpRFLVBP2KJDHYfpgV4nmEsQvxBhD54K+rXnA01m8HhX96FiRd+CQA4o60N+yYnuRQhjM+8dzU6m2vRN57AT188yO37ug2bmFkoQElscSFLZNKhURXIt9zqOLbAbSsimkckB1W194CTJEVEXpiqYeUfhxUtqUzO0VOvnrJnVxHhm/rHEGFWHfPIrCpy624xZizAo70z3mAsQoIhhP/6c6g76lTk0kkM/uyLmHrzGQDAGUuW4Jm+Pvhq+IaQ1YcC+MSZKwEA3312X8UqY6IUEcBoWBU3wut08y6DZ6iZ3Ws+U+xVVVc2rEKtGYlhBiQr/zjGatZJz86ug5rBO/WPoZlVeYzvehxopi27E5iqOhs92Ix8Iq5jLEJqalF72edRu/ydyKWmMfjjO5DYux31AG7buBHP7NsHX0BMyPSH13WiIRTA3sNx/E+FekX6Y0wREVCIuDDCq2WIONwFVFvDTxGxk1uV/3z9/mS3PcOMuxRoJiEpG4WI8U2RclCI2J0pZzDJkOeooKqqiDGPyDyYmnErzMwIM6zS5IzLGIqQtnAD6j5yJ8JdxyOXjGPgsc8i2fMqoj4fxrZswZ1//rOwIgTIR4r/9bpOAMBDz+4T9joiEauI5AuRIRdaM3bj3Rn6glEOrRkbSd7A7ELE3j0nTYqIvLBCImShSgz4FDDfoxPzkPOpGb6pf0C+6mcjdXxzRDzyiLg0umuELcDbPzyFwxPi0yMJzCxC6hpRd/n/QWjxMchOxzDwyK1I9e7MT8bE4whYHM21y9+dtgwA8PtdgzgwPCX89XjTp8W78y9EWurzrRmRG3j11oyz0WOe04l2IxsURdEKCLuFCHvotnKvE408J/EYO30zRVG4jFMlbIwOG9HNqs7n2xls867fp2gz9E5gyaqTyQyXOXwrTKeyWiHgZiHCFuABwHZqz4jHWIRE2lB/xZdR07ES2clRDPxwE1IDe/QihKMpdS5WLmjAWe9YAFUFvvfcPtdelwfxZEZTRkWYVd1pzeSvrw0OlV02nTjFozXjIMRSzxKx2ZohRURe7C4C4pF0l3SqiGhubn6KCJuYiYQDUBTn467GLBK3J2fYsrtIOKB5VdxCzxOh9oxQDEXIgqZFaLjy3xBs7URmfBD9P/w00kP7PSlCGB87fTkA4LEXeipqlJf5QyKhgOPWRjHcaM3om3cdKiKsBc5TEbFxzXf68EuFiMTYXQTEI+bdqUeEZ++SMcFxdBfIZwawFo/boWZuj+4aOXVFPmH1eQo2EwNbXrdhQ74IaVuG+iu/jEBjO9LDB9H/g1uQGe31tAgBgHe/YwGWt9ZhIpHBz1865MkZ7NA/Ls6oCrizgTfOyaxaV8NvOtHu0jvA+eK7JJlV5SWVsT6+C/BJutPDbRzmiHA0q7LWTLSW31NQk0eTM14YVRkbVrQCAF47NK6pTAQnjMvr9u1D6+Jj0HDFZgQaWpAa2Iv+H/5rPrLd4yIEAHw+BVedthxAvj1jNwPCbfoEFyJtbHxX4AbeCU7juzyXi9qNeDd+jd3WDJlVJUabmrHamuGQdKeH2zhTRHiO72qtmRC/VgZLV3V78d0BlzNEjCxsDGNFWz1yKsW9c2XW8rqvrjoVjZd/Cf7aKJKHdmLgkU3ITY1JUYQwLlvfiboaP94amMRze4a9Po4p+gUaVQFdERmbTiMjyDsW5zQ1U8fxgc9OkjfDcWvG5r1OJPKcxGNSNpJVgZlJd3bRwm1sKiLsB4Tn+G6MY5gZgxlW3Z6ccTvMbDYbV+bbM5Vy85GeeBy45RatCHn5pAux4EO3wRcMY2rPNgw8dhtyybhURQiQb3N+6OQlAIDvb93v8WnMITJVFQCaC9N0qiquZasVIg6vZfpQgHOPj5MQS6etGfKISEommwMLPbRsVmXVqQOPiBMHNQCEBQSa8frhNeJVqFnPiPuju0Y2rsy3Z7buJUXEMWyD7n334SwAr77rCrRedEN+g+6OLTj80y9CTSelK0IYl5+S3z/zh12D3FcyiEBkhgiQn8pjRnZRDyiTvJJVOUa8O2nHO1Xh7T50i0Sek3iIMSrXrlnVWWvGmUeE5w4EBi8504gXoWaqqhpSVcVv3S0GK0Re7x33LEdlXmDcoOsL4M2LbkDTGVcAAMaefRTDv/kGoOa4Lq/jzXGLo1jSVItEOoc/V0DSqmiPCGBY/yDousC27zqemuH4wOekHc92ktl9+KVkVUkxpqLaL0ScKyJOA834Ts3weYowwsyqbt6MhyZTmE5noSj5zbte0BENYyX5RJzR3Q1s3JjPCKmNIvWRLyJy0oVQc1kM//abGP/TwwDELK/jiaIoOH91OwDgqTcGPD7N3Gjx7lGRhQi7LojxjnGLeOeYYO3kmu/0nkNmVUlJZvNvLEXJp6Vage/UjM2IdwFTM2IUEfbk455Zlakhi6Jh24oTDzYUVBHyiViEjeeedhpS3d1obFuG+r/7KsJLT0AuOYXBn34Rky//BgDQFYkIWV7Hm/esWQgA+N3OAS29WEYS6aw2ViuqNQPo3jFRigi31gzHB76kg2u+UxU+aWOdiWjkOYmHGMPMrIZ3OR2lUlXV9kpoRlhAjkg8xb8Q8cKsemgs7w/pdHHZXTGYYXVrNxUipmF+kBtuwGcGBtC66hS0fvQuBJsWIj3ah/7v/xMSe18AAHSFQtg7NCR0bwwvTl3RgkgogKHJFF7uGfP6OCUZKKgh4aBP+9kVgejrAnuocmq85zmd6OSa7+ThV1VV3axKrRm5YB4RO9n7jsNlDF/ndHw3mclxe8Jiscg8WzOaWdXFQkQbP2zyVqrXfSIx8omYYXAQuPjivB8ECu497W/Q8uHPwheqQ2L/DvR/72akh3vyG3TXrMG+WAwByZUQRk3Ah7OPlb8906cZVWu5pCuXQmS+UC6naoWD02sZr+nEbE7VfBq2ChHt4df6PceJH1Ik8pzEQ5yMMzltzRjfTLbNqobeJ68RXl6xyEaYWXXcRbOqZrYT2OM2A/OJqCqwrZt8IiWJx4GvfAU47jhkXnwRbaF6JD78WTSf9XdQFB8mXv4NBn70OeQSE6gB8ht0X31V+nbMbN6zpgMA8NQb/R6fpDT9Lv3s6PlC/K8LccOoLa/tu04VEWNLxe3WTMqQ1UKKiGQ4K0Qc9usKX+dTgKDf3lOH8c3Ma4R3UohZVQ8vcgsmL4t0/Ztl4yo2xkvtmaKwVsw//zNuGxpCtGMVGq6+B3VHnYpcOomhJ+7ByG+/CeQyqAEw4eIGXd6cfcwCBHwK9hyOY+/hSa+PU5Q+waO7DF0R4e8dY/4Qv09xPK7KK8Ha+PDpdmsm7WAwQyTynMRDUll78e6AM5kMMGSIBPy25U+fT+Hq6Ab4GbyMGKdm3Iq4dutiagbWnnmOCpGZMEPqKacUWjHAN066EAs+ehcCzA/y8L8g/upTAIAuvx/xeBw1dd76fpwQDQe198N/vylne8atIl4zq4pQRLSJGfvXVwYvsyrbth70K/BbHI4A+Cgifp+91xYFFSLQK0s7c9VOWzMJhwvvGDxn3AFd0owIMKtmc6pW6IimX3AypBU2FhbgvdEXc7U9JTUsJfWGG/CZAwfQVFOH6fffkg8pC9Rg6u2/oO+hm5Ae3AugMJ47NYVABRchDL09I2ch0ic43p0h0qzKvG4RDss764J8lt4ZHz7t4OThV0ajKuBSIXLfffdhxYoVCIfDWLduHf70pz+58bKmYQYeO/84jlszDjNEGDxT/wDDkwTHQiQc9Gt/X26EmmWyOQxO5JdpyaCItEfDWLkg7xOhbbzQskFy992HdwO4a9E70Pixb6B+9VlQsxmM/uFBHP7ZnVCTcQQB3LZxY0WM55rl/EIhsn3/qNClb3Zxq4hnLVsRxbl+HXPudQvX5K9d0+msI0XX6bZ1Jw+/SQkzRAAXCpHHHnsMN910E2677Ta89NJLOPPMM3HxxRfjwIEDol/aNHw8Is4UEW79Sy6jZVmtOONZiADuhpoNTaaQzanw+xRty6fX6HHvVdyeYYbUtWuReu01NEPBjlM/hIVX/huCTQuRGetH/w8/jdjzPwWgIgJg8sknceef/1wR47lmWdJUi+MWR5FTgd/vHPT6OEfgukdEiCLC74GKPeypKp8AS9uKiJPWjAP1XyTCT/PVr34V11xzDT7+8Y9j9erV+PrXv46uri7cf//9ol/aNI4KEYdxu/ryI16KiPOWBzOqAvneKk/0UDPxhQiTljsiIWn6oadVe7CZ0ZAai6Guvhnhv/k8ms/5Byj+AOI7/4Te796IVO8uAMAZ0SjGpqdRc+GFFWlKnYvzV8vZnklnczhcUGlEe0T01Q8p5DgHvLFrGY88pFrDNdrJA58WYGlbEbH/8JvOyrdnBhBciKRSKWzfvh0XXHDBjI9fcMEFePbZZ4/4/GQyiVgsNuOXGzCzqicpd9rCOz57EKZTzldps90MtUE/ApwrZz1LRHy6ar8LezKssqEQbPZmf8zVhFkp6O4GTj1VM6R+beV6LPqHf0ftipORSycw/OS/Y+gXX4aajKMOhVbM8DB8ksa184D5RP709hDX7dlOGZxIQlXz7eqWOrGtsGihEMmpwCSHzbZGmNfNabw7AAT8Pq19P+Xg30oLM7PtEbH/8JuSMFUVEFyIDA0NIZvNoqOjY8bHOzo60N9/5Pz85s2b0djYqP3q6uoSeTwNJ3IVL7Oq49aMNuPOQRER4A9huJmuagxkkoX2SBhHtTdAVfM3n6rAGNO+fz/q/UFMnve/0H7ZHfDXNSI12I3+hz6FyVeeBAB0BYMY37Jl3rViisGW4E2ns1KpZCwIsKMxBJ9gNTEc9Gt+Cd4+kUnOW8R5tMB1FdypImK/NVOVZtXZY1OqqhYdpdq0aRPGx8e1Xz09PW4cDykuZlVn47tOWzO8Uv8AY7w7/90sbm7g7ZcoQ8TIeYWlZ7KObXJlcBB4//u1mPbG1k60/t1XEV3/fgBAbNvj6CukpPoA3Lp6dX5pXYXmg1hFURSceXQbALnGurUiPupOES8q1Iz3zqxaDus09N1iTj0i9qdmggE5WtUMoY8bbW1t8Pv9R6gfg4ODR6gkABAKhRAKuW8qdOYRcbZrRnNQc1NEnBciIhURN82qMmWIGHnP6g58+5m9+MPOQaSzOemMY9zo7gbOOw+Z7m6sBDB64nuw4Pz/DV8wjGx8FENPfB2JvdsBABEAQ088MW+9IOXYuLIVj27rkcrA7HZbs6kuiP5YgvsDSlxbVcHnoYrHglF9z4z7UzNaa0aya47Q09TU1GDdunV46qmnZnz8qaeewumnny7ypS3BI+I95VAR4eYR4aCIiEhVZWjpqi74I5i8LJsi8s6lzWitr0EskcHz8zHufXAQuOoq4KSTkOruRgPLBrn4RviCYUx3v4jeB2/QihDNkHrxxVVXhAD6JNVrh8YRS8iRL+N2Ea+HmvG9LkxwvpaFOcQkJJ0qIkEOrZlq8ogAwM0334z//M//xHe+8x28+eab+NSnPoUDBw7g2muvFf3SpvF0fDfNVxHhMb6rbasU6BFxZ2pGTkXE71O09oxs0xKOYcvqHn4YZ05MoHHhUWidlQ0y+KPbkYuPIQjg1g0b5r0hdS4WNoaxoq0eOYn2EHmhiAD8rwvcWzOaR8S+F09vxzv0iDgJNLNZBIlCuBPsIx/5CIaHh/GFL3wBfX19OP744/HEE09g2bJlol/aNFrEux2PiIPqNP91nALNeCoi88CsmsuphohqecyqjPNXd+BHLxzEU28M4Pb3rRG63dQ1DK2YVgC5tRej/bz/BSUQRGasH4d/dZc2lhtVFAz/5jdV4wWZi40rW9A9FMfWvcM4b/WRbWu3cStVlSHquqD73fhcy/i0ZpxFNszH1owrlvRPfvKT+OQnP+nGS9lCS1Z1sn3XZo6ILtPJE2im91Ur1yMyHE8hnVWhKEB7RI4wMyNnHr0A4aAPh8am8WbfBNYsjnp9JPvE48CDDwJ33onUwAAigRAiF16HhuPPBQBMvfUchp74OtRkHH4AG+vr8fTQEAJVrILMZuPKVjzyfA+27pVNEXHJrFonxqzK+6EqzMGLpweaeTg1I5lZVa6yyCOcjDQ5bc1wU0TYD4j0UzPuBJqxC+mChpCUZtDaGj/OOGoBgApvzxh3xQwMINq8GK1X3Y2G48+Fmsti9A/fweGffwlqMo4aAFNf/jL+Z2yMipBZbFiR94m83jvuipG7HNmcigGXVyM01orZwMu7NVPHdXzX3vWVfV0yk7McNZ+WVBGR6zQe4SR/nxUiqWzOViqg05lyhja+y0ER4W3wMtLkUqAZG92VzR9i5AK29OzNIzN1KgKWklrYFfOVlevQ/vdfQ037CmQnRzHw6G2IPf8zAEAn25h7yy3APM8GsYPRJ/KCx3uIhiaTrq9GEOUR4W285+HFY/cbuwMKzA6gqrqab/W1q86sWgnwiHgH9P6bFZzOlDN4SIYM3k8RRliyaiKdE5okKevEjJFzjm2HogCvHYppPfmKYXAQuOQSLSX1lVMuRfuHPwdfqB6JntfR991/RLLntfyyutWrsX+ebMwVycZC6q7XwWbM5O3magRNKRXUmuGl7vLw4iUctuONX2e1PVO1UzOVACsg7CWrGt4UNnwiTmfKGXU1fFZUA2ILkYaaANi1LSZQgpYxVXU2CyIhnLy0GQDw35XUnilszc1s24ZWfwBTF9+IlnM/DsXnx8Qrv8XAo7chGx9FDQrL6l57bd5szBWJthCx29tCRE9Vda+IZ60ZntcEVVURLzyYNYSCXL4nj7ymhMN2vLGtYtUSoJtV5ZqaoUIEQNpBlRjwKdqN1Y55SF+AxGnpneSBZj6fYsgMEFeIMI9IR1ReRQTQd41sqZRCZHAQuOgipLq7EamNovYjX0LDie+Bmsti5L8fwMiT9wK5DKKKkm/FVGFAmV1YIfJ6b8xTn0jPSL4Q6Wx2T8ES0ZpJZnLI5tgWcU6KSNB5gnXSYTteURTb3kRZk1XpCgG9SgzZUETybwr741ROHdSM2pr813NRRDiPvM1GDzVzQxGpjEJk695hacKsisL2xWzYgMxbb6GpaRFarvoKwl3HIZeYxOBPPo+J7b8EkN8VM0ytGMt0RMNY2VYP1eM8kZ7RKQDA0hb31EQRgWbsgQrgs/QO4NSaYR4RB+14PUvE2jnSDu51IpHrNB7htG/mJEvE6Uw5ozbIrzWjrc7mtChqNqIc8kZk3TMzm1ULGrCyrR7prIo/vnXY6+MUJx4HbrstPxmzbx9aF70DrR+9C8HmRUiP9aPv+/+MRPeLqAdw25o1+V0xNBVjiw2sPeNh3PuBkXwh0uWBIsLTO8auY3U1fm6L+1gh4qQ141QRAQwbeG0qIuQRkRAn23cBvTpN2PCI8Fp6xzNHZJLliHB6ipiNPjkjRgFQVdX1QCYnaO2Z1yVsz7Ai5J57CpMx69F4+f+Bv74Jyf7d6P/+PyMzchA1AMa2bMGdr75KfhAHaIZVCQqRpS3uFSINoYBmjOXVlhLRYuaxXFTziPBQRKgQmT9oBh7bLmYHrZkMn0AzY+Kf1dny2Yg0qwKGFEVBrZnx6bRW4MnuEQGAi09YBAB48vV+jMTF7+Axzawi5KUT3oP2D38Wvpowpvdux8Ajm5CbGkMNgIl4nFJSOXBaQRF5oy8m7OejHLmcioOj+SK+y8VCRFEU7umqIq5jPMyqWoilE0XEZmsmSTki8uK4NeMg6S7JSRFhX5/NqbbGiBnZnKq1d3gZvGbTUp9/Yh4WdNNl/pCW+hrHf69ucFJnI05Y0ohUJodHnj/g9XHyzCpCdpz6YbT91Y1QfH5Mvvo7DP70C1BT07oplfwgXGiPhrFyQd4n8rwHeSKDE0mkMjn4fYrramIT5z1ULA8pyrHFXMumEznkiDi5Ntl9+NXNqnLd+uU6jUc4zd/XPSJ2xnf5BpoBQCJlvxCJG5Y5iZiaAYDWQiEyEk8K+f5aPHUFqCFA/mnw709fDgD4wdb9yDgoJLlgKELOAvDamR9F8zkfAwCMb/0xhp/4GpDLkilVEBs99ImwtsySploEXH5qbqzj6x1j5u9oLZ/RXcAwncghR8RRa8bmPYeSVSXGuSJif9+MPjXj7Mk96PchUOixTqXtb4ZkBq+gX3HcLipFayGtcXhSrCJSCf4QxiUnLkJLfQ16xxPeRr7PKEIUvHHe/0Lj6ZcDAEaf/i7GnnkIAHBGWxuZUgUhQyHipj+E0cR5rJ9lkkQ4KiI8I965tGYo0Gz+4GTXDOCwNcNJEQH4GFbjBoOXqI2wolszlZCqOptw0I+/PbULAPDdZ/d5cwgW2X7PPThL8WHnxf+I6Pr3AwCGt9yH2F9+AgA4Y8kSPNPXR6ZUQWxYkTesvtkXmzGC6gY9bGLGxdFdhrb4jntrhp8iEuagiCS5mFXtPfymtNFhuW79cp3GI9KOzar2ZLJsTtV2BTh5UzJ4yIaa01zQxAwAtDWwQkRMa6YSFREA+OjGZfD7FPylewRv9sXcfXFjZLsvgF3v/xctqGzov76KyZeeAFAoQvbtg4/2xQijIxpGZ3Mtcirw0oFRV19bL0TcV0R4m1WFtGYcPuypqsplv5jtqRmH9zpRyHUaj+DWmrH4pjCOgDmR6Rh8FBEWiSzuRtNSn2/NjAhqzegZIvLGuxdjUWMtLjpuIQDgITdVEVaEbNuGVn8QiQ/eivpjz4SaTePwL/5/xF//PQAqQtxk/bJ89P8L+9wtRLxszfAONYtN5x+qIjzHdwsPe6lszpaXK51VwXajOgo003JE7LVmZNtILtdpPCLpVBFhxiGro1SGwkU6RUTQxAygt2biqayQxXeVqogA0Eyrj798SGjgm4axCAmGEP7rz6HuqFORSycx+NMvYvqt5wBQEeI265bn2zPb97usiIy6H2bG4B3zPpEUp4gA9q6zxsKBj0fE7q4ZuW79cp3GA1RV5egRsaeI1Ph9XJL/eKT+adsqOfZVZxMNBxD05/9/RfhEBsYrI1W1GKcsb8bqRVEk0jk8tq1H7IsZi5CaWtRe9nnULn8ncqlpDP74DiS6XwRARYgXMEXkpQOjrk1RJdJZDMTy7VJPzKp1nFszBUUkWsvvfRsK+MCsc3YKEWPopROfhh6iSWbVeQHzaABOChFnrRlexiEeqX9xzmuzi6EoiqaK8G7PTCTSmCj8P1TK+K4RRVFw9enLAADf37pfW9rFHUMR0hZuQN1H7kS463jkEpMYeOyzSPa8mo9s37iRihAPeEdHBJFQAPFUFjv7J1x5zYMFNSQSCmhFgZs01fLdQTVR8IhEOG3eBfI/n04WjBqv+U6GAZzmiFAhIhlpw9OG24Fm7E3kdPMug0fqnxtmVQBoLfhEhjgbVgcK/pBoOCAsB0U0H1i7BE11QRwcncZPtgtQRYxFSKQVdVd+GaHFxyA7NY6BR29Dqncnoj5fPrL9z3+mIsQD/D4F79R8Iu4Em7Gtu10tdcIm5soR5W5WZYoI36KqzsHiOx5hZoDe1rGsiFBrRk5SGQ6FiOYRsaeI8BjdBfik/sUF7GcoRmuDGEVE94dUllHVSDjox7XvXgUA+OJ/valNMjjGuEF32zYsaO1Ew0fvQk3bMmQmhjDwyCakBvYg6vNhmCLbPUczrLrkEzng4eguYPSI8DKrMo8I32tZmIMi4vSaH7apiPBK8uZN1V9lWIXo9yna0iWr2G/N8J3pri28uZ2YVVkhwjMEqBit9WJGePsq2B9i5BNnrsT6Zc2YTGbwTz96xXmLZvYG3cXHoOHKf0Mg2o70cA/6v/8vSA8d0IsQCirzHFaIuGVY9XJiBtADzWKJjOP3u6qqWo5IhLPfzUlrJqntFnO60sO6IqKqquH15br1y3UaD3BqVAWctGZYdcy3NeNEEZlwSRFhI7y8zar9FTwxY8TvU/DVv1mL+ho/nt83ggf+uNf+NxscBN7//lkbdL8Ef20Uyd6d6P/Bp5GdOExFiGSsXdoEv09B33gCh8amhb+e14VIo6GFEnPYnklmctpDJs9dM4DD1oymSDi79doZkJgxOkyKiFwktblqJ8Yhu1MzfGUyrTXDQRGp9NZMpSsiALC0tQ63v+84AMBXn9qF13vHrX+T7m5g40Zkfv97LFd8ePVdV6D9rz8HXzCM6T0vYODR25CbjlERIiF1NQGsWRQF4I5PhLUAOz0qRAJ+n5b54TTmnRUyisLf7xZ24MVLcHr4ZF9vJTJixugwKSJyoaeqcgiXsegR4S2T8TCr6oFmYivmVkEx71q8ewVOzBTjsvWduGBNB9JZFZ967GXzUuzgIPC3fwusWYNUdzci9U1I/s0X0XTGFVAUHyZefhKDP/si1HSSihCJWedSe0ZVVa0Q8UoRAfgZVplRNRIKcIlGMOJEEeHVjtfNqubvOcYHZSpEJINH9r7t1gxnRYTH+K5bUzOi9s3MJ0UEyI8Lbv7QCWhrqMFbA5P4zOOvlf/3LSgg6OpC5tFHcUYigejSE9B69TdQu/wk5FIJDP3qboz89t/1DbpUhEjL+uXuJKyOxFOIp7JQlPzmXa/gZVgVEe/OcJJgzasdr5tVrSgi+r3Oi6moclT9XB6P7H3bZlWOC+8AIKwFmtlflKXliIg2q2obeMWYVSt5amY2rQ0hfPnDJ+Kah17AT7YfxPb9o/jSB4/H6avagMWLgb6+GZ+fArAKwGDbUkRO/SA6jjsXis+P1OH9OPyLzcgMHwQAdNXWYt/YGC2vk5j1y/IJqzv78wvwRK1e6BnNK4kdkbCnExW8Qs1EGVUBoDZovwXOa9s6+zeyoojwzq3iiXwnchm+ZlW7gWacFBEt4t1+EqOWrCraI8ICzTgqIvFkRruALW6aX0/4563uwLevWof2SAjdQ3Fc8R9/wb/8+BWMjk7ihwAUw69o53FIfvhzWHTNfWg44T1QfH5M7ngK/d+7GZnhg/ADeFdjI/ZSESI9CxvdWYDntVGVwSvUTBvdFfBAVVuTv97b8oiwaz43s6oFRYQVQZIZVQFSRPQlQAEHZlW7u2Y4OagZumRoXxGZdNmsOpXKYjqVnbHDwS59BX9IJBwQ8iTkNRcetxCnrWrFvz25Ew9vPYAfbz+IJ//xexgZG8JiRYECAP4gApFWAICq5jD11nOI/eWnSPW9BSD/5DH15S+j5uabAQoqqwjWL2vGwdFpvLBvFGcevUDIa3i5dddII6d9M0JbM0H7LXAt0Mzhw2fIhiIi6+guQIUIl6Q5pmikbLZmeCkiTpfeqapqiHgX+9ZoCAVQ4/chlc1hOJ5EZ43zC+ChsXxbZvE8asvMJhoO4s5LT8Cla5dg089exduDkwg2L5rxOWomhcnXfofY8z9HZrRX+3gEwNDEBGoaGlw+NeGEdctb8PjLvUINqz0eh5kxGjmZVfXWjAhFxH5wJLdAMxs5IrynNHlChQiH7H2n47tOZTqGExMVOw+bMxetiLB9M/2xBEbiKXRy2PbZV8hamG9tmWKsX96CJ248E2/2xXDKhtMAqICa/8fLjPYhl9D3k9QC+NTSpfjirl3wkSm14pi9AC8gIJ5bntZMQRGZdmhW1Voz4hQRe60ZPh4RO75EmRUR+U7kMnoh4mR812GgGW9FxGYhwtZmK4ruNxEJa88Mc8oS6S0UIos8dP27SdDvw4mdTXjwrs8i1bsLqb63kOp7a0YR8q9LlmByyxZ8qbubipAKxY0FeNIUIsysyqs1I0ARcTKdmOQ0oGBHEUlymBAVhXwnchmerRnru2b4KiJO5tsBPUOkvob/7H0xeI/w9hYmZrwcP/SCK664AuoPfgAVyP/q6IC6cSPUJ5/E5gMH4KOdMRWN6AV46WxOK+K9LkQamVmVU2tGpEfEznQir/aIURFRVXNx+LwW7omg6q9OeqCZF8mqfBURJ4l/gDFV1Z03ahvnEV5NEZknGSKWuOKKfGtGVYH+fuC554ALL6QCZJ6wYUV+jPcv3fwLkb6xBHJq/jq2IBLi/v2twC1HRGRrxlHEO5/2iFFRMXvfofFdieE5vpvK5pCzsKyJd4XKFJFkxto5GG5NzDBaOI/wsgyRxVWmiBDzH1aIPN89YvoJ2CwHDBMzXgdd6WZV+5N/gGCzqqOld3yu+cavN6vE660ZUkSkI8nDrGp4U7BWjxl4V6jGEVg71bpbEzMM5hEZ4uARUVVVU0Tm89QMUZ2c2NmEcNCH4XgKuwcnuX5vWfwhgDHQLOWo4BI5vuss4p2PRyTgU8C652a9iUlOry0C+U7kMjynZgBrPhHeEe/GFo+dHxK3wswYeqiZ89bMSDyFZCYHRQE6Gr2VlwmCNzUBH05emveJbOXcnukZLSgizd4X8CzQLJ1VHe3MihUUFRGtmbCTQoRTZIOiKJbTVUkRkRimYAQdtGbsVKfGz+VVofp8iva97MiG7rdm8gUDj9YMa8u0NYSk/EEjCKdsWJEPqvvL3mGu3/eAJGFmQP5ayB4KnRhWJwqKiHStGY4hluwBOGFREeE1HMET+U7kMmkOioiiKLbmunnNlBtxEmpWya2ZQ1qGiPdPdQQhgg0rdcMqT5+IDFt3GYqi6FkiNg2rmWwO8UKRILQ1YydHhGOIZdji1nca35UYpoiEHIYE2ckS4b30DgDqHKT+TbLxXZemZnjum9HCzKpxYoaoCtZ2NaEm4MPhiSS6h+Jcvmcup6L7cP57LWut5/I9neI0XZUZVQGxishUOmu5IGTtpjoOKy201ozJe47uT5FPMaZChIMiAhhkMg89IvnvZX8hk66IuLOnhW3gnU5nHW0MBvQMEVJEiPlKOOjH2q4mAPzGePcOxTGRzCAc9GHVAjkKEaehZqwQqQ36HbXcS8GGAlTVemTDlPaw57xA0mIjSBGpfPgVIjZaMwIid5kiYif1bzLBChF3Kub6Gr/29+40XbWqM0SIqmEjyxPh5BPZcXAMAHD84kYh0fF2cBpqpk/MiGkx1xoeHK0qz/HCAxcPRSRkcfkemVUlJsnBrArYW8ssQipzsgdhMuWuWVVRFLRxSldlhUi1paoS1cWGlQXDKiefyI6D4wDy48Gy0ORwA29MM6qKUXYDfp+WO2XFi6eq+iQQT0XEtFlVgBWAF/KdyGV4mFUBo0fEnCKiqqpeoXJ8YzhJ/Yu7PDUDAC0NfEZ42dRMteyZIaqTk5c2I+hX0DeeQM/ItOPv90pBETmpq9Hx9+KF08V3+uiuuOuYNp1ocddLthA0ydMjYrY1I2I4ghdVX4jw2DUDWN83k8rm2LJUMVMzNjwXbk/NAPoIr5PWTCabw0CMeUSoNUPMX2pr/Jp6sbXbWXsmnc3hjd4YALkUEc2s6lARETExw7Cz6dyoUrMWuhPCNhURGt+VEN5mVdMpdwblhKdUxn5AbLVmCkYqNwsRHq2ZgYkkcioQ9Ctoq6cwM2J+o+2d2evMsLqrfwLJTA7RcADLW70f3WXoKqm9a4Ie7y6uENGmEy0oIuxBLxz0wc9hqWjI6vhumsyq0pLi5CS2uviO+UMUxbkaY4QVEexNb4XJZP5JwtXWDIcRXuYPWdgYdmVrMEF4ie4TcaaIsLbMiZ1Nnu+YMdLq8OFEX3gnsjVj/YFP84dwUEMA64qINhxB47vywSNZFbA+NWOsTnleBNjcfCxhpzXjviLCRniHHGzgpR0zRDWxblkz/D4FB0entSA/O+zoYUZVefwhgH5NsLuV243WjJ1QM21ihtNUouWId1JE5IVba4aZVU2PUokJl2FyJPthtIIe8e5excwj1Kx3jDJEiOqhIRTA8UvyxYOTMV6jIiITmiJi0zcmcvMuQ0+wNv/Ap2WIcFJE7NoByKwqIfzMqlZbM4UwM85vCjY7P2FREUlnc1pR5q4i4uyiAwB94yzenYyqRHWw0aFPZDqVxduFLb4yTcwAuiIykcxYikNg6K0ZN8yq5nOjeGaIAHYi3ml8V1q8CjRLCFpAxBSRCYuKiNFTUqkekUXUmiGqBLZ3xu7kzOu948jmVCyIhLAwKlcBHw0HEPTn29V2rgvuKiJWPCJ84xGsKiI0visxXk/N8FZENI/ItDVFhP3whgI+IbHIpWhl47vxpO2AJtaaoTAzolo4ZXkLgn4F+4en8NbAhOWvf6UQZHZSZ6NURlUgH3TY6mCs35XxXRsxCcyDx1sRMe0REZDkzQv5TuQy3FozmkfEmiLCWyZjcuRE0qIiknI/QwTQWzOJdM7WyDEA9BZaM4uoNUNUCZFwEGcdvQAA8OsdfZa/foek/hAGU0rtmNi1QsSN1owdRYTX1EyQ7Teb+wyiAjR5Id+JXIZbsqrl1owYmYyNrFn1iHiRqgrknw5YhW5Hhp1KZbQoaDKrEtXEe09cBAD49at9ltVEPdpdLn8Iw4l3jF37RI7v2slr0hQRTsMAVu45xgBN2r4rIZoi4nprRrRHJGPp4uRFmBlQ2DfjYISXtWUaQgGhT0AEIRvnr+lAjd+H3YOTeGtg0vTXjU+n0T0UByCvIsKuCcMWVz+oqqoXIiLHd4PWx3d5KyIhC4qIsVih1oxk5HIq0tn8zdqzqRnO1SmbmsnmVIvVujetGcCZYZUmZohqJRoO4qx3sPZMr+mve7WghnS11Go/e7JhN9RsKpXV9rmIfDCpY8GRlnJEmEeEl1nV/PZdZhngHaDJC/lO5CJMDQF45IhYXUAkxjhUG/Rr8cFW2jOTCfczRBhOZFiamCGqmUsK7Zn/stCekTU/xIgeambtmsD8IQGfInRMNVIoRCYtTCdOcc5pCltYtGq838hmTgaqvBBJGwoR58mqNqdmOCsiiqIY0lXN/5BMeuQRAXRFxE6kM4WZEdXMeavbURPwYe/hOHb2m5ueYUbVtTIXIlqombXWjLZ5tzYo9IbbULjGTlpYpcFbEdGnZsy3ZmQc3QWqvBBJGSpJbtt3LeaIiKjaozayRLxszdi96ADGeHdqzRDVRyQcxNnvsDY984qk0e5GNJXU4sMJu+aJzBAB9OukFdVZzxHhZVY1r4jIPLoLVHshou2ZURwvS7PsEdHeGPwrVDv7ZiY9Gt8FdBnWnkeEFBGiurEyPTMYS6A/loBPgRYTLyNOWzOijeu2FJGkKEXETCEiRoHnRXUXIhk+GSKAjV0zmlmV/z+BHmpmXRGpvNYMZYgQ1c15qzsQCvjQPRTHG32xsp/LgsyOam/w5GfdLLpZ1VrQod6aEfv/pnlELBQi+tQM74j3ue85ojyJvJDzVC7BCpEgh38cpmykzG7fZbn/AhSRqGGE1yzMrOqFItLWoF90rKCqqhZmRqmqRLXSEArgnGPaAczdnvnBX/YDyG/wlRm7QYdaaybkkiJiISZBzxHhHfFuXhGRMcwMqPZChFOqKmB/fFfEGyNipxBhmyE9UUQKrRmLMuzoVFr7e1xIHhGiijHTnvnDzkE8veswgn4FnzhzpZvHs0xdTUCLUbfSnokl3FFE2ANbJqeavuaLUkRS2Zw2slyKpMR7ZoBqL0Q4paoChtaMxUAzET07O1Mzcc6jZVZgMuxQPGVJhmVtmbaGkLQ/YAThBuce245w0If9w1N4vffI9kwqk8MXf/0GAOBj71qBlQsa3D6iZZgqMmRBKXXLI2IMJTP7wKdNzXBWRIC5lXiZN+8CVIgA4FSIBKzmiIhZegfoiYKWpmZS4jdWlqI9GoJPyf97HLYwOaNNzJA/hKhy6kMBnHtsvj1zxy9fP8K78L3n9mHv4TjaGmpw/blHeXFEy9gxrDKPSERwIeLzKZoqYsYnks7mtPsNb0UEmHuElxQRifG2NSMm4h2wt29GyxHh5Oi2QijgR1dLHQBg96D5qOqeUTa6S/4Qgrjh3KMRCQfwwv5RfOzB5zWVc2gyiXt+9zYA4F8uPKZiViG02Rjr1zfvir+OaYWIieus0efCa2rG71MQ9OenPRNzKPE0visxfBWR/PdIZXPIzdGvA8QGzNiZmtGTVb1x0h9VkIr3WChE3ioEOL2jQ36ZmSBEs3pRFN+/ZgMioQC27RvFxx7chngyg69seQsTiQyOXxLFX6/r8vqYprGTJaIvvBNfbDHDqplN58wfEvQrXO43DLNKfNWO7+7btw/XXHMNVqxYgdraWqxatQq33347UinrI5qiSPNURAz/wMbo+FKIVUSsm1W9DDQD8uOEgDVFZGd/vhd+zMKokDMRRKWxtqsJ3/94vhh5ft8IPvLAc3h02wEAwO3vO05b/1AJMBO7tdaMO4FmgDVFhHeGCIN5PuZSRKp2fHfnzp3I5XL49re/jddffx1f+9rX8K1vfQu33nqrqJe0TFKAIgKY84loyapCFBFrhUgqk9OMVE113si2q1ghcthcIZLLqdrG0WMWRoSdiyAqDWMx8tqhGFQ1v5PmlOUtXh/NEnbG+vXWjPjrWMRCqBnviRmGVUVE1kJEWNl40UUX4aKLLtJ+v3LlSuzatQv3338/7r77blEvawmerZmAT4FPAXIq68eV/0HQpTKBgWYmzapjU/knDp/ijqRZDKuKyIGRKUynswgFfFjeWifyaARRcaztasL3rjkVf/ed56EA2PRXq70+kmXsLMNkD1+uKiImChHeGSIMpqjPaVbVckTkbM24qsOPj4+jpaV0VZ5MJpFM6tVvLFY+JdApPM2qiqIgFPBjOp01uQ1RnEdEn5oxp4iMTuULlqa6GsdR93ZhhchALIlYIj1nQcQWfB3d0YCAhGutCcJr3rm0Gc/+67lIZnJoK0ygVBKthdbMkBWz6rQ747uAtX0zohQRpqgn5rjn6Aq8nNdK1061Z88e3Hvvvbj22mtLfs7mzZvR2Nio/erqEmus4pmsCljLEkkKXHpnlAznCroB9B0vzR61ZYD8haM9kr/wmDGsav6QDvKHEEQpIuFgRRYhgK6ImN1BlczoD4FutGas7JvhvXmXYXa1iDa+K6kiYvkueMcdd0BRlLK/XnjhhRlf09vbi4suugiXXXYZPv7xj5f83ps2bcL4+Lj2q6enx/r/kQWYWTXE6Yma9d+8XkJklCXNGKlGC60ZtvPFK6y0Z3YVFJHVi8gfQhDzEaaIjMRTpiYRjcqEG6b7iJXxXUGBkWYVEdnHdy3/a11//fW4/PLLy37O8uXLtf/u7e3FOeecg9NOOw0PPPBA2a8LhUIIhdyr3nl6RACDcWiON0U2p2ptIRFvjFDAj1DAh2Qmh1gijcY5lA5WiDTVeV+IPLtn2JRhlRUiZFQliPkJezDK5FTEEuk5r0/axEwo4Mp0kAyKSNikRyQhuSJi+W+lra0NbW1tpj730KFDOOecc7Bu3To8+OCD8Pnkqsb4FyLmWjPGPxc11x0JB5GcTJrqX44WpM8WCQoRYO7WzHQqi33DcQBUiBDEfKUm4EM0HEAskcHQZGrOQkTLEHGhLQMADSHzXjxRiojZh1/ZFRFhp+rt7cXZZ5+Nrq4u3H333Th8+DD6+/vR398v6iUtk+RoVgWMHpE53hSG1o2oN0bUwuTMSDz/Oc1et2YWmGvNvD04gZyaf2JaUKH9b4Ig5qZNi3mf27DKrnVuranQFZG5r7GiFZE5PSLVOr67ZcsW7N69G7t370ZnZ+eMP7Oy2Ewk3M2qJme6WfhMwKcIm/iIWJicYeO7XppVAT1L5MDIFBLpbEm1iE3MHLswAkWpnIAmgiCs0dpQg71DcVOGVbZnxq0IgoiF8V3hOSJzKiJVumvm6quvhqqqRX/JAs9kVcB8a0ZbeCewX6fvmzGhiLBCxGNFpD0SQiQUQE6F1nopBvlDCKI6aDFs5p6LCa8UESvJqpxNtOY9IrR9V1rEeUS8X8lsZd/MqDa+620hoiiKnrBapj2zy6CIEAQxf2m10ZpxzyNiJdBMUI5I4WHWdKBZtSkilUCKc9/MrEwmMsyMYWXfDAs0a6n3fiunblgtrYjQjhmCqA70DbxmFBHWmnFJEbEQaBYvtGa454hYfPgVsduMB3KeyiW0ZFXegWYmZTKRb4qIthnS/NSM14oIYMgSKTHCOzSZxNBkCopCW3cJYr6jKSIm9s3oC+9c8ogUrrHJTE57qC3FVMGsyn1qxqQiotkBSBGRj1Qm71cJcveImFzJLPBNoS++K9+aSWdzWrEiRSEyx+QMa8ssa6nj/nRBEIRcWNk3E9PGd925LtQb/B7xOR742J97poi48PDrBDlP5RI8d80AVloz4t8U2vjudPkfkFHjwjuXeqvlYIrI3sOTRePpd5JRlSCqBmZWHbZgVnVraibo92k+v7l8IqIUEeseETlv+XKeyiVShb6Z24Fm+gIi8YrIXDkio3F94Z0baYRz0dVSh5pCKuyh0ekj/nxnH/lDCKJasJQjMs0277r3QGU21GxKkEdEL0RKP/yqqkpmVZnhPjWjeURMtmbc8IjM8QOix7t7r4YAgN+nYGVbPQBg9+GJI/5810BhxwwpIgQx72ktKCKjU2lksuWvq4MTCQDu7syKmIx5Z+O79cJaM6Uffo0KPY3vSohXrRlR/UIjrM0ytyIiR7y7kVIjvNmcircGqDVDENVCU10NmFDL8o6Kkc7m0FNQUJe31blxNADGEd7S19lsTsV0muWIiGrNlL7nGO9HpIhISLpgVnW7NcNUCpHBO2YVkRFJFt4ZKWVYzSeu5hAO+rCstd6LoxEE4SJ+n6L7RMoYVg+NTiObUxEO+tARCbt1PFMjvNMG/4Y4RaRcIZJ/fUUBgn7v2+/FqOpChPv4rkkHsxsJgFGTUzNjEmWIMI4qoYgwf8jR7REp/CwEQYiHFSLlYt67C0nMy1vr4XPx2mBmAy9beKco/FsjTBEpFxmRNIzuyroSo7oLkQzn1oz2ppirEBFvqmKFSCJdfsad/XB7He9uxFiIGFcC7KREVYKoOlrr84bVoTKG1X1D+UJkWat7bRnAsG+mjCLCFt7V1wS4FwJmIt5lDzMDqrwQSQqLePe+NdNg+N7lVJHRKXnCzBgr2urhU/K5AIcNFx/aMUMQ1YeZLBFWiCxvc7dla0YR0T2B/P0ZZnyJepK3vLd7eU/mAmx8l1+gmTmzqr6uWpwi4vcp2l6Dcv1LGc2q4aAfXS35J5vdg5M4NDaNh7fux/P7RgAAx9LoLkFUDW0m0lX3DU8BAFa47B0z4xHRM0T4P3haUURELll1SlVHU6azedmf364Zsx4R8YoIkJ+ciaeyZSdnRqZYjog8HhEgb1jdPzyF637worYLB8j/HR+/hAoRgqgWWk2YVdm2bikVkZQ4RUSbmilnVq0ARaSqCxGvds24lQAYCQfQN16+Wh8rtGbcnL03w7GLIvjdzkGMTqXhU4B1y5px9jHt+KsTFkk14UMQhFhaGsqnq6azORxko7suKyJmPCJTgjJEAL24yOZUZLI5BIqo+7KHmQFVXIhkc6oWIc47R2SuBUhuKSJm9s3IaFYFgI+fsRI1fj9WLKjHWUe3UfFBEFUKM6uWSlc9WBjdrQ360RENuXk0a4oI5wwRYGa7JZHJoaFoIVIwq5IiIh/GYsH98V22rlqsIjLXvpl0NqedRSazKpAvjG48/2ivj0EQhMe0zaGIGCdm3B5P1SLeTYzvilBEjA/RyXRW86wY0TbvSuwRkbdEEoyxEOFmVg3OPTWTzua0gBu3FJFSHhGWIaIoQKMEC+8IgiBm06rtmyleiHQXCpEVLvtDAEOyahnVmY3vivCI+HyK9iBdyidSCYqIvCcTTCprLET4VNHa1EyZHBFjL7FBeCFS3tGt7ZmpDVJAGEEQUsLGdyeTmaLTIcyo6kXaspldM2zhnYipGQAIB8pPzmgeEcoRkQ+jUZWXnGemNcOKgtqgn5sSU4q59s2w0V3Z2jIEQRCMSCigPSwWS1fVRndd3DHDaDATaMbMqgI8IsDcQZpubHt3SvUWIqxK5FgMsEIklc0hl1OLfk7MhXh3hllFRDajKkEQBENRFCxszO+Pmb32ATCEmXmgiDBVO57KasMPs5nSxncFKSIsS6SEJUAb3yVFRD5SnFNVAb0yBWa2foy4NTGTf43yUzMj8fzHmyXLECEIgjBy6vJWAMBze4dnfDyVyeHgKFNEvPOIAPp0zGz0iHdBisgcloBKGN+t+kKEZ3vEaAYq9aaYcCFVlTHX1IyM8e4EQRCzOX1VvhB5ds/MQuTg6BRyat4IuiDi7ugukL/ms7ZRqfYMm5qpE+URmUMRYa0ZMqtKCO8wMwAI+BQwz2epyRk3FRFtA2+yvEdEtjAzgiAII6cVCpFXD47N8LwZjapebJZVFEX3iZQwrBqX3okgHCi/gVc3q5IiIh0iWjOKosy5bybmUqoqMLdHZIRNzZAiQhCExCxuqsWKtnrkVOD5vSPax7uHvDOqMuYKNZsSGGgGGGMjaHy34tAUEc6TK3NlibiqiLCpmenyOSIt9eQRIQhCbpgq8uc9Q9rHvDSqMlioWenWjDuKyJzju1SIyIcIRQTQ/7ETc3pE3J2aUdUjHd0jNL5LEESFwHwizxl8ItqyOw8LkcicrRlxS+8APTG1lCKije9Sa0Y+tEKEtyKitWbmUkTcaM3kXyOTU4sWRmM0vksQRIVw2sp8IbKzfwJDhb0zXm3dNaK1ZuZSRASZVUNmA81IEZGPtACzKmAINSupiLjXmqmv8Wvm2WKhZqSIEARRKbQ2hHDswggAYOveYaQyORxiW3e99IgUCoxi+2ZUVdUUEWHju0HWminhEUmTWVVahLVm5jAOxVwc31UUpWSWSCabQ0xbeEceEYIg5Of0VW0A8mO8PYXR3foaPxY0uD+6yyiniCQzObCcM1Hju3qid4nxXTKryktSlFnVdGvGncXH7HVis35IxqZp4R1BEJWF0Seib931ZnSXoXtEjlSd4waVpFaQIhE2qYiQR0RCRJtVSykibppVAX1MePbkDMsQaawNIiB45w1BEAQPTl3ZAp+S37jLws28SFQ1Ui5HZKqQIVIb9AtbLKoFmpX0iJAiIi3MI8J78ZxZj4gbOSJA6SwR8ocQBFFpRMNBnNDZBAD46YsHAXjrDwH01kyxvCbNHyIoQwTAnNlVZFaVGHGKiGytGeYRmflDMjpFe2YIgqg8WHuG5SAt83B0FyiviLDNu6IW3gFzKyIJas3IS0pQlVjOrJrO5jBdeLO4YVYFgGgt84jMas3QnhmCICoQVogwvG7NRMqYVacEZ4gAZhQRas1Ii4hdM0B5j4jxjeq2R2T21MwoZYgQBFGBrF/WMmPIwMswM8CQrFpGERGVIQKY8YjQ+K606Nt3+RqIQmUWELH2SDjo4+5NKUUpjwgtvCMIohKprfHjnUubAOTbIm0N3l7DynlE3FBEtGTVIr7EXE7V7nVhUkTkQ981w/cNUk4RcTNDhFFqamYknv99E3lECIKoMFieyLLWOk9Hd4E5PCKCN+8C5XNE2H0OIEVESkQHmhWTyfSJGXfaMkBpRYTFu7eQR4QgiArjw+uWYPWiKK7YsNTro+gekeSRO72mkmI37wLlc0SMKonMHhH37oiSIaoQYeFgY0U23k54oIiUmpoZIY8IQRAVSmdzHX5z45leHwOArohkCzu9ag1tGDcUkXCZje8sVdWnAAFBOSY8kLdEEoyoQqQjGgYADMaSR/yZ26O7QJmpGcoRIQiCcExdjR+sOzQxK13VDUWE+RLLKSLhoN/zFlY5qrcQ0TwifP9xFkTyOw8GJhJH/BlTRNwKMwPmzhFpqSePCEEQhF0URdF9IrOus24qIokiikgljO4CVVyIiNq+K5siou+a0Sv1TDaH8WlmViVFhCAIwgmREoZVV3NEiikiWl6WvEZVoIoLEfYPxHtqpr2giEwmMzMWHgH6mmhXWzNhfca9Z2QKALQiBACaaOEdQRCEI0pt4HUjRyRkUERmm2XZ0ARTTWRF7tMJRJRHpCEU0KrfwYmZqogXZtW2hhqc1NkIVQU+9t1tGJ9Oa2FmtPCOIAjCOaw1M+GBIsKmZlQVSGdnFiKkiEiOqEJEURRDe2amTyQ27b4ioigKvn3VeiyMhrF7cBL/38PbtbYR7ZkhCIJwTgNTnj3wiBj9H7N9IppHhBQROUllxSSrAkbD6kxFxItAMwBY2BjGd64+BfU1fjy7Zxifefw1ADS6SxAEwYOSHhEXpmZq/D5tamd2fhWbpCGzqqQws6qIf6BSiogXZlXGmsVRfPPKk+H3Kdg7FAdAYWYEQRA8KJWuOuWCIqIoip6uOsuwyhQRmTfvAlVciKQEmVUB3bBa2iPiTY7c2ce04wsfOE77PU3MEARBOKfUvpl4wSNSL1ARAQz7Zma3ZkgRkRtRHhEA6IgWWjMlFBE3c0Rmc+WGZfjk2asAAGu7Gj07B0EQxHxBV0RmB5rlC4M6gYoIAIRLhJpVilmVIt6FtmZmKyLetWaM3HLRsfiHM1agrSHk6TkIgiDmA5Ei47upTE7zIopszQC6GXW2IsI8I2RWlZSkK2ZVXRFJZ3OYLrwp3DarFoOKEIIgCD4U84hMp/SioFbg+C5Q+YpIVRYiqqoKS1YFdEXksEERMVbKXisiBEEQBD+KeUSYP6TG7xNynzFSavEdRbxLTCanggXQhQSaVScM6arsDRoO+hCkEDGCIIh5QzFFRAszE2xUBUovvtPGd6k1Ix/MHwKIUUSKpat6lSFCEARBiEXziBgKES3eXbA/BDDEvKeLKyJhas3Ih+hCpFi6qixGVYIgCIIvDaEjk1XjLsS7M7TFd5lZHhFSROSF+UN8CuD38TerAkemq3qxZ4YgCIIQj+YRMbZm2OiuwIV3jHAJRSRRIWbVqnw8Twoc3WWUUkSipIgQBEHMK5hHJJXJIZnJ4sX9Y7jnd28DAOpdUET0QLPZikhlbN+tyrsim+2uEWganZ2u6nWqKkEQBCGGBoPq8Xf/93n8pXsEQL4tc/mpS4W/PpuKOdIjQoqItOhhZuL+cWanq2oekRC1ZgiCIOYTfp+Cuho/plJZ/KV7BAGfgis2LMUN5x6ttelFwhSRI3NEKmN8t7oLEQFhZozZ6aqsd0iKCEEQxPzjqPYG7Dg4jvedtBj/9J53YHlbvWuvrS29OyJZNX+vk33pXVXeFUWGmTFmp6uSWZUgCGL+8v1rNiA2nUZXS53rr11aEamMpXdVWYiI3DPDmJ2uGmNm1dqq/CsnCIKY1zTWBtFY682DJiWrViBJFxQRVohMJDOYSmUMOSKkiBAEQRD80HJEZisiWo4ItWak451dTfjR/z5NaJXI0lWnUlkMxpI0NUMQBEEIoVSOiJasSuO78tFUV4NTV7QIf52OaBjdQ3EMxBKITVMhQhAEQfBnzmRVycd3XSmTkskk1q5dC0VR8PLLL7vxklJgTFfVA82oNUMQBEHwo7QiUhlmVVdOd8stt2Dx4sVuvJRUGNNVadcMQRAEIYJQkWTVbE7VwjtlH98VXoj85je/wZYtW3D33XeLfinp6CgoIr1jCUwXKlUyqxIEQRA8KZasalzuKrsiIvTxfGBgAJ/4xCfw+OOPo65u7tnqZDKJZDKp/T4Wi4k8nnDaC+mqew5Pah8jRYQgCILgiZYjYhjfNY7yyl6ICDudqqq4+uqrce2112L9+vWmvmbz5s1obGzUfnV1dYk6niuw1szuwXwhEg76EBS434YgCIKoPsJFxndZuJnfpyAg+X3H8unuuOMOKIpS9tcLL7yAe++9F7FYDJs2bTL9vTdt2oTx8XHtV09Pj9XjSQUzqx4amwZAbRmCIAiCP6EiZlVtdFdyNQSw0Zq5/vrrcfnll5f9nOXLl+POO+/E1q1bEQrNXPizfv16XHnllXjooYeO+LpQKHTE51cyTBFhUFuGIAiC4I3emtEVEW1iRnKjKmCjEGlra0NbW9ucn/eNb3wDd955p/b73t5eXHjhhXjsscewYcMGqy9bkRxZiJAiQhAEQfCFeUBSmRxUVYWiKJo6Irs/BBBoVl26dOmM3zc0NAAAVq1ahc7OTlEvKxXGdFUAiJIiQhAEQXDGOJ47kcwgGg5qiojso7tAle6acROjKkKtGYIgCII3dUE/ljTVAgC++fvdAIypqvLf5l074fLly6GqKtauXevWS0oBM6wCQCRErRmCIAiCLz6fgi984DgAwH/8aS9e6RmrmM27ACkiwiFFhCAIghDNeas78IG1i5FTgVt+skNL85Z9zwxAhYhwOoyKCJlVCYIgCEHc/r7j0Fpfg10DE/jmH/ItmpDkm3cBKkSEw9JVAVJECIIgCHG01Nfg84UWzduFIE1SRAhqzRAEQRCu8d4TFuGCNR3a70kRIWaaVak1QxAEQQhEURTceenxWlxEmBQRwqiIUI4IQRAEIZr2aBj/50MnoL7Gj9NWtXp9nDmhO6NgZrZmSBEhCIIgxHPJiYtx8fGL4PcpXh9lTkgREUxDKIDW+hoAM9s0BEEQBCGSSihCAFJEXOFbV63DYCyJhY3huT+ZIAiCIKoIKkRc4JTlLV4fgSAIgiCkhFozBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4htTbd1VVBQDEYjGPT0IQBEEQhFnYfZvdx8shdSEyMTEBAOjq6vL4JARBEARBWGViYgKNjY1lP0dRzZQrHpHL5dDb24tIJAJFUbw+jufEYjF0dXWhp6cH0WjU6+PMW+jv2R3o79kd6O/ZPejvWkdVVUxMTGDx4sXw+cq7QKRWRHw+Hzo7O70+hnREo9Gqf5O7Af09uwP9PbsD/T27B/1d55lLCWGQWZUgCIIgCM+gQoQgCIIgCM+gQqSCCIVCuP322xEKhbw+yryG/p7dgf6e3YH+nt2D/q7tIbVZlSAIgiCI+Q0pIgRBEARBeAYVIgRBEARBeAYVIgRBEARBeAYVIgRBEARBeAYVIhVOMpnE2rVroSgKXn75Za+PM6/Yt28frrnmGqxYsQK1tbVYtWoVbr/9dqRSKa+PNi+47777sGLFCoTDYaxbtw5/+tOfvD7SvGLz5s045ZRTEIlE0N7ejksvvRS7du3y+ljzns2bN0NRFNx0001eH6VioEKkwrnllluwePFir48xL9m5cydyuRy+/e1v4/XXX8fXvvY1fOtb38Ktt97q9dEqnsceeww33XQTbrvtNrz00ks488wzcfHFF+PAgQNeH23e8Mwzz+C6667D1q1b8dRTTyGTyeCCCy5APB73+mjzlm3btuGBBx7AiSee6PVRKgoa361gfvOb3+Dmm2/GT3/6Uxx33HF46aWXsHbtWq+PNa+56667cP/992Pv3r1eH6Wi2bBhA04++WTcf//92sdWr16NSy+9FJs3b/bwZPOXw4cPo729Hc888wzOOussr48z75icnMTJJ5+M++67D3feeSfWrl2Lr3/9614fqyIgRaRCGRgYwCc+8Ql8//vfR11dndfHqRrGx8fR0tLi9TEqmlQqhe3bt+OCCy6Y8fELLrgAzz77rEenmv+Mj48DAL1/BXHdddfhve99L84//3yvj1JxSL30jiiOqqq4+uqrce2112L9+vXYt2+f10eqCvbs2YN7770XX/nKV7w+SkUzNDSEbDaLjo6OGR/v6OhAf3+/R6ea36iqiptvvhlnnHEGjj/+eK+PM+949NFH8eKLL2Lbtm1eH6UiIUVEIu644w4oilL21wsvvIB7770XsVgMmzZt8vrIFYnZv2cjvb29uOiii3DZZZfh4x//uEcnn18oijLj96qqHvExgg/XX389duzYgUceecTro8w7enp6cOONN+Lhhx9GOBz2+jgVCXlEJGJoaAhDQ0NlP2f58uW4/PLL8atf/WrGRTubzcLv9+PKK6/EQw89JPqoFY3Zv2d2Uent7cU555yDDRs24Lvf/S58PqrfnZBKpVBXV4cf//jH+OAHP6h9/MYbb8TLL7+MZ555xsPTzT9uuOEGPP744/jjH/+IFStWeH2cecfjjz+OD37wg/D7/drHstksFEWBz+dDMpmc8WfEkVAhUoEcOHAAsVhM+31vby8uvPBC/OQnP8GGDRvQ2dnp4enmF4cOHcI555yDdevW4eGHH6YLCic2bNiAdevW4b777tM+tmbNGnzgAx8gsyonVFXFDTfcgJ///Od4+umncfTRR3t9pHnJxMQE9u/fP+NjH/vYx3Dsscfi05/+NLXCTEAekQpk6dKlM37f0NAAAFi1ahUVIRzp7e3F2WefjaVLl+Luu+/G4cOHtT9buHChhyerfG6++WZcddVVWL9+PU477TQ88MADOHDgAK699lqvjzZvuO666/DDH/4Qv/jFLxCJRDT/TWNjI2praz0+3fwhEokcUWzU19ejtbWVihCTUCFCECXYsmULdu/ejd27dx9R4JGQ6IyPfOQjGB4exhe+8AX09fXh+OOPxxNPPIFly5Z5fbR5AxuNPvvss2d8/MEHH8TVV1/t/oEIogTUmiEIgiAIwjPIdUcQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGf8P6ZyOHzGZunmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_41_2.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "backend = np\n",
    "\n",
    "def function(x):\n",
    "    return x*backend.sin(x**2 + 1)\n",
    "\n",
    "def analytical_gradient(x):\n",
    "    return backend.sin(x**2 + 1) + 2*x**2*backend.cos(x**2 + 1)\n",
    "\n",
    "\n",
    "x = np.linspace(-5, 5, 100)\n",
    "plt.plot(x, function(x), label=\"f(x)\")\n",
    "\n",
    "descent_x, descent_y = gradient_descent(1, 0.01, 300, solver=\"analytical\")\n",
    "\n",
    "# Change the backend to JAX\n",
    "backend = jnp\n",
    "jax_descend_x, jax_descend_y = gradient_descent(1, 0.01, 300, solver=\"jax\")\n",
    "\n",
    "plt.scatter(descent_x, descent_y, label=\"Gradient descent\", marker=\"v\", s=10, color=\"red\") \n",
    "plt.scatter(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\", s=5, color=\"black\")"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}