FYS-STK4155/doc/LectureNotes/_build/jupyter_execute/exercisesweek41.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4b4c06bc",
   "metadata": {
    "editable": true
   },
   "source": [
    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
    "doconce format html exercisesweek41.do.txt  -->\n",
    "<!-- dom:TITLE: Exercises week 41 -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcb25e64",
   "metadata": {
    "editable": true
   },
   "source": [
    "# Exercises week 41\n",
    "**October 4-11, 2024**\n",
    "\n",
    "Date: **Deadline is Friday October 11 at midnight**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb01f126",
   "metadata": {
    "editable": true
   },
   "source": [
    "# Overarching aims of the exercises this week\n",
    "\n",
    "The aim of the exercises this week is to get started with implementing\n",
    "gradient methods of relevance for project 2. This exercise will also\n",
    "be continued next week with the addition of automatic differentation.\n",
    "Everything you develop here will be used in project 2. \n",
    "\n",
    "In order to get started, we will now replace in our standard ordinary\n",
    "least squares (OLS) and Ridge regression codes (from project 1) the\n",
    "matrix inversion algorithm with our own gradient descent (GD) and SGD\n",
    "codes.  You can use the Franke function or the terrain data from\n",
    "project 1. **However, we recommend using a simpler function like**\n",
    "$f(x)=a_0+a_1x+a_2x^2$ or higher-order one-dimensional polynomials.\n",
    "You can obviously test your final codes against for example the Franke\n",
    "function. Automatic differentiation will be discussed next week.\n",
    "\n",
    "You should include in your analysis of the GD and SGD codes the following elements\n",
    "1. A plain gradient descent with a fixed learning rate (you will need to tune it) using the analytical expression of the gradients\n",
    "\n",
    "2. Add momentum to the plain GD code and compare convergence with a fixed learning rate (you may need to tune the learning rate), again using the analytical expression of the gradients.\n",
    "\n",
    "3. Repeat these steps for stochastic gradient descent with mini batches and a given number of epochs. Use a tunable learning rate as discussed in the lectures from week 39. Discuss the results as functions of the various parameters (size of batches, number of epochs etc)\n",
    "\n",
    "4. Implement the Adagrad method in order to tune the learning rate. Do this with and without momentum for plain gradient descent and SGD.\n",
    "\n",
    "5. Add RMSprop and Adam to your library of methods for tuning the learning rate.\n",
    "\n",
    "The lecture notes from weeks 39 and 40 contain more information and code examples. Feel free to use these examples.\n",
    "\n",
    "In summary, you should \n",
    "perform an analysis of the results for OLS and Ridge regression as\n",
    "function of the chosen learning rates, the number of mini-batches and\n",
    "epochs as well as algorithm for scaling the learning rate. You can\n",
    "also compare your own results with those that can be obtained using\n",
    "for example **Scikit-Learn**'s various SGD options.  Discuss your\n",
    "results. For Ridge regression you need now to study the results as functions of  the hyper-parameter $\\lambda$ and \n",
    "the learning rate $\\eta$.  Discuss your results.\n",
    "\n",
    "You will need your SGD code for the setup of the Neural Network and\n",
    "Logistic Regression codes. You will find the Python [Seaborn\n",
    "package](https://seaborn.pydata.org/generated/seaborn.heatmap.html)\n",
    "useful when plotting the results as function of the learning rate\n",
    "$\\eta$ and the hyper-parameter $\\lambda$ when you use Ridge\n",
    "regression.\n",
    "\n",
    "We recommend reading chapter 8 on optimization from the textbook of [Goodfellow, Bengio and Courville](https://www.deeplearningbook.org/). This chapter contains many useful insights and discussions on the optimization part of machine learning."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d576598a",
   "metadata": {
    "editable": true
   },
   "source": [
    "# Code examples from week 39 and 40"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d07a903",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Code with a Number of Minibatches which varies, analytical gradient\n",
    "\n",
    "In the code here we vary the number of mini-batches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4db17153",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.16413938]\n",
      " [2.88594444]]\n",
      "Eigenvalues of Hessian Matrix:[0.2965315 4.059181 ]\n",
      "theta from own gd\n",
      "[[4.16413938]\n",
      " [2.88594444]]\n",
      "theta from own sdg\n",
      "[[4.11131389]\n",
      " [2.89069029]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHFCAYAAADxOP3DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABTVUlEQVR4nO3deVhU5eIH8O8AMogKCiqyKWimLNd9zZVcB/Wq5LW0SNsXl8huv/JWV71lXnu6XS0ts1varTAr0TZGMwXR1DKXuuJuqKiQOwjKCMP7+2OakZEZZmGWc+Z8P8/Do3POmTPvmZnD+fKed1EJIQSIiIiIFMrP2wUgIiIi8iaGISIiIlI0hiEiIiJSNIYhIiIiUjSGISIiIlI0hiEiIiJSNIYhIiIiUjSGISIiIlI0hiEiIiJSNIYhIglauXIlVCqV6ScgIACRkZG45557cPToUa+Va+7cuVCpVF57fTk4ceIEVCoVXn/9dW8XhYjsFODtAhCRdStWrEDHjh1RUVGBH374AfPnz0dOTg4OHTqEZs2aebt4REQ+gWGISMKSk5PRo0cPAMDgwYOh1+sxZ84crFu3Dg888ICXS0feVFlZaao1JKL64W0yIhkxBqPff//dtKyiogLPPPMMunTpgtDQUISFhaFv37748ssvaz1fpVJh+vTp+Oijj5CQkIDg4GB07twZ33zzTa1tv/32W3Tp0gVqtRrx8fFWb/tUVFRg9uzZiI+PR2BgIKKjozFt2jRcuXLFbLu4uDiMHj0a33zzDbp27YqGDRsiISHB9NorV65EQkICGjVqhF69euHnn3+2+X4Ybyfm5OTgiSeeQPPmzREeHo60tDScPXu21rHPnTu31j7i4uIwderUWvvcvHkzHnnkEYSHhyMkJAT3338/ysvLUVxcjIkTJ6Jp06aIjIzEX//6V1RWVtbab3V1NebPn4/WrVsjKCgIPXr0wKZNm2ptd/ToUUyePBktW7aEWq1GQkICli5darZNbm4uVCoVPvroIzzzzDOIjo6GWq3GsWPHcO3aNfz1r39FfHw8goKCEBYWhh49emDVqlU23z8iMuCfFEQyUlBQAAC4/fbbTct0Oh0uXbqEv/71r4iOjsaNGzfw/fffIy0tDStWrMD9999vto9vv/0Wu3btwj/+8Q80btwYr732GsaPH4/Dhw+jbdu2AIBNmzZh7Nix6Nu3Lz799FPo9Xq89tprZiEMAIQQGDduHDZt2oTZs2djwIAB+PXXXzFnzhzs2LEDO3bsgFqtNm3/yy+/YPbs2XjhhRcQGhqKefPmIS0tDbNnz8amTZvw6quvQqVS4bnnnsPo0aNRUFCAhg0b2nxfHn74YYwaNQqZmZkoLCzEs88+i/vuuw+bN292+r1++OGHkZaWhk8//RR79+7F3/72N1RVVeHw4cNIS0vDo48+iu+//x4LFy5EVFQUZs2aZfb8JUuWoE2bNli0aBGqq6vx2muvQaPRYMuWLejbty8A4MCBA7jjjjvQunVr/Otf/0KrVq2wYcMGzJw5ExcuXMCcOXPM9jl79mz07dsXy5Ytg5+fH1q2bIlZs2bho48+wiuvvIKuXbuivLwc+/fvx8WLF50+diLFEUQkOStWrBAAxM6dO0VlZaW4evWqWL9+vWjVqpUYOHCgqKystPrcqqoqUVlZKR566CHRtWtXs3UAREREhCgtLTUtKy4uFn5+fmLBggWmZb179xZRUVHi+vXrpmWlpaUiLCxM1Py1sX79egFAvPbaa2avs3r1agFALF++3LSsTZs2omHDhuL06dOmZfv27RMARGRkpCgvLzctX7dunQAgvvrqK7vepyeffNJs+WuvvSYAiKKiIrNjnzNnTq19tGnTRkyZMqXWPmfMmGG23bhx4wQA8cYbb5gt79Kli+jWrZvpcUFBgQBg9f0bOnSoadmIESNETEyMKCkpMdvn9OnTRVBQkLh06ZIQQoicnBwBQAwcOLBW+ZOTk8W4ceNqLSci+/E2GZGE9enTBw0aNECTJk0wcuRINGvWDF9++WWtdiKff/45+vXrh8aNGyMgIAANGjTA+++/j4MHD9baZ0pKCpo0aWJ6HBERgZYtW+LkyZMAgPLycuzatQtpaWkICgoybdekSROMGTPGbF/Gmpeat5kA4C9/+QsaNWpU67ZQly5dEB0dbXqckJAAwNAeKjg4uNZyY5ls+fOf/2z2uFOnTg4935LRo0ebPTaWadSoUbWWW3oda+9fXl4e9Ho9KioqsGnTJowfPx7BwcGoqqoy/aSmpqKiogI7d+402+ddd91V63V69eoFrVaL559/Hrm5ubh+/brTx0ykVAxDRBL23//+F7t27cLmzZvx2GOP4eDBg5g0aZLZNllZWZg4cSKio6Px8ccfY8eOHdi1axcefPBBVFRU1NpneHh4rWVqtdp0Eb18+TKqq6vRqlWrWtvduuzixYsICAhAixYtzJarVCq0atWq1q2asLAws8eBgYF1LrdUfktuPSbjrbn6BANHymqpnNbevxs3bqCsrAwXL15EVVUV3nrrLTRo0MDsJzU1FQBw4cIFs+dHRkbW2uebb76J5557DuvWrUNKSgrCwsIwbtw4rw7BQCQ3bDNEJGEJCQmmRtMpKSnQ6/X4z3/+gy+++AITJkwAAHz88ceIj4/H6tWrzcYA0ul0Tr1ms2bNoFKpUFxcXGvdrcvCw8NRVVWF8+fPmwUiIQSKi4vRs2dPp8rgDmq12uJ74q62Ndbev8DAQDRu3BgNGjSAv78/0tPTMW3aNIv7iI+PN3tsaYynRo0aYd68eZg3bx5+//13Uy3RmDFjcOjQIdccDJGPY80QkYy89tpraNasGf7+97+juroagOECGRgYaHahLC4uttibzB7G3lxZWVlmNR5Xr17F119/bbbtkCFDABgCWU1r1qxBeXm5ab0UxMXF4ddffzVbtnnzZpSVlbnl9ay9fwMGDIC/vz+Cg4ORkpKCvXv3olOnTujRo0etH0u1eHWJiIjA1KlTMWnSJBw+fBjXrl1z9WER+STWDBHJSLNmzTB79mz83//9HzIzM3Hfffdh9OjRyMrKwpNPPokJEyagsLAQL7/8MiIjI52+VfLyyy9j5MiRGDZsGJ555hno9XosXLgQjRo1wqVLl0zbDRs2DCNGjMBzzz2H0tJS9OvXz9SbrGvXrkhPT3fVoddbeno6XnrpJfz973/HoEGDcODAASxZsgShoaFueT1/f38MGzYMs2bNQnV1NRYuXIjS0lLMmzfPtM3ixYvRv39/DBgwAE888QTi4uJw9epVHDt2DF9//bVdveF69+6N0aNHo1OnTmjWrBkOHjyIjz76CH379jVrh0VE1jEMEcnMjBkzsGTJEvzjH//ApEmT8MADD+DcuXNYtmwZPvjgA7Rt2xbPP/88Tp8+bXbhdcSwYcOwbt06vPjii7j77rvRqlUrPPnkk7h+/brZPlUqFdatW4e5c+dixYoVmD9/Ppo3b4709HS8+uqrZt3qve3ZZ59FaWkpVq5ciddffx29evXCZ599hrFjx7rl9aZPn46KigrMnDkT586dQ1JSEr799lv069fPtE1iYiL27NmDl19+GS+++CLOnTuHpk2bon379qZ2Q7bceeed+Oqrr/Dvf/8b165dQ3R0NO6//3688MILbjkuIl+kEkIIbxeCiIiIyFvYZoiIiIgUjWGIiIiIFI1hiIiIiBSNYYiIiIgUjWGIiIiIFI1hiIiIiBRNEeMMVVdX4+zZs2jSpInF4eyJiIhIeoQQuHr1KqKiouDn5776G0WEobNnzyI2NtbbxSAiIiInFBYWIiYmxm37V0QYatKkCQDDmxkSEuLl0hAREZE9SktLERsba7qOu4siwpDx1lhISAjDEBERkcy4u4kLG1ATERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaJ5PQzl5eVhzJgxiIqKgkqlwrp166xu+9hjj0GlUmHRokUeKx8RERH5Nq+HofLycnTu3BlLliypc7t169bhxx9/RFRUlIdKRkREREoQ4O0CaDQaaDSaOrc5c+YMpk+fjg0bNmDUqFEeKhkREREpgddrhmyprq5Geno6nn32WSQlJXm7OERERORjvF4zZMvChQsREBCAmTNn2v0cnU4HnU5nelxaWuqOohEREZEPkHTN0O7du7F48WKsXLkSKpXK7uctWLAAoaGhpp/Y2Fg3lpKIiIjkTNJhaOvWrTh37hxat26NgIAABAQE4OTJk3jmmWcQFxdn9XmzZ89GSUmJ6aewsNBzhSYiIiJZkfRtsvT0dAwdOtRs2YgRI5Ceno4HHnjA6vPUajXUarW7i0dEREQ+wOthqKysDMeOHTM9LigowL59+xAWFobWrVsjPDzcbPsGDRqgVatW6NChg6eLSkRERD7I62Ho559/RkpKiunxrFmzAABTpkzBypUrvVQqIiIiUgqvh6HBgwdDCGH39idOnHBfYYiIiEhxJN2AmoiIiMjdGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiIiISNG8Hoby8vIwZswYREVFQaVSYd26daZ1lZWVeO655/CnP/0JjRo1QlRUFO6//36cPXvWewUmIiIin+L1MFReXo7OnTtjyZIltdZdu3YNe/bswUsvvYQ9e/YgKysLR44cwZ///GcvlJSIiIh8kUoIIbxdCCOVSoW1a9di3LhxVrfZtWsXevXqhZMnT6J169Z27be0tBShoaEoKSlBSEiIi0pLRERE7uSp63eA2/bsJiUlJVCpVGjatKnVbXQ6HXQ6nelxaWmpB0pGREREcuT122SOqKiowPPPP4/JkyfXmRAXLFiA0NBQ009sbKwHS0lERERyIpswVFlZiXvuuQfV1dV4++2369x29uzZKCkpMf0UFhZ6qJREREQkN7K4TVZZWYmJEyeioKAAmzdvtnnfUK1WQ61We6h0REREJGeSD0PGIHT06FHk5OQgPDzc20UiIiIiH+L1MFRWVoZjx46ZHhcUFGDfvn0ICwtDVFQUJkyYgD179uCbb76BXq9HcXExACAsLAyBgYHeKjYRERH5CK93rc/NzUVKSkqt5VOmTMHcuXMRHx9v8Xk5OTkYPHiwXa/BrvVERETyo5iu9YMHD0ZdeUxCwyARERGRD5JNbzIiIiIid2AYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkXz+qCLRERE5CP0emDrVqCoCIiMBAYMAPz9vV0qmxiGiIiIqP6ysoCnngJOn765LCYGWLwYSEvzXrnswNtkREREVD9ZWcCECeZBCADOnDEsz8ryTrnsxDBEREREztPrDTVCluYSNS7LyDBsJ1EMQ0REROS8rVtr1wjVJARQWGjYTqIYhoiIiMh5RUWu3c4LGIaIiIjIeZGRrt3OCxiGiIiIyHkDBhh6jalUlterVEBsrGE7iWIYIiIikiq9HsjNBVatMvwrxUbI/v6G7vNA7UBkfLxokaTHG2IYIiIikqKsLCAuDkhJASZPNvwbFyfNbuppacAXXwDR0ebLY2IMyyU+zpBKCEt94XxLaWkpQkNDUVJSgpCQEG8Xh4iIqG7GcXtuvUQba1qkGjBcPAK1p67fDENERERSotcbaoCsdVdXqQw1LgUFkr715Aqeun7zNhkREZGU+MC4PSZyaPMEzk1GREQkLT4wbg8AWc1VxpohIiIiKfGBcXvkNlcZwxAREZGUyH3cHhnOVcYwREREJCVyH7dHhm2eGIaIiIikRs7j9siwzRMbUBMREUlRWhowdqxLx+3xCBm2eWIYIiIix7l4cD2ywt8fGDzY26VwjLHN05kzltsNGcdJklCbJ94mIyIix8hpmgjyPBm2eWIYIiIi+8msyzR5iczaPHE6DiIisg+niSAje2+T1vN2qqeu32wzRERE9nGky7Tc2rk4Q0ntpmoe69GjwPLlhtpAI2sjS8ukzRPDEBER2UeGXabdRkZTTdSbpWO9lfE2qQRvgdmDbYaIiMg+Muwy7RZKajdl7VhvJdGRpe3FMERERPaR+zQRriDDqSacVtexWiLBkaXtxTBERET2kWGXaZeT4VQTTrN1rNbI8DYpwxAREdlPZl2mXU5J7aacPQYZ3iZlA2oiInKMXKeJcAUltZty9BgkOLK0vbxeM5SXl4cxY8YgKioKKpUK69atM1svhMDcuXMRFRWFhg0bYvDgwcjPz/dOYYmIyMDYZXrSJMO/SghCgLLaTdk61ppkfpvU62GovLwcnTt3xpIlSyyuf+211/DGG29gyZIl2LVrF1q1aoVhw4bh6tWrHi4pEREpnpLaTdV1rLeS+W1SSY1ArVKpsHbtWowbNw6AoVYoKioKGRkZeO655wAAOp0OERERWLhwIR577DG79ssRqImIyKUsjb0TG2sIQjINBFZZG1PpkUeA9u3depuUI1ADKCgoQHFxMYYPH25aplarMWjQIGzfvt1qGNLpdNDpdKbHpaWlbi8rEREpiJLaTSngWCUdhoqLiwEAERERZssjIiJw8uRJq89bsGAB5s2b59ayERGRwslkqgmX8NKxlp7xTGWG19sM2UN1y71KIUStZTXNnj0bJSUlpp/CwkJ3F5GIiIjqSVQL/G/NEbyWmovBTfchLrGhR15X0jVDrVq1AmCoIYqs0cXv3LlztWqLalKr1VCr1W4vHxERkex5ecLZq2ev4vvF+dB+dQPao7fhtP52ALf/sdYzNUOSDkPx8fFo1aoVNm7ciK5duwIAbty4gS1btmDhwoVeLh0REZHMeWHCWVEtcODr48hefhra7aHYdiUJlehjWt8Q13Bny/3QDLqO/lOao8totxTDjNfDUFlZGY4dO2Z6XFBQgH379iEsLAytW7dGRkYGXn31VbRv3x7t27fHq6++iuDgYEyePNmLpSYiIpI54ySst3Yqd8MM9GXFZdj0Zj6063TIPtIOhfrbANxmWt++QQFSk05BM6ERBj2egKD/XQOKilCKMpe8vi1e71qfm5uLlJSUWsunTJmClStXQgiBefPm4d1338Xly5fRu3dvLF26FMnJyXa/BrvWExER1aDXA3Fx1uceM44mXVDg1C0zUS1w8Jvj0L53GtofQpB3ORmVCDStD8J1pLTYD83Aa9A8EYfbhrQxrLilpqoUQCjg9uu318OQJzAMERER1ZCbC1ioiKglJ8fuXmTl58qx+a18ZGdVQHu4LU7qY8zWtws4idTEE9DcFYzB05PRMOyWxtEWaqo8FYa8fpuMiIhIsrzcuNhtXDDhrKgWOLy+ANrlp6Dd2gRbLiXjBnqZ1qtRgcHh+5E6qByax9ug/bA4AG0s70yvN9QIeal+hmGIiIjIEi80LvYYJyecvXbhGnKW5CN7zTVoD8WjoKotgLam9fEBp5CaUABNWjBSpichuHkP+15n61brt+w8gGGIiIjoVh5sXOwVxklYz5yxXBvzR5sh0a8/jm4ogPbdU8jOa4QtF5OhQ0/TZoHQYVDYfqQOLIPm0VjcPiIeKr/WjpfH3poqN2EYIiIiqqmuWzZCGIJCRoZhigq53jIzTsI6YYLheGocq/jj8fuqR/DP4DM4XhUPIN60vo3/aaR2/A2paUFImZ6ERi2717889tZUuQkbUBMREdXkhsbFkmXhVmAhYvAUFmMtDDVfDXADA5vtR+qAUmgeiUXH1LZQ+dmYxd5Rxt5tt9RUsQE1ERGRN7igcbHUXb90HVvezof2i3BsKM5BK5xGJIpQhEhsxQDE+BfhsdvzkDpejTtnJKFxq27uLVAdNVWewDBERERUk5ONi6Xut9xT0L5dgOzcYOScT8J13Gzc/BtaY0Cz/Rh9RymWPlKAxDHtoPKLqWNvNjjTCy8tzdAW69ZG6x7A22REREQ1WbllY1LPAQk9peJKBfLeyYf28zJk72+NI5XxZutj/M8itf0xaMapMWRGIppENXHNC9e3F16NIFUaEoLQ0aM56KIrMAwREZFDjL3JAPNApPqjrYxEe5MV5BVC+/Zv0OY0xOZzSbiGRqZ1AahE/6b7oelbgtRHopE09jbXt/2x1gvPyffNU9dvhiEiIiJLLNVwxMYCixY5XMPhrgEbdaU6bF2Wj+zVpdDuj8WhG+3M1kf5FSG1/VFo/hyIoTMTERLjxmugG6b48NT1m22GiIiILElLM3SfdybQuHHAxpM/nIZ26W/Q5qixqTgJ5bjZuNkfVegXuh+aPleQ+nAU/pTWHio/D7VtsjVwohBAYaFhO4n1wmMYIiIissbf3/ELt4sHbLxRdgPb3s1H9qoSaPfH4IDuNgA3GzdH+hVD0+4oNGMCMHRmIpq26eJYeV1Fxr3wGIaIiIhqqs/tLRcN2Fj441lo3zoG7eZAfF+UhDJ0Na3zRxX6huQjtc9laB6MROe/3A6VXysHD9INZNwLj2GIiIjIqL63t5y8VVR5rRI/LM9HduYVaH+Nxn5dewBRpvURfuegaXsYmtH+GDYzEc3iOzt+bO5m5xQfGDDA82WzgWGIiIg8Q+ozwLvi9pYDt4rO/FwE7VvHkL2xAb4vSsRVdDGt9oMefZrkI7X3JWimRqDL3R3gF9DSsePxtLoGTjT2Jlu0SFqf+R8YhoiIPEXqYcAdjMf85ZfAJ58A58/fXCelGeBdNR+ZnbeAHpxShRWVkQBubt9SdR4j4w9DM8oPw59KQFi7To4dgxRYGzgxJsb+XnhewK71RESe4MbeRZJl6ZhrktKYPa6aj8zGgI3VUOE0YhCPAgio0KdxPjS9LkIzpSW6Te4IvwA/Z49AWlwU/Nm1nojIV7i4d5EsWDvmmqQ0A7yLekJVVQocGfE0Et5/BgIq+OHm8VfDEP7WtnwMH0/8EcNndkR4+z85XWRJc6YXnhf5SAQlIpIoW7dfAEMY0Otd81q5ucCqVYZ/XbFPZ8th7ZhvVbNBsTfVoydU0b7fseLBrZgYuwMtgsuQ9P4s3IU1OINos+0qwyKAOXPw1KK2mHTXDYS3DXVFyckFWDNEROROnhqITkq34WwdsyXeHnvGgZ5QVRVV+HHFAWg/uYTsPRHYez0BQIRp03DVRTRsHYGtIz9Bat/LaBp4DTh6FOrly4F5c2/u09dvk8oIwxARkTt5YiA6qd2Gc+ZYvD32jI2eUAJA7u2P4d34H/Hd6QRcFuaNm3s2yoem+3mk3t8cPdIT4B/Y7+bKrCxg7lzpfD5UCxtQExG5k6sa5lrjhvmg6s3eYwakNwO8hRq2YlUrPCmWYi1uBpYw1SWMaH0QmpECI2Z0QMukFpb3J8XPR0bYgJqIyNW80bXd3QPRSXE+KFvHbCSxsWfO5Z/Hhu9aYr3fJ7iKK2iEchQhElvFAFTDH92DD0DT7RxS08PRa2qiee2PNVL8fKgWhiEiUgZLbWrCwgzLXnjBfRdjdw9EJ8X5oOo65pq8PPaM/oYeP390ENn/vQDt7hb4uTwBAv1N65uqrmBEzAF8MGIHRsy4Ha06JQJIdOxFpPj5UC0MQ0Tk+6y1qbl0CZgzB3jzTWD5cvddlN05EJ1U54OydswtWgD33mvoSu+FQScvHL6IDYsPQasF1p/siIsi2Wx914YHoen6O1LvC0PvBxIREHRH/V7QE5+PEgfzdDG2GSIi32arzYaRSuX+hqzuuGjZGOTP621S3HXMdu6zuqoaP390ENr/nod2V3P8VJ4IUWNUmVCUYHjMAWiGVWHkzNsR2SXC4n7qVVZ3fj5S6kXoBp66fjMMEZFvc6Qxb2ysPBuyGmu+AMu34Xypt5IdF/+LRy/huzcPIfvbamw40QHnhXnj5s5Bh5HatQiayc3Q9+EkBAS5+SaJuz4fazWePvS5Mwy5EMMQkYJ98glw3332b+9sry5vsxQSYmMlPR+Uw6xc/IVKBQjgi+S5eOPEePxUlohq3Ay0ISjBsOgDSB1WhZEz2iOqWytPl9z1n49CeqkxDLkQwxCRQmVlAY8/bj45qC2ZmcCkSe4rkzv5ctsRGxf/mnN+VcMffwo6gtTOZ6GZ1BR3PJKEBsENPFteS1z5+bh7yAaJYNd6IqL6sGduLEu8PfhffchsPihHVOdugV8d7b78INAahfhq+FJ0fuUviOl5O4DbPVdAe7jy87G399mmTb4Zjl2MYYiIfI8jc2MZ1Xe8H3K5ywVXsPHNA9B+o0fQ8SN4x47njJraAugp40BrL3tD+yuv3Py/DzWsdjWGISLyPY7OjSWxwf+USlQL/PL5EWg/KEL2zmbYUZoEPQxd2wfBzkln5Vyz5wh7B7asidN/WMUwRES+x9EB7Lw8+J+SlZwqwcbFB6D9ugra4+1RVN0BQAfT+kT1MWiSTyP17sYQi6OhOnvWPSN5y429A1vWJIRh24wMwzhPDP4mDENE5HvsrR148UVgyBC2pfAgUS3wv6yj0L5/Ftk7mmJ7SRKq0Ne0PhjlGBKxH6l36qCZ1hZt+t0G4DbDynZvum8kbzmyNrBlXTj9h0XsTUZEvkfqAxEqTOnpUnz/5gFov7oB7dH2OFNtHlY7Bh6HJrkQqXeHYMDjSVCHqK3vTAlDCDiqZi+1AwfM2wlZI5Nek+xNRkTkLHfPB0aW/XFRFmfO4sRv1fhieySyd4Zh+5UE9EUFIlGE21CNK2iMlJYHoUm5Ds2TbRE/sB2Adva9Rlqa4RaPrw4h4IyavdRyc+0LQ0ppW2Un1gwRke9iLYLHXP/PJ8CsWWh49ZxpWSFikIlJmIxViMXNz0BERUP11pt1fwa+PGaSO/lYrSgHXXQhhiEiBeNF1S1EtcCBr49D+95pXM39GXPK/w8wm/ULqAbwRz2c6V/DAxvTRfj4fFtu50PTszAM/aGqqgpz587FJ598guLiYkRGRmLq1Kl48cUX4efnZ3sHYBgiInKFsuIybH4rH9lrddAeaYtT+hj4QY8TiEM0TsO+38h/sFZDoYD5tjzCR2pF2WboDwsXLsSyZcvw4YcfIikpCT///DMeeOABhIaG4qmnnvJ28YhIaRRU0ySqBQ5l/wbte4XQbmuCvEvJuIHepvVBuI4ZIR8ittSBMZ1MO7fQq6muwTLZLdwxbFvlEMmHoR07dmDs2LEYNWoUACAuLg6rVq3Czz//7OWSEZHiKOD2Tfm5cuQsyUd2VgW0h+Nxosq8cXPbgJNITTgBzV3BGDwtCcEbQ4HJ9XjBmmNC2Rosk93CHePD07O4muTDUP/+/bFs2TIcOXIEt99+O3755Rds27YNixYt8nbRiEhJrN2+kfmovqJa4MiGAmiXFyI7rzG2XErGDfQyrVejAoPC9yN1YDk0j7VG+2FxUPm1ubmD+vZKqvl8ewfLdHRQTSIbJB+GnnvuOZSUlKBjx47w9/eHXq/H/PnzMamO8RF0Oh10Op3pcWlpqSeKSkS+ysdu31y7cA05S/KhzbqG7IPxKKhqC6CtaX1cQCFSO/4GzfiGSJmehEYte1jfmTPTQgCWR4y2N1ixWzi5mOTD0OrVq/Hxxx8jMzMTSUlJ2LdvHzIyMhAVFYUpU6ZYfM6CBQswb948D5eUyMcpqK1MLT5w++boxhPIfucktHmNkHsxGTr0NK0LhA4Dw/YjdcBVaB5tjQ4j46Hyi7Vvx85MC2FtrCdbwUppU26Q5wiJi4mJEUuWLDFb9vLLL4sOHTpYfU5FRYUoKSkx/RQWFgoAoqSkxN3FJTJXVSVETo4QmZmGf6uqvF0i56xZI0RMjBCGS5ThJybGsFwJMjPNj93aT2amt0tqcu3iNZE97ycxo1OuaBdwolZR2/gXiscTt4ivXvxRXC26Wv8XtPQdiY0V4tlnLS+39t1Zs0YIlcrwU/M5xmVK+c6REEKIkpISj1y/JV8zdO3atVpd6P39/VFdXW31OWq1Gmp1HcO5E3mCrzS29dG2Mg6Rye2b45tPIvvtE9DmBSPnfDIqatT+NMANDGy2H5p+pdA8EoOE0e2g8otx3YvX1XtpwQL7axWtzbfFyXTJjSQ/ztDUqVPx/fff491330VSUhL27t2LRx99FA8++CAWLlxo1z44zhB5nK+MlWIczdbaLSKZjWbrNIm+DxVXKrBl6X5ovyhHdn5rHK2MN1sf638GmvbHkTpejTunJ6JJVBOPla3elHxblkw46OIfrl69ipdeeglr167FuXPnEBUVhUmTJuHvf/87AgMD7doHwxB5lEQvnE7JzQVSUmxvl5PjubYy3rhIZmUBjz4KXLxYe52HA+5vuaegfbsA2i0NsflcMq4j2LQuAJUY0HQ/NHeUIPXRGCSOaQeVn6qOvfkoBimfwUEX/9CkSRMsWrSIXelJPnygsa2J1Lo6e+PWo7VaPqOwMGD5cre9vq5Uh7y39yP7szJo82Nx+EZbAK1N66P9iqBpfxSpYwMxZEYiQmK6uqUcZqQcNnzl9jR5lOTDEJFkWbsgSC1A1IeU2sp4o+1SXV3qjRo2NLSVcaET205Du/Q3aHPU2PR7Mq6hu2mdP6rQP3Q/NH2vIPXhKCSPbw+VnwfbKlkKG2FhhmUvvODdUMT2beQkyd8mcwXeJiOXq+uvz7Aw6d1acpZUZsD21q1HD90m1JXqsO3dfGSvLoX2f7E4eKOd2fpIv2Kk3nYUmj8HYOiMRIS2DnX6terFVi1ZeHj9a8mcrXXypdvTZCLJ22SFhYWIjbVz7AkiX2Xrr8/Vq31nrJS6xpCxNlaMO3jr1qMba/lO7TgD7ZLjyN6sxqbiJJSjm2mdP6pwR0g+UvtehubBSHSacDtUfq0MK/V6Q0jz9C0qe2rJLl4E7roLWLPGuUBUn1tcvnR7mjzPkX74wcHB4sUXXxRlZWWu7+TvRp4ap4AUoKqq9pgpt46FEhsrxOef+9ZYKdbGkPHUcXhrnJ+cHPteNyfH5q50V3Vi0+t7xF975Igk9ZFau2jlVyweaJ8nPnt6u7h84orlnXhzvCd73wvjd8PRMbWM4wtZ2+e8eXXvU4ZjQZFtnrp+OxSGfvjhB9GrVy8RGRkpPvjgA3eVyeUYhshlHLk4ejtAuJo3B5B0YShxiDH8WrtIG8Ovlfei8Kez4r17N4sXwpaKqXhfDEKO8EOVAITwQ5Xo1+QX8crQHLEn86DQV+rrLou1sOCpgG1v2HDms7D1R4bxJzra+nF66ztCbiXJMGT04YcfipiYGNGlSxeRI4MvFsMQuYyjf336ygjU3lbPUFIvDoyIfKP8hsj5917xf71yRLL6iBiPNeIUzC/yF/1biK2p88XFY5ccP35X18Y4wpGaIUdrYBzZt7Xg56rvCM9ZSZF0GBJCiGvXromXXnpJBAcHi3HjxomjR4+6slwuxTBELsO/Pr3Hm9M01FHLd3rXWfGfKXkiLWqHCMEV0+rxWCP0UAm9K2pypPC9s7f2xpmyOFLrVFeoqe93ROnTzkiQ5MNQeXm52Lp1q8jIyBB+fn5CrVaLWbNmidLSUleWzyUYhshlvFlDQd699fhHjUHliv+KfTPeE7N7fy86BR2q9RVooTon7o/PFeWNW4hqZy7oltgbFjIy3Pse2GrX4+w54GitU11hy9nviLdvQ5JFnrp+O9S1ftmyZdi1axd27dqFgwcPwt/fH506dUKfPn3QpUsXfPLJJzhy5AjWrl2LHj16uL61t5PYtZ5cytibDDD8ujSS+lQbUh4ozxFeOI6ze4qx/q2j0G4MwMYziShBqGmdCtXo1egAUntdgGZKS3S/tyP8tuW5tku+vV38W7QwvC/ufD/cMRq3rSEcLMnMBCZNsr4/R74j7JYvWR67fjuSnGJiYsSECRPE66+/LrZt2yYqKipqbTN//nyRlJTkoqzmGqwZIpeTW+NoVv87pPJ6pdi69Bcxu2+O6NLwYK3Kguaq8+LeuG3ikye3ifOHLtTegat7NlVVCdG8ufdvldUsz7x5QoSFue4csKfWyV3HKYXbkGSRJGuG7PH7778jKioKer3elbutF9YMkVvIpaZF7pPGeuh9Lv71HNa/eQTrNwiUnS5BE1xFESKxFQMgoELPRgeg6XEBmvTm6JGeAP/AOsrgjsEan37aMKaTLXXVmLiaqz8bS+MM3codtTSrVgGTJ9vezpPvLQGQaM2QPaqrq0Vubq6rd1svrBkixbJ3XCSptnFyY41Wla5K/LDsV/FCvxzRreEBAQiLvb/KGrUQVxavcHDnbmhbppTaC2Otk7X3zR3td5Ty3sqQbGuGpIg1QzIkl1oXqZPirPP2ckON1rn881i/+DCy1/vhu9MJuCyamdaNRxbWYAIAAbN53p19PVe3LZPK1CieYqmWKDbWUDvmjjnolPTeyohsa4akiDVDMsP2La4j11F5XVSjVaWrEjve+594aUCO6BGcX2s3zVSXxD2tfxD/fShXVEVEub4GzdVty7w5vIA3eHLMH6W9tzLBmiEXYs2QjMi9fYsn2VN7JteaoXqU+/zBC9jw5mFka4HvTnXERRFutr5bw4PQdPsdqenh6DUlAQFBAe59nzzRrsZdNSZKw/dWciQ5USuRW9U1EaQQhkCUkQGMHcuqansntBwwQJ6TxjowQar+hh4/f3QQ2o8uQPtzc+wqT4RAP9MmoSjBiNh8aIZXY+TM29GqUwKABKdfz2H+/q4NmmlphnOAt5Fdj++tYjEMkXRw1mn7WKs9O3PGsLxm7Vlds84Dhsd33WV4T6X0Sz8y0q7N5j99AYvuvYwLItlseZeGh5DatRiae8PQ58FEBATd4ZLXs3s7d3N1wKKb+N4qEm+TkXSwe6ttzg4OZ6kmyd/fsD8jSzVL3nLjBhAcbF6+GgQAPfzRENdQhUCEogTDog9AM6wKI2e0R1S3Vo69nqca0LJjAJFDPHX99nPbnokcJbe/zj1Nrwfeesv+2rOa0tKAEycMbV4yMm7uryZjzVJWlitL7Zzt260GIQBQAQiAHksT3saWN3/B+fJgfH66Lx5cMcDxIATcrEEDbrZPM73YH48XLap/W5+4OEPbpMmTDf/GxUnj/SZSOIYhkg5j+5ZbL0ZGKpWhMaPU2rd4gvFC+vTT9m1vqW2Lv7/hvfviC8vPMdaIZGTUGUTcpbqqGrs/PohXhuZizsgf7XrOoy9FYOCMzmgQ3KD+BUhLM7w30dHmy2Ni6t9w33hr89YgK6UASqRgbDNE0lFX+xZX/XUuR9baCNXFWu2ZxNplXS64gu8WHYD222qsL7gdv1cbGjcPsncHrq4ldEcDWnYMIJI8hiGSFuNf55Z6Simxe2tdF1JLbPUOc2evKTuIaoF9qw8j+4NiaH8Mw46rSajGzcbNjXEVQyPzMepOgaqNkQg4X+z5XnCubkArsQBKRLUxDJH0sHvrTbYupDXZU3vmhXZZV06WYOPiA9B+UwXt8dtRXN0RQEfT+iT1UWj+dAapk0LR79EkBDbuY1iRtcQ3agm9HECJyDaGIZImdm81cOQC2ayZoRZp7Fjr23hg3CFRLfDrF0eQ/X4RtDubYXtpEvToa1rfCGUY0iofqXfqoJneDq37tgfQvvaOfKWWkB0DiCSPXeuJpMzekZFrstVF3tVzZgEoOVWC7986AO1XVdAeuw1nq80v7AmBx6H5UyFS7w5B/8eSoA5R279zuXdH57xXRE7z1PWbYYiUQa4XVFsXUkvsCTX1nHZAVAv8L+sotO+fhXZHU/xQkoQq3OzRFYxyDInYD02KDpppbRHXP8a+svsqNwRQIiVgGHIhhiGFs3fqCqmydiGtiz21DQ4GxNLTpdj01gFkf3kD2qPtceaW2p8Ogb9Bk1SI1ImNMeDxJAQ1DbKvrErBea+IHMYw5EIMQwrmrYlfPTE5pz3qMQGrqBbI//IYtP85g+ztodh2Jdms9qchruHOlvuhGXQdmifj0XZwa6deR1HkWkNJ5CUMQy7EMKRQzk5dUV/uqomqeSE9cAB45RXbz3Fw6pKy4jJsejMf2Wt10B5th0K9+QCE7RsUIDXpFDQTGmHQtGTW/hCRW3HWeqL68sb4Lo5Mouqomj3scnPtC0M2eiiJaoGD3xyH9r3TyP4hBFsvJ6MSvU3rg3AdKS32QzPwGjRPxOG2IfEA4p0rPxGRRDEMke/y9Pgu7hpp2NKtlXp0kS8rLsPmt/KhXaeD9nBbnNTfBuA20/p2ASeRmngCmruCMXh6MhqG9ay7LJ6+zSOFMhCRT2EYIt/l6fFd3FETVdctNzunLhHVAofXF0C7/BSytzZB3qVk3KhR+6NGBQaH70fqoHJoHm+D9sPiALRxrCyeagAshTIQkc9hmyHyXZ4e32XVKsNs5LbY247HnsbfgMUeShXz/onvC26Ddu11ZB9qixNVsWa7iA84hdSEAmjSgpEyPQnBzYPrXxZ3hxEplIGIPIoNqF2IYUjBPDm+i70DJNrTw8uRxt8AxJY8FH/3K37YJvD+wb7IudQZOtxs3BwIHQaF7UfqwDJoHo3F7SPiofJT2XVYXmuILrUyEJHHsQE1kSt4ckoHV051Yectt50PvYdP9iQg+2Bb/FZlHsTa+J9GasffoBkfhDtnJKFRy+4OHpBjZXHrRKNSKIMz2L6JSBYYhsj3eWriV39/u9vx2GRno+7FH4biUwwCADTADQxsth+pA0qheSQWHVPbQuXngpGfpTDRqBTK4Ci2byKSDYYh8hxv/pXsqYlfXVUTZWej7mq/ADzWIQ+p49W4c0YSGrfq5niZXVQWt040KoUyOMKdQywQkcuxzRB5htL+Sq5H8Du++STWv30cE9dOQnj1efih9ikqAFQ2j0SDs6egauDmv2mkMNGoFMpgL7ZvInIZthki36HEv5IdqImquFKBvHfykf1ZGbT5rXGkMh5AG2zCO/gCE1ANlXkgUqmgAhD47hLA3UEIcOz2n7tq/1x5C9LdXN2+ie2OiNzOz9sFsMeZM2dw3333ITw8HMHBwejSpQt2797t7WKRPWwNRAgYBiLU6z1aLKfo9YYeY6tWGf6tR5kL8grx9j1bMDriJ4Q302PE37pj8b5BOFIZjwBUYnDTveijCUPhX9+EKsZ8SgzExHg+QBpv/0XXUZasLEONSEqKYYiBlBTD46wsz5VBClzZvsnd7ykRAZDBbbLLly+ja9euSElJwRNPPIGWLVvi+PHjiIuLQ7t27ezaB2+T2ckdf4G6sru5N9XzNp+uVIety/KRvboU2v2xOHTD/Lsb5VeE1PZHoflzIIbOTERITI3vqZRqBqyVxZNjAFkqAyCd98hV33mOq0Tkueu3kLjnnntO9O/fv177KCkpEQBESUmJi0rlg9asESImRgjDr17DT0yMYXl9ZGaa79PaT2ama47DlaqqhMjJESIjw3KZVSrDj5X36MS2QvHOpC1iTMRO0QhXzZ7qj0oxMHSvWDAiR/zy+WFRra/27LG5UlVV7e/Ore9TbKxhO3dw13fXWcb3Q6Vy/v3w9ntKJBGeun5LPgwlJCSIjIwMMWHCBNGiRQvRpUsXsXz58jqfU1FRIUpKSkw/hYWFDEN1WbPG8i9uGxd7u+Tk2BeGcnJcdTSuYekCa+OipLuqE9+/tls80z1HJAQeq7VppF+ReLB9nvh81nZx+cQVbx+h63jzM7bnu2sMtZmZhn89ESCM5bq1bPaeU3I9b4hcjGHoD2q1WqjVajF79myxZ88esWzZMhEUFCQ+/PBDq8+ZM2eOgKHDjdkPw5AF7v4L1BV/JXuatQtsHT8vNFsqGqO0Vu1P/5B94tXhOWLvp4fkXftTF2/V/tnz3Q0PFyI62ju1RpYCdWysfa8t5xpVIhfyVBiSfJuhwMBA9OjRA9u3bzctmzlzJnbt2oUdO3ZYfI5Op4NOpzM9Li0tRWxsrG+3GXK2XYkn2vR4ckqM+rLVLdqKScjEp5iECL9z0LQ9DM1ofwybmYhm8U3dUkxJ8Va7MHtf91ae/N5J+bwkkgF2rf9DZGQkEhMTzZYlJCRgzZo1Vp+jVquhVqvdXTTpqE/jXk+M7FvfgQg92YDYVrdoK1K7/Y5nZx1El7s7wC+gpRsKJmGunIbEEc5+J4UwlCkjwzAyuTsbWjs72Ke33lMihZJ81/p+/frh8OHDZsuOHDmCNm3aeKlEEmOsdbn1Am4cw8dWF1xPjeyblgacOGH4SzYz0/BvQYHtIOTprsUOXmCFSgXExiL9pxnodm8C/AIkf0q5nnEMIOBmrYuRO8cAqs93UoibY/1IkbfeUyKlcutNOBf46aefREBAgJg/f744evSo+OSTT0RwcLD4+OOP7d6Hz/Ymc0V7Hym36XFnw+5bnNldJP4zJU/8vfkS+9sKuaEcslafNjLOsPXddXWbG281xPbke0okMWxAXcPXX38tkpOThVqtFh07drTZm+xWPhuGXNXjpL49X9zBzQ27K69Xiry39onn++SIzkGHTLv1Q5U4hRiht+d95UWpNk8HBmvfXXt/7O2N5c3u+94IYUQSwTDkQj4bhlzZ40Rqf4G6oWvx2b3F4oMH8sSE6O0iFFfMsxX0onej/4m5g3PEmZEPiOq6XnPkSCH+/W8hdDp3HT05wlpQCQ93TY2nB2soicgce5O5kM+OQO3qHidSGul41SpDGyFbMjOBSZMsrqqqqMLODw5A+8klaPdGYO/1BLP14aqLGNH6EFJTgeEzOqBFQnPro/5a4ssTzbqCJ79Pll7ryy8d78V4637uuANo146TrhJ5iaeu3wxDcianmbwd5WTQ+33/eaxffBjZG/zw3elEXBFNzTbv2Sgfmu7nkXp/c/RIT4B/YI33xdFu9VIdGkAKgbae05e4tRyxsZZ7MVratnlz4MIF26/DLu5EbsEw5EI+G4YA947h480Lq51BT3/4GH7872FoP74I7Z6W2H3NfBiGZqrLGBF7EKmaaoyY0QEtk1pYf01nxq2RUuCUUgCR0pxa9nyPHakRtKSOGkoich7nJnMhu+85yrWhojva+0hhvicrjWOrVSpRDYh/NZ8vwlQXazXl6B6cL17snyO2v/urqNI58Bna2warnm2X6sXad1Qq7VrkOKeWrTJL6fMnUhg2oHYhu95MKVz868OVQU4qF9Y/ylIdbf65nESsGI81pkVNVZfF3bE/iJUPbxVFv/zu/GvZ22jb2Ubq9WXtO/rZZ9IJIHKcU6s+n7sUwx2RD/FUGJL8CNQeYa2K3DhwoZTahFjj7Ei3t9LrDbdaLN0uEMJjI/eeP3gBG948DK02AhvP7EYiDiASRShCJLZiADo3PIK/dc1F6n1h6P1AIgKC7qj/i9oa9bcu9R2U0pa6vqMTJ9b9XCFuDjDo7nYtnhjR3NWcLQsHPyTyGQxDErn4S4at6ShuvbC6qF1RdVU1fv7oILT/PY/sXc2xqzwRAv1M6/dBjZYxakwZVoVVMy8gsksCgATrO3SGcdTfCRMMn7s9gcgT0yLY+o7ayxMBxFMjmruSvWVp0QI4f/7mY3unkyEiyWMYcvTi7+sc+cu+ng12Lx69hA2LDkKrFdhwogPOiySz9Z2DDiO1axE0k5uh78NJCAjq68iROMfaPGqWeKpmwMn50mrxRACR45xa9pb52DFg+3bXdyiQSg9AIgVjGJJKtb5UfiHae8E8ehSYO9ehW4vVVdXYk3kI2SvPQbsrHD+VJaK6Ru1PCEowLPoANEOrMHL6bYju0QFAh/odjzPS0gw1gTU/jwsXgKefdm6i2fqq73fPkwGkrto1qd5WsrfMgYGu/4NIKj0AiZTOrS2SJKLOBlhSaPAppcbb9sxVFhNjd4Pdi8cuiVUzfhD3t90qWqrO1dr0T0GHxXO9c0Tuor3iRvkNy+WRSg8/b5XFkQa+UplSRWojmtvD02WWUkcFIolibzIXqvPN9PZEpc78QnT3RdnWXGXz5tl1YZ7W8D/CD1Vmi5ugRIyP3CHeuz9PFP501nY5pBISvcne7+jnn0srgEgpyNrLU2WW4xAERF7AMORCNt9Mb01U6swvRE8FBEuvExZmCEIff2xXGPoXMgQgRLL6iHi2Z47Y/K89QnfVzvm8+FezOXu/o3IMIEokhRppIhlgGHIhp8cZcvdf1Y7+QrQVED7/3LUXwqoqQ/gJCzN7veoWLewq9zV1U3Fy60nnXpd/Ndcmx1tPZJkrJ1km8mEcZ8jTLDWadXcjZkcab9vTvfqeewzbGdVsiOlMA+0vv7TYSFqcN8zVpLJR7Ia6K2hd9RuA1ja2vAV7+Fnmje8ouYcchyAg8mEMQzW5auBCeznyCzE313b36ppBCDD07LrrLuDuu4GNG4FLl26us9RjpUZgEs1boPKRJ9FAiFqhxw8CFiKZZc70hJJKDz8p8vR3lNxDjkMQEPkwP28XQNGMvxBVVupYVCrDDNsXLtgeZdgS4y/Z1avNgxBwswt8VpbhcVYWqmNbGyYqnTwZquHDEHjpd6u1P7ZqhUyc+cuWfzWTrzN25wdqn/9SHYKAyIcxDHmTPb8Q77nHEIRuDTP1JQy1OzcenYZ1XedC3DUBKDrruv0bg5wzf9necQfQvLl79k0kFcYBPqOjzZfHxMhjCiAiH8Iw5G11/UJcvRpYtcpyNboLqIRA4MVi3LHvbQgI578MrvzLNisLaNfOUBtW12vxr2byBWlpwIkTQE4OkJlp+LeggEGIyMNUQrjpSishpaWlCA0NRUlJCUJCQhzfgSdGh7b0Glu3Gm5bSZGxTcO//gXMmmXenik21rmRma1NRlqTs/smIiLZqff1205sQG2Lp4bLt9QwVqoNhI21M2+8Ybid9c9/GiawbNHCUMPlTFisq7ecUYsWhvmhAgOdLzsREdEtGIbqYq2moo75t1zKzgbCAg40aHaFmBhDWyZLc3UtXuxcrZk9k5GeP2+YKJO9qYiIyIUYhqyxNa6PSgVkZBjGfXFX2xUb3W+NS6wFIZeEJJXKUNuzciVw7pwhoJ0/b+iu78qQyO70RETkJWxAbY0jA/+5S43eZuKWRsrVMASdusKOy2qLFi8GhgwBJk0yBLRZs+oe/DEjo/aYR7awOz0REXkJw5A1EqipuHbhGr79NRbLY+ahWLQyW3cJ4fbtJCzM/HF4uKH2xh7h4bVredwVEu0dc4nd6YmIyMV4m8waL9VUHN14AtnvnIQ2rxFyLyZDh54AeuIJ/A13YhNSGu1C+y6N0Du1OfBCuu0dfvaZoYbp1l5qX3xh+7mrVxtqhGpyV0g01oJNmGAIPjVrntidnoiI3Ihd663R64G4ONvD5RcU1OsCff3SdeQu2Y/sL65BezAOx6vamK1v438amg6/ITUtCCnTEtG4VeP6l68+z83Nta+7f06Ocw2dLfXeY3d6IiJF8lTXeoahuhh7kwGWayqc7E12bNNJaN85gewtwci9kIwKNDSta4AbGNhsPzT9SqF5JAYJo9tB5Wfl1lF9yufscz0REj0xrhMREUkew5AL1evNdEFNxfVL17Hl7XxovyiH9kBrHK2MN1sf638GmvbHkTpejTunJ6JJVBPPlM/Z57opJBIREdXEMORC3hiB+rfcU9C+XYDs3GDknE/CdQSb1gWgEgOa7ofmjhJoHo5G0tjbrNf+uKl89X4ub2cREZGbMQy5kCfezIorFch7Jx/az8uQvb81jtxS+xPtVwRN+6NIHRuIITMSERJzSznkeGtIjmUmIiLZ4HQcMnBi22lol/6G7M1B2HwuCdfQ3bTOH1XoH7ofmr5XkPpwFJLHt4fKz0rPM09N+eFqlqYQISIikhmGIQfoSnXYuiwf2s9Kkf2/WBy60Q5AjGl9pF8xUm87Cs2fAzB0RiJCW3exvVNvT/lBRESkcLxNZsPJHwy1P9ocNTYVJ6EcjU3r/FGFO0LyoelzGakPRaLThNsda/tj7JllbRBDF3XfJyIikiPeJvOSG2U3sO3dfGSvKoF2fwwO6G5DzdqfVn6/Q9PuCDSjAzDsqUQ0bdPZ+RdzZDRn3o4iIiJyC4YhAIU/noX2rWPQbg7E90VJKENX0zo/6NG3ST40vS8h9cFW6PyX2+EXEOGaF5bAlB9ERERKp8gwVHmtEj8sz0d25hVof43Gfl17AFGm9RF+5zAy/gg0o/wwbGYCwtp1ck9BODkpERGR1ykqDH342Hbk5jXFxrOJuIoupuV+0KN34wNI7X0RmqkR6HpPB/gFtHR/gYyTk9oazZmTkxIREbmNosLQzE/vAGBogNVCdR4j4w8jdZQfhs3oiPD2f/J8gYyTk951l+X1QnByUiIiIjfz83YBHLVgwQKoVCpkZGQ4/NzuwQcwLyUXP608gOIb4fjv8f645807EN4+zPUFJSIiIlmQVc3Qrl27sHz5cnTq5Fwbns1FiW7tmucwvd4w2KI1KhWQkQGMHcvaISIiIjeRTc1QWVkZ7r33Xrz33nto1qyZt4vjGo50rSciIiK3kE0YmjZtGkaNGoWhQ4fa3Fan06G0tNTsR5LYtZ6IiMjrZHGb7NNPP8WePXuwa9cuu7ZfsGAB5s2b5+ZSuQC71hMREXmd5GuGCgsL8dRTT+Hjjz9GUFCQXc+ZPXs2SkpKTD+FhYVuLqWTjF3rVVam8FCpgNhYw3Z6PZCbC6xaZfhXr/dkSYmIiHyW5OcmW7duHcaPHw//Gg2I9Xo9VCoV/Pz8oNPpzNZZ4qm5TZxinKgVMB9ryBiQvvjC8K8cZ7UnIiKqB09dvyVfMzRkyBD873//w759+0w/PXr0wL333ot9+/bZDEKSl5ZmCDzR0ebLY2JuBqEJE2o3tDbOap+V5ZlyEhER+SjJtxlq0qQJkpOTzZY1atQI4eHhtZbLVlqaofv81q2GxtKRkTdHnY6Lszw6tRDsek9EROQCkg9DiuHvX3tm+txczmpPRETkZrIMQ7m5ud4ugmfY26V+0ybzGiXWEhEREdlNlmFIMeztUv/KKzf/z4bVREREDpF8A2pFs9X13hI2rCYiInIIw5CUGWe1B+wPRMbG1hkZHIuIiIjIDgxDUmet631dOKcZERGR3RiG5CAtDThxAsjJATIzgRdftO95nNOMiIjIJjagtkWvrz3+jzd6a9Xsep+ba95o2hrOaUZERGQTa4bqkpVlGPQwJQWYPNnwb1yc9xsnOzKnGREREdWJYcga45xhUpwGo66G1cbHixZxvCEiIiI7MAxZotcbJka1Ng0G4P3eWrbmNFPCOEN6veGW4apVhn/Ze46IiJzANkOWbN0qj2kwrM1ppoQaoawsQ2Ct+TlxwEkiInICw5Al9vbCkkJvLUtzmvk64y3MW2vujLcwlVIzRkRELsHbZJbY2wuLvbU8Tw63MImISFYYhixhby3pcuQWJhERkR0Yhixhby3pktMtTCIikgWGIWvYW0uaeAuTiIhcTCWEpcYXvqW0tBShoaEoKSlBSEiIY0+WygjUZKDXGwa+PHPGcrshlcoQWAsK+DkREclcva7fDmBvMluU2FtLyoy3MCdMMASfmoGItzCJiMgJvE1G8sNbmERE5EKsGSJ5UvKAk0RE5FIMQyRfvIVJREQuwNtkREREpGgMQ0RERKRoDENERESkaAxDREREpGgMQ0RERKRoDENERESkaAxDREREpGgMQ0RERKRoDENERESkaAxDREREpGgMQ0RERKRoDENERESkaAxDREREpGgMQ0RERKRoDENERESkaAxDREREpGgMQ0RERKRoDENERESkaJIPQwsWLEDPnj3RpEkTtGzZEuPGjcPhw4e9XSwiIiLyEZIPQ1u2bMG0adOwc+dObNy4EVVVVRg+fDjKy8u9XTQiIiLyASohhPB2IRxx/vx5tGzZElu2bMHAgQPtek5paSlCQ0NRUlKCkJAQN5eQiIiIXMFT1+8At+3ZTUpKSgAAYWFhVrfR6XTQ6XSmx6WlpW4vFxEREcmT5G+T1SSEwKxZs9C/f38kJydb3W7BggUIDQ01/cTGxnqwlERERCQnsrpNNm3aNHz77bfYtm0bYmJirG5nqWYoNjaWt8mIiIhkhLfJbjFjxgx89dVXyMvLqzMIAYBarYZarfZQyYiIiEjOJB+GhBCYMWMG1q5di9zcXMTHx3u7SERERORDJB+Gpk2bhszMTHz55Zdo0qQJiouLAQChoaFo2LChl0tHREREcif5NkMqlcri8hUrVmDq1Kl27YNd64mIiOSHbYb+IPGsRkRERDInq671RERERK7GMERERESKxjBEREREisYwRERERIrGMERERESKxjBEREREisYwRERERIrGMERERESKxjBEREREisYwRERERIrGMERERESKxjBEREREisYwRERERIrGMERERESKxjBEREREisYwRERERIrGMERERESKxjBEREREisYwRERERIrGMERERESKxjBEREREisYwRERERIrGMERERESKxjBEREREisYwRERERIrGMERERESKxjBEREREisYwRERERIrGMERERESKxjBEREREisYwRERERIrGMERERESKxjBEREREisYwRERERIrGMERERESKxjBEREREisYwRERERIrGMERERESKJpsw9PbbbyM+Ph5BQUHo3r07tm7d6u0iERERkQ+QRRhavXo1MjIy8MILL2Dv3r0YMGAANBoNTp065e2iERERkcyphBDC24WwpXfv3ujWrRveeecd07KEhASMGzcOCxYssPn80tJShIaGoqSkBCEhIe4sKhEREbmIp67fkq8ZunHjBnbv3o3hw4ebLR8+fDi2b9/upVIRERGRrwjwdgFsuXDhAvR6PSIiIsyWR0REoLi42OJzdDoddDqd6XFJSQkAQ8IkIiIieTBet919E0vyYchIpVKZPRZC1FpmtGDBAsybN6/W8tjYWLeUjYiIiNzn4sWLCA0Nddv+JR+GmjdvDn9//1q1QOfOnatVW2Q0e/ZszJo1y/T4ypUraNOmDU6dOuXWN1NqSktLERsbi8LCQkW1leJx87iVgMfN41aCkpIStG7dGmFhYW59HcmHocDAQHTv3h0bN27E+PHjTcs3btyIsWPHWnyOWq2GWq2utTw0NFRRXyKjkJAQHreC8LiVhcetLEo9bj8/9zZxlnwYAoBZs2YhPT0dPXr0QN++fbF8+XKcOnUKjz/+uLeLRkRERDInizB099134+LFi/jHP/6BoqIiJCcnIzs7G23atPF20YiIiEjmZBGGAODJJ5/Ek08+6dRz1Wo15syZY/HWmS/jcfO4lYDHzeNWAh63e49bFoMuEhEREbmL5AddJCIiInInhiEiIiJSNIYhIiIiUjSGISIiIlI0WYaht99+G/Hx8QgKCkL37t2xdevWOrffsmULunfvjqCgILRt2xbLli2rtc2aNWuQmJgItVqNxMRErF271l3Fd5ojx52VlYVhw4ahRYsWCAkJQd++fbFhwwazbVauXAmVSlXrp6Kiwt2H4hBHjjs3N9fiMR06dMhsO1/7vKdOnWrxuJOSkkzbyOHzzsvLw5gxYxAVFQWVSoV169bZfI4vnN+OHrevnN+OHrevnN+OHrevnN8LFixAz5490aRJE7Rs2RLjxo3D4cOHbT7PE+e47MLQ6tWrkZGRgRdeeAF79+7FgAEDoNFocOrUKYvbFxQUIDU1FQMGDMDevXvxt7/9DTNnzsSaNWtM2+zYsQN333030tPT8csvvyA9PR0TJ07Ejz/+6KnDssnR487Ly8OwYcOQnZ2N3bt3IyUlBWPGjMHevXvNtgsJCUFRUZHZT1BQkCcOyS6OHrfR4cOHzY6pffv2pnW++HkvXrzY7HgLCwsRFhaGv/zlL2bbSf3zLi8vR+fOnbFkyRK7tveV89vR4/aV89vR4zaS+/nt6HH7yvm9ZcsWTJs2DTt37sTGjRtRVVWF4cOHo7y83OpzPHaOC5np1auXePzxx82WdezYUTz//PMWt/+///s/0bFjR7Nljz32mOjTp4/p8cSJE8XIkSPNthkxYoS45557XFTq+nP0uC1JTEwU8+bNMz1esWKFCA0NdVUR3cLR487JyREAxOXLl63uUwmf99q1a4VKpRInTpwwLZPD510TALF27do6t/GV87sme47bEjme3zXZc9y+cn7X5Mzn7QvntxBCnDt3TgAQW7ZssbqNp85xWdUM3bhxA7t378bw4cPNlg8fPhzbt2+3+JwdO3bU2n7EiBH4+eefUVlZWec21vbpac4c962qq6tx9erVWpPdlZWVoU2bNoiJicHo0aNr/WXpTfU57q5duyIyMhJDhgxBTk6O2TolfN7vv/8+hg4dWmuUdil/3s7whfPbFeR4fteHnM9vV/CV87ukpAQA6pyE1VPnuKzC0IULF6DX62vNVh8REVFrVnuj4uJii9tXVVXhwoULdW5jbZ+e5sxx3+pf//oXysvLMXHiRNOyjh07YuXKlfjqq6+watUqBAUFoV+/fjh69KhLy+8sZ447MjISy5cvx5o1a5CVlYUOHTpgyJAhyMvLM23j6593UVERtFotHn74YbPlUv+8neEL57cryPH8doYvnN/15SvntxACs2bNQv/+/ZGcnGx1O0+d47KZjqMmlUpl9lgIUWuZre1vXe7oPr3B2TKuWrUKc+fOxZdffomWLVualvfp0wd9+vQxPe7Xrx+6deuGt956C2+++abrCl5Pjhx3hw4d0KFDB9Pjvn37orCwEK+//joGDhzo1D69xdkyrly5Ek2bNsW4cePMlsvl83aUr5zfzpL7+e0IXzq/neUr5/f06dPx66+/Ytu2bTa39cQ5LquaoebNm8Pf379W2jt37lytVGjUqlUri9sHBAQgPDy8zm2s7dPTnDluo9WrV+Ohhx7CZ599hqFDh9a5rZ+fH3r27CmZvyTqc9w19enTx+yYfPnzFkLggw8+QHp6OgIDA+vcVmqftzN84fyuDzmf364it/O7Pnzl/J4xYwa++uor5OTkICYmps5tPXWOyyoMBQYGonv37ti4caPZ8o0bN+KOO+6w+Jy+ffvW2v67775Djx490KBBgzq3sbZPT3PmuAHDX4xTp05FZmYmRo0aZfN1hBDYt28fIiMj611mV3D2uG+1d+9es2Py1c8bMPTWOHbsGB566CGbryO1z9sZvnB+O0vu57eryO38rg+5n99CCEyfPh1ZWVnYvHkz4uPjbT7HY+e43U2tJeLTTz8VDRo0EO+//744cOCAyMjIEI0aNTK1qn/++edFenq6afvffvtNBAcHi6efflocOHBAvP/++6JBgwbiiy++MG3zww8/CH9/f/HPf/5THDx4UPzzn/8UAQEBYufOnR4/PmscPe7MzEwREBAgli5dKoqKikw/V65cMW0zd+5csX79enH8+HGxd+9e8cADD4iAgADx448/evz4rHH0uP/973+LtWvXiiNHjoj9+/eL559/XgAQa9asMW3ji5+30X333Sd69+5tcZ9y+LyvXr0q9u7dK/bu3SsAiDfeeEPs3btXnDx5Ugjhu+e3o8ftK+e3o8ftK+e3o8dtJPfz+4knnhChoaEiNzfX7Ht77do10zbeOsdlF4aEEGLp0qWiTZs2IjAwUHTr1s2sW96UKVPEoEGDzLbPzc0VXbt2FYGBgSIuLk688847tfb5+eefiw4dOogGDRqIjh07mp1cUuHIcQ8aNEgAqPUzZcoU0zYZGRmidevWIjAwULRo0UIMHz5cbN++3YNHZB9HjnvhwoWiXbt2IigoSDRr1kz0799ffPvtt7X26WuftxBCXLlyRTRs2FAsX77c4v7k8Hkbu05b+9766vnt6HH7yvnt6HH7yvntzPfcF85vS8cMQKxYscK0jbfOcdUfBSQiIiJSJFm1GSIiIiJyNYYhIiIiUjSGISIiIlI0hiEiIiJSNIYhIiIiUjSGISIiIlI0hiEiIiJSNIYhIiIiUjSGISIiIlI0hiEiIiJSNIYhIpKlVatWISgoCGfOnDEte/jhh9GpUyeUlJR4sWREJDecm4yIZEkIgS5dumDAgAFYsmQJ5s2bh//85z/YuXMnoqOjvV08IpKRAG8XgIjIGSqVCvPnz8eECRMQFRWFxYsXY+vWrQxCROQw1gwRkax169YN+fn5+O677zBo0CBvF4eIZIhthohItjZs2IBDhw5Br9cjIiLC28UhIplizRARydKePXswePBgLF26FJ9++imCg4Px+eefe7tYRCRDbDNERLJz4sQJjBo1Cs8//zzS09ORmJiInj17Yvfu3ejevbu3i0dEMsOaISKSlUuXLqFfv34YOHAg3n33XdPysWPHQqfTYf369V4sHRHJEcMQERERKRobUBMREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaL9P1H6jezKdyEWAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_5_1.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "# Importing various packages\n",
    "from math import exp, sqrt\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.inv(X.T @ X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = 2.0/n*X.T @ ((X @ theta)-y)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "# Can you figure out a better way of setting up the contributions to each batch?\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (2.0/M)* xi.T @ ((xi @ theta)-yi)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        theta = theta - eta*gradients\n",
    "print(\"theta from own sdg\")\n",
    "print(theta)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a815759a",
   "metadata": {
    "editable": true
   },
   "source": [
    "In the above code, we have use replacement in setting up the\n",
    "mini-batches. The discussion\n",
    "[here](https://sebastianraschka.com/faq/docs/sgd-methods.html) may be\n",
    "useful."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4dc76e73",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Momentum based GD\n",
    "\n",
    "The stochastic gradient descent (SGD) is almost always used with a\n",
    "*momentum* or inertia term that serves as a memory of the direction we\n",
    "are moving in parameter space.  This is typically implemented as\n",
    "follows"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2650be7b",
   "metadata": {
    "editable": true
   },
   "source": [
    "$$\n",
    "\\mathbf{v}_{t}=\\gamma \\mathbf{v}_{t-1}+\\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t) \\nonumber\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94cf4be5",
   "metadata": {
    "editable": true
   },
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"_auto1\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation} \n",
    "\\boldsymbol{\\theta}_{t+1}= \\boldsymbol{\\theta}_t -\\mathbf{v}_{t},\n",
    "\\label{_auto1} \\tag{1}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c744311",
   "metadata": {
    "editable": true
   },
   "source": [
    "where we have introduced a momentum parameter $\\gamma$, with\n",
    "$0\\le\\gamma\\le 1$, and for brevity we dropped the explicit notation to\n",
    "indicate the gradient is to be taken over a different mini-batch at\n",
    "each step. We call this algorithm gradient descent with momentum\n",
    "(GDM). From these equations, it is clear that $\\mathbf{v}_t$ is a\n",
    "running average of recently encountered gradients and\n",
    "$(1-\\gamma)^{-1}$ sets the characteristic time scale for the memory\n",
    "used in the averaging procedure. Consistent with this, when\n",
    "$\\gamma=0$, this just reduces down to ordinary SGD as discussed\n",
    "earlier. An equivalent way of writing the updates is"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0c1409c",
   "metadata": {
    "editable": true
   },
   "source": [
    "$$\n",
    "\\Delta \\boldsymbol{\\theta}_{t+1} = \\gamma \\Delta \\boldsymbol{\\theta}_t -\\ \\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t),\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a20fde4",
   "metadata": {
    "editable": true
   },
   "source": [
    "where we have defined $\\Delta \\boldsymbol{\\theta}_{t}= \\boldsymbol{\\theta}_t-\\boldsymbol{\\theta}_{t-1}$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80345cf7",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Algorithms and codes for Adagrad, RMSprop and Adam\n",
    "\n",
    "The algorithms we have implemented are well described in the text by [Goodfellow, Bengio and Courville, chapter 8](https://www.deeplearningbook.org/contents/optimization.html).\n",
    "\n",
    "The codes which implement these algorithms are discussed after our presentation of automatic differentiation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e073302",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Practical tips\n",
    "\n",
    "* **Randomize the data when making mini-batches**. It is always important to randomly shuffle the data when forming mini-batches. Otherwise, the gradient descent method can fit spurious correlations resulting from the order in which data is presented.\n",
    "\n",
    "* **Transform your inputs**. Learning becomes difficult when our landscape has a mixture of steep and flat directions. One simple trick for minimizing these situations is to standardize the data by subtracting the mean and normalizing the variance of input variables. Whenever possible, also decorrelate the inputs. To understand why this is helpful, consider the case of linear regression. It is easy to show that for the squared error cost function, the Hessian of the cost function is just the correlation matrix between the inputs. Thus, by standardizing the inputs, we are ensuring that the landscape looks homogeneous in all directions in parameter space. Since most deep networks can be viewed as linear transformations followed by a non-linearity at each layer, we expect this intuition to hold beyond the linear case.\n",
    "\n",
    "* **Monitor the out-of-sample performance.** Always monitor the performance of your model on a validation set (a small portion of the training data that is held out of the training process to serve as a proxy for the test set. If the validation error starts increasing, then the model is beginning to overfit. Terminate the learning process. This *early stopping* significantly improves performance in many settings.\n",
    "\n",
    "* **Adaptive optimization methods don't always have good generalization.** Recent studies have shown that adaptive methods such as ADAM, RMSPorp, and AdaGrad tend to have poor generalization compared to SGD or SGD with momentum, particularly in the high-dimensional limit (i.e. the number of parameters exceeds the number of data points). Although it is not clear at this stage why these methods perform so well in training deep neural networks, simpler procedures like properly-tuned SGD may work as well or better in these applications.\n",
    "\n",
    "Geron's text, see chapter 11, has several interesting discussions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23fed8cc",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Using Automatic differentation with OLS\n",
    "\n",
    "We conclude the part on optmization by showing how we can make codes\n",
    "for linear regression and logistic regression using **autograd**. The\n",
    "first example shows results with ordinary leats squares."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2b199ebe",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.89044267]\n",
      " [3.11620886]]\n",
      "Eigenvalues of Hessian Matrix:[0.31812561 4.63042739]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own gd\n",
      "[[3.89044267]\n",
      " [3.11620886]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHFCAYAAADxOP3DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABWl0lEQVR4nO3deXhTVfoH8G+6pWwttNDSDVq0UKDs+1KEkUVkSa2IwojoqKOCaC2iMMwMMKPyEzdUdBhmGHB0igqUBsENoQUUUBSQfbVAWWpZpIFCt+T8/ggJTZs0S5Pcm9zv53n6YO69uTn3tjFvznnPeVVCCAEiIiIihQqQugFEREREUmIwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRCRDy5cvh0qlMv8EBQUhJiYGDzzwAI4dOyZZu+bOnQuVSiXZ6/uCkydPQqVS4fXXX5e6KUTkoCCpG0BEti1btgwpKSkoKyvDd999h5dffhl5eXk4fPgwmjVrJnXziIj8AoMhIhlLTU1Fz549AQCDBw+GXq/HnDlzkJubi0ceeUTi1pGUKisrzb2GRFQ/HCYj8iGmwOjXX381bysrK8P06dPRtWtXhIeHIyIiAv369YNWq631fJVKhaeffhoffvgh2rdvj4YNG6JLly5Yt25drWPXr1+Prl27Qq1WIykpyeawT1lZGWbNmoWkpCSEhIQgLi4OU6dOxZUrVyyOS0xMxOjRo7Fu3Tp069YNDRo0QPv27c2vvXz5crRv3x6NGjVC79698eOPP9q9H6bhxLy8PDz11FNo3rw5IiMjkZGRgXPnztW69rlz59Y6R2JiIh5++OFa59y0aRMef/xxREZGIiwsDA899BBKS0tRVFSE8ePHo2nTpoiJicHzzz+PysrKWuc1GAx4+eWX0apVK4SGhqJnz57YuHFjreOOHTuGiRMnIioqCmq1Gu3bt8d7771ncUx+fj5UKhU+/PBDTJ8+HXFxcVCr1Th+/DiuX7+O559/HklJSQgNDUVERAR69uyJFStW2L1/RGTErxREPqSgoAAA0LZtW/O28vJyXL58Gc8//zzi4uJQUVGBb775BhkZGVi2bBkeeughi3OsX78eO3fuxN/+9jc0btwYCxYswD333IMjR46gTZs2AICNGzdCo9GgX79++Pjjj6HX67FgwQKLIAwAhBBIT0/Hxo0bMWvWLKSlpWHv3r2YM2cOtm/fju3bt0OtVpuP//nnnzFr1izMnj0b4eHhmDdvHjIyMjBr1ixs3LgRr7zyClQqFV588UWMHj0aBQUFaNCggd378thjj2HUqFHIzs5GYWEhZsyYgQcffBCbNm1y+V4/9thjyMjIwMcff4zdu3fjT3/6E6qqqnDkyBFkZGTgj3/8I7755hu8+uqriI2NRVZWlsXzFy1ahNatW2PhwoUwGAxYsGABRo4cic2bN6Nfv34AgIMHD6J///5o1aoV3njjDbRs2RJfffUVnnnmGVy8eBFz5syxOOesWbPQr18/LF68GAEBAYiKikJWVhY+/PBDvPTSS+jWrRtKS0uxf/9+XLp0yeVrJ1IcQUSys2zZMgFA7NixQ1RWVoqrV6+KL7/8UrRs2VIMGjRIVFZW2nxuVVWVqKysFI8++qjo1q2bxT4AIjo6Wuh0OvO2oqIiERAQIObPn2/e1qdPHxEbGytu3Lhh3qbT6URERISo/r+NL7/8UgAQCxYssHidTz75RAAQS5YsMW9r3bq1aNCggThz5ox52549ewQAERMTI0pLS83bc3NzBQCxdu1ah+7TlClTLLYvWLBAABDnz5+3uPY5c+bUOkfr1q3F5MmTa51z2rRpFselp6cLAOLNN9+02N61a1fRvXt38+OCggIBwOb9Gzp0qHnbiBEjRHx8vCgpKbE459NPPy1CQ0PF5cuXhRBC5OXlCQBi0KBBtdqfmpoq0tPTa20nIsdxmIxIxvr27Yvg4GA0adIEd911F5o1awatVlsrT2TlypUYMGAAGjdujKCgIAQHB2Pp0qU4dOhQrXMOGTIETZo0MT+Ojo5GVFQUTp06BQAoLS3Fzp07kZGRgdDQUPNxTZo0wZgxYyzOZep5qT7MBAD33XcfGjVqVGtYqGvXroiLizM/bt++PQBjPlTDhg1rbTe1yZ6xY8daPO7cubNTz7dm9OjRFo9NbRo1alSt7dZex9b927JlC/R6PcrKyrBx40bcc889aNiwIaqqqsw/d999N8rKyrBjxw6Lc9577721Xqd379744osvMHPmTOTn5+PGjRsuXzORUjEYIpKx//73v9i5cyc2bdqEJ554AocOHcKECRMsjsnJycH48eMRFxeHjz76CNu3b8fOnTvxhz/8AWVlZbXOGRkZWWubWq02f4j+9ttvMBgMaNmyZa3jam67dOkSgoKC0KJFC4vtKpUKLVu2rDVUExERYfE4JCSkzu3W2m9NzWsyDc3VJzBwpq3W2mnr/lVUVODatWu4dOkSqqqq8O677yI4ONji5+677wYAXLx40eL5MTExtc75zjvv4MUXX0Rubi6GDBmCiIgIpKenS7oEA5GvYc4QkYy1b9/enDQ9ZMgQ6PV6/Pvf/8aqVaswbtw4AMBHH32EpKQkfPLJJxZrAJWXl7v0ms2aNYNKpUJRUVGtfTW3RUZGoqqqChcuXLAIiIQQKCoqQq9evVxqgyeo1Wqr98RTuTW27l9ISAgaN26M4OBgBAYGYtKkSZg6darVcyQlJVk8trbGU6NGjTBv3jzMmzcPv/76q7mXaMyYMTh8+LB7LobIz7FniMiHLFiwAM2aNcNf//pXGAwGAMYPyJCQEIsPyqKiIquzyRxhms2Vk5Nj0eNx9epVfPbZZxbH3nnnnQCMAVl1q1evRmlpqXm/HCQmJmLv3r0W2zZt2oRr16555PVs3b+0tDQEBgaiYcOGGDJkCHbv3o3OnTujZ8+etX6s9eLVJTo6Gg8//DAmTJiAI0eO4Pr16+6+LCK/xJ4hIh/SrFkzzJo1Cy+88AKys7Px4IMPYvTo0cjJycGUKVMwbtw4FBYW4u9//ztiYmJcHir5+9//jrvuugvDhg3D9OnTodfr8eqrr6JRo0a4fPmy+bhhw4ZhxIgRePHFF6HT6TBgwADzbLJu3bph0qRJ7rr0eps0aRL+8pe/4K9//SvuuOMOHDx4EIsWLUJ4eLhHXi8wMBDDhg1DVlYWDAYDXn31Veh0OsybN898zNtvv42BAwciLS0NTz31FBITE3H16lUcP34cn332mUOz4fr06YPRo0ejc+fOaNasGQ4dOoQPP/wQ/fr1s8jDIiLbGAwR+Zhp06Zh0aJF+Nvf/oYJEybgkUceQXFxMRYvXoz//Oc/aNOmDWbOnIkzZ85YfPA6Y9iwYcjNzcWf//xn3H///WjZsiWmTJmCGzduWJxTpVIhNzcXc+fOxbJly/Dyyy+jefPmmDRpEl555RWLafVSmzFjBnQ6HZYvX47XX38dvXv3xqeffgqNRuOR13v66adRVlaGZ555BsXFxejYsSPWr1+PAQMGmI/p0KEDdu3ahb///e/485//jOLiYjRt2hTJycnmvCF7fve732Ht2rV46623cP36dcTFxeGhhx7C7NmzPXJdRP5IJYQQUjeCiIiISCrMGSIiIiJFYzBEREREisZgiIiIiBSNwRAREREpGoMhIiIiUjQGQ0RERKRoilhnyGAw4Ny5c2jSpInV5eyJiIhIfoQQuHr1KmJjYxEQ4Ln+G0UEQ+fOnUNCQoLUzSAiIiIXFBYWIj4+3mPnV0Qw1KRJEwDGmxkWFiZxa4iIiMgROp0OCQkJ5s9xT1FEMGQaGgsLC2MwRERE5GM8neLCBGoiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNMmDoS1btmDMmDGIjY2FSqVCbm6uzWOfeOIJqFQqLFy40GvtIyIiIv8meTBUWlqKLl26YNGiRXUel5ubi++//x6xsbFeahkREREpQZDUDRg5ciRGjhxZ5zFnz57F008/ja+++gqjRo3yUsuIiIhICSTvGbLHYDBg0qRJmDFjBjp27Ch1c4iIiMjPSN4zZM+rr76KoKAgPPPMMw4/p7y8HOXl5ebHOp3OE00jIiIiPyDrnqGffvoJb7/9NpYvXw6VSuXw8+bPn4/w8HDzT0JCggdbSURERL5M1sHQ1q1bUVxcjFatWiEoKAhBQUE4deoUpk+fjsTERJvPmzVrFkpKSsw/hYWF3ms0ERER+RRZD5NNmjQJQ4cOtdg2YsQITJo0CY888ojN56nVaqjVak83j4iIiPyA5MHQtWvXcPz4cfPjgoIC7NmzBxEREWjVqhUiIyMtjg8ODkbLli3Rrl07bzeViIiI/JDkwdCPP/6IIUOGmB9nZWUBACZPnozly5dL1CoiIiJSCsmDocGDB0MI4fDxJ0+e9FxjiIiISHFknUBNRERE5GkMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESma5MHQli1bMGbMGMTGxkKlUiE3N9e8r7KyEi+++CI6deqERo0aITY2Fg899BDOnTsnXYOJiIjIr0geDJWWlqJLly5YtGhRrX3Xr1/Hrl278Je//AW7du1CTk4Ojh49irFjx0rQUiIiIvJHKiGEkLoRJiqVCmvWrEF6errNY3bu3InevXvj1KlTaNWqlUPn1el0CA8PR0lJCcLCwtzUWiIiIvIkb31+B3nszB5SUlIClUqFpk2b2jymvLwc5eXl5sc6nc4LLSMiIiJfJPkwmTPKysowc+ZMTJw4sc4Icf78+QgPDzf/JCQkeLGVRESkSHo9kJ8PrFhh/Fevl7pF5CCfCYYqKyvxwAMPwGAw4P3336/z2FmzZqGkpMT8U1hY6KVWEhGRIuXkAImJwJAhwMSJxn8TE43bSfZ8YpissrIS48ePR0FBATZt2mR33FCtVkOtVnupdUREpGg5OcC4cUDNFNyzZ43bV60CMjKkaRs5RPY9Q6ZA6NixY/jmm28QGRkpdZOIiIiM9Hrg2WdrB0LArW2ZmRwykznJe4auXbuG48ePmx8XFBRgz549iIiIQGxsLMaNG4ddu3Zh3bp10Ov1KCoqAgBEREQgJCREqmYTEREBW7cCZ87Y3i8EUFhoPG7wYK81S1b0euP1nz8PxMQAaWlAYKDUrbIgeTD0448/YsiQIebHWVlZAIDJkydj7ty5WLt2LQCga9euFs/Ly8vDYKX+YRERkTycP+/e4/xNTo6x56x6wBgfD7z9tqyGDiUPhgYPHoy6ljqS0TJIRERElmJi3HucP/GhXCrZ5wwRERHJVlqasadDpbK+X6UCEhKMxymJj+VSMRgiIiJyVWCgccgHqB0QmR4vXCi7HBmPcyaXSgYYDBEREdVHRoZxyCcuznJ7fLyshoK8ysdyqSTPGSIiIvJ5GRmARiP7WVNe42O5VAyGiIiI3CEwULnT52sy5VKdPWs9b0ilMu6XSS4Vh8mIiIjIvXwsl4rBEBEREbmfD+VScZiMiIiIPMORXCoZrFDNYIiIiIg8p65cKpmsUM1hMiIiIvI+0wrVNdcjMq1QnZPjtaYwGCIiIiLvktkK1QyGiIiIyLscXaF62zavNIc5Q0RERM6QQcKvz3N05emiIs+24yYGQ0RERI6SScKvz3N05emWLT3bjps4TEZEROQIGSX8+jzTCtU1F2Q0UamAhASgf3+vNIfBEBERkT0yS/j1eTJboZrBEBERkT2OJvxu3eq9Nvk6Ga1QzZwhIiIiexxN+HX0ODJyZIVqL2AwREREZI+jCb+OHke31LVCtZcwGCIiIrLHlPB79qz1vCGVyrg/Lc37bbOG0/+dwpwhIiIie2SW8FunnBwgMREYMgSYONH4b2KiNLPd9HogPx9YscL4r0wTzBkMEREROUJGCb82yWn6v5yCMjtUQljr7/MvOp0O4eHhKCkpQVhYmNTNISIiXybXISi93hhs2Jr1ZhrKKyjwfHtNQVnNEMPUi+Zg8Oitz28GQ0RERP4gP9/Y+2JPXp5nE5bdGJR56/Obw2RERET+QC7T/31wTSYGQ0RERP5ALtP/5RKUOYHBEBERkT9wtN6Xp6f/yyUocwKDISIiIn8gl+n/cgnKnMBgiIiIyF/IYfq/XIIyJ3A2GRERkb+Rw/T/nBzg2Wctk6kTEoyBkINBGafWuxGDISIiPyaHD36yrp6/G299frM2GRER+S5rvQ/x8cZhGjmsCK10MijC6gjmDBERkW+SU+kJ8mkMhoiIyPfo9cYeIWuZHqZtmZmyLQxK8sJgiIiIfI8PrnJM8sWcISIi8j0+uMqxWzFp3K0YDBERke/xwVWO3YZJ427HYTIiIvI9PrjKsVswadwjJA+GtmzZgjFjxiA2NhYqlQq5ubkW+4UQmDt3LmJjY9GgQQMMHjwYBw4ckKaxREQkDz64ynG9KShp/FrRNayesR1/7LLdK68neTBUWlqKLl26YNGiRVb3L1iwAG+++SYWLVqEnTt3omXLlhg2bBiuXr3q5ZYSEZGsyKH0hDf5edJ40d5i/OuhLRgVtRPNY4Iw7vV++ORkP6+8tuQ5QyNHjsTIkSOt7hNCYOHChZg9ezYybv5Rf/DBB4iOjkZ2djaeeOIJbzaViIjkJiMD0GiUkUzsh0njhz//BbnvnIb2u0h8f60jBAaZ990WdAp3pe7He3s83w7Jg6G6FBQUoKioCMOHDzdvU6vVuOOOO7Bt2zabwVB5eTnKy8vNj3U6ncfbSkREEvGRVY7rzQ+SxvUVeny/7CByl16Cdk9rHK1sA6CNeX/vRvuh6X8Rmqnx6DDmNly91gzvhXu+XbIOhoqKigAA0dHRFtujo6Nx6tQpm8+bP38+5s2b59G2EREReZUpafzsWet5QyqVcX99k8bdPG3/xuUb+OatfdB+Wo7PjqWgWHQy7wtBOX7XfC80w25g7PNtEds9tX5td5GsgyETVY3kOCFErW3VzZo1C1lZWebHOp0OCQkJHmsfERGRx5mSxseNMwY+1QMidyWNu2na/qVjl7FuwUFo1wfhq/OdcB29zfvCUYJRrfdDc48Kd01PRVh8L9fb6yayDoZatmwJwNhDFFOt26+4uLhWb1F1arUaarXa4+0jIiLyKlPSuLWAZeHC+iWNm6bt1+x1Mk3bt5OU/kv+aWgX/gJtflNsLekEAwaa9yUEnoWmw3GkT2qCQVM7IbjhgFtPlMECkrIOhpKSktCyZUts2LAB3bp1AwBUVFRg8+bNePXVVyVuHRERAZDFh5mi1Ewaj4oybi8uBvLzXbv/9qbtq1TGafsajfncwiDw00eHkLukGNof47C/PBlAK/PTuoQeQXqf89A80RJd728HVUBc7XPLZAFJyYOha9eu4fjx4+bHBQUF2LNnDyIiItCqVStkZmbilVdeQXJyMpKTk/HKK6+gYcOGmDhxooStJiIiALL5MFMcU9J4Tg7w8MP1v/8OTtuv/HoT8vZEIPd/pVh7KBlnDR0AdDA2CVUY1HQf0n+nw9jnbkPiwHYA2tk+pyM9UUOHOn4N9SEklpeXJwDU+pk8ebIQQgiDwSDmzJkjWrZsKdRqtRg0aJDYt2+fU69RUlIiAIiSkhIPXAERkUKtXi2ESiWE8ePs1o9KZfxZvVrqFvo3d97/7Oza57Hy8wj+bbGpEa6KcXHbxIdPfisuHb/s+OtVVQkRH2/7tVQqIRISRMnly175/FYJYa1PzL/odDqEh4ejpKQEYWFhUjeHiMj36fVAYqLt3gTTzKaCAg6ZeYK7739+PjBkiN3DBiMPRwLaY2zbI9A80AC/e7YTQpuGOtV0Z15Pt24dwkeP9vjnt+TDZERE5IOcWQ1ZCWsAeZub778YMBCVkS0RfKkI1uZqG6CCLqQ5/u/dZuj9hxYICLI9ickhji4MeXOJHU9jMEREpDTuSHj2ldWQ/TW52w33v6qsCt8u3g/tf0ug3ZeErlXvYRXGQQAIwK1BIwEVAlRA0xWL0Teji/3XdOSeO7ow5M1Z5Z7GYIiISEnclfDsC6sh+3Nyt4v3/1rRNXz95n7krq7C+oIOuCy6mvedRwssCHsJz5a/hgblV8zbVQlOTNt39J73728MkOoqKhsYCPTpY/813YA5Q0RESmFr9o5pwT5nipuaclbsrYYsVc6QO69Vjpy4/0UHLuGz1w9D+1UovinujHLcyvGJVF3C6DaHkH5fMO5qewKhf33RMpBp3hx4/33gvvvst2nlSmD8eOttASzvucxyhiSvWk9ERF5gbx0ZwLiOTF3f1KszrYYM3PqwM3HXasiucve1ypGd+y8EsCbyMfRvehCxXZrjjx8Owvri3ihHKNoEnUJWj3xsfudnFF0Px/LjA5He6yxCH32wdh7SpUvA/fcbg8u6rFoFTJhgfZ+1e372rGPX6aWcIQZDRERK4EzCraNMqyHH1VhMLz5e2p4XT1yrHNm4/+dES9yLVcjY81dsv9YJAgHo1egAXhqaj305x3C8vBXe+HEwBk3rgqDQoPoHjzk5xp6juoLL6vc8J8d4PkcwZ4iIiNzGUwnPNVdDlkOSsq8kd9fTjcs3sHFPPNY2/C8uqq5ALcpwHjHYijQEQo8RkT9CM+w6xk5PRlzPjrZPVJ+ZaaZAylFarbFHy16GjmmYr39/x89dDwyGiIiUwJMJz6bVkOXCF5K7XXTp2GWsf81YAPXLc7ULoN7dej+yNd9j5IxUhMX3dOyk9Qke7QVSNX30kWOBEODVYVYGQ0RESpCWZvymbS/hNi3N+21zNz+71oIthdC+9Qty88LxbUkq9NUKoMYHnoOmwzGkP9gEg6akIqTxgDrOZEN9gkdnetdatAAuXLB/XPPmwOLFxl5Hnc7x89cDgyEiIiUwJdyOG2cMBqoHCVInPLubj1+rMAjsyj6M3H/+Cu2PsdhX1hZAgnl/59AjSO99Hpo/RqPbhBSoAmJtn8yRNX/qEzw607vWpw+wbp394956y/v5Zh4t9iETrE1GRHTT6tW1a0IlJPhnHTE5XmtVlRB5ecZaYHl5xsdCiPKr5eKrV34UU1LzRXzgWYsmB6JSDG66Syy8J1/8svm0469l7frj461fv6nOWc1aZ/bqnJlqjFmrkWa+gEAhPv5YiBYtHKp/JvLyzKf31uc31xkiIv/nr6sQu0pJ90NO12plQcLrjVtgcWgm5l2cCh3Czdsb4RruitsHzWg9Rs3oiIjbmjn/Ws6us2RtwcSEBPsLLppeC7DesxQZCTzzDDBnjv12t2hh/F3d/B156/ObwRAR+Td/XoWYfMfNgEEIYVH7y3Dz0TiswraAgRjb9jA09zfAnZkuFkAF6lfE1dXgMScH+OMfjesSWXs9R0ONzEzjMNlNDIbciMEQkUL5+yrEJC0HAgdhENi/6jBaP3QHGpdfsLq4nwBQ0awlgs+fRoA6uP7tcnB1Z+TluW8WoF4PtG7t+GKKtrz1FjBtmtd7hrjoIhH5JyWsQkzSyckx9r4MGQJMnGj8NzERyMlBVVkVNr+9B1k98nGbuhDT7v8VYTYCIQBQAVD/VoSA7d+5p21SrLO0dWv9AyEAeO458330Js4mIyL/VJ+F5IjqYqPHUZw5A9x7Lx7FcvwXk83bByLPsfO6KziRYp0lZ9pub9js7Fnj/V21Chg6tP5tcwB7hojIPylkFWLysjp6HFUABFT4O/6C5riAh9p8i5wXduCfOVGOndtdwYlpqnzNmmXmhqqMidHuXGfJ0bbPm1e7fEtNEvTcMhgiIv/kx6sQk3QK3/i0zh7HAAi0QiGKvtqLD04MxD2v9kWDscO9G5xIUUTX0QBs9mzg5EmLJGmrTD2327a5r411YDBERP5Jim/H5HcMVQZsX7IPM/vmo736BF540bHnBV4qrvZAguDE20V0nbnGwEAgOtqx87JqPRFRPUjxAUR+oexKGdbP3Yk/tt+COPUF9H+iE179fjAOV9yGC2ju2Elq9jh6OzgxvebJk8ZZY9nZxn8LCjw3g9KZa3S0R9ZLVes5tZ6I/JurC8mR8+S0wKGTLp/4DetfOwDtukB8ebYTStHYvC8MJbi71QGkpwN3ZaYgfFAX+6UrrK3hA/j0PXKYI9doWgvJzn3U/fwzwiMiuM6QOzAYIlI4JXwASc0HF7c8+e0ZaN88gdy8MGy90gn6ahOsTQVQNRMb446nOyGkccitJ65cCYwfX/uEXL/KObZWrq52H3VDh3LRRXdhMEREbsXgypK9xS0//dRYiVzi+yUMArtX3CyAujMGe8vaWezvFHoU6b3OQfPHaHSfmAJVgJV8M2tBnwl7HJ1np+eWK1C7EYMhInIbH+wB8Sh7pR8AY+BTfYq0F+9XxbUKbF60D9rsa1h78HYU6m/lswRAj0FN90IzWIexzyahzeBWdZ/MVtBnsnLlrZ4OclwdXy4YDLkRgyEicguW96jN0dIP1Xn4funO6PDFa/uh1Qp8fioVJdUKoDZEKe6K3QfNqCqMmtEBkckRjp20PvW+yGUMhtyIwRAR1Rs/DK1bscJYjsJZbr5fZ388j7VvHIN2Q0NsutQZlbiV4xOluoCxbQ9BMz4Ud2Z2QoOIBs6/QH3rfXFo1SXe+vxmOQ4iIkewvId1ri5aWc/7JQwCB7THoX3/LLTbW2BnaUcAt9rSLuQXaLqeRvqjkejzh44ICBrkWjtN6rOiOYdWZY/BEBGRI1jewzrT4pa2pkjb48T9qiqrwrZ/HYD2g9+Q+3MSfqlKBpAMAFDBgL6NDyA97RI001qh3cg2ANo43x5bXF3R3NbQavX6WwyIJMdgiIjIESzvYZ1pcctx4+wX4LTGzv0qLS7Fhrf2I3dlJdb90h6XRBfzPjXKMDRqL9LvKseYGSmITu3kyhU4xl7QZxr2q76ieR11zCCE8TmZmYBGwyEziTFniIjIEQ4uEqe4nCETa0NBNWeRVVfH/So+cAGfvXYY2i9DsOHXzijDrRyfCNVljE46CM29QRielYrGLRvXPLPnOLAujkUvT33zjIg5Q0REslJXDwjLexiDAI3GMkn4wgXg/vuN++3cr2MbTiJ34Ulot0Zg29VUCNzqYUkKOg1NpwJoHgrHwCdTERQ60EsXVYOp3IS1/B9r6wtxaNVnMBgiInKUsx+GShMYWLuHIzDQ6v0yvPEmfrjcFtp++dDuSsChitsAJJoP6dHwIDR9i6F5KhadMpKhCrCzBpC3WAv6bM0M49Cqz+AwGRGRszhN2jk371fF8VPY9e0NLN/RDtpjHVBkuFW5PAiVGBKxF5o7r2Hs9GQk9Imt81y17r0cfyccWq03DpMREcmVtR4Qsuq3gitYv+AAtJ+p8eXZDFxDE/O+MJRgZIKxAOrI5zsivFWPuk9ma4r6hAnG9Y7kNnWdQ6s+gz1DRETkVqe+u1kAdVMYttQogBoXcB6aDkehmdgYg6fVKIBaF3ulMGqS06rgdupvkW1cgdqNGAwREXmOMAjs+eQIchcXQftDDH6uUQA1VX0M6b3OQvN4FHo82N56AdS6OFL/zBo5DUPJcRjPB3CYjIiIZKvyeqWxAOpHV7H24G04rU8BkALAWAA1LXwfNIOvQJPZBm0G31oc0SX2Vv+2RU6rgnNoVdYYDBERkUN0Z3T48o390K4R+Px0R1wR3c37GqIUI2JuFkB9vj2at+vq3Mnr6jmp79RzTl0nO2QfDFVVVWHu3Ln43//+h6KiIsTExODhhx/Gn//8ZwQEBEjdPCIi18h12KRGu841Ssbat05Au6EBNl3sjAr0Nx8apbqAMcmHoRmvxtDnOqFBRF/XXtNe7a76Tj3/9Vfjdcnh/pI8CZl76aWXRGRkpFi3bp0oKCgQK1euFI0bNxYLFy50+BwlJSUCgCgpKfFgS4mIHLR6tRDx8UIYB3KMP/Hxxu0St8tQo12nES/uwWrzprbBv4gZvfLEd4v3iqryKre8plCpLO8FYNymUhn3V1UZ74+14xz9kcP9Jad56/Nb9gnUo0ePRnR0NJYuXWredu+996Jhw4b48MMPHToHE6iJSDZszYqScPaTvkKPI1PeRvul0yEAVO9zN0AFFYDcrnPR/uUHkXK3G4uf2kuMrp4ArdVaL4XhKE/fX7n29Pk4b31+y36caeDAgdi4cSOOHj0KAPj555/x7bff4u6775a4ZURETrJXuBMwFu60Vc/Lja5fvI7cWd/jkeStiFVfRJOlb9UKhAAgAAIqFXDPpX8jZURr9zbCXmJ09QRo0+rfcXGWxyQkADNmGIOmunjy/ubkGIO6IUOAiRON/yYmGreTT5B9ztCLL76IkpISpKSkIDAwEHq9Hi+//DImTJhg8znl5eUoLy83P9bpdN5oKhFR3Zz58PfAzKMLhy7iswWHjAVQizrhBvoAAO5APhLgpXZV70E5eNCx55gSoOsqhTF/PvDuu8Bzz3nnOkxs9fSdPWvcLod1jsgu2QdDn3zyCT766CNkZ2ejY8eO2LNnDzIzMxEbG4vJkydbfc78+fMxb948L7eUiMgOCQp3HttwEtq3T0K7tRm26VJhqFYANTGoEJrUX/B4x+3A/7zQLmuJ0o6onkBta4p6YCAQHV17uzXuur/2evpUKmNPlEbDITOZk30wNGPGDMycORMPPPAAAKBTp044deoU5s+fbzMYmjVrFrKyssyPdTodEhISvNJeIiKb6lO408GcFEOVATs/OAjt0ovQ7orHwfLbUb0AavcGh6Dp+ys0T8ag87i2UAUkAPnCsWCoPrO6nF1BGriVM5SWZv9YwPuFUSXu6SP3kX0wdP369VpT6AMDA2EwGGw+R61WQ61We7ppRETOSUszfrjbK9xZ88PfztTzcl05Ni3cC+3HN7D2SFucN6QaFz7EVvwe2xHd5DraDE/GmOnt0KpfewDt3dMuR9XVg2KLK7W76nMdriRAS9DTRx7i0blqbjB58mQRFxdnnlqfk5MjmjdvLl544QWHz8Gp9UQkhDBO0c7LEyI72/hvlRumhjvLNJW85jTx6lPJrR1fY6q4QaUSBkAsiHhFNIbOYvdEfCguBLZwbmq5s+1yRl6e81PhExJce01XrsPVpQ4cva68POevg4QQ3vv8ln0wpNPpxLPPPitatWolQkNDRZs2bcTs2bNFeXm5w+dgMEREslrbx1pbrH34m9bXsfEhq4dKnEKCCECViA04J57quFnsmvCqMNhbt6e+7XJWdrZjQcOf/+yeQNWZ63BknSNb7K1/pFIZX1eKoNtPcJ0hN+I6Q0QKJ8O1fRwZlhGb8qC683d2T3X4xWVo+9JDCFAJx9ftsTUE5In1cvLzjdPN7cnLc19ujSPX4cw6R7bugelvC7D8+5Lyb8uPsGq9GzEYIlIwd3zgeVHl9Upsec9YAFXs24d3xTT7T8rOBiZMkCbocITpd2Avl8fbvwN33S9rOV0JCcZ8JwZC9cKq9URE7uADM36unruKL1/fD+0aA9afulUA9Q44+F312DHjv3JN6A0MNCZ7jxtnDHys9aA4kyjtLu66X3Wtf0Q+walgqLCwkFPUici31OcDz4MlFs7v+RVrXzsC7YYG2HihMyrQz7yvheoCxtx+GJp7Q2D4IBYB58/VfbJ//QuYPdv7U8udYVpB2tqsOKl6UNx5v2ytf0Q+walgKCUlBVlZWZg5cyYaNWrkqTYREbmPqx949iqpO0kYBA6tOwHte2eQ+11z/FCaCuDWIoHJwQXQdDmF9D9EoO+jHREYcnP6d4MngDlz6j75mTPGoM3TU+TrS249KHK/X+Q1TtUm27BhA77++mskJydj2bJlnmoTkX/T6425CitWGP/1Qh0qRTN94JmGY2pSqYz5HdU/8ExJsTWH10wlFhysOaWv0OPb9/fi+Z75aBt6Ch01t+NPXw++GQgBfRrtx/wR+Tj42QkcKUvEazsHY8BTnREYUi04SE527DrPn781HGW6rprXCUgzHFWdqQdlwgTjv1K3Re73i7zDlSloH3zwgYiPjxddu3YVeT6wfgKn1pNsyGl6t5I4s/aMnens9qZLl14oFbmzdohHkreIFqpii6eGoEzc3eIH8c/fbxbndhc51nZX1rLx1BR5f8X7JVuyn1p/48YNzJ8/H2+88QaGDx+O1157Dbfffrt7IzU34WwykgU5Tu9WEkdn/Lgww+jCoYtY99ohaL8IwddFnXADDc2HNVVdwejE/dDcE4gR01PRJLaJc+12dSaWB/Od/BLvlyzJfmr99evXsWvXLqxevRrvvPMOgoODMXXqVMydOxdNmjj5ZvcwBkMkOR+b3u23HPnAW7ECmDjR7qmK/vQ2/re3M7RbmuE7XSoMuHWe1oFnoEk9Ac2kMKQ9lYrghsH1azfXsiGFkmUwtHjxYuzcuRM7d+7EoUOHEBgYiM6dO6Nv377o2rUr/ve//+Ho0aNYs2YNevbs6bFGO4vBEElOruu/UG0O/q4GIw+bMdj8uFuDQ9D0MRZA7XJfW6gCbOQouYpr2ZACyTIYSkhIQN++fc0/PXv2rFUQ9ZVXXkF2djb279/v9sa6isEQSc7B3gbz4nkkHTvDUgaocAbxSMZRpDU7CM3vrmLsc7eh9YB477SNQzmkILJcdLGwsNDuMY8++ij+8pe/uNwgIr8k5/Vf6svPPqCvnLmGvR2fQtqZ2RBQIaDawocGqKCCwMkRT6LoH2VoltTdu41zdi0bP/vdEHmKU1PrHREVFYVNmza5+7REvs2V6d2A/Kfh5+QYe1GGDDH2fA0ZYnzs4NRzuTi9/SzeHbcZQyN2oUViQ9zx1Z9wL1bjLOIsjlPFxUG1ejUGffknNEtqKk1jHeUnvxsib2BtMiJvcTYJ1s2L/rmdD8+OEwaBn1cehXbxeWi/j8buG+0t9ndUH4Omx1lo/hCBnokXEVD8q/Q9K8708vjw74aoOlnmDPkqBkMkG44mwebkAPfeW/v5cvkw88HZcZXXK7H1H/uh/VAH7f7bcEp/K8cnAHoMCNsPzaDfoMlMwu13tpawpVY4Exj74O+GyBYGQ27EYIhkxd43fL0eiI4GLl2y/nw5fJjJaXZcHffz6rmr+OqN/dCu0WP9yY74TTQzP60BrmN4y33QjKzA6Bnt0aJ9c8+201W2enkA499CzcB440Zg6FD75+XMRfIBskygJiI3sJcE+/LLtgMhQBZV1mVTHd1Kj4k+OgYb207B24dH4JsaBVCbqy5izO2HoBkXgmFZndCweR/Ptq++9Hrj9dn6zioE8Mc/Gut9BQYa78fjjzt2bm9XrieSMQZDRHKi19+qlWSPlB9mcpgdd7PHRAiB6mnpql+LMPTXv2IxOqACvXB78EmkdzkJzSMR6PdYtQKovmDrVtvDXSaXLhkD6NRU2z1I1vjizEUiD2EwRCQnW7cCly87dqyUH2YSV/vW36hA1aNTEFIjEAKAAAgYoMIy9ZN4aUVHtNe0hSog0SPtcJirU9y1WsfO//rrQJMmjgVCrMROVIvbp9YTUT042tsTESHth5kE1b5vXL6BtbO/x6Ntt+Lehp9DfeXXWoGQSQAEwssvoEOz8+5fCdpZrk5xz8kx3kNHXL0KnDvneJtYiZ3IAoMhIjlxtLfn2WelneKdnw+UlwNz5wJxlmvxID7ebbPdLh65hOWPfYt7YnegeaQBmlf64D/H0tAANxw7gdR5Mabk55pDXWfPGrfbCohMuULuFhEh/UxEIhniMBmRnNgbfgKAyEhg9mzvtsvE2hTvuDhg3jwgOdkta/Gc2HQK2rcLoN3cFN+WdIIBA837WgWegabjCUzudxX4pwMnk3Iosa7kZyGMPWiZmbeSn6tzJFfIFZ9+Ctx5p/vPS+TjGAwRyYlp+GncOOOHpbVF85YskaZXyNYU73PnjD1Eq1a5NLvNUGXAT/87DO2/ipH7YxwOlCcDuLXOT9cGh6HpXYT0p0wFUOMB/UBg/d8ly1lyiL2Apq5Zge7u0TLdD06lJ7KKwRCR3GRkGAMLT1QodzWRtz69HFZUXKtA3tt7oV1xHdpDyThn6ACgAwAgEFW4o9lepJsLoKYASLE8QV1Bo4dylpxWn+UH3NmjJZf7QSRjDIaI5CgjwxhYuLPIZn3Ke9Snl+OmK6dK8MXrB5CrBb4oTMVV9DTva4yrGBm/H5oxBtw9o6NjBVBtBY3x8fUPGt2hPssPODJc6ii53A8iGeMK1ERKUN9aVStWGGdC2ZOdDUyYYH5Y+P05aF8/Bu2mxsi/3BlVCDbviwkowth2R6F5oAF+l9kZ6jC1M1d0i1wrs5vKYtgbyrO1kritWnZ1UamMOVzLlwPFxfK6H0QuYDkON2IwRIrmjlpVDpbfEBs3Ye/FWGMB1B3R2FWjAGoH9XFoup+B5tHm6DW5AwKC/HxCq7PFea09v2bPV2SkcaFFW8ODnC1GfoTBkBsxGCJFc0cdMTu9HAIqXAlugZ6G7/GLPtG8XQUDBoTtgybtN2ieTUTysMRaz3WYXHuA7HG0OK8t1q5bq/VMThmRzLA2GRG5hzvqiNWRsGy4ufTho5X/wC9IRChuYHjLvdDcVYHRz6cgqmOX+rTeqD75TlKrb/6XtVp2nsgpI1IwBkNy5avfgkl+3FRHrOj2gdg36G/ouvVdtBDF5u1nEI+/4m8Iv70Fcu/73v0FUG3lO5kWLvSFYSF7xXnlck4iheIwmRz58rdgkp96JPIe/vwXaN89jdxvI/H9tY4QCEAA9EjDVnQN2It2txuQOnUQ+j/ZBYEhHgjW3ZHv5A388kLkERwmUyp/+BZM8uLEmjz6Cj2+X3YQuUsvQbunNY5WtgHQxnx470b7oel/EZqp8egw5g7P1/1yw5R+j+OXFyKfx2BITty8sB05QCnf6OtYk6f876/i693x0M7ais+OpaBYdDLvDkE5ftd8LzTDbmDs820R2z3Vu+12R76TJ/HLC5FfYDAkJ77wLdif+PM3emtBXrWk26u7j+G7b8rw713d8cXDXXEdjcxPDUcJRrXeD809Ktw1PRVh8b2kuw435Tt5BL+8EPkNBkNyIvdvwf7En7/Rr1oFTJkCXLhwa1t8PIoe/zNW7GoHbX5TbC35Awy49QGdEHgWmg7HoXmwCe54uhOCGw6QoOFW2FuJWcoaZPzyQuQ3GAzJibPfgpUyxONu/vyN/oUXgNdeq7VZnDmDqDlPYStWYTMGAwC6hB6Bpvd5pD/ZEl3vbwdVQJyXG+sAOdcg45cXIr/h58u/+hjTt2CVjaRUlcq4sFpamrFnIzHRuJjexInGfxMTjdupbs58o/clK1dCvPYarE0PNf5FCSxWTcE792xCwdYz2HOjHeZtHoxuE1I8nwhdH6Z8p7gawVp8vLQ9eHIewiMipzAYkhPTt2CgdkBU/VuwVmv8plzzA900xMOAqG5+9o2+5HQJPnl6K0oeeBwqmAKf2gIARIlfMe2ZACQOjPdiC90gIwM4edK4SnZ2tvHfggJphzKd+fJCRLLGYEhu7H0L1mjqHuIBjEM8er3Hm+qz/OAbfeH35/De+M0YHvkTWrRugH+8p0e4ocSxJ/tIkFeLaZHBCROM/0o9hOnolxep20lEdjFnSI7qWmo/P9+/kza9kQcl56RcG4RBYF/OMWj/cQ7aHVH46XoHALHm/T0D9wCOxr8yDvJ8Th1LFrBOGJHv8Ilg6OzZs3jxxRfxxRdf4MaNG2jbti2WLl2KHj16SN00z7G11L6fDfFY8NZUdzkn5VZTVVaFbxfvh/a/JdDuS0JBVVsAbQEYC6D2b7IfmrTL0DzTGm3VXQEHarGiRQtpgzx/TPpnnTAinyf7YOi3337DgAEDMGTIEHzxxReIiorCiRMn0LRpU6mbJg0/GOKxyttT3WX6jb60uBRfvb4P2tVVWFfQAZdFV/O+UNzAsOhbBVCjUzvfeqK+lbHtdfUaAsD770v3Ie3P6zqxThiRT5N9bbKZM2fiu+++w9Z6zOzxudpkdalHnSnZkqr+lF5vHHbMzzc+HjxYklyUX/dfwGevHYb2SzU2FHdGOULN+yJVlzAmaT8e7XMAPQc1QGhKku1eB1sBpcmMGcCCBR66CjvstW3lSuN+IqJqvPb5LWSuffv2IjMzU4wbN060aNFCdO3aVSxZsqTO55SVlYmSkhLzT2FhoQAgSkpKvNRqD1u9WgiVyvhj/Hgx/pi2rV4tdQudk5dneR22fvLy3Peaq1cLER9vef74eK/du8OfnxD/d1ee6Nd4r1BBb9GMNkEnRVaPPLH5nT2iasWnzrXT2nW1aCHEp5965bqsqqqq3aaaP4GB0raRiGSppKTEK5/fsg+G1Gq1UKvVYtasWWLXrl1i8eLFIjQ0VHzwwQc2nzNnzhwBoNaP3wRDQlj/0EtI8L1ASAghsrMdC4ays93zeqZgsub5PRhM6iv14rvFe8ULvfNEu5ATtV66V6P94qWheWJfzlFh0Bvq186qKmPgmJ1t/Leqyu3X4xRHg13AN/9+ichjvBUMyX6YLCQkBD179sS2bdvM25555hns3LkT27dvt/qc8vJylJeXmx/rdDokJCT4xzBZdf6SjJqfb1w00p68vPrnZXhxSO7G5RvYuHAftJ+W4bNjKfjVEGXeF4wK/C5yLzTDrmPs9GTE9ayR4yXl0KG7/6ZWrDAuDOqIhATfGuIlIo/y1jCZ7BOoY2Ji0KFDB4tt7du3x+rVq20+R61WQ61We7pp0vOXpE1vTnX3cD2pS8cuY/1rB6FdH4Qvz3XCdfQ27wtHCe5uvR8ajQojZ6QiLL6nZO20ylMJzs4k8/vyshBE5LNkHwwNGDAAR44csdh29OhRtG7dWqIWkdt5c6q7B5YmKNhSCO1bvyA3LxzflqRCj4HmffGB56DpcAyaBxpgcLcSBF+5aAwOYhrVcUbPtLNOnpzNZwp27c10M/HFZSGIyKfJPhh67rnn0L9/f7zyyisYP348fvjhByxZsgRLliyRumnkTt6a6u6GpQmEQWBX9mFol/yK3J2x2FfWFkCCeX/n0CPQ9DqP9CeijXW/cnc43+PizSUUPF241hTs3nuvY8f72rIQROTzZJ8zBADr1q3DrFmzcOzYMSQlJSErKwuPP/64w8+XzdR6f8nx8SRP3yMXlyaouFaBzYv2Ifd/17D2UDLO6G+t/hyIKqQ13Yf0ITqMzWyDpEG3AiObPS6mHi9bPS722gkYF1A8cwYICanfffNWztaqVcADD9guFeOLy0IQkUdxar0beSsbvU4ST+WmahxcmuDKqStixbTvxAOtvhNhuGJxaCNcFffGbRP/fWKruHT88q1zV5/J9c03QsTF2Z45pVIZZwDamu1lq501/4ZmzKjf35Y3Z/N9+qnte+GLy0IQkUdxar0bSR4MSTCVm+ywsTTBhVf/Ld67P18Mj9wpglFusTs64FfxeMpmsW7OD+LGbzccO6cjP3Wtn+TqOZ352/L2Ok/+tCwEEXkUp9a7kaTDZFJNkSb79HqIzVtQ+NlubN4SgEWH7sQPNzpZHJIScgKaboXQ/CESff7QEQFBAdbPZW+F5bpkZxsrsdtSUQHExQEXLzp3Xkf/tqRY1ZxDxkTkAE6t9xdSTJGmOlWVVeG7f+6H9r9XkLv3NhRU3cqXUcGAfk32QzPwMjTTWqHdyNsA3Fb3CetKQHaEvYThbducD4QAx/+2pChc6y/LQhCRX2Aw5GlSV5nnN3AAxgKoX79xswDqL+1xqVoBVDXKMCxqLzR3lWPMjBoFUB1hL+C1xdH1k+r7t+HI82VauJaIyBsYDHmalFXm/blKuAOKDxgLoOZ+ocY3xZ1Qhr7mfRGqyxiddBDp44IwfHonNIrqXceZbqoZWPbvb+y1qWMBUJuc6XGp79+Go8/PyDBOn2fwTEQKw5whT5OqyryrU7p93NGvCpD79ilov43A9qupELiV45MUdBrpnX+B5qGmGPBEKoJCnfguYC2wDAy0PU3cnoQEx3tcHJlmbw3z0YjIx3nr85vBkDeYAhPAej6GuwMTBSVtG6oM+P4/B6D9zyVodyfgcIVlfk/Phgeh6VcMzVOxSL0nGaoAlfMvUp/k6OrmzAHatXOtx8XW35Atfh70EpEyMBhyI8mDIcB6z4IzvQPO8GbhUwmUXSkzFkD95AY+O9oORYZo875gVGBI5F5ohpZi7PS2iO9VzyEme4GlM+pbhNTW39ADDxiLoXrjb4uIyIs4m8zfeDMfQ+qkbQ+4fOI3rH/tALTrAvHl2U4oRS/zvjCU4O5WB6DRACOf74jwVnUUQHWWq8nR1tR31mBdf0Pz5zPXh4jIRQyGvMlb04mlTNp2o5PfnoH2zRPIzQvD1iudLAqgxgWch6bDUWgmNsbgaZ0Q0ri/Zxrh7oCxvuez9Tfk7r8tzkIkIgVhMOSPTFXC7SVt25vS7WXCILB7xc0CqD/EYG9ZOwDx5v2dQo9C0/Mc0p+IRveJKVAFeCGYc3fAKPMAFIDiZyESkfIwZ8hfeTtp20WV1yuNBVA/uoq1B29HoT7OvC8AegxquheawTqMfTYJbQa38n4DXZ3JVZOvJK0rdBYiEckTE6jdSJHBEODdpG0n6M7o8MVr+6HVCnx+KhUlCDfva4hS3BW7D5pRVRg1owMikyMka6eZszO5avKVQEJBsxCJyDcwGHIjxQZDgGxyP87+eB5r3zgG7YaG2HSpMyoRYt4XpbqAsW0PQTM+FHdmdkKDiAZeb59djqwz5Oszu/x8FiIR+R7OJvMnUgYkEtWAEgaBA9rj0L5/FtrtLbCztCOAW/ky7UJ+gabraaQ/aiqAOsjrbXSKtZlcphWo/WVmlx/OQiQicgSDIU9zNRlVJj06zqgqq8K2fx2A9oPfkPtzEn6pSgaQDMBYALVv4wPQDLwEzbRWSLm7DYA2kra3Trbuf83A0hszu7zFT2YhEhE5i8NknuRqMqoPzea5fvE6vn5jH3JXVt4sgBpp3qdGGYZG7UX6XeUYPb0dWnaOkrClTvCh++9WUpWOISKygTlDbiRJMORqMqoPzOYpPnAB614/jNwvQrDh184ow60cH1MBVM29QRielYrGLRtL2FIX+MD99ygfmYVIRMrAYMiNJAmGXElGlfFsnmMbTiJ34Ulot0Zgm5UCqJpOBdA8FI6BTzpZAFVOZHz/vUqmsxCJSHmYQO3rXElGtVf6QYj6l3RwkKHKgB+WH4R26UVodyXgUMVtABLN+3s0PAhNX2MB1E4ZyVAFSLAGkLvJ6P5LypulY4iIZIDBkKe4kowq8Wyesitl2PT2Pmg/voG1R9uhyJBq3heESgyJ2AvNndcwdnoyEvp0ANDBI+2QDGdT3eKrSeBERC5gMOQprpTEkGA2z28FV7B+wQFoPwvAl2dTca1aAdQm0OHuhP1ITzcVQO3htteVJc6mIiJSJGXlDK1bhzCdznvd/s4mo3ppNs+p724WQN0Uhi1XOkFfLSaOCziPse2PIf33jW4WQA2p40x+hrOpiIhkhQnUbmS+mQDMt9JbU6WdTUb1wGweYRDY88kR5C4ugvaHGPxc1s5if6r6GNJ7nYXm8Sj0eLA9VAEqp87vVzibiohINhgMuZHVYMibH27OLqDohtk8ldcrseW9fcj98CrWHrwNp/W3qr8HQI+08H3QDL4CTWYbaQqgyhlnUxERyQKDITeyGgwBtYc9pF71ufrrR91coLC42OG26M7o8OUb+6FdI/D56Y64Ipqa9zVEKUbE3CyA+nx7NG8XaftEJP3fAhERcWq9V1SfKn35srSrDtta9fjxx43/vXWr1Q/kc7uKsPb1o9BuaIBNFzujAv3N+6JUFzAm+TA049UY+lwnNIjo6/nr8BecTUVEpBjK7hkyycw0Bj1SrTpsa9XjmuLjId5aiIPBXaB97wy025rjh9JUi0PaBhdA0/WUsQDqIx0QGMLeDCIi8k0cJnMju8FQ8+bAxYvWn+zpGUT2Vj2uRgAQUGEcVmENbgVnfRvvQ7pFAVQZ4/ATERE5iMNk3qBSGQOhCxdsH+PpVYftrXpcjQrGgOhtPIvK5rEYO7ICY55PQcvOndzfLk9QagFUIiKStQD7h/gp0xDY73/v2PGeWnXYyfMGQCABZ/DZyjI8/t9BvlUJfty42oHf2bPG7Tk50rSLiIgUT7nBUHy8MRdIo3HseA+tOnzmrItr+vhSSQi93tgjZG1E1rQtM9N4HBERkZcpa5hs3Tqg5grUer3zZTPqwVBlwM4PTAVQ43G4/D6cxAzE4SwC4ET6lr3grGZuTv/+wLZt0uTqsAAqERHJmLKCobQ0oGYCVmCgMWdl3Dhj4GNt1eGFC+sVOJTryrFp4V5jAdQjbXG+RgHU/zR+Bn+99gIEVFDZC4gcCc6s5eaYAj8Tb+bqsAAqERHJmHKHyarLyDAOmcXFWW43DaW5EDD8VnAF/5vyHe6L347m4RW4e04v/PPQIJw3tEQT6DA+YRuyn96GCyevY87VGVCtXg1VfFzdJ3UkOLOVm1NzCMqbuTosgEpERDKmrKn19qbm1XPat6kAqjavCbb81glVCDbviw04j7Epx5A+sSEGT+sEdZi67tc/dgxYssQYtJjYKwnhxDR9AN4rPMoCqERE5AJOrZeCk6sOC4PAzyuPIvcf5/HZ9y3QpOwCYnAeesTAgAB0VB9Des+bBVB/n4KAIDs9HzVff/Zs54IzJ6bpGy/AS7k6XhiKJCIichWDISeZCqBqP7qKtQduwyl9O9yDA8jFJCTgViBS1SIGQYsX1S8nx9mSEK7m3HgjV8c0FGltnSEWQCUiIgn5XM7Q/PnzoVKpkJmZ6bXXvHruKlZmbceDSd8hqnEphr7QHe/uvQOn9PF4ACuwGvciHpY9MkEXi7y/fo6rOTfeytXJyABOngTy8oDsbOO/BQUMhIiISFI+1TO0c+dOLFmyBJ07d/b4a53f8yvWvnYE2g0NsPFCZ1Sgn3lfC9UFjLn9MDT3BmL0f2dAdc7KCYQwDgFlZhrXMjINAXmyHEVaWt3LBNTk5mUDHMICqEREJDM+0zN07do1/P73v8e//vUvNGvWzO3nFwaBg2uPY/6IfPRpvB+x3aLxZPYgfHGhFyqgRnJwAZ7vmY9v39+L82URWHo0DWNHVCDg3Nk6TlotJwcw9hIlJgJDhgATJxr/TUx0X++RKTcHuJWLUxchmKtDRESK5zM9Q1OnTsWoUaMwdOhQvPTSS3UeW15ejvLycvNjnU5n9Th9hR7b/30Auf+5DO3eRByvvB3A7eb9fRrtR/rAi9A8nYCUu9tAFZBkeQJn1s+xVZneNMXdxSn8tZhyc555xnImmjWRkY6vwE1EROSnfCIY+vjjj7Fr1y7s3LnToePnz5+PefPmWd13/eJ1bHhzH7SrKrDueAouiFtDbiEox50t9iJ9+A2Meb4dYrqmWj2HmaO5NlFRwMMP2y5HYW04rT4yMoDwcGDo0LqPu3SJqz4TEZHiyT4YKiwsxLPPPouvv/4aoaGhDj1n1qxZyMrKMj/W6XRISEjAhOQfkVfcFzfQx7yvqeoKRifuh+aeQIyYnoomsb0cb5y9HB1TTg7g/XIUxcWOHcdVn+vPk3lgRETkcbIPhn766ScUFxejR48e5m16vR5btmzBokWLUF5ejsAaHzxqtRpqde1FDT8v7gmgIVoHnoEm9QQ0k8KQ9lQqghsOdK1xjq6fI0VgwlWfvcNa6RNvljohIqJ6k/0K1FevXsWpU6cstj3yyCNISUnBiy++iNRUO0NZuLWC5cyB63D/07ejy31toQpwsVq8NdY+EKuvFp2fb0yWticvz309Q1z12fNs5YGZAmF35YERESmUt1agln0wZM3gwYPRtWtXLFy40KHjXb6Zzgx/1HWsVIGJ6cMasN5rxQ9r19krfcJgk4io3liOQ2q2hj8efxxITq4d8NS1fo5U5Si46rPn2Ct94q1SJ0REVG8+2TPkLKcjS1vDHzU5mxtibzjNU5jg634rVhjXirInOxuYMMHz7SEi8kPsGZKKXm8MWByJEZ1dIygjAxg9Gnj/feDECeC224ApU4CQkPq3uy5c9dn9mKBOROQ32DNUk6PJzibO5IZY6xlq3hx48EHjGkPssfEdTFAnIvI4b/UM+Uw5Dq9xdnp7zZIbtpiG3mrmmVy8aBwmc3dpDvKsukqfeDIPjIiI3I7BUE2uDmvUFUQ5OvRmGnZjQOQbTAnqcXGW2+PjOVOPiMiHcJisJnvDH7bUtUaQM0NvHF7xPUxQJyLyCCZQS6WuafDWmIKXtDTbxzgz9MYp2b6HCepERD6Nw2TW2Br+qMnR3BBXht5YM4yIiMgrGAzZkpEBnDxpHP7KzgbmzXM9N8RU0LVmom1dOCWbiIjIK5gz5Iz65IY4upAjc4aIiIgAMGdInuqTG2KrNEZ1nJJNRETkdRwm86bqQ2+ZmUCLFpb7OSWbiIjI6zhMJiVOySYiIrKJw2RKwCnZREREkuMwGRERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESma7IOh+fPno1evXmjSpAmioqKQnp6OI0eOSN0sIiIi8hOyD4Y2b96MqVOnYseOHdiwYQOqqqowfPhwlJaWSt00IiIi8gMqIYSQuhHOuHDhAqKiorB582YMGjTIoefodDqEh4ejpKQEYWFhHm4hERERuYO3Pr+DPHZmDykpKQEARERE2DymvLwc5eXl5sc6nc7j7SIiIiLfJPthsuqEEMjKysLAgQORmppq87j58+cjPDzc/JOQkODFVhIREZEv8alhsqlTp2L9+vX49ttvER8fb/M4az1DCQkJHCYjIiLyIRwmq2HatGlYu3YttmzZUmcgBABqtRpqtdpLLSMiIiJfJvtgSAiBadOmYc2aNcjPz0dSUpLUTSIiIiI/IvtgaOrUqcjOzoZWq0WTJk1QVFQEAAgPD0eDBg0kbh0RERH5OtnnDKlUKqvbly1bhocfftihc3BqPRERke9hztBNMo/ViIiIyMf51NR6IiIiIndjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGg+Ewy9//77SEpKQmhoKHr06IGtW7dK3SQiIiLyAz4RDH3yySfIzMzE7NmzsXv3bqSlpWHkyJE4ffq01E0jIiIiH6cSQgipG2FPnz590L17d/zjH/8wb2vfvj3S09Mxf/58u8/X6XQIDw9HSUkJwsLCPNlUIiIichNvfX7LvmeooqICP/30E4YPH26xffjw4di2bZtErSIiIiJ/ESR1A+y5ePEi9Ho9oqOjLbZHR0ejqKjI6nPKy8tRXl5uflxSUgLAGGESERGRbzB9bnt6EEv2wZCJSqWyeCyEqLXNZP78+Zg3b16t7QkJCR5pGxEREXnOpUuXEB4e7rHzyz4Yat68OQIDA2v1AhUXF9fqLTKZNWsWsrKyzI+vXLmC1q1b4/Tp0x69mXKj0+mQkJCAwsJCReVK8bp53UrA6+Z1K0FJSQlatWqFiIgIj76O7IOhkJAQ9OjRAxs2bMA999xj3r5hwwZoNBqrz1Gr1VCr1bW2h4eHK+qPyCQsLIzXrSC8bmXhdSuLUq87IMCzKc6yD4YAICsrC5MmTULPnj3Rr18/LFmyBKdPn8aTTz4pddOIiIjIx/lEMHT//ffj0qVL+Nvf/obz588jNTUVn3/+OVq3bi1104iIiMjH+UQwBABTpkzBlClTXHquWq3GnDlzrA6d+TNeN69bCXjdvG4l4HV79rp9YtFFIiIiIk+R/aKLRERERJ7EYIiIiIgUjcEQERERKRqDISIiIlI0nwyG3n//fSQlJSE0NBQ9evTA1q1b6zx+8+bN6NGjB0JDQ9GmTRssXry41jGrV69Ghw4doFar0aFDB6xZs8ZTzXeZM9edk5ODYcOGoUWLFggLC0O/fv3w1VdfWRyzfPlyqFSqWj9lZWWevhSnOHPd+fn5Vq/p8OHDFsf52+/74YcftnrdHTt2NB/jC7/vLVu2YMyYMYiNjYVKpUJubq7d5/jD+9vZ6/aX97ez1+0v729nr9tf3t/z589Hr1690KRJE0RFRSE9PR1Hjhyx+zxvvMd9Lhj65JNPkJmZidmzZ2P37t1IS0vDyJEjcfr0aavHFxQU4O6770ZaWhp2796NP/3pT3jmmWewevVq8zHbt2/H/fffj0mTJuHnn3/GpEmTMH78eHz//ffeuiy7nL3uLVu2YNiwYfj888/x008/YciQIRgzZgx2795tcVxYWBjOnz9v8RMaGuqNS3KIs9dtcuTIEYtrSk5ONu/zx9/322+/bXG9hYWFiIiIwH333WdxnNx/36WlpejSpQsWLVrk0PH+8v529rr95f3t7HWb+Pr729nr9pf39+bNmzF16lTs2LEDGzZsQFVVFYYPH47S0lKbz/Hae1z4mN69e4snn3zSYltKSoqYOXOm1eNfeOEFkZKSYrHtiSeeEH379jU/Hj9+vLjrrrssjhkxYoR44IEH3NTq+nP2uq3p0KGDmDdvnvnxsmXLRHh4uLua6BHOXndeXp4AIH777Teb51TC73vNmjVCpVKJkydPmrf5wu+7OgBizZo1dR7jL+/v6hy5bmt88f1dnSPX7S/v7+pc+X37w/tbCCGKi4sFALF582abx3jrPe5TPUMVFRX46aefMHz4cIvtw4cPx7Zt26w+Z/v27bWOHzFiBH788UdUVlbWeYytc3qbK9ddk8FgwNWrV2sVu7t27Rpat26N+Ph4jB49utY3SynV57q7deuGmJgY3HnnncjLy7PYp4Tf99KlSzF06NBaq7TL+fftCn94f7uDL76/68OX39/u4C/v75KSEgCoswirt97jPhUMXbx4EXq9vla1+ujo6FpV7U2KioqsHl9VVYWLFy/WeYytc3qbK9dd0xtvvIHS0lKMHz/evC0lJQXLly/H2rVrsWLFCoSGhmLAgAE4duyYW9vvKleuOyYmBkuWLMHq1auRk5ODdu3a4c4778SWLVvMx/j77/v8+fP44osv8Nhjj1lsl/vv2xX+8P52B198f7vCH97f9eUv728hBLKysjBw4ECkpqbaPM5b73GfKcdRnUqlsngshKi1zd7xNbc7e04puNrGFStWYO7cudBqtYiKijJv79u3L/r27Wt+PGDAAHTv3h3vvvsu3nnnHfc1vJ6cue527dqhXbt25sf9+vVDYWEhXn/9dQwaNMilc0rF1TYuX74cTZs2RXp6usV2X/l9O8tf3t+u8vX3tzP86f3tKn95fz/99NPYu3cvvv32W7vHeuM97lM9Q82bN0dgYGCtaK+4uLhWVGjSsmVLq8cHBQUhMjKyzmNsndPbXLluk08++QSPPvooPv30UwwdOrTOYwMCAtCrVy/ZfJOoz3VX17dvX4tr8ufftxAC//nPfzBp0iSEhITUeazcft+u8If3d3348vvbXXzt/V0f/vL+njZtGtauXYu8vDzEx8fXeay33uM+FQyFhISgR48e2LBhg8X2DRs2oH///laf069fv1rHf/311+jZsyeCg4PrPMbWOb3NlesGjN8YH374YWRnZ2PUqFF2X0cIgT179iAmJqbebXYHV6+7pt27d1tck7/+vgHjbI3jx4/j0Ucftfs6cvt9u8If3t+u8vX3t7v42vu7Pnz9/S2EwNNPP42cnBxs2rQJSUlJdp/jtfe4w6nWMvHxxx+L4OBgsXTpUnHw4EGRmZkpGjVqZM6qnzlzppg0aZL5+F9++UU0bNhQPPfcc+LgwYNi6dKlIjg4WKxatcp8zHfffScCAwPF//3f/4lDhw6J//u//xNBQUFix44dXr8+W5y97uzsbBEUFCTee+89cf78efPPlStXzMfMnTtXfPnll+LEiRNi9+7d4pFHHhFBQUHi+++/9/r12eLsdb/11ltizZo14ujRo2L//v1i5syZAoBYvXq1+Rh//H2bPPjgg6JPnz5Wz+kLv++rV6+K3bt3i927dwsA4s033xS7d+8Wp06dEkL47/vb2ev2l/e3s9ftL+9vZ6/bxNff30899ZQIDw8X+fn5Fn+3169fNx8j1Xvc54IhIYR47733ROvWrUVISIjo3r27xbS8yZMnizvuuMPi+Pz8fNGtWzcREhIiEhMTxT/+8Y9a51y5cqVo166dCA4OFikpKRZvLrlw5rrvuOMOAaDWz+TJk83HZGZmilatWomQkBDRokULMXz4cLFt2zYvXpFjnLnuV199Vdx2220iNDRUNGvWTAwcOFCsX7++1jn97fcthBBXrlwRDRo0EEuWLLF6Pl/4fZumTtv6u/XX97ez1+0v729nr9tf3t+u/J37w/vb2jUDEMuWLTMfI9V7XHWzgURERESK5FM5Q0RERETuxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBH5pBUrViA0NBRnz541b3vsscfQuXNnlJSUSNgyIvI1rE1GRD5JCIGuXbsiLS0NixYtwrx58/Dvf/8bO3bsQFxcnNTNIyIfEiR1A4iIXKFSqfDyyy9j3LhxiI2Nxdtvv42tW7cyECIip7FniIh8Wvfu3XHgwAF8/fXXuOOOO6RuDhH5IOYMEZHP+uqrr3D48GHo9XpER0dL3Rwi8lHsGSIin7Rr1y4MHjwY7733Hj7++GM0bNgQK1eulLpZROSDmDNERD7n5MmTGDVqFGbOnIlJkyahQ4cO6NWrF3766Sf06NFD6uYRkY9hzxAR+ZTLly9jwIABGDRoEP75z3+at2s0GpSXl+PLL7+UsHVE5IsYDBEREZGiMYGaiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGj/DyfcRT7ljAaNAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_16_2.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients for OLS\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1d7695a",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Same code but now with momentum gradient descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "74404bdc",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.]\n",
      " [3.]]\n",
      "Eigenvalues of Hessian Matrix:[0.331075   4.25018094]\n",
      "0 [-10.99619572] [-12.63347711]\n",
      "1 [-0.06146873] [0.05293756]\n",
      "2 [-0.05668052] [0.0488139]\n",
      "3 [-0.0522653] [0.04501146]\n",
      "4 [-0.048194] [0.04150522]\n",
      "5 [-0.04443985] [0.0382721]\n",
      "6 [-0.04097813] [0.03529083]\n",
      "7 [-0.03778607] [0.03254179]\n",
      "8 [-0.03484266] [0.03000689]\n",
      "9 [-0.03212854] [0.02766946]\n",
      "10 [-0.02962583] [0.0255141]\n",
      "11 [-0.02731808] [0.02352663]\n",
      "12 [-0.02519009] [0.02169399]\n",
      "13 [-0.02322786] [0.0200041]\n",
      "14 [-0.02141849] [0.01844584]\n",
      "15 [-0.01975006] [0.01700897]\n",
      "16 [-0.0182116] [0.01568403]\n",
      "17 [-0.01679297] [0.0144623]\n",
      "18 [-0.01548486] [0.01333573]\n",
      "19 [-0.01427864] [0.01229692]\n",
      "20 [-0.01316638] [0.01133903]\n",
      "21 [-0.01214076] [0.01045576]\n",
      "22 [-0.01119504] [0.00964129]\n",
      "23 [-0.01032298] [0.00889027]\n",
      "24 [-0.00951885] [0.00819774]\n",
      "25 [-0.00877737] [0.00755917]\n",
      "26 [-0.00809364] [0.00697033]\n",
      "27 [-0.00746317] [0.00642737]\n",
      "28 [-0.00688182] [0.0059267]\n",
      "29 [-0.00634575] [0.00546503]\n",
      "theta from own gd\n",
      "[[3.98232596]\n",
      " [3.01522108]]\n",
      "0 [-0.00585143] [0.00503932]\n",
      "1 [-0.00539563] [0.00464677]\n",
      "2 [-0.00483858] [0.00416704]\n",
      "3 [-0.00429456] [0.00369852]\n",
      "4 [-0.00379682] [0.00326986]\n",
      "5 [-0.00335174] [0.00288656]\n",
      "6 [-0.00295713] [0.00254671]\n",
      "7 [-0.00260839] [0.00224638]\n",
      "8 [-0.00230059] [0.00198129]\n",
      "9 [-0.00202904] [0.00174743]\n",
      "10 [-0.00178952] [0.00154115]\n",
      "11 [-0.00157826] [0.00135922]\n",
      "12 [-0.00139195] [0.00119876]\n",
      "13 [-0.00122762] [0.00105724]\n",
      "14 [-0.0010827] [0.00093243]\n",
      "15 [-0.00095488] [0.00082235]\n",
      "16 [-0.00084215] [0.00072527]\n",
      "17 [-0.00074274] [0.00063965]\n",
      "18 [-0.00065505] [0.00056414]\n",
      "19 [-0.00057772] [0.00049754]\n",
      "20 [-0.00050952] [0.0004388]\n",
      "21 [-0.00044937] [0.000387]\n",
      "22 [-0.00039632] [0.00034132]\n",
      "23 [-0.00034953] [0.00030102]\n",
      "24 [-0.00030827] [0.00026549]\n",
      "25 [-0.00027188] [0.00023414]\n",
      "26 [-0.00023978] [0.0002065]\n",
      "27 [-0.00021147] [0.00018212]\n",
      "28 [-0.00018651] [0.00016062]\n",
      "29 [-0.00016449] [0.00014166]\n",
      "theta from own gd wth momentum\n",
      "[[3.99956181]\n",
      " [3.00037737]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients for OLS\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x#+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 30\n",
    "\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(theta)\n",
    "    theta -= eta*gradients\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "# Now improve with momentum gradient descent\n",
    "change = 0.0\n",
    "delta_momentum = 0.3\n",
    "for iter in range(Niterations):\n",
    "    # calculate gradient\n",
    "    gradients = training_gradient(theta)\n",
    "    # calculate update\n",
    "    new_change = eta*gradients+delta_momentum*change\n",
    "    # take a step\n",
    "    theta -= new_change\n",
    "    # save the change\n",
    "    change = new_change\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"theta from own gd wth momentum\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "398dee87",
   "metadata": {
    "editable": true
   },
   "source": [
    "## But noen of these can compete with Newton's method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f4f7ac9a",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.95845431]\n",
      " [3.1424112 ]]\n",
      "Eigenvalues of Hessian Matrix:[0.31758645 4.42644801]\n",
      "0 [-17.28364423] [-20.62918713]\n",
      "1 [-1.11891833e-14] [-3.34599467e-15]\n",
      "2 [4.62954328e-16] [6.26536626e-16]\n",
      "3 [-6.85866294e-16] [-8.50258788e-16]\n",
      "4 [4.62954328e-16] [6.26536626e-16]\n",
      "beta from own Newton code\n",
      "[[3.95845431]\n",
      " [3.1424112 ]]\n"
     ]
    }
   ],
   "source": [
    "# Using Newton's method\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "beta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(beta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "# Note that here the Hessian does not depend on the parameters beta\n",
    "invH = np.linalg.pinv(H)\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "beta = np.random.randn(2,1)\n",
    "Niterations = 5\n",
    "\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(beta)\n",
    "    beta -= invH @ gradients\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"beta from own Newton code\")\n",
    "print(beta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fe85919",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Including Stochastic Gradient Descent with Autograd\n",
    "In this code we include the stochastic gradient descent approach discussed above. Note here that we specify which argument we are taking the derivative with respect to when using **autograd**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f4ca22ee",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.21392756]\n",
      " [2.76329289]]\n",
      "Eigenvalues of Hessian Matrix:[0.24999088 4.37057274]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own gd\n",
      "[[4.21392756]\n",
      " [2.76329289]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHFCAYAAADxOP3DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABSlElEQVR4nO3deXwTZf4H8M/05mgLLWcvWu4W5D6FCoikHrBgRRQV0VVXBXVZPIDVFerFyk9ddRFZPGBXLbhKAY/VhqMFFFAQEOUSsEApVO4WCr3S5/dHTGjapJkkk2Qm83m/Xnn11ZnJ5JmZTOfb5/g+khBCgIiIiEingvxdACIiIiJ/YjBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEpEJLliyBJEnWV0hICNq2bYvbb78dBw4c8Fu55syZA0mS/Pb5WnD48GFIkoRXXnnF30UhIplC/F0AInJs8eLF6Nq1K8rLy/Htt9/ixRdfRF5eHvbt24fmzZv7u3hERAGBwRCRinXv3h39+vUDAAwfPhwmkwmzZ8/GypUrce+99/q5dORPVVVV1lpDIvIMm8mINMQSGP3222/WZeXl5Xj88cfRq1cvREdHIyYmBoMHD8aqVavqvV+SJDzyyCP44IMPkJqaisaNG6Nnz5744osv6m375ZdfolevXggPD0dKSorDZp/y8nLMmjULKSkpCAsLQ3x8PKZOnYrz58/bbJecnIzRo0fjiy++QO/evdGoUSOkpqZaP3vJkiVITU1FkyZNMGDAAGzbts3p+bA0J+bl5eHhhx9GixYtEBsbi8zMTBw/frzesc+ZM6fePpKTk3HPPffU2+e6devwwAMPIDY2FlFRUbj77rtRVlaG4uJiTJgwAc2aNUPbtm3xxBNPoKqqqt5+a2pq8OKLLyIpKQkRERHo168f1q5dW2+7AwcO4I477kCrVq0QHh6O1NRUvPXWWzbb5OfnQ5IkfPDBB3j88ccRHx+P8PBwHDx4EJcuXcITTzyBlJQUREREICYmBv369cPSpUudnj8iMuO/FEQaUlBQAADo3LmzdVlFRQXOnj2LJ554AvHx8aisrMSaNWuQmZmJxYsX4+6777bZx5dffomtW7fiueeeQ9OmTTFv3jzcfPPN2L9/P9q3bw8AWLt2LcaOHYvBgwdj2bJlMJlMmDdvnk0QBgBCCIwbNw5r167FrFmzkJ6ejl27dmH27NnYvHkzNm/ejPDwcOv2P/74I2bNmoWnn34a0dHRyMrKQmZmJmbNmoW1a9fipZdegiRJmDFjBkaPHo2CggI0atTI6Xm5//77cdNNNyE7OxuFhYV48skncdddd2HdunVun+v7778fmZmZWLZsGXbs2IG//vWvqK6uxv79+5GZmYk//elPWLNmDV5++WXExcVh+vTpNu+fP38+2rVrh9dffx01NTWYN28ebrjhBqxfvx6DBw8GAOzZswdXX301kpKS8Oqrr6JNmzbIzc3FY489htOnT2P27Nk2+5w1axYGDx6MhQsXIigoCK1atcL06dPxwQcf4IUXXkDv3r1RVlaGn3/+GWfOnHH72Il0RxCR6ixevFgAEFu2bBFVVVXiwoUL4uuvvxZt2rQR11xzjaiqqnL43urqalFVVSXuu+8+0bt3b5t1AETr1q1FaWmpdVlxcbEICgoSc+fOtS4bOHCgiIuLE5cvX7YuKy0tFTExMaL2n42vv/5aABDz5s2z+ZyPP/5YABCLFi2yLmvXrp1o1KiROHbsmHXZzp07BQDRtm1bUVZWZl2+cuVKAUB89tlnss7TlClTbJbPmzdPABAnTpywOfbZs2fX20e7du3E5MmT6+3z0Ucftdlu3LhxAoB47bXXbJb36tVL9OnTx/p7QUGBAODw/F133XXWZRkZGSIhIUGUlJTY7PORRx4RERER4uzZs0IIIfLy8gQAcc0119Qrf/fu3cW4cePqLSci+dhMRqRigwYNQmhoKCIjI3H99dejefPmWLVqVb1+Ip988gmGDBmCpk2bIiQkBKGhoXjvvfewd+/eevscMWIEIiMjrb+3bt0arVq1wpEjRwAAZWVl2Lp1KzIzMxEREWHdLjIyEmPGjLHZl6XmpXYzEwDceuutaNKkSb1moV69eiE+Pt76e2pqKgBzf6jGjRvXW24pkzN/+MMfbH7v0aOHS++3Z/To0Ta/W8p000031Vtu73Mcnb8NGzbAZDKhvLwca9euxc0334zGjRujurra+rrxxhtRXl6OLVu22Ozzlltuqfc5AwYMwFdffYWZM2ciPz8fly9fdvuYifSKwRCRiv3nP//B1q1bsW7dOjz44IPYu3cvJk6caLNNTk4OJkyYgPj4eHz44YfYvHkztm7dij/+8Y8oLy+vt8/Y2Nh6y8LDw60P0XPnzqGmpgZt2rSpt13dZWfOnEFISAhatmxps1ySJLRp06ZeU01MTIzN72FhYQ0ut1d+e+oek6VpzpPAwJWy2iuno/NXWVmJixcv4syZM6iursY///lPhIaG2rxuvPFGAMDp06dt3t+2bdt6+3zzzTcxY8YMrFy5EiNGjEBMTAzGjRvn1xQMRFrDPkNEKpaammrtND1ixAiYTCa8++67+PTTTzF+/HgAwIcffoiUlBR8/PHHNjmAKioq3PrM5s2bQ5IkFBcX11tXd1lsbCyqq6tx6tQpm4BICIHi4mL079/frTJ4Q3h4uN1z4q2+NY7OX1hYGJo2bYrQ0FAEBwdj0qRJmDp1qt19pKSk2PxuL8dTkyZNkJWVhaysLPz222/WWqIxY8Zg3759yhwMUYBjzRCRhsybNw/NmzfHs88+i5qaGgDmB2RYWJjNg7K4uNjuaDI5LKO5cnJybGo8Lly4gM8//9xm25EjRwIwB2S1LV++HGVlZdb1apCcnIxdu3bZLFu3bh0uXrzolc9zdP7S09MRHByMxo0bY8SIEdixYwd69OiBfv361XvZq8VrSOvWrXHPPfdg4sSJ2L9/Py5duqT0YREFJNYMEWlI8+bNMWvWLDz11FPIzs7GXXfdhdGjRyMnJwdTpkzB+PHjUVhYiOeffx5t27Z1u6nk+eefx/XXX49Ro0bh8ccfh8lkwssvv4wmTZrg7Nmz1u1GjRqFjIwMzJgxA6WlpRgyZIh1NFnv3r0xadIkpQ7dY5MmTcLf/vY3PPvssxg2bBj27NmD+fPnIzo62iufFxwcjFGjRmH69OmoqanByy+/jNLSUmRlZVm3eeONNzB06FCkp6fj4YcfRnJyMi5cuICDBw/i888/lzUabuDAgRg9ejR69OiB5s2bY+/evfjggw8wePBgm35YROQYgyEijXn00Ucxf/58PPfcc5g4cSLuvfdenDx5EgsXLsT777+P9u3bY+bMmTh27JjNg9cVo0aNwsqVK/HMM8/gtttuQ5s2bTBlyhRcvnzZZp+SJGHlypWYM2cOFi9ejBdffBEtWrTApEmT8NJLL9kMq/e3J598EqWlpViyZAleeeUVDBgwAP/9738xduxYr3zeI488gvLycjz22GM4efIkunXrhi+//BJDhgyxbpOWlobt27fj+eefxzPPPIOTJ0+iWbNm6NSpk7XfkDPXXnstPvvsM/zjH//ApUuXEB8fj7vvvhtPP/20V46LKBBJQgjh70IQERER+Qv7DBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1XeQZqqmpwfHjxxEZGWk3nT0RERGpjxACFy5cQFxcHIKCvFd/o4tg6Pjx40hMTPR3MYiIiMgNhYWFSEhI8Nr+dREMRUZGAjCfzKioKD+XhoiIiOQoLS1FYmKi9TnuLboIhixNY1FRUQyGiIiINMbbXVzYgZqIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0ze/B0IYNGzBmzBjExcVBkiSsXLnS4bYPPvggJEnC66+/7rPyERERUWDzezBUVlaGnj17Yv78+Q1ut3LlSnz33XeIi4vzUcmIiIhID0L8XYAbbrgBN9xwQ4PbFBUV4ZFHHkFubi5uuukmH5WMiIiI9MDvNUPO1NTUYNKkSXjyySfRrVs3fxeHiIiIAozfa4acefnllxESEoLHHntM9nsqKipQUVFh/b20tNQbRSMiIqIAoOqaoR9++AFvvPEGlixZAkmSZL9v7ty5iI6Otr4SExO9WEoiIiLSMlUHQxs3bsTJkyeRlJSEkJAQhISE4MiRI3j88ceRnJzs8H2zZs1CSUmJ9VVYWOi7QhMREZGmqLqZbNKkSbjuuutslmVkZGDSpEm49957Hb4vPDwc4eHh3i4eERERBQC/B0MXL17EwYMHrb8XFBRg586diImJQVJSEmJjY222Dw0NRZs2bdClSxdfF5WIiIgCkN+DoW3btmHEiBHW36dPnw4AmDx5MpYsWeKnUhEREZFe+D0YGj58OIQQsrc/fPiw9wpDREREuqPqDtRERERE3sZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka75PRjasGEDxowZg7i4OEiShJUrV1rXVVVVYcaMGbjqqqvQpEkTxMXF4e6778bx48f9V2AiIiIKKH4PhsrKytCzZ0/Mnz+/3rpLly5h+/bt+Nvf/obt27cjJycHv/zyC/7whz/4oaREREQUiCQhhPB3ISwkScKKFSswbtw4h9ts3boVAwYMwJEjR5CUlCRrv6WlpYiOjkZJSQmioqIUKi0RERF5k6+e3yFe27OXlJSUQJIkNGvWzOE2FRUVqKiosP5eWlrqg5IRERGRFvm9mcwV5eXlmDlzJu64444GI8S5c+ciOjra+kpMTPRhKYmIiEhLNBMMVVVV4fbbb0dNTQ0WLFjQ4LazZs1CSUmJ9VVYWOijUhIREZHWaKKZrKqqChMmTEBBQQHWrVvntN0wPDwc4eHhPiodERERaZnqgyFLIHTgwAHk5eUhNjbW30UiIiKiAOL3YOjixYs4ePCg9feCggLs3LkTMTExiIuLw/jx47F9+3Z88cUXMJlMKC4uBgDExMQgLCzMX8UmIiKiAOH3ofX5+fkYMWJEveWTJ0/GnDlzkJKSYvd9eXl5GD58uKzP4NB6IiIi7dHN0Prhw4ejoXhMRWmQiIiIKABpZjQZERERkTcwGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0rUQfxeAiIiIVMBkAjZuBE6cANq2BdLTgeBgf5fKJxgMERER6V1ODvDnPwPHjl1ZlpAAvPEGkJnpv3L5CJvJiIiI9CwnBxg/3jYQAoCiIvPynBz/lMuHGAwRERHplclkrhESov46y7Jp08zbBTAGQ0RERIHAZALy84GlS80/5QQwGzfWrxGqTQigsNC8XQBjnyEiIiKtc7fPz4kT8vYvdzuNYs0QERGRlnnS56dtW3mfIXc7jZKEsNdQGFhKS0sRHR2NkpISREVF+bs4REREyjCZgORkx01dkmSuISooMA+Trzt8/uqrgQ4dzIGTvXCg7vt9zFfPbzaTERERaZUrfX7OnrXflDZxIvDKK+bAp3ZAJEnmn6+/HvD5hthMRkREpFVy+/KsWuW4Ke2VV4AnngDi423XtWhhDp5iYjiajIiIiFRKbl+ejz5qePj8smXAoUNAXp55KH2LFsCpU+ZaoREjzE1xAZxviMEQERGRVqWnm5u6LE1adUkS0LKlObBxxNKUtmmTuSntjTeA06dttwnwBIwMhoiIiLQqONgcvAD1AyLL73feKW9fRUW6TcDIYIiIiEjLMjOBTz+t3+cnIcG8fOxYefs5dUq3CRg5moyIiEhL7M0un5lpDnrszTpvMpkDI2fD51u2lPf5AZiAkcEQERGRVjjLND18eP33WJrSxo9vePh8TIy8MgRgAka/N5Nt2LABY8aMQVxcHCRJwsqVK23WCyEwZ84cxMXFoVGjRhg+fDh2797tn8ISERH5iyeZpp01pWVmyuuMnZho3i7A+D0YKisrQ8+ePTF//ny76+fNm4fXXnsN8+fPx9atW9GmTRuMGjUKFy5c8HFJiYiI/ESJ2eUzM4HDh83D57OzzT8LCq7MXSanM3aAJmBU1XQckiRhxYoVGDduHABzrVBcXBymTZuGGTNmAAAqKirQunVrvPzyy3jwwQdl7ZfTcRARkabl55vz/TiTl2e/qcwV9priEhPNgVBDk756AafjAFBQUIDi4mIYDAbrsvDwcAwbNgybNm1yGAxVVFSgoqLC+ntpaanXy0pEROQ1vpxdvqHO2AFK1cFQcXExAKB169Y2y1u3bo0jR444fN/cuXORlZXl1bIREVEAsDcyS40PfV/PLh8c7HkNk4b4vc+QHFKdtkshRL1ltc2aNQslJSXWV2FhobeLSEREWpOTY55mYsQI4I471D3thI47N/uCqoOhNm3aALhSQ2Rx8uTJerVFtYWHhyMqKsrmRUTkUyaTuZ/H0qXmnwGYtVfTPBmZ5Q867tzsC6oOhlJSUtCmTRusXr3auqyyshLr16/H1Vdf7ceSERE1QEs1DnqkxMgsf5AzPJ7c4vc+QxcvXsTBgwetvxcUFGDnzp2IiYlBUlISpk2bhpdeegmdOnVCp06d8NJLL6Fx48a44447/FhqIiIHLDUOdR+0lhoHPrT8b+NG+dNOqK3fjA47N/uC34Ohbdu2YUSt4YLTp08HAEyePBlLlizBU089hcuXL2PKlCk4d+4cBg4cCKPRiMjISH8VmYjIPmc1DpJkrnEYO5YPL3/y5cgsb9BJ5+bq8mps/fcen3yWqvIMeQvzDBGRT/gyFwy5j9dJtY58ewy5b/8KY34o1h5PxXkRBEDneYaIiDRF6zUOvuLv4eyWkVnOJi5V88gsf59DhVwsvoi8t/bAuOoyjPuS8EtVCoAE6/poHEWJD8rBYIiISCm+zgWjRc4mGvUFuROXqjW4UMM5dFNNdQ12LNuP3P/8BuP3zbCpJA1VGGBdH4xqDIrcg4z+Z2G4owU6j41HTEvvl4vNZERESjGZzKPGnNU4FBSo90HrTY46l1sCEF93LlfRtBOyqe0cylC07QSMCw7CuDYYqwu74IyItVnfPuQIMrocgWFMOEY83BXRSdHWdb56fjMYIiJSkuVhBdivcVDhw8onLIGio1FcrgSKSjYRaam5Sclz6EWXTl/ChoV7YFxxEcbd8dhd0clmfSRKMbLtHhiGVcLwQDt0uLadw31xbjIiIi2y5IKx14yh5hoHb1NqOLvSTURaGpml0pQAokbgp5wDyF18HMYtkdh4thsq0M+6XkIN+jfZg4w+p2G4PQYD70lFaONBPiufHAyGiIiUxlww9SnRuVzvOZxU1EH/t59PYfVbv8BoFFh9uBOKazoD6GxdnxhchIyOh2C4MRQjp3ZFTIfuXi+TJxgMERF5g5ZqHHzB087lzOHk1w76FaUV+PadPcj9bwmMP7XBzstdAVzp2dwYZRjecg8y0i/BcF8iulyfAiko3vEOVYZ9hoiISDmO+uB42rmcuYF82kFf1Ajs+9+vML5/DLnfNMb6U2m4hCY22/RutBcZPX+D4dZoXH1/GsKjwhsuuxs1pewzRERE2uKsP48nw9lV1ETkN15OCXDmwFmsXbAPuV+aYPy1A46ZOgDoYF3fJug3GJIPION6CddN6YxW3VIBpDrfsQZSATAYIiIiz8ntz+Nu53LmcDJTsIN+1aUqbHl/D3KXnYNxZ0tsK0uFwJVJ0MNRjmtidiPj6gsw3BOH7jd3ghTU2rXyaqSfF5vJiIjIM64O+XanyYQ5nGy5cQ5FjcChvKMwvnMYuesjkFeciguwfSZ2Dz8AQ/ciZNwSifQH09AoppFnZfQwFQCbyYiISJ3qPohNJteGfLvTuVzrWaOVJvMclhwtwbq39iL380oYDySjoLodgCt5fVpIpzEqaT8M1wkYpnREXJ9OADo53J9LVJoKwB4GQ0REntBS0j4l2Ov/ERMj772e9udhDienqsurse3DfTBmn0buthh8dyENJlzJ6ROKSgxpthuGASXIuLs1et3WBUEhQ7xTGA3182IwRETkLg10DFWUo/4fZ8/Ke78S/XmYw6meI98eg/FfvyJ3nWWmd9ucPl3CfoWhayEybm6MYQ+lommb3r4pmIb6ebHPEBGROzQ4R5RHnPX/aIje+vN42cXii8hfsAe5K2vP9H5FM+k8rovbC8O1VTA82B7thiQ42JOXKdDPi32GiIjUSo8JAJ31/3BEj/15FGaZ6d34wW/I/c7xTO+GfmeRcWcL9JuUiuCwwX4ssaVg2unnxWCIiMhVGuoYqhi5/TpiYmybzdifxy1F205g9cJDMK6RsPpoF5wWtjl9UkKOIqPzYRhGh+HaqamITurhv8I2RCP9vBgMERG5SkMdQxUjt1/Hf/9r/k+f/XlccvnsZWx4ezeMKy4i92fLTO9XznkkSnFtm70wDCtHxp+Sf5/pPclv5XWJBvp5MRgiInKVhjqGKiY93fzfvLP+H5ah84D+Rtq5wDLTu3HJcRg3R2KDg5neDb1Pw3Bbcwz6YxpCGw/0Y4k9pPK5+hgMERG5Sm5gkJ7u+7J5i6v9P/Q20k6Gk7t/n+k9V8BoZ6b3hODjyOhwCIYbgzFySlfEdlL3TO+BhKPJiIjcYRlNBtgPDJQcTaamGhZ7QU5iom3/D72NtHPAMtO78ZMSGHe1xo7LtvN4WWZ6Nwy9BMMfE9D1xvaQgiQ/lbYOlXznfPX8ZjBEROQuOYGBNz7D3zUsDT0oFZiCQatqz/Ru/KYx8h3M9G7oYZ7pfcgDTmZ6V5IrwY2KvnMMhhTEYIiIvPafrjf/g9ZiDUt+PjBihPPt8vJU3YdELstM78b/mWA81B6Fpnib9ZaZ3g0Z5pneW3dv6ftCuhLcqOw7x2BIQQyGiHRORf/pyqbVGpalS4E77nC+XXY2MHGi98ujMMtM78aPz8G4owW2lqVBIMi63jLTu2Gweab3qzI7+bfpy5XgRoXfOSZdJCJSgqOHQVGRebmn/+l6q2ZIq7mMAnCk3cG1R2B85zCM6yOwrjgVF9DTZr1lpnfDzU1xzcPd0Cimr59KWoez5KCAeb0lOahWv3MKYDBERIHL25mivVnjpNVcRgEw0s4y07vxi0oYf2mHXxuY6X3UQx0Q30/Bmd6VJCdr+LFjwIsvAs8+q93vnAIYDBFR4PLmf7rernHSag2LWqZgcKHGzlRpwtb/7IUx+zSM22KwpYGZ3g2TWqP37V6c6V1JcoOW2bOB7t21+51TAIMhIgpc3vpP1xdzk7lbw1JZCSxYABw6BHToAEyZAoSFuVcGd/l7CgYZNXaWmd6N60KxpoGZ3g1jG2HYQ6mIjPPRTO9KciVomTYNOHhQ87V6bhM6UFJSIgCIkpISfxeFiHwpL08I85/1hl9ZWd7Zb16eZ+VfvlwISTK/au/Xsmz5ctvtn3xSiOBg222Dg83L/aG62nwOsrPNP6urvf+ZlnNW51rUSJKoAcQ77Z4TXcIO1btUzaRzYnz8JrFo0npRsLHQ++X0hepqIRIS5H1XLd9XV79zXuar5zeDISIKXJaHgZ2HY70/9K78kc/Olvdwyc72/BiWL6//QEtMtB8INVQWfwVErvIkgHLy8DdBEkeQKIJQLYJRJa6O/FFkjcgTm9/5SVRdrvLWEfnX8uXygyHL91Xud84HfPX85tB6IgpsOTnALbc0vI2rQ4Z9nUvHWf+XykqgcWPzdo4EBwOXLvm+ycwVHnZIP70oBy0edHKtAWy8/S1c9fc70axdtCel1Y7nnjP3C3Km9veVGagDD4MhIp1z52HQEEs+Fmd9K3yVj+X114G//MX5dv/4h7lviBq5keyv9kzvxp/j0b1iG5YicHMcuc1kAtq1M39f7VFrziowzxARkXI6yRz2LLcjtVpGTFkcOqTsdr4ms0O6GPMH/PzZr8hdbH+m9xYolPd5ATgaqkHBwcCbbzY8l54vv68qFOR8EyIijfPGkGHLiKl42+kXkJDg+2kyOnRQdjtfk5kC4Zbwz9BjfGc8+eVwrD7bFxWIQELwcfyx00Z8PG0TPt17lfn8Sw4yPkuSee64QBwN5Yyavq8qxGYyIgp83mzWUkPfCq33GZI5hcdEZGMVxmJ4y93IcDTTu6W5DbBfA6L3B78avq8uYDMZEZFSLM1ajjpSC+F+M0FwsP+nJggLA6ZPB/7v/xxvM326KgMhUSNQeEQgSca2Tz50EUteDkZ4VH/HG8nJcaSxgEBR3v6+avTcMhgiIvIHpR8a8+aZf772mm0NUXCwORCyrFeBs4fOYe1b+5D7ZTWMh9qjyHQbDmMG4lGEIDiuuesz/4/yzlFmpjnhpb3zq8VJe7VCw+eWzWREFPjUNhu3Nx8azjJQ++E/96pLVfhuyV7kLj3rcKb3WU3/iWcvPgVAggQvNW+5MWKNZPLSufXZ89urWYwUUFVVJZ5++mmRnJwsIiIiREpKisjKyhImk0n2Pph0kUjnfJUxWg4HGZJ9kuHXXjK9hASvfObBtYfFgtvXi7FttohIlNQ73G7hv4i/9MkTX7+wVZSdKnNcPqWS/TnLxixJ5s/yZpZsf2Tk9gUvnltmoP7dCy+8IGJjY8UXX3whCgoKxCeffCKaNm0qXn/9ddn7YDBEpHMffui7jNEN8ecD2ctB2Pkj58WKmVvEQ2nrRfuQw/U+JlY6LW5P+lYsvm+jOLb1uOMdeStg8HdA7MNA1Oe8eG599fxWfZ+hzZs3Y+zYsbjpppsAAMnJyVi6dCm2bdvm55IRkSbk5MhLSAh4P/+MzCHk2LhR2U6uzvL4AC5PLGuqNGHbB+aZ3nO3WmZ6H2hdH4IqDIneDcOA88i42zLT+9XOd+ytDr7emrRXDkdNSEVF5k79WVnmXFga6nBsw5/nViGqD4aGDh2KhQsX4pdffkHnzp3x448/4ptvvsHrr7/u76IRkdo5egjV5e5s3K72v/HXQ8NZEAbICsKObi6CceGvyF0bgrXHu+JcnZneO4cWwND1KDLGWWZ67+V52ZXijVxTcsgJRGtnR9dIh2Mb/jq3ClJ9MDRjxgyUlJSga9euCA4OhslkwosvvoiJDaRSr6ioQEVFhfX30tJSXxSViLzB3Q6/DT2E7HF1aL07naDlPgxatZJfDjkcTcPgZLuLxRexfuFeGFdeQu7eROyvbA/gStK+aJTguvg9MAyvguGh9kgemgIgRblyKyk93Xx9nOWaUjoho5xAtLaiInMAr6XO3P46t0ryaiOcApYuXSoSEhLE0qVLxa5du8R//vMfERMTI5YsWeLwPbNnzxYA6r3YZ4hIYzzpZyG3H0PLlq7323C3/42lz5C999Z+xcfLK5Pc/jX/+Iesc2F65VXxw4d7xNyMPDGi2XYRigqbTSwzvc8ZrtGZ3i3Xre7592bn9exsed9DX3fmVpqXzi07UP8uISFBzJ8/32bZ888/L7p06eLwPeXl5aKkpMT6KiwsZDBEpDWedviV+xD68EPXyuVpJ2hHDw1Xj9GVQFFmB/IHsaDe4uSQo+LB1PVi+ZObxbnD5107V2rkzRFr9sgNyu29fDG6UUleOLfsQP27S5cuISjIdgq14OBg1NTUOHxPeHg4wsPDvV00IvIWmRN3NtjhV26TVN25mpzxtBO0owzJdffR0DE21CHXXhOLzGPch1Q0xQVc22YPDOnlyHgwGR1GJEEKSrT/Bi1mG24oIaM3OGtCaoirfcf8fT18fW6V5NVQSwGTJ08W8fHx1qH1OTk5okWLFuKpp56SvQ8OrSdVCtScI0pQYqiusyYpd5si5NY4ORumv2aNe8foYs1UjalG7Pp4jyiNaClMDt5jAsS5sJZiwxs/iMqySnnnIZCHiitNTm2gpzVDAXo92Ez2u9LSUvHnP/9ZJCUliYiICNG+fXvx9NNPi4qKCtn7YDBEqhOgf7gUo1TA4Y1+DErlVHH3GGV+/uqxb4rJHTaKtkEnBCDEzVguTJDqBUQ1lvPhyrnwZ+JIrbJ3z8sMaGXtO0CvB4MhBTEYIlUJ4D9cilEyiZvS/RiUqnFy9xhlBlG3I9v6ayOUiRtafi8+G/CcqIxt49m5UEMmZ62qXRuclaVMoB7g18NXz2/OTUbkS2qbI0utLOfJ2VBduedJ6b4Ulj47gG35XJmHyd1jzM8HRoxwWsT7w/6N2F5JMIyPwpAH0hDRLOLK53pyLmR+PvLyvDs7uq94sx+OvfQMiYnmNA9yh9UH+PXw1fNb9R2oiQKKvzIQa01wsDlfz/jx5qDAXsDhSl4gpbMaO+oEnZAg/0Hm4jFaZnpf/aXAHLRFGxTbneFdAKhpHYd3i+60f348PRcBkG1YNm/Pwq5Eh2M9XQ8vcikYKiwsRGKig1EFROQc/3DJp0TA4U1KPMgaOMbql1/BluKOMF6TD+P2FthalooaDAYAnMZ8fIrxqAFgM9ZWkiABCF7wT+/VLAZAtmFZXB2x5y5Pg1O9XA8vc6mZrEmTJpg+fTpmzpyJJk2aeLNcimIzGalGgFdpe4W/hwv7wu/H+NuaXdiySeA/e/thTXF3lCLaZrNu4Qdg6FYEw81NMSLhAML/9pRnTSzullXJJkx3y+DN74SWmrPVcD28yGfPb1c6GH377bdiwIABom3btuL999/3Rh8mr2AHalINbw33Jk0qKSwRK2ZuEQ93Wy86NDDT+/v3bhCF39uZ6d1f6Rn8kcm59md7eySmv2e4d5U/r4eXqXo02b///W+RkJAgevXqJfLU8mVoAIMhUpUA/sNFDauuqBZb3v1JPHdtnhgatVMEo8rmKxCCSjEseod4cVSe2Prv3aK6QsVBsa8zOVs+0xcjMZVK7eBL/rgePqD60WSXL1/G3Llz8eqrr8JgMOD//u//0LFjRyUrrRTDZjJSHSVGkZAmWGZ6N64LwZqirjgnmtust8z0bhjbCMMfTkVkXKSfSuoGe81VgHeasHzZdKXV5uwAbFL21fPb7WDo0qVL2L59O5YvX44333wToaGhmDp1KubMmYPISHXdzAyGSJUC8A9XQB6Ti8pOliF/wR4YV16CcW8C9lV2sFlfe6b3UX9KQco1ATQoxZujr3wZoAR4PxwtUeXQ+oULF2Lr1q3YunUr9u7di+DgYPTo0QNTp05Fr1698NFHHyEtLQ0rVqxAv379vFVmosCg9HBvf/P2MGSVqqmuwc6P98P4wW8wfheNb853QxX6W9cHwYRBkbth6HsWhomx6H93KkIiBvuxxF7i7dFXvhyJqXRqB1I9l2qGEhMTMWjQIOurX79+9SZEfemll5CdnY2ff/5Z8cK6izVDRF7m6EHoShJCDTmx8zcY3zoA42oJq492xinR0mZ9ckghMjoVwDA6DNdOTUWzdtEO9hQgfNGE5Y+mKzZn+53qm8kc+e233xAXFweTyaTkbj3CYIjIi7Q0DNlNl89exjeL9iD30wsw7o7DT+WdbdbXnund8EA7dBzZDlKQ5KfS+oEvAhV/NV2x6devVNlMJkerVq2wbt06pXdLRGoVgFm1RY3A7lUHkft+EYybm2LDmW4oR1/regk16Nt4HzJ6n4ThtuYYdG8qwpoO9GOJ/cwXTVj+aroKtOZsskvxYEiSJAwbNkzp3RKRWgVIVu1Te09j9fz9MOYKGAs64kRNJwCdrOvjg04go8NBGG4IxsgpXdCiSxqANL+VV1V8lQXZUcbu+PiA75tG3sW5yYjIMxqdDqDyYiU2vbsHuR+fh/HH1th+ORVAC+v6RriEYS12I2PoJRjujUfq6A6QgtR1DKqRnm5uonLWhGUZeu+pup8R+PONk5dx1noi8ow/hyG70J9D1Aj8kluA3HcLYfymEfJPpqEMTW226RmxH4arTiDj1jozvZNzlk70gP0mLCU60eusoz5puAO1GjEYIvIyXzwI7X2mk6H85wrOY+38vcj9sgrGg+1x1JRgs4tW0ikYkvcjI0PCdQ93QpserZQto954c/SVDjrqU30MhhTEYIh0xV+jX3w5DNlBDYGQJEAAy1Ln4I2jmb/P9H7l2MNQgfTmu5ExuBSGyW1xVWYnBIUE1d07ecJb3z+tZoUmj2h2NBkR+ZE/Ex9mZgJjx3o/EDOZzMdo5/84SQjUQMKQve/iLjyNGgQjLfwgDGlFyMhsgmseSkPjFn3s75PDp5XhrdFXAdJRn9SJwRBRoPB2BmA5fDAMuSznazRpYCh/EASSUIgvrn8LVz13KxL6dwTQwLyJOs2c7VXeCC412lGftIHNZERyqbn2IID7U5gqTfjho30wfnQKuVubI6n0J3yESc7fmJ0NTJzY8DbskKs8bwWXnC9Ml3z1/GZjOZEcOTnmP8QjRgB33GH+mZxsXq4GriQ+1IDC747jvXs2YkLiZrSMKMXAP3bD39YOxzelPVGEBOc7AJzXEDTQ3GZdNm2aeTuSxxJc1v0uWmonPblfLEkXgSvBqgXnCyMPsZmMyBl/Nj/JrY1ytz+FP2q77Hxm2ZlyrH/bPNN77h7LTO9x1rdEowQj4/bCMLwShvuTgbsVyGnjy8zZaq5VVIqz4FKSzMHl2LHuH7ujpIsJCZwvjDwjdKCkpEQAECUlJf4uCmlNdbUQCQlCmP+c139JkhCJiebtlLZ8ef3PTkgwL68rL89xGWu/8vLc27891dXm/WVnm3/KOQd2PvM3qZW4FctsihGEajG46S4xe1ie2PSvXaLqclX9/UiS+WXvOLOynJcnO1veOcvOlnc+XDhml86zGtm79u58B5X8fApIvnp+Mxgiaogv/8DXZnnY2wu+JKn+g9QStDkKDuoGba7u3175XHzAn33lPVEDiJo6n2mCJEyQxJ+CFok/dV0vPn1ikzh3+Ly8c9RQoOos4PDFtfX0PKuRo2s/bZpvgkvSFQZDCmIwRG7zVe1Bbc5qowAhWrYUoqLC9n2OakvqPng9re2S+YC/dOaSMM7dJp7olyd6hu8RR5EgTA4+swaSqHGnhq262lwL5Og4Ggo4XA0gXeXPWkVvaejay7lPvPGPAwU0Xz2/2YGaqCG+Hs5rMgH//GfDfVkA4NQp8+SUtTukWvpTxMfbbpuQYNuvyZPO1k76hQgBlN75EG6M/Q4xsYBhVl+8sm04mlX8hkQcczhiQ4KA5G4H73fecXwcgONO0N7ukBtgndpl9Qlq6FxJkjkJp1LzkxEpiMEQUUMsE1DWfVhaKPkH3jJi7S9/kbf96dP1R+hkZgKHD5uz8GZnm38WFNh2LPUkeZ2TB7wEgajyUxh09gsMxHdIlI7h3k4b8VLGevc/syGeBhxyA0h3BFqSQDnn2hJ0crQXaQxHkxE1xFJ7MH68+Q967f+KlfwD72jEmhx1R+g4S3zoSm1XnVFQ1YeOyPqj8SxewLN4ASI+AdLf3wBihgG5Mj/TFUoEHN7KnB1oSQLlnutp08yBJEd7kYYwGCJyxtvDeRtqfnCmds2H3OHfltouZ0PTT52CSE6GVOuYz6EFWrpQPKmoCLjlFmD2bCAmBjh71sGGMofD16VUwOGNzNlyz7NWmo3knuuxY4FXXgn8VAIUUJiBmkguf09A2RA52ZZra2CWeQFgf9+J6LItGwK2bek1ACwNIA4aDl3nSbZntWclbuA8A9BWhmu1n2sKSMxATaQ2ltqDiRPNP5X6g69EnxFXm1oc9JUpRmtMEMvQZNuGeoEQav2uWCAEeNY/R+1Zib3ZJ8nX1H6uiTzAmiEif/OkZsjN/8Z/zT8K46LDWJMXjMvF5xCFCziBttiIdKRjI/LhYU2VHLGxwMcfKxNY2psPKzFRPf1UAikDtdrPNQUUXz2/GQyRvqnhIeWs+cHCUQduGTUMpcdKkbdgL4yfl8O4vx0OViXbrI+RzmJUwj4YRpowtv1PiH12qnvH4qq8vPp9ddy9Jmq4lnrBc00+4qvnNztQk355a3ZtV8kZsfbEE8DSpbI7cNee6d24tTk2l6ahGgOt60NQhaujd8PQ/zwMd7VCn4ldEBx2tXllvg8nJq3bROjJNfFGJ2iyj+eaAgxrhkifHA1l92fHVmfND07+Gy/87jiMbx+CcV0w1hzrirMixmb3HUMPI6PrERjGRGD4w6mISnBwL8itqVJC7ZohNV4Tch9rj0gBbCZTEIMhsmF52DtKIOfPUTEuPEDKTpZZZ3o37knA3soONutrz/Q+6oFktB+eJL8cjkZBOdPQ8Pna6p5jNV8Tcp1aal1J8xgMKYjBENmQ22HZXn8WP6qprsGu5QeQu+QEjN9F4Ztz3VCJcOv6IJgwoOkeZPQ9A8PEWAyYnIqQCA9awu090JxZs8YcrJw4ARw4AMyZY17urK+TRq8J2cEaPlIQ+wwReYuGpkko3nUSxvm/wLhawuojnXFSdAHQxbq+XfAxZHT6FYbRobh2Siqap1yl3IfXzsxcVGSeJuT06YZzzNQdGda9u7xklRq6JtQAOfOX1c2YTqQCDIZIf1Q8TUL5+XJsXLgbxuUXYPy5LXaVdwHQyrq+CS7i2ta7YUgvh+H+JHQalQwpKMF7BardUbZRI9enJZE71YWKrwm5wJW54ljDRyqiiWCoqKgIM2bMwFdffYXLly+jc+fOeO+999C3b19/F420SEXTJIgagT2fH0Lu+8dg/LYp1p/phnJc+V5LqEHfxvtg6HUShgnNMPi+NIQ1HdjAHr3I3WlJ5Iw8UtE1IQ8oXcPHTtjkI6oPhs6dO4chQ4ZgxIgR+Oqrr9CqVSscOnQIzZo183fR3MOb2/98NfmqA6f3n8Hq+fth/NoE468dcbymI4CO1vVxQSeQ0eEgDNcHY+TDndEyNQ1AmlfK4jJvTWrq52tCClGyho+dsMmXhMrNmDFDDB061KN9lJSUCACipKREoVK5aflyIRIShDD/qTe/EhLMy8n37F2PxETFr0fFhQqR//oOMWtwnujbeLeQYLL5yAhcEhmxW8VrY/PEzysPiBpTjaKfryk+uibkJdXV5usnSbbX0PKSJPP1rK5ueD/Ll9vfhySZX/w+6Iavnt+qH02WlpaGjIwMHDt2DOvXr0d8fDymTJmCBx54wOF7KioqUFFRYf29tLQUiYmJ/h1NxhEW6uSFmjpRI3Bg9WHkvnMUxo2NkHcyDWVoarNNj4j9yLjqBAzjozD0T2mIaBbh0WcGFNaeapunk9MyzQLVwqH1v4uIMD8kpk+fjltvvRXff/89pk2bhn/961+4++677b5nzpw5yMrKqrfcb8GQs5sbMN/chw8H7s0d4A+4cwXnsW7BXuR+UQXjgfY4YrLt1NxKOoVR7X7B9aOqccOAM4htUhGQ58FnAvz7pHmezF/GNAtUi89S43i13kkBoaGhYvDgwTbLHn30UTFo0CCH7ykvLxclJSXWV2FhoX+byfLy7FcZ131lZfmnfN4WgM2DVZerxLcLd4nZw/LEoKa7RBCqbQ4vDOXi2uY/iJdvyBM7lu0TpiqT+Xjj423PQ3y8ps+DXwTg9ykgVVeb//ZlZ5t/Omsas8jOlvf3Mjvbm6UnlfBVM5nqO1C3bdsWaWm2nUdTU1OxfPlyh+8JDw9HeHi4w/U+J3fkxOzZ5rwsgdRc5qh5sKjIvNzV5kE/1ggUbCiE8V8FyM0Lw9oTqSiFbU6f1LBDMKQdQ8bNjXHNQ2lo0qrPlZU5OcAtt9TfaVGRefny5d677g2dM63VsCj9fSLvcXf+MqZZIH/waqilgIkTJ9brQD1t2rR6tUUN8XsHark1Q5bOonL/g1I7S2dKR8cqtzOlhY9rBEoKS8Sqv24RU7rni46hBfWKHyOdERMSvxXv3bNBHN1S5HhH1dVCxMY2fN1jY71z3Rs6Z1qrYVH6+0TqpFQnbAoIvnp+qz4Y+v7770VISIh48cUXxYEDB8RHH30kGjduLD788EPZ+/B7MOTsj3jdV16ef8qpNLlBoJzj9cHokuqKavH9kt3ihevyRHrUThGCSpuPCkGlSI/aKZ4fmSe+X7JbVFfI/GO8Zo2887BmjcfHYKOhc9ZQQKHW0TpKfp9I3Szf3brfVTV/P8krGAzV8vnnn4vu3buL8PBw0bVrV7Fo0SKX3u/3YEgI880rNxgKlLZwpdr+vVgjcHRLkXjvng1iQuK3IkY6U2/XHUMLxJTu+WLVX7eIkkI3vz/PPCPvPDzzjHv7t8fVAFwL/3mzL4m+MM0CCfYZsjF69GiMHj3a38XwTGYmkJVl7hfkTKC0hSvV9q9giv+yE6X4+a/Z2J93HGuLuuDD6ttRgzjr+iiUYGRb80zvhj8lo/3wZADJ8o5DTZyds4a4cD59in1J9MVbCT6J7NBEMBQwnn4aeOcd5/kzAmXKAaWmWPAgxb9lpnfjv0+gbMM2PFD2OgaiCAMB3A3gBczEWxHTET6gNzLusMz0Pkje58k1fDjwwgvytlOKEhOaqm1SVE7ZoT/udsImchGDIVd4OvKm9pQDgO0f9ECcckCpKRZcrBEo3nUSqxf8gtzcKzO934zd+BRPAbB9iCZIRfh7xePAnz8FMofL+xxXDR8OxMYCZ8443iY29soffSVGeClRO6K2GhZO2UFE3uLVRjiVUKTN0dORN7VzbmRl1c83E8ht4Z62/TsZXVIjSeJydGvxVL81okfEvnqbROK8OBnUStT4s4/Mk0823M/Fci6UGuHlbESOFvsMWbAvCZFucDoOBXmcwdLTqTQcTTj4wANAp076aAv3tLbDQYr/GgCAhPH4FCtw5Rr0bbwHhp7mmd6HdDmF0BsNzj/DWxltHX1/LJ58Epg3z7Pvmb3zu2qV41pIy++OaljUnq9Ha/mRiMgtzECtII8iS09HMnHCQUWc2ndabLzhRXEmpKXNeTyCRHEzlou4oOPino4bRPYj34qTe07Zvtmfo5DkjOpKTBSiosL975mruYQstSisYSEileNoMrXwZCSTyWSuEbJXIyCE+b/wadPMIyYC4b9aBf9br7xYic3v7YHxv+dh3NkKP1zqCoG/IggzkI6NaIcjaB1VjrbDOuK5e5PQbWwbSEEO+rj4cxSSnFFdhYXAggXufc/kZGQ+fNjxdeFoHSIidqB2yoORTEoOCVc9R02Bb7whq7nFMtO78d2jMG6MQN5vabiIXjbb9IjYD0P3EzDcEon0h26TP9P71VebH/Amk+NtgoPN2ylN7vfn0CHX9+dKsO3o+8XROkREDIac8qRWwZNASkvcnC/KMtO78YsqGA+m4HB1CoAU6/qW0ikY2v0CwyiBUVM6oW2vLgC6uF6+TZsaDoQA8/pNm5QPDOR+fzp0cH1/egq2iYi8iMGQM57kNtFDkjgXaieqqwS+//deGJeegfGHWHx3MQ01GGzdPAwVGNp8NwwDS2G4uw163toZQSFDPC+jP4NSud+fKVOAV1917Xuml2CbiMjLGAw540luE1cCKa2OjpFZO/Fs63/hzTN3osTBTO+GcY0x7OE6M70rxZ9BqdzvT1iY698zPQTbRES+4NXu2SrhtTxDckbeyJlw0F+zh9fOfZSX515eGZkjtW5HtgCEaC6dFRMSvxXvTt4gjmw6pvQR2aeGWbDlfn9c+Z6p4biIiLyIeYYUpFieAndrb+x1Lk5MNP+nD3iWw8hdHnZ4tjCtXotgw3VOt/ugzz/Q5ZFR6HtnVwSH+aHGy0GeIp/m1ZH7/XHle6aG4yIi8hJf5RliMOQr9h5wAJCc7HyusoICZZvMPEwieWzrCRgXHIRxbTDWFXbED+iLeBQhCPW/SkKSIHnjGNzRUFCq5YAhUI+LiHSPwZCCVBEM2ZOfD4wY4Xw7JTMjm0wuB2CXTl/C+gW7YVx5CcY98dhT0dHmLXfiA3yAu81vr7svQF21E+7U7mmhP5cWykhE5CJfPb/Zgdqf/DEaSGaH54NZHyFnWxKMW6Kw8Vw3VKK/dZMgmDCg6R4Y+pyB4fYYDJh8O6Svm9hvdlNb7UTtvDpyAgiFmhO9jvmCiIjcxmDIn7wxGsjZA15mYPW350OxDMOtvycFH0NGp19huDEU107pipgOtqPCkJmprWzGcoIcN/MnERGRtrCZzJ8sTVbOht7L7W8j5wEvs2nuevwPIa1ikZF+GYb7EtE5IwVSkOT0fZogp8/U2LH+6c9FRERWbCbTA09yGNXlpBZD/PcT7A3rCeN7JtwZ1AqxNafsd3gGUNGsNVb9OgzhzRu7fWiqJTdJZHQ0szsTEelEkL8LoHuZmeaaiPh42+UJCfKbYZw84GuEwPFbH8NVY1Pwl89G4sGatwEANahT0yNJkCQJEe8tCMxACJA/hUV+vrz9eSu7s8lkLsPSpeafzqYTISIit7FmSA087W/j5AEfBCAexzESayDFxmLokBgc6/omErNfVn+HZ6UpHbx4I7uzVjptExEFCAZDauHGaCBRI3Bw7RH8+sw6ZMjY/otFxQh7wLLlcOClh7XT4VkpcoOX4cOBJUvcm5POE+y0TUTkc+xArTHnj5Rg3Vt7kft55e8zvSdiGPKRDx/nK/I3d/PquNJpfdUq32Z3diMHFBFRIPPV85t9hlSuurwamxf9hDnD83F15E+ITW6KW/5vEBbtuwaHqxMRikqERDdFaXhLiLp9gCwkyZyRWOlaDH/JyTEHDSNGAHfcYf6ZnGxe7oyl0zpwJaixqNtpXYn+XK6Q259p40ZlP5eISOfYTKZCh785BuPCX5GbF4a1x1PrzfTeNewQMmxmeu8H5Cz8vRbDw1FpaqdEM5IlyJGTJNKX+ZP8kYSTiIjYTKYGF45fQP7be5G76jKM+5JwoCrFZn1z6Ryui9+HjJHVGPVgeyQNjre/o0Cfo0rpZiS1TWHhj+lZiIhUjHOTKUhtwVBNdQ22Z++D8cOTyP2+GTaVdEM1Qq3rg1GNwVG7Yeh3Dhl3tXRtpne1PeCVFOjBgtJJOImINI5JFwPMsa0nsPrtg8hdE4w1x7rgjEgDkGZd3yHkCAxdjiDjD+EY/lBXRCf1dO+DAnmOqkBvRlIyCScREcnGYMhLLp2+hA0L98C44iJydyf8PtP7lWHdkSjFyLZ7YBhWCcMD7dDh2nYA2vmmcFqtPfLGXG5q40p/JiIiUgSbyRQiagR2ffoLjP8+gdzNlpnew63rg2BC/yZ7YehzGhkTYzBgcipCG4c2sEcv0XJCPz01I2k1YCUiUhD7DCnIWyfzt59PYfVbv8BoFDAe7ozfalrZrE8MLkJGx0Mw3BiKkVO7IqZDc8U+2y1yJihVe0BkOQbAN7l/iIjIbxgMKUipk1l+vhzfvrMHxk9LkburLX4s72KzvjHKMKLVbhiGXkbG/Sqb6T2QEvoF+qg5IiICwGBIUe6eTFEjsPeLQzAuLoLx28bIP9UNl3FlAtMgmHBf2Ie4Nukguma0Q+rzd6h3gtNAG4nFZiQiooDH0WR+cubAWayZvw/Gr0ww/toBx0wdAXS0rm8bVAxDykH8seMGDNn5FoJ/Ow4chPm1Kku9fW8CbSRWII+aIyIin9J9MFR5sRJbFu+F8eNzyN3RCj9c6gqBq63rI3AZ18TuhmHwRWT8MR7dxnaEtHITMP4ZbU2mqYeRWERERG7QXTNZZNNIHFx7BMZ3jsC4MQLritNwEZE2218V8QsM3Y7DcEsk0h9MQ6OYRldWarXvjZ5GYvkCm+mIiLyOzWReMK3/t8j7tTsOVycDSLYubymdwqikX2AYJTDq4Y6I69MZQGf7O3FlMk01NeMwoZ9ytJyegIiI6tFVMLT4lyEAohCKSgxtthuGgSUwTGqNXrd1QVDIEHk70XLfGyb085wSE8USEZGq6KqZ7MHuX2LMLS0x7KFUNG3T1L2dBcKoLDbxuEerTaRERBrlq2ayIK/t2Uvmzp0LSZIwbdo0l98779uhuGlOf/cDIcAcOCQkXGlaqkuSzDlv0tPd/wxvs4zEmjjR/JMPbnlcaSIlIiLN0FQwtHXrVixatAg9evTwXyEsfW+A+gER+94ENi03kRIRkUOaCYYuXryIO++8E++88w6aN/fztBaWvjfx8bbLExLYZySQMT0BEVFA0kwwNHXqVNx000247rrrnG5bUVGB0tJSm5fiMjOBw4fNfYOys80/CwoYCAWyQGgiJSKiejQxmmzZsmXYvn07tm7dKmv7uXPnIisry8ulArMg6w3TExARBSTV1wwVFhbiz3/+Mz788ENERETIes+sWbNQUlJifRUWFnq5lKQbbCIlIgo4qh9av3LlStx8880IrvXftslkgiRJCAoKQkVFhc06e3w1NE939DxEX8/HTkTkI8xA/buRI0fip59+sll27733omvXrpgxY4bTQIi8RO9ZmNlESkQUMFQfDEVGRqJ79+42y5o0aYLY2Nh6y8lHmIWZiIgCiOr7DPmVyWTOOL10qfmnyeTvEvmfyWSuEbLXumpZNm0azxUREWmG6muG7MnPz/f+h+i9GcgRrU5US0RE5ABrhuyxNAPVfehbmoFycvxTLjVgFmYiIgowDIbqYjNQw5iFmYiIAgyDobo4GWfDmIWZiIgCDIOhutgM1DBOVEtERAGGwVBdbAZyjlmYiYgogKg+A7USXMpgaTIBycnmztKOTk3LluamtLAwxcuqKczCTEREXuSrDNSsGaqroWYgi1OngA4d9D2qDLiShXniRPNPBkJERKRBDIbscdQMVBuH2RMREQUEBkOOZGYChw6Zm8Ts4TB7IiKigMBgqCGbNpmbxBzR+zB7IiKiAMBgqCEcZk9ERBTwGAw1hMPsiYiIAh6DoYYw2zIREVHA028wZDIB+fnA0qXmn/Y6QTPbMhERUcDTZzCUk2NOrDhiBHDHHeafycn2h8kz2zIREVFA018G6jVrzPmB6h62pabHUYDDbMtEREQ+5asM1PoKhs6eRVSPHo5npZckc41PQQEDHSIiIj/jdBzesGmT40AIYN4gIiIiHdJXMFRcLG875g0iIiLSDX0FQ23ayNuOeYOIiIh0Q1/B0NVXM28QERER2dBXMMS8QURERFSHvoIhgHmDiIiIyEaIvwvgF5mZwNixzBtEREREOg2GAHPgM3y4v0tBREREfqa/ZjIiIiKiWhgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHRN9cHQ3Llz0b9/f0RGRqJVq1YYN24c9u/f7+9iERERUYBQfTC0fv16TJ06FVu2bMHq1atRXV0Ng8GAsrIyfxeNiIiIAoAkhBD+LoQrTp06hVatWmH9+vW45pprZL2ntLQU0dHRKCkpQVRUlJdLSERERErw1fM7xGt79pKSkhIAQExMjMNtKioqUFFRYf29tLTU6+UiIiIibVJ9M1ltQghMnz4dQ4cORffu3R1uN3fuXERHR1tfiYmJPiwlERERaYmmmsmmTp2KL7/8Et988w0SEhIcbmevZigxMZHNZERERBrCZrI6Hn30UXz22WfYsGFDg4EQAISHhyM8PNxHJSMiIiItU30wJITAo48+ihUrViA/Px8pKSn+LhIREREFENUHQ1OnTkV2djZWrVqFyMhIFBcXAwCio6PRqFEjP5eOiIiItE71fYYkSbK7fPHixbjnnntk7YND64mIiLSHfYZ+p/JYjYiIiDROU0PriYiIiJTGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNc0EwwtWLAAKSkpiIiIQN++fbFx40Z/F4mIiIgCgCaCoY8//hjTpk3D008/jR07diA9PR033HADjh496u+iERERkcZJQgjh70I4M3DgQPTp0wdvv/22dVlqairGjRuHuXPnOn1/aWkpoqOjUVJSgqioKG8WlYiIiBTiq+e36muGKisr8cMPP8BgMNgsNxgM2LRpk59KRURERIEixN8FcOb06dMwmUxo3bq1zfLWrVujuLjY7nsqKipQUVFh/b2kpASAOcIkIiIibbA8t73diKX6YMhCkiSb34UQ9ZZZzJ07F1lZWfWWJyYmeqVsRERE5D1nzpxBdHS01/av+mCoRYsWCA4OrlcLdPLkyXq1RRazZs3C9OnTrb+fP38e7dq1w9GjR716MtWmtLQUiYmJKCws1FVfKR43j1sPeNw8bj0oKSlBUlISYmJivPo5qg+GwsLC0LdvX6xevRo333yzdfnq1asxduxYu+8JDw9HeHh4veXR0dG6+hJZREVF8bh1hMetLzxufdHrcQcFebeLs+qDIQCYPn06Jk2ahH79+mHw4MFYtGgRjh49ioceesjfRSMiIiKN00QwdNttt+HMmTN47rnncOLECXTv3h3/+9//0K5dO38XjYiIiDROE8EQAEyZMgVTpkxx673h4eGYPXu23aazQMbj5nHrAY+bx60HPG7vHrcmki4SEREReYvqky4SEREReRODISIiItI1BkNERESkawyGiIiISNc0GQwtWLAAKSkpiIiIQN++fbFx48YGt1+/fj369u2LiIgItG/fHgsXLqy3zfLly5GWlobw8HCkpaVhxYoV3iq+21w57pycHIwaNQotW7ZEVFQUBg8ejNzcXJttlixZAkmS6r3Ky8u9fSguceW48/Pz7R7Tvn37bLYLtOt9zz332D3ubt26WbfRwvXesGEDxowZg7i4OEiShJUrVzp9TyDc364ed6Dc364ed6Dc364ed6Dc33PnzkX//v0RGRmJVq1aYdy4cdi/f7/T9/niHtdcMPTxxx9j2rRpePrpp7Fjxw6kp6fjhhtuwNGjR+1uX1BQgBtvvBHp6enYsWMH/vrXv+Kxxx7D8uXLrdts3rwZt912GyZNmoQff/wRkyZNwoQJE/Ddd9/56rCccvW4N2zYgFGjRuF///sffvjhB4wYMQJjxozBjh07bLaLiorCiRMnbF4RERG+OCRZXD1ui/3799scU6dOnazrAvF6v/HGGzbHW1hYiJiYGNx6660226n9epeVlaFnz56YP3++rO0D5f529bgD5f529bgttH5/u3rcgXJ/r1+/HlOnTsWWLVuwevVqVFdXw2AwoKyszOF7fHaPC40ZMGCAeOihh2yWde3aVcycOdPu9k899ZTo2rWrzbIHH3xQDBo0yPr7hAkTxPXXX2+zTUZGhrj99tsVKrXnXD1ue9LS0kRWVpb198WLF4vo6GiliugVrh53Xl6eACDOnTvncJ96uN4rVqwQkiSJw4cPW5dp4XrXBkCsWLGiwW0C5f6uTc5x26PF+7s2OccdKPd3be5c70C4v4UQ4uTJkwKAWL9+vcNtfHWPa6pmqLKyEj/88AMMBoPNcoPBgE2bNtl9z+bNm+ttn5GRgW3btqGqqqrBbRzt09fcOe66ampqcOHChXqT3V28eBHt2rVDQkICRo8eXe8/S3/y5Lh79+6Ntm3bYuTIkcjLy7NZp4fr/d577+G6666rl6VdzdfbHYFwfytBi/e3J7R8fyshUO7vkpISAGhwElZf3eOaCoZOnz4Nk8lUb7b61q1b15vV3qK4uNju9tXV1Th9+nSD2zjap6+5c9x1vfrqqygrK8OECROsy7p27YolS5bgs88+w9KlSxEREYEhQ4bgwIEDipbfXe4cd9u2bbFo0SIsX74cOTk56NKlC0aOHIkNGzZYtwn0633ixAl89dVXuP/++22Wq/16uyMQ7m8laPH+dkcg3N+eCpT7WwiB6dOnY+jQoejevbvD7Xx1j2tmOo7aJEmy+V0IUW+Zs+3rLnd1n/7gbhmXLl2KOXPmYNWqVWjVqpV1+aBBgzBo0CDr70OGDEGfPn3wz3/+E2+++aZyBfeQK8fdpUsXdOnSxfr74MGDUVhYiFdeeQXXXHONW/v0F3fLuGTJEjRr1gzjxo2zWa6V6+2qQLm/3aX1+9sVgXR/uytQ7u9HHnkEu3btwjfffON0W1/c45qqGWrRogWCg4PrRXsnT56sFxVatGnTxu72ISEhiI2NbXAbR/v0NXeO2+Ljjz/Gfffdh//+97+47rrrGtw2KCgI/fv3V81/Ep4cd22DBg2yOaZAvt5CCLz//vuYNGkSwsLCGtxWbdfbHYFwf3tCy/e3UrR2f3siUO7vRx99FJ999hny8vKQkJDQ4La+usc1FQyFhYWhb9++WL16tc3y1atX4+qrr7b7nsGDB9fb3mg0ol+/fggNDW1wG0f79DV3jhsw/8d4zz33IDs7GzfddJPTzxFCYOfOnWjbtq3HZVaCu8dd144dO2yOKVCvN2AerXHw4EHcd999Tj9HbdfbHYFwf7tL6/e3UrR2f3tC6/e3EAKPPPIIcnJysG7dOqSkpDh9j8/ucdldrVVi2bJlIjQ0VLz33ntiz549Ytq0aaJJkybWXvUzZ84UkyZNsm7/66+/isaNG4u//OUvYs+ePeK9994ToaGh4tNPP7Vu8+2334rg4GDx97//Xezdu1f8/e9/FyEhIWLLli0+Pz5HXD3u7OxsERISIt566y1x4sQJ6+v8+fPWbebMmSO+/vprcejQIbFjxw5x7733ipCQEPHdd9/5/PgccfW4//GPf4gVK1aIX375Rfz8889i5syZAoBYvny5dZtAvN4Wd911lxg4cKDdfWrhel+4cEHs2LFD7NixQwAQr732mtixY4c4cuSIECJw729XjztQ7m9XjztQ7m9Xj9tC6/f3ww8/LKKjo0V+fr7N9/bSpUvWbfx1j2suGBJCiLfeeku0a9dOhIWFiT59+tgMy5s8ebIYNmyYzfb5+fmid+/eIiwsTCQnJ4u333673j4/+eQT0aVLFxEaGiq6du1qc3OphSvHPWzYMAGg3mvy5MnWbaZNmyaSkpJEWFiYaNmypTAYDGLTpk0+PCJ5XDnul19+WXTo0EFERESI5s2bi6FDh4ovv/yy3j4D7XoLIcT58+dFo0aNxKJFi+zuTwvX2zJ02tH3NlDvb1ePO1Dub1ePO1Dub3e+54Fwf9s7ZgBi8eLF1m38dY9LvxeQiIiISJc01WeIiIiISGkMhoiIiEjXGAwRERGRrjEYIiIiIl1jMERERES6xmCIiIiIdI3BEBEREekagyEiIiLSNQZDREREpGsMhoiIiEjXGAwRkSYtXboUERERKCoqsi67//770aNHD5SUlPixZESkNZybjIg0SQiBXr16IT09HfPnz0dWVhbeffddbNmyBfHx8f4uHhFpSIi/C0BE5A5JkvDiiy9i/PjxiIuLwxtvvIGNGzcyECIil7FmiIg0rU+fPti9ezeMRiOGDRvm7+IQkQaxzxARaVZubi727dsHk8mE1q1b+7s4RKRRrBkiIk3avn07hg8fjrfeegvLli1D48aN8cknn/i7WESkQewzRESac/jwYdx0002YOXMmJk2ahLS0NPTv3x8//PAD+vbt6+/iEZHGsGaIiDTl7NmzGDJkCK655hr861//si4fO3YsKioq8PXXX/uxdESkRQyGiIiISNfYgZqIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESka/8PYAjYcKf1AIYAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_22_2.png"
      }
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own sdg\n",
      "[[4.15847501]\n",
      " [2.79503363]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using SGD\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = (1.0/n)*training_gradient(y, X, theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "# Can you figure out a better way of setting up the contributions to each batch?\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        theta = theta - eta*gradients\n",
    "print(\"theta from own sdg\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81bbaf59",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Same code but now with momentum gradient descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f2ecc922",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.02970225]\n",
      " [3.04000457]]\n",
      "Eigenvalues of Hessian Matrix:[0.28225179 4.44767398]\n",
      "theta from own gd\n",
      "[[4.02930753]\n",
      " [3.04033524]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own sdg with momentum"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "[[4.0292555 ]\n",
      " [3.02413786]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using SGD\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 100\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = (1.0/n)*training_gradient(y, X, theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "change = 0.0\n",
    "delta_momentum = 0.3\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        # calculate update\n",
    "        new_change = eta*gradients+delta_momentum*change\n",
    "        # take a step\n",
    "        theta -= new_change\n",
    "        # save the change\n",
    "        change = new_change\n",
    "print(\"theta from own sdg with momentum\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ddf05caf",
   "metadata": {
    "editable": true
   },
   "source": [
    "## AdaGrad algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/adagrad.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/adagrad.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ac3a77b",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Similar (second order function now) problem but now with AdaGrad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "ec906255",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own AdaGrad\n",
      "[[1.99997644]\n",
      " [3.00011577]\n",
      " [3.99989296]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using AdaGrad and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-8\n",
    "for epoch in range(n_epochs):\n",
    "    Giter = 0.0\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        Giter += gradients*gradients\n",
    "        update = gradients*eta/(delta+np.sqrt(Giter))\n",
    "        theta -= update\n",
    "print(\"theta from own AdaGrad\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b540daef",
   "metadata": {
    "editable": true
   },
   "source": [
    "Running this code we note an almost perfect agreement with the results from matrix inversion."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fbc5639",
   "metadata": {
    "editable": true
   },
   "source": [
    "## RMSProp algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/rmsprop.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/rmsprop.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80820b57",
   "metadata": {
    "editable": true
   },
   "source": [
    "## RMSprop for adaptive learning rate with Stochastic Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "a51a1f9f",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own RMSprop\n",
      "[[2.00211924]\n",
      " [3.00515889]\n",
      " [4.01229102]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Value for parameter rho\n",
    "rho = 0.99\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-8\n",
    "for epoch in range(n_epochs):\n",
    "    Giter = 0.0\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "\t# Accumulated gradient\n",
    "\t# Scaling with rho the new and the previous results\n",
    "        Giter = (rho*Giter+(1-rho)*gradients*gradients)\n",
    "\t# Taking the diagonal only and inverting\n",
    "        update = gradients*eta/(delta+np.sqrt(Giter))\n",
    "\t# Hadamard product\n",
    "        theta -= update\n",
    "print(\"theta from own RMSprop\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1dbb8774",
   "metadata": {
    "editable": true
   },
   "source": [
    "## ADAM algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/adam.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/adam.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53f97773",
   "metadata": {
    "editable": true
   },
   "source": [
    "## And finally [ADAM](https://arxiv.org/pdf/1412.6980.pdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "88a61340",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own ADAM\n",
      "[[2.00013955]\n",
      " [2.99942911]\n",
      " [4.00048158]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Value for parameters beta1 and beta2, see https://arxiv.org/abs/1412.6980\n",
    "beta1 = 0.9\n",
    "beta2 = 0.999\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-7\n",
    "iter = 0\n",
    "for epoch in range(n_epochs):\n",
    "    first_moment = 0.0\n",
    "    second_moment = 0.0\n",
    "    iter += 1\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        # Computing moments first\n",
    "        first_moment = beta1*first_moment + (1-beta1)*gradients\n",
    "        second_moment = beta2*second_moment+(1-beta2)*gradients*gradients\n",
    "        first_term = first_moment/(1.0-beta1**iter)\n",
    "        second_term = second_moment/(1.0-beta2**iter)\n",
    "\t# Scaling with rho the new and the previous results\n",
    "        update = eta*first_term/(np.sqrt(second_term)+delta)\n",
    "        theta -= update\n",
    "print(\"theta from own ADAM\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e34a6cb",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Introducing [JAX](https://jax.readthedocs.io/en/latest/)\n",
    "\n",
    "Presently, instead of using **autograd**, we recommend using [JAX](https://jax.readthedocs.io/en/latest/)\n",
    "\n",
    "**JAX** is Autograd and [XLA (Accelerated Linear Algebra))](https://www.tensorflow.org/xla),\n",
    "brought together for high-performance numerical computing and machine learning research.\n",
    "It provides composable transformations of Python+NumPy programs: differentiate, vectorize, parallelize, Just-In-Time compile to GPU/TPU, and more."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04f74999",
   "metadata": {
    "editable": true
   },
   "source": [
    "### Getting started with Jax, note the way we import numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a5c7d572",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import jax\n",
    "import jax.numpy as jnp\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from jax import grad as jax_grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c584b1d",
   "metadata": {
    "editable": true
   },
   "source": [
    "### A warm-up example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "651b82b4",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/jax/_src/numpy/lax_numpy.py:173: UserWarning: Explicitly requested dtype float64 requested in asarray is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more.\n",
      "  return asarray(x, dtype=self.dtype)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x124b30190>]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABQCUlEQVR4nO3deXxU1eH+8c+dmaxkhZCFECDsS9hBFNkURami4K6tim1tUyOVUmu/WlutbeXXRUstYLRWXLHugDu4sCj7JoussgWSAAnJTNZJMnN/fwxEEQQCydyZyfN+veYFTG7Iw4iZh3POPccwTdNERERExE9sVgcQERGR5kXlQ0RERPxK5UNERET8SuVDRERE/ErlQ0RERPxK5UNERET8SuVDRERE/ErlQ0RERPzKYXWA7/J6veTn5xMbG4thGFbHERERkTNgmiZlZWW0adMGm+3UYxsBVz7y8/PJyMiwOoaIiIichby8PNq2bXvKawKufMTGxgK+8HFxcRanERERkTPhcrnIyMiofx8/lYArH8emWuLi4lQ+REREgsyZLJnQglMRERHxK5UPERER8SuVDxEREfErlQ8RERHxK5UPERER8SuVDxEREfErlQ8RERHxK5UPERER8auA22RMREREmojXA3uXQvlBiEmB9kPBZvd7jAaNfEydOpXBgwcTGxtLcnIy48ePZ9u2bcddM3HiRAzDOO5x/vnnN2poEREROXMz188k9+PJMC0Lnr8S3vyJ78dpWeR+PJmZ62f6NU+DyseiRYvIyclh+fLlLFiwgLq6OsaMGUNFRcVx111++eUUFBTUP95///1GDS0iIiJnzla0nRkHPiHXVn7c87m2CmYc+ARb0Xa/5mnQtMuHH3543K9nzZpFcnIya9asYcSIEfXPR0REkJqa2jgJRURE5Ox5PWRv+Ahs5cxITAAgu9RFbkIcMxLjySlxkl08Hy72+G0K5pzWfDidTgBatmx53PMLFy4kOTmZhIQERo4cyV/+8heSk5NP+nu43W7cbnf9r10u17lE+v6slbW8unofOw+V87fr+jbJ1xAREQk4e5eCK5/so7+ckZhAbkI8HsMgp6SU7FIX4PRdlzncL5HO+m4X0zSZMmUKw4YNIysrq/75sWPH8vLLL/Ppp5/y2GOPsWrVKi6++OLjCsa3TZ06lfj4+PpHRkbG2UY6pRqPl//3wVZeW72fvCOVTfI1REREAk75wfqfjqisAsBjGISZ5tHiceJ1Te2sy8fdd9/Nhg0beOWVV457/sYbb+SKK64gKyuLcePG8cEHH7B9+3bee++9k/4+999/P06ns/6Rl5d3tpFOqXVsBOd3bAXAexsLmuRriIiIBJyYlPqf/qWVb6bCME1qDYPchLiTXtfUzmraZdKkScybN4/FixfTtm3bU16blpZG+/bt2bFjx0k/HhERQURExNnEaLAr+qSx9Oti3t2QT/bITn75miIiIpZqPxTi2jDDVsGGSN/77YyDh9kcEX50DYhBtjfGd52fNGjkwzRN7r77bt566y0+/fRTMjMzT/s5xcXF5OXlkZaWdtYhG8vYrDTsNoNNB1zsKao4/SeIiIgEO5ud3D6XkZsYD0DrujouqKomu9RFTomTGYnx5PYZ49f9PhpUPnJycnjppZeYPXs2sbGxFBYWUlhYSFWVbw6pvLyce++9l2XLlrFnzx4WLlzIuHHjSEpKYsKECU3yB2iIli3CGdrJN/Xy7oZ8i9OIiIj4hzepKxm1BgBXllfWT3tke2PISR+NN6mrX/MYpmmaZ3yxYZz0+VmzZjFx4kSqqqoYP34869ato7S0lLS0NC666CL+9Kc/nfFCUpfLRXx8PE6nk7i4uNN/QgO9tiqP+97cQPfUWD6cPOL0nyAiIhLkiot3cMk7E6gzDP4adwM/6Nyj0Xc4bcj7d4PWfJyup0RFRfHRRx815Lf0uzG9UnjgbYOthWXsPFRO5+QYqyOJiIg0qVcW/pU6w6BjNVx8ywMQ5v8t1b+t2R0slxAdzvAuSYCmXkREpHn4pGQlAN1sfYm0uHhAMywfAFf2aQPAuxsKTjuaIyIiEsy2bn+PnWEmDtNkRN9fWh0HaKbl49JeKYTbbew8VM62g2VWxxEREWky/1vlOzSuR2Uklw4YaHEan2ZZPuIiwxjRtTUA723QhmMiIhKaamsr+cy9B4AOkRcR4bB+ygWaafkAGNfXt++Ipl5ERCRUfb76SY7YbSR4vFx8Xo7Vceo12/IxukcKEQ4bu4sq2JzfNIfZiYiIWOnNbW8B0Lm8NSO6t7M4zTeabfmIiXBwcXffSbvvaupFRERCjLN0D0vxnT6fnnA94Y7AecsPnCQWOHbXyztf5mvqRUREQsp7y/9BrWHQwQ1jzr/B6jjHadbl4+LuybQIt3OgtIq1+0qtjiMiItJo5uZ/DkDrii71R4sEimZdPqLC7Vza03eE8DtfasMxEREJDbt2f8JXdg8O06Rd+k9x2APr7T6w0ljgqn6+qZf3Nhbg8WrqRUREgt/ba2YA0K0igivOO9/iNCdq9uVjWOfWxEeFcbjMzYpdxVbHEREROSeeuhrec20HIKJ6MAPbJVqc6ETNvnyEO2z8oHcqAPM09SIiIkFu+bqnOWw3iPd46dr1Z9hsJz+R3krNvnwAjOvrm3r5YFMhNXVei9OIiIicvTnbXgcgs6wlVw/oZHGak1P5AIZktiI5NgJnVS1Ldhy2Oo6IiMhZKXMd4LNa3xICb91l9GoTZ3Gik1P5AOw2gyv6+LZb110vIiISrD5a8Rhum0FGjcmArOswjMCbcgGVj3rHpl7mf3WQqhqPxWlEREQabs7+hQC0cnbg6v7p1oY5BZWPo/pnJJDRMorKGg+fbD1odRwREZEG2bt3CV/aarGZJtWOm+icHGt1pO+l8nGUYRiM+9Z26yIiIsFk3trpAHSvDOPifhdYnObUVD6+5djUy2fbDuOqrrU4jYiIyJnxeup4p2QzAKazP1ceXccYqFQ+vqV7aixdkmOoqfPy4aZCq+OIiIickVVfPkuB3SDG48WMu4mMltFWRzollY9vMQyDq49utz5vvaZeREQkOMzd8j8AOpcncGX/rhanOT2Vj++4up9vdfDSr4s45Kq2OI2IiMipVZQXsqDmEADFzlH1W0cEMpWP78hoGc2Adgl4TXhnQ4HVcURERE5p/orHqbYZtKkxSWozjqSYCKsjnZbKx0mMP3pv9Nz1ByxOIiIicmrz8j4FINnZngkD2lqc5syofJzED3qnYbcZbNjvZNfhcqvjiIiInNT+/ctZbbgxTJM95Vdwac9UqyOdEZWPk0iKiWBY5yQA5mrhqYiIBKh3Vj8BQLeqMPp1u4CYCIfFic6Mysf3GN//6F0vX+ZjmqbFaURERI7n9dQx98hGAGylWVx9dK+qYKDy8T0u7ZlKZJiN3UUVbNjvtDqOiIjIcdZufIkDdoj2etlTO54RXVtbHemMqXx8j5gIR/3cmaZeREQk0Mzb8hIAXcriGd27K+GO4HlLD56kFjg2hPXOhnw8Xk29iIhIYKisLOKjat9O3CWlwxjfL3BPsD0ZlY9TGNG1NQnRYRwuc7Ps62Kr44iIiADwyYppVNoMUmu9HHFcyqD2iVZHahCVj1MId9j4QW/fTnFztOeHiIgEiHn75gOQ5szg6v4Z2GyGxYkaRuXjNI4NZX24qZCqGo/FaUREpLkrLFjHCrMSgJ3OsfVnkgUTlY/TGNQ+kfSEKMrddXy85aDVcUREpJl7Z9U/MQ2DrpV24lr2p0danNWRGkzl4zRsNqN+z4856zT1IiIi1jG9XuYWrQMg3NWTCf2Da6HpMSofZ+DYf9xF2w9TXO62OI2IiDRXX371P/baIcrrZYPzyvqT2IONyscZ6JwcS+/0eOq8Ju/qpFsREbHIvE0vANC1PJZ+HTuSGh9pcaKzo/Jxho6NfrytqRcREbFAdVUJH1btB8BZOjTo9vb4NpWPMzSubxvsNoP1eaU66VZERPxu4aonKLMZJNd62VlzEZdnBccJtiej8nGGWsdGMLyL76TbOdpuXURE/GzO7vcBSHe14ZKebYmNDLM40dlT+WiAY1Mvc9Yd0Em3IiLiN4cObmKZWQHArtLLmdA/+Pb2+DaVjwa4tGcK0eF29h2pZO2+EqvjiIhIM/HuysfxGgadqwyqwvowvEvwnGB7MiofDRAd7qifY9PCUxER8QfT62Xe4dUARDl7Mq5vG8Lswf32HdzpLXBs6uXdDQXU1HktTiMiIqHuq21v87XdJMJrstF5BeODdGOxb1P5aKChnZJIiYugtLKWz7YdsjqOiIiEuDkb/gtAt4poWrfMoG/beIsTnTuVjway24z6HeXeWrvf4jQiIhLKatxlfFC5D4Dy0vOZ0D8dwwiuE2xPRuXjLFw7oC0An249RElFjcVpREQkVC1a9W+cNoOkOi8by0czYUDwT7mAysdZ6ZYaS682cdR6TN7ZoD0/RESkaczb9S4AGa5UzuuYTNvEaIsTNQ6Vj7N0bPTjzTWaehERkcZXVLSVJV4XAHtLL61/3wkFKh9n6ap+bXDYDL7c72TnoTKr44iISIh5f+XjeAyDTtUGh7z9GNs7zepIjUbl4ywlxUQwqptvk5e31mrPDxERaVzzDq4AoIWzK5dnpRIT4bA4UeNR+TgH1xwdAnt73QE8Xm23LiIijWPrtnlss3kJM002ll7JNSGy0PQYlY9zMLpHMnGRDgqc1SzfVWx1HBERCRFzNzwDQPfyKKJjMhjaKcniRI1L5eMcRDjsjOvrO9xHC09FRKQx1NZW8n75LgCqnIMZ3z8duy349/b4NpWPc3TtQN/UywebCqlw11mcRkREgt3nq2dyxGaQWOdlQ9mlXBtiUy6g8nHO+mckkJnUgqpaDx9sKrQ6joiIBLm5O98GoENZMr3atqZLSqzFiRpfg8rH1KlTGTx4MLGxsSQnJzN+/Hi2bdt23DWmafLwww/Tpk0boqKiGDVqFJs3b27U0IHEMIz6VqqpFxERORelJbtZ5HECsL90dEjt7fFtDSofixYtIicnh+XLl7NgwQLq6uoYM2YMFRUV9df87W9/4/HHH2f69OmsWrWK1NRULr30UsrKQncvjAkD2mIYsGxXMXlHKq2OIyIiQer9Ff+gzjDo4Ia82gH16wpDTYPKx4cffsjEiRPp1asXffv2ZdasWezbt481a9YAvlGPadOm8bvf/Y5rrrmGrKwsnn/+eSorK5k9e3aT/AECQXpCFEM7tQK054eIiJy9uQVfABDv7Mzo7im0bBFucaKmcU5rPpxO39BQy5YtAdi9ezeFhYWMGTOm/pqIiAhGjhzJ0qVLT/p7uN1uXC7XcY9gdP3ADADeWJuHV3t+iIhIA+3Y+SFf2Tw4TJNNpVdy/aDQnHKBcygfpmkyZcoUhg0bRlZWFgCFhb4FlykpKcddm5KSUv+x75o6dSrx8fH1j4yMjLONZKnLeqUSG+Eg70gVK3YfsTqOiIgEmXnrnwKge0UEjqj2jOza2uJETeesy8fdd9/Nhg0beOWVV074mGEcfz+yaZonPHfM/fffj9PprH/k5eWdbSRLRYXbubKvb9/9N7TwVEREGqCutpp3XTsAqHEO4poB6TjsoXtD6ln9ySZNmsS8efP47LPPaNv2m2Gh1NRUgBNGOQ4dOnTCaMgxERERxMXFHfcIVtcd3fPj/Y0FlGvPDxEROUPL1j1Nkd0g3uNlg2tM/ftJqGpQ+TBNk7vvvpu33nqLTz/9lMzMzOM+npmZSWpqKgsWLKh/rqamhkWLFjF06NDGSRzABrRLpOPRPT/e31BgdRwREQkSc7e/AUBHVxI926bQNQT39vi2BpWPnJwcXnrpJWbPnk1sbCyFhYUUFhZSVVUF+KZbJk+ezKOPPsrbb7/Npk2bmDhxItHR0dxyyy1N8gcIJIZh1O94qqkXERE5E07nPj6r860VLCi9mOsGBefax4ZoUPl48skncTqdjBo1irS0tPrHq6++Wn/Nfffdx+TJk7nrrrsYNGgQBw4cYP78+cTGhnaLO+baAW2xGbByzxH2FFWc/hNERKRZ+2j5Y9QYBu3cJnvrBnFVn9Dc2+PbGjztcrLHxIkT668xDIOHH36YgoICqqurWbRoUf3dMM1Banwkw7v4Vihr9ENERE5nbv4SABKdHRnTM4346DCLEzW90F1Ka6FjC4XeXLsfj/b8EBGR77F7z0I22GqxmyZbSsdxfTOYcgGVjyZxac8U4iIdFDirWfp1kdVxREQkQM1bMx2AbpXhhLXoyLDOSRYn8g+VjyYQGWbn6n6+w+ZeXRWc+5aIiEjT8tTV8I5zKwDe0v5cMyAdu+3ke2KFGpWPJnLD0aGz+ZsPUlJRY3EaEREJNCvWP8NBu0Gsx8uXZWPr3zeaA5WPJpKVHkePtDhqPF7mrtdhcyIicrx5214HoFNZS/p3SKdDUguLE/mPykcTMQyDG48eCvTq6v2YphaeioiIT3lZAZ/UHAbgUOkIbhzcfEY9QOWjSY3vn064w8aWAhebDgTnab0iItL45q94jGqbQdsak3zvMMZmpVkdya9UPppQQnQ4l/XynXfz6up9FqcREZFAMXf/ZwAkOdszrl9bosLtFifyL5WPJnbj0QVEc9fnU13rsTiNiIhYLS/vC9YaNdhMk62lV9S/TzQnKh9NbGinVrRNjKKsuo4PNumwORGR5m7e6mN7e4TRKimLPm3jLU7kfyofTcxmM7h+oK/VvrZK262LiDRnXk8d80o2AWA4+3DDoAwMo3ns7fFtKh9+cN2gthgGLNtVzN5iHTYnItJcrdnwPPl2aOH18lXFOCb0T7c6kiVUPvwgPSGqfsvc11dr9ENEpLmas2U2AF3K4hnRsyOJLcItTmQNlQ8/OXYP9+tr8qjzeC1OIyIi/lZZfogF7oMAFJWOaFY7mn6XyoefXNozhZYtwjnocrNo+2Gr44iIiJ99vHIaVTaDNjUmzrBLms0hciej8uEnEQ471xyd23tlpfb8EBFpbubumw9AsqsdNw5u32wOkTsZlQ8/uuk83xDbp1sPUeistjiNiIj4S37+alYabgzT5OvSsdwwuK3VkSyl8uFHnZNjOa9DS7wmvL46z+o4IiLiJ/NWTQOga5WDHp3OJy0+ytpAFlP58LNjox+vrs7D69VhcyIioc70eplX/CUADmcWN53XzuJE1lP58LMf9E4jLtLB/pIqPt9ZZHUcERFpYus2vkSeHaK9XvK913BRt9ZWR7KcyoefRYbZ6zeV0cJTEZHQN++rlwDoUhbHVQN74bDrrVevgAVuHuIbclvw1UEOl7ktTiMiIk2lqvIIH1XnA+B0Xli/51Nzp/Jhge6pcfTLSKDOa/LmWu14KiISqj5dOY1ym0FKrZeEtAlktIy2OlJAUPmwyM3HFp6uysM0tfBURCQUzd37IQBtXOncPKSjxWkCh8qHRa7s04YW4XZ2F1WwbFex1XFERKSRFRauZ7lZCcDh6qu4pEeKxYkCh8qHRVpEOLi6fuGp9vwQEQk1766ahmkYdK2yMaLfaMIdess9Rq+EhW45eq/3h5sKKCrXwlMRkVBher3MPbwGgHBnT27W3h7HUfmwUFZ6PH0zEqj1mLyxRgtPRURCxcYtr7PHDpFek6iWt9EhqYXVkQKKyofFfni0Dc9esU87noqIhIg5G58DoGt5C64/v5+lWQKRyofFruybRmykg31HKvnia+14KiIS7NzVTj6s8m0iaVYN45KeWmj6XSofFosOd3DtAN/phi8v146nIiLBbuGqJyiz2Whd52VAnzsI046mJ9ArEgBuObbj6ZaDHHRVW5xGRETOxVtfvwtAW1caN2lvj5NS+QgAXVNiGdwhEY/X5LVVuu1WRCRYHT60mRVUABAfcyNtE7Wj6cmofASIHw5pD/gOm/No4amISFCat+JxPIZB52qD8cOusjpOwFL5CBCXZ6WSGB1GvrOahdsOWR1HREQayPR6mXdoFQCtKnsxqluyxYkCl8pHgIgMs3PdQN/C09krtPBURCTYbNk+j10Ok3CvSe/ud2G3GVZHClgqHwHklqNTL59uO0TekUqL04iISEO8tuYpALpXRHPThYMtThPYVD4CSGZSC4Z1TsI0fWs/REQkONS6K/jY7fu+nR4xmpS4SIsTBTaVjwDzo/N9ox+vrsrDXeexOI2IiJyJBSv+hdNuo1Wdl6tGTLI6TsBT+Qgwl/RIJi0+kuKKGj7YWGh1HBEROQNv75gLQNeqNlzYNc3iNIFP5SPAOOy2+tNuX1y+1+I0IiJyOsXFO1ht9+3tMTDzdgxDC01PR+UjAN14XgYOm8GavSVszndaHUdERE7G64HdS3j93V9SZxh0rIabR99odaqgoPIRgJJjI7k8KxWAl3Tei4hIQJm5fia5H0+GaVnw/JV86t4FwLUVFcz+/DfMXD/T2oBBQOUjQN12QQcA5qw7gLOq1towIiJSz1a0nRkHPiHXVs62sDC2RITjME2O2D3MOPAJtqLtVkcMeCofAWpwh0S6pcRSVevhrbX7rY4jIiIAXg/ZGz4ip6SUGYkJ/DkpEYD2tbX8NyGOnBIn2Rvm+6Zk5HupfAQowzD40QW+225fXL4X09R5LyIiltu7FFz5ZJe6+EVJKesjfft5fB0eTk5JKdmlTnAd8F0n30vlI4BN6J9OTISDXYcrWPZ1sdVxRESk/GD9TzvW1tX/PMw0yS51nfQ6OZHKRwCLiXBwzYB0AJ5ftsfaMCIiAjEpAJjAX1smAGAzTWoNg9yEuBOuk5NT+Qhwtx7d8XTBVwfZX6LzXkRELNV+KMS14cGklhQ5HDi8XhbtO1C/BiQ3IR7i0n3XyfdS+QhwXVJiubBzK7ymbrsVEbGczU5un8uYFxsDwE1l5SR4vWSXusgpcTIjMZ7cPmPAZrc4aGBT+QgCtx+97fZ/q/ZRXasV1CIiVsqr821/4DBNbnOW1T+f7Y0hJ3003qSuVkULGg6rA8jpje6RQtvEKPaXVDFvfT43DM6wOpKISLNVdGAtOKBfZSypP5oD5Yd8azzaDyVbIx5nRCMfQcBuM+rXfjy3dI9uuxURsciu3YtYZveNdgzPvBsjcwT0vg4yh2uqpQFUPoLEjYMziAyz8VWBi9V7S6yOIyLSLOUufgTTMMiqDOMmneNy1lQ+gkRCdDjj+/luu31u6R5rw4iINEOHD23mY9O3f0dW/E1Eh2vlwtlS+Qgitw/tAMCHmwopdFZbG0ZEpJl55tMHqTUMOlcZ3HrZ3VbHCWoNLh+LFy9m3LhxtGnTBsMwmDNnznEfnzhxIoZhHPc4//zzGytvs9YjLY7zMlvi8Zq8vGKv1XFERJqNMtcB5lb5DozrZLuUdq2iLU4U3BpcPioqKujbty/Tp0//3msuv/xyCgoK6h/vv//+OYWUb0w8Ovoxe8U+3HW67VZExB9mf/Y7Kmw22taYjBt5r9Vxgl6DJ6zGjh3L2LFjT3lNREQEqampZx1Kvt+YnimkxUdS4KzmnS8LuG5gW6sjiYiENHe1k1eKV4PdIKNyICO66v3tXDXJmo+FCxeSnJxM165dufPOOzl06ND3Xut2u3G5XMc95Ps57DZuO7rp2Kwvduu2WxGRJjZv8R8pthu0rvMyfND/YRiG1ZGCXqOXj7Fjx/Lyyy/z6aef8thjj7Fq1Souvvhi3G73Sa+fOnUq8fHx9Y+MDG2gdTo3n+e77XZzvouVu49YHUdEJGR56mp4Nm8BAO2d3bh2cBeLE4WGRi8fN954I1dccQVZWVmMGzeODz74gO3bt/Pee++d9Pr7778fp9NZ/8jLy2vsSCEnITqcawf4plue/WK3xWlERELXJ8v+xn4HxHq8dOtyr26vbSRNfqttWloa7du3Z8eOHSf9eEREBHFxccc95PTuuLADAPO/Osi+Yp12KyLS2Eyvl2d2vAFAF2c6tw3vb3Gi0NHk5aO4uJi8vDzS0tKa+ks1K52TYxnZtTWmCc8v22N1HBGRkLNi3X/YYvcQ4TVp2WoSbRKirI4UMhpcPsrLy1m/fj3r168HYPfu3axfv559+/ZRXl7Ovffey7Jly9izZw8LFy5k3LhxJCUlMWHChMbO3uwdG/14dVUeZdW11oYREQkxz2x8BoAerkR+NHKYxWlCS4PLx+rVq+nfvz/9+/uGn6ZMmUL//v35wx/+gN1uZ+PGjVx99dV07dqV22+/na5du7Js2TJiY2MbPXxzN6JLazq1bkG5u4431uy3Oo6ISMj4auscVhjV2E0TbBMZ0C7B6kghpcErZ0aNGnXK2zs/+uijcwokZ85mM7jjwkwenLOJ55bu4bYLOmC36RYwEZFz9ezqxwHoVd6C64ZfrttrG5nOdgly1wxIJz4qjL3FlXy29fv3UxERkTOTl/cFC+p82xhUVYznB721ZrGxqXwEuehwBzef1w6AZz7fZXEaEZHgN+uLP+E1DHpWOBh93vWE2fVW2dj0ioaA24e2x2EzWL7rCJsOOK2OIyIStIqKtjK32reGrqbkEm45+o87aVwqHyEgLT6KK/r4hgWfWaLRDxGRszV78e+pMQw6VRv07PEjEluEWx0pJKl8hIg7h3cE4N0NBRQ4qyxOIyISfMrLCnjFuQUAR/EQfjqis8WJQpfKR4jISo/n/I4tqfOaPPfFHqvjiIgEnTcW/Z5ym0F6jUli+k/ITGphdaSQpfIRQo6NfsxeuY9yd53FaUREgkeNu4wXDi0HIPFIb+4c0dXiRKFN5SOEXNQtmY6tW1BWXcerq3RAn4jImXp3yR85bDdIqvPijv4xgzskWh0ppKl8hBCbzeCnw3yjH89+vps6j9fiRCIigc/rqWPWvvkApJd04o4RvbSpWBNT+Qgx1wxIp1WLcA6UVvHh5kKr44iIBLzPlv+DPXaTGI+XgrqJjM1KtTpSyFP5CDGRYXZ+dH57AP6zZPcpt8IXEWnuTK+XZ7a/CkAXZxtuvrAfDm0q1uT0CoegWy9oT7jDxpd5pazaU2J1HBGRgLX6y1lsstUR4TXZ7bqFGwdnWB2pWVD5CEFJMRFcO6AtAE8v/triNCIigeu/G54GoIcrgbGDzyc2MsziRM2DykeIunN4JoYBH285xI6DZVbHEREJONu2v8sXVGIzTfaVXMuPL8y0OlKzofIRojq2juGynr5FU08v1pbrIiLf9eyqfwCQVR7Neb0uJjU+0uJEzYfKRwj7+Ujfbbdz1h+g0FltcRoRkcCxf/9yPqwtAuBQ0RX8bERHixM1LyofIax/u0TOy2xJrcdk1he7rY4jIhIwnv/iEbyGQY9KB+0zf0CXlFirIzUrKh8hLvvo6MfLK/bhqq61OI2IiPWKi7bzduU+AKqKLuLnIztZnKj5UfkIcaO6JtM1JYZydx2zV+yzOo6IiOVmL/k9bptBx2pwtJzAoPbaSt3fVD5CnM1m8LMRvlb/7Oe7cdd5LE4kImKdyvJD/K90MwBhxUPIHtVFW6lbQOWjGbiqbxtS4yI5VOZm7rp8q+OIiFjmjcUP4rIZtKkxKQm7kUt7pFgdqVlS+WgGwh02fjLMd/967uKv8Xi15bqIND+17gpeKFwKQKsjWdw5shs2m0Y9rKDy0UzcPKQdcZEOdh2uYL4OnBORZui9zx/hoN2gVZ2XPbU/ZHz/dKsjNVsqH81ETISDiUM7ADBj4U4dOCcizYrXU8esvR8AkFGSyR0jehEZZrc4VfOl8tGMTLwwk6gwO5sOuFiyo8jqOCIifrNo5TR22U1aeL18XfFDbh7SzupIzZrKRzPSskU4N5/n+x9uxmc7LU4jIuIfptfLf7fOBqBraSrXDx1ITITD4lTNm8pHM3PniEzC7AYrdh9hzd4jVscREWlyaze+yJe2WsK9JrtcN9dPQYt1VD6ambT4KCYcXWQ187OvLU4jItL0nv0yF4CeZfFcNnAoiS3CLU4kKh/NUPbIThgGfLL1EFsKXFbHERFpMtt3fsBisxybabLvyLXcOVwHyAUClY9mqGPrGH6QlQbAkws1+iEioWvWir8B0Ks8igt6X0JqfKTFiQRUPpqtX4zybbn+7oZ89hZXWJxGRKTx5eev5oPawwAUF4/VAXIBROWjmcpKj2dk19Z4TY1+iEhoeuHzP+IxDLpV2unSZTyZSS2sjiRHqXw0Y5Mu7gzAm2v3c6C0yuI0IiKNp+TI17xZsRuAmuKR3H30+50EBpWPZmxQh5Zc0LEVtR6TXI1+iEgIeWXRg1TbDDq4ISXjh3RPjbM6knyLykczN2m0718Dr67O46Cr2uI0IiLnrrKyiNmlGwGIKh7MPZd0sziRfJfKRzN3QcdWDGqfSE2dl6cW7bI6jojIOXt70e9x2gzSak1iUn9KVnq81ZHkO1Q+mjnDMJg0ugsAs1fupajcbXEiEZGzV1tbyXMFSwBIOtKTSaN7WJxITkblQxjRJYm+beOprvXynyUa/RCR4PXh53+h0G6QWOclvGU2/dslWh1JTkLlQ3yjHxf7Rj9eXLaXkooaixOJiDSc6fXy7O53AGhf0oG7RvezNpB8L5UPAWB0j2R6psVRWePh2S92Wx1HRKTBlqz6FzvtJtFeL7T4OedltrQ6knwPlQ8Bjo1++O58ee6LPTgray1OJCLSMP/56kUAupWmkH3pUIvTyKmofEi9y3ql0j01ljJ3Hf/9XGs/RCR4rN/4MutttYSZJkb4TxjaqZXVkeQUVD6kns1mcM/RO1+e/WIPpZVa+yEiweE/a6cD0NMVx88uuwzDMCxOJKei8iHHOTb6Ue6u45klWvshIoHv668XsJhyDNMk0v4jjXoEAZUPOY7NZjD5kq4AzPpit+58EZGA959ljwLQqyKSH192g0Y9goDKh5xgTM8UeqTFUVHj0b4fIhLQCgvW8VHdYQDiuZYLNOoRFFQ+5AS+0Q/f2o/nl+7hiEY/RCRA/XfRH6gzDLpW2rn98p9p1CNIqHzISY3pmUKvNhr9EJHA5Szdw9wq39q0NuYYjXoEEZUPOSnD+Gbtx/NL91CsM19EJMA898n9VNkMOrjhljG/sTqONIDKh3yvS3ok0zs9nsoaD08t1uiHiASOqsojvOHcAEDH2qFc0Lm1xYmkIVQ+5HsZhsGvLvWt/Xhh2R4OuaotTiQi4vPyxw9QareRUuvl5ssftjqONJDKh5zSRd2SGdAugepaL9M/22l1HBER6mqrefXw5wBk1fbj/E5pFieShlL5kFMyDIN7L+sGwCsr95F3pNLiRCLSbHk9sHsJb71xO4UOgwSPl9vGTLU6lZwFlQ85raGdkriwcytqPSZPfLLD6jgi0szMXD+T3I8nw7QszOev5LWy9QDc6Kpi5e7Hmbl+pqX5pOFUPuSM/HqMb/TjzbX72XW43OI0ItKc2Iq2M+PAJ+TayvkiKpJtEeFEe714jDpmHPgEW9F2qyNKA6l8yBkZ0C6R0d2T8Zrwz481+iEifuL1kL3hI3JKSpmRmMCfWiUC0LWmhmcS4skpcZK9Yb5vSkaCRoPLx+LFixk3bhxt2rTBMAzmzJlz3MdN0+Thhx+mTZs2REVFMWrUKDZv3txYecVCU8b49v1458t8thS4LE4jIs3C3qXgyie71MU1rnLyw8LANFkfGUlOSSnZpU5wHfBdJ0GjweWjoqKCvn37Mn369JN+/G9/+xuPP/4406dPZ9WqVaSmpnLppZdSVlZ2zmHFWr3axHNFH9+q8sfma5hTRPyg/CAAHmBLRLjvOcMgzDTJLnWdcJ0EhwaXj7Fjx/LnP/+Za6655oSPmabJtGnT+N3vfsc111xDVlYWzz//PJWVlcyePbtRAou1fnVJV2wGfLzlIGv2llgdR0RCXUwKAHNiWtSXjzDTpNYwyE2IO+E6CQ6NuuZj9+7dFBYWMmbMmPrnIiIiGDlyJEuXnnxIzO1243K5jntI4OqcHMN1A9sC8NcPt2KapsWJRCSktR+KMy6N/3d0rceFlVWs3ZNXvwYkNyEe4tKh/VCLg0pDNGr5KCwsBCAl5fgGmpKSUv+x75o6dSrx8fH1j4yMjMaMJE1g8iVdCXfYWLn7CAu3HbY6joiEMpudOxNiqLbZSKzz8O+Dvu852aUuckqczEiMJ7fPGLDZLQ4qDdEkd7t890hj0zS/95jj+++/H6fTWf/Iy8trikjSiNokRDFxaAfAN/rh9Wr0Q0SaxradC9hq8x3t8Mcj1YR962PZ3hhy0kfjTepqTTg5a47G/M1SU1MB3whIWto3290eOnTohNGQYyIiIoiIiGjMGOIHd43qxCsr97G1sIy5Xx5gQv+2VkcSkRBjer08uuh+TIdB78oIhkxaDQUrfYtLY1Kg/VCyNeIRlBp15CMzM5PU1FQWLFhQ/1xNTQ2LFi1i6FDNx4WShOhwfjGqE+C788Vdp3vsRaRxzV/6D9Y63ISZJqM7/Y7oyAjIHA69r/P9qOIRtBpcPsrLy1m/fj3r168HfItM169fz759+zAMg8mTJ/Poo4/y9ttvs2nTJiZOnEh0dDS33HJLY2cXi90xNJOUuAj2l1Qxe8U+q+OISAipcZfxz20vADCwLI3bRl9tcSJpTA0uH6tXr6Z///70798fgClTptC/f3/+8Ic/AHDfffcxefJk7rrrLgYNGsSBAweYP38+sbGxjZtcLBcVbuee0b651n9/upOy6lqLE4lIqPjv+zkccBi0qvPyg/MeI8yuDblDiWEG2L2SLpeL+Ph4nE4ncXFxp/8EsVSdx8uYfy5mV1EFv7y4M1OOngEjInK2Dh3cxJXv30SVzWC4azDTc/6LzXbymxYkcDTk/VtVUs6Jw27jN5f5Csd/luzmoKva4kQiEuz+Pv9uqmwGnaoNbhrzqIpHCFL5kHN2eVYqA9snUlXr4XFtuy4i52D9ptl86C0GoB0/YkS3VIsTSVNQ+ZBzZhgGD/ygBwCvrcnToXMicla8njoeXfk3APq5Ysm5+hcWJ5KmovIhjWJg+0Su6J2GacLUD7ZaHUdEgtCcz+5ni91DC6+X9kn/R7dU3agQqlQ+pNHcd3k3wuwGi7cfZvF2bbsuImeuzHWAf+37AIBuRzrzyyvGnOYzJJipfEijad+qBbee3wGAR9/fgkfbrovIGXryo7s4YjdIrzHp2+sRkuMirY4kTUjlQxrVpIs7ExfpYGthGW+u3W91HBEJArv3LOSViq8BSCodzc8v6mVxImlqKh/SqBJbhDPp4i4APDZ/G5U1dRYnEpFA99dFv6XOMOhZEcaVI39Ni4hGPXZMApDKhzS624a2J6NlFAddbnIX7bI6jogEsMUr/skXVOIwTQz3T7h+UIbVkcQPVD6k0UU47Nw/1nfr7VOLvuZAaZXFiUQkENW4y/jr5mcB6F2STM6VN2DXhmLNgsqHNImxWamcl9kSd52Xv+rWWxE5iZfm38M+OyTWeYlO+A3Du7S2OpL4icqHNAnDMPjDlT0xDJj3ZT5r9h6xOpKIBJDDhzbzVNFKANoW9eO+K4dbnEj8SeVDmkxWejw3DPTN3z7yzld4deutiBw1bcEvqbQZdKw26NHjt3RsHWN1JPEjlQ9pUvde1o2YCAdf7nfy9roDVscRkQCwYfOrzKs7BICjZDy/vLSHxYnE31Q+pEm1jo0g56LOAPzto61UuHXrrUhz5vXUMXXl/wOgr6sFE0b9hPioMItTib+pfEiT+/GwDrRrGc1Bl5snF35tdRwRsdA7ix5kk62OaK8Xd10ONw3WrbXNkcqHNLkIh73+1Nunl+xib3GFxYlExAoV5YVM2/MuAF2LM5k8biwOu96GmiP9Vxe/uKxXCsO7JFFT5+VP735ldRwRscBTH95Fkd0grcZLfPJ9XNg5yepIYhGVD/ELwzB4aFwvHDaDj7cc4tOtB62OJCJ+tHfvEl4s3w5AQtFI7r9qsMWJxEoqH+I3nZNj+PGwTAD++M5XVNd6LE4kIv7yt4X3UWcY9KhwMGzQPaQnRFkdSSyk8iF+NeniziTHRrC3uJL/fr7b6jgi4gdLVj7BYspxmCaeion8bFRnqyOJxVQ+xK9iI8PqF59O/3Qn+Tr3RSSk1bor+Oum/wDQu7Q1d115E5FhdotTidVUPsTvru7XhsEdEqmq9fCX97dYHUdEmtDLC+5h79HzW6LifsPoHilWR5IAoPIhfmcYBn+8KgubAe9tKODzHUVWRxKRJlBUtJWnDi8DIKO4D/dfNdLiRBIoVD7EEj3bxHHr+e0B+P3cTVp8KhKCps2/m3Kbjcxq6Nf7QToktbA6kgQIlQ+xzK8v60br2Ah2F1Xw1KJdVscRkUa0ecubzKspBCCmbAKTLulucSIJJCofYpm4yDB+f2VPAGYs3MmeIu18KhIKTK+Xvyz7M6Zh0McVzZ1X3q1FpnIclQ+x1Lg+aQzr7Nv59PdzN2GaptWRROQcvbPo92y01xHl9dI69tdc1D3Z6kgSYFQ+xFKGYfDI1b0It9tYsqOI9zYWWB1JRM5BRXkh/9w9F4CeJR347YRxFieSQKTyIZbr2DqGX4zqBMAj73xFWXWtxYlE5Gw9+f7R81tqTS7s/yhp8drJVE6k8iEB4RejOtGhVTSHytw8Nn+71XFE5Czs2/c5syt9//92qLiYO0b0tjiRBCqVDwkIkWF2Hrk6C4Dnl+1h3b4SixOJSEM9+sm91BoGPSod/Oyqh3DY9RYjJ6e/GRIwRnRtzYT+6Zgm3P/WRmo9XqsjicgZWrR8Ol/YKnCYJr0SJjEos5XVkSSAqXxIQHnwih4kRoextbCMpxdr7w+RYFBbW8nfNz0FQP+y1kyZcKvFiSTQqXxIQGkVE8Efxvn2/vjXJzvYdbjc4kQiclJeD+xeAhvf4L+v3cLeMEjweLl26D+JjQyzOp0EOIfVAUS+a3y/dN5el8/i7Ye5/62NvHLn+dhshtWxRASYuX4mtqLtZG/4CFz5HLHZeKFtG7DbyKhLYK99KdDP6pgS4DTyIQHHMAz+Mj6LqDA7K3Yf4dXVeVZHEpGjbEXbmXHgE3JtvlHJJxITKLPbSKqrY2OEC1uR7laT01P5kICU0TKaX4/pCsCj72/hkKva4kQigtdD9oaPyCkpZUZiAn9qlchbsb7D4oocDnJKnGRvmO+bkhE5BZUPCVh3XJhJ37bxlFXX8cDb2npdxHJ7l4Irn+xSF3eVlPJaXCym4ZsSzSkpJbvUCa4DvutETkHlQwKW3Wbw1+v6EGY3+HjLQeZ9mW91JJHmrfxg/U+jvd/8YyDMNMkudZ30OpGTUfmQgNY9NY5fXtwFgIfmbeZQmaZfRCwTkwLA5vAwHm+ZAIDdNKk1DHIT4k64TuT7qHxIwMse1YlebeIorazlQU2/iFin/VDK49L4eWoyXsOgY00N6/bk1a8ByU2Ih7h0aD/U6qQS4FQ+JOCF2W384/q+OGwG8786yDsbdPKtiCVsdn4UH4XTbifG4+GFgkMYQHapi5wSJzMS48ntMwZsdquTSoBT+ZCg0CMtjknHpl/mbuJwmdviRCLNz/8++i1fO+owTJPHit3Ee785AiHbG0NO+mi8SV0tTCjBQpuMSdC466JOfLi5kC0FLn4/ZxNP/mgAhqHNx0T8Ye/eL/hn/rtgszG8sgPn/2oO5C3zLS6NSYH2Q8nWiIecIY18SNDwTb/0wWEz+HBzIW+vO2B1JJFmodZdwa8/votKm42uVTZ+de2L2BwOyBwOva/z/ajiIQ2g8iFBpVebeO4ZfWz6ZTMHSqssTiQS+v7+9i1sc3iJ9XgZ23EqnVMSrY4kQU7lQ4LOL0Z1on+7BMrcddz72pd4vbr7RaSpLF75b15x+06YPs99CT++ZKzFiSQUqHxI0HHYbTx+Qz+iwuws21XMrKV7rI4kEpKKirby4KZcAAa6Evm/W/6qQx6lUah8SFDKTGrB767oAcBfP9zK9oNlFicSCS1eTx3/985tlNhttHObXH3BU6TGR1odS0KEyocErR8Oaceobq2pqfPyq1fXU1PnPf0nicgZmfX+z1lhqyLCa9IrfBITBvWwOpKEEJUPCVqGYfC3a/uQEB3G5nwX//xYR3mLNIZNX73O9OIVAPQtyeKBG+6wOJGEGpUPCWrJcZFMndAbgNxFX7N0Z5HFiUSCW0V5Ib9Z9gh1hkFWeQS3X/EvEqLDrY4lIUblQ4Le2N5p3DQ4A9OEX722niMVNVZHEglaf3z7JvY7ILnWS5/0vzGimw6Jk8an8iEh4Q/jetKpdQsOutzc98YGHT4nchbmfvoAH3iLsZsmmZXX8+srR1kdSUKUyoeEhOhwB0/c3J9wu42PtxzkpeV7rY4kElT27l3Co3vnAtDvSAYP3Hwv4Q69RUjTaPS/WQ8//DCGYRz3SE1NbewvI3KCXm3i+e3Y7gD8+b0tbCvU7bciZ6LWXcG9n0yq3z597LDpdGwdY3UsCWFNUmt79epFQUFB/WPjxo1N8WVETvDjCzswqltr3HVeJr2ylqoaj9WRRALe43NvYavdQ5zHS4eoB7hhcEerI0mIa5Ly4XA4SE1NrX+0bt26Kb6MyAkMw+Af1/clKSaC7QfLeWjeJqsjiQS0xSv+xUtVvu3Tu5RcxB9uuEanRUuTa5LysWPHDtq0aUNmZiY33XQTu3bt+t5r3W43LpfruIfIuUiKieCJm/phM+C11ft5c81+qyOJBKSiw1t4cPPTAPQvTWDyDX8mPirM4lTSHDR6+RgyZAgvvPACH330Ef/5z38oLCxk6NChFBcXn/T6qVOnEh8fX//IyMho7EjSDA3tnMQ9o7sC8OCcTezQ9usix/F66rjvXd/26e3dMLLPDPplJFgdS5oJw2ziexIrKiro1KkT9913H1OmTDnh4263G7fbXf9rl8tFRkYGTqeTuLi4powmIc7jNbn92ZV8vrOIzskxzLv7QqLDHVbHEgkIT82dyPTSNUR4TUZ4J/HYj3+m6RY5Jy6Xi/j4+DN6/27y+6hatGhB79692bFjx0k/HhERQVxc3HEPkcZgtxlMu6kfybER7DxUzoNvb9L+HyLAhs2vkVuyGoB+zr48dMuPVTzEr5q8fLjdbrZs2UJaWlpTfymREyTFRPDvm/tjM+CtdQd4dVWe1ZFELFVeVsBvlvu2T+9dHsk9107XOg/xu0YvH/feey+LFi1i9+7drFixguuuuw6Xy8Xtt9/e2F9K5IwM6diKX4/pBsAf5m5mfV6ptYFELGJ6vTz45o3kOwySa72M6fkEvTMSrY4lzVCjl4/9+/dz8803061bN6655hrCw8NZvnw57du3b+wvJXLGfjGyE5f2TKHG4+UXL62hqNx9+k8SCTGvzP8tnxgl2E2TIfbbuH3U+VZHkmaqyRecNlRDFqyINERZdS1XT/+CXUUVnN+xJS/9ZAgOu7aPlhDm9cDepVB+kB1lBfxwWy5VNhvDXJn8M/ttIsPsVieUENKQ928t/ZdmIzYyjKduHcj4GV+wfNcR/vrhVn53RU+rY4k0upnrZ2Ir2k72ho/AlU8N8Ls2qVRFhJNca5IxYJSKh1hK/+yTZqVLSiz/uL4vAP9Zspt3vsy3OJFI47MVbWfGgU/ItZUDMK1lAlsiwon0ejkUZtCyap/FCaW5U/mQZmds7zR+MaoTAPe9sYHN+U6LE4k0Iq+H7A0fkVNSyozEBO5PasmL8b4h8GqbjZwSJ9kb5vumZEQsovIhzdK9Y7oxvEsSVbUe7nx+NYfLtABVQsTepeDKJ7vUxW1OF+/GfnM6bU5JKdmlTnAd8F0nYhGVD2mW7DaD6TcPoGNSC/Kd1fz8xdVU1+pfghICyg8CcMRmY3FUVP3TYaZJdqnrhOtErKDyIc1WfHQYz9w+iLhIB2v3lfLAWxu1A6oEv5gUKg2DnNTW7An3bR4WZprUGga5CXHHXSdiFZUPadY6to5h5g8HYrcZvLXuAE8t/v4TmEWCQW1qX+5JSWZTRAQAtzhdrN2TV78GJDchHuLSof1Qi5NKc6byIc3esC5JPDTOd8vtXz/cyoKvNBwtwcnrqeNXr/yA5VG+4nGdq4z7j5QCkF3qIqfEyYzEeHL7jAGbbrUV66h8iAC3XdCBH53fDtOEX76yjg37S62OJNIgptfLQ6+MY5HdiWGaDPGk8VBt1HHXZHtjyEkfjTepq0UpRXy0yZjIUQ+N68W+I1Us3n6YHz+3irfvupCMltFWxxI5I0+8dStzPPsBuKxmGH/96ZOAt36HU2JSoP1QsjXiIQFAIx8iR4XZbcz84QB6psVRVF7DxFkrKa2ssTqWyGm99P5knqnYAMDI8u786Y4Z2GyGb2olczj0vs73o4qHBAiVD5FviYlwMOuOwaTFR/L14Qp+9uIa3HW6BVcC14dLpvL3Qx8DcL4rlUcmvqyt0yXgqXyIfEdKXCSz7hhMbISDlbuPcO/rG/B6dQuuBJ4V657ldztfxmsYDCiL5YEb36Jli3CrY4mclsqHyEl0T40j99aBOGwG73yZz5/e+0p7gEhA2bbjPSavf4wam0FWRTh3X/EWmcmxVscSOSMqHyLf48LOSfWH0M36Yg/TP91pcSIRn7z9y/j5kvsot9noUmXj1qGzGZyZanUskTOm8iFyCuP7p9fvAfLYgu28uHyvxYmkuSsu2s7PPrqTYruNDLfJdb2e4gf9ulkdS6RBVD5ETuOOCzP55cWdAfjD3E2882W+xYmkuaooL+Rnc69nv8MgudbLVe3+wS3Dz7c6lkiDqXyInIFfXdq1fhOyKa+tZ9H2w1ZHkmamxl3GXa9eyXaHl3iPlzEJD5B9+eVWxxI5KyofImfAMAz+eFUWV/ZJo9Zj8rMXVrP06yKrY0kz4amrYcrsy1nrcBPl9XKx46fcd+0tVscSOWsqHyJnyG4zePyGflzcPRl3nZefPLealbuPWB1LQo3XA7uXwMY3YPcSzLpafj97HItsLhymyUWeCTz0o8kYhmF1UpGzZpgBdv+gy+UiPj4ep9NJXFzc6T9BxM+qaz3c+cJqluwookW4nRd/OoQB7RKtjiVBbub6mdiKtpO94SNwfbOu6MmWScyMjwbT5HL3CB796XTC7Pp3owSehrx/62+wSANFhtn5z22DGNqpFRU1Hm5/dqUOopNzZivazowDn5BrK69/7vXYFr7iAfSqieUvP1HxkNCgv8UiZyEyzM4ztw/ivA4tKauu49b/rmTjfqfVsSRYeT1kb/iInJJSZiQmkJsQx8fRUfypVUsABlZV80q5k3BbQA1Ui5w1lQ+RsxQd7uDZOwYzoF0CzqpabnlmOWv2llgdS4LR3qXgyie71FVfQH6VnIRpGPSsdjOr8BCG64DvOpEQoPIhcg5iIhw8/+PzvjUCsoJlXxdbHUuCTfnB+p+2r60D0wTDwDBNXi44iHGS60SCmcqHyDmKjQzjuR8PZljnJCprPEyctVL7gEjDxKRgArPiY7kvOam+eJiGwTMJccddJxIKVD5EGkF0uINnbh9Ufxvunc+vZsFX+leqnBlP+iD+ktSax1v67prqU13Nuj1531oDEg9x6dB+qMVJRRqHyodII4kMs5P7o4GMzUqlxuMl+6U1vLY6z+pYEuCqKo+Q/eIoXo2NAuDCyipeLjiEHY6uAXEyIzGe3D5jwGa3NqxII1H5EGlE4Q4b/765P9cOaIvHa3LfGxuYuXAnAbadjgSI4uId3Db7IpY7KrGZJgNrW/FkVdhx12R7Y8hJH403qatFKUUan8PqACKhxmG38Y/r+9A6NoLcRV/ztw+3ccjl5g9X9sRm066U4rNrzxLu+uQXHAgziPN4uS7ux0y+dgqG6fXd1VJ+0LfGo/1QsjXiISFG5UOkCRiGwf+N7U7r2Aj+9O5XPLd0D0Xlbh67oS8RDr2RNHervnyZKWsepdRhI6XWy63tHuL2MTf4PmjYIXO4tQFFmpjKh0gT+smwTFrHRvDr19bz7oYCDrnc5N46kJYtwq2OJhaZ99lfeGTPK7jtNjLdcGefpxh3/jCrY4n4ldZ8iDSxq/q2YdbE84iNcLByzxHGz/iCnYfKrI4lTe07B8Th9fDk2z/jwb2v4LYZ9KoM4/6Rb6p4SLOkg+VE/GTHwTJ+/Pwq8o5UERvpYOYPBzC8S2urY0kjO9kBcV7gsVZJvBDnO6flvPIEHrxuLpkpLS1MKtK4dLCcSADqkhLLnLsuZHCHRMqq65g4axUvLtujO2FCzHcPiHMb8NvWreqLR3d3FI/f8bGKhzRrWvMh4ketYiJ46adDuP+tjby19gC/n7uZDfud/Gl8FpFhWoga9I4eEIetnBmJCbgNg3UREayJigTgkvIKHnd7MCL1rVeaN418iPhZhMPOY9f35f/GdsdmwOtr9nNd7lLyjlRaHU3O1bcOiPuh08UzCfH1xePqsnL+ebhYB8SJoPIhYgnDMMge2YkXfzKEli3C2XTAxbjpn7Nw2yGro8m5KD+ICbwe24K3YmPqn3aYJn8uOnLcdSLNmcqHiIUu7JzEO5OG0bdtPKWVtdzx3Coen7+NOo/X6mhyFgrr3PwyOYlHklpRZfN9ew0zTeoMg1wdECdST+VDxGLpCVG8ln0BN5/XDtOEJz7dyY1PL9c0TJCZ8+lfuWHdoyxsEY3t6CLiu0pKWasD4kROoPIhEgAiHHamXtObJ27uT2yEgzV7S/jBE0t458t8q6PJaVRWFvObWZfw+7yXKLHbSKjz4jUMckqc/KLUBeiAOJHv0pJrkQByVd829M9I4J7/rWPtvlImvbKORdsP84dxPYmLDDv9byBNx+s54cyVlRvf4uFVfyQvzHdmz7DKFNr1GUNiVT7ZxR8BzvpPz/bGgA6IEwG0yZhIQKrzeHnikx1M/2wnXhPS4iN59JreXNQt2epozc7JNg3zAM8kJjEzPgqvYdCqzstVsbdwz3UPYD92eOBJyopGPCSUNeT9WyMfIgHIYbcxZUw3hndtzW9e/5I9xZXcMWsV1w1sy++v6El8tEZB/OXYpmHYyskGDjjsPNC6FWsjfbfQtq0xeGjY/zi/R+/vfKIOiBP5Phr5EAlwVTUe/jF/G89+sRvThOTYCB65OovLeqVgGIbV8UKb1wPTssg9umnY6IpKlkdFUnH0TpbRFZU8Xh2BbfJGjWpIs6ft1UVCSFS4nd9f2ZPXf34BHZNacKjMTfZLa7jjuVXsKaqwOl5oO7pp2LDKatJqa/mkRXR98bjV6WLaoSJs2jRMpMFUPkSCxKAOLXn/nuHkXNSJMLvBwm2HGfPPxTw+fxvVtR6r4wWvk5w+e8yevcu4v3Urbk5PpSAsDI4OFIeZJvcdKf3m99CmYSINomkXkSC063A5D83bzJIdRQC0TYziN5d1Y1yfNthsmoo5EydbSApAXBue6DGKJYc2s9tbjPvoSEc3dw3bIsIJM01qDYOcklKyj95Ky+3van2HNHuadhEJcR1bx/DCj8/jyR8OIC0+kv0lVdzzv/VcNeNzPj9aSOTUvnv6LPjuYvllZA3/OfQ5WynBbbPRv9rN9a4ytkWEk6NNw0Qahe52EQlShmEwtncaI7u15tnPd5O7aBebDrj40X9XMLxLEvdd1p3ebeOtjhkYvnvba8aQ406fBehX7eb/Wrei2OH7tphRW8foFuOITK0mt/Azckqc9SMdvh8NZiTGQ/posrXYVKRBNO0iEiKKy91M/2wnLy3fS63H97/1yK6tufvizgzu0NLidNb43qmV6CRyw2vwYLDfYefdbx0CF+718ssSJze7ygi//V1mOjd+z/RMOrl9xuBN6spd/e7y459KJDA15P1b5UMkxOwrrmTax9uZ+2U+Hq/vf+/zMlty90WdGd4lqVndnpv78WRmHPjk+PUZwPSEOJ5KTCCxro4SxzcDwDbTZOG+AyR6jx7sd+1/ofd1vp9r0zCRU1L5EBH2FVfy5KKveWNNXv1ISJfkGG67oD0TBrQlJiIEZ12/XRBatIa3f06uvYIZiQnklJRyk6uce5NbsSIqqv5Twrxeam02LSQVOUcqHyJSr8BZxdOLd/Hqqjwqa3y3kcZEOLhuYFtuGdKOrimxFic8d987vQLkJsTxWXQUX0VE+G6VPTryk1JXR6eaWpZGR9UXjtyEuKNFxek7i0Wbh4mcMW2vLiL10uKjeGhcL351aVfeWrOfF5btZVdRBc8t3cNzS/eQlR7H+H7pXNWvDcmxkVbHPSvf3QIdIM9h5+Gklqz81igHhoFhmjx6uJh9YQ6ePDoiooWkIv6lkQ+RZsbrNfni6yJeWLaXz7Yeou7ouhCbAcO6tOayXimM7p5CanyQFJHvbIE+sqISl93Gushv8ju8XupsNhymSd3RqRUvBjZMsmvCoLL4m99PC0lFzkpATLvMnDmTv//97xQUFNCrVy+mTZvG8OGnnztV+RDxnyMVNby3IZ+31h1g3b7S4z6WlR7H6O4pjOrWmqz0eMLsgbctUE2dl92rPqTbRzcDcGdKa5ZHfzPSkVFbS9ca37bo3zu18sv1kLdCC0lFzpHl5ePVV1/l1ltvZebMmVx44YU89dRTPPPMM3z11Ve0a9fulJ+r8iFijT1FFby3sYCPtxxkfV4p3/7OEB1uZ0C7RM7LbMngDi3plR5HXKT/T9Z1Vtay8YCTlXuOsGr3EdbllTDG8zlPhE8HfCfOjm3bBtMwCDNNflbqrF9s+u27XXIT4pmRGE9O+miyL5nm9z+HSCiyvHwMGTKEAQMG8OSTT9Y/16NHD8aPH8/UqVNP+bkqHyLWKyp38+nWQ3yy5SDLdx3BWVV7wjVtE6PonhpHj7RYOifH0DYxivSEaFrHRmA/hy3ePV6Tg65qDpRWcaCkiu0Hy9haWMbWAhf5zuoTrr80ejv/8T4MUD+qcezOlcFV1ZxXXX1c8QA0tSLSBCwtHzU1NURHR/P6668zYcKE+ufvuece1q9fz6JFi075+SofIoHF6zXZcaiclbuLWbH7CGv3lpy0BBwTZjdIjY8kMTqcuMgwYiMdxEWGERVu59tbjJgmVNbUUVZdh6u6FldVHUcqaih0VdfvT3IybROjGNQ+kfMyW3FeZiKdWkVh/Ks3ubYK32jG902vXD0TKos0tSLSRCy926WoqAiPx0NKSspxz6ekpFBYWHjC9W63G7fbXf9rl8t1wjUiYh2bzaBbaizdUmO59YIOgG/6Y2uhiy0FLrYUlLG7uIIDJVUUuqqp9ZjkHaki70jVWX9Nh80gLSGS9IQoMpNi6JkWS/e0OLqlxp50uie3z2VHNxM7xRbonUaddR4RaVxNdqvtd3dRNE3zpDsrTp06lT/+8Y9NFUNEmkB8dBhDOrZiSMdWxz1f5/FysMxNQWkVzqpaXNW1vpGNqlqqaj0n/D7R4Q7iIh3EHh0hSYgOo01CFMmxkQ2auvEmdSUHyC7+CHDWP5/tjYH00XiTup7tH1VEmoDl0y4nG/nIyMjQtIuINJy2QBexjKXTLuHh4QwcOJAFCxYcVz4WLFjA1VdffcL1ERERRERENHYMEWmObHZthy4SBJpk2mXKlCnceuutDBo0iAsuuICnn36affv2kZ2dffpPFhERkZDWJOXjxhtvpLi4mEceeYSCggKysrJ4//33ad++fVN8OREREQki2l5dREREzllD3r8Db79kERERCWkqHyIiIuJXKh8iIiLiVyofIiIi4lcqHyIiIuJXKh8iIiLiVyofIiIi4lcqHyIiIuJXKh8iIiLiV02yvfq5OLbhqsvlsjiJiIiInKlj79tnsnF6wJWPsrIyADIyMixOIiIiIg1VVlZGfHz8Ka8JuLNdvF4v+fn5xMbGYhiG1XEs53K5yMjIIC8vT2fdNCG9zv6h19l/9Fr7h17nb5imSVlZGW3atMFmO/WqjoAb+bDZbLRt29bqGAEnLi6u2f/F9ge9zv6h19l/9Fr7h15nn9ONeByjBaciIiLiVyofIiIi4lcqHwEuIiKChx56iIiICKujhDS9zv6h19l/9Fr7h17nsxNwC05FREQktGnkQ0RERPxK5UNERET8SuVDRERE/ErlQ0RERPxK5SMIud1u+vXrh2EYrF+/3uo4IWfPnj385Cc/ITMzk6ioKDp16sRDDz1ETU2N1dGC3syZM8nMzCQyMpKBAweyZMkSqyOFlKlTpzJ48GBiY2NJTk5m/PjxbNu2zepYIW/q1KkYhsHkyZOtjhI0VD6C0H333UebNm2sjhGytm7ditfr5amnnmLz5s3885//JDc3lwceeMDqaEHt1VdfZfLkyfzud79j3bp1DB8+nLFjx7Jv3z6ro4WMRYsWkZOTw/Lly1mwYAF1dXWMGTOGiooKq6OFrFWrVvH000/Tp08fq6MEFd1qG2Q++OADpkyZwptvvkmvXr1Yt24d/fr1szpWyPv73//Ok08+ya5du6yOErSGDBnCgAEDePLJJ+uf69GjB+PHj2fq1KkWJgtdhw8fJjk5mUWLFjFixAir44Sc8vJyBgwYwMyZM/nzn/9Mv379mDZtmtWxgoJGPoLIwYMHufPOO3nxxReJjo62Ok6z4nQ6admypdUxglZNTQ1r1qxhzJgxxz0/ZswYli5dalGq0Od0OgH0d7eJ5OTkcMUVV3DJJZdYHSXoBNzBcnJypmkyceJEsrOzGTRoEHv27LE6UrPx9ddf8+9//5vHHnvM6ihBq6ioCI/HQ0pKynHPp6SkUFhYaFGq0GaaJlOmTGHYsGFkZWVZHSfk/O9//2Pt2rWsWrXK6ihBSSMfFnv44YcxDOOUj9WrV/Pvf/8bl8vF/fffb3XkoHWmr/W35efnc/nll3P99dfz05/+1KLkocMwjON+bZrmCc9J47j77rvZsGEDr7zyitVRQk5eXh733HMPL730EpGRkVbHCUpa82GxoqIiioqKTnlNhw4duOmmm3jnnXeO+0bt8Xiw2+388Ic/5Pnnn2/qqEHvTF/rY99M8vPzueiiixgyZAjPPfccNpu6+tmqqakhOjqa119/nQkTJtQ/f88997B+/XoWLVpkYbrQM2nSJObMmcPixYvJzMy0Ok7ImTNnDhMmTMBut9c/5/F4MAwDm82G2+0+7mNyIpWPILFv3z5cLlf9r/Pz87nssst44403GDJkCG3btrUwXeg5cOAAF110EQMHDuSll17SN5JGMGTIEAYOHMjMmTPrn+vZsydXX321Fpw2EtM0mTRpEm+//TYLFy6kS5cuVkcKSWVlZezdu/e45+644w66d+/Ob3/7W01znQGt+QgS7dq1O+7XMTExAHTq1EnFo5Hl5+czatQo2rVrxz/+8Q8OHz5c/7HU1FQLkwW3KVOmcOuttzJo0CAuuOACnn76afbt20d2drbV0UJGTk4Os2fPZu7cucTGxtavp4mPjycqKsridKEjNjb2hILRokULWrVqpeJxhlQ+RL5j/vz57Ny5k507d55Q7DRQePZuvPFGiouLeeSRRygoKCArK4v333+f9u3bWx0tZBy7jXnUqFHHPT9r1iwmTpzo/0Ai30PTLiIiIuJXWkEnIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+9f8Bor1Qczd146QAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_39_2.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def function(x):\n",
    "    return x**2\n",
    "\n",
    "def analytical_gradient(x):\n",
    "    return 2*x\n",
    "\n",
    "def gradient_descent(starting_point, learning_rate, num_iterations, solver=\"analytical\"):\n",
    "    x = starting_point\n",
    "    trajectory_x = [x]\n",
    "    trajectory_y = [function(x)]\n",
    "\n",
    "    if solver == \"analytical\":\n",
    "        grad = analytical_gradient    \n",
    "    elif solver == \"jax\":\n",
    "        grad = jax_grad(function)\n",
    "        x = jnp.float64(x)\n",
    "        learning_rate = jnp.float64(learning_rate)\n",
    "\n",
    "    for _ in range(num_iterations):\n",
    "        \n",
    "        x = x - learning_rate * grad(x)\n",
    "        trajectory_x.append(x)\n",
    "        trajectory_y.append(function(x))\n",
    "\n",
    "    return trajectory_x, trajectory_y\n",
    "\n",
    "x = np.linspace(-5, 5, 100)\n",
    "plt.plot(x, function(x), label=\"f(x)\")\n",
    "\n",
    "descent_x, descent_y = gradient_descent(5, 0.1, 10, solver=\"analytical\")\n",
    "jax_descend_x, jax_descend_y = gradient_descent(5, 0.1, 10, solver=\"jax\")\n",
    "\n",
    "plt.plot(descent_x, descent_y, label=\"Gradient descent\", marker=\"o\")\n",
    "plt.plot(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "245d5858",
   "metadata": {
    "editable": true
   },
   "source": [
    "### A more advanced example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "51bc9058",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x124b96d60>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGdCAYAAAAvwBgXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB8SklEQVR4nO29eZhcZZn3/z21dFUvVb2mO0t3VhAStkgCCQjIJsuIijqMDMgMDvq+/AQGZGZwAiqo+ObngAsygjLziigKuKKOiGFU0BEiISxhSyBJJ+mkt/Ra3dVd+3n/qHrOOd2pqj7L85zzVNf9ua5cF+l0dz0k1efc53t/7++tqKqqgiAIgiAIwgN8Xh+AIAiCIIjqhQoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8I+D1AcqRy+XQ29uLSCQCRVG8Pg5BEARBECZQVRUTExNYvHgxfL7ymofUhUhvby+6urq8PgZBEARBEDbo6elBZ2dn2c+RuhCJRCIA8v8j0WjU49MQBEEQBGGGWCyGrq4u7T5eDqkLEdaOiUajVIgQBEEQRIVhxlZBZlWCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCkIQ9hyfxrWf2IJHOen0U15B6+y5BEARBVBNf3fIWfv1qH9oaQvjrdZ1eH8cVSBEpQzqbQyab8/oYBEEQRJVweCIJAOgZmfL4JO5BhUgJ4skMTtv8e3zkga1IZagYIQiCIMQTS6QBAIOFgqQaoEKkBPuG4xiaTGL7/lF88w+7vT4OQRAEUQVMJDIAgMFYwuOTuAcVIiVIGlSQb/5hN97si3l4GoIgCKIamCBFhGAk03ohksmpuOUnO8gvQhAEQQhDVVVMJvOKyAApIkQikx+dWtJUi2g4gFcPjeM//6fb41MRBEEQ85V4Koucmv/vockksuw38xwqRErAFJGFjWF85pI1AICvPvUW9hye9PJYBEEQxDwlNp3W/junAsOT1dGeoUKkBMmCIhIK+HDZuk6ceXQbUpkc/vWnO5CrkiqVIAiCcA9mVGUMxKgQqWqYWTUU8EFRFGz+0Amor/Fj275R/NerfR6fjiAIgphvMKMqY3CiOnwirhUimzdvhqIouOmmm9x6SUckC/G64aAfANDZXIdLTlwMAOg+HPfsXARBEMT8hBQRgWzbtg0PPPAATjzxRDdejgtGRYRRF8oXJczIShAEQRC8iM1SRKplckZ4ITI5OYkrr7wS//Ef/4Hm5mbRL8cNvRDxax9j6kg1LSMiCIIg3GG2IlItWSLCC5HrrrsO733ve3H++eeLfimusGIjFNT/isKFoiRJke8EQRAEZ1ghoij531dLuqrQ7buPPvooXnzxRWzbts3U5yeTSSSTegUYi3mXZsqKDaaCAHpRQooIQRAEwRtmVu1qrsOBkSlSRJzS09ODG2+8EQ8//DDC4bCpr9m8eTMaGxu1X11dXaKONyfMrGr0iIQL/02KCEEQBMEbpoisWlAPgDwijtm+fTsGBwexbt06BAIBBAIBPPPMM/jGN76BQCCAbPZIVWHTpk0YHx/XfvX09Ig63pwk0keaVUMFdSRJighBEATBGaaIHNXeAKB60lWFtWbOO+88vPrqqzM+9rGPfQzHHnssPv3pT8Pv9x/xNaFQCKFQSNSRLMECzYytmbDWmiFFhCAIguALU0SWt9XDp+jpqu1Rc12FSkVYIRKJRHD88cfP+Fh9fT1aW1uP+LiMFBvf1c2qpIgQBEEQfGGFSFNtDdoaQhicSGIgNv8LEUpWLUGx8d0QKSIEQRCEIFiOSCQcQHs03x2ohnRVoVMzs3n66afdfDlHlB/fJUWEIAiC4AtTRCLhADoiYbyGmNB01b7xabzRG8OCSAgndjYJe525IEWkBKSIEARBEG4yoSkiQa0dI3JyZuveYVzz0Au467e7hL2GGagQKYG2fdegiLCihHJECIIgCJ6oqorJZF4RiYYDaI+w1ow4RSRVeOCu8XtbClAhUoJi47tsgsarHJHx6TRe7x335LUJgiAIccRTWbBJ3Ug4iI6CIiIyXVUrRALelgKuekQqiWLju6wocVsRSaSzePDP+3D/07sRS2Tww49vwOlHtbl6BoIgCEIcrC0T8CkIB32uKCJJKkTkJjmHIqKqKhS2EEAQmWwOP95+EF//77dmGJb2DU/h9KOEvjRBEAThIkajqqIomiIi0iOSyubvc0GPWzNUiJSg+PZd34w/N6olvBmNp/CRB57DWwOTAIAlTbVoCAWwa2ACk8n0HF9NEARBVBJGoyoAdBTGd1m6qt/H/8E3ncn3grxWRMgjUgLWfgkXMasCumIiihcPjOKtgUnU1/jx2UvW4Pf//G6cuqIFADA5a1U0QRAEUdnEDIoIALQ2hGakq4ogVVi1QmZVCVFVtagiEvQrYEWp6CwR5p4+qasJ15yxAqGAHw2FN+hEkgoRgiCI+cTErELE71PQ1pBXRURliaSKJIh7ARUiRWB9M2Dm+K6iKIYRXrGKyFQqX+jU1ejds4ZQ/r9JESEIgphfzG7NABCerirL1AwVIkUwFhmzK0Vt8Z1gRSReUD3qQ7oiwyrlSVJECIIg5hWzFREA6Igww6qo1owcZlUqRIrA2i6KcmTvTJuc8VIRoUKEIAhiXsEUkegMRUTs5EyKzKryYhzdnT2iq2WJiFZEUgVFpEZXRFghMkGtGYIgiHlFbPpIRUR0lghTRMisKiHFjKoM1xSRZEERCRkUEWrNEARBzEuKKSKi01VThQdqUkQkpNjoLsOtdNViikgklH+DklmVIAiCD8OTSSke7op5RIQrImRWlZdyikiooIiIbs2QIkIQBCGWiUQaZ9/1NM7/yjM4NDbt8VlYIXKkIiLMI0KtGXnRNu8WqRLdas2U84hMJjPIse1IBEEQhC0OjExhIplBfyyBf3hwG2IJ71KrY9r4rmFqZla6Km8oWVViWJFRLMLdLbNqsakZ4xuUFSoEQRCEPcam9MJj18AEPvnwi0hnvdmuXqw1IzpdNUmKiLyYUUREB5oVyxEJBXwIFKJdqT1DEAThjJF4CgDQ2VyLuho//mf3ED7z89egqu4rzsUCzUSnq5JHRGI0j0gRs2q48A8mOuK9mCKiKIruEyHDKkEQhCNGp/KFyAlLGnHv374TPgV47IUe3Pf0HlfPoaqq9nAZDc/cRatNzghIV6WpGYlhEzHFzapsakZ0oNmRighgyBIhRYQgCMIRo/G8CtFcX4PzVnfg8+8/DgBw12934c+7h1w7RzyVBbOAGBURQJ+cEaGIpLP5F6VkVQlhikix8d1woTgRrYjEC1Mz9TUzq2PaN0MQBMEHpog01+Vv/ledthzvO2kxAOCPbx927RysLRPwKUfcd0Smq9LSO4nRk1VLKyIip2ayORXTadaamXkG2jdDEATBB+YRaa6r0T521IIGAO4mWBuNqrPTvEVmiWjju1SIyEdZs6q2fVecIjJt+N71IVJECIIgRMAUkZZ6vRBhD3uxafdGeYsZVRki01U1syq1ZuQjUWZ8V8sRyYhTRKYKaodPObIYaii8UckjQhAE4Qy9NXNkIeKmIhIrMrrLYFkiQhSRwn0sSIqIfJRTRHSzqjhFJJ7S/SGzZTpSRAiCIPhgNKsyorWFhz0Xw82KZYgw2iN5RaSfsyKiqiolq8pMsoyBRzerilNEWIZIXehIRUb3iHiXAMg4PJHEuV95Gvf899teH4UgCMIys82qgKE146pHpHRrpqlOTGHEJmYA8ohIiTa+WyxZ1QVFZCpVfGIGmBnz7jXPvHUYew/H8X//Z6+Q+GGCIAhRJNJZ7Vo7QxEJy6WIaAMSmRzXoLWUIUGWpmYkpJwiEnLBrMri24spIlqOiAStmX1DcQD5J4c3emMen4YgCMI8LN494FMQMQwF6IWI+4pItIgiwu45qjpTxXBKyqDqU46IhGjju0XNqnp1Kgpt824xRUSi8d3u4bj238/ucS/8hyAIwilsdLeprmaGF4+pElOprGt7Z8oqIoYHYp75VawQ8fsU+H3KHJ8tFipEilDWrOqiIlJfU8QjIpFZdf+MQmTYw5MQBEFYQx/dnalCNBiKAbeus+YLEX6FUVoSoypAhUhRyo/vio94n9LMqvIqIqqqYt/QlPb757tHZkh9BEEQMsMKkSbD6C6Qb1PUFq79brVnyplVFUXRzKQ8C5GkJAvvACpEimJm+67QqRnNrCqvR2RoMoXJZAaKknd1T6ezeOXgmKdnIgiCMMtooTXTMqsQAYBoLZucccewWi5HBNDvRUmOSrwsm3cBKkSKUt6syv8NMRu28K6YR0SWiHfWllncWIt3HdUGAHh2N7VnCIKoDEaKZIgwmDLhViGit2aOVEQA3RLA8wFYlgwRgAqRojD/h1fJqtrCu6JTM/k36mQyw3WUyyrdhYmZFW31OH1VKwAyrBIEUTkUyxBhuJ2uqrdm5lBEeBYipIjIjRlFJJXNCcvOKKeIMI9INqcK9anMxb6CIrKstQ7vWpVXRF46MIbplNitxARBEDwotmeGwcZo3do3wwqeaKlCJMhfiSezquTohUhpRST/eWJuuuU8InVBP9ik2YSH6ar7hvNG1RVt9VjWWofFjWGksjls3z/q2ZkIgiDMMlrIEWku4hFxUxFRVbWsWRUQ1JohRURuklqyamlFJP95YhSJclMzPp+ChhrvR3hZmNny1nooioLTCqoItWcIgqgEmFm1ub5Ya8a9ULN4KgsmrrvZmqGpGclJZEqP7wb8PgQK4S8J4YpI8Tel1yO8+dHdQiHSVgcABp8IGVYJgpAfFmhWTBFxc2qGqSF+n6KNDc9GL0Q4Ts1Qa0ZeVFXVJKtS+fuaYVWUIlIm4h3wfgPv4ckk4qksfArQ1ZIvRE4rFCI7Do655jQnCIKwy9hUmULExX0zxjCz2dvWGSEB9xx2nwuSIiIfRumrdCFSCDUTpIiwiPe5FJEJjxSR/QV/yOKmWq13ubipFiva6pFTgef3jnhyLoIgCDMk0llNeS4+vuueR2SuiRlATGuGzKoSY6w4i7VmAGPMuxhFRFt6V8SsCniviBhHd42cRu0ZgiAqALbwzu9Tik6qRF3MEdHCzELFjaqAoNbMHMq/m3h/Aslg/9A+BZoXZDYiRqmMaIpIEbMq4H2oGfOHLGutm/FxyhMhCKISMGaIFGuHuKuIlE9VBWhqpuowju6W7NcxRURAqJmqqmWX3gEGRcTj1szy1pmKyMaV+UJkZ/8EhieTrp+LIAjCDKNljKqAu1MzrDUTrS2jiGgPv5SsWhUkyozuMvTFd/wVkWQmp41yFRvfBfR0Va/2zZRqzbQ1hHDswggAYNs+yhMhCEJORsoYVQHD1IwLgWbmFBH+rZmkZlYt/sDtJlSIzIL944SLhJkxwgJkMkbcoHKUGuXSx3fdn05RVdWQqlp/xJ+z4mQglnD1XARBEGbRwsyKZIgAHikiJcLMADGtGd2sWvpe5xZUiMxC27xbRhEJCVREpgpO7tqgH/4SHpWIh2bVwxNJTBVGd5e21B3x5ywuebggfRIEQciGtnm3yMQMoKsTqWxOyHXeiFeKCHlEJIb14Mo5iYUqIswfUiJDBPA20IxFuy9pri36Bm5tCAEAeUQIgpAWZlZtKtGaaagJaKs0RE/OmCpERHhEqBCRF5YNUmp0FxA7NcM27xZbeMdgZlUvPCLGaPditDUUFJFJUkQIgpATTREpUYj4fIpr19m59swAYqdmZBjfLX23q1KsKCJiWjPlM0QAbxWR7uHyhUhrfUERiZMiQhDVSiqTw7Z9I/jz7iGc1NWEC49b6PWRZjBS8Ig01ZW++UfDQUwkMsILkZhHrRnmEQn6vTerUiEyi3KbdxlsakaMWbV8hgige0TiHhQi+1kh0laiECFFhCCqkqlUBr94uRd/2DmIP+8e0pJLw0EfXrvjQgQkGBNlsHj3Uh4RQC8MRE/O6K0ZM4FmHJfeSTS+S4XILJjKES5rVq1iRWQo7xFZ0XakURUAWsmsShBVyRd+9QYe3daj/b6tIYTx6RQS6RwOjExh5YIGD083E23hXZlCJOrS5IypiHeBu2Zqyjx0u4X3pZBkmFJEBFSnjLk27wLeeURUVdUUkWKju4BuVh2fTmtvdIIg5j/b9+ezgz66cSl+df0ZeP7W8/COjnyu0O7BSS+PdgRzBZoBxnRVdxSRYlHzDJqaqTK08d0y/zhCFZFk+c27gL6TIJnJuXqzHzSM7nY1F1dEmmqDYFPHzJlOEMT8JpnJYm/ByH7dOUfhhM5G+HwKjmrPqyC7D8tTiCQz+sK7UmZVQE86FTk1o6qqpmy73ZrRtu9K4BGhQmQWmlm1XGsmwHJEvFFEjKO9bvpE2MRMZ3NdySra51PQUjCsDtEIL0FUBXsG48jmVETDASyMhrWPH1Vox8ikiLCFdz6lfDvEjX0zU6kssoUobbd3zTCzqgxTM96fQDISmiJSzqzK3hTeKCIBv09LXXXTJ7JvDqMqg0Z4CaK62NkfAwAcuzA6Y0cXU0T2SFSIjBjaMr4SoZGAO4UI+95+n1IySRswREbwbM1kqTUjLZWgiAC6YdVNnwgzqi5vLd6WYbDJmREyrBJEVbCrfwIAcOyiyIyPa4XI4ThUVXX9XMXQNu+WMaoCullV5NSM0ahaaskqYGjNiDCrUsS7fJgb3xWoiJiYmgEMMe8uKiL758gQYVBrhiCqi52FQuSYhTMLkWWt9fD7FEwmM+iXZP/UaLywZ6ZMhgigezZiAh/2zGSIAGIDzUgRkRAz47thzazqTY4I4M3iOxbvvrzE6C6DRngJorrQFJFZhUhNwIdlBQV1z2Dc9XMVY3SOzbsMN6Zm2IPnXAq40O27ZFaVDzOKiN6a8U4R8WKEl428tUfCZT9P94iQIkIQ852xqZSmdrBxXSO6YXXC1XOVYq6Fdwx9akbcNVa3ApS/3ocMIZq8Wlxp8ojIi5nxXaaIiBidNe0R8aA1EzMRvAMYF9+RIkIQ8x3Wlulsri06girbCO/IHAvvGG4oItpuszmKAfZgrKpAOsunEEnR1Iy8JE0sAmJtG69yRABDa8YlRSSdzWGqUCRFy8y7A9SaIYhqolRbhrFKshFeNr7bUl/+OhZ1YSAgYVYRMdyPeLVnyKwqMbpHpFxrpuAREaCITJlURNw2qxoLnrkVEVaIUGuGIOY7pYyqDE0RkcQjwqb55lJE9Ij3tLCJn6RpRcRYiPC575BZVWKsKCJJAYpInJmXTCoibnlEWFumrsY/5/IqbQMvtWYIYt7DMkSOWRgt+uerCoXI0GQS41PumetLwcyq5VJVAX1qJqfqLXPemFVEFEXRCgYehUgupyJTCFIjs6qEmDEPCVVEClMzdXN6RPI/JG4pIrFptg+hvJwJ6IrIVCqrmW8Jgph/5HIq3iooIqtLKCINoQAWNeYN7rsPe29YNZsjEg76ECgEnonyiWgKvAlVQs8ScV4UMX8IQIqIlJgxDzFFJJtTkcnynetmbxCzgWZueUTYD2K0du6FzQ2hgPbmJlWEIOYvh8amEU9lUeP3lU1c1tsz3vtEzOaIKIqiT85Mi7nOMnWjnBWAwTNLhAoRyTGjiBjfNDxVkWmD/FcrWaCZPjEztyKiKAra6ildlSDmO8wfsqq9AcEyLVtZDKupTE67Zs41vguIn5xh6oaZyRWei++ME581c7Ta3UDoCTZv3oxTTjkFkUgE7e3tuPTSS7Fr1y6RL+kYMx4R4z8cz8kZ5g+p8fvmrFK1HBHXCpG5V1UbaSHDKkHMe3b2sR0zxdsyDFkUkbFCW8anmGszi943Y2Y4ghHi6E3UJ2Z8ZaPl3UJoIfLMM8/guuuuw9atW/HUU08hk8ngggsuQDwuh3u6GJqLucwbw+fjaxxiaGFmcxhVAWNrxh3zF9u3YEYRAXTD6hC1Zghi3rJzoPzoLkOWLBFjhki5hXcMbd+MKEVEa82YUUQ4tmYkSlUFAHOPtzZ58sknZ/z+wQcfRHt7O7Zv346zzjpL5EvbRmvNmBinSmVyfBWRpLnRXcD9QDNNETHhEQEMI7xUiBDEvGXXHKO7DFaIHBydRiKdNaUAiMCsP4TBFBFR6aoJrTVjxiPC7+FXplRVwGWPyPj4OACgpaXFzZc1TS6nmk6b0xbfcdw3EzcZ7w7oPyCum1VNKiJtWroqtWYIYj6SSGfRPZRXt48tMbrLaK2vQVNdEKoK7D3snSJuds8MI2LIEhEBG981p4jw2zeTlChDBHCxEFFVFTfffDPOOOMMHH/88UU/J5lMIhaLzfjlJkYn8Vxz3Vq6KsclRNro7hwL7wBdEYmnssjmxK/XZq5x860ZMqsSxHxm9+AksjkVjbVBdERDZT9XURR954yH7Rl2PZprdJehKSLCpmYKiogpjwi/h99UtSoi119/PXbs2IFHHnmk5Ods3rwZjY2N2q+uri63jgdgpvHUbPa/CLNqvQlFpMFgGo27kNURszC+C+iO9KEKLUSyORVv9MaEJSoSRKVjjHY3Y3iUwbA6piki5h6ooi4pIl5NzcgwMQO4VIjccMMN+OUvf4k//OEP6OzsLPl5mzZtwvj4uParp6fHjeNpsH9gv0+ZMz00HBRhVjUXZgbkCyH2JnKjPTNhYXwXqPzWzM0/ehl/9Y0/4eYfveKK4kQQlcYuk0ZVBitE9nhYiIwwj4hFRUTY1IyJ4QgGz9YM84iUG7l2E6FmVVVVccMNN+DnP/85nn76aaxYsaLs54dCIYRC5SU+kZg1qgJAmDmYuZpVzcW7MxrCAYzEU64YVvVk1flvVt26dxi/eLkXAPDzlw5BAXDXZSfBb8JlTxDVwpt95aPdZ7NKAkXEbLw7Q/jUjOYRcTnQzERMhZsIPcV1112Hhx9+GD/84Q8RiUTQ39+P/v5+TE9Pi3xZ21iqTrUNvN4oIoAhS8QFRURvzZj0iDBFJJ6sqPZGNqfiC796AwCwblkz/D4FP3vpEP7lJ6SMEIQRsxMzDOYR6R6Kc02ktsJ4IYag0eR1jLWiRSsiplozWo4Ix9ZMNRQi999/P8bHx3H22Wdj0aJF2q/HHntM5MvaxpYiwtGsasUjArg7wjthMdCMmVXTWdW10DUe/OiFHrzRF0M0HMADV63DvX/7znwx8uIh3PKTHVSMEASA0XgKgxP5tqvZQmRJUy1qg36ksjn0jHrzMMra2A0mr2Oip2asKSL8WjOymVWFt2YqiaSN6pSrImJhagZwb9+MqqqWx3fDQT/qa/yIp7IYnkyZ/jovGZ9O4+7f5pN/bzr/HWhtCOGvTlgEVQX+8dGX8NMXD6Im4MPmD53g8UkJwlveLrRXOptrtQeiufD5FKxcUI/Xe2PYPTiJFWV204hiUmt/my1E3JmacTvQLJmRyyMixykkQY93n7s6lUER0ffNiE1XjaeyYEKA2dYMYGjPVIhh9d7fvY3heAqrFtTjqtOWaR9/74mLcM/la6EowCPPH0DPyJSHpyQI7+kdyysaXc11lr7O650z7Bprtnhyb2rGG7NqVU3NVAp67r8ZRYSN73qviIj2iLB496BfsWRuYobVSoh533N4Et99dh8A4LOXrDniSeGSExfjtJWtAIBfv9rn9vEIQioOFQqRRU1hS1/X2VwLABiIJbifyQxsIMBsIcIUkXgqK8TXYu2eQx6RqsCKIsJuyF7liADueURihraMlQVJbN9MJSy++9Kv30Qmp+LcY9tx9jHtRT/nvScuAgD8egcVIkR10zeeL0SWNNVa+jrdxO7Nw8mk5UJEV4B5X2cz2RwyBak5bOqew39qhgoRCdFT7kyYVYP83hQMy1MzLnlEmOISMWnwYrQVFJERyRWRXf0T+P3OQQR8Cm577+qSn3fRcQvhU4BXD41j/7C8ixsJQjS9Y3lFY1GjxUJES1x2/+Ekk81pCrZZj0hNwKepFbyVZ+O9w8w9h6tZtZrGdysNO/06vkvvrOWIRNxSRKatje4ytCwRydNVt+8fBQBsWNmi9bCL0doQwumr2gBQe4aobphHZLHF1oyX+UJsqShg/hoL6KoI7ywRYyFiThHhmKwqWaCZHKeQBBZOVjGKCMsRcbE1Y4WWQmtmSHKz6o6DYwCAEzub5vxcas8QhLEQsaaIsNUPXjycTBZa33mvm5VCRMzkDHuIrfH74DMRlihk1wwVIvKRtCBXaUvvOCoiUylrikhDoTCQvTUje7rqKwfzW6FP6myc83MvPG4h/D4Fr/fGsG+I2jNE9RFPZhArXBMWNVpTRNjqh9F4CjmXM3msGlUZoiZn2L3DbHtERGuGPCISwgoRK3G7PKdmmHRYb1ERca01Y1ERqQSz6nQqi7cKOzPMKCIt9TU4fRVNzxDVCzOqRsIB07unGM2FaPVMThUWm14KqxkiDFH7ZrQHXxP3G0DQ0jsqROTDSoWqL73jo4hkcyqm06w1Y9Ij4rJZ1ezmXQbrB49I7BF5o28c2ZyKtoaQ6ae7S6g9Q1QxhwpGVasTM0D+xseuW263Z7RUVZuKCO/CycroLkBTM1WDpUAzjv06AFoRApiv2N0e37X69GMsRGSNRn+lR2/LmB1NvmDNQgR8Ct7oi2HvYe8WeBGEF/SxDBGLbRmGvpnb3UIkblMREbVvJmFhpQhgyBHhGfFOHhH5sBTxzqZmOCkiU4UfEp9i/o3JvCTiWzPW9swwmAybU4GxKTlVEStGVUZzfQ3edVR+euYJas8QVYZdoyqjxaMRXvutGTEekaSFJauAoTXD4eE3LdmuGTlOIQkJCwuIeCsi8ZTuDzH7ZF5b8JJMczTMFsOuIhL0+9BUl/8aWUd4dxSMqid2zW1UNcKmZ/6L2jNEldE7nm/N2C1EWJaI24nLTBGJWC1EQqKmZszfbwBBrRlSROTDytQMb0WE/ZDUWZhvry28gVOZnNDWR0zziFhfXKdfdOQzrI5Pp7G3MPlykgVFBAAuXLMQQb+Cnf0Tnu3NIAgvsJshwvDKO6Y97Fm4xgL6dW+C804vKwq88fN4tGaS5BGRF1s5IpwUkamUtYkZYKapVaQqMqFNzVhf1tzqUT/YDK8dyqshnc21mlxslsa6IM4otGd+Q+0ZooroG7eXqspg03RuFyLM4yHN1IxVRSSoT8043WxPZlWJ0cZ3TZlVOSsiKeuKiLGSnk6JK0RiWo6IdUWkTeLJmVcK/hCragjj/DUdAIBn9wxzOhFByI2qqroiYrMQafFIJbWbI6Ilq05znprJ2JuaUVUgnXVYiFCyqrwkLCgieo4IL7OqtVRVAFAURWvPiC1EWMS7dUVES1KUsDWzozAxc6KJILNibFiRzxN58cAotzFugpCZkXgKyUwOigJ0NIZsfQ/PWjN2p2YEKSJ6XIQ1syrgvD1DZlWJsbR9l6NMBljfvMtg7RlRrZlEOqvJePY8IoWYdwkVETYxc1JXk62vX7WgHm0NISQzOc30ShDzGbbsrq0hZCkm3YgWdOhyu9bq5l2GvmtGVGvGmkcEcG5Y1ZbekSIiH9Yi3nWZjMlcTpjSzKrWfkjYOVg8PG/YU4CiAA0W1BqGHvMulyJyeCKJ3vEEFAU4fok9RURRFGxY0QIA2ErtGaIK6B13NroLeLdvhj3sWS9EClMzvAPNMtYUEUVRNAWDVyFCioiEJNPm57p5VqeAcXxXLkWE/fA1hAKmFjPNRlazKlNDjlrQYPnCZGTDynwh8pfuER7HIgip0f0h9iZmAP3hZHTK3X0zkzbNqkwJTmVyXFuwWqCZSUUEMGaJODsHFSISY0URqfH7wOI+ePhEmKJhxSMCALU1Yj0iWry7DaMqoI/vymZWZYvurASZFYP5RLbvH9X6rgQxX+lzmCEC5AMBgfxai3HOBtBy6IFm1h72jA8qPNdpaIFmFlpcvLJEyKwqMdpct4kKVVEUrkl32sI7iz8kmllVlCIyzcLM7KkGXq79LofuD7HXlmEc3d6A5rogptNZ8okQ855DDuPdgfzNr7HW/aBDdo21qoD6fXpLhOd11mqgGcBv8R0pIhKjvTFMVqhalggHuc6pIjIlSBHRJ2bsKSKNdfrCKLfXfpdCVVU9UdWhIuLzKTh1BWvPkE+EmN+wPTN2Ft4ZafVgms7u+C6gP/DxmpI0fi+zgWaAYUjCaWuGds3IixVFBNALlgQPRcSmR0TED4gRvTVjTxFhTz6qyn/8zS4HR6cxEk8h6FewelHE8fdj7Zm/7CWfCDG/YVMzixwWIi0ut2xVVcWkTbMqYPDipfi1X7XcKkuKCKfWjAUbghvIcQoJyOZULSTG9Fw3x22IdqdmRHtEYlqqqj1FJBTwaz/EY9NytGdYkNmxC6O2RxCNMMPqC/tGkCGfCDFPSWdzGJxgHhH7rRlAzxJxa6x/KpUFS1mwalYF9Ac+ntOJCW04woZZ1UEhksnmwMRp8ohIRsrwD2v2jSFGEbFYiATlbs0AQFPha8em3DOmlUNvyzjzhzCOXRhFNBxAPJXF670xLt+TIGRjIJZATgWCfgVt9fbCzBgtLObdpWk61pZRlJmrMcxSK2A6kXkLrTwM8dg3Y0xlJY+IZBhbG2b7ZkwR4To1Y9Gsyn6oRLdm7JpVAb2IcdMhX47Xe/kWIn7yiRBVgHHHjJ1RfiNavlDcHY+INjFjYbu5EREJ1kmLEe8AEOKw48z40E2FiGQwqSvgUxAwWYiEOa5l1iLeLfQLARcUEYetGQBoKhhWxyQpRPYNTQEAjmpv4PY9ySdCzHd6OUzMMNyeprM7McMQoYh4NTWTzOb/HxQlf7+TASpEClhdyQzwVUQSFsLUjNQWWjmixnd5KCJNtfmLzviU9x6RdDaHvkI6ZFdzHbfvy3wiz+8bQVaS6SCC4AkzqjqdmAH0oEO3WjN2M0QYIh74EnbuORxaM9rort9nSx0SARUiBexVp/w8InYc1ABQWyiGhJlVeXhE6uTxiPSOTSOn5n+gF0Sc9bmNrFkURUMogIlEBm/2kU+EmH9oiohDoypgGN91uTXjVBHh2QJPOrjnOFFEjIWILMhzEo+xo4iEOU7N2JkpB8RIhkZi086SVQE9S0SG1syBkXxbZmlLHdengYDfh/XLmwFQ3DsxP+njsGeG4fb4rt3Nu4w6AdOJthQRLUfEfiHCzKqy+EMAKkQ0tHh3C9VpOMhPEUnYVURYa0ZYxLuzZFVAzxKRwaxqLER4o/tEyLBKzD8OFVozixt5tGb0QsSNoMNJh4WItlxUQKCZPY8Ih9YMFSLyoY9SuduvA/Jz3cxXYMVBDRh6l8KW3hUUEUfju/mLjgytmZ6Rgj9ERCFi8InIkiJLELzgqYg01+WvCTnVHaWUKSIRSRQRVVUND79W7jkcWjMFsyoVIhKitUY8UEQShjeV1YAtbXxXgCKSzanak4TdZFVA94iMSxBo1lNQREQUIicsaURt0I+xqTTeHpzk/v0JD9m2DVi0CFiwQP+1aFH+41XAVCqjPUjw8IgE/T7tuuBGzLvT1gzv8d1UNqcFrLmdI5Ikj4i8WNm8y2Cf69TAZPx6qx4RXTLkH59u3DQZcTK+K1GgmcjWTNDvw9quJgDAC/vJJ1LxbNsGtLbm5xxPPRVT/f0IDQ3hgqEhhIaGMNXfD3zwg16f0hXYxEwkFHDkFzPi5gjvhNNChPN0ovHh1VqOiHOPCGvNyJKqClAhomHPrMonRyRp6NlZDQoSsQOBwSZmwkGfIxlPRrNqV4tzebkYzLC6fd+okO9PuEB3N3DqqcCGDZgaGUEIgOLzo2XJaviWnYQ/LTsJvuVr0br8nehJeH1Yd+A5McNg6azDLozw6gvv5BjfZfcbRbGmTPBozchoVrWvt88z7IzvhjltQrQ7MQMYJUP+ikhMM6o6ewIymlVVVfVsdn18Oq0ZZnlmiBhZtyxfiLywnwqRiiMeB771LeDzn0dqYgKrAByqqUXDiRdgwSkfQCDafsSXnK0A1zzxJm46/2jLm7MrCZ7+EIY+OeNGa8ZpoBm/zChgpifRyvVwvppV5+9PjkVsBZpxSla1M0/OqDOM7/K+yeuju87eJk0FY1oqk0MindNGjt2G+UPaGmpsS7RzcfKyZihKXnkZnEigPcLvCZIQSHc38P73I/faa/gsgC/XNSGy7hIsOfkS+MP5BN7s1Diy8VFozX2fHzVtS/HAH/fiiVf78KUPnoB3v2OBd/8PAmETM4s4TMwwWhrca804nZqpDfJuzdgLsOSRrMrMqrJs3gWoENHQPSLWFRHHHhEbOwe0MxRu6jk1b4DisU2WwSPMDADqa/wI+BRkcirGplOorRHTFpkLVoh0ClJDgHzeyjEdEezsn8D2faO4+IRFwl6L4EA8Djz4IHDnnUgNDGABgMwx78Lii2+EL5R/n6SHDyL2/M8w+fofgOzM9uLv3uzHZx9/HQdHp/H333keH1i7GHe87zg0F5725wt9hdbMEq6tmUIh4mprxlmgGb/WTOHh0+L1mueuGTKrSoiuSlhXRBIOx3f11oyNrZCGipp3loge7+6sEFEURYp01Z5RcUZVI8wnQu2ZCuDXv0buhhvwmYEB1PsD8J9/LRZcugm+UB2S/bsx+LMvoff/fhKTO7YcUYQMDAzg3GM7sOVTZ+GaM1bApwC/eLkX//v72+ddzH/vONszI6I1I78iwnu5qD6lae0WzLM1Q2ZVCdFT7ux4RPiYVe0oIkG/D0F/vh3DO11VX3jnXDhrlGByRuTEjJH1y/J5IlSISEw8Dvz7vwO33ILPAfhy00K0ffRuRNddAgAYf+5H6P/ezZh++zlAzSEIYHrVKqgNDVAVBerAANrb856R+lAAn71kDX72yXehvsaP5/eN4D/+tNe7/zcB6Jt3+SkibN/MkAvju85bM0wR4ePF0zyJVhURLq0Z+cyq8pzEYzTzkI2VzE4VEWZ2tfqmZIQFbeDl1ZoB5EhXPVAIMxNdiDDD6uuHxoUl3hIOiMeB227LKyH79+Peleux6Op7EFp4FLJT4xj48e0Y++P3ADWHOgC3LliARDyO8O7dwMQEkMsB7UcaV9d2NeH29x0HAPjKll14o3d+7BxSVRX9rBDhaFZtdVER0QLNbD5U1XIONEvabMdz3TVDhYh8aG8MG+EyjgPNbBRBRkTsQQD4bN5lMMOql6FmmkdE0Oguo7O5Fh3REDI5Fa8cHBP6WoRFWBFyzz14N4Cvn3A+mj78WfhC9UgcfAN9370Rib3bEQRw69KlmNiyBV/q74evzlzxetn6TrxnTQfSWRU3PfYS1yVpXhFLZLSHnIVR/oqIG2ZVNjXjONCMc46I1XZ8iMN+MypEJMZOMaDniDhURGwUQUZ4/5Aw9NaMc0XE61CzbE7FoVF3FBFFUbT2zHZqz8jD4CDwoQ8hd889OAvAjo2Xoe2vboLi82Py1acw8MgmZCeGEFUUTD35JL7U3Q3fe94D+MxfExRFweYPnYC2hhq8NTCJu3+7S9z/j0swNaSpLsh14o15REanUkI9NclMFqls/vreYHPEmj3spbMq0lkOu8WcekScmFVZxDt5ROTDVqAZM6tyUkTsjO8C4hbfaa0ZHh4Rj0PNBmIJpLI5BHwKV8NdKbQ8kX2UsCoFg4PAJZcgs2ULlkPBG+d9As3v/nsAwPjWH2P4iXuAXBZRnw/DU1MIXHihpQLESFtDCF/+8IkAgP/8n248u3uI2/+GF7AMEZ5qCAA0F64JqgqMTYlTRZgaAgD1NgPNjNdmHipX0uaSU2rNzHPsjO+GJAg0A4Dawjl4KyITHBbeMbxefMeMqkuaa+G3mF5rBy1hdf8oLcDzGlaEbNuGVl8A0+/7J0TXfwAAMPK7BzD2zEMAgK5gEMPxOAJh5zfc81Z34G9PXQoA+Kcfv6J5FCqRfgFGVQAI+H1aMSKyPcP+7kMBHwI2VYB88Fj+v3k88Nm95vOYmtGSVUkRkQ87u2a0pXecIt6tLNwzwttIxdAVER5m1byqEvNIEelxaWKGsXpRFLVBP2KJDHYfpgV4nmEsQvxBhD54K+rXnA01m8HhX96FiRd+CQA4o60N+yYnuRQhjM+8dzU6m2vRN57AT188yO37ug2bmFkoQElscSFLZNKhURXIt9zqOLbAbSsimkckB1W194CTJEVEXpiqYeUfhxUtqUzO0VOvnrJnVxHhm/rHEGFWHfPIrCpy624xZizAo70z3mAsQoIhhP/6c6g76lTk0kkM/uyLmHrzGQDAGUuW4Jm+Pvhq+IaQ1YcC+MSZKwEA3312X8UqY6IUEcBoWBU3wut08y6DZ6iZ3Ws+U+xVVVc2rEKtGYlhBiQr/zjGatZJz86ug5rBO/WPoZlVeYzvehxopi27E5iqOhs92Ix8Iq5jLEJqalF72edRu/ydyKWmMfjjO5DYux31AG7buBHP7NsHX0BMyPSH13WiIRTA3sNx/E+FekX6Y0wREVCIuDDCq2WIONwFVFvDTxGxk1uV/3z9/mS3PcOMuxRoJiEpG4WI8U2RclCI2J0pZzDJkOeooKqqiDGPyDyYmnErzMwIM6zS5IzLGIqQtnAD6j5yJ8JdxyOXjGPgsc8i2fMqoj4fxrZswZ1//rOwIgTIR4r/9bpOAMBDz+4T9joiEauI5AuRIRdaM3bj3Rn6glEOrRkbSd7A7ELE3j0nTYqIvLBCImShSgz4FDDfoxPzkPOpGb6pf0C+6mcjdXxzRDzyiLg0umuELcDbPzyFwxPi0yMJzCxC6hpRd/n/QWjxMchOxzDwyK1I9e7MT8bE4whYHM21y9+dtgwA8PtdgzgwPCX89XjTp8W78y9EWurzrRmRG3j11oyz0WOe04l2IxsURdEKCLuFCHvotnKvE408J/EYO30zRVG4jFMlbIwOG9HNqs7n2xls867fp2gz9E5gyaqTyQyXOXwrTKeyWiHgZiHCFuABwHZqz4jHWIRE2lB/xZdR07ES2clRDPxwE1IDe/QihKMpdS5WLmjAWe9YAFUFvvfcPtdelwfxZEZTRkWYVd1pzeSvrw0OlV02nTjFozXjIMRSzxKx2ZohRURe7C4C4pF0l3SqiGhubn6KCJuYiYQDUBTn467GLBK3J2fYsrtIOKB5VdxCzxOh9oxQDEXIgqZFaLjy3xBs7URmfBD9P/w00kP7PSlCGB87fTkA4LEXeipqlJf5QyKhgOPWRjHcaM3om3cdKiKsBc5TEbFxzXf68EuFiMTYXQTEI+bdqUeEZ++SMcFxdBfIZwawFo/boWZuj+4aOXVFPmH1eQo2EwNbXrdhQ74IaVuG+iu/jEBjO9LDB9H/g1uQGe31tAgBgHe/YwGWt9ZhIpHBz1865MkZ7NA/Ls6oCrizgTfOyaxaV8NvOtHu0jvA+eK7JJlV5SWVsT6+C/BJutPDbRzmiHA0q7LWTLSW31NQk0eTM14YVRkbVrQCAF47NK6pTAQnjMvr9u1D6+Jj0HDFZgQaWpAa2Iv+H/5rPrLd4yIEAHw+BVedthxAvj1jNwPCbfoEFyJtbHxX4AbeCU7juzyXi9qNeDd+jd3WDJlVJUabmrHamuGQdKeH2zhTRHiO72qtmRC/VgZLV3V78d0BlzNEjCxsDGNFWz1yKsW9c2XW8rqvrjoVjZd/Cf7aKJKHdmLgkU3ITY1JUYQwLlvfiboaP94amMRze4a9Po4p+gUaVQFdERmbTiMjyDsW5zQ1U8fxgc9OkjfDcWvG5r1OJPKcxGNSNpJVgZlJd3bRwm1sKiLsB4Tn+G6MY5gZgxlW3Z6ccTvMbDYbV+bbM5Vy85GeeBy45RatCHn5pAux4EO3wRcMY2rPNgw8dhtyybhURQiQb3N+6OQlAIDvb93v8WnMITJVFQCaC9N0qiquZasVIg6vZfpQgHOPj5MQS6etGfKISEommwMLPbRsVmXVqQOPiBMHNQCEBQSa8frhNeJVqFnPiPuju0Y2rsy3Z7buJUXEMWyD7n334SwAr77rCrRedEN+g+6OLTj80y9CTSelK0IYl5+S3z/zh12D3FcyiEBkhgiQn8pjRnZRDyiTvJJVOUa8O2nHO1Xh7T50i0Sek3iIMSrXrlnVWWvGmUeE5w4EBi8504gXoWaqqhpSVcVv3S0GK0Re7x33LEdlXmDcoOsL4M2LbkDTGVcAAMaefRTDv/kGoOa4Lq/jzXGLo1jSVItEOoc/V0DSqmiPCGBY/yDousC27zqemuH4wOekHc92ktl9+KVkVUkxpqLaL0ScKyJOA834Ts3weYowwsyqbt6MhyZTmE5noSj5zbte0BENYyX5RJzR3Q1s3JjPCKmNIvWRLyJy0oVQc1kM//abGP/TwwDELK/jiaIoOH91OwDgqTcGPD7N3Gjx7lGRhQi7LojxjnGLeOeYYO3kmu/0nkNmVUlJZvNvLEXJp6Vage/UjM2IdwFTM2IUEfbk455Zlakhi6Jh24oTDzYUVBHyiViEjeeedhpS3d1obFuG+r/7KsJLT0AuOYXBn34Rky//BgDQFYkIWV7Hm/esWQgA+N3OAS29WEYS6aw2ViuqNQPo3jFRigi31gzHB76kg2u+UxU+aWOdiWjkOYmHGMPMrIZ3OR2lUlXV9kpoRlhAjkg8xb8Q8cKsemgs7w/pdHHZXTGYYXVrNxUipmF+kBtuwGcGBtC66hS0fvQuBJsWIj3ah/7v/xMSe18AAHSFQtg7NCR0bwwvTl3RgkgogKHJFF7uGfP6OCUZKKgh4aBP+9kVgejrAnuocmq85zmd6OSa7+ThV1VV3axKrRm5YB4RO9n7jsNlDF/ndHw3mclxe8Jiscg8WzOaWdXFQkQbP2zyVqrXfSIx8omYYXAQuPjivB8ECu497W/Q8uHPwheqQ2L/DvR/72akh3vyG3TXrMG+WAwByZUQRk3Ah7OPlb8906cZVWu5pCuXQmS+UC6naoWD02sZr+nEbE7VfBq2ChHt4df6PceJH1Ik8pzEQ5yMMzltzRjfTLbNqobeJ68RXl6xyEaYWXXcRbOqZrYT2OM2A/OJqCqwrZt8IiWJx4GvfAU47jhkXnwRbaF6JD78WTSf9XdQFB8mXv4NBn70OeQSE6gB8ht0X31V+nbMbN6zpgMA8NQb/R6fpDT9Lv3s6PlC/K8LccOoLa/tu04VEWNLxe3WTMqQ1UKKiGQ4K0Qc9usKX+dTgKDf3lOH8c3Ma4R3UohZVQ8vcgsmL4t0/Ztl4yo2xkvtmaKwVsw//zNuGxpCtGMVGq6+B3VHnYpcOomhJ+7ByG+/CeQyqAEw4eIGXd6cfcwCBHwK9hyOY+/hSa+PU5Q+waO7DF0R4e8dY/4Qv09xPK7KK8Ha+PDpdmsm7WAwQyTynMRDUll78e6AM5kMMGSIBPy25U+fT+Hq6Ab4GbyMGKdm3Iq4dutiagbWnnmOCpGZMEPqKacUWjHAN066EAs+ehcCzA/y8L8g/upTAIAuvx/xeBw1dd76fpwQDQe198N/vylne8atIl4zq4pQRLSJGfvXVwYvsyrbth70K/BbHI4A+Cgifp+91xYFFSLQK0s7c9VOWzMJhwvvGDxn3AFd0owIMKtmc6pW6IimX3AypBU2FhbgvdEXc7U9JTUsJfWGG/CZAwfQVFOH6fffkg8pC9Rg6u2/oO+hm5Ae3AugMJ47NYVABRchDL09I2ch0ic43p0h0qzKvG4RDss764J8lt4ZHz7t4OThV0ajKuBSIXLfffdhxYoVCIfDWLduHf70pz+58bKmYQYeO/84jlszDjNEGDxT/wDDkwTHQiQc9Gt/X26EmmWyOQxO5JdpyaCItEfDWLkg7xOhbbzQskFy992HdwO4a9E70Pixb6B+9VlQsxmM/uFBHP7ZnVCTcQQB3LZxY0WM55rl/EIhsn3/qNClb3Zxq4hnLVsRxbl+HXPudQvX5K9d0+msI0XX6bZ1Jw+/SQkzRAAXCpHHHnsMN910E2677Ta89NJLOPPMM3HxxRfjwIEDol/aNHw8Is4UEW79Sy6jZVmtOONZiADuhpoNTaaQzanw+xRty6fX6HHvVdyeYYbUtWuReu01NEPBjlM/hIVX/huCTQuRGetH/w8/jdjzPwWgIgJg8sknceef/1wR47lmWdJUi+MWR5FTgd/vHPT6OEfgukdEiCLC74GKPeypKp8AS9uKiJPWjAP1XyTCT/PVr34V11xzDT7+8Y9j9erV+PrXv46uri7cf//9ol/aNI4KEYdxu/ryI16KiPOWBzOqAvneKk/0UDPxhQiTljsiIWn6oadVe7CZ0ZAai6Guvhnhv/k8ms/5Byj+AOI7/4Te796IVO8uAMAZ0SjGpqdRc+GFFWlKnYvzV8vZnklnczhcUGlEe0T01Q8p5DgHvLFrGY88pFrDNdrJA58WYGlbEbH/8JvOyrdnBhBciKRSKWzfvh0XXHDBjI9fcMEFePbZZ4/4/GQyiVgsNuOXGzCzqicpd9rCOz57EKZTzldps90MtUE/ApwrZz1LRHy6ar8LezKssqEQbPZmf8zVhFkp6O4GTj1VM6R+beV6LPqHf0ftipORSycw/OS/Y+gXX4aajKMOhVbM8DB8ksa184D5RP709hDX7dlOGZxIQlXz7eqWOrGtsGihEMmpwCSHzbZGmNfNabw7AAT8Pq19P+Xg30oLM7PtEbH/8JuSMFUVEFyIDA0NIZvNoqOjY8bHOzo60N9/5Pz85s2b0djYqP3q6uoSeTwNJ3IVL7Oq49aMNuPOQRER4A9huJmuagxkkoX2SBhHtTdAVfM3n6rAGNO+fz/q/UFMnve/0H7ZHfDXNSI12I3+hz6FyVeeBAB0BYMY37Jl3rViisGW4E2ns1KpZCwIsKMxBJ9gNTEc9Gt+Cd4+kUnOW8R5tMB1FdypImK/NVOVZtXZY1OqqhYdpdq0aRPGx8e1Xz09PW4cDykuZlVn47tOWzO8Uv8AY7w7/90sbm7g7ZcoQ8TIeYWlZ7KObXJlcBB4//u1mPbG1k60/t1XEV3/fgBAbNvj6CukpPoA3Lp6dX5pXYXmg1hFURSceXQbALnGurUiPupOES8q1Iz3zqxaDus09N1iTj0i9qdmggE5WtUMoY8bbW1t8Pv9R6gfg4ODR6gkABAKhRAKuW8qdOYRcbZrRnNQc1NEnBciIhURN82qMmWIGHnP6g58+5m9+MPOQaSzOemMY9zo7gbOOw+Z7m6sBDB64nuw4Pz/DV8wjGx8FENPfB2JvdsBABEAQ088MW+9IOXYuLIVj27rkcrA7HZbs6kuiP5YgvsDSlxbVcHnoYrHglF9z4z7UzNaa0aya47Q09TU1GDdunV46qmnZnz8qaeewumnny7ypS3BI+I95VAR4eYR4aCIiEhVZWjpqi74I5i8LJsi8s6lzWitr0EskcHz8zHufXAQuOoq4KSTkOruRgPLBrn4RviCYUx3v4jeB2/QihDNkHrxxVVXhAD6JNVrh8YRS8iRL+N2Ea+HmvG9LkxwvpaFOcQkJJ0qIkEOrZlq8ogAwM0334z//M//xHe+8x28+eab+NSnPoUDBw7g2muvFf3SpvF0fDfNVxHhMb6rbasU6BFxZ2pGTkXE71O09oxs0xKOYcvqHn4YZ05MoHHhUWidlQ0y+KPbkYuPIQjg1g0b5r0hdS4WNoaxoq0eOYn2EHmhiAD8rwvcWzOaR8S+F09vxzv0iDgJNLNZBIlCuBPsIx/5CIaHh/GFL3wBfX19OP744/HEE09g2bJlol/aNFrEux2PiIPqNP91nALNeCoi88CsmsuphohqecyqjPNXd+BHLxzEU28M4Pb3rRG63dQ1DK2YVgC5tRej/bz/BSUQRGasH4d/dZc2lhtVFAz/5jdV4wWZi40rW9A9FMfWvcM4b/WRbWu3cStVlSHquqD73fhcy/i0ZpxFNszH1owrlvRPfvKT+OQnP+nGS9lCS1Z1sn3XZo6ILtPJE2im91Ur1yMyHE8hnVWhKEB7RI4wMyNnHr0A4aAPh8am8WbfBNYsjnp9JPvE48CDDwJ33onUwAAigRAiF16HhuPPBQBMvfUchp74OtRkHH4AG+vr8fTQEAJVrILMZuPKVjzyfA+27pVNEXHJrFonxqzK+6EqzMGLpweaeTg1I5lZVa6yyCOcjDQ5bc1wU0TYD4j0UzPuBJqxC+mChpCUZtDaGj/OOGoBgApvzxh3xQwMINq8GK1X3Y2G48+Fmsti9A/fweGffwlqMo4aAFNf/jL+Z2yMipBZbFiR94m83jvuipG7HNmcigGXVyM01orZwMu7NVPHdXzX3vWVfV0yk7McNZ+WVBGR6zQe4SR/nxUiqWzOViqg05lyhja+y0ER4W3wMtLkUqAZG92VzR9i5AK29OzNIzN1KgKWklrYFfOVlevQ/vdfQ037CmQnRzHw6G2IPf8zAEAn25h7yy3APM8GsYPRJ/KCx3uIhiaTrq9GEOUR4W285+HFY/cbuwMKzA6gqrqab/W1q86sWgnwiHgH9P6bFZzOlDN4SIYM3k8RRliyaiKdE5okKevEjJFzjm2HogCvHYppPfmKYXAQuOQSLSX1lVMuRfuHPwdfqB6JntfR991/RLLntfyyutWrsX+ebMwVycZC6q7XwWbM5O3magRNKRXUmuGl7vLw4iUctuONX2e1PVO1UzOVACsg7CWrGt4UNnwiTmfKGXU1fFZUA2ILkYaaANi1LSZQgpYxVXU2CyIhnLy0GQDw35XUnilszc1s24ZWfwBTF9+IlnM/DsXnx8Qrv8XAo7chGx9FDQrL6l57bd5szBWJthCx29tCRE9Vda+IZ60ZntcEVVURLzyYNYSCXL4nj7ymhMN2vLGtYtUSoJtV5ZqaoUIEQNpBlRjwKdqN1Y55SF+AxGnpneSBZj6fYsgMEFeIMI9IR1ReRQTQd41sqZRCZHAQuOgipLq7EamNovYjX0LDie+Bmsti5L8fwMiT9wK5DKKKkm/FVGFAmV1YIfJ6b8xTn0jPSL4Q6Wx2T8ES0ZpJZnLI5tgWcU6KSNB5gnXSYTteURTb3kRZk1XpCgG9SgzZUETybwr741ROHdSM2pr813NRRDiPvM1GDzVzQxGpjEJk695hacKsisL2xWzYgMxbb6GpaRFarvoKwl3HIZeYxOBPPo+J7b8EkN8VM0ytGMt0RMNY2VYP1eM8kZ7RKQDA0hb31EQRgWbsgQrgs/QO4NSaYR4RB+14PUvE2jnSDu51IpHrNB7htG/mJEvE6Uw5ozbIrzWjrc7mtChqNqIc8kZk3TMzm1ULGrCyrR7prIo/vnXY6+MUJx4HbrstPxmzbx9aF70DrR+9C8HmRUiP9aPv+/+MRPeLqAdw25o1+V0xNBVjiw2sPeNh3PuBkXwh0uWBIsLTO8auY3U1fm6L+1gh4qQ141QRAQwbeG0qIuQRkRAn23cBvTpN2PCI8Fp6xzNHZJLliHB6ipiNPjkjRgFQVdX1QCYnaO2Z1yVsz7Ai5J57CpMx69F4+f+Bv74Jyf7d6P/+PyMzchA1AMa2bMGdr75KfhAHaIZVCQqRpS3uFSINoYBmjOXVlhLRYuaxXFTziPBQRKgQmT9oBh7bLmYHrZkMn0AzY+Kf1dny2Yg0qwKGFEVBrZnx6bRW4MnuEQGAi09YBAB48vV+jMTF7+Axzawi5KUT3oP2D38Wvpowpvdux8Ajm5CbGkMNgIl4nFJSOXBaQRF5oy8m7OejHLmcioOj+SK+y8VCRFEU7umqIq5jPMyqWoilE0XEZmsmSTki8uK4NeMg6S7JSRFhX5/NqbbGiBnZnKq1d3gZvGbTUp9/Yh4WdNNl/pCW+hrHf69ucFJnI05Y0ohUJodHnj/g9XHyzCpCdpz6YbT91Y1QfH5Mvvo7DP70C1BT07oplfwgXGiPhrFyQd4n8rwHeSKDE0mkMjn4fYrramIT5z1ULA8pyrHFXMumEznkiDi5Ntl9+NXNqnLd+uU6jUc4zd/XPSJ2xnf5BpoBQCJlvxCJG5Y5iZiaAYDWQiEyEk8K+f5aPHUFqCFA/mnw709fDgD4wdb9yDgoJLlgKELOAvDamR9F8zkfAwCMb/0xhp/4GpDLkilVEBs99ImwtsySploEXH5qbqzj6x1j5u9oLZ/RXcAwncghR8RRa8bmPYeSVSXGuSJif9+MPjXj7Mk96PchUOixTqXtb4ZkBq+gX3HcLipFayGtcXhSrCJSCf4QxiUnLkJLfQ16xxPeRr7PKEIUvHHe/0Lj6ZcDAEaf/i7GnnkIAHBGWxuZUgUhQyHipj+E0cR5rJ9lkkQ4KiI8I965tGYo0Gz+4GTXDOCwNcNJEQH4GFbjBoOXqI2wolszlZCqOptw0I+/PbULAPDdZ/d5cwgW2X7PPThL8WHnxf+I6Pr3AwCGt9yH2F9+AgA4Y8kSPNPXR6ZUQWxYkTesvtkXmzGC6gY9bGLGxdFdhrb4jntrhp8iEuagiCS5mFXtPfymtNFhuW79cp3GI9KOzar2ZLJsTtV2BTh5UzJ4yIaa01zQxAwAtDWwQkRMa6YSFREA+OjGZfD7FPylewRv9sXcfXFjZLsvgF3v/xctqGzov76KyZeeAFAoQvbtg4/2xQijIxpGZ3Mtcirw0oFRV19bL0TcV0R4m1WFtGYcPuypqsplv5jtqRmH9zpRyHUaj+DWmrH4pjCOgDmR6Rh8FBEWiSzuRtNSn2/NjAhqzegZIvLGuxdjUWMtLjpuIQDgITdVEVaEbNuGVn8QiQ/eivpjz4SaTePwL/5/xF//PQAqQtxk/bJ89P8L+9wtRLxszfAONYtN5x+qIjzHdwsPe6lszpaXK51VwXajOgo003JE7LVmZNtILtdpPCLpVBFhxiGro1SGwkU6RUTQxAygt2biqayQxXeVqogA0Eyrj798SGjgm4axCAmGEP7rz6HuqFORSycx+NMvYvqt5wBQEeI265bn2zPb97usiIy6H2bG4B3zPpEUp4gA9q6zxsKBj0fE7q4ZuW79cp3GA1RV5egRsaeI1Ph9XJL/eKT+adsqOfZVZxMNBxD05/9/RfhEBsYrI1W1GKcsb8bqRVEk0jk8tq1H7IsZi5CaWtRe9nnULn8ncqlpDP74DiS6XwRARYgXMEXkpQOjrk1RJdJZDMTy7VJPzKp1nFszBUUkWsvvfRsK+MCsc3YKEWPopROfhh6iSWbVeQHzaABOChFnrRlexiEeqX9xzmuzi6EoiqaK8G7PTCTSmCj8P1TK+K4RRVFw9enLAADf37pfW9rFHUMR0hZuQN1H7kS463jkEpMYeOyzSPa8mo9s37iRihAPeEdHBJFQAPFUFjv7J1x5zYMFNSQSCmhFgZs01fLdQTVR8IhEOG3eBfI/n04WjBqv+U6GAZzmiFAhIhlpw9OG24Fm7E3kdPMug0fqnxtmVQBoLfhEhjgbVgcK/pBoOCAsB0U0H1i7BE11QRwcncZPtgtQRYxFSKQVdVd+GaHFxyA7NY6BR29Dqncnoj5fPrL9z3+mIsQD/D4F79R8Iu4Em7Gtu10tdcIm5soR5W5WZYoI36KqzsHiOx5hZoDe1rGsiFBrRk5SGQ6FiOYRsaeI8BjdBfik/sUF7GcoRmuDGEVE94dUllHVSDjox7XvXgUA+OJ/valNMjjGuEF32zYsaO1Ew0fvQk3bMmQmhjDwyCakBvYg6vNhmCLbPUczrLrkEzng4eguYPSI8DKrMo8I32tZmIMi4vSaH7apiPBK8uZN1V9lWIXo9yna0iWr2G/N8J3pri28uZ2YVVkhwjMEqBit9WJGePsq2B9i5BNnrsT6Zc2YTGbwTz96xXmLZvYG3cXHoOHKf0Mg2o70cA/6v/8vSA8d0IsQCirzHFaIuGVY9XJiBtADzWKJjOP3u6qqWo5IhLPfzUlrJqntFnO60sO6IqKqquH15br1y3UaD3BqVAWctGZYdcy3NeNEEZlwSRFhI7y8zar9FTwxY8TvU/DVv1mL+ho/nt83ggf+uNf+NxscBN7//lkbdL8Ef20Uyd6d6P/Bp5GdOExFiGSsXdoEv09B33gCh8amhb+e14VIo6GFEnPYnklmctpDJs9dM4DD1oymSDi79doZkJgxOkyKiFwktblqJ8Yhu1MzfGUyrTXDQRGp9NZMpSsiALC0tQ63v+84AMBXn9qF13vHrX+T7m5g40Zkfv97LFd8ePVdV6D9rz8HXzCM6T0vYODR25CbjlERIiF1NQGsWRQF4I5PhLUAOz0qRAJ+n5b54TTmnRUyisLf7xZ24MVLcHr4ZF9vJTJixugwKSJyoaeqcgiXsegR4S2T8TCr6oFmYivmVkEx71q8ewVOzBTjsvWduGBNB9JZFZ967GXzUuzgIPC3fwusWYNUdzci9U1I/s0X0XTGFVAUHyZefhKDP/si1HSSihCJWedSe0ZVVa0Q8UoRAfgZVplRNRIKcIlGMOJEEeHVjtfNqubvOcYHZSpEJINH9r7t1gxnRYTH+K5bUzOi9s3MJ0UEyI8Lbv7QCWhrqMFbA5P4zOOvlf/3LSgg6OpC5tFHcUYigejSE9B69TdQu/wk5FIJDP3qboz89t/1DbpUhEjL+uXuJKyOxFOIp7JQlPzmXa/gZVgVEe/OcJJgzasdr5tVrSgi+r3Oi6moclT9XB6P7H3bZlWOC+8AIKwFmtlflKXliIg2q2obeMWYVSt5amY2rQ0hfPnDJ+Kah17AT7YfxPb9o/jSB4/H6avagMWLgb6+GZ+fArAKwGDbUkRO/SA6jjsXis+P1OH9OPyLzcgMHwQAdNXWYt/YGC2vk5j1y/IJqzv78wvwRK1e6BnNK4kdkbCnExW8Qs1EGVUBoDZovwXOa9s6+zeyoojwzq3iiXwnchm+ZlW7gWacFBEt4t1+EqOWrCraI8ICzTgqIvFkRruALW6aX0/4563uwLevWof2SAjdQ3Fc8R9/wb/8+BWMjk7ihwAUw69o53FIfvhzWHTNfWg44T1QfH5M7ngK/d+7GZnhg/ADeFdjI/ZSESI9CxvdWYDntVGVwSvUTBvdFfBAVVuTv97b8oiwaz43s6oFRYQVQZIZVQFSRPQlQAEHZlW7u2Y4OagZumRoXxGZdNmsOpXKYjqVnbHDwS59BX9IJBwQ8iTkNRcetxCnrWrFvz25Ew9vPYAfbz+IJ//xexgZG8JiRYECAP4gApFWAICq5jD11nOI/eWnSPW9BSD/5DH15S+j5uabAQoqqwjWL2vGwdFpvLBvFGcevUDIa3i5dddII6d9M0JbM0H7LXAt0Mzhw2fIhiIi6+guQIUIl6Q5pmikbLZmeCkiTpfeqapqiHgX+9ZoCAVQ4/chlc1hOJ5EZ43zC+ChsXxbZvE8asvMJhoO4s5LT8Cla5dg089exduDkwg2L5rxOWomhcnXfofY8z9HZrRX+3gEwNDEBGoaGlw+NeGEdctb8PjLvUINqz0eh5kxGjmZVfXWjAhFxH5wJLdAMxs5IrynNHlChQiH7H2n47tOZTqGExMVOw+bMxetiLB9M/2xBEbiKXRy2PbZV8hamG9tmWKsX96CJ248E2/2xXDKhtMAqICa/8fLjPYhl9D3k9QC+NTSpfjirl3wkSm14pi9AC8gIJ5bntZMQRGZdmhW1Voz4hQRe60ZPh4RO75EmRUR+U7kMnoh4mR812GgGW9FxGYhwtZmK4ruNxEJa88Mc8oS6S0UIos8dP27SdDvw4mdTXjwrs8i1bsLqb63kOp7a0YR8q9LlmByyxZ8qbubipAKxY0FeNIUIsysyqs1I0ARcTKdmOQ0oGBHEUlymBAVhXwnchmerRnru2b4KiJO5tsBPUOkvob/7H0xeI/w9hYmZrwcP/SCK664AuoPfgAVyP/q6IC6cSPUJ5/E5gMH4KOdMRWN6AV46WxOK+K9LkQamVmVU2tGpEfEznQir/aIURFRVXNx+LwW7omg6q9OeqCZF8mqfBURJ4l/gDFV1Z03ahvnEV5NEZknGSKWuOKKfGtGVYH+fuC554ALL6QCZJ6wYUV+jPcv3fwLkb6xBHJq/jq2IBLi/v2twC1HRGRrxlHEO5/2iFFRMXvfofFdieE5vpvK5pCzsKyJd4XKFJFkxto5GG5NzDBaOI/wsgyRxVWmiBDzH1aIPN89YvoJ2CwHDBMzXgdd6WZV+5N/gGCzqqOld3yu+cavN6vE660ZUkSkI8nDrGp4U7BWjxl4V6jGEVg71bpbEzMM5hEZ4uARUVVVU0Tm89QMUZ2c2NmEcNCH4XgKuwcnuX5vWfwhgDHQLOWo4BI5vuss4p2PRyTgU8C652a9iUlOry0C+U7kMjynZgBrPhHeEe/GFo+dHxK3wswYeqiZ89bMSDyFZCYHRQE6Gr2VlwmCNzUBH05emveJbOXcnukZLSgizd4X8CzQLJ1VHe3MihUUFRGtmbCTQoRTZIOiKJbTVUkRkRimYAQdtGbsVKfGz+VVofp8iva97MiG7rdm8gUDj9YMa8u0NYSk/EEjCKdsWJEPqvvL3mGu3/eAJGFmQP5ayB4KnRhWJwqKiHStGY4hluwBOGFREeE1HMET+U7kMmkOioiiKLbmunnNlBtxEmpWya2ZQ1qGiPdPdQQhgg0rdcMqT5+IDFt3GYqi6FkiNg2rmWwO8UKRILQ1YydHhGOIZdji1nca35UYpoiEHIYE2ckS4b30DgDqHKT+TbLxXZemZnjum9HCzKpxYoaoCtZ2NaEm4MPhiSS6h+Jcvmcup6L7cP57LWut5/I9neI0XZUZVQGxishUOmu5IGTtpjoOKy201ozJe47uT5FPMaZChIMiAhhkMg89IvnvZX8hk66IuLOnhW3gnU5nHW0MBvQMEVJEiPlKOOjH2q4mAPzGePcOxTGRzCAc9GHVAjkKEaehZqwQqQ36HbXcS8GGAlTVemTDlPaw57xA0mIjSBGpfPgVIjZaMwIid5kiYif1bzLBChF3Kub6Gr/29+40XbWqM0SIqmEjyxPh5BPZcXAMAHD84kYh0fF2cBpqpk/MiGkx1xoeHK0qz/HCAxcPRSRkcfkemVUlJsnBrArYW8ssQipzsgdhMuWuWVVRFLRxSldlhUi1paoS1cWGlQXDKiefyI6D4wDy48Gy0ORwA29MM6qKUXYDfp+WO2XFi6eq+iQQT0XEtFlVgBWAF/KdyGV4mFUBo0fEnCKiqqpeoXJ8YzhJ/Yu7PDUDAC0NfEZ42dRMteyZIaqTk5c2I+hX0DeeQM/ItOPv90pBETmpq9Hx9+KF08V3+uiuuOuYNp1ocddLthA0ydMjYrY1I2I4ghdVX4jw2DUDWN83k8rm2LJUMVMzNjwXbk/NAPoIr5PWTCabw0CMeUSoNUPMX2pr/Jp6sbXbWXsmnc3hjd4YALkUEc2s6lARETExw7Cz6dyoUrMWuhPCNhURGt+VEN5mVdMpdwblhKdUxn5AbLVmCkYqNwsRHq2ZgYkkcioQ9Ctoq6cwM2J+o+2d2evMsLqrfwLJTA7RcADLW70f3WXoKqm9a4Ie7y6uENGmEy0oIuxBLxz0wc9hqWjI6vhumsyq0pLi5CS2uviO+UMUxbkaY4QVEexNb4XJZP5JwtXWDIcRXuYPWdgYdmVrMEF4ie4TcaaIsLbMiZ1Nnu+YMdLq8OFEX3gnsjVj/YFP84dwUEMA64qINhxB47vywSNZFbA+NWOsTnleBNjcfCxhpzXjviLCRniHHGzgpR0zRDWxblkz/D4FB0entSA/O+zoYUZVefwhgH5NsLuV243WjJ1QM21ihtNUouWId1JE5IVba4aZVU2PUokJl2FyJPthtIIe8e5excwj1Kx3jDJEiOqhIRTA8UvyxYOTMV6jIiITmiJi0zcmcvMuQ0+wNv/Ap2WIcFJE7NoByKwqIfzMqlZbM4UwM85vCjY7P2FREUlnc1pR5q4i4uyiAwB94yzenYyqRHWw0aFPZDqVxduFLb4yTcwAuiIykcxYikNg6K0ZN8yq5nOjeGaIAHYi3ml8V1q8CjRLCFpAxBSRCYuKiNFTUqkekUXUmiGqBLZ3xu7kzOu948jmVCyIhLAwKlcBHw0HEPTn29V2rgvuKiJWPCJ84xGsKiI0visxXk/N8FZENI/ItDVFhP3whgI+IbHIpWhl47vxpO2AJtaaoTAzolo4ZXkLgn4F+4en8NbAhOWvf6UQZHZSZ6NURlUgH3TY6mCs35XxXRsxCcyDx1sRMe0REZDkzQv5TuQy3FozmkfEmiLCWyZjcuRE0qIiknI/QwTQWzOJdM7WyDEA9BZaM4uoNUNUCZFwEGcdvQAA8OsdfZa/foek/hAGU0rtmNi1QsSN1owdRYTX1EyQ7Teb+wyiAjR5Id+JXIZbsqrl1owYmYyNrFn1iHiRqgrknw5YhW5Hhp1KZbQoaDKrEtXEe09cBAD49at9ltVEPdpdLn8Iw4l3jF37RI7v2slr0hQRTsMAVu45xgBN2r4rIZoi4nprRrRHJGPp4uRFmBlQ2DfjYISXtWUaQgGhT0AEIRvnr+lAjd+H3YOTeGtg0vTXjU+n0T0UByCvIsKuCcMWVz+oqqoXIiLHd4PWx3d5KyIhC4qIsVih1oxk5HIq0tn8zdqzqRnO1SmbmsnmVIvVujetGcCZYZUmZohqJRoO4qx3sPZMr+mve7WghnS11Go/e7JhN9RsKpXV9rmIfDCpY8GRlnJEmEeEl1nV/PZdZhngHaDJC/lO5CJMDQF45IhYXUAkxjhUG/Rr8cFW2jOTCfczRBhOZFiamCGqmUsK7Zn/stCekTU/xIgeambtmsD8IQGfInRMNVIoRCYtTCdOcc5pCltYtGq838hmTgaqvBBJGwoR58mqNqdmOCsiiqIY0lXN/5BMeuQRAXRFxE6kM4WZEdXMeavbURPwYe/hOHb2m5ueYUbVtTIXIlqombXWjLZ5tzYo9IbbULjGTlpYpcFbEdGnZsy3ZmQc3QWqvBBJGSpJbtt3LeaIiKjaozayRLxszdi96ADGeHdqzRDVRyQcxNnvsDY984qk0e5GNJXU4sMJu+aJzBAB9OukFdVZzxHhZVY1r4jIPLoLVHshou2ZURwvS7PsEdHeGPwrVDv7ZiY9Gt8FdBnWnkeEFBGiurEyPTMYS6A/loBPgRYTLyNOWzOijeu2FJGkKEXETCEiRoHnRXUXIhk+GSKAjV0zmlmV/z+BHmpmXRGpvNYMZYgQ1c15qzsQCvjQPRTHG32xsp/LgsyOam/w5GfdLLpZ1VrQod6aEfv/pnlELBQi+tQM74j3ue85ojyJvJDzVC7BCpEgh38cpmykzG7fZbn/AhSRqGGE1yzMrOqFItLWoF90rKCqqhZmRqmqRLXSEArgnGPaAczdnvnBX/YDyG/wlRm7QYdaaybkkiJiISZBzxHhHfFuXhGRMcwMqPZChFOqKmB/fFfEGyNipxBhmyE9UUQKrRmLMuzoVFr7e1xIHhGiijHTnvnDzkE8veswgn4FnzhzpZvHs0xdTUCLUbfSnokl3FFE2ANbJqeavuaLUkRS2Zw2slyKpMR7ZoBqL0Q4paoChtaMxUAzET07O1Mzcc6jZVZgMuxQPGVJhmVtmbaGkLQ/YAThBuce245w0If9w1N4vffI9kwqk8MXf/0GAOBj71qBlQsa3D6iZZgqMmRBKXXLI2IMJTP7wKdNzXBWRIC5lXiZN+8CVIgA4FSIBKzmiIhZegfoiYKWpmZS4jdWlqI9GoJPyf97HLYwOaNNzJA/hKhy6kMBnHtsvj1zxy9fP8K78L3n9mHv4TjaGmpw/blHeXFEy9gxrDKPSERwIeLzKZoqYsYnks7mtPsNb0UEmHuElxQRifG2NSMm4h2wt29GyxHh5Oi2QijgR1dLHQBg96D5qOqeUTa6S/4Qgrjh3KMRCQfwwv5RfOzB5zWVc2gyiXt+9zYA4F8uPKZiViG02Rjr1zfvir+OaYWIieus0efCa2rG71MQ9OenPRNzKPE0visxfBWR/PdIZXPIzdGvA8QGzNiZmtGTVb1x0h9VkIr3WChE3ioEOL2jQ36ZmSBEs3pRFN+/ZgMioQC27RvFxx7chngyg69seQsTiQyOXxLFX6/r8vqYprGTJaIvvBNfbDHDqplN58wfEvQrXO43DLNKfNWO7+7btw/XXHMNVqxYgdraWqxatQq33347UinrI5qiSPNURAz/wMbo+FKIVUSsm1W9DDQD8uOEgDVFZGd/vhd+zMKokDMRRKWxtqsJ3/94vhh5ft8IPvLAc3h02wEAwO3vO05b/1AJMBO7tdaMO4FmgDVFhHeGCIN5PuZSRKp2fHfnzp3I5XL49re/jddffx1f+9rX8K1vfQu33nqrqJe0TFKAIgKY84loyapCFBFrhUgqk9OMVE113si2q1ghcthcIZLLqdrG0WMWRoSdiyAqDWMx8tqhGFQ1v5PmlOUtXh/NEnbG+vXWjPjrWMRCqBnviRmGVUVE1kJEWNl40UUX4aKLLtJ+v3LlSuzatQv3338/7r77blEvawmerZmAT4FPAXIq68eV/0HQpTKBgWYmzapjU/knDp/ijqRZDKuKyIGRKUynswgFfFjeWifyaARRcaztasL3rjkVf/ed56EA2PRXq70+kmXsLMNkD1+uKiImChHeGSIMpqjPaVbVckTkbM24qsOPj4+jpaV0VZ5MJpFM6tVvLFY+JdApPM2qiqIgFPBjOp01uQ1RnEdEn5oxp4iMTuULlqa6GsdR93ZhhchALIlYIj1nQcQWfB3d0YCAhGutCcJr3rm0Gc/+67lIZnJoK0ygVBKthdbMkBWz6rQ747uAtX0zohQRpqgn5rjn6Aq8nNdK1061Z88e3Hvvvbj22mtLfs7mzZvR2Nio/erqEmus4pmsCljLEkkKXHpnlAznCroB9B0vzR61ZYD8haM9kr/wmDGsav6QDvKHEEQpIuFgRRYhgK6ImN1BlczoD4FutGas7JvhvXmXYXa1iDa+K6kiYvkueMcdd0BRlLK/XnjhhRlf09vbi4suugiXXXYZPv7xj5f83ps2bcL4+Lj2q6enx/r/kQWYWTXE6Yma9d+8XkJklCXNGKlGC60ZtvPFK6y0Z3YVFJHVi8gfQhDzEaaIjMRTpiYRjcqEG6b7iJXxXUGBkWYVEdnHdy3/a11//fW4/PLLy37O8uXLtf/u7e3FOeecg9NOOw0PPPBA2a8LhUIIhdyr3nl6RACDcWiON0U2p2ptIRFvjFDAj1DAh2Qmh1gijcY5lA5WiDTVeV+IPLtn2JRhlRUiZFQliPkJezDK5FTEEuk5r0/axEwo4Mp0kAyKSNikRyQhuSJi+W+lra0NbW1tpj730KFDOOecc7Bu3To8+OCD8Pnkqsb4FyLmWjPGPxc11x0JB5GcTJrqX44WpM8WCQoRYO7WzHQqi33DcQBUiBDEfKUm4EM0HEAskcHQZGrOQkTLEHGhLQMADSHzXjxRiojZh1/ZFRFhp+rt7cXZZ5+Nrq4u3H333Th8+DD6+/vR398v6iUtk+RoVgWMHpE53hSG1o2oN0bUwuTMSDz/Oc1et2YWmGvNvD04gZyaf2JaUKH9b4Ig5qZNi3mf27DKrnVuranQFZG5r7GiFZE5PSLVOr67ZcsW7N69G7t370ZnZ+eMP7Oy2Ewk3M2qJme6WfhMwKcIm/iIWJicYeO7XppVAT1L5MDIFBLpbEm1iE3MHLswAkWpnIAmgiCs0dpQg71DcVOGVbZnxq0IgoiF8V3hOSJzKiJVumvm6quvhqqqRX/JAs9kVcB8a0ZbeCewX6fvmzGhiLBCxGNFpD0SQiQUQE6F1nopBvlDCKI6aDFs5p6LCa8UESvJqpxNtOY9IrR9V1rEeUS8X8lsZd/MqDa+620hoiiKnrBapj2zy6CIEAQxf2m10ZpxzyNiJdBMUI5I4WHWdKBZtSkilUCKc9/MrEwmMsyMYWXfDAs0a6n3fiunblgtrYjQjhmCqA70DbxmFBHWmnFJEbEQaBYvtGa454hYfPgVsduMB3KeyiW0ZFXegWYmZTKRb4qIthnS/NSM14oIYMgSKTHCOzSZxNBkCopCW3cJYr6jKSIm9s3oC+9c8ogUrrHJTE57qC3FVMGsyn1qxqQiotkBSBGRj1Qm71cJcveImFzJLPBNoS++K9+aSWdzWrEiRSEyx+QMa8ssa6nj/nRBEIRcWNk3E9PGd925LtQb/B7xOR742J97poi48PDrBDlP5RI8d80AVloz4t8U2vjudPkfkFHjwjuXeqvlYIrI3sOTRePpd5JRlSCqBmZWHbZgVnVraibo92k+v7l8IqIUEeseETlv+XKeyiVShb6Z24Fm+gIi8YrIXDkio3F94Z0baYRz0dVSh5pCKuyh0ekj/nxnH/lDCKJasJQjMs0277r3QGU21GxKkEdEL0RKP/yqqkpmVZnhPjWjeURMtmbc8IjM8QOix7t7r4YAgN+nYGVbPQBg9+GJI/5810BhxwwpIgQx72ktKCKjU2lksuWvq4MTCQDu7syKmIx5Z+O79cJaM6Uffo0KPY3vSohXrRlR/UIjrM0ytyIiR7y7kVIjvNmcircGqDVDENVCU10NmFDL8o6Kkc7m0FNQUJe31blxNADGEd7S19lsTsV0muWIiGrNlL7nGO9HpIhISLpgVnW7NcNUCpHBO2YVkRFJFt4ZKWVYzSeu5hAO+rCstd6LoxEE4SJ+n6L7RMoYVg+NTiObUxEO+tARCbt1PFMjvNMG/4Y4RaRcIZJ/fUUBgn7v2+/FqOpChPv4rkkHsxsJgFGTUzNjEmWIMI4qoYgwf8jR7REp/CwEQYiHFSLlYt67C0nMy1vr4XPx2mBmAy9beKco/FsjTBEpFxmRNIzuyroSo7oLkQzn1oz2ppirEBFvqmKFSCJdfsad/XB7He9uxFiIGFcC7KREVYKoOlrr84bVoTKG1X1D+UJkWat7bRnAsG+mjCLCFt7V1wS4FwJmIt5lDzMDqrwQSQqLePe+NdNg+N7lVJHRKXnCzBgr2urhU/K5AIcNFx/aMUMQ1YeZLBFWiCxvc7dla0YR0T2B/P0ZZnyJepK3vLd7eU/mAmx8l1+gmTmzqr6uWpwi4vcp2l6Dcv1LGc2q4aAfXS35J5vdg5M4NDaNh7fux/P7RgAAx9LoLkFUDW0m0lX3DU8BAFa47B0z4xHRM0T4P3haUURELll1SlVHU6azedmf364Zsx4R8YoIkJ+ciaeyZSdnRqZYjog8HhEgb1jdPzyF637worYLB8j/HR+/hAoRgqgWWk2YVdm2bikVkZQ4RUSbmilnVq0ARaSqCxGvds24lQAYCQfQN16+Wh8rtGbcnL03w7GLIvjdzkGMTqXhU4B1y5px9jHt+KsTFkk14UMQhFhaGsqnq6azORxko7suKyJmPCJTgjJEAL24yOZUZLI5BIqo+7KHmQFVXIhkc6oWIc47R2SuBUhuKSJm9s3IaFYFgI+fsRI1fj9WLKjHWUe3UfFBEFUKM6uWSlc9WBjdrQ360RENuXk0a4oI5wwRYGa7JZHJoaFoIVIwq5IiIh/GYsH98V22rlqsIjLXvpl0NqedRSazKpAvjG48/2ivj0EQhMe0zaGIGCdm3B5P1SLeTYzvilBEjA/RyXRW86wY0TbvSuwRkbdEEoyxEOFmVg3OPTWTzua0gBu3FJFSHhGWIaIoQKMEC+8IgiBm06rtmyleiHQXCpEVLvtDAEOyahnVmY3vivCI+HyK9iBdyidSCYqIvCcTTCprLET4VNHa1EyZHBFjL7FBeCFS3tGt7ZmpDVJAGEEQUsLGdyeTmaLTIcyo6kXaspldM2zhnYipGQAIB8pPzmgeEcoRkQ+jUZWXnGemNcOKgtqgn5sSU4q59s2w0V3Z2jIEQRCMSCigPSwWS1fVRndd3DHDaDATaMbMqgI8IsDcQZpubHt3SvUWIqxK5FgMsEIklc0hl1OLfk7MhXh3hllFRDajKkEQBENRFCxszO+Pmb32ATCEmXmgiDBVO57KasMPs5nSxncFKSIsS6SEJUAb3yVFRD5SnFNVAb0yBWa2foy4NTGTf43yUzMj8fzHmyXLECEIgjBy6vJWAMBze4dnfDyVyeHgKFNEvPOIAPp0zGz0iHdBisgcloBKGN+t+kKEZ3vEaAYq9aaYcCFVlTHX1IyM8e4EQRCzOX1VvhB5ds/MQuTg6BRyat4IuiDi7ugukL/ms7ZRqfYMm5qpE+URmUMRYa0ZMqtKCO8wMwAI+BQwz2epyRk3FRFtA2+yvEdEtjAzgiAII6cVCpFXD47N8LwZjapebJZVFEX3iZQwrBqX3okgHCi/gVc3q5IiIh0iWjOKosy5bybmUqoqMLdHZIRNzZAiQhCExCxuqsWKtnrkVOD5vSPax7uHvDOqMuYKNZsSGGgGGGMjaHy34tAUEc6TK3NlibiqiLCpmenyOSIt9eQRIQhCbpgq8uc9Q9rHvDSqMlioWenWjDuKyJzju1SIyIcIRQTQ/7ETc3pE3J2aUdUjHd0jNL5LEESFwHwizxl8ItqyOw8LkcicrRlxS+8APTG1lCKije9Sa0Y+tEKEtyKitWbmUkTcaM3kXyOTU4sWRmM0vksQRIVw2sp8IbKzfwJDhb0zXm3dNaK1ZuZSRASZVUNmA81IEZGPtACzKmAINSupiLjXmqmv8Wvm2WKhZqSIEARRKbQ2hHDswggAYOveYaQyORxiW3e99IgUCoxi+2ZUVdUUEWHju0HWminhEUmTWVVahLVm5jAOxVwc31UUpWSWSCabQ0xbeEceEYIg5Of0VW0A8mO8PYXR3foaPxY0uD+6yyiniCQzObCcM1Hju3qid4nxXTKryktSlFnVdGvGncXH7HVis35IxqZp4R1BEJWF0Seib931ZnSXoXtEjlSd4waVpFaQIhE2qYiQR0RCRJtVSykibppVAX1MePbkDMsQaawNIiB45w1BEAQPTl3ZAp+S37jLws28SFQ1Ui5HZKqQIVIb9AtbLKoFmpX0iJAiIi3MI8J78ZxZj4gbOSJA6SwR8ocQBFFpRMNBnNDZBAD46YsHAXjrDwH01kyxvCbNHyIoQwTAnNlVZFaVGHGKiGytGeYRmflDMjpFe2YIgqg8WHuG5SAt83B0FyiviLDNu6IW3gFzKyIJas3IS0pQlVjOrJrO5jBdeLO4YVYFgGgt84jMas3QnhmCICoQVogwvG7NRMqYVacEZ4gAZhQRas1Ii4hdM0B5j4jxjeq2R2T21MwoZYgQBFGBrF/WMmPIwMswM8CQrFpGERGVIQKY8YjQ+K606Nt3+RqIQmUWELH2SDjo4+5NKUUpjwgtvCMIohKprfHjnUubAOTbIm0N3l7DynlE3FBEtGTVIr7EXE7V7nVhUkTkQ981w/cNUk4RcTNDhFFqamYknv99E3lECIKoMFieyLLWOk9Hd4E5PCKCN+8C5XNE2H0OIEVESkQHmhWTyfSJGXfaMkBpRYTFu7eQR4QgiArjw+uWYPWiKK7YsNTro+gekeSRO72mkmI37wLlc0SMKonMHhH37oiSIaoQYeFgY0U23k54oIiUmpoZIY8IQRAVSmdzHX5z45leHwOArohkCzu9ag1tGDcUkXCZje8sVdWnAAFBOSY8kLdEEoyoQqQjGgYADMaSR/yZ26O7QJmpGcoRIQiCcExdjR+sOzQxK13VDUWE+RLLKSLhoN/zFlY5qrcQ0TwifP9xFkTyOw8GJhJH/BlTRNwKMwPmzhFpqSePCEEQhF0URdF9IrOus24qIokiikgljO4CVVyIiNq+K5siou+a0Sv1TDaH8WlmViVFhCAIwgmREoZVV3NEiikiWl6WvEZVoIoLEfYPxHtqpr2giEwmMzMWHgH6mmhXWzNhfca9Z2QKALQiBACaaOEdQRCEI0pt4HUjRyRkUERmm2XZ0ARTTWRF7tMJRJRHpCEU0KrfwYmZqogXZtW2hhqc1NkIVQU+9t1tGJ9Oa2FmtPCOIAjCOaw1M+GBIsKmZlQVSGdnFiKkiEiOqEJEURRDe2amTyQ27b4ioigKvn3VeiyMhrF7cBL/38PbtbYR7ZkhCIJwTgNTnj3wiBj9H7N9IppHhBQROUllxSSrAkbD6kxFxItAMwBY2BjGd64+BfU1fjy7Zxifefw1ADS6SxAEwYOSHhEXpmZq/D5tamd2fhWbpCGzqqQws6qIf6BSiogXZlXGmsVRfPPKk+H3Kdg7FAdAYWYEQRA8KJWuOuWCIqIoip6uOsuwyhQRmTfvAlVciKQEmVUB3bBa2iPiTY7c2ce04wsfOE77PU3MEARBOKfUvpl4wSNSL1ARAQz7Zma3ZkgRkRtRHhEA6IgWWjMlFBE3c0Rmc+WGZfjk2asAAGu7Gj07B0EQxHxBV0RmB5rlC4M6gYoIAIRLhJpVilmVIt6FtmZmKyLetWaM3HLRsfiHM1agrSHk6TkIgiDmA5Ei47upTE7zIopszQC6GXW2IsI8I2RWlZSkK2ZVXRFJZ3OYLrwp3DarFoOKEIIgCD4U84hMp/SioFbg+C5Q+YpIVRYiqqoKS1YFdEXksEERMVbKXisiBEEQBD+KeUSYP6TG7xNynzFSavEdRbxLTCanggXQhQSaVScM6arsDRoO+hCkEDGCIIh5QzFFRAszE2xUBUovvtPGd6k1Ix/MHwKIUUSKpat6lSFCEARBiEXziBgKES3eXbA/BDDEvKeLKyJhas3Ih+hCpFi6qixGVYIgCIIvDaEjk1XjLsS7M7TFd5lZHhFSROSF+UN8CuD38TerAkemq3qxZ4YgCIIQj+YRMbZm2OiuwIV3jHAJRSRRIWbVqnw8Twoc3WWUUkSipIgQBEHMK5hHJJXJIZnJ4sX9Y7jnd28DAOpdUET0QLPZikhlbN+tyrsim+2uEWganZ2u6nWqKkEQBCGGBoPq8Xf/93n8pXsEQL4tc/mpS4W/PpuKOdIjQoqItOhhZuL+cWanq2oekRC1ZgiCIOYTfp+Cuho/plJZ/KV7BAGfgis2LMUN5x6ttelFwhSRI3NEKmN8t7oLEQFhZozZ6aqsd0iKCEEQxPzjqPYG7Dg4jvedtBj/9J53YHlbvWuvrS29OyJZNX+vk33pXVXeFUWGmTFmp6uSWZUgCGL+8v1rNiA2nUZXS53rr11aEamMpXdVWYiI3DPDmJ2uGmNm1dqq/CsnCIKY1zTWBtFY682DJiWrViBJFxQRVohMJDOYSmUMOSKkiBAEQRD80HJEZisiWo4ItWak451dTfjR/z5NaJXI0lWnUlkMxpI0NUMQBEEIoVSOiJasSuO78tFUV4NTV7QIf52OaBjdQ3EMxBKITVMhQhAEQfBnzmRVycd3XSmTkskk1q5dC0VR8PLLL7vxklJgTFfVA82oNUMQBEHwo7QiUhlmVVdOd8stt2Dx4sVuvJRUGNNVadcMQRAEIYJQkWTVbE7VwjtlH98VXoj85je/wZYtW3D33XeLfinp6CgoIr1jCUwXKlUyqxIEQRA8KZasalzuKrsiIvTxfGBgAJ/4xCfw+OOPo65u7tnqZDKJZDKp/T4Wi4k8nnDaC+mqew5Pah8jRYQgCILgiZYjYhjfNY7yyl6ICDudqqq4+uqrce2112L9+vWmvmbz5s1obGzUfnV1dYk6niuw1szuwXwhEg76EBS434YgCIKoPsJFxndZuJnfpyAg+X3H8unuuOMOKIpS9tcLL7yAe++9F7FYDJs2bTL9vTdt2oTx8XHtV09Pj9XjSQUzqx4amwZAbRmCIAiCP6EiZlVtdFdyNQSw0Zq5/vrrcfnll5f9nOXLl+POO+/E1q1bEQrNXPizfv16XHnllXjooYeO+LpQKHTE51cyTBFhUFuGIAiC4I3emtEVEW1iRnKjKmCjEGlra0NbW9ucn/eNb3wDd955p/b73t5eXHjhhXjsscewYcMGqy9bkRxZiJAiQhAEQfCFeUBSmRxUVYWiKJo6Irs/BBBoVl26dOmM3zc0NAAAVq1ahc7OTlEvKxXGdFUAiJIiQhAEQXDGOJ47kcwgGg5qiojso7tAle6acROjKkKtGYIgCII3dUE/ljTVAgC++fvdAIypqvLf5l074fLly6GqKtauXevWS0oBM6wCQCRErRmCIAiCLz6fgi984DgAwH/8aS9e6RmrmM27ACkiwiFFhCAIghDNeas78IG1i5FTgVt+skNL85Z9zwxAhYhwOoyKCJlVCYIgCEHc/r7j0Fpfg10DE/jmH/ItmpDkm3cBKkSEw9JVAVJECIIgCHG01Nfg84UWzduFIE1SRAhqzRAEQRCu8d4TFuGCNR3a70kRIWaaVak1QxAEQQhEURTceenxWlxEmBQRwqiIUI4IQRAEIZr2aBj/50MnoL7Gj9NWtXp9nDmhO6NgZrZmSBEhCIIgxHPJiYtx8fGL4PcpXh9lTkgREUxDKIDW+hoAM9s0BEEQBCGSSihCAFJEXOFbV63DYCyJhY3huT+ZIAiCIKoIKkRc4JTlLV4fgSAIgiCkhFozBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4htTbd1VVBQDEYjGPT0IQBEEQhFnYfZvdx8shdSEyMTEBAOjq6vL4JARBEARBWGViYgKNjY1lP0dRzZQrHpHL5dDb24tIJAJFUbw+jufEYjF0dXWhp6cH0WjU6+PMW+jv2R3o79kd6O/ZPejvWkdVVUxMTGDx4sXw+cq7QKRWRHw+Hzo7O70+hnREo9Gqf5O7Af09uwP9PbsD/T27B/1d55lLCWGQWZUgCIIgCM+gQoQgCIIgCM+gQqSCCIVCuP322xEKhbw+yryG/p7dgf6e3YH+nt2D/q7tIbVZlSAIgiCI+Q0pIgRBEARBeAYVIgRBEARBeAYVIgRBEARBeAYVIgRBEARBeAYVIhVOMpnE2rVroSgKXn75Za+PM6/Yt28frrnmGqxYsQK1tbVYtWoVbr/9dqRSKa+PNi+47777sGLFCoTDYaxbtw5/+tOfvD7SvGLz5s045ZRTEIlE0N7ejksvvRS7du3y+ljzns2bN0NRFNx0001eH6VioEKkwrnllluwePFir48xL9m5cydyuRy+/e1v4/XXX8fXvvY1fOtb38Ktt97q9dEqnsceeww33XQTbrvtNrz00ks488wzcfHFF+PAgQNeH23e8Mwzz+C6667D1q1b8dRTTyGTyeCCCy5APB73+mjzlm3btuGBBx7AiSee6PVRKgoa361gfvOb3+Dmm2/GT3/6Uxx33HF46aWXsHbtWq+PNa+56667cP/992Pv3r1eH6Wi2bBhA04++WTcf//92sdWr16NSy+9FJs3b/bwZPOXw4cPo729Hc888wzOOussr48z75icnMTJJ5+M++67D3feeSfWrl2Lr3/9614fqyIgRaRCGRgYwCc+8Ql8//vfR11dndfHqRrGx8fR0tLi9TEqmlQqhe3bt+OCCy6Y8fELLrgAzz77rEenmv+Mj48DAL1/BXHdddfhve99L84//3yvj1JxSL30jiiOqqq4+uqrce2112L9+vXYt2+f10eqCvbs2YN7770XX/nKV7w+SkUzNDSEbDaLjo6OGR/v6OhAf3+/R6ea36iqiptvvhlnnHEGjj/+eK+PM+949NFH8eKLL2Lbtm1eH6UiIUVEIu644w4oilL21wsvvIB7770XsVgMmzZt8vrIFYnZv2cjvb29uOiii3DZZZfh4x//uEcnn18oijLj96qqHvExgg/XX389duzYgUceecTro8w7enp6cOONN+Lhhx9GOBz2+jgVCXlEJGJoaAhDQ0NlP2f58uW4/PLL8atf/WrGRTubzcLv9+PKK6/EQw89JPqoFY3Zv2d2Uent7cU555yDDRs24Lvf/S58PqrfnZBKpVBXV4cf//jH+OAHP6h9/MYbb8TLL7+MZ555xsPTzT9uuOEGPP744/jjH/+IFStWeH2cecfjjz+OD37wg/D7/drHstksFEWBz+dDMpmc8WfEkVAhUoEcOHAAsVhM+31vby8uvPBC/OQnP8GGDRvQ2dnp4enmF4cOHcI555yDdevW4eGHH6YLCic2bNiAdevW4b777tM+tmbNGnzgAx8gsyonVFXFDTfcgJ///Od4+umncfTRR3t9pHnJxMQE9u/fP+NjH/vYx3Dsscfi05/+NLXCTEAekQpk6dKlM37f0NAAAFi1ahUVIRzp7e3F2WefjaVLl+Luu+/G4cOHtT9buHChhyerfG6++WZcddVVWL9+PU477TQ88MADOHDgAK699lqvjzZvuO666/DDH/4Qv/jFLxCJRDT/TWNjI2praz0+3fwhEokcUWzU19ejtbWVihCTUCFCECXYsmULdu/ejd27dx9R4JGQ6IyPfOQjGB4exhe+8AX09fXh+OOPxxNPPIFly5Z5fbR5AxuNPvvss2d8/MEHH8TVV1/t/oEIogTUmiEIgiAIwjPIdUcQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGf8P6ZyOHzGZunmAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_41_1.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "backend = np\n",
    "\n",
    "def function(x):\n",
    "    return x*backend.sin(x**2 + 1)\n",
    "\n",
    "def analytical_gradient(x):\n",
    "    return backend.sin(x**2 + 1) + 2*x**2*backend.cos(x**2 + 1)\n",
    "\n",
    "\n",
    "x = np.linspace(-5, 5, 100)\n",
    "plt.plot(x, function(x), label=\"f(x)\")\n",
    "\n",
    "descent_x, descent_y = gradient_descent(1, 0.01, 300, solver=\"analytical\")\n",
    "\n",
    "# Change the backend to JAX\n",
    "backend = jnp\n",
    "jax_descend_x, jax_descend_y = gradient_descent(1, 0.01, 300, solver=\"jax\")\n",
    "\n",
    "plt.scatter(descent_x, descent_y, label=\"Gradient descent\", marker=\"v\", s=10, color=\"red\") \n",
    "plt.scatter(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\", s=5, color=\"black\")"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}