FYS-STK4155/doc/LectureNotes/_build/jupyter_execute/exercisesweek41.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b18bcd06",
   "metadata": {},
   "source": [
    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
    "doconce format html exercisesweek41.do.txt  -->\n",
    "<!-- dom:TITLE: Exercises week 41 -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7542d6aa",
   "metadata": {},
   "source": [
    "# Exercises week 41\n",
    "**October 4-11, 2024**\n",
    "\n",
    "Date: **Deadline is Friday October 11 at midnight**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80943a15",
   "metadata": {},
   "source": [
    "# Overarching aims of the exercises this week\n",
    "\n",
    "The aim of the exercises this week is to get started with implementing\n",
    "gradient methods of relevance for project 2. This exercise will also\n",
    "be continued next week with the addition of automatic differentation.\n",
    "Everything you develop here will be used in project 2.\n",
    "\n",
    "In order to get started, we will now replace in our standard ordinary\n",
    "least squares (OLS) and Ridge regression codes (from project 1) the\n",
    "matrix inversion algorithm with our own gradient descent (GD) and SGD\n",
    "codes.  You can use the Franke function or the terrain data from\n",
    "project 1. **However, we recommend using a simpler function like**\n",
    "$f(x)=a_0+a_1x+a_2x^2$ or higher-order one-dimensional polynomials.\n",
    "You can obviously test your final codes against for example the Franke\n",
    "function. Automatic differentiation will be discussed next week.\n",
    "\n",
    "You should include in your analysis of the GD and SGD codes the following elements\n",
    "1. A plain gradient descent with a fixed learning rate (you will need to tune it) using the analytical expression of the gradients\n",
    "\n",
    "2. Add momentum to the plain GD code and compare convergence with a fixed learning rate (you may need to tune the learning rate), again using the analytical expression of the gradients.\n",
    "\n",
    "3. Repeat these steps for stochastic gradient descent with mini batches and a given number of epochs. Use a tunable learning rate as discussed in the lectures from week 39. Discuss the results as functions of the various parameters (size of batches, number of epochs etc)\n",
    "\n",
    "4. Implement the Adagrad method in order to tune the learning rate. Do this with and without momentum for plain gradient descent and SGD.\n",
    "\n",
    "5. Add RMSprop and Adam to your library of methods for tuning the learning rate.\n",
    "\n",
    "The lecture notes from weeks 39 and 40 contain more information and code examples. Feel free to use these examples.\n",
    "\n",
    "In summary, you should \n",
    "perform an analysis of the results for OLS and Ridge regression as\n",
    "function of the chosen learning rates, the number of mini-batches and\n",
    "epochs as well as algorithm for scaling the learning rate. You can\n",
    "also compare your own results with those that can be obtained using\n",
    "for example **Scikit-Learn**'s various SGD options.  Discuss your\n",
    "results. For Ridge regression you need now to study the results as functions of  the hyper-parameter $\\lambda$ and \n",
    "the learning rate $\\eta$.  Discuss your results.\n",
    "\n",
    "You will need your SGD code for the setup of the Neural Network and\n",
    "Logistic Regression codes. You will find the Python [Seaborn\n",
    "package](https://seaborn.pydata.org/generated/seaborn.heatmap.html)\n",
    "useful when plotting the results as function of the learning rate\n",
    "$\\eta$ and the hyper-parameter $\\lambda$ when you use Ridge\n",
    "regression.\n",
    "\n",
    "We recommend reading chapter 8 on optimization from the textbook of [Goodfellow, Bengio and Courville](https://www.deeplearningbook.org/). This chapter contains many useful insights and discussions on the optimization part of machine learning."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2095d197",
   "metadata": {},
   "source": [
    "# Code examples from week 39 and 40"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f428decb",
   "metadata": {},
   "source": [
    "## Code with a Number of Minibatches which varies, analytical gradient\n",
    "\n",
    "In the code here we vary the number of mini-batches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ba38d454",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.41622083]\n",
      " [2.74692745]]\n",
      "Eigenvalues of Hessian Matrix:[0.26594878 4.45190996]\n",
      "theta from own gd\n",
      "[[4.41622083]\n",
      " [2.74692745]]\n",
      "theta from own sdg\n",
      "[[4.38222303]\n",
      " [2.74411787]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHHCAYAAAC88FzIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABVrElEQVR4nO3deVxU5f4H8M8AMqAJisomuOEKbplppiYmYGamqblUpt5s1brevN2yX2nd6mK73bK0Mqlraqao1S0FFdTKrYibuS9ouOAuKCTKzPP7Y5qRgRlm4cxZ5nzer9e8lHPOnPOcOcDz5Xm+z/MYhBACRERERDoVoHQBiIiIiJTEYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiI3DZhwgS0aNFC6WKoWm5uLgwGA5YtW6Z0UYjITQyGiFQoIyMDBoPB9goKCkLTpk0xYcIEHDt2TOniERH5lSClC0BEzv3zn/9Ey5YtcfnyZWzZsgUZGRn4/vvv8dtvvyEkJETp4hER+QUGQ0QqNmjQIHTv3h0AMGnSJDRu3BivvvoqvvrqK4waNUrh0pGSSktLUa9ePaWLQeQX2E1GpCF9+/YFABw8eNC27cqVK5gxYwZuuOEGhIeHo169eujbty9ycnLs3nv48GEYDAa88cYb+PDDD5GQkACj0Ygbb7wR27dvr3atlStXomPHjggJCUHHjh2xYsUKh2UqLS3FtGnTEB8fD6PRiHbt2uGNN96AEMLuOIPBgClTpuDLL79EYmIiQkND0atXL+zYsQMAMG/ePLRu3RohISFITk7G4cOHXX4eL7zwAgwGAw4cOIAJEyagQYMGCA8Px8SJE1FWVlbt3jMyMqqdw2Aw4IUXXqh2zn379uG+++5DeHg4mjRpgueffx5CCBQWFmLo0KEICwtDdHQ03nzzTYdlM5lMePbZZxEdHY169erhzjvvRGFhYbXjtm7dittuuw3h4eGoW7cu+vXrhx9++MHhfe7atQv33HMPGjZsiD59+gAAioqKMHHiRMTFxcFoNCImJgZDhw516/MjIgu2DBFpiLWCa9iwoW1bSUkJPv74Y4wdOxYPPvggLl68iPnz52PgwIHYtm0bunbtaneORYsW4eLFi3j44YdhMBjw2muvYfjw4Th06BDq1KkDAMjKysKIESOQmJiI9PR0nD171lbhViaEwJ133omcnBw88MAD6Nq1K9asWYOnnnoKx44dw9tvv213/KZNm/DVV19h8uTJAID09HTccccd+Mc//oH3338fjz32GM6fP4/XXnsNf/nLX7B+/Xq3PpdRo0ahZcuWSE9PR15eHj7++GNERkbi1Vdf9eTjtTN69Gh06NABs2bNwn//+1+8/PLLiIiIwLx583Drrbfi1Vdfxeeff46///3vuPHGG3HLLbfYvf+VV16BwWDA008/jVOnTmH27NlISUlBfn4+QkNDAQDr16/HoEGDcMMNN2DmzJkICAjAggULcOutt2LTpk3o0aOH3TnvvvtutGnTBv/6179sweaIESOwc+dOPP7442jRogVOnTqF7Oxs/P7770x2J3KXICLVWbBggQAg1q5dK06fPi0KCwvFsmXLRJMmTYTRaBSFhYW2YysqKkR5ebnd+8+fPy+ioqLEX/7yF9u2goICAUA0atRInDt3zrZ91apVAoD4+uuvbdu6du0qYmJixIULF2zbsrKyBADRvHlz27aVK1cKAOLll1+2u/7IkSOFwWAQBw4csG0DIIxGoygoKLBtmzdvngAgoqOjRUlJiW379OnTBQC7Yx2ZOXOmAGB3n0IIcdddd4lGjRpVu/cFCxZUOwcAMXPmzGrnfOihh2zbKioqRFxcnDAYDGLWrFm27efPnxehoaFi/Pjxtm05OTkCgGjatKndPS1dulQAEO+8844QQgiz2SzatGkjBg4cKMxms+24srIy0bJlS5GamlqtTGPHjrUr+/nz5wUA8frrr9f4ORFRzdhNRqRiKSkpaNKkCeLj4zFy5EjUq1cPX331lV0LTWBgIIKDgwEAZrMZ586dQ0VFBbp37468vLxq5xw9erRdy5K16+3QoUMAgBMnTiA/Px/jx49HeHi47bjU1FQkJibanevbb79FYGAgnnjiCbvt06ZNgxAC3333nd32AQMG2LVW9OzZE4CldaN+/frVtlvL5Mojjzxi93Xfvn1x9uxZlJSUuPV+RyZNmmT7f2BgILp37w4hBB544AHb9gYNGqBdu3YOy3n//ffb3dPIkSMRExODb7/9FgCQn5+P/fv345577sHZs2dx5swZnDlzBqWlpRgwYAA2btwIs9lc432GhoYiODgYubm5OH/+vNf3SqR37CYjUrE5c+agbdu2KC4uxieffIKNGzfCaDRWO+7TTz/Fm2++iT179uDq1au27S1btqx2bLNmzey+tgZG1sr0yJEjAIA2bdpUe2+7du3sAqwjR44gNjbWrtIHgA4dOtidy9m1rcFWfHy8w+3uVvA13VNYWJhb53B1zvDwcISEhKBx48bVtp89e7ba+6t+fgaDAa1bt7Z1de7fvx8AMH78eKdlKC4utgtcqz5Po9GIV199FdOmTUNUVBRuuukm3HHHHbj//vsRHR3t+iaJCACDISJV69Gjh2002bBhw9CnTx/cc8892Lt3L6677joAwMKFCzFhwgQMGzYMTz31FCIjIxEYGIj09HS7RGurwMBAh9cSVRKefcHZtWtbJlfvNxgMDvebTCaPzinlZ2dt9Xn99der5XVZWZ+xlTXXqLKpU6diyJAhWLlyJdasWYPnn38e6enpWL9+Pa6//nqPy0WkRwyGiDTCGuD0798f7733Hp555hkAwLJly9CqVStkZmbaVfozZ8706jrNmzcHcK3lorK9e/dWO3bt2rW4ePGiXevQnj177M6lNGvryoULF+y2V225klLVz08IgQMHDqBz584AgISEBABAWFgYUlJSanWthIQETJs2DdOmTcP+/fvRtWtXvPnmm1i4cGGtzkukF8wZItKQ5ORk9OjRA7Nnz8bly5cBXGutqNw6sXXrVmzevNmra8TExKBr16749NNPUVxcbNuenZ2NXbt22R17++23w2Qy4b333rPb/vbbb8NgMGDQoEFelUFqYWFhaNy4MTZu3Gi3/f333/fZNT/77DNcvHjR9vWyZctw4sQJ22dyww03ICEhAW+88QYuXbpU7f2nT592eY2ysjLb94FVQkIC6tevj/Ly8lreAZF+sGWISGOeeuop3H333cjIyMAjjzyCO+64A5mZmbjrrrswePBgFBQUYO7cuUhMTHRYybojPT0dgwcPRp8+ffCXv/wF586dw7vvvoukpCS7cw4ZMgT9+/fH//3f/+Hw4cPo0qULsrKysGrVKkydOtXW+qEGkyZNwqxZszBp0iR0794dGzduxL59+3x2vYiICPTp0wcTJ07EyZMnMXv2bLRu3RoPPvggACAgIAAff/wxBg0ahKSkJEycOBFNmzbFsWPHkJOTg7CwMHz99dc1XmPfvn0YMGAARo0ahcTERAQFBWHFihU4efIkxowZ47N7I/I3DIaINGb48OG2FoUHH3wQEyZMQFFREebNm4c1a9YgMTERCxcuxJdffonc3FyvrnHbbbfhyy+/xHPPPYfp06cjISEBCxYswKpVq+zOGRAQgK+++gozZszAF198gQULFqBFixZ4/fXXMW3aNGluWCIzZszA6dOnsWzZMixduhSDBg3Cd999h8jISJ9c79lnn8Wvv/6K9PR0XLx4EQMGDMD777+PunXr2o5JTk7G5s2b8dJLL+G9997DpUuXEB0djZ49e+Lhhx92eY34+HiMHTsW69atw3/+8x8EBQWhffv2WLp0KUaMGOGT+yLyRwYhR9YkERERkUoxZ4iIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGu6WKeIbPZjOPHj6N+/fpO1ygiIiIidRFC4OLFi4iNjUVAgO/ab3QRDB0/frzaqthERESkDYWFhYiLi/PZ+XURDFkXkCwsLERYWJjCpSEiIiJ3lJSUID4+3m4haF/QRTBk7RoLCwtjMERERKQxvk5xYQI1ERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6ZriwdDGjRsxZMgQxMbGwmAwYOXKlU6PfeSRR2AwGDB79mzZykdERET+TfFgqLS0FF26dMGcOXNqPG7FihXYsmULYmNjZSoZERER6UGQ0gUYNGgQBg0aVOMxx44dw+OPP441a9Zg8ODBMpWMiIiI9EDxliFXzGYzxo0bh6eeegpJSUlKF4eIiIj8jOItQ668+uqrCAoKwhNPPOH2e8rLy1FeXm77uqSkxBdFIyIiIj+g6pahn3/+Ge+88w4yMjJgMBjcfl96ejrCw8Ntr/j4eB+WkoiIiLRM1cHQpk2bcOrUKTRr1gxBQUEICgrCkSNHMG3aNLRo0cLp+6ZPn47i4mLbq7CwUL5CExERkaaoupts3LhxSElJsds2cOBAjBs3DhMnTnT6PqPRCKPR6OviERERkR9QPBi6dOkSDhw4YPu6oKAA+fn5iIiIQLNmzdCoUSO74+vUqYPo6Gi0a9dO7qISERGRH1I8GPrpp5/Qv39/29dPPvkkAGD8+PHIyMhQqFRERESkF4oHQ8nJyRBCuH384cOHfVcYIiIi0h1VJ1ATERER+RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLoWpHQBiIiISGNMJmDTJuDECSAmBujbFwgMVLpUXmMwRERERO7LzAT++lfg6NFr2+LigHfeAYYPV65ctcBuMiIiInJPZiYwcqR9IAQAx45ZtmdmKlOuWmIwRERERK6ZTJYWISGq77NumzrVcpzGMBgiIiIi1zZtqt4iVJkQQGGh5TiNYTBERERErp04Ie1xKsJgiIiIiFyLiZH2OBVhMERERESu9e1rGTVmMDjebzAA8fGW4zRG8WBo48aNGDJkCGJjY2EwGLBy5UrbvqtXr+Lpp59Gp06dUK9ePcTGxuL+++/H8ePHlSswERGRHgUGWobPA9UDIuvXs2drcr4hxYOh0tJSdOnSBXPmzKm2r6ysDHl5eXj++eeRl5eHzMxM7N27F3feeacCJSUiItK54cOBZcuApk3tt8fFWbZrdJ4hgxCOxsgpw2AwYMWKFRg2bJjTY7Zv344ePXrgyJEjaNasmVvnLSkpQXh4OIqLixEWFiZRaYmIiHRKphmo5aq/NTcDdXFxMQwGAxo0aOD0mPLycpSXl9u+LikpkaFkREREOhEYCCQnK10KySjeTeaJy5cv4+mnn8bYsWNrjBDT09MRHh5ue8XHx8tYSiIiItISzQRDV69exahRoyCEwAcffFDjsdOnT0dxcbHtVVhYKFMpiYiISGs00U1mDYSOHDmC9evXu+w3NBqNMBqNMpWOiIiItEz1wZA1ENq/fz9ycnLQqFEjpYtEREREfkTxYOjSpUs4cOCA7euCggLk5+cjIiICMTExGDlyJPLy8vDNN9/AZDKhqKgIABAREYHg4GClik1ERER+QvGh9bm5uejfv3+17ePHj8cLL7yAli1bOnxfTk4Okt3MZOfQeiIiIu3RzdD65ORk1BSPqWgaJCIiIvJDmhlNRkREROQLDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI14KULgAREZGqmUzApk3AiRNATAzQty8QGKh0qUhCDIaIiIicycwE/vpX4OjRa9vi4oB33gGGD1euXCQpdpMRERE5kpkJjBxpHwgBwLFjlu2ZmcqUy5+YTEBuLrB4seVfk0mRYjAYIiIiqspksrQICVF9n3Xb1KmKVd5+ITMTaNEC6N8fuOcey78tWigSZCoeDG3cuBFDhgxBbGwsDAYDVq5cabdfCIEZM2YgJiYGoaGhSElJwf79+5UpLBER6cOmTdVbhCoTAigstBxHnlNZq5viwVBpaSm6dOmCOXPmONz/2muv4d///jfmzp2LrVu3ol69ehg4cCAuX74sc0mJiEg3TpyQ9jitk7I7S4WtboonUA8aNAiDBg1yuE8IgdmzZ+O5557D0KFDAQCfffYZoqKisHLlSowZM0bOohIR+R5HLqlDTIy0x2mZ1EnknrS6devm+fm9oHjLUE0KCgpQVFSElJQU27bw8HD07NkTmzdvdvq+8vJylJSU2L2IiFRPRTkUute3r6XCNxgc7zcYgPh4y3H+zBfdWSpsdVN1MFRUVAQAiIqKstseFRVl2+dIeno6wsPDba/4+HiflpOIqNZUlkOhe4GBlpYPoHpAZP169mz/brXzVXeWClvdVB0MeWv69OkoLi62vQoLC5UuEhGRcyrMoSBYuoCWLQOaNrXfHhdn2V7beYZUMqzcKV8lkbvR6ibi47G7OBYf3Pu9Z+f2kuI5QzWJjo4GAJw8eRIxlSLEkydPomvXrk7fZzQaYTQafV08IiJpeFLpJCfLViyCJeAZOlT6PC4tTOboq+4sa6vbyJGWgKjSHwECAITAQ8dm4ONhbQFEe3ZuL6m6Zahly5aIjo7GunXrbNtKSkqwdetW9OrVS8GSERFJSIU5FFRJYKAlCB071vKvFIGQFrpEfdmdNXw4rv5nCS6HRdptLkQ8RmA5PjZPQgj+QP+Gv3h+bi8o3jJ06dIlHDhwwPZ1QUEB8vPzERERgWbNmmHq1Kl4+eWX0aZNG7Rs2RLPP/88YmNjMWzYMOUKTUQkJRXmUJCPuOoSNRgsXaJDhyqfj2Ttzjp2zHF5DQbLfjeTyIVZYO/qAmTNL0TW96HIPXU7/sAI9MUmxOAETiAGxcZIpHQ6hawRP6PPQ4m4GnQ9wsMlvi8HFA+GfvrpJ/Tv39/29ZNPPgkAGD9+PDIyMvCPf/wDpaWleOihh3DhwgX06dMHq1evRkhIiFJFJiKSlsSVDtWSL6c30FKXaA3dWe4mkZ/Zexbr3t+LrO9MyDqUgKOmVgBa2fZHBZxCfPMgpKXFI+WRNojpGgUg0bb/aslVae/JCYMQjn7y/EtJSQnCw8NRXFyMsLAwpYtDRFSdtesEcFzpSJGwS675Opdn8WLLtAmuLFpk6ZZTA0efSXy8JRCq8plcuXQFm+fvQtbSC8jKj8TPZe0hKmXkGHEZt0TsRFqvi0ibEItOw9vAEOAkkRry1d8MhoiI1MKDSsdvKTnppDUgrVotShmQ5uZa5o9yJSdH+Zahypw8l+pdX4koxXV2b+0cshepSSeQNqI++j6ciNCIULcvy2BIQgyGiEgz9DwDtZIjrEwmywSXzrqwrF2VBQW1ex7W67jqEq3tdSpfT+Lvp7P7z2Hd+3uQ9a0JWQdbodBkP/VApOE00lrsRVoaKnV9eVf2ktWrEX7HHQyGpMBgiIhI5eRolamJNy023gYacnWJShRcutP11TdiJ9Juuoi08THoNLwNAoJqOVj9z7KXHD2KcMDn9bfiCdRERKRzahhh5en0BrUJNKyTOTp6v1Rdos6CS+vw/RoCLmEW2Lfmz66vTaHIOZWIUnS1O6ZTyD6kJR2v1PV1Q+3L7KrsPsSWISIi8pyU3S9qyKPxpAznzknTiuWrLlEvuvwqd31lH2yJ301xdm+JNJxGavN9SEsVSHmkNWK7+WgyxCplLwHYMkRERCokdW6PGiaddHd6g5tvBhISpGnFsk7mKDU3h+//728L8GVea2TlN8FPpR0gcLPtEMddX72lL6unZfcRBkNEROQ+V90vX3wBNGniWWuHGiaddHdOnR9/VP88QW4GjbPerYclSLZ93dG4H2kdjyHtruvQ9+FE1G0sYdeXuxSaZZ3BEBERucedBWXHjrVfcNSdFiO1TDrpTi7P4sXunUvJpVPcDBovIxT3tvihUtdXGwBtfFs2VxSaZZ3BEBERucedLoyqK6+7kbArxUzHknG1MKsaWrGcuHLpCrYs2I3sJSZMMUShiTiFAFQPLgWAq42isfzYYAQY68hezhq5Cox9hAnURETkHndnT67K3blztDDppNzzBDm6/p+BmoiKxv7L8chacAxZm0KQczIRl1AfAHAXMrEMluH7dgGRFmY0rzT1QIkQTKAmIiIvuDNKyZuRTJGRNe93xt08GletMmqgZCtWZiZMkx9HYNFxy+UAhCIO6/EOvoYlsGliOI3UZvuQlhqJC+0+QsQ7L/hu+L6vOOuu9CWhA8XFxQKAKC4uVrooRES1V1EhRE6OEIsWWf6tqLi2b/lyIeLihLBU05ZXXJxluyfHVOXoPZ6+Fi3y1SciP0efR3x8zZ+hF66UXhEb380Xi9q/KMwwCFOVz9QEgzADIvP6F8UvS/YI01WT/Qlq+l5Ru4oKUfzNN7LU3+wmIyLSkpqGtQPOJ6szGCx/bTs7pqbuE6kmwVPbelu15YN5goRZYH/2YWR9/Duyvw/B+qJElKEuDqMFmuIoHM7r7OuuOQVxbTIJMRgiIr9Q05IVQgCNGgFnzzp/f6NGQEiIJd/FEUeVqqsJ/Nzhx5W11yoFUiXm65D1U0NkrTYja38rHKky4eFQrMRK3OX6nP4WbEK++ps5Q0REWuDOsPaaAiF39jvK7antJHhyjwbTgIrFS2F67AkYL5wEAIQB6Ik4LMI7OIJbEIxy9Gm4E2k9S5B2fzS6mEqBcW6cWMnh/BrHYIiISAvknJm3cqVa2wq2aVN5Vp1XMWEWOLDuCLI+OoKS7M14+sKz1Ya8N8UxLMcI5I95FW3fmYx6kd2u7cwtcu9CCs3R4w8YDBERaYGcf/Xv2mVZq6tv39pXsBkZwIABUpRKU84XXMC693Yj+9uryDrQEocrWiAA8TiM+wC7Nd8tAmBZyuP6H94DGk2z36mWSSn9mMNcLCIiUhk5/+p/+WXLoqUtWgBnzlgqWmt3l6dOnZK0aGp1tewqvn//V8y4JRc3XfcbGreqj7vf6oUP99yCwxXxCEY5/nrdfMQ7S4IG7LspK7MO5weqPwd2Q0qCwRARkRZYWwecBSUGAyB1gumxY8CoUZYlNqzX8JSfdt1YR329P2YDhsVsRaN6f6Dv5M54aVMytpZ2hBmBSDQewNTrN+DbF7fj3MkKvPVhffdO7qgV0Dr3TtOm9tvj4tQ9gaJGsJuMiEgL3Jnsb/584JFHXCdKu8u6CvuSJZYFWJ980j5vKTCw+vIblcvkbteND4ao+8L5ggtY//5uZH1zresLaGHb39hwBqnN9iItRSDl4QTE3dgaQOtrJ6jtUh5amJRSoxgMERFphTsLiQYEACNGSHdNa9dNkybA4cP2FfHp08Do0deOs/Kk66ameZMUbu24WnYVWzN2I/uLc8jKa4RtlxJhRi/b/jq4gj4NdiKtZzHSxkWh6+h2CAjq7fyEZ864vmh8fM0BZGCgfMPnNRKkSoHzDBERaY2rSspRgNGkiSV48daiRde6yyqrzXpiNc2bBHjX/VOLClyYBQ7m/I6sjw4ja0MI1hd1wEXY1xmJxgNISzyG1KF10e/RRNSLrOd+udyZr2npUuDuu907py+pJEjlpIsSYjBERLpTNSg4dgy47z7vz1fThH7eBCCuggNvJmr0ogK/cKQY6+fsRtY3V5C1vwUKKprZ7W9kOIvU+L1IG2BC6qOtEXejlzlQubmWpHRX1DBxoi+CVC9x0kUiIn8lR/dD1e6U3FzvzuNO7o83XTeu5k1yd3FXK2cV+LFjlu1/VuBXy65i26e7kbWkctfXTbbDK3d9pd4XhevHtENA0M2e3Zsj7k6NoPTEia4m9zQYgKlTLblLftRlxmCIiEhOSnU/uJqrxhFPcn88DfCkDA5cVOACBlwa9ygmTI7C2qKOKEFnu0M6BB9EWuJRpA2ri1se7oDroq93r2yeqG3ytFykDlI1gsEQEZFc3Gy9cJsnAYir0WiO1jarnJjt6r7cCfAql/fkSffu0Z3gwEUFboBA/bJTOFt2FSUIt3V9pd5qQuojCYjvmQAgwb3yeEsrEydqpQVLarVf+F79iouLBQBRXFysdFGISK8qKoSIixPCUhVWfxkMQsTHW45zx/Ll1c8XF2fZ7un74uMt2ysqhMjJEWLRIsu/7pRl+XJL2R3dj8FwrTyOrhsYKMnnUZHxH+fnqfRa2ek58dN/dgnTVZPr+/IF62dV9fOq+lkpKSfHrc9S5OTIUhy56m8mUBMRyUHKBNraJrhKlbPkbhL0m29ahuB72j1Xw30cXG9Z6ytrQzCunDiD/2KI6/OqJTnZ29F3crA+U1ctWJ4kttcCE6iJiPyJVN0PUiS4SjVXjbv5JY89VnMgFBAAmM3XvnbQPXfhSDFy3v9z1Ne+5jhU0RxAc8vbYcIxxCIGxx0vq6CWLihA/RMnujO5px8u/cFgiIhIDlIl0KopwdXdAM/VZINmMxAeDkycaAkU+vZFxVWBbfN2IGvxWWT93AjbLnWAqcqor94NdiL1xmKk3ReJ6JB3EDBmlGWnkhW4O61uck6c6A13Jvd0l0YmbmQwREQkB6kSaNWU4CrlyKfiYojZ7yDrp0aYdzAE6050QAk62R3SPvgg0jpYRn31e6TqqK9EIEiiCtxbKpmoUBJStGBp6PNgzhARkVysuT6A49YLd0aTqWnyPnfySxo3dnvmazMMOIo4tEQBzAhEhOEcUuL2IO1WE1IfboVmvZq6PolSLREqmqhQFST6PDgDtYQYDBGRatQ2gVZlCa4uA7ylS4G//c2j+Y0W3vA22j+RhuvHtENgsPq6VKrxxWzanl5fTV1REn4ectXfDnPNiIjIR4YPtyx4mpNjWe8rJ8dSKbjbamBNcAWuBRxWSiS4WvNLmlZptYmLA5Ytw6HGPbCmxcMQQsDs+AzV3DctCt3vT9RGIAR4lscltcxMS+DRvz9wzz2Wf1u0sGxXipKfh5eYM0REJLfaJtBKmeAqhUr5JWU7DiJvczkW/68D1oxuhYMVzQA8h7uQiLl4GJFwY+V2pWdh9pRSeVxST+IpFTXltbmJwRARkRapZIh2xeUKbP9s95+jviKw9WIfmCpVLUG4ipvDd+KGGyNwZEw2mjyTCoOz0WVqGgLvCSWW2lDzGmJaWXqkEgZDRERapdAQ7YKNhciaV4Cs3GCsO94BxVVGfbULPoS09oVIGxqKfo90QP3Yrtd2NpxXc46RFuewUWKpDTVNsVCVVpYeqYTBEBHVTG3JmSS74t+LkfPBHmR9XY7sfc1w4GoLAPG2/Q0N55HSdA/Sbq1A6kMt0bx3KwCtHJ9MbV18UlBiokI1d0VpceJGny72IYGKigrx3HPPiRYtWoiQkBDRqlUr8c9//lOYzWa3z8G1yYi85O36V6RpV/+4KjZ/tEO82D9H9K7/PxGIq3bfAkG4Im4J/0W8nJIjtmXsFBXlbq6nVpk366CpXU3rvklNZWuIOSTB58G1yf70r3/9C2+99RY+/fRTJCUl4aeffsLEiRPxyiuv4IknnnDrHBxaT+QFzpuiK5W7vtafaI8LooHd/rZ1CpDW/nekDQ1F8qMdUD+2vjIFBdTdWilX2VxNsQBYrrt4MXD33dJf3121/Dw4z9Cf7rjjDkRFRWH+/Pm2bSNGjEBoaCgWLlzo1jkYDBF5SOl5U0g6TiqjkqMllrW+vr6MrL3N/+z6uqZ611ec19eSlIZmNfY5Z3+wVGYwaPoPF9nqb5+2O0nglVdeEc2bNxd79+4VQgiRn58vIiMjxcKFC90+B7vJiDykhSZ4cs1BN8UFYxPxbOibDru++obli5cG5Iitn/zmedeXHF2qy5cLYTBU/z40GCwvPXbffvmlEIGBzn9GDQZL15RGuyHlqr9Vn0D9zDPPoKSkBO3bt0dgYCBMJhNeeeUV3HvvvU7fU15ejvLyctvXJSUlchSVyH+oOTlTCmruZpFKZibEiJEABCpPzVi//Axewt+xGy2ws871Vbq+unh9LZ/Pd6PmoeRKatzY8tk4o+SoMg1RfTC0dOlSfP7551i0aBGSkpKQn5+PqVOnIjY2FuPHj3f4nvT0dLz44osyl5TIj2hwnhC3+XE3i7Xra+3XpXj2t8mIgqi2zEAABASApZGPI+j470Bgy9pdVK4gRc1DyZXk73+4yET1y3E89dRTeOaZZzBmzBh06tQJ48aNw9/+9jekp6c7fc/06dNRXFxsexUWFspYYiIVMZksC3suXmz5t6a/ICuzzhNSdbkHK4PBsp6WiuYJcYu1BaNqpWptwVByCQMvmK6YsHX+b3hpQC76hv8PEfF1MSy9J3b8FoAYFDn9BW8AEHTquDTLIci19IIaKn1vf558yZ//cJGR6luGysrKEBBg/yMdGBgIs9n5KjdGoxFGo9HXRSNyTcnumNq0gGhxnhBX/KSb5fD3R5E19xCyc+tg7fEOuCA62u1vU6cA46PXA+78DShF4CBXkKJ0pa/WFkUNTnCoSj7NSJLA+PHjRdOmTcU333wjCgoKRGZmpmjcuLH4xz/+4fY5mEBNilByjp6aEk0BIaZOdW9uF1/OmyL3PDMaTQovLiwWq57dIiZ3yhVt6hyqVtwGhvNiRNMfxbx7N4hDG363vEnOe5XrWhUVlu9FR9/Xvk4UVnvitrV8VcuolvLVglz1t+qDoZKSEvHXv/5VNGvWzDbp4v/93/+J8vJyt8/BYIhkp+QvT2ul4U4F5U5w5ougRYlAcdEi9z6TRYt8VwY3VJRXiC0f7xAvDcgRfcPyRRCu2BUvEFdFn7B88c9bc8SWj3c4HvUlZ+Ag57WUqPRd/TypZbSWnBM+yojBkIQYDJGslP7l6e5f6kr95ahUoKhUy5AbweTh7wvFh+M2iJFNfxQNDeeqFalNnUPisY65YuX0LaK40M3fY3IGDnJfS85KX0stin44qzeDIQkxGCJZKf3L090WECX+slUyUFSim8VJC1jZRwvFqme3iCmdckVbd7u+pLp+bQMHZxWunEGKnJW+RloU/RXnGSLSKqVHvXiaQCqE/ZBkXyZ9Kzk8unJSeFW+SAp3MveO+egxGB8chwwswwpYEm8DUYGb6u9CWvdzSLunMbrf1x5BIb1qX4bhwy0J4VI9T1dJxFJeqyaBgfINn1c6cZtk4VEwVFhYiPj4eNcHEumZ0r88XY0ucWbdOuD0aeDJJ303YkbpQBEAIiKAs2erb/vwQ+lGBZlMqHhkCgKF/YSHgGWeHzMMeA9TEJvUAKl31kPyI+0R3qyzNNeuSqrAwd2JFf1tjh+O1tIFj+YZat++PWbMmIGysjJflYdI+5Seo8faAmK9lrtefhkYNcq3c/AoGShaK/OqgRDgeJuHLh6/iK+f34bHO2/AOONSBJ0+US0QsgqAQCxO4L13BIb+qyfCm4XX+vo+5WpaAsAyLYEa5t2RWk0/T1qdZoKq8SgYys7Oxpo1a9CmTRtkZGT4qEhEGqeGX57Dh1v+Um/atPbnkrKyUypQrKkyt17Xw/szXTFh24KdeCU1F/0a5COiaQjufLkH3tvRDxXunmbUKPVM9FjThIJyTayoVs5+nuLiNL0IKlXiTaLRp59+KuLi4kS3bt3Exo0bpc5jkhwTqEkRahjqak00nTrVs6RqT5O+PUloVWJ4tERJ7Ud+PCo+un+juDvO8aivhKDD4tGkDWLT2DmeJbArPfzZ1VQHTCK28MPRWmqn+tFkpaWl4vnnnxf16tUTI0aMEIcOHZKyXJJiMESKUdMvT0cVnqcvR5WdN3MGyR0oelmZlxwrEV89t1U83jlXtAs+WO3wcFwQw2M3i7n3bBAHc45ce6OrkWtVgyEl56lxZ6oDpUdIkm7JVX8bhPAkw/KasrIy5OXlYdmyZXj33XcRHByMxx9/HDNmzMB1110nZeNVrZWUlCA8PBzFxcUICwtTujhEyjGZgBdesOQHeSMnxz5B1llSrbUbrKYuBDmXKsnNBfr3d3mYKWst8o7HIGvhKWRtb4Afi5NQgTq2/YGoQK/rduCBhA24uXs5EkZ1R+CAZMfldvbZOFP1s5WDyQS0aOG8C8yaHHzgAJCQ4DqJuKCAuTMkKbnqb4+Coblz52L79u3Yvn07du/ejYCAAHTs2BE33XQTunTpgiVLluDAgQPIzMxE9+7dfVZoTzEYIr8gVfDgZmBgx1Fl525FqoYK0lpWJ5W5AHA2MBIdTL/hDJrY7UsIOoK0dkeQNsSI1Bb7UO/lZ90fbZeZCTz4IHDunOsyLloEjB3r2X3VlrvfCzk5lnuwTktQ+TN0J/Al8pJs9bcnzUhxcXFixIgR4vXXXxebNm0SZWVl1Y555ZVXRFJSkiTNVlJhNxlpnpTLV3jShVNTLo/Wuk7+7A4yV7lvEwzCBIO4C8sFIEQYLoi7YjaLD8ZuEAfWHa72frc/H6u1a9X7OXnafah0Hpyaup1JFqrPGXKmqKhIBAQESH3aWmEwRJrmi+UrnCUxO3o5q+w0klRbUV4htmXsFC+n5Ijn6r4hfkdTu/IdQZyYHvKmeCE5R/w471dx9Y+rDk5Si5mzlVxg1BVvAlqlAhIlFz4mxag+Z6iGliZs3LgR/fr1k/K0tcJuMtIsX3ZFOZpNOD4eeOstoHFj191x7naxrF0LDBjgWdlq6ffNx5A97xCy1gdi7dH2OCcibPsCYMLogC8xIGYX2vaNRqdXxqBBq4gazgbPupMc5f1Y84cAdXUxueg+VE1XZ21y00jTVJkzpFUMhkizalsJu1KbPCRXFalV06bAv//t08rqUtEl5L6/C9lf/YGs3XHYcyXBbn8YijEgZjfSkq8gdVJzJNza3LMLLF4M3HOP6+NqyvtxFnzOnq1sRa7WQM1KS7lpJDm56m+uTUakZr5evqI2SzVUXuvLYHAeEB0/br9cgwRMV0z4Zcley6ivbQ3wY3EirqKHbX8ATOh53S6k3XAWaWMbocf4DggKucn7C0oxc7aca3d5wjqhoKM1x5QO1ABl17Mj3WAwRKRmSq9z5oqzirQyIa7N8Dx0qNeVf+HW48j64CCy1wdi7dF2OCsSASTa9rcM+h0D2x5G2h3B6P9YBzRo3smr6zgk1fpUci4w6gm1BmqAOtazI7/HYIick3MeGHJMC4tEDh8OhIcDKSnOj/Hir/dLRZewYe5uZK0sq9T1FWvbH4Zi3BqzG2n9riDtQWvXV7Na3YpTNbWC+cv6VGoN1NT+BwH5BQZD5Jij/AYpVy8n92ilEj51yr3javjr3VxhRt6iPcj+/BSytoXjhwtJuIobbfsrd32ljo5Aj/EdUKduLbq+PKX27iR/pYU/CEjzGAxRdc5GblhXL1c6oVJvtFAJe/nXe+HW48ieexBZ6xx3fbUIKsTANgVIuyMYt06WuOvLG2ruTvJXWvmDgDSNo8nIHkduqFdtuy192e3p5hDtS5t3YMNH+5C9qgxZu+Kwu8qor/oowYCYXUjr9+eor/7NYAhwssI96YtaR+ORT3FovYQYDHnA10O5SRlydHs6GaItYAAgMLPu65hV9gSuIti2LwAm9LhuF9K6nUXaGGvXVx3UGvPd/BOfq+5waD0pgyM3/I9c3Z5/dudVPDoFQaeufX8UIg5TMRsryizXsHZ9pQ6ug1sf64CGLSXu+mK+m/9Sa5I3aR6DIbLHkRvSUcNfsSaTJTBw1AAs0ZD30lOl2PDBLsuor11dsPdKIfpiE2JwAicQg3x0QXL0PszptwFpD7b4s+srvnb35Qzz3YjIC+wmI3tamZ6/NtwNUmoTzKildcIH3Z7mCjN+WbIX2QtPImtrOL6/kOS06yt1dAR6TpCo68sV5rsR+R12k5Ey/H3khrtBSm2CGTW1TkjU7Xl0+wlkf3DAMuqrsC3OiA4AOtj2Nw88ioFtDiHtDh91fbmDMxUTkZcYDFF1WhjK7Q13g5TaBDMydEt5xMtuz9JTpdg419r11RS7ylsDuHZMfZTg1ujdSO17GWkPNkfrAc1hCIiTsOBeYL4bEXmJ3WTknBpyXqTibhfKgQNAQoL3XS1qG43nZrenef9B5C87gKz/WLq+friQiCsw2g4LgAk31tuNtG5nkDZGxq4vT6jtsyeiWmM3GSnPnZEbWgmY3O1Cef/92nW1qK11ooZuTwHL17MvP4J/hRY77fpKvb0Obn2sPSISOspTZm9xpmIi8hKDIfKeWpKE3eFu8HHwYO3Op8bReMOHA198AfMjjyHg3BnbZtuQ99OWZ3UdLuLW6F1IU1PXlye0lu+mlT8kiHSAwRB5R6okYbkqBHeDj4QE18fUdD4VtU6YK8z435f7cPi1pej9vzmIFNcCoVNogml4A8frtcVz1+cibXRD3PSXRNSp29Pn5fIpreS7aekPCb1hkKpPQgeKi4sFAFFcXKx0UfxDRYUQcXFCWKr76i+DQYj4eMtxNVm+vPp54uIs231VZoOh5jKXl7t3XE33tny55biq57Bu88X9/eno9uNiwQObxNjm34smhlPiLiwXJhiEqcp9mAFh9nFZFFVRIUROjhCLFln+dfW9KCfr94ej7y1/fiZaIOfvJHKLXPU3gyHyXE6O80Co8isnx/k5lKgQ3A1SpAhmHP1SjY+X/L5KT5eK717aLv7WLUckGffZXS4AFaIQscJc26CVpCPVHxIkPQapqiRX/c3RZOS5xYuBe+5xfdyiRcDYsdW3Kzk5nruLPUqxKKQPmtutXV9ZnxUha2sYvj+fZDfqywAzbqy3C2nXn8Hd3Q6g878fdH1Sjq6SD0e8qRMn7FQtjiYj9aptkrCSk+MNH26Z48dVkOLucTWRaB2l43lFlgkPsw3I/r0tTov2ANrb9jcLPIq01oeQdnsdDJhcadTX4hPAv924AOfdkY/aRhuSBSfs1D0GQ+S52iYJK10huBukKLQoZNmZMsuEhysuIWtnU+wsbwMg2rb/OlxE/6g/R31NaoY2qS0cj/pS48g2veMzUSelfyeR4hgMkedqO4RZTxWCG11l5gozfl2+H1mfnkDWljBsOp+EK+hu22+AGd3r7UZa19OWUV8TOyD4OjdGfaloZJtsI3TUPhJITc+ErtHT7yRyzKcZSSrBBGof8TZJ2N2RXVpPIq1hZMqxn0+IjEmbxD3NvxeRhlPVPoL4wKNiUrsNYunffhRn9p2tXRkUGtlmVwY5RuhoZSSQGp4J2dPL7yQN4mgyCTEY8iFvhzD7e4XgZGSKGRAmGMRdWG63qx4uijsit4p/j8gVe749KMwms7RlkWFkm9NryzFCR2sjgZR8JuSYv/9O0iiOJpMQR5OplBQjttTIZIL4c2SKwcFuMww4ijiMqvsNUq4/V6nrK9inZZK9+0iuETpKjgSqzeeq9i49PfLX30kaJlf9zWCIlOVHFcKJ/JPIfn8/Cr/6Bf938gnXb/D34dNyDSNXarg6Z5H2T370O8kfcGh9JceOHcPTTz+N7777DmVlZWjdujUWLFiA7t27u34zqZtCI7akUHamDJvmWUZ9Ze+MxY7LbQFEYQwK3TuBv49MkWuEjhIjgaRajqYqVsTK0/DvJPKe6oOh8+fPo3fv3ujfvz++++47NGnSBPv370fDhg2VLhr5uyoVk7lXb+xYdejPUV/1selcEsqrjPq6oe4eDGh5Etjpxvn9fWSKXCN05B4JZDJZWoQcNaoLYemWmzrVMk+VJ4EMW5qIFKP6brJnnnkGP/zwAzZt2uT1OdhNplJq/ivYQcV0HDGYgvewAtcqprjA4xiYcBBptwdiwGPt0ahNxLUcFlfDp/19Nlu5Pge5P29fdMs5a2myTlXhbUsTkcbJVX8H+OzMEvnqq6/QvXt33H333YiMjMT111+Pjz76qMb3lJeXo6SkxO5FKpOZaanA+ve3LO3Rv7/l68xMRYv1x7k/kH/v6xAjRsBcJSE3GkVYhpF4OexVvDN8A3b/9xB+vxKDj/f2xai3b7YEQsC1eZiAa5WZlTvzMPkLuT4HuT9vqbvlXLU0AZaWJpPJvfMRked8OlZNAkajURiNRjF9+nSRl5cn5s2bJ0JCQkRGRobT98ycOVMAqPbi0HqVUNEwaLPJLPK/2CNeuz1HpEb8JEJxSfyOuGqrvNuGxnsy3wiHT1vI9TnIdR0pFir25fmI/AiH1v8pODgY3bt3x48//mjb9sQTT2D79u3YvHmzw/eUl5ejvLzc9nVJSQni4+PZTaYGKlgQsejXU8h+fx+ysoDsI21x0hxp29cPuciFhF0gau4KlJM/zUAtdbdcbRc+JvJjHE32p5iYGCQmJtpt69ChA5YvX+70PUajEUaj0el+UpACCyL+ce4PbJq3C9mZF5H1Wwx+vdwOwLUAqC5K0T9yJ9L6/IERrfKBN9w4qbtdIEqMTFFjACbX5yDHdWq7HE1VXAqCSHGqD4Z69+6NvXv32m3bt28fmjdvrlCJVEyNlWBVMgyDFmaBHZn7kZVxHFmb62PjuSSU4wbbfgPM6FZ3D9K6nELaqAbo9ZcOMIb1sOzMFe4FQ2qtmDgiSR7Dh1uSmh191p5O0Mf1yoiU59NOOAls27ZNBAUFiVdeeUXs379ffP7556Ju3bpi4cKFbp9DF8txaGVdJh/lR5z430nx2cObxLhWm0R0QFG108UFHhN/abNRLHniB3F6zxnnJ9LyGkUqysXSDW+Xo6mKS0EQOcS1ySr5+uuvRceOHYXRaBTt27cXH374oUfv9/tgSEuVoETBRtnZMpGV/pP4e/cc0TlkT7XT1MUlcXuTbWL2Xbli19cHPFvrS4sVk/VzdRZcqjmIIwsm3BNVwwRqCXmUgKWFrqbKVJCQ7DHrnCqA43wLB3OqWLu+sj89jqzN12Hj2SRcQTD6YhNicAInEIOy0EZI6XoWqSPDcfOkRBjDapE3JtcaRVJ9vym1JAVJS2u/f4h8jAnUSpAr30LKX3gKJCTXmpv5FkW/nsLaD/YjK0sg+3AbFJnbAmgLALgLmXgPUxCLSrlFjeKAv78DDE+WpoxDh/q2YpLy+02JJSlIelwKgkgRDIasfLXWkKPrSBlwabUSdBBs/JHUHd/P34vsHrnI2hGD/zkY9ZXcZBcebbkag7fNhGX6qEqkfla+rJik/n7jiCQiIq+xmwyQr6vJF1Pua7h7RJgFflthHfVl6fq6jFC7Y7qF7kZal5NIu/vPrq96QdrrFqzKF99vXAKEiPyQXN1kDIYAeQIKXwVcGqsET/52Gmvf34esNQJZh9ugyBxlt79pwAmkJRxA6sBApDzWFk06NLY/gYaDPxtf3YMXuVhERGrGnCE5ydHV5KvcHqkngJPY5QuX8f2Hu5C1rKRS11cT2/5QlCG5yU6k9S5D2sSm6HBHAgwBNXTlyNkt6KtkVl/dg5Rz3xAR6QiDIUCefAtfVuIqqgStXV/Znx1H1g/XYcPZJFxGN7tjrF1fqSPD0fvBRBjDbnT/AnLlxvgymd6X9yBH4jcRkZ9hNxkgT1eTXF1xClSClbu+sg+3xglztN3+2IATSGt1AGm3BWLAI20QmdTEyZncIMez8kVuV2Ua69okIlIKc4Yk5NaH6et8C7VWgF4EUJcvXMYPH1m7vqKR/0d7u/3Wrq/Um0uRNjEOiUMSYAgwSFdmXz4ruZPpAe3k93AOHCKSmVzBkCZmoK4tt2ew9PUMsGqb2djNJTzMJrPYkblPvHlnjhjYaLsIRWm1CY6vD90lnu6ZI9a9kSf+OP+HMmWX4ln5aLkQWe/BF7Sy3AsR+RXOQC0hVc1ALdfMxu6Uo4auoOK35uO/+9taRn0VVO/6igkoQlrLA0gbaEDKY21r1/XlLV88q8WLgXvucX3cokXA2LG1uxagjdYWX3cbEhE5wW4yCcnWzOYupStAF11BZhhwFHFoiQKYYSlXKMrQr/FOpPUuRer4pkga2lrari+18Ieh+1LS4nIvROQ3OLTenyk95b6LYf4BEGiGQkwM/hyNujZD2sgw9H4wESENPBj1pVV9+1oqd1e5XX37yl82JWhxuRciIg8xGNKRUzsto76Kl+XiUTeO/zijDjA22dfFUheVz9skO60u90JE5IEApQtAVZhMlq6axYst/5pMXp+qvKQc69/8Bc/clItudXcjqmMT3Pt+b3xxKtm9E+h1HSvrvE1Nm9pvj4vTX34M1zwjIh1gy5Ca1HKiP2EW2PX1QWQtOIqsH+phw5kk/IHr7Y7pGroHN3U04fK+SBhLTsOgpq4gpXOpKuPkhRbsNiQiHWAwpBZermJ+evcZy4SHq03IOtQax82tAbS27Y8OOIm0Fvtto76iOrYH0B7I/EBdXUG+nPHZW0rndqkBuw2JSAc4mkwNPBixU15aYZnw8MtiZP0ahV/+6GB3aAj+QL9GO5HW+xJS749Fx7vaOB/15ckwf1+22nDotvqpZUoIItIVDq2XkOqDITeHcz8d/gHeKx6HMtSz294lZC/SOp1A2sgw9HkoESENQty/tjtBji9bbTh0WzvU1I1JRLrAofV64uZInN+Lw1GGeog1HMeU6OXom3gWiWO7IGLCnUBgO++u7aoryMvuO7dx6LZ2sNuQiPwUgyG51PBX9ZWwRgh24xTDbjqJWX3eQ7Mlr8Jw9ChwAsA6AC/4KLfGZLK0CDlqPBTC0mozdaol0djbFgKlh26ztYOISPcYDMnBQTfT1UbRWJ3wGD4ouA2bTt+EXYhDUxxDAKoHHsJggCEuDqOnxQGjRvmulaYqOVptlBy6rcakbSIikh3nGfK1P7uZRJWgIvDsSQzeNhMhpwtxCWGYafgnDBDVQyGDAQYAePNN4G9/c95KA1haaWoxL1E1crTaWIduG5wkeRsMlkRdqYduW7v/qgZ71sAyM1Pa6xERkWoxGPKR8pJy5LzxEy6MfQRmIVC1qre0AAlkhDyC/y3ZjfkVE2BYvhyGuDj7A60T/TVp4n4rjVTkaLWxDt0GqgdEvhq67ar7D5A+sCQiItViMCQRYRbY/c1BvDN8AwZHbkdEeAVefOoSGlw57fRDDgAQdvk0OkedtAx/Hz4cOHzYsgjookWWfwsKLNvlzq0xmSyviAjnx0jVaiP3jM+edP8REZHfY85QLZzZexZr5+xF9moTsg4l4KgpAUCCbX97wx44SAGqrnIA42zEjpy5NY5yaaqSutVGzhmflU7aJiIiVWEw5IErl67gx493IWvpBWT9LxJ5Ze0hcLNtvxGXcUvETqT1uoi0CbHoFNEOGODGid0JYORaFsHZUPqq4uKkn3BPrqHbXG+LiIgq4aSLNRBmgT3fHkLWJ0eR/UMock8lohTX2R3TOWQvUpNOIG1EffR9OBGhEaHXdlonFHQVwLg7oaA1UAEcL4tQ2y4lVxMgAkCjRsAXX1iCFq0OQZf6uRARkU9w0kWFnNl7Fuve34us7xx3fUUaTiOtxV6kpQEpj7RBTNd2AJxMeCj1uk7W3BpHw8GlaKVxlUsDAGfPWsqr5SCB620REVElug+GrF1f2V9eQFZ+JH520PXVN2In0m66iLTxMeg0vA0Cgvq4fwGpAxhf5tboKZfG14ElERFphu66yepfVx97Vxcga34hsr533PXVKWQf0pKOO+768pYWZjp2c4005OT4z7IMWnguREQ6xYVaJWT9MMclfIfcw51QaLIfwh1pOI3U5vuQliqQ8khrxHaLVqikXpCyMmcuDRERqQhzhnzgPwdvBhDmpOurt9LF85zUy0kwl4aIiHRIVy1Dk7t+gyEjo9D34UTUbVxX2ovI3d3ibAi8FCPLHAVZ8fHMpSEiIlmxm0xCPv8w5V7w09UQeCm6s5hLQ0RECmM3mVY4a6Hx1UrygDyrycs1ASIREZHCuDZZbSi14KeehsATERH5GIOh2lBqwU8uJ0FEWmIyWabuWLzY8q/UfyAS1RKDodpQqoXGuk6ZNVm6KqlWkyciqq3MTEuOY//+wD33WP5t0cKynUglNBcMzZo1CwaDAVOnTlW6KMq10FiHwAPVAyIOgScitbDmVFZtQbfmVDIgIpXQVDC0fft2zJs3D507d1a6KBZKttBYl5Noaj+BJOLifJO0TUTkCaVyKom8oJlg6NKlS7j33nvx0UcfoWHDhkoXx0LpFprhw4HDhy3LYyxaZPm3oICBEBEpT6mcSiIvaCYYmjx5MgYPHoyUlBSXx5aXl6OkpMTu5TNKt9BYh8CPHWv5V6rAiwmPRFQbHPVKGqKJeYaWLFmCvLw8bN++3a3j09PT8eKLL/q4VJUMHQqEh1uCBsASlEgZmMhN7kkkicj/cNQraYjqZ6AuLCxE9+7dkZ2dbcsVSk5ORteuXTF79myH7ykvL0d5ebnt65KSEsTHx/tmBkt/Cxx8ucwHEekHF34mCXA5jj+tXLkSd911FwIr/bCYTCYYDAYEBASgvLzcbp8jPvsw5Q4cfL1EhhzLfBCRflh/RwKOF37mH1fkglzBkOpzhgYMGIAdO3YgPz/f9urevTvuvfde5OfnuwyEfEbukRJyzNXBhEcikpLSOZVEblJ9zlD9+vXRsWNHu2316tVDo0aNqm2Xlbfrg3nTuiPX+mdMeCQiqQ0fbsmr5MLPpGKqD4ZUy5vAwZv8IlctUAaDpQVq6NDa/3JxN5ExMrJ21yEifeHCz6Ryqu8mcyQ3N9dp8rRsPB0p4e1MrHJ2XbmaRNJq/HjOHEtERH5Dk8GQKngy+3Rt8ovk7LqqaRLJyo4f51T6RETkNxgMecuT2adr07oj91wdzhIeK+NU+kRE5EcYDNWGuyMlatO6o8T6Z8OHAxkZNR/DkWVEROQnmEBdW+6MlKhN6461BWrkSEvg42iuDl+sf3bqlHvHcWQZERFpHFuGpOBqfbDatu4oMVcHp9InIiKdUP0M1FKQawbLGkkxE6uvZ6Cuei1OpU9ERAriDNT+RorWHV+tUO/sWu4miBMREWkYW4bkJmfrjhQcTRQZH28JhDiVPhER+RAXapWQqoIhLdJaAEdERH5Brvqbo8nINU6lT0REfow5Q0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdU30wlJ6ejhtvvBH169dHZGQkhg0bhr179ypdLCIiIvITqg+GNmzYgMmTJ2PLli3Izs7G1atXkZaWhtLSUqWLRkRERH7AIIQQShfCE6dPn0ZkZCQ2bNiAW265xa33lJSUIDw8HMXFxQgLC/NxCYmIiEgKctXfQT47s48UFxcDACIiIpweU15ejvLyctvXJSUlPi8XERERaZPqu8kqM5vNmDp1Knr37o2OHTs6PS49PR3h4eG2V3x8vIylJCIiIi3RVDfZo48+iu+++w7ff/894uLinB7nqGUoPj6e3WREREQawm6yKqZMmYJvvvkGGzdurDEQAgCj0Qij0ShTyYiIiEjLVB8MCSHw+OOPY8WKFcjNzUXLli2VLhIRERH5EdUHQ5MnT8aiRYuwatUq1K9fH0VFRQCA8PBwhIaGKlw6IiIi0jrV5wwZDAaH2xcsWIAJEya4dQ4OrSciItIe5gz9SeWxGhEREWmcpobWExEREUmNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka5pJhiaM2cOWrRogZCQEPTs2RPbtm1TukhERETkBzQRDH3xxRd48sknMXPmTOTl5aFLly4YOHAgTp06pXTRiIiISOM0EQy99dZbePDBBzFx4kQkJiZi7ty5qFu3Lj755BOli0ZEREQap/pg6MqVK/j555+RkpJi2xYQEICUlBRs3rxZwZIRERGRPwhSugCunDlzBiaTCVFRUXbbo6KisGfPHofvKS8vR3l5ue3r4uJiAEBJSYnvCkpERESSstbbQgifXkf1wZA30tPT8eKLL1bbHh8fr0BpiIiIqDbOnj2L8PBwn51f9cFQ48aNERgYiJMnT9ptP3nyJKKjox2+Z/r06XjyySdtX1+4cAHNmzfH77//7tMPU21KSkoQHx+PwsJChIWFKV0c2fC+ed96wPvmfetBcXExmjVrhoiICJ9eR/XBUHBwMG644QasW7cOw4YNAwCYzWasW7cOU6ZMcfgeo9EIo9FYbXt4eLiuvomswsLCeN86wvvWF963vuj1vgMCfJvirPpgCACefPJJjB8/Ht27d0ePHj0we/ZslJaWYuLEiUoXjYiIiDROE8HQ6NGjcfr0acyYMQNFRUXo2rUrVq9eXS2pmoiIiMhTmgiGAGDKlClOu8VcMRqNmDlzpsOuM3/G++Z96wHvm/etB7xv3963Qfh6vBoRERGRiql+0kUiIiIiX2IwRERERLrGYIiIiIh0jcEQERER6Zomg6E5c+agRYsWCAkJQc+ePbFt27Yaj//yyy/Rvn17hISEoFOnTvj222/t9gshMGPGDMTExCA0NBQpKSnYv3+/L2/BK57c90cffYS+ffuiYcOGaNiwIVJSUqodP2HCBBgMBrvXbbfd5uvb8Jgn952RkVHtnkJCQuyO8cfnnZycXO2+DQYDBg8ebDtGC89748aNGDJkCGJjY2EwGLBy5UqX78nNzUW3bt1gNBrRunVrZGRkVDvG098ZcvP0vjMzM5GamoomTZogLCwMvXr1wpo1a+yOeeGFF6o97/bt2/vwLjzn6X3n5uY6/D4vKiqyO87fnrejn12DwYCkpCTbMVp43unp6bjxxhtRv359REZGYtiwYdi7d6/L98lRh2suGPriiy/w5JNPYubMmcjLy0OXLl0wcOBAnDp1yuHxP/74I8aOHYsHHngAv/zyC4YNG4Zhw4bht99+sx3z2muv4d///jfmzp2LrVu3ol69ehg4cCAuX74s12255Ol95+bmYuzYscjJycHmzZsRHx+PtLQ0HDt2zO642267DSdOnLC9Fi9eLMftuM3T+wYsM7RWvqcjR47Y7ffH552ZmWl3z7/99hsCAwNx99132x2n9uddWlqKLl26YM6cOW4dX1BQgMGDB6N///7Iz8/H1KlTMWnSJLvAwJvvIbl5et8bN25Eamoqvv32W/z888/o378/hgwZgl9++cXuuKSkJLvn/f333/ui+F7z9L6t9u7da3dfkZGRtn3++Lzfeecdu/stLCxEREREtZ9vtT/vDRs2YPLkydiyZQuys7Nx9epVpKWlobS01Ol7ZKvDhcb06NFDTJ482fa1yWQSsbGxIj093eHxo0aNEoMHD7bb1rNnT/Hwww8LIYQwm80iOjpavP7667b9Fy5cEEajUSxevNgHd+AdT++7qoqKClG/fn3x6aef2raNHz9eDB06VOqiSsrT+16wYIEIDw93ej69PO+3335b1K9fX1y6dMm2TQvPuzIAYsWKFTUe849//EMkJSXZbRs9erQYOHCg7evafpZyc+e+HUlMTBQvvvii7euZM2eKLl26SFcwH3PnvnNycgQAcf78eafH6OF5r1ixQhgMBnH48GHbNq09byGEOHXqlAAgNmzY4PQYuepwTbUMXblyBT///DNSUlJs2wICApCSkoLNmzc7fM/mzZvtjgeAgQMH2o4vKChAUVGR3THh4eHo2bOn03PKzZv7rqqsrAxXr16ttthdbm4uIiMj0a5dOzz66KM4e/aspGWvDW/v+9KlS2jevDni4+MxdOhQ7Ny507ZPL897/vz5GDNmDOrVq2e3Xc3P2xuufr6l+Cy1wGw24+LFi9V+vvfv34/Y2Fi0atUK9957L37//XeFSiitrl27IiYmBqmpqfjhhx9s2/XyvOfPn4+UlBQ0b97cbrvWnndxcTEA1LgIq1x1uKaCoTNnzsBkMlVbhiMqKqpan7FVUVFRjcdb//XknHLz5r6revrppxEbG2v3DXPbbbfhs88+w7p16/Dqq69iw4YNGDRoEEwmk6Tl95Y3992uXTt88sknWLVqFRYuXAiz2Yybb74ZR48eBaCP571t2zb89ttvmDRpkt12tT9vbzj7+S4pKcEff/whyc+OFrzxxhu4dOkSRo0aZdvWs2dPZGRkYPXq1fjggw9QUFCAvn374uLFiwqWtHZiYmIwd+5cLF++HMuXL0d8fDySk5ORl5cHQJrflWp3/PhxfPfdd9V+vrX2vM1mM6ZOnYrevXujY8eOTo+Tqw7XzHIc5L1Zs2ZhyZIlyM3NtUsmHjNmjO3/nTp1QufOnZGQkIDc3FwMGDBAiaLWWq9evdCrVy/b1zfffDM6dOiAefPm4aWXXlKwZPKZP38+OnXqhB49etht98fnTcCiRYvw4osvYtWqVXa5M4MGDbL9v3PnzujZsyeaN2+OpUuX4oEHHlCiqLXWrl07tGvXzvb1zTffjIMHD+Ltt9/Gf/7zHwVLJp9PP/0UDRo0wLBhw+y2a+15T548Gb/99ptq8po01TLUuHFjBAYG4uTJk3bbT548iejoaIfviY6OrvF467+enFNu3ty31RtvvIFZs2YhKysLnTt3rvHYVq1aoXHjxjhw4ECtyyyF2ty3VZ06dXD99dfb7snfn3dpaSmWLFni1i8/tT1vbzj7+Q4LC0NoaKgk30NqtmTJEkyaNAlLly6t1pVQVYMGDdC2bVtNP29HevToYbsnf3/eQgh88sknGDduHIKDg2s8Vs3Pe8qUKfjmm2+Qk5ODuLi4Go+Vqw7XVDAUHByMG264AevWrbNtM5vNWLdunV1rQGW9evWyOx4AsrOzbce3bNkS0dHRdseUlJRg69atTs8pN2/uG7Bk2L/00ktYvXo1unfv7vI6R48exdmzZxETEyNJuWvL2/uuzGQyYceOHbZ78ufnDViGoJaXl+O+++5zeR21PW9vuPr5luJ7SK0WL16MiRMnYvHixXZTKDhz6dIlHDx4UNPP25H8/HzbPfnz8wYso7EOHDjg1h87anzeQghMmTIFK1aswPr169GyZUuX75GtDvco9VsFlixZIoxGo8jIyBC7du0SDz30kGjQoIEoKioSQggxbtw48cwzz9iO/+GHH0RQUJB44403xO7du8XMmTNFnTp1xI4dO2zHzJo1SzRo0ECsWrVK/Prrr2Lo0KGiZcuW4o8//pD9/pzx9L5nzZolgoODxbJly8SJEydsr4sXLwohhLh48aL4+9//LjZv3iwKCgrE2rVrRbdu3USbNm3E5cuXFblHRzy97xdffFGsWbNGHDx4UPz8889izJgxIiQkROzcudN2jD8+b6s+ffqI0aNHV9uuled98eJF8csvv4hffvlFABBvvfWW+OWXX8SRI0eEEEI888wzYty4cbbjDx06JOrWrSueeuopsXv3bjFnzhwRGBgoVq9ebTvG1WepBp7e9+effy6CgoLEnDlz7H6+L1y4YDtm2rRpIjc3VxQUFIgffvhBpKSkiMaNG4tTp07Jfn/OeHrfb7/9tli5cqXYv3+/2LFjh/jrX/8qAgICxNq1a23H+OPztrrvvvtEz549HZ5TC8/70UcfFeHh4SI3N9fu+7asrMx2jFJ1uOaCISGEePfdd0WzZs1EcHCw6NGjh9iyZYttX79+/cT48ePtjl+6dKlo27atCA4OFklJSeK///2v3X6z2Syef/55ERUVJYxGoxgwYIDYu3evHLfiEU/uu3nz5gJAtdfMmTOFEEKUlZWJtLQ00aRJE1GnTh3RvHlz8eCDD6rqF4aVJ/c9depU27FRUVHi9ttvF3l5eXbn88fnLYQQe/bsEQBEVlZWtXNp5Xlbh05XfVnvdfz48aJfv37V3tO1a1cRHBwsWrVqJRYsWFDtvDV9lmrg6X3369evxuOFsEwxEBMTI4KDg0XTpk3F6NGjxYEDB+S9MRc8ve9XX31VJCQkiJCQEBERESGSk5PF+vXrq53X3563EJbh4qGhoeLDDz90eE4tPG9H9wzA7mdWqTrc8GcBiYiIiHRJUzlDRERERFJjMERERES6xmCIiIiIdI3BEBEREekagyEiIiLSNQZDREREpGsMhoiIiEjXGAwRERGRrjEYIiIiIl1jMERERES6xmCIiDRp8eLFCA0NxYkTJ2zbJk6ciM6dO6O4uFjBkhGR1nBtMiLSJCEEunbtiltuuQXvvvsuZs6ciU8++QRbtmxB06ZNlS4eEWlIkNIFICLyhsFgwCuvvIKRI0ciOjoa7777LjZt2sRAiIg8xpYhItK0bt26YefOncjKykK/fv2ULg4RaRBzhohIs1avXo09e/bAZDIhKipK6eIQkUaxZYiINCkvLw/JycmYN28eMjIyEBYWhi+//FLpYhGRBjFniIg05/Dhwxg8eDCeffZZjB07Fq1atUKvXr2Ql5eHbt26KV08ItIYtgwRkaacO3cON998M5KTkzF37lzb9sGDB8NkMmH16tUKlo6ItIjBEBEREekaE6iJiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREuvb/zRf6FBIXu6AAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "# Importing various packages\n",
    "from math import exp, sqrt\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.inv(X.T @ X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "\n",
    "#while (iter <= Ni... or test)\n",
    "for iter in range(Niterations):\n",
    "    gradients = 2.0/n*X.T @ ((X @ theta)-y)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "# Can you figure out a better way of setting up the contributions to each batch?\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (2.0/M)* xi.T @ ((xi @ theta)-yi)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        theta = theta - eta*gradients\n",
    "print(\"theta from own sdg\")\n",
    "print(theta)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de04b41a",
   "metadata": {},
   "source": [
    "In the above code, we have use replacement in setting up the\n",
    "mini-batches. The discussion\n",
    "[here](https://sebastianraschka.com/faq/docs/sgd-methods.html) may be\n",
    "useful."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77fc1cca",
   "metadata": {},
   "source": [
    "## Momentum based GD\n",
    "\n",
    "The stochastic gradient descent (SGD) is almost always used with a\n",
    "*momentum* or inertia term that serves as a memory of the direction we\n",
    "are moving in parameter space.  This is typically implemented as\n",
    "follows"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "441d1f36",
   "metadata": {},
   "source": [
    "$$\n",
    "\\mathbf{v}_{t}=\\gamma \\mathbf{v}_{t-1}+\\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t) \\nonumber\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47434945",
   "metadata": {},
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"_auto1\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation} \n",
    "\\boldsymbol{\\theta}_{t+1}= \\boldsymbol{\\theta}_t -\\mathbf{v}_{t},\n",
    "\\label{_auto1} \\tag{1}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3ea5060",
   "metadata": {},
   "source": [
    "where we have introduced a momentum parameter $\\gamma$, with\n",
    "$0\\le\\gamma\\le 1$, and for brevity we dropped the explicit notation to\n",
    "indicate the gradient is to be taken over a different mini-batch at\n",
    "each step. We call this algorithm gradient descent with momentum\n",
    "(GDM). From these equations, it is clear that $\\mathbf{v}_t$ is a\n",
    "running average of recently encountered gradients and\n",
    "$(1-\\gamma)^{-1}$ sets the characteristic time scale for the memory\n",
    "used in the averaging procedure. Consistent with this, when\n",
    "$\\gamma=0$, this just reduces down to ordinary SGD as discussed\n",
    "earlier. An equivalent way of writing the updates is"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "923628c8",
   "metadata": {},
   "source": [
    "$$\n",
    "\\Delta \\boldsymbol{\\theta}_{t+1} = \\gamma \\Delta \\boldsymbol{\\theta}_t -\\ \\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t),\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c94031c",
   "metadata": {},
   "source": [
    "where we have defined $\\Delta \\boldsymbol{\\theta}_{t}= \\boldsymbol{\\theta}_t-\\boldsymbol{\\theta}_{t-1}$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3f0e9c9",
   "metadata": {},
   "source": [
    "## Algorithms and codes for Adagrad, RMSprop and Adam\n",
    "\n",
    "The algorithms we have implemented are well described in the text by [Goodfellow, Bengio and Courville, chapter 8](https://www.deeplearningbook.org/contents/optimization.html).\n",
    "\n",
    "The codes which implement these algorithms are discussed after our presentation of automatic differentiation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92253eff",
   "metadata": {},
   "source": [
    "## Practical tips\n",
    "\n",
    "* **Randomize the data when making mini-batches**. It is always important to randomly shuffle the data when forming mini-batches. Otherwise, the gradient descent method can fit spurious correlations resulting from the order in which data is presented.\n",
    "\n",
    "* **Transform your inputs**. Learning becomes difficult when our landscape has a mixture of steep and flat directions. One simple trick for minimizing these situations is to standardize the data by subtracting the mean and normalizing the variance of input variables. Whenever possible, also decorrelate the inputs. To understand why this is helpful, consider the case of linear regression. It is easy to show that for the squared error cost function, the Hessian of the cost function is just the correlation matrix between the inputs. Thus, by standardizing the inputs, we are ensuring that the landscape looks homogeneous in all directions in parameter space. Since most deep networks can be viewed as linear transformations followed by a non-linearity at each layer, we expect this intuition to hold beyond the linear case.\n",
    "\n",
    "* **Monitor the out-of-sample performance.** Always monitor the performance of your model on a validation set (a small portion of the training data that is held out of the training process to serve as a proxy for the test set. If the validation error starts increasing, then the model is beginning to overfit. Terminate the learning process. This *early stopping* significantly improves performance in many settings.\n",
    "\n",
    "* **Adaptive optimization methods don't always have good generalization.** Recent studies have shown that adaptive methods such as ADAM, RMSPorp, and AdaGrad tend to have poor generalization compared to SGD or SGD with momentum, particularly in the high-dimensional limit (i.e. the number of parameters exceeds the number of data points). Although it is not clear at this stage why these methods perform so well in training deep neural networks, simpler procedures like properly-tuned SGD may work as well or better in these applications.\n",
    "\n",
    "Geron's text, see chapter 11, has several interesting discussions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08209015",
   "metadata": {},
   "source": [
    "## Using Automatic differentation with OLS\n",
    "\n",
    "We conclude the part on optmization by showing how we can make codes\n",
    "for linear regression and logistic regression using **autograd**. The\n",
    "first example shows results with ordinary leats squares."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f1f7d4aa",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.32147704]\n",
      " [2.78166226]]\n",
      "Eigenvalues of Hessian Matrix:[0.32306518 4.38577204]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own gd\n",
      "[[4.32147704]\n",
      " [2.78166226]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHHCAYAAAC88FzIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABZmUlEQVR4nO3deVxU5f4H8M8AsmiAoiIgIyBugFtpkqaJiUzmNU3NsjK1tE3vzZt1y0qtX3Wx3W5q6q2kTcwFtbrV4AIu5XYzb+6ioiHuG6sizDy/P6YZGZiBmWGWc+Z83q/XvJBzzpx5njkznK/P832eRyWEECAiIiJSKB9PF4CIiIjIkxgMERERkaIxGCIiIiJFYzBEREREisZgiIiIiBSNwRAREREpGoMhIiIiUjQGQ0RERKRoDIaIiIhI0RgMEZHNxo8fj9jYWE8XQ9Jyc3OhUqmwYsUKTxeFiGzEYIhIgjIyMqBSqUwPPz8/tG7dGuPHj0dhYaGni0dE5FX8PF0AIrLu//7v/xAXF4dr165h27ZtyMjIwJYtW7B3714EBgZ6unhERF6BwRCRhA0ePBg9e/YEAEycOBEtWrTAW2+9hW+//RajR4/2cOnIk8rKytCkSRNPF4PIK7CbjEhG+vXrBwA4evSoadv169cxc+ZM9OjRA6GhoWjSpAn69euHnJwcs+ceP34cKpUK7777LhYtWoT4+HgEBATg1ltvxc6dO2u91urVq9G5c2cEBgaic+fOWLVqlcUylZWVYdq0aVCr1QgICEDHjh3x7rvvQghhdpxKpcKUKVOwfPlyJCYmIigoCL1798aePXsAAAsXLkS7du0QGBiIlJQUHD9+vN7349VXX4VKpcKRI0cwfvx4NG3aFKGhoZgwYQLKy8tr1T0jI6PWOVQqFV599dVa5zx8+DAefvhhhIaGomXLlpgxYwaEECgoKMCwYcMQEhKCiIgIvPfeexbLptPp8NJLLyEiIgJNmjTBPffcg4KCglrHbd++HXfddRdCQ0PRuHFj9O/fHz///LPFeu7fvx8PPvggmjVrhr59+wIAzpw5gwkTJiA6OhoBAQGIjIzEsGHDbHr/iMiALUNEMmK8wTVr1sy0rbi4GJ988gnGjBmDSZMmoaSkBJ9++ik0Gg127NiB7t27m51jyZIlKCkpwRNPPAGVSoW3334bI0aMwLFjx9CoUSMAQHZ2NkaOHInExESkp6fj4sWLphtudUII3HPPPcjJycFjjz2G7t27Q6vV4vnnn0dhYSE++OADs+M3b96Mb7/9FpMnTwYApKen4y9/+Qv+8Y9/YP78+Xj66adx+fJlvP3223j00UexYcMGm96X0aNHIy4uDunp6di1axc++eQThIeH46233rLn7TVz//33IyEhAbNnz8Z//vMfvPHGGwgLC8PChQtx55134q233sLXX3+N5557DrfeeivuuOMOs+e/+eabUKlUeOGFF3Du3DnMmTMHqamp2L17N4KCggAAGzZswODBg9GjRw/MmjULPj4+WLx4Me68805s3rwZvXr1Mjvnfffdh/bt2+Of//ynKdgcOXIk9u3bh7/+9a+IjY3FuXPnsHbtWvzxxx9MdieylSAiyVm8eLEAINatWyfOnz8vCgoKxIoVK0TLli1FQECAKCgoMB1bVVUlKioqzJ5/+fJl0apVK/Hoo4+atuXn5wsAonnz5uLSpUum7WvWrBEAxHfffWfa1r17dxEZGSmuXLli2padnS0AiJiYGNO21atXCwDijTfeMHv9UaNGCZVKJY4cOWLaBkAEBASI/Px807aFCxcKACIiIkIUFxebtk+fPl0AMDvWklmzZgkAZvUUQoh7771XNG/evFbdFy9eXOscAMSsWbNqnfPxxx83bauqqhLR0dFCpVKJ2bNnm7ZfvnxZBAUFiXHjxpm25eTkCACidevWZnVatmyZACA+/PBDIYQQer1etG/fXmg0GqHX603HlZeXi7i4ODFo0KBaZRozZoxZ2S9fviwAiHfeeafO94mI6sZuMiIJS01NRcuWLaFWqzFq1Cg0adIE3377rVkLja+vL/z9/QEAer0ely5dQlVVFXr27Ildu3bVOuf9999v1rJk7Ho7duwYAOD06dPYvXs3xo0bh9DQUNNxgwYNQmJiotm5fvjhB/j6+uJvf/ub2fZp06ZBCIEff/zRbPvAgQPNWiuSk5MBGFo3goODa203lqk+Tz75pNnv/fr1w8WLF1FcXGzT8y2ZOHGi6d++vr7o2bMnhBB47LHHTNubNm2Kjh07WiznI488YlanUaNGITIyEj/88AMAYPfu3cjLy8ODDz6Iixcv4sKFC7hw4QLKysowcOBAbNq0CXq9vs56BgUFwd/fH7m5ubh8+bLDdSVSOnaTEUnYvHnz0KFDBxQVFeGzzz7Dpk2bEBAQUOu4zz//HO+99x4OHjyIyspK0/a4uLhax7Zp08bsd2NgZLyZnjhxAgDQvn37Ws/t2LGjWYB14sQJREVFmd30ASAhIcHsXNZe2xhsqdVqi9ttvcHXVaeQkBCbzlHfOUNDQxEYGIgWLVrU2n7x4sVaz6/5/qlUKrRr187U1ZmXlwcAGDdunNUyFBUVmQWuNa9nQEAA3nrrLUybNg2tWrXCbbfdhr/85S945JFHEBERUX8liQgAgyEiSevVq5dpNNnw4cPRt29fPPjggzh06BBuuukmAMBXX32F8ePHY/jw4Xj++ecRHh4OX19fpKenmyVaG/n6+lp8LVEj4dkVrL12Q8tU3/NVKpXF/Tqdzq5zOvO9M7b6vPPOO7XyuoyM19jImGtU3dSpUzF06FCsXr0aWq0WM2bMQHp6OjZs2ICbb77Z7nIRKRGDISKZMAY4AwYMwNy5c/Hiiy8CAFasWIG2bdsiKyvL7KY/a9Ysh14nJiYGwI2Wi+oOHTpU69h169ahpKTErHXo4MGDZufyNGPrypUrV8y212y5cqaa758QAkeOHEHXrl0BAPHx8QCAkJAQpKamNui14uPjMW3aNEybNg15eXno3r073nvvPXz11VcNOi+RUjBniEhGUlJS0KtXL8yZMwfXrl0DcKO1onrrxPbt27F161aHXiMyMhLdu3fH559/jqKiItP2tWvXYv/+/WbH3n333dDpdJg7d67Z9g8++AAqlQqDBw92qAzOFhISghYtWmDTpk1m2+fPn++y1/ziiy9QUlJi+n3FihU4ffq06T3p0aMH4uPj8e6776K0tLTW88+fP1/va5SXl5s+B0bx8fEIDg5GRUVFA2tApBxsGSKSmeeffx733XcfMjIy8OSTT+Ivf/kLsrKycO+992LIkCHIz8/HggULkJiYaPEma4v09HQMGTIEffv2xaOPPopLly7ho48+QlJSktk5hw4digEDBuDll1/G8ePH0a1bN2RnZ2PNmjWYOnWqqfVDCiZOnIjZs2dj4sSJ6NmzJzZt2oTDhw+77PXCwsLQt29fTJgwAWfPnsWcOXPQrl07TJo0CQDg4+ODTz75BIMHD0ZSUhImTJiA1q1bo7CwEDk5OQgJCcF3331X52scPnwYAwcOxOjRo5GYmAg/Pz+sWrUKZ8+exQMPPOCyuhF5GwZDRDIzYsQIU4vCpEmTMH78eJw5cwYLFy6EVqtFYmIivvrqKyxfvhy5ubkOvcZdd92F5cuX45VXXsH06dMRHx+PxYsXY82aNWbn9PHxwbfffouZM2fim2++weLFixEbG4t33nkH06ZNc06FnWTmzJk4f/48VqxYgWXLlmHw4MH48ccfER4e7pLXe+mll/D7778jPT0dJSUlGDhwIObPn4/GjRubjklJScHWrVvx+uuvY+7cuSgtLUVERASSk5PxxBNP1PsaarUaY8aMwfr16/Hll1/Cz88PnTp1wrJlyzBy5EiX1IvIG6mEO7ImiYiIiCSKOUNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUTRHzDOn1epw6dQrBwcFW1ygiIiIiaRFCoKSkBFFRUfDxcV37jSKCoVOnTtVaFZuIiIjkoaCgANHR0S47vyKCIeMCkgUFBQgJCfFwaYiIiMgWxcXFUKvVZgtBu4IigiFj11hISAiDISIiIplxdYoLE6iJiIhI0RgMERERkaIxGCIiIiJFYzBEREREisZgiIiIiBSNwRAREREpGoMhIiIiUjQGQ0RERKRoDIaIiIhI0RgMERERkaIxGCIiIiJFYzBEREREisZgiIiIiBSNwRAREREpGoMhIiIiUjQGQ0RERKRoDIaIiIhI0TweDG3atAlDhw5FVFQUVCoVVq9ebfXYJ598EiqVCnPmzHFb+YiIiMi7eTwYKisrQ7du3TBv3rw6j1u1ahW2bduGqKgoN5WMiIiIlMDP0wUYPHgwBg8eXOcxhYWF+Otf/wqtVoshQ4a4qWRERESkBB5vGaqPXq/H2LFj8fzzzyMpKcnTxSEiIiIv4/GWofq89dZb8PPzw9/+9jebn1NRUYGKigrT78XFxa4oGhEREXkBSbcM/frrr/jwww+RkZEBlUpl8/PS09MRGhpqeqjVaheWkoiIiORM0sHQ5s2bce7cObRp0wZ+fn7w8/PDiRMnMG3aNMTGxlp93vTp01FUVGR6FBQUuK/QREREJCuS7iYbO3YsUlNTzbZpNBqMHTsWEyZMsPq8gIAABAQEuLp4RERE5AU8HgyVlpbiyJEjpt/z8/Oxe/duhIWFoU2bNmjevLnZ8Y0aNUJERAQ6duzo7qISERGRF/J4MPTf//4XAwYMMP3+7LPPAgDGjRuHjIwMD5WKiIiIlMLjwVBKSgqEEDYff/z4cdcVhoiIiBRH0gnURERERK7GYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGieTwY2rRpE4YOHYqoqCioVCqsXr3atK+yshIvvPACunTpgiZNmiAqKgqPPPIITp065bkCExERkVfxeDBUVlaGbt26Yd68ebX2lZeXY9euXZgxYwZ27dqFrKwsHDp0CPfcc48HSkpERETeSCWEEJ4uhJFKpcKqVaswfPhwq8fs3LkTvXr1wokTJ9CmTRubzltcXIzQ0FAUFRUhJCTESaUlIiIiV3LX/dvPZWd2kaKiIqhUKjRt2tTqMRUVFaioqDD9Xlxc7IaSERERkRx5vJvMHteuXcMLL7yAMWPG1BkhpqenIzQ01PRQq9VuLCURERHJiWyCocrKSowePRpCCHz88cd1Hjt9+nQUFRWZHgUFBW4qJREREcmNLLrJjIHQiRMnsGHDhnr7DQMCAhAQEOCm0hEREZGcST4YMgZCeXl5yMnJQfPmzT1dJCIiIvIiHg+GSktLceTIEdPv+fn52L17N8LCwhAZGYlRo0Zh165d+P7776HT6XDmzBkAQFhYGPz9/T1VbCIiIvISHh9an5ubiwEDBtTaPm7cOLz66quIi4uz+LycnBykpKTY9BocWk9ERCQ/ihlan5KSgrriMQlNg0REREReSDajyYiIiIhcgcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNE8PpqMiIiIvIBOB2zeDJw+DURGAv36Ab6+ni6VTRgMERERUcNkZQHPPAOcPHljW3Q08OGHwIgRniuXjdhNRkRERI7LygJGjTIPhACgsNCwPSvLM+WyA4MhIiIicoxOZ2gRsjRBsnHb1KmG4ySMwRARERE5ZvPm2i1C1QkBFBQYjpMwBkNERETkmNOnnXuchzAYIiIiIsdERjr3OA9hMERERESO6dfPMGpMpbK8X6UC1GrDcRLGYIiIiIgc4+trGD4P1A6IjL/PmSP5+YYYDBEREZHjRowAVqwAWrc23x4dbdgug3mGOOkiERERNcyIEcCwYZyBmoiIiBTM1xdISfF0KRzCbjIiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkaJ10kIiLyVjqdbGeFdicGQ0RERN4oKwt45hng5Mkb26KjDQurumq9MJkGX+wmIyIi8jZZWcCoUeaBEAAUFhq2Z2W55jVjY4EBA4AHHzT8jI11zWs5GYMhIiIib6LTGVqEhKi9z7ht6lTDcc7iieDLiRgMEREReZPNm2sHJdUJARQUGI5zBk8EX07GYIiIiMibnD7t3OPq4+7gywWYQE1ERPIg0+Rct4uMdO5x9XF38OUCbBkiIiLpk3Fyrtv162cYNaZSWd6vUgFqteE4Z3B38OUCDIaIiEjaZJ6c63a+vobh80DtgMj4+5w5zmtVc3fw5QIMhoiISLq8IDnXI0aMAFasAFq3Nt8eHW3Y7sx5hhoSfOl0QG4ukJlp+Omh66gSwtInzLsUFxcjNDQURUVFCAkJ8XRxiIjIVrm5hi6x+uTkACkpri6N+zU0T8qdeVaWJnlUqw2BkKXgy4ZJId11/2YCNRERSZcXJOc6zBkzSPv6ui9IHDECGDbMtuDL2PVZsz3G2PXp7NarejAYIiIi6fKC5FyHODNYaEjrkL3PtSX4qq/rU6UydH0OG2ZbGZ3A4zlDmzZtwtChQxEVFQWVSoXVq1eb7RdCYObMmYiMjERQUBBSU1ORl5fnmcISEZF7eUFyrt2cmSfVkFF4rhrBJ8F5iTweDJWVlaFbt26YN2+exf1vv/02/vWvf2HBggXYvn07mjRpAo1Gg2vXrrm5pERE5HbuHhklBc4KFhoyCs+VI/gk2PXp8WBo8ODBeOONN3DvvffW2ieEwJw5c/DKK69g2LBh6Nq1K7744gucOnWqVgsSERF5KXeOjJICZwQLDWldcvUIPgl2fXo8GKpLfn4+zpw5g9TUVNO20NBQJCcnY+vWrVafV1FRgeLiYrMHERHJ2IgRwPHjhlFjS5YYfubne18gBDgnWGhI65Kru7Ek2PUp6QTqM2fOAABatWpltr1Vq1amfZakp6fjtddec2nZiIjIzdwxMkoKS34Yg4XCQsutMyqVYX9dwUJDWpdc3Y1l7PocNcpQl+p19FDXp6Rbhhw1ffp0FBUVmR4FBQWeLhIREUlR9Un//u//gJgYzy/54Yw8qYa0LrmjG0tiXZ+SbhmKiIgAAJw9exaR1d70s2fPonv37lafFxAQgICAAFcXj4iI5MzSPD41eWjeG1OwYGmeIWuTGFbXkNYlZ7RM2cLKvEQVZVX4+b3foF1WhB9/v6lhr2EjSbcMxcXFISIiAuvXrzdtKy4uxvbt29G7d28PloyIiGTN2mipmjy55EdD8qQa0rrkzhF8vr4Qd/THoabJ+NdHKgyJ3IWw0CoMfO5mvL0jBXuudWj4a9jA4y1DpaWlOHLkiOn3/Px87N69G2FhYWjTpg2mTp2KN954A+3bt0dcXBxmzJiBqKgoDB8+3HOFJiIi+aprtJQl1ROG3b3kR0PypBrSutTQlql6XDlRhPVz90P7fSWy89rihK4tgLam/RE+Z5EWm4d+d5Zj0icNeimbeHxtstzcXAywsO7MuHHjkJGRASEEZs2ahUWLFuHKlSvo27cv5s+fjw4dbI8WuTYZERGZ2LreWU1LlgBjxji9OC7nzhmorZ3mug47vzgA7dcXkP1rGLaXJEJXrT3GHxXo12wfNL2LoZkQhS4j2kPlo3Lb/dvjwZA7MBgiIiKTzExDgrS9vHUxWBcp2H4K2vlHkb3BD+sKO+GyaGa2v5P/UWgST0Jzb2Pc8WQimoQ3qXUOLtRKRETkCvaOgnJWwnBNUhjG70TlZ0uw58WvcSjnFNYXdsRXVQ9AjyjT/qaqK0iNOgDNwCqkPdkWbXrHA4j3XIGrYcsQEREpi05nGDJvbbRUdcaEYWePJnPGivQeJvQCe7LykJ1xCmUbd+Kx0g8RjULT/gJEY37gswi87WakjWmOWx9JgF+gfW0w7CZzIgZDRERkxjiaDKg7IFKrHUsYrqvVx9qK9K4KvJzo/IELWDf/MLQ/6pGd3w6n9RG4F1lYgVEAhNkQdaFSQQU0qD4MhpyIwRAREdVirXVm0iSgfXvbu65qBj7nzwPPPmu51WfYMEOrlLUh/cYuufx8SXSZXS+9jm2LD0C79DK0u8Oxq7yTWchzE4pxzKc9WujPweLiGg2sD4MhJ2IwRESy5WV5JfWqWd8+fYBffnFd/Rv6/toycSNwo9Xn1VeBWbPqP68Hk7WPbjgB7aLj0G4MxIYziShFsNn+boGHkNblNDT3haBf4gX4/0VT/0kdrA8TqImIlM4L8krsYqm+vr7mkx06u/4NmcfHWneXJUIYAiLjZIb1cXTdLwcUnyxGzvwD0H5bAe2hGByrigEQY9rfUnUeg9ochiZNYNDT7RHZvSOAjoadmZm2vYgb6+MIBkNERFJk7UbrqeUhXM1afWvO+iyV+ts7cSNgOPbSJduObci6X/XQV+mxa8lBaL88B+2OZthanIgqJJv2+6ESt4fugyb5CjSPtEL3+zvCx+/2hpXThfVxBnaTERFJjXG0k0zyShqsvvrWJIX6OzpxIwCEhQGXL9e97peT63Zq1xlkzz8C7VofrC3oiIuiudn+do2OQ9PpBDTDgpDyVAKCo4KtnKmG+kbmySRniC1DRERSs3lz3YGBJ5eHcIX66luTFOrfkG6fZ54x5A6pVOYBhBPX/bp25Ro2L9gH7YoSZO+L+nONrwjT/mAUY2DkfmhSriPt8Vi0TYkFEGv/CxnXMRs1qv76SDj/jcEQEZHU2HqjlXgehs0crYcn6+9It4+xleTll4HOnZ267pfQCxz4/ii0n51E9s9NsPFCEq6ix42Xhh49mxyA5ubz0IwJQ/L4BDRqfJv9dbDElnXMJJ7/xmCIiEhqvCQPw2aO1sOT9e/Xz3Azt2XiRqB2K8mIEYZh9g1oKbl09DLWzT2A7B900B6Nx0ldOwDtTPujfE5DE38EmsG+GPh0R7TomGRfHe1RV31kkP/GnCEiIqlxcR6G5NgzIzQgnfrbOnEj4PjkjdVUXavC9sX7kb30ErS/tsDOsgTocaP+gbiKO5rvg+b2UqSNb42kYe2g8rE4+4/7NDD/jTlDRERKZU8ehjeoq741San+1rqH1Grg/feBFi0anB9zfMtJZC84Bm2OP9afSkARuprtTwrIg6ZzIdLuvQl3PJWEoLCeDa2Vc8kk/43BEBGRFNmSh+FNrNXX0jxDUqq/E7q7qis9U4qNCw5Au6oc2gNtcLgyDkC0aX+Y6hIGRR9E2kAd0p5uh+hb2wNo75y6uIJM8t/YTUZEJGUSHoHjEu6egdrD9FV6/L4yD9qM09BuC8WWK0mohL9pvy+q0DtkH9J6Xobm4Zbo8VAn+PrLqP62TkFgZYZqLsfhRAyGiIgUTkJB5dm957F23mFotQJrT3TAWX242f5YvwJo2udDM9Qfd05OQGibUI+U0ykamP/GnCEiIiJn8PCw7uul1/Hzon3QLiuC9vcI7L7aCUBL0/4mKMWA8P3Q3HEVmsdj0G5gDFQ+apeXyy1kkv/GliEiIqpNQi0pDWJtWLfxRuyCYd1CL5C39ji0//4D2s1ByD2XiDLcZHbMzUEHoOl2Fpr7m6LPxET43+Rv5WxewlJAasMIO3aTORGDISIiO0h8gjyrLOUbxce7ZVmToj+KsH7uAWi/u47sI3E4XmXestPK5xzSYg9Do1Fh0OQOCE9qaeVMXsyBAJvBkBMxGCIispEHWlKcwlIA16IFcOFC/c+1krxbF911Hf775QFov76A7P+GYVtJInTVMk/8UYG+zfZBc1sxNOMj0WVEe/j4+Vg4kZe0wLkIc4aIiMi96lqJXQhDQDR1qmEouZRu2NYCOFsCIcDmYd0nd56Gdt4RZG/wxbqTnXBJdDbt84EOY/2WYGDUQXQcEIUu6Q+iSeQt9Zdbji1wXogtQ0REZNDAYdAeYe+K95ZYqc/VS1excf4+ZK8qhXZfNPZXtDPbH4oipLbejyfarceA/fPhd75aUFVfUCPXFjg3Y8sQERG5l0wmyDNj74r31Rlzhvr1A2BIfN67Kg/ZGaeg/SUYmy4loQI3ZnT2gQ69btoPTY+LSBvTHL3GJcDvh9PAqJn2rbsl1xY4L8ZgiIiIDOS4QKyjgdmfLTDFz7+OH/6+Hdofdcg+1g6n9B0AdDAdpvYthKbdUaTd3QgDJ3dCWHyXG+dwNKiRyRIVSsJgiIiIDOpbib1GS4ok2BqYtWwJnD9v+vVKoxb4J17Gu38bC4Ebic1BKEdKy31Iu70cmsei0enutlD5tLZ8TkeDGjm2wHk5BkNERGQgkwnyzNgQwFW1iMCn/b9AwfrDOHU5EMfQFpuv9zOt+N418BDSOp+G5r4Q9H08EYFNb7XttR0NauTYAuflGAwREUmZu4dey22B2DoCOAHDr6PPz8WqFakAUgEALVQXcH+bbdAMEkib3B6R3TsC6Gj/azsa1PTrBzRvDly8aPl4KbbAeTmOJiMikipPDr2W0fw3+io98qfNRcuFbyCk4kZX2B9QYyrm4DsMRZ/QfdD0ugLNI61w8wMdLc/5Yy9H193KygJGjqz73CtXSi/w9ABOuuhEDIaISHY49LpOp3efRfa8PGjXqrD2j464IFrABzr0w2ZE4jR8fFRolhCOtGFNMGByIoKjgl1TEON1Aix3K9a8TrZMBdC8OXD2rGSDT3diMOREDIaISFbqu2E6cRkJubh2sQz7XvoaB9cWYMPJ9siofMiU8wMAwSjGnREHoEmpQNqkGMTfGeO+wtmz7pYc53LyIM4zRESkVBx6DaEXOPjDMWg/PYnSDdsxvvhf6IFC9ADwEIBX8TI+8n8O/rd2g+aBZrjt0UQ0apzsmsLU12U4YoRh+Lwt3YrOGElmSxempXXafvlFFt2ensBgiIhIahQ69Ppy/hWs++gAsn+ohPZIPAp08bgX/8MKvAhDOvQN0apCvF35d+DZFcCIFNcVyta8LV9f2wLTho4ks6U8lo7x9TUESHXVQcHYTUZEtpNRUq2suaMrRQLXsupaFXZ8fgDZmReh/bU5dpQmmnV9BaEMJ1Rt0UKcg8rSCVzdXeiKvC1Hk65tLQ9g+RhLr+NoHdyIOUNOxGCIyAm4qKT7NOSGaQsPXssTP59E9sJj0G5ohPWnEnBFNDXbnxhwBJqkk0i79yakdLmAwOGD6z+pK/JrXJm3ZW/Sta3laf3n5JC2Lk8ig9wz5gwRkXRY+x9pXesvkWOMLTajRhkScJ09+aGbr2XZuTJs/Hg/tKvKoT2gxqHrbQFEm/Y3U13GoOgDSLtTh7Sn4qFObgfgzwVRMzNtexFndRdWby07e9Z1eVuOzOVkSx6ZvWu0KSD3zFYMhoioblxU0n0stdj4+NTO9XB08kM3XEuhF/h9xWFoM05Duy0EWy4n4TpuzOjsiyrcFrwfaT0vQfNQC/QcmwBf/z6WT+bOmZotvfe2cDQQsyfpuiGvYwsvyz1zBIMhIqobRza5h7UWG2MgZAxSGpLb46JreW7feayddxharUD28Q44qzef0TnG9yQ07Y9BM9Qfd05OQNOYrrad2F1rpVl7723RkEDM1qTrhr6OJ88tEwyGiKhuCh3Z5FZ1tdgAhpv+ypXAu+82rPXNSdfyeul1/PLJfmi/uQLt/1rht6sJAFqa9jdGGQaE74Om31VoJrVB+0GxUPlEWz+hNe5YK62+994ady+ZYUtgaMwZsnaMpedw2Q8AgBPmI3ctnU6HGTNmIC4uDkFBQYiPj8frr78OBeR9E0kDF5V0PXtabBrCwWsp9AJ5a49j7n0bMbTVDoQFX8eAv3fH7G0pfwZCQPegg3ghORcb3vsNl4r88P3ZXvjriv7ooImDysfiWDDbGPNrWtdYOT462jn5TfW995Z4YtFaY2BY/fVrlufDD60fU5NUF971FCFxb775pmjevLn4/vvvRX5+vli+fLm46aabxIcffmjzOYqKigQAUVRU5MKSEnmpqiohoqOFUKmEMNyWzR8qlRBqteE4csySJZbf25qPJUsa9jp2XMsrJ66IrH9sFU8kbBRxfidqHRquOicejtssvnxyiziz55xz3of6yp6TY3gPcnKc93mz9b2v/lCrhVi50jmvb6+VKw3XsK7yWDrG11c6dbCDu+7fku8m++WXXzBs2DAMGTIEABAbG4vMzEzs2LHDwyUjqkYCc7a4jDu6KpQuPNy24xra+lbXCu8qFSCAFaET8WGzfdhWkggdbjPtb4Tr6Nt0HzS3FUEzPhJdR7aHj1/fhpXH3rLXlV/j6HfQ1vf0gw+AVq08//22JfHa0jGcgbpuLg21nODNN98UMTEx4tChQ0IIIXbv3i3Cw8PFV199ZfM52DJELmXpf2HR0bL4X5ddbPkfKdlv5UohWreuuyXC2a1vFq5lAVqLe7HS7GU7NDom/to1V3w/a4coOV3inNd2hYZ8B9nyKWnuun9LftJFvV6Pl156CW+//TZ8fX2h0+nw5ptvYvr06VafU1FRgYqKCtPvxcXFUKvVnHSRnE9pK4t7cwuYJ9gyismJn6Wrl65i08f7kL2qFGv3RCDs+hlE4jROIxKb0Q/BKMXAqAPQDLiOtCfbIravA0nP7uaM76AjkyCSW7ht0mSXhlpOkJmZKaKjo0VmZqb4/fffxRdffCHCwsJERkaG1efMmjVLwLCQjdmDLUPkVMb/Ubrrf/PkXer7/DihlVGv04s9WYfFe/fkiLTmO0Ugys1O7YMqkdxkj5h5R47YMv9/ovJqpVOr6HLO/A66uuXTVTlPXo4tQ39Sq9V48cUXMXnyZNO2N954A1999RUOHjxo8TlsGSK3cMf6UdbU1ULD1ht5sPXzs24dMHCgzae9mHcJ6+YehPZHHbKPtkOh3jwnJtr3FDTxR5F2ty9SpyQgLL6ZnQWXEGd/B1313eFSNg7jchx/Ki8vh4+P+QwAvr6+0Ov1Vp8TEBCAgIAAVxeNlM5T8+/U9YcV4B9dubD1c3HuXJ27K8srsT3jALSZl6D9rSX+W5YAgRszOgfiKlJa7EXa7WXQPBqNhL/EQ+UT1ZCSS4ezv4P2TIJoKy5lIwuSD4aGDh2KN998E23atEFSUhJ+++03vP/++3j00Uc9XTRSOk/Mv1PXH9aRIy0/h390pakBn5/8TQXQLsiHNtcfG04noBjmMzp3CTyMtKRT0IwKRr8nkxDY9NZa5/AKUp8Di0vZyIbku8lKSkowY8YMrFq1CufOnUNUVBTGjBmDmTNnwt/f36ZzcNV6cglXryxu7fXsnSDOFWVxBaV179nx+Sk9fxU58/ZDu+YqtAdjcKQy1uzQ5qqLGKQ+BM0gPdKeboeoWyLcUgWPc/d30F6e7Er3Euwm+1NwcDDmzJmDOXPmeLooRObcPf+OIzPlGkl9/TB351RIIfAyfn4stOgZ5/zJCpuIj5rvwS9FiahEL9N+P1Sid8h+aHpdhmZsOG55sBN8/KwsdurNpD4HFpeykQ3JL8dBJGmuXiqgOmf8wZTiH11j11/NQM/YvZeV5fzXi401/I/9wQcNP2Njnf86tmrevNami6IpRmIFRv1vJjYWdUcl/NHW7wSeStqE1dO342LBVWwq6oaX16ag5yOJ8PFT8J9yd34H7SX1bjwykXw3mTOwm4xczh0tDbY2uddFas3x9XX9ObubQ0LzQlV+9Q38xj5gePlq2/V/bhmLL1Ea0Q6a/tegeTwW8XfGuKVcHuGM748UWvsslUnK3Xgy4K77t13BUEFBAdRqtcsK4yoMhsgr1PeHtS5S/aPrzpwKdwdeNQi9wKGf8qH9pADrNgfg4wujEIVCi83zAoYVyFUnjkvrermCtw8754SODeKu+7ddbaudOnXCzJkzUV5e7qryEJE1tqxaXdc+Ka4f5s6cCnetDF/N5fwrWDFtKx5P2IRY/0IkDGmLqav6o+TCNURbCYQAQAUBVeFJp5ZFktzdReoJUu7GIxO7gqG1a9dCq9Wiffv2yMjIcFGRiMiquv6wrlxpeMjpj64rcyp0OkPLU2am4WdhoW3Pa0DgVXWtClsX7cFrA3LRJ3gPWrQNxn3v98a/D96BP3TRCMA1pIb9iue6r3d5WSSvvmHngGHYuU7n1mK5xIgRwPHjhhbOJUsMP/PzrX8na352veE9kDiHcoa++OILvPzyywgPD8ecOXPQr18/V5TNadhNRl7HW2agdlVOhaWulxYtgAsX6n+unV1yf2wtRPaCY9Cu98O6Uwm4Ipqa7U/wPwpN0kmkDW+M/k8noXGLxg3rHpTT9a0Lh51b5u3dhnaS/NpkZWVlYsaMGaJJkyZi5MiR4tixYw1fHMRFuGo9kYStXGlYQ6rmquHGbfauDWU8X31rfjm4jlXZ+TLxw2s7xDPdc0Un/yO1TtNMdUncF/2L+Pcjm8SJX05aPomjK6U3ZHV2qVmyxLbrsmSJp0vqPtY+u45+F7yAu+7fDQqGNm/eLJ555hnh4+MjAgMDxfPPPy9KSkqcWT6nYDBEJHHOWiTT1sVP7Qi89Dq9+N/yQ+Ltu3PEwGa/Cn9cq7XYaZ/g/4lXU3LE1n/vEVUVNi7AaW8QWN+NctkyeS0EmpNjWzCUk+PpkroHF362SJILtS5YsAA7d+7Ezp07ceDAAfj4+KBz58647bbb0K1bNyxduhRHjhxBVlYWevbs6arGLLuxm4xIBpzR/WNr10vLlsD58zd+V6sNCeZ/dkOcP3ABa+cegvYngezj7XFG38rs6W18T0LT/hg0f2mEgVMS0TQm1L5yGlnqEqlRFgC2zT7u62ueWyL1rhUOOzfHbkOLJDm0Xq1WIzk5Gbfddhtuu+029OjRA0FBQWbH/POf/8SSJUuwd+9epxfWUQyGiBQiM9MwkWJ9vvrKkGj+Z+B1/eZkbM04BO03V6Dd3Qq7riaYHd4YZUhpuR+afuXQTFSjgyYOKh+VlZPbyZYg0JE5puQwdJvDzm+w9bO7ZAkwZozryyMRklyOo6CgoN5jHnvsMcyYMcPhAhEROczWUWetW+OILg7aVSpoNwUi52wVStHd7JBugYeg6XoamtGhuH1SIgJCXLTYqS0rpTsyqkwOC4EaR0daShiu2Trm7ThbtUc5fQZqIQQ2bdqE/v37O/O0DcKWIZnwllEy5Dn1dL0IAJf9wpEstuKIrq3Zvpaq80iLOQyNRmDQ0x0Q0TXcPWW2RUNnH5d61wq/++w2tEKSLUO2UKlUkgqESCY4nJScodrCnUKlgqraTUX/56IXE6s+xhG0RSNcx+1N90GTXATNuAh0u68DfPxu91TJ69avn+H74Mjs44D05yuypXXM20l90Vkvp+DV/UgylDALLblF4X9PY/H3LfF+8zdwSph3J5xENP7mOw+tuzTHdzN24NLp68i5fDNe/CkFN4/pJO3FTuuafdwW7FqRB85W7TFcqJU8y8PrRZG8Xb10FZsX7kf2yhJo97bG3or2pn0+0OEu/IiBTXehXY9QdHl5GOIGxHqusM5gqQW15iiy6vj9kSd2G5pIcjSZXDEYkjAOJ3U9Of5htVJmoRfY/91RZC8+Ce2Wm7DxYhKu4caIVhX0uLXJfmhuuYC0B8KQPD4BjRo38mBFXKDme3PhAjB6tGGf0kdkkdeRbc4QkV3ctVCnHAMCZ5BjLpaFMpff1BKLg/+G2ecexUldOwDtTPta+5yGJv4I0gb7InVKJzRv39kDhXYjS/k1HJFF1CAMhsiz3DGcVI4BgTOsWAHcd1/t7cZcrJotBlIIGLOyIEaNAoRA9cyYwNILeKp0JtYjERcwGP2b70Pa7aXQPBqNxKHxUPkoPCdmxAjD8HlPXz8imWI3GXmWq4eTGpOza57b27sQli83TMxmay6JhwPG41tOInteHkYsewBh+nMWR3YIANdCWwF5RxDU8iaXl4mIPI85Q07EYEjiXDULrVKTs7OygJEjbTs2Jwe4dMl6wCgE8NprQPv2Tm1tKD1Titz5+6FdfRXag22QVxmH/shFLpg/JitSaE0kr8acIVIOV81Cu3lz3Ws5CQEUFBiOq3lzlesfeZ3O8D7aqrAQePFFy61yxm2zZt3Y5mBrkb5Kj/8tPwzt52eg3R6Kn68koRK9TPt9UYWUoG3AVRtOJvU5c5RCqd3P5JUYDJE0uCLnwdHkbDn/ka8vAKzp/Hn7jreWb2TB2b3nkT33ELRaFdae6IBzohOATqb9cX5/QNPhODT3BODOyQkIOXIbbGkYksScOXINlp3FWvezHZ8PIilhMETS4exZaB1Jzq7rj/zIkS7pMnIqe1pN1GrD6u32qGO9q4riCvz87/3QLitC9p4I7L7aCcCN8zdBKe5stQ+aO65B80Qs4ge0gcqnzY1zR9Yzy7KxW7NfP/vK7GxyDpadwdj6aK01UerroRFZwJwh8l72JmfXl2NUkxRvgPasYbVyJRAW5vCaV2L9BhyuiIH2kwJkbwlCzrkklKOJ2TG3BB2ApvtZaO5vit6PJcL/Jv+6T+rK/DFntOQoNSG/Os4NRm7ktvu3UICioiIBQBQVFXm6KORuK1cKoVIZHoZbmOFh3LZy5Y1jc3LMj6nvYekcnlZVJUR0dO36Vn/4+gqxfLntx1t5/FX1Ua3NET5nxCNtN4uvn94izu4951gdVq40lKn6idVqx99nS+eLjrb/fMb3qq7Pg1ptOM5bVVUJ8cortn1GlizxdGnJC7jr/s1giLyfrTfXJUvsDggkeQO0FgAaH8uW2Xe8lUd/5Ah/XBMDm/0q3r47R+z+5qDQ6/TOqUNVlSE4XbLE8NPR99dYN2cEsrYGyzk5jpVV6ix9j5T4PpBbuev+zW4yUgZbukns6WKqSWpdApbyWtRq66PzLBwvALOJD430UKGoUUtsnb4G/Sd3QZPwJhaOkgBnT62QmQk8+GD9xy1ZYpjjyZtY6x60xFunrCCP4NB6ImeyJTm7Xz0JvHWR2nBvO0bnlV8ox6b9bZDd4nNcPHMa16t80A55eA2vQgDwwY33QkAFHxXQbOnHuHvEbW6skAMaMrWCJe6YLV2K6kqYrsmYOzVnDgMhkhUGQ+Q9Gpok6+trSIgeNerGhIO2kuIN0EoAKPQCe1flQbv4FLRbg7H5UhIq0NO03wc6JN+0H53bAMNPz0fA5bOmfSq1jNa7cva6d/UFy1IZ7eZs9kzXwPXQSKYYDJF3cNZwZ2sTQNZFrZb8DfDCoYtYO/cQtD/qkZ3fDqf1HQB0MO1X+xZC0+4oNEMaYeCUBDSL6wKgC6B7Wfrz6VgLgp3dklNXsOzNLSK2BouvvAK8+qr31Z8UgcEQyZ+zJ4Cr2cW0Zg3wzTfWj3/gAcndACrLK7H10/3QLr0M7e5w7C5vj764jkicRgdUoRhNcEfLg9D0LYdmohod74qDyqd17RM5e+4nZ6srCB42zPktOa6aLV3KbA0WBw6U3PeAyFZMoCZ5c/X6Y7bMPaRWSyJZ9OiGE9AuOg7txkBsOJOIUgQDAO5FFj7EM1CjWnJ0VGuoPvqXbTdvqc62bMucP4C05y2SA1cvpkxUBy7U6kQMhryYqyeAk/AEcyWnSrBh7n5ov61A9qEYHK2KMdvfQnUBL7VYhKnnXwZQY2SYrcGAVGdbticIXrPGch0mTZL2bOJS4qrJMInqwdFkRLZwdpKsu89vB32VHruWHIT2y3PI3tkUvxQloQrJpv1+qMTtofugSb4CzSOt0H1UO/i0+9jyyYSof9kEKa8/Zc9IsZrdnnl5wKJFTlmA1i2k0AqlxO5BUhQGQyRvrh7u7OHh1Kd2nUH2/CPIXqfC2j864oJIBJBo2t+u0XFoOp2AZlgQUp5KQHBU9xtPzs11fGi51NefsjdINeY+ZWUZknylGOBZIqWWOVcspkwkEQyGSN5cPdzZzcOpr125hs0L9iF7ZQm0e6Ow51oHABGm/cEoxsDI/dCkXEfa47FomxILINbyyRrSquXsOXqczZEgVeoBXk1SbJmTekI9kYMYDJG8uXq4s4vPL/QCB74/iuzFhdBuaYyNF5JwFT1uvAT06NnkADQ3n0fa/c1w26OJaNTYxskOG9KqJaHuQYscCVKlHuBVJ7fAjUjmGAyR/Lk6n8HJ57909DLWzzsI7X+qkH20LQp07QC0M+2P8jkNTfwRpN3li9TJHdGiY5Jj5W5Iq5bUZ1t2JEiVeoBXnZwCNyIvwGCIvIOr8xkacP6qa1XY8fkBaJdchPbXFthZlgA9epv2B+Aa+jffi7TepdA82hpJw9pB5eOEIKMhrVpymG3Z3iBV6gFedXIK3Ii8gCyG1hcWFuKFF17Ajz/+iPLycrRr1w6LFy9Gz549638yOLSe3O/Ezyeh/fgYtDn+WH8qAUUINdufFJCHtKRCaIYHIaXbFQSUXXJdQqq9i7ZWf54chlPbOtqqvvlyAEMgdfy457ueJDylA5E7cZ6hP12+fBk333wzBgwYgKeeegotW7ZEXl4e4uPjER8fb9M5GAyRq5WdK0Pu/P3QriqH9kAbHK6MM9sfprqE1OiD0AzUIe3pdoi+NdK+kUINHV7t6PMdDaSkylqAZ9S8uWHYvafrxokOiQAwGDJ58cUX8fPPP2Pz5s0On8MjwZAU5gYhl9FX6fH7yjxoM05Duy0UW64koRL+pv2+qMJtwfuhufUSNA+3RI+HOsHXv9r1t2X2ZOMN2dPDq73ts5yVBTz+OHDxYu19Umr1kkvLHJELMRj6U2JiIjQaDU6ePImNGzeidevWePrppzFp0iSrz6moqEBFRYXp9+LiYqjVavcFQ56+eZFLnNt3HtlzD0OrFVh7ogPO6sPN9sf6FUDTPh+aof64c3ICQtuEWj6RvbMn2xo0SZ1UgiqdDoiJMbS6WCKlVhdva5kjspPbGjOExAUEBIiAgAAxffp0sWvXLrFw4UIRGBgoMjIyrD5n1qxZAkCtR1FRkesLvHKlECqVEIbb142HSmV4rFzp+jKQU1SUVIgN7+0SLyTniJuD9te6pE1QIv4Svl18NCpXHM7OF3qd3rYT5+TU/nxYeqxbJ0R0tPX9KpUQarUQVVUufR+cYtkyIVq0MC9/dLRnvg+2vv85Oe4vmyVVVYayLFli+CmH603kJEVFRW65f0u+Zcjf3x89e/bEL7/8Ytr2t7/9DTt37sTWrVstPsdjLUOuXjTU1jJI4X/fMiT0Anlrj0P77z+QvTkIOecSUYabzI65OegANN3OQnN/U/SZmAj/m/ytnK0OmZnAgw/Wf9wrrwBvvFH/cVJPov3HP4B33rG8T6Vyf+uWre//kiXAmDGuLw8RWcW1yf4UGRmJxMREs20JCQlYuXKl1ecEBAQgICDA1UWrzdNzg7B7zm5FfxRh/dwDyP7+OrR5cTheFQfgRvJzK59zSIs9DI1GhdSnO6BV5wQACQ17UWcP3Zby8OoVK6wHQoDhO+HuyQPlNMSeiNxC8sHQ7bffjkOHDpltO3z4MGJiYqw8w4M8OTeIFKfulyDddR3+++UBZC+5AO3OMGwrSYQON2Z09kcF+jbbB81txUgbF4muI9vDxy+8jjM6wNY5fFJSbGsZkupNW6cDnn66/uPcPXmgHOZQIiK3knww9Pe//x19+vTBP//5T4wePRo7duzAokWLsGjRIs8WzFJ3lKf+x8mp++t0cudpZM8/Au16X6w72QmXRGez/R39j0GTUIC04Y2R8nQimoTf4toC2ToZYkqKvG/amzcD58/bdqw7W7dcvYQLEcmPSzOSnOS7774TnTt3FgEBAaJTp05i0aJFdj3f6QlYK1fWTmyNjhZi+XLDT0sJ1K5MeJVbQqiLlV8sFz+9sVP8/ZYckRiQV+ttCMUVMbL1L2LhQxvF8S0Fniuopc+RWm2eVGxMyK/5mZJDQv6SJbZ9Lj312bTl/Scij2ICtRM5NQGrvvlhnnsOePddw7/dNTeIwhNChV5g35oj0H5WCO0vwdh0KQkVCDTt94EOvW7aj7RbLkLzYHP0GpcAv0CJNIrakvAu1+HVts6i3LKlof6eGmbPAQdEksV5hpzIaW+mraPF3nsPePZZ9928FDh1/4VDF7Fu3iFof9Qh+1g7nNKbdz1G+56CJv4oNEP8MHByJ4TFN/NQSZ1Ejjft+r4vRsuWAffd55YiEZG8cDSZFNk6WqxlS8P6Ru66eSkgIbSyvBLbPtsP7dLL0P4Wjl/LO0Ggj2l/EMrRv8U+aPqWQ/NYNDrd3RYqnygPltjJfH3lF8hWz82x9n+u559nIEREHsdgyB72jBZzxc3LWuuAlyaEHsv9A9qFx6HNDcCGMwkoQTez/V0CD0PT+RQ094Wg7+OJCGx6q4dKSlZZW1m+ZUtg/vwby00QEXkQu8nsYWt31Lp1hsDDma1CtswhJNfckj+VnCpBzrz90H57DdmHYnCkMtZsfwvVBQxqcwiaQQKDnmqHqFsiPFNQsp8cu/mIyOOYM+RETs8Zqqs7KiwMCAw0X/fI0YkPjTeQNWsMAY2l1wPMk7JldNPRV+nx29JD0H5xFtk7muLnoiRUoZFpvx8q0Sd0HzS9rkDzSCvc/EBH+Pj5eLDEXkJGnxEiUjYGQ07kktFkQO3uKGtvpSMjySy18lg7t1QWlbTB6d1nkT0vD9lrVVj7RwecFy3N9sf7nYCm03Fo7glEylMJCIl2w8K6SsJZyolIRhgMOZHT30xrN5SrV4GLFy0/x56gxdrw/bpIdKTYtSvXsGXRfmSvKIZ2TyR+v9bRbH8winFnxAFoUiqQNikG8XdKcGZxb1HftBCcpZyIJIbBkBO55M2s2dWg0wGpqfU/r76gxdbhyDVJZA4hoRc4+F0e9r37A47+rxQbSnpiHQZBD0MAqIIePRofhObmc0i7vxl6P5aIRo0b1XNWajApLCJMRGQnDq2XupqjxTIzbXtefSPS6hu+b40H16e6nH8F6+cegPY/lVDlHcIM/f9hFAx1eAHAKUQiK+JpNB91JwZN6YgWHRMBJNZ5TrdRSv6MpxcRJiKSMAZDzuKsdcnsXaPJA3MIVV2rws4vDkC75CK0vzbHjtJE6NEb9yILK/AEAPPGxkjVGUw5OxMYkAh07GP5pJ6gpPwZTy4iTEQkcQyGnMVZEx/a08LjxjmE/thaCO3HR6Hd0AjrTyXgiuhitr+z/yF8Kp6EqlJAVbOYUlws1lr+TGGhYbu35c94ahFhIiIZYM6QM9U10gyw7QZb3/D96lw4h1DZuTJs/Hg/tKvKoT2gxqHrbc32N1NdRmrrg9AMrELaU/FQXz0snyVBlJg/Y0sumlrtXXUmItljzpAcWZttNzra9qClrtmkjYwtLE7MbxF6gd9XHIY24zS020Kw5XISruPGjM6+qEJy8H5oel6C5qEW6Dk2Ab7+vW+cIHOjbS8khW4YJebP+PoaEuzfecf6MQ88wECIiBSJwZCzjRhhCFQakpRrLaiypSXIjoTgc/vOY+28w9BqBdYeb48z+o4Abgx9j/E9CU37Y9AM9cedkxPQNKar9deVUzeMEvNndLr6k/yXLgXS0xkQEZHiMBhyBWesS+ZIUFVPQvD10uv45ZP90H5zBdn/a4VdVxMA3Jj0sDHKMCB8HzT9rkIzqQ3aD4qFyifatvLakzPl6RFczg7cPF0fW9gyStHbWsOIiGzEYEiq7L3BWkkIFicLgZEjMTs0Hf8smoxSdDfb3z3oIDRdz0AzOhR9JiYiIKSXY+W1dbHYNWs8P4LLWcnugHxGpCmxNYyIyEZc6EmKsrIMya4DBgAPPmj4GRtr2G6JTme4IVu4sasgIKDCQ0XzUY7GCFedx8NxW/Dlkz/j9P/O4bfyTpi9LQUDnr0ZASEBDSu3sXuvdWvz7dHRhu2AIViq2UJhHMFlrX7OZgzcgBuBmpE9I/SMAain62MLOXVjEhG5GUeTSY2dSyborutweMYXSHj70XpPnffK54if9bDrFzu11KoFSG8El6VWHVtH6MltRJotiwxLqbxEROBoMmWqo4UH1ebqKWzdC9kL86Fd54u1JzshTQTClvmv2yc2Atyx6rulnKncXOmN4GpIsrvcRqTZ2o3JQIiIFIjBkJTYeIN96LYj2IgU0+ZSNLHt/J7sApFqzoqjye5SrU9dnDH1AxGRF2IwJCU23jijUIjkJnuRdssFjOm0C53+8z5wqo4neGDJjlq8LWdFrvVxxtQPRERehsGQRFzMu4T/rbiIO204dsFXIQh5qPOf+UXP1T9TNeD5LhBnjuCSAjnXxxlTPxAReRGOJvOQyvJKbJn/O2b0y0Wvm/ahZYemGJT1FAoQDX2t1b3+pFIBajVCHri77vyimp57zvNdIM4awSUV3lYfIiIFYzDkRvmbCrDgwU24N2obWjQpR7/JXfHGlhTsLEuCgA8SA45hfdtJhuHw9d1gbZlEz2jpUkPw5Gn1Db33dMBmL2+rDxGRQnFovbNYGE5eev4qcubth3bNVWQfbIO8yjizpzRXXURa9H480mU3km/3Q7M+CYZuFUsTE9Yc8p2ZaZiDyFZSWCDVSA4zNtvD2+pDRCQRHFovJxbmqzmnCscU8RGWY7Rpmx8q0TtkPzS9LkMzNhw3B+6H77S/Az+cBH748yDj7MXHj9d9g7U3MXflSsNPKdyovS1nxdvqQ0SkMGwZaqDL7y9G02mGCQ+rd2wZ834m+3wMVUICNPcEYMDTCQiJ/vP17ZxcsZb6Jv2zRopLRRAREVngrpYhBkN2qiiuwJaF+5C9ohhr/xeONRUatMZJi8lXAiqo1BZm9XXW7MVZWcDIkfZVwNZgi4iIyMPcFQwxgboeQi9w8Idj+NfIjRgSvhNhoVVI+0c3bN8BDK/IhNpKIAQY1gUzzUJcnT2zF9dlxAhD91fz5nZU6M/Yd+pUaSRVW6LTGWaszsw0/JRqOYmIyCswZ8iCy/lXsGH+AWi/r4Q2ry3+0LUF0BYAcC+yMBdTEAU7ZhauOZmiM2cvNk6i9+abhu6vS5fqf47UloqoTi6rwBMRkddgMATDYqc7vzgA7dcXoP1vc2wvTYQevU37/VGBO8L2YnLsfzBs16sA7OxZrJns7OzZi319gZkzgZdfNgQ4K1cCc+fW/zwpLRUBWM+jMq4Cz649IiJyAcUGQwXbT0E7/yjWrlOh7NQVBKMEpxGJ7UiEHr5I8D+KtMST0NzbGP2fTkLjZt2B2OGwOxACgPPnzX931ezF1Uc12RIMSWmpCBsXqcWwYZ4fDUdERF5FUQnUK6ZvwOYffaDdH42D1+NxL7LwIZ6BGje6ZEobt8TVV95Ey+mTzE+SmwsMGOBYAdTq2snQxlYQwPIK4g1pBTEmaNcXbNWXoO1Otr6/UpoviYiIXIoJ1C4wKr0HPtzdHwevx2MkVmAlRiEa5onMN129gJYvP2EIVqprSJeSpWRoV85eLMelIuS4CjwREXkFRQVD0T6FmNRpE1Y8uwXLIp+BCqL2KmDWRls1tEvJ0k18xAjD5Io5OcCSJYaf+fnOyYuR21IRcl0FnoiIZE9R3WRXLl9BaNNQx7pk6ut6sudc7iSXpSLk2LVHREQuxW4yF1D5/NkO5EiXTF1dT3W+qGGlebuToZ3FmFQ9Zozhp1QDCTl27RERkVdQVDBk4miXjLWuJ+Okh7yJN4zcuvaIiMgryC4Ymj17NlQqFaZOner4SYxD26218NTVmmMpz+fsWcPcPryJN5wr86iIiIgskNU8Qzt37sTChQvRtWvXhp3I2CUzapQh8LE0tL2u1hxLq5QbZ4KWQ36O1HEVeCIiciPZtAyVlpbioYcewr///W80a9as4Sd0RZeMXPJziIiIyEQ2wdDkyZMxZMgQpKam1ntsRUUFiouLzR4WsUuGiIhI8WTRTbZ06VLs2rULO3futOn49PR0vPbaa7ad3FVdMnIZ0k5ERKRwkm8ZKigowDPPPIOvv/4agYGBNj1n+vTpKCoqMj0KCgpcXMoasrIMc+YMGAA8+KDhZ2xs7VmtiYiIyOMkP+ni6tWrce+998K3WquKTqeDSqWCj48PKioqzPZZ4q5JmwBYX3ndGWuOERERKYi77t+SD4ZKSkpw4sQJs20TJkxAp06d8MILL6Bz5871nsNtwZBxFuWTJy3v5yzKRERENnPX/VvyOUPBwcG1Ap4mTZqgefPmNgVCNnNGjs/mzdYDIcDQWmRctJVDx4mIiCRB8sGQW2RlAc88Yx7IREcb5iKyp0vL3mU+mGRNRETkcbIMhnJzc513Mms5PoWFhu325PjYs8yHswIwIiIiahDJjyZzCZ3OsHL9118DTz5peZV047apUw3H28LWZT4uXDAEWjW71IwBGEedERERuY3ygqHqw94ffhg4f976sdVzfGxhy8rr770H/P3vzgvAiIiIqEGUFQx9+63lFpn62JoLBNS/zEfLlrYnWRMREZHLyTJnyGEvvGC5RaY+tuYCGdW1aGtmpm3nsCcAIyIiIocpKxg6dcq+443zAvXrZ/9rWVvmw54kayIiInI5ZXWT2cOY4zNnjnOHu9uaZO1IAEZERER2YzBkjTHHx9nD3G1JsnZ2AEZERERWKSsYioqy3iIDGJKbv/oKyMkxLJnhqvl+6kuy5jxDREREbiP5tcmcwbS2yZdfIuSRRwwbq1fbU4uocgZqIiIiq7g2mSvcc48h4LE08/OcOe5vkbGWZE1ERERuo6xgCKh72DsREREpjvKCIYAtMkRERGSirARqIiIiohoYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiST4YSk9Px6233org4GCEh4dj+PDhOHTokKeLRURERF5C8sHQxo0bMXnyZGzbtg1r165FZWUl0tLSUFZW5umiERERkRdQCSGEpwthj/PnzyM8PBwbN27EHXfcYdNziouLERoaiqKiIoSEhLi4hEREROQM7rp/+7nszC5SVFQEAAgLC7N6TEVFBSoqKky/FxcXu7xcREREJE+S7yarTq/XY+rUqbj99tvRuXNnq8elp6cjNDTU9FCr1W4sJREREcmJrLrJnnrqKfz444/YsmULoqOjrR5nqWVIrVazm4yIiEhG2E1Ww5QpU/D9999j06ZNdQZCABAQEICAgAA3lYyIiIjkTPLBkBACf/3rX7Fq1Srk5uYiLi7O00UiIiIiLyL5YGjy5MlYsmQJ1qxZg+DgYJw5cwYAEBoaiqCgIA+XjoiIiORO8jlDKpXK4vbFixdj/PjxNp2DQ+uJiIjkhzlDf5J4rEZEREQyJ6uh9URERETOxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRGAwRERGRojEYIiIiIkVjMERERESKxmCIiIiIFI3BEBERESkagyEiIiJSNAZDREREpGgMhoiIiEjRZBMMzZs3D7GxsQgMDERycjJ27Njh6SIRERGRF5BFMPTNN9/g2WefxaxZs7Br1y5069YNGo0G586d83TRiIiISOZkEQy9//77mDRpEiZMmIDExEQsWLAAjRs3xmeffebpohEREZHMST4Yun79On799Vekpqaatvn4+CA1NRVbt271YMmIiIjIG/h5ugD1uXDhAnQ6HVq1amW2vVWrVjh48KDF51RUVKCiosL0e1FREQCguLjYdQUlIiIipzLet4UQLn0dyQdDjkhPT8drr71Wa7tarfZAaYiIiKghLl68iNDQUJedX/LBUIsWLeDr64uzZ8+abT979iwiIiIsPmf69Ol49tlnTb9fuXIFMTEx+OOPP1z6ZkpNcXEx1Go1CgoKEBIS4uniuA3rzXorAevNeitBUVER2rRpg7CwMJe+juSDIX9/f/To0QPr16/H8OHDAQB6vR7r16/HlClTLD4nICAAAQEBtbaHhoYq6kNkFBISwnorCOutLKy3sii13j4+rk1xlnwwBADPPvssxo0bh549e6JXr16YM2cOysrKMGHCBE8XjYiIiGROFsHQ/fffj/Pnz2PmzJk4c+YMunfvjp9++qlWUjURERGRvWQRDAHAlClTrHaL1ScgIACzZs2y2HXmzVhv1lsJWG/WWwlYb9fWWyVcPV6NiIiISMIkP+kiERERkSsxGCIiIiJFYzBEREREisZgiIiIiBRNlsHQvHnzEBsbi8DAQCQnJ2PHjh11Hr98+XJ06tQJgYGB6NKlC3744Qez/UIIzJw5E5GRkQgKCkJqairy8vJcWQWH2FPvf//73+jXrx+aNWuGZs2aITU1tdbx48ePh0qlMnvcddddrq6G3eypd0ZGRq06BQYGmh3jjdc7JSWlVr1VKhWGDBliOkYO13vTpk0YOnQooqKioFKpsHr16nqfk5ubi1tuuQUBAQFo164dMjIyah1j798Md7O33llZWRg0aBBatmyJkJAQ9O7dG1qt1uyYV199tdb17tSpkwtrYT97652bm2vxc37mzBmz47ztelv67qpUKiQlJZmOkcP1Tk9Px6233org4GCEh4dj+PDhOHToUL3Pc8c9XHbB0DfffINnn30Ws2bNwq5du9CtWzdoNBqcO3fO4vG//PILxowZg8ceewy//fYbhg8fjuHDh2Pv3r2mY95++23861//woIFC7B9+3Y0adIEGo0G165dc1e16mVvvXNzczFmzBjk5ORg69atUKvVSEtLQ2Fhodlxd911F06fPm16ZGZmuqM6NrO33oBhhtbqdTpx4oTZfm+83llZWWZ13rt3L3x9fXHfffeZHSf1611WVoZu3bph3rx5Nh2fn5+PIUOGYMCAAdi9ezemTp2KiRMnmgUGjnyG3M3eem/atAmDBg3CDz/8gF9//RUDBgzA0KFD8dtvv5kdl5SUZHa9t2zZ4oriO8zeehsdOnTIrF7h4eGmfd54vT/88EOz+hYUFCAsLKzW91vq13vjxo2YPHkytm3bhrVr16KyshJpaWkoKyuz+hy33cOFzPTq1UtMnjzZ9LtOpxNRUVEiPT3d4vGjR48WQ4YMMduWnJwsnnjiCSGEEHq9XkRERIh33nnHtP/KlSsiICBAZGZmuqAGjrG33jVVVVWJ4OBg8fnnn5u2jRs3TgwbNszZRXUqe+u9ePFiERoaavV8SrneH3zwgQgODhalpaWmbXK43tUBEKtWrarzmH/84x8iKSnJbNv9998vNBqN6feGvpfuZku9LUlMTBSvvfaa6fdZs2aJbt26Oa9gLmZLvXNycgQAcfnyZavHKOF6r1q1SqhUKnH8+HHTNrldbyGEOHfunAAgNm7caPUYd93DZdUydP36dfz6669ITU01bfPx8UFqaiq2bt1q8Tlbt241Ox4ANBqN6fj8/HycOXPG7JjQ0FAkJydbPae7OVLvmsrLy1FZWVlrsbvc3FyEh4ejY8eOeOqpp3Dx4kWnlr0hHK13aWkpYmJioFarMWzYMOzbt8+0TynX+9NPP8UDDzyAJk2amG2X8vV2RH3fb2e8l3Kg1+tRUlJS6/udl5eHqKgotG3bFg899BD++OMPD5XQubp3747IyEgMGjQIP//8s2m7Uq73p59+itTUVMTExJhtl9v1LioqAoA6F2F11z1cVsHQhQsXoNPpai3D0apVq1p9xkZnzpyp83jjT3vO6W6O1LumF154AVFRUWYfmLvuugtffPEF1q9fj7feegsbN27E4MGDodPpnFp+RzlS744dO+Kzzz7DmjVr8NVXX0Gv16NPnz44efIkAGVc7x07dmDv3r2YOHGi2XapX29HWPt+FxcX4+rVq0757sjBu+++i9LSUowePdq0LTk5GRkZGfjpp5/w8ccfIz8/H/369UNJSYkHS9owkZGRWLBgAVauXImVK1dCrVYjJSUFu3btAuCcv5VSd+rUKfz444+1vt9yu956vR5Tp07F7bffjs6dO1s9zl33cNksx0GOmz17NpYuXYrc3FyzZOIHHnjA9O8uXbqga9euiI+PR25uLgYOHOiJojZY79690bt3b9Pvffr0QUJCAhYuXIjXX3/dgyVzn08//RRdunRBr169zLZ74/UmYMmSJXjttdewZs0as9yZwYMHm/7dtWtXJCcnIyYmBsuWLcNjjz3miaI2WMeOHdGxY0fT73369MHRo0fxwQcf4Msvv/Rgydzn888/R9OmTTF8+HCz7XK73pMnT8bevXslk9ckq5ahFi1awNfXF2fPnjXbfvbsWURERFh8TkRERJ3HG3/ac053c6TeRu+++y5mz56N7OxsdO3atc5j27ZtixYtWuDIkSMNLrMzNKTeRo0aNcLNN99sqpO3X++ysjIsXbrUpj9+UrvejrD2/Q4JCUFQUJBTPkNStnTpUkycOBHLli2r1ZVQU9OmTdGhQwdZX29LevXqZaqTt19vIQQ+++wzjB07Fv7+/nUeK+XrPWXKFHz//ffIyclBdHR0nce66x4uq2DI398fPXr0wPr1603b9Ho91q9fb9YaUF3v3r3NjgeAtWvXmo6Pi4tDRESE2THFxcXYvn271XO6myP1BgwZ9q+//jp++ukn9OzZs97XOXnyJC5evIjIyEinlLuhHK13dTqdDnv27DHVyZuvN2AYglpRUYGHH3643teR2vV2RH3fb2d8hqQqMzMTEyZMQGZmptkUCtaUlpbi6NGjsr7eluzevdtUJ2++3oBhNNaRI0ds+s+OFK+3EAJTpkzBqlWrsGHDBsTFxdX7HLfdw+1K/ZaApUuXioCAAJGRkSH2798vHn/8cdG0aVNx5swZIYQQY8eOFS+++KLp+J9//ln4+fmJd999Vxw4cEDMmjVLNGrUSOzZs8d0zOzZs0XTpk3FmjVrxO+//y6GDRsm4uLixNWrV91eP2vsrffs2bOFv7+/WLFihTh9+rTpUVJSIoQQoqSkRDz33HNi69atIj8/X6xbt07ccsston379uLatWseqaMl9tb7tddeE1qtVhw9elT8+uuv4oEHHhCBgYFi3759pmO88Xob9e3bV9x///21tsvlepeUlIjffvtN/PbbbwKAeP/998Vvv/0mTpw4IYQQ4sUXXxRjx441HX/s2DHRuHFj8fzzz4sDBw6IefPmCV9fX/HTTz+ZjqnvvZQCe+v99ddfCz8/PzFv3jyz7/eVK1dMx0ybNk3k5uaK/Px88fPPP4vU1FTRokULce7cObfXzxp76/3BBx+I1atXi7y8PLFnzx7xzDPPCB8fH7Fu3TrTMd54vY0efvhhkZycbPGccrjeTz31lAgNDRW5ublmn9vy8nLTMZ66h8suGBJCiI8++ki0adNG+Pv7i169eolt27aZ9vXv31+MGzfO7Phly5aJDh06CH9/f5GUlCT+85//mO3X6/VixowZolWrViIgIEAMHDhQHDp0yB1VsYs99Y6JiREAaj1mzZolhBCivLxcpKWliZYtW4pGjRqJmJgYMWnSJEn9wTCyp95Tp041HduqVStx9913i127dpmdzxuvtxBCHDx4UAAQ2dnZtc4ll+ttHDpd82Gs67hx40T//v1rPad79+7C399ftG3bVixevLjWeet6L6XA3nr379+/zuOFMEwxEBkZKfz9/UXr1q3F/fffL44cOeLeitXD3nq/9dZbIj4+XgQGBoqwsDCRkpIiNmzYUOu83na9hTAMFw8KChKLFi2yeE45XG9LdQZg9p311D1c9WcBiYiIiBRJVjlDRERERM7GYIiIiIgUjcEQERERKRqDISIiIlI0BkNERESkaAyGiIiISNEYDBEREZGiMRgiIiIiRWMwRERERIrGYIiIiIgUjcEQEclSZmYmgoKCcPr0adO2CRMmoGvXrigqKvJgyYhIbrg2GRHJkhAC3bt3xx133IGPPvoIs2bNwmeffYZt27ahdevWni4eEcmIn6cLQETkCJVKhTfffBOjRo1CREQEPvroI2zevJmBEBHZjS1DRCRrt9xyC/bt24fs7Gz079/f08UhIhlizhARydZPP/2EgwcPQqfToVWrVp4uDhHJFFuGiEiWdu3ahZSUFCxcuBAZGRkICQnB8uXLPV0sIpIh5gwRkewcP34cQ4YMwUsvvYQxY8agbdu26N27N3bt2oVbbrnF08UjIplhyxARycqlS5fQp08fpKSkYMGCBabtQ4YMgU6nw08//eTB0hGRHDEYIiIiIkVjAjUREREpGoMhIiIiUjQGQ0RERKRoDIaIiIhI0RgMERERkaIxGCIiIiJFYzBEREREisZgiIiIiBSNwRAREREpGoMhIiIiUjQGQ0RERKRoDIaIiIhI0f4fQ1h4o3hzkSwAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients for OLS\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1bc83f33",
   "metadata": {},
   "source": [
    "## Same code but now with momentum gradient descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "dc2a3f65",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.]\n",
      " [3.]]\n",
      "Eigenvalues of Hessian Matrix:[0.26136301 4.38259367]\n",
      "0 [-12.42926102] [-13.7797646]\n",
      "1 [-0.35774662] [0.30560116]\n",
      "2 [-0.33641182] [0.28737615]\n",
      "3 [-0.31634936] [0.27023801]\n",
      "4 [-0.29748336] [0.25412193]\n",
      "5 [-0.27974246] [0.23896696]\n",
      "6 [-0.26305957] [0.22471578]\n",
      "7 [-0.24737159] [0.2113145]\n",
      "8 [-0.23261919] [0.19871242]\n",
      "9 [-0.21874657] [0.18686188]\n",
      "10 [-0.20570127] [0.17571808]\n",
      "11 [-0.19343394] [0.16523885]\n",
      "12 [-0.1818982] [0.15538456]\n",
      "13 [-0.17105041] [0.14611795]\n",
      "14 [-0.16084954] [0.13740398]\n",
      "15 [-0.15125702] [0.12920967]\n",
      "16 [-0.14223657] [0.12150404]\n",
      "17 [-0.13375406] [0.11425795]\n",
      "18 [-0.12577742] [0.107444]\n",
      "19 [-0.11827649] [0.1010364]\n",
      "20 [-0.11122288] [0.09501094]\n",
      "21 [-0.10458992] [0.08934481]\n",
      "22 [-0.09835253] [0.08401658]\n",
      "23 [-0.09248712] [0.07900612]\n",
      "24 [-0.08697151] [0.07429446]\n",
      "25 [-0.08178482] [0.06986379]\n",
      "26 [-0.07690745] [0.06569735]\n",
      "27 [-0.07232095] [0.06177939]\n",
      "28 [-0.06800798] [0.05809507]\n",
      "29 [-0.06395221] [0.05463048]\n",
      "theta from own gd\n",
      "[[3.76990501]\n",
      " [3.19655615]]\n",
      "0 [-0.06013832] [0.05137251]\n",
      "1 [-0.05655187] [0.04830882]\n",
      "2 [-0.05210338] [0.04450874]\n",
      "3 [-0.04766156] [0.04071437]\n",
      "4 [-0.04348664] [0.03714799]\n",
      "5 [-0.03964077] [0.0338627]\n",
      "6 [-0.03612297] [0.03085765]\n",
      "7 [-0.03291338] [0.02811589]\n",
      "8 [-0.02998766] [0.02561663]\n",
      "9 [-0.02732158] [0.02333916]\n",
      "10 [-0.02489239] [0.02126405]\n",
      "11 [-0.02267913] [0.0193734]\n",
      "12 [-0.02066265] [0.01765085]\n",
      "13 [-0.01882546] [0.01608144]\n",
      "14 [-0.01715161] [0.01465158]\n",
      "15 [-0.01562659] [0.01334885]\n",
      "16 [-0.01423717] [0.01216195]\n",
      "17 [-0.01297129] [0.01108058]\n",
      "18 [-0.01181796] [0.01009536]\n",
      "19 [-0.01076718] [0.00919775]\n",
      "20 [-0.00980983] [0.00837994]\n",
      "21 [-0.0089376] [0.00763484]\n",
      "22 [-0.00814292] [0.006956]\n",
      "23 [-0.0074189] [0.00633751]\n",
      "24 [-0.00675926] [0.00577402]\n",
      "25 [-0.00615826] [0.00526063]\n",
      "26 [-0.00561071] [0.00479289]\n",
      "27 [-0.00511184] [0.00436673]\n",
      "28 [-0.00465732] [0.00397847]\n",
      "29 [-0.00424322] [0.00362473]\n",
      "theta from own gd wth momentum\n",
      "[[3.98520854]\n",
      " [3.01263545]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients for OLS\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x#+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 30\n",
    "\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(theta)\n",
    "    theta -= eta*gradients\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "# Now improve with momentum gradient descent\n",
    "change = 0.0\n",
    "delta_momentum = 0.3\n",
    "for iter in range(Niterations):\n",
    "    # calculate gradient\n",
    "    gradients = training_gradient(theta)\n",
    "    # calculate update\n",
    "    new_change = eta*gradients+delta_momentum*change\n",
    "    # take a step\n",
    "    theta -= new_change\n",
    "    # save the change\n",
    "    change = new_change\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"theta from own gd wth momentum\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ef007d0",
   "metadata": {},
   "source": [
    "## But noen of these can compete with Newton's method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "0e498aa4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.96856022]\n",
      " [2.98116672]]\n",
      "Eigenvalues of Hessian Matrix:[0.3407072  4.46918198]\n",
      "0 [-14.47784776] [-17.95282048]\n",
      "1 [2.49106291e-15] [-6.96409595e-15]\n",
      "2 [8.32667268e-17] [8.72456212e-17]\n",
      "3 [8.32667268e-17] [8.72456212e-17]\n",
      "4 [8.32667268e-17] [8.72456212e-17]\n",
      "beta from own Newton code\n",
      "[[3.96856022]\n",
      " [2.98116672]]\n"
     ]
    }
   ],
   "source": [
    "# Using Newton's method\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "beta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(beta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "# Note that here the Hessian does not depend on the parameters beta\n",
    "invH = np.linalg.pinv(H)\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "beta = np.random.randn(2,1)\n",
    "Niterations = 5\n",
    "\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(beta)\n",
    "    beta -= invH @ gradients\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"beta from own Newton code\")\n",
    "print(beta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40292cf3",
   "metadata": {},
   "source": [
    "## Including Stochastic Gradient Descent with Autograd\n",
    "In this code we include the stochastic gradient descent approach discussed above. Note here that we specify which argument we are taking the derivative with respect to when using **autograd**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "fa819b9d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.20603792]\n",
      " [2.82345148]]\n",
      "Eigenvalues of Hessian Matrix:[0.26589296 4.18756348]\n",
      "theta from own gd\n",
      "[[4.20603792]\n",
      " [2.82345148]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHHCAYAAAC88FzIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABY0UlEQVR4nO3deVxU5f4H8M8BBNQERZRFEHBDwdTU3E1NjPGaV6/ZomVmaZt282bd8v5K816LrG7ZLUttUVvUXFC73RrcQHFJzaVyB0VDFFFUQFCQmef3xzSjAzPMDMxyzpzP+/XiVZzznHOeM4vny7N8H0kIIUBERESkUj6ergARERGRJzEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiK7PfbYY4iNjfV0NWQtIyMDkiRh1apVnq4KEdmJwRCRDC1evBiSJJl+/Pz80KJFCzz22GPIy8vzdPWIiLyKn6crQETW/fOf/0RcXByuX7+On376CYsXL8a2bdtw8OBBBAYGerp6RERegcEQkYwNHToU3bt3BwBMnDgRoaGhmDNnDr777js88MADHq4deVJpaSkaNmzo6WoQeQV2kxEpSP/+/QEAJ06cMG2rqKjAjBkz0K1bNwQHB6Nhw4bo378/0tPTzY49deoUJEnCu+++i4ULF6J169YICAjAnXfeiT179lS71tq1a9GxY0cEBgaiY8eOWLNmjcU6lZaWYtq0aYiOjkZAQADi4+Px7rvvQghhVk6SJEyZMgUrV65EQkIC6tevj969e+O3334DACxYsABt2rRBYGAgBg4ciFOnTtl8PV5//XVIkoTs7Gw89thjaNy4MYKDgzFhwgSUlZVVu/fFixdXO4ckSXj99dernfP48eN45JFHEBwcjGbNmuG1116DEAK5ubkYMWIEgoKCEB4ejn//+98W66bT6fCPf/wD4eHhaNiwIf785z8jNze3Wrldu3ZBo9EgODgYDRo0wIABA7B9+3aL93n48GGMHTsWTZo0Qb9+/QAA+fn5mDBhAqKiohAQEICIiAiMGDHCrtePiAzYMkSkIMYHXJMmTUzbiouL8dlnn2HMmDGYNGkSSkpK8PnnnyM5ORm7d+9Gly5dzM6xdOlSlJSU4KmnnoIkSXj77bcxatQonDx5EvXq1QMArF+/Hvfddx8SEhKQkpKCwsJC0wP3VkII/PnPf0Z6ejqeeOIJdOnSBWlpaXjppZeQl5eH999/36x8ZmYmvvvuO0yePBkAkJKSgnvvvRd///vf8fHHH+PZZ5/F5cuX8fbbb+Pxxx/H5s2b7XpdHnjgAcTFxSElJQX79u3DZ599hubNm2POnDmOvLxmHnzwQXTo0AFvvfUW/ve//2H27NkICQnBggULcPfdd2POnDn45ptv8OKLL+LOO+/EXXfdZXb8G2+8AUmS8PLLL6OgoABz585FUlISDhw4gPr16wMANm/ejKFDh6Jbt26YOXMmfHx8sGjRItx9993IzMxEjx49zM55//33o23btnjzzTdNweZ9992HQ4cO4bnnnkNsbCwKCgqwYcMG/P777xzsTmQvQUSys2jRIgFAbNy4UVy4cEHk5uaKVatWiWbNmomAgACRm5trKltZWSnKy8vNjr98+bIICwsTjz/+uGlbTk6OACCaNm0qLl26ZNq+bt06AUD897//NW3r0qWLiIiIEFeuXDFtW79+vQAgYmJiTNvWrl0rAIjZs2ebXX/06NFCkiSRnZ1t2gZABAQEiJycHNO2BQsWCAAiPDxcFBcXm7ZPnz5dADAra8nMmTMFALP7FEKIv/zlL6Jp06bV7n3RokXVzgFAzJw5s9o5n3zySdO2yspKERUVJSRJEm+99ZZp++XLl0X9+vXF+PHjTdvS09MFANGiRQuze1qxYoUAID744AMhhBB6vV60bdtWJCcnC71ebypXVlYm4uLixJAhQ6rVacyYMWZ1v3z5sgAg3nnnnRpfJyKqGbvJiGQsKSkJzZo1Q3R0NEaPHo2GDRviu+++M2uh8fX1hb+/PwBAr9fj0qVLqKysRPfu3bFv375q53zwwQfNWpaMXW8nT54EAJw7dw4HDhzA+PHjERwcbCo3ZMgQJCQkmJ3rhx9+gK+vL/7617+abZ82bRqEEPjxxx/Ntg8ePNistaJnz54ADK0bjRo1qrbdWCdbnn76abPf+/fvj8LCQhQXF9t1vCUTJ040/b+vry+6d+8OIQSeeOIJ0/bGjRsjPj7eYj0fffRRs3saPXo0IiIi8MMPPwAADhw4gKysLIwdOxaFhYW4ePEiLl68iNLSUgwePBhbt26FXq+v8T7r168Pf39/ZGRk4PLly7W+VyK1YzcZkYzNmzcP7dq1Q1FREb744gts3boVAQEB1cotWbIE//73v3H06FHcuHHDtD0uLq5a2ZYtW5r9bgyMjA/T06dPAwDatm1b7dj4+HizAOv06dOIjIw0e+gDQIcOHczOZe3axmArOjra4nZ7H/A13VNQUJBd57B1zuDgYAQGBiI0NLTa9sLCwmrHV339JElCmzZtTF2dWVlZAIDx48dbrUNRUZFZ4Fr1/QwICMCcOXMwbdo0hIWFoVevXrj33nvx6KOPIjw83PZNEhEABkNEstajRw/TbLKRI0eiX79+GDt2LI4dO4bbbrsNAPD111/jsccew8iRI/HSSy+hefPm8PX1RUpKitlAayNfX1+L1xJVBjy7grVr17VOto6XJMnifp1O59A5nfnaGVt93nnnnWrjuoyM77GRcazRraZOnYrhw4dj7dq1SEtLw2uvvYaUlBRs3rwZd9xxh8P1IlIjBkNECmEMcAYNGoSPPvoIr7zyCgBg1apVaNWqFVJTU80e+jNnzqzVdWJiYgDcbLm41bFjx6qV3bhxI0pKSsxah44ePWp2Lk8ztq5cuXLFbHvVlitnqvr6CSGQnZ2NTp06AQBat24NAAgKCkJSUlKdrtW6dWtMmzYN06ZNQ1ZWFrp06YJ///vf+Prrr+t0XiK14JghIgUZOHAgevTogblz5+L69esAbrZW3No6sWvXLuzcubNW14iIiECXLl2wZMkSFBUVmbZv2LABhw8fNiv7pz/9CTqdDh999JHZ9vfffx+SJGHo0KG1qoOzBQUFITQ0FFu3bjXb/vHHH7vsml9++SVKSkpMv69atQrnzp0zvSbdunVD69at8e677+Lq1avVjr9w4YLNa5SVlZk+B0atW7dGo0aNUF5eXsc7IFIPtgwRKcxLL72E+++/H4sXL8bTTz+Ne++9F6mpqfjLX/6CYcOGIScnB/Pnz0dCQoLFh6w9UlJSMGzYMPTr1w+PP/44Ll26hA8//BCJiYlm5xw+fDgGDRqE//u//8OpU6fQuXNnrF+/HuvWrcPUqVNNrR9yMHHiRLz11luYOHEiunfvjq1bt+L48eMuu15ISAj69euHCRMm4Pz585g7dy7atGmDSZMmAQB8fHzw2WefYejQoUhMTMSECRPQokUL5OXlIT09HUFBQfjvf/9b4zWOHz+OwYMH44EHHkBCQgL8/PywZs0anD9/Hg899JDL7o3I2zAYIlKYUaNGmVoUJk2ahMceewz5+flYsGAB0tLSkJCQgK+//horV65ERkZGra6h0WiwcuVKvPrqq5g+fTpat26NRYsWYd26dWbn9PHxwXfffYcZM2bg22+/xaJFixAbG4t33nkH06ZNc84NO8mMGTNw4cIFrFq1CitWrMDQoUPx448/onnz5i653j/+8Q/8+uuvSElJQUlJCQYPHoyPP/4YDRo0MJUZOHAgdu7ciX/961/46KOPcPXqVYSHh6Nnz5546qmnbF4jOjoaY8aMwaZNm/DVV1/Bz88P7du3x4oVK3Dfffe55L6IvJEk3DFqkoiIiEimOGaIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqqkiz5Ber8fZs2fRqFEjq2sUERERkbwIIVBSUoLIyEj4+Liu/UYVwdDZs2errYpNREREypCbm4uoqCiXnV8VwZBxAcnc3FwEBQV5uDZERERkj+LiYkRHR5stBO0KqgiGjF1jQUFBDIaIiIgUxtVDXDiAmoiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFTN48HQ1q1bMXz4cERGRkKSJKxdu9Zq2aeffhqSJGHu3Lluqx8RERF5N48HQ6WlpejcuTPmzZtXY7k1a9bgp59+QmRkpJtqRkRERGrg5+kKDB06FEOHDq2xTF5eHp577jmkpaVh2LBhbqoZERERqYHHW4Zs0ev1GDduHF566SUkJiZ6ujpERETkZTzeMmTLnDlz4Ofnh7/+9a92H1NeXo7y8nLT78XFxa6oGhEREXkBWbcM7d27Fx988AEWL14MSZLsPi4lJQXBwcGmn+joaBfWkoiIiJRM1sFQZmYmCgoK0LJlS/j5+cHPzw+nT5/GtGnTEBsba/W46dOno6ioyPSTm5vrvkoTERGRosi6m2zcuHFISkoy25acnIxx48ZhwoQJVo8LCAhAQECAq6tHREREXsDjwdDVq1eRnZ1t+j0nJwcHDhxASEgIWrZsiaZNm5qVr1evHsLDwxEfH+/uqhIREZEX8ngw9PPPP2PQoEGm31944QUAwPjx47F48WIP1YqIiIjUwuPB0MCBAyGEsLv8qVOnXFcZIiIiUh1ZD6AmIiIicjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFTN48HQ1q1bMXz4cERGRkKSJKxdu9a078aNG3j55Zdx++23o2HDhoiMjMSjjz6Ks2fPeq7CRERE5FU8HgyVlpaic+fOmDdvXrV9ZWVl2LdvH1577TXs27cPqampOHbsGP785z97oKZERETkjSQhhPB0JYwkScKaNWswcuRIq2X27NmDHj164PTp02jZsqVd5y0uLkZwcDCKiooQFBTkpNoSERGRK7nr+e3nsjO7SFFRESRJQuPGja2WKS8vR3l5uen34uJiN9SMiIiIlMjj3WSOuH79Ol5++WWMGTOmxggxJSUFwcHBpp/o6Gg31pKIiIiURDHB0I0bN/DAAw9ACIFPPvmkxrLTp09HUVGR6Sc3N9dNtSQiIiKlUUQ3mTEQOn36NDZv3myz3zAgIAABAQFuqh0REZHK6XRAZiZw7hwQEQH07w/4+nq6VnaTfTBkDISysrKQnp6Opk2berpKREREZJSaCjz/PHDmzM1tUVHABx8Ao0Z5rl4O8HgwdPXqVWRnZ5t+z8nJwYEDBxASEoKIiAiMHj0a+/btw/fffw+dTof8/HwAQEhICPz9/T1VbSIiIkpNBUaPBqpOTM/LM2xftUoRAZHHp9ZnZGRg0KBB1baPHz8er7/+OuLi4iwel56ejoEDB9p1DU6tJyIicjKdDoiNNW8RupUkGVqIcnJq3WWmmqn1AwcORE3xmIzSIBEREZFRZqb1QAgwtBbl5hrK2dl44SmKmU1GREREMnLunHPLeRCDISIiInJcRIRzy3kQgyEiIiJyXP/+hjFBkmR5vyQB0dGGcjLHYIiIiIgc5+trmD4PVA+IjL/PnauIfEMMhoiIiKh2Ro0yTJ9v0cJ8e1SUYqbVAzKYTUZEREQKNmoUMGIEM1ATERGRivn6yn76fE3YTUZERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlXz83QFiIiIvJ5OB2RmAufOARERQP/+gK+vp2tFf2AwRERE5EqpqcDzzwNnztzcFhUFfPABMGqU5+pFJuwmIyIicpXUVGD0aPNACADy8gzbU1M9Uy8yw2CIiIjIFXQ6Q4uQENX3GbdNnWooRx7FYIiIiMgVMjOrtwjdSgggN9dQjjyKwRAREZErnDvn3HLkMgyGiIiIXCEiwrnlyGUYDBEREblC//6GWWOSZHm/JAHR0YZy5FEMhoiIiFzB19cwfR6oHhAZf587l/mGZIDBEBERkauMGgWsWgW0aGG+PSrKsF0teYZ0OiAjA1i2zPBfmc2gY9JFIiIiVxo1ChgxwrszUNeUYVsBSSclISwlQPAuxcXFCA4ORlFREYKCgjxdHSIiIu9RU7ADGJJLVg01jN2ENlrH3PX8ZjBEREREtWPMsG0p2BECaNoUKCy0fKwkGYKmnByrrWTuen5zzBARERE5zp4M29YCIWMZmSSdZDBEREREjrOVYdteMkg66fFgaOvWrRg+fDgiIyMhSRLWrl1rtl8IgRkzZiAiIgL169dHUlISsrKyPFNZIiIiMnBWECODpJMeD4ZKS0vRuXNnzJs3z+L+t99+G//5z38wf/587Nq1Cw0bNkRycjKuX7/u5poSERGRSV2DGBklnfT41PqhQ4di6NChFvcJITB37ly8+uqrGDFiBADgyy+/RFhYGNauXYuHHnrInVUlIiIiI2OG7bw8y+OGJAlo0AAoLbV+DpkknfR4y1BNcnJykJ+fj6SkJNO24OBg9OzZEzt37rR6XHl5OYqLi81+iIiIyIlsZdgWouZA6MUXZZNnSNbBUH5+PgAgLCzMbHtYWJhpnyUpKSkIDg42/URHR7u0nkRERKpkLcN2SIjtY5cvl00malkHQ7U1ffp0FBUVmX5yc3M9XSUiInI2mS/xoBqjRgGnTgHp6cDSpcDGjdYXp72VTKbVAzIYM1ST8PBwAMD58+cRcctArfPnz6NLly5WjwsICEBAQICrq0dERJ6igCUealyiwtv4+gIDBxr+PyMDuHjRvuNszEgrOVdSp2rZS9YtQ3FxcQgPD8emTZtM24qLi7Fr1y707t3bgzUjIiKPMWY9rprjJi/PsD011TP1ulVqKhAbCwwaBIwda/hvbKw86uZqjky5rzIjTegFfllxDHOGZmBQk/2Ibe+ehg2PtwxdvXoV2dnZpt9zcnJw4MABhISEoGXLlpg6dSpmz56Ntm3bIi4uDq+99hoiIyMxcuRIz1WaiIg8w1bWY0kCpk41LIzqqVYYa0tUGIM1b1+t3t4p982aAf37ozDrEjZ8eBTaH/RIy2mLfH08gPg/CrlnApTH1ybLyMjAoEGDqm0fP348Fi9eDCEEZs6ciYULF+LKlSvo168fPv74Y7Rr187ua3BtMiIiL5GRYWhlsSU9/Wa3jTvpdIYWIGuZme1Yj0vxbLwGxqBjZYeZeO/30dhdmgBxS0dVA5Ti7uaHoBlwDX0eaYquI27nQq3OwGCIiMhLLFtm6HayZelSYMwY19enKrkHa+5irXUMhmDobbyEV/C2advtgcehuf0sku8PQr+nEhEQZOgec9fz2+PdZERERHaztwvGU0s82DteRgbrcblSedIwHH/0bUQvfRuNb1wwbT+PZpiMj7FJSsL9LXZCk1SJ5Mlt0KJ7OwD29/g4G4MhIiJSDnuyHkdFeW6JB2cGawqbjXZi82lo559C2tZAbD6fiFK8CB/8Df2RiUjkITiwAmF3tsS0R5pj+aO3wS9QPhOhGAwREZFyGLMejx59M8uxkTG3jSeXeHBWsKaA1AGlBaXI+PgwtKll0B6NQfaNWAAxpv1hPgXQxB1H8lB/DJmiQWh8U4/V1RaOGSIiIuWxFCxERxsCIU8HC8bxMoDlYM3WbDJr423sPd5FhF7g0LpsaD/Pg3ZHEDIvJ6ICN6e+++EG+gYfgqbXFWgmRKDTfW3h41e3DD7uen4zGCIiImWSczdSbYM1mc1Gu5xzBZs+OgLt95XQZrdBnt68ey/WLxeadjnQjAjAoGc7ICjKuc9YBkNOxGCIiIjcrjbBmodno+kr9dj7zVFovyyAdk8IfipJhB436xyIaxjU7CA0/cuQPDEa7ZLjIPnYsfRGLXE2GRERkZLdukSFvTwwGy3/1wKs/+g4tOslrP+9PQpFAoAE0/4O/ieg6ZgLzehG6P9UAuqH3Om0a8sFgyEiIiK5cEPqgBtlN7Dj00NI+/YKtAfCsP9aBwDNTfuDUISkyCPQDK5A8jOt0bJ3awCta309JWAwREREJBe2ZqMBhhanCxcs77Pi1LYzSPvkJLTp/th0LgEl6GK2v1uDw0juUgDNmCbo9XgC6jXoVcsbUCYGQ0RERHJxa+oAa3Q64MEHDWUtDcbW6VD+v404+vXPyNzXEB//PgxHbrQFEGUqEipdRHLMMWg0wD1T2qF5onnXmNpwADUREZHcrFoFPPSQIfCxpMqsMqEXOPrDSWT/ayl6/fwRmukLTEVzEYW/4T2cD2oHTY/L0IwPwx0Pxdd52rs7cAA1ERGpm5ynzrtaaKj1QAgwdKHl5iLzkQX4+kBHaLNaoZvuF6zCTNxcCtUgCmewUnoQ0iLP5CdSAgZDREQkPwrIwOxSds4W+3h5EyzHXfCBDtvxV0gQqDrR3fT71KnAiBHqCSgdIP82MiIiUhdjBuaqiQfz8gzbU1M9Uy93snO2mJ8v8NfOW7DjsU8RhbxqgZDJHy1JyMy0fjKdzpDnaNkyw39rapnyMgyGiIhIPnQ6Q4uQpeGsxm1Tp3rtg7ryeiW2f/IrZryqw3kpDHor5QSAymYR+Kr8AXxwYAB63hNs3wWstTilphoyXw8aBIwda/hvbKw6Ak8wGCIiIjnJzLS+FAVgXwuHwuTuOovPxmdidNROhNYvRb9nO+Ff2wdjsXjUakuPJEnwm//RzS6vuuQnYkscxwwREZGMeCADs7tdv3Id2xYehnZFMbQHW+BQeVsAkQAAH+gwHN/h6du+wdCrK6yf5MUXzcdO2cpPZJx91r+/+XZbLXGSpIqxRgyGiMjz1DxriMy5IQOzuwm9QPam09AuOA3t1gZIv5CIa+hq2u8DHXredhjPx6zByLMfI+DyeeBqDSeUJGD5ciAl5eb35Nb8RJJkHtwYV7ufO7f698qRljgXrIUmFwyGiMiz1D5riMzVtoVDZkrOliB93mFo15Uj7VgMTlbGAog17Y/wyYemdRY0w3yRNKUDQn7JAka/bj3r9K2sBSijRhnyE1n6Ps2da/n7pIKWOHswGCIizzGOVaj6ADCOVVjFvCiqU9sWDg8TeoFfVx1H2uJz0O4MxrYribiBnqb99VCB/k0OQtO7GJrHI9HxL20h+YQbdup0wEArXVU1sRSgjBpl6NKyt6XVC1viaoMZqInIM3Q6w2wVa030VTLskspYajGMjrbewuEBhVmXsPGjo9D+oEPaybY4pw8329/K7zSGdjiF5BH1MWhyAm4Lv83yiTIyDLO3HJWeXveuK+P30FZLnIe+h8xATUTerTZjFTi2yPnk+po62sLhBroKHfZ8eQTary9C+3Mo9pa2Q19UIALn0A46lKABBjQ/Cs1d16B5OhZtBscAiLF9Yke7oJzZVajQljhnYzBERJ7h6FgFOYwtkmvgUFtyeE1r4uvr8UG7Z/flI+2jLKRt9MX6Mx1wWXQEAPwFqViJoYjGzddORLaA9OF/HH/tHOmCckWAUpuxRl6G3WRE5Bn2dg1s3Ahs3w7MnFl9n/HB4I6xRXIPHBxlbbyWO19TGaq4WoHtCw9B+20RtL9G4Nfr8Wb7G0tX8I8mn+DFS/8AAPM8QLV97Wx1Vd3KlV2FMgz23fX8ZjBERJ5hz1iFkBAgIAA4e9b6edwxpsHbAgeO1zJzMuN3aOfnQJsRiM3nE1GKm2N7JOjRveERaLpegObhpujxSDv4tW/j/NfO+BkDLH8fjLl+ZBCguBODISdiMEQkU9YeAFXHLtjDGYNJLfHGwMHeVjlXvaYeVlpQii2fHIY2tQzaIy2RdSPObH+YTwGSY49D8ycfDJkSj9D4pjd3uvK1U8CgcXfjAGoi8n7Wxiq0aAFcuwYUFtp/LlflQfHGpHQqyy0j9AKH/3sC2s/OQLujEbZe6ogK3Gna74cb6Bt8CMk9r0DzWDg6398OPn7NLZ/Mla+dDAeNqwWDISLyLEsPAJ0OSEpy7DyuyoMih8DB0bEctsqrILfMldNF2Pifw9B+X4m0E61xRtcGQBvT/hjfM9C0OwnNiADcPbkDgqK62HdiV792Mhg0rkYMhojI86o+AJYts/9YV2ck9nTg4OjAbXvKe0mW51vpK/XYt/QotF8WQLs7BD+VJECH3qb9gbiGgaEHoelfBs2kaLRLjoPkE+X4hdz92slwULNXEipQVFQkAIiioiJPV4WI7JGeLoThUWP7R5KEWL3adXWprBQiKspwHWvXj442lHO21astX1eSLN+3I+WNZauWt3ZuGcr/rUB8+VSmGBuzTYRKF6rddgf/bDH1jgyhnb1HlBWWOe/C7nrtVq82fPZuvUZUlCLeG2dx1/ObwRARyY+tAMTdDwZPBA7G16CmIPDWIMzR8sb7qnpMdLRsH7YVpRViy38OiOm900XX+oer3WIjFIm/ROwUCx7eIk5ty3VtZVz92jkaCHspdz2/OZuMiOTJ1lTjWbOA//s/93UZuHumj6Ozlmo7y0nm3TCnt59B2icnod3sj43nElAC83/Du9Y/As0d55H8UBP0fiIB9RrUc1/l7HntavP6euMMxlribDIiUjdrM808NdXY3TN9HB24XduB3jIbsHvt0jVs/eQQtKtLoT0UhaMVrQHcHNsTKl1EcswxJCcL3DMlHmEdOwDo4JnK2nrtapuo0xtnMMocgyEiki+5TTV2Z+Dg6MBtTw/0riWhFzimzYH201ykbWuAjIsdcR3dTft9oEPvRoeg6XEJmkebo+vY9vDx6+vBGtvJWqLOvDzD9poSdcphBqPKsJuMiEiOXUWOriYu89XHb1V8phibPjyMtP9WQHu8FU7rzGd1RfmehaZNNjTD62HwlAQ0jgn2UE1rqa7dXCpPinkrdpMREbmDXNccc3Q1cRmvPq6v1OOXlcehXZwP7a7G2FGUiEr0Mu33RznuCjkITZ8SaCZGIWF4a0g+kW6vp9PUtZurttP35RjUKwSDISJSr7p0ZbiDo6uJO1rehQ/PC0cuYsNHx6DVCqw/1Q7n9e0BtDftb1svB5oOv0MzqgEGPJOAhs27OeW6slDXbq7aBLZyDeoVgt1kRKRO9nZlZGcDO3bULWCoa9DhrAzUt27PygIWLjQEfkZ1eHhWXq/E7iVHoP2mENp9zfBzaQcI+Jj2N8RVDA4/BM3AciQ/FYtWA1s6fA3FcFY3l70zGL1tIeFbuO357dKJ+05QWVkpXn31VREbGysCAwNFq1atxD//+U+h1+vtPgfzDBFRNfYmdgwNrVtuI7kkzrNUjzrmsMndfVZ8Nn6rGN1ih2gsXa52uk6BR8Xfe6SLzf/eJ8pLyl18gzLizESdlZWGz+rSpYb/Vj2mNvmlFIRJF//wxhtviKZNm4rvv/9e5OTkiJUrV4rbbrtNfPDBB3afg8EQkcrYeoAIYdhnTzBUl4BBLonzrNXDwYfn9aLrYsOcvWJat3TRMeB4tUObSJfEg9HbxaInMkXe3nPuuTe5cleiTnuD+vR051zPzdz1/Jb9mKEdO3ZgxIgRGDZsGAAgNjYWy5Ytw+7duz1cMyKSJXvHTtR2irkQhu6HqVMN0/6tdVfpdIZ6WBqJYO85nKGmelhSZXBv9qbT0M4/Be3W+kgvSEQZupqKStCj522HoOleCM0joeg+rgN8/fu46EYUxtHxW7XFafhOIftgqE+fPli4cCGOHz+Odu3a4ZdffsG2bdvw3nvvebpqRGQvd81ycWRAtK0ZOzWxJ+mdXBLn2aqHFV+M24w38uNwsjIGQIxpe4RPPpJbZUHzJ18kTWmPpm1vd2JlvYw78mQpNL+U3Mg+GHrllVdQXFyM9u3bw9fXFzqdDm+88QYefvhhq8eUl5ejvLzc9HtxcbE7qkpElrhrloujLTE1zdixV01/bcvlL/Zanv/LM3fjJGJQDxXo1/gQNL2LoHk8ErePagvJJ9zJlfRirk7UWdtp+GTGx3YRz1qxYgW++eYbLF26FPv27cOSJUvw7rvvYsmSJVaPSUlJQXBwsOknOjrajTUmIhNjS03VlgljS01qqvOu5UhLjJGxK6NFC/OyzZrZd82a/tqWy1/sDp5fDwlnEYGOiXqs+8cuFOaVY/PlO/D3Hwai0+h2kHwkF1WUasUY1AM3Z48ZeTi/lJLIfmp9dHQ0XnnlFUyePNm0bfbs2fj6669x9OhRi8dYahmKjo7m1Hoid3L3YpPLlgFjx9out3QpMGaM+baq3Xh9+gCtW9ctm7NcMkLrdBAxhnpIqPmfe8NeCVi5EtLo+1xXJ3I+dy8k7CbMQP2HsrIy+PiYN2D5+vpCr9dbPSYgIAABAQGurhoR1cTdY2bq0hJjqSujrtmcPZwR+tyB80j7KAtpG3zgnzcbizABAhJ8bgmIBIBb2xIkL3h4qpbc1vFTGNkHQ8OHD8cbb7yBli1bIjExEfv378d7772Hxx9/3NNVI1Imdw1mdveYGWePnXDGbCB3zSgCUHG1Ajs+Owzt8ivQ/hKBX67HAwj7Y28f6OGD931eRKi+wHSMFBUFTJoEtG17s0Vsxw5DKxsfpsrjzoWEvYzsg6EPP/wQr732Gp599lkUFBQgMjISTz31FGbMmOHpqhEpjztT9rt7zIyzWmKqBosnTtQtA7Wjf7E7EKzmbM2F9pMcaDMCsDk/AVfR5eYtQ49uDY5C07UAmrEh6DlhDPzqjbV+7tRUQ9cgl3MgFZL9mCFn4HIcRHB/yn5PjZmpy9gJa8Hi++8DoaHuSQ1QQ7BadrEMGfMOQZtahrQjLXH8RpzZ4c2lC0iOOwaNRsKQKfFo1iHU/ut66XIOpGzuen4zGCJSA3cPZjYyPmQByy01rnrI1qYr0FpAYIkrWkysXF9AAiDwz9veRsrV51COQNM+X1Sib/BBaHpeQfKjYejyYDx8/BycJOypz4YScVV4t2Mw5EQMhkj1nLVwZG0oYZaLrYCgKmcHczaur4eEM4hCHHIQ5XsOmrYnofmzP+6e3AHBLYPrdm1PfjaUhKvCe4QsZ5Pl5uYyZw8pA/+CM+fJBIBKmOXiaJZmJy6noa/UI+u1rxBfw/V9INASuchJ+RbRfx8DySeq1terRi7JIeXMkczmpEgOtae2b98eM2bMQFlZmavqQ1R3qamGv7IHDTLknRk0yPC7MxP8KY0nEwAqITCtzYPeUhJHOxUcuoCvn9mOR+K2I9y/EK+/ZV8qkJYxkvOTHsolOaRc2cpsDhiCYp3OrdUi53IoGNqwYQPS0tLQtm1bLF682EVVIqoDd2Y8VhLjtPOqGWqNJMnQdeXslP1KCUzr8qC3I5C6UXYDmR/9gv/rm4FuDY4grGMzjJvfF9+c6osLohmKYGfzvysCEk99NpSiNpnNSXlqs9T9kiVLRFRUlOjatavYunVrbU7hVkVFRQKAKCoq8nRVyJUqK4WIihLC8M9T9R9JEiI62lBOjVavNrwGklT9dZEkw35XXM/S++CK69WF8bNjqb62ftLTLZ7y9I4zYsHDW8SoyJ0iCFeqHXZH/cNieu90seU/B0RF8bWar+/qz667PxueVFlpeM+WLjX819ZrunSpfZ+DpUvdUXvVcdfzu1bBkBBClJaWitdee000bNhQ3HfffeLkyZPOrJdTMRhSifT0Oj28VGH16uoBY3S08x92SgxMrQUENf00bWq6h7LCMpH25s/ib13TRQf/7OpFpYtibMw2sWRSpjj3y3n7r++ugMRdnw1PsnSPUVE13yP/XfEoRQRDmZmZ4vnnnxc+Pj4iMDBQvPTSS6KkpMSZ9XMKBkMqwb/g7OPoX8a1odQHiKWHpZUfPSBuBDURH4zcJDShu0UgysyK+KBS9Gn0i/jn3eli9+JDorLcjtfZ0wGJOz4bnlLblkpbrYZyDOy9iLue3w5NrZ8/fz727NmDPXv24MiRI/Dx8UHHjh3Rq1cvdO7cGcuXL0d2djZSU1PRvXt31/Tr1QKn1qsEpwjLR10WTfU044DvTZuA2bNtFh+IdGzBQABAC59z0LTJhuZePwye0gFN4hrX/vpyHnCuNHXNpeSpfFkkzzxD0dHR6NmzJ3r16oVevXqhW7duqF+/vlmZN998E0uXLsXBgwedXtnaYjCkEnJZJZy8IzC1M6B7s+Fs+A/sC80TLZA4oo3zZ3tR3Tnj86iEfFleSJbBkD3Onz+PyMhI6GQ0zZDBkIrwLzh5UHBgevFYITZ8dAz+33yO+y5/YfsAOQd0ZOCslkq22rmdLJMu2qN58+bYvHmzs09LZB83rhLuMt7wD66zFk11g8rrldi95AjSlhZCuzcUe0oTMBL5WIUvIABYbecxBnRqnXKuJM7KpcRV4b0Wl+Mg76TUgMLbUv7LtGsh7+dzSJuXDe1GP2zI64ArorFpnw90yEMUwpBvPRACDMGQvS2NSv08egsFt1SqnWK7yeSIwZCKKekh5K0rh8vgPSgvLse2BYegXVEM7W8tcLC8rdn+JtJlDIk6Ak2SDvf2LECzp0fbPumsWcCMGbbLuTrAdfXrq9MZxtxkZBh+HzjQ8CPX75E17EJXJLc9v106V00mOLVepWqTU8RTlJiXR+ayNp4SH47OEPc23yUa4Kr5ywmd6NnwNzFzQLrY+elv5tPenZmiwdWJJ139GV+92pBLyVJ+JTl+j2zxdOoCcpgsp9YrFVuGVEhprSzeMPvKw67mX0XGx4ehXXsN2iOxOFEZY7Y/3Oc8NK2OI3moL4Y81x5N24ZYPpGz3gtb07kBICQEWLGidi0trv6Mp6YC991Xc5nVq+X1PbKHDFoqyX7sJnMiBkMqU9ecIp6g5Lw8HiL0AgfXZEH7xVlodwZh2+VEVODmgqd+uIF+jQ9C06sImgkR6DS6nX3T3isqgAYNal5409cXKCsD/P2tl7E3qAIc7zZz9WfcnkAOMFzj1Cn5fI/I6yh2NhmRxzmysKJcWlm4crhdLp24jI0fHUHa/3TQnmiDs/p2ANqZ9sf65WJofA40IwIwaHICGkXe4fhFduywvQK5TmcoV9Pnx44FXE2MCwnb25rj6s+4rfMbnTkjr+8RUS0xGCLvY+9DaN06+fwjblw53NZsF5VN49ZV6LD3m6PQfnUB2j1NsetqAvToY9pfH2UY1OwQkvuXQfNkS7QdEgvJJ7puF7X382OrnCOBqxCG93jqVGDECNstLc6qozOOq+01iGSEwRB5H3sfQnPnGoILOYx58PUF3nsPeOCB6vtklpfH1fJ/LUDah8ehXe+DDbnxKBSJZvsTArKh6XgGmtGN0P/pRAQ2vtO5FXBWK52tALcqR1pzXN2S6MhxKm+tJO/AMUPkfWzlFLlVdLQ8xg5Zmn5tJIO8PK5UcbUCOz47jLRvr0D7SzgOXGtvtj8IRRjS4jA0gyuR/GxrRPeMdG2FnJmTxtp07prYMy7M1XlzOGaIZIJjhohqy5j92NZMGEAeY4eszQoyeu89rwuEcrbmIm1+DrTpAdiUn4Cr6GK2v3uDw0i+owCasSHo+VgH1GvQu+YTOnOGkDOzZ1vLiF4Te1paXJ3h297v0AcfMBAi7+DSifsywTxDMlZZKUR6uiFnS3q6c/PoTJ3qvHwxrqKS/EKlF0rFD7N2i792zhDt6p2sdpvNpALxSFym+PqZbeL8wQLHTu6qXDvOzElTWSnExo1ChIQ49712dd4cb8szRIrDPENOxG4ymXJ1Zl4l5O5RQh1rQegFjnx/AmlfnIF2223YUtgR5Qg07fdFJfoEHYKm52Voxoehy4Px8PHzcfxCrs61Y0+LkyOtUq7IguzqDNHekoGaFIkZqJ2ILUMy5OrMvELcbHWxdB25tLo4M9uxh105fUWsfmmnmNR+i4j2PVPtFqJ9z4hJ7beI1S/tFFdOX6n7BeXQqlabVilnt+YoKdM6kYPYMuREbBmSGXcmRbT1l/i33wLNmnkuG62CW4b0lXrsX34M2iXnod3dBDuLE6G7ZRhiAK5jQNOD0PS7Cs3EaLT/Uyv7kh7ay9OvXV1apZw1xklpmdaJHMQM1E7EYEhm3P0Qs7Zy+kMPGTI/29tN54o0/gpbTbvg0AVsmHccWq1A2ql4XBDNzPbH+5+EJiEXmvsa4q6nE9AgtIHrKuPJrN1yyHIuhzoQuRhnk5H3cnXCuKpGjTIksrs1kLl40ZDTp2oAYi0TsKvGN9maFSSEYUZPZqZH1lCqvF6Jn744DO3SS9Dub469ZQkADAGQD3TQ4AckNf4ZbbsG4/ZXRyBuUCsArdxTOU9m7ZZDlnM51IHIW7i0E04mOGZIZtLT7Rsnk57umus7OtbEHeObLI378PX1yDiQ0zvOiIXjtohRkTtFMK5Uu+0u9Y+Ir9rNEtcah3l2nIonx4TJYayXHOpA5GLuen7XYvoGUR0ZM/NKVsaPSJKhG8tVS0848he1TmdoEbLUhWXcNnWq7bWsbBk1ypC8Lj3dcD6g+jmNrVapqXW7VhXXr1zH+pS9mNY9A4mB2Yjp0wJPfnUXUs/2QhGC0VQqxJiY7VgyaRvO/VKA/V8fxiPHZyLwynm31M8qY6saUP2z5Oqs3XJYS04OdSDyEhwzRJ7hjCnGtR3D48hYk4gI945vcvY4EAuvkZB8kLXhFLQLf4c2swEyLiTiGm6O7fGBDr0aHUJy90vQjGuGbg+3h6+/783zhYUBhYXOqZ8zWBsT5sqs3XIY6yWHOhC5GKfWOxG7yWSqLlOM6zKd2JFuOnd3RTizC9HCa3TJr5l42md+tdNF+pwVj7fdKlb8bYe4dPKy9XPOmuXZLk5rXJm80xpj92nVbjpndp8qoQ5ELuSu5zeDIfKs2jzE6jqGx5GxJu4e3+Sk4Eu/cpXQQxL6KsfpIAkdJHE/lovBTfaKd4ali99Sjwu9Tm+7bpWVNWdQVuM4FVdngFZKHYhchHmGnIjdZF7EWd1I9nbTubsrog5pBwqzLmHDh0eR9r8beOPkGITjHCwNChQARGQL+Px+2rE621s3K/XzWq5IuaDEOhC5AKfWE1nirOnE1hbQjIoyH2vi6gUxqzIOLrcVfPXvD12FDrsXH4b2m0Jo94ZiT2kCBPpgADIQCetpCSQA0tk8x6dc25vqICTEdYPf5cjX1/OBnxzqQKRgDIZIWZyZo8hS/iFLf1HbGzg5g43gSwBIb/MkFsTtwoYzHXBZ3G52+O2Bx/Fk5CbgpB3XyssztPbY25pg76yk559nqwQRKQq7yUhZPLkEgzu7IizMkDqHcEzGPKzBzeCrsXQFQ1ocgSapEslPx6HFtWxg0yZg9mzb1wgNNSSfNLKVRNJWlyEANG0KnD/PYIiInILLcTgRgyEvooLpxCc2n4Z2/imsz/BHxYUraIwrOIcIZKI/BCTc2fAwNN0uQvNwU9z5aAf4BfpZnl7uKHvSGlgba2U8Xg5rYXH8DJHXYDDkRAyGvIwzchTJSGlBKTI+Pgxtahm0R2OQfSPWbH+YTwE0cceh+ZMPkibHIzS+qfkJrC3WaUnVrjdL+20Fk57I62MvVy2bQkQewWDoFnl5eXj55Zfx448/oqysDG3atMGiRYvQvXt3u45nMOSF5PxAtkHoBQ6ty4b28zxodwQh83IiKhBg2u+HG+jX+CCSexZBMyECne5rCx8/K8nibc2uq6pZM+DCBdvlbHUzyrH1hSu4E3kdzib7w+XLl9G3b18MGjQIP/74I5o1a4asrCw0adLE01UjT7J38LNMXM65go0fHoH2+0qknWiDPH1bAG1N+2P9cqFplwPNiAAMerYDgqLusO/EtmbXGb36KjB4sKF78ZFHbJe3NQBdbrOXbC2bIkmGZU5GjJDtZ4SIPEf2wdCcOXMQHR2NRYsWmbbFxcV5sEYkG3J7IN9CV6HD3m+OIu3rC9DuCcFPJYnQo7dpfyCuYVCzg9D0L0PyxGi0S46D5BPt+IXsnV2XkGB4rTIy7Csvt/WsbLVEcQV3IqoD2QdD3333HZKTk3H//fdjy5YtaNGiBZ599llMmjTJ6jHl5eUoLy83/V5cXOyOqiqDHLs3vET+rwVY/9FxaNdLWP97exSKRLP9CQHZSE48A83oRuj/VALqh9xZ94s6ulinA3mMZMOecUDOTLkA8HtCpDYuzW/tBAEBASIgIEBMnz5d7Nu3TyxYsEAEBgaKxYsXWz1m5syZAn8k2r31R/XLcdRlPS+qpqK0QmTM3S9e6ZUu7qh/uNqKFEG4IkZF7hQLx20Rp3eccU0lHFlaxEhJ61nZu/SKi9d04/eEyDO4Ntkf6tWrJ3r37m227bnnnhO9evWyesz169dFUVGR6Sc3N5fBUF3X8yIhhBA5mbli/tgtYmTETtEIRdVezm4NDon/65suMuf9IipKK9xTqdoEN0pYz8oY6FkLbCTJsH/jRiG+/lqI0NCay1YNCi3h94RIVrg22R9iYmIwZMgQfPbZZ6Ztn3zyCWbPno28vDy7zqH62WTOWs9LLtzYhXHt0jVs+fgQtKtLoT0cjWMVrcz2N5Mu4J6Y49BogHumtEPzxGYuqYdFt74OWVnAp586NrtO7l1BjqyFVhN7Z5N52/eEyAtwNtkf+vbti2PHjpltO378OGJiYjxUIwXypsGlLs4jI/QCR384Ce1nudBuuw1bCxNxHTdTOPiiEr2DDkHT4zI048Nwx0Px8PHrW+frOszS69CiBTBrFtC2rX3BjYwHoAOwf3yPLfYum+JN3xMicojsg6G//e1v6NOnD95880088MAD2L17NxYuXIiFCxd6umrK4ezBpZ6Smgrcd1/17Xl5hvwytcwjU/R7ETbPOwLtdxXQZrXC77rWAFqb9kf75kHT9gQ0w/1x9+QOaBzTuQ434QTW8umcPQu8/rrhdfCGh3VtZ7RJkmGpkfffNwSI9rZ4ecv3hIgcJvtuMgD4/vvvMX36dGRlZSEuLg4vvPBCjbPJqlJ9N5kn1/NyFp0OaN4cuHTJ8n4HujD0lXoc+PYYtEvOQ7urCXYUJ0J3y98FAbiOu0IOQdOvBJonotDh3taQfCRn3k3tqakrx5610Gri6OfZG74nRF6GGaidSPXBkDes5/XQQ8C339ouZ+VBdeHIRaz/8Bi0WoH1p+JRIMzH9rSrlwNNwu/QjGqAAc8mokFoAydV3MnsfWC//z4QFibPsUCOqGktNFuWLgXGjLG/vDd8T4i8DMcMyYmzBppWPU+fPsCOHa4fwOrraxhTM3p09bWpjINL586V7z/wK1faFwgBpi6MyuuV2LXoMLRLL0G7rzn2lrWHwM2xPbehBIPDD0MzqBzJT8ch7q44AApI5mlvF83f/nbz/5W8NteoUYZuv9osQutoN5vSvydEVHsunasmE3WamuesnCOWzuPr695cJkqYTl1VZWXNU6ar/Hx/zwfivhY7RDCuVNvdpf4R8XLPdJH+/n5RXlLu6TurHXvz6XjbtPDKSsO9L11qmErvaG4lRyjxe0LkpTi13olq3czmrIUf7V1V3B0LSsp9OnVVDkyvvogQhKEAehjuJ0S6hHuij0Jzjx73TG6LiC5hLqyom9R2HI23dfFY6z5z1ndIad8TIi/FMUNOVKsX01kDVR1dVdzbHlp1tWwZMHasXUVn4nVsuG0UNHcWIvnhUHQf1wG+/l74GtZlHI03Df61lF7AVm4lIlIUjhnyNGflHLF3VXFHz6sSZfWCYc9Q5gr/hnj+t8mY1S7U5XXyuLqMo/GmaeGjRhlWoWcLDhHVEYMha5yVc6S2Dx9vemg5QOgFfl11HNpF55D2UzB2XBmELEShBc7Ax1J5ABIA/28WI0QNgZBR1UDg/HnzQdPWyG01+rqSe+JIIlIEBkPWOLoaeF3PU1VWVu2OU6DCrEvY+NFRaH/QIe1kW5zTxwOIN+1/0+dVfKx/2hT43EoCgJdeMnQbqW2cx62BgE4H/PvfylqNnohIJjhmyBpn5Ryp7YDXqCjg1CmvfJjrKnTY8+URaL++CO3PodhT2sE06BkAGqAUg5ofguaua9A8HYs2A6OAN94wTHu+Nelis2bAvHnA/fe7fJkORXD1oGIiIjfjAGonqvNsMqBuD5faDnj1osGuZ/flI+2jLKRt9MX6Mx1wWTQx298xIAua2/OgeSAI/Z5KREBQgGGHpSAnJMSw7f/+zxAsOmvWnzVKanHioGIi8iIMhpyoTi+msx4uls5ji6MZdGWk4moFti88BO23RdD+GoFfr8eb7W8sXcGQFkegSarEPc+2QdSdFroT7QlyRoxw7fIUSmxxUlLwRkRUAwZDTlTnF9PZGag3bQJmz7ZdXmEtQyczfod2fg60GYHYfD4RpbjNtE+CHnc2PIzkrhehebgpeozvAL/AGoas2ZvaYNEiICnJduVq81q6usWJiIhqxKn1cuKsGSvG8/TvDyxe7NhgVxn+tV9aUIqMjw9Dm1qGtKMtkXUjDkBL0/4wnwIkxx6H5k8+GDIlHqHxHe0/ub2pDTIy7Dufo7PzdDpDi5Cl90cIw3v09NPAtWuOrYxORESyw2DIExxdA0kmXTVCL3BoXTbSvsiDdkcjbL3UERW407TfDzfQN/gQNL2uIHl8ODrf3w4+fs1rdzFnpxZwdFafPcHYhQvAI48Yfpd71xkREVnFYMhTrCXOi4oyH49krasmL8+w3cVdNVdOF2Hjfw5D+30ltNltkKdvC6CtaX+M7xlo2p2EZkQA7n66HYJyrvzRegVAamvttLbZG7wMHOh4K5s9HA3G3PR+EBGR83HMkJGnuqFquq6zlgRxgL5Sj73fHIX2ywKk7QnBTyUJ0N0SMwfiGgaGHoSmfxk0k6LRLjkOko/k/NYrR1IbrFvn/CnlDqyJZrFO7DIjIqozd40Z4qr1QjhvZXpns3eF8vT0Ol0m/7cC8eVTmWJszDYRKl2odvoO/tnib13ThXb2HlFWWFb9BKtXW15BvK6rpRvPW/Xcls7r7JXGKytrXhndhe8HEREZuGvVenaTebgbqkbOWhKkihtlN7Dz88PQLrsM7YEw7L/WAUAz0/4gFGFwxBFo7q5A8jOtENO3NYDWlk9mz0DjqVMNU+AdbS2xtyvRWNaZ61TVNK7LFpUupUJEpFTq7ibzQDeUQ+ztqrFj2vjp7Weg/fgk0tL9sfFcAkpg3tzYtf4RaO44D82YJuj1eALqNajn9jpa5cmZdLXJD6WwlAhERHLFqfXu4KyV6V2lf39DMFaLwcHXLl3D1k8OQbu6FNpDUTha0RpAlGl/qHQRyTHHoNEAQya3Q1jHDgA6OF5HF7VembGW2sAdQdKtLU55eYZWrosXLZfl+l9ERIqk7mDIHQ/yunBgCr7QCxzT5kD7aS60mQ2xpTAR19H95qlQiV6NDkPT4xI0jzZH17Ht4ePXt+51dNaCto5yZ7qBW4Ox+vVrHqx9a0oEIiJSBHV3k7mji8cZrCwJUvbam0jLboO0/1ZAe7wVTuuizA6L8j0LTZtsaIbXw+ApCWgcE+z8ujlrQVtHeDIztE5nedFYrv9FROR0XI7DiWyOGXLng7y2dDroM7bg97X7sf0nX3x2vD+2FXdCJW6O7fFHOQaEHERynxJoJkYhYXhrw7R3V3Pnauk6HRAWBhQWWt7vyvfMnkVjiYjIaThmyB0czQTtAReOXMSGj45BqxVIy7kdBeJus/3t6uUgucPv0IxqgAHPJKBh827ur6Qjs77q6o03rAdCgOvGeVlrjbp8GXj9daBjR7YKEREplLpbhoyctTK9E1Rer8SuRYeRtuwStPua4efSDhDwMe1viKsYHH4ImoHlSH4qFq0GtqzhbG7m6gHNOh3QvLl595Q1S5cCY8Y477pynnVIROSl2DLkTs7OUeOgM3vOIW1eNrQb/bAhLwFF6GS2v3PgMWg6n0PyA8Ho+2Qi/G/r6ZZ6OcxZC9pak5lpXyAEOHfAttxnHZK8yHBRZSKqmbqCocxMQKOx/A+Tqx/ktygvLkfm/EPQriiG9mALHCpvC8NiXgZNpMu4J+oINPfocc+zbRDZNR5AvFvqJmv2zupr2tS509vlPuuQ5EMmiyoTkWPUFQzde6/HVnvP3nQa2gWnkZZZH+kFiShDV9N+H+jQ47bD0HQvhOaRUHQf1wG+/n3cVj/FsLe1569/de5f4p5KH0DKIuds9kRUI3WNGQIQ5I7p1wCu5l9F+rzD0K69Bu3RWJysjDHbH+GTD03rLCQP9UXSlPZo2jbEZXXxGrZm/wGGVqHz550/Vkkpsw7JMziujMglOGbIVeq6Xpa10+oFfkvNgvaLs9DuDMa2K4m4gR6m/fVQgX6ND0HTuwiaxyNx+6i2kHzCnXJt1bBnvbCFC53/sFHArEPyMI4rI1I09QVDgNP+Ybp04jI2fnQE2v/poD3RFuf07QC0M+2P8/sdQ9vnQDOyPgZNTsBt4XfUve5qZ20av6tn/7kzfQApD8eVESmaOoMhIwf/YdJV6PDzV0eg/foitD83xe6rCdDj5tie+ijD3c0PQnPXNSQ/GYM2g2Mg+cho6ru38NTsP3dfl7OSlIPjyogUTX1jhm7dYccyG+cOnEfaR1nQrvfBhjPtcUmEwAc69EcmInAOgfV0CL09AskPNkG/JxMQ2DjQhXdCqsFZScrCcWVELsExQ65Uw+riFVcrsH3hIaStKIL2lwj8cj0eQJhp/yP4Cu/7vIhQfYFhww0ABVFAmw+Axl2rnY/IYZyVpDwcV0akaOprGbIwm+xkxu9IW3AK2owAbM5PwFU0Mh0rQY/uDY8g+Y4LGNthH9p/9iIkTywQSurAWUnKJqNs9kTegAu1OpFZMBQdjfJ/zsGm022gTS2D9khLZN2IMyvfXLqA5Lhj0GgkDJkSj2YdQvmQIvfIyAAGDbJdzo4uXvIQjvUichp2k7lAao8ULM+6GxkTOqEcN8f2+OEG+gQfgqbnFSQ/GoYuD8bDx6+f+cGcOkvuwFlJyufGbPZE5ByqCoYm7H4WxiHULX3PYGi7k0ge7o+7J3dAcMsuNR/Mh1Td8S9m2zgriYjI7VQVDA1usg/3DpCgmRSNeE0cJJ8o+w/mQ6puODvKPv37G14XW7OSnLn2GhGRyvl4ugKOeuuttyBJEqZOnerwsamnumLqmgFo/6dWkHwkxw42PqQkK8dJkmGgJB9S1RlnR1XtZjTOjkpN9Uy95Mg4Kwmo/lnjrCQiIpdQVDC0Z88eLFiwAJ06dXL/xfmQqh2dztAiZKmVw7ht6lRDOTIwZrtu0cJ8e1QUZywSEbmAYoKhq1ev4uGHH8ann36KJk2aeKYSfEg5zpGB53TTqFHAqVOGWWNLlxr+m5PDzxgRkQsoZszQ5MmTMWzYMCQlJWH27Nk1li0vL0d5ebnp9+LiYudVxFNLQSgVB57XHmclERG5hSKCoeXLl2Pfvn3Ys2ePXeVTUlIwa9Ys11WIDyn7ceA5ERHJnOy7yXJzc/H888/jm2++QWCgfet+TZ8+HUVFRaaf3NxcF9eSrOLAcyIikjnZZ6Beu3Yt/vKXv8D3lm4onU4HSZLg4+OD8vJys32WuCuDJVlhnE0GWF6zieOtiIjIAnc9v2XfMjR48GD89ttvOHDggOmne/fuePjhh3HgwAGbgRDJAAeeExGRjMl+zFCjRo3QsWNHs20NGzZE06ZNq20nGePAcyIikinZB0PkBtaWyXD28hkceE5ERDKkyGAoIyPD01XwHtaWyRgzBli2jMtnEBGR15P9AGpn4ABqK4wDm+39CHDAMxERuREHUJNr1bRMhjVcPoOIiLwQgyG1srVMhjVcPoOIiLwMgyG1quvyF1w+g4iIvASDIbWq6/IXXD6DiIi8BIMhtbK1TIY1XD6DiIi8DIMhtfL1NUyTB+wPiIzl5s5lskQiIvIaDIbUzNoyGdHRwEsvGVqObsXlM4iIyAsxzxC5LwM1ERGRA9z1/FZkBmpyMmvLZMh5+QwGakRE5CQMhkh5rC0hwqVCiIioFjhmiJTFuIRI1YSReXmG7ampnqkXEREpFoMhUo6alhDhUiFERFRLDIZIOWwtIcKlQoiIqBYYDJFy2LsECJcKISIiBzAYIuWwdwkQLhVCREQO4GwyZ+OUb9cxLiGSl2d53JAkGfZzqRAiInIAW4acKTUViI0FBg0Cxo41/Dc2ljOcnKWmJUS4VAgREdUSgyFn4ZRv97C2hAiXCiEiolrichzOoNMZWoCszXQydt/k5LDVwlnYHUlE5PW4HIeSODLlW67LWyiNnJcKISIiRWE3mTNwyjcREZFiMRhyBk75JiIiUiwGQ85gnPJddYaTkSQB0dGc8k1ERCRDDIacgVO+iYiIFIvBkLNwyjcREZEicTaZM40aBYwYwSnfRERECsJgyNk45ZuIiEhR2E1GREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFRN9sFQSkoK7rzzTjRq1AjNmzfHyJEjcezYMU9Xi4iIiLyE7IOhLVu2YPLkyfjpp5+wYcMG3LhxA/fccw9KS0s9XTUiIiLyApIQQni6Eo64cOECmjdvji1btuCuu+6y65ji4mIEBwejqKgIQUFBLq4hEREROYO7nt+KW7W+qKgIABASEmK1THl5OcrLy02/FxcXu7xeREREpEyy7ya7lV6vx9SpU9G3b1907NjRarmUlBQEBwebfqKjo91YSyIiIlISRXWTPfPMM/jxxx+xbds2REVFWS1nqWUoOjqa3WREREQKwm6yKqZMmYLvv/8eW7durTEQAoCAgAAEBAS4qWZERESkZLIPhoQQeO6557BmzRpkZGQgLi7O01UiIiIiLyL7YGjy5MlYunQp1q1bh0aNGiE/Px8AEBwcjPr163u4dkRERKR0sh8zJEmSxe2LFi3CY489Ztc5OLWeiIhIeThm6A8yj9WIiIhI4RQ1tZ6IiIjI2RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpmmKCoXnz5iE2NhaBgYHo2bMndu/e7ekqERERkRdQRDD07bff4oUXXsDMmTOxb98+dO7cGcnJySgoKPB01YiIiEjhFBEMvffee5g0aRImTJiAhIQEzJ8/Hw0aNMAXX3zh6aoRERGRwsk+GKqoqMDevXuRlJRk2ubj44OkpCTs3LnTgzUjIiIib+Dn6QrYcvHiReh0OoSFhZltDwsLw9GjRy0eU15ejvLyctPvRUVFAIDi4mLXVZSIiIicyvjcFkK49DqyD4ZqIyUlBbNmzaq2PTo62gO1ISIiorooLCxEcHCwy84v+2AoNDQUvr6+OH/+vNn28+fPIzw83OIx06dPxwsvvGD6/cqVK4iJicHvv//u0hdTboqLixEdHY3c3FwEBQV5ujpuw/vmfasB75v3rQZFRUVo2bIlQkJCXHod2QdD/v7+6NatGzZt2oSRI0cCAPR6PTZt2oQpU6ZYPCYgIAABAQHVtgcHB6vqQ2QUFBTE+1YR3re68L7VRa337ePj2iHOsg+GAOCFF17A+PHj0b17d/To0QNz585FaWkpJkyY4OmqERERkcIpIhh68MEHceHCBcyYMQP5+fno0qULtFpttUHVRERERI5SRDAEAFOmTLHaLWZLQEAAZs6cabHrzJvxvnnfasD75n2rAe/btfctCVfPVyMiIiKSMdknXSQiIiJyJQZDREREpGoMhoiIiEjVGAwRERGRqikyGJo3bx5iY2MRGBiInj17Yvfu3TWWX7lyJdq3b4/AwEDcfvvt+OGHH8z2CyEwY8YMREREoH79+khKSkJWVpYrb6FWHLnvTz/9FP3790eTJk3QpEkTJCUlVSv/2GOPQZIksx+NRuPq23CYI/e9ePHiavcUGBhoVsYb3++BAwdWu29JkjBs2DBTGSW831u3bsXw4cMRGRkJSZKwdu1am8dkZGSga9euCAgIQJs2bbB48eJqZRz9N8PdHL3v1NRUDBkyBM2aNUNQUBB69+6NtLQ0szKvv/56tfe7ffv2LrwLxzl63xkZGRY/5/n5+WblvO39tvTdlSQJiYmJpjJKeL9TUlJw5513olGjRmjevDlGjhyJY8eO2TzOHc9wxQVD3377LV544QXMnDkT+/btQ+fOnZGcnIyCggKL5Xfs2IExY8bgiSeewP79+zFy5EiMHDkSBw8eNJV5++238Z///Afz58/Hrl270LBhQyQnJ+P69evuui2bHL3vjIwMjBkzBunp6di5cyeio6Nxzz33IC8vz6ycRqPBuXPnTD/Lli1zx+3YzdH7BgwZWm+9p9OnT5vt98b3OzU11eyeDx48CF9fX9x///1m5eT+fpeWlqJz586YN2+eXeVzcnIwbNgwDBo0CAcOHMDUqVMxceJEs8CgNp8hd3P0vrdu3YohQ4bghx9+wN69ezFo0CAMHz4c+/fvNyuXmJho9n5v27bNFdWvNUfv2+jYsWNm99W8eXPTPm98vz/44AOz+83NzUVISEi177fc3+8tW7Zg8uTJ+Omnn7BhwwbcuHED99xzD0pLS60e47ZnuFCYHj16iMmTJ5t+1+l0IjIyUqSkpFgs/8ADD4hhw4aZbevZs6d46qmnhBBC6PV6ER4eLt555x3T/itXroiAgACxbNkyF9xB7Th631VVVlaKRo0aiSVLlpi2jR8/XowYMcLZVXUqR+970aJFIjg42Or51PJ+v//++6JRo0bi6tWrpm1KeL9vBUCsWbOmxjJ///vfRWJiotm2Bx98UCQnJ5t+r+tr6W723LclCQkJYtasWabfZ86cKTp37uy8irmYPfednp4uAIjLly9bLaOG93vNmjVCkiRx6tQp0zalvd9CCFFQUCAAiC1btlgt465nuKJahioqKrB3714kJSWZtvn4+CApKQk7d+60eMzOnTvNygNAcnKyqXxOTg7y8/PNygQHB6Nnz55Wz+lutbnvqsrKynDjxo1qi91lZGSgefPmiI+PxzPPPIPCwkKn1r0uanvfV69eRUxMDKKjozFixAgcOnTItE8t7/fnn3+Ohx56CA0bNjTbLuf3uzZsfb+d8VoqgV6vR0lJSbXvd1ZWFiIjI9GqVSs8/PDD+P333z1UQ+fq0qULIiIiMGTIEGzfvt20XS3v9+eff46kpCTExMSYbVfa+11UVAQANS7C6q5nuKKCoYsXL0Kn01VbhiMsLKxan7FRfn5+jeWN/3XknO5Wm/uu6uWXX0ZkZKTZB0aj0eDLL7/Epk2bMGfOHGzZsgVDhw6FTqdzav1rqzb3HR8fjy+++ALr1q3D119/Db1ejz59+uDMmTMA1PF+7969GwcPHsTEiRPNtsv9/a4Na9/v4uJiXLt2zSnfHSV49913cfXqVTzwwAOmbT179sTixYuh1WrxySefICcnB/3790dJSYkHa1o3ERERmD9/PlavXo3Vq1cjOjoaAwcOxL59+wA4599KuTt79ix+/PHHat9vpb3fer0eU6dORd++fdGxY0er5dz1DFfMchxUe2+99RaWL1+OjIwMs8HEDz30kOn/b7/9dnTq1AmtW7dGRkYGBg8e7Imq1lnv3r3Ru3dv0+99+vRBhw4dsGDBAvzrX//yYM3c5/PPP8ftt9+OHj16mG33xvebgKVLl2LWrFlYt26d2diZoUOHmv6/U6dO6NmzJ2JiYrBixQo88cQTnqhqncXHxyM+Pt70e58+fXDixAm8//77+OqrrzxYM/dZsmQJGjdujJEjR5ptV9r7PXnyZBw8eFA245oU1TIUGhoKX19fnD9/3mz7+fPnER4ebvGY8PDwGssb/+vIOd2tNvdt9O677+Ktt97C+vXr0alTpxrLtmrVCqGhocjOzq5znZ2hLvdtVK9ePdxxxx2me/L297u0tBTLly+36x8/ub3ftWHt+x0UFIT69es75TMkZ8uXL8fEiROxYsWKal0JVTVu3Bjt2rVT9PttSY8ePUz35O3vtxACX3zxBcaNGwd/f/8ay8r5/Z4yZQq+//57pKenIyoqqsay7nqGKyoY8vf3R7du3bBp0ybTNr1ej02bNpm1Btyqd+/eZuUBYMOGDabycXFxCA8PNytTXFyMXbt2WT2nu9XmvgHDCPt//etf0Gq16N69u83rnDlzBoWFhYiIiHBKveuqtvd9K51Oh99++810T978fgOGKajl5eV45JFHbF5Hbu93bdj6fjvjMyRXy5Ytw4QJE7Bs2TKzFArWXL16FSdOnFD0+23JgQMHTPfkze83YJiNlZ2dbdcfO3J8v4UQmDJlCtasWYPNmzcjLi7O5jFue4Y7NPRbBpYvXy4CAgLE4sWLxeHDh8WTTz4pGjduLPLz84UQQowbN0688sorpvLbt28Xfn5+4t133xVHjhwRM2fOFPXq1RO//fabqcxbb70lGjduLNatWyd+/fVXMWLECBEXFyeuXbvm9vuzxtH7fuutt4S/v79YtWqVOHfunOmnpKRECCFESUmJePHFF8XOnTtFTk6O2Lhxo+jatato27atuH79ukfu0RJH73vWrFkiLS1NnDhxQuzdu1c89NBDIjAwUBw6dMhUxhvfb6N+/fqJBx98sNp2pbzfJSUlYv/+/WL//v0CgHjvvffE/v37xenTp4UQQrzyyiti3LhxpvInT54UDRo0EC+99JI4cuSImDdvnvD19RVardZUxtZrKQeO3vc333wj/Pz8xLx588y+31euXDGVmTZtmsjIyBA5OTli+/btIikpSYSGhoqCggK33581jt73+++/L9auXSuysrLEb7/9Jp5//nnh4+MjNm7caCrjje+30SOPPCJ69uxp8ZxKeL+feeYZERwcLDIyMsw+t2VlZaYynnqGKy4YEkKIDz/8ULRs2VL4+/uLHj16iJ9++sm0b8CAAWL8+PFm5VesWCHatWsn/P39RWJiovjf//5ntl+v14vXXntNhIWFiYCAADF48GBx7Ngxd9yKQxy575iYGAGg2s/MmTOFEEKUlZWJe+65RzRr1kzUq1dPxMTEiEmTJsnqHwwjR+576tSpprJhYWHiT3/6k9i3b5/Z+bzx/RZCiKNHjwoAYv369dXOpZT32zh1uuqP8V7Hjx8vBgwYUO2YLl26CH9/f9GqVSuxaNGiauet6bWUA0fve8CAATWWF8KQYiAiIkL4+/uLFi1aiAcffFBkZ2e798ZscPS+58yZI1q3bi0CAwNFSEiIGDhwoNi8eXO183rb+y2EYbp4/fr1xcKFCy2eUwnvt6V7BmD2nfXUM1z6o4JEREREqqSoMUNEREREzsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARKdKyZctQv359nDt3zrRtwoQJ6NSpE4qKijxYMyJSGq5NRkSKJIRAly5dcNddd+HDDz/EzJkz8cUXX+Cnn35CixYtPF09IlIQP09XgIioNiRJwhtvvIHRo0cjPDwcH374ITIzMxkIEZHD2DJERIrWtWtXHDp0COvXr8eAAQM8XR0iUiCOGSIixdJqtTh69Ch0Oh3CwsI8XR0iUii2DBGRIu3btw8DBw7EggULsHjxYgQFBWHlypWerhYRKRDHDBGR4pw6dQrDhg3DP/7xD4wZMwatWrVC7969sW/fPnTt2tXT1SMihWHLEBEpyqVLl9CnTx8MHDgQ8+fPN20fNmwYdDodtFqtB2tHRErEYIiIiIhUjQOoiYiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKr2/81CzlL3QzUTAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own sdg\n",
      "[[4.20773884]\n",
      " [2.81706996]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using SGD\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = (1.0/n)*training_gradient(y, X, theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "# Can you figure out a better way of setting up the contributions to each batch?\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        theta = theta - eta*gradients\n",
    "print(\"theta from own sdg\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ca466b4",
   "metadata": {},
   "source": [
    "## Same code but now with momentum gradient descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "0d44a49c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.79928328]\n",
      " [3.09772988]]\n",
      "Eigenvalues of Hessian Matrix:[0.33500183 3.88706629]\n",
      "theta from own gd\n",
      "[[3.79937084]\n",
      " [3.09764763]]\n",
      "theta from own sdg with momentum\n",
      "[[3.77947186]\n",
      " [3.08314421]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using SGD\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 100\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = (1.0/n)*training_gradient(y, X, theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "change = 0.0\n",
    "delta_momentum = 0.3\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        # calculate update\n",
    "        new_change = eta*gradients+delta_momentum*change\n",
    "        # take a step\n",
    "        theta -= new_change\n",
    "        # save the change\n",
    "        change = new_change\n",
    "print(\"theta from own sdg with momentum\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b82627f6",
   "metadata": {},
   "source": [
    "## AdaGrad algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/adagrad.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/adagrad.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00d3aff0",
   "metadata": {},
   "source": [
    "## Similar (second order function now) problem but now with AdaGrad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6b85aacc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own AdaGrad\n",
      "[[2.00000205]\n",
      " [2.99998905]\n",
      " [4.00001059]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using AdaGrad and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-8\n",
    "for epoch in range(n_epochs):\n",
    "    Giter = 0.0\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        Giter += gradients*gradients\n",
    "        update = gradients*eta/(delta+np.sqrt(Giter))\n",
    "        theta -= update\n",
    "print(\"theta from own AdaGrad\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8ddde38",
   "metadata": {},
   "source": [
    "Running this code we note an almost perfect agreement with the results from matrix inversion."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff15b503",
   "metadata": {},
   "source": [
    "## RMSProp algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/rmsprop.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/rmsprop.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66f96d12",
   "metadata": {},
   "source": [
    "## RMSprop for adaptive learning rate with Stochastic Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "888f1b4e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own RMSprop\n",
      "[[1.99957289]\n",
      " [3.00371103]\n",
      " [3.99595959]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Value for parameter rho\n",
    "rho = 0.99\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-8\n",
    "for epoch in range(n_epochs):\n",
    "    Giter = 0.0\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "\t# Accumulated gradient\n",
    "\t# Scaling with rho the new and the previous results\n",
    "        Giter = (rho*Giter+(1-rho)*gradients*gradients)\n",
    "\t# Taking the diagonal only and inverting\n",
    "        update = gradients*eta/(delta+np.sqrt(Giter))\n",
    "\t# Hadamard product\n",
    "        theta -= update\n",
    "print(\"theta from own RMSprop\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e0860f7",
   "metadata": {},
   "source": [
    "## ADAM algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/adam.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/adam.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab4a9859",
   "metadata": {},
   "source": [
    "## And finally [ADAM](https://arxiv.org/pdf/1412.6980.pdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ccdd4d77",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own ADAM\n",
      "[[2.00001509]\n",
      " [2.99992841]\n",
      " [4.00006811]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Value for parameters beta1 and beta2, see https://arxiv.org/abs/1412.6980\n",
    "beta1 = 0.9\n",
    "beta2 = 0.999\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-7\n",
    "iter = 0\n",
    "for epoch in range(n_epochs):\n",
    "    first_moment = 0.0\n",
    "    second_moment = 0.0\n",
    "    iter += 1\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        # Computing moments first\n",
    "        first_moment = beta1*first_moment + (1-beta1)*gradients\n",
    "        second_moment = beta2*second_moment+(1-beta2)*gradients*gradients\n",
    "        first_term = first_moment/(1.0-beta1**iter)\n",
    "        second_term = second_moment/(1.0-beta2**iter)\n",
    "\t# Scaling with rho the new and the previous results\n",
    "        update = eta*first_term/(np.sqrt(second_term)+delta)\n",
    "        theta -= update\n",
    "print(\"theta from own ADAM\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25ac988c",
   "metadata": {},
   "source": [
    "## Introducing [JAX](https://jax.readthedocs.io/en/latest/)\n",
    "\n",
    "Presently, instead of using **autograd**, we recommend using [JAX](https://jax.readthedocs.io/en/latest/)\n",
    "\n",
    "**JAX** is Autograd and [XLA (Accelerated Linear Algebra))](https://www.tensorflow.org/xla),\n",
    "brought together for high-performance numerical computing and machine learning research.\n",
    "It provides composable transformations of Python+NumPy programs: differentiate, vectorize, parallelize, Just-In-Time compile to GPU/TPU, and more."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37d556d0",
   "metadata": {},
   "source": [
    "### Getting started with Jax, note the way we import numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "5b81d6e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "import jax\n",
    "import jax.numpy as jnp\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from jax import grad as jax_grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c42db672",
   "metadata": {},
   "source": [
    "### A warm-up example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "98eb2f26",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/jax/_src/numpy/lax_numpy.py:173: UserWarning: Explicitly requested dtype float64 requested in asarray is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more.\n",
      "  return asarray(x, dtype=self.dtype)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x139592670>]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABPqUlEQVR4nO3deXiU1eH28e8zk52sELIS9l1W2RQUURFFUVHcbUVr1bTRVm1rtfXVbr9SrbW0Noh1X+pSUQStoiyCKKvsyI6sCQkkZDLZl5nn/WMgggISSObMTO7Pdc0FzDxJbtKUuT3nPOdYtm3biIiIiPiJw3QAERERaVlUPkRERMSvVD5ERETEr1Q+RERExK9UPkRERMSvVD5ERETEr1Q+RERExK9UPkRERMSvwkwH+Dav10t+fj5xcXFYlmU6joiIiJwE27YpKysjIyMDh+PEYxsBVz7y8/PJysoyHUNEREROwZ49e2jXrt0Jrwm48hEXFwf4wsfHxxtOIyIiIifD7XaTlZXV8D5+IgFXPg5PtcTHx6t8iIiIBJmTWTKhBaciIiLiVyofIiIi4lcqHyIiIuJXKh8iIiLiVyofIiIi4lcqHyIiIuJXKh8iIiLiVyofIiIi4lcBt8mYiIiINBOvB3YtgvJCiE2FDsPB4fR7jEaNfEyaNIkhQ4YQFxdHSkoK48ePZ/PmzUddM2rUKCzLOuqRnZ3dpKFFRETk5E1ZPYWpc+6FyX3g5XHwzu2+Xyf3Yeqce5myeopf8zSqfCxYsICcnByWLFnC7NmzqaurY8yYMVRUVBx13R133MG+ffsaHo8//niThhYREZGT5yjaQm7eXKY6yo96fqqjgty8uTiKtvg1T6OmXWbNmnXUn1966SVSUlJYsWIFI0eObHg+JiaGtLS0pkkoIiIip87rIXvtx+AoJzcpEYBsl5upifHkJiWQU1JKdvEncIHHb1Mwp7XgtLS0FIDWrVsf9fx//vMfkpOT6dOnDw899BCVlZXH/Rw1NTW43e6jHs2hoLSaf8zZyl8+2tQsn19ERCQg7VoE7nyyXW6yS1zkJiUyoGMWuUmJ5JS4yHaVgjvPd52fnPKCU6/Xy7333suIESPo06dPw/M33XQTHTp0ICMjg7Vr1/LrX/+azZs38+677x7z80yaNInf//73pxrjpO0vq+bvc7YQHe7kZxd2JSZCa21FRKQFKC9s+G2P2joAPJZFuG2T7XIf87rmdsrvwDk5Oaxfv57PP//8qOfvvPPOht/37duX9PR0LrzwQrZv306XLl2+83keeugh7r///oY/u91usrKyTjXWcfXNTKB96xh2H6xk3qb9jOuX0eRfQ0REJODEpjb89h9JCQA4bJs6y2JqYvw3BeSI65rbKU273H333XzwwQd8+umntGvX7oTXDhs2DIBt27Yd8/XIyEji4+OPejQHy7K4rF86AB+s2dcsX0NERCTgdBgO8Rk8mZTIzogIAKbn7SPn0BTM1MQEiM/0XecnjSoftm1z9913M336dObNm0enTp2+92NWr14NQHp6+ikFbErjDpWPTzfvp7ym3nAaERERP3A4mdrvYl5M9P3Hfd/qGjrX1ZPtcpNTUkpuUgJT+43x634fjSofOTk5vPbaa7z++uvExcVRUFBAQUEBVVVVAGzfvp0//vGPrFixgp07dzJz5kxuueUWRo4cSb9+/ZrlL9AYvdPj6Zzcipp6L3M2+G9uS0RExCRvcneS6r0AXFH+zfYY2d5YcjIvxJvc3a95GrXm4+mnnwZ8G4kd6cUXX+TWW28lIiKCOXPmMHnyZCoqKsjKymLChAk8/PDDTRb4dFiWxbh+6fxz3jY+WLuP8QMzTUcSERFpdiMiMnk6zEG4bXPG4L9A6+iGHU6zDexw2qjyYdv2CV/PyspiwYIFpxWouV3WL4N/ztvGZ1sOUFpVR0J0uOlIIiIizeq/y6cCcEZVDH1G/gAsy2ieFnewXI+0OLqlxFLr8TJbUy8iIhLi6uoq+cyzG4De8RdhGS4e0ALLB9Bwm+0Ha/MNJxEREWle85b8C5fTQVK9l3Hn/tx0HKCFlo/Dt9x+vrWIkopaw2lERESaz/St0wHoUZVKn6y2htP4tMjy0TUllp5pcdR7bT7ZUGA6joiISLNwlexgqaMMgB7pNwXElAu00PIBcHn/w1Mv2nBMRERC04xFf6XesuhYA1ecc73pOA1abPk4vOHYou3FFJfXGE4jIiLS9N4v+AKAzOqedE+NM5zmGy22fHRo04q+mQl4vDYfrdfUi4iIhJZt2z9hc5iXMNumV5ds03GO0mLLB3wz+vH+Gt31IiIioeWdFVMA6FkRybihwwynOVrLLh+H1n0s23mQgtJqw2lERESaRn1dNbPKfQe6JnjPpkvbWMOJjtaiy0dmYjSDOyRh29rzQ0REQsfiVf+myGmR4PEy4Iy7TMf5jhZdPgCuGOAb/dDUi4iIhIrpm94GoLO7DZef2dVwmu9q8eVjbJ90HBas2VvKruKK7/8AERGRAOYu3cMCbwkAUWGXk5kYbTjRd7X48tE2LpIRXZMBjX6IiEjwm7X0b9RaFu1rbEYMnGA6zjG1+PIB32w49v4abTgmIiLB7b29vtPlW7s7c1n/TMNpjk3lA7j4jDQinA42F5axuaDMdBwREZFTsmPnfNY563HaNtHxPyA5NtJ0pGNS+QASosM5r4fvsB1NvYiISLB6f2UuAD0qIxh95tmG0xyfyschh6deZq7Jx7Ztw2lEREQax1Nfy0zXRgAs90AuPiPNcKLjU/k4ZHSvFKLDnew+WMmavaWm44iIiDTKsjUvUOi0iPN4SUi7hYTocNORjkvl45CYiDAu6p0KaOpFRESCz8xNbwHQpSyJcQO7GU5zYiofRzg89fLB2nw8Xk29iIhIcCgv28ec2gMAuMou4MJeKYYTnZjKxxFGdk8mPiqMQncNS3cUm44jIiJyUmYvfZJqh0VmrU3HTlcSExFmOtIJqXwcITLMyaV9fSfdzlytqRcREQkO7+2dB0Db0g6MP7Od4TTfT+XjW64c4NuQ5cN1+6ip9xhOIyIicmJ79nzBSqsWh22TX30lI7u3NR3pe6l8fMuwTq1Ji4/CXV3P/M0HTMcRERE5oZlf/guAHlXhnHXG2YQ7A/+tPfAT+pnDYTWcdDtjdZ7hNCIiIsfn9dQzs2Q9AJarH+MHBuZ26t+m8nEMVxy662XOxv2UVdcZTiMiInJsK9a9Qr4TWnm9FDKBQe2TTEc6KSofx3BGRjxd2raitt7LrPUFpuOIiIgc04wNrwPQrSyBsQO743BYhhOdHJWPY7Asi/GHFp7O1IZjIiISgCrL9/NJje8/kItd53LloSUDwUDl4zgOr/v4YlsR+8uqDacRERE52pxlk6lyWKTXevHEjaVnWrzpSCdN5eM4OrRpxcD2iXht+GDNPtNxREREjjJzz2wAUt3tuXJg4O/tcSSVjxO48tDC0xmaehERkQCSn/8lS/GNym8rGdtwo0SwUPk4gcv6ZeB0WKzZ42JHUYXpOCIiIgC8v/wfAPSodNKu3WDaJcUYTtQ4Kh8n0DYukhFdkwHt+SEiIoHB9nqZWbwagLDSMxp25g4mKh/fY/yhhafvrcrDtnXSrYiImLV6/evsdkK018tXZeMaziQLJiof3+PiM9KIDneys7iS1XtcpuOIiEgLN+OrVwDoXh7PsO7daN0qwnCixlP5+B6tIsMYc0Yq4Bv9EBERMaW6qoSPq303QZS6hnNVkGyn/m0qHyfh8P+476/dR53HaziNiIi0VPOWTabcYZFa52Vn/QVc2CvFdKRTovJxEs7pmkxybCQHK2r5bItOuhURETNm7pwFQIY7k0v7ZhEV7jSc6NSofJyEMKej4R7q6Zp6ERERAwoL17LY9m37sN11SdCcYHssKh8n6fDUy+wNhbh10q2IiPjZB8v+jtey6FblwI7uz7BOrU1HOmUqHyepT2Y8XVNiqdFJtyIi4me218uMAysAiCztxZUDM4PmBNtjUfk4SZZlNYx+TF+pqRcREfGf9RvfYYfTJtJrs650XNDe5XKYykcjHF73sWRHMfmuKsNpRESkpZix7gUAelS0okNaB7qnxhlOdHpUPhohq3UMQzu1xrZhpg6bExERP6itKeOjqj0AlLnOCvpRD1D5aLQjp1603bqIiDS3+cv+gdthkVzvZUPFBUF3gu2xqHw00qV904kIc7C5sIwN+9ym44iISIibseN/AGS50zm7axop8VGGE50+lY9GSogOZ/ShHeXe1cJTERFpRkUHNvKFtwyAna4xXH1m8E+5gMrHKZlwZjsAZqzO03brIiLSbP637Ek8lkWXaotiux8Xn5FmOlKTUPk4BSO7t6VNqwiKymtZuFXbrYuISNOzvV7e278MgJjSnoztm05MRJjhVE1D5eMUhDsdXDnAN/T1zgpNvYiISNPbuGUm2xxeIrw261zjGkbdQ4HKxyk6PO82e2MhpZXabl1ERJrWzLXPA9CjIpr4+HZBvZ36t6l8nKIzMuLpmRZHbb2X/63bZzqOiIiEkLqaCj6s2AFAVelQrj4zuLdT/zaVj1NkWVbD6Mc7K/caTiMiIqHksxX/osRh0brey9qy0SGxsdiRVD5Ow/gBmTgsWLGrhJ1FFabjiIhIiJi5fSYAHdyp9G+fQue2sYYTNS2Vj9OQEh/Fud3aAvCuRj9ERKQJHDy4jc88pQDsdo3m6hBaaHqYysdpmjDI90Px7qo8vF5tty4iIqfno6V/o96y6FQN+fUDubxf8G+n/m0qH6dpTO9U4iLD2FtSxbKdB03HERGRIDejYDEAce5ujO6dQkJMuOFETa9R5WPSpEkMGTKEuLg4UlJSGD9+PJs3bz7qmurqanJycmjTpg2xsbFMmDCBwsLCJg0dSKLCnVzWLx2Ad1Zo6kVERE7dlm0fsdHhIcy2WR9ie3scqVHlY8GCBeTk5LBkyRJmz55NXV0dY8aMoaLim8WW9913H++//z5vv/02CxYsID8/n6uvvrrJgweSw1MvH67bR2VtveE0IiISrGauegaAnhVRhEdnMbJ7W8OJmkej9mmdNWvWUX9+6aWXSElJYcWKFYwcOZLS0lKef/55Xn/9dS644AIAXnzxRXr16sWSJUs466yzmi55ABncIYmObWLYWVzJR+sKGsqIiIjIyaqvq+aDsm3gtKh1DWb8gEzCnaG5OuK0/lalpb7VuK1b+3ZdW7FiBXV1dYwePbrhmp49e9K+fXsWL158zM9RU1OD2+0+6hFsLMvimkOF4+0VewynERGRYLRo5dMUOy0SPV7WlI3hmsGh+x+yp1w+vF4v9957LyNGjKBPnz4AFBQUEBERQWJi4lHXpqamUlBQcMzPM2nSJBISEhoeWVlZpxrJqKvPbIdlwZKvD7K7uNJ0HBERCTLvbXkXgE7uZHplptAzLd5wouZzyuUjJyeH9evX8+abb55WgIceeojS0tKGx549wTlykJEYzTldkwHteCoiIo1T6trJfE8JAHmuC7k2hEc94BTLx913380HH3zAp59+Srt233yD0tLSqK2txeVyHXV9YWEhaWlpx/xckZGRxMfHH/UIVoenXqat2Ks9P0RE5KTNWvokdZZFhxrYUzeEK/qH3t4eR2pU+bBtm7vvvpvp06czb948OnXqdNTrgwYNIjw8nLlz5zY8t3nzZnbv3s3ZZ5/dNIkD2MVnpBEXGUaeq4olXxebjiMiIkFiRv5CABJLu3BR71QSYyIMJ2pejSofOTk5vPbaa7z++uvExcVRUFBAQUEBVVVVACQkJHD77bdz//338+mnn7JixQpuu+02zj777JC90+VIUeFOxh1qq9O054eIiJyEr3fMZZ2jnjDbZoPrspBeaHpYo8rH008/TWlpKaNGjSI9Pb3h8dZbbzVc8/e//51x48YxYcIERo4cSVpaGu+++26TBw9Uh+fpPly/j7LqOsNpREQk0M1YOQWAHpWRhLfqxMhuobm3x5Eatc+HbX//OoaoqChyc3PJzc095VDBbGBWIl3atmL7gQr+t3YfNwxtbzqSiIgEKE99LR+UbganRb1rIFcNbIfTYZmO1exCc/cSg3x7fvhuF9bUi4iInMiSVf9mv9Mi3uNlbdklIX+Xy2EqH83g6jMzcVjw5a4Svj5QbjqOiIgEqBmb3wagc1lr+mSl06VtrOFE/qHy0QxS46M479B+/P/9UqMfIiLyXWXuPObV+e6MLHSd3zBq3hKofDST6wb7fojeWbmXeo/XcBoREQk0Hy/9GzUOi6xamz31Z3F5/3TTkfxG5aOZXNgrlTatIjhQVsP8zQdMxxERkQAzM28BAG1KO3Jp30ziosINJ/IflY9mEhHm4KqBmQC89WVwbhkvIiLNY9euhayyanHYNhtd47h+SMuZcgGVj2Z1+Idp3qb97C+rNpxGREQCxcyV/wKgZ2U4cYk9GdIxyXAi/1L5aEbdUuMY2D4Rj9fm3ZV5puOIiEgA8Hrqeb/kKwDs0gFcO7gdlhX6e3scSeWjmV1/aOHpf5fvOalN2kREJLQtX/MC+5wWsR4v68ov5ZozW8beHkdS+Whm4/pnEBPh5OuiCr7cVWI6joiIGDZz05sAdC1PZET3jqTERxlO5H8qH80sNjKMy/r6bp/673ItPBURackqygv4pGY/AEUlIxu2ZWhpVD784LpDC0//t24f5TX1htOIiIgps5f+nWqHRUatzX7H+ZzfM8V0JCNUPvxgcIckOrdtRWWthw/W5JuOIyIihszYMxeAFHd7rhmcRbizZb4Nt8y/tZ9ZltUwtPampl5ERFqkvXuX8KVVg2XbbCm5lGtb0Hbq36by4ScTzmxHmMNi9R4XmwrcpuOIiIifvf/lUwD0qAqjfbsz6ZrSMg6ROxaVDz9pGxfJ6F6pALy5TKMfIiItie31MvPgWgAcrr7cMKS94URmqXz40Q1DfUNs767cS3Wdx3AaERHxl5XrXmWvE2K8XrbXXsmlfVvOIXLHovLhR+d2a0tmYjTu6no+Wr/PdBwREfGTGRteBaBbWQKXDehJdITTcCKzVD78yOmwGs57eUNTLyIiLUJlZRGfVBcAUOI6hxuGtuwpF1D58LtrB7fDYcGyHQfZfqDcdBwREWlmc5dOpsJhkVbnxdn6cnqlx5uOZJzKh5+lJ0Rzfg/fpjJvLtttOI2IiDS3mbs/ASC9NIubhnYwnCYwqHwYcOOhIbd3VuZRU6+FpyIioapg3yqW2pUA7Km4jMv7ZxhOFBhUPgwY1aMtqfGRHKyoZfaGQtNxRESkmby//O/YlkX3Sidn9RlFq8gw05ECgsqHAWFOxzc7nmrhqYhISLK9XmYUrQIgwt2bG4e23B1Nv03lw5DrBmdhWfD5tiJ2F1eajiMiIk1szYY32eWEKK9NdeT19M1MMB0pYKh8GJLVOoZzu7UF4I3lWngqIhJqZq5/BYAe5bFcPaw/lmUZThQ4VD4MuunQwtO3v9xDbb3XcBoREWkq1VUlfFTlm1avKh/B+IGZhhMFFpUPgy7slUJqfCRF5bV8sqHAdBwREWki85f/k3KHg5Q6L9273UxcVLjpSAFF5cOgcKeD6w8tPP3PEk29iIiEivd2fAhApjuDm87qYjhN4FH5MOz6oe1xWLD462LteCoiEgL2F65nsV3h+0P41fRtp4Wm36byYVhmYjQX9PTtePrGUo1+iIgEuw+W/R2vZdG1yuKysy4zHScgqXwEgJuG+RaeTlu5l+o67XgqIhKsbK+X9/YvByC+vLd2ND0OlY8AcF73FDITo3FV1vHR+n2m44iIyCnasHk6O8JsIr02HTvdSUyEdjQ9FpWPAOB0WA0732nhqYhI8Jq26jkAelTEcOOIwYbTBC6VjwBx3eAswhwWX+4qYXNBmek4IiLSSLU1Zcyu9v0HZCLn0zMt3nCiwKXyESBS4qO4qHcqAK8v3WU4jYiINNb85U9R6nSQXO/l/LPuMh0noKl8BJCbh3UA4N2VeVTU1BtOIyIijTFtywwAOpanMa5/B8NpApvKRwAZ3qUNnZJbUVZTz4zV+abjiIjISSoq2sQyh29vj27pPyAq3Gk4UWBT+QggDofFzYduu31l8U5s2zacSERETsY7n/8Vj2XRudripguuMx0n4Kl8BJhrB2URFe5gU0EZK3aVmI4jIiIn4aMi394enb396JjcynCawKfyEWASYsK5sr/v9MNXl2jhqYhIoFu7YQbbw23CbZvRg+81HScoqHwEoB+e7Vuo9OG6fRSV1xhOIyIix+T1wI6FTPviMQDOqIzhkkGDDIcKDiofAahPZgIDshKp89i8tXyP6TgiInKEKaunMHXOvTC5D3Uvj2OB0wXALZXFPDvvPqasnmI0XzBQ+QhQtxwa/Xh96W48Xi08FREJFI6iLeTmzWWqo5zPY6I56HTSpt7D1rA6cvPm4ijaYjpiwFP5CFCX9k0nKSacPFcV8zbtNx1HREQAvB6y135MTomL3KREJiclAJBVX8fTSQnklJSSvfYT35SMHJfKR4CKCndy3RDfeS9aeCoiEiB2LQJ3PtkuN7e7Svk6IgKA1VFR5JS4yHaVgjvPd50cl8pHAPvBsA5YFny25QA7iypMxxERkfLCht/Geb0Nvw+3bbJd7mNeJ9+l8hHAslrHcH6PFECjHyIiASHWdwZXjQX/TvBNuThtmzrLYmpi/Heuk2NT+Qhwh2+7/e+Xe3Tei4iIaR2GQ3wGv2ybTKXTQazHw/KdexrWgExNTID4TN91clwqHwHuvG5tfee9VNczfVWe6TgiIi2bw8nTfS9ifqsYAH7qchMOZLvc5JSUkpuUwNR+Y8Chs11OROUjwDkcFj88yzf6ofNeRETMW+/eB0C8x8OEsvKG57O9seRkXog3ubupaEEjzHQA+X7XDG7HE59sZkthOYu3FzO8a7LpSCIiLZLt9bKvZAuEw6CaTsTc8oxvcWlsKnQYTrZGPE6KRj6CQHxUOBPObAfAS4t2mg0jItKCffblM2wN9xLptRk37HfQ6Vzoe43vVxWPk6byESQmDvdNvczZWMjekkrDaUREWqYX1j0PwIDKZEYPGGw4TfBS+QgSXVPiOKdrMl5bt92KiJjw1abprAyrwWnbnNX1PhwOy3SkoKXyEUQmDu8IwFvL91Bdp617RUT86ZklfwOgb3krbhh1meE0wa3R5eOzzz7j8ssvJyMjA8uyeO+99456/dZbb8WyrKMel1xySVPlbdEu6JlCu6RoXJV1zFit225FRPxlz54vWIALgO6tf0RspO7XOB2NLh8VFRX079+f3Nzc415zySWXsG/fvobHG2+8cVohxcfpsBpOu31p0S7ddisi4ifPfvYHvJZF74owbr7oZtNxgl6jq9vYsWMZO3bsCa+JjIwkLS3tlEPJ8V0/uD1Pzt7Cxn1ulu44yFmd25iOJCIS0oqKNvG/ujywLNIcV9C5bazpSEGvWdZ8zJ8/n5SUFHr06MFPfvITiouLj3ttTU0Nbrf7qIccX0LMN7fdvvjFDsNpRERC36vzH6bWsuhSbTH+vJ+YjhMSmrx8XHLJJbzyyivMnTuXxx57jAULFjB27Fg8nmMvkJw0aRIJCQkNj6ysrKaOFHJuG9ERgE82FLK7WLfdiog0l/Kyffy3bCMAyZXncl4PHRjXFJq8fNxwww1cccUV9O3bl/Hjx/PBBx+wfPly5s+ff8zrH3roIUpLSxsee/bsaepIIadrShzndW+LbcPLi3eajiMiErLeXvD/KHc4yKy1OW/Y/bq9tok0+622nTt3Jjk5mW3bth3z9cjISOLj4496yPf70TmdAN9tt2XVdYbTiIiEntqaMl7dvwSA1NIBTBjcwXCi0NHs5WPv3r0UFxeTnp7e3F+qRRnZLZmuKbGU19QzbcVe03FERELOBwt/zwGnRXK9l549f0FMhG6vbSqNLh/l5eWsXr2a1atXA7Bjxw5Wr17N7t27KS8v51e/+hVLlixh586dzJ07lyuvvJKuXbty8cUXN3X2Fs2yLG49tOnYS4t24vHqtlsRkabi9dTzwu6PAWhX0oWJ5/Q0nCi0NLp8fPnllwwcOJCBAwcCcP/99zNw4EAeeeQRnE4na9eu5YorrqB79+7cfvvtDBo0iIULFxIZGdnk4Vu6q8/MJCE6nF3FlczbtN90HBGRkPHpkifY5YRYj5ek1J+RkRhtOlJIafQY0qhRo064udXHH398WoHk5MVEhHHj0PZMXbCdFz7fwUW9tQpbROR02V4vz215CxzQrTSDiVcPMh0p5OhslyB3y9kdcDosFn9dzMZ92iNFROR0fbnmRdY76on02tSF/5gz2yeajhRyVD6CXEZiNGP7+HaTff5zbTomInK6nl/7bwB6uRO54ZxzsCzdXtvUVD5CwO2HbruduTqf/WXVhtOIiASvzVs+4Asqcdg2rqobubSv7tRsDiofIWBg+yQGdUii1uPllUW7TMcREQlaLyx/AoA+5TFcdvYlhDv1Ntkc9F0NEXec6xv9eG3pLipr6w2nEREJPnv3LmFWXREALtc4bhza3nCi0KXyESIu6p1GhzYxuCrreEebjomINNrLX/wBr2XRqzKMof0nkBAdbjpSyFL5CBFOh8WPRvhGP57/fIc2HRMRaYTioi1Mr9wNQHXR+Q3/nkrzUPkIIdcObkdCdDg7iyuZs7HQdBwRkaDx+sL/R43DonM1tOt4E1mtY0xHCmkqHyEkJiKMm4f55iifW/i14TQiIsGhsnw/b7q+AiC8eBh3nNfFcKLQp/IRYiYO70i402L5zhJW73GZjiMiEvCmffYwbodFRq2NlfhDBrZPMh0p5Kl8hJjU+Ciu6J8JwLMa/RAROaG6mgpeKVgEQJuDfbh9ZA/DiVoGlY8Q9ONDt91+tG4few5WGk4jIhK4/vf5Hyh0WrSp91Jo3aIzsvxE5SME9UqP59xuyXhtbbkuInI8Xk89L+76CICskk5MPLc3Toe2UvcHlY8QlX1owdSby3dzsKLWcBoRkcCzYNlkvnbatPJ62Vn1A64dlGU6Uouh8hGihndpQ5/MeKrrvLy6WFuui4gcyfZ6eX7T6wB0d6VxzdmDiI5wGk7Vcqh8hCjLsrhrpG/04+XFO6mq9RhOJCISOFaue5U1jjoivDbbS2/klrM7mI7Uoqh8hLCxfdLIah3NwYpapq3YYzqOiEjAeGHNVAB6lyVw0Zlnk9QqwnCilkXlI4SFOR3ccW5nAJ5duIN6j9dwIhER87Zs+4jP7HIcts3u4gncfo62Uvc3lY8Qd+2gLJJiwtl9sJJZXxWYjiMiYtyLSx8H4IzyaPr3PF9bqRug8hHioiOcTBzeEYBnFnyNbevAORFpufLzv+SjugMAHCgay50jOxtO1DKpfLQAt5zdkahwB+vySlm8vdh0HBERY175/Pd4LIselU7Ssi6jT2aC6UgtkspHC9C6VQTXD/bdv/70gu2G04iImFFycDvvVPg2XqwtPq/hjkDxP5WPFuLH53bG6bBYuLWItXtdpuOIiPjdGwseptph0bEGPPFXM6JrG9ORWiyVjxYiq3UMV/bPAGDKpxr9EJGWpbKyiNdd6wCILh7C3ed3x7K0lbopKh8tSPYo3xDjxxsK2La/zHAaERH/mb7g/1HqsEivsykOv5GLz0gzHalFU/loQbqnxjGmdyq2DU/P/9p0HBERv6irq+TlfQsBSD7Ym+xRPXHoADmjVD5amJ+e3xWAGavz2FtSaTiNiEjzm/X5/7HPaZFU7yWv/gdcOSDTdKQWT+WjhRmQlcjwLm2o99o8+5lGP0QktNleLy/seB+ADiUdufW8vkSE6a3PNP0v0ALlHBr9eHP5HorKawynERFpPguX/4NtTpsYr5cdVT/ghiHtTUcSVD5apOFd2tC/XQI19V5e+HyH6TgiIs3muY2vAdDDlcoNIwYTHeE0nEhA5aNFsiyrYe3Hq4t34a6uM5xIRKTprV73H1ZZtYTbNjvLbuCHZ3c0HUkOUflooS7qlUq3lFjKaup5ZdFO03FERJrc86tyAejtjufSISNJiA43nEgOU/looRwOq2Htx/Of76Cipt5wIhGRprN9+2zm22VYtk2B6yp+fG4n05HkCCofLdi4ful0bBNDSWUdry3ZZTqOiEiTeWHpXwA4oyKKkQMvIzk20nAiOZLKRwsW5nQ0rP14duHXVNV6DCcSETl9BftW8WFtIQDug2O567zOhhPJt6l8tHBXDcykXVI0ReW1vLFst+k4IiKn7ZXPf0e9ZdG90smgfteTGh9lOpJ8i8pHCxfudPCTQ2e+PPPZdqrrNPohIsGr1LWTaeW+wzPtknPJPq+L4URyLCofwjWD2pGeEEWhu4a3V+w1HUdE5JS9seBhqhwWHWuge/fbyUiMNh1JjkHlQ4gMc3LXSN+c6NT526mt9xpOJCLSeFWVB/lP8WoAWh08k5+e391sIDkulQ8B4Iah7UmOjSTPVcX0VRr9EJHg895nj+ByWqTWecnsmEP7NjGmI8lxqHwIAFHh34x+5H66nTqPRj9EJHjU11XzUv58ANIO9iLnwt5mA8kJqXxIg5vPak9ybAS7D1YyfWWe6TgiIift4y/+TL7TItHjJSXzZ3RuG2s6kpyAyoc0iIkI466RvpXhT326VaMfIhIUbK+X57+eAUCnkvbkjDnTcCL5PiofcpQfnNWB5NhI9hys4h3d+SIiQeCLL3PZ6vQS4/WSknYvXTTqEfBUPuQo0RFOsg/tBvjUvG2680VEAt6/178EQI/Stvx0zAizYeSkqHzId9w8rEPDnS/TNPohIgFs7VdvscpZS5htk578U631CBIqH/Id0RHOhl1Pcz/V6IeIBK5nlv8DgD5lcfzkkssMp5GTpfIhx3TzsPakxPlGP95escd0HBGR7/h6xzwW4gagfeKP6JjcynAiOVkqH3JMUeFHjH7M20ZNvc58EZHAMnXhn7Atiz4VEdx56U2m40gjqHzIcd04tD2p8ZHkl1bz1nKNfohI4CgsXMts734AurS6ng5tNOoRTFQ+5Liiwp3knN8VgH/N20ZVrUY/RCQwTJ3zW+oti+5VDu4cd7fpONJIKh9yQjcMaU9mYjT7y2p4dclO03FERCh17ebD2q8B6BN9mc5wCUIqH3JCEWEOfj66GwBPz99OWXWd4UQi0mJ5PbBjIS+8exuVDgfta2yyr3jYdCo5BSof8r2uHphJ5+RWlFTW8cLnO03HEZEWZsrqKUydcy9M7kP1K5fzHvsAuL2skhkrfsOU1VPMBpRGU/mQ7xXmdHDvRd0BeG7h17gqaw0nEpGWxFG0hdy8uUx1lDMzthUHnU4y6urZ56wnN28ujqItpiNKI6l8yEkZ1zednmlxlNXU88xnX5uOIyIthddD9tqPySlxkZuUyD+SEgDoUlfL1KQEckpKyV77iW9KRoJGo8vHZ599xuWXX05GRgaWZfHee+8d9bpt2zzyyCOkp6cTHR3N6NGj2bp1a1PlFUMcDotfjOkBwEtf7GR/WbXhRCLSIuxaBO58sl1uRpdX4HY6wbZZGBNDTomLbFcpuPN810nQaHT5qKiooH///uTm5h7z9ccff5x//vOfTJ06laVLl9KqVSsuvvhiqqv1ZhXsRvdKoX9WIlV1HqZ8ut10HBFpCcoLAai0LNZGRfqesyzCbZtsl/s710lwaHT5GDt2LH/605+46qqrvvOabdtMnjyZhx9+mCuvvJJ+/frxyiuvkJ+f/50REgk+lmXxq0OjH68v3c3ekkrDiUQk5MWmAvBcYjz7w8IACLdt6iyLqYnx37lOgkOTrvnYsWMHBQUFjB49uuG5hIQEhg0bxuLFi4/5MTU1Nbjd7qMeErhGdG3D8C5tqPV4eXK2FnmJSDPrMJw9cam8EO8rGmPLK1i5c0/DGpCpiQkQnwkdhhsOKo3RpOWjoKAAgNTUoxtoampqw2vfNmnSJBISEhoeWVlZTRlJmphlWfz6kp4ATF+Vx6YClUURaUYOJ9mJ0XgcFu3q6njsQDEA2S43OSWl5CYlMLXfGHA4DQeVxjB+t8tDDz1EaWlpw2PPHp0hEuj6ZyVyad80bBv+Omuz6TgiEsLmLc5ld5gXy7b5W0kd1hGvZXtjycm8EG9yd2P55NSENeUnS0tLA6CwsJD09PSG5wsLCxkwYMAxPyYyMpLIyMimjCF+8MsxPfj4q0LmbtrP8p0HGdKxtelIIhJi6uoqefKrqRAOwyra0vO+2bBnsW9xaWwqdBhOtkY8glKTjnx06tSJtLQ05s6d2/Cc2+1m6dKlnH322U35pcSwzm1juX6Ib4rsLx9twrZtw4lEJNS8/NF97AqHBI+Xa4Y/iSMsDDqdC32v8f2q4hG0Gl0+ysvLWb16NatXrwZ8i0xXr17N7t27sSyLe++9lz/96U/MnDmTdevWccstt5CRkcH48eObOLqY9vMLuxEV7mDFrhLmbNxvOo6IhJCDB7fxQtHnAJxZNZAxxxk9l+DU6PLx5ZdfMnDgQAYOHAjA/fffz8CBA3nkkUcAeOCBB7jnnnu48847GTJkCOXl5cyaNYuoqKimTS7GpcZH8aMRnQB4fNYmPF6NfohI03jyw59S5nTQsQZuufRvWJb1/R8kQcOyA2y83O12k5CQQGlpKfHx8d//AWJUaVUdIx//lNKqOh6f0I/rhuhuJRE5PZs2z+S6xb/Btiwurr2WJ+54xHQkOQmNef82freLBLeE6HDuPr8rAH+bvZmqWp2vICKnzvZ6+dMXv8O2LPqVRZN95X2mI0kzUPmQ03bL8A60S4qm0F3Dcwt16JyInLoPF/6BNc46Ir02XZN/RdeUONORpBmofMhpiwxz8sChjceeXrBdh86JyCmprCziye3TAOjryuLuy64wnEiai8qHNInL+6XTPyuRyloPk+foFGMRabznZuWw32mRWuflzDP+RNs47QEVqlQ+pElYlsVvL+0FwJvLdrO1sMxwIhEJJnl5y3il9CsA2pWew49HDTAbSJqVyoc0maGdWjOmdype27fxmIjIyfrr3PupcVj0qHRy6bm/ITpCG4iFMpUPaVIPju1JmMNi7qb9LNpWZDqOiASBZaueY65disO2iam9hQmD25uOJM1M5UOaVOe2sdw8zPcPx/99uBGvNh4TkROor6vmz6v+CUC/0iTuvPQWnA5tKBbqVD6kyf3swm7ERYbxVb6baSv3mo4jIgHs7bm/ZLvTJt7jJSrml5zTLdl0JPEDlQ9pcm1iI7nnQt/GY3/9eDPlNfWGE4lIIHKV7OBf+z4FoFNxLx644gLDicRfVD6kWdw6vBMd28RwoKyGKZ9uMx1HRALQUx//FLfDQfsam67dfkO3VG0o1lKofEiziAhz8JtDt94+9/kO9hysNJxIRALJ5i0fMK16DwAxBy/l5xf1MZxI/EnlQ5rNRb1TGdG1DbX1XiZ9tNF0HBEJELbXy6RFj+K1LPqWRXHZuT8hqVWE6VjiRyof0mwsy+L/jeuNw4IP1xWw5Oti05FEJAB88sX/scKqJdJrU1t7Jzef1cF0JPEzlQ9pVj3T4rlxqO/W2z9+sAGPbr0VadGqKg/yxNa3AOhdksHdl11JuFNvRS2N/heXZnf/Rd2Ji/Ldevv2l3tMxxERg1765G4KnBZt671EJDzAqB4ppiOJASof0uzaxEby8wu7Ab5bb0sr6wwnEhET9uWv4IWStQCkHBjGg5efZTiRmKLyIX4xcXhHuqbEUlxRy9/nbDEdR0QMeGLevVQ7LLpXOejb5xd0TYk1HUkMUfkQvwh3Ovj9FWcA8MrinWzc5zacSET8afnqF/jE48Jh29ilN/Cz0d1NRxKDVD7Eb0Z0TebSvml4bXh0xlfYthafirQE9XXVTFr5DwD6lSbywzE/JC4q3HAqMUnlQ/zqt5f1JircwbKdB5m5Jt90HBHxg3fmPcBWp5c4jxfC72X8gEzTkcQwlQ/xq8zEaO4+33fuy//9b6POfREJcaWunTyVPxeAzsU9eHD8RViWTq1t6VQ+xO9+fG5nOrSJYX9ZDU/N22o6jog0o399nEOpw0FWjU3nrr/ljIwE05EkAKh8iN9FhTt59PLeADy/cAdbC8sMJxKR5rB12yzertoFQLzrEu6/uK/hRBIoVD7EiAt6pnJhzxTqvTYPv7dei09FQozt9TJp4cN4LIs+5ZGMH/UzEmN0fov4qHyIMb+74gyiwh0s3XGQd1fmmY4jIk1o7qLHWO6oIcJrE2lnc93gLNORJICofIgxWa1j+NmhnU///OFGXJW1hhOJSFOorirh8S3/AaCPK51fT7gOh0OLTOUbKh9i1I/P6Uy3QzufPv7xZtNxRKQJvDjrHvY5LZLrvXTv/Ht6pcebjiQBRuVDjIoIc/DH8X0AeH3pblbuLjGcSEROR8G+VbzgWg1Ax9KzuPeSoWYDSUBS+RDjzurchglntgPgt9PXU+/xGk4kIqfqsdk/o9ph0a3KwXWj/0iryDDTkSQAqXxIQPjNpT1JiA5n4z43Ly3aaTqOiJyCL1e/xBzbhWXbtHdM5JK+6aYjSYBS+ZCA0CY2kgfH9gTgydlb2HOw0nAiEWkMT30tf17xJAAD3An84pps7WQqx6XyIQHj+sFZDO3Ymspaj/b+EAkyr3/yS7aG2cR6vAzv8UeyWseYjiQBTOVDAobDYfHnq/sS4XSwYMsBHTwnEiRKS3fz7wLf+S0DK3rz49GjzAaSgKfyIQGla0os91zgO3ju9+9v4GCF9v4QCUheD+xYCOum8eQ7P8DldNCu1ubOK54izKm3Fjkx/YRIwLnrvC70SI3jYEUtf/rfBtNxROQIU1ZPYeqce2FyH3h5HNtn3sUMx0EAMp0ZLCp512xACQoqHxJwIsIc/GVCXywL3l2Zx2dbDpiOJCKHOIq2kJs3l6mOcmzgsTZJeCyLTrW1LHXuw1G0xXRECQIqHxKQBrZPYuLZHQH4zfR1VNbWmw0kIuD1kL32Y3JKXOQmJfJg2zYsjo7GYdvsiIggp6SU7LWf+KZkRE5A5UMC1i8v7kFmYjR7S6r4q7ZeFzFv1yJw55PtcpNd4uLD2FYAeC2LnBIX2a5ScOf5rhM5AZUPCVixkWH8+eq+ALy0aCfLdhw0nEikhSsvbPit2+Fs+H24bZPtch/zOpFjUfmQgHZe97ZcPzgL24YHpq2hqlbDuSLGxKYCMD86mtcT4gAIs23qLIupifHfuU7keFQ+JOD9dlwv0hOi2FlcyROfaPpFxJgOwymIT+VXKW0A6F9dzaqdexrWgExNTID4TOgw3HBQCXQqHxLw4qPCG6ZfXvhiB1/u1PSLiAker4cfJkRS7XCQXF/PC/v2A5DtcpNTUkpuUgJT+42BI6ZkRI5F5UOCwvk9Urh2UDtsG341bS3VdZp+EfG3ye9OpCAMwrxenimuI+KI17K9seRkXog3ubuxfBI8dNaxBI2Hx/Xms60H2FFUwRMfb+bhcb1NRxJpMZatfpVXK9eBZTGmfjjdfznVd1dLeaFvjUeH4WRrxENOkkY+JGgkRIcz6dD0y/Nf7GDJ18WGE4m0DO7SPfx2xWN4LIuB5a349Q1P+aZWOp0Lfa/x/ariIY2g8iFB5YKeqQ13v/ziv2soq64zHUkkpNleL7+dfgMFYRbpdTY3Dn+W1nFRpmNJkFP5kKDz/y7vTVbraPJcVfz+fZ39ItKc3vzkfuZbbsJsm7MibmfswL6mI0kIUPmQoBMbGcaT1w3AsmDair3MWl9gOpJISNq+fTZ/2zcHgMGlXXng+nsMJ5JQofIhQWlIx9bcNbIL4Dv7ZX9ZteFEIqGlprqUX8z/BTUOi16VYdxx+b+JjdQ9CtI0VD4kaN13UTd6psVxsKKWh95Zh23bpiOJhIy/TL+B7WE2SfVeBqf/H0M7p5iOJCFE5UOCVmSYk8k3DCDC6WDupv28vmy36UgiIWHeoseZVrsXgB7lY7l33CWGE0moUfmQoNYzLZ5fXdwDgD9+sIGthWWGE4kEt4KC1Tyy6WUAznQl88vrHyUiTG8V0rT0EyVB7/ZzOnFut2Sq67zc88Yq7X4qcoo89bU88NHtlDoddKqGCwbl0iMtznQsCUEqHxL0HA6Lv13XnzatIthUUMZfPtpkOpJIUHrm/dtY5aglxuulk/NebhnRy3QkCVEqHxISUuKieOLa/gC8tGgn8zYVGk4kElxWrX2NZ0rXANDr4BAevfGHWJZlOJWEqiYvH7/73e+wLOuoR8+ePZv6y4h8x/k9U7htREcAfvn2Wva7dfutyMkoLd3NA1/+Ba9l0d8dwx1XPkHrVhHf/4Eip6hZRj7OOOMM9u3b1/D4/PPPm+PLiHzHg2N70is9noMVtdz/3zV4vLr9VuREbK+XR967kQKnRXqtl74d/8aIrsmmY0mIa5byERYWRlpaWsMjOVk/yOIfkWFOnrpxANHhTj7fVkTup9tMRxIJaG99ch/z8G2f3r76h9w3drjpSNICNEv52Lp1KxkZGXTu3Jmbb76Z3buPv/9CTU0Nbrf7qIfI6eiaEscfx/cBYPKcLSzaXmQ4kUhg2rb9E57YNxeAAQe78OjNP9NtteIXTf5TNmzYMF566SVmzZrF008/zY4dOzj33HMpKzv2/guTJk0iISGh4ZGVldXUkaQFumZQO64d1A6vDT97Y7W2Xxf5luqqEn4x/5e+7dMrwrjuolyyWseYjiUthGU3857ULpeLDh068OSTT3L77bd/5/Wamhpqamoa/ux2u8nKyqK0tJT4+PjmjCYhrqrWw/jcL9hcWMbZndvw2o+H4XRo9b4IwKP/uZh36/NJqvdyceJf+e2ES01HkiDndrtJSEg4qffvZh9fS0xMpHv37mzbduy598jISOLj4496iDSF6AgnuTefSUyEk8VfF/OPOVtMRxIJCB8vnMS79fkAnFE5jl9dqe3Txb+avXyUl5ezfft20tPTm/tLiXxH15RYJl3dF4CnPt3Ggi0HDCcSMatg3yr+sPU1AAa7Unj4h7/TOg/xuyb/ifvlL3/JggUL2LlzJ4sWLeKqq67C6XRy4403NvWXEjkpVw7I5Mah7bFt+Pmbq9hzsNJ0JBEj6uuq+cWHt+N2OuhcDTee/yyZidGmY0kL1OTlY+/evdx444306NGD6667jjZt2rBkyRLatm3b1F9K5KQ9enlv+rdLwFVZx12vrqCqVue/SMvz93dvYW1YHTFeLyNTHmZMv86mI0kL1ewLThurMQtWRBoj31XF5U99TnFFLVcPzORv1/XX9tES2rwe2LUIygv5In8lP903E69lcVHVcJ64cyoOLcCWJtSY9+8wP2USMS4jMZqnbhrID59fxrur8uiflcjE4R1NxxJpclNWT8FRtIXstR+DO59Sh4NHM9PwhoXRvsZBu0F9VDzEKK0ykhZleJdkHhrrO2vojx9sYPnOg4YTiTQ9R9EWcvPmMtVRjg08mtyawrAwEuo97I70ElO63XREaeFUPqTFuf2cTlzeP4N6r81PXltJvqvKdCSRpuP1kL32Y3JKXOQmJXJPSjJzW8XgsG1Kw5zklJSSvfYT35SMiCEqH9LiWJbFYxP60jMtjqLyGu545Usqa+tNxxJpGrsWgTufbJeba91lLGjl27XUa1nklLjIdpWCO893nYghKh/SIsVEhPHcxMG0aRXBV/lufvHfNXh1Aq6EgvJCAHaHhTG31TfbpYfbNtku93euEzFB5UNarHZJMTzzw0GEOy0+Wl/AZO2AKqEgNpUDTgd3pqVw0OkEfMWjzrKYmhh/1HUipqh8SIs2uGNr/nyVbwfUf87bxvtr8g0nEjk97qT2ZKemkBfuu5nxNlcpK3fuaVgDMjUxAeIzocNww0mlJdOtttLiXTs4i637y/n3Z1/zy7fX0L51DP2zEk3HEmm0mupSsqddxZbICAB+UOrm/pJSgENTLha5SQmQeSHZDqfBpNLSaeRDBPj1JT25oGcKNfVebn95ubZgl6Djqa/l569fwrrwOsJsm/MdXfh1fcxR12R7Y8nJvBBvcndDKUV8tMOpyCHlNfVcN3UxG/a56dK2Fe/8ZDiJMRGmY4l8L9vr5cHXxvChXUi4bXN12A08/IOHj9rhlNhU31SLRjykmTTm/VsjHyKHxEaG8eJtQ8hIiGL7gQrufHUFNfXaC0EC3xNvX8eHdiGWbTO69kJ+c9NvfS84nNDpXOh7je9XFQ8JECofIkdIjY/ixduGEhcZxrIdB/nl22t1C64EtBfez+aV6s0AjKzozx9ue1Jbp0vAU/kQ+ZYeaXFM/eEgwhwW76/J57GPN5mOJHJMMz/9HZOLPwfg7NIs/nDri0SFa3RDAp/Kh8gxjOiazGMT+gHwzIKvef7zHYYTiRxt4bIpPLprGrZlMdidxEM3vEXrVlqjJMFB5UPkOCYMasevLu4B+A6he2fFXsOJRHzWbniHX3yVS71l0a88irvHvU2nlDjTsUROmsqHyAn8dFQXbj+nEwAPvLOWORu0JbWYtWPnAnKWPEKVw0HPSie3j3qLQZ20W6kEF5UPkROwLIvfXtqLq8/MxOO1yXl9JUu/LjYdS1qo/YXruXNuDi6ng441cNOgV7jgjM6mY4k0msqHyPdwOCwem9CP0b18m5D9+OUvWZ9XajqWtDDu0j3c8cFNFIRZpNd6mdD1Ka4a2s90LJFTovIhchLCnQ7+ddOZDO3YmrKaem55YRmbC8pMx5IWorqqhLumXcHXYTat671clvp7br1glOlYIqdM5UPkJEWFO3nu1sH0a5fAwYpabn5uCdv2l5uOJaHG64EdC2HdNNixkPqaCu55/RLWh9XTyuvlktif8bPLJ5hOKXJatL26SCO5Kmu56dmlbNjnJiUukrfuOptOya1Mx5IgN2X1FBxFW8he+zG4facr28AjbdvyXmw0DtvmmrCbefjmB7EsbSImgUfbq4s0o8SYCF778TB6pMaxv6yGm55dooPo5LQ5iraQmzeXqY5vRtOeSkrgvdhoAM6sb6viISFD5UPkFLRuFcF/7hhG15RY9pVWc8O/VUDkNHg9ZK/9mJwSF7lJiUxNjOc/8bE8m5gAwKiKSl5wF2LZXsNBRZqGyofIKUqOjeT1Hw+jU3Ir8lxVXPfMYr4+oDUgcgp2LQJ3Ptkud0MB+Uub1gAMq6ziqf1FWO4833UiIUDlQ+Q0pMRH8eadZzWMgFz3zBLdBSONV+7bvM4Gor02HFqK57Btni088J3rRIKdyofIaUo9VEB6pcdTVF7DDf9erH1ApHFiU/EAj7VO4ok2SWBZOGwbr2XxTGL8UdeJhAKVD5EmkBwbyRt3DKN/uwRKKuu48dklrNxdYjqWBInqlF7cn5rCfxJ857MMr6xi9c49R6wBSYD4TOgw3HBSkaah8iHSRA7fBTO4QxJl1fXc/OxS5m/ebzqWBLji4u3c8sZo5sVEATCmvIJnCg9gwaE1IKXkJiUwtd8YcDjNhhVpIiofIk0oLiqcV24fyrndkqmq8/Djl7/k3ZU6DVeO7eudn3Pze1eyMdxDuNfmHNrxt5rIo67J9saSk3kh3uTuhlKKND1tMibSDGrrvTwwbQ3vrfZtFvWbS3ty58guhlNJIPlyzRvct+JPuJwOUuq8TOzwCLdcdL1vh9Ndi3yLS2NTfVMtGvGQINCY9+8wP2USaVEiwhw8ed0A2sZF8uzCHfz5w03sd9fwm0t74XBok6iW7sPPHuOR7a9S43TQqQayB0zl0qHn+l50OKHTuWYDijQzlQ+RZuJwWPz2st6kxEXxfx9u5LnPd5DnquLJ6wYQHaH/km2pnp3xE/5VshCvw+KMynDuu/AthnXvZjqWiF+pfIg0sztGdqZtXCQPTFvLR+sLyHMt5tlbBpMaH2U6mjSXY0ydeG2bP715NW/X7wDLYnB5PL8e/x49M9uaTivid1rzIeIny3ce5K5XV3Cwopa0+CiemziYPpkJpmNJEzrW4XAAtXEZXJMYyY4wDwAjKzry+x/8l+T4aFNRRZqcDpYTCUBDOrbmvZ+OoGtKLAXuaq6duphZ6wtMx5ImdKzD4dwOiysSbHaEeXDYNuPqzubJO99T8ZAWTeVDxI/at4nh3Z8Ob7gVN/u1Ffz14014vAE1ACmn4hiHw+1zOhmXmU5eeDjhXi+Ti6r4821TiAzTmh9p2VQ+RPwsPiqcF28dwm0jOgKQ++l2bn1xGQcras0Gk9NzjMPhxmRlUBIWRiuPhzf2FXJ+eRHW7sWmk4oYp/IhYkCY08Gjl5/BP24YQHS4k4Vbi7j8qc9Zu9dlOpqcqkOHvtUDFvgOh7MssG3eyyugR23dUdeJtGQqHyIGXTkgk+k5w+nYJoY8VxXXPL2YVxbvJMDWgcvJiE1lT5iT29JT+VdSIlgW1qEC8l5cq6OuE2npVD5EDOuZFs/Me87hot6p1Hq8PDLjK+54ZYWmYYKI7fXy3NppXJOZzuoo3/boo8srWKPD4USOSeVDJADER4Xz7x8O4pFxvYlwOpizsZCx//iMRduKTEeT71Fc/DV3vTCCf7jmUenw/ZP6w1I3fz9QrMPhRI5D5UMkQFiWxY/O6cT0nOF0aduKQncNNz+/lEkfbaSm3mM6nng9sGMhrJvm+9Xr4aOF/2DCe5ezOLycMNumc31rfpwyigfqY476UB0OJ3I0bTImEoAqa+v54wcbeWPZbgC6pcTy12v7MyAr0WywFuhYG4dVWxZPtk7mjUN7dbSrtbmu3X3cesmPsCxLh8NJi6SD5USCXExEGJOu7suoHm357fT1bN1fztVTvuCOcztz30XdiQrXG5m/HN44DEc52cDmiHAebNuGbRERAHSuCeOxS2fSs13WER+kw+FETkTlQySAXXxGGkM7tuYPH2xg+qo8nvnsa2ZvLGTSVX0Z1rmN6Xih79DGYTjKyU1KZFVkJMujo6izfCcTjysr58+10VgZGYaDigQXrfkQCXBJrSL4+/UDePaWwaTERfL1gQqu//cS7ntrNfvLqk3HC22HNg7rV11L6/p6FsVENxSPH7lKmVR0EMud57tORE6ayodIkLiodyqz7zuPm4a1x7Jg+qo8LnxiAS9+sYN6j9d0vOB1jIWkh63fOpufprblrvQUDoaF+TYOA8Jtm/tKSr/5HNo4TKRRtOBUJAit2ePi/81Yz9q9vjfAnmlxPDi2J+d1b+tb8Cjf63gn0BKfwd+6D+ezA1+xyyrHY1mE2Ta9a2pYGxVFuG1TZ1nklLjIdrl9HzPxA63xkBZPp9qKhLj+WYlM/+kI/nxVXxJjwtlUUMatLy7n5ueWsm5v6fd/AjnmCbS1wF3RdbxUvIyvHRV4LIvzKqq5zl3G2qgockpcrNTGYSKnTQtORYKU02Fx07D2XNo3jdxPt/Hyol0s2l7M5f/6nMv7Z3Df6G50bhtrOmZg+Patr1nDjlpICtC5to7fJbemzOm7k6hHTS0XpdyONyOfKflzySkpbRjp8P1qkZuUAJkXkq3baEUaRdMuIiFib0klT36yhemr87BtcFhwad90cs7vSq/0lvn/peNOrcQkMzWilnosdoSH8UnsN2evxHg8PHTQxeXlFTgnfsCU0nXHmZ7JZGq/MXiTu/PTAT/1499KJDA15v1b5UMkxGzId/Pk7M3M2bi/4bnRvVLIOb8rA9snGUzmf1Pn3Etu3tyj12cA/0xM4NmkBOI8noaRDgCHbbN4115iDv+zOOF56HuN7/faOEzkhLTJmEgL1jsjnucmDmFDvpvc+dv4cN0+5mzcz5yN+xmQlcjE4R24tG86kWEh+MZ5ZEFo1ZbsNbPAWdEwtTK+rIIH2rZhVXQUAGVOJ1FeL9UOR8NC0lcS4r4pKkeeQKuNw0SajEY+RELc9gPlPD1/OzNX51N76JbcNq0iuGFoFjcMaU9W65jv+QyB77jTK8DTifHMjYlhc2SE71bZQ3cDdayto2NdHfNbxTSMjExNjCc3KdG3vsMbC/eu0+iGyEnSyIeINOjSNpYnru3Pry/pyVvLd/Pakt0UuKvJ/XQ7uZ9uZ2jH1lx1ZiaX9kknISbcdNxT8u0t0AG2hofzpzZJrDw0ygGAZWHZNk8VHmBDZARTkhKPmpLRQlIR/9DIh0gLU+/xMntDIa8t9d0dc/hfgAingwt6pjDmjFTO75FCUqsIs0FPltcDk/sw9dCdK8Mrqyh2On0jHYdEeL3UOhyE2Tb1h/bo8GLhwCa7Nhwqi7/5fFpIKnJKAmLBaW5uLn/9618pKCigf//+PPXUUwwdOvR7P07lQ8R/9pVWMWN1PtNX5rG5sKzheYcFgzokcWGvVM7r3pYeqXE4HIG3eVlVrYdtyz6i75ybAbg+PZUNUZENr3eqraVHbR2zYlsdf2rlZ6thz1ItJBU5TcbLx1tvvcUtt9zC1KlTGTZsGJMnT+btt99m8+bNpKSknPBjVT5EzNiQ7+aj9b7FqRv3uY96LT4qjCEdWzOkU2uGdGxNr/Q4YiL8P2tbVF7D2r0ulu44yPIdB1mXV8pY+wv+GfEvAL6MiuRHaSnYlkW4bXOnq/RQ0Tj6bpepiQnkJiWQk3kh2aMn+/3vIRKKjJePYcOGMWTIEP71L98/CF6vl6ysLO655x4efPDBE36syoeIeXtLKpm3aT9zN+5n+c6DVNZ6jnrdsqBjm1b0So+jZ1o8XdrGkpkUTUZiFG1jI09ri/c6j5eC0mr2llSxt6SSrfvL2bjPzcZ9ZRSV13zn+ktjtzGl/hGAhlGNw3euDKmqZmh19VHFA9DUikgzMFo+amtriYmJYdq0aYwfP77h+YkTJ+JyuZgxY8YJP17lQySw1Hu8bNjnZtmOgyzbcZCVu13HLAGHRYQ5yEiIIiEmgvioMOKjwomPDiMyzMmRncS2obymnrLqOtxV9bir6ygur6WwrJrj/atkWdCpTauGUZhhnVrTLiEC6x99meqo8I1mHG965copUFmkqRWRZmL0bpeioiI8Hg+pqalHPZ+amsqmTZu+c31NTQ01Nd/8Q+Z2u79zjYiYE+Z00K9dIv3aJfLjczsDcKCshs0FZWwqcLNhn5tdxZXklVRRWFZNbb2XncWVUFx5yl8zIsxBZmI0mYnRdE2JpWdaHD3T4+meGnvM6Z6p/S4+tJnYCbZA7zLqlPOISNMyfqvtpEmT+P3vf286hog0Qtu4SNrGRXJOt+Sjnq+t91LoribfVYW7uh53VZ1vZKO6npr6b03dYBET6SQ+Kpy4QyMkiTHhZCZFk9wqslELXL3J3ckBsos/Br45WC/bGwuZF+JN7n46f10RaWLGp12ONfKRlZWlaRcRaTxtgS5iTGOmXRxN/cUjIiIYNGgQc+fObXjO6/Uyd+5czj777O9cHxkZSXx8/FEPEZFTcngL9L7X+H5V8RAJSM0y7XL//fczceJEBg8ezNChQ5k8eTIVFRXcdtttzfHlREREJIg0S/m4/vrrOXDgAI888ggFBQUMGDCAWbNmfWcRqoiIiLQ82l5dRERETpvRNR8iIiIiJ6LyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+1Szbq5+Owxuuut1uw0lERETkZB1+3z6ZjdMDrnyUlZUBkJWVZTiJiIiINFZZWRkJCQknvCbgznbxer3k5+cTFxeHZVmm4xjndrvJyspiz549OuumGen77B/6PvuPvtf+oe/zN2zbpqysjIyMDByOE6/qCLiRD4fDQbt27UzHCDjx8fEt/gfbH/R99g99n/1H32v/0PfZ5/tGPA7TglMRERHxK5UPERER8SuVjwAXGRnJo48+SmRkpOkoIU3fZ//Q99l/9L32D32fT03ALTgVERGR0KaRDxEREfErlQ8RERHxK5UPERER8SuVDxEREfErlY8gVFNTw4ABA7Asi9WrV5uOE1J27tzJ7bffTqdOnYiOjqZLly48+uij1NbWmo4WEnJzc+nYsSNRUVEMGzaMZcuWmY4UUiZNmsSQIUOIi4sjJSWF8ePHs3nzZtOxQt5f/vIXLMvi3nvvNR0laKh8BKEHHniAjIwM0zFC0qZNm/B6vTzzzDN89dVX/P3vf2fq1Kn85je/MR0t6L311lvcf//9PProo6xcuZL+/ftz8cUXs3//ftPRQsaCBQvIyclhyZIlzJ49m7q6OsaMGUNFRYXpaCFr+fLlPPPMM/Tr1890lOBiS1D58MMP7Z49e9pfffWVDdirVq0yHSnkPf7443anTp1Mxwh6Q4cOtXNychr+7PF47IyMDHvSpEkGU4W2/fv324C9YMEC01FCUllZmd2tWzd79uzZ9nnnnWf//Oc/Nx0paGjkI4gUFhZyxx138OqrrxITE2M6TotRWlpK69atTccIarW1taxYsYLRo0c3POdwOBg9ejSLFy82mCy0lZaWAujnt5nk5ORw2WWXHfVzLScn4A6Wk2OzbZtbb72V7OxsBg8ezM6dO01HahG2bdvGU089xRNPPGE6SlArKirC4/GQmpp61POpqals2rTJUKrQ5vV6uffeexkxYgR9+vQxHSfkvPnmm6xcuZLly5ebjhKUNPJh2IMPPohlWSd8bNq0iaeeeoqysjIeeugh05GD0sl+n4+Ul5fHJZdcwrXXXssdd9xhKLnIqcnJyWH9+vW8+eabpqOEnD179vDzn/+c//znP0RFRZmOE5S0vbphBw4coLi4+ITXdO7cmeuuu473338fy7Ianvd4PDidTm6++WZefvnl5o4a1E72+xwREQFAfn4+o0aN4qyzzuKll17C4VBPPx21tbXExMQwbdo0xo8f3/D8xIkTcblczJgxw1y4EHT33XczY8YMPvvsMzp16mQ6Tsh57733uOqqq3A6nQ3PeTweLMvC4XBQU1Nz1GvyXSofQWL37t243e6GP+fn53PxxRczbdo0hg0bRrt27QymCy15eXmcf/75DBo0iNdee03/iDSRYcOGMXToUJ566inANy3Qvn177r77bh588EHD6UKDbdvcc889TJ8+nfnz59OtWzfTkUJSWVkZu3btOuq52267jZ49e/LrX/9a01wnQWs+gkT79u2P+nNsbCwAXbp0UfFoQnl5eYwaNYoOHTrwxBNPcODAgYbX0tLSDCYLfvfffz8TJ05k8ODBDB06lMmTJ1NRUcFtt91mOlrIyMnJ4fXXX2fGjBnExcVRUFAAQEJCAtHR0YbThY64uLjvFIxWrVrRpk0bFY+TpPIhcoTZs2ezbds2tm3b9p1Sp0HC03P99ddz4MABHnnkEQoKChgwYACzZs36ziJUOXVPP/00AKNGjTrq+RdffJFbb73V/4FEjkPTLiIiIuJXWkUnIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+9f8BYGdCfJk4eWcAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def function(x):\n",
    "    return x**2\n",
    "\n",
    "def analytical_gradient(x):\n",
    "    return 2*x\n",
    "\n",
    "def gradient_descent(starting_point, learning_rate, num_iterations, solver=\"analytical\"):\n",
    "    x = starting_point\n",
    "    trajectory_x = [x]\n",
    "    trajectory_y = [function(x)]\n",
    "\n",
    "    if solver == \"analytical\":\n",
    "        grad = analytical_gradient    \n",
    "    elif solver == \"jax\":\n",
    "        grad = jax_grad(function)\n",
    "        x = jnp.float64(x)\n",
    "        learning_rate = jnp.float64(learning_rate)\n",
    "\n",
    "    for _ in range(num_iterations):\n",
    "        \n",
    "        x = x - learning_rate * grad(x)\n",
    "        trajectory_x.append(x)\n",
    "        trajectory_y.append(function(x))\n",
    "\n",
    "    return trajectory_x, trajectory_y\n",
    "\n",
    "x = np.linspace(-5, 5, 100)\n",
    "plt.plot(x, function(x), label=\"f(x)\")\n",
    "\n",
    "descent_x, descent_y = gradient_descent(5, 0.1, 10, solver=\"analytical\")\n",
    "jax_descend_x, jax_descend_y = gradient_descent(5, 0.1, 10, solver=\"jax\")\n",
    "\n",
    "plt.plot(descent_x, descent_y, label=\"Gradient descent\", marker=\"o\")\n",
    "plt.plot(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a5f19b5",
   "metadata": {},
   "source": [
    "### A more advanced example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "d8f5eb38",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x1394faca0>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGdCAYAAAAvwBgXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB8EklEQVR4nO29eXgk5XXv/63e1WrtGmk2zQ4MGAbMMsNmG2w2G1/HdoIhxgtcmyfJBS4EJ8CYxNwktoc4xBu2MfHvF5sEMCQkQLzAhRiDbdZh2DEzwwyj2bSv3Wqp16r7R/dbVdK0WrW8b9Xb6vN5Hj3PjEZSvyO1qk99z/d8j6JpmgaCIAiCIAgfCPh9AIIgCIIg6hcqRAiCIAiC8A0qRAiCIAiC8A0qRAiCIAiC8A0qRAiCIAiC8A0qRAiCIAiC8A0qRAiCIAiC8A0qRAiCIAiC8I2Q3weohqqq6OvrQ1NTExRF8fs4BEEQBEFYQNM0pFIpLF++HIFAdc1D6kKkr68PPT09fh+DIAiCIAgHHDx4ECtXrqz6MVIXIk1NTQBK/5Hm5mafT0MQBEEQhBWSySR6enr01/FqSF2IsHZMc3MzFSIEQRAEUWNYsVWQWZUgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgJOG1gxP459/tg6pqfh/FM6TevksQBEEQ9cRXHnkTrx2axHHLm3H6ug6/j+MJpIhUIZMvQtPqpyolCIIg/GU4lQUAHByb9vkk3kGFyDwcGp/Ge//2CVxz3ytUjBAEQRCekMoUAABD5YKkHqBCZB7e6ktiJl/EL97ox4M7Dvl9HIIgCGKRo6oapnLlQiSZ8fk03kGFyDxkC6r+57/7+e/r6klBEARBeM9UrgAmwJMiQiCbL+p/TmYK+KuH36QWDUEQBCEM1pYBgME6uvmlQmQeMmVF5KiuBEIBBY//fhC/eKPf51MRBEEQi5VUJq//eTBJikjdwxSR45Y343+duwEAcOsjb2EsnfPzWARBEMQiJTljKCLDqWzdqPBUiMwD84hEQwFcc+4GHN2dwGg6h7/52Vs+n4wgCIJYjJgVkVxRxcR0vspHLx6oEJkHoxAJIhIK4Bt/dCICCvDIq314+cC4z6cjCIIgFhtmjwhQP4ZVzwqR2267DYqi4Prrr/fqIV3BWjOxcOlbdFJPK85YX0q52zec9u1cBEEQxOLErIgA9WNY9aQQ2b59O+666y5s2rTJi4fjglkRYcQjpUT8TKFY8XMIgiAIwinJOYoIFSKcmJqawuWXX44f/ehHaGtrE/1w3MiWi41oyPgWxcKloiSTVyt+DkEQBEE4hVozgrj66qtx8cUX47zzzhP9UFxhxUY0bCpEykVJlhQRgiAIgjOsNaMopb/XS5Cm0O27999/P15++WVs377d0sdns1lks0YFmEwmRR1t4bMUmEfEaM2wooQUEYIgCII3TBHpaYvjwNg0KSJuOXjwIK677jrce++9iMVilj5n27ZtaGlp0d96enpEHW9BsnljfJcRK/tFSBEhCIIgeMMUkfVLGgGQR8Q1O3bswNDQEE4++WSEQiGEQiE8/fTT+O53v4tQKIRi8cgX861bt2JyclJ/O3jwoKjjLUhG94gcqYhkSREhCIIgOMMUkQ1dCQD1k64qrDXzoQ99CG+88cas91155ZXYuHEjbrrpJgSDwSM+JxqNIhqNijqSLVixEQsfqYhk8qSIEARBEHxhhcj6JaVChKWrKsw0skgRVog0NTXh+OOPn/W+xsZGdHR0HPF+Gak0vsv8IubNvARBEATBA701U1ZEWLpqW2PEz2MJh5JV56HS+K5hViVFhCAIguALU0Q6GiNojYcB1McIr9Cpmbk89dRTXj6cK4zxXZMiEiJFhCAIguCPqmqYypUKkaZYGN1NMUxM5zGYzOCYpU1CHvPd4Sn0jqaxsi2Oo7vFPIYVSBGZB1JECIIgCK+YyhXAlu02xULoai75JUVOzvzi9X78z5+8hB8/s0/YY1iBCpF5YKqH2awaJbMqQRAEIQDWlokEA4iFg+hqKsVeiGzN5Iqq/ph+QoXIPLBiY7ZZlSWr+tOaGZnK4p3BlC+PTRAEQYiDGVWbYiXHRHdZERGZrporv5ZFQv6WAp56RGoFTdOMqRkJFJHJ6TzufHovfvzMPmQLKp76i3OwprPR0zMQBEEQ4mCKCCtEuprKhYhARSRLhYi85Iua3qurpIh4FfGeyRfxk2d7cedTezE5Y6yHPjA2TYUIQRDEIsJQRErTMt3NpdaMSI8Ia82EfW7NUCFSAXOEe6Xtu160ZnpH0vjjHz2P/snSk/Do7gTS2SIOT8xgKltY4LMJgiCIWuIIRUQvRMQpInlJFBHyiFTArHjMmpph23c9aM08vXsY/ZMZdCaiuP2SE/Hode/H0d2lkJupDBUiBEEQi4nkPK0Zlq4qAjKrSox5dNccreulIsJUjw9uXII/OmUlggEFibJklyJFhCAIYlExtzXDxndZuqoIcoUjl7v6ARUiFcjO88Nhf88VVRRVMRUqY7ocbBOPGN2zRLT0Z1JECIIgFhdzWzPRUFB4uqosUzNUiFRAH90Nz17MFzP93ewjEUE6W/r6jVHjMdkTdCorpjomCIIg/GGuIgIA3U1iDauymFWpEKlApTAzYLZCkhU8OVNVEaHWDEEQxKKCKSLNMeOaLzpdlRQRiWFFhnl0FwBCwQBCgZJnJCNaEcmVFZGIcQZWiKSoNUMQBLGoSM7MDjQDIDxdlcyqElNpzwxDN6yKVkTKqkc8alJEYqSIEARBLEYMRcTUmhGcrkqKiMSw8d3YHI8IYBQn3ikiRiHSRGZVgiAIrgxMZoR7/qxgmFWNQkR0uioVIhJjRRERna6qe0RMZlVSRAiCIPhxYHQaZ972K3zsjmdmpVf7wdxdM4D4dFVqzUjMfOO7gLF7RnSo2XT2SEWEPCIEQRD8eHsgCVUDdg2m8Gf37NAVAj+YO74LiE9XpWRViWFFRuXWTFkREfyETetTM5XGd6kQIQiCcMvEdE7/87N7R/Hlh94QlmJaDVXVMJWbvzUjKl1VV0SoEJGPaoqIsfjOI0XEbFaNlp6gU9mCL78sBEEQi4mxdKkdsn5JIwIK8OCOQ/jek3s8P8dUrqAvWm2qML4rKl1V375LrRn5MAqRIxWRWEh8zLumaboiMmt8t/wELaqaZxuACYIgFivjZUXkgxu78Ld/cDwA4B+f2I1HXj3s6TlYWyYSDMxS4qOhINoEpquSWVVijGTV+T0iIhWRbEEFS5A3j+/Gw0Gw1TcpSlclCIJwxXi6VIi0NUbwmdNX40/evw4A8Jf//jr2DKU8O0cloyqjS2C6ap7MqvJiJKv6o4ikTR6QBtMZAgEFiQiN8BIEQfCAKSJt8QgA4KaLNuKU1W3IFVU89+6YZ+eoZFRliEpXLRSNG15SRCSEmVX9mpqZLmeINISDCAaUWf9GI7wEQRB8GEvPLkQCAQXrOhsBGCqFF1TaM8MQla7KjKoAFSJSUtWsyqZmBBYiuj8keqQiQxt4CYIg+MAMoO2NEf19rBhIznh3ja2miIhKVzWPKlNrRkIyVcZ32dSM2NZM6fHNC+8YTBFJkSJCEAThijG9NWMoEawY8FIRSVYtRAQpIuXXMEXBEcq711AhUoHqgWbiFZHpChkiDFJECIIg3FNUNT1Ntc2kiDQ3lIoSL4Mjq7dmSorIAG9FxGRUVRQqRKSj+viud4qIOUOEIVOo2e7BFM7c9iv820sH/T4KQRCELSZn8np2R2vDkYpI0lOPyPyKSGvZv8K7MJJldBegQqQi1cd3JVFEJChEfvlGP/omM/jn3+3z+ygEQRC2YEbV5lgIIZNHojnm/SqNaoqIPiDBeTEfU0QqKf9e4/8JJKSaIqJv3xUYKFZp8y6DpavKsG+mdyQNANg5kMLolJhdCARBECJg8e5moyoANMfYNdZ7RaS5giLCXnOynF9zcpKkqgJUiFRE375bQRFhBlaRa6Ons0du3mUY47v+B5rtG53W//y8hzP3BEEQbmGKCGt9MGSbmokKyq5ihUiYFBE5YZVnRbOqz4pIk0Rm1f2jaf3Pz+4d8fEkBEEQ9hifRxHxY2qmamsmJLY1Q4qIpGQK1cZ3PfCIWFJE/C1EJqZzs5YwPbt31MfTEARB2GO8fP1qjc9+8WeFSDpXRKHozU6vqoqIKTKC57JTMqtKTjVFxGjN+OUR8d5IVYl9ZX9IS0MYAaX0976JGV/PRBAEYRW2Z6Z9ntYM4N0Nn1GIVFJESq85mgbki1SI1A3WzKo+Tc1IoojsL/tDNi5twgkrWwEAz5EqQhBEjTBmWnhnJhIK6MGVXt3wJassvTPfEPNsz1BrRnKMZNX5FZGcXzkikozvMkVkbWcjzlzfAYDaMwRB1A6sNdM2RxEBTIZVD3wiqqrp1/OFCxF+rzukiEiMpmm1oYj43JrpLRtVV3c04qz1nQCA5/aOcO1hEgRBiMIwqx7ZDmFjtF5MzqRzBT1YrblCa0ZRFL1Y4FmI5EkRkRfzRsJq47sZvz0iPisiveXWzNrOOE5Z3YZIMIC+yYzesiEIgpCZ8enZm3fNNHmYJcL2zISDyrzhYkaWCMfWDCki8mKuOCubVfk/IeZSbWqmqRxoliuoQrNMFoKFma3pbERDJIj3rmoFQO0ZgiBqg/F5PCKAeYRX/A2feXR3vp0vIrJEslSIyAtruShKZcmKPSFEKiLTVRSRRlNxwrwkXjOezunLola3NwIAziy3ZyhPhCAI2SmqGiZm5veIsMV3XnhEqo3uMqICWjNkVpUY8+hupeqUKSJFVRM2Y54ue0QaKygioWAADeX2kF8+kX1lf8jS5hgayj6WMzeUDKvP7R2FqpJPhCAIeUmaF97F5/eIeKuIVClEBCjxlKwqMdWMqsDskDNRqsh0WemIV1BEAMOwmvIp5p0lqq7pjOvvO3FlKxrCQYymc9g9lPLlXARBEFYYK/tDmmIhhCsoAl56RHRFJHpkQcQQ0Zohs6rEVBvdBWb/0ERMzuQKqi6ZVWrNAP7HvO8bYUbVRv19kVAAp61tBwA8u4d8IgRByIseZlbBHwJ4OzWT9Ks1U5g/uNNr/D+BZCykiAQCYkapGDM5o7hpqDC+C/gfasaMqqs7Gme9n/JECIKoBYx498qFiK6IeKA6V9szwxCxb4amZiRG37xb5YcjMkuE+UMiwcC8T5CEz6FmemtmnkLkhXdHPdvRQBAEYRcj3r3yi7+3UzOlx2huqOYRKbdmOC5bJbOqxOhm1XlaM4DYxXd6mFkFoyrDz30zmqbNSlU1857lLWiKhZDKFrBzgHwiBEHIyViVDBHACBZLzsimiPAf3yWzqoQwRSQ2T2sGMGWJCGjN6PHu8/hDAH9bM+PTeb2nuao9PuvfggFFV0kGkxnPz0YQBGEFPcxsHo+IL4qIJY8Iv5tftkCPFBEJ0T0iVRQRPUtEYGumUrw7w0+zKlNDlrXEKnpYmPlrtCx9EgRByMZCZlVj14x3hUh1syr/qZlcuaghj4iEGDki/igi+uhuhYV3DD8Vkfn8IYyORLkQmaJChCAIOTHMqpXbIcyv4U2gmYXWjJ4jQkvv6oJMofr4LmCqTgUqIo1VFJFElM24e1+ImKPdK9GZiAIARqeynp2JIAjCDoZZtboi4sUqDXvJqhynZoryjO/O/z+vU+woIhmO1SmDxbvPF2YGmBUR7wPN9pWX2q3piFf89w5qzRBE3TOdK+C5vaN4sXcM5x7ThdPXdfh9pFkws+p847sJkyKdyhQQTcz/euAWoxDxONCsUPKIVAp08xoqROZgZXw3pj8pBCgi2fnj3RnMI+LHrhkjVXW+1kxJERkhRYQg6orxdA4Pv3oYv941jOffHdWl/yfeGsSTf3GOv4ebw0S5NTOfRyQYUNAULU0AJmfyutIrgqSViHcBikhWovFdKkTmwFQOc5T7XKJ+KyJsfNdjj0i10V0GU0TGSBEhiLri2p++gt/tMZZeLm+JoW8yg32jaWTyxarXVC8pqhom9KmZ+VUIFkUgsgWuqpru9bO2a4Y8InWB74qIFY8Ia814YKQyM5bO6b+Uc0d3GWRWJYj6Q1U1vHxgHABw3YeOwn/f8H48c/MH0dIQhqYB7w6nfT6hQXImD7aXs7WhsiICmPfNiCtE0rmCvnyv2ePWDE3NSEzWQv5+VA8082lqxqdk1d5yW2Z5S2zeuxvWmhlNZ6FptIWXIOqBQ+MzmM4VEQkFcO0HN2BDVxMURcGGrgQAYM/wlM8nNGAZIk3RUNUXYS8mZ1iREw4qltK8RZhVZfCI+H8CyTCSVau0ZjyIeK+miDTF/MkR6S0vu5vPHwIYrZl8UfNkBp8gCP/ZOZAEAGxYkkDI9MK2YUm5EBmSrxBprdKWAbzZwGs2qiqKMu/HiUhWZWZVGaZm/D+BZGSstGbC/GUyhh1FJJ0roqh6pzr0LmBUBUrfG3Y+GuEliPqArXTYuLRp1vuZIrJXokJkLF02qs4zMcPwIl01ZcGoCgjeNUOFiHzUgiKSMD1p2cd7ATOqzje6y2A+ETKsEkR9sIsVIssqFyIyKiLzxbszvNg3YyVDBBC8fZdaM/JhyawqUhGxMDUTDQX1J4+X7Zn9eobI/IoIYIzEjZBhlSDqAtaaOWZp86z3s0Jk30hamo3cLMxsvoV3DFYciGwx66O70eptIhGtGZqakRgr47tGoJk/OSKAPzHvVlozANDRaBhWCYJY3GTyRV0tnduaWdHagFg4gFxRxaHxGT+OdwQs3n3hQkT81Ay78Vzoes97akbTNDKryowVRcRYeuePIgKYskQ8UkTyRVV/rK6m6uE+nTTCSxB1w56hKahaaW/L3GtDIKBgXadc7Rlj4V11FcKLqRm2JqSaFaD073xbM2zzLkCKiJRYGd+NcX5SmJnOWVREPB7hNRc8iSpGWsCcJUKKCEEsdsxG1UqTH7KN8C4U787wYmomU369iVVZKQKYWjOcbn5zpjYZTc1IiFGIVGvN8HcwM1hse+NCiojHI7zMsNUYCc4az6uE0ZohRYQgFju7yv6QjXP8IYz1ko3wslTV+eLdGV5MzWR0RaT6NZV3ayZn+jpkVpUQ9sSovn237BHhrIgUVQ0zedaaqV4hN+mKiDfpquyXsbmhupwJULoqQdQTTBE5Zo4/hCHb5Ayb5muNL9CaYVMzIlszdhURTq85rBAJBRQEAvPnl3gFFSJz8FMRmTGZXxsXaH8kPKjWzVhZzMQgsypB1A9WC5G9Q1NSpC2PL7DwjtEskyISNqZmeHwPWSEig1EVoELkCLIWnhiiFJHpst8joCzct/PaI8JaM9X2ITBIESGI+mAsncNwqnTDcUx35UJkTWccAaW0pHMo5e/NiWpaeLdwoJkxNSOqgNKnNBdUREr/rmmzjaZOkSnMDKBC5Ah081DV8V0xikg6Z/hDqsX9At57RJy0Zsamc54mvxIE4S0sP2RVe3xeFTcaCmJ1OXvI7/ZMMmNaeLdAIcKmZoqqpk8z8oa1WqpZAYDZN6Y82jMyZYgAVIjMQtM0/QdkZWqGtyLCMkTiC0zMAGaPiHytGXanoWmGMYwgiMXHrgXaMgxZDKvMH5JYYOEdADSEgwiW/ROi2jN6kvcCZ5ldiLi/AdYVkXpozWzbtg2nnXYampqa0NXVhY9//OPYtWuXyId0hfkHbC1HhHNrJmdtYgYw5Yh4VoiUFRELrZlQMKAbwWhyhiAWLzv7K++YmYsshlXmD1nIqAoAiqKYJmfEGFaN4YjqN5+KouiFE5dCxMINt5cIPcXTTz+Nq6++Gs8//zyeeOIJ5PN5XHDBBUin0yIf1jHmH3C1JwZv4xCD7Y2xoogkygWB1+O7VhQRwNjCO0JZIgSxaNk5yAqRyqO7DGkKkbS10V2G6MmZrAUrAMPIEuHXmpHFrGrtVcUhjz322Ky//+QnP0FXVxd27NiB97///SIf2hHsBxxQSmNN82E2DuWKatUJGzvom3dtKCJet2aseEQAoCMRxd7hNBlWCWKRoqoa3hm01prRJ2d8DjXTF94t4A9hiN43o0/NWFAmoqEgUihwUUTy9WxWnZycBAC0t7d7+bCWMY/uVjOLmo1FPJcQWdm8y2jyy6xqoTUDmGPeSREhiMXIwfFpTOeKiIQCC27kXr+kZFYdSmWF5nIshFGIWLuOiQ41yxSstWYAvovvsvVqVlVVFddffz3OOussHH/88RU/JpvNIplMznrzEn3PzAIO5kgwAFan8PSJTOtmVQkVEdutmVKWyBh5RAhiUfJ22R9yVFdiwbTlplgYS5tjAPxtz4ylywvvLLZm2Agvu/7xRjerLvCaY/4YLq2ZejKrmrn66qvx5ptv4v7775/3Y7Zt24aWlhb9raenx6vjAbA+060oCvfsf8A8vmvFIyLWRDWXpI3xXcDowY7UaCGSyRexezAlRQATQcjIrgFr/hCGDD6RCZutmWbBG3gz+pJVK4oIv5j3uhzfveaaa/Dzn/8cv/71r7Fy5cp5P27r1q2YnJzU3w4ePOjF8XSsKiKAKUuE4wgvW3hnxSNiHt/14sUyZWN8F6jt1oyqavjs//8CLvjWb3D747uoGCGICuwaZDtmqvtDGOaEVb9gCq11RUT01Awzq1rxiPBrzTCPSF2YVTVNw7XXXouHHnoITz31FNauXVv146PRKKLR6ivmRWJ1phtgqklefyLxQF94Z2lqpvSjU7VSNLyV4sUNdpJVgZJZFajNdNWHXjmM7b3jAIDv/3ovFCj40gVHLxgyRxD1BBvdXcioylgvgSIybjFVldGsm1VFtWaceET4Tc3Uxfju1VdfjXvuuQf33XcfmpqaMDAwgIGBAczMzIh8WMfYMg6xUDOeHhEbikhDOAg22CPasKqqmp5XwtIGF4KN79Zajkg6W8DfP7YTALBlbclU/b1f78E3n9hNyghBlMnki+gdLcUwWFZEWKiZj5Mzk+UbqhaLLWbWihbXmrFeEEQ5JnrXVWvmzjvvxOTkJM455xwsW7ZMf3vggQdEPqxj7CsivKdmrHtEFEXxLNQsnSuAvQbbV0RqqzXzg6f2YCiVxeqOOP7lC5vx1x89DgBwx5N78C0qRggCAPDO4BRUreQFW9JkTcVmrZmDY9PcwyCtwm7aEhZbzCKnZlTVSPL2empGNrOq8NZMLWFl8y5DiCJiY2oGKDm6k5mCcEWEGVUjwYBlKY8pIslMAbmCKk3lXY0Do9P40W/3AQBu+cixiIaC+MLZa6FpGr76i7fx3Sf3IBoO4upzN/h8UoLwl3eGSm2Zo7sTlluWnYkIWhrCmJzJY99IGscus2Zy5QmbMkxYaH8DYqdmWDEAeN+aYa914ZAc7Wb5Xx08RDer+q6IWCtEvBrhTelhZgsv42O0NIT1PQ21MsL79V++jVxBxVkbOnD+cd36+7/4vnW45SPHAgDuePIdvYVGEPVK30Spvd7TVj0/xIyiKHqeiB8+EU3TjGusxZs9kVMz5ptYq4FmAF+zaiTIJ4zTLVSImDAczD57RCxW6wnBYTuM5Ezp6zdZbMsAQCCgGCO8NdCeeXbvCB57awABBfjKR99zRMH1xfetxar2ODJ5FU/uHPLplAQhB4cnMgCAZa0Ntj5vZblwGUxmuJ9pIbIFVd8GnrCsOoubmmGvN8GAYml6xcgRIY/IosaOImIsvuOniLCId9kUEWNixl4nr1YMq4Wiir/92e8BAJ85fXXFKQBFUXDxpmUAgF+83u/p+QhCNvonS4rIitaYrc/rSPh3TTBfJ61eY0VGvLPXm5jFYkDE1AwVIhJiJ+UuFub3pGDoS+8smFUBQxGZEhxqlsqyDBHriggAdCZYuqrcisjTu4excyCFloYw/vy8o+f9uItPKBUiT+4cQtqjRFuCkBHWmlnWYk8RYTcnYz6M9TMvXTwSRKDKLjEzbGpmKlvQ1RReZPTXG2vXexGBZnUxvltrOEm5E6KIWJUNPVNE7I3uMvS7H8mzRHbsL2WGXPSepVWDjt6zvBlrOuLIFlT8itozRB3TX27NLLfZmtGn6Xy4OWHXSavXV2B2gCPv66yviogeaEZmVenwUxEpGamsL70D4Nn4rt0wM4bhEZG7EHn9UGkZ46aelqofN7s90yf8XAQhI8lMXr/mLLfZmmn3sV2b1idmrBci0VBQb1/wnpyx40kEOHtEJBvfleMUkmBnfJc9eXgpItmCCqb8WR3fTXi0gZdddKzGuzM6ayBLRNM0vH5oAgBw4srWBT/+4hOWAwB+vWvYs4WDBCETTA1pjYdtJzqz1Q9+TNKxGz07hQggbnKGDTpY9WmI2TVDUzPSoUtlNnL/eU3NmD0HDRYrZO/NqvYUkVowq/aOTiOZKSASCliKqj52WRPWdTYiV1Dxq7cHPTghQchF36QzfwgAtDf6t/physYKDTPNgiZnsjbCzAAyq9YNunnIhiLCK0dkujzf3hAO6vkbC9HklSJic/Muw+gHy1uIMDXkuGXNlkboaHqGqHeYUdXuxAxgtGamsgXP01X1VFWbioioyZlM3vqNL8A5WZUKEXmxFWim9+s4KSLMH2KjWk9Ey5KhaEXE5uZdRkcNbOB97WDJH3Liyur+EDOsEHlq97CwrZwEISusNeNEEWmOhXSDpNftmbQDsypg3jfD2yNifTgC4LxrRveIkFlVOrJ2FhCxqRlOZlW2eddOz5UFn4keJXXdmpHYrMoUkU0W/CGMY7qbsH4Ja8/Q9AxRXzBFxO7EDFBSFJkq4nUh4mRqBhC3b8ZozXg/NaMnq5IiIh8ZGyuZYxwdzIB58651RSRePueMYImT/QLaV0RKrZmZfFHKWPRCUcWbfWVFZIGJGTOl9kzJtPpzas8QdQbziNidmGF0lH0iXicusxu2JruFSFTMvhk7rzeAoNYMRbzLhwyKiJ1qvaFctMzkxBYiSX3XjD1FpDES1L+XMqoi7wxNIZNXkYiGsK4zYetzP1puz/xm97D+/SGIeqDPYYYIo8OnyRmj/W23NSMmJsHO603p40RMzchRAshxCknI2ki6k0IRiXijiBiBZvYKEUVR9BFeGffNsLbM8SuaLSctMo7ubsJRXQnkijQ9Q9QPqqphYJJ5RJwqIv4UIkzZtd+aEeMRydpVRDhmV2WpEJEXO0l3rFjhpojY3LwLGE9gkYpIJl/UjU12WzOAf3c/VnjtEDOqtjr6/PPKG3qf3TPK60gEITUj6SxyRRWKAnQ3OytE2vXWjD9m1YTN8V19amaG89SM0/FdjmZVSlaVEDvZ/0aOCCdFJGtv8y5g5I2Yt0ryhrUdFAVI2AwvAkxJihK2ZpwYVc1sWdsOAHhh3xinExGE3LCJme6mmKVx90oYNydee0Tst78Bw6TPuwVrTM1435phZlXaNSMhdnp2Ro6If4qIecJG1Ey+blSNhmy3LwCTMU2yxXeZfBE7+1MA7BlVzZy6ph3BgIIDY9P6JAFBLGb0ZXcOjaqAf9N0Uw4i3gGBUzN2I95FBJqRWVU+bOWIcF5650QRMZ9zWlB7hjnF7W7eZXRKuvju7f4kCqqGjsYIVjg03SWiIRy/vBkA8MI+as8Qi5++SXdGVcC/fTNOI96bRCkiNl5vALNHRIWmuVPAyawqMXYq1Cj3QDP7ikggoOjtGVGKSNJhqipD1lAzfdHdyhYoivM+6ZZ1HQCAF96l9gyx+NEzRBwaVQH/NvBOOTSr6lMzgpJVrVgBAKM1o2lAvui8EFFVDYVyK58KEclQVU038NhpzWS4Rbzbn5oBjBFeUYpIymGqKoO1ZmSLeX/NpT+EQT4Rop7on3QeZsbQp2ZqpDWj7/QSFWhm2SNifJyb9gx7nQPIrCod5h+OHbNqrqBC5WAUdWqkahAcaqaP7jpszfiVorgQTBFx6g9hnLqmHYoC7BtJYyiZ4XE0gpCWwy7i3RlMJU3nip7tmykUVf2F3/Y1NmJMSLptiZhxGmgGuDOsml/rSBGRDPMvhJUK1fzkMf9gneJeERGTXGqEmTlTRFripQJmknMqoRtSmTz2Dk8BcK+ItDSEcdyykk/keVJFiEVOv77wznkhkoiGEClP3HillLIbPcD+9l12s6dp/JacAuYlq9ZehhVF0QsHV4WI6XMjDiefeCPHKSSA/WCDAQUhCz8cc7HCo6p34hEBINwjwlozThWR1rK3ZHJankLkjcOT0LTSxZQFrrlhy1rmEyHDKrF4yRVUDJe9Xm6mZmbtm/GoPTNVvlGLBAOWl8wxGkw3nTwzm7I2c0QAc5aIi9aMPjETcOWP4wkVImWyNqvTUDCAYHmclUeV7GRqBjDHvPOr1M0YrRlnikhrvHTBSWUL+uy635iNqjzYso58IsTiZzCZgaaV5Hzm83AKa894NdZvGFXtj6uGggFdOZjmeMNnN1kV4JMlwgoRWfwhABUiOnqqqo0nRUwPNXP/5Jx2qYiIb804U0TMBQzvpVFOcRtkNpfNa0qFyJ6hKSmj7AmCB4dNEzNu76Q9V0Qcbt5liNjrZTfQzPyxbgoR2TbvAlSI6Njt1wGmmHcOWSLGQiZ7FTvzlAgPNHOoiISCAX3bpSw+kbf6kgD4KSJtjRFsXNoEAHiRVBFikcJjYobR6fEIb9rhxAyjQcA6DUetGQ6xEbLtmQGoENGxE2bGiHFMupsum6kaHCsiYgPNnHpEAMOwOiFBIVIoqjg0Xrqgbuiyt3G3GvoYL/lEiEVKH4eJGYbXoWZuCxERC0aNqRk7igiH1gwpIvJixLvbaM1wUkSKpgwTqzPljAbBG3gNRcR5IdIal8ew2j+ZQVHVEAkFsISDUZWhB5uRIkIsUvr0iRnnRlWGvm+mRlozMQEt8IyD1xwerRmzWVUW5DmJzzipTiOcPCLmcSo7Mh0gRjI043Z8FwBaG0oXnYkZ/7NEDoxNAwB62hoc7c6Zj81lRWTnQArjkmWmEAQPjD0z7hWRDo8VEadhZowGzi3wQtFYVGpPEXGvwhtmVXle/uU5ic+4UUTcTs2Yn9x2tyGKVkS4tmYkUERYIbKqPc7163Ymonqr58VeUkWIxUc/hz0zjHaPE5fTWedTMwD/1ow5kduWIsJec1yo8LJt3gWoENHRPSI2qlNWybqtktnyo5DFDBMzItzcjEJR1fNNnJpVgVLoFyCHWVVUIQKYfSJUiBCLj8Mc9swwvN5BNeUwuZoR4+zFc3rzybU1Q4WIfBg5It7OdJsf225bBjCZVQUoIkzOBFx6RBrkUUQOstaMiEJE94mQYZVYXKQyed0vxrM149XqB6aINLk1q3IqRMyTK3ZaxFxaM2RWlRdjE6J/iogTqUwf3xWgiLAws4Zw0NWTtlWimHeRhcjpZUXk9/1J7ivDCZ/Zvh1YtgxYssR4W7as9P46gLVlmmMhxz4LM2wD73SuKMzfZibtNkeEsxfPSYZI6ePd3/xmyawqL4ZHxP4oletCxIUiwlsyNJN0uXmXoZtVp/03cYpszXQ1x9DT3gBNA145MMH96xMes3070NEBKAqweTOmBwYQHRnBBSMjiI6MYHpgALjkEr9P6QnMqMrDHwIAjRHj5saLLJEUr0AzXh4RB6mqgDlHhEeyqjwv//KcxGecmVXd9+sAI5zGjhrDiJdzR0SYVd2mqjJkyRFJZfIYL7eHRCgiAHDq6pIqsoMMq7XLvn3A5s3Ali2YHhtDFEAgFEX7yvcgsPpE/Hb1iQisOQkda96LoRlvtsf6DcsQ4VWIKIqCTjY548EIL69AM143fEaYmV1FxH1rRsZkVfca2yLByfiuPjXjujVjvwhiiBzfdZuqypDFrHpwrHRX194Y4SIvV+KU1W146JXDeGn/uJCvTwhkaAi47jrg4YeRy2SwHkBfQzOaTv4ouk6+GMH4kUm87w8quP7pvfjC2WulusPkjZGq6t6oymhPRNA3mfHEJ8KrEOE1vmu0ZmwqIhx3zVAhIiFOFBEeDmbAvPzI/hND5Pguj9FdQJ5AswMC/SGM08p7Z149OIFCUbU9BUX4xL59wLnnorB/P84B8EJLN5pO+zhWbDofgXDpxbeQGoWaSZX2wQNQwlGgbTlue3QnHnm1D7d98gSc2NPq239BJGxihkeqKsPLEV5uu2Y4XWeNAQXvFZGcAxuCaKgQKeMo4p1TlcwUkZgbRURIa6a8eddla8YINMtD0zTfVk8fNIWZieKorgSaYyEkMwW83Z/CCZz22RCCSKeBH/8Y+OpXkRscRBOA6MkfxfIPfgFKsPS8z/a/g+QLD2J693OANvum48GXDuKrv/g93u5P4uM/eAafP2MNbv7wRkd+L5npL7dmVnBqzQAwtWbEe0TS5fFdt4Fm/Foz5ZtPu4oID49IUT6zKhUiZZyM0Eb1ZFU+gWZOPCIic0RSvMyqZUWkqGqYyhZcjQK74eC4OKMqIxBQcPLqNjy1axgv7R+jQkRm0mngllugfuc7+AqAb0TiaPnIdWg85iwAwEzva0g+9wAyB16v+Omjo6Nob2/HOccswdd+8Tb+85XD+MmzvZiYzuHbl73Xw/+IePommSLCsTXj4QjvFKdAM36tmbIqYVsR4deakamVKM9JfCbjQK4yklXdPTmzLhQRc+KfVpaMecHGd922ZmLhoP599TNLROTEjJlTV7cBAPlEZCadBm68Eep3voMPAPiH7vVYcsV30HjMWdCKeYz9910YeuAWvQhpBDCzdCm0aBRaQwO0chEClEZRv3npSfj/PncqAgrw8Kt9+Pnrff793zijaZo+vsuzNcNGeEcEm1U1TdO3m8tiVtU9ibY9IpQjsqhxMrkS5bT0zo1HhBVD5sV5vOCxZ4Yhg2HVq0LkFH1yZpx7cUhwgCkhP/gBPgDg1U3nY+lnbke4bRkKk4MYuPcmpHb8DADQBCD7sY9hamYGsf5+IJMBpqeBchFi5rzjunHNuRsAALc89CYGyi/etc74dF6/i+5u4bco0gg1E9uamc4Vma0HCYfqLtuKzjvQzPb4LiWrLm6cqBJ6a8alIuLUQQ0YiggAZHJ8CxGjNeO+leJ3qJmqajhUnpoRaVYFgJN6WhEKKBhIZnSTHyEJpnbM+wG8ceZl6PjwdVBCYUzvfg79P/7fyPXvRhjAl1etwsTMDCKPPALErLUkrv3QUThhRQsmZ/L4ywdfg6rWfiHKJmY6ExFH16j50GPeBbdm2MRMQDGUDbvw9uI5DjTjsGuGChGJcZas6v5JATifKQdKfb5QOSJ4Os9vRTVgbs24V0SMUDN/CpGhVBa5oopgQOHa565EQySI9yxvBgDsoPaMPAwNAZ/8ZKkIUQJ4+/w/Rev7PgMAmHj2fgw/9DWo2TR6gkFMP/YYvrZvHwIWCxBGOBjAty49CdFQAL99ZwT/8lyvgP+ItzBlZynn35t2j3JEdH9IJOTYKM874t3wiPjYmiGPiHw4CjTjrIg4ddqLMqzqrRkOiogRauZPuipry6xobfBkpJa1Z17qpUJECoaGgI9+FIXHH8eaYAi7PnYjmk7+KDRNxdgTP8Tkb+8BAPQ0NKB3ehqhCy8EAs6eJxu6EvjyR44FAGx7dCfeGUxx+2/4AfOHLG3mO23WUR7fFW1WdTu6CxjXZm7juwVn7XhqzSxyHEW8c1JE9OrY4RND1AhvSh/f5aGI+Lv4zsgQETe6a+bUNWRYlQZWhGzfjo5IA3KX/A0aN54NrZDHyH99A6mXfw4A6IlG8e7EBAKRiOuH/NwZq/H+o5cgW1Dx5//2ak23aAZ0oypfRYS1ZmbyRUzn+Kq5ZtxOzAD8b/YyDpasmj/eTSGSJ0VEXhzliHBSRPTHXsyKSLkQSfrkETnokVGVwSZndg0kda8N4QOmIqQzlkD80q8itvpEqNlpDP37rZje+TsAwNmdnehNJhHiUIQApQjzf/ijTWiKhvDm4SSe2j3E5ev6Qb+g1kw8YkzTiWzP6BkiLq5j8fK1OVdUUeAwFOBYEQm7b81kSRGRF6Zq2Pnh8FZEHLdmBCgimqaZIt75mVX9UkREbt2tBFuAp9ICPP8wFyENzYhf9nVElx+D4kwSgz/dqo/mnr1iBZ7u7+eihJjpbo7hss09AIAfP9PL9Wt7yUCSf4YIUN43kxCfrmrEu7tXRAA+11mn13y9NUNm1cWJk0VArJp1v33XmYOawTv1j32tYllO5jK+G2fpqv56RHravClEAGMBHrVnfMBchCTaEf/0bYh0r0NxahyD921FbnAvGgHccvrpeLq3F4GQmGzHz56+BooC/PadEewdnhLyGKIRZVYFzKFm4kZ4zWZVp0RDATCfK5dCxIECX/p4DoFmRQo0kxb2w7HlESk/KXJud804nCln8E79A4y2TCigOB55MyOLR8Sr1gxQWoAHADv20yZeTzEVIUual6Dx07ch0rkKheQwBu67CfmR/WgOBDDx+OP46jPPCCtCAGBVRxwf2tgFAPiXZ3uFPY4oRIWZMZhPRGSo2ZTLhXdASb3huWDUSZI3sHi378pzEp/R5aqg90vvnGz+NcM79Q+YvXmXx24YP3NEMvkihlKlOy4vCxFmWH3lwASXvjJhAXMR0rYcjZf/PcJty5GfGMDgfTejMN6H5kAAo+k0Quef73gyxg6fP3MNAODBHYdqzi+Uyhb068rSZpGKiPjWjJupGWB2irVb3HtEVMdhifrSO1JE5MNJ34wVIrmi6soVn3EwOmyGd+ofYNq863LhHcPPZNVD5R0zTdGQXhB5wdFdTWiKhTCdK2LnQG2PcNYEpiKka8kaJD799wg1dyE/ehCD992EwuSgUYTYzAdxw9kbOrF+SSPSuSL+Y8chzx6XB6wt0xoPz/JJ8KLDg0JEV0Rc5iHFON7wOQ2xZB+vaUC+6K4QIUVEMlRVQ6FcSISD1u/+zVMubuLV3US8A0BD+fN4mlWTnBbeMfwMNGNtmZXtcU83/wYCCk5eVR7j7aX2jFDMRciyo9H4x9sQTLQhN7gXA/fdjGJq1JciBCjJ+leUVZF/eW5/TY3yGhkiYr5nxr4ZcR6RNIfWDGAozxkerRmHIZZm64DT9gwVIpJiLiKcKCKAOxezW48Iz94lQ88Q4bQplwWazeSLXL0sVjhYjnZf5VGGiJnNa0uG1RepEBFDOg1873vAli0obN+O7p4TkLj0qwg2NCFz+G0M/PTLUKcnfStCGJ88eSWaoiG8O5LGb94Z9uUMThgQsHXXjDetmdL1ptGlosOzNeN26R3g3BJAZlVJcVqIhAIKyunqrsxD7qdmyq0ZnorIDL8MEaDUFmHfK6+zRPwwqjJOX1cqRJ5/d4wW4PGG7Y259lr8VW8vlmzYjMZL/g8C0Thmel/F0AN/DS2b9r0IAUr+hD86dSUA4O4aMq0aGSJiivjOhPiY9xQnjwjf1gyLeLd3zVcURX+NclyIkCIiJ+apFztpc4qicBmn4qWI8DSrJk1mVR4EAoruE5nwqRDxKkPEzAkrWtEQDmIsncM7Q7U5viklpuV1HwDwnRMvRPMnbkEgHMX0O89j6MG/gZbPSFGEMD53xhoAwFO7h9E7kvb3MBYRlarKaPcg5p1Xa0aEWdWJL9DIEnHYmqFkVTkxJmYCtj0EPJLunMp0DJHjuzzCzBh+GVa9DjMzEwkF9OmZ5/aOev74ixLz8joAb5z9aXRcdC2UQBCp1/4vhh/6OlDMS1WEAMDazkacc8wSaBpw7wv7/T6OJUSlqjKYWXVUYI5ImpNZlWeCtZsQS7c3vzkH60xEI89JfIT9YOwYVRnsh5lx6BHRNM3R5l8zMT3QjN++Bl6/vGb0UDMPDauapnke7z6X09d1AACef5cKEdfs2wecfnppeZ0SwNsXXYvWsz4NAJh45j6MPXYHoKnSFSGMy05bBQB49M2BmmjViVZEmHcsk1eFecd4LL0DgIYwvxa4m3a8m9iIQlEF80pTa0Yy3AS8uK1OC6qmPzEcKyJ6xDu/rAp9P4OLWOS5GKFm3qWrjqVzSJfvYFa0em9WBQyfyAv7xmpqYkI6hoaAiy5Cbt8+JCINyHzyr9B04oXQ1CJGH/seJn93HwCgJxyWsggBgPcf3YloKIBD4zPYVQNbefsFm1WboiEEy+YxUd4xblMzkdLrg9sWuKZprtrxugrvoCAy+yHJrCoZbpYAuU26M98FOFVEDMmQnyLCpmbc3kWY8SPUjPlDljbHHHtw3LJpJflEXGGejNm9G60t3Wj/zD8gvmEz1HwWww99HVOvPQagvLxuakrKIgQA4pEQzt7QCQB44q1Bn09TnXS2oHvFugWN7yqKeO8YP0WETwvcfNPqJLLBzc1vvmDcCJEiIhk5N4pI2J2D2dzScbtrhufUDK+7CDN+xLz7OTHDCAfNPpER385Rk6TTwI036pMxnSvfg47PfRORJWtQSI1i8KdbMbPnBQDiltfx5vzjugEAT7wtdyEykCy1ZRLREFev2FxEXheyhaIe/JVwsWsG4BccmZ11zXdhVnXwmpMtls6uKKWpT1mgQgRmj4iL6tRhWyRrWn7kNGxLRI5IOse/EPHDrHp4oiQtr2zzpy3DMHwilCdiGWZK/cEPSpMxmy5Ay2VfQzDegmz/Oxj4lxuQ69+NMMQvr+PJB4/tgqIArx+a1D0YMiJy2Z2ZZoHXBdZiBoBGl21mXtOJ7JofUNz5Ep2o8G4GM0Qi/2+tB+RdjDO5b824G90FxBQivORMM8YGXu8KEd1s1+qvVM8KkRf2jUJVNQQkuhuRkn37gA99CIV9+7AuGMLUuV9ExykfBQCk3/4NRn/5HWiFLCIAUo89hohHe2N40NUUw0k9rXjlwAT+++1BfOb01X4fqSL9go2qDNayFeEdY8puLBxAyKUngtd0ovma76QYYIneTm5+ZcwQAUgRAeBunMnt4ju3YWYA3/l2htjWjHdmVdER1VbZtLIF8UgQ49N57B6S36ToG0NDwGc/C5x4InL79qGpqRP5T9+G5nIRMvHbezDyX9+AVsiiWVGQTqcRufDCmilCGHp75vfytmdYqqro351WgYoIj827DEMRcefFyxTcXfPdvObImCECUCECwF2V6HZqxm2YmflzeQaaTS0Ss+pgUmwypFVKPpFyyirliVRmaAj48IdRuOcevC+VQtuak9B5xXcQXb4RxZkUhv79/2Dy2fsBoDSeOz2NUNw/748bLigXIs/tHdVfLGXDO0VE3Fg/T2WXlxcv61IFd6PCM7MqKSISwsWs6vDJmeWoiGQL7rYAM1RV00de3fZVzRgSrHeFiFcXUyuwMd7nKE9kNuk08I//CLznPSi8/DI6lADeOPMydH7qb0t+kIE9GLj7esy8+xIAucdzrbJ+SQJrOuLIFVX8Zrecu2e8KuKNqRn+SqleiLg0qgL8WuBMEXFeiDi/+c2VzapUiEhI1pVZ1WVrxuWTEsCs9dw82jPTpq/RFK3dZNVcQdW3eoo23FnB8IlQnogOm4r5i7/ALSMjaEy0I/apv0Pr+z4DRQkg9eqjGLjnL1GYHCyZUo87TurxXKsoiiJ9e8arIt64LvBXhngGM/Jqgbttxxs3vw6mZgp12pr5/ve/jzVr1iAWi2HLli148cUXRT+kbdyZVV22ZnSZzvmPwhyExqMQYb+8AcXduebS0lCSYJOZPIoevBAPpTLQtNLPtT3u/0jnCStKPpGJ6XxNhFkJp5ySyqZivrnuFHRdeQca1pwINZfByC++ibH/+32gmEcEwNRjj+Grb7wh/XiuVc4/bikA4MmdQ/o1SCa8mprxwqzKwyMS49yaifrQmqlLs+oDDzyAG264AbfeeitefvllnHjiibjwwgsxNDQk8mFt484j4nJqxsXyI0YgoOgFA4/JGXNfleeIF7vz0TQglRGvirALaXdLVIoplVk+kXpuz5gNqW++ibZACG+e+z/RfcnfIBhvQW7wXfTffR3Sbz4JAFgZDNasKbUap6xuQ1s8jMmZPF7qHff7OLPI5IsYLS+i82pqRoxZlbWYJWzNODarumjNuFD/RSL0NN/85jdx1VVX4corr8Rxxx2HH/7wh4jH4/jnf/5nkQ9rG1eFiAuZDDCPcrn7Uei/JBwUEWZU5TkxA5S+v43luwovfCK6tNzsr1HVzBnl9kzdLsCbY0htbluO+Gf+AS2bPwkASL70X+j/1y+hMHa41Io59ljsr2FTajWCAQUf3Chne2YoWWppxsIB/QZCFEwpFWJW1a9l7r1ucU5L7zI+KiIs3K1uFJFcLocdO3bgvPPOMx4sEMB5552H5557ruLnZLNZJJPJWW9e4G58121rhi28c/eLEueU+geIGd1ltHqYJeKVtGyHut07w2LaTzutZEgF8MoJ56Hriu8guuyo0lTMf/wdxn/1T7NbMW++uWhaMZUwUlblWoJn7JhpEB58JbQ1k+NvVp3OF139rLJuFREXN7/MrCrT5l1AYCEyMjKCYrGI7u7uWe/v7u7GwMBAxc/Ztm0bWlpa9Leenh5Rx5sFm5pxZ1Z12prhs5KZKSo8RnhFhJkxRKYozkWmiRnG8StakIiGMDmTx2uHJvw+jjeYY9oPHEBrtBGRj92Izo9cj0CkAZn9r6P/x9foUe09i7QVUwm2BO/g2Az2SLSHiMW7e5G/wxSXVLbA3Ts2xdGsyoYCNM35jSfgPsSSR2um7syqdti6dSsmJyf1t4MHD3ryuG5CXngFmrldyMYUER6rtEXEuzO8DDUbSJYDmSQqRMLBAD5wzBIAwH9LvmuEC0NDwMc+phtSb19xLFqvvAONx74fWrGA8ad+gsEH/grF1CjCAL587LHoXaStmErEIyGctka+sW4vi3iR3jGe6m6D6RrtRnl2PTXDwaxaNx6Rzs5OBINBDA7OvtgODg5i6dKlFT8nGo2iubl51psXuPOIuN01U66OXZhVAX57EACzwYv/tlovQ81kVEQA4Pxj5fQFcKc8FVN48kmsUQJ4/fRL0P3p2xBq6UJ+vA8D996I5AsPApqKlcEgph97DF9b5K2YSrB2nUwGZi/bmuFgQC8UePtE0hzV3VAwoN+suvHiuQ2xdJesWmcekUgkglNOOQW/+tWv9Pepqopf/epXOOOMM0Q9rCN8nZrRPSIuzaocY95FpKoyvAw1My6m8phVAeDcY7oQDCjYPTiF/aNpv4/Dn7kx7Yl2ZD/1d2j7wOehBIKYeuvX6P/Jdcj17wYAnN3RUTKk1kErphLmhYiy+EQMj4g3RbwRasb3upDifC3TpxPdFCKuc0Ro14wtbrjhBvzoRz/C3Xffjbfffht/9md/hnQ6jSuvvFLkw9rGz4h3vV/ISRGZcbkHATDuIpoEFCIiHfJmCkUVQ6mS8182RaQlHsaWtaW74EWnisyNaV97Mjqv+O6sbJDRn/8jtNwM4ihvzR0YqDsVxMymla1oCAcxls5h96AcPhGvi3hRhlWjzcxH3W3gMDnjth2/GHNEhG7fvfTSSzE8PIyvfOUrGBgYwEknnYTHHnvsCAOr3/jpEdEd1DIpIgLNql6lq45M5VBUNQQDCjoTUaGP5YTzju3Gs3tH8cTvB/HF963z+zh8MG3M7VACUM66DJ1nXgZFCSA3+C6G/+vvURg7DKC8K+axxxD60IfqUgUxEwkFcOqaNvz2nRE8/+4ojlna5PeRPG9riroupMtt5gSnhOiSFy/LqTXjx9K7csR7vXhEGNdccw3279+PbDaLF154AVu2bBH9kLbJ+7prhs/UjFGpu09o5NlXnYvhERFrVmXScndTFEEJwszmwsY2t/eOYTzt3TZiIbDR3DPOQG7fPiQamhG75P+g9axPl2LaX/mlng0SBHBWY2NpV8z559d9EcIw2jP++0TyRRXDHq9GEOUdM26q+CgiPBaMGmZVH6dmJFNE5DqNT7gZaWJPipxrRYSTWTXPoTXjydSMWEVExgwRMz3tcWxc2gRVK0V81yxDQ8AnP1kazR0cROuyo9F5xbfRsPZkqPkMRn7+jxh7/Ad6Nsj03/89fjcxUfO7YngjU77MUCrr+WoEUS1b3plIPELN3IZYss9zFWgmmSIitDVTK2S5mFXdJqu6Hd8tfX6Gw9QMb4OXmZa4GFPaXFgOwjLJjKpmLjiuGzsHUnji94P4w1NW+n0c+zA/yMsvYx2AsRPOw5ILr4YSDCM/dhjDD30d+ZH9AEox7fuSyboZy7XLCSsMn8g7Q1O+tmcGmJro4WoEESb2oqrpygWva5mRYO38ho8VEM6TVZ2bVd281olErtP4BJeId7dTM64DzfiN7wpNVvXIrCq7IgIA55XbM795Z5hL/ounDA0BH/1oKSVVCWDq3C+g8yPXQwmGMb3rWfTf/efIj+xf9DHtvGA+EQB4bu+Ir2fxYzWCrpRybNmmTcZ9XtcyHi3wjMt2vPGaQ62ZRYW7ZFVOOSKcFBE+23eZwUucIpKcyQsdVZQ1Q8TMCStasLQ5hulcsXZ2zzA/yJYtKGzfjo5IHLE//AqaN38CADDxu/sw/PA2aLnpuolp54V5jNdPjGWR3v3uMLNqkqNSym6oQgGFW6S5kdfkXBHJuGzHu5qacTGYIRK5TuMTrsyqnJJVXeeIcNoMCfA3eJlhdz65osqlaJoP/WLqQUS1UxRFwXnHdQEAHq+FMd50GrjllpIfpLcXba1L0fjZ29Gw/lSo+QyGH7kNk8/cB0Crq5h2XrBC5IV9o776RA6OTQMAVrZ5qIgIaM2kBWwRZ9dZNwpmlmPEu92buVy5eAmTIiIf+tI7F+O7uaLq6OLBqmOnDmoGz/FdkWbVeCSIcLB0URDZnulPehvI5JTzjyulDP/324O+mxSrwvbFfOc7+ACAf1h2NFo/+4+IdK5CITWKwXtvwvTO3wEAzu7srKuYdl5sWtmChnAQ49N57B5K+XaOg+Ol351V7d79/HSzKkdFZEqAssvjOmtc8921ZjTNMJ9ahX28k9c6kch1Gp/gEfEOGLKXHbIuHdQMw0TlrhDRNM1Ync1hUdRcFEURHmqmqhoGJ70dP3TK6evakYiGMJzK4vXDk34fpzJMCSnvi9mx7lR0X/Z1BOMtyPa/g4F/+XPkBveiEeWAsv5+asU4IBw0fCLP+9iqO1BWRHravCtERCgiRkI0P2WXFSJuvHjuFRHjtcJue4Y8IhLjZhHQrCeFA58Ir6V3PBL/gJLcVyjfmYuYmgFMFx1BWSJj0znkiioUBehqkrsQiYaC+MDRpSV4j79VeSu1r7AipKyEvLrpfCz5w79GIBLDzLs7MPjTrShOjaE5EMDE44/jq888g0CIhvGcwtozfi3A0zRNb814qYiY84V4ecdEBDPGebRmXIZYmv0ddi0BVIhIjJtFQKGAAjbh5sQ8lCm4c1AzeJlVWV8VABojYl5Q9BRFQYoI84d0JqLS/cJV4sMnlNozD+445DiPRgimIuT9AN448zJ0fPi60r6YN/4bQ//xt9DyGfSEwxRQxokz1jOfiD95IsOpLLIFFcGAgmWt3ptV80WNm3dMxPQfD0XEmJpxdvOpKIpjb2KWzKrywgw8Tl60Sk8KZ0l3mqbpLzxuFRFe47tsYqYhHBSWSNreWJLtRwUlitbCxIyZC45biiVNUQylsnj0zX6/j1NiVhGi4Pfn/Qla3/cZAMDksw9g9JffBtQiehoa0Ds1RQFlnDhhRQvikSAmpvPYNei9T4S1ZZa1xDxdFd8QDuovjrzaM6lM6es0x/jEuwN8lGceaz30QsRm0aar/5LdoMl1Gp9wO9LkNEvEXLi4H98tVf1uA81E7plhdJQLkTFBhQgLZFoq8cSMmUgogMu3rAIA3P1sr7+HAWYXIUoAOz9yHZpP+R/QNBWjj9+Jid/+KwCgJxrFuxMT5AfhSMknUkpZ9SPu/YAPbRmg7B3j7BNJlj0izQ0cFRGXXryiqumGUTeLTvUNvDZvfvOkiMhLzmV7hH1exqZHxNxndL1rRo94L7rqsbJCpEmAUZXRkSgrIuV9FrypNUUEAD69ZRXCQQUvH5jAG4d8NK2ai5BgCLv+4CYkTjgPmlrE6M+/ialXfgGgPBmTTCJERQh3WNx7PRUiAP9QM5ZJ0sRREXEb8T7rms9DESGPyOKgUFTBWrFOpUinrRn28cGA4loGZZKhueJ2QlpghgijvbG0DVdUa8brFeY86GqK4SMnLAMA/MQvVYTtjfnOd/C+UBTvfPKv0XjMWdAKeQw/vA3p3z8FADh7xQqajBHIlrWlQuSl3nGhoX+VODhWUhN7/ChE4ny9Y2xVRTPHm6qYS0VklgruRhFx2ZrhFfDGC7lO4wPmF22nVaLTpDt9YobDk6LB1NpxY/bSWzOCjKoA0KkrIuQRMXPFmWsAAD97rQ8jgtSieWGR7Y8/jjWRBuz91N+gYd0pUHMZDP3H32DmnecBlIuQ3l6ajBHI8StaEAkFMJrOoXd02tPHZhMzfhQiuomdU5ZIknlEGjh6RFwGR7JrfiQYcLXHx+nNb85FeKdI5DqND5inFBwXIg6z/3X3tEt/CACEg4puLnVjpBK5Z4bRLtojkpR/z0wl3ruqDSeubEGuqOL+Fw9498CsCNm+HZ2xBPKXfhWxnuOhZtMY+re/Rqb3VQBUhHhFNBTEphUtAICXer2Ne/ezNcM71IwVIjzbzMyL5/Rmj9duMaevOW6iKkQi12l8IFssPTEUpTSK6wSn+2Z09zSH6lRRFH3GnYsi4kEhIqI1o2ka+idrI1W1Ep8vqyL3PH9AN5YJxVyENDQjftnXEV1+DIozSQz+9MvIHn4bABUhXnNKOdhsx/5xzx4zky9iMFUq4ns8jHdn8A41M1ozPKdmStdqp9OJrHBwe/PpVIUnRURSdPNOMOB4H4Hz1gyf0V1GTJ9xd76QiRUiIlJVGZ2JkkdkfDrHPSshOVPQv68y75mZj4s3LUNnIoKBZAaPvyV4/4y5CEm0I/7p2xDpXofi1DgG79uK3OBeAFSE+MGpq8s+EQ8LkcMTM9A0oDES1G8WvKS1wQg14wEzq3JtzXBSRNwmabObXzsDEua4CJqakQwePxinDma9X8ipOmWObjepf160ZtripYtcUdW49YMZbMdMe2OEW4HnJdFQEH+82YNRXlMRsqR5CRo/fVtpb0xyGAP33YT8yH4jsp2KEM85ZXVJEdkzNIVxQS3MuRww+UN4LYmzgyhFhGdrhnlEcgUVRSe7xfJ8zKJObn55+CFFIddpfCDvIlWV4XZqhtcLZgOHUDO2KEqkWTUSCuhO9tE0X1MmM6rWSoZIJS7fshqhgIIXe8fwzJ4R/g9gLkI6V6PxM7cj3LYc+YkBDNx7EwrjfRTZ7jPtjRGsW9IIwLv2zCEf/SGAoVzwuDnRNM0wqwoY3wWcqSJGmJnL1kzYviJi3oVGiohk8Jir1o1DNp+YvGQ6Bo/UPy/GdwGgo9ye4T05M1CjEzNmlrbEcNnmHgDAX/z7a/xUo3Qa+N73gC1bUNi+HV0rjkPi8r9HqKkDueH9GLz3JhSTQ2gOBCiyXQJOLasiXrVnDvg4MQMArXF+yzAzeVW/yeTZmjErGU6us7za8TEHikiWY24Vb+Q6jQ/kiu7bI25bM053DsyFxwbetAeBZoCRrsrbsKorIjVciADA1g8fizUdcfRPZnDrI2+6/4IsqOzaa/FXvb1YsmEzEpf+HYKxBDKH3sLgvTeiODVqFCEU2e47zCeyY783kzN+TswAZo+I+0KExbsHlJLnhReKorga4WWFg9tCIOZAEcmabAhuRodFUPeFSJaLR8Rta4aTIuJyxh0AUh5MzQDiJmcGanhixkxjNIRvXnoSAgrw8Kt9+Nlrfc6/2L59wOmn6xt0v7PpfDR/4hYEwlFMv/MChh74CtRsmooQyWCTM68dmnS0UNMuB8phZr4VIrpHxP01wRjdDXP3u7hZMJrlpIg48YjoN76cXm94It+JPIbHXLXrQDNeHhEOG3jTHhUiLOZ9jHNrpr8GU1Xn4+RVbbjm3A0AgFseekNvO1mGtWLOOAO5N99EWzCM319wtbFB9/XHMfzQ16AVslSESMi6zka0N0aQK6h483BS6GNpmqZ7RHra/fndaS3niKRzRdej65Mz4pRdY8Go/enEDIeFd+Yz2ImM0EeHOSnwPKn7QoSLWVX3iDhTRHj163iYVb2YmgGADj3mna9ZdWARmFXNXPuho7BpZQuSmQL+8sHXrI8779sHnHZaqRUzOIjmtuVo/Ow/oum9H4amqZj43X0YffS7gKZSESIpiqLg5FUsT0Rse2ZiOq+roSvb/FFEmmIhMPHCbXtGxOZdhhtFhFc73thvZsMjImm8O0CFCB+zqtPWDGdFhMf4rhdTM4DI1szi8IgwwsEAvnXpSYiFA/jtOyP45hO7q48N7tsHbN4MHHUUcm+/jTYA39p4Nro+/+1SRsj0JIb+7VZMPnMfAKAnHKYiRGJOLbdnXuoVa1hl/pDu5qhvY++BgKIXDm4NqyI27zLcDAUYrRlOioiN15ws5+EInsh3Io9hZlU3VaLj1gzn8V0j0KwGzKoCNvAmM8ZdXa17RMysX5LALR85FgDwvV/vwSd+8Aze6itv6O3uLsUCs7d165Dbvh09xSKalh+D6CduwZI/uBmBaByZg2+i/8f/G5neVwCUN+hOTVERIjGnmRJWRS7AOzhebsv4pIYw9MV3LkPNUhn+m3cZboYCMgU+iggrJuzcdGYkbs3UfUCADIFmvKSyeNhd6l+hqOqfK9wjUm7N8Nw30z9RUkNaGsLCz+81nzl9NULBAL7+y7fx+qFJfOx7z+CL71uL68cm8e8APq9/pIKG9aeiecsfYlnP8QAATVORfP5BTPz2HkBTEQbwl1u24O9+9zvKCJGcuQvw1nY2CnkcvydmGK0NYewHB0Vkhn+8O6PBxQ2fsV+MT7KqE0VERrNq3V+FuJhVHRiHzB/Pz6xa+j84nZpJmz5PfI4I/w28feWJmeWttW9UnYuiKPjjzavwoY1d+Juf/R6/eKMfdz39Lh644acYnhjFckWBoihQQhEEG0t30Voxj/RbTyH54n8iP3oQANCsKBh99FHKCKkR2AK8l/aP46XeMWGFiJ9bd820cMoSMTbvCmjNhJ23wI39Ym4DzRaXR4QKES7Jqk5bM3wVEbfju6wtEw4qwuU7liPC9s3wmGvvmygXIouoLTOXruYYvn/5yfjE7wfxlUfeRN8kEG5bNutj1Ow0Uq8+itRL/4Xi1Kj+/jPjcTw9OkqtmBrjlDVteGn/OHbsH8clp/YIeYyD5dFd3wsRTlkiQlszLjwi3ALNHBRDvKc0eUKFiI9mVWOum5ci4q4149XoLgC0lQsRVSut/eaxZIu1ZhajIjKX847rxtlHdeLt/iROP+NMABqgld7yY4eg5Wb0jw0DSH7ta4jdeCNArZia49TV7bgL7wpNWJWpNQOUrgluMFoz4hQRZ60ZPjefTuwAMisi8p3IY/gUIs48InqgmSSKSMqj0V2g1Apjdz+8DKtMEVnWWh93/LFwEO9d1YYfbduKXN8u5Pp3Izfwzqwi5OaODmTSacS+/GUqQmoU8wI8HmFfcykUVRye8DfMjKGbVV3+P0XsmWG4mU7ktV/MiSJCOSISo0e8u/KIuNs1w0sRcTPfDniXIcLgHfPOPCIr6kARMfO5z30O2t13QwNKb+vXQzvjDGiPPYZtQ0MIxP19cSHcYV6AJ2KMt38yg6KqIRIKoKspyv3r26GFkyKSEjm+y0ERcTtCa+SI2FFEaHxXWnhu383Zbs3wVUTcJP4B3rZmAP6G1b4JtvCuvgoRAMDnPqe3ZrBnD/Dss8CFF5IhdZGwZW1p78wL+0YX+Ej7MKPqyrYG33eQ8Fp8l5wRp4i4aYGzAQX347v27QAZTo8tgrq/Svk5vstLpmMYkqGzeGQ9zMyjQoT5QsY4pKuqqqaHmS2vk9YMUT9sWdsBAHhhH/+EVVn8IQBPsyrLQxKRI+J8OpGXKmFEvNtpzfAdjuCJfCfymCwPj0jY2dRMlnOOiDHf7k4RSQge3WV0JEoy8AgHRWQknUWuqEJRgO5FEu9OEIwt60qKyJuHJ/WJEF7IVIgYgWbyju/GXSgivKZmHJlVOcdF8KTuCxGuUzM2lQjuS+9cJP4BwJRPHhEeoWZsYqa7KeYqE4YgZGRZSwNWtcehauA+PXNwvDy663OqKmCamnFhVs0XVd2/IaI1E3MzvstJlWCvGbmiWn3lgwlSRCQmV+S5fdff1kyDqTVjeTmaiSmPPSLtPAuRyfqamCHqD90n8i7f9swBScLMAKDFpIg4uYYBwFTGUIQTIsd3XXhEeCkigHUlPssp1VUE8p3IY/Icx3dzRXsFAO+I9wbTkztjs00E+DA1o7dm3HtEDtdRhghRn2xZx3wifA2rByVqzTCPiKoZcQJ2YW2ZeCQoRB3VvXgSKCKAdSWexnclhikiUQ4R7+avZwXeS+/MhYgT2dBrRYRva2bxp6oS9Q1TRN44NOnYBzaX8XRO//1b1eF/IRINBfXrWNKhT0TknhnANJ2Yt/8zYC2juMtrbDCgIBwsTThZvenkNTosAvlO5DE8A80A69VpwdTb4/XECAQU/SxOZty9V0T45Ygs5j0zBAGUWicrWhtQUDXs4OQTee3QBABgXWejZ7/3C8EMq05HeI14dzH/Hz2vKWevFZ8vqvrrTWPE/c2nXW8iKSISw6MQCQUUsPF7q/26jMlPwvOJ4Sb1z2uzartp34xVw9V81HWGCFE38PaJvH5oEgCwaWULl6/HAyPUzNkNijExI0YRMRKs7Ski5ptDNnnjBnYDa1URIbOqxGQ5mFUVRbG9b8ZcKPB8YrhJ/fM8R6QcXqRppWLEDSzevd5SVYn6go3x8vKJvF5WRDatbOXy9XjgVhFJ6hkighWRfBGaZv0GirXTwkHF1Y0vg73mWM2N4j0cwZO6L0R4mFUB+1ki5vwSnmmGDS5i3o1kVW+eqKFgQL/ouPGJ5AoqhsuGV5qaIRYzLNjstYOTjlRPM5qm4bWyInJijzyKSGtDOV3VsUdEXKoqYIzvqpo9T2C6fKPHQw0B7K8W4T0cwRP5TuQx7InkJlkVsJ/9L+pJ4WZFtdceEcBoz7iJeR9MZqBppaKug8MWX4KQldUdcXQ3R5Erqnj5gDufyEAyg+FUFsGAguOWyVOI6OmqDlXSpMA9M4DzoQCmiPDwhwBAjCkiFlV43SNCZlX54OERAWC7NSMq5c5da8b7QqSzsTTCO+oi5r3PNDGjKP7uyiAIkSiKYsS9u/SJvHawpIYc3d2k38DIQHuCTdO5NauKUUTCwYA+sWJHedYVEU7XV7uKCK89NyKgQkR3EvNRRKybVcWMUrEiIm1zBl/TNF8VETetGTYxQ0ZVoh7g5RNhEzMnSmRUBcxbuZ3dnIge3wXMC0ZrRxER9ZrDA/lO5DE8klUBs0fEbmuGb3XK7gKSNvdRzOSLYIMrXplVAWOE182+mT4KMyPqCKaIvHJgwvZ+KzMyGlUB91u5Re6ZYcQdtMDTOb4eEX1qhhSR2oebWdXhTDfv6pQ5xZMZe4oIa8soivFL5gUdHDbw6q0ZMqoSdcD6JY3oTESRLah6e8UuqqpJOboLAB2N7hKXRbdmAGd7vaY5DwPYsQNomkbjuzLDxnfdFyI2p2ZYyh3n6pTNztvd0Mn6l42RkKc+Cxbz7sas2j9JighRP5R8IixPxFl7pnc0jVSmgGgogGOWNvE8nmvcBh0arRlxikgD28ArgSJixSNSUDVd8Y7S+K5caJpmmFU5Tc1Yb82IcTAzRSRlUxHxwx8CmKZm3HhEJphHhBQRoj5gPpHnHfpEmBpy3PJm6bZVd5ZvTsbTOUeL71JZLxSR0vesVhQRUblVvJDvRB6SLxpPcs+nZgpiFBHdI2JzBp8VLl5liDB47JuhMDOi3jhzfScA4MV9Y5h0EPxlGFVbOZ6KD23loMOCqtn2ugGGItIi0CPiJCbBT49IdlaSt3wv+/KdyEPMYTSuFRHb4TJixneba0wRMVozzvrBU9mC7odZRoUIUSds6ErgmO4m5IsaHv/9gO3Pl9UfApRuCtl1zK6JXdM0vS0tcmqmIVxuzdhRRHhPzYStr/MwB2jKGHFQ14VI3lQl8vOI+Btoxn75mDxplXTO2827DNaamZjJo2AjpZDBtu42x0LSLO0iCC+4eNMyAMAv3ui39XmFooq3+lgh0sr7WFzodHiDks4Z039CWzMR++O73HNEbLzm6Jt3JVRDgDovRJgiEgwoCLqMWbffmmEeEd6tmfLUzIyzqRmvX8zb4mEoCts3Y1+GPTxBW3eJ+uQjJ5QKkd+9M4IJGymkuwenkMmraIqGsK6zUdTxXOHUsMpa0uGgIjQvI+5g8R1vRSRqRxHJi3m94UV9FyKcjKqAg0CzvJhwGedTM/4UIqFgAK0NzvfN0MQMUa9s6Epg49ImFFQNj781aPnzWH7I8StauO654kmHnrhs75rAWtLNsbDQFkS87KVL++gRsbNWRObRXaDOC5EspwwRwOwRsTk1w92sanhE7GyGnMr405oB3PlEaGKGqGc+Wm7P/NxGe4Ytutsk0aK7uRihZvauCUk9Q0TsdaypfJ2csuHF4z01wzwiVm5+RXkSeVHXhUieU6oq4GJqhvv4bkldKKiaLSPVFMsR8aEQcTPCS6mqRD3D2jPP7BnBuMXfH6aInCSpPwQwxbzbNKvqm3cbxPlDACBRLnSmbKzS4D81w1ozpIjUNLz2zJi/hvXWjJgKtTESBFNb7UzOGK0Z7ytm46LjXBGhVFWiHlm3JIHjljWjqGr4v28tPD2TyRexcyAFANjU0yr4dM7RVVKbicvsmidaEUlEWQvcgUeEW46I9decLMfXOhHIeSqPyHFKVQUcTM0IqlAVRdFVETs+kamcPx4RwJBhnXlE2OZdUkSI+sTO9MxbfUkUVQ2diQiWS9zOdLqDKunB6C5gVkSsX2P1qRlfFBFqzUgLV7Mq69dZ3TUjsGfH7gYmbUzOpLP+eUTaHRrTNE1DH5lViTrn4nJ75tm9owuqiuZFdzLmSTCYWdXuzYnemhFciOgeERutGWNqxvtAM1FxEbyQ81QewQqRcMj9L6TtXTMCVzI3O1FEMv4pIp0Ot22OpnPIFVQoCtDdLO/dHUGIZE1nI45fwdoz80/PFIoq7n/xIADglNVtXh3PEZ0OzaqetWZi9syqqqrpmSNxzhHvOQsqvNGaIUVEOvTWjA9mVaNCFaeI2OlfTvmqiDhrzTB/yJJElEt7jSBqlYtPWA4A+MUbffN+zE9fPIBdgym0xsO4fMsqr47mCHZNGJ+2F3Sot2ZEm1VtKiLmwQE/FBGW+M17txkv5DyVR+R4ju/a9IgYPTv+PwJ934wNRcSvZFXAtPbbpjGNJmYIogRrzzy3dxQjFVSEiekcvvnEbgDADecfjdbyPhdZaY1HdNP9mI2wtqSeIyLarGrvZo9dXxWF3zXfzs2v/npDioh8GIWI+x+O/V0zYpbeAUBzg31FhBmpREualWAZIIfHZ2xt26SJGYIosaojjk0rW6BqwF8//KYeTcD49n+/g/HpPI7uTuDTm+VWQ4BS2nW7gxFe5hERGe9e+vql62S2oFpqjUyzeIRIiJs3hxSRRYKI1oyVJyVgCjSTxSPiY2tmZVsDIsEAsgVVj2y3wsHxaQA0MUMQAHDjhRsRDip49M0BXH//q3ox8s5gCv/6/H4AwK3/4z0IcbjeeYGermqnEGGKiODWjPk6mbbQnmGKSJxTvDtgmpqx5RGR82cv56k8wlBEeJpV7QWaifSIWN03kzNV9QlO/Us7hIIBrC3vvNgzNGX583YPlvIQju5uEnIugqglzj6qEz/8zCkIBxX84o1+XH//qygUVfztz3+PoqrhguO6cdaGTr+PaRlj34z1lq2xeVfsdSwcDOiKhBWfCDOq8rzRY685RVVb0EdTl+O7vb29+MIXvoC1a9eioaEB69evx6233opczn5OhEjyPBWRsL2pGSPQzH9FxFzR8wrbscuGrgQAe4XIzv5SIXLMUipECAIAPnRsN+683ChG/vDOZ/Hbd0YQCQZwy8XH+n08WzhrzbCpGbGKCGAv1IxdY0UoIsDCqkhdju/u3LkTqqrirrvuwltvvYVvfetb+OEPf4gvf/nLIh7OMVx3zYTs5YjINDUzUe6rJqIh32Tb9TYLkeFUFqPpHBSFFBGCMHPecUYxwvbK/M+z12J1h5ybduej00G6qjE1I17ZbbIR864rIhwVZ/MN9ELeRNnHd4X8tC666CJcdNFF+t/XrVuHXbt24c4778Ttt98u4iEd4dfUjKZpQqUyu1MzbGy2rVH8XcR86IrIsLVCZFc5pnpNRyMaON5lEMRi4LzjuvGDy0/B/7p3B7qbY7jmgxv8PpJt7O6byeSL+jXdG0XEerqqrohwVJwDAQWRUAC5grqgIqJbASQ1q3pmCJicnER7e3vVj8lms8hmjeo3mUwKPZNhVuUwNVMuRHJFFaqqVV2vbS5WhJhVbU7NTJTH49p8HOnbsMRQRDRNW9BZvnOg9Nw4htQQgqjI+cd14/mtH0I4FPAlqNAtbN+M1Zh3dr1TFCP5VCR2RnhFKCIAEGOFyAKKiG4FkFQR8aQ82rNnD+644w78yZ/8SdWP27ZtG1paWvS3np4eoefimqxqUjZyCxmHTO0bEU8MY9eMtUJEV0R8LETWLWmEogCTM3lLFx62uIv8IQQxPx2JqPC4c1EYO6istWaYJy4RDVW9EeSFnQ28IqZmAOurRWRXRGyd6uabb4aiKFXfdu7cOetzDh8+jIsuugiXXHIJrrrqqqpff+vWrZicnNTfDh48aP9/ZANmVo1yGd819+usPSkCChAO8v+FMaZmrLVmxsuKCDOH+UEsHERPWxyANZ8Ia80cu4wKEYJYjOitGYuJy0aYmTeFl75vxooikuU/NQOYskQWGJJgr0mymlVtfVe+9KUv4Yorrqj6MevWrdP/3NfXh3PPPRdnnnkm/umf/mnBrx+NRhGNRu0cyRU8PSKhgIKAAqgaKzTm/2XQM0RCQSGLp9gv4lSusGCbCCjFKANAa9zfO6cNXQkcGJvGnuEpnLG+Y96PK6qaPrp7zNJmr45HEISHsNaMVY+IEWbmTRtKBkWEKeoLtmZ0RUTO1oytn9iSJUuwZMkSSx97+PBhnHvuuTjllFPw4x//GIGAfJUYz0JEURREQ0HM5IsLGlYzAhfeAcYvoqaVipGF7hDGy3cc7T7HPm/oSuDJnUPYu4Aisn80jWxBRSwcwKr2uEenIwjCS1hrZipbQCZfXNDYn/IozIxhyyMiSBExYiOs2QFkVUSEnOrw4cM455xzsGrVKtx+++0YHh7GwMAABgYGRDycY7Icc0QA61ki2bzYcJlYOKj/n6y0Z4ypGZ8LkSXWRniZP+To7iYEPegFEwThPU3RkH4ds9KeSXoUZsaQSRGh8d0KPPHEE9izZw/27NmDlStXzvo3TbO+S0Q0hlmV1xIilv1vTRERWZ02N4QwMpWzVK1PlFszfppVAWB9l7V0VVaIbCSjKkEsWhRFQUcigv7JDMamclixwHJLdtMlpUdE0NSMZUXEg9ccNwg51RVXXAFN0yq+yQTPZFXA+jZEfeGdwH6dnckZtt3SzxwRANiwpFRYDCQzVVNhd7HRXfKHEMSihhnorWzmZtc6KT0iAnJEABseEcEqvFvkLI88gqdHBDCHmll0MAstRKxPzoxLML4LAC3xsJ6muHc4Pe/H7SJFhCDqAjuGVSNV1SuPSPlmz0ZrhnuOCFt8Z3V8t54UkVohx3kjoVWZzJPWDFNEFkj9U1VNj3j3c3yXsaHcnpnPsDqdK2D/WGnrLmWIEMTiplNPV7WuiHjVmtGTVS0kWDOzKvccEQs3v+Yk70WRI7LY0JNVuSki1sJlvJDJrO6bSWUKKKqllpnf47vAwlHvuwenoGmlPRRMPSEIYnFibOC1oIh4PL5rZ9eMrohwn5pZWBHJFVUwVwS1ZiREN6ty84hYbM2w8V2BiojVQoSFmTVGglI4qheanGH+EGrLEMTiR+7WjP1AMz8UkVkrRag1Ix857mZVi60ZDzwiTJ5cyCNiGFX9b8sAwIauUoExX2uGot0Jon5o19NVJWzNlG/20rmiripXQtM0YYqIFY+IWaHn9VrHGzlP5RH8zao2p2aEKiJsA+8Cikja/3h3M6w1s39sWv/5mNnZT4UIQdQLnQnrG3g9T1Y1FRWs0KhEtqCC1Sncc0RYxHuVqRmzUVVEkjcPqBABx0KEmVUthst44xGprogY8e5yFCLdzVEkoiEUVQ29o7MnZzRNw65ytPuxNLpLEIuejkbWmqmuiBRVTV+W6dVNVTQU0HeFVWvPpE0ekjjvHBELN7+yj+4C9V6I+NSa0WfKOVfHZlif1LIiIoFRFSiFGK3vquwTGZ7KYiydQ0ABjupO+HE8giA8hJlVR9K5qjlUfRMzyBVVRIIBLF8g+IwXiqIYPpEqhlUWZhYLB7gnQdtVRGRF3pN5QN6n1gxTKURKiFYVEeYRkUURAeY3rLL8kDUdjVJX9wRB8IEpIrmCWvXFfv9oaaS/p73B07UPCQtDAaIyRABrrzmyj+4CdV6I8B/ftTY1YyQAilMhrE7NTEzL5REBTCO8cwoR8ocQRH3REAnqyvFYlRHefeU27trORk/OxWChZtWKpDSbmOGcqgpYU0QMT6K8N291W4ioqoZ8sST1cV96t0COiBdRxJanZiRZeGdm3kJET1QlfwhB1At6e6aKYbV3pFSIrO7wthCxsm9mWqAiwpThaq85pIhIDFNDAD9bM+IUkWaLu2bG9YV3cnhEAKMQeXdkCqppLG7XINsxQ4oIQdQLVgyrrBBZ47UiooeazX/DlxaUIQJYzBFhcRESKyLezDlJiLkQ8TrQzAtFhH3tmXwR+aI67//RMKvKo4j0tDUgEgwgk1dxuGxC+/XOIeweLCkkFGZGEPVDp4V0VTZht9ZjRYSZVavd8E0LyhABLOaIFAyzrKzUbSGSL/APebE6NZP0sBABSr8k83lAxiU0q4aCAaztbMSuwRT+4PvPzOoNdzVFsao97uPpCILwkoUUkaKq4eDYDABgTae31wYrG3jTOXGKiO4RIUWkNmGKSDioIMDJZR210K8DjNaMyATAUDCAeCSI6VwRqUy+YiGiaZrempHJrAoAG5c1YddgCmPpHCLBALasa8c5x3Tho5uWcft5EQQhP+0LKCLm0d1lLd6M7jIseUSyHkzNWFBEZB7frd9CpMA3QwSw1prJFVRdMREdRdwUC5ULkcq/JEnJFt6Z+YsLjsHazka8Z3kLzlzfIUTWJAhCfjoaq6ersrbMqo64p6O7ACzliOiKiMipGQu7ZqgQkRDeqaqANbOqOdcjITiKuDkWxmAyO+/kDPOHxCNB6XI5etrjuP68o/0+BkEQPsO2bM+3b0Y3qnrsDwFMOSLVAs18VkT08V3JrvFm5C2RBJPlvHkXsOYRYepEYyQovHpnPpH50lWZP6RNIn8IQRCEmY4F9s3sGymFma312B8CWNvAa3hEBBQiJkVkvuTZWlBE5D2ZYPKcw8wAa7tmvAgzYzTpI7zzKCIShpkRBEGYYWbV+XJEWGvG6wwRwLjZqx7xzqZmRLRmSl9T02ZPgpoxckRIEZEOka2ZSltjGV7EuzMW2jcznmYL7+TyhxAEQTCYIjI+nZuVK8To9SlVFTAlq1Zdele6MRXhczOrHPMp8V5se3eLvCcTDO+Fd4C11owXo7uMhfbNkCJCEITstDdGEAkGKm7kLhRVHBwrtWa8DjMDrI3vMkVExPhuJBiAUu7wzxfzro/vkiIiHzkBfTO9NVPFwexFqipjoX0zerw7eUQIgpCUcDCA965qBQA89+7orH/rm8ggX9QQCQWwrDnm+dmMQLMqyaplj4gIs6qiKMYN8DyG1VoY35X3ZILJCTGrLuxg9iJVlbHQvhkj3p0KEYIg5OXM9Z0AgGf3zi5EdH9Ie9yXfCGzR2Q+syibmhExvguY9s3McwOcyZNZVVp4b94F7E3NeKGINC+giOjx7o3kESEIQl7O3NABAHh+7+gsnwgrRPxoywCGIqJqpXUalZgWqIgAxlbd+WLedUWEWjPyIcasWvpauaJa0VQFAEk9VdULj0h5amaehUxjEsa7EwRBzOXEla1oCAcxms5h91BKf/++Ef+MqkDJ98E8GvP5RNICp2aAhS0BNL4rMULMqqaKc75RKm+nZso5IjOVf0EmyKxKEEQNEAkFcNradgDAM3uM9oyfYWZAyaOxUJbIdFZcjghgRRGRf9dM/RYiAhURYH6fiEw5ImNp8ogQBFEbnLm+1J55bu+I/r79o+WJmQ7/FmE2VYl5zxVU/aZUWGtmAUXESFaV9+Ve3pMJRsSumVBAAfNLzZf976VZtdrUjKZpuiLSRh4RgiAkhxUiL7w7hkJRRaGo4oCPo7sMfYS3wnV2Jme8DjQIGN8FDKWDFJEaRESyammUqvrkjJfju/rUTCZ/hKM7lS2gUPaxkCJCEITsvGd5C5pjIaSyBbzZl8ThiRkUVA3RUABLfRjdZegjvBUUEeYPiQQDXF9rzOgx7/PliOhmVXlf7uU9mWBEtGaAhY1Dfigi+aJ2xCQPm5hpCMu38I4gCGIuwYCC09eVVJFn946gV2/LNPoyustIxOZPV9XDzAQZVYGFl63S+K7EZAWYVYGFR3i9TFZtjIR0R3dyjk+EZYiQUZUgiFrB8ImM6kbV1T76Q4DqHhE93l2QPwQwvB/zJ6vS9l1pEaaIhKqHy6T08V3xrZlAQNF/SeZOzjBFhPwhBEHUCmduKAWbbe8dw67B0hivX6O7jES1QkRgvDuDFRgLe0TkfbmX92SCYR4RnsmqAKrG7eYKqv6k8EIRKT1O5ckZincnCKLWOKorgc5EBJm8il++0Q/AX6MqYJhVKw0F6KO7AhbeMQwV/sibX03TyKwqM+I9IkcWIuZiICHwiWlmvskZtvCOChGCIGoFRVFwRjnufaLcXva7NWMoIkfGJOhhZj4pIubXIRrflRARS+9KX2/+1gwrBuKRIEKclZj5aG4wJmfMGIUItWYIgqgdmE+E4XdrpqnK+C6LdxcVZgZUV0TMhQgpIhIiYtcMUN2s6uXEDGO+fTP6wjsyqxIEUUOYC5FYOIDuJv9Gd4EFPCJZsfHuwEKKSKk4URQgHPRvsmgh6rcQEbB9F6juEfEyQ4Qxn0fEWHhHhQhBELXDqvY4VrQ2APB/dBdYwCPigSKiJ6tWmJphr0OxUBCKQoWIdOSKpTAv/uO787dmvBzdZTBFZO7UDDOr0sI7giBqCUVRdFXEb38IYG1qRqRHpFqOSC2EmQH1XIiUf0CizKqVZDIvR3cZ8ykizOjVToUIQRA1xufPXIN1SxrxqVN7/D6K4RGpUIh4MTVTLUekFsLMAMC7W3PJEDU101I2h07M5I74Nz88IvNNzYzRnhmCIGqU41e04MkvneP3MQAAiej8yapeTs1UVUQkNqoC9ayICDKrdpd3Hgwls0f8m5ebdxmVpmY0TTMCzUgRIQiCcIzuEfFJEWFqRyVFRPeIUGtGTkRs3wWAJU1RAMBgqlIhwloz3isiSVO1PkUL7wiCILjAPCKlwMrZxYAnHhE2NVNlfJcUEUnJM7OqMEUkc8S/+dOaYR4RoxAZT5cKolg4IGw1NUEQRD1gDqdku2UYnuaIVBnfld0jIvfpBCJKEekqKyJDlRSRrPfju0x96Z+cwWTZoMrCzMioShAE4Y5gQNF3ycz1iXiaI1JBEcnorRm5bzjrthDJCjKrMkVkLJ3Tix2GH4rIscuasbwlhonpPP7knpeQK6gmoyoVIgRBEG5hqkhqTsy7JzkibHyXFJHaQ9T4bls8rCfYDU/NVkWSM94rIrFwEP985WlIREN4/t0x3Pwfr2NsioyqBEEQvEjME/M+nROviESrjO/qHhEyq8qJPjXDuTWjKAq6ypHDg3N8In4oIgCwcWkzfnD5yQgGFPznK4fx/V/vAUCKCEEQBA+a5gk1Y56RRqHJqqw1c6QiwooTMqtKiiizKgB0NZd9InMKET+SVRnvP3oJvvbx4wEA746kAQDttPCOIAjCNYkKoWZFVcNMnrVmRCarll7DcgUVmqbN+jca35WYoqqhqIqJeAfmN6z6kaxq5rLNq/Bn56zX/07x7gRBEO7RPSKm1syMqVXSKDRZ1Shy5oaa0fiuxJhNpCIUEWZYNbdmSjPmpcf1QxFh/OUFx+DjJy0HAJzY0+LbOQiCIBYLerqqSRGZLv85oIg1i8ZMX3uuYbVWzKp1GfHuVSFiTlc173pJCKyOFyIQUPCtS0/CX330OHQmor6dgyAIYrHQVMGsms4Z/hCRm29DwQCCAQVFVUOmUEQLDMVd3zVD47vykS0akllIwArpSumqTLKLR4IICWgH2UFRFCpCCIIgOFFpAy/LEIkLnJhhxOaJea8VRUTu0wnCbFQVUalWSlf1a2KGIAiCEEuiwnLR6Zz4iRnGfIvvDI+I3C/1cp9OEKw1ExWkTFQyq7LWjJcZIgRBEIR4DEXEaMGzPTNeKCLzLb7Tx3epNSMfOUGpqoxK6ap+ju4SBEEQ4miqML6rb971UBHJHGFWLY/vkiIiH6ILkUrpqqSIEARBLE50RWSWWVX85l0Gey2bu/03S2ZVeWGpqmFBrZlK6arkESEIglicGLtmjhzfjXswJTmfIpIhs6q8iFZEAHO6KlNESk/KZipECIIgFhXmXTOapuGxNwdw93P7AXijiLDk1PkUEdm379blq6KoPTNmDMMqU0SoNUMQBLEYaSoHmk1M5/HJO5/FKwcmAJTa9J9470rhj8+SU4/0iNSGIlKfhYgHisjcdFW9NeNjmBlBEATBH6aI5IoqXjkwgYZwEF9831pc9f51nqz0iM2zgbdWxnfr8lVRL0QEKiJz01VTWaaI1OW3nCAIYtHSHAuhMxHFxHQOl23uwf/+4FHoKr8GeAFTRObmiGSoNSMv+aJ4RWRuuqphVqXWDEEQxGIiFAzgsevfB1XVPC1AGPMrItSakRYvWzMsXZXliDQ3UCFCEASx2PBzbcaCyaqSKyJyl0mCyHpgVu1unp2uaphV67L2IwiCIATBFI+sSRFRVc1IESdFRD4uPK4bR3cl0BIXp06wHBGWrko5IgRBEIQIjBwRoxBh06Hmf5eVunxV7GqOCe/jsXTVfFHD8FQWyZmSIuKFg5ogCIKoH3RFxNSayZpGeWVXRISfLpvN4qSTToKiKHj11VdFP5w0mNNVD41N608QUkQIgiAInlRSRJhRNaAAoQD/LfM8EV6I3HjjjVi+fLnoh5ESlq66dzitvy9BOSIEQRAER6IVzKrm0V1FqeNC5NFHH8Xjjz+O22+/XeTDSEt3WRHZOzwFAIhHgggJNMgSBEEQ9QdrvVRSRGRvywACPSKDg4O46qqr8PDDDyMej1v6nGw2i2w2q/89mUyKOp4nMEVkz1CpEKG2DEEQBMGbSkvvjFRVuY2qgCBFRNM0XHHFFfjTP/1TnHrqqZY/b9u2bWhpadHfenp6RBzPM1iWiFGIkFGVIAiC4EssdOTSO6aORMPyKyK2TnjzzTdDUZSqbzt37sQdd9yBVCqFrVu32jrM1q1bMTk5qb8dPHjQ1ufLBktXPTwxA4AUEYIgCII/0SqKSKwGFBFbr4xf+tKXcMUVV1T9mHXr1uHJJ5/Ec889h2h0dtLcqaeeissvvxx33313xc+NRqNHfE4t0z1nRJgUEYIgCII3TBHJFCp4RGpAEbFViCxZsgRLlixZ8OO++93v4qtf/ar+976+Plx44YV44IEHsGXLFvunrFFYuiqDFBGCIAiCN/rUTP7IqZm6NauuWrVq1t8TiQQAYP369Vi5cqWIh5QSliPCaKZChCAIguAMW3qXzhWgqhoCAUVXRGRPVQXqdNeMV7B0VQa1ZgiCIAjerGhtQDwSxMR0Hve+sB+AoY7UgiLiyQnXrFkDTdNw0kknefFw0mBOVwWAJgozIwiCIDjTFAvjpos2AgBue3QnDo1P0/guYdBl8omQR4QgCIIQwWdPX43T1rQhnSti63++YYzvkiJCdJsVEWrNEARBEAIIBBTc9oebEAkF8Nt3RvDgjkMADCOrzFAhIhhSRAiCIAgvWL8kgRvOPxoA8E45SJMUEWJWlggpIgRBEIRIvnj2WpywokX/ey3kiMh/whqHpasCpIgQBEEQYgkFA/jGH21CKFCa2KyFZFUqRARjVkSaSREhCIIgBHPssmbc/OGNaAgHsXltu9/HWRC6RRdMN3lECIIgCI/54vvW4cqz1iIYUBb+YJ+hV0bBLGtpQCigIBwMUCFCEARBeEYtFCEAFSLCaWkI44efOQXRcAChIHXCCIIgCMIMFSIecN5x3X4fgSAIgiCkhG7RCYIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDam372qaBgBIJpM+n4QgCIIgCKuw1232Ol4NqQuRVCoFAOjp6fH5JARBEARB2CWVSqGlpaXqxyialXLFJ1RVRV9fH5qamqAoit/H8Z1kMomenh4cPHgQzc3Nfh9n0ULfZ2+g77M30PfZG+j7PBtN05BKpbB8+XIEAtVdIFIrIoFAACtXrvT7GNLR3NxMT3QPoO+zN9D32Rvo++wN9H02WEgJYZBZlSAIgiAI36BChCAIgiAI36BCpIaIRqO49dZbEY1G/T7Kooa+z95A32dvoO+zN9D32TlSm1UJgiAIgljckCJCEARBEIRvUCFCEARBEIRvUCFCEARBEIRvUCFCEARBEIRvUCFS42SzWZx00klQFAWvvvqq38dZVPT29uILX/gC1q5di4aGBqxfvx633norcrmc30dbFHz/+9/HmjVrEIvFsGXLFrz44ot+H2lRsW3bNpx22mloampCV1cXPv7xj2PXrl1+H2vRc9ttt0FRFFx//fV+H6VmoEKkxrnxxhuxfPlyv4+xKNm5cydUVcVdd92Ft956C9/61rfwwx/+EF/+8pf9PlrN88ADD+CGG27ArbfeipdffhknnngiLrzwQgwNDfl9tEXD008/jauvvhrPP/88nnjiCeTzeVxwwQVIp9N+H23Rsn37dtx1113YtGmT30epLTSiZvnlL3+pbdy4UXvrrbc0ANorr7zi95EWPd/4xje0tWvX+n2Mmmfz5s3a1Vdfrf+9WCxqy5cv17Zt2+bjqRY3Q0NDGgDt6aef9vsoi5JUKqUdddRR2hNPPKF94AMf0K677jq/j1QzkCJSowwODuKqq67Cv/7rvyIej/t9nLphcnIS7e3tfh+jpsnlctixYwfOO+88/X2BQADnnXcennvuOR9PtriZnJwEAHr+CuLqq6/GxRdfPOt5TVhD6qV3RGU0TcMVV1yBP/3TP8Wpp56K3t5ev49UF+zZswd33HEHbr/9dr+PUtOMjIygWCyiu7t71vu7u7uxc+dOn061uFFVFddffz3OOussHH/88X4fZ9Fx//334+WXX8b27dv9PkpNQoqIRNx8881QFKXq286dO3HHHXcglUph69atfh+5JrH6fTZz+PBhXHTRRbjkkktw1VVX+XRygnDG1VdfjTfffBP333+/30dZdBw8eBDXXXcd7r33XsRiMb+PU5NQxLtEDA8PY3R0tOrHrFu3Dp/61Kfws5/9DIqi6O8vFosIBoO4/PLLcffdd4s+ak1j9fsciUQAAH19fTjnnHNw+umn4yc/+QkCAarf3ZDL5RCPx/Hggw/i4x//uP7+z3/+85iYmMAjjzzi3+EWIddccw0eeeQR/OY3v8HatWv9Ps6i4+GHH8YnPvEJBINB/X3FYhGKoiAQCCCbzc76N+JIqBCpQQ4cOIBkMqn/va+vDxdeeCEefPBBbNmyBStXrvTxdIuLw4cP49xzz8Upp5yCe+65hy4onNiyZQs2b96MO+64A0CpdbBq1Spcc801uPnmm30+3eJA0zRce+21eOihh/DUU0/hqKOO8vtIi5JUKoX9+/fPet+VV16JjRs34qabbqJWmAXII1KDrFq1atbfE4kEAGD9+vVUhHDk8OHDOOecc7B69WrcfvvtGB4e1v9t6dKlPp6s9rnhhhvw+c9/Hqeeeio2b96Mb3/720in07jyyiv9Ptqi4eqrr8Z9992HRx55BE1NTRgYGAAAtLS0oKGhwefTLR6ampqOKDYaGxvR0dFBRYhFqBAhiHl44oknsGfPHuzZs+eIAo+ERHdceumlGB4exle+8hUMDAzgpJNOwmOPPXaEgZVwzp133gkAOOecc2a9/8c//jGuuOIK7w9EEPNArRmCIAiCIHyDXHcEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfjG/wM+Mbg9yFnJtwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "backend = np\n",
    "\n",
    "def function(x):\n",
    "    return x*backend.sin(x**2 + 1)\n",
    "\n",
    "def analytical_gradient(x):\n",
    "    return backend.sin(x**2 + 1) + 2*x**2*backend.cos(x**2 + 1)\n",
    "\n",
    "\n",
    "x = np.linspace(-5, 5, 100)\n",
    "plt.plot(x, function(x), label=\"f(x)\")\n",
    "\n",
    "descent_x, descent_y = gradient_descent(1, 0.01, 300, solver=\"analytical\")\n",
    "\n",
    "# Change the backend to JAX\n",
    "backend = jnp\n",
    "jax_descend_x, jax_descend_y = gradient_descent(1, 0.01, 300, solver=\"jax\")\n",
    "\n",
    "plt.scatter(descent_x, descent_y, label=\"Gradient descent\", marker=\"v\", s=10, color=\"red\") \n",
    "plt.scatter(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\", s=5, color=\"black\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}