FYS-STK4155/doc/LectureNotes/_build/jupyter_execute/exercisesweek41.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e3354581",
   "metadata": {
    "editable": true
   },
   "source": [
    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
    "doconce format html exercisesweek41.do.txt  -->\n",
    "<!-- dom:TITLE: Exercises week 41 -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d5fc499",
   "metadata": {
    "editable": true
   },
   "source": [
    "# Exercises week 41\n",
    "**October 9-13, 2023**\n",
    "\n",
    "Date: **Deadline is Sunday October 15 at midnight**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3ef3596",
   "metadata": {
    "editable": true
   },
   "source": [
    "# Overarching aims of the exercises this week\n",
    "\n",
    "The aim of the exercises this week is to get started with implementing\n",
    "gradient methods of relevance for project 2. This exercise will also\n",
    "be continued next week with the addition of automatic differentation.\n",
    "Everything you develop here will be used in project 2. \n",
    "\n",
    "In order to get started, we will now replace in our standard ordinary\n",
    "least squares (OLS) and Ridge regression codes (from project 1) the\n",
    "matrix inversion algorithm with our own gradient descent (GD) and SGD\n",
    "codes.  You can use the Franke function or the terrain data from\n",
    "project 1. **However, we recommend using a simpler function like**\n",
    "$f(x)=a_0+a_1x+a_2x^2$ or higher-order one-dimensional polynomials.\n",
    "You can obviously test your final codes against for example the Franke\n",
    "function. Automatic differentiation will be discussed next week.\n",
    "\n",
    "You should include in your analysis of the GD and SGD codes the following elements\n",
    "1. A plain gradient descent with a fixed learning rate (you will need to tune it) using the analytical expression of the gradients\n",
    "\n",
    "2. Add momentum to the plain GD code and compare convergence with a fixed learning rate (you may need to tune the learning rate), again using the analytical expression of the gradients.\n",
    "\n",
    "3. Repeat these steps for stochastic gradient descent with mini batches and a given number of epochs. Use a tunable learning rate as discussed in the lectures from week 39. Discuss the results as functions of the various parameters (size of batches, number of epochs etc)\n",
    "\n",
    "4. Implement the Adagrad method in order to tune the learning rate. Do this with and without momentum for plain gradient descent and SGD.\n",
    "\n",
    "5. Add RMSprop and Adam to your library of methods for tuning the learning rate.\n",
    "\n",
    "The lecture notes from weeks 39 and 40 contain more information and code examples. Feel free to use these examples.\n",
    "\n",
    "In summary, you should \n",
    "perform an analysis of the results for OLS and Ridge regression as\n",
    "function of the chosen learning rates, the number of mini-batches and\n",
    "epochs as well as algorithm for scaling the learning rate. You can\n",
    "also compare your own results with those that can be obtained using\n",
    "for example **Scikit-Learn**'s various SGD options.  Discuss your\n",
    "results. For Ridge regression you need now to study the results as functions of  the hyper-parameter $\\lambda$ and \n",
    "the learning rate $\\eta$.  Discuss your results.\n",
    "\n",
    "You will need your SGD code for the setup of the Neural Network and\n",
    "Logistic Regression codes. You will find the Python [Seaborn\n",
    "package](https://seaborn.pydata.org/generated/seaborn.heatmap.html)\n",
    "useful when plotting the results as function of the learning rate\n",
    "$\\eta$ and the hyper-parameter $\\lambda$ when you use Ridge\n",
    "regression.\n",
    "\n",
    "We recommend reading chapter 8 on optimization from the textbook of [Goodfellow, Bengio and Courville](https://www.deeplearningbook.org/). This chapter contains many useful insights and discussions on the optimization part of machine learning."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9940e21b",
   "metadata": {
    "editable": true
   },
   "source": [
    "# Code examples from week 39 and 40"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6559d6d2",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Code with a Number of Minibatches which varies, analytical gradient\n",
    "\n",
    "In the code here we vary the number of mini-batches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "021b66d2",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.89481038]\n",
      " [3.13155259]]\n",
      "Eigenvalues of Hessian Matrix:[0.28194659 4.81122914]\n",
      "theta from own gd\n",
      "[[3.89481038]\n",
      " [3.13155259]]\n",
      "theta from own sdg\n",
      "[[3.88291866]\n",
      " [3.19123037]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHFCAYAAADxOP3DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABT+ElEQVR4nO3dd3gU5doG8HvSQ0kgQEIaJCjSQu9CgIiACJgQsYBiOepRaUZsYAMsoBwLKJbDOR74LAkICQmKBYQEUECpAtI1QCgh1AQCCWT3/f4Yd8kmu9mSLTM79++69sq1M7Oz78zuZJ59y/NKQggBIiIiIo3y8XQBiIiIiDyJwRARERFpGoMhIiIi0jQGQ0RERKRpDIaIiIhI0xgMERERkaYxGCIiIiJNYzBEREREmsZgiIiIiDSNwRCRAi1cuBCSJBkffn5+iIyMxL333ouDBw96rFzTp0+HJEkee381OHz4MCRJwjvvvOPpohCRjfw8XQAismzBggVo3bo1ysrK8Msvv+DNN99Ebm4u9u3bh4YNG3q6eEREXoHBEJGCJSQkoFu3bgCAAQMGQKfTYdq0acjOzsbDDz/s4dKRJ127ds1Ya0hEtcNmMiIVMQRGp06dMi4rKyvDM888g06dOiE0NBRhYWHo3bs3cnJyqr1ekiRMmDABX3zxBdq0aYM6deqgY8eO+Pbbb6ttu2LFCnTq1AmBgYGIj4+32OxTVlaGqVOnIj4+HgEBAYiOjsb48eNx4cIFk+3i4uIwfPhwfPvtt+jcuTOCg4PRpk0b43svXLgQbdq0Qd26ddGjRw9s2bLF6vkwNCfm5ubiySefROPGjdGoUSOkpqbixIkT1Y59+vTp1fYRFxeHhx56qNo+16xZg8ceewyNGjVCSEgIHnjgAZSWlqKwsBB33303GjRogMjISDz77LO4du1atf3q9Xq8+eabaNasGYKCgtCtWzesXr262nYHDx7EmDFjEB4ejsDAQLRp0wYfffSRyTZ5eXmQJAlffPEFnnnmGURHRyMwMBCHDh3C5cuX8eyzzyI+Ph5BQUEICwtDt27dkJGRYfX8EZGMPymIVCQ/Px8AcNNNNxmXlZeX49y5c3j22WcRHR2Nq1ev4qeffkJqaioWLFiABx54wGQfK1aswObNm/Haa6+hXr16mD17NkaOHIn9+/ejRYsWAIDVq1cjOTkZvXv3xqJFi6DT6TB79myTIAwAhBBISUnB6tWrMXXqVCQmJmLnzp2YNm0aNm7ciI0bNyIwMNC4/e+//46pU6fipZdeQmhoKGbMmIHU1FRMnToVq1evxsyZMyFJEl544QUMHz4c+fn5CA4OtnpeHn30UQwbNgzp6ekoKCjAc889h/vvvx9r1qxx+Fw/+uijSE1NxaJFi7B9+3a8+OKLqKiowP79+5Gamop//vOf+Omnn/D2228jKioKkydPNnn9vHnz0Lx5c8yZMwd6vR6zZ8/G0KFDsXbtWvTu3RsAsGfPHtx8881o1qwZ3n33XTRt2hQ//vgjJk2ahDNnzmDatGkm+5w6dSp69+6NTz/9FD4+PggPD8fkyZPxxRdf4I033kDnzp1RWlqK3bt34+zZsw4fO5HmCCJSnAULFggAYtOmTeLatWvi4sWL4ocffhBNmzYV/fr1E9euXbP42oqKCnHt2jXxyCOPiM6dO5usAyAiIiJESUmJcVlhYaHw8fERs2bNMi7r2bOniIqKEleuXDEuKykpEWFhYaLyv40ffvhBABCzZ882eZ/FixcLAGL+/PnGZc2bNxfBwcHi2LFjxmU7duwQAERkZKQoLS01Ls/OzhYAxPLly206T+PGjTNZPnv2bAFAnDx50uTYp02bVm0fzZs3Fw8++GC1fU6cONFku5SUFAFAvPfeeybLO3XqJLp06WJ8np+fLwBYPH+33nqrcdmQIUNETEyMKC4uNtnnhAkTRFBQkDh37pwQQojc3FwBQPTr169a+RMSEkRKSkq15URkOzaTESlYr1694O/vj/r16+O2225Dw4YNkZOTU62fyJIlS9CnTx/Uq1cPfn5+8Pf3x2effYa9e/dW22dSUhLq169vfB4REYHw8HAcOXIEAFBaWorNmzcjNTUVQUFBxu3q16+PESNGmOzLUPNSuZkJAO666y7UrVu3WrNQp06dEB0dbXzepk0bAHJ/qDp16lRbbiiTNXfccYfJ8w4dOtj1enOGDx9u8txQpmHDhlVbbu59LJ2/devWQafToaysDKtXr8bIkSNRp04dVFRUGB+33347ysrKsGnTJpN93nnnndXep0ePHvj+++8xZcoU5OXl4cqVKw4fM5FWMRgiUrDPP/8cmzdvxpo1a/D4449j7969GD16tMk2WVlZuPvuuxEdHY0vv/wSGzduxObNm/GPf/wDZWVl1fbZqFGjassCAwONN9Hz589Dr9ejadOm1baruuzs2bPw8/NDkyZNTJZLkoSmTZtWa6oJCwszeR4QEFDjcnPlN6fqMRma5moTGNhTVnPltHT+rl69ikuXLuHs2bOoqKjAhx9+CH9/f5PH7bffDgA4c+aMyesjIyOr7fODDz7ACy+8gOzsbCQlJSEsLAwpKSkeTcFApDbsM0SkYG3atDF2mk5KSoJOp8N///tfLF26FKNGjQIAfPnll4iPj8fixYtNcgCVl5c79J4NGzaEJEkoLCystq7qskaNGqGiogKnT582CYiEECgsLET37t0dKoMrBAYGmj0nrupbY+n8BQQEoF69evD394evry/Gjh2L8ePHm91HfHy8yXNzOZ7q1q2LGTNmYMaMGTh16pSxlmjEiBHYt2+fcw6GyMuxZohIRWbPno2GDRvi1VdfhV6vByDfIAMCAkxulIWFhWZHk9nCMJorKyvLpMbj4sWL+Oabb0y2HThwIAA5IKssMzMTpaWlxvVKEBcXh507d5osW7NmDS5duuSS97N0/hITE+Hr64s6deogKSkJ27dvR4cOHdCtW7dqD3O1eDWJiIjAQw89hNGjR2P//v24fPmysw+LyCuxZohIRRo2bIipU6fi+eefR3p6Ou6//34MHz4cWVlZGDduHEaNGoWCggK8/vrriIyMdLip5PXXX8dtt92GQYMG4ZlnnoFOp8Pbb7+NunXr4ty5c8btBg0ahCFDhuCFF15ASUkJ+vTpYxxN1rlzZ4wdO9ZZh15rY8eOxSuvvIJXX30V/fv3x549ezBv3jyEhoa65P18fX0xaNAgTJ48GXq9Hm+//TZKSkowY8YM4zZz585F3759kZiYiCeffBJxcXG4ePEiDh06hG+++cam0XA9e/bE8OHD0aFDBzRs2BB79+7FF198gd69e5v0wyIiyxgMEanMxIkTMW/ePLz22msYPXo0Hn74YRQVFeHTTz/F//73P7Ro0QJTpkzBsWPHTG689hg0aBCys7Px8ssv45577kHTpk0xbtw4XLlyxWSfkiQhOzsb06dPx4IFC/Dmm2+icePGGDt2LGbOnGkyrN7TnnvuOZSUlGDhwoV455130KNHD3z99ddITk52yftNmDABZWVlmDRpEoqKitCuXTusWLECffr0MW7Ttm1bbNu2Da+//jpefvllFBUVoUGDBmjZsqWx35A1t9xyC5YvX473338fly9fRnR0NB544AG89NJLLjkuIm8kCSGEpwtBRERE5CnsM0RERESaxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTNJFnSK/X48SJE6hfv77ZdPZERESkPEIIXLx4EVFRUfDxcV39jSaCoRMnTiA2NtbTxSAiIiIHFBQUICYmxmX710QwVL9+fQDyyQwJCfFwaYiIiMgWJSUliI2NNd7HXUUTwZChaSwkJITBEBERkcq4uosLO1ATERGRpjEYIiIiIk1jMERERESaxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTGAwRERGRpjEYIiIiIk1jMERERESaxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTGAwRERGRpnk8GFq3bh1GjBiBqKgoSJKE7Oxsi9s+/vjjkCQJc+bMcVv5iIiIyLt5PBgqLS1Fx44dMW/evBq3y87Oxq+//oqoqCg3lYyIiIi0wM/TBRg6dCiGDh1a4zbHjx/HhAkT8OOPP2LYsGFuKhkRERFpgcdrhqzR6/UYO3YsnnvuObRr187TxSEiIiIv4/GaIWvefvtt+Pn5YdKkSTa/pry8HOXl5cbnJSUlrigaEREReQFF1wxt3boVc+fOxcKFCyFJks2vmzVrFkJDQ42P2NhYF5aSiIiI1EzRwdD69etRVFSEZs2awc/PD35+fjhy5AieeeYZxMXFWXzd1KlTUVxcbHwUFBS4r9BERESkKopuJhs7dixuvfVWk2VDhgzB2LFj8fDDD1t8XWBgIAIDA11dPCIiIvICHg+GLl26hEOHDhmf5+fnY8eOHQgLC0OzZs3QqFEjk+39/f3RtGlTtGrVyt1FJSIiIi/k8WBoy5YtSEpKMj6fPHkyAODBBx/EwoULPVQqIiIi0gqPB0MDBgyAEMLm7Q8fPuy6whAREZHmKLoDNREREZGrMRgiIiIiTWMwRERERJrGYIiIiIg0jcEQERERaRqDISIiItI0BkNERESkaQyGiIiISNMYDBEREZGmMRgiIiIiTWMwRERERJrGYIiIiIg0jcEQERERaRqDISIiItI0BkNERESkaQyGiIiISNMYDBEREZGmMRgiIiIiTWMwRERERJrGYIiIiIg0jcEQERERaRqDISIiItI0BkNERESkaQyGiIiISNMYDBEREZGmMRgiIiIiTWMwRERERJrGYIiIiIg0jcEQERERaRqDISIiItI0BkNERESkaQyGiIiISNMYDBEREZGmMRgiIiIiTWMwRERERJrGYIiIiIg0jcEQERERaRqDISIiItI0BkNERESkaR4PhtatW4cRI0YgKioKkiQhOzvbuO7atWt44YUX0L59e9StWxdRUVF44IEHcOLECc8VmIiIiLyKx4Oh0tJSdOzYEfPmzau27vLly9i2bRteeeUVbNu2DVlZWThw4ADuuOMOD5SUiIiIvJEkhBCeLoSBJElYtmwZUlJSLG6zefNm9OjRA0eOHEGzZs1s2m9JSQlCQ0NRXFyMkJAQJ5WWiIiIXMld928/l+3ZRYqLiyFJEho0aGBxm/LycpSXlxufl5SUuKFkREREpEYebyazR1lZGaZMmYIxY8bUGCHOmjULoaGhxkdsbKwbS0lERERqoppg6Nq1a7j33nuh1+vx8ccf17jt1KlTUVxcbHwUFBS4qZRERESkNqpoJrt27Rruvvtu5OfnY82aNVbbDQMDAxEYGOim0hEREZGaKT4YMgRCBw8eRG5uLho1auTpIhEREZEX8XgwdOnSJRw6dMj4PD8/Hzt27EBYWBiioqIwatQobNu2Dd9++y10Oh0KCwsBAGFhYQgICPBUsYmIiMhLeHxofV5eHpKSkqotf/DBBzF9+nTEx8ebfV1ubi4GDBhg03twaD0REZH6aGZo/YABA1BTPKagNEhERETkhVQzmoyIiIjIFTxeM0REREReTqcD1q8HTp4EIiOBxETA19fTpTJiMERERESuk5UFPPUUcOzY9WUxMcDcuUBqqufKVQmbyYiIiMg1srKAUaNMAyEAOH5cXp6V5ZlyVcGaISIiIm/k6aYpnU6uETI3EEoIQJKAtDQgObl6uQxl//NPtxSVNUNERETeJisLiIsDkpKAMWPkv3Fx7q2JWb++eo1QZUIABQXydpVVLvujj7q0iAYMhoiIiLyJUpqmTp60fztLZXcxBkNERETewlrTFCA3Tel0ri9LZKR929VUdhdjMEREROQtHG2acoXERHnUmCSZXy9JQGysvB1gvewuxGCIiIjIWzjSNOUqvr7y8HmgekBkeD5nzvXO0+4okwUMhoiIiLyFvU1TrpaaCixdCkRHmy6PiZGXV84zdPCge8pkhscnanUHTtRKRESaoNPJI7GOHzff90aS5EAkP9/9w+xrGuZv6DhdpcwlAEIB75+olYiIiJzE0DQ1apQc+FQOLsw1TbmzXAMGmF/nwY7TBmwmIyIi8ib2NE0pgQc7ThuwZoiIiMjbpKbKmZ0VPDmqkQc7ThswGCIiIvJGNTVNKYm7OnPXgM1kRERE5DnW8hG5AYMhIiIi8pya8hG5CYMhIiIi8ixLnb6rPncR5hkiIiIiZaiSj6ikY0eEhoUxzxARERFpRNVO3yUlbnlbBkNERERaZy1DtJdjMERERKRlWVlyBujKiQ9jYuROzUpL0Ogi7EBNRESkVYY5wapmgD5+XF6eleWZcrkZgyEiIiJn0+mAvDwgI0P+q9N5ukTV1TQnmGFZWpoyy+5kDIaIiIicKStLnjk+KQkYM0b+GxenvFoWa3OCCQEUFMjbeTkGQ0RERM5ia7OTEmqObJ0TTAFzh7kaO1ATEVHtaXw0EgDrzU6SJDc76fXA0097vsOyrXOCKWDuMFdj0kUiIqodjkaS5eXJTWKOMExDsXSp+86ZTic33x0/bj6AkyT5c8zPd01ga0MA7a77N5vJiIjIcRyNdF1tmpM80WG5pjnBDM/nzHFNIKSwflUMhoiIyDEcjWSqts1JVTssu6JfUdV9JiebnxMsJsZ1tVQKDKDZTEZERI6xtVkoN9d0igVvZa3ZyVbp6UBgoPObHmtqzkxOdk+fL8M5sjSKrUrTHJvJiIhI2TgayZQtzU62OHjQ+TUn1mpjcnLkgHX0aPmvqzq/K3Q4P4MhIiJyDEcjVZeaarnZ6euv5b+WAiNDrch//uPcpkclNWcqNIBmMERERI5JTLR+c4+NlbfTktRU4PBhuXkwPV3+m58P3HWX9Zqjxx5zfs2JkmpjFBpAMxgiIiLHeHI0ktL5+l5vdkpMlAONjAwgLAxYvNhyh+WWLW3bvz01J0qqjVFoAM2ki0RE5DhDs5C5jrlz5mgrz5A5ljotv/8+0Lhx9Q7LeXm27deemhMl1cYYAuhRo+TAp3LTnQcDaI4mIyKi2mMG6uoMnZar3mZrSrDoikSIdo7gcinD9yQnB/jqK+D06evrYmONAbTQC/y+5AAWf3gIb/0y3PtHk61btw4jRoxAVFQUJElCdna2yXohBKZPn46oqCgEBwdjwIAB+OOPPzxTWCIiMq9ys5ArRyOphaOdll3R9OjrK38uNXG0NsaeXEiVEy3OmSMHQo0by+chNxcVew8i93A8nuq0FvEBx9H53lZ46xf3NJd5PBgqLS1Fx44dMW/ePLPrZ8+ejffeew/z5s3D5s2b0bRpUwwaNAgXL150c0mJiIhsVJtOyzWNSHMkEWJWFvDOO5bXP/usY82Z9mSRtjC0X5w9CzFnDj649xeE17+MW57pjA9+748juhgE4zJuD99if7kcoKhmMkmSsGzZMqSkpACQa4WioqKQlpaGF154AQBQXl6OiIgIvP3223j88cdt2i+byYiIyK0yMuQAwZr0dMu1Ns5oenRVE5k9TYBWyqCHhGOIQTzy0VC6gDtu3IPkUQEYNLk9KgIqmHQxPz8fhYWFGDx4sHFZYGAg+vfvjw0bNlh8XXl5OUpKSkweREREbuOMTsvOaHp0xbB6e5sArZTBBwLNUIBtExeg8HIo/ncgEckze6JO4zq2l6mWFB0MFRYWAgAiIiJMlkdERBjXmTNr1iyEhoYaH7GxsS4tJxERkQmlDCF3xbB6GwOsvS9+jhdvzsNzg7bbtNuOvevCL8gzg9wVHQwZSFW+TEKIassqmzp1KoqLi42PgoICVxeRiIjoOqXkYLK1hurgQdv3aWPg9NrsIMzaOACbKzrbtl8PZipXdDDUtGlTAKhWC1RUVFSttqiywMBAhISEmDyIiIjcytkdoR1hrYbKYNo02+c8szFouYBQ3BWzEY8/4QN9ZJTna8lqoOhgKD4+Hk2bNsWqVauMy65evYq1a9fi5ptv9mDJiIiIbGBpag53JaOsXENVE0myeX6y48E34lJwE+gtrBcAroREIPvcAHxd0BujP+kHn3kfXn+fqu8LeDxTucczUF+6dAmHDh0yPs/Pz8eOHTsQFhaGZs2aIS0tDTNnzkTLli3RsmVLzJw5E3Xq1MEYW3rpExEReZqhI7SnpKYC06fLtT+WVO5IXbmsOh3E2nU4/t0OrM3V4+c/GuBCeR3ciAmYgWnQQ4IPrnekFpIECUDwgo+BhpU6QCs8U7nHg6EtW7YgKSnJ+Hzy5MkAgAcffBALFy7E888/jytXrmDcuHE4f/48evbsiZUrV6J+/fqeKjIREZG62Dnnme6qDgcmzEXkwrfQ4NppxAC47++HwWX/EAQE+cDn4gXjMqmm4CY1FUhOVmSmckXlGXIV5hkiIvJSnAbENnl5clJEKzbc/zH+91sCdAcO4TM8AkBY7k9jaOKaPl0Otuw5/zZ+bu66fzMYIiIidbI0CercuR5vdlEcK3Oe6QEcRwzicBgAcBhxiMYx6x2LHUnaaMfn5q77t6I7UBMREZllYXoHHD8uL7d1ZJRWVOpILWDaiVkPCYCEpzAXUb6n8F78h4i1JRAC7E/aqNDPjcEQERGpi6OToGqU0AvsWLwf0z8IwzP+c3AMpkP9TyECi9tMx8tftsHRq5F46k3LqWsssiX3kII/N493oCYiIrKLPVNMeHIUlwdVlFVg/ce7kP15CXJ234AjulYAWgEYgA/wBMbV/RxDWh9Bp/sTEDVxFEZXbuJyJPmhLa9R8OfGYIiIiNTFFVNMeIHSolL8+M4uZGdWYEV+W5wT1zM/B+MyBjfdhZRh1zD8uTZo3OpRyzsyJGq00L/IhKHPkC0JExX8uTEYIiIidXHGJKheouiP0/jmX/uQ/X0gVhV1QDl6Gdc1ks5ixA17kHKXPAN8ncY9bdupoX/RqFFysGMpILI3YaKCPzeOJiMiInWxMjLKoRFOKnJw1WHkzD2M7HVh2HAxwWTwewu/I0jukI+Uhxri5sfa1W7iU3OjviqLjbUvYaIDn5u77t+sGSIiInWpqeZCIdM7OJO+Qo8tX+xFzn9PI3trDPaU3wggzri+a509SOldhOQno5AwsiUkn+bOeeOqSRLDw+XlRUWO5XRS8OfGmiEiIlInczUX9tZWKNTVS1eRO3cncjIuI2dvS5zQX2868sM1DAjbiZSBl3DHMy0R2zPKgyV1gB2fG5MuOhGDISIihXFW5mgvykBdfLQY37/zB3JygO+OtkMJQo3r6uEihsbsRvIIPW5/rh0axjfwXEGdgRmo3Y/BEBGRgjBztNHxLSex/N2DyF5VB7lnO+AaAozrInyKkHzTPqSMDkbSpPYIahDkwZJ6BvsMERGR9zFkIK76O9yQgXjpUq8OiIReYO+3fyJ73jFkb2iCzaXtAFxvAmsV8BdSOh9FyqON0eOhtvDx6+e5wmoIa4aIiMg9DKOJLI1O8tJRYLqrOmz67A9k/+8ccn5vjoPX4k3W9663C8l9zyJ5YjO0vr2Fh0qpTKwZIiIi76LgDMTOduXcFayeswvZi8ux/GBrnBYdjOsCUI6BTXYiZfAVjHi2FSI7tfdMIb2ov1VtMRgiIiL3UHAGYmc49+d5rPjXH8j+xg8/nGiPy+hhXBeKYgxrvhspqT647dkE1I/q7sGSgv22qmAwRERE9nG0RkHBGYgddeSXY8h5709krwnBugvtoUNf47oY3xNIaXsQyffXR/8J7eFfp48HS1qJxvttmcM+Q0REZLva1Ch4QeZooRf4fckB5Hx6Etm/NsWOK61N1rcPOoCU7ieQ/M8IdBnTGpKP5KGSWqCyflvsM0RERMpS2xoFBWcgrolhBvicL0qQvavyDPCAD3ToG7oLKQMuIDmtBVoMuAnATR4tb4001G/LHgyGiIjIOp1OrhEyV6MjhBzMpKXJ0zfUFMykpspBk7naJQVljrZ1Bvhhz7RGkzadPFdQe9naH2v1ak11rGYzGRERWZeXByQlWd8uN9e2GgVXjmRycN+VZ4D/qag9yhBsXFd9Bvg6zimru9n6OVbmwY7VbCYjIiLlcPZIMF9f1zTD2Nmn6dDqI8h+P7/SDPCJxnXxfkeR0uGvSjPAJ1Z7veokJsrnw1K/LXNq07FaJcP3GQwREZF1ahgJZkOfJv0dKdj61T5kzy+qNAP89Vneu9bZg+ReRUgZZ5gBvpl7j8HVauq3ZYk9zaCVqWj4PpvJiIjIOqWPBLMySkoAOO8Xjo66rTgmYozLDTPAJ99yCXdMvhHNekeb7lMFtRoOMReo2MLWZlBLgamho7yNtUxsJiMiIuVQ+kgwK6OkJABhFUW4AYdwAaFVZoDvWv0FKqrVcEhqqlzLYwj29uwB3njD+utsaQZ1Vmd7N/LxdAGIiEglDCPBoqNNl8fEeD5Rn419ld6/82ecPu+Prwt6476P+6BhfIPqGxlqNaoGV4bmtqys2pdXCQz9tkaPBgYOtO01tjSD2jN8XyFYM0RERLarWqPgweYjwwzwOR8dw9F1ZfjEhtd0ntAXaBBkeQMV1mo4hbWO1YZm0EQbOpGrcNoVBkNERGQfV40Es4FhBvicBeeQvaM5Dl67EcCN8EEiXsSriMZx+KAWN3NXJiVUch8kZzaDqqGzfRUMhoiISNEqzwD/zcHWKLIwA3y9zjPg89yjAGpxM3dVrYYa+iA5KyGmM2uZ3ITBEBERKc65A2ew5dkM/LH+PFZe6I6VGAw95EDG8gzw/YD4BrW7mbuiVkNNE6M6oxlU6Z3tzeDQeiIib+SKJhkXN/MYZoA/+8NmPHp5LmJxPaA5gUisaPYk4scPQ79xCQioF+Cacjo7hYCaJkZ19udrrjYsNtauWia33b+FBhQXFwsAori42NNFISJyvcxMIWJihJBv5/IjJkZe7sp9VlQIkZsrRHq6/LeiosZd6nV6sWPxPjF9QK7oFLxXAEKMRKbQQRK6yu8DCL0kCSFJtTsGW2Vmyu8lSabH60gZcnNN92HpkZvrqqOxjSu+M0LY/Z2oyl33bwZDRETexHAjr3qzrU0wYcs+bbyZXrtyTeS+v1081SlPNPctMNncD+XilBQh9JYCBkkSIjbW7huqQ8wdT2ys/ecvPd22YCg93TXHYQtXfGecxF33bzaTERF5C1c0ydiyz7Aw4Nw5i9mGyz77Et/tbYGczAp8m98W50SYcZMgXMGQpjuRPPQqRvY/hwYPpVgvk61ZkGvLGc1Gzp7g1tkU3ozHDNRERGQfVwwLt2WfZ89aXKcHcPofL+AuHDZ2gDbMAJ88KgCDnk5A3fCe8vYZGbaVyV35aZyRQkDpI6tcmUpARRgMERFVpeR8MDVxxbDwWgYePgBicQx3+yxFZKcIJD/QAH0eTzA/A7wK89NYpfSRVSpMkOgKDIaIiCpTQz4YS5wRTFQNBMPDnVK09M91kO4bUPN73XyzsmtRHOWs/D2u4I0BqAPYZ4iIyMBJM217TG2HhVsKBK9cMd8nCIAeNk5yWbVPjKX3Gj0aeOcd+bm5WhSlfwY1UWKNo7NTCTiZu+7fDIaIyDOUdmNQeEdSmxkCOsC+YKKGQNBwmxAwDXz0kAAIXJJCUF+UQDJXHnPnzVrQ+eyzcv+hWuSnITs4+p1xA3fdvxU/a31FRQVefvllxMfHIzg4GC1atMBrr70GvV7v6aIRkTk6nTyCJiND/qvTVd8mK0sOPJKSgDFj5L9xcZ6dDVyFM22b5cjM8lYmJxWQcAaNcBym+ywPCUfF5xkIWboAkiRdv3kamOsTY20iVABYtAj480+5Nik9Xf6bn6/+QMiWa8MTHPnOeBuXDtx3gjfeeEM0atRIfPvttyI/P18sWbJE1KtXT8yZM8fmfTDPEJGb2JJrRqk5TdSQD8Ye9iS7szEx4L9bvSN2T/5M6L74svo+lywRokkT63l51JKE0NlcldTQmWqZINEV3HX/VnwH6o0bNyI5ORnDhg0DAMTFxSEjIwNbtmzxcMmIPEhpTUyAbfMvJSfXXCsgSUBamrydu4/H2zqS2jAs3DAD/JF3VmOMDbv857QouU9PVVlZwNNPA6dPX1/WuDHw7rvVaxVsHZV0/Lht26mBWuYmc0YqAZVSfDNZ3759sXr1ahw4cAAA8Pvvv+Pnn3/G7bff7uGSEXmIEpuYbGn6SEuTmwaU2hRlyAdTtanHQJLkfitqG8lURdmFMnz76m94rPU6RAWdQ99xHTD/r4G2vdhcIGi40Vf9XM+eBe65p/r30tZgMi3Ns99pZ7H12lBKk5lWubTeyQn0er2YMmWKkCRJ+Pn5CUmSxMyZM2t8TVlZmSguLjY+CgoK2ExG3kGpTUy2Nn28/LKym6KcOSeVu9XQxHH20Dnx+ePrxZ3RG0RdXDQ5tFBcEPc1WytK6zWR5/+yZxqMigohGjWy/Dmae11Fhdw8ZOm9qr5eyefcFlptFnQSzk32t4yMDBETEyMyMjLEzp07xeeffy7CwsLEwoULLb5m2rRpAvLAB5MHgyFSNcNNxJ4bj7vY2t/G1mDIkzcGZ81J5U5mynytSaT4tudr4paGW4UvrpkcTrTPCTG+fZ5Y9fZWUX6x/Po+7A0EZ8xw7PO09F5K+k47i7f1RXMzBkN/i4mJEfPmzTNZ9vrrr4tWrVpZfA1rhsgrKfkXpq1l++mnmmsFlHLzU2BHUov+DiyqTm4qz/wuiZHIFIAQCYEHxMt9c8WWL/YIvU5veV+2BoIVFUKEhTl+o8/MrN7hWknfaWdR8nWrAuxA/bfLly/Dx8e0a5Ovr2+NQ+sDAwMRGBjo6qIRuZeS0+bbOv/SgAHKnprAwJUdSZ3Y+b2itBwV/xiHQCGq5fjxgYAeEhYGPYF/Le+EGwa1BNCy5h2mpsqd120p3/r1ciJGW5jrJ5SaKidzvP9+669X81QQSp+bjACoYDqOESNG4M0330SzZs3Qrl07bN++He+99x7+8Y9/eLpoRO6l5NFO9sy/pOSpCVzNUtbl994DmjSxKUAqLSrFynd3IXtpBc7/dQ7Lccri2/lAIKTsNEL8jwJoYVsZbQ0EbQ1QGjWyfKOvmtfGErWM4DNH6XOTkcyl9U5OUFJSIp566inRrFkzERQUJFq0aCFeeuklUV5ebvM+mGeInMaTzSfWOp4qoYnJ3mYWtTRFOYOlzu/mHlXyz5zaXSQ+e2idGBGxSQThsnGze+HB/ii2Nv/MmGF5H2r4TjuLGvuiKYC77t+cjoPIVkqYwFPBafONlJgDydOsTfVRlSRBCGB51+l452AyfilpD1EpE8oNvvl4ukUOhsX9gbhV/7W+v6rzgjmDtTmtALlW6NSpmj9/NXynnYXXht04N5kTMRiiWlPSBJ7mgjJXztvEf+C1l5cn54Oygx4SjiEG8ciHHr7oErwXKb1PYWyX3Wi+6G1ItgRWrp5PzVIgY5CZadt30t3faVINBkNOxGCIakWJE3i6K0BRQm2YN8jIkBNkOmBZv/fR9a270Kx3tOWg3Bx3BerOCmQYdJMZigyGCgoKEBsb67LCuAqDIaoVW3/Vu6IpwpOUVBumdg7UDBmlp8tTYNjb1ObOmhUGMuQiipy1vnXr1njllVdQWlrqqvIQKY+Sh7S7CqcQcKoTdVviUnATWE4IUgPDSKr1620LhF5+2f2zvBtGoI0eLf9lIEQqY1cwtGrVKqxcuRItW7bEggULXFUmImVR8pB2V7F24xXCc3OIqYDQC+z99k/MGpKHnvV2I7pHNB648ikACfoqGYEsVs1XnQvN1mC7bVsGJER2sisYuvnmm/Hrr7/irbfewquvvorOnTsjLy/PRUUjUgg1TuCp08lNMxkZ8l97a3C0WBtWS7qrOmz49y483yMPrYMOo+2IG/DiygH4rTQBAHCyXktkd5oOXeMIk9eZ/VZVzT+j08mjsmzhTUE5kZs4NGv9Aw88gAMHDmDEiBEYNmwYRo4ciUOHDjm7bETKYEiaBlQPiJSYNM0Zs9prsTbMAWUXyrBi+mbjDPB9nmiPf20egAPX4hGAcgxtshmfjlmHE9tPYePF9kjd/ir8C4/JzVjp6fLfJUvkYLuymJjrfbIMn+fTT9dcGCUG5UQq4fBossuXL2Pbtm3IzMzEBx98AH9/f4wfPx7Tp09H/fr1nV3OWmEHanIKNQz/dVanZ2s5ZKyNoPPiDrXnD53Flsnp2P3zOaw83wMrMRh6yMcWimIMa74bySMl3PZMAkJibPx/Y+l82Tp6jJ3ayUspcjTZp59+is2bN2Pz5s3Yu3cvfH190aFDB/Tq1QudOnXCV199hQMHDmDZsmXo1q2bywptLwZD5DRKvsk7OwWAo8nwvHA4/pFfjiHnvT9x9sfNeLR0LmJx/dhOIBIrmj2JuCdvR/8J7RFQL8A5b2rP6DGlBeVETqLIYCg2Nha9evUyPrp161ZtQtSZM2ciPT0du3fvdnphHcVgiDTBFSkA7K0N85Lh+EIvsHPpAeR8ehLZmyKw/UobjEQWlmIUYJILGhCSJPf7cfax2fp5vv8+MHGicoJyIidy1/3brolaCwoKrG7zyCOP4JVXXnG4QETkIFd0erZnFnNrw/ElSR6On5ysyBt3RVkFfv50N3I+L0b2rhY4XNEKQCsAgB+u4lNpHCQzs8NLrjo2Wz+ns2ed835EGub0WevDw8OxZs0aZ++WiKxxVadnW2cxt2c4vkKSU1aeAf7b/LY4JzoZ1wXhCgY33YmUoVcxsv85NHiohtFclo6tNs2qtn5Ob7wBLFyo6mZIIk9zejAkSRL69+/v7N0SkTWGFADWOj27arSRSobjn957Bt/M3ovs7wOw6lQHlKGXcV2YdA4jWuxByl3+GPR0AuqG95RXZGTYtvPKx1bbvlPWPs/Kjh+XmydV0gxJpDROD4aIyEMMKQBGjZIDH3Odnl2ZAkDBw/EPrT6CnDn5yF7XEBtKEqDH9YAwzq8AKe3/QsqDoejzeAL8gvpW34G9x2ap75Q9QUtNn2dVKmiGJDdR8iAPBeNErUTexlMpABQ0HF/oBbZ+uRfZ84uQvSUaf5S3NFnfJXgvknudQsq4KLRPbQnJx0JCTUeODXD+qL6qn2dNvG2OPLKdF47kdNv9W2hAcXGxACCKi4s9XRQi96ioECI3V4j0dPlvRYV73jczUwhJkh9y2CA/DMsyMy2/LibG9DUxMZa3N6P8YrlYOWuLGJeQJ6J9TpjsyhfXxMCGW8UHd+aJIxuOufbYcnNN11t65Oba/t4VFUK8/LJt+01Pd+z4SN0M38+q3wdr157Cuev+zZohInIuNw7HLzlWgu//tRs5OQIrjiSgBKHGdXVxCUOjdyF5hB63P9sWYTc0dOx4KtdYHTwIzJ8v1xBZOraMDDnztzWG2eht5YrUCeQdnJ1jTEEUmWdIrRgMEbmZrU1eDvwTP7GtEMvfOYCcVcFYfaYjruF6ksNw6TSSW+1F8j3BGJjWHkENgmp3HJaaHR57DGjZ0vyxuSpoqW0zJHkvLw6UFZlniIjIJk4ejn9k9iKk50Uj55fG+LU0AUBT4yY3+ecjpfMRJP+jEXo+3Ba+Af1qXXwANXeCnj5drrEyd4yuGtXn6Q7y7sROwPZRyUhOJWMwRNrEf7bKYOM/5ykv+mARBhif96y7Gyl9zyBlUjO0vr0FgHjH3t/S96A2CSRdGbSkpspBmLnaKm+ZjsMLOwG7nIJHcqoFm8lIe/jPVjlsrN4fhB/h17ghUgZfwYhnbkJUl6ZWX2NVTd+DsLDaNzu4clSftwbzXjKdi9t5cRMq+ww5EYMhMuI/W0U5f+gsAjq1QXDpaZP5vgz0AMrqhaNi9z6ENHewA7Q51r4HTz0lBy3WWOsE7a1Biyt4cSdgt3B0YmWFc9f929z/H6otnU7+xZuRIf/V6TxdIgKsN30ActMHPy+XOrrxOD4ctRYDw7ahSctQjC39FIAEfZVZv4QkwUeSUOf/PnFuIGTL9+Crr2zbl7VmB0PfqdGj5b+8iVtmz3QuVJ2hCTU62nR5TIxqAyF3Yp8hZ2MTjHKpcO4sbyD0AruyDiL74xPGGeCB6/+wDwa2x6L46Rh16mMEnL8+/5fkqn4wtnwPTp8GGjeWJ0H1xNQmWsROwLVnz8TKZILBkDM5IwU/uQ7/2VrnpGad6jPA3wTgJgCAD3ToE7IbKQPOI/mpeNxwS0sArwK6l9zzT9zWz/f+++UfMeY6QQsB3HmnXF7ebJyDnYCdw9aRnGSCfYache3dyufFuTicopa1moYZ4HMyK/DNX21xToQZ11WeAX74c23QpE1jVxyBbez5Hpw7V/2cGEabGbDm1zm8uBMwOY4dqJ3ILSeTN1rl4z9byxzsWG6YAT7nhwCsLOyAMgQb1xlmgE++0w+Dn2mPuuF1XXkEtrP3e2CoLcvJMd+pWuUdVBXFSzsBk+PYgVpt2ASjfIb8L8D1f64G3pa0zh52diw/tPoI3h2Rh36hv6Np24Z4ZGEilhf2RBmCEedXgLTOa5E3ZwdOXQ7BwkN9MfLtXsoJhAD7vwe+vnJT2NKl5vfnaOd7DrSojp2AyUPYZ8hZ2N6tDNb6vGghaV1ltvQBsrFj+WedPsScg8Owu7wlgObG1Z2D9yLFZAb4WNccizPZ+z1wdud7DrSwjJ2AyQMYDDmLq1LwK52S8qjYeoPx9D9bd50zW8+HjbWVP+2OwG60hC8q0L/hTqTcchF3PH0DmvdpA6CNc8vuDvZ8D5xZ88uBFtaxEzC5W+0nvle+4uJiAUAUFxe79o0yM4WQJPkh/6uTH4ZlmZmufX93y8wUIibG9FhjYjxznIZzX7ksSjz37jpn9pyP3Nzq25l5vNp4nvjiiZ/F2UPnnFtWNbDxHInc3Jr3U1FR/fOv+vnExsrbuUJFhVzG9HT5r6veh8hJ3HX/ZjDkbOZudrGxyrkZO4uSgg9P32Bs5a5zZsf5OLG9UPz73jWiyCdc6GCmbIDQA0IfHeP58+dJhnNq7vOz5zvmrKDKEUr68UJkI3fdv9mB2tlSU4HDh+VRY+np8t/8fO+q9lZaJmc1ZK515zmz8XyMq/c5ojpH4PFFSXhc/wkAVMsCDUmCJEmQPpirvD4b7uyA7KzO954aaGFomqv6vTA0zWVlOff9iFSGwZAreHsKfqUFH2oYyefOc2bjcZ4vCwIgzwDfY0gYTkydBylGJaN4srLk4fFJScCYMfLfuDjX3tSdMdLJEwMtlPbjhUiB2IGa7Ke04EMNI/ncec5sPM7RA4vw7uxCRHVJ+HvJAOD1x5XTId4ST3ZArk3ne51OfoSFyckczXHFQAtOQ0NkFYMhsp/Sgg81jORzwzk7n38BK2b/gW+W++FdRCMKJ+CD6udDSBKkmBjc8eOE6jdxpY/isVbLIUlyLUdysuuCOEfOkbmRfVW5KteV0n68ECkQm8nIfobgo2rfCQNJAmJj3Rd8qCGZoovOmckM8C3qYeynffD1ib6YhA8AoHooJElyryBPnw9HKa2J1haW+utU5aomSaX9eCFSIAZDZD8lBh9Kz1zrpHMm9AI7lx7Aa7fkoUudvWh+czQmZfbHmvNdoIMf2gUexEt98vDi/7WGtGSJPPN7ZUo5H45SWy1HTTVZBmFhwE8/uW6ghdJ+vBApEJvJyDGuzOTsaFJCTydTtMbBc1ZRVoFf/r0b2f9XjJxd8civNAO8BD36huxCcr/zSE6Lx40DWwJo+fcr2wIjU5x3PpSQYFNttRzWarIAuf+Qr69rm/XmzpVrpyTJ/Jxfaq0pJHIWlw7cd5Jjx46J++67T4SFhYng4GDRsWNHsWXLFptf79Y8Q1rjrCRuhv2kpQnRuLF350Kx4ZyVni4Vy6ZsEg/esF40ks6YnI4gXBZ3NN0kPntonTi1u8g9ZTaXoyYsTIgZM9ybf8hZ+X7cJT3dtrxC6emuL4tWcqCRV3HX/Vvxs9afP38enTt3RlJSEp588kmEh4fjzz//RFxcHG644Qab9uGuWW/JQdY6l2pkxurTe8/g23/tRfb35meAHx6/BymjPDADvKXRWwaNGgHz57vvs1HTzOZ5efKwf2tyc93TcV0JtXtEdnDX/VvxwdCUKVPwyy+/YH0tOkQyGFIwazdaA8OIsPx8r/rn/eeaI8h+Px856xril5IE6HH92OL8CpDS/i8kPxCKvk8kwC/IA63aOp2cv8daUw8AZGa6NyCqGkDHxipvsl3D+bM20tHLvtdEzsJg6G9t27bFkCFDcOzYMaxduxbR0dEYN24cHnvsMYuvKS8vR3l5ufF5SUkJYmNjGQwpjT03WgN3/YJ2EaEX2PrlXuT8pwjZm6P/ngH+OsMM8MlPRKLDqJsg+Vjo9OouttZsAHIw4s6bulpqOdRUk0WkMO4KhhTfgfqvv/7CJ598gsmTJ+PFF1/Eb7/9hkmTJiEwMBAPPPCA2dfMmjULM2bMcHNJyW62dC6tSimjhOxw9dJVrJ23Cznpl5CzpyWO6doCaAsAyp8B3p7z7e7EfUrPiWTgysEGROQUiq8ZCggIQLdu3bBhwwbjskmTJmHz5s3YuHGj2dcovmZILb9oXe2rr4D777fvNY7UDHngfJccK8EP7+5G9jKB744koBihxnV1cQm3Re9Cygg9bn+2LcJuaOjSstSKPTVDgDwf3+jRLiuOqvG6J7Iba4b+FhkZibZt25osa9OmDTIzMy2+JjAwEIGBga4umszef3Dm+jrExMhDX7X0CzErS84UbCtHs0i78Xyf3HEKy/+1H9krg7HmTAdcxc3GdeHSadxx016k3BuMgWntEdSgt1Pf22UMOWpsrcFTypB2JVJLTRaRBik+GOrTpw/2799vsuzAgQNo3ry5h0pUib03Wk/OqaQktnaarkwI+3OhuOF87/vuL2R/cBTZPzfGr6UJACKM61r65yOl0xGkPNIIPR9uC9+AfrV6L4+onKOmps9LCVOeEBE5yqUD953gt99+E35+fuLNN98UBw8eFF999ZWoU6eO+PLLL23eh0vyFGRmms91Iknyo2ruDkN+FEt5RpSWH8VVrJ0HS49Gjew7Ny4637prOrHh3zvF8z1yxU3+f1Xbbc+6u8TMwblizzeHhF6nt/PkKFhmpvwZWDqX5r7zRES15K48Q4oPhoQQ4ptvvhEJCQkiMDBQtG7dWsyfP9+u1zv9ZDpyo83Nte2mn5vrnDIqla3nobbnxonn+8r5K2LF9N/EY63XigifUyYv90e5uK3xb+KT0WvF8a0nHTwpKlFRISdZDAtj4j4icgt3BUOKbyYDgOHDh2P48OGeLsZ19kwWaegjoLY5lVylNsdnz2treb4NM8DnfOOD74+3Rym6G9eFoBjDmu9GcrKEoc8lICSmu9l9eB1fX+DVV4GXXmJHYCLyKqoIhhTHkRut2uZUcpXaHJ89r3XgfB/deBw57x5Czpr6WHu+PSrQx7gu2uckktseQPKYehgwsT0C6vUxtzdtYEdgIvIyDIYc4UhgYxiVYy0Trbd3QLV2Hsxx5NzYcL5FTAx2nYpA9i15yNkUgW1X2gC4Put9u8CDSOl2HMmPhaPrfa3h4+flgSoRkUYxGHKEI4ENZ46W1XQezHH03Bje5847q60SACAExh1/CZ/eez3JoQQ9+oTsQorZGeCJiMhb+Xi6AKpkuNEC12/WBjXdvA2ZaKOjTZfHxGhnWD1g+Tw0aiQ/KnPCuTEXbgkAp/RNEIQrGBHxKz57aD0Kd5/F+uKOeOabAbhxoAJSNxARkVsoPgO1M7gsg6Wjk0UyE63M3HkAnHJuTu8+hTo926PO5dMwN7uXHkBZ/XCI/QdRN1IBWcmJiKgaTtTqRC49mQxsFOPPNUeQMzcf2XkN4VdyBmtwq/UXqXziVyIib8bpONSCI2s8RugFtqXvQ/a/T1WaAV5u3roXGbbtxNtTGRARkVUMhkhVrl2+hrXzdiH7y4t/zwB/vQN05Rng7+rvB0yyYYfensqAiIisYjBEimeYAT5nmcCKIwkoRhfjuuozwP+9TqcDZjOVARERWcdgiBTJMAN8zqpgrD5tfgb45LuDMDCtPYLDzMwAz1QGRERkIwZDpBj7vjmIba+vwL6d5VhX3hPrkQg95GDFoRngDUP4q474i4mxPuKPiIg0g6PJyGP0FXr8+r8/kP3ZWVzethfPV8xELK4HLaekCPzcYRzavnEfWt/eApKPuUHyNuCIPyIiVeJoMlInK4FH2YUyrJm7C9mLrmD5gdY4pW+PkcjCUoxH1fSIESjCnTunA1cTAJ8bHC8TR/wREVENWDNEzmMuCWVMDC69OBM5u1oge7kPfjiegEuob1zdAOdwyLc1wnTmkyMaOzrn57M2h4hIY1gzRNUpubknK0vurFwlttYfO4Y64x5EJpZiGeQ+OlE+J5Hc5iBS7quLpE7n4H/7acv7FQIoKJCPm7U7RETkAgyG1MJCrQvmzvV8R2CdDuKppwAhqtXu+ECe+uIjTEDb3qFIfjzSdAb4DCZHJDso+QcBEakWJ2pVA0OtS+VACJBz6IwaJa/3gIqyCqyduwMftfkQ0rFj5pu5APhAIBIn8cZMX3R/sC18/Cp97WxNesjkiJSVBcTFAUlJwJgx8t+4OI99/4nIezAYUjqdTq4RMte1y7AsLU3ezg0un7mM7Km/4uGW69G0TjEGpHXCzwcjbHuxudqdxES5hkuyEEpJkjz5LZMjaptCfxAQkXdgM5nSrV9f/QZQmRv61JzefQpbnluMPRuLsbK4O37CIGP+n4bSeXRoegawpRXLXO2ONyZHZFOOc1n7QSBJ8g+C5GSeZyJyCGuGlM7WvjJO7lPz55ojeC85Dy/XeQ9l7bth6A9P4ZniV/EjhqIAsfgs7nXkvr8DRZfrY2rBuNrV7hiSI0ZHmy6PiZGXe7pPlD3YlON89vwgICJyAGuGlM5NfWrMzQA/EluxFM+iav6fSKkQ/zgyDWjWDgjqJC+sbe1Oaqr8y17NNSoWRtQZm3LUFtgphYd+EBCRdjDPkNLpdHLNgrUJRx3Iw1N9Bvgo4zp/lOOY1AxNRJHt+X/MjXiLjdXG1BeGz8lSDQbzJTkuL0+uYbMmN5fpF4i8jLvu3wyG1MBQ4wCYD4iWLLm+3oqLJy7ih3d2IztL//cM8KHGdXVQiqHRO5E8XIfkxPMIuf8O6zusegPSan8Z3rBdx4U/CIhI2Zh0ka6zNOGowdNPAz4+FmtfTu44hW/e2Y/slYYZ4K/P8h4uncaIlvuQck+g6Qzwjub/0erUF2zKcR1v7GRPRIrCYEgtUlPlX8h33119nZk+Kfu++wvZHxxFzi+NsOlSewDXh7/f6H8YIzsdRvLDYej1SDv4Bpjp2GxvXyWt1ggZMF+Sa1n6QRATo41mWCJyKTaTqYWVPilCknC1QQSmt/wSy3bEY//VFibre9TdjZQ+Z5A8PgZtht9gfQZ4e5omcnKUmx3bXdiU4x5aD7qJNIZ9hpzIK4IhG/ukDEAu1mIA/HEVtzTaiZTBl3HHszchqktT+9/TUl8lQ9PE0qXyX3MjqCpvo5WAyJbzpZVzQUTkBO66fzPPkFrY2NfkzkZrsWjSBpw+cgU/nOmGJ9L7ORYIAdbz/yQnKyo7tsd5U74kIiINYc2QChT8egJbnv8aI9c9bX1jV4xWstQ0wRFU5rEph4jIKTiaTMOEXmD3soPI/vgEsjdGYNuVNvDBRBzGu4jGcfighj4prpjDy9IIMY6gMk+rI+qIiFSKwZBCVJRVYMN//kD2wvPI3tkC+RU3AbgJACBBj5tDdmPbjY8hZtt05Qwv5ggqIiLyAgyGPOjymctY9d4uZC+5im/+bIuzoqNxXSDKMDjidyTfdhUjnmuN8HYdAXQEshKUM7zYMOO8tRFUnHGeiIgUjH2G3OzM/rP49l97kb3CHysL2+MK6hjXNZTOY0T8H0i+0w+DJyegXtN65neipD4pHEFFREQuwqH1TuTpYOivvKPIfv8v5KxtgJ+L20OP64FLc99jSGn/J5LHhiBxXHv4Bamwss4Tc5IpKSCsSsllIyJSEQZDTuTuYKjyDPA5W6Kwq+wmk/WdgvchpWchkp+IRMe7brKeAFEN3BkAmAu+lJLkUcllIyJSGQZDTuSOk1l5Bvjle25Ege56rhlfVKBfg11IuaUEdzx9A+L6xrikDJpgaJZTYpJHJZeNiEiFGAw5katOZuUZ4L872g4XRAPjujooxW1Ru5AyogLDnmuHsBsaOu19NcvKlCQenfJCyWUjIlIp5hlSqMKdRVg+e5/ZGeCbSKdxh8kM8L08WFIvtH695WADkGtkCgrk7dyd50fJZSMiohqpbjqOWbNmQZIkpKWlue099333F94emofe9XchqmNjPP5VP3x/ujuuIhA3+h/Gs93y8PPHO3GyLAz/3Z+I4a/1QHBYsNvKpxlKTvKo5LIREVGNVFUztHnzZsyfPx8dOnRw6fvoK/T4beEeZP/3DLK3N/t7Bvjrs8D3qLsbyTefQcoEwwzwcS4tD/3N1uSNBw+6thzmMAElEZFqqSYYunTpEu677z785z//wRtvvOH0/ZeXlGPNnJ3IzriC5QdaoVCfYFxnmAE+edBl3PFMS0R3S6hhT+QyiYnyJKjHj9e83X/+A7z0knv65hhG0R0/DjRuDJw5Y347JqAkIlIs1QRD48ePx7Bhw3DrrbdaDYbKy8tRXl5ufF5SUmJ2u/OHzmLLM+nY8/M5/HCuO1ZiiDEHUH2UYFiz3UhOBoY+2w6hzbo572DIMb6+wD//CUybVvN2x465p2+OuWH05nhquhQiIrKJKoKhRYsWYdu2bdi8ebNN28+aNQszZswwu67g1xPIeecgzv7wG/5xaS4G4TgGAXgKwAlE4tvYcWj++G0YMLE9AkNudt5BeJq3JAJs2dK27VzdN8fSMHpzPDVdChER2UTxwVBBQQGeeuoprFy5EkFBQTa9ZurUqZg8ebLxeUlJCWJjY9G/6T7suNIDI7EJS/ECUGX290ipEP889irQpi0Q4kU1Qd6UCFAJfXN0Ovl8WgqEJEluMnv/fblZT62BJxGRRig+z1B2djZGjhwJ30o3E51OB0mS4OPjg/LycpN15hjyFADF8EUwTkixaCJOwWzeZ2/LB+NtiQAN+XysTQ7rys8vLw9ISrK+XW4uh9ETEdWCu/IMKX5o/cCBA7Fr1y7s2LHD+OjWrRvuu+8+7Nixw2ogVNmHozeg6H8rEG4pEAJM88GoXU01GIZlaWnydmrh6yvXaAHXAzoDd/XN4TB6IiKvovhgqH79+khISDB51K1bF40aNUJCgn2juh749GaEBV2xbWNvuJHZkwhQTVJT5Rqt6GjT5TEx7qnpUkJTHREROY3i+ww5nZZuZN5cg5GaCiQne6ZTeGKiHHhZa6rjMHoiIlVQZTCUl5fn+Iu1dCPz9sDP19czfXIMTXWjRsnfl8rfIw6jJyJSHcU3kzmdEvqcuJpOJ3fyNSQCtESSgNhY7wj83M3TTXVEROQ0ih9N5gxme6ObG24eG6v+fDD2JgLkjbt2vCV/ExGRArlrNJl2gyHA+25k9iQC9IbAj4iIvJq7giFV9hlyGk/1OXEFb0gE6G3BKRERqYK2gyFvYssw+tOn5UBIiQGgN2XJJiIiVdFeB2pvpeZh9IbmvarB3PHj8vKsLM+Ui4iINIHBkJoYRollZMh/K2eOVuswem/Mkk1ERKrCYEgtsrLkObmSkoAxY+S/cXHXa00M+ZOqpgswUOowem/Nkk1ERKrBYEgNbGlGUmv+JDU37xERkVdgMKR09jQjqTERoFqb94iIyGtoO8+QGuTlyU1i1uTmXh8lpqYh6jqd3NxnbXqU/HzlHgMREbkE8wyRzJFmJDXlT+I8X0RE5GFsJlM6LTQjqbF5j4iIvAabyZROS81IamreIyIil2MzGcm01IykpuY9IiLyGmwmUwM2IxEREbkMa4aczVVNPampQHIym5GIiIicjMGQM7l6slE2IxERETkdm8mchZONEhERqRKDIWfgZKNERESqxWDIGTjZKBERkWoxGHIGTjZKRESkWgyGnEELWaKJiIi8FIMhZ0hMlEeNGZIgViVJQGysvB0REREpCoMhZzBkiQaqB0TeliWaiIjIyzAYchZmiSYiIlIlJl10JmaJJiIiUh0GQ87GLNFERESqwmYyIiIi0jQGQ0RERKRpDIaIiIhI0xgMERERkaYxGCIiIiJNYzBEREREmsZgiIiIiDSNwRARERFpGoMhIiIi0jQGQ0RERKRpig+GZs2ahe7du6N+/foIDw9HSkoK9u/f7+liERERkZdQfDC0du1ajB8/Hps2bcKqVatQUVGBwYMHo7S01NNFIyIiIi8gCSGEpwthj9OnTyM8PBxr165Fv379bHpNSUkJQkNDUVxcjJCQEBeXkIiIiJzBXfdv1c1aX1xcDAAICwuzuE15eTnKy8uNz0tKSlxeLiIiIlInxTeTVSaEwOTJk9G3b18kJCRY3G7WrFkIDQ01PmJjY91YSiIiIlITVTWTjR8/HitWrMDPP/+MmJgYi9uZqxmKjY1lMxkREZGKsJmsiokTJ2L58uVYt25djYEQAAQGBiIwMNBNJSMiIiI1U3wwJITAxIkTsWzZMuTl5SE+Pt7TRSIiIiIvovhgaPz48UhPT0dOTg7q16+PwsJCAEBoaCiCg4M9XDoiIiJSO8X3GZIkyezyBQsW4KGHHrJpHxxaT0REpD7sM/Q3hcdqREREpHKqGlpPRERE5GwMhoiIiEjTGAwRERGRpjEYIiIiIk1jMERERESaxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTGAwRERGRpjEYIiIiIk1jMERERESaxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTGAwRERGRpjEYIiIiIk1jMERERESaxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTGAwRERGRpjEYIiIiIk1jMERERESaxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTGAwRERGRpjEYIiIiIk1jMERERESaxmCIiIiINE01wdDHH3+M+Ph4BAUFoWvXrli/fr2ni0REREReQBXB0OLFi5GWloaXXnoJ27dvR2JiIoYOHYqjR496umhERESkcpIQQni6ENb07NkTXbp0wSeffGJc1qZNG6SkpGDWrFlWX19SUoLQ0FAUFxcjJCTElUUlIiIiJ3HX/VvxNUNXr17F1q1bMXjwYJPlgwcPxoYNGzxUKiIiIvIWfp4ugDVnzpyBTqdDRESEyfKIiAgUFhaafU15eTnKy8uNz4uLiwHIESYRERGpg+G+7epGLMUHQwaSJJk8F0JUW2Ywa9YszJgxo9ry2NhYl5SNiIiIXOfs2bMIDQ112f4VHww1btwYvr6+1WqBioqKqtUWGUydOhWTJ082Pr9w4QKaN2+Oo0ePuvRkKk1JSQliY2NRUFCgqb5SPG4etxbwuHncWlBcXIxmzZohLCzMpe+j+GAoICAAXbt2xapVqzBy5Ejj8lWrViE5OdnsawIDAxEYGFhteWhoqKa+RAYhISE8bg3hcWsLj1tbtHrcPj6u7eKs+GAIACZPnoyxY8eiW7du6N27N+bPn4+jR4/iiSee8HTRiIiISOVUEQzdc889OHv2LF577TWcPHkSCQkJ+O6779C8eXNPF42IiIhUThXBEACMGzcO48aNc+i1gYGBmDZtmtmmM2/G4+ZxawGPm8etBTxu1x63KpIuEhEREbmK4pMuEhEREbkSgyEiIiLSNAZDREREpGkMhoiIiEjTVBkMffzxx4iPj0dQUBC6du2K9evX17j92rVr0bVrVwQFBaFFixb49NNPq22TmZmJtm3bIjAwEG3btsWyZctcVXyH2XPcWVlZGDRoEJo0aYKQkBD07t0bP/74o8k2CxcuhCRJ1R5lZWWuPhS72HPceXl5Zo9p3759Jtt52+f90EMPmT3udu3aGbdRw+e9bt06jBgxAlFRUZAkCdnZ2VZf4w3Xt73H7S3Xt73H7S3Xt73H7S3X96xZs9C9e3fUr18f4eHhSElJwf79+62+zh3XuOqCocWLFyMtLQ0vvfQStm/fjsTERAwdOhRHjx41u31+fj5uv/12JCYmYvv27XjxxRcxadIkZGZmGrfZuHEj7rnnHowdOxa///47xo4di7vvvhu//vqruw7LKnuPe926dRg0aBC+++47bN26FUlJSRgxYgS2b99usl1ISAhOnjxp8ggKCnLHIdnE3uM22L9/v8kxtWzZ0rjOGz/vuXPnmhxvQUEBwsLCcNddd5lsp/TPu7S0FB07dsS8efNs2t5brm97j9tbrm97j9tA7de3vcftLdf32rVrMX78eGzatAmrVq1CRUUFBg8ejNLSUouvcds1LlSmR48e4oknnjBZ1rp1azFlyhSz2z///POidevWJssef/xx0atXL+Pzu+++W9x2220m2wwZMkTce++9Tip17dl73Oa0bdtWzJgxw/h8wYIFIjQ01FlFdAl7jzs3N1cAEOfPn7e4Ty183suWLROSJInDhw8bl6nh864MgFi2bFmN23jL9V2ZLcdtjhqv78psOW5vub4rc+Tz9obrWwghioqKBACxdu1ai9u46xpXVc3Q1atXsXXrVgwePNhk+eDBg7Fhwwazr9m4cWO17YcMGYItW7bg2rVrNW5jaZ/u5shxV6XX63Hx4sVqk91dunQJzZs3R0xMDIYPH17tl6Un1ea4O3fujMjISAwcOBC5ubkm67TweX/22We49dZbq2VpV/Ln7QhvuL6dQY3Xd22o+fp2Bm+5vouLiwGgxklY3XWNqyoYOnPmDHQ6XbXZ6iMiIqrNam9QWFhodvuKigqcOXOmxm0s7dPdHDnuqt59912Ulpbi7rvvNi5r3bo1Fi5ciOXLlyMjIwNBQUHo06cPDh486NTyO8qR446MjMT8+fORmZmJrKwstGrVCgMHDsS6deuM23j7533y5El8//33ePTRR02WK/3zdoQ3XN/OoMbr2xHecH3Xlrdc30IITJ48GX379kVCQoLF7dx1jatmOo7KJEkyeS6EqLbM2vZVl9u7T09wtIwZGRmYPn06cnJyEB4eblzeq1cv9OrVy/i8T58+6NKlCz788EN88MEHzit4Ldlz3K1atUKrVq2Mz3v37o2CggK888476Nevn0P79BRHy7hw4UI0aNAAKSkpJsvV8nnby1uub0ep/fq2hzdd347ylut7woQJ2LlzJ37++Wer27rjGldVzVDjxo3h6+tbLdorKiqqFhUaNG3a1Oz2fn5+aNSoUY3bWNqnuzly3AaLFy/GI488gq+//hq33nprjdv6+Pige/fuivklUZvjrqxXr14mx+TNn7cQAv/73/8wduxYBAQE1Lit0j5vR3jD9V0bar6+nUVt13dteMv1PXHiRCxfvhy5ubmIiYmpcVt3XeOqCoYCAgLQtWtXrFq1ymT5qlWrcPPNN5t9Te/evattv3LlSnTr1g3+/v41bmNpn+7myHED8i/Ghx56COnp6Rg2bJjV9xFCYMeOHYiMjKx1mZ3B0eOuavv27SbH5K2fNyCP1jh06BAeeeQRq++jtM/bEd5wfTtK7de3s6jt+q4NtV/fQghMmDABWVlZWLNmDeLj462+xm3XuM1drRVi0aJFwt/fX3z22Wdiz549Ii0tTdStW9fYq37KlCli7Nixxu3/+usvUadOHfH000+LPXv2iM8++0z4+/uLpUuXGrf55ZdfhK+vr3jrrbfE3r17xVtvvSX8/PzEpk2b3H58lth73Onp6cLPz0989NFH4uTJk8bHhQsXjNtMnz5d/PDDD+LPP/8U27dvFw8//LDw8/MTv/76q9uPzxJ7j/v9998Xy5YtEwcOHBC7d+8WU6ZMEQBEZmamcRtv/LwN7r//ftGzZ0+z+1TD533x4kWxfft2sX37dgFAvPfee2L79u3iyJEjQgjvvb7tPW5vub7tPW5vub7tPW4DtV/fTz75pAgNDRV5eXkm39vLly8bt/HUNa66YEgIIT766CPRvHlzERAQILp06WIyLO/BBx8U/fv3N9k+Ly9PdO7cWQQEBIi4uDjxySefVNvnkiVLRKtWrYS/v79o3bq1ycWlFPYcd//+/QWAao8HH3zQuE1aWppo1qyZCAgIEE2aNBGDBw8WGzZscOMR2cae43777bfFDTfcIIKCgkTDhg1F3759xYoVK6rt09s+byGEuHDhgggODhbz5883uz81fN6GodOWvrfeen3be9zecn3be9zecn078j33huvb3DEDEAsWLDBu46lrXPq7gERERESapKo+Q0RERETOxmCIiIiINI3BEBEREWkagyEiIiLSNAZDREREpGkMhoiIiEjTGAwRERGRpjEYIiIiIk1jMERERESaxmCIiIiINI3BEBGpUkZGBoKCgnD8+HHjskcffRQdOnRAcXGxB0tGRGrDucmISJWEEOjUqRMSExMxb948zJgxA//973+xadMmREdHe7p4RKQifp4uABGRIyRJwptvvolRo0YhKioKc+fOxfr16xkIEZHdWDNERKrWpUsX/PHHH1i5ciX69+/v6eIQkQqxzxARqdaPP/6Iffv2QafTISIiwtPFISKVYs0QEanStm3bMGDAAHz00UdYtGgR6tSpgyVLlni6WESkQuwzRESqc/jwYQwbNgxTpkzB2LFj0bZtW3Tv3h1bt25F165dPV08IlIZ1gwRkaqcO3cOffr0Qb9+/fDvf//buDw5ORnl5eX44YcfPFg6IlIjBkNERESkaexATURERJrGYIiIiIg0jcEQERERaRqDISIiItI0BkNERESkaQyGiIiISNMYDBEREZGmMRgiIiIiTWMwRERERJrGYIiIiIg0jcEQERERaRqDISIiItK0/wd0Zw6C7VjIeQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_5_1.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "# Importing various packages\n",
    "from math import exp, sqrt\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.inv(X.T @ X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = 2.0/n*X.T @ ((X @ theta)-y)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "# Can you figure out a better way of setting up the contributions to each batch?\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (2.0/M)* xi.T @ ((xi @ theta)-yi)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        theta = theta - eta*gradients\n",
    "print(\"theta from own sdg\")\n",
    "print(theta)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8e7110b",
   "metadata": {
    "editable": true
   },
   "source": [
    "In the above code, we have use replacement in setting up the\n",
    "mini-batches. The discussion\n",
    "[here](https://sebastianraschka.com/faq/docs/sgd-methods.html) may be\n",
    "useful."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74a7859a",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Momentum based GD\n",
    "\n",
    "The stochastic gradient descent (SGD) is almost always used with a\n",
    "*momentum* or inertia term that serves as a memory of the direction we\n",
    "are moving in parameter space.  This is typically implemented as\n",
    "follows"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "619df8e8",
   "metadata": {
    "editable": true
   },
   "source": [
    "$$\n",
    "\\mathbf{v}_{t}=\\gamma \\mathbf{v}_{t-1}+\\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t) \\nonumber\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c759cfc4",
   "metadata": {
    "editable": true
   },
   "source": [
    "<!-- Equation labels as ordinary links -->\n",
    "<div id=\"_auto1\"></div>\n",
    "\n",
    "$$\n",
    "\\begin{equation} \n",
    "\\boldsymbol{\\theta}_{t+1}= \\boldsymbol{\\theta}_t -\\mathbf{v}_{t},\n",
    "\\label{_auto1} \\tag{1}\n",
    "\\end{equation}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a4571f9",
   "metadata": {
    "editable": true
   },
   "source": [
    "where we have introduced a momentum parameter $\\gamma$, with\n",
    "$0\\le\\gamma\\le 1$, and for brevity we dropped the explicit notation to\n",
    "indicate the gradient is to be taken over a different mini-batch at\n",
    "each step. We call this algorithm gradient descent with momentum\n",
    "(GDM). From these equations, it is clear that $\\mathbf{v}_t$ is a\n",
    "running average of recently encountered gradients and\n",
    "$(1-\\gamma)^{-1}$ sets the characteristic time scale for the memory\n",
    "used in the averaging procedure. Consistent with this, when\n",
    "$\\gamma=0$, this just reduces down to ordinary SGD as discussed\n",
    "earlier. An equivalent way of writing the updates is"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bebfdd9",
   "metadata": {
    "editable": true
   },
   "source": [
    "$$\n",
    "\\Delta \\boldsymbol{\\theta}_{t+1} = \\gamma \\Delta \\boldsymbol{\\theta}_t -\\ \\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t),\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9e635e8",
   "metadata": {
    "editable": true
   },
   "source": [
    "where we have defined $\\Delta \\boldsymbol{\\theta}_{t}= \\boldsymbol{\\theta}_t-\\boldsymbol{\\theta}_{t-1}$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb544f8b",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Algorithms and codes for Adagrad, RMSprop and Adam\n",
    "\n",
    "The algorithms we have implemented are well described in the text by [Goodfellow, Bengio and Courville, chapter 8](https://www.deeplearningbook.org/contents/optimization.html).\n",
    "\n",
    "The codes which implement these algorithms are discussed after our presentation of automatic differentiation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d088ad85",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Practical tips\n",
    "\n",
    "* **Randomize the data when making mini-batches**. It is always important to randomly shuffle the data when forming mini-batches. Otherwise, the gradient descent method can fit spurious correlations resulting from the order in which data is presented.\n",
    "\n",
    "* **Transform your inputs**. Learning becomes difficult when our landscape has a mixture of steep and flat directions. One simple trick for minimizing these situations is to standardize the data by subtracting the mean and normalizing the variance of input variables. Whenever possible, also decorrelate the inputs. To understand why this is helpful, consider the case of linear regression. It is easy to show that for the squared error cost function, the Hessian of the cost function is just the correlation matrix between the inputs. Thus, by standardizing the inputs, we are ensuring that the landscape looks homogeneous in all directions in parameter space. Since most deep networks can be viewed as linear transformations followed by a non-linearity at each layer, we expect this intuition to hold beyond the linear case.\n",
    "\n",
    "* **Monitor the out-of-sample performance.** Always monitor the performance of your model on a validation set (a small portion of the training data that is held out of the training process to serve as a proxy for the test set. If the validation error starts increasing, then the model is beginning to overfit. Terminate the learning process. This *early stopping* significantly improves performance in many settings.\n",
    "\n",
    "* **Adaptive optimization methods don't always have good generalization.** Recent studies have shown that adaptive methods such as ADAM, RMSPorp, and AdaGrad tend to have poor generalization compared to SGD or SGD with momentum, particularly in the high-dimensional limit (i.e. the number of parameters exceeds the number of data points). Although it is not clear at this stage why these methods perform so well in training deep neural networks, simpler procedures like properly-tuned SGD may work as well or better in these applications.\n",
    "\n",
    "Geron's text, see chapter 11, has several interesting discussions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "937063d3",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Using Automatic differentation with OLS\n",
    "\n",
    "We conclude the part on optmization by showing how we can make codes\n",
    "for linear regression and logistic regression using **autograd**. The\n",
    "first example shows results with ordinary leats squares."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e08215a6",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.23636536]\n",
      " [2.77184871]]\n",
      "Eigenvalues of Hessian Matrix:[0.30125775 4.66878535]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own gd\n",
      "[[4.23636536]\n",
      " [2.77184871]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHFCAYAAADxOP3DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABU2klEQVR4nO3deXhU1fkH8O9km7AlkAAhGyQihARkkX2JgMKMFigYccGKYNWKoEixKtRWoZVS+VmXCkrdoNUGKRBwq84gJIAIihC0soMsYQlhTUhCtsn5/THOkElmn7kz9879fp4nD+TeO3fOncnNvDnnPe/RCCEEiIiIiFQqLNgNICIiIgomBkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNEMrR8+XJoNBrrV0REBBITE3HPPffg0KFDQWvXvHnzoNFogvb8SnDs2DFoNBq89NJLwW4KEbkpItgNICLHli1bhm7duqGqqgpbt27FggULkJ+fj/3796NNmzbBbh4RUUhgMEQkYz169EC/fv0AACNGjIDJZMLzzz+PdevW4YEHHghy6yiYamtrrb2GROQbDpMRKYglMDp79qx1W1VVFZ588kn07t0bsbGxiIuLw+DBg/HRRx81ebxGo8Fjjz2G999/H5mZmWjevDl69eqFTz/9tMmxn332GXr37g2tVov09HSHwz5VVVWYO3cu0tPTERUVheTkZMyYMQOXL1+2OS4tLQ1jx47Fp59+ij59+qBZs2bIzMy0Pvfy5cuRmZmJFi1aYMCAAfjuu+9cvh6W4cT8/Hw8+uijaNu2LeLj45GTk4PTp083ufZ58+Y1OUdaWhqmTp3a5JwbN27Eww8/jPj4eMTExOD+++9HRUUFiouLcdddd6F169ZITEzE7373O9TW1jY5b319PRYsWICOHTsiOjoa/fr1w4YNG5ocd+jQIdx7771o3749tFotMjMzsWTJEptjCgoKoNFo8P777+PJJ59EcnIytFotDh8+jMrKSvzud79Deno6oqOjERcXh379+mHFihUuXz8iMuOfFEQKcvToUQBA165drduqq6tx8eJF/O53v0NycjJqamrw5ZdfIicnB8uWLcP9999vc47PPvsMO3bswJ/+9Ce0bNkSixYtwu23344DBw7guuuuAwBs2LAB48ePx+DBg/Hhhx/CZDJh0aJFNkEYAAghMGHCBGzYsAFz585FdnY2fvjhBzz//PPYtm0btm3bBq1Waz3++++/x9y5c/Hss88iNjYW8+fPR05ODubOnYsNGzbgL3/5CzQaDZ555hmMHTsWR48eRbNmzVy+Lg899BDGjBmD3NxcFBUV4amnnsJ9992HjRs3ev1aP/TQQ8jJycGHH36IwsJC/P73v0ddXR0OHDiAnJwc/OY3v8GXX36JF198EUlJSZg9e7bN4xcvXoxOnTrh1VdfRX19PRYtWoTbbrsNmzZtwuDBgwEAe/fuxZAhQ9CxY0f87W9/Q4cOHWAwGDBz5kycP38ezz//vM05586di8GDB2Pp0qUICwtD+/btMXv2bLz//vt44YUX0KdPH1RUVODHH3/EhQsXvL52ItURRCQ7y5YtEwDE9u3bRW1trbhy5Yr44osvRIcOHcRNN90kamtrHT62rq5O1NbWigcffFD06dPHZh8AkZCQIMrKyqzbiouLRVhYmFi4cKF128CBA0VSUpK4evWqdVtZWZmIi4sTDX9tfPHFFwKAWLRokc3zrFy5UgAQb731lnVbp06dRLNmzcTJkyet23bv3i0AiMTERFFRUWHdvm7dOgFAfPzxx269TtOnT7fZvmjRIgFAnDlzxuban3/++Sbn6NSpk5gyZUqTcz7++OM2x02YMEEAEC+//LLN9t69e4sbb7zR+v3Ro0cFAIev36hRo6zb9Hq9SElJEaWlpTbnfOyxx0R0dLS4ePGiEEKI/Px8AUDcdNNNTdrfo0cPMWHChCbbich9HCYjkrFBgwYhMjISrVq1wq233oo2bdrgo48+apInsmrVKgwdOhQtW7ZEREQEIiMj8e6772Lfvn1Nzjly5Ei0atXK+n1CQgLat2+P48ePAwAqKiqwY8cO5OTkIDo62npcq1atMG7cOJtzWXpeGg4zAcCdd96JFi1aNBkW6t27N5KTk63fZ2ZmAjDnQzVv3rzJdkubXPnlL39p833Pnj09erw9Y8eOtfne0qYxY8Y02W7veRy9fps3b4bJZEJVVRU2bNiA22+/Hc2bN0ddXZ316xe/+AWqqqqwfft2m3PecccdTZ5nwIAB+PzzzzFnzhwUFBTg6tWrXl8zkVoxGCKSsX/961/YsWMHNm7ciEceeQT79u3DpEmTbI7Jy8vDXXfdheTkZHzwwQfYtm0bduzYgV//+teoqqpqcs74+Pgm27RarfVD9NKlS6ivr0eHDh2aHNd424ULFxAREYF27drZbNdoNOjQoUOToZq4uDib76Oiopxut9d+expfk2VozpfAwJO22muno9evpqYG5eXluHDhAurq6vD6668jMjLS5usXv/gFAOD8+fM2j09MTGxyzr///e945plnsG7dOowcORJxcXGYMGFCUEswECkNc4aIZCwzM9OaND1y5EiYTCa88847WL16NSZOnAgA+OCDD5Ceno6VK1fa1ACqrq726jnbtGkDjUaD4uLiJvsab4uPj0ddXR3OnTtnExAJIVBcXIz+/ft71QYpaLVau6+JVLk1jl6/qKgotGzZEpGRkQgPD8fkyZMxY8YMu+dIT0+3+d5ejacWLVpg/vz5mD9/Ps6ePWvtJRo3bhz279/vn4shCnHsGSJSkEWLFqFNmzZ47rnnUF9fD8D8ARkVFWXzQVlcXGx3Npk7LLO58vLybHo8rly5gk8++cTm2FtuuQWAOSBraM2aNaioqLDul4O0tDT88MMPNts2btyI8vJySZ7P0euXnZ2N8PBwNG/eHCNHjkRhYSF69uyJfv36Nfmy14vnTEJCAqZOnYpJkybhwIEDqKys9PdlEYUk9gwRKUibNm0wd+5cPP3008jNzcV9992HsWPHIi8vD9OnT8fEiRNRVFSEP//5z0hMTPR6qOTPf/4zbr31VowePRpPPvkkTCYTXnzxRbRo0QIXL160Hjd69Gjo9Xo888wzKCsrw9ChQ62zyfr06YPJkyf769J9NnnyZPzxj3/Ec889h+HDh2Pv3r1YvHgxYmNjJXm+8PBwjB49GrNnz0Z9fT1efPFFlJWVYf78+dZjXnvtNQwbNgzZ2dl49NFHkZaWhitXruDw4cP45JNP3JoNN3DgQIwdOxY9e/ZEmzZtsG/fPrz//vsYPHiwTR4WETnGYIhIYR5//HEsXrwYf/rTnzBp0iQ88MADKCkpwdKlS/Hee+/huuuuw5w5c3Dy5EmbD15PjB49GuvWrcMf/vAH3H333ejQoQOmT5+Oq1ev2pxTo9Fg3bp1mDdvHpYtW4YFCxagbdu2mDx5Mv7yl7/YTKsPtqeeegplZWVYvnw5XnrpJQwYMAD/+c9/MH78eEme77HHHkNVVRVmzpyJkpISdO/eHZ999hmGDh1qPSYrKwu7du3Cn//8Z/zhD39ASUkJWrdujS5duljzhly5+eab8fHHH+OVV15BZWUlkpOTcf/99+PZZ5+V5LqIQpFGCCGC3QgiIiKiYGHOEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVTRZ2h+vp6nD59Gq1atbJbzp6IiIjkRwiBK1euICkpCWFh0vXfqCIYOn36NFJTU4PdDCIiIvJCUVERUlJSJDu/KoKhVq1aATC/mDExMUFuDREREbmjrKwMqamp1s9xqagiGLIMjcXExDAYIiIiUhipU1yYQE1ERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqFvRgaPPmzRg3bhySkpKg0Wiwbt06h8c+8sgj0Gg0ePXVVwPWPiIiIgptQQ+GKioq0KtXLyxevNjpcevWrcM333yDpKSkALWMiIiI1CAi2A247bbbcNtttzk95tSpU3jsscdgMBgwZsyYALWMiIiI1CDoPUOu1NfXY/LkyXjqqafQvXv3YDeHiIiIQkzQe4ZcefHFFxEREYGZM2e6/Zjq6mpUV1dbvy8rK5OiaURERBQCZN0ztHPnTrz22mtYvnw5NBqN249buHAhYmNjrV+pqakStpKIiIiUTNbB0JYtW1BSUoKOHTsiIiICEREROH78OJ588kmkpaU5fNzcuXNRWlpq/SoqKgpco4mIiEhRZD1MNnnyZIwaNcpmm16vx+TJk/HAAw84fJxWq4VWq5W6eURERBQCgh4MlZeX4/Dhw9bvjx49it27dyMuLg4dO3ZEfHy8zfGRkZHo0KEDMjIyAt1UIiIiCkFBD4a+++47jBw50vr97NmzAQBTpkzB8uXLg9QqIiIiUougB0MjRoyAEMLt448dOyZdY4iIiEh1ZJ1ATURERCQ1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhULejB0ObNmzFu3DgkJSVBo9Fg3bp11n21tbV45plncMMNN6BFixZISkrC/fffj9OnTwevwURERBRSgh4MVVRUoFevXli8eHGTfZWVldi1axf++Mc/YteuXcjLy8PBgwfxy1/+MggtJSIiolCkEUKIYDfCQqPRYO3atZgwYYLDY3bs2IEBAwbg+PHj6Nixo1vnLSsrQ2xsLEpLSxETE+On1hIREZGUAvX5HSHZmSVSWloKjUaD1q1bOzymuroa1dXV1u/LysoC0DIiIiJSoqAPk3miqqoKc+bMwb333us0Qly4cCFiY2OtX6mpqQFsJRERESmJYoKh2tpa3HPPPaivr8cbb7zh9Ni5c+eitLTU+lVUVBSgVhIREZHSKGKYrLa2FnfddReOHj2KjRs3uhw31Gq10Gq1AWodERERKZnsgyFLIHTo0CHk5+cjPj4+2E0iIiKiEBL0YKi8vByHDx+2fn/06FHs3r0bcXFxSEpKwsSJE7Fr1y58+umnMJlMKC4uBgDExcUhKioqWM0mIiKiEBH0qfUFBQUYOXJkk+1TpkzBvHnzkJ6ebvdx+fn5GDFihFvPwan1REREyqOaqfUjRoyAs3hMRmWQiIiIKAQpZjYZERERkRQYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUigt0AIiIikgmTCdiyBThzBkhMBLKzgfDwYLdKcgyGiIiICMjLA554Ajh58tq2lBTgtdeAnJzgtSsAOExGRESkdnl5wMSJtoEQAJw6Zd6elxecdgUIgyEiIiI1M5nMPUJCNN1n2TZrlvm4EMVgiIiISM22bGnaI9SQEEBRkfm4EMVgiIiISM3OnPHvcQrEYIiIiEjNEhP9e5wCMRgiIiJSs+xs86wxjcb+fo0GSE01HxeiGAwRERGpWXi4efo80DQgsnz/6qshXW+IwRAREZHa5eQAq1cDycm221NSzNsb1hkymYCCAmDFCvO/ITDLjEUXiYiIyBzwjB/vvAJ1iBZm1Ahhr7BAaCkrK0NsbCxKS0sRExMT7OYQEREpj6UwY+OwwTKU1rgHyQ8C9fnNYTIiIiJyLsQLMzIYIiIiIudCvDAjc4aIiIjUzJ2V6kO8MCODISIiUi53PsjJMXcTokO8MCOHyYiISJny8oC0NGDkSODee83/pqWF/ArrfuPJSvUhXpiRwRARESmPJx/k1JSnCdFSFmaUQd0iBkNERKQsIT6zKSC8SYj2pDCju2TSu8ecISIiUhZPPshHjAhYs2TLXl6VtwnR7hRmdJejukWW3r3Vq4FRozw/rxcYDBERkbKE+Mwmv3KUIP3ww+493l5CdHi470Gmq949jcbcu/f99749j5uCPky2efNmjBs3DklJSdBoNFi3bp3NfiEE5s2bh6SkJDRr1gwjRozAnj17gtNYIiIKvhCf2eQ3zvKq5s0D4uODlxDtbu/e119L8/yNBD0YqqioQK9evbB48WK7+xctWoSXX34Zixcvxo4dO9ChQweMHj0aV65cCXBLiYhIFuQ4s8nXJGB/JxG7k1dl+X8wVqp3t9euuFia528k6MHQbbfdhhdeeAE5dhKvhBB49dVX8eyzzyInJwc9evTAP//5T1RWViI3NzcIrSUioqDzZWaTFDOXfE0CliKJ2J2elwsXgPnzmyZEt21rDqTi4qRLQne3165DB2mev5GgB0POHD16FMXFxdDpdNZtWq0Ww4cPx9dOus6qq6tRVlZm80VERCHEm5lNUgQdvk7xl6pEgLs9L126AMeOAfn55hydtm2Bc+fMwaSUM7vc7d0bMsT/z22HrIOh4p+7xxISEmy2JyQkWPfZs3DhQsTGxlq/UlNTJW0nEZEqyKAejI2cnGsf5Lm55n+PHnUcCPk76PB1ir+UJQI8yasKDwcuXjT3tp0/b7tfqrpNUtYt8oKsgyELTaMXSgjRZFtDc+fORWlpqfWrqKhI6iYSEYU2mdSDacIys2nSJPO/jobGpAg6fF28VMrFT131vADmBGqTCaipCU7dJinqFnlJ1sFQh5/HChv3ApWUlDTpLWpIq9UiJibG5ouIiLyk9GrPUgUdvk7xl7JEQMOeF0cuXDDX8UlJCd6K9J707klI1sFQeno6OnTogPXr11u31dTUYNOmTRgSoHFEIiJVC4Vqz1IFHb5O8Ze6REBODvC737keajp3zr3zSVW3yZ3ePYkFPRgqLy/H7t27sXv3bgDmpOndu3fjxIkT0Gg0mDVrFv7yl79g7dq1+PHHHzF16lQ0b94c9957b3AbTkSkBlIO5QSKVEGHr1P8pS4RkJcHvPSS/wLVANdtqq+rx+6VBwLyXEGvQP3dd99h5MiR1u9nz54NAJgyZQqWL1+Op59+GlevXsX06dNx6dIlDBw4EEajEa1atQpWk4mI1CMUqj1bgo5Tp+z3cGk05v2eBh2WoaiJE83naHhud5KAfX28M8569Dzl7evjhdO7irH+zcMwrNdg/YkMnBcZkj8nAGiE8McrJW9lZWWIjY1FaWkp84eIiDxRUGBOlnYlP//aEg321sIKwtCHDUveE2A/6PAlYdfekhepqeZAxp1z+vp4e9x931zxx+vjRNXlKmxZugeG1Vdg3JOE/1V1tdnfAqdQgRTJP78ZDBERkWMmk3nWmKtelaNHzQGPo7WwXnst4EmxTUgRdFj4GgD6O4BcscI8689TbdvaTq/31+vzM1EvsPeTIzC8dxLGrS2x6UJ3VKGZdb8G9ejXYh90vc9Bf08bZE1MQdvEtgyG/IHBEBGRD9ztVXG0CrnEvQsekWOvlRQ87RmyBLWHD5vXA/Pj63Ph0EWsf30/jF+YYDxyPU7V2+YeJYWdgb7zYehvC8ct0zPQNiPeui9Qn98MhoiIyDVXvSqWHiRHydaNe5BIWq569Bryc7BaW1mLbe/uhXHlJRgK22NnZTeIBvO1onEVw+P3QD+sHLoHUpA1rjM0YfaTyBkM+RGDISIKuFDsgXB2Td7kFpG0HPXoNeaHobDDG47D+PYxGDZFI784E1dg+1l7Q/RB6Huchu6OVsie1h3RraPdOm+gPr+DPpuMiCjkyDlvxheWejD2hMKss1BjqfBs72fx4YfN65J5GaiXnihF/pv7Yfi4GsaDnfBTXScAnaz722rOQ9fxAHSjBUY/ej2SbuwKoKvD8wUbgyEiIn9ylDdjqdYsh7wZKbhbg+bsWXMPk9J7yZQiJwcYP97nXkpTjQk7/70fhg/OwfhdG2wr6w4TBlr3R6IGQ1vvgW5AKfT3J6D33RkIixjq76uRDIfJiIj8Rc15M57kqIRCL5kKnNxxBoYlh2HcGI4vT3bDRRFns79r5FHoup2AfkIzDJ+WiVZJ/q//x2EyIiKl8aRac6jlzTgrINhYqPeSKVTl+UpsemMPjOsqYdiTgn01nQFc6/GLRSluSdoH/cgajH6wI9I1x6/1NiU0D17D/YDBEBGRv6g9b8ZRjkpjQpgDplmzzEM4odZLphCiXuCH1Qdh/OcZGLbFYMul7qhBf+v+MJgwoOVe6PtegG5SPAZMyURE9CDzUPD9vwqpnDgGQ0RE/iL1wptKYMlRef114Le/dXxcKPeSyVjJnnNYv+QgjAYB47EuKK7PAHBtyYuO4Seh7/ITdL+IxC2PZaJN+g22JwjRnDgGQ0RE/uJqDSwgYGs8BVV4OJCQ4N6xodpLFiguSjjUlNdg61t7YPhPKYw/JKDwaiaAdtb9zVGBEe32Qp9dCf1DqeiqT4cmLMXxczla70zhvX0MhoiIXHG3ZpA7eTNXrwIffaTIv549wl4y6dkp4SBSUnB6yu+Rtz8Lhi3NUFCShQr0sXlYn2b7oOt5Fro7YzH04SxoY/o3PrN9gcyJs9xzR474dh43MRgiInLG05pBlryZ3/wGuHCh6f6LFxU9nOA2qVaKJzMHw1Xi5CkkLpiBfKzGZzD/fCWElUCXdhB6vQajpndFQo9MAJmeP2egcuLs3XMS49R6IiJHvF1rS81T7BuScqV4FaurqIYppROiLp+FvUUs6qHBeU17/OvWf2P01BT0nNjV4XIXHglElfFG91wZgFiAy3H4A4MhIvKYLwENl6a4RsqV4lXk2FcnYVz6Ewz5Uag+fR6fYpzrB/n758tVLSlfg3w791yggiEOkxER2eNLfoTap9g35KcKyGpTXlyOgjf2wrDuKoz7O+JgbToAc2LzPVjh3kn8/fPlLCfO0tv36qvev7eu7jkJMRgiIrLHl4CGycO2nK1ppiZOEvHr6+qxe+UBGN8/C8M3sdh6uTtqMcD60HDUYXDMHuj6XcIdN1YBL7nxfFL8fDlb78zX3r4g/nHAYIiIyB5fAhomD1NjdoYLTQmJ2HLDDLx9aATWn+iKc8I2sTk94gT0XY9BNzYKN8/IRGzHXj8/MBv48Lng/XxJ1dsXxD8OmDNERGSPr/kRTB4mi59/FoQQNgnP9T9/NxGrsRY5aIkruLnDXuiyq6B/JA2dR3Z0nPgcij9fdu65QOUMhUl2ZiIiJbPkRwDXPmAs3MmPsAwnJCfbbk9JUeYHFXlM1AvsyTuAK7+a1iQQAoAwmD/w34uahk2v7sLFimh8dGYgZvxnOK6/pZPzGWCh+PPV8J4LMPYMEdE17hYXVBNfZ0PxNVWVC4cu4svF+2H43ATjketxff0BFEDCmYWh9vOVl2dTo4uzyYgosDwtLqgWvuZHMHlY/nwIKGora7H9vb0wrrwEQ2E7fFeRCYEh1v23wOheG7xNHg6lny9Hdb0CgMEQEYXs4ot+E0ofOGTLiz8Cjmw8DuPbx2Eo0GJjcSauoJfN/h7aQ9D3OAX9xFa4KasDMN6NdqhlZqEjztY9CwAOkxGpHaslk1q5WWG87GQZ8t/YB8PH1TAe6IQjdZ1sDm+rOY/RHQ9AN0pAN/16JN3Y4dpOqQsVhgoHhUo5TEZEgRHIxReJ5MLFCuwCGpTdOw3jo9Kw9coNqMNA6+4I1GJo7B7oBlyG/v4E9LknA2ERQ+0/j9SFCl1RSk5RkAuQMhgiUjtWSyY1cvFHgAYCsdXngOoy1CESXSKPQtftBPTjm2HEo5loldTb/eeSslChM0rKAwzyMCGDISK1Y7VkUqHqg8egdeO4PwzZiGULOyP9pnQA6d4/obeJ+N727CgtD9BVoVKJMWeISGpy76ZmToPv5P4eq9nP7404dRo/HTZh3bdJ+GJ7a2guXoARetePD+Ziut727AQrD9DX+8BOIclA5QxBqEBpaakAIEpLS4PdFFKbNWuESEkRwnxrm79SUszb5WTNGiE0GvNXw7ZatsmtvXKilPfYV3V1QuTnC5Gba/63ri7YLXLp8ivvifIW7WzemxNIEbdjjbgDq0Qdwmzft8Y/+6mpwbtOyz1pr12u7sn8fMfX1fArP9+/7fXHfdDoPKVAQD6/GQwRScWXX2bBYO+XWWqq/NopJ0p7j72lkICv+kq1yH+lUMwZlC+ejHpNmKARpkbvjQkQ9Q2+nAZDwbq+urqmr7cngVpurnvBUG6uf9rr7/ugQeBd+umnAfn85jAZkRSUOl2dwz3uk9N7LOX75ub084C3C+blLg6tPwbD2ydg3NIM+SVZqEBLhMGEY0hDMk56t+ZUeDiwYgVw551+a6tHHEwzb8LREJ6vj/eExPdBoD6/mUBNJAWlTldncUH3yeU9lnLGkIvp59BogFmzzInBjT/oJGrX5eOl2LB4L4yf1sJw6DocN9kmNrfXnMPMhA+RWuzkvXHFZALatfP+8b7ydYanq2Rkf65qL5f7wEcMhoikwOnqoU8O77HUM4a8/aDzY7vqquqw41/7YFxxAYbv4vFNeRbqMdi6PwrVGNZmD/SDyqCbkoied3RB2Kq2wL0eXKc9wbw3fZ3hGcjaRnK4D/yAwRCRFDhdPfQF+z32pdfGXd580PmhXce3noTxHz/BsDESG05n4rK4wWZ/t6gj0GedhG5Ccwx/NAst2t9oewJ/vOaBujftDSX6o2cnULWNgn0f+AlzhoikwOnqoS/Y73Eg8kK8eQ4vHlNeXI5NS/fBsLYSxv2pOFBznc2hbTSXMCp5P3Q310E37Tp0HJzs/Nyu3htnAnlvrloFTJ8OnD9/bZtlKBFoMs3c2j7A/d41qfMAJb4PmDNEpGTBLsFP0gv2exyI4QlveijcfL5ja3biw78Cxm9i8dXl7qhFf+u+cNRhUKu90PW7CP2v2qLf5EyERw12crZGXL03lu+DeW8+/TTwf//XdPvJk9eGEv3RsyN1HmCw7wN/kXSumkxwaj0FDaerh75gvceBqiXjaQ0qN9s1HPk2m9IiTohHMjeJvKe3iUvHLvvW5oZtd/TeBPPeXLXK9WtkmTqvlPpOEr2egfr85jAZkdQ4XT30BeM9DuQwnb2ZYamp9nsofm6XOHUKGjvtqocGJ5GCG/ADhiccgP6mKuge7oTrb+kETZjGt3ba4+y9Cdb7lpgInDvn+thgVr/2hgSvZ6A+v2UfDNXV1WHevHn497//jeLiYiQmJmLq1Kn4wx/+gLAw9ypIMBgiopBkZ/kCAJ7nlbjDxQedqBfY9+kRGJedwpUN3+DZK3MAAGG41q56aKCBwN4HXkKXvz+OqJZR/mmbkribUwUAubnApEmSNkfumDP0sxdffBFLly7FP//5T3Tv3h3fffcdHnjgAcTGxuKJJ54IdvOIiIInkKuh28k9uXDoIja8sR+Gz0ww/tQZJ03XA7gewHAU4nosxmNIwrUcorBUc7u6y2mB0EDzJIdL5jOwQonsg6Ft27Zh/PjxGDNmDAAgLS0NK1aswHfffRfklhGR5DjE6Jq3q6F7obayFt8s3wfDioswFrbFjoosCAyx7o/GVdwUvwe6weXQ//oGJI49AWz9iu9fQ+4GOO3a+acoIrlF9sHQsGHDsHTpUhw8eBBdu3bF999/j6+++gqvvvpqsJtGRFKSsrJyqJFwxtBPBSdg+McxGDdFYeOZTJShp83+HtpD0PU4Bf0drZD9SBaaxfWzPYGScl4CwTJDz1kxSwBYsoSBYwDJPhh65plnUFpaim7duiE8PBwmkwkLFizAJCfjqNXV1aiurrZ+X1ZWFoimEpG/SF1ZmRwqO1mG/Df2wfhJFQz703CkrhOAjtb98ZoLGJ16APrR9Rg9rTOS+3UB0CVo7VWchlPRHaXsPvVU8NZFUynZB0MrV67EBx98gNzcXHTv3h27d+/GrFmzkJSUhClTpth9zMKFCzF//vwAt5SI/CIQlZXJylRjwq4VB2D8oASGb9tgW1kW6jDQuj8CtRgSuwf6AZehu689bry3G8Iihjg5I7nkKNerXTvgjTeuJcVTwMh+NllqairmzJmDGTNmWLe98MIL+OCDD7B//367j7HXM5SamsrZZERKEMgVt1Xq5I4zWP/mYRi+DMeXJzNwQcTb7L8+8hj03Y5DNy4aI2dkoVVSqyC11AtKyjNTUluDhLPJflZZWdlkCn14eDjq6+sdPkar1UKr1UrdNCLP8ZefayGy8KOcXL14FZvf3ANDXjmMe5Kxp7oLgGuJvDEoxS2J+6AbUQPdb9Jw3Yg0AGlBaq0PlJZnJnV1aHKb7IOhcePGYcGCBejYsSO6d++OwsJCvPzyy/j1r38d7KYReUZpv6iDJUQWfgwmUS/w49pDMCw7DeO2Vth8sTuqcS2xOQwm9G+xD7obz0M/KQ4DpmQisvmgILbYD5hnRj6Q/TDZlStX8Mc//hFr165FSUkJkpKSMGnSJDz33HOIinKvYBeLLlLQOfpFLUVxPKUL9gKoCnVu33msX3wARoOA8ej1OFPfwWZ/Svhp6DsfgX5MBG6Z0Q1xndsEqaUSsPzMOJqhxZ8ZxWIFaj9iMERBxV/UngtkZWWFqimvwdfv7IXxP5dh2J2AXVczbfY3QyVGtNsD/bBK6H6dgm6/uE6a5S7kgHlmIYs5Q0ShYssW5zVFhACKiszH8Re1WSArKyuEqBc4tP4YjO+cgGFzM+SXZKECvW2O6RV9APqeZ6CbGINhj3SHNqa//ZNJLdC5cVLlmYVKjl+oXIeEGAwRSY0Jwd4JYGVlubp8vBQbl+yD4ZMaGA+n41hdOoB06/72mnPQpR2ATgeMnt4VHXpmAMgIWnsBBCc3Too8s0BcRyCCFOYqusWjYbKioiKkpqZK2R5JcJiMgioUu/D5l6YkTDUm7PjXPhj+fR7GnXH45koWTA3+Zo1CNYa12QPdwDLopyai5x1dEBbh3oLVARGs3Dh/55kF4joCEaSEQK6iLHOGWrRogdmzZ2POnDlo0aKFZI3yNwZDFFShlhDMvzT96sS2UzAuOYgiw16cOq/FYVyPLchGPcw/C92ijkCXeRL625tj+KNZaNFepr97g50b5yjPzGLVKveKGQbiOgIRpAT7/fCTgH1+Cw9s3bpVDBgwQCQmJor33nvPk4cGVWlpqQAgSktLg90UUqs1a4TQaMxf5l+B5i/LtjVrgt1C91iuo+E1KPE6gqj8bLn49PlvxcxeBSIj6oi4HWvECaTYvJ7nwtqL9cP/LI59VRTs5rovP7/pz4W9r/x86dqwZo0QKSn2nzclxb2fT6mvo67OcRst91Jqqvk4X8jh/fCDQH1+e9S/OmTIEHzzzTf461//iueeew59+vRBQUGBJEEaUUixJAQnJ9tuT0lRRFc1ANfLZADmZTJMpoA2S+7q6+pRuGI/XrytADe3KUSbhEiMnd8ff/9+OLJqdmM1JiIZtn+9txXnMGrzc+h09lvPn9BkMg/Nrlhh/jdQ74cccuNycoCXX7a/z1JvKC/P+Tmkvg5PJlT4Qg7vh4J4Ndh8//334+DBgxg3bhzGjBmD22+/HYcPH/Z324hCS04OcOyYOTcoN9f879GjygiEgMD9Eg8BxT+U4P1pX+G+9K1IjLqAG+/thjlfjED+5T6oRRTSIoowrVs+Pmj5KDQQTX8Rextc5uWZh0ZGjgTuvdf8b1qa6wDAH+RQLNNkAmbPtr/P3ddU6usIVJAih/dDSbztUqqoqBBbtmwRs2bNEmFhYUKr1YrZs2eLsrIyf/Zc+QWHyYj8IDfXvW733NxgtzTgrl66Kr5ctFM81T9f9Ire3+QlaYErYlzCdvH6xAJx0HhU1Jvq/T+MEewhTMvwj702+HP4xxl/vKZSX0eghq/k8H74QaA+vz2aWr906VLs2LEDO3bswL59+xAeHo6ePXtixowZ6N27N/79738jKysLa9euRb9+/VyfkIiUg39pWol6gf3//QnGd47jdMFBFJdG4zjSbBKf+zbfC12vEujvbo3BD2YhquVA25P4s4fA1RCmRmPuERk/Xrpk2fBwcxL9xInm57NXLPPVV+Vfb0jq68jONg+Pu5pQkZ3t3fkt5PB+KIknkVNKSoqYOHGieOmll8RXX30lqqqqmhyzYMEC0b17d79Fa/7AniEiPwiRvzS9deHwRbFy1lbxYNfNIjX8pN3E5wvh7cTm2xaIkr3nXJ/Qnz0EckqWtZfEnJoamOR6f74OUl6HowkVlq9Vq3x/jobPFaz3ww8C9fnt9+U4zp49i6SkJJhklETJqfVEfqKiZTJqK2vxzfJ9MH54EYZdbbGjIsua3XM78rAaE6GBgM0CF568Dv4subBihTlHyJXcXGDSJNfH+SpYdaj8XcZCyuuwV6LCQonFHSUiyzpD7hBCYPPmzRg+fLg/T+sTBkNEfmTvl3hqqnyXyfDgg+CnghMwvnUMhoIobDyTiTLE2uzvrj2EW7ufwAuH74W2rMQ2ELLw5APXX8Glvwp71tQAb7wBHDkCdO4MTJ8OuLkgtmwoKWBftQq4666m2+XY1iCRZZ0hpeIwGZGf1dWZhxpyc83/ynVozN4QQYN6M2WnysRHv98upvcoENdHHm0yWhGvOS/u6bhVvPfAZnFyx2nzOaVIfPZ1GMMfQ5hPPSVEeLjt48LDzduVRglDQ4GqN6Rwih0mkyP2DBGpkIMqv+aBLYE/Nn8JL1bORB0irfsiUIshsXug638Z+snt0eeeDIRHNerdkWJIyh/DGL70iDz9NPB//+f43E89BSxa5Fl7gk3uQ0OhuEyPBBQ7TCZHqgmG5H7zk7oF8ufTxVIE9dDgJFKQjqO4LrIIuozj0P8yGiMezURMiovfEXL+ELM3hBkfD8ycCTz7rP3Xu6YGaN7cee2d8HCgslJ5Q2ZyJrc8L5kK1Oe3jFb4I58Es9gaKZ/UVYsD+PN59eJVfPubd5wWiAyDQEcUoeiV1ThUk4Yl/xuOXy4Y6DoQAq5NjdbYzRgyb09N9X1qtDcshT3nzwfi4szbLlwAnn/e8ev9xhuu32+TyXwc+Q9LVcgKg6FQYOkeb/zL393y86RuUgcqEv98inqB/605iL+NK4AufifaxGvwynvu/QWZlFDv+RNa6rcATQMiqeu3uBO0fvQRMG8ecPGi7XZHr/eRI+49t7vHkXvkHFSrkaQZSTIR0gnUTMIjX0hdtViin8+SvedE7mNbxZTOW0Ri2Jkmp50Ytkr6ujuBTtJ1kQwuhPDu9X7lFfdeq1dekea61CxUFnCWEBOo/Sikc4bknL9A8uYir8bjmiz2+Onns6a8Btve3QvDysswft8euyq72azo1QyVGNFuD3RDK6F/MAXd9J2guS7df/VmHAlUHpSDZPAmydHevN7MGQoupZWqCLBAfX57tBwHyRBXJlYWOSW5e7LwqreBtJc/n6Je4PCG4zC8dRzGLdHIP5uFcvS2OaZX9AHobjgD/Z0xGPpwFqJb97c9ZyCWIggPl/6PDE+W2vDm9Y6KMi9u6mw22ezZDISkkpNjfu/k8ntBpRgMKR2T8JTD3l+A/q4064lABNIe/HyWnijFhsX7YPy0BoZD6ThWlwYgzXpIO8056DodhF4vMGpaFyT2zgCQ4ficOTnmHhN7r7mS/ur2JGj19veBZdr8yy/b9hCFh5sDIaVNq1eaQATV5BSHyZTO3+XnSRruDnMEUiCGWF0MwQgAAmEY2eIbbK3oDVODv88iUYNhrfdAP6gUuvs7oNedXREW4WTOh6NeNzn1xnnDkynYd93l2++DUKhATSGFw2TkHq5MLH9yWFHcHndWz05ONrd/xQrvAomvv3aai6IBoEE9NBXlMCECGVE/QZ9ZBN2E5hg+LRMtO/Rx73lc9bop+a9uT3p7fP19EBVl/lkkUhtJ07NlIqRnk1koofy8mjRcrsLd2TqBWFG8MWezWQAh4uOdz15y4eqb77l17RuG/VEc+6rIt2vwZUaco+VF5LDsiDdLbfD3AYUIzibzo5AeJmtI6cMBocLZatTOBKvSrKOqxRcuND3WxbBefV09flhzCIblZ2D8JgZhl85jPfSu2+DtUJw/ZsQ56lWaNMncIyaHHC93l9po+DugfXvzvpIS/j4gxeJyHH6kmmCIgs9RbpA7gln+oPGH6JQp5uEzexoFGGd/PAfj4gNYbzCh/PhFaEUVziARW2AuFleEVCTijO8rvNvja96Tp+9XMHO8XE3BlluCPpEfMGeISGmc5QY5YwkIgllptuFsloICx4EQYJ299I8er2Pp8Vux+2o33I4teA1PIBXXPoivNGuHK0/OR2Kv16G5685rj218roce8r7dvsyI8+b9CmaOl7Mp2I6COkvV6WAEb0QKwuU4iPzF1RRoe+SY5O5mgFGwP+HnQCgPqzERKbC99lZV55G0YAY0YRrzh3Fysv0TOVs3yxVfSkt4834BtlPZA80StE6aZP7XMlvOWYI+YA7e/L3eHFEIYTBE5C/e1ONJSZHfX+1uBhi9ks4jd9pmrEp4HGEQTYfBGn4Qjx9/bQFRe7xdp8yX9Z18LUQql0KmntQhIiK7GAwR+Yu7vRSvvGJOls7PN+fKyCAQqquqw9Y3f8BzNxVgyJg2KEIK6u1n+UBAA5GaijknpmPS3fUIP3va8YkbfxC//bbj4wDPezB8WTTV10Kkvj7enUVX3cEq9EQ+Y84Qkb+4U7cnJQV4/HFZDIkd3VwEw9KjMBZEYcOZTJShp3XfE3gNqzERArANiTQa8/eWAMOTD2Kplv/wttK0q/fLEX/kePkz2ZlV6Il8xmCIyF9kXgDzyukryF+yF8ZPqmDc3xGHatMBpFr3x2kuYnTKfuhHmTD60cEIK3IjwPDkg9jXZGdnZSO8Wd/J2fvliD/eR38nO7sbhAczQZ9I7iStYiQTqii6SPIhk4J3plqT+O79vWLB6HxxU2yhiECNTZMiUCOyY3aLP9+SL75dvkfUVdspKOiq6KAnBQHz870rPmnv9fSw+KNTjt6vp57y//toeb0cXbu9AoruXoOj4pnuFp4kkiEWXfQj1hmigAtSAczTu4phfOMwjF9qsP5EBs6Ltjb7O0cch77bMejGRWPk9EzEpPjhfvCkIKCn62YFak23QK1rJuV6cK7qEBEpEIsu+hGDIQpVVy9exZZ/7IVxzRUYfkzGj9VdbPa3Qhlu7rAP+hHV0D3cCZ1v7iRNQ9z9IHY3cAL8U11abjxZdNWbauSsQk8hhkUXiagJUS+w56PDMC4/BcPWlth8oTuq0Ne6X4N69GuxD/o+56C7uw0G/ToLkc0HOj+pPz5A3c3Z8STZWaqE62CSOtm5YfFMInIbgyEimTt/4ALWLz4A4xcmGH+6HqfruwC41gOUHHYG+s6HoR8Tjlumd0N8l+7un9yfs5rc/SB2N3AKxSnjTHYmkiUGQ0QyU1Neg+3L9sHw4SUYdrfHrspuEBhi3d8MlRjedg/0wyqheyAZmWM7QxPmRU9CMJdwcCdwCsUp4zKfcUikVswZChSO5ZMDol7g8IbjML59HMYt0dhYnIVytLI5pmf0AehvOAPdxBgM+00WoltH+/akSsjH8SbhWimY7EzkFuYMNXDq1Ck888wz+Pzzz3H16lV07doV7777Lvr27ev6wXLA1aSpkdITpdi4ZB8Mn9TAeCgNR+vSAKRZ97fTnMPojgeh1wmMnt4Fib0zAGT4rwFKyMcJ5V4Ub+oiEZFkZB8MXbp0CUOHDsXIkSPx+eefo3379jhy5Ahat24d7Ka5h6tJEwBTjQnfvb8PxtzzMOyIw/YrWTBhkHV/JGowrPUe6AaWQjc5Ab3vzkBYxFDpGuRuns2aNeZ/g/VB7W11aSVgsjORbMh+mGzOnDnYunUrtviwyGDQhsmUMBRBkin65jSMbx6BYUMEvjzVDZdEG5v9XSOPQp95ArrxzTBiehZadmgZuMa5W+/GItg9mRxmJlIl1hn6WVZWFvR6PU6ePIlNmzYhOTkZ06dPx8MPP+zwMdXV1aiurrZ+X1ZWhtTU1MAHQ1IWWCPZqSipwOale2FYWwnj3hTsq+lss78NLuLxtitwU8ZZdL/7BnSYniPffJzG/F3kkIjIDcwZ+tlPP/2EN998E7Nnz8bvf/97fPvtt5g5cya0Wi3uv/9+u49ZuHAh5s+fH+CW2iG3qcH869qvRL3A96sOwvjPMzBuj8GWS91Rg/7W/WEwYWDLvdD3u4B7MgrR9dOXoTl1EjgPYCuARUHsbfF0XS4hzMfNmmXOdeHPDRGFENn3DEVFRaFfv374+uuvrdtmzpyJHTt2YNu2bXYfw54hO5jE7RdnfzyH9UsOwmAQWH+8K87Wt7fZ3zH8JPRdfoJ+bCRueSwLrTvFBm5JCW/Y+7lwhT2ZRBQg7Bn6WWJiIrKysmy2ZWZmYo0lsdMOrVYLrVYrddNck0uBtdWrgTvvbLqdSdwuVZdVY+vbe2FcVQrDDx2w+2o3AO2s+1ugHCPb74Uu+yp0D6aiqz4dmrCUaycwmczBhr33P1C9Lc56BBvOalqzBli82PX5lFTkkIjIDbIPhoYOHYoDBw7YbDt48CA6dZJojSV/ksPU4FWrHK9xxKGPJkS9wIEvjsL4bhEMW5qj4FwWKtHH5pg+zfZB3+ssdHfGYshDWdDGDDDvMJmAzZtsg45gT2F3p0ew4awmd4IhJRU5JCJyh+8L30vr22+/FREREWLBggXi0KFD4t///rdo3ry5+OCDD9w+R2lpqQAgSktLJWypE2vWCJGSIoT5o8/8lZpq3i718zZ8Tmdf+fnStkXGLv50Saya/bV4KGOT6Bhe1OSl6RBWLO6/bov44NGvxNkfS+yfxN57nJIixKxZ7r3+ubn+v7A1a4TQaJo+l0Zj/mr881dXZ26zvcdYHpeaaj6OiCgAAvX5LfucIQD49NNPMXfuXBw6dAjp6emYPXu209lkjamyArWraf2NebtKtgLVVdXh23/ugyH3Aoy74vFteRbqce290KIK2XF7oBt0BfoHknBDThdowjSOT+gsJ8jd28vfeTjelnXwZFV5IiKJcWq9H8kiGAo0T+vIhHhS7LGvTsLw5k8wFkRhw+lMlCLWZn+W9jB0Waegz2mBm6ZloXnb5u6d2J2gIyzMfJyj/VLUmvIleZ9LRRCRTDCBmnzjSZJramrIrZJdXlyO/CV7YfzoKgz7O+JQbTqAa4nNcZqLGJWyH/pbTBg9rTNSB14P4HrPn8idnCBLIBTIvDFfyjoodakIlo4gIi8xGFIyZ7/8PUlyVer6Tg3U19Wj8MMDML5/FoZvWuPr0izUYoB1fzjqMDhmD/T9L0H3q3bo+6tuCI8a4uSMbnI36Jg1yzzEFKglJXxd8V1pS0WwdAQR+YDDZErl6pe/OxWGw8OBDz+8liOiMKd3FWP9m4dhWK/B+hMZOC/a2uy/LuI49BnHoRunxc0zMhGT4uS997ZXwZPhKMvsskD0XITyiu+NybmOExH5hDlDfhRywZC7v/wdJcNa/Oc/9usPydTVi1fx1Vt7YVh9BcY9SfhfVVeb/a1Qhps77INueBX0v0lD55vdLL/gS6+CnIMONSRDc/0/opDGYMiPQioY8vSXv4KTYUW9wN5PjsDw3kkYt7bEpgvdUYVm1v0a1KNfi33Q9T4H/T1tMOjXWYhsHnntBO709vijV0HOQYeC33+3yKnKOxH5HYMhPwqpYMibX/4KSiw9f+ACvlxyAMYvTDAeuR6n6m1zWpLDzkDX+TD0t4XjlukZaJsRb/9E7vT2+LNXQc5Bh4Lef4+tWAHce6/r41RUOoIolHA2GdnnzSwhGSfD1lbWYtu7e2FceQmGwvbYWdkNAtcSm6NxFcPj90A/rBy6B1KQNa4zNGEukoMd9fY0Xn7En9Wh5TwDS8bvv898TRQnIgKDIeWR+y9/N3ohDm84DuPbx2DYFI2NxVkoRy+b/TdEH4S+x2no7miF7GndEd26n2fP72wtMACYNg0YO9b9wPLUKfeOC+WgQ67ksv4fESkagyGlkfMvfwdDUxV/+AvWH+sKw8fVMB7shJ/qOgG4ltzcVnMeuo4HoBstMPrR65F0Y1cAXZuc3i2uensA4Nw5IDkZePxx9875298CzZoFf7iLmpLD+n9EpHjMGVIiOSbsOhiaqoe5TROxGmthblMkajC09R7oBpRCf38Cet+dgbCIMP+0w90cEov4eODiRefLZsghEZqck3POFhF5jQnUfhS0YEjKxFU5/fI3mVCXlIrwkjOwt4JXPTQoRgf8tccH0N3eEiOmZ6Flh5bStMWTZUg0GiAuznUwZDmWU7TlLZQTxYlUisGQHwUlGApERdwg/vKvPF+JTW/sgXFdJS7+7yT+WXef6wcFYnqzO8UmG5s/H3j9deD8edfHcoo2EVHAcDaZkrk7m8lXAUzYFfUCP6w+COM/z8CwLQZbLnVHDfoDAO7BCvdO4sl6ad5qmEPiri5dzD1q97kR0AXiGoiIKKAYDPmbq9lMGo15narx42XfhV+y5xzWLzkIg0Fg/bEuKK7PAJBh3d8x/CT0XX7Cr264CKxy44SBmuGWk2MOOKdNMydLu+JJuzhFm4go5HCYzN8UXBG3uqwaX7+zF4b/lML4QwIKr2ba7G+OCoxsvwe6YVehfygVXfXp0IRp5LskRU2NedaYo+Gvhu0C5HkNREQqxmEypfKmKGKQiHqBg4ajMLxTBONXzVBQkoUK9LE5pk+zfdD1PAv9XbEY8lAWtDEDmp5IrtObo6KAf/zD+cy7hu2S4zUQEZHkGAz5m8yLIl46ehkbFu+D8b+1MBy6DidM1wG4zro/IawEurSD0Os1GDW9KxJ6ZALIdHg+K8vQlL2k8WBOb/akXXK9BiIikhSHyfxNZkNGdVV1+Paf+2BccQGGnfH4tjwLAJCNLUjEGZxHPDSt22D0kAropiSi58Su5qEvb8l1erMn7ZLrNRARqQyHyZRKBkNGx746CePSn2DIj8KG05koxQ3WfbcjD0swA4kovvaAlinAg68BOSN8f3K5LknhSbvkeg1ERCQJ9gxJJRBFEX/uwajadxSF26qwojADhgPpOFibbnNYG80ljE7Zh4c7b8QtBc9Bg0ZvOSssExGRDLHooh+FWgXq+rp6HPvdYrRd+gJiqq9NHS9CCp7Aa/gYv8TgmD3Q9bsE/X3t0PdX3cxPm5bmeN2uUJ0txSEvIiLF4jBZKPDjcMuZ3WdhXHIIxvUaRB0/iHcxC2jUw5OCk1iDO1C59F9o8chk2xMUFDhfwFQIoKjIHDiEyhBRIKqAExGR4jEYkqmqy1XYsnQPjGuuwPBjEv5X1RVAAsJgwjHcA0Cg8dKmGgDQaNBiwbPAQ/fa9oAoaMq/XwSqCjgRESkegyGpuTlMI+oF9n5yBMZlJ2H4qiU2XeiOKvS17tegHn2b78e064xI/dGLHh6ZT/n3qxCqAk5ERNJjMCQlF8M0Fw5dxJeL98PwuQnGI9fjVP31AK63HpoUdgb6zoehuzUco2ZkoG1GFrDie+BeN567cQ9Pdrb5uV1N+c/O9upSZWXLFvUNCRIRkdcYDEnFwTCNOHkKuOMOPKN9BS9Vz4TAEOu+aFzF8Pg90A0th/7XKcga1xmasEY9Nd728Mhgyn/AqG1IkIiIfNI47YT8wckwjQYCAho8Vv0yNBC4IfognuxbAOPCnbh4AfjifD/M/mgEuo+/3n7xQ0sPj8ZBYUSNxjyF314Pj6XCcnKy7faUlNDKoVHTkCAREfmMU+t91SgnqCy9F36YuwLDVsxw+dDz/1iDtr/xIgCx9DoB9nt4XAU2oT7dXGZVwImIyDucWq8EeXkQM5+A5tS1/JRSpOBbTMQwNx7etlW1d8/r6xpaoV5hWU1DgkRE5DP2DHnh5I4zOPDMu7g5/48QsB1rrIcG9qa925Wf71tQEuo9PL4KRBVwIiKSDCtQ+5GvL2bl+UpsXroXhrwKGPcmY391Oo4hDck46TjoCQ8H6us5TBNsDBiJiBSLw2RBJOoF/pd3CIZlp2Hc3gpbLnZHNfpZ94/EBqTCydRtwPwhDARumKbhh3779uZtJSUMAEJ9SJCIiHzGYOhnJXvOYf2SgzAaBIzHuqC4viuArtb9qeGnoL/+CPRjInFr6lHgt26cdNYsc26PN3k9nrA3HNQQl6AgIiJySLXBUE15Dba+tQfGVaUwfJ+AwquZANpZ9zdHBUa02wt9diV0D6Yi49Z0aMJ+npJe4Gbi8/jxwEsvSTtM42jZiYa4BAUREZFDqsoZ+m71bny94jIMW5qhoCQLFWhpc1zvZvuh71kM3Z2xGPpwFrQxWvsnDPTUbUd5L5Z2OKu2LFWbiIiIJMacIQn0m5gO4NqLmRBWAl3aQeh0Goye0RUJPboB6Ob6RIGcuu1sSY+4OPcCIYBLUBARETmgqmAoEtW4qc0u6AaVQT81ETfkdEFYRHvvTuZrrR93uFp5/YknPD8nl6AgIiKyoaphstOHTyOxs5+XYJBq6rarITCNBmjbFjh3zrPz+lrbiIiIKEACNUymuLXJFi5cCI1Gg1mzZnn82BbtWvi/QZap25Mmmf/1Vz6OOyuvnztnDogcrVPWkLM1y4iIiFRMUcHQjh078NZbb6Fnz57Bbor03B3Ouu8+87/OAiIuQUFEROSQYoKh8vJy/OpXv8Lbb7+NNm3aBLs50nN3RfU2beyvRN9QqK1KT0RE5EeKSaCeMWMGxowZg1GjRuGFF15wemx1dTWqq6/VAiorK5O6ef6XnW0OYlzNFnv7beDYMXNNI1agJiIi8pgigqEPP/wQu3btwo4dO9w6fuHChZg/f77ErZJYeDjw8MPA8887P+7kyWvT5ZkYTURE5DHZD5MVFRXhiSeewAcffIDo6Gi3HjN37lyUlpZav4qKiq7tNJmAggJgxQrzv5Y1xOSoSxf3juN0eSIiIq/Jvmdo586dKCkpQd++fa3bTCYTNm/ejMWLF6O6uhrhjYaAtFottFo71aM//hiYO9d+AUM55tO4mzfk7nFERETUhOzrDF25cgXHjx+32fbAAw+gW7dueOaZZ9CjRw+X57DWKUDD+tM/s8y0kmOCcaCX/SAiIpIRLsfxs1atWjUJeFq0aIH4+Hi3AiGXhDAHFbNmmZOQLUGFVMUUPRHIZT+IiIhUSvY5QwHRcN0uwLwMRloaMHIkcO+95n/T0szb7ZEyD8my7EfjqfOcLk9EROQXsh8m8wenw2QN5eYCWq399cAcDac5W0jVn4GKHHqqiIiIAihQw2QMhhr68ktg6lTn64E1zNFxtJCqnPOQiIiIFIJrkwWSZd0uwPV6YJbhNJPJ3CNkL5a0bJs1S95T94mIiEiFwVDjNbwaJiKXlLh3jjNn3FtItWEeEhEREcmSuoKh9993nojsSV0fdwsdsiAiERGRrMl+ar1f/fKXwKRJjhORLeuBuarrk53tfo8PCyISERHJmrqCIcAc+Dhaw8uTuj6eBE5EREQkW+oaJnPH+PHAvHlAmza22xvX9bEEToDzPCROfyciIpI1BkMNWYotPv88cPGieVtcHDB/vnk6feNp8iyISEREpHjqqjPkrE6BLzWDWBCRiIjI71h00Y9cvpiWBVHdLbZIREREkmPRxUBizSAiIiLVYjAEsGYQERGRijEYAjwrtkhEREQhhcEQcK1mUOMp8haWtctYM4iIiCjkMBgCWDOIiIhIxRgMWbBmEBERkSqpbzkOZ3JyzBWoWTOIiIhINRgMNeZs7TIiIiIKORwmIyIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNc4m8weTidPxiYiIFIrBkK/y8oAnnrBd9T4lxVzRmoUaiYiIZI/DZL7IywMmTrQNhADg1Cnz9ry84LSLiIiI3MZgyFsmk7lHSIim+yzbZs0yH0dERESyxWDIW1u2NO0RakgIoKjIfBwRERHJFoMhb50549/jiIiIKCgYDHkrMdG/xxEREVFQMBjyVna2edaYRmN/v0YDpKaajyMiIiLZYjDkrfBw8/R5oGlAZPn+1VdZb4iIiEjmGAz5IicHWL0aSE623Z6SYt7OOkNERESyx6KLvsrJAcaPZwVqIiIihWIw5A/h4cCIEcFuBREREXmBw2RERESkagyGiIiISNUYDBEREZGqyT4YWrhwIfr3749WrVqhffv2mDBhAg4cOBDsZhEREVGIkH0wtGnTJsyYMQPbt2/H+vXrUVdXB51Oh4qKimA3jYiIiEKARgh7y67L17lz59C+fXts2rQJN910k1uPKSsrQ2xsLEpLSxETEyNxC4mIiMgfAvX5rbip9aWlpQCAuLg4h8dUV1ejurra+n1ZWZnk7SIiIiJlkv0wWUNCCMyePRvDhg1Djx49HB63cOFCxMbGWr9SU1MD2EoiIiJSEkUNk82YMQOfffYZvvrqK6SkpDg8zl7PUGpqKofJiIiIFITDZI08/vjj+Pjjj7F582angRAAaLVaaLXaALWMiIiIlEz2wZAQAo8//jjWrl2LgoICpKenB7tJREREFEJkHwzNmDEDubm5+Oijj9CqVSsUFxcDAGJjY9GsWbMgt46IiIiUTvY5QxqNxu72ZcuWYerUqW6dg1PriYiIlIc5Qz+TeaxGRERECqeoqfVERERE/sZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RQTDL3xxhtIT09HdHQ0+vbtiy1btgS7SURERBQCFBEMrVy5ErNmzcKzzz6LwsJCZGdn47bbbsOJEyeC3TQiIiJSOI0QQgS7Ea4MHDgQN954I958803rtszMTEyYMAELFy50+fiysjLExsaitLQUMTExUjaViIiI/CRQn9+y7xmqqanBzp07odPpbLbrdDp8/fXXQWoVERERhYqIYDfAlfPnz8NkMiEhIcFme0JCAoqLi+0+prq6GtXV1dbvS0tLAZgjTCIiIlIGy+e21INYsg+GLDQajc33Qogm2ywWLlyI+fPnN9mempoqSduIiIhIOhcuXEBsbKxk55d9MNS2bVuEh4c36QUqKSlp0ltkMXfuXMyePdv6/eXLl9GpUyecOHFC0hdTbsrKypCamoqioiJV5UrxunndasDr5nWrQWlpKTp27Ii4uDhJn0f2wVBUVBT69u2L9evX4/bbb7duX79+PcaPH2/3MVqtFlqttsn22NhYVf0QWcTExPC6VYTXrS68bnVR63WHhUmb4iz7YAgAZs+ejcmTJ6Nfv34YPHgw3nrrLZw4cQLTpk0LdtOIiIhI4RQRDN199924cOEC/vSnP+HMmTPo0aMH/vvf/6JTp07BbhoREREpnCKCIQCYPn06pk+f7tVjtVotnn/+ebtDZ6GM183rVgNeN69bDXjd0l63IoouEhEREUlF9kUXiYiIiKTEYIiIiIhUjcEQERERqRqDISIiIlI1RQZDb7zxBtLT0xEdHY2+fftiy5YtTo/ftGkT+vbti+joaFx33XVYunRpk2PWrFmDrKwsaLVaZGVlYe3atVI132ueXHdeXh5Gjx6Ndu3aISYmBoMHD4bBYLA5Zvny5dBoNE2+qqqqpL4Uj3hy3QUFBXavaf/+/TbHhdr7PXXqVLvX3b17d+sxSni/N2/ejHHjxiEpKQkajQbr1q1z+ZhQuL89ve5Qub89ve5Qub89ve5Qub8XLlyI/v37o1WrVmjfvj0mTJiAAwcOuHxcIO5xxQVDK1euxKxZs/Dss8+isLAQ2dnZuO2223DixAm7xx89ehS/+MUvkJ2djcLCQvz+97/HzJkzsWbNGusx27Ztw913343Jkyfj+++/x+TJk3HXXXfhm2++CdRlueTpdW/evBmjR4/Gf//7X+zcuRMjR47EuHHjUFhYaHNcTEwMzpw5Y/MVHR0diEtyi6fXbXHgwAGba+rSpYt1Xyi+36+99prN9RYVFSEuLg533nmnzXFyf78rKirQq1cvLF682K3jQ+X+9vS6Q+X+9vS6LZR+f3t63aFyf2/atAkzZszA9u3bsX79etTV1UGn06GiosLhYwJ2jwuFGTBggJg2bZrNtm7duok5c+bYPf7pp58W3bp1s9n2yCOPiEGDBlm/v+uuu8Stt95qc4xerxf33HOPn1rtO0+v256srCwxf/586/fLli0TsbGx/mqiJDy97vz8fAFAXLp0yeE51fB+r127Vmg0GnHs2DHrNiW83w0BEGvXrnV6TKjc3w25c932KPH+bsid6w6V+7shb97vULi/hRCipKREABCbNm1yeEyg7nFF9QzV1NRg586d0Ol0Ntt1Oh2+/vpru4/Ztm1bk+P1ej2+++471NbWOj3G0TkDzZvrbqy+vh5XrlxpsthdeXk5OnXqhJSUFIwdO7bJX5bB5Mt19+nTB4mJibjllluQn59vs08N7/e7776LUaNGNanSLuf32xuhcH/7gxLvb18o+f72h1C5v0tLSwHA6SKsgbrHFRUMnT9/HiaTqclq9QkJCU1WtbcoLi62e3xdXR3Onz/v9BhH5ww0b667sb/97W+oqKjAXXfdZd3WrVs3LF++HB9//DFWrFiB6OhoDB06FIcOHfJr+73lzXUnJibirbfewpo1a5CXl4eMjAzccsst2Lx5s/WYUH+/z5w5g88//xwPPfSQzXa5v9/eCIX72x+UeH97IxTub1+Fyv0thMDs2bMxbNgw9OjRw+FxgbrHFbMcR0MajcbmeyFEk22ujm+83dNzBoO3bVyxYgXmzZuHjz76CO3bt7duHzRoEAYNGmT9fujQobjxxhvx+uuv4+9//7v/Gu4jT647IyMDGRkZ1u8HDx6MoqIivPTSS7jpppu8OmeweNvG5cuXo3Xr1pgwYYLNdqW8354KlfvbW0q/vz0RSve3t0Ll/n7sscfwww8/4KuvvnJ5bCDucUX1DLVt2xbh4eFNor2SkpImUaFFhw4d7B4fERGB+Ph4p8c4OmegeXPdFitXrsSDDz6I//znPxg1apTTY8PCwtC/f3/Z/CXhy3U3NGjQIJtrCuX3WwiB9957D5MnT0ZUVJTTY+X2fnsjFO5vXyj5/vYXpd3fvgiV+/vxxx/Hxx9/jPz8fKSkpDg9NlD3uKKCoaioKPTt2xfr16+32b5+/XoMGTLE7mMGDx7c5Hij0Yh+/fohMjLS6TGOzhlo3lw3YP6LcerUqcjNzcWYMWNcPo8QArt370ZiYqLPbfYHb6+7scLCQptrCtX3GzDP1jh8+DAefPBBl88jt/fbG6Fwf3tL6fe3vyjt/vaF0u9vIQQee+wx5OXlYePGjUhPT3f5mIDd426nWsvEhx9+KCIjI8W7774r9u7dK2bNmiVatGhhzaqfM2eOmDx5svX4n376STRv3lz89re/FXv37hXvvvuuiIyMFKtXr7Yes3XrVhEeHi7++te/in379om//vWvIiIiQmzfvj3g1+eIp9edm5srIiIixJIlS8SZM2esX5cvX7YeM2/ePPHFF1+II0eOiMLCQvHAAw+IiIgI8c033wT8+hzx9LpfeeUVsXbtWnHw4EHx448/ijlz5ggAYs2aNdZjQvH9trjvvvvEwIED7Z5TCe/3lStXRGFhoSgsLBQAxMsvvywKCwvF8ePHhRChe397et2hcn97et2hcn97et0WSr+/H330UREbGysKCgpsfm4rKyutxwTrHldcMCSEEEuWLBGdOnUSUVFR4sYbb7SZljdlyhQxfPhwm+MLCgpEnz59RFRUlEhLSxNvvvlmk3OuWrVKZGRkiMjISNGtWzebm0suPLnu4cOHCwBNvqZMmWI9ZtasWaJjx44iKipKtGvXTuh0OvH1118H8Irc48l1v/jii6Jz584iOjpatGnTRgwbNkx89tlnTc4Zau+3EEJcvnxZNGvWTLz11lt2z6eE99syddrRz22o3t+eXneo3N+eXneo3N/e/JyHwv1t75oBiGXLllmPCdY9rvm5gURERESqpKicISIiIiJ/YzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIgUacWKFYiOjsapU6es2x566CH07NkTpaWlQWwZESkN1yYjIkUSQqB3797Izs7G4sWLMX/+fLzzzjvYvn07kpOTg908IlKQiGA3gIjIGxqNBgsWLMDEiRORlJSE1157DVu2bGEgREQeY88QESnajTfeiD179sBoNGL48OHBbg4RKRBzhohIsQwGA/bv3w+TyYSEhIRgN4eIFIo9Q0SkSLt27cKIESOwZMkSfPjhh2jevDlWrVoV7GYRkQIxZ4iIFOfYsWMYM2YM5syZg8mTJyMrKwv9+/fHzp070bdv32A3j4gUhj1DRKQoFy9exNChQ3HTTTfhH//4h3X7+PHjUV1djS+++CKIrSMiJWIwRERERKrGBGoiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqv0/Jx4UF4ulq+AAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_16_2.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients for OLS\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "623ad923",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Same code but now with momentum gradient descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b7fe53c1",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.]\n",
      " [3.]]\n",
      "Eigenvalues of Hessian Matrix:[0.35058127 4.29679459]\n",
      "0 [-8.31895514] [-8.15055258]\n",
      "1 [-0.75472506] [0.63957747]\n",
      "2 [-0.69314603] [0.58739348]\n",
      "3 [-0.63659131] [0.53946725]\n",
      "4 [-0.58465096] [0.49545139]\n",
      "5 [-0.5369485] [0.45502684]\n",
      "6 [-0.49313815] [0.41790059]\n",
      "7 [-0.45290234] [0.38380352]\n",
      "8 [-0.41594943] [0.35248847]\n",
      "9 [-0.38201155] [0.32372846]\n",
      "10 [-0.35084272] [0.29731502]\n",
      "11 [-0.32221699] [0.27305669]\n",
      "12 [-0.29592687] [0.25077762]\n",
      "13 [-0.2717818] [0.23031634]\n",
      "14 [-0.24960675] [0.21152452]\n",
      "15 [-0.229241] [0.19426595]\n",
      "16 [-0.21053692] [0.17841553]\n",
      "17 [-0.19335893] [0.16385836]\n",
      "18 [-0.17758251] [0.15048894]\n",
      "19 [-0.16309331] [0.13821034]\n",
      "20 [-0.14978631] [0.12693357]\n",
      "21 [-0.13756504] [0.11657689]\n",
      "22 [-0.12634093] [0.10706523]\n",
      "23 [-0.1160326] [0.09832963]\n",
      "24 [-0.10656534] [0.09030678]\n",
      "25 [-0.09787053] [0.08293853]\n",
      "26 [-0.08988514] [0.07617146]\n",
      "27 [-0.08255129] [0.06995653]\n",
      "28 [-0.07581582] [0.06424868]\n",
      "29 [-0.06962991] [0.05900655]\n",
      "theta from own gd\n",
      "[[3.81759234]\n",
      " [3.15457792]]\n",
      "0 [-0.06394871] [0.05419212]\n",
      "1 [-0.05873105] [0.04977051]\n",
      "2 [-0.0523738] [0.04438319]\n",
      "3 [-0.04619338] [0.03914571]\n",
      "4 [-0.04057027] [0.03438051]\n",
      "5 [-0.03557316] [0.0301458]\n",
      "6 [-0.03117156] [0.02641575]\n",
      "7 [-0.02730775] [0.02314144]\n",
      "8 [-0.02392053] [0.020271]\n",
      "9 [-0.02095266] [0.01775594]\n",
      "10 [-0.01835274] [0.01555268]\n",
      "11 [-0.01607534] [0.01362274]\n",
      "12 [-0.01408051] [0.01193226]\n",
      "13 [-0.01233322] [0.01045155]\n",
      "14 [-0.01080274] [0.00915458]\n",
      "15 [-0.00946219] [0.00801855]\n",
      "16 [-0.00828799] [0.0070235]\n",
      "17 [-0.0072595] [0.00615193]\n",
      "18 [-0.00635865] [0.00538851]\n",
      "19 [-0.00556958] [0.00471983]\n",
      "20 [-0.00487843] [0.00413413]\n",
      "21 [-0.00427304] [0.00362111]\n",
      "22 [-0.00374279] [0.00317175]\n",
      "23 [-0.00327833] [0.00277816]\n",
      "24 [-0.00287151] [0.00243341]\n",
      "25 [-0.00251517] [0.00213144]\n",
      "26 [-0.00220306] [0.00186694]\n",
      "27 [-0.00192967] [0.00163526]\n",
      "28 [-0.00169021] [0.00143234]\n",
      "29 [-0.00148047] [0.00125459]\n",
      "theta from own gd wth momentum\n",
      "[[3.99630114]\n",
      " [3.00313452]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients for OLS\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x#+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 30\n",
    "\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(theta)\n",
    "    theta -= eta*gradients\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "# Now improve with momentum gradient descent\n",
    "change = 0.0\n",
    "delta_momentum = 0.3\n",
    "for iter in range(Niterations):\n",
    "    # calculate gradient\n",
    "    gradients = training_gradient(theta)\n",
    "    # calculate update\n",
    "    new_change = eta*gradients+delta_momentum*change\n",
    "    # take a step\n",
    "    theta -= new_change\n",
    "    # save the change\n",
    "    change = new_change\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"theta from own gd wth momentum\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "749000b3",
   "metadata": {
    "editable": true
   },
   "source": [
    "## But noen of these can compete with Newton's method"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e7b4abe1",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.08185019]\n",
      " [2.82781715]]\n",
      "Eigenvalues of Hessian Matrix:[0.32244056 3.90871918]\n",
      "0 [-14.19393543] [-14.29174301]\n",
      "1 [-6.35255737e-15] [-1.10851799e-14]\n",
      "2 [-5.89805982e-17] [-2.66490332e-16]\n",
      "3 [9.81853487e-16] [1.10741066e-15]\n",
      "4 [-5.89805982e-17] [-2.66490332e-16]\n",
      "beta from own Newton code\n",
      "[[4.08185019]\n",
      " [2.82781715]]\n"
     ]
    }
   ],
   "source": [
    "# Using Newton's method\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "def CostOLS(beta):\n",
    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "beta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(beta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "# Note that here the Hessian does not depend on the parameters beta\n",
    "invH = np.linalg.pinv(H)\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "beta = np.random.randn(2,1)\n",
    "Niterations = 5\n",
    "\n",
    "# define the gradient\n",
    "training_gradient = grad(CostOLS)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = training_gradient(beta)\n",
    "    beta -= invH @ gradients\n",
    "    print(iter,gradients[0],gradients[1])\n",
    "print(\"beta from own Newton code\")\n",
    "print(beta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d47c2b1c",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Including Stochastic Gradient Descent with Autograd\n",
    "In this code we include the stochastic gradient descent approach discussed above. Note here that we specify which argument we are taking the derivative with respect to when using **autograd**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e2b04ba9",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[3.41716708]\n",
      " [3.50046106]]\n",
      "Eigenvalues of Hessian Matrix:[0.24252405 4.30774404]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own gd\n",
      "[[3.41716708]\n",
      " [3.50046106]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHFCAYAAADxOP3DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABUPUlEQVR4nO3deXgT1foH8O+0pSlroYXS0hZaXFhkB1lElqrgBoK1IuUnItf1sijiBqJCUUBwgyt6uagXvHobUFoW9bogtBRlkX1R2bRgWcpOC4UWmpzfH2NC0yTNJE0yk8z38zx5MJPJ5MyEMK/nvOc9khBCgIiIiEinQtRuABEREZGaGAwRERGRrjEYIiIiIl1jMERERES6xmCIiIiIdI3BEBEREekagyEiIiLSNQZDREREpGsMhoiIiEjXGAwRadDChQshSZL1ERYWhri4OAwdOhT79+9XrV1TpkyBJEmqfX4gOHjwICRJwltvvaV2U4hIoTC1G0BEzi1YsAAtW7ZEaWkpfvrpJ0ybNg05OTnYs2cPGjRooHbziIiCAoMhIg1r06YNunTpAgDo27cvTCYTJk+ejGXLlmHkyJEqt47UdOXKFWuvIRFVD4fJiAKIJTA6fvy4dVtpaSmeffZZdOjQAZGRkYiKikKPHj2wfPlyu/dLkoQxY8bg008/RatWrVCrVi20b98eX331ld2+X3/9NTp06ACDwYDk5GSnwz6lpaWYOHEikpOTER4ejvj4eIwePRrnzp2z2S8pKQkDBgzAV199hY4dO6JmzZpo1aqV9bMXLlyIVq1aoXbt2ujatSs2b97s8npYhhNzcnLw97//HQ0bNkR0dDRSU1Nx9OhRu3OfMmWK3TGSkpLw8MMP2x1z9erVeOyxxxAdHY169erhoYceQklJCQoLCzFkyBDUr18fcXFxeO6553DlyhW745rNZkybNg1NmzZFREQEunTpglWrVtntt3//fgwbNgwxMTEwGAxo1aoV3n//fZt9cnNzIUkSPv30Uzz77LOIj4+HwWDAgQMHcPHiRTz33HNITk5GREQEoqKi0KVLFxiNRpfXj4hk/F8KogCSn58PALj++uut28rKynDmzBk899xziI+Px+XLl/HDDz8gNTUVCxYswEMPPWRzjK+//hqbNm3C1KlTUadOHcyaNQv33nsv9u7di+bNmwMAVq1ahUGDBqFHjx5YtGgRTCYTZs2aZROEAYAQAoMHD8aqVaswceJE9OrVCzt37sTkyZOxfv16rF+/HgaDwbr/jh07MHHiREyaNAmRkZHIyMhAamoqJk6ciFWrVmH69OmQJAkvvvgiBgwYgPz8fNSsWdPldXn00Udx9913IzMzEwUFBXj++efx4IMPYvXq1R5f60cffRSpqalYtGgRtm3bhpdeegnl5eXYu3cvUlNT8fjjj+OHH37AzJkz0aRJE4wfP97m/XPnzkWzZs0we/ZsmM1mzJo1C3feeSfWrFmDHj16AAB+/fVX3HTTTWjatCnefvttxMbG4rvvvsNTTz2FU6dOYfLkyTbHnDhxInr06IF58+YhJCQEMTExGD9+PD799FO8/vrr6NixI0pKSrB7926cPn3a43Mn0h1BRJqzYMECAUBs2LBBXLlyRZw/f158++23IjY2VvTu3VtcuXLF6XvLy8vFlStXxCOPPCI6duxo8xoA0bhxY1FcXGzdVlhYKEJCQsSMGTOs27p16yaaNGkiLl26ZN1WXFwsoqKiRMV/Nr799lsBQMyaNcvmcxYvXiwAiPnz51u3NWvWTNSsWVMcPnzYum379u0CgIiLixMlJSXW7cuWLRMAxIoVKxRdp1GjRtlsnzVrlgAgjh07ZnPukydPtjtGs2bNxIgRI+yOOXbsWJv9Bg8eLACId955x2Z7hw4dRKdOnazP8/PzBQCn1++2226zbrv99ttFQkKCKCoqsjnmmDFjREREhDhz5owQQoicnBwBQPTu3duu/W3atBGDBw+2205EynGYjEjDunfvjho1aqBu3bq444470KBBAyxfvtwuT+SLL75Az549UadOHYSFhaFGjRr4+OOP8dtvv9kdMyUlBXXr1rU+b9y4MWJiYnDo0CEAQElJCTZt2oTU1FRERERY96tbty4GDhxocyxLz0vFYSYAuP/++1G7dm27YaEOHTogPj7e+rxVq1YA5HyoWrVq2W23tMmVe+65x+Z5u3bt3Hq/IwMGDLB5bmnT3Xffbbfd0ec4u355eXkwmUwoLS3FqlWrcO+996JWrVooLy+3Pu666y6UlpZiw4YNNse877777D6na9eu+OabbzBhwgTk5ubi0qVLHp8zkV4xGCLSsP/85z/YtGkTVq9ejSeeeAK//fYb0tPTbfbJzs7GkCFDEB8fj88++wzr16/Hpk2b8Le//Q2lpaV2x4yOjrbbZjAYrDfRs2fPwmw2IzY21m6/yttOnz6NsLAwNGrUyGa7JEmIjY21G6qJioqyeR4eHl7ldkftd6TyOVmG5qoTGLjTVkftdHb9Ll++jAsXLuD06dMoLy/He++9hxo1atg87rrrLgDAqVOnbN4fFxdnd8x//OMfePHFF7Fs2TKkpKQgKioKgwcPVrUEA1GgYc4QkYa1atXKmjSdkpICk8mEjz76CEuWLEFaWhoA4LPPPkNycjIWL15sUwOorKzMo89s0KABJElCYWGh3WuVt0VHR6O8vBwnT560CYiEECgsLMSNN97oURt8wWAwOLwmvsqtcXb9wsPDUadOHdSoUQOhoaEYPnw4Ro8e7fAYycnJNs8d1XiqXbs2MjIykJGRgePHj1t7iQYOHIg9e/Z452SIghx7hogCyKxZs9CgQQO8+uqrMJvNAOQbZHh4uM2NsrCw0OFsMiUss7mys7NtejzOnz+PL7/80mbfW2+9FYAckFWUlZWFkpIS6+takJSUhJ07d9psW716NS5cuOCTz3N2/Xr16oXQ0FDUqlULKSkp2LZtG9q1a4cuXbrYPRz14lWlcePGePjhh5Geno69e/fi4sWL3j4toqDEniGiANKgQQNMnDgRL7zwAjIzM/Hggw9iwIAByM7OxqhRo5CWloaCggK89tpriIuL83io5LXXXsMdd9yBfv364dlnn4XJZMLMmTNRu3ZtnDlzxrpfv379cPvtt+PFF19EcXExevbsaZ1N1rFjRwwfPtxbp15tw4cPxyuvvIJXX30Vffr0wa+//oq5c+ciMjLSJ58XGhqKfv36Yfz48TCbzZg5cyaKi4uRkZFh3WfOnDm4+eab0atXL/z9739HUlISzp8/jwMHDuDLL79UNBuuW7duGDBgANq1a4cGDRrgt99+w6effooePXrY5GERkXMMhogCzNixYzF37lxMnToV6enpGDlyJE6cOIF58+bh3//+N5o3b44JEybg8OHDNjded/Tr1w/Lli3Dyy+/jAceeACxsbEYNWoULl26ZHNMSZKwbNkyTJkyBQsWLMC0adPQsGFDDB8+HNOnT7eZVq+2559/HsXFxVi4cCHeeustdO3aFZ9//jkGDRrkk88bM2YMSktL8dRTT+HEiRO44YYb8PXXX6Nnz57WfVq3bo2tW7fitddew8svv4wTJ06gfv36uO6666x5Q67ccsstWLFiBd59911cvHgR8fHxeOihhzBp0iSfnBdRMJKEEELtRhARERGphTlDREREpGsMhoiIiEjXGAwRERGRrjEYIiIiIl1jMERERES6xmCIiIiIdE0XdYbMZjOOHj2KunXrOixnT0RERNojhMD58+fRpEkThIT4rv9GF8HQ0aNHkZiYqHYziIiIyAMFBQVISEjw2fF1EQzVrVsXgHwx69Wrp3JriIiISIni4mIkJiZa7+O+ootgyDI0Vq9ePQZDREREAcbXKS5MoCYiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdUz0YysvLw8CBA9GkSRNIkoRly5Y53feJJ56AJEmYPXu239pHREREwU31YKikpATt27fH3Llzq9xv2bJl2LhxI5o0aeKnlhEREZEehKndgDvvvBN33nlnlfscOXIEY8aMwXfffYe7777bTy0jIiIiPVC9Z8gVs9mM4cOH4/nnn8cNN9ygdnOIiIgoyKjeM+TKzJkzERYWhqeeekrxe8rKylBWVmZ9Xlxc7IumERERURDQdM/Qli1bMGfOHCxcuBCSJCl+34wZMxAZGWl9JCYm+rCVREREFMg0HQytXbsWJ06cQNOmTREWFoawsDAcOnQIzz77LJKSkpy+b+LEiSgqKrI+CgoK/NdoIiIiCiiaHiYbPnw4brvtNpttt99+O4YPH46RI0c6fZ/BYIDBYPB184iIiCgIqB4MXbhwAQcOHLA+z8/Px/bt2xEVFYWmTZsiOjraZv8aNWogNjYWLVq08HdTiYiIKAipHgxt3rwZKSkp1ufjx48HAIwYMQILFy5UqVVERESkF6oHQ3379oUQQvH+Bw8e9F1jiIiISHc0nUBNRERE5GsMhoiIiEjXGAwRERGRrjEYIiIiIl1jMERERES6xmCIiIiIdI3BEBEREekagyEiIiLSNQZDREREpGsMhoiIiEjXGAwRERGRrjEYIiIiIl1jMERERES6xmCIiIiIdI3BEBEREekagyEiIiLSNQZDREREpGsMhoiIiEjXGAwRERGRrjEYIiIiIl1jMERERES6xmCIiIiIdI3BEBEREekagyEiIiLSNQZDREREpGsMhoiIiEjXGAwRERGRrjEYIiIiIl1jMERERES6xmCIiIiIdI3BEBEREekagyEiIiLStTC1G0BEREQBzmQC1q4Fjh0D4uKAXr2A0FC1W6UYgyEiIiLyXHY28PTTwOHDV7clJABz5gCpqeq1yw0cJiMiIiLPZGcDaWm2gRAAHDkib8/OVqddbmIwRERERO4zmeQeISHsX7NsGzdO3k/jGAwRERGR+9aute8RqkgIoKBA3k/jGAwRERGR+44d8+5+KmIwRERERO6Li/PufipSPRjKy8vDwIED0aRJE0iShGXLlllfu3LlCl588UW0bdsWtWvXRpMmTfDQQw/h6NGj6jWYiIiI5OnzCQmAJDl+XZKAxER5P41TPRgqKSlB+/btMXfuXLvXLl68iK1bt+KVV17B1q1bkZ2djX379uGee+5RoaVERERkFRoqT58H7AMiy/PZswOi3pAkhKM0cHVIkoSlS5di8ODBTvfZtGkTunbtikOHDqFp06aKjltcXIzIyEgUFRWhXr16XmotEREROawzlJgoB0LVrDPkr/t3wBVdLCoqgiRJqF+/vtN9ysrKUFZWZn1eXFzsh5YRERHpUGoqMGgQK1D7S2lpKSZMmIBhw4ZVGSHOmDEDGRkZfmwZERGRjoWGAn37qt0Kj6meM6TUlStXMHToUJjNZnzwwQdV7jtx4kQUFRVZHwUFBX5qJREREQWagOgZunLlCoYMGYL8/HysXr3a5bihwWCAwWDwU+uIiIgokGk+GLIEQvv370dOTg6io6PVbhIREREFEdWDoQsXLuDAgQPW5/n5+di+fTuioqLQpEkTpKWlYevWrfjqq69gMplQWFgIAIiKikJ4eLhazSYiIqIgofrU+tzcXKSkpNhtHzFiBKZMmYLk5GSH78vJyUFfhclanFpPREQUeHQztb5v376oKh7TUBkkIiIiCkIBM5uMiIiIyBcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXQtTuwFERERUBZMJWLsWOHYMiIsDevUCQkPVblVQYTBERESkVdnZwNNPA4cPX92WkADMmQOkpqrXriDDYTIiIiItys4G0tJsAyEAOHJE3p6drU67ghCDISIiIq0xmeQeISHsX7NsGzdO3o+qjcEQERGR1qxda98jVJEQQEGBvB9VG4MhIiIirTl2zLv7UZWYQE1EROQP7swKi4tTdkyl+1GV2DNERETka9nZQFISkJICDBsm/5mU5DwJulcvedaYJDl+XZKAxER5P6o2BkNERES+5MmssNBQefo8YB8QWZ7Pnu37ekMmE5CbCxiN8p9BmrDNYIiIiMhXqjMrLDUVWLIEiI+33Z6QIG/3dZ0hd3uzApgkhKNvKLgUFxcjMjISRUVFqFevntrNISIivcjNlYMIV3JygL59Hb+mRgVqS29W5RDB0ivlj2AM/rt/M4GaiIjIV7wxKyw01Hmg5AuuerMkSe7NGjTI50GZ6bJ/huU4TEZEROQrWp8V5ignSOUaR8IssOmTX/FMpzVoHXPSJ59RGXuGiIiIfMUyK+zIEcc9LZIkv67GrDBn656lpSl7v5drHP264gCMbx6GcUMyfi9v/dfWYq9+hjMMhoiIiHzFMissLU0OfCoGRP6cFVaZs5ygI0fk9ijhhd6sgz8exqLXD8C4Jg47S1sAuBYAUAsluKfpDgy6vxTpb1f7Y1xiAjUREZGvOeqFSUyUAw9/rz5vMsmzwpwNhUkSEBLifBq9pTcrP9+jIO747pP4fMqvMH4XhfUX2lq318Bl3NF4G9LvN2HgpHaoE1uHCdRERERBIzVVTjj296wwR5TkBFkCIS/1Zp07VITsKTthXFELq890gBl95MPBjL71dyB94HncN7ktoq7p5ubJeAeDISIiIn/w96wwZ5Tm+owbJ0+hr5xTpLA36+Kpi/jyte0wfh6Kbwo74DKu5kV1rb0b6f1OY8grLdCkU0f32u8DDIaIiIj0RGmuz6BBwFtvudWbdfnCZXw/azuM/7mC5YfaowQ3WV+7wbAf6b2OYOjEZFxzS5vqnoVXqT61Pi8vDwMHDkSTJk0gSRKWLVtm87oQAlOmTEGTJk1Qs2ZN9O3bF7/88os6jSUiIgp07qx7ZunNSk+X/3QQCJkum5DzzjY83ioPcfUuYOBrXZF5qCdKUAdJYQWY2CMXO5fsw+7S6zBpZV9cc0szn56eJ1QPhkpKStC+fXvMnTvX4euzZs3CO++8g7lz52LTpk2IjY1Fv379cP78eT+3lIiIKAh4Yd0zYRb4ecEveKbTGiTWPIlbnu2ID/f0xhkRhcYhJ/BU+zVY/+Fu/FGWgOnr+qLtfde7bpea66AJDQEgli5dan1uNptFbGyseOONN6zbSktLRWRkpJg3b57i4xYVFQkAoqioyJvNJSIiClxZWUIkJAghp0jLj8REebsTu5ftF5N65ohrwg7avK2+dFY8cn2e+GHWFlFeVu6dtiQkiKJPP/XL/VvTOUP5+fkoLCxE//79rdsMBgP69OmDdevW4YknnnD4vrKyMpSVlVmfFxf7p2gTERFRwFA4wy0/rwCLpv0OY14T7Cq9HpVrAaU/GIrbX+wAQz0PC0dWVfNo+HDPjukmTQdDhYWFAIDGjRvbbG/cuDEOHTrk9H0zZsxARkaGT9tGREQVqLGYKFWfkxlullpAmd9FYcOFtgASATiqBXST3Xvd4modND/RdDBkIVUa0xRC2G2raOLEiRg/frz1eXFxMRITE33WPiIiXXO2rMOcOf4vKEgeq6oWUEqD7UgfWILUV9t4txaQq5pHfqLpYCg2NhaA3EMUV2Eq4IkTJ+x6iyoyGAwwGAw+bx8Rke5VNcSRlibXqWFApFlV1QLqVns30vufxpBXWyKuQyffNMDL65t5StPBUHJyMmJjY7Fy5Up07CgXZbp8+TLWrFmDmTNnqtw6IiKdczXEIUly4b5BgzhkpiFV1QJqY9iP9N5HMPSl5mje1w+1gLywvpk3qB4MXbhwAQcOHLA+z8/Px/bt2xEVFYWmTZti3LhxmD59Oq677jpcd911mD59OmrVqoVhw4ap2GoiIlK0rENBgbyfFiov65jpsgl5c3fC+OF5ZO1tgzOiq/W15LA/MfTGP5D+bJO/psBf57+GWWoeHTni1xyhylQPhjZv3oyUlBTrc0uuz4gRI7Bw4UK88MILuHTpEkaNGoWzZ8+iW7du+P7771G3bl21mkxERIDyIY4jR+S6MUyu9ithFtj0ya8wvncKi3e0wDHz1WUvYkOOY0jbPUgfE41uf7sBUkhTdRppqXmUluZ4HTQ/BUhctZ6IiDyTmwtU+J9Zpxo2BE6duvqcydU+9cvyAzC+eRiLNibj9/Kr1Z7rS+dw33W7kP5obfR9uj1CwzUUkDpKwk9MRPH06YgcPtzn928GQ0RE5BmTCUhKcn+IwzIbmMnVXmNfC0hmXwtIw5OLHJRnKC4p8cv9m8EQERF5zjKbDHBviEOS5B6i/PzAHDLTQF2lwp0n8HnGbzB+b6kFJKtYC+ieV9qjdkxtv7bLm/x1/1Y9Z4iIiAJYaqrcw1N5iKNePaCoyPn7Ajm5WsW6SmfzzyE7YxeMK2oj52x7/9QC0gEGQ0REVD2pqYDZDIwaBZw8KW+rKhCqSCN1ZhTzZl0lhb1LJSdK8OXrO2D8IhTfqlELSAcYDBERUfVkZwNDhng280cjdWYU8WZdJRe9S5qqBaQDzBkiIiLPWZKo3V1SIRBzhpTOnsvJqXroz0nvkpAkQADz4qdi0tHROCsaWF+zrwWkD8wZIiIi7fNkbSnLbLLZswMnEAKUD+lVtV8VvUuSEDBDwt1H5mMMJmqnFpAOMBgiIiLPeZLzk5AgB0JV5daoMVvL1WcqHdKLi3N+LBfBYwgEmqIAPz/xMTr84xGEhvep5kmREgyGiIjIc0oDhHffBRo3VhbYqDFbS8lnulo6wjL0d+qU/dBhQgKOP/EKNnxegEEKmtO5T11AS0URg1yI2g0gIqIAZgkQLENflUkSkJgIjB0LpKfLuTSuAqG0NPveE8tsrexsrzXd7c+0LB0B2J+v5fnQoXIyeaVjmQ8fQaNXnsTmXTWUtWn/fjdPgqqDwRAREXlOSYCgNDfI1WwtQJ6tZTJ52trqf6alrlJ8vO2+CQnA4sWA0QhH85JCIG8bK32ACxHRcDlz6cMPvXueVCUGQ0REVD1VBQju1N1xlYxdsVCjt3jymampwMGD8qyxzEwgJwclG3djtfE4cPgwnPSRIQQCMeI46gwd6HQfq8OHvXueVCXmDBERUfWlpsr1daqT9OyN2Vru8vQzQ0NxuctNci2giVew/FAIBiIatyg5Vp063m0bVRuDISIi8o7Q0OotreHObC1vcfMzTZdNWPPeTiz66DyW7G2Ls6Lr1X1CQgCzgmNdc41320bVxqKLRESkDZYCjq5ma3mzUKOCzxQJCfj5lS9h/OAcPt/ZAsfMsdaXY0OO44F2ci2grg+1hNQ82XX7DxyQAyJ/nmeA8tf9mzlDRESkDd5MxvbCZwpIEEJg1JFJ6P54e8zZ3gfHzLGoL53Doy3ysOqtbTh8qSFmb+uDbo+0gVQj7OqxKqvY/vBw/58nVYnBEBERaYe3krG98JkFSMB9yMI88xOohRIMbboOy1/aiMJzNfHhnt645dmOCHVUCygqyvG2iu139zxNJnk5EKNR/pMzzbyKw2RERKQ9fqxAXbjtGH56ejF2/nwJRWU1cQKNcBTx2IBu6N94J9LvN+GeV9qjdkztqg/kbEV7i6wsx0GOq/NUowilRvjr/s1giIiIdOds/jlkZ+xCwZKNeKRkDhJxNdA4IcVgZ++x6PTxaERd06CKo1TgasFaT/OAsrOB++5z/rqjACuIMGeIiIjIi0pOlGDRU+swKG4jGjevha8/OYlXS15APGwDmBicxG15ryJqR47yg/uiRpLJBDz+eNX7PP44h8y8gFPriYiCnRqLnmrE5QuX8d3M7TB+egUrDrVHCW4CAITAhA8wGhKEfQFEIeSenHHj5NpJSq6VL2ok5eYCp09Xvc/p0/J+t96q/Lhkh8EQEZFW+CJo0WG+iaUWkPHD88jaZ1sLKDnsT6R3/QOP3nEYsa8WOj9IxZ4cJbWTfFEjKTdX+X4MhqqFwRARkRb4ImhxltBrWYDUV7OzqsPDgFCYBX5e8AuMc0//VQuoo/U1m1pAI2+AFNJUnpWlhNKeHKUr2vfqpex45FcMhoiI1OatoKViIBETU/UCpO4OA/mDBwHhL8sPwPjmYRg3JuOP8jbW7Q2ks7jv+l1If6wu+oxth9DwPrZv9HZPjqVeUVqafG0rXndPawf17Qu8/rqy/ahaOJuMiEhN3pqF5CiQUCInRxs3U2cBoSWQqBAQ5ucVYNG032HMa4Jdpddbd62FEgxqth3pw2vg9hc7ILxOuPPP81W1a0ffQ2KiHAi52wtnMgGNG1edNxQdDRw/rp2A1sv8df9mzxARkZrcmYXkLGhxVd+mKlpYDNRkctmLZRo1Fh98Vh+ZKxthw4W2ABIBADVwGXfGbkP6/SYMfLk9asf0VPaZrnpyhJCntK9d617uljcWrK3Yxvnzq55aP39+0AZC/sSp9UREaqruLKSqAgkltLAYqIKAMPT4UWQtDcGGC20hwYxbGmzFRyPW4vgfF7H8WDcM/cdNrosiVuasCnTIX7fG2bOBlBS5Byk7W/lxLQvWpqdfDWA9rR6dmirXEnJUqTrIawz5E3uGiIjUVN3cFVeBhDPeTuitzkw4hQFhL8PPuPcuCUNebYm4Dp2q0dgKKvbkLF8uB0CVg5XqJJx7IzHem71N5BBzhoiI1FTd3BWjERg2zL3PrJiH442bbDVv+Fe+WYkad/V3/Tm+zG/yRQVpN/KgyDFWoCYi0oPqrtTuyTCXZTFQQA4AUlLkgMqTISHLDb9yEGHpTXFyLNNlE1a/vQ2PtcxD3F0dUYAEmO3LH8okSU5C9uW0dG9XkHaVBwXIs/lYPVoTGAwREamtOiu1W+rbVA6kLCw9Gj/8AGRmyr0r+fnyax4EMTbcvOELs8DGj3djXMc1SKx5Erc+1xEf7e2N02iIKVIGJAj7etCeTkt3l7crSPtieQ7yGeYMERFpgad5IUrq28yZY1uhWMHsLTz5JHDpkhygOWuHwht+/nQjPvo2AYt+rqoW0AhIX9V3PNzmybR0d3m77pAvlucgn2EwRESkFZZZSO6y9CwpDSSUBDEnTwIPPnj1OI7yfxTeyF96NRSL0BeAo1pAvW3PQ61EYW9XkPbF8hzkMwyGiIiCgTuBhLu9Ec5mUym8kZ9EQ9wTu1FZLSBPA8KqKJnp5u0K0lyeI6BwNhkRkd7k5srJ0u5wNJvKZII5oSmkwqMOU5/NAC7WisGVHb+iwbXR1Wy0h9yd6ebNCtKW5HLAcXDF2WQu+ev+zWCIiMgXfLECvbe4ms5flZwclLS+EV++vgPGL0JhKDyERRgKAAjB1WMJAJIkqXvD93Rquze/O28GVzrEYMiLGAwRkV/5YgV6b3PWa+HCew0nY8Kp53ERV6s9j6sxF9MwCbWuFNvuHB0tLxehxjm7qhsEAI0aya+HV7GGmbfaotXAWONYZ4iIKBB5WHfH75xN53ch61RfXERtNA87hJduysWu7P14d1ET+0AIAM6cUe+clVTmPnlSDlJ93b7Ky3MwENIczQdD5eXlePnll5GcnIyaNWuiefPmmDp1Ksxms9pNIyKypUahPZOpeuteHTwo1yB66SWgbl046yMyQ8IRNEHH9mZs+Gg3DpQ1xbSf+qLNPc3lc3ZEzeKCSpPET57UVpBKqtD8bLKZM2di3rx5+OSTT3DDDTdg8+bNGDlyJCIjI/G0sx8gEVF1eDqs4Y0V6N3hjeG45cttjiHhr3yfCruY/3oea3wH78aGAMd2AXmn5Ovi73MGlH0/7k5ZHzdOno3HXhtd0nwwtH79egwaNAh33303ACApKQlGoxGbN29WuWVEpDpf5GJUJ8DwZ6E9Z8nBbiwqWjj1X2g8+UkIVD1MEJKYCAwditDnn7O/Lpa8I1e8VVxQ6ffjamp7Rb4I2CigaH6Y7Oabb8aqVauwb98+AMCOHTvw448/4q677lK5ZUSkquzs6q+r5eiY1cn38VehvWoMxx3bfhxzUtfgpto7cGXy6w4DIZtp8g0bAg88ALz1luPrMnu2sjZ7o7igO99PxTXflGI1aP0SGmc2m8WECROEJEkiLCxMSJIkpk+fXuV7SktLRVFRkfVRUFAgAIiioiI/tZqIfCorSwhJEkK+9V99SJL8yMpy/5jl5UIkJNgfs+KxExPl/Vwdw1HblB5DiZwc5+2s+MjJEUIIceaPs+LDh/LELQ22iBCUC0CIPlB4DFcPSRIiNLR6100JT7+frCwhGjZ063qRdhQVFfnl/q35nqHFixfjs88+Q2ZmJrZu3YpPPvkEb731Fj755BOn75kxYwYiIyOtj8TERD+2mIh8xmQCVq0CHnvM+0nK3lhYs7or0CulsAdj3Yw1uCd2Ixo3r4XH/tMLq892ghmh6F5nFybcuLp6bbAQ4uq19uU5e/r9pKbKPUeNGjl/ryTJtX9YDVq/fBpqeUFCQoKYO3euzbbXXntNtGjRwul72DNEFISysqruGaju/+VnZio7bmamZ21NTPSsx8oRhT1DfZBjfdo2Yq+Y3j9H/LHmT7eOofgxbpxvz7m634+lN7Fyr111ehPJ5/zVM6T5BOqLFy8iJMS2Ays0NLTKqfUGgwEGg8HXTSMif3GWLFwVd/M/vJnv4+6Co0oSwSvuExNTZXKwGRIOIwFHQpvipW65SH8uHm3uvR7A9Vd3cifBWIlBg+S8Il8VF6zu9+PuYrakLz4NtbxgxIgRIj4+Xnz11VciPz9fZGdni4YNG4oXXnhB8TH8FVkSkQ+4yhXxVs+Qv/J9KnPUi5SQYNtT4WCfy7UjhRkQJti21wRJmAGxb/S7wmwyu/5sZ+er9OGr61KZt76f8nL570Zmpvynr9tN1eKv+7fmg6Hi4mLx9NNPi6ZNm4qIiAjRvHlzMWnSJFFWVqb4GAyGiAKYu8M51bk5+3soRUkiuJN9TJCECRAnEW2z3Vw5kFLShvh4zwMhfw4xcahLd/x1/+baZESkbUajPHVeCW+sBu6vhTVdrZ0lSUB8PMpLyxF6qtDpqvCldWNQI/M/qHH+jOdDUyYTMG0aMHmy43YAwHPPyd+F2guOcuFTXeFCrV7EYIgogOXmyjWElPDWTdEfC2u6c16u5OT4rqJ1xWuqlQVHtdIO8jl/3b81n0BNRDqnJNE3OhpYvNh7i2BaFtb0JW8W+PPWsVwlfvvjuiihlXZQ0HArGCooKGDNHiLyL0vtnrQ0ecimYkBkGcKZPx+49VZ12ucpb1Rk9sWxGGiQDrlVdLFly5Z45ZVXUFJS4qv2EBHZs0yLjo+33Z6QUL38IBVcvnAZKyZtxLCHQnEY8TA7zAaCHOglJMiPysUMK+7DYoFE1eZWztC6devwzDPPoKCgANOmTcPIkSN92TavYc4QkR/4I48jQHNFTJdNyJ2zA8aPSpC1vy3OifoAgHuRjSVIgwRhGxJVTAQHri6G6qhXLMCCQSJ3aDqB+j//+Q8mTZqEhg0b4t1330VfjXepMhgi8rHqrPQepIRZYOO/f4Fx7ml8vqslCs2Nra/FhRTigfZ7kT62IW6suwfSM+Oqnh3FGVRVC9AgmVzTdDAEAJcuXcKMGTPw9ttvo3///njzzTdx7bXXert9XsFgiPxKb/8wO6sO7ajnQgfXZvfS/TC+dQTGn5sjv7ypdXsD6SzSWuxC+mN10XtMO4SGVzhvdytQB+m18wgD8aDmt/u3pwWKSkpKxNq1a8W4ceNESEiIMBgMYvz48aK4uLj61Y+8jEUXyW+UVBMOJu6sJB7E1+b3nENiWr8c0cawz+b0auGCSG/2o1jx8kZRdl55oVhSSEnRSgpomiy6OG/ePGzatAmbNm3Cb7/9htDQULRr1w7du3dHhw4d8N///hf79u3D0qVL0aVLF99FcG5izxD5hTs9JMFCaa2cjAxgypTAujYuemKObT+Oz6fugfH7aGwsaWPdXgOXcWfsNqTfb8LAl9ujdkxtNVof/JQUrUxIAPLz2YMWwDQ5TJaYmIju3btbH126dLFbEHX69OnIzMzE7t27vd5YTzEYIp/T6z/MSqtDR0UBZ844fk2L18bJ0MuFl6Zj0YYkGL+sjdyz7WGG3N4QmJDSYAfS7ylB6uS2aJBcX51264nSQNxbBSlJFZosulhQUOByn0ceeQSvvPKKxw0iCkhr1zoPhAC5R6SgQN4vmP5hVlrfxlkgBPj/2rjKvXHSw2c+fBi1Ro3A/7AEqyFPZe9eZxfS+5/BkMmtENuuk+/bTlcpLTTpzeKWFLS8XoE6JiYGq1ev9vZhibRNr/8wu6oOLUlAgwZVB0MW/rg2X3wBjBoFnDp1dVvFZFuTCXj6aQgh7Kr/hEBeC+x9jEbXfpF44OXrkdy7re/bTI4pDcS9WZCSgpZbRReVkCQJffr08fZhibRNr/8wW6pDA/aFAS3Pn35a2bF8fW1eeAEYMsQ2EALkHr20NJg+X4Itoz8GDh92VgYRIRCIQyEmvBSK5N6sxq8qSyDOgpTkBV4Phoh0Sc//MLuqDj1pkm+vjckk548YjfKfJpP9PkuWAG++6fQQQggUPvA03v6XwmTnYOvhC0RKAvHZs7WTh0aaxmCIyBv0/g9zaipw8KCcrJqZKf+Zny9vr3htHBECGDrUs2uTnS0nrqekyIncKSny8+zsq/uYTPLQWBUkAPE4imY4pOxzg62HL1AF0TItpC6Piy4GEs4mI79hpWDnXnjBee+MJLl/81JaykDprCMA5R8tRNiUl6vOgdLazDdiQcogpsmp9YGKwRD5Ff9htuft0gMKj3ds6QZsf3Ie7tz8mrJ25uTIyd5cC4xIEzQ5tZ6IFAgNDa7p897g7dIDCo83rMseCNyCO6EgGGrU6GrgumSJ4yUe2MNni4E/BQkGQ0R65c8bmbdLDyjcLxbH8WftVjhvboQ6l046nSUGAHj//avnn5oKDBrEG31VuCYYBREGQ0R65O8bmbdLDyjc753ZoYh7ugOQPc9xfpHF888D999vu409fM45y9c6ckTezqFECjCcTUakN5YbWeVhJsuNrOJMLG/xUukB02UTVr25FY89DhxBE5id9fX8dby4MffJzy2zjhISbPdr1EguxDhrlpsnpGN/FaZ0GFhato0b57jEAZFGMYGaSE/UXEPNEoQBbiUmC7PAxn//AuPc0/h8V0sUmhsDAO5FNpYgDRIqVYuu6njMcak+rglGfsQEaiLyPjXXULP0zihMTN6VtQ/Gt49i0abmyC+/uip8A+ks0lrsQvpjyRAJnyPk2WeUJzpz6Kv69Lr0DAU1BkNEalGjl0LtG5mLxOQ/cv+EcdofMK6Nxy9l1wO4HgBQGxcwqNkOpD9UA/1f6IDwOr3/OmBH4L57q38d2WOknF6XnqGgxmCISA1qzcRReoP69Vd5OMQXQUGl3plj249jccYeGFdG4+eSNgCaAgDCUYY7Y7cjfYgJAya1R+2YnoqO5zZvfxfBHlgpWZw3ISE4l56h4CV0oKioSAAQRUVFajeFSIisLCEkSQj5VnL1IUnyIyvLd59dXi5EQoLjz3f0SEjwSXvO/HFWfPhQnrilwRYhwWT9uBCUi9uiNouPH84TZw+e8/rn2vH2d5GVJV8zP1xDVVmuW+Vr54+/w6Qr/rp/M4GayJ/UTGC2cJbI7Kw9gFemSpecKMGK13bA+EUovj3eEVcQbn2tR51dSL/9DO5/tRVi28VU63MU8/Z3oXR5kGDBpWfID7gchxcxGCJNMJmA994DnnnG9b6+nonj6EbmTDUCtLLiMnw3czuMn5mw4s/2uIirq8K3i9iL9D7HMPTla5F0c0IVR/ERb86K0kKQq4ZgHxIk1XE2GVEwcSf4AHw/E6diIvOqVcDrrzvf180ZZqbLJuTO2QHjRyXI2t8W50Q362vNww5hWPd8pD+fgNb3tADQovrn4ilvJpOrOUtPTZydR0GCwRD5n97+b9LZ8ElVvDkTx9n1ttzIvBAU2NcC6mR9LS6kEA+034v0sQ1x44jWkEKaVfOEvMSbs6LUnqVHRNXCYIj8S2/rGVVVrdcRb8/EUXK9qxEUuK4FVBe9x7RDaHhsdc7CN7w5K4rTzYkCGnOGyH/0lmAKKM9LAbx/HVz1SGVlyZ9jyXdxFRT8le9iWwvoOutu9rWAwu2PpTUeVsW24+Y1JCJl/HX/5tpk5B96Xc/InWGRhATvBUJKeqQef1zeLzRU7ikC7NcO++v52aenYHbaj+hWZzeuSWmKl3/oi1/KrkM4yjAodiMWPbUOx49L+O/BnhgwtWtgBELA1arY8fG22939LhRcQ8yezUCISKPYM0T+odf1jJSe97vvAmPHeu9mqfRzMzKAV1+V/9vBkNqFWjF4E8/jtYvjIf76f6cQmHBL1Hak33MRqVPaoX6zSO+0WU3eymPjdHMir+LUei9iMKQBRiMwbJjr/TIzgfR05cfVejK2WsMnSq93dDRw/Lj1sy8cKcKGUQtx6ocd+P1iLFbhVqxFL/TEOvQxbET7bgbcNPsBxHb0Ye6L1r9TVwK9/UQawqn15Fog/aPriwRTNZOxlV57y/BJWpoc+DjKS/HF8InS63j6NC5/uwrf/BQJ42cmSAUHMQtv4TbI13QSZsCMEITADJQByANwz9u+u8bBkGDP6eZEgcen9a01IiiX4wi0sv+uloGQJCESE+X9lFBzSQtPrr2j9yQm+q6d5eVCREUpWnLjEXwoACHuRZYwQRImV+/x1TVW8zslIk3ichxeFHTDZIE6K8vbM3fUqPZbnWvv7568qVOByZNd7tYXOfhduha7wjoh8spJSC7fAe9fY71WcK6uyn+nbroJWLcuMHqLiRTgbDJyLJBnZXlr5o471X69KdCu/aRJQHQ0nP3fjhkSzoTFYMrbdXHwu72orzQQArx/jdX6TgNZdrYcQKakyPlhKSlArVq2z5OS5P2IqEoBEQwdOXIEDz74IKKjo1GrVi106NABW7ZsUbtZ6gj0m0ZqKnDwoDxrLDNT/jM/372eLLWq/Vbn2ju6cVV1ozKZ5BlhRqP8p5sB1h+5f2LaHWsxrmgKBCSYKzcVcmdL1OJ/ou/4zgg9dcKt41t56xqzgrN7LD2Ulf8+Vv57cuSIvB8DIqIqaT6B+uzZs+jZsydSUlLwzTffICYmBr///jvq16+vdtPUEQw3jeommKpV7dfTa+9saM1yo6rcK+ZhEvGx7cexOGMPjCuj8XNJGwBNAfTFUTTCByFPoaH5asAjVZ7u7em18tY1ZgVn5dypai6EHPWOGyevRcchMyKHNB8MzZw5E4mJiViwYIF1W1JSknoNUhtvGsqWUYiPl28aRqP3cic8ufauhtYq36jcDJzO/H4WWRm7YPyyLnLPtYdAYwCVawHdgfoJR6vOV3J1TSvz9rIh3lwaI9i56qGsLFgXiSXyJp+mZ3tBq1atxLhx40RaWppo1KiR6NChg5g/f36V7yktLRVFRUXWR0FBQfDMJvP2rKxAZZl5VPk6WJ5HR3t/pp0n1z4nR9GMLpGTc/X4Vc3iSkwU5w+fE/8d9aMYELNR1ECZzS496uwU/7gvVxzbcdx719Tfs8kcfaecTXZVZqayv1OVH5mZarecyG3+mk2m+WDIYDAIg8EgJk6cKLZu3SrmzZsnIiIixCeffOL0PZMnTxaQ0yJsHkERDAnBm4aFo+nqlYMgb18bd6+90htXZqbiwOl2/M9mU7uIPWLG7Tkif21B9c7Ncn6Vr2loqO1zX5YE8HcJgkCkNMB2FHATBRhOrf9LeHg4unTpgnXr1lm3PfXUU9i0aRPWr1/v8D1lZWUoKyuzPi8uLkZiYmLwTK0HWPbfouLU4pgY4OGHfT89251r784yJMeOKaoanY5MbAq7Cend85H+fAJa33OtW813Se3p2oFUTFQNrqqaV8ayBBTAWIH6L3FxcWjdurXNtlatWiErK8vpewwGAwwGg6+bpq7UVDnPRO83jYrJ2Lm5ymd7VSd3wp1r704ujMIZgJNfuIQWt/8B6XghUA+AKdm737ujBHd/5pqwgnPVqqpqXhkXiSVSRPPBUM+ePbF3716bbfv27UOzZs1UapGG8KZhy58z7ZRee4XLcexa9juMb5owBnGIRSFCHFQHEpIEKSoKLf/7KjDryNUXAm25Cqo+S82uyj2UoaG20+sTEvTXW0zkAc0HQ8888wxuuukmTJ8+HUOGDMHPP/+M+fPnY/78+Wo3jbRGqzPtnNy4rjSMxfK4JzFlWFv8UnYdgOuxB3OxBGkwo1IRMEmCJARw+rT98Z1N0afg5qiHkhWoiTyi+ZwhAPjqq68wceJE7N+/H8nJyRg/fjwee+wxxe8PuuU4yDG1VohXymTCqY+XY9OH27FidzLmlz4EM+R2hKMMd8ZuR/oQEwZ3PATDKy/Y1xm6dMlxMASof25ERD7gr/t3QARD1cVgSEecrX8GyAGDCr0n9rWA5D4f21pA7VC/WeTVN1VOIjaZgNtuc/1hOTkcOiWioMEEaiJPWIakHn/cvhclKspvzbhQeAErXtsB45Ia+O5EB1xBb+trPersQvrtZ3D/q60Q266z4wNUzkkyGpV9sJYrjxMRaRSDIQpOjoaTzpzxaW5NWXEZvn1jO4yfmfBlQXtcRE/ra+0i9iK9zzEMfflaJN3c1v2DazUfiogoCHCYTItYZ8VzlrwhX9casnzcpcvYPm4hNq84gi8Lb8Q3uNOaB3RN2CHv1QLyZj4U/34RUYDgMJleebhIp9cE+o3SnZXlPcytEWaBDR/txi/Tl+GuQ/9EZxxDZwBPADiKOHyX/CTaTE5Dl+GtIIV4qQSEq9oyQgCPPur6OGr//SIi0qAQ17uQ31iSfyvfzC1Tp7Ozff/5SUlyxeRhw+Q/k5J8/7ne5MNaQ7uy9mFij1w0NxzGm0/sx98OTUYsbI8TJxVi5MEpuLHuHkghktufUSVLPlR8vOPXJ0+u+vtS++8XEZFGcZhMK/w8vGPH2WrplsKAgVLDxp3lLxT0DP2++hAWzciHcW38X7WA5Flgh9AM8TgCh+GOr78rkwmYNk0Ofhx9NmD/fan994uIyAP+un+zZ0gr3Bne8TaTSR46cRQXW7aNG2db2VarLMtfSE56ZSRJXkesVy+nhzi6tRCz712DbnV249pbm+HlH/ril7LrEI4yDI7bgB/u+ycSnAVCgG+/K4sPP3T+2YD996Xm3y8iIo1jMKQV/lxKorJgulFacmsA+4CoinWazvx+Fh8+lIeUBtuQ0DkGzyzrg59L2iAEJvSL2oJ/j1yL4wdLsfRod6TcF62sLb6a5u7J96Xm3y8iIo1jArVWqDl1OthulM7Wbaq0TlPlWkAm9EQvrMUDWIyoiEtodVcS7s9oi8ZtKtUCUnuauyffl9pt9odAT/4nItUwGNIKd1Y397ZgvFE6WVm+rKQc3760EcbPTFhR0AGX/qoFdC+y8QFGIxaF8vtLAfycAOybA7SplCul5ncFePZ9qd1mX+MsOSKqDqEDRUVFAoAoKipSuylVy8oSQpLkh3zLkh+WbVlZvvnc8nIhEhLsP7fi5ycmyvsFoPKycrFy5hbxt+vyRH3prM2pXRN2UGS2zBBmODj3qq67Wt+VEJ5/X2q22Zcs5+XO90dEAcFf928GQ1qTlSXf6Cr+o56Y6Pt/0IPsRmk2mcW6f+0UY9vlisYhx21OqUnIUfFMpxzx88JfhPnyFfvrrTQQVOu7sny2J9+Xmm32BUtg6Mn3R0Sa56/7N6fWa5FauQ+OhhoSE23ybLRMmAV2Ze+H8e2jWLT5GhwsT7S+1hAnMSExE3f2vYgWw7si9Ja+8jWt7lR8NfNUPP2+gim3xsulFIhIW1iBWs8qL9LpL07ybLR+o/x99SEYp+fD+GMCfi27HsD1AIDauIDBSTvwTNef0Omnf0AqOAJ8CvlhyScpK1P2Ic6SltX6rgDPvy812+xtwZb8T0SqYDBEtgLkRnl0ayEWT90D4w+NsKnkBgDyshfhKMNdcduQ/oAZAyZ1QK2840DaBPukYUvV5SlTlH2gVpPHA+T78plgTP4nIr/jMBkFjDO/n0VWxi5kflkXa861h/irTFYITLg1ajvSB13EvZPboX6zSPkNSqouW5a28MYCqOR/3lzAlog0h8NkpC0q5ZlUrgV0Bb2tr91UdyfSbz+L+ye3tq8FBCgrTnj4MJCRIfcQVV4AtYoijaQRVS1gy++PiBRiBWpyzc8LuJYVl2H5SxsxtOk6xMSF4P8+6ImvTnTFFYSjfcRevHFHLvLXHsZPxe0w5os+aNymkeMDKc0Tue46xwugJiQEzppseuZsAVt+f0SkEIfJqGp+WsDVdNmEnNk7YPyoBNkH2qJY1EUvrEUcjiEkVMK1PRrhgeebofU91yo/qLszjYJplpUe8fsjCjr+un8zGCLnfLzSuTALbPhoN4zvn8Hnu1vhuDkGgFwNei7GoAkq9Ox4Uk2Y+SRERAGNq9aT+nywgKswC+xcsg8Te+SiueEwbnqiLd7b2QfHzTGIks7gn/GvIwtpiEOlIS7L7C93huY8XLSViIj0hcEQOefFGi6/rz6E12/LRZtav6P9/dfjjQ19cbA8EbVxAf+X9BO+mrwJx87WxJPSvyBBQKp8AEvPzrhxco+PUswnISIiFzibjJyrZg0XxbWAGsqLpSI3V3lPlDu1dQK0mCQREfkHg6HqCPaETQ9WOj+9/wyyMnbD+LWlFlAsAEe1gLrbH8+X1YT1XpyQiIicYjDkKUfrQnmS5KtlCmu4XDh5Ccun7oAxqwa+O9ER5e7UAqqI1YSJiEgFnE3mCT9NN9cMB4GfiE/Az11H453NvfFlQQdcQi3ra+0j9iK97zE8MOlaJN2coPxzOPuLiIgq4NR6L/LqxfTxdHPNMplgWp2LXxdsxDd5dfDGkQdxFlHWl6+tcRDp3Q9i6HMJ7tUCqswSaAKOe6KCLdAkIiKnuByHVrkz3VxLOSoe5jfZ1gJqi+PmW62vxYccwwMd9yL9qRh0frAVpJCk6rfTMvvL0RDk7NkMhIiIyOsYDLnLl0m+vuJmfpMwC+zK3g/j20exaPM1OFje1vpalHQG97fcjfQn6qHX6HYICfNB/k4gzf4K9iR6IiIdYDDkrkBL8nWW32QpYlhh2On31YdgnJ4P448J+LXsegDXAwBq4wLuTd6O9BEG9Hu+A2rU+itB2mSSp8N7MxCoHFwMGaLd4EIPSfRERDrAnCF3BVKSr4L8pvKYOMzt9h9kror9qxaQzL4WUC3b9/oiEAik4EJvSfRERCpgArUX+Ww2GaDtJF+FC5X2RQ7WoK+DWkCRjt/gi0DA2TEtsrK0cU0B/SbRExH5Gdcm07JAWeJBYd5Sn4iNeC9tDY7uOoPvT3fGyH/3ch4ImUxy742joMXTJTOqOqbF44+7d0xf8sGabUREpB7mDHkqAJJ8L9eLRriC/TK+6Qb07aPsoL6YTefqmABw+jQwbRrw6qvKjulLgZhET0RETrFnqDosSzykp8t/aiAQKi8tx8o3tuBv169F7IAuKEACzPbLnsokCUhMtFlOwyVfBAJK9/3HP7TROxRoSfRERFQlffUMrV0LFBdrshenOoRZYP2HllpArXFCXF32Yqo0BfPFoxCQIMHxchpuXQdfBAJK9z19Whv1mzxYs42IiLRLXz1DAwYAw4bJScVJSXLSboASZoEdn+/FhO65SA4/gp5PtsXcXX1wQjRClHQGT7TKQ+7s7fjX5ZGQsrIgJXgpv8kSCEhe7G3q1QuIinK9H6CNoSfLmm2A/XXwNMgkIiLV6Gs2GQBrLrrWZn4pdGDVISyaYakFdHXZC/taQDVs3+jN4oC+mE03dSowebLr/XJy1O8ZsnBUCiAxkZWyiYi8hFPrnZgxYwZeeuklPP3005g9e7ai9zgMhoCAmQJ9dGshFk/dA+MPjdyrBeRL3g4ETCagcWN5KMwRrX5XrEBNROQzXJvMgU2bNmH+/Plo166ddw6o1XXEAJzefwZZGbth/Lou1pxrD4FYAEAITLgtehuG3nPpr1pA3dVpoLPZdIBnValDQ4H586uuX6TFoSdLEn0gYiBHRAQggIKhCxcu4P/+7//w4Ycf4vXXX/fuwbWQhwLgQuEFLJ+6A8asGvjuREeUo7f1tZvq7sSwO87i/smtEXNDFxVbWUHlQKC6FaS5SKv/BFK1byIiHwuYYbIRI0YgKioK7777Lvr27YsOHTo4HSYrKytDWVmZ9XlxcTESExPth8ksVMxDKSsuwzcztsP4XxO+LOiAS7g61NU+Yi/S+x7D0JevRbOeCaq0TzFvVqVmj4VvcSkRIgoQHCarYNGiRdi6dSs2bdqkaP8ZM2YgIyPD9Y4qTYEuLy1HzuwdMP77IrL3t0MRullfu7bGQaR3P4j0FxLRakALAC382jaPuKpKLUlyVepBg5QPmQXq0JPWefu7IiIKApoPhgoKCvD000/j+++/R0REhKL3TJw4EePHj7c+t/QM2fBzHkpVtYDiQ47hgY57kf5UDDo/2ApSSJLP2+NVvqhKTb7B74qIyI7mg6EtW7bgxIkT6Nz5avBgMpmQl5eHuXPnoqysDKGVghmDwQCDwVD1gf2QhyLMAjuX7IPxnWNYtPlaHDK1tb4WLZ1GWstfkP5EPfQa3Q4hYQFcrZjLUwQOfldERHY0Hwzdeuut2LVrl822kSNHomXLlnjxxRftAqEqffWVXypQH1h1CMbp+TD+mIjfLl8d6qqD8xicvKNCLaDeVR8oUHhalZq5Qf7HpUSIiOxoPhiqW7cu2rRpY7Otdu3aiI6OttvuUq9egI8SsI5sPobFr+2F8YcYbL7YGkAzAHItoLvjtiF9qMDdL7VHrYY3++TzVeXJ8hSczaQOLiVCRGRH88GQlp3efwZLpuyG8et6yCtqBwH5/6YttYDSB5fi3lfbIrKpSrWA/MWyPEVamnwzdVSVumJulrPZTEeOyNs5m8l33P2uiIh0IGCm1leHN6fm2dcCurrshW0toEbVbbZ/eHuZDldVqU0meV04Z0m8Wq00HWy4lAgRBQAux+FF1b2YQVMLqDJfDFW5Cq5yc+WFcl3R0hpkwYo5W0SkcawzpLKgqwVUma+GqlzVCOJsJu1gPSciIgAMhmwEdS2gitQsvMfZTI6xl4aISDW6D4bcqgUkxcg3rMU7AvuGpWbhPc5msseZdUREqtJtMOR2LaBgumGpOVTF2Uy2OLOOiEh1ukqg/i1nL/737tEKtYBkBpTirrjtFWoB1bI9QLAtbKmFJGbOZuLMOiIiFzibzIssFxM4C6A+ACAU5bg1enuFWkCRjt8cjDcsyzm5Gqry9TnpPU9GC0EpEZGG+SsYCvHZkTUpBD3r7sTc+9fg6O6z+O54Rzz8YDkif/qffGMymezf4k5+TaCwDFUBV3u3LPw5VGWZzZSeLv+pp0AI4Mw6IiKN0FUwtOu7I/ixuB1Gf94HMXvXyr0jKSnAsGHyn0lJ8vBNRcF6w0pNlYf34uNttyckBN6wX6DizDoiIk3Q1TCZtZvNnRygYB/K0PtQlZq0MlxJRKRRHCbzFVc1dgC5xo5lyMwyFbzycJKFJMmJv4E4FZyBkLq0MlxJRKRz+guG3M0BCtYbVna2smFC8i0OVxIRqU5/wZAnOUDBdsOyDBNWDgottW0qBkQmkzxUaDQ6TzKn6klNBQ4elIdaMzPlP/PzA+/vFRFRgNJfztDWrZ7nAAXDsJI7pQKWLw+eQpNERBRwWGfIi2wuZu3a+k5aVZoQnpEBTJkSPIUmiYgo4DCB2leCNQdIKaXDhHPmKE8yJyIiCmD6C4aA4MsBcofSmjVnzjh/LRALTRIRETmh24VakZoKDBoU+DlA7lKyanxUFHD6tOtjBVqhSSIiIgf02TNkocflIJQMEz71lLJjsTIyEREFAX0HQ3rlaphw0qTgLTRJRERUiX6HyfTO1TDhnDlyzSFJsh1O00OSORER6QqDIT2zDBM6Yuk9clRnaPbs4E4yJyIiXWEwRM7pNcmciIh0hcEQVa2q3iMiIqIgwARqIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREuqb5YGjGjBm48cYbUbduXcTExGDw4MHYu3ev2s0iIiKiIKH5YGjNmjUYPXo0NmzYgJUrV6K8vBz9+/dHSUmJ2k0jIiKiICAJIYTajXDHyZMnERMTgzVr1qB3796K3lNcXIzIyEgUFRWhXr16Pm4hEREReYO/7t9hPjuyjxQVFQEAoqKinO5TVlaGsrIy6/Pi4mKft4uIiIgCk+aHySoSQmD8+PG4+eab0aZNG6f7zZgxA5GRkdZHYmKiH1tJREREgSSghslGjx6Nr7/+Gj/++CMSEhKc7ueoZygxMZHDZERERAGEw2SVjB07FitWrEBeXl6VgRAAGAwGGAwGP7WMiIiIApnmgyEhBMaOHYulS5ciNzcXycnJajeJiIiIgojmg6HRo0cjMzMTy5cvR926dVFYWAgAiIyMRM2aNVVuHREREQU6zecMSZLkcPuCBQvw8MMPKzoGp9YTEREFHuYM/UXjsRoREREFuICaWk9ERETkbQyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0LWCCoQ8++ADJycmIiIhA586dsXbtWrWbREREREEgIIKhxYsXY9y4cZg0aRK2bduGXr164c4778Sff/6pdtOIiIgowElCCKF2I1zp1q0bOnXqhH/+85/Wba1atcLgwYMxY8YMl+8vLi5GZGQkioqKUK9ePV82lYiIiLzEX/dvzfcMXb58GVu2bEH//v1ttvfv3x/r1q1TqVVEREQULMLUboArp06dgslkQuPGjW22N27cGIWFhQ7fU1ZWhrKyMuvzoqIiAHKESURERIHBct/29SCW5oMhC0mSbJ4LIey2WcyYMQMZGRl22xMTE33SNiIiIvKd06dPIzIy0mfH13ww1LBhQ4SGhtr1Ap04ccKut8hi4sSJGD9+vPX5uXPn0KxZM/z5558+vZhaU1xcjMTERBQUFOgqV4rnzfPWA543z1sPioqK0LRpU0RFRfn0czQfDIWHh6Nz585YuXIl7r33Xuv2lStXYtCgQQ7fYzAYYDAY7LZHRkbq6i+RRb169XjeOsLz1heet77o9bxDQnyb4qz5YAgAxo8fj+HDh6NLly7o0aMH5s+fjz///BNPPvmk2k0jIiKiABcQwdADDzyA06dPY+rUqTh27BjatGmD//3vf2jWrJnaTSMiIqIAFxDBEACMGjUKo0aN8ui9BoMBkydPdjh0Fsx43jxvPeB587z1gOft2/MOiKKLRERERL6i+aKLRERERL7EYIiIiIh0jcEQERER6RqDISIiItK1gAyGPvjgAyQnJyMiIgKdO3fG2rVrq9x/zZo16Ny5MyIiItC8eXPMmzfPbp+srCy0bt0aBoMBrVu3xtKlS33VfI+5c97Z2dno168fGjVqhHr16qFHjx747rvvbPZZuHAhJEmye5SWlvr6VNziznnn5uY6PKc9e/bY7Bds3/fDDz/s8LxvuOEG6z6B8H3n5eVh4MCBaNKkCSRJwrJly1y+Jxh+3+6ed7D8vt0972D5fbt73sHy+54xYwZuvPFG1K1bFzExMRg8eDD27t3r8n3++I0HXDC0ePFijBs3DpMmTcK2bdvQq1cv3Hnnnfjzzz8d7p+fn4+77roLvXr1wrZt2/DSSy/hqaeeQlZWlnWf9evX44EHHsDw4cOxY8cODB8+HEOGDMHGjRv9dVouuXveeXl56NevH/73v/9hy5YtSElJwcCBA7Ft2zab/erVq4djx47ZPCIiIvxxSoq4e94We/futTmn6667zvpaMH7fc+bMsTnfgoICREVF4f7777fZT+vfd0lJCdq3b4+5c+cq2j9Yft/unnew/L7dPW+LQP99u3vewfL7XrNmDUaPHo0NGzZg5cqVKC8vR//+/VFSUuL0PX77jYsA07VrV/Hkk0/abGvZsqWYMGGCw/1feOEF0bJlS5ttTzzxhOjevbv1+ZAhQ8Qdd9xhs8/tt98uhg4d6qVWV5+75+1I69atRUZGhvX5ggULRGRkpLea6BPunndOTo4AIM6ePev0mHr4vpcuXSokSRIHDx60bguE77siAGLp0qVV7hMsv++KlJy3I4H4+65IyXkHy++7Ik++72D4fQshxIkTJwQAsWbNGqf7+Os3HlA9Q5cvX8aWLVvQv39/m+39+/fHunXrHL5n/fr1dvvffvvt2Lx5M65cuVLlPs6O6W+enHdlZrMZ58+ft1vs7sKFC2jWrBkSEhIwYMAAu/+zVFN1zrtjx46Ii4vDrbfeipycHJvX9PB9f/zxx7jtttvsqrRr+fv2RDD8vr0hEH/f1RHIv29vCJbfd1FREQBUuQirv37jARUMnTp1CiaTyW61+saNG9utam9RWFjocP/y8nKcOnWqyn2cHdPfPDnvyt5++22UlJRgyJAh1m0tW7bEwoULsWLFChiNRkRERKBnz57Yv3+/V9vvKU/OOy4uDvPnz0dWVhays7PRokUL3HrrrcjLy7PuE+zf97Fjx/DNN9/g0Ucftdmu9e/bE8Hw+/aGQPx9eyIYft/VFSy/byEExo8fj5tvvhlt2rRxup+/fuMBsxxHRZIk2TwXQthtc7V/5e3uHlMNnrbRaDRiypQpWL58OWJiYqzbu3fvju7du1uf9+zZE506dcJ7772Hf/zjH95reDW5c94tWrRAixYtrM979OiBgoICvPXWW+jdu7dHx1SLp21cuHAh6tevj8GDB9tsD5Tv213B8vv2VKD/vt0RTL9vTwXL73vMmDHYuXMnfvzxR5f7+uM3HlA9Qw0bNkRoaKhdtHfixAm7qNAiNjbW4f5hYWGIjo6uch9nx/Q3T87bYvHixXjkkUfw+eef47bbbqty35CQENx4442a+T+J6px3Rd27d7c5p2D+voUQ+Pe//43hw4cjPDy8yn219n17Ihh+39URyL9vbwm033d1BMvve+zYsVixYgVycnKQkJBQ5b7++o0HVDAUHh6Ozp07Y+XKlTbbV65ciZtuusnhe3r06GG3//fff48uXbqgRo0aVe7j7Jj+5sl5A/L/MT788MPIzMzE3Xff7fJzhBDYvn074uLiqt1mb/D0vCvbtm2bzTkF6/cNyLM1Dhw4gEceecTl52jt+/ZEMPy+PRXov29vCbTfd3UE+u9bCIExY8YgOzsbq1evRnJyssv3+O03rjjVWiMWLVokatSoIT7++GPx66+/inHjxonatWtbs+onTJgghg8fbt3/jz/+ELVq1RLPPPOM+PXXX8XHH38satSoIZYsWWLd56effhKhoaHijTfeEL/99pt44403RFhYmNiwYYPfz88Zd887MzNThIWFiffff18cO3bM+jh37px1nylTpohvv/1W/P7772Lbtm1i5MiRIiwsTGzcuNHv5+eMu+f97rvviqVLl4p9+/aJ3bt3iwkTJggAIisry7pPMH7fFg8++KDo1q2bw2MGwvd9/vx5sW3bNrFt2zYBQLzzzjti27Zt4tChQ0KI4P19u3vewfL7dve8g+X37e55WwT67/vvf/+7iIyMFLm5uTZ/by9evGjdR63feMAFQ0II8f7774tmzZqJ8PBw0alTJ5tpeSNGjBB9+vSx2T83N1d07NhRhIeHi6SkJPHPf/7T7phffPGFaNGihahRo4Zo2bKlzY9LK9w57z59+ggAdo8RI0ZY9xk3bpxo2rSpCA8PF40aNRL9+/cX69at8+MZKePOec+cOVNcc801IiIiQjRo0EDcfPPN4uuvv7Y7ZrB930IIce7cOVGzZk0xf/58h8cLhO/bMnXa2d/bYP19u3vewfL7dve8g+X37cnf82D4fTs6ZwBiwYIF1n3U+o1LfzWQiIiISJcCKmeIiIiIyNsYDBEREZGuMRgiIiIiXWMwRERERLrGYIiIiIh0jcEQERER6RqDISIiItI1BkNERESkawyGiIiISNcYDBEREZGuMRgiooBkNBoRERGBI0eOWLc9+uijaNeuHYqKilRsGREFGq5NRkQBSQiBDh06oFevXpg7dy4yMjLw0UcfYcOGDYiPj1e7eUQUQMLUbgARkSckScK0adOQlpaGJk2aYM6cOVi7di0DISJyG3uGiCigderUCb/88gu+//579OnTR+3mEFEAYs4QEQWs7777Dnv27IHJZELjxo3Vbg4RBSj2DBFRQNq6dSv69u2L999/H4sWLUKtWrXwxRdfqN0sIgpAzBkiooBz8OBB3H333ZgwYQKGDx+O1q1b48Ybb8SWLVvQuXNntZtHRAGGPUNEFFDOnDmDnj17onfv3vjXv/5l3T5o0CCUlZXh22+/VbF1RBSIGAwRERGRrjGBmoiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRrDIaIiIhI1xgMERERka4xGCIiIiJdYzBEREREusZgiIiIiHSNwRARERHpGoMhIiIi0jUGQ0RERKRr/w+Zn7hsMecGZwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_22_2.png"
      }
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own sdg\n",
      "[[3.38135654]\n",
      " [3.49216685]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using SGD\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 1000\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = (1.0/n)*training_gradient(y, X, theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "xnew = np.array([[0],[2]])\n",
    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
    "ypredict = Xnew.dot(theta)\n",
    "ypredict2 = Xnew.dot(theta_linreg)\n",
    "\n",
    "plt.plot(xnew, ypredict, \"r-\")\n",
    "plt.plot(xnew, ypredict2, \"b-\")\n",
    "plt.plot(x, y ,'ro')\n",
    "plt.axis([0,2.0,0, 15.0])\n",
    "plt.xlabel(r'$x$')\n",
    "plt.ylabel(r'$y$')\n",
    "plt.title(r'Random numbers ')\n",
    "plt.show()\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "# Can you figure out a better way of setting up the contributions to each batch?\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        theta = theta - eta*gradients\n",
    "print(\"theta from own sdg\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6d7455a",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Same code but now with momentum gradient descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "40a68e4f",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[4.12427537]\n",
      " [2.85355539]]\n",
      "Eigenvalues of Hessian Matrix:[0.33486875 3.91080327]\n",
      "theta from own gd\n",
      "[[4.12417157]\n",
      " [2.85365229]]\n",
      "theta from own sdg with momentum\n",
      "[[4.25050227]\n",
      " [2.8249367 ]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using SGD\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 100\n",
    "x = 2*np.random.rand(n,1)\n",
    "y = 4+3*x+np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "# Hessian matrix\n",
    "H = (2.0/n)* XT_X\n",
    "EigValues, EigVectors = np.linalg.eig(H)\n",
    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "eta = 1.0/np.max(EigValues)\n",
    "Niterations = 100\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "\n",
    "for iter in range(Niterations):\n",
    "    gradients = (1.0/n)*training_gradient(y, X, theta)\n",
    "    theta -= eta*gradients\n",
    "print(\"theta from own gd\")\n",
    "print(theta)\n",
    "\n",
    "\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "t0, t1 = 5, 50\n",
    "def learning_schedule(t):\n",
    "    return t0/(t+t1)\n",
    "\n",
    "theta = np.random.randn(2,1)\n",
    "\n",
    "change = 0.0\n",
    "delta_momentum = 0.3\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        eta = learning_schedule(epoch*m+i)\n",
    "        # calculate update\n",
    "        new_change = eta*gradients+delta_momentum*change\n",
    "        # take a step\n",
    "        theta -= new_change\n",
    "        # save the change\n",
    "        change = new_change\n",
    "print(\"theta from own sdg with momentum\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1f7f38d",
   "metadata": {
    "editable": true
   },
   "source": [
    "## AdaGrad algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/adagrad.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/adagrad.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b85cc96c",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Similar (second order function now) problem but now with AdaGrad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "3ced9e13",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own AdaGrad\n",
      "[[1.99999773]\n",
      " [3.0000164 ]\n",
      " [3.99998703]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using AdaGrad and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-8\n",
    "for epoch in range(n_epochs):\n",
    "    Giter = 0.0\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        Giter += gradients*gradients\n",
    "        update = gradients*eta/(delta+np.sqrt(Giter))\n",
    "        theta -= update\n",
    "print(\"theta from own AdaGrad\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed1fa9e4",
   "metadata": {
    "editable": true
   },
   "source": [
    "Running this code we note an almost perfect agreement with the results from matrix inversion."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "787b2b5e",
   "metadata": {
    "editable": true
   },
   "source": [
    "## RMSProp algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/rmsprop.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/rmsprop.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac61b6e0",
   "metadata": {
    "editable": true
   },
   "source": [
    "## RMSprop for adaptive learning rate with Stochastic Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "5febac4f",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own RMSprop\n",
      "[[2.01023308]\n",
      " [2.95235306]\n",
      " [4.04597076]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Value for parameter rho\n",
    "rho = 0.99\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-8\n",
    "for epoch in range(n_epochs):\n",
    "    Giter = 0.0\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "\t# Accumulated gradient\n",
    "\t# Scaling with rho the new and the previous results\n",
    "        Giter = (rho*Giter+(1-rho)*gradients*gradients)\n",
    "\t# Taking the diagonal only and inverting\n",
    "        update = gradients*eta/(delta+np.sqrt(Giter))\n",
    "\t# Hadamard product\n",
    "        theta -= update\n",
    "print(\"theta from own RMSprop\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee8a6301",
   "metadata": {
    "editable": true
   },
   "source": [
    "## ADAM algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
    "\n",
    "<!-- dom:FIGURE: [figures/adam.png, width=600 frac=0.8] -->\n",
    "<!-- begin figure -->\n",
    "\n",
    "<img src=\"figures/adam.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
    "<!-- end figure -->"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2d0f16b",
   "metadata": {
    "editable": true
   },
   "source": [
    "## And finally [ADAM](https://arxiv.org/pdf/1412.6980.pdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "59349e6f",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Own inversion\n",
      "[[2.]\n",
      " [3.]\n",
      " [4.]]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "theta from own ADAM\n",
      "[[1.99998193]\n",
      " [3.0000747 ]\n",
      " [3.99992263]]\n"
     ]
    }
   ],
   "source": [
    "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
    "# OLS example\n",
    "from random import random, seed\n",
    "import numpy as np\n",
    "import autograd.numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from autograd import grad\n",
    "\n",
    "# Note change from previous example\n",
    "def CostOLS(y,X,theta):\n",
    "    return np.sum((y-X @ theta)**2)\n",
    "\n",
    "n = 1000\n",
    "x = np.random.rand(n,1)\n",
    "y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
    "\n",
    "X = np.c_[np.ones((n,1)), x, x*x]\n",
    "XT_X = X.T @ X\n",
    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
    "print(\"Own inversion\")\n",
    "print(theta_linreg)\n",
    "\n",
    "\n",
    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
    "training_gradient = grad(CostOLS,2)\n",
    "# Define parameters for Stochastic Gradient Descent\n",
    "n_epochs = 50\n",
    "M = 5   #size of each minibatch\n",
    "m = int(n/M) #number of minibatches\n",
    "# Guess for unknown parameters theta\n",
    "theta = np.random.randn(3,1)\n",
    "\n",
    "# Value for learning rate\n",
    "eta = 0.01\n",
    "# Value for parameters beta1 and beta2, see https://arxiv.org/abs/1412.6980\n",
    "beta1 = 0.9\n",
    "beta2 = 0.999\n",
    "# Including AdaGrad parameter to avoid possible division by zero\n",
    "delta  = 1e-7\n",
    "iter = 0\n",
    "for epoch in range(n_epochs):\n",
    "    first_moment = 0.0\n",
    "    second_moment = 0.0\n",
    "    iter += 1\n",
    "    for i in range(m):\n",
    "        random_index = M*np.random.randint(m)\n",
    "        xi = X[random_index:random_index+M]\n",
    "        yi = y[random_index:random_index+M]\n",
    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
    "        # Computing moments first\n",
    "        first_moment = beta1*first_moment + (1-beta1)*gradients\n",
    "        second_moment = beta2*second_moment+(1-beta2)*gradients*gradients\n",
    "        first_term = first_moment/(1.0-beta1**iter)\n",
    "        second_term = second_moment/(1.0-beta2**iter)\n",
    "\t# Scaling with rho the new and the previous results\n",
    "        update = eta*first_term/(np.sqrt(second_term)+delta)\n",
    "        theta -= update\n",
    "print(\"theta from own ADAM\")\n",
    "print(theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d490115",
   "metadata": {
    "editable": true
   },
   "source": [
    "## Introducing [JAX](https://jax.readthedocs.io/en/latest/)\n",
    "\n",
    "Presently, instead of using **autograd**, we recommend using [JAX](https://jax.readthedocs.io/en/latest/)\n",
    "\n",
    "**JAX** is Autograd and [XLA (Accelerated Linear Algebra))](https://www.tensorflow.org/xla),\n",
    "brought together for high-performance numerical computing and machine learning research.\n",
    "It provides composable transformations of Python+NumPy programs: differentiate, vectorize, parallelize, Just-In-Time compile to GPU/TPU, and more."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bc7785a",
   "metadata": {
    "editable": true
   },
   "source": [
    "### Getting started with Jax, note the way we import numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "f1090d5e",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import jax\n",
    "import jax.numpy as jnp\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from jax import grad as jax_grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac6bf976",
   "metadata": {
    "editable": true
   },
   "source": [
    "### A warm-up example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ed55121d",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/jax/_src/numpy/lax_numpy.py:173: UserWarning: Explicitly requested dtype float64 requested in asarray is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more.\n",
      "  return asarray(x, dtype=self.dtype)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10febc640>]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAABQCUlEQVR4nO3deXxU1eH+8c+dmaxkhZCFECDsS9hBFNkURami4K6tim1tUyOVUmu/WlutbeXXRUstYLRWXLHugDu4sCj7JoussgWSAAnJTNZJMnN/fwxEEQQCydyZyfN+veYFTG7Iw4iZh3POPccwTdNERERExE9sVgcQERGR5kXlQ0RERPxK5UNERET8SuVDRERE/ErlQ0RERPxK5UNERET8SuVDRERE/ErlQ0RERPzKYXWA7/J6veTn5xMbG4thGFbHERERkTNgmiZlZWW0adMGm+3UYxsBVz7y8/PJyMiwOoaIiIichby8PNq2bXvKawKufMTGxgK+8HFxcRanERERkTPhcrnIyMiofx8/lYArH8emWuLi4lQ+REREgsyZLJnQglMRERHxK5UPERER8SuVDxEREfErlQ8RERHxK5UPERER8SuVDxEREfErlQ8RERHxK5UPERER8auA22RMREREmojXA3uXQvlBiEmB9kPBZvd7jAaNfEydOpXBgwcTGxtLcnIy48ePZ9u2bcddM3HiRAzDOO5x/vnnN2poEREROXMz188k9+PJMC0Lnr8S3vyJ78dpWeR+PJmZ62f6NU+DyseiRYvIyclh+fLlLFiwgLq6OsaMGUNFRcVx111++eUUFBTUP95///1GDS0iIiJnzla0nRkHPiHXVn7c87m2CmYc+ARb0Xa/5mnQtMuHH3543K9nzZpFcnIya9asYcSIEfXPR0REkJqa2jgJRURE5Ox5PWRv+Ahs5cxITAAgu9RFbkIcMxLjySlxkl08Hy72+G0K5pzWfDidTgBatmx53PMLFy4kOTmZhIQERo4cyV/+8heSk5NP+nu43W7cbnf9r10u17lE+v6slbW8unofOw+V87fr+jbJ1xAREQk4e5eCK5/so7+ckZhAbkI8HsMgp6SU7FIX4PRdlzncL5HO+m4X0zSZMmUKw4YNIysrq/75sWPH8vLLL/Ppp5/y2GOPsWrVKi6++OLjCsa3TZ06lfj4+PpHRkbG2UY6pRqPl//3wVZeW72fvCOVTfI1REREAk75wfqfjqisAsBjGISZ5tHiceJ1Te2sy8fdd9/Nhg0beOWVV457/sYbb+SKK64gKyuLcePG8cEHH7B9+3bee++9k/4+999/P06ns/6Rl5d3tpFOqXVsBOd3bAXAexsLmuRriIiIBJyYlPqf/qWVb6bCME1qDYPchLiTXtfUzmraZdKkScybN4/FixfTtm3bU16blpZG+/bt2bFjx0k/HhERQURExNnEaLAr+qSx9Oti3t2QT/bITn75miIiIpZqPxTi2jDDVsGGSN/77YyDh9kcEX50DYhBtjfGd52fNGjkwzRN7r77bt566y0+/fRTMjMzT/s5xcXF5OXlkZaWdtYhG8vYrDTsNoNNB1zsKao4/SeIiIgEO5ud3D6XkZsYD0DrujouqKomu9RFTomTGYnx5PYZ49f9PhpUPnJycnjppZeYPXs2sbGxFBYWUlhYSFWVbw6pvLyce++9l2XLlrFnzx4WLlzIuHHjSEpKYsKECU3yB2iIli3CGdrJN/Xy7oZ8i9OIiIj4hzepKxm1BgBXllfWT3tke2PISR+NN6mrX/MYpmmaZ3yxYZz0+VmzZjFx4kSqqqoYP34869ato7S0lLS0NC666CL+9Kc/nfFCUpfLRXx8PE6nk7i4uNN/QgO9tiqP+97cQPfUWD6cPOL0nyAiIhLkiot3cMk7E6gzDP4adwM/6Nyj0Xc4bcj7d4PWfJyup0RFRfHRRx815Lf0uzG9UnjgbYOthWXsPFRO5+QYqyOJiIg0qVcW/pU6w6BjNVx8ywMQ5v8t1b+t2R0slxAdzvAuSYCmXkREpHn4pGQlAN1sfYm0uHhAMywfAFf2aQPAuxsKTjuaIyIiEsy2bn+PnWEmDtNkRN9fWh0HaKbl49JeKYTbbew8VM62g2VWxxEREWky/1vlOzSuR2Uklw4YaHEan2ZZPuIiwxjRtTUA723QhmMiIhKaamsr+cy9B4AOkRcR4bB+ygWaafkAGNfXt++Ipl5ERCRUfb76SY7YbSR4vFx8Xo7Vceo12/IxukcKEQ4bu4sq2JzfNIfZiYiIWOnNbW8B0Lm8NSO6t7M4zTeabfmIiXBwcXffSbvvaupFRERCjLN0D0vxnT6fnnA94Y7AecsPnCQWOHbXyztf5mvqRUREQsp7y/9BrWHQwQ1jzr/B6jjHadbl4+LuybQIt3OgtIq1+0qtjiMiItJo5uZ/DkDrii71R4sEimZdPqLC7Vza03eE8DtfasMxEREJDbt2f8JXdg8O06Rd+k9x2APr7T6w0ljgqn6+qZf3Nhbg8WrqRUREgt/ba2YA0K0igivOO9/iNCdq9uVjWOfWxEeFcbjMzYpdxVbHEREROSeeuhrec20HIKJ6MAPbJVqc6ETNvnyEO2z8oHcqAPM09SIiIkFu+bqnOWw3iPd46dr1Z9hsJz+R3krNvnwAjOvrm3r5YFMhNXVei9OIiIicvTnbXgcgs6wlVw/oZHGak1P5AIZktiI5NgJnVS1Ldhy2Oo6IiMhZKXMd4LNa3xICb91l9GoTZ3Gik1P5AOw2gyv6+LZb110vIiISrD5a8Rhum0FGjcmArOswjMCbcgGVj3rHpl7mf3WQqhqPxWlEREQabs7+hQC0cnbg6v7p1oY5BZWPo/pnJJDRMorKGg+fbD1odRwREZEG2bt3CV/aarGZJtWOm+icHGt1pO+l8nGUYRiM+9Z26yIiIsFk3trpAHSvDOPifhdYnObUVD6+5djUy2fbDuOqrrU4jYiIyJnxeup4p2QzAKazP1ceXccYqFQ+vqV7aixdkmOoqfPy4aZCq+OIiIickVVfPkuB3SDG48WMu4mMltFWRzollY9vMQyDq49utz5vvaZeREQkOMzd8j8AOpcncGX/rhanOT2Vj++4up9vdfDSr4s45Kq2OI2IiMipVZQXsqDmEADFzlH1W0cEMpWP78hoGc2Adgl4TXhnQ4HVcURERE5p/orHqbYZtKkxSWozjqSYCKsjnZbKx0mMP3pv9Nz1ByxOIiIicmrz8j4FINnZngkD2lqc5syofJzED3qnYbcZbNjvZNfhcqvjiIiInNT+/ctZbbgxTJM95Vdwac9UqyOdEZWPk0iKiWBY5yQA5mrhqYiIBKh3Vj8BQLeqMPp1u4CYCIfFic6Mysf3GN//6F0vX+ZjmqbFaURERI7n9dQx98hGAGylWVx9dK+qYKDy8T0u7ZlKZJiN3UUVbNjvtDqOiIjIcdZufIkDdoj2etlTO54RXVtbHemMqXx8j5gIR/3cmaZeREQk0Mzb8hIAXcriGd27K+GO4HlLD56kFjg2hPXOhnw8Xk29iIhIYKisLOKjat9O3CWlwxjfL3BPsD0ZlY9TGNG1NQnRYRwuc7Ps62Kr44iIiADwyYppVNoMUmu9HHFcyqD2iVZHahCVj1MId9j4QW/fTnFztOeHiIgEiHn75gOQ5szg6v4Z2GyGxYkaRuXjNI4NZX24qZCqGo/FaUREpLkrLFjHCrMSgJ3OsfVnkgUTlY/TGNQ+kfSEKMrddXy85aDVcUREpJl7Z9U/MQ2DrpV24lr2p0danNWRGkzl4zRsNqN+z4856zT1IiIi1jG9XuYWrQMg3NWTCf2Da6HpMSofZ+DYf9xF2w9TXO62OI2IiDRXX371P/baIcrrZYPzyvqT2IONyscZ6JwcS+/0eOq8Ju/qpFsREbHIvE0vANC1PJZ+HTuSGh9pcaKzo/Jxho6NfrytqRcREbFAdVUJH1btB8BZOjTo9vb4NpWPMzSubxvsNoP1eaU66VZERPxu4aonKLMZJNd62VlzEZdnBccJtiej8nGGWsdGMLyL76TbOdpuXURE/GzO7vcBSHe14ZKebYmNDLM40dlT+WiAY1Mvc9Yd0Em3IiLiN4cObmKZWQHArtLLmdA/+Pb2+DaVjwa4tGcK0eF29h2pZO2+EqvjiIhIM/HuysfxGgadqwyqwvowvEvwnGB7MiofDRAd7qifY9PCUxER8QfT62Xe4dUARDl7Mq5vG8Lswf32HdzpLXBs6uXdDQXU1HktTiMiIqHuq21v87XdJMJrstF5BeODdGOxb1P5aKChnZJIiYugtLKWz7YdsjqOiIiEuDkb/gtAt4poWrfMoG/beIsTnTuVjway24z6HeXeWrvf4jQiIhLKatxlfFC5D4Dy0vOZ0D8dwwiuE2xPRuXjLFw7oC0An249RElFjcVpREQkVC1a9W+cNoOkOi8by0czYUDwT7mAysdZ6ZYaS682cdR6TN7ZoD0/RESkaczb9S4AGa5UzuuYTNvEaIsTNQ6Vj7N0bPTjzTWaehERkcZXVLSVJV4XAHtLL61/3wkFKh9n6ap+bXDYDL7c72TnoTKr44iISIh5f+XjeAyDTtUGh7z9GNs7zepIjUbl4ywlxUQwqptvk5e31mrPDxERaVzzDq4AoIWzK5dnpRIT4bA4UeNR+TgH1xwdAnt73QE8Xm23LiIijWPrtnlss3kJM002ll7JNSGy0PQYlY9zMLpHMnGRDgqc1SzfVWx1HBERCRFzNzwDQPfyKKJjMhjaKcniRI1L5eMcRDjsjOvrO9xHC09FRKQx1NZW8n75LgCqnIMZ3z8duy349/b4NpWPc3TtQN/UywebCqlw11mcRkREgt3nq2dyxGaQWOdlQ9mlXBtiUy6g8nHO+mckkJnUgqpaDx9sKrQ6joiIBLm5O98GoENZMr3atqZLSqzFiRpfg8rH1KlTGTx4MLGxsSQnJzN+/Hi2bdt23DWmafLwww/Tpk0boqKiGDVqFJs3b27U0IHEMIz6VqqpFxERORelJbtZ5HECsL90dEjt7fFtDSofixYtIicnh+XLl7NgwQLq6uoYM2YMFRUV9df87W9/4/HHH2f69OmsWrWK1NRULr30UsrKQncvjAkD2mIYsGxXMXlHKq2OIyIiQer9Ff+gzjDo4Ia82gH16wpDTYPKx4cffsjEiRPp1asXffv2ZdasWezbt481a9YAvlGPadOm8bvf/Y5rrrmGrKwsnn/+eSorK5k9e3aT/AECQXpCFEM7tQK054eIiJy9uQVfABDv7Mzo7im0bBFucaKmcU5rPpxO39BQy5YtAdi9ezeFhYWMGTOm/pqIiAhGjhzJ0qVLT/p7uN1uXC7XcY9gdP3ADADeWJuHV3t+iIhIA+3Y+SFf2Tw4TJNNpVdy/aDQnHKBcygfpmkyZcoUhg0bRlZWFgCFhb4FlykpKcddm5KSUv+x75o6dSrx8fH1j4yMjLONZKnLeqUSG+Eg70gVK3YfsTqOiIgEmXnrnwKge0UEjqj2jOza2uJETeesy8fdd9/Nhg0beOWVV074mGEcfz+yaZonPHfM/fffj9PprH/k5eWdbSRLRYXbubKvb9/9N7TwVEREGqCutpp3XTsAqHEO4poB6TjsoXtD6ln9ySZNmsS8efP47LPPaNv2m2Gh1NRUgBNGOQ4dOnTCaMgxERERxMXFHfcIVtcd3fPj/Y0FlGvPDxEROUPL1j1Nkd0g3uNlg2tM/ftJqGpQ+TBNk7vvvpu33nqLTz/9lMzMzOM+npmZSWpqKgsWLKh/rqamhkWLFjF06NDGSRzABrRLpOPRPT/e31BgdRwREQkSc7e/AUBHVxI926bQNQT39vi2BpWPnJwcXnrpJWbPnk1sbCyFhYUUFhZSVVUF+KZbJk+ezKOPPsrbb7/Npk2bmDhxItHR0dxyyy1N8gcIJIZh1O94qqkXERE5E07nPj6r860VLCi9mOsGBefax4ZoUPl48skncTqdjBo1irS0tPrHq6++Wn/Nfffdx+TJk7nrrrsYNGgQBw4cYP78+cTGhnaLO+baAW2xGbByzxH2FFWc/hNERKRZ+2j5Y9QYBu3cJnvrBnFVn9Dc2+PbGjztcrLHxIkT668xDIOHH36YgoICqqurWbRoUf3dMM1Banwkw7v4Vihr9ENERE5nbv4SABKdHRnTM4346DCLEzW90F1Ka6FjC4XeXLsfj/b8EBGR77F7z0I22GqxmyZbSsdxfTOYcgGVjyZxac8U4iIdFDirWfp1kdVxREQkQM1bMx2AbpXhhLXoyLDOSRYn8g+VjyYQGWbn6n6+w+ZeXRWc+5aIiEjT8tTV8I5zKwDe0v5cMyAdu+3ke2KFGpWPJnLD0aGz+ZsPUlJRY3EaEREJNCvWP8NBu0Gsx8uXZWPr3zeaA5WPJpKVHkePtDhqPF7mrtdhcyIicrx5214HoFNZS/p3SKdDUguLE/mPykcTMQyDG48eCvTq6v2YphaeioiIT3lZAZ/UHAbgUOkIbhzcfEY9QOWjSY3vn064w8aWAhebDgTnab0iItL45q94jGqbQdsak3zvMMZmpVkdya9UPppQQnQ4l/XynXfz6up9FqcREZFAMXf/ZwAkOdszrl9bosLtFifyL5WPJnbj0QVEc9fnU13rsTiNiIhYLS/vC9YaNdhMk62lV9S/TzQnKh9NbGinVrRNjKKsuo4PNumwORGR5m7e6mN7e4TRKimLPm3jLU7kfyofTcxmM7h+oK/VvrZK262LiDRnXk8d80o2AWA4+3DDoAwMo3ns7fFtKh9+cN2gthgGLNtVzN5iHTYnItJcrdnwPPl2aOH18lXFOCb0T7c6kiVUPvwgPSGqfsvc11dr9ENEpLmas2U2AF3K4hnRsyOJLcItTmQNlQ8/OXYP9+tr8qjzeC1OIyIi/lZZfogF7oMAFJWOaFY7mn6XyoefXNozhZYtwjnocrNo+2Gr44iIiJ99vHIaVTaDNjUmzrBLms0hciej8uEnEQ471xyd23tlpfb8EBFpbubumw9AsqsdNw5u32wOkTsZlQ8/uuk83xDbp1sPUeistjiNiIj4S37+alYabgzT5OvSsdwwuK3VkSyl8uFHnZNjOa9DS7wmvL46z+o4IiLiJ/NWTQOga5WDHp3OJy0+ytpAFlP58LNjox+vrs7D69VhcyIioc70eplX/CUADmcWN53XzuJE1lP58LMf9E4jLtLB/pIqPt9ZZHUcERFpYus2vkSeHaK9XvK913BRt9ZWR7KcyoefRYbZ6zeV0cJTEZHQN++rlwDoUhbHVQN74bDrrVevgAVuHuIbclvw1UEOl7ktTiMiIk2lqvIIH1XnA+B0Xli/51Nzp/Jhge6pcfTLSKDOa/LmWu14KiISqj5dOY1ym0FKrZeEtAlktIy2OlJAUPmwyM3HFp6uysM0tfBURCQUzd37IQBtXOncPKSjxWkCh8qHRa7s04YW4XZ2F1WwbFex1XFERKSRFRauZ7lZCcDh6qu4pEeKxYkCh8qHRVpEOLi6fuGp9vwQEQk1766ahmkYdK2yMaLfaMIdess9Rq+EhW45eq/3h5sKKCrXwlMRkVBher3MPbwGgHBnT27W3h7HUfmwUFZ6PH0zEqj1mLyxRgtPRURCxcYtr7PHDpFek6iWt9EhqYXVkQKKyofFfni0Dc9esU87noqIhIg5G58DoGt5C64/v5+lWQKRyofFruybRmykg31HKvnia+14KiIS7NzVTj6s8m0iaVYN45KeWmj6XSofFosOd3DtAN/phi8v146nIiLBbuGqJyiz2Whd52VAnzsI046mJ9ArEgBuObbj6ZaDHHRVW5xGRETOxVtfvwtAW1caN2lvj5NS+QgAXVNiGdwhEY/X5LVVuu1WRCRYHT60mRVUABAfcyNtE7Wj6cmofASIHw5pD/gOm/No4amISFCat+JxPIZB52qD8cOusjpOwFL5CBCXZ6WSGB1GvrOahdsOWR1HREQayPR6mXdoFQCtKnsxqluyxYkCl8pHgIgMs3PdQN/C09krtPBURCTYbNk+j10Ok3CvSe/ud2G3GVZHClgqHwHklqNTL59uO0TekUqL04iISEO8tuYpALpXRHPThYMtThPYVD4CSGZSC4Z1TsI0fWs/REQkONS6K/jY7fu+nR4xmpS4SIsTBTaVjwDzo/N9ox+vrsrDXeexOI2IiJyJBSv+hdNuo1Wdl6tGTLI6TsBT+Qgwl/RIJi0+kuKKGj7YWGh1HBEROQNv75gLQNeqNlzYNc3iNIFP5SPAOOy2+tNuX1y+1+I0IiJyOsXFO1ht9+3tMTDzdgxDC01PR+UjAN14XgYOm8GavSVszndaHUdERE7G64HdS3j93V9SZxh0rIabR99odaqgoPIRgJJjI7k8KxWAl3Tei4hIQJm5fia5H0+GaVnw/JV86t4FwLUVFcz+/DfMXD/T2oBBQOUjQN12QQcA5qw7gLOq1towIiJSz1a0nRkHPiHXVs62sDC2RITjME2O2D3MOPAJtqLtVkcMeCofAWpwh0S6pcRSVevhrbX7rY4jIiIAXg/ZGz4ip6SUGYkJ/DkpEYD2tbX8NyGOnBIn2Rvm+6Zk5HupfAQowzD40QW+225fXL4X09R5LyIiltu7FFz5ZJe6+EVJKesjfft5fB0eTk5JKdmlTnAd8F0n30vlI4BN6J9OTISDXYcrWPZ1sdVxRESk/GD9TzvW1tX/PMw0yS51nfQ6OZHKRwCLiXBwzYB0AJ5ftsfaMCIiAjEpAJjAX1smAGAzTWoNg9yEuBOuk5NT+Qhwtx7d8XTBVwfZX6LzXkRELNV+KMS14cGklhQ5HDi8XhbtO1C/BiQ3IR7i0n3XyfdS+QhwXVJiubBzK7ymbrsVEbGczU5un8uYFxsDwE1l5SR4vWSXusgpcTIjMZ7cPmPAZrc4aGBT+QgCtx+97fZ/q/ZRXasV1CIiVsqr821/4DBNbnOW1T+f7Y0hJ3003qSuVkULGg6rA8jpje6RQtvEKPaXVDFvfT43DM6wOpKISLNVdGAtOKBfZSypP5oD5Yd8azzaDyVbIx5nRCMfQcBuM+rXfjy3dI9uuxURsciu3YtYZveNdgzPvBsjcwT0vg4yh2uqpQFUPoLEjYMziAyz8VWBi9V7S6yOIyLSLOUufgTTMMiqDOMmneNy1lQ+gkRCdDjj+/luu31u6R5rw4iINEOHD23mY9O3f0dW/E1Eh2vlwtlS+Qgitw/tAMCHmwopdFZbG0ZEpJl55tMHqTUMOlcZ3HrZ3VbHCWoNLh+LFy9m3LhxtGnTBsMwmDNnznEfnzhxIoZhHPc4//zzGytvs9YjLY7zMlvi8Zq8vGKv1XFERJqNMtcB5lb5DozrZLuUdq2iLU4U3BpcPioqKujbty/Tp0//3msuv/xyCgoK6h/vv//+OYWUb0w8Ovoxe8U+3HW67VZExB9mf/Y7Kmw22taYjBt5r9Vxgl6DJ6zGjh3L2LFjT3lNREQEqampZx1Kvt+YnimkxUdS4KzmnS8LuG5gW6sjiYiENHe1k1eKV4PdIKNyICO66v3tXDXJmo+FCxeSnJxM165dufPOOzl06ND3Xut2u3G5XMc95Ps57DZuO7rp2Kwvduu2WxGRJjZv8R8pthu0rvMyfND/YRiG1ZGCXqOXj7Fjx/Lyyy/z6aef8thjj7Fq1Souvvhi3G73Sa+fOnUq8fHx9Y+MDG2gdTo3n+e77XZzvouVu49YHUdEJGR56mp4Nm8BAO2d3bh2cBeLE4WGRi8fN954I1dccQVZWVmMGzeODz74gO3bt/Pee++d9Pr7778fp9NZ/8jLy2vsSCEnITqcawf4plue/WK3xWlERELXJ8v+xn4HxHq8dOtyr26vbSRNfqttWloa7du3Z8eOHSf9eEREBHFxccc95PTuuLADAPO/Osi+Yp12KyLS2Eyvl2d2vAFAF2c6tw3vb3Gi0NHk5aO4uJi8vDzS0tKa+ks1K52TYxnZtTWmCc8v22N1HBGRkLNi3X/YYvcQ4TVp2WoSbRKirI4UMhpcPsrLy1m/fj3r168HYPfu3axfv559+/ZRXl7Ovffey7Jly9izZw8LFy5k3LhxJCUlMWHChMbO3uwdG/14dVUeZdW11oYREQkxz2x8BoAerkR+NHKYxWlCS4PLx+rVq+nfvz/9+/uGn6ZMmUL//v35wx/+gN1uZ+PGjVx99dV07dqV22+/na5du7Js2TJiY2MbPXxzN6JLazq1bkG5u4431uy3Oo6ISMj4auscVhjV2E0TbBMZ0C7B6kghpcErZ0aNGnXK2zs/+uijcwokZ85mM7jjwkwenLOJ55bu4bYLOmC36RYwEZFz9ezqxwHoVd6C64ZfrttrG5nOdgly1wxIJz4qjL3FlXy29fv3UxERkTOTl/cFC+p82xhUVYznB721ZrGxqXwEuehwBzef1w6AZz7fZXEaEZHgN+uLP+E1DHpWOBh93vWE2fVW2dj0ioaA24e2x2EzWL7rCJsOOK2OIyIStIqKtjK32reGrqbkEm45+o87aVwqHyEgLT6KK/r4hgWfWaLRDxGRszV78e+pMQw6VRv07PEjEluEWx0pJKl8hIg7h3cE4N0NBRQ4qyxOIyISfMrLCnjFuQUAR/EQfjqis8WJQpfKR4jISo/n/I4tqfOaPPfFHqvjiIgEnTcW/Z5ym0F6jUli+k/ITGphdaSQpfIRQo6NfsxeuY9yd53FaUREgkeNu4wXDi0HIPFIb+4c0dXiRKFN5SOEXNQtmY6tW1BWXcerq3RAn4jImXp3yR85bDdIqvPijv4xgzskWh0ppKl8hBCbzeCnw3yjH89+vps6j9fiRCIigc/rqWPWvvkApJd04o4RvbSpWBNT+Qgx1wxIp1WLcA6UVvHh5kKr44iIBLzPlv+DPXaTGI+XgrqJjM1KtTpSyFP5CDGRYXZ+dH57AP6zZPcpt8IXEWnuTK+XZ7a/CkAXZxtuvrAfDm0q1uT0CoegWy9oT7jDxpd5pazaU2J1HBGRgLX6y1lsstUR4TXZ7bqFGwdnWB2pWVD5CEFJMRFcO6AtAE8v/triNCIigeu/G54GoIcrgbGDzyc2MsziRM2DykeIunN4JoYBH285xI6DZVbHEREJONu2v8sXVGIzTfaVXMuPL8y0OlKzofIRojq2juGynr5FU08v1pbrIiLf9eyqfwCQVR7Neb0uJjU+0uJEzYfKRwj7+Ujfbbdz1h+g0FltcRoRkcCxf/9yPqwtAuBQ0RX8bERHixM1LyofIax/u0TOy2xJrcdk1he7rY4jIhIwnv/iEbyGQY9KB+0zf0CXlFirIzUrKh8hLvvo6MfLK/bhqq61OI2IiPWKi7bzduU+AKqKLuLnIztZnKj5UfkIcaO6JtM1JYZydx2zV+yzOo6IiOVmL/k9bptBx2pwtJzAoPbaSt3fVD5CnM1m8LMRvlb/7Oe7cdd5LE4kImKdyvJD/K90MwBhxUPIHtVFW6lbQOWjGbiqbxtS4yI5VOZm7rp8q+OIiFjmjcUP4rIZtKkxKQm7kUt7pFgdqVlS+WgGwh02fjLMd/967uKv8Xi15bqIND+17gpeKFwKQKsjWdw5shs2m0Y9rKDy0UzcPKQdcZEOdh2uYL4OnBORZui9zx/hoN2gVZ2XPbU/ZHz/dKsjNVsqH81ETISDiUM7ADBj4U4dOCcizYrXU8esvR8AkFGSyR0jehEZZrc4VfOl8tGMTLwwk6gwO5sOuFiyo8jqOCIifrNo5TR22U1aeL18XfFDbh7SzupIzZrKRzPSskU4N5/n+x9uxmc7LU4jIuIfptfLf7fOBqBraSrXDx1ITITD4lTNm8pHM3PniEzC7AYrdh9hzd4jVscREWlyaze+yJe2WsK9JrtcN9dPQYt1VD6ambT4KCYcXWQ187OvLU4jItL0nv0yF4CeZfFcNnAoiS3CLU4kKh/NUPbIThgGfLL1EFsKXFbHERFpMtt3fsBisxybabLvyLXcOVwHyAUClY9mqGPrGH6QlQbAkws1+iEioWvWir8B0Ks8igt6X0JqfKTFiQRUPpqtX4zybbn+7oZ89hZXWJxGRKTx5eev5oPawwAUF4/VAXIBROWjmcpKj2dk19Z4TY1+iEhoeuHzP+IxDLpV2unSZTyZSS2sjiRHqXw0Y5Mu7gzAm2v3c6C0yuI0IiKNp+TI17xZsRuAmuKR3H30+50EBpWPZmxQh5Zc0LEVtR6TXI1+iEgIeWXRg1TbDDq4ISXjh3RPjbM6knyLykczN2m0718Dr67O46Cr2uI0IiLnrrKyiNmlGwGIKh7MPZd0sziRfJfKRzN3QcdWDGqfSE2dl6cW7bI6jojIOXt70e9x2gzSak1iUn9KVnq81ZHkO1Q+mjnDMJg0ugsAs1fupajcbXEiEZGzV1tbyXMFSwBIOtKTSaN7WJxITkblQxjRJYm+beOprvXynyUa/RCR4PXh53+h0G6QWOclvGU2/dslWh1JTkLlQ3yjHxf7Rj9eXLaXkooaixOJiDSc6fXy7O53AGhf0oG7RvezNpB8L5UPAWB0j2R6psVRWePh2S92Wx1HRKTBlqz6FzvtJtFeL7T4OedltrQ6knwPlQ8Bjo1++O58ee6LPTgray1OJCLSMP/56kUAupWmkH3pUIvTyKmofEi9y3ql0j01ljJ3Hf/9XGs/RCR4rN/4MutttYSZJkb4TxjaqZXVkeQUVD6kns1mcM/RO1+e/WIPpZVa+yEiweE/a6cD0NMVx88uuwzDMCxOJKei8iHHOTb6Ue6u45klWvshIoHv668XsJhyDNMk0v4jjXoEAZUPOY7NZjD5kq4AzPpit+58EZGA959ljwLQqyKSH192g0Y9goDKh5xgTM8UeqTFUVHj0b4fIhLQCgvW8VHdYQDiuZYLNOoRFFQ+5AS+0Q/f2o/nl+7hiEY/RCRA/XfRH6gzDLpW2rn98p9p1CNIqHzISY3pmUKvNhr9EJHA5Szdw9wq39q0NuYYjXoEEZUPOSnD+Gbtx/NL91CsM19EJMA898n9VNkMOrjhljG/sTqONIDKh3yvS3ok0zs9nsoaD08t1uiHiASOqsojvOHcAEDH2qFc0Lm1xYmkIVQ+5HsZhsGvLvWt/Xhh2R4OuaotTiQi4vPyxw9QareRUuvl5ssftjqONJDKh5zSRd2SGdAugepaL9M/22l1HBER6mqrefXw5wBk1fbj/E5pFieShlL5kFMyDIN7L+sGwCsr95F3pNLiRCLSbHk9sHsJb71xO4UOgwSPl9vGTLU6lZwFlQ85raGdkriwcytqPSZPfLLD6jgi0szMXD+T3I8nw7QszOev5LWy9QDc6Kpi5e7Hmbl+pqX5pOFUPuSM/HqMb/TjzbX72XW43OI0ItKc2Iq2M+PAJ+TayvkiKpJtEeFEe714jDpmHPgEW9F2qyNKA6l8yBkZ0C6R0d2T8Zrwz481+iEifuL1kL3hI3JKSpmRmMCfWiUC0LWmhmcS4skpcZK9Yb5vSkaCRoPLx+LFixk3bhxt2rTBMAzmzJlz3MdN0+Thhx+mTZs2REVFMWrUKDZv3txYecVCU8b49v1458t8thS4LE4jIs3C3qXgyie71MU1rnLyw8LANFkfGUlOSSnZpU5wHfBdJ0GjweWjoqKCvn37Mn369JN+/G9/+xuPP/4406dPZ9WqVaSmpnLppZdSVlZ2zmHFWr3axHNFH9+q8sfma5hTRPyg/CAAHmBLRLjvOcMgzDTJLnWdcJ0EhwaXj7Fjx/LnP/+Za6655oSPmabJtGnT+N3vfsc111xDVlYWzz//PJWVlcyePbtRAou1fnVJV2wGfLzlIGv2llgdR0RCXUwKAHNiWtSXjzDTpNYwyE2IO+E6CQ6NuuZj9+7dFBYWMmbMmPrnIiIiGDlyJEuXnnxIzO1243K5jntI4OqcHMN1A9sC8NcPt2KapsWJRCSktR+KMy6N/3d0rceFlVWs3ZNXvwYkNyEe4tKh/VCLg0pDNGr5KCwsBCAl5fgGmpKSUv+x75o6dSrx8fH1j4yMjMaMJE1g8iVdCXfYWLn7CAu3HbY6joiEMpudOxNiqLbZSKzz8O+Dvu852aUuckqczEiMJ7fPGLDZLQ4qDdEkd7t890hj0zS/95jj+++/H6fTWf/Iy8trikjSiNokRDFxaAfAN/rh9Wr0Q0SaxradC9hq8x3t8Mcj1YR962PZ3hhy0kfjTepqTTg5a47G/M1SU1MB3whIWto3290eOnTohNGQYyIiIoiIiGjMGOIHd43qxCsr97G1sIy5Xx5gQv+2VkcSkRBjer08uuh+TIdB78oIhkxaDQUrfYtLY1Kg/VCyNeIRlBp15CMzM5PU1FQWLFhQ/1xNTQ2LFi1i6FDNx4WShOhwfjGqE+C788Vdp3vsRaRxzV/6D9Y63ISZJqM7/Y7oyAjIHA69r/P9qOIRtBpcPsrLy1m/fj3r168HfItM169fz759+zAMg8mTJ/Poo4/y9ttvs2nTJiZOnEh0dDS33HJLY2cXi90xNJOUuAj2l1Qxe8U+q+OISAipcZfxz20vADCwLI3bRl9tcSJpTA0uH6tXr6Z///70798fgClTptC/f3/+8Ic/AHDfffcxefJk7rrrLgYNGsSBAweYP38+sbGxjZtcLBcVbuee0b651n9/upOy6lqLE4lIqPjv+zkccBi0qvPyg/MeI8yuDblDiWEG2L2SLpeL+Ph4nE4ncXFxp/8EsVSdx8uYfy5mV1EFv7y4M1OOngEjInK2Dh3cxJXv30SVzWC4azDTc/6LzXbymxYkcDTk/VtVUs6Jw27jN5f5Csd/luzmoKva4kQiEuz+Pv9uqmwGnaoNbhrzqIpHCFL5kHN2eVYqA9snUlXr4XFtuy4i52D9ptl86C0GoB0/YkS3VIsTSVNQ+ZBzZhgGD/ygBwCvrcnToXMicla8njoeXfk3APq5Ysm5+hcWJ5KmovIhjWJg+0Su6J2GacLUD7ZaHUdEgtCcz+5ni91DC6+X9kn/R7dU3agQqlQ+pNHcd3k3wuwGi7cfZvF2bbsuImeuzHWAf+37AIBuRzrzyyvGnOYzJJipfEijad+qBbee3wGAR9/fgkfbrovIGXryo7s4YjdIrzHp2+sRkuMirY4kTUjlQxrVpIs7ExfpYGthGW+u3W91HBEJArv3LOSViq8BSCodzc8v6mVxImlqKh/SqBJbhDPp4i4APDZ/G5U1dRYnEpFA99dFv6XOMOhZEcaVI39Ni4hGPXZMApDKhzS624a2J6NlFAddbnIX7bI6jogEsMUr/skXVOIwTQz3T7h+UIbVkcQPVD6k0UU47Nw/1nfr7VOLvuZAaZXFiUQkENW4y/jr5mcB6F2STM6VN2DXhmLNgsqHNImxWamcl9kSd52Xv+rWWxE5iZfm38M+OyTWeYlO+A3Du7S2OpL4icqHNAnDMPjDlT0xDJj3ZT5r9h6xOpKIBJDDhzbzVNFKANoW9eO+K4dbnEj8SeVDmkxWejw3DPTN3z7yzld4deutiBw1bcEvqbQZdKw26NHjt3RsHWN1JPEjlQ9pUvde1o2YCAdf7nfy9roDVscRkQCwYfOrzKs7BICjZDy/vLSHxYnE31Q+pEm1jo0g56LOAPzto61UuHXrrUhz5vXUMXXl/wOgr6sFE0b9hPioMItTib+pfEiT+/GwDrRrGc1Bl5snF35tdRwRsdA7ix5kk62OaK8Xd10ONw3WrbXNkcqHNLkIh73+1Nunl+xib3GFxYlExAoV5YVM2/MuAF2LM5k8biwOu96GmiP9Vxe/uKxXCsO7JFFT5+VP735ldRwRscBTH95Fkd0grcZLfPJ9XNg5yepIYhGVD/ELwzB4aFwvHDaDj7cc4tOtB62OJCJ+tHfvEl4s3w5AQtFI7r9qsMWJxEoqH+I3nZNj+PGwTAD++M5XVNd6LE4kIv7yt4X3UWcY9KhwMGzQPaQnRFkdSSyk8iF+NeniziTHRrC3uJL/fr7b6jgi4gdLVj7BYspxmCaeion8bFRnqyOJxVQ+xK9iI8PqF59O/3Qn+Tr3RSSk1bor+Oum/wDQu7Q1d115E5FhdotTidVUPsTvru7XhsEdEqmq9fCX97dYHUdEmtDLC+5h79HzW6LifsPoHilWR5IAoPIhfmcYBn+8KgubAe9tKODzHUVWRxKRJlBUtJWnDi8DIKO4D/dfNdLiRBIoVD7EEj3bxHHr+e0B+P3cTVp8KhKCps2/m3Kbjcxq6Nf7QToktbA6kgQIlQ+xzK8v60br2Ah2F1Xw1KJdVscRkUa0ecubzKspBCCmbAKTLulucSIJJCofYpm4yDB+f2VPAGYs3MmeIu18KhIKTK+Xvyz7M6Zh0McVzZ1X3q1FpnIclQ+x1Lg+aQzr7Nv59PdzN2GaptWRROQcvbPo92y01xHl9dI69tdc1D3Z6kgSYFQ+xFKGYfDI1b0It9tYsqOI9zYWWB1JRM5BRXkh/9w9F4CeJR347YRxFieSQKTyIZbr2DqGX4zqBMAj73xFWXWtxYlE5Gw9+f7R81tqTS7s/yhp8drJVE6k8iEB4RejOtGhVTSHytw8Nn+71XFE5Czs2/c5syt9//92qLiYO0b0tjiRBCqVDwkIkWF2Hrk6C4Dnl+1h3b4SixOJSEM9+sm91BoGPSod/Oyqh3DY9RYjJ6e/GRIwRnRtzYT+6Zgm3P/WRmo9XqsjicgZWrR8Ol/YKnCYJr0SJjEos5XVkSSAqXxIQHnwih4kRoextbCMpxdr7w+RYFBbW8nfNz0FQP+y1kyZcKvFiSTQqXxIQGkVE8Efxvn2/vjXJzvYdbjc4kQiclJeD+xeAhvf4L+v3cLeMEjweLl26D+JjQyzOp0EOIfVAUS+a3y/dN5el8/i7Ye5/62NvHLn+dhshtWxRASYuX4mtqLtZG/4CFz5HLHZeKFtG7DbyKhLYK99KdDP6pgS4DTyIQHHMAz+Mj6LqDA7K3Yf4dXVeVZHEpGjbEXbmXHgE3JtvlHJJxITKLPbSKqrY2OEC1uR7laT01P5kICU0TKaX4/pCsCj72/hkKva4kQigtdD9oaPyCkpZUZiAn9qlchbsb7D4oocDnJKnGRvmO+bkhE5BZUPCVh3XJhJ37bxlFXX8cDb2npdxHJ7l4Irn+xSF3eVlPJaXCym4ZsSzSkpJbvUCa4DvutETkHlQwKW3Wbw1+v6EGY3+HjLQeZ9mW91JJHmrfxg/U+jvd/8YyDMNMkudZ30OpGTUfmQgNY9NY5fXtwFgIfmbeZQmaZfRCwTkwLA5vAwHm+ZAIDdNKk1DHIT4k64TuT7qHxIwMse1YlebeIorazlQU2/iFin/VDK49L4eWoyXsOgY00N6/bk1a8ByU2Ih7h0aD/U6qQS4FQ+JOCF2W384/q+OGwG8786yDsbdPKtiCVsdn4UH4XTbifG4+GFgkMYQHapi5wSJzMS48ntMwZsdquTSoBT+ZCg0CMtjknHpl/mbuJwmdviRCLNz/8++i1fO+owTJPHit3Ee785AiHbG0NO+mi8SV0tTCjBQpuMSdC466JOfLi5kC0FLn4/ZxNP/mgAhqHNx0T8Ye/eL/hn/rtgszG8sgPn/2oO5C3zLS6NSYH2Q8nWiIecIY18SNDwTb/0wWEz+HBzIW+vO2B1JJFmodZdwa8/votKm42uVTZ+de2L2BwOyBwOva/z/ajiIQ2g8iFBpVebeO4ZfWz6ZTMHSqssTiQS+v7+9i1sc3iJ9XgZ23EqnVMSrY4kQU7lQ4LOL0Z1on+7BMrcddz72pd4vbr7RaSpLF75b15x+06YPs99CT++ZKzFiSQUqHxI0HHYbTx+Qz+iwuws21XMrKV7rI4kEpKKirby4KZcAAa6Evm/W/6qQx6lUah8SFDKTGrB767oAcBfP9zK9oNlFicSCS1eTx3/985tlNhttHObXH3BU6TGR1odS0KEyocErR8Oaceobq2pqfPyq1fXU1PnPf0nicgZmfX+z1lhqyLCa9IrfBITBvWwOpKEEJUPCVqGYfC3a/uQEB3G5nwX//xYR3mLNIZNX73O9OIVAPQtyeKBG+6wOJGEGpUPCWrJcZFMndAbgNxFX7N0Z5HFiUSCW0V5Ib9Z9gh1hkFWeQS3X/EvEqLDrY4lIUblQ4Le2N5p3DQ4A9OEX722niMVNVZHEglaf3z7JvY7ILnWS5/0vzGimw6Jk8an8iEh4Q/jetKpdQsOutzc98YGHT4nchbmfvoAH3iLsZsmmZXX8+srR1kdSUKUyoeEhOhwB0/c3J9wu42PtxzkpeV7rY4kElT27l3Co3vnAtDvSAYP3Hwv4Q69RUjTaPS/WQ8//DCGYRz3SE1NbewvI3KCXm3i+e3Y7gD8+b0tbCvU7bciZ6LWXcG9n0yq3z597LDpdGwdY3UsCWFNUmt79epFQUFB/WPjxo1N8WVETvDjCzswqltr3HVeJr2ylqoaj9WRRALe43NvYavdQ5zHS4eoB7hhcEerI0mIa5Ly4XA4SE1NrX+0bt26Kb6MyAkMw+Af1/clKSaC7QfLeWjeJqsjiQS0xSv+xUtVvu3Tu5RcxB9uuEanRUuTa5LysWPHDtq0aUNmZiY33XQTu3bt+t5r3W43LpfruIfIuUiKieCJm/phM+C11ft5c81+qyOJBKSiw1t4cPPTAPQvTWDyDX8mPirM4lTSHDR6+RgyZAgvvPACH330Ef/5z38oLCxk6NChFBcXn/T6qVOnEh8fX//IyMho7EjSDA3tnMQ9o7sC8OCcTezQ9usix/F66rjvXd/26e3dMLLPDPplJFgdS5oJw2ziexIrKiro1KkT9913H1OmTDnh4263G7fbXf9rl8tFRkYGTqeTuLi4powmIc7jNbn92ZV8vrOIzskxzLv7QqLDHVbHEgkIT82dyPTSNUR4TUZ4J/HYj3+m6RY5Jy6Xi/j4+DN6/27y+6hatGhB79692bFjx0k/HhERQVxc3HEPkcZgtxlMu6kfybER7DxUzoNvb9L+HyLAhs2vkVuyGoB+zr48dMuPVTzEr5q8fLjdbrZs2UJaWlpTfymREyTFRPDvm/tjM+CtdQd4dVWe1ZFELFVeVsBvlvu2T+9dHsk9107XOg/xu0YvH/feey+LFi1i9+7drFixguuuuw6Xy8Xtt9/e2F9K5IwM6diKX4/pBsAf5m5mfV6ptYFELGJ6vTz45o3kOwySa72M6fkEvTMSrY4lzVCjl4/9+/dz8803061bN6655hrCw8NZvnw57du3b+wvJXLGfjGyE5f2TKHG4+UXL62hqNx9+k8SCTGvzP8tnxgl2E2TIfbbuH3U+VZHkmaqyRecNlRDFqyINERZdS1XT/+CXUUVnN+xJS/9ZAgOu7aPlhDm9cDepVB+kB1lBfxwWy5VNhvDXJn8M/ttIsPsVieUENKQ928t/ZdmIzYyjKduHcj4GV+wfNcR/vrhVn53RU+rY4k0upnrZ2Ir2k72ho/AlU8N8Ls2qVRFhJNca5IxYJSKh1hK/+yTZqVLSiz/uL4vAP9Zspt3vsy3OJFI47MVbWfGgU/ItZUDMK1lAlsiwon0ejkUZtCyap/FCaW5U/mQZmds7zR+MaoTAPe9sYHN+U6LE4k0Iq+H7A0fkVNSyozEBO5PasmL8b4h8GqbjZwSJ9kb5vumZEQsovIhzdK9Y7oxvEsSVbUe7nx+NYfLtABVQsTepeDKJ7vUxW1OF+/GfnM6bU5JKdmlTnAd8F0nYhGVD2mW7DaD6TcPoGNSC/Kd1fz8xdVU1+pfghICyg8CcMRmY3FUVP3TYaZJdqnrhOtErKDyIc1WfHQYz9w+iLhIB2v3lfLAWxu1A6oEv5gUKg2DnNTW7An3bR4WZprUGga5CXHHXSdiFZUPadY6to5h5g8HYrcZvLXuAE8t/v4TmEWCQW1qX+5JSWZTRAQAtzhdrN2TV78GJDchHuLSof1Qi5NKc6byIc3esC5JPDTOd8vtXz/cyoKvNBwtwcnrqeNXr/yA5VG+4nGdq4z7j5QCkF3qIqfEyYzEeHL7jAGbbrUV66h8iAC3XdCBH53fDtOEX76yjg37S62OJNIgptfLQ6+MY5HdiWGaDPGk8VBt1HHXZHtjyEkfjTepq0UpRXy0yZjIUQ+N68W+I1Us3n6YHz+3irfvupCMltFWxxI5I0+8dStzPPsBuKxmGH/96ZOAt36HU2JSoP1QsjXiIQFAIx8iR4XZbcz84QB6psVRVF7DxFkrKa2ssTqWyGm99P5knqnYAMDI8u786Y4Z2GyGb2olczj0vs73o4qHBAiVD5FviYlwMOuOwaTFR/L14Qp+9uIa3HW6BVcC14dLpvL3Qx8DcL4rlUcmvqyt0yXgqXyIfEdKXCSz7hhMbISDlbuPcO/rG/B6dQuuBJ4V657ldztfxmsYDCiL5YEb36Jli3CrY4mclsqHyEl0T40j99aBOGwG73yZz5/e+0p7gEhA2bbjPSavf4wam0FWRTh3X/EWmcmxVscSOSMqHyLf48LOSfWH0M36Yg/TP91pcSIRn7z9y/j5kvsot9noUmXj1qGzGZyZanUskTOm8iFyCuP7p9fvAfLYgu28uHyvxYmkuSsu2s7PPrqTYruNDLfJdb2e4gf9ulkdS6RBVD5ETuOOCzP55cWdAfjD3E2882W+xYmkuaooL+Rnc69nv8MgudbLVe3+wS3Dz7c6lkiDqXyInIFfXdq1fhOyKa+tZ9H2w1ZHkmamxl3GXa9eyXaHl3iPlzEJD5B9+eVWxxI5KyofImfAMAz+eFUWV/ZJo9Zj8rMXVrP06yKrY0kz4amrYcrsy1nrcBPl9XKx46fcd+0tVscSOWsqHyJnyG4zePyGflzcPRl3nZefPLealbuPWB1LQo3XA7uXwMY3YPcSzLpafj97HItsLhymyUWeCTz0o8kYhmF1UpGzZpgBdv+gy+UiPj4ep9NJXFzc6T9BxM+qaz3c+cJqluwookW4nRd/OoQB7RKtjiVBbub6mdiKtpO94SNwfbOu6MmWScyMjwbT5HL3CB796XTC7Pp3owSehrx/62+wSANFhtn5z22DGNqpFRU1Hm5/dqUOopNzZivazowDn5BrK69/7vXYFr7iAfSqieUvP1HxkNCgv8UiZyEyzM4ztw/ivA4tKauu49b/rmTjfqfVsSRYeT1kb/iInJJSZiQmkJsQx8fRUfypVUsABlZV80q5k3BbQA1Ui5w1lQ+RsxQd7uDZOwYzoF0CzqpabnlmOWv2llgdS4LR3qXgyie71FVfQH6VnIRpGPSsdjOr8BCG64DvOpEQoPIhcg5iIhw8/+PzvjUCsoJlXxdbHUuCTfnB+p+2r60D0wTDwDBNXi44iHGS60SCmcqHyDmKjQzjuR8PZljnJCprPEyctVL7gEjDxKRgArPiY7kvOam+eJiGwTMJccddJxIKVD5EGkF0uINnbh9Ufxvunc+vZsFX+leqnBlP+iD+ktSax1v67prqU13Nuj1531oDEg9x6dB+qMVJRRqHyodII4kMs5P7o4GMzUqlxuMl+6U1vLY6z+pYEuCqKo+Q/eIoXo2NAuDCyipeLjiEHY6uAXEyIzGe3D5jwGa3NqxII1H5EGlE4Q4b/765P9cOaIvHa3LfGxuYuXAnAbadjgSI4uId3Db7IpY7KrGZJgNrW/FkVdhx12R7Y8hJH403qatFKUUan8PqACKhxmG38Y/r+9A6NoLcRV/ztw+3ccjl5g9X9sRm066U4rNrzxLu+uQXHAgziPN4uS7ux0y+dgqG6fXd1VJ+0LfGo/1QsjXiISFG5UOkCRiGwf+N7U7r2Aj+9O5XPLd0D0Xlbh67oS8RDr2RNHervnyZKWsepdRhI6XWy63tHuL2MTf4PmjYIXO4tQFFmpjKh0gT+smwTFrHRvDr19bz7oYCDrnc5N46kJYtwq2OJhaZ99lfeGTPK7jtNjLdcGefpxh3/jCrY4n4ldZ8iDSxq/q2YdbE84iNcLByzxHGz/iCnYfKrI4lTe07B8Th9fDk2z/jwb2v4LYZ9KoM4/6Rb6p4SLOkg+VE/GTHwTJ+/Pwq8o5UERvpYOYPBzC8S2urY0kjO9kBcV7gsVZJvBDnO6flvPIEHrxuLpkpLS1MKtK4dLCcSADqkhLLnLsuZHCHRMqq65g4axUvLtujO2FCzHcPiHMb8NvWreqLR3d3FI/f8bGKhzRrWvMh4ketYiJ46adDuP+tjby19gC/n7uZDfud/Gl8FpFhWoga9I4eEIetnBmJCbgNg3UREayJigTgkvIKHnd7MCL1rVeaN418iPhZhMPOY9f35f/GdsdmwOtr9nNd7lLyjlRaHU3O1bcOiPuh08UzCfH1xePqsnL+ebhYB8SJoPIhYgnDMMge2YkXfzKEli3C2XTAxbjpn7Nw2yGro8m5KD+ICbwe24K3YmPqn3aYJn8uOnLcdSLNmcqHiIUu7JzEO5OG0bdtPKWVtdzx3Coen7+NOo/X6mhyFgrr3PwyOYlHklpRZfN9ew0zTeoMg1wdECdST+VDxGLpCVG8ln0BN5/XDtOEJz7dyY1PL9c0TJCZ8+lfuWHdoyxsEY3t6CLiu0pKWasD4kROoPIhEgAiHHamXtObJ27uT2yEgzV7S/jBE0t458t8q6PJaVRWFvObWZfw+7yXKLHbSKjz4jUMckqc/KLUBeiAOJHv0pJrkQByVd829M9I4J7/rWPtvlImvbKORdsP84dxPYmLDDv9byBNx+s54cyVlRvf4uFVfyQvzHdmz7DKFNr1GUNiVT7ZxR8BzvpPz/bGgA6IEwG0yZhIQKrzeHnikx1M/2wnXhPS4iN59JreXNQt2epozc7JNg3zAM8kJjEzPgqvYdCqzstVsbdwz3UPYD92eOBJyopGPCSUNeT9WyMfIgHIYbcxZUw3hndtzW9e/5I9xZXcMWsV1w1sy++v6El8tEZB/OXYpmHYyskGDjjsPNC6FWsjfbfQtq0xeGjY/zi/R+/vfKIOiBP5Phr5EAlwVTUe/jF/G89+sRvThOTYCB65OovLeqVgGIbV8UKb1wPTssg9umnY6IpKlkdFUnH0TpbRFZU8Xh2BbfJGjWpIs6ft1UVCSFS4nd9f2ZPXf34BHZNacKjMTfZLa7jjuVXsKaqwOl5oO7pp2LDKatJqa/mkRXR98bjV6WLaoSJs2jRMpMFUPkSCxKAOLXn/nuHkXNSJMLvBwm2HGfPPxTw+fxvVtR6r4wWvk5w+e8yevcu4v3Urbk5PpSAsDI4OFIeZJvcdKf3m99CmYSINomkXkSC063A5D83bzJIdRQC0TYziN5d1Y1yfNthsmoo5EydbSApAXBue6DGKJYc2s9tbjPvoSEc3dw3bIsIJM01qDYOcklKyj95Ky+3van2HNHuadhEJcR1bx/DCj8/jyR8OIC0+kv0lVdzzv/VcNeNzPj9aSOTUvnv6LPjuYvllZA3/OfQ5WynBbbPRv9rN9a4ytkWEk6NNw0Qahe52EQlShmEwtncaI7u15tnPd5O7aBebDrj40X9XMLxLEvdd1p3ebeOtjhkYvnvba8aQ406fBehX7eb/Wrei2OH7tphRW8foFuOITK0mt/Azckqc9SMdvh8NZiTGQ/posrXYVKRBNO0iEiKKy91M/2wnLy3fS63H97/1yK6tufvizgzu0NLidNb43qmV6CRyw2vwYLDfYefdbx0CF+718ssSJze7ygi//V1mOjd+z/RMOrl9xuBN6spd/e7y459KJDA15P1b5UMkxOwrrmTax9uZ+2U+Hq/vf+/zMlty90WdGd4lqVndnpv78WRmHPjk+PUZwPSEOJ5KTCCxro4SxzcDwDbTZOG+AyR6jx7sd+1/ofd1vp9r0zCRU1L5EBH2FVfy5KKveWNNXv1ISJfkGG67oD0TBrQlJiIEZ12/XRBatIa3f06uvYIZiQnklJRyk6uce5NbsSIqqv5Twrxeam02LSQVOUcqHyJSr8BZxdOLd/Hqqjwqa3y3kcZEOLhuYFtuGdKOrimxFic8d987vQLkJsTxWXQUX0VE+G6VPTryk1JXR6eaWpZGR9UXjtyEuKNFxek7i0Wbh4mcMW2vLiL10uKjeGhcL351aVfeWrOfF5btZVdRBc8t3cNzS/eQlR7H+H7pXNWvDcmxkVbHPSvf3QIdIM9h5+Gklqz81igHhoFhmjx6uJh9YQ6ePDoiooWkIv6lkQ+RZsbrNfni6yJeWLaXz7Yeou7ouhCbAcO6tOayXimM7p5CanyQFJHvbIE+sqISl93Gushv8ju8XupsNhymSd3RqRUvBjZMsmvCoLL4m99PC0lFzkpATLvMnDmTv//97xQUFNCrVy+mTZvG8OGnnztV+RDxnyMVNby3IZ+31h1g3b7S4z6WlR7H6O4pjOrWmqz0eMLsgbctUE2dl92rPqTbRzcDcGdKa5ZHfzPSkVFbS9ca37bo3zu18sv1kLdCC0lFzpHl5ePVV1/l1ltvZebMmVx44YU89dRTPPPMM3z11Ve0a9fulJ+r8iFijT1FFby3sYCPtxxkfV4p3/7OEB1uZ0C7RM7LbMngDi3plR5HXKT/T9Z1Vtay8YCTlXuOsGr3EdbllTDG8zlPhE8HfCfOjm3bBtMwCDNNflbqrF9s+u27XXIT4pmRGE9O+miyL5nm9z+HSCiyvHwMGTKEAQMG8OSTT9Y/16NHD8aPH8/UqVNP+bkqHyLWKyp38+nWQ3yy5SDLdx3BWVV7wjVtE6PonhpHj7RYOifH0DYxivSEaFrHRmA/hy3ePV6Tg65qDpRWcaCkiu0Hy9haWMbWAhf5zuoTrr80ejv/8T4MUD+qcezOlcFV1ZxXXX1c8QA0tSLSBCwtHzU1NURHR/P6668zYcKE+ufvuece1q9fz6JFi075+SofIoHF6zXZcaiclbuLWbH7CGv3lpy0BBwTZjdIjY8kMTqcuMgwYiMdxEWGERVu59tbjJgmVNbUUVZdh6u6FldVHUcqaih0VdfvT3IybROjGNQ+kfMyW3FeZiKdWkVh/Ks3ubYK32jG902vXD0TKos0tSLSRCy926WoqAiPx0NKSspxz6ekpFBYWHjC9W63G7fbXf9rl8t1wjUiYh2bzaBbaizdUmO59YIOgG/6Y2uhiy0FLrYUlLG7uIIDJVUUuqqp9ZjkHaki70jVWX9Nh80gLSGS9IQoMpNi6JkWS/e0OLqlxp50uie3z2VHNxM7xRbonUaddR4RaVxNdqvtd3dRNE3zpDsrTp06lT/+8Y9NFUNEmkB8dBhDOrZiSMdWxz1f5/FysMxNQWkVzqpaXNW1vpGNqlqqaj0n/D7R4Q7iIh3EHh0hSYgOo01CFMmxkQ2auvEmdSUHyC7+CHDWP5/tjYH00XiTup7tH1VEmoDl0y4nG/nIyMjQtIuINJy2QBexjKXTLuHh4QwcOJAFCxYcVz4WLFjA1VdffcL1ERERRERENHYMEWmObHZthy4SBJpk2mXKlCnceuutDBo0iAsuuICnn36affv2kZ2dffpPFhERkZDWJOXjxhtvpLi4mEceeYSCggKysrJ4//33ad++fVN8OREREQki2l5dREREzllD3r8Db79kERERCWkqHyIiIuJXKh8iIiLiVyofIiIi4lcqHyIiIuJXKh8iIiLiVyofIiIi4lcqHyIiIuJXKh8iIiLiV02yvfq5OLbhqsvlsjiJiIiInKlj79tnsnF6wJWPsrIyADIyMixOIiIiIg1VVlZGfHz8Ka8JuLNdvF4v+fn5xMbGYhiG1XEs53K5yMjIIC8vT2fdNCG9zv6h19l/9Fr7h17nb5imSVlZGW3atMFmO/WqjoAb+bDZbLRt29bqGAEnLi6u2f/F9ge9zv6h19l/9Fr7h15nn9ONeByjBaciIiLiVyofIiIi4lcqHwEuIiKChx56iIiICKujhDS9zv6h19l/9Fr7h17nsxNwC05FREQktGnkQ0RERPxK5UNERET8SuVDRERE/ErlQ0RERPxK5SMIud1u+vXrh2EYrF+/3uo4IWfPnj385Cc/ITMzk6ioKDp16sRDDz1ETU2N1dGC3syZM8nMzCQyMpKBAweyZMkSqyOFlKlTpzJ48GBiY2NJTk5m/PjxbNu2zepYIW/q1KkYhsHkyZOtjhI0VD6C0H333UebNm2sjhGytm7ditfr5amnnmLz5s3885//JDc3lwceeMDqaEHt1VdfZfLkyfzud79j3bp1DB8+nLFjx7Jv3z6ro4WMRYsWkZOTw/Lly1mwYAF1dXWMGTOGiooKq6OFrFWrVvH000/Tp08fq6MEFd1qG2Q++OADpkyZwptvvkmvXr1Yt24d/fr1szpWyPv73//Ok08+ya5du6yOErSGDBnCgAEDePLJJ+uf69GjB+PHj2fq1KkWJgtdhw8fJjk5mUWLFjFixAir44Sc8vJyBgwYwMyZM/nzn/9Mv379mDZtmtWxgoJGPoLIwYMHufPOO3nxxReJjo62Ok6z4nQ6admypdUxglZNTQ1r1qxhzJgxxz0/ZswYli5dalGq0Od0OgH0d7eJ5OTkcMUVV3DJJZdYHSXoBNzBcnJypmkyceJEsrOzGTRoEHv27LE6UrPx9ddf8+9//5vHHnvM6ihBq6ioCI/HQ0pKynHPp6SkUFhYaFGq0GaaJlOmTGHYsGFkZWVZHSfk/O9//2Pt2rWsWrXK6ihBSSMfFnv44YcxDOOUj9WrV/Pvf/8bl8vF/fffb3XkoHWmr/W35efnc/nll3P99dfz05/+1KLkocMwjON+bZrmCc9J47j77rvZsGEDr7zyitVRQk5eXh733HMPL730EpGRkVbHCUpa82GxoqIiioqKTnlNhw4duOmmm3jnnXeO+0bt8Xiw2+388Ic/5Pnnn2/qqEHvTF/rY99M8vPzueiiixgyZAjPPfccNpu6+tmqqakhOjqa119/nQkTJtQ/f88997B+/XoWLVpkYbrQM2nSJObMmcPixYvJzMy0Ok7ImTNnDhMmTMBut9c/5/F4MAwDm82G2+0+7mNyIpWPILFv3z5cLlf9r/Pz87nssst44403GDJkCG3btrUwXeg5cOAAF110EQMHDuSll17SN5JGMGTIEAYOHMjMmTPrn+vZsydXX321Fpw2EtM0mTRpEm+//TYLFy6kS5cuVkcKSWVlZezdu/e45+644w66d+/Ob3/7W01znQGt+QgS7dq1O+7XMTExAHTq1EnFo5Hl5+czatQo2rVrxz/+8Q8OHz5c/7HU1FQLkwW3KVOmcOuttzJo0CAuuOACnn76afbt20d2drbV0UJGTk4Os2fPZu7cucTGxtavp4mPjycqKsridKEjNjb2hILRokULWrVqpeJxhlQ+RL5j/vz57Ny5k507d55Q7DRQePZuvPFGiouLeeSRRygoKCArK4v333+f9u3bWx0tZBy7jXnUqFHHPT9r1iwmTpzo/0Ai30PTLiIiIuJXWkEnIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+9f8Bor1Qczd146QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_39_2.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def function(x):\n",
    "    return x**2\n",
    "\n",
    "def analytical_gradient(x):\n",
    "    return 2*x\n",
    "\n",
    "def gradient_descent(starting_point, learning_rate, num_iterations, solver=\"analytical\"):\n",
    "    x = starting_point\n",
    "    trajectory_x = [x]\n",
    "    trajectory_y = [function(x)]\n",
    "\n",
    "    if solver == \"analytical\":\n",
    "        grad = analytical_gradient    \n",
    "    elif solver == \"jax\":\n",
    "        grad = jax_grad(function)\n",
    "        x = jnp.float64(x)\n",
    "        learning_rate = jnp.float64(learning_rate)\n",
    "\n",
    "    for _ in range(num_iterations):\n",
    "        \n",
    "        x = x - learning_rate * grad(x)\n",
    "        trajectory_x.append(x)\n",
    "        trajectory_y.append(function(x))\n",
    "\n",
    "    return trajectory_x, trajectory_y\n",
    "\n",
    "x = np.linspace(-5, 5, 100)\n",
    "plt.plot(x, function(x), label=\"f(x)\")\n",
    "\n",
    "descent_x, descent_y = gradient_descent(5, 0.1, 10, solver=\"analytical\")\n",
    "jax_descend_x, jax_descend_y = gradient_descent(5, 0.1, 10, solver=\"jax\")\n",
    "\n",
    "plt.plot(descent_x, descent_y, label=\"Gradient descent\", marker=\"o\")\n",
    "plt.plot(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a994430",
   "metadata": {
    "editable": true
   },
   "source": [
    "### A more advanced example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "863db98d",
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/jax/_src/numpy/lax_numpy.py:173: UserWarning: Explicitly requested dtype float64 requested in asarray is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more.\n",
      "  return asarray(x, dtype=self.dtype)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x10febcf10>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGdCAYAAAAvwBgXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAA9hAAAPYQGoP6dpAAB8SklEQVR4nO29eZhcZZn3/z21dFUvVb2mO0t3VhAStkgCCQjIJsuIijqMDMgMDvq+/AQGZGZwAiqo+ObngAsygjLziigKuKKOiGFU0BEiISxhSyBJJ+mkt/Ra3dVd+3n/qHrOOd2pqj7L85zzVNf9ua5cF+l0dz0k1efc53t/7++tqKqqgiAIgiAIwgN8Xh+AIAiCIIjqhQoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8gwoRgiAIgiA8I+D1AcqRy+XQ29uLSCQCRVG8Pg5BEARBECZQVRUTExNYvHgxfL7ymofUhUhvby+6urq8PgZBEARBEDbo6elBZ2dn2c+RuhCJRCIA8v8j0WjU49MQBEEQBGGGWCyGrq4u7T5eDqkLEdaOiUajVIgQBEEQRIVhxlZBZlWCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCIDyDChGCIAiCkIQ9hyfxrWf2IJHOen0U15B6+y5BEARBVBNf3fIWfv1qH9oaQvjrdZ1eH8cVSBEpQzqbQyab8/oYBEEQRJVweCIJAOgZmfL4JO5BhUgJ4skMTtv8e3zkga1IZagYIQiCIMQTS6QBAIOFgqQaoEKkBPuG4xiaTGL7/lF88w+7vT4OQRAEUQVMJDIAgMFYwuOTuAcVIiVIGlSQb/5hN97si3l4GoIgCKIamCBFhGAk03ohksmpuOUnO8gvQhAEQQhDVVVMJvOKyAApIkQikx+dWtJUi2g4gFcPjeM//6fb41MRBEEQ85V4Koucmv/vockksuw38xwqRErAFJGFjWF85pI1AICvPvUW9hye9PJYBEEQxDwlNp3W/junAsOT1dGeoUKkBMmCIhIK+HDZuk6ceXQbUpkc/vWnO5CrkiqVIAiCcA9mVGUMxKgQqWqYWTUU8EFRFGz+0Amor/Fj275R/NerfR6fjiAIgphvMKMqY3CiOnwirhUimzdvhqIouOmmm9x6SUckC/G64aAfANDZXIdLTlwMAOg+HPfsXARBEMT8hBQRgWzbtg0PPPAATjzxRDdejgtGRYRRF8oXJczIShAEQRC8iM1SRKplckZ4ITI5OYkrr7wS//Ef/4Hm5mbRL8cNvRDxax9j6kg1LSMiCIIg3GG2IlItWSLCC5HrrrsO733ve3H++eeLfimusGIjFNT/isKFoiRJke8EQRAEZ1ghoij531dLuqrQ7buPPvooXnzxRWzbts3U5yeTSSSTegUYi3mXZsqKDaaCAHpRQooIQRAEwRtmVu1qrsOBkSlSRJzS09ODG2+8EQ8//DDC4bCpr9m8eTMaGxu1X11dXaKONyfMrGr0iIQL/02KCEEQBMEbpoisWlAPgDwijtm+fTsGBwexbt06BAIBBAIBPPPMM/jGN76BQCCAbPZIVWHTpk0YHx/XfvX09Ig63pwk0keaVUMFdSRJighBEATBGaaIHNXeAKB60lWFtWbOO+88vPrqqzM+9rGPfQzHHnssPv3pT8Pv9x/xNaFQCKFQSNSRLMECzYytmbDWmiFFhCAIguALU0SWt9XDp+jpqu1Rc12FSkVYIRKJRHD88cfP+Fh9fT1aW1uP+LiMFBvf1c2qpIgQBEEQfGGFSFNtDdoaQhicSGIgNv8LEUpWLUGx8d0QKSIEQRCEIFiOSCQcQHs03x2ohnRVoVMzs3n66afdfDlHlB/fJUWEIAiC4AtTRCLhADoiYbyGmNB01b7xabzRG8OCSAgndjYJe525IEWkBKSIEARBEG4yoSkiQa0dI3JyZuveYVzz0Au467e7hL2GGagQKYG2fdegiLCihHJECIIgCJ6oqorJZF4RiYYDaI+w1ow4RSRVeOCu8XtbClAhUoJi47tsgsarHJHx6TRe7x335LUJgiAIccRTWbBJ3Ug4iI6CIiIyXVUrRALelgKuekQqiWLju6wocVsRSaSzePDP+3D/07sRS2Tww49vwOlHtbl6BoIgCEIcrC0T8CkIB32uKCJJKkTkJjmHIqKqKhS2EEAQmWwOP95+EF//77dmGJb2DU/h9KOEvjRBEAThIkajqqIomiIi0iOSyubvc0GPWzNUiJSg+PZd34w/N6olvBmNp/CRB57DWwOTAIAlTbVoCAWwa2ACk8n0HF9NEARBVBJGoyoAdBTGd1m6qt/H/8E3ncn3grxWRMgjUgLWfgkXMasCumIiihcPjOKtgUnU1/jx2UvW4Pf//G6cuqIFADA5a1U0QRAEUdnEDIoIALQ2hGakq4ogVVi1QmZVCVFVtagiEvQrYEWp6CwR5p4+qasJ15yxAqGAHw2FN+hEkgoRgiCI+cTErELE71PQ1pBXRURliaSKJIh7ARUiRWB9M2Dm+K6iKIYRXrGKyFQqX+jU1ejds4ZQ/r9JESEIgphfzG7NABCerirL1AwVIkUwFhmzK0Vt8Z1gRSReUD3qQ7oiwyrlSVJECIIg5hWzFREA6Igww6qo1owcZlUqRIrA2i6KcmTvTJuc8VIRoUKEIAhiXsEUkegMRUTs5EyKzKryYhzdnT2iq2WJiFZEUgVFpEZXRFghMkGtGYIgiHlFbPpIRUR0lghTRMisKiHFjKoM1xSRZEERCRkUEWrNEARBzEuKKSKi01VThQdqUkQkpNjoLsOtdNViikgklH+DklmVIAiCD8OTSSke7op5RIQrImRWlZdyikiooIiIbs2QIkIQBCGWiUQaZ9/1NM7/yjM4NDbt8VlYIXKkIiLMI0KtGXnRNu8WqRLdas2U84hMJjPIse1IBEEQhC0OjExhIplBfyyBf3hwG2IJ71KrY9r4rmFqZla6Km8oWVViWJFRLMLdLbNqsakZ4xuUFSoEQRCEPcam9MJj18AEPvnwi0hnvdmuXqw1IzpdNUmKiLyYUUREB5oVyxEJBXwIFKJdqT1DEAThjJF4CgDQ2VyLuho//mf3ED7z89egqu4rzsUCzUSnq5JHRGI0j0gRs2q48A8mOuK9mCKiKIruEyHDKkEQhCNGp/KFyAlLGnHv374TPgV47IUe3Pf0HlfPoaqq9nAZDc/cRatNzghIV6WpGYlhEzHFzapsakZ0oNmRighgyBIhRYQgCMIRo/G8CtFcX4PzVnfg8+8/DgBw12934c+7h1w7RzyVBbOAGBURQJ+cEaGIpLP5F6VkVQlhikix8d1woTgRrYjEC1Mz9TUzq2PaN0MQBMEHpog01+Vv/ledthzvO2kxAOCPbx927RysLRPwKUfcd0Smq9LSO4nRk1VLKyIip2ayORXTadaamXkG2jdDEATBB+YRaa6r0T521IIGAO4mWBuNqrPTvEVmiWjju1SIyEdZs6q2fVecIjJt+N71IVJECIIgRMAUkZZ6vRBhD3uxafdGeYsZVRki01U1syq1ZuQjUWZ8V8sRyYhTRKYKaodPObIYaii8UckjQhAE4Qy9NXNkIeKmIhIrMrrLYFkiQhSRwn0sSIqIfJRTRHSzqjhFJJ7S/SGzZTpSRAiCIPhgNKsyorWFhz0Xw82KZYgw2iN5RaSfsyKiqiolq8pMsoyBRzerilNEWIZIXehIRUb3iHiXAMg4PJHEuV95Gvf899teH4UgCMIys82qgKE146pHpHRrpqlOTGHEJmYA8ohIiTa+WyxZ1QVFZCpVfGIGmBnz7jXPvHUYew/H8X//Z6+Q+GGCIAhRJNJZ7Vo7QxEJy6WIaAMSmRzXoLWUIUGWpmYkpJwiEnLBrMri24spIlqOiAStmX1DcQD5J4c3emMen4YgCMI8LN494FMQMQwF6IWI+4pItIgiwu45qjpTxXBKyqDqU46IhGjju0XNqnp1Kgpt824xRUSi8d3u4bj238/ucS/8hyAIwilsdLeprmaGF4+pElOprGt7Z8oqIoYHYp75VawQ8fsU+H3KHJ8tFipEilDWrOqiIlJfU8QjIpFZdf+MQmTYw5MQBEFYQx/dnalCNBiKAbeus+YLEX6FUVoSoypAhUhRyo/vio94n9LMqvIqIqqqYt/QlPb757tHZkh9BEEQMsMKkSbD6C6Qb1PUFq79brVnyplVFUXRzKQ8C5GkJAvvACpEimJm+67QqRnNrCqvR2RoMoXJZAaKknd1T6ezeOXgmKdnIgiCMMtooTXTMqsQAYBoLZucccewWi5HBNDvRUmOSrwsm3cBKkSKUt6syv8NMRu28K6YR0SWiHfWllncWIt3HdUGAHh2N7VnCIKoDEaKZIgwmDLhViGit2aOVEQA3RLA8wFYlgwRgAqRojD/h1fJqtrCu6JTM/k36mQyw3WUyyrdhYmZFW31OH1VKwAyrBIEUTkUyxBhuJ2uqrdm5lBEeBYipIjIjRlFJJXNCcvOKKeIMI9INqcK9anMxb6CIrKstQ7vWpVXRF46MIbplNitxARBEDwotmeGwcZo3do3wwqeaKlCJMhfiSezquTohUhpRST/eWJuuuU8InVBP9ik2YSH6ar7hvNG1RVt9VjWWofFjWGksjls3z/q2ZkIgiDMMlrIEWku4hFxUxFRVbWsWRUQ1JohRURuklqyamlFJP95YhSJclMzPp+ChhrvR3hZmNny1nooioLTCqoItWcIgqgEmFm1ub5Ya8a9ULN4KgsmrrvZmqGpGclJZEqP7wb8PgQK4S8J4YpI8Tel1yO8+dHdQiHSVgcABp8IGVYJgpAfFmhWTBFxc2qGqSF+n6KNDc9GL0Q4Ts1Qa0ZeVFXVJKtS+fuaYVWUIlIm4h3wfgPv4ckk4qksfArQ1ZIvRE4rFCI7Do655jQnCIKwy9hUmULExX0zxjCz2dvWGSEB9xx2nwuSIiIfRumrdCFSCDUTpIiwiPe5FJEJjxSR/QV/yOKmWq13ubipFiva6pFTgef3jnhyLoIgCDMk0llNeS4+vuueR2SuiRlATGuGzKoSY6w4i7VmAGPMuxhFRFt6V8SsCniviBhHd42cRu0ZgiAqALbwzu9Tik6qRF3MEdHCzELFjaqAoNbMHMq/m3h/Aslg/9A+BZoXZDYiRqmMaIpIEbMq4H2oGfOHLGutm/FxyhMhCKISMGaIFGuHuKuIlE9VBWhqpuowju6W7NcxRURAqJmqqmWX3gEGRcTj1szy1pmKyMaV+UJkZ/8EhieTrp+LIAjCDKNljKqAu1MzrDUTrS2jiGgPv5SsWhUkyozuMvTFd/wVkWQmp41yFRvfBfR0Va/2zZRqzbQ1hHDswggAYNs+yhMhCEJORsoYVQHD1IwLgWbmFBH+rZmkZlYt/sDtJlSIzIL944SLhJkxwgJkMkbcoHKUGuXSx3fdn05RVdWQqlp/xJ+z4mQglnD1XARBEGbRwsyKZIgAHikiJcLMADGtGd2sWvpe5xZUiMxC27xbRhEJCVREpgpO7tqgH/4SHpWIh2bVwxNJTBVGd5e21B3x5ywuebggfRIEQciGtnm3yMQMoKsTqWxOyHXeiFeKCHlEJIb14Mo5iYUqIswfUiJDBPA20IxFuy9pri36Bm5tCAEAeUQIgpAWZlZtKtGaaagJaKs0RE/OmCpERHhEqBCRF5YNUmp0FxA7NcM27xZbeMdgZlUvPCLGaPditDUUFJFJUkQIgpATTREpUYj4fIpr19m59swAYqdmZBjfLX23q1KsKCJiWjPlM0QAbxWR7uHyhUhrfUERiZMiQhDVSiqTw7Z9I/jz7iGc1NWEC49b6PWRZjBS8Ig01ZW++UfDQUwkMsILkZhHrRnmEQn6vTerUiEyi3KbdxlsakaMWbV8hgige0TiHhQi+1kh0laiECFFhCCqkqlUBr94uRd/2DmIP+8e0pJLw0EfXrvjQgQkGBNlsHj3Uh4RQC8MRE/O6K0ZM4FmHJfeSTS+S4XILJjKES5rVq1iRWQo7xFZ0XakURUAWsmsShBVyRd+9QYe3daj/b6tIYTx6RQS6RwOjExh5YIGD083E23hXZlCJOrS5IypiHeBu2Zqyjx0u4X3pZBkmFJEBFSnjLk27wLeeURUVdUUkWKju4BuVh2fTmtvdIIg5j/b9+ezgz66cSl+df0ZeP7W8/COjnyu0O7BSS+PdgRzBZoBxnRVdxSRYlHzDJqaqTK08d0y/zhCFZFk+c27gL6TIJnJuXqzHzSM7nY1F1dEmmqDYFPHzJlOEMT8JpnJYm/ByH7dOUfhhM5G+HwKjmrPqyC7D8tTiCQz+sK7UmZVQE86FTk1o6qqpmy73ZrRtu9K4BGhQmQWmlm1XGsmwHJEvFFEjKO9bvpE2MRMZ3NdySra51PQUjCsDtEIL0FUBXsG48jmVETDASyMhrWPH1Vox8ikiLCFdz6lfDvEjX0zU6kssoUobbd3zTCzqgxTM96fQDISmiJSzqzK3hTeKCIBv09LXXXTJ7JvDqMqg0Z4CaK62NkfAwAcuzA6Y0cXU0T2SFSIjBjaMr4SoZGAO4UI+95+n1IySRswREbwbM1kqTUjLZWgiAC6YdVNnwgzqi5vLd6WYbDJmREyrBJEVbCrfwIAcOyiyIyPa4XI4ThUVXX9XMXQNu+WMaoCullV5NSM0ahaaskqYGjNiDCrUsS7fJgb3xWoiJiYmgEMMe8uKiL758gQYVBrhiCqi52FQuSYhTMLkWWt9fD7FEwmM+iXZP/UaLywZ6ZMhgigezZiAh/2zGSIAGIDzUgRkRAz47thzazqTY4I4M3iOxbvvrzE6C6DRngJorrQFJFZhUhNwIdlBQV1z2Dc9XMVY3SOzbsMN6Zm2IPnXAq40O27ZFaVDzOKiN6a8U4R8WKEl428tUfCZT9P94iQIkIQ852xqZSmdrBxXSO6YXXC1XOVYq6Fdwx9akbcNVa3ApS/3ocMIZq8Wlxp8ojIi5nxXaaIiBidNe0R8aA1EzMRvAMYF9+RIkIQ8x3Wlulsri06girbCO/IHAvvGG4oItpuszmKAfZgrKpAOsunEEnR1Iy8JE0sAmJtG69yRABDa8YlRSSdzWGqUCRFy8y7A9SaIYhqolRbhrFKshFeNr7bUl/+OhZ1YSAgYVYRMdyPeLVnyKwqMbpHpFxrpuAREaCITJlURNw2qxoLnrkVEVaIUGuGIOY7pYyqDE0RkcQjwqb55lJE9Ij3tLCJn6RpRcRYiPC575BZVWKsKCJJAYpInJmXTCoibnlEWFumrsY/5/IqbQMvtWYIYt7DMkSOWRgt+uerCoXI0GQS41PumetLwcyq5VJVAX1qJqfqLXPemFVEFEXRCgYehUgupyJTCFIjs6qEmDEPCVVEClMzdXN6RPI/JG4pIrFptg+hvJwJ6IrIVCqrmW8Jgph/5HIq3iooIqtLKCINoQAWNeYN7rsPe29YNZsjEg76ECgEnonyiWgKvAlVQs8ScV4UMX8IQIqIlJgxDzFFJJtTkcnynetmbxCzgWZueUTYD2K0du6FzQ2hgPbmJlWEIOYvh8amEU9lUeP3lU1c1tsz3vtEzOaIKIqiT85Mi7nOMnWjnBWAwTNLhAoRyTGjiBjfNDxVkWmD/FcrWaCZPjEztyKiKAra6ildlSDmO8wfsqq9AcEyLVtZDKupTE67Zs41vguIn5xh6oaZyRWei++ME581c7Ta3UDoCTZv3oxTTjkFkUgE7e3tuPTSS7Fr1y6RL+kYMx4R4z8cz8kZ5g+p8fvmrFK1HBHXCpG5V1UbaSHDKkHMe3b2sR0zxdsyDFkUkbFCW8anmGszi943Y2Y4ghHi6E3UJ2Z8ZaPl3UJoIfLMM8/guuuuw9atW/HUU08hk8ngggsuQDwuh3u6GJqLucwbw+fjaxxiaGFmcxhVAWNrxh3zF9u3YEYRAXTD6hC1Zghi3rJzoPzoLkOWLBFjhki5hXcMbd+MKEVEa82YUUQ4tmYkSlUFAHOPtzZ58sknZ/z+wQcfRHt7O7Zv346zzjpL5EvbRmvNmBinSmVyfBWRpLnRXcD9QDNNETHhEQEMI7xUiBDEvGXXHKO7DFaIHBydRiKdNaUAiMCsP4TBFBFR6aoJrTVjxiPC7+FXplRVwGWPyPj4OACgpaXFzZc1TS6nmk6b0xbfcdw3EzcZ7w7oPyCum1VNKiJtWroqtWYIYj6SSGfRPZRXt48tMbrLaK2vQVNdEKoK7D3snSJuds8MI2LIEhEBG981p4jw2zeTlChDBHCxEFFVFTfffDPOOOMMHH/88UU/J5lMIhaLzfjlJkYn8Vxz3Vq6KsclRNro7hwL7wBdEYmnssjmxK/XZq5x860ZMqsSxHxm9+AksjkVjbVBdERDZT9XURR954yH7Rl2PZprdJehKSLCpmYKiogpjwi/h99UtSoi119/PXbs2IFHHnmk5Ods3rwZjY2N2q+uri63jgdgpvHUbPa/CLNqvQlFpMFgGo27kNURszC+C+iO9KEKLUSyORVv9MaEJSoSRKVjjHY3Y3iUwbA6piki5h6ooi4pIl5NzcgwMQO4VIjccMMN+OUvf4k//OEP6OzsLPl5mzZtwvj4uParp6fHjeNpsH9gv0+ZMz00HBRhVjUXZgbkCyH2JnKjPTNhYXwXqPzWzM0/ehl/9Y0/4eYfveKK4kQQlcYuk0ZVBitE9nhYiIwwj4hFRUTY1IyJ4QgGz9YM84iUG7l2E6FmVVVVccMNN+DnP/85nn76aaxYsaLs54dCIYRC5SU+kZg1qgJAmDmYuZpVzcW7MxrCAYzEU64YVvVk1flvVt26dxi/eLkXAPDzlw5BAXDXZSfBb8JlTxDVwpt95aPdZ7NKAkXEbLw7Q/jUjOYRcTnQzERMhZsIPcV1112Hhx9+GD/84Q8RiUTQ39+P/v5+TE9Pi3xZ21iqTrUNvN4oIoAhS8QFRURvzZj0iDBFJJ6sqPZGNqfiC796AwCwblkz/D4FP3vpEP7lJ6SMEIQRsxMzDOYR6R6Kc02ktsJ4IYag0eR1jLWiRSsiplozWo4Ix9ZMNRQi999/P8bHx3H22Wdj0aJF2q/HHntM5MvaxpYiwtGsasUjArg7wjthMdCMmVXTWdW10DUe/OiFHrzRF0M0HMADV63DvX/7znwx8uIh3PKTHVSMEASA0XgKgxP5tqvZQmRJUy1qg36ksjn0jHrzMMra2A0mr2Oip2asKSL8WjOymVWFt2YqiaSN6pSrImJhagZwb9+MqqqWx3fDQT/qa/yIp7IYnkyZ/jovGZ9O4+7f5pN/bzr/HWhtCOGvTlgEVQX+8dGX8NMXD6Im4MPmD53g8UkJwlveLrRXOptrtQeiufD5FKxcUI/Xe2PYPTiJFWV204hiUmt/my1E3JmacTvQLJmRyyMixykkQY93n7s6lUER0ffNiE1XjaeyYEKA2dYMYGjPVIhh9d7fvY3heAqrFtTjqtOWaR9/74mLcM/la6EowCPPH0DPyJSHpyQI7+kdyysaXc11lr7O650z7Bprtnhyb2rGG7NqVU3NVAp67r8ZRYSN73qviIj2iLB496BfsWRuYobVSoh533N4Et99dh8A4LOXrDniSeGSExfjtJWtAIBfv9rn9vEIQioOFQqRRU1hS1/X2VwLABiIJbifyQxsIMBsIcIUkXgqK8TXYu2eQx6RqsCKIsJuyF7liADueURihraMlQVJbN9MJSy++9Kv30Qmp+LcY9tx9jHtRT/nvScuAgD8egcVIkR10zeeL0SWNNVa+jrdxO7Nw8mk5UJEV4B5X2cz2RwyBak5bOqew39qhgoRCdFT7kyYVYP83hQMy1MzLnlEmOISMWnwYrQVFJERyRWRXf0T+P3OQQR8Cm577+qSn3fRcQvhU4BXD41j/7C8ixsJQjS9Y3lFY1GjxUJES1x2/+Ekk81pCrZZj0hNwKepFbyVZ+O9w8w9h6tZtZrGdysNO/06vkvvrOWIRNxSRKatje4ytCwRydNVt+8fBQBsWNmi9bCL0doQwumr2gBQe4aobphHZLHF1oyX+UJsqShg/hoL6KoI7ywRYyFiThHhmKwqWaCZHKeQBBZOVjGKCMsRcbE1Y4WWQmtmSHKz6o6DYwCAEzub5vxcas8QhLEQsaaIsNUPXjycTBZa33mvm5VCRMzkDHuIrfH74DMRlihk1wwVIvKRtCBXaUvvOCoiUylrikhDoTCQvTUje7rqKwfzW6FP6myc83MvPG4h/D4Fr/fGsG+I2jNE9RFPZhArXBMWNVpTRNjqh9F4CjmXM3msGlUZoiZn2L3DbHtERGuGPCISwgoRK3G7PKdmmHRYb1ERca01Y1ERqQSz6nQqi7cKOzPMKCIt9TU4fRVNzxDVCzOqRsIB07unGM2FaPVMThUWm14KqxkiDFH7ZrQHXxP3G0DQ0jsqROTDSoWqL73jo4hkcyqm06w1Y9Ij4rJZ1ezmXQbrB49I7BF5o28c2ZyKtoaQ6ae7S6g9Q1QxhwpGVasTM0D+xseuW263Z7RUVZuKCO/CycroLkBTM1WDpUAzjv06AFoRApiv2N0e37X69GMsRGSNRn+lR2/LmB1NvmDNQgR8Ct7oi2HvYe8WeBGEF/SxDBGLbRmGvpnb3UIkblMREbVvJmFhpQhgyBHhGfFOHhH5sBTxzqZmOCkiU4UfEp9i/o3JvCTiWzPW9swwmAybU4GxKTlVEStGVUZzfQ3edVR+euYJas8QVYZdoyqjxaMRXvutGTEekaSFJauAoTXD4eE3LdmuGTlOIQkJCwuIeCsi8ZTuDzH7ZF5b8JJMczTMFsOuIhL0+9BUl/8aWUd4dxSMqid2zW1UNcKmZ/6L2jNEldE7nm/N2C1EWJaI24nLTBGJWC1EQqKmZszfbwBBrRlSROTDytQMb0WE/ZDUWZhvry28gVOZnNDWR0zziFhfXKdfdOQzrI5Pp7G3MPlykgVFBAAuXLMQQb+Cnf0Tnu3NIAgvsJshwvDKO6Y97Fm4xgL6dW+C804vKwq88fN4tGaS5BGRF1s5IpwUkamUtYkZYKapVaQqMqFNzVhf1tzqUT/YDK8dyqshnc21mlxslsa6IM4otGd+Q+0ZooroG7eXqspg03RuFyLM4yHN1IxVRSSoT8043WxPZlWJ0cZ3TZlVOSsiKeuKiLGSnk6JK0RiWo6IdUWkTeLJmVcK/hCragjj/DUdAIBn9wxzOhFByI2qqroiYrMQafFIJbWbI6Ilq05znprJ2JuaUVUgnXVYiFCyqrwkLCgieo4IL7OqtVRVAFAURWvPiC1EWMS7dUVES1KUsDWzozAxc6KJILNibFiRzxN58cAotzFugpCZkXgKyUwOigJ0NIZsfQ/PWjN2p2YEKSJ6XIQ1syrgvD1DZlWJsbR9l6NMBljfvMtg7RlRrZlEOqvJePY8IoWYdwkVETYxc1JXk62vX7WgHm0NISQzOc30ShDzGbbsrq0hZCkm3YgWdOhyu9bq5l2GvmtGVGvGmkcEcG5Y1ZbekSIiH9Yi3nWZjMlcTpjSzKrWfkjYOVg8PG/YU4CiAA0W1BqGHvMulyJyeCKJ3vEEFAU4fok9RURRFGxY0QIA2ErtGaIK6B13NroLeLdvhj3sWS9EClMzvAPNMtYUEUVRNAWDVyFCioiEJNPm57p5VqeAcXxXLkWE/fA1hAKmFjPNRlazKlNDjlrQYPnCZGTDynwh8pfuER7HIgip0f0h9iZmAP3hZHTK3X0zkzbNqkwJTmVyXFuwWqCZSUUEMGaJODsHFSISY0URqfH7wOI+ePhEmKJhxSMCALU1Yj0iWry7DaMqoI/vymZWZYvurASZFYP5RLbvH9X6rgQxX+lzmCEC5AMBgfxai3HOBtBy6IFm1h72jA8qPNdpaIFmFlpcvLJEyKwqMdpct4kKVVEUrkl32sI7iz8kmllVlCIyzcLM7KkGXq79LofuD7HXlmEc3d6A5rogptNZ8okQ855DDuPdgfzNr7HW/aBDdo21qoD6fXpLhOd11mqgGcBv8R0pIhKjvTFMVqhalggHuc6pIjIlSBHRJ2bsKSKNdfrCKLfXfpdCVVU9UdWhIuLzKTh1BWvPkE+EmN+wPTN2Ft4ZafVgms7u+C6gP/DxmpI0fi+zgWaAYUjCaWuGds3IixVFBNALlgQPRcSmR0TED4gRvTVjTxFhTz6qyn/8zS4HR6cxEk8h6FewelHE8fdj7Zm/7CWfCDG/YVMzixwWIi0ut2xVVcWkTbMqYPDipfi1X7XcKkuKCKfWjAUbghvIcQoJyOZULSTG9Fw3x22IdqdmRHtEYlqqqj1FJBTwaz/EY9NytGdYkNmxC6O2RxCNMMPqC/tGkCGfCDFPSWdzGJxgHhH7rRlAzxJxa6x/KpUFS1mwalYF9Ac+ntOJCW04woZZ1UEhksnmwMRp8ohIRsrwD2v2jSFGEbFYiATlbs0AQFPha8em3DOmlUNvyzjzhzCOXRhFNBxAPJXF670xLt+TIGRjIJZATgWCfgVt9fbCzBgtLObdpWk61pZRlJmrMcxSK2A6kXkLrTwM8dg3Y0xlJY+IZBhbG2b7ZkwR4To1Y9Gsyn6oRLdm7JpVAb2IcdMhX47Xe/kWIn7yiRBVgHHHjJ1RfiNavlDcHY+INjFjYbu5EREJ1kmLEe8AEOKw48z40E2FiGQwqSvgUxAwWYiEOa5l1iLeLfQLARcUEYetGQBoKhhWxyQpRPYNTQEAjmpv4PY9ySdCzHd6OUzMMNyeprM7McMQoYh4NTWTzOb/HxQlf7+TASpEClhdyQzwVUQSFsLUjNQWWjmixnd5KCJNtfmLzviU9x6RdDaHvkI6ZFdzHbfvy3wiz+8bQVaS6SCC4AkzqjqdmAH0oEO3WjN2M0QYIh74EnbuORxaM9rort9nSx0SARUiBexVp/w8InYc1ABQWyiGhJlVeXhE6uTxiPSOTSOn5n+gF0Sc9bmNrFkURUMogIlEBm/2kU+EmH9oiohDoypgGN91uTXjVBHh2QJPOrjnOFFEjIWILMhzEo+xo4iEOU7N2JkpB8RIhkZi086SVQE9S0SG1syBkXxbZmlLHdengYDfh/XLmwFQ3DsxP+njsGeG4fb4rt3Nu4w6AdOJthQRLUfEfiHCzKqy+EMAKkQ0tHh3C9VpOMhPEUnYVURYa0ZYxLuzZFVAzxKRwaxqLER4o/tEyLBKzD8OFVozixt5tGb0QsSNoMNJh4WItlxUQKCZPY8Ih9YMFSLyoY9SuduvA/Jz3cxXYMVBDRh6l8KW3hUUEUfju/mLjgytmZ6Rgj9ERCFi8InIkiJLELzgqYg01+WvCTnVHaWUKSIRSRQRVVUND79W7jkcWjMFsyoVIhKitUY8UEQShjeV1YAtbXxXgCKSzanak4TdZFVA94iMSxBo1lNQREQUIicsaURt0I+xqTTeHpzk/v0JD9m2DVi0CFiwQP+1aFH+41XAVCqjPUjw8IgE/T7tuuBGzLvT1gzv8d1UNqcFrLmdI5Ikj4i8WNm8y2Cf69TAZPx6qx4RXTLkH59u3DQZcTK+K1GgmcjWTNDvw9quJgDAC/vJJ1LxbNsGtLbm5xxPPRVT/f0IDQ3hgqEhhIaGMNXfD3zwg16f0hXYxEwkFHDkFzPi5gjvhNNChPN0ovHh1VqOiHOPCGvNyJKqClAhomHPrMonRyRp6NlZDQoSsQOBwSZmwkGfIxlPRrNqV4tzebkYzLC6fd+okO9PuEB3N3DqqcCGDZgaGUEIgOLzo2XJaviWnYQ/LTsJvuVr0br8nehJeH1Yd+A5McNg6azDLozw6gvv5BjfZfcbRbGmTPBozchoVrWvt88z7IzvhjltQrQ7MQMYJUP+ikhMM6o6ewIymlVVVfVsdn18Oq0ZZnlmiBhZtyxfiLywnwqRiiMeB771LeDzn0dqYgKrAByqqUXDiRdgwSkfQCDafsSXnK0A1zzxJm46/2jLm7MrCZ7+EIY+OeNGa8ZpoBm/zChgpifRyvVwvppV5+9PjkVsBZpxSla1M0/OqDOM7/K+yeuju87eJk0FY1oqk0MindNGjt2G+UPaGmpsS7RzcfKyZihKXnkZnEigPcLvCZIQSHc38P73I/faa/gsgC/XNSGy7hIsOfkS+MP5BN7s1Diy8VFozX2fHzVtS/HAH/fiiVf78KUPnoB3v2OBd/8PAmETM4s4TMwwWhrca804nZqpDfJuzdgLsOSRrMrMqrJs3gWoENHQPSLWFRHHHhEbOwe0MxRu6jk1b4DisU2WwSPMDADqa/wI+BRkcirGplOorRHTFpkLVoh0ClJDgHzeyjEdEezsn8D2faO4+IRFwl6L4EA8Djz4IHDnnUgNDGABgMwx78Lii2+EL5R/n6SHDyL2/M8w+fofgOzM9uLv3uzHZx9/HQdHp/H333keH1i7GHe87zg0F5725wt9hdbMEq6tmUIh4mprxlmgGb/WTOHh0+L1mueuGTKrSoiuSlhXRBIOx3f11oyNrZCGipp3loge7+6sEFEURYp01Z5RcUZVI8wnQu2ZCuDXv0buhhvwmYEB1PsD8J9/LRZcugm+UB2S/bsx+LMvoff/fhKTO7YcUYQMDAzg3GM7sOVTZ+GaM1bApwC/eLkX//v72+ddzH/vONszI6I1I78iwnu5qD6lae0WzLM1Q2ZVCdFT7ux4RPiYVe0oIkG/D0F/vh3DO11VX3jnXDhrlGByRuTEjJH1y/J5IlSISEw8Dvz7vwO33ILPAfhy00K0ffRuRNddAgAYf+5H6P/ezZh++zlAzSEIYHrVKqgNDVAVBerAANrb856R+lAAn71kDX72yXehvsaP5/eN4D/+tNe7/zcB6Jt3+SkibN/MkAvju85bM0wR4ePF0zyJVhURLq0Z+cyq8pzEYzTzkI2VzE4VEWZ2tfqmZIQFbeDl1ZoB5EhXPVAIMxNdiDDD6uuHxoUl3hIOiMeB227LKyH79+Peleux6Op7EFp4FLJT4xj48e0Y++P3ADWHOgC3LliARDyO8O7dwMQEkMsB7UcaV9d2NeH29x0HAPjKll14o3d+7BxSVRX9rBDhaFZtdVER0QLNbD5U1XIONEvabMdz3TVDhYh8aG8MG+EyjgPNbBRBRkTsQQD4bN5lMMOql6FmmkdE0Oguo7O5Fh3REDI5Fa8cHBP6WoRFWBFyzz14N4Cvn3A+mj78WfhC9UgcfAN9370Rib3bEQRw69KlmNiyBV/q74evzlzxetn6TrxnTQfSWRU3PfYS1yVpXhFLZLSHnIVR/oqIG2ZVNjXjONCMc46I1XZ8iMN+MypEJMZOMaDniDhURGwUQUZ4/5Aw9NaMc0XE61CzbE7FoVF3FBFFUbT2zHZqz8jD4CDwoQ8hd889OAvAjo2Xoe2vboLi82Py1acw8MgmZCeGEFUUTD35JL7U3Q3fe94D+MxfExRFweYPnYC2hhq8NTCJu3+7S9z/j0swNaSpLsh14o15REanUkI9NclMFqls/vreYHPEmj3spbMq0lkOu8WcekScmFVZxDt5ROTDVqAZM6tyUkTsjO8C4hbfaa0ZHh4Rj0PNBmIJpLI5BHwKV8NdKbQ8kX2UsCoFg4PAJZcgs2ULlkPBG+d9As3v/nsAwPjWH2P4iXuAXBZRnw/DU1MIXHihpQLESFtDCF/+8IkAgP/8n248u3uI2/+GF7AMEZ5qCAA0F64JqgqMTYlTRZgaAgD1NgPNjNdmHipX0uaSU2rNzHPsjO+GJAg0A4Dawjl4KyITHBbeMbxefMeMqkuaa+G3mF5rBy1hdf8oLcDzGlaEbNuGVl8A0+/7J0TXfwAAMPK7BzD2zEMAgK5gEMPxOAJh5zfc81Z34G9PXQoA+Kcfv6J5FCqRfgFGVQAI+H1aMSKyPcP+7kMBHwI2VYB88Fj+v3k88Nm95vOYmtGSVUkRkQ87u2a0pXecIt6tLNwzwttIxdAVER5m1byqEvNIEelxaWKGsXpRFLVBP2KJDHYfpgV4nmEsQvxBhD54K+rXnA01m8HhX96FiRd+CQA4o60N+yYnuRQhjM+8dzU6m2vRN57AT188yO37ug2bmFkoQElscSFLZNKhURXIt9zqOLbAbSsimkckB1W194CTJEVEXpiqYeUfhxUtqUzO0VOvnrJnVxHhm/rHEGFWHfPIrCpy624xZizAo70z3mAsQoIhhP/6c6g76lTk0kkM/uyLmHrzGQDAGUuW4Jm+Pvhq+IaQ1YcC+MSZKwEA3312X8UqY6IUEcBoWBU3wut08y6DZ6iZ3Ws+U+xVVVc2rEKtGYlhBiQr/zjGatZJz86ug5rBO/WPoZlVeYzvehxopi27E5iqOhs92Ix8Iq5jLEJqalF72edRu/ydyKWmMfjjO5DYux31AG7buBHP7NsHX0BMyPSH13WiIRTA3sNx/E+FekX6Y0wREVCIuDDCq2WIONwFVFvDTxGxk1uV/3z9/mS3PcOMuxRoJiEpG4WI8U2RclCI2J0pZzDJkOeooKqqiDGPyDyYmnErzMwIM6zS5IzLGIqQtnAD6j5yJ8JdxyOXjGPgsc8i2fMqoj4fxrZswZ1//rOwIgTIR4r/9bpOAMBDz+4T9joiEauI5AuRIRdaM3bj3Rn6glEOrRkbSd7A7ELE3j0nTYqIvLBCImShSgz4FDDfoxPzkPOpGb6pf0C+6mcjdXxzRDzyiLg0umuELcDbPzyFwxPi0yMJzCxC6hpRd/n/QWjxMchOxzDwyK1I9e7MT8bE4whYHM21y9+dtgwA8PtdgzgwPCX89XjTp8W78y9EWurzrRmRG3j11oyz0WOe04l2IxsURdEKCLuFCHvotnKvE408J/EYO30zRVG4jFMlbIwOG9HNqs7n2xls867fp2gz9E5gyaqTyQyXOXwrTKeyWiHgZiHCFuABwHZqz4jHWIRE2lB/xZdR07ES2clRDPxwE1IDe/QihKMpdS5WLmjAWe9YAFUFvvfcPtdelwfxZEZTRkWYVd1pzeSvrw0OlV02nTjFozXjIMRSzxKx2ZohRURe7C4C4pF0l3SqiGhubn6KCJuYiYQDUBTn467GLBK3J2fYsrtIOKB5VdxCzxOh9oxQDEXIgqZFaLjy3xBs7URmfBD9P/w00kP7PSlCGB87fTkA4LEXeipqlJf5QyKhgOPWRjHcaM3om3cdKiKsBc5TEbFxzXf68EuFiMTYXQTEI+bdqUeEZ++SMcFxdBfIZwawFo/boWZuj+4aOXVFPmH1eQo2EwNbXrdhQ74IaVuG+iu/jEBjO9LDB9H/g1uQGe31tAgBgHe/YwGWt9ZhIpHBz1865MkZ7NA/Ls6oCrizgTfOyaxaV8NvOtHu0jvA+eK7JJlV5SWVsT6+C/BJutPDbRzmiHA0q7LWTLSW31NQk0eTM14YVRkbVrQCAF47NK6pTAQnjMvr9u1D6+Jj0HDFZgQaWpAa2Iv+H/5rPrLd4yIEAHw+BVedthxAvj1jNwPCbfoEFyJtbHxX4AbeCU7juzyXi9qNeDd+jd3WDJlVJUabmrHamuGQdKeH2zhTRHiO72qtmRC/VgZLV3V78d0BlzNEjCxsDGNFWz1yKsW9c2XW8rqvrjoVjZd/Cf7aKJKHdmLgkU3ITY1JUYQwLlvfiboaP94amMRze4a9Po4p+gUaVQFdERmbTiMjyDsW5zQ1U8fxgc9OkjfDcWvG5r1OJPKcxGNSNpJVgZlJd3bRwm1sKiLsB4Tn+G6MY5gZgxlW3Z6ccTvMbDYbV+bbM5Vy85GeeBy45RatCHn5pAux4EO3wRcMY2rPNgw8dhtyybhURQiQb3N+6OQlAIDvb93v8WnMITJVFQCaC9N0qiquZasVIg6vZfpQgHOPj5MQS6etGfKISEommwMLPbRsVmXVqQOPiBMHNQCEBQSa8frhNeJVqFnPiPuju0Y2rsy3Z7buJUXEMWyD7n334SwAr77rCrRedEN+g+6OLTj80y9CTSelK0IYl5+S3z/zh12D3FcyiEBkhgiQn8pjRnZRDyiTvJJVOUa8O2nHO1Xh7T50i0Sek3iIMSrXrlnVWWvGmUeE5w4EBi8504gXoWaqqhpSVcVv3S0GK0Re7x33LEdlXmDcoOsL4M2LbkDTGVcAAMaefRTDv/kGoOa4Lq/jzXGLo1jSVItEOoc/V0DSqmiPCGBY/yDousC27zqemuH4wOekHc92ktl9+KVkVUkxpqLaL0ScKyJOA834Ts3weYowwsyqbt6MhyZTmE5noSj5zbte0BENYyX5RJzR3Q1s3JjPCKmNIvWRLyJy0oVQc1kM//abGP/TwwDELK/jiaIoOH91OwDgqTcGPD7N3Gjx7lGRhQi7LojxjnGLeOeYYO3kmu/0nkNmVUlJZvNvLEXJp6Vage/UjM2IdwFTM2IUEfbk455Zlakhi6Jh24oTDzYUVBHyiViEjeeedhpS3d1obFuG+r/7KsJLT0AuOYXBn34Rky//BgDQFYkIWV7Hm/esWQgA+N3OAS29WEYS6aw2ViuqNQPo3jFRigi31gzHB76kg2u+UxU+aWOdiWjkOYmHGMPMrIZ3OR2lUlXV9kpoRlhAjkg8xb8Q8cKsemgs7w/pdHHZXTGYYXVrNxUipmF+kBtuwGcGBtC66hS0fvQuBJsWIj3ah/7v/xMSe18AAHSFQtg7NCR0bwwvTl3RgkgogKHJFF7uGfP6OCUZKKgh4aBP+9kVgejrAnuocmq85zmd6OSa7+ThV1VV3axKrRm5YB4RO9n7jsNlDF/ndHw3mclxe8Jiscg8WzOaWdXFQkQbP2zyVqrXfSIx8omYYXAQuPjivB8ECu497W/Q8uHPwheqQ2L/DvR/72akh3vyG3TXrMG+WAwByZUQRk3Ah7OPlb8906cZVWu5pCuXQmS+UC6naoWD02sZr+nEbE7VfBq2ChHt4df6PceJH1Ik8pzEQ5yMMzltzRjfTLbNqobeJ68RXl6xyEaYWXXcRbOqZrYT2OM2A/OJqCqwrZt8IiWJx4GvfAU47jhkXnwRbaF6JD78WTSf9XdQFB8mXv4NBn70OeQSE6gB8ht0X31V+nbMbN6zpgMA8NQb/R6fpDT9Lv3s6PlC/K8LccOoLa/tu04VEWNLxe3WTMqQ1UKKiGQ4K0Qc9usKX+dTgKDf3lOH8c3Ma4R3UohZVQ8vcgsmL4t0/Ztl4yo2xkvtmaKwVsw//zNuGxpCtGMVGq6+B3VHnYpcOomhJ+7ByG+/CeQyqAEw4eIGXd6cfcwCBHwK9hyOY+/hSa+PU5Q+waO7DF0R4e8dY/4Qv09xPK7KK8Ha+PDpdmsm7WAwQyTynMRDUll78e6AM5kMMGSIBPy25U+fT+Hq6Ab4GbyMGKdm3Iq4dutiagbWnnmOCpGZMEPqKacUWjHAN066EAs+ehcCzA/y8L8g/upTAIAuvx/xeBw1dd76fpwQDQe198N/vylne8atIl4zq4pQRLSJGfvXVwYvsyrbth70K/BbHI4A+Cgifp+91xYFFSLQK0s7c9VOWzMJhwvvGDxn3AFd0owIMKtmc6pW6IimX3AypBU2FhbgvdEXc7U9JTUsJfWGG/CZAwfQVFOH6fffkg8pC9Rg6u2/oO+hm5Ae3AugMJ47NYVABRchDL09I2ch0ic43p0h0qzKvG4RDss764J8lt4ZHz7t4OThV0ajKuBSIXLfffdhxYoVCIfDWLduHf70pz+58bKmYQYeO/84jlszDjNEGDxT/wDDkwTHQiQc9Gt/X26EmmWyOQxO5JdpyaCItEfDWLkg7xOhbbzQskFy992HdwO4a9E70Pixb6B+9VlQsxmM/uFBHP7ZnVCTcQQB3LZxY0WM55rl/EIhsn3/qNClb3Zxq4hnLVsRxbl+HXPudQvX5K9d0+msI0XX6bZ1Jw+/SQkzRAAXCpHHHnsMN910E2677Ta89NJLOPPMM3HxxRfjwIEDol/aNHw8Is4UEW79Sy6jZVmtOONZiADuhpoNTaaQzanw+xRty6fX6HHvVdyeYYbUtWuReu01NEPBjlM/hIVX/huCTQuRGetH/w8/jdjzPwWgIgJg8sknceef/1wR47lmWdJUi+MWR5FTgd/vHPT6OEfgukdEiCLC74GKPeypKp8AS9uKiJPWjAP1XyTCT/PVr34V11xzDT7+8Y9j9erV+PrXv46uri7cf//9ol/aNI4KEYdxu/ryI16KiPOWBzOqAvneKk/0UDPxhQiTljsiIWn6oadVe7CZ0ZAai6Guvhnhv/k8ms/5Byj+AOI7/4Te796IVO8uAMAZ0SjGpqdRc+GFFWlKnYvzV8vZnklnczhcUGlEe0T01Q8p5DgHvLFrGY88pFrDNdrJA58WYGlbEbH/8JvOyrdnBhBciKRSKWzfvh0XXHDBjI9fcMEFePbZZ4/4/GQyiVgsNuOXGzCzqicpd9rCOz57EKZTzldps90MtUE/ApwrZz1LRHy6ar8LezKssqEQbPZmf8zVhFkp6O4GTj1VM6R+beV6LPqHf0ftipORSycw/OS/Y+gXX4aajKMOhVbM8DB8ksa184D5RP709hDX7dlOGZxIQlXz7eqWOrGtsGihEMmpwCSHzbZGmNfNabw7AAT8Pq19P+Xg30oLM7PtEbH/8JuSMFUVEFyIDA0NIZvNoqOjY8bHOzo60N9/5Pz85s2b0djYqP3q6uoSeTwNJ3IVL7Oq49aMNuPOQRER4A9huJmuagxkkoX2SBhHtTdAVfM3n6rAGNO+fz/q/UFMnve/0H7ZHfDXNSI12I3+hz6FyVeeBAB0BYMY37Jl3rViisGW4E2ns1KpZCwIsKMxBJ9gNTEc9Gt+Cd4+kUnOW8R5tMB1FdypImK/NVOVZtXZY1OqqhYdpdq0aRPGx8e1Xz09PW4cDykuZlVn47tOWzO8Uv8AY7w7/90sbm7g7ZcoQ8TIeYWlZ7KObXJlcBB4//u1mPbG1k60/t1XEV3/fgBAbNvj6CukpPoA3Lp6dX5pXYXmg1hFURSceXQbALnGurUiPupOES8q1Iz3zqxaDus09N1iTj0i9qdmggE5WtUMoY8bbW1t8Pv9R6gfg4ODR6gkABAKhRAKuW8qdOYRcbZrRnNQc1NEnBciIhURN82qMmWIGHnP6g58+5m9+MPOQaSzOemMY9zo7gbOOw+Z7m6sBDB64nuw4Pz/DV8wjGx8FENPfB2JvdsBABEAQ088MW+9IOXYuLIVj27rkcrA7HZbs6kuiP5YgvsDSlxbVcHnoYrHglF9z4z7UzNaa0aya47Q09TU1GDdunV46qmnZnz8qaeewumnny7ypS3BI+I95VAR4eYR4aCIiEhVZWjpqi74I5i8LJsi8s6lzWitr0EskcHz8zHufXAQuOoq4KSTkOruRgPLBrn4RviCYUx3v4jeB2/QihDNkHrxxVVXhAD6JNVrh8YRS8iRL+N2Ea+HmvG9LkxwvpaFOcQkJJ0qIkEOrZlq8ogAwM0334z//M//xHe+8x28+eab+NSnPoUDBw7g2muvFf3SpvF0fDfNVxHhMb6rbasU6BFxZ2pGTkXE71O09oxs0xKOYcvqHn4YZ05MoHHhUWidlQ0y+KPbkYuPIQjg1g0b5r0hdS4WNoaxoq0eOYn2EHmhiAD8rwvcWzOaR8S+F09vxzv0iDgJNLNZBIlCuBPsIx/5CIaHh/GFL3wBfX19OP744/HEE09g2bJlol/aNFrEux2PiIPqNP91nALNeCoi88CsmsuphohqecyqjPNXd+BHLxzEU28M4Pb3rRG63dQ1DK2YVgC5tRej/bz/BSUQRGasH4d/dZc2lhtVFAz/5jdV4wWZi40rW9A9FMfWvcM4b/WRbWu3cStVlSHquqD73fhcy/i0ZpxFNszH1owrlvRPfvKT+OQnP+nGS9lCS1Z1sn3XZo6ILtPJE2im91Ur1yMyHE8hnVWhKEB7RI4wMyNnHr0A4aAPh8am8WbfBNYsjnp9JPvE48CDDwJ33onUwAAigRAiF16HhuPPBQBMvfUchp74OtRkHH4AG+vr8fTQEAJVrILMZuPKVjzyfA+27pVNEXHJrFonxqzK+6EqzMGLpweaeTg1I5lZVa6yyCOcjDQ5bc1wU0TYD4j0UzPuBJqxC+mChpCUZtDaGj/OOGoBgApvzxh3xQwMINq8GK1X3Y2G48+Fmsti9A/fweGffwlqMo4aAFNf/jL+Z2yMipBZbFiR94m83jvuipG7HNmcigGXVyM01orZwMu7NVPHdXzX3vWVfV0yk7McNZ+WVBGR6zQe4SR/nxUiqWzOViqg05lyhja+y0ER4W3wMtLkUqAZG92VzR9i5AK29OzNIzN1KgKWklrYFfOVlevQ/vdfQ037CmQnRzHw6G2IPf8zAEAn25h7yy3APM8GsYPRJ/KCx3uIhiaTrq9GEOUR4W285+HFY/cbuwMKzA6gqrqab/W1q86sWgnwiHgH9P6bFZzOlDN4SIYM3k8RRliyaiKdE5okKevEjJFzjm2HogCvHYppPfmKYXAQuOQSLSX1lVMuRfuHPwdfqB6JntfR991/RLLntfyyutWrsX+ebMwVycZC6q7XwWbM5O3magRNKRXUmuGl7vLw4iUctuONX2e1PVO1UzOVACsg7CWrGt4UNnwiTmfKGXU1fFZUA2ILkYaaANi1LSZQgpYxVXU2CyIhnLy0GQDw35XUnilszc1s24ZWfwBTF9+IlnM/DsXnx8Qrv8XAo7chGx9FDQrL6l57bd5szBWJthCx29tCRE9Vda+IZ60ZntcEVVURLzyYNYSCXL4nj7ymhMN2vLGtYtUSoJtV5ZqaoUIEQNpBlRjwKdqN1Y55SF+AxGnpneSBZj6fYsgMEFeIMI9IR1ReRQTQd41sqZRCZHAQuOgipLq7EamNovYjX0LDie+Bmsti5L8fwMiT9wK5DKKKkm/FVGFAmV1YIfJ6b8xTn0jPSL4Q6Wx2T8ES0ZpJZnLI5tgWcU6KSNB5gnXSYTteURTb3kRZk1XpCgG9SgzZUETybwr741ROHdSM2pr813NRRDiPvM1GDzVzQxGpjEJk695hacKsisL2xWzYgMxbb6GpaRFarvoKwl3HIZeYxOBPPo+J7b8EkN8VM0ytGMt0RMNY2VYP1eM8kZ7RKQDA0hb31EQRgWbsgQrgs/QO4NSaYR4RB+14PUvE2jnSDu51IpHrNB7htG/mJEvE6Uw5ozbIrzWjrc7mtChqNqIc8kZk3TMzm1ULGrCyrR7prIo/vnXY6+MUJx4HbrstPxmzbx9aF70DrR+9C8HmRUiP9aPv+/+MRPeLqAdw25o1+V0xNBVjiw2sPeNh3PuBkXwh0uWBIsLTO8auY3U1fm6L+1gh4qQ141QRAQwbeG0qIuQRkRAn23cBvTpN2PCI8Fp6xzNHZJLliHB6ipiNPjkjRgFQVdX1QCYnaO2Z1yVsz7Ai5J57CpMx69F4+f+Bv74Jyf7d6P/+PyMzchA1AMa2bMGdr75KfhAHaIZVCQqRpS3uFSINoYBmjOXVlhLRYuaxXFTziPBQRKgQmT9oBh7bLmYHrZkMn0AzY+Kf1dny2Yg0qwKGFEVBrZnx6bRW4MnuEQGAi09YBAB48vV+jMTF7+Axzawi5KUT3oP2D38Wvpowpvdux8Ajm5CbGkMNgIl4nFJSOXBaQRF5oy8m7OejHLmcioOj+SK+y8VCRFEU7umqIq5jPMyqWoilE0XEZmsmSTki8uK4NeMg6S7JSRFhX5/NqbbGiBnZnKq1d3gZvGbTUp9/Yh4WdNNl/pCW+hrHf69ucFJnI05Y0ohUJodHnj/g9XHyzCpCdpz6YbT91Y1QfH5Mvvo7DP70C1BT07oplfwgXGiPhrFyQd4n8rwHeSKDE0mkMjn4fYrramIT5z1ULA8pyrHFXMumEznkiDi5Ntl9+NXNqnLd+uU6jUc4zd/XPSJ2xnf5BpoBQCJlvxCJG5Y5iZiaAYDWQiEyEk8K+f5aPHUFqCFA/mnw709fDgD4wdb9yDgoJLlgKELOAvDamR9F8zkfAwCMb/0xhp/4GpDLkilVEBs99ImwtsySploEXH5qbqzj6x1j5u9oLZ/RXcAwncghR8RRa8bmPYeSVSXGuSJif9+MPjXj7Mk96PchUOixTqXtb4ZkBq+gX3HcLipFayGtcXhSrCJSCf4QxiUnLkJLfQ16xxPeRr7PKEIUvHHe/0Lj6ZcDAEaf/i7GnnkIAHBGWxuZUgUhQyHipj+E0cR5rJ9lkkQ4KiI8I965tGYo0Gz+4GTXDOCwNcNJEQH4GFbjBoOXqI2wolszlZCqOptw0I+/PbULAPDdZ/d5cwgW2X7PPThL8WHnxf+I6Pr3AwCGt9yH2F9+AgA4Y8kSPNPXR6ZUQWxYkTesvtkXmzGC6gY9bGLGxdFdhrb4jntrhp8iEuagiCS5mFXtPfymtNFhuW79cp3GI9KOzar2ZLJsTtV2BTh5UzJ4yIaa01zQxAwAtDWwQkRMa6YSFREA+OjGZfD7FPylewRv9sXcfXFjZLsvgF3v/xctqGzov76KyZeeAFAoQvbtg4/2xQijIxpGZ3Mtcirw0oFRV19bL0TcV0R4m1WFtGYcPuypqsplv5jtqRmH9zpRyHUaj+DWmrH4pjCOgDmR6Rh8FBEWiSzuRtNSn2/NjAhqzegZIvLGuxdjUWMtLjpuIQDgITdVEVaEbNuGVn8QiQ/eivpjz4SaTePwL/5/xF//PQAqQtxk/bJ89P8L+9wtRLxszfAONYtN5x+qIjzHdwsPe6lszpaXK51VwXajOgo003JE7LVmZNtILtdpPCLpVBFhxiGro1SGwkU6RUTQxAygt2biqayQxXeVqogA0Eyrj798SGjgm4axCAmGEP7rz6HuqFORSycx+NMvYvqt5wBQEeI265bn2zPb97usiIy6H2bG4B3zPpEUp4gA9q6zxsKBj0fE7q4ZuW79cp3GA1RV5egRsaeI1Ph9XJL/eKT+adsqOfZVZxMNBxD05/9/RfhEBsYrI1W1GKcsb8bqRVEk0jk8tq1H7IsZi5CaWtRe9nnULn8ncqlpDP74DiS6XwRARYgXMEXkpQOjrk1RJdJZDMTy7VJPzKp1nFszBUUkWsvvfRsK+MCsc3YKEWPopROfhh6iSWbVeQHzaABOChFnrRlexiEeqX9xzmuzi6EoiqaK8G7PTCTSmCj8P1TK+K4RRVFw9enLAADf37pfW9rFHUMR0hZuQN1H7kS463jkEpMYeOyzSPa8mo9s37iRihAPeEdHBJFQAPFUFjv7J1x5zYMFNSQSCmhFgZs01fLdQTVR8IhEOG3eBfI/n04WjBqv+U6GAZzmiFAhIhlpw9OG24Fm7E3kdPMug0fqnxtmVQBoLfhEhjgbVgcK/pBoOCAsB0U0H1i7BE11QRwcncZPtgtQRYxFSKQVdVd+GaHFxyA7NY6BR29Dqncnoj5fPrL9z3+mIsQD/D4F79R8Iu4Em7Gtu10tdcIm5soR5W5WZYoI36KqzsHiOx5hZoDe1rGsiFBrRk5SGQ6FiOYRsaeI8BjdBfik/sUF7GcoRmuDGEVE94dUllHVSDjox7XvXgUA+OJ/valNMjjGuEF32zYsaO1Ew0fvQk3bMmQmhjDwyCakBvYg6vNhmCLbPUczrLrkEzng4eguYPSI8DKrMo8I32tZmIMi4vSaH7apiPBK8uZN1V9lWIXo9yna0iWr2G/N8J3pri28uZ2YVVkhwjMEqBit9WJGePsq2B9i5BNnrsT6Zc2YTGbwTz96xXmLZvYG3cXHoOHKf0Mg2o70cA/6v/8vSA8d0IsQCirzHFaIuGVY9XJiBtADzWKJjOP3u6qqWo5IhLPfzUlrJqntFnO60sO6IqKqquH15br1y3UaD3BqVAWctGZYdcy3NeNEEZlwSRFhI7y8zar9FTwxY8TvU/DVv1mL+ho/nt83ggf+uNf+NxscBN7//lkbdL8Ef20Uyd6d6P/Bp5GdOExFiGSsXdoEv09B33gCh8amhb+e14VIo6GFEnPYnklmctpDJs9dM4DD1oymSDi79doZkJgxOkyKiFwktblqJ8Yhu1MzfGUyrTXDQRGp9NZMpSsiALC0tQ63v+84AMBXn9qF13vHrX+T7m5g40Zkfv97LFd8ePVdV6D9rz8HXzCM6T0vYODR25CbjlERIiF1NQGsWRQF4I5PhLUAOz0qRAJ+n5b54TTmnRUyisLf7xZ24MVLcHr4ZF9vJTJixugwKSJyoaeqcgiXsegR4S2T8TCr6oFmYivmVkEx71q8ewVOzBTjsvWduGBNB9JZFZ967GXzUuzgIPC3fwusWYNUdzci9U1I/s0X0XTGFVAUHyZefhKDP/si1HSSihCJWedSe0ZVVa0Q8UoRAfgZVplRNRIKcIlGMOJEEeHVjtfNqubvOcYHZSpEJINH9r7t1gxnRYTH+K5bUzOi9s3MJ0UEyI8Lbv7QCWhrqMFbA5P4zOOvlf/3LSgg6OpC5tFHcUYigejSE9B69TdQu/wk5FIJDP3qboz89t/1DbpUhEjL+uXuJKyOxFOIp7JQlPzmXa/gZVgVEe/OcJJgzasdr5tVrSgi+r3Oi6moclT9XB6P7H3bZlWOC+8AIKwFmtlflKXliIg2q2obeMWYVSt5amY2rQ0hfPnDJ+Kah17AT7YfxPb9o/jSB4/H6avagMWLgb6+GZ+fArAKwGDbUkRO/SA6jjsXis+P1OH9OPyLzcgMHwQAdNXWYt/YGC2vk5j1y/IJqzv78wvwRK1e6BnNK4kdkbCnExW8Qs1EGVUBoDZovwXOa9s6+zeyoojwzq3iiXwnchm+ZlW7gWacFBEt4t1+EqOWrCraI8ICzTgqIvFkRruALW6aX0/4563uwLevWof2SAjdQ3Fc8R9/wb/8+BWMjk7ihwAUw69o53FIfvhzWHTNfWg44T1QfH5M7ngK/d+7GZnhg/ADeFdjI/ZSESI9CxvdWYDntVGVwSvUTBvdFfBAVVuTv97b8oiwaz43s6oFRYQVQZIZVQFSRPQlQAEHZlW7u2Y4OagZumRoXxGZdNmsOpXKYjqVnbHDwS59BX9IJBwQ8iTkNRcetxCnrWrFvz25Ew9vPYAfbz+IJ//xexgZG8JiRYECAP4gApFWAICq5jD11nOI/eWnSPW9BSD/5DH15S+j5uabAQoqqwjWL2vGwdFpvLBvFGcevUDIa3i5dddII6d9M0JbM0H7LXAt0Mzhw2fIhiIi6+guQIUIl6Q5pmikbLZmeCkiTpfeqapqiHgX+9ZoCAVQ4/chlc1hOJ5EZ43zC+ChsXxbZvE8asvMJhoO4s5LT8Cla5dg089exduDkwg2L5rxOWomhcnXfofY8z9HZrRX+3gEwNDEBGoaGlw+NeGEdctb8PjLvUINqz0eh5kxGjmZVfXWjAhFxH5wJLdAMxs5IrynNHlChQiH7H2n47tOZTqGExMVOw+bMxetiLB9M/2xBEbiKXRy2PbZV8hamG9tmWKsX96CJ248E2/2xXDKhtMAqICa/8fLjPYhl9D3k9QC+NTSpfjirl3wkSm14pi9AC8gIJ5bntZMQRGZdmhW1Voz4hQRe60ZPh4RO75EmRUR+U7kMnoh4mR812GgGW9FxGYhwtZmK4ruNxEJa88Mc8oS6S0UIos8dP27SdDvw4mdTXjwrs8i1bsLqb63kOp7a0YR8q9LlmByyxZ8qbubipAKxY0FeNIUIsysyqs1I0ARcTKdmOQ0oGBHEUlymBAVhXwnchmerRnru2b4KiJO5tsBPUOkvob/7H0xeI/w9hYmZrwcP/SCK664AuoPfgAVyP/q6IC6cSPUJ5/E5gMH4KOdMRWN6AV46WxOK+K9LkQamVmVU2tGpEfEznQir/aIURFRVXNx+LwW7omg6q9OeqCZF8mqfBURJ4l/gDFV1Z03ahvnEV5NEZknGSKWuOKKfGtGVYH+fuC554ALL6QCZJ6wYUV+jPcv3fwLkb6xBHJq/jq2IBLi/v2twC1HRGRrxlHEO5/2iFFRMXvfofFdieE5vpvK5pCzsKyJd4XKFJFkxto5GG5NzDBaOI/wsgyRxVWmiBDzH1aIPN89YvoJ2CwHDBMzXgdd6WZV+5N/gGCzqqOld3yu+cavN6vE660ZUkSkI8nDrGp4U7BWjxl4V6jGEVg71bpbEzMM5hEZ4uARUVVVU0Tm89QMUZ2c2NmEcNCH4XgKuwcnuX5vWfwhgDHQLOWo4BI5vuss4p2PRyTgU8C652a9iUlOry0C+U7kMjynZgBrPhHeEe/GFo+dHxK3wswYeqiZ89bMSDyFZCYHRQE6Gr2VlwmCNzUBH05emveJbOXcnukZLSgizd4X8CzQLJ1VHe3MihUUFRGtmbCTQoRTZIOiKJbTVUkRkRimYAQdtGbsVKfGz+VVofp8iva97MiG7rdm8gUDj9YMa8u0NYSk/EEjCKdsWJEPqvvL3mGu3/eAJGFmQP5ayB4KnRhWJwqKiHStGY4hluwBOGFREeE1HMET+U7kMmkOioiiKLbmunnNlBtxEmpWya2ZQ1qGiPdPdQQhgg0rdcMqT5+IDFt3GYqi6FkiNg2rmWwO8UKRILQ1YydHhGOIZdji1nca35UYpoiEHIYE2ckS4b30DgDqHKT+TbLxXZemZnjum9HCzKpxYoaoCtZ2NaEm4MPhiSS6h+Jcvmcup6L7cP57LWut5/I9neI0XZUZVQGxishUOmu5IGTtpjoOKy201ozJe47uT5FPMaZChIMiAhhkMg89IvnvZX8hk66IuLOnhW3gnU5nHW0MBvQMEVJEiPlKOOjH2q4mAPzGePcOxTGRzCAc9GHVAjkKEaehZqwQqQ36HbXcS8GGAlTVemTDlPaw57xA0mIjSBGpfPgVIjZaMwIid5kiYif1bzLBChF3Kub6Gr/29+40XbWqM0SIqmEjyxPh5BPZcXAMAHD84kYh0fF2cBpqpk/MiGkx1xoeHK0qz/HCAxcPRSRkcfkemVUlJsnBrArYW8ssQipzsgdhMuWuWVVRFLRxSldlhUi1paoS1cWGlQXDKiefyI6D4wDy48Gy0ORwA29MM6qKUXYDfp+WO2XFi6eq+iQQT0XEtFlVgBWAF/KdyGV4mFUBo0fEnCKiqqpeoXJ8YzhJ/Yu7PDUDAC0NfEZ42dRMteyZIaqTk5c2I+hX0DeeQM/ItOPv90pBETmpq9Hx9+KF08V3+uiuuOuYNp1ocddLthA0ydMjYrY1I2I4ghdVX4jw2DUDWN83k8rm2LJUMVMzNjwXbk/NAPoIr5PWTCabw0CMeUSoNUPMX2pr/Jp6sbXbWXsmnc3hjd4YALkUEc2s6lARETExw7Cz6dyoUrMWuhPCNhURGt+VEN5mVdMpdwblhKdUxn5AbLVmCkYqNwsRHq2ZgYkkcioQ9Ctoq6cwM2J+o+2d2evMsLqrfwLJTA7RcADLW70f3WXoKqm9a4Ie7y6uENGmEy0oIuxBLxz0wc9hqWjI6vhumsyq0pLi5CS2uviO+UMUxbkaY4QVEexNb4XJZP5JwtXWDIcRXuYPWdgYdmVrMEF4ie4TcaaIsLbMiZ1Nnu+YMdLq8OFEX3gnsjVj/YFP84dwUEMA64qINhxB47vywSNZFbA+NWOsTnleBNjcfCxhpzXjviLCRniHHGzgpR0zRDWxblkz/D4FB0entSA/O+zoYUZVefwhgH5NsLuV243WjJ1QM21ihtNUouWId1JE5IVba4aZVU2PUokJl2FyJPthtIIe8e5excwj1Kx3jDJEiOqhIRTA8UvyxYOTMV6jIiITmiJi0zcmcvMuQ0+wNv/Ap2WIcFJE7NoByKwqIfzMqlZbM4UwM85vCjY7P2FREUlnc1pR5q4i4uyiAwB94yzenYyqRHWw0aFPZDqVxduFLb4yTcwAuiIykcxYikNg6K0ZN8yq5nOjeGaIAHYi3ml8V1q8CjRLCFpAxBSRCYuKiNFTUqkekUXUmiGqBLZ3xu7kzOu948jmVCyIhLAwKlcBHw0HEPTn29V2rgvuKiJWPCJ84xGsKiI0visxXk/N8FZENI/ItDVFhP3whgI+IbHIpWhl47vxpO2AJtaaoTAzolo4ZXkLgn4F+4en8NbAhOWvf6UQZHZSZ6NURlUgH3TY6mCs35XxXRsxCcyDx1sRMe0REZDkzQv5TuQy3FozmkfEmiLCWyZjcuRE0qIiknI/QwTQWzOJdM7WyDEA9BZaM4uoNUNUCZFwEGcdvQAA8OsdfZa/foek/hAGU0rtmNi1QsSN1owdRYTX1EyQ7Teb+wyiAjR5Id+JXIZbsqrl1owYmYyNrFn1iHiRqgrknw5YhW5Hhp1KZbQoaDKrEtXEe09cBAD49at9ltVEPdpdLn8Iw4l3jF37RI7v2slr0hQRTsMAVu45xgBN2r4rIZoi4nprRrRHJGPp4uRFmBlQ2DfjYISXtWUaQgGhT0AEIRvnr+lAjd+H3YOTeGtg0vTXjU+n0T0UByCvIsKuCcMWVz+oqqoXIiLHd4PWx3d5KyIhC4qIsVih1oxk5HIq0tn8zdqzqRnO1SmbmsnmVIvVujetGcCZYZUmZohqJRoO4qx3sPZMr+mve7WghnS11Go/e7JhN9RsKpXV9rmIfDCpY8GRlnJEmEeEl1nV/PZdZhngHaDJC/lO5CJMDQF45IhYXUAkxjhUG/Rr8cFW2jOTCfczRBhOZFiamCGqmUsK7Zn/stCekTU/xIgeambtmsD8IQGfInRMNVIoRCYtTCdOcc5pCltYtGq838hmTgaqvBBJGwoR58mqNqdmOCsiiqIY0lXN/5BMeuQRAXRFxE6kM4WZEdXMeavbURPwYe/hOHb2m5ueYUbVtTIXIlqombXWjLZ5tzYo9IbbULjGTlpYpcFbEdGnZsy3ZmQc3QWqvBBJGSpJbtt3LeaIiKjaozayRLxszdi96ADGeHdqzRDVRyQcxNnvsDY984qk0e5GNJXU4sMJu+aJzBAB9OukFdVZzxHhZVY1r4jIPLoLVHshou2ZURwvS7PsEdHeGPwrVDv7ZiY9Gt8FdBnWnkeEFBGiurEyPTMYS6A/loBPgRYTLyNOWzOijeu2FJGkKEXETCEiRoHnRXUXIhk+GSKAjV0zmlmV/z+BHmpmXRGpvNYMZYgQ1c15qzsQCvjQPRTHG32xsp/LgsyOam/w5GfdLLpZ1VrQod6aEfv/pnlELBQi+tQM74j3ue85ojyJvJDzVC7BCpEgh38cpmykzG7fZbn/AhSRqGGE1yzMrOqFItLWoF90rKCqqhZmRqmqRLXSEArgnGPaAczdnvnBX/YDyG/wlRm7QYdaaybkkiJiISZBzxHhHfFuXhGRMcwMqPZChFOqKmB/fFfEGyNipxBhmyE9UUQKrRmLMuzoVFr7e1xIHhGiijHTnvnDzkE8veswgn4FnzhzpZvHs0xdTUCLUbfSnokl3FFE2ANbJqeavuaLUkRS2Zw2slyKpMR7ZoBqL0Q4paoChtaMxUAzET07O1Mzcc6jZVZgMuxQPGVJhmVtmbaGkLQ/YAThBuce245w0If9w1N4vffI9kwqk8MXf/0GAOBj71qBlQsa3D6iZZgqMmRBKXXLI2IMJTP7wKdNzXBWRIC5lXiZN+8CVIgA4FSIBKzmiIhZegfoiYKWpmZS4jdWlqI9GoJPyf97HLYwOaNNzJA/hKhy6kMBnHtsvj1zxy9fP8K78L3n9mHv4TjaGmpw/blHeXFEy9gxrDKPSERwIeLzKZoqYsYnks7mtPsNb0UEmHuElxQRifG2NSMm4h2wt29GyxHh5Oi2QijgR1dLHQBg96D5qOqeUTa6S/4Qgrjh3KMRCQfwwv5RfOzB5zWVc2gyiXt+9zYA4F8uPKZiViG02Rjr1zfvir+OaYWIieus0efCa2rG71MQ9OenPRNzKPE0visxfBWR/PdIZXPIzdGvA8QGzNiZmtGTVb1x0h9VkIr3WChE3ioEOL2jQ36ZmSBEs3pRFN+/ZgMioQC27RvFxx7chngyg69seQsTiQyOXxLFX6/r8vqYprGTJaIvvBNfbDHDqplN58wfEvQrXO43DLNKfNWO7+7btw/XXHMNVqxYgdraWqxatQq33347UinrI5qiSPNURAz/wMbo+FKIVUSsm1W9DDQD8uOEgDVFZGd/vhd+zMKokDMRRKWxtqsJ3/94vhh5ft8IPvLAc3h02wEAwO3vO05b/1AJMBO7tdaMO4FmgDVFhHeGCIN5PuZSRKp2fHfnzp3I5XL49re/jddffx1f+9rX8K1vfQu33nqrqJe0TFKAIgKY84loyapCFBFrhUgqk9OMVE113si2q1ghcthcIZLLqdrG0WMWRoSdiyAqDWMx8tqhGFQ1v5PmlOUtXh/NEnbG+vXWjPjrWMRCqBnviRmGVUVE1kJEWNl40UUX4aKLLtJ+v3LlSuzatQv3338/7r77blEvawmerZmAT4FPAXIq68eV/0HQpTKBgWYmzapjU/knDp/ijqRZDKuKyIGRKUynswgFfFjeWifyaARRcaztasL3rjkVf/ed56EA2PRXq70+kmXsLMNkD1+uKiImChHeGSIMpqjPaVbVckTkbM24qsOPj4+jpaV0VZ5MJpFM6tVvLFY+JdApPM2qiqIgFPBjOp01uQ1RnEdEn5oxp4iMTuULlqa6GsdR93ZhhchALIlYIj1nQcQWfB3d0YCAhGutCcJr3rm0Gc/+67lIZnJoK0ygVBKthdbMkBWz6rQ747uAtX0zohQRpqgn5rjn6Aq8nNdK1061Z88e3Hvvvbj22mtLfs7mzZvR2Nio/erqEmus4pmsCljLEkkKXHpnlAznCroB9B0vzR61ZYD8haM9kr/wmDGsav6QDvKHEEQpIuFgRRYhgK6ImN1BlczoD4FutGas7JvhvXmXYXa1iDa+K6kiYvkueMcdd0BRlLK/XnjhhRlf09vbi4suugiXXXYZPv7xj5f83ps2bcL4+Lj2q6enx/r/kQWYWTXE6Yma9d+8XkJklCXNGKlGC60ZtvPFK6y0Z3YVFJHVi8gfQhDzEaaIjMRTpiYRjcqEG6b7iJXxXUGBkWYVEdnHdy3/a11//fW4/PLLy37O8uXLtf/u7e3FOeecg9NOOw0PPPBA2a8LhUIIhdyr3nl6RACDcWiON0U2p2ptIRFvjFDAj1DAh2Qmh1gijcY5lA5WiDTVeV+IPLtn2JRhlRUiZFQliPkJezDK5FTEEuk5r0/axEwo4Mp0kAyKSNikRyQhuSJi+W+lra0NbW1tpj730KFDOOecc7Bu3To8+OCD8Pnkqsb4FyLmWjPGPxc11x0JB5GcTJrqX44WpM8WCQoRYO7WzHQqi33DcQBUiBDEfKUm4EM0HEAskcHQZGrOQkTLEHGhLQMADSHzXjxRiojZh1/ZFRFhp+rt7cXZZ5+Nrq4u3H333Th8+DD6+/vR398v6iUtk+RoVgWMHpE53hSG1o2oN0bUwuTMSDz/Oc1et2YWmGvNvD04gZyaf2JaUKH9b4Ig5qZNi3mf27DKrnVuranQFZG5r7GiFZE5PSLVOr67ZcsW7N69G7t370ZnZ+eMP7Oy2Ewk3M2qJme6WfhMwKcIm/iIWJicYeO7XppVAT1L5MDIFBLpbEm1iE3MHLswAkWpnIAmgiCs0dpQg71DcVOGVbZnxq0IgoiF8V3hOSJzKiJVumvm6quvhqqqRX/JAs9kVcB8a0ZbeCewX6fvmzGhiLBCxGNFpD0SQiQUQE6F1nopBvlDCKI6aDFs5p6LCa8UESvJqpxNtOY9IrR9V1rEeUS8X8lsZd/MqDa+620hoiiKnrBapj2zy6CIEAQxf2m10ZpxzyNiJdBMUI5I4WHWdKBZtSkilUCKc9/MrEwmMsyMYWXfDAs0a6n3fiunblgtrYjQjhmCqA70DbxmFBHWmnFJEbEQaBYvtGa454hYfPgVsduMB3KeyiW0ZFXegWYmZTKRb4qIthnS/NSM14oIYMgSKTHCOzSZxNBkCopCW3cJYr6jKSIm9s3oC+9c8ogUrrHJTE57qC3FVMGsyn1qxqQiotkBSBGRj1Qm71cJcveImFzJLPBNoS++K9+aSWdzWrEiRSEyx+QMa8ssa6nj/nRBEIRcWNk3E9PGd925LtQb/B7xOR742J97poi48PDrBDlP5RI8d80AVloz4t8U2vjudPkfkFHjwjuXeqvlYIrI3sOTRePpd5JRlSCqBmZWHbZgVnVraibo92k+v7l8IqIUEeseETlv+XKeyiVShb6Z24Fm+gIi8YrIXDkio3F94Z0baYRz0dVSh5pCKuyh0ekj/nxnH/lDCKJasJQjMs0277r3QGU21GxKkEdEL0RKP/yqqkpmVZnhPjWjeURMtmbc8IjM8QOix7t7r4YAgN+nYGVbPQBg9+GJI/5810BhxwwpIgQx72ktKCKjU2lksuWvq4MTCQDu7syKmIx5Z+O79cJaM6Uffo0KPY3vSohXrRlR/UIjrM0ytyIiR7y7kVIjvNmcircGqDVDENVCU10NmFDL8o6Kkc7m0FNQUJe31blxNADGEd7S19lsTsV0muWIiGrNlL7nGO9HpIhISLpgVnW7NcNUCpHBO2YVkRFJFt4ZKWVYzSeu5hAO+rCstd6LoxEE4SJ+n6L7RMoYVg+NTiObUxEO+tARCbt1PFMjvNMG/4Y4RaRcIZJ/fUUBgn7v2+/FqOpChPv4rkkHsxsJgFGTUzNjEmWIMI4qoYgwf8jR7REp/CwEQYiHFSLlYt67C0nMy1vr4XPx2mBmAy9beKco/FsjTBEpFxmRNIzuyroSo7oLkQzn1oz2ppirEBFvqmKFSCJdfsad/XB7He9uxFiIGFcC7KREVYKoOlrr84bVoTKG1X1D+UJkWat7bRnAsG+mjCLCFt7V1wS4FwJmIt5lDzMDqrwQSQqLePe+NdNg+N7lVJHRKXnCzBgr2urhU/K5AIcNFx/aMUMQ1YeZLBFWiCxvc7dla0YR0T2B/P0ZZnyJepK3vLd7eU/mAmx8l1+gmTmzqr6uWpwi4vcp2l6Dcv1LGc2q4aAfXS35J5vdg5M4NDaNh7fux/P7RgAAx9LoLkFUDW0m0lX3DU8BAFa47B0z4xHRM0T4P3haUURELll1SlVHU6azedmf364Zsx4R8YoIkJ+ciaeyZSdnRqZYjog8HhEgb1jdPzyF637worYLB8j/HR+/hAoRgqgWWk2YVdm2bikVkZQ4RUSbmilnVq0ARaSqCxGvds24lQAYCQfQN16+Wh8rtGbcnL03w7GLIvjdzkGMTqXhU4B1y5px9jHt+KsTFkk14UMQhFhaGsqnq6azORxko7suKyJmPCJTgjJEAL24yOZUZLI5BIqo+7KHmQFVXIhkc6oWIc47R2SuBUhuKSJm9s3IaFYFgI+fsRI1fj9WLKjHWUe3UfFBEFUKM6uWSlc9WBjdrQ360RENuXk0a4oI5wwRYGa7JZHJoaFoIVIwq5IiIh/GYsH98V22rlqsIjLXvpl0NqedRSazKpAvjG48/2ivj0EQhMe0zaGIGCdm3B5P1SLeTYzvilBEjA/RyXRW86wY0TbvSuwRkbdEEoyxEOFmVg3OPTWTzua0gBu3FJFSHhGWIaIoQKMEC+8IgiBm06rtmyleiHQXCpEVLvtDAEOyahnVmY3vivCI+HyK9iBdyidSCYqIvCcTTCprLET4VNHa1EyZHBFjL7FBeCFS3tGt7ZmpDVJAGEEQUsLGdyeTmaLTIcyo6kXaspldM2zhnYipGQAIB8pPzmgeEcoRkQ+jUZWXnGemNcOKgtqgn5sSU4q59s2w0V3Z2jIEQRCMSCigPSwWS1fVRndd3DHDaDATaMbMqgI8IsDcQZpubHt3SvUWIqxK5FgMsEIklc0hl1OLfk7MhXh3hllFRDajKkEQBENRFCxszO+Pmb32ATCEmXmgiDBVO57KasMPs5nSxncFKSIsS6SEJUAb3yVFRD5SnFNVAb0yBWa2foy4NTGTf43yUzMj8fzHmyXLECEIgjBy6vJWAMBze4dnfDyVyeHgKFNEvPOIAPp0zGz0iHdBisgcloBKGN+t+kKEZ3vEaAYq9aaYcCFVlTHX1IyM8e4EQRCzOX1VvhB5ds/MQuTg6BRyat4IuiDi7ugukL/ms7ZRqfYMm5qpE+URmUMRYa0ZMqtKCO8wMwAI+BQwz2epyRk3FRFtA2+yvEdEtjAzgiAII6cVCpFXD47N8LwZjapebJZVFEX3iZQwrBqX3okgHCi/gVc3q5IiIh0iWjOKosy5bybmUqoqMLdHZIRNzZAiQhCExCxuqsWKtnrkVOD5vSPax7uHvDOqMuYKNZsSGGgGGGMjaHy34tAUEc6TK3NlibiqiLCpmenyOSIt9eQRIQhCbpgq8uc9Q9rHvDSqMlioWenWjDuKyJzju1SIyIcIRQTQ/7ETc3pE3J2aUdUjHd0jNL5LEESFwHwizxl8ItqyOw8LkcicrRlxS+8APTG1lCKije9Sa0Y+tEKEtyKitWbmUkTcaM3kXyOTU4sWRmM0vksQRIVw2sp8IbKzfwJDhb0zXm3dNaK1ZuZSRASZVUNmA81IEZGPtACzKmAINSupiLjXmqmv8Wvm2WKhZqSIEARRKbQ2hHDswggAYOveYaQyORxiW3e99IgUCoxi+2ZUVdUUEWHju0HWminhEUmTWVVahLVm5jAOxVwc31UUpWSWSCabQ0xbeEceEYIg5Of0VW0A8mO8PYXR3foaPxY0uD+6yyiniCQzObCcM1Hju3qid4nxXTKryktSlFnVdGvGncXH7HVis35IxqZp4R1BEJWF0Seib931ZnSXoXtEjlSd4waVpFaQIhE2qYiQR0RCRJtVSykibppVAX1MePbkDMsQaawNIiB45w1BEAQPTl3ZAp+S37jLws28SFQ1Ui5HZKqQIVIb9AtbLKoFmpX0iJAiIi3MI8J78ZxZj4gbOSJA6SwR8ocQBFFpRMNBnNDZBAD46YsHAXjrDwH01kyxvCbNHyIoQwTAnNlVZFaVGHGKiGytGeYRmflDMjpFe2YIgqg8WHuG5SAt83B0FyiviLDNu6IW3gFzKyIJas3IS0pQlVjOrJrO5jBdeLO4YVYFgGgt84jMas3QnhmCICoQVogwvG7NRMqYVacEZ4gAZhQRas1Ii4hdM0B5j4jxjeq2R2T21MwoZYgQBFGBrF/WMmPIwMswM8CQrFpGERGVIQKY8YjQ+K606Nt3+RqIQmUWELH2SDjo4+5NKUUpjwgtvCMIohKprfHjnUubAOTbIm0N3l7DynlE3FBEtGTVIr7EXE7V7nVhUkTkQ981w/cNUk4RcTNDhFFqamYknv99E3lECIKoMFieyLLWOk9Hd4E5PCKCN+8C5XNE2H0OIEVESkQHmhWTyfSJGXfaMkBpRYTFu7eQR4QgiArjw+uWYPWiKK7YsNTro+gekeSRO72mkmI37wLlc0SMKonMHhH37oiSIaoQYeFgY0U23k54oIiUmpoZIY8IQRAVSmdzHX5z45leHwOArohkCzu9ag1tGDcUkXCZje8sVdWnAAFBOSY8kLdEEoyoQqQjGgYADMaSR/yZ26O7QJmpGcoRIQiCcExdjR+sOzQxK13VDUWE+RLLKSLhoN/zFlY5qrcQ0TwifP9xFkTyOw8GJhJH/BlTRNwKMwPmzhFpqSePCEEQhF0URdF9IrOus24qIokiikgljO4CVVyIiNq+K5siou+a0Sv1TDaH8WlmViVFhCAIwgmREoZVV3NEiikiWl6WvEZVoIoLEfYPxHtqpr2giEwmMzMWHgH6mmhXWzNhfca9Z2QKALQiBACaaOEdQRCEI0pt4HUjRyRkUERmm2XZ0ARTTWRF7tMJRJRHpCEU0KrfwYmZqogXZtW2hhqc1NkIVQU+9t1tGJ9Oa2FmtPCOIAjCOaw1M+GBIsKmZlQVSGdnFiKkiEiOqEJEURRDe2amTyQ27b4ioigKvn3VeiyMhrF7cBL/38PbtbYR7ZkhCIJwTgNTnj3wiBj9H7N9IppHhBQROUllxSSrAkbD6kxFxItAMwBY2BjGd64+BfU1fjy7Zxifefw1ADS6SxAEwYOSHhEXpmZq/D5tamd2fhWbpCGzqqQws6qIf6BSiogXZlXGmsVRfPPKk+H3Kdg7FAdAYWYEQRA8KJWuOuWCIqIoip6uOsuwyhQRmTfvAlVciKQEmVUB3bBa2iPiTY7c2ce04wsfOE77PU3MEARBOKfUvpl4wSNSL1ARAQz7Zma3ZkgRkRtRHhEA6IgWWjMlFBE3c0Rmc+WGZfjk2asAAGu7Gj07B0EQxHxBV0RmB5rlC4M6gYoIAIRLhJpVilmVIt6FtmZmKyLetWaM3HLRsfiHM1agrSHk6TkIgiDmA5Ei47upTE7zIopszQC6GXW2IsI8I2RWlZSkK2ZVXRFJZ3OYLrwp3DarFoOKEIIgCD4U84hMp/SioFbg+C5Q+YpIVRYiqqoKS1YFdEXksEERMVbKXisiBEEQBD+KeUSYP6TG7xNynzFSavEdRbxLTCanggXQhQSaVScM6arsDRoO+hCkEDGCIIh5QzFFRAszE2xUBUovvtPGd6k1Ix/MHwKIUUSKpat6lSFCEARBiEXziBgKES3eXbA/BDDEvKeLKyJhas3Ih+hCpFi6qixGVYIgCIIvDaEjk1XjLsS7M7TFd5lZHhFSROSF+UN8CuD38TerAkemq3qxZ4YgCIIQj+YRMbZm2OiuwIV3jHAJRSRRIWbVqnw8Twoc3WWUUkSipIgQBEHMK5hHJJXJIZnJ4sX9Y7jnd28DAOpdUET0QLPZikhlbN+tyrsim+2uEWganZ2u6nWqKkEQBCGGBoPq8Xf/93n8pXsEQL4tc/mpS4W/PpuKOdIjQoqItOhhZuL+cWanq2oekRC1ZgiCIOYTfp+Cuho/plJZ/KV7BAGfgis2LMUN5x6ttelFwhSRI3NEKmN8t7oLEQFhZozZ6aqsd0iKCEEQxPzjqPYG7Dg4jvedtBj/9J53YHlbvWuvrS29OyJZNX+vk33pXVXeFUWGmTFmp6uSWZUgCGL+8v1rNiA2nUZXS53rr11aEamMpXdVWYiI3DPDmJ2uGmNm1dqq/CsnCIKY1zTWBtFY682DJiWrViBJFxQRVohMJDOYSmUMOSKkiBAEQRD80HJEZisiWo4ItWak451dTfjR/z5NaJXI0lWnUlkMxpI0NUMQBEEIoVSOiJasSuO78tFUV4NTV7QIf52OaBjdQ3EMxBKITVMhQhAEQfBnzmRVycd3XSmTkskk1q5dC0VR8PLLL7vxklJgTFfVA82oNUMQBEHwo7QiUhlmVVdOd8stt2Dx4sVuvJRUGNNVadcMQRAEIYJQkWTVbE7VwjtlH98VXoj85je/wZYtW3D33XeLfinp6CgoIr1jCUwXKlUyqxIEQRA8KZasalzuKrsiIvTxfGBgAJ/4xCfw+OOPo65u7tnqZDKJZDKp/T4Wi4k8nnDaC+mqew5Pah8jRYQgCILgiZYjYhjfNY7yyl6ICDudqqq4+uqrce2112L9+vWmvmbz5s1obGzUfnV1dYk6niuw1szuwXwhEg76EBS434YgCIKoPsJFxndZuJnfpyAg+X3H8unuuOMOKIpS9tcLL7yAe++9F7FYDJs2bTL9vTdt2oTx8XHtV09Pj9XjSQUzqx4amwZAbRmCIAiCP6EiZlVtdFdyNQSw0Zq5/vrrcfnll5f9nOXLl+POO+/E1q1bEQrNXPizfv16XHnllXjooYeO+LpQKHTE51cyTBFhUFuGIAiC4I3emtEVEW1iRnKjKmCjEGlra0NbW9ucn/eNb3wDd955p/b73t5eXHjhhXjsscewYcMGqy9bkRxZiJAiQhAEQfCFeUBSmRxUVYWiKJo6Irs/BBBoVl26dOmM3zc0NAAAVq1ahc7OTlEvKxXGdFUAiJIiQhAEQXDGOJ47kcwgGg5qiojso7tAle6acROjKkKtGYIgCII3dUE/ljTVAgC++fvdAIypqvLf5l074fLly6GqKtauXevWS0oBM6wCQCRErRmCIAiCLz6fgi984DgAwH/8aS9e6RmrmM27ACkiwiFFhCAIghDNeas78IG1i5FTgVt+skNL85Z9zwxAhYhwOoyKCJlVCYIgCEHc/r7j0Fpfg10DE/jmH/ItmpDkm3cBKkSEw9JVAVJECIIgCHG01Nfg84UWzduFIE1SRAhqzRAEQRCu8d4TFuGCNR3a70kRIWaaVak1QxAEQQhEURTceenxWlxEmBQRwqiIUI4IQRAEIZr2aBj/50MnoL7Gj9NWtXp9nDmhO6NgZrZmSBEhCIIgxHPJiYtx8fGL4PcpXh9lTkgREUxDKIDW+hoAM9s0BEEQBCGSSihCAFJEXOFbV63DYCyJhY3huT+ZIAiCIKoIKkRc4JTlLV4fgSAIgiCkhFozBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4BhUiBEEQBEF4htTbd1VVBQDEYjGPT0IQBEEQhFnYfZvdx8shdSEyMTEBAOjq6vL4JARBEARBWGViYgKNjY1lP0dRzZQrHpHL5dDb24tIJAJFUbw+jufEYjF0dXWhp6cH0WjU6+PMW+jv2R3o79kd6O/ZPejvWkdVVUxMTGDx4sXw+cq7QKRWRHw+Hzo7O70+hnREo9Gqf5O7Af09uwP9PbsD/T27B/1d55lLCWGQWZUgCIIgCM+gQoQgCIIgCM+gQqSCCIVCuP322xEKhbw+yryG/p7dgf6e3YH+nt2D/q7tIbVZlSAIgiCI+Q0pIgRBEARBeAYVIgRBEARBeAYVIgRBEARBeAYVIgRBEARBeAYVIhVOMpnE2rVroSgKXn75Za+PM6/Yt28frrnmGqxYsQK1tbVYtWoVbr/9dqRSKa+PNi+47777sGLFCoTDYaxbtw5/+tOfvD7SvGLz5s045ZRTEIlE0N7ejksvvRS7du3y+ljzns2bN0NRFNx0001eH6VioEKkwrnllluwePFir48xL9m5cydyuRy+/e1v4/XXX8fXvvY1fOtb38Ktt97q9dEqnsceeww33XQTbrvtNrz00ks488wzcfHFF+PAgQNeH23e8Mwzz+C6667D1q1b8dRTTyGTyeCCCy5APB73+mjzlm3btuGBBx7AiSee6PVRKgoa361gfvOb3+Dmm2/GT3/6Uxx33HF46aWXsHbtWq+PNa+56667cP/992Pv3r1eH6Wi2bBhA04++WTcf//92sdWr16NSy+9FJs3b/bwZPOXw4cPo729Hc888wzOOussr48z75icnMTJJ5+M++67D3feeSfWrl2Lr3/9614fqyIgRaRCGRgYwCc+8Ql8//vfR11dndfHqRrGx8fR0tLi9TEqmlQqhe3bt+OCCy6Y8fELLrgAzz77rEenmv+Mj48DAL1/BXHdddfhve99L84//3yvj1JxSL30jiiOqqq4+uqrce2112L9+vXYt2+f10eqCvbs2YN7770XX/nKV7w+SkUzNDSEbDaLjo6OGR/v6OhAf3+/R6ea36iqiptvvhlnnHEGjj/+eK+PM+949NFH8eKLL2Lbtm1eH6UiIUVEIu644w4oilL21wsvvIB7770XsVgMmzZt8vrIFYnZv2cjvb29uOiii3DZZZfh4x//uEcnn18oijLj96qqHvExgg/XX389duzYgUceecTro8w7enp6cOONN+Lhhx9GOBz2+jgVCXlEJGJoaAhDQ0NlP2f58uW4/PLL8atf/WrGRTubzcLv9+PKK6/EQw89JPqoFY3Zv2d2Uent7cU555yDDRs24Lvf/S58PqrfnZBKpVBXV4cf//jH+OAHP6h9/MYbb8TLL7+MZ555xsPTzT9uuOEGPP744/jjH/+IFStWeH2cecfjjz+OD37wg/D7/drHstksFEWBz+dDMpmc8WfEkVAhUoEcOHAAsVhM+31vby8uvPBC/OQnP8GGDRvQ2dnp4enmF4cOHcI555yDdevW4eGHH6YLCic2bNiAdevW4b777tM+tmbNGnzgAx8gsyonVFXFDTfcgJ///Od4+umncfTRR3t9pHnJxMQE9u/fP+NjH/vYx3Dsscfi05/+NLXCTEAekQpk6dKlM37f0NAAAFi1ahUVIRzp7e3F2WefjaVLl+Luu+/G4cOHtT9buHChhyerfG6++WZcddVVWL9+PU477TQ88MADOHDgAK699lqvjzZvuO666/DDH/4Qv/jFLxCJRDT/TWNjI2praz0+3fwhEokcUWzU19ejtbWVihCTUCFCECXYsmULdu/ejd27dx9R4JGQ6IyPfOQjGB4exhe+8AX09fXh+OOPxxNPPIFly5Z5fbR5AxuNPvvss2d8/MEHH8TVV1/t/oEIogTUmiEIgiAIwjPIdUcQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGdQIUIQBEEQhGf8P6ZyOHzGZunmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/exercisesweek41_41_2.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "backend = np\n",
    "\n",
    "def function(x):\n",
    "    return x*backend.sin(x**2 + 1)\n",
    "\n",
    "def analytical_gradient(x):\n",
    "    return backend.sin(x**2 + 1) + 2*x**2*backend.cos(x**2 + 1)\n",
    "\n",
    "\n",
    "x = np.linspace(-5, 5, 100)\n",
    "plt.plot(x, function(x), label=\"f(x)\")\n",
    "\n",
    "descent_x, descent_y = gradient_descent(1, 0.01, 300, solver=\"analytical\")\n",
    "\n",
    "# Change the backend to JAX\n",
    "backend = jnp\n",
    "jax_descend_x, jax_descend_y = gradient_descent(1, 0.01, 300, solver=\"jax\")\n",
    "\n",
    "plt.scatter(descent_x, descent_y, label=\"Gradient descent\", marker=\"v\", s=10, color=\"red\") \n",
    "plt.scatter(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\", s=5, color=\"black\")"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}