Files
FYS-STK4155/doc/LectureNotes/_build/jupyter_execute/exercisesweek41.ipynb
T
Morten Hjorth-Jensen 4f275a4832 update jupyter-book
2024-11-12 09:48:10 +01:00

1376 lines
202 KiB
Plaintext

{
"cells": [
{
"cell_type": "markdown",
"id": "b18bcd06",
"metadata": {},
"source": [
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
"doconce format html exercisesweek41.do.txt -->\n",
"<!-- dom:TITLE: Exercises week 41 -->"
]
},
{
"cell_type": "markdown",
"id": "7542d6aa",
"metadata": {},
"source": [
"# Exercises week 41\n",
"**October 4-11, 2024**\n",
"\n",
"Date: **Deadline is Friday October 11 at midnight**"
]
},
{
"cell_type": "markdown",
"id": "80943a15",
"metadata": {},
"source": [
"# Overarching aims of the exercises this week\n",
"\n",
"The aim of the exercises this week is to get started with implementing\n",
"gradient methods of relevance for project 2. This exercise will also\n",
"be continued next week with the addition of automatic differentation.\n",
"Everything you develop here will be used in project 2.\n",
"\n",
"In order to get started, we will now replace in our standard ordinary\n",
"least squares (OLS) and Ridge regression codes (from project 1) the\n",
"matrix inversion algorithm with our own gradient descent (GD) and SGD\n",
"codes. You can use the Franke function or the terrain data from\n",
"project 1. **However, we recommend using a simpler function like**\n",
"$f(x)=a_0+a_1x+a_2x^2$ or higher-order one-dimensional polynomials.\n",
"You can obviously test your final codes against for example the Franke\n",
"function. Automatic differentiation will be discussed next week.\n",
"\n",
"You should include in your analysis of the GD and SGD codes the following elements\n",
"1. A plain gradient descent with a fixed learning rate (you will need to tune it) using the analytical expression of the gradients\n",
"\n",
"2. Add momentum to the plain GD code and compare convergence with a fixed learning rate (you may need to tune the learning rate), again using the analytical expression of the gradients.\n",
"\n",
"3. Repeat these steps for stochastic gradient descent with mini batches and a given number of epochs. Use a tunable learning rate as discussed in the lectures from week 39. Discuss the results as functions of the various parameters (size of batches, number of epochs etc)\n",
"\n",
"4. Implement the Adagrad method in order to tune the learning rate. Do this with and without momentum for plain gradient descent and SGD.\n",
"\n",
"5. Add RMSprop and Adam to your library of methods for tuning the learning rate.\n",
"\n",
"The lecture notes from weeks 39 and 40 contain more information and code examples. Feel free to use these examples.\n",
"\n",
"In summary, you should \n",
"perform an analysis of the results for OLS and Ridge regression as\n",
"function of the chosen learning rates, the number of mini-batches and\n",
"epochs as well as algorithm for scaling the learning rate. You can\n",
"also compare your own results with those that can be obtained using\n",
"for example **Scikit-Learn**'s various SGD options. Discuss your\n",
"results. For Ridge regression you need now to study the results as functions of the hyper-parameter $\\lambda$ and \n",
"the learning rate $\\eta$. Discuss your results.\n",
"\n",
"You will need your SGD code for the setup of the Neural Network and\n",
"Logistic Regression codes. You will find the Python [Seaborn\n",
"package](https://seaborn.pydata.org/generated/seaborn.heatmap.html)\n",
"useful when plotting the results as function of the learning rate\n",
"$\\eta$ and the hyper-parameter $\\lambda$ when you use Ridge\n",
"regression.\n",
"\n",
"We recommend reading chapter 8 on optimization from the textbook of [Goodfellow, Bengio and Courville](https://www.deeplearningbook.org/). This chapter contains many useful insights and discussions on the optimization part of machine learning."
]
},
{
"cell_type": "markdown",
"id": "2095d197",
"metadata": {},
"source": [
"# Code examples from week 39 and 40"
]
},
{
"cell_type": "markdown",
"id": "f428decb",
"metadata": {},
"source": [
"## Code with a Number of Minibatches which varies, analytical gradient\n",
"\n",
"In the code here we vary the number of mini-batches."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "ba38d454",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Own inversion\n",
"[[3.86119759]\n",
" [3.07853436]]\n",
"Eigenvalues of Hessian Matrix:[0.35325267 4.24060666]\n",
"theta from own gd\n",
"[[3.86119759]\n",
" [3.07853436]]\n",
"theta from own sdg\n",
"[[3.90968018]\n",
" [3.12597955]]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHHCAYAAAC88FzIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABZ4ElEQVR4nO3deVyU1f4H8M8AMiAKisgmo+Iu4JKaW5qYqJkKZqZpllpa3bRyafN2y7y3rlqaWlnaJvUzLFOUycolA7UyLZerYioaKi6IKygKwsz5/THNyMAMszDLM/N83q8XL+VZz5kZeL6c5XsUQggBIiIiIpnycXcBiIiIiNyJwRARERHJGoMhIiIikjUGQ0RERCRrDIaIiIhI1hgMERERkawxGCIiIiJZYzBEREREssZgiIiIiGSNwRARWW38+PFo2rSpu4shaVlZWVAoFFi9erW7i0JEVmIwRCRBqampUCgUhi8/Pz80atQI48ePx5kzZ9xdPCIir+Ln7gIQkXn//ve/ERsbi5KSEvz2229ITU3Fzz//jIMHDyIgIMDdxSMi8goMhogkbNCgQejSpQsAYOLEiQgLC8O8efOgVqsxcuRIN5eO3Km4uBhBQUHuLgaRV2A3GZEH6d27NwDg+PHjhm23bt3Ca6+9hs6dOyMkJARBQUHo3bs3MjMzjc49ceIEFAoF5s+fj48++gjNmzeHUqnEnXfeid9//73KvdatW4eEhAQEBAQgISEBa9euNVmm4uJizJgxAyqVCkqlEq1bt8b8+fMhhDA6TqFQYMqUKfjmm28QFxeHwMBA9OjRAwcOHAAALFu2DC1atEBAQAASExNx4sQJi6/H66+/DoVCgWPHjmH8+PGoV68eQkJCMGHCBNy4caNK3VNTU6tcQ6FQ4PXXX69yzaNHj2Ls2LEICQlBw4YN8eqrr0IIgby8PKSkpCA4OBiRkZFYsGCBybJpNBr885//RGRkJIKCgpCcnIy8vLwqx+3cuRP33nsvQkJCULt2bfTp0we//PKLyXoeOnQIY8aMQf369dGrVy8AQH5+PiZMmICYmBgolUpERUUhJSXFqtePiHTYMkTkQfQPuPr16xu2FRUV4ZNPPsHo0aMxadIkXLt2DZ9++ikGDhyIXbt2oWPHjkbXSEtLw7Vr1/Dkk09CoVDgrbfewvDhw/HXX3+hVq1aAIBNmzbhgQceQFxcHObMmYNLly4ZHrgVCSGQnJyMzMxMPP744+jYsSM2btyIF154AWfOnMHChQuNjt++fTvUajUmT54MAJgzZw6GDBmCF198ER988AGefvppXLlyBW+99RYee+wx/PTTT1a9LiNHjkRsbCzmzJmDPXv24JNPPkF4eDjmzZtny8trZNSoUWjbti3mzp2L7777Dm+88QZCQ0OxbNky3HPPPZg3bx6+/PJLPP/887jzzjtx9913G53/5ptvQqFQ4KWXXkJBQQEWLVqEpKQk7Nu3D4GBgQCAn376CYMGDULnzp0xa9Ys+Pj4YPny5bjnnnuwfft2dO3a1eiaDz74IFq2bIn//ve/hmDzgQceQHZ2Np555hk0bdoUBQUF2Lx5M06dOsXB7kTWEkQkOcuXLxcAxI8//iguXLgg8vLyxOrVq0XDhg2FUqkUeXl5hmPLy8tFaWmp0flXrlwRERER4rHHHjNsy83NFQBEgwYNxOXLlw3bMzIyBADx7bffGrZ17NhRREVFiatXrxq2bdq0SQAQTZo0MWxbt26dACDeeOMNo/uPGDFCKBQKcezYMcM2AEKpVIrc3FzDtmXLlgkAIjIyUhQVFRm2z5w5UwAwOtaUWbNmCQBG9RRCiPvvv180aNCgSt2XL19e5RoAxKxZs6pc84knnjBsKy8vFzExMUKhUIi5c+catl+5ckUEBgaKcePGGbZlZmYKAKJRo0ZGdVq1apUAIBYvXiyEEEKr1YqWLVuKgQMHCq1Wazjuxo0bIjY2VvTv379KmUaPHm1U9itXrggA4u233672dSKi6rGbjEjCkpKS0LBhQ6hUKowYMQJBQUFQq9VGLTS+vr7w9/cHAGi1Wly+fBnl5eXo0qUL9uzZU+Wao0aNMmpZ0ne9/fXXXwCAc+fOYd++fRg3bhxCQkIMx/Xv3x9xcXFG1/r+++/h6+uLZ5991mj7jBkzIITADz/8YLS9X79+Rq0V3bp1A6Br3ahbt26V7foyWfLUU08Zfd+7d29cunQJRUVFVp1vysSJEw3/9/X1RZcuXSCEwOOPP27YXq9ePbRu3dpkOR999FGjOo0YMQJRUVH4/vvvAQD79u1DTk4OxowZg0uXLuHixYu4ePEiiouL0a9fP2zbtg1arbbaegYGBsLf3x9ZWVm4cuWK3XUlkjt2kxFJ2JIlS9CqVSsUFhbis88+w7Zt26BUKqsc9/nnn2PBggU4fPgwysrKDNtjY2OrHNu4cWOj7/WBkf5hevLkSQBAy5Ytq5zbunVrowDr5MmTiI6ONnroA0Dbtm2NrmXu3vpgS6VSmdxu7QO+ujoFBwdbdQ1L1wwJCUFAQADCwsKqbL906VKV8yu/fgqFAi1atDB0debk5AAAxo0bZ7YMhYWFRoFr5fdTqVRi3rx5mDFjBiIiItC9e3cMGTIEjz76KCIjIy1XkogAMBgikrSuXbsaZpMNGzYMvXr1wpgxY3DkyBHUqVMHALBixQqMHz8ew4YNwwsvvIDw8HD4+vpizpw5RgOt9Xx9fU3eS1Qa8OwM5u5d0zJZOl+hUJjcr9FobLqmI187favP22+/XWVcl57+PdbTjzWqaOrUqRg6dCjWrVuHjRs34tVXX8WcOXPw008/4Y477rC5XERyxGCIyEPoA5y+ffvi/fffx8svvwwAWL16NZo1a4b09HSjh/6sWbPsuk+TJk0A3G65qOjIkSNVjv3xxx9x7do1o9ahw4cPG13L3fStK1evXjXaXrnlypEqv35CCBw7dgzt27cHADRv3hwAEBwcjKSkpBrdq3nz5pgxYwZmzJiBnJwcdOzYEQsWLMCKFStqdF0iueCYISIPkpiYiK5du2LRokUoKSkBcLu1omLrxM6dO7Fjxw677hEVFYWOHTvi888/R2FhoWH75s2bcejQIaNj77vvPmg0Grz//vtG2xcuXAiFQoFBgwbZVQZHCw4ORlhYGLZt22a0/YMPPnDaPb/44gtcu3bN8P3q1atx7tw5w2vSuXNnNG/eHPPnz8f169ernH/hwgWL97hx44bhc6DXvHlz1K1bF6WlpTWsAZF8sGWIyMO88MILePDBB5GamoqnnnoKQ4YMQXp6Ou6//34MHjwYubm5WLp0KeLi4kw+ZK0xZ84cDB48GL169cJjjz2Gy5cv47333kN8fLzRNYcOHYq+ffvilVdewYkTJ9ChQwds2rQJGRkZmDp1qqH1QwomTpyIuXPnYuLEiejSpQu2bduGo0ePOu1+oaGh6NWrFyZMmIDz589j0aJFaNGiBSZNmgQA8PHxwSeffIJBgwYhPj4eEyZMQKNGjXDmzBlkZmYiODgY3377bbX3OHr0KPr164eRI0ciLi4Ofn5+WLt2Lc6fP4+HHnrIaXUj8jYMhog8zPDhww0tCpMmTcL48eORn5+PZcuWYePGjYiLi8OKFSvwzTffICsry6573Hvvvfjmm2/wr3/9CzNnzkTz5s2xfPlyZGRkGF3Tx8cHarUar732Gr7++mssX74cTZs2xdtvv40ZM2Y4psIO8tprr+HChQtYvXo1Vq1ahUGDBuGHH35AeHi4U+73z3/+E/v378ecOXNw7do19OvXDx988AFq165tOCYxMRE7duzAf/7zH7z//vu4fv06IiMj0a1bNzz55JMW76FSqTB69Ghs2bIF//d//wc/Pz+0adMGq1atwgMPPOCUehF5I4VwxahJIiIiIonimCEiIiKSNQZDREREJGsMhoiIiEjWGAwRERGRrDEYIiIiIlljMERERESyJos8Q1qtFmfPnkXdunXNrlFERERE0iKEwLVr1xAdHQ0fH+e138giGDp79myVVbGJiIjIM+Tl5SEmJsZp15dFMKRfQDIvLw/BwcFuLg0RERFZo6ioCCqVymghaGeQRTCk7xoLDg5mMERERORhnD3EhQOoiYiISNYYDBEREZGsMRgiIiIiWWMwRERERLLGYIiIiIhkjcEQERERyRqDISIiIpI1BkNEREQkawyGiIiISNYYDBEREZGsMRgiIiIiWWMwRERERLLGYIiIiIhkjcEQERERyRqDISIiIpI1BkNEREQkawyGiIiISNbcHgxt27YNQ4cORXR0NBQKBdatW2f22KeeegoKhQKLFi1yWfmIiIjIu7k9GCouLkaHDh2wZMmSao9bu3YtfvvtN0RHR7uoZERERCQHfu4uwKBBgzBo0KBqjzlz5gyeeeYZbNy4EYMHD3ZRyYiIiEgO3N4yZIlWq8UjjzyCF154AfHx8e4uDhEREXkZt7cMWTJv3jz4+fnh2Weftfqc0tJSlJaWGr4vKipyRtGIiIjIC0i6ZWj37t1YvHgxUlNToVAorD5vzpw5CAkJMXypVConlpKIiIg8maSDoe3bt6OgoACNGzeGn58f/Pz8cPLkScyYMQNNmzY1e97MmTNRWFho+MrLy3NdoYmIiMijSLqb7JFHHkFSUpLRtoEDB+KRRx7BhAkTzJ6nVCqhVCqdXTwiIiLyAm4Phq5fv45jx44Zvs/NzcW+ffsQGhqKxo0bo0GDBkbH16pVC5GRkWjdurWri0pEREReyO3B0B9//IG+ffsavp8+fToAYNy4cUhNTXVTqYiIiEgu3B4MJSYmQghh9fEnTpxwXmGIiIhIdiQ9gJqIiIjI2RgMERERkawxGCIiIiJZYzBEREREssZgiIiIiGSNwRARERHJGoMhIiIikjUGQ0RERCRrDIaIiIhI1hgMERERkawxGCIiIiJZYzBEREREssZgiIiIiGSNwRARERHJGoMhIiIikjUGQ0RERCRrDIaIiIhI1hgMERERkawxGCIiIiJZYzBEREREssZgiIiIiGSNwRARERHJGoMhIiIikjUGQ0RERCRrDIaIiIhI1hgMERERkawxGCIiIiJZYzBEREREssZgiIiIiGSNwRARERHJGoMhIiIikjUGQ0RERCRrDIaIiIhI1hgMERERkawxGCIiIiJZYzBEREREssZgiIiIiGSNwRARERHJGoMhIiIikjW3B0Pbtm3D0KFDER0dDYVCgXXr1hn2lZWV4aWXXkK7du0QFBSE6OhoPProozh79qz7CkxERERexe3BUHFxMTp06IAlS5ZU2Xfjxg3s2bMHr776Kvbs2YP09HQcOXIEycnJbigpEREReSOFEEK4uxB6CoUCa9euxbBhw8we8/vvv6Nr1644efIkGjdubNV1i4qKEBISgsLCQgQHBzuotERERORMrnp++zntyk5SWFgIhUKBevXqmT2mtLQUpaWlhu+LiopcUDIiIiLyRG7vJrNFSUkJXnrpJYwePbraCHHOnDkICQkxfKlUKheWkoiIiDyJxwRDZWVlGDlyJIQQ+PDDD6s9dubMmSgsLDR85eXluaiURERE5Gk8optMHwidPHkSP/30k8V+Q6VSCaVS6aLSERERkVU0GmD7duDcOSAqCujdG/D1dXeppB8M6QOhnJwcZGZmokGDBu4uEhEREdkqPR147jng9Onb22JigMWLgeHD3VcuSCAYun79Oo4dO2b4Pjc3F/v27UNoaCiioqIwYsQI7NmzB+vXr4dGo0F+fj4AIDQ0FP7+/u4qNhEREVkrPR0YMQKoPIH9zBnd9tWr3RoQuX1qfVZWFvr27Vtl+7hx4/D6668jNjbW5HmZmZlITEy06h6cWk9EROQmGg3QtKlxi1BFCoWuhSg3t0qXmWym1icmJqK6eExCaZCIiIjIVtu3mw+EAF1rUV6e7jgrGzkczWNmkxEREZEHOnfOscc5AYMhIiIicp6oKMce5wQMhoiIiMh5evfWjQlSKEzvVygAlUp3nJswGCIiIiLn8fXVTZ8HqgZE+u8XLXJrviEGQ0RERORcw4frps83amS8PSbG7dPqAQnMJiMiIiIZGD4cSElhBmoiIiKSMV9ft02frw67yYiIiEjWGAwRERGRrDEYIiIiIlljMERERESyxmCIiIiIZI3BEBEREckagyEiIiKSNQZDREREJGsMhoiIiEjWGAwRERGRrDEYIiIiIlnj2mRERESeQqOR5EKnVpNo+RkMEREReYL0dOC554DTp29vi4kBFi/WrQgvdRIuP7vJiIiIpC49HRgxwjiQAIAzZ3Tb09PdUy5rSbz8CiGEcGsJXKCoqAghISEoLCxEcHCwu4tDRERkPY0GaNq0aiChp1DoWlhycyXR5VRFDcrvquc3W4aIiIikbPt284EEAAgB5OXpjpMiDyg/gyEiIiIpO3fOsce5mgeUn8EQERGRlIWHW3dcVJRzy2Eva8vlxvIzGCIiIpKq9HRg/Pjqj1EoAJVKN01dinr31o0JUihM75dA+RkMERERSZG5GVgV6QOMRYukOXga0JVr8WLd/ysHRBIpP4MhIiIiqdFodDl5LE34btQIWL3a7Xl6LBo+XFfORo2Mt8fESKL8TLpIREQkNZZmYOmlpgL9+jm9OA4xfDiQksIM1ERERGQFa2dWFRQ4txyO5usLJCa6uxRVsJuMiIhIajxgBpY3YcsQERE5h0QX5fQI+hlYZ86YHjekz9os1RlkHoYtQ0RE5Hjp6bolGPr2BcaM0f3btKnb16DyGB4wA8ubMBgiIiLHkviinB5D4jOwbKbRAFlZwMqVun81GneXyIALtRIRkeN4+qKiUuQN3Y3p6bpUARU/FzExutavaoI6Vz2/OWaIiIgcx5ZFOSU4q0iSJDoDy2r6lsLKbS/6lkIJtHKxm4yIiBzHAxblpEqc2X1VXfJI/bapU93eZcZgiIiIHIdTwj2Lswe629JS6EZuD4a2bduGoUOHIjo6GgqFAuvWrTPaL4TAa6+9hqioKAQGBiIpKQk5OTnuKSwREVXPAxblpL+5YqC7h7QUuj0YKi4uRocOHbBkyRKT+9966y28++67WLp0KXbu3ImgoCAMHDgQJSUlLi4pERFZxCnhnsFV3Vce0lLo9mBo0KBBeOONN3D//fdX2SeEwKJFi/Cvf/0LKSkpaN++Pb744gucPXu2SgsSERFJhLdNCfdGruq+qmFL4bl952t2fyu5PRiqTm5uLvLz85GUlGTYFhISgm7dumHHjh1mzystLUVRUZHRFxERudDw4cCJE0BmJpCWpvs3N5eBkFS4qvvKxpZCoRXYv/oo3kjKQtc62WjTJ6Jm97eSpKfW5+fnAwAiIoxfjIiICMM+U+bMmYPZs2c7tWxERGSBp08J92au6r7SaIDQUF2X3JdfAhcu3N4XEwMsWoSye4di29t7oP7yGtTZzXCivBWAVn8f5JrGDEkHQ/aaOXMmpk+fbvi+qKgIKpXKjSUiIiKSEFesfWYq0WJYGDB2LK5364fvfg5BxnQffP/ADRSik+GQANxE/4j9SB54C32ejEaru+wvgrUkHQxFRkYCAM6fP4+oCtHp+fPn0bFjR7PnKZVKKJVKZxePiIjIM+m7r0aM0AU+FQMiRwx0N5NoUVy8CCxahAnogdUYAgDwgQYpWIdBkXsRf08E7pg/FkFR3QDAZcNcJD1mKDY2FpGRkdiyZYthW1FREXbu3IkePXq4sWREREROZE8iRFvPcdZA979nqpla7UsBQECBBXge8bWO4ouW/8aNeo2wDvfjyfx/o1faZAR1jXf5+nVubxm6fv06jh07Zvg+NzcX+/btQ2hoKBo3boypU6fijTfeQMuWLREbG4tXX30V0dHRGDZsmPsKTURE3s9da4LZs46XnWt/YfhwICWlaj0BXUBlY91Lrpbgf9NS0e30aZiZPwYfCDRGHg7+6yvg9derX6ajwgQqpxJulpmZKQBU+Ro3bpwQQgitViteffVVERERIZRKpejXr584cuSITfcoLCwUAERhYaETakBERF5nzRohYmKE0D2qdV8xMbrtzr6vQmF8X+D2ttmzhUhLEyIzU4jycsvnKBRCfPON7vjK5zmo7hcOXxSfT9ouHmj0q6iDIvEQ0qqWxdRXaKj5fQqFECqVKLx82SXPb65aT0REVJG5hUX1Y2mclStJo9EthVFd/p+KYmKAhQuBadOqP8fX17jLrLoWIyvrnrP5BNTvnoB6ez38XNgOWtxuNXpAsQarxQjr6mBB0fr1CBkyxOnPbwZDREREepYCEv0sq9xcx3eZZWXp1gazVuWBz7acB1QN6izUXUCBQv8w9BI/I7usldG+DgFHkHznOSRPDEfn0a2gaBZb/Uy10FDg0iWLRS365BOETJzo9Oe3pAdQExERuZQ7Fxa1NcGhvW0Z5pbbsFB3BQTq3bqAsLKz8EMZkkJ3490HtiJ3+2nsu9ka/96WiC6PxkFRy89yosVnn7WurH/PKnc2BkNERER67lxY1JXrc5kK6qys038HbsXFkzew+VJnPLO6D5r2iql6kKWZaq+8Yt0yHT17WlmhmnH7bDIiIiLJcOfCopYSITqBOHMWhzKOQf3BaeRtu4kPrDin58t9gMYhlg80N1NN373ozDxHNuKYISIicix3TUl3BP24GUuZmZ0xZgi4PYAZsD0gsmMM0cM+K5GmfQiALvnhCTRFI5yBDxxY9+o+D6ZSAqhUukBo+HCXPb/ZTUZERLapLrlferoumOjbFxgzRvdv06YuT6JnNxsXFnU4c91L5ui7k1atqnpONWXUQoFTUOEr7YNQogT3NfwdH4z5BUFvvw4fBRxXd0ufB4ks6MuWISIisl51yf0A90xJdwYLLRZOV7E1JSfHdHLCyq9rpRaYi78eRYNXnoSAccuH9u90iEsi/oNGj/bDgOkJqBNZ5/YBjqq7A1IUuOr5zWCIiIisY+nhVt10aWd3LzmDVLr70tOBJ56o+to2aAB89JEhoBBagX1fH4H6o3xk7IzA3pttcT/SsRjPQYXbgU1JvQjUWvY+fEdWkwuopnV3UIoCBkMOxGCIiKiGbE0IaE5mJpCY6IgSyYO5APRvZSu+Ruap5lCvvA71oRbI09zuKlNAi551DyLlrgsYlXgejRsrXBfUWZszycLnwVXPb84mIyIiyyzl37GWM6ake6u/Fzw1FwhpAZwfOx2DcNKQAbo2ijEg6gCS7y3D4BltEB7f3oUFrsCdKQrswGCIiIgsc9RDy5W5dDydhQDUB0AMziBFkYGw1mFIfjAA/aa2Q2Bod+vvYao7TH/vmnQPujNFgR0YDBERkWU1fWjpx4joH7ZULW25Fn+l7UILK45d/cVN+Iy92/abmBoo3aCB7t+K45OqW8vMHEs5kyT2eeDUeiIiskz/cKsuY7D+QeqOKele4OaF69jx6If4uNFrGOG/Dk983Nmq83xirJyGX5F+LFLllqdLl6oO1D5zRnesLekR3J2iwEYcQE1ERNYxlxCw4lRpwL1T0j1MQfYFfLfgMM6v/QVjr76PGJwx7DuNKIT4FKOOtggmQ9CaJEG0dTC8vfeq4TR9ziZzIAZDREQOYs3DTSpT0iVIaAWObMiF+v1TyPg5FDuuJWAY1mE1RgAQRt01QqGAQv+INrdkhT25m6yd6WWKPbMBq/s8WPiscDYZETkGH0zkSJbWmwJ0/+f0eYPyknLs+CQbGalXoN7fBDllzQA0A6BbAuNDPA0FRJXWH4UQuqAnNBQIDKya6NLe1raaDIa351xzn4fqEni6uBWRwRCRN5PQLxvyIgx2qqfR4GbGJhxY/ju27K6PhedG4QI6GHbXwi3c02A/UvrfwAO9CxA++bz5awmhG8Pz44+6190Rf9TUZDC8o2Z/mcufpB+f5OJs5ewmI/JWDkiFT0TWO7snHwenfoI7fnkPDbUFhu15iMEreBMithlS7vfFgGnxCI75+1m0cqVuzS5L0tKA0aMdU1BLi9Ga4sgM4jZkpy4qLuZCrURkp+qStem3TZ1qvMAmEdlEaAX2rz6KN5KycGfQIUzp/CuStr+GBhUCIUCXC+hzxXj83/wCjFjQ43YgBLgnH091M71McfTsL0sJPIUA8vJ0x7kIgyEibyTBXzZE3qDsRhm2vL0Hz3XcimbK0+jwYCu8uiURe260xmI8h8qDoAHcHg9k6g8Qa1IWqFSOz8czfLiudbjySvcNGtxOkaAXE+PYlmQJZqfmmCEibyTBXzZELuegyQNXTxZiw4JsqNUC359MQCE6GfYF4Cb6R+zHkx13QrXRyj9AKo630rfSjBhhfsaYs/LxmBsMDzh30oUEs1MzGCLyRhL8ZUPkUvZOHvg7gLqQdRDbM8uxdP9dyLzaEeXoaTikoeIChrY4jJQH/ZE0rR1qh3UDVv4FbLSiXKb+ANG30lQub1gY8PDDutlkGo1zAiJzg+GdOUDeluzUxcXOK0fFW3IANZEXsjRA0pGDIYmkxo7JA9pyLf6a/h4aLnsTIbcuGLbnIQbPYTEO+3dASqc8JE9ogK7j4+DrX+nnxhGrtOtbsjIygC+/BC7cLofXzQK1JoHn8OFMuuhIDIZIlqz8ZUPkVWyYqVRyrQyZ7x6A+uubKPszBx+JSaiS+FB3EhRrLPy8OOoPEDnNArUigSeDIQdiMESyVcNU+EQex8oWmtcaLME7lx5FMerABxqcQFM0wmnTs4psDWQA+/4AsSGQ85oWXYlkoOZsMiJvNnw4cOKErmk+LU33b24uAyHyXlZOCsi5VB/FqINon3N4u8n7UJkLhADrZ1+am6HVqBHw+utAaakuWDOX0kKOs0D1Y5ZGj9b966YgjwOoibwdswWTp6nJLDArJwX0b3cez7/4JzqNaQPF1+GAFXkPrQq0Ks/QyskBPv4YmDXr9jHmxv9wFqjbsGWIiIikIz1d11XUt68uM3Pfvrrv09OrPa24oBgZ/9yJxycBZxANrel13nUZf1QqPLb3GXQe2xYKH4XjZ1/q/wBRKnUtQpVbe/RLTlSuE2eBug3HDBGR5+His97JxsHD+fsLsH7BEWRsUOLHgnYoQSAA4H6km1wF3uzYHWfMvrRn/A9ngVbBMUNERKbY2XJAEmfFEjJi6lRkpx/BnIFZ6F7nIKI6hGPSF72xvqArShCIpn55eLbDVkyZHwtt2lfwiYkxvo65TMrVLU9hb+JDe8b/OKMcZBWOGSIizyGxla7JgawIHhR5eZj8wDlsRaJh851B2UjufgEp/4hGwv0tofBR/b3nDmDkA9a3IJpLfBgTY9/sS3vH/zi6HGQVdpMRkWeQ47RjObFy9fZHkYpLDeOQMuAmhkxvhehOkY4th6O6YGuahFEOXcFW1NFVz2+2DBGRZ7Cl24Gz5zxOwZVaCLfiuGVrwhE4/E7nFcRRsy9tWXLCmeWQKnuXS3ESjhkiIs/AacdeRWgF9q48jNcTs9Cp9p+Imnw/8hBjdhaYfvX2wJQBri2ovTj+xzx9d7e1s+xcgMEQEXkGTjv2eKVFpdg0Zzcmt9uKxv7n0GlMG8zemoi9N9tCQIEPA6ZBAaGb/l6RpwYP5pIwmhvILQdWDJTH1KnmE1M6CccMEZFn4LRjj3T5+BX8sOAQMr71wYbT8biG27+Da6MYA6IOIOW+cgye0QYN24Z55xIychj/Yy0bx1JxzBARUUX6bocRI3SBj6m1nzyt5cBL/ZV1ChmL/oJ6awi2X20HDe4y7Iv0OY/kVkeQPCoQ9zybgMDQ7sYnV87g7A3Bg7eP/7GFRLu7GQwRkefgtGNJ0pZr8fvnh5Dx6UWo9zRCdmlLAI0N+xOUOUjpcgbJjzdEl0fawscvovoLMnjwXhLt7pZ8N5lGo8Hrr7+OFStWID8/H9HR0Rg/fjz+9a9/QVF5UJoZ7CYj8jLsdnC7m5dv4seFB6D+pgTf5rTBee3tuWC+KMfd9Q4gpW8Rhj4bi2aJjau5EsmKjd3d7Cb727x58/Dhhx/i888/R3x8PP744w9MmDABISEhePbZZ91dPCJyB7YcuEVB9gWsn38Y6g3+2JTfDjfR1bAvGIUYpMpG8lCBQc/Ho37sHW4sKUmWRLu7JR8M/frrr0hJScHgwYMBAE2bNsXKlSuxa9cuN5eMiMi7Ca3AkQ25yHjvFNS/hGLHtQQI3M6Lo/I9g5S4Y0geUwd9prSDf52ebiwteQwJdndLPhjq2bMnPvroIxw9ehStWrXC//73P/z8889455133F00IiKvU15Sjh2fZCMj9QrU+5sgp6wZgGaG/Z0C/0RK9/NIfjIKHR5sBYVPI/MXcyZ2lXo2iQ2Ul3ww9PLLL6OoqAht2rSBr68vNBoN3nzzTTz88MNmzyktLUVpaanh+6KiIlcUlYjII13Pv45N7xxExppyfJfbFpdEB8M+f5TinrD9SE66iaHTWyLmzrYA2rqvsIDksheTnSTU3S35YGjVqlX48ssvkZaWhvj4eOzbtw9Tp05FdHQ0xo0bZ/KcOXPmYPbs2S4uKRGR5zi7Jx/fLjiKjM2B2HKhPW7h9hT3UMVlDI49hORhvhgwLR7BMU5c/sJWXKyXnEDys8lUKhVefvllTJ482bDtjTfewIoVK3D48GGT55hqGVKpVJxNRkSyJbQCB9JzkPHhWah/C8cfN+KM9jf3O4mUDrlIfqQe7noyAX4BEvxbmYv1yg5nk/3txo0b8PExXjXE19cXWq3W7DlKpRJKpdLZRSMikrSyG2XYtuQAMlZcgzq7OU5qWgFoBQBQQIvudbKRfNclJD8dg7ZDmkPh08S9BbaEi/WSk0g+GBo6dCjefPNNNG7cGPHx8di7dy/eeecdPPbYY+4uGhGR5Fw9WYgNC7KRkQH8cCoehehk2BeIG+gfeQDJA29h8PTWiGzfzo0ltYNEsxeT55N8MPTee+/h1VdfxdNPP42CggJER0fjySefxGuvvebuohERScKJn09DvfA41Jl1sfVKO5Tj9hT3cMUFDG15GMkj/JE0rR1qh3VzY0lrSKLZi8nzSX7MkCMwAzV5PU4zlhVtuRZ70g4j4+MCqP+Iwv6S1kb745THkHzHaSRPaICu4+Pg6+8lnwUu1is7HDNERNbhNGNZKLlagsx3DyDjq5v49khLnNXGAdANgvaBBr1DDiC5z1UMndIULfu3ANDCreV1ColmLybPx5YhIk9mbpqx/sHAacYe3Wp28cglfDf/T6i/98PGswkoRh3Dvjq4hnsbHUTyEA3umxGHBi1D3VhSFzP1B4BKxcV6vZCrnt82BUN5eXlQqVROK4yzMBgih5HSg5XTjC2TaqtZNZ+jnM0nkLH4BNTb6+OXogRocfu9a+RzDsltc5D8UG30fbYdlMEynjUrpZ9FchpJBkNBQUGYMWMGXn75ZdSuXdtphXI0BkPkEFJ7sGZlAX37Wj4uM1Oe04yl2mpm4nNUWi8CX4c9jTmnHsbhW82NDu8YeBjJd+YjeVIEOo1pA4WPwtUlJnIbVz2/fSwfctvmzZuxceNGtGzZEqmpqU4qEpEE6R+slVth9Flv09NdXyZOMzZPo9EFHKb+1tNvmzpVd5wr/f05EpU+R7WuFmDssdfR9tb/4Icy9A/djfdGbMWJn09j7402mL01EZ3HtmUgROQkdo0Z+uKLL/DKK68gPDwcixYtQu/evS2f5EZsGaIakWp3FFuGzJPga5O/9xzq9OqAoBsXYCqk0QK4WScc5fv/REisjMb/EFVDki1Deo8++iiOHDmCwYMHY9CgQRgxYgRyc3MdXTYiabAl660r9e6tC8IUZloLFArdoFKJ/7HiFBJoNRNageyMY/jvgCx0r3MQD3U6gjpmAiFA98s46HoBQk7ud1qZiMg0u4IhvQEDBmDixIlYu3Yt4uLi8OKLL+L69euOKhuRNEjgwWqSfpoxUDUgkvs0Yzcl5ysvKUfmO3sxrdNWtFCeQsKwFnhlcyJ2FicgChL9HBGRbXmGli5dit9//x2///47/vzzT/j4+CAhIQFPPfUUOnTogK+++gpxcXFIT09Hly5dnFVmIteSctbb4cN1A4FNDeyW8zRjfauZpeR8Dmg1KzpdhA0LDkK9TuD7k3G4Iu4w7FOiBEnh+5HcvwTD71YCT1pxQWZPJnI5m8YMqVQqdOvWDd27d0f37t3RuXNnBAYGGh3z3//+F2lpaTh48KDDC2svjhmiGvGErLecZlyVftA7YDo53+rVQEqKXa9b3s6zUC/IgfqnIGReao8y+Bv2hSkuYkjzP5E8vBYGzGiHoPAg3Q5P+BwRSYwkp9Zb4/z584iOjobG1bM0qsFgSMYcFSRY82CVayuMlFWXnA+wOlWC0ArsXXkY6o/PQ70rAntvtjXa39r/LyR3OIXk8aHoMTHe/PIX/BwR2cRjgyEhBLZt24Y+ffo48rI1wmBIphydF8gTs96yxcj0a5CRYTEHUWnSYGS9dwDqr4qh/rMlTmuiDYf5QIOewQeR3OsKkqc0RutBzawvj7M/R3zPyYt4bDAkRQyGZMhZCfc85UGj0QBvvqkL/C5fvr1dCtmX3c1CqgQB4JJvOFpoDqMQ9Q3ba6MYA6MOIPm+cgye0QYN24bZ/3moyeeounOllhiUqIYYDDkQgyGZkWpeIFdJTweeeAK4dKnqPnbHWJ2DKBGZOOrTBkNbHUXyqED0m9oOAfUCbh/gjsCjunsC0sy4TVQDDIYciMGQB6rJX84STLjnMunpwAMPVH+MtweDFmhXfAmfR8ZaPO74U28h9r0Z8PEzkYHEHUt9WLpnaKjpAFh/jIzfc/Jcrnp+2zS1nsglavoXt1TzAjlDxaAxPBx49lnL51RMEultwaAZNy/fxI8LD0D9TQkuHa0NaxZPaT7qTsBUIGRpqQ+FQrfUR0rK7cCjpt2r1iwvYi4Q0h8js/ecyBYMhkhazP31q18DzJq/uKWcF8iRTAWNtvCGYLAaBdkXsH7+Yag3+GNTfjvcRFcAusHPZxCNKJw1nXXWUg4iWzKSJyY6pjvN0j2t5eXvOZG9GAyRdNjzF7cpLky45zbmgkZbeHowWInQChz+/i+ol+RB/UsodlxLgMDt97ix72kkxx1Hyti6CG+0ED6PPPT3iSamuFeXuduWlkdHBPe23NMSL3vPiRyFwRBJh61/cZujX6ZixAjdw83Wh53UVRc0WstL1iwrLynHrx9nQ/35Faj3N0FOWXMAzQ37O9c+hORuBUh5KgrtR7SCwifm7z2dgEA/+zJ3WxtQhIcD48fXPLi35Z7meMMfAEROxGCIpMORY328eZmKmnaZKBQeHQxez7+OjQsOQJ2uwXe5bXFJdDDs80cp7gnbj+Skmxg6vSVi7owDEGf6QsOH25eB2tqWR8Axwb2199QPoPbGPwAAz0lrQR6JwRBJh6PH+tj7sJO6mnSZNGgAfPSRxwWDZ/44h2/fyYH6x0BsudAet9DDsC9UcRmDYw8hZbgvBkxLQN3oO62/sK+v7QOKrW15LCiw7nrWvJ/W3POjj3T/euMfAMyfRM4mZKCwsFAAEIWFhe4uClWnvFyImBghFAohdL/ujb8UCiFUKt1xcpaZafr1qe4rNFSI2bM95rXTarRi39eHxb/vyRRdamdXqU5zvxNieudMsfXdfaLsZpl7Crlmje7zWrFgKpVuuxDWv0+ZmY67pxC69zgzU4i0NN2/HvKem7VmjenfCQqF7qti3cnruOr5zTxDJC1cu8kyaxb8bNQISE3VtU64o0XMji6Nshtl2Pr+Aai/vAZ1dnOc1MQY9imgRfc62Ui+6xJSpqjQ5r5mUPgonF0Ly6qrp7MWZpVTd5HcE6gSky46EoMhD+OJa4C5mpSDRhu6NK6eLMQP87OhVgM/nIpHIUIM+wJxA/0jDyDl3lsYPKMNIhIauqoGjiPl98kTyDmBKgFg0kWSM28d6+NIUh0gbsVU8hPhXaFeeBzqzLrYeqUdytHTcFi44gKGtjyMlJFK9HsuAbXDupm+j6e0jkj1ffIUckqgSm7FliEiTyaloMCKBVDzEYkYnIYWt8sYpzyG5DtOI+XxMHQdH2d6+YuKPHEwrZTeJ0/CliHZYzeZA7klGOIvP5IbKx9cb+CfuBgUi6b9YjHk2RZo0a+J9fdwx5pg5D7OGndFHoPdZJ7ME/9yJaqhot1HYc2vqn/hv0AxgD0xQOFiAFYGQ47KUE6ew9sTqJJkWGiPJpvp/3Kt3FWgHzORbs0SkUSe4ejGXMwfkoW7Q/6HYc83t3xCRbb+TNiSoZy8h37cVaNGxttjYtgSSA7DbjJH4jRQ8nKaWxr89mk21KmXod6nwuFbtwMg3QKoMYhAPqye9G7Lz8TKlcCYMZavmZYGjB5tbQnIU3DogSyxm8wTOWptLfJOHvrLvLigGJsXHoR6zS2sP9YGF0R7w75auIXE0ANI6XcdQ6e1QOS5Jaankptjy8+EozOUk2exJ1s4kZUYDDkSp4GSOR42juzcvvNY/84RqDcG4MeCdijB7Snu9RRXcV/jbKTcr8DAafEIady5wplmppJbvKEVPxPWrgnGxUiJyEYMhhyJf7mSKVbk3nF3QCS0AtkZx6D+8AzUv4ZhZ3ECgAjD/qZ+eUhJ+AspjwSj11MJqFX7LvMXq5gnassW4I03LBfAmp8JDqYlIifhmCFH4jRQqkzC48jKbpTh56UHoV5RBPWBpvir3HhWV9egg0jucREpTzdCfEoL+5a/sPQzAQANG+peH39/667JDOVEssE8Qw7k0jxDUk2/76HjVTyexJLGFZ0uwoYFB6FeJ/D9yThcEfUN+5QoQVL4fqQMLMGQ6a0R1TGimivZwNzPREUVuwyt+azy80wkCxxA7amkmH7fw8aruJWlh6ytD2EJjCM7teMMvl14DOqfgpB5qT3KKix/Eaa4iCHN/0TKCH/0n5aAoPCuji+AuZ+JivRdhs8/r5s1ZumzysG0RORAbBlyFqn85cqMvdazFDTaE1S6oWVIaAX2rjwM9cfnod4Vgb032xrtb+3/F5I7nELKhFB0fzwevv4uWjH91i3d63Xhgg21geM/q1L52SQii1z2/BYyUFhYKACIwsJCdxfFtcrLhYiJEUIXClX9UiiEUKl0x8ndmjW618PUa6RQCPHCC9XvX7PG9HX174Gpcx34HpQUlogNb/wunk7IEjG+Z4xu4YNy0St4n3h7cKY4/P1x+2+yZk3Vz1NMjPm6V5aZaf6zaOnLUZ/VmtaBiFzKVc9vtgx5M4mNV3EpW/76t2aQs4+P7jhz+6sbBO2kcWSXj1/B9/MPQb3eB5tOt0FH/A9ROIdziMJudEL/qENIGVyO+6a3QcO2YTZf32QdatLCaG3SxOrU5LPKVlIij+Oq57dHLMdx5swZjB07Fg0aNEBgYCDatWuHP/74w93Fkj4JjFdxi/R0XXDTt6/u4du3r+57c8s+WJMs01wgpN9f3TIQDlxO4PhPJ7FwWBYS6+1DeIu6eGTpXSg/fQ4H0B5Z6IuVGIMs9EVRdBzS3z+LcR/3qnkgZGlNMEC3Jlh1rxHgmJQS9n5WHVUHIvJKkh9AfeXKFdx1113o27cvfvjhBzRs2BA5OTmoX7++5ZPlTo55j+zJ6eOoYLC661TMvWPDWBVtuRa7Ug9B/dlFZOyJwaHSFqi4sOnUWu/jnbJnqpynOHfWcTmMHJVZ3VLSRGvY+1lldngiqobkg6F58+ZBpVJh+fLlhm2xsbFuLJEHkVvGXntXNXdUMGjpOlbOgLpx8Qa2LD4I9Tcl+DanDc5rE25fAuXoU38/Uvpew9ApTRD76DzA1DPekau4O6qFsbqkiZbU9LMq11ZSIrKK5LvJ1Go1unTpggcffBDh4eG444478PHHH1d7TmlpKYqKioy+ZEn/8AFuj4vQ88aMvfauaq4PGiu/RnoKRfWvkUKhS/pXg6CyIPsCPpuwHcOidiKsIZD8Rld8cuRunNeGIxiFGKX6FWlTfsWFv65jy+VOeHZNH8QqTrhmFXdHtjCa6zJUqYAXXtC9ls74rMqxlZSIrOfU4dkOoFQqhVKpFDNnzhR79uwRy5YtEwEBASI1NdXsObNmzRIAqnzJbjaZnqkZNCqV982gSUuzbmZSWpru+PJy3QyntDQhZs+2bjZZ5WMszSYzQ6vRikPfHhNz780UPev+TyigMbpsY9888Uz7LLF53m5Req3UMfW1lzNmxFV87TMzb5/rrM+qi2b1EZFjuWo2meSDoVq1aokePXoYbXvmmWdE9+7dzZ5TUlIiCgsLDV95eXnyDoaEMP/w8SbWTt3OzDT90PXxqXpsgwa3H8Q1fFCX3SwTW9/dJ2Z0zhQta/1V5Vada2eLf9+TKfZ9fVhoNVrH1rem9KkHHBQMVstZn1VX1oGIHMJVwZDkxwxFRUUhLi7OaFvbtm2xZs0as+colUoolUpnF82zyCFjr7VjpC5cAEaNqnqMVlv1nEuXbv/fjkHQ185ew6aFB6FO1+C73La4JDoY9vmjFPeE7UdK/5sYMq0lYu6MAxBn9lp219cRY8JcmVndWZ9VKWaHJyJJkHyeoTFjxiAvLw/bK4x7mDZtGnbu3Ilff/3Vqmt4VZ4hOWXPtaeulnL6rFoFTJtW/VibiuxYSPXMH+fw7Ts5UP8YiC0X2uMWbgfmoYrLGBJ7CMnDfTFgWgLqRte1rhzmuHotPG/4/HlDHYhkghmo/7Zr1y7h5+cn3nzzTZGTkyO+/PJLUbt2bbFixQqrr+E1GajllD23JnWtrjvL3izI1XQ1aTVase/rw+Lf92SKLrWzq5zaolaumNE5U2x9d58ou1nmsJfIqvoSEXkwZqCuYP369Zg5cyZycnIQGxuL6dOnY9KkSVaf7xUtQ3LKnuuIupr769/eLMhpacDo0YZvb12/hW0fHIT6y2tQZzfHSU3M7WJCi+51spHS6xKSJ6vQ5r5mUPiYmanmKI5u7WDrCRFJgKue3x4RDNWUxwdD1iwXYWNXjmQ5u67WLlFSWWYmrsbegR/mZ0OtBn44FY9ChBh2B+IGBkQeQPK9tzB4RhtEJDS0/R5SYc+CtERETuCq57fkB1AT5JU919l1tTELsoAC1wMb4oH7g5B5tTbK0dOwL8KnAENbHkbygwHo91wCaod1s708UmNPBm8iIg/HYMgTyCl7rrPrakMWZC10XVvjbn6IzTfvBADEKY8hpdNpJD8Whq7j4+Djd7d95ZAiezN4W3NddrkRkYRJPgM1QV7Zc11RVzNZkDWVfhxOIwYj8TWu1IvFOylZyPnxJLJLWuC/vyai+8QE+Ph52Y+PvRm8q2ProrlERG7AliFPIKc1xlxU14vxffD9gBU4lpGNvEuByEUsfkFP3IVfEYu/0CTsJloOb4dlzyegQcvQGt3LYzi6VY5dbkTkIRgMeYLquna8bY0xJ9b16MZcqN87iYzt9fFrUQK06AOgDwAgxvcsnmi7A8kP1UHiMw9DGSzDpJ2ObJVzVpcbEZETcDaZJzE1y0el8s7suQ6oq+aWBr99mg116mVk7GuMI7eaGe2/I/BPJHc9j+RJEbhjdBvnT3+XOv1MPkutctbM5LN21l5mpnMG/XOcEpFX4GwyqsqO5SBM8oQHhZ11LS4oxuaFB6Fecwvrj7XBBdHesK8WbqFvg/1IvqcYQ6e1QOMebQG0dXJFPIgjW+XcOeifqQGIyEYMhjxNTddt8qQHhZV1PbfvPNa/cwQZGwPwY0F7lOL2FPd6iqsY3CQbycMUGDgtHiGNuzixwF7AUet3uWvQP8cpEZEd2E0mJ16SxVpoBbIzjkH94Rlk/BqGXcUJRvtj/U4hpV0ukscGo9dTCahVu5abSmoFqbbS1bRcjuxys/WeckhOSiQTzEDtQLINhio+0MLDgfHjq5863bChbr+/v8uKaK2yG2X4eelBqFcUIeNALHLLGxvt7xZ0EMk9LyL5H40Qn9LCM8b/eFIrnT1cvYisu8cpEZHDccwQ1YypB60lFy7ocu8sWyaJh3HR6SJsWHAQGWsFvj8Vj6viDsO+ANxEUvgBJA8swZDprRHVMaGaK0mQHLpzHNXlZi05JSclIodiy5A3MvegtZZC4baH8akdZ/DtwmPI2FIHWZfboQy3W6kaKi5gSIvDSH7AH/2nJSAoPMjl5XMIW7tzpNqVZi1XlZ8tQ0Reh91kDiSrYMjSg9YaLhxbIbQCe1cehvrj88jYFYl9N9sY7W/jfxzJHfOQPD4U3R+Ph6+/BwUB5tjy0L582bu70hzJHeOUiMip2E1G9rG0pII1nLzwa2lRKbLeO4CMlTfw7eEWOK25PcXdBxrcFXwQyb2vIPmZJmg1sDmA5g4vg9NY0wpibTdNRoYu6PHmrjRHklNyUiJyKAZD3saR4yEceK3Lx6/g+/mHkPGtDzacScB13J7iHoTrGBh9EMn3lWPw820R1rqDw+7rUtYOiLZ2OvmKFczgbCtXj1MiIq/AbjJvY20XjDVqOLbi+E8nkbE4F+qt9fBzYQI0FWLvKJ98JLc+iuRRgbjnuXYIqBdg302kMp7GlrQF1nTnhIXpBrRbwvEvpknlc0FENcIxQw4kq2DImgdtdDRw7RpQVGT6GnaOrdCWa7Er9RAyPr0I9d4YHCptYbS/fcARJHc5h+SJ4ej8cJuar/oulanp9uS3sTTt/LnndC0ZlqSlAaNH21tyIiJJc9Xzu4ZPI3I4jUbXurNype5fjca28/XjJoDbD1Y9/fdjxpgPhADdw9nKsRU3Lt7At6/uwsTW2xGtvIgekxIw97dEHCptAT+UoV/9PVg8fCtyt5/G/262xn+2J+LOcXGOCYRGjKgagOjH06Sn1+z6trA0TqviGCw9fXdOo0bGx8bE6LanpFh3b0dncCYikiG2DFnLFc3ujmzpMLfQ6YIFwPTp1T+8GzQAzp83W7/zBy/guwWHkbHBH5vz2+Emahv2BaMQ9zXORnIyMGhaG9Q7tV8XoFy4oEvq2KhRzV87qWUaXrlSF2BaYqoVx9znSsozo9gFRUQu4rKeHSEDhYWFAoAoLCy07wJr1ggREyOE7rGk+4qJ0W13lDVrhFAojO8B6LYpFPbdq7xciMxMIdLSdP/qv698D1NfmZmGy2g1WnHo22Ni7r2Zoked/UIBjdGhTXzzxDPts8TmebtF6bXS2/Wp/Jo56rWzow5O5azy6D8TlT8XNflM1JQrfhaIiP5W4+e3lRgMWeLoIMVUgFJebj5w0N9LpdIdV1NpaVY9uMtT/09sfXefmNE5U7SolVvlkC61s8W/78kU//vmiNBqtNa9ZpXrZO8D1Mo6iLS0mr9e1tC/f+bqXJP3z1TwoVK5LxBydMBORFQNVwVD7CarjqO7Y8x1g02aBMyaZfl8R8wcsnK2WTIy8C2SDd/7oxT9Gu5HctJNDJ3eEo26mBmrYkvSR5XKvm4eKWYaduY6XFLolpJa1yQRyQJnkzmQ3S+mIx+61U29tvYtcMTMIQtjUbRQ4DRiEItc1FMUYkjsISQP98WAaQmoG13X8vVtndpvT8Ai1fE05sZpeUN+GykGoETk9ZiBWgoctfCjRqN7SJpLoGctB8wcEgofnBz5Ipq88ywEFPDB7ftroWvF+KnZRGRNO4geE+PhF9DLthvYmqjx3DnbWz6kmml4+HDdLDB3t+I4AxdBJSIvxmCoOtYGH5aOq+kSGfqWjt697Tr91vVb2PbBQWSsuAb1oeY4pXkG96MRFuM5qHC7XJqwSPh9+B7Gj3jA/rLaGrDl5FTtfrFmBp1UMw37+npny4ijfhaIiCSI3WTVcVR3jLVTr/XXdMCYk6snC/HD/GxkZAA/5MWjCCGGfYG4gQGRB5Ay4CaGJV5F/YCb5lsxTLXaAOZbPyy9ZhU1aABcumT6NbC2zlIYTyMHUu2aJCKvxm4yKXBUd4y1fy3Png18/LHdLR252/Lw7eK/kJFZF9uutEM5ehr2RfgUYGjLw0gZGYB+U9shMLSb5fKYGgPToIHu34pBTMWWnIqvWXUqJ4SsyJa1t7y1JUZqpNo1SUTkAGwZskZNB8ba8lc1YHVLh7Zci91fHkbGxwVQ747GgZJWRvvjlTlI7nQGKY+H2Z712dyAb1NMteSYes30VCpg4kTXzaAjx/HmQeJEJDmcTeZAkshA7aCp1yVXS/DT4gPI+Oomvj3aCue0kYZ9vihH73oHkNynEMnPxqL5PU2sL19FtkyP1zPVTaJ/zUxloF61yv6szeRe7JokIhdhN5nU1LQ7pgYDfi8euYTv5v+JjO/8sOlcAopxp2FfXRTh3phspAzVYtCMOIQ2v8P+MurZM+C74vpb+teputeMA3I9F7smicjLMBhyJRumXh/dmIuMd09C/XN9/FqUAC1uT3GP8T2L5LY5SBlTB30mJ0AZ3MOx5azJ9Ghrz+3dWxcIWuo6tHMGHRERkbUYDLmamb+qNbc0+O3TbGQsvwz1/xrjyK1mAGIN++8I/BMp3c4j+YlIdBzVGgqfaOeVsSatMdaeywG5REQkERwz5EbFBcXYvPAgMlbfwvrjbXFRhBn21cIt9G2wHylJxRg6rSVU3ZwY/FRmy/R4PXunVnNArvfhmCIichAOoHYgKQVD5/adx7fzj0C9OQA/FrRHKQIM++oprmJwk2yk3O+DgdPjERxjZVltefhYe6y5Ad+m1HT9LT48vYe59fcsJdEkIjKBwZADuTMYElqB7IxjyPjgDNQ7wrCrOMFofzO/k0hpn4vkR+rhrifiUat2LdtuYMvDx9YHlbV5htiSQ0D16+8BNVuslohkicGQA7k6GCq7UYaflx5Exv8VQX0wFrnljY32dws6iOSeF5EyOQZxQ5tD4VNNAsLq2PLwsfdBZWsGapInrmpPRE7AYMiBXPFiFp4qxIYF2VBnCHx/Kh5XRT3DvgDcRFL4ASQPLMGQ6a0R1TGi5je05eED8EHlCOzOM4+r2hOREzDPkAc4teMM1O8cg/qnOsi63A5lFZa/aKi4gCEtDiNlhD+SpiYgKLyrY29uKRdQxbw/gPXH8kFlGsfCVI+r2hORB/O4YGju3LmYOXMmnnvuOSxatMil9xZagT1ph6H++DzUv0di3802ABoZ9rfxP47kjnlIeawBuk2Ig6+/E3PkOOPhwweVaea6GM+c0W3nWBgm0SQij+ZRwdDvv/+OZcuWoX379i67Z2lRKTLfPQD1Vzeg/rMlzmjbAmgLAPCBBncFH0Ry7ytIfqYJWg1sDqB5zW9qTXeMMx4+fFBVpdHoWoRM9SbbsqCst2MSTSLyYB4TDF2/fh0PP/wwPv74Y7zxxhtOvdfl41fw3dvZUK/3xYYzCbiOLoZ9QbiOgdEHkTK4HPfNaIuw1h0ce/PVq4Gnn9at5aVnqjvG1oePVB9UUh+HY0t3pJy7GJlEk4g8mA3LmLvX5MmTMXjwYCQlJVk8trS0FEVFRUZflhzbchLvpGQhsd4+hLeoi0eX9cLqMz1wHXUR5ZOPJ9tuw/ezf8fFK35Yc6Y7Hv2oF8JaN3BE1W578UXgwQeNAyFA9zAeMULXXaOnf/gAtx82epUfPrYc60rp6bqB3X376hZt7dtX933FerpbRoZ1x7GL8fb6e40aGW+PiWFXIhFJm/AAK1euFAkJCeLmzZtCCCH69OkjnnvuObPHz5o1SwCo8lVYWGg4RlOmETs+PiBe7p4p4pQ5Qven7O2v9gGHxb96ZYrfP88WmjKNs6soxKpVokohKn+pVEKUlxuft2aNEDExVY9bs6bqPWw51tnWrBFCoahaR4VC9+WOMlVWXi5EWJjl9wUQIjPT3aWVjvJy3euRlqb7t/JnlojISoWFhVWe384g+an1eXl56NKlCzZv3mwYK5SYmIiOHTuaHUBdWlqK0tJSw/dFRUVQqVQ499c5/P7ZKWSsKsX6Y61xXhtuOMYfJZhS5wv0jzuNjmPbIfLp4VVbSpzVpaPRAJGRwMWLlo81NTXZGRmonclTctJYO108OBi4fJldQEREDsap9X/bvXs3CgoK0KlTJ8M2jUaDbdu24f3330dpaSl8Kz2ElEollEpllWvFNgNKcHuKezAKcV/jbEyOz0LPfUvgc+4ssAu6r3+HAR98oOu2Apw7tXr7dusCIcB0d4yZxV9NsuVYZ/GUcTjWdn0VFem609gNRETkkSQfDPXr1w8HDhww2jZhwgS0adMGL730UpVAqDolqI0mvqeRHH8cKWProvc/EuC/KR8Y8a+qA4svXgRGjgReeAHo3t25U6ttGW9iz4wvKbQGVeQpOWlsea05o4yIyGNJPhiqW7cuEhKM1/MKCgpCgwYNqmy35JfPj6HH2Dug8InRbahu2rTe22/r1uNy5tRqax+6DRvaPuNLiskCPSUnjX7GXnWtWHpSaMkiIiK7eMxsMkdIGNbCeB0wS901ehUXJa2scqZne+gfupZ88IFtAZc+WWDlOupbtNw1a0tf38oz2/QUCt3ir+7OSVNxFp413N2SRUREdvHIYCgrK8sx2acd+fCqybX0D11zwQGg664bMcL6a1pKFgjoWrQ0GpuK6hBSnepvyvDhwOzZ1h3r7pYsIiKyi0cGQw7jyIeXuWtpNLpZSStX6v41F3zoc7RUbiFq2BBYtQp46y3bymPr2mWu5kk5aV55pfqWO6m0ZBERkV0kP2bIqXr3BsLCrJ/JZUp12ZttHa8zfLhu7JEjBjt7wiBlR9bXmSpmVwaYXZmIyMvIOxjy9dWNwxk5svrjGjTQjRuyZZkBexf3dNTUd08ZpCyFqf7W0LdkmQpuFy2SVksWERHZRPJJFx3BYtKmF1/UzRozRaHQPQSBqg9Clcr0g1AKSQX1ZbC0Hpm7Ext6GqmlKSAi8mKuSrrIYEjP1AKplYMdax+E1mYuNpVN2pH0rVOA6RYtqY3NISIiqoAZqF1txAjg/vurD3as7dKRyngddu0QERFZxGCoIm8cr+Mpg5SJiIjchMGQM+iTCloar+OqqdieMkiZiIjIDeSdZ8hZPCmpIBERkcwxGHIWT0oqSEREJGPsJnMmjtchIiKSPAZDzsbxOkRERJLGbjIiIiKSNQZDREREJGsMhoiIiEjWGAwRERGRrDEYIiIiIlljMERERESyxmCIiIiIZI3BEBEREckagyEiIiKSNQZDREREJGsMhoiIiEjWGAwRERGRrDEYIiIiIlljMERERESyxmCIiIiIZI3BEBEREckagyEiIiKSNQZDREREJGsMhoiIiEjWGAwRERGRrDEYIiIiIlljMERERESyxmCIiIiIZI3BEBEREckagyEiIiKSNQZDREREJGuSD4bmzJmDO++8E3Xr1kV4eDiGDRuGI0eOuLtYRERE5CUkHwxt3boVkydPxm+//YbNmzejrKwMAwYMQHFxsbuLRkRERF5AIYQQ7i6ELS5cuIDw8HBs3boVd999t1XnFBUVISQkBIWFhQgODnZyCYmIiMgRXPX89nPalZ2ksLAQABAaGmr2mNLSUpSWlhq+Lyoqcnq5iIiIyDNJvpusIq1Wi6lTp+Kuu+5CQkKC2ePmzJmDkJAQw5dKpXJhKYmIiMiTeFQ32T/+8Q/88MMP+PnnnxETE2P2OFMtQyqVit1kREREHoTdZJVMmTIF69evx7Zt26oNhABAqVRCqVS6qGRERETkySQfDAkh8Mwzz2Dt2rXIyspCbGysu4tEREREXkTywdDkyZORlpaGjIwM1K1bF/n5+QCAkJAQBAYGurl0RERE5OkkP2ZIoVCY3L58+XKMHz/eqmtwaj0REZHn4Zihv0k8ViMiIiIP51FT64mIiIgcjcEQERERyRqDISIiIpI1BkNEREQkawyGiIiISNYYDBEREZGsMRgiIiIiWWMwRERERLLGYIiIiIhkjcEQERERyRqDISIiIpI1BkNEREQkawyGiIiISNYYDBEREZGsMRgiIiIiWWMwRERERLLGYIiIiIhkjcEQERERyRqDISIiIpI1BkNEREQkawyGiIiISNYYDBEREZGsMRgiIiIiWWMwRERERLLGYIiIiIhkjcEQERERyRqDISIiIpI1BkNEREQkawyGiIiISNYYDBEREZGsMRgiIiIiWWMwRERERLLGYIiIiIhkjcEQERERyRqDISIiIpI1BkNEREQkawyGiIiISNYYDBEREZGseUwwtGTJEjRt2hQBAQHo1q0bdu3a5e4iERERkRfwiGDo66+/xvTp0zFr1izs2bMHHTp0wMCBA1FQUODuohEREZGH84hg6J133sGkSZMwYcIExMXFYenSpahduzY+++wzdxeNiIiIPJzkg6Fbt25h9+7dSEpKMmzz8fFBUlISduzY4caSERERkTfwc3cBLLl48SI0Gg0iIiKMtkdERODw4cMmzyktLUVpaanh+8LCQgBAUVGR8wpKREREDqV/bgshnHofyQdD9pgzZw5mz55dZbtKpXJDaYiIiKgmLl26hJCQEKddX/LBUFhYGHx9fXH+/Hmj7efPn0dkZKTJc2bOnInp06cbvr969SqaNGmCU6dOOfXFlJqioiKoVCrk5eUhODjY3cVxGdab9ZYD1pv1loPCwkI0btwYoaGhTr2P5IMhf39/dO7cGVu2bMGwYcMAAFqtFlu2bMGUKVNMnqNUKqFUKqtsDwkJkdWHSC84OJj1lhHWW15Yb3mRa719fJw7xFnywRAATJ8+HePGjUOXLl3QtWtXLFq0CMXFxZgwYYK7i0ZEREQeziOCoVGjRuHChQt47bXXkJ+fj44dO2LDhg1VBlUTERER2cojgiEAmDJlitluMUuUSiVmzZplsuvMm7HerLccsN6stxyw3s6tt0I4e74aERERkYRJPukiERERkTMxGCIiIiJZYzBEREREssZgiIiIiGTNI4OhJUuWoGnTpggICEC3bt2wa9euao//5ptv0KZNGwQEBKBdu3b4/vvvjfYLIfDaa68hKioKgYGBSEpKQk5OjjOrYBdb6v3xxx+jd+/eqF+/PurXr4+kpKQqx48fPx4KhcLo695773V2NWxmS71TU1Or1CkgIMDoGG98vxMTE6vUW6FQYPDgwYZjPOH93rZtG4YOHYro6GgoFAqsW7fO4jlZWVno1KkTlEolWrRogdTU1CrH2Po7w9VsrXd6ejr69++Phg0bIjg4GD169MDGjRuNjnn99dervN9t2rRxYi1sZ2u9s7KyTH7O8/PzjY7ztvfb1M+uQqFAfHy84RhPeL/nzJmDO++8E3Xr1kV4eDiGDRuGI0eOWDzPFc9wjwuGvv76a0yfPh2zZs3Cnj170KFDBwwcOBAFBQUmj//1118xevRoPP7449i7dy+GDRuGYcOG4eDBg4Zj3nrrLbz77rtYunQpdu7ciaCgIAwcOBAlJSWuqpZFttY7KysLo0ePRmZmJnbs2AGVSoUBAwbgzJkzRsfde++9OHfunOFr5cqVrqiO1WytN6DL0FqxTidPnjTa743vd3p6ulGdDx48CF9fXzz44INGx0n9/S4uLkaHDh2wZMkSq47Pzc3F4MGD0bdvX+zbtw9Tp07FxIkTjQIDez5DrmZrvbdt24b+/fvj+++/x+7du9G3b18MHToUe/fuNTouPj7e6P3++eefnVF8u9lab70jR44Y1Ss8PNywzxvf78WLFxvVNy8vD6GhoVV+vqX+fm/duhWTJ0/Gb7/9hs2bN6OsrAwDBgxAcXGx2XNc9gwXHqZr165i8uTJhu81Go2Ijo4Wc+bMMXn8yJEjxeDBg422devWTTz55JNCCCG0Wq2IjIwUb7/9tmH/1atXhVKpFCtXrnRCDexja70rKy8vF3Xr1hWff/65Ydu4ceNESkqKo4vqULbWe/ny5SIkJMTs9eTyfi9cuFDUrVtXXL9+3bDNE97vigCItWvXVnvMiy++KOLj4422jRo1SgwcONDwfU1fS1ezpt6mxMXFidmzZxu+nzVrlujQoYPjCuZk1tQ7MzNTABBXrlwxe4wc3u+1a9cKhUIhTpw4Ydjmae+3EEIUFBQIAGLr1q1mj3HVM9yjWoZu3bqF3bt3IykpybDNx8cHSUlJ2LFjh8lzduzYYXQ8AAwcONBwfG5uLvLz842OCQkJQbdu3cxe09XsqXdlN27cQFlZWZXF7rKyshAeHo7WrVvjH//4By5duuTQsteEvfW+fv06mjRpApVKhZSUFGRnZxv2yeX9/vTTT/HQQw8hKCjIaLuU3297WPr5dsRr6Qm0Wi2uXbtW5ec7JycH0dHRaNasGR5++GGcOnXKTSV0rI4dOyIqKgr9+/fHL7/8Ytgul/f7008/RVJSEpo0aWK03dPe78LCQgCodhFWVz3DPSoYunjxIjQaTZVlOCIiIqr0Gevl5+dXe7z+X1uu6Wr21Luyl156CdHR0UYfmHvvvRdffPEFtmzZgnnz5mHr1q0YNGgQNBqNQ8tvL3vq3bp1a3z22WfIyMjAihUroNVq0bNnT5w+fRqAPN7vXbt24eDBg5g4caLRdqm/3/Yw9/NdVFSEmzdvOuRnxxPMnz8f169fx8iRIw3bunXrhtTUVGzYsAEffvghcnNz0bt3b1y7ds2NJa2ZqKgoLF26FGvWrMGaNWugUqmQmJiIPXv2AHDM70qpO3v2LH744YcqP9+e9n5rtVpMnToVd911FxISEswe56pnuMcsx0H2mzt3Lr766itkZWUZDSZ+6KGHDP9v164d2rdvj+bNmyMrKwv9+vVzR1FrrEePHujRo4fh+549e6Jt27ZYtmwZ/vOf/7ixZK7z6aefol27dujatavRdm98vwlIS0vD7NmzkZGRYTR2ZtCgQYb/t2/fHt26dUOTJk2watUqPP744+4oao21bt0arVu3Nnzfs2dPHD9+HAsXLsT//d//ubFkrvP555+jXr16GDZsmNF2T3u/J0+ejIMHD0pmXJNHtQyFhYXB19cX58+fN9p+/vx5REZGmjwnMjKy2uP1/9pyTVezp9568+fPx9y5c7Fp0ya0b9++2mObNWuGsLAwHDt2rMZldoSa1FuvVq1auOOOOwx18vb3u7i4GF999ZVVv/yk9n7bw9zPd3BwMAIDAx3yGZKyr776ChMnTsSqVauqdCVUVq9ePbRq1cqj329TunbtaqiTt7/fQgh89tlneOSRR+Dv71/tsVJ+v6dMmYL169cjMzMTMTEx1R7rqme4RwVD/v7+6Ny5M7Zs2WLYptVqsWXLFqPWgIp69OhhdDwAbN682XB8bGwsIiMjjY4pKirCzp07zV7T1eypN6AbYf+f//wHGzZsQJcuXSze5/Tp07h06RKioqIcUu6asrfeFWk0Ghw4cMBQJ29+vwHdFNTS0lKMHTvW4n2k9n7bw9LPtyM+Q1K1cuVKTJgwAStXrjRKoWDO9evXcfz4cY9+v03Zt2+foU7e/H4DutlYx44ds+qPHSm+30IITJkyBWvXrsVPP/2E2NhYi+e47Blu09BvCfjqq6+EUqkUqamp4tChQ+KJJ54Q9erVE/n5+UIIIR555BHx8ssvG47/5ZdfhJ+fn5g/f774888/xaxZs0StWrXEgQMHDMfMnTtX1KtXT2RkZIj9+/eLlJQUERsbK27evOny+plja73nzp0r/P39xerVq8W5c+cMX9euXRNCCHHt2jXx/PPPix07dojc3Fzx448/ik6dOomWLVuKkpISt9TRFFvrPXv2bLFx40Zx/PhxsXv3bvHQQw+JgIAAkZ2dbTjGG99vvV69eolRo0ZV2e4p7/e1a9fE3r17xd69ewUA8c4774i9e/eKkydPCiGEePnll8UjjzxiOP6vv/4StWvXFi+88IL4888/xZIlS4Svr6/YsGGD4RhLr6UU2FrvL7/8Uvj5+YklS5YY/XxfvXrVcMyMGTNEVlaWyM3NFb/88otISkoSYWFhoqCgwOX1M8fWei9cuFCsW7dO5OTkiAMHDojnnntO+Pj4iB9//NFwjDe+33pjx44V3bp1M3lNT3i///GPf4iQkBCRlZVl9Lm9ceOG4Rh3PcM9LhgSQoj33ntPNG7cWPj7+4uuXbuK3377zbCvT58+Yty4cUbHr1q1SrRq1Ur4+/uL+Ph48d133xnt12q14tVXXxURERFCqVSKfv36iSNHjriiKjaxpd5NmjQRAKp8zZo1SwghxI0bN8SAAQNEw4YNRa1atUSTJk3EpEmTJPULQ8+Wek+dOtVwbEREhLjvvvvEnj17jK7nje+3EEIcPnxYABCbNm2qci1Peb/1U6crf+nrOm7cONGnT58q53Ts2FH4+/uLZs2aieXLl1e5bnWvpRTYWu8+ffpUe7wQuhQDUVFRwt/fXzRq1EiMGjVKHDt2zLUVs8DWes+bN080b95cBAQEiNDQUJGYmCh++umnKtf1tvdbCN108cDAQPHRRx+ZvKYnvN+m6gzA6GfWXc9wxd8FJCIiIpIljxozRERERORoDIaIiIhI1hgMERERkawxGCIiIiJZYzBEREREssZgiIiIiGSNwRARERHJGoMhIiIikjUGQ0RERCRrDIaIiIhI1hgMEZFHWrlyJQIDA3Hu3DnDtgkTJqB9+/YoLCx0Y8mIyNNwbTIi8khCCHTs2BF333033nvvPcyaNQufffYZfvvtNzRq1MjdxSMiD+Ln7gIQEdlDoVDgzTffxIgRIxAZGYn33nsP27dvZyBERDZjyxARebROnTohOzsbmzZtQp8+fdxdHCLyQBwzREQea8OGDTh8+DA0Gg0iIiLcXRwi8lBsGSIij7Rnzx4kJiZi2bJlSE1NRXBwML755ht3F4uIPBDHDBGRxzlx4gQGDx6Mf/7znxg9ejSaNWuGHj16YM+ePejUqZO7i0dEHoYtQ0TkUS5fvoyePXsiMTERS5cuNWwfPHgwNBoNNmzY4MbSEZEnYjBEREREssYB1ERERCRrDIaIiIhI1hgMERERkawxGCIiIiJZYzBEREREssZgiIiIiGSNwRARERHJGoMhIiIikjUGQ0RERCRrDIaIiIhI1hgMERERkawxGCIiIiJZ+380bKW5m+/9eQAAAABJRU5ErkJggg==",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%matplotlib inline\n",
"\n",
"# Importing various packages\n",
"from math import exp, sqrt\n",
"from random import random, seed\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"n = 100\n",
"x = 2*np.random.rand(n,1)\n",
"y = 4+3*x+np.random.randn(n,1)\n",
"\n",
"X = np.c_[np.ones((n,1)), x]\n",
"XT_X = X.T @ X\n",
"theta_linreg = np.linalg.inv(X.T @ X) @ (X.T @ y)\n",
"print(\"Own inversion\")\n",
"print(theta_linreg)\n",
"# Hessian matrix\n",
"H = (2.0/n)* XT_X\n",
"EigValues, EigVectors = np.linalg.eig(H)\n",
"print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
"\n",
"theta = np.random.randn(2,1)\n",
"eta = 1.0/np.max(EigValues)\n",
"Niterations = 1000\n",
"\n",
"#while (iter <= Ni... or test)\n",
"for iter in range(Niterations):\n",
" gradients = 2.0/n*X.T @ ((X @ theta)-y)\n",
" theta -= eta*gradients\n",
"print(\"theta from own gd\")\n",
"print(theta)\n",
"\n",
"xnew = np.array([[0],[2]])\n",
"Xnew = np.c_[np.ones((2,1)), xnew]\n",
"ypredict = Xnew.dot(theta)\n",
"ypredict2 = Xnew.dot(theta_linreg)\n",
"\n",
"n_epochs = 50\n",
"M = 5 #size of each minibatch\n",
"m = int(n/M) #number of minibatches\n",
"t0, t1 = 5, 50\n",
"\n",
"def learning_schedule(t):\n",
" return t0/(t+t1)\n",
"\n",
"theta = np.random.randn(2,1)\n",
"\n",
"for epoch in range(n_epochs):\n",
"# Can you figure out a better way of setting up the contributions to each batch?\n",
" for i in range(m):\n",
" random_index = M*np.random.randint(m)\n",
" xi = X[random_index:random_index+M]\n",
" yi = y[random_index:random_index+M]\n",
" gradients = (2.0/M)* xi.T @ ((xi @ theta)-yi)\n",
" eta = learning_schedule(epoch*m+i)\n",
" theta = theta - eta*gradients\n",
"print(\"theta from own sdg\")\n",
"print(theta)\n",
"\n",
"plt.plot(xnew, ypredict, \"r-\")\n",
"plt.plot(xnew, ypredict2, \"b-\")\n",
"plt.plot(x, y ,'ro')\n",
"plt.axis([0,2.0,0, 15.0])\n",
"plt.xlabel(r'$x$')\n",
"plt.ylabel(r'$y$')\n",
"plt.title(r'Random numbers ')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "de04b41a",
"metadata": {},
"source": [
"In the above code, we have use replacement in setting up the\n",
"mini-batches. The discussion\n",
"[here](https://sebastianraschka.com/faq/docs/sgd-methods.html) may be\n",
"useful."
]
},
{
"cell_type": "markdown",
"id": "77fc1cca",
"metadata": {},
"source": [
"## Momentum based GD\n",
"\n",
"The stochastic gradient descent (SGD) is almost always used with a\n",
"*momentum* or inertia term that serves as a memory of the direction we\n",
"are moving in parameter space. This is typically implemented as\n",
"follows"
]
},
{
"cell_type": "markdown",
"id": "441d1f36",
"metadata": {},
"source": [
"$$\n",
"\\mathbf{v}_{t}=\\gamma \\mathbf{v}_{t-1}+\\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t) \\nonumber\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "47434945",
"metadata": {},
"source": [
"<!-- Equation labels as ordinary links -->\n",
"<div id=\"_auto1\"></div>\n",
"\n",
"$$\n",
"\\begin{equation} \n",
"\\boldsymbol{\\theta}_{t+1}= \\boldsymbol{\\theta}_t -\\mathbf{v}_{t},\n",
"\\label{_auto1} \\tag{1}\n",
"\\end{equation}\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "f3ea5060",
"metadata": {},
"source": [
"where we have introduced a momentum parameter $\\gamma$, with\n",
"$0\\le\\gamma\\le 1$, and for brevity we dropped the explicit notation to\n",
"indicate the gradient is to be taken over a different mini-batch at\n",
"each step. We call this algorithm gradient descent with momentum\n",
"(GDM). From these equations, it is clear that $\\mathbf{v}_t$ is a\n",
"running average of recently encountered gradients and\n",
"$(1-\\gamma)^{-1}$ sets the characteristic time scale for the memory\n",
"used in the averaging procedure. Consistent with this, when\n",
"$\\gamma=0$, this just reduces down to ordinary SGD as discussed\n",
"earlier. An equivalent way of writing the updates is"
]
},
{
"cell_type": "markdown",
"id": "923628c8",
"metadata": {},
"source": [
"$$\n",
"\\Delta \\boldsymbol{\\theta}_{t+1} = \\gamma \\Delta \\boldsymbol{\\theta}_t -\\ \\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t),\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "5c94031c",
"metadata": {},
"source": [
"where we have defined $\\Delta \\boldsymbol{\\theta}_{t}= \\boldsymbol{\\theta}_t-\\boldsymbol{\\theta}_{t-1}$."
]
},
{
"cell_type": "markdown",
"id": "f3f0e9c9",
"metadata": {},
"source": [
"## Algorithms and codes for Adagrad, RMSprop and Adam\n",
"\n",
"The algorithms we have implemented are well described in the text by [Goodfellow, Bengio and Courville, chapter 8](https://www.deeplearningbook.org/contents/optimization.html).\n",
"\n",
"The codes which implement these algorithms are discussed after our presentation of automatic differentiation."
]
},
{
"cell_type": "markdown",
"id": "92253eff",
"metadata": {},
"source": [
"## Practical tips\n",
"\n",
"* **Randomize the data when making mini-batches**. It is always important to randomly shuffle the data when forming mini-batches. Otherwise, the gradient descent method can fit spurious correlations resulting from the order in which data is presented.\n",
"\n",
"* **Transform your inputs**. Learning becomes difficult when our landscape has a mixture of steep and flat directions. One simple trick for minimizing these situations is to standardize the data by subtracting the mean and normalizing the variance of input variables. Whenever possible, also decorrelate the inputs. To understand why this is helpful, consider the case of linear regression. It is easy to show that for the squared error cost function, the Hessian of the cost function is just the correlation matrix between the inputs. Thus, by standardizing the inputs, we are ensuring that the landscape looks homogeneous in all directions in parameter space. Since most deep networks can be viewed as linear transformations followed by a non-linearity at each layer, we expect this intuition to hold beyond the linear case.\n",
"\n",
"* **Monitor the out-of-sample performance.** Always monitor the performance of your model on a validation set (a small portion of the training data that is held out of the training process to serve as a proxy for the test set. If the validation error starts increasing, then the model is beginning to overfit. Terminate the learning process. This *early stopping* significantly improves performance in many settings.\n",
"\n",
"* **Adaptive optimization methods don't always have good generalization.** Recent studies have shown that adaptive methods such as ADAM, RMSPorp, and AdaGrad tend to have poor generalization compared to SGD or SGD with momentum, particularly in the high-dimensional limit (i.e. the number of parameters exceeds the number of data points). Although it is not clear at this stage why these methods perform so well in training deep neural networks, simpler procedures like properly-tuned SGD may work as well or better in these applications.\n",
"\n",
"Geron's text, see chapter 11, has several interesting discussions."
]
},
{
"cell_type": "markdown",
"id": "08209015",
"metadata": {},
"source": [
"## Using Automatic differentation with OLS\n",
"\n",
"We conclude the part on optmization by showing how we can make codes\n",
"for linear regression and logistic regression using **autograd**. The\n",
"first example shows results with ordinary leats squares."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "f1f7d4aa",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Own inversion\n",
"[[3.63498731]\n",
" [3.34342385]]\n",
"Eigenvalues of Hessian Matrix:[0.32784744 4.2127597 ]\n",
"theta from own gd\n",
"[[3.63498731]\n",
" [3.34342385]]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHHCAYAAAC88FzIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABZ5klEQVR4nO3deXxM5+IG8OckkUksCYkliQxiFyQpSqm1tKpqKYbSBS1tXdoqbdHeVv26hGpVFy260N5WWkMstxstCWppXTtVQkMjRKwJQcjM+/tjzMgks+fMep7v55MPOXPmzHvmTHKevKskhBAgIiIiUqggbxeAiIiIyJsYhoiIiEjRGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiIiISNEYhoiIiEjRGIaIiIhI0RiGiMhho0aNQoMGDbxdDJ+WmZkJSZKwbNkybxeFiBzEMETkgxYvXgxJkkxfISEhqFu3LkaNGoXc3FxvF4+IKKCEeLsARGTd//3f/yEhIQHXrl3Dtm3bsHjxYvz222/Yv38/wsLCvF08IqKAwDBE5MP69OmDdu3aAQDGjBmDmjVrYtasWVi9ejWGDh3q5dKRNxUVFaFKlSreLgZRQGAzGZEf6dKlCwDg6NGjpm3Xr1/Hq6++irZt2yIyMhJVqlRBly5dkJGRYfbcY8eOQZIkvPPOO1i4cCEaNWoElUqF22+/Hdu3by/3WitXrkSrVq0QFhaGVq1aYcWKFRbLVFRUhMmTJ0OtVkOlUqFZs2Z45513IIQw20+SJEyYMAFarRaJiYkIDw9Hx44dsW/fPgDAggUL0LhxY4SFhaF79+44duyY3ffjtddegyRJOHLkCEaNGoXq1asjMjISo0ePxpUrV8qd++LFi8sdQ5IkvPbaa+WOefjwYTz88MOIjIxErVq18Morr0AIgZycHAwYMAARERGIiYnBu+++a7FsOp0OL730EmJiYlClShX0798fOTk55fb7/fffce+99yIyMhKVK1dGt27dsHnzZovn+eeff2LEiBGoUaMGOnfuDADIy8vD6NGjER8fD5VKhdjYWAwYMMCh94+IDFgzRORHjDe4GjVqmLYVFhbis88+w/DhwzF27FhcunQJn3/+OXr37o0//vgDKSkpZsdYsmQJLl26hCeffBKSJOHtt9/GoEGD8Pfff6NSpUoAgLVr12Lw4MFITExEamoqzp07Z7rhliaEQP/+/ZGRkYHHH38cKSkpWLNmDV544QXk5ubivffeM9t/06ZNWL16NcaPHw8ASE1Nxf33348XX3wRH3/8Mf71r3/hwoULePvtt/HYY49h/fr1Dr0vQ4cORUJCAlJTU7Fz50589tlnqF27NmbNmuXM22tm2LBhaNGiBWbOnIkffvgBb7zxBqKiorBgwQLcddddmDVrFr755hs8//zzuP3229G1a1ez57/55puQJAlTpkxBfn4+5s6di169emH37t0IDw8HAKxfvx59+vRB27ZtMX36dAQFBWHRokW46667sGnTJrRv397smBqNBk2aNMFbb71lCpuDBw/GgQMH8PTTT6NBgwbIz8/HL7/8gn/++Yed3YkcJYjI5yxatEgAEL/++qs4c+aMyMnJEcuWLRO1atUSKpVK5OTkmPYtKSkRxcXFZs+/cOGCqFOnjnjsscdM27KzswUAER0dLc6fP2/avmrVKgFA/Pe//zVtS0lJEbGxseLixYumbWvXrhUARP369U3bVq5cKQCIN954w+z1hwwZIiRJEkeOHDFtAyBUKpXIzs42bVuwYIEAIGJiYkRhYaFp+7Rp0wQAs30tmT59ugBgdp5CCPHAAw+I6Ojocue+aNGicscAIKZPn17umE888YRpW0lJiYiPjxeSJImZM2eatl+4cEGEh4eLkSNHmrZlZGQIAKJu3bpm57R06VIBQLz//vtCCCH0er1o0qSJ6N27t9Dr9ab9rly5IhISEsTdd99drkzDhw83K/uFCxcEADF79myb7xMR2cZmMiIf1qtXL9SqVQtqtRpDhgxBlSpVsHr1arMamuDgYISGhgIA9Ho9zp8/j5KSErRr1w47d+4sd8xhw4aZ1SwZm97+/vtvAMCpU6ewe/dujBw5EpGRkab97r77biQmJpod68cff0RwcDCeeeYZs+2TJ0+GEAI//fST2faePXua1VZ06NABgKF2o1q1auW2G8tkz1NPPWX2fZcuXXDu3DkUFhY69HxLxowZY/p/cHAw2rVrByEEHn/8cdP26tWro1mzZhbL+eijj5qd05AhQxAbG4sff/wRALB7925kZWVhxIgROHfuHM6ePYuzZ8+iqKgIPXv2xMaNG6HX622eZ3h4OEJDQ5GZmYkLFy64fK5ESsdmMiIfNm/ePDRt2hQFBQX44osvsHHjRqhUqnL7ffnll3j33Xfx119/4caNG6btCQkJ5fatV6+e2ffGYGS8mR4/fhwA0KRJk3LPbdasmVnAOn78OOLi4sxu+gDQokULs2NZe21j2FKr1Ra3O3qDt3VOERERDh3D3jEjIyMRFhaGmjVrltt+7ty5cs8v+/5JkoTGjRubmjqzsrIAACNHjrRahoKCArPgWvZ6qlQqzJo1C5MnT0adOnVwxx134P7778ejjz6KmJgY+ydJRAAYhoh8Wvv27U2jyQYOHIjOnTtjxIgROHToEKpWrQoA+PrrrzFq1CgMHDgQL7zwAmrXro3g4GCkpqaadbQ2Cg4OtvhaokyHZ3ew9toVLZO950uSZPFxnU7n1DHlfO+MtT6zZ88u16/LyHiNjYx9jUqbOHEi+vXrh5UrV2LNmjV45ZVXkJqaivXr1+O2225zulxESsQwROQnjAGnR48e+OijjzB16lQAwLJly9CwYUOkp6eb3fSnT5/u0uvUr18fwK2ai9IOHTpUbt9ff/0Vly5dMqsd+uuvv8yO5W3G2pWLFy+abS9bcyWnsu+fEAJHjhxBUlISAKBRo0YAgIiICPTq1atCr9WoUSNMnjwZkydPRlZWFlJSUvDuu+/i66+/rtBxiZSCfYaI/Ej37t3Rvn17zJ07F9euXQNwq7aidO3E77//jq1bt7r0GrGxsUhJScGXX36JgoIC0/ZffvkFf/75p9m+9913H3Q6HT766COz7e+99x4kSUKfPn1cKoPcIiIiULNmTWzcuNFs+8cff+y21/zqq69w6dIl0/fLli3DqVOnTO9J27Zt0ahRI7zzzju4fPlyueefOXPG7mtcuXLF9DkwatSoEapVq4bi4uIKngGRcrBmiMjPvPDCC9BoNFi8eDGeeuop3H///UhPT8cDDzyAvn37Ijs7G/Pnz0diYqLFm6wjUlNT0bdvX3Tu3BmPPfYYzp8/jw8//BAtW7Y0O2a/fv3Qo0cPvPzyyzh27BiSk5Oxdu1arFq1ChMnTjTVfviCMWPGYObMmRgzZgzatWuHjRs34vDhw257vaioKHTu3BmjR4/G6dOnMXfuXDRu3Bhjx44FAAQFBeGzzz5Dnz590LJlS4wePRp169ZFbm4uMjIyEBERgf/+9782X+Pw4cPo2bMnhg4disTERISEhGDFihU4ffo0HnzwQbedG1GgYRgi8jODBg0y1SiMHTsWo0aNQl5eHhYsWIA1a9YgMTERX3/9NbRaLTIzM116jXvvvRdarRb//ve/MW3aNDRq1AiLFi3CqlWrzI4ZFBSE1atX49VXX8V3332HRYsWoUGDBpg9ezYmT54szwnL5NVXX8WZM2ewbNkyLF26FH369MFPP/2E2rVru+X1XnrpJezduxepqam4dOkSevbsiY8//hiVK1c27dO9e3ds3boVr7/+Oj766CNcvnwZMTEx6NChA5588km7r6FWqzF8+HCsW7cO//nPfxASEoLmzZtj6dKlGDx4sFvOiygQScITvSaJiIiIfBT7DBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIpYp4hvV6PkydPolq1albXKCIiIiLfIoTApUuXEBcXh6Ag99XfKCIMnTx5styq2EREROQfcnJyEB8f77bjKyIMGReQzMnJQUREhJdLQ0RERI4oLCyEWq02WwjaHRQRhoxNYxEREQxDREREfsbdXVzYgZqIiIgUjWGIiIiIFI1hiIiIiBSNYYiIiIgUjWGIiIiIFI1hiIiIiBSNYYiIiIgUjWGIiIiIFI1hiIiIiBSNYYiIiIgUjWGIiIiIFI1hiIiIiBSNYYiIiIgUjWGIiIiIFI1hiIiIiBSNYYiIiIgUjWGIiIiIFM3rYWjjxo3o168f4uLiIEkSVq5caXXfp556CpIkYe7cuR4rHxEREQU2r4ehoqIiJCcnY968eTb3W7FiBbZt24a4uDgPlYyIiIiUIMTbBejTpw/69Oljc5/c3Fw8/fTTWLNmDfr27euhkhEREZESeL1myB69Xo9HHnkEL7zwAlq2bOnt4hAREVGA8XrNkD2zZs1CSEgInnnmGYefU1xcjOLiYtP3hYWF7igaERERBQCfrhnasWMH3n//fSxevBiSJDn8vNTUVERGRpq+1Gq1G0tJRERE/synw9CmTZuQn5+PevXqISQkBCEhITh+/DgmT56MBg0aWH3etGnTUFBQYPrKycnxXKGJiIjIr/h0M9kjjzyCXr16mW3r3bs3HnnkEYwePdrq81QqFVQqlbuLR0RERAHA62Ho8uXLOHLkiOn77Oxs7N69G1FRUahXrx6io6PN9q9UqRJiYmLQrFkzTxeViIiIApDXw9D//vc/9OjRw/T9pEmTAAAjR47E4sWLvVQqIiIiUgqvh6Hu3btDCOHw/seOHXNfYYiIiEhxfLoDNREREZG7MQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaJ5PQxt3LgR/fr1Q1xcHCRJwsqVK02P3bhxA1OmTEHr1q1RpUoVxMXF4dFHH8XJkye9V2AiIiIKKF4PQ0VFRUhOTsa8efPKPXblyhXs3LkTr7zyCnbu3In09HQcOnQI/fv390JJiYiIKBBJQgjh7UIYSZKEFStWYODAgVb32b59O9q3b4/jx4+jXr16Dh23sLAQkZGRKCgoQEREhEylJSIiInfy1P07xG1HdpOCggJIkoTq1atb3ae4uBjFxcWm7wsLCz1QMiIiIvJHXm8mc8a1a9cwZcoUDB8+3GZCTE1NRWRkpOlLrVZ7sJRERETkT/wmDN24cQNDhw6FEAKffPKJzX2nTZuGgoIC01dOTo6HSklERET+xi+ayYxB6Pjx41i/fr3ddkOVSgWVSuWh0hEREZE/8/kwZAxCWVlZyMjIQHR0tLeLRERERAHE62Ho8uXLOHLkiOn77Oxs7N69G1FRUYiNjcWQIUOwc+dOfP/999DpdMjLywMAREVFITQ01FvFJiIiogDh9aH1mZmZ6NGjR7ntI0eOxGuvvYaEhASLz8vIyED37t0deg0OrSciIvI/ihla3717d9jKYz40DRIRERHZo9MBmzYBp04BsbFAly5AcLC3S2WT18MQERERBYj0dODZZ4ETJ25ti48H3n8fGDTIe+Wyw2+G1hMREZEPS08HhgwxD0IAkJtr2J6e7p1yOYBhiIiIiCpGpzPUCFnq2mLcNnGiYT8fxDBEREREFbNpU/kaodKEAHJyDPv5IIYhIiIiqphTp+Tdz8MYhoiIiKhiYmPl3c/DGIaIiIioYrp0MYwakyTLj0sSoFYb9vNBDENERERUMcHBhuHzQPlAZPx+7lyfnW+IYYiIiIgqbtAgYNkyoG5d8+3x8YbtPjzPECddJCIiInkMGgQMGMAZqImIiEjBgoMBB9cO9RUMQ0RERIHED9cG8zaGISIiokDhp2uDeRs7UBMREQUCP14bzNsYhoiIiPydn68N5m0MQ0RERBWl0wGZmUBamuFfT4cOP18bzNvYZ4iIiKgifKGfjp+vDeZtrBkiIiJyla/00/HztcG8jWGIiIjIFb7UT8fP1wbzNoYhIiIiV/hSPx0/XxvM2xiGiIiIXOFr/XT8eG0wb2MHaiIiIlf4Yj8dP10bzNsYhoiIiJyl0xm+oqKA8+ct7yNJhloZT/fT8cO1wUzKLiWSnOyRl2UYIiIicoalofRlsZ+O8yy9r3FxHnlphiEiIiJHGYfSWxpBVlp8vCEIsZ+OY6y9rydPeuTlGYaIiIgcYWsovVFUFLB0qaGZijVCjnHkfXUzjiYjIiJyhL2h9ICh/1BwMIOQMxx5X92MYYiIiMgRvjaUPlD4wPvFZjIiIgp8ZUcpuTLc3BeH0gcCH3i/WDNERESBLT0daNAA6NEDGDHC8G+DBs6vG8YlL9zD3vvqAQxDREQUuORcSDU4GBg+3HZHXw6ld56tpUQ8hGGIiIgCk9wLqaanA++8Y/3x55+Xdyi9TgdkZgJpaYZ/7ZXT2f19ibWlRMp+7yaSEF4cy+YhhYWFiIyMREFBASIiIrxdHCIi8oTMTEOTmD0ZGfZnbNbpDE1rtkY9qdVAdrY8NUOWJiCMjzfUoFgKXM7u76vK9O0qTE5GZFSU2+/frBkiIqLAJOfoL0eGf8u1Qr2zTXtyNgV6282lRHSDh2Ljvhp4ofNWj7wswxAREQUmOUd/eWpYvbNNe3I3BXqR7roOG97fjQlJGxAffhbdnknGwj87e+S1GYaIiCgwyTn6y1PD6u3VQAlhXgPl7P4+Rnddh8y5uzG+9QbUDT+H7hNTMG9fN+Tp66C6dBEjGm7xSDk4zxAREQUm4yilIUMMwad07Ym9hVTLzkvUqZMhWOXmWq6FkWuFemdroFytsZJj3iUX6a7rsPGjvdB+cQnpB5vjtD7F9FgN6QIGNt4PzcNh6DkpGdf0nbAk0v1lYhgiIqLAZRylZKlzsbWFVK11Rh4+3DCazNlg5Qxna6BcqbHyQmfrkmsl2Dhv380A1AL54jbTY8YANPTRcNw1MQmhVW8FymuF19xSnnKEl23YsEHcf//9IjY2VgAQK1asMHtcr9eLV155RcTExIiwsDDRs2dPcfjwYadeo6CgQAAQBQUFMpaciIj8RkmJEBkZQixZYvi3pMTyfsuXCyFJQhjizq0vSTJ8vfCCEPHx5o+p1YbnyVXO+HjLZTCWQ62+VX5n97d3fnKdhxDixtUb4te3d4gnW2wQtaR8s5eLks6Jx5psFD+9vl1cL7pu9Rieun97vWaoqKgIycnJeOyxxzDIQiJ9++238cEHH+DLL79EQkICXnnlFfTu3Rt//vknwsLCvFBiIiLyOzdHKdlkrzOyJAHffgscPQps2eKeJiZnm/ac2d+R85s4ERgwwOXzKblWgswP9kK76DJWHGqBM6KN6bEo6TweaHIAmkfDcddzyahU2Ydm6nZr1HISytQM6fV6ERMTI2bPnm3advHiRaFSqURaWprDx2XNEBER2ZWRYbl2pexXRob7y7J8uXM1UI7s76bzu3H1hlib+j8xtvkGUVM6Y3aoaOmsGNNsg1jz1v9s1gBZo5iaIVuys7ORl5eHXr16mbZFRkaiQ4cO2Lp1Kx588EGLzysuLkZxcbHp+8LCQreXlYiI/JwvrUo/aJChhsbRTs6O7C/j+d24cgMZ7++F9ssirDjcEudEW9Nj0dI5DGp2AJqRVdD9mSRUqtzVsdf1Ip8OQ3l5eQCAOnXqmG2vU6eO6TFLUlNTMWPGDLeWjYiIAoyvrUrvSNOeM/tX8PyMAWjp4itYmZVoFoBqSmcxqNmf0Iyuiu7PJCEkzPcDUGk+HYZcNW3aNEyaNMn0fWFhIdRqtRdLREREPs84L5G7h897iwvnd+PKDax/bw+0X13FiqyWOF8qANWSzmBQ84PQjK6Kbk/7XwAqzafDUExMDADg9OnTiC2VVE+fPo2UlBSrz1OpVFCpVO4uHhERBRJHOiOPGQMsXerxuXlk4WBn6xvFeqybswvar65i5ZGWOC/amXarLZ3BoBYHoXmsGrqOb+3XAag0nw5DCQkJiImJwbp160zhp7CwEL///jvGjRvn3cIREVHgsTYvUVSU4d/p029t88eFUK2cn4iri12dn8ZHU2th5ZDLuGAhAA19vBq6TkhCcGhgBKDSvL5q/eXLl3HkyBEAwG233YY5c+agR48eiIqKQr169TBr1izMnDnTbGj93r17nRpaz1XriYjIKaVnaM7KMg9BRsbalGXL/CsQAYBOhxtr12P/579jzdYIzD75EM4j2vRwnaB8DGrxl6EGaEISgkMrUANWgdmuPXX/9noYyszMRI8ePcptHzlyJBYvXgwhBKZPn46FCxfi4sWL6Ny5Mz7++GM0bdrU4ddgGCIiIpfodECDBtbX/zL2s8nO9osms+uXr+OXd/ZA+3UxVv3dChdFddNjdYLyMTjxIDSPR6LLv1pXLAAZVXC2a8WEIU9gGCIiIpdkZgIW/mAvJyPDuZFfHlRcWGwIQN9cx6q/W6MAtxb7igk6jcEt/4Lm8Uh0HidTADJKTzf0TyobM5yoUfPU/dun+wwREZGCeXExURNfmnvICcWFxVg7ew+0S65j9d+tUYD2psdig/IwuOUhaMZWx51PtkJwaDf5C+CB2a7lxDBERES+xwuLiVrka3MP2WAMQEu/uY7V2a1RWCYADWl1MwA91RpBITHuLcymTdabFgFDIMrJMeznAzVqDENERORbrDWv5OYatltqXnFXLZKPzz107eI1Qw1Q2o1yASgu6BSGtD4MzRM10OmJVu4PQKX5WY0awxAREfkOV5pX3FmL5OzCqR5w7eI1rHl7D7RpJVh9rDUuoYPpsbpBpzAkyRCAOo5thaAQL9VY+VGNGsAO1ERE5Euc7bAsQyddh1gKXGq1IQh5oNnu2sVr+HmWIQD993hrXMKte1l88EkMScqC5sko3PF4SwSFBLm9PHYZR+HZq1GzMwqPHaiJiEh5nGlekaOTrqPNa84unCqDq+evGgLQdzr893gSLpeqATIGoKFPRaHDYy0RFBLn3MHd3TndB2vUbGEYIiIi3+FM80pFO+k627zm7MKpLrh6/ip+mrkH2u/0+P6f1riMO0yPqYNzMSQ5C5ono10LQEae6pxubTbv+HiP1ag5is1kRETkO5xpXlm6FBgxwv4xlywBhg833+ap5jUHXDl7BT/N2gPtdwLf5yShCFVNj9ULPoEhKUegeaom2o9KrHgTmDfOmzNQ+waGISIiP2K8YQOWm1eMN2xXJ0T0gVmlr5y9gh9T90CrFfjBQgDSpByBZlxNtB/dElKQJM+L+sB5O4t9hoiISJkcbV5xddi7l+bAKcovwo8z90KrBX44kYQr6Gh6rH7wCQxNOYTRPf9B8yQVpLpxQJcWgFxBCPC7uX88iWGIiIh8jyMdll3tpOvBOXCMAWjpUuDHXPMA1CAkB5rbjkIzvjbaVTkI6bmJwNtu7MfjZ3P/eBLDEBER+SZHOiy70knXzXPgFOUX4YfUPdBqJfyQm4yrpQJQQsg/0LT5G5rxtdH24RaQgtQ3mwU1zk0y6Qo5ztsXlkhxA/YZIiIi/+fMTVqmOXBKu5x3GT+k7oV2uYQfc5NxFZVNjzUMOQ5N22xoJtRBmxHNzfsAebIfT0XP2wtLpLDPEBERkaOcGfYu0xw4l/Mu4/u39kK7PAg/nkzGNXQyPWYMQEOfroPbhjeHFFTf8kHc0Y/HWjCsyHm7skSKH2EYIiLyRQHaHOEznGleK3UtrlSKxOqMali6ohJ+OmUegBqFHIemnaEGyGYAKk3ufjz2am9caVb0sxXoXcFmMiIiX+MrK7Yrgb3QmZ4O/YSnEXTqpGlTDuLxLN7HCgxC40rHoGl3DJqnY5AyrJnzw+BdnR7AEmfmEHImbMtZRidxniEZMQwRkd/wockAlazwRCF2j/kIXda8DAGg9FSHekiQIHBs0gdoMHtCxeYBkqv/kjv7HqWluT65ZQV56v7tA6u5ERERAPvNEYChOUKn82ixlKLwRCG+HrcZA2J/R4w6GAlrPikXhAAgCAKSJCFBOxuS0FfsRY39eIBbgdfImTW8nOl75Cw/W4HeFQxDRES+wp03NLKo4J8CUwCqpVbhkfl3YnVeB7THdqhxwvpNUs5rYezHU7eu+fb4eMdrAt05h5BxcsuyYc1IkgC1uvzkln6EHaiJiHwFJ8XziIJ/CrD6rX3QrqyENadTcB13mh5rHnoUmg45GNN2JzDXgYOtWydP53ZHJpm0xZ21N362Ar0r2GeIiMhXeLGjaqC7eLwAq9/cB+2qUKzNT8Z1qEyPtQg9Cs0dOdBMrIuWAxob+gA5ei0A3+jc7oa5k8qx1LFfrXbrCvTsQC0jhiEikoVcw92tHccTNzQFuXi8AKve2Aft6lCszU/BDYSaHktUHYHmjhPQPBePlgMal3+yvWtRmq90bnd0gduK8PCUDx67fwsFKCgoEABEQUGBt4tCRP5q+XIh4uOFMNxmDF/x8Ybtch5n+XIhJMnwVXof4zZnX09hzv99QSx6fJO4r9YfohKKzd7CRFWWmN4tQ+xfmeXYwaxdC0tfkiSEWi1ESYl7T9CRMpf9fKnVfvu58dT9mzVDRET2yDXc3dHjeKE5wp9dyL6Ila/vg/b7MPx6JtmsBqilKguajrnQPBePxP4WaoDssXQtbPGFJswAmrCTzWQyYhgiIpfJNX+Ls8cJoBuaO5w/egEr39hvCEBnU1CCSqbHWqmyoOmUC80kNVrc36jiL6bTAa+9Brzxhv193TDXjpJxbTIiIl8g19pRzh7HmbW2FOJc1nmsfPMAtN+HY925ZJTg1lDu1mGHoel0EprJ9dD8viYAmsj3wsHBQM+ejoUhP55rR8kYhoiIbJFruDuHzbvkXNZ5rHjjALQ/hGN9mQCUFHYImjtPQTO5Hpr1aQqgqfsKYpxrx17ndj+ea0fJGIaIiGyRa/4WBcziK5ezh85h5Vt/QvtDONadS4GuVABKDjsETedT0DxfH017NwPQzDOFUsBcO0rGPkNERLbIvXYUh81bdPbQOax48wC0P1TB+vPJ0JX6W908ACV4sZQInM7tftInjX2GiIh8gVw1AqxZuOXmjbjwf4exYc1VfLSjI9ZdaAMdupp2SQn/C5rOedC80ABN7vZgDZA9FZ0p2hdYCnS+MHGkF7FmiIjIEXLVCARKzYKLCj5YjOBpL6LqlTOmbTmIx7N4H8fCW0DT5TQ0Lyagcc/6XixlAJNrmggP4dB6GTEMEZEs3D0DdYDKP3AG6W8exKnVf2B60YsAhNkCqIabkARpuW/diAOOXNNEeBDDkIwYhoiIPOv0fkMA0v5cDRsuJgEAjqEB6lpbCd4Hb8QBxw/XvmOfISIi8it5e/OR/tZBaNdEYOPFJOhL9QF6QrUY6mIZ5msi13F6B6sYhoiIAoGXmt7y9uZj+RsHoV0biY0FSRDoZnrs9ioHoOl2BkOmNEJCrgoY4cABnb0RK6zJsUI4vYNVDENElijhF6wSzlEpPDw6yFYAal9lPzTdz2LI1MZo0LnlrSdluuFGzFFRzuHEkda5dRlYH8FV68kpcq1O7suUcI7eVFIiREaGEEuWGP5150rmxpXVLa2iLuMq9yd35YkPh2SKrpG7hASd2Ut1qLJPvHN/hjj2W471A5SUGD5j1laAd3bVdw+dt0s8ef2dZXzfyr53vvC+WeCp+zfDEFFpvvwLVi5KOEdv8mTQNAYMS+HClYBRRu6OU+KDwZmiS8Ru1wJQWXLdiN183g6zFHr84Q8NS2VUq32rjDcxDN1UUlIi/v3vf4sGDRqIsLAw0bBhQ/F///d/Qq/XO3wMhiFyiK/8gnUnJZyjN3k6aGZkWL+Wpb8yMhw+5IntJ8X7gzJFZwsB6I6qe8W7/TPE8S0n7B/IWu2IHDdiN5y30yydR3S09Z8rX/tDw5drr0phGLrpzTffFNHR0eL7778X2dnZQqvViqpVq4r333/f4WMwDJFDfOEXrLsp4Ry9xRtBc8kSx67nkiU2D3Ni+0kx94FMcWe1PeWe2rHqXjFnQIb4Z1uu4+WyVztS0RuxTOftMmuh19YX/9Bwiafu3z7fgXrLli0YMGAA+vbtCwBo0KAB0tLS8Mcff3i5ZBRwlDDsVAnn6C2bNlmfzA4w3BLlHjpegdFBJ7afwrK3DkO7rga2XEoCcGufTtX2QtPzAgZPbQJ1h9bOlcnaDMe5uYbtxhmOK/IeeHNUlE5n6LRd9vzsccf1J9n4fBjq1KkTFi5ciMOHD6Np06bYs2cPfvvtN8yZM8fbRaNAo4Rhp0o4R2+RK2g6M8rPydFBOb+fxLLUw9Cui8bWy61ROgDdWW0vNL0uYPC0poi/Pcmxc7FUdmtBwbjtiSeAyEhDIHB19GJFRkVVdBSlvdBrD//Q8E1urXeSgU6nE1OmTBGSJImQkBAhSZJ46623bD7n2rVroqCgwPSVk5PDZjKyT+7RLr5ICefoLXI0QbrS+dZOp+T8txaKd/tniDuq7jV/GDrROWK3eH9Qpjix/aRn3wNHzsseVzpjy9G52dEmOjZBy4J9hm5KS0sT8fHxIi0tTezdu1d89dVXIioqSixevNjqc6ZPny4AlPtiGCK7/GzYqUuUcI7eUNGgWZHO1xZu8gVhtcQU1XvlAlCXiN3ig8GZInfHKfnfA2eCghyfN2c6Y9vq5+NMOZwJfPxDo8IYhm6Kj48XH330kdm2119/XTRr1szqc1gzRBXiR8NOXaaEc/QGV4OmDJ2vj204Jr694z3xb9Us0Q0ZIgglpgDUNXKX+HCImwJQac4GBTkCgiOdse29v8bPvyPlsBd63RX8FIodqG+6cuUKgoLMl/ULDg6GXq+3+hyVSgWVSuXuolGgGjQIGDAg8GZnLttX4uhRYMuWwDpHbxs0yNBB2NKsyHPnWp8V2cXO18d+O4FlM49Am1kTfxS1AjARACBBjy6Re6G5uwCDX26O2JSUCp6Yg+z15SnLynk5JTjY/nMd6efjaDmCgw0zXA8ZYuibVPo8jd9HRwPnzt3abu/6k9f5fBjq168f3nzzTdSrVw8tW7bErl27MGfOHDz22GPeLhoFMkd+wfoTW8sWDB/uvXIFIlfCtBOdr7M35mDZrKPQbqiF7UUtAcQDMASgrpF7obmnAINe8mAAKs1WULDF3Z2Kc3Pl3c9e6A3EP6YCnM+HoQ8//BCvvPIK/vWvfyE/Px9xcXF48skn8eqrr3q7aET+wdGhziQfZ8O0g6P3nhxdjIXFagBqAEAQdOhafS80vQsx6KUWiElKcbqosrMWFGxx9+jFM2fk3Q+wH3oD6Y8pBZCEcDS6+6/CwkJERkaioKAAERER3i4OkefodECDBtZvSsYhyNnZ/MvVm4zXyUrzkh4STiAeCcgGAHSrvheaey9h0MstUKdVLQ8X1kE6HZCZCQwdCpw/b3kfT33+vvkGePhh+/t9/TXw0EPuKwc5zVP37yD7uxCR33KmLwp5z83mJSEMQ19L0wOQILAu7D58PGwjTu47j/UXbsO4tK6+G4QAwzn17Al8+qkh9EiS+ePG7+fOdX8Qr1tX3v0o4DAMEQUyzjjt846sO47U3plo83ALDMYynLjZB8goCIAEYPS1BXhy86Ooc9jPgqux2axs0IiP91wTrbFjty1qteVJGkkRnOozlJOTA7Va7a6yEJHcOOO0T8r65Ri0s49B+1sMdl9tDqA+AGAvmqCoRjzeiF+Advu+gFT2if7az8vbIzRLd+wGyo8AAzxTQ0U+y6k+Q1WqVMHkyZMxdepUVK5c2Z3lkhX7DJFi2emLwj5DnlM+ABkEowR3Re2Bpm8RBk5LRK2mNXy/n1dFl7TwFkujKtVqDnv3YZ66fzsVhrZs2YLnnnsOJ06cwJtvvolRo0a5rWByYhgiRTOOJgMs/0Xsb7UMfuTwmmxo3zkO7W+x2HOtmWl7MErQM3o3NH2vYuBLiajZLPrWkzIzgR497B88I8M7I5ZsTdPgD58jfw1yCuWTYcjoq6++wssvv4zatWtj7ty56OLj7awMQ6R4/IvYYw799De07/4D7eZY7HU0AJWWlgaMGGH/hZYs8fwcUdamaWCwJjfx6TAEGGaGnjlzJubMmYN7770Xs2fPRkJCgtzlkwXDEBH4F7Eb/fXjzQC0JQ77rjU1bQ/BDfSM3gPN/Vcx8OWWiG4SZf9gvlozxGkayAv8Igzt3LkTy5Ytw4cffojQ0FA8/fTTePXVV1G1alW5y1khDENEJLeD3x+Fdk4OtFvqYn9xE9P2ENxAr5q7obn/Ggb+uxWiGtVw7sDu6OclRxD21ZBGAc1T92+nRpPNnz8f27dvx/bt23Hw4EEEBQWhVatWeOqpp5CcnIxvv/0WiYmJSE9PR7t27dxVZiIir/hz9RFo3zsB7da6OFDcBEAjAIYAdHctQwAa8HIrRDW63fUXsbf2FeDcyCe5+vhwmgYKYE7VDKnVanTo0AF33HEH7rjjDrRt2xbh4eFm+7z11ltYsmQJ9u/fL3thXcWaISIXsWkNB1bdDEDb4vFncWPT9kq4jrtr7YGmfzEGvNwKNRKqy/vCcvTzkrOPD2uGyAt8vpnMmtOnTyMuLg46nU7Ow1YIwxCRC/x91FAFHFh1BEvnnIB2mxoHrzcyba+E67in9m5o+l9H/5fcEIDKqkgYlbuPD6dpIC/wyWYyR9SuXRvr16+X+7BE5EkKW9xV6IWhBmhu7s0A1BiAoRYoFMW4p/YeaAZcR/+XW6N6/faeK5izC76W5sxSLI68htzNd0Q+RPYwJEkSunXrJvdhiagsdzVh6XSGGiFLf/0LYbjxTZxomFHYj298Qi+wf0UWtO+fhPZ3Nf663gSAoSN0KIrRu85uaAbeQP+XWiOyngcDkFzc0cfH2or08fGBMU0Dm4UVS/YwREQe4M4mLLlrFHyI0AvsS78ZgP6oh0PXmwIwDIUvH4A6eLewFeWupVi8vbSGuyi4WZgYhoj8j7ubsAJs1JDQC+xddhjaD05B+0d9HL5xKwCpcA29Y/ZAM7AE/aa18v8AVJpxcVJ7fXxcmTS3Is13vsiRn6lADIBkInsHal/EDtTkce5swnL3xHcBMGpI6AX2aG8GoO31kXXj1oSwKlzDvTF7oHmgBP1eao2I+AD+neDppVj8sZnJkZ+pqCggPJy1Rl7gsfu3UICCggIBQBQUFHi7KKQEy5cLER8vhOH2Y/iKjzdsr6iMDPPjWvvKyHD9NUpKDOWVJMvHliQh1GrDfu5WUmI4lyVLDP/aeE29Ti92LjkopnXMEI0rZZsVWYWrYmDsVvHNv34TBTlO/B5w4vV9lqXPo1otz+fR3uvI9bl3J0d/piz9HEiS75+fn/PU/ZthiEhOy5dbDhFy/eJcssSxX9RLlshzHmXPxZM3AAdursYANPWO8gEoDFfEA7FbxZIJm0VhbqFbXt9vuDvUuftz706O/kx5+w8DhfLU/ZvNZBS4PF1lH2hNWN5c3NXGZIECQPYzc/Hp70nQ/i8BR0vqmx4Ow1XcF7cHmsF69J3aGtXiqsn++gACbmqBCvH3Ncsc/ZmyxYebjP2d30666IsYhhTIGyNDPBFUPD3xnTf6gNi5ueoh4QTikYBs6BGMcFzBfXX3QDNYoO+0JFSNqeDaiP5+c/c0f+9jZu9nyhFLlgDDh8taLDLw1P07yG1HJvIW41/1ZW9mxpEh6enueV1PjMIyTnwH3KqlMHLHxHfGUUPDhxv+9cTN387Q/iAI1EMO/l1zPr6buAX5p/RYdqIjhr3fqeJByIHXN5tagPx/9KGtnylHOTs9AfkchiEKLPYmDAQMEwa6Y7kYd83rUpZx4ru6dc23x8f7ffON0Asc/W67Q/vO+CAKQ9+TKQCV5u83d0/z1OfenWz9TEVHWw9JkmRoOnZlegLyKQxDFFi8+Ve9cV4XT/ziHDQIOHbM0PSwZInh3+xsvwxCQi/wx6IDeOH2TDRUncDj8x1c8d1dN9dAuLl7kic/9+5k6Wfq2DFg4ULD456oiSWv4aSLFFi8+Ve9p9du8uOJ74wBSPvJWSzb3RjHdS1Nj51FJM4G10K07gws3l4rMlmgI9w5WWEgCqQ1yyz9TAX6EiQEgDVDFGjc9Ve9TmfoKJqWZvjXWjNbADdhVZTQC2z7bD8mt8tEg9Bc3DGmFd7d0R3HdfGogssYpt6CZc9vxekzIai5dD4kSfLOX+Oe7pcVCAL9cx9ANbFkGUeTUWBxx2grV0am+eNMvG6gL9Hj9y8OQLvgHJbtaYIc3a2bZRVcRr96e6EZFoR7X0xC5ZqVzZ/szaH9vvD6/oife5IZh9bLiGFIYeRcgoDzzTjNGICWzj+PZXub4IQuzvRYVVxCv/p7oRkWjHunJCM8Ktz2wbx9c/X26xMpHMOQjBiGFEiOv+o534zD9CV6bPv8ALQLLAeg/vX3QjM8BL1fSLIfgIiIbmIYkhHDkEJV9K96f59Mzs30JXps/XQ/tAsvYNnepsjV3+qHVQ2F6N9gnyEAvZiMsOphXiwpEfkrT92/OZqMAldFR1txvply9CV6bFloCEDL9zVFrj7J9JgxAA0dEYJ7XkhGWPU7vVhSIiLHMQwRWePp+WZ8tH+KvkSPzfP3QfvpRSzf3xQnSwWgCBSgf8I+aIZXYgAiIr/FMERkjSfnm/HGWmo26K7rsHnBfkMAOtAMp/TJpsciUICBCbvx5J1/4vYeVVCpYT2gS0efCG5ERK5gnyEiW+QcmWbvNbw8Yk13XYffPtkH7ecFWH6gOfL0dUyPRaIAAxrug+ahUPRumo1K0573meBGRIGLHahlxDBEFWJtZNqcOUDNmhVr1vLyiDXddR02fWwIQOl/lg9AAxvtg+ZhFXpNSoIqQuUzwY2IlIFhSEYMQ2STI311yu5z5gwwaVLFa0e8MGLNGICWflaI9IPNcVpf2/RYdenizQAUhl6TkxFaNbTUEznVABF5FkeTEXmCo311So9MS08Hhg0rXzuSm2uoNXGmdsRDI9Z013XY+NFeaL+4dDMApZgeqyFdwMDG+6F5OAw9JyUjtKqVPlDOLIKrwKkGiMh/MQyRcllr8rEVanQ6Q3iyVKEqhKF2ZOJEYMAAx2pHHB2JlpXl2H6llFwrwcZ5+24GoBbIF7eZHqshXcADjfdD82g47pqYZD0AlcapBogoQDEMkTJdvw489ZTzoUbu2hF7I9aMXnsNaNXKbo1TybUSbPhwL7SLLiP9rxY4UyoARUnn8UCTA4YA9FwyKlV2chScp6caICLyEIYhUp70dODJJ4GzZ63vYy3UOFs7Yq8/knGF9MGD7R/TSo1TybUSZH5gCEArDrXAGdHG9JgxAA0dVRk9nk1yPgCV5smpBoiIPCjI2wVwRG5uLh5++GFER0cjPDwcrVu3xv/+9z9vF4v8kbFpzFYQKq1s+HGmdiQ93dDhuEcPYMQIw78NGhi2lzZoEDBjhu3jlQ5nMASgX2buwBMtNiK28kXcPaUNFv7VFWdELURL5zCm2UaseWsH8i5Xw2eHuuCeaW1RqXIlx8pujTG4AbdGjxkZv587l52nicjv+HzN0IULF3DnnXeiR48e+Omnn1CrVi1kZWWhRo0a3i4a+Rtb/X2sKRt+HK0dOXPGuU7WTZo4VJy98zfjw6eCsOJwS5wTbU3bo6VzGNTsADQjq6D7M0moVLmro2fonEGDDGW31OncmUVwiYh8iM8PrZ86dSo2b96MTTf/InYFh9YTAMeHsQO2h4nbm4hx6VLgueecG4LuYNm6IwMb0B0AUFM6i0HN/oRmdFV0fyYJIWEe/NvGR5cOIaLA4qn7t883k61evRrt2rWDRqNB7dq1cdttt+HTTz+1+Zzi4mIUFhaafRE5PcrJWpOPsXakbl3z7fHxhu01azreydrIWONUtvnpJj0k/AM1/kJzPNliI359eydOXamOBQe7oteLbTwbhIBbUw0MH274l0GIiPyYz4ehv//+G5988gmaNGmCNWvWYNy4cXjmmWfw5ZdfWn1OamoqIiMjTV9qtdqDJSbZ6XSGmpO0NMO/Op1rx3G0v0+tWvbnCho0CDh2zDAZ4pIlhn+zsw3bXRiCfqNYj12dn4YQAnqYByI9JEgQKBj5DE5crYn5f3ZFzxe8EICIiAKUzzeThYaGol27dtiyZYtp2zPPPIPt27dj69atFp9TXFyM4uJi0/eFhYVQq9VsJvNH1iZFnDPHEFqcaaYxzqBsaxh7rVqG1woNtfy4Ixxs8rrx41r8ur06tF9fw8ojrXBB1MADSMf7eBZq3DpfER8Piet+EZECcQbqm2JjY5GYmGi2rUWLFli+fLnV56hUKqhUKncXjdzN2qSIJ04AQ4eab3NkKQzjaKghQwzNUZb6+8yfX7EgBNjtZC0AnA+pjab3tcF5RJu21wnKR2xiNI6OXom41hcRfDYfiI2FxP44RERu5fNh6M4778ShQ4fMth0+fBj169f3UolIVtY64l6/bpgLyNGKS0eXwvDEaCgbocvYBDa25BOcRzTqBOVjcOJBDB0Tic7jWiM4tFvFX5+IiJzi881k27dvR6dOnTBjxgwMHToUf/zxB8aOHYuFCxfioYcecugYihhN5o+je6w1gQ0fDixebBie7gxnFgp18/tVXFiMfY/PRcKKdxGtu3Ue/0CNGdJrCG/VCJrHjQHIx6+THPzx80lEXsdV60v5/vvvMW3aNGRlZSEhIQGTJk3C2LFjHX5+wIchRxcb9SXWmsDkIOMK784oLizG2tl7oF1yHav/bo0CRCIIOnTBJiRKB9GsQTFue7Yr7hyXrIwAZOSPn08i8gkMQzIK6DBkLVQY+8A4s4K6pxg7Mtsafl4RS5YYapecKY+LtRbXLl4zBKC0G1id3RqFiDQ9FhuUhyGtDkEztjrufKo1gkJ8fvCm/Pzx80lEPoNhSEYBG4bshQpnmo2cfd2KNHk4M/mhK5ypGXKh1uLaxWtY8/YeaNNKsPpYa1zCrc9UXNApDGl9GJonaqDTE62UGYCMvPX5JKKAwdFkZJ/cK6g7Qo4mD2cnP3SUswuFWqu1sNAZ2xiAli4pwX+Pt8YldDDtXjfoFIYkGQJQx7GtEBTCVdsBeOfzSUTkAoYhf+bC5H4V4kR4sMnRyQ+d4exCobbWKRMCkCTon3kWq3+PgfY7US4AxQefxJCkLGiejMIdj7dkALLE059PIiIXMQz5M2dWUK8oB8IDJk4EBgywH0bsLXZqS61awEcfAZMnV2xovAO1FkG5JzD37eumtcDUwbkYkpwFzZPR6PBYSwSFxDlX9kBXtvm0dm3HnueOcExE5ASGIX/m6ArqjjYb2SJnk4etyQ+tKT0p4qBBwODBFeu35GBtRCvpANq2ATRP1UT7UYkICqlr/0lKZK35NDoaOH/e/Z9PIqIKYBjyZ47MqOxos5E9cjd5WJv8UK0GHnzQsA6ZrZof40KhLrpWuQbCHNjvw18TId3l+uv4BHfP8WOr+dS4zd2fTyKiCuBoskBg6a9ytVq+GZUBx0eAOTvHj7UbtRtu4FfOXsGPqXug1Qr8lNMSB9AKdZGLINiotfD3kU7unuPHkRFjUVFAWJghHBnJ/fkkooDEofUyCvgwBLj/r397i5z6aHgoyi/CjzP3QqsFfjiRhCuoYnrsiaBPMV//BHBzVfhyZswApk4Ftmzxz5mTPTHHj6Mh+ddfDe+bP76PROQ1DEMycvnN5BIC5ow3V8Byk4e3JtArc52KmrXBD2/vh1Yr4cdc8wDUICQHmtuOQjO+Nto90gLSyhXla05KM9ZSGfnLzMmemuMnLQ0YMcL+fs5OhElEBM+FIQXPCGdHerrhZtKjh+GXfY8ehu/T071dMu8x9vOpW6YTcXy8fEFIpzPUNqSlGf4tHUQssXCdLsQl4tu5eViW2xFXUAWNgrMxv/lcHH1qNv5ecwRvb+2C20cmQgqSDGU+dsxQC2StPKUZpxHw9c+BMx3eK8KTIxqJiNyENUOWKGkJAVdqv9xVY+Zk/5arn3+DsDEPA8DNteANjCvDf9PkNXTqUx0N02dDsnVMZ5cH8dEmQTOeqrHx0+ZTIvIPHuvmIhSgoKBAABAFBQX2dy4pESI+XgjDr/byX5IkhFpt2M/fLV9e/lzj4w3bvVEWSbL8fkuSqUyXTl0SaU9vFkNifxM5qCt0Vq6TXpKEiI526JgiI8P69bb1lZHh+ffJUY6ekxznYLx2Zd/rsu8zEZGTnLp/VwCbycryVPOCtxlrv8qeqzeagexM6CgAFI0chyGxm1ErNhjDP+yEM6duIB65Vtt5JSGAc+esTxIJGCaJ1OlcnwHZl2dONs5BJUmWH5ckw4guOeb48UTzKRGRGzEMlaWEJQTszSYN3AoKnmAngEpCoMrlfJzNu4FrCEfjSsfwVJN1FXvN0qHW1f4svtwPxjgHFVA+ELljjh9j36uMDEPTW0aGoWmMQYiI/ADDUFlK6BDqa7VfDgbLcU3WYde3h3D4Wn08uLCnfK9trxalLDlrVdzJ0zU2xokwhw83/Ms+QkTkJxiGyvJk84K3+FDtV+GJQvySdtahfYct7ImUYc0Mo8CcDTDWxMbarkUpy99mTmaNDRGRXQxDZXm6ecEbvFz7VfBPAf7z1G/oH/M7aqlVuPe//0IO4qG398QzZ27935HrFB3teKi1VotS9jr7Yz8Y1tgQEdnEofXWeGKJC2/xwnDoi8cLsPrNfdCuCsXa/GRch8r0WPPQo5je8D8Y9pdhrh+rdTNqdfky2bpOgPOTRJadNqBTJ/+dgZqIyM9xBmoZcQZqCzwwm3TpALQmPwU3EGp6rEXoUWjuyIFmYl20HNAY0sYNrq99Zus6BVqoDeTPJBFRGZ4KQ1y13pYKrozu06ytGl92dXgnXTxegFVv7IN2dSjW5qfgBjqbHktUHYHmjhPQPBePlgMaA2h064mlF/G0Ras1/Fs6BNi6ToMGAQMG+F+AsBR6Vq1y76KrREQKxZohpZOhpuFC9kWsenM/tKtV+OVMslkNUPkAZMXcucBzzzn+ooEcAizVZkVHG+ZNKisQZ0UnIrqJzWQy8vkw5IdNHxeyL2Ll6/ug/T4Mv5YJQC1VWdB0zIXmuXgk9rcRgEr75hvg4YcdL0CghgBrS8HYwiUviChAsZlMKZxcj8ubzh+9gJVv7DcEoLMpKMGt6QVaqbKg6ZQLzSQ1WtzfBEAT5w5edhSXPUIYQsDEiYZmsEAIAbYmw7Sl9LxQgdqsS0TkRgxD3mStFsC4JIYP1HqcyzqPlW8egPb7cKw7l2wWgFqHHYam00loJtdD8/usBCBHa72M8wY5ulgqEHghwN5kmPb486zoRERexDDkLfaWxPBirce5rPNY8cYBaH8Ix/oyASgp7BA0d56CZnI9NOvTFEBT6wdyptbLOG+QpRFu9rg7BBgDXW6uYa6jWrUMNVlyN2dW9Dz8eVZ0IiIvYhjyFkeXxMjMBHrKtPSEDWcPncOKNw9A+0MVrD+fDF2pAJQcdgiazqegeb4+mvZuBqCZ/QO6UutlbYSbPVlZju/rLEuBzkju5kxXw4yxz5A/z4pORORF7EDtLWlpwIgR9veLigI+/dQtzWVnDp7Fijf/hPanKsg4nwxdqWycEv4XNJ3zMGRyfTTtneDcgY2TOloLNPY6/JauiXn2WcujqEqLjzcsOSF3DZojnZklSb7mTHuTYVp7fcAnmlSJiOTmqfs3l+PwFkdrAc6fN9yQ09NledkzB89i4cMbcXf0DsQmVseT33TFr+fbQocQ3BZ+EG/dk4nDa49h15XmeGltd+eDEFDxhWCN8wY99BDwzDP2X+/ECfkXlXW0M7MQhuZMna7ir+noEiOl+ePyIEREPobNZN5i7DDsaC1ABfoP5R84gxVvHYT2p6rIvJAEHbqaHrst/CA0XU5D82ICGvdsAaCF08cvR86FYJs4OCpN7n5DznRmlrMTt73JMP1xAkkiIh/HMOQtZTsM2+LCqKn8A2eQ/uZBaH+uiswLydCXCkBtwg9C09UQgBrdJVMAKk3OhWC9taiss+FKzjBmb9bsQBg5R0TkQxiGvMlYCzB2rKE5zB47N9zT+40BqBo2XEwyC0BtK/8JTdd8aKY0RMPubghApdmr9XKmw6+cx3KGs+FK7jAWyEvBEBH5GIYhbxs0CIiMBHr1sr+vhRtu3t58pL91ENo1EdhYJgC1q/wnNN3yMeTFhmjYPRFAoowFt6F0rZckWV4Idu5cx5p35DyWM5xpxlSrOZKLiMiPcTSZL7A3iqjM6Ku8vflY/sZBaNdGYmNBEkSpfvC3VzkATbczGDKlERK6qj13DpbIuWK8N1af9/RoMqo4P1zahois49pkMnLbmynnL17jjRewWPtxYfZn+GZbI2h/icSmMgGofZX90HQ/iyFTG6NB53hXz8Y95HyPvHGjszXPkLvDGDnHj5a2ISLHMAzJyC1vpjt+8Vo45qXw2piFKXjr6kSzANShyn5oehgCUP07fSwABRpPzUBNrrNWi8d5mIj8GsOQjGR/Myv6i9dGDcfJ7bnY9rwW+7ZfQ+bV9tiIbtDD8NgdVfdBc9c5DJnaBPU6Ormwqa9hcwbJpaKTfBKRz2IYkpGsb2ZFf/FaqP0pqR2LNQnjMPNgf2wubG1WA9Sx6j5oep7D4CkBEICM2JxBcsrMBHr0sL9fRgZH6BH5GU+FIY4mc5YzsyuX/cVrpUYpKD8PffKn43O0hECyKQANmdYU6g6t5T8Hb3JlzTIiW+Sc5JOIFMnvluOYOXMmJEnCxIkTvVMAV3/x6nQoGTcBlirigmDY9mX4OPyzOQdbLrXGcyu7Q90hrqKl9S22lrgwbpNraQtSDm9NzElEAcOvwtD27duxYMECJCUlea8QTv7izfn9JN4bmInxVb9ESP4pSFZ2D4JAtav5UF8/Kk85fVFF1yzT6QxNImlphn8Zmgi4NSdU2fXcjCSJc0ERkU1+E4YuX76Mhx56CJ9++ilq1KjhvYI48Iu3pHYc3pujQ8dq+1DvjjhMWtUd56+FO3b8QK7Kr0hzRnq6oa9Wjx7AiBGGfxs0kG0BW/Jjjixw646JOYkoYPhNGBo/fjz69u2LXg7M1FxcXIzCwkKzrwoz1kosXWpYPgMo94tXwFC5MTT/Q0z6b09su9waEvToErEHmq6nHXudQK7Kd7U5w9jPqGytkrGfEQMRGZe2qVtmkEF8PPuhEZFdftGB+ttvv8XOnTuxfft2h/ZPTU3FjBkz5CuApdFP0dHQ3dAjuPCCaVMO1JiIuViJgegauRuauwswaFozxLVJBnStgAbven6NLV/iyjpj9voZSZKhn9GAAfzLX+nsLXBLRGSFz4ehnJwcPPvss/jll18QFhbm0HOmTZuGSZMmmb4vLCyEWu3i0hRWRj/pz52HBIFXMANH0AR5qAMpIhKDe1/GvJfOIDYlxfw43lpjy5e48h5UZPQeKQ8XuCUiF/h8M9mOHTuQn5+PNm3aICQkBCEhIdiwYQM++OADhISEQGehE61KpUJERITZl0tu1kpYHwEmYYL0Mbpq6iBtTyusL2iL8Uu7ITaljuXjsSrf+feAw6aJiMjNfL5mqGfPnti3b5/ZttGjR6N58+aYMmUKgt1Uk5K9MQfbXliO4SdO2BwBVkecxrh/BQFJtR07MKvynXsPOGyaiIjczOfDULVq1dCqVSuzbVWqVEF0dHS57RX1d+Y/0M76G9qNtbHjSiIeRB0Md+SJztZKsCrf8ffAlX5GRERETvD5MORuR9cfh/btbGg31sHOqy0A1AMABEGH2KqXgMsOHIS1Eu7DvlZERORmilyb7Mi6mwFoUx3sutrCtF8QdOhRYw80fS7jgZdaoHbzKMNcNvZqJbgApPtZGtGnVhuCkBL6WhERKRAXapWR8c18pcf3+O+2Rth9tbnpsWCUoEfUHmj6FOGBlxNRq0VN8ycbR5MBlmslbHV85srs8uL7SUSkKAxDMjK+mUABgAgEowR3Re2Bpm8RHni5JWo2i7Z9AFdqJbgyOxERUYUwDMnI+GbeVSMDw/sFY+BLifYDUFnO1EpYW5ndkdokIiIiAsAwJCtPvZkADKGpQQPrEwWynxEREZFDPHX/9vlJF/1ORVdmL42rtBMREbmd4ofWy06uGZPZ54iIiMgjWDMkNzlmTOYq7URERB7DMCQ344zJkpVFPCTJMBLN2ozJ9lZpBwyrtLPJjIiISBYMQ3IzzpgMlA9EjsyYLGefIyIiIrKLYcgdnFmZvWwn6dxcx16Dq7QTERHJgh2o3cWRldktdZKuVcux43M9NCIiIlkwDAHuW+bB1srs1iZmPHvW9jG5SjsREZGsGIa8MYTdkU7SlnCVdiIiItkpu8+Qt4aw2+skbVSzzKKxlvocERERUYUot2bIXu2MJBmGsA8YIH8tjKOdn+fONXTC5irtREREbqPcMOTMEHZr/X5c5Wjn57p15X9tIiIiMqPcZjK5ls1wRUUnZiQiIiLZKDcMybFshqsqOjEjERERyUa5YcjbtTPOTMxIREREbqPcPkPG2pkhQwzBp3RHak/VzjgyMSMRERG5lXLDEHCrdsbSPENz53qmdsbWxIxERETkdsoMQ2VnnD56FNiyhbUzRERECqS8MGRrxunhw71XLiIiIvIKZXWgXr3aOzNOExERkc9SVhiaMsX2emATJxqa0IiIiEgxlBWGTp60/ljpGaeJiIhIMZQVhhzhjhmniYiIyGcxDJXljhmniYiIyGcpKwzFxXE9MCIiIjKjrDA0a5bhX64HRkRERDcpKwz178/1wIiIiMiM8iZdLL0eWG4ucOYMUKsWEBVlGFbPmiEiIiJFUV4YAgyB5/x5YOpUyzNRs4aIiIhIMZTVTGaUns6ZqImIiAiAEsOQTmdYm4wzURMRERGUGIY2bSpfI1QaZ6ImIiJSFOWFIUdnmOZM1ERERIqgvDDk6AzTnImaiIhIEXw+DKWmpuL2229HtWrVULt2bQwcOBCHDh1y/YBduhhGjXEmaiIiIoIfhKENGzZg/Pjx2LZtG3755RfcuHED99xzD4qKilw7YHCwYfg8wJmoiYiICJIQloZV+a4zZ86gdu3a2LBhA7p27erQcwoLCxEZGYmCggJEREQYNqanG0aVle5MrVYbghDnGSIiIvI6i/dvN/C7SRcLCgoAAFFRUVb3KS4uRnFxsen7wsLC8juVnon61ClDH6EuXVgjREREpDB+VTOk1+vRv39/XLx4Eb/99pvV/V577TXMmDGj3HZ3J0siIiKSj6dqhvwqDI0bNw4//fQTfvvtN8THx1vdz1LNkFqtZhgiIiLyI2wmK2PChAn4/vvvsXHjRptBCABUKhVUKpWHSkZERET+zOfDkBACTz/9NFasWIHMzEwkJCR4u0hEREQUQHw+DI0fPx5LlizBqlWrUK1aNeTl5QEAIiMjER4e7uXSERERkb/z+T5DkpXJERctWoRRo0Y5dAxPtTkSERGRfNhn6CYfz2pERETk53x+BmoiIiIid2IYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkVjGCIiIiJFYxgiIiIiRWMYIiIiIkXzmzA0b948NGjQAGFhYejQoQP++OMPbxeJiIiIAoBfhKHvvvsOkyZNwvTp07Fz504kJyejd+/eyM/P93bRiIiIyM/5RRiaM2cOxo4di9GjRyMxMRHz589H5cqV8cUXX3i7aEREROTnfD4MXb9+HTt27ECvXr1M24KCgtCrVy9s3brViyUjIiKiQBDi7QLYc/bsWeh0OtSpU8dse506dfDXX39ZfE5xcTGKi4tN3xcUFAAACgsL3VdQIiIikpXxvi2EcOvr+HwYckVqaipmzJhRbrtarfZCaYiIiKgizp07h8jISLcd3+fDUM2aNREcHIzTp0+bbT99+jRiYmIsPmfatGmYNGmS6fuLFy+ifv36+Oeff9z6ZvqawsJCqNVq5OTkICIiwtvF8RieN89bCXjePG8lKCgoQL169RAVFeXW1/H5MBQaGoq2bdti3bp1GDhwIABAr9dj3bp1mDBhgsXnqFQqqFSqctsjIyMV9SEyioiI4HkrCM9bWXjeyqLU8w4Kcm8XZ58PQwAwadIkjBw5Eu3atUP79u0xd+5cFBUVYfTo0d4uGhEREfk5vwhDw4YNw5kzZ/Dqq68iLy8PKSkp+Pnnn8t1qiYiIiJyll+EIQCYMGGC1WYxe1QqFaZPn26x6SyQ8bx53krA8+Z5KwHP273nLQl3j1cjIiIi8mE+P+kiERERkTsxDBEREZGiMQwRERGRojEMERERkaL5ZRiaN28eGjRogLCwMHTo0AF//PGHzf21Wi2aN2+OsLAwtG7dGj/++KPZ40IIvPrqq4iNjUV4eDh69eqFrKwsd56CS5w5708//RRdunRBjRo1UKNGDfTq1avc/qNGjYIkSWZf9957r7tPw2nOnPfixYvLnVNYWJjZPoF4vbt3717uvCVJQt++fU37+MP13rhxI/r164e4uDhIkoSVK1fafU5mZibatGkDlUqFxo0bY/HixeX2cfZ3hqc5e97p6em4++67UatWLURERKBjx45Ys2aN2T6vvfZauevdvHlzN56F85w978zMTIuf87y8PLP9Au16W/rZlSQJLVu2NO3jD9c7NTUVt99+O6pVq4batWtj4MCBOHTokN3neeIe7ndh6LvvvsOkSZMwffp07Ny5E8nJyejduzfy8/Mt7r9lyxYMHz4cjz/+OHbt2oWBAwdi4MCB2L9/v2mft99+Gx988AHmz5+P33//HVWqVEHv3r1x7do1T52WXc6ed2ZmJoYPH46MjAxs3boVarUa99xzD3Jzc832u/fee3Hq1CnTV1pamidOx2HOnjdgmKG19DkdP37c7PFAvN7p6elm57x//34EBwdDo9GY7efr17uoqAjJycmYN2+eQ/tnZ2ejb9++6NGjB3bv3o2JEydizJgxZsHAlc+Qpzl73hs3bsTdd9+NH3/8ETt27ECPHj3Qr18/7Nq1y2y/li1bml3v3377zR3Fd5mz52106NAhs/OqXbu26bFAvN7vv/++2fnm5OQgKiqq3M+3r1/vDRs2YPz48di2bRt++eUX3LhxA/fccw+KioqsPsdj93DhZ9q3by/Gjx9v+l6n04m4uDiRmppqcf+hQ4eKvn37mm3r0KGDePLJJ4UQQuj1ehETEyNmz55tevzixYtCpVKJtLQ0N5yBa5w977JKSkpEtWrVxJdffmnaNnLkSDFgwAC5iyorZ8970aJFIjIy0urxlHK933vvPVGtWjVx+fJl0zZ/uN6lARArVqywuc+LL74oWrZsabZt2LBhonfv3qbvK/peepoj521JYmKimDFjhun76dOni+TkZPkK5maOnHdGRoYAIC5cuGB1HyVc7xUrVghJksSxY8dM2/ztegshRH5+vgAgNmzYYHUfT93D/apm6Pr169ixYwd69epl2hYUFIRevXph69atFp+zdetWs/0BoHfv3qb9s7OzkZeXZ7ZPZGQkOnToYPWYnubKeZd15coV3Lhxo9xid5mZmahduzaaNWuGcePG4dy5c7KWvSJcPe/Lly+jfv36UKvVGDBgAA4cOGB6TCnX+/PPP8eDDz6IKlWqmG335evtCns/33K8l/5Ar9fj0qVL5X6+s7KyEBcXh4YNG+Khhx7CP//846USyislJQWxsbG4++67sXnzZtN2pVzvzz//HL169UL9+vXNtvvb9S4oKAAAm4uweuoe7ldh6OzZs9DpdOWW4ahTp065NmOjvLw8m/sb/3XmmJ7mynmXNWXKFMTFxZl9YO6991589dVXWLduHWbNmoUNGzagT58+0Ol0spbfVa6cd7NmzfDFF19g1apV+Prrr6HX69GpUyecOHECgDKu9x9//IH9+/djzJgxZtt9/Xq7wtrPd2FhIa5evSrLz44/eOedd3D58mUMHTrUtK1Dhw5YvHgxfv75Z3zyySfIzs5Gly5dcOnSJS+WtGJiY2Mxf/58LF++HMuXL4darUb37t2xc+dOAPL8rvR1J0+exE8//VTu59vfrrder8fEiRNx5513olWrVlb389Q93G+W4yDXzZw5E99++y0yMzPNOhM/+OCDpv+3bt0aSUlJaNSoETIzM9GzZ09vFLXCOnbsiI4dO5q+79SpE1q0aIEFCxbg9ddf92LJPOfzzz9H69at0b59e7PtgXi9CViyZAlmzJiBVatWmfWd6dOnj+n/SUlJ6NChA+rXr4+lS5fi8ccf90ZRK6xZs2Zo1qyZ6ftOnTrh6NGjeO+99/Cf//zHiyXznC+//BLVq1fHwIEDzbb72/UeP3489u/f7zP9mvyqZqhmzZoIDg7G6dOnzbafPn0aMTExFp8TExNjc3/jv84c09NcOW+jd955BzNnzsTatWuRlJRkc9+GDRuiZs2aOHLkSIXLLIeKnLdRpUqVcNttt5nOKdCvd1FREb799luHfvn52vV2hbWf74iICISHh8vyGfJl3377LcaMGYOlS5eWa0ooq3r16mjatKlfX29L2rdvbzqnQL/eQgh88cUXeOSRRxAaGmpzX1++3hMmTMD333+PjIwMxMfH29zXU/dwvwpDoaGhaNu2LdatW2faptfrsW7dOrPagNI6duxotj8A/PLLL6b9ExISEBMTY7ZPYWEhfv/9d6vH9DRXzhsw9LB//fXX8fPPP6Ndu3Z2X+fEiRM4d+4cYmNjZSl3Rbl63qXpdDrs27fPdE6BfL0BwxDU4uJiPPzww3Zfx9eutyvs/XzL8RnyVWlpaRg9ejTS0tLMplCw5vLlyzh69KhfX29Ldu/ebTqnQL7egGE01pEjRxz6Y8cXr7cQAhMmTMCKFSuwfv16JCQk2H2Ox+7hTnX99gHffvutUKlUYvHixeLPP/8UTzzxhKhevbrIy8sTQgjxyCOPiKlTp5r237x5swgJCRHvvPOOOHjwoJg+fbqoVKmS2Ldvn2mfmTNniurVq4tVq1aJvXv3igEDBoiEhARx9epVj5+fNc6e98yZM0VoaKhYtmyZOHXqlOnr0qVLQgghLl26JJ5//nmxdetWkZ2dLX799VfRpk0b0aRJE3Ht2jWvnKMlzp73jBkzxJo1a8TRo0fFjh07xIMPPijCwsLEgQMHTPsE4vU26ty5sxg2bFi57f5yvS9duiR27doldu3aJQCIOXPmiF27donjx48LIYSYOnWqeOSRR0z7//3336Jy5crihRdeEAcPHhTz5s0TwcHB4ueffzbtY++99AXOnvc333wjQkJCxLx588x+vi9evGjaZ/LkySIzM1NkZ2eLzZs3i169eomaNWuK/Px8j5+fNc6e93vvvSdWrlwpsrKyxL59+8Szzz4rgoKCxK+//mraJxCvt9HDDz8sOnToYPGY/nC9x40bJyIjI0VmZqbZ5/bKlSumfbx1D/e7MCSEEB9++KGoV6+eCA0NFe3btxfbtm0zPdatWzcxcuRIs/2XLl0qmjZtKkJDQ0XLli3FDz/8YPa4Xq8Xr7zyiqhTp45QqVSiZ8+e4tChQ544Fac4c97169cXAMp9TZ8+XQghxJUrV8Q999wjatWqJSpVqiTq168vxo4d61O/MIycOe+JEyea9q1Tp4647777xM6dO82OF4jXWwgh/vrrLwFArF27ttyx/OV6G4dOl/0ynuvIkSNFt27dyj0nJSVFhIaGioYNG4pFixaVO66t99IXOHve3bp1s7m/EIYpBmJjY0VoaKioW7euGDZsmDhy5IhnT8wOZ8971qxZolGjRiIsLExERUWJ7t27i/Xr15c7bqBdbyEMw8XDw8PFwoULLR7TH663pXMGYPYz6617uHSzgERERESK5Fd9hoiIiIjkxjBEREREisYwRERERIrGMERERESKxjBEREREisYwRERERIrGMERERESKxjBEREREisYwRERERIrGMERERESKxjBERH4pLS0N4eHhOHXqlGnb6NGjkZSUhIKCAi+WjIj8DdcmIyK/JIRASkoKunbtig8//BDTp0/HF198gW3btqFu3breLh4R+ZEQbxeAiMgVkiThzTffxJAhQxATE4MPP/wQmzZtYhAiIqexZoiI/FqbNm1w4MABrF27Ft26dfN2cYjID7HPEBH5rZ9//hl//fUXdDod6tSp4+3iEJGfYs0QEfmlnTt3onv37liwYAEWL16MiIgIaLVabxeLiPwQ+wwRkd85duwY+vbti5deegnDhw9Hw4YN0bFjR+zcuRNt2rTxdvGIyM+wZoiI/Mr58+fRqVMndO/eHfPnzzdt79u3L3Q6HX7++Wcvlo6I/BHDEBERESkaO1ATERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGiMQwRERGRojEMERERkaIxDBEREZGi/T/Qezoehl70YgAAAABJRU5ErkJggg==",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Using Autograd to calculate gradients for OLS\n",
"from random import random, seed\n",
"import numpy as np\n",
"import autograd.numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from autograd import grad\n",
"\n",
"def CostOLS(beta):\n",
" return (1.0/n)*np.sum((y-X @ beta)**2)\n",
"\n",
"n = 100\n",
"x = 2*np.random.rand(n,1)\n",
"y = 4+3*x+np.random.randn(n,1)\n",
"\n",
"X = np.c_[np.ones((n,1)), x]\n",
"XT_X = X.T @ X\n",
"theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
"print(\"Own inversion\")\n",
"print(theta_linreg)\n",
"# Hessian matrix\n",
"H = (2.0/n)* XT_X\n",
"EigValues, EigVectors = np.linalg.eig(H)\n",
"print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
"\n",
"theta = np.random.randn(2,1)\n",
"eta = 1.0/np.max(EigValues)\n",
"Niterations = 1000\n",
"# define the gradient\n",
"training_gradient = grad(CostOLS)\n",
"\n",
"for iter in range(Niterations):\n",
" gradients = training_gradient(theta)\n",
" theta -= eta*gradients\n",
"print(\"theta from own gd\")\n",
"print(theta)\n",
"\n",
"xnew = np.array([[0],[2]])\n",
"Xnew = np.c_[np.ones((2,1)), xnew]\n",
"ypredict = Xnew.dot(theta)\n",
"ypredict2 = Xnew.dot(theta_linreg)\n",
"\n",
"plt.plot(xnew, ypredict, \"r-\")\n",
"plt.plot(xnew, ypredict2, \"b-\")\n",
"plt.plot(x, y ,'ro')\n",
"plt.axis([0,2.0,0, 15.0])\n",
"plt.xlabel(r'$x$')\n",
"plt.ylabel(r'$y$')\n",
"plt.title(r'Random numbers ')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "1bc83f33",
"metadata": {},
"source": [
"## Same code but now with momentum gradient descent"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "dc2a3f65",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Own inversion\n",
"[[4.]\n",
" [3.]]\n",
"Eigenvalues of Hessian Matrix:[0.32280554 4.06602883]\n",
"0 [-16.21191032] [-17.88160136]\n",
"1 [-0.05113442] [0.04607198]\n",
"2 [-0.04707482] [0.04241429]\n",
"3 [-0.04333751] [0.03904698]\n",
"4 [-0.0398969] [0.03594701]\n",
"5 [-0.03672945] [0.03309314]\n",
"6 [-0.03381347] [0.03046585]\n",
"7 [-0.03112899] [0.02804714]\n",
"8 [-0.02865763] [0.02582045]\n",
"9 [-0.02638248] [0.02377054]\n",
"10 [-0.02428795] [0.02188338]\n",
"11 [-0.02235971] [0.02014604]\n",
"12 [-0.02058455] [0.01854663]\n",
"13 [-0.01895033] [0.0170742]\n",
"14 [-0.01744584] [0.01571866]\n",
"15 [-0.0160608] [0.01447074]\n",
"16 [-0.01478572] [0.0133219]\n",
"17 [-0.01361187] [0.01226426]\n",
"18 [-0.01253121] [0.01129059]\n",
"19 [-0.01153635] [0.01039422]\n",
"20 [-0.01062047] [0.00956901]\n",
"21 [-0.0097773] [0.00880932]\n",
"22 [-0.00900107] [0.00810994]\n",
"23 [-0.00828647] [0.00746609]\n",
"24 [-0.0076286] [0.00687335]\n",
"25 [-0.00702296] [0.00632767]\n",
"26 [-0.0064654] [0.00582531]\n",
"27 [-0.00595211] [0.00536283]\n",
"28 [-0.00547956] [0.00493707]\n",
"29 [-0.00504454] [0.00454511]\n",
"theta from own gd\n",
"[[3.98561349]\n",
" [3.01296221]]\n",
"0 [-0.00464405] [0.00418427]\n",
"1 [-0.00427535] [0.00385208]\n",
"2 [-0.00382532] [0.0034466]\n",
"3 [-0.00338661] [0.00305133]\n",
"4 [-0.00298614] [0.0026905]\n",
"5 [-0.00262892] [0.00236865]\n",
"6 [-0.00231304] [0.00208405]\n",
"7 [-0.00203465] [0.00183321]\n",
"8 [-0.00178959] [0.00161242]\n",
"9 [-0.001574] [0.00141817]\n",
"10 [-0.00138436] [0.00124731]\n",
"11 [-0.00121757] [0.00109702]\n",
"12 [-0.00107086] [0.00096484]\n",
"13 [-0.00094183] [0.00084859]\n",
"14 [-0.00082835] [0.00074634]\n",
"15 [-0.00072855] [0.00065642]\n",
"16 [-0.00064076] [0.00057733]\n",
"17 [-0.00056356] [0.00050776]\n",
"18 [-0.00049565] [0.00044658]\n",
"19 [-0.00043593] [0.00039277]\n",
"20 [-0.00038341] [0.00034545]\n",
"21 [-0.00033721] [0.00030383]\n",
"22 [-0.00029658] [0.00026722]\n",
"23 [-0.00026085] [0.00023502]\n",
"24 [-0.00022942] [0.0002067]\n",
"25 [-0.00020177] [0.0001818]\n",
"26 [-0.00017746] [0.00015989]\n",
"27 [-0.00015608] [0.00014063]\n",
"28 [-0.00013727] [0.00012368]\n",
"29 [-0.00012073] [0.00010878]\n",
"theta from own gd wth momentum\n",
"[[3.99967105]\n",
" [3.00029638]]\n"
]
}
],
"source": [
"# Using Autograd to calculate gradients for OLS\n",
"from random import random, seed\n",
"import numpy as np\n",
"import autograd.numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from autograd import grad\n",
"\n",
"def CostOLS(beta):\n",
" return (1.0/n)*np.sum((y-X @ beta)**2)\n",
"\n",
"n = 100\n",
"x = 2*np.random.rand(n,1)\n",
"y = 4+3*x#+np.random.randn(n,1)\n",
"\n",
"X = np.c_[np.ones((n,1)), x]\n",
"XT_X = X.T @ X\n",
"theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
"print(\"Own inversion\")\n",
"print(theta_linreg)\n",
"# Hessian matrix\n",
"H = (2.0/n)* XT_X\n",
"EigValues, EigVectors = np.linalg.eig(H)\n",
"print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
"\n",
"theta = np.random.randn(2,1)\n",
"eta = 1.0/np.max(EigValues)\n",
"Niterations = 30\n",
"\n",
"# define the gradient\n",
"training_gradient = grad(CostOLS)\n",
"\n",
"for iter in range(Niterations):\n",
" gradients = training_gradient(theta)\n",
" theta -= eta*gradients\n",
" print(iter,gradients[0],gradients[1])\n",
"print(\"theta from own gd\")\n",
"print(theta)\n",
"\n",
"# Now improve with momentum gradient descent\n",
"change = 0.0\n",
"delta_momentum = 0.3\n",
"for iter in range(Niterations):\n",
" # calculate gradient\n",
" gradients = training_gradient(theta)\n",
" # calculate update\n",
" new_change = eta*gradients+delta_momentum*change\n",
" # take a step\n",
" theta -= new_change\n",
" # save the change\n",
" change = new_change\n",
" print(iter,gradients[0],gradients[1])\n",
"print(\"theta from own gd wth momentum\")\n",
"print(theta)"
]
},
{
"cell_type": "markdown",
"id": "0ef007d0",
"metadata": {},
"source": [
"## But noen of these can compete with Newton's method"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "0e498aa4",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Own inversion\n",
"[[3.8324348]\n",
" [3.0346417]]\n",
"Eigenvalues of Hessian Matrix:[0.36106642 4.2729462 ]\n",
"0 [-12.98689762] [-16.40152131]\n",
"1 [-1.54737334e-14] [-3.53431592e-14]\n",
"2 [4.59701721e-16] [5.2369504e-16]\n",
"3 [4.59701721e-16] [5.2369504e-16]\n",
"4 [4.59701721e-16] [5.2369504e-16]\n",
"beta from own Newton code\n",
"[[3.8324348]\n",
" [3.0346417]]\n"
]
}
],
"source": [
"# Using Newton's method\n",
"from random import random, seed\n",
"import numpy as np\n",
"import autograd.numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from autograd import grad\n",
"\n",
"def CostOLS(beta):\n",
" return (1.0/n)*np.sum((y-X @ beta)**2)\n",
"\n",
"n = 100\n",
"x = 2*np.random.rand(n,1)\n",
"y = 4+3*x+np.random.randn(n,1)\n",
"\n",
"X = np.c_[np.ones((n,1)), x]\n",
"XT_X = X.T @ X\n",
"beta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
"print(\"Own inversion\")\n",
"print(beta_linreg)\n",
"# Hessian matrix\n",
"H = (2.0/n)* XT_X\n",
"# Note that here the Hessian does not depend on the parameters beta\n",
"invH = np.linalg.pinv(H)\n",
"EigValues, EigVectors = np.linalg.eig(H)\n",
"print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
"\n",
"beta = np.random.randn(2,1)\n",
"Niterations = 5\n",
"\n",
"# define the gradient\n",
"training_gradient = grad(CostOLS)\n",
"\n",
"for iter in range(Niterations):\n",
" gradients = training_gradient(beta)\n",
" beta -= invH @ gradients\n",
" print(iter,gradients[0],gradients[1])\n",
"print(\"beta from own Newton code\")\n",
"print(beta)"
]
},
{
"cell_type": "markdown",
"id": "40292cf3",
"metadata": {},
"source": [
"## Including Stochastic Gradient Descent with Autograd\n",
"In this code we include the stochastic gradient descent approach discussed above. Note here that we specify which argument we are taking the derivative with respect to when using **autograd**."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "fa819b9d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Own inversion\n",
"[[3.9972423 ]\n",
" [3.14884659]]\n",
"Eigenvalues of Hessian Matrix:[0.33571966 4.32437259]\n",
"theta from own gd\n",
"[[3.9972423 ]\n",
" [3.14884659]]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAHHCAYAAAC88FzIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABZhklEQVR4nO3deXgT1foH8O+0pWlZWijQjZYdgdICsooCFqkgAhKQRVAEFK8LLghyXa6y/K5aERBcUPCqgCKoQAmIsgktVEVBEdn3gmUpZW1LC4Um5/dHSCBt0mbPTOb7eZ4+NTOTyZkkZV7Pec97JCGEABEREZFKBfi6AURERES+xGCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiOw2cuRI1K9f39fNkLWMjAxIkoSlS5f6uilEZCcGQ0QyNH/+fEiSZP4JCgpCnTp1MHLkSJw8edLXzSMi8itBvm4AEdn2f//3f2jQoAGuXr2K3377DfPnz8fPP/+M3bt3IyQkxNfNIyLyCwyGiGSsV69eaNeuHQBg9OjRqFWrFqZOnYqVK1di8ODBPm4d+VJhYSGqVKni62YQ+QUOkxEpSJcuXQAAR44cMW+7du0aJk6ciLZt2yI8PBxVqlRBly5dkJ6ebvHcY8eOQZIkTJ8+HZ9++ikaNWoEjUaD9u3bY9u2bWVeS6fTITExESEhIUhMTMTy5cuttqmwsBDjx49HfHw8NBoNmjZtiunTp0MIYXGcJEl49tlnsWTJEiQkJCA0NBSdOnXCrl27AABz585F48aNERISguTkZBw7dqzC92Py5MmQJAmHDx/GyJEjUb16dYSHh2PUqFEoKioqc+3z588vcw5JkjB58uQy5zx48CAeeeQRhIeHo3bt2njjjTcghEB2djb69euHsLAwREdHY8aMGVbbptfr8dprryE6OhpVqlTBAw88gOzs7DLH/f7777jvvvsQHh6OypUr4+6778Yvv/xi9Tr37t2LYcOGoUaNGujcuTMAICcnB6NGjUJcXBw0Gg1iYmLQr18/u94/IjJizxCRgphucDVq1DBvy8/Px2effYahQ4fiiSeeQEFBAT7//HP07NkTW7duRevWrS3OsWjRIhQUFODJJ5+EJEl49913MWDAABw9ehSVKlUCAKxbtw4PPvggEhISkJqaivPnz5tvuLcSQuCBBx5Aeno6Hn/8cbRu3Rpr167FhAkTcPLkScycOdPi+MzMTKxcuRJjxowBAKSmpqJPnz7497//jY8//hjPPPMMLl68iHfffRePPfYYNm7caNf7MnjwYDRo0ACpqanYvn07PvvsM0RGRmLq1KmOvL0WhgwZgubNm+Odd97BDz/8gDfffBMRERGYO3cu7rnnHkydOhVff/01XnrpJbRv3x5du3a1eP5bb70FSZLw8ssvIzc3F7NmzUJKSgp27NiB0NBQAMDGjRvRq1cvtG3bFpMmTUJAQADmzZuHe+65B5mZmejQoYPFOQcNGoQmTZrg7bffNgebDz74IPbs2YPnnnsO9evXR25uLtavX49//vmHye5E9hJEJDvz5s0TAMRPP/0kzp49K7Kzs8XSpUtF7dq1hUajEdnZ2eZjS0pKRHFxscXzL168KKKiosRjjz1m3paVlSUAiJo1a4oLFy6Yt69YsUIAEN9//715W+vWrUVMTIy4dOmSedu6desEAFGvXj3zNp1OJwCIN9980+L1Bw4cKCRJEocPHzZvAyA0Go3Iysoyb5s7d64AIKKjo0V+fr55+6uvvioAWBxrzaRJkwQAi+sUQoj+/fuLmjVrlrn2efPmlTkHADFp0qQy5/zXv/5l3lZSUiLi4uKEJEninXfeMW+/ePGiCA0NFSNGjDBvS09PFwBEnTp1LK7pu+++EwDE+++/L4QQwmAwiCZNmoiePXsKg8FgPq6oqEg0aNBA3HvvvWXaNHToUIu2X7x4UQAQ06ZNK/d9IqLycZiMSMZSUlJQu3ZtxMfHY+DAgahSpQpWrlxp0UMTGBiI4OBgAIDBYMCFCxdQUlKCdu3aYfv27WXOOWTIEIueJdPQ29GjRwEAp0+fxo4dOzBixAiEh4ebj7v33nuRkJBgca4ff/wRgYGBeP755y22jx8/HkIIrF692mJ79+7dLXorOnbsCMDYu1GtWrUy201tqshTTz1l8bhLly44f/488vPz7Xq+NaNHjzb/d2BgINq1awchBB5//HHz9urVq6Np06ZW2/noo49aXNPAgQMRExODH3/8EQCwY8cOHDp0CMOGDcP58+dx7tw5nDt3DoWFhejevTs2b94Mg8FQ7nWGhoYiODgYGRkZuHjxotPXSqR2HCYjkrHZs2fjtttuQ15eHr744gts3rwZGo2mzHELFizAjBkzsH//fly/ft28vUGDBmWOrVu3rsVjU2BkupkeP34cANCkSZMyz23atKlFgHX8+HHExsZa3PQBoHnz5hbnsvXapmArPj7e6nZ7b/DlXVNYWJhd56jonOHh4QgJCUGtWrXKbD9//nyZ55d+/yRJQuPGjc1DnYcOHQIAjBgxwmYb8vLyLALX0p+nRqPB1KlTMX78eERFReGOO+5Anz598OijjyI6OrriiyQiAAyGiGStQ4cO5tlkWq0WnTt3xrBhw3DgwAFUrVoVALBw4UKMHDkSWq0WEyZMQGRkJAIDA5GammqRaG0SGBho9bVEqYRnT7D12q62qaLnS5Jkdb9er3fonO5870y9PtOmTSuT12Vi+oxNTLlGtxo7diz69u0LnU6HtWvX4o033kBqaio2btyI22+/3eF2EakRgyEihTAFON26dcNHH32EV155BQCwdOlSNGzYEGlpaRY3/UmTJjn1OvXq1QNws+fiVgcOHChz7E8//YSCggKL3qH9+/dbnMvXTL0rly5dstheuufKnUq/f0IIHD58GC1btgQANGrUCAAQFhaGlJQUl16rUaNGGD9+PMaPH49Dhw6hdevWmDFjBhYuXOjSeYnUgjlDRAqSnJyMDh06YNasWbh69SqAm70Vt/ZO/P7779iyZYtTrxETE4PWrVtjwYIFyMvLM29fv3499u7da3Hs/fffD71ej48++shi+8yZMyFJEnr16uVUG9wtLCwMtWrVwubNmy22f/zxxx57zS+//BIFBQXmx0uXLsXp06fN70nbtm3RqFEjTJ8+HZcvXy7z/LNnz1b4GkVFRebvgUmjRo1QrVo1FBcXu3gFROrBniEihZkwYQIGDRqE+fPn46mnnkKfPn2QlpaG/v37o3fv3sjKysKcOXOQkJBg9SZrj9TUVPTu3RudO3fGY489hgsXLuDDDz9EixYtLM7Zt29fdOvWDf/5z39w7NgxtGrVCuvWrcOKFSswduxYc++HHIwePRrvvPMORo8ejXbt2mHz5s04ePCgx14vIiICnTt3xqhRo3DmzBnMmjULjRs3xhNPPAEACAgIwGeffYZevXqhRYsWGDVqFOrUqYOTJ08iPT0dYWFh+P7778t9jYMHD6J79+4YPHgwEhISEBQUhOXLl+PMmTN46KGHPHZtRP6GwRCRwgwYMMDco/DEE09g5MiRyMnJwdy5c7F27VokJCRg4cKFWLJkCTIyMpx6jfvuuw9LlizB66+/jldffRWNGjXCvHnzsGLFCotzBgQEYOXKlZg4cSK+/fZbzJs3D/Xr18e0adMwfvx491ywm0ycOBFnz57F0qVL8d1336FXr15YvXo1IiMjPfJ6r732Gnbu3InU1FQUFBSge/fu+Pjjj1G5cmXzMcnJydiyZQv++9//4qOPPsLly5cRHR2Njh074sknn6zwNeLj4zF06FBs2LABX331FYKCgtCsWTN89913ePDBBz1yXUT+SBLeyJokIiIikinmDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1VdQZMhgMOHXqFKpVq2ZzjSIiIiKSFyEECgoKEBsbi4AAz/XfqCIYOnXqVJlVsYmIiEgZsrOzERcX57HzqyIYMi0gmZ2djbCwMB+3hoiIiOyRn5+P+Ph4i4WgPUEVwZBpaCwsLIzBEBERkcJ4OsWFCdRERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkagyGiIiISNUYDBEREZGqMRgiIiIiVWMwRERERKrGYIiIiIhUjcEQERERqRqDISIiIlI1BkNERESkaj4PhjZv3oy+ffsiNjYWkiRBp9PZPPapp56CJEmYNWuW19pHRERE/s3nwVBhYSFatWqF2bNnl3vc8uXL8dtvvyE2NtZLLSMiIiI1CPJ1A3r16oVevXqVe8zJkyfx3HPPYe3atejdu7eXWkZERERq4POeoYoYDAYMHz4cEyZMQIsWLXzdHCIiIvIzPu8ZqsjUqVMRFBSE559/3u7nFBcXo7i42Pw4Pz/fE00jIiIiPyDrnqE///wT77//PubPnw9Jkux+XmpqKsLDw80/8fHxHmwlERERKZmsg6HMzEzk5uaibt26CAoKQlBQEI4fP47x48ejfv36Np/36quvIi8vz/yTnZ3tvUYTERGRosh6mGz48OFISUmx2NazZ08MHz4co0aNsvk8jUYDjUbj6eYRERGRH/B5MHT58mUcPnzY/DgrKws7duxAREQE6tati5o1a1ocX6lSJURHR6Np06bebioRERH5IZ8HQ3/88Qe6detmfjxu3DgAwIgRIzB//nwftYqIiIjUwufBUHJyMoQQdh9/7NgxzzWGiIiIVEfWCdREREREnsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkar5PBjavHkz+vbti9jYWEiSBJ1OZ953/fp1vPzyy0hKSkKVKlUQGxuLRx99FKdOnfJdg4mIiMiv+DwYKiwsRKtWrTB79uwy+4qKirB9+3a88cYb2L59O9LS0nDgwAE88MADPmgpERER+SNJCCF83QgTSZKwfPlyaLVam8ds27YNHTp0wPHjx1G3bl27zpufn4/w8HDk5eUhLCzMTa0lIiIiT/LW/TvIY2f2kLy8PEiShOrVq9s8pri4GMXFxebH+fn5XmgZERERKZHPh8kccfXqVbz88ssYOnRouRFiamoqwsPDzT/x8fFebCUREREpiWKCoevXr2Pw4MEQQuCTTz4p99hXX30VeXl55p/s7GwvtZKIiIiURhHDZKZA6Pjx49i4cWOF44YajQYajcZLrSMiIiIlk30wZAqEDh06hPT0dNSsWdPXTSIiIiI/4vNg6PLlyzh8+LD5cVZWFnbs2IGIiAjExMRg4MCB2L59O1atWgW9Xo+cnBwAQEREBIKDg33VbCIiIvITPp9an5GRgW7dupXZPmLECEyePBkNGjSw+rz09HQkJyfb9RqcWk9ERKQ8qplan5ycjPLiMRmVQSIiIiI/pJjZZERERESewGCIiIiIVI3BEBEREakagyEiIiKSH70eyMz0yksxGCIiIiJ5SUsD6tcH+vTxysv5fDYZERERkVlaGjBwIODF2eTsGSIiIiJ50OuBF17waiAEMBgiIiIiucjMBE6c8PrLMhgiIiIieTh92icvy2CIiIiI5CEmxicvy2CIiIiI5KFLFyAuDpAkr74sgyEiIiJf0+uBjAxg8WLjb73e1y3yjcBA4P33jf/txYCIwRAREZEvmWrqdOsGDBtm/F2/vnG7uykh6BowAFi6FKhTx2svyWCIiIjIV0w1dUrPoDp50rjdmYDIVsDjzaDLVQMGAMeOAatWeeXlJCG8PJnfB/Lz8xEeHo68vDyEhYX5ujlERETGIKV+fdtTySXJmD+TlWUcPrJHWpqxTs+t54yLA4YOBaZPL1u/xzQUtXSpMQCRGW/dv9kzRERE5AsV1dQRAsjOtn99Llu9TCdOANOmWS9kaNo2dqw8h8y8hMEQERGRL9hbU8ee41yp3Oxo0OWHGAwRERH5gr01dew5zh2Vm31U8FAOGAwRERH5QkU1dSQJiI83HlcRdwQyPip4KAcMhoiIiHyhvJo6psezZtmXPO1KIONI0OWnGAwRERH5iq2aOnFxjs3wsrdys6tBl59iMERERORLppo66enAokXG31lZjk11r6iXSZKACRNcD7r8FOsMERER+QtrdYbi4409PwMGGGedZWYac4xiYow9Sr7oEbKzHd66fzMYIiIi8idyCXhssVUY8v33y/RQMRhyIwZDREREMmAqDGlnJWxWoCYiIiL/UV5hSB9XwmYwRERERJ7n7uVH3IjBEBEREXmeO5cfcbMgr78iEREReY5cE6jdufyIm7FniIiIyF+kpQH16wPdugHDhhl/169v3O5r7lx+xM0YDBERETlKrwcyMoDFi42/fZD0W4ZpplbpvJyTJ43bfR0QuXP5ETdjMEREROQIOfa+yHimlgV3LT/iZqwzREREZC8H6+R4TUaGMSirSHo6kJzs6dZUTGYVqJlATUREZI+Kel8kydj70q+f94d6PDVTy1PJ2IGBdgVlhhKD669lBw6TERER2UPGdXI8MlPLR8OBxfnFWP1/2/Cv5ptxW63zHn0tE/YMERER2UPGdXLMM7VOnrTecyVJxv32ztSyNRxoSsZ283Bg3j95+HHaHuhWAD9mJ+Ey2t/Yk++21ygPgyEiIiJ7+LJOjq3hqlu3P/EEMHmyMfC5NYhxdKaWl4YDT/5xGitnHIJufWWkn2+J67jTvC824DT6NT+Eex8UGPB/Tr+E3RgMERER2cPdvS/2srXK+9Chxqn9t26vWdP4+/x5y2NnzbK/J8eR4UAHk7H3rToC3YfZ0P1SC1sLEwHcDBybBx+Btm02tE/URrvhzREQFIP8/HzAC8GQz3OGNm/ejL59+yI2NhaSJEGn01nsF0Jg4sSJiImJQWhoKFJSUnDo0CHfNJaIiNTLF3VybNUOOnECmDat7PYLF4yB0JQpwKJFxtljWVmODWm5cTjQUGLAb5/txit3ZKCZ5igS+jbCa+uSbwRCQKequzC1Vwb2/3gUe4sb4e1fk9FhVAsEBHk3PPF5z1BhYSFatWqFxx57DAOsfFjvvvsuPvjgAyxYsAANGjTAG2+8gZ49e2Lv3r0ICQnxQYuJiMjv2DtrylQnx1pPjSO9L/a2ydZwlS2mYazPPjMGQc4EZi4OBxbnFyP9g13QLSrCigNNkWNINO8LRjG6194JbY8r6PtSU8S0TnK8fR4gqzpDkiRh+fLl0Gq1AIy9QrGxsRg/fjxeeuklAEBeXh6ioqIwf/58PPTQQ3adl3WGiIjIJlvDUO+/bzu48cb6X/bWDrLF2ZpC164BlSuXX6AxMBAoKgKCgwEYE6BXT98DnQ74MTsRBbh5r62GfPSuuxtaLdBrQiLC4uy/D7POEICsrCzk5OQgJSXFvC08PBwdO3bEli1bbAZDxcXFKC4uNj/Oz/dONjoRESmMs7Om7KyT4xJXZ6U5+/xff624UrVej/NfrsKSjbWgW1cZG0slQMcE5KBfs4PQDquM5OeSoAm7s5yT+Z6sg6GcnBwAQFRUlMX2qKgo8z5rUlNTMWXKFI+2jYiIFE7ORRQB12elVfR8W71bdgZRzz5RjG/Q1fy4WfARaNtkQzu6FtqPSEBAULQrrfcqWQdDznr11Vcxbtw48+P8/HzEx8f7sEVERCQ7Hpw15RYVzV6zxZ5ZbeUNDdoZhJ1GDDpW2Y3+Xc6h33N10ez+RgAa2d9OGZF1MBQdbYwqz5w5g5hbPpwzZ86gdevWNp+n0Wig0Wg83TwiIlIyORdRBG7OXhs4sGztIFvsmdVWwdDg9QWLoA+LhCY/F5KVpxsAFFaOxKKMJohtX8fKEcrj86n15WnQoAGio6OxYcMG87b8/Hz8/vvv6NSpkw9bRkREiufLIor2srXKe3w8MGGCsTfnVhWt/l7B0KBBCJx59CU8kj8bAhIMpcIhIUkIkCRU++oTvwmEABn0DF2+fBmHDx82P87KysKOHTsQERGBunXrYuzYsXjzzTfRpEkT89T62NhY84wzIiIip/iqiKKjBgww5i1Zy+9JTa14VtutuUFnzpQ7NBgAIA4nYZAC8Gns/2FEwYcIzc8175ecLSHgjdl3rhA+lp6eLgCU+RkxYoQQQgiDwSDeeOMNERUVJTQajejevbs4cOCAQ6+Rl5cnAIi8vDwPXAERESnWsmVCSJLxxxgSGX9M25Yt83ULXbNsmRBxcZbXZseP/quFxueXlAiRni7EokXG3yUl7mlDXJxd76237t+yqjPkKawzRERENllLJo6Pd38RRW9LS4O4kRtkLfenXM7WKLLSBqv5SabcpgoWfPXW/ZvBEBERkdyHcRxw7fI1bPpgB9pN6oPwkrNWk4MFYD1AMg0NOlu9+lZ6PVC/vu1hOTtei0UXiYjI9/woSCiXN4ooelDBqQKsmb4bujQDfjieiNYoQgbO2jzeZiAEuG99NbmXLrgFgyEiIrLOmWUqyGvO7D6Lle/ug25NKH462xLXcHOWdTMccPyE7l5fTe6lC27BYIiIiMpydpkK8qjDG45DNzMLus0R+LUgEeKWCtCNKx1D/9bHoH28Jjo2bAz0sOOEM2cCUVGe6fVTQumCG5gzREREltyQ60HuIQwCfy7cB92nudD9UQd7iptY7O9YeSeebb4RyckCde5vDenursbPxPQZVlQ2wJOfoRvawJwhIiLyDQXlevij60XXsemjXdAtLMCKvU1wQp8AIAEAEITrSI7YCW33yxjS7jBqfTgZ+PME8CeAGbAcxrRVvdrduUG2lFdB21ttsJOsK1ATEZEPKCjXw19czrmMZRO24JEGvyCyaiHufbkNZu+6Gyf0saiCyxhYZwsWPv0Lco8WYv35thjz0HnUeuWJskGraRgzLc129eqKqlS7kxzaYAcOkxERkaWMDKBbt4qPc1ctGpXK3XMW30/bD91qDdbntkQxQgAAAdCjL75H75jtSOwRi9tnDEdIzSo3n+joMGZFMwK9MWPQyddgnSE3YjBEROQAOeSb+KkjG00J0DXwS34SxC0DNI2CjuP1ugsw9MLH0Fw6c/NJpWfwuTNYlfmMQeYMERGRbygo10PuhEFg+6L90M09A922Othd3ARAPfP+tpX3QtspF9oxddBCvxPS4MkVz+Bz1zAmZwyasWeIiIisc3aZCrUUarThetF1ZH6yG7ov86Hb0xjZ+pv5MoEoQXKNndB2L8AD4xqjbqcb+xwZ+srMdL1nSCEzBjlM5kYMhoiInORoYCPzYRdPKcwtxNrpu6BbVoJVWS1wUdQw76uMQtwXuwvaviXoPaEFIhrVKHsCR4a+unRxfRhTIXlhHCYjIiLfc2SZCpUNu5zddw7fv7sPutXBWH+mJa7iDvO+WtI5PNB4H7RDNEh5MQmhEXeUcyY4NvTljmFMzhi0wGCIiIhcp9cbe4Ss9VQIYbxJjx0L9Oun6CGzoxn/YMWso9BlVMfPeUkwoIt5X4Ogf9C/1VFoR9bAnf9KRGBwl3LOVIqj1ZpNU9at9cLZs6SGgqpDewOHyYiIyHUKGXZxlDAI7Pj2AHRzcqDbGoOdV5ta7G8Tug/aTmegfSYWif2bQAqwugRqxZydwedsfpZCZgxymIyIiNzDGwnNfjTsUnK1BJkf7zImQO9uhH/0zQA0A2BMgO5afRe09+Sj37hGqHdXcwDNXX9RZ4e+HBnGdMfr+SkGQ0RE/sxbCc0KH3YpzC3Euhm7oFtaglVZCbggbjfvC0UR7ovZCW2fEvSekICaTW4v50wucHXoyx6lA+NvvwXGjfPc6ykEh8mIiPyVrYRm0//5uzOhWSHDLrc6d+A8Vk3bB90PlbAuJwlXUNm8r6Z0Hg803gvtoGCkvJiEyrUql3MmN/NUT56twHjmTKBWLVmWQuDUejdiMEREquOLOjKm4AuwPuwig9lkWZuzsWLmUejSw5GZlwQDbl57/aBsaJOOov+o6rjziRYICvGjwRNvBsZuxGDIjRgMEZHq+Cqh2dlCjR4iDAJ/LzkI3Senofs9Bn+XSoBuHbof2o450D4dg5YDb3M+AdpR3ixMqZACi9YwgZqIiJznq4TmAQOM0+d9WIG65GoJfpm7G7oFedDtaohjJU0BGIOgAOjRtfpOaLsZE6Drd76ZHO017s7jqiiwysy0HQgBxt6i7GzjcQqa6edODIaISJ5UvqSDy3yZ0OzsDCcXFJ0rwvr3dkG35Bq+P5KA86K1eV8oitAzZie0vUvQ+6XmqNXUQwnQ9nB3YUp7Ais/munnKRwmIyL5UemSDm6lwIRmR50/dAGr3t0L3Q9BWHu6pUUCdIR0AX0b7oV2UCX0GO/lBGhb3D1cZW8ekIJrQDFnyI0YDBEpiEITPWVp6VJg0KCy2519L2XQW3f8lxNY8d4RLN8YhsxLSdDfMsBRL/AEtElHoB0Rjs5PJcovAdqdQYkjgRWg2MCYOUNEpD4qWdLBK9LSgBdftL7PmToyPuqtEwaBXWmHoPv4FHS/ReGvK80BxJn3two5AG3H09A+HYNWg26DFBBn+2S+5s7hKkfzgFhgsVwMhohIPpjo6R62etdMZsxwPBDy4gKs+mt6YwL0/IvQ7WyIrJLbANwGwJgA3Tl8F7TJl6Ad1wgNut5MjpY9d+ZxORpYeaOgo4IxGCIi+WCip+vK610DjD0B48cbb372rmHlhd66KxeuYP2MncYE6MPNcU60Mu8LwRX0iN4Jba9r6DOhOWo3b+306/hUly7G4KOi4aoudizw6kxgJYOZfnLFYIiI5EPhSzrIgrt71zzYW3fhyEX8MG0PdN8HYc2pJBSho3lfDeki+jbYA+3AIPQYn4QqkR3LOZNCuHM9MGcDKx/M9FMCBkNEJB/u/D9ntXJ375qbj/tny0msmHEYuo3VsOliS+jR2byvbuAJaBOPQPtoGDo/lYhKlTuXcyaFMSWfFxcDkycDn35q/J6bODpcxYVW3YrBEBHJB/+Bd527e9dcPE4YBHYvv5EAvSUK2680B1AHgDH/Z1Slr9Cr4UG0erAxmkx+GFIlGSdAO8tW8vmUKUCTJs4PVzEPyG04tZ6I5EdmSzooirvrCzlxPv01PbZ8tge6Ly5A93cDHCmpd/NwGNA5bBdeum0F7j8+B0Fnb+lR8sdaUt4oFSGDkgeewjpDbsRgiEiBnP0H3o9vDHZz94KpdpzvSnIvbJi1C7pvi7HyUDOcFbXNh2lwFT2i/jYmQL/UDJEHMtVRS0rBa4LJBYMhN2IwRKQSrFx9k7t716yczxBTB5lJz+DDPd2w5mQSClHVvK+6dAl96u9G/4FB6DEuEVWjb+xTU4Cg4MrPcsGii0REgP09PV6uhSN77p5GfeN8uXOX468FO7Fubxw+Ov0orp0OMR8SF3gK2oRD0A6vhq5jkqwnQKuplpQvS0Wwh9QhDIaISL7s7enxRi0ce28ucroJuWEatTAI7P3+CHQfnYDu10j8UTQQwEDz/kTNIWjbn4T2ySi0GdYMUkBs+SdUUy0pX5WKYA+p44QK5OXlCQAiLy/P100hInstWyaEJAlhDGdu/kiS8WfZspvHpqeXPc7aT3q6822Ji7M8V1ycZRscOU7mSopLxM8f/y1eapcuGlfKsnz7oRedw3aI6X3SxaGfjjl+ck9/VnJSUmL8/K19j03f5fh443Hu4sjfjQJ46/7NnCEikh9H80oWLwaGDav4vIsWAUOHOtYWe2cDKXyB2auXrhoToL+5gpUHmyO3VAJ0SuROaO8rRt8JzRCVWLucM1XA3bPd5M7dyezl8cN8LG/dvwM8dmYiImc5klcCeG44oqLhN8A4/Hbtmn3H6fWOvb6HXTqeh0VjfsHg+C2oXeM6+kxpj88OdEWuqI1w5OHh+r9gybgtOHvyOlad6YDRC7q4FggBN2tJATcDAhN/qyWl1wMREcbvRs2alvvi4twfIDv6d0Nmss8Z0uv1mDx5MhYuXIicnBzExsZi5MiReP311yGV/kMiIv/gaF6JpypX23tz+fhjeSYFW8lfOrE9FytnHIRufVWkX2iJEtxlPrxOwGloWxyE9pFq6PpMIoKr3lXOyV2ghmKB1vJ2atcGHn7YmLvmiVwyNeVjuZnsg6GpU6fik08+wYIFC9CiRQv88ccfGDVqFMLDw/H888/7unlE5AmO9vR4qnK1vTeNI0fcez5rHE3MtnIzPiNF4XnxMZbjZrCRoDkMbdsT6P9kJNo+0hxSgJfWffPnRUNtDZmeO2f8nnrqOrm2n/M8mpHkBr179xaPPfaYxbYBAwaIhx9+2O5zMIGaSGGcTTy1lsAcH+980qi9yb4zZ7onKbikxHjMokXG36brczAxW//dEmGAJAylXl8PSeghiVdDZoh3708XB9Ycde59IdtM311b3wFPJE2Xfm1vJmx7mLfu37IPht566y1Rr149ceDAASGEEDt27BCRkZFi4cKFdp+DwRCRB9m6gbvKNCum9D/sFc2KcWd77L25FBe7fhOyFfBMmGDX7KCreVfFj1O2iiebbhQnESP0Nm7GBgXeEBXF17PlnP27kSkGQzfo9Xrx8ssvC0mSRFBQkJAkSbz99tvlPufq1asiLy/P/JOdnc1giMgTPD2V3N09Pc62wZ6biys3ofKmQ5dzQzVIkrhcNVIMidssqiJfAELcjXTf3ozVbtEi+97/RYs81wY5/N24CYOhGxYvXizi4uLE4sWLxc6dO8WXX34pIiIixPz5820+Z9KkSQJAmR8GQ0Ru5K16Jp7qeXKEvTcXZ25CFQ2r2PFzN9IFIERswCnxedwbvr8Zq5mve4ZM5PB34wasM3RDfHw8XnnlFYwZM8a87c0338TChQuxf/9+q88pLi5GcXGx+XF+fj7i4+NZZ4jUxZOVkJVWz8Qd74WnKlDbu35VOb5p8joavvoQ2g1vjoCfN3tuPSw5VdeWK7XVUfIwrk12Q1FREQICLMshBQYGwmAw2HyORqOBRqPxdNOI5MvT5fiVtL6Uu94Le5e2cHQJDDdMc37o0+5AcgvjA0+VGeASD/bx1MxG8ijZF13s27cv3nrrLfzwww84duwYli9fjvfeew/9+/f3ddOI5Mk0rbd0sGJasDQtzfXXUEo9E2+8F65yZZqzJBlXor81sPFEUUMlvI9yYqqjVKeO5XZPFFokt5D9MFlBQQHeeOMNLF++HLm5uYiNjcXQoUMxceJEBAcH23UOLsdBiuLKUIS3hq/sHdpxZijGXRQwlJd/Ih+r3/kbKXMeRA39OQSg7D/Hpi1lSsxWtJyDtZ6c+HjLoob2fNcU8D7KFocVXea1+7dHM5JkglPrSTFcnZ3lreRNJdQz8WUiaznJq6f+yhFzhm0S99XaKiqhWABC9Mcycw0gq8noEyY4NzuovCRae79rckkIJlXy1v1b9jlDRKphq2qtaSjCnu51bw1fKSEvwldDeVZ6ZK7XisaquKfx7uH++O1yEoAo876mwUfRtE0EDt/+Hpp8P8P28hSpqY73MtjKX3Lku6aUIVEiFzAYIpKDihYElSTjQp/9+pV/A/RmOX45rS9lbTjCF0sT3AgyhBAWw1qB586g37nJ+AqJAJLQscpuaDufg/b5umh2f0MADQEkAx8+ZzvgcTQx2xZHv2tc4oFUwKGcoezsbMTHx3uyPR7BnCGSPXfl4PhiWq+v8yJszXJ67z1g3DivvRfX8q7AULc+NPm5ZfN7ABgAFFaOREHGdsS2r2PlCC9x9LvGqeLkQ966fzs0m6xZs2aYOHEiioqKPNUeInVy11CEJ2YSVcTUYzF0qDEQyswEFi823nT1eve9jjXlzXIaMsTYJsBj70X+iXx89+KvGFb/F/SvvgEhNgIhwPiPbbWiXMQWHnL69dzC0e+aL75TcqDXG7/D3vouk085FAytX78ea9euRZMmTTB//nwPNYlIhdw5FOGrab1pacYehG7dgGHDjL/r1/fctOuKhnsA4JtvgO++c+t7kbMzF58+shn3R25D7XgNhsy6E4uP34UwFNh3Al/n1jjzXVPbVHFvf5fJ55yaWv/ll1/iP//5DyIjIzFr1ix0cbR4l5dxmIxkzxNDEd4cvrKVkFvR9G9XODLcY+qxcvK9OLT+GJbPPAZdZk38drkFxC3/H9mkUhb6334cj3Y+ihbvPW5fe3xZiNKV75qvh0S9wRffZbJJ9lPrCwsLxRtvvCGqVKkiHnzwQXH06FE3TXBzP06tJ0VQ6mrTFa2t5alp9h5cEFN/XS+2zt8jXrszXSRoDpU5ZYcqu8TbPdLF3u8PC4PeYPk+eKLcgLvXmVLqd83TfPVdJptkv1BrYWGhyMzMFC+88IIICAgQISEhYsKECaKgoMCd7XMLBkOkGEpcbdpXdWjc/LrFBcViXeof4pnEDFEn4JTFKYJwTfSouU18/NAmcWLbKdsn8USQ4WrtKUfOK/fvmqexppLsyLLO0Jw5c7Bt2zZs27YN+/btQ0BAABITE/HUU0+hVatW+Oabb5CQkIC0tDS0a9fOEx1ZRP5twADjlGYlDUX4qg6NG9bgKjhVgDXTd0OXZsAPxxORh7bmfVVRgF5xu6F9wID7/52I6vXs+DfN3eUG3FF7qry2Ku275mmsqaRaDuUMxcfHo2PHjrjjjjtwxx13oG3btggNDbU45u2338aiRYuwe/dutzfWWcwZIvIgXy7NYQoWAOuFH60EC2d2n8X30/Zj+eoQ/HS2Ja7h5qLOkdJZ9Gu6D9qHQnHPC0kIqR7iXLvckVvDZTC8TwnLzKiMt+7fbl+b7MyZM4iNjYVeRtMQGQwReZCv69DYsQbX4Q3HoZuZBd3mCPxakAiBAARAjy7IROuAnWja2IBWY5PR8fEkBAbLJLDgjdn7fP1dpjK8df92ewXqyMhIbNy40d2nJfIONcyWcTdfL83Rrx8QHm4MHgAgORmi6934c/FB6DpnQPdHHewpbgKgnvkpE4Lfx+vSWwgrPmushngQwNtxQNT78pkpxCEb7/P1d5l8x6MZSTLBBGqyi6cSVdXCFwm5Vl7zQqXaYnTA/8okQKdE/CE+GpQhzqb+z/qML7nNpGIyr+8wuVw2vHX/dvswmRxxmIwqxNoi7uHl2kbixmd2a11kw41Hj+ArXK/T0JgAPaEFajSorqw8HA7Z+BZ7iWVBsTlDcsRgiMqlpBukK/zkH/fcPWexauoe9P16CGoacq2W0RcAUCcO0vFjlteotDwcJxLEifyJLNcmI/JLmZm2AyHAeBPKzjYep1QKX17gyMbjmNE3A13C/0Z0Yk18+RVQ20YgBAASAOnkibKfmdLycNS2DAaRj7g9gZpIcZR2g3SUJ2vVeIgwCPy1eD90c89g+dY62F0qAfqu4G3ANTtOVPozc+cacN7CekBEHsdgiEiJN0h7VbSYqSQBY8cab7Y+vrleL7qOzE92Q/dlPnR7GiNb3xxAcwBAIEpwd42d6N+9AA+Ma4y6xe0BO0a7ynxmbijU6BOBgfIYtiPyUwyGiJR6g7SHI0OAPrjZFuYWYu30XdAtK8GqrBa4KG4376uMQtwXuwvaviXoPaEFIhq1uflEfbRznxmnTlvykzwyIlcxGCLy5xukDIcAz+47h1XT9kG3OhjrclriKu4w76slncMDjfdBO0SDlBeTEBpxh/WTuPKZuXvJDKWyVqwyLs74vqrlPSC6gbPJiEzsqGSsODKZPXU04x+smHUUuozq+DkvCQbcDFIaBP2D/q2OQjuyBu78V6JjFaCtfWa1awMff3xzFpYtau4VYSkJUghOrXcjBkNkN3+7QfqoVo0wCOz49gB0c3Kg2xqDnVebWuy/PXQftHecgfaZWCQNaAIpQLJxJjssWQI88wxw7tzNbezhsM3XpST87W+MPIrBkBsxGCK/48gNxUu1akquliDz413GBOjdjfCPPs68LxAl6Fp9F7T35KPfuEaod1dcOWdyAHs4HOfrhXU5NEcOUOzaZETkYY7eUDyYI1N0rgjrZuyCbsl1fH80ARduSYAORRHui9kJbZ8S9J6QgJpNbi/nTE5Q0Ew5WfFVHpkCSzyQerBniEiObPX8uNIT4qbhiXMHzhsToH+ohHU5SbiCyuZ9NaXz6NtoL/oPDkbKi0moXKtyOWdykUzyoRTHF++br4fmSLHYM0SkVrZ6fmbOBF580fmeEBdq1Rz7+QR0M45Alx6OzLwkGNDZvK9+UDa0SUehHRGOu55MRFCIl0oQyHCmnCL4opSEzEs8EDEYImUor1fDnxIyyxtKGDSo/Oe68YYiDAI7lx6E7pPT0P0ejR1XmgG4mefTOnQ/tB1zoH06Bi0H3gYpIN6l13OKPxfL9CRflJJg4Eoyx2CI5K+8HBnAfxIyK8qBsZeTN5SSqyX4Ze5u6BbkQberIY6VNAVgnAUWAD26Vt8JbTdjAnT9zs0ANHPqddzGn4tlepq3ay0xcCWZY84QyVt5OTK2vrpKnUlkby5HRRzI9Sg6V4T17+2Cbsk1fH8kAedFTfO+EFxBz+id0Pa+jj4TmqNW05rlnMlH5LSquxJ7KL3VZh+VeCDl49R6N2IwpFAVJV2WR4n/uC5ebFxR3ll2XvP5Qxfww7S9WL4qCGtPt7RIgI6QLqBvw73QDqqEe19MRJXIKs63x1NK38DPngXGjfNtsUxOGa+YnAJXUgwmUBNVlHRZHiUmZLoyRFBBrsfxX05gxXtHoNsYhs2XkqC/JQG6XuAJaJOOQDsiHJ2fSkRQSOcyz5eN8pLLa9XyTa8Mp4zbh8ugkIyxZ4jky9WeEgB4/XVg8mRl9A5VNJRQnlI9IcIgsCvtEHQfn4Lutyj8daW5xeEtQw6gf8fT0D4dg1aDbnOtArS3yLHAIqeMO06Jw4nkMxwmcyMGQwrlrhwaJQ1X2Lrh2xIRYfw/7f/8B3o9jAnQ8y9Ct7Mhskrqmg8LgB6dw3dBm3wJ/cY2RMPkuuWcVIbkGnSw1hGRR3GYjKii2UL2UtJwhWko4cknLdfaskFcuABMmoSP5wRgcs5TOCdamfeF4Ap6RO+Ettc19JnQHLWbt/Zgwz1MrnVqOGWcyC8E+LoBRDaZ6qEAN4dCTEo/Lo8pkBo71tjDIHcDBhiHvOwgARCQ0Pf0p7ggaqCGdBGPNvwZaf/+DefOGLDidEeM+qILajev5dEme5xcgw5OGSfyCwyGSN5MPSV16lhuj4sDli0DJkywb1jk1p4DJSh9veUIgEBdZGPbU1/gzOWqWHCkM/pPvUOeM8GcJdegw9R7aSs4lyRjPhdrHRHJGoMhkr8BA4Bjx4x5F4sWGX9nZRn3TZ/uWG+PQoYrxF2dca1mNBwZHGzTtSoqVa7ksTb5lFyDDnt6L029fBkZxkkBGRnK6KEkUhEGQ6QMpnW1hg69mRNiq1pzeWQ8XKG/psfPH+/ES+0y0KTySTx0fjYEJBjsPYGMrw2AMQBwNiCwN+jwxayk8novly41/nf9+sZE62HDjL/r1zcmyxORLHA2GSmTozPNZDrF+eqlq/jpvZ3QfVuMlYea4ayobd6nwVVMDHsfz1+fjqpXykmmlum1WUyhPnQI+PRTYzK7iTOz/KzVGfJ2gUVbrE0ZX7FCfuUAiBSEU+tvcfLkSbz88stYvXo1ioqK0LhxY8ybNw/t2rWz6/l+GwypuV6HIzWIZHbjuZh1CT+8uwe67wOw5mQSClHVvK+6dAl96u+G9sFA9ByfhKrRVY2f81tvAZMmlT2ZzK7NzFrQUpqzbVfK916u5QCIFIRT62+4ePEi7rrrLnTr1g2rV69G7dq1cejQIdSoUcPXTfMtR8r/K+Xm4QhHhoRkUOH2xLbTWDHtIHQbqiLjQkuU4K6bzQs8BW3CIWiHV0PXMUmoVLlUBejAQGDiRCAxURnVe+2tlSSEMSAYOxbo18/+76RpyFTu5FoOgIjKkH0wNHXqVMTHx2PevHnmbQ0aNPBhi2TAkfL//rpm0tmzxptieXknNWsC335rvNF4OfgTBoG93x+B7qMT0P0aiT+KEgDcDOASNYegbX8S2iej0GZYM0gBsRWfdMAAY9Ag58BWr3csl8ufAwK5lgMgojJkHwytXLkSPXv2xKBBg7Bp0ybUqVMHzzzzDJ544gmbzykuLkZxcbH5cX5+vjea6h3l3WxK/5+2rXwFJRUhtCYtDRgypPwbriQZc1S6d/das/TX9Pjt8z3QfXEBur/r4/D1xgAaG5sDA+6sthvarhfQ74X6aHJvEwBNHH8RufeKOLuenLWAQOk9mnItB0BEZQmZ02g0QqPRiFdffVVs375dzJ07V4SEhIj58+fbfM6kSZMEgDI/eXl5Xmy5h6SnC2EMA8r/+eknIeLibO+XJCHi44UoKfH1FTmmpKT86wKECAwUYskSrzTnysUr4ofJW8XopptEpJRr0YxgXBW9I38X/3t0s8jZleuV9vjcokX2fT9L/6SnW55n2bKyn3NcnHG7Upi+q5LkX3+DRF6Ul5fnlfu37IOhSpUqiU6dOllse+6558Qdd9xh8zlXr14VeXl55p/s7Gz/CYbsvdm8/rpzNyG5szcY9OB1XTx2SXz9zM9iUNyvoiryLV42HJfEw/V/FkvG/SryT+Z7rA2yZe/nU15AsGyZ9QBCkow/SgqITNdS+nqUeC1EPuCtYEj2w2QxMTFISEiw2Na8eXMsW7bM5nM0Gg00Go2nm+Yb7u5SV1q+go/yME7+cRorph+Ebn1VpJdKgK4TcBr9Eg6i//Bq6PpMIoKr3lXOmW5Q+hCQLY6sJ2etPpAjw8BKeL9MNYiUkPhOpGKyD4buuusuHDhwwGLbwYMHUa9ePR+1yMcqutmYpusmJwNvvlnx+ZSWr+ClPAxhENi3ypQAXRvbClvg1gToBM1h9G9zHI90zcZtiZUQEFcH6NLKvhu0vya1AzeLIw4caPwulhcQWQsI/HEGlhIS34nUzqP9Tm6wdetWERQUJN566y1x6NAh8fXXX4vKlSuLhQsX2n0Ob3WzeY09Xe/+mq/gwevSX9eLX+fuFP/ukC5uq3TU8rTQizur/S3evT9dHFhz1PmcFn8aAiqPrfdnyhTjUG96uvXPyN5h4EWLvH5JROR9zBm6xffffy8SExOFRqMRzZo1E59++qlDz/e7YEgI6zeb+HghvvvOeKNZtMh44zHdaP3pxuvGPIyreVfFj1O2in812ySiAs5YnC4YV8X9tbeKT4dvEqf/PlP29R0NaCpK/lZqgGpLScnN76Kt4Kc0GeSEEZF8eOv+rYgK1K5STQXqc+eAF1+0HGaoWdP4+/z5m9vksnyBK1xYliHvnzz8OG0PdCuAH7OTcBnVzPvCkIfe9XZD21/CfeMTERZX6vviSlVhe5cQSU9XzhCQu5ne34qGgVm1mUgVWIFaybyVHHtrzZm0NGDw4LI3kAsXjL+nTAGaNPGffAUH8zBObc/ByukHsXxdZaSfb4nruNO8LyYgB9rmB6F9uAqSn0sqPwHalZwWFuGrWHk5R75ekJWI/BaDIXfzRXKsPTNwPvvM//5vuoIChPt/PArdB/9A93Mt/F6YCCDavK958BFo22ZD+0RttBveHAFB0TbPY8GVgIZF+OzDGVhE5GUMhtzJkWUy3MkfZ+A4wVBiwNb5e6H77Bx0f9XFgWsNATQ077+j6i7073Ie/Z6ri6a9GgFo5PiLuBLQ2DsTsEsXx9vlbzgDi4i8iMGQu/iyPoonhl8UUgfn2uVr2DhrJ3SLirDiQFPkGBLN+yrhGrrX+hvaHlfwwISmiGmd5PoLuhLQcAjIMXJfeoSI/AaDIXfxZe+Mu4dfZF4HJ/9EPlZP2w2dDvjxnxbIRzvzvmrIR++6u6HVAr0mJCIsrr17X9zVgIZDQEREssNgyF18mRzrzuEXXw31VeD0jjNYOe0AdOtCseFcqzIJ0P2aHYR2WGUkP5cETdid5ZzJDVwNaDgEREQkK5xa7y6+njZtCmIA670V9gQxrkwb94ADq00J0DXx22XLIa7bKmWhf5vj0I6uhQ4jExAQFODx9pShkKFEIiKl8tbUegZD7iKH+igu1N4B4POAzlBiwB9f7cPyT89Ctz0e+69ZJjh3rLIb2s7noH2+Lprd39DGWYiIyF+wzpDSyCE51tXhFx8M9V27fA0ZH+6C7utCrNjXBKcMLcz7KuEa7qm5E9oeRXjgpdsQ2yaxnDMRERE5h8GQO8khOdaVGTheqoOTfyIfa2bshm65wA/HE5GPtuZ9VVGA++N3ob8W6PVSC4TXbWf7RERERG6grmGyCxcQ9vffns/xUGouiQeH+nJ25mLlu/uNCdBnW+IaNOZ9UQG56HfbfmiHhuKesS2hCdOUcyYiIlILDpN5QmIicOrUzccREcZenP/8x73BilLro7h5qO/Q+mPQzToGXWYEthQkQqCreV+TSlnof/txaB+viY6PtUBAUNdyzuRmSg1WiYjII9TVMwTAalxZsybw6aes8WLiZCK2MAj88dU+6D7Nhe7POOwtbmyxv0OV3dDedQ7a5+LR7P6GkAIkD11AOWReQ4mIiG7ibDI3qjAYAow9Hz6qoSNLdvaeXLt8DZs+2gXd15exYu9tOGm4mU8UhOvoFrET2nsv44HxtyGufYzD53crWzWUHCk/QEREXsNgyI3sCoYAY++Hvy1m6gEFpwqwdsZuLF9mwA/HE5GHcPO+qihAr7jd0D5gwP3/TkT1euFlT+CrxWxlVEOJiIgqxmDIjewOhgDPFUVUuDO7z+L7afuhW6PBT7ktUYwQ875I6Sz6Nd0H7UOhuOeFJIRUD7F9Il/1zvi6KCYRETmMCdS+Ym8NHRUk4R7ecBy6mVnQbY7ArwWJELi5nEejoOPo3zoL2scicMfjLRAYbEcCtL8tZktERH6BwVBp9tTQ8dMkXGEQ2L5oP1bMOYXsradx9XogTiMGW9ACAgFoV3kvtHfmQvtsHBL6NoIUUM+xF/CnxWyJiMhvMBgysXcxUzktZOqG3qnrRdexefYu6L4qgG5vE7TX78P7eAHxuBm0FITWRvHEt1HrldEAEpxvr78sZktERH7FB6tb+lCNGta321tDp6JhHsA4zKPXu9JK+6SlGROCu3UDhg0z/q5f37i9ApdzLmPZhC0Y3vBnRFYtRMq/2+CjXXejvf43LMVAxMGy96ba1XOo9dq/7Dp3uXzZO2OqoQTc/LxNvLVcChERyZK6gqEjR4ApU4zFFm8VF2dfj44jwzyeZOqdKt0WU++UlaAld89ZfD4yE32jtqJWTBAGTu+EhVmdcUlUR23pLEY3ycDX1Z6GBIEy1X/cFeiZemdKByMmkmSc0eep3hnTcil16lhut/fzJyIiv6Su2WSmbHRnh5cWLzb2wlRk0SJg6FDXG26NA1PEj2w6gRXvZ2F5Rg38kp8EcUvs2zDoOPq3MiZAdxrdAoG/ZnpntpUpkAOsV7j2RlCiguR3IiJ/wNlknuTschlySMK1s3fqsdBFmHd9OICbSc5tQveh/51noB1TBy36NbZMgPZWPo/SF7MlIiK/o85gyFlySMK1Mxi5cj0IgSjB3TV2QntPAfqNb4y6nZoDaG79Cd4M9AYMME6fZ+8MERHJAIMhRzi6kKknhmPsDEZG9T2H2TMLENGojX3n9Xagx94ZIiKSCXUlULuDvUm4Lsz2subsvnOY91gm+j8UjBOoA0PZNGcAgLiRhNxj+TOIaGRj9pw1nG1FREQqpc4Eancor9fHTUtOZG3Ohu69I9BlVMfPeUkwwHj+/kjDUgwsO/PLHUnITq5YT0RE5G5cm8yNvPVmAnBpQVBhEPh7yUEs//g0dFtjsPNqU4v9t4fug/aOM9A+E4sk7IL04ljPBC2cbUVERDLAYMiNvBoMObggaMnVEvw8Zzd0C/Kg29UIx/Vx5kMCUYKu1XdBe08++o1rhHp3xVmeg0ELERH5MU6t94TMTCA/37OBg52zvbbNzMTs0UH4/mgCLojW5u2hKELPmJ0YcH8xHkjOQ3hgobG9d1hJnGYSMhERkcvUFQz16XPzvz21sKqds70mrOyCTegMAKgpnUffRnuhHRSMe8clofLmU8a8nc9lsBAse5+IiMjPqWuYDIC5k81TFY9NOUM2pqgbIOEE4tAtcDMeaHkc2hHhuOvJRASF3IhL3ZR87RbWkql9FZQREZHqMGfIjawGQ0C5yczOEAaBnUsPYv//fYtBeyYDAAJw8+01QIIEgePjPkC9ac9CCig1hb2i5GvA2N5jxzzfOyOnoIyIiFTJW8GQuusMuWFh1ZKrJdj0/g682GYTGmpOoPWQpnhoz0QMxFKchGUtooD4OEjLlqH+jOfKBkJAxUttAMb9b73ldHvtotcbe4SsxcnuWrSViIhIJtSVM2SLg+ttXblwBetn7MTy767h+yMJOH9LAnQIrqBn9E480DsSoeP+BHL32p9vY287Jk0CEhM91zNj5/pnyMxkAjcRESkegyHArqTn84cu4Idpe6FbFYS1p5NQhI7mfRHSBfRtuBfaQZVw74uJqBJ5cx8SIt3aDrOxY43re/lwRpzLi7YSERHJgLqDoQrW2zr+ywmseO8IdBvDsPlSEvQ3Zn8BQN3AE+ifdATaEeHo/FQigkI6Wz2HQ0zrg1U0VAZ4tmfGm4u22ouz2oiIyEPUGwxZWW9LGAR2pR2C7uNT0P0Whb+uNAdws9Bhy5AD0HY4De1T0Wg9pCmkgLiy53WFaX2wBx+073hP9cx4ctFWZ4IazmojIiIPUlwC9TvvvANJkjB27FjXTnRjYVV9n37I/OhvjG+Xgcaaf9Bq0G2YlJ6Mv640RwD06Bq+A+/1y8CR9H/w95WmmLIpGbcPbWY9AdodBgwApkyx71hP9cx4atFWZxavNc1qK91bdvKkcbuTC98SERGZKGpq/bZt2zB48GCEhYWhW7dumDVrll3PM0/NW7UKYfn5KK4SgXW/h0G3pAQrDzfHOVHLfGwIrqBH9E5oe11DnwnNUbt5rXLO7CEurG/mVu5ctNWZqfpyeR+IiMgnuBxHKZcvX8bDDz+M//3vf3jzzTedOseGj/Zg0d934cfTrVGEKubtNaSL6NNgD7QPBqHnS0mWCdC+YOqZGTjQ+PjWAMKZnhln820GDDAmabuaq1PRVH1Jsp4QzlltRETkBYoJhsaMGYPevXsjJSWlwmCouLgYxcXF5sf5+fkAgO5rXkY7xEGP9/FHYEdoWxyG9tEwdHk6EZUquyEB2p0GDDD2lljLlXGkZ8bVfBt3rH/mbFDDWW1EROQFigiGvvnmG2zfvh3btm2z6/jU1FRMsZF3E4cTWIaBwDdLIA20M1HZV1ztmbE1NGXKt/FWFWlngxo5zmojIiK/I/ucoezsbLRr1w7r169Hy5YtAQDJyclo3bq1zZwhaz1D8fHxN5fjUEOuiZzybTIyjMnSFUlPt+wZqmCdN1V8jkREKsblOG74888/kZubizZt2iAoKAhBQUHYtGkTPvjgAwQFBUFvZUkIjUaDsLAwix8LbliGQ/YcGZryNNNU/dIz00wkyZiYXXqqvqdmtREREd1C9sFQ9+7dsWvXLuzYscP8065dOzz88MPYsWMHAl25ETqaa6LXG3s5Fi82/pbz2lxyyrdxJagx5U7VsVznzVQagXWGiIjIVbLPGapWrRoSExMttlWpUgU1a9Yss91hjuSaKK3wn9zybVxJCHfXrDYiIiIrZB8MeYSjFZTlkojsCE9WkXaWK0GNO2a1ERERWSH7BGp3MCdgAQgrr8ifNXJKRHaUKYgDrNcqkmMQR0REdAMTqD3F0VwTRxKR5ZZTxHwbIiKiCqlrmGzVKuC++xzrwbE3wXjFCmD4cPnlFDHfhoiIqFzqGiZzppvN3ho51jgzJMeghYiICACHyeSjoho5gO2AxRRnjh1b8ZCZMyu6ExERkcsYDFXEnho55QU69hQ3NCU6l85NMs1WY0DkHLnlcBERkSwxGLJHeYnIY8fadw5buUcVregO2NezRJbY00ZERHZiMGSvAQOAY8eM62ctWmT8nZVlTE62h63ihnJaNsNfsKeNiIgcoK7ZZK6yVvjP1eKGclo2wx9U1NMmScaetn79mJxOREQA2DPkOlcXE5XbshlKx542IiJyEIMhd3CluKGzK7qTdexpIyIiB3GYzF2cLW5o6lkaONAY+FhbNqO8niWyxJ42IiJyEIsuykVaWtkV3ePjK17RnSyZ1pKrKIdLjmvJERGRBW/dv9kzJBdcNsMxtqp1s6eNiIgcxGBITqzNVqOyrPWi3boOnCmHy9ox7GkjIqJSOExGymKqIVT6a2ttHTiu9UZEpGjeun8zGCLlMOUD2Zo6z3wgIiK/woVaiUpjDSEiIvIABkOkHKwhREREHsBgiJSDNYSIiMgDGAyRcrBaNxEReQCDIVIOV9eBIyIisoLBECmLK+vAERERWcGii6Q8rNZNRERuxGCIlInVuomIyE04TEZERESqxmCIiIiIVI3BEBEREakagyEiIiJSNSZQ+wOuzk5EROQ0BkNKl5YGvPCC5QKmcXHG4oSsuUNERFQhDpMpWVoaMHBg2ZXcT540bk9L8027iIiIFITBkFLp9cYeISHK7jNtGzvWeBwRERHZxGBIqTIzy/YI3UoIIDvbeBwRERHZxGBIqU6fdu9xREREKsVgSKliYtx7HBERkUoxGFKqLl2Ms8Ykyfp+SQLi443HERERkU0MhpQqMNA4fR4oGxCZHs+axXpDREREFWAwpGQDBgBLlwJ16lhuj4szbmedISIiogrJPhhKTU1F+/btUa1aNURGRkKr1eLAgQO+bpZ8DBgAHDsGpKcDixYZf2dlMRAiIiKyk+wrUG/atAljxoxB+/btUVJSgtdeew09evTA3r17UaVKFV83Tx4CA4HkZF+3goiISJEkIaxV7ZOvs2fPIjIyEps2bULXrl3tek5+fj7Cw8ORl5eHsLAwD7eQiIiI3MFb92/Z9wyVlpeXBwCIiIiweUxxcTGKi4vNj/Pz8z3eLiIiIlIm2ecM3cpgMGDs2LG46667kJiYaPO41NRUhIeHm3/i4+O92EoiIiJSEkUNkz399NNYvXo1fv75Z8TFxdk8zlrPUHx8PIfJiIiIFITDZKU8++yzWLVqFTZv3lxuIAQAGo0GGo3GSy0jIiIiJZN9MCSEwHPPPYfly5cjIyMDDRo08HWTiIiIyI/IPhgaM2YMFi1ahBUrVqBatWrIyckBAISHhyM0NNTHrSMiIiKlk33OkGRj7a158+Zh5MiRdp2DU+uJiIiUhzlDN8g8ViMiIiKFU9TUeiIiIiJ3YzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIiIiUjUGQ0RERKRqigmGZs+ejfr16yMkJAQdO3bE1q1bfd0kIiIi8gOKCIa+/fZbjBs3DpMmTcL27dvRqlUr9OzZE7m5ub5uGhERESmcIoKh9957D0888QRGjRqFhIQEzJkzB5UrV8YXX3zh66YRERGRwsk+GLp27Rr+/PNPpKSkmLcFBAQgJSUFW7Zs8WHLiIiIyB8E+boBFTl37hz0ej2ioqIstkdFRWH//v1Wn1NcXIzi4mLz47y8PABAfn6+5xpKREREbmW6bwshPPo6sg+GnJGamoopU6aU2R4fH++D1hAREZErzp8/j/DwcI+dX/bBUK1atRAYGIgzZ85YbD9z5gyio6OtPufVV1/FuHHjzI8vXbqEevXq4Z9//vHomyk3+fn5iI+PR3Z2NsLCwnzdHK/hdfO61YDXzetWg7y8PNStWxcREREefR3ZB0PBwcFo27YtNmzYAK1WCwAwGAzYsGEDnn32WavP0Wg00Gg0ZbaHh4er6ktkEhYWxutWEV63uvC61UWt1x0Q4NkUZ9kHQwAwbtw4jBgxAu3atUOHDh0wa9YsFBYWYtSoUb5uGhERESmcIoKhIUOG4OzZs5g4cSJycnLQunVrrFmzpkxSNREREZGjFBEMAcCzzz5rc1isIhqNBpMmTbI6dObPeN28bjXgdfO61YDX7dnrloSn56sRERERyZjsiy4SEREReRKDISIiIlI1BkNERESkagyGiIiISNUUGQzNnj0b9evXR0hICDp27IitW7eWe/ySJUvQrFkzhISEICkpCT/++KPFfiEEJk6ciJiYGISGhiIlJQWHDh3y5CU4xZHr/t///ocuXbqgRo0aqFGjBlJSUsocP3LkSEiSZPFz3333efoyHObIdc+fP7/MNYWEhFgc44+fd3JycpnrliQJvXv3Nh+jhM978+bN6Nu3L2JjYyFJEnQ6XYXPycjIQJs2baDRaNC4cWPMnz+/zDGO/pvhbY5ed1paGu69917Url0bYWFh6NSpE9auXWtxzOTJk8t83s2aNfPgVTjO0evOyMiw+j3PycmxOM7fPm9rf7uSJKFFixbmY5TweaempqJ9+/aoVq0aIiMjodVqceDAgQqf5417uOKCoW+//Rbjxo3DpEmTsH37drRq1Qo9e/ZEbm6u1eN//fVXDB06FI8//jj++usvaLVaaLVa7N6923zMu+++iw8++ABz5szB77//jipVqqBnz564evWqty6rQo5ed0ZGBoYOHYr09HRs2bIF8fHx6NGjB06ePGlx3H333YfTp0+bfxYvXuyNy7Gbo9cNGCu03npNx48ft9jvj593WlqaxTXv3r0bgYGBGDRokMVxcv+8CwsL0apVK8yePduu47OystC7d29069YNO3bswNixYzF69GiLwMCZ75C3OXrdmzdvxr333osff/wRf/75J7p164a+ffvir7/+sjiuRYsWFp/3zz//7InmO83R6zY5cOCAxXVFRkaa9/nj5/3+++9bXG92djYiIiLK/H3L/fPetGkTxowZg99++w3r16/H9evX0aNHDxQWFtp8jtfu4UJhOnToIMaMGWN+rNfrRWxsrEhNTbV6/ODBg0Xv3r0ttnXs2FE8+eSTQgghDAaDiI6OFtOmTTPvv3TpktBoNGLx4sUeuALnOHrdpZWUlIhq1aqJBQsWmLeNGDFC9OvXz91NdStHr3vevHkiPDzc5vnU8nnPnDlTVKtWTVy+fNm8TQmf960AiOXLl5d7zL///W/RokULi21DhgwRPXv2ND929b30Nnuu25qEhAQxZcoU8+NJkyaJVq1aua9hHmbPdaenpwsA4uLFizaPUcPnvXz5ciFJkjh27Jh5m9I+byGEyM3NFQDEpk2bbB7jrXu4onqGrl27hj///BMpKSnmbQEBAUhJScGWLVusPmfLli0WxwNAz549zcdnZWUhJyfH4pjw8HB07NjR5jm9zZnrLq2oqAjXr18vs9hdRkYGIiMj0bRpUzz99NM4f/68W9vuCmev+/Lly6hXrx7i4+PRr18/7Nmzx7xPLZ/3559/joceeghVqlSx2C7nz9sZFf19u+O9VAKDwYCCgoIyf9+HDh1CbGwsGjZsiIcffhj//POPj1roXq1bt0ZMTAzuvfde/PLLL+btavm8P//8c6SkpKBevXoW25X2eefl5QFAuYuweuserqhg6Ny5c9Dr9WWW4YiKiiozZmySk5NT7vGm346c09ucue7SXn75ZcTGxlp8Ye677z58+eWX2LBhA6ZOnYpNmzahV69e0Ov1bm2/s5y57qZNm+KLL77AihUrsHDhQhgMBtx55504ceIEAHV83lu3bsXu3bsxevRoi+1y/7ydYevvOz8/H1euXHHL344STJ8+HZcvX8bgwYPN2zp27Ij58+djzZo1+OSTT5CVlYUuXbqgoKDAhy11TUxMDObMmYNly5Zh2bJliI+PR3JyMrZv3w7APf9Wyt2pU6ewevXqMn/fSvu8DQYDxo4di7vuuguJiYk2j/PWPVwxy3GQ89555x188803yMjIsEgmfuihh8z/nZSUhJYtW6JRo0bIyMhA9+7dfdFUl3Xq1AmdOnUyP77zzjvRvHlzzJ07F//973992DLv+fzzz5GUlIQOHTpYbPfHz5uARYsWYcqUKVixYoVF7kyvXr3M/92yZUt07NgR9erVw3fffYfHH3/cF011WdOmTdG0aVPz4zvvvBNHjhzBzJkz8dVXX/mwZd6zYMECVK9eHVqt1mK70j7vMWPGYPfu3bLJa1JUz1CtWrUQGBiIM2fOWGw/c+YMoqOjrT4nOjq63ONNvx05p7c5c90m06dPxzvvvIN169ahZcuW5R7bsGFD1KpVC4cPH3a5ze7gynWbVKpUCbfffrv5mvz98y4sLMQ333xj1z9+cvu8nWHr7zssLAyhoaFu+Q7J2TfffIPRo0fju+++KzOUUFr16tVx2223KfrztqZDhw7ma/L3z1sIgS+++ALDhw9HcHBwucfK+fN+9tlnsWrVKqSnpyMuLq7cY711D1dUMBQcHIy2bdtiw4YN5m0GgwEbNmyw6A24VadOnSyOB4D169ebj2/QoAGio6MtjsnPz8fvv/9u85ze5sx1A8YM+//+979Ys2YN2rVrV+HrnDhxAufPn0dMTIxb2u0qZ6/7Vnq9Hrt27TJfkz9/3oBxCmpxcTEeeeSRCl9Hbp+3Myr6+3bHd0iuFi9ejFGjRmHx4sUWJRRsuXz5Mo4cOaLoz9uaHTt2mK/Jnz9vwDgb6/Dhw3b9z44cP28hBJ599lksX74cGzduRIMGDSp8jtfu4Q6lfsvAN998IzQajZg/f77Yu3ev+Ne//iWqV68ucnJyhBBCDB8+XLzyyivm43/55RcRFBQkpk+fLvbt2ycmTZokKlWqJHbt2mU+5p133hHVq1cXK1asEDt37hT9+vUTDRo0EFeuXPH69dni6HW/8847Ijg4WCxdulScPn3a/FNQUCCEEKKgoEC89NJLYsuWLSIrK0v89NNPok2bNqJJkybi6tWrPrlGaxy97ilTpoi1a9eKI0eOiD///FM89NBDIiQkROzZs8d8jD9+3iadO3cWQ4YMKbNdKZ93QUGB+Ouvv8Rff/0lAIj33ntP/PXXX+L48eNCCCFeeeUVMXz4cPPxR48eFZUrVxYTJkwQ+/btE7NnzxaBgYFizZo15mMqei/lwNHr/vrrr0VQUJCYPXu2xd/3pUuXzMeMHz9eZGRkiKysLPHLL7+IlJQUUatWLZGbm+v167PF0eueOXOm0Ol04tChQ2LXrl3ihRdeEAEBAeKnn34yH+OPn7fJI488Ijp27Gj1nEr4vJ9++mkRHh4uMjIyLL63RUVF5mN8dQ9XXDAkhBAffvihqFu3rggODhYdOnQQv/32m3nf3XffLUaMGGFx/HfffSduu+02ERwcLFq0aCF++OEHi/0Gg0G88cYbIioqSmg0GtG9e3dx4MABb1yKQxy57nr16gkAZX4mTZokhBCiqKhI9OjRQ9SuXVtUqlRJ1KtXTzzxxBOy+gfDxJHrHjt2rPnYqKgocf/994vt27dbnM8fP28hhNi/f78AINatW1fmXEr5vE1Tp0v/mK51xIgR4u677y7znNatW4vg4GDRsGFDMW/evDLnLe+9lANHr/vuu+8u93ghjCUGYmJiRHBwsKhTp44YMmSIOHz4sHcvrAKOXvfUqVNFo0aNREhIiIiIiBDJycli48aNZc7rb5+3EMbp4qGhoeLTTz+1ek4lfN7WrhmAxd+sr+7h0o0GEhEREamSonKGiIiIiNyNwRARERGpGoMhIiIiUjUGQ0RERKRqDIaIiIhI1RgMERERkaoxGCIiIiJVYzBEREREqsZgiIiIiFSNwRARERGpGoMhIlKkxYsXIzQ0FKdPnzZvGzVqFFq2bIm8vDwftoyIlIZrkxGRIgkh0Lp1a3Tt2hUffvghJk2ahC+++AK//fYb6tSp4+vmEZGCBPm6AUREzpAkCW+99RYGDhyI6OhofPjhh8jMzGQgREQOY88QESlamzZtsGfPHqxbtw533323r5tDRArEnCEiUqw1a9Zg//790Ov1iIqK8nVziEih2DNERIq0fft2JCcnY+7cuZg/fz7CwsKwZMkSXzeLiBSIOUNEpDjHjh1D79698dprr2Ho0KFo2LAhOnXqhO3bt6NNmza+bh4RKQx7hohIUS5cuIA777wTycnJmDNnjnl77969odfrsWbNGh+2joiUiMEQERERqRoTqImIiEjVGAwRERGRqjEYIiIiIlVjMERERESqxmCIiIiIVI3BEBEREakagyEiIiJSNQZDREREpGoMhoiIiEjVGAwRERGRqjEYIiIiIlVjMERERESq9v/RUjIqJ5MsSwAAAABJRU5ErkJggg==",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"theta from own sdg\n",
"[[3.97120525]\n",
" [3.17838946]]\n"
]
}
],
"source": [
"# Using Autograd to calculate gradients using SGD\n",
"# OLS example\n",
"from random import random, seed\n",
"import numpy as np\n",
"import autograd.numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from autograd import grad\n",
"\n",
"# Note change from previous example\n",
"def CostOLS(y,X,theta):\n",
" return np.sum((y-X @ theta)**2)\n",
"\n",
"n = 100\n",
"x = 2*np.random.rand(n,1)\n",
"y = 4+3*x+np.random.randn(n,1)\n",
"\n",
"X = np.c_[np.ones((n,1)), x]\n",
"XT_X = X.T @ X\n",
"theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
"print(\"Own inversion\")\n",
"print(theta_linreg)\n",
"# Hessian matrix\n",
"H = (2.0/n)* XT_X\n",
"EigValues, EigVectors = np.linalg.eig(H)\n",
"print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
"\n",
"theta = np.random.randn(2,1)\n",
"eta = 1.0/np.max(EigValues)\n",
"Niterations = 1000\n",
"\n",
"# Note that we request the derivative wrt third argument (theta, 2 here)\n",
"training_gradient = grad(CostOLS,2)\n",
"\n",
"for iter in range(Niterations):\n",
" gradients = (1.0/n)*training_gradient(y, X, theta)\n",
" theta -= eta*gradients\n",
"print(\"theta from own gd\")\n",
"print(theta)\n",
"\n",
"xnew = np.array([[0],[2]])\n",
"Xnew = np.c_[np.ones((2,1)), xnew]\n",
"ypredict = Xnew.dot(theta)\n",
"ypredict2 = Xnew.dot(theta_linreg)\n",
"\n",
"plt.plot(xnew, ypredict, \"r-\")\n",
"plt.plot(xnew, ypredict2, \"b-\")\n",
"plt.plot(x, y ,'ro')\n",
"plt.axis([0,2.0,0, 15.0])\n",
"plt.xlabel(r'$x$')\n",
"plt.ylabel(r'$y$')\n",
"plt.title(r'Random numbers ')\n",
"plt.show()\n",
"\n",
"n_epochs = 50\n",
"M = 5 #size of each minibatch\n",
"m = int(n/M) #number of minibatches\n",
"t0, t1 = 5, 50\n",
"def learning_schedule(t):\n",
" return t0/(t+t1)\n",
"\n",
"theta = np.random.randn(2,1)\n",
"\n",
"for epoch in range(n_epochs):\n",
"# Can you figure out a better way of setting up the contributions to each batch?\n",
" for i in range(m):\n",
" random_index = M*np.random.randint(m)\n",
" xi = X[random_index:random_index+M]\n",
" yi = y[random_index:random_index+M]\n",
" gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
" eta = learning_schedule(epoch*m+i)\n",
" theta = theta - eta*gradients\n",
"print(\"theta from own sdg\")\n",
"print(theta)"
]
},
{
"cell_type": "markdown",
"id": "2ca466b4",
"metadata": {},
"source": [
"## Same code but now with momentum gradient descent"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "0d44a49c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Own inversion\n",
"[[4.25680728]\n",
" [2.80142034]]\n",
"Eigenvalues of Hessian Matrix:[0.28283539 4.95706971]\n",
"theta from own gd\n",
"[[4.25522704]\n",
" [2.80262454]]\n",
"theta from own sdg with momentum\n",
"[[4.23872708]\n",
" [2.82197736]]\n"
]
}
],
"source": [
"# Using Autograd to calculate gradients using SGD\n",
"# OLS example\n",
"from random import random, seed\n",
"import numpy as np\n",
"import autograd.numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from autograd import grad\n",
"\n",
"# Note change from previous example\n",
"def CostOLS(y,X,theta):\n",
" return np.sum((y-X @ theta)**2)\n",
"\n",
"n = 100\n",
"x = 2*np.random.rand(n,1)\n",
"y = 4+3*x+np.random.randn(n,1)\n",
"\n",
"X = np.c_[np.ones((n,1)), x]\n",
"XT_X = X.T @ X\n",
"theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
"print(\"Own inversion\")\n",
"print(theta_linreg)\n",
"# Hessian matrix\n",
"H = (2.0/n)* XT_X\n",
"EigValues, EigVectors = np.linalg.eig(H)\n",
"print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
"\n",
"theta = np.random.randn(2,1)\n",
"eta = 1.0/np.max(EigValues)\n",
"Niterations = 100\n",
"\n",
"# Note that we request the derivative wrt third argument (theta, 2 here)\n",
"training_gradient = grad(CostOLS,2)\n",
"\n",
"for iter in range(Niterations):\n",
" gradients = (1.0/n)*training_gradient(y, X, theta)\n",
" theta -= eta*gradients\n",
"print(\"theta from own gd\")\n",
"print(theta)\n",
"\n",
"\n",
"n_epochs = 50\n",
"M = 5 #size of each minibatch\n",
"m = int(n/M) #number of minibatches\n",
"t0, t1 = 5, 50\n",
"def learning_schedule(t):\n",
" return t0/(t+t1)\n",
"\n",
"theta = np.random.randn(2,1)\n",
"\n",
"change = 0.0\n",
"delta_momentum = 0.3\n",
"\n",
"for epoch in range(n_epochs):\n",
" for i in range(m):\n",
" random_index = M*np.random.randint(m)\n",
" xi = X[random_index:random_index+M]\n",
" yi = y[random_index:random_index+M]\n",
" gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
" eta = learning_schedule(epoch*m+i)\n",
" # calculate update\n",
" new_change = eta*gradients+delta_momentum*change\n",
" # take a step\n",
" theta -= new_change\n",
" # save the change\n",
" change = new_change\n",
"print(\"theta from own sdg with momentum\")\n",
"print(theta)"
]
},
{
"cell_type": "markdown",
"id": "b82627f6",
"metadata": {},
"source": [
"## AdaGrad algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
"\n",
"<!-- dom:FIGURE: [figures/adagrad.png, width=600 frac=0.8] -->\n",
"<!-- begin figure -->\n",
"\n",
"<img src=\"figures/adagrad.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
"<!-- end figure -->"
]
},
{
"cell_type": "markdown",
"id": "00d3aff0",
"metadata": {},
"source": [
"## Similar (second order function now) problem but now with AdaGrad"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "6b85aacc",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Own inversion\n",
"[[2.]\n",
" [3.]\n",
" [4.]]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"theta from own AdaGrad\n",
"[[2.0000319 ]\n",
" [2.99982304]\n",
" [4.00017365]]\n"
]
}
],
"source": [
"# Using Autograd to calculate gradients using AdaGrad and Stochastic Gradient descent\n",
"# OLS example\n",
"from random import random, seed\n",
"import numpy as np\n",
"import autograd.numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from autograd import grad\n",
"\n",
"# Note change from previous example\n",
"def CostOLS(y,X,theta):\n",
" return np.sum((y-X @ theta)**2)\n",
"\n",
"n = 1000\n",
"x = np.random.rand(n,1)\n",
"y = 2.0+3*x +4*x*x\n",
"\n",
"X = np.c_[np.ones((n,1)), x, x*x]\n",
"XT_X = X.T @ X\n",
"theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
"print(\"Own inversion\")\n",
"print(theta_linreg)\n",
"\n",
"\n",
"# Note that we request the derivative wrt third argument (theta, 2 here)\n",
"training_gradient = grad(CostOLS,2)\n",
"# Define parameters for Stochastic Gradient Descent\n",
"n_epochs = 50\n",
"M = 5 #size of each minibatch\n",
"m = int(n/M) #number of minibatches\n",
"# Guess for unknown parameters theta\n",
"theta = np.random.randn(3,1)\n",
"\n",
"# Value for learning rate\n",
"eta = 0.01\n",
"# Including AdaGrad parameter to avoid possible division by zero\n",
"delta = 1e-8\n",
"for epoch in range(n_epochs):\n",
" Giter = 0.0\n",
" for i in range(m):\n",
" random_index = M*np.random.randint(m)\n",
" xi = X[random_index:random_index+M]\n",
" yi = y[random_index:random_index+M]\n",
" gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
" Giter += gradients*gradients\n",
" update = gradients*eta/(delta+np.sqrt(Giter))\n",
" theta -= update\n",
"print(\"theta from own AdaGrad\")\n",
"print(theta)"
]
},
{
"cell_type": "markdown",
"id": "d8ddde38",
"metadata": {},
"source": [
"Running this code we note an almost perfect agreement with the results from matrix inversion."
]
},
{
"cell_type": "markdown",
"id": "ff15b503",
"metadata": {},
"source": [
"## RMSProp algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
"\n",
"<!-- dom:FIGURE: [figures/rmsprop.png, width=600 frac=0.8] -->\n",
"<!-- begin figure -->\n",
"\n",
"<img src=\"figures/rmsprop.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
"<!-- end figure -->"
]
},
{
"cell_type": "markdown",
"id": "66f96d12",
"metadata": {},
"source": [
"## RMSprop for adaptive learning rate with Stochastic Gradient Descent"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "888f1b4e",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Own inversion\n",
"[[2.]\n",
" [3.]\n",
" [4.]]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"theta from own RMSprop\n",
"[[2.00123675]\n",
" [2.99488443]\n",
" [4.00536626]]\n"
]
}
],
"source": [
"# Using Autograd to calculate gradients using RMSprop and Stochastic Gradient descent\n",
"# OLS example\n",
"from random import random, seed\n",
"import numpy as np\n",
"import autograd.numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from autograd import grad\n",
"\n",
"# Note change from previous example\n",
"def CostOLS(y,X,theta):\n",
" return np.sum((y-X @ theta)**2)\n",
"\n",
"n = 1000\n",
"x = np.random.rand(n,1)\n",
"y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
"\n",
"X = np.c_[np.ones((n,1)), x, x*x]\n",
"XT_X = X.T @ X\n",
"theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
"print(\"Own inversion\")\n",
"print(theta_linreg)\n",
"\n",
"\n",
"# Note that we request the derivative wrt third argument (theta, 2 here)\n",
"training_gradient = grad(CostOLS,2)\n",
"# Define parameters for Stochastic Gradient Descent\n",
"n_epochs = 50\n",
"M = 5 #size of each minibatch\n",
"m = int(n/M) #number of minibatches\n",
"# Guess for unknown parameters theta\n",
"theta = np.random.randn(3,1)\n",
"\n",
"# Value for learning rate\n",
"eta = 0.01\n",
"# Value for parameter rho\n",
"rho = 0.99\n",
"# Including AdaGrad parameter to avoid possible division by zero\n",
"delta = 1e-8\n",
"for epoch in range(n_epochs):\n",
" Giter = 0.0\n",
" for i in range(m):\n",
" random_index = M*np.random.randint(m)\n",
" xi = X[random_index:random_index+M]\n",
" yi = y[random_index:random_index+M]\n",
" gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
"\t# Accumulated gradient\n",
"\t# Scaling with rho the new and the previous results\n",
" Giter = (rho*Giter+(1-rho)*gradients*gradients)\n",
"\t# Taking the diagonal only and inverting\n",
" update = gradients*eta/(delta+np.sqrt(Giter))\n",
"\t# Hadamard product\n",
" theta -= update\n",
"print(\"theta from own RMSprop\")\n",
"print(theta)"
]
},
{
"cell_type": "markdown",
"id": "2e0860f7",
"metadata": {},
"source": [
"## ADAM algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
"\n",
"<!-- dom:FIGURE: [figures/adam.png, width=600 frac=0.8] -->\n",
"<!-- begin figure -->\n",
"\n",
"<img src=\"figures/adam.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
"<!-- end figure -->"
]
},
{
"cell_type": "markdown",
"id": "ab4a9859",
"metadata": {},
"source": [
"## And finally [ADAM](https://arxiv.org/pdf/1412.6980.pdf)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "ccdd4d77",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Own inversion\n",
"[[2.]\n",
" [3.]\n",
" [4.]]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"theta from own ADAM\n",
"[[2.00003737]\n",
" [2.99982141]\n",
" [4.00019066]]\n"
]
}
],
"source": [
"# Using Autograd to calculate gradients using RMSprop and Stochastic Gradient descent\n",
"# OLS example\n",
"from random import random, seed\n",
"import numpy as np\n",
"import autograd.numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from autograd import grad\n",
"\n",
"# Note change from previous example\n",
"def CostOLS(y,X,theta):\n",
" return np.sum((y-X @ theta)**2)\n",
"\n",
"n = 1000\n",
"x = np.random.rand(n,1)\n",
"y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
"\n",
"X = np.c_[np.ones((n,1)), x, x*x]\n",
"XT_X = X.T @ X\n",
"theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
"print(\"Own inversion\")\n",
"print(theta_linreg)\n",
"\n",
"\n",
"# Note that we request the derivative wrt third argument (theta, 2 here)\n",
"training_gradient = grad(CostOLS,2)\n",
"# Define parameters for Stochastic Gradient Descent\n",
"n_epochs = 50\n",
"M = 5 #size of each minibatch\n",
"m = int(n/M) #number of minibatches\n",
"# Guess for unknown parameters theta\n",
"theta = np.random.randn(3,1)\n",
"\n",
"# Value for learning rate\n",
"eta = 0.01\n",
"# Value for parameters beta1 and beta2, see https://arxiv.org/abs/1412.6980\n",
"beta1 = 0.9\n",
"beta2 = 0.999\n",
"# Including AdaGrad parameter to avoid possible division by zero\n",
"delta = 1e-7\n",
"iter = 0\n",
"for epoch in range(n_epochs):\n",
" first_moment = 0.0\n",
" second_moment = 0.0\n",
" iter += 1\n",
" for i in range(m):\n",
" random_index = M*np.random.randint(m)\n",
" xi = X[random_index:random_index+M]\n",
" yi = y[random_index:random_index+M]\n",
" gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
" # Computing moments first\n",
" first_moment = beta1*first_moment + (1-beta1)*gradients\n",
" second_moment = beta2*second_moment+(1-beta2)*gradients*gradients\n",
" first_term = first_moment/(1.0-beta1**iter)\n",
" second_term = second_moment/(1.0-beta2**iter)\n",
"\t# Scaling with rho the new and the previous results\n",
" update = eta*first_term/(np.sqrt(second_term)+delta)\n",
" theta -= update\n",
"print(\"theta from own ADAM\")\n",
"print(theta)"
]
},
{
"cell_type": "markdown",
"id": "25ac988c",
"metadata": {},
"source": [
"## Introducing [JAX](https://jax.readthedocs.io/en/latest/)\n",
"\n",
"Presently, instead of using **autograd**, we recommend using [JAX](https://jax.readthedocs.io/en/latest/)\n",
"\n",
"**JAX** is Autograd and [XLA (Accelerated Linear Algebra))](https://www.tensorflow.org/xla),\n",
"brought together for high-performance numerical computing and machine learning research.\n",
"It provides composable transformations of Python+NumPy programs: differentiate, vectorize, parallelize, Just-In-Time compile to GPU/TPU, and more."
]
},
{
"cell_type": "markdown",
"id": "37d556d0",
"metadata": {},
"source": [
"### Getting started with Jax, note the way we import numpy"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "5b81d6e4",
"metadata": {},
"outputs": [],
"source": [
"import jax\n",
"import jax.numpy as jnp\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"from jax import grad as jax_grad"
]
},
{
"cell_type": "markdown",
"id": "c42db672",
"metadata": {},
"source": [
"### A warm-up example"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "98eb2f26",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/jax/_src/numpy/lax_numpy.py:173: UserWarning: Explicitly requested dtype float64 requested in asarray is not available, and will be truncated to dtype float32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more.\n",
" return asarray(x, dtype=self.dtype)\n"
]
},
{
"data": {
"text/plain": [
"[<matplotlib.lines.Line2D at 0x130bd2520>]"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABPqUlEQVR4nO3deXiU1eH28e8zk52sELIS9l1W2RQUURFFUVHcbUVr1bTRVm1rtfXVbr9SrbW0Noh1X+pSUQStoiyCKKvsyI6sCQkkZDLZl5nn/WMgggISSObMTO7Pdc0FzDxJbtKUuT3nPOdYtm3biIiIiPiJw3QAERERaVlUPkRERMSvVD5ERETEr1Q+RERExK9UPkRERMSvVD5ERETEr1Q+RERExK9UPkRERMSvwkwH+Dav10t+fj5xcXFYlmU6joiIiJwE27YpKysjIyMDh+PEYxsBVz7y8/PJysoyHUNEREROwZ49e2jXrt0Jrwm48hEXFwf4wsfHxxtOIyIiIifD7XaTlZXV8D5+IgFXPg5PtcTHx6t8iIiIBJmTWTKhBaciIiLiVyofIiIi4lcqHyIiIuJXKh8iIiLiVyofIiIi4lcqHyIiIuJXKh8iIiLiVyofIiIi4lcBt8mYiIiINBOvB3YtgvJCiE2FDsPB4fR7jEaNfEyaNIkhQ4YQFxdHSkoK48ePZ/PmzUddM2rUKCzLOuqRnZ3dpKFFRETk5E1ZPYWpc+6FyX3g5XHwzu2+Xyf3Yeqce5myeopf8zSqfCxYsICcnByWLFnC7NmzqaurY8yYMVRUVBx13R133MG+ffsaHo8//niThhYREZGT5yjaQm7eXKY6yo96fqqjgty8uTiKtvg1T6OmXWbNmnXUn1966SVSUlJYsWIFI0eObHg+JiaGtLS0pkkoIiIip87rIXvtx+AoJzcpEYBsl5upifHkJiWQU1JKdvEncIHHb1Mwp7XgtLS0FIDWrVsf9fx//vMfkpOT6dOnDw899BCVlZXH/Rw1NTW43e6jHs2hoLSaf8zZyl8+2tQsn19ERCQg7VoE7nyyXW6yS1zkJiUyoGMWuUmJ5JS4yHaVgjvPd52fnPKCU6/Xy7333suIESPo06dPw/M33XQTHTp0ICMjg7Vr1/LrX/+azZs38+677x7z80yaNInf//73pxrjpO0vq+bvc7YQHe7kZxd2JSZCa21FRKQFKC9s+G2P2joAPJZFuG2T7XIf87rmdsrvwDk5Oaxfv57PP//8qOfvvPPOht/37duX9PR0LrzwQrZv306XLl2+83keeugh7r///oY/u91usrKyTjXWcfXNTKB96xh2H6xk3qb9jOuX0eRfQ0REJODEpjb89h9JCQA4bJs6y2JqYvw3BeSI65rbKU273H333XzwwQd8+umntGvX7oTXDhs2DIBt27Yd8/XIyEji4+OPejQHy7K4rF86AB+s2dcsX0NERCTgdBgO8Rk8mZTIzogIAKbn7SPn0BTM1MQEiM/0XecnjSoftm1z9913M336dObNm0enTp2+92NWr14NQHp6+ikFbErjDpWPTzfvp7ym3nAaERERP3A4mdrvYl5M9P3Hfd/qGjrX1ZPtcpNTUkpuUgJT+43x634fjSofOTk5vPbaa7z++uvExcVRUFBAQUEBVVVVAGzfvp0//vGPrFixgp07dzJz5kxuueUWRo4cSb9+/ZrlL9AYvdPj6Zzcipp6L3M2+G9uS0RExCRvcneS6r0AXFH+zfYY2d5YcjIvxJvc3a95GrXm4+mnnwZ8G4kd6cUXX+TWW28lIiKCOXPmMHnyZCoqKsjKymLChAk8/PDDTRb4dFiWxbh+6fxz3jY+WLuP8QMzTUcSERFpdiMiMnk6zEG4bXPG4L9A6+iGHU6zDexw2qjyYdv2CV/PyspiwYIFpxWouV3WL4N/ztvGZ1sOUFpVR0J0uOlIIiIizeq/y6cCcEZVDH1G/gAsy2ieFnewXI+0OLqlxFLr8TJbUy8iIhLi6uoq+cyzG4De8RdhGS4e0ALLB9Bwm+0Ha/MNJxEREWle85b8C5fTQVK9l3Hn/tx0HKCFlo/Dt9x+vrWIkopaw2lERESaz/St0wHoUZVKn6y2htP4tMjy0TUllp5pcdR7bT7ZUGA6joiISLNwlexgqaMMgB7pNwXElAu00PIBcHn/w1Mv2nBMRERC04xFf6XesuhYA1ecc73pOA1abPk4vOHYou3FFJfXGE4jIiLS9N4v+AKAzOqedE+NM5zmGy22fHRo04q+mQl4vDYfrdfUi4iIhJZt2z9hc5iXMNumV5ds03GO0mLLB3wz+vH+Gt31IiIioeWdFVMA6FkRybihwwynOVrLLh+H1n0s23mQgtJqw2lERESaRn1dNbPKfQe6JnjPpkvbWMOJjtaiy0dmYjSDOyRh29rzQ0REQsfiVf+myGmR4PEy4Iy7TMf5jhZdPgCuGOAb/dDUi4iIhIrpm94GoLO7DZef2dVwmu9q8eVjbJ90HBas2VvKruKK7/8AERGRAOYu3cMCbwkAUWGXk5kYbTjRd7X48tE2LpIRXZMBjX6IiEjwm7X0b9RaFu1rbEYMnGA6zjG1+PIB32w49v4abTgmIiLB7b29vtPlW7s7c1n/TMNpjk3lA7j4jDQinA42F5axuaDMdBwREZFTsmPnfNY563HaNtHxPyA5NtJ0pGNS+QASosM5r4fvsB1NvYiISLB6f2UuAD0qIxh95tmG0xyfyschh6deZq7Jx7Ztw2lEREQax1Nfy0zXRgAs90AuPiPNcKLjU/k4ZHSvFKLDnew+WMmavaWm44iIiDTKsjUvUOi0iPN4SUi7hYTocNORjkvl45CYiDAu6p0KaOpFRESCz8xNbwHQpSyJcQO7GU5zYiofRzg89fLB2nw8Xk29iIhIcCgv28ec2gMAuMou4MJeKYYTnZjKxxFGdk8mPiqMQncNS3cUm44jIiJyUmYvfZJqh0VmrU3HTlcSExFmOtIJqXwcITLMyaV9fSfdzlytqRcREQkO7+2dB0Db0g6MP7Od4TTfT+XjW64c4NuQ5cN1+6ip9xhOIyIicmJ79nzBSqsWh22TX30lI7u3NR3pe6l8fMuwTq1Ji4/CXV3P/M0HTMcRERE5oZlf/guAHlXhnHXG2YQ7A/+tPfAT+pnDYTWcdDtjdZ7hNCIiIsfn9dQzs2Q9AJarH+MHBuZ26t+m8nEMVxy662XOxv2UVdcZTiMiInJsK9a9Qr4TWnm9FDKBQe2TTEc6KSofx3BGRjxd2raitt7LrPUFpuOIiIgc04wNrwPQrSyBsQO743BYhhOdHJWPY7Asi/GHFp7O1IZjIiISgCrL9/NJje8/kItd53LloSUDwUDl4zgOr/v4YlsR+8uqDacRERE52pxlk6lyWKTXevHEjaVnWrzpSCdN5eM4OrRpxcD2iXht+GDNPtNxREREjjJzz2wAUt3tuXJg4O/tcSSVjxO48tDC0xmaehERkQCSn/8lS/GNym8rGdtwo0SwUPk4gcv6ZeB0WKzZ42JHUYXpOCIiIgC8v/wfAPSodNKu3WDaJcUYTtQ4Kh8n0DYukhFdkwHt+SEiIoHB9nqZWbwagLDSMxp25g4mKh/fY/yhhafvrcrDtnXSrYiImLV6/evsdkK018tXZeMaziQLJiof3+PiM9KIDneys7iS1XtcpuOIiEgLN+OrVwDoXh7PsO7daN0qwnCixlP5+B6tIsMYc0Yq4Bv9EBERMaW6qoSPq303QZS6hnNVkGyn/m0qHyfh8P+476/dR53HaziNiIi0VPOWTabcYZFa52Vn/QVc2CvFdKRTovJxEs7pmkxybCQHK2r5bItOuhURETNm7pwFQIY7k0v7ZhEV7jSc6NSofJyEMKej4R7q6Zp6ERERAwoL17LY9m37sN11SdCcYHssKh8n6fDUy+wNhbh10q2IiPjZB8v+jtey6FblwI7uz7BOrU1HOmUqHyepT2Y8XVNiqdFJtyIi4me218uMAysAiCztxZUDM4PmBNtjUfk4SZZlNYx+TF+pqRcREfGf9RvfYYfTJtJrs650XNDe5XKYykcjHF73sWRHMfmuKsNpRESkpZix7gUAelS0okNaB7qnxhlOdHpUPhohq3UMQzu1xrZhpg6bExERP6itKeOjqj0AlLnOCvpRD1D5aLQjp1603bqIiDS3+cv+gdthkVzvZUPFBUF3gu2xqHw00qV904kIc7C5sIwN+9ym44iISIibseN/AGS50zm7axop8VGGE50+lY9GSogOZ/ShHeXe1cJTERFpRkUHNvKFtwyAna4xXH1m8E+5gMrHKZlwZjsAZqzO03brIiLSbP637Ek8lkWXaotiux8Xn5FmOlKTUPk4BSO7t6VNqwiKymtZuFXbrYuISNOzvV7e278MgJjSnoztm05MRJjhVE1D5eMUhDsdXDnAN/T1zgpNvYiISNPbuGUm2xxeIrw261zjGkbdQ4HKxyk6PO82e2MhpZXabl1ERJrWzLXPA9CjIpr4+HZBvZ36t6l8nKIzMuLpmRZHbb2X/63bZzqOiIiEkLqaCj6s2AFAVelQrj4zuLdT/zaVj1NkWVbD6Mc7K/caTiMiIqHksxX/osRh0brey9qy0SGxsdiRVD5Ow/gBmTgsWLGrhJ1FFabjiIhIiJi5fSYAHdyp9G+fQue2sYYTNS2Vj9OQEh/Fud3aAvCuRj9ERKQJHDy4jc88pQDsdo3m6hBaaHqYysdpmjDI90Px7qo8vF5tty4iIqfno6V/o96y6FQN+fUDubxf8G+n/m0qH6dpTO9U4iLD2FtSxbKdB03HERGRIDejYDEAce5ujO6dQkJMuOFETa9R5WPSpEkMGTKEuLg4UlJSGD9+PJs3bz7qmurqanJycmjTpg2xsbFMmDCBwsLCJg0dSKLCnVzWLx2Ad1Zo6kVERE7dlm0fsdHhIcy2WR9ie3scqVHlY8GCBeTk5LBkyRJmz55NXV0dY8aMoaLim8WW9913H++//z5vv/02CxYsID8/n6uvvrrJgweSw1MvH67bR2VtveE0IiISrGauegaAnhVRhEdnMbJ7W8OJmkej9mmdNWvWUX9+6aWXSElJYcWKFYwcOZLS0lKef/55Xn/9dS644AIAXnzxRXr16sWSJUs466yzmi55ABncIYmObWLYWVzJR+sKGsqIiIjIyaqvq+aDsm3gtKh1DWb8gEzCnaG5OuK0/lalpb7VuK1b+3ZdW7FiBXV1dYwePbrhmp49e9K+fXsWL158zM9RU1OD2+0+6hFsLMvimkOF4+0VewynERGRYLRo5dMUOy0SPV7WlI3hmsGh+x+yp1w+vF4v9957LyNGjKBPnz4AFBQUEBERQWJi4lHXpqamUlBQcMzPM2nSJBISEhoeWVlZpxrJqKvPbIdlwZKvD7K7uNJ0HBERCTLvbXkXgE7uZHplptAzLd5wouZzyuUjJyeH9evX8+abb55WgIceeojS0tKGx549wTlykJEYzTldkwHteCoiIo1T6trJfE8JAHmuC7k2hEc94BTLx913380HH3zAp59+Srt233yD0tLSqK2txeVyHXV9YWEhaWlpx/xckZGRxMfHH/UIVoenXqat2Ks9P0RE5KTNWvokdZZFhxrYUzeEK/qH3t4eR2pU+bBtm7vvvpvp06czb948OnXqdNTrgwYNIjw8nLlz5zY8t3nzZnbv3s3ZZ5/dNIkD2MVnpBEXGUaeq4olXxebjiMiIkFiRv5CABJLu3BR71QSYyIMJ2pejSofOTk5vPbaa7z++uvExcVRUFBAQUEBVVVVACQkJHD77bdz//338+mnn7JixQpuu+02zj777JC90+VIUeFOxh1qq9O054eIiJyEr3fMZZ2jnjDbZoPrspBeaHpYo8rH008/TWlpKaNGjSI9Pb3h8dZbbzVc8/e//51x48YxYcIERo4cSVpaGu+++26TBw9Uh+fpPly/j7LqOsNpREQk0M1YOQWAHpWRhLfqxMhuobm3x5Eatc+HbX//OoaoqChyc3PJzc095VDBbGBWIl3atmL7gQr+t3YfNwxtbzqSiIgEKE99LR+UbganRb1rIFcNbIfTYZmO1exCc/cSg3x7fvhuF9bUi4iInMiSVf9mv9Mi3uNlbdklIX+Xy2EqH83g6jMzcVjw5a4Svj5QbjqOiIgEqBmb3wagc1lr+mSl06VtrOFE/qHy0QxS46M479B+/P/9UqMfIiLyXWXuPObV+e6MLHSd3zBq3hKofDST6wb7fojeWbmXeo/XcBoREQk0Hy/9GzUOi6xamz31Z3F5/3TTkfxG5aOZXNgrlTatIjhQVsP8zQdMxxERkQAzM28BAG1KO3Jp30ziosINJ/IflY9mEhHm4KqBmQC89WVwbhkvIiLNY9euhayyanHYNhtd47h+SMuZcgGVj2Z1+Idp3qb97C+rNpxGREQCxcyV/wKgZ2U4cYk9GdIxyXAi/1L5aEbdUuMY2D4Rj9fm3ZV5puOIiEgA8Hrqeb/kKwDs0gFcO7gdlhX6e3scSeWjmV1/aOHpf5fvOalN2kREJLQtX/MC+5wWsR4v68ov5ZozW8beHkdS+Whm4/pnEBPh5OuiCr7cVWI6joiIGDZz05sAdC1PZET3jqTERxlO5H8qH80sNjKMy/r6bp/673ItPBURackqygv4pGY/AEUlIxu2ZWhpVD784LpDC0//t24f5TX1htOIiIgps5f+nWqHRUatzX7H+ZzfM8V0JCNUPvxgcIckOrdtRWWthw/W5JuOIyIihszYMxeAFHd7rhmcRbizZb4Nt8y/tZ9ZltUwtPampl5ERFqkvXuX8KVVg2XbbCm5lGtb0Hbq36by4ScTzmxHmMNi9R4XmwrcpuOIiIifvf/lUwD0qAqjfbsz6ZrSMg6ROxaVDz9pGxfJ6F6pALy5TKMfIiItie31MvPgWgAcrr7cMKS94URmqXz40Q1DfUNs767cS3Wdx3AaERHxl5XrXmWvE2K8XrbXXsmlfVvOIXLHovLhR+d2a0tmYjTu6no+Wr/PdBwREfGTGRteBaBbWQKXDehJdITTcCKzVD78yOmwGs57eUNTLyIiLUJlZRGfVBcAUOI6hxuGtuwpF1D58LtrB7fDYcGyHQfZfqDcdBwREWlmc5dOpsJhkVbnxdn6cnqlx5uOZJzKh5+lJ0Rzfg/fpjJvLtttOI2IiDS3mbs/ASC9NIubhnYwnCYwqHwYcOOhIbd3VuZRU6+FpyIioapg3yqW2pUA7Km4jMv7ZxhOFBhUPgwY1aMtqfGRHKyoZfaGQtNxRESkmby//O/YlkX3Sidn9RlFq8gw05ECgsqHAWFOxzc7nmrhqYhISLK9XmYUrQIgwt2bG4e23B1Nv03lw5DrBmdhWfD5tiJ2F1eajiMiIk1szYY32eWEKK9NdeT19M1MMB0pYKh8GJLVOoZzu7UF4I3lWngqIhJqZq5/BYAe5bFcPaw/lmUZThQ4VD4MuunQwtO3v9xDbb3XcBoREWkq1VUlfFTlm1avKh/B+IGZhhMFFpUPgy7slUJqfCRF5bV8sqHAdBwREWki85f/k3KHg5Q6L9273UxcVLjpSAFF5cOgcKeD6w8tPP3PEk29iIiEivd2fAhApjuDm87qYjhN4FH5MOz6oe1xWLD462LteCoiEgL2F65nsV3h+0P41fRtp4Wm36byYVhmYjQX9PTtePrGUo1+iIgEuw+W/R2vZdG1yuKysy4zHScgqXwEgJuG+RaeTlu5l+o67XgqIhKsbK+X9/YvByC+vLd2ND0OlY8AcF73FDITo3FV1vHR+n2m44iIyCnasHk6O8JsIr02HTvdSUyEdjQ9FpWPAOB0WA0732nhqYhI8Jq26jkAelTEcOOIwYbTBC6VjwBx3eAswhwWX+4qYXNBmek4IiLSSLU1Zcyu9v0HZCLn0zMt3nCiwKXyESBS4qO4qHcqAK8v3WU4jYiINNb85U9R6nSQXO/l/LPuMh0noKl8BJCbh3UA4N2VeVTU1BtOIyIijTFtywwAOpanMa5/B8NpApvKRwAZ3qUNnZJbUVZTz4zV+abjiIjISSoq2sQyh29vj27pPyAq3Gk4UWBT+QggDofFzYduu31l8U5s2zacSERETsY7n/8Vj2XRudripguuMx0n4Kl8BJhrB2URFe5gU0EZK3aVmI4jIiIn4aMi394enb396JjcynCawKfyEWASYsK5sr/v9MNXl2jhqYhIoFu7YQbbw23CbZvRg+81HScoqHwEoB+e7Vuo9OG6fRSV1xhOIyIix+T1wI6FTPviMQDOqIzhkkGDDIcKDiofAahPZgIDshKp89i8tXyP6TgiInKEKaunMHXOvTC5D3Uvj2OB0wXALZXFPDvvPqasnmI0XzBQ+QhQtxwa/Xh96W48Xi08FREJFI6iLeTmzWWqo5zPY6I56HTSpt7D1rA6cvPm4ijaYjpiwFP5CFCX9k0nKSacPFcV8zbtNx1HREQAvB6y135MTomL3KREJiclAJBVX8fTSQnklJSSvfYT35SMHJfKR4CKCndy3RDfeS9aeCoiEiB2LQJ3PtkuN7e7Svk6IgKA1VFR5JS4yHaVgjvPd50cl8pHAPvBsA5YFny25QA7iypMxxERkfLCht/Geb0Nvw+3bbJd7mNeJ9+l8hHAslrHcH6PFECjHyIiASHWdwZXjQX/TvBNuThtmzrLYmpi/Heuk2NT+Qhwh2+7/e+Xe3Tei4iIaR2GQ3wGv2ybTKXTQazHw/KdexrWgExNTID4TN91clwqHwHuvG5tfee9VNczfVWe6TgiIi2bw8nTfS9ifqsYAH7qchMOZLvc5JSUkpuUwNR+Y8Chs11OROUjwDkcFj88yzf6ofNeRETMW+/eB0C8x8OEsvKG57O9seRkXog3ubupaEEjzHQA+X7XDG7HE59sZkthOYu3FzO8a7LpSCIiLZLt9bKvZAuEw6CaTsTc8oxvcWlsKnQYTrZGPE6KRj6CQHxUOBPObAfAS4t2mg0jItKCffblM2wN9xLptRk37HfQ6Vzoe43vVxWPk6byESQmDvdNvczZWMjekkrDaUREWqYX1j0PwIDKZEYPGGw4TfBS+QgSXVPiOKdrMl5bt92KiJjw1abprAyrwWnbnNX1PhwOy3SkoKXyEUQmDu8IwFvL91Bdp617RUT86ZklfwOgb3krbhh1meE0wa3R5eOzzz7j8ssvJyMjA8uyeO+99456/dZbb8WyrKMel1xySVPlbdEu6JlCu6RoXJV1zFit225FRPxlz54vWIALgO6tf0RspO7XOB2NLh8VFRX079+f3Nzc415zySWXsG/fvobHG2+8cVohxcfpsBpOu31p0S7ddisi4ifPfvYHvJZF74owbr7oZtNxgl6jq9vYsWMZO3bsCa+JjIwkLS3tlEPJ8V0/uD1Pzt7Cxn1ulu44yFmd25iOJCIS0oqKNvG/ujywLNIcV9C5bazpSEGvWdZ8zJ8/n5SUFHr06MFPfvITiouLj3ttTU0Nbrf7qIccX0LMN7fdvvjFDsNpRERC36vzH6bWsuhSbTH+vJ+YjhMSmrx8XHLJJbzyyivMnTuXxx57jAULFjB27Fg8nmMvkJw0aRIJCQkNj6ysrKaOFHJuG9ERgE82FLK7WLfdiog0l/Kyffy3bCMAyZXncl4PHRjXFJq8fNxwww1cccUV9O3bl/Hjx/PBBx+wfPly5s+ff8zrH3roIUpLSxsee/bsaepIIadrShzndW+LbcPLi3eajiMiErLeXvD/KHc4yKy1OW/Y/bq9tok0+622nTt3Jjk5mW3bth3z9cjISOLj4496yPf70TmdAN9tt2XVdYbTiIiEntqaMl7dvwSA1NIBTBjcwXCi0NHs5WPv3r0UFxeTnp7e3F+qRRnZLZmuKbGU19QzbcVe03FERELOBwt/zwGnRXK9l549f0FMhG6vbSqNLh/l5eWsXr2a1atXA7Bjxw5Wr17N7t27KS8v51e/+hVLlixh586dzJ07lyuvvJKuXbty8cUXN3X2Fs2yLG49tOnYS4t24vHqtlsRkabi9dTzwu6PAWhX0oWJ5/Q0nCi0NLp8fPnllwwcOJCBAwcCcP/99zNw4EAeeeQRnE4na9eu5YorrqB79+7cfvvtDBo0iIULFxIZGdnk4Vu6q8/MJCE6nF3FlczbtN90HBGRkPHpkifY5YRYj5ek1J+RkRhtOlJIafQY0qhRo064udXHH398WoHk5MVEhHHj0PZMXbCdFz7fwUW9tQpbROR02V4vz215CxzQrTSDiVcPMh0p5OhslyB3y9kdcDosFn9dzMZ92iNFROR0fbnmRdY76on02tSF/5gz2yeajhRyVD6CXEZiNGP7+HaTff5zbTomInK6nl/7bwB6uRO54ZxzsCzdXtvUVD5CwO2HbruduTqf/WXVhtOIiASvzVs+4Asqcdg2rqobubSv7tRsDiofIWBg+yQGdUii1uPllUW7TMcREQlaLyx/AoA+5TFcdvYlhDv1Ntkc9F0NEXec6xv9eG3pLipr6w2nEREJPnv3LmFWXREALtc4bhza3nCi0KXyESIu6p1GhzYxuCrreEebjomINNrLX/wBr2XRqzKMof0nkBAdbjpSyFL5CBFOh8WPRvhGP57/fIc2HRMRaYTioi1Mr9wNQHXR+Q3/nkrzUPkIIdcObkdCdDg7iyuZs7HQdBwRkaDx+sL/R43DonM1tOt4E1mtY0xHCmkqHyEkJiKMm4f55iifW/i14TQiIsGhsnw/b7q+AiC8eBh3nNfFcKLQp/IRYiYO70i402L5zhJW73GZjiMiEvCmffYwbodFRq2NlfhDBrZPMh0p5Kl8hJjU+Ciu6J8JwLMa/RAROaG6mgpeKVgEQJuDfbh9ZA/DiVoGlY8Q9ONDt91+tG4few5WGk4jIhK4/vf5Hyh0WrSp91Jo3aIzsvxE5SME9UqP59xuyXhtbbkuInI8Xk89L+76CICskk5MPLc3Toe2UvcHlY8QlX1owdSby3dzsKLWcBoRkcCzYNlkvnbatPJ62Vn1A64dlGU6Uouh8hGihndpQ5/MeKrrvLy6WFuui4gcyfZ6eX7T6wB0d6VxzdmDiI5wGk7Vcqh8hCjLsrhrpG/04+XFO6mq9RhOJCISOFaue5U1jjoivDbbS2/klrM7mI7Uoqh8hLCxfdLIah3NwYpapq3YYzqOiEjAeGHNVAB6lyVw0Zlnk9QqwnCilkXlI4SFOR3ccW5nAJ5duIN6j9dwIhER87Zs+4jP7HIcts3u4gncfo62Uvc3lY8Qd+2gLJJiwtl9sJJZXxWYjiMiYtyLSx8H4IzyaPr3PF9bqRug8hHioiOcTBzeEYBnFnyNbevAORFpufLzv+SjugMAHCgay50jOxtO1DKpfLQAt5zdkahwB+vySlm8vdh0HBERY175/Pd4LIselU7Ssi6jT2aC6UgtkspHC9C6VQTXD/bdv/70gu2G04iImFFycDvvVPg2XqwtPq/hjkDxP5WPFuLH53bG6bBYuLWItXtdpuOIiPjdGwseptph0bEGPPFXM6JrG9ORWiyVjxYiq3UMV/bPAGDKpxr9EJGWpbKyiNdd6wCILh7C3ed3x7K0lbopKh8tSPYo3xDjxxsK2La/zHAaERH/mb7g/1HqsEivsykOv5GLz0gzHalFU/loQbqnxjGmdyq2DU/P/9p0HBERv6irq+TlfQsBSD7Ym+xRPXHoADmjVD5amJ+e3xWAGavz2FtSaTiNiEjzm/X5/7HPaZFU7yWv/gdcOSDTdKQWT+WjhRmQlcjwLm2o99o8+5lGP0QktNleLy/seB+ADiUdufW8vkSE6a3PNP0v0ALlHBr9eHP5HorKawynERFpPguX/4NtTpsYr5cdVT/ghiHtTUcSVD5apOFd2tC/XQI19V5e+HyH6TgiIs3muY2vAdDDlcoNIwYTHeE0nEhA5aNFsiyrYe3Hq4t34a6uM5xIRKTprV73H1ZZtYTbNjvLbuCHZ3c0HUkOUflooS7qlUq3lFjKaup5ZdFO03FERJrc86tyAejtjufSISNJiA43nEgOU/looRwOq2Htx/Of76Cipt5wIhGRprN9+2zm22VYtk2B6yp+fG4n05HkCCofLdi4ful0bBNDSWUdry3ZZTqOiEiTeWHpXwA4oyKKkQMvIzk20nAiOZLKRwsW5nQ0rP14duHXVNV6DCcSETl9BftW8WFtIQDug2O567zOhhPJt6l8tHBXDcykXVI0ReW1vLFst+k4IiKn7ZXPf0e9ZdG90smgfteTGh9lOpJ8i8pHCxfudPCTQ2e+PPPZdqrrNPohIsGr1LWTaeW+wzPtknPJPq+L4URyLCofwjWD2pGeEEWhu4a3V+w1HUdE5JS9seBhqhwWHWuge/fbyUiMNh1JjkHlQ4gMc3LXSN+c6NT526mt9xpOJCLSeFWVB/lP8WoAWh08k5+e391sIDkulQ8B4Iah7UmOjSTPVcX0VRr9EJHg895nj+ByWqTWecnsmEP7NjGmI8lxqHwIAFHh34x+5H66nTqPRj9EJHjU11XzUv58ANIO9iLnwt5mA8kJqXxIg5vPak9ybAS7D1YyfWWe6TgiIift4y/+TL7TItHjJSXzZ3RuG2s6kpyAyoc0iIkI466RvpXhT326VaMfIhIUbK+X57+eAUCnkvbkjDnTcCL5PiofcpQfnNWB5NhI9hys4h3d+SIiQeCLL3PZ6vQS4/WSknYvXTTqEfBUPuQo0RFOsg/tBvjUvG2680VEAt6/178EQI/Stvx0zAizYeSkqHzId9w8rEPDnS/TNPohIgFs7VdvscpZS5htk578U631CBIqH/Id0RHOhl1Pcz/V6IeIBK5nlv8DgD5lcfzkkssMp5GTpfIhx3TzsPakxPlGP95escd0HBGR7/h6xzwW4gagfeKP6JjcynAiOVkqH3JMUeFHjH7M20ZNvc58EZHAMnXhn7Atiz4VEdx56U2m40gjqHzIcd04tD2p8ZHkl1bz1nKNfohI4CgsXMts734AurS6ng5tNOoRTFQ+5Liiwp3knN8VgH/N20ZVrUY/RCQwTJ3zW+oti+5VDu4cd7fpONJIKh9yQjcMaU9mYjT7y2p4dclO03FERCh17ebD2q8B6BN9mc5wCUIqH3JCEWEOfj66GwBPz99OWXWd4UQi0mJ5PbBjIS+8exuVDgfta2yyr3jYdCo5BSof8r2uHphJ5+RWlFTW8cLnO03HEZEWZsrqKUydcy9M7kP1K5fzHvsAuL2skhkrfsOU1VPMBpRGU/mQ7xXmdHDvRd0BeG7h17gqaw0nEpGWxFG0hdy8uUx1lDMzthUHnU4y6urZ56wnN28ujqItpiNKI6l8yEkZ1zednmlxlNXU88xnX5uOIyIthddD9tqPySlxkZuUyD+SEgDoUlfL1KQEckpKyV77iW9KRoJGo8vHZ599xuWXX05GRgaWZfHee+8d9bpt2zzyyCOkp6cTHR3N6NGj2bp1a1PlFUMcDotfjOkBwEtf7GR/WbXhRCLSIuxaBO58sl1uRpdX4HY6wbZZGBNDTomLbFcpuPN810nQaHT5qKiooH///uTm5h7z9ccff5x//vOfTJ06laVLl9KqVSsuvvhiqqv1ZhXsRvdKoX9WIlV1HqZ8ut10HBFpCcoLAai0LNZGRfqesyzCbZtsl/s710lwaHT5GDt2LH/605+46qqrvvOabdtMnjyZhx9+mCuvvJJ+/frxyiuvkJ+f/50REgk+lmXxq0OjH68v3c3ekkrDiUQk5MWmAvBcYjz7w8IACLdt6iyLqYnx37lOgkOTrvnYsWMHBQUFjB49uuG5hIQEhg0bxuLFi4/5MTU1Nbjd7qMeErhGdG3D8C5tqPV4eXK2FnmJSDPrMJw9cam8EO8rGmPLK1i5c0/DGpCpiQkQnwkdhhsOKo3RpOWjoKAAgNTUoxtoampqw2vfNmnSJBISEhoeWVlZTRlJmphlWfz6kp4ATF+Vx6YClUURaUYOJ9mJ0XgcFu3q6njsQDEA2S43OSWl5CYlMLXfGHA4DQeVxjB+t8tDDz1EaWlpw2PPHp0hEuj6ZyVyad80bBv+Omuz6TgiEsLmLc5ld5gXy7b5W0kd1hGvZXtjycm8EG9yd2P55NSENeUnS0tLA6CwsJD09PSG5wsLCxkwYMAxPyYyMpLIyMimjCF+8MsxPfj4q0LmbtrP8p0HGdKxtelIIhJi6uoqefKrqRAOwyra0vO+2bBnsW9xaWwqdBhOtkY8glKTjnx06tSJtLQ05s6d2/Cc2+1m6dKlnH322U35pcSwzm1juX6Ib4rsLx9twrZtw4lEJNS8/NF97AqHBI+Xa4Y/iSMsDDqdC32v8f2q4hG0Gl0+ysvLWb16NatXrwZ8i0xXr17N7t27sSyLe++9lz/96U/MnDmTdevWccstt5CRkcH48eObOLqY9vMLuxEV7mDFrhLmbNxvOo6IhJCDB7fxQtHnAJxZNZAxxxk9l+DU6PLx5ZdfMnDgQAYOHAjA/fffz8CBA3nkkUcAeOCBB7jnnnu48847GTJkCOXl5cyaNYuoqKimTS7GpcZH8aMRnQB4fNYmPF6NfohI03jyw59S5nTQsQZuufRvWJb1/R8kQcOyA2y83O12k5CQQGlpKfHx8d//AWJUaVUdIx//lNKqOh6f0I/rhuhuJRE5PZs2z+S6xb/Btiwurr2WJ+54xHQkOQmNef82freLBLeE6HDuPr8rAH+bvZmqWp2vICKnzvZ6+dMXv8O2LPqVRZN95X2mI0kzUPmQ03bL8A60S4qm0F3Dcwt16JyInLoPF/6BNc46Ir02XZN/RdeUONORpBmofMhpiwxz8sChjceeXrBdh86JyCmprCziye3TAOjryuLuy64wnEiai8qHNInL+6XTPyuRyloPk+foFGMRabznZuWw32mRWuflzDP+RNs47QEVqlQ+pElYlsVvL+0FwJvLdrO1sMxwIhEJJnl5y3il9CsA2pWew49HDTAbSJqVyoc0maGdWjOmdype27fxmIjIyfrr3PupcVj0qHRy6bm/ITpCG4iFMpUPaVIPju1JmMNi7qb9LNpWZDqOiASBZaueY65disO2iam9hQmD25uOJM1M5UOaVOe2sdw8zPcPx/99uBGvNh4TkROor6vmz6v+CUC/0iTuvPQWnA5tKBbqVD6kyf3swm7ERYbxVb6baSv3mo4jIgHs7bm/ZLvTJt7jJSrml5zTLdl0JPEDlQ9pcm1iI7nnQt/GY3/9eDPlNfWGE4lIIHKV7OBf+z4FoFNxLx644gLDicRfVD6kWdw6vBMd28RwoKyGKZ9uMx1HRALQUx//FLfDQfsam67dfkO3VG0o1lKofEiziAhz8JtDt94+9/kO9hysNJxIRALJ5i0fMK16DwAxBy/l5xf1MZxI/EnlQ5rNRb1TGdG1DbX1XiZ9tNF0HBEJELbXy6RFj+K1LPqWRXHZuT8hqVWE6VjiRyof0mwsy+L/jeuNw4IP1xWw5Oti05FEJAB88sX/scKqJdJrU1t7Jzef1cF0JPEzlQ9pVj3T4rlxqO/W2z9+sAGPbr0VadGqKg/yxNa3AOhdksHdl11JuFNvRS2N/heXZnf/Rd2Ji/Ldevv2l3tMxxERg1765G4KnBZt671EJDzAqB4ppiOJASof0uzaxEby8wu7Ab5bb0sr6wwnEhET9uWv4IWStQCkHBjGg5efZTiRmKLyIX4xcXhHuqbEUlxRy9/nbDEdR0QMeGLevVQ7LLpXOejb5xd0TYk1HUkMUfkQvwh3Ovj9FWcA8MrinWzc5zacSET8afnqF/jE48Jh29ilN/Cz0d1NRxKDVD7Eb0Z0TebSvml4bXh0xlfYthafirQE9XXVTFr5DwD6lSbywzE/JC4q3HAqMUnlQ/zqt5f1JircwbKdB5m5Jt90HBHxg3fmPcBWp5c4jxfC72X8gEzTkcQwlQ/xq8zEaO4+33fuy//9b6POfREJcaWunTyVPxeAzsU9eHD8RViWTq1t6VQ+xO9+fG5nOrSJYX9ZDU/N22o6jog0o399nEOpw0FWjU3nrr/ljIwE05EkAKh8iN9FhTt59PLeADy/cAdbC8sMJxKR5rB12yzertoFQLzrEu6/uK/hRBIoVD7EiAt6pnJhzxTqvTYPv7dei09FQozt9TJp4cN4LIs+5ZGMH/UzEmN0fov4qHyIMb+74gyiwh0s3XGQd1fmmY4jIk1o7qLHWO6oIcJrE2lnc93gLNORJICofIgxWa1j+NmhnU///OFGXJW1hhOJSFOorirh8S3/AaCPK51fT7gOh0OLTOUbKh9i1I/P6Uy3QzufPv7xZtNxRKQJvDjrHvY5LZLrvXTv/Ht6pcebjiQBRuVDjIoIc/DH8X0AeH3pblbuLjGcSEROR8G+VbzgWg1Ax9KzuPeSoWYDSUBS+RDjzurchglntgPgt9PXU+/xGk4kIqfqsdk/o9ph0a3KwXWj/0iryDDTkSQAqXxIQPjNpT1JiA5n4z43Ly3aaTqOiJyCL1e/xBzbhWXbtHdM5JK+6aYjSYBS+ZCA0CY2kgfH9gTgydlb2HOw0nAiEWkMT30tf17xJAAD3An84pps7WQqx6XyIQHj+sFZDO3Ymspaj/b+EAkyr3/yS7aG2cR6vAzv8UeyWseYjiQBTOVDAobDYfHnq/sS4XSwYMsBHTwnEiRKS3fz7wLf+S0DK3rz49GjzAaSgKfyIQGla0os91zgO3ju9+9v4GCF9v4QCUheD+xYCOum8eQ7P8DldNCu1ubOK54izKm3Fjkx/YRIwLnrvC70SI3jYEUtf/rfBtNxROQIU1ZPYeqce2FyH3h5HNtn3sUMx0EAMp0ZLCp512xACQoqHxJwIsIc/GVCXywL3l2Zx2dbDpiOJCKHOIq2kJs3l6mOcmzgsTZJeCyLTrW1LHXuw1G0xXRECQIqHxKQBrZPYuLZHQH4zfR1VNbWmw0kIuD1kL32Y3JKXOQmJfJg2zYsjo7GYdvsiIggp6SU7LWf+KZkRE5A5UMC1i8v7kFmYjR7S6r4q7ZeFzFv1yJw55PtcpNd4uLD2FYAeC2LnBIX2a5ScOf5rhM5AZUPCVixkWH8+eq+ALy0aCfLdhw0nEikhSsvbPit2+Fs+H24bZPtch/zOpFjUfmQgHZe97ZcPzgL24YHpq2hqlbDuSLGxKYCMD86mtcT4gAIs23qLIupifHfuU7keFQ+JOD9dlwv0hOi2FlcyROfaPpFxJgOwymIT+VXKW0A6F9dzaqdexrWgExNTID4TOgw3HBQCXQqHxLw4qPCG6ZfXvhiB1/u1PSLiAker4cfJkRS7XCQXF/PC/v2A5DtcpNTUkpuUgJT+42BI6ZkRI5F5UOCwvk9Urh2UDtsG341bS3VdZp+EfG3ye9OpCAMwrxenimuI+KI17K9seRkXog3ubuxfBI8dNaxBI2Hx/Xms60H2FFUwRMfb+bhcb1NRxJpMZatfpVXK9eBZTGmfjjdfznVd1dLeaFvjUeH4WRrxENOkkY+JGgkRIcz6dD0y/Nf7GDJ18WGE4m0DO7SPfx2xWN4LIuB5a349Q1P+aZWOp0Lfa/x/ariIY2g8iFB5YKeqQ13v/ziv2soq64zHUkkpNleL7+dfgMFYRbpdTY3Dn+W1nFRpmNJkFP5kKDz/y7vTVbraPJcVfz+fZ39ItKc3vzkfuZbbsJsm7MibmfswL6mI0kIUPmQoBMbGcaT1w3AsmDair3MWl9gOpJISNq+fTZ/2zcHgMGlXXng+nsMJ5JQofIhQWlIx9bcNbIL4Dv7ZX9ZteFEIqGlprqUX8z/BTUOi16VYdxx+b+JjdQ9CtI0VD4kaN13UTd6psVxsKKWh95Zh23bpiOJhIy/TL+B7WE2SfVeBqf/H0M7p5iOJCFE5UOCVmSYk8k3DCDC6WDupv28vmy36UgiIWHeoseZVrsXgB7lY7l33CWGE0moUfmQoNYzLZ5fXdwDgD9+sIGthWWGE4kEt4KC1Tyy6WUAznQl88vrHyUiTG8V0rT0EyVB7/ZzOnFut2Sq67zc88Yq7X4qcoo89bU88NHtlDoddKqGCwbl0iMtznQsCUEqHxL0HA6Lv13XnzatIthUUMZfPtpkOpJIUHrm/dtY5aglxuulk/NebhnRy3QkCVEqHxISUuKieOLa/gC8tGgn8zYVGk4kElxWrX2NZ0rXANDr4BAevfGHWJZlOJWEqiYvH7/73e+wLOuoR8+ePZv6y4h8x/k9U7htREcAfvn2Wva7dfutyMkoLd3NA1/+Ba9l0d8dwx1XPkHrVhHf/4Eip6hZRj7OOOMM9u3b1/D4/PPPm+PLiHzHg2N70is9noMVtdz/3zV4vLr9VuREbK+XR967kQKnRXqtl74d/8aIrsmmY0mIa5byERYWRlpaWsMjOVk/yOIfkWFOnrpxANHhTj7fVkTup9tMRxIJaG99ch/z8G2f3r76h9w3drjpSNICNEv52Lp1KxkZGXTu3Jmbb76Z3buPv/9CTU0Nbrf7qIfI6eiaEscfx/cBYPKcLSzaXmQ4kUhg2rb9E57YNxeAAQe78OjNP9NtteIXTf5TNmzYMF566SVmzZrF008/zY4dOzj33HMpKzv2/guTJk0iISGh4ZGVldXUkaQFumZQO64d1A6vDT97Y7W2Xxf5luqqEn4x/5e+7dMrwrjuolyyWseYjiUthGU3857ULpeLDh068OSTT3L77bd/5/Wamhpqamoa/ux2u8nKyqK0tJT4+PjmjCYhrqrWw/jcL9hcWMbZndvw2o+H4XRo9b4IwKP/uZh36/NJqvdyceJf+e2ES01HkiDndrtJSEg4qffvZh9fS0xMpHv37mzbduy598jISOLj4496iDSF6AgnuTefSUyEk8VfF/OPOVtMRxIJCB8vnMS79fkAnFE5jl9dqe3Txb+avXyUl5ezfft20tPTm/tLiXxH15RYJl3dF4CnPt3Ggi0HDCcSMatg3yr+sPU1AAa7Unj4h7/TOg/xuyb/ifvlL3/JggUL2LlzJ4sWLeKqq67C6XRy4403NvWXEjkpVw7I5Mah7bFt+Pmbq9hzsNJ0JBEj6uuq+cWHt+N2OuhcDTee/yyZidGmY0kL1OTlY+/evdx444306NGD6667jjZt2rBkyRLatm3b1F9K5KQ9enlv+rdLwFVZx12vrqCqVue/SMvz93dvYW1YHTFeLyNTHmZMv86mI0kL1ewLThurMQtWRBoj31XF5U99TnFFLVcPzORv1/XX9tES2rwe2LUIygv5In8lP903E69lcVHVcJ64cyoOLcCWJtSY9+8wP2USMS4jMZqnbhrID59fxrur8uiflcjE4R1NxxJpclNWT8FRtIXstR+DO59Sh4NHM9PwhoXRvsZBu0F9VDzEKK0ykhZleJdkHhrrO2vojx9sYPnOg4YTiTQ9R9EWcvPmMtVRjg08mtyawrAwEuo97I70ElO63XREaeFUPqTFuf2cTlzeP4N6r81PXltJvqvKdCSRpuP1kL32Y3JKXOQmJXJPSjJzW8XgsG1Kw5zklJSSvfYT35SMiCEqH9LiWJbFYxP60jMtjqLyGu545Usqa+tNxxJpGrsWgTufbJeba91lLGjl27XUa1nklLjIdpWCO893nYghKh/SIsVEhPHcxMG0aRXBV/lufvHfNXh1Aq6EgvJCAHaHhTG31TfbpYfbNtku93euEzFB5UNarHZJMTzzw0GEOy0+Wl/AZO2AKqEgNpUDTgd3pqVw0OkEfMWjzrKYmhh/1HUipqh8SIs2uGNr/nyVbwfUf87bxvtr8g0nEjk97qT2ZKemkBfuu5nxNlcpK3fuaVgDMjUxAeIzocNww0mlJdOtttLiXTs4i637y/n3Z1/zy7fX0L51DP2zEk3HEmm0mupSsqddxZbICAB+UOrm/pJSgENTLha5SQmQeSHZDqfBpNLSaeRDBPj1JT25oGcKNfVebn95ubZgl6Djqa/l569fwrrwOsJsm/MdXfh1fcxR12R7Y8nJvBBvcndDKUV8tMOpyCHlNfVcN3UxG/a56dK2Fe/8ZDiJMRGmY4l8L9vr5cHXxvChXUi4bXN12A08/IOHj9rhlNhU31SLRjykmTTm/VsjHyKHxEaG8eJtQ8hIiGL7gQrufHUFNfXaC0EC3xNvX8eHdiGWbTO69kJ+c9NvfS84nNDpXOh7je9XFQ8JECofIkdIjY/ixduGEhcZxrIdB/nl22t1C64EtBfez+aV6s0AjKzozx9ue1Jbp0vAU/kQ+ZYeaXFM/eEgwhwW76/J57GPN5mOJHJMMz/9HZOLPwfg7NIs/nDri0SFa3RDAp/Kh8gxjOiazGMT+gHwzIKvef7zHYYTiRxt4bIpPLprGrZlMdidxEM3vEXrVlqjJMFB5UPkOCYMasevLu4B+A6he2fFXsOJRHzWbniHX3yVS71l0a88irvHvU2nlDjTsUROmsqHyAn8dFQXbj+nEwAPvLOWORu0JbWYtWPnAnKWPEKVw0HPSie3j3qLQZ20W6kEF5UPkROwLIvfXtqLq8/MxOO1yXl9JUu/LjYdS1qo/YXruXNuDi6ng441cNOgV7jgjM6mY4k0msqHyPdwOCwem9CP0b18m5D9+OUvWZ9XajqWtDDu0j3c8cFNFIRZpNd6mdD1Ka4a2s90LJFTovIhchLCnQ7+ddOZDO3YmrKaem55YRmbC8pMx5IWorqqhLumXcHXYTat671clvp7br1glOlYIqdM5UPkJEWFO3nu1sH0a5fAwYpabn5uCdv2l5uOJaHG64EdC2HdNNixkPqaCu55/RLWh9XTyuvlktif8bPLJ5hOKXJatL26SCO5Kmu56dmlbNjnJiUukrfuOptOya1Mx5IgN2X1FBxFW8he+zG4facr28AjbdvyXmw0DtvmmrCbefjmB7EsbSImgUfbq4s0o8SYCF778TB6pMaxv6yGm55dooPo5LQ5iraQmzeXqY5vRtOeSkrgvdhoAM6sb6viISFD5UPkFLRuFcF/7hhG15RY9pVWc8O/VUDkNHg9ZK/9mJwSF7lJiUxNjOc/8bE8m5gAwKiKSl5wF2LZXsNBRZqGyofIKUqOjeT1Hw+jU3Ir8lxVXPfMYr4+oDUgcgp2LQJ3Ptkud0MB+Uub1gAMq6ziqf1FWO4833UiIUDlQ+Q0pMRH8eadZzWMgFz3zBLdBSONV+7bvM4Gor02HFqK57Btni088J3rRIKdyofIaUo9VEB6pcdTVF7DDf9erH1ApHFiU/EAj7VO4ok2SWBZOGwbr2XxTGL8UdeJhAKVD5EmkBwbyRt3DKN/uwRKKuu48dklrNxdYjqWBInqlF7cn5rCfxJ857MMr6xi9c49R6wBSYD4TOgw3HBSkaah8iHSRA7fBTO4QxJl1fXc/OxS5m/ebzqWBLji4u3c8sZo5sVEATCmvIJnCg9gwaE1IKXkJiUwtd8YcDjNhhVpIiofIk0oLiqcV24fyrndkqmq8/Djl7/k3ZU6DVeO7eudn3Pze1eyMdxDuNfmHNrxt5rIo67J9saSk3kh3uTuhlKKND1tMibSDGrrvTwwbQ3vrfZtFvWbS3ty58guhlNJIPlyzRvct+JPuJwOUuq8TOzwCLdcdL1vh9Ndi3yLS2NTfVMtGvGQINCY9+8wP2USaVEiwhw8ed0A2sZF8uzCHfz5w03sd9fwm0t74XBok6iW7sPPHuOR7a9S43TQqQayB0zl0qHn+l50OKHTuWYDijQzlQ+RZuJwWPz2st6kxEXxfx9u5LnPd5DnquLJ6wYQHaH/km2pnp3xE/5VshCvw+KMynDuu/AthnXvZjqWiF+pfIg0sztGdqZtXCQPTFvLR+sLyHMt5tlbBpMaH2U6mjSXY0ydeG2bP715NW/X7wDLYnB5PL8e/x49M9uaTivid1rzIeIny3ce5K5XV3Cwopa0+CiemziYPpkJpmNJEzrW4XAAtXEZXJMYyY4wDwAjKzry+x/8l+T4aFNRRZqcDpYTCUBDOrbmvZ+OoGtKLAXuaq6duphZ6wtMx5ImdKzD4dwOiysSbHaEeXDYNuPqzubJO99T8ZAWTeVDxI/at4nh3Z8Ob7gVN/u1Ffz14014vAE1ACmn4hiHw+1zOhmXmU5eeDjhXi+Ti6r4821TiAzTmh9p2VQ+RPwsPiqcF28dwm0jOgKQ++l2bn1xGQcras0Gk9NzjMPhxmRlUBIWRiuPhzf2FXJ+eRHW7sWmk4oYp/IhYkCY08Gjl5/BP24YQHS4k4Vbi7j8qc9Zu9dlOpqcqkOHvtUDFvgOh7MssG3eyyugR23dUdeJtGQqHyIGXTkgk+k5w+nYJoY8VxXXPL2YVxbvJMDWgcvJiE1lT5iT29JT+VdSIlgW1qEC8l5cq6OuE2npVD5EDOuZFs/Me87hot6p1Hq8PDLjK+54ZYWmYYKI7fXy3NppXJOZzuoo3/boo8srWKPD4USOSeVDJADER4Xz7x8O4pFxvYlwOpizsZCx//iMRduKTEeT71Fc/DV3vTCCf7jmUenw/ZP6w1I3fz9QrMPhRI5D5UMkQFiWxY/O6cT0nOF0aduKQncNNz+/lEkfbaSm3mM6nng9sGMhrJvm+9Xr4aOF/2DCe5ezOLycMNumc31rfpwyigfqY476UB0OJ3I0bTImEoAqa+v54wcbeWPZbgC6pcTy12v7MyAr0WywFuhYG4dVWxZPtk7mjUN7dbSrtbmu3X3cesmPsCxLh8NJi6SD5USCXExEGJOu7suoHm357fT1bN1fztVTvuCOcztz30XdiQrXG5m/HN44DEc52cDmiHAebNuGbRERAHSuCeOxS2fSs13WER+kw+FETkTlQySAXXxGGkM7tuYPH2xg+qo8nvnsa2ZvLGTSVX0Z1rmN6Xih79DGYTjKyU1KZFVkJMujo6izfCcTjysr58+10VgZGYaDigQXrfkQCXBJrSL4+/UDePaWwaTERfL1gQqu//cS7ntrNfvLqk3HC22HNg7rV11L6/p6FsVENxSPH7lKmVR0EMud57tORE6ayodIkLiodyqz7zuPm4a1x7Jg+qo8LnxiAS9+sYN6j9d0vOB1jIWkh63fOpufprblrvQUDoaF+TYOA8Jtm/tKSr/5HNo4TKRRtOBUJAit2ePi/81Yz9q9vjfAnmlxPDi2J+d1b+tb8Cjf63gn0BKfwd+6D+ezA1+xyyrHY1mE2Ta9a2pYGxVFuG1TZ1nklLjIdrl9HzPxA63xkBZPp9qKhLj+WYlM/+kI/nxVXxJjwtlUUMatLy7n5ueWsm5v6fd/AjnmCbS1wF3RdbxUvIyvHRV4LIvzKqq5zl3G2qgockpcrNTGYSKnTQtORYKU02Fx07D2XNo3jdxPt/Hyol0s2l7M5f/6nMv7Z3Df6G50bhtrOmZg+Patr1nDjlpICtC5to7fJbemzOm7k6hHTS0XpdyONyOfKflzySkpbRjp8P1qkZuUAJkXkq3baEUaRdMuIiFib0klT36yhemr87BtcFhwad90cs7vSq/0lvn/peNOrcQkMzWilnosdoSH8UnsN2evxHg8PHTQxeXlFTgnfsCU0nXHmZ7JZGq/MXiTu/PTAT/1499KJDA15v1b5UMkxGzId/Pk7M3M2bi/4bnRvVLIOb8rA9snGUzmf1Pn3Etu3tyj12cA/0xM4NmkBOI8noaRDgCHbbN4115iDv+zOOF56HuN7/faOEzkhLTJmEgL1jsjnucmDmFDvpvc+dv4cN0+5mzcz5yN+xmQlcjE4R24tG86kWEh+MZ5ZEFo1ZbsNbPAWdEwtTK+rIIH2rZhVXQUAGVOJ1FeL9UOR8NC0lcS4r4pKkeeQKuNw0SajEY+RELc9gPlPD1/OzNX51N76JbcNq0iuGFoFjcMaU9W65jv+QyB77jTK8DTifHMjYlhc2SE71bZQ3cDdayto2NdHfNbxTSMjExNjCc3KdG3vsMbC/eu0+iGyEnSyIeINOjSNpYnru3Pry/pyVvLd/Pakt0UuKvJ/XQ7uZ9uZ2jH1lx1ZiaX9kknISbcdNxT8u0t0AG2hofzpzZJrDw0ygGAZWHZNk8VHmBDZARTkhKPmpLRQlIR/9DIh0gLU+/xMntDIa8t9d0dc/hfgAingwt6pjDmjFTO75FCUqsIs0FPltcDk/sw9dCdK8Mrqyh2On0jHYdEeL3UOhyE2Tb1h/bo8GLhwCa7Nhwqi7/5fFpIKnJKAmLBaW5uLn/9618pKCigf//+PPXUUwwdOvR7P07lQ8R/9pVWMWN1PtNX5rG5sKzheYcFgzokcWGvVM7r3pYeqXE4HIG3eVlVrYdtyz6i75ybAbg+PZUNUZENr3eqraVHbR2zYlsdf2rlZ6thz1ItJBU5TcbLx1tvvcUtt9zC1KlTGTZsGJMnT+btt99m8+bNpKSknPBjVT5EzNiQ7+aj9b7FqRv3uY96LT4qjCEdWzOkU2uGdGxNr/Q4YiL8P2tbVF7D2r0ulu44yPIdB1mXV8pY+wv+GfEvAL6MiuRHaSnYlkW4bXOnq/RQ0Tj6bpepiQnkJiWQk3kh2aMn+/3vIRKKjJePYcOGMWTIEP71L98/CF6vl6ysLO655x4efPDBE36syoeIeXtLKpm3aT9zN+5n+c6DVNZ6jnrdsqBjm1b0So+jZ1o8XdrGkpkUTUZiFG1jI09ri/c6j5eC0mr2llSxt6SSrfvL2bjPzcZ9ZRSV13zn+ktjtzGl/hGAhlGNw3euDKmqZmh19VHFA9DUikgzMFo+amtriYmJYdq0aYwfP77h+YkTJ+JyuZgxY8YJP17lQySw1Hu8bNjnZtmOgyzbcZCVu13HLAGHRYQ5yEiIIiEmgvioMOKjwomPDiMyzMmRncS2obymnrLqOtxV9bir6ygur6WwrJrj/atkWdCpTauGUZhhnVrTLiEC6x99meqo8I1mHG965copUFmkqRWRZmL0bpeioiI8Hg+pqalHPZ+amsqmTZu+c31NTQ01Nd/8Q+Z2u79zjYiYE+Z00K9dIv3aJfLjczsDcKCshs0FZWwqcLNhn5tdxZXklVRRWFZNbb2XncWVUFx5yl8zIsxBZmI0mYnRdE2JpWdaHD3T4+meGnvM6Z6p/S4+tJnYCbZA7zLqlPOISNMyfqvtpEmT+P3vf286hog0Qtu4SNrGRXJOt+Sjnq+t91LoribfVYW7uh53VZ1vZKO6npr6b03dYBET6SQ+Kpy4QyMkiTHhZCZFk9wqslELXL3J3ckBsos/Br45WC/bGwuZF+JN7n46f10RaWLGp12ONfKRlZWlaRcRaTxtgS5iTGOmXRxN/cUjIiIYNGgQc+fObXjO6/Uyd+5czj777O9cHxkZSXx8/FEPEZFTcngL9L7X+H5V8RAJSM0y7XL//fczceJEBg8ezNChQ5k8eTIVFRXcdtttzfHlREREJIg0S/m4/vrrOXDgAI888ggFBQUMGDCAWbNmfWcRqoiIiLQ82l5dRERETpvRNR8iIiIiJ6LyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+1Szbq5+Owxuuut1uw0lERETkZB1+3z6ZjdMDrnyUlZUBkJWVZTiJiIiINFZZWRkJCQknvCbgznbxer3k5+cTFxeHZVmm4xjndrvJyspiz549OuumGen77B/6PvuPvtf+oe/zN2zbpqysjIyMDByOE6/qCLiRD4fDQbt27UzHCDjx8fEt/gfbH/R99g99n/1H32v/0PfZ5/tGPA7TglMRERHxK5UPERER8SuVjwAXGRnJo48+SmRkpOkoIU3fZ//Q99l/9L32D32fT03ALTgVERGR0KaRDxEREfErlQ8RERHxK5UPERER8SuVDxEREfErlY8gVFNTw4ABA7Asi9WrV5uOE1J27tzJ7bffTqdOnYiOjqZLly48+uij1NbWmo4WEnJzc+nYsSNRUVEMGzaMZcuWmY4UUiZNmsSQIUOIi4sjJSWF8ePHs3nzZtOxQt5f/vIXLMvi3nvvNR0laKh8BKEHHniAjIwM0zFC0qZNm/B6vTzzzDN89dVX/P3vf2fq1Kn85je/MR0t6L311lvcf//9PProo6xcuZL+/ftz8cUXs3//ftPRQsaCBQvIyclhyZIlzJ49m7q6OsaMGUNFRYXpaCFr+fLlPPPMM/Tr1890lOBiS1D58MMP7Z49e9pfffWVDdirVq0yHSnkPf7443anTp1Mxwh6Q4cOtXNychr+7PF47IyMDHvSpEkGU4W2/fv324C9YMEC01FCUllZmd2tWzd79uzZ9nnnnWf//Oc/Nx0paGjkI4gUFhZyxx138OqrrxITE2M6TotRWlpK69atTccIarW1taxYsYLRo0c3POdwOBg9ejSLFy82mCy0lZaWAujnt5nk5ORw2WWXHfVzLScn4A6Wk2OzbZtbb72V7OxsBg8ezM6dO01HahG2bdvGU089xRNPPGE6SlArKirC4/GQmpp61POpqals2rTJUKrQ5vV6uffeexkxYgR9+vQxHSfkvPnmm6xcuZLly5ebjhKUNPJh2IMPPohlWSd8bNq0iaeeeoqysjIeeugh05GD0sl+n4+Ul5fHJZdcwrXXXssdd9xhKLnIqcnJyWH9+vW8+eabpqOEnD179vDzn/+c//znP0RFRZmOE5S0vbphBw4coLi4+ITXdO7cmeuuu473338fy7Ianvd4PDidTm6++WZefvnl5o4a1E72+xwREQFAfn4+o0aN4qyzzuKll17C4VBPPx21tbXExMQwbdo0xo8f3/D8xIkTcblczJgxw1y4EHT33XczY8YMPvvsMzp16mQ6Tsh57733uOqqq3A6nQ3PeTweLMvC4XBQU1Nz1GvyXSofQWL37t243e6GP+fn53PxxRczbdo0hg0bRrt27QymCy15eXmcf/75DBo0iNdee03/iDSRYcOGMXToUJ566inANy3Qvn177r77bh588EHD6UKDbdvcc889TJ8+nfnz59OtWzfTkUJSWVkZu3btOuq52267jZ49e/LrX/9a01wnQWs+gkT79u2P+nNsbCwAXbp0UfFoQnl5eYwaNYoOHTrwxBNPcODAgYbX0tLSDCYLfvfffz8TJ05k8ODBDB06lMmTJ1NRUcFtt91mOlrIyMnJ4fXXX2fGjBnExcVRUFAAQEJCAtHR0YbThY64uLjvFIxWrVrRpk0bFY+TpPIhcoTZs2ezbds2tm3b9p1Sp0HC03P99ddz4MABHnnkEQoKChgwYACzZs36ziJUOXVPP/00AKNGjTrq+RdffJFbb73V/4FEjkPTLiIiIuJXWkUnIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+pfIhIiIifqXyISIiIn6l8iEiIiJ+9f8BYGdCfJk4eWcAAAAASUVORK5CYII=",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"def function(x):\n",
" return x**2\n",
"\n",
"def analytical_gradient(x):\n",
" return 2*x\n",
"\n",
"def gradient_descent(starting_point, learning_rate, num_iterations, solver=\"analytical\"):\n",
" x = starting_point\n",
" trajectory_x = [x]\n",
" trajectory_y = [function(x)]\n",
"\n",
" if solver == \"analytical\":\n",
" grad = analytical_gradient \n",
" elif solver == \"jax\":\n",
" grad = jax_grad(function)\n",
" x = jnp.float64(x)\n",
" learning_rate = jnp.float64(learning_rate)\n",
"\n",
" for _ in range(num_iterations):\n",
" \n",
" x = x - learning_rate * grad(x)\n",
" trajectory_x.append(x)\n",
" trajectory_y.append(function(x))\n",
"\n",
" return trajectory_x, trajectory_y\n",
"\n",
"x = np.linspace(-5, 5, 100)\n",
"plt.plot(x, function(x), label=\"f(x)\")\n",
"\n",
"descent_x, descent_y = gradient_descent(5, 0.1, 10, solver=\"analytical\")\n",
"jax_descend_x, jax_descend_y = gradient_descent(5, 0.1, 10, solver=\"jax\")\n",
"\n",
"plt.plot(descent_x, descent_y, label=\"Gradient descent\", marker=\"o\")\n",
"plt.plot(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\")"
]
},
{
"cell_type": "markdown",
"id": "8a5f19b5",
"metadata": {},
"source": [
"### A more advanced example"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "d8f5eb38",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.collections.PathCollection at 0x130a7b520>"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGdCAYAAAAvwBgXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB8EklEQVR4nO29eXgk5XXv/63e1WrtGmk2zQ4MGAbMMsNmG2w2G1/HdoIhxgtcmyfJBS4EJ8CYxNwktoc4xBu2MfHvF5sEMCQkQLzAhRiDbdZh2DEzwwyj2bSv3Wqp16r7R/dbVdK0WrW8b9Xb6vN5Hj3PjEZSvyO1qk99z/d8j6JpmgaCIAiCIAgfCPh9AIIgCIIg6hcqRAiCIAiC8A0qRAiCIAiC8A0qRAiCIAiC8A0qRAiCIAiC8A0qRAiCIAiC8A0qRAiCIAiC8A0qRAiCIAiC8I2Q3weohqqq6OvrQ1NTExRF8fs4BEEQBEFYQNM0pFIpLF++HIFAdc1D6kKkr68PPT09fh+DIAiCIAgHHDx4ECtXrqz6MVIXIk1NTQBK/5Hm5mafT0MQBEEQhBWSySR6enr01/FqSF2IsHZMc3MzFSIEQRAEUWNYsVWQWZUgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgCN+gQoQgCIIgJOG1gxP459/tg6pqfh/FM6TevksQBEEQ9cRXHnkTrx2axHHLm3H6ug6/j+MJpIhUIZMvQtPqpyolCIIg/GU4lQUAHByb9vkk3kGFyDwcGp/Ge//2CVxz3ytUjBAEQRCekMoUAABD5YKkHqBCZB7e6ktiJl/EL97ox4M7Dvl9HIIgCGKRo6oapnLlQiSZ8fk03kGFyDxkC6r+57/7+e/r6klBEARBeM9UrgAmwJMiQiCbL+p/TmYK+KuH36QWDUEQBCEM1pYBgME6uvmlQmQeMmVF5KiuBEIBBY//fhC/eKPf51MRBEEQi5VUJq//eTBJikjdwxSR45Y343+duwEAcOsjb2EsnfPzWARBEMQiJTljKCLDqWzdqPBUiMwD84hEQwFcc+4GHN2dwGg6h7/52Vs+n4wgCIJYjJgVkVxRxcR0vspHLx6oEJkHoxAJIhIK4Bt/dCICCvDIq314+cC4z6cjCIIgFhtmjwhQP4ZVzwqR2267DYqi4Prrr/fqIV3BWjOxcOlbdFJPK85YX0q52zec9u1cBEEQxOLErIgA9WNY9aQQ2b59O+666y5s2rTJi4fjglkRYcQjpUT8TKFY8XMIgiAIwinJOYoIFSKcmJqawuWXX44f/ehHaGtrE/1w3MiWi41oyPgWxcKloiSTVyt+DkEQBEE4hVozgrj66qtx8cUX47zzzhP9UFxhxUY0bCpEykVJlhQRgiAIgjOsNaMopb/XS5Cm0O27999/P15++WVs377d0sdns1lks0YFmEwmRR1t4bMUmEfEaM2wooQUEYIgCII3TBHpaYvjwNg0KSJuOXjwIK677jrce++9iMVilj5n27ZtaGlp0d96enpEHW9BsnljfJcRK/tFSBEhCIIgeMMUkfVLGgGQR8Q1O3bswNDQEE4++WSEQiGEQiE8/fTT+O53v4tQKIRi8cgX861bt2JyclJ/O3jwoKjjLUhG94gcqYhkSREhCIIgOMMUkQ1dCQD1k64qrDXzoQ99CG+88cas91155ZXYuHEjbrrpJgSDwSM+JxqNIhqNijqSLVixEQsfqYhk8qSIEARBEHxhhcj6JaVChKWrKsw0skgRVog0NTXh+OOPn/W+xsZGdHR0HPF+Gak0vsv8IubNvARBEATBA701U1ZEWLpqW2PEz2MJh5JV56HS+K5hViVFhCAIguALU0Q6GiNojYcB1McIr9Cpmbk89dRTXj6cK4zxXZMiEiJFhCAIguCPqmqYypUKkaZYGN1NMUxM5zGYzOCYpU1CHvPd4Sn0jqaxsi2Oo7vFPIYVSBGZB1JECIIgCK+YyhXAlu02xULoai75JUVOzvzi9X78z5+8hB8/s0/YY1iBCpF5YKqH2awaJbMqQRAEIQDWlokEA4iFg+hqKsVeiGzN5Iqq/ph+QoXIPLBiY7ZZlSWr+tOaGZnK4p3BlC+PTRAEQYiDGVWbYiXHRHdZERGZrporv5ZFQv6WAp56RGoFTdOMqRkJFJHJ6TzufHovfvzMPmQLKp76i3OwprPR0zMQBEEQ4mCKCCtEuprKhYhARSRLhYi85Iua3qurpIh4FfGeyRfxk2d7cedTezE5Y6yHPjA2TYUIQRDEIsJQRErTMt3NpdaMSI8Ia82EfW7NUCFSAXOEe6Xtu160ZnpH0vjjHz2P/snSk/Do7gTS2SIOT8xgKltY4LMJgiCIWuIIRUQvRMQpInlJFBHyiFTArHjMmpph23c9aM08vXsY/ZMZdCaiuP2SE/Hode/H0d2lkJupDBUiBEEQi4nkPK0Zlq4qAjKrSox5dNccreulIsJUjw9uXII/OmUlggEFibJklyJFhCAIYlExtzXDxndZuqoIcoUjl7v6ARUiFcjO88Nhf88VVRRVMRUqY7ocbBOPGN2zRLT0Z1JECIIgFhdzWzPRUFB4uqosUzNUiFRAH90Nz17MFzP93ewjEUE6W/r6jVHjMdkTdCorpjomCIIg/GGuIgIA3U1iDauymFWpEKlApTAzYLZCkhU8OVNVEaHWDEEQxKKCKSLNMeOaLzpdlRQRiWFFhnl0FwBCwQBCgZJnJCNaEcmVFZGIcQZWiKSoNUMQBLGoSM7MDjQDIDxdlcyqElNpzwxDN6yKVkTKqkc8alJEYqSIEARBLEYMRcTUmhGcrkqKiMSw8d3YHI8IYBQn3ikiRiHSRGZVgiAIrgxMZoR7/qxgmFWNQkR0uioVIhJjRRERna6qe0RMZlVSRAiCIPhxYHQaZ972K3zsjmdmpVf7wdxdM4D4dFVqzUjMfOO7gLF7RnSo2XT2SEWEPCIEQRD8eHsgCVUDdg2m8Gf37NAVAj+YO74LiE9XpWRViWFFRuXWTFkREfyETetTM5XGd6kQIQiCcMvEdE7/87N7R/Hlh94QlmJaDVXVMJWbvzUjKl1VV0SoEJGPaoqIsfjOI0XEbFaNlp6gU9mCL78sBEEQi4mxdKkdsn5JIwIK8OCOQ/jek3s8P8dUrqAvWm2qML4rKl1V375LrRn5MAqRIxWRWEh8zLumaboiMmt8t/wELaqaZxuACYIgFivjZUXkgxu78Ld/cDwA4B+f2I1HXj3s6TlYWyYSDMxS4qOhINoEpquSWVVijGTV+T0iIhWRbEEFS5A3j+/Gw0Gw1TcpSlclCIJwxXi6VIi0NUbwmdNX40/evw4A8Jf//jr2DKU8O0cloyqjS2C6ap7MqvJiJKv6o4ikTR6QBtMZAgEFiQiN8BIEQfCAKSJt8QgA4KaLNuKU1W3IFVU89+6YZ+eoZFRliEpXLRSNG15SRCSEmVX9mpqZLmeINISDCAaUWf9GI7wEQRB8GEvPLkQCAQXrOhsBGCqFF1TaM8MQla7KjKoAFSJSUtWsyqZmBBYiuj8keqQiQxt4CYIg+MAMoO2NEf19rBhIznh3ja2miIhKVzWPKlNrRkIyVcZ32dSM2NZM6fHNC+8YTBFJkSJCEAThijG9NWMoEawY8FIRSVYtRAQpIuXXMEXBEcq711AhUoHqgWbiFZHpChkiDFJECIIg3FNUNT1Ntc2kiDQ3lIoSL4Mjq7dmSorIAG9FxGRUVRQqRKSj+viud4qIOUOEIVOo2e7BFM7c9iv820sH/T4KQRCELSZn8np2R2vDkYpI0lOPyPyKSGvZv8K7MJJldBegQqQi1cd3JVFEJChEfvlGP/omM/jn3+3z+ygEQRC2YEbV5lgIIZNHojnm/SqNaoqIPiDBeTEfU0QqKf9e4/8JJKSaIqJv3xUYKFZp8y6DpavKsG+mdyQNANg5kMLolJhdCARBECJg8e5moyoANMfYNdZ7RaS5giLCXnOynF9zcpKkqgJUiFRE375bQRFhBlaRa6Ons0du3mUY47v+B5rtG53W//y8hzP3BEEQbmGKCGt9MGSbmokKyq5ihUiYFBE5YZVnRbOqz4pIk0Rm1f2jaf3Pz+4d8fEkBEEQ9hifRxHxY2qmamsmJLY1Q4qIpGQK1cZ3PfCIWFJE/C1EJqZzs5YwPbt31MfTEARB2GO8fP1qjc9+8WeFSDpXRKHozU6vqoqIKTKC57JTMqtKTjVFxGjN+OUR8d5IVYl9ZX9IS0MYAaX0976JGV/PRBAEYRW2Z6Z9ntYM4N0Nn1GIVFJESq85mgbki1SI1A3WzKo+Tc1IoojsL/tDNi5twgkrWwEAz5EqQhBEjTBmWnhnJhIK6MGVXt3wJassvTPfEPNsz1BrRnKMZNX5FZGcXzkikozvMkVkbWcjzlzfAYDaMwRB1A6sNdM2RxEBTIZVD3wiqqrp1/OFCxF+rzukiEiMpmm1oYj43JrpLRtVV3c04qz1nQCA5/aOcO1hEgRBiMIwqx7ZDmFjtF5MzqRzBT1YrblCa0ZRFL1Y4FmI5EkRkRfzRsJq47sZvz0iPisiveXWzNrOOE5Z3YZIMIC+yYzesiEIgpCZ8enZm3fNNHmYJcL2zISDyrzhYkaWCMfWDCki8mKuOCubVfk/IeZSbWqmqRxoliuoQrNMFoKFma3pbERDJIj3rmoFQO0ZgiBqg/F5PCKAeYRX/A2feXR3vp0vIrJEslSIyAtruShKZcmKPSFEKiLTVRSRRlNxwrwkXjOezunLola3NwIAziy3ZyhPhCAI2SmqGiZm5veIsMV3XnhEqo3uMqICWjNkVpUY8+hupeqUKSJFVRM2Y54ue0QaKygioWAADeX2kF8+kX1lf8jS5hgayj6WMzeUDKvP7R2FqpJPhCAIeUmaF97F5/eIeKuIVClEBCjxlKwqMdWMqsDskDNRqsh0WemIV1BEAMOwmvIp5p0lqq7pjOvvO3FlKxrCQYymc9g9lPLlXARBEFYYK/tDmmIhhCsoAl56RHRFJHpkQcQQ0Zohs6rEVBvdBWb/0ERMzuQKqi6ZVWrNAP7HvO8bYUbVRv19kVAAp61tBwA8u4d8IgRByIseZlbBHwJ4OzWT9Ks1U5g/uNNr/D+BZCykiAQCYkapGDM5o7hpqDC+C/gfasaMqqs7Gme9n/JECIKoBYx498qFiK6IeKA6V9szwxCxb4amZiRG37xb5YcjMkuE+UMiwcC8T5CEz6FmemtmnkLkhXdHPdvRQBAEYRcj3r3yi7+3UzOlx2huqOYRKbdmOC5bJbOqxOhm1XlaM4DYxXd6mFkFoyrDz30zmqbNSlU1857lLWiKhZDKFrBzgHwiBEHIyViVDBHACBZLzsimiPAf3yWzqoQwRSQ2T2sGMGWJCGjN6PHu8/hDAH9bM+PTeb2nuao9PuvfggFFV0kGkxnPz0YQBGEFPcxsHo+IL4qIJY8Iv5tftkCPFBEJ0T0iVRQRPUtEYGumUrw7w0+zKlNDlrXEKnpYmPlrtCx9EgRByMZCZlVj14x3hUh1syr/qZlcuaghj4iEGDki/igi+uhuhYV3DD8Vkfn8IYyORLkQmaJChCAIOTHMqpXbIcyv4U2gmYXWjJ4jQkvv6oJMofr4LmCqTgUqIo1VFJFElM24e1+ImKPdK9GZiAIARqeynp2JIAjCDoZZtboi4sUqDXvJqhynZoryjO/O/z+vU+woIhmO1SmDxbvPF2YGmBUR7wPN9pWX2q3piFf89w5qzRBE3TOdK+C5vaN4sXcM5x7ThdPXdfh9pFkws+p847sJkyKdyhQQTcz/euAWoxDxONCsUPKIVAp08xoqROZgZXw3pj8pBCgi2fnj3RnMI+LHrhkjVXW+1kxJERkhRYQg6orxdA4Pv3oYv941jOffHdWl/yfeGsSTf3GOv4ebw0S5NTOfRyQYUNAULU0AJmfyutIrgqSViHcBikhWovFdKkTmwFQOc5T7XKJ+KyJsfNdjj0i10V0GU0TGSBEhiLri2p++gt/tMZZeLm+JoW8yg32jaWTyxarXVC8pqhom9KmZ+VUIFkUgsgWuqpru9bO2a4Y8InWB74qIFY8Ia814YKQyM5bO6b+Uc0d3GWRWJYj6Q1U1vHxgHABw3YeOwn/f8H48c/MH0dIQhqYB7w6nfT6hQXImD7aXs7WhsiICmPfNiCtE0rmCvnyv2ePWDE3NSEzWQv5+VA8082lqxqdk1d5yW2Z5S2zeuxvWmhlNZ6FptIWXIOqBQ+MzmM4VEQkFcO0HN2BDVxMURcGGrgQAYM/wlM8nNGAZIk3RUNUXYS8mZ1iREw4qltK8RZhVZfCI+H8CyTCSVau0ZjyIeK+miDTF/MkR6S0vu5vPHwIYrZl8UfNkBp8gCP/ZOZAEAGxYkkDI9MK2YUm5EBmSrxBprdKWAbzZwGs2qiqKMu/HiUhWZWZVGaZm/D+BZGSstGbC/GUyhh1FJJ0roqh6pzr0LmBUBUrfG3Y+GuEliPqArXTYuLRp1vuZIrJXokJkLF02qs4zMcPwIl01ZcGoCgjeNUOFiHzUgiKSMD1p2cd7ATOqzje6y2A+ETKsEkR9sIsVIssqFyIyKiLzxbszvNg3YyVDBBC8fZdaM/JhyawqUhGxMDUTDQX1J4+X7Zn9eobI/IoIYIzEjZBhlSDqAtaaOWZp86z3s0Jk30hamo3cLMxsvoV3DFYciGwx66O70eptIhGtGZqakRgr47tGoJk/OSKAPzHvVlozANDRaBhWCYJY3GTyRV0tnduaWdHagFg4gFxRxaHxGT+OdwQs3n3hQkT81Ay78Vzoes97akbTNDKryowVRcRYeuePIgKYskQ8UkTyRVV/rK6m6uE+nTTCSxB1w56hKahaaW/L3GtDIKBgXadc7Rlj4V11FcKLqRm2JqSaFaD073xbM2zzLkCKiJRYGd+NcX5SmJnOWVREPB7hNRc8iSpGWsCcJUKKCEEsdsxG1UqTH7KN8C4U787wYmomU369iVVZKQKYWjOcbn5zpjYZTc1IiFGIVGvN8HcwM1hse+NCiojHI7zMsNUYCc4az6uE0ZohRYQgFju7yv6QjXP8IYz1ko3wslTV+eLdGV5MzWR0RaT6NZV3ayZn+jpkVpUQ9sSovn237BHhrIgUVQ0zedaaqV4hN+mKiDfpquyXsbmhupwJULoqQdQTTBE5Zo4/hCHb5Ayb5muNL9CaYVMzIlszdhURTq85rBAJBRQEAvPnl3gFFSJz8FMRmTGZXxsXaH8kPKjWzVhZzMQgsypB1A9WC5G9Q1NSpC2PL7DwjtEskyISNqZmeHwPWSEig1EVoELkCLIWnhiiFJHpst8joCzct/PaI8JaM9X2ITBIESGI+mAsncNwqnTDcUx35UJkTWccAaW0pHMo5e/NiWpaeLdwoJkxNSOqgNKnNBdUREr/rmmzjaZOkSnMDKBC5Ah081DV8V0xikg6Z/hDqsX9At57RJy0Zsamc54mvxIE4S0sP2RVe3xeFTcaCmJ1OXvI7/ZMMmNaeLdAIcKmZoqqpk8z8oa1WqpZAYDZN6Y82jMyZYgAVIjMQtM0/QdkZWqGtyLCMkTiC0zMAGaPiHytGXanoWmGMYwgiMXHrgXaMgxZDKvMH5JYYOEdADSEgwiW/ROi2jN6kvcCZ5ldiLi/AdYVkXpozWzbtg2nnXYampqa0NXVhY9//OPYtWuXyId0hfkHbC1HhHNrJmdtYgYw5Yh4VoiUFRELrZlQMKAbwWhyhiAWLzv7K++YmYsshlXmD1nIqAoAiqKYJmfEGFaN4YjqN5+KouiFE5dCxMINt5cIPcXTTz+Nq6++Gs8//zyeeOIJ5PN5XHDBBUin0yIf1jHmH3C1JwZv4xCD7Y2xoogkygWB1+O7VhQRwNjCO0JZIgSxaNk5yAqRyqO7DGkKkbS10V2G6MmZrAUrAMPIEuHXmpHFrGrtVcUhjz322Ky//+QnP0FXVxd27NiB97///SIf2hHsBxxQSmNN82E2DuWKatUJGzvom3dtKCJet2aseEQAoCMRxd7hNBlWCWKRoqoa3hm01prRJ2d8DjXTF94t4A9hiN43o0/NWFAmoqEgUihwUUTy9WxWnZycBAC0t7d7+bCWMY/uVjOLmo1FPJcQWdm8y2jyy6xqoTUDmGPeSREhiMXIwfFpTOeKiIQCC27kXr+kZFYdSmWF5nIshFGIWLuOiQ41yxSstWYAvovvsvVqVlVVFddffz3OOussHH/88RU/JpvNIplMznrzEn3PzAIO5kgwAFan8PSJTOtmVQkVEdutmVKWyBh5RAhiUfJ22R9yVFdiwbTlplgYS5tjAPxtz4ylywvvLLZm2Agvu/7xRjerLvCaY/4YLq2ZejKrmrn66qvx5ptv4v7775/3Y7Zt24aWlhb9raenx6vjAbA+060oCvfsf8A8vmvFIyLWRDWXpI3xXcDowY7UaCGSyRexezAlRQATQcjIrgFr/hCGDD6RCZutmWbBG3gz+pJVK4oIv5j3uhzfveaaa/Dzn/8cv/71r7Fy5cp5P27r1q2YnJzU3w4ePOjF8XSsKiKAKUuE4wgvW3hnxSNiHt/14sUyZWN8F6jt1oyqavjs//8CLvjWb3D747uoGCGICuwaZDtmqvtDGOaEVb9gCq11RUT01Awzq1rxiPBrzTCPSF2YVTVNw7XXXouHHnoITz31FNauXVv146PRKKLR6ivmRWJ1phtgqklefyLxQF94Z2lqpvSjU7VSNLyV4sUNdpJVgZJZFajNdNWHXjmM7b3jAIDv/3ovFCj40gVHLxgyRxD1BBvdXcioylgvgSIybjFVldGsm1VFtWaceET4Tc3Uxfju1VdfjXvuuQf33XcfmpqaMDAwgIGBAczMzIh8WMfYMg6xUDOeHhEbikhDOAg22CPasKqqmp5XwtIGF4KN79Zajkg6W8DfP7YTALBlbclU/b1f78E3n9hNyghBlMnki+gdLcUwWFZEWKiZj5Mzk+UbqhaLLWbWihbXmrFeEEQ5JnrXVWvmzjvvxOTkJM455xwsW7ZMf3vggQdEPqxj7CsivKdmrHtEFEXxLNQsnSuAvQbbV0RqqzXzg6f2YCiVxeqOOP7lC5vx1x89DgBwx5N78C0qRggCAPDO4BRUreQFW9JkTcVmrZmDY9PcwyCtwm7aEhZbzCKnZlTVSPL2empGNrOq8NZMLWFl8y5DiCJiY2oGKDm6k5mCcEWEGVUjwYBlKY8pIslMAbmCKk3lXY0Do9P40W/3AQBu+cixiIaC+MLZa6FpGr76i7fx3Sf3IBoO4upzN/h8UoLwl3eGSm2Zo7sTlluWnYkIWhrCmJzJY99IGscus2Zy5QmbMkxYaH8DYqdmWDEAeN+aYa914ZAc7Wb5Xx08RDer+q6IWCtEvBrhTelhZgsv42O0NIT1PQ21MsL79V++jVxBxVkbOnD+cd36+7/4vnW45SPHAgDuePIdvYVGEPVK30Spvd7TVj0/xIyiKHqeiB8+EU3TjGusxZs9kVMz5ptYq4FmAF+zaiTIJ4zTLVSImDAczD57RCxW6wnBYTuM5Ezp6zdZbMsAQCCgGCO8NdCeeXbvCB57awABBfjKR99zRMH1xfetxar2ODJ5FU/uHPLplAQhB4cnMgCAZa0Ntj5vZblwGUxmuJ9pIbIFVd8GnrCsOoubmmGvN8GAYml6xcgRIY/IosaOImIsvuOniLCId9kUEWNixl4nr1YMq4Wiir/92e8BAJ85fXXFKQBFUXDxpmUAgF+83u/p+QhCNvonS4rIitaYrc/rSPh3TTBfJ61eY0VGvLPXm5jFYkDE1AwVIhJiJ+UuFub3pGDoS+8smFUBQxGZEhxqlsqyDBHriggAdCZYuqrcisjTu4excyCFloYw/vy8o+f9uItPKBUiT+4cQtqjRFuCkBHWmlnWYk8RYTcnYz6M9TMvXTwSRKDKLjEzbGpmKlvQ1RReZPTXG2vXexGBZnUxvltrOEm5E6KIWJUNPVNE7I3uMvS7H8mzRHbsL2WGXPSepVWDjt6zvBlrOuLIFlT8itozRB3TX27NLLfZmtGn6Xy4OWHXSavXV2B2gCPv66yviogeaEZmVenwUxEpGamsL70D4Nn4rt0wM4bhEZG7EHn9UGkZ46aelqofN7s90yf8XAQhI8lMXr/mLLfZmmn3sV2b1idmrBci0VBQb1/wnpyx40kEOHtEJBvfleMUkmBnfJc9eXgpItmCCqb8WR3fTXi0gZdddKzGuzM6ayBLRNM0vH5oAgBw4srWBT/+4hOWAwB+vWvYs4WDBCETTA1pjYdtJzqz1Q9+TNKxGz07hQggbnKGDTpY9WmI2TVDUzPSoUtlNnL/eU3NmD0HDRYrZO/NqvYUkVowq/aOTiOZKSASCliKqj52WRPWdTYiV1Dxq7cHPTghQchF36QzfwgAtDf6t/physYKDTPNgiZnsjbCzAAyq9YNunnIhiLCK0dkujzf3hAO6vkbC9HklSJic/Muw+gHy1uIMDXkuGXNlkboaHqGqHeYUdXuxAxgtGamsgXP01X1VFWbioioyZlM3vqNL8A5WZUKEXmxFWim9+s4KSLMH2KjWk9Ey5KhaEXE5uZdRkcNbOB97WDJH3Liyur+EDOsEHlq97CwrZwEISusNeNEEWmOhXSDpNftmbQDsypg3jfD2yNifTgC4LxrRveIkFlVOrJ2FhCxqRlOZlW2eddOz5UFn4keJXXdmpHYrMoUkU0W/CGMY7qbsH4Ja8/Q9AxRXzBFxO7EDFBSFJkq4nUh4mRqBhC3b8ZozXg/NaMnq5IiIh8ZGyuZYxwdzIB58651RSRePueMYImT/QLaV0RKrZmZfFHKWPRCUcWbfWVFZIGJGTOl9kzJtPpzas8QdQbziNidmGF0lH0iXicusxu2JruFSFTMvhk7rzeAoNYMRbzLhwyKiJ1qvaFctMzkxBYiSX3XjD1FpDES1L+XMqoi7wxNIZNXkYiGsK4zYetzP1puz/xm97D+/SGIeqDPYYYIo8OnyRmj/W23NSMmJsHO603p40RMzchRAshxCknI2ki6k0IRiXijiBiBZvYKEUVR9BFeGffNsLbM8SuaLSctMo7ubsJRXQnkijQ9Q9QPqqphYJJ5RJwqIv4UIkzZtd+aEeMRydpVRDhmV2WpEJEXO0l3rFjhpojY3LwLGE9gkYpIJl/UjU12WzOAf3c/VnjtEDOqtjr6/PPKG3qf3TPK60gEITUj6SxyRRWKAnQ3OytE2vXWjD9m1YTN8V19amaG89SM0/FdjmZVSlaVEDvZ/0aOCCdFJGtv8y5g5I2Yt0ryhrUdFAVI2AwvAkxJihK2ZpwYVc1sWdsOAHhh3xinExGE3LCJme6mmKVx90oYNydee0Tst78Bw6TPuwVrTM1435phZlXaNSMhdnp2Ro6If4qIecJG1Ey+blSNhmy3LwCTMU2yxXeZfBE7+1MA7BlVzZy6ph3BgIIDY9P6JAFBLGb0ZXcOjaqAf9N0Uw4i3gGBUzN2I95FBJqRWVU+bOWIcF5650QRMZ9zWlB7hjnF7W7eZXRKuvju7f4kCqqGjsYIVjg03SWiIRy/vBkA8MI+as8Qi5++SXdGVcC/fTNOI96bRCkiNl5vALNHRIWmuVPAyawqMXYq1Cj3QDP7ikggoOjtGVGKSNJhqipD1lAzfdHdyhYoivM+6ZZ1HQCAF96l9gyx+NEzRBwaVQH/NvBOOTSr6lMzgpJVrVgBAKM1o2lAvui8EFFVDYVyK58KEclQVU038NhpzWS4Rbzbn5oBjBFeUYpIymGqKoO1ZmSLeX/NpT+EQT4Rop7on3QeZsbQp2ZqpDWj7/QSFWhm2SNifJyb9gx7nQPIrCod5h+OHbNqrqBC5WAUdWqkahAcaqaP7jpszfiVorgQTBFx6g9hnLqmHYoC7BtJYyiZ4XE0gpCWwy7i3RlMJU3nip7tmykUVf2F3/Y1NmJMSLptiZhxGmgGuDOsml/rSBGRDPMvhJUK1fzkMf9gneJeERGTXGqEmTlTRFripQJmknMqoRtSmTz2Dk8BcK+ItDSEcdyykk/keVJFiEVOv77wznkhkoiGEClP3HillLIbPcD+9l12s6dp/JacAuYlq9ZehhVF0QsHV4WI6XMjDiefeCPHKSSA/WCDAQUhCz8cc7HCo6p34hEBINwjwlozThWR1rK3ZHJankLkjcOT0LTSxZQFrrlhy1rmEyHDKrF4yRVUDJe9Xm6mZmbtm/GoPTNVvlGLBAOWl8wxGkw3nTwzm7I2c0QAc5aIi9aMPjETcOWP4wkVImWyNqvTUDCAYHmclUeV7GRqBjDHvPOr1M0YrRlnikhrvHTBSWUL+uy635iNqjzYso58IsTiZzCZgaaV5Hzm83AKa894NdZvGFXtj6uGggFdOZjmeMNnN1kV4JMlwgoRWfwhABUiOnqqqo0nRUwPNXP/5Jx2qYiIb804U0TMBQzvpVFOcRtkNpfNa0qFyJ6hKSmj7AmCB4dNEzNu76Q9V0Qcbt5liNjrZTfQzPyxbgoR2TbvAlSI6Njt1wGmmHcOWSLGQiZ7FTvzlAgPNHOoiISCAX3bpSw+kbf6kgD4KSJtjRFsXNoEAHiRVBFikcJjYobR6fEIb9rhxAyjQcA6DUetGQ6xEbLtmQGoENGxE2bGiHFMupsum6kaHCsiYgPNnHpEAMOwOiFBIVIoqjg0Xrqgbuiyt3G3GvoYL/lEiEVKH4eJGYbXoWZuCxERC0aNqRk7igiH1gwpIvJixLvbaM1wUkSKpgwTqzPljAbBG3gNRcR5IdIal8ew2j+ZQVHVEAkFsISDUZWhB5uRIkIsUvr0iRnnRlWGvm+mRlozMQEt8IyD1xwerRmzWVUW5DmJzzipTiOcPCLmcSo7Mh0gRjI043Z8FwBaG0oXnYkZ/7NEDoxNAwB62hoc7c6Zj81lRWTnQArjkmWmEAQPjD0z7hWRDo8VEadhZowGzi3wQtFYVGpPEXGvwhtmVXle/uU5ic+4UUTcTs2Yn9x2tyGKVkS4tmYkUERYIbKqPc7163Ymonqr58VeUkWIxUc/hz0zjHaPE5fTWedTMwD/1ow5kduWIsJec1yo8LJt3gWoENHRPSI2qlNWybqtktnyo5DFDBMzItzcjEJR1fNNnJpVgVLoFyCHWVVUIQKYfSJUiBCLj8Mc9swwvN5BNeUwuZoR4+zFc3rzybU1Q4WIfBg5It7OdJsf225bBjCZVQUoIkzOBFx6RBrkUUQOstaMiEJE94mQYZVYXKQyed0vxrM149XqB6aINLk1q3IqRMyTK3ZaxFxaM2RWlRdjE6J/iogTqUwf3xWgiLAws4Zw0NWTtlWimHeRhcjpZUXk9/1J7ivDCZ/Zvh1YtgxYssR4W7as9P46gLVlmmMhxz4LM2wD73SuKMzfZibtNkeEsxfPSYZI6ePd3/xmyawqL4ZHxP4oletCxIUiwlsyNJN0uXmXoZtVp/03cYpszXQ1x9DT3gBNA145MMH96xMes3070NEBKAqweTOmBwYQHRnBBSMjiI6MYHpgALjkEr9P6QnMqMrDHwIAjRHj5saLLJEUr0AzXh4RB6mqgDlHhEeyqjwv//KcxGecmVXd9+sAI5zGjhrDiJdzR0SYVd2mqjJkyRFJZfIYL7eHRCgiAHDq6pIqsoMMq7XLvn3A5s3Ali2YHhtDFEAgFEX7yvcgsPpE/Hb1iQisOQkda96LoRlvtsf6DcsQ4VWIKIqCTjY548EIL69AM143fEaYmV1FxH1rRsZkVfca2yLByfiuPjXjujVjvwhiiBzfdZuqypDFrHpwrHRX194Y4SIvV+KU1W146JXDeGn/uJCvTwhkaAi47jrg4YeRy2SwHkBfQzOaTv4ouk6+GMH4kUm87w8quP7pvfjC2WulusPkjZGq6t6oymhPRNA3mfHEJ8KrEOE1vmu0ZmwqIhx3zVAhIiFOFBEeDmbAvPzI/hND5Pguj9FdQJ5AswMC/SGM08p7Z149OIFCUbU9BUX4xL59wLnnorB/P84B8EJLN5pO+zhWbDofgXDpxbeQGoWaSZX2wQNQwlGgbTlue3QnHnm1D7d98gSc2NPq239BJGxihkeqKsPLEV5uu2Y4XWeNAQXvFZGcAxuCaKgQKeMo4p1TlcwUkZgbRURIa6a8eddla8YINMtD0zTfVk8fNIWZieKorgSaYyEkMwW83Z/CCZz22RCCSKeBH/8Y+OpXkRscRBOA6MkfxfIPfgFKsPS8z/a/g+QLD2J693OANvum48GXDuKrv/g93u5P4uM/eAafP2MNbv7wRkd+L5npL7dmVnBqzQAwtWbEe0TS5fFdt4Fm/Foz5ZtPu4oID49IUT6zKhUiZZyM0Eb1ZFU+gWZOPCIic0RSvMyqZUWkqGqYyhZcjQK74eC4OKMqIxBQcPLqNjy1axgv7R+jQkRm0mngllugfuc7+AqAb0TiaPnIdWg85iwAwEzva0g+9wAyB16v+Omjo6Nob2/HOccswdd+8Tb+85XD+MmzvZiYzuHbl73Xw/+IePommSLCsTXj4QjvFKdAM36tmbIqYVsR4deakamVKM9JfCbjQK4yklXdPTmzLhQRc+KfVpaMecHGd922ZmLhoP599TNLROTEjJlTV7cBAPlEZCadBm68Eep3voMPAPiH7vVYcsV30HjMWdCKeYz9910YeuAWvQhpBDCzdCm0aBRaQwO0chEClEZRv3npSfj/PncqAgrw8Kt9+Pnrff793zijaZo+vsuzNcNGeEcEm1U1TdO3m8tiVtU9ibY9IpQjsqhxMrkS5bT0zo1HhBVD5sV5vOCxZ4Yhg2HVq0LkFH1yZpx7cUhwgCkhP/gBPgDg1U3nY+lnbke4bRkKk4MYuPcmpHb8DADQBCD7sY9hamYGsf5+IJMBpqeBchFi5rzjunHNuRsAALc89CYGyi/etc74dF6/i+5u4bco0gg1E9uamc4Vma0HCYfqLtuKzjvQzPb4LiWrLm6cqBJ6a8alIuLUQQ0YiggAZHJ8CxGjNeO+leJ3qJmqajhUnpoRaVYFgJN6WhEKKBhIZnSTHyEJpnbM+wG8ceZl6PjwdVBCYUzvfg79P/7fyPXvRhjAl1etwsTMDCKPPALErLUkrv3QUThhRQsmZ/L4ywdfg6rWfiHKJmY6ExFH16j50GPeBbdm2MRMQDGUDbvw9uI5DjTjsGuGChGJcZas6v5JATifKQdKfb5QOSJ4Os9vRTVgbs24V0SMUDN/CpGhVBa5oopgQOHa565EQySI9yxvBgDsoPaMPAwNAZ/8ZKkIUQJ4+/w/Rev7PgMAmHj2fgw/9DWo2TR6gkFMP/YYvrZvHwIWCxBGOBjAty49CdFQAL99ZwT/8lyvgP+ItzBlZynn35t2j3JEdH9IJOTYKM874t3wiPjYmiGPiHw4CjTjrIg4ddqLMqzqrRkOiogRauZPuipry6xobfBkpJa1Z17qpUJECoaGgI9+FIXHH8eaYAi7PnYjmk7+KDRNxdgTP8Tkb+8BAPQ0NKB3ehqhCy8EAs6eJxu6EvjyR44FAGx7dCfeGUxx+2/4AfOHLG3mO23WUR7fFW1WdTu6CxjXZm7juwVn7XhqzSxyHEW8c1JE9OrY4RND1AhvSh/f5aGI+Lv4zsgQETe6a+bUNWRYlQZWhGzfjo5IA3KX/A0aN54NrZDHyH99A6mXfw4A6IlG8e7EBAKRiOuH/NwZq/H+o5cgW1Dx5//2ak23aAZ0oypfRYS1ZmbyRUzn+Kq5ZtxOzAD8b/YyDpasmj/eTSGSJ0VEXhzliHBSRPTHXsyKSLkQSfrkETnokVGVwSZndg0kda8N4QOmIqQzlkD80q8itvpEqNlpDP37rZje+TsAwNmdnehNJhHiUIQApQjzf/ijTWiKhvDm4SSe2j3E5ev6Qb+g1kw8YkzTiWzP6BkiLq5j8fK1OVdUUeAwFOBYEQm7b81kSRGRF6Zq2Pnh8FZEHLdmBCgimqaZIt75mVX9UkREbt2tBFuAp9ICPP8wFyENzYhf9nVElx+D4kwSgz/dqo/mnr1iBZ7u7+eihJjpbo7hss09AIAfP9PL9Wt7yUCSf4YIUN43kxCfrmrEu7tXRAA+11mn13y9NUNm1cWJk0VArJp1v33XmYOawTv1j32tYllO5jK+G2fpqv56RHravClEAGMBHrVnfMBchCTaEf/0bYh0r0NxahyD921FbnAvGgHccvrpeLq3F4GQmGzHz56+BooC/PadEewdnhLyGKIRZVYFzKFm4kZ4zWZVp0RDATCfK5dCxIECX/p4DoFmRQo0kxb2w7HlESk/KXJud804nCln8E79A4y2TCigOB55MyOLR8Sr1gxQWoAHADv20yZeTzEVIUual6Dx07ch0rkKheQwBu67CfmR/WgOBDDx+OP46jPPCCtCAGBVRxwf2tgFAPiXZ3uFPY4oRIWZMZhPRGSo2ZTLhXdASb3huWDUSZI3sHi378pzEp/R5aqg90vvnGz+NcM79Q+YvXmXx24YP3NEMvkihlKlOy4vCxFmWH3lwASXvjJhAXMR0rYcjZf/PcJty5GfGMDgfTejMN6H5kAAo+k0Quef73gyxg6fP3MNAODBHYdqzi+Uyhb068rSZpGKiPjWjJupGWB2irVb3HtEVMdhifrSO1JE5MNJ34wVIrmi6soVn3EwOmyGd+ofYNq863LhHcPPZNVD5R0zTdGQXhB5wdFdTWiKhTCdK2LnQG2PcNYEpiKka8kaJD799wg1dyE/ehCD992EwuSgUYTYzAdxw9kbOrF+SSPSuSL+Y8chzx6XB6wt0xoPz/JJ8KLDg0JEV0Rc5iHFON7wOQ2xZB+vaUC+6K4QIUVEMlRVQ6FcSISD1u/+zVMubuLV3US8A0BD+fN4mlWTnBbeMfwMNGNtmZXtcU83/wYCCk5eVR7j7aX2jFDMRciyo9H4x9sQTLQhN7gXA/fdjGJq1JciBCjJ+leUVZF/eW5/TY3yGhkiYr5nxr4ZcR6RNIfWDGAozxkerRmHIZZm64DT9gwVIpJiLiKcKCKAOxezW48Iz94lQ88Q4bQplwWazeSLXL0sVjhYjnZf5VGGiJnNa0uG1RepEBFDOg1873vAli0obN+O7p4TkLj0qwg2NCFz+G0M/PTLUKcnfStCGJ88eSWaoiG8O5LGb94Z9uUMThgQsHXXjDetmdL1ptGlosOzNeN26R3g3BJAZlVJcVqIhAIKyunqrsxD7qdmyq0ZnorIDL8MEaDUFmHfK6+zRPwwqjJOX1cqRJ5/d4wW4PGG7Y259lr8VW8vlmzYjMZL/g8C0Thmel/F0AN/DS2b9r0IAUr+hD86dSUA4O4aMq0aGSJiivjOhPiY9xQnjwjf1gyLeLd3zVcURX+NclyIkCIiJ+apFztpc4qicBmn4qWI8DSrJk1mVR4EAoruE5nwqRDxKkPEzAkrWtEQDmIsncM7Q7U5viklpuV1HwDwnRMvRPMnbkEgHMX0O89j6MG/gZbPSFGEMD53xhoAwFO7h9E7kvb3MBYRlarKaPcg5p1Xa0aEWdWJL9DIEnHYmqFkVTkxJmYCtj0EPJLunMp0DJHjuzzCzBh+GVa9DjMzEwkF9OmZ5/aOev74ixLz8joAb5z9aXRcdC2UQBCp1/4vhh/6OlDMS1WEAMDazkacc8wSaBpw7wv7/T6OJUSlqjKYWXVUYI5ImpNZlWeCtZsQS7c3vzkH60xEI89JfIT9YOwYVRnsh5lx6BHRNM3R5l8zMT3QjN++Bl6/vGb0UDMPDauapnke7z6X09d1AACef5cKEdfs2wecfnppeZ0SwNsXXYvWsz4NAJh45j6MPXYHoKnSFSGMy05bBQB49M2BmmjViVZEmHcsk1eFecd4LL0DgIYwvxa4m3a8m9iIQlEF80pTa0Yy3AS8uK1OC6qmPzEcKyJ6xDu/rAp9P4OLWOS5GKFm3qWrjqVzSJfvYFa0em9WBQyfyAv7xmpqYkI6hoaAiy5Cbt8+JCINyHzyr9B04oXQ1CJGH/seJn93HwCgJxyWsggBgPcf3YloKIBD4zPYVQNbefsFm1WboiEEy+YxUd4xblMzkdLrg9sWuKZprtrxugrvoCAy+yHJrCoZbpYAuU26M98FOFVEDMmQnyLCpmbc3kWY8SPUjPlDljbHHHtw3LJpJflEXGGejNm9G60t3Wj/zD8gvmEz1HwWww99HVOvPQagvLxuakrKIgQA4pEQzt7QCQB44q1Bn09TnXS2oHvFugWN7yqKeO8YP0WETwvcfNPqJLLBzc1vvmDcCJEiIhk5N4pI2J2D2dzScbtrhufUDK+7CDN+xLz7OTHDCAfNPpER385Rk6TTwI036pMxnSvfg47PfRORJWtQSI1i8KdbMbPnBQDiltfx5vzjugEAT7wtdyEykCy1ZRLREFev2FxEXheyhaIe/JVwsWsG4BccmZ11zXdhVnXwmpMtls6uKKWpT1mgQgRmj4iL6tRhWyRrWn7kNGxLRI5IOse/EPHDrHp4oiQtr2zzpy3DMHwilCdiGWZK/cEPSpMxmy5Ay2VfQzDegmz/Oxj4lxuQ69+NMMQvr+PJB4/tgqIArx+a1D0YMiJy2Z2ZZoHXBdZiBoBGl21mXtOJ7JofUNz5Ep2o8G4GM0Qi/2+tB+RdjDO5b824G90FxBQivORMM8YGXu8KEd1s1+qvVM8KkRf2jUJVNQQkuhuRkn37gA99CIV9+7AuGMLUuV9ExykfBQCk3/4NRn/5HWiFLCIAUo89hohHe2N40NUUw0k9rXjlwAT+++1BfOb01X4fqSL9go2qDNayFeEdY8puLBxAyKUngtd0ovma76QYYIneTm5+ZcwQAUgRAeBunMnt4ju3YWYA3/l2htjWjHdmVdER1VbZtLIF8UgQ49N57B6S36ToG0NDwGc/C5x4InL79qGpqRP5T9+G5nIRMvHbezDyX9+AVsiiWVGQTqcRufDCmilCGHp75vfytmdYqqro351WgYoIj827DEMRcefFyxTcXfPdvObImCECUCECwF2V6HZqxm2YmflzeQaaTS0Ss+pgUmwypFVKPpFyyirliVRmaAj48IdRuOcevC+VQtuak9B5xXcQXb4RxZkUhv79/2Dy2fsBoDSeOz2NUNw/748bLigXIs/tHdVfLGXDO0VE3Fg/T2WXlxcv61IFd6PCM7MqKSISwsWs6vDJmeWoiGQL7rYAM1RV00de3fZVzRgSrHeFiFcXUyuwMd7nKE9kNuk08I//CLznPSi8/DI6lADeOPMydH7qb0t+kIE9GLj7esy8+xIAucdzrbJ+SQJrOuLIFVX8Zrecu2e8KuKNqRn+SqleiLg0qgL8WuBMEXFeiDi/+c2VzapUiEhI1pVZ1WVrxuWTEsCs9dw82jPTpq/RFK3dZNVcQdW3eoo23FnB8IlQnogOm4r5i7/ALSMjaEy0I/apv0Pr+z4DRQkg9eqjGLjnL1GYHCyZUo87TurxXKsoiiJ9e8arIt64LvBXhngGM/Jqgbttxxs3vw6mZgp12pr5/ve/jzVr1iAWi2HLli148cUXRT+kbdyZVV22ZnSZzvmPwhyExqMQYb+8AcXduebS0lCSYJOZPIoevBAPpTLQtNLPtT3u/0jnCStKPpGJ6XxNhFkJp5ySyqZivrnuFHRdeQca1pwINZfByC++ibH/+32gmEcEwNRjj+Grb7wh/XiuVc4/bikA4MmdQ/o1SCa8mprxwqzKwyMS49yaifrQmqlLs+oDDzyAG264AbfeeitefvllnHjiibjwwgsxNDQk8mFt484j4nJqxsXyI0YgoOgFA4/JGXNfleeIF7vz0TQglRGvirALaXdLVIoplVk+kXpuz5gNqW++ibZACG+e+z/RfcnfIBhvQW7wXfTffR3Sbz4JAFgZDNasKbUap6xuQ1s8jMmZPF7qHff7OLPI5IsYLS+i82pqRoxZlbWYJWzNODarumjNuFD/RSL0NN/85jdx1VVX4corr8Rxxx2HH/7wh4jH4/jnf/5nkQ9rG1eFiAuZDDCPcrn7Uei/JBwUEWZU5TkxA5S+v43luwovfCK6tNzsr1HVzBnl9kzdLsCbY0htbluO+Gf+AS2bPwkASL70X+j/1y+hMHa41Io59ljsr2FTajWCAQUf3Chne2YoWWppxsIB/QZCFEwpFWJW1a9l7r1ucU5L7zI+KiIs3K1uFJFcLocdO3bgvPPOMx4sEMB5552H5557ruLnZLNZJJPJWW9e4G58121rhi28c/eLEueU+geIGd1ltHqYJeKVtGyHut07w2LaTzutZEgF8MoJ56Hriu8guuyo0lTMf/wdxn/1T7NbMW++uWhaMZUwUlblWoJn7JhpEB58JbQ1k+NvVp3OF139rLJuFREXN7/MrCrT5l1AYCEyMjKCYrGI7u7uWe/v7u7GwMBAxc/Ztm0bWlpa9Leenh5Rx5sFm5pxZ1Z12prhs5KZKSo8RnhFhJkxRKYozkWmiRnG8StakIiGMDmTx2uHJvw+jjeYY9oPHEBrtBGRj92Izo9cj0CkAZn9r6P/x9foUe09i7QVUwm2BO/g2Az2SLSHiMW7e5G/wxSXVLbA3Ts2xdGsyoYCNM35jSfgPsSSR2um7syqdti6dSsmJyf1t4MHD3ryuG5CXngFmrldyMYUER6rtEXEuzO8DDUbSJYDmSQqRMLBAD5wzBIAwH9LvmuEC0NDwMc+phtSb19xLFqvvAONx74fWrGA8ad+gsEH/grF1CjCAL587LHoXaStmErEIyGctka+sW4vi3iR3jGe6m6D6RrtRnl2PTXDwaxaNx6Rzs5OBINBDA7OvtgODg5i6dKlFT8nGo2iubl51psXuPOIuN01U66OXZhVAX57EACzwYv/tlovQ81kVEQA4Pxj5fQFcKc8FVN48kmsUQJ4/fRL0P3p2xBq6UJ+vA8D996I5AsPApqKlcEgph97DF9b5K2YSrB2nUwGZi/bmuFgQC8UePtE0hzV3VAwoN+suvHiuQ2xdJesWmcekUgkglNOOQW/+tWv9Pepqopf/epXOOOMM0Q9rCN8nZrRPSIuzaocY95FpKoyvAw1My6m8phVAeDcY7oQDCjYPTiF/aNpv4/Dn7kx7Yl2ZD/1d2j7wOehBIKYeuvX6P/Jdcj17wYAnN3RUTKk1kErphLmhYiy+EQMj4g3RbwRasb3upDifC3TpxPdFCKuc0Ro14wtbrjhBvzoRz/C3Xffjbfffht/9md/hnQ6jSuvvFLkw9rGz4h3vV/ISRGZcbkHATDuIpoEFCIiHfJmCkUVQ6mS8182RaQlHsaWtaW74EWnisyNaV97Mjqv+O6sbJDRn/8jtNwM4ihvzR0YqDsVxMymla1oCAcxls5h96AcPhGvi3hRhlWjzcxH3W3gMDnjth2/GHNEhG7fvfTSSzE8PIyvfOUrGBgYwEknnYTHHnvsCAOr3/jpEdEd1DIpIgLNql6lq45M5VBUNQQDCjoTUaGP5YTzju3Gs3tH8cTvB/HF963z+zh8MG3M7VACUM66DJ1nXgZFCSA3+C6G/+vvURg7DKC8K+axxxD60IfqUgUxEwkFcOqaNvz2nRE8/+4ojlna5PeRPG9riroupMtt5gSnhOiSFy/LqTXjx9K7csR7vXhEGNdccw3279+PbDaLF154AVu2bBH9kLbJ+7prhs/UjFGpu09o5NlXnYvhERFrVmXScndTFEEJwszmwsY2t/eOYTzt3TZiIbDR3DPOQG7fPiQamhG75P+g9axPl2LaX/mlng0SBHBWY2NpV8z559d9EcIw2jP++0TyRRXDHq9GEOUdM26q+CgiPBaMGmZVH6dmJFNE5DqNT7gZaWJPipxrRYSTWTXPoTXjydSMWEVExgwRMz3tcWxc2gRVK0V81yxDQ8AnP1kazR0cROuyo9F5xbfRsPZkqPkMRn7+jxh7/Ad6Nsj03/89fjcxUfO7YngjU77MUCrr+WoEUS1b3plIPELN3IZYss9zFWgmmSIitDVTK2S5mFXdJqu6Hd8tfX6Gw9QMb4OXmZa4GFPaXFgOwjLJjKpmLjiuGzsHUnji94P4w1NW+n0c+zA/yMsvYx2AsRPOw5ILr4YSDCM/dhjDD30d+ZH9AEox7fuSyboZy7XLCSsMn8g7Q1O+tmcGmJro4WoEESb2oqrpygWva5mRYO38ho8VEM6TVZ2bVd281olErtP4BJeId7dTM64DzfiN7wpNVvXIrCq7IgIA55XbM795Z5hL/ounDA0BH/1oKSVVCWDq3C+g8yPXQwmGMb3rWfTf/efIj+xf9DHtvGA+EQB4bu+Ir2fxYzWCrpRybNmmTcZ9XtcyHi3wjMt2vPGaQ62ZRYW7ZFVOOSKcFBE+23eZwUucIpKcyQsdVZQ1Q8TMCStasLQ5hulcsXZ2zzA/yJYtKGzfjo5IHLE//AqaN38CADDxu/sw/PA2aLnpuolp54V5jNdPjGWR3v3uMLNqkqNSym6oQgGFW6S5kdfkXBHJuGzHu5qacTGYIRK5TuMTrsyqnJJVXeeIcNoMCfA3eJlhdz65osqlaJoP/WLqQUS1UxRFwXnHdQEAHq+FMd50GrjllpIfpLcXba1L0fjZ29Gw/lSo+QyGH7kNk8/cB0Crq5h2XrBC5IV9o776RA6OTQMAVrZ5qIgIaM2kBWwRZ9dZNwpmlmPEu92buVy5eAmTIiIf+tI7F+O7uaLq6OLBqmOnDmoGz/FdkWbVeCSIcLB0URDZnulPehvI5JTzjyulDP/324O+mxSrwvbFfOc7+ACAf1h2NFo/+4+IdK5CITWKwXtvwvTO3wEAzu7srKuYdl5sWtmChnAQ49N57B5K+XaOg+Ol351V7d79/HSzKkdFZEqAssvjOmtc8921ZjTNMJ9ahX28k9c6kch1Gp/gEfEOGLKXHbIuHdQMw0TlrhDRNM1Ync1hUdRcFEURHmqmqhoGJ70dP3TK6evakYiGMJzK4vXDk34fpzJMCSnvi9mx7lR0X/Z1BOMtyPa/g4F/+XPkBveiEeWAsv5+asU4IBw0fCLP+9iqO1BWRHravCtERCgiRkI0P2WXFSJuvHjuFRHjtcJue4Y8IhLjZhHQrCeFA58Ir6V3PBL/gJLcVyjfmYuYmgFMFx1BWSJj0znkiioUBehqkrsQiYaC+MDRpSV4j79VeSu1r7AipKyEvLrpfCz5w79GIBLDzLs7MPjTrShOjaE5EMDE44/jq888g0CIhvGcwtozfi3A0zRNb814qYiY84V4ecdEBDPGebRmXIZYmv0ddi0BVIhIjJtFQKGAAjbh5sQ8lCm4c1AzeJlVWV8VABojYl5Q9BRFQYoI84d0JqLS/cJV4sMnlNozD+445DiPRgimIuT9AN448zJ0fPi60r6YN/4bQ//xt9DyGfSEwxRQxokz1jOfiD95IsOpLLIFFcGAgmWt3ptV80WNm3dMxPQfD0XEmJpxdvOpKIpjb2KWzKrywgw8Tl60Sk8KZ0l3mqbpLzxuFRFe47tsYqYhHBSWSNreWJLtRwUlitbCxIyZC45biiVNUQylsnj0zX6/j1NiVhGi4Pfn/Qla3/cZAMDksw9g9JffBtQiehoa0Ds1RQFlnDhhRQvikSAmpvPYNei9T4S1ZZa1xDxdFd8QDuovjrzaM6lM6es0x/jEuwN8lGceaz30QsRm0aar/5LdoMl1Gp9wO9LkNEvEXLi4H98tVf1uA81E7plhdJQLkTFBhQgLZFoq8cSMmUgogMu3rAIA3P1sr7+HAWYXIUoAOz9yHZpP+R/QNBWjj9+Jid/+KwCgJxrFuxMT5AfhSMknUkpZ9SPu/YAPbRmg7B3j7BNJlj0izQ0cFRGXXryiqumGUTeLTvUNvDZvfvOkiMhLzmV7hH1exqZHxNxndL1rRo94L7rqsbJCpEmAUZXRkSgrIuV9FrypNUUEAD69ZRXCQQUvH5jAG4d8NK2ai5BgCLv+4CYkTjgPmlrE6M+/ialXfgGgPBmTTCJERQh3WNx7PRUiAP9QM5ZJ0sRREXEb8T7rms9DESGPyOKgUFTBWrFOpUinrRn28cGA4loGZZKhueJ2QlpghgijvbG0DVdUa8brFeY86GqK4SMnLAMA/MQvVYTtjfnOd/C+UBTvfPKv0XjMWdAKeQw/vA3p3z8FADh7xQqajBHIlrWlQuSl3nGhoX+VODhWUhN7/ChE4ny9Y2xVRTPHm6qYS0VklgruRhFx2ZrhFfDGC7lO4wPmF22nVaLTpDt9YobDk6LB1NpxY/bSWzOCjKoA0KkrIuQRMXPFmWsAAD97rQ8jgtSieWGR7Y8/jjWRBuz91N+gYd0pUHMZDP3H32DmnecBlIuQ3l6ajBHI8StaEAkFMJrOoXd02tPHZhMzfhQiuomdU5ZIknlEGjh6RFwGR7JrfiQYcLXHx+nNb85FeKdI5DqND5inFBwXIg6z/3X3tEt/CACEg4puLnVjpBK5Z4bRLtojkpR/z0wl3ruqDSeubEGuqOL+Fw9498CsCNm+HZ2xBPKXfhWxnuOhZtMY+re/Rqb3VQBUhHhFNBTEphUtAICXer2Ne/ezNcM71IwVIjzbzMyL5/Rmj9duMaevOW6iKkQi12l8IFssPTEUpTSK6wSn+2Z09zSH6lRRFH3GnYsi4kEhIqI1o2ka+idrI1W1Ep8vqyL3PH9AN5YJxVyENDQjftnXEV1+DIozSQz+9MvIHn4bABUhXnNKOdhsx/5xzx4zky9iMFUq4ns8jHdn8A41M1ozPKdmStdqp9OJrHBwe/PpVIUnRURSdPNOMOB4H4Hz1gyf0V1GTJ9xd76QiRUiIlJVGZ2JkkdkfDrHPSshOVPQv68y75mZj4s3LUNnIoKBZAaPvyV4/4y5CEm0I/7p2xDpXofi1DgG79uK3OBeAFSE+MGpq8s+EQ8LkcMTM9A0oDES1G8WvKS1wQg14wEzq3JtzXBSRNwmabObXzsDEua4CJqakQwePxinDma9X8ipOmWObjepf160ZtripYtcUdW49YMZbMdMe2OEW4HnJdFQEH+82YNRXlMRsqR5CRo/fVtpb0xyGAP33YT8yH4jsp2KEM85ZXVJEdkzNIVxQS3MuRww+UN4LYmzgyhFhGdrhnlEcgUVRSe7xfJ8zKJObn55+CFFIddpfCDvIlWV4XZqhtcLZgOHUDO2KEqkWTUSCuhO9tE0X1MmM6rWSoZIJS7fshqhgIIXe8fwzJ4R/g9gLkI6V6PxM7cj3LYc+YkBDNx7EwrjfRTZ7jPtjRGsW9IIwLv2zCEf/SGAoVzwuDnRNM0wqwoY3wWcqSJGmJnL1kzYviJi3oVGiohk8Jir1o1DNp+YvGQ6Bo/UPy/GdwGgo9ye4T05M1CjEzNmlrbEcNnmHgDAX/z7a/xUo3Qa+N73gC1bUNi+HV0rjkPi8r9HqKkDueH9GLz3JhSTQ2gOBCiyXQJOLasiXrVnDvg4MQMArXF+yzAzeVW/yeTZmjErGU6us7za8TEHikiWY24Vb+Q6jQ/kiu7bI25bM053DsyFxwbetAeBZoCRrsrbsKorIjVciADA1g8fizUdcfRPZnDrI2+6/4IsqOzaa/FXvb1YsmEzEpf+HYKxBDKH3sLgvTeiODVqFCEU2e47zCeyY783kzN+TswAZo+I+0KExbsHlJLnhReKorga4WWFg9tCIOZAEcmabAhuRodFUPeFSJaLR8Rta4aTIuJyxh0AUh5MzQDiJmcGanhixkxjNIRvXnoSAgrw8Kt9+Nlrfc6/2L59wOmn6xt0v7PpfDR/4hYEwlFMv/MChh74CtRsmooQyWCTM68dmnS0UNMuB8phZr4VIrpHxP01wRjdDXP3u7hZMJrlpIg48YjoN76cXm94It+JPIbHXLXrQDNeHhEOG3jTHhUiLOZ9jHNrpr8GU1Xn4+RVbbjm3A0AgFseekNvO1mGtWLOOAO5N99EWzCM319wtbFB9/XHMfzQ16AVslSESMi6zka0N0aQK6h483BS6GNpmqZ7RHra/fndaS3niKRzRdej65Mz4pRdY8Go/enEDIeFd+Yz2ImM0EeHOSnwPKn7QoSLWVX3iDhTRHj163iYVb2YmgGADj3mna9ZdWARmFXNXPuho7BpZQuSmQL+8sHXrI8779sHnHZaqRUzOIjmtuVo/Ow/oum9H4amqZj43X0YffS7gKZSESIpiqLg5FUsT0Rse2ZiOq+roSvb/FFEmmIhMPHCbXtGxOZdhhtFhFc73thvZsMjImm8O0CFCB+zqtPWDGdFhMf4rhdTM4DI1szi8IgwwsEAvnXpSYiFA/jtOyP45hO7q48N7tsHbN4MHHUUcm+/jTYA39p4Nro+/+1SRsj0JIb+7VZMPnMfAKAnHKYiRGJOLbdnXuoVa1hl/pDu5qhvY++BgKIXDm4NqyI27zLcDAUYrRlOioiN15ws5+EInsh3Io9hZlU3VaLj1gzn8V0j0KwGzKoCNvAmM8ZdXa17RMysX5LALR85FgDwvV/vwSd+8Aze6itv6O3uLsUCs7d165Dbvh09xSKalh+D6CduwZI/uBmBaByZg2+i/8f/G5neVwCUN+hOTVERIjGnmRJWRS7AOzhebsv4pIYw9MV3LkPNUhn+m3cZboYCMgU+iggrJuzcdGYkbs3UfUCADIFmvKSyeNhd6l+hqOqfK9wjUm7N8Nw30z9RUkNaGsLCz+81nzl9NULBAL7+y7fx+qFJfOx7z+CL71uL68cm8e8APq9/pIKG9aeiecsfYlnP8QAATVORfP5BTPz2HkBTEQbwl1u24O9+9zvKCJGcuQvw1nY2CnkcvydmGK0NYewHB0Vkhn+8O6PBxQ2fsV+MT7KqE0VERrNq3V+FuJhVHRiHzB/Pz6xa+j84nZpJmz5PfI4I/w28feWJmeWttW9UnYuiKPjjzavwoY1d+Juf/R6/eKMfdz39Lh644acYnhjFckWBoihQQhEEG0t30Voxj/RbTyH54n8iP3oQANCsKBh99FHKCKkR2AK8l/aP46XeMWGFiJ9bd820cMoSMTbvCmjNhJ23wI39Ym4DzRaXR4QKES7Jqk5bM3wVEbfju6wtEw4qwuU7liPC9s3wmGvvmygXIouoLTOXruYYvn/5yfjE7wfxlUfeRN8kEG5bNutj1Ow0Uq8+itRL/4Xi1Kj+/jPjcTw9OkqtmBrjlDVteGn/OHbsH8clp/YIeYyD5dFd3wsRTlkiQlszLjwi3ALNHBRDvKc0eUKFiI9mVWOum5ci4q4149XoLgC0lQsRVSut/eaxZIu1ZhajIjKX847rxtlHdeLt/iROP+NMABqgld7yY4eg5Wb0jw0DSH7ta4jdeCNArZia49TV7bgL7wpNWJWpNQOUrgluMFoz4hQRZ60ZPjefTuwAMisi8p3IY/gUIs48InqgmSSKSMqj0V2g1Apjdz+8DKtMEVnWWh93/LFwEO9d1YYfbduKXN8u5Pp3Izfwzqwi5OaODmTSacS+/GUqQmoU8wI8HmFfcykUVRye8DfMjKGbVV3+P0XsmWG4mU7ktV/MiSJCOSISo0e8u/KIuNs1w0sRcTPfDniXIcLgHfPOPCIr6kARMfO5z30O2t13QwNKb+vXQzvjDGiPPYZtQ0MIxP19cSHcYV6AJ2KMt38yg6KqIRIKoKspyv3r26GFkyKSEjm+y0ERcTtCa+SI2FFEaHxXWnhu383Zbs3wVUTcJP4B3rZmAP6G1b4JtvCuvgoRAMDnPqe3ZrBnD/Dss8CFF5IhdZGwZW1p78wL+0YX+Ej7MKPqyrYG33eQ8Fp8l5wRp4i4aYGzAQX347v27QAZTo8tgrq/Svk5vstLpmMYkqGzeGQ9zMyjQoT5QsY4pKuqqqaHmS2vk9YMUT9sWdsBAHhhH/+EVVn8IQBPsyrLQxKRI+J8OpGXKmFEvNtpzfAdjuCJfCfymCwPj0jY2dRMlnOOiDHf7k4RSQge3WV0JEoy8AgHRWQknUWuqEJRgO5FEu9OEIwt60qKyJuHJ/WJEF7IVIgYgWbyju/GXSgivKZmHJlVOcdF8KTuCxGuUzM2lQjuS+9cJP4BwJRPHhEeoWZsYqa7KeYqE4YgZGRZSwNWtcehauA+PXNwvDy663OqKmCamnFhVs0XVd2/IaI1E3MzvstJlWCvGbmiWn3lgwlSRCQmV+S5fdff1kyDqTVjeTmaiSmPPSLtPAuRyfqamCHqD90n8i7f9swBScLMAKDFpIg4uYYBwFTGUIQTIsd3XXhEeCkigHUlPssp1VUE8p3IY/Icx3dzRXsFAO+I9wbTkztjs00E+DA1o7dm3HtEDtdRhghRn2xZx3wifA2rByVqzTCPiKoZcQJ2YW2ZeCQoRB3VvXgSKCKAdSWexnclhikiUQ4R7+avZwXeS+/MhYgT2dBrRYRva2bxp6oS9Q1TRN44NOnYBzaX8XRO//1b1eF/IRINBfXrWNKhT0TknhnANJ2Yt/8zYC2juMtrbDCgIBwsTThZvenkNTosAvlO5DE8A80A69VpwdTb4/XECAQU/SxOZty9V0T45Ygs5j0zBAGUWicrWhtQUDXs4OQTee3QBABgXWejZ7/3C8EMq05HeI14dzH/Hz2vKWevFZ8vqvrrTWPE/c2nXW8iKSISw6MQCQUUsPF7q/26jMlPwvOJ4Sb1z2uzartp34xVw9V81HWGCFE38PaJvH5oEgCwaWULl6/HAyPUzNkNijExI0YRMRKs7Ski5ptDNnnjBnYDa1URIbOqxGQ5mFUVRbG9b8ZcKPB8YrhJ/fM8R6QcXqRppWLEDSzevd5SVYn6go3x8vKJvF5WRDatbOXy9XjgVhFJ6hkighWRfBGaZv0GirXTwkHF1Y0vg73mWM2N4j0cwZO6L0R4mFUB+1ki5vwSnmmGDS5i3o1kVW+eqKFgQL/ouPGJ5AoqhsuGV5qaIRYzLNjstYOTjlRPM5qm4bWyInJijzyKSGtDOV3VsUdEXKoqYIzvqpo9T2C6fKPHQw0B7K8W4T0cwRP5TuQx7InkJlkVsJ/9L+pJ4WZFtdceEcBoz7iJeR9MZqBppaKug8MWX4KQldUdcXQ3R5Erqnj5gDufyEAyg+FUFsGAguOWyVOI6OmqDlXSpMA9M4DzoQCmiPDwhwBAjCkiFlV43SNCZlX54OERAWC7NSMq5c5da8b7QqSzsTTCO+oi5r3PNDGjKP7uyiAIkSiKYsS9u/SJvHawpIYc3d2k38DIQHuCTdO5NauKUUTCwYA+sWJHedYVEU7XV7uKCK89NyKgQkR3EvNRRKybVcWMUrEiIm1zBl/TNF8VETetGTYxQ0ZVoh7g5RNhEzMnSmRUBcxbuZ3dnIge3wXMC0ZrRxER9ZrDA/lO5DE8klUBs0fEbmuGb3XK7gKSNvdRzOSLYIMrXplVAWOE182+mT4KMyPqCKaIvHJgwvZ+KzMyGlUB91u5Re6ZYcQdtMDTOb4eEX1qhhSR2oebWdXhTDfv6pQ5xZMZe4oIa8soivFL5gUdHDbw6q0ZMqoSdcD6JY3oTESRLah6e8UuqqpJOboLAB2N7hKXRbdmAGd7vaY5DwPYsQNomkbjuzLDxnfdFyI2p2ZYyh3n6pTNztvd0Mn6l42RkKc+Cxbz7sas2j9JighRP5R8IixPxFl7pnc0jVSmgGgogGOWNvE8nmvcBh0arRlxikgD28ArgSJixSNSUDVd8Y7S+K5caJpmmFU5Tc1Yb82IcTAzRSRlUxHxwx8CmKZm3HhEJphHhBQRoj5gPpHnHfpEmBpy3PJm6bZVd5ZvTsbTOUeL71JZLxSR0vesVhQRUblVvJDvRB6SLxpPcs+nZgpiFBHdI2JzBp8VLl5liDB47JuhMDOi3jhzfScA4MV9Y5h0EPxlGFVbOZ6KD23loMOCqtn2ugGGItIi0CPiJCbBT49IdlaSt3wv+/KdyEPMYTSuFRHb4TJixneba0wRMVozzvrBU9mC7odZRoUIUSds6ErgmO4m5IsaHv/9gO3Pl9UfApRuCtl1zK6JXdM0vS0tcmqmIVxuzdhRRHhPzYStr/MwB2jKGHFQ14VI3lQl8vOI+Btoxn75mDxplXTO2827DNaamZjJo2AjpZDBtu42x0LSLO0iCC+4eNMyAMAv3ui39XmFooq3+lgh0sr7WFzodHiDks4Z039CWzMR++O73HNEbLzm6Jt3JVRDgDovRJgiEgwoCLqMWbffmmEeEd6tmfLUzIyzqRmvX8zb4mEoCts3Y1+GPTxBW3eJ+uQjJ5QKkd+9M4IJGymkuwenkMmraIqGsK6zUdTxXOHUsMpa0uGgIjQvI+5g8R1vRSRqRxHJi3m94UV9FyKcjKqAg0CzvJhwGedTM/4UIqFgAK0NzvfN0MQMUa9s6Epg49ImFFQNj781aPnzWH7I8StauO654kmHnrhs75rAWtLNsbDQFkS87KVL++gRsbNWRObRXaDOC5EspwwRwOwRsTk1w92sanhE7GyGnMr405oB3PlEaGKGqGc+Wm7P/NxGe4Ytutsk0aK7uRihZvauCUk9Q0TsdaypfJ2csuHF4z01wzwiVm5+RXkSeVHXhUieU6oq4GJqhvv4bkldKKiaLSPVFMsR8aEQcTPCS6mqRD3D2jPP7BnBuMXfH6aInCSpPwQwxbzbNKvqm3cbxPlDACBRLnSmbKzS4D81w1ozpIjUNLz2zJi/hvXWjJgKtTESBFNb7UzOGK0Z7ytm46LjXBGhVFWiHlm3JIHjljWjqGr4v28tPD2TyRexcyAFANjU0yr4dM7RVVKbicvsmidaEUlEWQvcgUeEW46I9decLMfXOhHIeSqPyHFKVQUcTM0IqlAVRdFVETs+kamcPx4RwJBhnXlE2OZdUkSI+sTO9MxbfUkUVQ2diQiWS9zOdLqDKunB6C5gVkSsX2P1qRlfFBFqzUgLV7Mq69dZ3TUjsGfH7gYmbUzOpLP+eUTaHRrTNE1DH5lViTrn4nJ75tm9owuqiuZFdzLmSTCYWdXuzYnemhFciOgeERutGWNqxvtAM1FxEbyQ81QewQqRcMj9L6TtXTMCVzI3O1FEMv4pIp0Ot22OpnPIFVQoCtDdLO/dHUGIZE1nI45fwdoz80/PFIoq7n/xIADglNVtXh3PEZ0OzaqetWZi9syqqqrpmSNxzhHvOQsqvNGaIUVEOvTWjA9mVaNCFaeI2OlfTvmqiDhrzTB/yJJElEt7jSBqlYtPWA4A+MUbffN+zE9fPIBdgym0xsO4fMsqr47mCHZNGJ+2F3Sot2ZEm1VtKiLmwQE/FBGW+M17txkv5DyVR+R4ju/a9IgYPTv+PwJ934wNRcSvZFXAtPbbpjGNJmYIogRrzzy3dxQjFVSEiekcvvnEbgDADecfjdbyPhdZaY1HdNP9mI2wtqSeIyLarGrvZo9dXxWF3zXfzs2v/npDioh8GIWI+x+O/V0zYpbeAUBzg31FhBmpREualWAZIIfHZ2xt26SJGYIosaojjk0rW6BqwF8//KYeTcD49n+/g/HpPI7uTuDTm+VWQ4BS2nW7gxFe5hERGe9e+vql62S2oFpqjUyzeIRIiJs3hxSRRYKI1oyVJyVgCjSTxSPiY2tmZVsDIsEAsgVVj2y3wsHxaQA0MUMQAHDjhRsRDip49M0BXH//q3ox8s5gCv/6/H4AwK3/4z0IcbjeeYGermqnEGGKiODWjPk6mbbQnmGKSJxTvDtgmpqx5RGR82cv56k8wlBEeJpV7QWaifSIWN03kzNV9QlO/Us7hIIBrC3vvNgzNGX583YPlvIQju5uEnIugqglzj6qEz/8zCkIBxX84o1+XH//qygUVfztz3+PoqrhguO6cdaGTr+PaRlj34z1lq2xeVfsdSwcDOiKhBWfCDOq8rzRY685RVVb0EdTl+O7vb29+MIXvoC1a9eioaEB69evx6233opczn5OhEjyPBWRsL2pGSPQzH9FxFzR8wrbscuGrgQAe4XIzv5SIXLMUipECAIAPnRsN+683ChG/vDOZ/Hbd0YQCQZwy8XH+n08WzhrzbCpGbGKCGAv1IxdY0UoIsDCqkhdju/u3LkTqqrirrvuwltvvYVvfetb+OEPf4gvf/nLIh7OMVx3zYTs5YjINDUzUe6rJqIh32Tb9TYLkeFUFqPpHBSFFBGCMHPecUYxwvbK/M+z12J1h5ybduej00G6qjE1I17ZbbIR864rIhwVZ/MN9ELeRNnHd4X8tC666CJcdNFF+t/XrVuHXbt24c4778Ttt98u4iEd4dfUjKZpQqUyu1MzbGy2rVH8XcR86IrIsLVCZFc5pnpNRyMaON5lEMRi4LzjuvGDy0/B/7p3B7qbY7jmgxv8PpJt7O6byeSL+jXdG0XEerqqrohwVJwDAQWRUAC5grqgIqJbASQ1q3pmCJicnER7e3vVj8lms8hmjeo3mUwKPZNhVuUwNVMuRHJFFaqqVV2vbS5WhJhVbU7NTJTH49p8HOnbsMRQRDRNW9BZvnOg9Nw4htQQgqjI+cd14/mtH0I4FPAlqNAtbN+M1Zh3dr1TFCP5VCR2RnhFKCIAEGOFyAKKiG4FkFQR8aQ82rNnD+644w78yZ/8SdWP27ZtG1paWvS3np4eoefimqxqUjZyCxmHTO0bEU8MY9eMtUJEV0R8LETWLWmEogCTM3lLFx62uIv8IQQxPx2JqPC4c1EYO6istWaYJy4RDVW9EeSFnQ28IqZmAOurRWRXRGyd6uabb4aiKFXfdu7cOetzDh8+jIsuugiXXHIJrrrqqqpff+vWrZicnNTfDh48aP9/ZANmVo1yGd819+usPSkCChAO8v+FMaZmrLVmxsuKCDOH+UEsHERPWxyANZ8Ia80cu4wKEYJYjOitGYuJy0aYmTeFl75vxooikuU/NQOYskQWGJJgr0mymlVtfVe+9KUv4Yorrqj6MevWrdP/3NfXh3PPPRdnnnkm/umf/mnBrx+NRhGNRu0cyRU8PSKhgIKAAqgaKzTm/2XQM0RCQSGLp9gv4lSusGCbCCjFKANAa9zfO6cNXQkcGJvGnuEpnLG+Y96PK6qaPrp7zNJmr45HEISHsNaMVY+IEWbmTRtKBkWEKeoLtmZ0RUTO1oytn9iSJUuwZMkSSx97+PBhnHvuuTjllFPw4x//GIGAfJUYz0JEURREQ0HM5IsLGlYzAhfeAcYvoqaVipGF7hDGy3cc7T7HPm/oSuDJnUPYu4Aisn80jWxBRSwcwKr2uEenIwjCS1hrZipbQCZfXNDYn/IozIxhyyMiSBExYiOs2QFkVUSEnOrw4cM455xzsGrVKtx+++0YHh7GwMAABgYGRDycY7Icc0QA61ki2bzYcJlYOKj/n6y0Z4ypGZ8LkSXWRniZP+To7iYEPegFEwThPU3RkH4ds9KeSXoUZsaQSRGh8d0KPPHEE9izZw/27NmDlStXzvo3TbO+S0Q0hlmV1xIilv1vTRERWZ02N4QwMpWzVK1PlFszfppVAWB9l7V0VVaIbCSjKkEsWhRFQUcigv7JDMamclixwHJLdtMlpUdE0NSMZUXEg9ccNwg51RVXXAFN0yq+yQTPZFXA+jZEfeGdwH6dnckZtt3SzxwRANiwpFRYDCQzVVNhd7HRXfKHEMSihhnorWzmZtc6KT0iAnJEABseEcEqvFvkLI88gqdHBDCHmll0MAstRKxPzoxLML4LAC3xsJ6muHc4Pe/H7SJFhCDqAjuGVSNV1SuPSPlmz0ZrhnuOCFt8Z3V8t54UkVohx3kjoVWZzJPWDFNEFkj9U1VNj3j3c3yXsaHcnpnPsDqdK2D/WGnrLmWIEMTiplNPV7WuiHjVmtGTVS0kWDOzKvccEQs3v+Yk70WRI7LY0JNVuSki1sJlvJDJrO6bSWUKKKqllpnf47vAwlHvuwenoGmlPRRMPSEIYnFibOC1oIh4PL5rZ9eMrohwn5pZWBHJFVUwVwS1ZiREN6ty84hYbM2w8V2BiojVQoSFmTVGglI4qheanGH+EGrLEMTiR+7WjP1AMz8UkVkrRag1Ix857mZVi60ZDzwiTJ5cyCNiGFX9b8sAwIauUoExX2uGot0Jon5o19NVJWzNlG/20rmiripXQtM0YYqIFY+IWaHn9VrHGzlP5RH8zao2p2aEKiJsA+8Cikja/3h3M6w1s39sWv/5mNnZT4UIQdQLnQnrG3g9T1Y1FRWs0KhEtqCC1Sncc0RYxHuVqRmzUVVEkjcPqBABx0KEmVUthst44xGprogY8e5yFCLdzVEkoiEUVQ29o7MnZzRNw65ytPuxNLpLEIuejkbWmqmuiBRVTV+W6dVNVTQU0HeFVWvPpE0ekjjvHBELN7+yj+4C9V6I+NSa0WfKOVfHZlif1LIiIoFRFSiFGK3vquwTGZ7KYiydQ0ABjupO+HE8giA8hJlVR9K5qjlUfRMzyBVVRIIBLF8g+IwXiqIYPpEqhlUWZhYLB7gnQdtVRGRF3pN5QN6n1gxTKURKiFYVEeYRkUURAeY3rLL8kDUdjVJX9wRB8IEpIrmCWvXFfv9oaaS/p73B07UPCQtDAaIyRABrrzmyj+4CdV6I8B/ftTY1YyQAilMhrE7NTEzL5REBTCO8cwoR8ocQRH3REAnqyvFYlRHefeU27trORk/OxWChZtWKpDSbmOGcqgpYU0QMT6K8N291W4ioqoZ8sST1cV96t0COiBdRxJanZiRZeGdm3kJET1QlfwhB1At6e6aKYbV3pFSIrO7wthCxsm9mWqAiwpThaq85pIhIDFNDAD9bM+IUkWaLu2bG9YV3cnhEAKMQeXdkCqppLG7XINsxQ4oIQdQLVgyrrBBZ47UiooeazX/DlxaUIQJYzBFhcRESKyLezDlJiLkQ8TrQzAtFhH3tmXwR+aI67//RMKvKo4j0tDUgEgwgk1dxuGxC+/XOIeweLCkkFGZGEPVDp4V0VTZht9ZjRYSZVavd8E0LyhABLOaIFAyzrKzUbSGSL/APebE6NZP0sBABSr8k83lAxiU0q4aCAaztbMSuwRT+4PvPzOoNdzVFsao97uPpCILwkoUUkaKq4eDYDABgTae31wYrG3jTOXGKiO4RIUWkNmGKSDioIMDJZR210K8DjNaMyATAUDCAeCSI6VwRqUy+YiGiaZrempHJrAoAG5c1YddgCmPpHCLBALasa8c5x3Tho5uWcft5EQQhP+0LKCLm0d1lLd6M7jIseUSyHkzNWFBEZB7frd9CpMA3QwSw1prJFVRdMREdRdwUC5ULkcq/JEnJFt6Z+YsLjsHazka8Z3kLzlzfIUTWJAhCfjoaq6ersrbMqo64p6O7ACzliOiKiMipGQu7ZqgQkRDeqaqANbOqOdcjITiKuDkWxmAyO+/kDPOHxCNB6XI5etrjuP68o/0+BkEQPsO2bM+3b0Y3qnrsDwFMOSLVAs18VkT08V3JrvFm5C2RBJPlvHkXsOYRYepEYyQovHpnPpH50lWZP6RNIn8IQRCEmY4F9s3sGymFma312B8CWNvAa3hEBBQiJkVkvuTZWlBE5D2ZYPKcw8wAa7tmvAgzYzTpI7zzKCIShpkRBEGYYWbV+XJEWGvG6wwRwLjZqx7xzqZmRLRmSl9T02ZPgpoxckRIEZEOka2ZSltjGV7EuzMW2jcznmYL7+TyhxAEQTCYIjI+nZuVK8To9SlVFTAlq1Zdele6MRXhczOrHPMp8V5se3eLvCcTDO+Fd4C11owXo7uMhfbNkCJCEITstDdGEAkGKm7kLhRVHBwrtWa8DjMDrI3vMkVExPhuJBiAUu7wzxfzro/vkiIiHzkBfTO9NVPFwexFqipjoX0zerw7eUQIgpCUcDCA965qBQA89+7orH/rm8ggX9QQCQWwrDnm+dmMQLMqyaplj4gIs6qiKMYN8DyG1VoY35X3ZILJCTGrLuxg9iJVlbHQvhkj3p0KEYIg5OXM9Z0AgGf3zi5EdH9Ie9yXfCGzR2Q+syibmhExvguY9s3McwOcyZNZVVp4b94F7E3NeKGINC+giOjx7o3kESEIQl7O3NABAHh+7+gsnwgrRPxoywCGIqJqpXUalZgWqIgAxlbd+WLedUWEWjPyIcasWvpauaJa0VQFAEk9VdULj0h5amaehUxjEsa7EwRBzOXEla1oCAcxms5h91BKf/++Ef+MqkDJ98E8GvP5RNICp2aAhS0BNL4rMULMqqaKc75RKm+nZso5IjOVf0EmyKxKEEQNEAkFcNradgDAM3uM9oyfYWZAyaOxUJbIdFZcjghgRRGRf9dM/RYiAhURYH6fiEw5ImNp8ogQBFEbnLm+1J55bu+I/r79o+WJmQ7/FmE2VYl5zxVU/aZUWGtmAUXESFaV9+Ve3pMJRsSumVBAAfNLzZf976VZtdrUjKZpuiLSRh4RgiAkhxUiL7w7hkJRRaGo4oCPo7sMfYS3wnV2Jme8DjQIGN8FDKWDFJEaRESyammUqvrkjJfju/rUTCZ/hKM7lS2gUPaxkCJCEITsvGd5C5pjIaSyBbzZl8ThiRkUVA3RUABLfRjdZegjvBUUEeYPiQQDXF9rzOgx7/PliOhmVXlf7uU9mWBEtGaAhY1Dfigi+aJ2xCQPm5hpCMu38I4gCGIuwYCC09eVVJFn946gV2/LNPoyustIxOZPV9XDzAQZVYGFl63S+K7EZAWYVYGFR3i9TFZtjIR0R3dyjk+EZYiQUZUgiFrB8ImM6kbV1T76Q4DqHhE93l2QPwQwvB/zJ6vS9l1pEaaIhKqHy6T08V3xrZlAQNF/SeZOzjBFhPwhBEHUCmduKAWbbe8dw67B0hivX6O7jES1QkRgvDuDFRgLe0TkfbmX92SCYR4RnsmqAKrG7eYKqv6k8EIRKT1O5ckZincnCKLWOKorgc5EBJm8il++0Q/AX6MqYJhVKw0F6KO7AhbeMQwV/sibX03TyKwqM+I9IkcWIuZiICHwiWlmvskZtvCOChGCIGoFRVFwRjnufaLcXva7NWMoIkfGJOhhZj4pIubXIRrflRARS+9KX2/+1gwrBuKRIEKclZj5aG4wJmfMGIUItWYIgqgdmE+E4XdrpqnK+C6LdxcVZgZUV0TMhQgpIhIiYtcMUN2s6uXEDGO+fTP6wjsyqxIEUUOYC5FYOIDuJv9Gd4EFPCJZsfHuwEKKSKk4URQgHPRvsmgh6rcQEbB9F6juEfEyQ4Qxn0fEWHhHhQhBELXDqvY4VrQ2APB/dBdYwCPigSKiJ6tWmJphr0OxUBCKQoWIdOSKpTAv/uO787dmvBzdZTBFZO7UDDOr0sI7giBqCUVRdFXEb38IYG1qRqRHpFqOSC2EmQH1XIiUf0CizKqVZDIvR3cZ8ykizOjVToUIQRA1xufPXIN1SxrxqVN7/D6K4RGpUIh4MTVTLUekFsLMAMC7W3PJEDU101I2h07M5I74Nz88IvNNzYzRnhmCIGqU41e04MkvneP3MQAAiej8yapeTs1UVUQkNqoC9ayICDKrdpd3Hgwls0f8m5ebdxmVpmY0TTMCzUgRIQiCcIzuEfFJEWFqRyVFRPeIUGtGTkRs3wWAJU1RAMBgqlIhwloz3isiSVO1PkUL7wiCILjAPCKlwMrZxYAnHhE2NVNlfJcUEUnJM7OqMEUkc8S/+dOaYR4RoxAZT5cKolg4IGw1NUEQRD1gDqdku2UYnuaIVBnfld0jIvfpBCJKEekqKyJDlRSRrPfju0x96Z+cwWTZoMrCzMioShAE4Y5gQNF3ycz1iXiaI1JBEcnorRm5bzjrthDJCjKrMkVkLJ3Tix2GH4rIscuasbwlhonpPP7knpeQK6gmoyoVIgRBEG5hqkhqTsy7JzkibHyXFJHaQ9T4bls8rCfYDU/NVkWSM94rIrFwEP985WlIREN4/t0x3Pwfr2NsioyqBEEQvEjME/M+nROviESrjO/qHhEyq8qJPjXDuTWjKAq6ypHDg3N8In4oIgCwcWkzfnD5yQgGFPznK4fx/V/vAUCKCEEQBA+a5gk1Y56RRqHJqqw1c6QiwooTMqtKiiizKgB0NZd9InMKET+SVRnvP3oJvvbx4wEA746kAQDttPCOIAjCNYkKoWZFVcNMnrVmRCarll7DcgUVmqbN+jca35WYoqqhqIqJeAfmN6z6kaxq5rLNq/Bn56zX/07x7gRBEO7RPSKm1syMqVXSKDRZ1Shy5oaa0fiuxJhNpCIUEWZYNbdmSjPmpcf1QxFh/OUFx+DjJy0HAJzY0+LbOQiCIBYLerqqSRGZLv85oIg1i8ZMX3uuYbVWzKp1GfHuVSFiTlc173pJCKyOFyIQUPCtS0/CX330OHQmor6dgyAIYrHQVMGsms4Z/hCRm29DwQCCAQVFVUOmUEQLDMVd3zVD47vykS0akllIwArpSumqTLKLR4IICWgH2UFRFCpCCIIgOFFpAy/LEIkLnJhhxOaJea8VRUTu0wnCbFQVUalWSlf1a2KGIAiCEEuiwnLR6Zz4iRnGfIvvDI+I3C/1cp9OEKw1ExWkTFQyq7LWjJcZIgRBEIR4DEXEaMGzPTNeKCLzLb7Tx3epNSMfOUGpqoxK6ap+ju4SBEEQ4miqML6rb971UBHJHGFWLY/vkiIiH6ILkUrpqqSIEARBLE50RWSWWVX85l0Gey2bu/03S2ZVeWGpqmFBrZlK6arkESEIglicGLtmjhzfjXswJTmfIpIhs6q8iFZEAHO6KlNESk/KZipECIIgFhXmXTOapuGxNwdw93P7AXijiLDk1PkUEdm379blq6KoPTNmDMMqU0SoNUMQBLEYaSoHmk1M5/HJO5/FKwcmAJTa9J9470rhj8+SU4/0iNSGIlKfhYgHisjcdFW9NeNjmBlBEATBH6aI5IoqXjkwgYZwEF9831pc9f51nqz0iM2zgbdWxnfr8lVRL0QEKiJz01VTWaaI1OW3nCAIYtHSHAuhMxHFxHQOl23uwf/+4FHoKr8GeAFTRObmiGSoNSMv+aJ4RWRuuqphVqXWDEEQxGIiFAzgsevfB1XVPC1AGPMrItSakRYvWzMsXZXliDQ3UCFCEASx2PBzbcaCyaqSKyJyl0mCyHpgVu1unp2uaphV67L2IwiCIATBFI+sSRFRVc1IESdFRD4uPK4bR3cl0BIXp06wHBGWrko5IgRBEIQIjBwRoxBh06Hmf5eVunxV7GqOCe/jsXTVfFHD8FQWyZmSIuKFg5ogCIKoH3RFxNSayZpGeWVXRISfLpvN4qSTToKiKHj11VdFP5w0mNNVD41N608QUkQIgiAInlRSRJhRNaAAoQD/LfM8EV6I3HjjjVi+fLnoh5ESlq66dzitvy9BOSIEQRAER6IVzKrm0V1FqeNC5NFHH8Xjjz+O22+/XeTDSEt3WRHZOzwFAIhHgggJNMgSBEEQ9QdrvVRSRGRvywACPSKDg4O46qqr8PDDDyMej1v6nGw2i2w2q/89mUyKOp4nMEVkz1CpEKG2DEEQBMGbSkvvjFRVuY2qgCBFRNM0XHHFFfjTP/1TnHrqqZY/b9u2bWhpadHfenp6RBzPM1iWiFGIkFGVIAiC4EssdOTSO6aORMPyKyK2TnjzzTdDUZSqbzt37sQdd9yBVCqFrVu32jrM1q1bMTk5qb8dPHjQ1ufLBktXPTwxA4AUEYIgCII/0SqKSKwGFBFbr4xf+tKXcMUVV1T9mHXr1uHJJ5/Ec889h2h0dtLcqaeeissvvxx33313xc+NRqNHfE4t0z1nRJgUEYIgCII3TBHJFCp4RGpAEbFViCxZsgRLlixZ8OO++93v4qtf/ar+976+Plx44YV44IEHsGXLFvunrFFYuiqDFBGCIAiCN/rUTP7IqZm6NauuWrVq1t8TiQQAYP369Vi5cqWIh5QSliPCaKZChCAIguAMW3qXzhWgqhoCAUVXRGRPVQXqdNeMV7B0VQa1ZgiCIAjerGhtQDwSxMR0Hve+sB+AoY7UgiLiyQnXrFkDTdNw0kknefFw0mBOVwWAJgozIwiCIDjTFAvjpos2AgBue3QnDo1P0/guYdBl8omQR4QgCIIQwWdPX43T1rQhnSti63++YYzvkiJCdJsVEWrNEARBEAIIBBTc9oebEAkF8Nt3RvDgjkMADCOrzFAhIhhSRAiCIAgvWL8kgRvOPxoA8E45SJMUEWJWlggpIgRBEIRIvnj2WpywokX/ey3kiMh/whqHpasCpIgQBEEQYgkFA/jGH21CKFCa2KyFZFUqRARjVkSaSREhCIIgBHPssmbc/OGNaAgHsXltu9/HWRC6RRdMN3lECIIgCI/54vvW4cqz1iIYUBb+YJ+hV0bBLGtpQCigIBwMUCFCEARBeEYtFCEAFSLCaWkI44efOQXRcAChIHXCCIIgCMIMFSIecN5x3X4fgSAIgiCkhG7RCYIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDSpECIIgCILwDam372qaBgBIJpM+n4QgCIIgCKuw1232Ol4NqQuRVCoFAOjp6fH5JARBEARB2CWVSqGlpaXqxyialXLFJ1RVRV9fH5qamqAoit/H8Z1kMomenh4cPHgQzc3Nfh9n0ULfZ2+g77M30PfZG+j7PBtN05BKpbB8+XIEAtVdIFIrIoFAACtXrvT7GNLR3NxMT3QPoO+zN9D32Rvo++wN9H02WEgJYZBZlSAIgiAI36BChCAIgiAI36BCpIaIRqO49dZbEY1G/T7Kooa+z95A32dvoO+zN9D32TlSm1UJgiAIgljckCJCEARBEIRvUCFCEARBEIRvUCFCEARBEIRvUCFCEARBEIRvUCFS42SzWZx00klQFAWvvvqq38dZVPT29uILX/gC1q5di4aGBqxfvx633norcrmc30dbFHz/+9/HmjVrEIvFsGXLFrz44ot+H2lRsW3bNpx22mloampCV1cXPv7xj2PXrl1+H2vRc9ttt0FRFFx//fV+H6VmoEKkxrnxxhuxfPlyv4+xKNm5cydUVcVdd92Ft956C9/61rfwwx/+EF/+8pf9PlrN88ADD+CGG27ArbfeipdffhknnngiLrzwQgwNDfl9tEXD008/jauvvhrPP/88nnjiCeTzeVxwwQVIp9N+H23Rsn37dtx1113YtGmT30epLTSiZvnlL3+pbdy4UXvrrbc0ANorr7zi95EWPd/4xje0tWvX+n2Mmmfz5s3a1Vdfrf+9WCxqy5cv17Zt2+bjqRY3Q0NDGgDt6aef9vsoi5JUKqUdddRR2hNPPKF94AMf0K677jq/j1QzkCJSowwODuKqq67Cv/7rvyIej/t9nLphcnIS7e3tfh+jpsnlctixYwfOO+88/X2BQADnnXcennvuOR9PtriZnJwEAHr+CuLqq6/GxRdfPOt5TVhD6qV3RGU0TcMVV1yBP/3TP8Wpp56K3t5ev49UF+zZswd33HEHbr/9dr+PUtOMjIygWCyiu7t71vu7u7uxc+dOn061uFFVFddffz3OOussHH/88X4fZ9Fx//334+WXX8b27dv9PkpNQoqIRNx8881QFKXq286dO3HHHXcglUph69atfh+5JrH6fTZz+PBhXHTRRbjkkktw1VVX+XRygnDG1VdfjTfffBP333+/30dZdBw8eBDXXXcd7r33XsRiMb+PU5NQxLtEDA8PY3R0tOrHrFu3Dp/61Kfws5/9DIqi6O8vFosIBoO4/PLLcffdd4s+ak1j9fsciUQAAH19fTjnnHNw+umn4yc/+QkCAarf3ZDL5RCPx/Hggw/i4x//uP7+z3/+85iYmMAjjzzi3+EWIddccw0eeeQR/OY3v8HatWv9Ps6i4+GHH8YnPvEJBINB/X3FYhGKoiAQCCCbzc76N+JIqBCpQQ4cOIBkMqn/va+vDxdeeCEefPBBbNmyBStXrvTxdIuLw4cP49xzz8Upp5yCe+65hy4onNiyZQs2b96MO+64A0CpdbBq1Spcc801uPnmm30+3eJA0zRce+21eOihh/DUU0/hqKOO8vtIi5JUKoX9+/fPet+VV16JjRs34qabbqJWmAXII1KDrFq1atbfE4kEAGD9+vVUhHDk8OHDOOecc7B69WrcfvvtGB4e1v9t6dKlPp6s9rnhhhvw+c9/Hqeeeio2b96Mb3/720in07jyyiv9Ptqi4eqrr8Z9992HRx55BE1NTRgYGAAAtLS0oKGhwefTLR6ampqOKDYaGxvR0dFBRYhFqBAhiHl44oknsGfPHuzZs+eIAo+ERHdceumlGB4exle+8hUMDAzgpJNOwmOPPXaEgZVwzp133gkAOOecc2a9/8c//jGuuOIK7w9EEPNArRmCIAiCIHyDXHcEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfgGFSIEQRAEQfjG/wM+Mbg9yFnJtwAAAABJRU5ErkJggg==",
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"backend = np\n",
"\n",
"def function(x):\n",
" return x*backend.sin(x**2 + 1)\n",
"\n",
"def analytical_gradient(x):\n",
" return backend.sin(x**2 + 1) + 2*x**2*backend.cos(x**2 + 1)\n",
"\n",
"\n",
"x = np.linspace(-5, 5, 100)\n",
"plt.plot(x, function(x), label=\"f(x)\")\n",
"\n",
"descent_x, descent_y = gradient_descent(1, 0.01, 300, solver=\"analytical\")\n",
"\n",
"# Change the backend to JAX\n",
"backend = jnp\n",
"jax_descend_x, jax_descend_y = gradient_descent(1, 0.01, 300, solver=\"jax\")\n",
"\n",
"plt.scatter(descent_x, descent_y, label=\"Gradient descent\", marker=\"v\", s=10, color=\"red\") \n",
"plt.scatter(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\", s=5, color=\"black\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.15"
}
},
"nbformat": 4,
"nbformat_minor": 5
}