{ "cells": [ { "cell_type": "markdown", "id": "1da77599", "metadata": {}, "source": [ "# Exercises week 38\n", "\n", "## September 15-19\n", "\n", "## Resampling and the Bias-Variance Trade-off\n" ] }, { "cell_type": "markdown", "id": "1b63f3b9", "metadata": {}, "source": [ "**Python Code can be found at https://github.uio.no/larsbog/FYS-STK4155**" ] }, { "cell_type": "markdown", "id": "e9f27b0e", "metadata": {}, "source": [ "### Learning goals\n", "\n", "After completing these exercises, you will know how to\n", "\n", "- Derive expectation and variances values related to linear regression\n", "- Compute expectation and variances values related to linear regression\n", "- Compute and evaluate the trade-off between bias and variance of a model\n", "\n", "### Deliverables\n", "\n", "Complete the following exercises while working in a jupyter notebook. Then, in canvas, include\n", "\n", "- The jupyter notebook with the exercises completed\n", "- An exported PDF of the notebook (https://code.visualstudio.com/docs/datascience/jupyter-notebooks#_export-your-jupyter-notebook)\n" ] }, { "cell_type": "markdown", "id": "984af8e3", "metadata": {}, "source": [ "## Use the books!\n", "\n", "This week deals with various mean values and variances in linear regression methods (here it may be useful to look up chapter 3, equation (3.8) of [Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer](https://www.springer.com/gp/book/9780387848570)).\n", "\n", "For more discussions on Ridge regression and calculation of expectation values, [Wessel van Wieringen's](https://arxiv.org/abs/1509.09169) article is highly recommended.\n", "\n", "The exercises this week are also a part of project 1 and can be reused in the theory part of the project.\n", "\n", "### Definitions\n", "\n", "We assume that there exists a continuous function $f(\\boldsymbol{x})$ and a normal distributed error $\\boldsymbol{\\varepsilon}\\sim N(0, \\sigma^2)$ which describes our data\n" ] }, { "cell_type": "markdown", "id": "c16f7d0e", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{y} = f(\\boldsymbol{x})+\\boldsymbol{\\varepsilon}\n", "$$\n" ] }, { "cell_type": "markdown", "id": "9fcf981a", "metadata": { "vscode": { "languageId": "plaintext" } }, "source": [ "We further assume that this continous function can be modeled with a linear model $\\mathbf{\\tilde{y}}$ of some features $\\mathbf{X}$.\n" ] }, { "cell_type": "markdown", "id": "d4189366", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{y} = \\boldsymbol{\\tilde{y}} + \\boldsymbol{\\varepsilon} = \\boldsymbol{X}\\boldsymbol{\\beta} +\\boldsymbol{\\varepsilon}\n", "$$\n" ] }, { "cell_type": "markdown", "id": "f4fca21b", "metadata": {}, "source": [ "We therefore get that our data $\\boldsymbol{y}$ has an expectation value $\\boldsymbol{X}\\boldsymbol{\\beta}$ and variance $\\sigma^2$, that is $\\boldsymbol{y}$ follows a normal distribution with mean value $\\boldsymbol{X}\\boldsymbol{\\beta}$ and variance $\\sigma^2$.\n" ] }, { "cell_type": "markdown", "id": "5de0c7e6", "metadata": {}, "source": [ "## Exercise 1: Expectation values for ordinary least squares expressions\n" ] }, { "cell_type": "markdown", "id": "d878c699", "metadata": {}, "source": [ "**a)** With the expressions for the optimal parameters $\\boldsymbol{\\hat{\\beta}_{OLS}}$ show that\n" ] }, { "cell_type": "markdown", "id": "08b7007d", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}(\\boldsymbol{\\hat{\\beta}_{OLS}}) = \\boldsymbol{\\beta}.\n", "$$\n" ] }, { "cell_type": "markdown", "id": "3a6e76d0", "metadata": {}, "source": [ "
\n", "\n", "$$\n", "\\mathbb{E}(\\boldsymbol{\\hat{\\beta}_{OLS}}) = \\mathbb{E} \\left((X^T X)^{-1} X^T y\\right) = \\mathbb E \\left((X^T X)^{-1} X^T \\tilde y + (X^T X)^{-1} X^T \\epsilon \\right) = \\beta + \\mathbb{E} \\left((X^T X)^{-1} X^T \\epsilon \\right) = \\beta\n", "$$\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "46e93394", "metadata": {}, "source": [ "**b)** Show that the variance of $\\boldsymbol{\\hat{\\beta}_{OLS}}$ is\n" ] }, { "cell_type": "markdown", "id": "be1b65be", "metadata": {}, "source": [ "$$\n", "\\mathbf{Var}(\\boldsymbol{\\hat{\\beta}_{OLS}}) = \\sigma^2 \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1}.\n", "$$\n" ] }, { "cell_type": "markdown", "id": "e39cd224", "metadata": {}, "source": [ "
\n", "\n", "$$\n", "\\begin{aligned}\n", "\\mathbf{Var}(\\hat \\beta_{OLS}) &= \\mathbb E (\\beta_{OLS} - \\mathbb{E}(\\hat{\\beta}_{OLS}))^2 \\\\ &= \\mathbb E (\\beta_{OLS} - \\beta)^2\\\\ &= \\mathbb E (\\beta + (X^T X)^{-1} X^T \\epsilon - \\beta)^2\\\\ &= \\mathbb E ((X^T X)^{-1} X^T \\epsilon)^2\\\\ &= \\sigma^2 \\mathbb E ((X^T X)^{-1} X^T X ((X^T X)^{-1})^T)\\\\ &= \\sigma^2 ((X^T X)^{-1})^T\\\\ &= \\sigma^2 (X^T X)^{-1}\n", "\\end{aligned}\n", "$$\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "d2143684", "metadata": {}, "source": [ "We can use the last expression when we define a [confidence interval](https://en.wikipedia.org/wiki/Confidence_interval) for the parameters $\\boldsymbol{\\hat{\\beta}_{OLS}}$.\n", "A given parameter ${\\boldsymbol{\\hat{\\beta}_{OLS}}}_j$ is given by the diagonal matrix element of the above matrix.\n" ] }, { "cell_type": "markdown", "id": "f5c2dc22", "metadata": {}, "source": [ "## Exercise 2: Expectation values for Ridge regression\n" ] }, { "cell_type": "markdown", "id": "3893e3e7", "metadata": {}, "source": [ "**a)** With the expressions for the optimal parameters $\\boldsymbol{\\hat{\\beta}_{Ridge}}$ show that\n" ] }, { "cell_type": "markdown", "id": "79dc571f", "metadata": {}, "source": [ "$$\n", "\\mathbb{E} [ \\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}} ]=(\\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I}_{pp})^{-1} (\\mathbf{X}^{\\top} \\mathbf{X})\\boldsymbol{\\beta}\n", "$$\n" ] }, { "cell_type": "markdown", "id": "028209a1", "metadata": {}, "source": [ "We see that $\\mathbb{E} [ \\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}} ] \\not= \\mathbb{E} [\\hat{\\boldsymbol{\\beta}}^{\\mathrm{OLS}} ]$ for any $\\lambda > 0$.\n" ] }, { "cell_type": "markdown", "id": "73186528", "metadata": {}, "source": [ "
\n", "\n", "$$\n", "\\mathbb E \\hat \\beta_\\mathrm{Ridge} = \\mathbb E \\left((X^TX + \\lambda I)^{-1} X^T \\tilde y\\right) + \\mathbb E \\left((X^T X + \\lambda I)^{-1} X^T \\epsilon\\right) = \\mathbb E \\left((X^TX + \\lambda I)^{-1} X^T \\tilde y\\right) = (X^TX + \\lambda I)^{-1} X^T X \\beta\n", "$$\n", "\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "b4e721fc", "metadata": {}, "source": [ "**b)** Show that the variance is\n" ] }, { "cell_type": "markdown", "id": "090eb1e1", "metadata": {}, "source": [ "$$\n", "\\mathbf{Var}[\\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}}]=\\sigma^2[ \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1} \\mathbf{X}^{T}\\mathbf{X} \\{ [ \\mathbf{X}^{\\top} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}\\}^{T}\n", "$$\n" ] }, { "cell_type": "markdown", "id": "6b8e8697", "metadata": {}, "source": [ "We see that if the parameter $\\lambda$ goes to infinity then the variance of the Ridge parameters $\\boldsymbol{\\beta}$ goes to zero.\n" ] }, { "cell_type": "markdown", "id": "06835e6f", "metadata": {}, "source": [ "
\n", "\n", "$$\n", "\\begin{aligned}\n", "\\mathrm{Var}(\\hat \\beta_\\mathrm{Ridge}) &= \\mathbb E (\\hat \\beta_\\mathrm{Ridge} - \\mathbb E \\hat \\beta_\\mathrm{Ridge})^2\\\\ \n", "&= \\mathbb E \\left[\\left((X^TX + \\lambda I)^{-1} X^T \\tilde y\\right) + \\left((X^T X + \\lambda I)^{-1} X^T \\epsilon\\right) - (X^TX + \\lambda I)^{-1} X^T X \\beta \\right]^2 \\\\\n", "&= \\mathbb E \\left[(X^T X + \\lambda I)^{-1} X^T \\epsilon\\right]^2\\\\\n", "&= \\sigma^2 (X^T X + \\lambda I)^{-1} X^T X ((X^T X + \\lambda I)^{-1})^T\n", "\\end{aligned}\n", "$$\n", "\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "74bc300b", "metadata": {}, "source": [ "## Exercise 3: Deriving the expression for the Bias-Variance Trade-off\n" ] }, { "cell_type": "markdown", "id": "eeb86010", "metadata": {}, "source": [ "The aim of this exercise is to derive the equations for the bias-variance tradeoff to be used in project 1.\n", "\n", "The parameters $\\boldsymbol{\\hat{\\beta}_{OLS}}$ are found by optimizing the mean squared error via the so-called cost function\n" ] }, { "cell_type": "markdown", "id": "522a0d1d", "metadata": {}, "source": [ "$$\n", "C(\\boldsymbol{X},\\boldsymbol{\\beta}) =\\frac{1}{n}\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2=\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]\n", "$$\n" ] }, { "cell_type": "markdown", "id": "831db06c", "metadata": {}, "source": [ "**a)** Show that you can rewrite this into an expression which contains\n", "\n", "- the variance of the model (the variance term)\n", "- the expected deviation of the mean of the model from the true data (the bias term)\n", "- the variance of the noise\n", "\n", "In other words, show that:\n" ] }, { "cell_type": "markdown", "id": "8cc52b3c", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathrm{Bias}[\\tilde{y}]+\\mathrm{var}[\\tilde{y}]+\\sigma^2,\n", "$$\n" ] }, { "cell_type": "markdown", "id": "8cb50416", "metadata": {}, "source": [ "with\n" ] }, { "cell_type": "markdown", "id": "e49bdbb4", "metadata": {}, "source": [ "$$\n", "\\mathrm{Bias}[\\tilde{y}]=\\mathbb{E}\\left[\\left(\\boldsymbol{y}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]\\right)^2\\right],\n", "$$\n" ] }, { "cell_type": "markdown", "id": "eca5554a", "metadata": {}, "source": [ "and\n" ] }, { "cell_type": "markdown", "id": "b1054343", "metadata": {}, "source": [ "$$\n", "\\mathrm{var}[\\tilde{y}]=\\mathbb{E}\\left[\\left(\\tilde{\\boldsymbol{y}}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]\\right)^2\\right]=\\frac{1}{n}\\sum_i(\\tilde{y}_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2.\n", "$$\n", "\n", "In order to arrive at the equation for the bias, we have to approximate the unknown function $f$ with the output/target values $y$.\n" ] }, { "cell_type": "markdown", "id": "7dcfe676", "metadata": {}, "source": [ "
\n", "\n", "$$\n", "\\begin{aligned}\n", "\\mathbb E[(y-\\tilde y)^2]\n", "&= \\mathbb E[(f+\\varepsilon-\\tilde y)^2] \\\\\n", "&= \\mathbb E[(f - \\mathbb E[\\tilde y] + \\mathbb E[\\tilde y]-\\tilde y + \\varepsilon)^2] \\\\\n", "&= \\mathbb E[(f - \\mathbb E[\\tilde y])^2] + \\mathbb E[(\\tilde y - \\mathbb E[\\tilde y])^2] + \\mathbb E[\\varepsilon^2] \\\\\n", "&\\quad + 2\\,\\mathbb E[(f - \\mathbb E[\\tilde y])(\\mathbb E[\\tilde y]-\\tilde y)]\n", "+ 2\\,\\mathbb E[(f - \\mathbb E[\\tilde y])\\varepsilon]\n", "+ 2\\,\\mathbb E[(\\mathbb E[\\tilde y]-\\tilde y)\\varepsilon] \\\\\n", "&= (f - \\mathbb E[\\tilde y])^2 + \\mathrm{Var}(\\tilde y) + \\sigma^2,\n", "\\end{aligned}\n", "$$\n", "\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "70fbfcd7", "metadata": {}, "source": [ "**b)** Explain what the terms mean and discuss their interpretations.\n" ] }, { "cell_type": "markdown", "id": "6f47d057", "metadata": {}, "source": [ "
\n", "\n", "1. Bias: Difference between true function and average prediction. Systematic error. High bias = underfitting.\n", "\n", "2. Variance: How much predictions change if we retrain on different data. Sensitivity to training data. High variance = overfitting.\n", "\n", "3. Irreducible noise $\\sigma^2$: Random noise in data we can’t model away. Sets the minimum possible error.\n", "\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "b8f8b9d1", "metadata": {}, "source": [ "## Exercise 4: Computing the Bias and Variance\n" ] }, { "cell_type": "markdown", "id": "9e012430", "metadata": {}, "source": [ "Before you compute the bias and variance of a real model for different complexities, let's for now assume that you have sampled predictions and targets for a single model complexity using bootstrap resampling.\n", "\n", "**a)** Using the expression above, compute the mean squared error, bias and variance of the given data. Check that the sum of the bias and variance correctly gives (approximately) the mean squared error.\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "b5bf581c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MSE (218.794) = Bias (210.484) + Variance (8.306) = 218.790\n" ] } ], "source": [ "import numpy as np\n", "\n", "n = 100\n", "bootstraps = 1000\n", "\n", "predictions = np.random.rand(bootstraps, n) * 10 + 10\n", "# The definition of targets has been updated, and was wrong earlier in the week.\n", "targets = np.random.rand(1, n)\n", "\n", "def calculate_key_metrics(y_pred, y_test):\n", " y_pred = np.array(y_pred).T\n", " y_test = np.array(y_test).T\n", " error = np.mean( np.mean((y_test - y_pred)**2, axis=1, keepdims=True) )\n", " bias = np.mean( (y_test - np.mean(y_pred, axis=1, keepdims=True))**2 )\n", " variance = np.mean( np.var(y_pred, axis=1, keepdims=True) )\n", "\n", " return error, bias, variance\n", "\n", "def print_key_metrics(predictions, targets):\n", " mse, bias, variance = calculate_key_metrics(predictions, targets)\n", "\n", " print(f\"MSE ({mse:.3f}) = Bias ({bias:.3f}) + Variance ({variance:.3f}) = {bias + variance:.3f}\")\n", "\n", "print_key_metrics(predictions, targets)" ] }, { "cell_type": "markdown", "id": "7b1dc621", "metadata": {}, "source": [ "**b)** Change the prediction values in some way to increase the bias while decreasing the variance.\n" ] }, { "cell_type": "code", "execution_count": 2, "id": "c8e777a6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MSE (731.318) = Bias (729.240) + Variance (2.077) = 731.316\n" ] } ], "source": [ "predictions = predictions * 0.5 + 20\n", "\n", "print_key_metrics(predictions, targets)" ] }, { "cell_type": "markdown", "id": "dbce0baf", "metadata": {}, "source": [ "\n", "**c)** Change the prediction values in some way to increase the variance while decreasing the bias.\n" ] }, { "cell_type": "code", "execution_count": 3, "id": "a30ea2b6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MSE (830.999) = Bias (0.332) + Variance (830.621) = 830.954\n" ] } ], "source": [ "predictions = (predictions - np.mean(predictions, axis=0)) * 20\n", "print_key_metrics(predictions, targets)" ] }, { "cell_type": "markdown", "id": "8da63362", "metadata": {}, "source": [ "**d)** Perform a bias-variance analysis of a polynomial OLS model fit to a one-dimensional function by computing and plotting the bias and variances values as a function of the polynomial degree of your model.\n" ] }, { "cell_type": "code", "execution_count": 4, "id": "dd5855e4", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from sklearn.preprocessing import (\n", " PolynomialFeatures,\n", ") # use the fit_transform method of the created object!\n", "from sklearn.linear_model import LinearRegression\n", "from sklearn.metrics import mean_squared_error\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.utils import resample" ] }, { "cell_type": "code", "execution_count": 5, "id": "7e35fa37", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAGwCAYAAACHJU4LAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjUsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvWftoOwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAZXJJREFUeJzt3XdcVFf6P/DPnUoH6aAUey9YYrBi7LFrYomJDUsSjXGTbBKTTdvvZjW7Wd1k488YuybGklgSYzRGBRV7b9gQEBVEQDoMU87vD3BkKAoK3AE+79drXnru3HvnmTsX5uHcc88jCSEEiIiIiKyQQu4AiIiIiErDRIWIiIisFhMVIiIislpMVIiIiMhqMVEhIiIiq8VEhYiIiKwWExUiIiKyWiq5A3gaJpMJd+7cgaOjIyRJkjscIiIiKgMhBDIyMuDr6wuF4tF9JtU6Ublz5w78/PzkDoOIiIieQFxcHOrVq/fIdap1ouLo6AgUvFEnJye5wyEiIqIySE9Ph5+fn/l7/FGqdaLy4HKPk5MTExUiIqJqpizDNjiYloiIiKwWExUiIiKyWkxUiIiIyGpV6zEqZWU0GqHX6+UOgx5DrVZDqVTKHQYREVmRGp2oCCGQkJCA1NRUuUOhMnJxcYG3tzfnxSEiIqCmJyoPkhRPT0/Y2dnxy8+KCSGQnZ2NxMREAICPj4/cIRERkRWosYmK0Wg0Jylubm5yh0NlYGtrCwBITEyEp6cnLwMREVHNHUz7YEyKnZ2d3KFQOTz4vDimiIiIUJMTlQd4uad64edFRESF1fhEhYiIiKovJipERERktZioEBERUTGL9l3HqohoZOcZZI2DiYoVmjRpEiRJwquvvlrsuZkzZ0KSJEyaNAkAcO/ePbz22mvw9/eHVquFt7c3+vfvj4iICPM2gYGBkCSp2GP+/PlV+r6IiKh6SMnKw//2XsOnv15C1/l7sXD3VaRly3OTQ429Pbkok0ngfnaebK9fx04DhaLsA0X9/Pywfv16LFy40Hzbbm5uLtatWwd/f3/zeqNGjUJeXh5Wr16NBg0a4O7du9izZw+Sk5Mt9vf3v/8d06ZNs1hWlvLaRERU+6w5HINcvQkAcD9bj0X7rmN0Jz8426mrPJZak6jcz85Dh3/8Kdvrn/xbH7g5aMu8fvv27REVFYXNmzdj/PjxAIDNmzfD398f9evXBwCkpqbiwIEDCAsLQ8+ePQEAAQEBeOaZZ4rtz9HREd7e3hX2foiIqGbKyTNi9aEYi2VD2/qiroutLPHw0o8VmzJlClauXGlur1ixApMnTza3HRwc4ODggK1bt0Kn08kUJRER1SSbTsbhfpHLPNN7NpAtHiYqVuzll1/GwYMHERsbi9jYWERERODll182P69SqbBq1SqsXr0aLi4u6Nq1Kz744AOcO3eu2L7ee+89c2Lz4HHgwIEqfkdERGTNDEYTtocfxluqjXBDGgAgpKkHmnk7yRYTExUr5uHhgUGDBmHVqlVYuXIlBg0aBHd3d4t1Ro0ahTt37uCXX37BgAEDEBYWhvbt22PVqlUW6/31r3/FmTNnLB4dO3as4ndERETW7PcLCXg+czNmq7YiQjsbIYrTmNGjoawx1ZoxKnXsNDj5tz6yvv6TmDJlCmbNmgUAWLRoUYnr2NjYoG/fvujbty8++ugjTJ06FZ988on5ziAAcHd3R6NGjZ4weiIiqumEEPgx7BSWK8MAACYooPfugGcbuMoaV61JVBQKqVyDWa3FgAEDkJeXB0mS0L9//zJt06JFC2zdurXSYyMioprjUFQyOiVuhq06/w7Z9cZeGN+rneylTWS99PPpp58Wm9ujWbNmcoZkdZRKJSIjI3Hp0qVi1YSTk5Px3HPP4fvvv8e5c+cQHR2NTZs24V//+heGDRtmsW5GRgYSEhIsHunp6VX8boiIyFqt2HcRE1W7AAAGocBOx5Ho31L+u0Vl71Fp2bIl/vzz4W3DKpXsIVkdJ6eSBzE5ODigc+fOWLhwIaKioqDX6+Hn54dp06bhgw8+sFj3448/xscff2yxbMaMGfj2228rNXYiIrJ+F++kwTdmM1zVmQCAX03BGNrzWSjLMf9XZZE9K1CpVGWe30On01nchltTewSKDoQtqvBlnXnz5mHevHmPXD8mJuaRzxMRUe22LPwq/qL8zdzeoB6BVR3qyRrTA7Lf9XPt2jX4+vqiQYMGGD9+PG7evFnquvPmzYOzs7P54efnV6WxEhER1TRxKdkwXvwF/op7AIBwYxt07RoCG7XysdtWBVkTlc6dO2PVqlXYuXMnFi9ejOjoaHTv3h0ZGRklrj937lykpaWZH3FxcVUeMxERUU2y/MANTFP8Ym6vxDC8Ehwga0yFyXrpZ+DAgeb/t2nTBp07d0ZAQAA2btyI0NDQYutrtVpotdXvzh0iIiJrdD8rDwdOnMZsRRIA4JypPup3GgCXJ5xSozLIPkalMBcXFzRp0gTXr1+XOxQiIqIab+2RWETpXdEVX+NFZThuwwufdZdvuvySyD5GpbDMzExERUXBx8dH7lCIiIhqtFy9EasKig/mwAZrjP3h2Pp51KtjJ3doFmRNVN555x2Eh4cjJiYGhw4dwogRI6BUKjFu3Dg5wyIiIqrxNp28hZSsPItl02WeLr8ksl76uXXrFsaNG4fk5GR4eHigW7duOHLkCDw8POQMi4iIqEYzmgQ2hZ+CB7JwD3UAAD2aeKCFr3zFB0sja6Kyfv16OV+eiIioVtp5IQHDM9ZjvPZPbDZ2xwLDC3i1R2e5wyqRVY1RobKJiYmBJEk4c+ZMhe/7+vXr8PLygp2dHSIiIip8/0REJC8hBH7YdxpjlfuglQwYroxAE586CG7oJndoJWKiYoUmTZpkUf/Izc0NAwYMwLlz5wAAfn5+iI+PR6tWrSr0de/cuYO+ffuiW7duCA0NxeDBg3H+/HmLdfR6Pd577z20bt0a9vb28PX1xYQJE3Dnzp0KjYWIiCrH4RvJaJ+4GXZS/kzvG4whGNcrSPbig6VhomKlBgwYgPj4eMTHx2PPnj1QqVQYPHgwUFCo0Nvbu0LrIt2/fx/9+/dH9+7dsXHjRvzvf//Dq6++iv79+yM6Otq8XnZ2Nk6dOoWPPvoIp06dwubNm3HlyhUMHTq0wmIhIqLKs2JfJCYVFB80Cgm/O4zCACsoPlgaJipWSqvVwtvbG97e3mjXrh3ef/99xMXF4d69e8Uu/RiNRoSGhqJ+/fqwtbVF06ZN8dVXX1nsLywsDM888wzs7e3h4uKCrl27IjY2FihIPgYNGoSuXbti9erV5irN8+bNw8yZM9GvXz8kJiYCAJydnbF7926MHj0aTZs2xbPPPotvvvkGJ0+efGT5AyIikl9kfDq8ojfDXcqvlfeb6VkM7vksVErrTQesasI3KllmZia+//57NGrUCG5ubsjKyrJ43mQyoV69eti0aRPc3Nxw6NAhTJ8+HT4+Phg9ejQMBgOGDx+OadOm4ccff0ReXh6OHTtm7uazs7PDoUOHSnztDz/8EB9++OEj40tLS4MkSXBxcanAd01ERBVtWfg1vFGo+OB69Qgs72DddfNqZ6Jy6Bvg8KLHr+fTFnipyJ1J68YC8Wcfv23wTKDLrCcOcfv27XBwcAAAZGVlwcfHB9u3b4dCUTzrVavV+Oyzz8zt+vXr4/Dhw9i4cSNGjx6N9PR0pKWlYfDgwWjYMP8e+ebNmz9xbIXl5ubivffew7hx4+DkZH23tRERUb7bqTnQnf8Fgeq7AIADxlZ4plsv2Gqso/hgaWpnoqLLADLKMPjTuW7xZdlJZdtWV3JhxbLq1asXFi9eDBSMH/l//+//YeDAgTh27FiJ6y9atAgrVqzAzZs3kZOTg7y8PLRr1w4A4OrqikmTJqF///7o27cv+vTpg9GjRz/1DMB6vR6jR4+GEMIcKxERWacVB25gqrJw8cGh+DI4UNaYysJ6L0pVJq0j4Oj7+Iede/Ft7dzLtq3W8alCtLe3R6NGjdCoUSN06tQJy5YtQ1ZWFpYuXVps3fXr1+Odd95BaGgo/vjjD5w5cwaTJ09GXt7DGQdXrlyJw4cPo0uXLtiwYQOaNGmCI0eOPHF8D5KU2NhY7N69m70pRERWLC1bj+vHd6Kd4gYA4KIpAH4dnoervfUUHyxN7exR6TLryS/LFL0UVEUkSYJCoUBOTk6x5yIiItClSxe8/vrr5mVRUVHF1gsKCkJQUBDmzp2L4OBgrFu3Ds8++2y5Y3mQpFy7dg379u2Dm5t13ntPRET5vj8aiyN5DfCBMhTTlNvxnXEI3rHC6fJLUjsTlWpAp9MhISEBKLj088033yAzMxNDhgwptm7jxo2xZs0a7Nq1C/Xr18fatWtx/Phx1K9fHwAQHR2N7777DkOHDoWvry+uXLmCa9euYcKECeWOS6/X44UXXsCpU6ewfft2GI1Gc5yurq7QaKw/Oyciqk1y9UasjIiGDhqsM/bGemMvDG7jAz9X6yo+WBomKlZq586d5jEkjo6OaNasGTZt2oSQkBDExMRYrDtjxgycPn0aY8aMgSRJGDduHF5//XX8/vvvQMFdPZcvX8bq1auRnJwMHx8fzJw5EzNmzCh3XLdv38Yvv+Rf43wwBuaBffv2ISQk5CneNRERVbTNp24jKfPhUAATFJjes7GsMZWHJIQQcgfxpNLT0+Hs7Iy0tLRiYyRyc3MRHR2N+vXrw8bGRrYYqXz4uRERVRyjSaDPgnBEJz2c1qJbI3d8P1Xeuj6P+v4uij0qRERENdTuSwkYnboMHuo0LDEMxjVRDzN6NpA7rHJhokJERFQDCSGwZt9ZLFX+AXtJh+cVR/FKnbXo1qiEO1qtWO28PZmIiKiGOxadgnYJP8O+oPjgT8YemBDSymqLD5aGiQoREVENtDzsMiardgIFxQe3243AoNZPN9GnHJioEBER1TBXEjLgFvUzPAqKD/5u6ozne3ax6uKDpal+ERMREdEjLQ2/hmmFig+uUw3H6E7WXXywNExUiIiIapD4tBxkn/sFDRT5k3FGGFuiY5fesNNUz/tnmKgQERHVICsO3MD0QsUHV2AoJgYHyBrT02CiQkREVEOk5ehx5dgutFPk13uLNPnDt/0guDlo5Q7tiTFRqUUkScLWrVvlDoOIiCrJD0djIQw6RJny7+75zjgYU3tUrwneimKiYmWGDBmCAQMGlPjcgQMHIEkSzp0790T7jo+Px8CBA58yQiIiskb5xQdjcMDUBn3y/o3QvLdhaDECAW72cof2VJioWJnQ0FDs3r0bt27dKvbcypUr0bFjR7Rp06Zc+8zLyy9G5e3tDa22+nb/ERFR6baevo17GfmTuwkosMfUAdN6NpE7rKfGRMXKDB48GB4eHli1apXF8szMTGzatAnDhw/HuHHjULduXdjZ2aF169b48ccfLdYNCQnBrFmzMGfOHLi7u6N///5ACZd+3nvvPTRp0gR2dnZo0KABPvroI+j1evPzn376Kdq1a4e1a9ciMDAQzs7OGDt2LDIyMszrmEwm/Otf/0KjRo2g1Wrh7++Pzz//3Px8XFwcRo8eDRcXF7i6umLYsGHFqj8TEdHTMZkEvtt/w2JZl4ZuaFPPRbaYKgoTFSujUqkwYcIErFq1CoULW2/atAlGoxEvv/wyOnTogN9++w0XLlzA9OnT8corr+DYsWMW+1m9ejU0Gg0iIiLw7bfflvhajo6OWLVqFS5duoSvvvoKS5cuxcKFCy3WiYqKwtatW7F9+3Zs374d4eHhmD9/vvn5uXPnYv78+fjoo49w6dIlrFu3Dl5eXgAAvV6P/v37w9HREQcOHEBERAQcHBwwYMAAcy8PERE9vd2Rd9EyZTeaSTfNy2b0bChrTBVFEoW/DauZR5WJzs3NRXR0NOrXrw8bGxuL51ZfXI01l9Y8dv8tXFvgf73/Z7HsjT1v4FLKpcduO6HFBExsObHM76Wwy5cvo3nz5ti3bx9CQkIAAD169EBAQADWrl1bbP3BgwejWbNm+PLLL4GCHpX09HScOnXKYj1JkrBlyxYMHz68xNf98ssvsX79epw4cQIo6FH597//jYSEBDg6OgIA3n33Xezfvx9HjhxBRkYGPDw88M0332Dq1KnF9vf999/jH//4ByIjI821JfLy8uDi4oKtW7eiX79+xbZ51OdGRETFCSHw8qI/sOTeBDhIufjD2AELXD/B73N6WG1dn0d9fxdVPWd/eUpZ+iwkZic+dj1ve+9iy1J0KWXaNkuf9cTxNWvWDF26dMGKFSsQEhKC69ev48CBA/j73/8Oo9GIf/7zn9i4cSNu376NvLw86HQ62NnZWeyjQ4cOj32dDRs24Ouvv0ZUVBQyMzNhMBiKnTCBgYHmJAUAfHx8kJiY//4jIyOh0+nQu3fvEvd/9uxZXL9+3WJ7FCQjUVFR5TomRERUshOx99EmfjMc1LkAgEThgldDGlltklJetTJRsVfbw9PO87HruWpdS1xWlm3t1U83yjo0NBRvvPEGFi1ahJUrV6Jhw4bo2bMnvvjiC3z11Vf473//i9atW8Pe3h5z5swpdinF3v7Rr3/48GGMHz8en332Gfr37w9nZ2esX78e//nPfyzWU6vVFm1JkmAymQAAtra2j3yNzMxMdOjQAT/88EOx5zw8PB57DIiI6PGW74vE3wuKD5qEhF/sRuGHNtWv+GBpamWiMrHlxCe+LFP0UlBlGT16NN58802sW7cOa9aswWuvvQZJkhAREYFhw4bh5ZdfBgoGs169ehUtWrQo1/4PHTqEgIAAfPjhh+ZlsbGx5dpH48aNYWtriz179pR46ad9+/bYsGEDPD09H9u1R0RE5Xftbgacr2+BpzoVAPC7qRMG9OgCdTUsPliamvNOahgHBweMGTMGc+fORXx8PCZNmgQUJAe7d+/GoUOHEBkZiRkzZuDu3bvl3n/jxo1x8+ZNrF+/HlFRUfj666+xZcuWcu3DxsYG7733Ht59912sWbMGUVFROHLkCJYvXw4AGD9+PNzd3TFs2DAcOHAA0dHRCAsLw+zZs0u8/ZqIiMpnafh1TFduN7d/UI7AmGpafLA0TFSsWGhoKO7fv4/+/fvD19cXAPC3v/0N7du3R//+/RESEgJvb+9SB8c+ytChQ/GXv/wFs2bNQrt27XDo0CF89NFH5d7PRx99hLfffhsff/wxmjdvjjFjxpjHsNjZ2WH//v3w9/fHyJEj0bx5c4SGhiI3N5c9LERETykhLRcZ535BQ0U8AOCIqTk6dOkNe23NulhSK+/6IevFz42IqGzm/XYJ/Y++gvaK6wCAacb3MO+9t+FeDer6lOeuH/aoEBERVTPpuXpcOrbbnKRcNvnBM2hwtUhSyouJChERUTWz7uhN9DEeMLeXGgdhWo+aMcFbUTXrQhYREVENpzMYseJgNO4ZJiLc1BajlPuR13wEAt2rd/HB0jBRISIiqka2nb6DxAwdAAX2mtpjr6k9tvVsJndYlabGX/qpxmOFayV+XkREpTOZBJbst5zZ+9kGrmjrV/2LD5amxiYqD2ZUzc7OljsUKocHn1fRGXGJiAjYczkRaffuWCyrKcUHS1NjL/0olUq4uLhYzOlRU+oe1ERCCGRnZyMxMREuLi5QKpVyh0REZHXW7juHvdq3cM7UAIuNQ5Hk0QUhTWp2SZIam6gAgLd3flHBB8kKWT8XFxfz50ZERA+diElBszs/w0mdg27Ki7gpPGHT8+Ua/0d4jU5UJEmCj48PPD09odfr5Q6HHkOtVrMnhYioFMvDL+OTwsUHbUdibVtfucOqdDU6UXlAqVTyC5CIiKqt64mZcLi6Bd7q+wCAP0wd0adHtxpVfLA0Nf8dEhERVXPL91/HdOVv5vb3ymEY+4y/rDFVFSYqREREViwxPRcpZ7ajseI2AOCoqRnaBveFQw0rPlgaJipERERWbOWhGExVbDO3l4shmNglUNaYqhITFSIiIiuVkavHhSO70ElxFQBw1VQX7u2GwNOx9lSXZ6JCRERkpdYfi8Mrxoe9Kd8ZB2NqDS0+WJracYGLiIiomskzmLD8YDSaGPvAATmor0hATtORaODhIHdoVYqJChERkRX65ewdJKTnIgFtsd/UFnWQjhUhTeUOq8rx0g8REZGVMZkEvitSfLBx/UAE+deRLSa5MFEhIiKyMmFXE3H1bqbFshk9GsgWj5yYqBAREVmZ1fvOY7n63whRnAYg0NjTAb2aesodliw4RoWIiMiKnLp5H01u/Yze6tPorTyNrw3D4dPjH1AoanbxwdJYTY/K/PnzIUkS5syZI3coREREslkedgVTCooPAsBB214Y1q6urDHJySoSlePHj2PJkiVo06aN3KEQERHJ5sa9TNhc2QIfKQUA8IexA/p07w6Nyiq+rmUh+zvPzMzE+PHjsXTpUtSpU/tGMxMRET2wbH8Upiu3m9trFMMxrpYUHyyN7InKzJkzMWjQIPTp0+ex6+p0OqSnp1s8iIiIaoLEjFwkndmOpopbAIDjpiZo9Ww/ONqo5Q5NVrIOpl2/fj1OnTqF48ePl2n9efPm4bPPPqv0uIiIiKra6kMxCJV+MbeXm4bis661p/hgaWTrUYmLi8Obb76JH374ATY2ZSuuNHfuXKSlpZkfcXFxlR4nERFRZcvUGXD28G50VlwGAFw3+cKl7RB4OdWe4oOlka1H5eTJk0hMTET79u3Ny4xGI/bv349vvvkGOp0OSqXSYhutVgutVitDtERERJVn/bGbGG/cBhR87S0xDsaMno3kDssqyJao9O7dG+fPn7dYNnnyZDRr1gzvvfdesSSFiIioJtIbTVh14Bo+hw4AcFe4ILPJCDTyrF3FB0sjW6Li6OiIVq1aWSyzt7eHm5tbseVEREQ11a9n7+BWugET8T5aGmLgLSXj9ZBmcodlNTgzLRERkUyEEFgSfsPcvigCYesfhA4BrrLGZU2sKlEJCwuTOwQiIqIqE3b1Hq7czbBYNqNnQ9nisUayz6NCRERUW63edx59FCchwQQAaOhhj97NamfxwdIwUSEiIpLBmbhUNIzbjGWa/2C35l08I0ViRo+Gtbb4YGmYqBAREclgWfgVhKp2AAAaKe5AYe+KYUG+codldZioEBERVbGYpCxoIrfAt6D44J/GIIR07wmtilNzFMVEhYiIqIot3R+FaYWKD65WjMBLnWt38cHSMFEhIiKqQvcydEg4/RuaK/LLwJw0NUaLZ/rCqZYXHywNExUiIqIqtOZwDELxsPjgMtMQTO7WQNaYrBkTFSIioiqSpTPg5KE96KK8BACIMvnAsc1QeDuz+GBpmKgQERFVkQ3H4/CScau5/Z1xMKax+OAjMVEhIiKqAnqjCVsPnMRAxTEAQKJwQVqjEWjs5Sh3aFbNqqbQJyIiqql+OxePc2m2GCTNwzTVdlw2+SO0V3O5w7J6TFSIiIgqmRAC34ZHAQAuC3+8rX8d7f1d8EFAHblDs3q89ENERFTJ9l9LwuWE4sUHJYnT5T8OExUiIqJKtizsMhQFhQcBoIG7Pfo295I1puqCiQoREVElOncrFY1iN+BPzTsYp9wDLfIwvUcDFh8sIyYqRERElWhp+FWEqn5HA0UC5qmXI8jhPoYH1ZU7rGqDiQoREVEliU3OgurSFtSTkgAAe4xB6NGtB2zULD5YVkxUiIiIKsmy/TcwvXDxQWkoxncOkDWm6oaJChERUSVIztThzqnf0FxxEwBw2tQITZ/pD2dbFh8sDyYqRERElWDN4VhMKVR8cKlxCKZ0Z/HB8mKiQkREVMGy8ww4cWgPuiovAgBumLxh12YofJxt5Q6t2mGiQkREVME2nbiFcYaHxQeXGgdhWs/GssZUXTFRISIiqkAGowm/hR/CQMVRAMA94YTkhqPQ1JvFB58EExUiIqIKtONCAm6l6fGzsQfyhBKrDAMQGtJM7rCqLSYqREREFUQIgSXhUbgDd7xrmIHuuq9w2vtFPFPfVe7Qqi0mKkRERBUk4noyLt5JN7fvwhUTerVm8cGnwESFiIiogizZH2XRru9uj74tvGWLpyZgokJERFQBLtxOQ52obXhZuRs20AEApnVvACWLDz4VldwBEBER1QTLwq/ir6qN8FPcwxzVzxit+goj27P44NNijwoREdFTikvJhri0DX6KewCAC6b6GNWtDYsPVgAmKkRERE9p+YEbmKb41dxeJQ3Fyyw+WCGYqBARET2FlKw8xJ7YgVaKGADAWVMDNOw0EM52LD5YEZioEBERPYW1h2MxmcUHKw0TFSIioieUk2fEkUN70UN5HgAQa/KEpvUw1HVh8cGKwkSFiIjoCf10Mg5j9Sw+WJmYqBARET0Bg9GE7eFHMEhxBACQJJyQUH8Umvs4yR1ajcJEhYiI6AnsvJgAz/TzMBZ8la429MOUXs3lDqvG4YRvRERE5ZRffPAGzpu64IiuBSao/sApr1F4q4Gb3KHVOOxRISIiKqfDUck4fzsNAHAPLviPYTReCgli8cFKwESFiIionL7df8Oi7e9qhwGtWHywMjBRISIiKodLd9Jx99pJ2CLXvGxaDxYfrCwco0JERFQOS8Ov4jv1AjhJ2Vhj7Ie1mrF4sUM9ucOqsZioEBERldGt+9kwXNyGAFUiACBIugapawMWH6xEvPRDRERURkWLD67EMLzyLIsPViYmKkRERGVwPysP0cd3oo0iGgBw3hSIgI4DUcdeI3doNRoTFSIiojL4/kgsJmObub3UNBShLD5Y6ZioEBERPUau3oiIiDD0VJ4DANw0eUDZcij8XO3kDq3GY6JCRET0GD+dvIUx+i3m9lLjIEzt2UTWmGoLJipERESPYDQJ/BJ+BEMUhwEAKcIBdwJHoqWvs9yh1QpMVIiIiB5h18UE9MvYApVkAgCsNvTHlF4t5Q6r1uA8KkRERKXILz4YhVuGociCDV5UhuO450jMacjig1WFPSpERESlOHIjBWdvpSEZzlhoeBHddF9jXEh7Fh+sQkxUiIiISrFkf5RFu66rPQay+GCVYqJCRERUgsj4dIRdSbRYNq17A6iU/OqsSrIe7cWLF6NNmzZwcnKCk5MTgoOD8fvvv8sZEhEREQBgWfg1bNN8hHdUG+CBVNSxU+PFDn5yh1XryDqYtl69epg/fz4aN24MIQRWr16NYcOG4fTp02jZkiOqiYhIHrdTc6A7/wvaqm+greIGWkkxONNlGWw1LD5Y1WRNVIYMGWLR/vzzz7F48WIcOXKEiQoREclmxYEbmKr8xdxei+fx7+BAWWOqrazm9mSj0YhNmzYhKysLwcHBJa6j0+mg0+nM7fT09CqMkIiIaoO0bD2uHd+FdoobAIBLpgDU6zAIriw+KAvZRwSdP38eDg4O0Gq1ePXVV7Flyxa0aNGixHXnzZsHZ2dn88PPj9cKiYioYn1/NBaTxMPig98ZB2Nqj4ayxlSbyZ6oNG3aFGfOnMHRo0fx2muvYeLEibh06VKJ686dOxdpaWnmR1xcXJXHS0RENVeu3oj9B8PxnPIMAOCWcIdoOYLFB2Uk+6UfjUaDRo0aAQA6dOiA48eP46uvvsKSJUuKravVaqHVamWIkoiIaoMtp29jdN4WoGDM7DLD85jG4oOykr1HpSiTyWQxDoWIiKgqGE0CW8OOYqjiEADgvnBAXMAotKrL4oNykrVHZe7cuRg4cCD8/f2RkZGBdevWISwsDLt27ZIzLCIiqoV2X7qLPumboVYZAQBrjH0xicUHZSdropKYmIgJEyYgPj4ezs7OaNOmDXbt2oW+ffvKGRYREdUyQgh8Gx6FjqIOkoUj7JGLI26jMLuRu9yh1XqyJirLly+X8+WJiIgAAMdj7uNMXCrOYBC+N/ZBW+kGXurF4oPWwOrGqBAREVW1JeEPiw/mQotbzu0xqLWPrDFRPiYqRERUq129m4E9l4sWH6zP4oNWgp8CERHVakvDr+F15VZ4IQUA4GKnxuhOnFDUWjBRISKiWis+LQdZ537Fu+qNOKB9E6HK3zAhOBB2GtmnGaMCTFSIiKjWWnkwGtMU+cUHNZIRsQo/TAwOkDssKoSJChER1UppOXpcPvYHghTXAQCRJj/4tB8MNwfOgG5NmKgQEVGttO7oTUwwbTW3vzMOwdQeDWSNiYpjokJERLWOzmBE2MFw9FGeBgDcFm4wtBiBADd7uUOjIsqdqOj1eqhUKly4cKFyIiIiIqpkW0/fxgu5W8zt5Sw+aLXKnaio1Wr4+/vDaDRWTkRERESVyGQS+DnsGIYpIwAAqcIe0f6j0Kaei9yhUQme6NLPhx9+iA8++AApKSkVHxEREVEl+jPyLvqkbYZGyv+De62xLyb1aiV3WFSKJ7pR/JtvvsH169fh6+uLgIAA2NtbXtM7depURcVHRERUoZaGX8MXipMAAJ1QI8J1JGY1ZvFBa/VEicrw4cMrPhIiIqJKdiImBcdvpmMg5mOE8iDckYaxvTqy+KAVe6JE5ZNPPqn4SIiIiCrZt+E3AAA6aLDe+Bx8nW0Q3obFB63ZU80RfPLkSURGRgIAWrZsiaCgoIqKi4iIqEJdT8zAn5F3LZaFdm8ANYsPWrUnSlQSExMxduxYhIWFwcUlf5R0amoqevXqhfXr18PDw6Oi4yQiInoqS8OvwxvJSIAbAMDZVo2xLD5o9Z4ojXzjjTeQkZGBixcvIiUlBSkpKbhw4QLS09Mxe/bsio+SiIjoKdxNz0Xa2V9xQDsH/1H/PzSQ7uCVZwNgr2XxQWv3RJ/Qzp078eeff6J58+bmZS1atMCiRYvQr1+/ioyPiIjoqa2IiMZUxa9QS0aMUh7ETnTFxC6BcodFZfBEPSomkwlqtbrYcrVaDZPJVBFxERERVYj0XD0uHfkDHRVXAQCXTX7wCBoMD0cWH6wOnihRee655/Dmm2/izp075mW3b9/GX/7yF/Tu3bsi4yMiInoqPx69iQmmbeb2UuMgTOvRUNaYqOyeKFH55ptvkJ6ejsDAQDRs2BANGzZE/fr1kZ6ejv/9738VHyUREdET0BmM2HvwAPoq8yd4uyNcoWs2AvXdWXywuniiMSp+fn44deoU/vzzT1y+fBkA0Lx5c/Tp06ei4yMiInpi287cwcicLeZvu+WGgQjt2VTusKgcyp2o6PV62Nra4syZM+jbty/69u1bOZERERE9hfzig8exVnkAAJAu7HC93igE+deROzQqB1ZPJiKiGmnv5UT0Si1cfLAPJvVqLXdYVE6snkxERDXS2rBzeEn5JwBAJ1Q4UOcFhDTlhKTVDasnExFRjXMyNgWHbmbj/5SvYIZyO46ZmuLFkA4sPlgNsXoyERHVOEvCb0APFTYZQ/CTsQcaOEn4va2v3GHREyh3omIwGCBJEqZMmYJ69epVTlRERERP6HpiJnYXKj4ooMC47s2hUbH4YHVU7k9NpVLh3//+NwwGQ+VERERE9BSWHbgBIR62HW1UGPuMv5wh0VN44plpw8PDKz4aIiKip5CYnovk09vxlfobtJRiAACvPBsABxYfrLae6JMbOHAg3n//fZw/fx4dOnQoNph26NChFRUfERFRma08FIOpim3orLiMYcpDGGv4DJO6srRLdfZEicrrr78OAFiwYEGx5yRJ4hwrRERU5TJy9Th/ZDfeU+TPmH7NVBcNgkLg6Wgjd2j0FJ4oUWGFZCIisjbrj8XhZeM2QJnf/s44CK/2aCR3WPSUyjVG5fnnn0daWpq5PX/+fKSmpprbycnJaNGiRcVGSERE9Bh5BhN2HziIfooTAIAEUQfZTUegoYeD3KHRUypXorJr1y7odDpz+5///KfF7LQGgwFXrlyp2AiJiIge49ezdzA8ZzMUUv7tPssNAxEa0lzusKgClCtREYXv9yqhTUREVNWEENgYdgKjzMUHbXG17ii0Z/HBGoGz3xARUbUWduUeetz/GVopf36vH4x9MIHFB2uMciUqkiQVq5PAuglERCSnVfvO45VCxQfDXUaiV1NPucOiClKuu36EEJg0aRK0Wi0AIDc3F6+++qp5HpXC41eIiIgq2+mb93H75nXcUbvBScrGFmM3jArpBIWCf0TXFOVKVCZOnGjRfvnll4utM2HChKePioiIqAy+238D10U9DMibjxDFGWTYB+DHdnXlDosqULkSlZUrV1ZeJEREROUQnZSFnRcTCloSwkxB+KB7MxYfrGH4aRIRUbW0tGjxQa0K41h8sMZhokJERNXOvQwdLp8MR2vphnnZ+GcD4GijljUuqngsJ0lERNXO6kMxeFfxPZ5VReKQsQX+YpqDyV0D5Q6LKgF7VIiIqFrJ0hlw+vBuPKuIBAB4Sqno1a4pvJxYfLAmYqJCRETVyvrjcRhv3GZuf2cchKk9WXywpmKiQkRE1YbeaMKu8IMYoDgOAEgULshoMhKNPFl8sKZiokJERNXG9nN3MCxni7n44ArDAEwNaSZ3WFSJmKgQEVG1IITAhn0n8YJyPwAgQ9jiku8odAhwlTs0qkRMVIiIqFoIv3oPXVM2QyvpAQDrjM/hlV5t5Q6LKhkTFSIiqhZW7buICco/AAB5Qom9ziPRuxmLD9Z0TFSIiMjqnY1LhXvc73CWsgEA24xdMSqkM4sP1gKc8I2IiKzed/tv4DdjD9wVdTBD+St+thmB1UG+codFVYCJChERWbWYpCz8fiEegIQDpjY4YGqD9/s0g1allDs0qgK89ENERFZt2cEbMBUqPuigVeGlziw+WFvImqjMmzcPnTp1gqOjIzw9PTF8+HBcuXJFzpCIiMiKJGXq8NuJ6xbLxnf2hxOLD9YasiYq4eHhmDlzJo4cOYLdu3dDr9ejX79+yMrKkjMsIiKyEmsOxWCR9C9s0PwdzylOQaMUmNy1vtxhURWSdYzKzp07LdqrVq2Cp6cnTp48iR49ehRbX6fTQafTmdvp6elVEicREVW9LJ0BJw7twVvKSwAAd1UalrQaBG9nFh+sTaxqjEpaWhoAwNW15FkG582bB2dnZ/PDz8+viiMkIqKqsvFEHF4ybjW3lxoHYVrPxrLGRFXPahIVk8mEOXPmoGvXrmjVqlWJ68ydOxdpaWnmR1xcXJXHSURElU9vNOH38EMYqDgGALgnnHG/4Ug09nKUOzSqYlZze/LMmTNx4cIFHDx4sNR1tFottFptlcZFRERVb8f5eAzJ3gylKv92n5WGAQjt1VzusEgGVtGjMmvWLGzfvh379u1DvXr15A6HiIhkJITAj/tO4UVlOAAgS2hxzmckOgXWkTs0koGsiYoQArNmzcKWLVuwd+9e1K/PkdxERLXdgWtJCE7+GTYFxQd/ND6Hl0PaQZI4XX5tJOuln5kzZ2LdunXYtm0bHB0dkZCQAABwdnaGra2tnKEREZFMVoVdwIKC4oN6ocSfTqPwQwsvucMimcjao7J48WKkpaUhJCQEPj4+5seGDRvkDIuIiGRy/lYa/GM3w0XKn0/rF1MwhoV0hpLFB2stWXtUhBBlWIuIiGqLJfujcMfUAOHGNuipPIeNmpFYHVRX7rBIRlZz1w8REdVuN5OzseN8PEyiCSbq34ef4S7G9usBGzWLD9ZmVnHXDxERUdHigylqX7zcOUDOkMgKMFEhIiLZJWfqsPGE5SSe457xh7Mdiw/WdkxUiIhIdmsOx+JtsQb9FMchwQSVQsKUbpyygjhGhYiIZJadZ8CxQ3vxo2oHpmEH9hiD8Fvr/8LXhdNUEHtUiIhIZptO3MI4w8Pig3tM7TG9RwNZYyLrwUSFiIhkYzCa8Fv4IQxSHAEA3BNOSGowAs28neQOjawEExUiIpLNjgsJGJS1GUop/3afVYYBmNKrhdxhkRVhokJERLIQQmB92CmMLlR88Iz3KHSu7yp3aGRFmKgQEZEsDkUl45l7P8NWygMArDc+h5dD2rL4IFlgokJERLJYue8CJhQUHzQIBXY5jkS/lt5yh0VWhokKERFVuYt30uAbsxmuUiYA4FcWH6RSMFEhIqIq993+G2gi3TK3N2pGYFT7erLGRNaJE74REVGVikvJxvZz8dhmCsUaYz/0VJxF1x4hLD5IJWKiQkREVWr5wWgYC6oPXhV+uKUMxKFnWXyQSsZLP0REVGXuZ+Vhw3HL4oNjO/nDxU4jW0xk3ZioEBFRlVl7JBZtjeehgAkAoFRICO3O4oNUOl76ISKiKpGrN+JwxF6s1/wD0SYvfGkYA03bUajL4oP0CExUiIioSmw6eQtj9FsBJVBfcRd1pAyMZ/FBegxe+iEiokpnNAn8GnYYgwuKDyYLR9ytPxLNfVh8kB6NiQoREVW6nRcSMDBzM1RS/tiU1Yb+mMzig1QGTFSIiKhSCSHww77TGKMMAwBkCy1Oeo1CcAM3uUOjaoCJChERVarDN5LRIfFn2Ek6AMAGYwheCgli8UEqEyYqRERUqVbsi8Qk1S6goPjg7w4jMaAViw9S2TBRISKiShMZnw7v6J/hJmUAAH4zPYshIcEsPkhlxkSFiIgqzXfhUQhV7jC3N6iH48UOLD5IZcd5VIiIqFLcup+NX87F45x4B1OVO+AhpeLZ7s+x+CCVCxMVIiKqFCsOxsBoEohCXcw1TIOdWkIEiw9SOfHSDxERVbjU7DysP37TYtnoTgGoY8/ig1Q+TFSIiKjCfX8kFjl5enNbqZAQ2o3FB6n8mKgQEVGFytUbcTAiDOGav2CCchdsoMPgNj7wc7WTOzSqhjhGhYiIKtTPp25hdN5W+Cvv4e+K1TBBgXE9essdFlVT7FEhIqIKYzQJ/BJ2BEMVhwAAKcIBCYHD0dLXWe7QqJpiokJERBXmj4sJ6JexxVx8cI2xHyb3aiV3WFSNMVEhIqIKIYTA9/vOYKxyLwAgR2hwzGMUujRk8UF6ckxUiIioQhyNTkG7u5thX1B8cKOxJ8aFtGfxQXoqTFSIiKhCrNh3CZNUOwEARiHhN4eRGMjig/SUmKgQEdFTu5yQDvcbW+AhpQMAdpg6Y3DPLlAp+TVDT4dnEBERPbWl4dcwTbnd3P5RNRwvdvCTNSaqGZioEBHRU7mTmoM9Z6NxyNQKOqHGQWNLdOrSG7YaFh+kp8cJ34iI6KmsOBiNVJMtPjSFYqHhBXioc/B9MIsPUsVgjwoRET2xtGw9fjz2sPhgEpzRqWNnuDloZY2Lag4mKkRE9MS+PxqLrDyjua2QgKndGsgaE9UsTFSIiOiJ5OqNCD+4H1OUv8MOuQCA51v7wN+NxQep4jBRISKiJ7Ll9G2MyduMj9VrcUj7BtpLVzGjR0O5w6IahokKERGVm9EksDXsqLn4oIAEl/pBaF2PxQepYjFRISKictt96S76pG+GWsofn7LG2BeTQlrKHRbVQExUiIioXIQQ+D7sDMYVFB/MFWocdhuF7o3d5Q6NaiAmKkREVC4nYu+jTfxmOEj5A2g3GXtiXC8WH6TKwUSFiIjKZfm+SEwuVHzwV7uReL61j9xhUQ3FRIWIiMrs2t0MuFzfDA8pDQDwu+kZDOzZBWoWH6RKwjOLiIjKbGn4dUxT/mZu/6gajjGdWHyQKg8TFSIiKpOEtFxcOnsEvlIyAOCwsQU6BPeGnYZl46jy8OwiIqIyWRkRjQtGf3Qxfo0Jyt04o2iG/3QJlDssquFk7VHZv38/hgwZAl9fX0iShK1bt8oZDhERlSI9V48fjuYXH7wPJ3xlHAW/DgPhzuKDVMlkTVSysrLQtm1bLFq0SM4wiIjoMdYdvYlMncHcZvFBqiqyXvoZOHAgBg4cKGcIRET0GDqDEVsPnIYDBDKRX3BwYCsfBLrbyx0a1QLVaoyKTqeDTqczt9PT02WNh4ioNth2+g6m5K7BAO0x/GDsg8WGoZjeg70pVDWq1V0/8+bNg7Ozs/nh58db4oiIKpPJJPBz2DEMVx6Ek5SDl5R70CHQHW39XOQOjWqJapWozJ07F2lpaeZHXFyc3CEREdVoey4nolfaZmgKig+uNfbFxF6t5A6LapFqdelHq9VCq+UIcyKiqrJ23zksUu4BAOiEGhGuozCziYfcYVEtUq16VIiIqOqciElB8zs/w1HKAQD8bOyO0b06sPggVSlZe1QyMzNx/fp1czs6OhpnzpyBq6sr/P395QyNiKjWWx5+GZ+qfgcAmISEbbYj8X0bX7nDolpG1kTlxIkT6NWrl7n91ltvAQAmTpyIVatWyRgZEVHtdj0xEw5Xt8BLnQoA2GXqiH49u7H4IFU5WROVkJAQCCHkDIGIiEqwLPw6Zii3m9s/KEdgCYsPkgyYGhMRkYW76blIPfsrGinuAACOmpqhXXAf2Gur1f0XVEMwUSEiIgsrI2Jw2NAEX+pfRJJwwjIxDBNZfJBkwkSFiIjMMnL1+OFILNLggG+MI9BV9zU8ggbDw5FTQ5A8mKgQEZHZj8duIqNQ8cE8SYNpPRrKGhPVbkxUiIgIAJBnMGH5wWiLZQNaeqM+iw+SjDgyioiIAADbztzG9OxlMKoUWGEYgAS4sfggyY6JChERFRQfPI7Vyt3QSgaMVB7AbJ8fEORfR+7QqJbjpR8iIsK+K4nomboZWil/fMpGYwimhTSTOywiJipERASs2Xce45V/AgB0QoUDdUYhpCmLD5L8mKgQEdVyJ2Pvo8ntn+FUUHxws7E7XgjpyOKDZBWYqBAR1XLLwy8jtFDxwa22IzGkLYsPknVgokJEVItF3cuE3ZUt8JbuAwB2mzqgb49u0Kj49UDWgWciEVEttnz/dUwvVHxwrXI4xj7jL2tMRIUxUSEiqqUSM3KRdHo7mihuAwCOmZqizbN94cDig2RFmKgQEdVSqyJioDXlIFG4AACWm4ZiUlcWHyTrwrSZiKgWytQZsPZILDJMXbBL1wkDFMfh1mEwPB1t5A6NyAITFSKiWmj9sZvIyM2f3C0PavwquuDPHo3kDouoGF76ISKqZUoqPtivhRcaejjIFhNRaZioEBHVMr+evYMOGftQF/fMy2b0bChrTESl4aUfIqJaRAiBjWEnsEa9GEqY8KPxOfxa7x20Z/FBslJMVIiIapGwK/fQ4/7P0Kryx6dkwRbTezSQOyyiUvHSDxFRLbJq33m8UlB8ME8osc9lJJ5r5il3WESlYqJCRFRLnIlLRaNbP8NJygYAbDF2x6iQTlAoWHyQrBcTFSKiWqJw8UEA2Gw7AsPasfggWTcmKkREtUBMUhY0kVvgK6UAAHYb2+O5bt2hVSnlDo3okTiYlqgEQgjoDCbk5BmRo89/5BY8cvJMD5flGZFrMFqul/dg/fz1dHl5UOgyYNJnQ9Jnw6Syg4NbPfh7OCDA1R6Bbnbwd7ODj7MtlOyCp0qydH8Upil/M7fXKobjm84sPkjWj4kKVSsmU0EC8SBp0OcnCRb/N5jMycLDZcZCy0zI0Rlg1OdA5GXDpM8B9NmQ8nIgGfIfCQYHRIoAi9d+XbkNTlI2bKCDLfJgI+XBBTrYIA+2Uv4yW+jwf4aXEWYKMm/XXrqKzdpPLfalS1Uj9ronYoUXLgkv/C68cFvyQbxLEHzcXeHvaodANzsEuNkjwM0O9erYQaNiByg9mXsZOtw9vR3NlHEAgJOmxmj+bF842ajlDo3osZioUIUonEAUTh4KJxDm5KIgmSiaYDzojTDqc2DKy4bIy++BiDW4Is2gMvdS+El38awiErYFCYOtlJ8s2CDP3K6DPNhBjVn62RZx/lO1FMOVh2CDPCgkUfKbUQI/oxve1r9usThUtQNuUsZjj4UrLNfJgbbYOlpJjybSbTTBbYvlXZK+RmSSwdzuKF1GW0UU4uCFHIcAKN3rw9fdFQGuD5OYADc72Gn4o0ylW3M4BsMRZm4vMw3Bp914SzJVD/ztVsOZTMLi0kSu3lRi8pBrbptKWGaELk+PPL0OGXqlxT78867DzpAKlTG3oJfhYc+CjfQgedAhzNQO+wr1MjgiGz9q/mFe72HSkVfsPQzV/R9ixMNZM9tJUfi3+rvHvvd0YVtsmRICdpLusdvaongc+QlH6YmKXiiRA02xBOi+cMBeYzvkQoNcaOCIbARIdxEgJUIr6c3r6YQa8XC12LaP8hReVW0vWAHAbSD+lituCk/EmLzxq/DETeGFe3YNYHJvBv9Cl5ICCxIZFzvNY98v1VxZOgPWHI5Ftn4m9hmD0F95HE5th8LLicUHqXpgoiKT4gnEw7EPuXrL3oeHYyRMxZbpCv7V63Jhq0sGDNlAXg4kYw4UhhyojLkFlyYeJg0rjQOgw8Mvr+cVRzBCGZF/SaMgaSh8OePB9qdNjTAi7+8W7+MbzVJ0UF4DHjMeL81gb5Go6KFEK0VMmY5V0aQhF2X74rUpIdm4LdwRafJDLrTIFRrkIP+RCy1yhAY50CIHGlw1+RXb9q+G12GjUsCksgHUtoDaDtDYQlLZQqm1g0ajha1aCRuNEqFqJWzUivy2Wol4TU/z/5Ny9DiRnIW4pExkJN2C8v4NeBnj4YQsiCLj2wOku8Xi8JFS4COloLPisnnZHl0QQmP+iuMx983LXlH+gUxhi2RNXcA1EM7udQv1wuQnNB6OWkgSx8XUZBuOxyEtRw9AhZ9NPfCzqQf+7Mnig1R9MFEpwmgSxcY+PBgUWeIlDb3RYjyEeQClLg/GvByYCsZBSIac/ATCkJ9EqI25SBR1cEZY/sL4i+on1EGGuafBrSC5yE8g8qAtSCD+T/8Kfjd1Nm/XWrqBjdq/Wb4ZJUpMIDYZe1okKv5SIvoqTz722NigeE9Ejihb0lA02dBBDZ1QIxfq/ORAaAp6HCwThlxokQJHi20vmQIwVx+KHJG/jkGhhUllC5PSFkJtm59EaOygVNugl9YJNmqlOYHIUr+N39RK2GqU0KoUsNXkP2enVsK14P82aiX6q5V4V62EjeZhsqFWDirTey2fThBCICkzDzdTstAsKRuxKdmITc5CbHI2ViWNxi5dp/weGMVdBEoJ8JcS4S6lW+zlpig6YZfA+6ofYf+g9ygFyEy2wU3hhRjhhVPCC5uFFxIUPsio0wJu7p4IdLcvGBuTn8z4unBwb3WnNxYvPtinuRcaeTqWug2RtWGiUsSKg9H4fEckAqV4hCjOWlzOsCmUNLgXXKowQImJ+vct9vGF6juMUYWV+hoPEohfjc/ijSJjKMYo98Fbul/qpg88mLDpgZwy9jIAyL+8UujqRNEeCp1QIReaYglEtPAutq9fTF1wSjRGbkHSkFOop0JfkEAIpS0ybD3QTOsIrVoJ24Kehr9odj1MIgr+tdUU/r8C9iolPihIIB48b6tWQqseWyiBqN6DTCVJgoejFh6OWnQIcC3ybFek5ehxMzkbsSlZOJycjfXJWbh77x5EcjQcsm8iQErEWWE53sAd6Q+TlAIOUi5aSLFogViL5ZOS/oo/Eh/2dvlLd9FLcQa3JG/kOQVA4x4IP3eX/CTG3Q7+rvbwc7Xlba3VwG/n4pGRmgTA3rzs1Z4cm0LVCxOVImw0+b98m0lx+FS95rHr54rio+b1j7sO8uC1ShoHITRAKX/EFk4gDMLyNVKFI7YbOxfvkRDaIr0WWmQqnOGkVuUnBBolzqgGY6JqACS1HZRaW2g16ofJQqHkwUatxD8LEoiHycWz5v2Yl2mUsFEpoKrmCYS1cLZVo3U9Z7Su51zsuZw8I+LuZyMoKQvPpWQjpqAnJiFJiWnp78APdxEgJSBASkSAlIB6UhLUktFiH7FFEtCO0hV8pl6d38gGjLES7sS4I0Z44abwwmHhhZvwQpZ9AEwezc2XkgoP8LXX8leL3IQQWL/vBA5rZ2GvKQjfGobAxr89OgYWTYaJrBt/mxRhq85PAMo8DkLSQ4LJYmxBjPDGSVNjc89C8csZ+QnEDeFjsS+NUoH3FX+BjUrKH/+gyr+E8SCBsNFoChIGBeqolXhDY9kToVf3gINaCfcivQ82TCBqLFuNEk28HNHEq3hXvt7YD7fv55gvJR1IzkZcUjpykmKhTItBPVM8AqS7uCU8LLYLUCRatJWSgJ90D364B+CCefktnTu6RX2NQ1HJ5mV9FSdggzyk2fpBcq0PNw+v/Lli3O3Ml5Vc7NQcF1MF9l9LQteUzbBX6TBEeQR3hBvq93hB7rCIyo2JShEPEpWLpgDMzpuJXGhgUNrApLSBUD0YA2EHSWMHhdoWCo0tni9IIB70NNiq38SRIsmCo1oJz4JEoXAC8Xmh/3M8AFUktVKBQHd7BLrbAyicjDwLk0ngbkYuYpKy8Y+ULMQkZ+Nmcn6PzJ7kbrild0eAdBeB0l34F/xb9HJjjMmr2Gu+pvoF7RXXASOAe0Bqoj1ihRdihRcOCi/8ILxwT+2LvDpN4erhlT9XjOvDAb6ejlrWnakgq/ZdwH+VfwAFxQf3OI/E+ubFPzMia8dEpYiQph448G4vi2SCCQTVNAqFBB9nW/g42yK4oZvFc0J0Q0pWnrknZm9yNmKTspCclAAp5Qacc28jULqLO3Artt+idym5SFlwkW6gLW5YLJ+XOA5L4oeY27bIxQhlBOIV3jC4BMLWPQD+bo4IcM+/pBToZg9fFxv2BpbRuVupqH/zZzir85PLbcauGBXyDJNAqpaYqBRhr1Xx+jrVapIkwc1BCzcHLdr71yn2fEauHrHJ2biZko0GyVnmnpibSVn4IDMUgdLdgnli8u9U8kVysbllYoTlX/YNpAT8U7284AWAvHQl4qI8cVN4Ikp4Y5/wRBzyB/dK7o0Q4O5ocYeSn6sdbNQc3PvA0vCreF+1w9z+yWYE1gTVlTUmoifFb2QiKhdHGzVa1XVGq7rFB/fm6nvh1v1sxCRl43JKNnYlZ+F2Uir0SdHQpMfCD/m3VxctTxAgJVi0NZIRDaV4NEQ8gLMPn8gBWlxbgfBrDycrC5KuwVtxH1l2flC41YeXuwcC3AtfUrKDYy2aKj42OQuqS1tQV50/duhPYxBCQnryLi2qtpioEFGFsVEr0cjTsYR5OrrBYDThTmouYpKzMC0l/3LSg8tLV1Ia4QN9aKE7lPJ7ZIrOInxPOCMbljOqjlHuw1hVGKAHkADci3c2j4v5w5Q/b0yaTV0YXRvCw8Pb4jbrQDc7uNpratTg3mX7b2C68ldze41iOP7H4oNUjTFRIaIqoVIq4F8wvX9RJlMPJGbozBPdbU/JQkxSFtKTbkO6Hw2PvDsIUCQUm7kXAAIky7uUPKQ0eEhp6IirDyc8NAE/3+2Ot2+9ZrHuUMUhpKndIeoEwtG9HvzdHS1us/Z2sqlW4zqSM3WIP7UdzQuKD542NUKzzv3gbFt7epSo5mGiQkSyUygkeDvbwNvZBp0bFB6k2wFCCKRm6/PHwaRk4y9J+ZPfxSZnIzY5G8uyByLC1NI8JiZAugsvKbXYa8QWuUtJizx8rfkmv5EK5NzX4ObV/NpJF4QXfhNeuK3whsE5ELbugajn5mRxm3XdOrZWN9ng6sOxmIJfzO1lpiH4W7f6ssZE9LSYqBCRVZMkCXXsNahjr0FQCYN7M3Uh+TP3JmfhVEo2tiRnIeFeCgzJ0bDLioU/8m+vPi6aWmznV6QnxlbKQ1PpFpriVpEXAIYn/x1/FCp34SfdRTPFbeQ6BkDtXh++7i4Wt1n7u9rBVlO1Y0Ky8wxYcygaN40hqCNlQAs97NoMhY9z8eKcRNUJExUiqtYctCq08HVCC1+nIs+EQGcwIi4lBzdTstA3KRtNC2bvvZmcjYz7TvhYP9Firhg/KRFayVDsNWKL1FLqqziFj9VrgVzAFCchIa4OYk3eiBWeOC3y/82w84PCrQE83T0s6ij5u9lVyqWYjcfjkJpjwFZ0w9a8rvBEKr7v2bjCX4eoqjFRIaIaS6tSopGnAxp5OhR7zmgSuJM6LP8SUkoWjiVn42ZSOrKS4qC8HwNvUzwCpbvwlpJxv0hhTP9C88UoJAFfpMBXmYJgXHq4kgE4Hx+IIbH/tNi2i+IC1Fo7CNcGqOPuUzAe5mFvjLtD+Qf3GowmLD1QuPighNbNmpY4YzFRdcNEhYhqJaVCgp9r/hws3eBe6JnOEELgXoYOsSnZiEnKwhsp2QWz9+bP4vuHriPSDA7mnhh/6S7cpIxirxEris8E+w/VCjQQCUAykJFkW3CHkieOCW9sEl64q/SBsU59OLj5wd/DIb8EQcEgZB/nkita/3Y+HrdTcyyWzejZsIKOFJG8mKgQERUhSRI8nWzg6WSDTiUU8UvNDinoicnGwYLbrO/dS4QpJRpO2XHm26tPCctLL0oYUU+6Z247SjloJcWgFWIsXyANeC9pGpZc6mVe5IQsdFJFQVcwuLeum1N+CQI3e/y47zS+Vn+D5YaBOCsaIcjfBZ0Ci4/nIaqOmKgQEZWTi50GLnYatPVzKfZcdp4BN1PyJ71rmJKFlwrVUUpKTcMXhrEWs/fWlZKgkkzF9lO0N6a14gaWq+YDWYAhU4Hb0e7m+WKmSPfRT3kSQ5WH8Yl+IoJ7fFCj5oah2o2JChFRBbLTqNDM2wnNvIsO7gXyDCbcut8XsSnZuJGUhX0p2biVlI7cpBio02JRV8QXJDCJuGGyrK5eeL4YlWTKnxgPiQDOm5frhRJXnLvj4xYsPkg1BxMVIqIqolEp0MDDAQ08HACLu6WDYTQJJKTnmmfsHWGuo5Q/NuaiPgCLDEMf1lGS7sJJshyX8pOxB4b27MxCqlSjMFEhIrICSoWEui62qOtiiy5FnhNCICmzF26mZCEmKRu7U7IRm5SJlIKK1nV0t6GCCWg1Av/q5CfTOyCqHExUiIisnCRJ8HDUwsNRiw4BxQf3puXooVRIcGDld6qBeFYTEVVzrOVDNZl1FaogIiIiKoSJChEREVktq0hUFi1ahMDAQNjY2KBz5844duyY3CERERGRFZA9UdmwYQPeeustfPLJJzh16hTatm2L/v37IzExsQxbExERUU0me6KyYMECTJs2DZMnT0aLFi3w7bffws7ODitWrJA7NCIiIpKZrHf95OXl4eTJk5g7d655mUKhQJ8+fXD48OFi6+t0Ouh0OnM7PT29UuNbfXE11lxa89j1Wri2wP96/89i2Rt73sCllEulbvPAhBYTMLHlRHM7S5+FoVuHlim+r5/7Gi3dWprb4XHh+PuRvz92OzuVHX4d8avFsv+c+A92RO947LY96vXAJ8GfWCwbs30MknKSHrvtWx3ewqAGg8zt6LRoTP1j6mO3A4D1g9bDw87D3N50dRO+PfvtY7cLdArE8v7LLZa9t/89nLh74rHbvtD4BbzW7jWLZb039S5TvPO7z0cn707m9vGE43j/wPtl2nbPi3ss2ovPLMZP13567HYdvTriix5fWCwL3RWKmPSYUrd54NW2r+LFJi+a2/ey72Hsb2PLFO+yfstQ37m+uf3bjd+w4OSCx27nbuuODYM3WCz77PBn2H9r/2O3fb7+83i749sWy4ZsGYJsQ/Zjt/342Y/R06+nuX0x+SJm75392O0A4Jfhv8BebW9u83dEcfwdUfN+RxQ9B6uarIlKUlISjEYjvLwsp3v28vLC5cuXi60/b948fPbZZ1UWX5Y+C4nZj78E5W3vXWxZii6lTNtm6bMs2kKIMm0HAHqj3qKda8wt07aFf9E+kJ6XXqZt03RpxZYl5SSVadtcQ65F22gylvm9GoXRop2tzy7Tto7q4mXuU3WpZdo2Q1+8Gm5Z480z5hVrl3XbkuIoy7aputRiy5Jzksu0bbbe8gveKMrx2ZgsP5tcQ9nOw5Kk6dLKtG16XvE/Uu7l3Cv281SSXKPleag36sscrxDCos3fEcXxd0TN+x1Rlp+rylSt5lGZO3cu3nrrLXM7PT0dfn6VNwujvdoennaej13PVVt8AiZXrWuZti36C0GSpDJtBwBqpeXcCTZKmzJta6eyK7bMSeNUpm2dtc7Flrnbuj92OwCwUdlYtJUKZZnfq1JSWrTt1HZl2tbN1q3YMhetS5m2LekXWFnj1Sg1xdpl3bakOMqyrYu2eIE8N1u3En+ZFmWntjwnlFI5PhuF5WdjoyrbeVjSeeOsdS7Ttk6a4nV0PGw9SvyCLcpGaXkeqpXqMr/XooX++DuiOP6OqHm/I8ryc1WZJFH0T4QqlJeXBzs7O/z0008YPny4efnEiRORmpqKbdu2PXL79PR0ODs7Iy0tDU5OxX9xERERkfUpz/e3rINpNRoNOnTogD17Hl5vM5lM2LNnD4KDg+UMjYiIiKyA7Jd+3nrrLUycOBEdO3bEM888g//+97/IysrC5MmT5Q6NiIiIZCZ7ojJmzBjcu3cPH3/8MRISEtCuXTvs3Lmz2ABbIiIiqn1kHaPytDhGhYiIqPqpNmNUiIiIiB6FiQoRERFZLSYqREREZLWYqBAREZHVYqJCREREVouJChEREVktJipERERktZioEBERkdViokJERERWS/Yp9J/Gg0l109PT5Q6FiIiIyujB93ZZJsev1olKRkYGAMDPz0/uUIiIiKicMjIy4Ozs/Mh1qnWtH5PJhDt37sDR0RGSJFXovtPT0+Hn54e4uDjWEXoMHquy47EqOx6rsuOxKjseq/KprOMlhEBGRgZ8fX2hUDx6FEq17lFRKBSoV69epb6Gk5MTT+Yy4rEqOx6rsuOxKjseq7LjsSqfyjhej+tJeYCDaYmIiMhqMVEhIiIiq8VEpRRarRaffPIJtFqt3KFYPR6rsuOxKjseq7LjsSo7HqvysYbjVa0H0xIREVHNxh4VIiIislpMVIiIiMhqMVEhIiIiq8VEhYiIiKxWrUxU9u/fjyFDhsDX1xeSJGHr1q2P3SYsLAzt27eHVqtFo0aNsGrVqiqJVW7lPVZhYWGQJKnYIyEhocpilsu8efPQqVMnODo6wtPTE8OHD8eVK1ceu92mTZvQrFkz2NjYoHXr1tixY0eVxCunJzlWq1atKnZe2djYVFnMclm8eDHatGljnnArODgYv//++yO3qY3n1APlPV619bwqav78+ZAkCXPmzHnkenKcW7UyUcnKykLbtm2xaNGiMq0fHR2NQYMGoVevXjhz5gzmzJmDqVOnYteuXZUeq9zKe6weuHLlCuLj480PT0/PSovRWoSHh2PmzJk4cuQIdu/eDb1ej379+iErK6vUbQ4dOoRx48YhNDQUp0+fxvDhwzF8+HBcuHChSmOvak9yrFAwO2bh8yo2NrbKYpZLvXr1MH/+fJw8eRInTpzAc889h2HDhuHixYslrl9bz6kHynu8UEvPq8KOHz+OJUuWoE2bNo9cT7ZzS9RyAMSWLVseuc67774rWrZsabFszJgxon///pUcnXUpy7Hat2+fACDu379fZXFZq8TERAFAhIeHl7rO6NGjxaBBgyyWde7cWcyYMaMKIrQeZTlWK1euFM7OzlUal7WqU6eOWLZsWYnP8Zwq7lHHq7afVxkZGaJx48Zi9+7domfPnuLNN98sdV25zq1a2aNSXocPH0afPn0slvXv3x+HDx+WLSZr165dO/j4+KBv376IiIiQOxxZpKWlAQBcXV1LXYfnVr6yHCsAyMzMREBAAPz8/B77V3JNZDQasX79emRlZSE4OLjEdXhOPVSW44Vafl7NnDkTgwYNKnbOlESuc6taFyWsKgkJCfDy8rJY5uXlhfT0dOTk5MDW1la22KyNj48Pvv32W3Ts2BE6nQ7Lli1DSEgIjh49ivbt28sdXpUxmUyYM2cOunbtilatWpW6XmnnVm0Y0/NAWY9V06ZNsWLFCrRp0wZpaWn48ssv0aVLF1y8eLHSi5PK7fz58wgODkZubi4cHBywZcsWtGjRosR1eU6V73jV5vNq/fr1OHXqFI4fP16m9eU6t5ioUIVq2rQpmjZtam536dIFUVFRWLhwIdauXStrbFVp5syZuHDhAg4ePCh3KFavrMcqODjY4q/iLl26oHnz5liyZAn+7//+rwoilU/Tpk1x5swZpKWl4aeffsLEiRMRHh5e6pdvbVee41Vbz6u4uDi8+eab2L17t9UPHmaiUgbe3t64e/euxbK7d+/CycmJvSll8Mwzz9SqL+xZs2Zh+/bt2L9//2P/Iivt3PL29q7kKK1DeY5VUWq1GkFBQbh+/XqlxWctNBoNGjVqBADo0KEDjh8/jq+++gpLliwptm5tP6dQzuNVVG05r06ePInExESLnm6j0Yj9+/fjm2++gU6ng1KptNhGrnOLY1TKIDg4GHv27LFYtnv37kde86SHzpw5Ax8fH7nDqHRCCMyaNQtbtmzB3r17Ub9+/cduU1vPrSc5VkUZjUacP3++VpxbRZlMJuh0uhKfq63n1KM86ngVVVvOq969e+P8+fM4c+aM+dGxY0eMHz8eZ86cKZakQM5zq1KH6lqpjIwMcfr0aXH69GkBQCxYsECcPn1axMbGCiGEeP/998Urr7xiXv/GjRvCzs5O/PWvfxWRkZFi0aJFQqlUip07d8r4LqpGeY/VwoULxdatW8W1a9fE+fPnxZtvvikUCoX4888/ZXwXVeO1114Tzs7OIiwsTMTHx5sf2dnZ5nVeeeUV8f7775vbERERQqVSiS+//FJERkaKTz75RKjVanH+/HmZ3kXVeJJj9dlnn4ldu3aJqKgocfLkSTF27FhhY2MjLl68KNO7qBrvv/++CA8PF9HR0eLcuXPi/fffF5IkiT/++EMInlPFlPd41dbzqiRF7/qxlnOrViYqD26hLfqYOHGiEEKIiRMnip49exbbpl27dkKj0YgGDRqIlStXyhR91Srvsfriiy9Ew4YNhY2NjXB1dRUhISFi7969Mr6DqlPScQJgca707NnTfOwe2Lhxo2jSpInQaDSiZcuW4rfffpMh+qr1JMdqzpw5wt/fX2g0GuHl5SWef/55cerUKZneQdWZMmWKCAgIEBqNRnh4eIjevXubv3QFz6liynu8aut5VZKiiYq1nFuSyP+lQURERGR1OEaFiIiIrBYTFSIiIrJaTFSIiIjIajFRISIiIqvFRIWIiIisFhMVIiIislpMVIiIiMhqMVEhIiIiq8VEhWqNsLAwSJKE1NTUMm8TGBiI//73v5UaV2WTJAlbt26tsP1NmjQJw4cPr7D9FZaXl4dGjRrh0KFDFbbPkJAQzJkzp8L2V5nKe76tWrUKLi4ulRqTtfj2228xZMgQucMgGTBRIaswadIkSJKEV199tdhzM2fOhCRJmDRpkiyxPU56ejo+/PBDNGvWDDY2NvD29kafPn2wefNm1MSJn7/66iusWrXK3K7IRODbb79F/fr10aVLF/MySZLMD2dnZ3Tt2hV79+6tkNezNsePH8f06dMrdJ+Fj5+9vT0aN26MSZMm4eTJkxX6OpVtypQpOHXqFA4cOCB3KFTFmKiQ1fDz88P69euRk5NjXpabm4t169bB399f1thKk5qaii5dumDNmjWYO3cuTp06hf3792PMmDF49913kZaWJneIFc7Z2blS/ooXQuCbb75BaGhosedWrlyJ+Ph4REREwN3dHYMHD8aNGzcqPAa5eXh4wM7OrsL3++D4Xbx4EYsWLUJmZiY6d+6MNWvWVPhrFaXX6ytkPxqNBi+99BK+/vrrCtkfVR9MVMhqtG/fHn5+fti8ebN52ebNm+Hv74+goCCLdXU6HWbPng1PT0/Y2NigW7duOH78uMU6O3bsQJMmTWBra4tevXohJiam2GsePHgQ3bt3h62tLfz8/DB79mxkZWWVOeYPPvgAMTExOHr0KCZOnIgWLVqgSZMmmDZtGs6cOQMHBwcAwP379zFhwgTUqVMHdnZ2GDhwIK5du2bez4Mu/O3bt6Np06aws7PDCy+8gOzsbKxevRqBgYGoU6cOZs+eDaPRaN4uMDAQ//d//4dx48bB3t4edevWxaJFix4Zc1xcHEaPHg0XFxe4urpi2LBh5mNz+fJl2NnZYd26deb1N27cCFtbW1y6dAkoculn0qRJCA8Px1dffWX+qz06OhqNGjXCl19+afG6Z86cgSRJuH79eolxnTx5ElFRURg0aFCx51xcXODt7Y1WrVph8eLFyMnJwe7duwEA4eHheOaZZ6DVauHj44P3338fBoOhxNf4+9//jlatWhVb3q5dO3z00UcW7+/LL7+Ej48P3NzcMHPmTIsv3Mr8PAtf+lmwYAFat24Ne3t7+Pn54fXXX0dmZmaJ7+1RHhy/wMBA9OvXDz/99BPGjx+PWbNm4f79++b1HvfzEB8fj0GDBsHW1hb169fHunXrisUsSRIWL16MoUOHwt7eHp9//jkAYNu2bWjfvj1sbGzQoEEDfPbZZxafU2pqKqZOnQoPDw84OTnhueeew9mzZy3ex5AhQ/DLL79Y/DFDtUCllz0kKoOJEyeKYcOGiQULFojevXubl/fu3VssXLhQDBs2zKKK5+zZs4Wvr6/YsWOHuHjxopg4caKoU6eOSE5OFkIIcfPmTaHVasVbb70lLl++LL7//nvh5eUlAIj79+8LIYS4fv26sLe3FwsXLhRXr14VERERIigoSEyaNMn8OgEBAWLhwoUlxmw0GkWdOnXE9OnTH/v+hg4dKpo3by72798vzpw5I/r37y8aNWok8vLyhBBCrFy5UqjVatG3b19x6tQpER4eLtzc3ES/fv3E6NGjxcWLF8Wvv/4qNBqNWL9+vUV8jo6OYt68eeLKlSvi66+/Fkql0qJaLACxZcsWIYQQeXl5onnz5mLKlCni3Llz4tKlS+Kll14STZs2FTqdTgghxKJFi4Szs7OIjY0VcXFxok6dOuKrr74q9lkJIURqaqoIDg4W06ZNE/Hx8SI+Pl4YDAbx+eefixYtWlgcg9mzZ4sePXqUeowWLFggmjVrVmx54fiFECIlJUUAEF9//bW4deuWsLOzE6+//rqIjIwUW7ZsEe7u7uKTTz4xr1+4ImxcXJxQKBTi2LFj5udPnTolJEkSUVFR5vfn5OQkXn31VREZGSl+/fVXYWdnJ7777rsq+TwLn28LFy4Ue/fuFdHR0WLPnj2iadOm4rXXXjM/v3LlSuHs7FzqMS3p+D1w+vRpAUBs2LBBiDL+PPTp00e0a9dOHDlyRJw8eVL07NlT2NraWsQMQHh6eooVK1aIqKgoERsbK/bv3y+cnJzEqlWrRFRUlPjjjz9EYGCg+PTTTy32PWTIEHH8+HFx9epV8fbbbws3Nzfzz7QQQmRlZQmFQiH27dv3yPdMNQsTFbIKD778EhMThVarFTExMSImJkbY2NiIe/fuWSQqmZmZQq1Wix9++MG8fV5envD19RX/+te/hBBCzJ07t9gX5XvvvWeRqISGhhZLMg4cOCAUCoXIyckR4jGJyt27dwUAsWDBgke+t6tXrwoAIiIiwrwsKSlJ2Nraio0bNwpR8IUDQFy/ft28zowZM4SdnZ3IyMgwL+vfv7+YMWOGuR0QECAGDBhg8XpjxowRAwcONLcLf1GtXbtWNG3aVJhMJvPzOp1O2Nrail27dpmXDRo0SHTv3l307t1b9OvXz2L9womKKKE0vBBC3L59WyiVSnH06FEhCj4fd3d3sWrVqlKP05tvvimee+65YssLx5+VlSVef/11oVQqxdmzZ8UHH3xQ7P0sWrRIODg4CKPRWGJ8AwcOtPiyf+ONN0RISIjF+wsICBAGg8G87MUXXxRjxowRogo+z9LONyGE2LRpk3BzczO3nyZRycnJEQDEF198IUQZfh4iIyMFAHH8+HHz89euXRMAiiUqc+bMsdhP7969xT//+U+LZWvXrhU+Pj7m13FychK5ubkW6zRs2FAsWbLEYlmdOnUeeR5RzaOSu0eHqDAPDw8MGjQIq1atghACgwYNgru7u8U6UVFR0Ov16Nq1q3mZWq3GM888g8jISABAZGQkOnfubLFdcHCwRfvs2bM4d+4cfvjhB/MyIQRMJhOio6PRvHnzR8Za1oGykZGRUKlUFvG4ubmhadOm5ngBwM7ODg0bNjS3vby8EBgYaL589GBZYmLiI99XcHBwqXeOnD17FtevX4ejo6PF8tzcXERFRZnbK1asQJMmTaBQKHDx4kVIklSm9/qAr68vBg0ahBUrVuCZZ57Br7/+Cp1OhxdffLHUbXJycmBjY1Pic+PGjYNSqUROTg48PDywfPlytGnTBp9++imCg4Mt4uvatSsyMzNx69atEsc2TZs2DVOmTMGCBQugUCiwbt06LFy40GKdli1bQqlUmts+Pj44f/48UAWfZ2F//vkn5s2bh8uXLyM9PR0GgwG5ubnIzs5+6rEsD87fB8fucT8PV69ehUqlQvv27c3PN2rUCHXq1Cm2744dO1q0z549i4iICPNlIAAwGo3m93L27FlkZmbCzc3NYrucnByL8xIAbG1tkZ2d/VTvnaoXJipkdaZMmYJZs2YBwGPHWzyNzMxMzJgxA7Nnzy72XFkG73p4eMDFxQWXL1+ukHjUarVFW5KkEpeZTKYnfo3MzEx06NDB4svoAQ8PD/P/z549i6ysLCgUCsTHx8PHx6fcrzV16lS88sorWLhwIVauXIkxY8Y88svV3d3dnAwUtXDhQvTp0wfOzs4WcT6JIUOGQKvVYsuWLdBoNNDr9XjhhRcs1qmI4/60n2dMTAwGDx6M1157DZ9//jlcXV1x8OBBhIaGIi8v76kTlQdJVf369YEy/DxcvXq1zPu2t7e3aGdmZuKzzz7DyJEji61rY2ODzMxM+Pj4ICwsrNjzRQdup6SkPPU5QNULExWyOgMGDEBeXh4kSUL//v2LPd+wYUNoNBpEREQgICAAKLiz4Pjx4+bbZJs3b45ffvnFYrsjR45YtNu3b49Lly6hUaNGTxSnQqHA2LFjsXbtWnzyySfw9fW1eD4zMxM2NjZo3rw5DAYDjh49ar7tNjk5GVeuXEGLFi2e6LULK/q+jhw5UmpvUPv27bFhwwZ4enrCycmpxHVSUlIwadIkfPjhh4iPj8f48eNx6tQp2Nralri+RqOxGBD6wPPPPw97e3ssXrwYO3fuxP79+x/5PoKCgrB48WIIIYr14Hh7e5f4OTVv3hw///yzxTYRERFwdHREvXr1SnwdlUqFiRMnYuXKldBoNBg7dmyp760klf15PnDy5EmYTCb85z//gUKRf9/Dxo0bK2z///3vf+Hk5IQ+ffoAZfh5aNq0KQwGA06fPo0OHToAAK5fv24xGLc07du3x5UrV0rdd/v27ZGQkACVSoXAwMBS9xMVFYXc3Nxig+upZuNdP2R1lEolIiMjcenSJYvu9wfs7e3x2muv4a9//St27tyJS5cuYdq0acjOzjbf2vrqq6/i2rVr+Otf/4orV65g3bp1FnN/AMB7772HQ4cOYdasWThz5gyuXbuGbdu2mXtzyuLzzz+Hn5+f+VbPS5cu4dq1a1ixYgWCgoKQmZmJxo0bY9iwYZg2bRoOHjyIs2fP4uWXX0bdunUxbNiwpz5eERER+Ne//oWrV69i0aJF2LRpE958880S1x0/fjzc3d0xbNgwHDhwANHR0QgLC8Ps2bNx69YtoODY+fn54W9/+xsWLFgAo9GId955p9TXDwwMxNGjRxETE4OkpCRzD4FSqcSkSZMwd+5cNG7cuNglqqJ69eqFzMxMXLx4sczv/fXXX0dcXBzeeOMNXL58Gdu2bcMnn3yCt956y/zlXpKpU6di79692LlzJ6ZMmVLm1wNQ6Z/nA40aNYJer8f//vc/3LhxA2vXrsW33377RPtKTU1FQkICYmNjsXv3brzwwgtYt24dFi9ebO6xeNzPQ7NmzdCnTx9Mnz4dx44dw+nTpzF9+nTY2to+9tLgxx9/jDVr1uCzzz7DxYsXERkZifXr1+Nvf/sbAKBPnz4IDg7G8OHD8ccffyAmJgaHDh3Chx9+iBMnTpj3c+DAATRo0MDikhrVfExUyCo5OTmV+hc/AMyfPx+jRo3CK6+8gvbt2+P69evYtWuX+Xq5v78/fv75Z2zduhVt27bFt99+i3/+858W+2jTpg3Cw8Nx9epVdO/eHUFBQfj444+L9Yw8iqurK44cOYKXX34Z//jHPxAUFITu3bvjxx9/xL///W84OzsDBfNYdOjQAYMHD0ZwcDCEENixY0exSwFP4u2338aJEycQFBSEf/zjH1iwYEGJPVEoGDexf/9++Pv7Y+TIkWjevDlCQ0ORm5sLJycnrFmzBjt27MDatWuhUqlgb2+P77//HkuXLsXvv/9e4j7feecdKJVKtGjRAh4eHrh586b5uQeXKSZPnvzY9+Hm5oYRI0aUeFmqNHXr1sWOHTtw7NgxtG3bFq+++ipCQ0PNX4Clady4Mbp06YJmzZoVG8tUFpX5eT7Qtm1bLFiwAF988QVatWqFH374AfPmzXuifU2ePBk+Pj5o1qwZXnvtNTg4OODYsWN46aWXzOuU5edhzZo18PLyQo8ePTBixAhMmzYNjo6OpY4teqB///7Yvn07/vjjD3Tq1AnPPvssFi5caO4RlSQJO3bsQI8ePTB58mQ0adIEY8eORWxsLLy8vMz7+fHHHzFt2rQnOgZUfUmiJk6dSVRLBAYGYs6cOVY7RfyBAwfQu3dvxMXFWXzhlObcuXPo27cvoqKiLAadVjQhBBo3bozXX38db731VqW9Tk1369Yt+Pn54c8//0Tv3r0r9bUuXryI5557DlevXjX/AUC1A8eoEFGF0+l0uHfvHj799FO8+OKLZUpSUPBX/RdffIHo6Gi0bt26UmK7d+8e1q9fj4SEhDL19NBDe/fuRWZmJlq3bo34+Hi8++67CAwMRI8ePSr9tePj47FmzRomKbUQExUiqnA//vgjQkND0a5du3JP017ZNZ08PT3h7u6O7777rsRba6l0er0eH3zwAW7cuAFHR0d06dIFP/zwQ4Ve8irNg0G/VPvw0g8RERFZLQ6mJSIiIqvFRIWIiIisFhMVIiIislpMVIiIiMhqMVEhIiIiq8VEhYiIiKwWExUiIiKyWkxUiIiIyGr9f5uxg1jzmXabAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "n = 40\n", "bootstraps = 20\n", "\n", "x = np.linspace(-3, 3, n)\n", "y = np.exp(-(x**2)) + 1.5 * np.exp(-((x - 2) ** 2)) + np.random.normal(0, 0.1, size=n)\n", "\n", "biases = []\n", "variances = []\n", "mses = []\n", "\n", "p_degrees = list(range(1, 5))\n", "for p in p_degrees:\n", " predictions = np.zeros((bootstraps, int(n*0.2)), dtype=float)\n", " targets = np.zeros((bootstraps, int(n*0.2)), dtype=float)\n", " targets_nf = np.zeros((bootstraps, int(n*0.2)), dtype=float)\n", "\n", " X = PolynomialFeatures(degree=p).fit_transform(x.reshape(-1, 1))\n", " X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, shuffle=False)\n", " for b in range(bootstraps):\n", " x_sample, y_sample = resample(X_train, y_train)\n", " \n", " model = LinearRegression().fit(X_train, y_train)\n", "\n", " predictions[b, :] = model.predict(X_test)\n", " targets[b, :] = y_test\n", " #targets_nf[b, :] = y_test\n", "\n", " mse, bias, variance = calculate_key_metrics(predictions, targets)\n", " mses.append(mse)\n", " biases.append(bias)\n", " variances.append(variance)\n", "\n", "plt.plot(p_degrees, np.array(mses), label=\"MSE\", lw=3)\n", "plt.plot(p_degrees, biases, label=\"Bias^2\", linestyle=\"dashed\", lw=2)\n", "plt.plot(p_degrees, variances, label=\"Variance\", linestyle=\"dashed\", lw=2)\n", "#plt.plot(range(1, 5), np.array(biases) + np.array(variances), label=\"Bias^2 + Variance\", linestyle=\"dashed\")\n", "plt.xlabel(\"Model Complexity (Polynomial Degree)\")\n", "plt.ylabel(\"Error\")\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "253b8461", "metadata": {}, "source": [ "**e)** Discuss the bias-variance trade-off as function of your model complexity (the degree of the polynomial).\n" ] }, { "cell_type": "markdown", "id": "4fa16eb2", "metadata": {}, "source": [ "
\n", " The assymmetry of bias and variance reaches a minimum at the same point where the total model error reaches a minimum wrt to the model complexity. At a polynomial degree of 3 we have the best model complexity.\n", "
\n" ] }, { "cell_type": "markdown", "id": "70cd5da0", "metadata": {}, "source": [ "\n", "**f)** Compute and discuss the bias and variance as function of the number of data points (choose a suitable polynomial degree to show something interesting).\n" ] }, { "cell_type": "code", "execution_count": 6, "id": "74e2cec7", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAGwCAYAAABVdURTAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjUsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvWftoOwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAX29JREFUeJzt3Xl4U1X+P/D3TdImTZukW9IFCi2LAooUWkBgRpixWEYEcQP9oSAiDioKdlRkHGBwRosbosCIC4vrgPhFRlFBrKKsolRUFBERWdumW5ZuSZvc3x9tb5suNC1pb5O+X8+Th96bc24+J4XkwznnniOIoiiCiIiIKEAo5A6AiIiIyJeY3BAREVFAYXJDREREAYXJDREREQUUJjdEREQUUJjcEBERUUBhckNEREQBRSV3AB3N7Xbj3Llz0Ol0EARB7nCIiIjIC6Iowm63Iz4+HgrF+ftmulxyc+7cOSQkJMgdBhEREbXB6dOn0b179/OW6XLJjU6nA2reHL1eL3c4RERE5AWbzYaEhATpe/x8ulxyUzsUpdfrmdwQERH5GW+mlHBCMREREQUUJjdEREQUUJjcEBERUUDpcnNuiIioa3G5XKisrJQ7DPJCcHBwi7d5e4PJDRERBSRRFJGbmwuLxSJ3KOQlhUKBpKQkBAcHX9B1mNwQEVFAqk1sTCYTtFotF27t5GoX2c3JyUGPHj0u6PfF5IaIiAKOy+WSEpuoqCi5wyEvGY1GnDt3DlVVVQgKCmrzdTihmIiIAk7tHButVit3KNQKtcNRLpfrgq7D5IaIiAIWh6L8i69+X0xuiIiIKKAwuSEiIqKAwgnFPlLqqEK+3QGz3QGFAKQmRsodEhERUZfEnhsfePurU7hk8XaMeWYnJr+0D49/dETukIiIyE/dfvvtEAQBs2fPbvTcvffeC0EQcPvttwMA8vPzcffdd6NHjx5Qq9WIjY1Feno69uzZI9VJTEyEIAiNHkuXLu3QdnUk9tz4QGRo9exuLSpgEopht8odERER1ed2iyguc8oaQ4Q2GAqFdxNmExISsGHDBjz33HMICQkBAFRUVODtt99Gjx49pHI33HADnE4nXnvtNfTq1Qt5eXnIyspCYWGhx/Uee+wxzJo1y+OcTqfzSbs6IyY3PmDSqzFP9S7mqTYDAO4oXQBRvJqz9ImIOoniMidS/v2prDEc/EcaosLUXpUdMmQIjh8/js2bN2Pq1KkAgM2bN6NHjx5ISkoCAFgsFuzatQs7d+7E6NGjAQA9e/bEsGHDGl1Pp9MhNjbWp+3pzDgs5QMmnRpFYl0GHCkWwVrOfUyIiKjt7rjjDqxbt046Xrt2LWbMmCEdh4WFISwsDFu2bIHD4ZApys6JyY0PGHVqmMUI6dgEC8x2/kUjIqK2u/XWW7F7926cPHkSJ0+exJ49e3DrrbdKz6tUKqxfvx6vvfYawsPDMWrUKPz973/H999/3+ha8+fPl5Kh2seuXbs6uEUdh8mND6hVSpSro6Vjo2CB2cbkhoiI2s5oNGL8+PFYv3491q1bh/HjxyM6OtqjzA033IBz587h/fffx7hx47Bz504MGTIE69ev9yj30EMP4dChQx6P1NTUDm5Rx+kUc25WrVqFp59+Grm5uRg0aBBWrFjR5JghAIwZMwZffPFFo/NXX301Pvzwww6ItmlimAmwV/9sFCww2ytki4WIiDxFaINx8B9pssfQWnfccQfmzJkD1HxXNkWj0WDs2LEYO3YsFi5ciDvvvBOLFy+W7qgCgOjoaPTp0+cCovcvsic3GzduREZGBlavXo3hw4dj+fLlSE9Px9GjR2EymRqV37x5M5zOuhnvhYWFGDRoEG666aYOjtyTUh8vJTcmwYJsDksREXUaCoXg9WTezmTcuHFwOp0QBAHp6ele1RkwYAC2bNnS7rF1ZrInN8uWLcOsWbOkSVKrV6/Ghx9+iLVr1+KRRx5pVD4y0nNxvA0bNkCr1Tab3DgcDo+JVjabzedtAIAIvQ7WM1oYhLLqOTccliIiogukVCpx5MgR6ef6CgsLcdNNN+GOO+7AZZddBp1Oh2+++QZPPfUUrr32Wo+ydrsdubm5Hue0Wi30en0HtKLjyTrnxul04uDBg0hLq+sqVCgUSEtLw759+7y6xpo1a3DzzTcjNDS0yeczMzNhMBikR0JCgs/ir8+or5tUbBIsMNvK2+V1iIioa9Hr9U0mIWFhYRg+fDiee+45XHHFFbj00kuxcOFCzJo1CytXrvQou2jRIsTFxXk8Hn744Q5sRceSteemoKAALpcLMTExHudjYmLw888/t1j/wIEDOHz4MNasWdNsmQULFiAjI0M6ttls7ZLgmHQamMVw9MVZaAUH7LZin78GEREFvoaTgRuqP+SUmZmJzMzM85b//ffffRabv5B9WOpCrFmzBgMHDmx28jEAqNVqqNXtP85q0qmRD0PdCXvu+YoTERFRO5E1uYmOjoZSqUReXp7H+by8vBZXUiwtLcWGDRvw2GOPtXOU3jHq1FhQdQNWV01EvmiAs966N0RERNRxZJ1zExwcjJSUFGRlZUnn3G43srKyMGLEiPPW3bRpExwOh8eCRnIy6dQ4IcbhZ7EHCmGA3eFGmbNK7rCIiIi6HNkX8cvIyMArr7yC1157DUeOHMHdd9+N0tJS6e6padOmYcGCBY3qrVmzBpMmTUJUVJQMUTdm0msaneMdU0RERB1P9jk3U6ZMQX5+PhYtWoTc3FwkJydj27Zt0iTjU6dOQaHwzMGOHj2K3bt345NPPpEp6sbC1Cpog5Uoc7qkc2a7A4nRTd/FRURERO1D9uQGAObMmSOtwNjQzp07G527+OKLIYpiB0TWOglhQD/LbpgEC/LECJjtg+UOiYiIqMvpFMlNoIgNU+H5sv8AAHa7LsEvtllyh0RERNTlyD7nJpCEGSJQLlbvHWISuDM4ERGRHJjc+JBJX72QH6TkhptnEhGRb/3+++8QBAGHDh3y+bV//fVXxMTEQKvVYs+ePT6/fkdhcuNDJp0GZlQnN+FCKSw2u9whERGRn7n99tshCIL0iIqKwrhx4/D9998DABISEpCTk4NLL73Up6977tw5jB07Fn/4wx8wc+ZMXHPNNfjhhx88ylRWVmL+/PkYOHAgQkNDER8fj2nTpuHcuXM+jeVCMbnxIZNOjfyanhsAqLJylWIiImq9cePGIScnBzk5OcjKyoJKpcI111wD1GygGRsbC5XKd9Nmi4uLkZ6ejj/+8Y945513sGLFCsyePRvp6ek4ceKEVK6srAzZ2dlYuHAhsrOzsXnzZhw9ehQTJ070WSy+wOTGh0x6tTQsBQBCCZMbIiJqPbVajdjYWMTGxiI5ORmPPPIITp8+jfz8/EbDUi6XCzNnzkRSUhJCQkJw8cUX4/nnn/e43s6dOzFs2DCEhoYiPDwco0aNwsmTJ4GahGX8+PEYNWoUXnvtNWn38czMTNx777246qqrYDabAQAGgwE7duzA5MmTcfHFF+Pyyy/HypUrcfDgQZw6darD36fm8G4pHzLpNNhXL7nROArgrHIjWMUckoiI2qakpARvvvkm+vTpg6ioKJSWlno873a70b17d2zatAlRUVHYu3cv7rrrLsTFxWHy5MmoqqrCpEmTMGvWLPz3v/+F0+nEgQMHIAgCAECr1WLv3r1Nvvajjz6KRx999LzxWa1WCIKA8PDw85brSExufKh688y6X65JsCC/xIFu4SGyxkVERPXsXQnsW9VyubhBwP/b4Hnu7ZuBnO9arjviXmBk0+u3eWPr1q0ICwsDavZSjIuLw9atWxstagsAQUFBWLJkiXSclJSEffv24Z133sHkyZNhs9lgtVpxzTXXoHfv3gCA/v37tzm2+ioqKjB//nzccsst0Ov1PrmmLzC58aFwbRCKhEjp2CRYYLZVMLkhIupMHHbA7sUEWEO3xufKCryr67iwG0r+9Kc/4cUXXwRq5sP85z//wV/+8hccOHCgyfKrVq3C2rVrcerUKZSXl8PpdCI5ORkAEBkZidtvvx3p6ekYO3Ys0tLSMHnyZMTFxV1QjJWVlZg8eTJEUZRi7Sw4XuJDgiDAoY3FGTEa2e4+MIvhXOuGiKizUesAXXzLD21047raaO/qqnUXFGJoaCj69OmDPn36YOjQoXj11VdRWlqKV155pVHZDRs24MEHH8TMmTPxySef4NChQ5gxYwacTqdUZt26ddi3bx9GjhyJjRs34qKLLsL+/fvbHF9tYnPy5Ens2LGjU/XagD03vldiuAh/OP2CdHwxkxsios5l5Jy2Dxk1HKbqIIIgQKFQoLy8vNFze/bswciRI3HPPfdI544fP96o3ODBgzF48GAsWLAAI0aMwNtvv43LL7+81bHUJjbHjh3D559/3mk2sK6PyY2PGXVqj2P23BARUWs5HA7k5lbfcVtcXIyVK1eipKQEEyZMaFS2b9++eP3117F9+3YkJSXhjTfewNdff42kpCQAwIkTJ/Dyyy9j4sSJiI+Px9GjR3Hs2DFMmzat1XFVVlbixhtvRHZ2NrZu3QqXyyXFGRkZieDg4Atuuy8wufExU4PkJp+rFBMRUStt27ZNmhOj0+nQr18/bNq0CWPGjMHvv//uUfavf/0rvv32W0yZMgWCIOCWW27BPffcg48//hiouRvq559/xmuvvYbCwkLExcXh3nvvxV//+tdWx3X27Fm8//77ACDN6an1+eefY8yYMRfQat8RxM64vXY7stlsMBgMsFqt7TJG+Pynx/Dcp79Ix1f2M2HN7UN9/jpERNS8iooKnDhxAklJSdBoNHKHQ1463++tNd/f7LnxMZNejbuUH+BK5bcwoRiLrM/IHRIREVGXwuTGx0w6NUQhD8MVPwMABDtXKSYiIupIvBXcx+pvngkAQeVmuNxdauSPiIhIVkxufKx6f6kI6TgaFhSVOs9bh4iIiHyHyY2PRYUGe27BgGKYeccUERFRh2Fy42MqpQIOjUk6NgkWrnVDRETUgZjctAMxLEb62SRYkG9jckNERNRRmNy0g2BDDNxi9Vby1T03HJYiIiLqKExu2kGUPhRFqN40zchhKSIiog7F5KYdmHQa5IvVk4qNsMBsZc8NERHJSxAEbNmyRe4wOgQX8WsHJr0ab7muhN5VinyEo8BeJndIRETkByZMmIDKykps27at0XO7du3CFVdcge+++w6XXXZZq6+dk5ODiIgIL0r6PyY37cCkU2ORa6x03L2kStZ4iIjIP8ycORM33HADzpw5g+7du3s8t27dOqSmprY6sXE6nQgODkZsbKyPo+28OCzVDoyNdgZ3oIvtT0pERG1wzTXXwGg0Yv369R7nS0pKsGnTJkyaNAm33HILunXrBq1Wi4EDB+K///2vR9kxY8Zgzpw5mDdvHqKjo5Geng40MSw1f/58XHTRRdBqtejVqxcWLlyIyspK6fl//vOfSE5OxhtvvIHExEQYDAbcfPPNsNvtUhm3242nnnoKffr0gVqtRo8ePfD4449Lz58+fRqTJ09GeHg4IiMjce211zba1bw9MLlpByad506mjio3bBXsvSEiovNTqVSYNm0a1q9f7/Gf4k2bNsHlcuHWW29FSkoKPvzwQxw+fBh33XUXbrvtNhw4cMDjOq+99hqCg4OxZ88erF69usnX0ul0WL9+PX766Sc8//zzeOWVV/Dcc895lDl+/Di2bNmCrVu3YuvWrfjiiy+wdOlS6fkFCxZg6dKlWLhwIX766Se8/fbbiImpXg6lsrIS6enp0Ol02LVrF/bs2YOwsDCMGzcOTmf7rtwviF2sS6E1W6a3VUWlC/0Wfgw9ymAULCgX1Xg943r0Mena5fWIiMhTRUUFTpw4gaSkJGg0nv/hfO3H1/D6T6+3eI0BkQOw4soVHufuy7oPPxX91GLdaQOmYfol09sQOfDzzz+jf//++PzzzzFmzBgAwBVXXIGePXvijTfeaFT+mmuuQb9+/fDMM88ANT03NpsN2dnZHuUEQcB7772HSZMmNfm6zzzzDDZs2IBvvvkGqOm5efrpp5Gbmwudrvr76+GHH8aXX36J/fv3w263w2g0YuXKlbjzzjsbXe/NN9/Ev//9bxw5cgSCUL08itPpRHh4OLZs2YKrrrqqUZ3z/d5a8/3NOTftQBOkxBWa43gdiwAAa6vGwWwbz+SGiKgTKK0shbnM3GK52NDGc1SKHEVe1S2tLG1zfP369cPIkSOxdu1ajBkzBr/++it27dqFxx57DC6XC0888QTeeecdnD17Fk6nEw6HA1qt1uMaKSkpLb7Oxo0b8cILL+D48eMoKSlBVVVVo6QhMTFRSmwAIC4uDmZzdfuPHDkCh8OBK6+8ssnrf/fdd/j111896qMmgTl+/Hir3pPWYnLTTsSwGKCk+mduwUBE1HmEBoXCpDW1WC5SHdnkOW/qhgaFtjk+1Ewsvu+++7Bq1SqsW7cOvXv3xujRo/Hkk0/i+eefx/LlyzFw4ECEhoZi3rx5jYZ5QkPP//r79u3D1KlTsWTJEqSnp8NgMGDDhg149tlnPcoFBQV5HAuCALfbDQAICQk572uUlJQgJSUFb731VqPnjEZji+/BhWBy006UulgpuTEKFnzHVYqJiDqF6ZdMb/OQUcNhqvYyefJkzJ07F2+//TZef/113H333RAEAXv27MG1116LW2+9FaiZ0PvLL79gwIABrbr+3r170bNnTzz66KPSuZMnT7bqGn379kVISAiysrKaHJYaMmQINm7cCJPJ1G7TQJrDCcXtJNxggE2szmpNKIaZ+0sREZGXwsLCMGXKFCxYsAA5OTm4/fbbgZqEYseOHdi7dy+OHDmCv/71r8jLy2v19fv27YtTp05hw4YNOH78OF544QW89957rbqGRqPB/Pnz8fDDD+P111/H8ePHsX//fqxZswYAMHXqVERHR+Paa6/Frl27cOLECezcuRP3338/zpw50+qYW4PJTTsx6eutUixYOSxFREStMnPmTBQXFyM9PR3x8fEAgH/84x8YMmQI0tPTMWbMGMTGxjY7Qfh8Jk6ciAceeABz5sxBcnIy9u7di4ULF7b6OgsXLsTf/vY3LFq0CP3798eUKVOkOTlarRZffvklevTogeuvvx79+/fHzJkzUVFR0e49Obxbqp28uus3XPLJVIxQVs+qnx77Hl6b/ed2ez0iIqpzvrtuqPPy1d1S7LlpJ0adGmaES8due66s8RAREXUVsic3q1atQmJiIjQaDYYPH95oIaKGLBYL7r33XsTFxUGtVuOiiy7CRx991GHxesuk08As1iU3ypLWj4kSERFR68l6t9TGjRuRkZGB1atXY/jw4Vi+fDnS09Nx9OhRmEyNb7VzOp0YO3YsTCYT3n33XXTr1g0nT55EeHh4k9eXk0mv9khuwioLUe50ISRYKWtcREREgU7W5GbZsmWYNWsWZsyYAQBYvXo1PvzwQ6xduxaPPPJIo/Jr165FUVER9u7dK917n5iYeN7XcDgccDjqJvPabDaft6MpJp1amlAMaa2bCvSMurC1D4iIiOj8ZBuWcjqdOHjwINLS0uqCUSiQlpaGffv2NVnn/fffx4gRI3DvvfciJiYGl156KZ544gm4XK5mXyczMxMGg0F6JCQktEt7GgpTq7BfmYKxjqeQXPES1rrG8Y4pIqIO1sXumfF7vvp9yZbcFBQUwOVySRts1YqJiUFubtOTb3/77Te8++67cLlc+Oijj7Bw4UI8++yz+Pe//93s6yxYsABWq1V6nD592udtaYogCFDro3BM7A4LdAAErnVDRNRBanv3y8rK5A6FWqF2pWWl8sKmcPjVCsVutxsmkwkvv/wylEolUlJScPbsWTz99NNYvHhxk3XUajXUanWHx4qaoamThXX/sPK5SjERUYdQKpUIDw/3WHOldvNG6pzcbjfy8/Oh1WqhUl1YeiJbchMdHQ2lUtloZcW8vDzExjberAw1G3YFBQV5ZHT9+/dHbm4unE4ngoOD2z3u1jDqPJMqDksREXWc2u+S2gSHOj+FQoEePXpccCIqW3ITHByMlJQUZGVlSasrut1uZGVlYc6cOU3WGTVqFN5++2243W4oFNUjar/88gvi4uI6XWKDmtvB0xVfo5eQA51Qht/sD8odEhFRlyEIAuLi4mAymVBZWSl3OOSF4OBg6fv9Qsg6LJWRkYHp06cjNTUVw4YNw/Lly1FaWirdPTVt2jR069YNmZmZAIC7774bK1euxNy5c3Hffffh2LFjeOKJJ3D//ffL2YxmGXVqTFBtRYriGABghm223CEREXU5SqXygudwkH+RNbmZMmUK8vPzsWjRIuTm5iI5ORnbtm2TJhmfOnXKI4NLSEjA9u3b8cADD+Cyyy5Dt27dMHfuXMyfP1/GVjTPpPNc66bKmiNrPERERF2B7BOK58yZ0+ww1M6dOxudGzFiBPbv398BkV04k16D3+slNwquUkxERNTuZN9+IZBV99xESMfqinxUutyyxkRERBTomNy0I1ODzTNNggUFJbxjioiIqD0xuWlHEdpgFKKu58YoWLiQHxERUTtjctOOFAoBVdq6DUBNsHCtGyIionbG5Ka96eq2l6jdPJOIiIjaD5ObdqbWx6BKrH6bTUIxh6WIiIjamey3ggc6oyEEh8Q+EEXgF3cCh6WIiIjaGZObdmbSqXGj85/ScRqHpYiIiNoVh6XamUmn8Thmzw0REVH7YnLTzkwNdwbnnBsiIqJ2xeSmnZn0nslNQYkDbrcoWzxERESBjnNu2plRp8YViu8wV7UZJljwXNUNKC67ElFhai9qExERUWsxuWln0WFqaAQnUhTHAADxQiHMdgeTGyIionbCYal2FqRUwKE2SsfVC/lx3g0REVF7YXLTAdxhDVYptvF2cCIiovbC5KYDqAxx0s8moZg9N0RERO2IyU0HiNKHoUgMA2p6bvKZ3BAREbUbJjcdwKRTwyxGAACMsMBsK5c7JCIiooDF5KYDVCc34QAAtVCFMmuh3CEREREFLCY3HcCk1yAf4dKx254razxERESBjMlNB6jfcwMAitI8iCJXKSYiImoPXMSvA5h0GnzuSoZFDINZDMeP7njYHVXQa4LkDo2IiCjgMLnpACa9GgfE/jjg6i+dM9scTG6IiIjaAYelOoAmSAmdxjOPNNu5kB8REVF7YHLTQUw6z72kuNYNERFR++CwVAcx6TQoys+BSbBAjUrk2/t7UYuIiIhai8lNBzHq1HhFPQ9hQgWOu+Ow0T5W7pCIiIgCEoelOkj928G5eSYREVH7YXLTQUx6tbSQn04oh9VqlTskIiKigMTkpoOYdBqPhfxc9hxZ4yEiIgpUTG46SP3NMwFAUZInazxERESBislNBzHpPbdg0DoLUVHpkjUmIiKiQMTkpoMYGwxLmQQL17ohIiJqB0xuOoheo0KxMlI6NgkWrlJMRETUDpjcdBBBEOAKNUnH1beDs+eGiIjI15jcdCCFLk76ORx2mDksRURE5HOdIrlZtWoVEhMTodFoMHz4cBw4cKDZsuvXr4cgCB4PjUbTofG2lVYfhSsdT2Ngxau4s/JBDksRERG1A9mTm40bNyIjIwOLFy9GdnY2Bg0ahPT0dJjN5mbr6PV65OTkSI+TJ092aMxtZdJrcFzsBju0AAQOSxEREbUD2ZObZcuWYdasWZgxYwYGDBiA1atXQ6vVYu3atc3WEQQBsbGx0iMmJqbZsg6HAzabzeMhF5Pes4eJw1JERES+J2ty43Q6cfDgQaSlpdUFpFAgLS0N+/bta7ZeSUkJevbsiYSEBFx77bX48ccfmy2bmZkJg8EgPRISEnzeDm8ZdWqPYyY3REREvidrclNQUACXy9Wo5yUmJga5ublN1rn44ouxdu1a/O9//8Obb74Jt9uNkSNH4syZM02WX7BgAaxWq/Q4ffp0u7TFGyadGkOEX3CvcguWqNZBYzshWyxERESBSiV3AK01YsQIjBgxQjoeOXIk+vfvj5deegn/+te/GpVXq9VQq9WNzsvBpNNgpOJHPBi0CQCwp2IgqlxuqJSyjw4SEREFDFm/VaOjo6FUKpGX57nPUl5eHmJjY726RlBQEAYPHoxff/21naL0HaNODTPqVik2woLCUqesMREREQUaWZOb4OBgpKSkICsrSzrndruRlZXl0TtzPi6XCz/88APi4uK8KC2vqNBgFKJu80wu5EdEROR7sg9LZWRkYPr06UhNTcWwYcOwfPlylJaWYsaMGQCAadOmoVu3bsjMzAQAPPbYY7j88svRp08fWCwWPP300zh58iTuvPNOmVvSMoVCgDPECFRVHxtRXLPWjUHu0IiIiAKG7MnNlClTkJ+fj0WLFiE3NxfJycnYtm2bNMn41KlTUCjqOpiKi4sxa9Ys5ObmIiIiAikpKdi7dy8GDBggYytaQRcLFFf/WL2/FHtuiIiIfEkQRVGUO4iOZLPZYDAYYLVaodfrO/z171y3Hy/9Pg5KQcT37iR8fsUmzE3r2+FxEBER+ZPWfH/zNp0OZjRoUVgzDMWdwYmIiHyPyU0HM+o0MIvVd0xFw4p8W5ncIREREQUUJjcdzKRTS8mNSnCjwpYvd0hEREQBRfYJxV2NSafGYTERWrcDZjEcFnu53CEREREFFCY3Hcyk1+CuqsnScZBbgCiKEARB1riIiIgCBYelOpipweaZlS4RxWWVssVDREQUaJjcdLDosMb7XPGOKSIiIt9hctPBglUKRIYGe5zjFgxERES+wzk3MugbWoEFlYthFCz4xn0xzPZX5A6JiIgoYDC5kUGoPhzJtuMAgByhAAc4LEVEROQzHJaSQYReD4sYCgAwoRj53F+KiIjIZ5jcyMBYbyE/k2CB2caeGyIiIl9hciOD+qsUa4RKlFqL5A6JiIgoYDC5kYFJr4YZEdKx254jazxERESBhMmNDEz1Ns8EAGWpWdZ4iIiIAgmTGxmYdGrk10tu9FWFKHFUyRoTERFRoGByIwOTXu3Rc8NJxURERL7D5EYG2mAV7EHR0rFJsMDM28GJiIh8gov4ycQWmoR/WaciX4zAT2IPxDC5ISIi8gkmNzIJMsRgTdF46ZjDUkRERL7BYSmZmPQaj2OuUkxEROQbTG5kYtKpPY4554aIiMg3OCwlE5NOjXDYES8UwiRYUGjTyx0SERFRQGByIxOTXo1/B63FNcqvAAC3WfvIHRIREVFA4LCUTEw6jcdCfoqSPFnjISIiChRMbmRSvXlm3f5SWmcBHFUuWWMiIiIKBExuZNJwfymTYOEdU0RERD7A5EYm+hAVipR1PTcmoZh3TBEREfkAkxuZCIKAqpAY6dgIK8w2JjdEREQXismNjAR9XXJTPSzFVYqJiIguFJMbGYXojXCKSoCbZxIREfkMkxsZmfQhyEf1pGKjUMxhKSIiIh9gciMjk04trXWjRhXybaVyh0REROT3uEKxjEx6NWY5/4YSaFAODS4pqZI7JCIiIr/H5EZGJp1GGpYCN88kIiLyCQ5LycjYYGfwwhIHXG5RtniIiIgCQadIblatWoXExERoNBoMHz4cBw4c8Krehg0bIAgCJk2a1O4xtgeT3jO5cYvVCQ4RERG1nezJzcaNG5GRkYHFixcjOzsbgwYNQnp6Osxm83nr/f7773jwwQfxxz/+scNi9bWoUDW6CwWYq/w/PK5agwmKvRyaIiIiukCyJzfLli3DrFmzMGPGDAwYMACrV6+GVqvF2rVrm63jcrkwdepULFmyBL169Trv9R0OB2w2m8ejs1AqBPQKrcADQf+HqaosXK44AjMX8iMiIrogsiY3TqcTBw8eRFpaWl1ACgXS0tKwb9++Zus99thjMJlMmDlzZouvkZmZCYPBID0SEhJ8Fr9P6DxXKeZaN0RERBdG1uSmoKAALpcLMTExHudjYmKQm5vbZJ3du3djzZo1eOWVV7x6jQULFsBqtUqP06dP+yR2XwnWx8ItCkDtQn4cliIiIrogfnUruN1ux2233YZXXnkF0dHRXtVRq9VQq9VelJRHtF6LQuhhhLVmfykmN0RERBdC1uQmOjoaSqUSeXl5Hufz8vIQGxvbqPzx48fx+++/Y8KECdI5t9sNAFCpVDh69Ch69+7dAZH7jlGnhlkMh1Gwwggr8m1lcodERETk12QdlgoODkZKSgqysrKkc263G1lZWRgxYkSj8v369cMPP/yAQ4cOSY+JEyfiT3/6Ew4dOtT55tN4of4WDEGCC+W2ArlDIiIi8muyD0tlZGRg+vTpSE1NxbBhw7B8+XKUlpZixowZAIBp06ahW7duyMzMhEajwaWXXupRPzy8OjFoeN5fGHUamMW6VYpFW9NzjYiIiMg7sic3U6ZMQX5+PhYtWoTc3FwkJydj27Zt0iTjU6dOQaGQ/Y71dmPSq/FrvS0YVGVmiKIIQRBkjYuIiMhftTq5qaysREhICA4dOuSz3pI5c+Zgzpw5TT63c+fO89Zdv369T2KQi6lmzk2tSLEI1vJKhGuDZY2LiIjIX7U6uQkKCkKPHj3gcrnaJ6IuxqhT47gYjz2uS2BGOM6K0TDbHUxuiIiI2qhNw1KPPvoo/v73v+ONN95AZGSk76PqQtQqJX7UDMHUsoHSObPNgYtidLLGRURE5K/alNysXLkSv/76K+Lj49GzZ0+EhoZ6PJ+dne2r+LoEk04NS1mldMwtGIiIiNquTcmNv+7C3VmZdBr8klciHXOVYiIiorZrU3KzePFi30fShZl0dSsoC3BzfykiIqILcEG3gh88eBBHjhwBAFxyySUYPHiwr+LqUox6NVYFLUeq4heEwIG/2z6UOyQiIiK/1abkxmw24+abb8bOnTulRfQsFgv+9Kc/YcOGDTAajb6OM6CZdBpEoAQxggUAYLcVyx0SERGR32rT6nj33Xcf7HY7fvzxRxQVFaGoqAiHDx+GzWbD/fff7/soA5xJp4a53kJ+op2rFBMREbVVm3putm3bhk8//RT9+/eXzg0YMACrVq3CVVdd5cv4ugSTTo3vxAjpWFGSd97yRERE1Lw29dy43W4EBQU1Oh8UFCTt0k3eM+k995fSVxWh1FEla0xERET+qk3JzZ///GfMnTsX586dk86dPXsWDzzwAK688kpfxtclNNyCwSgUI5+3gxMREbVJm5KblStXwmazITExEb1790bv3r2RlJQEm82GFStW+D7KABeqVsGmqlvp2SRYuNYNERFRG7Vpzk1CQgKys7Px6aef4ueffwYA9O/fH2lpab6Or8twh8YA5dU/Vyc3XKWYiIioLS5oV/CxY8di7Nix7RNZFyPo4uqSG1hwlAv5ERERtUmrh6W4K3j7CDNEoFys3gmcw1JERERtx13BOwmTXoNFVbejTNTgnBiFJA5LERERtQl3Be8kTDoN1rnGSMdh7LkhIiJqE+4K3knU3zwTADfPJCIiaqNWJzdVVVUQBAF33HEHunfv3j5RdUEmfYPkhsNSREREbdLqCcUqlQpPP/00qqq4gq4vmXQa6FCGgcJvuFJxEKHlZ+Gs4mrPRERErdXmFYq/+OIL30fThZl0aqQrv8YH6n9gTfCz+JPiEPJLODRFRETUWm2ac/OXv/wFjzzyCH744QekpKQ0mlA8ceJEX8XXZYRrg1AkNFil2FaBbuEhssZFRETkb9qU3Nxzzz0AgGXLljV6ThAEroHTBoIgoEprApzVxyZwrRsiIqK2aFNyw52/24kuFiis/tEkFOMMkxsiIqJWa9Wcm6uvvhpWq1U6Xrp0KSwWi3RcWFiIAQMG+DbCLiREH41KUQnUDEvl23jHFBERUWu1KrnZvn07HI663oQnnngCRUVF0nFVVRWOHj3q2wi7EKMhBPkwALXJDScUExERtVqrkhtRFM97TBfGpNPALIYDAKJgQ4G1TO6QiIiI/E6bbgWn9mHUqZEvRgAAFIIIpy1P7pCIiIj8TquSG0EQIAhCo3PkGyadWuq5AQDYc+UMh4iIyC+16m4pURRx++23Q62u3iqgoqICs2fPlta5qT8fh1rPpNPg+5rkxipqUVVug8stQqlgAklEROStViU306dP9zi+9dZbG5WZNm3ahUfVRZn0arziGo/VrglwIBgAUFTqhLHBpppERETUvFYlN+vWrWu/SAhRocEoFzSoP0/bbK9gckNERNQKnFDciaiUCkSFNtwdnEN9RERErcHkppMxNeilybcxuSEiImqNNm2/QO3HpAtGev67MKEYJQiB2f6Y3CERERH5lU7Rc7Nq1SokJiZCo9Fg+PDhOHDgQLNlN2/ejNTUVISHhyM0NBTJycl44403OjTe9mTSazBV+SluUX2Oq5VfcViKiIiolWRPbjZu3IiMjAwsXrwY2dnZGDRoENLT02E2m5ssHxkZiUcffRT79u3D999/jxkzZmDGjBnYvn17h8feHkw6DfJrbgc3wgKzlftLERERtYbsyc2yZcswa9YszJgxAwMGDMDq1auh1Wqxdu3aJsuPGTMG1113Hfr374/evXtj7ty5uOyyy7B79+4Oj709mPR1C/kFCy6U2fLlDomIiMivyJrcOJ1OHDx4EGlpaXUBKRRIS0vDvn37WqwviiKysrJw9OhRXHHFFU2WcTgcsNlsHo/OrOEqxSJXKSYiImoVWZObgoICuFwuxMTEeJyPiYlBbm7zX+pWqxVhYWEIDg7G+PHjsWLFCowdO7bJspmZmTAYDNIjISHB5+3wJaNOAzPqkhtlmZkblBIREbWC7MNSbaHT6XDo0CF8/fXXePzxx5GRkYGdO3c2WXbBggWwWq3S4/Tp0x0eb2tU99xESMeRriLYyqtkjYmIiMifyHoreHR0NJRKJfLyPHe/zsvLQ2xsbLP1FAoF+vTpAwBITk7GkSNHkJmZiTFjxjQqq1arpb2w/EH1zuAG6dgkWJBfUgGDNkjWuIiIiPyFrD03wcHBSElJQVZWlnTO7XYjKysLI0aM8Po6brc7YDbt1AQpURocLR2bBAvMXMiPiIjIa7Iv4peRkYHp06cjNTUVw4YNw/Lly1FaWooZM2YANRtxduvWDZmZmUDNHJrU1FT07t0bDocDH330Ed544w28+OKLMrfEd9xhMUBJ9c8mwcK1boiIiFpB9uRmypQpyM/Px6JFi5Cbm4vk5GRs27ZNmmR86tQpKBR1HUylpaW45557cObMGYSEhKBfv3548803MWXKFBlb4VtKXSy+tA6EGRE46O6LJDvXuiEiIvKWIHaxW3FsNhsMBgOsViv0er3c4TRp3oZvseXQOen4zj8k4R/XDJA1JiIiIjm15vvbL++WCnQmvcbjmMNSRERE3mNy0wk13BnczGEpIiIirzG56YSM9ZIbFapQYCuTNR4iIiJ/wuSmEzLpNJil3Ipv1LPxi3o6utu/lzskIiIiv8HkphMy6at7bqIFGxSCiLDKQpQ7XXKHRURE5BeY3HRCDTfPrF7rhvNuiIiIvMHkphMKU6tgVUZJx1zIj4iIyHtMbjohQRDgDjNJx0ahmFswEBEReYnJTScl6Oo2DjWBw1JERETeYnLTSYXpo1AhVu8EzmEpIiIi7zG56aSMeo00qdgkWJDP5IaIiMgrTG46KZNeDTMiAAARQgmKbHa5QyIiIvILTG46KWOY5+3gVdZcWeMhIiLyFyq5A6CmmfQarKpKx/9cI5EvhiOnJETukIiIiPwCk5tOyqRT44DYHxBrTpQBlS43gpTsbCMiIjofflN2Ug13BgeAghJOKiYiImoJk5tOKkIbDJVC8DjHhfyIiIhaxmGpTkqhEBAXpkCU/ShMQjFsCIXZnip3WERERJ0ek5tOLCmsEq87FwEAPnMlI8c+We6QiIiIOj0OS3Vian0MqsTqX5FJsHBYioiIyAtMbjoxoyEEBTAA3IKBiIjIa0xuOjGTrm4hvyhYUWArlTskIiKiTo/JTSdm0tXtL6UURDhtZrlDIiIi6vSY3HRi9XtuAAA2bsFARETUEiY3nZhJr0Z+zeaZAKAqz4fbLZ63DhERUVfH5KYTqz8sBQBRKEZRmVPWmIiIiDo7JjedWHRYMPJRl9yYYEE+75giIiI6LyY3nZhKqYBDYwQAFIo6AODt4ERERC3gCsWdXH5Yf/TNex2VNb+qp20VcodERETUqbHnppOLNoRKiQ3Yc0NERNQiJjednEmn9jjmnBsiIqLzY3LTyTVMbsx2DksRERGdD+fcdHImnRo3Kr/AMOFnmAQL1lr+IXdIREREnRqTm07OpNdghOIn3KDcBQBYa8+ROyQiIqJOjcNSnVzDLRgUpXkQRa5STERE1BwmN51cw1WKw13FsDuqZI2JiIioM+sUyc2qVauQmJgIjUaD4cOH48CBA82WfeWVV/DHP/4RERERiIiIQFpa2nnL+zuTXg2zWLe/lEkohtnGO6aIiIiaI3tys3HjRmRkZGDx4sXIzs7GoEGDkJ6eDrPZ3GT5nTt34pZbbsHnn3+Offv2ISEhAVdddRXOnj3b4bF3BE2QEiXBUdKxSbDwjikiIqLzkD25WbZsGWbNmoUZM2ZgwIABWL16NbRaLdauXdtk+bfeegv33HMPkpOT0a9fP7z66qtwu93Iysrq8Ng7ijs0RvrZJHB/KSIiovORNblxOp04ePAg0tLS6gJSKJCWloZ9+/Z5dY2ysjJUVlYiMjKyyecdDgdsNpvHw98odbHSzybBwmEpIiKi85A1uSkoKIDL5UJMTIzH+ZiYGOTm5np1jfnz5yM+Pt4jQaovMzMTBoNBeiQkJPgk9o5kCA9HiagBABjBYSkiIqLzkX1Y6kIsXboUGzZswHvvvQeNRtNkmQULFsBqtUqP06dPd3icF6r+7eAcliIiIjo/WRfxi46OhlKpRF5ensf5vLw8xMbGNlsPAJ555hksXboUn376KS677LJmy6nVaqjV6maf9wcmnQbb3UMR4bbDjHDk28rlDomIiKjTkjW5CQ4ORkpKCrKysjBp0iQAkCYHz5kzp9l6Tz31FB5//HFs374dqampHRixPIw6NeZV3SId9ymplDUeIiKizkz27RcyMjIwffp0pKamYtiwYVi+fDlKS0sxY8YMAMC0adPQrVs3ZGZmAgCefPJJLFq0CG+//TYSExOluTlhYWEICwuTtS3tpdHmmTbOuSEiImqO7MnNlClTkJ+fj0WLFiE3NxfJycnYtm2bNMn41KlTUCjqpga9+OKLcDqduPHGGz2us3jxYvzzn//s8Pg7gknvmdzYKqpQUemCJkgpW0xERESdlSB2sY2KbDYbDAYDrFYr9Hq93OF4xVpeiUFLPgEABKMSKriw/eG/ICFSK3doREREHaI1399+fbdUV6HXqDBMdRzfqu/CL5rpmKPawtvBiYiImsHkxg8IggB1mAERQgnAhfyIiIjOi8mNn1DUX6UYxTBzrRsiIqImMbnxE1p9FBxiEADAyM0ziYiImsXkxk+Y9Brkw1D9M4eliIiImsXkxk+Y9BppC4YowY5CW4ncIREREXVKTG78hFGnhlmMkI4rbXnnLU9ERNRVMbnxE/U3zwQARYl3u6YTERF1NUxu/IRJp/FIbtQV+ahyuWWNiYiIqDNicuMnTHo1zKhLboywoKDEKWtMREREnZHse0uRdyK1wdgvDsQsZwbMYjhOiLG42e5ArEEjd2hERESdCpMbP6FQCHCGdccOW7R0rnqtG4OscREREXU2HJbyI0ad5+7gXKWYiIioMSY3fsTUMLnhQn5ERESNcFjKj5j0agwUfkN3IR86oQxmew+5QyIiIup0mNz4EaNOg8ygV3Gp4ndUiQrcY7tJ7pCIiIg6HQ5L+ZH6C/mpBDcqbPlyh0RERNTpMLnxIw1XKRZsXKWYiIioISY3fsSk13gs5KcqN0MURVljIiIi6myY3PgRU4PNMyPFYhSXVcoaExERUWfD5MaPRIepkV9vWMoES81CfkRERFSLyY0fCVYpUKExSsdGwcK1boiIiBpgcuNnxLAY6WeTYOEqxURERA0wufEzSn3D5IbDUkRERPVxET8/E6nXo0DUQ4QAixjKYSkiIqIGmNz4GZNejWGO/8Bd0+k2nsNSREREHjgs5WdMOrWU2ABAPpMbIiIiD0xu/IxJp/E45pwbIiIiT0xu/IxRp/Y45t1SREREnjjnxs+YdGoMFX7GzarPYIIFa6r+ghJHGsLU/FUSERGByY3/MenViBWKcINyNwDgS/dlMNsqEGYMkzs0IiKiToHDUn5GG6yCPShaOuZCfkRERJ6Y3Pghd6hJ+tnI5IaIiMgDkxs/JOhipZ9NsMBs4x1TREREtZjc+CF9eCRKxeq7pkyChWvdEBER1cPkxg+ZdGqYxfDqnzksRURE5EH25GbVqlVITEyERqPB8OHDceDAgWbL/vjjj7jhhhuQmJgIQRCwfPnyDo21szDp1DAjAgCgF8pQbLPKHRIREVGnIWtys3HjRmRkZGDx4sXIzs7GoEGDkJ6eDrPZ3GT5srIy9OrVC0uXLkVsbGyTZboCk16N/JqeGwBwWXNljYeIiKgzkTW5WbZsGWbNmoUZM2ZgwIABWL16NbRaLdauXdtk+aFDh+Lpp5/GzTffDLVa3WSZrsCk00jDUgCgKMmTNR4iIqLORLZF/JxOJw4ePIgFCxZI5xQKBdLS0rBv3z6fvY7D4YDDUTcnxWaz+ezacjHp1Hjd3Q+aKifMCMdvrjBUVLqgCVLKHRoREZHsZEtuCgoK4HK5EBMT43E+JiYGP//8s89eJzMzE0uWLPHZ9ToDk06D7e5h2O4eJp3LtzuQEKmVNS4iIqLOQPYJxe1twYIFsFqt0uP06dNyh3TB9CEqBKs8f3X5JbxjioiICHL23ERHR0OpVCIvz3O+SF5enk8nC6vV6oCbnyMIAkw6Nc4Ul0vnzDYmN0RERJCz5yY4OBgpKSnIysqSzrndbmRlZWHEiBFyheU3jLrqhE0DB4ywIN/OVYqJiIgg967gGRkZmD59OlJTUzFs2DAsX74cpaWlmDFjBgBg2rRp6NatGzIzM4GaScg//fST9PPZs2dx6NAhhIWFoU+fPnI2pcOZwoLxjXo2ogUbjrh74CP7u3KHRERE1CnImtxMmTIF+fn5WLRoEXJzc5GcnIxt27ZJk4xPnToFhaKuc+ncuXMYPHiwdPzMM8/gmWeewejRo7Fz505Z2iAXkz4EFQgGajfP5LAUERERIHdyAwBz5szBnDlzmnyuYcKSmJgIURQ7KLLOzaSrXsivu1CAaMGGQluJ3CERERF1CrInN9Q21asUG6Rjp40L+RERUTVRFCGKgFjzs1sERFSfAwB3E8+jXhnRo0z1CXfD591izWtVn3eLNa9bc06tUsi2RAmTGz9l0mlwtt4qxYKdyQ0Fns7wAQ3pXF292mvW1nNL5ev9KcXVOAaxidjrX9/zmk3H0FS9+rGjwTVq24Z65xrG0Pg9rT6u/7to+P41ir3mGmjwe/FsX+O4atvZOK6m67nrtVOECLe7ibikvxDumvdQhCiKcAlBHq8T5C6HQnTVBOGuvgDc1e+JKEKECIXoRhk0KBNCpN8RRDeMrprP3prXqH4fRQg1f9ZeI0cwohwaqR2hYini3bkeryeg9o2qi0MEkC328/j7lSieQQwKIdS8jiiK1T/DXfuGQICIfDEch0TP+ah/VmQjFBUQqoOGABEK1NYHBKH652x3XxwXu0n19CjFROXemnI1ZevVRb1zG1x/hh1aDIjT46O5f2zHT4jmMbnxU0adGt+KEdJxcIUZLrcIpUKQNS4ib1RUunDWUo4zxeU4XVRW82cp7EW5QPFJqBxFEES39MF5SozBz2IPj2uMV+yHsubDvLkPaAEidrsG4hyipXqxKESaMlt6XtHgw7r+4xXXeIj1biodqTiMIcKxunpCbT23R/1TYgw2uv7kEe8s5VbECsVQwA14vJbnl8Q291B86R4k1dOhDP8Mes3jy0PhEXvd6z9RNRVnRKNUd4TiR8xUftSonucDsCIU91TO84h3rvL/8EflD1DUaxuaeL+y3IOxrGqyR933ghdBj9JGbRQgQiGIUvuXVE7DJ+6hUr0Bwu94PXhpk7+bhq89wrECVoRJde9R/g8Zqk0QIEIpND994Xt3EiY6H/c4907wEgxTHG22Tq2VVdfimaop0rEGDnykmdViPQCY7FiIY2J/6ThdkY2Xgp9rsV6FGIR+jtc8zk1VfYSpqqxm69T6xJWCuyr/5nHusaD16C4UtFj30co7cNxVl9xECTb8O2hdi/UA4GP3MNhFLeScRMLkxk+Z9GqYUddzY4QFhSUOmPQaWeMiAoBKlxvnGiYvxWXSsdlePQF+tvJ9DFUcxTDBjO5CAbRCzcT4IM/rvVY1FourZnicWx60CkGCq8VYZop/wzl3XXKTpMj1+kN6resvqKqX3Fyh+AGzVR+0WG+va0Cj5OY65R4MUJxsse6pShO+RF1yo4ETNyh3eRXvyqpJAOqSm3gUIk35bYv18kV9o3O9FDlIVfzSYt2fxYRG5xKFXEQILc8DDIHnjRBKuBEteLdFTm2SWEuACJXgbrZ8/XIN1aZPra3rbb3qug1fs231LrSuWxSafqJR3YZt9Z4Cdb19cmFy46eiQtUoqJfcmIRimO1MbqhjuNwicm0V9Xpdqv/MLyyAWPQ7NGVn0R3VCUuCYMYgIR+l0OBG5z89rpOi+AVXevHl6+sPeG8p2vgBr2ii58Drup34C9Ql1u/zqe5LcYmN97SziVoo4G5QtmH/FuCoueOzlgNBOOU2SmXr99u4a96ZugEYz2XazAjH9+6kRq/RsN5v7rhG8X7v7gWHGFTv2p5x1z5+dXfzqOeCAu+7RjRZVoRQM0RW/XM+DB51T4kxeL1qrBQjPNpW9/pVaPz+fu5ORmGVvq4/TfR8v2p/NyfExgvirnBdB52r3ON30tT7/I37Yo96BaIBGc7ZjX6nYqPXFVAgGhq9bkcTxC52+5HNZoPBYIDVaoVe3/h/K/5k2r9fwutVDwMA3q76M2JvXY0/94tpsR5RS9xuEfkljkbJS15hEU5aKnHaWomqmrkqVyi+w4Oqd5Ag5J/3f+ulohqXONZ6fJUuVr2GGartcIhBOCNG44xoxGnRCLMYAVe9QYjDYiJ2uS/zuN505XbpC/R8X4S7XANxtl5vhhEW/FHxvUc9NPNF+Ik71eNLtLdwFkmKvLoBIQGAUDc4BAiAIMAuhOEokgABUAgCBAG4WPwdGqGypvmKuvdBUABCXd0CIQI2ZTgEVNdTiS50Ry7EmjKioIBQ87MAAW5BgFBzDZvCAJcQDEX12ByCRSe0KK8pr6h7bUGoec3qa4kKAS5F9X+MBEGAAEAJV/V1IUBQCNJ5Qah+ZYWi+s/at6G2nULtNWrKCQ2fF1B9TQFQNCjTsJ5CqItHei/rx1BzUHvO4/kmYqgtU7+dCo+yzcTQVDvrX7uZGFATo8d753EsNHrvPNtZFwOaei/RVCyNY2j03tSv500MNc9Deq0WYqhfz4dTJVrz/c2eG3+miwGKq3/kWjfUGqIoorDU2WjYKLfIisrCU1DZTyHObUZ3IR8Jghl/EArQXTDDKNhwk2MRToj9pGsJAC5TnDjv61WKShSIBoShHCWou3viraAb8Fn4VOgi49EtMhQJkVokRGgxWK+GSqGQPihvaPKLYky9L6rq8019WT5Y7+e6L6rJ5//Ab/JLmfPZiPwFkxs/FqSPxYy8h2AWw5EjRuF2O5MbqiaKImzlVThdXOaRvOQU2VFReAq51gocq6ybhyLAjV3qeYhHYfWQihJoojccAJAgmPF1veTmjBgNlyggB1HVPS9uI86I1Y+CoFi4DQkIiUpAt8gwZERo0T0iBAmR1X/qNEFNvwgR0QVgcuPHjHotNrjrVmw2c3+pLqXEUdVo2OhMkR3lRWehtJxEZGUuEoR8dBfyMVCRj78I+YhDIZSCiPcxAvfjPulaIhQQRQEKRfOj1LliBM6IRpSibl5XSJASyvC+mB35MeIidUioSV7G1iQvhpAg9ngQUYdjcuPHTDrP3c45LBVYyp0unLWU4XRROc4Ul+F0cTlOF5aitOgcYDkJvSMPW92Xe8xhWaJah+mqHdWngpu/dnchv9G578ReKHAbcEaMxmnRhDOiETmCCS59AlSRPRAbFY7uESG4JkKL2TW9L1GhwUxeiKjTYXLjx4wN7oz61VyCnUfNiA8PQZxBwy7/Ts5R5cI5S0WjW6VzCy0IKT4KXflZac5LglCAkYIZ3YSC6kmpqE5evqwYCFu9tT7OitHNvl6xGCYlLkfc1WvGqBQC4sND0D0iBLsinpGGjIZFhOCGSC2MYWqfTggkIuoITG78mEmnRi/hHC4RfscDqndRYgvBoTf64CsxCufESFiDTHDruyMoohtM4TrEGzSICw9BfLgG8YYQxBo00AQ1M7GCLliVy40ca0V10lKv96WoIA9i8Sloy8+gG/JxwN0fP4i9pHoXCaexST3/vD0vtRKEAvwo1iU3P4qJ+MSVIt11dBZGlGu7QxHZE1FR0dKw0ahILaZEhCBWr4FKqTjvaxAR+RsmN34sVq/BBMU+PBD0f9K5Rnet2AG3TUD+SQMyKu/GHvdA6SkDSpCsLYBo6IaQ8HjERYQirjYBqvkzRqfml18zXG4RebaKJhaqqx5KSrJ/gz44XdP7ko9Lav7UC2XVF6jpWHu6cjJ+cNUlN/VXmK2vXAzG6ZqJurV/Foh6mHRqaYJuQkQfFEX8P1wcqcWVESGIM4QgWMXfHxF1LUxu/Ngl8Xo8phuH0WXfYaBwotnVWhWCiBhYUCZ6DmMNVxzBy+7ngGLAWaREnhiJc4hCjhiJr8UonBOjkIMoOLTxKI3oLyU91cNe1T1AcYYQRIcF5rwLUaxd66W61+VMcfWfuYUWuIp+R5D9DOLE6tulnVDhuQZL0C8L/j8MV/zc4us0nP9SBg1erfoLCkQDzohG2DVxEMN7QhcVh4TIUHSPCEFipBZ/iAjB38JD2PtGRNQAkxs/plIqsGL2BLy5/zKszrOiwpIDwXoW2oo8xAsFiBeKECcUIr7m0XA+RpxQJP0cLLiQIOQjAY0nmhY5wzDk1MvAKYt07kblF0gQ8nFOjEK+EI0qXTwUhm6IjIhCXE3S0y08RPpZr1F1ugRIFEUUl1U22etSm8z0dv2G8cr96C4UYIRgRoKQD6Ngrb5AvX89uWJEo/11zohGDEddclMpKnFOjPLoeTkjGnE6uDcuMemlIaPqXpjncUWkFt3CQxCq5j9TIqLW4Kemn4sPD8HD4/p5nKuodCHHWoEcSznOWSuwq+bP/pZyhFvLkWOpgN1RhV/E7ni9aqxHAtTUCrM5YlSjcxMVe3GF8oe6E+XVD2uOtrrHp6bn5w33YHzmHoLQYGXNfJ+aIS9DdeITX+/PkGDf90BYyysbJSzniuyoKDwNpfUkol150rDRECEfmyrv9NgJt7fiHO5Vvd/i68QKxQhGJZz1NkXa5BqNva5LUKCKgTu8B0IiuyM+UicNIV0WoUX3yBDoOfGbiMinmNwEIE2QEknRoUiKDm22jL2iEjnWK3DWcjtyLBX43lqOc5YKFBUXo8pyBsqSczC68xGPQljq3Y1Tq36vT30GoQwGoQz9cRoAUFilw2fuISh1uvCruQQnzRbsVt+PXDESOWIUjomR+KImGbKrY+DWd4cmIg6x4WHViVBNz0+cQYNYgwZBDeb/lDqq6s15qZ6wWz+ZsVVUQQkXMlWv4hIhH+lCPuKEwuoN9hQ1q+DX00vI8UhuTosmj+fzxPCa26TrbpfOU5jgMvTAqB6x6B4ZVq/3ZRQSIrQI13KtFyKijsTkpovSaYKg0wThohhdk8+LooiiUidyrBU4ZylHr5o/z9X0CC0pfhjq0jOIRSHihOpHN6EQcShErFAEtVAFNNHrEysUIkawIEawYBB+a/CiAKxAlUWBPETgdud8HBO7S0+bhGL0Dy2FaEgARBcEyykYKs6hu5CP7oIZvYV8jBbysdOdjO1V06V6LihxlfIbhAulLb4vMUKxx/EvYnfcUbUAVfruUEX2RExkuJS8jIoIQUKENmDnHBER+SsmN9QkQRAQFaZGVJgal3ZraofXkXC5ReTbHThrKUeOtRw/WCqw3VqO3OIylFpyIVrP4ucKz+RJj3LkiJEwoRjKJnZOBgCV4EY3FMIqevY8Xa34Cv+seh0orHeyidulT4k5jc6dFo0IF0phEUM95rycFo04BxMcYd2hjOgJY1QkMiLr5r4kRGhh0nGtFyIif8LkhtpMqRAQWzNcBEQ0WcZR5UKe1YFz1nKcs5Qjx3oxVlquQJ6lFI7iMxBsZ2FwmhFf0/tT+6dRsKIAnklVnFDY5GvUVyJq4Gzw11oQgEWaR6APNyIqKhrda5KXSyO0GBdRPeTF292JiAIHkxtqV2qVEj2itOgRpW22TKmjCjk1c35yrOU4bKke+hpprcC5mgnQ5ZUuZLsvwttV5YgXCuGG4NEDU6LtBiG8B8IjY9A9UovMml6X7hHVk5i51gsRUdfB5IZkF6pWoY9Jhz6m5uf/WMsrcdbyB+RYKnDaWg4IAhIiQjCyJoHhWi9ERFSLyQ11eoIgIFwbjHBtMC6Jb2r+DxERUR321RMREVFAYXJDREREAYXJDREREQUUJjdEREQUUJjcEBERUUBhckNEREQBhckNERERBRQmN0RERBRQmNwQERFRQGFyQ0RERAGFyQ0REREFFCY3REREFFCY3BAREVFA6RTJzapVq5CYmAiNRoPhw4fjwIED5y2/adMm9OvXDxqNBgMHDsRHH33UYbESERFR5yZ7crNx40ZkZGRg8eLFyM7OxqBBg5Ceng6z2dxk+b179+KWW27BzJkz8e2332LSpEmYNGkSDh8+3OGxExERUecjiKIoyhnA8OHDMXToUKxcuRIA4Ha7kZCQgPvuuw+PPPJIo/JTpkxBaWkptm7dKp27/PLLkZycjNWrV7f4ejabDQaDAVarFXq93setISIiovbQmu9vVYdF1QSn04mDBw9iwYIF0jmFQoG0tDTs27evyTr79u1DRkaGx7n09HRs2bKlyfIOhwMOh0M6ttlsPou/Ka/9+Bpe/+n1FssNiByAFVeu8Dh3X9Z9+KnopxbrThswDdMvmS4dl1aWYuKWiV7F98KfX8AlUZdIx1+c/gKP7X+sxXpalRYfXPeBx7lnv3kWH51oeUjwiu5XYPGIxR7npmydgoLyghbrZqRkYHyv8dLxCesJ3PnJnS3WA4AN4zfAqDVKx5t+2YTV37WcACfqE7EmfY3Huflfzsc3ed+0WPfGvjfi7uS7Pc5duelKr+Jd+selGBo7VDr+OvdrPLKrcYLflKybsjyOXzz0It499m6L9VJjUvHkFU96nJu5fSZ+t/3eYt3Zg2bjpotuko7zy/Jx84c3exXvq1e9iiRDknT84W8fYtnBZS3Wiw6JxsZrNnqcW7JvCb4882WLda9Ouhp/S/2bx7kJ701AWVVZi3UXXb4IoxNGS8c/Fv6I+z+7v8V6APD+pPcRGhQqHfMzojF+RgTeZ0TDv4MdTdbkpqCgAC6XCzExMR7nY2Ji8PPPPzdZJzc3t8nyubm5TZbPzMzEkiVLfBj1+ZVWlsJc1vSQWn2xobGNzhU5iryqW1pZ6nEsiqJX9QCg0lXpcVzhqvCqbv0P51o2p82rulaHtdG5gvICr+pWVFV4HLvcLq/b6hJdHsdllWVe1dUF6RqdszgsXtW1V9obnfM2XqfL2ejY27pNxeFNXYvD0uhcYXmhV3XLKj2TApfYit+N2/N3U1Hl3d/DplgdVq/q2pyN/2OTX57f6N9TUypcnn8PK12VXsfbsHOcnxGN8TMi8D4jvPl31Z5kTW46woIFCzx6emw2GxISEtrt9UKDQmHSmlosF6mObPKcN3UbfogIguBVPQAIUgZ5HGuUGq/qalXaRuf0wXqv6hrUhkbnokOiW6wHABqVxuNYqVB63ValoPQ41gZpvaobFRLV6Fy4Otyruk196Hkbb7AyuNGxt3WbisObuuHq8EbnokKimvwAbkgb5Pl3Qim04nej8PzdaFTe/T1s6u+NQW3wqq4+uHE3tjHE2OSXckMapeffwyBlkNdtFQTB45ifEY3xMyLwPiO8+XfVnmSdc+N0OqHVavHuu+9i0qRJ0vnp06fDYrHgf//7X6M6PXr0QEZGBubNmyedW7x4MbZs2YLvvvuuxdfknBsiIiL/05rvb1nvlgoODkZKSgqysurGAt1uN7KysjBixIgm64wYMcKjPADs2LGj2fJERETUtcg+LJWRkYHp06cjNTUVw4YNw/Lly1FaWooZM2YAAKZNm4Zu3bohMzMTADB37lyMHj0azz77LMaPH48NGzbgm2++wcsvvyxzS4iIiKgzkD25mTJlCvLz87Fo0SLk5uYiOTkZ27ZtkyYNnzp1CgpFXQfTyJEj8fbbb+Mf//gH/v73v6Nv377YsmULLr30UhlbQURERJ2F7OvcdDTOuSEiIvI/fjPnhoiIiMjXmNwQERFRQGFyQ0RERAGFyQ0REREFFCY3REREFFCY3BAREVFAYXJDREREAYXJDREREQUUJjdEREQUUGTffqGj1S7IbLPZ5A6FiIiIvFT7ve3NxgpdLrmx2+0AgISEBLlDISIiolay2+0wGAznLdPl9pZyu904d+4cdDodBEGQOxyfsNlsSEhIwOnTp7vEfllsb2BjewNbV2svumCb26u9oijCbrcjPj7eY0PtpnS5nhuFQoHu3bvLHUa70Ov1XeIfTi22N7CxvYGtq7UXXbDN7dHelnpsanFCMREREQUUJjdEREQUUJjcBAC1Wo3FixdDrVbLHUqHYHsDG9sb2Lpae9EF29wZ2tvlJhQTERFRYGPPDREREQUUJjdEREQUUJjcEBERUUBhckNEREQBhcmNn8jMzMTQoUOh0+lgMpkwadIkHD161KNMRUUF7r33XkRFRSEsLAw33HAD8vLyZIvZl5YuXQpBEDBv3jzpXKC19+zZs7j11lsRFRWFkJAQDBw4EN988430vCiKWLRoEeLi4hASEoK0tDQcO3ZM1pjbyuVyYeHChUhKSkJISAh69+6Nf/3rXx57xvh7e7/88ktMmDAB8fHxEAQBW7Zs8Xjem/YVFRVh6tSp0Ov1CA8Px8yZM1FSUtLBLfHO+dpbWVmJ+fPnY+DAgQgNDUV8fDymTZuGc+fOeVwjUNrb0OzZsyEIApYvX+5xPtDae+TIEUycOBEGgwGhoaEYOnQoTp06JT3fkZ/ZTG78xBdffIF7770X+/fvx44dO1BZWYmrrroKpaWlUpkHHngAH3zwATZt2oQvvvgC586dw/XXXy9r3L7w9ddf46WXXsJll13mcT6Q2ltcXIxRo0YhKCgIH3/8MX766Sc8++yziIiIkMo89dRTeOGFF7B69Wp89dVXCA0NRXp6OioqKmSNvS2efPJJvPjii1i5ciWOHDmCJ598Ek899RRWrFghlfH39paWlmLQoEFYtWpVk897076pU6fixx9/xI4dO7B161Z8+eWXuOuuuzqwFd47X3vLysqQnZ2NhQsXIjs7G5s3b8bRo0cxceJEj3KB0t763nvvPezfvx/x8fGNnguk9h4/fhx/+MMf0K9fP+zcuRPff/89Fi5cCI1GI5Xp0M9skfyS2WwWAYhffPGFKIqiaLFYxKCgIHHTpk1SmSNHjogAxH379skY6YWx2+1i3759xR07doijR48W586dK4oB2N758+eLf/jDH5p93u12i7GxseLTTz8tnbNYLKJarRb/+9//dlCUvjN+/Hjxjjvu8Dh3/fXXi1OnThXFAGwvAPG9996Tjr1p308//SQCEL/++mupzMcffywKgiCePXu2g1vQOg3b25QDBw6IAMSTJ0+KYoC298yZM2K3bt3Ew4cPiz179hSfe+456blAa++UKVPEW2+9tdk6Hf2ZzZ4bP2W1WgEAkZGRAICDBw+isrISaWlpUpl+/fqhR48e2Ldvn2xxXqh7770X48eP92gXArC977//PlJTU3HTTTfBZDJh8ODBeOWVV6TnT5w4gdzcXI/2GgwGDB8+3C/bO3LkSGRlZeGXX34BAHz33XfYvXs3/vKXvwAB2N6GvGnfvn37EB4ejtTUVKlMWloaFAoFvvrqK1ni9iWr1QpBEBAeHg4EYHvdbjduu+02PPTQQ7jkkksaPR9I7XW73fjwww9x0UUXIT09HSaTCcOHD/cYuuroz2wmN37I7XZj3rx5GDVqFC699FIAQG5uLoKDg6UPiloxMTHIzc2VKdILs2HDBmRnZyMzM7PRc4HW3t9++w0vvvgi+vbti+3bt+Puu+/G/fffj9deew2oaS9q2lefv7b3kUcewc0334x+/fohKCgIgwcPxrx58zB16lQgANvbkDfty83Nhclk8nhepVIhMjLS79+DiooKzJ8/H7fccou0sWKgtffJJ5+ESqXC/fff3+TzgdRes9mMkpISLF26FOPGjcMnn3yC6667Dtdffz2++OILQIbP7C63K3gguPfee3H48GHs3r1b7lDazenTpzF37lzs2LHDY8w2ULndbqSmpuKJJ54AAAwePBiHDx/G6tWrMX36dLnD87l33nkHb731Ft5++21ccsklOHToEObNm4f4+PiAbC/VqaysxOTJkyGKIl588UW5w2kXBw8exPPPP4/s7GwIgiB3OO3O7XYDAK699lo88MADAIDk5GTs3bsXq1evxujRozs8Jvbc+Jk5c+Zg69at+Pzzz9G9e3fpfGxsLJxOJywWi0f5vLw8xMbGyhDphTl48CDMZjOGDBkClUoFlUqFL774Ai+88AJUKhViYmICqr1xcXEYMGCAx7n+/ftLdxrUtqnhnQX+2t6HHnpI6r0ZOHAgbrvtNjzwwANSL12gtbchb9oXGxsLs9ns8XxVVRWKior89j2oTWxOnjyJHTt2SL02CLD27tq1C2azGT169JA+v06ePIm//e1vSExMBAKsvdHR0VCpVC1+hnXkZzaTGz8hiiLmzJmD9957D5999hmSkpI8nk9JSUFQUBCysrKkc0ePHsWpU6cwYsQIGSK+MFdeeSV++OEHHDp0SHqkpqZi6tSp0s+B1N5Ro0Y1urX/l19+Qc+ePQEASUlJiI2N9WivzWbDV1995ZftLSsrg0Lh+fGjVCql/wEGWnsb8qZ9I0aMgMViwcGDB6Uyn332GdxuN4YPHy5L3BeiNrE5duwYPv30U0RFRXk8H0jtve222/D99997fH7Fx8fjoYcewvbt24EAa29wcDCGDh163s+wDv+O8vkUZWoXd999t2gwGMSdO3eKOTk50qOsrEwqM3v2bLFHjx7iZ599Jn7zzTfiiBEjxBEjRsgaty/Vv1tKDLD2HjhwQFSpVOLjjz8uHjt2THzrrbdErVYrvvnmm1KZpUuXiuHh4eL//vc/8fvvvxevvfZaMSkpSSwvL5c19raYPn262K1bN3Hr1q3iiRMnxM2bN4vR0dHiww8/LJXx9/ba7Xbx22+/Fb/99lsRgLhs2TLx22+/le4O8qZ948aNEwcPHix+9dVX4u7du8W+ffuKt9xyi4ytat752ut0OsWJEyeK3bt3Fw8dOuTxGeZwOKRrBEp7m9LwbikxwNq7efNmMSgoSHz55ZfFY8eOiStWrBCVSqW4a9cu6Rod+ZnN5MZPAGjysW7dOqlMeXm5eM8994gRERGiVqsVr7vuOjEnJ0fWuH2pYXITaO394IMPxEsvvVRUq9Viv379xJdfftnjebfbLS5cuFCMiYkR1Wq1eOWVV4pHjx6VLd4LYbPZxLlz54o9evQQNRqN2KtXL/HRRx/1+KLz9/Z+/vnnTf6bnT59uih62b7CwkLxlltuEcPCwkS9Xi/OmDFDtNvtMrXo/M7X3hMnTjT7Gfb5559L1wiU9jalqeQm0Nq7Zs0asU+fPqJGoxEHDRokbtmyxeMaHfmZLYj1lwQlIiIi8nOcc0NEREQBhckNERERBRQmN0RERBRQmNwQERFRQGFyQ0RERAGFyQ0REREFFCY3REREFFCY3BAREVFAYXJDRF5LTEzE8uXLvS6/c+dOCILQaLM8qjNmzBjMmzdP7jCIAgqTG6IAJAjCeR///Oc/23Tdr7/+GnfddZfX5UeOHImcnBwYDIY2vZ63apMoQRCgUChgMBgwePBgPPzww8jJyWn19QRBwJYtWy44LpfLhaVLl6Jfv34ICQlBZGQkhg8fjldffVUqs3nzZvzrX/+64NciojoquQMgIt+r/4W+ceNGLFq0yGPH3rCwMOlnURThcrmgUrX8cWA0GlsVR3BwMGJjY1tV50IcPXoUer0eNpsN2dnZeOqpp7BmzRrs3LkTAwcO7LA4ai1ZsgQvvfQSVq5cidTUVNhsNnzzzTcoLi6WykRGRnZ4XESBjj03RAEoNjZWehgMBgiCIB3//PPP0Ol0+Pjjj5GSkgK1Wo3du3fj+PHjuPbaaxETE4OwsDAMHToUn376qcd1Gw5LCYKAV199Fddddx20Wi369u2L999/X3q+4bDU+vXrER4eju3bt6N///4ICwvDuHHjPJKxqqoq3H///QgPD0dUVBTmz5+P6dOnY9KkSS2222QyITY2FhdddBFuvvlm7NmzB0ajEXfffbdU5uuvv8bYsWMRHR0Ng8GA0aNHIzs726ONAHDddddBEATp2Jv3p6H3338f99xzD2666SYkJSVh0KBBmDlzJh588EGpTP1hqfo9UPUft99+u1T+f//7H4YMGQKNRoNevXphyZIlqKqqavG9IepKmNwQdVGPPPIIli5diiNHjuCyyy5DSUkJrr76amRlZeHbb7/FuHHjMGHCBJw6deq811myZAkmT56M77//HldffTWmTp2KoqKiZsuXlZXhmWeewRtvvIEvv/wSp06d8viyf/LJJ/HWW29h3bp12LNnD2w2W5uHiEJCQjB79mzs2bMHZrMZAGC32zF9+nTs3r0b+/fvR9++fXH11VfDbrcDNckPAKxbtw45OTnScVven9jYWHz22WfIz8/3Kt7aYbzax2effQaNRoMrrrgCALBr1y5MmzYNc+fOxU8//YSXXnoJ69evx+OPP96m94coYLXLXuNE1GmsW7dONBgM0vHnn38uAhC3bNnSYt1LLrlEXLFihXTcs2dP8bnnnpOOAYj/+Mc/pOOSkhIRgPjxxx97vFZxcbEUCwDx119/leqsWrVKjImJkY5jYmLEp59+WjquqqoSe/ToIV577bXNxtnwder7+OOPRQDiV1991WRdl8sl6nQ68YMPPvBo13vvvdfCu9P4/Wnoxx9/FPv37y8qFApx4MCB4l//+lfxo48+8igzevRoce7cuY3qFhQUiL169RLvuece6dyVV14pPvHEEx7l3njjDTEuLq7FWIm6EvbcEHVRqampHsclJSV48MEH0b9/f4SHhyMsLAxHjhxpsefmsssuk34ODQ2FXq+XekmaotVq0bt3b+k4Li5OKm+1WpGXl4dhw4ZJzyuVSqSkpLSpjaiZU4SaITQAyMvLw6xZs9C3b18YDAbo9XqUlJS02M62vD8DBgzA4cOHsX//ftxxxx0wm82YMGEC7rzzzvO+VmVlJW644Qb07NkTzz//vHT+u+++w2OPPYawsDDpMWvWLOTk5KCsrKyV7wxR4OKEYqIuKjQ01OP4wQcfxI4dO/DMM8+gT58+CAkJwY033gin03ne6wQFBXkcC4IAt9vdqvK1CUh7OHLkCFBvLs306dNRWFiI559/Hj179oRarcaIESNabGdb3x+FQoGhQ4di6NChmDdvHt58803cdtttePTRR5GUlNRknbvvvhunT5/GgQMHPCZ6l5SUYMmSJbj++usb1dFoNF69H0RdAZMbIgIA7NmzB7fffjuuu+46oOaL9Pfff+/QGAwGA2JiYvD1119L80xcLheys7ORnJzc6uuVl5fj5ZdfxhVXXCHd6bVnzx785z//wdVXXw0AOH36NAoKCjzqBQUFweVyeZzz1fszYMAAAEBpaWmTzy9btgzvvPMO9u7di6ioKI/nhgwZgqNHj6JPnz6tfl2iroTJDREBAPr27YvNmzdjwoQJEAQBCxcuPG8PTHu57777kJmZiT59+qBfv35YsWIFiouLpWGl8zGbzaioqIDdbsfBgwfx1FNPoaCgAJs3b5bK9O3bF2+88YZ0a/ZDDz2EkJAQj+skJiYiKysLo0aNglqtRkRERJvenxtvvBGjRo3CyJEjERsbixMnTmDBggW46KKL0K9fv0blP/30Uzz88MNYtWoVoqOjkZubC9RMjDYYDFi0aBGuueYa9OjRAzfeeCMUCgW+++47HD58GP/+979b8S4TBTbOuSEioKbHICIiAiNHjsSECROQnp6OIUOGdHgc8+fPxy233IJp06ZhxIgRCAsLQ3p6ulfDLhdffDHi4+ORkpKCpUuXIi0tDYcPH5Z6SwBgzZo1KC4uxpAhQ3Dbbbfh/vvvh8lk8rjOs88+ix07diAhIQGDBw8G2vj+pKen44MPPsCECRNw0UUXYfr06ejXrx8++eSTJtcV2r17N1wuF2bPno24uDjpMXfuXOl6W7duxSeffIKhQ4fi8ssvx3PPPYeePXt6/f4SdQWC2J6D3UREF8jtdqN///6YPHkyV/IlIq9wWIqIOpWTJ0/ik08+wejRo+FwOLBy5UqcOHEC/+///T+5QyMiP8FhKSLqVBQKBdavX4+hQ4di1KhR+OGHH/Dpp5+if//+codGRH6Cw1JEREQUUNhzQ0RERAGFyQ0REREFFCY3REREFFCY3BAREVFAYXJDREREAYXJDREREQUUJjdEREQUUJjcEBERUUD5/wHeYwdJGFN+AAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "p = 3\n", "\n", "\n", "biases = []\n", "variances = []\n", "mses = []\n", "\n", "p_degrees = list(range(1, 5))\n", "N_values = [10, 20, 40, 80, 160]\n", "for N in N_values:\n", " n = N\n", "\n", " x = np.linspace(-3, 3, n)\n", " X = PolynomialFeatures(degree=p).fit_transform(x.reshape(-1, 1))\n", " y = np.exp(-(x**2)) + 1.5 * np.exp(-((x - 2) ** 2)) + np.random.normal(0, 0.1, size=n)\n", " predictions = np.zeros((bootstraps, int(n*0.2)), dtype=float)\n", " targets = np.zeros((bootstraps, int(n*0.2)), dtype=float)\n", "\n", " X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, shuffle=False)\n", " for b in range(bootstraps):\n", " x_sample, y_sample = resample(X_train, y_train)\n", " \n", " model = LinearRegression().fit(X_train, y_train)\n", "\n", " predictions[b, :] = model.predict(X_test)\n", " targets[b, :] = y_test\n", " mse, bias, variance = calculate_key_metrics(predictions, targets)\n", " mses.append(mse)\n", " biases.append(bias)\n", " variances.append(variance)\n", "\n", "plt.plot(N_values, np.array(mses), label=\"MSE\", lw=3)\n", "plt.plot(N_values, biases, label=\"Bias^2\", linestyle=\"dashed\", lw=2)\n", "plt.plot(N_values, variances, label=\"Variance\", linestyle=\"dashed\", lw=2)\n", "#plt.plot(range(1, 5), np.array(biases) + np.array(variances), label=\"Bias^2 + Variance\", linestyle=\"dashed\")\n", "plt.xlabel(\"Training Data Size\")\n", "plt.ylabel(\"Error\")\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "46250fbc", "metadata": {}, "source": [ "## Exercise 5: Interpretation of scaling and metrics\n" ] }, { "cell_type": "markdown", "id": "5af53055", "metadata": {}, "source": [ "In this course, we often ask you to scale data and compute various metrics. Although these practices are \"standard\" in the field, we will require you to demonstrate an understanding of _why_ you need to scale data and use these metrics. Both so that you can make better arguements about your results, and so that you will hopefully make fewer mistakes.\n", "\n", "First, a few reminders: In this course you should always scale the columns of the feature matrix, and sometimes scale the target data, when it is worth the effort. By scaling, we mean subtracting the mean and dividing by the standard deviation, though there are many other ways to scale data. When scaling either the feature matrix or the target data, the intercept becomes a bit harder to implement and understand, so take care.\n", "\n", "Briefly answer the following:\n", "\n", "**a)** Why do we scale data?\n" ] }, { "cell_type": "markdown", "id": "809a620d", "metadata": {}, "source": [ "
\n", "Because it allows for\n", "\n", "- You can get a feeling for parameter values\n", "- Similar Hyperparameters are usable over multiple different problem sets\n", "- No Numerical Inaccuracies due to float limitations\n", "- No Bias of Features in regualrized regression (see next questions)\n", "- There's no major disadvantage\n", "
\n" ] }, { "cell_type": "markdown", "id": "fc04426b", "metadata": {}, "source": [ "\n", "\n", "**b)** Why does the OLS method give practically equivelent models on scaled and unscaled data?\n" ] }, { "cell_type": "markdown", "id": "b2028408", "metadata": {}, "source": [ "
\n", "Because the cost function depends only on the deviation between output and target variables. There is no influence of parameter values on the cost value. Thus the minimum of the cost function is invariant wrt to scaling of the features.\n", "
\n" ] }, { "cell_type": "markdown", "id": "48c44d22", "metadata": {}, "source": [ "\n", "**c)** Why does the Ridge method **not** give practically equivelent models on scaled and unscaled data? Why do we only consider the model on scaled data correct?\n" ] }, { "cell_type": "markdown", "id": "2b19fbb4", "metadata": {}, "source": [ "
\n", " Because the cost function directly depends on the parameter values. But if for example we scale feature 1 by an factor of 3, the parameter of feature 1 needs to be multiplied by 1/3 for an optimal result. But because the cost function also depends on the parameter values, this may not be the minimum of the cost function anymore.\n", "
\n" ] }, { "cell_type": "markdown", "id": "d9979876", "metadata": {}, "source": [ "\n", "**d)** Why do we say that the Ridge method gives a biased model?\n" ] }, { "cell_type": "markdown", "id": "c9265015", "metadata": {}, "source": [ "
\n", "\n", "The expectation value for the ridge parameters only approaches the true value $\\beta$ in the limit $\\lambda \\to 0$, this is why we call it biased. As independent of number of training samples our estimate will always differ from the true value given $\\lambda \\neq 0$ (in which case it would be OLS). The larger the parameter value (i.e. parameter not equal to zero), the larger our cost value. Even though it may be needed that the paramter is bigger for our targets and predictions to align perfectly. This may be a problem if the goal of our analysis is the closest possible parameter estimates given near infinite number of training samples (because in this case the deviation from the true value approaches 0).\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "8287f760", "metadata": {}, "source": [ "\n", "**e)** Is the MSE of the OLS method affected by scaling of the feature matrix? Is it affected by scaling of the target data?\n" ] }, { "cell_type": "markdown", "id": "edcdff21", "metadata": {}, "source": [ "
\n", "\n", " It is only affected by scaling of the target data. The MSE is in units of $[y]^2$. So if our scaled unit is $[y_{scaled}] = 10[y]$, then our MSE will be scaled by a factor of 100.\n", "\n", "
\n" ] }, { "cell_type": "markdown", "id": "5145de2d", "metadata": {}, "source": [ "\n", "**f)** Read about the R2 score, a metric we will ask you to use a lot later in the course. Is the R2 score of the OLS method affected by scaling of the feature matrix? Is it affected by scaling of the target data?\n" ] }, { "cell_type": "markdown", "id": "a552db5a", "metadata": {}, "source": [ "
\n", " The R2 score is invariant under scaling of the feature and target data. \n", "
\n" ] }, { "cell_type": "markdown", "id": "23517608", "metadata": {}, "source": [ "\n", "**g)** Give interpretations of the following R2 scores: 0, 0.5, 1.\n" ] }, { "cell_type": "markdown", "id": "03e5cb8c", "metadata": {}, "source": [ "
\n", " \n", "- 0: There is no correlation between the prediction and the target values. The target values can be equally good described by just giving the mean of the values.\n", "- 0.5: Half of the variation in the target values can be explained by the models predictions.\n", "- 1: The prediction perfectly aligns with the target values.\n", "\n", "
\n" ] }, { "cell_type": "markdown", "id": "5938cb96", "metadata": {}, "source": [ "\n", "**h)** What is an advantage of the R2 score over the MSE?\n" ] }, { "cell_type": "markdown", "id": "a73b6cd3", "metadata": {}, "source": [ "
\n", "\n", "- It is invariant under scaling of the target values\n", "- It has a limited range of values with a particular meaning.\n", "\n", "
\n" ] } ], "metadata": { "kernelspec": { "display_name": "lecture-materials", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.7" } }, "nbformat": 4, "nbformat_minor": 5 }