FYS-STK4155/doc/LectureNotes/_build/jupyter_execute/chapter3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Resampling Methods\n",
    "\n",
    "## Introduction\n",
    "\n",
    "Resampling methods are an indispensable tool in modern\n",
    "statistics. They involve repeatedly drawing samples from a training\n",
    "set and refitting a model of interest on each sample in order to\n",
    "obtain additional information about the fitted model. For example, in\n",
    "order to estimate the variability of a linear regression fit, we can\n",
    "repeatedly draw different samples from the training data, fit a linear\n",
    "regression to each new sample, and then examine the extent to which\n",
    "the resulting fits differ. Such an approach may allow us to obtain\n",
    "information that would not be available from fitting the model only\n",
    "once using the original training sample.\n",
    "\n",
    "Two resampling methods are often used in Machine Learning analyses,\n",
    "1. The **bootstrap method**\n",
    "\n",
    "2. and **Cross-Validation**\n",
    "\n",
    "In addition there are several other methods such as the Jackknife and the Blocking methods. We will discuss in particular\n",
    "cross-validation and the bootstrap method. \n",
    "\n",
    "\n",
    "Resampling approaches can be computationally expensive, because they\n",
    "involve fitting the same statistical method multiple times using\n",
    "different subsets of the training data. However, due to recent\n",
    "advances in computing power, the computational requirements of\n",
    "resampling methods generally are not prohibitive. In this chapter, we\n",
    "discuss two of the most commonly used resampling methods,\n",
    "cross-validation and the bootstrap. Both methods are important tools\n",
    "in the practical application of many statistical learning\n",
    "procedures. For example, cross-validation can be used to estimate the\n",
    "test error associated with a given statistical learning method in\n",
    "order to evaluate its performance, or to select the appropriate level\n",
    "of flexibility. The process of evaluating a model’s performance is\n",
    "known as model assessment, whereas the process of selecting the proper\n",
    "level of flexibility for a model is known as model selection. The\n",
    "bootstrap is widely used.\n",
    "\n",
    "\n",
    "* Our simulations can be treated as *computer experiments*. This is particularly the case for Monte Carlo methods\n",
    "\n",
    "* The results can be analysed with the same statistical tools as we would use analysing experimental data.\n",
    "\n",
    "* As in all experiments, we are looking for expectation values and an estimate of how accurate they are, i.e., possible sources for errors.\n",
    "\n",
    "## Reminder on Statistics\n",
    "\n",
    "\n",
    "* As in other experiments, many numerical  experiments have two classes of errors:\n",
    "\n",
    "  * Statistical errors\n",
    "\n",
    "  * Systematical errors\n",
    "\n",
    "\n",
    "* Statistical errors can be estimated using standard tools from statistics\n",
    "\n",
    "* Systematical errors are method specific and must be treated differently from case to case. \n",
    "\n",
    "The\n",
    "advantage of doing linear regression is that we actually end up with\n",
    "analytical expressions for several statistical quantities.  \n",
    "Standard least squares and Ridge regression  allow us to\n",
    "derive quantities like the variance and other expectation values in a\n",
    "rather straightforward way.\n",
    "\n",
    "\n",
    "It is assumed that $\\varepsilon_i\n",
    "\\sim \\mathcal{N}(0, \\sigma^2)$ and the $\\varepsilon_{i}$ are\n",
    "independent, i.e.:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{align*} \n",
    "\\mbox{Cov}(\\varepsilon_{i_1},\n",
    "\\varepsilon_{i_2}) & = \\left\\{ \\begin{array}{lcc} \\sigma^2 & \\mbox{if}\n",
    "& i_1 = i_2, \\\\ 0 & \\mbox{if} & i_1 \\not= i_2.  \\end{array} \\right.\n",
    "\\end{align*}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The randomness of $\\varepsilon_i$ implies that\n",
    "$\\mathbf{y}_i$ is also a random variable. In particular,\n",
    "$\\mathbf{y}_i$ is normally distributed, because $\\varepsilon_i \\sim\n",
    "\\mathcal{N}(0, \\sigma^2)$ and $\\mathbf{X}_{i,\\ast} \\, \\boldsymbol{\\beta}$ is a\n",
    "non-random scalar. To specify the parameters of the distribution of\n",
    "$\\mathbf{y}_i$ we need to calculate its first two moments. \n",
    "\n",
    "Recall that $\\boldsymbol{X}$ is a matrix of dimensionality $n\\times p$. The\n",
    "notation above $\\mathbf{X}_{i,\\ast}$ means that we are looking at the\n",
    "row number $i$ and perform a sum over all values $p$.\n",
    "\n",
    "\n",
    "The assumption we have made here can be summarized as (and this is going to be useful when we discuss the bias-variance trade off)\n",
    "that there exists a function $f(\\boldsymbol{x})$ and  a normal distributed error $\\boldsymbol{\\varepsilon}\\sim \\mathcal{N}(0, \\sigma^2)$\n",
    "which describe our data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\boldsymbol{y} = f(\\boldsymbol{x})+\\boldsymbol{\\varepsilon}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We approximate this function with our model from the solution of the linear regression equations, that is our\n",
    "function $f$ is approximated by $\\boldsymbol{\\tilde{y}}$ where we want to minimize $(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2$, our MSE, with"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\boldsymbol{\\tilde{y}} = \\boldsymbol{X}\\boldsymbol{\\beta}.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can calculate the expectation value of $\\boldsymbol{y}$ for a given element $i$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{align*} \n",
    "\\mathbb{E}(y_i) & =\n",
    "\\mathbb{E}(\\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta}) + \\mathbb{E}(\\varepsilon_i)\n",
    "\\, \\, \\, = \\, \\, \\, \\mathbf{X}_{i, \\ast} \\, \\beta, \n",
    "\\end{align*}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "while\n",
    "its variance is"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{align*} \\mbox{Var}(y_i) & = \\mathbb{E} \\{ [y_i\n",
    "- \\mathbb{E}(y_i)]^2 \\} \\, \\, \\, = \\, \\, \\, \\mathbb{E} ( y_i^2 ) -\n",
    "[\\mathbb{E}(y_i)]^2  \\\\  & = \\mathbb{E} [ ( \\mathbf{X}_{i, \\ast} \\,\n",
    "\\beta + \\varepsilon_i )^2] - ( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta})^2 \\\\ &\n",
    "= \\mathbb{E} [ ( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta})^2 + 2 \\varepsilon_i\n",
    "\\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta} + \\varepsilon_i^2 ] - ( \\mathbf{X}_{i,\n",
    "\\ast} \\, \\beta)^2 \\\\  & = ( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta})^2 + 2\n",
    "\\mathbb{E}(\\varepsilon_i) \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta} +\n",
    "\\mathbb{E}(\\varepsilon_i^2 ) - ( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta})^2 \n",
    "\\\\ & = \\mathbb{E}(\\varepsilon_i^2 ) \\, \\, \\, = \\, \\, \\,\n",
    "\\mbox{Var}(\\varepsilon_i) \\, \\, \\, = \\, \\, \\, \\sigma^2.  \n",
    "\\end{align*}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hence, $y_i \\sim \\mathcal{N}( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta}, \\sigma^2)$, that is $\\boldsymbol{y}$ follows a normal distribution with \n",
    "mean value $\\boldsymbol{X}\\boldsymbol{\\beta}$ and variance $\\sigma^2$ (not be confused with the singular values of the SVD). \n",
    "\n",
    "\n",
    "With the OLS expressions for the parameters $\\boldsymbol{\\beta}$ we can evaluate the expectation value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\mathbb{E}(\\boldsymbol{\\beta}) = \\mathbb{E}[ (\\mathbf{X}^{\\top} \\mathbf{X})^{-1}\\mathbf{X}^{T} \\mathbf{Y}]=(\\mathbf{X}^{T} \\mathbf{X})^{-1}\\mathbf{X}^{T} \\mathbb{E}[ \\mathbf{Y}]=(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\mathbf{X}^{T}\\mathbf{X}\\boldsymbol{\\beta}=\\boldsymbol{\\beta}.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This means that the estimator of the regression parameters is unbiased.\n",
    "\n",
    "We can also calculate the variance\n",
    "\n",
    "The variance of $\\boldsymbol{\\beta}$ is"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{eqnarray*}\n",
    "\\mbox{Var}(\\boldsymbol{\\beta}) & = & \\mathbb{E} \\{ [\\boldsymbol{\\beta} - \\mathbb{E}(\\boldsymbol{\\beta})] [\\boldsymbol{\\beta} - \\mathbb{E}(\\boldsymbol{\\beta})]^{T} \\}\n",
    "\\\\\n",
    "& = & \\mathbb{E} \\{ [(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y} - \\boldsymbol{\\beta}] \\, [(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y} - \\boldsymbol{\\beta}]^{T} \\}\n",
    "\\\\\n",
    "% & = & \\mathbb{E} \\{ [(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y}] \\, [(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y}]^{T} \\} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n",
    "% \\\\\n",
    "% & = & \\mathbb{E} \\{ (\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y} \\, \\mathbf{Y}^{T} \\, \\mathbf{X} \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1}  \\} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n",
    "% \\\\\n",
    "& = & (\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\, \\mathbb{E} \\{ \\mathbf{Y} \\, \\mathbf{Y}^{T} \\} \\, \\mathbf{X} \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n",
    "\\\\\n",
    "& = & (\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\, \\{ \\mathbf{X} \\, \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T} \\,  \\mathbf{X}^{T} + \\sigma^2 \\} \\, \\mathbf{X} \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n",
    "% \\\\\n",
    "% & = & (\\mathbf{X}^T \\mathbf{X})^{-1} \\, \\mathbf{X}^T \\, \\mathbf{X} \\, \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^T \\,  \\mathbf{X}^T \\, \\mathbf{X} \\, (\\mathbf{X}^T % \\mathbf{X})^{-1}\n",
    "% \\\\\n",
    "% & & + \\, \\, \\sigma^2 \\, (\\mathbf{X}^T \\mathbf{X})^{-1} \\, \\mathbf{X}^T  \\, \\mathbf{X} \\, (\\mathbf{X}^T \\mathbf{X})^{-1} - \\boldsymbol{\\beta} \\boldsymbol{\\beta}^T\n",
    "\\\\\n",
    "& = & \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}  + \\sigma^2 \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n",
    "\\, \\, \\, = \\, \\, \\, \\sigma^2 \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1},\n",
    "\\end{eqnarray*}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "where we have used  that $\\mathbb{E} (\\mathbf{Y} \\mathbf{Y}^{T}) =\n",
    "\\mathbf{X} \\, \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T} \\, \\mathbf{X}^{T} +\n",
    "\\sigma^2 \\, \\mathbf{I}_{nn}$. From $\\mbox{Var}(\\boldsymbol{\\beta}) = \\sigma^2\n",
    "\\, (\\mathbf{X}^{T} \\mathbf{X})^{-1}$, one obtains an estimate of the\n",
    "variance of the estimate of the $j$-th regression coefficient:\n",
    "$\\boldsymbol{\\sigma}^2 (\\boldsymbol{\\beta}_j ) = \\boldsymbol{\\sigma}^2 \\sqrt{\n",
    "[(\\mathbf{X}^{T} \\mathbf{X})^{-1}]_{jj} }$. This may be used to\n",
    "construct a confidence interval for the estimates.\n",
    "\n",
    "\n",
    "In a similar way, we can obtain analytical expressions for say the\n",
    "expectation values of the parameters $\\boldsymbol{\\beta}$ and their variance\n",
    "when we employ Ridge regression, allowing us again to define a confidence interval. \n",
    "\n",
    "It is rather straightforward to show that"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\mathbb{E} \\big[ \\boldsymbol{\\beta}^{\\mathrm{Ridge}} \\big]=(\\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I}_{pp})^{-1} (\\mathbf{X}^{\\top} \\mathbf{X})\\boldsymbol{\\beta}^{\\mathrm{OLS}}.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see clearly that \n",
    "$\\mathbb{E} \\big[ \\boldsymbol{\\beta}^{\\mathrm{Ridge}} \\big] \\not= \\boldsymbol{\\beta}^{\\mathrm{OLS}}$ for any $\\lambda > 0$. We say then that the ridge estimator is biased.\n",
    "\n",
    "We can also compute the variance as"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\mbox{Var}[\\boldsymbol{\\beta}^{\\mathrm{Ridge}}]=\\sigma^2[  \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}  \\mathbf{X}^{T} \\mathbf{X} \\{ [  \\mathbf{X}^{\\top} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}\\}^{T},\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and it is easy to see that if the parameter $\\lambda$ goes to infinity then the variance of Ridge parameters $\\boldsymbol{\\beta}$ goes to zero. \n",
    "\n",
    "With this, we can compute the difference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\mbox{Var}[\\boldsymbol{\\beta}^{\\mathrm{OLS}}]-\\mbox{Var}(\\boldsymbol{\\beta}^{\\mathrm{Ridge}})=\\sigma^2 [  \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}[ 2\\lambda\\mathbf{I} + \\lambda^2 (\\mathbf{X}^{T} \\mathbf{X})^{-1} ] \\{ [  \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}\\}^{T}.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The difference is non-negative definite since each component of the\n",
    "matrix product is non-negative definite. \n",
    "This means the variance we obtain with the standard OLS will always for $\\lambda > 0$ be larger than the variance of $\\boldsymbol{\\beta}$ obtained with the Ridge estimator. This has interesting consequences when we discuss the so-called bias-variance trade-off below. \n",
    "\n",
    "\n",
    "\n",
    "## Resampling methods\n",
    "\n",
    "With all these analytical equations for both the OLS and Ridge\n",
    "regression, we will now outline how to assess a given model. This will\n",
    "lead us to a discussion of the so-called bias-variance tradeoff (see\n",
    "below) and so-called resampling methods.\n",
    "\n",
    "One of the quantities we have discussed as a way to measure errors is\n",
    "the mean-squared error (MSE), mainly used for fitting of continuous\n",
    "functions. Another choice is the absolute error.\n",
    "\n",
    "In the discussions below we will focus on the MSE and in particular since we will split the data into test and training data,\n",
    "we discuss the\n",
    "1. prediction error or simply the **test error** $\\mathrm{Err_{Test}}$, where we have a fixed training set and the test error is the MSE arising from the data reserved for testing. We discuss also the \n",
    "\n",
    "2. training error $\\mathrm{Err_{Train}}$, which is the average loss over the training data.\n",
    "\n",
    "As our model becomes more and more complex, more of the training data tends to  used. The training may thence adapt to more complicated structures in the data. This may lead to a decrease in the bias (see below for code example) and a slight increase of the variance for the test error.\n",
    "For a certain level of complexity the test error will reach minimum, before starting to increase again. The\n",
    "training error reaches a saturation.\n",
    "\n",
    "\n",
    "\n",
    "Two famous\n",
    "resampling methods are the **independent bootstrap** and **the jackknife**. \n",
    "\n",
    "The jackknife is a special case of the independent bootstrap. Still, the jackknife was made\n",
    "popular prior to the independent bootstrap. And as the popularity of\n",
    "the independent bootstrap soared, new variants, such as **the dependent bootstrap**.\n",
    "\n",
    "The Jackknife and independent bootstrap work for\n",
    "independent, identically distributed random variables.\n",
    "If these conditions are not\n",
    "satisfied, the methods will fail.  Yet, it should be said that if the data are\n",
    "independent, identically distributed, and we only want to estimate the\n",
    "variance of $\\overline{X}$ (which often is the case), then there is no\n",
    "need for bootstrapping. \n",
    "\n",
    "\n",
    "The Jackknife works by making many replicas of the estimator $\\widehat{\\beta}$. \n",
    "The jackknife is a resampling method where we systematically leave out one observation from the vector of observed values $\\boldsymbol{x} = (x_1,x_2,\\cdots,X_n)$. \n",
    "Let $\\boldsymbol{x}_i$ denote the vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\boldsymbol{x}_i = (x_1,x_2,\\cdots,x_{i-1},x_{i+1},\\cdots,x_n),\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "which equals the vector $\\boldsymbol{x}$ with the exception that observation\n",
    "number $i$ is left out. Using this notation, define\n",
    "$\\widehat{\\beta}_i$ to be the estimator\n",
    "$\\widehat{\\beta}$ computed using $\\vec{X}_i$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Runtime: 0.1375 sec\n",
      "Jackknife Statistics :\n",
      "original           bias      std. error\n",
      " 100.029        100.019        0.150581\n"
     ]
    }
   ],
   "source": [
    "from numpy import *\n",
    "from numpy.random import randint, randn\n",
    "from time import time\n",
    "\n",
    "def jackknife(data, stat):\n",
    "    n = len(data);t = zeros(n); inds = arange(n); t0 = time()\n",
    "    ## 'jackknifing' by leaving out an observation for each i                                                                                                                      \n",
    "    for i in range(n):\n",
    "        t[i] = stat(delete(data,i) )\n",
    "\n",
    "    # analysis                                                                                                                                                                     \n",
    "    print(\"Runtime: %g sec\" % (time()-t0)); print(\"Jackknife Statistics :\")\n",
    "    print(\"original           bias      std. error\")\n",
    "    print(\"%8g %14g %15g\" % (stat(data),(n-1)*mean(t)/n, (n*var(t))**.5))\n",
    "\n",
    "    return t\n",
    "\n",
    "\n",
    "# Returns mean of data samples                                                                                                                                                     \n",
    "def stat(data):\n",
    "    return mean(data)\n",
    "\n",
    "\n",
    "mu, sigma = 100, 15\n",
    "datapoints = 10000\n",
    "x = mu + sigma*random.randn(datapoints)\n",
    "# jackknife returns the data sample                                                                                                                                                \n",
    "t = jackknife(x, stat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bootstrap\n",
    "\n",
    "Bootstrapping is a nonparametric approach to statistical inference\n",
    "that substitutes computation for more traditional distributional\n",
    "assumptions and asymptotic results. Bootstrapping offers a number of\n",
    "advantages: \n",
    "1. The bootstrap is quite general, although there are some cases in which it fails.  \n",
    "\n",
    "2. Because it does not require distributional assumptions (such as normally distributed errors), the bootstrap can provide more accurate inferences when the data are not well behaved or when the sample size is small.  \n",
    "\n",
    "3. It is possible to apply the bootstrap to statistics with sampling distributions that are difficult to derive, even asymptotically. \n",
    "\n",
    "4. It is relatively simple to apply the bootstrap to complex data-collection plans (such as stratified and clustered samples).\n",
    "\n",
    "Since $\\widehat{\\beta} = \\widehat{\\beta}(\\boldsymbol{X})$ is a function of random variables,\n",
    "$\\widehat{\\beta}$ itself must be a random variable. Thus it has\n",
    "a pdf, call this function $p(\\boldsymbol{t})$. The aim of the bootstrap is to\n",
    "estimate $p(\\boldsymbol{t})$ by the relative frequency of\n",
    "$\\widehat{\\beta}$. You can think of this as using a histogram\n",
    "in the place of $p(\\boldsymbol{t})$. If the relative frequency closely\n",
    "resembles $p(\\vec{t})$, then using numerics, it is straight forward to\n",
    "estimate all the interesting parameters of $p(\\boldsymbol{t})$ using point\n",
    "estimators.  \n",
    "\n",
    "\n",
    "\n",
    "In the case that $\\widehat{\\beta}$ has\n",
    "more than one component, and the components are independent, we use the\n",
    "same estimator on each component separately.  If the probability\n",
    "density function of $X_i$, $p(x)$, had been known, then it would have\n",
    "been straight forward to do this by: \n",
    "1. Drawing lots of numbers from $p(x)$, suppose we call one such set of numbers $(X_1^*, X_2^*, \\cdots, X_n^*)$. \n",
    "\n",
    "2. Then using these numbers, we could compute a replica of $\\widehat{\\beta}$ called $\\widehat{\\beta}^*$. \n",
    "\n",
    "By repeated use of (1) and (2), many\n",
    "estimates of $\\widehat{\\beta}$ could have been obtained. The\n",
    "idea is to use the relative frequency of $\\widehat{\\beta}^*$\n",
    "(think of a histogram) as an estimate of $p(\\boldsymbol{t})$.\n",
    "\n",
    "\n",
    "But\n",
    "unless there is enough information available about the process that\n",
    "generated $X_1,X_2,\\cdots,X_n$, $p(x)$ is in general\n",
    "unknown. Therefore, [Efron in 1979](https://projecteuclid.org/euclid.aos/1176344552)  asked the\n",
    "question: What if we replace $p(x)$ by the relative frequency\n",
    "of the observation $X_i$; if we draw observations in accordance with\n",
    "the relative frequency of the observations, will we obtain the same\n",
    "result in some asymptotic sense? The answer is yes.\n",
    "\n",
    "\n",
    "Instead of generating the histogram for the relative\n",
    "frequency of the observation $X_i$, just draw the values\n",
    "$(X_1^*,X_2^*,\\cdots,X_n^*)$ with replacement from the vector\n",
    "$\\boldsymbol{X}$. \n",
    "\n",
    "\n",
    "The independent bootstrap works like this: \n",
    "\n",
    "1. Draw with replacement $n$ numbers for the observed variables $\\boldsymbol{x} = (x_1,x_2,\\cdots,x_n)$. \n",
    "\n",
    "2. Define a vector $\\boldsymbol{x}^*$ containing the values which were drawn from $\\boldsymbol{x}$. \n",
    "\n",
    "3. Using the vector $\\boldsymbol{x}^*$ compute $\\widehat{\\beta}^*$ by evaluating $\\widehat \\beta$ under the observations $\\boldsymbol{x}^*$. \n",
    "\n",
    "4. Repeat this process $k$ times. \n",
    "\n",
    "When you are done, you can draw a histogram of the relative frequency\n",
    "of $\\widehat \\beta^*$. This is your estimate of the probability\n",
    "distribution $p(t)$. Using this probability distribution you can\n",
    "estimate any statistics thereof. In principle you never draw the\n",
    "histogram of the relative frequency of $\\widehat{\\beta}^*$. Instead\n",
    "you use the estimators corresponding to the statistic of interest. For\n",
    "example, if you are interested in estimating the variance of $\\widehat\n",
    "\\beta$, apply the etsimator $\\widehat \\sigma^2$ to the values\n",
    "$\\widehat \\beta ^*$.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "## The bias-variance tradeoff\n",
    "\n",
    "\n",
    "We will discuss the bias-variance tradeoff in the context of\n",
    "continuous predictions such as regression. However, many of the\n",
    "intuitions and ideas discussed here also carry over to classification\n",
    "tasks. Consider a dataset $\\mathcal{L}$ consisting of the data\n",
    "$\\mathbf{X}_\\mathcal{L}=\\{(y_j, \\boldsymbol{x}_j), j=0\\ldots n-1\\}$. \n",
    "\n",
    "Let us assume that the true data is generated from a noisy model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\boldsymbol{y}=f(\\boldsymbol{x}) + \\boldsymbol{\\epsilon}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "where $\\epsilon$ is normally distributed with mean zero and standard deviation $\\sigma^2$.\n",
    "\n",
    "In our derivation of the ordinary least squares method we defined then\n",
    "an approximation to the function $f$ in terms of the parameters\n",
    "$\\boldsymbol{\\beta}$ and the design matrix $\\boldsymbol{X}$ which embody our model,\n",
    "that is $\\boldsymbol{\\tilde{y}}=\\boldsymbol{X}\\boldsymbol{\\beta}$. \n",
    "\n",
    "Thereafter we found the parameters $\\boldsymbol{\\beta}$ by optimizing the means squared error via the so-called cost function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "C(\\boldsymbol{X},\\boldsymbol{\\beta}) =\\frac{1}{n}\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2=\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right].\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can rewrite this as"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\frac{1}{n}\\sum_i(f_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2+\\frac{1}{n}\\sum_i(\\tilde{y}_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2+\\sigma^2.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The three terms represent the square of the bias of the learning\n",
    "method, which can be thought of as the error caused by the simplifying\n",
    "assumptions built into the method. The second term represents the\n",
    "variance of the chosen model and finally the last terms is variance of\n",
    "the error $\\boldsymbol{\\epsilon}$.\n",
    "\n",
    "To derive this equation, we need to recall that the variance of $\\boldsymbol{y}$ and $\\boldsymbol{\\epsilon}$ are both equal to $\\sigma^2$. The mean value of $\\boldsymbol{\\epsilon}$ is by definition equal to zero. Furthermore, the function $f$ is not a stochastics variable, idem for $\\boldsymbol{\\tilde{y}}$.\n",
    "We use a more compact notation in terms of the expectation value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathbb{E}\\left[(\\boldsymbol{f}+\\boldsymbol{\\epsilon}-\\boldsymbol{\\tilde{y}})^2\\right],\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and adding and subtracting $\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]$ we get"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathbb{E}\\left[(\\boldsymbol{f}+\\boldsymbol{\\epsilon}-\\boldsymbol{\\tilde{y}}+\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2\\right],\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "which, using the abovementioned expectation values can be rewritten as"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathbb{E}\\left[(\\boldsymbol{y}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2\\right]+\\mathrm{Var}\\left[\\boldsymbol{\\tilde{y}}\\right]+\\sigma^2,\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "that is the rewriting in terms of the so-called bias, the variance of the model $\\boldsymbol{\\tilde{y}}$ and the variance of $\\boldsymbol{\\epsilon}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error: 0.01312157412031145\n",
      "Bias^2: 0.012073649480472317\n",
      "Var: 0.0010479246398391328\n",
      "0.01312157412031145 >= 0.012073649480472317 + 0.0010479246398391328 = 0.01312157412031145\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABa3klEQVR4nO2dd3xb1Rn3v0fLlne8ZxJnELIHAQIJM0CYIQVCofBSyiqllJa2lND2hZD2bQO0UGhLIaV005JSCGGmNAkjgTTLibPINon3lretcd4/riRLsiTLsWxL9vl+PnyQ7j269/hG+t3nPucZQkqJQqFQKKIf3VBPQKFQKBThQQm6QqFQDBOUoCsUCsUwQQm6QqFQDBOUoCsUCsUwwTBUJ05PT5djx44dqtMrFApFVLJjx45aKWWGv31DJuhjx45l+/btQ3V6hUKhiEqEEF8E2qdcLgqFQjFMUIKuUCgUwwQl6AqFQjFMGDIfuj+sViulpaV0dHQM9VRGDLGxseTn52M0God6KgqFop9ElKCXlpaSmJjI2LFjEUIM9XSGPVJK6urqKC0tpbCwcKino1Ao+klECXpHR4cS80FECEFaWho1NTVDPRXFMGVNURlPrTtIeWM7uSlmHlo0iSWz84Z6WsOWiBJ0QIn5IKOut2KgWFNUxiOv76HdagegrLGdR17fA6BEfYBQi6IKhWJAeGrdQbeYu2i32nlq3cEhmtHwJ+IsdIVCETn0x2VS3tjep+2K/qMsdB+ee+45Jk+ezC233MKaNWtYsWJF0PHf//732bBhwyDNTqEYPFwuk7LGdiTdLpM1RWUhfT43xdyn7Yr+owTdh+eff54PPviAv//97zz55JPcd999Qcd/61vfYuXKlYM0O4Vi8Oivy+ShRZMwG/Ve28xGPQ8tmhS2OSq8iViXy+Nv7WN/eVNYjzklN4nHrpkacP+9997LsWPHuOKKK7j11luJiYkhPT0dgGuvvZbrr7+e2267jRdffJGPP/6Yv//974wZM4a6ujoqKyvJzs4O63wViqGkvy4Tl2tGRbkMHhEr6EPBCy+8wPvvv8/GjRt56623mDNnjnvfqlWrmD9/PoWFhfzyl79ky5Yt7n1z5sxh8+bNXH/99UMxbYViQMhNMVPmR7z74jJZMjtPCfggErGCHsySHgwqKirIyOiuUJmVlcWKFSu46KKLeOONN0hNTXXvy8zMpLy8fCimqVAMGA8tmsQPXiumy+5wb1Muk8gmYgV9qDGbzVgsFq9te/bsIS0trYd4d3R0YDarhR7F8GLJ7Dw2fF7N2t3a9z0nKZaHrzg9qMWtEomGFrUoGoDJkydz5MgR9/utW7fy3nvvUVRUxC9+8QuOHz/u3nfo0CGmTZs2FNNUKAYUg6478ez3X53bq5j3JypG0X+UoAfg/PPPp6ioCCklnZ2d3H333bz88svk5ubyy1/+kjvuuAMpJVarlSNHjjB37tyhnrJCEXYOVDaTmRgDwMn6tqBjVSLR0KME3YeSkhLS09OJi4vjkksuYf369cTExLB79273IunixYvZuHEjQgjefvttbrjhBgwG5b1SDC+sdgdHq1u4dEoWACd6EXSVSDT09CroQoiXhRDVQoi9QcZcKITYJYTYJ4T4KLxTHDp++MMf0tYW/Etss9n43ve+N0gzUigGj+O1rXTZHZw5NpWUOGOvgq4SiYaeUCz0PwGXB9ophEgBngcWSymnAkvDMrMIICsri8WLFwcds3TpUlJSUgZnQgrFIHKgQssDOT0nkYJRcZxsCG5pq0SioadXP4GU8mMhxNggQ74CvC6lPOEcXx2muSkUiiHkYGUzBp1gXHoCo1Pj2F8RPNFPJRINPeFw/J4GGIUQHwKJwLNSyr+E4bgKhWII+byymQmZCZgMOgpS4/jP/krsDole511yWYUqRg7hEHQDcAawEDADnwkhtkgpD/kOFELcA9wDMHr06DCcWqFQDBSfVzRxVqGWQDc6NQ6rXVLZ1EGeh09c1TyPLMIR5VIKrJNStkopa4GPgZn+BkopV0kp50op53pmYUYSer2eWbNmMXXqVGbOnMkvf/lLHA5H0M+UlJTwyiuvDPjc7rrrLvbv3x90zJo1a3odo1D0hqXdSrmlg0nZSYAm6NAzdPHxt/apUMUIIhyC/iawQAhhEELEAWcDB8Jw3CHBbDaza9cu9u3bxwcffMB7773H448/HvQzgyXoL730ElOmTAk6Rgm6IhwcrGwGtAVRgIJUzSr3jHRZU1RGQ5vV7+dVqOLQEErY4j+Az4BJQohSIcSdQoh7hRD3AkgpDwDvA8XAVuAlKWXAEMdwsqaojPkrN1C47B3mr9wQ9oy0zMxMVq1axW9+8xuklJSUlHDeeecxZ84c5syZw6effgrAsmXL+OSTT5g1axbPPPNMwHGelJSUcPrpp3PLLbcwefJkbrjhBneI5Pr165k9ezbTp0/njjvuoLOzE4ALL7yQ7du3A5CQkMCPfvQjZs6cybx586iqquLTTz9l7dq1PPTQQ8yaNYujR4/y3HPPMWXKFGbMmMFNN90U1uujGD74/pZe3XYCgNOzNUHPTTGjE94WejArvC+hip9XNvHVl7fS1mU7xdkrXPQq6FLKm6WUOVJKo5QyX0r5BynlC1LKFzzGPCWlnCKlnCal/NWAztjJYKUZjxs3DrvdTnV1NZmZmXzwwQfs3LmTV199lQceeACAlStXct5557Fr1y4efPDBgON8OXjwIPfddx8HDhwgKSmJ559/no6ODm6//XZeffVV9uzZg81m43e/+12Pz7a2tjJv3jx2797N+eefz+9//3vOPfdcFi9ezFNPPcWuXbsYP348K1eupKioiOLiYl544QU/s1CMdPz9ltbsKifOpCc7KRYAo15HborZy0IPZoX3JVTx3T2VfHSohl0nGk/1T1A4idpM0aFIM7Zardx9991Mnz6dpUuXBnRthDquoKCA+fPnA3DrrbeyadMmDh48SGFhIaeddhoAX/3qV/n44497fNZkMnH11VcDcMYZZ1BSUuL3HDNmzOCWW27hb3/7m8pmHcEcqW7G4ZB+9/n7LdkdEptdejURH50a52WhB7LCDTrtmKE+Oe8v14rg7Qtz/4ORSNQK+mClGR87dgy9Xk9mZibPPPMMWVlZ7N69m+3bt9PV1eX3M6GO8/yx+HsfDKPR6B6v1+ux2fw/rr7zzjt885vfZOfOnZx55pkBxymGL28Xl3PJ0x/z0aEav/sD/WY8y+YCFIyK40R999iHFk3CpPeWEAHYHPTpyXlvmSbkvcW5K3onagV9MNKMa2pquPfee7n//vsRQmCxWMjJyUGn0/HXv/4Vu12zahITE2lubnZ/LtA4X06cOMFnn30GwCuvvMKCBQuYNGkSJSUl7kqPf/3rX7ngggtCnrPnXBwOBydPnuSiiy7iiSeewGKx0NLSckrXQhGdNLR28dib+wAobfCfuh/oN5MU6/1ENzotjtqWTreve8nsPC5IbyKWTvaYvsaW2G9zhWl3j+MEe3Kuae6ksqkDgH3lFr9jFKETtYI+UGnG7e3t7rDFSy65hMsuu4zHHnsMgPvuu48///nPzJw5k88//5z4+HhAc2vo9XpmzpzJM888E3CcL5MmTeK3v/0tkydPpqGhgW984xvExsbyxz/+kaVLlzJ9+nR0Oh333ntvyPO/6aabeOqpp5g9ezaHDx/m1ltvZfr06cyePZsHHnhAlSkYYfzknf1Y2rVIlNoW/0+K/n5LoLldPC3rAnfoomaldxatZmuVYJFuO4m6TrKp4bfiCS7TbSOXWo7HfIUdpntYrNsU8CnAJeJnjU3laE0rHVb/xo8iNKLWqTpQacaBrGmAiRMnUlxc7H7/xBNPAJr7Y8OGDV5j/Y3zxWAw8Le//a3H9oULF1JUVNRj+4cffuh+7Wlp33DDDdxwww0AzJ8/38tnv2nTpoB/jyL6CZal+dGhGl7fWcb9F03gla0nqG3p9HsM1/jla/fR2N4dhtjaZWfTG89z2X/+TVx7JaPj5gHf4mR9G5OyE/nw/dew8FWW6De7PyMEFIpKPnSmoqTpWnjK+CKpRhNwVY9zu/zmS+fms7Wkns8rm5lVkBKGKzMyiVpBB9WvUDGyCZalee6ENH74+h7GZcRz/8UTWLevkroAFjpov6Wn1h30EvTLxFZWiFXEtWufK2jVopFP7N3MvpSFPNt0AWlYWKDb43WsFNFCFybaiSGOTmKEncdsv4Jn/g0LH4UZN7rH7i2zMDYtjnnj0gDYX96kBL0fRLWgRzNjx45l795BCddXDFMCRXotX7sPh5R02By8ctfZxBr1pCfEBLTQXfi6RX5s+DtxovsmkEoz8bTz4q4uflK0iWQy+KnxJYzCew6paGs49SQSh3ZOAWA5CW85Q3idor633MKM/BTyR5lJjDV4+dEtbVZ+tf4Q3730NBJjjSFfl5FM1PrQFYqRTiC/dGO7lck5Sbz37fOYO1arxZKWYKKuNbCFDtri6GLdJi7TbaOAagp03lExQsBpopQGh5l7zh/HR9fD1fqtPY6TIjR3YKNM6HkSazusXwFogn2yvh0BLHhiI80dNlZvP+n22/996xf8cXMJW47VB523ohtloSsUUUpuipkyP6KebDbyz3vmeYXBpifEUNscxEJ/+7ts6ngZjJIvd/1fckUt/qJoXzQ9DYm5ZF6xBZgMlVtg+8tAd4x7qtAs9AZ/gg5gKQW6F0T/s6/KHSJptUuW/bsYKSWrt50EcEfBKHpHWegKRZQSKDpl+TVTeuQ0pCeYaO60+Y8iefu7sP0PCCRCQJlMJ1fU+T1npqmLzMu+273h6qfhulWQXODcIBjl4XLx5LAjj+dt1+BI0sbudQq6b7x7h83B/3v3ACV1WphllUUJeqgoC12hiFJ8I730OsGEzAS+NCe/x9i0BK3Rc31rV8+48x1/cr+0S0ElqX4EXUByfo9FTUB777EtpaUTfvpfGmfcA4d2aW4W4DX7ebxoX0zC6Ou4je6EIn/UtnSRGGPAaNApC70PKAvdB1f53GnTprF06dJee4oG4/bbb+e1114L4+wUCm+WzM5j87KL2fv4IhxSctnUbL/j0p2C7ndhVHZb7dWMwo6eXFHbvT+5AJY3woN7e4q5H1LM2gJmfco0uOY5p/UuqDXmAvCz3WaO17ayr9xCrMG/BAlg8axcCkaZqVKCHjJK0H1wlc/du3cvJpOpR0ErlTqviET2lFlwSJgdIOQvLcEE4D90UXS7bcqlFj7ottCNZs0q7wMGvY6kWAONbV3aDeDBvbC8kZrRV1CQaibGoOfb/yziWG0rF56e6ddtJIEvn1lAVlIslcrlEjLRLejFq+GZabA8Rft/8eqwHv68887jyJEjfPjhh5x33nksXryYKVOmYLfbeeihhzjzzDOZMWMGL774IgBSSu6//34mTZrEJZdcQnW1aq+qGBx2nWwEYEZ+st/96fGahV7jz0I/43b3yzKZDkCeqAVjvGZhh2CV+5Iab+pRK722uZPTMhNZce1UikstSAlLz8jn59dNJy/FjADiTJq4T85JYnpeMjnJscrl0gei14devFqLaXX65/zFuPYHm83Ge++9x+WXXw7Azp072bt3L4WFhaxatYrk5GS2bdtGZ2cn8+fP57LLLqOoqIiDBw+yf/9+qqqqmDJlCnfccUe/56JQ9MauE42MTo1z+8p9SU8MYqFf/bT2/x1/clvoOXOugiVPnfJ8UuJMNLR5n6u2pZPpecksnpnLun2VvLe3kul5yWSWrGVJzAqILeWvxhv4v11f4stz8xFCkJUcS3OHjbYuG3Gm6JWrwSJ6r9D6Fd1i7sIV49oPQXfVcgHNQr/zzjv59NNPOeussygsLATgP//5D8XFxW7/uMVi4fDhw3z88cfcfPPN6PV6cnNzufjii095HgpFX9hd2uiOOfdHnMmA2ainLlBy0dVPw9VPU/7mXpKKykjsh5gDjIozej0NOBySutYu0hNNCCH4xdKZ3DG/kMyStV6G2aKuD9hnTOP62Fag0F2PvdLSwbiMAGGQCjfRK+jOWNaQt4eIy4fui2eBLSklv/71r1m0aJHXmHfffbdf51YoToWqpg4qLB2BU+aLV8N7D5Nue5TaTz+Cff+EK57wa/iUN3aEpWLpqHgTh6q66w01tHVhd0gynE8QcSaDdgN6xtswyxSNrNS/AB8XwBk3dgt6kxL0UIheH3pyz9CsoNvDyKJFi/jd736H1ar5CA8dOkRrayvnn38+r776Kna7nYqKCjZu3Djgc1EoipydfvwKevFqWHMftNeThoU6kqC9Ht78pt81J1eRr/4yysfl4qr0mJ7o4xLqxTDLStYEXUW6hEb0WugLH/X2ocMprcifCnfddRclJSXMmTMHKSUZGRmsWbOGL33pS2zYsIEpU6YwevRozjnnnAGfi2Lk4Vth8fScRAw6wdTcpJ6D168Ah2Z4pAsLpTJD227v8uueLLe0M2dMSr/nmBpvoq3LTofVTqxR7w6XTPf18Sfna+tfvjgNs26XS/A6NAqNXgVdCPEycDVQLaWcFmTcmWjNpG+SUg588LXri7h+hXY3D5T00Ef8NYC48MILufDCC93vdTodP/vZz/jZz37WY+xvfvObfp1foQiGvwqLFZZ28lLMxPoJ//O0gNNFE7sd4/3uA2jttNHYZg2LhZ4Sp8WiN7ZZyU7WU9McQNB7McziYwwkxhiUhR4ioVjofwJ+A/wl0AAhhB54AvhPeKYVIj4ZagpFtBOsvjn4r7DokFoGqF88LOA0mqgnCYcU6ITs4Z6ssGiimhcmlwtovvPs5Fi3hZ7h63IJwTDLSlax6KHSq6BLKT8WQoztZdi3gH8DZ4ZjUgrFSCRYfXOXqAeqsNjaFaAxy8JHNR+6w0q6sGBHTyMJJIsO9D7uybJGTTTD5UMH3H70mpZOTM6Eox70YphlJ6lY9FDp96KoECIP+BLwuxDG3iOE2C6E2F5T479hrZT+O5MrBgZ1vSOHQPXNPftxBhLbTF/L18WMG9k26/9RLxNIQ6udcsyRzcP2r7PGPt9rqOtmEZ4oF83l0tDqbH/X3EV6gqlPjdBdZCXFKpdLiIQjyuVXwMNSSkdvA6WUq6SUc6WUczMyMnrsj42Npa6uTonMICGlpK6ujtjY2KGeioLA1rfndn8VFnUCfnjl5IDH/c7+iczpXMUDtm8BcIP1cV7rOrdH4+byxnZ0ArIC3Rz6gD8LvUeES4hkJ8dQ3dyJ3aF0oTfCEeUyF/in886bDlwphLBJKdf09UD5+fmUlpYSyHpXhJ/Y2Fjy8wc+1FPRO4Hqm3tazJ4VFl1jbztnbNBWjMFuFMWljYxJiyfZbKSssZ3spFgM+v7bea5F0Qanb7+2uZOc5FMzHLKTYrE7JLUtnWQlKeMjGP0WdClloeu1EOJPwNunIuagNVt2ZWMqFCONhxZN8vKhA5iNeh5aNMlrnKuX7tIXPqXC0sGPrwpsnUPgG0VmUgzX/nYzuclmlp6Rz9vFFXTZHMxfuaHfDddjDHriTXp3PRdX2v+pkJ2s3dAqLR1K0Huh11uxEOIfaOGIk4QQpUKIO4UQ9woh7h346SkUI4cls/O8ClXlpZj5+XXTvYR1TVEZ81duYOyyd9hW0sDcMaMw7HstaJE6f24as1HPhadlIiU0d1j51frDdNk0r6lrMdbVCu5UGRVvorGtyyvt/1TwzBZVBCeUKJebQz2YlPL2fs1GoRjhuKxvf/hGwQDIfW9gO/wiBrtT7AIUqYsx6NyfM+kFP79uOuv2VZKdFIsAmjq8y0K7FmP7Y6WPijNR39blTvvvEYMeIlnJ2ufUwmjvRG/qv0IxwvAXBfOo+EO3mLvwaMTsugk0tneXsrU5JHaHZNORWs4/LT2g5RvI9x4qKXFGGtqs7rT/HjHoIZIeH4NBJ1QseggoQVcoogRfgV0otpEqemY2A+4s0ECJSD9/7wDNHTbOPy0jYJhif8MXU+NNNLR2BU77DxGdTpCZGKNcLiGgBF2hiBJ8BfZx418JGNbtzAINZGXXtnShE7BgQnpAH7vvYmxfcRXoCpj23weyklUseigoQVcoogRf4fXq++mLMws0kJVt1Atm5KeQEmcKaTH2VBgVZ6K5w+a2rDP6IejZqhVdSERvtUWFYoSxZHYeLZ02frxmLwDVIoNs/ORsmFPdC6L+QiEBbHbJ+RPTvY7dXwH3xZUteqS6RUv7N5+63GQlxfLJ4SA3MAWgBF2hiCoKUuMAeGXUi2S31wACraWyE6NZa17hxDMRqbyxnWSzkcZ2KxI4/7Se2drhJMWZLXq4qvmU0/5dZCfH0tJpo6XTRkKModciZiMVJegKRRRR/N9/AOOZ3rZV03IkblFPLvBbQtrT+t7weRV3/Gk7ibGGwB2OThFfkV08MxeAw9UtTMjsX7chz1Z0e8ssvRYxG6koQVcoIoRerc7i1RSXNjBOlJMoPBc7nWL+4N5ez+FamJw/Pj0sKf6ec/cV2Zc3HwegrcverwVRwJ0hWmFpD1rEbKQLuloUVSgigDVFZTz0r92UNbYj0QTxoX/t9s7WXL+CYsc4ZohjPQ8QYi/dvBQzJr2ORdOywjNxJ/5EttPWXa+vPwuiAJNzEjEb9byxs4zyxnYW6zaxyfQAx2K+wibTAyzWbep33PxwQFnoCkUEsHztPqw+1QStDsnytfvcVmdVYwtVpDJD50fQQ+ylm5YQw+ZlF5OecGpp+IHoTUxPNe3fRUqciVvnjeYPm47zpbjd/NT+EnFCS1jKF7WsNL5EqtEEXNWv80Q7ykJXKCIAz0zOQNuLY7X+MT0FXfSpl25GYky/Fij90VsSUn9dLgD3nD8ek0GH3dblFnMXcaKLHxhfDek4a4rK2Hiwut/ziUSUoCsUUcKegq+gw8FUUeKxVcDcO4a8FWOg5KQUsxa6GA5Bz0iM4dazx/B21xxKHD1dRnHtlSEd55n/HuLFj46637sKnhUue4f5Kzf0uyjZUKIEXaGIAEY564cH215szeO0FDCnZAFCWwi9bhVc/fTgTDIIgZKTcpyW+6nWcfHlngvGYcDBb+xLeu4Mwe0kpaSqqYOT9ZqLyLWY67l2EY5Kk0OF8qErFBHAY9dM5fv/2oXNp+/X9y/T0u+llBSXWrhk8mi4ofdolqHAX3LS6u1ag+pwWOgAmYmx3DIJ/nxwAd/Sv8EYndN1YjSH5HZqarfRYXVQbmmny+YYdhEzykJXKCKAJbPzeHzxNPf7zMQY9EKy+b1XqH5sDGW/PI/61i6m56f0+GwkuwxGxWuLof2NcvHknusux46et02X435Suea5kNxOVc1a+QApNWs8lLZ/0YSy0BWKCGH26FEAPH/LHK6Un/Dsax/wTOcS1vFrJnZpIj2zfRswxv0Zf/HfkZRkkxZvwmToX9q/L9nJsUzKSmRL0q18885f9+mzngW+Tta3hdT2L5pQFrpCESFUNmnCkpMcC+tX8G3dav5r+j5f179Nk4wjlSYmFf3E6zPBXAaRwO3njuXZL88Ke1TN2eNS2fFFA1a700dVvDpo1yYXVU2d7tcn6tsGrNLkUKEsdIUiQqhwVhPMSTa7E4Um6Mr5ge5Vvm9YTRcGYpq8OwtFustgXEYC4zL6l/bvj7ML0/jLZ1+wt8zC7MYPtC5NVuffHKBrE3Rb6Ea94GR9G49cqfVjHS51YXoVdCHEy8DVQLWUcpqf/bcAD6MVlGgGviGl3B3uiSoUw52Kxg70OqFFhCTna8LkRCcksVg1f7EHw81lECpnFaYCsOVYPbOLVnSLuQtX1yYfQa9u6iDZbCQtwcSJ+jZgYCpNDhWhuFz+BFweZP9x4AIp5XTgJ8CqMMxLEYFE8uLbcKDC0kFWrB39s9OdYu7jpvATyTHcXAahkpEYw/iMeP53vC5w2QM/26uaOslKimF0apxb0IcTvQq6lPJjoD7I/k+llA3Ot1uA0HKQFVHFcIvXjUQqjxaT3X7EwzKXONAiMirJYNv0x/1WUhyI5hTRwLxxaWwvaaApNtfv/jZzdo9tVc0dZCXFMjo1jpMjUdD7yJ3Ae4F2CiHuEUJsF0Jsr6nxU5hfEbFE+uJbNLDirf08+9/D/ncWr6aiqYMc4W076YAymc68jme5bdsYvzfQJbPz2LzsYo6vvIrNyy4eEWIOcPa4NFo6bfyw83bapHetmDZp4saW7/PRIW+dqW7qJDNRE/SmDhuWNv8lF6KVsAm6EOIiNEF/ONAYKeUqKeVcKeXcjIyBLa6vCC+RvvgWDaz/vIo3d/t/opHvPkyFTCVH1PXYl+vcpm6g3sxz+tHfbp/KMutdlDrScUhBqSOd73R9g73WHP61vXsdwuHQskSzkmLIH6U1CjlR3zasXIlhiXIRQswAXgKukFL2/EYqhpRwdHcZCYtvA90Fp66li9YuG21dNuJM3j+9pvZO2oklW/T0bpbLtO7X6gbqJjMplsL0eCoa21lrW8DargU9xry/t5I1RWUsmZ1HfVsXNod0u1wA/r2zlFe3nYzYOP6+0m8LXQgxGngd+D9SykP9n5IinITL9z3cF98Geo2gw2qnpdOGlHCgornH/gqpWZu+Lhcp4Ulbt998ON1Aw8G8cakgINbQLWWLdZu4QfchADpHF5+8/jxrisrcIYtZSTEUpGrX8fWdpcPKldiroAsh/gF8BkwSQpQKIe4UQtwrhLjXOeRRIA14XgixSwixfQDnq+gj4fJ9D/fFt4FeI6hv7S73ur+iqcf+CpOW/elrobcSy1qHZnkOpxtouDi7MI0Oq4P7L55AXoqZxbpNrDS+xAGpXc8ujNwt3mDXO6uodiYVZSbFkhhrJDXeRFOHze9xo/VJqFeXi5Ty5l723wXcFbYZKcJKOH3fwyle15dA16OssZ3rnt/MkzfM7FdfzLoWD0Ev9yPoU+6Erd3+cgC7MPILw72ITqI+4WWgOHuc9mSj1+nYvOxiSh+9Gxt6DsgxnKcr5hPHDMpkJnd1/Y1NTXcC3e3sClLjaO20eXVWchGtT0IqU3SYEw2+76qmDlLijMQY9L0PHiACXae0eBM7TzRSdKLhlATd5Zd3HTvGoPNroVcmTEUnDpORnAhNDZCcj37hoyyfcSPL+3zWkUNOspk5o1N4o6iUey8YR66ujo/sM3Cg42b9Bj5xzOCIzOMi/S532r+rUNjo1DjKGtrQddq9ns6i+UlI1XIZ5vTX9z3QEQBSSi7/1cc8v/Fo74MHkEDX6fzTtGgsS4COQsHw9Mu7sNod7C+3YLN7W4Xllg4yE80YvlsMyxu1hs9D3LQiWrjhjAIOVbWwp8xChzmbbY7TMWDjQt1u0mnkiMylw5xNVXOHu1gYQMEoM41tVn66ZNqwcSUqC32Y0mG1o9cJ9xfzVKI3BqOSn6XdSkOblR1fNPQ+eAAJdJ22lWg+7aZTEHR/fnmHBIddcry2lYlZie7tlZYOclJi+/EXjFyumpHD42/t47Udpay4YgVb/1nGVFFCnOhkgq6MwzKfuCtWUF3U4Xa3gGah2xySswpT2bzs4iH8C8KHEvRhyld+v4XU+Bh+f9sZp+z7Hozi/7Ut2mPwvnILUsqwV+XrC/6u06vbtDjmQItnwQi2TrG/oslL0Css7UzKTgw4XhGYZLORRVOzeXNXOd9f9CWKWcft5o/BIZgQ08SbtnnI6VdR9dFmspK667K7QhdP1rdR4Hwd7SiXyzBESsn+iib+e6CK9Qf61gzX08Xiz6cM4Y0AqGnWFgsb2qzuaoORxLHaFuDUXC7B1il+9MZet/tKSkmFpYPspMhZ14g2rj8jH0u7laf/c4guh+DMG74HyxuZcMmdNFsFNc2dzqSibgvdJeLDqaaLEvRhSG1LFx1WzUe74u39dPhY2YHwjcUORDgXVF0WOsA+P9EfvfHqthPc85ftA+Lnb+6wuhfSTkXQ/fnlXbR02txx7k0dNtq67OQql8sps2BCOllJMfzlsxIAzhyrNQuZkKk99Xz+/M3UNreTuf+P7lrpOcmxGHSCkw1K0BURTKnzC3r7uWM5Ud/GHzYd9zvOd8Hz8bf29XCx+BLuCABvQbf0+fMvbyrhg/1VA5IQdLy2FQAhTk3QPWP3ATKpZ5PpAZbqNpJCM5fYPuKpdQepdD6ZZCcrQT9V9DrBdXPycUiYlJVISpxW22VC3UYAtjRn4EBHVmeJViu9eDUGvY7cFDMn6qMz5twfStCHIaUN2hf05rNGs2hqFr/ZcKSHm8RfZmRDkEJFAxUBUNvSiV4nGJcef0oW+tGalh5PE+FKCDpao7lbJmQknJKgQ3fhLDMdzNftI19XyzTdFzSSyEPGVzmj6QMqLB6dihSnzPVztEKvZxaOcm/L+vRxEmhjs2Oq9l40dtdKByZkJrC3LLghEU21XpSgD0Ncgp4/ysyPr5qCQ0p+8vZ+rzH+FjwDkZdiHrBKfrXNXaTFm5ial+w34aY3bA7/zqFw+Pnf21MJwOHqFo7VtIT8Q/7DpuNsL+nO+JRS4kBHpmgEYIruCwCOyAKWG//i3alIccpMyEzg1zfP5hsXTnBvE02ljBfl7JHjAMgSzmgqZ6308yemc7y2lS/qWv0eM9rKRitBjwDCbQGUNrSRGm8iPsZAQWocDyycyHt7K3mnuMI9JlTBizHoBjTJoralk/SEGKbmJmlPCR4p8r3R5SfDz0V//fxrisq8FpQdkpB+yA6HZOV7B/jrli/c21q77HRiIk1oN6zTxQkA9ssxjKKFCksHOqE1bVD0j2tm5rpdXAAk5zNelONwSp1b0JM1a/7CSZkAfHjQfznvaCsbrQR9CFlTVMasx//Dd17dFVYL4GRDO/mjur/UXz9/HDPyk/nBv3cz72frKVz2DroA4YEpZiM6j11njBk1oEkWtS2dpCdqgg7+65wE+6w/wuHnf2rdQezS2/oP5Ydc29qJ1S4pqeteaKtzztMl6IminTGikn/YLuYb1m/z5q4yMhJjMOrVzzHsLHyUCQbtxqzDQRoWr85PY9PjKUyPZ+NB/9Fg0VY2Wn2DhgjXo1yjH99sfy2A0oY2L0E36HVcMS2b1k47lU0dSOghVqAJ4Y+vmoxDwncvPY0vzy1g54kGGttCt5r7Sm1LF+kJJqbmJgN9Wxitbu4W9KzEmLD6+U81ZLO8UXOfuB/hi1dT+9JSAFLpvlndol9PomjnGPmY9DqWzIrOzMSIZ8aNMO5CADJopEak9+j8dOGkDD47Wkd7V08XZKAnvUgqneGJEvQhojcfdjDhKG1ow2r3726QUlLW0O4u4O/ib1tO+B2vF8JLCM8ep9Xezk6K5WsLxtJhdfCPrSf9fra/SCmpaekkIyGG1HgTucmxfVoYrW7qjlt/+WtnhtXPnxXA/dHbD7nC+e/W2GalcdtqeOsB6lq0eaaL7r/tHsM7vBP7f/nPTSl88N0L3N3nFeFlTVEZzx7R3CpVpPrt/HTRpEw6bQ62HOvZyiHaykYrQR8ierP0fIWjrqWTP2w6zhXPfsKCJzby509L/H6upqWTTpvDy0IPdj6HlF5CWOmqGZ0cy+nZSZw7Po2/fFbSo/ZIOGjutNFlc5DuLJY0JTe514gDT2o8XC4tp5DJGQx/NwWTvvf1hHKP5KiSt58Eazv1UouFThMef1tyAfovPa/qtQwwT6072KOa4qX2j5j35gWwPAWemcZZrRsxG/V86MftEm1lo1Xq/xARqLof9LQAmjqsXPbMx9S1djEjP5mMxBg+O1rHXeeN6/5Q8WpYv4LShhhgBfnvfhW21Ll9hZ/F/pBMWUOTjOM39iX80X4FOhwkxsV7ndsdE+3MqLvtnDHc+7edfHKkloucC0jhotbpMklP1GKGp+Ymsf7zKr8dffzhqm8N0NoVXkF3ZRRmJ8W6b3JfPXdMrz9kzxvnF44MZukPU4fmTkrF1dhCaMW3FAOOryHjqpceh9ONaDlJ7LsPcG7Gi2w8WMNyP+UnoqlstLLQh4hAWYSj4ow9LIDPK5qpa+3i2Ztmsfb+BVw0KYOik41IKTUhf6IQXr8bLCc5KbXqgAWiRusev+Y+ePObZFODTkCKro0fG1/hXdMjjBMVmNor3ZlzgLuri0vQLzo9k5Q4I//eURr2a1DrrBHustCn5iYF7OjjD08fenMQC33HFw088npxn54yjta0kGw28tkjF7Px+xcCMDknqdfPVVjaydM1IHBQIrMAqJVJJNJGrHCulzgjLBQDj++T7g8Mq4kTPmtC1nbmN67hRH0bhY+8G/Gx5sFQgj5EuB7l9E5rIM6k51dfnsVj10zlqXUHvUIYD1VpAjd3rFbMf/boUdS3dvHFp69pWW/t3THPpU5BzxO12gaHFew9FzUn6Uq5Rv8ZlTKVtn/fr90UildT1dRBrFFHklmzkGMMeqbnJvN2cQVj+xhW2dJp4xt/2xEwxtcVpeIW9DzNkt0f4sJoTXMnafEm97kC8dqOk/xj60nW7i4P6bgAx2paGZcRjxCCZLMRCC1btLyxg0JKyaGeLxzZANTJJHeEi2eEhWLg8TWccl2/Cx8usX3ifh3psebBUII+hFwyJcsdbTJntJbd5i+JYd2+SuJNenKdmYSusTs/WqtlvXlQKjNIw0Kc8B/S58lYoSXOlMgs7abw1gNUnjhMdlKs+7FzTVEZW4933zD68mX/6GAN7+2tZOPn/kPCfAU9NzmWZLORA5WhWeg1zR0Upmsuo2A+9D1Ov/yvNxwJ2Uo/VtvCuHStoUVSrHZzC03Q28kxdTBGV+W20OtI1iJchB6ueU75zQcRXx94pUj3O05H30NUIxEl6EPI0WottTzWqKOyqSNgEsPW4/VMyEp0i+yEzAQSYgwUtab1OGapzCA/gBXiS6FT0I/LHG2DtZ2qshIym/a6F4x2vbOKTh8RDPXL/ulRbR6umii+1DZ3IgSkOq3sN3eV09Zl45X/nQjpSaC6uZMxafEIAa2dNr8JWp02Owcrmzk9O5Hjta0hWemuolzjM7WbhUGvIyHG0Kugd9kc1LR0kjNhJmP1tXzhEnSZSJq+Fb70ghLzIcBVfuH4yqtYZbiVNmny2t8mTV6NuF1Eaqx5MEJpEv2yEKJaCOF3FUdoPCeEOCKEKBZCzAn/NIcnR5yCfubYVKosHQG/QJ02BxM92p/pdYJZBSnsFFN6jC2T6eQLj6w3nRH0ph7jwNNCz3Zvq7Qnkm2vACRYTvID6/Ms1m3qeZ7G9l5F99OjWhjYsQCCXtPSRWqcCb1OuOPyrXbpPn6wJwGHQ1Lb0klWUgwJMQZ2nWz0+3Sz6qNjWO2Sb140gck5SSFZ6a4bkMtCB63mdkBBL14Nz0yjasVpSAl58ZKxU8+hjmSaZBx1YhTpY6YpMY8A/txyFsusd1HqSMchBaWOdJZZ73I34vYkUmPNgxGKhf4n4PIg+68AJjr/uwf4Xf+nNTI4XN2CQSeYNy6N5k5b0Gp7E336Wc4encLn9lzaDMnubQ4pNAtd57TQkwtgyfNw7W+11wCi+588XnSSRT3HnL5eKaFSjiLblR4NxIkunja+wGem+9DhYCLd8exlje08+OoufrxmT4/5lje2c7y2Fb1OUBLEh+5yt/Q1xbqx3YrVLslI1AR9+xcNfj/vqjQ5Mz+Fby+cGJKV7qqPPTa9O5Y/yWykqd2PW6dYizXHcpJytCemnOLfMiZRu2mU3H2AepJJG63izCOB3BQzax0LWND1HOM6/86CrucA2GR6gF8angfgYrEjomPNg9GroEspPwbqgwy5FviL1NgCpAghcsI1weHMkeoWxqbHu2PGbz93bI/IF5MzHfy0LO9uNnNGj8IuBcVnP+0Ua0FN4mS6MJJ/9TJYbunuSznjRu31cgs81gDX/R7M2gJroa7CbaE3kkAXJrKE9z+3QTjI0TVynq6YFuJ4TP+ye58E/r7lRA9L2mWdLzw9k7KGdjptPZOotLR/7emhrynW1c1aNE5mYiwJMQba/GT5gSb8yWYjBalmLpuS5bbS7QGKekG3oBd4JGclmw0929AVr4Y37nWvY1RITdBz7eWM3afZNbtPNmJ3SNIS/D8lKQYX30XSxbpN/ML4Ivm6WkbrtCfbWwzr+cuZX0RNqKIn4fCh5wGeqYSlzm09EELcI4TYLoTYXlPjvxjOSOJoTQsTMxPcMc9Tc5N7JDEsmZ0L0KPj/KyCFACKjLOdYt1I6dJ3AHokFfVgxo3w8HG47vcUxrRoPvTkAipNYwG8LHRPrtNvooJ0puu+4HjMVzge8xV2xtzDNbpNPSzpT4/Wkhpv4vJp2Tik1ubLF08Lva8p1q4Y9MykGBJiDcQY/H+VjXrBtLwkhBDodIJb543meG2ru2a8P07Wt5HmLG7mIinWx+Xissxl942kXGo3yRxRx+hW7anF1SvVtU6gGFp8F0lXmP6KSWj/hi5XZYVM48w9jw/hLE+dQU0sklKuAlYBzJ07N1hTnGFPp83OF3WtXD0jxx3zXdnUwQ1n5HtZBo+/tQ+zUe9dQQ4YFW9iXHo8O090i6+rbG5Bb4LuYsaNFDaeQf27n2P5ehGVH6+FDyFb+H8gu0hXhBEbHzjmMld/GIBUWnja+ALfawLQGu1KKfn0SB3njE9jXIZ2IzpW08resiavJsw1zd2C/tCiSV4NqSF4inWNMwY9I0FzuWQnx1Ld1On1+ViDDqtDMj0vxb1tbJq20FnW2M6YNO+kKhcn/PSY7OFDX7+iR4RRuUwnmRbiRSckF5DVHsPOE41AdySPYujxShRa3h1RlUUDJqxaLoe1VbtpR9m6Rzgs9DKgwON9vnObIggltW04pGZ5u3znVR61SVwLbYc/e4uJsgTd3n/1OMas0SkUnXAmGNEt6HkpoTe8LXQu/B2va6U67UzAlSUptDA7D5JEO/N0+/mPYy6etb0MwsGvTM/DM9OgeDXHa1upbOrg3PFpFDpF8+3iih6Lll12h/tvdllOrtDMhBhD0BRrV1JRZlIMibEGTHpdj6eb+y6agN0hmZ7Xvc7gujGWNQSOYDhZ3+5uIOwi2WzE0tqu/Y3LU7SkLR8qZBo5os4daz4mLd7tvlEul8hHJyR5opZS6cyIdjbBiCbCIehrgduc0S7zAIuUsqK3D410XBEu4zMSiDXqSYkzujvX8PZ34fV7wHKSw448JjiOu9tmeTJn9ChqWzqZ97P1jF32Dr9wuj0uefqjkJMiXHHcx2tbqLQ4RfI7H8PyRi3MDu806Mt02zkuczgqc722C9BE7vV72PzsVwGYb9tKcpyR1HgT/z1Q5bcY2abD3SGWS2bn8ekjCylMj+eCSRlBfZjVzR0kxBiIMxmINxlo6bSxZHYeDy2aRG6KmfLGdl52Loh6Crrr5umqiuiLze6grLGdglTvp5zkpoO02wVdjeUQoONquUwlT9S7Y83HpnXfFNLilYUekTjXklzkixp3trWrCUY0EUrY4j+Az4BJQohSIcSdQoh7hRD3Ooe8CxwDjgC/B+4bsNkOIw5XNyOEJujgrBli6dREe/vLgMQi46gilYm6Mq+2WS5cZW2rnNaqS2b6kvwzOjUOnYDjNZpVnZ5gwuTyR8+4EebegaeoX6LfCcA6x9wAR5R86phKLrWM2fgtKF5NYXo85q56dsbc4/a9/8v4GADPWH/qzlJ1UZAa59fn7klNc6e7IURCrIGWDluP7jKu0sQ7vuh2IcUa9WQkxlDW6P/4FZYO7A7Z00I//i4AFvy7aQDKZQY547vDE10uHSG0kg6KCOSKJ7zeFohqd7Z1NJZoCCXK5WYpZY6U0iilzJdS/kFK+YKU8gXnfiml/KaUcryUcrqUcvvATzv6OVLdQv4oM2aT5tbITo6lsqkd3nsYlzQfkZqFeppwWgo+FsMr//NfEhdCT/4xGXTkj4rjeF0bVU0d7gVaN1c/DdetclsyOaKemeIoH9jP8Hs8hxR85pjCufp9CJt2Exqrq0aPnVTRghCawLkKVmXoLFqW6pr7tCeTZ6Yx+ugrnCgr7/FE4km1h6Anxhho6bLx5Puf+30K+MV/Dnm9zwtSGM11Iylo3KrdaJYnw/Jkkju0a2+R3YvTn9in0STNgKAtaRwW4skZN8293+WvHxVnwqCaV0QmM26EuXfiMlryRQ31JNFqSInKEg3qWzZEHKluYUJGtzjkJMdSWdfoVZflsEMT9InCaWn7WAwVFv9uAxehZroVpsc7XS4d7gVaLzyiYjDGc5l+O7vkRKpkSo+hB2U+jSRyjs7Zw9RSyria9VSTSqvsdjvUSk3Q010lZR1W7cnEcpLRoppGGY9l7cMBRb2muZNMDwtdSu/StcGuQ16K2b/LpXg1J159CICCTx72+rdIQould1notTKJ26zLeEp3JyxvpPzWT9zHduGKY1cRLhGOy2hJLqDAmWV98vxfRN2CKChBH3TWFJVx7s/X83llM9tKGtxukaykWGo7dHTJ7oXIwzKfWDqd4VSih8XQWyZbqJluhenxHK9ppaqpg0x/gu5ixo3wo3Jypl0AwAf2M/DNuXSVEXD1zSQ5n7Gd2pOCZ0ZqrdNCT/Po4uN6MhkttNovJ7uSAi5MVTd1uC30+JrdAGyJuY+jMbdwPOYrvGn8EQC/MTzLZ7Hf9rox5I3SLHSHw6Na5fJkeP1uTrQZMWAjxyfSJ1logt4kNZE+5MhHouOf7Wczdtk7XPb0RwD89J397n9Tl8slTQl65OPM1Si4558AnEw/f4gndGooQR9EXD5elyXZ0mnjkdf3sG3ti2R/uhyAaka5xx+S+UwQ5QgkR8d8uYfFEKgEL/Stq0phejytXXbqWrv8W+g+f8MP92oLoj+23cl3uu6jTKY7pVhQJrXiR/mi1h3tUZio7T3uKegymRSaMYqeLpICp6CfkJl+F6ZaO220dtnJTIyF4tUk7PkTAC0yDr2QCIG7y/tM3TGyqdEWmZcnwzPTyGs9QJfNQd0TM7Wywx6W+EmZQZ6oxSC8b1XJTgu9iXhAsMcwFQAr2vV3ja5t6XKvXyTEGMhIjCFdNX+OGlwhv72t4UQqStAHEX/p7ZfaP2Lmzh+SbdXC4Kpkt6AfceQxUZRSLxO4rerLPY7nmSQBuEvx9rWriivSBSA7Obj4PLXuIB0eHWDWOhYwv/M5FsS+wbY5T3BIFpBAG23EuXs3jr3kbsDHQpfJ3e4WwDNxc7SnoPtZmHLFoGcmxsD6FSTateO00n0zOiLziKfdo66N8wSWk+Tu1bI4y9p7pmGckJnu83uS5LTQLbH5sLyRZ+03+L9AaOsXy9fuY/7KDdQ0d7LteH1UlmIdiaTGm4gz6d0hwK7wYVexumDrOpGA6lg0iPjzaf/AsBoTNnd2ZoUz27BZmqkgjbGiksdttwX0h/t2U1lTVMZT6w7y4Ku7eGrdQR5aNKlXYfcU9B6LoiH8DaBF1ty2bQztdk2Az+l8FvM2PT8vKGPJnBvJfustjtnHAlrNmBqZTDoWpIR6mcDbjnks1X9MnOgiUbSTShMnRC4svLbHuTxj0LGUkiA0N0iL7HYxVctRZIkGfJrPAJAntaJkZTKdWRz12ndSZrJIt63HZ1wWuqXwSoCApQZcNLZb3VE21c2dPPK6ljkajenkIwkhBPmjzJxsaOvOBnYlkFlOau8hYv3rykIfRHx92ot1m9yNKFw+20qnoG92aNESnzkms9axICR/uG/YXqjhi7kpZnfNmGAFwvz9DS70QgQtrlWYk8Hx7EWw3MIC8xvskJPYIqdS2PkKZ3St4jHbHTxpvM9dl6bA0MjJtPP8/nDcWaKJMZCcTwLaD64ZT0FPITNACYM8odWZKZfe5YdbZCz1JPm10E0xsZj1EkvSJKSU+LlPBCVa62uPRApGOcNm/WQDY3VFokUmStAHkYcWTSLWGeM9mkpWGl9yW5BJtBJLp1vQX7OfTxoWtshpIfvD+1qx0IVeJxjjTILpzYceqAu6XQZItnFa9GPT4ylxlqX197eYjXpmXXWPuy7N6KnzOGFN7DEOvAtzsfBREozauVs8BZ0UMvDf+SiJVhJoc/v7Xbjb91GNXTr/YZILtOieH5aTHG/G0m6lsqkDCRh1fZP1aKyvPRIpSI2jtKEd2Rggsai9PmJdL0rQB5Els/N48NLTALhZv9Grt6EQmpVeKVOpkUlsdMxiFM198of3tWKhJ2PT44kx6Nzt1oL9DZ5+e7NR7/XeF5dF39Zpo6HNythl7/DYm/sArRNQoE7qo1PNlDW0+61dXt3ciVEvSDEbYcaNJCzSon9apRmEDik1Cz1DNCIlPSJxhMFEnqij1EPQpYQTDq0hxWr7BZxvft27YiXd9VwOOjsq3XvB+B5/d16KOWASUTTW1x6J5I8y09JpozHptMCDIrQsgPKhDzKuIk0X63f12Jcl6qmSo3hTdyl29Pzuwa8wMcu/leqP3AAJM6EIyfVz8t09NHvD5bf/P3/4H5Z2q1uIH/53MZ0eC6auJ4s1RWW8t7fSvd3SYUUn4JErJnPz2aP9nmN0ahw2h6TC0tGjUJarqJfOaSHHz7keXn+floU/h4v+xatbT9Dx+h5etl/Jy/YrWazbxMPG1eSKOkRyPix8lNwPYyiv1iERlMs0nrDe6G5y8CmzSOyyUbjsHXJTzO51CJegH67SyjbcsaCQ7zmfNl7fWcrcMamMTotzu75CLTSmiCxc37fSM37AqA/v9DvGYSllbVFZxK2JKEEfZHadbCTepGdCisArBBvIpoHtYgqvJZ/JzDRdn8QcQq9Y6Fo4dVU9dAnW5dOyfQ8ZlLFp8azZVYaUkiWz89hbZuElZ/2UPI/jzl+5gS4fS9sh4ZE39vCbjUf8Lty6flQnnZUPD1c1897eSj49WsvOLxqZmpfkHhtj0GMy6Gh29hV9+gPvzNC1jgWs7VxAXoqZzQ9qFSHzju1hZ4MZ8Vgj24rK2LHuILhuhgIa2rQFTdc6BEBbl43PK5v53/F6dAI+OlTjnvd1c7qjcVzb/F1jReTjKj99Mm0+pxlTiLE29hjjkIJNbzwP3BdR/65K0AeZXYeOMUMeR990Ai3duNv3nG1oorQrFSqb+cm1U/t87FCExNd69BSsvn4xx6TF0dxho7HNyqh4E3nOH8L2H1/iVS42mMsn0PldtVSO1LSw5Xg9z288gl1KpuQk8dVzx/Cl2d7hjAkxBlo6u6NK/OE5j7yUOCztVndRryWz8/jaH7ey6Uituw2ei3arnR++XkynTbrXChySoNfNN/pIET24jIn391ayofNWVohVbvdoiSOLFmKZpvuCFaziyXcMLJkdObXTlaAPIh07V3OgPoa79fvQ8lEkblFPLiA7fyHs0LoUXTMzN/jBAtCbkARbOO2rALlqlZTUtTIq3kRZQzuxRl2PzMhArqBg589JNmPQCX7y9n6sdsl1s/P44VWTA9YVT4jRCnSBVgjLZWH7zqP7tavqYru7G9SJ+rYeYu6izdrTl3+q100R2STFGkk2G1l/oIrWrnOxCAOzdMd4z3E2e2UhJqwUxdxDvOjkrq6/AZEj6GpRdBDZ+8GfsWFgls4z9lkTcx7cS/bkcwG4ZEomKXEDky7en4VTX1y1Sr6o07LqyhrbtXrkPn74YBmtgc6v1wkm5yQxKs7ES7fN5ekvzwraJEKz0LUb1fmnZfTY7+t6cj1Wu+qiOxyS0oZ2EmL6ZuOoyJXhSUGqmVZnrsEH8iyest+EARtL9R/ShZE9shCAXF3dEM6yJ8pCH0R2NWt+39m6I947nOntp2UloNcJbjl7zIDNoT8Lp77kj4pDCNxNoEsb2skbFddjnKcrKJCl7u/8f7vrbEx6nbsiZTASYrtdLllJsRh0gszEGCosHX5dT64mIK751LR00mlzcM2MXF7bGXodbBW5MjwZnRrH3rLuRa5XjY9ztv4gdTKRf9kvZLdjAvN0n9Nhzib0djIDj7LQB5Ei/XTyqCFTNHrvcKa3j8tIYM/yy5g/Ib3nh8NEoDjyU4nAiDXqyU02e1nogfqZLpmdx+ZlF/OrL88K+fzJZmNIYg4uC11zudQ0d5KVFMunjyzk+Mqr2Lzs4h5ukYzEGAw64Rb0YzXaTenqmTkh1y5XkSvDlwcWTuTLcwvceSN/ty+kTZpIE80UiGp2OcYjgbj2iogqCaAEfRDZZZjOLEOJ90ZnASsXcaaBfWjybZLb17ovvoxJi6OkrpW2Lhv1rV0B49EH6vwuPH3o1c0dWlmAIOh1gpyUWMoatKqLz64/RGKMgRn5KTx2zVS/N50LJ3W7csI1b0Vkcnp2Ek/cMIOV188gL8XMW44FPGm8jzZzDrPEEXY7xndnC7tKAkSAqCuXyyBR3dxBWauO22fPgPICzc3ijIke7LoQ4YzAGJMWz/t7K9y+6EAW+kCd34XmctF8ntVNnYzLCNxZyEWes1XdK1tPsOVYPSuvm05qvClgtFD+KDMfHqzhokkZ/PFrZ4V1/orIxPu7ehXwODN/ei9vtaQ7y0s0artcHcWGuMaLEvRBYpez+/vssy+EsXuHdC7hpDA9joY2K/srNH9jKII+ECT6hC2ePS416Pg1RWXsPmmh3WpnxxcNnJaVwJfP7O517u+mc7hKyxA9rY/5AYrhxeyuHcA17HKM5zL9Dvf2SEg2CsnlIoS4XAhxUAhxRAixzM/+0UKIjUKIIiFEsRDiyvBPNbJoaO3yu8DXYbX7jXxYt6+KxBgD0zwaFg8HXE0cPj2irfa7FhsHm4QYAx1WB62dNiztVq3OSwB8Y/ElWqTOm7vKg54jJ8VMZmLMgK5xKCKfqSlWDNjY7Rjvtb3ckRZyL9+BIpQm0Xrgt8AVwBTgZiHEFJ9hPwZWSylnAzcBz4d7opHGj9fs5drfbKbDJ6b7p+/s55KnP6KupTu5pa3Lxnt7K7hqRg6xvYTvuVhTVMb8lRsoXPYO81duiNh62q5Y9E1HajHqhbst3GAT7ww3PO4sABZsHv5i8Tttjl6LmCXEGNj6o0v8hkUqRg6xl/yQ03Wl7JIT3NukBDMdXGr/iCff/9z9NDfYhGKhnwUckVIek1J2Af8EfItUS8CVi50MBDd1ohyHQ/Lp0VpqWzp5c1e30Na2dLJ6eyltXXb+7tHAed2+Stq67F7p4cE41TK4Q4Ero7PM6WvW9bECYbhIiPUR9CCLouGMxVeMQGbcyMxxeex2jMfu0L7vQkCaroWVxpfIbSri0mc+ZscX/ss3DyShCHoecNLjfalzmyfLgVuFEKXAu8C3/B1ICHGPEGK7EGJ7TU2NvyFRwdGaFhrarOh1gpc+OY50poP/9bMv6LI5mJyTxF8+K3Fb7//eUUZBqpm5Y0YFO6ybUy2DOxSYTXp3yd2h8p+D5kOH7vDDYC6XQLHjKqZcESozZ51BC2aOk+O1PU50kaXT1pPW7hp8AyxcYYs3A3+SUuYDVwJ/FUL0OLaUcpWUcq6Ucm5GRvQ+tm4t0ZpRfPOiCRyubuHDQzV0WO38dcsXLDw9kx9fNZnali7e3FVGhaWdzUdruW52flDr1dPFEij5JlItSFct9d5CFgcSl4V+rFarhBjM5RLOWHzFyGR2QQoAu6S3H11K2GnXXDFvFJVhd/gvJTFQhCLoZUCBx/t85zZP7gRWA0gpPwNigWG7crT1eD0ZiTGMHmVGCPjaH7dx+v99n/rWLibnJHHu+DQm5yTx0ifHeX1nGVLCdXMCr3z7ulgCEYkW5JqiMvaUaY0k3ttbOWRuIU8fuhBab8hADFQsvGLksKfUQhwd7HJM8Np+VOZSgdYJq6nDxtMfDO5TdShhi9uAiUKIQjQhvwn4is+YE8BC4E9CiMlogh69PpUgSCnZeryevBQzj7yxB99GPS98eIQJmQncfV4h3129m+c3HuHMsaPc0SD+8Odi8SUSLUjfaJHmDtuQ9c70dLmkxcdg0Ae3VVQ1RMWpsqaojB+t2Us8HRT5CPpGx0yv93/aXMJDi04ftLn1aqFLKW3A/cA64ABaNMs+IcQKIcRi57DvAXcLIXYD/wBulzJAT7Iop7ShnQpLB8drW/1W5rNJTaCvnpFLVlIMrSEshgZzpUSyBRlJvn6Xy6Wl0zZkkTaKkYHre19LCvvlGA7YC3BIQZ0jgc2O6RRQzSbTA+RSQ2uXHZvdMWhRayElFkkp30Vb7PTc9qjH6/3A/PBOLTLZ5vSfW5wd3RfrNpFMG1vkFH5v+AVP22/grcYFmAw67r1gPM+tP8yV03OCHTJgway8FDObl10c/j8iTERStIhnlcTe0v4Viv7g+f2W6LjC+gSLdZv4qeFldjhO42r9FvJ1tTxs+Cfftn2LX/7nEH/6tCQsPQh6Q2WK9pFtJfUkxRpIiDEwt/m/rDS+5NUbdKXuJVKNJuAqbj93LLecPQaTIfiDUKidhiKNcFZu7C/xHjVw+mKhB+repFAEwt/3/j79Wg5TQDNxLNBpYr1Iv504Wwd//qwkbD0IekMV5+oF30el9QeqOXNsKj+4/HSWG//iJeaghS39wPgqAEKIXsUconeRLpKiRXQ6QbyzMmNGiIIeTfH+isjB3/d+neNMNjmmIXBwrk5rgh4rrFym205bl//1sYF4klUWehD8tWsDiDPpWaLfjBQtfj8X117pd3swonGRLtJ6ZybEGmjtsgeNQfcknN2bFCMH3+99jEHHH+xXUUAV00QJozx04crEo6yxLPB7nIF4klWCHoRA0SdbjtVD1QoCRpUnh5YROhyIpBtRQoyBKjpDdrlE0hqAIrrw/N7vLbNw9a83sY9C7tWv7R5kNDP9oqWwBow6gdUjJn2gnmSVyyUIgX7YNS2d7i5DfvGob64YPBJitcYUoS6KqoxRRTiYlpfMpVOyADgvsRIQWlvJa54j++wbiDPpOWd82qC4VJWFHoRg0SfE5GuF7X0xpw55TeSRSkKM5tf0dbkEWviM1sVoReTxoysnk5di5swr3wCPdTMBjMuIRwgxKBFrykIPgr/FD5Nep/3gFz6qdRvyxGiGK54YxBkqPHGFLnouigZb+IzWxWhF5DE2PZ7li6f6DYIYn5HA0Rr/623hRlnoQXD9sJ98/3PKLR0Y9YInrnf94J1W+PoVQ9p9SNFNQoyRxFiDV4ni3hY+I2kNQDE8GZ+RwNrd5bR32UPukXuqKEHvhSWz8zCb9Hz9rzv45Y2zWDwzt3vnjBuVgEcQX5s/lvNP8y4hpBY+FUPNuIx4pNTqDE3JTer9A/1ACXovSCl54aOjFKSauXJa9lBPRxGEaXnJPTpCRVLyk2LksG3tixTsfIpMWUOynAk8zLHalgEXdOVD90NdSyf7yi3UtXSydf2/KTrRyN1Nv8Pw3IyI6OytCJ1ISn5SjAy2rX2RaTt+TDY16AScKfYjcPDxJx8O+LmVhe7D3jILX/n9Fpo6bM4tZlJpYqn+Q7B0wVsPaJuVqyUqiLTkJ8Xwp2DnU5g9MshjhZV8UUtjmZ+ouDCjBJ3usLayxnaEgBSzkWdvmkX9O49T1SaYp9vf/Q9kbdcWQpWgRw1q4VMxmGTKGnyzDseJcspk6oCfe8QLeo8O8BLauuxICV+zrgajnyrAwZKKFArFiKZaZJDt0w5ivCjnf47JOBxyQPvujngfur+wtkWOj5n35gUQqH/QCErtVygUfePknIdol94ds0aLajqIobKpY0DPPeIF3Td8bbFuEyuNL/W4w7oxmlVqv0KhCMiZi7/O3jN+SiUZOKSgkgyYdBXAgCcYjXiXi29Y2w8Mq3uUxHWTXKCShxQKRa+cufjrsPjrAGQDVzZ3sHzfeo5Wt3DexIwBO++It9AfWjQJg4dPK1fUBhgp4MG9SswVCkWfyUiIITHWwNGa1gE9T0iCLoS4XAhxUAhxRAixLMCYG4UQ+4UQ+4QQr4R3mgPHktl5TMtLRq8TCLQFDb8ov7lCoTgFtq19karHJzK+8wAHtn7AtrUvDti5enW5CCH0wG+BS4FSYJsQYq2zj6hrzETgEWC+lLJBCJE5UBOG8LcNs7RbuXRyFi/8nzOguFWLNbd6+NaV31yhUJwCriQjs+hinKjgU8dUztjxLf65cz2x1z4zJC3ozgKOSCmPSSm7gH8C1/qMuRv4rZSyAUBKWR3WWXoQ7rZhzR1Wjte2MtWVkjvjRrjmOc1f7lHXWLlaFApFX/FMMhqvK6eSNNqI5ctyHa1vfDvs7Q5DEfQ8wDPFqdS5zZPTgNOEEJuFEFuEEJeHa4K+BKuedyrsL28CYNq2ZbA8BZ6Zpu14cC8sb1R+c4VCccpkyu5oufGiHIBjMgch4CviA+xrvxvW84VrUdQATAQuBG4Gfi+ESPEdJIS4RwixXQixvaYmQFhgL4S7et6+7R8BMLV9GyC1phVvPaBqtigUin7juSY3XlRgxEalM2NUCPiS4/2wak0ogl4GFHi8z3du86QUWCultEopjwOH0ATeCynlKinlXCnl3IyMUwvdCXfbsL3795JJA5nC0r3R2k7l6z+kcNk7zF+5QXWBVygUp8TJOQ/haiU6XpRzIOZ2LtPvcO/XCbRSImEiFEHfBkwUQhQKIUzATcBanzFr0KxzhBDpaC6YY2GbpQfhrp63rzOTqbqSHtszZW1YfPQKhWLkcubir3N87E1IQCckBuHoOSiMpUR6FXQppQ24H1gHHABWSyn3CSFWCCEWO4etA+qEEPuBjcBDUsq6sM3Sg1DahlU3dTDnJx+w/kBV0GN1WO0ckblMEyU99pXLNPfr/vjoFQrFyGb8115EzL0zUCER2szh67MQUqaolPJd4F2fbY96vJbAd53/DTi9Vc/7+HAt9a1dPLXuIBdNygxYDOfzymbs6Jlq9La+26SJJ23eC6Gqw41CoThlrn6a13eU8SXH+3jKUZs08aT1yywP02mGZabolmPaw8Hnlc38N4iVvrdM85tPvewOd5hiJRkss97FWscCr7Gqw41CoegP32+/je9Y76PUkY5DCkod6Syz3sWfW84K2zmGZS2Xz47WccnkLA5XN/PrDUe4dEoWQvS00veVW0g2G8k/5wY4dykAW4rK+OD1PeDoDo1UHW4UCkV/yU0xs7ZxAWu7vI3FvDAai8POQj9Z30ZZYzsLJqTxzQsnsKfMwoeHtBBJKSVFJxr47/4q/rOvkq3H65mWl+Ql9qH46BUKhaKvDEY7xGFnoX/mdLecMz6dcRnxPLv+MM+tP4zu+Cc8u7mKndYxXuOvmp7T4xiqw41CoQg3g9EOcdgJ+pZjdaTGm5iYmYBOJ/jGheP58Zq9fPVELHnE8RPDy8zSHUUYTOgveIgJC3qEyysUCsWAMNDG4rASdCklW47WMW9cqjuyZencfPat+wPTrXu4Qf8RJuH0jTuAnY/DBUuHbsIKhUIRRoaVoJ+sb6fc0sG947pjyGMMen4mn0MYekaBSkupby9XhUKhiFqiVtD9ldDttGnW9znj0uDt78KOP4G0BzxGFemEL6RfoVAohpaoFHRXCV1X1UVXev7U3CTSE0xMeO8WKPnIPV4AUmrFcFy0SRM/ty7l2UGeu0KhUAwUUSnogUroFp1o5PICK8JDzF0IoYm6RFAu03jSdiM7ki4drCkrFArFgBOVgh4oDd8uJfMq/gZ6v7sBGNf5d0CL//y5ShZSKBTDiKgU9NwUM2U+on66KOFM3WGu0W0O+Dm70CGcnw93/KdCoVAMNVEp6A8tmuTlQwf4g+lp8kRt0M8Z5n6N41dfNdDTUygUiiEhKlP/Xen5Jr02/bwUM7kicLVeCVSlzYOrnx6kGSoUCsXgE5WCDpqoJ8QauPms0WxedjEiOd/vOJvU8e2u+7iw+kHVpEKhUAxrolbQ61u7qG/tYnxGvLZh4aNg9K5a1iZNfNd6L2sdC1STCoVCMeyJSh86wLGaFgDGZyRoG2Y4G1KsX4GjsdQdmuhZ11w1qVAoFMOZqBX0o76CDpqoz7iR81Zu6BEFA6pJhUKhGN5Ercvl6P4dmLCS91wePDMNile79w1G3WGFQqGINEISdCHE5UKIg0KII0KIZUHGXS+EkEKIueGboh+KV3P00H7GiQr0wgGWk/DWA25RV00qFArFSKRXl4sQQg/8FrgUKAW2CSHWSin3+4xLBL4N/G8gJurF+hUcdTzIVFHSvc3aDutXuH3pqkmFQqEYaYRioZ8FHJFSHpNSdgH/BK71M+4nwBNARxjn55fOxkpOyEzGi3LvHZbSgT61QqFQRCyhCHoecNLjfalzmxshxBygQEr5TrADCSHuEUJsF0Jsr6mp6fNkXXyRMBMHOsbrfATdIxZ9TVEZ81duoHDZO8xfuUHFoCsUimFPvxdFhRA64Gnge72NlVKuklLOlVLOzcjIOOVzHptyH4C3hW40a7HodJfXLWtsR9JdXleJukKhGM6EIuhlQIHH+3znNheJwDTgQyFECTAPWDuQC6NH42YCUJisBwQkF8A1z7n954HK66rEIoVCMZwJJQ59GzBRCFGIJuQ3AV9x7ZRSWoB013shxIfA96WU28M71W6OVreQmxxL/Pd2+t0fKIFIJRYpFIrhTK8WupTSBtwPrAMOAKullPuEECuEEIsHeoL+OFrTwvjMhID7U+KMfrerxCKFQjGcCSlTVEr5LvCuz7ZHA4y9sP/T8s+aojKefP9zyi0dxJv0rCkq6xGauKaojJYOW4/PGvVCJRYpFIphTdRkiroWOsstWlTk1+3/YPGaqbQ9cbpXluhT6w5idcgen483GVRcukKhGNZEjaC7FjrPEXsAmKs7hE5I4torvLJEA/nJLe3WQZurQqFQDAVRI+guoV6q/4TJosQ7Bt2VJUpgP7nynysUiuFO1Ai6S5CX6DfxXswPyRKN3gOcWaKqMJdCoRipRI2gu4S6XKb7H5Ccz5qiMrdrRi8EoApzKRSKkUPUCLqrguJLpltpkybvnUYz28Z/y50dCmCX0m2ZKzFXKBQjgagRdNBEffmPHyfu+t9q2aEeWaLf2T9RZYcqFIoRTXR2LHJ2JvKk/BX/dcFUdqhCoRgpRJWFHgwV3aJQKEY6w0bQVXSLQqEY6USny8UProXPp9YdpLyxndwUs1oQVSgUI4roEvTi1VoCkaVUa2ax8FEvX7pqO6dQKEYy0SPoxau1FH+rc5HT1RgaeiyQKhQKxUgkenzo61d0i7kLj5R/hUKhGOlEj6AHagCtGkMrFAoFEE2C7tEAOqTtCoVCMcKIHkFf+KjWCNoTj8bQCoVCMdKJHkGfcaPWCNon5V8tiCoUCoVGSFEuQojLgWcBPfCSlHKlz/7vAncBNqAGuENK+UWY5+o35V+hUCgUGr1a6EIIPfBb4ApgCnCzEGKKz7AiYK6UcgbwGvBkuCeqUCgUiuCE4nI5CzgipTwmpewC/glc6zlASrlRStnmfLsFUCuVCoVCMciEIuh5wEmP96XObYG4E3jP3w4hxD1CiO1CiO01NTWhz1KhUCgUvRLWRVEhxK3AXOApf/ullKuklHOllHMzMjLCeWqFQqEY8YSyKFoGFHi8z3du80IIcQnwI+ACKWVneKanUCgUilARUsrgA4QwAIeAhWhCvg34ipRyn8eY2WiLoZdLKQ+HdGIhaoBTjYRJB2pP8bMDTaTOTc2rb6h59Z1Indtwm9cYKaVfF0evgg4ghLgS+BVa2OLLUsr/J4RYAWyXUq4VQvwXmA5UOD9yQkq5+BQmGhJCiO1SyrkDdfz+EKlzU/PqG2pefSdS5zaS5hVSHLqU8l3gXZ9tj3q8viSck1IoFApF34meTFGFQqFQBCVaBX3VUE8gCJE6NzWvvqHm1XcidW4jZl4h+dAVCoVCEflEq4WuUCgUCh+UoCsUCsUwISoEXQixVAixTwjhEEIEDPMRQlwuhDgohDgihFg2SHNLFUJ8IIQ47Pz/qADj7EKIXc7/1g7gfIJeAyFEjBDiVef+/wkhxg7UXPo4r9uFEDUe1+iuQZjTy0KIaiHE3gD7hRDiOeeci4UQcwZ6Tn2Y24VCCIvH9RrwxgBCiAIhxEYhxH7n7/HbfsYMyTULcW5Dcc1ihRBbhRC7nfN63M+Y8P0mpZQR/x8wGZgEfIhW1dHfGD1wFBgHmIDdwJRBmNuTwDLn62XAEwHGtQzCXHq9BsB9wAvO1zcBr0bIvG4HfjPI36vzgTnA3gD7r0SrSySAecD/ImhuFwJvD/L1ygHmOF8noiUc+v47Dsk1C3FuQ3HNBJDgfG0E/gfM8xkTtt9kVFjoUsoDUsqDvQzrtSrkAHEt8Gfn6z8DSwbhnIEI5Rp4zvc1YKEQQkTAvAYdKeXHQH2QIdcCf5EaW4AUIUROhMxt0JFSVkgpdzpfNwMH6Fmob0iuWYhzG3Sc16HF+dbo/M83EiVsv8moEPQQ6WtVyHCRJaV0ZchWAlkBxsU6K01uEUIsGaC5hHIN3GOklDbAAqQN0Hz6Mi+A652P6a8JIQr87B9shuo7FSrnOB/l3xNCTB3MEzvdArPRLE5PhvyaBZkbDME1E0LohRC7gGrgAyllwGvW399kSJmig4GzfEC2n10/klK+Odjz8STY3DzfSCmlECJQHOgYKWWZEGIcsEEIsUdKeTTcc41i3gL+IaXsFEJ8Hc1iuXiI5xTJ7ET7TrUIrTTHGmDiYJxYCJEA/Bv4jpSyaTDOGSq9zG1IrpmU0g7MEkKkAG8IIaZJKf2ujfSXiBF02f/yASFVhTwVgs1NCFElhMiRUlY4Hy2rAxyjzPn/Y0KID9EsiHALeijXwDWmVGiF15KBujDPo8/zklJ6zuElIqPr1YB9p/qLp1hJKd8VQjwvhEiXUg5oESohhBFNMP8upXzdz5Ahu2a9zW2orpnHORuFEBuBywFPQQ/bb3I4uVy2AROFEIVCCBPa4sKARZN4sBb4qvP1V4EeTxNCiFFCiBjn63RgPrB/AOYSyjXwnO8NwAbpXI0ZQHqdl4+fdTGaD3SoWQvc5ozcmAdYPNxrQ4oQItvlZxVCnIX2Wx7QG7PzfH8ADkgpnw4wbEiuWShzG6JrluG0zBFCmIFLgc99hoXvNzmYK76n+h/wJTRfXCdQBaxzbs8F3vUYdyXa6vZRNFfNYMwtDVgPHAb+C6Q6t89Fa6gNcC6wBy26Yw9w5wDOp8c1AFYAi52vY4F/AUeArcC4QbpOvc3r58A+5zXaCJw+CHP6B1qFUKvz+3UncC9wr3O/QOune9T57+Y3wmqI5na/x/XaApw7CHNagLagVwzscv53ZSRcsxDnNhTXbAZaz+ViNKv8Uef2AflNqtR/hUKhGCYMJ5eLQqFQjGiUoCsUCsUwQQm6QqFQDBOUoCsUCsUwQQm6QqFQDBOUoCsUCsUwQQm6QqFQDBP+P2eKFNz8Jg6yAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_37_1.png"
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from sklearn.linear_model import LinearRegression, Ridge, Lasso\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.utils import resample\n",
    "\n",
    "np.random.seed(2018)\n",
    "\n",
    "n = 500\n",
    "n_boostraps = 100\n",
    "degree = 18  # A quite high value, just to show.\n",
    "noise = 0.1\n",
    "\n",
    "# Make data set.\n",
    "x = np.linspace(-1, 3, n).reshape(-1, 1)\n",
    "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2) + np.random.normal(0, 0.1, x.shape)\n",
    "\n",
    "# Hold out some test data that is never used in training.\n",
    "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2)\n",
    "\n",
    "# Combine x transformation and model into one operation.\n",
    "# Not neccesary, but convenient.\n",
    "model = make_pipeline(PolynomialFeatures(degree=degree), LinearRegression(fit_intercept=False))\n",
    "\n",
    "# The following (m x n_bootstraps) matrix holds the column vectors y_pred\n",
    "# for each bootstrap iteration.\n",
    "y_pred = np.empty((y_test.shape[0], n_boostraps))\n",
    "for i in range(n_boostraps):\n",
    "    x_, y_ = resample(x_train, y_train)\n",
    "\n",
    "    # Evaluate the new model on the same test data each time.\n",
    "    y_pred[:, i] = model.fit(x_, y_).predict(x_test).ravel()\n",
    "\n",
    "# Note: Expectations and variances taken w.r.t. different training\n",
    "# data sets, hence the axis=1. Subsequent means are taken across the test data\n",
    "# set in order to obtain a total value, but before this we have error/bias/variance\n",
    "# calculated per data point in the test set.\n",
    "# Note 2: The use of keepdims=True is important in the calculation of bias as this \n",
    "# maintains the column vector form. Dropping this yields very unexpected results.\n",
    "error = np.mean( np.mean((y_test - y_pred)**2, axis=1, keepdims=True) )\n",
    "bias = np.mean( (y_test - np.mean(y_pred, axis=1, keepdims=True))**2 )\n",
    "variance = np.mean( np.var(y_pred, axis=1, keepdims=True) )\n",
    "print('Error:', error)\n",
    "print('Bias^2:', bias)\n",
    "print('Var:', variance)\n",
    "print('{} >= {} + {} = {}'.format(error, bias, variance, bias+variance))\n",
    "\n",
    "plt.plot(x[::5, :], y[::5, :], label='f(x)')\n",
    "plt.scatter(x_test, y_test, label='Data points')\n",
    "plt.scatter(x_test, np.mean(y_pred, axis=1), label='Pred')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Polynomial degree: 0\n",
      "Error: 0.32149601703519126\n",
      "Bias^2: 0.3123314713548606\n",
      "Var: 0.009164545680330616\n",
      "0.32149601703519126 >= 0.3123314713548606 + 0.009164545680330616 = 0.3214960170351912\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Polynomial degree: 1\n",
      "Error: 0.08426840630693411\n",
      "Bias^2: 0.07968918676726028\n",
      "Var: 0.004579219539673833\n",
      "0.08426840630693411 >= 0.07968918676726028 + 0.004579219539673833 = 0.08426840630693411\n",
      "Polynomial degree: 2\n",
      "Error: 0.10398646080125035\n",
      "Bias^2: 0.10077114273548986\n",
      "Var: 0.0032153180657605086\n",
      "0.10398646080125035 >= 0.10077114273548986 + 0.0032153180657605086 = 0.10398646080125036\n",
      "Polynomial degree: 3\n",
      "Error: 0.06547790180152352\n",
      "Bias^2: 0.062082386342319454\n",
      "Var: 0.0033955154592040936\n",
      "0.06547790180152352 >= 0.062082386342319454 + 0.0033955154592040936 = 0.06547790180152355\n",
      "Polynomial degree: 4\n",
      "Error: 0.06844519414009442\n",
      "Bias^2: 0.06453579006728317\n",
      "Var: 0.003909404072811237\n",
      "0.06844519414009442 >= 0.06453579006728317 + 0.003909404072811237 = 0.06844519414009441\n",
      "Polynomial degree: 5\n",
      "Error: 0.05227921801205707\n",
      "Bias^2: 0.048187277304303125\n",
      "Var: 0.004091940707753964\n",
      "0.05227921801205707 >= 0.048187277304303125 + 0.004091940707753964 = 0.05227921801205709\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Polynomial degree: 6\n",
      "Error: 0.03781367141738898\n",
      "Bias^2: 0.03365768507152761\n",
      "Var: 0.004155986345861379\n",
      "0.03781367141738898 >= 0.03365768507152761 + 0.004155986345861379 = 0.03781367141738899\n",
      "Polynomial degree: 7\n",
      "Error: 0.027609773491022498\n",
      "Bias^2: 0.02299949826036597\n",
      "Var: 0.004610275230656537\n",
      "0.027609773491022498 >= 0.02299949826036597 + 0.004610275230656537 = 0.027609773491022505\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Polynomial degree: 8\n",
      "Error: 0.017355848195591973\n",
      "Bias^2: 0.010331721306655588\n",
      "Var: 0.007024126888936384\n",
      "0.017355848195591973 >= 0.010331721306655588 + 0.007024126888936384 = 0.017355848195591973\n",
      "Polynomial degree: 9\n",
      "Error: 0.026605727637189085\n",
      "Bias^2: 0.010018312644140933\n",
      "Var: 0.016587414993048166\n",
      "0.026605727637189085 >= 0.010018312644140933 + 0.016587414993048166 = 0.0266057276371891\n",
      "Polynomial degree: 10\n",
      "Error: 0.021592704588043153\n",
      "Bias^2: 0.010516485576652981\n",
      "Var: 0.011076219011390184\n",
      "0.021592704588043153 >= 0.010516485576652981 + 0.011076219011390184 = 0.021592704588043167\n",
      "Polynomial degree: 11\n",
      "Error: 0.07160048164228314\n",
      "Bias^2: 0.01443680008897583\n",
      "Var: 0.0571636815533073\n",
      "0.07160048164228314 >= 0.01443680008897583 + 0.0571636815533073 = 0.07160048164228312\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Polynomial degree: 12\n",
      "Error: 0.1154777721897675\n",
      "Bias^2: 0.01628578269590588\n",
      "Var: 0.09919198949386163\n",
      "0.1154777721897675 >= 0.01628578269590588 + 0.09919198949386163 = 0.11547777218976751\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Polynomial degree: 13\n",
      "Error: 0.22842468702166951\n",
      "Bias^2: 0.01975416527163567\n",
      "Var: 0.20867052175003387\n",
      "0.22842468702166951 >= 0.01975416527163567 + 0.20867052175003387 = 0.22842468702166954\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA/L0lEQVR4nO3deXxU5d3//9dnZrKHhKysCQkQEvbFEFzAFQTUira2isutdbdau/kr2vahLbf93nrXttalLrdFrVVRqbYuLBVX3ICwKihCIIFsZCNkX2bm+v1xJiEJAQZIcpLJ5/noPGbONvNJiu85uc51rkuMMSillApcDrsLUEop1b006JVSKsBp0CulVIDToFdKqQCnQa+UUgHOZXcBHcXHx5uUlBS7y1BKqT5lw4YNZcaYhM629bqgT0lJITs72+4ylFKqTxGRvCNt06YbpZQKcBr0SikV4DTolVIqwPW6NnqlVP/Q3NxMfn4+DQ0NdpfSp4SGhjJ8+HCCgoL8PkaDXilli/z8fAYMGEBKSgoiYnc5fYIxhvLycvLz80lNTfX7OG26UUrZoqGhgbi4OA354yAixMXFHfdfQRr0SinbaMgfvxP5nQVM0B+sa+Yvq3eyNb/S7lKUUqpXCZg2enHAn1d/S5BLmDR8oN3lKKX6AKfTycSJE1uXr7jiCu6++24bK+oeARP0UaFBDIoKYVdJjd2lKKX6iLCwMDZv3nzUfTweD06n84jL/h5np4BpugEYlRBJTmmt3WUopfq4lJQUFi1axLRp03jttdcOW3755ZeZOHEiEyZMYNGiRa3HRUZG8otf/ILJkyfz+eef2/gTtBcwZ/QAoxMjeX1jAcYYvcijVB/yu7e2sb2wqkvfc9zQKO77zvij7lNfX8+UKVNal++55x4uv/xyAOLi4ti4cSMAd999d+tyYWEhp556Khs2bCAmJobzzz+ff/3rX1xyySXU1tYyY8YM/vjHP3bpz3KyAi7oaxrd7K9qZHB0qN3lKKV6uaM13bQEfsfl9evXc/bZZ5OQYA0UedVVV/Hxxx9zySWX4HQ6+d73vtetNZ+IwAn66mK+u/5KvnCcR07pDA16pfqQY5152yEiIuKoy50JDQ3tNe3ybQVOG31YLBEHvmGcI08vyCqluk1WVhYfffQRZWVleDweXn75Zc466yy7yzqqwDmjdwVD3CgySov4SINeKeWHjm308+bN44EHHjjqMUOGDOGBBx7gnHPOwRjDhRdeyIIFC7q50pMTOEEPSPwYMg5s5m8a9EopP3g8nk7X5+bmHnV54cKFLFy48LDjamp6Z/YETtMNQEIGQzxF5JUcsLsSpZTqNfwKehGZJyI7RGSXiBx225iI3CoiX4rIZhH5RETGtdl2j++4HSIytyuLP0xCOk48RNTupaqhuVs/Siml+opjBr2IOIHHgfnAOGBh2yD3eckYM9EYMwX4X+BPvmPHAVcA44F5wF9979c9EtIBGC0FekFWKaV8/DmjzwJ2GWN2G2OagKVAuysPxpi2dzpEAMb3egGw1BjTaIzZA+zyvV/3iEvDIKRJATka9EopBfh3MXYYsK/Ncj4wo+NOInI78HMgGDi3zbFfdDh22AlV6o/gcBiYzJiKAr4s1aBXSinowouxxpjHjTGjgEXAb47nWBG5WUSyRSS7tLT0pOqQhHTGuor0jF4ppXz8CfoCIKnN8nDfuiNZClxyPMcaY542xmQaYzJbbis+YQnpJJsC9pR07bgZSqnAk5uby4QJEw5bf+ONN7J9+3YbKuoe/gT9eiBNRFJFJBjr4uqbbXcQkbQ2ixcCO32v3wSuEJEQEUkF0oB1J1/2UcSnE2Sa8Vbk0ujuvI+sUkodzTPPPMO4cR37nPRdxwx6Y4wbuANYBXwNvGqM2SYii0XkYt9ud4jINhHZjNVOf63v2G3Aq8B2YCVwuzGme9M3IQOAkVJAblldt36UUqrvc7vdXHXVVYwdO5bLLruMuro6zj77bLKzswG47bbbyMzMZPz48dx3332tx919992MGzeOSZMmcdddd9lVvl/8ujPWGLMcWN5h3b1tXv/kKMf+Hvj9iRZ43BLGAFg9b0prSB88oMc+Wil1glbcDcVfdu17Dp4I848+nAHAjh07+Nvf/sYZZ5zB9ddfz1//+td223//+98TGxuLx+PhvPPOY+vWrQwbNow33niDb775BhGhsrKya2vvYoF1ZyxAaDTeyMGMdhRqX3ql1DElJSVxxhlnAHD11VfzySeftNv+6quvMm3aNKZOncq2bdvYvn070dHRhIaGcsMNN/D6668THh5uR+l+C6ixblo4EtIZV1vAxxr0SvUNfpx5d5eOkxS1Xd6zZw8PPfQQ69evJyYmhuuuu46GhgZcLhfr1q3jvffeY9myZTz22GO8//77PV263wLvjB4gIYNUCsgpqba7EqVUL7d3797Waf9eeuklZs6c2bqtqqqKiIgIoqOj2b9/PytWrACswcsOHjzIBRdcwJ///Ge2bNliS+3+CtCgH0OYqae2LA+v1xx7f6VUv5Wens7jjz/O2LFjOXDgALfddlvrtsmTJzN16lQyMjK48sorW5t4qqurueiii5g0aRIzZ87kT3/6k13l+yUgm26It8a8SfLkU1BZT1Js724/U0rZIyUlhW+++eaw9R9++GHr6+eee67TY9et696e4l0pQM/orS6WLT1vlFKqPwvMoI+Ixxsao6NYKqUUgRr0IjgSM8hwFekZvVKq3wvMoAeIH8NoKSCnpNbuSpRSylaBG/QJGUSZKspKjjb+mlJKBb4ADnprKISE+lwqaptsLkYppewTwEFv9bwZ7dCeN0qpw51zzjmsWrWq3bqHH364XT/6o7n33ntZvXp1d5TW5QI36KOG4Q2K0J43SqlOLVy4kKVLl7Zbt3TpUhYuXHjMYz0eD4sXL2b27NndVV6XCtygF0ES0hmjg5sppTpx2WWX8c4779DUZDXt5ubmUlhYyMsvv9zpsMQpKSksWrSIadOm8dprr3HdddexbNkyABYvXsz06dOZMGECN998M8ZYd+SfffbZLFq0iKysLMaMGcOaNWsA64virrvuYsKECUyaNIlHH30UgA0bNnDWWWdxyimnMHfuXIqKirrkZw3MO2N9JCGd9KJV/J823SjVqz247kG+qTj8DtWTkRGbwaKsRUfcHhsbS1ZWFitWrGDBggUsXbqUH/zgB/zqV786bFjiSZMmARAXF8fGjRsBWLlyZet73XHHHdx7rzVy+zXXXMPbb7/Nd77zHcAa737dunUsX76c3/3ud6xevZqnn36a3NxcNm/ejMvloqKigubmZn784x/z73//m4SEBF555RV+/etfs2TJkpP+XQTuGT1AQjrxpoKi/fvtrkQp1Qu1bb5pabbpbFjiFpdffnmn7/PBBx8wY8YMJk6cyPvvv8+2bdtat333u98F4JRTTiE3NxeA1atXc8stt+ByWefasbGx7Nixg6+++oo5c+YwZcoU7r//fvLz87vk5wzoM/qWMW/Cq3Kob/IQFuy0uSClVGeOdubdnRYsWMDPfvYzNm7cSF1dHbGxsZ0OS9wiIiLisPdoaGjgRz/6EdnZ2SQlJfHb3/623TEhISEAOJ1O3G73EWsxxjB+/PjWkTS7UsCf0QOMlgJ2l2nzjVKqvcjISM455xyuv/56Fi5ceMRhiY+mJdTj4+Opqalpbbc/mjlz5vDUU0+1Bn9FRQXp6emUlpa2Bn1zc3O7vwxORmAHfUwKXmeI9rxRSh3RwoUL2bJlCwsXLjzisMRHM3DgQG666SYmTJjA3LlzmT59+jGPufHGG0lOTmbSpElMnjyZl156ieDgYJYtW8aiRYuYPHkyU6ZM4bPPPuuKHxFpuTrcW2RmZpqWSXm7gvevp/NhUTCbZz3Fz89P77L3VUqdnK+//pqxY8faXUaf1NnvTkQ2GGMyO9s/sM/osaYVzHAVklOqY94opfqngA96EjIYbErYt7/M7kqUUsoWfgW9iMwTkR0isktE7u5k+89FZLuIbBWR90RkRJttHhHZ7Hu82ZXF+yVhDA4MUr4Lt8fb4x+vlDqy3tZ03BecyO/smEEvIk7gcWA+MA5YKCLjOuy2Ccg0xkwClgH/22ZbvTFmiu9x8XFXeLJ8Y96MMPnkH6jv8Y9XSnUuNDSU8vJyDfvjYIyhvLyc0NDQ4zrOn370WcAuY8xuABFZCiwAWu8iMMZ80Gb/L4Crj6uK7hQ7CiNO0hxWz5uU+MP7wSqlet7w4cPJz8+ntLTU7lL6lNDQUIYPH35cx/gT9MOAfW2W84EZR9n/BqBt59NQEckG3MADxph/HVeFJ8sVjDcmldGlBewqrWE2g3r045VSnQsKCiI1NdXuMvqFLr0zVkSuBjKBs9qsHmGMKRCRkcD7IvKlMSanw3E3AzcDJCcnd2VJADgTM8io2MR72pdeKdUP+XMxtgBIarM83LeuHRGZDfwauNgY09iy3hhT4HveDXwITO14rDHmaWNMpjEmMyEh4bh+AL8kpJNEMXklB7r+vZVSqpfzJ+jXA2kikioiwcAVQLveMyIyFXgKK+RL2qyPEZEQ3+t44AzatO33mPh0XHhoKs3RCz9KqX7nmE03xhi3iNwBrAKcwBJjzDYRWQxkG2PeBP4ARAKviQjAXl8Pm7HAUyLixfpSecAY0/NB75tWcEjTXkqrG0mMOr4r1kop1Zf51UZvjFkOLO+w7t42rzudZsUY8xkw8WQK7BLxVtCnST67Sms06JVS/Urg3xkLEByBO2o4ox2F5OgFWaVUP9M/gh5wJo7VaQWVUv1Svwl6SUhnpBSyu6TK7lKUUqpH9ZugJyGdEJqoLdljdyVKKdWj+k/Q+6YVHFi7m+qGZpuLUUqpntN/gt7XxXK0FLBbx6ZXSvUj/Sfow2JwhyeSptMKKqX6mf4T9IAjMYM0RyG7SjXolVK9y5/e/Zb73+6e+0n7V9AnpJPmKCBnf7XdpSilVCuP1/DS2r0UVHbPnBn9KuhJSCeCeipL9tpdiVJKtVqfW0FZTSMXThrSLe/f74IeILRyJ01unVZQKdU7vLO1iNAgB+dmJHbL+/ezoLemFRxJAXsrtOeNUsp+Hq9hxVfFnJuRSHhwl04R0qp/BX1EAu6QaO15o5TqNVqabS6Y2D3NNtDfgl4ESchgtEODXinVO3R3sw30t6AHnInpjHEUkqM3TSmlbNYTzTbQD4OehAxiqKKk+LDZEJVSqket29P9zTbQH4PeN+aNo3wHXq9OK6iUss/yL7u/2Qb6Y9D7ulgmefIprmqwuRilVH/VU8020B+DPno4Hle4Na2gXpBVStmkpdnmwolDu/2z+l/Qi+CNG8Mo0dmmlFL2eefLQkKDHJyTkQDAe3nv8d7e97rls/pf0AOuQemkOwvI0cHNlFI28HgNK7/az3kZg1qbbR7b/Bh/3/b3bvm8fhn0kpDBICooKN5vdylKqX6oY2+bnQd2sqtyF/NT53fL5/XLoG+5IGvKdthciFKqP3rny0LCgpytzTYr9qzAIQ7mjJjTLZ/nV9CLyDwR2SEiu0Tk7k62/1xEtovIVhF5T0RGtNl2rYjs9D2u7criT5ivi2VCQx6VdU02F6OU6k+sZptDvW2MMazMXcmMwTOIC4vrls88ZtCLiBN4HJgPjAMWisi4DrttAjKNMZOAZcD/+o6NBe4DZgBZwH0iEtN15Z+gmBS8jiBGS6G20yuletTaPeWU1TS1NttsL9/Ovup93dZsA/6d0WcBu4wxu40xTcBSYEHbHYwxHxhj6nyLXwDDfa/nAu8aYyqMMQeAd4F5XVP6SXC6cMeMYrR2sVRK9bDlXxYd1mzjcrg4N/ncbvtMf4J+GLCvzXK+b92R3ACsOJ5jReRmEckWkezS0lI/Sjp5rkFjSdMxb5RSPahjs43XeFmZu5KZQ2cSHRLdbZ/bpRdjReRqIBP4w/EcZ4x52hiTaYzJTEhI6MqSjsiRkE6SlJBXXN4jn6eUUi3NNi0zSW0q2cT+uv3d2mwD/gV9AZDUZnm4b107IjIb+DVwsTGm8XiOtUVCOg4MTSXf2l2JUqqfeGerr9km3RrbZsWeFYQ6Qzk76exu/Vx/gn49kCYiqSISDFwBvNl2BxGZCjyFFfIlbTatAs4XkRjfRdjzfevs5+tiOaA6h4Zmj83FKKUCndvjZdW2Ys4dm0hYsBO31827ee9yVtJZhAeFd+tnHzPojTFu4A6sgP4aeNUYs01EFovIxb7d/gBEAq+JyGYRedN3bAXw31hfFuuBxb519osbjcHBKClgT5m20yulupd1k1QTF/p626wrXkdFQwXzU7q32QbAryHTjDHLgeUd1t3b5vXsoxy7BFhyogV2G1cITdEjSKuwZpsaOyTK7oqUUgHsnS/bN9us3LOSiKAIZg6f2e2f3T/vjPVxDRrLaNExb5RS3atjs02Tp4nVe1dzXvJ5hDhDuv3z+3XQOxPTGekoZvf+SrtLUUoFsI7NNp8WfEp1U3W397Zp0a+DnoQMXHioK95pdyVKqQDWsdlmRe4KBoYMZMaQGT3y+f076OPHABBauQuPTiuolOoGbo/XuknK12xT11zHh/s+ZM6IOQQ5gnqkBg16YIQ3n4ID9TYXo5QKROv2VFBe28RFvmabjws+pt5d32PNNtDfgz4kksaIoaQ58tlVWm13NUqpAPS2r9nm7Da9bRLCEpiWOK3HaujfQQ84EjMYrdMKKqW6gdvjZdVXxZzna7apbqpmTf4a5qbMxelw9lgd/T7ogwaNZbSjkN379YxeKdW11vqabVp623yw7wOavE3MS+3ZQXz7fdCTkE4oTRzcv9vuSpRSAeadDs02y/csZ1jkMCbFT+rROjTofWPeOMt2YIz2vFFKdY2OzTYHGg7wReEXzEuZh4j0aC0a9L6eN0Oa91Jeq9MKKqW6Rsdmm3fz3sVjPD3a26aFBn14LE2h8aRJgV6QVUp1mXe+LCI8uE1vm9yVpEanMiZmTI/XokEPmPh0Rjs06JVSXaP1JqkMq9mmpK6E7OJs5qfM7/FmG9CgByB48FjSpICcEu15o5Q6eWv3VFBR28RFvpmk/pP7HwyGualzbalHgx6QxAwGSD1lxXvtLkUpFQDe3tq+2WZF7goyYjMYGT3Slno06KH1gqyUfmNzIUqpvq5lSOLzxg4iNMhJfnU+W0u32nIRtoUGPbR2sYypy6W20W1zMUqpvuyL3VazzYUTBwPWRViAuSn2NNuABr0lchDNQVGMlgJ2l+q0gkqpE3dYb5s9K5mcMJlhkcNsq0mDHkAEd2waaY4CHdxMKXXCOjbb7K7czY4DO2xttgEN+lbBQ8ZZ0wqW6Bm9UurEdNZsIwjnjzjf1ro06H2cienESxVFRQV2l6KU6qPaNtsYY1ixZwXTB08nITzB1ro06FskZADgKdlhcyFKqb6oY7PNNxXfkFuV2+MjVXbGr6AXkXkiskNEdonI3Z1sP1NENoqIW0Qu67DNIyKbfY83u6rwLufrYhlZtYtmj9fmYpRSfc2hZhvrJqkVuStwiYs5yXNsrgxcx9pBRJzA48AcIB9YLyJvGmO2t9ltL3AdcFcnb1FvjJly8qV2s+gk3M4wUt0F7K2oY1RCpN0VKaX6kHe+LCQi2MnZ6QkYY1i5ZyWnDT2NgaED7S7NrzP6LGCXMWa3MaYJWAosaLuDMSbXGLMV6Lunwg4HTQNHkSb5OuaNUuq4tIxt09Jss6V0C0W1Rbb3tmnhT9APA/a1Wc73rfNXqIhki8gXInJJZzuIyM2+fbJLS0uP4627VtDgsYxyFJJTqkGvlPLf57vLOVDXzAW+ZpuVuSsJdgRzTtI5Nldm6YmLsSOMMZnAlcDDIjKq4w7GmKeNMZnGmMyEBPuuTgcNymCYlLOvqMS2GpRSfc/yL4tam208Xg+rcldx5vAziQzuHU3A/gR9AZDUZnm4b51fjDEFvufdwIfA1OOor2f5et40FeuYN0op/zR3aLbZsH8DZfVlvaK3TQt/gn49kCYiqSISDFwB+NV7RkRiRCTE9zoeOAPYfvSjbOQb8yakcqdOK6iU8ssXHZptlu9ZTrgrnDOHn2lzZYccM+iNMW7gDmAV8DXwqjFmm4gsFpGLAURkuojkA98HnhKRbb7DxwLZIrIF+AB4oENvnd4lJhWPuEjy7GN/VaPd1Sil+oB3th5qtmn2NLN672rOST6HMFeY3aW1Omb3SgBjzHJgeYd197Z5vR6rSafjcZ8BE0+yxp7jdNEYlcroCmu2qcHRoXZXpJTqxZo73CT1cf6nHGw8yPyU3tHbpoXeGduBY9BYa8wb7XmjlDqGlmabC30zSa3cs5IBwQM4fejpNlfWngZ9ByGDM0h2lJBbXG53KUqpXq6l2easMQk0uBt4f9/7zBkxhyBnkN2ltaNB34EkZuDEUFekY94opY6sY7PNJwWfUNtcy7yU3tPbpoUGfUfxVs8bV8W3NheilOrNPs9p32yzYs8KYkNjmT54us2VHU6DvqO40XhxkNiYR1VDs93VKKV6qZabpM4ak0Btcy0f5X/E3JS5uBx+9XHpURr0HQWF0hCZxGgd80YpdQTNHi8rtxUze5zVbPPBvg9o9DT2mrFtOtKg74RJSGe0FJKjQa+U6sTnOeVUth3bZs9KBkcMZnLCZJsr65wGfSfChowlVYrIKam0uxSlVC/UtrfNwcaDfFr4KfNS5uGQ3hmpvbMqmzkSxxIsHqoLdtpdilKql2n2eFm1/VCzzXt738PtdfeqsW060qDvTII125SjXLtYKqXaa2m2aZ1Jas8KkgckMy52nM2VHZkGfWd80wpG1+ym0e2xuRilVG/yztYiIkNcnDkmgbL6MtYVr2Ne6jxExO7SjkiDvjMhA6gLG8xIKSS3rM7uapRSvURLs815YxMJDXLyn9z/4DVeLki9wO7SjkqD/gg8sWN0WkGlVDufdWi2WZm7krSYNEYNPGw+pV5Fg/4IwoaOY5QUkVNSZXcpSqleYnmbZpuimiI2lWzqdSNVdkaD/ghcgzIIl0YqCnPsLkUp1Qs0uj1Wbxtfs82q3FUAvXJsm4406I/EN62gt0R73ijV3xljuPufX1JZ18z3M62ZVVfkrmBC3ASSopKOcbT9NOiPxDetYETVLrxenVZQqf7sL+/t5I1NBfxizhjOGB1PXlUe28u39+q+821p0B9JeCwNwbGM8OZTUFlvdzVKKZu8sSmfh1fv5HvThnPHuaMBa8gDQZibMtfm6vyjQX8UTTFppDkK2KWzTSnVL63dXc4vl23l1JGx/M93JyIiGGNYsWcF0wZNY3DEYLtL9IsG/VEEDxlnTSu4v9ruUpRSPSyntIabX9hAUmw4T12dSbDLisudlTvJOZjTJ3rbtNCgP4rQIWOJljpKivLsLkUp1YMqapu4/rn1uBzCc9dlER1+aGrAlXtW4hQns0fMtrHC46NBfzS+C7LNxd9068ccrGvmrS2FOtyCUr1AQ7OHm/+eTdHBBp7+r0yS48Jbt7U028wYMoO4sDgbqzw+fgW9iMwTkR0isktE7u5k+5kislFE3CJyWYdt14rITt/j2q4qvEf4phUMrdzVLW9f1+Tm8Q92Met/3+fHL2/i9hc30eT2dstnKaWOzRjDL5dtJTvvAH/6wWROGRHTbvu28m3k1+T3ib7zbR1zzisRcQKPA3OAfGC9iLxpjNneZre9wHXAXR2OjQXuAzIBA2zwHXuga8rvZgMG0+iKZEjDXipqm4iNCO6St210e3hp7V4e/2AXZTVNfGdMOHNDd/Lzrc3c+bLw6JVTCXLqH1tK9bQ/v/stb24p5Jfz0rlo0tDDtq/YswKXw8V5I86zoboT58/khlnALmPMbgARWQosAFqD3hiT69vW8XR0LvCuMabCt/1dYB7w8klX3hNEaIgeTVpjAbtKashKjT2pt3N7vLy+sYC/vLeTgsp6zkiN4vVTtpG89S9QX8HE4Wcze9v1/OwV4eHLp+DSsFeqxyzbkM8j7+/i8swkbjvr8LFrPF4PK3NXMnPYTKKCo2yo8MT5kyTDgH1tlvN96/zh17EicrOIZItIdmlpqZ9v3TOcg8Yy2lFAzkl0sfR6DW9vLeT8P3/ML/+5lfiIIJbPreYfTT8jee19MGg8nPlLRpR9yKqk51mxNZ+7XtuCR2/UUqpHfJZTxj2vb+WM0XHcf+mEw4YcNsbwUPZDlNSVcMmoS+wp8iT0iunKjTFPA08DZGZm9qp0Cx86jsjtL5NfWAAkH9exxhg+3FHKH1btYHtRFWMGRfLSRWGctutB5KM1EJcGC5fCmHkgAmExjFx1DyuSXczbfA0up4P//d4kHI7eO861Un3drpIabn1hAyPiIvjrVad02mz63Lbn+MfX/+CacddwbvK5NlR5cvwJ+gKg7WAOw33r/FEAnN3h2A/9PLZXcCRaY940Fn0NnOb3cV/sLucPq3awIe8AybHhPHnxEM7f/zSO1UshPBYueAhOuQ6ch7ptcdqPwNPImNW/5a1kFxdtWEiQU/j9JRM17JXqBuU1jfzwuXUEuxw8e910osOCDtvnrZy3+NOGPzE/ZT53Zd7VqycYORJ/gn49kCYiqVjBfQVwpZ/vvwr4fyLScun6fOCe467STr5pBYMqvvVr9635lfxh1Q7W7CxjUFQID35nJJc1/BPn+4+B8cAZd8KsX0BodOdvMPNn4Glm/Ae/51/JLi5Z931cDgeLF4zvk//AlOqtGpo93PT3bEqqGll686kkxYYfts9nBZ9x76f3MmPwDO6feX+vnfz7WI4Z9MYYt4jcgRXaTmCJMWabiCwGso0xb4rIdOANIAb4joj8zhgz3hhTISL/jfVlAbC45cJsnxGdTLMjhLj6XOqbPIQFOzvdbef+av74n29Zua2YmPAgfjN/DNeGfULQxz+Cmv0w/rsw+z6ISTn2Z571S3A3MnnNQ7yWHMRlX1yKyynce9E4DXuluoDXa/jFa1vYuLeSJ66axtTkmMP22Va2jZ9++FNGDRzFw+c8TLCza3rd2cGvNnpjzHJgeYd197Z5vR6rWaazY5cAS06iRns5HNQNGEVahXVBdsKw9mfi+yrq+PPqb3ljUwERwS5+OjuNm4flEf7BtVCyDYZnweUvQtL04/vcc38DniYyP3uEl5OdLPz0YoKcDu6Zn6Fhr9RJeug/O3hnaxH3zM9gvm+2qLb2Ve3jR+/9iNjQWJ6Y/QSRwZE2VNl1esXF2N5OEtMZVbmGDW2Cfn9VA4++v5NX1u/DIcJNs0Zy+/hmotf8Cj55FwaOgO8/B+MusS60HveHCsxZDJ5mTlv7BH9PDuK/Pp5PkFO46/x0DXulTtAr6/fy1w9zWJiVzM1njjxse3l9ObesvgWv8fLE7CdICE+wocqupUHvh/Bh44ja+QZvFJVwIC2BJz/K4bnPcvF4DVdkJfGTGdEkZP8JnnseggfA+fdD1s3gCjm5DxaBef8DnkbOzF7C35Jd3PCBEOR08NPZY7rmh1OqH/lkZxm/fuMrZqXFd3rdq665jh+99yPK6st45vxnSI1OtanSrqVB7wfXoLEAbN68nqe/KKGmyc2lU4bxs7OTSNrxLDz7MLjrYfpNcNYiiOjCMTBE4II/gqeJ8zY9xxPJLm5bDUFOB7efM7rrPkepAPft/mpu+8cGRiVE8vhV0w7rRtnsbebnH/6cHRU7eOTcR5iUMMmmSrueBr0/fGPeRFXv5vSx0/jFnDTG7F8JL14FVfmQfqHVzBLfTcHrcMB3HgGPm/lbn+EvyUH8ZBW4HMItndzBp5Rqr7S6kR8+u57QYCdLfjidqND23SiNMdz36X18Wvgpi09fzJnDz7Sp0u6hQe+P2FSMw8W9MxzETGmCNxdA4SYYMhkufRJSZ3V/DQ4nLHgcPE0s2PYEzcku7loBLqeDG2YGxp+XSnWH+iYPN/49m/LaRl695TSGDQw7bJ+HNz7MW7vf4o4pd3Bp2qU2VNm9NOj94QxC4kYTs+3vsOlxiBoGlz4FE39gnW33WB0u+O7T4Gnism8epSnJya/ehiCn8F+npfRcHUr1EV6v4WevbGZrfiVPXn0Kk4YPPGyfF79+kSVfLeHy9Mu5edLNPV9kD9Cg99fw6VC5z+r2eOrtEHz4zRU9whkElz0Lr17Dld8+TOPwu7j33+ByOLhyxvEN0aBUoHtw5Tes3FbMby4cy9zxh0/7tzJ3JQ+ue5DZybO5J+uegO3NpkHvrwv/CPMfhOAIuysBVzB8/3lYeiXX5fyR5mG/5FdvgMsp/CAz6djHK9UPvLg2j6c+3s01p47otHlzXdE6frXmV0xNnMr/zPofnI7Ob4YMBH3zfl47uEJ6R8i3CAqFK15EUmdxU8UfuGvYVyz651be2JRvd2VK2e6jb0u599/bODs9gfu+c/gd5TsqdvCTD37CiKgRPHLuI4S6Qm2qtGdo0PdlQWGwcCmSdCq3VzzIj4d8wy9e3cKbWwrtrkwp23xTXMXtL24kLTGSx66cdti8DoU1hdy2+jbCg8J5YvYTRIccYdypAKJB39cFR8BVryLDTuFnlf/DzYN38rNXNrPiyyK7K1Oqx5VUNXD9s+sJD3by7A+nExnSvnW6sqGSW969hQZPA0/NforBEYe32wciDfpAEDIArl6GDJ7Aoqrfc11iDj9+eRP/2VZsd2VK9Zhmj5cfvbiRA3XNLLluOkOi23ejrHfXc/v7t1NYU8ij5z7K6Jj+c8OhBn2gCI2Gq19HEtL5Tc39XJmwh9tf2sj73+zv0o8xxlDT6GZfRR1b9lXywY4SPvq2lGaPTmqu7PXQf3aQnXeAB7438bDBB91eN7/86Jd8WfolD575IKcMOsWmKu2hvW4CSXgsXPNv5PmL+N2B+2mMu49bXxD+79pMzhrT+cBMjW4PB2qbqaht4kBdExW1hx4ty9ZzMxW1jRyobaapk1AfHBXKNaeN4IrpScRFnuQYP0odp/e+3s9TH+1mYVYyC6a0n63UGMP9X9zPh/kf8psZv2H2iNk2VWkfMaZXzdxHZmamyc7OtruMvq2mBJ67EHOwgF+E/pZ3DiRxxfQkqhvcVNQ1caC2iYq6Jipqmqht8hzxbaLDgoiNCCYmPIjYiBBiI4KIiQgmNjy49Tk2Mpiy6kZe+CKPNTvLCHY5uHjyUK47PeWwsyqlukNBZT0XPrKGIdFhvPGj0wkNat9N8vHNj/Pklie5aeJN3DntTpuq7H4issEYk9nZNj2jD0SRifBfbyLPXcBDNYtxJvw3r20QYsKDiYsMJiY8mJEJkcSEB7cL77hQiHc1EOOsY4CpwdVUDQ1lUH8AGg5CQ6X1XFx56HV9JThcnD/lSnLO+z7Pbq7inxsKWLYhn+kpMVx3eirnjx/U6TycSp2sJreXO17aiNtj+OtV0w4L+Vd3vMqTW57k0tGX8uOpP7apSvvpGX0gO5gPz15ghfLMn0NzvfW6vvJQcLd93Vx39PdzhULoQAgbaF0TaHldXQR7Pra2T/w+1VNv4pW8ATz/eS77KuoZEh3K1adqs47qeve/vZ1nPtnDY1dO5aJJQ9tte2/ve/z8w59zxtAz+Mu5fyHIcfh8sIHkaGf0GvSB7kAePP8dqMyzlkOirZAO8wV1aLQvuAe2CfGO633HBB3lppKSr2Htk7DlFWvI5pRZeGbcygeeaTz3xT4+2WU16yyYPJRrtVlHdYH/bCvm5hc2cM2pI/jvSya027apZBM3/ecmxsSM4ZnznyE8yKYhS3qQBn1/52mGxmorrLv7Nu+6Ctj4PKx7xhrCOSYFsm4hZ9gClmyo4PWNBdQ3e8hKieXa01OYO37QYTe0KHUs+yrquPCRNSTHhfPP204nxHXo33VOZQ7/teK/iA2N5e/z/05M6OHzwQYiDXrV8zxu+OYt+OJJ2PcFBEfClKuonnQ9r+wJPqxZZ2FWMrERfXfyZdVzmtxevv/kZ+wureXtO2cyIu7Q0CTFtcVcs+Ia3F43L8x/geEDOp3KOiBp0Ct7FW6yAv+rf4LXDWnn48m6lfebxvH853ntmnWuOyOF8UO1WUcd2e/e2sazn+byxFXT2k3sXe+u59oV17K3ei/Pzn2WsXFjbayy52nQq96hej9kL4Hsv0FtKSRkwIxb2Dn4Ip5bv79ds851Z6Rw/jht1lHtrfyqiFv/sZHrTk/htxePb11vjOGXH/+SVbmreOy8xwJuhih/nHTQi8g84C+AE3jGGPNAh+0hwN+BU4By4HJjTK6IpABfAzt8u35hjLn1aJ+lQd8PuBvhq9dh7RNQtMW64HvKtVRN/CGv7jSHNetcPj2JeO2t0+/tLa/jwkfXMDI+gtduPZ1g16GTgP/b+n88sukRfjrtp9ww8QYbq7TPSQW9iDiBb4E5QD6wHlhojNneZp8fAZOMMbeKyBXApcaYy31B/7YxZkInb90pDfp+xBjY+4UV+F+/BQiMvchq1qkd2dqs43II541N5AeZSZw1JkHP8vuhRreHy574nLzyWt65cxZJsYd60Xy470PufP9O5qfO54FZDwTs5CHHcrI3TGUBu4wxu31vthRYAGxvs88C4Le+18uAx6S//raV/0RgxGnWo3IfrP8/2PA8zu3/Zs6QycyZcRu7LpzDq5tKeX1jPqu27SdxQAjfO2U43z9lOCMTIu3+CVQP+f07X/NlwUGevuaUdiG/68AuFn28iLFxY/nd6b/rtyF/LP6c0V8GzDPG3OhbvgaYYYy5o80+X/n2yfct5wAzgEhgG9ZfBFXAb4wxazr5jJuBmwGSk5NPycvL64IfTfVJTbWw9RXr4m3ZDohIhKlX0zz5Kj4oieTV7H18sKMUj9cwPSWG72cmceHEIUSE6E3egertrYXc8dImbpyZym8uGte6/mDjQRa+s5C65jqWXrS03ww5fCQn23RzMkFfDUQaY8pF5BTgX8B4Y0zVkT5Pm24UYDXr7P4A1j4NO1eB8cKImTD1akqS5vL6Vwd4df0+dpfVEhHs5KJJQ/nB9OFMS47Rs7oAsqeslu88+glpgyJ59ZbTWofScHvd3Lr6Vjbu38iSuUuYkjjF3kJ7gZNtuikA2k5EOty3rrN98kXEBUQD5cb6FmkEMMZs8H0BjAE0ydXRicCoc61HVRFseQk2/QP+dSuJwQO4deL3uOWKa9jQPJFXN+Tz1tZCXsnex8iECH6QmcR3pw0jcUBgTw8X6BqaPdz+4kZcTuGxK6e1Gy/poeyHWFu0lsWnL9aQ94M/Z/QurKaX87ACfT1wpTFmW5t9bgcmtrkY+11jzA9EJAGoMMZ4RGQksMa3X8WRPk/P6NURGQN5n8GmF2Dbv6yhFhLHwdSrqc24jHdymngtex/rcw/gdAjnpCfyg8zhnJORqIOq9UG/euNLXlq7lyXXZXJuxqDW9a/vfJ37PruPq8dezaKsRTZW2Lt0RffKC4CHsbpXLjHG/F5EFgPZxpg3RSQUeAGYClQAVxhjdovI94DFQDPgBe4zxrx1tM/SoFd+aaiybsDa9AIUbABHEGRcAFOvIScqi9c2FvHPjfmUVjcSHxnC96YN4/uZwxmdOMDuypUf/r25gJ8s3cwtZ43knvmHbnzaVLKJ61ddz/RB0/nr7L/icui1mRZ6w5QKbPu3W806W5dCXTkMGApTrsQ9+So+Ko3glfX7eP+bEtxew7TkgfwgM4kLJw1hQGhgj2bYV+WU1nDxo58wdkgUL998autfY8W1xVz+9uVEBkXy0oUv9YtJvY+HBr3qH9xNsGO5Ffo571kXcFNmwdRrKE2ay7++quCV7H3sKqkhLMjJBROH8N1pw8hMiWk3KJayT32Th0v/+ikl1Y28c+fM1nlf2w5v8NIFLzFy4EibK+19NOhV/3OwADa/ZDXtVOZZwzNPvAwz9Wo2uVN4bUM+b20poqbRTViQkxkjY5k5Op5ZaQmMGRSpPXdssmjZVl7J3sdzP5zO2emJQPvhDR4991HOSjrL5ip7Jw161X95vZD3CWx8Ab5+E9wNMGgCTL2Guozv8lmh4ZNdZXy8s5TdpbUAJA4IYWZaPLPS4jljdLz23ukhr2/M5+evbuH2c0bx/83NaF3/zJfP8JeNf+En037CjRNvtLHC3k2DXimwZtP6apkV+kWbwRkMaedbj9GzKTCxfLrTCv1Pd5VxoK4ZgIzBA5iVFs/MtASyUmIJC9Zmnq62c381Fz/2KZOGR/PijTNah7loGd5gXuo8Hpz1oP6ldRQa9Ep1VPyl1Za//U2oLrTWJY6D0bNh9Gy8w2ewvbSRNTvLWLOzlOzcAzR5vAS7HExPiWHm6ARmpcUzbkgUDoeGz8moa3Kz4LFPqahtYvlPZjEoyvoLKqcyh6uWX8WIqBE8N+85wlxhNlfau2nQK3UkxljTIO5abT3yPgNvMwRFwMizYPR5MHoO9RHDWZdbwZpvS/lkVxnfFFcDEBsRzBmj45k1Op6ZafEMHahhdLx+8eoWXt+UzwvXz2BmWjygwxuciJO9M1apwCUCg8ZZjzPuhMYayF0DO9+FXe9avXiAsLg0zho9m7MyZsPcMyhpED7dVcaab8tYs6uMt7ZYfxWMSohgVloCM0fHc+qoOCJ1DJ6jejV7H//cmM+d56W1hrzb6+auj+6iuLaYJXOXaMh3AT2jV+pIjIHyHCvwd62G3E+si7muMEiZaTXzpM3BxKTybUkta3aWsmZnGWv3lNPQ7MXlEKYlxzArLZ5ZYxKYOCwaZy9v5vF6DTtLasgprWFQVCjJseHERwZ3S9v4juJqFjz+CdOSY3jhhhmtv5sH1z3IP77+B4tPX8ylaZd2+ecGKm26UaorNNdD7qeHgr98l7U+JsXXtj8HUmfR6AhlQ94B1uws45OdZXxVeBBjIDosiDNGx7W277cdbtcuHq/h66Iq1u2pYO2ectbtqWi9CN0iPNhJcmw4SbHhJLd5JMWGMzwmjNCg4784Xdvo5uLHPuFgvZvlP5nZ2rPpjZ1vcO9n9+rwBidAg16p7lCxx9e2/x7s+Qia66yePCNOb72oS0IG5bVNfJpTzie+M/6igw0ApMZH+Prux3PaqLgeuVPX7fGyrbCKtXvKWbu7gvW5FVQ1uAFIig0jKyWOGSNjGTckipLqBvaW17G3op69FXXsq6hjb0Ud9c2edu852Hfm3/pFEBfm+zKI6PSvAWMMP3tlM29uKeQfN87g9FFWk83mks38cNUPyRyUyROzn9DhDY6TBr1S3c3dCHs/t4J/52oo/dpaH5cG4y+1HoljMUBO6aFmni92l1PX5MHpEKYmDWRWWgKzxsQzaVh0l8yk1eT28mVBJV/srmDtngo25FZQ22QFdWp8BDNSY5kxMpYZqXF+XUg2xlBW09Qa/Hnlde2+BIqrGtrtHxbU8a+BMCpqm3jk/V38fM4Y7jwvDbCGN7ji7SsIDwrn5Qtf1uENToAGvVI97WA+fLsKtv/Lats3XohPbxP61g1BTW4vG/ceaA3+LwusZp4BoS7OGGX15DkzLYHkOP+aeRqaPWzeV9naFLMh7wANzV4A0hIjW0N9RmosiVFdfyNYQ7OH/AP1rcHf8mhZrvN9ycxKi+e5H2bhdEi74Q1evOBFRg0c1eV19XrGQMNBq3kwasgJvYUGvVJ2qimx7srd9i8r9DGQMPZQ6CeMad31QG0Tn+b4evPsLKXQ18wzIi68dYiG00bFER1mNfPUNbnZtLeStbvL+WJPBZv3VdLk9iICGYOjrDP21FiyUmOJs3mCdWMM5bVNFB9sIH3wAIKcDowxLPp4EStzVwbe8AbGQFON9f9/TQnU7IfaUuu5ZV1tyaHXnkYYngU3vntCH6dBr1RvUb3fF/pvWH32MZA43hf6l0B8Wuuuxhh2l9Xyie+mrc9zyqlt8uAQmJI0EICt+Qdxew0OgQnDoslKiWXGyDiyUmKJDj+xNn+P10N5QzlxoXE4Hd17F3CfHN6gqa59QLcEd8d1taXWdZuOxAHh8RCZaD0ifM+RgyB2pDXc9gnQoFeqN6oqOhT6ez+31g2aaAX++Eshrn0TRpPby+Z9la3NPA6BrFTr4mnmiJgTvpjrNV52HtjJuuJ1rCtex4b9G6huqibIEUTSgCSSo5IZMWAEI6JHWM9RI0gMTzzpLpc9NryBxw1N1dY9Ek211ll2U02b5WrrubGmk20dlhurobm2888Jj7PCOiLBej4syH1hHh4H3fAFqkGvVG9XVQjb/22F/r611rrBkw6d6cd23bC8xhhyKnNYV7yO9cXryd6fTWVjJQDDI4eTNSSL9Jh09tftJ68qj7yqPPZV76PR09j6HmGuMJIGJDEiakS7R/KAZGJDY48Z2i3DGyQPSOb5+c8f3/AGnmbrjLm62Pq9VRdZj6oiaziL2vL24d2m7qMTCI6E4AgI8T0HD2iz7HtExB8K7ZZQj4gHp73zG2jQK9WXHMw/FPr56611Q6YcCv2YlON6O2MMuVW5rC9e3xruFQ3WbJ5DI4YyffB0soZkMX3QdIZEdn4h0Gu87K/dT151HnkH86znqjz2Vu0lvzoft3G37jsgaID1V0BL+EclkxKVQnJUMlHBUUce3sAYqD/QJrRbArzQCvXqQmt9bSnQIbccLhgwxHpEJEBIJwF9tPAOjoCgcHD03SknNeiV6qsq9x4K/YIN1rqh06zAHzrVau8Vp+/Zehggv6GMdQe+Yd2Br1lfsZ3SxgMAJIbGkRU3kayESUyPn8LwyKHtjkUch8LO6wXjAa8HvO42rz2+127wenB7miis209ebRF5dUXk1e0nr76EvfWlFDYewLQJ5VhXOEHipMJdy5KY05nS0HAo0KuLrTuPOwqPs2YNGzDY6pHSEuhRvnUDhvqaQ/puSHeFfhH0bq+bP234E0MihjA0YihDI61HVHCUDm2qAsOBPKu75rY3oHBTu02FLifrQkNZHxrCurBQil3WzUZxbg9ZDQ1Mb2ggq76RZLebnvyvoVEg3+UiLyiIvCAXea4gCoOcfL+qhjlN+IK7bYi3vB7qC/TB4LK3t1Bf0S8GNSuvL2fZt8uod9e3Wx/uCmdo5FDrC6DN89DIoQyNGEpcWBwO6d9nAqpvMAOTqc+6iaopl1NV9g07irJZX/kt6yq/paCxHIAYVwSZUSO5IWokWQNSSQ2JQzBWP37je/Z6fMtHexjrgqE4rGYRh9P6y6Hldeuyb504rTPq1tfWfiHiYJTDxah2xzsgLBZCo61B5VS3C5igHxQxiLVXrqWysZLC2kKKaooorCmkqLaIgpoCimqL2FK6haqmqnbHBTmCGBIxhCGR1l8CQyKHMCxyWOsXQmJ4IkEOnURadY1mTzNVTVVUN1VT3VTd+rqqqard69btjVVUNx/a1+11t3u/qOAopg+ezjWDpzN98HRGDxytJy7qMAET9AAiQkxoDDGhMYyPG9/pPrXNta1fAIU1hRTWFlrLNUWsKVhDWX1Zu/0d4iAxPLH17D/UGUqIK4RQZyihrlBCnB1et3lu2TfMGUaIK6TdvnaO42GMwWu8eI0Xt3Fbz17r2WM8eLwe69n3uu1+Hbe1vMaAFy/GmNY2Wa85tGwwh+1jMNbrjssdnlvr9r1vy7q2bb8dt3X8edvt0+a4lt9Dy8/e8WdsXe/1tl/ueEyb30fb5Zrmmnbh3fEvzo5cDhdRwVGtj+jQaJIGJDEgeABRIVEMCB5gvQ6OYkTUCMbEjNFgV8fkV9qIyDzgL4ATeMYY80CH7SHA34FTgHLgcmNMrm/bPcANgAe40xizqsuqPwERQRGkxaSRFpPW6fZGTyPFtcXtvwx8Xwg5lTk0ehppcDdYz56Gw86w/OUSV7svjGBnMECnQdfuGSukrdz0f5+OAaaOTRCc4sQhDpwO69khjkPrOj47nO3WOR1OIoMiSYlKaRfQLaHd+rrNc4gzRK8pqS53zKAXESfwODAHyAfWi8ibxpjtbXa7AThgjBktIlcADwKXi8g44ApgPDAUWC0iY4zpvUkT4gxp7RbmD7fX3Rr+DZ4GGt3WF0Db5UZPI/Xueho97V+3fmH4nsH6C0IQrP9J63LLf/xtl4XD1x1pn5bgaXl2iAOXuKxnh+tQOB1lv9b10n57S7gBrZ/Z8edoWXe0fVrr9q1rt+zTMQQ729ayrrPA7Lit5XM7Bnrbn0mpvs6fM/osYJcxZjeAiCwFFgBtg34B8Fvf62XAY2L9l7QAWGqMaQT2iMgu3/t93jXl28/lcOFyuIgIirC7FKWU6pQ/pyzDgH1tlvN96zrdxxjjBg4CcX4ei4jcLCLZIpJdWlrqf/VKKaWOqVf8bWqMedoYk2mMyUxISLC7HKWUCij+BH0BkNRmebhvXaf7iIgLiMa6KOvPsUoppbqRP0G/HkgTkVQRCca6uPpmh33eBK71vb4MeN9YfdreBK4QkRARSQXSgHVdU7pSSil/HPNirDHGLSJ3AKuwulcuMcZsE5HFQLYx5k3gb8ALvoutFVhfBvj2exXrwq0buL0397hRSqlAFDBj3SilVH92tLFuesXFWKWUUt1Hg14ppQJcr2u6EZFSIO8k3iIeKDvmXr1PX60btHa7aO326K21jzDGdNo/vdcF/ckSkewjtVP1Zn21btDa7aK126Mv1q5NN0opFeA06JVSKsAFYtA/bXcBJ6iv1g1au120dnv0udoDro1eKaVUe4F4Rq+UUqoNDXqllApwARP0IjJPRHaIyC4RudvuevwlIkki8oGIbBeRbSLyE7trOl4i4hSRTSLytt21HA8RGSgiy0TkGxH5WkROs7smf4jIz3z/Vr4SkZdFJNTumo5GRJaISImIfNVmXayIvCsiO33PMXbW2Jkj1P0H37+XrSLyhogMtLFEvwVE0LeZ7nA+MA5Y6JvGsC9wA78wxowDTgVu70O1t/gJ8LXdRZyAvwArjTEZwGT6wM8gIsOAO4FMY8wErIEGr7C3qmN6DpjXYd3dwHvGmDTgPd9yb/Mch9f9LjDBGDMJ+Ba4p6eLOhEBEfS0me7QGNMEtEx32OsZY4qMMRt9r6uxwuawWbh6KxEZDlwIPGN3LcdDRKKBM7FGXsUY02SMqbS1KP+5gDDf3A/hQKHN9RyVMeZjrFFt21oAPO97/TxwSU/W5I/O6jbG/Mc3ix7AF1hzbPR6gRL0fk1Z2NuJSAowFVhrcynH42Hgl4DX5jqOVypQCjzra3Z6RkR6/cS/xpgC4CFgL1AEHDTG/Mfeqk7IIGNMke91MTDIzmJO0PXACruL8EegBH2fJyKRwD+Bnxpjquyuxx8ichFQYozZYHctJ8AFTAOeMMZMBWrpnc0H7fjashdgfVENBSJE5Gp7qzo5vkmK+lQ/bxH5NVaz64t21+KPQAn6Pj1loYgEYYX8i8aY1+2u5zicAVwsIrlYzWXnisg/7C3Jb/lAvjGm5a+nZVjB39vNBvYYY0qNMc3A68DpNtd0IvaLyBAA33OJzfX4TUSuAy4CrjJ95EakQAl6f6Y77JVERLDaib82xvzJ7nqOhzHmHmPMcGNMCtbv/H1jTJ84uzTGFAP7RCTdt+o8rJnQeru9wKkiEu77t3MefeAicifaTj96LfBvG2vxm4jMw2qqvNgYU2d3Pf4KiKD3XRxpme7wa+BVY8w2e6vy2xnANVhnw5t9jwvsLqqf+DHwoohsBaYA/8/eco7N9xfIMmAj8CXWf8O9+pZ8EXkZ+BxIF5F8EbkBeACYIyI7sf5KecDOGjtzhLofAwYA7/r+W33S1iL9pEMgKKVUgAuIM3qllFJHpkGvlFIBToNeKaUCnAa9UkoFOA16pZQKcBr0SikV4DTolVIqwP3/s0buYom3ux0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_38_6.png"
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from sklearn.linear_model import LinearRegression, Ridge, Lasso\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.utils import resample\n",
    "\n",
    "np.random.seed(2018)\n",
    "\n",
    "n = 40\n",
    "n_boostraps = 100\n",
    "maxdegree = 14\n",
    "\n",
    "\n",
    "# Make data set.\n",
    "x = np.linspace(-3, 3, n).reshape(-1, 1)\n",
    "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)\n",
    "error = np.zeros(maxdegree)\n",
    "bias = np.zeros(maxdegree)\n",
    "variance = np.zeros(maxdegree)\n",
    "polydegree = np.zeros(maxdegree)\n",
    "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2)\n",
    "\n",
    "for degree in range(maxdegree):\n",
    "    model = make_pipeline(PolynomialFeatures(degree=degree), LinearRegression(fit_intercept=False))\n",
    "    y_pred = np.empty((y_test.shape[0], n_boostraps))\n",
    "    for i in range(n_boostraps):\n",
    "        x_, y_ = resample(x_train, y_train)\n",
    "        y_pred[:, i] = model.fit(x_, y_).predict(x_test).ravel()\n",
    "\n",
    "    polydegree[degree] = degree\n",
    "    error[degree] = np.mean( np.mean((y_test - y_pred)**2, axis=1, keepdims=True) )\n",
    "    bias[degree] = np.mean( (y_test - np.mean(y_pred, axis=1, keepdims=True))**2 )\n",
    "    variance[degree] = np.mean( np.var(y_pred, axis=1, keepdims=True) )\n",
    "    print('Polynomial degree:', degree)\n",
    "    print('Error:', error[degree])\n",
    "    print('Bias^2:', bias[degree])\n",
    "    print('Var:', variance[degree])\n",
    "    print('{} >= {} + {} = {}'.format(error[degree], bias[degree], variance[degree], bias[degree]+variance[degree]))\n",
    "\n",
    "plt.plot(polydegree, error, label='Error')\n",
    "plt.plot(polydegree, bias, label='bias')\n",
    "plt.plot(polydegree, variance, label='Variance')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The bias-variance tradeoff summarizes the fundamental tension in\n",
    "machine learning, particularly supervised learning, between the\n",
    "complexity of a model and the amount of training data needed to train\n",
    "it.  Since data is often limited, in practice it is often useful to\n",
    "use a less-complex model with higher bias, that is  a model whose asymptotic\n",
    "performance is worse than another model because it is easier to\n",
    "train and less sensitive to sampling noise arising from having a\n",
    "finite-sized training dataset (smaller variance). \n",
    "\n",
    "\n",
    "\n",
    "The above equations tell us that in\n",
    "order to minimize the expected test error, we need to select a\n",
    "statistical learning method that simultaneously achieves low variance\n",
    "and low bias. Note that variance is inherently a nonnegative quantity,\n",
    "and squared bias is also nonnegative. Hence, we see that the expected\n",
    "test MSE can never lie below $Var(\\epsilon)$, the irreducible error.\n",
    "\n",
    "\n",
    "What do we mean by the variance and bias of a statistical learning\n",
    "method? The variance refers to the amount by which our model would change if we\n",
    "estimated it using a different training data set. Since the training\n",
    "data are used to fit the statistical learning method, different\n",
    "training data sets  will result in a different estimate. But ideally the\n",
    "estimate for our model should not vary too much between training\n",
    "sets. However, if a method has high variance  then small changes in\n",
    "the training data can result in large changes in the model. In general, more\n",
    "flexible statistical methods have higher variance.\n",
    "\n",
    "\n",
    "You may also find this recent [article](https://www.pnas.org/content/116/32/15849) of interest."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "============================\n",
      "Underfitting vs. Overfitting\n",
      "============================\n",
      "\n",
      "This example demonstrates the problems of underfitting and overfitting and\n",
      "how we can use linear regression with polynomial features to approximate\n",
      "nonlinear functions. The plot shows the function that we want to approximate,\n",
      "which is a part of the cosine function. In addition, the samples from the\n",
      "real function and the approximations of different models are displayed. The\n",
      "models have polynomial features of different degrees. We can see that a\n",
      "linear function (polynomial with degree 1) is not sufficient to fit the\n",
      "training samples. This is called **underfitting**. A polynomial of degree 4\n",
      "approximates the true function almost perfectly. However, for higher degrees\n",
      "the model will **overfit** the training data, i.e. it learns the noise of the\n",
      "training data.\n",
      "We evaluate quantitatively **overfitting** / **underfitting** by using\n",
      "cross-validation. We calculate the mean squared error (MSE) on the validation\n",
      "set, the higher, the less likely the model generalizes correctly from the\n",
      "training data.\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAykAAAFLCAYAAADSycMSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAACUQElEQVR4nOzddXiV5RvA8e9zzro32KgBo7u7WwFFpBktAqIiij8Du7AVC5CUkJZSVJTukO6G0TE2NjZY7/398Z6NMdZsO7H7c127YOet59S9935SaZqGEEIIIYQQQlgKg7kLIIQQQgghhBApSZIihBBCCCGEsCiSpAghhBBCCCEsiiQpQgghhBBCCIsiSYoQQgghhBDCokiSIoQQQgghhLAokqQIIYQQQgghLIokKVZIKRWklIpSSkUopcKUUtuVUiOVUhb/fiqlHJRSS0zPQVNKtTZ3mYQoyKw5nqSklHrfFFPam7ssQhQU1hw/MrsfUUp9qJSKU0pFpvgpa57SFkwW/yES6eqiaZo7UBr4AngTmJEXF1JKGXP5lFuBAcD1XD6vECJnrDmeoJQqB/QCruX2uYUQmbLm+JHZ/cgiTdPcUvycy+XriwxIkmLlNE0L1zTtD6APMFgpVR1AKeWolPpGKXVRKXVDKTVZKeWcdJxS6g2l1DWl1FWl1DBTLUJ507ZZSqmflVJ/K6XuAm2UUsWVUkuVUsFKqfNKqdEpzmVQSo1VSp1VSoUopRYrpXzSKW+spmnfa5q2FUjIy9dGCJE91hZPUpiIfmMUm9uviRAia6wtfsj9iOWTJMVGaJr2H3AZaGF66AugIlAbKA+UAN4HUEp1BF4F2pu2tU7jlP2ATwF3YDuwEjhoOk874BWl1OOmfV8CngZaAcWB2+g3DUIIK2RN8UQp1QuI0TTt7xw8VSFELrOm+JEFXZRSoUqpo0qp5x/hPCIHJEmxLVcBH6WUAkYAYzRNC9U0LQL4DOhr2q83MFPTtKOapt0DPkzjXL9rmrZN07REoAbgq2nax6aah3PAtBTnGwm8o2naZU3TYkzn66mUssuj5ymEyHsWH0+UUu6msrycS89ZCJE7LD5+ZMFioArgCwwH3ldKBebgPCKH5CbStpQAQtG/UC7AXj0+AKCApL6cxYE9KY67lMa5Uj5WGiiulApL8ZgR2JJi+3KlVGKK7QlAEeBKtp+FEMISWEM8+RD4VdO0oEyfjRAiP1lD/MiQpmnHUvy6XSn1A9ATWJCd84ickyTFRiilGqAHha3ALSAKqKZpWlpfymuAf4rfS6axj5bi/5eA85qmVUjn8peAoZqmbct2wYUQFseK4kk7wF8p9YLpd19gsVLqS03TvszC8UKIXGZF8SO7NPQES+QT6e5l5ZRSHkqpJ4GFwFxN0w6bmkSnAd8ppfxM+5VI0WdzMfCMUqqKUsoFeC+Ty/wHRCil3lRKOSuljEqp6qZABDAZ+FQpVdp0LV+lVNcMyuyolHIy/eqglHJSKapYhBDmYYXxpB1QHb2ve230LibPIWPihMh3Vhg/MrwfUUp1VUp5K11DYDTwe3ZfF5FzkqRYr5VKqQj0WoN3gPHAMym2vwmcAXYqpe4Aa4FKAJqmrQJ+BDYk7WM6JiatC2malgA8iX4TcB69ZmQ64Gna5QfgD2C1qUw7gUYZlP0kes1KCeBf0/9LZ+1pCyHygFXGE03TQjRNu570g96t47amaZHZfQGEEDlmlfHDJKP7kb6mMkUAc4AvNU2bneErIXKV0jQt872ETVNKVQGOAI6apsWbuzxCCOsl8UQIkVMSP0RK0pJSQCmlupmaOb2BL4GVEhCEEDkh8UQIkVMSP0R6JEkpuJ4DbgJn0btIyPzfQoickngihMgpiR8iTdLdSwghhBBCCGFRpCVFCCGEEEIIYVEkSRFCCCGEEEJYFElSxCNRSlVVSu2RdU6yTynVRSm1yNzlEMISSCzJOaVUEaXUcaWUo7nLIoQlkHiS+5RSS5VSnfLzmladpCilgpRSsUqpwqke36+U0pRSAabf/U0v7i2lVLhS6ohSaohpW4Bp38hUP33yqMwVlFLRSqm5GeyjlFJfKqVCTD9fpvyiKaXaKqX2KaXuKKXOKaVGPEJ5MrvWVKXUSaVUYtJrlsonwDdaDgY3KaWKK6UuZ/MYB9Mf43SPU0o9oZTaqpQKU0pdV0pNV0q5p9g+y/S5Sfl+G7Nb/hTnG2O6zh2l1C8pbxSUUp8opQ4rpeKVUh+mPE7TtJVANaVUzZxeW+QOG44lY0wx4o5S6qpS6jullJ1pm59SaoHp8XCl1DalVEbrCWRWnnRjiVKqolLqd6VUsFIqVCn1r1KqUqpT5EssUUqtSvX+xCqlDmewfzul1Aml1D2l1AZlWiTOtM1HKbXI9HxvKaXmKaU8slv+LF7rG6XUaaVUhGmfQUnbNE27gb7ORI7/Fojss6a4oZQapfSb9hil1KxM9lVKqXFKqSum8m5USlVLsT3dz2IultdR6X9P7yj97+urqbb3Vvq9QIRS6phS6ulUp8ivePKhUiou1XtXNgvH/WJ638uneGyjKaYnnedkdsuehesGmGLLPdN71z7Ftgzfd/SZ18bldpkyYtVJisl5IDDpF6VUDcAl1T6/oi8yVBooBAwEbqTax0vTNLcUP3lVwz0R2J3JPiOAp4FaQE2gC/rsFyil7IHlwBT0xYv6AOOVUrVyWJ50r2VyEHgB2Jf6QKVUMaANsCKtE5u+vB9mcO3OwD/ZLO/rQHAm+3iif5GKA1XQF2n6OtU+X6V6vxOyWQ4AlL5q7lj0la9LA2WBj1LscgZ4A/grnVMsQG4sLIUtxpI/gLqapnmgr8xeC33VZAA30/H1AB9gNvCXUsoth+XJKJZ4mcpSCSiCvmp08srN+RlLNE3rlPL9AbYDv6Vz3cLAMvRVsH2APUDK93Mc4A2UAcqZnltG5UxXFq51F/019QQGAz8opZqm2D6PB2O3yB/WEjeuon9ef8nCvr2AoUAL9M/iDvTnkCSzz2KWKKWGZJAwfQhUQH/N2gBvKKU6mo4rAcwFXgU80O8L5qv7K9rn973JolTv3bmMdlZKNUePF2kZleI8qStyskTpFbFD0tm8ANiP/jl8B1iilPI1bcvwfdc07T/AQylVPyflyglbSFJ+BVJm8YPRVwZNqQEwS9O0u5qmxWuatt+0ymm+Ukr1BcKAdZnsOhj4VtO0y5qmXQG+BYaYtvmgfyl/1XS7geNA1RTXGWqqYbit9BrLjFZzz+haaJo2UdO0dUB0Gsd2APZpmpbWtqzoDPyd1Z2VUmWAAcDnGe2nadp8TdP+0TTtnqZpt4FpQLNsXOdJpdQBpbfEbFcZt3QMBmZomnbUdK1PePD1m236rEWkc/xG4Imslk3kKZuLJZqmndU0LSzpMCARKG/adk7TtPGapl3TNC1B07SpgAOmlaBN18mVWKJp2n+aps3QNC1U07Q44DugklKqkOnYfI0lSUw13S14+H1O0h04qmnab6ayfQjUUkpVNm0vA6zQNO2Opmnh6BVIKWucKyul1ii99eikUqp3BsXJ8Fqapn2gadoJTdMSNU3bBWwBmqQ4fhdQNpP3SOQ+q4gbmqYt0zRtBRCShd3LAFtNMSIBPSFIvsfI7LOolGps+tsZppQ6qJRqnYMiDwY+0TTttqZpx9H/jg8xbfMHwjRNW2W6D/oLPXFKuvE3SzzJCqW3ZP8EvJSDY7MTj9M6viJQF/hA07QoTdOWAoeBHqZdMnzfTTaSj/cstpCk7ETP7KoovctOX/QXNvU+E5VSfZVSpR7lYkqpSaYvXlo/hzI4zgP4GD3zz0w19BaMJAdNjyU16y8AnlFKGZVSTdBrGraartMVeBv9D54vevBYkJNrZUENIEfNkaYWoZbAmmwc9hP6c4vK5uVaAkdTPfaC6cZhr1Iq6QuKUqoOek3Tc+g1DVOAP1T6fb3Tev2KpLj5ysxxIEA9QhcRkWtsMZaglOqnlLoD3EJv5ZiSzn610ZOUM6bf8zKWtASua5qWdMOU37EkySBgi6ZpQelsf+A5aZp2F30th6TnNRF4UinlrfSF6HoAq0zlcjWVaT7gh/55mqSUSv1HP6vXSqaUcka/8T2aYv949Pcup63qImesIm5k00KgnNK7adqjJwxptiyk/iyaWjn+Qm+18QFeA5amqK3PlOm7VIz048ke4LhS6inTfdDTQAyQ9PzzO550Md1PHFVKZbbGyxhgs6Zp6b1Xnyu9W+C2lMldDuJxWqoB5zRNS1lpmvJ1zcr7fpx8jDG2kKTA/ZqMDugv4JVU23uhv6HvAedNteQNUu1zK9WXvUpaF9I07QVN07zS+cmoxv0T9Br3rPRzdAPCU/weDrgplTxWZAHwPvqXcgvwjqZpl0zbRgKfa5p23PRH6zOgdgYZd2bXyogX6bcQZKYlcDDVlyVdSqlugFHTtOXZuYhSqgP6F+39FA//iN6M7If+mZillEpqaRkBTNE0bZepdnk2+uvcOJ1LpPX6AbinsW9akp6/Vxb3F3nL1mJJUsuiB1ARmMzD3UySEp9fgY9MLQKQR7FEKeWPfnOfMsnyIp9iSSqDgFkZbE/9nDD9nvT93oee2IWYfhKASaZtTwJBmqbNTKo9B5aif4Zycq2UJqPfXPyb6vEIJJaYgzXEjey4hl7xeRK9UrAX+s11WlJ/FgcAf2ua9reppWUNelLRORvXT+pymjqeuAOYavnnoFcAxJj+fc6U2EP+xpPF6N3KfYHhwPtKqcC0dlRKlUSvAH0/re3Am+hdxksAU4GVSqmk1qHsxuO0ZBZjsvK+52uMsaUkpR96U+BDzfam5sKxmqZVQ+8zfABYkeqPZ+FUX/bjuVU4Uw1le/QuDlkRid6lK4kHEKlpmqb0pv+F6AHRAT0DfkMpldT8Vhq9f2iYUioMCEXv5lFCKfW2uj8ga3Jm18pCOW+T6g+oUurPFNceC4xNEXT/TLFrcnOqUqp/inI91ARuqpH8ivt96bNEKdUYPXj11DTtVNLjmqbt0zQtxHTj8Dd6X+7ups2lgf+l/GMBlASKp1POtF4/yHqATHr9wrLz3ESesbVYkkzTtNPotZ2TUj5uqgldCezUNC1lV8pcjyWm2tTVwCRN01LWAuZLLEl1/uZAUWBJBrulfk5Jzyvp+70YOGUquwd6y0dSLXppoFGqWNIfKKqUKpWinJFZvFZSub9GH1/UO4047Y7EEnOw6LiRA++jt46UBJzQx1muV0o9MNYmnc9iaaBXqs99c/SWkQdagtBjUb8U+ya1LiR9J1LHkwjTOdqj3xO0Rr8PagVMN8VHyMd4omnaMU3TrpoqNbcDPwA909oX+B74OEVFUOpz7dI0LULTtBhTBek27id36cZjU1kPpdjWD73VNun5JcX8zGJMVt73/I0xmqZZ7Q8QBLQ3/X8jcAdwBewADQhI57jqpu2FgADT/+2yeM3J6G90Wj9H0znmFfT+ktdNP5HoWeq+dPbfDgxP8ftQ9BsI0D/8+1Pt/z0wwfT/f4H+2XgN071Wqv22AkNSPTYAWJPBuT8EPkxn23GgShbLWBuIS/H6haLXWF7P4D2uA9wEumTh/D8D403/n4LeMpXV128+8GmK39uid2NJvd/ctF4L9LEy5/Pi+yE/Wf+x1ViSxvED0GsJk353NMWMeYAh1b65GkvQB5jvB75Ip1x5HktSHTcNmJPJPiOAbSl+dwXuAZVNv0cCtVJsr42emIE+mDrd55Tda5ke+wg4AhRK43g70/6lc/v7IT/pvmdWETdSHT8OfXxMRvv8Cbyc6rEwoH6K39P8LAJvAdOy+FyGpFcW9IH+HVL8/jGw0PT/14DlqfZfAbxm+n++x5MUx78JLEtnWxh6S3ZS/NbQJwLql87+q4DRpv9nOR6jtw4PSePxiujji91TPLYZGJmN930a+piW/PmO5deF8qTwDwaIckkvZOoAgT5tWnXT4+7oXQ1Om7ZlK0DksJwu6DV2ST/foNfe+aaz/0jTF6UE+gxVR1N8iMqZglFb9Cy6HHo/5BGm7d1MgaOa6XdPoFcGZUv3WqbtDugZ9Tb0pkwnTDcz6DVCIYBTOudOMxCgD846l43Xzy7V69cdPYAVRe8Clnr/6qZA0Ced8/VEb/Y0AI+h1yK0Nm2rjz4LSyPT6+uKPkjMPZ1zdUQPNlXRm0DXk+ImDLA3vWbz0f84OKUsM3of00nm/i4V9B8bjiXDAD/T/6uavt9JCbk9egvKirTKnJuxBL227j9MlSlpHJsvsSTFcc7o3RzaZrKfr2m/Hqbv7pc8mHhtQB8r52z6mQRsN21zBy6gz+Rkb/ppQDo3QFm41lvAaaBoOsc3BY6Z+7tUkH6sJW6kKJMT+sQzv5r+n+Y1gQ/QKyaLoP+dHIheOeJl2p7uZxG9Fv468DhgNF2nNeCfxr5DSD9J+QLYhF65URm9K1JH07ZW6GPsapt+r2OKH4+Zfs+3eAJ0NZVRAQ3Ru/oNTmdfPx6M3xp6V3Jn9PuHx5PeF/RW17tARdOxWY7HpJOkmLbtRP+74WQ6Zximvx+Zve+mfU4BDfPtO5ZfF8qTwqcIEKkeTx0gfjJ9oSLRs9Y/Mf2h4H6ASF0j8WoelvtDYG6K31tgqn0z/a7QmzJDTT9fASrF9t6mD2sEcBk9ABpSbB+IPmPDHfQb7l8yKEtm19poen1S/rROsf030k8GPiTtQDCKdG5Wsvj6tQYup3osEmhh+v9M9FmM0qxhQu8bHG56fQ4CfVOdqyP61Kxh6IHxN9JJUkz7v4qeFN0xXdsxxbZZabx+Q1JsP0yKmlj5Mc+PDceSmabP5l3Tc/wa0x9u9D/0Gnrte8rytkhxfK7EEvRxYZqpHCmvVSrF8fkWS9BbOS6QItal2HaUFDWW6N3rTqC3WG0kRe04+k3NSvQbolD0QaYVUmyvhD6IONi0z3pMN1bplCuja2no/e9Tvn5vp9g+EVOtq/zkz481xQ3Tdyj136IPTdtKpfw+ot/ATkT/+3cHfexVx2x8FhuhJxihpuf7V8rveor9hpB+kuKIPonNHfQY9mqq7aPQK2gjgHPA/1Jtz5d4gj5GOMT0GpxI/R0kVUxNtU0Dypv+74t+3xGBfu+xkxQtSaZ9shSPyThJCUCPLVHoY0/ap9iW2fvegCy22ufWT9IfECFyROkz1cxGz6yz9GFSSv2NHgjyZIo/a6GU6gIM1DQto2lJhSgQJJbknNLXh9gE1NFyPu2qEDZD4knuU0otRZ+0Jd9eH0lSRL5TSr0B/KRpWnanEhZCiGQSS4QQuUXiieWRJEUIIYQQQghhUWxlCmIhhBBCCCGEjZAkRQghhBBCCGFR7PLipIULF9YCAgLy4tRCiBzau3fvLU3TfM1djuySeCKE5bGFeBJ06y7xiRrl/dwyPkgIkWcyiiV5kqQEBASwZ8+evDi1ECKHlFIXzF2GnJB4IoTlsYV48szM/7gVGcvKl5qbuVRCFFwZxRLp7iWEEEKIAkcphYZMHiSEpZIkRQghhBAFjgJkglMhLJckKUIIIYQocJSSJEUIS5YnY1JEwRIXF8fly5eJjpaFji2Bk5MT/v7+2Nvbm7soQmSLxBLLY9vxRElnLxsm8cSy5CSWSJIiHtnly5dxd3cnICAApZS5i1OgaZpGSEgIly9fpkyZMuYujhDZIrHEsth6PNFbUiRNsVUSTyxHTmOJdPcSjyw6OppChQpJELAASikKFSokNUfCKkkssSy2Hk/kU2bbJJ5YjpzGEklSRK6QIGA55L0Q1kw+v5bFlt8PGZNi+2z582ttcvJeSJIibIJSigEDBiT/Hh8fj6+vL08++WS2zhMQEMCtW7ceeR8hhHWSWFJwKGQKYpG3JJ48GklShE1wdXXlyJEjREVFAbBmzRpKlChh5lIJIayNxJKCQ1pSRF6TePJoJEkRNqNz58789ddfACxYsIDAwMDkbaGhoTz99NPUrFmTxo0bc+jQIQBCQkJ47LHHqFatGsOGDXtgEOXcuXNp2LAhtWvX5rnnniMhISF/n5AQwiwklhQMSiHtKCLPSTzJOUlShM3o27cvCxcuJDo6mkOHDtGoUaPkbR988AF16tTh0KFDfPbZZwwaNAiAjz76iObNm3P06FG6devGxYsXATh+/DiLFi1i27ZtHDhwAKPRyLx588zyvIQQ+UtiScGgUDK7l8hzEk9yTqYgFrnqo5VHOXb1Tq6es2pxDz7oUi3T/WrWrElQUBALFiygc+fOD2zbunUrS5cuBaBt27aEhIRw584dNm/ezLJlywB44okn8Pb2BmDdunXs3buXBg0aABAVFYWfn19uPi0hRAYklog8Jy0pBYbEE+skSYqwKU899RSvvfYaGzduJCQkJMfn0TSNwYMH8/nnn+di6YQQ1kJiie1TIFmKyBcST3JGkhSRq7JSq5CXhg4dipeXFzVq1GDjxo3Jj7do0YJ58+bx3nvvsXHjRgoXLoyHhwctW7Zk/vz5vPvuu6xatYrbt28D0K5dO7p27cqYMWPw8/MjNDSUiIgISpcubaZnJkTBIrFE5DWlZMX5gkLiiXWSJEXYFH9/f0aPHv3Q4x9++CFDhw6lZs2auLi4MHv2bEDvDxoYGEi1atVo2rQppUqVAqBq1aqMGzeOxx57jMTEROzt7Zk4caLNBgIhxIMkltg+haw4L/KHxJOcUXnxBa1fv762Z8+eXD+vsEzHjx+nSpUq5i6GSCGt90QptVfTtPpmKlKOSTwpOCSWWCZbjScvL9zPgUthbHq9jZlLJfKCxBPLk91YIrN7CSGEEKLA0VtSzF0KIUR6JEkRQgghRIGjj0mRLEUISyVJihBCCCEKHGlJEcKySZIihBBCiIJHSZIihCWTJEUIIYQQBY7SV0oRQlgoSVKEEEIIUeAoJVMQC2HJJEkRVi8kJITatWtTu3ZtihYtSokSJZJ/j42NzZVrbNmyhWrVqlG7dm2ioqJy5ZwAn3322QO/N23aNNfOLYTIHoklBYtCFpwXeUfiyaOTJEVYvUKFCnHgwAEOHDjAyJEjGTNmTPLvDg4OxMfHP/I15s2bx1tvvcWBAwdwdnbOhVLrUgeC7du359q5hRDZI7GkYFEyJkXkIYknj06SFGGThgwZwsiRI2nUqBFvvPEGH374Id98803y9urVqxMUFATA3LlzadiwIbVr1+a5554jISHhgXNNnz6dxYsX895779G/f382btzIk08+mbx91KhRzJo1C4CAgAA++OAD6tatS40aNThx4gQAkZGRPPPMM9SoUYOaNWuydOlSxo4dS1RUFLVr16Z///4AuLm5AXoXhNdff53q1atTo0YNFi1aBMDGjRtp3bo1PXv2pHLlyvTv31+6KwiRhySW2C6FTEEs8pfEk+yRJEXYrMuXL7N9+3bGjx+f7j7Hjx9n0aJFbNu2jQMHDmA0Gpk3b94D+wwbNoynnnqKr7/++qFtaSlcuDD79u3j+eefTw4+n3zyCZ6enhw+fJhDhw7Rtm1bvvjiC5ydnTlw4MBD5122bBkHDhzg4MGDrF27ltdff51r164BsH//fr7//nuOHTvGuXPn2LZtW3ZfGiFENkgssU3SkiLMQeJJ1tk98hmESGnVWLh+OHfPWbQGdPoi24f16tULo9GY4T7r1q1j7969NGjQAICoqCj8/PxyVMwk3bt3B6BevXosW7YMgLVr17Jw4cLkfby9vTM8x9atWwkMDMRoNFKkSBFatWrF7t278fDwoGHDhvj7+wNQu3ZtgoKCaN68+SOVWQiLI7FEYkkeU8qyx6TExCew81wo64/foExhV4Y0K2PuIlkviSdWGU8kSRE2y9XVNfn/dnZ2JCYmJv8eHR0N6E2XgwcP5vPPP8/yedM7VxJHR0cAjEZjrvQ5TS3p/Hl5DSHEfRJLbJWy2JaUqNgEHv9+MxdD7wFgZ1A8Xr0oxTxzb9yBMA+JJ1knSYrIXTmoVcgPAQEB/PnnnwDs27eP8+fPA9CuXTu6du3KmDFj8PPzIzQ0lIiICEqXLp3uuUqXLs2xY8eIiYkhKiqKdevWZVpb0KFDByZOnMj3338PwO3bt/H29sbe3p64uDjs7e0f2L9FixZMmTKFwYMHExoayubNm/n666+T+5EKYfMklqRJYknuUQostS3l36PXuRh6j8+61aBhGR8e/34zM7cF8XbnKuYumnWSeJImS48nMiZFFAg9evQgNDSUatWqMWHCBCpWrAhA1apVGTduHI899hg1a9akQ4cOyf0r01OyZEl69+5N9erV6d27N3Xq1Mn0+u+++y63b9+mevXq1KpViw0bNgAwYsQIatasmTw4LUm3bt2oWbMmtWrVom3btnz11VcULVo0h89eCJFbJJbYDoXljklZuu8yJbyc6dugJOX93Ohcoxjzd13kTnScuYsmcpHEk4ypvJjNo379+tqePXty/bzCMh0/fpwqVaR2x5Kk9Z4opfZqmlbfTEXKMYknBYfEEstkq/Hk3RWH+fvwdfa918HMpXrQtfAomn6xnpfalOfVxyoBcPhyOF0mbOXtzpUZ0bKcmUtoHSSeWJ7sxhJpSRFCCCFEgaNQFjnt8vL9V9A06F7XP/mxGv6eNClbiF+2BhGXkJjB0ULYDklShBBCCCEsgKZpLN17mfqlvQko7PrAtgGNS3P9TjSHLoebqXRC5C9JUoQQQghR4FjiFMSHLodzNvguPer5P7StQRl9etj9F2/nd7GEMAtJUoQQQghR4FjiwPmtZ24B8Hi1hwcj+7k74e/tzD5JUkQBIUmKEEIIIQocpSxvTMqBS2GUKeyKj6tDmtvrlvJm34Ww/C2UEGYiSYoQQgghCiRLSlE0TePApTDqlPRKd5+6pby4fieaq2FR+VcwIcxEkhRhEz799FOqVatGzZo1qV27Nrt27cqza7Vu3RqZElcI2yXxpGBQCovKUq6GRxMcEUPtUl7p7lO3tD4uRbp8WQ+JJzknK84Lq7djxw7+/PNP9u3bh6OjI7du3SI2NtbcxRJCWCGJJwWHQllSjpI8IL52Bi0pVYp54GRvYN+FMJ6sWTyfSiZySuLJo5GWFGEWwcGwe7f+76O6du0ahQsXxtHREYDChQtTvHhxPv74Yxo0aED16tUZMWJEct/j1q1bM2bMGOrXr0+VKlXYvXs33bt3p0KFCrz77rsABAUFUblyZfr370+VKlXo2bMn9+7de+jaq1evpkmTJtStW5devXoRGRkJwNixY6latSo1a9bktddee/QnKYRIU27GEpB4UpAohUWNSTlwMQxHOwOVi3qku4+90UDNEl7SkpJHJJ5YFklSRL5bsEAjoHwCnXrdJaB8AgsWPNoficcee4xLly5RsWJFXnjhBTZt2gTAqFGj2L17N0eOHCEqKoo///wz+RgHBwf27NnDyJEj6dq1KxMnTuTIkSPMmjWLkJAQAE6ePMkLL7zA8ePH8fDwYNKkSQ9c99atW4wbN461a9eyb98+6tevz/jx4wkJCWH58uUcPXqUQ4cOJQcWIUTuyu1YAhJPChIL6+3FgUthVC/hiYNdxrdmdUp7cfRqONFxCflUsoJB4onlkSRF5KvgYBg2MhHPHttw67sRzx7bGDYy8ZFqLdzc3Ni7dy9Tp07F19eXPn36MGvWLDZs2ECjRo2oUaMG69ev5+jRo8nHPPXUUwDUqFGDatWqUaxYMRwdHSlbtiyXLl0CoGTJkjRr1gyAAQMGsHXr1geuu3PnTo4dO0azZs2oXbs2s2fP5sKFC3h6euLk5MSzzz7LsmXLcHFxyfmTE0KkKS9iCUg8KUj0lhRzl0IXl5DI4SvhGQ6aT1K3lDdxCRpHr8qijrlF4ollkjEpIl8FBYGzdzQOfhEAOPhF4OwVTVCQK76+OT+v0WikdevWtG7dmho1ajBlyhQOHTrEnj17KFmyJB9++CHR0dHJ+yc1vRoMhuT/J/0eHx8P6NNTppT6d03T6NChAwsWLHioPP/99x/r1q1jyZIlTJgwgfXr1+f8yQkhHpJXsQQknhQUSik0C2lLOXEtgpj4xAwHzSepY9rnwKVw6pX2yduCFRASTyyTtKSIfBUQAFG3nYi96Q5A7E13osKcCAjI+TlPnjzJ6dOnk38/cOAAlSpVAvT+n5GRkSxZsiTb57148SI7duwAYP78+TRv3vyB7Y0bN2bbtm2cOXMGgLt373Lq1CkiIyMJDw+nc+fOfPfddxw8eDCnT00IkY68iCUg8aQgsaTFHA9cynzQfBI/dyd8XB04czMyj0tVcEg8sUzSkiLyla8vTJ9sYNjIZjh7RRMV5sT0yYZHqqmIjIzkpZdeIiwsDDs7O8qXL8/UqVPx8vKievXqFC1alAYNGmT7vJUqVWLixIkMHTqUqlWr8vzzz6d6Lr7MmjWLwMBAYmJiABg3bhzu7u507dqV6OhoNE1j/PjxOX9yQog05UUsAYknBYqynDEpBy6FU9jNkRJezlnav5yvK2clSck1Ek8sk8qLmS3q16+v2dI8zSJjx48fp0qVKtk6JjhYb14NCOCRg0BeCAoK4sknn+TIkSPmLkqOpPWeKKX2appW30xFyjGJJwWHLcYSkHhiSVLGky9WneCXrec59WknM5cKOv+wBV93R2YPbZil/ccuPcSaYzfY+16HPC6Z9ZJ4YnmyG0ukJUWYha+v5QYAIYT1kFgickopLGJMSnxCImeCI2lRoXCWjynv58bC3Ze4fTcWb1eHPCxdwSLxxLLImBQh0hAQEGCVtRRCCMsj8cQyWcqYlKCQe8TGJ1KxiHuWjynn6wbA2WDp8lXQFKR4IkmKEEIIIQocZSFjUk7d0GeUqlRUkhQhUpIkReQKS1q1t6CT90JYM/n8WhZbfj8UyiKe34nrERiU3oUrq0p4O+NoZ5AZvjJhCe+v0OXkvZAkRTwyJycnQkJCJBhYAE3TCAkJwcnJydxFESLbJJZYFluPJxbTknI9goDCrjjZG7N8jNGgKFPYlbPBd/OwZNZN4onlyGkskYHz4pH5+/tz+fJlgh91aVaRK5ycnPD39zd3MYTINokllseW44mljEk5eSOCytno6pWknJ8bR67IqvPpkXhiWXISSyRJEY/M3t6eMmXKmLsYQggrJ7FE5KtUq3SbQ3RcAkEhd+lau3i2jy3v68aqw9eIjkvIVitMQSHxxPpZfHev4GDYvVv/VwghHoXEEyFEkqQUJafdgXIjnpy+EYmmQaVszOyVpJyfG4kaBIVIly9hmyw6SVmwQCOgfAKdet0loHwCCxZYQLusEMIqSTwRQqSU1JCSkxwlt+LJSdPMXhVz0t3L1xWAszclSRG2yWK7ewUHw7CRiXj22IaDXwQON90ZNrIZ7dsbZaEdIUS2SDwRQqSmTG0p2U0vcjOenLx+Bwc7AwGFXLNZCihb2A2lkBm+hM2y2JaUoCBw9o7GwU+vZXDwi8DZK5qgILMWSwhhhSSeCCFSu9+Skr00JSgInH3u5Uo8OXkjkgp+bhgN2R8f4+xgpISXs6yVImxWviQpOem3GRAAUbediL2pN4HG3nQnKsyJgIA8KaIQwkpIPBFC5IZ79/R/szumZOutc7j23EJirD5YPfamW47jycnrd7K1iGNqZX3dOHdLkhRhm/I8Sclpv01fX5g+2UD40mZELmhN+NJmTJ9skK4ZQhRgEk+EELlhwQKNb8fr8aN8leyNKTkdehtnByOJd1wAiNpfPkfxJOxeLDfuxFAxB4Pmk5TyceZSaFSOjxfCkuVpkpKy36Zb34149tjGsJGJWa61CAxUBJ0xsmqJK0FnjAQGmn+6QCGEeUg8EULkhqRY4ljtAgCe3bZnK5ZcCo2iQVkvdo9rjqPRyIixYTmKJ6du6C0gj9KSUsrHhfCoOMKj4nJ8DiEsVZ4mKbnRD9zXFxo0QGo8hSjgJJ4IIXJDUiyxc4sFwMEvMlux5NLte5T0caFYUQN1Snty7ObtHJUjaWavnEw/nKSUj96acyn0Xo7PIYSlytMkRfqBCyFyi8QTIURuSIol8ZGOAMQGZ31MSUR0HGH34pKTg/qlfTh27Q53Y+KzXY5T1yNwd7SjmKdTto9N4u8tSYqwXXmapEg/cCFEboiIjiPBIZrvforjzp8NiFzQSuKJECJHku5Nog8GAHDnj0ZZjiVJ4z9KmpKDegHeJCRqHLwUlu1ynLwRQcWi7iiV866npQqZkpTbkqQI25Pn66QEBiratzcSFORKQIB0sxBCZM+RK+HU+HB18u+Fh4K9wUBlDyfWxrhxc60X9Up706CMN452RjOWVAhhLQIDFbd8jHy7AQ7sNVC+VNYShaRkoKSPMwB1S3mjFOy5cJum5Qtn+fqapnHqRgSdqhfLfuFT8HCyx9PZnovSkiJsUL4s5ujrK8mJECJnCrk58nbnyrg52pOoacTEJ3LzTjRXwqI4cT2C9Sdvomng6mCkRQVfnq5TgraV/XCws9hloIQQFsDTQ09MvHyyPrNXUreqpJYUT2d7Kvq5s+dC9salBEfEEHYvjkpF3LJ1XFpK+bhwUWb4EjYob5KU4JPwS0ewdwZXP3AvCj5lwLcy+FUBJ888uawQwvYUi7vIiLOjwcFVjyXuxaBkBahbCQo3ISJe8d/5UNafuMmaYzf45+h1Crk60LdhSYY0LYOvu6O5n4IQwlLcOgWzngRnLxredeM5I9idjofSNcE74P4Kj+m4FHoPN0c7vFzskx+rF+DNygNXSUjUsrwoY9Kg+YqPMLNXklI+Lhy/dueRzyOEpcmbJMVoBwY7iAqDW6ch8gYkxJo2KvCrCqUaQ/n2ULaVfvMhhBBpUUZAg4ircHU/3A3Wfwewc8K9eF3aBTSjXf3H+OjJVmw5e5sF/11k0sazTNtynj71S/JS2/L4eeR8cKoQwkYoAyTEQfApyoZd5S37CPh9gb7NpRCUbARlWkLFx8Gn7EOHX7odhb+38wPjSOqV8mb+roucuhFBlWIeWSrGyeuPPrNXEn8fZ9Ycu5GtJEkIa5A3SYpPORjy5/3fExMh/BIEn4CrB+DSLji0GPbMAKMjVOgANXtDxY5gJ7WeQogUCpWDZ/6+/3t8DISchZvH4Mo+uLQTtoyHzV9j51KYNtWepk2bPpzv1IqpW86x4L+LLNl7meEtyjCydTlcHPKll6sQwhIVKg/P/gvAit2X+GTpTtYNKorfvTNweTdc3AEn/4Z/xuq9P2r0hBq9wbs0oLeklCn8YMVq/QBvQB+XktUk5dSNCAq7OVDI7dHveUr5uBCbkMiNO9EU93J+5PMJYSny56+1waB/wb1L67UTAPGxpmCwCo4ugxN/grMP1B0EDZ4Fr1L5UjQhhJWxc4QiVfWfGj31x6Juw5l1cOIv2D8Xdk+njG9lPm84nJGNn+LrjZf5cf0Zlu67wkdPVaN91SLmfQ5CCLMzGBQRuBBdpB4UagH1n9E3hJ6H06vh6ApYP07/KdcOreEILt9OoGXFBwfZlvJxobCbI/sv3GZg49JZuvbJG5GPtNJ8SiVTTEMsSYqwJeYbWWrnoHf16vQFjDkGA5ZCQDPY/iP8UAuWjdDHtgghRGacvfWEpddMeO00PDUB7Jzgr/9Rek59JhT7l6VDquLqaGTYnD28OG8foXdjMz+vEMJmGU13QAlaqoHzPmWg0XMwdBW8fAhavwU3jqIW9GGleo1WUesg4f66KEopavl7cvhKeJaum5iocfpGRK4lKUlrtsgMX8LWWMb0N0Y7fXxKn7l6QGj8AhxfCRMb6clK2EVzl1AIYS2cPKDuQBixEZ5dA6Wbw6YvqLesBavq7OKt9qVYfew6j3+/mQ0nbpq7tEIIMzGYxpUkJGYwu5d3aWg9Fl45zLlWPxCHkZZH34WJDfWWFlOCU8PfkzPBkVla1PFKWBT3YhOolAuD5gGKezljUPp4GSFsiWUkKSl5lYTHP4VXDkOz0XDsd/ipPqz5AGIizV06IYS1UApKNoTA+TByG5RthXHjpzx3sBcbO1ynkIs9z8zazWd/HycuIdHcpRVC5LOkQeaJqVtS0mLnwGHvDnSO/ZwrHaeD0QF+GwwzOsCVfdQo4YmmwdGrmc+ylTQTV261pDjYGSjm6SyrzgubY3lJChAcDLuPFSa49sfw0l6o3gO2fa/XXBz7I7nmQgghMhIcDLt3Q7CxOvSdB0P/BY8SlNg4hr89v+DVWvFM3XyOPlN2cD082tzFFULkIztTkhKfkLV7ihOX7qFhILFUd3h+m96tNOwiTGtLkxOf4UFklrp8HbocjtGgqJrFQfZZUdLHWbp7CZtjcUnKggUaAeUT6NTrLgHlE1jwdwno9jMMXa33O188EBYPgshgcxdVCGHBHoolCzR96vNn10CXHzEEH2f06Wf5p85Ozl6/zVMTtrLvYvYWZBNCWK+k7l5ZaUlZsEBj/PR7JEbZU7kKLFhk0LuVjtoNjUbicmgOa53GknBiVabnOnApjMpF3XF2MD7yc0hS0ttFWlKEzbGoJCU4GIaNTMSzxzbc+m7Es8c2ho1MJDgYKNUIRmyCdh/AqX9gUiN93IoQQqSSYSwxGKDeYBi1B6p0ofLxH9lV5EsqGK/Sd8pOlu+/bO7iCyHyQVJ3rwzHpHA/njiWu4bBOe7BeOLkqU8ANHw9sfaejLj8Fix/HmIi0jxXYqLGwcth1CrplavPpZSPCzcjYoiKTcjV8wphThaVpAQFgbN3NA5++pfbwS8CZ69ogoJMOxjtoMWr8NwW8CwJiwbAn2MgTgaLCSHuyzSWALgW1mcD6z0Hp7tXmJvwJv8rvIsxiw7w07rTaNKtVAibZkhKUjL5rifFE4ODngCkGU+K1+H3hvOYEP802qGFMKWlvo5TKudu3SUiOp7auZyk+PvoUw9fCZP7IWE7LCpJCQiAqNtOxN7UB5PF3nQnKsyJgIBUO/pV1rtsNB0Ne36BaW31xd2EEIJsxBKAql1h5DaUf32eC/+OpX4zmbTmEG8tO0y8DKgXwmYZk7p7ZdKSkhRPtER9//TiSdVSvnwT35tjj83X14Kb0QF2TXlgHO3BS2EAuZ+kmNZKkSRF2BKLSlJ8fWH6ZAPhS5sRuaA14UubMX2yAV/fNHa2c4DHPtHXV4m4DlNbw/E/09hRCFHQZCuWAHgUg4EroM271L2zjs3en7Jzz3+Mmr+fmHjpPiGELcpqdy9fX5j2swINYo+VSjee1CjhCcD2uErw/FYo3wFWvQHLhkPsXUAfj+LmaEc5X7dcfS4lTIs4Xr4t41KE7bCoJAUgMFARdMbIqiWuBJ0xEhioMj6gfHt4bhMUKgeL+sOmrwm+qekz+sjYeiEKrGzHEoMRWr2OGrAUX0L51/VDIo+vYeDUPWzZES/xRAgbk7xOSha6dnbvpaGMGkN6OacbTwq7OVLc00mf4cvZG/rOh7bvwuEl8MvjEH6ZPefDKO3hSWhIJvEom4p4OGFnUFyRtVKEDbG4JAX0WosGDUi/1jM1r1LwzD9Qsy9sGMeG0c/Sre+t+zP6CCEKpGzHEoDy7WDEJhx9SjHb4SsqXZ5Pv2k7KVMlRuKJEDYkeZ2ULPTqjDQt0li5vF2G8aSGvycHL4fpvxgM0PJ16P8bhAYR9WMbHK7vY/8m11y/PzEaFMW8nKS7l7ApFpmk5Ii9E8HNJ/P+1vfpWWkZy/t3IqDXqvszcAghRFZ5l+ZW13/552wHPnGYxXvFJ1Fo0FqGvxQr8UQIG2HM4sB5IHkleVcHuwz3a1y2EBdC7hF06+79Byt0ILT7Gm6EO7LI6WOerL70wRnCckkJL2dpSRE2xXaSFCDogmLSmZG8GDeamuo8fxR/jYr+px+cgUMIIbLg/FV3ntk6jZnxjzPC7i8muP5Ika7bOH463txFE0LkgvtjUjJvSomI1r/3bk4ZJykdqhYBYM2xGw88fjaiCk8em84JrRQ/23/P0GLLHp4h7BGV8HLhsiQpwobYVJKSNAPH79fbMzB2LIW4w9+dulLO/bi5iyaEsDIBAXA31JV3rr7IZ3GBdDHuZHbx95i8d7OsRSCEDUia3Ssrk/gldfdyd8w4SfH3dqFqMQ9WH7v+wOMBARDiYaBv1LtsSKzNOPuZvFL5BwJK516XrxLeztyIiCY2XmYlFLbBqpOU4GAeGCCfckaf9fOfp8Pcf/H0BJ/fO8GVveYtrBDCoqUfT5ozfu63DP9nEk2Mx3kt+HXGzFovs34JYeUMpjugzGb3AojMYksK6K0pey/cZu3WmOR4kuAQjXPALUIPVqb/r7/z69FA3m3yJb773s7aoJgs8Pd2RtPgenh0rpxPCHOz2iRlwQKNgPIJdOp194EBaCln9Pl7Vw1cXlwFjh4wuytc2GHmUgshLFFW4slnK/tj7DOHWnYXePnyq7wzd6OsoyKEFUseOJ+VMSmxpiQlk5YUgMSLfiRqMOC9S8nxZNHuSySi8c+E0vz5mycdJ06CRs/Dzknw58u5kqj4J01DHCbTEAvbYJVJSnAwDBuZiGePbbj13fjQALQHZvTxKQtD/wH3IjC3BwRtM2vZhRCWJVvxpMqTGPsvpoLdDYafG81nv22SlemFsFL3u3tl/h1OHpOSSZISHAzvv+qGlqBwanRSjyfPJzB/10Waly9M3YquejzxM0DHz/XZv/bNgT9GQeKjtc6W8E5aK0XGpQjbYJVJSlAQOHtH4+AXAYCDX0TGA9A8isOQv8CzBMzrCUFb86uoQggLl+14Uq4tdgN+o6xdMIHHXmTiyu35VVQhRC4yZKMlJWlMSmbdvfR4EoMy6ud08IvArcpVbkRE069RqQd3VkpfR6X1W3BgHvz+4iO1qBTzdEYpZIYvYTOsMklJGiAfe9MdgNib7kSFOREQkMFB7kVNiUpJmNcbLu7Kj6IKISxcjuJJ2VbYDVxCaeMt2u15joUbD+RDSYUQuSk7LSmR0fEYFDjbGzPcLymexIW6AKBp4FDvFD4uDrSvUiTtg1qPhTbvwMEFj9T1y8HOQBF3WStF2A6rTFJSDpCPXNCa8KXNmD7ZkPmCbW5+MPgPvevXvJ4ymF4IkeN4osq0xNh/EeWN16m+fjBr9p7MnwILIXLF/SmIs9aS4uZoh1IZrxSfFE/CFrXk3t/1iTpcCg9PjWEty+Bgl8EtV6s37nf9WvWGnt3kQAlvWStF2I7MR4BZqMBARfv2RoKCXAkIyMaK0u5FYfBKmNlZH6PyzCrwq5KXRRVCWLicxhNj+TYk9J5LpUX9if1jAPs9V1CnfIk8LasQIndkN0lxd7LP0nnvx5MiBAQUwde3RtYK1OYdiI+G7T+Bozu0/yBrx6VQwsuZ/ZduZ/s4ISyRVbakJHlgQGt2ePrDoN/B6Ai/doPbQXlRPCGEFclpPHGo0pHoLlOopc5wb24/LtyUGwQhrEF2VpyPjI7H1THjrl4p5SieKAUdPoF6z8DW8bDth2wcrPP3duZaWHSWEi8hLJ1VJymPxKcMDFwOcVEw52mIvGnuEgkhrJR7vZ6EtvmKZhzg7NQBhN+NMXeRhBCZMJi6biVmo7tXnlMKnvgWqnWHNe/Dvl+zdXgJb2fiEzVuRshaKcL6FdwkBaBIVei/BCJvwLxeEBNh7hIJIayUb6vhXKw3lrbxW9n+8wjiZLFHISxadrp7RcTE45bF7l6PzGCEblOgXDtY+TKc/CfLh5bwkmmIhe0o2EkKQMkG0GsWXD8MiwdBfKy5SySEsFKlnhzLqTID6RS5grUz3pE1VISwYMmze2Xha3o3Jh73/GhJSWLnAL3nQLGa8NsQuLQ7S4f5m9ZKkcHzwhZIkgJQ8XHo8gOcXQ9/vpLjWTWEEAWcUlQc+CPHfNrT6drPbFk+2dwlEkKkw2C6A8pSd6/ofOrulZKjG/T7TZ/wZ0EfCDmb6SElvPSpj2UaYmELJElJUnfg/QWVNn9t7tIIIayVwUDlkfM46VSTRgff5fD2VeYukRAiDdkaOB8Tj2t+JykAbr4wYKleeTqvF9wLzXB3ZwcjhVwduHz7Xj4VUIi8kydJSmRMvHV2c2j1JtQKhA2fwsFF5i6NEMJKGRycKP7cUm4ai1By9TCunDlk7iIJIVIxZHExx8RETR84n8lq83mmUDkIXADhl2FBIMRlPCje39tZxqQIm5AnScr5W3fp/vN21hy7kaVmVIuhFHT5EQJawB+j4OJOc5dICGGl3L39MAxYSiIGtPl9uBsWbO4iCSFSsDNkbXavu7HxAPk7JiW1Uo2h22S4tBNWjs6wW3oJb2fp7iVsQp4kKcW9nAmOiGH4nD10+mELvx+4QnxCYl5cKvclDVbzLAkL+8saKkKIHCtRtgqXOkzDN+Eml6f0RJOJOYSwGEndveIzS1Ji9Jn6zNaSkqR6d2jzLhxaBFu+TXc3f28XrtyOss4eLUKkkCdJSiFXBza+1prv+tRCQ+PlhQdo++0m5u26QHScFUzL6eID/RZDYhzM7wvRd8xdIiGElarVrCPbq75PpagDHJvxnEzMIYSFUEqhFCRm8p2MjIkDyP+B82lp+RrU6A3rP4Fjv6e5SwkvZ2LiE7kVKZUiwrrl2cB5O6OBbnX8+efllkwdWA9vVwfeWX6Ell9tYNrmc9yNic+rS+eOwuX1FpVbp2DZCEi0kpYgIYTFad17NKt9+lHt2jLO/P29uYsjhDAxKpXpmJSIaP1+xSKSFKXgqZ/AvwEsHwnXjzy0S9I0xDJ4Xli7PJ/dy2BQPFatKCteaMrcZxtR3s+NT/8+TrMv1/P92lOE3bPgTL9sa+j4OZxapQ+mF0KIHFBK0fy5H9hp14DSuz/h5qG15i6SEAL9HiWz2b0iTZWqZu/ulcTeCfrMBSdPWBgId0Me2Fwiaa0UGZcirFy+TUGslKJ5hcLMH96YZS80pUGAD9+vPU3TL9bz6V/HuHEn49kqzKbhCKgzELZ8A0eXm7s0Qggr5eLoQNGhc7lIURyXP0NM8HlzF0mIAs+oVKYD5yMtqSUliXtR6DsPIm7Ab4MhIS55k6w6L2yFWdZJqVvKm2mD6vPvKy3pULUIM7aep8WXG3h7+WEuhlhY86RS8MS3ULIRrHgBbhw1d4mEEFYqoHhRrneaBYnxhMzoBbEWFu+EKGCMBkVm8/okt6RYUpICUKIePPUjBG2B1e8lP+zuZI+ns72sOi+snlkXc6xU1J0f+tZh42tt6FnfnyV7LtP6mw28vHA/J69HmLNoD7Jz1MenOHroM35F3TZ3iYQQVqpZo4asqTyOolFnuDB7mAykF8KMDFkaOG+agthSunulVKsvNHoedv0MBxcmP1zCy1nGpAirZxErzpcq5MJn3Wqw5c02PNu8DGuO3eDx7zczfM4e9l+0kITAvaieqIRfhqXDZSC9ECLHnu79DL95DqL0lb+4tnq8uYsjRIGlt6RkrbuXWVacz4rHPoHSzWHly3DtIKAPnpcxKcLaWUSSkqSIhxPvPFGVbW+25ZX2FdgdFEq3SdvpN20n287cMv+c36UaQacv4cwa2PyVecsihLBadkYD7YZ/xUbVEN8dn3L39BZzF0mIAsloMGS6TkpkTDyOdgbsjRZ1y3Sf0R56zQJnH1g0EKJuU8K06rzZ75uEeAQW+Y3zdnXglfYV2fZmW959ogpnbkbSf/ounp60nX+PXjfvKvb1h0KtfrDxCzi12nzlEEJYtcLuTngETuOS5kv8wkFod66Zu0hCFDhGQ+YrzkfGxFtmV6+U3Hz13h53rsKyEfh7OXEvNoGwe3GZHyuEhbLIJCWJq6Mdw1qUZcubbfisWw1u343luV/38vj3m1m+/7J5VrFPGkhfpDosGyYr0gshcqxuxQB2N/wR+/i73PilHyRY+PpRQtgYo8raFMQWN2g+LSUb6MsmnF5Ny2szAZnhS1g3i05SkjjaGenXqBTr/9eKH/rWRikYs+ggbb7daJ5V7B1coM8c0IDFgyE+Jn+vL4SwGb06P8Zcv/9RNGwfN1a8Y+7iCFGgGAxZm4LYYtZIyUyDYVCzD+WPTaCZ4TBXwmTwvLBeVpGkJLEzGuhau0TyKvaFXB15Z/kRWny1gambzybPwJEvfMpCt5/h2gH45638u64QwqYopej9zBiWGTtS5PBk7h783dxFEqLAMGZhMceImHhcHawkSVEKnvyOxMKV+NF+AqHXgsxdIiFyzKqSlCRJq9gvf6Ep84c1olIRdz77+wTNvljP+DWnuH03n1axr/wENB0Ne2bAod/y55pCCJvj5eJA2QE/cCixLOr359FCZaFHIfKDUWVtdi+LH5OSkoMrhj5zcCaW5gfeeGChRyGsiVUmKUmUUjQtX5i5wxqx4sVmNC7rw4/rTtPsy/WM+/MY18PzYRX7dh9AqSbw5ytw63TeX08IYZNqlynKseY/EZegcWtWP+lGKkQ+MBhUpuuk3I21kjEpKSjfSnzvMppSdw/B+k/MXRwhcsSqk5SUapf0YsrA+qwe05LHqxVl5vYgWn61gbeWHeZCyN28u7DRDnrM0Bd8XDxYVpAWQuRYnw7N+LXIm/jeOUbwsjfMXRwhbF5WW1KsZkxKCueKPs6fDp1g2w9w8h9zF0eIbLOZJCVJxSLufNenNhtfa02v+v4s3XeZNt9sZPSC/Zy4fidvLupZArpNhZtHYZXcWAghckYpxYAhL7DQ2AXfY7O4e2CZuYskhE0zGBSZTRQaEROPm6N9/hQoF5X0ceHd6H5oRWvCipEQdsncRRIiW2wuSUlS0seFT7vVYOsbbRjeoizrjt+g4/dbGDZ7N/vyYhX7Cu2h+auw/1cZnyKEyDEvFwcq9v+GA4nlUH+MkvEpQuQhO4MiITH9LCU2PpHY+ETcHI35WKrcUcrHhbBYI7efmKpPb770WRmfIqyKzSYpSfw8nHircxW2j23Hqx0qsufCbbpP2k7fqTvYcjo4d1djbfMOlGxsGp9yJvfOK4QoUOqWLcrhJt8Rn6ARMnsAxOfTZCBCFDAGgyIhg9uAu6ZZQ61tTAroSQpAkFYUunwPl3bBhk/NWyghssHmk5Qkni72jG5XIXkV+/O37jJwxn90nbiNf47k0ir2RjvoOQOM9rBkCMTlw8B9IYRN6v94S2YWfo3C4UcI+V2mORciLxhVxivOJy1t4OZkfd29ShfSk5RLofegRk+oNwS2fgdn1pq3YEJkUYFJUpIkrWK/+Y02fN69BmH34hg5V1/Fftm+y8Q96ir2nv7w9GS4fhjWvJ87hRZCFDgGg6L/M6NYbOhIocPTiT72t7mLJITNMRoyHjgfEZ3UkmJ93b38vfUk5WKIaUKfjl+AX1VY9hxEXDdjyYTImgKXpCRxtDMS2PD+KvYGpXh18UHafLORX3c+4ir2lTpC4xfgvylw4q/cK7QQokAp7OaIf5/xHEssTfzSkXDnqrmLJIRNMaiMF3OMMt0LONlbX5LiZG+kiIcjF0NNSYq9M/ScCbF3YdkIyGAsjhCWoMAmKUmSVrFf9XILpg+qj6+7I++tOELzLzcwedNZIqJzOMis/YdQrBb8/iKEX87VMgshCo6mlUqws+7XGOKjCZ4zGBIfoQJFCPEAo0Fl2N0rxoqTFNDHpVwITbE0gl9l6PQlnN8EW8ebr2BCZEGBT1KSGAyK9lWLsOz5piwY3pgqxdz5YpVpFfvVJwnN7ir2do56jUVCHCwdLjcWQogcG9SlA794vojvrf8IW/2FuYsjhM0wGjJuSYmO1/92O1tpklLSx0Ufk5JS3UFQrTts+Awu/WeeggmRBZKkpKKUokm5Qvz6bCN+f7EZTcsV5sf1Z2j2xXo+XnmMa+FRWT9ZoXLwxLdwcTts/ibvCi2EsGl2RgPdnnmdv2iO+85viDu/3dxFEsImGFTGLSnRcXqXKGtuSbl+J/rBLuxKwZPf6Wu8LXkWosLMVj4hMiJJSgZqlfRi8sB6rBnTkk7VizJ7h76K/dilhwi6lcVV7Gv1hZp9YNMXcEFuLIQQOVPC2wWHrj9wJbEQ9xY8A1F5sN6TEAWMnUERn2GSktTdyzpvl0r5uKBpcCUsVQWrsxf0mAF3rujLJuTmcgxC5BLr/NblswpF3BlvWsW+T4OSLNt/hbbfbuSlBfs5fi0Lq9g/8S14lda7fcmNhRAihzrUKc/flT7DJSaYm/OflxsLIR6RIZPZvWyhJQW4P3g+pZINoc3bcHQ57J+bzyUTInOSpGRDSR8Xxj1dg61vtmF4y7JsOHGTTj9sYeis3ey9EJr+gY7u+vopkddh5ctyYyGEyLEhvXvwq3N//C6t4s72X8xdHCGsmlEpEjMak5LUkmJnpUlKyrVS0tJ8DAS0gFVvwK3T+VgyITInSUoO+Lk78VanKmx7sy3/61CR/Rdv0+PnHfSZsoPNp9JZxb5EPWj7Hhz7HfbNyf9CCyFsgpO9kZbPjGOHVg2HtW+RePOkuYskhNXKbJ2UpIHzjlba3cvXzREne8P9tVJSMxih+1R9sp8lQyE+Jn8LKEQGrPNbZyE8Xex5qV0Fto1ty3tPVuVCyD0G/fIfT03Yxj9Hrj08GK/paCjbGla9CcGnzFJmIYT1K1/EkxvtfuReoj0hcwbKjYUQOWQwKDLIUYiOTUApcLSzztslpRSlfFzS7u6VxKM4dJ0E1w/Buo/zr3BCZMI6v3UWxsXBjmebl2HTG635onsN7kTHMXLuPjp8t4kle1OsYm8wQLcp4OACS6XGQgiRc11b1GNxibH4Rp7k+vJ3zF0cIaySUZFJS0oijnYGlFL5WKrclWmSAlC5MzQYBjsmwJl1+VMwITIhSUoucrQz0rdhKdb/rzU/BdbB3mjgtd8O0vrrjczeHqT3bXUvCl0nwvXDUmMhhMgxpRT9Bo1kmbEjRY9O4+6x1eYukhBWJ/OB8wlWO2g+SUlTkpJmV/SUHhsHvpVh+UiIDM6fwgmRAUlS8oDRoOhSqzirXm7BL0PqU9TTiQ/+OErzL9czaeMZ7pRuDw2Gm2os1pq7uEIIK+XhZE+Z/t9zOrEE8UtHoEXeNHeRhI3TNI2v/z1h7mLkmqwMnLfWQfNJSvu4cC82geCITHpv2Dvr0xJHh8PvL8okP8LsJEnJQ0op2lYuwpKRTVg4ojFVi3vy1T8nafbFer5XA4kvVBmWPy81FkKIHKtTthj7Go7HKT6Sq7OflRsLkac2ngxm4oaz5i5Grsl04HxcotWukZKknJ8bAGeDs7C+W9Hq8NgncPpf+G9aHpdMiIxl+s1TSr2klPLOj8LYKqUUjcsWYs7Qhqwc1Zzm5Qvzw5bLdL85lPh7YUQvfU5uLETeio81dwkAiSd5pVfnx1nkPZwSwZu5se4ncxdH2KjERI1vVp+krLe9uYsC5E48yTxJsf7uXuV8k5KUyKwd0HAEVHgMVr8LN47mYcmEyFhWqgeKALuVUouVUh2VNY8eswA1/D35eYC+in35Go0YF9cPp/Pr+H3qB5zP6ir2QmSHpukTNVgGiSd5wGBQdBr6AVtUPby3fkz05cPmLpKwQf8evc7lq1f5Q71q7qIkeeR4YjQoEjLq7hWfiKOVJylFPZxwcTBmPUlRSp/ty8kTljwLcVGZHyNETpzfkuHmTJMUTdPeBSoAM4AhwGml1GdKqXK5Ub6CqryfO+N71+bZMZ9x0qMpHa9O4oXxcxg1fx9Hr4abu3jCluybA8dXmrsUgMSTvOTr4YR990mEay6E/TpQbixErkpI1Ph29UkmuM3ENfq6uYsD5E48MagstKRY6fTDSQwGRVlf16x190ri5gvdfobg47D6vbwrnCi47oXCshEZ7pKlb56mTwlx3fQTD3gDS5RSXz1qGQu6koVcqTRiDvauXszxnMqOk1d44setPDPzP/YEZbCKvRDpCA6G3bv1fwk+Bf+MhTKtzF2sZBJP8k7jGpXZUOVjisac5/wCi6ntFlYsKZ7M3XKFeqF/0iJ+B6rd++YuVrJHjSdGg3p4TbMUYmyguxfoXb7O3sxiS0qS8u2h8Yuwexqc/CdvCiYKjAfuTTQNVo6GuxmPyc7KmJSXlVJ7ga+AbUANTdOeB+oBPXKh3MLNF0P3yfhGnWN73fW89lhFDl4Op+fkHfSevIONJ29mPnWgEMCCBRoB5RPo1OsuFSvdI3T6s2DnpK/PYwEknuS9br0G8YdzN8qcm0/wnhXmLo6wYsnxpE8EvyxZz4f2c9DKtIYmo8xdNCB34kmm3b1sYOA86EnKlbAoomITsndg+w+gSA34/QWIsIwWNGF9Ut6bBJRP4L/Js/UeHplUeGTlm+cDdNc07XFN037TNC0OQNO0RODJRy+6APQaiyajcNz/C6NKnGHrm234oEtVLt2+x5CZu3nyp638fTiNVeyFMAkOhmEjE/HssQ23vhv5fMjz+MQcIrz1BPAoZu7iJZF4ksfsjQbqDv2O4wTg+NdLxN6+Yu4iCSuUMp749F7HRO9vuRfjTGiryfrCxJbhkeOJQSkSE9PfHh1vOy0pAOduZbM1xc4Res6A2Hv6+ikZvVhCpCH1vUm9Pgupfnkssf6tM63wyMqYlA80TbuQzrbjOSuySFO796FoDVjxAi7RwTzTrAybXm/DVz1qci82gRfm7aP9d5v4bc+l+6vYC2ESFATO3tE4+EXQwnCIkR4rmHF8MKcMT5i7aMkknuQPf19vbj02CfvEaC7/MlhuLES2pYwnr9ktprohiNFbvudcsMVUeORKPDEayKQlJQFnW0hS/FyBLE5DnJpvJej4OZzbADsn5nLJhK1LGUsciOPnol9yL8GZY5WmZFrhYTHVIQK9xqLHL/qA1xV6jYWDnYHeDUqy9tVWTOhXB0c7I68vOUTrrzcya9v57DfdCpsVEABRt51wu5XIt/aTORFbijfWfEZAgLlLJsyhRdNmrCk1hrIRuzn9++fmLo6wMknxpFnsMUbY/cXMu51Zcfgpm4snRtPA+fS6VOvdvaw/SQko5IpSZH9cSpJ6Q6BKF1j7EVw9kJtFEzYuKZbERzrwut0iqhkuMPLfCZSoXDTTYyVJsTS+FaHTF3BuI+y4v96B0aB4smZx/h7dnJlDGlDM04kPVx6j+ZfrmbjhDHei48xXZmERfH1h+s+Kr2Pn4JEQxaCF85kw0RVfX3OXTJjLYwPfYJt9UwIOfkvwyZ3mLo6wIr6+MOn7K3zn9hMnYgJ4bcY0pk822Fw8MZpqctPrSR0dl4CjDYxJcbI3UtLbJevTEKemFHT5Edz8YOmzEJPD84gCx9cXpvwMbewPMNzub+bEPsbTr3fMUiyx/m+eLao7WK+xWPcxXNn3wCalFG0q+7Hk+aYsGtGY6iU8+frfkzT7fD1f/3uCkMgYMxVaWILAslPpVHY1N2qN469dNQgMlGVICjInBzv8B0/jluZF3OKhxEXdMXeRhLXQNBqFDsdT3eN60xmcOOlmk/HEaLoLSmsa4sREjZj4RJzsrL8lBaBcdqchTs3FR5+EJeQs/PNm7hVM2Lwi5Y8x3uNHLhrL8GliP+q0zlqSK0mKJUqusShiqrGISHO3RmULMXtoQ/58qTktKhZm0sazNPtyPR/+cZSrYbJGQoFz/TCseQ8qdqJ0z+E2V+Mpcqa0vz9nW4ynaPxVjs8Yae7iCCtxc+33VI7YybpSo2ndpb7NxhODQU+8EtPo7hUTr4/lsoXuXqAPnj8XHPloE/CUaQEt/gf758KRpblXOGG7EhMpvPYVPFQ0xt4ziMGBfRfDsnSoJCmWysUHuk+D20Hw12sZ7lq9hCeT+tdjzZhWPFmzOHN3XqDlVxt4Y8lBzuW0aVdYl9i7sGQoOPtA14l6oiuESfP2T7OxyCBq3vqLo//OMHdxhKW7dhDvbZ+ygXo06WPbNeZGU6xMqyUlOk4f82kLUxADlPNzIyY+kSuPWonZeiz4N4CVr+j3KEJk4Maa76gTu4fdFV+leMW6FHZzYP/F21k61ja+ebYqoBm0fB0OLYSDizLdvbyfG9/0qsWmN9owoHFp/jh4lXbjN/HivH0cuSKr2Nu0f8bCrdPQfQq4FjJ3aYQFavrs1xwzVqbUjne4ceGEuYsjLFVMJPfmDyZEc+NCs6/wdnM0d4nylNHUkpLWDF/R8UlJiu20pACcyeng+SRGe+gxXf//0mGQIGNiRTquHqDwzs9YnVCPqk+9ilKKOqW82S8tKTai5RtQqgn89SqEnsvSISW8nPnwqWpsfbMtz7cqx+ZTwTz501YG//If/52XVextzpGlsG8ONB8DZVubuzTCQjk5OuLWbxaapgifO4i4WBm/Jh6W+PcbOEUE8bnTqwS2qWPu4uQ5g6klJa0uUNFxSd29bONWqWIRPUk5cT3tLuTZ4h0AXb6Hy7tho8weKNIQEwFLhhKqvJhf9A18TBUedUp5cf7WXULvxmZ6Ctv45tkyo53e7ctg1LvzxKf/pgYHw+7d+r8Ahd0ceaNjZba91ZbXH6/EkSvh9J6yg16Tt7NBVrG3DbeDYOUYvem9zdvmLo2wcKXKVeFko0+pGHeS3TP/l+G+qeOJKAAOL8FwcB4T47vSqUsvHG1kwHhGkltSMuruZSOvg5eLA/7ezrnXs6J6D6gzALaMh3ObMtxV4kkB9PcbaLfPMyrqeRpWK5/8cN1S3gAcuJR5ly9JUqyBV0l4agJc3Q/rP05zlwULNALKJ9Cp110CyiewYMH9gOvhZM+Lbcqz9c22fNilKlduR/GMaRX7vw5dSzM4CyuQEAdLngU0vendaG/uEgkr0KDzUHYX6kLTa7+yd92SNPfJKJ4IGxV6Dm3lyxygEjtKDufxapmvYWALDBl194qzre5eADVKeHI4N7t/d/oKCleAZcMhMu0MROJJAXRwIRycz9FyI9ilVaFd5SLJm2r6e2I0KPZdCMv0NHZ5WESRm6o+BfWfhe0/cSquFd4NOyTPthIcDMNGJuLZY5u+oudNd4aNbEb79sYHZmRxdjAypFkZ+jUqzYoDV5i88Swvzt9H2cKujGxdjqdrl8DBTvJWq7F+HFzZA71m6U3vQmRRzWE/c/GbgwRsfpU18fWpXTcg2/FE2JD4WFgylOgExaiYF5jSpQaqgEy+YUzu7vXwtqTuXrawTkqS6iU8WXXkOuH34vB0yYWKLQdX6PkLTGtHzOLn2VRuDnNOHmXruZsYFLg62HF1dUU8e1zEwS9S4klBcOsM/PkqlGrKTwndKeF1N7mrIYCLgx2Vi7qzLwuD523nm1cALA4fx5HgqvhsGUHjWpeTayOCgsDZOxoHP72fqYNfBI7u0fz9d9pNqw52BnrXL8maV1sxsV9dnB2MvLHkEK2/3sBMWcXeOpxZB9u+11cBrtZNmtJFtjg6u7OtyCxcuYfjxkGUqRSdbjwxusVg7xzL/v1mLLDIW+s+gqv7eTV6GA1r1OPeFc8CE0vsTC0p8WlkKbY2cB70WmyAI1dzsTWlaA12F/oUx4tr2P3362w5eYtmhQN4sU15Snt44t7qGA5++mB9iSc2Li4aljwDdg5Ed53MptO3qe7jx61bD1Z6lCnsyvXw6ExPJ0mKlQgOhmdecOSFuFG4ON7l10F9GPF8HMHBEBAAUbediL3pDkD4zjKE3XDi1fczblo1GhRP1CzGny81Z9YzDfD3duGjFKvYh0fJjB2WKCToOnGLRxDvUwUe/1ya0kW2BQfDsNdr896dYbR0PMh7w19h2MjEh+LJ3ePFuDK1FXdjE+nWUz5btih89z+wYwL/OHVhI02Z+kr5AhVLktdJSaMlJcbGxqQAVC+uJym52eUrOBg6/NiVfxIa8IbTAqpHXWTFJ5UZVKcSMwY2InJHJTQNEiIduDK1pcQTGxb1x3tw/RDhbSbxxRJnouMTWDHT46F44uJg5F4WKsQlSbESSbWbF328eC/uGZo6HeG9Vp8TFAS+vjB9soHwpc24M7c14dsqUXTgNtwDN+LZY1vyzUd6lFK0ruTH4pFNWPxck+RV7Jt/sZ4v/zlBcITMAmQpFs6P58jHw4i7F0mj72YwZaZzctcct75Ze7+FSIonvzk254+EJrzmtJD2rZc9EE/CfmtGyKqaFO23kxIjNuLZUz5btmbFnMvELxnJ4YhKvBzWg+vrKuDx5O4CFUuSV5xPc0yKbc3uBeDtqg+ez80k5cCJGDweP8gbccO5gTdTfL+iqN91goKgiJ/ip5HliNhUFaNbLMWf3SLxxEZtmfo7zoen8vPRERRr9ziT/riOpoFzq8MPxRMXBzui4iRJsRkpazeXJrZkcWRbXqv9HRXs9Bk1AgMVQWeMfPepK97FH+z65ewVTVBQ1q7TsIxP8ir2LSv5MnnTWZp/uZ4Pfj/y6AtAiUcSHAxnZn9Fq1Jb+EAbxLU2Ibz8v0ScvHL+fouC6X488eDtuGe5mFiEybXeIMFwHtDjyYqlRjyLxMhny0YF34jHd8uzODpGM9phBPeiXIg87Vvg3m9DlhZztJ2WFDANnr+cO0lKQqLG1MP7UfbxhIT4MSp2NEW4zfiGbxBQWn9NAwMVCz4KIO5SYezc9UrPgvL5KihCzl6gxrlR7I+pwPhyzXFpux+nSleT15VO/X472RuzNLRAkhQrkbK1JHJBa0bNmkWEYwW81g6DiBvJ+3TuDNFh97t+xd50JyrMiYCA7F2veglPJvary9pXW9G1dnHm7bpIq6828NpvBx99ISiRIyF7NvF2469YmtCc3xJa4eAXgYtXNPdCHv39FgVLynhyfUFnBiyfTSF1h8S/niXsrt5PuE4diL0jny1bFb96HM1K7OSd+Gc4rxUDBQkRBe/9TpqCOLGAzO4F+t/3i6H3CL/36F265/93kd0XQ+gZUI2wxS3ZOn8Y72/+gK4V/sT33JTk/erWVYSvqYmWoL/esTfdCsTnq0CIj8XxjyEoA4zmBeKww6XcLQwOCcSFugAPxxMXByOxCYnEJ6TRzzIFSVKsSFJryaolrhw97onns7P1xXKWPguJejBNncyEL23G9MmGHM+iUc7Xja961mKzaRX7Pw9dpcN3m3hh3l5ZxT4/Rdyg4qFhnAotz5tXRwOK2JvuxEQ48f343Hu/RcGRMp6s2NCYG00/oGniXv6d9jYJiVquxxJhQU6tptjZ75h5KpAVCc0BSLjjjL2dInxJwXq/jRm0pETZYHcvyL3B8+H34hi/+iSNy/rwzYslk+PJq4tGQ8VOsPpduLwX0O9Npv3gxJ21tQCIPlS2QHy+CoQ17+MWvo/n/pnA2Rum9VAMGnE3PQhb2CLNeOJsSvwz6/IlUxBbGV9fUnypq8IT38DvL8KmL5MX8wsMVLRvbyQoyJWAAHIlCBQ3rWI/qm15Zm47z5wdF/j78HVaVvTlxdblaFS20KNfRKQtMQGWPoshLoJz9VZw7eXKOHtFExXmxPTJBgIDFd275e77LQqGB+LJYy8RFLSNHldn8tuyhvTt2TdPYokws/DLsPw5tCLVma/6oIXGcffvhkTddGfWDAPt26sC9X4bCtBijklSDp5vVr5wjs/z4/rThEXF8f6T1VBKpYgnCp6eBFNawW9DYORmcPYmMFDRtl1x+sw4R3zPM/TsXULfV1ivY3/Arp+h0fN0rfQUK0cm4lrhOi7tD9Czbmne+tKOoCC7h+KJs4MpScmky5dtVQ8URHUGQK1+sOkrOLM2+WFfX2jQIPf/yBR2c+T1xyuzbWxb3uhYiaNXwukzdSc9f97OhhOyin2e2PAZBG2BJ76l85BqybVVQWeMBAbqAT6v3m9RgChF6cFTue1YnDaH32T9niOAfLZsSnysftOYEMfKSp9xOuoeHzxdhVXzvZLjSUF7v5PXSUmru1d8Ag52huRExlYkDZ4/eCksx+c4FxzJ7O1B9G1QkqrFPR7ewcUHes2EiGuwfGTy9GlF/BTvPV2Ry2H3WLr3co6vLyxAyFm9krxEPejwcXLrfLshN/BwsufTESXSjSdZbUmRJMUWPPEt+FWFpcP1WrJ84OFkzwut9VXsP3qqGlfDonhm1m46/7iVPw9dlVXsc8upf2HLN1BnINTpD8hNo8g7yskTj0EL8FL3cFn5HKevhZm7SCI3rXkfLu8m7LHveGdTNE3LFWJoa/8CHU+MxqR1Uh7+mxUTl4iTjS5w3KRsIbaduZXpmID0fPXPSZzsjbzaoVL6O/nXh8c/hVP/wPYfkh9uW9mP2iW9+HHdaWLiZV02qxQXBYsHg8GoLyht5wDAgeAb7LxyjcBGJZNbS9LiYtqW2TTEtvntK2gcXKD3HEiI02vJ4mPz7dLODkYGNw1g4+tt+LpnTWLiExg1fz/tx29i8e5LxMbnLAAKIOwiLBsBRWpA56/NXRpRQDj61yTqsa9orI6w65f/5crgWmEBji7Xu2U0foG3TpQlNiGRz7oVnJXl03N/xfm0u3vZ2qD5JG0q+3EnOp4DOWhNOXw5nH+OXmdYizL4ujtmvHPDEVCtG6z7GM5vAfRlD8Z0qMjV8Gj+PHgtB6UXZvfXa3DjMHSfBl6lADhx/Q6vLNxPzRKejGlfMcPDnRykJaVgKVweuk6Ay7th9Tv5fnkHOwO96pdkzZhW/Ny/Lq6ORt5YeohWX29gxtbz3IuNz/cyWYs0V4uPi4ZFA0FLhN6zwd7ZbOUTBY9X02cIrtCbAXFLmPHLJGkZtSJpxpPgk/D7KPBvyL/Fn2fVkeuMbleBgMKuZiunpTBmMibFVpOU5hUKYzQoNpy8me1jx685iZeLPUObl8l8Z6XgqZ/Ap5y+EvmdqwC0rFCYCn5uzNx+XrqJW7A048ne2XBgLrR8HSp0AOD4tTsMm70HV0c7pgysn+n3xsVexqQUPNWehiaj4L+pcHCRWYpgNCg61SjGylHNmT20ISV9XPjkz2M0+2I9P607LavYp5LuavGrXodrB6DbFChUzqxlFAWTb++fCPWowrDgL5iyfLW5iyOyIM14EhMBiwaAvTNhT07nnT9OUbWYByNaljV3cS1C8jop6SzmaGszeyXxcLKnXmlvNpzI3mqKey+EsuFkMM+1LIeHk33WDnJ0hz5zIfZecm8PpRRDmgVw5Mod9l64nf0nIPJcmvHkyl74+zUo1xat1VgOXQ7jxXn76PTDFu5ExTF1UH2Kejplem4ZOF9Qtf8QSjeDlS/D9SNmK4ZSilYVfVn8XBOWjGxCnVLefLvmFM1kFftkwcGkuVp8xKY5sG8OtHgNKnc2dzFFQWXvhM8zi7Czs6PNwddYtuuUuUskMpB2PEkgZvGL+gDXnjP5aNNtwu7F8k2vWtgb5c8/pFgnJY2eydHxttuSAtCmkh/Hrt3henh0lvbXNI1v/j1FYTdHBjctnb2L+VXWe3tc2pXc26NbnRJ4ONkxc3tQNksu8lpa8eTNV4JJWDAYzc2P+f7v8dgP23hqwjY2nrzJS23Ls+WNttQu6ZWl8yePSZHuXgWM0R56zgRnL1jUH+6Fpt1cl4/qB/jwy5AG/DW6Oa1TrGL//u9HuHz7nnkKZQGCgsDZ+8HV4puW34Hrpv9B2TbJU0oLYTbepXHoPYNKhkvY/fUKa/eHmDWWiPSlFU/eaD4ex7O/Q/sPWBtVkeX7r/Bim/Jpz8ZUQCXlamm3pCTY3PTDKbWprM+WsOlU1rp8bTwZzI5zIYxqUw4XhxysYFG9+/3eHgcWcDfcjlYlS/HP4etcC4/K/vlEnkkdT5z9wvi16zNw9wYvxY/h7X+v4eZkx6fdqrP9rXb877FKeLpksWWN+wukRktLSgHkXgR6/wrhV7g24VnKVoh9uDuRGVQr7smEfnVZZ1rFfsF/F2n99Ub+t/ggZ25GmK1c5hIQAFG376/w7Hkrnlmth6G5FYOev+izZghhZnaVHiOm5ds8ZdjG9gUf0XnILbPHEvGw1PGkyZ3TvFvvc6LLdye01kjGLjtM5aLuvNimvHkLamEMGQ6cT8TRRrt7AVQq4k4xT6csdfmKT0jk81XHCSjkQr9G2WxFSan9RxDQgvjfX+HppntZON6X+ASNt2dcyPk5Ra5LHU9ei11Gq+JbeTP2GfYnlGXaoPosf6EZ/RuVxtM568lJkqQkN7Pxyrb77SvoSjYgouU3FLu3ns+fGflAdyJz14KWNa1iv+n1NgxsUpq/Dl+lw3ebGfnrXg5fLjir2Kdc0TtmUVMm8jNFPMIw9punzzEvhIWIrP46K4I68Y7zHDp0nYFnz60WEUvEfSnjSaGVZZjo8R0RTpVx7PUT76w4wp2oOL7rUxsHG51SN6cK6sB50Ltlt67kx9Yzt4iMyfhmcem+y5y6EckbHSs/2mfIaMettrO4Gl6YxX16U6rLarQ4I2vPX+baDZkN1FKkjCcddobxvMdyZsd3IKxiL9a82pIOVYs80vmT1kmR7l4F2AnXIcw6OZBRnkt5yrAdB78InL2iCQoyd8l0xb2c+aBLNba92ZYXW5dn29lbdJmwlYEzdrHzXEiBmPEjMFARdNrAwfc+pLn/Duy6T4CiNR7Yx9zd9YQIuqB4eeuPnNFKMNH+R8r7ncXZ+57FxBKhCwxUXDh2jy3PDMXLE7xGzmP50TBWHbnOq49VpEoxD4knqSS1pKS5Tkp8ok0nKQB9GpQkMiaeWdvOp7vP3Zh4vl19irqlvOhUvegjX/P8jcIM2TidQsZwJjn8gKNjDEbXGJbvzP5MYyLvBAYqLm8/xKQ2r7IrsTLbyv2Pif3rJreCPEoscbI3oJR09yrQAgLglX++Zmd0Nb6yn0LlsOtEhTkREGDukj2okJsjrz1eie1j2/Jmx8ocvxZB36k76Tl5B+tP3LD5ZMU3aDp+F2dB8zFQo+cD29Kd/UuIfBQQALeCCzHk+vsATLf/Bs9aRyhdWj6PFiUxkcKbR+By9xSGPrO5RFE++P0oDQN8GN6irMSTNNgZM1hxPi7BZhdzTFK7pBftq/gxdfO5dGff/OCPowRHxvDOE1VzZV2dgADYdbYhr4a8TCPDCT6In0vCXQe237j0yOcWuSjiBm5/B3Iz0Y0ZxT7kp4ENcTSN0XrUWKKUwtneKIs5FmS+vvDzJCf6zlpG8F0/Zrh+zdwJN/H1tczaeXcne55vXY6tb7bhk67VuB4ezdBZe+j0wxb+OGijq9if3wz/jIUKj0Pb9x7YlN7sX5b0nomCIanp/9Cingz9dwZl1XUm13ybFcdOA5YZTwqkDePg5N/w+GfEl27Jywv3A/Bt71qEhiiJJ2lIWsyxIHb3SjKmQ0XuRMczY8u5h7Yt33+ZJXsv81Kb8tQr7Z0r10uKJ7N/fZvvD41igNtqPi+xk+3nb3I1LEriiSWIiyZxYX8S7obwmvFNxvVvk5yg5Na9ibO9URZzLOgCAxW7DxclvONCinuH0y2qH4vn3bPo2jQneyMDmwSw8fXWfNOrFnEJiYxesJ+2325kwX8XiYnP+ENtNW6d0RdsLFQeekx7aKB8WrP1WFJ3PVGwBAYqgs4Yefunjtxr+TkdjPvQ1n7I2ElXLDqeFBgHF8KWb6HOQGj0HD+uO82+i2F82r0GJX1cJJ6kw2BIvyUlKi7BZtdJSalacU861yjKjK3nuRJ2f5atE9fv8O7yIzQM8GF0uwq5es2keNJs7MfEBHSif8x4WqiDvDfzksQTc9M0+OMlDFd282rsSEb0eRo/j/trn+RWLHF2MGa6TkoO5pAT1sbXF3zb14QS09AWDcB4aiRePYZi73cXh5vuDBvZjPbtjfj6mrukD7I3GuhZz5/udUqw+th1Jm44y1vLDvP92lMMb1GWfo1K5WwaREtwLxQW9NETk8CF4OT50C5Js2s43HTHwS+C2JvuFtldTxQcvr76D9oIEu6d4Lm9v/DGpRJ49e2EvXeURccTm3ZxJ/zxEpRuDk+MZ8e5UCZsOEPPev48Vas4IPEkPem1pGiaZlrM0fZbUgBe7VCRdcdv0mH8Jp5vVY6Qu7HM3XkBD2d7vu9bG7s8WFdHjydGqDUNfunIzzd/4smg4nj2uIaDX6TEE3PZ8g0cXsy3cT0p0qQvbSr5PbA5t2JJVlpSrPQOT+RIlSe5XPlDevABV+MV38b3TpEBu1psEDAYFB2rF+PxakXZcvoWEzecYdxfx5m44QzPNCvD4CYB2Zqf2+ziY2HxIAi7CIP+AJ8yae6W1CQ+bGQznL2iiQpzYvpkg8W+T6IAUQpj568IPX+OcSG/cKmQLzsSq1lFPLE5oedhYX/w9Ic+v3IzKpHRC/cTUNiVj56qlrybxJO0pTe7V0y8PtNUQUlSyvu5s3pMSz77+zjfrjmF0aDo06Akr7SvgJ975iuIPxJHdwhcgDahDbM8x9HN+WNC8JR4Yg5Hl8P6cay1a8lyp36sfrzSQ7vkVixxcch8TIokKQWMU9uXmfXbaV6qMZeLmh/zrnWxmto0pRQtK/rSsqIvey+EMnHDWcavOcWUTWcZ0KQ0zzYvk/fB9FFpGqwcDUFboNsUKN0kw90DAxXt2xsJCnIlIAAJ1MJyGO1J7D6bs191YLL3d/SI/ZBjwZWsJp7YhHuhMK8XaAnQbzEJTt68PH0XEdFx/PpsQ1wdH/wTL/HkYel194qJ05MURxsfOJ9S6UKuTBlYnyNXwnFztCOgsGv+XdyrFPe6LMR3WWem2Y0nMP4dIm4WkniSny7ugmXPccW9Fi8GD2Xq0Jrp9lbJjVjiJGNSRGq+fgqnHt+x7kJrPjP+Qr3dsVZZm1avtL6K/aqXW9C2ShGmbT5H8y838N6KI1wKteBV7Dd9CQcXQOu3oVbfLB3i6wsNGsgNhbA8hf29OFV/CdHRrsy0/5oiDqF891OcfFbzQ3wMLBoAYReg73woXIHv1pxix7kQPu5ancpF015VXuLJg+5393rw8WjT2Ednh4LRkpJS9RKe+ZugmPjWasBU54+pbTzDt8ELubO0iVXen1ilkLOwoC+xbsXoFvoCnWoH0Kpixi/8o8YSlyyMSZEkpQDq28+BWuN+Jc6zEr/3HUJg68PmLlKOVSnmwU+BdVj3v9Z0r1OChbsv0vqbjby66ACnb1jYKvb75sDGz6FWP2j1hrlLI0Su6DqoNA6DFlPMLpK5vh/zT+imdKcyFbkkMRGWPwcXtkHXSVC6Kf8evc6EDWfoXd+f3vVLmruEVsNgugtKSNWSEm2q4XWyK3hJijl1eXYgn8b358mS/3Bz5rsEZq0uTzyKyGCY2wMNeM3hPWIcfHj3yap5fllnB2lJEekoXMIDl+FLMDh7wryecDvI3EV6JGUKu/JFj5psfqMNg5sEsOrIdTp8t5nnft3DwUthj3z+R54S8eQ/sPIVKNcOnvoRcmGueSEshXe1utj1nUNVw0XGhI5jxMzt3M1kBeuC7JHiiabBv2/rfcc7fAw1e3HmZiT/W3yQWv6efNy1eq6X15bZmbKUhFRNKdFxBWtMiqUo6+vG0dIDWGTXBZdDk2HbD+YuksV7pHgSE6nfA0ZcZ13dCfxxyZm3OlWmsJtjrpczNWd7O2lJERnwKA4DlurdBn7trmfTVq6YpzPvd6nKtrFtGd2uAjvOhtB14jYGTN/F9rO3crQw5CMvgHbpP/htiL6SfO85YLSiQf5CZFXFx1BP/UgLwyH6XfuC4bP+S66NFvc9cjzZ9j3s+hkaPQ9NRxMeFceIX/fgaGfg5wH15KY6m5K7e6V6G5JbUgrAFMSWJrBhKcZG9uFGqSdg7QdwYL65i2SxHimexMfC4oFw/TART03n9R321C/tnW8tsc4OBmlJEZnwqwz9FsGdqzC3O0SHm7tEucLH1YFXO1Rk+1vteKtTZU5cj6DftF10/3k7a49lfRX7R1606PoRvZbCoxj0/w0c3XL+pISwdHUGQLv36WrczuOXv+O5OXtsZ12jXPDI8WTvLFj7IVTvoS/YmKgxav4+LoXeY1L/uhT3cs7D0tumpO5eiYnpdPeSpC/fdapejEJuzryvXoSyreH3UXD8T3MXy+I8UjxJTIBlw+HsenjqR947VoKI6Hg+7VYjeTKJvObiYMe92Ixb3CVJEVCqMfT5FW4egwWBEBeV+TFWws3Rjuda3V/FPjgihmFz9FXsfz9whfjUoyVTeaRFi0LP6YmfvSsM+h3c/DI/Rghr1/xVaPoSg42rqXv+Z56fu08SFZNHiidHluldRst3gKcng8HARyuPseX0LT59ugaNyhbKw5LbruQpiFOPSUmeglhuk/Kbg52Bfg1LsvpUGJc6TIfidWDJM3Buk7mLZlFyHE80Df4cA8dWwGOf8o99e1YcuMqLbcpTqah7Hpf6Pid7Y3K3yvTIt0/oKnSA7lPhwnZ9zv34GHOXKFclrWK/4bXWjO9di4REjZcXHqDd+E3M35X+KvZJixbF3tS/uFletCjsIsx+ChLiYNAK8CoF5MLYFiEsnVLQ4ROoM5CX7ZZT8fR0XpBEBXiEeHJylV7rWaqx3mXUzoEZW8/z684LdKlQljYBMlA+pwzpLOaY1JLiKAPnzaJfo9IYlGLOvmC9F0Kh8nol6sWd5i6axchRPNE0+Oct2DcbWvyPkJrDeWf5YaoV96BvjfL5en/ikoWZ8yRJEfdV7wFP/QRn18HiwXp/RRtjbzTQva4//77SkskD6uHpbM/byw/T8qsNTN9y7qHBvkmLFoUvbUbkgtaEL22W+ZSId67C7C4QcwcGLgdffTGkR+6LLoS1UAq6/ADVezDWfiGlTs/muV/3FvgxKjmKJ2fW6ou/Fq2pd811cOGvQ9cY9+cxYs4VYcG4khJPHkFSS4p097IsRT2d6FitKIt2XyLKzhMGrtC7Tc/tCZf3mrt4FiHb8UTT9O6ipjFtWpt3eXv5YSKi42nvVovylbR8vT9xzsJ3S5IU8aC6A+GJb+HUKlg61CYTFUhaxb4ov7/YjLnPNqJMYVfG/XWc5l+u58d1pwm/d38K1cBARdAZI6uWuBJ0xkhgYAb9NZMSlLshMGAZFK8N5EJfdCGsjcGoL1hapQsf2P9KqTPzeGbm7gI/61e24snZ9XrLtm8lGLgMnDzZdS6EVxYdIPaaF8rtLm59Nkk8eQT3B86nvZijdPcyn0FNSnMnOp5l+y+DexEY9Ae4+MDcbnBln7mLZxGyHE80DTZ8qk+8UX8odPycyZvP8+/RGzzfvBLvjnHN9/uTrKxBJN8+8bAGw6Djl3B8pT4rlY0mKqCvYt+8QmEWjmjCsheaUq+0N+PXnKLpF+v4/O/j3LwTDWRx0aLwKzDrCYi4DgOWgH/95E2P1BddCGtltIcev0ClznxsP4sqF+cycMauByoBCqIsxZPTa2F+X/App9ciO3tz+HI4z87eg5+LMzE7quHgFwlIPHkUhvRaUuKlJcXcGpbxoaa/J1M2ndPHj3qWgMErwckT5jwtLSommcYTTYN1H8Pmr6HOQOj8LRtPBfPVvyfoUqs4zQqVMcv9iXT3EjnXeCR0/gZO/qWvahwXbe4S5bm6pbyZPrgB/7zSgnZVijBtyzmaf7WBd5YfznwV+7CLMKsziRHBHKu/nGDnxg9sznFfdCGsnZ0D9JoNVbrwvt2vNLw2j95TdiRXAIg0nPwHbUEgd10rcuvJleBamDM3Ixg88z88ne2ZEtiIqBvuEk9yidGgiE+VpCSt3yBJivkopRjVpjwXQ+/xx8Gr+oPepWHI3+DiDb8+DRd3mbWMFk/TYM17sHU8N0s9Q3CTHzl+I5LRC/ZTqYg7X/aoQZkyyiz3J9LdSzyahsPhye/g9Gp9Gt2Y7K3gbq2DxCsX9eDHwDpseK01Per689uey7T+ZiNjFh3gVFqr2Aefgl86Eht+mzZzltJyRLWH+nTmqC+6ELbCzgF6zoRq3RhrnEfXsFn0+Hkb54Ijs3S4tcaSHDm8hMQF/dl3tRq1Ji+kdA0vfpwVQf/puzAaFPOGNaJ6WWeJJ7nIaFBprDhv6u5lJ7dJ5tShahEqF3VnwoYz9yc38CqpJyquvnqicnZ9ts5ZYOJJYgL8+Qps/4nJB4ZR9eNxlG8STvcJO3F1tGPaoPq4ONiZ7f5EunuJR1d/qN6v/MJ2fbaqe6FZOswWBomXLuTK591rsOXNNgxtFsC/R6/z2HebGT5nDweSVrG/egBmdiQxLpaWs1dyullMun06s9UXXQhbY7SHHjOgzkBeUEt5MXoaPSdtZe+FjGOKLcSSLNvzC9rSYWy73JDAuLHEdz+IZ+8tfL1nF7FxGnOfbURAYVdA4kluMiqVZncvO4PCzii3SeaklOKlthU4F3yXVUeu3d/gWQKG/gM+ZWF+Hzj2e5bOV2DiSXysPiPg3ll8tetVPvZ5Cre+m/DqsZ3IMCOTejahpI9L8u7miCfSkiJyR60+0Gcu3DgKMzrA7aAMd7e1QeJFPJx454mqbHuzLS+3q8B/50N5euI2vpgwgYRfOqPZO3Ok0b+cSSifaZ/OLPVFF8JWGYz6DIJNRtFXW8U3hh8ZMm0Lfx26lubuthZL0qVpsOEz+HMM4b4dCNzwK7G+encjh0J3UXaJvNus8UNrGEg8yR1GgyL1klnRcQlZuokSea9j9aKU93PjuzWniI1P8Ua5+cGQP6FYbX1G0v+mZXieAhNPosP13i9HlnKp8kd8dfx/yePXlAGiN9QiPszlocPyO564ONhluo8kKSJrKnfWFyS8ewumd4Cr+9Pd1VYHiXu7OjCmQ0W2jW3LzNqneO3W+5yMLcxQ4+eccnQj6raj9BEXIjNKwWPj4LFxtE3YxkLnr3hr/mZ+WHsaLVWXG1uNJQ9IiNNX1N70JdQeQFyP+dy+5U1cqH4ToSUown5vSMta+bfIWkFjUJCYRncvR0lSLILRoHirU2XOBt9lxtbzD2509tbvTSp1gr9fgzXvQ2LaCwQWiHhy5yrM7AwXtsHTk7nTcCT2DY8nb44NduPeFW+LuDeRlhSRu0o3gWdXg50T/NIp3eZVmx4knpiI25ZPaXPiQ1RAcw49toDTUW689vseqo/ZRvShMkQubCl9xIXIiFLQ9CXoMYOqiadY6/EJK9ZtYtSC/Q9MUWzTsQT07rO/doMDc6HVm9B1Ar5F7Xnrm9sY3WJIjHQidGFzpnzmKbEkD+ktKamnIE6Q6YctSLsqRXisahF+XHeay7dTTWTj4AK9f4X6z8K2H+C3QRB796Fz2Hw8ubIPpraB20Ek9F3EzLuN6TN7E67lbhG5vRKRC1oRvqS5xdybZGVMSuZtLUKk5FsJhq/T5+5fPAjavAstXyP4liIoSA8CSYOwho1shrNXNFFhTml+KYKDeeAYixcTActHwok/od4QjJ2/oa/Rnp5NqvDnoWtM2ngGj/aHKOLqwtAmZenWyh+Qmjgh0lWjJ8rTH9+F/fnH9UOGHxlF9xuRfP5EPRLCXQkIsNFYAhB8Ehb0hfDL0G2q3q0WWLbvMrODDlGumCtvNmpEvQ+crOc5Wak0B87HJ8jMXhbmg6eq0f7bTXy08hjTBtV/cKPRTl/jrVB5WP0O/NIRAheAp/8D8cFm48mRZbDiBXD15UCHxbz5VwInbxyjZUVfPulaDZdXXS3uOUmSIvKGm58+V/nK0bBhHJd276fhlxOJc3Ug6rb+pQ8MVLRvbyQoyDXNL8WCBRrDRibi7B39wDEWK/gULBqAFnKaS1W/wLnRSHyNenntjAaerlOCp2oVZ+3xG0zceJbP1x5hxq7TDG9Rln6NSuHqKF81IdJUqjFq+HocF/RldsKXjA/ty9M/3yNmdyUiD5di+mQDQWdsKJYAHF0Bv79IotGZE43+wrd4Qwolavy4/jTfrz1Nk7KFmDywHp7O9uYuaYFgSGPgfFSstKRYmhJezrzSvgKfrzrBzG3neaZZmQd3UAqavKAnKkuGwpSWrPf5hS7/a/FAfDh32sDqvTEcuRPE7+F3mPx1NLHxiVQp5oG67cHir0vg6KCsI54kxMPaD2DHBMI8GzLO4y2WLA3D39uZyQPq8Xi1IijTgqWWkpwkyUp3L5W6D3BuqF+/vrZnz55cP68wvwdqGAprRK6bjOOmd7mY6MdLiaM4cKMG4UubEXTGmO4XIjgYAson4NljGw5+EcTedM/0GLM6shT+GE10ghPdFs5g990GGQYvTdPYfjaEiRvOsP1sCF4u9gxpGsDgJgF4uzqY4QnolFJ7NU2rn/melkXiie16IJ54RBL920s4nVnGv3H1eSNhBKH3vAmd25qgkw5pxgariyXxsbDuI9gxgVtODWjy0y/cti9EdKQdrf53mKPhN+hetwRfdK+Jg4VPfWtL8aTJ5+toXr4wX/eqlfxY4NSdxCUksuT5pvldRJGBxESN5+ftZfWxG0wdWJ8OVYskb3sgnqgzxM/rjwo5xdd3+jHVqSPRNz2IPeVPxc4XuHonCkc7AzVKeFLMyxk7g+LQpTucuRkBSs93YoPdCF/S3HLjyZ2rsHQ4XNjKtHP9+LLQk8TZKdoXL8fEUeWtoiWw4rurOP1p53RjiVTviixLq8by7NmRrF1eiUU9nmE57/NlsUB+9KpLUJBbul/q9AevuVpWIIiJhFVvwoG5xBVpQI1xv3D38SDc/DbicNOdYSOb0b79w8FLKUWz8oVpVr4w+y/eZtLGs3y/9jRTN5+jf6NSDGtRliIeTuZ5TkJYiIfjiStnz88gdFNtvmj5EavsxvKKepEdfezZcKAuvTt4PnQOq4klACFn9drdaweIqjGc8sM+waXbbtz8DuMSb+DIbY1X21Zh9ONlkms+Rf5Ia0xKdHwCbtICbnEMBsX3ferQd+oORi/Yz/TB9WlWvnAa8aQcF++spdTpUYytNJeWCfsY4/cCN/wi8XTw4fU+FelQtegD7/Hu3dB5cAiuT+0EwME3EpeSoQQF+VpePDm5Cla8QGJcNC+dfY+/SlQBID7EmYVTy/PFACNOllbmNGS26rx8A0WWpJy6z8EvAoeb7jw7ohloEB3fnHZXJvB9sW953/5Xmra5TNlCEwD/NM+VNHjN4aZ7cu2nxQ1eu7gTVjwPoeehxWscdBvLbftY3PwOA1m/GapTyptpg+pz8noEP288wy/bgpi9/QI96vkzslVZShdyzacnJITlyDiePM+xSm787PclCxw+ZYrHU3y4IZwbhmqMbF0O+xTrVlhFLElMhL2/wOr39bVi+szjSOST2HlGJidXBrtEotbXomkvfyQ/yX/pLeZYyNXya6ILImcHI9MG16fv1J30n76LvnUD+OnFCnj22PlAPDF6aNhVfIN9AX585DCbfxzG8trOT/hibmd8/R7+ogUEwL0rXtib4omWqHBuv59Qu3pAoXx/nmmKvqOvIL93Fgl+NXj+7mhWl7h/H2Ff6J7lVtSkIbMuX5KkiCxJq8bSwTUWZZ+Ia92THJ/3OE94tmBYwCK+6/ghDoua6NOM1hkIhge7LWR1YL1ZxETq6xXsnASeJWHwH1CmJaWDIeq2Icc3Q5WKuvN93zq82qESUzaf5bc9l1m0+yJdahXn+dblqFzUI0+flhCWJLN4smb2QOoX7shX9T7jhdpz6eawnxfWDuWvw3X5qmdNavp7ARYeS0Cv5Fg5Gs5vhrKtoetE8PTH80IMdvVPJu8We8uVyJPFLCu5KkCMSmb3sjZ+7k78+VJzvlx1gtk7gij8zCWUvWltIb8IfPpsw65wBJoGc872ZO3Fp5hc431mNB0DG9ZD52/AvegD50wdT2ISEqn87F5GL/2PBYUaU7eUtzme6n1n1sHKl+HOFWIavEjguQ4cuh3FnXXVcaoRZLkVNRnIbPC8JCkiS9KqsYy964BS4FwokuLDNxF1vhAzNw7j47mPUWjLKP2P84F58MR4KFr9gfNlNrA+32kaHF8J/7wFdy7rUxl2+Agc9akKc+tmqFQhFz7tVoPR7SowY+t55u68wO8HrtK+ShFebFOOOuYOgkLkg6zGk5fW/UiPd7tRbPNolsd+wIo77Rk8sRedG1bj9ccr4eXiYHmxBCAuGrb/CFu+BYMdPPk91BuCBvx96Brv/34E1wpxhO+oAEHFibrtbFnJVQFjMKg01kmR2b0snYuDHR91rU6zUsUZ+N4VHCtfxmCXiJagSIyxI3JHRRwqXMGh6B0uRRWi/aJ/uPbbJNz++wzObYI270CDYfrMYCap44nBuQndJm1n+Ow9LH+hGaUKPbwIYp67cxX+fRuOLodCFbgT+Cd9VyVy5mYkUwbW42YNP4aNLGGZFTWZyKwlRQbOiyxL7vOZ4osAPPRYYKDSuzgcXKA3S0aFQd1B0PotcC+S8UXM4dpBWP0enN8EftXgyfFQqnGau+b21IRh92KZtT2ImduCCI+Ko2m5QrzYpjxNyxXK9X7ptjTQVVi/bMWTmAjY+AXazp+JMrgyPqYrf9h34vn2VenfqLTlDDLXNP1GYt1HcDsIqnWDxz8Dj+IE3brLB38cZdOpYKqX8OCbXrUoZPSwuGlBs8qW4snj320moLALUwbefzp1Pl7NEzWLMe7pGvldRJED2YonIWf1hR/Prge/qtDhYyjfnvT6Wp4NjqT7pO0UcnNg+fPN8HTJp1n3Yu/C9gn62i+J8dDyNaIbvsiAWQc5dCWcaYPq06qiHjisctpkoNfk7Sx5vlm6sUSSFJEtaX0RMvxy3AuFjV/AnhlgdITGI6Hxi+BqAf07g0/C5q/h8BJw9tIXU2swTO83ns8iY+JZsOsi07ac42ZEDLX8PXmhTXk6VCmCwZA7yYot3VQI25DteHLjKPz7DpzbwE1jUb6Ofoo9Ho8x+rEqdKlZHDujmZIVTYPTa2DTF3Blr17Z8fg4KNeW23djmbjhDHN2XMDBzsCrHSoyqElp85U1l9hSPOn0wxZKeDkzffD9p1PlvX8Y0LgU7zxRNb+LKHIoW/FE0/QFqdd+CLfPQ5mW0GosBDRL89y7zoXQf/ou2lcpws8D6ubt5BZxUbBvDmwZD5HXocpT0OEjEr3K8PKiA6w8eJWJ/eryRM1ieVeGfDJwxi7mDmsss3uJ3OHr+/CNQ1qPJXPxgc5fQaPnYP0n+pdu52SoNwQaDgefMukcmEc0DS7t0secHPsD7J2h2cvQfIyeqJiJm6Mdw1uWZWCT0izbd4XJm87y3K97qeDnxgttypn3BkyIPJLteFKkGgxaAWfW4bvuI76+NpUb0Sv4eUlHpq3pyOA2Nehau0T+ddOJj9VvdHb8pLfIepaCrpOgVl/CohOYvfY007ee425MPD3q+vP645Xwk5n9LI7RwAPdvTRNk8UcrVC24olSUO1pqNQZ9vyid82c1RlKNYWmo6BiRzDcf/8blS3EGx0r8dnfJ5i78wIDmwTk/hO4GwL7ZsPOn+HuTSjdDPr8CiUbAvD9mlOsPHiVNztWtokEBWR2L5FHst20WKgc9JoFrU7owWDXZD1RqPAY1OkPFR4H+zz84303BI4u02snrh8CR09o8arltOqYONkb6deoFL3r+/PnoWv8vPEsYxYdZPyaUzzXshw96/nLH05hU3LUTaF8O1S5tnB6DX6bv+bDy3O4d+83lv3elJGr2lKjUXv6NCyFv3ce9R8PPgWHFsL+uRB5A3zKwVMToFZfzt+OZe7fJ1n430XuxibQvkoRXn+8EpWKuudNWcQjSz1wPjYhEU1DYq0VynY8sXPQe3jUGwx7Z+tjyRb20ysc6g6Cmr3BuzQAw5qXZfvZED756zj1SvtQtXguTHiTmAhBW/Tu8UeWQUIMlG0DLWc90Kqz7vgNflx3ml6mmUFthYtDxmmIdPcS2ZYrKzzfuQp7Zuq1BpE3wMlTr7mo+DiUawvOuTCA/PYFOL0aTv0L5zbofTr9qkHDYVCjNzi6ZXi4JfTxTEzUWHfiJhM3nOHApTB83R0Z1rwM/RuXzvYc/rbUPUPYhlxbLf7KXrT/ppJ4ZDnGhBiCtCKsTajHjaKtqdygPe1qlMTL5REWUk1M0FtKTq+Gk3/r/1cGvR97wxHc8W/J6mPBrNh/ha1nbmFnUDxRs9gDM/dZQjzJTbYUT7pN2oargx1zhzUCIDwqjlofrebdJ6owrIXt3BDaulyJJwnxcPIv+G+anjwAlGyUfH8S4lKOTj9uxcvFnj9GNc9ZIht7Dy5s0+9NTv4Nd66AgzvU7AUNR4BflQd2vxR6jyd+3EJJHxeWPt+UiDCjzcSSt5Yd5oseNWVMisgdub7Cc0K8PmD98G/6FzYqVH/ct4rexFmkGvhW0qcDdi8GDqlqRjUNosMh4rrerzT4BFw/DBd36bN0AXiXgcpPQK2+UDRrgyBz7eYpl2iaxo5z+ir2286E4Olsz+CmATzTNOur2NvSTYWwfnmyWnz0HTi+kqj9i7G/tA07LY4YzZ5DWlluuFfH2b86JSvUIqBsRRw8iz48/iwxAe7egohrEHJGjydX9sHl3RBzB1DgX5/Eqk9zrkhHNl8zsulUMDvOhRAbn4i/tzN96pekT8OS+Lnfbxm2tHiSG2wpnvSavB07g4EFI/QJU27ciabRZ+sY93R1BjQubY5iimzKk3hy+wIcXqx3Db9+SH/M2YdbPrX5Ncib0lXq0r1dS/AoDs4+Dy23QHyMXgkbfgVunYSbx/VYcu2gXmlq76K3mtToARU7PXx/A8TEJ9Dz5x0Ehdzlr5dasG21s03Fkk/+PMb7XarJmBSRO3J9hWejHZRvp/8kJuhf4PNb4NJOEo/+jmHf7FT7O+rdwgz2kBCrDzBLjHtwHw9/KNkASr6kn7dQ+XRn7UhLyoXmjG4xcL4Qz46olebq8vlFKUXTcoVpWq4wBy6FMWnDGX5cd5rpW/7f3r3HSFWecRz/vefM7MzuDsuCO8uyFFgUaGHlJrIqWBOLlyhUGxuroBCDqNS22tiY1kqq9hJrq+0fpheo1qq1q1FjYpqqbeOlVhSxqLVeUJDBG3ITkIWZndvpH2eHnYVlF9jZPWf2fD+JiRF2eGM4T85vnvd9n/e1sGWMrjiNKfYoL/0yLT5aI824RJUzLpHa2+S8/6w+f/MZjUq8qGltj6ti3aNS53gSpU1UeTsiY1kK5dOyckkZJ7//1x1jK1U7XttHnaNE9VStcqbp9Z1hvfHUbu1pdz/o2Hi1Lj1prOZPG6kZo2sPOlBbqCexeS/LCudkZ2wtXdbiaT1BV5bpOswxlXHnbbDdq3z0Sz0ZNlY67Xr3n88/ceeUfPCS6j58SdeGn5G1/lFpfcfvNbZ7xjUUkZy8ew15Ntn180KVUuN0afZ3pLGnSk2n9rrN/bYn1umNj3drxaKZqsxXaemynK/eTfqKYY4oqX6d8GzZ7tW/Y052v3n8eU4jh32spvAGjR+1SSNCO3TxV3dr0oR2N6CEom5BqI67Q5lqx7hdl+jQPi2jUOwyO2La0nqyQkOTyrQ7WrHS0fIbvf/GYvroWq1cfKLe3bJHv3t2g+5ZldB9LzLFHuWl36fFR2Iyk+YrPmm+WlsdLVma0di6dZoyea1G132kqeM/lq09clJJWcorpQrtU0TbnFptdWqVcBq00RmpdDIsbXY/MhbZp+Pi1TpveqNmjBmmlqbhvc5NSCQkK5LR9sdmKjQ0qezuSlVVZ5RIlO+LxWBjW0bpbGc4TWXcf+/tBQr+0e/1pKZROmGRdMIitbY6+uZNu9Wy8BGNC29W9drhuvzrO9Q8MSllUx2BJepu4RrS4P5s3QT3nMuB3ZYe/POtLfrjCxt12ewmnd3coDVr/P1ucjQY5oiSGogJz53fPK7Rxsdmat+CTVpfX6/01uN0+4/62L49DE1N0r4dUaWeOl4NC1/aX/BuvW2OrrrSPy8WE0cM0a8vmq7rzpyo3z+3QQ//x51iP39qo64+nSn28LeBmha/bZt0+ZV5teekz+fu1Jr6Or2wdZxW3DFHiQ22IkMy+mRXUtv3pPXZvrRGZnJK5/Kaa1mKhC3FIiHVD4lqRE1E8SGRI756NBaT9u4Mq2FR5zaUT++fo1jPR+IwgGzrUJ0UblQsFwNdT1KZKr2ZPk7vxkYqNz2sFcuXKbGhdO8Hm3cndf0jr6u5sUY3nPslSeXzbnIk6KSg5Pp7wnOhk2GFcwoNTfa5fXukB1bjcemH37d0y68OaB0P62PruJ+MHu5Osb+2aIr9469/ojMm1evq08frBKbYw6cGYlp8IiFVxNLKOekuz3O4xn2eZ80Kq6YhLDX0/llHc/i9rU0aUt+1lsTiKbW10fH0C8sY5fNs9yp3A11PwsP3SZLsyoyqJn+oRKKpJO8mmVxe17S+qvZsXncumKFIyP17WG7vJoejtyuI+ZoARyUel2bN6p8iUGjb5jO2srsrld7qXt15NO3b1lZHTeNzOufCvWoan1Nr6+FdFHHVVUYV2Wif/uyBVl8T1Q3nTtKqH8zVdWdO1CubduqC367SgpUv6d/vbfd6eUC3+rOWSG49SbdVKLOray3JfD4wtaSpScq2da0lub3+riVBc1AnpWPrF52U8uNFPclnLFWe9K5idanD/pye6sntT63TmsRO3XrBFB0b79pyLcd3k56w3Qtlp7Nt26Kq6oy7NSKeUm7vkbVviw/AV9TvUcXWIVq6bM5hHTKLx6W7VvR/67g/DK0K65q5E3T5qePU+rI7xf7Su1d7vSzAE/G4dPdKS5ddLn3659myq9tlp6O6564BrCUDsA0FR88yRrnOIylKpt1OSuEbbKCgu3pSUZ1V4+IXddNTr+qBK05SuJfByz3Vk9e2bdGKf72vS08eo/Onj+r2zy/Xd5PuzJ00osdfJ6TAlzrbtrZiMamt7cjbt3297WMgWsf9qToS0tIvd06xX3ib1ysCvFF4ll99VZJCmjGDWoJOtqUu273as25I6e1bXgRTd/XkhY+m6LsPvaZfPPmObpw3ucefP1Q9ee71nG5+/jUdP6pGy3v4jMFUT3qb90ZIgW/F4317+Epx20df1+AHkZCtBS1jtNDrhQAeisels846up+llgxuIctSNl98uxdnUtCzA+vJ1+KjtPaDnfrD8xs15Qu1Om9a4yF/trt60p6Wfrl6jWKRkO5aPKvXv3tBqSeEFAxabLMAUArUksHNsoyKGin7ryCOhjiTgsO3fN5kvbN5j6576DVVhW2dMbn7rUwH1pNU0tLUb72iXe0ZPbzsFDUMZeZZAU8gBrUFC4wS62098Ui1Euvtsp7MCsA71JLByzZSjtu90EcVIUt3XXaimhtrdPUDa/Xcu9sO+XsL9eS+e41mfm+1tmfa9JtLTlBzY9/mvA02hBQMev192weAYKCWDE6WZQ4IKYXbvQgpODI10bDuXdKi8fUxLfnTGt3x93VdBoUWe3vnNt2yapU+S7br/iUtOv2L9QO8Wv9juxcAAAgs2xjli64gTmZyCttGtkW3DEeutqpCD151sm55/C3d+fR6/eOtLbrwxNE6bUKdJGnDtr16YPUmPf/edo09pkr3Lmlh+PIhEFIAAEBg2Qd1UnJ0UdAnNdGw7vjGNJ3VPEK/ePId/eSvb3X59aGVYS2fN0mLThnLVdc9IKQAAIDAcg/Od72CmJCCUji7uUFnNzfow8/2adWG7YqEbI2rq9aEETFVVfAK3hv+DwEAgMCyzcFnUpg2j1IaPbxKFw0f4/Uyyg5PIQAACCzbMsoeuN2LLTiA5wgpAAAgsGzLdJk4n+RMCuALhBQAABBYtmWUc7p2UioJKYDnCCkAACCwLGOULxplkcrkFeFMCuA5nkIAABBYtqWDOils9wK8R0gBAACBdeDtXu3ZPCEF8AFCCgAACCyrY7J84fC8e7sXr0eA13gKAQBAYNnGDSmFLV/c7gX4AyEFAAAEVqGTkivqpFRWEFIArxFSAABAYNlFIcVxHHfiPNu9AM/xFAIAgMAKWZ3bvdqz7l3EEbZ7AZ4Leb0AAAAAr1im8+B8e8fAFM6kAN4jpAAAgMAq3u6V3R9S2GgCeI2nEAAABJZVtN0rmc5JkirppACeI6QAAIDAsvdv95JSWTeksN0L8B4hBQAABJbd8SaU67jZS2K7F+AHPIUAACCwig/OpzIdnZQQnRTAa4QUAAAQWIWD89mikMIVxID3CCkAACCwim/3KoQUDs4D3iOkAACAwCqElDxnUgBf4SkEAACBVbjdq7iTwu1egPcIKQAAILCsbrZ7EVIA7xFSAABAYO2fk+I4SmXZ7gX4BU8hAAAIrO4OznMFMeA9QgoAAAgsq+jgfDKTU0XI2v/fAHiHkAIAAAKr8+C81J7JKxri1QjwA55EAAAQWFbHm1A2n1cqk+PQPOAThBQAABBYoY6Uks+LkAL4CCEFAAAElt3xJpTrGObIzV6AP/AkAgCAwLIKVxDnHaWyOVXSSQF8gZACAAACq/gK4mQ6pwghBfAFQgoAAAisQicl1zHMkTMpgD8QUgAAQGAVOikf7NinzbuSXEEM+ETI6wUAAAB4pRBSfva3t1VdYeviltEerwiAREgBAAABdkx1haJhS6eOj+vH5zersbbS6yUBECEFAAAE2DGxiP5389myLSPTcT4FgPcIKQAAINBCNudQAL/hqQQAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK8ZxnNJ/qDHbJG0q+QcD6IuxjuPEvV7EkaKeAL5EPQFQCoesJf0SUgAAAADgaLHdCwAAAICvEFIAAAAA+AohBQAAAICvEFIAAAAA+AohBQcxxswyxvzXGBM1xlQbY940xhzv9boAlB/qCYBSoZ4EC7d7oVvGmJ9KikqqlPSR4zi3erwkAGWKegKgVKgnwUFIQbeMMRWS1khKSZrtOE7O4yUBKFPUEwClQj0JDrZ74VCOkRSTNETuNxYAcLSoJwBKhXoSEHRS0C1jzOOSHpQ0TtJIx3G+7fGSAJQp6gmAUqGeBEfI6wXAf4wxiyVlHMf5izHGlrTKGPMVx3Ge9nptAMoL9QRAqVBPgoVOCgAAAABf4UwKAAAAAF8hpAAAAADwFUIKAAAAAF8hpAAAAADwFUIKAAAAAF8hpAAAAADwFUIKAAAAAF8hpAAAAADwlf8DE192GNEnrKkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1008x360 with 3 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_40_1.png"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\"\"\"\n",
    "============================\n",
    "Underfitting vs. Overfitting\n",
    "============================\n",
    "\n",
    "This example demonstrates the problems of underfitting and overfitting and\n",
    "how we can use linear regression with polynomial features to approximate\n",
    "nonlinear functions. The plot shows the function that we want to approximate,\n",
    "which is a part of the cosine function. In addition, the samples from the\n",
    "real function and the approximations of different models are displayed. The\n",
    "models have polynomial features of different degrees. We can see that a\n",
    "linear function (polynomial with degree 1) is not sufficient to fit the\n",
    "training samples. This is called **underfitting**. A polynomial of degree 4\n",
    "approximates the true function almost perfectly. However, for higher degrees\n",
    "the model will **overfit** the training data, i.e. it learns the noise of the\n",
    "training data.\n",
    "We evaluate quantitatively **overfitting** / **underfitting** by using\n",
    "cross-validation. We calculate the mean squared error (MSE) on the validation\n",
    "set, the higher, the less likely the model generalizes correctly from the\n",
    "training data.\n",
    "\"\"\"\n",
    "\n",
    "print(__doc__)\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "\n",
    "def true_fun(X):\n",
    "    return np.cos(1.5 * np.pi * X)\n",
    "\n",
    "np.random.seed(0)\n",
    "\n",
    "n_samples = 30\n",
    "degrees = [1, 4, 15]\n",
    "\n",
    "X = np.sort(np.random.rand(n_samples))\n",
    "y = true_fun(X) + np.random.randn(n_samples) * 0.1\n",
    "\n",
    "plt.figure(figsize=(14, 5))\n",
    "for i in range(len(degrees)):\n",
    "    ax = plt.subplot(1, len(degrees), i + 1)\n",
    "    plt.setp(ax, xticks=(), yticks=())\n",
    "\n",
    "    polynomial_features = PolynomialFeatures(degree=degrees[i],\n",
    "                                             include_bias=False)\n",
    "    linear_regression = LinearRegression()\n",
    "    pipeline = Pipeline([(\"polynomial_features\", polynomial_features),\n",
    "                         (\"linear_regression\", linear_regression)])\n",
    "    pipeline.fit(X[:, np.newaxis], y)\n",
    "\n",
    "    # Evaluate the models using crossvalidation\n",
    "    scores = cross_val_score(pipeline, X[:, np.newaxis], y,\n",
    "                             scoring=\"neg_mean_squared_error\", cv=10)\n",
    "\n",
    "    X_test = np.linspace(0, 1, 100)\n",
    "    plt.plot(X_test, pipeline.predict(X_test[:, np.newaxis]), label=\"Model\")\n",
    "    plt.plot(X_test, true_fun(X_test), label=\"True function\")\n",
    "    plt.scatter(X, y, edgecolor='b', s=20, label=\"Samples\")\n",
    "    plt.xlabel(\"x\")\n",
    "    plt.ylabel(\"y\")\n",
    "    plt.xlim((0, 1))\n",
    "    plt.ylim((-2, 2))\n",
    "    plt.legend(loc=\"best\")\n",
    "    plt.title(\"Degree {}\\nMSE = {:.2e}(+/- {:.2e})\".format(\n",
    "        degrees[i], -scores.mean(), scores.std()))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Degree of polynomial:   1\n",
      "Mean squared error on training data: 439230.69504801\n",
      "Mean squared error on test data: 481979.17861098\n",
      "Degree of polynomial:   2\n",
      "Mean squared error on training data: 115822.95008046\n",
      "Mean squared error on test data: 123711.53703498\n",
      "Degree of polynomial:   3\n",
      "Mean squared error on training data: 9011.85263220\n",
      "Mean squared error on test data: 10913.84780262\n",
      "Degree of polynomial:   4\n",
      "Mean squared error on training data: 303.47610036\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean squared error on test data: 426.30787294\n",
      "Degree of polynomial:   5\n",
      "Mean squared error on training data: 3.80354994\n",
      "Mean squared error on test data: 5.98822371\n",
      "Degree of polynomial:   6\n",
      "Mean squared error on training data: 3.66204648\n",
      "Mean squared error on test data: 8.14812206\n",
      "Degree of polynomial:   7\n",
      "Mean squared error on training data: 0.47075725\n",
      "Mean squared error on test data: 2.00607783\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Degree of polynomial:   8\n",
      "Mean squared error on training data: 0.04912436\n",
      "Mean squared error on test data: 0.21596432\n",
      "Degree of polynomial:   9\n",
      "Mean squared error on training data: 0.02522069\n",
      "Mean squared error on test data: 0.08576932\n",
      "Degree of polynomial:  10\n",
      "Mean squared error on training data: 0.02511518\n",
      "Mean squared error on test data: 1.20015436\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Degree of polynomial:  11\n",
      "Mean squared error on training data: 0.01640891\n",
      "Mean squared error on test data: 1.35533773\n",
      "Degree of polynomial:  12\n",
      "Mean squared error on training data: 0.00813803\n",
      "Mean squared error on test data: 0.17446471\n",
      "Degree of polynomial:  13\n",
      "Mean squared error on training data: 0.00759119\n",
      "Mean squared error on test data: 1.08131003\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Degree of polynomial:  14\n",
      "Mean squared error on training data: 0.00472199\n",
      "Mean squared error on test data: 0.81333793\n",
      "Degree of polynomial:  15\n",
      "Mean squared error on training data: 0.00410478\n",
      "Mean squared error on test data: 92.09145189\n",
      "Degree of polynomial:  16\n",
      "Mean squared error on training data: 0.00315593\n",
      "Mean squared error on test data: 234.39716546\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Degree of polynomial:  17\n",
      "Mean squared error on training data: 0.00242998\n",
      "Mean squared error on test data: 1271.05295709\n",
      "Degree of polynomial:  18\n",
      "Mean squared error on training data: 0.00228740\n",
      "Mean squared error on test data: 108.42208194\n",
      "Degree of polynomial:  19\n",
      "Mean squared error on training data: 0.00156372\n",
      "Mean squared error on test data: 1388.41078073\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Degree of polynomial:  20\n",
      "Mean squared error on training data: 0.00137982\n",
      "Mean squared error on test data: 1761.43341615\n",
      "Degree of polynomial:  21\n",
      "Mean squared error on training data: 0.00118170\n",
      "Mean squared error on test data: 15061.31603087\n",
      "Degree of polynomial:  22\n",
      "Mean squared error on training data: 0.00092354\n",
      "Mean squared error on test data: 890.63488525\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Degree of polynomial:  23\n",
      "Mean squared error on training data: 0.00085887\n",
      "Mean squared error on test data: 5483.16796929\n",
      "Degree of polynomial:  24\n",
      "Mean squared error on training data: 0.00084589\n",
      "Mean squared error on test data: 1695.57143061\n",
      "Degree of polynomial:  25\n",
      "Mean squared error on training data: 0.00078806\n",
      "Mean squared error on test data: 131343.30655001\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Degree of polynomial:  26\n",
      "Mean squared error on training data: 0.00076916\n",
      "Mean squared error on test data: 17709.14370264\n",
      "Degree of polynomial:  27\n",
      "Mean squared error on training data: 0.00068970\n",
      "Mean squared error on test data: 2975.38903780\n",
      "Degree of polynomial:  28\n",
      "Mean squared error on training data: 0.00062588\n",
      "Mean squared error on test data: 3848.64522721\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Degree of polynomial:  29\n",
      "Mean squared error on training data: 0.00060728\n",
      "Mean squared error on test data: 2988.64001211\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-5-8dc29df57a8c>:73: RuntimeWarning: divide by zero encountered in log10\n",
      "  plt.plot(polynomial, np.log10(trainingerror), label='Training Error')\n",
      "<ipython-input-5-8dc29df57a8c>:74: RuntimeWarning: divide by zero encountered in log10\n",
      "  plt.plot(polynomial, np.log10(testerror), label='Test Error')\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEHCAYAAACjh0HiAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA/GUlEQVR4nO3dd3zU9f3A8dfn7rInGUBICCPsJSPsoYiLYd2DOkCtuKpYW62/altta2tdtQ5UXLi1ihNxoBLC3lsQBJIwJQTIgOz7/P74XCCE7Nzlm7t7Px/cI3ff7/e+3/dxyb3vs5XWGiGEEP7LZnUAQgghrCWJQAgh/JwkAiGE8HOSCIQQws9JIhBCCD8niUAIIfycw8qLK6WigVeAPoAGbtRaL63p+Li4ON2xY8fmCU4IIXzE6tWrD2mt42vab2kiAP4LfK21vlwpFQiE1nZwx44dWbVqVfNEJoQQPkIplVnbfssSgVIqChgDTAXQWpcAJVbFI4QQ/srKNoJOQDbwulJqrVLqFaVUWNWDlFLTlFKrlFKrsrOzmz9KIYTwcVYmAgcwEHhBaz0AOAbcX/UgrfVMrXWq1jo1Pr7GKi4hhBCNZGUi2APs0Vovdz3+CJMYhBBCNCPLEoHW+gCwWynV3bVpHPCjVfEIIYS/srrX0J3AO64eQzuBGyyORwgh/I6liUBrvQ5ItTIGIYTwd/4xsnjPKlj0tNVRCCFEi+QXiWD3gtfhu7+i139gdShCCNHi+EUi+Cj2dpaW96L8s9+a0oEQQogT/CIRTD+vF590/Sd7y6MpfvtqyN1rdUhCCNFi+EUisNkUD189hsdbPURpYQGFb10JJcesDksIIVoEv0gEACGBdv5y02U8FPA7gg5tpujDW8DptDosIYSwnN8kAoDWEcHceONtPOm8huDtX1Ay/1GrQxJCCMv5VSIA6NUukoFX/5mPyscQuPDfODd9YnVIQghhKb9LBADjerUl75zHWeXsRvnHt8C+dVaHJIQQlvHLRABww5jufNvnCX4pj+D4m1dC/gGrQxJCCEv4bSJQSnHvZaN5vs3foPAoBW9cCaWFVoclhBDNzm8TAUCA3cb9U6/k0ZDfEX5oPQUf3g5aWx2WEEI0K79OBABRoQHcdPNdPKeuJnzbxxTNf9LqkIQQoln5fSIA6BAbxuBrH2GOcziB6f+g7Mc5VockhPCkeX+FD66TGgAXSQQuQ1PiKJn0HJudHSj8+C4oLbI6JCGEJ2gN696BLZ/DT3OtjqZFkERQyaVDurA8ZToRZTnkLX/L6nCEEJ5wcAscywabA779M5SVWB2R5SQRVHHuxCvZ6OxI2aJnwFludThCCHfblW5+jn8MDu+AVa9aG08LIImgig5x4Sxpex0xRVkUb/rC6nCEEO62Kx1adYLUGyHlbEh7FI4ftjoqS0kiqEb/864nw9mG/O8fl8YkIXxJeRlkLIJOY0ApOO8fUJwH6Y9bHZmlJBFUY0hKPF9GXEZc7iacuxZaHY4Qwl0OrIfiXJMIANr0hoHXw4qZcOhna2OzkCSCaiilSBp7E9k6kiPz/PubghA+paJ9oCIRAIx9ABzB8N1frYmpBZBEUIPx/TvzkX0isfvT4cBGq8MRQrjDzgXQuheEtz65Lbw1jL4Hts4BP60BkERQg0CHjYBh0yjQweR9L6ONhfB6ZcWQtezU0kCFYbdDVHv45k9+uWCVJIJaXDayDx/qcYRt/xyOZFodjhCiKfasgrJC6HTm6fsCQuCch+DABtjwfrOHZjVJBLVoFRbIL71vwqmhMP0Zq8MRQjTFrgWgbNBhRPX7+1wGianw/d/8bk1zSQR1uHzsUD5zjsSx/i04lmN1OEKIxtqVDgn9ISS6+v1Kwfn/hPz9sOTZ5ozMcpII6tCldThrkq4nwFlM2bIXrQ5HCNEYJcdgz0roXE21UGXJQ6H3JbD4v5C3r3liawEkEdTD+LPPYl75QMqXveR3RUYhfELmUnCWVd9QXNU5D5ljf/iHx8NqKSQR1MOoLnHMjbyKoNJc9Jo3rQ5HCNFQuxaAPRDaD6v72FYdYdhtsO5dv1nPXBJBPSilGHbWBFY6u1Gy8BkoL7U6JCFEQ+xKh6QhEBhav+NH/x5CY+DbB/1imhlJBPV0Uf9E3nFcStCxfbD5E6vDEULUV+ER2L++ftVCFYKjYOyfIGOhX6xZIImgnoID7CQPu4RtzkSKFzzlF98ShPAJGYsA3bBEADBwKsR194s1CyQRNMB1wzvxqvNCgnK2wM/fWR2OEKI+dqVDQCgkDmrY8+wOOP8Rs2bBypc9E1sLIYmgAeIjgqDvFezXsZSlP2V1OEK0LC11Iadd6WYQmSOw4c/tco65zf8X5O51f2wthCSCBpoyuhuvlF2AY/cSM2RdCAH7N8A/E2HHD1ZHcqr8A5C9teHVQhWUgglPmO6kX93n3thaEEkEDdSrXSS7ki8njzCci562OhwhWobv/2bm8Wlpgy4rZhNtbCIAiOkEZ91vZifd4purFlqeCJRSdqXUWqXUHKtjqa9rxvTmzbJzUFvnwKHtVocjhLUyl8DP8yA62fw8utvqiE7atQCCo6Ftv6adZ/gd0KYvzL0XivLcElpLYnkiAKYDW6wOoiHGdm/N/MhLKcWBXvq81eEIYR2tTWkgvC1c85F5vPZtq6M6adcC6DgKbPamncceAL/6r6lq+v5v7omtBbE0ESilkoCJwCtWxtFQNpvi4tH9+bJ8COUbP/b5rmVC1Gj7PMhaCmfeB/Hdocs4WPuWWRvYakcy4GhW9dNON0biIBh6K6x8BXavcM85WwirSwRPA/cBNa4EoZSappRapZRalZ2d3WyB1eWyQUl8bx+NoyQXdnxvdThCND+nE374m5mSYcB1ZtugqZC3t2V0r65YlrKuieYa4uwHILIdfDHdp2YYsCwRKKUmAQe11qtrO05rPVNrnaq1To2Pj2+m6OoWGuggus/5HNXhlG/40OpwhGh+P35ilnEd+8DJrpndLoDwNrB6lqWhAWZZyvA2ENfNfecMioCJT8LBH2GJ76xRYmWJYCTwK6VUBvA+cLZSqgVVLtbtgjOSmVs+BL11rsxKKvxLeRn88IhZ/7fPZSe32wNgwLWw/Rtr+91rbUoEncaYLqDu1H089LoI0v4NOTvce26LWJYItNb/p7VO0lp3BK4GftBaX2tVPI0xrHMM8wPH4CgvhJ++sjocIZrP+nfNiNuzHzy9IXbg9aCd1jYaZ/8Exw66r32gqgv+DY4gmHO3T0w3Y3UbgVdz2G3E9z6LX3Qryjd8ZHU4QjSP0iJIe9Qs69h9wun7W3WElLNhzZtNH21cVty4qaB3LTA/mzJ+oDaRCWbdgl3psN771zhuEYlAa52mtZ5kdRyNMaFfe74oH4b6+Tszy6EQvm7Va6ZBeNxfaq52GTQV8vbAz03sSPHF3TDzzIYPVNuVDtEdoFWHpl2/NoNugPZD4Zs/wbFDnrtOM2gRicCbDescQ1rgmdh0KWzxmjFxQjROcT4sfAI6n1V7b5zuEyCsddMajXctNFVQ4W3h6z/C6jfq9zxnuZk+2lOlgQo2G1z4X/N/8s0Dnr2Wh0kiaCKH3Ub73iPI0m0o3yi9h4SPW/YCHM+Bs/9S+3H2ABhwDWz7unFr/5YVw5zfmW/1dyw3E799MR02/K/u5x7YAEW5Jll5WuueMOpu2PB+y5tnqQEkEbjBhH7t+LR8OLZdCyH/F6vDEcIzjh+GJc9Cj0mQVI8pnQdeD7oc1r7T8GstfgZytpuumiHRcNXbZoTwJ7fCj5/V/tydrvaBjqMbft3GGP0HiEkxiavkeM3HlZWYiSqXPg8fXAdP94NZk+C7h+Gnr+FYTvPEWw2HZVf2IcM7x/J0wBju0p/Cj5/C0FusDkkI91v0H1MNcvaD9Ts+prP5Vr7mTRh9T/2necjZAemPQ6+Loeu5ZltACEx+H966BD66Ca4Ohm7nV//8XekQ3wMi2tTvek0VEGyqiN6YBOmPmUZkMB/se1ZA1jIzEnnfGigrMvuik6HdADPyeckzZnZTgNguZknN9oNN+0N8j6ZPj1EPkgjcwGG30bXPYH7akEzXDR9ik0QgfE3ePlgxE/pdZapD6mvQDfDhFNgxH7qeU/fxWsPcP5iF5i949NR9QeFw7Ufwxq/MN+pr/nd69U9ZiZnyomKkc3PpNBr6X2tKMnn7Ye9qU6IBsAVAwhmQehO0H2I+4CMTTj635DjsW2uSxu4VsP1b0zYCEBgBSanmef2ugtgUj4QvicBNJvRN4NM1I/jj3vfhSKZneysI0dzSHzeNsGP/r2HP6z4BwuJh9ev1SwSbZpu69vGPnfphWSE4Cq77BGZNhPcmw7UfQ4fhJ/fvXQWlxz3fUFyd8/5uYt/+rfmw7/9rSB5mvvkHhNT8vMBQ6DjS3MAkw8M7Yc9K2L0cdq80//8dR0siaOmGp8TyaOBo/qjfN7/Mo++xOiQh3OPwTlO9M2iqGSPQEI5A6H+NaVvIPwARbWs+tvAofP1/5oNz8G9qPi40Bq7/DF4fD+9cAVM+O7kM5a50ULaTH6rNKTQG7t4ANkfTRjMrZT7wY1PgjKvNtuJ8cAS7J85qSGOxmwTYbfTt3Y91uhvOjTK4TPiQ+f8y1Rtj7m3c8080Gtcx0viHv8PxQzDpP3XXi4e3hus/Nx++b10KBzaZ7bvSTTVMSKvGxdpU9gD3T2kBZo4je4D7z+siicCNJvRL4JOy4dgOboaDW60OR4im+2UzbPzQdICo7dt8bWJTzFQPa94wM5ZWZ89qWPkqDJlmSgT1EZUIUz43C9O/eZGpZ9+9wppqIS8nicCNRqTEsjBwFE5ssElKBcKLaW1673z7IARFwsjpTTvfoKmmh8zO+afvKy+DOdNNohnbwIFZrTrClC9MddBr48FZKomgESQRuFGA3UZq7+4s171N9ZAPTEYl/ERBtunL/sMjpqrlsU7w7EDT+DnmD6YKpil6TILQuOpHGi9/0UxnPf7fEBzZ8HPHdTFtBgEhpgoreXjdzxGnkMZiN5vQN4GP1w5n+JGZpt9wYj0G3gjRnEoLTTXK3tUnb0ezzD5lM1NL97zQ/O4mpkLbPk2/piPQ9KJZNsMMuqzo45+7B+b/E7qeDz1/1fjzt+kFN82D3N0QGNb0eP2MJAI3G9kljgcDRlCmXsexcbYkAtF4ZcWmK6Q7Gz7LSuDF0Sf7uEclQ+JAUzefOMg0tHrqg3TgFDN4at07J3vVffVHM2X1hMeb3sga18XcRINJInCzALuNEX06k77xDMZu+hh13t+bZWSg8DHlZaavfMFBuHO1+3qMbP7EJIHxj0HvS0zvm+YS18X0hV/zBoy828xDtHWOGYkr424sJW0EHjChbwIflw5HFeyHzCVWhyO80dJnzYCio5lmXIo7aA3Lnoe47qYE0JxJoMKgqWZR+Z/mwtx7Ib4nDP9t88chTiGJwANGdoljZeBQilWI9B4SDXdwq6k373mh+aBc/Ix7Oh5kLoH962HYbZ7p614fPS+EkBj4+GazXsGFT3u0f7yoH0kEHhBgtzGmdzLfOQehf/zM1MsKUR/lZfDZ7RAYDhOfghF3wsHNsKOJC7yAaagNiTk5WtUKjiDTaFx63Aw0Sx5mXSziBEkEHjKhXwKzS4ehCo9U33daiOosfdb04pn4hKm66XsFRCTA4v827bw5O2Drl5B6Y+3z3jSH4XeYJHDOw9bGIU6QROAhI1PiWBc4kOO2CJApJ0R9VK4S6n2p2eYIhKG3mqkTGrN2b4XlL5k5cIbc7JZQmySyHfzq2aaPTRBuI4nAQwIdNs7uncTc8iHorV/WvmCFEFWrhCrX4afeYKYjXvJM485deNTM89PnssZPEyF8miQCD5rY11U9VHrMdJUToiZVq4QqC46C1Kmw+VMzxXlDrXkTSo/B8NvdEanwQZIIPGhklzi2BPYl1xHrvi6AwvdUVyVU1VBXT59lMxp27vIys6BMx9FmsJgQ1ZBE4EGBDhvjerXj87Kh6O3zzILaQlRWW5VQZVGJpuF4zZtm7eD62vK5mXZhmJQGRM0kEXjYxH5tmV08DFVeDFvmWB2OaGlqqxKqasSdptvlylfrf/5lM8zawd0uaFqcwqdJIvCwUV3i2RHUnSOO1rD9G6vDES1JfaqEKmvTG7qcAytegtKiuo/fvdKMTh56G9jkT13UTH47PCzQYeO8XgmklfVC70qveWEO4V/qWyVU1Yi74Fg2rH+v7mOXPW8amvv/ummxCp8niaAZTOzXlrSSXmZw2YENVocjWoKGVAlV1mkMJPSHpc/V/qXiaBb8+JmZ8TMovMnhCt8miaAZjOoSz7qA/ubBzjQrQxEtQUOrhCpTCkbeBTk/m4nbarL8JUCZJSaFqIMkgmYQ6LDRpVNndto6SCLwd42tEqqs50UQnVzztBPF+aZ3Ue+LISqpSeEK/yCJoJkMT4klraQnzqxl9WvoE77HWQ5f39+4KqHK7A4zdfOeFZC17PT9a9+B4jwYdkfT4hV+o9ZEoJSKqcctupli9WojUuJY5OyDrazQ/AEL/1KUC+9eBStfNn36G1olVNWAa83KZYurTDvhLIflL0D7oZAkq+OJ+qlrhbJ9rltt5Vc7kOy2iHxUj7YRbAvqS7m2Y9+5wDT6Cf+QswPemwyHd8Ck/5gZQJsqMAwG/wbSn4BD2yGuq9n+01dm4ReZ2VM0QF1VQ1u01p211p1qugE5zRGot7PZFH1T2rNZdUVLO4H/2JUOr4yDYwfhuk/ckwQqDLkF7IGw5NmT25bNMOsQ95jkvusIn1dXIhhej3PU5xgBjEiJZX5pL9i3xswIKXzbylfhrUsgrDXc/IP7S4Hh8WaMwPr3If8XM0115mIYdqtpRxCinupKBCMq7iilOlXeoZS6FEBrLS2f9TQ8JY7F5b1R2mn+YIVvKi8z6/F+eQ+knA2/mWemefCEEXdCeYkZbbxshpmuesB1nrmW8Fl1JYInKt2vOn3mg025sFKqvVJqvlLqR6XUZqXU9KaczxukxIexJ6w3xSpYupH6qsIj8M5lZsbP4b+Fye+b0b2eEpsCPSbCilfMDLcDr4PgSM9dT/ikuhKBquF+dY8bqgz4vda6FzAMuEMp1auJ52zRlFIM6dKWVfSQdgJflL0NXh4HGYvhoufh/EfAZvf8dUdOh+Jc0E4ZQCYapa5EoGu4X93jBtFa79dar3Hdzwe2AIlNOac3GJESx/yS3qhD2yBvn9XhCHf5+Xt45RzTTXTKF6Z7Z3NpP8TMLtr/19CqY/NdV/iMulqUOiulPsd8+6+4j+txp5qf1jBKqY7AAGB5NfumAdMAkpO9v5fq8JRYXnf2MQ92LoD+k60NSDTdroXwzuUQ3xMmvwetOjR/DL/+oPmvKXxGXYngokr3n6iyr+rjRlFKhWPaH+7WWudV3a+1ngnMBEhNTW1SKaQlaB8TyrHobuQXRxGxSxKBT/jxU3CEwE3fQFCE1dEI0WC1JgKt9YLKj5VSAUAfYK/W+mBTL+4632zgHa31x009n7cYlhLP4k29OX9nGkrrxs03I1qOzKWmekaSgPBSdU0x8aJSqrfrfhSwHngTWKuUatJXWaWUAl7FDFp7qinn8jYjUuJIK+2Fyt8Ph7ZZHY5oisIjcPBH6DCi7mOFaKHqaiwerbXe7Lp/A7BNa90XGATc18RrjwSuA85WSq1z3SY08ZxeYXhKLIsqtxMI75W1HNCQLOMqhfeqq42gpNL9c4EPAbTWB1QTqzO01otoehdUr9QmMpiguE4cLEyg9c40GDrN6pBEY2UtAVsAJKVaHYkQjVZXieCoUmqSUmoA5hv81wBKKQcQ4ungfFlF9ZDOSDcjUYV3ylwK7QZAgPw5CO9VVyK4Bfgt8DqmV88B1/ZxwJeeDMzXjUiJZUFpb1RxPuxfZ3U4ojFKC2HfWugg1ULCu9XVa2gbcEE1278BvvFUUP5gWOdY/uR0DaTeOV+qFrzRnlXgLIVkaSgW3q3WRKCUeqa2/Vrru9wbjv9oFRZIQkISGQUpdNy5AMbca3VIoqGylgIKkodaHYkQTVJXY/GtwCbgf9S9QI1ooOEpsXy3oic37f4WVXIcAkOtDkk0ROYSaN3LrBQmhBerq40gATOq93xMV88A4DOt9Rta6zc8HZyvG5ESy8Ky3qjyEte3S+E1ystgz0ppHxA+odZEoLXO0Vq/qLUeixlHEA38qJSSCc/dYEinGFbTg3LlgF0ynsCrHNgAJQUyfkD4hLpKBAAopQYC04Frga+A1Z4Myl9EBAfQJbENWxw9ZX0Cb1NRgpMRxcIH1DXFxN+UUquBe4AFQKrW+iat9Y/NEp0fGJESy7zCHuj9G+D4YavDEfWVuQSiO0BkO6sjEaLJ6ioRPIipDjoD+BewRim1QSm1USm1wdPB+YMRKXEsLO+NQpuFzkXLpzVkLZPSgPAZdfUactuaA6J6gzq0YoutK8W2UIJ2pkHvi60OSdTl0HY4fkjaB4TPqGtAWWZzBeKvQgLt9E2OZd2hPgyVdgLvkLXE/JQSgfARdbURzKnrBPU5RtRuREosXxf2gCO74Ijk3hYvcymExUNsF6sjEcIt6qoaGlVpecrqKMCnF5xvDiNS4njg+z7m3di1AFpdb3VIojZZSyB5mCwoJHxGQ5aqrElJ3YeI2vRvH81uR3vyHbFE7FwAAyURtFi5e+FoFgy9zepIhHCbei1VqZRqAyS6Nu/VWv/i6cD8SaDDxuCOsSw/0JdzdqaB0wm2eg3xEM3txPgBaSgWvqOuNoL+SqllQBrwmOu2QCm1zLVGgXCTESlxfH28u+mNclCGabRYmUsgMBza9LU6EiHcpq6vnbOA6Vrrnlrrc1y3HsDdrn3CTUakxLK43LV8ZX2mmygtgh3zYcmzUJzv2eDESVmuhertddWqCuE96vptDtNaL6+6UWu9TCkV5qGY/FLvdpEUBLchO7A98TvTYPgdpx6gNRzcAjt+MLfMxVBWZPZlLoWr3pbqJE87ftiU1npfanUkQrhVXYngK6XUl8CbwG7XtvbA9biWrRTu4bDbGNophkW7+3BJRjqUlUBRrpmDqOLDv8C1QFxcdxh0A6ScDdlbYN5fIP0xOOt+S1+Dz9vt+k4k7QPCx9TVWHyXUmo8pvfQicZi4Hmt9VxPB+dvhqfE8fVPPbgk8CuYMQwO7zA7QlpB57Hmgz9lLEQlnXxS13Ph4FZI+xe07Qc9JlgTvD/IdC1UnzjI6kiEcKs6Kzq11l9hZhwVHjYiJZannb3JC+9MZEQC9P+1+fBPOANs9uqfpBRM+o8pGXw8DW7+HuK7N2/g/iJrKSQOlIXqhc9pdKWyUmqmOwMR0L1NBAFh0TycPAtu+BLG/MF88NSUBCoEBJs2goBgeG8yFB5tjnD9S8lxs1C9zC8kfFBd3UdjarjFAlIH4WY2m2J451iW7jiE1rphT45KgivfhKOZ8PHN4Cz3TJD+au8qcJbJ/ELCJ9VVIsgGVmEWoqm4rXLdWns2NP80PCWWfblFrNh1mAO5RRw9XkJhSTlOZz0SQ4cRMP7fsP1bmP9PzwfrTzJdC9W3l4Xqhe+pq41gJzBOa51VdYdSanc1x4smGtklDoCrZi47bV+g3UZQgI0gh53gABtBDhutI4J57PJ+tI9xLXyfehPsXw8Ln4C2fWVaa3fJWgJtekNItNWRCOF2dSWCp4FWwGmJADPKWLhZp7gw3rt5GPtzCykqdVJcVl7jz+JSJ/N/OsgjX27hxetcPVmUgglPmDEHn94Ocd2gjcwL2CTlZbB7pWm8F8IH1dV99Pla9j3r/nAEmOqh+nrm++08NW8bqzMPM6hDjNnoCIIr34KZZ8L7k+Hm+RAa46Fo/cCB9VB6TMYPCJ9V38XrL63mNk4pJe0EFvvN6E60jgjikS+3nNrAHJlgehLl7oXZN0njcVNkuiaaS5aGYuGb6tt99CbgFeAa1+1l4I/AYqXUdR6KTdRDaKCDe87txpqso3y96cCpO9sPgYlPmlHJ3z9sTYC+IGsptOpokqsQPqi+icAB9NRaX6a1vgyzGI0GhmISgrDQ5YOS6NYmnH9/vZXScuepOwdNMQ3Ii/8LGz+yJkBvprVJBFIaED6svomgfZU1CA66th0GSt0flmgIh93G/eN7kJFznHeXV9Ouf8GjZiDUZ7+VpTAb6tA2OJ4j7QPCp9U3EaQppeYopaYopaYAn7u2hQFHPRadqLex3VszvHMs//1+O3lFVXKzIxAuewV0uelWKuov07VQvZQIhA+rbyK4A3gd6O+6vQHcobU+prUe65nQREMopfjThJ4cPlbCSwt2nH5AVBIMmgrr3oXDu5o9Pq+VVbFQfYrVkQjhMfVKBNp0R1kE/AB8D6TrBs+BIDytb1IUF/VvxysLd7E/t/D0A0bdA8oO6S24VKC1mff/lx+h5JjV0ZgeQ8nDZaF64dPqtcySUupK4HHMkpUKeFYpda/Wukmtj0qpC4D/AnbgFa31o005n4A/nNedrzYe4Mlvt/HEFWecujMyAVJvhBUzYfQ9zf8t1+mEgl8gfx/kVbnl74e8veZ+xYI73S6AX3/QvDFWlrsHcrNg+O3WxSBEM6jvensPAIO11gcBlFLxwHdAoxOBUsoOPA+cC+wBViqlPtday4K9TdA+JpQpIzrwyqJd3DSqEz0TIk89YNTdsPp1Uyq45IXmDe7jm2FTlV8ZW4BJUJGJ0G4AdJ9g7udsh1Wvwa506DTGfTFoXf9v9yfGD0hDsfBt9U0Etook4JJDE6awdhkC/Ky13gmglHofswCOJIIm+u3Yrvxv1R7+9dVW3rxxyKk7I9rC4N/Ashkw+vcQ16V5gsr+ySSBMyZDr4sgwvXhHxpb/RKbpUWwfR58+yDcnOaeZTizf4JXzjHX7jQaOo6GjqMgLK7647OWQGCEmbNJCB9W37+ur5VS3yilpiqlpgJfAk1doSyRk8tfgikVJFY9SCk1TSm1Sim1Kjs7u4mX9A9RoQHceXYX0rdls3B7Nf9nI6eDPcgsb9lcVsw01zzvH9B9PLTrD+HxNX/ABwTD2X82E+hVLUU0htYw53egbBDdHta9Bx9OgcdTYMZwmHsf/Pi5aZ+okOlaqL6u9SCE8HL1bSy+F5gJ9HPdZmqtm2UgmdZ6ptY6VWudGh8f3xyX9AnXDe9AUqsQ/jl36+lTWIe3hiE3w8YPIXub54MpyjUfvH0vr/nbd3X6XmGW3/z+76aE0BTr3oXMxXDe3+Ha2XB/Jtz0HYz7C4S3gbVvwf+ug8c6wwujTGLI3iLjB4RfqHd5W2s9W2t9j+v2iRuuvRdoX+lxkmubcIMgh517z+/Olv15fLK2mv/WkdPBEQIL/u35YNa+YyZtGzKtYc+z2cwHd26WKVE01vHDpoqp/TDof63ZZg+A9oNN9dj1n8IfM+HGb2DsAxDaCta8YY7rLL2jhe+ra4WyfKVUXjW3fKVUXhOvvRLoqpTqpJQKBK7GDFQTbnJhv3b0TYziyW9/oqi0yqRzYXEwdBpsmg0Ht3ouCKfTfIi3H2qqgxqq81nQ5VwzEK5ytU1DzPsLFOfBpKdqropyBELyMDjzXpjyBdyfBXethaTUxl1TCC9SayLQWkdorSOruUVorSNre25dtNZlwG+Bb4AtwP+01pubck5xKpvNDDLbl1vE64szTj9gxF0QGAYLPNhr9+d5cGQXDL2l8ec492EozoeFTzb8uZlLTbXP8DvMwjL15QiCmM4Nv54QXsgNXTEaT2s9V2vdTWudorV+xMpYfNXwlFjG9WjNjPk/c/hYyak7Q2Ng6K2w+VP4xUM5ePlLppdOz181/hxteptFYVbMhCMZ9X9eealpII5KhjNlbkQhamJpIhDN4/7xPThWUsYz328/fefwOyAoAtI8UCo4tB12fG8GsdkDmnausQ+YUdHf/63+z1n6nGnwnfC4KfkIIaolicAPdG0TwVWD2/P2skx2Hz5+6s7QGBh2G2z5HA5sdO+FV8wEe6CZ46ipItuZpLVpNuxdXffxRzIh7d/QYxJ0v6Dp1xfCh0ki8BPTx3XDphTPz//59J3DboegKPeWCoryTJfN3pea7qruMHI6hMbBt38x4wJqojXMvdeMGRjfDL2ihPBykgj8RNuoYCYPac9Hq/ecXioIiTbz6WydYwZwucP696CkwPRMcpfgSDjrfshcBNu+rvm4rXNg+zcw9k9m1lUhRK0kEfiR287qUkup4DYIdlOpwOk0jcSJqZA4qOnnq2zQVIjtYrqElpedvr84H776I7TpaxrChRB1kkTgR9pGBXN1TaWC4CgYfif8NBf2rmnahXb8AId3eOaD2B4A5zxkVg5b+9bp++f/y8xgOuk/YK/vVFpC+DdJBH7mtrNSsCnFjLRqSgVDb4GQVk0vFax4yUzb0Ouipp2nJj0mmVHC8/8JxQUnt+9fD8tfMKWG9oM9c20hfJAkAj+TEBXC1UPa8+GqPew5UrVUEAkj7jT163tWNe4COTtg+7cw6AYzWtcTlDKT1x07CEueNduc5WbMQEgMnPNXz1xXCB8licAPVZQKnp9fzZKWQ6aZqaF/+Iep62+oFS+bNQZSb2h6oLVpP9iUOJY8A/kHYPUs0630/H+aUo0Qot4kEfihhKgQrhrcng9X7T69VBAUAWPug53z4YNrTeNrfRXnw7p3oPfFZt0DTxv3VzN6eO698N3DZgGbfld6/rpC+BhJBH7qZFtBNaWCobfA+MdNF81Xzq3/Yvfr3zeTuw1pwrxCDRGbAoNvMoPhygph4lOytrAQjSCJwE+1iw7hysFJ1ZcKlDL9/6/72Kwl/PJY2Lmg9hNqbUYStxvYvDN2jrnPzCU09gGI69p81xXCh0gi8GO3n2WWqay2VABmCuhp800PoLcuMfX/NY3o3TnfdOkcekvzfisPi4Xp681azEKIRpFE4MfaRZ9sK9h7tLD6g2I6w03zoOt5MPcP8MV0KCs5/bjlL0FYPPS+xLNBV8cd6xkL4cfkL8jP3VZRKqhutHGF4Ei4+l2zmteaN+DNX0FBpbWQD++Cbd+4uowGeThiIYS7SSLwc4nRIVyZ2p7/rdrNvppKBWC+dY/7C1z2Kuxba9oN9m8w+1a+YhZ4T72xeYIWQriVJALB7WMr2gpqKRVU6Hs53Pi1GcD12vlmhtE1b5mFZyITPBypEMITJBEIEqNDuCK1PR+srKNUUKHdAJiWBm36wKe3QXGuTPAmhBeTRCAAuP2sFABeqKkHUVURbWDqHBj8G+hzGbQf4sHohBCeJIlAAJDUKrRhpQIwDcMTn4TLX5OBXEJ4MUkE4oTbz0pBo+tfKhBC+ARJBOKEpFahXD7IlAr259azVCCE8HqSCMQp7hibglNLqUAIfyJLOIlTVLQVvLUsk9Jyzb3ndycmzEPrCgghWgRJBOI0f57Uk7BAO7OWZDB3437+cF43fj20A3abNAgL4YukakicJjTQwYOTevHV9NH0SYzkz59tZtKzi1iZcdjq0IQQHiCJQNSoa5sI3r5pKDOuGUju8RKueHEpd7+/ll/yiqwOTQjhRpIIRK2UUkzom8B3vz+TO8/uwtxNBzj7iTReWrCDkrJGLGUphGhxJBGIegkNdPD787oz73djGJ4Sy7++2soF/00nfVt23U8WQrRoSte00EgLlJqaqletWmV1GAKYv/UgD3+xmYyc4/RMiCQ4wIbClCDMT1AoXP9OPO4YF8blgxIZmNwKJaORhWgWSqnVWusalw6URCAarbisnNcXZ7BkRw4Vv0dag0ajNTi1dj0GXI8378ujsLScznFhXDYoicsGJtE2KtjS1yGEr5NEIFqUguIy5m7Yz0er97Ai4zA2BaO6xnPFoCTO7dWG4AC71SEK4XMkEYgWK+PQMT5avYfZa/awP7eIqJAAfnVGOy4flES/pCipOhLCTSQRiBav3KlZsuMQH67awzebD1Bc5qRbm3CmjUnhsoGJkhCEaCJJBMKr5BaWMmfDPt5bkcWmvXkM7RTDI5f0pUvrcKtDE8Jr1ZUIpPuoaFGiQgK4ZmgHPr9jFP+6tC9b9ucx/r/pPPXtTxSVllsdnhA+yZJEoJR6XCm1VSm1QSn1iVIq2oo4RMtlsykmD0nm+9+fxcS+CTzzw89c8HQ6i7Yfsjo0IXyOVSWCeUAfrXU/YBvwfxbFIVq4+Iggnr56AG/fNBSAa19dzvT315KdX2xxZEL4DktmH9Vaf1vp4TLg8saeq7S0lD179lBUJPPftCTBwcEkJSUREBDglvON6hrH13ePYUbaDl5I+5n5Ww9y//ieXD24PTaZFVWIJrG8sVgp9QXwgdb67Rr2TwOmASQnJw/KzMw8Zf+uXbuIiIggNjZWepe0EFprcnJyyM/Pp1OnTm4//88HC3jgk40s33WYQR1a8cglfejRNtLt1xHCV1jWa0gp9R3QtppdD2itP3Md8wCQClyq6xFIdb2GtmzZQo8ePSQJtDBaa7Zu3UrPnj09dv6PVu/hn3O3kF9URkp8OJEhDiKCA4gMdhAZEkBkcEClbeZ+dEggPRMicNiln4TwH3UlAo9VDWmtz6ltv1JqKjAJGFefJFDHuZrydOEBnn5PlFJckdqecT3b8NKCHWTmHCevqJSD+UX8fLCMvKJS8ovKKHee/quVGB3CNcOSuXpwsqy+JgQWtREopS4A7gPO1FoftyIG4RtiwgL5vwnVlzq01hwvKSevqJS8QpMc9h4p5H+rdvPY1z/x9HfbubBfO6aM6EC/pOjmDVyIFsSqpSqfA4KAea5vjsu01rdaFEuT5OTkMG7cOAAOHDiA3W4nPj4egBUrVhAYWPM3zlWrVvHmm2/yzDPP1HqNESNGsGTJkibHmpaWxkUXXXRKvf0TTzzBOefUWnjzWkopwoIchAU5SIgy2wZ3hIsHJLLtl3zeWprJ7DVmiov+7aOZMqIDE/omEOSQ+Y6Ef7G8sbghamoj8FQ9dEM99NBDhIeH84c//OHEtrKyMhyOlrE0dFpaGk888QRz5syp8RitNVprbDZbtY9rUt3rbEnvTU3yikr5ePUe3lyayc5Dx4gNC2TykGR+PTSZdtEhVocnhFtY1kZghYe/2MyP+/Lces5e7SL564W9G/ScqVOnEhwczNq1axk5ciRXX30106dPp6ioiJCQEF5//XW6d+9+ygfzQw89RFZWFjt37iQrK4u7776bu+66C4Dw8HAKCgpIS0vjoYceIi4ujk2bNjFo0CDefvttlFLMnTuXe+65h7CwMEaOHMnOnTtr/cCvLCMjg/PPP5+hQ4eyevVqZsyYwbRp0048njt3Ls899xxfffUVSikefPBBrrrqKtLS0vjzn/9Mq1at2Lp1K9u2bWvw/6/VIoMDmDqyE9cP78jiHYd4Y0kmz6f9zAsLdjC2e2v6JEaSGB1CUqtQklqFkBAVLA3Nwuf4VCJoSfbs2cOSJUuw2+3k5eWxcOFCHA4H3333HX/605+YPXv2ac/ZunUr8+fPJz8/n+7du3Pbbbed1g9/7dq1bN68mXbt2jFy5EgWL15Mamoqt9xyC+np6XTq1InJkyfXGNfChQvp37//icezZ8/Gbrezfft23njjDYYNG0ZGRsYpj2fPns26detYv349hw4dYvDgwYwZMwaANWvWsGnTJo90E21ONptidNd4RneNZ/fh47yzPIsv1u/j+62/ULnQbLcp2kYGk9gqhKRWrgQRHUKH2FD6JkURGih/UsL7+NRvbUO/uXvSFVdcgd1u6ppzc3OZMmUK27dvRylFaWlptc+ZOHEiQUFBBAUF0bp1a3755ReSkpJOOWbIkCEntvXv35+MjAzCw8Pp3LnziQ/jyZMnM3PmzGqvMXr06NNKChkZGXTo0IFhw4ad2Fb58aJFi5g8eTJ2u502bdpw5plnsnLlSiIjIxkyZIjXJ4Gq2seEcv/4Htw/vgclZU725xay50ghe44cZ++RivuFLNuRw/68vScShcOm6J0YRWqHVgzu2IpBHWKIjwiy9sUIUQ8+lQhakrCwsBP3//znPzN27Fg++eQTMjIyOOuss6p9TlDQyQ8Nu91OWVlZo45parzVPa7v83xNoMNGh9gwOsRW/zpLypwcyC3i5+x8VmceYWXGEd5elsmri3YB0CkujEGuxJDaMYbOcWHS3Vm0OJIImkFubi6JiYkAzJo1y+3n7969Ozt37iQjI4OOHTvywQcfuPX8o0eP5qWXXmLKlCkcPnyY9PR0Hn/8cbZu3erW63ijQIeN5NhQkmNDObtHG8As4blpbx6rMw+zMuMI32/5hY9W7wFMd9dBHVqR2sEkhj6JkdJLSVhOEkEzuO+++5gyZQr/+Mc/mDhxotvPHxISwowZM7jgggsICwtj8ODBNR5btY3gwQcfJDW1xs4EAFxyySUsXbqUM844A6UUjz32GG3btpVEUIMgh51BHVoxqEMrpo0xPa92HjrGqgyTGFZlHGbej78AJpGckRRFascYUl3PiQ6VQW6ieUn3UR9RUFBAeHg4WmvuuOMOunbtyu9+9ztLY5L3pmbZ+cWszjRJYVXmETbtzaXMNQq6a+twUju2IrVDDN3bRhAR7CA8yEF4sENKD6JR/Kr7qD97+eWXeeONNygpKWHAgAHccsstVockahEfEcQFfdpyQR8zHVdhSTnr9xw9kRy+3LCf91bsPu15AXZ1IimEBToqJYkAurUOZ2K/BDrHy2puomGkRCA8Rt6bxnM6NdsPFrDr0DGOFZdxrKSM/KIyCorLKCgq41hxGfmu+wXFZeQXlZKRY2Zr6ZkQyaR+CUzsm0DHON9uzBf1IyUCIbyQzabo3jaC7m0j6v2cA7lFzN24nzkb9vH4Nz/x+Dc/0Scxkkn92jGxbwLtY0I9GLHwZpIIhPARbaOCuXFUJ24c1Ym9RwuZu2E/czbu59GvtvLoV1s5IymKSf3aMaFfAokyfYaoRBKBED4oMTqEm8d05uYxndl9+DhfbtzPlxv288jcLTwydwttI4NpGxVMu+hg2kaaqTMSooPNz6gQWkcEyVQafkQSgRA+rn1MKLeemcKtZ6aQcegYX28+wM8HCziQW8TWA/nM35pNYWn5Kc+xKWgdYZJF+5hQOsSYsRIdY8PoEBtK64ggGRjnQyQRNFFTpqEGMyNoYGAgI0aMOG3frFmzuPfee08MRgN499136dWrlxtfgfAnHePCuPXMlFO2aa3JKyxjf14h+3OL2H+0iAO55v6+3ELW7z7K3I37T1nkJzjARnJMKMkxYXSMDaVDbCgdYsPokxgli/14IUkETRQbG8u6deuA6qehrktaWhrh4eHVJgKAq666iueee67G51ed/rm+0163pOmxhbWUUkSFBhAVGlDj2s+l5U72Hikk8/BxMnOOkZlznMyc42QdPsbC7dkUlzlPHNsxNpQBya0YkBzNwORWdG8bQYBUM7VovvVJ8NX9cGCje8/Zti+Mf7RBT1m9ejX33HMPBQUFxMXFMWvWLBISEnjmmWd48cUXcTgc9OrVi0cffZQXX3wRu93O22+/zbPPPsvo0aPrPH/V6Z9nzpx5yuMNGzZw2223sWrVKhwOB0899RRjx45l1qxZfPzxxxQUFFBeXs6CBQsa+78i/EyA3UbHuDBXd9T4U/Y5nZqD+cXszC5g/Z5c1mYdYeH2Q3yydi9gSg/9EqMZkFxxa0WbyGALXoWoiW8lghZAa82dd97JZ599Rnx8PB988AEPPPAAr732Go8++ii7du0iKCiIo0ePEh0dza233lprKeKDDz5g0aJFJx4vXboUOHX657S0tFMeP/nkkyil2LhxI1u3buW88847sVbAmjVr2LBhAzExMZ7/zxB+wWZTtI0y7QkjusQB5u9g79FC1mYdNbfdR3h9cQYvpZuSQ+uIIGLDg4gMdhAVEkBkSID5GRxAVIjj5OOQAMKDHDhsCrvrZlMKh11hV5W22RQOmyLQbpNG7kbwrUTQwG/unlBcXMymTZs499xzASgvLychIQGAfv36cc0113DxxRdz8cUX1+t8NVUNVZ3+ufLjRYsWceeddwLQo0cPOnTocCIRnHvuuZIEhMcppVyL+YRy4RntADMZ34/78libdZQf9+dx9HgpeYWlZB0+Tm6huX+spLyOM9ctNNBOZHAAkSGOE8klMiSAyGCH66fZFxroIMCuCHAlj4r7AXabSSoO8zPAbjvxHF9tIPetRNACaK3p3bv3iW/ulX355Zekp6fzxRdf8Mgjj7BxY+OrsWTaaOFtghx2V9tBqxqPKS13kl9UdiIx5BaWUlBcRrlT49SasnJNudaUO/Up25xaU+bUFJc6yS8qJa+olLxCc54DeUVsO5hPXmEZeUWlNHYyhSCHjfiIIFpHBNE6Ivjk/chTH8eEBXpdqUQSgZsFBQWRnZ3N0qVLGT58OKWlpWzbto2ePXuye/duxo4dy6hRo3j//fcpKCggIiKCvDz3Lq85evRo3nnnHc4++2y2bdtGVlYW3bt3Z82aNW69jhDuFmC3ERMW6LGeR06n5lhJGXlFZRwvLqO0XFPmdFJa7qS0XLt+nrxfVq4pKXeSV1jKwfxiDuYVcTC/mJ+zC1i6M4fcwtMXmVIKWoUGEhsWSGx4ILHhQcSFmZ+x4YHEhgURF25eY0WpJNBhMze7zZJShyQCN7PZbHz00Ufcdddd5ObmUlZWxt133023bt249tpryc3NRWvNXXfdRXR0NBdeeCGXX345n332WbWNxVXbCGbMmFFnDLfffju33XYbffv2xeFwMGvWrFMWtBHCX9lsiojgACKCA+o+uB6KSsvJzi/mYH4x2fkmSRwqKCGnoJicghJyjhWzZV8ehwqKySuq3yJSgXbbKYmh4v4/L+nLkE6eqdaVSeeEx8h7I8RJJWVODh8r4VBBMTnHTLIoKnVSUlZOSbmTkjInJeXa/CxzUlJeXum+kzvGdqF3u6hGXVsmnRNCiBYg0GE70buqpfGuFg0hhBBu5xOJwJuqt/yFvCdCeA+vTwTBwcHk5OTIB08LorUmJyeH4OCWVwQWQpzO69sIkpKS2LNnD9nZ2VaHIioJDg4mKSnJ6jCEEPXg9YkgICDglBG2QgghGsbrq4aEEEI0jSQCIYTwc5IIhBDCz3nVyGKlVDaQWWlTHHDIonA8zVdfm7wu7+Orr82fXlcHrXV8dQeDlyWCqpRSq2obNu3NfPW1yevyPr762uR1nSRVQ0II4eckEQghhJ/z9kQw0+oAPMhXX5u8Lu/jq69NXpeLV7cRCCGEaDpvLxEIIYRoIkkEQgjh57w2ESilLlBK/aSU+lkpdb/V8biLUipDKbVRKbVOKbWq7me0XEqp15RSB5VSmypti1FKzVNKbXf9rHkl8xaqhtf1kFJqr+t9W6eUmmBljI2hlGqvlJqvlPpRKbVZKTXdtd2r37NaXpcvvGfBSqkVSqn1rtf2sGt7J6XUctfn4wdKqVoXgfbKNgKllB3YBpwL7AFWApO11j9aGpgbKKUygFSttdcPdFFKjQEKgDe11n1c2x4DDmutH3Ul8FZa6z9aGWdD1fC6HgIKtNZPWBlbUyilEoAErfUapVQEsBq4GJiKF79ntbyuK/H+90wBYVrrAqVUALAImA7cA3ystX5fKfUisF5r/UJN5/HWEsEQ4Get9U6tdQnwPnCRxTGJKrTW6cDhKpsvAt5w3X8D8wfpVWp4XV5Pa71fa73GdT8f2AIk4uXvWS2vy+tpo8D1MMB108DZwEeu7XW+Z96aCBKB3ZUe78FH3ljMm/itUmq1Umqa1cF4QBut9X7X/QNAGyuDcbPfKqU2uKqOvKr6pCqlVEdgALAcH3rPqrwu8IH3TCllV0qtAw4C84AdwFGtdZnrkDo/H701EfiyUVrrgcB44A5XNYRP0qZe0vvqJqv3ApAC9Af2A09aGk0TKKXCgdnA3VrrvMr7vPk9q+Z1+cR7prUu11r3B5IwtSU9GnoOb00Ee4H2lR4nubZ5Pa31XtfPg8AnmDfWl/ziqrOtqLs9aHE8bqG1/sX1B+kEXsZL3zdXPfNs4B2t9ceuzV7/nlX3unzlPaugtT4KzAeGA9FKqYqFx+r8fPTWRLAS6OpqGQ8ErgY+tzimJlNKhbkas1BKhQHnAZtqf5bX+RyY4ro/BfjMwljcpuKD0uUSvPB9czU8vgps0Vo/VWmXV79nNb0uH3nP4pVS0a77IZgONFswCeFy12F1vmde2WsIwNXV62nADrymtX7E2oiaTinVGVMKALOM6Lve/LqUUu8BZ2Gmxf0F+CvwKfA/IBkzpfiVWmuvanit4XWdhali0EAGcEulenWvoJQaBSwENgJO1+Y/YerTvfY9q+V1Tcb737N+mMZgO+aL/f+01n9zfZa8D8QAa4FrtdbFNZ7HWxOBEEII9/DWqiEhhBBuIolACCH8nCQCIYTwc5IIhBDCz0kiEEIIPyeJQLQ4Sqly12yQm5RSHyqlQms5dqpS6rnmjK/Stf+mlDqnjmNmKaUur+OYjpVnMhWiuUkiEC1Roda6v2tmzxLgVqsDqo7W+i9a6++sjqMy18y8QjSIJALR0i0EurjmxP/UNUHYMtdAmhOUUhFKqV2uqQRQSkVWPFZKpSml/u2at32bUmq065hgpdTryqz/sFYpNda1farrWvOUWR/it0qpe1zHLFNKxbiOO/FtXyn1F6XUSlcpZqZrNGuNlFKDXHPIrwfuqLTdrpR63HWuDUqpW1zbbUqpGUqpra645la6dobr9a0BrlBKnaeUWqqUWuMqUYVXuuYC14SG31QZWSv8mCQC0WK55koZjxkR+jCwVmvdDzMq9M3Kx7qmF04DJro2XY2Zj73U9dihtR4C3I0ZCQzmA1hrrftiRpm+oZQKdu3rA1wKDAYeAY5rrQcAS4Hrqwn3Oa31YFcpJgSYVMfLex24U2t9RpXtNwG5WuvBrmvfrJTq5IqlI9ALuA4zn0xlOa7JCr8DHgTOcT1eBdzjSpDPApdrrQcBr7lelxA46j5EiGYX4ppWF0yJ4FXMNAeXAWitf1BKxSqlIqs87xXgPsw0FjcAN1faVzGB2mrMByrAKMyHI1rrrUqpTKCba998V3LJV0rlAl+4tm8ETimNuIxVSt0HhGKG9W+u9JxTuOaGiXatawDwFibhgZlfql+ldoUooKsr1g9dE6QdUErNr3LaD1w/h2GSxWJXoSQQk7y6Y5LbPNd2O2bGTSEkEYgWqdA1re4JddS0AKC1XuxqeD0LsGutKzfAVsyzUk79fu8rz8virPTYWfX5rlLEDMzKcruVWa0smMZRmJLCN1WuUdcyiscqPX+e1npylef3BTZrrauWJISQqiHhNRYC1wC4PugPVZ0r3+VN4F1M1UtDztkNM6naT42IreJD/5CrPr7WXkKu6YKPuiZDoyIGl2+A2yq1dXRTZibaxcBlrraCNphJ7qqzDBiplOrien6Y67X9BMQrpYa7tgcopXo38HUKHyUlAuEtHgJeU0ptAI5zclrkqt4B/gG8V49zzgBeUEptBMqAqVrr4vqUPirTWh9VSr2Mmcb4AGaa9LrcgHk9Gvi20vZXMFVXa1wNztmYZQZnA+OAHzGr860BcquJJVspNRV4TykV5Nr8oNZ6m6u66RmlVBTmb/9pTBWW8HMy+6jwKa4Pu4u01tdZHYu7KaXCXYuUxwIrgJFa6wNWxyW8n5QIhM9QSj2LaXStqz7dW81xNTQHAn+XJCDcRUoEQgjh56SxWAgh/JwkAiGE8HOSCIQQws9JIhBCCD8niUAIIfzc/wM63rHG7RTZMQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_41_11.png"
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Common imports\n",
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.linear_model import LinearRegression, Ridge, Lasso\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.utils import resample\n",
    "from sklearn.metrics import mean_squared_error\n",
    "# Where to save the figures and data files\n",
    "PROJECT_ROOT_DIR = \"Results\"\n",
    "FIGURE_ID = \"Results/FigureFiles\"\n",
    "DATA_ID = \"DataFiles/\"\n",
    "\n",
    "if not os.path.exists(PROJECT_ROOT_DIR):\n",
    "    os.mkdir(PROJECT_ROOT_DIR)\n",
    "\n",
    "if not os.path.exists(FIGURE_ID):\n",
    "    os.makedirs(FIGURE_ID)\n",
    "\n",
    "if not os.path.exists(DATA_ID):\n",
    "    os.makedirs(DATA_ID)\n",
    "\n",
    "def image_path(fig_id):\n",
    "    return os.path.join(FIGURE_ID, fig_id)\n",
    "\n",
    "def data_path(dat_id):\n",
    "    return os.path.join(DATA_ID, dat_id)\n",
    "\n",
    "def save_fig(fig_id):\n",
    "    plt.savefig(image_path(fig_id) + \".png\", format='png')\n",
    "\n",
    "infile = open(data_path(\"EoS.csv\"),'r')\n",
    "\n",
    "# Read the EoS data as  csv file and organize the data into two arrays with density and energies\n",
    "EoS = pd.read_csv(infile, names=('Density', 'Energy'))\n",
    "EoS['Energy'] = pd.to_numeric(EoS['Energy'], errors='coerce')\n",
    "EoS = EoS.dropna()\n",
    "Energies = EoS['Energy']\n",
    "Density = EoS['Density']\n",
    "#  The design matrix now as function of various polytrops\n",
    "\n",
    "Maxpolydegree = 30\n",
    "X = np.zeros((len(Density),Maxpolydegree))\n",
    "X[:,0] = 1.0\n",
    "testerror = np.zeros(Maxpolydegree)\n",
    "trainingerror = np.zeros(Maxpolydegree)\n",
    "polynomial = np.zeros(Maxpolydegree)\n",
    "\n",
    "trials = 100\n",
    "for polydegree in range(1, Maxpolydegree):\n",
    "    polynomial[polydegree] = polydegree\n",
    "    for degree in range(polydegree):\n",
    "        X[:,degree] = Density**(degree/3.0)\n",
    "\n",
    "# loop over trials in order to estimate the expectation value of the MSE\n",
    "    testerror[polydegree] = 0.0\n",
    "    trainingerror[polydegree] = 0.0\n",
    "    for samples in range(trials):\n",
    "        x_train, x_test, y_train, y_test = train_test_split(X, Energies, test_size=0.2)\n",
    "        model = LinearRegression(fit_intercept=True).fit(x_train, y_train)\n",
    "        ypred = model.predict(x_train)\n",
    "        ytilde = model.predict(x_test)\n",
    "        testerror[polydegree] += mean_squared_error(y_test, ytilde)\n",
    "        trainingerror[polydegree] += mean_squared_error(y_train, ypred) \n",
    "\n",
    "    testerror[polydegree] /= trials\n",
    "    trainingerror[polydegree] /= trials\n",
    "    print(\"Degree of polynomial: %3d\"% polynomial[polydegree])\n",
    "    print(\"Mean squared error on training data: %.8f\" % trainingerror[polydegree])\n",
    "    print(\"Mean squared error on test data: %.8f\" % testerror[polydegree])\n",
    "\n",
    "plt.plot(polynomial, np.log10(trainingerror), label='Training Error')\n",
    "plt.plot(polynomial, np.log10(testerror), label='Test Error')\n",
    "plt.xlabel('Polynomial degree')\n",
    "plt.ylabel('log10[MSE]')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cross-validation\n",
    "\n",
    "When the repetitive splitting of the data set is done randomly,\n",
    "samples may accidently end up in a fast majority of the splits in\n",
    "either training or test set. Such samples may have an unbalanced\n",
    "influence on either model building or prediction evaluation. To avoid\n",
    "this $k$-fold cross-validation structures the data splitting. The\n",
    "samples are divided into $k$ more or less equally sized exhaustive and\n",
    "mutually exclusive subsets. In turn (at each split) one of these\n",
    "subsets plays the role of the test set while the union of the\n",
    "remaining subsets constitutes the training set. Such a splitting\n",
    "warrants a balanced representation of each sample in both training and\n",
    "test set over the splits. Still the division into the $k$ subsets\n",
    "involves a degree of randomness. This may be fully excluded when\n",
    "choosing $k=n$. This particular case is referred to as leave-one-out\n",
    "cross-validation (LOOCV). \n",
    "\n",
    "\n",
    "* Define a range of interest for the penalty parameter.\n",
    "\n",
    "* Divide the data set into training and test set comprising samples $\\{1, \\ldots, n\\} \\setminus i$ and $\\{ i \\}$, respectively.\n",
    "\n",
    "* Fit the linear regression model by means of ridge estimation  for each $\\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\\boldsymbol{\\sigma}_{-i}^2(\\lambda)$, as"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{align*}\n",
    "\\boldsymbol{\\beta}_{-i}(\\lambda) & =  ( \\boldsymbol{X}_{-i, \\ast}^{T}\n",
    "\\boldsymbol{X}_{-i, \\ast} + \\lambda \\boldsymbol{I}_{pp})^{-1}\n",
    "\\boldsymbol{X}_{-i, \\ast}^{T} \\boldsymbol{y}_{-i}\n",
    "\\end{align*}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Evaluate the prediction performance of these models on the test set by $\\log\\{L[y_i, \\boldsymbol{X}_{i, \\ast}; \\boldsymbol{\\beta}_{-i}(\\lambda), \\boldsymbol{\\sigma}_{-i}^2(\\lambda)]\\}$. Or, by the prediction error $|y_i - \\boldsymbol{X}_{i, \\ast} \\boldsymbol{\\beta}_{-i}(\\lambda)|$, the relative error, the error squared or the R2 score function.\n",
    "\n",
    "* Repeat the first three steps  such that each sample plays the role of the test set once.\n",
    "\n",
    "* Average the prediction performances of the test sets at each grid point of the penalty bias/parameter. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{align*}\n",
    "\\frac{1}{n} \\sum_{i = 1}^n \\log\\{L[y_i, \\mathbf{X}_{i, \\ast}; \\boldsymbol{\\beta}_{-i}(\\lambda), \\boldsymbol{\\sigma}_{-i}^2(\\lambda)]\\}.\n",
    "\\end{align*}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the various values of $k$\n",
    "\n",
    "1. shuffle the dataset randomly.\n",
    "\n",
    "2. Split the dataset into $k$ groups.\n",
    "\n",
    "3. For each unique group:\n",
    "\n",
    "a. Decide which group to use as set for test data\n",
    "\n",
    "b. Take the remaining groups as a training data set\n",
    "\n",
    "c. Fit a model on the training set and evaluate it on the test set\n",
    "\n",
    "d. Retain the evaluation score and discard the model\n",
    "\n",
    "\n",
    "5. Summarize the model using the sample of model evaluation scores\n",
    "\n",
    "The code here uses Ridge regression with cross-validation (CV)  resampling and $k$-fold CV in order to fit a specific polynomial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEGCAYAAABvtY4XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA3c0lEQVR4nO3dd3gU5fr/8fedRhIICYRQQwmdAKEX6UUpggLiEURQ0CMqKvaf7dg5ikeP9SiKigVRiiIioIBUkRpa6D20UBJKII2UfX5/ZOGLECCJmZ3dzf26rlwkO7NzfzYhd2afmXlGjDEopZTyPj52B1BKKWUNbfBKKeWltMErpZSX0gavlFJeShu8Ukp5KT+7A1ysXLlypkaNGnbHUEopj7F27dokY0xEXsvcqsHXqFGD2NhYu2MopZTHEJH9V1qmQzRKKeWltMErpZSX0gavlFJeyq3G4POSlZXFoUOHyMjIsDuKKoTAwEAiIyPx9/e3O4pSxY7bN/hDhw4REhJCjRo1EBG746gCMMZw4sQJDh06RFRUlN1xlCp23H6IJiMjg/DwcG3uHkhECA8P13dfStnE7Rs8oM3dg+nPTin7uP0QjVLKvRzdtJP9X36POXOGErf0p2nvDvqH3E1pg1dK5YtxOFg18imaffUhFXOyABieHELGIT/+e1tTqoQF2ZxQXcojhmjUtcXHx9OoUSO7YygvZRwOVvW/i7ZfvMOWFp04vCaOtJOnuf6RoWw5fIaJd/+LhHVb7I6pLqF78H9TdnY2fn7F79tYXF93cTVp5X7iUwJwDBjBdT98jvjk7hsO7RBKizBfKr59G6dX/ET6to0EhYXYnFad51G/oa/8soWtCWeKdJvRlUvz0k0Nr7rON998w9tvv42IEBMTg6+vL4GBgaxfv5727dtz5513cv/995OWlkatWrWYMGECZcqU4YMPPuCTTz7Bz8+P6OhoJk+ezJIlS3jkkUeA3AOQS5cuJSTk8l+IwYMHM2zYMPr06QPA8OHD6du3Ly1btmTYsGGkpqYC8L///Y927dpd83Vu2bKFESNGkJmZicPh4Mcff6ROnTqXvbaJEycSHx/P3XffTVJSEhEREXz55ZdUq1aN4cOH/+V1P/jggzz44IMkJiYSHBzMZ599Rv369Qv6I1Bubk9iCmN+3U6rO0by3IjWiM9fx9sbNIpi04ef0fieQawadA9t5k61Kam6lKUNXkQeAe4FBPjMGPOelfWssGXLFsaMGcPy5cspV64cJ0+e5PHHH+fQoUMsX74cX19fYmJi+PDDD+ncuTMvvvgir7zyCu+99x5jx45l3759lChRgtOnTwPw9ttv89FHH9G+fXtSUlIIDAzMs+6gQYOYOnUqffr0ITMzkwULFjBu3DiMMcyfP5/AwEB27drF7bffnq8J2j755BMeeeQR7rjjDjIzM8nJycnztQE8/PDD3HXXXdx1111MmDCB0aNHM2PGDIC/vO7u3bvzySefUKdOHVatWsWoUaNYuHBhkXzflfs4eOtQ+kZE89RTL+Ljk/fB1MZ338bKWb/S9qev2D5jHvX793BxSpUnY4wlH0AjYDMQTO4fkt+B2ld7TosWLcyltm7detljrvTBBx+Y55577i+P3XXXXearr74yxhhz+vRpU7Vq1QvLdu/ebZo1a2aMMaZnz55m4MCBZuLEiebs2bPGGGPeeOMN07p1a/P++++bgwcPXrFuenq6qVq1qsnIyDAzZswwQ4YMuVBv6NChplGjRqZJkyYmKCjIGGPMvn37TMOGDa+4vUmTJpno6GgzduxYs3Pnziu+NmOMCQ8PN5mZmcYYYzIzM014ePhlr/vs2bMmMDDQNGnS5MJH/fr186xt989QFd6mr380BsyKB5655rpnE0+a4yFlzc5q9U1OVrYL0iljjAFizRV6qpUHWRsAq4wxacaYbGAJcIuF9VyqZMmS11xn9uzZPPjgg6xbt45WrVqRnZ3NM888w+eff056ejrt27dn+/bteT43MDCQLl26MHfuXKZMmcKgQYMAePfdd6lQoQIbN24kNjaWzMzMfOUdMmQIM2fOJCgoiBtvvLHQe9rnX7fD4SAsLIwNGzZc+Ni2bVuhtqncl98rL3M0rDxN3/zXNdctVa4MO157h7Gt/sH87cddkE5di5UNfjPQUUTCRSQYuBGoeulKIjJSRGJFJDYxMdHCOIXTrVs3pk2bxokTJwAuDGOcFxoaSpkyZfjjjz8AmDhxIp07d8bhcHDw4EG6du3Km2++SXJyMikpKezZs4fGjRvz9NNP06pVqys2eMgdpvnyyy/5448/6NWrFwDJyclUqlQJHx8fJk6cSE5OTr5ex969e6lZsyajR4+mX79+xMXFXfG1tWvXjsmTJwMwadIkOnbseNn2SpcuTVRUFNOmTQNy3wlu3LgxX1mUZ9g+/Tfq793E/hGjCAy59g4NQNuHhrG7TVc+WLj7/Dt5ZSPLxuCNMdtE5E1gHpAKbAAu60bGmPHAeICWLVu63f+Ihg0b8vzzz9O5c2d8fX1p1qzZZet8/fXXFw6y1qxZky+//JKcnByGDh1KcnIyxhhGjx5NWFgYL7zwAosWLcLHx4eGDRvSu3fvK9bu0aMHw4YNo1+/fgQEBAAwatQoBg4cyDfffEOvXr3y9U4CYOrUqUycOBF/f38qVqzIc889R9myZS97bV999RUffvghI0aM4K233rpwkDUvkyZN4oEHHmDMmDFkZWUxePBgmjRpkq88yv2lv/4myYGlaPzCo/l+jp+vD6PbVibh6ZeIC02kyV1e86bdI4mr/sqKyOvAIWPMx1dap2XLlubSA4bbtm2jQYMGVsdTFtKfoec5fiaDV4a/yo01QujzzrMFem5mWgZny1fiQN0Ymq1bYlFCdZ6IrDXGtMxrmaUXOolIeee/1cgdf//OynpKqaIxY8NhZtdtT/1nHy7wcwOCA9nZ9zZiNizj6KadFqRT+WX1law/ishW4BfgQWPMaYvreZxNmzbRtGnTv3y0adOm0NubO3fuZdsbMGBAESZW3s44HGR88BHdS2dRK6JUobZR47nHEGPYN/a9og2nCsTS8+CNMZcfnVN/0bhxYzZs2FBk2+vZsyc9e/Yssu2p4mf3r0sY/eO7rKoTAfQv1DYqxdRnQ5P21Jk5mcy0dwgIzvt6D2UtnYtGKfUXJz8ezzlff+qP/uff2o55YBQrIxuxav3uIkqmCsqjpipQSlnLkZ1DraXz2NK8I80rRfytbTW8ZxAjjoXT+WAW+lbeHroHr5S6YOesBZRLOYmjX/+/va0APx9ubFyJnUvXkna6aOeQUvmjDT4fSpX6vwNNc+bMoW7duuzfv5+XX36ZKlWqXDiY+cwzz1xxG1ebzrdLly75mk9GKavFL1hBhl8AdYYPKpLtDZFj/DruXraOm1gk21MFo0M0BbBgwQJGjx7N3LlzqV69OgCPPfYYTz75pM3JlCoa/6nRmWnvduKLKuWLZHvRfbtyLDQCvymT4dkHi2SbKv90Dz6fli5dyr333susWbOoVavWFdczxvDUU0/RqFEjGjduzJQpUy5bJz09ncGDB9OgQQMGDBhAenq6ldGVypfdx1PYm5hK5xZRRbZNHz9f9nXvS/SmlSQfcb+pSLyd5+3Bd+ly+WO33QajRkFaGtx44+XLhw/P/UhKgltv/euyxYuvWfLcuXP079+fxYsXXzbf+bvvvsu3334LwJtvvklKSgobNmxg48aNJCUl0apVKzp16vSX54wbN47g4GC2bdtGXFwczZs3v2YGpax2/OV/893sOdR4rGivPg27YxAB078k7quptNS9eJfSPfh88Pf3p127dnzxxReXLXvssccuzKbYs2dPli1bxu23346vry8VKlSgc+fOrFmz5i/PWbp0KUOHDgUgJiaGmJgYl7wOpa4mdOF8KmanUrlCWJFut+7N15NUqgzy889Ful11bZ63B3+1Pe7g4KsvL1cuX3vsl/Lx8WHq1Kl0796d119/neeee67A21DKnaWdSqbOro2s6z+MmkW8bR8/X6Y+/wETT5RgcXYOJfx8i7iCuhLdg8+n4OBgZs+ezaRJk/Lckz+vY8eOTJkyhZycHBITE1m6dCmtW7f+yzqdOnXiu+9yp+XZvHkzcXFxlmZX6lp2/TCHAEc2Jfv2smT7DQb04IhvMMv3nLBk+ypvnrcHb6OyZcvy22+/0alTJyIi8r4IZMCAAaxYsYImTZogIvznP/+hYsWKxMfHX1jngQceYMSIETRo0IAGDRrQokULF70CpfKWPutXzvn6U2dgHsewikC7WuEM3zyXtDcWwVdvW1JDXc5l0wXnh04X7J30Z+j+/jPkORqcOsRNv35jWY11ba4nctsGIk4fR3x08KCo2DZdsFLK/R1NzuDjau1JeOYlS+tk9+xN+bMn2LtghaV1PM23K/fz5LSNOBxFv7OtDV6pYm7dn5sITT9LhzrlLK1Tc2ju3Z2OT5thaR1PU+6FZ6g7aTw+PlLk2/aIBu9Ow0iqYPRn5/7K/fcNFn92Hw0qhFhbp24Ue6rUIXTx75bW8STZmVl0WPozTc4lWbJ9t2/wgYGBnDhxQhuFBzLGcOLECQIDdS5wd1Ypbg3x9Zvi42t9Ozja+QZOZRnOpGZYXssTxC9aQanMdHwvuRiyqFh6Fo2IPAb8EzDAJmCEMaZAP9nIyEgOHTpEYqJe5uyJAgMDiYyMtDuGuoKknfuomnSIw7cNc0k9/9de5bbxKxm39xS9G1dySU13lvTrAmoDkTf1sGT7ljV4EakCjAaijTHpIjIVGAx8VZDt+Pv7ExVVdHNjKKX+z/6f51IOKNv7epfUa169DCGBfizdkqANHvBfsZyjYeWp2KiOJdu3+j2ZHxAkIn5AMJBgcT2lVAFkL15Cmn8gNW/o4JJ6fr4+jI37kYdH9cU4HC6p6a6MMcSbEuxs082yGpY1eGPMYeBt4ABwBEg2xsy7dD0RGSkisSISq8MwSrnWp01v4pP7XsWvRIDLapZvWJfKp46yb9FKl9V0R/tPpPFEtwc48PJYy2pY1uBFpAzQD4gCKgMlRWTopesZY8YbY1oaY1pe6epQpVTRO52WycKcUAIG9Hdp3aqD+gFwfMYcl9Z1N7G7jwHQOqqsZTWsHKK5HthnjEk0xmQB04F2FtZTShXAjt+Xc8vmBbQq79qznCo2qsOBiKoELV3k0rruptrzT/LLt09QO6LUtVcuJCsb/AGgrYgEi4gA3YFtFtZTShWAY8pU/jPnfRpXKe3y2kdatafOtnVkphXf0yUrbYolJ6K8JRc4nWflGPwq4AdgHbmnSPoA462qp5QqmFLrY4mvUovgMqGuL377EN5tP4S4vcddX9sNJO2Op2riQTLaXGdpHUvPojHGvGSMqW+MaWSMGWaMOWdlPaVU/uRkZRO1ZwsnGjWzpX79gb34ou0tLD1SPPfg46fNBiD8JmumZz7P7a9kVUoVvYPL11EqMw2fttbuQV5JaJA/7cOEs7N+taW+3Ry/L+BMYClq9uhoaR1t8EoVQ0eX5s7oWLFnZ9syjNrwC89/+Dhnjxe/m4BMr9qcOQPvw9ff2ltyaINXqhiaEd2V6x//jsjWTWzLENK3F37GwZ5pxet0yYMn05hcsRkZD1h/A3Jt8EoVQ+sPnCayUS1bb7xRu98NpPuVIOO3ubZlsMOm31dQ68RB2tcKt7yWNnilipmzx5IYPf55ep/db2uOwFLB7KrXjIpr/rQ1h6uV/+C/TJn8HLXLW3f++3na4JUqZuJ/XUKf7ctoYO307/mS1rEzNY7Fk7Rzn91RXMI4HNTYuIr4mDYuefekDV6pYiZlyTIAqvfqYm8QIPS+e+hy76f8kWLtwUZ3cWD5OsqlnMTRtatL6mmDV6qYCdqwlv3lqxFayf65n+rF1CY5sgbLdp+0O4pLHJ2ee/575MC+LqmnDV6pYqbynq0k1m1kdwwAfHyEO7P2c907LxaL6YNLLPydQ+FVqNws2iX1tMErVYwcP3qCY8FhZLZqbXeUC9rnnOTWVTM5sGK93VEslZGVw8jrH2bWv95zWU1t8EoVI5tPZ3PT8Pfxe/hhu6NcUMU5XHH0x1k2J7HWqn0nOe5Xknq9rbn/al60wStVjGw6dAYRaFjZ9TNIXkmVFg1JKFuJgCWL7Y5iqZR3P+Ce9bO4rqb157+fpw1eqWKkzYuj+XT+B5Qs4V5nrRxq3o5aW9aQfS7T7iiWifnhK24+spFAf1+X1dQGr1QxUnNLLGUDXddg8su3x/UcL1mG7et22B3FEodjN1E16RDp3Xu4tK57/RlXSlkmcc9+yp9JYm+z5nZHuUzUAyNofqIqT6YH4h7n9xStQ99NpwoQefstLq2re/BKFRMJv+de4FS6fRubk1yubKkSRFcqzbJdiXZHsUTggnkcLBdJZOsYl9bVBq9UMZG2Yg0OhGrXd7A7Sp4e2L+MD5/oS/qpM3ZHKVJp57I4mZbFofbdXV7bsgYvIvVEZMNFH2dE5FGr6imlrm5TUARz2txIqXJl7I6Sp+oNaxKReord073rJiBLdiYxYuBL8PbbLq9t5T1ZdxhjmhpjmgItgDTgJ6vqKaWubkLVtsx/9DW7Y1xR7Vt6kenrR+qv8+yOUqQWro+nTLA/raLKury2q4ZougN7jDH2zk+qVDGVmJhMSuJJGlex4Qbb+RRcJpTdtRpTbtUfdkcpMplpGTx/f0/G7JiDn6/rR8RdVXEw8H1eC0RkpIjEikhsYqJ3HmBRym4J02ez+b3baJu42+4oV5XcrjO1D+3i1P4Eu6MUie3f/0xY+lmqdGhhS33LG7yIBAA3A9PyWm6MGW+MaWmMaRkRYf/sdkp5o7SVq3Eg1OjUyu4oVxU2eCCftBnI2j3esbOXPvVHUgOCqD/UtadHnueKPfjewDpjzDEX1FJK5SEwbiOHIyLd9gDreXWub8dHPe9lwWnPP8EvJyubWst/Z0fT9gSGlLQlgyu+i7dzheEZpZRrVNi3neO1Gtgd45r8fH3oUC2E5N8X2x3lb9v183zKpZzC0b+/bRksbfAiUhK4AZhuZR2l1JUlH0mk8qmjZDZ27UU2hTV88zw+Hjeaw2u32B3lb5mZGszY7vdQ757BtmWwtMEbY1KNMeHGmGQr6yilrmzX8VT+dcMD+PbtY3eUfKnonD74sAdPH5yd42DKgUwOjHiAkPKumz3yUp4/0KWUuqrNKYZvm/ehepe2dkfJl2rtmnM8JBy/Bb/bHaXQNv70Ox1Wz6V/dDlbc2iDV8rLnVm2gibnkigfUsLuKPkiPj7sb9GeWnErPXb6YMcHH/Da/E/oXNfeMwO1wSvl5Xp/9Cpj5n6EiNgdJd98buxNaEYKu2cvtDtKgaWfPkv0qoXs6NCDEiWDbc2iDV4pL5Z9LpNqCXtJrd/Q7igFUnvYP7ht2FvMCapqd5QC2zr+W0pmphM0/E67o2iDV8qbHVq5gRI5Wfg1b2Z3lAIJrRiOadeORbtP2R2lwHy//57jIeE0GNTX7ija4JXyZknLVgEQ3r61zUkK7uZS6Qz4+i0S93jOFFbHT5zF70gCe3r2x9ff/vspaYNXyotlr99Apq8fkdd51h48wHUR/ty9dibx33rOZTTT4o7R9673qPjef+yOAmiDV8qrfde2P6/c/xb+gZ5xBs3Fat3QgcSQsvj89pvdUfLFkZ3Dz8t20LZWOFFVXD81cF60wSvlpYwx/JkeSGbnrnZHKRTx8WFfi47U3rDcI06X3DLpZ6aP+Qf3B52wO8oF2uCV8lJJ+xO4eck0Wvqk2B2l0Pz69iE0I4WdM+bbHeWaMj/5lBwfX9r2d58/qNrglfJSR37/g5cWfEajTPfZoyyouncO5FipsuyI3Wp3lKs6tT+BxmsWsu36frbNHJkXbfBKeanU1WsBiOzkGVMU5KVURFmefW8270S0wBhjd5wr2vHafwnIyabik6PtjvIX2uCV8lIBmzdxNKw8oVXK2x3lb+nRuDIHT6Sxbb97vhPJzMik5tSv2RTdmhqd3et0VG3wSnmpcnu2cySqvt0x/rbra4ay6LP7SH7pVbuj5GnOtkTuvPVlMsa8YXeUy2iDV8oLZaSkUSnpMOeiG9kd5W8rVy6UjLLlKP/7r3ZHuYwxhgl/7iOzYSNa9HOfg6vnaYNXygvtOJVJk0cmk/LAQ3ZHKRLJvfpSK2E3Cevd62Dr9ulzuXvcvxhVvyQ+Pu43mZs2eKW80NYjZ0gPCKRu/ep2RykSVUcMAeDAF5NsTvJXGa+/Sde9a7mxfT27o+TJ6lv2hYnIDyKyXUS2ich1VtZTSuUqNeEznlg1hcgyQXZHKRJVWjRkT5U6lJ31k91RLti7YDnN1i1m663DCQ4rbXecPFk9G877wG/GmFtFJACwd3JkpYqJ2r/PpA645bBBYe0a9QQ/rTnAs4kp1IgoZXccTj/3IikBwUSPfcHuKFdk2R68iIQCnYAvAIwxmcaY01bVU0rlcmTnUPXgbpLrRdsdpUg1efBO5tVvx8y4I3ZHIX7xKpqvXsCmW+5069NQrRyiiQISgS9FZL2IfC4il13iJSIjRSRWRGITExMtjKNU8XBkw1ZKZaYhTZraHaVIVQoNok+pc/h8+D7G4bA1y6fbU5hw3UAavPmirTmuxcoG7wc0B8YZY5oBqcAzl65kjBlvjGlpjGkZEWHv/QuV8gbHl60GIKxdK5uTFL07U3fx0M8fsfu3P2zLsOPoWSbvz+DYC68RVq2SbTnyw8oGfwg4ZIxZ5fz6B3IbvlLKQscPHiMpOJRqndzrqsqiUO/BEWT6+nHyo09ty7B3+P10Orqd+zvVsi1DflnW4I0xR4GDInL+/KHugHudxKqUF5oW04PBr84gsLT9ByKLWmiV8sS1vYEGC2aSfvqsy+tv/uoHes/9jvuDkyhTMsDl9QvK6vPgHwYmiUgc0BR43eJ6ShV7246coUHlULtjWCb4oQcofS6VTe9+5tK6OVnZBD3/LAllKtLsTfc9c+ZiljZ4Y8wG5/h6jDGmvzHG8+6gq5QHST58nM/fuYfe8bF2R7FMg9v6sKdSTfat2ezSumvHfJB7Ne3TLxJYyjPO+NYrWZXyIocWr6BBYjyRESF2R7GM+PiweNIcno4ZyNaEMy6pefrgUeq8/So7ohrS/ImRLqlZFPLd4EWkg4iMcH4eISJR1sVSShXGWecc8JU7e+4c8Plxa5uaBAf4MnXGCpfUG7v8MJ+0vZWALz7Dx8/XJTWLQr4avIi8BDwNPOt8yB/41qpQSqnC8YmL40TJMMJre8ccNFcSGuzPWydX8vzDfUjYsM3SWiv3JDF5/VF8nnqSqK6eNdtKfvfgBwA3k3suO8aYBMB73wMq5aHK7t5GQvW6iHjPFAVX0nLUHRiEg/8aY1mNjDMpBPa4nkFH1zO6Wx3L6lglvw0+0+TeL8sA5HVFqlLKXlk5DtaH1+Bo+252R3GJCtF12NCpDzFzfyRpd7wlNTbefi9N925kWLdoggI8Z2jmvPw2+Kki8ikQJiL3Ar8Drj1HSSl1VXsTU3mq12hSvWQO+Pyo/Oar+Dpy2PvAk0W+7Q3jJtJmzmRW9r+LRnfeUuTbd4V8NXhjzNvkXon6I1APeNEY86GVwZRSBbPtQCIYQ4NK7jl1rRUiW8ew7qYh1Fk2jx3bDxTZdhPWb6X6kw+xt3Itmn37cZFt19Xye5C1JLDQGPMUuXvuQSLib2kypVSBVHxzDCvHDadmuHfMAZ9f9ce9Tf+Hv+DVZQnkjiT/PemZOfz+/Dv4Ohz4/zSdEiU945z3vOR3iGYpUEJEqgC/AcOAr6wKpZQquJLbt5ASGo6/v9W3eXAvYZUiuGdAK5bvSmTulAV/a1vZOQ4em7KBl2L6s3n2Yqq2jimilPbIb4MXY0wacAu5s0P+A2hoXSylVEEYY6i8fycnazewO4othrapzofrv6fDXf04tDquUNswDgdLb72XXUvW8K++DWnXrUURp3S9fDd45+327gBmOx/zvEPKSnmpE7v3E556GkeTJnZHsYWPj9D8Py+Q7etLdr9+pCQVbFaU7HOZrO41iG4zJvCyYyf3dPCO6zjz2+AfIXcu9+nGmC3Oq1gXWhdLKVUQCUtWAhDS2vP3OgurctMGHPz4S6oeO8D+9tfne7bJ1BOn2XTdDbSZ/wMrBo2kw+dvW5zUdfLb4NMAB3C7c2bImUBXy1IppQpki28on7a+hcgunnWlZVFrPHwg6//9PvV3bSCu040knE6/6vqblq3ndN1omqz/g1VPvMJ1kz9FfLxniq78Ho2ZBDwJbCa30Sul3MjywIqsG/Ag91XWu6K1fPYh1pcsyUfrTrL+3aU81rwM/WIqER5VFci9Z+2WPzcwPsGH39Yf4NMqdTnz8XjaDOprc/Kil98Gn2iM+cXSJEqpQstYt4GYurXtjuE2mo0ewStJqbw2aytn//MuZZZP5nDZimQGBBJx8hjVgWWPf8e93erR6tX5hAR651nfkp/zRkWkO3A7sAA4d/5xY8z0ogzTsmVLExvrvfNYK2WFjDMp+IeFsub2+2g7yXMvyrHKvmVrOfrFRAJ2bMcn8xyZFSri26E99R8cQanSnj/rioisNca0zGtZfvfgRwD1yZ1F8vwQjQGu2uBFJB44C+QA2VcKoZQqvANLV1PXOAhoobc8zktUhxZEdSieB5/z2+BbGWPqXXu1PHU1xiQV8rlKqWs4vWINAOU7trE5iXI3+T1cvFxEoi1NopQqFLNhI6kBQVRupr+i6q/y2+DbAhtEZIeIxInIJufpktdigHkislZE8rzPlYiMFJFYEYlNTEzMb26llFPIzq0cqFrHo+40pFwjv0M0vQq5/Q7GmMMiUh6YLyLbjTFLL17BGDMeGA+5B1kLWUepYskYw9gOw+gSFUrxnKRAXU2+GrwxZn9hNm6MOez897iI/AS0JnfiMqVUETh0Kp2lFerTs2cju6MoN2TZJVsiUlJEQs5/DvQg90IppVQRiV+2lut3raJhmQC7oyg3ZOU1uRWAZSKyEVgNzDbG/GZhPaWKHb9pUxk/fQx1Izx3znJlHcsmjjbG7AWK59R2SrlI4NbNHI6IpGqZULujKDfkPbPqKFUMVdi3neM169sdQ7kpbfBKeajkI4lUPnWUrEaN7Y6i3JQ2eKU81KGFywEIbtvK5iTKXWmDV8pDrawSTfd7xlGpbw+7oyg3pQ1eKQ+16WgKqTXrEFGxrN1RlJvSBq+Uh2o+4X0Gndxidwzlxiw7TVIpZZ3Uk8kMnf8NqyqVtjuKcmO6B6+UBzqwaAU+GIJa6wFWdWXa4JXyQMl/rgKgcrd2NidR7kwbvFIeyHf9Ok6UDCOiXpTdUZQb0wavlAcyx45zOKoB4qO/wurK9CCrUh4mIyuH2/u/wIPtqhJjdxjl1vTPv1IeZsfRs+Q4DNE1ytkdRbk5bfBKeZizX09k/PQxNArRX191dTpEo5SH8V+6lDYHN1O6aoTdUZSb010ApTxM2R2bORhVXw+wqmvS/yFKeZDM9HNUP7yblGg9vKquzfIGLyK+IrJeRGZZXUspb3dgyUoCcrLxb93S7ijKA7hiDP4RYBugk2Yo9TftPnCC5Mr1qdSji91RlAewdA9eRCKBPsDnVtZRqrhYWLYW997/AZWa6G361LVZPUTzHvD/AMeVVhCRkSISKyKxiYmJFsdRyrNt2n+SJpGhiIjdUZQHsKzBi0hf4LgxZu3V1jPGjDfGtDTGtIyI0NO+lLqSlKRT/PDsjdyxaZ7dUZSHsHIPvj1ws4jEA5OBbiLyrYX1lPJq++f/QcmsDCLq1bQ7ivIQljV4Y8yzxphIY0wNYDCw0Bgz1Kp6Snm7s0tzb7JdtUcnm5MoT6HnwSvlIQLWrSWhbCXKRkXaHUV5CJc0eGPMYmNMX1fUUspbVdq5iSN1GtkdQ3kQnYtGKQ+QmJzOxJieNO2pd3BS+adDNEp5gLiEM3x83W2E3f4Pu6MoD6INXikPsH/lBspknKVRFb0gXOWfNnilPEDrt/7FtB9fJDhAR1VV/mmDV8rNZZ/LJGrPFk7EtLA7ivIw2uCVcnPxi1dSMjMd3w4d7I6iPIw2eKXcXNK8xQBU6dPd3iDK42iDV8rN+a1cwfHS5ajYqK7dUZSH0SM2Srm5/7YZRLuON/Gw3qJPFZD+j1HKjR1JTmdFQAQl+/a2O4ryQNrglXJju2Yv5pbNC2hZIcjuKMoDaYNXyo0FfD+JMfM+pkHVMnZHUR5IG7xSbix8Yyz7ohriH1jC7ijKA2mDV8pNpSSdIurQLs40b2V3FOWhtMEr5ab2/PgrfsZBqZ432B1FeSht8Eq5qVPLV5Pp60ft/j3sjqI8lJ4Hr5Sb+m/LW5nYqAdfhIXYHUV5KMv24EUkUERWi8hGEdkiIq9YVUspb5OcnsWWhGQaNq1tdxTlwawcojkHdDPGNAGaAr1EpK2F9ZTyGrsn/cT/pr9Op1JZdkdRHsyyIRpjjAFSnF/6Oz+MVfWU8iZZv8yi295YaBxldxTlwSw9yCoiviKyATgOzDfGrMpjnZEiEisisYmJiVbGUcpjRMSuYHftxgSWCrY7ivJgljZ4Y0yOMaYpEAm0FpHLbglvjBlvjGlpjGkZERFhZRylPMLpA0eolbCbs2072h1FeTiXnCZpjDkNLAJ6uaKeUp5s3/Q5AJTpo6dHqr/HyrNoIkQkzPl5EHADsN2qekp5i81HzrIhsgE1b+xqdxTl4aw8D74S8LWI+JL7h2SqMWaWhfWU8njGGMaFNabx69/waXCg3XGUh7PyLJo4oJlV21fKG+3ed5SkE2fp0r2O3VGUF9CpCpRyIyfe/oB1Hw6hS4Sv3VGUF9AGr5QbKbVoPkllK1ApqordUZQX0LlolNtK2h3P/um/krVxM/779pBifHhn6PME+ApDF3xLed8cAtu1pfbgmwkpH2533L8tJekUdXduZF3/YdSwO4zyCtrglVvZffwsM9Yn0ODlJ+mzchblgBzx4WiZCuyr1YjQIH8ys3MI27mVlpuX4T9lPOce92d90/YEPPUE0bf1QUTsfhmFsnPSTzR3ZFOq/012R1FeQhu8sl3GmRQ2vvE/Qr+fyKB+L3A2OIRHmrRlZZNowm/qRfWu11ElOJAqQIfzTxq5mIwzKeyas4gz302l7sJfmDP2Y146WZYXb4omJjLMvhdUSI4fpnM6KIR6/7jR7ijKS2iDV7Y5tT+B7S++Sb0fv6ZNajJ7qtThpeahdLilK+VD+lzz+YGlSxE9+CYYfBMZZ1LwX7Gb/bFJvPT8BJ5JWkPMpPEEechUu5nZDt5q0Is+rTpzp96eTxURPciqXC4+KZWxXy2hRJ2aXPfNBxyo3Zgt3/1MzQPbueWu3pQPKfj534GlSzGoZ1MWPNGZkQHHaTNnMscaxBC/ZLUFr6Dordx7glVlo6j0zzvtjqK8iO7BK5fZPn0ucdPn83TVrvj7+NBwyMPE3H0bTTsV3T1HSwf60/uzsWxq34JKD99HUI8uxH00gZh/Di6yGlY48skEOicJHevobB6q6EjurL7uoWXLliY2NtbuGKoI5WRlEzduIkHvv0v9vZs4GRzKN98tZEi36ELtqRfE8W17OHtDb6on7GHRx99zw/23WVqvsBzZOZwsU54D0c1pvmq+3XGUhxGRtcaYlnkt0yEaZYnUc9nM+fRHjlSqTrNH7qb0qURWPvoSJQ7u59F+zS1v7gDlG9SiwoZVzOz3T+7fG8gnS/ZYXrMwtn4/k3IpJzEDb7E7ivIyOkSjitSxrbuY+ccOPjziT/DxFCaElWPt0y/QZPTdVC4R4PI8pcqV4eYfxrFo6ka+mLac+r9Mpsvbz7s8x9WkTfialIBgGt4/zO4oystog1d/W05WNpu/mIzjs89pvP4P6ldvQscxn3P3iNZEf3yX3fHw8/Xh3duasGjyh3SZ+SUr01Jo+/EbdscCIP30WaKXz2druxtoXbqU3XGUl9EGrwptX1Iqe155i5hvPqLJmSROlAxjzcAR1Hr+CT5q2sDueH/h5+tD16mfsLbLYdqOG8ua8hG0evlxu2Oxau4KGvoFEHzPcLujKC+kDV4VyOF1W9n/6Ve8Xft61p3IYuSWg5SvVpvDd79Bo/uHcl2Q+05x61cigMYLfmZTq840e/UpNpaPoMkoe4dFvk4LY9ezU1g6uLutOZR30gavrior4xy7Zswj+adfKP/nYmod3kUVIGZURW4c3J8bn/6QymU8576hAcGBRC35jX0t2uH32qvE9upNy5rlbMly+FAif2w/ysiudfHx09kjVdHT0yTVX5xLTWPf3D/YlJzDPN/yJK5Yy4xP7ifLx5ddtRpzpntPqt13F5XdbAimoE7uO8SIb9YSTxA/PtCO2uVdP/69YvB9VJ47E7+tW6hSqazL6yvvcLXTJC3bgxeRqsA3QAXAAOONMe9bVU8VXEZWDjuPnSVj/OeYdespvX0zNeO3UT8ni42Nb2DnsGdpd/11rG34NXX+cSPRFezZ07VC2ahIPnyoLIM+XMTaW4YT9uVblKsT5bL6GWdTqf/LFOLrN6OZNndlESuHaLKBJ4wx60QkBFgrIvONMVstrKkukZGVQ2Lcdk6uiyNj207M7t0EHojnQHBZHu08EoeBRePfoULKKQ5E1mb9TUMo0bkj3W+6gUG1qjm3EmPra7BKtfBgvupQhqqvzyahWxyB61dRqlwZl9Te9M5ntEpL5vAjD7ukniqeXDZEIyI/A/8zxlzxUj0dorm2jLOpnD1ynJTEUyRVqcHptCwCli6mxNbNmOPH8Tt2lBKJx0h3CPcNfoXTaVl89/1ztDsQB0CqfyDHIqpwsH5T1j79bxpUKk1D33Qi61YvtuPAGz/7nob3DWVrw9Y0WLMYf4sn+zIOB3ur1sMvJ4tqCXsRH73eUBWeLUM0lwSoQe79WVflsWwkMBKgWrVqly7Ol+Qz6ZjTpzHGYBwOHA4HGAeOkFAcgUGYjHQkMQljHJgcBxiDyckhO6ICjpIlMWfP4JuQkPu4I3cbGAcZVWvgCA7G5+Qp/A7udy53XPhIrxdNTlAwfseO4Ld7N47MTExm1oV/E67rzLkSwZTavpmwjWshKwuTnZ37b1YWsf3vIiOgBFX/XEjU6sX4pKXhm5GOb0Y6fhkZPP/gO6TmCHfM/pybV/1C8Ll0ArMzCQRK+/rT9cmfAPjv7PEM3LyQbPHhROlwksPKkV65Gn1jKlEhJJCMOi+zrXQJIppFE16zGjV9fKgJdC7cj9PrNLn3dtYcOEyrMU+xutdttFr4k6VNd+3nU2iZsJvYV96lujZ3ZSHLG7yIlAJ+BB41xpy5dLkxZjwwHnL34AtT45+Pfsq0Lx+77PGHb3qKX6I70y5+A99N+ddly0fc+hKLarWix84VjP/p35ctv23IWFZXbUT/LYt4b9Z/L1veZ/j7bKlQi6Hr5zBm3seXLX/83k+JL1uF+1b9wLOLv7ps+eig5pwsXZaHVq7hhtXzORcQyLkSgWQFBJFVIpDyJXzwLVUSn0YN2eV/DlOyFJQpg5Qti1+5snzTrxVhJQMoc09jkoMCCKkYQQU/Xyo4t9/mQiW9gfO1tHrtSVYcOED0lAl8MXEh/7zrekvqGGN4KbUyMSNe57WnR1lSQ6nzLB2iERF/YBYw1xjzzrXWL+wQzdTZa6g2fxaIIL4+gICPcLR1R1KrRhGcdIwqyxeCjy/monWSWrfnXMXKBB1PIHzd6ty9Nh8B8cFHhJOtriOrbDmCjh4mdPtmEB/EB/DxBR/hTPPWmJDSlDiWQKn9+/AtEYBPiQB8AvzxLRFATu26+AcH4p+WSkB6Kj4BAfiWCMA/MAC/EiXwKxGAj6/uwbkL43DwxmcLGL8vk9cHNGZIm8K9o7ya3+ISuP+79bxzWxNuaR5Z5NtXxc/Vhmgsa/CSe9+0r4GTxphH8/McHYNXdsvKcXDv12uI/mYcPQZ1p+kDRXch1LmUVA7WbcL0jgN5/Ls38NM/7qoI2DWbZHtgGNBNRDY4P/ReZMqt+fv68NHAhgzYt4p6o+9l+4x5Rbbt9Y+8QO0je+jds4U2d+USeqGTUnlI2nOAc63bUjo1mcPf/Uj9W/7ejTjil6ymUvcObGnekearFxRRSqV0PnilCqxcrWr4LlnC6ZCyVBs8gC3fzyz0tjLOpJBzx1BSA4OpPuWrogup1DVog1fqCio2qkPQn0s5Fl6Rr79fysyNCQXehjGGb175jOoJezj49v8Ij6pqQVKl8qaTjSl1FeXqRiFb4tg3dRNTv1/PqcV/cvu9NxEQfO1ZMx0Ow4szN/Otf11KTpzHHXfojJHKtXQPXqlrCC8bwqR/tmVUNWHwE0M4WDeGXXOWXPU5Z44lEdu+N3snz+T+zrUYMqSbi9Iq9X+0wSuVDwF+Pvy/UTey9d3PCDudSJ0+XVjfvDObJkwl81zWhfVO7jvEykdfIrN2XZqvms9DFbN4ulc9cs8aVsq19CwapQrozLEktjwzhujJXxCSkUqTJ34gNCKMh2Z+xODl0wHYVisG3/fepW5f3XNX1rLlQqfC0AavPEnGmRS2TPuVBVUacyQ5gzaLf6Z69lnK3zaAmt2v00nElEtog1dKKS+l58ErpVQxpA1eKaW8lDZ4pZTyUtrglVLKS2mDV0opL6UNXimlvJQ2eKWU8lLa4JVSyku51YVOIpII7C/k08sBSUUYp6horoLRXAWjuQrGG3NVN8ZE5LXArRr83yEisVe6mstOmqtgNFfBaK6CKW65dIhGKaW8lDZ4pZTyUt7U4MfbHeAKNFfBaK6C0VwFU6xyec0YvFJKqb/ypj14pZRSF9EGr5RSXsqrGryIvCYicSKyQUTmiUhluzMBiMhbIrLdme0nEQmzOxOAiPxDRLaIiENEbD11TER6icgOEdktIs/YmeViIjJBRI6LyGa7s1xMRKqKyCIR2er8GT5idyYAEQkUkdUistGZ6xW7M50nIr4isl5EZtmd5WIiEi8im5x9q0jveORVDR54yxgTY4xpCswCXrQ5z3nzgUbGmBhgJ/CszXnO2wzcAiy1M4SI+AIfAb2BaOB2EYm2M9NFvgJ62R0iD9nAE8aYaKAt8KCbfM/OAd2MMU2ApkAvEWlrb6QLHgG22R3iCroaY5oW9bnwXtXgjTFnLvqyJOAWR5CNMfOMMdnOL1cCkXbmOc8Ys80Ys8PuHEBrYLcxZq8xJhOYDPSzORMAxpilwEm7c1zKGHPEGLPO+flZchtXFXtTgcmV4vzS3/lh+++hiEQCfYDP7c7iSl7V4AFE5N8ichC4A/fZg7/Y3cCvdodwM1WAgxd9fQg3aFaeQkRqAM2AVTZHAS4MhWwAjgPzjTHukOs94P8BDptz5MUA80RkrYiMLMoNe1yDF5HfRWRzHh/9AIwxzxtjqgKTgIfcJZdznefJfWs9yZ1yKc8lIqWAH4FHL3kHaxtjTI5zmDQSaC0ijezMIyJ9gePGmLV25riKDsaY5uQOUT4oIp2KasN+RbUhVzHGXJ/PVScBc4CXLIxzwbVyichwoC/Q3bjw4oMCfL/sdBioetHXkc7H1FWIiD+5zX2SMWa63XkuZYw5LSKLyD2GYedB6vbAzSJyIxAIlBaRb40xQ23MdIEx5rDz3+Mi8hO5Q5ZFclzM4/bgr0ZE6lz0ZT9gu11ZLiYivch9e3izMSbN7jxuaA1QR0SiRCQAGAzMtDmTWxMRAb4Athlj3rE7z3kiEnH+LDERCQJuwObfQ2PMs8aYSGNMDXL/by10l+YuIiVFJOT850APivCPoVc1eGCsc/ghjtxvlFucOgb8DwgB5jtPhfrE7kAAIjJARA4B1wGzRWSuHTmcB6AfAuaSe7BwqjFmix1ZLiUi3wMrgHoickhE7rE7k1N7YBjQzfl/aoNzD9VulYBFzt/BNeSOwbvVaYlupgKwTEQ2AquB2caY34pq4zpVgVJKeSlv24NXSinlpA1eKaW8lDZ4pZTyUtrglVLKS2mDV0opL6UNXrklEUm59lpXfO5DzlkpjYiUu+hxEZEPnMviRKT5RcsqnZ9lUES6FNWMgyKyOD8zdTpnFCx3jXV+F5EyRZFLFQ/a4JU3+hO4Hth/yeO9gTrOj5HAuIuWPQ585pJ0hTcRGGV3COU5tMErt+bc637LeQHbJhEZ5HzcR0Q+ltx59ueLyBwRuRXAGLPeGBOfx+b6Ad84ZzxcCYSJSCXnsoHAZReYiEhrEVnhnEd8uYjUcz4+XERmOGvHO981PO5cb6WIlL1oM8OcFyJtFpHWzueHS+49C7aIyOeAXFRzhnPiqS2XTD41E7i9sN9LVfxog1fu7hZy5xVvQu5e+VvOpnwLUIPc+eOHkXs17rXkOWuliEQBp4wx5/J4znagozGmGbmzk75+0bJGzhytgH8Dac71VgB3XrResHPyrVHABOdjLwHLjDENgZ+Aahetf7cxpgXQEhgtIuEAxphTQInzXyt1LR432ZgqdjoA3xtjcoBjIrKE3IbaAZhmjHEAR52TWhVWJSDxCstCga+d8xwZcuc3P2+Rcy72syKSDPzifHwTEHPRet9D7tzyIlLaOVdLJ3L/OGCMmS0ipy5af7SIDHB+XpXcIaUTzq+PA5Uv+lqpK9I9eFWcXGnWynRyZxnMy2vkNvJGwE2XrHfxHr/joq8d/HXn6dL5QK44P4iIdCH3ncp1zrsirb+kZqAzr1LXpA1eubs/gEGSexOJCHL3fFeTeyB1oHMsvgLQJR/bmgnc6RzXbwskG2OOkHsbxRpXeE4o/zd18fBCvobzxw06OGsmkzsd7BDn472B82fHhJI7XJQmIvXJvR0fzvUEqAjEFzKHKma0wSt39xMQB2wEFgL/zxhzlNx50A8BW4FvgXVAMoCIjHbOkhkJxDkPYkLu/QH2ArvJPWNmFIAxJhXYIyK186j/H+ANEVlP4Yc0M5zP/wQ4PxvlK0AnEdlC7lDNAefjvwF+IrINGEvuLR7PawGsvOj2j0pdlc4mqTyWiJQyxqQ4DzquBto7m39htjUAaGGM+VeRhixCIvI+MNMYs8DuLMoz6EFW5clmOQ9YBgCvFba5AxhjfvKAs1M2a3NXBaF78Eop5aV0DF4ppbyUNnillPJS2uCVUspLaYNXSikvpQ1eKaW81P8HIT4w0H032FcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_47_0.png"
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import KFold\n",
    "from sklearn.linear_model import Ridge\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "\n",
    "# A seed just to ensure that the random numbers are the same for every run.\n",
    "# Useful for eventual debugging.\n",
    "np.random.seed(3155)\n",
    "\n",
    "# Generate the data.\n",
    "nsamples = 100\n",
    "x = np.random.randn(nsamples)\n",
    "y = 3*x**2 + np.random.randn(nsamples)\n",
    "\n",
    "## Cross-validation on Ridge regression using KFold only\n",
    "\n",
    "# Decide degree on polynomial to fit\n",
    "poly = PolynomialFeatures(degree = 6)\n",
    "\n",
    "# Decide which values of lambda to use\n",
    "nlambdas = 500\n",
    "lambdas = np.logspace(-3, 5, nlambdas)\n",
    "\n",
    "# Initialize a KFold instance\n",
    "k = 5\n",
    "kfold = KFold(n_splits = k)\n",
    "\n",
    "# Perform the cross-validation to estimate MSE\n",
    "scores_KFold = np.zeros((nlambdas, k))\n",
    "\n",
    "i = 0\n",
    "for lmb in lambdas:\n",
    "    ridge = Ridge(alpha = lmb)\n",
    "    j = 0\n",
    "    for train_inds, test_inds in kfold.split(x):\n",
    "        xtrain = x[train_inds]\n",
    "        ytrain = y[train_inds]\n",
    "\n",
    "        xtest = x[test_inds]\n",
    "        ytest = y[test_inds]\n",
    "\n",
    "        Xtrain = poly.fit_transform(xtrain[:, np.newaxis])\n",
    "        ridge.fit(Xtrain, ytrain[:, np.newaxis])\n",
    "\n",
    "        Xtest = poly.fit_transform(xtest[:, np.newaxis])\n",
    "        ypred = ridge.predict(Xtest)\n",
    "\n",
    "        scores_KFold[i,j] = np.sum((ypred - ytest[:, np.newaxis])**2)/np.size(ypred)\n",
    "\n",
    "        j += 1\n",
    "    i += 1\n",
    "\n",
    "\n",
    "estimated_mse_KFold = np.mean(scores_KFold, axis = 1)\n",
    "\n",
    "## Cross-validation using cross_val_score from sklearn along with KFold\n",
    "\n",
    "# kfold is an instance initialized above as:\n",
    "# kfold = KFold(n_splits = k)\n",
    "\n",
    "estimated_mse_sklearn = np.zeros(nlambdas)\n",
    "i = 0\n",
    "for lmb in lambdas:\n",
    "    ridge = Ridge(alpha = lmb)\n",
    "\n",
    "    X = poly.fit_transform(x[:, np.newaxis])\n",
    "    estimated_mse_folds = cross_val_score(ridge, X, y[:, np.newaxis], scoring='neg_mean_squared_error', cv=kfold)\n",
    "\n",
    "    # cross_val_score return an array containing the estimated negative mse for every fold.\n",
    "    # we have to the the mean of every array in order to get an estimate of the mse of the model\n",
    "    estimated_mse_sklearn[i] = np.mean(-estimated_mse_folds)\n",
    "\n",
    "    i += 1\n",
    "\n",
    "## Plot and compare the slightly different ways to perform cross-validation\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(np.log10(lambdas), estimated_mse_sklearn, label = 'cross_val_score')\n",
    "plt.plot(np.log10(lambdas), estimated_mse_KFold, 'r--', label = 'KFold')\n",
    "\n",
    "plt.xlabel('log10(lambda)')\n",
    "plt.ylabel('mse')\n",
    "\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "More examples of the application of cross-validation follow here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-7-49b0ef2e51e2>:63: RuntimeWarning: divide by zero encountered in log10\n",
      "  plt.plot(polynomial, np.log10(estimated_mse_sklearn), label='Test Error')\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEGCAYAAABsLkJ6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAuP0lEQVR4nO3deXhU5dn48e+dHZKwhSxAgACGfQkQkUVUxLUuoHXHDbWWWpeuVl/9Ve1b+2prq7VqLSqiVhHFva5gjQjIkrAFwqZsmQBJCGYn6zy/P+aAISaZSTKTk5m5P9fFNXPOnJlzHwbOPed5nnM/YoxBKaVU8AmxOwCllFL20ASglFJBShOAUkoFKU0ASikVpDQBKKVUkAqzOwBP9O7d26SkpNgdhlJK+ZWsrKzDxpj45l73iwSQkpJCZmam3WEopZRfEZF9Lb2uTUBKKRWkNAEopVSQ0gSglFJByi/6AJpSW1uLw+GgqqrK7lD8WlRUFMnJyYSHh9sdilKqg/ltAnA4HMTGxpKSkoKI2B2OXzLGUFRUhMPhYNCgQXaHo5TqYH7bBFRVVUVcXJye/NtBRIiLi9OrKKWClN8mAEBP/l6gf4dKBS+/TgBKKRWoSo7W8uD7WymrqvXZPjQBtFFRURFpaWmkpaWRlJREv379ji/X1NS4fX9GRgarVq1q8rWFCxcSHx9//PPS0tLIycnx9iEopTqprH3f8aO/f8W/V+9j7Z4jPtuP33YC2y0uLo6NGzcC8OCDDxITE8NvfvMbj9+fkZFBTEwMU6dObfL1K6+8kqeeeqrZ99fV1REWFtbssqfvU0p1Hk6n4dnl3/LXz3bSp3sUb86bwvgBPX22P70C8KKsrCxOP/10Jk6cyLnnnsvBgwcBePLJJxk5ciRjx47lqquuYu/evTz77LM8/vjjpKWl8dVXX3n0+RkZGUyfPp2LL76YkSNH/mC5qqqKuXPnMmbMGMaPH88XX3wBuK4oLr74Ys4880xmzpzps+NXSrVdQVkV1y9Yy58/2cF5o5P46K7pPj35Q4BcATz0wVZyDpR69TNH9u3GAxeN8nh7Ywx33HEH7733HvHx8SxevJj77ruPBQsW8Mgjj7Bnzx4iIyMpLi6mR48ezJs3r8WrhsWLF7NixYrjy19//TUA69evZ8uWLQwaNIiMjIwTlv/6178iImRnZ7N9+3bOOeccdu7cefx9mzdvplevXu34W1FK+ULGjgJ+/cYmKmrqeOTSMVx5cv8OGaAREAmgM6iurmbLli2cffbZANTX19OnTx8Axo4dy5w5c5g9ezazZ8/26POaawKaNGnSCWP2Gy6vWLGCO+64A4Dhw4czcODA4wng7LPP1pO/Up1MTZ2Txz7bwfzluxmWGMvr10wmNTG2w/YfEAmgNb/UfcUYw6hRo47/Um/oww8/ZPny5XzwwQc8/PDDZGdnt3k/0dHRLS57+j6llL32F1Vyx6L1bHKUcO3kAdx/wUiiwkM7NAbtA/CSyMhICgsLjyeA2tpatm7ditPpJDc3lxkzZvDoo49SUlJCeXk5sbGxlJWVeTWG6dOn8+qrrwKwc+dO9u/fz7Bhw7y6D6VU+723MY8fPfkVew5X8M85E/jj7DEdfvIHTQBeExISwpIlS/jd737HuHHjSEtLY9WqVdTX13Pttdce75i988476dGjBxdddBHvvPNOs53AixcvPmEYaHNDRhu67bbbcDqdjBkzhiuvvJKFCxcSGRnpi8NVSrVBTZ2T3y3ZzF2vb2RYUiwf3TWd88f0sS0eMcbYtnNPpaenm8YTwmzbto0RI0bYFFFg0b9LpXyvqraen7+6ns+3F/DzGUP45VlDCQv17W9wEckyxqQ393pA9AEopVRndrSmnltfyeSrXYf54+zRXDt5oN0hAZoAlFLKp8qr67h54TrW7T3CXy4by+Xp/e0O6Ti/TgDGGC1m1k7+0ASolL8qrarlxgVr2eQo4YmrxnPxuL52h3QCv+0EjoqKoqioSE9g7XBsPoCoqCi7Q1Eq4HxXUcOc59aQnVfC09dM6HQnf/DjK4Dk5GQcDgeFhYV2h+LXjs0IppTynsPl1Vz7/Bp2H67gX9dN5MzhiXaH1CS/TQDh4eE6i5VSqtPJL61izvNrcHxXyYIbTubU1N52h9Qsv00ASinV2eQVH2XOc6spLKvmpbmTOGVwnN0htUgTgFJKecH+okqufm41pVW1vHLLKUzwcSVPb9AEoFQ7VNfVExnW8bfwq87l28Jy5jy3hqq6ehb9ZDKj+3W3OySPaAJQqo0+zj7IHYs2cN8FI7hxaooOSQ4gNXVOvthRwO7CCsqraymrqqO8qo6y6jrKqmopr677fl1VHTX1TuKiI3j91skMT+pmd/ge0wSgVBst33WYOqfhoQ9yyHaU8PAlY+gSoVcD/mzbwVLeyMzlvY0HOFLhmto1NESIiQwjNirs+GNCbBSDe1vrosLoFhXORWP7MiCuq81H0DqaAJRqo82OYqadFMcpg+J4fNlOth8q41/XTaR/L/86CQS74soa3t90gDcyc9mSV0p4qHD2yEQuT+/PKYN60SU8NGCv7nyWAERkAXAhUGCMGd3otV8DjwHxxpjDvopBKV+pqq1nx6Eybj1tMHfOTGVMv+7c+foGLnpqBf+4ejzTU+PtDlG1oN5pWPnNYd7IzOWznHxq6pyM6NONBy4ayay0fvSKjrA7xA7hyyuAhcBTwMsNV4pIf+AcYL8P962UT20/VEad0zA22dXZN2N4Ah/cfio/fSWLGxas5bfnDmfe6YMD9pdjQ/VOQ2hI5z/Oypo6duWXs2xbPkuyHBwsqaJ7l3CumTSAyyYm+03HrTf5LAEYY5aLSEoTLz0O3A2856t9K+Vr2Y5iAMYk9zi+LqV3NG/fNpW739rMo59sJzuvmD9fNo6YyMBtac3YUcDPX13P7Wem8rMzhtgdDuDqwN1zuIId+WXsPFTmeswvY/+RSowBETgtNZ77LxjJWSMTgnoUV4f+yxSRWUCeMWaTu19GInIrcCvAgAEDOiA6pTy32VFCXHQEfbufWEcpOjKMp64ez7jk7jzy8XZ25Zfzr+smMjg+xqZIfSfnQCk/f3U9Bnj0k+1U1dbzi7NSO/yqx/FdJe9uyGNHfjk7D5Wx+3A5tfWuGmGhIcKg3tGM7tudS8cnMywphvEDepLYTetfQQcmABHpCvwPruYft4wx84H54JoQxoehKdVqmx0ljEnu3uTJTkS49bQhjOrbndtfW8+sp1by+JVpnDWyc9aDaYv80ipufmkdsVHhvHXbVB5fupO/f76L6jonvztvWIclgW0HS7nuhbUcLq8muWcXhiXGMnNEAsOSYhmaGMvg+Oig/oXvTkdeAQwBBgHHfv0nA+tFZJIx5lAHxqFUu1TW1LGroIxzRye1uN20k3rzwR2nMu/fWdzyciY3Tk1hypA4hiXG0r9XV79oN29KZU0dN7+0jpKjtbw5bwr9enThzz8eS2RYCM9++S1VtfU8cNFInyeBjbnF3LBgLV3CQ1n2q9M4KSHWp/sLRB2WAIwx2UDCsWUR2Quk6ygg5W9yDpTiNDDWg07D5J5dWTJvKr9/bwsLV+1l4aq9AESGhZCaGMPQhFiGJsUyNDGGoYmx9OvRpVN3HNc7DXcu2kjOgVKeuz6dUX1dfwchIcIfZ48mMiyUBSv3UFPv5I+zRhPioyS3ZncRNy1cR6+YCF67ZbIOvW0jXw4DXQScAfQWEQfwgDHmBV/tT6mOstlRAsCYZM9GjUSFh/Lny8bx+4tGsSu/jF355ezMd3VOrvq2iLc35B3fNjoilJMSY0mIjaSmzkl1Xb316Dz+2Hhdcs8unDMqiXNHJTK+f0+fnXQB/vTRNpZty+ehi0cxc8SJTVoiwv+7cASR4SH8M+Nbqmud/PmysV6/0vlyZyE/fSWTfj268Ootk0nqru35beXLUUBXu3k9xVf7VsqXsvNKSOwW2eqOxJjIMMYP6Mn4RkXCSo7Wsiu/jJ1WYtiZX0bukUoiw0OJDA0hOjKMnl1DiAwPISI0hMiw0OPPw8NCyDlQyosr9zB/+W7iYyM5e2Qi545KYsrgOCLCvDfn0ytf7+WFFXu4cWoKN0xNaXIbEeHuc4cRFRbK48t2UlPv5G9XjCPcS5Off7LFVX4jNSGWV26eRFxMpFc+N1gF7vg0pXxkk6OYMf16eO3zuncJJz2lF+kpvdr8GaVVtXyxvYBPtx7i3Q15vLZmP7FRYcwcnsC5o5I4fVg8XSPa/t/9i+0FPPD+Vs4akcD/u3Bki9uKCHedlUpkeAiPfLydmrp6nrx6fLs7Y9/Z4OA3b25mbHJ3Fs6dRPcu4e36PKUJQKlWKauqZXdhBZek9bM7lBN0iwpnVlo/ZqX1o6q2nhW7DvPp1kMs25bPuxsPEBkWwvTUeC5O68s5IxOJCvf8ZJxzoJTbX1vPiD7d+PtV4z1u0pl3+hAiw0J46IMc5r2SxT+vndiq/Tb06pp93P/uFqYMjuO569OJDuB7KzqS/i0q1Qpb8koBz9v/7RAVHspZIxM5a2QidfVO1u39jk+3HuIzKyF0iwpjVlo/rkjvz+h+3VrsdG443POFG05u9Yl37rRBRIaFct+72dzyUibzr5/Y6iuR+cu/5U8fbefM4Qk8M2dCm5OI+iFNAEq1QnZeMQBj/KRsQFhoCFOGxDFlSBy/v3Akq3cX8UZmLm9k5vLK6n0MT4rl8vT+zE7r+4P29IrqOm5auI7So7W8OW9qmztbrzllABFhIdy9ZBNznl/DrHF9GRgXzYC4riT37NJs05AxhieW7eLvn+/igjF9ePzKNK/2aSgQYzr/PVbp6ekmMzPT7jCU4vbX1rNhfzEr7znT7lDapeRoLf/ZfIA3Mh1syi0mPFSYOTyRK05O5rTUeESEn76SyX+3F/DCDSczY3iC+w9144NNB7jvnWxKq+qOrxOBvt270L9XFwb2ciWFgXFdGdgrmvc25vH8ij1cNjGZR3/s/dFEwUBEsowx6c29rlcASrXCZkfJ8QJw/qx7l3DmnDKQOacMZGd+GW9m5vL2+jw+2XqIhNhIUhNjWPlNEX+YNcorJ3+Ai8b15cKxfThcXsP+IxXsK6pkX1El+49Usq+ogs+353O4vOaE99wwZSAPXDTKp0Nbg5kmAKU8VFxZw/4jlVw9KbBqUw1NjOW+C0Zy93nD+e/2At7MzOWLHYXccuogrp+S4tV9iQjxsZHEx0YyceAPRz1VVNdZCaGSsBBh5oiETn1jnL/TBKCUh7LzXDeABcIVQFPCQ0M4d1QS545K4mhNPVHhHd/eHh0Zxog+3RjRx3+mVfRnmgCU8tCxO4BH9w3MBNCQTm0ZHLRLXSkPZTtKSInrSveuegOSCgyaAJTyUHZeyQkTwCjl7zQBKOWBw+XV5BUfZVyAtv+r4KQJQCkPZB+rAOonN4Ap5QlNAEp5YLOjBBEYpQlABRBNAEp5IDuvmCHxMQE9wbsKPpoAlPLAZkeJRzOAKeVPNAEo5cahkioKyqoD9gYwFbw0ASjlxmZHMYAOAVUBRxOAUm5k55UQGiKM1PIEKsBoAlDKjc2OElITYrQ8ggo4mgCUaoExhuy8wCgBrVRjmgCUaoHju6McqahhrLb/qwCkCUCpFgR6CWgV3DQBKNWCzY4SwkOFYUmxdoeilNdpAlCqBdl5xQxP6tbsxOVK+TNNAEo1wxjDZkcJY7T5RwUoTQBKNWNvUSVlVXVaAloFLJ8lABFZICIFIrKlwbq/iMh2EdksIu+ISA9f7V+p9jp+B3C/HrbGoZSv+PIKYCFwXqN1S4HRxpixwE7gXh/uX6l2yXaUEBkWQmpijN2hKOUTPksAxpjlwJFG6z4zxtRZi6uBZF/tX6n22pxXwsi+3QgP1ZZSFZjs/Jd9E/Bxcy+KyK0ikikimYWFhR0YllJQ7zRszdMS0Cqw2ZIAROQ+oA54tbltjDHzjTHpxpj0+Pj4jgtOKWB3YTkVNfV6B7AKaB0+vZGI3AhcCMw0xpiO3r9Sntjs0DuAVeDr0AQgIucBdwOnG2MqO3LfSrVGdl4JXSNCGRyvHcAqcPlyGOgi4GtgmIg4RORm4CkgFlgqIhtF5Flf7V+p9tjsKGZ03+6EhojdoSjlMz67AjDGXN3E6hd8tT+lWrL3cAX9enbxaERPXb2TrQdKuXbywA6ITCn76Pg2FfA2O4qZ8dcMZjyWwetr91NT52xx+5355VTXObX9XwU8TQAq4C1YsYfoiDDiYiK55+1sZjyWwaIWEkF2XjGAjgBSAU8TgApoBaVVfJh9kMsmJvPubVN5ce7JxMdGcq+VCF5b88NEsNlRQmxUGAN7dbUpaqU6hiYAFdD+vXofdU7DjVNTEBFmDEvgndumstBKBP/zjisR/Hv1Pqrr6gHXCKAx/boToh3AKsBpAlABq6q2nlfX7OfMYQmk9I4+vl5EOMNKBC/dNImEbpHc/+4WZvwlg5e/3sv2g2VaAloFhQ6/EUypjvKfzQcpqqhh7rRBTb4uIpw+NJ7TUnvz1a7DPLFsJ79/bysA47T9XwUBTQAqIBljeHHlHlITYph2UlyL24oIpw2NZ3pqb1Z8c5jPtxVw+lAtP6ICX4sJQER6efAZTmNMsXfCUco71u39jq0HSnn4ktGIeNaWLyJMT41neqqe/FVwcHcFcMD609L/oFBggNciUsoLXly5h+5dwrl0vFYcV6o57hLANmPM+JY2EJENXoxHqXZzfFfJp1sP8ZPTBtMlQidzV6o57kYBTfHgMzzZRqkO88rqfYgI109JsTsUpTo1dwlg6rEnInLCUAoRuRTAGFPlg7iUapPKmjpeX5vLuaMS6deji93hKNWpuUsAjzV4/laj1+73cixKtds7G/IoOVrb7NBPpdT33CUAaeZ5U8tK2coYw8KVexnVtxvpA3vaHY5SnZ67BGCaed7UslK2WvlNEbsKypk7bZDHQz+VCmbuRgENFpH3cf3aP/Yca1mvsVWn8uLKPfSOieCicX3sDkUpv+AuAcxq8PyxRq81XlbKNnsPV/DfHQXccWYqkWE69FMpT7SYAIwxXzZcFpFwYDSQZ4wp8GVgSrXGwlV7CQsRrj1F70lUylMt9gGIyLMiMsp63h3YBLwMbBCRpqZ8VKrDlVXVsiTLwQVj+pDQLcrucJTyG+46gacbY7Zaz+cCO40xY4CJwN0+jUwpDy3JclBeXadDP5VqJXcJoKbB87OBdwGMMYd8FZBSreF0Gl5atZcJA3owrn8Pu8NRyq+4SwDFInKhiIwHpgGfAIhIGKC3WSrbfbGjgL1FlfrrX6k2cDcK6KfAk0AS8IsGv/xnAh/6MjClPPHiyr0kdYvivNFJdoeilN9xNwpoJ3BeE+s/BT71VVBKeWJnfhkrvjnMb88dRniozm6qVGu5mxDmyZZeN8bc6d1wlPLcwlV7iQwL4epJOvRTqbZw1wQ0D9gCvIH7iWGU6jClVbW8vd7B7LR+9IqOsDscpfySuwTQB7gcuBKoAxYDS3QKSGW3/2w6SFWtk2v0xi+l2qzFhlNjTJEx5lljzAxc9wH0AHJE5Dp3HywiC0SkQES2NFjXS0SWisgu61FLNqo2WZKVS2pCDGOTu9sdilJ+y6OeMxGZANwFXAt8DGR58LaF/LAD+R7gc2NMKvC5taxUq3xTUM76/cVcnp6sVT+Vagd3ncB/AC4AtgGvA/caY+o8+WBjzHIRSWm0ehZwhvX8JSAD+J3n4SoFb613EBoizB7fz+5QlPJr7voA7gf2AOOsP3+yfnEJYIwxY1u5v0RjzEHr+SEgsbkNReRW4FaAAQO0nVe51DsNb693cMbQeBJite6PUu3hLgH47PZKY4wRkWYnlTHGzAfmA6Snp+vkMwqA5bsKyS+t5qGLk+0ORSm/5+5GsH1e3l++iPQxxhwUkT6AlpRWrbIky0HPruGcObzZi0ellIfclYP+j7sP8GSbBt4HbrCe3wC814r3qiBXXFnD0q35zErrR0SY3vmrVHu5awI6tcE0kE0RYGSTL4gswtXh21tEHMADwCPAGyJyM7APuKLVEaug9cGmA9TUO7k8XZt/lPKG1kwJ2ZyaplYaY5qbMGamB5+p1A+8meVgRJ9ujOqrY/+V8gaPpoQUkUTg2Ji7PGNMvq8DU6qhHYfK2Owo4fcXNnnBqZRqA3f3AaQBzwLdgTxrdbKIFAM/M8Zs8Gl0SlmWZOUSFiLMSutrdyhKBQx3TUALgZ8aY9Y0XCkik63XxvkmLKW+V1vv5J0NecwckUBcTKTd4SgVMNwNpYhufPIHMMasBqJ9E5JSJ8rYUcjh8houn9jf7lCUCijurgA+FpEPgZeBXGtdf+B6rOkhlfK1JVm59I6J4PRh8XaHolRAcdcJfKeInI9rNNDxTmDgaWPMR74OTqmi8mo+31bA3GkpOuuXUl7m7goAY8zHuCqAKtXh3tt4gDqn4TJt/lHK69r8k0pE5nszEKWa8maWg7HJ3RmWFGt3KEoFHHfDQHs19xLwI++Ho9T3tuSVsO1gKf87a5TdoSgVkNw1ARXiKtnQcNYNYy0n+CoopcBV+C0iNISLxunYf6V8wV0C2A3MNMbsb/yCiOQ2sb1SXlFT5+S9jXmcPSqRHl110nelfMFdH8ATQHPz9v7Zu6Eo9b3Pt+XzXWUtl0/Uwm9K+Yq7YaBPt/DaP7wfjlIuS7IcJHaLZHqqjv1XylfcDgMFEJFLm1hdAmQbY3RSF+VVBWVVZOws5NbTBhMaopO+K+UrHiUA4GZgCvCFtXwGkAUMEpE/GGNe8UFsKki9uyGPeqfhMm3+UcqnPE0AYcCIY2WgrfLQLwOnAMsBTQDKK4wxvJnpYMKAHgyJj7E7HKUCmqc3gvVvNAdAgbXuCFDr/bBUsNrkKGFXQTmXp+udv0r5mqdXABnW3L9vWsuXWeuigWJfBKaC05uZuUSFh3DB2D52h6JUwPM0AfwcuBQ41Vp+CXjLGGOAGb4ITAUXYwzPfrmbRWv3c8n4ZLpFhdsdklIBz6MEYIwxIrIC1/y/BlhrnfyVarfSqlp+++YmPt2azwVj+vCQln5QqkN4Ogz0CuAvQAauMhD/EJHfGmOW+DA2FQR2HCpj3r+z2H+kkvsvGMHNpw5CRId+KtURPG0Cug84+diYfxGJB5YBmgBUm723MY973somJiqM1245hVMGx9kdklJBxdMEENLohq8i2lFKWgW3mjonf/poGwtX7eXklJ48fc0EErpF2R2WUkHH0wTwiYh8Ciyylq8EdEYw1Wr5pVXc9up6svZ9x03TBnHvj4brTF9K2cTTTuDfisiPgWnWqvnGmHd8F5YKRKt3F3H7axuorKnjyavHc7GWeVbKVp5eAWCMeQt4y4exqABljOH5r/bwyCfbGdirK6/95BSGJuoMX0rZzd2MYGW4hn3+4CVco0O7tWWnIvJL4Bbrs7OBucaYqrZ8lur8Hv5wG8+v2MN5o5L4y+VjidUx/kp1Cu7KQXv9Z5qI9APuBEYaY46KyBvAVcBCb+9L2a+6rp5Fa/dz4dg+/OPq8TrEU6lOxK7etzCgi4iEAV2BAzbFoXzs62+LqKip59IJ/fTkr1Qn0+EJwBiTBzwG7AcOAiXGmM8abycit4pIpohkFhYWdnSYykuW5uTTNSKUqUN62x2KUqqRDk8AItITmAUMAvoC0SJybePtjDHzjTHpxpj0+HidFcofOZ2GZdvyOS01nqjwULvDUUo1YkcT0FnAHmNMoTGmFngbmGpDHMrHsvNKyC+t5qyRiXaHopRqgh0JYD8wWUS6iqtReCawzYY4lI8t25ZPiMCZwxPsDkUp1QQ7+gDW4KohtB7XENAQYH5Hx6F8b2lOPukpvegVHWF3KEqpJtgyCsgY84AxZrgxZrQx5jpjTLUdcSjfyT1SyfZDZZw9Qpt/lOqstAiL8omlOa4ZRM/W9n+lOi1NAMonlubkk5oQQ0rvaLtDUUo1QxOA8rriyhrW7j2iv/6V6uQ0ASivy9hRSL3T6PBPpTo5TQDK65bm5BMfG0lacg+7Q1FKtUATgPKq6rp6MnYUcNaIBEJCtPaPUp2ZJgDlVat3H6Gipl7b/5XyA5oAlFctzTlEl3At/qaUP9AEoLzGGMOynAJOG9pbi78p5Qc0ASivyc4r4VBpFWePTLI7FKWUBzQBKK9ZlqPF35TyJ5oAlNd8lpNP+kAt/qaUv9AEoLziePE3Hf2jlN/QBKC8Ytk2V/E3vftXKf+hCUB5xdKcfE5KiGGQFn9Tym9oAlDtVlJZy5o9WvxNKX+jCUC1W8bOAuqdRhOAUn5GE4Bqt89y8ukdo8XflPI3mgBUu1TX1fPljkIt/qaUH9IEoNpl9e4jlFfXafOPUn5IE4Bql2U5+XQJD2XaSVr8TSl/owlAtZkxhmXb8pmeqsXflPJHmgBUm23JK+VgSZU2/yjlpzQBqDZbuk2LvynlzzQBqDZbmpPPxIE9iYuJtDsUpVQbaAJQbZJ7pJJtB0u1+UcpPxZmdwCq88h2lPD8it0IEBoSQmiI6zEsRAi1/oSFCCEhwrcF5QA6+YtSfsyWBCAiPYDngdGAAW4yxnxtRyzqe3/6aBsbcr8jsVsUdfWGeqeh3rge6+qdOA3UOZ2u9U7DpJReWvxNKT9m1xXA34FPjDGXiUgE0NWmOJQl50ApX+8u4p7zhzPv9CF2h6OU6gAdngBEpDtwGnAjgDGmBqjp6DjUiRas3EOX8FCuPnmA3aEopTqIHZ3Ag4BC4EUR2SAiz4vID9oRRORWEckUkczCwsKOjzKIFJRV8f7GA1yenkz3ruF2h6OU6iB2JIAwYALwT2PMeKACuKfxRsaY+caYdGNMenx8fEfHGFT+vXo/tU4nc6cNsjsUpVQHsqMPwAE4jDFrrOUlNJEAvGH17iJyj1QyqHc0A+Oi6R0TgYhWrGyoqraeV1fvY+bwBO3QVSrIdHgCMMYcEpFcERlmjNkBzARyfLGvt9c7eCPTcXw5JjKMgXFdSYmLPv6Y0jualLiuxMdGBmVyeG9jHkUVNdykv/6VCjp2jQK6A3jVGgG0G5jri508fMkYbjvjJPYUVbDvcAV7iyrZW1RBzsFSPt16iDqnOb5tl/BQpg6J457zh5OaGOuLcDodYwwLVuxleFIsU4bE2R2OUqqD2ZIAjDEbgXRf7yc8NMT1C793NAw78bW6eicHiqtcyaGogt2FFby93sF5f/+K6yYP5BdnpdKja4SvQ7TVym+K2JFfxl8uGxuUVz9KBbugvRM4LDSEAXFdGRDXFXB1Mt85M5W/Ld3By1/v5d2NefzyrKHMOWUAYaGBWTHjhRW76R0TycVpfe0ORSllg8A8s7VRr+gI/jh7DB/dNZ1RfbvxwPtbOf/vX7F8Z+ANQ/2moJwvdhRy3eSBRIZpLX+lgpEmgCYMT+rGv28+hfnXTaSm3sn1C9Zy88J17C4stzs0r3lx5R4iwkKYM1lv/FIqWGkCaIaIcM6oJD775Wncc/5w1uw5wrlPLOeP/8mh5Git3eG1y3cVNby13sHstL701lLOSgUtTQBuRIaFMu/0IXzxmzO4dHwyL6zcw5mPZfDZ1kN2h9Zmi9btp6rWyU2n6tBPpYKZJgAPxcdG8uhlY/ng9lPp0yOKu17fyK78MrvDarXaeicvr9rHqSf1ZnhSN7vDUUrZSBNAK43u150FN5xM14hQbn9tA1W19XaH1CofZR/kUGkVN+uvf6WCniaANkjoFsXfrkxjR34ZD33gk5uYfcIYwwsr9jA4PprTh2p9JaWCnSaANjp9aDzzTh/CorX7+WDTAbvD8Ujmvu/Y7Chh7rRBhITojV9KBTtNAO3w63OGMmFAD+59O5t9RRV2h+PWghV76N4lnB9P6Gd3KEqpTkATQDuEh4bw5NXjCRG4/bUNVNd1TH+AMYatB0qoqXN6/J7cI5V8uvUQ15wygK4RQXsDuFKqAU0A7ZTcsyt/uXwc2XklPPrxjg7Z57+W7+aCJ1cw+f8+5+EPc/jWgxvUFq7aS4gIN0xJ8X2ASim/oD8FveDcUUncODWFBSv3MGVIHGePTPTZvjJ2FPDoJ9uZMSyeyLBQXly5l+e+2sOklF5cNak/PxrTh6jwE0s7lFXVsnhdLheM7UNS9yifxaaU8i+aALzk3h8NJ3PfEX7z5iY+ums6/Xp08fo+dheWc8eiDYxI6sbTcybQNSKMgrIq3srKY/G6/fzqjU08+P5WLhnfj6smDWBEH9c4/zcyHZRX1+nQT6XUCcQY434rm6Wnp5vMzEy7w3Br7+EKLvzHCoYnxfL6rZO9WkW0rKqWS55ZxZGKGt6/fRrJPbue8Loxhq93F7F4XS4fbzlETZ2Tcf17cNXJ/Xkm4xsSY6NY8rOpXotHKdX5iUiWMabZ0vvaB+BFKb2jefiS0WTu+47Hl+302uc6nYZfLt7InsMVPH3NhB+c/MFVu2jqkN78/arxrLl3Jr+/cCRHa+q49+1sco8c1V//Sqkf0CYgL5uV1o+vvy3imYxvmTw4jump7b/h6ollO1m2rYCHLh7l0cxdPaMjuOnUQcydlsKG3GJyDpRyzqikdsehlAosegXgAw9cNIrUhBh+uXgjBWVV7fqsj7MP8uR/v+GK9GSunzKwVe8VESYM6Mm1kwcSqjd+KaUa0QTgA10iQnnqmgmUV9fxi9c3Uu9sWz/L9kOl/PrNTYwf0IP/nT1ap21USnmVJgAfGZoYy0MXj2LVt0Xc8tI6NuYWt+r931XU8JOXM4mNCuNf107UWbuUUl6nfQA+dEV6f45U1PLsl98y++mVTBkcx8/OGML01N4t/pqvq3dy+6L15JdUs/ink0nopmP3lVLep1cAPiQi/OyMIay850zuv2AEuw+Xc/2CtVz01Ao+3Hyw2aah//t4Oyu/KeLhS0YzfkDPDo5aKRUsNAF0gJjIMG6ZPpjld8/g0R+PobK6np+/tp6z/vYlr6/df0INobeyHLywYg83Tk3h8vT+NkatlAp0eiOYDeqdhs+2HuKZjG/JzishITaSW6YPYnTf7ty4cB0TB/Tk5ZsnEe7FG8mUUsHH3Y1g2gdgg9AQ4fwxfThvdBIrvynin19+w58+2g5Acs8uPD1ngp78lVI+pwnARiLCqam9OTW1N5tyi1mS5eC6KQPpFR1hd2hKqSCgCaCTGNe/B+P697A7DKVUELGtnUFEQkVkg4j8x64YlFIqmNnZ0HwXsM3G/SulVFCzJQGISDJwAfC8HftXSill3xXAE8DdQLOT2orIrSKSKSKZhYWFHRaYUkoFiw5PACJyIVBgjMlqaTtjzHxjTLoxJj0+vv0llZVSSp3IjiuAacDFIrIXeB04U0T+bUMcSikV1Do8ARhj7jXGJBtjUoCrgP8aY67t6DiUUirY6e2mSikVpPyiFpCIFAL7GqzqDRy2KRxfC9Rj0+PyP4F6bMF0XAONMc12ovpFAmhMRDJbKnDkzwL12PS4/E+gHpse1/e0CUgppYKUJgCllApS/poA5tsdgA8F6rHpcfmfQD02PS6LX/YBKKWUaj9/vQJQSinVTpoAlFIqSPldAhCR80Rkh4h8IyL32B2Pt4jIXhHJFpGNIuLXEyCLyAIRKRCRLQ3W9RKRpSKyy3rsaWeMbdHMcT0oInnW97ZRRH5kZ4xtISL9ReQLEckRka0icpe13q+/sxaOKxC+sygRWSsim6xje8haP0hE1ljnx8Ui0uL0gn7VByAiocBO4GzAAawDrjbG5NgamBdYtZHSjTF+f4OKiJwGlAMvG2NGW+v+DBwxxjxiJe6expjf2RlnazVzXA8C5caYx+yMrT1EpA/QxxizXkRigSxgNnAjfvydtXBcV+D/35kA0caYchEJB1bgmmPlV8DbxpjXReRZYJMx5p/NfY6/XQFMAr4xxuw2xtTgKiY3y+aYVCPGmOXAkUarZwEvWc9fwvUf0a80c1x+zxhz0Biz3npehmuipn74+XfWwnH5PeNSbi2GW38McCawxFrv9jvztwTQD8htsOwgQL5QXF/eZyKSJSK32h2MDyQaYw5azw8BiXYG42W3i8hmq4nIr5pJGhORFGA8sIYA+s4aHRcEwHdmTau7ESgAlgLfAsXGmDprE7fnR39LAIHsVGPMBOB84OdWc0NAMq52R/9pe2zZP4EhQBpwEPirrdG0g4jEAG8BvzDGlDZ8zZ+/syaOKyC+M2NMvTEmDUjG1ToyvLWf4W8JIA/o32A52Vrn94wxedZjAfAOri80kORbbbLH2mYLbI7HK4wx+dZ/RCfwHH76vVntyG8Brxpj3rZW+/131tRxBcp3dowxphj4ApgC9BCRMOslt+dHf0sA64BUq6c7Atd8Au/bHFO7iUi01UmFiEQD5wBbWn6X33kfuMF6fgPwno2xeM2xE6TlEvzwe7M6FF8Athlj/tbgJb/+zpo7rgD5zuJFpIf1vAuugTHbcCWCy6zN3H5nfjUKCMAasvUEEAosMMY8bG9E7Scig3H96gcIA17z5+MSkUXAGbjK0+YDDwDvAm8AA3CV9r7CGONXHarNHNcZuJoSDLAX+GmDdnO/ICKnAl8B2Xw/T/f/4Gov99vvrIXjuhr//87G4urkDcX1Q/4NY8wfrHPJ60AvYANwrTGmutnP8bcEoJRSyjv8rQlIKaWUl2gCUEqpIKUJQCmlgpQmAKWUClKaAJRSKkhpAlCdhojUW9UZt4jImyLStYVtbxSRpzoyvgb7/oOInOVmm4UicpmbbVIaVhZVqqNpAlCdyVFjTJpVabMGmGd3QE0xxvzeGLPM7jgasirlKtUqmgBUZ/UVcJJVk/5dq3DXausGmONEJFZE9li3/CMi3Y4ti0iGiDxq1U3fKSLTrW2iRORFcc2/sEFEZljrb7T2tVRc8zPcLiK/srZZLSK9rO2O/7oXkd+LyDrrqmW+dfdps0RkolXDfRPw8wbrQ0XkL9ZnbRaRn1rrQ0TkGRHZbsX1UYN977WObz1wuYicIyJfi8h66woqpsE+v7QKDX7a6E5YFcQ0AahOx6plcj6uOzgfAjYYY8biuovz5YbbWmV+M4ALrFVX4aqHXmsthxljJgG/wHXnLrhOvMYYMwbXXaEviUiU9dpo4FLgZOBhoNIYMx74Gri+iXCfMsacbF21dAEudHN4LwJ3GGPGNVp/M1BijDnZ2vdPRGSQFUsKMBK4Dle9l4aKrCKCy4D7gbOs5UzgV1Zi/AdwmTFmIrDAOi6lCHO/iVIdpotV3hZcVwAv4CpH8GMAY8x/RSRORLo1et/zwN24yk3MBX7S4LVjhc2ycJ1IAU7FdVLEGLNdRPYBQ63XvrCSSpmIlAAfWOuzgROuPiwzRORuoCuu2++3NnjPCazaLT2seQUAXsGV6MBV/2lsg36D7kCqFeubVuGyQyLyRaOPXWw9TsaVJFZaFyERuJLWMFxJbam1PhRXBUylNAGoTuWoVd72ODctKgAYY1ZaHapnAKHGmIYdq8fqoNTj2b/3hnVTnA2WnY3fb101PINrJrdccc0OFkXbCK4rg08b7cPddIUVDd6/1BhzdaP3jwG2GmMaXzkopU1AqtP7CpgDYJ3gDzeuVW95GXgNVxNLaz5zKK5iZzvaENuxk/1hq729xVE/VtneYqtIGcdisHwK/KxBX8ZQcVWGXQn82OoLSMRVfK4pq4FpInKS9f5o69h2APEiMsVaHy4io1p5nCpA6RWA6uweBBaIyGagku/LEzf2KvBHYJEHn/kM8E8RyQbqgBuNMdWeXG00ZIwpFpHncJUTPoSrXLk7c3EdjwE+a7D+eVxNVOutjuRCXNP5vQXMBHJwzYa3HihpIpZCEbkRWCQikdbq+40xO61mpSdFpDuu//NP4GqqUkFOq4GqgGCd5GYZY66zOxZvE5EYa/LvOGAtMM0Yc8juuJT/0ysA5fdE5B+4OlPdtZf7q/9YHcgRwP/qyV95i14BKKVUkNJOYKWUClKaAJRSKkhpAlBKqSClCUAppYKUJgCllApS/x+qr1vTxriA8wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_49_1.png"
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Common imports\n",
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.linear_model import LinearRegression, Ridge, Lasso\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.model_selection import KFold\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "\n",
    "# Where to save the figures and data files\n",
    "PROJECT_ROOT_DIR = \"Results\"\n",
    "FIGURE_ID = \"Results/FigureFiles\"\n",
    "DATA_ID = \"DataFiles/\"\n",
    "\n",
    "if not os.path.exists(PROJECT_ROOT_DIR):\n",
    "    os.mkdir(PROJECT_ROOT_DIR)\n",
    "\n",
    "if not os.path.exists(FIGURE_ID):\n",
    "    os.makedirs(FIGURE_ID)\n",
    "\n",
    "if not os.path.exists(DATA_ID):\n",
    "    os.makedirs(DATA_ID)\n",
    "\n",
    "def image_path(fig_id):\n",
    "    return os.path.join(FIGURE_ID, fig_id)\n",
    "\n",
    "def data_path(dat_id):\n",
    "    return os.path.join(DATA_ID, dat_id)\n",
    "\n",
    "def save_fig(fig_id):\n",
    "    plt.savefig(image_path(fig_id) + \".png\", format='png')\n",
    "\n",
    "infile = open(data_path(\"EoS.csv\"),'r')\n",
    "\n",
    "# Read the EoS data as  csv file and organize the data into two arrays with density and energies\n",
    "EoS = pd.read_csv(infile, names=('Density', 'Energy'))\n",
    "EoS['Energy'] = pd.to_numeric(EoS['Energy'], errors='coerce')\n",
    "EoS = EoS.dropna()\n",
    "Energies = EoS['Energy']\n",
    "Density = EoS['Density']\n",
    "#  The design matrix now as function of various polytrops\n",
    "\n",
    "Maxpolydegree = 30\n",
    "X = np.zeros((len(Density),Maxpolydegree))\n",
    "X[:,0] = 1.0\n",
    "estimated_mse_sklearn = np.zeros(Maxpolydegree)\n",
    "polynomial = np.zeros(Maxpolydegree)\n",
    "k =5\n",
    "kfold = KFold(n_splits = k)\n",
    "\n",
    "for polydegree in range(1, Maxpolydegree):\n",
    "    polynomial[polydegree] = polydegree\n",
    "    for degree in range(polydegree):\n",
    "        X[:,degree] = Density**(degree/3.0)\n",
    "        OLS = LinearRegression()\n",
    "# loop over trials in order to estimate the expectation value of the MSE\n",
    "    estimated_mse_folds = cross_val_score(OLS, X, Energies, scoring='neg_mean_squared_error', cv=kfold)\n",
    "#[:, np.newaxis]\n",
    "    estimated_mse_sklearn[polydegree] = np.mean(-estimated_mse_folds)\n",
    "\n",
    "plt.plot(polynomial, np.log10(estimated_mse_sklearn), label='Test Error')\n",
    "plt.xlabel('Polynomial degree')\n",
    "plt.ylabel('log10[MSE]')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAon0lEQVR4nO3deXxV9Z3/8dcnOzsBwhoQ0Kgs4hbFimLdKtYFa+uIC0XbxzitWpxxZqrTzuho6+/XUX8zTltbtR1spVaq09rSSqUuVFAUAdllCZsSEJKwhATI/vn9cU/wEm4IwVzOXd7Px+M+OPv93ITknfP9nvM95u6IiIi0lBF2ASIikpgUECIiEpMCQkREYlJAiIhITAoIERGJKSvsAjpKnz59fOjQoWGXISKSVBYvXlzh7gWx1qVMQAwdOpRFixaFXYaISFIxs49aW6cmJhERiUkBISIiMSkgREQkppTpg4ilvr6e0tJSampqwi5FjlFeXh6FhYVkZ2eHXYpI2knpgCgtLaVbt24MHToUMwu7HGknd2fnzp2UlpYybNiwsMsRSTsp3cRUU1ND7969FQ5Jyszo3bu3zgBFQpLSAQEoHJKcvn8i4UnpJiYRkVSwv66BygP17D3QQFVNPXtrItN7a+qpqmkgv3MON48d0uHvq4AQEQlJU5NTXl3Lx7v2s2XXfkp3H6CsqoaKqjrKq2upqK6lvKqW/XWNRzzOmUN6KiAkfjZv3szVV1/NypUrwy5FJOU0NDaxeec+Vn9SxdrtVazZXsXGimpKdx+grqHpkG17ds6moGsufbrmcnphT/p0zaVPtxzyO+fQPS+bbnlZdO+UTffg3255WeRmZcalbgVEyBoaGsjKSr9vQ7p+bkkPFdW1LNq8m8Uf7WLRR7tZtW3vwSDIzDCG9enCyX27cdmIfgzO70Rhr84Mzu9MYX4n8rLj88v+WKTNT+hDf1zFh9v2dugxRw7szoPXjGpzu+eee47HH38cM2PMmDFkZmaSl5fHkiVLGDduHF/96lf5xje+wf79+znxxBOZNm0a+fn5/PCHP+Spp54iKyuLkSNHMmPGDN566y3uueceINKBO3fuXLp163bYe06aNInJkydz1VVXAXDbbbdx9dVXU1xczOTJk9m3bx8AP/7xjzn//PPb/AyrVq3i9ttvp66ujqamJn77299SVFR02GebPn06mzdv5mtf+xoVFRUUFBTw7LPPMmTIEG677bZDPvddd93FXXfdRXl5OZ07d+ZnP/sZp556anu+BSIJoa6hiUWbdzFnbRlz1pazvqwagJysDMYM6sGUz53AiAHdOaV/N04s6JpQIXAkaRMQYVm1ahXf//73mT9/Pn369GHXrl3ce++9lJaWMn/+fDIzMxkzZgw/+tGPuOiii3jggQd46KGHeOKJJ/jBD37Apk2byM3NZc+ePQA8/vjjPPnkk4wbN47q6mry8vJivu+NN97Iiy++yFVXXUVdXR1vvPEGP/3pT3F3XnvtNfLy8igpKeGmm246qkEOn3rqKe655x5uueUW6urqaGxsjPnZAL71rW8xZcoUpkyZwrRp05g6dSq///3vAQ753JdeeilPPfUURUVFLFiwgDvvvJM333yzQ77uIvHW0NjEOxt28oclW/nLhzuorm0gJzODscN78ZWzCzlnaD6jB/WIW/PP8ZA2AXE0f+nHw5tvvskNN9xAnz59AOjVqxcAN9xwA5mZmVRWVrJnzx4uuugiAKZMmcINN9wAwJgxY7jlllu47rrruO666wAYN24c9957L7fccgvXX389hYWFMd/3yiuv5J577qG2tpZXX32V8ePH06lTJyorK7n77rtZunQpmZmZrFu37qg+x+c+9zkeeeQRSktLuf766ykqKmr1s7377rv87ne/A2Dy5Ml8+9vfPnic5s9dXV3N/PnzD35WgNra2qOqRSRMmyv2Mf29j/jD0m1UVNfSLS+Lq04bwKUj+jLupD50yU2dX6up80mSTJcuXdrc5pVXXmHu3Ln88Y9/5JFHHmHFihXcf//9XHXVVcyaNYtx48Yxe/bsmM0yeXl5fP7zn2f27Nn85je/YdKkSQD813/9F/369WPZsmU0NTW1egbS0s0338zYsWN55ZVX+OIXv8jTTz/dvg8caP7cTU1N9OzZk6VLlx7TcUSOJ3fn3Q07mfbOJt5YU0amGZeO6MuXzhzE50/pmzRNRu2V8jfKhe2SSy7hpZdeYufOnQAHm2Ga9ejRg/z8fObNmwfA9OnTueiii2hqamLLli1cfPHF/Md//AeVlZVUV1ezYcMGTjvtNO677z7OOecc1qxZ0+p733jjjTz77LPMmzePCRMmAFBZWcmAAQPIyMhg+vTpNDYe+fK5Zhs3bmT48OFMnTqViRMnsnz58lY/2/nnn8+MGTMAeP7557nwwgsPO1737t0ZNmwYL730EhD5AVy2bNlR1SJyPC3YuJMbn36Pm3++gA8+3sO3Lj6J+fdfwtOTi5kwekDKhgPoDCLuRo0axXe/+10uuugiMjMzOfPMMw/b5pe//OXBTurhw4fz7LPP0tjYyK233kplZSXuztSpU+nZsyf/9m//xpw5c8jIyGDUqFFceeWVrb73F77wBSZPnszEiRPJyckB4M477+TLX/4yzz33HBMmTDiqMxmAF198kenTp5OdnU3//v35zne+Q69evQ77bL/4xS/40Y9+xO23385jjz12sJM6lueff55vfvObfP/736e+vp5JkyZx+umnH1U9IvG2vqyKh/+0mrnryunbLZeHJ47ib4oHp3QgtGTuHnYNHaK4uNhbdrauXr2aESNGhFSRdBR9H+V4OlDXyI/eLOFn8zbSOSeLuy4+kcnnDaVTTmoGg5ktdvfiWOt0BiEiEnhv407+6aVllO4+wJfPKuQ7XzyV3l1zwy4rNAqIJLdixQomT558yLLc3FwWLFhwTMebPXs299133yHLhg0bxssvv3zMNYokuvrGJp54fR0/+esGTujVmRl3nMd5w3uHXVboUj4g3D2lRwQ97bTTOvRKoCuuuIIrrriiw473WaVKE6gkrvKqWr75q8Us+mg3f1NcyIPXjEqpS1U/i7hexWRmE8xsrZmtN7P7Y6z/hpmtMLOlZva2mY0Mlg81swPB8qVm9tSxvH9eXh47d+7UL5kk1fzAoKO9FFekvVZurWTij99m5bZK/nvSGTz6ldMVDlHi9pUws0zgSeByoBRYaGYz3f3DqM1+7e5PBdtfC/wnMCFYt8Hdz/gsNRQWFlJaWkp5eflnOYyEqPmRoyIdbc7aMr75q8X06pzD/37jfEYP6hF2SQknnlF5LrDe3TcCmNkMYCJwMCDcPXpwpC5Ah/6pn52drUdVishh/rhsG//wm6Wc0r8bv7j9XAq6pW9H9JHEs4lpELAlar40WHYIM7vLzDYAjwJTo1YNM7MlZvaWmR1+p1Vk3zvMbJGZLdJZgogcjRcXbWHqjCWcNSSfF+44T+FwBKHfSe3uT7r7icB9wL8Giz8Bhrj7mcC9wK/NrHuMfZ9x92J3Ly4oKDh+RYtIUvrjsm3c99vlXHBSH375tXPpnpcddkkJLZ4BsRUYHDVfGCxrzQzgOgB3r3X3ncH0YmADcHJ8yhSRdPDmmh38w2+Wcs4JvXhmcnHK3vjWkeIZEAuBIjMbZmY5wCRgZvQGZlYUNXsVUBIsLwg6uTGz4UARsDGOtYpIClteuoc7n/+AEQO68/PbFA5HK26d1O7eYGZ3A7OBTGCau68ys4eBRe4+E7jbzC4D6oHdwJRg9/HAw2ZWDzQB33D3XYe/i4jIkW2vrOFvn1tE7y65PHv7OWpWaoeUHotJRNLbgbpGbnzmXTaUVfPbO8/n1P6HdWWmPY3FJCJp6YE/rGTF1kqemVyscDgGoV/FJCISD79fspWXFpdy98UncfnIfmGXk5QUECKScjZV7OO7L6/gnKH53HNpUds7SEwKCBFJKQ2NTUx9YQlZmRn896QzycrUr7ljpT4IEUkpz8zbyIqtlfzklrMY2LNT2OUkNUWriKSM9WXVPPF6CVeO7s8XTxsQdjlJTwEhIimhscm577fL6ZSdyUMTR4VdTkpQQIhISpix8GMWf7SbB64eSd9ueoZIR1BAiEjSq9xfz+Oz1zJ2WC+uP+uwQaPlGCkgRCTp/dfr66g8UM+/XzsqpR8xfLwpIEQkqa3dXsX09z7i5rFDGDFAd0t3JAWEiCS177/yIV1zs/jHy08Ju5SUo4AQkaQ1f0MF80oq+NYlJ5HfJSfsclKOAkJEkpK78/jstQzokcet550QdjkpSQEhIknpzTVlfPDxHqZeWkReth4AFA8KCBFJOk1NzmOz1zK0d2e+cnZh2OWkLAWEiCSd2au2s2Z7FX9/2clkazC+uNFXVkSSirvz07c2MLR3Z645fWDY5aQ0BYSIJJV31u9keWklf3fRiWRm6Ka4eFJAiEhS+elb6+nbLVdDahwHCggRSRrLtuzhnfU7+foFw8jN0pVL8aaAEJGk8czcjXTPy+LmsUPCLiUtKCBEJCls23OAV1dt56Zzh9AtLzvsctKCAkJEksKv3vsId9dd08eRAkJEEl5NfSMzFm7h0hH9GNyrc9jlpA0FhIgkvD8u28aufXXcdv7QsEtJKwoIEUlo7s4v5m+mqG9Xzj+xd9jlpBUFhIgktGWllazatpevnj9UT4s7zhQQIpLQfrPwYzplZ3LdGRpW43iLa0CY2QQzW2tm683s/hjrv2FmK8xsqZm9bWYjo9b9S7DfWjO7Ip51ikhi2lfbwMyl27hqzABd2hqCuAWEmWUCTwJXAiOBm6IDIPBrdz/N3c8AHgX+M9h3JDAJGAVMAH4SHE9E0sgrKz5hX10jk84ZHHYpaSmeZxDnAuvdfaO71wEzgInRG7j73qjZLoAH0xOBGe5e6+6bgPXB8UQkjfxm4RaGF3Th7BPywy4lLcUzIAYBW6LmS4NlhzCzu8xsA5EziKnt3PcOM1tkZovKy8s7rHARCV/JjioWf7SbSecMVud0SELvpHb3J939ROA+4F/bue8z7l7s7sUFBQXxKVBEQvHS4lKyMozrz9IT48ISz4DYCkQ3HBYGy1ozA7juGPcVkRTS2OTMXLqNz59SQJ+uuWGXk7biGRALgSIzG2ZmOUQ6nWdGb2BmRVGzVwElwfRMYJKZ5ZrZMKAIeD+OtYpIAlmwaSfb99Zw7Rl65kOYsuJ1YHdvMLO7gdlAJjDN3VeZ2cPAInefCdxtZpcB9cBuYEqw7yozexH4EGgA7nL3xnjVKiKJ5Q9LttElJ5PLR/QLu5S0FreAAHD3WcCsFsseiJq+5wj7PgI8Er/qRCQR1TY0MmvlJ1wxqj+dcnR1e5hC76QWEYk2Z005VTUNTDxTzUthU0CISEL5w9Kt9OmawzgNzBc6BYSIJIy9NfW8saaMq8cMJCtTv57Cpu+AiCSMV1dup66hiYkamC8hKCBEJGH8afknDO7ViTMG9wy7FEEBISIJonJ/PfPXV/DF0QM0tEaCUECISEJ4ffUOGpqcCaP7h12KBBQQIpIQ/rxyOwN65HF6Yc+wS5GAAkJEQldd28DcknKuGNWfjAw1LyUKBYSIhG7OmjLqGpq4Us1LCUUBISKhe3Xldvp0zaF4aK+wS5EoCggRCVVNfSNz1pbxhVH9yVTzUkJRQIhIqOauK2d/XaOalxKQAkJEQvXqqu306JTNecM19lKiUUCISGgam5y/ri3nklP7kq2xlxKOviMiEpolH+9m1746Lh3RN+xSJAYFhIiE5vXVZWRlGONPLgi7FIlBASEioXlj9Q7OHdaL7nnZYZciMSggRCQUH+/cT0lZNZfqudMJSwEhIqF4ffUOAC5T/0PCUkCISCjeWLODk/p25YTeXcIuRVqhgBCR466qpp4FG3fp6qUEp4AQkeNu7roKGpqcy9T/kNAUECJy3L2xZgc9O2dz1pD8sEuRI1BAiMhx5e7MXVfB+KICDc6X4BQQInJcrdleRUV1LRcW9Qm7FGmDAkJEjqu3SyoAuLBId08nOgWEiBxXc0vKKerblf498sIuRdoQ14AwswlmttbM1pvZ/THW32tmH5rZcjN7w8xOiFrXaGZLg9fMeNYpIsdHTX0j72/apbOHJJEVrwObWSbwJHA5UAosNLOZ7v5h1GZLgGJ3329m3wQeBW4M1h1w9zPiVZ+IHH8LN++itqGJC09W/0MyiOcZxLnAenff6O51wAxgYvQG7j7H3fcHs+8BhXGsR0RC9nZJBTmZGYwdpmdPJ4MjBoSZ3Ro1Pa7FurvbOPYgYEvUfGmwrDVfB/4cNZ9nZovM7D0zu66N9xKRJDC3pIKzT8inc07cGi+kA7V1BnFv1PSPWqz7WkcVEQRRMfBY1OIT3L0YuBl4wsxOjLHfHUGILCovL++ockQkDsqqalj9yV41LyWRtgLCWpmONd/SVmBw1HxhsOzQg5hdBnwXuNbda5uXu/vW4N+NwF+BM1vu6+7PuHuxuxcXFKjTSySRvbM+uLz1JP2sJou2AsJbmY4139JCoMjMhplZDjAJOORqJDM7E3iaSDiURS3PN7PcYLoPMA6I7twWkSQzr6SC/M7ZjBrYPexS5Ci11RB4qpktJ3K2cGIwTTA//Eg7untD0E8xG8gEprn7KjN7GFjk7jOJNCl1BV4yM4CP3f1aYATwtJk1EQmxH7S4+klEkoi7M6+kgguKCsjQ8BpJo62AGPFZDu7us4BZLZY9EDV9WSv7zQdO+yzvLSKJY+2OKsqrNLxGsjliQLj7R9HzZtYbGE/kL/3F8SxMRFLHvHXNw2soIJJJW5e5/snMRgfTA4CVRK5emm5mfx//8kQkFcxbX8FJfbsyoEensEuRdmirk3qYu68Mpm8HXnP3a4CxdOBlriKSumrqG1mwcafOHpJQWwFRHzV9KUF/grtXAU3xKkpEUseizbupbWhivMZfSjptdVJvMbNvEbkL+izgVQAz6wRkx7k2EUkB80rKyc40xg7X8BrJpq0ziK8Do4DbgBvdfU+w/Dzg2fiVJSKpYp6G10habV3FVAZ8I8byOcCceBUlIqmhvKqWDz/Zyz9fcUrYpcgxOGJAtPUchuCmNhGRmJqH11D/Q3Jq65zvc0RGZH0BWEDb4y+JiBw0t6Rcw2sksbYCoj+RB/7cRGRU1VeAF9x9VbwLE5Hk5u68XVLBuJP6aHiNJHXETmp3b3T3V919CpGO6fXAX4/iWRAikubW7aimrKpWzUtJrM3LCoJRVa8ichYxFPgh8HJ8yxKRZDevJPKMlgt0g1zSaquT+jlgNJEb5B6KuqtaROSI5pVUcGJBFwb21PAayaqt+yBuBYqAe4D5ZrY3eFWZ2d74lyciyaimvpEFm3ZyoZqXklpb90G0FSAiIodZ/NFuauqbGK/HiyY1BYCIdLi5zcNrDOsddinyGSggRKTDvV1SwVlD8umSq+E1kpkCQkQ6VEV1Lau27WX8yep/SHYKCBHpUM3Da+j5D8lPASEiHWruuopgeI0eYZcin5ECQkQ6jLvz9vpyzj+pD5kaXiPpKSBEpMOUlFWzY28t49W8lBIUECLSYeauax5eQx3UqUABISIdpnl4jUEaXiMlKCBEpEPUNmh4jVSjgBCRDrF4c2R4DV3emjoUECLSIeaWVJCdaZw3XMNrpAoFhIh0iHkl5RpeI8UoIETkM9sZDK+h5qXUEteAMLMJZrbWzNab2f0x1t9rZh+a2XIze8PMTohaN8XMSoLXlHjWKSKfzdsHh9dQB3UqiVtAmFkm8CRwJTASuMnMRrbYbAlQ7O5jgP8FHg327QU8CIwFzgUeNLP8eNUqIp/NvJIKenbOZvQgDa+RSuJ5BnEusN7dN7p7HTADmBi9gbvPcff9wex7QGEwfQXwmrvvcvfdwGvAhDjWKiLHyN2ZV1LOuBM1vEaqiWdADAK2RM2XBsta83Xgz8e4r4iEZN2OYHgNPT0u5STE5QZmditQDFzUzv3uAO4AGDJkSBwqE5G2NA+voec/pJ54nkFsBQZHzRcGyw5hZpcB3wWudffa9uzr7s+4e7G7FxcU6D+nSBjmlpRT1LcrA3poeI1UE8+AWAgUmdkwM8sBJgEzozcwszOBp4mEQ1nUqtnAF8wsP+ic/kKwTEQSyIG6RhZs2qWzhxQVtyYmd28ws7uJ/GLPBKa5+yozexhY5O4zgceArsBLZgbwsbtf6+67zOx7REIG4GF33xWvWkXk2CzYtJO6Bg2vkari2gfh7rOAWS2WPRA1fdkR9p0GTItfdSLyWc0rqSAnK4OxwzS8RirSndQicszmritn7LBedMrJDLsUiQMFhIgck217DlBSVs143T2dshQQInJM5pXo8tZUp4AQkWMyd10F/bvncXK/rmGXInGigBCRdmtsct5eX8GFRX0IrkCUFKSAEJF2W1a6h8oD9WpeSnEKCBFpt7nryjGDC07S/Q+pTAEhIu02Z00ZZwzuSX6XnLBLkThSQIhIu5RV1bCstJJLT+0bdikSZwoIEWmXv66NXN56yan9Qq5E4k0BISLt8ubqMgb0yGPEgG5hlyJxpoAQkaNW29DIvJJyLj61ry5vTQMKCBE5ags37WZfXaP6H9KEAkJEjtoba3aQm5XB+Sfq8tZ0oIAQkaPi7ryxuozzT+yt0VvThAJCRI7KhvJ9fLxrP5eM0NVL6UIBISJHZc6ayFOBL1H/Q9pQQIjIUfnLh9s5tX83BvXsFHYpcpwoIESkTWVVNSz6aDdXjh4QdilyHCkgRKRNr324A3eYMLp/2KXIcaSAEJE2vbpyO0N7d9bDgdKMAkJEjqhyfz3vbtjJFaP76+7pNKOAEJEjemPNDhqanAmj1LyUbhQQInJEs1dtp3/3PE4v7Bl2KXKcKSBEpFX7aht4a105XxjVj4wMNS+lGwWEiLTqtQ93UFPfxNVjBoZdioRAASEirZq5bBsDe+RRfEJ+2KVICBQQIhLTrn11zF1XzjVnDFTzUppSQIhITLNWfEJDk3Pt6WpeSldxDQgzm2Bma81svZndH2P9eDP7wMwazOwrLdY1mtnS4DUznnWKyOFmLt3GSX27MnJA97BLkZBkxevAZpYJPAlcDpQCC81sprt/GLXZx8BtwD/FOMQBdz8jXvWJSOu27TnA+5t38Y+Xn6yb49JY3AICOBdY7+4bAcxsBjAROBgQ7r45WNcUxzpEpJ3+sHQbANeoeSmtxbOJaRCwJWq+NFh2tPLMbJGZvWdm18XawMzuCLZZVF5e/hlKFZFm7s6Li7ZwztB8hvbpEnY5EqJE7qQ+wd2LgZuBJ8zsxJYbuPsz7l7s7sUFBQXHv0KRFLRg0y42Vexj0jlDwi5FQhbPgNgKDI6aLwyWHRV33xr8uxH4K3BmRxYnIrH9ZuEWuuVl8cXT9OyHdBfPgFgIFJnZMDPLASYBR3U1kpnlm1luMN0HGEdU34WIxEfl/npmrfiE684YRKeczLDLkZDFLSDcvQG4G5gNrAZedPdVZvawmV0LYGbnmFkpcAPwtJmtCnYfASwys2XAHOAHLa5+EpE4+MOyrdQ2NHHjOYPb3lhSXjyvYsLdZwGzWix7IGp6IZGmp5b7zQdOi2dtInIod+eF97cwelB3Rg/qEXY5kgASuZNaRI6j9zbuYvUne7ll7AlhlyIJQgEhIgBMe2cTvbrk8KUz23M1uqQyBYSIsLliH6+v3sEtY4eQl63OaYlQQIgIv5i/mawM49bz1Lwkn1JAiKS5sqoaXnj/Y649fRD9uueFXY4kEAWESJr7+bxN1Dc2cdfFhw1WIGlOASGSxnZW1zL93Y+49vSBDC/oGnY5kmAUECJp7Jl5G6lpaOTuS04KuxRJQAoIkTS1Zdd+nn1nM186cxAn9e0WdjmSgBQQImnqsdlryTD45ytOCbsUSVAKCJE0tOTj3cxcto2/vXA4A3p0CrscSVAKCJE0U9/YxL/8bgX9u+fxdxfpyiVpXVwH6xORxPP0WxtYs72Kn321mK65+hUgrdMZhEgaWbejih++sZ6rThvA5SP7hV2OJDgFhEia2F/XwJ3Pf0D3Ttn8+7Wjwi5HkoDOL0XSgLvzb79fxYbyan719bEUdMsNuyRJAjqDEEkDT721kd9+UMq3Lili3El9wi5HkoQCQiTF/e6DUv7j1TVcc/pA/v7SorDLkSSigBBJYS8u2sI/vbSMzw3vzeM3jCEjw8IuSZJI2vdBHKhr5Nn5mw5ZZnz6Q2QWvZyYy4+0zyHbRK1ouUnr79P6cQ+ZbeXYhx730AMczWdruQ9HU2crx4qeN4yMDCPTjMwMyDAjK9PIMCMzWJ6RYWRlRG/36at5u6xgPicrg9ysDHKyMsjJzDiknnTk7vz0rQ08+upaLizqw9OTzyY3Sw8CkvZJ+4DYX9fAo6+uDbsM6WDNgRF5ZR4Mj+b5zrmZdM3NOvjq0jydF5nulptFz87Z9OqSQ36XHLrlZiVN6OzZX8c/vbSc11fv4OoxA/h/f3O6wkGOSdoHRK8uOaz53oSY69yjpvGYyyProvfxVpa3ssMRjt3acY907OhjtTJ5xH1ae/+WNbT8GrRZS4t1Te40udPYBI1NzdNOQ9R0UzDf6JHp5u0aoqYbm6ChsYm6xiZq65v/baT2kPlPl9c1NlFT38jO6jo+3rmf6toGqmsb2F/XGPsDBbIzjfzOOZHA6JxDr6459OuWx4AeefTrEfm3f/c8+nbPDe2XcWOT89KiLTw6ey1VNfU8eM1Ibjt/aNIEmySetA8IM9MzeIXGJmdfXQP7aiOvvTUNVO6vZ9e+ushrfx27g+nd++tYvW0vc/aWxQyWPl1z6Nc9j4E9OzGoZycG9oxMN88XdM3t0L6AygP1zFy2jf+Zt5HNO/dzztB8Hrp2NCMHdu+w95D0lPYBIQKQmWF0z8ume172Ue/j7lTVNrC9subT194aPqmsYXvlAbbs2s97G3dSVdNwyH7ZmUb/HnkM7NEcIJ0Y0DM6UDq1OgRGU5OzfW8Nm3fuY3lpJe9t3Mk76yuob3ROL+zBT245iytH99dZg3QIBYTIMTL7NFRO7tf68xT21tTzyZ4atu05wNY9B9h28FXDgk272L63hsamQ5vjuuRk0ikni7zsSKd7TX0jB+obqa5toL7x022HF3ThtvOHcvWYgYwp7KFgkA6lgBCJs+552XTvn80p/WOHSGOTU1bVHCCRf8v21nKgvvFgf0qn7Ew6ZWfSJTeLwvxOnNC7MyMGdKdPV90RLfGjgBAJWWaGMaBHJwb06MTZJ4RdjcindKOciIjEpIAQEZGY4hoQZjbBzNaa2Xozuz/G+vFm9oGZNZjZV1qsm2JmJcFrSjzrFBGRw8UtIMwsE3gSuBIYCdxkZiNbbPYxcBvw6xb79gIeBMYC5wIPmll+vGoVEZHDxfMM4lxgvbtvdPc6YAYwMXoDd9/s7suBphb7XgG85u673H038BoQ+3ZnERGJi3gGxCBgS9R8abCsw/Y1szvMbJGZLSovLz/mQkVE5HBJ3Unt7s+4e7G7FxcUFIRdjohISolnQGwFBkfNFwbL4r2viIh0AGs5SmiHHdgsC1gHXErkl/tC4GZ3XxVj218Af3L3/w3mewGLgbOCTT4Aznb3XUd4v3Lgo89Qch+g4jPsHy+qq31UV/uorvZJxbpOcPeYTTBxCwgAM/si8ASQCUxz90fM7GFgkbvPNLNzgJeBfKAG2O7uo4J9vwZ8JzjUI+7+bNwKjbzfIncvjud7HAvV1T6qq31UV/ukW11xHWrD3WcBs1oseyBqeiGR5qNY+04DpsWzPhERaV1Sd1KLiEj8KCA+9UzYBbRCdbWP6mof1dU+aVVXXPsgREQkeekMQkREYlJAiIhITAqIgJl9z8yWm9lSM/uLmQ0MuyYAM3vMzNYEtb1sZj3DrgnAzG4ws1Vm1mRmoV/219bIwWExs2lmVmZmK8OupZmZDTazOWb2YfA9vCfsmpqZWZ6ZvW9my4LaHgq7pmZmlmlmS8zsT2HXEs3MNpvZiuB316KOPLYC4lOPufsYdz8D+BPwQBvbHy+vAaPdfQyRGw//JeR6mq0Ergfmhl3IUY4cHJZfkHgDTTYA/+juI4HzgLsS6OtVC1zi7qcDZwATzOy8cEs66B5gddhFtOJidz+jo++FUEAE3H1v1GwXICF67939L+7eEMy+Ryv3jRxv7r7a3deGXUegzZGDw+Luc4FWRwAIg7t/4u4fBNNVRH7pHe1AmnHlEdXBbHbwCv1n0cwKgauAn4ddy/GkgIhiZo+Y2RbgFhLnDCLa14A/h11EAvosIwenNTMbCpwJLAi5lIOCppylQBmRYf8TobYngG9z+KMJEoEDfzGzxWZ2R0ceOK0CwsxeN7OVMV4TAdz9u+4+GHgeuDtR6gq2+S6RpoHnE6kuSV5m1hX4LfD3Lc6gQ+XujUFTbyFwrpmNDrMeM7saKHP3xWHWcQQXuPtZRJpY7zKz8R114LgOtZFo3P2yo9z0eSJDhDwYx3IOaqsuM7sNuBq41I/jjSvt+HqFTaP/tpOZZRMJh+fd/Xdh1xOLu+8xszlE+nDC7OQfB1wbjC2XB3Q3s1+5+60h1nSQu28N/i0zs5eJNLl2SN9gWp1BHImZFUXNTgTWhFVLNDObQOTU9lp33x92PQlqIVBkZsPMLAeYBMwMuaaEZWYG/A+w2t3/M+x6oplZQfOVembWCbickH8W3f1f3L3Q3YcS+b/1ZqKEg5l1MbNuzdPAF+jAMFVAfOoHQfPJciJf5ES59O/HQDfgteAytqfCLgjAzL5kZqXA54BXzGx2WLUEnfh3A7OJdLi+GGtY+TCY2QvAu8ApZlZqZl8PuyYifxFPBi4J/k8tDf46TgQDgDnBz+FCIn0QCXVZaYLpB7xtZsuA94FX3P3Vjjq4htoQEZGYdAYhIiIxKSBERCQmBYSIiMSkgBARkZgUECIiEpMCQlKSmVW3vVWr+94djArrZtYnarmZ2Q+DdcvN7KyodQOaR/k0s8931IifZvbXoxktNxjRs08b27xuZvkdUZekBwWEyOHeAS4DPmqx/EqgKHjdAfw0at29wM+OS3XHbjpwZ9hFSPJQQEhKC/7qfyy4CXKFmd0YLM8ws59Y5Fkbr5nZLDP7CoC7L3H3zTEONxF4Lhhx9D2gp5kNCNZ9GTjsBiUzO9fM3g2eIzDfzE4Jlt9mZr8P3ntzcNZyb7Dde2bWK+owk4Ob2Vaa2bnB/r0t8tySVWb2c8Ci3vP3wcBtq1oM3jYTuOlYv5aSfhQQkuquJ/JcgdOJnBU8FvxSvx4YSuT5EZOJ3BHelpijxprZMGC3u9fG2GcNcKG7n0lkhOD/E7VudFDHOcAjwP5gu3eBr0Zt1zkYvO5OYFqw7EHgbXcfBbwMDIna/mvufjZQDEw1s94A7r4byG2eF2lLWg3WJ2npAuAFd28EdpjZW0R+IV8AvOTuTcD2YFC4YzUAKG9lXQ/gl8FYX07k+QbN5gTPY6gys0rgj8HyFcCYqO1egMizJcysezBW0Xgi4YK7v2Jmu6O2n2pmXwqmBxNpEtsZzJcBA6PmRVqlMwiRo9faqLEHiIzyGcv3iATBaOCaFttFn3E0Rc03cegfby3Hw2l1fBwz+zyRM6XPBU9lW9LiPfOCekXapICQVDcPuNEiD6EpIPKX9/tEOqK/HPRF9AM+fxTHmgl8NejXOA+odPdPiDwKdmgr+/Tg06HHbzvGz9Dcb3JB8J6VRIZzvjlYfiXQfHVSDyLNXfvN7FQijxQl2M6A/sDmY6xD0owCQlLdy8ByYBnwJvBtd99O5FkIpcCHwK+AD4BKADObGoxUWwgsDzqBIfKMkI3AeiJXLN0J4O77gA1mdlKM938U+L9mtoRjb9KtCfZ/CmgeDfYhYLyZrSLS1PRxsPxVIMvMVgM/IPKY2mZnA+9FPcJW5Ig0mqukLTPr6u7VQaft+8C4IDyO5VhfAs5293/t0CI7kJn9NzDT3d8IuxZJDuqklnT2p6DDNwf43rGGA4C7v5wEVwetVDhIe+gMQkREYlIfhIiIxKSAEBGRmBQQIiISkwJCRERiUkCIiEhM/x+hgaiTDp/jowAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "filenames": {
       "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_50_0.png"
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import KFold\n",
    "from sklearn.linear_model import Ridge\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "\n",
    "# A seed just to ensure that the random numbers are the same for every run.\n",
    "np.random.seed(3155)\n",
    "# Generate the data.\n",
    "n = 100\n",
    "x = np.linspace(-3, 3, n).reshape(-1, 1)\n",
    "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)\n",
    "# Decide degree on polynomial to fit\n",
    "poly = PolynomialFeatures(degree = 10)\n",
    "\n",
    "# Decide which values of lambda to use\n",
    "nlambdas = 500\n",
    "lambdas = np.logspace(-3, 5, nlambdas)\n",
    "# Initialize a KFold instance\n",
    "k = 5\n",
    "kfold = KFold(n_splits = k)\n",
    "estimated_mse_sklearn = np.zeros(nlambdas)\n",
    "i = 0\n",
    "for lmb in lambdas:\n",
    "    ridge = Ridge(alpha = lmb)\n",
    "    estimated_mse_folds = cross_val_score(ridge, x, y, scoring='neg_mean_squared_error', cv=kfold)\n",
    "    estimated_mse_sklearn[i] = np.mean(-estimated_mse_folds)\n",
    "    i += 1\n",
    "plt.figure()\n",
    "plt.plot(np.log10(lambdas), estimated_mse_sklearn, label = 'cross_val_score')\n",
    "plt.xlabel('log10(lambda)')\n",
    "plt.ylabel('MSE')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}