{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Resampling Methods\n", "\n", "## Introduction\n", "\n", "Resampling methods are an indispensable tool in modern\n", "statistics. They involve repeatedly drawing samples from a training\n", "set and refitting a model of interest on each sample in order to\n", "obtain additional information about the fitted model. For example, in\n", "order to estimate the variability of a linear regression fit, we can\n", "repeatedly draw different samples from the training data, fit a linear\n", "regression to each new sample, and then examine the extent to which\n", "the resulting fits differ. Such an approach may allow us to obtain\n", "information that would not be available from fitting the model only\n", "once using the original training sample.\n", "\n", "Two resampling methods are often used in Machine Learning analyses,\n", "1. The **bootstrap method**\n", "\n", "2. and **Cross-Validation**\n", "\n", "In addition there are several other methods such as the Jackknife and the Blocking methods. We will discuss in particular\n", "cross-validation and the bootstrap method. \n", "\n", "\n", "Resampling approaches can be computationally expensive, because they\n", "involve fitting the same statistical method multiple times using\n", "different subsets of the training data. However, due to recent\n", "advances in computing power, the computational requirements of\n", "resampling methods generally are not prohibitive. In this chapter, we\n", "discuss two of the most commonly used resampling methods,\n", "cross-validation and the bootstrap. Both methods are important tools\n", "in the practical application of many statistical learning\n", "procedures. For example, cross-validation can be used to estimate the\n", "test error associated with a given statistical learning method in\n", "order to evaluate its performance, or to select the appropriate level\n", "of flexibility. The process of evaluating a model’s performance is\n", "known as model assessment, whereas the process of selecting the proper\n", "level of flexibility for a model is known as model selection. The\n", "bootstrap is widely used.\n", "\n", "\n", "* Our simulations can be treated as *computer experiments*. This is particularly the case for Monte Carlo methods\n", "\n", "* The results can be analysed with the same statistical tools as we would use analysing experimental data.\n", "\n", "* As in all experiments, we are looking for expectation values and an estimate of how accurate they are, i.e., possible sources for errors.\n", "\n", "## Reminder on Statistics\n", "\n", "\n", "* As in other experiments, many numerical experiments have two classes of errors:\n", "\n", " * Statistical errors\n", "\n", " * Systematical errors\n", "\n", "\n", "* Statistical errors can be estimated using standard tools from statistics\n", "\n", "* Systematical errors are method specific and must be treated differently from case to case. \n", "\n", "The\n", "advantage of doing linear regression is that we actually end up with\n", "analytical expressions for several statistical quantities. \n", "Standard least squares and Ridge regression allow us to\n", "derive quantities like the variance and other expectation values in a\n", "rather straightforward way.\n", "\n", "\n", "It is assumed that $\\varepsilon_i\n", "\\sim \\mathcal{N}(0, \\sigma^2)$ and the $\\varepsilon_{i}$ are\n", "independent, i.e.:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{align*} \n", "\\mbox{Cov}(\\varepsilon_{i_1},\n", "\\varepsilon_{i_2}) & = \\left\\{ \\begin{array}{lcc} \\sigma^2 & \\mbox{if}\n", "& i_1 = i_2, \\\\ 0 & \\mbox{if} & i_1 \\not= i_2. \\end{array} \\right.\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The randomness of $\\varepsilon_i$ implies that\n", "$\\mathbf{y}_i$ is also a random variable. In particular,\n", "$\\mathbf{y}_i$ is normally distributed, because $\\varepsilon_i \\sim\n", "\\mathcal{N}(0, \\sigma^2)$ and $\\mathbf{X}_{i,\\ast} \\, \\boldsymbol{\\beta}$ is a\n", "non-random scalar. To specify the parameters of the distribution of\n", "$\\mathbf{y}_i$ we need to calculate its first two moments. \n", "\n", "Recall that $\\boldsymbol{X}$ is a matrix of dimensionality $n\\times p$. The\n", "notation above $\\mathbf{X}_{i,\\ast}$ means that we are looking at the\n", "row number $i$ and perform a sum over all values $p$.\n", "\n", "\n", "The assumption we have made here can be summarized as (and this is going to be useful when we discuss the bias-variance trade off)\n", "that there exists a function $f(\\boldsymbol{x})$ and a normal distributed error $\\boldsymbol{\\varepsilon}\\sim \\mathcal{N}(0, \\sigma^2)$\n", "which describe our data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{y} = f(\\boldsymbol{x})+\\boldsymbol{\\varepsilon}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We approximate this function with our model from the solution of the linear regression equations, that is our\n", "function $f$ is approximated by $\\boldsymbol{\\tilde{y}}$ where we want to minimize $(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2$, our MSE, with" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{\\tilde{y}} = \\boldsymbol{X}\\boldsymbol{\\beta}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can calculate the expectation value of $\\boldsymbol{y}$ for a given element $i$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{align*} \n", "\\mathbb{E}(y_i) & =\n", "\\mathbb{E}(\\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta}) + \\mathbb{E}(\\varepsilon_i)\n", "\\, \\, \\, = \\, \\, \\, \\mathbf{X}_{i, \\ast} \\, \\beta, \n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "while\n", "its variance is" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{align*} \\mbox{Var}(y_i) & = \\mathbb{E} \\{ [y_i\n", "- \\mathbb{E}(y_i)]^2 \\} \\, \\, \\, = \\, \\, \\, \\mathbb{E} ( y_i^2 ) -\n", "[\\mathbb{E}(y_i)]^2 \\\\ & = \\mathbb{E} [ ( \\mathbf{X}_{i, \\ast} \\,\n", "\\beta + \\varepsilon_i )^2] - ( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta})^2 \\\\ &\n", "= \\mathbb{E} [ ( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta})^2 + 2 \\varepsilon_i\n", "\\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta} + \\varepsilon_i^2 ] - ( \\mathbf{X}_{i,\n", "\\ast} \\, \\beta)^2 \\\\ & = ( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta})^2 + 2\n", "\\mathbb{E}(\\varepsilon_i) \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta} +\n", "\\mathbb{E}(\\varepsilon_i^2 ) - ( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta})^2 \n", "\\\\ & = \\mathbb{E}(\\varepsilon_i^2 ) \\, \\, \\, = \\, \\, \\,\n", "\\mbox{Var}(\\varepsilon_i) \\, \\, \\, = \\, \\, \\, \\sigma^2. \n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Hence, $y_i \\sim \\mathcal{N}( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta}, \\sigma^2)$, that is $\\boldsymbol{y}$ follows a normal distribution with \n", "mean value $\\boldsymbol{X}\\boldsymbol{\\beta}$ and variance $\\sigma^2$ (not be confused with the singular values of the SVD). \n", "\n", "\n", "With the OLS expressions for the parameters $\\boldsymbol{\\beta}$ we can evaluate the expectation value" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}(\\boldsymbol{\\beta}) = \\mathbb{E}[ (\\mathbf{X}^{\\top} \\mathbf{X})^{-1}\\mathbf{X}^{T} \\mathbf{Y}]=(\\mathbf{X}^{T} \\mathbf{X})^{-1}\\mathbf{X}^{T} \\mathbb{E}[ \\mathbf{Y}]=(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\mathbf{X}^{T}\\mathbf{X}\\boldsymbol{\\beta}=\\boldsymbol{\\beta}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This means that the estimator of the regression parameters is unbiased.\n", "v\n", "We can also calculate the variance\n", "\n", "The variance of $\\boldsymbol{\\beta}$ is" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{eqnarray*}\n", "\\mbox{Var}(\\boldsymbol{\\beta}) & = & \\mathbb{E} \\{ [\\boldsymbol{\\beta} - \\mathbb{E}(\\boldsymbol{\\beta})] [\\boldsymbol{\\beta} - \\mathbb{E}(\\boldsymbol{\\beta})]^{T} \\}\n", "\\\\\n", "& = & \\mathbb{E} \\{ [(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y} - \\boldsymbol{\\beta}] \\, [(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y} - \\boldsymbol{\\beta}]^{T} \\}\n", "\\\\\n", "% & = & \\mathbb{E} \\{ [(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y}] \\, [(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y}]^{T} \\} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n", "% \\\\\n", "% & = & \\mathbb{E} \\{ (\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y} \\, \\mathbf{Y}^{T} \\, \\mathbf{X} \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1} \\} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n", "% \\\\\n", "& = & (\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\, \\mathbb{E} \\{ \\mathbf{Y} \\, \\mathbf{Y}^{T} \\} \\, \\mathbf{X} \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n", "\\\\\n", "& = & (\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\, \\{ \\mathbf{X} \\, \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T} \\, \\mathbf{X}^{T} + \\sigma^2 \\} \\, \\mathbf{X} \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n", "% \\\\\n", "% & = & (\\mathbf{X}^T \\mathbf{X})^{-1} \\, \\mathbf{X}^T \\, \\mathbf{X} \\, \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^T \\, \\mathbf{X}^T \\, \\mathbf{X} \\, (\\mathbf{X}^T % \\mathbf{X})^{-1}\n", "% \\\\\n", "% & & + \\, \\, \\sigma^2 \\, (\\mathbf{X}^T \\mathbf{X})^{-1} \\, \\mathbf{X}^T \\, \\mathbf{X} \\, (\\mathbf{X}^T \\mathbf{X})^{-1} - \\boldsymbol{\\beta} \\boldsymbol{\\beta}^T\n", "\\\\\n", "& = & \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T} + \\sigma^2 \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n", "\\, \\, \\, = \\, \\, \\, \\sigma^2 \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1},\n", "\\end{eqnarray*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where we have used that $\\mathbb{E} (\\mathbf{Y} \\mathbf{Y}^{T}) =\n", "\\mathbf{X} \\, \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T} \\, \\mathbf{X}^{T} +\n", "\\sigma^2 \\, \\mathbf{I}_{nn}$. From $\\mbox{Var}(\\boldsymbol{\\beta}) = \\sigma^2\n", "\\, (\\mathbf{X}^{T} \\mathbf{X})^{-1}$, one obtains an estimate of the\n", "variance of the estimate of the $j$-th regression coefficient:\n", "$\\boldsymbol{\\sigma}^2 (\\boldsymbol{\\beta}_j ) = \\boldsymbol{\\sigma}^2 \\sqrt{\n", "[(\\mathbf{X}^{T} \\mathbf{X})^{-1}]_{jj} }$. This may be used to\n", "construct a confidence interval for the estimates.\n", "\n", "\n", "In a similar way, we can obtain analytical expressions for say the\n", "expectation values of the parameters $\\boldsymbol{\\beta}$ and their variance\n", "when we employ Ridge regression, allowing us again to define a confidence interval. \n", "\n", "It is rather straightforward to show that" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E} \\big[ \\boldsymbol{\\beta}^{\\mathrm{Ridge}} \\big]=(\\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I}_{pp})^{-1} (\\mathbf{X}^{\\top} \\mathbf{X})\\boldsymbol{\\beta}^{\\mathrm{OLS}}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see clearly that \n", "$\\mathbb{E} \\big[ \\boldsymbol{\\beta}^{\\mathrm{Ridge}} \\big] \\not= \\boldsymbol{\\beta}^{\\mathrm{OLS}}$ for any $\\lambda > 0$. We say then that the ridge estimator is biased.\n", "\n", "We can also compute the variance as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mbox{Var}[\\boldsymbol{\\beta}^{\\mathrm{Ridge}}]=\\sigma^2[ \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1} \\mathbf{X}^{T} \\mathbf{X} \\{ [ \\mathbf{X}^{\\top} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}\\}^{T},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and it is easy to see that if the parameter $\\lambda$ goes to infinity then the variance of Ridge parameters $\\boldsymbol{\\beta}$ goes to zero. \n", "\n", "With this, we can compute the difference" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mbox{Var}[\\boldsymbol{\\beta}^{\\mathrm{OLS}}]-\\mbox{Var}(\\boldsymbol{\\beta}^{\\mathrm{Ridge}})=\\sigma^2 [ \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}[ 2\\lambda\\mathbf{I} + \\lambda^2 (\\mathbf{X}^{T} \\mathbf{X})^{-1} ] \\{ [ \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}\\}^{T}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The difference is non-negative definite since each component of the\n", "matrix product is non-negative definite. \n", "This means the variance we obtain with the standard OLS will always for $\\lambda > 0$ be larger than the variance of $\\boldsymbol{\\beta}$ obtained with the Ridge estimator. This has interesting consequences when we discuss the so-called bias-variance trade-off below. \n", "\n", "\n", "\n", "## Resampling methods\n", "\n", "With all these analytical equations for both the OLS and Ridge\n", "regression, we will now outline how to assess a given model. This will\n", "lead us to a discussion of the so-called bias-variance tradeoff (see\n", "below) and so-called resampling methods.\n", "\n", "One of the quantities we have discussed as a way to measure errors is\n", "the mean-squared error (MSE), mainly used for fitting of continuous\n", "functions. Another choice is the absolute error.\n", "\n", "In the discussions below we will focus on the MSE and in particular since we will split the data into test and training data,\n", "we discuss the\n", "1. prediction error or simply the **test error** $\\mathrm{Err_{Test}}$, where we have a fixed training set and the test error is the MSE arising from the data reserved for testing. We discuss also the \n", "\n", "2. training error $\\mathrm{Err_{Train}}$, which is the average loss over the training data.\n", "\n", "As our model becomes more and more complex, more of the training data tends to used. The training may thence adapt to more complicated structures in the data. This may lead to a decrease in the bias (see below for code example) and a slight increase of the variance for the test error.\n", "For a certain level of complexity the test error will reach minimum, before starting to increase again. The\n", "training error reaches a saturation.\n", "\n", "\n", "\n", "Two famous\n", "resampling methods are the **independent bootstrap** and **the jackknife**. \n", "\n", "The jackknife is a special case of the independent bootstrap. Still, the jackknife was made\n", "popular prior to the independent bootstrap. And as the popularity of\n", "the independent bootstrap soared, new variants, such as **the dependent bootstrap**.\n", "\n", "The Jackknife and independent bootstrap work for\n", "independent, identically distributed random variables.\n", "If these conditions are not\n", "satisfied, the methods will fail. Yet, it should be said that if the data are\n", "independent, identically distributed, and we only want to estimate the\n", "variance of $\\overline{X}$ (which often is the case), then there is no\n", "need for bootstrapping. \n", "\n", "\n", "The Jackknife works by making many replicas of the estimator $\\widehat{\\beta}$. \n", "The jackknife is a resampling method where we systematically leave out one observation from the vector of observed values $\\boldsymbol{x} = (x_1,x_2,\\cdots,X_n)$. \n", "Let $\\boldsymbol{x}_i$ denote the vector" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{x}_i = (x_1,x_2,\\cdots,x_{i-1},x_{i+1},\\cdots,x_n),\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "which equals the vector $\\boldsymbol{x}$ with the exception that observation\n", "number $i$ is left out. Using this notation, define\n", "$\\widehat{\\beta}_i$ to be the estimator\n", "$\\widehat{\\beta}$ computed using $\\vec{X}_i$." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Runtime: 0.14109 sec\n", "Jackknife Statistics :\n", "original bias std. error\n", " 100.203 100.193 0.149917\n" ] } ], "source": [ "from numpy import *\n", "from numpy.random import randint, randn\n", "from time import time\n", "\n", "def jackknife(data, stat):\n", " n = len(data);t = zeros(n); inds = arange(n); t0 = time()\n", " ## 'jackknifing' by leaving out an observation for each i \n", " for i in range(n):\n", " t[i] = stat(delete(data,i) )\n", "\n", " # analysis \n", " print(\"Runtime: %g sec\" % (time()-t0)); print(\"Jackknife Statistics :\")\n", " print(\"original bias std. error\")\n", " print(\"%8g %14g %15g\" % (stat(data),(n-1)*mean(t)/n, (n*var(t))**.5))\n", "\n", " return t\n", "\n", "\n", "# Returns mean of data samples \n", "def stat(data):\n", " return mean(data)\n", "\n", "\n", "mu, sigma = 100, 15\n", "datapoints = 10000\n", "x = mu + sigma*random.randn(datapoints)\n", "# jackknife returns the data sample \n", "t = jackknife(x, stat)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Bootstrap\n", "\n", "Bootstrapping is a nonparametric approach to statistical inference\n", "that substitutes computation for more traditional distributional\n", "assumptions and asymptotic results. Bootstrapping offers a number of\n", "advantages: \n", "1. The bootstrap is quite general, although there are some cases in which it fails. \n", "\n", "2. Because it does not require distributional assumptions (such as normally distributed errors), the bootstrap can provide more accurate inferences when the data are not well behaved or when the sample size is small. \n", "\n", "3. It is possible to apply the bootstrap to statistics with sampling distributions that are difficult to derive, even asymptotically. \n", "\n", "4. It is relatively simple to apply the bootstrap to complex data-collection plans (such as stratified and clustered samples).\n", "\n", "Since $\\widehat{\\beta} = \\widehat{\\beta}(\\boldsymbol{X})$ is a function of random variables,\n", "$\\widehat{\\beta}$ itself must be a random variable. Thus it has\n", "a pdf, call this function $p(\\boldsymbol{t})$. The aim of the bootstrap is to\n", "estimate $p(\\boldsymbol{t})$ by the relative frequency of\n", "$\\widehat{\\beta}$. You can think of this as using a histogram\n", "in the place of $p(\\boldsymbol{t})$. If the relative frequency closely\n", "resembles $p(\\vec{t})$, then using numerics, it is straight forward to\n", "estimate all the interesting parameters of $p(\\boldsymbol{t})$ using point\n", "estimators. \n", "\n", "\n", "\n", "In the case that $\\widehat{\\beta}$ has\n", "more than one component, and the components are independent, we use the\n", "same estimator on each component separately. If the probability\n", "density function of $X_i$, $p(x)$, had been known, then it would have\n", "been straight forward to do this by: \n", "1. Drawing lots of numbers from $p(x)$, suppose we call one such set of numbers $(X_1^*, X_2^*, \\cdots, X_n^*)$. \n", "\n", "2. Then using these numbers, we could compute a replica of $\\widehat{\\beta}$ called $\\widehat{\\beta}^*$. \n", "\n", "By repeated use of (1) and (2), many\n", "estimates of $\\widehat{\\beta}$ could have been obtained. The\n", "idea is to use the relative frequency of $\\widehat{\\beta}^*$\n", "(think of a histogram) as an estimate of $p(\\boldsymbol{t})$.\n", "\n", "\n", "But\n", "unless there is enough information available about the process that\n", "generated $X_1,X_2,\\cdots,X_n$, $p(x)$ is in general\n", "unknown. Therefore, [Efron in 1979](https://projecteuclid.org/euclid.aos/1176344552) asked the\n", "question: What if we replace $p(x)$ by the relative frequency\n", "of the observation $X_i$; if we draw observations in accordance with\n", "the relative frequency of the observations, will we obtain the same\n", "result in some asymptotic sense? The answer is yes.\n", "\n", "\n", "Instead of generating the histogram for the relative\n", "frequency of the observation $X_i$, just draw the values\n", "$(X_1^*,X_2^*,\\cdots,X_n^*)$ with replacement from the vector\n", "$\\boldsymbol{X}$. \n", "\n", "\n", "The independent bootstrap works like this: \n", "\n", "1. Draw with replacement $n$ numbers for the observed variables $\\boldsymbol{x} = (x_1,x_2,\\cdots,x_n)$. \n", "\n", "2. Define a vector $\\boldsymbol{x}^*$ containing the values which were drawn from $\\boldsymbol{x}$. \n", "\n", "3. Using the vector $\\boldsymbol{x}^*$ compute $\\widehat{\\beta}^*$ by evaluating $\\widehat \\beta$ under the observations $\\boldsymbol{x}^*$. \n", "\n", "4. Repeat this process $k$ times. \n", "\n", "When you are done, you can draw a histogram of the relative frequency\n", "of $\\widehat \\beta^*$. This is your estimate of the probability\n", "distribution $p(t)$. Using this probability distribution you can\n", "estimate any statistics thereof. In principle you never draw the\n", "histogram of the relative frequency of $\\widehat{\\beta}^*$. Instead\n", "you use the estimators corresponding to the statistic of interest. For\n", "example, if you are interested in estimating the variance of $\\widehat\n", "\\beta$, apply the etsimator $\\widehat \\sigma^2$ to the values\n", "$\\widehat \\beta^*$.\n", "\n", "Before we proceed however, we need to remind ourselves about a central\n", "theorem in statistics, namely the so-called **central limit theorem**.\n", "This theorem plays a central role in understanding why the Bootstrap\n", "(and other resampling methods) work so well on independent and\n", "identically distributed variables.\n", "\n", "\n", "Suppose we have a PDF $p(x)$ from which we generate a series $N$\n", "of averages $\\langle x_i \\rangle$. Each mean value $\\langle x_i \\rangle$\n", "is viewed as the average of a specific measurement, e.g., throwing \n", "dice 100 times and then taking the average value, or producing a certain\n", "amount of random numbers. \n", "For notational ease, we set $\\langle x_i \\rangle=x_i$ in the discussion\n", "which follows. \n", "\n", "If we compute the mean $z$ of $m$ such mean values $x_i$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "z=\\frac{x_1+x_2+\\dots+x_m}{m},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "the question we pose is which is the PDF of the new variable $z$.\n", "\n", "\n", "The probability of obtaining an average value $z$ is the product of the \n", "probabilities of obtaining arbitrary individual mean values $x_i$,\n", "but with the constraint that the average is $z$. We can express this through\n", "the following expression" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\tilde{p}(z)=\\int dx_1p(x_1)\\int dx_2p(x_2)\\dots\\int dx_mp(x_m)\n", " \\delta(z-\\frac{x_1+x_2+\\dots+x_m}{m}),\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where the $\\delta$-function enbodies the constraint that the mean is $z$.\n", "All measurements that lead to each individual $x_i$ are expected to\n", "be independent, which in turn means that we can express $\\tilde{p}$ as the \n", "product of individual $p(x_i)$. The independence assumption is important in the derivation of the central limit theorem.\n", "\n", "\n", "\n", "If we use the integral expression for the $\\delta$-function" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\delta(z-\\frac{x_1+x_2+\\dots+x_m}{m})=\\frac{1}{2\\pi}\\int_{-\\infty}^{\\infty}\n", " dq\\exp{\\left(iq(z-\\frac{x_1+x_2+\\dots+x_m}{m})\\right)},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and inserting $e^{i\\mu q-i\\mu q}$ where $\\mu$ is the mean value\n", "we arrive at" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\tilde{p}(z)=\\frac{1}{2\\pi}\\int_{-\\infty}^{\\infty}\n", " dq\\exp{\\left(iq(z-\\mu)\\right)}\\left[\\int_{-\\infty}^{\\infty}\n", " dxp(x)\\exp{\\left(iq(\\mu-x)/m\\right)}\\right]^m,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "with the integral over $x$ resulting in" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\int_{-\\infty}^{\\infty}dxp(x)\\exp{\\left(iq(\\mu-x)/m\\right)}=\n", " \\int_{-\\infty}^{\\infty}dxp(x)\n", " \\left[1+\\frac{iq(\\mu-x)}{m}-\\frac{q^2(\\mu-x)^2}{2m^2}+\\dots\\right].\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The second term on the rhs disappears since this is just the mean and \n", "employing the definition of $\\sigma^2$ we have" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\int_{-\\infty}^{\\infty}dxp(x)e^{\\left(iq(\\mu-x)/m\\right)}=\n", " 1-\\frac{q^2\\sigma^2}{2m^2}+\\dots,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "resulting in" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\left[\\int_{-\\infty}^{\\infty}dxp(x)\\exp{\\left(iq(\\mu-x)/m\\right)}\\right]^m\\approx\n", " \\left[1-\\frac{q^2\\sigma^2}{2m^2}+\\dots \\right]^m,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and in the limit $m\\rightarrow \\infty$ we obtain" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\tilde{p}(z)=\\frac{1}{\\sqrt{2\\pi}(\\sigma/\\sqrt{m})}\n", " \\exp{\\left(-\\frac{(z-\\mu)^2}{2(\\sigma/\\sqrt{m})^2}\\right)},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "which is the normal distribution with variance\n", "$\\sigma^2_m=\\sigma^2/m$, where $\\sigma$ is the variance of the PDF $p(x)$\n", "and $\\mu$ is also the mean of the PDF $p(x)$. \n", "\n", "\n", "Thus, the central limit theorem states that the PDF $\\tilde{p}(z)$ of\n", "the average of $m$ random values corresponding to a PDF $p(x)$ \n", "is a normal distribution whose mean is the \n", "mean value of the PDF $p(x)$ and whose variance is the variance\n", "of the PDF $p(x)$ divided by $m$, the number of values used to compute $z$.\n", "\n", "The central limit theorem leads to the well-known expression for the\n", "standard deviation, given by" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\sigma_m=\n", "\\frac{\\sigma}{\\sqrt{m}}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The latter is true only if the average value is known exactly. This is obtained in the limit\n", "$m\\rightarrow \\infty$ only. Because the mean and the variance are measured quantities we obtain \n", "the familiar expression in statistics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\sigma_m\\approx \n", "\\frac{\\sigma}{\\sqrt{m-1}}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In many cases however the above estimate for the standard deviation,\n", "in particular if correlations are strong, may be too simplistic. Keep\n", "in mind that we have assumed that the variables $x$ are independent\n", "and identically distributed. This is obviously not always the\n", "case. For example, the random numbers (or better pseudorandom numbers)\n", "we generate in various calculations do always exhibit some\n", "correlations.\n", "\n", "\n", "\n", "The theorem is satisfied by a large class of PDFs. Note however that for a\n", "finite $m$, it is not always possible to find a closed form /analytic expression for\n", "$\\tilde{p}(x)$.\n", "\n", "\n", "The following code starts with a Gaussian distribution with mean value\n", "$\\mu =100$ and variance $\\sigma=15$. We use this to generate the data\n", "used in the bootstrap analysis. The bootstrap analysis returns a data\n", "set after a given number of bootstrap operations (as many as we have\n", "data points). This data set consists of estimated mean values for each\n", "bootstrap operation. The histogram generated by the bootstrap method\n", "shows that the distribution for these mean values is also a Gaussian,\n", "centered around the mean value $\\mu=100$ but with standard deviation\n", "$\\sigma/\\sqrt{n}$, where $n$ is the number of bootstrap samples (in\n", "this case the same as the number of original data points). The value\n", "of the standard deviation is what we expect from the central limit\n", "theorem." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Bootstrap Statistics :\n", "original bias std. error\n", " 99.9348 15.1379 99.9341 0.151076\n" ] } ], "source": [ "%matplotlib inline\n", "\n", "import numpy as np\n", "from time import time\n", "from scipy.stats import norm\n", "import matplotlib.pyplot as plt\n", "\n", "# Returns mean of bootstrap samples \n", "# Bootstrap algorithm\n", "def bootstrap(data, datapoints):\n", " t = np.zeros(datapoints)\n", " n = len(data)\n", " # non-parametric bootstrap \n", " for i in range(datapoints):\n", " t[i] = np.mean(data[np.random.randint(0,n,n)])\n", " # analysis \n", " print(\"Bootstrap Statistics :\")\n", " print(\"original bias std. error\")\n", " print(\"%8g %8g %14g %15g\" % (np.mean(data), np.std(data),np.mean(t),np.std(t)))\n", " return t\n", "\n", "# We set the mean value to 100 and the standard deviation to 15\n", "mu, sigma = 100, 15\n", "datapoints = 10000\n", "# We generate random numbers according to the normal distribution\n", "x = mu + sigma*np.random.randn(datapoints)\n", "# bootstrap returns the data sample \n", "t = bootstrap(x, datapoints)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see that our new variance and from that the standard deviation, agrees with the central limit theorem.\n", "\n", "We plot then the histogram together with a best fit for the data set." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEGCAYAAACUzrmNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAroUlEQVR4nO3dd3xUVfrH8c8TEoqhBEgIVYKChUVBQVFBSVBXQVCaLoiKi8jPttZ1V3/rrqiLrv50LYtdUVwVpIMIdkJRUUA6AlIE6b2EHvL8/rgTjRAyKXPnTHner9e8Zubmzp3vIRmeOefee66oKsYYY0xRElwHMMYYE/msWBhjjAnKioUxxpigrFgYY4wJyoqFMcaYoBJdByip1NRUzcjICPv77t27l+Tk5LC/b6hZOyKLtSOyxHI7Zs+evVVV00q7zagrFhkZGcyaNSvs75udnU1mZmbY3zfUrB2RxdoRWWK5HSKyuizbtGEoY4wxQVmxMMYYE5QVC2OMMUFZsTDGGBOUFQtjjDFBWbEwxhgTlBULY4wxQVmxMMYYE5QVC2OMMUFF3RncxkS0rKzCl0+eHN4cxoSY9SyMMcYEZcXCGGNMUDYMZUxpHG+4yZgYZT0LY4wxQVmxMMYYE5QVC2OMMUFZsTDGGBOU7eA2pgR++AEefRSqLL2Pkyqup1GljTSquIFGFTeQmrQLEdcJjfGHFQtjinEinSq89BIMGAB//Sskz/6RlQfqMGvzqaw6UIdVB2oD8N/Tn+CKmjPCENqY8LJiYUwQGzdC376wdSt89RWccgrw0fhj1vtmV1O6LHyMQU1e4OpaU8If1Bgf2T4LY4owbhycdRa0bFmgUBzH+dUW82nzv3Dn8j8xZONl4QtpTBhYz8KYQuTmJXD7/8Bnn8HIkdCmTfFe17zyCiY3v4dL5z/N3iMVua3eOH+DGhMmViyMKcTTP/+BJcDcuVC1aslee1ryz0xpcTeXzPMKxv0nfuBHRGPCyoqFMUdZmJPBM2uvYdbkkheKfCdV2sDUFndxybxnyDlSiQGKHSlloprtszCmgMN55bhhyYM80eh1GjYs27bqV9zK1LPuYsSWdnxgnQsT5XwrFiLSQEQmi8hiEVkkIncVsk6miOwSkbmB2z/8ymNMcTyxpjfp5XdwU52JIdlerfI7ef3UZ7j/fti7NySbNMYJP3sWucB9qtoUOA+4XUSaFrLeNFVtEbg96mMeY4o0Z09jBq3rwuunPB3SIaM21RbSrh088UTotmlMuPlWLFR1g6p+H3i8B/gBqOfX+xlTFofyEumz5AGePvkV6lfcGvLtP/kkvPIKrFgR8k0bExaiqv6/iUgGMBVopqq7CyzPBEYBa4H1wJ9VdVEhr+8P9AdIT09vOWzYMN8zHy0nJ4fKlSuH/X1DzdpRiGXLeHNsK1auq8k/b/vk115FUSdVLFtWsvc45RTef/9EFi2qysCBC39ZbL+PyBLL7cjKypqtqq1Ku03fi4WIVAamAANVdfRRP6sK5Klqjoh0BJ5X1SZFba9Vq1Y6a9Ys/wIfR3Z2NpmZmWF/31CzdhxrZstb6LTgcea16kftCjt+/UFR180uxcWPDuYl0WzmYP7T+AUurzkTgOyHH7bfRwSJ5XaISJmKha+HzopIEl7P4b2jCwVAwV6Gqk4UkZdEJFVVQz8OYEwhDh6EG5c8wHONX/xtoYCQXw2vQsJhnmv8Incvv4P51W+ifEJuSLdvjJ/8PBpKgDeBH1T138dZp3ZgPUTk3ECebX5lMuZob78NDSpspmetL8PyflfUnMHJldbzwtpuYXk/Y0LFz55FG+B6YIGIzA0s+1/gRABVfQXoAdwqIrnAfqCnhmMnijHAkSPw9NMwuOG7YT1h7tnGL3LB94Ponf55+N7UmDLyrVio6nSgyI+gqg4CBvmVwZiijBkDqanQtsKCsL7vKSes5aY6E3lw5c3cGNZ3Nqb07AxuE5dUvcNZ//pXN9NwPNTwv3y+oyVLl1YJ/5sbUwpWLExcmjwZcnLgyivdvH+VxP3c22AEI0bUdxPAmBKyiQRN/ChwdNOT857i/lqTSbh4krM4fWtP5OFvb2bDBqhTx1kMY4rFehYm7szZ05hFezOc72BOSdpL+/abefVVpzGMKRYrFibuPPVzT+6uP5IKCYddR6Fr13W8+qp3vocxkcyKhYkrq/bX5rPtrehfd4LrKABkZOyjWTMYMcJ1EmOKZsXCxJVnfr6G/nUnUDVxn+sov7jzTnj+ee8ILWMilRULEze2HKrG+5sv5s56o1xH+Y2OHWH7dvj2W9dJjDk+KxYmbvxnXTeuTpty7BxQjpUrB3fcAS+84DqJMcdnxcLEhZwceHn9lfy5QWRe3/SPf4RJk2D9etdJjCmcFQsTF4YNg7bVFtDkhHWuoxQqJQV69cIOozURy4qFiQtvvw19a7s7Aa847rgDO4zWRCwrFibm/fgjLF8Ol9f4znWUIjVtCmeeCcOHu05izLGsWJiYN2QI9O4NSQlHXEcJyg6jNZHKioWJaUeOwDvvQJ8+rpMUT4cOsGMHzJjhOokxv2XFwsS0yZMhLc0b3okG5crBzTd7+1iMiSRWLExMe/vt6OlV5Lv2Whg50nZ0m8hiU5SbmLV7N0yYAM895zpJEZYtg0ce+c2iE4EzzpjMpEnQpYuTVMYcw3oWJmaNGAHt23uXTo02vXvDu++6TmHMr6xnYWJP4CJHb895nvsbfABZXzsOVHI9esCf/ww7d3on7BnjmvUsTExavq8uy/bVp0ON6Jydr3p1uOQSGBVZcx6aOGbFwsSkIZsuo3f6F1FxbsXx2FCUiSRWLEzMyVNhyMbL6FP7E9dRyqRjR5g/H37+2XUSY6xYmBg0eedZ1EzaTfPKK1xHKZOKFaF7dxg61HUSY6xYmBj09sbLuDHKexX5bCjKRAorFiam7N4NH269gGtrfe46SkhceKF3RNT8+a6TmHhnxcLElNGjoV3KPNLK73IdJSQSErzexXvvuU5i4p0VCxNThg+HnrW+dB0jpHr3hvffh7w810lMPPOtWIhIAxGZLCKLRWSRiNxVyDoiIi+IyHIRmS8iZ/uVx8S+HTtg+nToVPMb11FCqlkzqFkTpk51ncTEMz/P4M4F7lPV70WkCjBbRD5T1cUF1ukANAncWgMvB+6NKbFx4+Dii6HKzv2uo5Rd4Cz0fNftuYZ3e51I5oYrHAUy8c63noWqblDV7wOP9wA/APWOWu0q4B31zABSRKSOX5lMbBsxAq65xnUKf/RK/5LRWy/kwAHXSUy8Css+CxHJAM4Cjp57oR5Q8JSjtRxbUIwJascOmDYNOnVyncQf9Sps5azKy5kwwXUSE698n0hQRCoDo4C7VXV3KbfRH+gPkJ6eTnZ2dugCFlNOTo6T9w21WG3Hxx/XpnnzmsyevQh69XIXrIRyatQgu5h5z6qfw4svbiY1dXHwlcMsVv+uopUf7RD18WK/IpIETAA+UdV/F/LzV4FsVR0aeL4UyFTVDcfbZqtWrXTWrFl+RT6u7OxsMjMzw/6+oRar7bjiCu+ooWuv5Zjx/kiW3asXmcU8RXvLoWo0WTSWDRugUiWfg5VQrP5dRavC2iEis1W1VWm36VvPQkQEeBP4obBCETAeuENEhuHt2N5VVKEw5jcCRWHn4WSmzfiAYbuvhtdjYOf2caSV38XZZ8Mnn9hFkUz4+TkM1Qa4HlggInMDy/4X70JgqOorwESgI7Ac2Af80cc8JkaN29aW9ilzqJIYu4UiX/fu3rTlVixMuPlWLFR1OiBB1lHgdr8ymPgwYnM7rk3/wnWMsOjaFf7+dzh0CMqXd53GxBM7g9tEtZ2Hk5m26ww614y+q+GVRt260LQpfBEftdFEECsWJqqN39aGrJS5cTEEla97dxg50nUKE2+sWJioNnxzJlfXmuI6Rlh16wbjx0NuruskJp5YsTBRa+fhZKbuOjNuhqDyNWwIGRkwJb5qpHHMioWJWuO3taF9yhyqJu5zHSXsevSwoSgTXlYsTNQasaVd3A1B5eveHcaMgSNHXCcx8cKKhYlKOTnlmLoz/oag8jVuDOnp8HV8Nt84YMXCRKWvv04lM2VeXA5B5cs/Qc+YcLBiYaLStGmpdE+L76sB9ejhFQu7gp4JB99nnTUm1PbtgzlzqtOpRWxdEa9YCkyS2BSovP1tZp7zJK1nv+Quk4kL1rMwUeeTT+DUU/dQI2mP6yjOdU+dyqgtF7mOYeKAFQsTdcaMgbZtt7iOERF6pE1h5JaL8PFKA8YAxSwWItJZRKywGOcOH4aPPoK2bbe6jhIRmldegaDMnes6iYl1xS0AfwB+FJGnROQ0PwMZU5QpU7zDRtPSDrmOEhFEoHvaNDsqyviuWMVCVa/Du4b2CuBtEflGRPqLSBVf0xlzlDFjvGm6za+6pU5lzBjXKUysK/bQUuD62SOBYUAdoCvwvYj8yadsxvxGXh6MHWvF4mjnVl3Cjh2wbJnrJCaWFXefxVUiMgbIBpKAc1W1A9AcuM+/eMb8auZMqFYNTj3VdZLIkiDKVVd5hdQYvxS3Z9ENeFZVz1DV/1PVzQCqug+4ybd0xhRgQ1DH17UrNhRlfFXcYrFRVX9zuqyIPAmgqnbNLuM7VSsWRcnMhKVLYf1610lMrCpusbi0kGUdQhnEmKL88APs3w8tW7pOEpnKl4cOHWDcONdJTKwqsliIyK0isgA4TUTmF7itAuaHJ6IxXq+iSxfvUFFTOBuKMn4K1rN4H+gMjAvc599aBg6nNSYsbAgquMsvhxkzYOdO10lMLApWLFRVfwJuB/YUuCEiNfyNZoxnzRpYvRouvNB1kshWubK37+Kjj1wnMbGoOD0LgNnArMD97ALPjfHd2LHQqRMk2hzJQdlQlPFLkR8/Ve0UuG8UnjjGHGvMGLjnHtcpokOnTnD33d7BAJUquU5jYkmRxUJEzi7q56r6fWjjmLhW4FoN+bYeqsr3C8dxaWHH45ljpKXBWWfB559D586u05hYEqxj/0wRP1OgfQizGHOMD7ddwKXlp1Cp44Df/qBXLyd5IlaBQtt1XXfG9D+Zzqc9BZMnOwxlYkmwYahjv+oZE0ajt15Iz1r2H15JdEmdzj9XX0duXoJdCtOETLBhqPaq+qWIdCvs56o62p9YxsCe3EpM2dmcd09/3HWUqNKw4iYaVNjM9F1nkOk6jIkZwb54tAO+xDu34mgKHLdYiMhgoBOwWVWbFfLzTLzzN1YFFo1W1UeDRzbxYtL21lxQdRHVEve6jhJ1uqZOZ8zWC61YmJAJNgz1cOD+j6XY9tvAIOCdItaZln/ElTFHG7O1Ld3SprmOEZW6pk7jigVP8JzaWe8mNIo7RXlNEXlBRL4Xkdki8ryI1CzqNYGJB7eHJKWJOwfzkpi0rTVXpX7lOkpU+l3yT5RPyGXOHNdJTKwQLcaV3kXkM2Aq8G5gUW8gU1UvCfK6DGBCEcNQo4C1wHrgz6q66Djb6Q/0B0hPT285bNiwoJlDLScnh8qVK4f9fUMtottR4Oo9MxY04P2PW/DC/R8WumpOjRpU3h7930X8bMcrI1tTvnY1+vb9yZftFxTRf1clEMvtyMrKmq2qrUq7zeIWi4VH/4cvIgtU9Ywgr8vg+MWiKpCnqjki0hF4XlWbBMvSqlUrnTUr/CePZ2dnk5mZGfb3DbWIbkeBwz/7LfkzTZNXc2+DEYWumt2rF5lDh4YrmW/8bMc3u5rS7+CLLCr0K1hoRfTfVQnEcjtEpEzForhTlH8qIj1FJCFwuwb4pLRvCt5lWlU1J/B4IpAkIqll2aaJDUc0gfHbLqBrqu2vKIvWVX9g505YssR1EhMLgk1RvkdEdgM3480TdShwG0ZgWKi0RKS2iLfrTUTODWTZVpZtmtgwfdcZ1KuwlUaVNrqOEtUSRG2uKBMyRRYLVa2iqlUD9wmqmhi4Jahq1aJeKyJDgW+AU0VkrYjcJCK3iMgtgVV6AAtFZB7wAtBTizMmZmLemC1t6Wa9ipDo1g1GjXKdwsSCYp/gKSLVgSZAxfxlR19qtSBVLXI+BlUdhHdorTG/UPUOmf3ojAddR4kJFz1yMavnj2L1+bfQsOKmX39g04CYEiruobP98I6G+gR4JHA/wL9YJl59n3MKFRIO87vkn1xHiQmJCXlcWfNrRm+xi4GYsinuDu67gHOA1YH5os4CdvoVysSv0VsupGvqdDuRLIS6p01l9FYrFqZsilssDqjqAQARqaCqS4BT/Ytl4pWdtR16F1f/noV7G7HxYHXXUUwUK26xWCsiKcBY4DMRGQes9iuUiU9L9jZgV24y51SxYz1DqULCYTrW+JaxW9u6jmKiWLGKhap2VdWdqjoA+DvwJtDFx1wmDo3Z6g1BJYgdFBdq3dKm2VCUKZPi9iwQkbNF5E7gTGCtqh7yL5aJR6O3XkjXtOmuY8Sky2t8x7e7T2f74Squo5goVdyjof4BDAFqAqnAWyLykJ/BTHxZswZW7q/DRdXmuY4Sk5LLHeDi6t/z4bYLXEcxUaq4PYvewDmq+nBg2vLzgOv9i2Xizdix0LnmNyQlHHEdJWZ1T5tmh9CaUitusVhPgZPxgArAutDHMfFq1CjsKCifXVHjG7J3NmdPbiXXUUwUCnZZ1f/gXRFvF7AoMFW5ApcC3/kfz8SD9ethwQK47IyZrqPEtJSkvbSptpBJ21tzjeswJuoEm+4jfy7w2UDB6ciyfUlj4tKoUdC5M1RYc9h1lJjXLXUao7ZcZMXClFiwy6oOyX8sIuWBUwJPl6qqfbJNSAwfDg88ADztOknsuyr1K/684lYOHICKFYOvb0y+4h4NlQn8CLwIvAQsE5GL/Itl4sW6dbBoEVx6qesk8SGt/C7OqvIjn33mOomJNsXdwf0M8HtVbaeqFwGXAc/6F8vEi5Ej4aqroHx510niR7fUaYwc6TqFiTbFLRZJqro0/4mqLgOS/Ilk4snw4XCNDaCHVfe0qXz4IRw86DqJiSbFvZ7FbBF5A3g38Lw3v+78NqZkAtfa/vlAGktmvc7F/+oBT+U6DhU/6lbYRvPm8PHHXq/OmOIobs/iFmAxcGfgthi41a9QJj6M3NKOLqlfUT7BCkW49ewJw4a5TmGiSdCehYiUA+ap6mnAv/2PZOLF8C2ZDMgYEnxFE3Ldu8Nf/wp790Jysus0JhoE7Vmo6hFgqYicGIY8Jk6sPpDOj/vq0z7le9dR4lJqKpx/Pnz4oeskJloUdxiqOt4Z3F+IyPj8m5/BTGwbueUiuqZNs7mgHLKhKFMSxd3B/XdfU5i4M2JzJo81Guw6Rlzr0gXuvBN27oSUFMdhTMQLNjdURbyd242BBcCbqmp7I02Z/LQ/nRUH6pKVMsd1lLhWrRpcfLE34++NN7pOYyJdsGGoIUArvELRAe/kPGPKZOSWdnRLnUZiQp7rKHGvZ08YOtR1ChMNghWLpqp6naq+CvQAbDJ8U2bDt2RyTa1s1zEMcMUV8O23sHmz6yQm0gXbZ/HLZIGqmisiPscxsW7VKu9IqHbV5rqOEt8CJ0YmAx0rPMSotvO5td54mDzZbS4TsYL1LJqLyO7AbQ9wZv5jEdkdjoAmtowY4V3kyIagIkfPWl8ybHN71zFMhCuyWKhqOVWtGrhVUdXEAo+rhiukiR3vvw/XpGW7jmEKuKzGTBbsbcTaA6muo5gIVtzzLIwps3nzYMcOaJcyz3UUU0CFhMN0Sf2K4VuyXEcxEcyKhQmbd96B66+HBFHXUcxRetX6gmGbrViY4/OtWIjIYBHZLCILj/NzEZEXRGS5iMwXkbP9ymLcy82F996DG25wncQUJitlDqsPpLNiheskJlL52bN4G7i8iJ93AJoEbv2Bl33MYhz79FNo1AhOOSX4uib8EhPy6JE2lQ8+cJ3ERCrfioWqTgW2F7HKVcA76pkBpIhIHb/yGLeGDIE+fVynMEXpWetL3nsP1EYJTSFEffzLEJEMYIKqNivkZxOAf6nq9MDzL4C/quoxF1USkf54vQ/S09NbDnMw+1lOTg6VK1cO+/uGmot27NmTSK9e5zF06AyqVMmFZcvKvM2cGjWovL2o7yLRIZLaoQrXPXo9Dz20mNNP31Oi19rnI7IU1o6srKzZqtqqtNss7kSCTqnqa8BrAK1atdLMzMywZ8jOzsbF+4aai3a89hp06ACdO7f1FjzySJm3md2rF5kxME9FpLXjttv6M29eS24t4aXN7PMRWfxoh8ujodYBDQo8rx9YZmKMDUFFjz59vOui79vnOomJNC6LxXjghsBRUecBu1R1g8M8xgfLl3u3yy5zncQUR/3rszhPZjC65UBvSpD8m4l7fh46OxT4BjhVRNaKyE0icouI3BJYZSKwElgOvA7c5lcW484778C110JSkuskprj61p7Emxs7uo5hIoxv+yxUtVeQnytwu1/vb9zLy/OKxZgxrpOYkuic+g23/ngPK/bX5eRK613HMRHCzuA2vpk2DapWhRYtXCcxJVEh4TC9a33O2xtt7ND8KiqOhjLRaUjPSfRJXoW0H+E6iimhP9b5mE4LHmdAxhDKic0QbKxnYXyydy+M2dqWa2t94TqKKYXmlVeQnrSDz3e0dB3FRAgrFsYXY8fC+VUXUadCZJxwZkqub51JDN7QwXUMEyGsWBhfvP023JD+qesYpgyurfU5n2w/h22H7dI1xoqF8cHSpTB/PnRJne46iimDlKS9XFFzBu9tusR1FBMBrFiYkHvpJejXDyqWOxx8ZRPR+taZxJsbOtrkgsaKhQmtPXvg3XfhlluCr2siX1bKHHblJjNnjuskxjUrFiak3n3Xmx2iQYPg65rIlyDKH+tMYvBg10mMa1YsTMiowqBBcMcdrpOYULqx9icMHQo5Oa6TGJesWJiQmTwZRKBdO9dJTCg1rLiJrCzvCDcTv6xYmJDJ71WIuE5iQu2ee+C55+DIEddJjCtWLExIrFkDU6bAdde5TmL8cMEFULMmfPih6yTGFSsWJiReeQWuvx5i4IqUphAiXu/i2WddJzGuWLEwZXbgALz5JtxuE87HtO7dYdUqmD3bdRLjghULU2bDh8PZZ0OTJq6TGD8lJcGdd1rvIl5ZsTBlZofLxo9+/WDiRFi3znUSE252PQtTJt99B9sWrOPy/7sBnrbrHsS6lBRv39SgQfDEE67TmHCyYmHK5D//gdvqjbML5MS6rKxfHt65vy6tZ7/EQ1/1JHnqJIehTDjZMJQptZ9+gkmToG9t+w8jnpxcaT0XpcxjiF12Na5Yz8IUX4FvlwADl97HrZW3Uz3J5oGIN/fWH0HfpX/hljxIsK+cccF+zaZUVu2vzegtF3JP/ZGuoxgH2lRbSLXEvXz0keskJlysWJhSeXxNb26rN44aSXtcRzEOiHi9i2eecZ3EhIsVC1Niq/bXZoz1KuJej7Qpv0zzYmKfFQtTYgNXX2e9CkNSwhEefRQefBC7kl4csGJhSmTl/jqM3dqWu61XYYBevbzrXHz9dU3XUYzP7GgoUyKPr7Z9FeZX5S7J4nE9jz+9dD8PfHbxb8+3mTzZXTATctazMMWW36u4p/4I11FMBLmi5gwqVzrEu5sudR3F+MjXYiEil4vIUhFZLiIPFPLzG0Vki4jMDdz6+ZnHlM3jq3tze72xdl6F+Q0R6N/tWx5edSMH85JcxzE+8a1YiEg54EWgA9AU6CUiTQtZ9QNVbRG4veFXHlM2K1di+yrMcZ3ReBPNklfxyvorXUcxPvGzZ3EusFxVV6rqIWAYcJWP72d8NHAg1qswRXr8pDd4fPW17Mmt5DqK8YGoT8e8iUgP4HJV7Rd4fj3QWlXvKLDOjcATwBZgGXCPqv5cyLb6A/0B0tPTWw4bNsyXzEXJycmhcgxcBq407Vi+PJn772/OO/94nyrJh3xKVjI5NWpQeft21zHKLNba8fjgLOql7aJP5+/hlFNcxyqxWP6cZ2VlzVbVVqXdputiURPIUdWDIvI/wB9UtX1R223VqpXOmjXLl8xFyc7OJjMzM+zvG2olbUdeHrRpAzfdBP3eywr+gjDJ7tWLzKFDXccos1hrx8r9dThn9sssObcPaV+NdR2rxGL5cy4iZSoWfg5DrQMaFHheP7DsF6q6TVUPBp6+AbT0MY8phTfe8HZg9u3rOomJBidV2sC16V/w+JrerqOYEPOzWMwEmohIIxEpD/QExhdcQUTqFHh6JfCDj3lMCW3eDA89BK+8YjOLmuJ7qOG7vLPx9yxd6jqJCSXf/gtQ1VzgDuATvCIwXFUXicijIpJ/yMSdIrJIROYBdwI3+pXHlNxf/gI33ABnnuk6iYkm6eV3MCBjCH37wpEjrtOYUPH1DG5VnQhMPGrZPwo8fhB40M8MpnSmTIEvv4TFi10nMdHo9npjGX7kTgYNgrvucp3GhIINLphjHDoEt94Kzz8PMXBgiHEgQZTBg+Gxx2DFCtdpTCjY3FDmGM88AyedBF26uE5iolmT/lk8mHI1N7W8gC9b3EuCBI68tDmjopIVC/Mbq87rxTOzX2Fmy1uQ9htdxzFR7u76oxixOZNX13fm1nrjg7/ARCwbhjK/UIU7l/+J+xoMp1ElKxSm7MpJHoNPe4q/r+rL6gPpruOYMrBiYX7x8suw5kA69zUY7jqKiSFNk1dzX4Ph3Lz0PrtIUhSzYmEAmDEDBgyAUb/7B+UTcl3HMTHm/gbD2H64KoM3dnQdxZSSFQvD5s1w9dXw5pvQ+IT1ruOYGJSYkMdbpz3JAytvZuVK12lMaVixiHO5udCzJ/TpA507u05jYtkZlVcxIGMInTrBrl2u05iSsqOh4lWWNyng31b0JzGnMY/kPQBZeUFeZEzZ3F5vLEua3cU118BHH0Gi/Q8UNaxnEcdGb7mQYZuzeP/0f/722snG+OjZZ73JKe++23USUxJWLOLU0n0NuGXZPYz83QBSy+92HcfEkcRLs/hgbycmv/UTg5o87/VysyJn+ntTOCsWcWjXLui28FH+2Wgw51S1qUFN+FVL3MuEMx5k4Jrr+HjbOa7jmGKwYhFn9uxJ5JJL4NLqs7i5zgTXcUwca1RpIyN/N4AbljzIwpwM13FMEFYs4si2bXDvvc256CJ4tvGLiLhOZOJdm2oL+ffJL9F54eOst6O2I5oVizixZYs3LHzOOTt4+mmsUJiIcV3tz7mt7jjatMEumBTBrFjEgU2bvEJx1VVw880rrVCYiHP/iR/w8MPQrh18843rNKYwVixi3IYLupN50mqu2fsWj03PQn5cZkeemIh0443w1lvel5oPP3SdxhzNikUMW74c2s19jhvSP+UfGe+4jmNMUB06wIQJ0L8/vPGG6zSmICsWMeq//4Xzz4d764/gwYbvu45jTLGdey5MnQpPPAGPPorNVBshrFjEmN274brrvA/aF1/ALfWsP2+iT5Mm8NVXMH48dO0KGza4TmRsZpYY8t13cO21cPHFMGsWnHCC60TGlMBR+9JqA18lJzHwzE9p3hyeesqb8NIO0HDDehYxIC/P+yB17gxPPgmvvmqFwsSGCgmHefRR+PRTeP556NgR1qxxnSo+WbGIYqowbhy0aAETH5/LzJP+QPdBWb/OtWNHPZkY0aKF13Nu2xZatvS+EOXZ3JdhZcNQUUjV2x/xt7/BgQMwcCB0euYe656b2BT40pME/A3o0iiDm+6/n5dfbsoDD0CPHjbVeThYzyLKfP01tG8Pt90G99wDc+Z4w09WKEy8+F3yT3xz9u0MHAgvvginngqvvAL797tOFtusWESB7du9D0ObNtCrl3e00+LF3hXuEuw3aOKQCFzxdBbTErN4p9qfmPjQV5yUso0nTnqdbdtcp4tN1nmLUAfb/Z6J21rz302X8uWOs7msxkweTP+UyzJmkvTuEXjXdUJjIkObagsZf8ZDLMzJ4P9+7slJJ0Hr1tCtG3TpArVru04YG6xYRAhV74zrKVO826SvR9AseRXX1/6Mt057imqJe11HNCaiNav8E0NO/xd7jzzHx+vPZfTDF/Lgn1rTLHkV3VOn0nHiHTRpYkO2pWXFwpF9+7wZNr/9w7+Zsqs5U3Y2JwGlXco82qXM45+tZtKw4ibXMY2JOsnlDtA9bSrd06ZyMC+Jz3e0ZNSWi/j3JbB3r9fraN0azjvPO1u8enXXiaODr8VCRC4HngfKAW+o6r+O+nkF4B2gJbAN+IOq/uRnpnA6cADWr4d167xew+LF8MMP3v2GDdC4MbTc3ZTfV5/FwEZv0qjiBvvWY0wIVUg4zBU1Z3BFzRnAU6w/WJNv15zOtwtP51/Pnc6sPaeSduIJNGnifR5F6pOT4z3OyICKFV23IHL4VixEpBzwInApsBaYKSLjVXVxgdVuAnaoamMR6Qk8CfzBr0wllZvrHWGxbx9s3FiRxYu9x3v3wq47/sbO3MrsyK3Cztxkdhyuwo7cKmw6VJ11h1JZf7AmOUcqUaf8dupW2MrJVzajaVPo1w+aNoWTTgoc7pf1pOtmGhM36lbYRte06XRNmw5Abl4CK1//guXLvS902dkVefFF7/Hq1d7Jrenpv72lpUG1alC16q/3VatClSpQqZJXYPLvK1SInWEvP3sW5wLLVXUlgIgMA64CChaLq4ABgccjgUEiIqqhnzpsxw649MSlHCGBI5pArpYrcF+Ow1qOQ5rEobzEX+4V4YSEg5xQ7gAJVX9Hjf0/cULCQSqVO0i1cp2onrSHlMQcqifm0LDiJponrqB2+e3UrbCNuuW3UjNpNwkSaMrawO3TULfMGFNaiQl5nPI/WZwSeH5mr15kDh0K9SGvnvzyBXDT7ups2lqdzXOrs+VwNVblJrP7yAnsyk1md+Dxnnqns3+/N6KQf3/4sFcwkg7vJUlySZIjJCXkkiS5JMoREuUICSjlJI9ycsS7Jw8RJQElQfIQvMciypOfnMW557r5txIf/l/2NizSA7hcVfsFnl8PtFbVOwqsszCwztrA8xWBdbYeta3+QP/A01MBF9fTSgW2Bl0r8lk7Iou1I7LEcjsaqmpaaTcYFTu4VfU14DWXGURklqq2cpkhFKwdkcXaEVmsHcfn5yld64AGBZ7XDywrdB0RSQSq4e3oNsYYE0H8LBYzgSYi0khEygM9gfFHrTMe6BN43AP40o/9FcYYY8rGt2EoVc0VkTuAT/AOnR2sqotE5FFglqqOB94E/isiy4HteAUlUjkdBgsha0dksXZEFmvHcfi2g9sYY0zssGnojDHGBGXFwhhjTFBxXyxE5C4RWSgii0Tk7sCy5iLyjYgsEJEPRaRqEa8vJyJzRGRC2EIXnqPU7RCRFBEZKSJLROQHETk/rOGPzVOWttwTeN1CERkqImGbsEFEBovI5sD5Q/nLaojIZyLyY+C+emC5iMgLIrJcROaLyNnH2WbLQJuXB9b3/XzgULdDRE4QkY8Cf1+LRORfR68TDe04atvjC27XTz79XZUXkddEZFng99I9aBBVjdsb0AxYCJyAt7P/c6Ax3pFc7QLr9AUeK2Ib9wLvAxOitR3AEKBf4HF5ICUa2wLUA1YBlQLPhwM3hjH7RcDZwMICy54CHgg8fgB4MvC4IzAJEOA84NvjbPO7wM8lsH6HaGtH4HeZVeDva1o0tqPANroFPvML/coehr+rR4B/Bh4nAKlBc4SjsZF6A64G3izw/O/AX4Bd/LrzvwGw+Divrw98AbTHbbEodTvwzm1Zlb+e61sZ21IP+BmogVdoJgC/D3P+jKM+1EuBOoHHdYClgcevAr0KW6/AsjrAkgLPewGvRls7Ctn288DN0dgOoDIwHWhKmIqFT+34GUguSYZ4H4ZaCFwoIjVF5AS8qtwAWIQ3bxV4/3k1OM7rn8P7j8z1pePL0o5GwBbgrcBw2hsikhyO0MdR6rao6jrgaWANsAHYpaquZ+NKV9UNgccbgfTA4/zClm9tYFlB9QLLi1onXMrSjl+ISArQGe9LlgtlbcdjwDPAPt8SFk+p2xH4HQA8JiLfi8gIEUkniLguFqr6A95Mt58CHwNzgSN4wxy3ichsoApw6OjXikgnYLOqzg5b4OMoSzvwvoGfDbysqmcBe/G6tU6U8XdSHa+gNALqAskicl14kgen3le6qD9WvbTtEG+WhqHACxqYYNSlkrZDRFoAJ6vqGN9ClUIpfh+JeKMiX6vq2cA3eF+yihTXxQJAVd9U1ZaqehGwA1imqktU9feq2hLvj3tFIS9tA1wpIj8Bw4D2IuLsYqdlaMdaYK2qfht4PhKveDhThrZcAqxS1S2qehgYDVwQvuSF2iQidQAC95sDy4s7HU79IOuES1nake814EdVfc6vkMVQlnacD7QKfOanA6eISLavaY+vLO3YhtczGh14PoJifObjvliISK3A/YkEdlwVWJYAPAS8cvTrVPVBVa2vqhl4Z55/qarOvsWWoR0bgZ9F5NTAoov57TTyYVfatuANP50XOPpG8NryQ3hSH1fBKW36AOMKLL8hcPTKeXhDZhsKvjDwfLeInBdozw0FXh9upW4HgIj8E2//2N1hyFqUsvw+XlbVuoHPfFu8LzGZ4Yl9jLK0Q4EPgczAouJ95sO1gyZSb3hHZiwG5gEXB5bdBSwL3P7FrztW6wITC9lGJg53cJe1HUALYBYwHxgLVI/itjwCLMHb9/FfoEIYcw/F21dyGK/HdhNQE298/ke8I7tqBNYVvIuDrQAWAK0KbGdugcetAm1ZAQwiDAcihLodeN9uFa9wzw3c+kVbO47adgbhOxrKj7+rhsDUwGf+C+DEYDlsug9jjDFBxf0wlDHGmOCsWBhjjAnKioUxxpigrFgYY4wJyoqFMcaYoKxYGGOMCcqKhTHGmKCsWBhTRiJyTuDaARVFJDlwzYZmrnMZE0p2Up4xIRCYzqIiUAlvrq0nHEcyJqSsWBgTAiJSHu8CTQeAC1T1iONIxoSUDUMZExo18S6MUwWvh2FMTLGehTEhICLj8aaqb4R3ZbI7HEcyJqQSXQcwJtqJyA3AYVV9X0TKAV+LSHtV/dJ1NmNCxXoWxhhjgrJ9FsYYY4KyYmGMMSYoKxbGGGOCsmJhjDEmKCsWxhhjgrJiYYwxJigrFsYYY4L6f+GEyuj6OBONAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_47_0.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# the histogram of the bootstrapped data (normalized data if density = True)\n", "n, binsboot, patches = plt.hist(t, 50, density=True, facecolor='red', alpha=0.75)\n", "# add a 'best fit' line \n", "y = norm.pdf(binsboot, np.mean(t), np.std(t))\n", "lt = plt.plot(binsboot, y, 'b', linewidth=1)\n", "plt.xlabel('x')\n", "plt.ylabel('Probability')\n", "plt.grid(True)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The bias-variance tradeoff\n", "\n", "\n", "We will discuss the bias-variance tradeoff in the context of\n", "continuous predictions such as regression. However, many of the\n", "intuitions and ideas discussed here also carry over to classification\n", "tasks. Consider a dataset $\\mathcal{L}$ consisting of the data\n", "$\\mathbf{X}_\\mathcal{L}=\\{(y_j, \\boldsymbol{x}_j), j=0\\ldots n-1\\}$. \n", "\n", "Let us assume that the true data is generated from a noisy model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{y}=f(\\boldsymbol{x}) + \\boldsymbol{\\epsilon}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $\\epsilon$ is normally distributed with mean zero and standard deviation $\\sigma^2$.\n", "\n", "In our derivation of the ordinary least squares method we defined then\n", "an approximation to the function $f$ in terms of the parameters\n", "$\\boldsymbol{\\beta}$ and the design matrix $\\boldsymbol{X}$ which embody our model,\n", "that is $\\boldsymbol{\\tilde{y}}=\\boldsymbol{X}\\boldsymbol{\\beta}$. \n", "\n", "Thereafter we found the parameters $\\boldsymbol{\\beta}$ by optimizing the means squared error via the so-called cost function" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "C(\\boldsymbol{X},\\boldsymbol{\\beta}) =\\frac{1}{n}\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2=\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right].\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can rewrite this as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\frac{1}{n}\\sum_i(f_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2+\\frac{1}{n}\\sum_i(\\tilde{y}_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2+\\sigma^2.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The three terms represent the square of the bias of the learning\n", "method, which can be thought of as the error caused by the simplifying\n", "assumptions built into the method. The second term represents the\n", "variance of the chosen model and finally the last terms is variance of\n", "the error $\\boldsymbol{\\epsilon}$.\n", "\n", "To derive this equation, we need to recall that the variance of $\\boldsymbol{y}$ and $\\boldsymbol{\\epsilon}$ are both equal to $\\sigma^2$. The mean value of $\\boldsymbol{\\epsilon}$ is by definition equal to zero. Furthermore, the function $f$ is not a stochastics variable, idem for $\\boldsymbol{\\tilde{y}}$.\n", "We use a more compact notation in terms of the expectation value" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathbb{E}\\left[(\\boldsymbol{f}+\\boldsymbol{\\epsilon}-\\boldsymbol{\\tilde{y}})^2\\right],\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and adding and subtracting $\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]$ we get" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathbb{E}\\left[(\\boldsymbol{f}+\\boldsymbol{\\epsilon}-\\boldsymbol{\\tilde{y}}+\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2\\right],\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "which, using the abovementioned expectation values can be rewritten as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathbb{E}\\left[(\\boldsymbol{y}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2\\right]+\\mathrm{Var}\\left[\\boldsymbol{\\tilde{y}}\\right]+\\sigma^2,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "that is the rewriting in terms of the so-called bias, the variance of the model $\\boldsymbol{\\tilde{y}}$ and the variance of $\\boldsymbol{\\epsilon}$." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Error: 0.01312157412031145\n", "Bias^2: 0.012073649480472317\n", "Var: 0.0010479246398391328\n", "0.01312157412031145 >= 0.012073649480472317 + 0.0010479246398391328 = 0.01312157412031145\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABa3klEQVR4nO2dd3xb1Rn3v0fLlne8ZxJnELIHAQIJM0CYIQVCofBSyiqllJa2lND2hZD2bQO0UGhLIaV005JSCGGmNAkjgTTLibPINon3lretcd4/riRLsiTLsWxL9vl+PnyQ7j269/hG+t3nPucZQkqJQqFQKKIf3VBPQKFQKBThQQm6QqFQDBOUoCsUCsUwQQm6QqFQDBOUoCsUCsUwwTBUJ05PT5djx44dqtMrFApFVLJjx45aKWWGv31DJuhjx45l+/btQ3V6hUKhiEqEEF8E2qdcLgqFQjFMUIKuUCgUwwQl6AqFQjFMGDIfuj+sViulpaV0dHQM9VRGDLGxseTn52M0God6KgqFop9ElKCXlpaSmJjI2LFjEUIM9XSGPVJK6urqKC0tpbCwcKino1Ao+klECXpHR4cS80FECEFaWho1NTVDPRXFMGVNURlPrTtIeWM7uSlmHlo0iSWz84Z6WsOWiBJ0QIn5IKOut2KgWFNUxiOv76HdagegrLGdR17fA6BEfYBQi6IKhWJAeGrdQbeYu2i32nlq3cEhmtHwJ+IsdIVCETn0x2VS3tjep+2K/qMsdB+ee+45Jk+ezC233MKaNWtYsWJF0PHf//732bBhwyDNTqEYPFwuk7LGdiTdLpM1RWUhfT43xdyn7Yr+owTdh+eff54PPviAv//97zz55JPcd999Qcd/61vfYuXKlYM0O4Vi8Oivy+ShRZMwG/Ve28xGPQ8tmhS2OSq8iViXy+Nv7WN/eVNYjzklN4nHrpkacP+9997LsWPHuOKKK7j11luJiYkhPT0dgGuvvZbrr7+e2267jRdffJGPP/6Yv//974wZM4a6ujoqKyvJzs4O63wViqGkvy4Tl2tGRbkMHhEr6EPBCy+8wPvvv8/GjRt56623mDNnjnvfqlWrmD9/PoWFhfzyl79ky5Yt7n1z5sxh8+bNXH/99UMxbYViQMhNMVPmR7z74jJZMjtPCfggErGCHsySHgwqKirIyOiuUJmVlcWKFSu46KKLeOONN0hNTXXvy8zMpLy8fCimqVAMGA8tmsQPXiumy+5wb1Muk8gmYgV9qDGbzVgsFq9te/bsIS0trYd4d3R0YDarhR7F8GLJ7Dw2fF7N2t3a9z0nKZaHrzg9qMWtEomGFrUoGoDJkydz5MgR9/utW7fy3nvvUVRUxC9+8QuOHz/u3nfo0CGmTZs2FNNUKAYUg6478ez3X53bq5j3JypG0X+UoAfg/PPPp6ioCCklnZ2d3H333bz88svk5ubyy1/+kjvuuAMpJVarlSNHjjB37tyhnrJCEXYOVDaTmRgDwMn6tqBjVSLR0KME3YeSkhLS09OJi4vjkksuYf369cTExLB79273IunixYvZuHEjQgjefvttbrjhBgwG5b1SDC+sdgdHq1u4dEoWACd6EXSVSDT09CroQoiXhRDVQoi9QcZcKITYJYTYJ4T4KLxTHDp++MMf0tYW/Etss9n43ve+N0gzUigGj+O1rXTZHZw5NpWUOGOvgq4SiYaeUCz0PwGXB9ophEgBngcWSymnAkvDMrMIICsri8WLFwcds3TpUlJSUgZnQgrFIHKgQssDOT0nkYJRcZxsCG5pq0SioadXP4GU8mMhxNggQ74CvC6lPOEcXx2muSkUiiHkYGUzBp1gXHoCo1Pj2F8RPNFPJRINPeFw/J4GGIUQHwKJwLNSyr+E4bgKhWII+byymQmZCZgMOgpS4/jP/krsDole511yWYUqRg7hEHQDcAawEDADnwkhtkgpD/kOFELcA9wDMHr06DCcWqFQDBSfVzRxVqGWQDc6NQ6rXVLZ1EGeh09c1TyPLMIR5VIKrJNStkopa4GPgZn+BkopV0kp50op53pmYUYSer2eWbNmMXXqVGbOnMkvf/lLHA5H0M+UlJTwyiuvDPjc7rrrLvbv3x90zJo1a3odo1D0hqXdSrmlg0nZSYAm6NAzdPHxt/apUMUIIhyC/iawQAhhEELEAWcDB8Jw3CHBbDaza9cu9u3bxwcffMB7773H448/HvQzgyXoL730ElOmTAk6Rgm6IhwcrGwGtAVRgIJUzSr3jHRZU1RGQ5vV7+dVqOLQEErY4j+Az4BJQohSIcSdQoh7hRD3AkgpDwDvA8XAVuAlKWXAEMdwsqaojPkrN1C47B3mr9wQ9oy0zMxMVq1axW9+8xuklJSUlHDeeecxZ84c5syZw6effgrAsmXL+OSTT5g1axbPPPNMwHGelJSUcPrpp3PLLbcwefJkbrjhBneI5Pr165k9ezbTp0/njjvuoLOzE4ALL7yQ7du3A5CQkMCPfvQjZs6cybx586iqquLTTz9l7dq1PPTQQ8yaNYujR4/y3HPPMWXKFGbMmMFNN90U1uujGD74/pZe3XYCgNOzNUHPTTGjE94WejArvC+hip9XNvHVl7fS1mU7xdkrXPQq6FLKm6WUOVJKo5QyX0r5BynlC1LKFzzGPCWlnCKlnCal/NWAztjJYKUZjxs3DrvdTnV1NZmZmXzwwQfs3LmTV199lQceeACAlStXct5557Fr1y4efPDBgON8OXjwIPfddx8HDhwgKSmJ559/no6ODm6//XZeffVV9uzZg81m43e/+12Pz7a2tjJv3jx2797N+eefz+9//3vOPfdcFi9ezFNPPcWuXbsYP348K1eupKioiOLiYl544QU/s1CMdPz9ltbsKifOpCc7KRYAo15HborZy0IPZoX3JVTx3T2VfHSohl0nGk/1T1A4idpM0aFIM7Zardx9991Mnz6dpUuXBnRthDquoKCA+fPnA3DrrbeyadMmDh48SGFhIaeddhoAX/3qV/n44497fNZkMnH11VcDcMYZZ1BSUuL3HDNmzOCWW27hb3/7m8pmHcEcqW7G4ZB+9/n7LdkdEptdejURH50a52WhB7LCDTrtmKE+Oe8v14rg7Qtz/4ORSNQK+mClGR87dgy9Xk9mZibPPPMMWVlZ7N69m+3bt9PV1eX3M6GO8/yx+HsfDKPR6B6v1+ux2fw/rr7zzjt885vfZOfOnZx55pkBxymGL28Xl3PJ0x/z0aEav/sD/WY8y+YCFIyK40R999iHFk3CpPeWEAHYHPTpyXlvmSbkvcW5K3onagV9MNKMa2pquPfee7n//vsRQmCxWMjJyUGn0/HXv/4Vu12zahITE2lubnZ/LtA4X06cOMFnn30GwCuvvMKCBQuYNGkSJSUl7kqPf/3rX7ngggtCnrPnXBwOBydPnuSiiy7iiSeewGKx0NLSckrXQhGdNLR28dib+wAobfCfuh/oN5MU6/1ENzotjtqWTreve8nsPC5IbyKWTvaYvsaW2G9zhWl3j+MEe3Kuae6ksqkDgH3lFr9jFKETtYI+UGnG7e3t7rDFSy65hMsuu4zHHnsMgPvuu48///nPzJw5k88//5z4+HhAc2vo9XpmzpzJM888E3CcL5MmTeK3v/0tkydPpqGhgW984xvExsbyxz/+kaVLlzJ9+nR0Oh333ntvyPO/6aabeOqpp5g9ezaHDx/m1ltvZfr06cyePZsHHnhAlSkYYfzknf1Y2rVIlNoW/0+K/n5LoLldPC3rAnfoomaldxatZmuVYJFuO4m6TrKp4bfiCS7TbSOXWo7HfIUdpntYrNsU8CnAJeJnjU3laE0rHVb/xo8iNKLWqTpQacaBrGmAiRMnUlxc7H7/xBNPAJr7Y8OGDV5j/Y3zxWAw8Le//a3H9oULF1JUVNRj+4cffuh+7Wlp33DDDdxwww0AzJ8/38tnv2nTpoB/jyL6CZal+dGhGl7fWcb9F03gla0nqG3p9HsM1/jla/fR2N4dhtjaZWfTG89z2X/+TVx7JaPj5gHf4mR9G5OyE/nw/dew8FWW6De7PyMEFIpKPnSmoqTpWnjK+CKpRhNwVY9zu/zmS+fms7Wkns8rm5lVkBKGKzMyiVpBB9WvUDGyCZalee6ENH74+h7GZcRz/8UTWLevkroAFjpov6Wn1h30EvTLxFZWiFXEtWufK2jVopFP7N3MvpSFPNt0AWlYWKDb43WsFNFCFybaiSGOTmKEncdsv4Jn/g0LH4UZN7rH7i2zMDYtjnnj0gDYX96kBL0fRLWgRzNjx45l795BCddXDFMCRXotX7sPh5R02By8ctfZxBr1pCfEBLTQXfi6RX5s+DtxovsmkEoz8bTz4q4uflK0iWQy+KnxJYzCew6paGs49SQSh3ZOAWA5CW85Q3idor633MKM/BTyR5lJjDV4+dEtbVZ+tf4Q3730NBJjjSFfl5FM1PrQFYqRTiC/dGO7lck5Sbz37fOYO1arxZKWYKKuNbCFDtri6GLdJi7TbaOAagp03lExQsBpopQGh5l7zh/HR9fD1fqtPY6TIjR3YKNM6HkSazusXwFogn2yvh0BLHhiI80dNlZvP+n22/996xf8cXMJW47VB523ohtloSsUUUpuipkyP6KebDbyz3vmeYXBpifEUNscxEJ/+7ts6ngZjJIvd/1fckUt/qJoXzQ9DYm5ZF6xBZgMlVtg+8tAd4x7qtAs9AZ/gg5gKQW6F0T/s6/KHSJptUuW/bsYKSWrt50EcEfBKHpHWegKRZQSKDpl+TVTeuQ0pCeYaO60+Y8iefu7sP0PCCRCQJlMJ1fU+T1npqmLzMu+273h6qfhulWQXODcIBjl4XLx5LAjj+dt1+BI0sbudQq6b7x7h83B/3v3ACV1WphllUUJeqgoC12hiFJ8I730OsGEzAS+NCe/x9i0BK3Rc31rV8+48x1/cr+0S0ElqX4EXUByfo9FTUB777EtpaUTfvpfGmfcA4d2aW4W4DX7ebxoX0zC6Ou4je6EIn/UtnSRGGPAaNApC70PKAvdB1f53GnTprF06dJee4oG4/bbb+e1114L4+wUCm+WzM5j87KL2fv4IhxSctnUbL/j0p2C7ndhVHZb7dWMwo6eXFHbvT+5AJY3woN7e4q5H1LM2gJmfco0uOY5p/UuqDXmAvCz3WaO17ayr9xCrMG/BAlg8axcCkaZqVKCHjJK0H1wlc/du3cvJpOpR0ErlTqviET2lFlwSJgdIOQvLcEE4D90UXS7bcqlFj7ottCNZs0q7wMGvY6kWAONbV3aDeDBvbC8kZrRV1CQaibGoOfb/yziWG0rF56e6ddtJIEvn1lAVlIslcrlEjLRLejFq+GZabA8Rft/8eqwHv68887jyJEjfPjhh5x33nksXryYKVOmYLfbeeihhzjzzDOZMWMGL774IgBSSu6//34mTZrEJZdcQnW1aq+qGBx2nWwEYEZ+st/96fGahV7jz0I/43b3yzKZDkCeqAVjvGZhh2CV+5Iab+pRK722uZPTMhNZce1UikstSAlLz8jn59dNJy/FjADiTJq4T85JYnpeMjnJscrl0gei14devFqLaXX65/zFuPYHm83Ge++9x+WXXw7Azp072bt3L4WFhaxatYrk5GS2bdtGZ2cn8+fP57LLLqOoqIiDBw+yf/9+qqqqmDJlCnfccUe/56JQ9MauE42MTo1z+8p9SU8MYqFf/bT2/x1/clvoOXOugiVPnfJ8UuJMNLR5n6u2pZPpecksnpnLun2VvLe3kul5yWSWrGVJzAqILeWvxhv4v11f4stz8xFCkJUcS3OHjbYuG3Gm6JWrwSJ6r9D6Fd1i7sIV49oPQXfVcgHNQr/zzjv59NNPOeussygsLATgP//5D8XFxW7/uMVi4fDhw3z88cfcfPPN6PV6cnNzufjii095HgpFX9hd2uiOOfdHnMmA2ainLlBy0dVPw9VPU/7mXpKKykjsh5gDjIozej0NOBySutYu0hNNCCH4xdKZ3DG/kMyStV6G2aKuD9hnTOP62Fag0F2PvdLSwbiMAGGQCjfRK+jOWNaQt4eIy4fui2eBLSklv/71r1m0aJHXmHfffbdf51YoToWqpg4qLB2BU+aLV8N7D5Nue5TaTz+Cff+EK57wa/iUN3aEpWLpqHgTh6q66w01tHVhd0gynE8QcSaDdgN6xtswyxSNrNS/AB8XwBk3dgt6kxL0UIheH3pyz9CsoNvDyKJFi/jd736H1ar5CA8dOkRrayvnn38+r776Kna7nYqKCjZu3Djgc1EoipydfvwKevFqWHMftNeThoU6kqC9Ht78pt81J1eRr/4yysfl4qr0mJ7o4xLqxTDLStYEXUW6hEb0WugLH/X2ocMprcifCnfddRclJSXMmTMHKSUZGRmsWbOGL33pS2zYsIEpU6YwevRozjnnnAGfi2Lk4Vth8fScRAw6wdTcpJ6D168Ah2Z4pAsLpTJD227v8uueLLe0M2dMSr/nmBpvoq3LTofVTqxR7w6XTPf18Sfna+tfvjgNs26XS/A6NAqNXgVdCPEycDVQLaWcFmTcmWjNpG+SUg588LXri7h+hXY3D5T00Ef8NYC48MILufDCC93vdTodP/vZz/jZz37WY+xvfvObfp1foQiGvwqLFZZ28lLMxPoJ//O0gNNFE7sd4/3uA2jttNHYZg2LhZ4Sp8WiN7ZZyU7WU9McQNB7McziYwwkxhiUhR4ioVjofwJ+A/wl0AAhhB54AvhPeKYVIj4ZagpFtBOsvjn4r7DokFoGqF88LOA0mqgnCYcU6ITs4Z6ssGiimhcmlwtovvPs5Fi3hZ7h63IJwTDLSlax6KHSq6BLKT8WQoztZdi3gH8DZ4ZjUgrFSCRYfXOXqAeqsNjaFaAxy8JHNR+6w0q6sGBHTyMJJIsO9D7uybJGTTTD5UMH3H70mpZOTM6Eox70YphlJ6lY9FDp96KoECIP+BLwuxDG3iOE2C6E2F5T479hrZT+O5MrBgZ1vSOHQPXNPftxBhLbTF/L18WMG9k26/9RLxNIQ6udcsyRzcP2r7PGPt9rqOtmEZ4oF83l0tDqbH/X3EV6gqlPjdBdZCXFKpdLiIQjyuVXwMNSSkdvA6WUq6SUc6WUczMyMnrsj42Npa6uTonMICGlpK6ujtjY2KGeioLA1rfndn8VFnUCfnjl5IDH/c7+iczpXMUDtm8BcIP1cV7rOrdH4+byxnZ0ArIC3Rz6gD8LvUeES4hkJ8dQ3dyJ3aF0oTfCEeUyF/in886bDlwphLBJKdf09UD5+fmUlpYSyHpXhJ/Y2Fjy8wc+1FPRO4Hqm3tazJ4VFl1jbztnbNBWjMFuFMWljYxJiyfZbKSssZ3spFgM+v7bea5F0Qanb7+2uZOc5FMzHLKTYrE7JLUtnWQlKeMjGP0WdClloeu1EOJPwNunIuagNVt2ZWMqFCONhxZN8vKhA5iNeh5aNMlrnKuX7tIXPqXC0sGPrwpsnUPgG0VmUgzX/nYzuclmlp6Rz9vFFXTZHMxfuaHfDddjDHriTXp3PRdX2v+pkJ2s3dAqLR1K0Huh11uxEOIfaOGIk4QQpUKIO4UQ9woh7h346SkUI4cls/O8ClXlpZj5+XXTvYR1TVEZ81duYOyyd9hW0sDcMaMw7HstaJE6f24as1HPhadlIiU0d1j51frDdNk0r6lrMdbVCu5UGRVvorGtyyvt/1TwzBZVBCeUKJebQz2YlPL2fs1GoRjhuKxvf/hGwQDIfW9gO/wiBrtT7AIUqYsx6NyfM+kFP79uOuv2VZKdFIsAmjq8y0K7FmP7Y6WPijNR39blTvvvEYMeIlnJ2ufUwmjvRG/qv0IxwvAXBfOo+EO3mLvwaMTsugk0tneXsrU5JHaHZNORWs4/LT2g5RvI9x4qKXFGGtqs7rT/HjHoIZIeH4NBJ1QseggoQVcoogRfgV0otpEqemY2A+4s0ECJSD9/7wDNHTbOPy0jYJhif8MXU+NNNLR2BU77DxGdTpCZGKNcLiGgBF2hiBJ8BfZx418JGNbtzAINZGXXtnShE7BgQnpAH7vvYmxfcRXoCpj23weyklUseigoQVcoogRf4fXq++mLMws0kJVt1Atm5KeQEmcKaTH2VBgVZ6K5w+a2rDP6IejZqhVdSERvtUWFYoSxZHYeLZ02frxmLwDVIoNs/ORsmFPdC6L+QiEBbHbJ+RPTvY7dXwH3xZUteqS6RUv7N5+63GQlxfLJ4SA3MAWgBF2hiCoKUuMAeGXUi2S31wACraWyE6NZa17hxDMRqbyxnWSzkcZ2KxI4/7Se2drhJMWZLXq4qvmU0/5dZCfH0tJpo6XTRkKModciZiMVJegKRRRR/N9/AOOZ3rZV03IkblFPLvBbQtrT+t7weRV3/Gk7ibGGwB2OThFfkV08MxeAw9UtTMjsX7chz1Z0e8ssvRYxG6koQVcoIoRerc7i1RSXNjBOlJMoPBc7nWL+4N5ez+FamJw/Pj0sKf6ec/cV2Zc3HwegrcverwVRwJ0hWmFpD1rEbKQLuloUVSgigDVFZTz0r92UNbYj0QTxoX/t9s7WXL+CYsc4ZohjPQ8QYi/dvBQzJr2ORdOywjNxJ/5EttPWXa+vPwuiAJNzEjEb9byxs4zyxnYW6zaxyfQAx2K+wibTAyzWbep33PxwQFnoCkUEsHztPqw+1QStDsnytfvcVmdVYwtVpDJD50fQQ+ylm5YQw+ZlF5OecGpp+IHoTUxPNe3fRUqciVvnjeYPm47zpbjd/NT+EnFCS1jKF7WsNL5EqtEEXNWv80Q7ykJXKCIAz0zOQNuLY7X+MT0FXfSpl25GYky/Fij90VsSUn9dLgD3nD8ek0GH3dblFnMXcaKLHxhfDek4a4rK2Hiwut/ziUSUoCsUUcKegq+gw8FUUeKxVcDcO4a8FWOg5KQUsxa6GA5Bz0iM4dazx/B21xxKHD1dRnHtlSEd55n/HuLFj46637sKnhUue4f5Kzf0uyjZUKIEXaGIAEY564cH215szeO0FDCnZAFCWwi9bhVc/fTgTDIIgZKTcpyW+6nWcfHlngvGYcDBb+xLeu4Mwe0kpaSqqYOT9ZqLyLWY67l2EY5Kk0OF8qErFBHAY9dM5fv/2oXNp+/X9y/T0u+llBSXWrhk8mi4ofdolqHAX3LS6u1ag+pwWOgAmYmx3DIJ/nxwAd/Sv8EYndN1YjSH5HZqarfRYXVQbmmny+YYdhEzykJXKCKAJbPzeHzxNPf7zMQY9EKy+b1XqH5sDGW/PI/61i6m56f0+GwkuwxGxWuLof2NcvHknusux46et02X435Suea5kNxOVc1a+QApNWs8lLZ/0YSy0BWKCGH26FEAPH/LHK6Un/Dsax/wTOcS1vFrJnZpIj2zfRswxv0Zf/HfkZRkkxZvwmToX9q/L9nJsUzKSmRL0q18885f9+mzngW+Tta3hdT2L5pQFrpCESFUNmnCkpMcC+tX8G3dav5r+j5f179Nk4wjlSYmFf3E6zPBXAaRwO3njuXZL88Ke1TN2eNS2fFFA1a700dVvDpo1yYXVU2d7tcn6tsGrNLkUKEsdIUiQqhwVhPMSTa7E4Um6Mr5ge5Vvm9YTRcGYpq8OwtFustgXEYC4zL6l/bvj7ML0/jLZ1+wt8zC7MYPtC5NVuffHKBrE3Rb6Ea94GR9G49cqfVjHS51YXoVdCHEy8DVQLWUcpqf/bcAD6MVlGgGviGl3B3uiSoUw52Kxg70OqFFhCTna8LkRCcksVg1f7EHw81lECpnFaYCsOVYPbOLVnSLuQtX1yYfQa9u6iDZbCQtwcSJ+jZgYCpNDhWhuFz+BFweZP9x4AIp5XTgJ8CqMMxLEYFE8uLbcKDC0kFWrB39s9OdYu7jpvATyTHcXAahkpEYw/iMeP53vC5w2QM/26uaOslKimF0apxb0IcTvQq6lPJjoD7I/k+llA3Ot1uA0HKQFVHFcIvXjUQqjxaT3X7EwzKXONAiMirJYNv0x/1WUhyI5hTRwLxxaWwvaaApNtfv/jZzdo9tVc0dZCXFMjo1jpMjUdD7yJ3Ae4F2CiHuEUJsF0Jsr6nxU5hfEbFE+uJbNLDirf08+9/D/ncWr6aiqYMc4W076YAymc68jme5bdsYvzfQJbPz2LzsYo6vvIrNyy4eEWIOcPa4NFo6bfyw83bapHetmDZp4saW7/PRIW+dqW7qJDNRE/SmDhuWNv8lF6KVsAm6EOIiNEF/ONAYKeUqKeVcKeXcjIyBLa6vCC+RvvgWDaz/vIo3d/t/opHvPkyFTCVH1PXYl+vcpm6g3sxz+tHfbp/KMutdlDrScUhBqSOd73R9g73WHP61vXsdwuHQskSzkmLIH6U1CjlR3zasXIlhiXIRQswAXgKukFL2/EYqhpRwdHcZCYtvA90Fp66li9YuG21dNuJM3j+9pvZO2oklW/T0bpbLtO7X6gbqJjMplsL0eCoa21lrW8DargU9xry/t5I1RWUsmZ1HfVsXNod0u1wA/r2zlFe3nYzYOP6+0m8LXQgxGngd+D9SykP9n5IinITL9z3cF98Geo2gw2qnpdOGlHCgornH/gqpWZu+Lhcp4Ulbt998ON1Aw8G8cakgINbQLWWLdZu4QfchADpHF5+8/jxrisrcIYtZSTEUpGrX8fWdpcPKldiroAsh/gF8BkwSQpQKIe4UQtwrhLjXOeRRIA14XgixSwixfQDnq+gj4fJ9D/fFt4FeI6hv7S73ur+iqcf+CpOW/elrobcSy1qHZnkOpxtouDi7MI0Oq4P7L55AXoqZxbpNrDS+xAGpXc8ujNwt3mDXO6uodiYVZSbFkhhrJDXeRFOHze9xo/VJqFeXi5Ty5l723wXcFbYZKcJKOH3fwyle15dA16OssZ3rnt/MkzfM7FdfzLoWD0Ev9yPoU+6Erd3+cgC7MPILw72ITqI+4WWgOHuc9mSj1+nYvOxiSh+9Gxt6DsgxnKcr5hPHDMpkJnd1/Y1NTXcC3e3sClLjaO20eXVWchGtT0IqU3SYEw2+76qmDlLijMQY9L0PHiACXae0eBM7TzRSdKLhlATd5Zd3HTvGoPNroVcmTEUnDpORnAhNDZCcj37hoyyfcSPL+3zWkUNOspk5o1N4o6iUey8YR66ujo/sM3Cg42b9Bj5xzOCIzOMi/S532r+rUNjo1DjKGtrQddq9ns6i+UlI1XIZ5vTX9z3QEQBSSi7/1cc8v/Fo74MHkEDX6fzTtGgsS4COQsHw9Mu7sNod7C+3YLN7W4Xllg4yE80YvlsMyxu1hs9D3LQiWrjhjAIOVbWwp8xChzmbbY7TMWDjQt1u0mnkiMylw5xNVXOHu1gYQMEoM41tVn66ZNqwcSUqC32Y0mG1o9cJ9xfzVKI3BqOSn6XdSkOblR1fNPQ+eAAJdJ22lWg+7aZTEHR/fnmHBIddcry2lYlZie7tlZYOclJi+/EXjFyumpHD42/t47Udpay4YgVb/1nGVFFCnOhkgq6MwzKfuCtWUF3U4Xa3gGah2xySswpT2bzs4iH8C8KHEvRhyld+v4XU+Bh+f9sZp+z7Hozi/7Ut2mPwvnILUsqwV+XrC/6u06vbtDjmQItnwQi2TrG/oslL0Css7UzKTgw4XhGYZLORRVOzeXNXOd9f9CWKWcft5o/BIZgQ08SbtnnI6VdR9dFmspK667K7QhdP1rdR4Hwd7SiXyzBESsn+iib+e6CK9Qf61gzX08Xiz6cM4Y0AqGnWFgsb2qzuaoORxLHaFuDUXC7B1il+9MZet/tKSkmFpYPspMhZ14g2rj8jH0u7laf/c4guh+DMG74HyxuZcMmdNFsFNc2dzqSibgvdJeLDqaaLEvRhSG1LFx1WzUe74u39dPhY2YHwjcUORDgXVF0WOsA+P9EfvfHqthPc85ftA+Lnb+6wuhfSTkXQ/fnlXbR02txx7k0dNtq67OQql8sps2BCOllJMfzlsxIAzhyrNQuZkKk99Xz+/M3UNreTuf+P7lrpOcmxGHSCkw1K0BURTKnzC3r7uWM5Ud/GHzYd9zvOd8Hz8bf29XCx+BLuCABvQbf0+fMvbyrhg/1VA5IQdLy2FQAhTk3QPWP3ATKpZ5PpAZbqNpJCM5fYPuKpdQepdD6ZZCcrQT9V9DrBdXPycUiYlJVISpxW22VC3UYAtjRn4EBHVmeJViu9eDUGvY7cFDMn6qMz5twfStCHIaUN2hf05rNGs2hqFr/ZcKSHm8RfZmRDkEJFAxUBUNvSiV4nGJcef0oW+tGalh5PE+FKCDpao7lbJmQknJKgQ3fhLDMdzNftI19XyzTdFzSSyEPGVzmj6QMqLB6dihSnzPVztEKvZxaOcm/L+vRxEmhjs2Oq9l40dtdKByZkJrC3LLghEU21XpSgD0Ncgp4/ysyPr5qCQ0p+8vZ+rzH+FjwDkZdiHrBKfrXNXaTFm5ial+w34aY3bA7/zqFw+Pnf21MJwOHqFo7VtIT8Q/7DpuNsL+nO+JRS4kBHpmgEYIruCwCOyAKWG//i3alIccpMyEzg1zfP5hsXTnBvE02ljBfl7JHjAMgSzmgqZ6308yemc7y2lS/qWv0eM9rKRitBjwDCbQGUNrSRGm8iPsZAQWocDyycyHt7K3mnuMI9JlTBizHoBjTJoralk/SEGKbmJmlPCR4p8r3R5SfDz0V//fxrisq8FpQdkpB+yA6HZOV7B/jrli/c21q77HRiIk1oN6zTxQkA9ssxjKKFCksHOqE1bVD0j2tm5rpdXAAk5zNelONwSp1b0JM1a/7CSZkAfHjQfznvaCsbrQR9CFlTVMasx//Dd17dFVYL4GRDO/mjur/UXz9/HDPyk/nBv3cz72frKVz2DroA4YEpZiM6j11njBk1oEkWtS2dpCdqgg7+65wE+6w/wuHnf2rdQezS2/oP5Ydc29qJ1S4pqeteaKtzztMl6IminTGikn/YLuYb1m/z5q4yMhJjMOrVzzHsLHyUCQbtxqzDQRoWr85PY9PjKUyPZ+NB/9Fg0VY2Wn2DhgjXo1yjH99sfy2A0oY2L0E36HVcMS2b1k47lU0dSOghVqAJ4Y+vmoxDwncvPY0vzy1g54kGGttCt5r7Sm1LF+kJJqbmJgN9Wxitbu4W9KzEmLD6+U81ZLO8UXOfuB/hi1dT+9JSAFLpvlndol9PomjnGPmY9DqWzIrOzMSIZ8aNMO5CADJopEak9+j8dOGkDD47Wkd7V08XZKAnvUgqneGJEvQhojcfdjDhKG1ow2r3726QUlLW0O4u4O/ib1tO+B2vF8JLCM8ep9Xezk6K5WsLxtJhdfCPrSf9fra/SCmpaekkIyGG1HgTucmxfVoYrW7qjlt/+WtnhtXPnxXA/dHbD7nC+e/W2GalcdtqeOsB6lq0eaaL7r/tHsM7vBP7f/nPTSl88N0L3N3nFeFlTVEZzx7R3CpVpPrt/HTRpEw6bQ62HOvZyiHaykYrQR8ierP0fIWjrqWTP2w6zhXPfsKCJzby509L/H6upqWTTpvDy0IPdj6HlF5CWOmqGZ0cy+nZSZw7Po2/fFbSo/ZIOGjutNFlc5DuLJY0JTe514gDT2o8XC4tp5DJGQx/NwWTvvf1hHKP5KiSt58Eazv1UouFThMef1tyAfovPa/qtQwwT6072KOa4qX2j5j35gWwPAWemcZZrRsxG/V86MftEm1lo1Xq/xARqLof9LQAmjqsXPbMx9S1djEjP5mMxBg+O1rHXeeN6/5Q8WpYv4LShhhgBfnvfhW21Ll9hZ/F/pBMWUOTjOM39iX80X4FOhwkxsV7ndsdE+3MqLvtnDHc+7edfHKkloucC0jhotbpMklP1GKGp+Ymsf7zKr8dffzhqm8N0NoVXkF3ZRRmJ8W6b3JfPXdMrz9kzxvnF44MZukPU4fmTkrF1dhCaMW3FAOOryHjqpceh9ONaDlJ7LsPcG7Gi2w8WMNyP+UnoqlstLLQh4hAWYSj4ow9LIDPK5qpa+3i2Ztmsfb+BVw0KYOik41IKTUhf6IQXr8bLCc5KbXqgAWiRusev+Y+ePObZFODTkCKro0fG1/hXdMjjBMVmNor3ZlzgLuri0vQLzo9k5Q4I//eURr2a1DrrBHustCn5iYF7OjjD08fenMQC33HFw088npxn54yjta0kGw28tkjF7Px+xcCMDknqdfPVVjaydM1IHBQIrMAqJVJJNJGrHCulzgjLBQDj++T7g8Mq4kTPmtC1nbmN67hRH0bhY+8G/Gx5sFQgj5EuB7l9E5rIM6k51dfnsVj10zlqXUHvUIYD1VpAjd3rFbMf/boUdS3dvHFp69pWW/t3THPpU5BzxO12gaHFew9FzUn6Uq5Rv8ZlTKVtn/fr90UildT1dRBrFFHklmzkGMMeqbnJvN2cQVj+xhW2dJp4xt/2xEwxtcVpeIW9DzNkt0f4sJoTXMnafEm97kC8dqOk/xj60nW7i4P6bgAx2paGZcRjxCCZLMRCC1btLyxg0JKyaGeLxzZANTJJHeEi2eEhWLg8TWccl2/Cx8usX3ifh3psebBUII+hFwyJcsdbTJntJbd5i+JYd2+SuJNenKdmYSusTs/WqtlvXlQKjNIw0Kc8B/S58lYoSXOlMgs7abw1gNUnjhMdlKs+7FzTVEZW4933zD68mX/6GAN7+2tZOPn/kPCfAU9NzmWZLORA5WhWeg1zR0Upmsuo2A+9D1Ov/yvNxwJ2Uo/VtvCuHStoUVSrHZzC03Q28kxdTBGV+W20OtI1iJchB6ueU75zQcRXx94pUj3O05H30NUIxEl6EPI0WottTzWqKOyqSNgEsPW4/VMyEp0i+yEzAQSYgwUtab1OGapzCA/gBXiS6FT0I/LHG2DtZ2qshIym/a6F4x2vbOKTh8RDPXL/ulRbR6umii+1DZ3IgSkOq3sN3eV09Zl45X/nQjpSaC6uZMxafEIAa2dNr8JWp02Owcrmzk9O5Hjta0hWemuolzjM7WbhUGvIyHG0Kugd9kc1LR0kjNhJmP1tXzhEnSZSJq+Fb70ghLzIcBVfuH4yqtYZbiVNmny2t8mTV6NuF1Eaqx5MEJpEv2yEKJaCOF3FUdoPCeEOCKEKBZCzAn/NIcnR5yCfubYVKosHQG/QJ02BxM92p/pdYJZBSnsFFN6jC2T6eQLj6w3nRH0ph7jwNNCz3Zvq7Qnkm2vACRYTvID6/Ms1m3qeZ7G9l5F99OjWhjYsQCCXtPSRWqcCb1OuOPyrXbpPn6wJwGHQ1Lb0klWUgwJMQZ2nWz0+3Sz6qNjWO2Sb140gck5SSFZ6a4bkMtCB63mdkBBL14Nz0yjasVpSAl58ZKxU8+hjmSaZBx1YhTpY6YpMY8A/txyFsusd1HqSMchBaWOdJZZ73I34vYkUmPNgxGKhf4n4PIg+68AJjr/uwf4Xf+nNTI4XN2CQSeYNy6N5k5b0Gp7E336Wc4encLn9lzaDMnubQ4pNAtd57TQkwtgyfNw7W+11wCi+588XnSSRT3HnL5eKaFSjiLblR4NxIkunja+wGem+9DhYCLd8exlje08+OoufrxmT4/5lje2c7y2Fb1OUBLEh+5yt/Q1xbqx3YrVLslI1AR9+xcNfj/vqjQ5Mz+Fby+cGJKV7qqPPTa9O5Y/yWykqd2PW6dYizXHcpJytCemnOLfMiZRu2mU3H2AepJJG63izCOB3BQzax0LWND1HOM6/86CrucA2GR6gF8angfgYrEjomPNg9GroEspPwbqgwy5FviL1NgCpAghcsI1weHMkeoWxqbHu2PGbz93bI/IF5MzHfy0LO9uNnNGj8IuBcVnP+0Ua0FN4mS6MJJ/9TJYbunuSznjRu31cgs81gDX/R7M2gJroa7CbaE3kkAXJrKE9z+3QTjI0TVynq6YFuJ4TP+ye58E/r7lRA9L2mWdLzw9k7KGdjptPZOotLR/7emhrynW1c1aNE5mYiwJMQba/GT5gSb8yWYjBalmLpuS5bbS7QGKekG3oBd4JGclmw0929AVr4Y37nWvY1RITdBz7eWM3afZNbtPNmJ3SNIS/D8lKQYX30XSxbpN/ML4Ivm6WkbrtCfbWwzr+cuZX0RNqKIn4fCh5wGeqYSlzm09EELcI4TYLoTYXlPjvxjOSOJoTQsTMxPcMc9Tc5N7JDEsmZ0L0KPj/KyCFACKjLOdYt1I6dJ3AHokFfVgxo3w8HG47vcUxrRoPvTkAipNYwG8LHRPrtNvooJ0puu+4HjMVzge8xV2xtzDNbpNPSzpT4/Wkhpv4vJp2Tik1ubLF08Lva8p1q4Y9MykGBJiDcQY/H+VjXrBtLwkhBDodIJb543meG2ru2a8P07Wt5HmLG7mIinWx+Xissxl942kXGo3yRxRx+hW7anF1SvVtU6gGFp8F0lXmP6KSWj/hi5XZYVM48w9jw/hLE+dQU0sklKuAlYBzJ07N1hTnGFPp83OF3WtXD0jxx3zXdnUwQ1n5HtZBo+/tQ+zUe9dQQ4YFW9iXHo8O090i6+rbG5Bb4LuYsaNFDaeQf27n2P5ehGVH6+FDyFb+H8gu0hXhBEbHzjmMld/GIBUWnja+ALfawLQGu1KKfn0SB3njE9jXIZ2IzpW08resiavJsw1zd2C/tCiSV4NqSF4inWNMwY9I0FzuWQnx1Ld1On1+ViDDqtDMj0vxb1tbJq20FnW2M6YNO+kKhcn/PSY7OFDX7+iR4RRuUwnmRbiRSckF5DVHsPOE41AdySPYujxShRa3h1RlUUDJqxaLoe1VbtpR9m6Rzgs9DKgwON9vnObIggltW04pGZ5u3znVR61SVwLbYc/e4uJsgTd3n/1OMas0SkUnXAmGNEt6HkpoTe8LXQu/B2va6U67UzAlSUptDA7D5JEO/N0+/mPYy6etb0MwsGvTM/DM9OgeDXHa1upbOrg3PFpFDpF8+3iih6Lll12h/tvdllOrtDMhBhD0BRrV1JRZlIMibEGTHpdj6eb+y6agN0hmZ7Xvc7gujGWNQSOYDhZ3+5uIOwi2WzE0tqu/Y3LU7SkLR8qZBo5os4daz4mLd7tvlEul8hHJyR5opZS6cyIdjbBiCbCIehrgduc0S7zAIuUsqK3D410XBEu4zMSiDXqSYkzujvX8PZ34fV7wHKSw448JjiOu9tmeTJn9ChqWzqZ97P1jF32Dr9wuj0uefqjkJMiXHHcx2tbqLQ4RfI7H8PyRi3MDu806Mt02zkuczgqc722C9BE7vV72PzsVwGYb9tKcpyR1HgT/z1Q5bcY2abD3SGWS2bn8ekjCylMj+eCSRlBfZjVzR0kxBiIMxmINxlo6bSxZHYeDy2aRG6KmfLGdl52Loh6Crrr5umqiuiLze6grLGdglTvp5zkpoO02wVdjeUQoONquUwlT9S7Y83HpnXfFNLilYUekTjXklzkixp3trWrCUY0EUrY4j+Az4BJQohSIcSdQoh7hRD3Ooe8CxwDjgC/B+4bsNkOIw5XNyOEJujgrBli6dREe/vLgMQi46gilYm6Mq+2WS5cZW2rnNaqS2b6kvwzOjUOnYDjNZpVnZ5gwuTyR8+4EebegaeoX6LfCcA6x9wAR5R86phKLrWM2fgtKF5NYXo85q56dsbc4/a9/8v4GADPWH/qzlJ1UZAa59fn7klNc6e7IURCrIGWDluP7jKu0sQ7vuh2IcUa9WQkxlDW6P/4FZYO7A7Z00I//i4AFvy7aQDKZQY547vDE10uHSG0kg6KCOSKJ7zeFohqd7Z1NJZoCCXK5WYpZY6U0iilzJdS/kFK+YKU8gXnfiml/KaUcryUcrqUcvvATzv6OVLdQv4oM2aT5tbITo6lsqkd3nsYlzQfkZqFeppwWgo+FsMr//NfEhdCT/4xGXTkj4rjeF0bVU0d7gVaN1c/DdetclsyOaKemeIoH9jP8Hs8hxR85pjCufp9CJt2Exqrq0aPnVTRghCawLkKVmXoLFqW6pr7tCeTZ6Yx+ugrnCgr7/FE4km1h6Anxhho6bLx5Puf+30K+MV/Dnm9zwtSGM11Iylo3KrdaJYnw/Jkkju0a2+R3YvTn9in0STNgKAtaRwW4skZN8293+WvHxVnwqCaV0QmM26EuXfiMlryRQ31JNFqSInKEg3qWzZEHKluYUJGtzjkJMdSWdfoVZflsEMT9InCaWn7WAwVFv9uAxehZroVpsc7XS4d7gVaLzyiYjDGc5l+O7vkRKpkSo+hB2U+jSRyjs7Zw9RSyria9VSTSqvsdjvUSk3Q010lZR1W7cnEcpLRoppGGY9l7cMBRb2muZNMDwtdSu/StcGuQ16K2b/LpXg1J159CICCTx72+rdIQould1notTKJ26zLeEp3JyxvpPzWT9zHduGKY1cRLhGOy2hJLqDAmWV98vxfRN2CKChBH3TWFJVx7s/X83llM9tKGtxukaykWGo7dHTJ7oXIwzKfWDqd4VSih8XQWyZbqJluhenxHK9ppaqpg0x/gu5ixo3wo3Jypl0AwAf2M/DNuXSVEXD1zSQ5n7Gd2pOCZ0ZqrdNCT/Po4uN6MhkttNovJ7uSAi5MVTd1uC30+JrdAGyJuY+jMbdwPOYrvGn8EQC/MTzLZ7Hf9rox5I3SLHSHw6Na5fJkeP1uTrQZMWAjxyfSJ1logt4kNZE+5MhHouOf7Wczdtk7XPb0RwD89J397n9Tl8slTQl65OPM1Si4558AnEw/f4gndGooQR9EXD5elyXZ0mnjkdf3sG3ti2R/uhyAaka5xx+S+UwQ5QgkR8d8uYfFEKgEL/Stq0phejytXXbqWrv8W+g+f8MP92oLoj+23cl3uu6jTKY7pVhQJrXiR/mi1h3tUZio7T3uKegymRSaMYqeLpICp6CfkJl+F6ZaO220dtnJTIyF4tUk7PkTAC0yDr2QCIG7y/tM3TGyqdEWmZcnwzPTyGs9QJfNQd0TM7Wywx6W+EmZQZ6oxSC8b1XJTgu9iXhAsMcwFQAr2vV3ja5t6XKvXyTEGMhIjCFdNX+OGlwhv72t4UQqStAHEX/p7ZfaP2Lmzh+SbdXC4Kpkt6AfceQxUZRSLxO4rerLPY7nmSQBuEvx9rWriivSBSA7Obj4PLXuIB0eHWDWOhYwv/M5FsS+wbY5T3BIFpBAG23EuXs3jr3kbsDHQpfJ3e4WwDNxc7SnoPtZmHLFoGcmxsD6FSTateO00n0zOiLziKfdo66N8wSWk+Tu1bI4y9p7pmGckJnu83uS5LTQLbH5sLyRZ+03+L9AaOsXy9fuY/7KDdQ0d7LteH1UlmIdiaTGm4gz6d0hwK7wYVexumDrOpGA6lg0iPjzaf/AsBoTNnd2ZoUz27BZmqkgjbGiksdttwX0h/t2U1lTVMZT6w7y4Ku7eGrdQR5aNKlXYfcU9B6LoiH8DaBF1ty2bQztdk2Az+l8FvM2PT8vKGPJnBvJfustjtnHAlrNmBqZTDoWpIR6mcDbjnks1X9MnOgiUbSTShMnRC4svLbHuTxj0LGUkiA0N0iL7HYxVctRZIkGfJrPAJAntaJkZTKdWRz12ndSZrJIt63HZ1wWuqXwSoCApQZcNLZb3VE21c2dPPK6ljkajenkIwkhBPmjzJxsaOvOBnYlkFlOau8hYv3rykIfRHx92ot1m9yNKFw+20qnoG92aNESnzkms9axICR/uG/YXqjhi7kpZnfNmGAFwvz9DS70QgQtrlWYk8Hx7EWw3MIC8xvskJPYIqdS2PkKZ3St4jHbHTxpvM9dl6bA0MjJtPP8/nDcWaKJMZCcTwLaD64ZT0FPITNACYM8odWZKZfe5YdbZCz1JPm10E0xsZj1EkvSJKSU+LlPBCVa62uPRApGOcNm/WQDY3VFokUmStAHkYcWTSLWGeM9mkpWGl9yW5BJtBJLp1vQX7OfTxoWtshpIfvD+1qx0IVeJxjjTILpzYceqAu6XQZItnFa9GPT4ylxlqX197eYjXpmXXWPuy7N6KnzOGFN7DEOvAtzsfBREozauVs8BZ0UMvDf+SiJVhJoc/v7Xbjb91GNXTr/YZILtOieH5aTHG/G0m6lsqkDCRh1fZP1aKyvPRIpSI2jtKEd2Rggsai9PmJdL0rQB5Els/N48NLTALhZv9Grt6EQmpVeKVOpkUlsdMxiFM198of3tWKhJ2PT44kx6Nzt1oL9DZ5+e7NR7/XeF5dF39Zpo6HNythl7/DYm/sArRNQoE7qo1PNlDW0+61dXt3ciVEvSDEbYcaNJCzSon9apRmEDik1Cz1DNCIlPSJxhMFEnqij1EPQpYQTDq0hxWr7BZxvft27YiXd9VwOOjsq3XvB+B5/d16KOWASUTTW1x6J5I8y09JpozHptMCDIrQsgPKhDzKuIk0X63f12Jcl6qmSo3hTdyl29Pzuwa8wMcu/leqP3AAJM6EIyfVz8t09NHvD5bf/P3/4H5Z2q1uIH/53MZ0eC6auJ4s1RWW8t7fSvd3SYUUn4JErJnPz2aP9nmN0ahw2h6TC0tGjUJarqJfOaSHHz7keXn+floU/h4v+xatbT9Dx+h5etl/Jy/YrWazbxMPG1eSKOkRyPix8lNwPYyiv1iERlMs0nrDe6G5y8CmzSOyyUbjsHXJTzO51CJegH67SyjbcsaCQ7zmfNl7fWcrcMamMTotzu75CLTSmiCxc37fSM37AqA/v9DvGYSllbVFZxK2JKEEfZHadbCTepGdCisArBBvIpoHtYgqvJZ/JzDRdn8QcQq9Y6Fo4dVU9dAnW5dOyfQ8ZlLFp8azZVYaUkiWz89hbZuElZ/2UPI/jzl+5gS4fS9sh4ZE39vCbjUf8Lty6flQnnZUPD1c1897eSj49WsvOLxqZmpfkHhtj0GMy6Gh29hV9+gPvzNC1jgWs7VxAXoqZzQ9qFSHzju1hZ4MZ8Vgj24rK2LHuILhuhgIa2rQFTdc6BEBbl43PK5v53/F6dAI+OlTjnvd1c7qjcVzb/F1jReTjKj99Mm0+pxlTiLE29hjjkIJNbzwP3BdR/65K0AeZXYeOMUMeR990Ai3duNv3nG1oorQrFSqb+cm1U/t87FCExNd69BSsvn4xx6TF0dxho7HNyqh4E3nOH8L2H1/iVS42mMsn0PldtVSO1LSw5Xg9z288gl1KpuQk8dVzx/Cl2d7hjAkxBlo6u6NK/OE5j7yUOCztVndRryWz8/jaH7ey6Uituw2ei3arnR++XkynTbrXChySoNfNN/pIET24jIn391ayofNWVohVbvdoiSOLFmKZpvuCFaziyXcMLJkdObXTlaAPIh07V3OgPoa79fvQ8lEkblFPLiA7fyHs0LoUXTMzN/jBAtCbkARbOO2rALlqlZTUtTIq3kRZQzuxRl2PzMhArqBg589JNmPQCX7y9n6sdsl1s/P44VWTA9YVT4jRCnSBVgjLZWH7zqP7tavqYru7G9SJ+rYeYu6izdrTl3+q100R2STFGkk2G1l/oIrWrnOxCAOzdMd4z3E2e2UhJqwUxdxDvOjkrq6/AZEj6GpRdBDZ+8GfsWFgls4z9lkTcx7cS/bkcwG4ZEomKXEDky7en4VTX1y1Sr6o07LqyhrbtXrkPn74YBmtgc6v1wkm5yQxKs7ES7fN5ekvzwraJEKz0LUb1fmnZfTY7+t6cj1Wu+qiOxyS0oZ2EmL6ZuOoyJXhSUGqmVZnrsEH8iyest+EARtL9R/ShZE9shCAXF3dEM6yJ8pCH0R2NWt+39m6I947nOntp2UloNcJbjl7zIDNoT8Lp77kj4pDCNxNoEsb2skbFddjnKcrKJCl7u/8f7vrbEx6nbsiZTASYrtdLllJsRh0gszEGCosHX5dT64mIK751LR00mlzcM2MXF7bGXodbBW5MjwZnRrH3rLuRa5XjY9ztv4gdTKRf9kvZLdjAvN0n9Nhzib0djIDj7LQB5Ei/XTyqCFTNHrvcKa3j8tIYM/yy5g/Ib3nh8NEoDjyU4nAiDXqyU02e1nogfqZLpmdx+ZlF/OrL88K+fzJZmNIYg4uC11zudQ0d5KVFMunjyzk+Mqr2Lzs4h5ukYzEGAw64Rb0YzXaTenqmTkh1y5XkSvDlwcWTuTLcwvceSN/ty+kTZpIE80UiGp2OcYjgbj2iogqCaAEfRDZZZjOLEOJ90ZnASsXcaaBfWjybZLb17ovvoxJi6OkrpW2Lhv1rV0B49EH6vwuPH3o1c0dWlmAIOh1gpyUWMoatKqLz64/RGKMgRn5KTx2zVS/N50LJ3W7csI1b0Vkcnp2Ek/cMIOV188gL8XMW44FPGm8jzZzDrPEEXY7xndnC7tKAkSAqCuXyyBR3dxBWauO22fPgPICzc3ijIke7LoQ4YzAGJMWz/t7K9y+6EAW+kCd34XmctF8ntVNnYzLCNxZyEWes1XdK1tPsOVYPSuvm05qvClgtFD+KDMfHqzhokkZ/PFrZ4V1/orIxPu7ehXwODN/ei9vtaQ7y0s0artcHcWGuMaLEvRBYpez+/vssy+EsXuHdC7hpDA9joY2K/srNH9jKII+ECT6hC2ePS416Pg1RWXsPmmh3WpnxxcNnJaVwJfP7O517u+mc7hKyxA9rY/5AYrhxeyuHcA17HKM5zL9Dvf2SEg2CsnlIoS4XAhxUAhxRAixzM/+0UKIjUKIIiFEsRDiyvBPNbJoaO3yu8DXYbX7jXxYt6+KxBgD0zwaFg8HXE0cPj2irfa7FhsHm4QYAx1WB62dNiztVq3OSwB8Y/ElWqTOm7vKg54jJ8VMZmLMgK5xKCKfqSlWDNjY7Rjvtb3ckRZyL9+BIpQm0Xrgt8AVwBTgZiHEFJ9hPwZWSylnAzcBz4d7opHGj9fs5drfbKbDJ6b7p+/s55KnP6KupTu5pa3Lxnt7K7hqRg6xvYTvuVhTVMb8lRsoXPYO81duiNh62q5Y9E1HajHqhbst3GAT7ww3PO4sABZsHv5i8Tttjl6LmCXEGNj6o0v8hkUqRg6xl/yQ03Wl7JIT3NukBDMdXGr/iCff/9z9NDfYhGKhnwUckVIek1J2Af8EfItUS8CVi50MBDd1ohyHQ/Lp0VpqWzp5c1e30Na2dLJ6eyltXXb+7tHAed2+Stq67F7p4cE41TK4Q4Ero7PM6WvW9bECYbhIiPUR9CCLouGMxVeMQGbcyMxxeex2jMfu0L7vQkCaroWVxpfIbSri0mc+ZscX/ss3DyShCHoecNLjfalzmyfLgVuFEKXAu8C3/B1ICHGPEGK7EGJ7TU2NvyFRwdGaFhrarOh1gpc+OY50poP/9bMv6LI5mJyTxF8+K3Fb7//eUUZBqpm5Y0YFO6ybUy2DOxSYTXp3yd2h8p+D5kOH7vDDYC6XQLHjKqZcESozZ51BC2aOk+O1PU50kaXT1pPW7hp8AyxcYYs3A3+SUuYDVwJ/FUL0OLaUcpWUcq6Ucm5GRvQ+tm4t0ZpRfPOiCRyubuHDQzV0WO38dcsXLDw9kx9fNZnali7e3FVGhaWdzUdruW52flDr1dPFEij5JlItSFct9d5CFgcSl4V+rFarhBjM5RLOWHzFyGR2QQoAu6S3H11K2GnXXDFvFJVhd/gvJTFQhCLoZUCBx/t85zZP7gRWA0gpPwNigWG7crT1eD0ZiTGMHmVGCPjaH7dx+v99n/rWLibnJHHu+DQm5yTx0ifHeX1nGVLCdXMCr3z7ulgCEYkW5JqiMvaUaY0k3ttbOWRuIU8fuhBab8hADFQsvGLksKfUQhwd7HJM8Np+VOZSgdYJq6nDxtMfDO5TdShhi9uAiUKIQjQhvwn4is+YE8BC4E9CiMlogh69PpUgSCnZeryevBQzj7yxB99GPS98eIQJmQncfV4h3129m+c3HuHMsaPc0SD+8Odi8SUSLUjfaJHmDtuQ9c70dLmkxcdg0Ae3VVQ1RMWpsqaojB+t2Us8HRT5CPpGx0yv93/aXMJDi04ftLn1aqFLKW3A/cA64ABaNMs+IcQKIcRi57DvAXcLIXYD/wBulzJAT7Iop7ShnQpLB8drW/1W5rNJTaCvnpFLVlIMrSEshgZzpUSyBRlJvn6Xy6Wl0zZkkTaKkYHre19LCvvlGA7YC3BIQZ0jgc2O6RRQzSbTA+RSQ2uXHZvdMWhRayElFkkp30Vb7PTc9qjH6/3A/PBOLTLZ5vSfW5wd3RfrNpFMG1vkFH5v+AVP22/grcYFmAw67r1gPM+tP8yV03OCHTJgway8FDObl10c/j8iTERStIhnlcTe0v4Viv7g+f2W6LjC+gSLdZv4qeFldjhO42r9FvJ1tTxs+Cfftn2LX/7nEH/6tCQsPQh6Q2WK9pFtJfUkxRpIiDEwt/m/rDS+5NUbdKXuJVKNJuAqbj93LLecPQaTIfiDUKidhiKNcFZu7C/xHjVw+mKhB+repFAEwt/3/j79Wg5TQDNxLNBpYr1Iv504Wwd//qwkbD0IekMV5+oF30el9QeqOXNsKj+4/HSWG//iJeaghS39wPgqAEKIXsUconeRLpKiRXQ6QbyzMmNGiIIeTfH+isjB3/d+neNMNjmmIXBwrk5rgh4rrFym205bl//1sYF4klUWehD8tWsDiDPpWaLfjBQtfj8X117pd3swonGRLtJ6ZybEGmjtsgeNQfcknN2bFCMH3+99jEHHH+xXUUAV00QJozx04crEo6yxLPB7nIF4klWCHoRA0SdbjtVD1QoCRpUnh5YROhyIpBtRQoyBKjpDdrlE0hqAIrrw/N7vLbNw9a83sY9C7tWv7R5kNDP9oqWwBow6gdUjJn2gnmSVyyUIgX7YNS2d7i5DfvGob64YPBJitcYUoS6KqoxRRTiYlpfMpVOyADgvsRIQWlvJa54j++wbiDPpOWd82qC4VJWFHoRg0SfE5GuF7X0xpw55TeSRSkKM5tf0dbkEWviM1sVoReTxoysnk5di5swr3wCPdTMBjMuIRwgxKBFrykIPgr/FD5Nep/3gFz6qdRvyxGiGK54YxBkqPHGFLnouigZb+IzWxWhF5DE2PZ7li6f6DYIYn5HA0Rr/623hRlnoQXD9sJ98/3PKLR0Y9YInrnf94J1W+PoVQ9p9SNFNQoyRxFiDV4ni3hY+I2kNQDE8GZ+RwNrd5bR32UPukXuqKEHvhSWz8zCb9Hz9rzv45Y2zWDwzt3vnjBuVgEcQX5s/lvNP8y4hpBY+FUPNuIx4pNTqDE3JTer9A/1ACXovSCl54aOjFKSauXJa9lBPRxGEaXnJPTpCRVLyk2LksG3tixTsfIpMWUOynAk8zLHalgEXdOVD90NdSyf7yi3UtXSydf2/KTrRyN1Nv8Pw3IyI6OytCJ1ISn5SjAy2rX2RaTt+TDY16AScKfYjcPDxJx8O+LmVhe7D3jILX/n9Fpo6bM4tZlJpYqn+Q7B0wVsPaJuVqyUqiLTkJ8Xwp2DnU5g9MshjhZV8UUtjmZ+ouDCjBJ3usLayxnaEgBSzkWdvmkX9O49T1SaYp9vf/Q9kbdcWQpWgRw1q4VMxmGTKGnyzDseJcspk6oCfe8QLeo8O8BLauuxICV+zrgajnyrAwZKKFArFiKZaZJDt0w5ivCjnf47JOBxyQPvujngfur+wtkWOj5n35gUQqH/QCErtVygUfePknIdol94ds0aLajqIobKpY0DPPeIF3Td8bbFuEyuNL/W4w7oxmlVqv0KhCMiZi7/O3jN+SiUZOKSgkgyYdBXAgCcYjXiXi29Y2w8Mq3uUxHWTXKCShxQKRa+cufjrsPjrAGQDVzZ3sHzfeo5Wt3DexIwBO++It9AfWjQJg4dPK1fUBhgp4MG9SswVCkWfyUiIITHWwNGa1gE9T0iCLoS4XAhxUAhxRAixLMCYG4UQ+4UQ+4QQr4R3mgPHktl5TMtLRq8TCLQFDb8ov7lCoTgFtq19karHJzK+8wAHtn7AtrUvDti5enW5CCH0wG+BS4FSYJsQYq2zj6hrzETgEWC+lLJBCJE5UBOG8LcNs7RbuXRyFi/8nzOguFWLNbd6+NaV31yhUJwCriQjs+hinKjgU8dUztjxLf65cz2x1z4zJC3ozgKOSCmPSSm7gH8C1/qMuRv4rZSyAUBKWR3WWXoQ7rZhzR1Wjte2MtWVkjvjRrjmOc1f7lHXWLlaFApFX/FMMhqvK6eSNNqI5ctyHa1vfDvs7Q5DEfQ8wDPFqdS5zZPTgNOEEJuFEFuEEJeHa4K+BKuedyrsL28CYNq2ZbA8BZ6Zpu14cC8sb1R+c4VCccpkyu5oufGiHIBjMgch4CviA+xrvxvW84VrUdQATAQuBG4Gfi+ESPEdJIS4RwixXQixvaYmQFhgL4S7et6+7R8BMLV9GyC1phVvPaBqtigUin7juSY3XlRgxEalM2NUCPiS4/2wak0ogl4GFHi8z3du86QUWCultEopjwOH0ATeCynlKinlXCnl3IyMUwvdCXfbsL3795JJA5nC0r3R2k7l6z+kcNk7zF+5QXWBVygUp8TJOQ/haiU6XpRzIOZ2LtPvcO/XCbRSImEiFEHfBkwUQhQKIUzATcBanzFr0KxzhBDpaC6YY2GbpQfhrp63rzOTqbqSHtszZW1YfPQKhWLkcubir3N87E1IQCckBuHoOSiMpUR6FXQppQ24H1gHHABWSyn3CSFWCCEWO4etA+qEEPuBjcBDUsq6sM3Sg1DahlU3dTDnJx+w/kBV0GN1WO0ckblMEyU99pXLNPfr/vjoFQrFyGb8115EzL0zUCER2szh67MQUqaolPJd4F2fbY96vJbAd53/DTi9Vc/7+HAt9a1dPLXuIBdNygxYDOfzymbs6Jlq9La+26SJJ23eC6Gqw41CoThlrn6a13eU8SXH+3jKUZs08aT1yywP02mGZabolmPaw8Hnlc38N4iVvrdM85tPvewOd5hiJRkss97FWscCr7Gqw41CoegP32+/je9Y76PUkY5DCkod6Syz3sWfW84K2zmGZS2Xz47WccnkLA5XN/PrDUe4dEoWQvS00veVW0g2G8k/5wY4dykAW4rK+OD1PeDoDo1UHW4UCkV/yU0xs7ZxAWu7vI3FvDAai8POQj9Z30ZZYzsLJqTxzQsnsKfMwoeHtBBJKSVFJxr47/4q/rOvkq3H65mWl+Ql9qH46BUKhaKvDEY7xGFnoX/mdLecMz6dcRnxPLv+MM+tP4zu+Cc8u7mKndYxXuOvmp7T4xiqw41CoQg3g9EOcdgJ+pZjdaTGm5iYmYBOJ/jGheP58Zq9fPVELHnE8RPDy8zSHUUYTOgveIgJC3qEyysUCsWAMNDG4rASdCklW47WMW9cqjuyZencfPat+wPTrXu4Qf8RJuH0jTuAnY/DBUuHbsIKhUIRRoaVoJ+sb6fc0sG947pjyGMMen4mn0MYekaBSkupby9XhUKhiFqiVtD9ldDttGnW9znj0uDt78KOP4G0BzxGFemEL6RfoVAohpaoFHRXCV1X1UVXev7U3CTSE0xMeO8WKPnIPV4AUmrFcFy0SRM/ty7l2UGeu0KhUAwUUSnogUroFp1o5PICK8JDzF0IoYm6RFAu03jSdiM7ki4drCkrFArFgBOVgh4oDd8uJfMq/gZ6v7sBGNf5d0CL//y5ShZSKBTDiKgU9NwUM2U+on66KOFM3WGu0W0O+Dm70CGcnw93/KdCoVAMNVEp6A8tmuTlQwf4g+lp8kRt0M8Z5n6N41dfNdDTUygUiiEhKlP/Xen5Jr02/bwUM7kicLVeCVSlzYOrnx6kGSoUCsXgE5WCDpqoJ8QauPms0WxedjEiOd/vOJvU8e2u+7iw+kHVpEKhUAxrolbQ61u7qG/tYnxGvLZh4aNg9K5a1iZNfNd6L2sdC1STCoVCMeyJSh86wLGaFgDGZyRoG2Y4G1KsX4GjsdQdmuhZ11w1qVAoFMOZqBX0o76CDpqoz7iR81Zu6BEFA6pJhUKhGN5Ercvl6P4dmLCS91wePDMNile79w1G3WGFQqGINEISdCHE5UKIg0KII0KIZUHGXS+EkEKIueGboh+KV3P00H7GiQr0wgGWk/DWA25RV00qFArFSKRXl4sQQg/8FrgUKAW2CSHWSin3+4xLBL4N/G8gJurF+hUcdTzIVFHSvc3aDutXuH3pqkmFQqEYaYRioZ8FHJFSHpNSdgH/BK71M+4nwBNARxjn55fOxkpOyEzGi3LvHZbSgT61QqFQRCyhCHoecNLjfalzmxshxBygQEr5TrADCSHuEUJsF0Jsr6mp6fNkXXyRMBMHOsbrfATdIxZ9TVEZ81duoHDZO8xfuUHFoCsUimFPvxdFhRA64Gnge72NlVKuklLOlVLOzcjIOOVzHptyH4C3hW40a7HodJfXLWtsR9JdXleJukKhGM6EIuhlQIHH+3znNheJwDTgQyFECTAPWDuQC6NH42YCUJisBwQkF8A1z7n954HK66rEIoVCMZwJJQ59GzBRCFGIJuQ3AV9x7ZRSWoB013shxIfA96WU28M71W6OVreQmxxL/Pd2+t0fKIFIJRYpFIrhTK8WupTSBtwPrAMOAKullPuEECuEEIsHeoL+OFrTwvjMhID7U+KMfrerxCKFQjGcCSlTVEr5LvCuz7ZHA4y9sP/T8s+aojKefP9zyi0dxJv0rCkq6xGauKaojJYOW4/PGvVCJRYpFIphTdRkiroWOsstWlTk1+3/YPGaqbQ9cbpXluhT6w5idcgen483GVRcukKhGNZEjaC7FjrPEXsAmKs7hE5I4torvLJEA/nJLe3WQZurQqFQDAVRI+guoV6q/4TJosQ7Bt2VJUpgP7nynysUiuFO1Ai6S5CX6DfxXswPyRKN3gOcWaKqMJdCoRipRI2gu4S6XKb7H5Ccz5qiMrdrRi8EoApzKRSKkUPUCLqrguJLpltpkybvnUYz28Z/y50dCmCX0m2ZKzFXKBQjgagRdNBEffmPHyfu+t9q2aEeWaLf2T9RZYcqFIoRTXR2LHJ2JvKk/BX/dcFUdqhCoRgpRJWFHgwV3aJQKEY6w0bQVXSLQqEY6USny8UProXPp9YdpLyxndwUs1oQVSgUI4roEvTi1VoCkaVUa2ax8FEvX7pqO6dQKEYy0SPoxau1FH+rc5HT1RgaeiyQKhQKxUgkenzo61d0i7kLj5R/hUKhGOlEj6AHagCtGkMrFAoFEE2C7tEAOqTtCoVCMcKIHkFf+KjWCNoTj8bQCoVCMdKJHkGfcaPWCNon5V8tiCoUCoVGSFEuQojLgWcBPfCSlHKlz/7vAncBNqAGuENK+UWY5+o35V+hUCgUGr1a6EIIPfBb4ApgCnCzEGKKz7AiYK6UcgbwGvBkuCeqUCgUiuCE4nI5CzgipTwmpewC/glc6zlASrlRStnmfLsFUCuVCoVCMciEIuh5wEmP96XObYG4E3jP3w4hxD1CiO1CiO01NTWhz1KhUCgUvRLWRVEhxK3AXOApf/ullKuklHOllHMzMjLCeWqFQqEY8YSyKFoGFHi8z3du80IIcQnwI+ACKWVneKanUCgUilARUsrgA4QwAIeAhWhCvg34ipRyn8eY2WiLoZdLKQ+HdGIhaoBTjYRJB2pP8bMDTaTOTc2rb6h59Z1Indtwm9cYKaVfF0evgg4ghLgS+BVa2OLLUsr/J4RYAWyXUq4VQvwXmA5UOD9yQkq5+BQmGhJCiO1SyrkDdfz+EKlzU/PqG2pefSdS5zaS5hVSHLqU8l3gXZ9tj3q8viSck1IoFApF34meTFGFQqFQBCVaBX3VUE8gCJE6NzWvvqHm1XcidW4jZl4h+dAVCoVCEflEq4WuUCgUCh+UoCsUCsUwISoEXQixVAixTwjhEEIEDPMRQlwuhDgohDgihFg2SHNLFUJ8IIQ47Pz/qADj7EKIXc7/1g7gfIJeAyFEjBDiVef+/wkhxg7UXPo4r9uFEDUe1+iuQZjTy0KIaiHE3gD7hRDiOeeci4UQcwZ6Tn2Y24VCCIvH9RrwxgBCiAIhxEYhxH7n7/HbfsYMyTULcW5Dcc1ihRBbhRC7nfN63M+Y8P0mpZQR/x8wGZgEfIhW1dHfGD1wFBgHmIDdwJRBmNuTwDLn62XAEwHGtQzCXHq9BsB9wAvO1zcBr0bIvG4HfjPI36vzgTnA3gD7r0SrSySAecD/ImhuFwJvD/L1ygHmOF8noiUc+v47Dsk1C3FuQ3HNBJDgfG0E/gfM8xkTtt9kVFjoUsoDUsqDvQzrtSrkAHEt8Gfn6z8DSwbhnIEI5Rp4zvc1YKEQQkTAvAYdKeXHQH2QIdcCf5EaW4AUIUROhMxt0JFSVkgpdzpfNwMH6Fmob0iuWYhzG3Sc16HF+dbo/M83EiVsv8moEPQQ6WtVyHCRJaV0ZchWAlkBxsU6K01uEUIsGaC5hHIN3GOklDbAAqQN0Hz6Mi+A652P6a8JIQr87B9shuo7FSrnOB/l3xNCTB3MEzvdArPRLE5PhvyaBZkbDME1E0LohRC7gGrgAyllwGvW399kSJmig4GzfEC2n10/klK+Odjz8STY3DzfSCmlECJQHOgYKWWZEGIcsEEIsUdKeTTcc41i3gL+IaXsFEJ8Hc1iuXiI5xTJ7ET7TrUIrTTHGmDiYJxYCJEA/Bv4jpSyaTDOGSq9zG1IrpmU0g7MEkKkAG8IIaZJKf2ujfSXiBF02f/yASFVhTwVgs1NCFElhMiRUlY4Hy2rAxyjzPn/Y0KID9EsiHALeijXwDWmVGiF15KBujDPo8/zklJ6zuElIqPr1YB9p/qLp1hJKd8VQjwvhEiXUg5oESohhBFNMP8upXzdz5Ahu2a9zW2orpnHORuFEBuBywFPQQ/bb3I4uVy2AROFEIVCCBPa4sKARZN4sBb4qvP1V4EeTxNCiFFCiBjn63RgPrB/AOYSyjXwnO8NwAbpXI0ZQHqdl4+fdTGaD3SoWQvc5ozcmAdYPNxrQ4oQItvlZxVCnIX2Wx7QG7PzfH8ADkgpnw4wbEiuWShzG6JrluG0zBFCmIFLgc99hoXvNzmYK76n+h/wJTRfXCdQBaxzbs8F3vUYdyXa6vZRNFfNYMwtDVgPHAb+C6Q6t89Fa6gNcC6wBy26Yw9w5wDOp8c1AFYAi52vY4F/AUeArcC4QbpOvc3r58A+5zXaCJw+CHP6B1qFUKvz+3UncC9wr3O/QOune9T57+Y3wmqI5na/x/XaApw7CHNagLagVwzscv53ZSRcsxDnNhTXbAZaz+ViNKv8Uef2AflNqtR/hUKhGCYMJ5eLQqFQjGiUoCsUCsUwQQm6QqFQDBOUoCsUCsUwQQm6QqFQDBOUoCsUCsUwQQm6QqFQDBP+P2eKFNz8Jg6yAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_61_1.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "from sklearn.linear_model import LinearRegression, Ridge, Lasso\n", "from sklearn.preprocessing import PolynomialFeatures\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.pipeline import make_pipeline\n", "from sklearn.utils import resample\n", "\n", "np.random.seed(2018)\n", "\n", "n = 500\n", "n_boostraps = 100\n", "degree = 18 # A quite high value, just to show.\n", "noise = 0.1\n", "\n", "# Make data set.\n", "x = np.linspace(-1, 3, n).reshape(-1, 1)\n", "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2) + np.random.normal(0, 0.1, x.shape)\n", "\n", "# Hold out some test data that is never used in training.\n", "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2)\n", "\n", "# Combine x transformation and model into one operation.\n", "# Not neccesary, but convenient.\n", "model = make_pipeline(PolynomialFeatures(degree=degree), LinearRegression(fit_intercept=False))\n", "\n", "# The following (m x n_bootstraps) matrix holds the column vectors y_pred\n", "# for each bootstrap iteration.\n", "y_pred = np.empty((y_test.shape[0], n_boostraps))\n", "for i in range(n_boostraps):\n", " x_, y_ = resample(x_train, y_train)\n", "\n", " # Evaluate the new model on the same test data each time.\n", " y_pred[:, i] = model.fit(x_, y_).predict(x_test).ravel()\n", "\n", "# Note: Expectations and variances taken w.r.t. different training\n", "# data sets, hence the axis=1. Subsequent means are taken across the test data\n", "# set in order to obtain a total value, but before this we have error/bias/variance\n", "# calculated per data point in the test set.\n", "# Note 2: The use of keepdims=True is important in the calculation of bias as this \n", "# maintains the column vector form. Dropping this yields very unexpected results.\n", "error = np.mean( np.mean((y_test - y_pred)**2, axis=1, keepdims=True) )\n", "bias = np.mean( (y_test - np.mean(y_pred, axis=1, keepdims=True))**2 )\n", "variance = np.mean( np.var(y_pred, axis=1, keepdims=True) )\n", "print('Error:', error)\n", "print('Bias^2:', bias)\n", "print('Var:', variance)\n", "print('{} >= {} + {} = {}'.format(error, bias, variance, bias+variance))\n", "\n", "plt.plot(x[::5, :], y[::5, :], label='f(x)')\n", "plt.scatter(x_test, y_test, label='Data points')\n", "plt.scatter(x_test, np.mean(y_pred, axis=1), label='Pred')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Polynomial degree: 0\n", "Error: 0.32149601703519126\n", "Bias^2: 0.3123314713548606\n", "Var: 0.009164545680330616\n", "0.32149601703519126 >= 0.3123314713548606 + 0.009164545680330616 = 0.3214960170351912\n", "Polynomial degree:" ] }, { "name": "stdout", "output_type": "stream", "text": [ " 1\n", "Error: 0.08426840630693411\n", "Bias^2: 0.07968918676726028\n", "Var: 0.004579219539673833\n", "0.08426840630693411 >= 0.07968918676726028 + 0.004579219539673833 = 0.08426840630693411\n", "Polynomial degree: 2\n", "Error: 0.10398646080125035\n", "Bias^2: 0.10077114273548986\n", "Var: 0.0032153180657605086\n", "0.10398646080125035 >= 0.10077114273548986 + 0.0032153180657605086 = 0.10398646080125036\n", "Polynomial degree: 3\n", "Error: 0.06547790180152352\n", "Bias^2: 0.062082386342319454\n", "Var: 0.0033955154592040936\n", "0.06547790180152352 >= 0.062082386342319454 + 0.0033955154592040936 = 0.06547790180152355\n", "Polynomial degree: 4\n", "Error: 0.06844519414009442\n", "Bias^2: 0.06453579006728317\n", "Var: 0.003909404072811237\n", "0.06844519414009442 >= 0.06453579006728317 + 0.003909404072811237 = 0.06844519414009441\n", "Polynomial degree: 5\n", "Error: 0.05227921801205707\n", "Bias^2: 0.048187277304303125\n", "Var: 0.004091940707753964\n", "0.05227921801205707 >= 0.048187277304303125 + 0.004091940707753964 = 0.05227921801205709\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Polynomial degree: 6\n", "Error: 0.03781367141738898\n", "Bias^2: 0.03365768507152761\n", "Var: 0.004155986345861379\n", "0.03781367141738898 >= 0.03365768507152761 + 0.004155986345861379 = 0.03781367141738899\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Polynomial degree: 7\n", "Error: 0.027609773491022498\n", "Bias^2: 0.02299949826036597\n", "Var: 0.004610275230656537\n", "0.027609773491022498 >= 0.02299949826036597 + 0.004610275230656537 = 0.027609773491022505\n", "Polynomial degree: 8\n", "Error: 0.017355848195591973\n", "Bias^2: 0.010331721306655588\n", "Var: 0.007024126888936384\n", "0.017355848195591973 >= 0.010331721306655588 + 0.007024126888936384 = 0.017355848195591973\n", "Polynomial degree: 9\n", "Error: 0.026605727637189085\n", "Bias^2: 0.010018312644140933\n", "Var: 0.016587414993048166\n", "0.026605727637189085 >= 0.010018312644140933 + 0.016587414993048166 = 0.0266057276371891\n", "Polynomial degree: 10\n", "Error: 0.021592704588043153\n", "Bias^2: 0.010516485576652981\n", "Var: 0.011076219011390184\n", "0.021592704588043153 >= 0.010516485576652981 + 0.011076219011390184 = 0.021592704588043167\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Polynomial degree: 11\n", "Error: 0.07160048164228314\n", "Bias^2: 0.01443680008897583\n", "Var: 0.0571636815533073\n", "0.07160048164228314 >= 0.01443680008897583 + 0.0571636815533073 = 0.07160048164228312\n", "Polynomial degree: 12\n", "Error: 0.1154777721897675\n", "Bias^2: 0.01628578269590588\n", "Var: 0.09919198949386163\n", "0.1154777721897675 >= 0.01628578269590588 + 0.09919198949386163 = 0.11547777218976751\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Polynomial degree: 13\n", "Error: 0.22842468702166951\n", "Bias^2: 0.01975416527163567\n", "Var: 0.20867052175003387\n", "0.22842468702166951 >= 0.01975416527163567 + 0.20867052175003387 = 0.22842468702166954\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA/L0lEQVR4nO3deXxU5d3//9dnZrKHhKysCQkQEvbFEFzAFQTUira2isutdbdau/kr2vahLbf93nrXttalLrdFrVVRqbYuLBVX3ICwKihCIIFsZCNkX2bm+v1xJiEJAQZIcpLJ5/noPGbONvNJiu85uc51rkuMMSillApcDrsLUEop1b006JVSKsBp0CulVIDToFdKqQCnQa+UUgHOZXcBHcXHx5uUlBS7y1BKqT5lw4YNZcaYhM629bqgT0lJITs72+4ylFKqTxGRvCNt06YbpZQKcBr0SikV4DTolVIqwPW6NnqlVP/Q3NxMfn4+DQ0NdpfSp4SGhjJ8+HCCgoL8PkaDXilli/z8fAYMGEBKSgoiYnc5fYIxhvLycvLz80lNTfX7OG26UUrZoqGhgbi4OA354yAixMXFHfdfQRr0SinbaMgfvxP5nQVM0B+sa+Yvq3eyNb/S7lKUUqpXCZg2enHAn1d/S5BLmDR8oN3lKKX6AKfTycSJE1uXr7jiCu6++24bK+oeARP0UaFBDIoKYVdJjd2lKKX6iLCwMDZv3nzUfTweD06n84jL/h5np4BpugEYlRBJTmmt3WUopfq4lJQUFi1axLRp03jttdcOW3755ZeZOHEiEyZMYNGiRa3HRUZG8otf/ILJkyfz+eef2/gTtBcwZ/QAoxMjeX1jAcYYvcijVB/yu7e2sb2wqkvfc9zQKO77zvij7lNfX8+UKVNal++55x4uv/xyAOLi4ti4cSMAd999d+tyYWEhp556Khs2bCAmJobzzz+ff/3rX1xyySXU1tYyY8YM/vjHP3bpz3KyAi7oaxrd7K9qZHB0qN3lKKV6uaM13bQEfsfl9evXc/bZZ5OQYA0UedVVV/Hxxx9zySWX4HQ6+d73vtetNZ+IwAn66mK+u/5KvnCcR07pDA16pfqQY5152yEiIuKoy50JDQ3tNe3ybQVOG31YLBEHvmGcI08vyCqluk1WVhYfffQRZWVleDweXn75Zc466yy7yzqqwDmjdwVD3CgySov4SINeKeWHjm308+bN44EHHjjqMUOGDOGBBx7gnHPOwRjDhRdeyIIFC7q50pMTOEEPSPwYMg5s5m8a9EopP3g8nk7X5+bmHnV54cKFLFy48LDjamp6Z/YETtMNQEIGQzxF5JUcsLsSpZTqNfwKehGZJyI7RGSXiBx225iI3CoiX4rIZhH5RETGtdl2j++4HSIytyuLP0xCOk48RNTupaqhuVs/Siml+opjBr2IOIHHgfnAOGBh2yD3eckYM9EYMwX4X+BPvmPHAVcA44F5wF9979c9EtIBGC0FekFWKaV8/DmjzwJ2GWN2G2OagKVAuysPxpi2dzpEAMb3egGw1BjTaIzZA+zyvV/3iEvDIKRJATka9EopBfh3MXYYsK/Ncj4wo+NOInI78HMgGDi3zbFfdDh22AlV6o/gcBiYzJiKAr4s1aBXSinowouxxpjHjTGjgEXAb47nWBG5WUSyRSS7tLT0pOqQhHTGuor0jF4ppXz8CfoCIKnN8nDfuiNZClxyPMcaY542xmQaYzJbbis+YQnpJJsC9pR07bgZSqnAk5uby4QJEw5bf+ONN7J9+3YbKuoe/gT9eiBNRFJFJBjr4uqbbXcQkbQ2ixcCO32v3wSuEJEQEUkF0oB1J1/2UcSnE2Sa8Vbk0ujuvI+sUkodzTPPPMO4cR37nPRdxwx6Y4wbuANYBXwNvGqM2SYii0XkYt9ud4jINhHZjNVOf63v2G3Aq8B2YCVwuzGme9M3IQOAkVJAblldt36UUqrvc7vdXHXVVYwdO5bLLruMuro6zj77bLKzswG47bbbyMzMZPz48dx3332tx919992MGzeOSZMmcdddd9lVvl/8ujPWGLMcWN5h3b1tXv/kKMf+Hvj9iRZ43BLGAFg9b0prSB88oMc+Wil1glbcDcVfdu17Dp4I848+nAHAjh07+Nvf/sYZZ5zB9ddfz1//+td223//+98TGxuLx+PhvPPOY+vWrQwbNow33niDb775BhGhsrKya2vvYoF1ZyxAaDTeyMGMdhRqX3ql1DElJSVxxhlnAHD11VfzySeftNv+6quvMm3aNKZOncq2bdvYvn070dHRhIaGcsMNN/D6668THh5uR+l+C6ixblo4EtIZV1vAxxr0SvUNfpx5d5eOkxS1Xd6zZw8PPfQQ69evJyYmhuuuu46GhgZcLhfr1q3jvffeY9myZTz22GO8//77PV263wLvjB4gIYNUCsgpqba7EqVUL7d3797Waf9eeuklZs6c2bqtqqqKiIgIoqOj2b9/PytWrACswcsOHjzIBRdcwJ///Ge2bNliS+3+CtCgH0OYqae2LA+v1xx7f6VUv5Wens7jjz/O2LFjOXDgALfddlvrtsmTJzN16lQyMjK48sorW5t4qqurueiii5g0aRIzZ87kT3/6k13l+yUgm26It8a8SfLkU1BZT1Js724/U0rZIyUlhW+++eaw9R9++GHr6+eee67TY9et696e4l0pQM/orS6WLT1vlFKqPwvMoI+Ixxsao6NYKqUUgRr0IjgSM8hwFekZvVKq3wvMoAeIH8NoKSCnpNbuSpRSylaBG/QJGUSZKspKjjb+mlJKBb4ADnprKISE+lwqaptsLkYppewTwEFv9bwZ7dCeN0qpw51zzjmsWrWq3bqHH364XT/6o7n33ntZvXp1d5TW5QI36KOG4Q2K0J43SqlOLVy4kKVLl7Zbt3TpUhYuXHjMYz0eD4sXL2b27NndVV6XCtygF0ES0hmjg5sppTpx2WWX8c4779DUZDXt5ubmUlhYyMsvv9zpsMQpKSksWrSIadOm8dprr3HdddexbNkyABYvXsz06dOZMGECN998M8ZYd+SfffbZLFq0iKysLMaMGcOaNWsA64virrvuYsKECUyaNIlHH30UgA0bNnDWWWdxyimnMHfuXIqKirrkZw3MO2N9JCGd9KJV/J823SjVqz247kG+qTj8DtWTkRGbwaKsRUfcHhsbS1ZWFitWrGDBggUsXbqUH/zgB/zqV786bFjiSZMmARAXF8fGjRsBWLlyZet73XHHHdx7rzVy+zXXXMPbb7/Nd77zHcAa737dunUsX76c3/3ud6xevZqnn36a3NxcNm/ejMvloqKigubmZn784x/z73//m4SEBF555RV+/etfs2TJkpP+XQTuGT1AQjrxpoKi/fvtrkQp1Qu1bb5pabbpbFjiFpdffnmn7/PBBx8wY8YMJk6cyPvvv8+2bdtat333u98F4JRTTiE3NxeA1atXc8stt+ByWefasbGx7Nixg6+++oo5c+YwZcoU7r//fvLz87vk5wzoM/qWMW/Cq3Kob/IQFuy0uSClVGeOdubdnRYsWMDPfvYzNm7cSF1dHbGxsZ0OS9wiIiLisPdoaGjgRz/6EdnZ2SQlJfHb3/623TEhISEAOJ1O3G73EWsxxjB+/PjWkTS7UsCf0QOMlgJ2l2nzjVKqvcjISM455xyuv/56Fi5ceMRhiY+mJdTj4+Opqalpbbc/mjlz5vDUU0+1Bn9FRQXp6emUlpa2Bn1zc3O7vwxORmAHfUwKXmeI9rxRSh3RwoUL2bJlCwsXLjzisMRHM3DgQG666SYmTJjA3LlzmT59+jGPufHGG0lOTmbSpElMnjyZl156ieDgYJYtW8aiRYuYPHkyU6ZM4bPPPuuKHxFpuTrcW2RmZpqWSXm7gvevp/NhUTCbZz3Fz89P77L3VUqdnK+//pqxY8faXUaf1NnvTkQ2GGMyO9s/sM/osaYVzHAVklOqY94opfqngA96EjIYbErYt7/M7kqUUsoWfgW9iMwTkR0isktE7u5k+89FZLuIbBWR90RkRJttHhHZ7Hu82ZXF+yVhDA4MUr4Lt8fb4x+vlDqy3tZ03BecyO/smEEvIk7gcWA+MA5YKCLjOuy2Ccg0xkwClgH/22ZbvTFmiu9x8XFXeLJ8Y96MMPnkH6jv8Y9XSnUuNDSU8vJyDfvjYIyhvLyc0NDQ4zrOn370WcAuY8xuABFZCiwAWu8iMMZ80Gb/L4Crj6uK7hQ7CiNO0hxWz5uU+MP7wSqlet7w4cPJz8+ntLTU7lL6lNDQUIYPH35cx/gT9MOAfW2W84EZR9n/BqBt59NQEckG3MADxph/HVeFJ8sVjDcmldGlBewqrWE2g3r045VSnQsKCiI1NdXuMvqFLr0zVkSuBjKBs9qsHmGMKRCRkcD7IvKlMSanw3E3AzcDJCcnd2VJADgTM8io2MR72pdeKdUP+XMxtgBIarM83LeuHRGZDfwauNgY09iy3hhT4HveDXwITO14rDHmaWNMpjEmMyEh4bh+AL8kpJNEMXklB7r+vZVSqpfzJ+jXA2kikioiwcAVQLveMyIyFXgKK+RL2qyPEZEQ3+t44AzatO33mPh0XHhoKs3RCz9KqX7nmE03xhi3iNwBrAKcwBJjzDYRWQxkG2PeBP4ARAKviQjAXl8Pm7HAUyLixfpSecAY0/NB75tWcEjTXkqrG0mMOr4r1kop1Zf51UZvjFkOLO+w7t42rzudZsUY8xkw8WQK7BLxVtCnST67Sms06JVS/Urg3xkLEByBO2o4ox2F5OgFWaVUP9M/gh5wJo7VaQWVUv1Svwl6SUhnpBSyu6TK7lKUUqpH9ZugJyGdEJqoLdljdyVKKdWj+k/Q+6YVHFi7m+qGZpuLUUqpntN/gt7XxXK0FLBbx6ZXSvUj/Sfow2JwhyeSptMKKqX6mf4T9IAjMYM0RyG7SjXolVK9y5/e/Zb73+6e+0n7V9AnpJPmKCBnf7XdpSilVCuP1/DS2r0UVHbPnBn9KuhJSCeCeipL9tpdiVJKtVqfW0FZTSMXThrSLe/f74IeILRyJ01unVZQKdU7vLO1iNAgB+dmJHbL+/ezoLemFRxJAXsrtOeNUsp+Hq9hxVfFnJuRSHhwl04R0qp/BX1EAu6QaO15o5TqNVqabS6Y2D3NNtDfgl4ESchgtEODXinVO3R3sw30t6AHnInpjHEUkqM3TSmlbNYTzTbQD4OehAxiqKKk+LDZEJVSqket29P9zTbQH4PeN+aNo3wHXq9OK6iUss/yL7u/2Qb6Y9D7ulgmefIprmqwuRilVH/VU8020B+DPno4Hle4Na2gXpBVStmkpdnmwolDu/2z+l/Qi+CNG8Mo0dmmlFL2eefLQkKDHJyTkQDAe3nv8d7e97rls/pf0AOuQemkOwvI0cHNlFI28HgNK7/az3kZg1qbbR7b/Bh/3/b3bvm8fhn0kpDBICooKN5vdylKqX6oY2+bnQd2sqtyF/NT53fL5/XLoG+5IGvKdthciFKqP3rny0LCgpytzTYr9qzAIQ7mjJjTLZ/nV9CLyDwR2SEiu0Tk7k62/1xEtovIVhF5T0RGtNl2rYjs9D2u7criT5ivi2VCQx6VdU02F6OU6k+sZptDvW2MMazMXcmMwTOIC4vrls88ZtCLiBN4HJgPjAMWisi4DrttAjKNMZOAZcD/+o6NBe4DZgBZwH0iEtN15Z+gmBS8jiBGS6G20yuletTaPeWU1TS1NttsL9/Ovup93dZsA/6d0WcBu4wxu40xTcBSYEHbHYwxHxhj6nyLXwDDfa/nAu8aYyqMMQeAd4F5XVP6SXC6cMeMYrR2sVRK9bDlXxYd1mzjcrg4N/ncbvtMf4J+GLCvzXK+b92R3ACsOJ5jReRmEckWkezS0lI/Sjp5rkFjSdMxb5RSPahjs43XeFmZu5KZQ2cSHRLdbZ/bpRdjReRqIBP4w/EcZ4x52hiTaYzJTEhI6MqSjsiRkE6SlJBXXN4jn6eUUi3NNi0zSW0q2cT+uv3d2mwD/gV9AZDUZnm4b107IjIb+DVwsTGm8XiOtUVCOg4MTSXf2l2JUqqfeGerr9km3RrbZsWeFYQ6Qzk76exu/Vx/gn49kCYiqSISDFwBvNl2BxGZCjyFFfIlbTatAs4XkRjfRdjzfevs5+tiOaA6h4Zmj83FKKUCndvjZdW2Ys4dm0hYsBO31827ee9yVtJZhAeFd+tnHzPojTFu4A6sgP4aeNUYs01EFovIxb7d/gBEAq+JyGYRedN3bAXw31hfFuuBxb519osbjcHBKClgT5m20yulupd1k1QTF/p626wrXkdFQwXzU7q32QbAryHTjDHLgeUd1t3b5vXsoxy7BFhyogV2G1cITdEjSKuwZpsaOyTK7oqUUgHsnS/bN9us3LOSiKAIZg6f2e2f3T/vjPVxDRrLaNExb5RS3atjs02Tp4nVe1dzXvJ5hDhDuv3z+3XQOxPTGekoZvf+SrtLUUoFsI7NNp8WfEp1U3W397Zp0a+DnoQMXHioK95pdyVKqQDWsdlmRe4KBoYMZMaQGT3y+f076OPHABBauQuPTiuolOoGbo/XuknK12xT11zHh/s+ZM6IOQQ5gnqkBg16YIQ3n4ID9TYXo5QKROv2VFBe28RFvmabjws+pt5d32PNNtDfgz4kksaIoaQ58tlVWm13NUqpAPS2r9nm7Da9bRLCEpiWOK3HaujfQQ84EjMYrdMKKqW6gdvjZdVXxZzna7apbqpmTf4a5qbMxelw9lgd/T7ogwaNZbSjkN379YxeKdW11vqabVp623yw7wOavE3MS+3ZQXz7fdCTkE4oTRzcv9vuSpRSAeadDs02y/csZ1jkMCbFT+rROjTofWPeOMt2YIz2vFFKdY2OzTYHGg7wReEXzEuZh4j0aC0a9L6eN0Oa91Jeq9MKKqW6Rsdmm3fz3sVjPD3a26aFBn14LE2h8aRJgV6QVUp1mXe+LCI8uE1vm9yVpEanMiZmTI/XokEPmPh0Rjs06JVSXaP1JqkMq9mmpK6E7OJs5qfM7/FmG9CgByB48FjSpICcEu15o5Q6eWv3VFBR28RFvpmk/pP7HwyGualzbalHgx6QxAwGSD1lxXvtLkUpFQDe3tq+2WZF7goyYjMYGT3Slno06KH1gqyUfmNzIUqpvq5lSOLzxg4iNMhJfnU+W0u32nIRtoUGPbR2sYypy6W20W1zMUqpvuyL3VazzYUTBwPWRViAuSn2NNuABr0lchDNQVGMlgJ2l+q0gkqpE3dYb5s9K5mcMJlhkcNsq0mDHkAEd2waaY4CHdxMKXXCOjbb7K7czY4DO2xttgEN+lbBQ8ZZ0wqW6Bm9UurEdNZsIwjnjzjf1ro06H2cienESxVFRQV2l6KU6qPaNtsYY1ixZwXTB08nITzB1ro06FskZADgKdlhcyFKqb6oY7PNNxXfkFuV2+MjVXbGr6AXkXkiskNEdonI3Z1sP1NENoqIW0Qu67DNIyKbfY83u6rwLufrYhlZtYtmj9fmYpRSfc2hZhvrJqkVuStwiYs5yXNsrgxcx9pBRJzA48AcIB9YLyJvGmO2t9ltL3AdcFcnb1FvjJly8qV2s+gk3M4wUt0F7K2oY1RCpN0VKaX6kHe+LCQi2MnZ6QkYY1i5ZyWnDT2NgaED7S7NrzP6LGCXMWa3MaYJWAosaLuDMSbXGLMV6Lunwg4HTQNHkSb5OuaNUuq4tIxt09Jss6V0C0W1Rbb3tmnhT9APA/a1Wc73rfNXqIhki8gXInJJZzuIyM2+fbJLS0uP4627VtDgsYxyFJJTqkGvlPLf57vLOVDXzAW+ZpuVuSsJdgRzTtI5Nldm6YmLsSOMMZnAlcDDIjKq4w7GmKeNMZnGmMyEBPuuTgcNymCYlLOvqMS2GpRSfc/yL4tam208Xg+rcldx5vAziQzuHU3A/gR9AZDUZnm4b51fjDEFvufdwIfA1OOor2f5et40FeuYN0op/zR3aLbZsH8DZfVlvaK3TQt/gn49kCYiqSISDFwB+NV7RkRiRCTE9zoeOAPYfvSjbOQb8yakcqdOK6iU8ssXHZptlu9ZTrgrnDOHn2lzZYccM+iNMW7gDmAV8DXwqjFmm4gsFpGLAURkuojkA98HnhKRbb7DxwLZIrIF+AB4oENvnd4lJhWPuEjy7GN/VaPd1Sil+oB3th5qtmn2NLN672rOST6HMFeY3aW1Omb3SgBjzHJgeYd197Z5vR6rSafjcZ8BE0+yxp7jdNEYlcroCmu2qcHRoXZXpJTqxZo73CT1cf6nHGw8yPyU3tHbpoXeGduBY9BYa8wb7XmjlDqGlmabC30zSa3cs5IBwQM4fejpNlfWngZ9ByGDM0h2lJBbXG53KUqpXq6l2easMQk0uBt4f9/7zBkxhyBnkN2ltaNB34EkZuDEUFekY94opY6sY7PNJwWfUNtcy7yU3tPbpoUGfUfxVs8bV8W3NheilOrNPs9p32yzYs8KYkNjmT54us2VHU6DvqO40XhxkNiYR1VDs93VKKV6qZabpM4ak0Btcy0f5X/E3JS5uBx+9XHpURr0HQWF0hCZxGgd80YpdQTNHi8rtxUze5zVbPPBvg9o9DT2mrFtOtKg74RJSGe0FJKjQa+U6sTnOeVUth3bZs9KBkcMZnLCZJsr65wGfSfChowlVYrIKam0uxSlVC/UtrfNwcaDfFr4KfNS5uGQ3hmpvbMqmzkSxxIsHqoLdtpdilKql2n2eFm1/VCzzXt738PtdfeqsW060qDvTII125SjXLtYKqXaa2m2aZ1Jas8KkgckMy52nM2VHZkGfWd80wpG1+ym0e2xuRilVG/yztYiIkNcnDkmgbL6MtYVr2Ne6jxExO7SjkiDvjMhA6gLG8xIKSS3rM7uapRSvURLs815YxMJDXLyn9z/4DVeLki9wO7SjkqD/gg8sWN0WkGlVDufdWi2WZm7krSYNEYNPGw+pV5Fg/4IwoaOY5QUkVNSZXcpSqleYnmbZpuimiI2lWzqdSNVdkaD/ghcgzIIl0YqCnPsLkUp1Qs0uj1Wbxtfs82q3FUAvXJsm4406I/EN62gt0R73ijV3xljuPufX1JZ18z3M62ZVVfkrmBC3ASSopKOcbT9NOiPxDetYETVLrxenVZQqf7sL+/t5I1NBfxizhjOGB1PXlUe28u39+q+821p0B9JeCwNwbGM8OZTUFlvdzVKKZu8sSmfh1fv5HvThnPHuaMBa8gDQZibMtfm6vyjQX8UTTFppDkK2KWzTSnVL63dXc4vl23l1JGx/M93JyIiGGNYsWcF0wZNY3DEYLtL9IsG/VEEDxlnTSu4v9ruUpRSPSyntIabX9hAUmw4T12dSbDLisudlTvJOZjTJ3rbtNCgP4rQIWOJljpKivLsLkUp1YMqapu4/rn1uBzCc9dlER1+aGrAlXtW4hQns0fMtrHC46NBfzS+C7LNxd9068ccrGvmrS2FOtyCUr1AQ7OHm/+eTdHBBp7+r0yS48Jbt7U028wYMoO4sDgbqzw+fgW9iMwTkR0isktE7u5k+5kislFE3CJyWYdt14rITt/j2q4qvEf4phUMrdzVLW9f1+Tm8Q92Met/3+fHL2/i9hc30eT2dstnKaWOzRjDL5dtJTvvAH/6wWROGRHTbvu28m3k1+T3ib7zbR1zzisRcQKPA3OAfGC9iLxpjNneZre9wHXAXR2OjQXuAzIBA2zwHXuga8rvZgMG0+iKZEjDXipqm4iNCO6St210e3hp7V4e/2AXZTVNfGdMOHNDd/Lzrc3c+bLw6JVTCXLqH1tK9bQ/v/stb24p5Jfz0rlo0tDDtq/YswKXw8V5I86zoboT58/khlnALmPMbgARWQosAFqD3hiT69vW8XR0LvCuMabCt/1dYB7w8klX3hNEaIgeTVpjAbtKashKjT2pt3N7vLy+sYC/vLeTgsp6zkiN4vVTtpG89S9QX8HE4Wcze9v1/OwV4eHLp+DSsFeqxyzbkM8j7+/i8swkbjvr8LFrPF4PK3NXMnPYTKKCo2yo8MT5kyTDgH1tlvN96/zh17EicrOIZItIdmlpqZ9v3TOcg8Yy2lFAzkl0sfR6DW9vLeT8P3/ML/+5lfiIIJbPreYfTT8jee19MGg8nPlLRpR9yKqk51mxNZ+7XtuCR2/UUqpHfJZTxj2vb+WM0XHcf+mEw4YcNsbwUPZDlNSVcMmoS+wp8iT0iunKjTFPA08DZGZm9qp0Cx86jsjtL5NfWAAkH9exxhg+3FHKH1btYHtRFWMGRfLSRWGctutB5KM1EJcGC5fCmHkgAmExjFx1DyuSXczbfA0up4P//d4kHI7eO861Un3drpIabn1hAyPiIvjrVad02mz63Lbn+MfX/+CacddwbvK5NlR5cvwJ+gKg7WAOw33r/FEAnN3h2A/9PLZXcCRaY940Fn0NnOb3cV/sLucPq3awIe8AybHhPHnxEM7f/zSO1UshPBYueAhOuQ6ch7ptcdqPwNPImNW/5a1kFxdtWEiQU/j9JRM17JXqBuU1jfzwuXUEuxw8e910osOCDtvnrZy3+NOGPzE/ZT53Zd7VqycYORJ/gn49kCYiqVjBfQVwpZ/vvwr4fyLScun6fOCe467STr5pBYMqvvVr9635lfxh1Q7W7CxjUFQID35nJJc1/BPn+4+B8cAZd8KsX0BodOdvMPNn4Glm/Ae/51/JLi5Z931cDgeLF4zvk//AlOqtGpo93PT3bEqqGll686kkxYYfts9nBZ9x76f3MmPwDO6feX+vnfz7WI4Z9MYYt4jcgRXaTmCJMWabiCwGso0xb4rIdOANIAb4joj8zhgz3hhTISL/jfVlAbC45cJsnxGdTLMjhLj6XOqbPIQFOzvdbef+av74n29Zua2YmPAgfjN/DNeGfULQxz+Cmv0w/rsw+z6ISTn2Z571S3A3MnnNQ7yWHMRlX1yKyynce9E4DXuluoDXa/jFa1vYuLeSJ66axtTkmMP22Va2jZ9++FNGDRzFw+c8TLCza3rd2cGvNnpjzHJgeYd197Z5vR6rWaazY5cAS06iRns5HNQNGEVahXVBdsKw9mfi+yrq+PPqb3ljUwERwS5+OjuNm4flEf7BtVCyDYZnweUvQtL04/vcc38DniYyP3uEl5OdLPz0YoKcDu6Zn6Fhr9RJeug/O3hnaxH3zM9gvm+2qLb2Ve3jR+/9iNjQWJ6Y/QSRwZE2VNl1esXF2N5OEtMZVbmGDW2Cfn9VA4++v5NX1u/DIcJNs0Zy+/hmotf8Cj55FwaOgO8/B+MusS60HveHCsxZDJ5mTlv7BH9PDuK/Pp5PkFO46/x0DXulTtAr6/fy1w9zWJiVzM1njjxse3l9ObesvgWv8fLE7CdICE+wocqupUHvh/Bh44ja+QZvFJVwIC2BJz/K4bnPcvF4DVdkJfGTGdEkZP8JnnseggfA+fdD1s3gCjm5DxaBef8DnkbOzF7C35Jd3PCBEOR08NPZY7rmh1OqH/lkZxm/fuMrZqXFd3rdq665jh+99yPK6st45vxnSI1OtanSrqVB7wfXoLEAbN68nqe/KKGmyc2lU4bxs7OTSNrxLDz7MLjrYfpNcNYiiOjCMTBE4II/gqeJ8zY9xxPJLm5bDUFOB7efM7rrPkepAPft/mpu+8cGRiVE8vhV0w7rRtnsbebnH/6cHRU7eOTcR5iUMMmmSrueBr0/fGPeRFXv5vSx0/jFnDTG7F8JL14FVfmQfqHVzBLfTcHrcMB3HgGPm/lbn+EvyUH8ZBW4HMItndzBp5Rqr7S6kR8+u57QYCdLfjidqND23SiNMdz36X18Wvgpi09fzJnDz7Sp0u6hQe+P2FSMw8W9MxzETGmCNxdA4SYYMhkufRJSZ3V/DQ4nLHgcPE0s2PYEzcku7loBLqeDG2YGxp+XSnWH+iYPN/49m/LaRl695TSGDQw7bJ+HNz7MW7vf4o4pd3Bp2qU2VNm9NOj94QxC4kYTs+3vsOlxiBoGlz4FE39gnW33WB0u+O7T4Gnism8epSnJya/ehiCn8F+npfRcHUr1EV6v4WevbGZrfiVPXn0Kk4YPPGyfF79+kSVfLeHy9Mu5edLNPV9kD9Cg99fw6VC5z+r2eOrtEHz4zRU9whkElz0Lr17Dld8+TOPwu7j33+ByOLhyxvEN0aBUoHtw5Tes3FbMby4cy9zxh0/7tzJ3JQ+ue5DZybO5J+uegO3NpkHvrwv/CPMfhOAIuysBVzB8/3lYeiXX5fyR5mG/5FdvgMsp/CAz6djHK9UPvLg2j6c+3s01p47otHlzXdE6frXmV0xNnMr/zPofnI7Ob4YMBH3zfl47uEJ6R8i3CAqFK15EUmdxU8UfuGvYVyz651be2JRvd2VK2e6jb0u599/bODs9gfu+c/gd5TsqdvCTD37CiKgRPHLuI4S6Qm2qtGdo0PdlQWGwcCmSdCq3VzzIj4d8wy9e3cKbWwrtrkwp23xTXMXtL24kLTGSx66cdti8DoU1hdy2+jbCg8J5YvYTRIccYdypAKJB39cFR8BVryLDTuFnlf/DzYN38rNXNrPiyyK7K1Oqx5VUNXD9s+sJD3by7A+nExnSvnW6sqGSW969hQZPA0/NforBEYe32wciDfpAEDIArl6GDJ7Aoqrfc11iDj9+eRP/2VZsd2VK9Zhmj5cfvbiRA3XNLLluOkOi23ejrHfXc/v7t1NYU8ij5z7K6Jj+c8OhBn2gCI2Gq19HEtL5Tc39XJmwh9tf2sj73+zv0o8xxlDT6GZfRR1b9lXywY4SPvq2lGaPTmqu7PXQf3aQnXeAB7438bDBB91eN7/86Jd8WfolD575IKcMOsWmKu2hvW4CSXgsXPNv5PmL+N2B+2mMu49bXxD+79pMzhrT+cBMjW4PB2qbqaht4kBdExW1hx4ty9ZzMxW1jRyobaapk1AfHBXKNaeN4IrpScRFnuQYP0odp/e+3s9TH+1mYVYyC6a0n63UGMP9X9zPh/kf8psZv2H2iNk2VWkfMaZXzdxHZmamyc7OtruMvq2mBJ67EHOwgF+E/pZ3DiRxxfQkqhvcVNQ1caC2iYq6Jipqmqht8hzxbaLDgoiNCCYmPIjYiBBiI4KIiQgmNjy49Tk2Mpiy6kZe+CKPNTvLCHY5uHjyUK47PeWwsyqlukNBZT0XPrKGIdFhvPGj0wkNat9N8vHNj/Pklie5aeJN3DntTpuq7H4issEYk9nZNj2jD0SRifBfbyLPXcBDNYtxJvw3r20QYsKDiYsMJiY8mJEJkcSEB7cL77hQiHc1EOOsY4CpwdVUDQ1lUH8AGg5CQ6X1XFx56HV9JThcnD/lSnLO+z7Pbq7inxsKWLYhn+kpMVx3eirnjx/U6TycSp2sJreXO17aiNtj+OtV0w4L+Vd3vMqTW57k0tGX8uOpP7apSvvpGX0gO5gPz15ghfLMn0NzvfW6vvJQcLd93Vx39PdzhULoQAgbaF0TaHldXQR7Pra2T/w+1VNv4pW8ATz/eS77KuoZEh3K1adqs47qeve/vZ1nPtnDY1dO5aJJQ9tte2/ve/z8w59zxtAz+Mu5fyHIcfh8sIHkaGf0GvSB7kAePP8dqMyzlkOirZAO8wV1aLQvuAe2CfGO633HBB3lppKSr2Htk7DlFWvI5pRZeGbcygeeaTz3xT4+2WU16yyYPJRrtVlHdYH/bCvm5hc2cM2pI/jvSya027apZBM3/ecmxsSM4ZnznyE8yKYhS3qQBn1/52mGxmorrLv7Nu+6Ctj4PKx7xhrCOSYFsm4hZ9gClmyo4PWNBdQ3e8hKieXa01OYO37QYTe0KHUs+yrquPCRNSTHhfPP204nxHXo33VOZQ7/teK/iA2N5e/z/05M6OHzwQYiDXrV8zxu+OYt+OJJ2PcFBEfClKuonnQ9r+wJPqxZZ2FWMrERfXfyZdVzmtxevv/kZ+wureXtO2cyIu7Q0CTFtcVcs+Ia3F43L8x/geEDOp3KOiBp0Ct7FW6yAv+rf4LXDWnn48m6lfebxvH853ntmnWuOyOF8UO1WUcd2e/e2sazn+byxFXT2k3sXe+u59oV17K3ei/Pzn2WsXFjbayy52nQq96hej9kL4Hsv0FtKSRkwIxb2Dn4Ip5bv79ds851Z6Rw/jht1lHtrfyqiFv/sZHrTk/htxePb11vjOGXH/+SVbmreOy8xwJuhih/nHTQi8g84C+AE3jGGPNAh+0hwN+BU4By4HJjTK6IpABfAzt8u35hjLn1aJ+lQd8PuBvhq9dh7RNQtMW64HvKtVRN/CGv7jSHNetcPj2JeO2t0+/tLa/jwkfXMDI+gtduPZ1g16GTgP/b+n88sukRfjrtp9ww8QYbq7TPSQW9iDiBb4E5QD6wHlhojNneZp8fAZOMMbeKyBXApcaYy31B/7YxZkInb90pDfp+xBjY+4UV+F+/BQiMvchq1qkd2dqs43II541N5AeZSZw1JkHP8vuhRreHy574nLzyWt65cxZJsYd60Xy470PufP9O5qfO54FZDwTs5CHHcrI3TGUBu4wxu31vthRYAGxvs88C4Le+18uAx6S//raV/0RgxGnWo3IfrP8/2PA8zu3/Zs6QycyZcRu7LpzDq5tKeX1jPqu27SdxQAjfO2U43z9lOCMTIu3+CVQP+f07X/NlwUGevuaUdiG/68AuFn28iLFxY/nd6b/rtyF/LP6c0V8GzDPG3OhbvgaYYYy5o80+X/n2yfct5wAzgEhgG9ZfBFXAb4wxazr5jJuBmwGSk5NPycvL64IfTfVJTbWw9RXr4m3ZDohIhKlX0zz5Kj4oieTV7H18sKMUj9cwPSWG72cmceHEIUSE6E3egertrYXc8dImbpyZym8uGte6/mDjQRa+s5C65jqWXrS03ww5fCQn23RzMkFfDUQaY8pF5BTgX8B4Y0zVkT5Pm24UYDXr7P4A1j4NO1eB8cKImTD1akqS5vL6Vwd4df0+dpfVEhHs5KJJQ/nB9OFMS47Rs7oAsqeslu88+glpgyJ59ZbTWofScHvd3Lr6Vjbu38iSuUuYkjjF3kJ7gZNtuikA2k5EOty3rrN98kXEBUQD5cb6FmkEMMZs8H0BjAE0ydXRicCoc61HVRFseQk2/QP+dSuJwQO4deL3uOWKa9jQPJFXN+Tz1tZCXsnex8iECH6QmcR3pw0jcUBgTw8X6BqaPdz+4kZcTuGxK6e1Gy/poeyHWFu0lsWnL9aQ94M/Z/QurKaX87ACfT1wpTFmW5t9bgcmtrkY+11jzA9EJAGoMMZ4RGQksMa3X8WRPk/P6NURGQN5n8GmF2Dbv6yhFhLHwdSrqc24jHdymngtex/rcw/gdAjnpCfyg8zhnJORqIOq9UG/euNLXlq7lyXXZXJuxqDW9a/vfJ37PruPq8dezaKsRTZW2Lt0RffKC4CHsbpXLjHG/F5EFgPZxpg3RSQUeAGYClQAVxhjdovI94DFQDPgBe4zxrx1tM/SoFd+aaiybsDa9AIUbABHEGRcAFOvIScqi9c2FvHPjfmUVjcSHxnC96YN4/uZwxmdOMDuypUf/r25gJ8s3cwtZ43knvmHbnzaVLKJ61ddz/RB0/nr7L/icui1mRZ6w5QKbPu3W806W5dCXTkMGApTrsQ9+So+Ko3glfX7eP+bEtxew7TkgfwgM4kLJw1hQGhgj2bYV+WU1nDxo58wdkgUL998autfY8W1xVz+9uVEBkXy0oUv9YtJvY+HBr3qH9xNsGO5Ffo571kXcFNmwdRrKE2ay7++quCV7H3sKqkhLMjJBROH8N1pw8hMiWk3KJayT32Th0v/+ikl1Y28c+fM1nlf2w5v8NIFLzFy4EibK+19NOhV/3OwADa/ZDXtVOZZwzNPvAwz9Wo2uVN4bUM+b20poqbRTViQkxkjY5k5Op5ZaQmMGRSpPXdssmjZVl7J3sdzP5zO2emJQPvhDR4991HOSjrL5ip7Jw161X95vZD3CWx8Ab5+E9wNMGgCTL2Guozv8lmh4ZNdZXy8s5TdpbUAJA4IYWZaPLPS4jljdLz23ukhr2/M5+evbuH2c0bx/83NaF3/zJfP8JeNf+En037CjRNvtLHC3k2DXimwZtP6apkV+kWbwRkMaedbj9GzKTCxfLrTCv1Pd5VxoK4ZgIzBA5iVFs/MtASyUmIJC9Zmnq62c381Fz/2KZOGR/PijTNah7loGd5gXuo8Hpz1oP6ldRQa9Ep1VPyl1Za//U2oLrTWJY6D0bNh9Gy8w2ewvbSRNTvLWLOzlOzcAzR5vAS7HExPiWHm6ARmpcUzbkgUDoeGz8moa3Kz4LFPqahtYvlPZjEoyvoLKqcyh6uWX8WIqBE8N+85wlxhNlfau2nQK3UkxljTIO5abT3yPgNvMwRFwMizYPR5MHoO9RHDWZdbwZpvS/lkVxnfFFcDEBsRzBmj45k1Op6ZafEMHahhdLx+8eoWXt+UzwvXz2BmWjygwxuciJO9M1apwCUCg8ZZjzPuhMYayF0DO9+FXe9avXiAsLg0zho9m7MyZsPcMyhpED7dVcaab8tYs6uMt7ZYfxWMSohgVloCM0fHc+qoOCJ1DJ6jejV7H//cmM+d56W1hrzb6+auj+6iuLaYJXOXaMh3AT2jV+pIjIHyHCvwd62G3E+si7muMEiZaTXzpM3BxKTybUkta3aWsmZnGWv3lNPQ7MXlEKYlxzArLZ5ZYxKYOCwaZy9v5vF6DTtLasgprWFQVCjJseHERwZ3S9v4juJqFjz+CdOSY3jhhhmtv5sH1z3IP77+B4tPX8ylaZd2+ecGKm26UaorNNdD7qeHgr98l7U+JsXXtj8HUmfR6AhlQ94B1uws45OdZXxVeBBjIDosiDNGx7W277cdbtcuHq/h66Iq1u2pYO2ectbtqWi9CN0iPNhJcmw4SbHhJLd5JMWGMzwmjNCg4784Xdvo5uLHPuFgvZvlP5nZ2rPpjZ1vcO9n9+rwBidAg16p7lCxx9e2/x7s+Qia66yePCNOb72oS0IG5bVNfJpTzie+M/6igw0ApMZH+Prux3PaqLgeuVPX7fGyrbCKtXvKWbu7gvW5FVQ1uAFIig0jKyWOGSNjGTckipLqBvaW17G3op69FXXsq6hjb0Ud9c2edu852Hfm3/pFEBfm+zKI6PSvAWMMP3tlM29uKeQfN87g9FFWk83mks38cNUPyRyUyROzn9DhDY6TBr1S3c3dCHs/t4J/52oo/dpaH5cG4y+1HoljMUBO6aFmni92l1PX5MHpEKYmDWRWWgKzxsQzaVh0l8yk1eT28mVBJV/srmDtngo25FZQ22QFdWp8BDNSY5kxMpYZqXF+XUg2xlBW09Qa/Hnlde2+BIqrGtrtHxbU8a+BMCpqm3jk/V38fM4Y7jwvDbCGN7ji7SsIDwrn5Qtf1uENToAGvVI97WA+fLsKtv/Lats3XohPbxP61g1BTW4vG/ceaA3+LwusZp4BoS7OGGX15DkzLYHkOP+aeRqaPWzeV9naFLMh7wANzV4A0hIjW0N9RmosiVFdfyNYQ7OH/AP1rcHf8mhZrvN9ycxKi+e5H2bhdEi74Q1evOBFRg0c1eV19XrGQMNBq3kwasgJvYUGvVJ2qimx7srd9i8r9DGQMPZQ6CeMad31QG0Tn+b4evPsLKXQ18wzIi68dYiG00bFER1mNfPUNbnZtLeStbvL+WJPBZv3VdLk9iICGYOjrDP21FiyUmOJs3mCdWMM5bVNFB9sIH3wAIKcDowxLPp4EStzVwbe8AbGQFON9f9/TQnU7IfaUuu5ZV1tyaHXnkYYngU3vntCH6dBr1RvUb3fF/pvWH32MZA43hf6l0B8Wuuuxhh2l9Xyie+mrc9zyqlt8uAQmJI0EICt+Qdxew0OgQnDoslKiWXGyDiyUmKJDj+xNn+P10N5QzlxoXE4Hd17F3CfHN6gqa59QLcEd8d1taXWdZuOxAHh8RCZaD0ifM+RgyB2pDXc9gnQoFeqN6oqOhT6ez+31g2aaAX++Eshrn0TRpPby+Z9la3NPA6BrFTr4mnmiJgTvpjrNV52HtjJuuJ1rCtex4b9G6huqibIEUTSgCSSo5IZMWAEI6JHWM9RI0gMTzzpLpc9NryBxw1N1dY9Ek211ll2U02b5WrrubGmk20dlhurobm2888Jj7PCOiLBej4syH1hHh4H3fAFqkGvVG9XVQjb/22F/r611rrBkw6d6cd23bC8xhhyKnNYV7yO9cXryd6fTWVjJQDDI4eTNSSL9Jh09tftJ68qj7yqPPZV76PR09j6HmGuMJIGJDEiakS7R/KAZGJDY48Z2i3DGyQPSOb5+c8f3/AGnmbrjLm62Pq9VRdZj6oiaziL2vL24d2m7qMTCI6E4AgI8T0HD2iz7HtExB8K7ZZQj4gHp73zG2jQK9WXHMw/FPr56611Q6YcCv2YlON6O2MMuVW5rC9e3xruFQ3WbJ5DI4YyffB0soZkMX3QdIZEdn4h0Gu87K/dT151HnkH86znqjz2Vu0lvzoft3G37jsgaID1V0BL+EclkxKVQnJUMlHBUUce3sAYqD/QJrRbArzQCvXqQmt9bSnQIbccLhgwxHpEJEBIJwF9tPAOjoCgcHD03SknNeiV6qsq9x4K/YIN1rqh06zAHzrVau8Vp+/Zehggv6GMdQe+Yd2Br1lfsZ3SxgMAJIbGkRU3kayESUyPn8LwyKHtjkUch8LO6wXjAa8HvO42rz2+127wenB7miis209ebRF5dUXk1e0nr76EvfWlFDYewLQJ5VhXOEHipMJdy5KY05nS0HAo0KuLrTuPOwqPs2YNGzDY6pHSEuhRvnUDhvqaQ/puSHeFfhH0bq+bP234E0MihjA0YihDI61HVHCUDm2qAsOBPKu75rY3oHBTu02FLifrQkNZHxrCurBQil3WzUZxbg9ZDQ1Mb2ggq76RZLebnvyvoVEg3+UiLyiIvCAXea4gCoOcfL+qhjlN+IK7bYi3vB7qC/TB4LK3t1Bf0S8GNSuvL2fZt8uod9e3Wx/uCmdo5FDrC6DN89DIoQyNGEpcWBwO6d9nAqpvMAOTqc+6iaopl1NV9g07irJZX/kt6yq/paCxHIAYVwSZUSO5IWokWQNSSQ2JQzBWP37je/Z6fMtHexjrgqE4rGYRh9P6y6Hldeuyb504rTPq1tfWfiHiYJTDxah2xzsgLBZCo61B5VS3C5igHxQxiLVXrqWysZLC2kKKaooorCmkqLaIgpoCimqL2FK6haqmqnbHBTmCGBIxhCGR1l8CQyKHMCxyWOsXQmJ4IkEOnURadY1mTzNVTVVUN1VT3VTd+rqqqard69btjVVUNx/a1+11t3u/qOAopg+ezjWDpzN98HRGDxytJy7qMAET9AAiQkxoDDGhMYyPG9/pPrXNta1fAIU1hRTWFlrLNUWsKVhDWX1Zu/0d4iAxPLH17D/UGUqIK4RQZyihrlBCnB1et3lu2TfMGUaIK6TdvnaO42GMwWu8eI0Xt3Fbz17r2WM8eLwe69n3uu1+Hbe1vMaAFy/GmNY2Wa85tGwwh+1jMNbrjssdnlvr9r1vy7q2bb8dt3X8edvt0+a4lt9Dy8/e8WdsXe/1tl/ueEyb30fb5Zrmmnbh3fEvzo5cDhdRwVGtj+jQaJIGJDEgeABRIVEMCB5gvQ6OYkTUCMbEjNFgV8fkV9qIyDzgL4ATeMYY80CH7SHA34FTgHLgcmNMrm/bPcANgAe40xizqsuqPwERQRGkxaSRFpPW6fZGTyPFtcXtvwx8Xwg5lTk0ehppcDdYz56Gw86w/OUSV7svjGBnMECnQdfuGSukrdz0f5+OAaaOTRCc4sQhDpwO69khjkPrOj47nO3WOR1OIoMiSYlKaRfQLaHd+rrNc4gzRK8pqS53zKAXESfwODAHyAfWi8ibxpjtbXa7AThgjBktIlcADwKXi8g44ApgPDAUWC0iY4zpvUkT4gxp7RbmD7fX3Rr+DZ4GGt3WF0Db5UZPI/Xueho97V+3fmH4nsH6C0IQrP9J63LLf/xtl4XD1x1pn5bgaXl2iAOXuKxnh+tQOB1lv9b10n57S7gBrZ/Z8edoWXe0fVrr9q1rt+zTMQQ729ayrrPA7Lit5XM7Bnrbn0mpvs6fM/osYJcxZjeAiCwFFgBtg34B8Fvf62XAY2L9l7QAWGqMaQT2iMgu3/t93jXl28/lcOFyuIgIirC7FKWU6pQ/pyzDgH1tlvN96zrdxxjjBg4CcX4ei4jcLCLZIpJdWlrqf/VKKaWOqVf8bWqMedoYk2mMyUxISLC7HKWUCij+BH0BkNRmebhvXaf7iIgLiMa6KOvPsUoppbqRP0G/HkgTkVQRCca6uPpmh33eBK71vb4MeN9YfdreBK4QkRARSQXSgHVdU7pSSil/HPNirDHGLSJ3AKuwulcuMcZsE5HFQLYx5k3gb8ALvoutFVhfBvj2exXrwq0buL0397hRSqlAFDBj3SilVH92tLFuesXFWKWUUt1Hg14ppQJcr2u6EZFSIO8k3iIeKDvmXr1PX60btHa7aO326K21jzDGdNo/vdcF/ckSkewjtVP1Zn21btDa7aK126Mv1q5NN0opFeA06JVSKsAFYtA/bXcBJ6iv1g1au120dnv0udoDro1eKaVUe4F4Rq+UUqoNDXqllApwARP0IjJPRHaIyC4RudvuevwlIkki8oGIbBeRbSLyE7trOl4i4hSRTSLytt21HA8RGSgiy0TkGxH5WkROs7smf4jIz3z/Vr4SkZdFJNTumo5GRJaISImIfNVmXayIvCsiO33PMXbW2Jkj1P0H37+XrSLyhogMtLFEvwVE0LeZ7nA+MA5Y6JvGsC9wA78wxowDTgVu70O1t/gJ8LXdRZyAvwArjTEZwGT6wM8gIsOAO4FMY8wErIEGr7C3qmN6DpjXYd3dwHvGmDTgPd9yb/Mch9f9LjDBGDMJ+Ba4p6eLOhEBEfS0me7QGNMEtEx32OsZY4qMMRt9r6uxwuawWbh6KxEZDlwIPGN3LcdDRKKBM7FGXsUY02SMqbS1KP+5gDDf3A/hQKHN9RyVMeZjrFFt21oAPO97/TxwSU/W5I/O6jbG/Mc3ix7AF1hzbPR6gRL0fk1Z2NuJSAowFVhrcynH42Hgl4DX5jqOVypQCjzra3Z6RkR6/cS/xpgC4CFgL1AEHDTG/Mfeqk7IIGNMke91MTDIzmJO0PXACruL8EegBH2fJyKRwD+Bnxpjquyuxx8ichFQYozZYHctJ8AFTAOeMMZMBWrpnc0H7fjashdgfVENBSJE5Gp7qzo5vkmK+lQ/bxH5NVaz64t21+KPQAn6Pj1loYgEYYX8i8aY1+2u5zicAVwsIrlYzWXnisg/7C3Jb/lAvjGm5a+nZVjB39vNBvYYY0qNMc3A68DpNtd0IvaLyBAA33OJzfX4TUSuAy4CrjJ95EakQAl6f6Y77JVERLDaib82xvzJ7nqOhzHmHmPMcGNMCtbv/H1jTJ84uzTGFAP7RCTdt+o8rJnQeru9wKkiEu77t3MefeAicifaTj96LfBvG2vxm4jMw2qqvNgYU2d3Pf4KiKD3XRxpme7wa+BVY8w2e6vy2xnANVhnw5t9jwvsLqqf+DHwoohsBaYA/8/eco7N9xfIMmAj8CXWf8O9+pZ8EXkZ+BxIF5F8EbkBeACYIyI7sf5KecDOGjtzhLofAwYA7/r+W33S1iL9pEMgKKVUgAuIM3qllFJHpkGvlFIBToNeKaUCnAa9UkoFOA16pZQKcBr0SikV4DTolVIqwP3/s0buYom3ux0AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_62_6.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "from sklearn.linear_model import LinearRegression, Ridge, Lasso\n", "from sklearn.preprocessing import PolynomialFeatures\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.pipeline import make_pipeline\n", "from sklearn.utils import resample\n", "\n", "np.random.seed(2018)\n", "\n", "n = 40\n", "n_boostraps = 100\n", "maxdegree = 14\n", "\n", "\n", "# Make data set.\n", "x = np.linspace(-3, 3, n).reshape(-1, 1)\n", "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)\n", "error = np.zeros(maxdegree)\n", "bias = np.zeros(maxdegree)\n", "variance = np.zeros(maxdegree)\n", "polydegree = np.zeros(maxdegree)\n", "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2)\n", "\n", "for degree in range(maxdegree):\n", " model = make_pipeline(PolynomialFeatures(degree=degree), LinearRegression(fit_intercept=False))\n", " y_pred = np.empty((y_test.shape[0], n_boostraps))\n", " for i in range(n_boostraps):\n", " x_, y_ = resample(x_train, y_train)\n", " y_pred[:, i] = model.fit(x_, y_).predict(x_test).ravel()\n", "\n", " polydegree[degree] = degree\n", " error[degree] = np.mean( np.mean((y_test - y_pred)**2, axis=1, keepdims=True) )\n", " bias[degree] = np.mean( (y_test - np.mean(y_pred, axis=1, keepdims=True))**2 )\n", " variance[degree] = np.mean( np.var(y_pred, axis=1, keepdims=True) )\n", " print('Polynomial degree:', degree)\n", " print('Error:', error[degree])\n", " print('Bias^2:', bias[degree])\n", " print('Var:', variance[degree])\n", " print('{} >= {} + {} = {}'.format(error[degree], bias[degree], variance[degree], bias[degree]+variance[degree]))\n", "\n", "plt.plot(polydegree, error, label='Error')\n", "plt.plot(polydegree, bias, label='bias')\n", "plt.plot(polydegree, variance, label='Variance')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The bias-variance tradeoff summarizes the fundamental tension in\n", "machine learning, particularly supervised learning, between the\n", "complexity of a model and the amount of training data needed to train\n", "it. Since data is often limited, in practice it is often useful to\n", "use a less-complex model with higher bias, that is a model whose asymptotic\n", "performance is worse than another model because it is easier to\n", "train and less sensitive to sampling noise arising from having a\n", "finite-sized training dataset (smaller variance). \n", "\n", "\n", "\n", "The above equations tell us that in\n", "order to minimize the expected test error, we need to select a\n", "statistical learning method that simultaneously achieves low variance\n", "and low bias. Note that variance is inherently a nonnegative quantity,\n", "and squared bias is also nonnegative. Hence, we see that the expected\n", "test MSE can never lie below $Var(\\epsilon)$, the irreducible error.\n", "\n", "\n", "What do we mean by the variance and bias of a statistical learning\n", "method? The variance refers to the amount by which our model would change if we\n", "estimated it using a different training data set. Since the training\n", "data are used to fit the statistical learning method, different\n", "training data sets will result in a different estimate. But ideally the\n", "estimate for our model should not vary too much between training\n", "sets. However, if a method has high variance then small changes in\n", "the training data can result in large changes in the model. In general, more\n", "flexible statistical methods have higher variance.\n", "\n", "\n", "You may also find this recent [article](https://www.pnas.org/content/116/32/15849) of interest." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "============================\n", "Underfitting vs. Overfitting\n", "============================\n", "\n", "This example demonstrates the problems of underfitting and overfitting and\n", "how we can use linear regression with polynomial features to approximate\n", "nonlinear functions. The plot shows the function that we want to approximate,\n", "which is a part of the cosine function. In addition, the samples from the\n", "real function and the approximations of different models are displayed. The\n", "models have polynomial features of different degrees. We can see that a\n", "linear function (polynomial with degree 1) is not sufficient to fit the\n", "training samples. This is called **underfitting**. A polynomial of degree 4\n", "approximates the true function almost perfectly. However, for higher degrees\n", "the model will **overfit** the training data, i.e. it learns the noise of the\n", "training data.\n", "We evaluate quantitatively **overfitting** / **underfitting** by using\n", "cross-validation. We calculate the mean squared error (MSE) on the validation\n", "set, the higher, the less likely the model generalizes correctly from the\n", "training data.\n", "\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAykAAAFLCAYAAADSycMSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAACUQElEQVR4nOzddXiV5RvA8e9zzro32KgBo7u7WwFFpBktAqIiij8Du7AVC5CUkJZSVJTukO6G0TE2NjZY7/398Z6NMdZsO7H7c127YOet59S9935SaZqGEEIIIYQQQlgKg7kLIIQQQgghhBApSZIihBBCCCGEsCiSpAghhBBCCCEsiiQpQgghhBBCCIsiSYoQQgghhBDCokiSIoQQQgghhLAokqQIIYQQQgghLIokKVZIKRWklIpSSkUopcKUUtuVUiOVUhb/fiqlHJRSS0zPQVNKtTZ3mYQoyKw5nqSklHrfFFPam7ssQhQU1hw/MrsfUUp9qJSKU0pFpvgpa57SFkwW/yES6eqiaZo7UBr4AngTmJEXF1JKGXP5lFuBAcD1XD6vECJnrDmeoJQqB/QCruX2uYUQmbLm+JHZ/cgiTdPcUvycy+XriwxIkmLlNE0L1zTtD6APMFgpVR1AKeWolPpGKXVRKXVDKTVZKeWcdJxS6g2l1DWl1FWl1DBTLUJ507ZZSqmflVJ/K6XuAm2UUsWVUkuVUsFKqfNKqdEpzmVQSo1VSp1VSoUopRYrpXzSKW+spmnfa5q2FUjIy9dGCJE91hZPUpiIfmMUm9uviRAia6wtfsj9iOWTJMVGaJr2H3AZaGF66AugIlAbKA+UAN4HUEp1BF4F2pu2tU7jlP2ATwF3YDuwEjhoOk874BWl1OOmfV8CngZaAcWB2+g3DUIIK2RN8UQp1QuI0TTt7xw8VSFELrOm+JEFXZRSoUqpo0qp5x/hPCIHJEmxLVcBH6WUAkYAYzRNC9U0LQL4DOhr2q83MFPTtKOapt0DPkzjXL9rmrZN07REoAbgq2nax6aah3PAtBTnGwm8o2naZU3TYkzn66mUssuj5ymEyHsWH0+UUu6msrycS89ZCJE7LD5+ZMFioArgCwwH3ldKBebgPCKH5CbStpQAQtG/UC7AXj0+AKCApL6cxYE9KY67lMa5Uj5WGiiulApL8ZgR2JJi+3KlVGKK7QlAEeBKtp+FEMISWEM8+RD4VdO0oEyfjRAiP1lD/MiQpmnHUvy6XSn1A9ATWJCd84ickyTFRiilGqAHha3ALSAKqKZpWlpfymuAf4rfS6axj5bi/5eA85qmVUjn8peAoZqmbct2wYUQFseK4kk7wF8p9YLpd19gsVLqS03TvszC8UKIXGZF8SO7NPQES+QT6e5l5ZRSHkqpJ4GFwFxN0w6bmkSnAd8ppfxM+5VI0WdzMfCMUqqKUsoFeC+Ty/wHRCil3lRKOSuljEqp6qZABDAZ+FQpVdp0LV+lVNcMyuyolHIy/eqglHJSKapYhBDmYYXxpB1QHb2ve230LibPIWPihMh3Vhg/MrwfUUp1VUp5K11DYDTwe3ZfF5FzkqRYr5VKqQj0WoN3gPHAMym2vwmcAXYqpe4Aa4FKAJqmrQJ+BDYk7WM6JiatC2malgA8iX4TcB69ZmQ64Gna5QfgD2C1qUw7gUYZlP0kes1KCeBf0/9LZ+1pCyHygFXGE03TQjRNu570g96t47amaZHZfQGEEDlmlfHDJKP7kb6mMkUAc4AvNU2bneErIXKV0jQt872ETVNKVQGOAI6apsWbuzxCCOsl8UQIkVMSP0RK0pJSQCmlupmaOb2BL4GVEhCEEDkh8UQIkVMSP0R6JEkpuJ4DbgJn0btIyPzfQoickngihMgpiR8iTdLdSwghhBBCCGFRpCVFCCGEEEIIYVEkSRFCCCGEEEJYFElSxCNRSlVVSu2RdU6yTynVRSm1yNzlEMISSCzJOaVUEaXUcaWUo7nLIoQlkHiS+5RSS5VSnfLzmladpCilgpRSsUqpwqke36+U0pRSAabf/U0v7i2lVLhS6ohSaohpW4Bp38hUP33yqMwVlFLRSqm5GeyjlFJfKqVCTD9fpvyiKaXaKqX2KaXuKKXOKaVGPEJ5MrvWVKXUSaVUYtJrlsonwDdaDgY3KaWKK6UuZ/MYB9Mf43SPU0o9oZTaqpQKU0pdV0pNV0q5p9g+y/S5Sfl+G7Nb/hTnG2O6zh2l1C8pbxSUUp8opQ4rpeKVUh+mPE7TtJVANaVUzZxeW+QOG44lY0wx4o5S6qpS6jullJ1pm59SaoHp8XCl1DalVEbrCWRWnnRjiVKqolLqd6VUsFIqVCn1r1KqUqpT5EssUUqtSvX+xCqlDmewfzul1Aml1D2l1AZlWiTOtM1HKbXI9HxvKaXmKaU8slv+LF7rG6XUaaVUhGmfQUnbNE27gb7ORI7/Fojss6a4oZQapfSb9hil1KxM9lVKqXFKqSum8m5USlVLsT3dz2IultdR6X9P7yj97+urqbb3Vvq9QIRS6phS6ulUp8ivePKhUiou1XtXNgvH/WJ638uneGyjKaYnnedkdsuehesGmGLLPdN71z7Ftgzfd/SZ18bldpkyYtVJisl5IDDpF6VUDcAl1T6/oi8yVBooBAwEbqTax0vTNLcUP3lVwz0R2J3JPiOAp4FaQE2gC/rsFyil7IHlwBT0xYv6AOOVUrVyWJ50r2VyEHgB2Jf6QKVUMaANsCKtE5u+vB9mcO3OwD/ZLO/rQHAm+3iif5GKA1XQF2n6OtU+X6V6vxOyWQ4AlL5q7lj0la9LA2WBj1LscgZ4A/grnVMsQG4sLIUtxpI/gLqapnmgr8xeC33VZAA30/H1AB9gNvCXUsoth+XJKJZ4mcpSCSiCvmp08srN+RlLNE3rlPL9AbYDv6Vz3cLAMvRVsH2APUDK93Mc4A2UAcqZnltG5UxXFq51F/019QQGAz8opZqm2D6PB2O3yB/WEjeuon9ef8nCvr2AoUAL9M/iDvTnkCSzz2KWKKWGZJAwfQhUQH/N2gBvKKU6mo4rAcwFXgU80O8L5qv7K9rn973JolTv3bmMdlZKNUePF2kZleI8qStyskTpFbFD0tm8ANiP/jl8B1iilPI1bcvwfdc07T/AQylVPyflyglbSFJ+BVJm8YPRVwZNqQEwS9O0u5qmxWuatt+0ymm+Ukr1BcKAdZnsOhj4VtO0y5qmXQG+BYaYtvmgfyl/1XS7geNA1RTXGWqqYbit9BrLjFZzz+haaJo2UdO0dUB0Gsd2APZpmpbWtqzoDPyd1Z2VUmWAAcDnGe2nadp8TdP+0TTtnqZpt4FpQLNsXOdJpdQBpbfEbFcZt3QMBmZomnbUdK1PePD1m236rEWkc/xG4Imslk3kKZuLJZqmndU0LSzpMCARKG/adk7TtPGapl3TNC1B07SpgAOmlaBN18mVWKJp2n+aps3QNC1U07Q44DugklKqkOnYfI0lSUw13S14+H1O0h04qmnab6ayfQjUUkpVNm0vA6zQNO2Opmnh6BVIKWucKyul1ii99eikUqp3BsXJ8Fqapn2gadoJTdMSNU3bBWwBmqQ4fhdQNpP3SOQ+q4gbmqYt0zRtBRCShd3LAFtNMSIBPSFIvsfI7LOolGps+tsZppQ6qJRqnYMiDwY+0TTttqZpx9H/jg8xbfMHwjRNW2W6D/oLPXFKuvE3SzzJCqW3ZP8EvJSDY7MTj9M6viJQF/hA07QoTdOWAoeBHqZdMnzfTTaSj/cstpCk7ETP7KoovctOX/QXNvU+E5VSfZVSpR7lYkqpSaYvXlo/hzI4zgP4GD3zz0w19BaMJAdNjyU16y8AnlFKGZVSTdBrGraartMVeBv9D54vevBYkJNrZUENIEfNkaYWoZbAmmwc9hP6c4vK5uVaAkdTPfaC6cZhr1Iq6QuKUqoOek3Tc+g1DVOAP1T6fb3Tev2KpLj5ysxxIEA9QhcRkWtsMZaglOqnlLoD3EJv5ZiSzn610ZOUM6bf8zKWtASua5qWdMOU37EkySBgi6ZpQelsf+A5aZp2F30th6TnNRF4UinlrfSF6HoAq0zlcjWVaT7gh/55mqSUSv1HP6vXSqaUcka/8T2aYv949Pcup63qImesIm5k00KgnNK7adqjJwxptiyk/iyaWjn+Qm+18QFeA5amqK3PlOm7VIz048ke4LhS6inTfdDTQAyQ9PzzO550Md1PHFVKZbbGyxhgs6Zp6b1Xnyu9W+C2lMldDuJxWqoB5zRNS1lpmvJ1zcr7fpx8jDG2kKTA/ZqMDugv4JVU23uhv6HvAedNteQNUu1zK9WXvUpaF9I07QVN07zS+cmoxv0T9Br3rPRzdAPCU/weDrgplTxWZAHwPvqXcgvwjqZpl0zbRgKfa5p23PRH6zOgdgYZd2bXyogX6bcQZKYlcDDVlyVdSqlugFHTtOXZuYhSqgP6F+39FA//iN6M7If+mZillEpqaRkBTNE0bZepdnk2+uvcOJ1LpPX6AbinsW9akp6/Vxb3F3nL1mJJUsuiB1ARmMzD3UySEp9fgY9MLQKQR7FEKeWPfnOfMsnyIp9iSSqDgFkZbE/9nDD9nvT93oee2IWYfhKASaZtTwJBmqbNTKo9B5aif4Zycq2UJqPfXPyb6vEIJJaYgzXEjey4hl7xeRK9UrAX+s11WlJ/FgcAf2ua9reppWUNelLRORvXT+pymjqeuAOYavnnoFcAxJj+fc6U2EP+xpPF6N3KfYHhwPtKqcC0dlRKlUSvAH0/re3Am+hdxksAU4GVSqmk1qHsxuO0ZBZjsvK+52uMsaUkpR96U+BDzfam5sKxmqZVQ+8zfABYkeqPZ+FUX/bjuVU4Uw1le/QuDlkRid6lK4kHEKlpmqb0pv+F6AHRAT0DfkMpldT8Vhq9f2iYUioMCEXv5lFCKfW2uj8ga3Jm18pCOW+T6g+oUurPFNceC4xNEXT/TLFrcnOqUqp/inI91ARuqpH8ivt96bNEKdUYPXj11DTtVNLjmqbt0zQtxHTj8Dd6X+7ups2lgf+l/GMBlASKp1POtF4/yHqATHr9wrLz3ESesbVYkkzTtNPotZ2TUj5uqgldCezUNC1lV8pcjyWm2tTVwCRN01LWAuZLLEl1/uZAUWBJBrulfk5Jzyvp+70YOGUquwd6y0dSLXppoFGqWNIfKKqUKpWinJFZvFZSub9GH1/UO4047Y7EEnOw6LiRA++jt46UBJzQx1muV0o9MNYmnc9iaaBXqs99c/SWkQdagtBjUb8U+ya1LiR9J1LHkwjTOdqj3xO0Rr8PagVMN8VHyMd4omnaMU3TrpoqNbcDPwA909oX+B74OEVFUOpz7dI0LULTtBhTBek27id36cZjU1kPpdjWD73VNun5JcX8zGJMVt73/I0xmqZZ7Q8QBLQ3/X8jcAdwBewADQhI57jqpu2FgADT/+2yeM3J6G90Wj9H0znmFfT+ktdNP5HoWeq+dPbfDgxP8ftQ9BsI0D/8+1Pt/z0wwfT/f4H+2XgN071Wqv22AkNSPTYAWJPBuT8EPkxn23GgShbLWBuIS/H6haLXWF7P4D2uA9wEumTh/D8D403/n4LeMpXV128+8GmK39uid2NJvd/ctF4L9LEy5/Pi+yE/Wf+x1ViSxvED0GsJk353NMWMeYAh1b65GkvQB5jvB75Ip1x5HktSHTcNmJPJPiOAbSl+dwXuAZVNv0cCtVJsr42emIE+mDrd55Tda5ke+wg4AhRK43g70/6lc/v7IT/pvmdWETdSHT8OfXxMRvv8Cbyc6rEwoH6K39P8LAJvAdOy+FyGpFcW9IH+HVL8/jGw0PT/14DlqfZfAbxm+n++x5MUx78JLEtnWxh6S3ZS/NbQJwLql87+q4DRpv9nOR6jtw4PSePxiujji91TPLYZGJmN930a+piW/PmO5deF8qTwDwaIckkvZOoAgT5tWnXT4+7oXQ1Om7ZlK0DksJwu6DV2ST/foNfe+aaz/0jTF6UE+gxVR1N8iMqZglFb9Cy6HHo/5BGm7d1MgaOa6XdPoFcGZUv3WqbtDugZ9Tb0pkwnTDcz6DVCIYBTOudOMxCgD846l43Xzy7V69cdPYAVRe8Clnr/6qZA0Ced8/VEb/Y0AI+h1yK0Nm2rjz4LSyPT6+uKPkjMPZ1zdUQPNlXRm0DXk+ImDLA3vWbz0f84OKUsM3of00nm/i4V9B8bjiXDAD/T/6uavt9JCbk9egvKirTKnJuxBL227j9MlSlpHJsvsSTFcc7o3RzaZrKfr2m/Hqbv7pc8mHhtQB8r52z6mQRsN21zBy6gz+Rkb/ppQDo3QFm41lvAaaBoOsc3BY6Z+7tUkH6sJW6kKJMT+sQzv5r+n+Y1gQ/QKyaLoP+dHIheOeJl2p7uZxG9Fv468DhgNF2nNeCfxr5DSD9J+QLYhF65URm9K1JH07ZW6GPsapt+r2OKH4+Zfs+3eAJ0NZVRAQ3Ru/oNTmdfPx6M3xp6V3Jn9PuHx5PeF/RW17tARdOxWY7HpJOkmLbtRP+74WQ6Zximvx+Zve+mfU4BDfPtO5ZfF8qTwqcIEKkeTx0gfjJ9oSLRs9Y/Mf2h4H6ASF0j8WoelvtDYG6K31tgqn0z/a7QmzJDTT9fASrF9t6mD2sEcBk9ABpSbB+IPmPDHfQb7l8yKEtm19poen1S/rROsf030k8GPiTtQDCKdG5Wsvj6tQYup3osEmhh+v9M9FmM0qxhQu8bHG56fQ4CfVOdqyP61Kxh6IHxN9JJUkz7v4qeFN0xXdsxxbZZabx+Q1JsP0yKmlj5Mc+PDceSmabP5l3Tc/wa0x9u9D/0Gnrte8rytkhxfK7EEvRxYZqpHCmvVSrF8fkWS9BbOS6QItal2HaUFDWW6N3rTqC3WG0kRe04+k3NSvQbolD0QaYVUmyvhD6IONi0z3pMN1bplCuja2no/e9Tvn5vp9g+EVOtq/zkz481xQ3Tdyj136IPTdtKpfw+ot/ATkT/+3cHfexVx2x8FhuhJxihpuf7V8rveor9hpB+kuKIPonNHfQY9mqq7aPQK2gjgHPA/1Jtz5d4gj5GOMT0GpxI/R0kVUxNtU0Dypv+74t+3xGBfu+xkxQtSaZ9shSPyThJCUCPLVHoY0/ap9iW2fvegCy22ufWT9IfECFyROkz1cxGz6yz9GFSSv2NHgjyZIo/a6GU6gIM1DQto2lJhSgQJJbknNLXh9gE1NFyPu2qEDZD4knuU0otRZ+0Jd9eH0lSRL5TSr0B/KRpWnanEhZCiGQSS4QQuUXiieWRJEUIIYQQQghhUWxlCmIhhBBCCCGEjZAkRQghhBBCCGFR7PLipIULF9YCAgLy4tRCiBzau3fvLU3TfM1djuySeCKE5bGFeBJ06y7xiRrl/dwyPkgIkWcyiiV5kqQEBASwZ8+evDi1ECKHlFIXzF2GnJB4IoTlsYV48szM/7gVGcvKl5qbuVRCFFwZxRLp7iWEEEKIAkcphYZMHiSEpZIkRQghhBAFjgJkglMhLJckKUIIIYQocJSSJEUIS5YnY1JEwRIXF8fly5eJjpaFji2Bk5MT/v7+2Nvbm7soQmSLxBLLY9vxRElnLxsm8cSy5CSWSJIiHtnly5dxd3cnICAApZS5i1OgaZpGSEgIly9fpkyZMuYujhDZIrHEsth6PNFbUiRNsVUSTyxHTmOJdPcSjyw6OppChQpJELAASikKFSokNUfCKkkssSy2Hk/kU2bbJJ5YjpzGEklSRK6QIGA55L0Q1kw+v5bFlt8PGZNi+2z582ttcvJeSJIibIJSigEDBiT/Hh8fj6+vL08++WS2zhMQEMCtW7ceeR8hhHWSWFJwKGQKYpG3JJ48GklShE1wdXXlyJEjREVFAbBmzRpKlChh5lIJIayNxJKCQ1pSRF6TePJoJEkRNqNz58789ddfACxYsIDAwMDkbaGhoTz99NPUrFmTxo0bc+jQIQBCQkJ47LHHqFatGsOGDXtgEOXcuXNp2LAhtWvX5rnnniMhISF/n5AQwiwklhQMSiHtKCLPSTzJOUlShM3o27cvCxcuJDo6mkOHDtGoUaPkbR988AF16tTh0KFDfPbZZwwaNAiAjz76iObNm3P06FG6devGxYsXATh+/DiLFi1i27ZtHDhwAKPRyLx588zyvIQQ+UtiScGgUDK7l8hzEk9yTqYgFrnqo5VHOXb1Tq6es2pxDz7oUi3T/WrWrElQUBALFiygc+fOD2zbunUrS5cuBaBt27aEhIRw584dNm/ezLJlywB44okn8Pb2BmDdunXs3buXBg0aABAVFYWfn19uPi0hRAYklog8Jy0pBYbEE+skSYqwKU899RSvvfYaGzduJCQkJMfn0TSNwYMH8/nnn+di6YQQ1kJiie1TIFmKyBcST3JGkhSRq7JSq5CXhg4dipeXFzVq1GDjxo3Jj7do0YJ58+bx3nvvsXHjRgoXLoyHhwctW7Zk/vz5vPvuu6xatYrbt28D0K5dO7p27cqYMWPw8/MjNDSUiIgISpcubaZnJkTBIrFE5DWlZMX5gkLiiXWSJEXYFH9/f0aPHv3Q4x9++CFDhw6lZs2auLi4MHv2bEDvDxoYGEi1atVo2rQppUqVAqBq1aqMGzeOxx57jMTEROzt7Zk4caLNBgIhxIMkltg+haw4L/KHxJOcUXnxBa1fv762Z8+eXD+vsEzHjx+nSpUq5i6GSCGt90QptVfTtPpmKlKOSTwpOCSWWCZbjScvL9zPgUthbHq9jZlLJfKCxBPLk91YIrN7CSGEEKLA0VtSzF0KIUR6JEkRQgghRIGjj0mRLEUISyVJihBCCCEKHGlJEcKySZIihBBCiIJHSZIihCWTJEUIIYQQBY7SV0oRQlgoSVKEEEIIUeAoJVMQC2HJJEkRVi8kJITatWtTu3ZtihYtSokSJZJ/j42NzZVrbNmyhWrVqlG7dm2ioqJy5ZwAn3322QO/N23aNNfOLYTIHoklBYtCFpwXeUfiyaOTJEVYvUKFCnHgwAEOHDjAyJEjGTNmTPLvDg4OxMfHP/I15s2bx1tvvcWBAwdwdnbOhVLrUgeC7du359q5hRDZI7GkYFEyJkXkIYknj06SFGGThgwZwsiRI2nUqBFvvPEGH374Id98803y9urVqxMUFATA3LlzadiwIbVr1+a5554jISHhgXNNnz6dxYsX895779G/f382btzIk08+mbx91KhRzJo1C4CAgAA++OAD6tatS40aNThx4gQAkZGRPPPMM9SoUYOaNWuydOlSxo4dS1RUFLVr16Z///4AuLm5AXoXhNdff53q1atTo0YNFi1aBMDGjRtp3bo1PXv2pHLlyvTv31+6KwiRhySW2C6FTEEs8pfEk+yRJEXYrMuXL7N9+3bGjx+f7j7Hjx9n0aJFbNu2jQMHDmA0Gpk3b94D+wwbNoynnnqKr7/++qFtaSlcuDD79u3j+eefTw4+n3zyCZ6enhw+fJhDhw7Rtm1bvvjiC5ydnTlw4MBD5122bBkHDhzg4MGDrF27ltdff51r164BsH//fr7//nuOHTvGuXPn2LZtW3ZfGiFENkgssU3SkiLMQeJJ1tk98hmESGnVWLh+OHfPWbQGdPoi24f16tULo9GY4T7r1q1j7969NGjQAICoqCj8/PxyVMwk3bt3B6BevXosW7YMgLVr17Jw4cLkfby9vTM8x9atWwkMDMRoNFKkSBFatWrF7t278fDwoGHDhvj7+wNQu3ZtgoKCaN68+SOVWQiLI7FEYkkeU8qyx6TExCew81wo64/foExhV4Y0K2PuIlkviSdWGU8kSRE2y9XVNfn/dnZ2JCYmJv8eHR0N6E2XgwcP5vPPP8/yedM7VxJHR0cAjEZjrvQ5TS3p/Hl5DSHEfRJLbJWy2JaUqNgEHv9+MxdD7wFgZ1A8Xr0oxTxzb9yBMA+JJ1knSYrIXTmoVcgPAQEB/PnnnwDs27eP8+fPA9CuXTu6du3KmDFj8PPzIzQ0lIiICEqXLp3uuUqXLs2xY8eIiYkhKiqKdevWZVpb0KFDByZOnMj3338PwO3bt/H29sbe3p64uDjs7e0f2L9FixZMmTKFwYMHExoayubNm/n666+T+5EKYfMklqRJYknuUQostS3l36PXuRh6j8+61aBhGR8e/34zM7cF8XbnKuYumnWSeJImS48nMiZFFAg9evQgNDSUatWqMWHCBCpWrAhA1apVGTduHI899hg1a9akQ4cOyf0r01OyZEl69+5N9erV6d27N3Xq1Mn0+u+++y63b9+mevXq1KpViw0bNgAwYsQIatasmTw4LUm3bt2oWbMmtWrVom3btnz11VcULVo0h89eCJFbJJbYDoXljklZuu8yJbyc6dugJOX93Ohcoxjzd13kTnScuYsmcpHEk4ypvJjNo379+tqePXty/bzCMh0/fpwqVaR2x5Kk9Z4opfZqmlbfTEXKMYknBYfEEstkq/Hk3RWH+fvwdfa918HMpXrQtfAomn6xnpfalOfVxyoBcPhyOF0mbOXtzpUZ0bKcmUtoHSSeWJ7sxhJpSRFCCCFEgaNQFjnt8vL9V9A06F7XP/mxGv6eNClbiF+2BhGXkJjB0ULYDklShBBCCCEsgKZpLN17mfqlvQko7PrAtgGNS3P9TjSHLoebqXRC5C9JUoQQQghR4FjiFMSHLodzNvguPer5P7StQRl9etj9F2/nd7GEMAtJUoQQQghR4FjiwPmtZ24B8Hi1hwcj+7k74e/tzD5JUkQBIUmKEEIIIQocpSxvTMqBS2GUKeyKj6tDmtvrlvJm34Ww/C2UEGYiSYoQQgghCiRLSlE0TePApTDqlPRKd5+6pby4fieaq2FR+VcwIcxEkhRhEz799FOqVatGzZo1qV27Nrt27cqza7Vu3RqZElcI2yXxpGBQCovKUq6GRxMcEUPtUl7p7lO3tD4uRbp8WQ+JJzknK84Lq7djxw7+/PNP9u3bh6OjI7du3SI2NtbcxRJCWCGJJwWHQllSjpI8IL52Bi0pVYp54GRvYN+FMJ6sWTyfSiZySuLJo5GWFGEWwcGwe7f+76O6du0ahQsXxtHREYDChQtTvHhxPv74Yxo0aED16tUZMWJEct/j1q1bM2bMGOrXr0+VKlXYvXs33bt3p0KFCrz77rsABAUFUblyZfr370+VKlXo2bMn9+7de+jaq1evpkmTJtStW5devXoRGRkJwNixY6latSo1a9bktddee/QnKYRIU27GEpB4UpAohUWNSTlwMQxHOwOVi3qku4+90UDNEl7SkpJHJJ5YFklSRL5bsEAjoHwCnXrdJaB8AgsWPNoficcee4xLly5RsWJFXnjhBTZt2gTAqFGj2L17N0eOHCEqKoo///wz+RgHBwf27NnDyJEj6dq1KxMnTuTIkSPMmjWLkJAQAE6ePMkLL7zA8ePH8fDwYNKkSQ9c99atW4wbN461a9eyb98+6tevz/jx4wkJCWH58uUcPXqUQ4cOJQcWIUTuyu1YAhJPChIL6+3FgUthVC/hiYNdxrdmdUp7cfRqONFxCflUsoJB4onlkSRF5KvgYBg2MhHPHttw67sRzx7bGDYy8ZFqLdzc3Ni7dy9Tp07F19eXPn36MGvWLDZs2ECjRo2oUaMG69ev5+jRo8nHPPXUUwDUqFGDatWqUaxYMRwdHSlbtiyXLl0CoGTJkjRr1gyAAQMGsHXr1geuu3PnTo4dO0azZs2oXbs2s2fP5sKFC3h6euLk5MSzzz7LsmXLcHFxyfmTE0KkKS9iCUg8KUj0lhRzl0IXl5DI4SvhGQ6aT1K3lDdxCRpHr8qijrlF4ollkjEpIl8FBYGzdzQOfhEAOPhF4OwVTVCQK76+OT+v0WikdevWtG7dmho1ajBlyhQOHTrEnj17KFmyJB9++CHR0dHJ+yc1vRoMhuT/J/0eHx8P6NNTppT6d03T6NChAwsWLHioPP/99x/r1q1jyZIlTJgwgfXr1+f8yQkhHpJXsQQknhQUSik0C2lLOXEtgpj4xAwHzSepY9rnwKVw6pX2yduCFRASTyyTtKSIfBUQAFG3nYi96Q5A7E13osKcCAjI+TlPnjzJ6dOnk38/cOAAlSpVAvT+n5GRkSxZsiTb57148SI7duwAYP78+TRv3vyB7Y0bN2bbtm2cOXMGgLt373Lq1CkiIyMJDw+nc+fOfPfddxw8eDCnT00IkY68iCUg8aQgsaTFHA9cynzQfBI/dyd8XB04czMyj0tVcEg8sUzSkiLyla8vTJ9sYNjIZjh7RRMV5sT0yYZHqqmIjIzkpZdeIiwsDDs7O8qXL8/UqVPx8vKievXqFC1alAYNGmT7vJUqVWLixIkMHTqUqlWr8vzzz6d6Lr7MmjWLwMBAYmJiABg3bhzu7u507dqV6OhoNE1j/PjxOX9yQog05UUsAYknBYqynDEpBy6FU9jNkRJezlnav5yvK2clSck1Ek8sk8qLmS3q16+v2dI8zSJjx48fp0qVKtk6JjhYb14NCOCRg0BeCAoK4sknn+TIkSPmLkqOpPWeKKX2appW30xFyjGJJwWHLcYSkHhiSVLGky9WneCXrec59WknM5cKOv+wBV93R2YPbZil/ccuPcSaYzfY+16HPC6Z9ZJ4YnmyG0ukJUWYha+v5QYAIYT1kFgickopLGJMSnxCImeCI2lRoXCWjynv58bC3Ze4fTcWb1eHPCxdwSLxxLLImBQh0hAQEGCVtRRCCMsj8cQyWcqYlKCQe8TGJ1KxiHuWjynn6wbA2WDp8lXQFKR4IkmKEEIIIQocZSFjUk7d0GeUqlRUkhQhUpIkReQKS1q1t6CT90JYM/n8WhZbfj8UyiKe34nrERiU3oUrq0p4O+NoZ5AZvjJhCe+v0OXkvZAkRTwyJycnQkJCJBhYAE3TCAkJwcnJydxFESLbJJZYFluPJxbTknI9goDCrjjZG7N8jNGgKFPYlbPBd/OwZNZN4onlyGkskYHz4pH5+/tz+fJlgh91aVaRK5ycnPD39zd3MYTINokllseW44mljEk5eSOCytno6pWknJ8bR67IqvPpkXhiWXISSyRJEY/M3t6eMmXKmLsYQggrJ7FE5KtUq3SbQ3RcAkEhd+lau3i2jy3v68aqw9eIjkvIVitMQSHxxPpZfHev4GDYvVv/VwghHoXEEyFEkqQUJafdgXIjnpy+EYmmQaVszOyVpJyfG4kaBIVIly9hmyw6SVmwQCOgfAKdet0loHwCCxZYQLusEMIqSTwRQqSU1JCSkxwlt+LJSdPMXhVz0t3L1xWAszclSRG2yWK7ewUHw7CRiXj22IaDXwQON90ZNrIZ7dsbZaEdIUS2SDwRQqSmTG0p2U0vcjOenLx+Bwc7AwGFXLNZCihb2A2lkBm+hM2y2JaUoCBw9o7GwU+vZXDwi8DZK5qgILMWSwhhhSSeCCFSu9+Skr00JSgInH3u5Uo8OXkjkgp+bhgN2R8f4+xgpISXs6yVImxWviQpOem3GRAAUbediL2pN4HG3nQnKsyJgIA8KaIQwkpIPBFC5IZ79/R/szumZOutc7j23EJirD5YPfamW47jycnrd7K1iGNqZX3dOHdLkhRhm/I8Sclpv01fX5g+2UD40mZELmhN+NJmTJ9skK4ZQhRgEk+EELlhwQKNb8fr8aN8leyNKTkdehtnByOJd1wAiNpfPkfxJOxeLDfuxFAxB4Pmk5TyceZSaFSOjxfCkuVpkpKy36Zb34149tjGsJGJWa61CAxUBJ0xsmqJK0FnjAQGmn+6QCGEeUg8EULkhqRY4ljtAgCe3bZnK5ZcCo2iQVkvdo9rjqPRyIixYTmKJ6du6C0gj9KSUsrHhfCoOMKj4nJ8DiEsVZ4mKbnRD9zXFxo0QGo8hSjgJJ4IIXJDUiyxc4sFwMEvMlux5NLte5T0caFYUQN1Snty7ObtHJUjaWavnEw/nKSUj96acyn0Xo7PIYSlytMkRfqBCyFyi8QTIURuSIol8ZGOAMQGZ31MSUR0HGH34pKTg/qlfTh27Q53Y+KzXY5T1yNwd7SjmKdTto9N4u8tSYqwXXmapEg/cCFEboiIjiPBIZrvforjzp8NiFzQSuKJECJHku5Nog8GAHDnj0ZZjiVJ4z9KmpKDegHeJCRqHLwUlu1ynLwRQcWi7iiV866npQqZkpTbkqQI25Pn66QEBiratzcSFORKQIB0sxBCZM+RK+HU+HB18u+Fh4K9wUBlDyfWxrhxc60X9Up706CMN452RjOWVAhhLQIDFbd8jHy7AQ7sNVC+VNYShaRkoKSPMwB1S3mjFOy5cJum5Qtn+fqapnHqRgSdqhfLfuFT8HCyx9PZnovSkiJsUL4s5ujrK8mJECJnCrk58nbnyrg52pOoacTEJ3LzTjRXwqI4cT2C9Sdvomng6mCkRQVfnq5TgraV/XCws9hloIQQFsDTQ09MvHyyPrNXUreqpJYUT2d7Kvq5s+dC9salBEfEEHYvjkpF3LJ1XFpK+bhwUWb4EjYob5KU4JPwS0ewdwZXP3AvCj5lwLcy+FUBJ888uawQwvYUi7vIiLOjwcFVjyXuxaBkBahbCQo3ISJe8d/5UNafuMmaYzf45+h1Crk60LdhSYY0LYOvu6O5n4IQwlLcOgWzngRnLxredeM5I9idjofSNcE74P4Kj+m4FHoPN0c7vFzskx+rF+DNygNXSUjUsrwoY9Kg+YqPMLNXklI+Lhy/dueRzyOEpcmbJMVoBwY7iAqDW6ch8gYkxJo2KvCrCqUaQ/n2ULaVfvMhhBBpUUZAg4ircHU/3A3Wfwewc8K9eF3aBTSjXf3H+OjJVmw5e5sF/11k0sazTNtynj71S/JS2/L4eeR8cKoQwkYoAyTEQfApyoZd5S37CPh9gb7NpRCUbARlWkLFx8Gn7EOHX7odhb+38wPjSOqV8mb+roucuhFBlWIeWSrGyeuPPrNXEn8fZ9Ycu5GtJEkIa5A3SYpPORjy5/3fExMh/BIEn4CrB+DSLji0GPbMAKMjVOgANXtDxY5gJ7WeQogUCpWDZ/6+/3t8DISchZvH4Mo+uLQTtoyHzV9j51KYNtWepk2bPpzv1IqpW86x4L+LLNl7meEtyjCydTlcHPKll6sQwhIVKg/P/gvAit2X+GTpTtYNKorfvTNweTdc3AEn/4Z/xuq9P2r0hBq9wbs0oLeklCn8YMVq/QBvQB+XktUk5dSNCAq7OVDI7dHveUr5uBCbkMiNO9EU93J+5PMJYSny56+1waB/wb1L67UTAPGxpmCwCo4ugxN/grMP1B0EDZ4Fr1L5UjQhhJWxc4QiVfWfGj31x6Juw5l1cOIv2D8Xdk+njG9lPm84nJGNn+LrjZf5cf0Zlu67wkdPVaN91SLmfQ5CCLMzGBQRuBBdpB4UagH1n9E3hJ6H06vh6ApYP07/KdcOreEILt9OoGXFBwfZlvJxobCbI/sv3GZg49JZuvbJG5GPtNJ8SiVTTEMsSYqwJeYbWWrnoHf16vQFjDkGA5ZCQDPY/iP8UAuWjdDHtgghRGacvfWEpddMeO00PDUB7Jzgr/9Rek59JhT7l6VDquLqaGTYnD28OG8foXdjMz+vEMJmGU13QAlaqoHzPmWg0XMwdBW8fAhavwU3jqIW9GGleo1WUesg4f66KEopavl7cvhKeJaum5iocfpGRK4lKUlrtsgMX8LWWMb0N0Y7fXxKn7l6QGj8AhxfCRMb6clK2EVzl1AIYS2cPKDuQBixEZ5dA6Wbw6YvqLesBavq7OKt9qVYfew6j3+/mQ0nbpq7tEIIMzGYxpUkJGYwu5d3aWg9Fl45zLlWPxCHkZZH34WJDfWWFlOCU8PfkzPBkVla1PFKWBT3YhOolAuD5gGKezljUPp4GSFsiWUkKSl5lYTHP4VXDkOz0XDsd/ipPqz5AGIizV06IYS1UApKNoTA+TByG5RthXHjpzx3sBcbO1ynkIs9z8zazWd/HycuIdHcpRVC5LOkQeaJqVtS0mLnwGHvDnSO/ZwrHaeD0QF+GwwzOsCVfdQo4YmmwdGrmc+ylTQTV261pDjYGSjm6SyrzgubY3lJChAcDLuPFSa49sfw0l6o3gO2fa/XXBz7I7nmQgghMhIcDLt3Q7CxOvSdB0P/BY8SlNg4hr89v+DVWvFM3XyOPlN2cD082tzFFULkIztTkhKfkLV7ihOX7qFhILFUd3h+m96tNOwiTGtLkxOf4UFklrp8HbocjtGgqJrFQfZZUdLHWbp7CZtjcUnKggUaAeUT6NTrLgHlE1jwdwno9jMMXa33O188EBYPgshgcxdVCGHBHoolCzR96vNn10CXHzEEH2f06Wf5p85Ozl6/zVMTtrLvYvYWZBNCWK+k7l5ZaUlZsEBj/PR7JEbZU7kKLFhk0LuVjtoNjUbicmgOa53GknBiVabnOnApjMpF3XF2MD7yc0hS0ttFWlKEzbGoJCU4GIaNTMSzxzbc+m7Es8c2ho1MJDgYKNUIRmyCdh/AqX9gUiN93IoQQqSSYSwxGKDeYBi1B6p0ofLxH9lV5EsqGK/Sd8pOlu+/bO7iCyHyQVJ3rwzHpHA/njiWu4bBOe7BeOLkqU8ANHw9sfaejLj8Fix/HmIi0jxXYqLGwcth1CrplavPpZSPCzcjYoiKTcjV8wphThaVpAQFgbN3NA5++pfbwS8CZ69ogoJMOxjtoMWr8NwW8CwJiwbAn2MgTgaLCSHuyzSWALgW1mcD6z0Hp7tXmJvwJv8rvIsxiw7w07rTaNKtVAibZkhKUjL5rifFE4ODngCkGU+K1+H3hvOYEP802qGFMKWlvo5TKudu3SUiOp7auZyk+PvoUw9fCZP7IWE7LCpJCQiAqNtOxN7UB5PF3nQnKsyJgIBUO/pV1rtsNB0Ne36BaW31xd2EEIJsxBKAql1h5DaUf32eC/+OpX4zmbTmEG8tO0y8DKgXwmYZk7p7ZdKSkhRPtER9//TiSdVSvnwT35tjj83X14Kb0QF2TXlgHO3BS2EAuZ+kmNZKkSRF2BKLSlJ8fWH6ZAPhS5sRuaA14UubMX2yAV/fNHa2c4DHPtHXV4m4DlNbw/E/09hRCFHQZCuWAHgUg4EroM271L2zjs3en7Jzz3+Mmr+fmHjpPiGELcpqdy9fX5j2swINYo+VSjee1CjhCcD2uErw/FYo3wFWvQHLhkPsXUAfj+LmaEc5X7dcfS4lTIs4Xr4t41KE7bCoJAUgMFARdMbIqiWuBJ0xEhioMj6gfHt4bhMUKgeL+sOmrwm+qekz+sjYeiEKrGzHEoMRWr2OGrAUX0L51/VDIo+vYeDUPWzZES/xRAgbk7xOSha6dnbvpaGMGkN6OacbTwq7OVLc00mf4cvZG/rOh7bvwuEl8MvjEH6ZPefDKO3hSWhIJvEom4p4OGFnUFyRtVKEDbG4JAX0WosGDUi/1jM1r1LwzD9Qsy9sGMeG0c/Sre+t+zP6CCEKpGzHEoDy7WDEJhx9SjHb4SsqXZ5Pv2k7KVMlRuKJEDYkeZ2ULPTqjDQt0li5vF2G8aSGvycHL4fpvxgM0PJ16P8bhAYR9WMbHK7vY/8m11y/PzEaFMW8nKS7l7ApFpmk5Ii9E8HNJ/P+1vfpWWkZy/t3IqDXqvszcAghRFZ5l+ZW13/552wHPnGYxXvFJ1Fo0FqGvxQr8UQIG2HM4sB5IHkleVcHuwz3a1y2EBdC7hF06+79Byt0ILT7Gm6EO7LI6WOerL70wRnCckkJL2dpSRE2xXaSFCDogmLSmZG8GDeamuo8fxR/jYr+px+cgUMIIbLg/FV3ntk6jZnxjzPC7i8muP5Ika7bOH463txFE0LkgvtjUjJvSomI1r/3bk4ZJykdqhYBYM2xGw88fjaiCk8em84JrRQ/23/P0GLLHp4h7BGV8HLhsiQpwobYVJKSNAPH79fbMzB2LIW4w9+dulLO/bi5iyaEsDIBAXA31JV3rr7IZ3GBdDHuZHbx95i8d7OsRSCEDUia3Ssrk/gldfdyd8w4SfH3dqFqMQ9WH7v+wOMBARDiYaBv1LtsSKzNOPuZvFL5BwJK516XrxLeztyIiCY2XmYlFLbBqpOU4GAeGCCfckaf9fOfp8Pcf/H0BJ/fO8GVveYtrBDCoqUfT5ozfu63DP9nEk2Mx3kt+HXGzFovs34JYeUMpjugzGb3AojMYksK6K0pey/cZu3WmOR4kuAQjXPALUIPVqb/r7/z69FA3m3yJb773s7aoJgs8Pd2RtPgenh0rpxPCHOz2iRlwQKNgPIJdOp194EBaCln9Pl7Vw1cXlwFjh4wuytc2GHmUgshLFFW4slnK/tj7DOHWnYXePnyq7wzd6OsoyKEFUseOJ+VMSmxpiQlk5YUgMSLfiRqMOC9S8nxZNHuSySi8c+E0vz5mycdJ06CRs/Dzknw58u5kqj4J01DHCbTEAvbYJVJSnAwDBuZiGePbbj13fjQALQHZvTxKQtD/wH3IjC3BwRtM2vZhRCWJVvxpMqTGPsvpoLdDYafG81nv22SlemFsFL3u3tl/h1OHpOSSZISHAzvv+qGlqBwanRSjyfPJzB/10Waly9M3YquejzxM0DHz/XZv/bNgT9GQeKjtc6W8E5aK0XGpQjbYJVJSlAQOHtH4+AXAYCDX0TGA9A8isOQv8CzBMzrCUFb86uoQggLl+14Uq4tdgN+o6xdMIHHXmTiyu35VVQhRC4yZKMlJWlMSmbdvfR4EoMy6ud08IvArcpVbkRE069RqQd3VkpfR6X1W3BgHvz+4iO1qBTzdEYpZIYvYTOsMklJGiAfe9MdgNib7kSFOREQkMFB7kVNiUpJmNcbLu7Kj6IKISxcjuJJ2VbYDVxCaeMt2u15joUbD+RDSYUQuSk7LSmR0fEYFDjbGzPcLymexIW6AKBp4FDvFD4uDrSvUiTtg1qPhTbvwMEFj9T1y8HOQBF3WStF2A6rTFJSDpCPXNCa8KXNmD7ZkPmCbW5+MPgPvevXvJ4ymF4IkeN4osq0xNh/EeWN16m+fjBr9p7MnwILIXLF/SmIs9aS4uZoh1IZrxSfFE/CFrXk3t/1iTpcCg9PjWEty+Bgl8EtV6s37nf9WvWGnt3kQAlvWStF2I7MR4BZqMBARfv2RoKCXAkIyMaK0u5FYfBKmNlZH6PyzCrwq5KXRRVCWLicxhNj+TYk9J5LpUX9if1jAPs9V1CnfIk8LasQIndkN0lxd7LP0nnvx5MiBAQUwde3RtYK1OYdiI+G7T+Bozu0/yBrx6VQwsuZ/ZduZ/s4ISyRVbakJHlgQGt2ePrDoN/B6Ai/doPbQXlRPCGEFclpPHGo0pHoLlOopc5wb24/LtyUGwQhrEF2VpyPjI7H1THjrl4p5SieKAUdPoF6z8DW8bDth2wcrPP3duZaWHSWEi8hLJ1VJymPxKcMDFwOcVEw52mIvGnuEgkhrJR7vZ6EtvmKZhzg7NQBhN+NMXeRhBCZMJi6biVmo7tXnlMKnvgWqnWHNe/Dvl+zdXgJb2fiEzVuRshaKcL6FdwkBaBIVei/BCJvwLxeEBNh7hIJIayUb6vhXKw3lrbxW9n+8wjiZLFHISxadrp7RcTE45bF7l6PzGCEblOgXDtY+TKc/CfLh5bwkmmIhe0o2EkKQMkG0GsWXD8MiwdBfKy5SySEsFKlnhzLqTID6RS5grUz3pE1VISwYMmze2Xha3o3Jh73/GhJSWLnAL3nQLGa8NsQuLQ7S4f5m9ZKkcHzwhZIkgJQ8XHo8gOcXQ9/vpLjWTWEEAWcUlQc+CPHfNrT6drPbFk+2dwlEkKkw2C6A8pSd6/ofOrulZKjG/T7TZ/wZ0EfCDmb6SElvPSpj2UaYmELJElJUnfg/QWVNn9t7tIIIayVwUDlkfM46VSTRgff5fD2VeYukRAiDdkaOB8Tj2t+JykAbr4wYKleeTqvF9wLzXB3ZwcjhVwduHz7Xj4VUIi8kydJSmRMvHV2c2j1JtQKhA2fwsFF5i6NEMJKGRycKP7cUm4ai1By9TCunDlk7iIJIVIxZHExx8RETR84n8lq83mmUDkIXADhl2FBIMRlPCje39tZxqQIm5AnScr5W3fp/vN21hy7kaVmVIuhFHT5EQJawB+j4OJOc5dICGGl3L39MAxYSiIGtPl9uBsWbO4iCSFSsDNkbXavu7HxAPk7JiW1Uo2h22S4tBNWjs6wW3oJb2fp7iVsQp4kKcW9nAmOiGH4nD10+mELvx+4QnxCYl5cKvclDVbzLAkL+8saKkKIHCtRtgqXOkzDN+Eml6f0RJOJOYSwGEndveIzS1Ji9Jn6zNaSkqR6d2jzLhxaBFu+TXc3f28XrtyOss4eLUKkkCdJSiFXBza+1prv+tRCQ+PlhQdo++0m5u26QHScFUzL6eID/RZDYhzM7wvRd8xdIiGElarVrCPbq75PpagDHJvxnEzMIYSFUEqhFCRm8p2MjIkDyP+B82lp+RrU6A3rP4Fjv6e5SwkvZ2LiE7kVKZUiwrrl2cB5O6OBbnX8+efllkwdWA9vVwfeWX6Ell9tYNrmc9yNic+rS+eOwuX1FpVbp2DZCEi0kpYgIYTFad17NKt9+lHt2jLO/P29uYsjhDAxKpXpmJSIaP1+xSKSFKXgqZ/AvwEsHwnXjzy0S9I0xDJ4Xli7PJ/dy2BQPFatKCteaMrcZxtR3s+NT/8+TrMv1/P92lOE3bPgTL9sa+j4OZxapQ+mF0KIHFBK0fy5H9hp14DSuz/h5qG15i6SEAL9HiWz2b0iTZWqZu/ulcTeCfrMBSdPWBgId0Me2Fwiaa0UGZcirFy+TUGslKJ5hcLMH96YZS80pUGAD9+vPU3TL9bz6V/HuHEn49kqzKbhCKgzELZ8A0eXm7s0Qggr5eLoQNGhc7lIURyXP0NM8HlzF0mIAs+oVKYD5yMtqSUliXtR6DsPIm7Ab4MhIS55k6w6L2yFWdZJqVvKm2mD6vPvKy3pULUIM7aep8WXG3h7+WEuhlhY86RS8MS3ULIRrHgBbhw1d4mEEFYqoHhRrneaBYnxhMzoBbEWFu+EKGCMBkVm8/okt6RYUpICUKIePPUjBG2B1e8lP+zuZI+ns72sOi+snlkXc6xU1J0f+tZh42tt6FnfnyV7LtP6mw28vHA/J69HmLNoD7Jz1MenOHroM35F3TZ3iYQQVqpZo4asqTyOolFnuDB7mAykF8KMDFkaOG+agthSunulVKsvNHoedv0MBxcmP1zCy1nGpAirZxErzpcq5MJn3Wqw5c02PNu8DGuO3eDx7zczfM4e9l+0kITAvaieqIRfhqXDZSC9ECLHnu79DL95DqL0lb+4tnq8uYsjRIGlt6RkrbuXWVacz4rHPoHSzWHly3DtIKAPnpcxKcLaWUSSkqSIhxPvPFGVbW+25ZX2FdgdFEq3SdvpN20n287cMv+c36UaQacv4cwa2PyVecsihLBadkYD7YZ/xUbVEN8dn3L39BZzF0mIAsloMGS6TkpkTDyOdgbsjRZ1y3Sf0R56zQJnH1g0EKJuU8K06rzZ75uEeAQW+Y3zdnXglfYV2fZmW959ogpnbkbSf/ounp60nX+PXjfvKvb1h0KtfrDxCzi12nzlEEJYtcLuTngETuOS5kv8wkFod66Zu0hCFDhGQ+YrzkfGxFtmV6+U3Hz13h53rsKyEfh7OXEvNoGwe3GZHyuEhbLIJCWJq6Mdw1qUZcubbfisWw1u343luV/38vj3m1m+/7J5VrFPGkhfpDosGyYr0gshcqxuxQB2N/wR+/i73PilHyRY+PpRQtgYo8raFMQWN2g+LSUb6MsmnF5Ny2szAZnhS1g3i05SkjjaGenXqBTr/9eKH/rWRikYs+ggbb7daJ5V7B1coM8c0IDFgyE+Jn+vL4SwGb06P8Zcv/9RNGwfN1a8Y+7iCFGgGAxZm4LYYtZIyUyDYVCzD+WPTaCZ4TBXwmTwvLBeVpGkJLEzGuhau0TyKvaFXB15Z/kRWny1gambzybPwJEvfMpCt5/h2gH45638u64QwqYopej9zBiWGTtS5PBk7h783dxFEqLAMGZhMceImHhcHawkSVEKnvyOxMKV+NF+AqHXgsxdIiFyzKqSlCRJq9gvf6Ep84c1olIRdz77+wTNvljP+DWnuH03n1axr/wENB0Ne2bAod/y55pCCJvj5eJA2QE/cCixLOr359FCZaFHIfKDUWVtdi+LH5OSkoMrhj5zcCaW5gfeeGChRyGsiVUmKUmUUjQtX5i5wxqx4sVmNC7rw4/rTtPsy/WM+/MY18PzYRX7dh9AqSbw5ytw63TeX08IYZNqlynKseY/EZegcWtWP+lGKkQ+MBhUpuuk3I21kjEpKSjfSnzvMppSdw/B+k/MXRwhcsSqk5SUapf0YsrA+qwe05LHqxVl5vYgWn61gbeWHeZCyN28u7DRDnrM0Bd8XDxYVpAWQuRYnw7N+LXIm/jeOUbwsjfMXRwhbF5WW1KsZkxKCueKPs6fDp1g2w9w8h9zF0eIbLOZJCVJxSLufNenNhtfa02v+v4s3XeZNt9sZPSC/Zy4fidvLupZArpNhZtHYZXcWAghckYpxYAhL7DQ2AXfY7O4e2CZuYskhE0zGBSZTRQaEROPm6N9/hQoF5X0ceHd6H5oRWvCipEQdsncRRIiW2wuSUlS0seFT7vVYOsbbRjeoizrjt+g4/dbGDZ7N/vyYhX7Cu2h+auw/1cZnyKEyDEvFwcq9v+GA4nlUH+MkvEpQuQhO4MiITH9LCU2PpHY+ETcHI35WKrcUcrHhbBYI7efmKpPb770WRmfIqyKzSYpSfw8nHircxW2j23Hqx0qsufCbbpP2k7fqTvYcjo4d1djbfMOlGxsGp9yJvfOK4QoUOqWLcrhJt8Rn6ARMnsAxOfTZCBCFDAGgyIhg9uAu6ZZQ61tTAroSQpAkFYUunwPl3bBhk/NWyghssHmk5Qkni72jG5XIXkV+/O37jJwxn90nbiNf47k0ir2RjvoOQOM9rBkCMTlw8B9IYRN6v94S2YWfo3C4UcI+V2mORciLxhVxivOJy1t4OZkfd29ShfSk5RLofegRk+oNwS2fgdn1pq3YEJkUYFJUpIkrWK/+Y02fN69BmH34hg5V1/Fftm+y8Q96ir2nv7w9GS4fhjWvJ87hRZCFDgGg6L/M6NYbOhIocPTiT72t7mLJITNMRoyHjgfEZ3UkmJ93b38vfUk5WKIaUKfjl+AX1VY9hxEXDdjyYTImgKXpCRxtDMS2PD+KvYGpXh18UHafLORX3c+4ir2lTpC4xfgvylw4q/cK7QQokAp7OaIf5/xHEssTfzSkXDnqrmLJIRNMaiMF3OMMt0LONlbX5LiZG+kiIcjF0NNSYq9M/ScCbF3YdkIyGAsjhCWoMAmKUmSVrFf9XILpg+qj6+7I++tOELzLzcwedNZIqJzOMis/YdQrBb8/iKEX87VMgshCo6mlUqws+7XGOKjCZ4zGBIfoQJFCPEAo0Fl2N0rxoqTFNDHpVwITbE0gl9l6PQlnN8EW8ebr2BCZEGBT1KSGAyK9lWLsOz5piwY3pgqxdz5YpVpFfvVJwnN7ir2do56jUVCHCwdLjcWQogcG9SlA794vojvrf8IW/2FuYsjhM0wGjJuSYmO1/92O1tpklLSx0Ufk5JS3UFQrTts+Awu/WeeggmRBZKkpKKUokm5Qvz6bCN+f7EZTcsV5sf1Z2j2xXo+XnmMa+FRWT9ZoXLwxLdwcTts/ibvCi2EsGl2RgPdnnmdv2iO+85viDu/3dxFEsImGFTGLSnRcXqXKGtuSbl+J/rBLuxKwZPf6Wu8LXkWosLMVj4hMiJJSgZqlfRi8sB6rBnTkk7VizJ7h76K/dilhwi6lcVV7Gv1hZp9YNMXcEFuLIQQOVPC2wWHrj9wJbEQ9xY8A1F5sN6TEAWMnUERn2GSktTdyzpvl0r5uKBpcCUsVQWrsxf0mAF3rujLJuTmcgxC5BLr/NblswpF3BlvWsW+T4OSLNt/hbbfbuSlBfs5fi0Lq9g/8S14lda7fcmNhRAihzrUKc/flT7DJSaYm/OflxsLIR6RIZPZvWyhJQW4P3g+pZINoc3bcHQ57J+bzyUTInOSpGRDSR8Xxj1dg61vtmF4y7JsOHGTTj9sYeis3ey9EJr+gY7u+vopkddh5ctyYyGEyLEhvXvwq3N//C6t4s72X8xdHCGsmlEpEjMak5LUkmJnpUlKyrVS0tJ8DAS0gFVvwK3T+VgyITInSUoO+Lk78VanKmx7sy3/61CR/Rdv0+PnHfSZsoPNp9JZxb5EPWj7Hhz7HfbNyf9CCyFsgpO9kZbPjGOHVg2HtW+RePOkuYskhNXKbJ2UpIHzjlba3cvXzREne8P9tVJSMxih+1R9sp8lQyE+Jn8LKEQGrPNbZyE8Xex5qV0Fto1ty3tPVuVCyD0G/fIfT03Yxj9Hrj08GK/paCjbGla9CcGnzFJmIYT1K1/EkxvtfuReoj0hcwbKjYUQOWQwKDLIUYiOTUApcLSzztslpRSlfFzS7u6VxKM4dJ0E1w/Buo/zr3BCZMI6v3UWxsXBjmebl2HTG635onsN7kTHMXLuPjp8t4kle1OsYm8wQLcp4OACS6XGQgiRc11b1GNxibH4Rp7k+vJ3zF0cIaySUZFJS0oijnYGlFL5WKrclWmSAlC5MzQYBjsmwJl1+VMwITIhSUoucrQz0rdhKdb/rzU/BdbB3mjgtd8O0vrrjczeHqT3bXUvCl0nwvXDUmMhhMgxpRT9Bo1kmbEjRY9O4+6x1eYukhBWJ/OB8wlWO2g+SUlTkpJmV/SUHhsHvpVh+UiIDM6fwgmRAUlS8oDRoOhSqzirXm7BL0PqU9TTiQ/+OErzL9czaeMZ7pRuDw2Gm2os1pq7uEIIK+XhZE+Z/t9zOrEE8UtHoEXeNHeRhI3TNI2v/z1h7mLkmqwMnLfWQfNJSvu4cC82geCITHpv2Dvr0xJHh8PvL8okP8LsJEnJQ0op2lYuwpKRTVg4ojFVi3vy1T8nafbFer5XA4kvVBmWPy81FkKIHKtTthj7Go7HKT6Sq7OflRsLkac2ngxm4oaz5i5Grsl04HxcotWukZKknJ8bAGeDs7C+W9Hq8NgncPpf+G9aHpdMiIxl+s1TSr2klPLOj8LYKqUUjcsWYs7Qhqwc1Zzm5Qvzw5bLdL85lPh7YUQvfU5uLETeio81dwkAiSd5pVfnx1nkPZwSwZu5se4ncxdH2KjERI1vVp+krLe9uYsC5E48yTxJsf7uXuV8k5KUyKwd0HAEVHgMVr8LN47mYcmEyFhWqgeKALuVUouVUh2VNY8eswA1/D35eYC+in35Go0YF9cPp/Pr+H3qB5zP6ir2QmSHpukTNVgGiSd5wGBQdBr6AVtUPby3fkz05cPmLpKwQf8evc7lq1f5Q71q7qIkeeR4YjQoEjLq7hWfiKOVJylFPZxwcTBmPUlRSp/ty8kTljwLcVGZHyNETpzfkuHmTJMUTdPeBSoAM4AhwGml1GdKqXK5Ub6CqryfO+N71+bZMZ9x0qMpHa9O4oXxcxg1fx9Hr4abu3jCluybA8dXmrsUgMSTvOTr4YR990mEay6E/TpQbixErkpI1Ph29UkmuM3ENfq6uYsD5E48MagstKRY6fTDSQwGRVlf16x190ri5gvdfobg47D6vbwrnCi47oXCshEZ7pKlb56mTwlx3fQTD3gDS5RSXz1qGQu6koVcqTRiDvauXszxnMqOk1d44setPDPzP/YEZbCKvRDpCA6G3bv1fwk+Bf+MhTKtzF2sZBJP8k7jGpXZUOVjisac5/wCi6ntFlYsKZ7M3XKFeqF/0iJ+B6rd++YuVrJHjSdGg3p4TbMUYmyguxfoXb7O3sxiS0qS8u2h8Yuwexqc/CdvCiYKjAfuTTQNVo6GuxmPyc7KmJSXlVJ7ga+AbUANTdOeB+oBPXKh3MLNF0P3yfhGnWN73fW89lhFDl4Op+fkHfSevIONJ29mPnWgEMCCBRoB5RPo1OsuFSvdI3T6s2DnpK/PYwEknuS9br0G8YdzN8qcm0/wnhXmLo6wYsnxpE8EvyxZz4f2c9DKtIYmo8xdNCB34kmm3b1sYOA86EnKlbAoomITsndg+w+gSA34/QWIsIwWNGF9Ut6bBJRP4L/Js/UeHplUeGTlm+cDdNc07XFN037TNC0OQNO0RODJRy+6APQaiyajcNz/C6NKnGHrm234oEtVLt2+x5CZu3nyp638fTiNVeyFMAkOhmEjE/HssQ23vhv5fMjz+MQcIrz1BPAoZu7iJZF4ksfsjQbqDv2O4wTg+NdLxN6+Yu4iCSuUMp749F7HRO9vuRfjTGiryfrCxJbhkeOJQSkSE9PfHh1vOy0pAOduZbM1xc4Res6A2Hv6+ikZvVhCpCH1vUm9Pgupfnkssf6tM63wyMqYlA80TbuQzrbjOSuySFO796FoDVjxAi7RwTzTrAybXm/DVz1qci82gRfm7aP9d5v4bc+l+6vYC2ESFATO3tE4+EXQwnCIkR4rmHF8MKcMT5i7aMkknuQPf19vbj02CfvEaC7/MlhuLES2pYwnr9ktprohiNFbvudcsMVUeORKPDEayKQlJQFnW0hS/FyBLE5DnJpvJej4OZzbADsn5nLJhK1LGUsciOPnol9yL8GZY5WmZFrhYTHVIQK9xqLHL/qA1xV6jYWDnYHeDUqy9tVWTOhXB0c7I68vOUTrrzcya9v57DfdCpsVEABRt51wu5XIt/aTORFbijfWfEZAgLlLJsyhRdNmrCk1hrIRuzn9++fmLo6wMknxpFnsMUbY/cXMu51Zcfgpm4snRtPA+fS6VOvdvaw/SQko5IpSZH9cSpJ6Q6BKF1j7EVw9kJtFEzYuKZbERzrwut0iqhkuMPLfCZSoXDTTYyVJsTS+FaHTF3BuI+y4v96B0aB4smZx/h7dnJlDGlDM04kPVx6j+ZfrmbjhDHei48xXZmERfH1h+s+Kr2Pn4JEQxaCF85kw0RVfX3OXTJjLYwPfYJt9UwIOfkvwyZ3mLo6wIr6+MOn7K3zn9hMnYgJ4bcY0pk822Fw8MZpqctPrSR0dl4CjDYxJcbI3UtLbJevTEKemFHT5Edz8YOmzEJPD84gCx9cXpvwMbewPMNzub+bEPsbTr3fMUiyx/m+eLao7WK+xWPcxXNn3wCalFG0q+7Hk+aYsGtGY6iU8+frfkzT7fD1f/3uCkMgYMxVaWILAslPpVHY1N2qN469dNQgMlGVICjInBzv8B0/jluZF3OKhxEXdMXeRhLXQNBqFDsdT3eN60xmcOOlmk/HEaLoLSmsa4sREjZj4RJzsrL8lBaBcdqchTs3FR5+EJeQs/PNm7hVM2Lwi5Y8x3uNHLhrL8GliP+q0zlqSK0mKJUqusShiqrGISHO3RmULMXtoQ/58qTktKhZm0sazNPtyPR/+cZSrYbJGQoFz/TCseQ8qdqJ0z+E2V+Mpcqa0vz9nW4ynaPxVjs8Yae7iCCtxc+33VI7YybpSo2ndpb7NxhODQU+8EtPo7hUTr4/lsoXuXqAPnj8XHPloE/CUaQEt/gf758KRpblXOGG7EhMpvPYVPFQ0xt4ziMGBfRfDsnSoJCmWysUHuk+D20Hw12sZ7lq9hCeT+tdjzZhWPFmzOHN3XqDlVxt4Y8lBzuW0aVdYl9i7sGQoOPtA14l6oiuESfP2T7OxyCBq3vqLo//OMHdxhKW7dhDvbZ+ygXo06WPbNeZGU6xMqyUlOk4f82kLUxADlPNzIyY+kSuPWonZeiz4N4CVr+j3KEJk4Maa76gTu4fdFV+leMW6FHZzYP/F21k61ja+ebYqoBm0fB0OLYSDizLdvbyfG9/0qsWmN9owoHFp/jh4lXbjN/HivH0cuSKr2Nu0f8bCrdPQfQq4FjJ3aYQFavrs1xwzVqbUjne4ceGEuYsjLFVMJPfmDyZEc+NCs6/wdnM0d4nylNHUkpLWDF/R8UlJiu20pACcyeng+SRGe+gxXf//0mGQIGNiRTquHqDwzs9YnVCPqk+9ilKKOqW82S8tKTai5RtQqgn89SqEnsvSISW8nPnwqWpsfbMtz7cqx+ZTwTz501YG//If/52XVextzpGlsG8ONB8DZVubuzTCQjk5OuLWbxaapgifO4i4WBm/Jh6W+PcbOEUE8bnTqwS2qWPu4uQ5g6klJa0uUNFxSd29bONWqWIRPUk5cT3tLuTZ4h0AXb6Hy7tho8weKNIQEwFLhhKqvJhf9A18TBUedUp5cf7WXULvxmZ6Ctv45tkyo53e7ctg1LvzxKf/pgYHw+7d+r8Ahd0ceaNjZba91ZbXH6/EkSvh9J6yg16Tt7NBVrG3DbeDYOUYvem9zdvmLo2wcKXKVeFko0+pGHeS3TP/l+G+qeOJKAAOL8FwcB4T47vSqUsvHG1kwHhGkltSMuruZSOvg5eLA/7ezrnXs6J6D6gzALaMh3ObMtxV4kkB9PcbaLfPMyrqeRpWK5/8cN1S3gAcuJR5ly9JUqyBV0l4agJc3Q/rP05zlwULNALKJ9Cp110CyiewYMH9gOvhZM+Lbcqz9c22fNilKlduR/GMaRX7vw5dSzM4CyuQEAdLngU0vendaG/uEgkr0KDzUHYX6kLTa7+yd92SNPfJKJ4IGxV6Dm3lyxygEjtKDufxapmvYWALDBl194qzre5eADVKeHI4N7t/d/oKCleAZcMhMu0MROJJAXRwIRycz9FyI9ilVaFd5SLJm2r6e2I0KPZdCMv0NHZ5WESRm6o+BfWfhe0/cSquFd4NOyTPthIcDMNGJuLZY5u+oudNd4aNbEb79sYHZmRxdjAypFkZ+jUqzYoDV5i88Swvzt9H2cKujGxdjqdrl8DBTvJWq7F+HFzZA71m6U3vQmRRzWE/c/GbgwRsfpU18fWpXTcg2/FE2JD4WFgylOgExaiYF5jSpQaqgEy+YUzu7vXwtqTuXrawTkqS6iU8WXXkOuH34vB0yYWKLQdX6PkLTGtHzOLn2VRuDnNOHmXruZsYFLg62HF1dUU8e1zEwS9S4klBcOsM/PkqlGrKTwndKeF1N7mrIYCLgx2Vi7qzLwuD523nm1cALA4fx5HgqvhsGUHjWpeTayOCgsDZOxoHP72fqYNfBI7u0fz9d9pNqw52BnrXL8maV1sxsV9dnB2MvLHkEK2/3sBMWcXeOpxZB9u+11cBrtZNmtJFtjg6u7OtyCxcuYfjxkGUqRSdbjwxusVg7xzL/v1mLLDIW+s+gqv7eTV6GA1r1OPeFc8CE0vsTC0p8WlkKbY2cB70WmyAI1dzsTWlaA12F/oUx4tr2P3362w5eYtmhQN4sU15Snt44t7qGA5++mB9iSc2Li4aljwDdg5Ed53MptO3qe7jx61bD1Z6lCnsyvXw6ExPJ0mKlQgOhmdecOSFuFG4ON7l10F9GPF8HMHBEBAAUbediL3pDkD4zjKE3XDi1fczblo1GhRP1CzGny81Z9YzDfD3duGjFKvYh0fJjB2WKCToOnGLRxDvUwUe/1ya0kW2BQfDsNdr896dYbR0PMh7w19h2MjEh+LJ3ePFuDK1FXdjE+nWUz5btih89z+wYwL/OHVhI02Z+kr5AhVLktdJSaMlJcbGxqQAVC+uJym52eUrOBg6/NiVfxIa8IbTAqpHXWTFJ5UZVKcSMwY2InJHJTQNEiIduDK1pcQTGxb1x3tw/RDhbSbxxRJnouMTWDHT46F44uJg5F4WKsQlSbESSbWbF328eC/uGZo6HeG9Vp8TFAS+vjB9soHwpc24M7c14dsqUXTgNtwDN+LZY1vyzUd6lFK0ruTH4pFNWPxck+RV7Jt/sZ4v/zlBcITMAmQpFs6P58jHw4i7F0mj72YwZaZzctcct75Ze7+FSIonvzk254+EJrzmtJD2rZc9EE/CfmtGyKqaFO23kxIjNuLZUz5btmbFnMvELxnJ4YhKvBzWg+vrKuDx5O4CFUuSV5xPc0yKbc3uBeDtqg+ez80k5cCJGDweP8gbccO5gTdTfL+iqN91goKgiJ/ip5HliNhUFaNbLMWf3SLxxEZtmfo7zoen8vPRERRr9ziT/riOpoFzq8MPxRMXBzui4iRJsRkpazeXJrZkcWRbXqv9HRXs9Bk1AgMVQWeMfPepK97FH+z65ewVTVBQ1q7TsIxP8ir2LSv5MnnTWZp/uZ4Pfj/y6AtAiUcSHAxnZn9Fq1Jb+EAbxLU2Ibz8v0ScvHL+fouC6X488eDtuGe5mFiEybXeIMFwHtDjyYqlRjyLxMhny0YF34jHd8uzODpGM9phBPeiXIg87Vvg3m9DlhZztJ2WFDANnr+cO0lKQqLG1MP7UfbxhIT4MSp2NEW4zfiGbxBQWn9NAwMVCz4KIO5SYezc9UrPgvL5KihCzl6gxrlR7I+pwPhyzXFpux+nSleT15VO/X472RuzNLRAkhQrkbK1JHJBa0bNmkWEYwW81g6DiBvJ+3TuDNFh97t+xd50JyrMiYCA7F2veglPJvary9pXW9G1dnHm7bpIq6828NpvBx99ISiRIyF7NvF2469YmtCc3xJa4eAXgYtXNPdCHv39FgVLynhyfUFnBiyfTSF1h8S/niXsrt5PuE4diL0jny1bFb96HM1K7OSd+Gc4rxUDBQkRBe/9TpqCOLGAzO4F+t/3i6H3CL/36F265/93kd0XQ+gZUI2wxS3ZOn8Y72/+gK4V/sT33JTk/erWVYSvqYmWoL/esTfdCsTnq0CIj8XxjyEoA4zmBeKww6XcLQwOCcSFugAPxxMXByOxCYnEJ6TRzzIFSVKsSFJryaolrhw97onns7P1xXKWPguJejBNncyEL23G9MmGHM+iUc7Xja961mKzaRX7Pw9dpcN3m3hh3l5ZxT4/Rdyg4qFhnAotz5tXRwOK2JvuxEQ48f343Hu/RcGRMp6s2NCYG00/oGniXv6d9jYJiVquxxJhQU6tptjZ75h5KpAVCc0BSLjjjL2dInxJwXq/jRm0pETZYHcvyL3B8+H34hi/+iSNy/rwzYslk+PJq4tGQ8VOsPpduLwX0O9Npv3gxJ21tQCIPlS2QHy+CoQ17+MWvo/n/pnA2Rum9VAMGnE3PQhb2CLNeOJsSvwz6/IlUxBbGV9fUnypq8IT38DvL8KmL5MX8wsMVLRvbyQoyJWAAHIlCBQ3rWI/qm15Zm47z5wdF/j78HVaVvTlxdblaFS20KNfRKQtMQGWPoshLoJz9VZw7eXKOHtFExXmxPTJBgIDFd275e77LQqGB+LJYy8RFLSNHldn8tuyhvTt2TdPYokws/DLsPw5tCLVma/6oIXGcffvhkTddGfWDAPt26sC9X4bCtBijklSDp5vVr5wjs/z4/rThEXF8f6T1VBKpYgnCp6eBFNawW9DYORmcPYmMFDRtl1x+sw4R3zPM/TsXULfV1ivY3/Arp+h0fN0rfQUK0cm4lrhOi7tD9Czbmne+tKOoCC7h+KJs4MpScmky5dtVQ8URHUGQK1+sOkrOLM2+WFfX2jQIPf/yBR2c+T1xyuzbWxb3uhYiaNXwukzdSc9f97OhhOyin2e2PAZBG2BJ76l85BqybVVQWeMBAbqAT6v3m9RgChF6cFTue1YnDaH32T9niOAfLZsSnysftOYEMfKSp9xOuoeHzxdhVXzvZLjSUF7v5PXSUmru1d8Ag52huRExlYkDZ4/eCksx+c4FxzJ7O1B9G1QkqrFPR7ewcUHes2EiGuwfGTy9GlF/BTvPV2Ry2H3WLr3co6vLyxAyFm9krxEPejwcXLrfLshN/BwsufTESXSjSdZbUmRJMUWPPEt+FWFpcP1WrJ84OFkzwut9VXsP3qqGlfDonhm1m46/7iVPw9dlVXsc8upf2HLN1BnINTpD8hNo8g7yskTj0EL8FL3cFn5HKevhZm7SCI3rXkfLu8m7LHveGdTNE3LFWJoa/8CHU+MxqR1Uh7+mxUTl4iTjS5w3KRsIbaduZXpmID0fPXPSZzsjbzaoVL6O/nXh8c/hVP/wPYfkh9uW9mP2iW9+HHdaWLiZV02qxQXBYsHg8GoLyht5wDAgeAb7LxyjcBGJZNbS9LiYtqW2TTEtvntK2gcXKD3HEiI02vJ4mPz7dLODkYGNw1g4+tt+LpnTWLiExg1fz/tx29i8e5LxMbnLAAKIOwiLBsBRWpA56/NXRpRQDj61yTqsa9orI6w65f/5crgWmEBji7Xu2U0foG3TpQlNiGRz7oVnJXl03N/xfm0u3vZ2qD5JG0q+3EnOp4DOWhNOXw5nH+OXmdYizL4ujtmvHPDEVCtG6z7GM5vAfRlD8Z0qMjV8Gj+PHgtB6UXZvfXa3DjMHSfBl6lADhx/Q6vLNxPzRKejGlfMcPDnRykJaVgKVweuk6Ay7th9Tv5fnkHOwO96pdkzZhW/Ny/Lq6ORt5YeohWX29gxtbz3IuNz/cyWYs0V4uPi4ZFA0FLhN6zwd7ZbOUTBY9X02cIrtCbAXFLmPHLJGkZtSJpxpPgk/D7KPBvyL/Fn2fVkeuMbleBgMKuZiunpTBmMibFVpOU5hUKYzQoNpy8me1jx685iZeLPUObl8l8Z6XgqZ/Ap5y+EvmdqwC0rFCYCn5uzNx+XrqJW7A048ne2XBgLrR8HSp0AOD4tTsMm70HV0c7pgysn+n3xsVexqQUPNWehiaj4L+pcHCRWYpgNCg61SjGylHNmT20ISV9XPjkz2M0+2I9P607LavYp5LuavGrXodrB6DbFChUzqxlFAWTb++fCPWowrDgL5iyfLW5iyOyIM14EhMBiwaAvTNhT07nnT9OUbWYByNaljV3cS1C8jop6SzmaGszeyXxcLKnXmlvNpzI3mqKey+EsuFkMM+1LIeHk33WDnJ0hz5zIfZecm8PpRRDmgVw5Mod9l64nf0nIPJcmvHkyl74+zUo1xat1VgOXQ7jxXn76PTDFu5ExTF1UH2Kejplem4ZOF9Qtf8QSjeDlS/D9SNmK4ZSilYVfVn8XBOWjGxCnVLefLvmFM1kFftkwcGkuVp8xKY5sG8OtHgNKnc2dzFFQWXvhM8zi7Czs6PNwddYtuuUuUskMpB2PEkgZvGL+gDXnjP5aNNtwu7F8k2vWtgb5c8/pFgnJY2eydHxttuSAtCmkh/Hrt3henh0lvbXNI1v/j1FYTdHBjctnb2L+VXWe3tc2pXc26NbnRJ4ONkxc3tQNksu8lpa8eTNV4JJWDAYzc2P+f7v8dgP23hqwjY2nrzJS23Ls+WNttQu6ZWl8yePSZHuXgWM0R56zgRnL1jUH+6Fpt1cl4/qB/jwy5AG/DW6Oa1TrGL//u9HuHz7nnkKZQGCgsDZ+8HV4puW34Hrpv9B2TbJU0oLYTbepXHoPYNKhkvY/fUKa/eHmDWWiPSlFU/eaD4ex7O/Q/sPWBtVkeX7r/Bim/Jpz8ZUQCXlamm3pCTY3PTDKbWprM+WsOlU1rp8bTwZzI5zIYxqUw4XhxysYFG9+/3eHgcWcDfcjlYlS/HP4etcC4/K/vlEnkkdT5z9wvi16zNw9wYvxY/h7X+v4eZkx6fdqrP9rXb877FKeLpksWWN+wukRktLSgHkXgR6/wrhV7g24VnKVoh9uDuRGVQr7smEfnVZZ1rFfsF/F2n99Ub+t/ggZ25GmK1c5hIQAFG376/w7Hkrnlmth6G5FYOev+izZghhZnaVHiOm5ds8ZdjG9gUf0XnILbPHEvGw1PGkyZ3TvFvvc6LLdye01kjGLjtM5aLuvNimvHkLamEMGQ6cT8TRRrt7AVQq4k4xT6csdfmKT0jk81XHCSjkQr9G2WxFSan9RxDQgvjfX+HppntZON6X+ASNt2dcyPk5Ra5LHU9ei11Gq+JbeTP2GfYnlGXaoPosf6EZ/RuVxtM568lJkqQkN7Pxyrb77SvoSjYgouU3FLu3ns+fGflAdyJz14KWNa1iv+n1NgxsUpq/Dl+lw3ebGfnrXg5fLjir2Kdc0TtmUVMm8jNFPMIw9punzzEvhIWIrP46K4I68Y7zHDp0nYFnz60WEUvEfSnjSaGVZZjo8R0RTpVx7PUT76w4wp2oOL7rUxsHG51SN6cK6sB50Ltlt67kx9Yzt4iMyfhmcem+y5y6EckbHSs/2mfIaMettrO4Gl6YxX16U6rLarQ4I2vPX+baDZkN1FKkjCcddobxvMdyZsd3IKxiL9a82pIOVYs80vmT1kmR7l4F2AnXIcw6OZBRnkt5yrAdB78InL2iCQoyd8l0xb2c+aBLNba92ZYXW5dn29lbdJmwlYEzdrHzXEiBmPEjMFARdNrAwfc+pLn/Duy6T4CiNR7Yx9zd9YQIuqB4eeuPnNFKMNH+R8r7ncXZ+57FxBKhCwxUXDh2jy3PDMXLE7xGzmP50TBWHbnOq49VpEoxD4knqSS1pKS5Tkp8ok0nKQB9GpQkMiaeWdvOp7vP3Zh4vl19irqlvOhUvegjX/P8jcIM2TidQsZwJjn8gKNjDEbXGJbvzP5MYyLvBAYqLm8/xKQ2r7IrsTLbyv2Pif3rJreCPEoscbI3oJR09yrQAgLglX++Zmd0Nb6yn0LlsOtEhTkREGDukj2okJsjrz1eie1j2/Jmx8ocvxZB36k76Tl5B+tP3LD5ZMU3aDp+F2dB8zFQo+cD29Kd/UuIfBQQALeCCzHk+vsATLf/Bs9aRyhdWj6PFiUxkcKbR+By9xSGPrO5RFE++P0oDQN8GN6irMSTNNgZM1hxPi7BZhdzTFK7pBftq/gxdfO5dGff/OCPowRHxvDOE1VzZV2dgADYdbYhr4a8TCPDCT6In0vCXQe237j0yOcWuSjiBm5/B3Iz0Y0ZxT7kp4ENcTSN0XrUWKKUwtneKIs5FmS+vvDzJCf6zlpG8F0/Zrh+zdwJN/H1tczaeXcne55vXY6tb7bhk67VuB4ezdBZe+j0wxb+OGijq9if3wz/jIUKj0Pb9x7YlN7sX5b0nomCIanp/9Cingz9dwZl1XUm13ybFcdOA5YZTwqkDePg5N/w+GfEl27Jywv3A/Bt71qEhiiJJ2lIWsyxIHb3SjKmQ0XuRMczY8u5h7Yt33+ZJXsv81Kb8tQr7Z0r10uKJ7N/fZvvD41igNtqPi+xk+3nb3I1LEriiSWIiyZxYX8S7obwmvFNxvVvk5yg5Na9ibO9URZzLOgCAxW7DxclvONCinuH0y2qH4vn3bPo2jQneyMDmwSw8fXWfNOrFnEJiYxesJ+2325kwX8XiYnP+ENtNW6d0RdsLFQeekx7aKB8WrP1WFJ3PVGwBAYqgs4Yefunjtxr+TkdjPvQ1n7I2ElXLDqeFBgHF8KWb6HOQGj0HD+uO82+i2F82r0GJX1cJJ6kw2BIvyUlKi7BZtdJSalacU861yjKjK3nuRJ2f5atE9fv8O7yIzQM8GF0uwq5es2keNJs7MfEBHSif8x4WqiDvDfzksQTc9M0+OMlDFd282rsSEb0eRo/j/trn+RWLHF2MGa6TkoO5pAT1sbXF3zb14QS09AWDcB4aiRePYZi73cXh5vuDBvZjPbtjfj6mrukD7I3GuhZz5/udUqw+th1Jm44y1vLDvP92lMMb1GWfo1K5WwaREtwLxQW9NETk8CF4OT50C5Js2s43HTHwS+C2JvuFtldTxQcvr76D9oIEu6d4Lm9v/DGpRJ49e2EvXeURccTm3ZxJ/zxEpRuDk+MZ8e5UCZsOEPPev48Vas4IPEkPem1pGiaZlrM0fZbUgBe7VCRdcdv0mH8Jp5vVY6Qu7HM3XkBD2d7vu9bG7s8WFdHjydGqDUNfunIzzd/4smg4nj2uIaDX6TEE3PZ8g0cXsy3cT0p0qQvbSr5PbA5t2JJVlpSrPQOT+RIlSe5XPlDevABV+MV38b3TpEBu1psEDAYFB2rF+PxakXZcvoWEzecYdxfx5m44QzPNCvD4CYB2Zqf2+ziY2HxIAi7CIP+AJ8yae6W1CQ+bGQznL2iiQpzYvpkg8W+T6IAUQpj568IPX+OcSG/cKmQLzsSq1lFPLE5oedhYX/w9Ic+v3IzKpHRC/cTUNiVj56qlrybxJO0pTe7V0y8PtNUQUlSyvu5s3pMSz77+zjfrjmF0aDo06Akr7SvgJ975iuIPxJHdwhcgDahDbM8x9HN+WNC8JR4Yg5Hl8P6cay1a8lyp36sfrzSQ7vkVixxcch8TIokKQWMU9uXmfXbaV6qMZeLmh/zrnWxmto0pRQtK/rSsqIvey+EMnHDWcavOcWUTWcZ0KQ0zzYvk/fB9FFpGqwcDUFboNsUKN0kw90DAxXt2xsJCnIlIAAJ1MJyGO1J7D6bs191YLL3d/SI/ZBjwZWsJp7YhHuhMK8XaAnQbzEJTt68PH0XEdFx/PpsQ1wdH/wTL/HkYel194qJ05MURxsfOJ9S6UKuTBlYnyNXwnFztCOgsGv+XdyrFPe6LMR3WWem2Y0nMP4dIm4WkniSny7ugmXPccW9Fi8GD2Xq0Jrp9lbJjVjiJGNSRGq+fgqnHt+x7kJrPjP+Qr3dsVZZm1avtL6K/aqXW9C2ShGmbT5H8y838N6KI1wKteBV7Dd9CQcXQOu3oVbfLB3i6wsNGsgNhbA8hf29OFV/CdHRrsy0/5oiDqF891OcfFbzQ3wMLBoAYReg73woXIHv1pxix7kQPu5ancpF015VXuLJg+5393rw8WjT2Ednh4LRkpJS9RKe+ZugmPjWasBU54+pbTzDt8ELubO0iVXen1ilkLOwoC+xbsXoFvoCnWoH0Kpixi/8o8YSlyyMSZEkpQDq28+BWuN+Jc6zEr/3HUJg68PmLlKOVSnmwU+BdVj3v9Z0r1OChbsv0vqbjby66ACnb1jYKvb75sDGz6FWP2j1hrlLI0Su6DqoNA6DFlPMLpK5vh/zT+imdKcyFbkkMRGWPwcXtkHXSVC6Kf8evc6EDWfoXd+f3vVLmruEVsNgugtKSNWSEm2q4XWyK3hJijl1eXYgn8b358mS/3Bz5rsEZq0uTzyKyGCY2wMNeM3hPWIcfHj3yap5fllnB2lJEekoXMIDl+FLMDh7wryecDvI3EV6JGUKu/JFj5psfqMNg5sEsOrIdTp8t5nnft3DwUthj3z+R54S8eQ/sPIVKNcOnvoRcmGueSEshXe1utj1nUNVw0XGhI5jxMzt3M1kBeuC7JHiiabBv2/rfcc7fAw1e3HmZiT/W3yQWv6efNy1eq6X15bZmbKUhFRNKdFxBWtMiqUo6+vG0dIDWGTXBZdDk2HbD+YuksV7pHgSE6nfA0ZcZ13dCfxxyZm3OlWmsJtjrpczNWd7O2lJERnwKA4DlurdBn7trmfTVq6YpzPvd6nKtrFtGd2uAjvOhtB14jYGTN/F9rO3crQw5CMvgHbpP/htiL6SfO85YLSiQf5CZFXFx1BP/UgLwyH6XfuC4bP+S66NFvc9cjzZ9j3s+hkaPQ9NRxMeFceIX/fgaGfg5wH15KY6m5K7e6V6G5JbUgrAFMSWJrBhKcZG9uFGqSdg7QdwYL65i2SxHimexMfC4oFw/TART03n9R321C/tnW8tsc4OBmlJEZnwqwz9FsGdqzC3O0SHm7tEucLH1YFXO1Rk+1vteKtTZU5cj6DftF10/3k7a49lfRX7R1606PoRvZbCoxj0/w0c3XL+pISwdHUGQLv36WrczuOXv+O5OXtsZ12jXPDI8WTvLFj7IVTvoS/YmKgxav4+LoXeY1L/uhT3cs7D0tumpO5eiYnpdPeSpC/fdapejEJuzryvXoSyreH3UXD8T3MXy+I8UjxJTIBlw+HsenjqR947VoKI6Hg+7VYjeTKJvObiYMe92Ixb3CVJEVCqMfT5FW4egwWBEBeV+TFWws3Rjuda3V/FPjgihmFz9FXsfz9whfjUoyVTeaRFi0LP6YmfvSsM+h3c/DI/Rghr1/xVaPoSg42rqXv+Z56fu08SFZNHiidHluldRst3gKcng8HARyuPseX0LT59ugaNyhbKw5LbruQpiFOPSUmeglhuk/Kbg52Bfg1LsvpUGJc6TIfidWDJM3Buk7mLZlFyHE80Df4cA8dWwGOf8o99e1YcuMqLbcpTqah7Hpf6Pid7Y3K3yvTIt0/oKnSA7lPhwnZ9zv34GHOXKFclrWK/4bXWjO9di4REjZcXHqDd+E3M35X+KvZJixbF3tS/uFletCjsIsx+ChLiYNAK8CoF5MLYFiEsnVLQ4ROoM5CX7ZZT8fR0XpBEBXiEeHJylV7rWaqx3mXUzoEZW8/z684LdKlQljYBMlA+pwzpLOaY1JLiKAPnzaJfo9IYlGLOvmC9F0Kh8nol6sWd5i6axchRPNE0+Oct2DcbWvyPkJrDeWf5YaoV96BvjfL5en/ikoWZ8yRJEfdV7wFP/QRn18HiwXp/RRtjbzTQva4//77SkskD6uHpbM/byw/T8qsNTN9y7qHBvkmLFoUvbUbkgtaEL22W+ZSId67C7C4QcwcGLgdffTGkR+6LLoS1UAq6/ADVezDWfiGlTs/muV/3FvgxKjmKJ2fW6ou/Fq2pd811cOGvQ9cY9+cxYs4VYcG4khJPHkFSS4p097IsRT2d6FitKIt2XyLKzhMGrtC7Tc/tCZf3mrt4FiHb8UTT9O6ipjFtWpt3eXv5YSKi42nvVovylbR8vT9xzsJ3S5IU8aC6A+GJb+HUKlg61CYTFUhaxb4ov7/YjLnPNqJMYVfG/XWc5l+u58d1pwm/d38K1cBARdAZI6uWuBJ0xkhgYAb9NZMSlLshMGAZFK8N5EJfdCGsjcGoL1hapQsf2P9KqTPzeGbm7gI/61e24snZ9XrLtm8lGLgMnDzZdS6EVxYdIPaaF8rtLm59Nkk8eQT3B86nvZijdPcyn0FNSnMnOp5l+y+DexEY9Ae4+MDcbnBln7mLZxGyHE80DTZ8qk+8UX8odPycyZvP8+/RGzzfvBLvjnHN9/uTrKxBJN8+8bAGw6Djl3B8pT4rlY0mKqCvYt+8QmEWjmjCsheaUq+0N+PXnKLpF+v4/O/j3LwTDWRx0aLwKzDrCYi4DgOWgH/95E2P1BddCGtltIcev0ClznxsP4sqF+cycMauByoBCqIsxZPTa2F+X/App9ciO3tz+HI4z87eg5+LMzE7quHgFwlIPHkUhvRaUuKlJcXcGpbxoaa/J1M2ndPHj3qWgMErwckT5jwtLSommcYTTYN1H8Pmr6HOQOj8LRtPBfPVvyfoUqs4zQqVMcv9iXT3EjnXeCR0/gZO/qWvahwXbe4S5bm6pbyZPrgB/7zSgnZVijBtyzmaf7WBd5YfznwV+7CLMKsziRHBHKu/nGDnxg9sznFfdCGsnZ0D9JoNVbrwvt2vNLw2j95TdiRXAIg0nPwHbUEgd10rcuvJleBamDM3Ixg88z88ne2ZEtiIqBvuEk9yidGgiE+VpCSt3yBJivkopRjVpjwXQ+/xx8Gr+oPepWHI3+DiDb8+DRd3mbWMFk/TYM17sHU8N0s9Q3CTHzl+I5LRC/ZTqYg7X/aoQZkyyiz3J9LdSzyahsPhye/g9Gp9Gt2Y7K3gbq2DxCsX9eDHwDpseK01Per689uey7T+ZiNjFh3gVFqr2Aefgl86Eht+mzZzltJyRLWH+nTmqC+6ELbCzgF6zoRq3RhrnEfXsFn0+Hkb54Ijs3S4tcaSHDm8hMQF/dl3tRq1Ji+kdA0vfpwVQf/puzAaFPOGNaJ6WWeJJ7nIaFBprDhv6u5lJ7dJ5tShahEqF3VnwoYz9yc38CqpJyquvnqicnZ9ts5ZYOJJYgL8+Qps/4nJB4ZR9eNxlG8STvcJO3F1tGPaoPq4ONiZ7f5EunuJR1d/qN6v/MJ2fbaqe6FZOswWBomXLuTK591rsOXNNgxtFsC/R6/z2HebGT5nDweSVrG/egBmdiQxLpaWs1dyullMun06s9UXXQhbY7SHHjOgzkBeUEt5MXoaPSdtZe+FjGOKLcSSLNvzC9rSYWy73JDAuLHEdz+IZ+8tfL1nF7FxGnOfbURAYVdA4kluMiqVZncvO4PCzii3SeaklOKlthU4F3yXVUeu3d/gWQKG/gM+ZWF+Hzj2e5bOV2DiSXysPiPg3ll8tetVPvZ5Cre+m/DqsZ3IMCOTejahpI9L8u7miCfSkiJyR60+0Gcu3DgKMzrA7aAMd7e1QeJFPJx454mqbHuzLS+3q8B/50N5euI2vpgwgYRfOqPZO3Ok0b+cSSifaZ/OLPVFF8JWGYz6DIJNRtFXW8U3hh8ZMm0Lfx26lubuthZL0qVpsOEz+HMM4b4dCNzwK7G+encjh0J3UXaJvNus8UNrGEg8yR1GgyL1klnRcQlZuokSea9j9aKU93PjuzWniI1P8Ua5+cGQP6FYbX1G0v+mZXieAhNPosP13i9HlnKp8kd8dfx/yePXlAGiN9QiPszlocPyO564ONhluo8kKSJrKnfWFyS8ewumd4Cr+9Pd1VYHiXu7OjCmQ0W2jW3LzNqneO3W+5yMLcxQ4+eccnQj6raj9BEXIjNKwWPj4LFxtE3YxkLnr3hr/mZ+WHsaLVWXG1uNJQ9IiNNX1N70JdQeQFyP+dy+5U1cqH4ToSUown5vSMta+bfIWkFjUJCYRncvR0lSLILRoHirU2XOBt9lxtbzD2509tbvTSp1gr9fgzXvQ2LaCwQWiHhy5yrM7AwXtsHTk7nTcCT2DY8nb44NduPeFW+LuDeRlhSRu0o3gWdXg50T/NIp3eZVmx4knpiI25ZPaXPiQ1RAcw49toDTUW689vseqo/ZRvShMkQubCl9xIXIiFLQ9CXoMYOqiadY6/EJK9ZtYtSC/Q9MUWzTsQT07rO/doMDc6HVm9B1Ar5F7Xnrm9sY3WJIjHQidGFzpnzmKbEkD+ktKamnIE6Q6YctSLsqRXisahF+XHeay7dTTWTj4AK9f4X6z8K2H+C3QRB796Fz2Hw8ubIPpraB20Ek9F3EzLuN6TN7E67lbhG5vRKRC1oRvqS5xdybZGVMSuZtLUKk5FsJhq/T5+5fPAjavAstXyP4liIoSA8CSYOwho1shrNXNFFhTml+KYKDeeAYixcTActHwok/od4QjJ2/oa/Rnp5NqvDnoWtM2ngGj/aHKOLqwtAmZenWyh+Qmjgh0lWjJ8rTH9+F/fnH9UOGHxlF9xuRfP5EPRLCXQkIsNFYAhB8Ehb0hfDL0G2q3q0WWLbvMrODDlGumCtvNmpEvQ+crOc5Wak0B87HJ8jMXhbmg6eq0f7bTXy08hjTBtV/cKPRTl/jrVB5WP0O/NIRAheAp/8D8cFm48mRZbDiBXD15UCHxbz5VwInbxyjZUVfPulaDZdXXS3uOUmSIvKGm58+V/nK0bBhHJd276fhlxOJc3Ug6rb+pQ8MVLRvbyQoyDXNL8WCBRrDRibi7B39wDEWK/gULBqAFnKaS1W/wLnRSHyNenntjAaerlOCp2oVZ+3xG0zceJbP1x5hxq7TDG9Rln6NSuHqKF81IdJUqjFq+HocF/RldsKXjA/ty9M/3yNmdyUiD5di+mQDQWdsKJYAHF0Bv79IotGZE43+wrd4Qwolavy4/jTfrz1Nk7KFmDywHp7O9uYuaYFgSGPgfFSstKRYmhJezrzSvgKfrzrBzG3neaZZmQd3UAqavKAnKkuGwpSWrPf5hS7/a/FAfDh32sDqvTEcuRPE7+F3mPx1NLHxiVQp5oG67cHir0vg6KCsI54kxMPaD2DHBMI8GzLO4y2WLA3D39uZyQPq8Xi1IijTgqWWkpwkyUp3L5W6D3BuqF+/vrZnz55cP68wvwdqGAprRK6bjOOmd7mY6MdLiaM4cKMG4UubEXTGmO4XIjgYAson4NljGw5+EcTedM/0GLM6shT+GE10ghPdFs5g990GGQYvTdPYfjaEiRvOsP1sCF4u9gxpGsDgJgF4uzqY4QnolFJ7NU2rn/melkXiie16IJ54RBL920s4nVnGv3H1eSNhBKH3vAmd25qgkw5pxgariyXxsbDuI9gxgVtODWjy0y/cti9EdKQdrf53mKPhN+hetwRfdK+Jg4VPfWtL8aTJ5+toXr4wX/eqlfxY4NSdxCUksuT5pvldRJGBxESN5+ftZfWxG0wdWJ8OVYskb3sgnqgzxM/rjwo5xdd3+jHVqSPRNz2IPeVPxc4XuHonCkc7AzVKeFLMyxk7g+LQpTucuRkBSs93YoPdCF/S3HLjyZ2rsHQ4XNjKtHP9+LLQk8TZKdoXL8fEUeWtoiWw4rurOP1p53RjiVTviixLq8by7NmRrF1eiUU9nmE57/NlsUB+9KpLUJBbul/q9AevuVpWIIiJhFVvwoG5xBVpQI1xv3D38SDc/DbicNOdYSOb0b79w8FLKUWz8oVpVr4w+y/eZtLGs3y/9jRTN5+jf6NSDGtRliIeTuZ5TkJYiIfjiStnz88gdFNtvmj5EavsxvKKepEdfezZcKAuvTt4PnQOq4klACFn9drdaweIqjGc8sM+waXbbtz8DuMSb+DIbY1X21Zh9ONlkms+Rf5Ia0xKdHwCbtICbnEMBsX3ferQd+oORi/Yz/TB9WlWvnAa8aQcF++spdTpUYytNJeWCfsY4/cCN/wi8XTw4fU+FelQtegD7/Hu3dB5cAiuT+0EwME3EpeSoQQF+VpePDm5Cla8QGJcNC+dfY+/SlQBID7EmYVTy/PFACNOllbmNGS26rx8A0WWpJy6z8EvAoeb7jw7ohloEB3fnHZXJvB9sW953/5Xmra5TNlCEwD/NM+VNHjN4aZ7cu2nxQ1eu7gTVjwPoeehxWscdBvLbftY3PwOA1m/GapTyptpg+pz8noEP288wy/bgpi9/QI96vkzslVZShdyzacnJITlyDiePM+xSm787PclCxw+ZYrHU3y4IZwbhmqMbF0O+xTrVlhFLElMhL2/wOr39bVi+szjSOST2HlGJidXBrtEotbXomkvfyQ/yX/pLeZYyNXya6ILImcHI9MG16fv1J30n76LvnUD+OnFCnj22PlAPDF6aNhVfIN9AX585DCbfxzG8trOT/hibmd8/R7+ogUEwL0rXtib4omWqHBuv59Qu3pAoXx/nmmKvqOvIL93Fgl+NXj+7mhWl7h/H2Ff6J7lVtSkIbMuX5KkiCxJq8bSwTUWZZ+Ia92THJ/3OE94tmBYwCK+6/ghDoua6NOM1hkIhge7LWR1YL1ZxETq6xXsnASeJWHwH1CmJaWDIeq2Icc3Q5WKuvN93zq82qESUzaf5bc9l1m0+yJdahXn+dblqFzUI0+flhCWJLN4smb2QOoX7shX9T7jhdpz6eawnxfWDuWvw3X5qmdNavp7ARYeS0Cv5Fg5Gs5vhrKtoetE8PTH80IMdvVPJu8We8uVyJPFLCu5KkCMSmb3sjZ+7k78+VJzvlx1gtk7gij8zCWUvWltIb8IfPpsw65wBJoGc872ZO3Fp5hc431mNB0DG9ZD52/AvegD50wdT2ISEqn87F5GL/2PBYUaU7eUtzme6n1n1sHKl+HOFWIavEjguQ4cuh3FnXXVcaoRZLkVNRnIbPC8JCkiS9KqsYy964BS4FwokuLDNxF1vhAzNw7j47mPUWjLKP2P84F58MR4KFr9gfNlNrA+32kaHF8J/7wFdy7rUxl2+Agc9akKc+tmqFQhFz7tVoPR7SowY+t55u68wO8HrtK+ShFebFOOOuYOgkLkg6zGk5fW/UiPd7tRbPNolsd+wIo77Rk8sRedG1bj9ccr4eXiYHmxBCAuGrb/CFu+BYMdPPk91BuCBvx96Brv/34E1wpxhO+oAEHFibrtbFnJVQFjMKg01kmR2b0snYuDHR91rU6zUsUZ+N4VHCtfxmCXiJagSIyxI3JHRRwqXMGh6B0uRRWi/aJ/uPbbJNz++wzObYI270CDYfrMYCap44nBuQndJm1n+Ow9LH+hGaUKPbwIYp67cxX+fRuOLodCFbgT+Cd9VyVy5mYkUwbW42YNP4aNLGGZFTWZyKwlRQbOiyxL7vOZ4osAPPRYYKDSuzgcXKA3S0aFQd1B0PotcC+S8UXM4dpBWP0enN8EftXgyfFQqnGau+b21IRh92KZtT2ImduCCI+Ko2m5QrzYpjxNyxXK9X7ptjTQVVi/bMWTmAjY+AXazp+JMrgyPqYrf9h34vn2VenfqLTlDDLXNP1GYt1HcDsIqnWDxz8Dj+IE3brLB38cZdOpYKqX8OCbXrUoZPSwuGlBs8qW4snj320moLALUwbefzp1Pl7NEzWLMe7pGvldRJED2YonIWf1hR/Prge/qtDhYyjfnvT6Wp4NjqT7pO0UcnNg+fPN8HTJp1n3Yu/C9gn62i+J8dDyNaIbvsiAWQc5dCWcaYPq06qiHjisctpkoNfk7Sx5vlm6sUSSFJEtaX0RMvxy3AuFjV/AnhlgdITGI6Hxi+BqAf07g0/C5q/h8BJw9tIXU2swTO83ns8iY+JZsOsi07ac42ZEDLX8PXmhTXk6VCmCwZA7yYot3VQI25DteHLjKPz7DpzbwE1jUb6Ofoo9Ho8x+rEqdKlZHDujmZIVTYPTa2DTF3Blr17Z8fg4KNeW23djmbjhDHN2XMDBzsCrHSoyqElp85U1l9hSPOn0wxZKeDkzffD9p1PlvX8Y0LgU7zxRNb+LKHIoW/FE0/QFqdd+CLfPQ5mW0GosBDRL89y7zoXQf/ou2lcpws8D6ubt5BZxUbBvDmwZD5HXocpT0OEjEr3K8PKiA6w8eJWJ/eryRM1ieVeGfDJwxi7mDmsss3uJ3OHr+/CNQ1qPJXPxgc5fQaPnYP0n+pdu52SoNwQaDgefMukcmEc0DS7t0secHPsD7J2h2cvQfIyeqJiJm6Mdw1uWZWCT0izbd4XJm87y3K97qeDnxgttypn3BkyIPJLteFKkGgxaAWfW4bvuI76+NpUb0Sv4eUlHpq3pyOA2Nehau0T+ddOJj9VvdHb8pLfIepaCrpOgVl/CohOYvfY007ee425MPD3q+vP645Xwk5n9LI7RwAPdvTRNk8UcrVC24olSUO1pqNQZ9vyid82c1RlKNYWmo6BiRzDcf/8blS3EGx0r8dnfJ5i78wIDmwTk/hO4GwL7ZsPOn+HuTSjdDPr8CiUbAvD9mlOsPHiVNztWtokEBWR2L5FHst20WKgc9JoFrU7owWDXZD1RqPAY1OkPFR4H+zz84303BI4u02snrh8CR09o8arltOqYONkb6deoFL3r+/PnoWv8vPEsYxYdZPyaUzzXshw96/nLH05hU3LUTaF8O1S5tnB6DX6bv+bDy3O4d+83lv3elJGr2lKjUXv6NCyFv3ce9R8PPgWHFsL+uRB5A3zKwVMToFZfzt+OZe7fJ1n430XuxibQvkoRXn+8EpWKuudNWcQjSz1wPjYhEU1DYq0VynY8sXPQe3jUGwx7Z+tjyRb20ysc6g6Cmr3BuzQAw5qXZfvZED756zj1SvtQtXguTHiTmAhBW/Tu8UeWQUIMlG0DLWc90Kqz7vgNflx3ml6mmUFthYtDxmmIdPcS2ZYrKzzfuQp7Zuq1BpE3wMlTr7mo+DiUawvOuTCA/PYFOL0aTv0L5zbofTr9qkHDYVCjNzi6ZXi4JfTxTEzUWHfiJhM3nOHApTB83R0Z1rwM/RuXzvYc/rbUPUPYhlxbLf7KXrT/ppJ4ZDnGhBiCtCKsTajHjaKtqdygPe1qlMTL5REWUk1M0FtKTq+Gk3/r/1cGvR97wxHc8W/J6mPBrNh/ha1nbmFnUDxRs9gDM/dZQjzJTbYUT7pN2oargx1zhzUCIDwqjlofrebdJ6owrIXt3BDaulyJJwnxcPIv+G+anjwAlGyUfH8S4lKOTj9uxcvFnj9GNc9ZIht7Dy5s0+9NTv4Nd66AgzvU7AUNR4BflQd2vxR6jyd+3EJJHxeWPt+UiDCjzcSSt5Yd5oseNWVMisgdub7Cc0K8PmD98G/6FzYqVH/ct4rexFmkGvhW0qcDdi8GDqlqRjUNosMh4rrerzT4BFw/DBd36bN0AXiXgcpPQK2+UDRrgyBz7eYpl2iaxo5z+ir2286E4Olsz+CmATzTNOur2NvSTYWwfnmyWnz0HTi+kqj9i7G/tA07LY4YzZ5DWlluuFfH2b86JSvUIqBsRRw8iz48/iwxAe7egohrEHJGjydX9sHl3RBzB1DgX5/Eqk9zrkhHNl8zsulUMDvOhRAbn4i/tzN96pekT8OS+Lnfbxm2tHiSG2wpnvSavB07g4EFI/QJU27ciabRZ+sY93R1BjQubY5iimzKk3hy+wIcXqx3Db9+SH/M2YdbPrX5Ncib0lXq0r1dS/AoDs4+Dy23QHyMXgkbfgVunYSbx/VYcu2gXmlq76K3mtToARU7PXx/A8TEJ9Dz5x0Ehdzlr5dasG21s03Fkk/+PMb7XarJmBSRO3J9hWejHZRvp/8kJuhf4PNb4NJOEo/+jmHf7FT7O+rdwgz2kBCrDzBLjHtwHw9/KNkASr6kn7dQ+XRn7UhLyoXmjG4xcL4Qz46olebq8vlFKUXTcoVpWq4wBy6FMWnDGX5cd5rpW/7f3r3HSFWecRz/vefM7MzuDsuCO8uyFFgUaGHlJrIqWBOLlyhUGxuroBCDqNS22tiY1kqq9hJrq+0fpheo1qq1q1FjYpqqbeOlVhSxqLVeUJDBG3ITkIWZndvpH2eHnYVlF9jZPWf2fD+JiRF2eGM4T85vnvd9n/e1sGWMrjiNKfYoL/0yLT5aI824RJUzLpHa2+S8/6w+f/MZjUq8qGltj6ti3aNS53gSpU1UeTsiY1kK5dOyckkZJ7//1x1jK1U7XttHnaNE9VStcqbp9Z1hvfHUbu1pdz/o2Hi1Lj1prOZPG6kZo2sPOlBbqCexeS/LCudkZ2wtXdbiaT1BV5bpOswxlXHnbbDdq3z0Sz0ZNlY67Xr3n88/ceeUfPCS6j58SdeGn5G1/lFpfcfvNbZ7xjUUkZy8ew15Ntn180KVUuN0afZ3pLGnSk2n9rrN/bYn1umNj3drxaKZqsxXaemynK/eTfqKYY4oqX6d8GzZ7tW/Y052v3n8eU4jh32spvAGjR+1SSNCO3TxV3dr0oR2N6CEom5BqI67Q5lqx7hdl+jQPi2jUOwyO2La0nqyQkOTyrQ7WrHS0fIbvf/GYvroWq1cfKLe3bJHv3t2g+5ZldB9LzLFHuWl36fFR2Iyk+YrPmm+WlsdLVma0di6dZoyea1G132kqeM/lq09clJJWcorpQrtU0TbnFptdWqVcBq00RmpdDIsbXY/MhbZp+Pi1TpveqNmjBmmlqbhvc5NSCQkK5LR9sdmKjQ0qezuSlVVZ5RIlO+LxWBjW0bpbGc4TWXcf+/tBQr+0e/1pKZROmGRdMIitbY6+uZNu9Wy8BGNC29W9drhuvzrO9Q8MSllUx2BJepu4RrS4P5s3QT3nMuB3ZYe/POtLfrjCxt12ewmnd3coDVr/P1ucjQY5oiSGogJz53fPK7Rxsdmat+CTVpfX6/01uN0+4/62L49DE1N0r4dUaWeOl4NC1/aX/BuvW2OrrrSPy8WE0cM0a8vmq7rzpyo3z+3QQ//x51iP39qo64+nSn28LeBmha/bZt0+ZV5teekz+fu1Jr6Or2wdZxW3DFHiQ22IkMy+mRXUtv3pPXZvrRGZnJK5/Kaa1mKhC3FIiHVD4lqRE1E8SGRI756NBaT9u4Mq2FR5zaUT++fo1jPR+IwgGzrUJ0UblQsFwNdT1KZKr2ZPk7vxkYqNz2sFcuXKbGhdO8Hm3cndf0jr6u5sUY3nPslSeXzbnIk6KSg5Pp7wnOhk2GFcwoNTfa5fXukB1bjcemH37d0y68OaB0P62PruJ+MHu5Osb+2aIr9469/ojMm1evq08frBKbYw6cGYlp8IiFVxNLKOekuz3O4xn2eZ80Kq6YhLDX0/llHc/i9rU0aUt+1lsTiKbW10fH0C8sY5fNs9yp3A11PwsP3SZLsyoyqJn+oRKKpJO8mmVxe17S+qvZsXncumKFIyP17WG7vJoejtyuI+ZoARyUel2bN6p8iUGjb5jO2srsrld7qXt15NO3b1lZHTeNzOufCvWoan1Nr6+FdFHHVVUYV2Wif/uyBVl8T1Q3nTtKqH8zVdWdO1CubduqC367SgpUv6d/vbfd6eUC3+rOWSG49SbdVKLOray3JfD4wtaSpScq2da0lub3+riVBc1AnpWPrF52U8uNFPclnLFWe9K5idanD/pye6sntT63TmsRO3XrBFB0b79pyLcd3k56w3Qtlp7Nt26Kq6oy7NSKeUm7vkbVviw/AV9TvUcXWIVq6bM5hHTKLx6W7VvR/67g/DK0K65q5E3T5qePU+rI7xf7Su1d7vSzAE/G4dPdKS5ddLn3659myq9tlp6O6564BrCUDsA0FR88yRrnOIylKpt1OSuEbbKCgu3pSUZ1V4+IXddNTr+qBK05SuJfByz3Vk9e2bdGKf72vS08eo/Onj+r2zy/Xd5PuzJ00osdfJ6TAlzrbtrZiMamt7cjbt3297WMgWsf9qToS0tIvd06xX3ib1ysCvFF4ll99VZJCmjGDWoJOtqUu273as25I6e1bXgRTd/XkhY+m6LsPvaZfPPmObpw3ucefP1Q9ee71nG5+/jUdP6pGy3v4jMFUT3qb90ZIgW/F4317+Epx20df1+AHkZCtBS1jtNDrhQAeisels846up+llgxuIctSNl98uxdnUtCzA+vJ1+KjtPaDnfrD8xs15Qu1Om9a4yF/trt60p6Wfrl6jWKRkO5aPKvXv3tBqSeEFAxabLMAUArUksHNsoyKGin7ryCOhjiTgsO3fN5kvbN5j6576DVVhW2dMbn7rUwH1pNU0tLUb72iXe0ZPbzsFDUMZeZZAU8gBrUFC4wS62098Ui1Euvtsp7MCsA71JLByzZSjtu90EcVIUt3XXaimhtrdPUDa/Xcu9sO+XsL9eS+e41mfm+1tmfa9JtLTlBzY9/mvA02hBQMev192weAYKCWDE6WZQ4IKYXbvQgpODI10bDuXdKi8fUxLfnTGt3x93VdBoUWe3vnNt2yapU+S7br/iUtOv2L9QO8Wv9juxcAAAgs2xjli64gTmZyCttGtkW3DEeutqpCD151sm55/C3d+fR6/eOtLbrwxNE6bUKdJGnDtr16YPUmPf/edo09pkr3Lmlh+PIhEFIAAEBg2Qd1UnJ0UdAnNdGw7vjGNJ3VPEK/ePId/eSvb3X59aGVYS2fN0mLThnLVdc9IKQAAIDAcg/Od72CmJCCUji7uUFnNzfow8/2adWG7YqEbI2rq9aEETFVVfAK3hv+DwEAgMCyzcFnUpg2j1IaPbxKFw0f4/Uyyg5PIQAACCzbMsoeuN2LLTiA5wgpAAAgsGzLdJk4n+RMCuALhBQAABBYtmWUc7p2UioJKYDnCCkAACCwLGOULxplkcrkFeFMCuA5nkIAABBYtqWDOils9wK8R0gBAACBdeDtXu3ZPCEF8AFCCgAACCyrY7J84fC8e7sXr0eA13gKAQBAYNnGDSmFLV/c7gX4AyEFAAAEVqGTkivqpFRWEFIArxFSAABAYNlFIcVxHHfiPNu9AM/xFAIAgMAKWZ3bvdqz7l3EEbZ7AZ4Leb0AAAAAr1im8+B8e8fAFM6kAN4jpAAAgMAq3u6V3R9S2GgCeI2nEAAABJZVtN0rmc5JkirppACeI6QAAIDAsvdv95JSWTeksN0L8B4hBQAABJbd8SaU67jZS2K7F+AHPIUAACCwig/OpzIdnZQQnRTAa4QUAAAQWIWD89mikMIVxID3CCkAACCwim/3KoQUDs4D3iOkAACAwCqElDxnUgBf4SkEAACBVbjdq7iTwu1egPcIKQAAILCsbrZ7EVIA7xFSAABAYO2fk+I4SmXZ7gX4BU8hAAAIrO4OznMFMeA9QgoAAAgsq+jgfDKTU0XI2v/fAHiHkAIAAAKr8+C81J7JKxri1QjwA55EAAAQWFbHm1A2n1cqk+PQPOAThBQAABBYoY6Uks+LkAL4CCEFAAAElt3xJpTrGObIzV6AP/AkAgCAwLIKVxDnHaWyOVXSSQF8gZACAAACq/gK4mQ6pwghBfAFQgoAAAisQicl1zHMkTMpgD8QUgAAQGAVOikf7NinzbuSXEEM+ETI6wUAAAB4pRBSfva3t1VdYeviltEerwiAREgBAAABdkx1haJhS6eOj+vH5zersbbS6yUBECEFAAAE2DGxiP5389myLSPTcT4FgPcIKQAAINBCNudQAL/hqQQAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK4QUAAAAAL5CSAEAAADgK8ZxnNJ/qDHbJG0q+QcD6IuxjuPEvV7EkaKeAL5EPQFQCoesJf0SUgAAAADgaLHdCwAAAICvEFIAAAAA+AohBQAAAICvEFIAAAAA+AohBQcxxswyxvzXGBM1xlQbY940xhzv9boAlB/qCYBSoZ4EC7d7oVvGmJ9KikqqlPSR4zi3erwkAGWKegKgVKgnwUFIQbeMMRWS1khKSZrtOE7O4yUBKFPUEwClQj0JDrZ74VCOkRSTNETuNxYAcLSoJwBKhXoSEHRS0C1jzOOSHpQ0TtJIx3G+7fGSAJQp6gmAUqGeBEfI6wXAf4wxiyVlHMf5izHGlrTKGPMVx3Ge9nptAMoL9QRAqVBPgoVOCgAAAABf4UwKAAAAAF8hpAAAAADwFUIKAAAAAF8hpAAAAADwFUIKAAAAAF8hpAAAAADwFUIKAAAAAF8hpAAAAADwlf8DE192GNEnrKkAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_64_1.png" } }, "output_type": "display_data" } ], "source": [ "\"\"\"\n", "============================\n", "Underfitting vs. Overfitting\n", "============================\n", "\n", "This example demonstrates the problems of underfitting and overfitting and\n", "how we can use linear regression with polynomial features to approximate\n", "nonlinear functions. The plot shows the function that we want to approximate,\n", "which is a part of the cosine function. In addition, the samples from the\n", "real function and the approximations of different models are displayed. The\n", "models have polynomial features of different degrees. We can see that a\n", "linear function (polynomial with degree 1) is not sufficient to fit the\n", "training samples. This is called **underfitting**. A polynomial of degree 4\n", "approximates the true function almost perfectly. However, for higher degrees\n", "the model will **overfit** the training data, i.e. it learns the noise of the\n", "training data.\n", "We evaluate quantitatively **overfitting** / **underfitting** by using\n", "cross-validation. We calculate the mean squared error (MSE) on the validation\n", "set, the higher, the less likely the model generalizes correctly from the\n", "training data.\n", "\"\"\"\n", "\n", "print(__doc__)\n", "\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from sklearn.pipeline import Pipeline\n", "from sklearn.preprocessing import PolynomialFeatures\n", "from sklearn.linear_model import LinearRegression\n", "from sklearn.model_selection import cross_val_score\n", "\n", "\n", "def true_fun(X):\n", " return np.cos(1.5 * np.pi * X)\n", "\n", "np.random.seed(0)\n", "\n", "n_samples = 30\n", "degrees = [1, 4, 15]\n", "\n", "X = np.sort(np.random.rand(n_samples))\n", "y = true_fun(X) + np.random.randn(n_samples) * 0.1\n", "\n", "plt.figure(figsize=(14, 5))\n", "for i in range(len(degrees)):\n", " ax = plt.subplot(1, len(degrees), i + 1)\n", " plt.setp(ax, xticks=(), yticks=())\n", "\n", " polynomial_features = PolynomialFeatures(degree=degrees[i],\n", " include_bias=False)\n", " linear_regression = LinearRegression()\n", " pipeline = Pipeline([(\"polynomial_features\", polynomial_features),\n", " (\"linear_regression\", linear_regression)])\n", " pipeline.fit(X[:, np.newaxis], y)\n", "\n", " # Evaluate the models using crossvalidation\n", " scores = cross_val_score(pipeline, X[:, np.newaxis], y,\n", " scoring=\"neg_mean_squared_error\", cv=10)\n", "\n", " X_test = np.linspace(0, 1, 100)\n", " plt.plot(X_test, pipeline.predict(X_test[:, np.newaxis]), label=\"Model\")\n", " plt.plot(X_test, true_fun(X_test), label=\"True function\")\n", " plt.scatter(X, y, edgecolor='b', s=20, label=\"Samples\")\n", " plt.xlabel(\"x\")\n", " plt.ylabel(\"y\")\n", " plt.xlim((0, 1))\n", " plt.ylim((-2, 2))\n", " plt.legend(loc=\"best\")\n", " plt.title(\"Degree {}\\nMSE = {:.2e}(+/- {:.2e})\".format(\n", " degrees[i], -scores.mean(), scores.std()))\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 1\n", "Mean squared error on training data: 439230.69504801\n", "Mean squared error on test data: 481979.17861098\n", "Degree of polynomial: 2\n", "Mean squared error on training data: 115822.95008046\n", "Mean squared error on test data: 123711.53703498\n", "Degree of polynomial: 3\n", "Mean squared error on training data: 9011.85263220\n", "Mean squared error on test data: 10913.84780262\n", "Degree of polynomial: 4\n", "Mean squared error on training data: 303.47610036\n", "Mean squared error on test data: 426.30787294" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Degree of polynomial: 5\n", "Mean squared error on training data: 3.80354994\n", "Mean squared error on test data: 5.98822371\n", "Degree of polynomial: 6\n", "Mean squared error on training data: 3.66204648\n", "Mean squared error on test data: 8.14812206\n", "Degree of polynomial: 7\n", "Mean squared error on training data: 0.47075725\n", "Mean squared error on test data: 2.00607783\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 8\n", "Mean squared error on training data: 0.04912436\n", "Mean squared error on test data: 0.21596432\n", "Degree of polynomial: 9\n", "Mean squared error on training data: 0.02522069\n", "Mean squared error on test data: 0.08576932\n", "Degree of polynomial: 10\n", "Mean squared error on training data: 0.02511518\n", "Mean squared error on test data: 1.20015436\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 11\n", "Mean squared error on training data: 0.01640891\n", "Mean squared error on test data: 1.35533774\n", "Degree of polynomial: 12\n", "Mean squared error on training data: 0.00813803\n", "Mean squared error on test data: 0.17446471\n", "Degree of polynomial: 13\n", "Mean squared error on training data: 0.00759119\n", "Mean squared error on test data: 1.08131001\n", "Degree of polynomial: 14\n", "Mean squared error on training data: 0.00472199\n", "Mean squared error on test data: 0.81333802\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 15\n", "Mean squared error on training data: 0.00410478\n", "Mean squared error on test data: 92.09160813\n", "Degree of polynomial: 16\n", "Mean squared error on training data: 0.00315593\n", "Mean squared error on test data: 234.40530431\n", "Degree of polynomial: 17\n", "Mean squared error on training data: 0.00242999\n", "Mean squared error on test data: 1270.94936405\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 18\n", "Mean squared error on training data: 0.00228741\n", "Mean squared error on test data: 108.11945731\n", "Degree of polynomial: 19\n", "Mean squared error on training data: 0.00156372\n", "Mean squared error on test data: 1376.61081005\n", "Degree of polynomial: 20\n", "Mean squared error on training data: 0.00137945\n", "Mean squared error on test data: 1931.97211078\n", "Degree of polynomial: 21\n", "Mean squared error on training data: 0.00118678\n", "Mean squared error on test data: 14496.70992192\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 22\n", "Mean squared error on training data: 0.00092686\n", "Mean squared error on test data: 873.95463048\n", "Degree of polynomial: 23\n", "Mean squared error on training data: 0.00085890\n", "Mean squared error on test data: 5535.20053452\n", "Degree of polynomial: 24\n", "Mean squared error on training data: 0.00084714\n", "Mean squared error on test data: 1289.22422186\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 25\n", "Mean squared error on training data: 0.00079022\n", "Mean squared error on test data: 136582.88824397\n", "Degree of polynomial: 26\n", "Mean squared error on training data: 0.00076923\n", "Mean squared error on test data: 18194.23521766\n", "Degree of polynomial: 27\n", "Mean squared error on training data: 0.00069302\n", "Mean squared error on test data: 2579.13493762\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 28\n", "Mean squared error on training data: 0.00062728\n", "Mean squared error on test data: 3984.82493809\n", "Degree of polynomial: 29\n", "Mean squared error on training data: 0.00060724\n", "Mean squared error on test data: 3204.07047448\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ ":73: RuntimeWarning: divide by zero encountered in log10\n", " plt.plot(polynomial, np.log10(trainingerror), label='Training Error')\n", ":74: RuntimeWarning: divide by zero encountered in log10\n", " plt.plot(polynomial, np.log10(testerror), label='Test Error')\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEHCAYAAACjh0HiAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA/bUlEQVR4nO3dd3xUVdrA8d+ZmfROEiCQECD0Jh1CUyxY0LVg47Wg8trruqvrrq6ru+uua3tdCyo27BXsoIISegsdBEFKQgkdUiB15rx/nAmEkJ6Z3MzM8/185jMz99659xmGzDOnK601QgghApfN6gCEEEJYSxKBEEIEOEkEQggR4CQRCCFEgJNEIIQQAU4SgRBCBDiHlRdXSsUCbwC9AA3cpLVeVN3xCQkJun379k0TnBBC+Inly5cf0FonVrff0kQA/Bf4Xmt9uVIqGAiv6eD27duTmZnZNJEJIYSfUEpl1bTfskSglIoBRgE3AGitS4ASq+IRQohAZWUbQQdgP/C2UmqlUuoNpVRE5YOUUrcopTKVUpn79+9v+iiFEMLPWZkIHEB/4BWtdT/gKPBQ5YO01pO11gO11gMTE6ut4hJCCNFAViaCncBOrfUS9/PPMYlBCCFEE7IsEWit9wA7lFJd3ZvOAn6xKh4hhAhUVvcauhv4wN1jaCtwo8XxCCFEwLE0EWitVwEDrYxBCCECXWCMLN6ZCfOftzoKIYRolgIiEeyY8zbM+ht69SdWhyKEEM1OQCSCz+PvYJGzB86v7jKlAyGEEMcFRCK4d0wPvuj8L3Y5Yyl+/2rI3WV1SEII0WwERCKw2RSPXz2Kp+Meo7SwgML3roSSo1aHJYQQzUJAJAKAsGA7j04cx2NBvyfkwHqKPrsVXC6rwxJCCMsFTCIAaBkVyk033c6zrmsI3fwNJbOftDokIYSwXEAlAoAebaLpf/Vf+dw5iuB5/8G17gurQxJCCEsFXCIAOKtHa/LOfppMVxec026F3ausDkkIISwTkIkA4MZRXfmx1zPsdUZx7N0rIX+P1SEJIYQlAjYRKKV4YNxIXm71dyg8QsE7V0JpodVhCSFEkwvYRAAQZLfx0A1X8mTY74k8sJqCz+4Ara0OSwghmlRAJwKAmPAgJt58Dy+pq4ncNI2i2c9aHZIQwttyd8HBLVZH0WwEfCIASI2PYNC1T/CtK53guf+k7JdvrQ5JCOFNX90Bb50LRXlWR9IsSCJwG5KWQMmFL7HelUrhtHugtMjqkIQQ3lBWAtmL4eh+WPC81dE0C5IIKrhscCeWpN1LVNlB8pa8Z3U4Qghv2L0SyoogJgUWvQxHdlgdkeUkEVRyztgrWetqT9n8F8DltDocIYSnZS0w91e9b+5/+rt1sTQTkggqSU2IZGHr62hRlE3xum+sDkcI4WlZCyGxG7TpC0PvgLWfwq7lVkdlKUkEVeg75nq2u1qR/9PT0p1UCH/icsKOJdAu3Twf8XuISIQfHgnov3VJBFUYnJbId1HjSMhdh2vbPKvDEUJ4yt51UJwHqcPN89BoOOPPkL0QNgZub0FJBFVQSpE8eiL7dTSHZz5tdThCCE/JWmjuU9NPbOs/wVQVzXzU9CgKQJIIqnF+3458bh9LfM5c2LPW6nCEEJ6QtQBiUyEm+cQ2uwPO+Qcc2gqZb1oXm4UkEVQj2GEjaOgtFOhQ8n6S0cZC+DytTYmgvFqoos7nQMfRkPEkHDvU9LFZTBJBDcYN78Vn+iwiNn8Nh7OsDkcI0RgHNsGxg5A67NR9SsGYf0JRLsx9puljs5gkghrERQSzt+dEXBoK575gdThCiMYoHz9QVSIAaN0L+l0LSycH3DxEkghqcfnoIXzlGo5j9Xtw9KDV4QghGiprIUS2hhYdqz/mzEfAHgyzHmuysJoDSQS16NQykhXJ1xPkKqZs8atWhyOEaIjj7QPpphqoOlGtYfi9sOFryFrUdPFZTBJBHZx/5hnMdPbHufg1KDlqdThCiPo6kg15u6puKK5s2F0QlQQ/Pgwul/djawYkEdTBiE4JTI++ipDSXPSKd60ORwhRX8fHD1TTPlBRcASc+Vcz7cT6ad6Nq5mQRFAHSimGnnEBy1xdKJn3AjhLrQ5JCFEfWQsgNBYSu9ft+NPGQ+veMOvxgJiSXhJBHV3cty0fOC4j5OhuWP+F1eEIIeoja6EpDdjq+JVns8GYJyA3G5a84t3YmgFJBHUUGmSn3dBL2eRqS/Gc5wJ6giohfEr+Hji0pW7VQhV1PB26nA9zn4WC/d6JrZmQRFAP16V34E3XRYQc3AC/zbI6HCFEXdSnfaCyc/4Opccg41+ejamZkURQD4lRIdD7CnJ0PGVzn7M6HCFEXWQthKAIaH1a/V+b2AUGTYTlU2Dveo+H1lxIIqinCSO78EbZeTh2LISdmVaHI0TzUJQLX98NuTutjuRUWQuh3RAzuVxDnPFnCI2BGX/y2yphSQT11KNNNNvaXU4eEbjmP291OEI0D3OfhhXvmjWAm5Njh2DfL9CuAdVC5cJbwOiHYfs8v12zwPJEoJSyK6VWKqV85l/4mlE9ebfsbNTGb+HAZqvDEcJah7bBktfA5oBVH0JpodURnbBjCaAb1j5Q0YAbTdfTHx/xy+6klicC4F5gg9VB1Mfori2ZHX0ZpTjQze0XkBBNbdZjJglc8goUHYFfvrI6ohOyFpi5g9oOaNx57A44799weDssnuSR0JoTSxOBUioZGAu8YWUc9WWzKS4Z2ZfvnINxrp0WsKsaCUH2EvjlSxh2D/S+AlqkmYbV5iJrIbQdCEGhjT9X2mjoeoGZpjp/T+PP14xYXSJ4HngQqHZCD6XULUqpTKVU5v79zacv77gByfxkH4mjJBe2/GR1OEI0Pa3NfDyRrWH4PWYytwE3QPYi2NcMCvnFBbB7VeOrhSoa809wlsBPf/fcOZsByxKBUupCYJ/WenlNx2mtJ2utB2qtByYmJjZRdLULD3YQ2+tcjuhInGs+szocIZre+mmwc5mZujk4wmzre42pimkOpYKdS0E7PZsI4tMg/Q5Y9YGZi8hPWFkiGA78Tim1HfgYOFMp9b6F8dTbeae1Y7pzMHrjdJmVVASW0iLTNtCqN/T9nxPbI+Kh++9g9UfWNxpnLQRlh5TBnj3vyD9CREuY8ZDfdCe1LBForf+stU7WWrcHrgZ+1lpfa1U8DTG0YwtmB4/C4SyEX2dYHY4QTWfpa2Zq53P/CTb7yfsG3mjGFaz/0pLQjstaCEmnQUiUZ88bGg1nPWpKHGs/9+y5LWJ1G4FPc9htJPY8g706Duca//gPIUStjh4wDaadz4WOZ5y6P3U4xHeG5W83/lpFefDbT/X/5V1WbAZ8erJaqKK+10BSX5j5qF/UBjSLRKC1ztBaX2h1HA1xQZ8UvnEORf02CwoPWx2OEN6X8aT58hvzj6r3lzca71gCe39p3LW+uhPev8wMVquPXSvAWey9RGCzwXlPQv5uWPBf71yjCTWLRODLhnZsQUbw6dh0KWzwmTFxQjTM/k2Q+Zb5ok/sWv1xff+n8Y3Gm2eaJSPD42H6A7B7Zd1fW75Qfbv0hl+/Nqnp0GucSQRHsr13nSYgiaCRHHYbKT2Hka1b4VwrvYeEn5v5KASFm/l3ahLeAnpcDKs/hpJj9b9OaSFM/yMkdIHb5kNEAnx6vZkyoi6yFkLLHiYObzr7cUCZf5f60Br2rIW8nGbR4CyJwAMu6NOGL53p2LbNg/y9VocjhHdsmwubZsCoP0BkHbpyD7gRinMbtpDT/P8zo3gveAai28CV75ovzS9urX0dYWeZqZbyVrVQRbEpZrH79V+cmO66Jnk5MO9ZeLE/vDoCnusGz3SG98eZ1dDWf2mm7Gji5NDA6fhERekd43k+aBT36C/NKMsht1odkhCe5XLBDw9DTDsYcnvdXpM6DBK6mkbjftfU/VoHt5hE0PsKszgMQPJAM8XD9D+aL9LTH6j+9XvWQElB0yQCMIlg5XtmdtJbMk7tReUshU3fw4r34LeZoF2mQX34fVBWBDlrIGc1bH0BXGXmNSExkNQHWvcxPZ+S+pgG+IbOoFoLSQQe4LDb6NxrEL+uaUfnNZ9hk0Qg/M2aj80X7Lg36z5dQ3mj8Q9/hj3roHWv2l+jNXz3B3CEmqUiKxr0v7BjKcx+ApIHQNqZVZ+j/Jd5Y2YcrY/gcLOAzdSJsPJ9GDDBbN+/CVa+a6rHju53j8C+D/pdawamVVZaBPs3mKRQnhwy3zTJAuCq96H7RV55C5IIPOSC3kl8uWIYf9r1MRzOgrhUq0MSwjNKjpopFdoOMI2j9XHa1Wbg2fIpMPaZ2o9f/wVsnQ3nPw1RrU7epxRc9LypW/98Itw611TNVJa9CFp0hOik+sXaGL3GwdLX4ed/mF/8qz8y1VM2B3Q5D/pdB53OrvkXfVAotOlnbuWcZXBws0kKXmz4ljYCD0lPi2dO8EjzZN1Ua4MRwpMWvgT5OXDuv8yXcX2Et4Cel8CaT2rvb1+UB9//2VSHDJpY9THBEXDVe6a65bMJZrxARS6XeyGaJioNlFMKzn/SjLH49j7TqH3O3+H+DXD1B9D1vIZV69gd0LK7SagRCR4Pu5wkAg8Jstvo3bMPq3QXXH4y2lAI8veY7pHdfwfthjbsHANuhOK82huNM/4NBXvhwudPrWevKKEzXPKymevnh4dP3nfgVyg81HTtAxW16QfXfAY3/QB3LTNtB5Etmz6OBpBE4EEX9Enii7J0bPvWw76NVocjROM4y0xPFmcJnPN4w8/TbqhpNM6sYaRxzhpY8qqZniK5DmsH9LgY0u+CZa/Dmk9PbC8fP2BFIgDofI55v/UtOVlMEoEHDUuLZ17wCFzYYJ2UCoQPKso1VZtTb4ZnOsHqD2HobabOvaGUMl/wuzJN/X5lLpdpIA5rYebwqauzHzNVQN/ce2IEc9ZCiGoDce0bHm8AkkTgQUF2GwN7dmWJ7mmqh5rBQBEhanVoKyyaBO9cBE91hM9vgt9mmUbOK95xD5pqpNOuNj2BqioVrHzPTOA25h8QFlf3c9qD4Iq3ITgSPr3OtDFkLTSlAR/7RW416TXkYRf0TmLaynTSD0+G3Ssav0SeEJ7mLDPrCGyaAb9+b+rVARK7meqWrudD8qCa6+nrKywOel5qqnHG/OPE+gVHD8Ksv5lf9qeNr/95o1rDFVNMEvvwStOobVW1kA+TROBhwzsl8EjQMMrU2zjWTpVEIBqurBhKj9XvV3JtXE544yzIWQW2IGg/3FTbdDkPWnTw3HWqMuAG061y3VTof73ZNutRKM6Hsc82/Fd8++GmmmjmX83z1OGeiDagSNWQhwXZbQzr1ZG5rtPQ66aZPzwh6stZBlPGwmunm66SnrLxW5MEzn4cHtwC138FQ2/3fhIASBkCid1PVA9lLzYDsIbeAa16NO7cw+6GHpdATErNk+GJKkki8IILeicxrTQdVZBTt/lHhKhs0Yum+uZIlmfHpSyaZBpSh90NoTGeO29dlDca715hun5+ez9EJ8Ppf/LMuS9/G+5cIu0DDSCJwAuGd0pgWfAQilWY9B4S9bdvI8z+l5lOILE7LHjBMx0Pdi6HHYvNXEGerP+vjz5XmUbjj6+FfevNIKyQSM+c22Y70fYg6kUSgRcE2W2M6tmOWa4B6F++grISq0MSvsJZBl/dYXrCjH3O/HLftx62/NT4cy9+GUKi6zcBnKeFxULPy8yCLp3HQDefXI/K70gi8JIL+iQxtXQoqvCwmTtFiLpY9KKpNhn7jBmV2vsKiEoypYLGyN1ppjjuf73n1/Ctr2F3QZv+cP5TUo3TTEgi8JLhaQmsCu7PMVuU3yxwLbysYpVQz8vMNkcwDLkNts2B3asafu4lr5n75jAzbquecMvspmmgFnUiicBLgh02zuyZzHTnYPTG7xq2SpMIHJWrhCr+Uh54IwRHwcIXG3bu4gJY/g70+B3EtvNMvMKvSCLworG93dVDpUfNwhRCVKdylVBFoTFmjvv1XzRsbdxVH5iVwtLv8kyswu9IIvCi4Z0S2BDcm1xHvExNLapXVZVQZUNvN6WERZPqd26XExa/AsmDzSpfQlRBEoEXBTtsnNWjDV+XDUFvnmkm9BKiopqqhCqKSYZel8OKd6HwcN3Pv+l7OLwN0u/wTLzCL0ki8LKxfVoztXgoylkMG761OhzR3NRUJVTZsLuh9Cgse7Me53/ZrDPczTtLHAr/IInAy0Z0SmRLSFcOO1rC5h+sDkc0J3WpEqqodS9IO8v0ACotqv343SvN/PxDbvXaoufCP0gi8LJgh40xPZLIKOuB3jbXzL0uRF2rhCobfg8c3WeWfqzNoknm/P2va1yswu9JImgCY/u0JqOkhxlctmeN1eGI5qA+VUIVdTjdrOm78MWaf1Tk7Yb108wAsqaeU0j4HEkETWBEp0RWBfU1T7ZmWBmKaA7qWyVUkVJmLdyDm2vukrz0ddCu5jGATDR7kgiaQLDDRqcOHdlqS5VEEOgaWiVUUY9LTAPwwmqmnSg5CplvmXl8ZMlGUQeSCJpIelo8GSXdcWUvrltDn/A/Lid8/1DDqoQqsjtMd9DsRbBj2an7V38ERUcg/c5GhSsCR42JQCnVog632CaK1acNS0tgvqsXtrJCsz6rCCxFufDhVbDsdbMQS32rhCrrdx2ExsLC/5683eUyA8ja9DcLwQhRB7X1KdvtvtVUfrUDMoFJLbq1jmJTSG+c2o59awZ0GGV1SKKpHNwCH42HQ1vgwv+DgTc1/pwhkTBoIsx7zpw/Ps1s3/wjHPwNxr0pM3uKOqutamiD1rqj1rpDdTfgYFME6utsNkXvtBTWq87orXOsDkc0lW1zzRrBR/fBdV94JgmUG3wr2INOnoxu8csQ3RZ6XOy56wi/V1siSK/DOepyjACGpcUzu7SHWaqv8IjV4QhvW/YmvHcpRLSEm3/2fCkwqhWcdjWs+hAK9kPOGpN4hrgThBB1VFsiGFb+QCl10uThSqnLALTW0vJZR+lpCSxw9kRplxnxKfyTswymPwDf3Q9pZ8L/zoQWHb1zrWH3gLMElk42bQNBEdB/gneuJfxWbYngmQqPK0+f+UhjLqyUSlFKzVZK/aKUWq+Uurcx5/MFaYkR7IzoSbEKlW6k/qrwMHwwznwxp98F4z/27oCuhM7Q9QJzvXWfm2Uow2K9dz3hl2pLBKqax1U9r68y4A9a6x7AUOBOpVSPRp6zWVNKMbhTazLphpZE4H/2b4LXz4LtC+Dil+HcJ5pmkfjh95juos5Ss5qZEPVUWyLQ1Tyu6nm9aK1ztNYr3I/zgQ1A28ac0xcMS0tgdklP1IFNZhoA4R9++wneONt0E53wDfS7tumu3W6oqYLqfcWJ3kNC1ENt3Uc7KqW+xvz6L3+M+7nHFhxVSrUH+gFLqth3C3ALQLt2vt9LNT0tnrddvcyTrXOg73hrAxKNt20efHA5JHaH8R9BXGrTx3DdF01/TeE3aksEFfugPVNpX+XnDaKUisS0P9yntc6rvF9rPRmYDDBw4MBGlUKag5QW4RyN7UJ+cQxRWzMkEfiDX74ERxhM/AFCoqyORoh6qzERaK1P6vCulAoCegG7tNb7Gntx9/mmAh9orac19ny+YmhaIgvW9eTcbXNQWsvAH1+XtQhSBksSED6rtikmXlVK9XQ/jgFWA+8CK5VSjfopq5RSwJuYQWvPNeZcvmZYWgIZpT1Q+TlwYJPV4YjGKDwM+36B1GG1HytEM1VbY/FIrfV69+MbgU1a697AAODBRl57OHAdcKZSapX7dkEjz+kT0tPimX+8nSDD0lhEI2UvATS0k3GVwnfV1kZQUuHxOcBnAFrrPaqR1Rla6/k0vguqT2oVHUpIQgf2FSbRcuscmTPel2UvBFsQJA+0OhIhGqy2EsERpdSFSql+mF/w3wMopRxAmLeD82fl1UN6+1wzElX4pqxF0KYfBMmfg/BdtSWCW4G7gLcxvXr2uLefBXznzcD83bC0eOaU9kQV50POKqvDEQ1RWmgWiE+VaiHh22rrNbQJOK+K7T8AP3grqEAwtGM8f3G5B1JvnS1VC75oZya4SqGdNBQL31ZjIlBKVbMWnqG1vsez4QSOuIhgkpKS2V6QRvutc2DUA1aHJOorexGgoJ0sACN8W22NxbcB64BPqX2BGlFP6WnxzFranYk7fkSVHIPgcKtDEvWRtRBa9oCwOKsjEaJRamsjSMKM6j0X09UzCPhKa/2O1vodbwfn74alxTOvrCfKWeL+dSl8hrMMdi6T9gHhF2pMBFrrg1rrV7XWozHjCGKBX5RS1zVFcP5ucIcWLKcbTuWQ8QS+Zs8aKCmQ8QPCL9RWIgBAKdUfuBe4FpgBLPdmUIEiKjSITm1bscHRHbbJ8pU+pbwEJyOKhR+obYqJvyullgP3A3OAgVrriVrrX5okugAwLC2emYXd0Dlr4Nghq8MRdZW1EGJTIbqN1ZEI0Wi1lQgewVQHnQb8G1ihlFqjlFqrlFrj7eACwbC0BOY5e6LQZr1Z0fxpDdmLpTQg/EZtvYY8tuaAqNqA1Dg22DpTbAsnZGsG9LzE6pBEbQ5shmMHpH1A+I3aBpRlNVUggSos2E7vdvGsOtCLIdJg7BuyF5p7KREIP1FbG8G3tZ2gLseImg1Li+f7wm5weBscltzb7GUtgohEiO9kdSRCeERtVUMjKixPWRUF+PWC801hWFoCD//Uy3wa2+ZA3PVWhyRqkr3QrBMsCwoJP1GfpSqrU1L7IaImfVNi2eFIId8Rb5av7C+JoNnK3QVHsmHI7VZHIoTH1GmpSqVUK6Cte/MurfVebwcWSIIdNga1j2fJnt6cvXUOuFxgq9MQD9HUjo8fkIZi4T9qayPoq5RaDGQAT7lvc5RSi91rFAgPGZaWwPfHupreKPtkmEazlbUQgiOhVW+rIxHCY2r72TkFuFdr3V1rfbb71g24z71PeMiwtHgWOOuxfGVpEWyZDQtfhOJ8r8YmKsh2L1Rvr61WVQjfUdv/5git9ZLKG7XWi5VSEV6KKSD1bBNNQWgr9genkLhtDgy76+QDtDYlhS0/m1vWQigrMvuyFsFV70t1krcdO2Q+g56XWR2JEB5VWyKYoZT6DngX2OHelgJcj3vZSuEZDruNIR1aMH9HLy7dPhfKSqDoiCkdbPnZ/PovcC8Ql9AVBtwIaWfC/g0w81GY+xSc8ZCVb8H/7XD/JpL2AeFnamssvkcpdT6m99DxxmLgZa31dG8HF2jS0xL4/tduXBo8AyYNhUNbzI6wOOg42nzxp42GmOQTL+p8DuzbCBn/htZ9oNsF1gQfCLLcC9W3HWB1JEJ4VK0VnVrrGZgZR4WXDUuL53lXT/IiOxIdlQR9/8d8+SedBjZ71S9SCi78P1MymHYL3PwTJHZt2sADRfYiaNtfFqoXfqfBlcpKqcmeDERA11ZRBEXE8li7t+HG72DUH80XT3VJoFxQqGkjCAqFj8ZD4ZEmiTeglBwzC9XL/ELCD9XWfbRFNbd4QOogPMxmU6R3jGfRloNorev34phkuPJdOJIF024Gl9M7QQaqXZngKpP5hYRfqq1EsB/IxCxEU37LdN9aeje0wJSeFk9ObhFLth0iJ7eQw0dLOFZShstVh8SQOgzO/w9s/hFm/8v7wQaSLPdC9SmyUL3wP7W1EWwFztJaZ1feoZTaUcXxopGGd0oA4OrJi0/ZF2y3EeKwERJkJzTIPG4ZFcpTl/chpYV74fuBEyFnNcx7Blr3lmmtPSV7IbTqCWGxVkcihMfVlgieB+KAUxIBZpSx8LAOCRF8dPNQ9uQVUlTqoqjUSXHZyfdFpS6Ky5wUl7qY/es+nvhuA69e5+7JohRc8Azs2wBf3gEJXaCVzAvYKM4y2LHMNN4L4Ydq6z76cg37XvR8OAJM9VBdvfDTZp6buYnlWYcYkNrCbHSEwJXvweTT4ePxcPNsCG/hpWgDwJ7VUHpUxg8Iv1XXxesvq+J2llJK2gks9r8jO9AyKoQnvttwcgNzdJLpSZS7C6ZOlMbjxshyTzTXThqKhX+qa/fRicAbwDXu2+vAn4AFSqnrvBSbqIPwYAf3n9OFFdlH+H7dnpN3pgyGsc+akck/PW5NgP4gexHEtTfJVQg/VNdE4AC6a63Haa3HYRaj0cAQTEIQFrp8QDJdWkXyn+83Uup0nbxzwATTgLzgv7D2c2sC9GVam0QgpQHhx+qaCFIqrUGwz73tEFDq+bBEfTjsNh46vxvbDx7jwyVVtOuf96QZCPXVXbIUZn0d2ATHDkr7gPBrdU0EGUqpb5VSE5RSE4Cv3dsigCNei07U2eiuLUnvGM9/f9pMXlGl3OwIhnFvgHaabqWi7rLcC9VLiUD4sbomgjuBt4G+7ts7wJ1a66Na69HeCU3Uh1KKv1zQnUNHS3htzpZTD4hJhgE3wKoP4dC2Jo/PZ2WXL1SfZnUkQnhNnRKBNt1R5gM/Az8Bc3W950AQ3tY7OYaL+7bhjXnbyMktPPWAEfeDssPcZlwq0NrM+7/3Fyg5anU0psdQu3RZqF74tTots6SUuhJ4GrNkpQJeVEo9oLVuVOujUuo84L+AHXhDa/1kY84n4I9jujJj7R6e/XETz1xx2sk7o5Ng4E2wdDKMvL/pf+W6XFCwF/J3Q16lW34O5O0yj8sX3OlyHvzPJ00bY0W5OyE3G9LvsC4GIZpAXdfbexgYpLXeB6CUSgRmAQ1OBEopO/AycA6wE1imlPpaay0L9jZCSotwJgxL5Y3525g4ogPdk6JPPmDEfbD8bZj7NFz6atMGN+1mWFfpv4wtyCSo6LbQph90vcA8PrgZMt+CbXOhwyjPxaB13X/dHx8/IA3Fwr/VNRHYypOA20EaMYW122DgN631VgCl1MeYBXAkETTSXaM782nmTv49YyPv3jT45J1RrWHQ/8LiSTDyj5DQqWmC2v+rSQKnjYceF0OU+8s/PL7qJTZLi2DzTPjxEbg5wzPLcO7/Fd4421y7w0hoPxLaj4CIhKqPz14IwVFmziYh/Fhd/7q+V0r9oJS6QSl1A/Ad0NgVytpyYvlLMKWCtpUPUkrdopTKVEpl7t+/v5GXDAwx4UHcfWYn5m7az7zNVfybDb8X7CFmecumsnSyueaYf0LX86FNX4hMrP4LPigUzvyrmUCvcimiIbSGb38PygaxKbDqI/hsAjydBpPSYfqD8MvXpn2iXJZ7ofra1oMQwsfVtbH4AWAy0Md9m6y1bpKBZFrryVrrgVrrgYmJiU1xSb9wXXoqyXFh/Gv6xlOnsI5sCYNvhrWfwf5N3g+mKNd88fa+vPpf31XpfYVZfvOnf5gSQmOs+hCyFsCYf8C1U+GhLJg4C856FCJbwcr34NPr4KmO8MoIkxj2b5DxAyIg1Lm8rbWeqrW+3337wgPX3gWkVHie7N4mPCDEYeeBc7uyISePL1ZW8c86/F5whMGc/3g/mJUfmEnbBt9Sv9fZbOaLOzfblCga6tghU8WUMhT6Xmu22YMgZRCM/ANc/yX8KQtu+gFGPwzhcbDiHXNcR+kdLfxfbSuU5Sul8qq45Sul8hp57WVAZ6VUB6VUMHA1ZqCa8JCL+rShd9sYnv3xV4pKK006F5EAQ26BdVPNlNXe4nKZL/GUIaY6qL46ngGdzjED4SpW29THzEehOA8ufK76qihHMLQbCqc/ABO+gYey4Z6VkDywYdcUwofUmAi01lFa6+gqblFa6+iaXlsbrXUZcBfwA7AB+FRrvb4x5xQns9nMILPduUW8vWD7qQcMuweCI7xbKvhtJhzeBkNubfg5znkcivNh3rP1f23WIlPtk36nWVimrhwh0KJj/a8nhA/yQFeMhtNaT9dad9Fap2mtn7AyFn+VnhbPWd1aMmn2bxw6WnLyzvAWMOQ2WP8F7PVSDl7ymuml0/13DT9Hq55mUZilk+Hw9rq/zllqGohj2sHpMjeiENWxNBGIpvHQ+d04WlLGCz9tPnVn+p0QEg0ZXhjLd2AzbPnJDGKzBzXuXKMfNqOif/p73V+z6CXT4HvB06bkI4SokiSCANC5VRRXDUrh/cVZ7Dh07OSd4S1g6O2w4WvYs9azF146GezBZo6jxopuY5LWuqmwa3ntxx/Ogoz/QLcLoet5jb++EH5MEkGAuPesLtiU4uXZv526c+gdEBLj2VJBUZ7pstnzMtNd1ROG3wvhCfDjo2ZcQHW0hukPmDED5zdBryghfJwkggDROiaU8YNT+Hz5zlNLBWGxZj6djd/C7lWeueDqj6CkwPRM8pTQaDjjIciaD5u+r/64jd/C5h9g9F/MrKtCiBpJIgggt5/RqYZSwe0Q6qFSgctlGonbDoS2Axp/vooG3ADxnUyXUGfZqfuL82HGn6BVb9MQLoSolSSCANI6JpSrqysVhMZA+t2waQbsWtG4C235GQ5t8c4XsT0Izn7MrBy28r1T98/+t5nB9ML/A3tdp9ISIrBJIggwt5+Rhk0pJmVUUSoYciuExUHGvxt3kaWvmWkbelzcuPNUp9uFZpTw7H9BccGJ7TmrYckrptSQMsg71xbCD0kiCDBJMWFcPTiFzzJ3svNw5VJBNAy7Gzb/CDszG3aBg1vM6wfcaEbreoNSZvK6o/tg4Ytmm8tpxgyEtYCz/+ad6wrhpyQRBKDyUsHLs6tY0nLwLWZq6J//Yer662vp62aNgYE3Nj7QmqQMMiWOhS9A/h5YPsV0Kz33X6ZUI4SoM0kEASgpJoyrBqXwWeaOU0sFIVEw6kHYmgGfXGO6gdZVcT6s+gB6XmLWPfC2s/5mRg9PfwBmPW4WsOlzpfevK4SfkUQQoE60FVRRKhhyK5z/NGz6Ad4cA4e21u2kqz82k7sNbsS8QvURnwaDJprBcGWFMPY5WVtYiAaQRBCg2sSGceWg5KpLBUqZ/v/XTTNrCb9+JmydU/MJtTYjidv0b9oZO0c9aOYSGv0wJHRuuusK4UckEQSwO84wy1RWWSoAMwX0LbNND6D3LoUlk6sf0bt1tunSOeTWpv1VHhEP9642azELIRpEEkEAaxN7oq1g15HCqg9q0REmzoTOY2DGA/DNvVBWcupxS16DiEToeal3g66KJ9YzFiKAyV9QgLu9vFRQ1WjjcqHRcPWHZjWvFe/Au7+DggprIR/aZtoTBtxo5vEXQvgUSQQBrm1sGFcOTOHTzB3srq5UAOZX91mPwrg3YfdKeH005Kwx+5a9YRZ4H3hT0wQthPAoSQSCO0aXtxXUUCoo1/tyuOl7M4DrrXPNDKMr3jMLz0QneTlSIYQ3SCIQtI0N44qBKXyyrJZSQbk2/eCWDGjVC768HYpzZYI3IXyYJAIBwB1npAHwSnU9iCqLagU3fAuD/hd6jYOUwV6MTgjhTZIIBADJceH1KxWAaRge+yxc/pYM5BLCh0kiEMfdcUYaGl33UoEQwi9IIhDHJceFc/kAUyrIya1jqUAI4fMkEYiT3Dk6DZeWUoEQgUSWcBInKW8reG9xFqVOzQPndqVFhJfWFRBCNAuSCMQp/nphdyKC7UxZuJ3pa3P4w5gu/M/gdjjsUoAUwh/JX7Y4RXiwg0cu7MGMe0fSq200j361noteWsDSbYesDk0I4QWSCES1OreK4v2JQ5h0TX9yj5Vw5WuLuPfjlezNK7I6NCGEB0kiEDVSSnFB7yRm/eF07j6zEzPW7eHMZzJ4dc4WSsoasJSlEKLZkUQg6iQ82MEfxnRl5u9HkZ4Wz5MzNnLe83OZs2l/7S8WQjRrSle30EgzNHDgQJ2ZmWl1GAKYvXEfj3+znu0Hj9GtdRShQXaUAoUpRZh7UJiN5c87JERy+YC29G8Xh5LRyEI0CaXUcq11tUsHSiIQDVZc5uTtBdtZuOUg5f+PtAaNNvcVHwNaa9bvzuNYiZOOCRGMG5DMuP7JtI4JtfaNCOHnJBGIZuVocRnfrc3h88ydLN1+CJuCEZ0TuWJAMuf0aEVokN3qEIXwO5IIRLO1/cBRpq7YydTlO9mdW0RMWBC/O60Nlw9Ipk9yjFQdCeEhkghEs+d0aRZuOcBnmTv5Yf0eistcdGkVyS2j0hjXv60kBCEaSRKB8Cm5haV8u2Y3Hy3NZt2uPIZ0aMETl/amU8tIq0MTwmfVlgik+6hoVmLCgrhmSCpf3zmCf1/Wmw05eZz/37k89+OvFJU6rQ5PCL9kSSJQSj2tlNqolFqjlPpCKRVrRRyi+bLZFOMHt+PnP57BhX3a8MLPv3He83OZv/mA1aEJ4XesKhHMBHpprfsAm4A/WxSHaOYSIkP4v6v68v7EIQBc++YS7v14Jfvziy2OTAj/Ycnso1rrHys8XQxc3tBzlZaWsnPnToqKZP6b5iQ0NJTk5GSCgoI8cr4RnRP4/r5RTMrYwqsZW5i9cR8Pnd+dqwelYLNJY7IQjWF5Y7FS6hvgE631+9XsvwW4BaBdu3YDsrKyTtq/bds2oqKiiI+Pl94lzYTWmoMHD5Kfn0+HDh08fv7f9hXwyJdrWbz1EANS43ji0l50ax3t8esI4S8s6zWklJoFtK5i18Na66/cxzwMDAQu03UIpKpeQxs2bKBbt26SBJoZrTUbN26ke/fuXjv/tBW7+Od3v5BfVEZaYiTRYQ6iQoOIDnUQHRZEdGhQhW3mcWxYMN2TomRtBRFQaksEXqsa0lqfXdN+pdQNwIXAWXVJArWcqzEvF17g7c9EKcW4Acmc2a0lr87ZQtbBY+QVlbIvv4jf9pWRV1RKflEZTtep/7XaxoZxzdB2XDUwhfjIEK/GKYQvsKSNQCl1HvAgcLrW+pgVMQj/EBcRzJ8vqLrUobXmWImTvKJS8gpNcth1uJBPM3fw1Pe/8vyszVzUpw0ThqXSJzm2aQMXohmxaqnKl4AQYKb7l+NirfVtFsXSKAcPHuSss84CYM+ePdjtdhITEwFYunQpwcHVr/ebmZnJu+++ywsvvFDjNYYNG8bChQsbHWtGRgYXX3zxSfX2zzzzDGefXWPhzWcppYgIcRAR4iApxmwb1B4u6deWTXvzeW9RlpniYsVOTkuJZUJ6KmP7JBHikPmORGCxvLG4PqprI/BWPXR9PfbYY0RGRvLHP/7x+LaysjIcjuaxNHRGRgbPPPMM3377bbXHaK3RWmOz2ap8Xp2q3mdz+myqk1dUyrTlO3l3URZbDxwlPiKYqwencM2QVNrEhlkdnhAeYVkbgRUe/2Y9v+zO8+g5e7SJ5m8X9azXa2644QZCQ0NZuXIlw4cP5+qrr+bee++lqKiIsLAw3n77bbp27XrSF/Njjz1GdnY2W7duJTs7m/vuu4977rkHgMjISAoKCsjIyOCxxx4jISGBdevWMWDAAN5//32UUkyfPp3777+fiIgIhg8fztatW2v8wq9o+/btnHvuuQwZMoTly5czadIkbrnlluPPp0+fzksvvcSMGTNQSvHII49w1VVXkZGRwV//+lfi4uLYuHEjmzZtqve/r9WiQ4O4YXgHrk9vz4ItB3hnYRaTMrbwSsYWzuzWkp5tYkiOC6NtXBgpceG0jgklSBqahZ/xq0TQnOzcuZOFCxdit9vJy8tj3rx5OBwOZs2axV/+8hemTp16yms2btzI7Nmzyc/Pp2vXrtx+++2n9MNfuXIl69evp02bNgwfPpwFCxYwcOBAbr31VubOnUuHDh0YP358tXHNmzePvn37Hn8+depU7HY7mzdv5p133mHo0KFs3779pOdTp05l1apVrF69mgMHDjBo0CBGjRoFwIoVK1i3bp1Xuok2JZtNMbJzIiM7J7Lj0DE+WJLNN6t389PGfVQsNNsUJMWYxJAcF0ZyXDjJsWGkxofTOzmG8GD5kxK+x6/+19b3l7s3XXHFFdjtpq45NzeXCRMmsHnzZpRSlJaWVvmasWPHEhISQkhICC1btmTv3r0kJyefdMzgwYOPb+vbty/bt28nMjKSjh07Hv8yHj9+PJMnT67yGiNHjjylpLB9+3ZSU1MZOnTo8W0Vn8+fP5/x48djt9tp1aoVp59+OsuWLSM6OprBgwf7fBKoLKVFOA+d342Hzu9GSZmLnNxCdh4uZOfhY+w6XP64kMVbDpKTt+t4onDYFD3bxjAwNY5B7eMYkNqCxCjplSSaP79KBM1JRETE8cd//etfGT16NF988QXbt2/njDPOqPI1ISEnvjTsdjtlZWUNOqax8Vb1vK6v8zfBDhup8RGkxlf9PkvKXOzJLeK3/fkszzrMsu2HeX9xFm/O3wZAh4QIBrgTw8D2LeiYECHdnUWzI4mgCeTm5tK2bVsApkyZ4vHzd+3ala1bt7J9+3bat2/PJ5984tHzjxw5ktdee40JEyZw6NAh5s6dy9NPP83GjRs9eh1fFOyw0S4+nHbx4ZzZrRVglvBctyuP5VmHWLb9MD9t2Mvny3cC0CIimAGpcQxMjWNg+zh6tY2RXkrCcpIImsCDDz7IhAkT+Oc//8nYsWM9fv6wsDAmTZrEeeedR0REBIMGDar22MptBI888ggDB1bbmQCASy+9lEWLFnHaaaehlOKpp56idevWkgiqEeKwMyA1jgGpcdwyyvS82nrgKJnbTWLI3H6Imb/sBUwiOS05hgGpLdzVSXHEhlff5VgIb5Duo36ioKCAyMhItNbceeeddO7cmd///veWxiSfTfX25xezPMskhcysw6zblUuZexR0p5aRx9sYurWOIjLEQWSog8gQByEOm1QtiXoLqO6jgez111/nnXfeoaSkhH79+nHrrbdaHZKoQWJUCOf1as15vcx0XIUlTlbvPHI8OXy3JoePlu445XVBdkWke5BcZIiDKHeCiAwNokvLSMb2SaJjoqzmJupHSgTCa+SzaTiXS7N5XwHbDhzlaHEZR0vKyC8qo6C4jIKiMo4Wl5HvflxQXEZ+USnbD5rZWronRXNhnyTG9k6ifYJ/N+aLupESgRA+yGZTdG0dRdfWUXV+zZ7cIqavzeHbNbt5+odfefqHX+nVNpoL+7RhbO8kUlqEezFi4cskEQjhJ1rHhHLTiA7cNKIDu44UMn1NDt+uzeHJGRt5csZGTkuO4cI+bbigTxJtZfoMUYEkAiH8UNvYMG4e1ZGbR3Vkx6FjfLc2h+/W5PDE9A08MX0DraNDaR0TSpvYUFpHh5n7mFCSYsJIigmlZVSIrNkQQCQRCOHnUlqEc9vpadx2ehrbDxzl+/V7+G1fATm5hWzck8/sjfspLHWe9BqbgpZRoSTFhpISF067FuaW0sKMmWgdHYpdlgj1G5IIGqkx01CDmRE0ODiYYcOGnbJvypQpPPDAA8cHowF8+OGH9OjRw4PvQASS9gkR3HZ62knbtNbkFZaRk1dIzpEicnKLyMktJCe3iN1HClm14wjfrc05aZGfYLuN5Lgwkxjct9T4cHq1jZFZW32QJIJGio+PZ9WqVUDV01DXJiMjg8jIyCoTAcBVV13FSy+9VO3rK0//XNdpr5vT9NjCWkopYsKDiAkPqnbt5zKni5zcIrIOHiP7kLntcN+v2nGE3MIT82e1jg6lX7tY9y2O3m1jCA2S0dPNmX99E8x4CPas9ew5W/eG85+s10uWL1/O/fffT0FBAQkJCUyZMoWkpCReeOEFXn31VRwOBz169ODJJ5/k1VdfxW638/777/Piiy8ycuTIWs9fefrnyZMnn/R8zZo13H777WRmZuJwOHjuuecYPXo0U6ZMYdq0aRQUFOB0OpkzZ05D/1VEgHHYbaS4q4aqknuslK0HClizM5eV2YdZueMIM9btMa+1KbonRdOvXSz928XRr10s7VqEy8C4ZsS/EkEzoLXm7rvv5quvviIxMZFPPvmEhx9+mLfeeosnn3ySbdu2ERISwpEjR4iNjeW2226rsRTxySefMH/+/OPPFy1aBJw8/XNGRsZJz5999lmUUqxdu5aNGzcyZsyY42sFrFixgjVr1tCiRQvv/2OIgBETHkS/dnH0axfHhGHtAThQUMyq7COs3HGYldlHmOpeAAggOtRBQlQIMWFBJ91iw4KIrvg8PJjIEAcOu8KmFHabwq4UNhvHH9tt5mazKYLtNil9NIB/JYJ6/nL3huLiYtatW8c555wDgNPpJCkpCYA+ffpwzTXXcMkll3DJJZfU6XzVVQ1Vnv654vP58+dz9913A9CtWzdSU1OPJ4JzzjlHkoBoEgmRIZzdoxVn9zCT8Tldmk1781mZfYRfcnI5fKyUvMJSDhaUsHX/UXILS8krKqWxY1yDHTaiQ4OIDnMQFRpEdKiD6LCg49ui3dvCgx0EOWwE2xVBdhvBDhtBdnMLttsIcqjjj8tf66+lGP9KBM2A1pqePXse/+Ve0XfffcfcuXP55ptveOKJJ1i7tuHVWDJttPA1dncVUfekqtshwIyozi8qI7ew9Pgtv6gUp9Y4XRqX1jhd5jin1pS5tHns3ldc5iKvqJS8wjLyikrJLyojr7CUXUcKzbbCUkqcrgbFH2y3kRAZTGJUCIlRISREhpz6ODKEhKgQIoLtPpU0JBF4WEhICPv372fRokWkp6dTWlrKpk2b6N69Ozt27GD06NGMGDGCjz/+mIKCAqKiosjL8+zymiNHjuSDDz7gzDPPZNOmTWRnZ9O1a1dWrFjh0esI4Wk224mGa28pKnWSX1TGsZIySp0uSso0pU6Xeex0UerUlJadeF5S5iKvqIz9+cXmVlDMriNFrN6Zy8GCYlxVlGBCg2zER5ikkBARTEJkCAlRwSe2RZptYUF2Qhw2Qhx2gh2mVGJFt1xJBB5ms9n4/PPPueeee8jNzaWsrIz77ruPLl26cO2115Kbm4vWmnvuuYfY2FguuugiLr/8cr766qsqG4srtxFMmjSp1hjuuOMObr/9dnr37o3D4WDKlCknLWgjRCALDbK72xEa/zfhdGkOHS05niD25xdzsKCYAwXFHCgo4UBBMbtzi1i7K5eDR0tO6oJbHYdNHU8KIe77YLuNf13amyEd4xsdc1Vk0jnhNfLZCHGCy6XJLSzlQIFJGgcLSigqdVJcZkodJU4XxaUuSpxO973ZXr7/jtFp9GwT06Bry6RzQgjRDNhsiriIYOIiguncqu6TCTYFmUxECCECnF8kAl+q3goU8pkI4Tt8PhGEhoZy8OBB+eJpRrTWHDx4kNDQUKtDEULUgc+3ESQnJ7Nz5072799vdSiigtDQUJKTk60OQwhRBz6fCIKCgk4aYSuEEKJ+fL5qSAghRONIIhBCiAAniUAIIQKcT40sVkrtB7IqbEoADlgUjrf563uT9+V7/PW9BdL7StVaJ1b3Ap9KBJUppTJrGjbty/z1vcn78j3++t7kfZ0gVUNCCBHgJBEIIUSA8/VEMNnqALzIX9+bvC/f46/vTd6Xm0+3EQghhGg8Xy8RCCGEaCRJBEIIEeB8NhEopc5TSv2qlPpNKfWQ1fF4ilJqu1JqrVJqlVIqs/ZXNF9KqbeUUvuUUusqbGuhlJqplNrsvo+zMsaGqOZ9PaaU2uX+3FYppS6wMsaGUEqlKKVmK6V+UUqtV0rd697u059ZDe/LHz6zUKXUUqXUavd7e9y9vYNSaon7+/ETpVRwjefxxTYCpZQd2AScA+wElgHjtda/WBqYByiltgMDtdY+P9BFKTUKKADe1Vr3cm97CjiktX7SncDjtNZ/sjLO+qrmfT0GFGitn7EytsZQSiUBSVrrFUqpKGA5cAlwAz78mdXwvq7E9z8zBURorQuUUkHAfOBe4H5gmtb6Y6XUq8BqrfUr1Z3HV0sEg4HftNZbtdYlwMfAxRbHJCrRWs8FDlXafDHwjvvxO5g/SJ9SzfvyeVrrHK31CvfjfGAD0BYf/8xqeF8+TxsF7qdB7psGzgQ+d2+v9TPz1UTQFthR4flO/OSDxXyIPyqlliulbrE6GC9opbXOcT/eA7SyMhgPu0sptcZddeRT1SeVKaXaA/2AJfjRZ1bpfYEffGZKKbtSahWwD5gJbAGOaK3L3IfU+v3oq4nAn43QWvcHzgfudFdD+CVt6iV9r26yaq8AaUBfIAd41tJoGkEpFQlMBe7TWudV3OfLn1kV78svPjOttVNr3RdIxtSWdKvvOXw1EewCUio8T3Zv83la613u+33AF5gP1p/sddfZltfd7rM4Ho/QWu91/0G6gNfx0c/NXc88FfhAaz3NvdnnP7Oq3pe/fGbltNZHgNlAOhCrlCpfeKzW70dfTQTLgM7ulvFg4Grga4tjajSlVIS7MQulVAQwBlhX86t8ztfABPfjCcBXFsbiMeVflG6X4oOfm7vh8U1gg9b6uQq7fPozq+59+clnlqiUinU/DsN0oNmASQiXuw+r9TPzyV5DAO6uXs8DduAtrfUT1kbUeEqpjphSAJhlRD/05fellPoIOAMzLe5e4G/Al8CnQDvMlOJXaq19quG1mvd1BqaKQQPbgVsr1Kv7BKXUCGAesBZwuTf/BVOf7rOfWQ3vazy+/5n1wTQG2zE/7D/VWv/d/V3yMdACWAlcq7UurvY8vpoIhBBCeIavVg0JIYTwEEkEQggR4CQRCCFEgJNEIIQQAU4SgRBCBDhJBKLZUUo53bNBrlNKfaaUCq/h2BuUUi81ZXwVrv13pdTZtRwzRSl1eS3HtK84k6kQTU0SgWiOCrXWfd0ze5YAt1kdUFW01o9qrWdZHUdF7pl5hagXSQSiuZsHdHLPif+le4Kwxe6BNMcppaKUUtvcUwmglIouf66UylBK/cc9b/smpdRI9zGhSqm3lVn/YaVSarR7+w3ua81UZn2Iu5RS97uPWayUauE+7vivfaXUo0qpZe5SzGT3aNZqKaUGuOeQXw3cWWG7XSn1tPtca5RSt7q325RSk5RSG91xTa9w7e3u97cCuEIpNUYptUgptcJdooqscM057gkNf6g0slYEMEkEotlyz5VyPmZE6OPASq11H8yo0HcrHuueXjgDGOvedDVmPvZS93OH1nowcB9mJDCYL2Ctte6NGWX6jlIq1L2vF3AZMAh4Ajimte4HLAKuryLcl7TWg9ylmDDgwlre3tvA3Vrr0yptnwjkaq0Hua99s1KqgzuW9kAP4DrMfDIVHXRPVjgLeAQ42/08E7jfnSBfBC7XWg8A3nK/LyFw1H6IEE0uzD2tLpgSwZuYaQ7GAWitf1ZKxSuloiu97g3gQcw0FjcCN1fYVz6B2nLMFyrACMyXI1rrjUqpLKCLe99sd3LJV0rlAt+4t68FTiqNuI1WSj0IhGOG9a+v8JqTuOeGiXWvawDwHibhgZlfqk+FdoUYoLM71s/cE6TtUUrNrnTaT9z3QzHJYoG7UBKMSV5dMcltpnu7HTPjphCSCESzVOieVve4WmpaANBaL3A3vJ4B2LXWFRtgy+dZcVK3//cV52VxVXjuqvx6dyliEmZluR3KrFYWSsMoTEnhh0rXqG0ZxaMVXj9Taz2+0ut7A+u11pVLEkJI1ZDwGfOAawDcX/QHKs+V7/Yu8CGm6qU+5+yCmVTt1wbEVv6lf8BdH19jLyH3dMFH3JOhUR6D2w/A7RXaOrooMxPtAmCcu62gFWaSu6osBoYrpTq5Xx/hfm+/AolKqXT39iClVM96vk/hp6REIHzFY8BbSqk1wDFOTItc2QfAP4GP6nDOScArSqm1QBlwg9a6uC6lj4q01keUUq9jpjHeg5kmvTY3Yt6PBn6ssP0NTNXVCneD837MMoNTgbOAXzCr860AcquIZb9S6gbgI6VUiHvzI1rrTe7qpheUUjGYv/3nMVVYIsDJ7KPCr7i/7C7WWl9ndSyeppSKdC9SHg8sBYZrrfdYHZfwfVIiEH5DKfUiptG1tvp0X/Wtu6E5GPiHJAHhKVIiEEKIACeNxUIIEeAkEQghRICTRCCEEAFOEoEQQgQ4SQRCCBHg/h8qfdt7vKJVhAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_65_10.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Common imports\n", "import os\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from sklearn.linear_model import LinearRegression, Ridge, Lasso\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.utils import resample\n", "from sklearn.metrics import mean_squared_error\n", "# Where to save the figures and data files\n", "PROJECT_ROOT_DIR = \"Results\"\n", "FIGURE_ID = \"Results/FigureFiles\"\n", "DATA_ID = \"DataFiles/\"\n", "\n", "if not os.path.exists(PROJECT_ROOT_DIR):\n", " os.mkdir(PROJECT_ROOT_DIR)\n", "\n", "if not os.path.exists(FIGURE_ID):\n", " os.makedirs(FIGURE_ID)\n", "\n", "if not os.path.exists(DATA_ID):\n", " os.makedirs(DATA_ID)\n", "\n", "def image_path(fig_id):\n", " return os.path.join(FIGURE_ID, fig_id)\n", "\n", "def data_path(dat_id):\n", " return os.path.join(DATA_ID, dat_id)\n", "\n", "def save_fig(fig_id):\n", " plt.savefig(image_path(fig_id) + \".png\", format='png')\n", "\n", "infile = open(data_path(\"EoS.csv\"),'r')\n", "\n", "# Read the EoS data as csv file and organize the data into two arrays with density and energies\n", "EoS = pd.read_csv(infile, names=('Density', 'Energy'))\n", "EoS['Energy'] = pd.to_numeric(EoS['Energy'], errors='coerce')\n", "EoS = EoS.dropna()\n", "Energies = EoS['Energy']\n", "Density = EoS['Density']\n", "# The design matrix now as function of various polytrops\n", "\n", "Maxpolydegree = 30\n", "X = np.zeros((len(Density),Maxpolydegree))\n", "X[:,0] = 1.0\n", "testerror = np.zeros(Maxpolydegree)\n", "trainingerror = np.zeros(Maxpolydegree)\n", "polynomial = np.zeros(Maxpolydegree)\n", "\n", "trials = 100\n", "for polydegree in range(1, Maxpolydegree):\n", " polynomial[polydegree] = polydegree\n", " for degree in range(polydegree):\n", " X[:,degree] = Density**(degree/3.0)\n", "\n", "# loop over trials in order to estimate the expectation value of the MSE\n", " testerror[polydegree] = 0.0\n", " trainingerror[polydegree] = 0.0\n", " for samples in range(trials):\n", " x_train, x_test, y_train, y_test = train_test_split(X, Energies, test_size=0.2)\n", " model = LinearRegression(fit_intercept=False).fit(x_train, y_train)\n", " ypred = model.predict(x_train)\n", " ytilde = model.predict(x_test)\n", " testerror[polydegree] += mean_squared_error(y_test, ytilde)\n", " trainingerror[polydegree] += mean_squared_error(y_train, ypred) \n", "\n", " testerror[polydegree] /= trials\n", " trainingerror[polydegree] /= trials\n", " print(\"Degree of polynomial: %3d\"% polynomial[polydegree])\n", " print(\"Mean squared error on training data: %.8f\" % trainingerror[polydegree])\n", " print(\"Mean squared error on test data: %.8f\" % testerror[polydegree])\n", "\n", "plt.plot(polynomial, np.log10(trainingerror), label='Training Error')\n", "plt.plot(polynomial, np.log10(testerror), label='Test Error')\n", "plt.xlabel('Polynomial degree')\n", "plt.ylabel('log10[MSE]')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Cross-validation\n", "\n", "When the repetitive splitting of the data set is done randomly,\n", "samples may accidently end up in a fast majority of the splits in\n", "either training or test set. Such samples may have an unbalanced\n", "influence on either model building or prediction evaluation. To avoid\n", "this $k$-fold cross-validation structures the data splitting. The\n", "samples are divided into $k$ more or less equally sized exhaustive and\n", "mutually exclusive subsets. In turn (at each split) one of these\n", "subsets plays the role of the test set while the union of the\n", "remaining subsets constitutes the training set. Such a splitting\n", "warrants a balanced representation of each sample in both training and\n", "test set over the splits. Still the division into the $k$ subsets\n", "involves a degree of randomness. This may be fully excluded when\n", "choosing $k=n$. This particular case is referred to as leave-one-out\n", "cross-validation (LOOCV). \n", "\n", "\n", "* Define a range of interest for the penalty parameter.\n", "\n", "* Divide the data set into training and test set comprising samples $\\{1, \\ldots, n\\} \\setminus i$ and $\\{ i \\}$, respectively.\n", "\n", "* Fit the linear regression model by means of ridge estimation for each $\\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\\boldsymbol{\\sigma}_{-i}^2(\\lambda)$, as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{align*}\n", "\\boldsymbol{\\beta}_{-i}(\\lambda) & = ( \\boldsymbol{X}_{-i, \\ast}^{T}\n", "\\boldsymbol{X}_{-i, \\ast} + \\lambda \\boldsymbol{I}_{pp})^{-1}\n", "\\boldsymbol{X}_{-i, \\ast}^{T} \\boldsymbol{y}_{-i}\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* Evaluate the prediction performance of these models on the test set by $\\log\\{L[y_i, \\boldsymbol{X}_{i, \\ast}; \\boldsymbol{\\beta}_{-i}(\\lambda), \\boldsymbol{\\sigma}_{-i}^2(\\lambda)]\\}$. Or, by the prediction error $|y_i - \\boldsymbol{X}_{i, \\ast} \\boldsymbol{\\beta}_{-i}(\\lambda)|$, the relative error, the error squared or the R2 score function.\n", "\n", "* Repeat the first three steps such that each sample plays the role of the test set once.\n", "\n", "* Average the prediction performances of the test sets at each grid point of the penalty bias/parameter. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{align*}\n", "\\frac{1}{n} \\sum_{i = 1}^n \\log\\{L[y_i, \\mathbf{X}_{i, \\ast}; \\boldsymbol{\\beta}_{-i}(\\lambda), \\boldsymbol{\\sigma}_{-i}^2(\\lambda)]\\}.\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For the various values of $k$\n", "\n", "1. shuffle the dataset randomly.\n", "\n", "2. Split the dataset into $k$ groups.\n", "\n", "3. For each unique group:\n", "\n", "a. Decide which group to use as set for test data\n", "\n", "b. Take the remaining groups as a training data set\n", "\n", "c. Fit a model on the training set and evaluate it on the test set\n", "\n", "d. Retain the evaluation score and discard the model\n", "\n", "\n", "5. Summarize the model using the sample of model evaluation scores\n", "\n", "The code here uses Ridge regression with cross-validation (CV) resampling and $k$-fold CV in order to fit a specific polynomial." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEGCAYAAABvtY4XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA3c0lEQVR4nO3dd3gU5fr/8fedRhIICYRQQwmdAKEX6UUpggLiEURQ0CMqKvaf7dg5ikeP9SiKigVRiiIioIBUkRpa6D20UBJKII2UfX5/ZOGLECCJmZ3dzf26rlwkO7NzfzYhd2afmXlGjDEopZTyPj52B1BKKWUNbfBKKeWltMErpZSX0gavlFJeShu8Ukp5KT+7A1ysXLlypkaNGnbHUEopj7F27dokY0xEXsvcqsHXqFGD2NhYu2MopZTHEJH9V1qmQzRKKeWltMErpZSX0gavlFJeyq3G4POSlZXFoUOHyMjIsDuKKoTAwEAiIyPx9/e3O4pSxY7bN/hDhw4REhJCjRo1EBG746gCMMZw4sQJDh06RFRUlN1xlCp23H6IJiMjg/DwcG3uHkhECA8P13dfStnE7Rs8oM3dg+nPTin7uP0QjVLKvRzdtJP9X36POXOGErf0p2nvDvqH3E1pg1dK5YtxOFg18imaffUhFXOyABieHELGIT/+e1tTqoQF2ZxQXcojhmjUtcXHx9OoUSO7YygvZRwOVvW/i7ZfvMOWFp04vCaOtJOnuf6RoWw5fIaJd/+LhHVb7I6pLqF78H9TdnY2fn7F79tYXF93cTVp5X7iUwJwDBjBdT98jvjk7hsO7RBKizBfKr59G6dX/ET6to0EhYXYnFad51G/oa/8soWtCWeKdJvRlUvz0k0Nr7rON998w9tvv42IEBMTg6+vL4GBgaxfv5727dtz5513cv/995OWlkatWrWYMGECZcqU4YMPPuCTTz7Bz8+P6OhoJk+ezJIlS3jkkUeA3AOQS5cuJSTk8l+IwYMHM2zYMPr06QPA8OHD6du3Ly1btmTYsGGkpqYC8L///Y927dpd83Vu2bKFESNGkJmZicPh4Mcff6ROnTqXvbaJEycSHx/P3XffTVJSEhEREXz55ZdUq1aN4cOH/+V1P/jggzz44IMkJiYSHBzMZ599Rv369Qv6I1Bubk9iCmN+3U6rO0by3IjWiM9fx9sbNIpi04ef0fieQawadA9t5k61Kam6lKUNXkQeAe4FBPjMGPOelfWssGXLFsaMGcPy5cspV64cJ0+e5PHHH+fQoUMsX74cX19fYmJi+PDDD+ncuTMvvvgir7zyCu+99x5jx45l3759lChRgtOnTwPw9ttv89FHH9G+fXtSUlIIDAzMs+6gQYOYOnUqffr0ITMzkwULFjBu3DiMMcyfP5/AwEB27drF7bffnq8J2j755BMeeeQR7rjjDjIzM8nJycnztQE8/PDD3HXXXdx1111MmDCB0aNHM2PGDIC/vO7u3bvzySefUKdOHVatWsWoUaNYuHBhkXzflfs4eOtQ+kZE89RTL+Ljk/fB1MZ338bKWb/S9qev2D5jHvX793BxSpUnY4wlH0AjYDMQTO4fkt+B2ld7TosWLcyltm7detljrvTBBx+Y55577i+P3XXXXearr74yxhhz+vRpU7Vq1QvLdu/ebZo1a2aMMaZnz55m4MCBZuLEiebs2bPGGGPeeOMN07p1a/P++++bgwcPXrFuenq6qVq1qsnIyDAzZswwQ4YMuVBv6NChplGjRqZJkyYmKCjIGGPMvn37TMOGDa+4vUmTJpno6GgzduxYs3Pnziu+NmOMCQ8PN5mZmcYYYzIzM014ePhlr/vs2bMmMDDQNGnS5MJH/fr186xt989QFd6mr380BsyKB5655rpnE0+a4yFlzc5q9U1OVrYL0iljjAFizRV6qpUHWRsAq4wxacaYbGAJcIuF9VyqZMmS11xn9uzZPPjgg6xbt45WrVqRnZ3NM888w+eff056ejrt27dn+/bteT43MDCQLl26MHfuXKZMmcKgQYMAePfdd6lQoQIbN24kNjaWzMzMfOUdMmQIM2fOJCgoiBtvvLHQe9rnX7fD4SAsLIwNGzZc+Ni2bVuhtqncl98rL3M0rDxN3/zXNdctVa4MO157h7Gt/sH87cddkE5di5UNfjPQUUTCRSQYuBGoeulKIjJSRGJFJDYxMdHCOIXTrVs3pk2bxokTJwAuDGOcFxoaSpkyZfjjjz8AmDhxIp07d8bhcHDw4EG6du3Km2++SXJyMikpKezZs4fGjRvz9NNP06pVqys2eMgdpvnyyy/5448/6NWrFwDJyclUqlQJHx8fJk6cSE5OTr5ex969e6lZsyajR4+mX79+xMXFXfG1tWvXjsmTJwMwadIkOnbseNn2SpcuTVRUFNOmTQNy3wlu3LgxX1mUZ9g+/Tfq793E/hGjCAy59g4NQNuHhrG7TVc+WLj7/Dt5ZSPLxuCNMdtE5E1gHpAKbAAu60bGmPHAeICWLVu63f+Ihg0b8vzzz9O5c2d8fX1p1qzZZet8/fXXFw6y1qxZky+//JKcnByGDh1KcnIyxhhGjx5NWFgYL7zwAosWLcLHx4eGDRvSu3fvK9bu0aMHw4YNo1+/fgQEBAAwatQoBg4cyDfffEOvXr3y9U4CYOrUqUycOBF/f38qVqzIc889R9myZS97bV999RUffvghI0aM4K233rpwkDUvkyZN4oEHHmDMmDFkZWUxePBgmjRpkq88yv2lv/4myYGlaPzCo/l+jp+vD6PbVibh6ZeIC02kyV1e86bdI4mr/sqKyOvAIWPMx1dap2XLlubSA4bbtm2jQYMGVsdTFtKfoec5fiaDV4a/yo01QujzzrMFem5mWgZny1fiQN0Ymq1bYlFCdZ6IrDXGtMxrmaUXOolIeee/1cgdf//OynpKqaIxY8NhZtdtT/1nHy7wcwOCA9nZ9zZiNizj6KadFqRT+WX1law/ishW4BfgQWPMaYvreZxNmzbRtGnTv3y0adOm0NubO3fuZdsbMGBAESZW3s44HGR88BHdS2dRK6JUobZR47nHEGPYN/a9og2nCsTS8+CNMZcfnVN/0bhxYzZs2FBk2+vZsyc9e/Yssu2p4mf3r0sY/eO7rKoTAfQv1DYqxdRnQ5P21Jk5mcy0dwgIzvt6D2UtnYtGKfUXJz8ezzlff+qP/uff2o55YBQrIxuxav3uIkqmCsqjpipQSlnLkZ1DraXz2NK8I80rRfytbTW8ZxAjjoXT+WAW+lbeHroHr5S6YOesBZRLOYmjX/+/va0APx9ubFyJnUvXkna6aOeQUvmjDT4fSpX6vwNNc+bMoW7duuzfv5+XX36ZKlWqXDiY+cwzz1xxG1ebzrdLly75mk9GKavFL1hBhl8AdYYPKpLtDZFj/DruXraOm1gk21MFo0M0BbBgwQJGjx7N3LlzqV69OgCPPfYYTz75pM3JlCoa/6nRmWnvduKLKuWLZHvRfbtyLDQCvymT4dkHi2SbKv90Dz6fli5dyr333susWbOoVavWFdczxvDUU0/RqFEjGjduzJQpUy5bJz09ncGDB9OgQQMGDBhAenq6ldGVypfdx1PYm5hK5xZRRbZNHz9f9nXvS/SmlSQfcb+pSLyd5+3Bd+ly+WO33QajRkFaGtx44+XLhw/P/UhKgltv/euyxYuvWfLcuXP079+fxYsXXzbf+bvvvsu3334LwJtvvklKSgobNmxg48aNJCUl0apVKzp16vSX54wbN47g4GC2bdtGXFwczZs3v2YGpax2/OV/893sOdR4rGivPg27YxAB078k7quptNS9eJfSPfh88Pf3p127dnzxxReXLXvssccuzKbYs2dPli1bxu23346vry8VKlSgc+fOrFmz5i/PWbp0KUOHDgUgJiaGmJgYl7wOpa4mdOF8KmanUrlCWJFut+7N15NUqgzy889Ful11bZ63B3+1Pe7g4KsvL1cuX3vsl/Lx8WHq1Kl0796d119/neeee67A21DKnaWdSqbOro2s6z+MmkW8bR8/X6Y+/wETT5RgcXYOJfx8i7iCuhLdg8+n4OBgZs+ezaRJk/Lckz+vY8eOTJkyhZycHBITE1m6dCmtW7f+yzqdOnXiu+9yp+XZvHkzcXFxlmZX6lp2/TCHAEc2Jfv2smT7DQb04IhvMMv3nLBk+ypvnrcHb6OyZcvy22+/0alTJyIi8r4IZMCAAaxYsYImTZogIvznP/+hYsWKxMfHX1jngQceYMSIETRo0IAGDRrQokULF70CpfKWPutXzvn6U2dgHsewikC7WuEM3zyXtDcWwVdvW1JDXc5l0wXnh04X7J30Z+j+/jPkORqcOsRNv35jWY11ba4nctsGIk4fR3x08KCo2DZdsFLK/R1NzuDjau1JeOYlS+tk9+xN+bMn2LtghaV1PM23K/fz5LSNOBxFv7OtDV6pYm7dn5sITT9LhzrlLK1Tc2ju3Z2OT5thaR1PU+6FZ6g7aTw+PlLk2/aIBu9Ow0iqYPRn5/7K/fcNFn92Hw0qhFhbp24Ue6rUIXTx75bW8STZmVl0WPozTc4lWbJ9t2/wgYGBnDhxQhuFBzLGcOLECQIDdS5wd1Ypbg3x9Zvi42t9Ozja+QZOZRnOpGZYXssTxC9aQanMdHwvuRiyqFh6Fo2IPAb8EzDAJmCEMaZAP9nIyEgOHTpEYqJe5uyJAgMDiYyMtDuGuoKknfuomnSIw7cNc0k9/9de5bbxKxm39xS9G1dySU13lvTrAmoDkTf1sGT7ljV4EakCjAaijTHpIjIVGAx8VZDt+Pv7ExVVdHNjKKX+z/6f51IOKNv7epfUa169DCGBfizdkqANHvBfsZyjYeWp2KiOJdu3+j2ZHxAkIn5AMJBgcT2lVAFkL15Cmn8gNW/o4JJ6fr4+jI37kYdH9cU4HC6p6a6MMcSbEuxs082yGpY1eGPMYeBt4ABwBEg2xsy7dD0RGSkisSISq8MwSrnWp01v4pP7XsWvRIDLapZvWJfKp46yb9FKl9V0R/tPpPFEtwc48PJYy2pY1uBFpAzQD4gCKgMlRWTopesZY8YbY1oaY1pe6epQpVTRO52WycKcUAIG9Hdp3aqD+gFwfMYcl9Z1N7G7jwHQOqqsZTWsHKK5HthnjEk0xmQB04F2FtZTShXAjt+Xc8vmBbQq79qznCo2qsOBiKoELV3k0rruptrzT/LLt09QO6LUtVcuJCsb/AGgrYgEi4gA3YFtFtZTShWAY8pU/jPnfRpXKe3y2kdatafOtnVkphXf0yUrbYolJ6K8JRc4nWflGPwq4AdgHbmnSPoA462qp5QqmFLrY4mvUovgMqGuL377EN5tP4S4vcddX9sNJO2Op2riQTLaXGdpHUvPojHGvGSMqW+MaWSMGWaMOWdlPaVU/uRkZRO1ZwsnGjWzpX79gb34ou0tLD1SPPfg46fNBiD8JmumZz7P7a9kVUoVvYPL11EqMw2fttbuQV5JaJA/7cOEs7N+taW+3Ry/L+BMYClq9uhoaR1t8EoVQ0eX5s7oWLFnZ9syjNrwC89/+Dhnjxe/m4BMr9qcOQPvw9ff2ltyaINXqhiaEd2V6x//jsjWTWzLENK3F37GwZ5pxet0yYMn05hcsRkZD1h/A3Jt8EoVQ+sPnCayUS1bb7xRu98NpPuVIOO3ubZlsMOm31dQ68RB2tcKt7yWNnilipmzx5IYPf55ep/db2uOwFLB7KrXjIpr/rQ1h6uV/+C/TJn8HLXLW3f++3na4JUqZuJ/XUKf7ctoYO307/mS1rEzNY7Fk7Rzn91RXMI4HNTYuIr4mDYuefekDV6pYiZlyTIAqvfqYm8QIPS+e+hy76f8kWLtwUZ3cWD5OsqlnMTRtatL6mmDV6qYCdqwlv3lqxFayf65n+rF1CY5sgbLdp+0O4pLHJ2ee/575MC+LqmnDV6pYqbynq0k1m1kdwwAfHyEO7P2c907LxaL6YNLLPydQ+FVqNws2iX1tMErVYwcP3qCY8FhZLZqbXeUC9rnnOTWVTM5sGK93VEslZGVw8jrH2bWv95zWU1t8EoVI5tPZ3PT8Pfxe/hhu6NcUMU5XHH0x1k2J7HWqn0nOe5Xknq9rbn/al60wStVjGw6dAYRaFjZ9TNIXkmVFg1JKFuJgCWL7Y5iqZR3P+Ce9bO4rqb157+fpw1eqWKkzYuj+XT+B5Qs4V5nrRxq3o5aW9aQfS7T7iiWifnhK24+spFAf1+X1dQGr1QxUnNLLGUDXddg8su3x/UcL1mG7et22B3FEodjN1E16RDp3Xu4tK57/RlXSlkmcc9+yp9JYm+z5nZHuUzUAyNofqIqT6YH4h7n9xStQ99NpwoQefstLq2re/BKFRMJv+de4FS6fRubk1yubKkSRFcqzbJdiXZHsUTggnkcLBdJZOsYl9bVBq9UMZG2Yg0OhGrXd7A7Sp4e2L+MD5/oS/qpM3ZHKVJp57I4mZbFofbdXV7bsgYvIvVEZMNFH2dE5FGr6imlrm5TUARz2txIqXJl7I6Sp+oNaxKReord073rJiBLdiYxYuBL8PbbLq9t5T1ZdxhjmhpjmgItgDTgJ6vqKaWubkLVtsx/9DW7Y1xR7Vt6kenrR+qv8+yOUqQWro+nTLA/raLKury2q4ZougN7jDH2zk+qVDGVmJhMSuJJGlex4Qbb+RRcJpTdtRpTbtUfdkcpMplpGTx/f0/G7JiDn6/rR8RdVXEw8H1eC0RkpIjEikhsYqJ3HmBRym4J02ez+b3baJu42+4oV5XcrjO1D+3i1P4Eu6MUie3f/0xY+lmqdGhhS33LG7yIBAA3A9PyWm6MGW+MaWmMaRkRYf/sdkp5o7SVq3Eg1OjUyu4oVxU2eCCftBnI2j3esbOXPvVHUgOCqD/UtadHnueKPfjewDpjzDEX1FJK5SEwbiOHIyLd9gDreXWub8dHPe9lwWnPP8EvJyubWst/Z0fT9gSGlLQlgyu+i7dzheEZpZRrVNi3neO1Gtgd45r8fH3oUC2E5N8X2x3lb9v183zKpZzC0b+/bRksbfAiUhK4AZhuZR2l1JUlH0mk8qmjZDZ27UU2hTV88zw+Hjeaw2u32B3lb5mZGszY7vdQ757BtmWwtMEbY1KNMeHGmGQr6yilrmzX8VT+dcMD+PbtY3eUfKnonD74sAdPH5yd42DKgUwOjHiAkPKumz3yUp4/0KWUuqrNKYZvm/ehepe2dkfJl2rtmnM8JBy/Bb/bHaXQNv70Ox1Wz6V/dDlbc2iDV8rLnVm2gibnkigfUsLuKPkiPj7sb9GeWnErPXb6YMcHH/Da/E/oXNfeMwO1wSvl5Xp/9Cpj5n6EiNgdJd98buxNaEYKu2cvtDtKgaWfPkv0qoXs6NCDEiWDbc2iDV4pL5Z9LpNqCXtJrd/Q7igFUnvYP7ht2FvMCapqd5QC2zr+W0pmphM0/E67o2iDV8qbHVq5gRI5Wfg1b2Z3lAIJrRiOadeORbtP2R2lwHy//57jIeE0GNTX7ija4JXyZknLVgEQ3r61zUkK7uZS6Qz4+i0S93jOFFbHT5zF70gCe3r2x9ff/vspaYNXyotlr99Apq8fkdd51h48wHUR/ty9dibx33rOZTTT4o7R9673qPjef+yOAmiDV8qrfde2P6/c/xb+gZ5xBs3Fat3QgcSQsvj89pvdUfLFkZ3Dz8t20LZWOFFVXD81cF60wSvlpYwx/JkeSGbnrnZHKRTx8WFfi47U3rDcI06X3DLpZ6aP+Qf3B52wO8oF2uCV8lJJ+xO4eck0Wvqk2B2l0Pz69iE0I4WdM+bbHeWaMj/5lBwfX9r2d58/qNrglfJSR37/g5cWfEajTPfZoyyouncO5FipsuyI3Wp3lKs6tT+BxmsWsu36frbNHJkXbfBKeanU1WsBiOzkGVMU5KVURFmefW8270S0wBhjd5wr2vHafwnIyabik6PtjvIX2uCV8lIBmzdxNKw8oVXK2x3lb+nRuDIHT6Sxbb97vhPJzMik5tSv2RTdmhqd3et0VG3wSnmpcnu2cySqvt0x/rbra4ay6LP7SH7pVbuj5GnOtkTuvPVlMsa8YXeUy2iDV8oLZaSkUSnpMOeiG9kd5W8rVy6UjLLlKP/7r3ZHuYwxhgl/7iOzYSNa9HOfg6vnaYNXygvtOJVJk0cmk/LAQ3ZHKRLJvfpSK2E3Cevd62Dr9ulzuXvcvxhVvyQ+Pu43mZs2eKW80NYjZ0gPCKRu/ep2RykSVUcMAeDAF5NsTvJXGa+/Sde9a7mxfT27o+TJ6lv2hYnIDyKyXUS2ich1VtZTSuUqNeEznlg1hcgyQXZHKRJVWjRkT5U6lJ31k91RLti7YDnN1i1m663DCQ4rbXecPFk9G877wG/GmFtFJACwd3JkpYqJ2r/PpA645bBBYe0a9QQ/rTnAs4kp1IgoZXccTj/3IikBwUSPfcHuKFdk2R68iIQCnYAvAIwxmcaY01bVU0rlcmTnUPXgbpLrRdsdpUg1efBO5tVvx8y4I3ZHIX7xKpqvXsCmW+5069NQrRyiiQISgS9FZL2IfC4il13iJSIjRSRWRGITExMtjKNU8XBkw1ZKZaYhTZraHaVIVQoNok+pc/h8+D7G4bA1y6fbU5hw3UAavPmirTmuxcoG7wc0B8YZY5oBqcAzl65kjBlvjGlpjGkZEWHv/QuV8gbHl60GIKxdK5uTFL07U3fx0M8fsfu3P2zLsOPoWSbvz+DYC68RVq2SbTnyw8oGfwg4ZIxZ5fz6B3IbvlLKQscPHiMpOJRqndzrqsqiUO/BEWT6+nHyo09ty7B3+P10Orqd+zvVsi1DflnW4I0xR4GDInL+/KHugHudxKqUF5oW04PBr84gsLT9ByKLWmiV8sS1vYEGC2aSfvqsy+tv/uoHes/9jvuDkyhTMsDl9QvK6vPgHwYmiUgc0BR43eJ6ShV7246coUHlULtjWCb4oQcofS6VTe9+5tK6OVnZBD3/LAllKtLsTfc9c+ZiljZ4Y8wG5/h6jDGmvzHG8+6gq5QHST58nM/fuYfe8bF2R7FMg9v6sKdSTfat2ezSumvHfJB7Ne3TLxJYyjPO+NYrWZXyIocWr6BBYjyRESF2R7GM+PiweNIcno4ZyNaEMy6pefrgUeq8/So7ohrS/ImRLqlZFPLd4EWkg4iMcH4eISJR1sVSShXGWecc8JU7e+4c8Plxa5uaBAf4MnXGCpfUG7v8MJ+0vZWALz7Dx8/XJTWLQr4avIi8BDwNPOt8yB/41qpQSqnC8YmL40TJMMJre8ccNFcSGuzPWydX8vzDfUjYsM3SWiv3JDF5/VF8nnqSqK6eNdtKfvfgBwA3k3suO8aYBMB73wMq5aHK7t5GQvW6iHjPFAVX0nLUHRiEg/8aY1mNjDMpBPa4nkFH1zO6Wx3L6lglvw0+0+TeL8sA5HVFqlLKXlk5DtaH1+Bo+252R3GJCtF12NCpDzFzfyRpd7wlNTbefi9N925kWLdoggI8Z2jmvPw2+Kki8ikQJiL3Ar8Drj1HSSl1VXsTU3mq12hSvWQO+Pyo/Oar+Dpy2PvAk0W+7Q3jJtJmzmRW9r+LRnfeUuTbd4V8NXhjzNvkXon6I1APeNEY86GVwZRSBbPtQCIYQ4NK7jl1rRUiW8ew7qYh1Fk2jx3bDxTZdhPWb6X6kw+xt3Itmn37cZFt19Xye5C1JLDQGPMUuXvuQSLib2kypVSBVHxzDCvHDadmuHfMAZ9f9ce9Tf+Hv+DVZQnkjiT/PemZOfz+/Dv4Ohz4/zSdEiU945z3vOR3iGYpUEJEqgC/AcOAr6wKpZQquJLbt5ASGo6/v9W3eXAvYZUiuGdAK5bvSmTulAV/a1vZOQ4em7KBl2L6s3n2Yqq2jimilPbIb4MXY0wacAu5s0P+A2hoXSylVEEYY6i8fycnazewO4othrapzofrv6fDXf04tDquUNswDgdLb72XXUvW8K++DWnXrUURp3S9fDd45+327gBmOx/zvEPKSnmpE7v3E556GkeTJnZHsYWPj9D8Py+Q7etLdr9+pCQVbFaU7HOZrO41iG4zJvCyYyf3dPCO6zjz2+AfIXcu9+nGmC3Oq1gXWhdLKVUQCUtWAhDS2vP3OgurctMGHPz4S6oeO8D+9tfne7bJ1BOn2XTdDbSZ/wMrBo2kw+dvW5zUdfLb4NMAB3C7c2bImUBXy1IppQpki28on7a+hcgunnWlZVFrPHwg6//9PvV3bSCu040knE6/6vqblq3ndN1omqz/g1VPvMJ1kz9FfLxniq78Ho2ZBDwJbCa30Sul3MjywIqsG/Ag91XWu6K1fPYh1pcsyUfrTrL+3aU81rwM/WIqER5VFci9Z+2WPzcwPsGH39Yf4NMqdTnz8XjaDOprc/Kil98Gn2iM+cXSJEqpQstYt4GYurXtjuE2mo0ewStJqbw2aytn//MuZZZP5nDZimQGBBJx8hjVgWWPf8e93erR6tX5hAR651nfkp/zRkWkO3A7sAA4d/5xY8z0ogzTsmVLExvrvfNYK2WFjDMp+IeFsub2+2g7yXMvyrHKvmVrOfrFRAJ2bMcn8xyZFSri26E99R8cQanSnj/rioisNca0zGtZfvfgRwD1yZ1F8vwQjQGu2uBFJB44C+QA2VcKoZQqvANLV1PXOAhoobc8zktUhxZEdSieB5/z2+BbGWPqXXu1PHU1xiQV8rlKqWs4vWINAOU7trE5iXI3+T1cvFxEoi1NopQqFLNhI6kBQVRupr+i6q/y2+DbAhtEZIeIxInIJufpktdigHkislZE8rzPlYiMFJFYEYlNTEzMb26llFPIzq0cqFrHo+40pFwjv0M0vQq5/Q7GmMMiUh6YLyLbjTFLL17BGDMeGA+5B1kLWUepYskYw9gOw+gSFUrxnKRAXU2+GrwxZn9hNm6MOez897iI/AS0JnfiMqVUETh0Kp2lFerTs2cju6MoN2TZJVsiUlJEQs5/DvQg90IppVQRiV+2lut3raJhmQC7oyg3ZOU1uRWAZSKyEVgNzDbG/GZhPaWKHb9pUxk/fQx1Izx3znJlHcsmjjbG7AWK59R2SrlI4NbNHI6IpGqZULujKDfkPbPqKFUMVdi3neM169sdQ7kpbfBKeajkI4lUPnWUrEaN7Y6i3JQ2eKU81KGFywEIbtvK5iTKXWmDV8pDrawSTfd7xlGpbw+7oyg3pQ1eKQ+16WgKqTXrEFGxrN1RlJvSBq+Uh2o+4X0Gndxidwzlxiw7TVIpZZ3Uk8kMnf8NqyqVtjuKcmO6B6+UBzqwaAU+GIJa6wFWdWXa4JXyQMl/rgKgcrd2NidR7kwbvFIeyHf9Ok6UDCOiXpTdUZQb0wavlAcyx45zOKoB4qO/wurK9CCrUh4mIyuH2/u/wIPtqhJjdxjl1vTPv1IeZsfRs+Q4DNE1ytkdRbk5bfBKeZizX09k/PQxNArRX191dTpEo5SH8V+6lDYHN1O6aoTdUZSb010ApTxM2R2bORhVXw+wqmvS/yFKeZDM9HNUP7yblGg9vKquzfIGLyK+IrJeRGZZXUspb3dgyUoCcrLxb93S7ijKA7hiDP4RYBugk2Yo9TftPnCC5Mr1qdSji91RlAewdA9eRCKBPsDnVtZRqrhYWLYW997/AZWa6G361LVZPUTzHvD/AMeVVhCRkSISKyKxiYmJFsdRyrNt2n+SJpGhiIjdUZQHsKzBi0hf4LgxZu3V1jPGjDfGtDTGtIyI0NO+lLqSlKRT/PDsjdyxaZ7dUZSHsHIPvj1ws4jEA5OBbiLyrYX1lPJq++f/QcmsDCLq1bQ7ivIQljV4Y8yzxphIY0wNYDCw0Bgz1Kp6Snm7s0tzb7JdtUcnm5MoT6HnwSvlIQLWrSWhbCXKRkXaHUV5CJc0eGPMYmNMX1fUUspbVdq5iSN1GtkdQ3kQnYtGKQ+QmJzOxJieNO2pd3BS+adDNEp5gLiEM3x83W2E3f4Pu6MoD6INXikPsH/lBspknKVRFb0gXOWfNnilPEDrt/7FtB9fJDhAR1VV/mmDV8rNZZ/LJGrPFk7EtLA7ivIw2uCVcnPxi1dSMjMd3w4d7I6iPIw2eKXcXNK8xQBU6dPd3iDK42iDV8rN+a1cwfHS5ajYqK7dUZSH0SM2Srm5/7YZRLuON/Gw3qJPFZD+j1HKjR1JTmdFQAQl+/a2O4ryQNrglXJju2Yv5pbNC2hZIcjuKMoDaYNXyo0FfD+JMfM+pkHVMnZHUR5IG7xSbix8Yyz7ohriH1jC7ijKA2mDV8pNpSSdIurQLs40b2V3FOWhtMEr5ab2/PgrfsZBqZ432B1FeSht8Eq5qVPLV5Pp60ft/j3sjqI8lJ4Hr5Sb+m/LW5nYqAdfhIXYHUV5KMv24EUkUERWi8hGEdkiIq9YVUspb5OcnsWWhGQaNq1tdxTlwawcojkHdDPGNAGaAr1EpK2F9ZTyGrsn/cT/pr9Op1JZdkdRHsyyIRpjjAFSnF/6Oz+MVfWU8iZZv8yi295YaBxldxTlwSw9yCoiviKyATgOzDfGrMpjnZEiEisisYmJiVbGUcpjRMSuYHftxgSWCrY7ivJgljZ4Y0yOMaYpEAm0FpHLbglvjBlvjGlpjGkZERFhZRylPMLpA0eolbCbs2072h1FeTiXnCZpjDkNLAJ6uaKeUp5s3/Q5AJTpo6dHqr/HyrNoIkQkzPl5EHADsN2qekp5i81HzrIhsgE1b+xqdxTl4aw8D74S8LWI+JL7h2SqMWaWhfWU8njGGMaFNabx69/waXCg3XGUh7PyLJo4oJlV21fKG+3ed5SkE2fp0r2O3VGUF9CpCpRyIyfe/oB1Hw6hS4Sv3VGUF9AGr5QbKbVoPkllK1ApqordUZQX0LlolNtK2h3P/um/krVxM/779pBifHhn6PME+ApDF3xLed8cAtu1pfbgmwkpH2533L8tJekUdXduZF3/YdSwO4zyCtrglVvZffwsM9Yn0ODlJ+mzchblgBzx4WiZCuyr1YjQIH8ys3MI27mVlpuX4T9lPOce92d90/YEPPUE0bf1QUTsfhmFsnPSTzR3ZFOq/012R1FeQhu8sl3GmRQ2vvE/Qr+fyKB+L3A2OIRHmrRlZZNowm/qRfWu11ElOJAqQIfzTxq5mIwzKeyas4gz302l7sJfmDP2Y146WZYXb4omJjLMvhdUSI4fpnM6KIR6/7jR7ijKS2iDV7Y5tT+B7S++Sb0fv6ZNajJ7qtThpeahdLilK+VD+lzz+YGlSxE9+CYYfBMZZ1LwX7Gb/bFJvPT8BJ5JWkPMpPEEechUu5nZDt5q0Is+rTpzp96eTxURPciqXC4+KZWxXy2hRJ2aXPfNBxyo3Zgt3/1MzQPbueWu3pQPKfj534GlSzGoZ1MWPNGZkQHHaTNnMscaxBC/ZLUFr6Dordx7glVlo6j0zzvtjqK8iO7BK5fZPn0ucdPn83TVrvj7+NBwyMPE3H0bTTsV3T1HSwf60/uzsWxq34JKD99HUI8uxH00gZh/Di6yGlY48skEOicJHevobB6q6EjurL7uoWXLliY2NtbuGKoI5WRlEzduIkHvv0v9vZs4GRzKN98tZEi36ELtqRfE8W17OHtDb6on7GHRx99zw/23WVqvsBzZOZwsU54D0c1pvmq+3XGUhxGRtcaYlnkt0yEaZYnUc9nM+fRHjlSqTrNH7qb0qURWPvoSJQ7u59F+zS1v7gDlG9SiwoZVzOz3T+7fG8gnS/ZYXrMwtn4/k3IpJzEDb7E7ivIyOkSjitSxrbuY+ccOPjziT/DxFCaElWPt0y/QZPTdVC4R4PI8pcqV4eYfxrFo6ka+mLac+r9Mpsvbz7s8x9WkTfialIBgGt4/zO4oystog1d/W05WNpu/mIzjs89pvP4P6ldvQscxn3P3iNZEf3yX3fHw8/Xh3duasGjyh3SZ+SUr01Jo+/EbdscCIP30WaKXz2druxtoXbqU3XGUl9EGrwptX1Iqe155i5hvPqLJmSROlAxjzcAR1Hr+CT5q2sDueH/h5+tD16mfsLbLYdqOG8ua8hG0evlxu2Oxau4KGvoFEHzPcLujKC+kDV4VyOF1W9n/6Ve8Xft61p3IYuSWg5SvVpvDd79Bo/uHcl2Q+05x61cigMYLfmZTq840e/UpNpaPoMkoe4dFvk4LY9ezU1g6uLutOZR30gavrior4xy7Zswj+adfKP/nYmod3kUVIGZURW4c3J8bn/6QymU8576hAcGBRC35jX0t2uH32qvE9upNy5rlbMly+FAif2w/ysiudfHx09kjVdHT0yTVX5xLTWPf3D/YlJzDPN/yJK5Yy4xP7ifLx5ddtRpzpntPqt13F5XdbAimoE7uO8SIb9YSTxA/PtCO2uVdP/69YvB9VJ47E7+tW6hSqazL6yvvcLXTJC3bgxeRqsA3QAXAAOONMe9bVU8VXEZWDjuPnSVj/OeYdespvX0zNeO3UT8ni42Nb2DnsGdpd/11rG34NXX+cSPRFezZ07VC2ahIPnyoLIM+XMTaW4YT9uVblKsT5bL6GWdTqf/LFOLrN6OZNndlESuHaLKBJ4wx60QkBFgrIvONMVstrKkukZGVQ2Lcdk6uiyNj207M7t0EHojnQHBZHu08EoeBRePfoULKKQ5E1mb9TUMo0bkj3W+6gUG1qjm3EmPra7BKtfBgvupQhqqvzyahWxyB61dRqlwZl9Te9M5ntEpL5vAjD7ukniqeXDZEIyI/A/8zxlzxUj0dorm2jLOpnD1ynJTEUyRVqcHptCwCli6mxNbNmOPH8Tt2lBKJx0h3CPcNfoXTaVl89/1ztDsQB0CqfyDHIqpwsH5T1j79bxpUKk1D33Qi61YvtuPAGz/7nob3DWVrw9Y0WLMYf4sn+zIOB3ur1sMvJ4tqCXsRH73eUBWeLUM0lwSoQe79WVflsWwkMBKgWrVqly7Ol+Qz6ZjTpzHGYBwOHA4HGAeOkFAcgUGYjHQkMQljHJgcBxiDyckhO6ICjpIlMWfP4JuQkPu4I3cbGAcZVWvgCA7G5+Qp/A7udy53XPhIrxdNTlAwfseO4Ld7N47MTExm1oV/E67rzLkSwZTavpmwjWshKwuTnZ37b1YWsf3vIiOgBFX/XEjU6sX4pKXhm5GOb0Y6fhkZPP/gO6TmCHfM/pybV/1C8Ll0ArMzCQRK+/rT9cmfAPjv7PEM3LyQbPHhROlwksPKkV65Gn1jKlEhJJCMOi+zrXQJIppFE16zGjV9fKgJdC7cj9PrNLn3dtYcOEyrMU+xutdttFr4k6VNd+3nU2iZsJvYV96lujZ3ZSHLG7yIlAJ+BB41xpy5dLkxZjwwHnL34AtT45+Pfsq0Lx+77PGHb3qKX6I70y5+A99N+ddly0fc+hKLarWix84VjP/p35ctv23IWFZXbUT/LYt4b9Z/L1veZ/j7bKlQi6Hr5zBm3seXLX/83k+JL1uF+1b9wLOLv7ps+eig5pwsXZaHVq7hhtXzORcQyLkSgWQFBJFVIpDyJXzwLVUSn0YN2eV/DlOyFJQpg5Qti1+5snzTrxVhJQMoc09jkoMCCKkYQQU/Xyo4t9/mQiW9gfO1tHrtSVYcOED0lAl8MXEh/7zrekvqGGN4KbUyMSNe57WnR1lSQ6nzLB2iERF/YBYw1xjzzrXWL+wQzdTZa6g2fxaIIL4+gICPcLR1R1KrRhGcdIwqyxeCjy/monWSWrfnXMXKBB1PIHzd6ty9Nh8B8cFHhJOtriOrbDmCjh4mdPtmEB/EB/DxBR/hTPPWmJDSlDiWQKn9+/AtEYBPiQB8AvzxLRFATu26+AcH4p+WSkB6Kj4BAfiWCMA/MAC/EiXwKxGAj6/uwbkL43DwxmcLGL8vk9cHNGZIm8K9o7ya3+ISuP+79bxzWxNuaR5Z5NtXxc/Vhmgsa/CSe9+0r4GTxphH8/McHYNXdsvKcXDv12uI/mYcPQZ1p+kDRXch1LmUVA7WbcL0jgN5/Ls38NM/7qoI2DWbZHtgGNBNRDY4P/ReZMqt+fv68NHAhgzYt4p6o+9l+4x5Rbbt9Y+8QO0je+jds4U2d+USeqGTUnlI2nOAc63bUjo1mcPf/Uj9W/7ejTjil6ymUvcObGnekearFxRRSqV0PnilCqxcrWr4LlnC6ZCyVBs8gC3fzyz0tjLOpJBzx1BSA4OpPuWrogup1DVog1fqCio2qkPQn0s5Fl6Rr79fysyNCQXehjGGb175jOoJezj49v8Ij6pqQVKl8qaTjSl1FeXqRiFb4tg3dRNTv1/PqcV/cvu9NxEQfO1ZMx0Ow4szN/Otf11KTpzHHXfojJHKtXQPXqlrCC8bwqR/tmVUNWHwE0M4WDeGXXOWXPU5Z44lEdu+N3snz+T+zrUYMqSbi9Iq9X+0wSuVDwF+Pvy/UTey9d3PCDudSJ0+XVjfvDObJkwl81zWhfVO7jvEykdfIrN2XZqvms9DFbN4ulc9cs8aVsq19CwapQrozLEktjwzhujJXxCSkUqTJ34gNCKMh2Z+xODl0wHYVisG3/fepW5f3XNX1rLlQqfC0AavPEnGmRS2TPuVBVUacyQ5gzaLf6Z69lnK3zaAmt2v00nElEtog1dKKS+l58ErpVQxpA1eKaW8lDZ4pZTyUtrglVLKS2mDV0opL6UNXimlvJQ2eKWU8lLa4JVSyku51YVOIpII7C/k08sBSUUYp6horoLRXAWjuQrGG3NVN8ZE5LXArRr83yEisVe6mstOmqtgNFfBaK6CKW65dIhGKaW8lDZ4pZTyUt7U4MfbHeAKNFfBaK6C0VwFU6xyec0YvFJKqb/ypj14pZRSF9EGr5RSXsqrGryIvCYicSKyQUTmiUhluzMBiMhbIrLdme0nEQmzOxOAiPxDRLaIiENEbD11TER6icgOEdktIs/YmeViIjJBRI6LyGa7s1xMRKqKyCIR2er8GT5idyYAEQkUkdUistGZ6xW7M50nIr4isl5EZtmd5WIiEi8im5x9q0jveORVDR54yxgTY4xpCswCXrQ5z3nzgUbGmBhgJ/CszXnO2wzcAiy1M4SI+AIfAb2BaOB2EYm2M9NFvgJ62R0iD9nAE8aYaKAt8KCbfM/OAd2MMU2ApkAvEWlrb6QLHgG22R3iCroaY5oW9bnwXtXgjTFnLvqyJOAWR5CNMfOMMdnOL1cCkXbmOc8Ys80Ys8PuHEBrYLcxZq8xJhOYDPSzORMAxpilwEm7c1zKGHPEGLPO+flZchtXFXtTgcmV4vzS3/lh+++hiEQCfYDP7c7iSl7V4AFE5N8ichC4A/fZg7/Y3cCvdodwM1WAgxd9fQg3aFaeQkRqAM2AVTZHAS4MhWwAjgPzjTHukOs94P8BDptz5MUA80RkrYiMLMoNe1yDF5HfRWRzHh/9AIwxzxtjqgKTgIfcJZdznefJfWs9yZ1yKc8lIqWAH4FHL3kHaxtjTI5zmDQSaC0ijezMIyJ9gePGmLV25riKDsaY5uQOUT4oIp2KasN+RbUhVzHGXJ/PVScBc4CXLIxzwbVyichwoC/Q3bjw4oMCfL/sdBioetHXkc7H1FWIiD+5zX2SMWa63XkuZYw5LSKLyD2GYedB6vbAzSJyIxAIlBaRb40xQ23MdIEx5rDz3+Mi8hO5Q5ZFclzM4/bgr0ZE6lz0ZT9gu11ZLiYivch9e3izMSbN7jxuaA1QR0SiRCQAGAzMtDmTWxMRAb4Athlj3rE7z3kiEnH+LDERCQJuwObfQ2PMs8aYSGNMDXL/by10l+YuIiVFJOT850APivCPoVc1eGCsc/ghjtxvlFucOgb8DwgB5jtPhfrE7kAAIjJARA4B1wGzRWSuHTmcB6AfAuaSe7BwqjFmix1ZLiUi3wMrgHoickhE7rE7k1N7YBjQzfl/aoNzD9VulYBFzt/BNeSOwbvVaYlupgKwTEQ2AquB2caY34pq4zpVgVJKeSlv24NXSinlpA1eKaW8lDZ4pZTyUtrglVLKS2mDV0opL6UNXrklEUm59lpXfO5DzlkpjYiUu+hxEZEPnMviRKT5RcsqnZ9lUES6FNWMgyKyOD8zdTpnFCx3jXV+F5EyRZFLFQ/a4JU3+hO4Hth/yeO9gTrOj5HAuIuWPQ585pJ0hTcRGGV3COU5tMErt+bc637LeQHbJhEZ5HzcR0Q+ltx59ueLyBwRuRXAGLPeGBOfx+b6Ad84ZzxcCYSJSCXnsoHAZReYiEhrEVnhnEd8uYjUcz4+XERmOGvHO981PO5cb6WIlL1oM8OcFyJtFpHWzueHS+49C7aIyOeAXFRzhnPiqS2XTD41E7i9sN9LVfxog1fu7hZy5xVvQu5e+VvOpnwLUIPc+eOHkXs17rXkOWuliEQBp4wx5/J4znagozGmGbmzk75+0bJGzhytgH8Dac71VgB3XrResHPyrVHABOdjLwHLjDENgZ+Aahetf7cxpgXQEhgtIuEAxphTQInzXyt1LR432ZgqdjoA3xtjcoBjIrKE3IbaAZhmjHEAR52TWhVWJSDxCstCga+d8xwZcuc3P2+Rcy72syKSDPzifHwTEHPRet9D7tzyIlLaOVdLJ3L/OGCMmS0ipy5af7SIDHB+XpXcIaUTzq+PA5Uv+lqpK9I9eFWcXGnWynRyZxnMy2vkNvJGwE2XrHfxHr/joq8d/HXn6dL5QK44P4iIdCH3ncp1zrsirb+kZqAzr1LXpA1eubs/gEGSexOJCHL3fFeTeyB1oHMsvgLQJR/bmgnc6RzXbwskG2OOkHsbxRpXeE4o/zd18fBCvobzxw06OGsmkzsd7BDn472B82fHhJI7XJQmIvXJvR0fzvUEqAjEFzKHKma0wSt39xMQB2wEFgL/zxhzlNx50A8BW4FvgXVAMoCIjHbOkhkJxDkPYkLu/QH2ArvJPWNmFIAxJhXYIyK186j/H+ANEVlP4Yc0M5zP/wQ4PxvlK0AnEdlC7lDNAefjvwF+IrINGEvuLR7PawGsvOj2j0pdlc4mqTyWiJQyxqQ4DzquBto7m39htjUAaGGM+VeRhixCIvI+MNMYs8DuLMoz6EFW5clmOQ9YBgCvFba5AxhjfvKAs1M2a3NXBaF78Eop5aV0DF4ppbyUNnillPJS2uCVUspLaYNXSikvpQ1eKaW81P8HIT4w0H032FcAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_71_0.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from sklearn.model_selection import KFold\n", "from sklearn.linear_model import Ridge\n", "from sklearn.model_selection import cross_val_score\n", "from sklearn.preprocessing import PolynomialFeatures\n", "\n", "# A seed just to ensure that the random numbers are the same for every run.\n", "# Useful for eventual debugging.\n", "np.random.seed(3155)\n", "\n", "# Generate the data.\n", "nsamples = 100\n", "x = np.random.randn(nsamples)\n", "y = 3*x**2 + np.random.randn(nsamples)\n", "\n", "## Cross-validation on Ridge regression using KFold only\n", "\n", "# Decide degree on polynomial to fit\n", "poly = PolynomialFeatures(degree = 6)\n", "\n", "# Decide which values of lambda to use\n", "nlambdas = 500\n", "lambdas = np.logspace(-3, 5, nlambdas)\n", "\n", "# Initialize a KFold instance\n", "k = 5\n", "kfold = KFold(n_splits = k)\n", "\n", "# Perform the cross-validation to estimate MSE\n", "scores_KFold = np.zeros((nlambdas, k))\n", "\n", "i = 0\n", "for lmb in lambdas:\n", " ridge = Ridge(alpha = lmb)\n", " j = 0\n", " for train_inds, test_inds in kfold.split(x):\n", " xtrain = x[train_inds]\n", " ytrain = y[train_inds]\n", "\n", " xtest = x[test_inds]\n", " ytest = y[test_inds]\n", "\n", " Xtrain = poly.fit_transform(xtrain[:, np.newaxis])\n", " ridge.fit(Xtrain, ytrain[:, np.newaxis])\n", "\n", " Xtest = poly.fit_transform(xtest[:, np.newaxis])\n", " ypred = ridge.predict(Xtest)\n", "\n", " scores_KFold[i,j] = np.sum((ypred - ytest[:, np.newaxis])**2)/np.size(ypred)\n", "\n", " j += 1\n", " i += 1\n", "\n", "\n", "estimated_mse_KFold = np.mean(scores_KFold, axis = 1)\n", "\n", "## Cross-validation using cross_val_score from sklearn along with KFold\n", "\n", "# kfold is an instance initialized above as:\n", "# kfold = KFold(n_splits = k)\n", "\n", "estimated_mse_sklearn = np.zeros(nlambdas)\n", "i = 0\n", "for lmb in lambdas:\n", " ridge = Ridge(alpha = lmb)\n", "\n", " X = poly.fit_transform(x[:, np.newaxis])\n", " estimated_mse_folds = cross_val_score(ridge, X, y[:, np.newaxis], scoring='neg_mean_squared_error', cv=kfold)\n", "\n", " # cross_val_score return an array containing the estimated negative mse for every fold.\n", " # we have to the the mean of every array in order to get an estimate of the mse of the model\n", " estimated_mse_sklearn[i] = np.mean(-estimated_mse_folds)\n", "\n", " i += 1\n", "\n", "## Plot and compare the slightly different ways to perform cross-validation\n", "\n", "plt.figure()\n", "\n", "plt.plot(np.log10(lambdas), estimated_mse_sklearn, label = 'cross_val_score')\n", "plt.plot(np.log10(lambdas), estimated_mse_KFold, 'r--', label = 'KFold')\n", "\n", "plt.xlabel('log10(lambda)')\n", "plt.ylabel('mse')\n", "\n", "plt.legend()\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "More examples of the application of cross-validation follow here." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ ":63: RuntimeWarning: divide by zero encountered in log10\n", " plt.plot(polynomial, np.log10(estimated_mse_sklearn), label='Test Error')\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEGCAYAAABsLkJ6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAvfklEQVR4nO3dd3wUdfrA8c+TTkIgBEJNIPTeA1JFROwFzwK2U9RDPcud3unp6et3nnfe2U49xHKoiFiQIvYKCtKFUAOhCCSQBFJISAiB1P3+/tgBY0yyKbuZbPZ5v1682Jmd2XmG1Xl2vt/5Pl8xxqCUUsr3+NkdgFJKKXtoAlBKKR+lCUAppXyUJgCllPJRmgCUUspHBdgdQE20adPGxMbG2h2GUkp5lU2bNh01xkRV9b5XJIDY2Fji4+PtDkMppbyKiBys7n1tAlJKKR+lCUAppXyUJgCllPJRXtEHUJmSkhJSU1MpLCy0OxSvFhISQnR0NIGBgXaHopRqYF6bAFJTUwkPDyc2NhYRsTscr2SMITs7m9TUVLp27Wp3OEqpBua1TUCFhYW0bt1aL/71ICK0bt1a76KU8lFemwAAvfi7gf4bKuW7vDoBKKVUU5VTUMzfP9vJ8cISjx3Da/sA7Jadnc2kSZMASE9Px9/fn6go54C7DRs2EBQUVO3+K1asICgoiDFjxvzqvblz5/Lggw/SqVOnM+vef/99+vXr58YzUEo1RsYYlmxO459fJJJfWMrY7m04r187jxxLE0AdtW7dmq1btwLw+OOP07x5c/785z/XeP8VK1bQvHnzShMAwNSpU5k1a1aV+5eWlhIQEFDlck33U0o1HklHC3js4wTW7MtmeJdW/OvKgfRuH+6x42kTkBtt2rSJCRMmMHz4cC644AKOHDkCwMyZM+nXrx+DBg1i2rRpJCcn89prr/HCCy8wZMgQVq1aVaPPX7FiBePHj+fyyy+nX79+v1ouLCxk+vTpDBw4kKFDh7J8+XLAeUdx+eWXc+655565a1FKNR7FpQ5e+u4nLnhxJdtT8/jnlAEsumO0Ry/+0ETuAP7+2U4SDx9362f269iCv13Wv8bbG2O49957+eSTT4iKimLBggU8+uijzJkzh6eeeoqkpCSCg4PJzc0lIiKCO++8s9q7hgULFrB69eozy+vWrQNg8+bN7Nixg65du7JixYpfLP/nP/9BREhISGD37t2cf/757N2798x+27dvJzIysh7/Kkopd9uYnMNflyTwU+YJLhnUgb9d2o+2LUIa5NhNIgE0BkVFRezYsYPJkycDUFZWRocOHQAYNGgQN9xwA1OmTGHKlCk1+ryqmoBGjhz5i2f2yy+vXr2ae++9F4A+ffrQpUuXMwlg8uTJevFXqhHJO1nCU1/vZv6GQ3SKaMZbt4xgYp+2DRpDk0gAtfml7inGGPr373/ml3p5X3zxBStXruSzzz7jySefJCEhoc7HCQsLq3a5pvsppexhjOHz7Uf4+2eJ5BQU8bvxXbl/ci9Cgxr+cqx9AG4SHBxMVlbWmQRQUlLCzp07cTgcpKSkMHHiRJ5++mny8vI4ceIE4eHh5OfnuzWG8ePH89577wGwd+9eDh06RO/evd16DKVU3RWVlnH72/HcO38LHSNC+PSecTx6ST9bLv6gCcBt/Pz8WLx4MX/5y18YPHgwQ4YMYe3atZSVlXHjjTee6Zi97777iIiI4LLLLuOjjz6qshN4wYIFDBky5MyftWvXuozh97//PQ6Hg4EDBzJ16lTmzp1LcHCwJ05XKVUHLy77ie92Z/LoxX356PdjGdCppa3xiDHG1gBqIi4uzlScEGbXrl307dvXpoiaFv23VMrztqbk8ptX1nDN8BievnpQgxxTRDYZY+Kqel/vAJRSysMKS8r408KttG8RwqOXNp4fW02iE1gppRqzF5btZX9WAfNuHUmLkMZTet2r7wC8ofmqsdN/Q6U8a9PBY7y+8gDXjezM2b2qnJ/dFl6bAEJCQsjOztYLWD2cng8gJKRhBp0o5WsKS8p4cNE2OrRsxqOXNJ6mn9O8tgkoOjqa1NRUsrKy7A7Fq52eEUwp5X7/+XYPB44W8N7tZ9E8uPFdbhtfRDUUGBios1gppRqt+OQc3lidxI2jOjO2Rxu7w6mU1zYBKaVUY3WquIwHF2+nU0QzHrmo8TX9nOa1dwBKKdVYPfvNHpKOFvD+784irBE2/ZymdwBK1VFJmYONyTn6IEITY4xhaWIGl89azbTZ61i772it9t+QlMNba5P47egujOneOJt+TtMEoFQdvbf+INe8to4HFm6jsKTM7nCUG8Qn53DNa+v43bx48gtLST56kuvf+JGp/1vHjweyXe5/sriUBxdvI6ZVKH+5sE8DRFw/jffeRKlGbktKLsEBfny0JY0DWSf4301xtG+pj9R6o32Z+Tz99R6WJmYQFR7Mk1cO4Nq4GMochg82HOLlFfuZOns9Y3u05v7zehEXW3lp9We+3sPB7JN8MGNUo276Oc1rawEpZbdzn1tBj7bNuXp4NPcv2EpYcAD/u2k4Qzu3sjs0VUPpeYW8uGwvC+NTCA0K4M4J3bh1XNdfVecsLCnj3fUHee2H/Rw9Ucz4nm24f3IvhpX7rtcfyGba7PXcMiaWxy+3v0Q92FgLSETmiEimiOyo5L0/iYgRkcbdQKZUFY4XlnDgaAGDoltyfv/2LPn9WEIC/Zk6ez0fbkq1O7wGk19YwvPf7uFI3im7Q6mVvFMlPP31biY8u5wPN6dyy5iu/PDgOdxzbs9KSzOHBPpz+/hurHxoIn+9uA87Dx/nN6+sZfpbG9iemktBkbPpp0vrUB660HtKsHvyHmUuMAuYV36liMQA5wOHPHhspTxqR1oeAAOjIwDo3T6cT+4ey93vb+ZPi7axO/04D1/UF38/sTFKzzLG8PCHCXyRcITv92Sy+M4xhAT62x1WtU4UlTL/x0PMWr6P44UlTBnSiQcm9yImMrRG+4cGBTDj7O7ccFYX3l6XzOyVB7h81ho6R4aSeuwUC2aMtq22f114LFJjzEoRia3krReAh4BPPHVspTwtIdVKAOXqubcKC+LtW0fy5Be7eH1VEnszTjDzuqG0bNZ4in+509trk/ki4QgXD2zPVzvSeWRJAs9fOxgR+5JeSZmDw7mnSMk5Rcqxk6TknCTl2ClSck6SeuwkR08UAzChVxQPXdib/h3rVo8/LDiA35/Tg5tGdeHttcm8viqJOyd0Z2RX75p2tUFTlYhcAaQZY7a5+o9ERGYAMwA6d+7cANEpVXPb0/KIiWxGZFjQL9YH+vvx+OX96d0+nP/7ZAdXvryG12+Oo3tUc5si9Ywth47x5Je7mNSnLbOuG8as5ft4fuleBnRqyW3jGnaE/tc70pm7NomUnFMcyTuFo1y3ZoCf0DGiGTGRzTivbztiIkMZERvptgt1eEgg95zbk7sn9nDL5zW0BksAIhIK/BVn849LxpjZwGxwdgJ7MDSlam17ai6DOkVU+f51IzvTo21z7nxnE1NeXsPM64YysXfDTvjtKccKirnn/S20DQ/hP9cOxs9PuGdiD3YezuNfX+6ib/twxjRQ6YOPtqTyp4XbiG0dxojYVnSO7ER0ZCgxrUKJiWxG+xYhBPh7/ml3O+966qMhxwF0B7oC20QkGYgGNotI+waMQal6O1ZQTErOKQZGV998MCI2kk/vHUdMq1Bum7uR//2w3+sHjTkchgcWbiUrv4hXbhhGRKjzDsjPT/jPtUPo1iaMu9/fTErOSY/HsnhTKg8s3Maobq35/L5xvDhtKA+c35tr42IY3b010a1CG+Ti780a7F/HGJNgjGlrjIk1xsQCqcAwY0x6Q8WglDskWB3Ag2own2uniGYsvms0Fw3owL+/2s39C7a6ddDYyeJS8k6VuO3zXHlt5X6W78nisUv7Mjgm4hfvNQ8OYPZv4yh1GO54ZxOnij03OG7BxkM8uHgb43q04c2bR3hVx2tj4snHQOcD64DeIpIqIrd56lhKNaTTCaB/DSf0Dg0KYNb1Q/nz+b34eOthrv3fOrc8Nvn59sOM/vf3DPvHUq6bvZ45q5M8+st7/YFsnvtmD5cN7shNo7pUuk3XNmHMnDaUXenHeXjJdo/c8bz340H+8mECZ/eM4vXfxtEsqHE/edSY6UAwpWrpjnfi2ZtxguV/PqfW+y5NzOD+BVsJCfTnfzcNY3iX2ndG5p0q4W+f7ODjrYcZHBPB2O6tWbYrg70ZJwDo26EFk/u14/x+7ejfsYVb2qcz8wu5ZOZqwoMD+PTecS5r27+8fB/PfrOHRy/uy+/O7lbv4582b10y//fJTs7t05ZXbxxGcIBe/KvjaiCYJgClamnMv78jLjaSmdcNrdP+P2Xk87t58aTlnuKfUwYwdUTNn3Jbu+8of1q0jcz8Iu47tyd3T+x+pp07+WgBSxMzWJqYQfzBHBwGOrYMYXK/dkzu156zukUSWIc28TKH4cY3fmRLyjE+vnssfdq3cLmPMYbfv7eZb3amM+/WsxjXs/6dwnNWJ/HE54lM7teOWdcP1Yt/DWgCUMqNsvKLGPHkMh67pC+3j6/7L9u8kyXcM38zq346ys2ju/DYpf2qvTgXlpTx7Dd7eHN1Et3ahPHC1CG/aoMvL/tEEd/tzmRpYgarfsqisMRBeEgAlwzswI2jujCghs1XAM99s4dZy/fx7NWDuCYupsb7FRSVcuUra8jML+Kze8bVeLBVZV5feYAnv9zFhf3bM/O6oQQFaOduTWgCUMqNlu/OZPrcjSyYMYqzurWu12eVljl4+uvdvL4qiVHdInnlhuG/GlcAsPNwHvcv2MrejBP8dnQXHrmob63avU8Vl7F631G+3pHOFwmHKSxxMCQmghtHdeHSQR2qHb27fE8m09/ayLVx0Txz9eBan2Py0QIun7WaTq1CWXLXmDq117+6Yj9Pf72bSwZ14MWpQ+p0F+OrNAEo5Ub/XfYTL363l4THL3DbHK9LNqfy8JIE2oYHM/umOPp1dDaxlDkMs1ce4Pmle2gVGsQzVw/inHqOJcg7VcKSzam8u/4g+7MKiAgN5Oph0dwwqgtd24T9YtvDuae4eOYq2rcI4eO7x9a5zMOKPc6keemgjsycNqRWfRKzvv+J577dyxVDOvKfawbrY521pAlAKTe6/e2NJGefZNkDE9z6udtScpnxTjzHT5Xy/LWDGdCpJQ8s3MrG5GNcPLA9T04ZSKtK7g7qyhjDugPZvLf+EN/sTKfUYRjfsw03nNWF8/q2xWFg6ux1/JRxgs/uHfer5FBbr6zYxzNf7+GhC3tz/cjOiAh+ws9/I4iACPiJIMCs5ft4cdlP/GZoJ569ZnCTrqvkKZoAlHKjkU8uY1yPNjw/dYjbPzvzeCF3vruJzYdyCQn0I9DPjyem9GfKkE4eHWmaebyQBRtTeH/DIY7kFdK+RQjd24axZl82r9wwjIsHdqj3MYwx3PP+Fr5IOFKr/a4ZHs1TVw3Si38duUoAOnpCqRrKOF5IZn6RyxHAddW2RQjzZ4zin5/v4nDuKZ6YMoBOEc08cqyKx713Uk/uOqc73+/O5N0fD7FybxbTx8a65eIPzl/6z10zmAm9ojhRVIrD+uHpMAZjwGHA4HxtjMFhoF2LYK4ZHoOfXvw9RhOAUjW03aoAOshDCQAgOMCff0wZ4LHPr06Avx/n92/P+f3bc6ygmIhQ91YxbRbkz7Ujav4UkfI8TQBK1VBCai5+Av06eC4BNBbu7G9QjZd2qStVQ9vT8ujVLlxLD6gmQxOAUjVgjCEhNe8XE8Ao5e00AShVA4fzCskuKPZo+79SDU0TgFI1sD0lF/h5DmClmgJNAErVwPa0PAL9hb4dwu0ORSm30QSgVA0kpObRu324VqBUTYomAKVcMMawPTWXgdXMAayUN9IEoJQLh3JOcrywVDuAVZOjCUApF06PANZHQFVTowlAKRcS0vIICvCjVzvtAFZNiyYApVzYnppL3w4tdBYq1eTof9FKVcPhMOxIO84gbf5RTZAmAKWqkZRdwImiUo+VgFbKTpoAlKpGQgOUgFbKLpoAlKrG9tQ8QgL96BHV3O5QlHI7TQBKVSMhLZf+HVvqZOSqSdL/qpWqQpnVAazP/6umShOAUlXYn3WCUyVl2v6vmiyPJQARmSMimSKyo9y6Z0Vkt4hsF5GPRCTCU8dXqr4aYg5gpezkyTuAucCFFdYtBQYYYwYBe4FHPHh8peolITWXsCB/urbRDmDVNHksARhjVgI5FdZ9a4wptRbXA9GeOr5S9bU9LY/+nVri7yd2h6KUR9jZB3Ar8FVVb4rIDBGJF5H4rKysBgxLKSgpc5B4WEcAq6bNlgQgIo8CpcB7VW1jjJltjIkzxsRFRUU1XHBKAXsz8ikqdTAoJsLuUJTymICGPqCI3AJcCkwyxpiGPr5SNXFmBLDeAagmrEETgIhcCDwETDDGnGzIYytVG9vT8ggPCaBL61C7Q1HKYzz5GOh8YB3QW0RSReQ2YBYQDiwVka0i8pqnjq9UfSSk5jEouiUi2gGsmi6P3QEYY66rZPWbnjqeUtXZl5lPTGRojSZ1LyotY3f6cW4b160BIlPKPg3eB6BUQ9ty6BhXvrKW9i1CuHNCN6aN7ExIYNWJYE96PiVlRgeAqSZPS0GoJm/OmmTCgwPo3DqUxz9L5OxnlvPm6iROFZdVur3OAax8hSYA1aSl5xXyVcIRpo6IYeEdo/lgxii6RzXnH58nMv6Z75m9cj8ni0t/sU9Cah6tQgOJbtXMpqiVahiaAFST9u76g5QZw81jYgEY1a0182eMYuEdo+nboQX/+nI3455ezqsr9nOiyJkItqflMTA6QjuAVZOnfQCqySosKeP9DYc4r287YiJ/+TjnyK6RvHPbWWw6eIyXvv+Jp7/ezf9W7mf6mK7szchnUp+2NkWtVMPRBKCarE+3HianoJjpY2Or3GZ4l1bMnT6SbSm5zPzuJ15YtheAAdr+r3yAJgDVJBljmLMmiT7twxndrbXL7QfHRPDmLSPYkZbH8t2ZTOyj5UdU01dtAhCRyBp8hsMYk+uecJRyj/UHctidns/TVw2sVVv+gE4t9de/8hmu7gAOW3+q+z/IH+jstoiUcoO5a5NoFRrIFUM62R2KUo2WqwSwyxgztLoNRGSLG+NRqt5Sck6yNDGDOyd0r3bAl1K+ztVjoKNr8Bk12UapBjNvXTIiwk2ju9gdilKNmqsEMOb0CxHpWv4NEfkNgDGm0ANxKVUnBUWlfLAxhYsGtKdDSx3IpVR1XCWA58q9/rDCe4+5ORal6m3JljTyC0uZPrar642V8nGuEoBU8bqyZaVs5XAY5q5JYlB0S4Z1jrA7HKUaPVcJwFTxurJlpWy1at9R9mcVMH1srJZxUKoGXD0F1E1EPsX5a//0a6xlvcdWjcpba5KICg/mkoEd7Q5FKa/gKgFcUe71cxXeq7islG32Z51gxZ4s7j+vF0EBWuNQqZqoNgEYY34ovywigcAAIM0Yk+nJwJSqjXlrkwny9+P6s3RMolI1Ve1PJRF5TUT6W69bAtuAecAWEalsykelGtzxwhIWb0rl0sEdiAoPtjscpbyGq3vl8caYndbr6cBeY8xAYDjwkEcjU6qGFm5MoaC4jFv10U+lasVVAigu93oy8DGAMSbdUwEpVRtlDsPb65IZEdtKi7gpVUuuEkCuiFwqIkOBscDXACISAOgwS2W773ZlkJJzSgd+KVUHrp4CugOYCbQH/ljul/8k4AtPBqZUTcxdm0zHliGc36+d3aEo5XVcPQW0F7iwkvXfAN94KiilamJ3+nHW7s/mLxf2IcBfH/1UqrZcTQgzs7r3jTH3uTccpWpu7ppkQgL9uG5kjN2hKOWVXDUB3QnsABbiemIYpRpM3qkSPtqSxm+GdSIiNMjucJTySq4SQAfgGmAqUAosABbrFJDKbp9tO0xRqYPrR2rNf6XqqtqGU2NMtjHmNWPMRJzjACKARBG5ydUHi8gcEckUkR3l1kWKyFIR+cn6u1V9T0D5psWbUundLpwBnVrYHYpSXqtGPWciMgz4A3Aj8BWwqQa7zeXXHcgPA98ZY3oC31nLStXKvsx8tqbkck1ctFb9VKoeXHUCPwFcAuwCPgAeMcaU1uSDjTErRSS2wuorgHOs128DK4C/1DxcpWBRfCoBfsKUoTrhu1L14aoP4DEgCRhs/fmX9YtLAGOMGVTL47UzxhyxXqcDVT68LSIzgBkAnTtrgS/lVFrmYMmWNM7p3ZY2zbXuj1L14SoBeGx4pTHGiEiVk8oYY2YDswHi4uJ08hkFwMqfssjKL+KauGi7Q1HK67kaCHbQzcfLEJEOxpgjItIB0JLSqlYWb0olMiyIib3b2h2KUl7PVTnoz119QE22KedT4Gbr9c3AJ7XYV/m4YwXFLEvMZMqQTjrpi1Ju4KoJaFy5aSArI0C/St8QmY+zw7eNiKQCfwOeAhaKyG3AQeDaWkesfNan2w5TXObQ5h+l3KQ2U0JWpbiylcaYqiaMmVSDz1TqVxZtSqF/xxb07aDP/ivlDjWaElJE2gGnn7lLM8ZkeDowpcrbdeQ4O9KO8/hlld5wKqXqwNU4gCHAa0BLIM1aHS0iucBdxpgtHo1OKcviTakE+guXD9Fn/5VyF1dNQHOBO4wxP5ZfKSKjrPcGeyYspX5WUubg4y1pnNe3HZFhWvhNKXdx9ShFWMWLP4AxZj0Q5pmQlPql5bszyS4o1s5fpdzM1R3AVyLyBTAPSLHWxQC/xZoeUilPW7QplajwYM7uGWV3KEo1Ka46ge8TkYtwPg10phMYeNkY86Wng1Pq6Ikilu/O5LZxXXXWL6XczNUdAMaYr3BWAFWqwX28JY1Sh+Hq4dr8o5S71fknlYjMdmcgSlVkjGHxplQGx0TQs1243eEo1eS4egw0sqq3gIvdH45SP9t5+Di70/P555QBdoeiVJPkqgkoC2fJhvKzbhhrWatxKY9aFJ9CUIAflw3qaHcoSjVJrhLAAWCSMeZQxTdEJKWS7ZVyi6LSMj7ZdpgL+renZWig3eEo1SS56gN4Eahq3t5n3BuKUj/7blcmuSdLtPNXKQ9y9Rjoy9W895L7w1HKaVF8Ch1ahjCuRxu7Q1GqyXL5GCiAiPymktV5QIIxRid1UW6VebyQH/Zmcdc53fH300nflfKUGiUA4DZgNLDcWj4H2AR0FZEnjDHveCA25aOWbEnDYeCqYdr8o5Qn1TQBBAB9T5eBtspDzwPOAlYCmgCUWxhjWBSfQlyXVnSLam53OEo1aTUdCBZTYQ6ATGtdDlDi/rCUr9qaksv+rALt/FWqAdT0DmCFNffvImv5amtdGJDricCUb1q0KZWQQD8uGdTB7lCUavJqmgDuBn4DjLOW3wY+NMYYYKInAlO+xeEwvLJiH/M3HOLqYdGEh+iz/0p5Wo0SgDHGiMhqnPP/GmCDdfFXqt7yTpXwp4VbWbYrk8sHd+TvV/S3OySlfEJNHwO9FngWWIGzDMRLIvKgMWaxB2NTPmDXkePc+e4m0o6d4vHL+nHzmFhE9NFPpRpCTZuAHgVGnH7mX0SigGWAJgBVZx9tSeWRJQm0bBbIBzNGERdbVe1BpZQn1DQB+FUY8JVNPUpJK99WXOrgH58n8s76g5zVNZKXrh9K2/AQu8NSyufUNAF8LSLfAPOt5amAzgimau1I3il+/95mthzKZcbZ3Xjogt4605dSNqlpJ/CDInIVMNZaNdsY85HnwlJN0dr9R7n3/S0UlpTxyg3DuHigPuqplJ1qegeAMeZD4EMPxqKaKGMM/1t5gGe+3k23qOa8duNwerTVUb5K2c3VjGD5OB/7/NVbOJ8ObVGXg4rI/cDt1mcnANONMYV1+SzV+P3j813MWZPEJQM78PTVg2geXOPfHUopD3JVDtrtE7GKSCfgPqCfMeaUiCwEpgFz3X0sZb/CkjI+2HiIywd35L/Thugjnko1Inb1vgUAzUQkAAgFDtsUh/KwdQeyOVlcxpVDO+nFX6lGpsETgDEmDXgOOAQcAfKMMd9W3E5EZohIvIjEZ2VlNXSYyk2WJmYQGuTP6O6t7Q5FKVVBgycAEWkFXAF0BToCYSJyY8XtjDGzjTFxxpi4qKiohg5TuYHDYViWmMGEXlGEBPrbHY5SqgI7moDOA5KMMVnGmBJgCTDGhjiUhyWk5ZGZX8R5fdvZHYpSqhJ2JIBDwCgRCRVno/AkYJcNcSgPW5qYgb+fcG6ftnaHopSqhB19AD/irCG0GecjoH7A7IaOQ3ne0sQM4rq0olVYkN2hKKUqYctTQMaYvxlj+hhjBhhjbjLGFNkRh/KcQ9kn2ZORz+R+2vyjVGOlRViURyzd5ZxBVBOAUo2XJgDlEUsT0+nVrjldWofZHYpSqgqaAJTb5Z4sZmPyMf31r1QjpwlAud3yPZmUOYw+/qlUI6cJQLnd0sQM2oYHMzg6wu5QlFLV0ASg3KqotIwf9mQxqW87/Py09o9SjZkmAOVW6/ZnU1BcxuR+OvhLqcZOE4Byq9PF38Z0b2N3KEopFzQBKLdxOAzLdmVwdk8t/qaUN9AEoNxmx+E8Mo4XcZ4+/qmUV9AEoNxmaWIGfoIWf1PKS2gCUG6zNDGDuNhIIrX4m1JeQROAcouUnJPsTs/nfG3+UcpraAJQbrE00Vn8TUf/KuU9NAEot1iamEHPts2JbaPF35TyFpoAVL3lnSxhQ3KOFn9TystoAlD1dqb4myYApbyKJgBVb0sTM4gKD2aIFn9TyqtoAlD1UlRaxoo9mZzXt60Wf1PKy2gCUPWy/kAOBcVl+vSPUl5IE4Cql6WJ6TQL9GdsDy3+ppS30QSg6swYw7LETM7u1UaLvynlhTQBqDrbkXac9OOF2vyjlJfSBKDqbGliOn4CkzQBKOWVNAGoOvs2MYO4Llr8TSlvpQlA1cnp4m86+lcp7xVgdwCq8UhIzWP2qgMIEOAn+PkJAX6Cf7k/p9cfyCoA0NG/SnkxWxKAiEQAbwADAAPcaoxZZ0cs6mdPfpnItpQ82rcModThoKzMUGYMZQ5DqcP59+nXDofhrK6RdNXib0p5LbvuAP4LfG2MuVpEgoBQm+JQlp2H81h/IIdHLurDHRO62x2OUqoBNHgCEJGWwNnALQDGmGKguKHjUL80Z3UyoUH+TBvZ2e5QlFINxI5O4K5AFvCWiGwRkTdE5FftCCIyQ0TiRSQ+Kyur4aP0IZn5hXy27TDXDI+mZbNAu8NRSjUQOxJAADAMeNUYMxQoAB6uuJExZrYxJs4YExcVFdXQMfqUd9cdpMTh4JaxXe0ORSnVgOzoA0gFUo0xP1rLi6kkAbjD+gPZHMo5SWzrMGJbhxIVHoyIVqwsr7CkjHd/PMSkPm21Q1cpH9PgCcAYky4iKSLS2xizB5gEJHriWEs2p7IwPvXMcmiQP12sZBDbxvl3l9ZhdG0TRlsfTQ6fbE0jp6CYW8fpr3+lfI1dTwHdC7xnPQF0AJjuiYM8eeVA7p7Yg6SjBRzMPklydgHJRwvYk57P0sQMSh3mzLbNAv0Z1S2Shy/qS+/24Z4Ip9ExxvDm6iT6dmjB6G6t7Q5HKdXAbEkAxpitQJynjxPo70eX1mF0af3rpo3SMgdH8gqt5FDA/qwClmxO5eKZq7jhrM7cf14vWjXxEger9x1lb8YJnrtmsE/e/Sjl63x2JHCAvx8xkaHERIYCzk7m+yb15IWle3l3/UE+2XqYP57XkxtHdSHQv2lWzHhzdRJtmgdz2eAOdoeilLJB07yy1VFkWBD/mDKAr/5wNgM7teTvnyVy0X9X8cPepvcY6r7MfFbsyeKmUV0IDtBa/kr5Ik0AlejdPpx3bhvJ67+No7TMwc1zNnDr3I3szzphd2hu89aaZIIC/LhhlA78UspXaQKogogwuV87vrn/bB65qA8bknK44IWV/OPzRPJOldgdXr0cKyjmw82pXDmkE22aB9sdjlLKJpoAXAgO8OeOCd1Z/udzuHp4NHPWJDHxuRV8uzPd7tDq7P0Nhygsceijn0r5OE0ANRQVHsxTVw3is3vG0TEihPs+2MJPGfl2h1VrxaUO5q1LZlyPNj7zuKtSqnKaAGppQKeWzLl5BGFBAdzz/hZOFZfZHVKtfJlwhIzjRdymv/6V8nmaAOqgbYsQnp86hD0Z+Tzx+U67w6kxYwxz1iTRLSqMCb20vpJSvk4TQB1N6BXFXed0Z/6GFD7ddtjucGok/uAxtqfmcevYrvj56cAvpXydJoB6eGByL4Z1juCvSxJIPlpgdzguvbkqiZbNArlqWLTdoSilGgFNAPUQ6O/HzOuG4u8n3Dt/C0WlDdMfYIxhR1perY6XknOSbxPTuf6szjQL0oFfSilNAPUW3SqUZ64eREJaHk99tbtBjvnaDwe49KXVjP739/zz80T2ZboeoPbWmmT8RLh5dKznA1RKeQWfrQXkThf0b88tY2J5a00yY7q3YXK/dh471vI9mTzzzW7O7dOW4AA/5q5N5o3VSYyIbcW0EZ25eGCHX/3Czy8sYWF8CpcM6kD7liEei00p5V00AbjJIxf3If5gDn9etI0v/zCeThHN3H6MA1knuG/+Fvq2b8HL1w+jWZA/WflFfLg5lQUbU/jTom08/tlOrhzaiWkjOtOvYwsAFsancqKoVB/9VEr9ghhjXG9ls7i4OBMfH293GC4lHy3g0pdW07t9OB/MGOXWKqL5hSVMeXkNx06W8Ok9Y4luFfqL940xrD+QwwcbD/HVjnSKSx0Mjm7JtJGdeWXFPtqFh7D4rjFui0cp1fiJyCZjTJWl97UPwI1i24Tx5JUD2HTwGC8s3eu2z3U4DPcv2Epy9klevn7Yry7+4KxdNLp7a/47bSgb/jqJ/7u0H6dKynhkSQIpOaf0179S6le0CcjNrhjSiXX7s3n1h/2M6taas90w4OqFZXtZtiuTJ67oz+jurmfuiggN4tZxXZk+NpYtKbkkHj7O+f3b1zsOpVTToncAHvC3y/rTs21zHli4lcz8wnp91lcJR3jp+31MjYvhplFdarWviDCscytuHNUFfx34pZSqQBOABzQL8mfW9cM4UVTKHz/YSpmjbv0su9OP86dF2xjWOYInpvTXaRuVUm6lCcBDerUL5++X92ft/mxuf3sjW1Nya7X/sYJifjcvnvCQAF67cbjO2qWUcjvtA/Cga+NiOHayhFdX7GfKy2sY3a01d53TnfE921T7a760zME98zeTkVfEgjtG0baFPruvlHI/vQPwIBHhzgndWfvwuTx2SV+Sjhbw2zkbuGzWar7YfqTKpqF/f7WbNfuyefLKAQzt3KqBo1ZK+QpNAA0gLDiA28d344eHzuGZqwZxsqiMu9/fzHnP/8AHGw79oqbPh5tSeXN1EreMieWauBgbo1ZKNXU6EMwGZQ7DtzvTeWXFfhLS8mgbHszt47syoGNLbpm7keGdWzHvtpFuHUimlPI9rgaCaR+ADfz9hIsGduDCAe1Zsy+bV3/Yx7++dBaSi27VjJdvGKYXf6WUx2kCsJGIMK5nG8b1bMO2lFwWb0rlptFdiAwLsjs0pZQP0ATQSAyOiWBwTITdYSilfIht7Qwi4i8iW0Tkc7tiUEopX2ZnQ/MfgF02Hl8ppXyaLQlARKKBS4A37Di+Ukop++4AXgQeAhxVbSAiM0QkXkTis7KyGiwwpZTyFQ2eAETkUiDTGLOpuu2MMbONMXHGmLioqPqXVFZKKfVLdtwBjAUuF5Fk4APgXBF514Y4lFLKpzV4AjDGPGKMiTbGxALTgO+NMTc2dBxKKeXrdLipUkr5KK+oBSQiWcDBcqvaAEdtCsfTmuq56Xl5n6Z6br50Xl2MMVV2onpFAqhIROKrK3DkzZrquel5eZ+mem56Xj/TJiCllPJRmgCUUspHeWsCmG13AB7UVM9Nz8v7NNVz0/OyeGUfgFJKqfrz1jsApZRS9aQJQCmlfJTXJQARuVBE9ojIPhF52O543EVEkkUkQUS2iohXT4AsInNEJFNEdpRbFykiS0XkJ+vvVnbGWBdVnNfjIpJmfW9bReRiO2OsCxGJEZHlIpIoIjtF5A/Weq/+zqo5r6bwnYWIyAYR2Wad29+t9V1F5Efr+rhARKqdXtCr+gBExB/YC0wGUoGNwHXGmERbA3MDqzZSnDHG6weoiMjZwAlgnjFmgLXuGSDHGPOUlbhbGWP+YmectVXFeT0OnDDGPGdnbPUhIh2ADsaYzSISDmwCpgC34MXfWTXndS3e/50JEGaMOSEigcBqnHOsPAAsMcZ8ICKvAduMMa9W9TnedgcwEthnjDlgjCnGWUzuCptjUhUYY1YCORVWXwG8bb1+G+f/iF6livPyesaYI8aYzdbrfJwTNXXCy7+zas7L6xmnE9ZioPXHAOcCi631Lr8zb0sAnYCUcsupNJEvFOeX962IbBKRGXYH4wHtjDFHrNfpQDs7g3Gze0Rku9VE5FXNJBWJSCwwFPiRJvSdVTgvaALfmTWt7lYgE1gK7AdyjTGl1iYur4/elgCasnHGmGHARcDdVnNDk2Sc7Y7e0/ZYvVeB7sAQ4AjwH1ujqQcRaQ58CPzRGHO8/Hve/J1Vcl5N4jszxpQZY4YA0ThbR/rU9jO8LQGkATHllqOtdV7PGJNm/Z0JfITzC21KMqw22dNts5k2x+MWxpgM639EB/A6Xvq9We3IHwLvGWOWWKu9/jur7Lyaynd2mjEmF1gOjAYiRCTAesvl9dHbEsBGoKfV0x2Ecz6BT22Oqd5EJMzqpEJEwoDzgR3V7+V1PgVutl7fDHxiYyxuc/oCabkSL/zerA7FN4Fdxpjny73l1d9ZVefVRL6zKBGJsF43w/lgzC6cieBqazOX35lXPQUEYD2y9SLgD8wxxjxpb0T1JyLdcP7qBwgA3vfm8xKR+cA5OMvTZgB/Az4GFgKdcZb2vtYY41UdqlWc1zk4mxIMkAzcUa7d3CuIyDhgFZDAz/N0/xVne7nXfmfVnNd1eP93NghnJ68/zh/yC40xT1jXkg+ASGALcKMxpqjKz/G2BKCUUso9vK0JSCmllJtoAlBKKR+lCUAppXyUJgCllPJRmgCUUspHaQJQjYaIlFnVGXeIyCIRCa1m21tEZFZDxlfu2E+IyHkutpkrIle72Ca2fGVRpRqaJgDVmJwyxgyxKm0WA3faHVBljDH/Z4xZZncc5VmVcpWqFU0AqrFaBfSwatJ/bBXuWm8NgDlDRMJFJMka8o+ItDi9LCIrRORpq276XhEZb20TIiJviXP+hS0iMtFaf4t1rKXinJ/hHhF5wNpmvYhEWtud+XUvIv8nIhutu5bZ1ujTKonIcKuG+zbg7nLr/UXkWeuztovIHdZ6PxF5RUR2W3F9We7Yydb5bQauEZHzRWSdiGy27qCalzvmD1ahwW8qjIRVPkwTgGp0rFomF+Ecwfl3YIsxZhDOUZzzym9rlfldAVxirZqGsx56ibUcYIwZCfwR58hdcF54jTFmIM5RoW+LSIj13gDgN8AI4EngpDFmKLAO+G0l4c4yxoyw7lqaAZe6OL23gHuNMYMrrL8NyDPGjLCO/TsR6WrFEgv0A27CWe+lvGyriOAy4DHgPGs5HnjASowvAVcbY4YDc6zzUooA15so1WCaWeVtwXkH8CbOcgRXARhjvheR1iLSosJ+bwAP4Sw3MR34Xbn3Thc224TzQgowDudFEWPMbhE5CPSy3ltuJZV8EckDPrPWJwC/uPuwTBSRh4BQnMPvd5bb5xes2i0R1rwCAO/gTHTgrP80qFy/QUugpxXrIqtwWbqILK/wsQusv0fhTBJrrJuQIJxJqzfOpLbUWu+PswKmUpoAVKNyyipve4aLFhUAjDFrrA7VcwB/Y0z5jtXTdVDKqNl/7+XrpjjKLTsq7m/dNbyCcya3FHHODhZC3QjOO4NvKhzD1XSFBeX2X2qMua7C/gOBncaYincOSmkTkGr0VgE3AFgX+KMVa9Vb5gHv42xiqc1n9sJZ7GxPHWI7fbE/arW3V/vUj1W2N9cqUsbpGCzfAHeV68voJc7KsGuAq6y+gHY4i89VZj0wVkR6WPuHWee2B4gSkdHW+kAR6V/L81RNlN4BqMbucWCOiGwHTvJzeeKK3gP+CcyvwWe+ArwqIglAKXCLMaaoJncb5RljckXkdZzlhNNxlit3ZTrO8zHAt+XWv4GziWqz1ZGchXM6vw+BSUAiztnwNgN5lcSSJSK3APNFJNha/ZgxZq/VrDRTRFri/H/+RZxNVcrHaTVQ1SRYF7krjDE32R2Lu4lIc2vy79bABmCsMSbd7riU99M7AOX1ROQlnJ2prtrLvdXnVgdyEPAPvfgrd9E7AKWU8lHaCayUUj5KE4BSSvkoTQBKKeWjNAEopZSP0gSglFI+6v8BGJYmwBdsB44AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_73_1.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Common imports\n", "import os\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from sklearn.linear_model import LinearRegression, Ridge, Lasso\n", "from sklearn.metrics import mean_squared_error\n", "from sklearn.model_selection import KFold\n", "from sklearn.model_selection import cross_val_score\n", "\n", "\n", "# Where to save the figures and data files\n", "PROJECT_ROOT_DIR = \"Results\"\n", "FIGURE_ID = \"Results/FigureFiles\"\n", "DATA_ID = \"DataFiles/\"\n", "\n", "if not os.path.exists(PROJECT_ROOT_DIR):\n", " os.mkdir(PROJECT_ROOT_DIR)\n", "\n", "if not os.path.exists(FIGURE_ID):\n", " os.makedirs(FIGURE_ID)\n", "\n", "if not os.path.exists(DATA_ID):\n", " os.makedirs(DATA_ID)\n", "\n", "def image_path(fig_id):\n", " return os.path.join(FIGURE_ID, fig_id)\n", "\n", "def data_path(dat_id):\n", " return os.path.join(DATA_ID, dat_id)\n", "\n", "def save_fig(fig_id):\n", " plt.savefig(image_path(fig_id) + \".png\", format='png')\n", "\n", "infile = open(data_path(\"EoS.csv\"),'r')\n", "\n", "# Read the EoS data as csv file and organize the data into two arrays with density and energies\n", "EoS = pd.read_csv(infile, names=('Density', 'Energy'))\n", "EoS['Energy'] = pd.to_numeric(EoS['Energy'], errors='coerce')\n", "EoS = EoS.dropna()\n", "Energies = EoS['Energy']\n", "Density = EoS['Density']\n", "# The design matrix now as function of various polytrops\n", "\n", "Maxpolydegree = 30\n", "X = np.zeros((len(Density),Maxpolydegree))\n", "X[:,0] = 1.0\n", "estimated_mse_sklearn = np.zeros(Maxpolydegree)\n", "polynomial = np.zeros(Maxpolydegree)\n", "k =5\n", "kfold = KFold(n_splits = k)\n", "\n", "for polydegree in range(1, Maxpolydegree):\n", " polynomial[polydegree] = polydegree\n", " for degree in range(polydegree):\n", " X[:,degree] = Density**(degree/3.0)\n", " OLS = LinearRegression(fit_intercept=False)\n", "# loop over trials in order to estimate the expectation value of the MSE\n", " estimated_mse_folds = cross_val_score(OLS, X, Energies, scoring='neg_mean_squared_error', cv=kfold)\n", "#[:, np.newaxis]\n", " estimated_mse_sklearn[polydegree] = np.mean(-estimated_mse_folds)\n", "\n", "plt.plot(polynomial, np.log10(estimated_mse_sklearn), label='Test Error')\n", "plt.xlabel('Polynomial degree')\n", "plt.ylabel('log10[MSE]')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that we have kept the intercept in the first column of design matrix $\\boldsymbol{X}$. When we call the corresponding **Scikit-Learn** function we need thus to set the intercept to **False**. Libraries like **Scikit-Learn** normally scale the design matrix and does not fit intercept. See the discussions below.\n", "\n", "## More on Rescaling data\n", "\n", "We end this chapter by adding some words on scaling and how to deal with the intercept for regression cases.\n", "\n", "When you are comparing your own code with for example **Scikit-Learn**'s\n", "library, there are some technicalities to keep in mind. The examples\n", "here demonstrate some of these aspects with potential pitfalls.\n", "\n", "The discussion here focuses on the role of the intercept, how we can\n", "set up the design matrix, what scaling we should use and other topics\n", "which tend confuse us.\n", "\n", "The intercept can be interpreted as the expected value of our\n", "target/output variables when all other predictors are set to zero.\n", "Thus, if we cannot assume that the expected outputs/targets are zero\n", "when all predictors are zero (the columns in the design matrix), it\n", "may be a bad idea to implement a model which penalizes the intercept.\n", "Furthermore, in for example Ridge and Lasso regression, the default solutions\n", "from the library **Scikit-Learn** (when not shrinking $\\beta_0$) for the unknown parameters\n", "$\\boldsymbol{\\beta}$, are derived under the assumption that both $\\boldsymbol{y}$ and\n", "$\\boldsymbol{X}$ are zero centered, that is we subtract the mean values.\n", "\n", "\n", "If our predictors represent different scales, then it is important to\n", "standardize the design matrix $\\boldsymbol{X}$ by subtracting the mean of each\n", "column from the corresponding column and dividing the column with its\n", "standard deviation. Most machine learning libraries do this as a default. This means that if you compare your code with the results from a given library,\n", "the results may differ. \n", "\n", "The\n", "[Standadscaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html)\n", "function in **Scikit-Learn** does this for us. For the data sets we\n", "have been studying in our various examples, the data are in many cases\n", "already scaled and there is no need to scale them. You as a user of different machine learning algorithms, should always perform a\n", "survey of your data, with a critical assessment of them in case you need to scale the data.\n", "\n", "If you need to scale the data, not doing so will give an *unfair*\n", "penalization of the parameters since their magnitude depends on the\n", "scale of their corresponding predictor.\n", "\n", "Suppose as an example that you \n", "you have an input variable given by the heights of different persons.\n", "Human height might be measured in inches or meters or\n", "kilometers. If measured in kilometers, a standard linear regression\n", "model with this predictor would probably give a much bigger\n", "coefficient term, than if measured in millimeters.\n", "This can clearly lead to problems in evaluating the cost/loss functions.\n", "\n", "\n", "\n", "Keep in mind that when you transform your data set before training a model, the same transformation needs to be done\n", "on your eventual new data set before making a prediction. If we translate this into a Python code, it would could be implemented as follows" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "data": { "text/plain": [ "'\\n#Model training, we compute the mean value of y and X\\ny_train_mean = np.mean(y_train)\\nX_train_mean = np.mean(X_train,axis=0)\\nX_train = X_train - X_train_mean\\ny_train = y_train - y_train_mean\\n\\n# The we fit our model with the training data\\ntrained_model = some_model.fit(X_train,y_train)\\n\\n\\n#Model prediction, we need also to transform our data set used for the prediction.\\nX_test = X_test - X_train_mean #Use mean from training data\\ny_pred = trained_model(X_test)\\ny_pred = y_pred + y_train_mean\\n'" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\"\"\"\n", "#Model training, we compute the mean value of y and X\n", "y_train_mean = np.mean(y_train)\n", "X_train_mean = np.mean(X_train,axis=0)\n", "X_train = X_train - X_train_mean\n", "y_train = y_train - y_train_mean\n", "\n", "# The we fit our model with the training data\n", "trained_model = some_model.fit(X_train,y_train)\n", "\n", "\n", "#Model prediction, we need also to transform our data set used for the prediction.\n", "X_test = X_test - X_train_mean #Use mean from training data\n", "y_pred = trained_model(X_test)\n", "y_pred = y_pred + y_train_mean\n", "\"\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us try to understand what this may imply mathematically when we\n", "subtract the mean values, also known as *zero centering*. For\n", "simplicity, we will focus on ordinary regression, as done in the above example.\n", "\n", "The cost/loss function for regression is" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "C(\\beta_0, \\beta_1, ... , \\beta_{p-1}) = \\frac{1}{n}\\sum_{i=0}^{n} \\left(y_i - \\beta_0 - \\sum_{j=1}^{p-1} X_{ij}\\beta_j\\right)^2,.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Recall also that we use the squared value since this leads to an increase of the penalty for higher differences between predicted and output/target values.\n", "\n", "What we have done is to single out the $\\beta_0$ term in the definition of the mean squared error (MSE).\n", "The design matrix\n", "$X$ does in this case not contain any intercept column.\n", "When we take the derivative with respect to $\\beta_0$, we want the derivative to obey" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\frac{\\partial C}{\\partial \\beta_j} = 0,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "for all $j$. For $\\beta_0$ we have" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\frac{\\partial C}{\\partial \\beta_0} = -\\frac{2}{n}\\sum_{i=0}^{n-1} \\left(y_i - \\beta_0 - \\sum_{j=1}^{p-1} X_{ij} \\beta_j\\right).\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Multiplying away the constant $2/n$, we obtain" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\sum_{i=0}^{n-1} \\beta_0 = \\sum_{i=0}^{n-1}y_i - \\sum_{i=0}^{n-1} \\sum_{j=1}^{p-1} X_{ij} \\beta_j.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We assume \n", "that every column of $\\boldsymbol{X}$ is centered, which we can do by subtracting the mean," ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "X = X - np.mean(X,axis=0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This means that we need to rewrite $X_{ij}$ as $\\tilde{X}_{ij}=X_{ij}-\\mu_j$, where" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mu_j = \\frac{1}{n}\\sum_{i=0}^{n-1}X_{ij}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us special first to the case where we have only two parameters $\\beta_0$ and $\\beta_1$.\n", "Our result for $\\beta_0$ simplifies then to" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "n\\beta_0 = \\sum_{i=0}^{n-1}y_i - \\sum_{i=0}^{n-1} X_{i1} \\beta_1.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Assuming that the matrix elements $X_{i1}$ are centered, what we have is" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1}y_i - \\beta_1\\frac{1}{n}\\sum_{i=0}^{n-1} \\left(X_{i1}-\\mu_{1}\\right),\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mu_1=\\frac{1}{n}\\sum_{i=0}^{n-1} (X_{i1},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and if we define the mean value of the outputs as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mu_y=\\frac{1}{n}\\sum_{i=0}^{n-1}y_i,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "we have" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\beta_0 = \\mu_y - \\beta_1\\frac{1}{n}\\sum_{i=0}^{n-1} (X_{i1}-\\mu_{1}),\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and it is easy to see that the last sum equals zero! This means that we have" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\beta_0 = \\mu_y,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of $\\beta$.\n", "We have thus" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1} y_i = \\overline{\\boldsymbol{y}},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "the average value of $\\boldsymbol{y}$.\n", "\n", "Replacing $y_i$ with $y_i - \\beta_0 = y_i - \\overline{\\boldsymbol{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "C(\\boldsymbol{\\beta}) = (\\boldsymbol{\\tilde{y}} - \\tilde{X}\\boldsymbol{\\beta})^T(\\boldsymbol{\\tilde{y}} - \\tilde{X}\\boldsymbol{\\beta}).\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If we minimize with respect to $\\boldsymbol{\\beta}$ we have then" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\hat{\\boldsymbol{\\beta}} = (\\tilde{X}^T\\tilde{X})^{-1}\\tilde{X}^T\\boldsymbol{\\tilde{y}},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $\\boldsymbol{\\tilde{y}} = \\boldsymbol{y} - \\overline{\\boldsymbol{y}}$\n", "and $\\tilde{X}_{ij} = X_{ij} - \\frac{1}{n}\\sum_{k=0}^{n-1}X_{kj}$.\n", "\n", "For Ridge regression we need to add $\\lambda \\boldsymbol{\\beta}^T\\boldsymbol{\\beta}$ to the cost function and get then" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\hat{\\boldsymbol{\\beta}} = (\\tilde{X}^T\\tilde{X} + \\lambda I)^{-1}\\tilde{X}^T\\boldsymbol{\\tilde{y}}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What does this mean? And why do we insist on all this? Let us look at some examples.\n", "\n", "\n", "This code shows a simple first-order fit to a data set using the above transformed data, where we consider the role of the intercept first, by either excluding it or including it (*code example thanks to Øyvind Sigmundson Schøyen*). Here our scaling of the data is done by subtracting the mean values only.\n", "Note also that we do not split the data into training and test." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "True beta: [2, 0.5, 3.7]\n", "Fitted beta: [2.08376632 0.19569961 3.97898392]\n", "Sklearn fitted beta: [2.08376632 0.19569961 3.97898392]\n", "MSE with intercept column\n", "0.004113634617443137\n", "MSE with intercept column from SKL\n", "0.0041136346174431284\n", "Manual intercept: 2.083766322923905\n", "Fitted beta (wiothout intercept): [0.19569961 3.97898392]\n", "Sklearn intercept: 2.0837663229239025\n", "Sklearn fitted beta (without intercept): [0.19569961 3.97898392]\n", "MSE with Manual intercept\n", "0.004113634617443136\n", "MSE with Sklearn intercept\n", "0.004113634617443135\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA7MElEQVR4nO3deVhV1frA8e8GEY+CoqKIQ+KQCsggaKKkgeY8hFNqZtqtzGuD10rTrFRuA4UN2vBrNLVraTmWDQ4BaQ4VCIihpCSKSIooCMjM+v2BnESmI9M5wPt5nvMIZ6999rs4+LLO2nu/S1NKIYQQwnSZGTsAIYQQ5ZNELYQQJk4StRBCmDhJ1EIIYeIkUQshhIlrVBMvamtrqxwcHCq1b0ZGBs2aNavegEyc9Ln+a2j9BenzrQoLC7uklGpT2rYaSdQODg6EhoZWat+QkBB8fHyqNyATJ32u/xpaf0H6fKs0TTtT1jaZ+hBCCBMniVoIIUycJGohhDBxNTJHXZrc3FzOnTtHVlZWue1atGjB8ePHaykq0yB9rhuaNGlCx44dsbCwMHYoooGptUR97tw5rK2tcXBwQNO0MtulpaVhbW1dW2GZBOmz6VNKkZyczLlz5+jSpYuxwxENTK1NfWRlZdG6detyk7QQpkrTNFq3bl3hJ0IhakKtzlFLkhZ1mfz+CmOptakPIYSor7aHJ/D9tvX4ND3LwIN5LBrliF+fDtX2+g3qqg9zc3Pc3d31j7i4OAYOHAhAXFwcX3zxhZEjFELUNdvDE1i15SfGHF1Ls1MhpFxNZcnWKLaHJ1TbMRpUotbpdEREROgfDg4OHDx4EJBELYSonNU/hPO+WSCDXK5y2nE612hCZm4+gbtiqu0YDSpRl8bKygqAxYsXs3//ftzd3XnrrbeMHJUQoi7IzcnmoailNM9JZH7+k5i3c9NvO5+SWW3HMcoc9Ypv/yD6/NVSt+Xn52Nubn7Lr+nUvjnLxjmX2yYzMxN3d3cAunTpwrZt2/TbAgICWLlyJTt37rzlYwshGqYtc3zwCM9hk86V/W1duYM8/bb2NrpqO06DOplYNPUhhBBV9dUL03A7nEJU7yZ8ZP8U5P+zTWdhzsIRPavtWEZJ1OWNfOvajRBCiIZn7/pX6Lk1krgOGqM//YnGf2Ven5NOo4ONjoUjelbrVR8NakRdHmtra9LS0owdhhDCxF04F0vu+s9JawY931mLVYtW+PUBvz4dCAkJ4YkZPtV+zAZ/MrGIq6sr5ubmuLm5yclEIUSpMjPSuPrZvbj1T8F82SK6Ot1RK8dtUCPq9PT0Mp+zsLAgKCiotkMSQtQR+Xl5fPPUUMa2jeXkkA8YMHRarR3boBG1pmk2mqZt1jTthKZpxzVNG1DTgQkhhCn5eq4vrj+nsTNzIO61mKTB8BH1KuBHpdRkTdMaA01rMCYhhDApW/wfwO2XS0Q7WTI54JtaP36FiVrTtBbAYGA2gFIqB8ip2bCEEMI0BG98k25f/c5Ze427P/kR80a1P2OsKaXKb6Bp7sBHQDTgBoQB85VSGTe1mwPMAbCzs/PcuHFjsddp0aIF3bt3rzCgyt7wUpdJn+uOU6dOkZqaesv7paen6++CbSjqQ58zUi6gW7kcy2y4MP8JWnV0Krd9Vfrs6+sbppTqW9o2QxJ1X+Aw4K2U+lXTtFXAVaXUC2Xt07dvX3XzKuTHjx/H0dGxwmAb4nXU0ue6w9Df45vJitx1T1ZmBmfeGEJB6jku9FuEz73zK9yniquQl5moDTmZeA44p5T69fr3mwGPSkUihBB1QH5eHt89P4rbc0+QdfcrBiXpmlRholZK/Q3Ea5pWdD/kUAqnQYQQol766vFhOH2XxObcu+gzYpaxwzH4hpcngA2aph0F3IFXaiyiGvTyyy/j7OyMq6sr7u7u/Ppr4YcEBwcHLl26VKK9MebXtm/fjr+/PwBJSUn079+fPn36sH//fkaPHk1KSgopKSm8//775b7O+fPnmTx5coXHe+UV472VlSkt6+DggIuLi76meFGZ2rLalva+VuSZZ56Ra+obsG0Bc3AN+ZsTPS2Y5L/V2OEUUkpV+8PT01PdLDo6usRzpbl69apB7W7VwYMHlZeXl8rKylJKKZWUlKQSEhKUUkp17txZJSUlldinWbNm1XLsvLy8crff2OcBAwboY/nyyy/VQw89VKL96dOnlbOzc7XEVpk+VtQfQ1y9elUFBwerMWPG3NJ+Zb1XVW17o7i4ODVs2LBStxn6e3yz4ODgSu1Xl9XFPu/b8q4Kc+mldg92VJcvnrvl/avSZyBUlZFTjXNn4g+L4e+oUjfp8vPAvBJhtXOBUQFlbk5MTMTW1hZLS0sAbG1tS7TJzMxk4sSJTJw4kUceeaTYtsDAQL766iuys7OZMGECK1asAMDPz4/4+HiysrKYP38+c+bMAQpH448++ih79+7lvffeY+TIkcyfP5+dO3ei0+nYsWMHdnZ2xY7x559/Ymlpia2tLRERESxatIjMzExCQ0M5dOgQjo6OhIaGsnjxYmJjY3F3d2fYsGEEBgaW6EtcXBxjx47l2LFjrF27lm+++YZr164RGxvLhAkTeP3111m8eLG+9KuzszMbNmzgf//7H6tXryYnJ4f+/fvz/vvvY25uXqI/f/31FytXrkTTNFxdXfn8889JSkpi7ty5nD17FoC3334bb29vli9fTmxsLKdOneLSpUssWrSIadOmsXjxYo4fP467uzuzZs1iwYIFt/CG/6Os96BIRkYG9957L+fOnSM/P58XXniBqVOnEhYWxlNPPUV6ejq2trasXbsWe3t7OnfuTHJyMn///Tft2rWrVEyi7vk7/iTq9XfJbgwdV66mZZvqK6pUVQ2m1sfw4cOJj4+nR48ezJs3j59//rnY9vT0dMaNG8f06dNLJOndu3dz8uRJfvvtNyIiIggLC2Pfvn0ArFmzhrCwMEJDQ1m9ejXJyclAYXLo378/kZGR3HnnnWRkZODl5UVkZCSDBw/m448/LhHjgQMH8PAoPE/r7u6Ov78/U6dOJSIiAp3un9q2AQEBdOvWjYiIiFKTdGkiIiLYtGkTUVFRbNq0ifj4eAICAvSlXzds2MDx48fZtGkTBw4cICIiAnNzczZs2FCiPy1btuSll14iKCiIyMhIVq1aBcD8+fNZsGABv//+O1u2bOHhhx/WH//o0aMEBQVx6NAh/P39SUxMJCAggEGDBhEREcGCBQuIiYkptlTajY+UlBT9a/n6+uLu7k7//v3LfQ+K/Pjjj7Rv357IyEiOHTvGyJEjyc3N5YknnmDz5s2EhYXxr3/9i6VLl+r38fDw4MCBAwb9bEXdl5OdxeX1szHvm0HG/Fk49rvb2CEVY5wRdTkj38waumzLysqKsLAw9u/fT3BwMFOnTiUgIIDZs2cDcM8997Bo0SJmzJhRYt/du3eze/du+vTpAxQm9ZMnTzJ48GBWr16tX4AgPj6ekydP0rp1a8zNzZk0aZL+NRo3bszYsWMB8PT0ZM+ePSWOk5iYSJs2baq76wAMHTqUFi1aAODk5MSZM2fo1KlTsTY//fQTYWFh9OvXDyj8hNG2bVuAYv0JCgpiypQp+k8lrVq1AmDv3r1ER/9znvnq1av6Wir33HMPOp0OnU6Hr68vYWFh2NvbFzt+z549DaoXHhwcXOwTUVnvQREXFxeefvppnn32WcaOHcugQYM4duwYx44dY9iwYUDhdd03xtO2bVvOnz9fYSyi7svPy2P3KxMYqx0jdNRKvMc8UvFOtaxBFWUyNzfHx8cHHx8fXFxcWLdunT5Re3t78+OPP3LfffehaVqx/ZRSLFmyhEcffbTY8yEhIezdu5dDhw7RtGlTfHx8yMrKAqBJkybFbuiwsLDQv665uTl5eXncTKfTVepmCkMUTfmUd3ylFLNmzeLVV18tse3m/pSmoKCAw4cP06RJkxLbbv6Z3vw9QExMDFOnTi31tUNCQrCxsSn1+bLegyI9evTgyJEjfP/99zz//PMMHTqUCRMm4OzszKFDh0o9XlZWVrFPMaL++nrBaFz2xrNt2jAmmGCShgY09RETE8PJkyf130dERNC5c2f99/7+/rRs2ZLHHnusxL4jRoxgzZo1+tFhQkICFy9eJDU1lZYtW9K0aVNOnDjB4cOHqxSjo6Mjp06dqrBdddbOtrCwIDc3FygcdW/evJmLFy8CcPnyZc6cOVNinyFDhvD111/rpxguX74MFE4vvfPOO/p2N46Od+zYQVZWFsnJyYSEhODh4VGiH0Uj6tIepSVpwKD34Pz58zRt2pT777+fhQsXcuTIEXr27ElSUpI+Uefm5vLHH3/o9/nzzz/p3bu3IT9CUYfteOMJXPbGc7JbI8Yu2WDscMrUYBJ1eno6s2bNwsnJCVdXV6Kjo1m+fHmxNqtWrSIzM5NFixYVe3748OHcd999DBgwABcXFyZPnkxaWhojR44kLy8PR0dHFi9ejJeXV5ViHDx4MOHh4agK7hZt3bo13t7e9O7dm4ULF1bpmHPmzMHV1ZUZM2bg5OTESy+9xPDhw3F1dWXYsGEkJiaW2MfZ2ZmlS5dy11134ebmxlNPPQUUTkGEhobi6uqKk5MTH3zwgX4fV1dXfH198fLy4oUXXsDe3r5aaoAb8h5ERUVxxx134O7uzooVK3j++edp3Lgxmzdv5tlnn8XNza3YpX65ubmcOnWKvn1LvUlM1BMHv/uMjuv28rctDPxwOxaNLSveyVjKuhykKg9TvDzPlN3Y5yeffFLt2bPHiNFUv2XLlqnAwMBiz5ny+7x161b1/PPPl7pNLs8znCn3OeGvaPVz/17qcJ9eKmL/jmp73Zq6PK/BjKjriueee45r164ZO4wGLS8vj6efftrYYYgakpuTTfKXc8jolU3KY1Nxu3O8sUOqUIM6mVgX2NnZMX684b84UVFRzJw5s9hzlpaW+rsuTcHNU0ymbsqUKcYOQdSgn99+gLtzjpLl9zL9/B43djgGkURdx7m4uBh0SZsQAjYuGM3tQaf5bupIxtSRJA0N6GSiEKJh2/nuM/T+8TQJ7c25e8Fnxg7nlkiiFkLUe7/v/pJ2n3xHUivo98FmLHV1azVBSdRCiHot4XQ0Gcv9KdCgub8/9p17GTukW9agErW5uXmx+hFxcXEMHDgQqLjkZmJiov4WcFMye/ZsNm/eXOL5F198kb1795a7b0hISLllQmva2rVri92mPW3atGI3JQlRVXm5OSRueJw0u3yS5vjhObRunihuUIm6qABR0cPBwUGfqCpK1G+++WaJYk2mzN/fn7vvLr+wTGUSdWm3nlfWzYn63//+N6+//nq1vb4QBz94lL55YdjNmMuYf5csjVBXNKhEXZqixQEWL17M/v37cXd3L/UuuS1btjBy5EigMMH4+fkxbNgwHBwcePfdd3nzzTfp06cPXl5e+luqP/74Y/r164ebmxuTJk3SXx89e/ZsnnzySQYOHEjXrl3Zvn07UJg4bxy1P/7446xduxYoTLz9+vWjd+/ezJkzp8K7F28caTs4OLBs2TI8PDxwcXHhxIkTxMXF8cEHH/DWW2/h7u7O/v37SUpKYtKkSfTr149+/frpq8ctX76cmTNn4u3tzcyZM7lw4QITJkzAzc0NNzc3fbL/3//+p78D8NFHHyU/P1//M16wYAHOzs4MHTqUpKQktm/fTmhoKDNmzMDd3Z3MzEwGDRrE3r17q/WPgWi4Ni3ygy8OsafpKO6Y/JSxw6kSo12e9+CPD5Z4boTDCMZ0GENmXibz9s4rsf2e7vfg192PK1lXeCqk+A/+s5EVn8Utqr0M0KVLF33FNSgsHbpy5Up27txZYr/Tp0/TsmXLYoWNjh07Rnh4OFlZWXTv3p3XXnuN8PBwFixYwPr16/nPf/5TrK71888/z6effsoTTzwBFE6l/PLLL5w4cYKxY8eWuBb6Zo8//jgvvvgiADNnzmTnzp2MGzeuwj4XsbW15ciRI7z//vusXLmSTz75hLlz52JlZcUzzzwDwH333ceCBQu48847OXv2LCNGjOD48eMAREdH88svv6DT6Zg6dSp33XUX27ZtIz8/n/T09GIlUi0sLJg3bx4bNmzggQceICMjg759+/LWW2/h7+/PihUrePXVV/n0009ZuXJlsVu1u3fvTmRkJJ6engb3TYgi28MTCNwVQ59zG3g0KIYzHc0Z+thHxg6ryhrUddRFUx+3qrTyo76+vlhbW2NtbU2LFi30SdPFxYWjR48Chcn8+eefJyUlhfT0dEaMGKHf38/PDzMzM5ycnEhKSqowhuDgYF5//XWuXbvG5cuXcXZ2vqVEPXHiRKCwxOrWraUvL1RemdLx48frq8kFBQWxfv16oHDev0WLFnz++edllkg1MzPTV8W7//779bGUpqi8qCRqcau2hyewZGsU3a/9zgMHDnK5BQR6PEnen2n49Wlu7PCqxGiJuqwRcFpaGrpGunJHyC2btDRoBF1ddDpdidKZN46uzczM9N+bmZnpP7rPnj2b7du34+bmxtq1awkJCSl1/6JpjEaNGlFQUKB/vuiYWVlZzJs3j9DQUDp16sTy5ctLxFORouOVVeIUyi9T2qxZs3JfX5VTIvVmpZU4LSLlRUVlBe6KoXnmWRYc/gKzAnjHaxJn6Ejgrhj8+pjOai2V0eDnqIuUVzq0R48exMXF3fJrpqWlYW9vT25urn6llPJ07tyZ6OhosrOzSUlJ4aeffgL+Sdi2trakp6eXepVHZdzc5/LKlN5o6NCh/N///R9QWHA/NTW13BKpBQUF+pi/+OIL7rzzzlKPD1JeVFReckoKARYfkddY8cWdnoRaDgDgfEqmkSOrOknU15VXcrNZs2Z069bNoFrRN/rvf/9L//798fb2pleviq/d7NSpE/feey+9e/fm3nvv1a8oY2NjwyOPPELv3r0ZMWKEfnqhqsaNG8e2bdv0JxPLK1N6o1WrVhEcHIyLiwuenp5ER0eXWyK1WbNm/Pbbb/Tu3ZugoCD9XPvs2bOZO3eu/mTihQsX0Ol0sk6huGUZaam8Y/42dzU+xXavSWy1nq7f1t6m7n9C0yq6eqAy+vbtq0JDQ4s9d/z4cRwdHSvcN62GluKqqm3bthEWFsZLL71U7a9tqn2uLlZWVvq57iKl9fmtt96iefPmPPTQQ7UZ3i0x9Pf4ZiEhIfj4+FR/QCastvqcm5PNt/fegS4zi4N9R/F5/jD9Np2FOa9OdKm1qY+q9FnTtDClVKlF0GVEbaAJEybg4OBg7DDqNRsbG2bNmmXsMEQdkp+Xx7ZZ3jieyCGlczs8Jy+kg40ODehgo6vVJF2TGtRVH1V146rawnA3j6bL8uCDJS/ZFKI8X8/1xS08g6OeVtz7/k+YN2pULxLzzWRELYSokzY9NQa3Xy4R7WTJxM9+wbxR/R13SqIWQtQ5oTs/wkULI9rZgtGf7zPt9Q6rgSRqIUSdEvzlStx+XwytuzPm84PomtXtm1kMIYlaCFFn7Pp0OS1f/pRvz7Wn47wdNGlqZeyQaoUkaiFEnbBvy7u0Xr2JVGtwefJDmtu0NnZItaZBJeqXX34ZZ2dnXF1dcXd31y8A6+DgwKVLl0q0L6qsV5u2b9+Ov78/AElJSfTv358+ffqwf/9+Ro8eTUpKCikpKbz//vvlvs758+eZPHlyhcd75ZVXqiXuyqiotOzN+vfvj7u7O7fddhtt2rQpVle8KqQOtukL++lrGr/yHjkW0CLgFW538zZ2SLVLKVXtD09PT3Wz6OjoEs+V5urVqwa1u1UHDx5UXl5eKisrSymlVFJSkkpISFBKKdW5c2eVlJRUYp9mzZpVy7Hz8vLK3X5jnwcMGKCP5csvv1QPPfRQifanT59Wzs7O1RJbZfpYUX8McfXqVRUcHKzGjBlzy/t+9tln6rHHHivxfG5ubqViCQkJUQ8//LBBbQ39Pb5ZcHBwpfary6qrz+dOH1chXr3Ub2691IEdH1XLa9aUqvQZCFVl5FSjjajPzHygxOPy9dFVQWZmqdtTthaWJc27cqXEtookJiZia2urL05ka2tL+/bti7XJzMxk1KhRfPzxxyX2DwwMpF+/fri6urJs2TL9835+fnh6euLs7MxHH/1TTtHKyoqnn34aNzc3Dh06hJWVFUuXLsXNzQ0vLy8uXLhQ4hh//vknlpaW2NraEhERwaJFi9ixY4f+Fuuikf/ixYuJjY3F3d2dhQsXltrfuLg4fc2MtWvXMnHiREaOHMntt9/OokWLgMIa3EWlX2fMmAGUX1P6xv6sX78eV1dX3Nzc9CVaK6pnPWDAAG6//Xb9z7eiGuCGuLlW9tq1a3n88X9Wlx47dqy+GNbu3bsZMGAAHh4eTJkyRX99t9TBNl2pVy6RvX4q+e6ZXPnPDAaOrzuLd1SnBjP1MXz4cOLj4+nRowfz5s3j559/LrY9PT2dcePGMX369BIruezevZuTJ0/y22+/ERERQVhYGPv27QNgzZo1hIWFERoayurVq0lOTgYgIyOD/v37ExkZyZ133klGRgZeXl5ERkYyePDgUv8YHDhwAA8PDwDc3d3x9/dn6tSpREREFKsoFxAQQLdu3YiIiCAwMNCg/kdERLBp0yaioqLYtGkT8fHxBAQE6Eu/btiwoVhN6YiICMzNzfXFpG7sT8uWLXnppZcICgoiMjKSVatWATB//nwWLFjA77//zpYtW4rdIHT06FGCgoI4dOgQ/v7+JCYmEhAQwKBBg4iIiGDBggXExMQUWyrtxkdKSkqZfYuOjmbv3r18+eWXZba5dOkSL730Env37uXIkSP07duXN998EyiseFhUB1uYjssX4jn00nA65sfTZvpbDJ/9vLFDMhqjXSHe+fP1pT6flpaGmU5X5naARi1blru9NFZWVoSFhbF//36Cg4OZOnUqAQEBzJ49G4B77rmHRYsW6UeWN9q9eze7d+/WF0lKT0/n5MmTDB48mNWrV+sXIIiPj+fkyZO0bt0ac3NzJk2apH+Nxo0b61dv8fT0ZM+ePSWOU1rd6+oydOhQWrRoAYCTkxNnzpyhU6dOxdr89NNPZdaUvrE/QUFBTJkyBVtbWwBatWoFlF/P+p577kGn06HT6fD19SUsLAx7e/tix+/Zs2el6oXfWCu7LIcPHyY6Ohpv78K5zZycHAYMGKDfLnWwTUtGWioHZo/itrP5/LLkGYYMnmDskIyq/t7KUwpzc3N8fHzw8fHBxcWFdevW6RO1t7c3P/74I/fdd1+JeslKKZYsWcKjjz5a7PmQkBD27t3LoUOHaNq0KT4+PvqSpE2aNMHc3Fzf1sLCQv+6ZdWE1ul0pKamVmeX9W6sf13W8VU5NaVv7k9pyqtnffPPtLSa1DExMfoFBm4WEhKCjY1NqdturJVdVk1vpRTDhg0rc9QtdbBNR25ONrtmDsbxdD6RY7sz7f7njB2S0TWYqY+YmJhiZ/YjIiLo3Lmz/nt/f39atmzJY489VmLfESNGsGbNGv3oMCEhgYsXL5KamkrLli1p2rQpJ06c4PDhw1WK0dHR0aBSquXVzr5VFhYW5ObmApRbU/pGQ4YM4euvv9ZP8xStEVlePesdO3aQlZVFcnIyISEheHh4lOhH0Yi6tEdZSfpmDg4OREREUFBQQHx8PL/99hsAXl5eHDhwQP/zzcjI4M8//9TvJ3WwTUN+Xh7bHigsshR5lx3TVn5r7JBMQoNJ1Onp6cyaNQsnJydcXV2Jjo5m+fLlxdqsWrWKzMxM/cm2IsOHD+e+++5jwIABuLi4MHnyZNLS0hg5ciR5eXk4OjqyePFivLy8qhTj4MGDCQ8Pr3Dh2tatW+Pt7U3v3r3LPJloqDlz5uDq6sqMGTPKrSl9I2dnZ5YuXcpdd92Fm5sbTz1VuH5lefWsXV1d8fX1xcvLixdeeAF7e/tya4BXlre3N126dMHJyYknn3xSP+ffpk0b1q5dy/Tp03F1dWXAgAGcOHECQOpgm5CtL07EJSKDo57WTHlvr7HDMR1lXQ5SlYcpXp5nym7s85NPPqn27NljxGiq37Jly1RgYGCx50zpfX7zzTfVJ598YlBbuTzPcLfa58NfvqLyX2yuvv7PIJVXyUstja3eXZ4nSvfcc89x7do1Y4fRoEgdbOPb7D+TtkfeIrLZQPxe31uvK+FVhkE/DU3T4oA0IB/IU2WsQiCqzs7OjvHjxxvcPioqSn8dcxFLS0v9XZem4OYpJlMjdbCNa/vKefTcGEpkt7aM/GozjSwaGzskk3Mrf7Z8lVIl77MWRuXi4lKpS9qEMAW7PlmGw9pg/raF/u9upomu/NXuGyqZ+hBCGMW+ze/S+p2vSLUGh3c+wb5zxQtAN1QGLW6radpp4AqggA+VUh+V0mYOMAfAzs7Oc+PGjcW2t2jRgu7du1d4rPz8/Aqv161vpM91x6lTpyp1rXt6erpRinwZU3l9zricQMGn/6X1RY2Euf+iTbd+tRxdzajK++zr61vm4raGJuoOSqkETdPaAnuAJ5RS+8pqXx9XIa9J0ue6Q1YhN1xZfb50/gzZHw+DvGskeL/KHSNnlty5jjLqKuRKqYTr/14EtgF3VCoSIzM3N8fd3R1nZ2fc3Nx44403it3FVppbLcUphCjb2ZOR7HtyFOY5qWRM+F+9StI1qcJErWlaM03TrIu+BoYDx2o6sJpQVIDojz/+YM+ePfzwww+sWLGi3H0kUQtRPS5fiCf60el0P64I6/Qvenj4GDukOsOQEbUd8IumaZHAb8B3SqkfazYs2B6egHdAEF0Wf4d3QBDbwxOq9fXbtm3LRx99xLvvvotSiri4OAYNGoSHhwceHh4cPHgQKFmKs6x2Qoiypade5sCskXRKVJycdgdj5pasJyPKVuHleUqpvwC3WohFb3t4Aku2RpGZW1gLOSElkyVbowDw69Oh2o7TtWtX8vPzuXjxIm3btmXPnj00adKEkydPMn36dEJDQwkICGDlypXs3LkTgGvXrpXaTghRutycbHY/4INjXAFHx/Vg6gvrjB1SnWOSt/8E7orRJ+kimbn5BO6KqdZEfaPc3Fwef/xxfR3mGwv2VKadEAJUQQHBb83EPiGXCJ92TA/cYeyQ6iSTTNTnUzJv6fnK+uuvvzA3N6dt27asWLECOzs7IiMjKSgoKLVUJ8Bbb71lUDshGqrt4QkE7ophWqc0olc8wjxtF0H3T2Dak2uMHVqdZZKJur2NjoRSknJ7m+qrF5yUlMTcuXN5/PHH0TSN1NRUOnbsiJmZGevWrdMvQXVzKc6y2gkhik9b6mLfpF3Keb7qOQiLQf9FM5P76yrLJH9yC0f0RGdR/GYInYU5C0f0rNLrFq0P6OzszN13383w4cP16x/OmzePdevW4ebmxokTJ/TF6G8uxVlWOyHEP9OWcy+txnv/RXIzLFmaM5uVe2SV96owyRF10Tx04K4Yzqdk0t5Gx8IRPas8P13e6Pf222/n6NGj+u9fe+01oLCwflBQULG2pbUTQhROTz5x4Q1GH0ok9jaNF3ovIddMV+3Tlg2NSSZqKEzWNXXiUAhRM5658BpDDiUR08WcuMnPkn7SBqjeacuGyCSnPoQQdYsqKODQZ8/Sp+lpjvVoxLMuyzBv0hyonmnLhq5WE7UhdUWEMFXy+1u6/Lw8vnl5KgPOfIBF94GYv7gT21atAehgo+PViS7y6biKam3qo0mTJiQnJ9O6detSV6AWwpQppUhOTpbLMW+Sm5PNtlneOEdk8M2UOxn75Bf0MzdnYt/OhISE8MQMH2OHWC/UWqLu2LEj586dIykpqdx2WVlZDe4/g/S5bmjSpAkdO3Y0dhgmIzvzGt/O9MblWBZH+1oz+cUdmNXB0rV1Qa0lagsLC7p06VJhu5CQEPr06VMLEZkO6bOoazLSUtk1czDOJ3KIHNiKKR/9LOsc1iA5mSiEuCXZWdf44bkROJ7IIcLHjmlrDkiSrmGSqIUQBsu6lk7M2+OZ2Oo4J2bdwfQPQowdUoMgiVoIYZCLCbHsurc/jZOOEeq6nAlLpApebZFELYSoUHzsMcLvH0e32AKirIZwx6QFxg6pQZFELYQo11/Rv3HiwSm0v6CIub8/U1bIike1TRK1EKJMf0bu58wjs2h7GWL/5cukpWuNHVKDJKdqhRCl+jv+FGZb53DNpoCM+8bj91igsUNqsCRRCyFKOBK8haZ7n6VDo3Tyln1OrzuGGTukBk0StRCimIPffQbLXufv1o0wf+1rerkPMnZIDZ4kaiGE3r7N72L5ynsAWD/6BLdLkjYJkqiFEADs/TwAmzfWkdMIzP67mEGjZhk7JHGdJGohBH8c/A6zj9aRaQlWAS/Rx3eSsUMSN5BELUQDF7VvB91+eoRM79Y08XuT3l6jjB2SuIkkaiEasO2BczE/HITO1Q6H/3yPbbtOxg5JlEJueBGigdri/wDdPvuZJpca0WTGF5KkTZiMqIVogL5aOgXnrceIt9dw+XQzHbo4GTskUQ5J1ELUU9vDEwjcFcP5lEza2+hYOKInfn06sHHheFy+PUlcJzP6r9+JrX3FC3oI45JELUQ9tD08gSVbo8jMzQcgISWTJVujUIfeo2t2GKe6t2TwZ7to2UYWna0LZI5aiHoocFeMPkkXeTDtAyZcfI9GnTwYtSVUknQdIolaiHrofErmP9+ofJ4/s4IxP8bw2WUXXP+zhcaWdWth4YZOErUQ9VB7G13hFyqfFX8txzs8jUinxnzSdimNLBobNzhxyyRRC1EPLRzRE2uzbF45+QJ3RGXyu4sOf8eXWTjG1dihiUqQk4lC1EO+nRuzIu1lHKNzONDHmk97v0TAaGf8+si8dF0kiVqIeib2j19pvHk2Y1td4Ic5U3n4qf/jYWMHJapEErUQ9cjOd5+h1brvsBicRfq9X3JP/xHGDklUA0nUQtQTm54ei/P3sSS1giZjAnGUJF1vSKIWoo7LzLjKtw8PwTU8g1NdzOn7wWbsO/cydliiGhmcqDVNMwdCgQSl1NiaC0kIYajUK5fYvWg4LuGZHPW0xu+TECx1TY0dlqhmt3J53nzgeE0FIoS4NWf/PErKO3cxul0cf9znwdQNv0mSrqcMStSapnUExgCf1Gw4QghDfPfes/z58FTystKIG/U/Jr+4wdghiRqkKaUqbqRpm4FXAWvgmdKmPjRNmwPMAbCzs/PcuHFjpQJKT0/HysqqUvvWVdLn+q86+xu3+RU8g+JJtoELsx/ArseAannd6tbQ3mOoWp99fX3DlFJ9S92olCr3AYwF3r/+tQ+ws6J9PD09VWUFBwdXet+6Svpc/1VHf7OuZahN9/VV0T17qW+HO6lzf/1R9cBqUEN7j5WqWp+BUFVGTjXkZKI3MF7TtNFAE6C5pmn/U0rdX6k/G0KIW3Y1JZldjw/FJSybKA8rxn38E7pmzY0dlqglFc5RK6WWKKU6KqUcgGlAkCRpIWpPwl/Hubz6LgbdlsBRP0fu/eJ3SdINjFxHLYQJ+/6D5yj4div9Pa5yYcwapnqPM3ZIwghuKVErpUKAkBqJRAhRzKZFfjjtjCHZRuPcsPfpI0m6wZIRtRAmJjvzGtsf8cE1NI3Y28xw+79NdOrW29hhCSOSRC2ECUlLvcyuh31wjcolyr0Z4z4NkvloIYlaCFNxPi6G7PVTcO58kaO39WXqGzuNHZIwEbLCixA1aHt4At4BQUQlpOIdEMT28IRS2/348Qv8vmQ8LfMvkTf+Q0nSohgZUQtRQ7aHJ7Bka1ThauCdICElkyVbowCKrbTy1ZJJOO6I5nILM84/tQ6XfsOMFbIwUTKiFqKGBO6KKUzSN8jMzSdwVwxQeNJw08z+uGyL5mxHM7qs34iTJGlRChlRC1FDzqdklvl8+tUr7J45GNeYPI656Rj9STDNrFvUcoSirpARtRA1pL2NrtTnXa1Tufi2D207pBA5sjNTNh2RJC3KJYlaiBqycERPdBbmxZ4bk/kNc+OWYVtwkRZTVzHt7R+NFJ2oS2TqQ4gaUnTCsHBOOo15V95j5C+nudyiERee2IaLq5dxAxR1hiRqIWqQX58OjHG25at/DcIjNI3THc1wemcdDo6llx0WojSSqIWoQefPxHBkzkQ8zhTwh0sTRq0JkflocctkjlqIGnLkx7U0+8wXzS6bX+/uxIQvf5ckLSpFRtRCVLOLCbHsmz+Z29pf5GLb9ri+8CmxCVcwbyT/3UTlyIhaiGq09/MATkwai+OxLP7K7obDogN0ut3N2GGJOk7+xAtRDbIzr7F1/ih6/3KRq80gbr4f0/79qrHDEvWEJGohqijxTAy/Bk7DfV8WJ3pa4Lnyc7xlFC2qkUx9CFEFe9f70+wzH4a0SuT47IGM33KE2yRJi2omI2ohKiHxzAkOLpiGw6lsjo3vQOd/fc7Ero7GDkvUU5KohbhFu9asoOmHG+l1FaIGtuaeJd/LKiyiRkmiFsJAOdlZbJ13Ny4Hk7liDeeemcq0h5YbOyzRAEiiFsIACX8dJ+2LB7G6nMyJXk0Z8NZG2jvIVIeoHZKohShHfl4eW5ZNp0fWYbpa55E66wX6+80zdliigZFELUQZ4mOPEfrUfbjE5HLcqQV2b22hf+eexg5LNECSqIUoxfcfPEeLNdvong4Rg9sycdUPWOqaGjss0UBJohbiBrk52ezwn4Lj5pMk28CFxTOZ/sBzxg5LNHCSqIW47vSJMHI3z2W89ifbBvTE96Uvaduhm7HDEkIStRD5eXlsfm4ydr/E0NX3KsfufJupox40dlhC6EmiFg3amRNhhD8zG9dTeZzuZEa631o8+g0zdlhCFCOJWjRYO999hlZrv6PbNYgcYs+kt3/AorGlscMSogRJ1KLBycnO4shnT2H+7U/kNDYj6emHmDb9GWOHJUSZJFGLBuXgd59hfnAVXhanCbp7BL3vfw1b+y7GDkuIckmiFg1CduY1ti6eiONPZzjXWRE+/z2GDL/f2GEJYRBJ1KLe+/6D57DcsA33JIjtbEZ3/3fo5TnE2GEJYTBJ1KLeunAuln2vzaT3nitcbg5bh/TiY+sH6bAHFpol4Neng7FDFMIgkqhFvZOSnMih9S9yV8oOfJrDDwNv5x27f5OkWgKQkJLJkq1RAJKsRZ0gS3GJeiM/L49trz1K1Ogh6Dbv53hTT3If3seHPf6rT9JFMnPzCdwVY6RIhbg1MqIW9cLhH9aRtOp1esUV8HcrSL93JD7z3wbgfEpsqfucT8msxQiFqLwKE7WmaU2AfYDl9fablVLLajowIQyRfvUKe958kO5fxWDRCCLv7si4VzfTzLqFvk17Gx0JpSTl9ja62gxViEozZOojGxiilHID3IGRmqZ51WhUQlQgNyebXR8/R9ab7owtCCa6rxVW6z9k2rt7iiVpgIUjeqKzMC/2nM7CnIUjpLa0qBsqHFErpRSQfv1bi+sPVZNBCVGeoC9WkvvxGmxSFfHjbLnst46pHj5lti86YRi4K4bzKZm0t9GxcERPOZEo6gyD5qg1TTMHwoDuwHtKqV9rNCohShEbdZBw/8dwjMoivSnEDe/FxKUbDarP4dengyRmUWdphQNmAxtrmg2wDXhCKXXspm1zgDkAdnZ2nhs3bqxUQOnp6VhZWVVq37pK+ly+gvw8Lv66ge6bDmOZC8dcm2I1eQHWth1rOMrqI+9xw1CVPvv6+oYppfqWulEpdUsP4EXgmfLaeHp6qsoKDg6u9L51lfS5bD9veUf9tcJFqWXN1Vf3u6oDOz6q2cBqiLzHDUNV+gyEqjJyqiFXfbQBcpVSKZqm6YBhwGuV+pMhhIGOHviW2NeW0jU2l8vj8rni+y6TX5yBZiaX/ouGx5A5antg3fV5ajPgK6XUzpoNSzRUKcmJ/PjsvfQ6fImuwPEBtox4eiMt28j8smi4KhyeKKWOKqX6KKVclVK9lVL+tRGYqJ+2hyfgHRBEVEIq3gFBbA9PAEAVFPDrzk+IHDsEt18uEdelEer/Apj2yX5J0qLBkzsTRa3ZHp7Akq1RZObmQ6d/am6kRO/BI349/bPC2NazI2l9R+H3+EpjhyuEyZBELWpN4K6YwiR9XZv8v3ns9Pt4brlG+vBrHO6/iHFLF9LIorERoxTC9EiiFrWmqLaGmcqh8YH/453D8Vhfg0jHxtzxr0/p7jrQyBEKYZokUYtaY9/ckh7pvzIxfAO3n1actdf45E5fjrefxH2SpIUokyRqUeMy0lL54Y15rFGR9Gocz7f2duzr2pXXrB7FsrGOV6XmhhDlkkQtasylxNPsfXUuHQ+fxfkqnB5mRpTbUt5p7MqUztnYx1tLzQ0hDCCJWlS7xDMn+dn/QboeScYtE8500EiaNIixC1Zj0diSKUBISAhPzPAxdqhC1AmSqEW1if3jVy4Ff4jThW+xj25Non0jrkycxMiHVxg7NCHqNEnUospCvn6Hyxs+of3ZHHqOvshx2+HYvTOHu/r6Gjs0IeoFSdSiUvLz8vjhwyXwzfd0O1NA88Zw0tUa26mfcoeLXMEhRHWSRC1uSX5eHpF7PiclaDXdvs0itSlE3mWH98J38OjuYuzwhKiXJFELg6QkJ7LrlTk0vniKCd3Oc87KnkNTXBn2n1V4tbY3dnhC1GuSqEW54mOPceD1x3H4/QKu1wqv4Aid+iZ9Rs5iSiP59RGiNsj/NFGqiwmnCXlrLt13n8UtB2I7m3Fp3GhGzX0Vc0nQQtQq+R8nijm081OuhG3l7oJDeGgWHO1iR6v7/sXYqf8xdmhCNFiSqAUAu9asIGvz1/T4K5/k7gWEj/Sj05hFTOzSy9ihCdHgSaJuwAry8/l21XwsdwbR+bwivQlEDmxNvwWv000usRPCZEiiboBORh7gwrFgOsZuotGJa1inNyZy+G3cveQD+tl3MXZ4QoibSKJuIM7GHOHQJyuwijyFQ3wB2pB08tu1xmbCY7iPfAhv6xbGDlEIUQZJ1PXY1ZRkwr//kKvrN+BwtgDXAkiygSivlnQZu5iuI2bSTVb1FsLkSaKuZy4mxBLywVK4eAq/dqcZRB678toT3bc5bUbdy11T5vNt1AWe2hXD+Z9/oL2NTkqNCmHiJFHXA5cvxBP04VIa/RZOl7g8XPLgfBsIc5tIqzumM3KZD9r1kXOxBWb5Z4FZQJK1ECZKEnUdlZKcyF+/70H9sY2EQ3/gfMyc1KYQ07sZzYeOYuis57FobFliv5sXmAXIzM0ncFeMJGohTJQk6jokPfUyez9aSv7BgzjE5tD47hQ6Njfnb9c+xPr2Yti/VuCla1ruaxQtMGvo80II45NEbeLycnP47YfPOL/uAxxis+iZBRmWcPp2S1q4PUav6QsZU8rIuSztbXQklJKU29voqjNsIUQ1kkRtgrIzr7H3sxVkxoUxpGkMnvlXUWfbc6ZLY8wHDODuR1+hb4tWlXrthSN6FpujBtBZmLNQFpgVwmRJojYRuTnZ/LTuJa7+9AO3ncygawYktFWcvscTrfck3J8dTzOrql/rXDQPHbgrhvMpmXLVhxB1gCRqI0pPvUz8iVByf1vH96vm0uO4Rk4jOO3QiHN93fB59L/Y1sCdgn59OkhiFqIOkURdS/Lz8jh64FtOBX2NOnWSFuczaHdR0XxMMnfp8tjV3ZFjzh0ZNOcl3G673djhCiFMiCTqGpJ45gRh33xMo4J02mWf4dKZU3T4yYLeQHYj+NtO4w83K6LMhmLvMJBNF9qxcERP2t0mI10hRHGSqKtBfl4eMWHBRH39Duanz9L672zaJkM34KxXFq266rjS0YnI0fm07TeE/mMfJu5UOv+9flLv6cZ5cuOJEKJMkqgr4fjve4ne/QV5MdE00mUxqt15OhfkUPCdPdeawIV25lxwbInOuQ8e4x6hU3cXbrvpNQJ3hcmNJ0IIg5hMot4enkDgrhimdUpjaUBQrV2JUHTcsq6ASEtN5tyJUFJPHuTc9u3Yn8mhVRo4AXlmcNLZjCiXsZjfdgfmQ9vieed4g5aqkhtPhBCGMolEXaz+RKfaqz9xc92LxCupfP2/90he/wfNz8fTMjETDcXQIYmF7fPbc7G9BfG3taOV5yDuGPcwLpVcgVtuPBFCGMokEnVt1Z9IvXyB038c5mJsFBl/nyXu1Gn+k5mGZef2dDW/SMqxJG4/Xli8KLMxJLYzI61DC8IHvkAn18H4tetUbbHIjSdCCEOZRKKuyjRARloqqZf/JuPKRc7+cYDko7+Sf+Uy5mkZWGTk0CQjD3svRVeLFA7G6nD43ZKidFu0GqDFbQlkay2J6diO460sCbfuw+ev++NxC7dm36obbzyBNDrIjSdCiDKYRKK+eRqgkcqhU94ZHBsl8M3bu7Fq2ZqmWi4X4/4g70gsltfyaJpRQLNrYJ0JuaNTcG1+jXPnbXDZV1iU6FpjSG8G15pqJFl0ILvNQHJ1uRy1v4plm3ZYt+/KxycaEZndgUxlBTlAs8JHBxtdqZXnqlvRjSchISE8McOnxo8nhKibTCJRLxzRk8VbInn72iJs3shnRyaYqX+2Jw9Jx6vtVQ5faUFuYjOuNdVIbdmISx0tyLdqRsseY8nv6kYLiyZkzTDHwekOWrapeGSaFp5A5NYokOkHIYQJM4lEXfRxP+XLdqQ6JJHepAlmNi1obm9Ps3a30bmPDxk9PfGytqmR40rdCyGEKaswUWua1glYD9gBCvhIKbWqugPx69MB+uwlJCSEqT4+1f3y5R5XErMQwpQZMqLOA55WSh3RNM0aCNM0bY9SKrqGYxNCCAFUuAS1UipRKXXk+tdpwHFAhqBCCFFLNKVUxa2KGmuaA7AP6K2UunrTtjnAHAA7OzvPjRs3Viqg9PR0rKysKrVvXSV9rv8aWn9B+nyrfH19w5RSfUvdqJQy6AFYAWHAxIraenp6qsoKDg6u9L51lfS5/mto/VVK+nyrgFBVRk6tcOoDQNM0C2ALsEEptbVSfy6EEEJUSoWJWtM0DfgUOK6UerPmQxJCCHEjQ0bU3sBMYIimaRHXH6NrOC4hhBDXVXh5nlLqF0CrhViEEEKU4pau+jD4RTUtCThTyd1tgUvVGE5dIH2u/xpaf0H6fKs6K6XalLahRhJ1VWiaFqrKukSlnpI+138Nrb8gfa5OBl31IYQQwngkUQshhIkzxUT9kbEDMALpc/3X0PoL0udqY3Jz1EIIIYozxRG1EEKIG0iiFkIIE2e0RK1p2khN02I0TTuladriUrZbapq26fr2X69X7quzDOjvU5qmRWuadlTTtJ80TetsjDirU0V9vqHdJE3TlKZpdf5SLkP6rGnavdff6z80TfuitmOsbgb8bt+maVqwpmnh13+/6/SdzZqmrdE07aKmacfK2K5pmrb6+s/jqKZpHlU+aFnVmmryAZgDsUBXoDEQCTjd1GYe8MH1r6cBm4wRay321xdoev3rf9fl/hra5+vtrCksnXsY6GvsuGvhfb4dCAdaXv++rbHjroU+fwT8+/rXTkCcseOuYp8HAx7AsTK2jwZ+oPCObi/g16oe01gj6juAU0qpv5RSOcBG4J6b2twDrLv+9WZg6PUCUXVRhf1VSgUrpa5d//Yw0LGWY6xuhrzHAP8FXgOyajO4GmJInx8B3lNKXQFQSl2s5RirmyF9VkDz61+3AM7XYnzVTim1D7hcTpN7gPWq0GHARtM0+6oc01iJugMQf8P35yi5aoy+jVIqD0gFWtdKdNXPkP7e6CEK/yLXZRX2+fpHwk5Kqe9qM7AaZMj73APooWnaAU3TDmuaNrLWoqsZhvR5OXC/pmnngO+BJ2onNKO51f/vFTKJVcjFPzRNux/oC9xl7FhqkqZpZsCbwGwjh1LbGlE4/eFD4aemfZqmuSilUowZVA2bDqxVSr2hadoA4HNN03orpQqMHVhdYawRdQLQ6YbvO15/rtQ2mqY1ovAjU3KtRFf9DOkvmqbdDSwFxiulsmsptppSUZ+tgd5AiKZpcRTO5X1Tx08oGvI+nwO+UUrlKqVOA39SmLjrKkP6/BDwFYBS6hDQhMLiRfWVQf/fb4WxEvXvwO2apnXRNK0xhScLv7mpzTfArOtfTwaC1PWZ+jqowv5qmtYH+JDCJF3X5y2hgj4rpVKVUrZKKQellAOF8/LjlVKhxgm3Whjye72dwtE0mqbZUjgV8lctxljdDOnzWWAogKZpjhQm6qRajbJ2fQM8cP3qDy8gVSmVWKVXNOKZ09EUjiZigaXXn/On8D8rFL6ZXwOngN+ArsY+21vD/d0LXAAirj++MXbMNd3nm9qGUMev+jDwfdYonPKJBqKAacaOuRb67AQcoPCKkAhguLFjrmJ/vwQSgVwKPyE9BMwF5t7wHr93/ecRVR2/13ILuRBCmDi5M1EIIUycJGohhDBxkqiFEMLESaIWQggTJ4laCCFMnCRqIYQwcZKohRDCxP0/5zUzMkNVbe8AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_109_1.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "from sklearn.linear_model import LinearRegression\n", "\n", "\n", "np.random.seed(2021)\n", "\n", "def MSE(y_data,y_model):\n", " n = np.size(y_model)\n", " return np.sum((y_data-y_model)**2)/n\n", "\n", "\n", "def fit_beta(X, y):\n", " return np.linalg.pinv(X.T @ X) @ X.T @ y\n", "\n", "\n", "true_beta = [2, 0.5, 3.7]\n", "\n", "x = np.linspace(0, 1, 11)\n", "y = np.sum(\n", " np.asarray([x ** p * b for p, b in enumerate(true_beta)]), axis=0\n", ") + 0.1 * np.random.normal(size=len(x))\n", "\n", "degree = 3\n", "X = np.zeros((len(x), degree))\n", "\n", "# Include the intercept in the design matrix\n", "for p in range(degree):\n", " X[:, p] = x ** p\n", "\n", "beta = fit_beta(X, y)\n", "\n", "# Intercept is included in the design matrix\n", "skl = LinearRegression(fit_intercept=False).fit(X, y)\n", "\n", "print(f\"True beta: {true_beta}\")\n", "print(f\"Fitted beta: {beta}\")\n", "print(f\"Sklearn fitted beta: {skl.coef_}\")\n", "ypredictOwn = X @ beta\n", "ypredictSKL = skl.predict(X)\n", "print(f\"MSE with intercept column\")\n", "print(MSE(y,ypredictOwn))\n", "print(f\"MSE with intercept column from SKL\")\n", "print(MSE(y,ypredictSKL))\n", "\n", "\n", "plt.figure()\n", "plt.scatter(x, y, label=\"Data\")\n", "plt.plot(x, X @ beta, label=\"Fit\")\n", "plt.plot(x, skl.predict(X), label=\"Sklearn (fit_intercept=False)\")\n", "\n", "\n", "# Do not include the intercept in the design matrix\n", "X = np.zeros((len(x), degree - 1))\n", "\n", "for p in range(degree - 1):\n", " X[:, p] = x ** (p + 1)\n", "\n", "# Intercept is not included in the design matrix\n", "skl = LinearRegression(fit_intercept=True).fit(X, y)\n", "\n", "# Use centered values for X and y when computing coefficients\n", "y_offset = np.average(y, axis=0)\n", "X_offset = np.average(X, axis=0)\n", "\n", "beta = fit_beta(X - X_offset, y - y_offset)\n", "intercept = np.mean(y_offset - X_offset @ beta)\n", "\n", "print(f\"Manual intercept: {intercept}\")\n", "print(f\"Fitted beta (wiothout intercept): {beta}\")\n", "print(f\"Sklearn intercept: {skl.intercept_}\")\n", "print(f\"Sklearn fitted beta (without intercept): {skl.coef_}\")\n", "ypredictOwn = X @ beta\n", "ypredictSKL = skl.predict(X)\n", "print(f\"MSE with Manual intercept\")\n", "print(MSE(y,ypredictOwn+intercept))\n", "print(f\"MSE with Sklearn intercept\")\n", "print(MSE(y,ypredictSKL))\n", "\n", "plt.plot(x, X @ beta + intercept, \"--\", label=\"Fit (manual intercept)\")\n", "plt.plot(x, skl.predict(X), \"--\", label=\"Sklearn (fit_intercept=True)\")\n", "plt.grid()\n", "plt.legend()\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The intercept is the value of our output/target variable\n", "when all our features are zero and our function crosses the $y$-axis (for a one-dimensional case). \n", "\n", "Printing the MSE, we see first that both methods give the same MSE, as\n", "they should. However, when we move to for example Ridge regression,\n", "the way we treat the intercept may give a larger or smaller MSE,\n", "meaning that the MSE can be penalized by the value of the\n", "intercept. Not including the intercept in the fit, means that the\n", "regularization term does not include $\\beta_0$. For different values\n", "of $\\lambda$, this may lead to differeing MSE values. \n", "\n", "To remind the reader, the regularization term, with the intercept in Ridge regression, is given by" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\lambda \\vert\\vert \\boldsymbol{\\beta} \\vert\\vert_2^2 = \\lambda \\sum_{j=0}^{p-1}\\beta_j^2,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "but when we take out the intercept, this equation becomes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\lambda \\vert\\vert \\boldsymbol{\\beta} \\vert\\vert_2^2 = \\lambda \\sum_{j=1}^{p-1}\\beta_j^2.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For Lasso regression we have" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\lambda \\vert\\vert \\boldsymbol{\\beta} \\vert\\vert_1 = \\lambda \\sum_{j=1}^{p-1}\\vert\\beta_j\\vert.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It means that, when scaling the design matrix and the outputs/targets,\n", "by subtracting the mean values, we have an optimization problem which\n", "is not penalized by the intercept. The MSE value can then be smaller\n", "since it focuses only on the remaining quantities. If we however bring\n", "back the intercept, we will get a MSE which then contains the\n", "intercept.\n", "\n", "\n", "Armed with this wisdom, we attempt first to simply set the intercept equal to **False** in our implementation of Ridge regression for our well-known vanilla data set." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Beta values for own Ridge implementation\n", "[ 1.03032441e+00 6.28336218e-02 -6.24175744e-01 5.21169159e-02\n", " 2.80847477e-01 2.12552073e-01 8.13220609e-02 -1.69634577e-02\n", " -6.50846112e-02 -7.38962192e-02 -5.94226022e-02 -3.50227564e-02\n", " -9.80609614e-03 1.08299273e-02 2.41882037e-02 2.93492130e-02\n", " 2.64742912e-02 1.63249532e-02 -5.01831250e-05 -2.15098090e-02]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[ 1.03032441e+00 6.28336218e-02 -6.24175744e-01 5.21169159e-02\n", " 2.80847477e-01 2.12552073e-01 8.13220608e-02 -1.69634577e-02\n", " -6.50846112e-02 -7.38962192e-02 -5.94226022e-02 -3.50227564e-02\n", " -9.80609615e-03 1.08299273e-02 2.41882037e-02 2.93492130e-02\n", " 2.64742912e-02 1.63249532e-02 -5.01831152e-05 -2.15098090e-02]\n", "MSE values for own Ridge implementation\n", "4.3632959273186007e-07\n", "MSE values for Scikit-Learn Ridge implementation\n", "4.363295916523824e-07\n", "Beta values for own Ridge implementation\n", "[ 1.03630548 -0.01963611 -0.37900111 -0.07062318 0.12182967 0.16343471\n", " 0.13003291 0.07490892 0.02365049 -0.01449782 -0.03814292 -0.04909093\n", " -0.05009826 -0.04389027 -0.03279636 -0.01866537 -0.00289724 0.01348565\n", " 0.02976145 0.04543942]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[ 1.03630548 -0.01963611 -0.37900111 -0.07062318 0.12182967 0.16343471\n", " 0.13003291 0.07490892 0.02365049 -0.01449782 -0.03814292 -0.04909093\n", " -0.05009826 -0.04389027 -0.03279636 -0.01866537 -0.00289724 0.01348565\n", " 0.02976145 0.04543942]\n", "MSE values for own Ridge implementation\n", "5.194042826640948e-06\n", "MSE values for Scikit-Learn Ridge implementation\n", "5.194042826840599e-06\n", "Beta values for own Ridge implementation\n", "[ 1.04220758 -0.10931453 -0.17641709 -0.06020587 0.02208512 0.05789007\n", " 0.06491736 0.05785343 0.04537385 0.03196357 0.01969145 0.00934499\n", " 0.00107405 -0.00526348 -0.00992331 -0.01318643 -0.01531845 -0.01655318\n", " -0.01708852 -0.01708781]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[ 1.04220758 -0.10931453 -0.17641709 -0.06020587 0.02208512 0.05789007\n", " 0.06491736 0.05785343 0.04537385 0.03196357 0.01969145 0.00934499\n", " 0.00107405 -0.00526348 -0.00992331 -0.01318643 -0.01531845 -0.01655318\n", " -0.01708852 -0.01708781]\n", "MSE values for own Ridge implementation\n", "2.094082198961287e-05\n", "MSE values for Scikit-Learn Ridge implementation\n", "2.0940821989631478e-05\n", "Beta values for own Ridge implementation\n", "[ 1.01219292 -0.06043581 -0.10391807 -0.05651951 -0.01898855 0.00312361\n", " 0.01463049 0.01975848 0.02123176 0.02068067 0.01905883 0.01691985\n", " 0.01458337 0.01223198 0.00996754 0.00784393 0.00588657 0.00410387\n", " 0.00249435 0.00105081]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[ 1.01219292 -0.06043581 -0.10391807 -0.05651951 -0.01898855 0.00312361\n", " 0.01463049 0.01975848 0.02123176 0.02068067 0.01905883 0.01691985\n", " 0.01458337 0.01223198 0.00996754 0.00784393 0.00588657 0.00410387\n", " 0.00249435 0.00105081]\n", "MSE values for own Ridge implementation\n", "0.00031535148309579146\n", "MSE values for Scikit-Learn Ridge implementation\n", "0.00031535148309581185\n", "Beta values for own Ridge implementation\n", "[ 8.38916861e-01 1.31276579e-01 8.97497404e-03 -1.72271878e-02\n", " -2.11744554e-02 -1.91492986e-02 -1.57201944e-02 -1.23002365e-02\n", " -9.30466214e-03 -6.81048318e-03 -4.78184120e-03 -3.15130074e-03\n", " -1.84923989e-03 -8.13661243e-04 7.46984697e-06 6.56636616e-04\n", " 1.16805821e-03 1.56912044e-03 1.88168312e-03 2.12318726e-03]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[ 8.38916861e-01 1.31276579e-01 8.97497404e-03 -1.72271878e-02\n", " -2.11744554e-02 -1.91492986e-02 -1.57201944e-02 -1.23002365e-02\n", " -9.30466214e-03 -6.81048318e-03 -4.78184120e-03 -3.15130074e-03\n", " -1.84923989e-03 -8.13661243e-04 7.46984697e-06 6.56636616e-04\n", " 1.16805821e-03 1.56912044e-03 1.88168312e-03 2.12318726e-03]\n", "MSE values for own Ridge implementation\n", "0.01507238889517716\n", "MSE values for Scikit-Learn Ridge implementation\n", "0.015072388895177083\n", "Beta values for own Ridge implementation\n", "[0.37396662 0.14174745 0.0764924 0.04892055 0.03447512 0.02586427\n", " 0.02024962 0.01633913 0.01347916 0.0113104 0.0096208 0.00827728\n", " 0.00719176 0.00630331 0.00556826 0.0049544 0.00443743 0.0039987\n", " 0.0036237 0.003301 ]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[0.37396662 0.14174745 0.0764924 0.04892055 0.03447512 0.02586427\n", " 0.02024962 0.01633913 0.01347916 0.0113104 0.0096208 0.00827728\n", " 0.00719176 0.00630331 0.00556826 0.0049544 0.00443743 0.0039987\n", " 0.0036237 0.003301 ]\n", "MSE values for own Ridge implementation\n", "0.2640931530791003\n", "MSE values for Scikit-Learn Ridge implementation\n", "0.26409315307910036\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAr4ElEQVR4nO3deXyU5b338c8vIRAwCArUBRTQAsqSBAhQRFCrCMpO4gGV1qUWrVXrUan1eB61ejxi9Wld6nHpUXvqsS5NQEAQRYHiIwoJCsgigoga3NhlC2T5PX/MEIc4EAiZ3DOT77svXs69f+9A85vrvmauy9wdERGRqlKCDiAiIvFJBUJERKJSgRARkahUIEREJCoVCBERiapB0AFqS8uWLb1du3ZBxxARSSiLFi3a6O6tom1LmgLRrl07ioqKgo4hIpJQzOyzA23TIyYREYlKBUJERKJSgRARkaiSpg8imtLSUoqLiykpKQk6isSZ9PR02rRpQ1paWtBRROJWUheI4uJimjZtSrt27TCzoONInHB3Nm3aRHFxMe3btw86jkjcSupHTCUlJbRo0ULFQfZjZrRo0UItS5FqJHWBAFQcJCr9uxCpXtIXCBGRZPa3x8bzzCNXxOTcKhAxZmaMGzeucrmsrIxWrVoxdOhQAL755huGDh1KVlYWnTt35sILLwRg3bp1NG7cmOzs7Mo/f/vb3+o0e2SGzp078/Of/5zS0lIAioqKuOGGG6Ie165dOzZu3HhE13722Wcr77thw4Z069aN7Oxsfve73x3yOR566CF27dp1RDlE4plXVHDv2r/y97VTYnL+pO6kjgdHHXUUy5YtY/fu3TRu3JhZs2bRunXryu133HEHAwcO5De/+Q0AS5curdx26qmnsnjx4rqOvJ99GcrLyxk4cCAvv/wyl156KTk5OeTk5MTsuldccQVXXBF6V9SuXTvmzJlDy5YtD+scDz30EOPGjaNJkyaxiCgSuOXvTuHjo0v516aDY3J+tSDqwIUXXsj06dMBeOGFF7j44osrt3311Ve0adOmcjkzM/Owzv3WW2/RvXt3unXrxpVXXsmePXsoLCxk9OjRAEyZMoXGjRuzd+9eSkpKOOWUUwA4++yzufXWW+nduzcdO3bk7bffPuh1UlNT6d27N+vXrwdg7ty5la2gTZs2cf7559OlSxeuuuoqImcpvOeee+jUqRNnnnkmF198MQ8++CAAn3zyCYMHD6Znz57079+fjz766JDu94EHHqBXr15kZmZy5513ArBz506GDBlCVlYWXbt25aWXXuKRRx7hyy+/5JxzzuGcc845jJ+oSOIoePMRzGHkyENvWR+O+tOCuPFGqO1349nZ8NBD1e42duxY7r77boYOHcrSpUu58sorK38h//rXv2bMmDH8+c9/5rzzzuOKK67gxBNPBEK/RLOzsyvP8+ijj9K/f//K5ZKSEi6//HLeeustOnbsyM9//nMef/xxrrvuusqWx9tvv03Xrl0pLCykrKyMPn36VB5fVlbGwoULmTFjBr///e958803D3gPJSUlLFiwgIcffvgH237/+99z5plncscddzB9+nSefvppAAoLCykoKGDJkiWUlpbSo0cPevbsCcD48eN54okn6NChAwsWLODaa69l9uzZB/05vvHGG6xevZqFCxfi7gwfPpx58+axYcMGTjzxxMoivG3bNpo1a8Yf//jHGrU8RBJF/rZ36V/RjONPObw3loeq/hSIAGVmZrJu3TpeeOGFyj6GfQYNGsTatWuZOXMmr732Gt27d2fZsmVA9Y+YVq1aRfv27enYsSMAl112GY899hg33ngjp556KitXrmThwoXcdNNNzJs3j/Ly8v0KzL5WRs+ePVm3bl3Ua+wrUp9++ilDhgyJ2sKZN28ekyZNAmDIkCEcc8wxALzzzjuMGDGC9PR00tPTGTZsGAA7duxg/vz5XHTRRZXn2LNnz8F+hECoQLzxxht079698jyrV6+mf//+3Hzzzdx6660MHTp0v3sUSVarCl9jWbM9PNx4SMyuUX8KxCG804+l4cOHc8sttzB37lw2bdq037Zjjz2WSy65hEsuuYShQ4cyb968ynfaNTVgwABee+010tLSOO+887j88sspLy/ngQceqNynUaNGQOjxUVlZWdTz7CtSGzdupF+/fkydOpXhw4cfUbaKigqaN29+2P0r7s5tt93G1Vdf/YNt77//PjNmzODf//3fOffcc7njjjuOKKNIvCuY+ScARo+IzeMlUB9Enbnyyiu588476dat237rZ8+eXflJm+3bt/PJJ59w8sknH9I5O3XqxLp161izZg0Azz33HGeddRYA/fv356GHHqJv3760atWKTZs2sWrVKrp27Vqj/C1btmTixIncd999P9g2YMAA/v73vwPw2muvsWXLFgD69evHtGnTKCkpYceOHbz66qsAHH300bRv355//OMfQOgX/5IlS6rNMGjQIJ555hl27NgBwPr16/n222/58ssvadKkCePGjWPChAm8//77ADRt2pTt27fX6H5F4l3+5v9H3y0ZtOnUK2bXqD8tiIC1adMm6sdCFy1axHXXXUeDBg2oqKjgqquuolevXqxbt+4HfRBXXnnlfudIT0/n2Wef5aKLLqKsrIxevXpxzTXXANCnTx+++eYbBgwYAIQec3399ddH9AWxkSNHctddd/2gQ/vOO+/k4osvpkuXLpxxxhmVBa5Xr14MHz6czMxMjjvuOLp160azZs0AeP755/nVr37Ff/zHf1BaWsrYsWPJyso66PXPP/98Vq5cSd++fQHIyMjgf//3f1mzZg0TJkwgJSWFtLQ0Hn/8cSDUzzF48GBOPPFE5syZU+P7Fok3a5fM5YPmu3mw4dCYXsciP3FS6yc3Gww8DKQC/+3uE6tsvwm4CigDNgBXuvtn4W3lwIfhXT9394M+18jJyfGqEwatXLmS008/vTZuRWpox44dZGRksGvXLgYMGMBTTz1Fjx49go4F6N+HJK4H/nMIvy2dwae5b9Ou65lHdC4zW+TuUT+zHrMWhJmlAo8BA4FioNDMprr7iojdPgBy3H2Xmf0K+AMwJrxtt7tnxyqf1I3x48ezYsUKSkpKuOyyy+KmOIgksvwN/6QnTY64OFQnlo+YegNr3H0tgJm9CIwAKguEu0e2+98DxiFJZV/fhIjUjs9XvMvC5ju5L3VQzK8Vy07q1sAXEcvF4XUH8gvgtYjldDMrMrP3zGxktAPMbHx4n6INGzYccWARkXg3adofAMi94OaYXysuOqnNbByQA5wVsbqtu683s1OA2Wb2obt/Enmcuz8FPAWhPog6CywiEpD8r94i09Lp0HNgzK8VyxbEeuCkiOU24XX7MbPzgNuB4e5e+W0pd18f/u9aYC7QPYZZRUTi3per32d+8+3kHXNGnVwvlgWiEOhgZu3NrCEwFpgauYOZdQeeJFQcvo1Yf4yZNQq/bgn0I6LvQkSkPpo8ZSJukDfoX+vkejErEO5eBlwHvA6sBF529+VmdreZ7fvI6gNABvAPM1tsZvsKyOlAkZktAeYAE6t8+ilhxHq477PPPptOnTqRlZVFr1699vt28oUXXsjWrVt/cMxdd91VOWheTW3atKky1/HHH0/r1q0rl/fu3XtI55g7dy7z588/ohwi9UlB8SxO39aQ0/vE9vsP+8S0D8LdZwAzqqy7I+L1eQc4bj7QLdq2RFMXw30///zz5OTk8OyzzzJhwgRmzZoFwIwZM6o5suZatGhRme2uu+4iIyODW2655bDOMXfuXDIyMjjjjLppLosksm/XLeefzbZyO3U31piG2qgDsRzuO1Lfvn0rh+OG/Sfuuffee+nYsSNnnnkmq1atqtynsLCQzMxMsrOzmTBhQuVQHOXl5UyYMKFyaO0nn3zykDIsWrSIs846i549ezJo0CC++uorAB555BE6d+5MZmYmY8eOZd26dTzxxBP86U9/Ijs7u9rhxkXqu1deuY+KFMg997o6u2ZcfIqpLtw480YWf724Vs+ZfXw2Dw1+qNr9YjXcd1UzZ85k5MiRP1i/aNEiXnzxRRYvXkxZWdl+w25fccUV/OUvf6Fv3777zdb29NNP06xZMwoLC9mzZw/9+vXj/PPPp3379ge8fmlpKddffz1TpkyhVatWvPTSS9x+++0888wzTJw4kU8//ZRGjRqxdetWmjdvzjXXXFOjlodIfVSwbiY/Tk0j88y8OrtmvSkQQYrVcN/7XHrppezdu5cdO3ZE3f/tt99m1KhRlTOr7RuNdevWrWzfvr1ybKNLLrmkckC9N954g6VLl5Kfnw+E5lhYvXr1QQvEqlWrWLZsGQMHhj5+V15ezgknnFD5M7j00ksZOXJk1CImIge2+ctPmH30Jm7xn2Apdffgp94UiEN5px9LsRzu+/nnn6dnz55MmDCB66+/vnJuhiPh7jz66KMMGnTo39Z0d7p06cK77777g23Tp09n3rx5TJs2jXvvvZcPP/wwyhlEJJqpk+6jLBVy+/2qTq+rPog6EovhviOZGffccw/vvffeD6bvHDBgAK+88gq7d+9m+/btTJs2DYDmzZvTtGlTFixYAMCLL75YecygQYN4/PHHKS0tBeDjjz9m586dB83QqVMnNmzYUFkgSktLWb58ORUVFXzxxRecc8453H///Wzbto0dO3ZoOG6RQ5T/yTTabk+l50/rdjQiFYg6crDhvnNycsjMzKRv376Vw33D930Q+/488sgjB71G48aNufnmm/ebFAigR48ejBkzhqysLC644ILK80Oor+GXv/wl2dnZ7Ny5s3I47quuuorOnTvTo0cPunbtytVXX33ASYX2adiwIfn5+dx6661kZWWRnZ3N/PnzKS8vZ9y4cXTr1o3u3btzww030Lx5c4YNG8bkyZPVSS1yENu+/Zw3mn5LbsPsOn28BDEe7rsuabjvmtk3HDfAxIkT+eqrr6LOO52M9O9DEsHzT1zLuG8eZ37vJ+l7wfhaP38gw31LYpg+fTr33XcfZWVltG3blr/+9a9BRxKRCPmrXqF1gxT6nH9lnV9bBaKeGzNmDGPGjKl+RxGpczs2f83Mo77il2WZpKTW/a/rpC8Q7n5E02xKckqWR6uS3GZMup+SNMjrfVUg10/qTur09HQ2bdqkXwayH3dn06ZNpKenBx1F5KDyV+Tzo11GvwuuDuT6Sd2CaNOmDcXFxWgyIakqPT19vyFOROLNrm0bmdG4mJ+VdSY1rWEgGZK6QKSlpR30m78iIvHq9ckPsLMh5PW4PLAMSV0gREQSVcGHL9MizThr2PWBZUjqPggRkUS0Z+d3TG20jpHlHWjQMLi+MrUgRETizKzJD7K9EeR2uzTQHCoQIiJxpmDxCzRLg3NH3hRoDj1iEhGJI6Ulu5iS9gkjSk+hYeOMQLOoBSEiEkfmTH2YLelObuexQUdRgRARiSf5RX8jIw3OHzUh6Ch6xCQiEi/K9pYwOWUVQ0tOJj2jedBx1IIQEYkXb0//LzY2dvI6/kvQUQAVCBGRuJH/3rM0aQAX5N4adBRAj5hEROJCRXkZk3wFF+xuTZNmLYOOA6gFISISF+a/9hRfH1VB3gmjg45SSS0IEZE4kP/OX2hUBkNyfxd0lEpqQYiIBKyivIyCsg8ZVHI8TVucGHScSmpBiIgErPDN/6E4o5y8DiOCjrKfmBYIMxtsZqvMbI2Z/aDdZGY3mdkKM1tqZm+ZWduIbZeZ2erwn8timVNEJEgF854krRyG5d4WdJT9xKxAmFkq8BhwAdAZuNjMOlfZ7QMgx90zgXzgD+FjjwXuBPoAvYE7zeyYWGUVEQmKV1SQv+cDztveiubHta3+gDoUyxZEb2CNu691973Ai8B+7Sd3n+Puu8KL7wH75oAcBMxy983uvgWYBQyOYVYRkUB8MPcFPm1aRm77C4OO8gOxLBCtgS8ilovD6w7kF8Brh3OsmY03syIzK9K80yKSiArm/hepFTBiVHw9XoI46aQ2s3FADvDA4Rzn7k+5e46757Rq1So24UREYsQrKsjfWcg5246l5Umdgo7zA7EsEOuBkyKW24TX7cfMzgNuB4a7+57DOVZEJJEtf3cKHx9dSu7Jg4KOElUsC0Qh0MHM2ptZQ2AsMDVyBzPrDjxJqDh8G7HpdeB8Mzsm3Dl9fnidiEjSyJ/1MOYwatS/BR0lqph9Uc7dy8zsOkK/2FOBZ9x9uZndDRS5+1RCj5QygH+YGcDn7j7c3Teb2T2EigzA3e6+OVZZRUSCkP/de/SvaMZx7bsGHSWqmH6T2t1nADOqrLsj4vV5Bzn2GeCZ2KUTEQnORwtnsLzZHh5pMjToKAcUF53UIiL1TcHrfwJg9Ij4GXupKo3FJCISgILN79DXM2jdMSfoKAekFoSISB37ZPFsPmi+m7zjfxp0lINSgRARqWMF0x8EIHfYbwNOcnB6xCQiUscKNs4jhya07dIv6CgHpRaEiEgd+nzFuyxsvpO8lgOCjlItFQgRkTo0adofAMi98JaAk1RPj5hEROpQ/ldvkWXp/Lj7uUFHqZZaECIideTL1e/zzjHbyTsmvvse9lGBEBGpI5OnTAQgd9CNwQY5RHrEJCJSR/KLZ9E5pRGn94nf4TUiqQUhIlIHvl23nHnNtpLbtHfQUQ6ZWhAiInXglVfuoyIF8n56Q9BRDpkKhIhIHchf9xodUtPo1m900FEOmR4xiYjE2Kbi1cxutpncJj2xlMT5tasWhIhIjE2dfB/lKZA34NdBRzksKhAiIjGWv/ZV2qU2oMc5lwQd5bAkTltHRCQBbfv2c2Y13UBuo6yEerwEakGIiMTUtIL7KE2FvDOuCTrKYVOBEBGJoYKPp9C6QQq9B14edJTDlljtHRGRBLJ905e8lvEVuQ26kZKaeO/HEy+xiEiCmDHpfvY0gNxevwg6So2oQIiIxEjBykkcl5ZCvwuuDjpKjegRk4hIDOzatpHpjYsZzemkpjUMOk6NqAUhIhIDr09+gF0NIbfHZUFHqTEVCBGRGMj/8CVapBlnDbs+6Cg1pkdMIiK1bM/O75jW6DNGlnegQcP0oOPUmFoQIiK1bNbkB9neCPIyfxZ0lCMS0xaEmQ02s1VmtsbMfhdl+wAze9/Myswsr8q2cjNbHP4zNZY5RURqU/7iv9O8xPjpiBuDjnJEYtaCMLNU4DFgIFAMFJrZVHdfEbHb58DlwC1RTrHb3bNjlU9EJBb27t7BlLS1DC9tT8PGGUHHOSKxfMTUG1jj7msBzOxFYARQWSDcfV14W0UMc4iI1Jk5Ux9ma7qT1zmxRm6NJpaPmFoDX0QsF4fXHap0Mysys/fMbGStJhMRiZH8oufI2AsDR90cdJQjFs+d1G3dfb2ZnQLMNrMP3f2TyB3MbDwwHuDkk08OIqOISKWyvSW8kvoxw0rakp7RPOg4RyyWLYj1wEkRy23C6w6Ju68P/3ctMBfoHmWfp9w9x91zWrVqdWRpRUSO0LxXH2NjYyev678EHaVWxLJAFAIdzKy9mTUExgKH9GkkMzvGzBqFX7cE+hHRdyEiEo8KFvyVJnth8OjfBh2lVsSsQLh7GXAd8DqwEnjZ3Zeb2d1mNhzAzHqZWTFwEfCkmS0PH346UGRmS4A5wMQqn34SEYkr5aV7meQruHB3a5o0axl0nFoR0z4Id58BzKiy7o6I14WEHj1VPW4+0C2W2UREatP8mU/x9VEV5J4wOugoteagLQgzGxfxul+VbdfFKpSISKIpmP80jcpgSO4PvhOcsKp7xHRTxOtHq2y7spaziIgkpIryMgrKPmTwjuNp2uLEoOPUmuoKhB3gdbRlEZF6qfDN/6E4o5zcDiOCjlKrqisQfoDX0ZZFROql/HlPkFYOw3JvCzpKraquk/o0M1tKqLVwavg14eVTYppMRCQBeEUF+XsWM7C8Fc2Paxt0nFpVXYE4vU5SiIgkqA/mvsC6pmX8n2OGBB2l1h20QLj7Z5HLZtYCGAB87u6LYhlMRCQR5M95jNQUGDH634KOUuuq+5jrq2bWNfz6BGAZoU8vPWdmN8Y+nohI/PKKCvJ3FXHOtmNp0aZD0HFqXXWd1O3dfVn49RXALHcfBvRBH3MVkXpu2fzJrD66lLy2g4OOEhPVFYjSiNfnEv5WtLtvBzSHg4jUa/lvPkJKBYwcmVyfXtqnuk7qL8zsekJzOfQAZgKYWWMgLcbZRETiWsF3C+hf0Yzj2ncNOkpMVNeC+AXQhdC0oGPcfWt4/U+AZ2MXS0Qkvn20cAbLm+0hr/XAoKPETHWfYvoWuCbK+jmERlkVEamXCl7/EwCjRtwacJLYOWiBMLODzt/g7sNrN46ISGLI3/wOZ3hTWnfMCTpKzFTXB9GX0LzSLwAL0PhLIiJ8sng2i5vv5o+NkmvspaqqKxDHAwOBi4FLgOnAC+6+/KBHiYgksYLpDwIwetiEgJPE1kE7qd293N1nuvtlhDqm1wBzNReEiNRn+Rvn0WvrUbTt0q/6nRNYtTPKheeGHkKoFdEOeASYHNtYIiLx6bPl71DYfCcTGyTnl+MiVddJ/TegK6EvyP0+4lvVIiL10qRpDwCQe+EtASeJvepaEOOAncBvgBvMKvuoDXB3PzqG2URE4k7+17PJtsb8uPu5QUeJueq+B1HdF+lEROqN9R8XMf+Y7dxjyV8coPpvUouISNjkKfcDkDf4poCT1I1qO6lFRCSkYP2bdE5pxGm9Lww6Sp1QC0JE5BB88+ky5jXbSt7RfYKOUmfUghAROQSvvHIfFSmQd+4NQUepMyoQIiKHoOCzmXRITaPrGaOCjlJn9IhJRKQam4pXM7vZZvKa5GAp9efXploQIiLVmDr5PspTIHfAtUFHqVMqECIi1chf+yrtUhvQ45xLgo5Sp2LaVjKzwWa2yszWmNnvomwfYGbvm1mZmeVV2XaZma0O/7ksljlFRA5k6zefMavpBvIaZderx0sQwxaEmaUCjxEaLrwYKDSzqe6+ImK3zwlNZ3pLlWOPBe4EcgAHFoWP3RKrvCIi0bw6aSKlqZB7xtVBR6lzsSyHvYE17r7W3fcCLwL7za7h7uvcfSlQUeXYQcAsd98cLgqzgOQfOlFE4k7+x6/QZkcqvQdeHnSUOhfLAtGa0Gx0+xSH19XasWY23syKzKxow4YNNQ4qIhLN9k1fMjPja0Y36EpKav3rsk3oB2ru/pS757h7TqtWrYKOIyJJZsak+9nTAPLOuCroKIGIZYFYD5wUsdwmvC7Wx4qI1Ir8lQUcvzOFMwaPDzpKIGJZIAqBDmbW3swaAmOBqYd47OvA+WZ2jJkdA5wfXiciUid2bdvIjMbrGWWnk5rWMOg4gYhZgXD3MuA6Qr/YVwIvu/tyM7vbzIYDmFkvMysGLgKeNLPl4WM3A/cQKjKFwN3hdSIidWLmpD+wqyHk9bki6CiBMXcPOkOtyMnJ8aKioqBjiEiSuPTmdrye9jlf372LBg3Tg44TM2a2yN1zom1L6E5qEZFYKNmxlWmNPmNUecekLg7VqX+f2xIRqcasyf+X7Y0gL+tnQUcJlAqEiEgVBUteoHmacc7w3wQdJVB6xCQiEmHv7h1MSVvLiNJTaNg4I+g4gVILQkQkwpypD7M13cnrUr9Gbo1GBUJEJEJ+0XM0TYOBo26pfuckp0dMIiJhZXtLmJz6McP2tKXRUUcHHSdwakGIiITNe/UxNjV2cjv9S9BR4oIKhIhIWP6CZ2nSAAaP/m3QUeKCHjGJiADlpXuZxEqG7G5Dk2Ytg44TF9SCEBEB5s98im+aVJB74uigo8QNFQgRESB//n+TngoXjr416ChxQwVCROq9ivIyJpUtY1DJ8TRtcWLQceKG+iBEpN5bOOuvFGeUk9dxZNBR4ooKhIjUe/lvP0FaOQzLvS3oKHFFj5hEpF7zigoK9ixhYHkrmv3o5KDjxBW1IESkXnt/zt9Z17SMvFOGBh0l7qhAiEi9VjD3v0itgOGj9HipKj1iEpF6yysq+MeuIn5afiwt2nQIOk7cUQtCROqtD9+ZxJqjS8lrd0HQUeKSWhAiUm8VvPUoKQ4jR+rxUjQqECJSb+V/t4ABFc35UbsuQUeJS3rEJCL10soFr7Ki2R5yW58XdJS4pQIhIvVSwesPATBaj5cOSI+YRKReyt/yDv28KSd26BF0lLilFoSI1DtrPniLJc1LyD3+p0FHiWsqECJS7xTMeBCA3OEa2vtg9IhJROqdgo1v04ujOLlz36CjxLWYtiDMbLCZrTKzNWb2uyjbG5nZS+HtC8ysXXh9OzPbbWaLw3+eiGVOEak/Plv+DoXNd5LXckDQUeJezFoQZpYKPAYMBIqBQjOb6u4rInb7BbDF3X9sZmOB+4Ex4W2fuHt2rPKJSP1UMO0PAOQOuSXgJPEvli2I3sAad1/r7nuBF4ERVfYZAfxP+HU+cK6ZWQwziUg9V/D1HLK3NubUbHVQVyeWBaI18EXEcnF4XdR93L0M2Aa0CG9rb2YfmNk/zax/tAuY2XgzKzKzog0bNtRuehFJOus/LmL+MdvJO7Zf0FESQrx+iukr4GR37w7cBPzdzI6uupO7P+XuOe6e06pVqzoPKSKJZfKU+wHIHfSvASdJDLEsEOuBkyKW24TXRd3HzBoAzYBN7r7H3TcBuPsi4BOgYwyzikg9kL9+Fl22NeK03hcGHSUhxLJAFAIdzKy9mTUExgJTq+wzFbgs/DoPmO3ubmatwp3cmNkpQAdgbQyzikiS++bTZcxrvo28o38SdJSEEbNPMbl7mZldB7wOpALPuPtyM7sbKHL3qcDTwHNmtgbYTKiIAAwA7jazUqACuMbdN8cqq4gkv1deuQ83yD33+qCjJAxz96Az1IqcnBwvKioKOoaIxKmBN7bg89TtfPRACZYSr92vdc/MFrl7TrRt+imJSNLbVLyaOc02k9skR8XhMGioDRFJelMm/SflKZB31q+DjpJQVCBEJOnlfzqd9qkN6H72xUFHSShqa4lIUtv6zWe82XQDuY2y9XjpMKkFISJJbVrBfZSmQl6/a4KOknBUIEQkqRWsnsJJDVLpPfCKoKMkHLW3RCRpbd/0JTMzvmZ0g256vFQDakGISNKaXjCRPQ0gr88vg46SkFQgRCRpFXw0ieMbpHDGBeODjpKQ1OYSkaS0a9tGZjRez2jrTEqq3gvXhH5qIpKUZk76A7saQl6OOqdrSgVCRJJS/rKXaZlm9B9ybdBREpYeMYlI0inZsZVp6Z8xqqITDRqmBx0nYakFISJJZ9bk/8uOhpCbOS7oKAlNBUJEkk7+kr9zTJrx0xGaWvRI6BGTiCSVvbt3MDXtU0aUnkpaepOg4yQ0tSBEJKnMnvIQW9Od3C4aufVIqUCISFLJX/QcTdNg4Khbgo6S8PSISUSSRtneEl5JXc2wPW1pdNTRQcdJeGpBiEjS+Oe0R9nU2Mk7bUzQUZKCCoSIJByvqODLNe+zdNEMlq59lyWbV7LUv+ajpnvIKINBoyYEHTEpqECISFwr2bGVFQuns2T5Wyz9cjFLdn/K0vRtbGrslfu0tVQyK1oy0n/M6HN/RZNmLQNMnDxUIEQkLnhFBetXL/q+VbBlJUv9G1Y13UN5uLe0cSp0s6MYXdGJzCZdyTrtLLr1Hkbz49oGGz5JqUCISJ3b/d1mVhTOCLUKvlrMkt3rWJq+jc0RrYJ21oDMipaM5sdktelNZtb5nJp1DqlpDQNMXr+oQIhIzHhFBcWrCln6/mssWTufpVs+YinfsKrpXirCrYImKaFWQV5FJzKP6kbWaWfTrfdQmv3o5GDDiwqEiNSO3d9tZvnCV1myYvb3rYLG37ElPaJVkNKArPKW5NGBrJP6hFoF2edovoY4pb8VETksXlHBFx8tCLUKPn2XpVtWscS+YXXG962Co1Kgm2XwL+WnkZmRSdZpZ9G11xC1ChKMCoSIHNCubRtZtmAaS1fODbUKSj5jaePv2BrRKmif0oCs8laMsQ5kndyHzOxBnJJ5lloFSUB/gyKCV1Tw+cp3Wfr+zFCrYGu4VdC0FLfQPhmp0I0MxlacTmZGJpmnDaBb72Ec3apNsOElZmJaIMxsMPAwkAr8t7tPrLK9EfA3oCewCRjj7uvC224DfgGUAze4++uxzCpSX+zc8i3LFk5j6cp/suTrxSwNtwq2Rcyrc0q4VXAxHclqG2oVtO82QK2CeiZmf9tmlgo8BgwEioFCM5vq7isidvsFsMXdf2xmY4H7gTFm1hkYC3QBTgTeNLOO7l4eq7wi8cIrKnCvOPT/RrwGcPfK5W0b1/PhktdZ8ul74VbBt6yp0irIpCmXVHQhs2kmWaefTdfeQ2na4sQAfwISL2L5dqA3sMbd1wKY2YvACCCyQIwA7gq/zgf+bGYWXv+iu+8BPjWzNeHzvVvbITd/+QlnPti5tk8b97z6XQ7vfFbLJwS8llN65R/HLfoyB9rHwusqX1PltVf+DA64z8H+G4OfX1WnWgOy/Edcah3JavcTMrMH0a7rmWoVyAHF8l9Ga+CLiOVioM+B9nH3MjPbBrQIr3+vyrGtq17AzMYD4wFOPrlmn45okNaIrvyoRscmOqMWfys5tXm2SqH3C7V4vvBdm4demf1wObRflX2ciNcHX//9dQyziPMc7H++L4tV3vf3+ar5b+XrlO+vHV7fpGEGXTudSbc+w8k49vha/VlK8kvotw7u/hTwFEBOTk6N3m4e3aoNL//xi+p3FBGpZ2I5H8R64KSI5TbhdVH3MbMGQDNCndWHcqyIiMRQLAtEIdDBzNqbWUNCnc5Tq+wzFbgs/DoPmO3uHl4/1swamVl7oAOwMIZZRUSkipg9Ygr3KVwHvE7oY67PuPtyM7sbKHL3qcDTwHPhTujNhIoI4f1eJtShXQb8Wp9gEhGpWxZ6w574cnJyvKioKOgYIiIJxcwWuXtOtG2ak1pERKJSgRARkahUIEREJCoVCBERiSppOqnNbAPw2RGcoiWwsZbiBClZ7gN0L/EqWe4lWe4Djuxe2rp7q2gbkqZAHCkzKzpQT34iSZb7AN1LvEqWe0mW+4DY3YseMYmISFQqECIiEpUKxPeeCjpALUmW+wDdS7xKlntJlvuAGN2L+iBERCQqtSBERCQqFQgREYlKBaIKM7vZzNzMWgadpabM7B4zW2pmi83sDTNL2AmGzewBM/sofD+Tzax50JlqyswuMrPlZlZhZgn38UozG2xmq8xsjZn9Lug8NWVmz5jZt2a2LOgsR8rMTjKzOWa2Ivxv6ze1eX4ViAhmdhJwPvB50FmO0APununu2cCrwB0B5zkSs4Cu7p4JfAzcFnCeI7EMGA3MCzrI4TKzVOAx4AKgM3CxmSXqZO5/BQYHHaKWlAE3u3tn4CfAr2vz70UFYn9/An7L9/PXJyR3/y5i8SgS+H7c/Q13LwsvvkdodsGE5O4r3X1V0DlqqDewxt3Xuvte4EVgRMCZasTd5xGafybhuftX7v5++PV2YCXQurbOn9BzUtcmMxsBrHf3Jfsmrk9kZnYv8HNgG3BOwHFqy5XAS0GHqKdaA5GTtxcDfQLKIlGYWTugO7Cgts5ZrwqEmb0JHB9l0+3AvxF6vJQQDnYv7j7F3W8Hbjez24DrgDvrNOBhqO5ewvvcTqg5/XxdZjtch3IvIrXNzDKAAuDGKk8Qjki9KhDufl609WbWDWgP7Gs9tAHeN7Pe7v51HUY8ZAe6lyieB2YQxwWiunsxs8uBocC5Hudf3DmMv5dEsx44KWK5TXidBMzM0ggVh+fdfVJtnrteFYgDcfcPgR/tWzazdUCOuyfkSI9m1sHdV4cXRwAfBZnnSJjZYEL9Qme5+66g89RjhUAHM2tPqDCMBS4JNpJY6B3t08BKd/9jbZ9fndTJaaKZLTOzpYQem9XqR9/q2J+BpsCs8Md2nwg6UE2Z2SgzKwb6AtPN7PWgMx2q8AcFrgNeJ9QR+rK7Lw82Vc2Y2QvAu0AnMys2s18EnekI9AN+Bvw0/P+PxWZ2YW2dXENtiIhIVGpBiIhIVCoQIiISlQqEiIhEpQIhIiJRqUCIiEhUKhCSlMxsxxEce114xNL9RvW1kEfC25aaWY+IbSeY2avh12fve32kzGzuoYz8ambrqhuB2MzeNLNjaiOX1A8qECI/9A5wHvBZlfUXAB3Cf8YDj0dsuwn4S52kq7nngGuDDiGJQwVCklr4Xf8D4S8OfmhmY8LrU8zsv8JzTcwysxlmlgfg7h+4+7oopxsB/M1D3gOam9kJ4W25wMwo1+9tZu+a2QdmNt/MOoXXX25mr4SvvS7carkpvN97ZnZsxGl+Fv4C1DIz6x0+vkV4ro/lZvbfgEVc8xUzWxTeNj7iPFOBi2v6s5T6RwVCkt1oIBvIItQqeCD8S3000I7Q3AY/I/Tt5upEG9G0dXj4iS3uvifKMR8B/d29O6F5Of4zYlvXcI5ewL3ArvB+7xIaiXefJuG5Pa4FngmvuxP4f+7eBZgMnByx/5Xu3hPIAW4wsxYA7r4FaLRvWaQ6GotJkt2ZwAvuXg58Y2b/JPQL+UzgH+5eAXxtZnOO4BonABsOsK0Z8D9m1oHQvBxpEdvmhMfw325m24Bp4fUfApkR+70AoXkMzOxoC82qN4BQccHdp5vZloj9bzCzUeHXJxF6JLYpvPwtcGLEssgBqQUhcugONKLpbiD9AMfcQ6gQdAWGVdkvssVREbFcwf5v3qqOh3PA8XHM7GxCLaW+7p4FfFDlmunhvCLVUoGQZPc2MMbMUs2sFaF33gsJdUTnhvsijgPOPoRzTQV+Hu7X+Amwzd2/IjQVarsDHNOM74fFvryG97Cv3+TM8DW3EZq29JLw+guAfZ9OakbocdcuMzuN0DSUhPczQnNVrKthDqlnVCAk2U0GlgJLgNnAb8NzfBQQ6kNYAfwv8D6h2fcwsxvCo662AZaGO4EhNK/GWmANoU8sXQvg7juBT8zsx1Gu/wfgPjP7gJo/0i0JH/8EsG/k0d8DA8xsOaFHTfvmUZ8JNDCzlcBEQtO07tMTeC9iCleRg9JorlJvmVmGu+8Id9ouBPrVdIKo8DP/nu7+77UashaZ2cPAVHd/K+gskhjUSS312avhDt+GwD1HMnugu09OgE8HLVNxkMOhFoSIiESlPggREYlKBUJERKJSgRARkahUIEREJCoVCBERier/A5PsezvJwAMMAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_117_1.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from sklearn.model_selection import train_test_split\n", "from sklearn import linear_model\n", "\n", "def MSE(y_data,y_model):\n", " n = np.size(y_model)\n", " return np.sum((y_data-y_model)**2)/n\n", "\n", "\n", "# A seed just to ensure that the random numbers are the same for every run.\n", "# Useful for eventual debugging.\n", "np.random.seed(3155)\n", "\n", "n = 100\n", "x = np.random.rand(n)\n", "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)\n", "\n", "Maxpolydegree = 20\n", "X = np.zeros((n,Maxpolydegree))\n", "#We include explicitely the intercept column\n", "for degree in range(Maxpolydegree):\n", " X[:,degree] = x**degree\n", "# We split the data in test and training data\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n", "\n", "p = Maxpolydegree\n", "I = np.eye(p,p)\n", "# Decide which values of lambda to use\n", "nlambdas = 6\n", "MSEOwnRidgePredict = np.zeros(nlambdas)\n", "MSERidgePredict = np.zeros(nlambdas)\n", "lambdas = np.logspace(-4, 2, nlambdas)\n", "for i in range(nlambdas):\n", " lmb = lambdas[i]\n", " OwnRidgeBeta = np.linalg.pinv(X_train.T @ X_train+lmb*I) @ X_train.T @ y_train\n", " # Note: we include the intercept column and no scaling\n", " RegRidge = linear_model.Ridge(lmb,fit_intercept=False)\n", " RegRidge.fit(X_train,y_train)\n", " # and then make the prediction\n", " ytildeOwnRidge = X_train @ OwnRidgeBeta\n", " ypredictOwnRidge = X_test @ OwnRidgeBeta\n", " ytildeRidge = RegRidge.predict(X_train)\n", " ypredictRidge = RegRidge.predict(X_test)\n", " MSEOwnRidgePredict[i] = MSE(y_test,ypredictOwnRidge)\n", " MSERidgePredict[i] = MSE(y_test,ypredictRidge)\n", " print(\"Beta values for own Ridge implementation\")\n", " print(OwnRidgeBeta)\n", " print(\"Beta values for Scikit-Learn Ridge implementation\")\n", " print(RegRidge.coef_)\n", " print(\"MSE values for own Ridge implementation\")\n", " print(MSEOwnRidgePredict[i])\n", " print(\"MSE values for Scikit-Learn Ridge implementation\")\n", " print(MSERidgePredict[i])\n", "\n", "# Now plot the results\n", "plt.figure()\n", "plt.plot(np.log10(lambdas), MSEOwnRidgePredict, 'r', label = 'MSE own Ridge Test')\n", "plt.plot(np.log10(lambdas), MSERidgePredict, 'g', label = 'MSE Ridge Test')\n", "\n", "plt.xlabel('log10(lambda)')\n", "plt.ylabel('MSE')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The results here agree when we force **Scikit-Learn**'s Ridge function to include the first column in our design matrix.\n", "We see that the results agree very well. Here we have thus explicitely included the intercept column in the design matrix.\n", "What happens if we do not include the intercept in our fit?\n", "Let us see how we can change this code by zero centering." ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Beta values for own Ridge implementation\n", "[ 3.43579948e-02 -5.43330971e-01 -3.10141414e-03 2.47116868e-01\n", " 2.18613217e-01 1.02054837e-01 -4.25617662e-04 -5.90475506e-02\n", " -7.68534263e-02 -6.68929213e-02 -4.24906604e-02 -1.40927184e-02\n", " 1.11482289e-02 2.88529063e-02 3.67047975e-02 3.38135733e-02\n", " 2.02198702e-02 -3.46383924e-03 -3.63025821e-02]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[ 3.43579948e-02 -5.43330971e-01 -3.10141413e-03 2.47116868e-01\n", " 2.18613217e-01 1.02054837e-01 -4.25617658e-04 -5.90475506e-02\n", " -7.68534263e-02 -6.68929213e-02 -4.24906604e-02 -1.40927184e-02\n", " 1.11482289e-02 2.88529063e-02 3.67047975e-02 3.38135733e-02\n", " 2.02198702e-02 -3.46383925e-03 -3.63025821e-02]\n", "Intercept from own implementation:\n", "1.0330308045181225\n", "Intercept from Scikit-Learn Ridge implementation\n", "1.033030804518383\n", "MSE values for own Ridge implementation\n", "3.139255958275475e-06\n", "MSE values for Scikit-Learn Ridge implementation\n", "3.139255958572018e-06\n", "Beta values for own Ridge implementation\n", "[-0.05807125 -0.29822833 -0.08551306 0.08156108 0.13679863 0.12333649\n", " 0.08251519 0.03815288 0.00111756 -0.02498832 -0.04010697 -0.04566964\n", " -0.04355837 -0.03562355 -0.02348765 -0.00848904 0.00831018 0.0260906\n", " 0.04423486]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[-0.05807125 -0.29822833 -0.08551306 0.08156108 0.13679863 0.12333649\n", " 0.08251519 0.03815288 0.00111756 -0.02498832 -0.04010697 -0.04566964\n", " -0.04355837 -0.03562355 -0.02348765 -0.00848904 0.00831018 0.0260906\n", " 0.04423486]\n", "Intercept from own implementation:\n", "1.0411487294305548\n", "Intercept from Scikit-Learn Ridge implementation\n", "1.0411487294305266\n", "MSE values for own Ridge implementation\n", "1.9601304850163794e-05\n", "MSE values for Scikit-Learn Ridge implementation\n", "1.9601304850085328e-05\n", "Beta values for own Ridge implementation\n", "[-0.1416398 -0.14021063 -0.05383795 0.01367553 0.04784395 0.05796251\n", " 0.05447415 0.044613 0.03267527 0.02098261 0.01066519 0.00217499\n", " -0.00440346 -0.00917248 -0.01231917 -0.01405935 -0.0146081 -0.01416528\n", " -0.01290947]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[-0.1416398 -0.14021063 -0.05383795 0.01367553 0.04784395 0.05796251\n", " 0.05447415 0.044613 0.03267527 0.02098261 0.01066519 0.00217499\n", " -0.00440346 -0.00917248 -0.01231917 -0.01405935 -0.0146081 -0.01416528\n", " -0.01290947]\n", "Intercept from own implementation:\n", "1.0495569966278282\n", "Intercept from Scikit-Learn Ridge implementation\n", "1.0495569966278269\n", "MSE values for own Ridge implementation\n", "5.4959161509370406e-05\n", "MSE values for Scikit-Learn Ridge implementation\n", "5.4959161509366834e-05\n", "Beta values for own Ridge implementation\n", "[-0.13535942 -0.08593216 -0.03568439 -0.0036367 0.01397146 0.02229529\n", " 0.02503753 0.0245528 0.02228115 0.01908936 0.01549377 0.01179792\n", " 0.00817631 0.00472512 0.00149311 -0.00149956 -0.00424967 -0.00676387\n", " -0.00905423]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[-0.13535942 -0.08593216 -0.03568439 -0.0036367 0.01397146 0.02229529\n", " 0.02503753 0.0245528 0.02228115 0.01908936 0.01549377 0.01179792\n", " 0.00817631 0.00472512 0.00149311 -0.00149956 -0.00424967 -0.00676387\n", " -0.00905423]\n", "Intercept from own implementation:\n", "1.039967668952797\n", "Intercept from Scikit-Learn Ridge implementation\n", "1.0399676689527975\n", "MSE values for own Ridge implementation\n", "7.571105947979326e-05\n", "MSE values for Scikit-Learn Ridge implementation\n", "7.57110594797945e-05\n", "Beta values for own Ridge implementation\n", "[-0.05100875 -0.04063602 -0.02723445 -0.01713366 -0.0100706 -0.00517114\n", " -0.00174276 0.00068734 0.00243186 0.00369758 0.00462287 0.0053018\n", " 0.00579953 0.006162 0.00642221 0.00660427 0.00672607 0.0068011\n", " 0.00683964]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[-0.05100875 -0.04063602 -0.02723445 -0.01713366 -0.0100706 -0.00517114\n", " -0.00174276 0.00068734 0.00243186 0.00369758 0.00462287 0.0053018\n", " 0.00579953 0.006162 0.00642221 0.00660427 0.00672607 0.0068011\n", " 0.00683964]\n", "Intercept from own implementation:\n", "0.999955585168597\n", "Intercept from Scikit-Learn Ridge implementation\n", "0.999955585168597\n", "MSE values for own Ridge implementation\n", "0.0007698473260556339\n", "MSE values for Scikit-Learn Ridge implementation\n", "0.000769847326055633\n", "Beta values for own Ridge implementation\n", "[-0.00834567 -0.00803064 -0.00673407 -0.00554552 -0.00458878 -0.0038335\n", " -0.00323332 -0.00274989 -0.0023548 -0.00202756 -0.00175331 -0.00152117\n", " -0.001323 -0.0011526 -0.00100519 -0.00087697 -0.00076495 -0.00066668\n", " -0.00058016]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[-0.00834567 -0.00803064 -0.00673407 -0.00554552 -0.00458878 -0.0038335\n", " -0.00323332 -0.00274989 -0.0023548 -0.00202756 -0.00175331 -0.00152117\n", " -0.001323 -0.0011526 -0.00100519 -0.00087697 -0.00076495 -0.00066668\n", " -0.00058016]\n", "Intercept from own implementation:\n", "0.9637117593816477\n", "Intercept from Scikit-Learn Ridge implementation\n", "0.9637117593816477\n", "MSE values for own Ridge implementation\n", "0.0023813163025848865\n", "MSE values for Scikit-Learn Ridge implementation\n", "0.002381316302584886\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEKCAYAAAA1qaOTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA0tUlEQVR4nO3dd3xUVdrA8d+TDoQOIp2ICIYWpIkURUBRIthYiUoRfLHAsor6CjZEdC0oYAHWBjaWIliiUgTFF1dUitKLFFFBpISWEFLnef+YSzaEFAgzuZPk+X4+89m595x77nNGdp6ce+6cK6qKMcYYc66C3A7AGGNMyWAJxRhjjE9YQjHGGOMTllCMMcb4hCUUY4wxPmEJxRhjjE/4NaGISE8R2Soi20VkVC7l4SIy2yn/UUQaZCsb7ezfKiJXO/vqishSEdkkIhtF5B/Z6j8pIntEZI3zutaffTPGGHMq8dfvUEQkGPgF6AHsBlYCcaq6KVude4EWqnq3iPQDblDVW0QkGpgJtANqAUuAi4DzgJqq+pOIlAdWA9er6iYReRJIUtUX/dIhY4wx+fLnCKUdsF1Vd6pqGjAL6JOjTh/gXef9XKCbiIizf5aqpqrqr8B2oJ2q7lXVnwBUNRHYDNT2Yx+MMcacoRA/tl0b+CPb9m6gfV51VDVDRI4CVZ39P+Q49pTE4VweawX8mG33cBEZAKwCHlDVwzmDEpGhwFCAcuXKtW7SpMlZd8wYY0qz1atXH1TV6jn3+zOh+I2IRALzgPtU9ZizeyowDlDnf18CBuc8VlXfAN4AaNOmja5atapIYjbGmJJCRH7Lbb8/L3ntAepm267j7Mu1joiEABWBhPyOFZFQvMlkhqp+dLKCqu5T1UxV9QBv4r3kZowxpoj4M6GsBBqJSJSIhAH9gPgcdeKBgc77m4Gv1XuXQDzQz7kLLApoBKxw5lfeBjar6oTsDYlIzWybNwAbfN4jY4wxefLbJS9nTmQ4sAgIBqap6kYReQpYparxeJPD+yKyHTiEN+ng1JsDbAIygGGqmikinYD+wHoRWeOc6hFVnQ+8ICIxeC957QLu8lffjDHGnM5vtw0XB7nNoaSnp7N7925SUlJcisoEooiICOrUqUNoaKjboRjjOhFZraptcu4vlpPy/rR7927Kly9PgwYN8F5hM6WdqpKQkMDu3buJiopyOxxjApYtvZJDSkoKVatWtWRisogIVatWtVGrMQWwhJILSyYmJ/s3YUzBLKEYY0wpkpHp8VvbllACkIhw++23Z21nZGRQvXp1YmNjAdi3bx+xsbG0bNmS6Ohorr3Wuw7mrl27KFOmDDExMVmv9957r0hjzx5DdHQ0AwYMID09HYBVq1YxYsSIXI9r0KABBw8ePKdzT58+PavfYWFhNG/enJiYGEaNOm1d0jxNmjSJ5OTkc4rDmED2697D3DP1A3YfOFZw5bOlqqX21bp1a81p06ZNp+0rauXKldOWLVtqcnKyqqrOnz9fW7Zsqb169VJV1aFDh+qkSZOy6q9du1ZVVX/99Vdt2rRp0QecTfYYMjIytGvXrvrBBx8UeFz9+vX1wIEDPoujsO3ld1wg/NswJhDg/enHad+pNkIJUNdeey1ffPEFADNnziQuLi6rbO/evdSpUydru0WLFmfV9ldffUWrVq1o3rw5gwcPJjU1lZUrV3LjjTcC8Omnn1KmTBnS0tJISUnhggsuAOCKK67g4Ycfpl27dlx00UV8++23+Z4nODiYdu3asWePd4GEb775JmuUlZCQwFVXXUXTpk2588470Wy3r48bN47GjRvTqVMn4uLiePFF7wLSO3bsoGfPnrRu3ZrOnTuzZcuWM+rv+PHjadu2LS1atGDMmDEAHD9+nF69etGyZUuaNWvG7NmzeeWVV/jzzz/p2rUrXbt2PYtP1Jji4fX5y5n6xXd+a98SSgGuuOL015Qp3rLk5NzL33nHW37w4OllZ6pfv37MmjWLlJQU1q1bR/v2/11Xc9iwYQwZMoSuXbvyzDPP8Oeff2aV7dix45RLXjm/9FNSUhg0aBCzZ89m/fr1ZGRkMHXqVFq1asWaNWsA+Pbbb2nWrBkrV67kxx9/POXcGRkZrFixgkmTJjF27Nh8+5CSksKPP/5Iz549TysbO3YsnTp1YuPGjdxwww38/vvvAKxcuZJ58+axdu1aFixYQPbfCQ0dOpRXX32V1atX8+KLL3LvvfcW+Dl++eWXbNu2jRUrVrBmzRpWr17NsmXLWLhwIbVq1WLt2rVs2LCBnj17MmLECGrVqsXSpUtZunRpgW0bU9w8+OWDjFjan7T0TL+0b79DCVAtWrRg165dzJw5M2uO5KSrr76anTt3snDhQhYsWECrVq3YsMG70kzDhg2zEkNutm7dSlRUFBdddBEAAwcOZPLkydx33300bNiQzZs3s2LFCkaOHMmyZcvIzMykc+fOWcefHMW0bt2aXbt25XqOk0nt119/pVevXrmOoJYtW8ZHH3mXYuvVqxeVK1cG4LvvvqNPnz5EREQQERHBddddB0BSUhLLly+nb9++WW2kpqbm9xEC3oTy5Zdf0qpVq6x2tm3bRufOnXnggQd4+OGHiY2NPaWPxpREU7/4jqTK33Nz2VcJCw32yzksoRTgm2/yLitbNv/yatXyLy9I7969efDBB/nmm29ISEg4paxKlSrceuut3HrrrcTGxrJs2TJat25d+JMBXbp0YcGCBYSGhtK9e3cGDRpEZmYm48ePz6oTHh4OeC9nZWRk5NrOyaR28OBBOnbsSHx8PL179z6n2DweD5UqVco3WeZGVRk9ejR33XX6Sjw//fQT8+fP57HHHqNbt2488cQT5xSjMYFs3JIXkfAqTB5xh9/OYZe8AtjgwYMZM2YMzZs3P2X/119/nXUnUmJiIjt27KBevXpn1Gbjxo3ZtWsX27dvB+D999/n8ssvB6Bz585MmjSJDh06UL16dRISEti6dSvNmjUrVPzVqlXjueee49lnnz2trEuXLvz73/8GYMGCBRw+7H10TceOHfnss89ISUkhKSmJzz//HIAKFSoQFRXFhx9+CHgTxdq1awuM4eqrr2batGkkJSUBsGfPHvbv38+ff/5J2bJluf3223nooYf46aefAChfvjyJiYmF6q8xgWrRql/YW/FTOobfy3mVy/ntPDZCCWB16tTJ9Tbb1atXM3z4cEJCQvB4PNx55520bduWXbt2ZV1uOmnw4MGntBEREcH06dPp27cvGRkZtG3blrvvvhuA9u3bs2/fPrp06QJ4L7v99ddf5/Sjvuuvv54nn3zytLmcMWPGEBcXR9OmTbnsssuyEmLbtm3p3bs3LVq0oEaNGjRv3pyKFSsCMGPGDO655x6efvpp0tPT6devHy1btsz3/FdddRWbN2+mQ4cOAERGRvLBBx+wfft2HnroIYKCgggNDWXq1KmAd56mZ8+eWXMpxpQE3235haDjtZh673C/nscWh8yxOOTmzZu5+OKLXYrIgHeeIzIykuTkZLp06cIbb7zBJZdc4nZY9m/DFGtp6Zk+mzvJa3FIu+RlAs7QoUOJiYnhkksu4aabbgqIZGJMcTV/xRafJpP82CUvE3BOzq0YY87NwaPJxH7UiSZzb2bTC//y+/ksoRhjTAn197feRcskcO8ltxXJ+SyhGGNMCZSWnsm8PydQztOOe3t1KpJzWkIxxpgS6PEZ8aRX2M7f63xIUFDRPH7BEooxxpRAM9fNJiQoimcH3FBk57S7vAKQv5ev//zzz2nVqlXW8a+//joATz75ZNZCjHlp0KABzZs3p0WLFlx++eX89ttvWWWXXXZZrscMGjSIuXPnnt2HkMP69euz+lSlShWioqKIiYmhe/fuZ9zGJ598wqZNm84pDmOKi+3Pz2Bx/yVFcnfXSTZCCUDlypVjw4YNnDhxgjJlyrB48WJq166dVf7EE0/Qo0cP/vGPfwCwbt26rLKC1vJKT09n6NChrFixgjp16pCamprnmlx5Wbp0KdWqVWPMmDE8/fTTvPnmmwAsX778rNo5G82bN8/q16BBg4iNjeXmm28+qzY++eQTYmNjiY6O9kOExgSO5JR0ykaEckXLC4r0vDZCCVD+Wr4+MTGRjIwMqlatCnjX5mrcuHGhYuzQoUPW0vTg/RU6eJdFGT58OI0bN6Z79+7s378/q878+fNp0qQJrVu3ZsSIEVmjruPHjzN48GDatWtHq1at+PTTT88ohi+//JIOHTpwySWX0Ldv36wlVkaNGkV0dDQtWrTgwQcfZPny5cTHx/PQQw8RExPDjh07CtVnYwLdolW/UH5MXSZ8XPQrPdgIpQBXvHPFafv+1vRv3Nv2XpLTk7l2xrWnlQ+KGcSgmEEcTD7IzXNO/Sv6m0HfnNF5+/Xrx1NPPUVsbCzr1q1j8ODBWcuXDBs2jFtuuYXXXnuN7t27c8cdd1CrVi2A05ZeefXVV09ZSbdKlSr07t2b+vXr061bN2JjY4mLiyMo6Oz/tli4cCHXX3/9afs//vhjtm7dyqZNm9i3bx/R0dEMHjyYlJQU7rrrLpYtW0ZUVNQpSfKZZ57hyiuvZNq0aRw5coR27drRvXt3ypXLe92hgwcP8vTTT7NkyRLKlSvH888/z4QJExg2bBgff/wxW7ZsQUQ4cuQIlSpVonfv3oUa2RhTnIz8cCKesCNcFVP0I3FLKAHKX8vXA7z11lusX7+eJUuW8OKLL7J48WLeOfkQlzPQtWtXDh06RGRkJOPGjTutfNmyZcTFxREcHEytWrW48sorAdiyZQsXXHABUVFRAMTFxfHGG28A3pFGfHx81hxOSkoKv//+e75Lnfzwww9s2rSJjh07ApCWlkaHDh2oWLEiERERDBkyhNjY2KxRkDEl3cZd+9kU+g5NUgfQLKpGkZ/fEkoB8htRlA0tm295tbLVznhEkht/Ll/fvHlzmjdvTv/+/YmKijqrhLJ06VIqVarEbbfdxpgxY5gwYcIZH5sXVWXevHlndflNVenRowczZ848rWzFihV89dVXzJ07l9dee42vv/76nGM0JtANe2cKhKbwUu+Rrpzf5lACmD+Wr09KSuKbbA9pWbNmDfXr1z/r2EJCQpg0aRLvvfcehw4dOqWsS5cuzJ49m8zMTPbu3Zu1am/jxo3ZuXNn1k0As2fPzjrm6quv5tVXX816FPDPP/9cYAyXXnop3333XdZS/MePH+eXX34hKSmJo0ePcu211zJx4sSsZe5taXpTkh1JSmFZymRqHLmOa9s1cSUGSygBLL/l69u0aUOLFi3o0KFD1vL1cPojgF955ZVTjlVVXnjhBRo3bkxMTAxjxow5ZXTy9NNPU6dOnaxXfmrWrElcXByTJ08+Zf8NN9xAo0aNiI6OZsCAAVlLx5cpU4YpU6ZkPRe+fPnyWUvTP/7446Snp9OiRQuaNm3K448/XuDnU716dd555x3i4uKyPostW7aQmJhIbGwsLVq0oFOnTlkjqH79+jF+/HhatWplk/KmxKlQNpyXL5vHKzecfhm6qNjy9bZ8fZE6uTS9qjJs2DAaNWrE/fff73ZYZ8T+bRjjZcvXm4Dw5ptvEhMTQ9OmTTl69Giuj+Y1xpydx9//jBajhrP7wDFX47BJeVOk7r///mIzIjGmuJi0+llSQv7ivEovuxqHJZRcqOo5PfbWlDyl+dKwCWyvz19OUuXvubnsq0W6zEpu7JJXDhERESQkJNgXiMmiqiQkJBAREeF2KMacZuzi8ciJKky+8w63Q7ERSk516tRh9+7dHDhwwO1QTACJiIgo8K43Y4raolW/sLfip3TSRzmvct6rShQVvyYUEekJvAwEA2+p6nM5ysOB94DWQAJwi6rucspGA0OATGCEqi4SkbpO/RqAAm+o6stO/SrAbKABsAv4m6oePtuYQ0NDs37JbYwxgSyyTDiNjw9h6r3D3Q4F8OMlLxEJBiYD1wDRQJyI5FxcZghwWFUvBCYCzzvHRgP9gKZAT2CK014G8ICqRgOXAsOytTkK+EpVGwFfOdvGGFNidWxany3j33RlmZXc+HMOpR2wXVV3qmoaMAvok6NOH+Bd5/1coJt4Z8P7ALNUNVVVfwW2A+1Uda+q/gSgqonAZqB2Lm29C1zvn24ZY4z7Hnh7LtO/XOF2GKfwZ0KpDfyRbXs3//3yP62OqmYAR4GqZ3KsiDQAWgE/OrtqqOpe5/1feC+LnUZEhorIKhFZZfMkxpji6ODRZCZuu5vRC552O5RTFMu7vEQkEpgH3Keqp/2SR723aOV6m5aqvqGqbVS1TfXq1f0cqTHG+N7f33oXLZPAo10fdDuUU/gzoewB6mbbruPsy7WOiIQAFfFOzud5rIiE4k0mM1T1o2x19olITadOTWA/xhhTwqSlZzLvzwmUO9KOYbGdCz6gCPkzoawEGolIlIiE4Z1kj89RJx4Y6Ly/GfjaGV3EA/1EJFxEooBGwApnfuVtYLOq5lwzPXtbA4Eze+SfMcYUI4998CnpFbYztNmDBAUF1g+w/ZZQnDmR4cAivJPnc1R1o4g8JSK9nWpvA1VFZDswEufOLFXdCMwBNgELgWGqmgl0BPoDV4rIGud18ulTzwE9RGQb0N3ZNsaYEmXfsUOUPdKa5wbc6HYop7HVhnOsNmyMMYHO41FXRye22rAxxhRz7y1ZRUamJ+AudZ1kCcUYY4qBRat+YeB/2nHDCxPdDiVPllCMMaYYGPnhRMgM5dl+t7sdSp4soRhjTIDbuGs/m0LfoXHKgIBZZiU3ttqwMcYEuGHvTIHQFF68bqTboeTLEooxxgQwj0dZefRzanAdse0vdjucfFlCMcaYABYUJBx49nt+/eusn8ZR5CyhGGNMgEpLzyQlLYMK5cJp2uA8t8MpkE3KG2NMgHp8RjyVx0axYOVWt0M5I5ZQjDEmQE1ZM54gTwRdWzZ0O5QzYpe8jDEmAL0+fzlJlb/nprKvEBFWPL6qbYRijDEBaOziF5GUyky5c7DboZwxSyjGGBNglq37lb0VP+Gy0Hs5r3I5t8M5Y8VjHGWMMaVIp2YNeGHbYq65pJnboZwVSyjGGBNggoKEh27q5nYYZ80ueRljTADp9c/xtH5kJBmZHrdDOWuWUIwxJkAcPJrMgmMvsPvENkKCi9/Xs13yMsaYADHi7ffQMgd57LKH3A6lUCyhGGNMAEhLz2Tunpco52nLsNjObodTKJZQjDEmADw+I570CtsZVnt2wD7ityDF7yKdMcaUQJ0vbkKrlPt4dsCNbodSaDZCMcaYABDb/mJi2wfu8+LPhI1QjDHGZTe+8DJzlq11O4xzZgnFGGNctHj1Nj5Ovp9XlnzodijnzBKKMca46L45EyAzlCmDhrsdyjmzhGKMMS7Z/PsBNoW+w0Up/Wlxwfluh3PObFLeGGNccu/0KRCawkvXPeB2KD5hIxRjjHFJSFAI9Y/dSmz7i90OxSdshGKMMS5Z/PijbofgUzZCMcaYIpaWnsmEj5fi8ajbofiUJRRjjClij8+I54F1V/Lkv79wOxSfsktexhhTxKaueZGQkAY88reebofiU34doYhITxHZKiLbRWRULuXhIjLbKf9RRBpkKxvt7N8qIldn2z9NRPaLyIYcbT0pIntEZI3zutaffTPGmMJ4ff5yEisvp0+NkUSElay/6f2WUEQkGJgMXANEA3EiEp2j2hDgsKpeCEwEnneOjQb6AU2BnsAUpz2Ad5x9uZmoqjHOa74v+2OMMb4wdvGLSEplXhtyh9uh+Jw/RyjtgO2qulNV04BZQJ8cdfoA7zrv5wLdRESc/bNUNVVVfwW2O+2hqsuAQ36M2xhj/OKvQ0nsD1lJh9B7OL9KpNvh+Jw/x1u1gT+ybe8G2udVR1UzROQoUNXZ/0OOY2ufwTmHi8gAYBXwgKoezllBRIYCQwHq1at3Zj0xxhgfOL9KJEee2kHSiTS3Q/GLknSX11SgIRAD7AVeyq2Sqr6hqm1UtU316tWLMDxjTGl28GgyySnpRJYJK5GjE/BvQtkD1M22XcfZl2sdEQkBKgIJZ3jsKVR1n6pmqqoHeBPnEpkxxgSCvpPGU+HxC/kzIdHtUPzGnwllJdBIRKJEJAzvJHt8jjrxwEDn/c3A16qqzv5+zl1gUUAjYEV+JxORmtk2bwA25FXXGGOK0qFjJ/i/E69RNaMFtaqWdzscv/HbHIozJzIcWAQEA9NUdaOIPAWsUtV44G3gfRHZjneivZ9z7EYRmQNsAjKAYaqaCSAiM4ErgGoishsYo6pvAy+ISAygwC7gLn/1zRhjzsbwt95FyxzkkQ4Puh2KX4l3QFA6tWnTRletWuV2GMaYEiwtPZPIURcT5qnEsZd+JChI3A7pnInIalVtk3N/yfpVjTHGBJgX5i0mvcI2htWeXSKSSX4soRhjjB898rerCZn3Jff16ep2KH5nCcUYY/woKEgY1beH22EUiZL0OxRjjAkoFz00hKvG/dPtMIqMJRRjjPGDxau3sa3cdI6nH3c7lCJjCcUYY/zg/jkTITOUyQOHux1KkbGEYowxPrb59wNsDJ3ORSn9iWlYs+ADSgiblDfGGB+7d/oUCE3hpesecDuUImUJxRhjfGzo5b0pu7wsse0vdjuUImUJxRhjfCzuilbEXdHK7TCKnM2hGGOMj6SlZ9L+sYdZsHKr26G4whKKMcb4yJh/f8aK0Bf4cu1at0NxRb4JRURuz/a+Y46y0nMvnDHGnIHJP79ISGIDnh1wo9uhuKKgEcrIbO9fzVE22MexGGNMsfXGgu9JrPwdfWqMJCKsdE5PF5RQJI/3uW0bY0ypNXbxi0hKZV4bcofbobimoDSqebzPbdsYY0olj0epEV6P6LIPlNjnxZ+JghJKExFZh3c00tB5j7N9gV8jM8aYYiIoSPjp2Yluh+G6ghJK6fpVjjHGnKWtfxxkweoNjOh9eYl/gFZB8p1DUdXfsr+AJOASoJqzbYwxpdo906Zw/9quLF27w+1QXFfQbcOfi0gz531NYAPeu7veF5H7/B+eMcYErkPHTvBN8mucdySWbq0udDsc1xV0l1eUqm5w3t8BLFbV64D22G3DxphS7u9vv4eWPcAjVzzodigBoaCEkp7tfTdgPoCqJgIefwVljDGBLiPTw4e7X6LskTb8/boubocTEAqalP9DRP4O7MY7d7IQQETKAKF+js0YYwLW0jU7yAg5yrBGT5f6yfiTCkooQ4CngO7ALap6xNl/KTDdj3EZY0xA69G6EYca/1ZqfxWfm3w/CVXdD9ydy/6lwFJ/BWWMMYFs6x8HqV+jEpUiI9wOJaDkm1BEJD6/clXt7dtwjDEm8F058S4O6y6SXlpll7uyKWis1gH4A5gJ/Iit32WMKeW++nk7f1b4mMs8oy2Z5FBQQjkf6AHEAbcCXwAzVXWjvwMzxphAdN+siRAWytQ7/u52KAGnoF/KZ6rqQlUdiHcifjvwjT0LxRhTGm394yAbQqdzUUp/WlxwvtvhBJwCb08QkXCgF95RSgPgFeBj/4ZljDGBZ9S/P4DQE4yPHVlw5VKooEn594BmeH/QODbbr+aNMabU+fDBEby9qB29L412O5SAJKp5P9ZERDzAcWcze0UBVFUr+DE2v2vTpo2uWrXK7TCMMcWAx6M2Ce8QkdWq2ibn/oLmUIJUtbzzqpDtVb64JxNjjDlTGZkeKo7syMCX33I7lIBW0Fpe50REeorIVhHZLiKjcikPF5HZTvmPItIgW9loZ/9WEbk62/5pIrJfRDbkaKuKiCwWkW3O/1b2Z9+MMaXHEzM+I6ny91QpZ39H58dvCUVEgoHJwDVANBAnIjkvPA4BDqvqhcBE4Hnn2GigH9AU6AlMcdoDeMfZl9Mo4CtVbQR85WwbY8w5e+2n8YQkNuDZATe6HUpA8+cIpR2wXVV3qmoaMAvok6NOH+Bd5/1coJuIiLN/lqqmquqveG9XbgegqsuAQ7mcL3tb7wLX+7AvxphS6o0F35NY+Tt6n3e/rdtVAH8mlNp4f2V/0m5nX651VDUDOApUPcNjc6qhqnud938BNXKrJCJDRWSViKw6cODAmfTDGFOKPbX4JSSlEpPvtEdAFcSvcyhuUe+ta7nevqaqb6hqG1VtU7169SKOzBhT3DzR4wHuqT+Z86tEuh1KwPPn+G0PUDfbdh1nX251dotICFARSDjDY3PaJyI1VXWv87ji/ecSvDHGAAy9pgPeZQ1NQfw5QlkJNBKRKBEJwzvJnnP14nhgoPP+ZuBrZ3QRD/Rz7gKLAhoBKwo4X/a2BgKf+qAPxphSausfB2n28L18v+l3t0MpNvyWUJw5keHAImAzMEdVN4rIUyJyctn7t4GqIrIdGIlzZ5az+OQcYBPep0QOU9VMABGZCXwPNBaR3SIyxGnrOaCHiGzD+0Cw5/zVN2NMyXfPtClsLDuVhMTjBVc2QAG/lC/p7JfyxpjcHDp2gmrP1Kd6Wnv2TfzM7XACTl6/lLd74IwxJocR095Hyx7gkcsedDuUYsUSijHGZJOR6WHOHy9R1tOGv1/Xxe1wihVLKMYYk83+I8dpFHIlsRdfZYtBniVLKMYYk02tquXZ+PxUt8MolkrkDxuNMaYwPl2+kalffOd2GMWWJRRjjHEMmzuGYf+5joNHk90OpViyhGKMMcBXP29nT4WP6BB6D9UqlnU7nGLJEooxxgD3zZoImaFMHjjc7VCKLUsoxphSb+sfB9kQOp2LUvoT07Cm2+EUW5ZQjDGl3oLVG5CMcoy/aaTboRRrtvSKLb1ijAGSTqQRWSbM7TCKhbyWXrERijGmVPt+0++kpWdaMvEBSyjGmFLr2PFULn+7Bw0fjnM7lBLBEooxplTaf/g4UY9eR3qFXxjQ6na3wykRbOkVY0yp8/v+ozT/ZyzHKi7njirTeKZ/74IPMgWyhGKMKXUufXYQxyr8wP31ZjFhSF+3wykxLKEYY0qd6bc9z4/bhvJE3DVuh1Ki2ByKMaZU+M+GXXR7ahwej3J1m4ssmfiBJRRjTIk3f8UWLn+3E0tPTOD7zb+7HU6JZQnFGFOizVm2lth5XdCgdOZc+390bFrf7ZBKLEsoxpgS662FP9BvwRUEecL54uZl3Ny5hdshlWg2KW+MKbF2JyQQmlqTr4cssJFJEbC1vGwtL2NKnDU79matGpySlkFEmP3t7Eu2lpcxplS4/605tJp+AU/PWghgyaQI2SdtjCkx7nhlGu8k/A8VkjoyqNtlbodT6lhCMcaUCDe98AofnfgHVY9exZaxH9tjfF1gl7yMMcXelM//w0cn/kHNIzew85l4SyYusRGKMabYu/vajmzdO4vnR99kcyYushGKMaZYysj00P6xh5n3n/UEBQkv/88tlkxcZgnFGFPspKRl0PjhQawIfYF/LY13OxzjsHRujClWjh1P5eLHb+XPih/RTcax6NFH3A7JOCyhGGOKjYNHk2ny5I0kVFrE9RET+fjh+9wOyWTj10teItJTRLaKyHYRGZVLebiIzHbKfxSRBtnKRjv7t4rI1QW1KSLviMivIrLGecX4s2/GmKIXFhKMIAys/JYlkwDktxGKiAQDk4EewG5gpYjEq+qmbNWGAIdV9UIR6Qc8D9wiItFAP6ApUAtYIiIXOcfk1+ZDqjrXX30yxrhj2+4EgoKEhrWqsO+l+QQFidshmVz4c4TSDtiuqjtVNQ2YBfTJUacP8K7zfi7QTUTE2T9LVVNV9Vdgu9PembRpjClB1uzYS/MJl9P6hRvweNSSSQDzZ0KpDfyRbXu3sy/XOqqaARwFquZzbEFtPiMi60RkooiE+6ITxhj3fLfxN9pN6UJqmV083mWMJZMAV5JuGx4NNAHaAlWAh3OrJCJDRWSViKw6cOBAUcZnjDkLi1b9wuXvdCIj7CBvdl7CAzde6XZIpgD+TCh7gLrZtus4+3KtIyIhQEUgIZ9j82xTVfeqVyowHe/lsdOo6huq2kZV21SvXr2QXTPG+JPHo9z8wSA8QanM7LmUO3te6nZI5gz487bhlUAjEYnC+6XfD7g1R514YCDwPXAz8LWqqojEA/8WkQl4J+UbASsAyatNEampqnudOZjrgQ1+7Jsxxo+CgoQvBs8gKSWVa9s1cTscc4b8llBUNUNEhgOLgGBgmqpuFJGngFWqGg+8DbwvItuBQ3gTBE69OcAmIAMYpqqZALm16ZxyhohUx5t01gB3+6tvxhj/mPTJN0z7YR4/PfMyXVpEuR2OOUv2xEZ7YqMxAWHsv+fz5KabCEuOYtOD/6FhrSpuh2TyYE9sNMYErJFvf8iTW66nzPFo1t2/zJJJMWVLrxhjXDV08nu8uf8Oyh/rwIZHvqDeeRXdDskUko1QjDGuurBGLc47di3bxyyyZFLMWUIxxrhixtc/AfC/N3dn38TPOK9yOZcjMufKEooxpkh5PEqHx0dz+7eteTV+mdvhGB+yORRjTJHJyPRwyaMjWF9mMhcfv4t7enVyOyTjQzZCMcYUiZS0DJo8PJj1ZSbTOu0BNjw3lZBg+woqSey/pjGmSIybPZ8d5d/lSnmKFePG20KPJZBd8jLGFIln+vem/oLlDL2mg9uhGD+xEYoxxm92HzjG+ff35oOvVgNYMinhLKEYY/xix5+HaPLP7uwrv4Cfdu10OxxTBOySlzHG59bt/It2r/UgNXIbjzX6iHH9r3M7JFMELKEYY3xq3c6/aDO5C+ll9vBCzBc8dFM3t0MyRcQueRljfOrCWlWpQ3v+1XGxJZNSxkYoxhif+HT5Ri6sVZ2mDc5j50vvux2OcYGNUIwx5+zdxSu5Ib4LXV++w+1QjIssoRhjzsmr8csYtLQbwRkVmTPoVbfDMS6yhGKMKbSnZy1kxIqehKXU4fuh33JFywvcDsm4yOZQjDGFkpaeyTMr/pcy2oTV9y/i4nrV3Q7JuMwSijHmrHk8SlhoMN/evYDqFctRv0Ylt0MyAcAueRljzkq/l6ZwwUO3k5aeSZuLalsyMVksoRhjztg1zzzP7KRhpHqSSEnLcDscE2AsoRhjCuTxKB2feJSFGaOofyyOHc/OpUK5cLfDMgHGEooxpkCdxjzK8uB/0uT4//DLc+9TNiLU7ZBMALJJeWNMgW5tdw26QvnuuX/ag7FMnmyEYozJVdKJNJ744HMAhl/Xme/HPWvJxOTLEoox5jSHjp2g4SM3Mm7Hdcz7z3q3wzHFhF3yMsac4s+ERJqO68ORSt9wW8XXualTc7dDMsWEJRRjTJZf9x6m+QvXcLziKu45/32m3H2b2yGZYsQSijEBKC09k2PJqSQmp3I0OYUyYaE0rlsNgA++Ws2R5GSSU1NJTk3leGoKzerWY2CPtng8yk3jXyYlI5XUjFRSM72vHhd14tmB13PwaDKtxg4iQ1NJJ5UMTSFTUulT/w4+uG8oE+IXcjxyDaMbzuOfA/q4/CmY4sYSijE5JJ1IY8/BYxw4msSR4ydIPJFCkAi3XB4DwOvzl7N17x5OpKVyIj2VlPRUKpetwNR7bgeg7/jX2HFoB2meVNI8KaR5UqkX2ZBlY58C4IIH+rPf8wuZpOKRVDxBqdSjEztefA+AkAcbkFn+t1NiqnO0L39MmANA/yXdIOLoKeWNNt/BwB5tCQoSPkl+EIIynZIQIJyQbSHA9YQEB7Gf9QQRTrDzCtPylA2NAODVu+Lot/EyOjat7/sP1pR4llBMsZaSlsG+w0nsP5JE60a1CQoS5q/Ywndbf+HI8SSOpiRx9EQiJ9JPsOSJxwCIe2kq3+xeSKomkSZJZEgSoqGcmLgGgIse6cfeSh+fcp7gpHrccrn3S/7RL8eSUPnLU8rD90QzFW9CWbJnHkfKrkYknCAiEMLxJGZm1Q0NCifCU5lQwgklglDCaVYtJqv8mqp/53jaccJDwokICadMaAStmjXKKn/2kg8JEqFseDjlIyIoFxHOBedXyyrfefcBypcNp0LZcMJCg0+Js1JkBKkTNuf7mVoyMYVlCcUUub8OJbH+170cTEzi4LFEDiUlceh4Ig9dfw11qlfg9fnLmb48nuSMJI5nJJLiSSLFk8gPD86kUZ2q9HluAp8deRYNSYLQlKx29ww/Rq2q5Xni07dYHfbSaedNSx9NWGgwe47t5Qi/EUokEVQmgnqUD62SVe+OSwaxZe8VVIiIpHxEWcqFR1CtfIWs8tkDJ3MsOYXIiHAiy4RTvkw4lSLLZJUfnrQ03/5vHf9WvuWfjX4g3/JRfXvkWx5Vs3K+5cb4i18Tioj0BF4GgoG3VPW5HOXhwHtAayABuEVVdzllo4EhQCYwQlUX5demiEQBs4CqwGqgv6qm+bN/JUlauvcv6LDQYJJT0tn0+35S0tI5npJGcqr3dWmTBjSsVYU1O/YyeeGXHElOJDE1iaS0JI6nJzG2z1B6XxrNq/HLeOzrx0iXRDKCksgMScQTksSbnZdwZ89LeWTGh0w/NPi0GDpvWcvN1Vvw1aaf+VEmIpQniEhCiCREIkk8kQpAq7qN2XG4L2U1kvJSnvJhkVSIiKRsuPfX25PihrNrXxxVK0RSo1J5zqsUyXmVymX9te699PRUnp/FM/175/tZdWt1YWE+YmNKPL8lFBEJBiYDPYDdwEoRiVfVTdmqDQEOq+qFItIPeB64RUSigX5AU6AWsERELnKOyavN54GJqjpLRP7ltD3VX/0rSNKJNJJOpHEi1fulfDwljcgy4TRtcB4AM77+iaSUFE6kpXEiLY3U9HQurlObWy6PweNRBr82jbSMdFIz0kjNTCMtI40rm7RlVN8eHElKoeszo0j3pJHhSSdd08jQNG68+AYm3vk3tu1OoN2LfckkDY+k4SEdj6QR1+ABpo8YzFc/b6fH7EvRoDQIdl6i3FbxdT64byhz/7OWgd+1Pa1Pw7f9m1fviuOb9Vt56+Cg/xZIEFCedbt60vvSaEKCgwkihArUJYJIymh5ymkkF5zvfV7G7Z27EPHDB1QqG0nlcpFUiYykesXydGnmfTjTrJH3MidoWJ6f7ZO39eLJ23rlWd6pWQM6NWtw1v/NjDHnxp8jlHbAdlXdCSAis4A+QPaE0gd40nk/F3hNRMTZP0tVU4FfRWS70x65tSkim4ErgVudOu867fotoZx3/7UcDt6KShqeoDQISqNm6pXsmTAPgIpPXIAncs8px9Q9+jd+nzAbgNsXX3n6xOqawdxy+dsEBQnvJgyFIM8p5Qd++gej+vbA41HWyHREwhAJI0jDEEL57dClAIQEB+EhgxDCCaY8IYQRTBi1Knuvs9eqWpFm9COUMEI1jFANJSwojJ7NWwPQvnEDbtvwOhEhYYSHhBEeGkpEaBh92l7ijb1rO2KidlCjcnlqVIqkUmTEKb+gvqdXR+7p9XWen92VMQ25MqZhnuX2a2xjiid/JpTawB/ZtncD7fOqo6oZInIU7yWr2sAPOY6t7bzPrc2qwBFVzcil/ilEZCgwFKBevXpn16Ns6pdtSmRKVUIkjFDCCNFQmtVtmlV+U43RHE9L9n4hB4cRHhpGTMv/fok+HTObIBHCQ0MpExZG2bAwLqx1Xlb58lt+pVxEGOUiwigTHkpkRFjWgnxVKpRB/3lqMsouqmZljk5almf5xfWqs+651/Isb1y3Gh/cNzTP8moVy9qjXo0xpyl1k/Kq+gbwBkCbNm20sO2sfGZ8vuVzHsz7kg3Ao7dcnW95h+jCJztjjHGDP9fy2gPUzbZdx9mXax0RCQEq4p2cz+vYvPYnAJWcNvI6lzHGGD/yZ0JZCTQSkSgRCcM7yR6fo048MNB5fzPwtaqqs7+fiIQ7d281Albk1aZzzFKnDZw2P/Vj34wxxuTgt0tezpzIcGAR3lt8p6nqRhF5ClilqvHA28D7zqT7IbwJAqfeHLwT+BnAMFXNBMitTeeUDwOzRORp4GenbWOMMUVEvH/cl05t2rTRVatWuR2GMcYUKyKyWlXb5Nxvz0MxxhjjE5ZQjDHG+IQlFGOMMT5hCcUYY4xPlOpJeRE5APxWYMXcVQMO+jAcN1lfAk9J6QdYXwLVufSlvqpWz7mzVCeUcyEiq3K7y6E4sr4EnpLSD7C+BCp/9MUueRljjPEJSyjGGGN8whJK4b3hdgA+ZH0JPCWlH2B9CVQ+74vNoRhjjPEJG6EYY4zxCUsoxhhjfMISig+IyAMioiJSze1YCktExonIOhFZIyJfikgtt2MqDBEZLyJbnL58LCKV3I6psESkr4hsFBGPiBTLW1VFpKeIbBWR7SIyyu14CktEponIfhHZ4HYs50JE6orIUhHZ5Pzb+ocv27eEco5EpC5wFfC727Gco/Gq2kJVY4DPgSdcjqewFgPNVLUF8Asw2uV4zsUG4EYg7+c5BzARCQYmA9cA0UCciES7G1WhvQP0dDsIH8gAHlDVaOBSYJgv/5tYQjl3E4H/BYr13Q2qeizbZjmKaX9U9UtVzXA2f8D79M5iSVU3q+pWt+M4B+2A7aq6U1XTgFlAH5djKhRVXYb3mU3FmqruVdWfnPeJwGagtq/aL3XPlPclEekD7FHVtSLidjjnTESeAQYAR4GuLofjC4OB2W4HUYrVBv7Itr0baO9SLCYHEWkAtAJ+9FWbllAKICJLgPNzKXoUeATv5a5iIb++qOqnqvoo8KiIjAaGA2OKNMAzVFA/nDqP4h3ezyjK2M7WmfTFGF8TkUhgHnBfjqsT58QSSgFUtXtu+0WkORAFnByd1AF+EpF2qvpXEYZ4xvLqSy5mAPMJ0IRSUD9EZBAQC3TTAP+h1Vn8NymO9gB1s23XcfYZF4lIKN5kMkNVP/Jl25ZQCklV1wPnndwWkV1AG1UtliuRikgjVd3mbPYBtrgZT2GJSE+8c1qXq2qy2/GUciuBRiIShTeR9ANudTek0k28f/2+DWxW1Qm+bt8m5c1Jz4nIBhFZh/cynk9vJyxCrwHlgcXOLdD/cjugwhKRG0RkN9AB+EJEFrkd09lwbo4YDizCO/k7R1U3uhtV4YjITOB7oLGI7BaRIW7HVEgdgf7Alc7/P9aIyLW+atyWXjHGGOMTNkIxxhjjE5ZQjDHG+IQlFGOMMT5hCcUYY4xPWEIxxhjjE5ZQjHGISNI5HDvcWVH3lFWnxesVp2ydiFySraymiHzuvL/i5PtzJSLfnMnqxCKyq6AVskVkiYhU9kVcpuSzhGKMb3wHdAd+y7H/GqCR8xoKTM1WNhJ4s0iiK7z3gXvdDsIUD5ZQjMnBGVWMd37ouV5EbnH2B4nIFOd5K4tFZL6I3Aygqj+r6q5cmusDvKdePwCVRKSmU3YTsDCX87cTke9F5GcRWS4ijZ39g0TkE+fcu5xR0Uin3g8iUiVbM/2dH61tEJF2zvFVnWfdbBSRtwDJds5PRGS1UzY0WzvxQFxhP0tTulhCMeZ0NwIxQEu8o47xThK4EWiA99ke/fH+gr0gua24W9tZjuSwqqbmcswWoLOqtsL7XJp/Zitr5sTRFngGSHbqfY93peiTyjrPtrkXmObsGwP8R1WbAh8D9bLVH6yqrYE2wAgRqQqgqoeB8JPbxuTH1vIy5nSdgJmqmgnsE5H/w/sF3gn4UFU9wF8isvQczlETOJBHWUXgXRFphPe5NKHZypY6z7FIFJGjwGfO/vVAi2z1ZoL3OR4iUkG8T67sgjcZoapfiMjhbPVHiMgNzvu6eC/RJTjb+4Fa2baNyZWNUIzxr7xW3D0BRORxzDi8iaMZcF2OetlHNJ5s2x5O/QMx55pKea6xJCJX4B2JdVDVlsDPOc4Z4cRrTL4soRhzum+BW0QkWESq4/3LfgXeifebnLmUGsAVZ9BWPDDAmZe5FDiqqnvxPp64QR7HVOS/y7wPKmQfTs77dHLOeRTvo4RvdfZfA5y8e6si3stvySLSBO+jYXHqCd7ntewqZBymFLGEYszpPgbWAWuBr4H/dZ5xMw/vHMgm4APgJ7xPt0RERjgrA9cB1jmT3uB9rsxOYDveO7ruBVDV48AOEbkwl/O/ADwrIj9T+MvSKc7x/wJOrow7FugiIhvxXvr63dm/EAgRkc3Ac3gfnXxSa+CHbI9VNiZPttqwMWdBRCJVNcmZpF4BdCzsA9WcOYvWqvqYT4P0IRF5GYhX1a/cjsUEPpuUN+bsfO5McIcB487l6Zyq+nExuHtqgyUTc6ZshGKMMcYnbA7FGGOMT1hCMcYY4xOWUIwxxviEJRRjjDE+YQnFGGOMT/w/KpKrMndbP4YAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_119_1.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from sklearn.model_selection import train_test_split\n", "from sklearn import linear_model\n", "from sklearn.preprocessing import StandardScaler\n", "\n", "def MSE(y_data,y_model):\n", " n = np.size(y_model)\n", " return np.sum((y_data-y_model)**2)/n\n", "# A seed just to ensure that the random numbers are the same for every run.\n", "# Useful for eventual debugging.\n", "np.random.seed(315)\n", "\n", "n = 100\n", "x = np.random.rand(n)\n", "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)\n", "\n", "Maxpolydegree = 20\n", "X = np.zeros((n,Maxpolydegree-1))\n", "\n", "for degree in range(1,Maxpolydegree): #No intercept column\n", " X[:,degree-1] = x**(degree)\n", "\n", "# We split the data in test and training data\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n", "\n", "#For our own implementation, we will need to deal with the intercept by centering the design matrix and the target variable\n", "X_train_mean = np.mean(X_train,axis=0)\n", "#Center by removing mean from each feature\n", "X_train_scaled = X_train - X_train_mean \n", "X_test_scaled = X_test - X_train_mean\n", "#The model intercept (called y_scaler) is given by the mean of the target variable (IF X is centered)\n", "#Remove the intercept from the training data.\n", "y_scaler = np.mean(y_train) \n", "y_train_scaled = y_train - y_scaler \n", "\n", "p = Maxpolydegree-1\n", "I = np.eye(p,p)\n", "# Decide which values of lambda to use\n", "nlambdas = 6\n", "MSEOwnRidgePredict = np.zeros(nlambdas)\n", "MSERidgePredict = np.zeros(nlambdas)\n", "\n", "lambdas = np.logspace(-4, 2, nlambdas)\n", "for i in range(nlambdas):\n", " lmb = lambdas[i]\n", " OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)\n", " intercept_ = y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data\n", " #Add intercept to prediction\n", " ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_ \n", " #Add intercept to prediction\n", " ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler \n", " RegRidge = linear_model.Ridge(lmb)\n", " RegRidge.fit(X_train,y_train)\n", " ypredictRidge = RegRidge.predict(X_test)\n", " MSEOwnRidgePredict[i] = MSE(y_test,ypredictOwnRidge)\n", " MSERidgePredict[i] = MSE(y_test,ypredictRidge)\n", " print(\"Beta values for own Ridge implementation\")\n", " print(OwnRidgeBeta) #Intercept is given by mean of target variable\n", " print(\"Beta values for Scikit-Learn Ridge implementation\")\n", " print(RegRidge.coef_)\n", " print('Intercept from own implementation:')\n", " print(intercept_)\n", " print('Intercept from Scikit-Learn Ridge implementation')\n", " print(RegRidge.intercept_)\n", " print(\"MSE values for own Ridge implementation\")\n", " print(MSEOwnRidgePredict[i])\n", " print(\"MSE values for Scikit-Learn Ridge implementation\")\n", " print(MSERidgePredict[i])\n", "\n", "\n", "# Now plot the results\n", "plt.figure()\n", "plt.plot(np.log10(lambdas), MSEOwnRidgePredict, 'b--', label = 'MSE own Ridge Test')\n", "plt.plot(np.log10(lambdas), MSERidgePredict, 'g--', label = 'MSE SL Ridge Test')\n", "plt.xlabel('log10(lambda)')\n", "plt.ylabel('MSE')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see here, when compared to the code which includes explicitely the\n", "intercept column, that our MSE value is actually smaller. This is\n", "because the regularization term does not include the intercept value\n", "$\\beta_0$ in the fitting. This applies to Lasso regularization as\n", "well. It means that our optimization is now done only with the\n", "centered matrix and/or vector that enter the fitting procedure. Note\n", "also that the problem with the intercept occurs mainly in these type\n", "of polynomial fitting problem.\n", "\n", "The next example is indeed an example where all these discussions about the role of intercept are not present.\n", "\n", "## More complicated Example: The Ising model\n", "\n", "The one-dimensional Ising model with nearest neighbor interaction, no\n", "external field and a constant coupling constant $J$ is given by" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " H = -J \\sum_{k}^L s_k s_{k + 1},\n", "\\label{_auto1} \\tag{1}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $s_i \\in \\{-1, 1\\}$ and $s_{N + 1} = s_1$. The number of spins\n", "in the system is determined by $L$. For the one-dimensional system\n", "there is no phase transition.\n", "\n", "We will look at a system of $L = 40$ spins with a coupling constant of\n", "$J = 1$. To get enough training data we will generate 10000 states\n", "with their respective energies." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from mpl_toolkits.axes_grid1 import make_axes_locatable\n", "import seaborn as sns\n", "import scipy.linalg as scl\n", "from sklearn.model_selection import train_test_split\n", "import tqdm\n", "sns.set(color_codes=True)\n", "cmap_args=dict(vmin=-1., vmax=1., cmap='seismic')\n", "\n", "L = 40\n", "n = int(1e4)\n", "\n", "spins = np.random.choice([-1, 1], size=(n, L))\n", "J = 1.0\n", "\n", "energies = np.zeros(n)\n", "\n", "for i in range(n):\n", " energies[i] = - J * np.dot(spins[i], np.roll(spins[i], 1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we use ordinary least squares\n", "regression to predict the energy for the nearest neighbor\n", "one-dimensional Ising model on a ring, i.e., the endpoints wrap\n", "around. We will use linear regression to fit a value for\n", "the coupling constant to achieve this.\n", "\n", "A more general form for the one-dimensional Ising model is" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " H = - \\sum_j^L \\sum_k^L s_j s_k J_{jk}.\n", "\\label{_auto2} \\tag{2}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we allow for interactions beyond the nearest neighbors and a state dependent\n", "coupling constant. This latter expression can be formulated as\n", "a matrix-product" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " \\boldsymbol{H} = \\boldsymbol{X} J,\n", "\\label{_auto3} \\tag{3}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $X_{jk} = s_j s_k$ and $J$ is a matrix which consists of the\n", "elements $-J_{jk}$. This form of writing the energy fits perfectly\n", "with the form utilized in linear regression, that is" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " \\boldsymbol{y} = \\boldsymbol{X}\\boldsymbol{\\beta} + \\boldsymbol{\\epsilon},\n", "\\label{_auto4} \\tag{4}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We split the data in training and test data as discussed in the previous example" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "X = np.zeros((n, L ** 2))\n", "for i in range(n):\n", " X[i] = np.outer(spins[i], spins[i]).ravel()\n", "y = energies\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the ordinary least squares method we choose the cost function" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " C(\\boldsymbol{X}, \\boldsymbol{\\beta})= \\frac{1}{n}\\left\\{(\\boldsymbol{X}\\boldsymbol{\\beta} - \\boldsymbol{y})^T(\\boldsymbol{X}\\boldsymbol{\\beta} - \\boldsymbol{y})\\right\\}.\n", "\\label{_auto5} \\tag{5}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We then find the extremal point of $C$ by taking the derivative with respect to $\\boldsymbol{\\beta}$ as discussed above.\n", "This yields the expression for $\\boldsymbol{\\beta}$ to be" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{\\beta} = \\frac{\\boldsymbol{X}^T \\boldsymbol{y}}{\\boldsymbol{X}^T \\boldsymbol{X}},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "which immediately imposes some requirements on $\\boldsymbol{X}$ as there must exist\n", "an inverse of $\\boldsymbol{X}^T \\boldsymbol{X}$. If the expression we are modeling contains an\n", "intercept, i.e., a constant term, we must make sure that the\n", "first column of $\\boldsymbol{X}$ consists of $1$. We do this here" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "X_train_own = np.concatenate(\n", " (np.ones(len(X_train))[:, np.newaxis], X_train),\n", " axis=1\n", ")\n", "X_test_own = np.concatenate(\n", " (np.ones(len(X_test))[:, np.newaxis], X_test),\n", " axis=1\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Doing the inversion directly turns out to be a bad idea since the matrix\n", "$\\boldsymbol{X}^T\\boldsymbol{X}$ is singular. An alternative approach is to use the **singular\n", "value decomposition**. Using the definition of the Moore-Penrose\n", "pseudoinverse we can write the equation for $\\boldsymbol{\\beta}$ as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{\\beta} = \\boldsymbol{X}^{+}\\boldsymbol{y},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where the pseudoinverse of $\\boldsymbol{X}$ is given by" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{X}^{+} = \\frac{\\boldsymbol{X}^T}{\\boldsymbol{X}^T\\boldsymbol{X}}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using singular value decomposition we can decompose the matrix $\\boldsymbol{X} = \\boldsymbol{U}\\boldsymbol{\\Sigma} \\boldsymbol{V}^T$,\n", "where $\\boldsymbol{U}$ and $\\boldsymbol{V}$ are orthogonal(unitary) matrices and $\\boldsymbol{\\Sigma}$ contains the singular values (more details below).\n", "where $X^{+} = V\\Sigma^{+} U^T$. This reduces the equation for\n", "$\\omega$ to" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " \\boldsymbol{\\beta} = \\boldsymbol{V}\\boldsymbol{\\Sigma}^{+} \\boldsymbol{U}^T \\boldsymbol{y}.\n", "\\label{_auto6} \\tag{6}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that solving this equation by actually doing the pseudoinverse\n", "(which is what we will do) is not a good idea as this operation scales\n", "as $\\mathcal{O}(n^3)$, where $n$ is the number of elements in a\n", "general matrix. Instead, doing $QR$-factorization and solving the\n", "linear system as an equation would reduce this down to\n", "$\\mathcal{O}(n^2)$ operations." ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "def ols_svd(x: np.ndarray, y: np.ndarray) -> np.ndarray:\n", " u, s, v = scl.svd(x)\n", " return v.T @ scl.pinv(scl.diagsvd(s, u.shape[0], v.shape[0])) @ u.T @ y" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "beta = ols_svd(X_train_own,y_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When extracting the $J$-matrix we need to make sure that we remove the intercept, as is done here" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "J = beta[1:].reshape(L, L)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A way of looking at the coefficients in $J$ is to plot the matrices as images." ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ ":7: UserWarning: FixedFormatter should only be used together with FixedLocator\n", " cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAM2CAYAAACjUj0CAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABOKUlEQVR4nO3dfYxc1Zkg/Kerev2FcRS/cSxjh0/hmVmjgN8RTYZ9N7DEsYmB2IhvOUPAAbKI+QNQloAWxAyaDAKyY4mQZdGYIYNpIFhLYJFZEiXaSZhgBSVkZwnYDhjDGmO+guOPNth0Vb1/GDfptN19yz5ddarv7ychpKrje5/73HNv1dPn1LldjUajEQAAAJRGpd0BAAAA0FoKQQAAgJJRCAIAAJSMQhAAAKBkFIIAAAAloxAEAAAome52BwBA59u9e3f09vbGqlWr4pVXXomurq44/PDD44wzzogLLrggDj300IG2119/ffzgBz+In/zkJzFr1qxht1uv1+P73/9+/OAHP4iXX3456vV6zJw5M+bNmxeXXXbZoO0CAMUpBAE4KG+99VZcdtll8dJLL8W8efPi7LPPjkajEb/+9a/j7//+7+P73/9+3HPPPXH00Uc3ve3rrrsunnzyyfjSl74UZ511VlQqlfjNb34Ty5cvj6eeeioefvjh+OQnPzkKRwUAY5tCEIADtnv37rjyyivj9ddfj/vuuy/+4i/+YuC9r3zlK3HxxRfHFVdcEVdccUU88cQTMXHixMLbfu655+KJJ56I66+/Pi699NJB733+85+Pq6++OpYvXx7/6T/9p2THAwBl4TeCABywxx57LF544YX45je/OagI3Ov444+P//yf/3Ns3Lgx7r333qa2/etf/zoiIv7dv/t3Q9770pe+FJ/+9Kfjf//v/31AcQNA2SkEAThgjz32WEyaNCnOPvvs/bY566yzYtq0afHEE080te1DDjkkIiIeeeSRqNfrQ97/yU9+Er29vc0FDABEhEIQgANUq9Xi+eefj3/7b/9tjB8/fr/turq64qSTTopXX3013nnnncLbnz9/fnziE5+IFStWxLx58+K2226Ln/3sZ7Fz586IiBg3btxBHwMAlJVCEIADsnXr1ti9e3dMmzZtxLaf/vSnIyLi7bffLrz9qVOnxj/8wz/EZz7zmdi0aVP84z/+Y1x++eXR09MT//E//sf4P//n/xxw7ABQdgpBAA5Io9GIiIhqtTpi2+7u7kH/pqjjjz8+nnrqqbjnnnviggsuiFmzZsWHH34Y/+t//a+44IILmp5uCgDsYdVQAA7I1KlT49/8m38Tv/vd70Zsu3ckcO/IYDO6u7vj1FNPjVNPPTUiIl555ZV48MEHY8WKFfG3f/u38cUvfjEmTJjQ9HYBoMyMCAJwQLq6umLu3Lnx/PPPx65du/bbrtFoxK9+9av4zGc+01QheNddd8XKlSuHvH700UfHjTfeGBdeeGH8/ve/j5dffvmA4geAMlMIAnDAFi9eHDt27Ijvf//7+23zk5/8JDZu3BhnnXVWU9t+7LHH4p577tnvdNLZs2dHRDT1bEIAYA+FIAAH7Oyzz465c+fGf/kv/yX+5V/+Zcj7a9asiZtuuilmzZoVl112WVPbPuuss2Ljxo1xzz33DHlv165d8dhjj8WRRx4ZRx999AHHDwBl1dVo9pf7APAH3n333bjyyivj+eefj/nz58dJJ50U1Wo1/vVf/zWeeOKJmDFjRvzX//pf49hjj42IiOuvvz5+8IMfxJlnnjnwrMA/9KUvfSn+4i/+Ij744INYunRp/OpXv4q5c+fGF77whZg6dWps3rw5nnjiiXjzzTfjH//xH+PP//zPW33IANDxFIIAHLTdu3fHY489Fo8++mhs2LAh+vv74/DDD4+FCxfGhRdeGIceeuhA272F4P7ccMMNcckll0RERH9/fzz00EPxP//n/4z169dHX19fTJ06NU4++eT4+te/HkcdddRoHxoAjEkKQQAAgJLxG0EAAICSUQgCAACUjEIQAACgZLrbHQAAAACjY9u2bbFt27Yhr1ssBgAA4CC9/957MXHq1HaHMcQHH3wQn//852Pr1q2DXs+2EKzXh3+/Uhm5TUTE0UfX0gQUEa+8Uk22rUoUCL6gesIZvoXiKpr8TKXMV0opc5/rMXa6Du/6SbX6HtaO3Kc8xo6m4w/I9fM253t+x37fGQP9PmW+UnyfnjUr4l/+Jd136Vz94//3/8W2119vdxgDpsyaFUv/5V9i8+bNUasNPo9jfmroa6+1OwIAAOhcvk8Xt+3112NrhgmbMWPGkNfy/RMSAAAAoyJ5Ibhx48b4q7/6q+jp6Ymenp647rrr4r333ku9GwAAgKxUMvxvf5JODd2yZUt89atfjd27d8dll10WtVot7r333li3bl2sXLkyxo0bl3J3AAAAHICkheD3vve9ePPNN+OJJ56IY445JiIijj/++Lj00kvjsccei/PPPz/l7gAAADgASaeGrlq1Knp6egaKwIiIk08+OY466qhYtWpVyl0BAABkpd3TQJuZGpqsENy6dWts3Lgx5syZM+S9OXPmxAsvvJBqVwAAAByEZIXgW2+9FRER06dPH/LetGnTYvv27bF9+/ZUuwMAAOAAJfuNYF9fX0RETJw4cch748ePj4iInTt3xqGHHlpoe5UCJWqRNo1Grg+uTPlQ1JQKbq1I8jOVb+Tpcp/vMXa+Du76ibX+Htb63DvZA3T8iMj38zbvs9PB33c6vN+njD7f79P5GWk6Zqu1ZNXQRqMxYpuurq7C26vXh3+/Uhm5TUREtVorvM+R1GrpLoJKFAi+oHrSm2yBuIomP1Mp85VSytzneoydrsO7flKtvoe1I/cpj7Gj6fgDcv28zfme37Hfd8ZAv0+ZrxTfp484IuLVVxWUOUnWQyZNmhQREbt27Rry3t7XJk+enGp3AAAAHKBkI4KHHXZYRES88847Q957++23Y8qUKQPFIgAAwFjTFXlNDR1uPmayOKdMmRKzZs3a5+qgL774Yhx33HGpdgUAAMBBSFqwzp8/P1avXh3r168feO2ZZ56JDRs2xMKFC1PuCgAAgAOUbGpoRMTll18ejz/+eFxyySWxdOnS2LVrVyxfvjzmzJkTixYtSrkrAACArHTSqqFJ45w6dWo88MAD8ad/+qdx5513xj/90z/FvHnzYvny5TFu3LiUuwIAAOAAJR0RjIg4+uij4x/+4R8OejvHHBPx2mv7f79ej+guEH2jVvyRFSOpJMxWf39OfysYHS1fZrsN28p1ye7Uy97nei5Tcox0ilz7aq5x5arQY1MKtkst1/znGlcZ7q0pv0+Tj+SFIAAAQBmVdmooAAAA+VMIAgAAlIypoQAAAAmYGgoAAEC2FIIAAAAlY2ooAABAAl2R10jbcA/+yClOAAAAWkAhCAAAUDKmhgIAACRg1VAAAACyZUQQAAAgga4YfoGWVrNYDAAAAAMUggAAACVjaigAAEAC1Y/+y8VwsRgRBAAAKBmFIAAAQMmYGgoAAJBAV+Q10jbcqqHZFoIb1tdHaFGJev9IbSIq3elORZH9FdVVbSTbVq2WbiZyvUDXrRRuly5fRfZXVMq4cpUyXxHlyFmxY6wUapc6/zlyjM1xDTUn1/5VhmNMLdfjTHkuc72+U+Y+1bby7A3l5XwAAACUTLYjggAAAJ2kEnmNtA0XS05xAgAA0AIKQQAAgJIxNRQAACABU0MBAADIlkIQAACgZEwNBQAASMDUUAAAALKlEAQAACgZU0MBAAAS6Iq8Rtq6hnkvpzgBAABoAYUgAABAyZgaCgAAkIBVQwEAAMiWQhAAAKBkTA0FAABIoCuGX6mz1awaCgAAwIAxPyJY768n21alO13dXKsl21RUq+k2VqtVk22r7u8MTSmSr0rhdun6fRTcZ1GpY6M9Up5H/as5rc59O+47ufav1vfVSqF2OX/eduq5LNrv97TN876Ta+7Jx5gvBAEAAFqh+tF/uRguFuU9AABAySgEAQAASsbUUAAAgAS6Iq+RNquGAgAAMEAhCAAAUDKmhgIAACRQibxG2oaLJac4AQAAaAGFIAAAQMmYGgoAAJCAqaEAAABkSyEIAABQMqaGAgAAJGBqKAAAANlSCAIAAJSMqaEAAAAJdEVeI21dw7yXU5wAAAC0gEIQAACgZLKdGlofoUatFGiTWr2/nmxble50sTdqww36NqerWht5f41qVAu0q9WqKUKKiIhKpMt9Sq2Pq1Jon6mvjZTHmTK2XPtFGbT+PBbr+ymVoa+mzH2u+cr1/pVSrv0rIt+c0Zx0fWzs9werhgIAAJAthSAAAEDJZDs1FAAAoJN0xfArdbaaVUMBAAAYoBAEAAAoGVNDAQAAEqh+9F8uhovFiCAAAEDJKAQBAABKxtRQAACABLoir5E2q4YCAAAwQCEIAABQMqaGAgAAJFCJvEbahoslpzgBAABoAYUgAABAyZgaCgAAkICpoQAAAGRLIQgAAFAypoYCAAAk0ElTQ7MtBCtRH7HFyG3yVe9PF3ulO113q9WKtquO2KZaLbixRPsrKmW/qSe81FPG1cnXRjNyzX+ucj3GIuexUrBdSp2cr5SK5r4M99aUUvb7XI8xtVz7WK5yPMb8IiqPzZs3R+2PvuhnWwgCAABw8JYsWRKbNm0a9JpCEAAAIIGuyGvks+uj//f29hoRBAAAKJMZM2YMeS2nghUAAIAWSD4ieO6558bzzz8/5PUFCxbEnXfemXp3AAAAWSjtqqGNRiPWr18f8+bNi/nz5w96b+bMmSl3BQAAwAFKWgi+/vrrsXPnzvjCF74QixYtSrlpAAAAEklaCL788ssREXHMMcek3CwAAED2uuLjlTpzMFwsSaewvvTSSxHxcSG4c+fOlJsHAAAggeSF4CGHHBK33nprzJ07N+bOnRvz5s2LVatWpdwNAAAAByH51NC+vr7Yvn173H777bFt27a4//7749prr40PP/wwFi9eXHxjlQI1apE2JVCvt36fRVLfaFRHP5ADkq7fpO2BBbfW4f0+3+jT5b/jj7HFikbV+q7f2flKus9CO+3we2tLt9TEPluc+9Q6+VwWv+fkmf88oxr7qh/9l4vhYulqNBqNVDt66KGHol6vx5IlSwZe++CDD+LMM8+M999/P372s59FtVowNSNVN5VKeyqgDFW6013q/f0F9lcw9dVq7eAD+kitlu6SqkS6flNP+qFUIK4x0O9T5iyllPnv6GNsgyL5akfX7+R8pVQ09x19by2oDLlPrVPPZTP3nFzzn+NnUYf/LbuQ5448Mna99lq7wxgw/ogj4v999dV9vpf0dFx00UWDisCIiAkTJsSiRYvi3XffHVhMBgAAgPZJ/kD5fZk6dWpEWDwGAAAYu7oir2m5LVk19K233oozzjgj7rrrriHvbdiwISIiZs2alWp3AAAAHKBkheD06dNj27ZtsXLlytixY8fA62+88UY8+uijcdJJJ8W0adNS7Q4AAIADlHRq6M033xxXXXVVXHjhhXHeeedFX19f9Pb2Rnd3d9x8880pdwUAAJCVSuQ1NXS4WJLGOW/evPjud78bEydOjG9/+9tx3333xQknnBAPPfTQwEPmAQAAaK+kj49Iaow/PiLHJX0jIroLjBHX68WW/y3yKIqiisRVVL2/c/tNux5f0MnLr6dkGf02asM9P9fc5/rYGvlqTsrHpuR8beea/5F31p7vmR2br6JK8PyIfz3yyNid0eMjxh1xRBy/n8dHtGTVUAAAgLGutFNDAQAAyJ9CEAAAoGRMDQUAAEjA1FAAAACypRAEAAAoGVNDAQAAEuiKvEbauoZ5L6c4AQAAaAGFIAAAQMmYGgoAAJCAVUMBAADIlkIQAACgZEwNBQAASKArhl+ps9WsGgoAAMAAhSAAAEDJmBoKAACQQDUi6u0O4g9Uh3nPiCAAAEDJKAQBAABKJtupofURatRKgTZ72qUbnC2yv6JyjaveXySuSqF2le50cfX3J9tUdFUbybbVqOW0LlRnSNn3W6/S8vhzzVcZ7ocpdXJcRT9vU8q1f6VULK7W33MiOru/ptJMv8+1j9EeXZHXSJtVQwEAABigEAQAACiZbKeGAgAAdJJK5DXSNlwsOcUJAABACygEAQAASsbUUAAAgARMDQUAACBbRgQBAAASMCIIAABAthSCAAAAJWNqKAAAQAJdkddIW9cw7+UUJwAAAC2gEAQAACgZU0MBAAASqEREo91B/AGrhgIAADBAIQgAAFAypoYCAAAk0BXDr9TZalYNBQAAYIARwSZUot7uEDpKvT9dvird6f5mUatVk22rkvAK6u8vsL+IqLfh7zft2GcRuV6TKfOV8hhzPY9F4ira98uQr9b3+0q211oRZTiPuR5jRDmOM9e4Ukp1jGM/U51FIQgAAJBAuuGGNIaLR2EOAABQMgpBAACAkjE1FAAAIIGuyGukzaqhAAAADFAIAgAAlIypoQAAAAnkNso2XDy5xQoAAMAoUwgCAACUjKmhAAAACeQ2ymZqKAAAAAMUggAAACVjaigAAEACuY2ymRoKAADAAIUgAABAyZgaCgAAkEBX5DXS1jXMeznFCQAAQAsoBAEAAErG1NAm1NXNTUmZr/7+ZJuKarWWbFuN2nAD7s2pdI+cr3o9orvAVZsyXxERlain3WAirb4mKwX3mTJfKY8x1/NYTKVQ/GW4T+v37dPqY2xH7qPgPtsh1z6W67011/M41lUiotHuIP6AqaEAAAAMUAgCAACUjKmhAAAACZgaCgAAQLYUggAAACVjaigAAEACpoYCAACQLYUgAABAyZgaCgAAkEBXDD8dMydGBAEAAEpGIQgAAFAypoYCAAAk0EmjbJ0UKwAAAAkoBAEAAErG1FAAAIAEOmmUrZNiBQAAIAGFIAAAQMmYGgoAAJBAJ42yZVsIVqI+YouR20TUO+p0HJgieWiHlHGlPI+1WjXZtrqqtWTbqhXcVH//yG2qCeOKiGjUupJuL5XW9/3W33dyvY5aHVelcLs885VSGY4RRluq75kR5biO0t13xn6uOkm2hSAAAAAHb/PmzVH7o1EHhSAAAEACXV1dEV0ZzaT6KJYlS5bEpk2bBr2lEAQAABjDent7jQgCAACUyYwZM4a8dkCF4E033RSvvvpqrFixYtDrGzdujNtuuy2effbZiIg49dRT4/rrr4+pU6ceyG4AAAA6R3f3nv9yMUwsTUe5cuXKeOSRR6Knp2fQ61u2bImvfvWrsXv37rjsssuiVqvFvffeG+vWrYuVK1fGuHHjmg8cAACA5AoXgrVaLe6+++6466679vn+9773vXjzzTfjiSeeiGOOOSYiIo4//vi49NJL47HHHovzzz8/TcQAAAAclEKF4K5du+K8886LdevWxeLFi2P16tVD2qxatSp6enoGisCIiJNPPjmOOuqoWLVqlUIQAAAY26rVvKaGVvf//OxCT3XctWtX7NixI5YtWxa33XZbdP/RwW3dujU2btwYc+bMGfJv58yZEy+88EKTEQMAADBaCpWrkydPjh/96EdDCsC93nrrrYiImD59+pD3pk2bFtu3b4/t27fHoYceehChAgAAkEKhQrBSqUSlsv/Bw76+voiImDhx4pD3xo8fHxERO3fubK4QHGZ/zbQpNOTZ8dpwlEXOT8rdtXRvxTUa+x9uHy1FUt+OuEqj5feddFvr9LiK3XZyzVdKct8+Yz/3uW+tpQp+1+ngI2xCOY4yibG8aui+NBqNEdt0ffRU+8Lq9eHfr1RGbhMR9RJ03EqMnIe0OyyW+5RyPY/Vam3kRgXVaiMXb0VTnzKuiIhGrcnrd6xqw30n5fXdyXEV7fu55isluW+fMuQ+It97RUs18V0n1/6aUrLz2OKBBIaX5GxMmjQpIvb8lvCP7X1t8uTJKXYFAADAQUoyInjYYYdFRMQ777wz5L233347pkyZMlAsAgAAjEljbdXQkUyZMiVmzZq1z9VBX3zxxTjuuONS7AYAAIAEkk3UnT9/fqxevTrWr18/8NozzzwTGzZsiIULF6baDQAAAAcp2bjl5ZdfHo8//nhccsklsXTp0ti1a1csX7485syZE4sWLUq1GwAAgDx10KqhyUYEp06dGg888ED86Z/+adx5553xT//0TzFv3rxYvnx5jBs3LtVuAAAAOEhdjSLPfmgHj48ozOMj2sfjI0rG4yMGlGEZ/VzvO3LfPmXIfUS+94qW8viIQTw+ogl//ucRGze2O4qPfeYzEb/61T7fymjccrCRLqpKgTZ72uX5wSSu5uT6QVKkeCuqSPHWaFQLtUsZV0REV8LCMmVR2eoP36L3nZRyvY5y/eKTa75yVewYK4Xa5doncu33Oec+15yllPIYy5AvmlC2VUMBAADoHApBAACAkslo3BIAAKCDlXHVUAAAADqDQhAAAKBkMhq3BAAA6GBWDQUAACBXCkEAAICSyWjcEgAAoINZNRQAAIBcKQQBAABKJqNxSwAAgA5m1VAAAABypRAEAAAomYzGLQEAADqYVUMBAADIlUIQAACgZDIatwQAAOhgHbRqaEZRDlaJ+ogtRm4TUU846Flkf52uSL4qhduN/XylPMZGrStZu65q7WDDGaRW2/9NpFkpY2vU0uW/DPeKzj7GYvf8lDo7X61XhmPMVVlyn/I4R7q+i37X2dM2XVytPMZ2bMtUxLw4HwAAACWT7YggAABAR7FqKAAAALlSCAIAAJRMRuOWAAAAHayDVg01IggAAFAyCkEAAICSyWjcEgAAoINZNRQAAIBcKQQBAABKJqNxSwAAgA5m1VAAAABypRAEAAAomYzGLQEAADqYVUMBAADIlUIQAACgZDIatwQAAOhgVg0FAAAgVxmVq4PVR6hRKwXa7GlXTxRRORTLV0VeM9eodSXdXle1lmxbKWPrqjaSbauW7hAL3ZuKcq21T8rc6xPNka+xI9frqAzy7PvOYU6yLQQBAAA6ilVDAQAAyFVG5SoAAEAHs1gMAAAAuVIIAgAAlExG45YAAAAdzGIxAAAA5EohCAAAUDIZjVsCAAB0MKuGAgAAkCuFIAAAQMlkNG4JAADQwawaCgAAQK4UggAAACWT0bglAABAB7NqKAAAALlSCAIAAJRMRuOWAAAAHayDVg3NKEoOVCXqybZVLzBIXCnYLqVWH2Ou2pH7iIhaLd22KgnvOo1aun5RJK56vdi9vb//4OMZ2GfC853yOqI5Zch9J99biyrD5+2e/eb5mdva66jSluu2c/NFJxr7d20AAAAGMSIIAACQglVDAQAAyJVCEAAAoGQyGrcEAADoYB20aqgRQQAAgJJRCAIAAJRMRuOWAAAAHcyqoQAAAORKIQgAAFAyGY1bAgAAdDCrhgIAAJArhSAAAEDJZDRuCQAA0MGsGgoAAECuFIIAAAAlk9G4JQAAQAezaigAAAC5yqhcHawS9RFbjNwmop6w1i2yv05X7Bhbn/tcz2Pr42p97lPr70+3rUp3uuMsGleRdtVq7eCC+QONWleybaWU6zXJ2JDrfTpXOV9DuZ5LcTUn5z5GMZs3b45abfD3k2wLQQAAgI6S6aqhS5YsiU2bNg16K6MoAQAASK23t9eIIAAAQJnMmDFjyGsHVAjedNNN8eqrr8aKFSsGvX7uuefG888/P6T9ggUL4s477zyQXQEAAHSGDlo1tOkoV65cGY888kj09PQMer3RaMT69etj3rx5MX/+/EHvzZw5s9ndAAAAMEoKF4K1Wi3uvvvuuOuuu/b5/uuvvx47d+6ML3zhC7Fo0aJkAQIAAJBWoUJw165dcd5558W6deti8eLFsXr16iFtXn755YiIOOaYY9JGCAAA0AkyXTV0Xwo9YGTXrl2xY8eOWLZsWdx2223RvY+De+mllyLi40Jw586dBxIqAAAAo6xQuTp58uT40Y9+tM8CcK+XXnopDjnkkLj11lvjySefjJ07d8ZnPvOZuOaaa+KMM85IFjAAAAAHp1AhWKlUolIZfvDw5Zdfjr6+vti+fXvcfvvtsW3btrj//vvj2muvjQ8//DAWL17cXGQj7K9om0JDnoWl3VpHa3nuU0oXWVv6V0fnPq16vfX7LHJrajT2Pw1jrMi17zNKOvjzttN7TbFu3+lHWUzL+1jBe045+n45+lgSY3nV0P05//zzo16vx5IlSwZeO+OMM+LMM8+MO+64I84666yoDjNHdYiRvuFVKoW+BdaTXlBt+NaZozbkPqWU57Hl/avDc59ayvtsf//IbQqmP6rV2siNCmrUupJtK6Vc+z6joMM/bzv5fli025fl+0lL+1gT95wy9P1kcfmDXlaSnY2LLrpoUBEYETFhwoRYtGhRvPvuuwOLyQAAANBeoz5uOXXq1IiweAwAADDGjbVVQ0fy1ltvxRlnnLHPZwxu2LAhIiJmzZqVYlcAAAAcpCSF4PTp02Pbtm2xcuXK2LFjx8Drb7zxRjz66KNx0kknxbRp01LsCgAAgIOUbNzy5ptvjquuuiouvPDCOO+886Kvry96e3uju7s7br755lS7AQAAyFMHrRqabLGYefPmxXe/+92YOHFifPvb34777rsvTjjhhHjooYcGHjIPAABA+3U1Go1Gu4PYJ4+PyFeHP8Kgo5dm7vDcp+bxEe2Ta99nFHT4520n3w89PmIwj49oTpZxleHxEStWRGzf3u4oPnbooRF/+Zf7fCujccvBRuq8lQJt9rQb4xdUtP5Drh25pzllyX1/f7q+X6SorNeLtavV0j1QvpLwLl3vH/v9opO/9BeV6/XdyZ9r7VAsX5VC7cqQr9RSfc8si1S5KEVGy7ZqKAAAAJ1DIQgAAFAyGY1bAgAAdLAyrhoKAABAZ1AIAgAAlExG45YAAAAdzKqhAAAA5EohCAAAUDIZjVsCAAB0MKuGAgAAkCuFIAAAQMlkNG4JAADQwawaCgAAQK4UggAAACWT0bglAABAB7NqKAAAALlSCAIAAJRMRuOWAAAAHayDVg3NKMrRUc900DPXuFJKeYyVqCfblty3V8pzmVK9v0hclULtKt3p8t/fn2xT0VVtJNtWo9baa7JSuF2e/SulMtxbxbV3f/r9H8r1sy3X/Od6HZEPZxUAAKBkxvyIIAAAQEtYNRQAAIBcKQQBAABKJqNxSwAAgA7WQauGGhEEAAAoGYUgAABAyWQ0bgkAANDBrBoKAABArhSCAAAAJZPRuCUAAEAHs2ooAAAAuVIIAgAAlExG45YAAAAdrLs7olZrdxQfs2ooAAAAeykEAQAASsbUUAAAgBSsGgoAAECuMipX81eJertD2Kd6wnq+2DFWss1Fq+Wah9Rxpexjre+vrVfvTxdXpTtdvmq1/f9VsFmVhJ8exfLV+vtOyr6aUso8FDnGSuF2rY2rqFzjSinXuMoi1/4KI1EIAgAApNDdHVHPqKC3aigAAAB7KQQBAABKxtRQAACAFKwaCgAAQK4UggAAACWT0bglAABAB+vujmg02h3Fx6waCgAAwF4KQQAAgJIxNRQAACCFajWvqaFWDQUAAGAvhSAAAEDJmBoKAACQQk4Pk4+waigAAAAfy6xkBQAA6FDDLM7SFhaLAQAAYC+FIAAAQMmYGgoAAJBCd3dEV1e7o/jYMFNDsy0EK1EfscXIbSLqCQc9U24rpSJ56HS55j6lIsdYKdwubZ/Qx/ZoR/7r/em2VelOdx319yfbVKG46vWC7VLmK+V5zPQeVuwYi33eppTrPacMfSIi3/ynlHP+UynDMXJw9BAAAICSyXZEEAAAoKNUq3lNDa3sf9zPiCAAAEDJKAQBAABKxtRQAACAFLq796xwlgtTQwEAANhLIQgAAFAypoYCAACkUK0OOx2z5YZZwTSjKAEAAGgFhSAAAEDJmBoKAACQQnd3RKPR7ig+ZmooAAAAeykEAQAASsbUUAAAgBSq1XZHUJgRQQAAgJJRCAIAAJSMqaEAAAApdHdOeWVEEAAAoGSyLVnrI9SolQJt9rSrJ4qo2P46XbGctj4XKc9jSmXoExFpjzPXa7JYXJVC7XLtF/X+dLnvqqZ7RlKtVuyH9f39I7epdKfLfZH9tYPrsTm5Xo9lkWu/yFWu+XIddb7NmzdHrVYb9Fq2hSAAAEAnybForkTEkiVLYtOmTYNeVwgCAACMYb29vUYEAQAAymTGjBlDXitcCD799NNx9913xwsvvBCVSiWOP/74uPrqq+OEE04YaLNx48a47bbb4tlnn42IiFNPPTWuv/76mDp16sFHDwAAkLEcf18+bty+X+9qNBoj/uL/2WefjYsvvjiOPfbYOOecc6K/vz8efPDBePvtt+PBBx+Mz372s7Fly5Y455xzYvfu3XHxxRdHrVaLe++9N2bOnBkrV66McfuLYD/qI/y+tVIZuU1EOX4o2+pjLJr7lHL9gXjLF81pQ7+PKMeP1wvFVfAElOFe0erFYor2/ZSrduf4YZ5arv0+1/tEq7Xj8zYi3/y3Mq525T6lHM9jpXMvx8J27253BEPtrwwr9JH5d3/3dzFjxox45JFHYuLEiRERsXjx4li4cGEsW7Ys7rvvvvje974Xb775ZjzxxBNxzDHHRETE8ccfH5deemk89thjcf7556c5EgAAAA7KiHX51q1bY+3atXH66acPFIEREZ/61KfixBNPjF//+tcREbFq1aro6ekZKAIjIk4++eQ46qijYtWqVaMQOgAAQD5qtT0zSnL574/WhxlkxBHByZMnx1NPPTWoCNxry5YtUa1WY+vWrbFx48ZYsGDBkDZz5syJn/70p81lEAAAgFEz4ohgtVqNI488MqZPnz7o9bVr18Zzzz0Xc+fOjbfeeisiYkibiIhp06bF9u3bY/v27YlCBgAA4GAc0M/q+/r64pvf/GZERFxxxRXR19cXEbHPUcPx48dHRMTOnTvj0EMPLbyPIj8mLfaD05Q/Us5V64+x9T/2zTP77Yiq1f0+5621Ja4CJyDP3hqRMrKRlxlLr0jf7/TFHVov136f632i9dqzuEau+W9tXJ2/sEmu53Fs6+9vz2fk/nR17f+9pgvB999/P6688spYu3ZtfP3rX4+enp547rnnCgQxTBT7YNXQ4qwa2j5WDW1ertdkrqsnpmTV0OZYNXRvI6uGtotVQwezamhzcjyPnV9cjy1NnY5t27bF0qVL4xe/+EWcc845cc0110RExKRJkyIiYteuXUP+zd7XJk+efLCxAgAAkEDhv53+7ne/i6997WuxZs2auOCCC+Jv/uZvBkb5DjvssIiIeOedd4b8u7fffjumTJkyUCwCAACMRbVaXqPJw43CFioEd+zYMVAEXnLJJXHDDTcMen/KlCkxa9aseOGFF4b82xdffDGOO+645iIGAABg1BSaGnrLLbfEmjVr4uKLLx5SBO41f/78WL16daxfv37gtWeeeSY2bNgQCxcuTBMtAAAAB62r0Rh+XZv169fHwoULY8qUKXHDDTdEtTr0R/2LFi2K9957L84888yoVquxdOnS2LVrVyxfvjwOP/zwePjhh2PcuHFNBWaxmOIsFtM+FotpXq7XZK6LZqRksZjmWCxmbyOLxbSLxWIGs1hMc3I8j2VYLObdd/PqO5VKxKc+te/3RiwEH3roofjrv/7rYXewbt26iIh45ZVX4tZbb41f/vKXMWHChDjllFPiuuuui6lTpzYdtEKwOIVg+ygEm5frNZnrF+KUFILNUQjubaQQbBeF4GAKwebkeB4Vgq13UIVguygEi1MIto9CsHm5XpO5fiFOSSHYHIXg3kYKwXZRCA6mEGxOjudRIdh6wxWCCT8yYbAcb0AR+RaVxeKqFGqX+otPZ+es9XKNK6VGrblnww6nUuCTqF4vVuSlLN5yLSpz7V/u+aOhPff8lDo7/8Xk2vfzvI7y7aup1Gp7/svFPn7VN2Dsnw0AAAAGUQgCAACUjKmhAAAACfT35zU1dLjVYIwIAgAAlIxCEAAAoGRMDQUAAEigVuucxw8ZEQQAACgZhSAAAEDJmBoKAACQQH+/qaEAAABkSiEIAABQMqaGAgAAJJDbqqFdXft/z4ggAABAySgEAQAASsbUUAAAgARyWzXU1FAAAAAGKAQBAABKxtRQAACABHJbNbQyzLBftoVgJeojthi5TVqt3l9R9YQDu8WOsVjuWx9XMSnjSqlIXJXC7dL21VzPZa5yzVeucRX9wCzSrlqtHVwwf6BWqybbVnfCT9t6Rl8wOkGu9/yU11AZ7quppfyeWYY+luoY88xUeTkfAAAAJZPtiCAAAEAnyW3V0OGmhhoRBAAAKBmFIAAAQMmYGgoAAJBAbquGVodZ98yIIAAAQMkoBAEAAErG1FAAAIAEcls11NRQAAAABigEAQAASsbUUAAAgARyWzW0e5hqz4ggAABAySgEAQAASsbUUAAAgARyWzV0uFiMCAIAAJSMQhAAAKBkTA0FAABIILdVQ2u1/b9nRBAAAKBkjAiOAZWotzuEfUoZVz3h3yw6O65Ktue7qJQ5S6nT89rJUvb9Rq3r4APau8eEn5Ap/zpc6U53DdX78+z3uV6Pud6/UseVa/5TGilnlQJtPm479vOV7hjzvIbKSiEIAACQgFVDAQAAyJZCEAAAoGRMDQUAAEjAqqEAAABkSyEIAABQMqaGAgAAJGDVUAAAALKlEAQAACgZU0MBAAASsGooAAAA2VIIAgAAlIypoQAAAAlYNRQAAIBsKQQBAABKxtRQAACABKwaCgAAQLYUggAAACWT7dTQ+gg1aqVAmz3t6okigqFS9q9i/blYu9RyvY7akYsiWt0viurk89iOvp9yak+1OszcnCbVatVk2+qqNkZs02gUa5cyrpRS9vtcr6Fc44rI9z5dBjnmPr+I0rNqKAAAANlSCAIAAJRMtlNDAQAAOolVQwEAAMiWQhAAAKBkTA0FAABIwKqhAAAAZEshCAAAUDKmhgIAACRg1VAAAACyZUQQAAAgAYvFAAAAkC2FIAAAQMmYGgoAAJCAxWIAAADIlkIQAACgZEwNBQAASMCqoQAAAGQr2xHBStRHbDFyG5pVL/C3gUrhdnmenyKxt0OxfLWn36fMWcr4c+1jKaU8xs4+j519z6/Vqsm2Va0O88v/JhWNq0i7lHE1al3JtpVSqz8/in7e5izXe1hKucaVUrrzOPZzlavNmzdH7Y9Wjsm2EAQAAOgkua4aumTJkti0adOg9xSCAAAAY1hvb68RQQAAgDKZMWPGkNcKF4JPP/103H333fHCCy9EpVKJ448/Pq6++uo44YQTBtqce+658fzzzw/5twsWLIg777zzwKIGAADoAJ20amihQvDZZ5+Nyy+/PI499ti45ppror+/Px588MH4yle+Eg8++GB89rOfjUajEevXr4958+bF/PnzB/37mTNnHtQBAAAAkE6hQvDv/u7vYsaMGfHII4/ExIkTIyJi8eLFsXDhwli2bFncd9998frrr8fOnTvjC1/4QixatGhUgwYAAODAjVgIbt26NdauXRuXXnrpQBEYEfGpT30qTjzxxPj5z38eEREvv/xyREQcc8wxoxQqAABAvnJdNXRfRiwEJ0+eHE899dSgInCvLVu2RLW659lCL730UkR8XAju3LkzJk2adCDxAgAAMIpGfKpjtVqNI488MqZPnz7o9bVr18Zzzz0Xc+fOjYg9heAhhxwSt956a8ydOzfmzp0b8+bNi1WrVo1O5AAAAByQA3p8RF9fX3zzm9+MiIgrrrgiIvZMDe3r64vt27fH7bffHtu2bYv7778/rr322vjwww9j8eLFze2kMmKNWqwNTSma0WKpT3d+ynGmCx5lG/p92j12+Nns4PtOx59HuY+IiEajmnBrxRRJfTviarV29MAO7vYf6dzvAp2f+5Qko6gxt2roH3r//ffjyiuvjLVr18bXv/716OnpiYiI888/P+r1eixZsmSg7RlnnBFnnnlm3HHHHXHWWWcNTCMtpF4f/v1KZeQ2NK1e4EIvmvpKpDs/ReLqdIXy1aZ+nzL/KftFy3X4faejz6PcD6hWh/nBR5NqtZE/l4umPmVcjVpXsm2l1OrPog7v9hHRud8FxkLuU0p2HlXXWWnqbGzbti2WLl0av/jFL+Kcc86Ja665ZuC9iy66aFARGBExYcKEWLRoUbz77rsDi8kAAADQXoVHBH/3u9/F1772tVizZk1ccMEF8Td/8zfR1TXyX+ymTp0aEXsWjwEAABirOmnV0EIjgjt27BgoAi+55JK45ZZbBhWBb731Vpxxxhlx1113Dfm3GzZsiIiIWbNmNRk2AAAAo6FQIXjLLbfEmjVr4uKLL44bbrhhyPvTp0+Pbdu2xcqVK2PHjh0Dr7/xxhvx6KOPxkknnRTTpk1LFzUAAAAHbMSpoevXr4/HH388pkyZEn/2Z38Wjz/++JA2ixYtiptvvjmuuuqquPDCC+O8886Lvr6+6O3tje7u7rj55ptHJXgAAIBcjKlVQ5999tmI2LNQzL5GAyP2FILz5s2L7373u3HPPffEt7/97ZgwYUL09PTEtddeO/CQeQAAANqvq9FoNNodxD55fERbeHxE+3h8RAfo8PtOR59HuR/g8RHt4/ERzevU7wJjIfcpeXxEcTfeGPHee+2O4mNTp0b87d/u+70DeqB8K4x0sVcKtNnTzlXcjGL5qhRql2vxVoY+kWvuI/ItRordT1p/38k1X7nKNV8pt1WkeCuqSPHWaFQLtUsZVyXht5N6f57nsege2/F5m+u9orVxFct9arnew1LFle+3k3TG3KqhAAAAjB0KQQAAgJLJdmooAABAJ+mkVUONCAIAAJSMQhAAAKBkTA0FAABIwKqhAAAAZEshCAAAUDKmhgIAACRg1VAAAACypRAEAAAoGVNDAQAAErBqKAAAANlSCAIAAJSMqaEAAAAJWDUUAACAbCkEAQAASsbUUAAAgAQ6adVQhSCjphL1ZNuqZzp43eq4Km3Y55795nkuU26r2DFWCrVrfVzF5HodFVG076fMV65SHmOj1pWsXSXhN4qUX6K6qo1k2yqar1YrQ7+PcG+F1PRcAACAkjEiCAAAkIBVQwEAAMiWQhAAAKBkTA0FAABIoJNWDTUiCAAAUDIKQQAAgJIxNRQAACABq4YCAACQLYUgAABAyZgaCgAAkIBVQwEAAMiWQhAAAKBkTA0FAABIwKqhAAAAZEshCAAAUDKmhgIAACRg1VAAAACyle2IYCXqI7YYuU1a9Uzr5lbnoR1SHmOu5zGlnPtEzrGlkusx5hpXMcXu+ble352d+2Lq/emOsavaSLatWq2abFuVhN+aiowYVKJYn865f6W8Jlv5XaBo7nPW6fEz+rItBAEAADqJVUMBAADIlkIQAACgZEwNBQAASMCqoQAAAGRLIQgAAFAypoYCAAAkYNVQAAAAsqUQBAAAKBlTQwEAABKwaigAAADZUggCAACUjKmhAAAACVg1FAAAgGwpBAEAAErG1FAAAIAErBoKAABAthSCAAAAJTPmp4bWE9a6lagn21bKuBgbivWvStJ+WFSu11FKRY6xUrBdSrned1odV9Hc55qvMuQ+peGmMjWrO+E3nZTTvarVkQ+y0agWalerVVOExAHK9XMtJd9bi7NqKAAAANlSCAIAAJTMmJ8aCgAA0ApWDQUAACBbCkEAAICSMTUUAAAgAauGAgAAkC2FIAAAQMmYGgoAAJCAVUMBAADIlkIQAACgZEwNBQAASMCqoQAAAGRLIQgAAFAypoYCAAAkYNVQAAAAsqUQBAAAKBlTQwEAABLopFVDFYJNqCccQK1EPdm2UipyjJXC7fI8xpRxpewTKXOfWq7nMqVix1hpeS7acb6LyDWuMsg197neJ/r70+WrWh3mxzZNqtWqydp1J/42V+/P83MypZH7a+vv9xH5ftdMt608+0MZbN68OWp/9INBhSAAAMAYtmTJkti0adOg1xSCAAAACTQa9Wg02h3Fx/bEUone3l4jggAAAGUyY8aMIa8Vnqi7evXquOiii2Lu3Lnx7//9v49vfetb0dfXN6jNxo0b46/+6q+ip6cnenp64rrrrov33nvv4CMHAAAgmUIjgqtXr46lS5fGnDlz4hvf+EZs3rw57r///vjNb34Tvb29UalUYsuWLfHVr341du/eHZdddlnUarW49957Y926dbFy5coYN27caB8LAABAG6VbVCqdfY/9FSoE77jjjpgxY0Y88MADMWHChIjYM7x4yy23xNNPPx2nnHJKfO9734s333wznnjiiTjmmGMiIuL444+PSy+9NB577LE4//zzEx0IAAAAB2PEqaG7du2KT37yk3H++ecPFIERET09PRERsW7duoiIWLVqVfT09AwUgRERJ598chx11FGxatWq1HEDAABkph57RgVz+W//j/4YcURw/Pjxce+99w55fc2aNRERcdhhh8XWrVtj48aNsWDBgiHt5syZEz/96U9H2g0AAAAt0vSqoZs2bYpf/OIXcdttt8Xs2bPji1/8Yrz22msRETF9+vQh7adNmxbbt2+P7du3x6GHHnrwEQMAAHBQmioEf//738dpp50WERETJ06MG2+8McaPHz+weujEiROH/Jvx48dHRMTOnTsVggAAwBhWi4iMHiQYXft9p6lCsKurK5YtWxa7d++OFStWxKWXXhrLli2LadOmFfq3TakUeLJFgTaFn4/RcnlGVjSqIqcn12NMqR1HWCz3yffajp3mqcUnQOY/1ur7jtx/rJPv+SmjajSqCbdWTJHc1/f/E6AD3WuGW0q/tZF31/o+3dH5ouM0VQh+4hOfiIULF0ZExOmnnx5nnnlm3HrrrfHf/tt/i4g9C8v8sb2vTZ48ubnIRrqrVSqF7nz1TC+CyjA/3GynIvkqmPpsjzGlVvevorlPvt8SnMtC2nACcr2HtVo77jtyv0en3/NTnsdqNd2y8LXayEVl0dx3N/1Dn+HV+/O8jlrax9r0gdux+SqqPX/NZj8O+GxMmDAhTj311Ni8eXN8+tOfjoiId955Z0i7t99+O6ZMmRKTJk068CgBAACyV8/wv30bsRBcv359nHbaadHb2zvkvb6+vujq6opx48bFrFmz4oUXXhjS5sUXX4zjjjtupN0AAADQIiMWgkcccURs3749Hn744di9e/fA65s2bYof/vCHceKJJ8bkyZNj/vz5sXr16li/fv1Am2eeeSY2bNgwMJ0UAACA9utqNBojLmvz+OOPx3XXXRcnnHBCfPnLX44tW7ZEb29vfPjhh/Hggw/G7Nmz47333oszzzwzqtVqLF26NHbt2hXLly+Pww8/PB5++OEYN25cc5H5jWBb+I1gc/xGsGT8RrBt/EawfTr9nu83gs3zG8HwG8HRUoLfCB555LZ47bV8cn/EEZV49dUp+3yvUCEYEfHkk0/G8uXL47e//W1MmjQpPve5z8U111wTRx111ECbV155JW699db45S9/GRMmTIhTTjklrrvuupg6dWrzUSsE20Ih2ByFYMkoBNtGIdg+nX7PVwg2TyEYCsHRohBsuSSFYMspBNtCIdgchWDJKATbRiHYPp1+z1cINk8hGArB0aIQbLnhCsHEtw4AAICyqsVwK3W23v7H/BSCbZLrX3yKbatSqF2ux5hS6+Nqfe5zlmsfK0NcKZXhvlOGuHI9xpTbatS6km2rq8DoYqNRLTQKWWR0sRmV7nTnsr8/2aay7WNlkCr3eX4KlZfzAQAAUDJGBAEAAJLonKmhRgQBAABKRiEIAABQMqaGAgAAJFGPPdND82dEEAAAoGQUggAAACVjaigAAEAS9chr1dD9P//UiCAAAEDJKAQBAABKxtRQAACAJGph1VAAAACypBAEAAAoGVNDAQAAksjtgfJWDQUAAOAjCkEAAICSMTUUAAAgidxWDTU1FAAAgI+M+RHBStTbHcKoqyes51PmK+W2cj1GmpfyXNI+rb6+K4Xb5Xl95xpXSmW456eMq1ErFlejtv+/5u/VVU07+lCrVZNtq5owtiK56HRluFeQjzFfCAIAALSGqaEAAABkSiEIAABQMqaGAgAAJNGIyOq3no39vmNEEAAAoGQUggAAACVjaigAAEASua0auv9xPyOCAAAAJaMQBAAAKBlTQwEAAJIwNRQAAIBMKQQBAABKxtRQAACAJOqR19TQ6n7fMSIIAABQMgpBAACAkjE1FAAAIIncVg3dfyxGBAEAAEpGIQgAAFAy2U4NrY9Qo1YKtNnTrp4oomL7a4eUx5hSrvlKGVer+1fRfp+zMlyTucr1XpFSrtd3Sq0/xkrLc+E+0Zxabf+rAh6IajXdtLaUsXUljKtR60q2rVz7WMrrKN228sxVWvWP/svF/mMpw9kAAADgDygEAQAASibbqaEAAACdJbcHypsaCgAAwEcUggAAACVjaigAAEASHigPAABAphSCAAAAJWNqKAAAQBJWDQUAACBTCkEAAICSMTUUAAAgCauGAgAAkCmFIAAAQMmYGgoAAJBEPYZbqbP1rBoKAADARxSCAAAAJWNqKAAAQBKd80B5hWATKgnn+9YTDsam3FYRlYL7zDVfuSqWr0qhdjnnK2Vsre5jRft+Srmey5S5T9n3c5XreUzZ73O953dyvykq9TE2al3JttVVTfeFuFarJttWZYRvwPV6RKW7WD/s708Q0Edy7a+prsk874Tl5XwAAACUjBFBAACAJDxQHgAAgEwpBAEAAErG1FAAAIAkTA0FAAAgUwpBAACAkjE1FAAAIIlGDPcQ99Zr7PcdI4IAAAAloxAEAAAoGVNDAQAAkrBqKAAAAJlSCAIAAJSMqaEAAABJmBoKAABAphSCAAAAJWNqKAAAQBL1yGtq6P4fbq8QBAAAGMM2b94ctdrgAjXbQrAyTPW6t8XIbRgrcj3X9YSzq3M9xtRyPc5icRW776TsF7lq9TFW2rLPPPtqSrn2+5S5zzWusmjUupJtq6uabpSlVquO2Ka/v9i2uhN+m64X3GehbWXZ98f+52OulixZEps2bRr0WraFIAAAQGfJc9XQ3t7ezhkRBAAA4ODNmDFjyGvGZwEAAEqm8Ijg6tWr484774y1a9fG5MmT4/TTT4+rr746DjnkkIE25557bjz//PND/u2CBQvizjvvTBMxAABAluox3EqdrXeQq4auXr06li5dGnPmzIlvfOMbsXnz5rj//vvjN7/5TfT29kalUolGoxHr16+PefPmxfz58wf9+5kzZx5c/AAAACRTqBC84447YsaMGfHAAw/EhAkTImLPPNNbbrklnn766TjllFPi9ddfj507d8YXvvCFWLRo0agGDQAAwIEb8TeCu3btik9+8pNx/vnnDxSBERE9PT0REbFu3bqIiHj55ZcjIuKYY44ZjTgBAAAyt/eB8rn8dxBTQ8ePHx/33nvvkNfXrFkTERGHHXZYRES89NJLEfFxIbhz586YNGnSSJsHAACgxZpeNXTTpk3x6KOPxre+9a2YPXt2fPGLX4yIPYXgIYccErfeemvMnTs35s6dG/PmzYtVq1YlDxoAAIAD19RzBH//+9/HaaedFhEREydOjBtvvDHGjx8fEXumhvb19cX27dvj9ttvj23btsX9998f1157bXz44YexePHi5iKrFKhRi7TJVOdGvkex1Hf6UY4s7REW3FqB5KfP/Ng/l4W1Jf9EtOOWn26H+faJXO87uea+DWeyg7/rpNZotHZ/RVNfT7pApL7f+fJ8oPy+dDUaxS+rrVu3xs9//vPYvXt3rFixItasWRPLli2LBQsWxEMPPRT1ej2WLFky0P6DDz6IM888M95///342c9+FtVqtXjMI11VlUrqK6+l6h18QRVNfSWrpXNHR8rzWChfBZOfun+V4VwW0qb8055bfsp+n2ufyPW+k2vuW34v7PDvOql1VdNVgrXa8N9Jm0l9d1PDKsOr94/xvl+CP2wceeTD8dprO9odxoAjjpgcr7564T7fa6oQ/EN7i7z+/v7453/+5/22+853vhN33XVX/I//8T/iT/7kT4rvQCGYLYXgxxSCJaMQbBuF4OjI9b6Ta+4Vgu2lEGxyWzn2fYVgyw1XCB7w2ZgwYUKceuqpsXnz5njvvff2227q1KkRsWfxGAAAgLGr3auE7uu/fRuxEFy/fn2cdtpp0dvbO+S9vr6+6Orqivfffz/OOOOMuOuuu4a02bBhQ0REzJo1a6RdAQAA0AIjFoJHHHFEbN++PR5++OHYvXv3wOubNm2KH/7wh3HiiSfGzJkzY9u2bbFy5crYsePjodA33ngjHn300TjppJNi2rRpo3MEAAAANGXEWc3d3d1x4403xnXXXRd/+Zd/GV/+8pdjy5Yt0dvbG5VKJW666aaIiLj55pvjqquuigsvvDDOO++86Ovri97e3uju7o6bb7551A8EAACgvfY+UD4X+/99Z+HFYp588slYvnx5/Pa3v41JkybF5z73ubjmmmviqKOOGmjz4x//OO65555Yu3ZtTJgwIXp6euLaa68deMh8czFbLCZXFov5mMViSsZiMW1jsZjRket9J9fcWyymvSwW0+S2cuz7pVgsZkW89tr2docx4IgjDo1XX/3Lfb53wKuGjjqFYLYUgh9TCJaMQrBtFIKjI9f7Tq65Vwi2l0KwyW3l2PcVgi03XCGYsOvmKcuLIPG2Usr1y0pKufaJInFVCrfLs3+VRRnyn+u9QgHRnFzvO7n2L5qX8lw2agm/h43wDbheL17g9fcffDx7pSx2U+Yr1Xksx5Vdj+GmY7be/mMpx/kAAABgwJgfEQQAAGiN4Z/d13oH8RxBAAAAxhaFIAAAQMmYGgoAAJBE5zxH0IggAABAySgEAQAASsbUUAAAgCSsGgoAAECmFIIAAAAlY2ooAABAElYNBQAAIFMKQQAAgJIxNRQAACAJq4YCAACQKYUgAABAyZgaCgAAkEQ9hlups/WsGgoAAMBHxvyIYCWrivxj9YQ1eBmOsQyKncdKoXapc1+GPpbrMaaUa77cK8YG57E5RfJVKdgutVLcD/tHOsZKgTZ7dFUbBx/QR2q1arJtVRJ+y+/vT7ct8jHmC0EAAIDW8EB5AAAAMqUQBAAAKBlTQwEAAJLwQHkAAAAypRAEAAAoGVNDAQAAkjA1FAAAgEwpBAEAAErG1FAAAIAkGjHcQ9xbr7Hfd4wIAgAAlIxCEAAAoGRMDQUAAEjCqqEAAABkSiEIAABQMqaGAgAAJGFqKAAAAJlSCAIAAJTMmJ8aWk9Y61ayejjkx1IeY0q55iulVue+UnCfZch9RNrjLJbXdpzzdMfY6nwVVSyuSqF2ud4Pc40rpVzvO7nmPmW/z1mu8Y/UL5q53zdqCe/TCb+Z9/en21Z3griOOCJiw4aD307+6pHX1ND99888744AAACMGoUgAABAyYz5qaEAAACtYdVQAAAAMqUQBAAAKBlTQwEAAJKox3ArdbaeVUMBAAD4iEIQAACgZEwNBQAASMID5QEAAMiUQhAAAKBkTA0FAABIwgPlAQAAyJRCEAAAoGRMDQUAAEjC1FAAAAAypRAEAAAoGVNDAQAAkuicB8pnWwgeeXQjXntt/+83GhFd1caI22nU9n/wzaonHECtDHNS2inlMZYhX2WR67lsfVyVQu1yvY5yVeQYK4XbdXL/KibXPpFrXLnmPte4IvL9zG1t/ovd7yPSxtXfn2xT0Z3wW369P1WfyPM+UVbOBgAAQMlkOyIIAADQWawaCgAAQKYUggAAACVjaigAAEAS9Rhupc7W238sRgQBAABKRiEIAABQMqaGAgAAJNE5D5Q3IggAAFAyCkEAAICSMTUUAAAgCQ+UBwAAIFMKQQAAgJIxNRQAACAJq4YCAACQKYUgAABAyZgaCgAAkEQ9hpuO2Xr7jyXbQnDWrJHbHHHE6McBAACdxvfk9pg16/9pdwiD7I1n8+bNUasN/u1iV6PRaLQjKAAAAEbXBx98EJ///Odj69atg15XCAIAAIxR27Zti23btg15XSEIAABQMlYNBQAAKBmFIAAAQMkoBAEAAEpGIQgAAFAyCkEAAICSUQgCAACUjEIQAACgZBSCAAAAJdPd7gCatXHjxrjtttvi2WefjYiIU089Na6//vqYOnVqmyMb+84999x4/vnnh7y+YMGCuPPOO9sQ0dh30003xauvvhorVqwY9LrroDX2l3/XQnpPP/103H333fHCCy9EpVKJ448/Pq6++uo44YQTBtro96OjSO71+dGzevXquPPOO2Pt2rUxefLkOP300+Pqq6+OQw45ZKCNvj86iuRe32cs66hCcMuWLfHVr341du/eHZdddlnUarW49957Y926dbFy5coYN25cu0McsxqNRqxfvz7mzZsX8+fPH/TezJkz2xTV2LZy5cp45JFHoqenZ9DrroPW2F/+XQvpPfvss3H55ZfHscceG9dcc0309/fHgw8+GF/5ylfiwQcfjM9+9rP6/Sgpknt9fvSsXr06li5dGnPmzIlvfOMbsXnz5rj//vvjN7/5TfT29kalUtH3R0mR3Ov7jHmNDvL3f//3jT/7sz9rvPzyywOv/fznP2/Mnj278f3vf7+NkY19//f//t/G7NmzG//9v//3docy5vX39ze+853vNP7kT/6kMXv27MZXvvKVQe+7DkbXSPl3LaS3aNGixqmnntrYuXPnwGvvvPNO48QTT2xccskljUZDvx8tRXKvz4+es88+u/Ef/sN/aLz//vsDrz3wwAON2bNnN/75n/+50Wjo+6OlSO71fca6jvqN4KpVq6KnpyeOOeaYgddOPvnkOOqoo2LVqlVtjGzse/nllyMiBuWe9Hbt2hVnn312fOc734lFixbF9OnTh7RxHYyeIvl3LaS1devWWLt2bZx++ukxceLEgdc/9alPxYknnhi//vWvI0K/Hw1Fc6/Pj45du3bFJz/5yTj//PNjwoQJA6/vnYWwbt26iND3R0PR3Ov7jHUdMzV069atsXHjxliwYMGQ9+bMmRM//elP2xBVebz00ksR8fHNcOfOnTFp0qR2hjQm7dq1K3bs2BHLli2LhQsXxmmnnTbofdfB6Bop/xGuhdQmT54cTz311KBCZK8tW7ZEtVrV70dJkdxH6POjZfz48XHvvfcOeX3NmjUREXHYYYfp+6OkSO4j9H3Gvo4ZEXzrrbciIvb5F/pp06bF9u3bY/v27a0OqzReeumlOOSQQ+LWW2+NuXPnxty5c2PevHn+GpnY5MmT40c/+lEsXLhwn++7DkbXSPmPcC2kVq1W48gjjxzSp9euXRvPPfdczJ07V78fJUVyH6HPt8qmTZvi0UcfjW9961sxe/bs+OIXv6jvt8i+ch+h7zP2dcyIYF9fX0TEPv9yOX78+IjY85eaQw89tKVxlcXLL78cfX19sX379rj99ttj27Ztcf/998e1114bH374YSxevLjdIY4JlUolKpX9/33GdTC6Rsp/hGuhFfr6+uKb3/xmRERcccUV+n0L/XHuI/T5Vvj9738/MANh4sSJceONN8b48eP1/RbYX+4j9H3Gvo4pBBuNxohturq6WhBJOZ1//vlRr9djyZIlA6+dccYZceaZZ8Ydd9wRZ5111sA0IkaP66D9XAuj6/33348rr7wy1q5dG1//+tejp6cnnnvuuRH/nX5/8PaV+wh9vhW6urpi2bJlsXv37lixYkVceumlsWzZspg2bVqhf8uB21/uFyxYoO8z5nXM1NC9c7J37do15L29r02ePLmlMZXJRRddNOhGGBExYcKEWLRoUbz77rsDP6hmdLkO2s+1MHq2bdsWS5cujV/84hdxzjnnxDXXXBMR+n0r7C/3Efp8K3ziE5+IhQsXxuLFi6O3tzcOO+ywuPXWW/X9Fthf7iP0fca+jikE9/5w95133hny3ttvvx1TpkzxA9422Psw2507d7Y5knJwHeTLtXBwfve738XFF18czz33XFxwwQXxrW99a2CkQ78fXcPlfjj6/OiYMGFCnHrqqbF58+b49Kc/HRH6fqv8Ye7fe++9/bbT9xkrOqYQnDJlSsyaNSteeOGFIe+9+OKLcdxxx7UhqnJ466234owzzoi77rpryHsbNmyIiIhZs2a1OqxSch20l2thdOzYsSO+9rWvxZo1a+KSSy6JW265ZVAhot+PnpFyr8+PnvXr18dpp50Wvb29Q97r6+uLrq6uGDdunL4/Cork/v3339f3GfM6phCMiJg/f36sXr061q9fP/DaM888Exs2bBh2lT8OzvTp02Pbtm2xcuXK2LFjx8Drb7zxRjz66KNx0kknFfodA2m4DtrHtTA6brnlllizZk1cfPHFccMNN+yzjX4/OkbKvT4/eo444ojYvn17PPzww7F79+6B1zdt2hQ//OEP48QTT4zJkyfr+6OgSO5nzpyp7zPmdTWKrD6Riffeey/OPPPMqFarsXTp0ti1a1csX748Dj/88Hj44Ydj3Lhx7Q5xzPrxj38cV111VRx77LFx3nnnRV9fX/T29saHH34YDz30kIetjpLTTjstZs6cGStWrBh4zXXQOvvKv2shrfXr18fChQtjypQpccMNN+xz4YVFixbp96OgaO71+dHz+OOPx3XXXRcnnHBCfPnLX44tW7YM5PbBBx+M2bNn6/ujpEju9X3Guo4qBCMiXnnllbj11lvjl7/8ZUyYMCFOOeWUuO666wbmazN6fvzjH8c999wTa9eujQkTJkRPT09ce+21boSjaF+FSITroFX2l3/XQjoPPfRQ/PVf//WwbdatWxcR+n1qzeRenx89Tz75ZCxfvjx++9vfxqRJk+Jzn/tcXHPNNXHUUUcNtNH3R0eR3Ov7jGUdVwgCAABwcDrqN4IAAAAcPIUgAABAySgEAQAASkYhCAAAUDIKQQAAgJJRCAIAAJSMQhAAAKBkFIIAAAAloxAEAAAomf8fcpttBfj//P0AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_150_1.png" } }, "output_type": "display_data" } ], "source": [ "fig = plt.figure(figsize=(20, 14))\n", "im = plt.imshow(J, **cmap_args)\n", "plt.title(\"OLS\", fontsize=18)\n", "plt.xticks(fontsize=18)\n", "plt.yticks(fontsize=18)\n", "cb = fig.colorbar(im)\n", "cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is interesting to note that OLS\n", "considers both $J_{j, j + 1} = -0.5$ and $J_{j, j - 1} = -0.5$ as\n", "valid matrix elements for $J$.\n", "In our discussion below on hyperparameters and Ridge and Lasso regression we will see that\n", "this problem can be removed, partly and only with Lasso regression. \n", "\n", "In this case our matrix inversion was actually possible. The obvious question now is what is the mathematics behind the SVD?\n", "\n", "\n", "\n", "\n", "\n", "Let us now \n", "focus on Ridge and Lasso regression as well. We repeat some of the\n", "basic parts of the Ising model and the setup of the training and test\n", "data. The one-dimensional Ising model with nearest neighbor\n", "interaction, no external field and a constant coupling constant $J$ is\n", "given by" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " H = -J \\sum_{k}^L s_k s_{k + 1},\n", "\\label{_auto7} \\tag{7}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $s_i \\in \\{-1, 1\\}$ and $s_{N + 1} = s_1$. The number of spins in the system is determined by $L$. For the one-dimensional system there is no phase transition.\n", "\n", "We will look at a system of $L = 40$ spins with a coupling constant of $J = 1$. To get enough training data we will generate 10000 states with their respective energies." ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from mpl_toolkits.axes_grid1 import make_axes_locatable\n", "import seaborn as sns\n", "import scipy.linalg as scl\n", "from sklearn.model_selection import train_test_split\n", "import sklearn.linear_model as skl\n", "import tqdm\n", "sns.set(color_codes=True)\n", "cmap_args=dict(vmin=-1., vmax=1., cmap='seismic')\n", "\n", "L = 40\n", "n = int(1e4)\n", "\n", "spins = np.random.choice([-1, 1], size=(n, L))\n", "J = 1.0\n", "\n", "energies = np.zeros(n)\n", "\n", "for i in range(n):\n", " energies[i] = - J * np.dot(spins[i], np.roll(spins[i], 1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A more general form for the one-dimensional Ising model is" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " H = - \\sum_j^L \\sum_k^L s_j s_k J_{jk}.\n", "\\label{_auto8} \\tag{8}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we allow for interactions beyond the nearest neighbors and a more\n", "adaptive coupling matrix. This latter expression can be formulated as\n", "a matrix-product on the form" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " H = X J,\n", "\\label{_auto9} \\tag{9}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $X_{jk} = s_j s_k$ and $J$ is the matrix consisting of the\n", "elements $-J_{jk}$. This form of writing the energy fits perfectly\n", "with the form utilized in linear regression, viz." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " \\boldsymbol{y} = \\boldsymbol{X}\\boldsymbol{\\beta} + \\boldsymbol{\\epsilon}.\n", "\\label{_auto10} \\tag{10}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We organize the data as we did above" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "X = np.zeros((n, L ** 2))\n", "for i in range(n):\n", " X[i] = np.outer(spins[i], spins[i]).ravel()\n", "y = energies\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.96)\n", "\n", "X_train_own = np.concatenate(\n", " (np.ones(len(X_train))[:, np.newaxis], X_train),\n", " axis=1\n", ")\n", "\n", "X_test_own = np.concatenate(\n", " (np.ones(len(X_test))[:, np.newaxis], X_test),\n", " axis=1\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will do all fitting with **Scikit-Learn**," ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "clf = skl.LinearRegression().fit(X_train, y_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When extracting the $J$-matrix we make sure to remove the intercept" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "J_sk = clf.coef_.reshape(L, L)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And then we plot the results" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ ":7: UserWarning: FixedFormatter should only be used together with FixedLocator\n", " cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAM2CAYAAACjUj0CAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAB7LklEQVR4nO3de3xU1b3//3cmcRjiOMZpICGkIQSMCJTbUbDUqsUICip4FLygqHhpe7CtWEvRH60txwqoRyrar9ripSDeOHKqFr/aWusd4etBi4jhHiGEBGMIQwghJpnfH0Ag5rI3zCfJDvN6Ph48fDj7k898smftPbOy1qyVEI1GowIAAAAAxA1fexcAAAAAAGhbdAQBAAAAIM7QEQQAAACAOENHEAAAAADiDB1BAAAAAIgzdAQBAAAAIM7QEQTQaqZPn65TTjlFhYWFruK8asmSJTrllFMa/evXr5+++93v6uabb9ann37a3mW2ieXLl+uUU07RkiVL2vR5t23bpmuuuUYDBgzQsGHDVFZW1qbP79ann36qn/70p/re976n/v3768wzz9TUqVOPqn0UFhbqlFNO0UMPPVT/2CmnnKLp06e3+HMjRozQNddc0+Cxr776SpWVlY7P6fVrEQBgJ6m9CwCAyy+/XN/97nfbuwxHl19+uf7t3/6t/v+rq6u1du1aPffcc/p//+//6aWXXlJWVlY7Vtj6evXqpXvvvVdDhgxp0+edM2eOPvroI91yyy3q0qWLwuFwmz6/G++8845+9KMfqXfv3po0aZLC4bC2b9+uF198Ua+99poeeugh5eXluc4XDod17733HnHH7M4771Tnzp3r///tt9/W7bffrv/5n/9RcnLyEeUCABy76AgCaHeDBw/W4MGD27sMR4MGDdLYsWMbPT5kyBBNnTpVTzzxhH7zm9+0fWFtKDU1tclz0NrWrl2rU089VVOmTGnz53brd7/7nfr06aPnn39exx13XP3jkyZN0tixYzVz5kydc845Skpy99abnJx8VOf6m53NVatWKRKJHHEeAMCxjamhABCjCy64QJ07d9a//vWv9i7lmPX111/r+OOPb+8ymlVWVqaCggINGzasQSdQklJSUjRu3DiVlpY6TpMGAKCt0BEE0O6++b2k6dOn6/zzz9eqVat09dVXa+DAgRo+fLjuvvtuVVVVNfjZ4uJiTZs2TWeccYa+853vaNy4cXr55ZcbPceyZct04403atiwYerXr5++//3v69e//nWDkZKDz7to0SKdfvrpOv300/XOO+841p+QkKBOnTopGo02eHzDhg2aMmWKTjvtNA0cOFBXXHGF3n333UY//69//UuTJk3S4MGD9f3vf18PPfSQHn744SbPSVO1uTkH0WhUDz/8sEaNGqXvfOc7Gj58uH7xi19o+/btDeKeffZZXXTRRRo4cKCGDRumKVOmaP369fXHm/qOYG1trebPn69Ro0bVfy/urrvuavA9voM/9/777+u3v/2tvvvd72rgwIG69tprlZ+f3+y5Pfhz27Zt04oVK+q/M3fw8f/5n//RRRddpO985zu64447jrieDz74QP/f//f/6fTTT9e//du/6Y477lBlZaXefvttjR07VgMHDtTYsWO1bNmyZmuUpM6dOysxMVH/+Mc/9OWXXzY6/pOf/ESfffaZsrOzG7wmCxYs0IUXXqgBAwZoxIgRuv/++7V3715JTX9H8Js2btyoYcOG6fzzz1dpaamkht8RnD59uh5++GFJ0rnnntvou4NutMU1diTXPADABlNDAXhSWVmZbrjhBl1wwQW6+OKL9c4772jhwoXy+/2aNm2aJKmkpETjx49XNBrVNddcoxNPPFH/+Mc/9Itf/EI7duzQjTfeKEl67733dNNNN2nIkCH66U9/qoSEBL3//vt6/vnntWvXLj344IP1z7t9+3Y98sgjuuWWW7Rjxw4NGjRIb7zxRou1fvrppyovL9eIESPqH1u7dq2uuuoqpaam6oc//KGOO+44/fWvf9XNN9+s//qv/9Lo0aMlSatXr9akSZOUmpqqKVOmaO/evVqwYIF8vsZ/p2uqNrfn4NFHH9Uf/vAHTZw4sX4BnwULFmj16tX661//qsTERL388sv6zW9+o3Hjxumaa65RWVmZ/vznP+uaa67R3//+d51wwglN/v5Tp07V66+/rpEjR2rSpEnavHmznn32WX344YdavHixQqFQfeyMGTPUtWtX/cd//Id27dql+fPn66abbtI///nPJqdMHvxO4qxZs3TSSSfpRz/6kU455RTt2rVLkjRz5kz9+7//u8aPH6+MjIwjrmf69Onq3bu3fv7zn2vFihVasmSJiouLtWbNGl1zzTU64YQT9Mc//lE/+9nP9MYbbzT42cN17txZo0eP1iuvvKK8vDyNGDFCZ555ps444wx17969yd/tt7/9rZ599ln94Ac/0JVXXqnNmzfriSeeUEFBQX3nrSVFRUWaPHmygsGg/vznPys1NbVRzOWXX66Kigr9/e9/1x133KGTTz7ZMe/h2uoae/XVV11d8wAAQ1EAaCW//OUvo7m5udGtW7e6ivvm/y9YsKBB3AUXXBA988wzG8QNHTo0WlJSUv9YXV1d9Lbbbov2798/WlpaGo1Go9Ebbrgh+oMf/CC6b9++BvkmTJgQHTx4cKPnXbp0aYO4F198MZqbmxtduHBh9Kuvvqr/V1RUFP373/8ezcvLiw4YMCC6adOm+p+5+uqro3l5edE9e/bUP/b1119Hr7rqqujw4cPra5k0aVL09NNPj3711Vf1cZ999lm0T58+TZ6Tb9bm9hxccMEF0ZtvvrnBzz777LPRiy++OPrFF19Eo9Fo9MYbb4yOGTOmQcxbb70VHT16dPSjjz6KRqPR6IcffhjNzc2Nvvjii9FoNBp9++23o7m5udG77767wc+9+uqr0dzc3OicOXMa/Nyll14arampqY977LHHorm5udH33nsv2pIf/OAH0auvvrr+/w/mu+GGGxrEHU09tbW10Wg0Gq2trY1+73vfi+bm5kbffvvt+p994YUXXNVYUVERvfXWW6O5ubkN/o0ZMyb69NNP1z9PNBqNrl+/PnrKKadEZ8yY0SDHAw88EM3NzY2uX78+unXr1mhubm503rx59cdzc3Ojv/zlL6NfffVVdOTIkdGzzjqr0fX1zXM1b948V9dhNNr0tdgW15jbax4AYIepoQA864ILLmjw/3369Kmf/lZXV6c33nhDp512mpKSklRWVqaysjLt3LlTI0eOVHV1td5//31J0mOPPaYXX3xRfr+/PtfOnTsVDAabXFL/tNNOa7Ke//zP/9R3v/vd+n/nnHOOfvKTn6hr16567rnn1LNnz/rcK1as0Nlnn62qqqr62iKRiM477zyVlpbq008/1a5du7RixQpdfPHFDVbB7Nu3r773ve81WcPhtR3JOUhPT9fy5cv15z//uf4cXnHFFQ1WOk1PT9emTZv08MMP13+X7eyzz9bSpUsbrJZ6uDfffFOS9MMf/rDB4xdccIF69uypf/zjHw0eHzlypBITE+v//9RTT5WkJqdTunH66afHVM+5555bP/rq8/n07W9/W4FAQGeddVZ9TGZmpqsajz/+eM2dO1evvvqqfvKTn2jw4MFKSkrS+vXrNXPmTP3Hf/yHamtrJUlvvfVW/Sjb4W644Qa9/PLLLa4+W1FRoRtvvFHbtm3TU089VV+ftfa4xlq65gEAtpgaCsCzvrlFgN/vV11dnaT9HzJ3796tN954o9mpmwe//5aYmKitW7fqwQcf1IYNG7RlyxaVlJQ0+7zf+ta3mnz8hhtu0Jlnnqm6ujrl5+frj3/8o3r06KH77ruvflqiJG3dulWStHDhQi1cuLDZ2jp16qS6ujr16NGj0fGcnJwmv094eG1Hcg6mTZumH//4x7rnnns0a9Ys9evXTyNGjNCECRPUpUsXSdKUKVP0ySef6KGHHtJDDz2k3r17a8SIERo/fnyzHZPCwkKFQqEmpyX26tWr0Xcsm3pNJdW/rkfqm/mOtJ5vxiUlJTXKebCj6LbGXr166ZZbbtEtt9yi3bt367XXXtODDz6of/7zn3r99dc1evRobdu2TZIafGdQkkKhULPTTw/6+9//Lp/Pp7q6Oq1evbr+DxBuVVdX10+tPSgQCDSa+tse11hL1zwAwBYdQQCe1dT35A46OLIyatQoXXHFFU3GfPvb35YkPf7447r33nvVs2dPnXbaaRo5cqQGDhyohQsX6pVXXmn0c4ePWB2ud+/eGj58uCTpzDPP1LBhw3TVVVfpmmuu0eLFi+s/xB6sbeLEic3uG9e7d28VFRVJUoNRlIM6derU5M8dXtuRnIM+ffro9ddf17vvvqt//vOfevfddzVv3jw9+eSTev7559WrVy+lp6frpZde0vLly/WPf/xD7777rv74xz/qySef1BNPPKGhQ4c2yh/9xgI5h6urq2u0gmZLr+nR+OZrdaT1NPVaJyQkHHEdb731lt5//3394he/aPB6nnDCCRo/frxyc3M1YcIE/e///q9Gjx5d/9odjRNOOEGPPvqo7rzzTs2ePVtnnXWWTjzxRNc///HHH2vSpEkNHrvkkks0e/bsBo+1xzVm3T4AAM2jIwigQwqHw+rcubNqamrqO2cHFRUVac2aNercubP27dunhx56SMOGDdMTTzzRYNGOwxewOBrf+c539POf/1yzZs3SjBkz9H/+z/+RJHXv3l3S/g+736xtw4YNKiwsVOfOnes/RBcUFDTK/cUXXzg+v9tzUFtbq/z8fAWDQZ177rk699xzJUmvvvqqpk6dqsWLF2v69Olau3atJNVPfZWk//3f/9W1116rhQsXNtkR7N69u9577z2VlpY2Gl3bvHmzunXr5vh7WGqvej777DMtWLBA5513XpPn6eAiLYFAQJLqR5C3bt2qXr161ceVlJRo1qxZuvrqq5Went7kc+Xl5em0007TjBkzdNNNN+m//uu/NHPmTNe19unTR08++WSDx7p27doozgvXGACg9fCnNwAdUlJSks466yy9/fbbjbYfmD17tqZMmaKdO3eqqqpKe/fuVXZ2doMPqJ9//rlWrFghSaqpqTnqOq699loNGTJE//jHP/Tqq69K2v+hun///vqf//mfBtPjvv76a91555366U9/qpqaGn3rW9/S4MGD9de//rXBVL2tW7e62rbC7Tmora3VpEmTdM899zSIGThwoKRDozA/+9nPNG3atAajVX379tVxxx3X7EjNwZVSH3vssQaPv/HGG9q8ebPOOeccx9/DUnvVM2bMGPl8Ps2ZM6fJzdtfeOEFSarvhJ999tmS9m/XcbglS5bo//7f/6tgMOj4nGeddZbOO+88vfDCC1q5cmWzcQdfu4OjpSeeeKKGDx/e4F/v3r0b/ZxXrjEAQOtgRBBAq5s7d26Tm4FfcMEF9SNPR+P222/X8uXLNXHiRE2cOFEZGRl666239M9//lOXX355/SjMwIEDtWTJEgWDQfXs2VPr16/X4sWL6z8g79mz54im1h0uISFBM2fO1CWXXKLf/e53+t73vqcTTzxRM2bM0LXXXqtLL71UV155pVJSUrR06VL961//0s9//nOddNJJkqRf/vKXuuaaa3TZZZfpiiuuUHV1tRYuXOj6e1Fuz8E111yjRx55RFOmTNH3v/99VVVV6fnnn1fnzp116aWXStr/HcgZM2bouuuu0/nnn69oNKqXXnpJ+/bt01VXXdXk85999tk699xztWDBApWUlGjYsGEqKCjQs88+q29/+9uNFm1pbe1VT3Z2tu644w7dc8899dsf5OTkqKqqSu+//77++c9/6pprrtGQIUMk7V8kZ/z48Vq4cKF27Nih7373u9qwYYOee+45jRs3Tn369HG1+fydd96p9957T3fddZeWLFnSaOqrdOh7d/Pnz9dZZ51V3xl1wwvXGACgddARBNDq/vrXvzb5eE5OTkwdwaysLL3wwguaN2+eXnjhBVVWVurb3/627rjjjgarMT744IOaNWuWXnzxRVVXV6t79+66+eab1atXL/3kJz/Rhx9+qFGjRh11HSeffLJuuOEGPfroo5o9e7ZmzZqlwYMH69lnn9VDDz2kJ598UjU1NerZs6dmz56tSy65pP5nBw8erPnz52vu3Ln6/e9/r5SUFF1zzTXauHGjXn/9dbNz8NOf/lQpKSl68cUXNWfOHCUmJmrIkCG677776qcmjh8/Xscdd5wWLFigBx54QHV1derfv7/+9Kc/adiwYU0+f0JCgh588EH96U9/0l/+8he9+eab+ta3vqXLL79cP/nJTxwXPrHWnvVMmjRJffv21aJFi+r3xQsEAurTp48eeOABjRkzpkH8zJkzlZ2drcWLF+vNN99URkaGpkyZUr83nxsZGRn68Y9/rAceeEBPPvmkbr755kYxY8aM0d/+9jctWbJEK1asOKKOoFeuMQCAvYRoS9+sBwC0qqa+yyZJP/rRj5Sfn6+33nqr7YsCAADHPL4jCADtaPz48brhhhsaPFZaWqrly5drwIAB7VQVAAA41jE1FADa0cUXX6xHH31UP//5zzVs2DBFIhG98MILqqur05QpU9q7PAAAcIxiaigAtKO6ujotWrRIL7zwgrZu3apOnTppyJAh+tnPfqY+ffq0d3kAAOAYRUcQAAAAAI5RkUikya2N6AgCAAAAQIz2lpWp84Ete7ykqqpKZ511VoM9iyUPdwQ3b5Za2n/25JOl9eud8/z3f9vVdMcFn9gl69bNLNW2mjSzXE38saCRU0+VPv/cOc5ylfbuod1muXbrBLNcJ+zcYpbL1cnv319avdox7PPE/gYFHXJqr2qzXJ9v9Jvlys42S6X33nOOOe886e9/d46zrOvk5G1mueq6dTfL5Vv1iVmuvacMcozp3Fnau9c5V+dCF28MblnexDp1ssu1caNZqrrB/+YY4/NJbra13LPHoKADamvtcm3ebJcrM9Mu17e+5Rzj9tz7ttvdJySZfkZRWZlZqvKkxissH62Ujf/bcsC//Zv0vw4xBw0eHHtBB7nYP9StvV2yzHItWhR7jmBQuuKK2PN43RNnnqmI4esYq1Bmpia/9562b9+u2m/cXD27WExNjfT11y3HOB2XpJ07beqRJFXbfRi2fJezfMN0c07dxlnWJcO/V5j+5aOlv1YcKbfty0Xc1569st23MTcs/4zlppPhNs7yd7S9kAwZ3g/dvo6u4ixPvuW5d/Np3q19++xyGbK8Hi1Pl+Vbt2Vdprx6n5BMT5rp+XdzHbXHtWb4ucLymnTzt2rsFyks1K4vvmjvMhrp1sQfeNg+AgAAAADijHlHcOvWrbrllls0dOhQDR06VNOmTVOZ4bQAAAAAAPAinwf/Ncd0AtnOnTt17bXXqrq6WjfeeKNqa2v1+OOPa+3atVq8eLH8frvvBgEAAAAAjo5pR/Cpp55ScXGxXnnlFfXq1UuSNHDgQF1//fX6y1/+ogkTJlg+HQAAAADgKJhODV26dKmGDh1a3wmUpOHDh6tnz55aunSp5VMBAAAAgKe09zTQI5kaatYR3LVrl7Zu3ap+/fo1OtavXz999tlnVk8FAAAAAIiBWUewpKREkpSW1nhPuy5dumj37t3avdtuLzgAAAAAwNEx21D+448/1hVXXKG7775b48ePb3Bs7ty5evTRR/XOO+802VEEAAAAgI7u/2Rne2ofwRN79NB/FBQ0ecxssRg3/cmEhATX+davb3lf4L59pTVrnPM89ZTrp3R072Ur7JJlZpql2lKTYZarvNw5ZsAAadUq57iUlFirOSQrxW4n04hCZrlCpZvMcrk6+UOGSCtXOoatShoSez2HGdDHbkfmVfl2qwf37m2WSm+84Rxz8cXSyy87x1nW1Te4xSxXXWaWWS7fR3b3w8r+Qx1jkpOlykrnXMkFLt4Y3LK8iQUCdrny881S1Z0x3DHG53O3kXdFhUFBBxjuqa316+1yZWfb5erSxTnG7bn3FdrdJySZfkZRaalZqrKkrma5wvkftBwwfLj0gUPMQWecEXtBBzXzgf1oVKbnmOV69NHYc4RC0o03xp4HdsymhiYnJ0uS9u3b1+jYwceCwaDV0wEAAAAAjpLZiGBGxv5RqS+//LLRsR07digUCtV3FgEAAADgWJMg420ZYtTSfEyzOkOhkDIzM5tcHXTNmjXq37+/1VMBAAAAAGJg2mEdOXKkli1bpo0bN9Y/9sEHH2jz5s0aPXq05VMBAAAAAI6S2dRQSbrpppv00ksv6brrrtPkyZO1b98+zZ8/X/369dPYsWMtnwoAAAAAPMVpE/e21iYbyktSOBzW008/rT59+mjevHn685//rLy8PM2fP19+v91KgQAAAACAo2e2j6C1X/xC+uqr5o8/8YQ0ebJznifmu1h32aVp0+36zTNmmKVSYaFdrr41LvaFcLl/RGXvAQYV7We4mrL6lrtcDtoFN0uvu+WrcbFFg98vVbuIs1x7XVJ1kjcXerJs+6mpzjGhkBRxsZPJ6tWx13PQ8NPstu5wtUWJS+vK7ZZxz00tcw4Kh6Uy57jqYNigov38nxhuGWS5t8KgQWapXn7P+Xy53Tbl4tOKDCo6ID3dLtd775mlWt7pLLNcw053sy+Eu/0jSr60HX9I22e3HUV1ut22NZb3/Jxyh62YXG7XJElF6XZbNmWk231uXbXarl0M6O1i/x4nCQlS586x5/G4x7KzFfHQPoKhHj30w9beRxAAAAAA4lncTg0FAAAAAHgfHUEAAAAAiDNMDQUAAAAAA0wNBQAAAAB4Fh1BAAAAAIgzTA0FAAAAAAMJ8tZIW0ILx7xUJwAAAACgDdARBAAAAIA4w9RQAAAAADDAqqEAAAAAAM9iRBAAAAAADCSo5QVa2hqLxQAAAAAA6tERBAAAAIA4w9RQAAAAADCQeOCfV7RUCyOCAAAAABBn6AgCAAAAQJxhaigAAAAAGEiQt0baWlo1NCEajUbbrJIjsG2bVFvb/PGsLGnLFuc8Dz9sV9O9s+vMck2bbtdErr7aLJX693eO8fmkOhenwldeFntBB5WWmqUqCuaa5cpIrTbL5YrfL1U7P2dRqd/0aTNKV5nl+qBigFmu4VVvmuVSaqpzzIAB0irnc7Fkg93v+O+nubjRuVSXmWWWq6LCLJVCG1Y6Bw0ZIq10EefmdWwHnj33SZXOQcnJUqVz3PJPkw0q2u+kk8xSqXdvu1y+Khfny60kF3+Ld3nP14YNsddzODe1ubSywu49181nFLf8pUUtB2RkSEUOMQcFArEXdJDhBV6dbnffqamJPUdCgtS5c+x5vG5RdrYqvviivcuoF+zRQxMLCpo85qUOKwAAAACgDTA1FAAAAAAM+OStkbaWavFSnQAAAACANkBHEAAAAADiDFNDAQAAAMAAU0MBAAAAAJ5FRxAAAAAA4gxTQwEAAADAAFNDAQAAAACeRUcQAAAAAOIMU0MBAAAAwECCvDXSltDCMS/VCQAAAABoA3QEAQAAACDOMDUUAAAAAAywaigAAAAAwLPoCAIAAABAnGFqKAAAAAAYSFDLK3W2NVYNBQAAAADU8+yIYG2tVFPTcozTcUmaPt2mHkn68RS7fvMjf6gzy/V/HrWra0BmmXNQOCxfuXPcO6vDBhXtd1Z/s1TK+L+LzHKtO32iWa7c0g+cg4YPlz76yDEsY9Cg2As6XE2KWapMu1SSeptleqcgyzHmLEnvlA9wjBs3LvZ6Dqqscq7LtSq7VIGAXa5NKUMcY3LcxqVWGlS0X6WSzXIlV9nVFQza1VXn4nf0SaoLOMcNG1xtUNF+mwr9ZrlMWTb8KhcXpN/v6sPOGvU1KOiQvqkuPgu4NCRlh1muSFVXs1z+1FTnIDcxkqpl1179waBZrp07zVIpbde62JMkJUk5ObHngRnPdgQBAAAAoCNJPPDPK1qqhamhAAAAABBn6AgCAAAAQJxhaigAAAAAGEiQt0baWDUUAAAAAFCPjiAAAAAAxBmmhgIAAACAAZ+8NdLWUi1eqhMAAAAA0AboCAIAAABAnGFqKAAAAAAYYGooAAAAAMCz6AgCAAAAQJxhaigAAAAAGGBqKAAAAADAs+gIAgAAAECcYWooAAAAABhIkLdG2hJaOOalOgEAAAAAbYCOIAAAAADEGc9ODa2p2f/PKcZJVZVNPZL04IN2uX79G7s++MwZ1Wa5Lr4s7Bjz8svSxde5iLt/nUVJkqQtFblmuSoGTzTLVWPYvtS7t22cJTcXm0tZ5SvNcpVlDzHLdVb2FhdRWe7iatJjrqc+VY3fLFdFhVkqZaRUmuVKTU12GeciyPCmnxwwSyUVFpqleqvQ7n54xhnOMcnJ7k5rsuzuExUVdu3e0t/esHvvPuMM53YfkhSpcY7rG9hkUNEhlYEcs1zJNRGzXOXlZqkUKvyo5YDhw6WPHGIOSDpjuEFF+5WV27X9r74yS6V9wdjvO4mJUneDWryOVUMBAAAAAJ5FRxAAAAAA4oxnp4YCAAAAQEeSoJZX6mxrrBoKAAAAAKhHRxAAAAAA4gxTQwEAAADAQOKBf17RUi2MCAIAAABAnKEjCAAAAABxhqmhAAAAAGAgQd4aaWPVUAAAAABAPTqCAAAAABBnmBoKAAAAAAZ88tZIW0u1eKlOAAAAAEAboCMIAAAAAHGGqaEAAAAAYICpoQAAAAAAz6IjCAAAAABxhqmhAAAAAGCgI00NPeY7ghmlq+ySpfc3SzXz9gqzXJNuDJnlenn+DhdRXV3F3TwjN/aCDpg92yyVstKr7ZJV2L2Om8q7OsbkdJU2VTjHbfjEoKDDjOxjd6soyx5ilstSJCXLMSbkNq7czXXkUsD59XarpsYslRQImKUqWO0cM2CAVFDgIi7d8Jc0/B3LUu3uh8fvNEulTz91jhk2zF1ct27JsRd0QGamWSqVl9vlSk+3y+Xm7SMUchmXmhp7QYdJDtTZJSutssslu887ZX2Gt3g87CKmPvbDDwwqOpCrTx+7XL2DZrlKdvpjzuHzUu8oDm3fvl21tbUNHjvmO4IAAAAAEM8mTpyobdu2NXiMjiAAAAAAGEiQt6aGJhz476JFixgRBAAAAIB40q1bt0aPeanDCgAAAABoA+Yjgpdddpk+beJb5aNGjdK8efOsnw4AAAAAPCFuVw2NRqPauHGj8vLyNHLkyAbHunfvbvlUAAAAAICjZNoRLCwsVGVlpc4991yNHTvWMjUAAAAAwIhpR3DDhg2SpF69elmmBQAAAADPS9ChlTq9oKVaTKewrl+/XtKhjmBlZaVlegAAAACAAfOO4PHHH69Zs2Zp8ODBGjx4sPLy8rR06VLLpwEAAAAAxCAhGo1GrZJdcsklWrNmjUaNGqWLLrpIkUhECxYsUH5+vubMmaNx48ZZPRUAAAAAeMrK7Gzt++KL9i6jXqcePTSkoKDJY6YdwWeffVZ1dXWaOHFi/WNVVVW68MILtXfvXr3zzjtKTEx0lWvTJqmmpvnjubnSunXOeXKrVrl6Plf697fLVVFhlmrSLSGzXAvu3+Ec1LWrtMM57uYZXQ0q2m/2bLNUCger7ZIZvo6bysOOMTk5+68NJwe+rmtmZJ8tZrnKgllmuSwlufjGdCgkRSIu4qpcXEcuRQJ211F5uVkqZWXWmeVatdp5csqAAdIqF7fzAel2516pqWapysrtJuAc+BZGmxk2TFq+3Dmuib2Kj1owaJfLUmGhXS43zSsjQyoqchEXdHFjOhKWL0BpqVmqLVV290OnXzEclsrK3OUK538Qe0EH9eljl8vwdSzZ6Y85h88ndeliUIzHdaSOoOnU0CuvvLJBJ1CSAoGAxo4dq9LS0vrFZAAAAAAA7cd8Q/mmhMP7RzpYPAYAAADAsSpB3tpQvk1WDS0pKdGYMWP08MMPNzq2efNmSVJmZqbV0wEAAAAAjpJZRzAtLU2RSESLFy9WxWHfmyoqKtKSJUs0bNgwdYmHicEAAAAA4HGmU0PvuusuTZkyRVdccYXGjx+vPXv2aNGiRUpKStJdd91l+VQAAAAA4Ck+eWtqaEu1mNaZl5enP/zhD+rcubPuv/9+Pfnkkxo0aJCeffbZ+k3mAQAAAADty3T7CFOlpVJdC8uTu9zCoDJot9RwcqGL/SpcWr4z1yzXsG+7WFvapdvuz3CMeeAB6bbbnHM9cLfd4kC3zUg2yzVjhlkqVVXZ5cooX+Mc1LevtMY5blOgr0FFh3j1671utnxwy1fuYp1wt+uJf/RR7AUdUNR/pFmujAq7e5jS081S1QWdt8Dx+Vp+S2gNvkK7bVMst5qp62N3fbe0TdNBfr9U7WLXHX+N3T2/LmB3zzc89QrVuNxPwIW6FOctg9y2e8utYazzWb5/WG7fkbP65ZYDLr5Yetkh5qAzzoi9oIMsG2xxsV2u3r1jz+HzmW7L41X/ys5WtYe2j/D36KGBzWwf0SarhgIAAADAsS5up4YCAAAAALyPjiAAAAAAxBmmhgIAAACAAaaGAgAAAAA8i44gAAAAAMQZpoYCAAAAgIEEeWukLaGFY16qEwAAAADQBugIAgAAAECcYWooAAAAABhg1VAAAAAAgGfREQQAAACAOMPUUAAAAAAwkKCWV+psa6waCgAAAACoR0cQAAAAAOIMU0MBAAAAwECipLr2LuIwiS0cY0QQAAAAAOIMHUEAAAAAiDMJ0Wg02t5FNKW6uuXjfr9zjCSVltrUI0kZpavskvXubZZqS2myWa5OnZxj0tKkkhLnuAcfjL2eg+6+2y7Xrbfa5Tr/fLtc/fs7x2RlSVu2uIjLtJ2UUFZu9zejcKDSLJeqquxyBQLOMcnJUqVz/WsK7K7J7GyzVEpOcnHTdGlLsd8sV3q6c4zbe77/kxWxF3RQnz52uYJBs1QlX9pdj2+95Rxz+eXS88+7iLvErn1ZsmyrWSkRs1xrCkOOMX37SmvWOOfq28d4IlpBgVmqkuNzzHKdcIJZKuf7odubjiQVF8de0AGRlCyzXBUVZqmUkWp0ffvtrkev2pidrZovvmjvMuol9eihXs1c04wIAgAAAECcoSMIAAAAAHGGVUMBAAAAwIBP3hppa6kWL9UJAAAAAGgDdAQBAAAAIM4wNRQAAAAADDA1FAAAAADgWYwIAgAAAIABRgQBAAAAAJ5FRxAAAAAA4gxTQwEAAADAQIK8NdKW0MIxL9UJAAAAAGgDdAQBAAAAIM4wNRQAAAAADPgkRdu7iMOwaigAAAAAoB4dQQAAAACIM0wNBQAAAAADCWp5pc62xqqhAAAAAIB6CdFo1EvfZ6xXUiLV1jZ/PCNDKipyzpNR8IFZTdWnDTfL5X/rb2a51mWPNMuVmekck5wsVVY6x+3eHXs9B/3Xf9nlmj3bLtcVV9jleu455xifT6qrcxFXUx17QYfZUuw3zWclK9PFyXCruNg5xuWNpy49w6Cg/QoLzVIpPd0ulz/J7txvKXT+m2RWlrRli3OuJMN5Lhnpdr9jWbnd313DVS7e/Nxy0yhc3ngqq+x+x+Ryu9+xSHbXY4YMz30g4BwTDktlZY5hm8rDBgUdkpNp9x7yzod27x9nnWl3TUYqWm6voZAUibjLFfrQ7jOd+ve3y1VebpertDT2HJ06ScOGxZ7H47ZnZ6v2iy/au4x6iT16qFtBQZPHmBoKAAAAAAYS27uAb2ipHqaGAgAAAECcoSMIAAAAAHGGqaEAAAAAYCBB3hppY9VQAAAAAEA9OoIAAAAAEGeYGgoAAAAABrw2ytZSPV6rFQAAAADQyugIAgAAAECcYWooAAAAABjw2igbU0MBAAAAAPXoCAIAAABAnGFqKAAAAAAY8NooG1NDAQAAAAD16AgCAAAAQJxhaigAAAAAGEiQt0baElo45qU6AQAAAABtgI4gAAAAAMQZz04NTUsslRLqWojoqoykHc6J+vc3q8mvarNcq9JHmuVSlV2q5KoyF0FhV3FVx4UNKtrv0kvNUmncOLtcL/+lpTZ6ZO6Z7fx3mTvvlGbPds51661+g4oOycy0y+XLX2OXrDTVLFUkmOEYE3IZV1Meez0HBYN2uUpL7XJl5L9ll6z3CLNUGQEX9zDXUswyhQOVZrlUY/jWXV7uHBMOu4qrSbK751s21ozeKWa5VFhhlqpIzveSDElFVc7nNSdpi0FFh3ntE7NUg8652CyX8vPNUgV693WOCbjLtS7b7jNdbrnhe2SfPna5LK5JX3yMP/kkRdu7iMMwNRQAAAAAUI+OIAAAAADEGc9ODQUAAACAjoSpoQAAAAAAz6IjCAAAAABxhqmhAAAAAGCAqaEAAAAAAM+iIwgAAAAAcYapoQAAAABgIEEtT8f0EkYEAQAAACDO0BEEAAAAgDjD1FAAAAAAMNCRRtk6Uq0AAAAAAAN0BAEAAAAgzjA1FAAAAAAMdKRRto5UKwAAAADAAB1BAAAAAIgzTA0FAAAAAAMdaZTNux3BpCSprs45xkF1IGRUkKunc21A1Qq7ZKedZpaqqDjsGJMhqajKRVzFOoOK9jvppFyzXC//d7VZrpl3+81y/XqGQ3uXJPl053TnuHtm296G7ry10ixXZXZfs1zJVWVmuUIBN+3C7yquWnbtorTULJUy0t20MXdKEkeY5dI+u1SmCgrMUpWl5JjlCqYkm+XyV0XcBbp4AwzV2F2PW1IGmOUKVpmlUjgQMMuVkerunuMqriY15noayMszSxWqcdnG3Ei1+z2rHNqF3+8c0ypSUsxSFRXbfRbIqCiPPUlSkulriNh5tyMIAAAAAIjZ9u3bVVtb2+AxOoIAAAAAYCAhIUFKSGjvMg45UMvEiRO1bdu2BofoCAIAAADAMWzRokWMCAIAAABAPOnWrVujx46qI/irX/1KBQUFWrhwYYPHt27dqjlz5mjFiv0LoZxzzjmaPn26wmHnhUUAAAAAoENLSrJdYTJWLdRyxFUuXrxYL7zwgoYOHdrg8Z07d+raa69VdXW1brzxRtXW1urxxx/X2rVrtXjxYvn9dqvoAQAAAACOnuuOYG1trR555BE9/PDDTR5/6qmnVFxcrFdeeUW9evWSJA0cOFDXX3+9/vKXv2jChAk2FQMAAAAAYuKqI7hv3z6NHz9ea9eu1bhx47Rs2bJGMUuXLtXQoUPrO4GSNHz4cPXs2VNLly6lIwgAAADg2JaY6K2poYmJzR5ytdPkvn37VFFRoblz52rOnDlK+sYvt2vXLm3dulX9+vVr9LP9+vXTZ599doQVAwAAAABai6vuajAY1N/+9rdGHcCDSkpKJElpaWmNjnXp0kW7d+/W7t27dcIJJ8RQKgAAAADAgquOoM/nk8/X/ODhnj17JEmdO3dudKxTp06SpMrKyiPrCKakOMe4WI3Us0vUfGOxHa/IyLCMy42llFbKJFm2il//2iyVXA7QSy1ciwfdeWeMpTSS7MFMkpLbYUViFwtfWd533F6T7rhsYy408Xe/VpeV5SbKsE0Yrnjt2bWz/SF3cSGXcUayvHrCwq4aoS03i+3Fy4J8hu3QTSa3T2d7edjd9E3fPow/iR3TjuVVQ5sSjUYdYxIO7GrvWnm5VFfX/PFwWCorc0xTHbR7N7F8TX0frbBLdtppZqmKip0/KGZkSEVFzrkyKtYZVLTfOstOZXa1Wa6Zsw07lTNaaO8H+XwtXxcH3DPb7gO/JN15a6VZrkrLTmWV8z3AtWDQOcbvl6qd20+1YVewtNQslTLSXbQxl0q+tGtj+/Y5x2RlSVu2uIgLGraJ8nKzVGUpOWa53DRVt/xVEeegUEiKuIirqYm9oAO2VNi9d1uer3CFi0boVnq6c4zLe47luTdnWVtVlVmqSKBri8fdNntJKi42KOiA3KCLD1guFVl2Ki0+0yUlSTl290LEzuSdPDl5/we7fU28mx98LGh5JwYAAAAAHDWTMa6MA3OXvvzyy0bHduzYoVAoVN9ZBAAAAIBj0rG2aqiTUCikzMzMJlcHXbNmjfr372/xNAAAAAAAA2Zf8hg5cqSWLVumjRs31j/2wQcfaPPmzRo9erTV0wAAAAAAYmQ2bnnTTTfppZde0nXXXafJkydr3759mj9/vvr166exY8daPQ0AAAAAeFMHWjXUbEQwHA7r6aefVp8+fTRv3jz9+c9/Vl5enubPny9/vCxrDAAAAAAdwFF1V998880mH8/JydGf/vSnmAoCAAAAALQuD41bNrS6MKXFrXOGhKWVBc77DA3pb7dn3Jp8u5HN3oPsNpT3l9vtm7V6tfM5zciQVq92zlXV227vv97ZZqmkKrs9jW65xa5NuNn77847XcZNt9svTpIm32i36u/tt5ulUnq64a7TFc4h4bBUVuH8moeDdvedjFSzVJLhdl5pJflmuSp7D3AVl+rmXJTb7TNmKVyzwyxXtVre/6y9VAbsrsdCu+alzEy7XGHDvSUjKc6b04f8UqTK+Z4Tqio3qOgwllt+FRaaparr09csV8jxs1NYoRp3n68qDPesrk612/uvyu7Uq+TE2D/T+XxSF4NaPC/eVg0FAAAAAHQcdAQBAAAAIM54aNwSAAAAADqweFw1FAAAAADQMdARBAAAAIA446FxSwAAAADowFg1FAAAAADgVXQEAQAAACDOeGjcEgAAAAA6MFYNBQAAAAB4FR1BAAAAAIgzHhq3BAAAAIAOjFVDAQAAAABeRUcQAAAAAOKMh8YtAQAAAKADY9VQAAAAAIBX0REEAAAAgDjjoXFLAAAAAOjAOtCqoQnRaDTahqW4t2WLVFPT/PGcHGnTJsc0ZSk5ZiWFy52fr120dJ6OVGamc0xyslRZ6RhWqWSDgvbbvdsslXbtssuVm11tl8zN6+jy3E++xe7cS9IT8+vMct05w24iwj2/MTz/bvj9UrWL56yosHvO0lK7XMGgWapIMMMsV6hqh3NQ167SDhdxgUDsBbWCLeUhs1ydOpmlUtpJLtqz23b/0UexF3RASa/hZrnSaovMclm+31anZznGuD31+fkGBR1mQB+7e2tljd8sV3KS4T2/qqrl46GQFIm4SrWqwO76HpBq115Xldrdp/v3t8nji4e5iGPGSNu3t3cVh3TrJi1d2uSheHg5AAAAAACH8dC4JQAAAAB0YKwaCgAAAADwKjqCAAAAABBnPDRuCQAAAAAdWAdaNZQRQQAAAACIM3QEAQAAACDOeGjcEgAAAAA6MFYNBQAAAAB4FR1BAAAAAIgzHhq3BAAAAIAOjFVDAQAAAABeRUcQAAAAAOKMh8YtAQAAAKADY9VQAAAAAIBX0REEAAAAgDjjoXFLAAAAAOjAWDUUAAAAAOBVHuquNrT7pCxFo80fD0mKpOY45gkHqs1q2lTu/HxuZWaapZL/kxVmuSLpuY4xIUmRmmTnuKRKg4r2Sz6uyixXbTBslksbNpilqszu6xiTLKlSzuf+9tsNCjrMnTPs/mZ0z911ZrkuHuc3y/Xyf9vdKxQMejNXQYFZqlDFOrNc6t3bXVxqqnNMRUVstRxmVUHILNeA4CazXHWZdu9F1TXO15BfUrVcxBm21bSTDK/HDeVmqSKZzvdptwJmmaQB6TsMs0lKcnGtuZQsu9eyrMLunh+uKG45IBSSystd5RqQHXM5h9jdwtS/v12uwsLYcyQmSt27x54HdjzbEQQAAACADoVVQwEAAAAAXuWh7ioAAAAAdGAsFgMAAAAA8Co6ggAAAAAQZzw0bgkAAAAAHRiLxQAAAAAAvIqOIAAAAADEGQ+NWwIAAABAB8aqoQAAAAAAr6IjCAAAAABxxkPjlgAAAADQgbFqKAAAAADAq+gIAgAAAECc8dC4JQAAAAB0YKwaCgAAAADwKjqCAAAAABBnPDRuCQAAAAAdWAdaNdRDVTZ0Qqdqhwi/QgGnGGldgd+mIEm5VavMcqki0yxVZf+hZrn27naOCYWkvXud44JdkmMv6ABfTY1ZroxgxCyXklLNUiVXlbkICruKS08PG1R0yD2/cb7W3Lp4nN01+fJf6sxyLXjaua5Jk6QFzznHnXGGRUX7pdo1MYUt35hSUuxy/fWvzjEXX+wqLnLOxQYF7Wd5uqozc8xy+T9aYZZLg+zeP0xt2GCXKxAwS+Xmc4dbb77nfC8ZMUJ67z3nXCP6GxR0mJIv7SaMnXSS3T0/XLDSLFfdoCEtHvdJqsvMcpXL8COKkoIhs1y+CrvPO1mZQaNMTEb0El4NAAAAAIgznh0RBAAAAIAOhVVDAQAAAABeRUcQAAAAAOKMh8YtAQAAAKAD60CrhjIiCAAAAABxho4gAAAAAMQZD41bAgAAAEAHxqqhAAAAAACvoiMIAAAAAHHGQ+OWAAAAANCBsWooAAAAAMCr6AgCAAAAQJzx0LglAAAAAHRgrBoKAAAAAPAqOoIAAAAAEGc8NG4JAAAAAB0Yq4YCAAAAALwqIRqNRtu7iKb885/S3r3NHx89Wnr1Vec8Z5xhV1O4eI1ZrnVJfc1y5SZtMsu1JSnHMSYrS9qyxTlXVkrEoKIDLP+yUlpqlqoyNcssV3JStXOQ3y9VO8eVVfgNKjokHHRRWztY8Jzd7znp6jrnIJ9PqnOOu+12u7+xPTDb8NwbXkeRCrvfMVTq4h6WkyNtsrvXuVGX7Xw/bA++Crt76/LPQ44xw4ZJy5c75zr5ZIOCDvj6a7tcltI+e9MumZsPKMnJUmWlc1wgEHs9hysoMEsVSbW7jkI1ZWa5HO+HoZAUcXetrSpwvo7cGhBYZ5ZrVVWuWa7evWPPkZAgde4cex7Pe+ABqby8vas4JCVFuu02bd++XbW1tQ0OeWjcEgAAAAA6MI+uGjpx4kRt27atwSEPVQkAAAAAsLZo0SJGBAEAAAAgnnTr1q3RY0fVEfzVr36lgoICLVy4sMHjl112mT799NNG8aNGjdK8efOO5qkAAAAAoGPoQKuGHnGVixcv1gsvvKChQ4c2eDwajWrjxo3Ky8vTyJEjGxzr3r37kT4NAAAAAKCVuO4I1tbW6pFHHtHDDz/c5PHCwkJVVlbq3HPP1dixY80KBAAAAADYctUR3Ldvn8aPH6+1a9dq3LhxWrZsWaOYDRs2SJJ69eplWyEAAAAAdAQeXTW0Ka42gtq3b58qKio0d+5czZkzR0lN/HLr16+XdKgjWOlm3xsAAAAAQJtz1V0NBoP629/+1mQH8KD169fr+OOP16xZs/Tqq6+qsrJS3/72tzV16lSNGTPGrGAAAAAAQGwSotFo9Eh/aMSIEerevXuDVUMvueQSrVmzRqNGjdJFF12kSCSiBQsWKD8/X3PmzNG4ceMs6wYAAAAAb5k/X4pE2ruKQ0Ih6cYbmzxkNoF1woQJqqur08SJE+sfGzNmjC688ELdd999uuiii5TYwhzVb/rnP6W9e5s/Pnq09OqrznnOOMP1UzoKF68xy7Uuqa9ZrtykTWa5tiTlOMZkZUlbtjjnykoxvAgs51qXlpqlqkzNMsuVnFTtHOT3S9XOcWUVfoOKDgkHXdTWDhY8Z/d7Trq6zjnI55PqnONuu93VrHtXHphteO4Nr6NIhd3vGCp1cQ/LyZE22d3r3KjLdr4ftgdfhd29dfnnIceYYcOk5cudc518skFBB3z9tV0uS2mfvWmXzM0HlORkyc1XbQKB2Os5XEGBWapIqt11FKopM8vleD8MhVx/mF9V4HwduTUgsM4s16qqXLNcvXvHniMhQercOfY8sGP2Tn7llVc26ARKUiAQ0NixY1VaWlq/mAwAAAAAoH21+pI24XBYEovHAAAAADjGHWurhjopKSnRmDFjmtxjcPPmzZKkzMxMi6cCAAAAAMTIpCOYlpamSCSixYsXq6Kiov7xoqIiLVmyRMOGDVOXLl0sngoAAAAAECOzccu77rpLU6ZM0RVXXKHx48drz549WrRokZKSknTXXXdZPQ0AAAAAeFNSkremhrZQi9liMXl5efrDH/6gzp076/7779eTTz6pQYMG6dlnn63fZB4AAAAA0P6Oqrv65ptNL5+cl5envLy8mAoCAAAAALSuo9pQvi2UlEi1tc0fz8iQioqc8xhuGafVq+1yXXaZXS6/DPcZO+w7ns0Kh6UyF3v5fPJJzOUctCp1hFmuAf1d7BfnUsmXdnupnXSSc4zLbQRt24Tkrl24FQyapVpXYLeP4KOPOsc88IB0220u4u63a2OWexLefbdZKtNZL/4KF/cTl/edykDYoKL9LN8/stINr8maGrNUlUp2jHG7lV1yqYsNZt2yvOekptrlst6vz4nbvewM24Qk0/MfSbHbbzckuz00y2pa3vvP7UcdSQrLcH/Dqiq7XOXlZqlKvhX7/tc+nxQXS4Y884ztPSxWwaB01VVNHrL7hAEAAAAA6BDoCAIAAABAnPHQkjYAAAAA0IHF46qhAAAAAICOgY4gAAAAAMQZD41bAgAAAEAHlpjoramhiYnNHmJEEAAAAADiDB1BAAAAAIgzHhq3BAAAAIAOjFVDAQAAAABeRUcQAAAAAOKMh8YtAQAAAKADY9VQAAAAAIBX0REEAAAAgDjjoXFLAAAAAOjAWDUUAAAAAOBVdAQBAAAAIM54aNwSAAAAADqwDrRqaEI0Go22YSnulZZKdXXNH+/aVdqxwzFNXWpXs5J8FRGzXNWBkFmuqiqzVAoEnGP8fqm62jmuoiL2eg5yU5dblufL8jp3c74yMqSiIhdxqS5eoCNRUGCXKzvbLFVZhd8sVzjo4py5bPy3Tber64H7W7gPHqEJV9hNAnnqKbNUSi7e5ByUkyNtco4rS8kxqGi/sMrMclUGwma5XnvNLJX+/Uzn91G377ebKuzeby1vOSkpdrmGZLo4Xy6VRJ3PV1qaVFLinCuts93nE0lSTY1dLssXs08fs1Qlu5NbPO723FtLq3XxJu9SdWqGWS7/6pUGSfxS//6x5/G6v/9d2ru3vas4pHNn6bzzmjzE1FAAAAAAiDMeGrcEAAAAgA6MVUMBAAAAAF5FRxAAAAAA4oyHxi0BAAAAoAPrQKuGMiIIAAAAAHGGjiAAAAAAxBkPjVsCAAAAQAfGqqEAAAAAAK+iIwgAAAAAccZD45YAAAAA0IGxaigAAAAAwKvoCAIAAABAnPHQuCUAAAAAdGBJSVJtbXtXcQirhgIAAAAADqIjCAAAAABxhqmhAAAAAGCBVUMBAAAAAF6VEI1Go+1dRFN275ZaqiwUkiIR5zyhpEqzmuoCyWa5KirMUmnvXrtcu3Y5x+TmSuvWuYhL2hR7QQelp5ulqpTd67h7t1kqpXWpcw7y+aQ6F3E1NbEXdLjSUrtclo3f8i9u2dnOMS7Pf2WV3d/YrrvOLJVeeM5F23Hp+cV2v+PlF7i5mbu86X/0UewFHbAmfYRZrr4pRWa5ShIzzHKl7XFxn87JkTa5iAsGYy/ooJQUu1zl5WapypK6muUqLnaO6dtXWrPGOa5Pn9jrOZzlbToUqDbL9cFHfrNcw/s73E/c3nMkqaoq9oIOqAzatbHkGpf1u7CqIBRzjuOOk0491aAYr/vkE6nart3HzO+XBg1q8pCHxi0BAAAAoANLSnL3R/u2wqqhAAAAAICD6AgCAAAAQJxhaigAAAAAWGDVUAAAAACAV9ERBAAAAIA446FxSwAAAADowJKSWt4Dr62xaigAAAAA4CA6ggAAAAAQZ5gaCgAAAAAWEhO9NTWUVUMBAAAAAAfREQQAAACAOMPUUAAAAACw4KXN5CVWDQUAAAAAHOKxLisAAAAAdFAtLM7SLlgsBgAAAABwEB1BAAAAAIgzTA0FAAAAAAtJSVJCQntXcUgLU0MTolEv7Xh4mHfekaqqmj8+cqT0t785plmXPdKspBNPNEulk06yy+WvKLNLFgg4xyQnS5WVds/pwqbiZLNcOdl1Zrn01ltmqUr6jXCMSUuTSkqcc6WVrDKo6JBI9gCzXKHidWa5lJpqliqSFHaMCYWkSMQ5l5vLyK2aGrtcr7xil+vy8XbX0a9/4zw5ZeZM6de/ds41blzs9RxkufDbgN5298yicrv7YX6+c8yIEdKbb7qIS18Te0EHlHyrr1kuS2kJO+ySuWlg4bBU5uI9PhiMvZ7DrCvwm+XKDRaZ5SoLZJjlCsvhvLo995KKqpzfP9zKKLV7/67rb/feXVgYe47ERKl799jzeF5JiVRb295VHJKYuP8DZBOYGgoAAAAAcYapoQAAAABgITHRW1NDfc2P+zEiCAAAAABxho4gAAAAAMQZpoYCAAAAgIWkJKnOcGHCWDE1FAAAAABwEB1BAAAAAIgzTA0FAAAAAAuJiS1Ox2xzLaxg6qEqAQAAAABtgY4gAAAAAMQZpoYCAAAAgIWkJCkabe8qDmFqKAAAAADgIDqCAAAAABBnmBoKAAAAABYSE9u7AtcYEQQAAACAOENHEAAAAADiDFNDAQAAAMBCUsfpXjEiCAAAAABxJiEa9dJGF4eUl0t1dc0fD4elsjLnPOGqIrOaVFxslmpL6hCzXOnpZqlUWOgck5MjbdrkHGdZV3KghcZwhIqK7f7+UVFhlkqBgHNMVpa0ZYtzXGpq7PUcLrlih10yy+L++le7XP37O8e4bfwpKTGXU6+83C6X4bn/9f0hs1wzf+Pi+vb5Wn5TOOCe2XbX952XrTPLZdruq6rscgWDzjGhkBSJOIZVB+zahL+m0ixXye5ks1xpXezei/Thh84xw4dLH3zgGPZm1XCDgg4ZNMguV7jCxZuWS2XBLLNcTk3f75eqq93l8le4+EDqluFnzZJv9TXLZdb2fXEwBuXivarN+Xzavn27amtrGzzcccYuAQAAAMDD6jw44dInaeLEidq2bVuDx+kIAgAAAMAxbNGiRYwIAgAAAEA86datW6PHXHcE3333XT3yyCP67LPP5PP5NHDgQN16660adNhE8q1bt2rOnDlasWKFJOmcc87R9OnTFQ6HY68eAAAAADyspqa9K2jM72/6cVcdwRUrVuimm27SySefrKlTp6qmpkbPPPOMrr76aj3zzDMaMGCAdu7cqWuvvVbV1dW68cYbVVtbq8cff1xr167V4sWL5W+uAgAAAABAm3LVEbznnnvUrVs3vfDCC+rcubMkady4cRo9erTmzp2rJ598Uk899ZSKi4v1yiuvqFevXpKkgQMH6vrrr9df/vIXTZgwofV+CwAAAACAa47L2uzatUv5+fk6//zz6zuBkpSamqrTTz9dH3/8sSRp6dKlGjp0aH0nUJKGDx+unj17aunSpa1QOgAAAAB4R23t/umhXvn3jfVhGnAcEQwGg3rttdcadAIP2rlzpxITE7Vr1y5t3bpVo0aNahTTr18/vf3220d2BgEAAAAArcZxRDAxMVHZ2dlKS0tr8Hh+fr5WrlypwYMHq6SkRJIaxUhSly5dtHv3bu3evduoZAAAAABALBKi0Wj0SH9oz549uuqqq7R27VotWLBAxx13nK644grdfffdGj9+fIPYuXPn6tFHH9U777zTZEcRAAAAAI4Fu3dLR967aj0JCdIJJzR97Ij3Edy7d69+/OMfKz8/Xz/84Q81dOhQrVy50kURCUf0POXlUl1d88fDYamszDlPuKroiJ63RcXFZqm2pA4xy5WebpZKhYXOMTk50qZNznGWdSUHWmgMR6io2HEg3LWKCrNUCgScY7KypC1bnONSU2Ov53DJFTvsklkW99e/2uXq3985xm3jT0mJuZx65eV2uQzP/a/vD5nlmvkbF9e3z9fym8IB98y2u77vvGydWS7Tdl9VZZcrGHSOCYWkSMQxrDpg1yb8NZVmuUp2J5vlSuti916kDz90jhk+XPrgA8ewN6uGGxR0yGE7g8UsXOHiTculsmCWWS6npu/3S9XV7nL5K1x8IHXL8LNmybf6muUya/s+u3s0YndEr0YkEtHkyZO1fPlyXXrppZo6daokKTl5/0123759jX7m4GNBN282AAAAAIBW53pE8KuvvtINN9ygzz//XJdffrl++9vf1o/yZWRkSJK+/PLLRj+3Y8cOhUKh+s4iAAAAAByLamtdTWBpMy0NwrrqCFZUVNR3Aq+77jrdcccdDY6HQiFlZmbqs88+a/Sza9asUX83U64AAAAAAG3C1dTQmTNn6vPPP9ekSZMadQIPGjlypJYtW6aNGzfWP/bBBx9o8+bNGj16tE21AAAAAICYOY4Ibty4US+99JJCoZBOPfVUvfTSS41ixo4dq5tuukkvvfSSrrvuOk2ePFn79u3T/Pnz1a9fP40dO7ZVigcAAAAAr6ipOYamhq5YsULS/oVimhsNHDt2rMLhsJ5++mnNmjVL8+bNUyAQUF5enqZNmya/3390lQMAAAAAzDl2BK+88kpdeeWVrpLl5OToT3/6U8xFAQAAAABaz1FtKN8WnDZjdLmtkUIBl5vAuGG5aZzldhqWe0q5ydW1q7TDxb5ylr9jfr5ZqpLudns4piUY7q+X5GLtJrcbaFq2Ccn2tTQUkd2+ZaFSF/sDutxHsDI9x6Ci/SxfyvAnb5rlWpkywizXa685x9x5p3TPPS7iptvNx/nprXb7Xf3+92ap1MQC3Uct7as1zkF9+0prXMQZbh5bVBU2y5Xx8VKzXDr1VLNUbu4TyclSpYstFZMLXLw+RyIz0yzVumK7+3Ru0HBvaKebq9t9YyXb90jLXIZ7Eppc3wkJUufOsefxuJKS/SuHekViopSW1vQxdnUEAAAAgDhDRxAAAAAA4ozrDeUBAAAAAM2rqfHW1NCWvmrHiCAAAAAAxBk6ggAAAAAQZ5gaCgAAAAAGamv3Tw/tCBgRBAAAAIA4Q0cQAAAAAOIMU0MBAAAAwEBNDVNDAQAAAAAeRUcQAAAAAOIMU0MBAAAAwIDXVg1NSGj+GCOCAAAAABBn6AgCAAAAQJxhaigAAAAAGPDaqqFMDQUAAAAA1KMjCAAAAABxhqmhAAAAAGDAa6uG+loY9kuIRqPRtivFvS+/lOrqmj+eliaVlDjnSdu3xa4oQ5GULLNcIUXMctUFQ44xPl/Lr019XHGRQUUHJNn9zaIutatZLl9NtVkuFRY6x+TkSJs22T2nW6mpZqlWFTi3MbcMm4X69HGOcdv23byUbmUFy8xyrSkOm+WyfJMbEFjnHJSbK61zjvvpw7kGFe037/cuXmyXbv6R3QSc6dPNUik72znGbbsvL4+1mkOCQbtcfhnepy1vOlVVzjHJyVJlpXPchg2x13M4N7W55ebm6tI7n9i9f5x1msN5dXvuJdvGb5irMruvWa7kKoP3Ip9PSkmJPY/HrV4tVRvedmLl90v9+zd9jKmhAAAAABBnmBoKAAAAAAa8tmpoS1NDGREEAAAAgDhDRxAAAAAA4gxTQwEAAADAgNdWDU1MbP4YI4IAAAAAEGfoCAIAAABAnGFqKAAAAAAY8NqqoUwNBQAAAADUoyMIAAAAAHGGqaEAAAAAYMBrq4YmtdDbY0QQAAAAAOIMHUEAAAAAiDNMDQUAAAAAA15bNbSlWhgRBAAAAIA4Q0cQAAAAAOIMU0MBAAAAwIDXVg2trW3+GCOCAAAAABBnPDsiGAhI0WjLMZ07u0j0r3yTeiSpZOBIs1xpNWVmuVRaapaqMBByjMnKkgoLnXNl1VQZVHRAZqZZqvJys1QKp9hdQmUpOc7P5zauZodBRYdsKXduF24NCG4yy1Wd6Xwu2kNWerVZrsqasFmuvilFZrmUkmKXqyrVXVyqc9zvfx9bKYe7+Ud2fyv946N1Zrn+/TK7uh5+2DkmI0MqLnYRl2rX7i1v1CXRrma59uwxSyUp2TEiJ0faVOwiLhi0KKjeO+UDzHKdlVRplis72yyVuyEbl8M6kWBGjMUcEki1y5VcvMUsl8kQV1KS7XsHYubZjiAAAAAAdCSsGgoAAAAA8Cw6ggAAAAAQZ5gaCgAAAAAGWDUUAAAAAOBZdAQBAAAAIM4wNRQAAAAADLBqKAAAAADAs+gIAgAAAECcYWooAAAAABhg1VAAAAAAgGfREQQAAACAOMPUUAAAAAAwwKqhAAAAAADPoiMIAAAAAHGGqaEAAAAAYIBVQwEAAAAAnkVHEAAAAADijGenhp7wxWqpurr5gCFDFNqw0jnRaaeZ1dTZ8GzVBcNmud76xC7X8cc7x2RlSdu3O8elficn9oIOKC02S6WswA67ZOV2jSLosk0Eg84x1eoaYzUNddppl6su065d+D9aYZZLffo4x4RC8lVEnOOS7NrFa6/5zXJ973sZZrlqy81SKUMuk1VVOYZ8+WVstRxu+nS7XP9+md3fXZf8d51Zrp/e6lzXvHnS7NnOuWbPtmuryampZrk6V5ilUtoJlWa5IjXJruJcnYpAZmzFfEP/FLtcbn9PN9LTzVJJFS7m7rmc3xdMia2Uw/mKi8xyrSrPMss1INvFe5+ThITYc3QArBoKAAAAAPAsOoIAAAAAEGc8OzUUAAAAADoSVg0FAAAAAHgWHUEAAAAAiDNMDQUAAAAAA6waCgAAAADwLDqCAAAAABBnmBoKAAAAAAZYNRQAAAAA4FmMCAIAAACAARaLAQAAAAB4Fh1BAAAAAIgzTA0FAAAAAAMsFgMAAAAA8Cw6ggAAAAAQZ5gaCgAAAAAGWDUUAAAAAOBZCdFoNNreRTSlurrl436/c4wkvfeeTT2SNOKMSrNckZpks1yBgFkqrV7tHDNkiLRypYu4QXWxF3TAM8/Z/c3iqivs6jL9k09VlXNMKCRFInbP6ZZhI6uW3yyXpY8/do4ZNkxavtw57jvfib2eg5Irdtglq6gwS/VmQY5ZrhGnuWjTbtt+YWHsBR1Q16evWa7iYrNUmj3bLte837u4H/p8Up1z3POL7e7TF1xglsr0Nh0M2uXyV3mz3UuS0tPNUpUpbJYrXLHFLJfjixkOS2Vl7nIZNrKVhV3Ncll+Puxbsyr2JMcdJ516aux5PO7BB6Vdu9q7ikNOPFH62c+k7du3q/YbK8cwNRQAAAAADHh11dCJEydq27ZtDY7REQQAAACAY9iiRYsYEQQAAACAeNKtW7dGj7nuCL777rt65JFH9Nlnn8nn82ngwIG69dZbNWjQoPqYyy67TJ9++mmjnx01apTmzZt3dFUDAAAAQAfQkVYNddURXLFihW666SadfPLJmjp1qmpqavTMM8/o6quv1jPPPKMBAwYoGo1q48aNysvL08iRIxv8fPfu3WP6BQAAAAAAdlx1BO+55x5169ZNL7zwgjp37ixJGjdunEaPHq25c+fqySefVGFhoSorK3Xuuedq7NixrVo0AAAAAODoOXYEd+3apfz8fF1//fX1nUBJSk1N1emnn673339fkrRhwwZJUq9evVqpVAAAAADwLq+uGtoUx45gMBjUa6+91qATeNDOnTuVmJgoSVq/fr2kQx3ByspKJSfb7ZUHAAAAALDhuPtrYmKisrOzlZaW1uDx/Px8rVy5UoMHD5a0vyN4/PHHa9asWRo8eLAGDx6svLw8LV26tHUqBwAAAAAclYRoNBo90h/as2ePrrrqKq1du1YLFizQ0KFDdckll2jNmjUaNWqULrroIkUiES1YsED5+fmaM2eOxo0b1wrlAwAAAIA3/Od/Sjt3tncVh5x0kvSrXzV97Ij3Edy7d69+/OMfKz8/Xz/84Q81dOhQSdKECRNUV1eniRMn1seOGTNGF154oe677z5ddNFF9dNI3aiubvm43+8cI0nvvef6KR2NOKPSLFekxm7abCBglkqrVzvHDBkirVzpIm5QXewFHfDMc46D165ddYVdXaaTwKuqnGNCISkSsXtOtwwbWbX8Zrksffyxc8ywYdLy5c5x3/lO7PUclFyxwy5ZRYVZqjcLcsxyjTjNRZt22/YLC2Mv6IC6Pn3NchUXm6XS7Nl2ueb93sX90OeT6pzjnl9sd5++4AKzVKa36WDQLpe/ypvtXpKUnm6Wqkxhs1zhii1muRxfzHBYKitzl8uwka0s7GqWy/LzYd+aVbEnOe446dRTY88DM0d0145EIpo8ebKWL1+uSy+9VFOnTq0/duWVVzboBEpSIBDQ2LFjVVpaWr+YDAAAAACgfbkeEfzqq690ww036PPPP9fll1+u3/72t0pISHD8uXB4/1+CKivtRtMAAAAAwGs60qqhrkYEKyoq6juB1113nWbOnNmgE1hSUqIxY8bo4YcfbvSzmzdvliRlZmYeYdkAAAAAgNbgqiM4c+ZMff7555o0aZLuuOOORsfT0tIUiUS0ePFiVRz2HZSioiItWbJEw4YNU5cuXeyqBgAAAAAcNcepoRs3btRLL72kUCikU089VS+99FKjmLFjx+quu+7SlClTdMUVV2j8+PHas2ePFi1apKSkJN11112tUjwAAAAAeEVNjbemhrZUi2NHcMWKFZL2LxTT1GigtL8jmJeXpz/84Q967LHHdP/99ysQCGjo0KG67bbb6jeZBwAAAAC0P8eO4JVXXqkrr7zSVbK8vDzl5eXFXBQAAAAAoPUc8T6CbSXJRWVuYkYMcrkHjAsr8+32whnSx3AVVcPh5/R0d/sbutpiyHBc/Krz7fY/q66xex0rKuz2xAvL5e/o4rxWBux+R0lK/ugDs1x+y424DJ188gCXcc4xyaV2e11tqskyy5VjeOpHpK8xy1UdcN6vzy+pOhByjjPc/6y83CyVMlJdbHzr0uzZdvcdN3v/XX65y7jxdnu03jPbbk/CO6/YZJar5Gu7/TPTjnP5HunmvbRPn9iK+YZ1G+zOv+Utf12V3f0wt8ahXYTDtjcBl77+2m4fwSG97fYdfvMjd++RLQkEpOEGtXjdMbdqKAAAAADg2EFHEAAAAADijGenhgIAAABAR9KRVg1lRBAAAAAA4gwdQQAAAACIM0wNBQAAAAADrBoKAAAAAPAsOoIAAAAAEGeYGgoAAAAABlg1FAAAAADgWXQEAQAAACDOMDUUAAAAAAywaigAAAAAwLPoCAIAAABAnGFqKAAAAAAYYNVQAAAAAIBn0REEAAAAgDjD1FAAAAAAMNCRVg099juCSXa/YiBglkrrCpPNclVVmaVSZqa7ODfnoi7JH1sxh/EZXlH+8h1mucKlpWa5tgT7OsZkhaUtFWHHuMJ8i4oO6dVruFmutJOqzXJpwwazVF9/bRhXURFTLYcrKDZLpZwzU8xylUS7muVKq6l0DvIny+8irqjK+fpwKzXVLJVUXm6WKtmwsAsucDcx6IILnGPumW03yejO6XVmuYafmWOW64O/2L1/KMnlhwoXn2Mqq2wneGVn2+XyV0XskhXb3fM1aJBzjMsTUVRsd/6Hddtilqtkb5ZZLjeny4mPeYiew0sCAAAAAHHm2B8RBAAAAIA2wKqhAAAAAADPoiMIAAAAAHGGqaEAAAAAYKAjrRrKiCAAAAAAxBk6ggAAAAAQZ5gaCgAAAAAGWDUUAAAAAOBZdAQBAAAAIM4wNRQAAAAADLBqKAAAAADAs+gIAgAAAECcYWooAAAAABhg1VAAAAAAgGfREQQAAACAOMPUUAAAAAAwwKqhAAAAAADP8uyIYCQi1dU1fzwclsrLnfOEP/nIrKa+55xjlsuU4Z8dSnb6XcV9/bVzjJvXx62vo13Ncu3ZY5ZKOZkBs1xBly9jMOgck5kZWy3flFZbZJdsQ7ldroDd+TeVmmqWKqXKLJXtRSm7a7Jkd7JjTFqyu7iMj5dalLTfeeeZpSoxvId1rjBL5frtw03cnVdsiq2Ywww/M8cs1wfvtfBh4gg9v9judbz8kmp3gS7uc+WlMRbzDRnpducsopBZrqQ+Q8xyyeHempwsVVa5Gy+xvLVmZKaY5UqrKTPLVVQVjjlHYqJBITDl2Y4gAAAAAHQkrBoKAAAAAPAsOoIAAAAAEGeYGgoAAAAABlg1FAAAAADgWXQEAQAAACDOMDUUAAAAAAywaigAAAAAwLPoCAIAAABAnGFqKAAAAAAYYNVQAAAAAIBn0REEAAAAgDjD1FAAAAAAMMCqoQAAAAAAz6IjCAAAAABxhqmhAAAAAGCAVUMBAAAAAJ5FRxAAAAAA4oxnp4amJFc7RPgVDjrFSGvSR9gUJKlvjfPzuVZVZZbqg9Uhs1yBgHNMWpq0bZtz3JDMHbEXdFCCXSqlGDb70nKzVOEkF7nCWQpXbHEOK3eR60ikpJilimT2NcsVCthdk2nvvekiaITSPnMRd9ppsRd0gOV1VJbU1SxXWo3h9Z2a6iLIp7Qudc5hp54aczn1kuzuFXv2mKVS2gmVZrmqA8mu4oJB55iSr3NirOaQD/5i176eX2zX7i8f76INujRtut8x5t57pWkznONuucWiokMiFXbjBBUVZqmUUbXJLFd1pnN7dXsL6JtSFGM1hz9pil0uNxeuSxmyavvH/hgUq4YCAAAAADyLjiAAAAAAxBnPTg0FAAAAgI6EVUMBAAAAAJ5FRxAAAAAA4gxTQwEAAADAAKuGAgAAAAA8i44gAAAAAMQZpoYCAAAAgAFWDQUAAAAAeBYdQQAAAACIM0wNBQAAAAADrBoKAAAAAPAsOoIAAAAAEGeYGgoAAAAABlg1FAAAAADgWXQEAQAAACDOMDUUAAAAAAx0pFVDE6LRaLTtSnGvurrl436/c4w1f02lWa66QLJZri+/NEvlSlqaVFLiHHfSSXbPWVxslysrsMMsV1FNV7NcGakuGrTLhh+p8htUdEggYJrOzHvv2eUacYaL6zs5Wap0EWf4DlCyN2SW66uvzFKpb3qZXbL8fOeY4cOlDz5wDKscNNygoP2SZXfP31Rsd89PTTVLpZAiLoJCUsRFnOUnnyTDv1Mb3sCmzbC7t947u845yOeT6pzjJl1nO8Hr4YftcoWq7N5zy5Ls3nOdmpjbZi/ZfkbJzrbL5S9YZ5Yrkp4bc46EBOmEEwyK8biePaUvvmjvKg7p0UPavFnavn27ar/xhUFGBAEAAADgGDZx4kRt27atwWN0BAEAAADAQDRaJy/Nt9xfi0+LFi1iRBAAAAAA4km3bt0aPeZ6UvmyZct05ZVXavDgwfr+97+v3/3ud9qzZ0+DmK1bt+qWW27R0KFDNXToUE2bNk1lZYbfIwEAAAAAxMzViOCyZcs0efJk9evXT7fffru2b9+uBQsWaPXq1Vq0aJF8Pp927typa6+9VtXV1brxxhtVW1urxx9/XGvXrtXixYvl99suXgEAAAAA3tLCDu7tpumxP1cdwfvuu0/dunXT008/rcCB1be6deummTNn6t1339XZZ5+tp556SsXFxXrllVfUq1cvSdLAgQN1/fXX6y9/+YsmTJhg9IsAAAAAAGLhODV03759OumkkzRhwoT6TqAkDR06VJK0du1aSdLSpUs1dOjQ+k6gJA0fPlw9e/bU0qVLresGAAAAAI+p0/5RQa/8a34LGscRwU6dOunxxx9v9Pjnn38uScrIyNCuXbu0detWjRo1qlFcv3799Pbbbzs9DQAAAACgjRzxqqHbtm3T8uXLNWfOHOXm5uq8887TFwd2TUxLS2sU36VLF+3evVu7d+/WCfGwiyQAAAAAeNwRdQTLy8s1YsQISVLnzp01Y8YMderUqX710M6dOzf6mU6dOkmSKisr6QgCAAAAOIbVSvLQRoJKaPbIEXUEExISNHfuXFVXV2vhwoW6/vrrNXfuXHXp0sXVzx4JN4uMtvlCpP5ks1Su9+1woYmB2FbX1s+ZlWWZratZpgyzTJLkskG7aPihOFmk98DfpYy4vL6T7e4DbqSFDHOZXrdhu1TDh5vF2b46dtlycsxSGXPZwEKGDbEDu/dey2wuPwn4nOMWLIixlNYUsnvPNbzruOK22Xv28sjNNUvl1V8RsTmijuCJJ56o0aNHS5LOP/98XXjhhZo1a5YeffRRSfsXlvmmg48Fg8EjKqy6uuXjfr9zjDV/TaVZrrqA3QeML780S+VKWppUUuIcd9JJds9ZXGyXKyuwwyxXUY1hpzLVRYN22fAjVbY9wcPWifKU996zyzXiDBfXd3KyVOkirqYm9oIOKNlr9/b71VdmqdQ33XCP2Px855jhw6UPPnAMqxzkslPpQrLs7vmbiu3u+ampZqkUUsRFUEiKuIgzbPdKOuJvrjTP8AY2bYbdvfXe2c0v4FDP55PqnOMmXWf552Xp4YftcoWq7N5zy5Ls3nOdmpjbZi/ZfkbJzrbL5S9YZ5Yrkh57pzIhQWJyoLcc9Z0jEAjonHPO0fbt29W16/4L88smeiQ7duxQKBRSchv/FR0AAAAA2ladB/81zbEjuHHjRo0YMUKLFi1qdGzPnj1KSEiQ3+9XZmamPvvss0Yxa9asUf/+/Z2eBgAAAADQRhw7gj169NDu3bv13HPPqfqwKWnbtm3T66+/rtNPP13BYFAjR47UsmXLtHHjxvqYDz74QJs3b66fTgoAAAAAaH+Ok/CTkpI0Y8YMTZs2Tddcc40uvvhi7dy5U4sWLZLP59OvfvUrSdJNN92kl156Sdddd50mT56sffv2af78+erXr5/Gjh3b6r8IAAAAALSvlqdjtr3mx/1cfRt77NixOu644zR//nzNmjVLycnJOuOMMzR16lT17NlTkhQOh/X0009r1qxZmjdvngKBgPLy8jRt2jT523x5TwAAAABAc1wvyzV69GjHKZ45OTn605/+FHNRAAAAAIDWY7g+MwAAAADEs1p5a2po85vbJ0Sj0eaPtqctW1rekygnR9q0yTFNUcBuF9+MKufnc2tNlV1dfYNbzHIpJcU5xuXmOhHD7Uct97ErLLTLlRMoskvm5ty73ceuoiLmcg63qthu76YB6XZ7Splyszmbyz29VF4eczn1DPdTqwvaXZO+GruNXN98z/nrAyNGSG++6ZxrRPoag4oOsNwX7wj30m1RZqZdrg0bnGP69pXWuDivffrEXs8BlVV2++JZXo6WTWLGDOeYBQukSZNcxD1l+6Fz2nS783/ddWapTPfYSy53eP/OyJCKXL7HW96nU+3eb32Fhp8PLS6k446TTj019jwel529U1984Z2OYI8ePhUUNL25t+0OpAAAAAAAz2NqKAAAAACY6DhTQxkRBAAAAIA4Q0cQAAAAAOIMU0MBAAAAwESd9k8P9T5GBAEAAAAgztARBAAAAIA4w9RQAAAAADBRJ2+tGprQ7BFGBAEAAAAgztARBAAAAIA4w9RQAAAAADBRK1YNBQAAAAB4Eh1BAAAAAIgzTA0FAAAAABNe21CeVUMBAAAAAAfQEQQAAACAOMPUUAAAAAAw4bVVQ5kaCgAAAAA4ICEajUbbu4gm1dW1fNznc46RpOJim3okqarKLlcwaJYqEuhqlsuNUEiKRJzjSkvtnjMnaYtdstRUu1wffmiX64wznGOSk6XKSrvndCvJcPKAYa6SL+3+lpW2Z5NzUE6OtMlFnOX5srxXJIXNclneWt1ckuGwVFbmIi7Jxc3Jrfx8s1TvVA01y9W/v1kqheXmpLo7+etK7dpXdrZZKvmTXHxWcClS0bZ/P3f7fnv33bbPe+9su3P24yl25+yR37lor2598knLx0eMkN5801WqskEjYq/nAMvPTrkpO8xyrSuP/bNmUtL+t9FjXXZ2gb74oqa9y6jXo0eSCgqymzzG1FAAAAAAMMHUUAAAAACAR9ERBAAAAIA4w9RQAAAAADAR1f5N5b2i+eVgGBEEAAAAgDhDRxAAAAAA4gxTQwEAAADAhNdWDW1+3I8RQQAAAACIM3QEAQAAACDOMDUUAAAAAEwwNRQAAAAA4FF0BAEAAAAgzjA1FAAAAABM1MlbU0MTmz3CiCAAAAAAxBk6ggAAAAAQZ5gaCgAAAAAmvLZqaPO1MCIIAAAAAHGGjiAAAAAAxJmEaDQabe8imlRaKtXVNX+8a1dpxw7HNNUpXc1K+vhjs1QadtI6u2Tp6Xa5amqcY8JhqazMMSySFDYoaL+KCrNU2rDBLtegQXa5Qoq4CApJERdxhYWxF3SYyuy+ZrmSk6rNclXLb5arqso5xu3ptxTasNIuWf/+drlKS+1yubnvZGVJW7Y4hq2ryjIoaL/cdMMXO8numxiRmmSzXIa3fFfXkFsZQbtzH1HILJfle1FGkvNnGLefddaU2n3WkaSHHrLL9cgfWvgsd4Tm3Gc3fnHJJS0fz82V1rn8qJaZGXs9ByUH7M7XlkK785VV8E7sSTp1koYNiz2Px2Vnf6wvvrD7rBOrHj38KigY3OQxRgQBAAAAIM7QEQQAAACAOMOqoQAAAABgwmsbyjc/3ZgRQQAAAACIM3QEAQAAACDOMDUUAAAAAEywoTwAAAAAwKPoCAIAAABAnGFqKAAAAACYYNVQAAAAAIBH0REEAAAAgDjD1FAAAAAAMMGqoQAAAAAAj6IjCAAAAABxhqmhAAAAAGCiTi2t1Nn2WDUUAAAAAHAAHUEAAAAAiDNMDQUAAAAAEx1nQ/mEaDQabcNKXCsvl+pamF4bDktlZc55SkvNSlJmpl2u8nK7XDU1drmSXPxpICNDKipyjrOsKyXFLlcoUG2X7JNP7HJlZzvHdO0q7djhGFaX2jX2eg7jq7E7Z2UVfrNc4YKVZrlcnX+3Nx43F5Jbhrk++CTZLFefPmapXHF76sNVLm5OLr2zIcMsl5vm5VZ6ul0uf/EW56CsLGmLc9y6qiyDivbLrbC7tiv7DDHLlVy8ySxXWUqOY4zbdh8IGBR0mOQqF0/q0pw/hc1y/fIXdt+7uurqlifFPfOMdNVV7nLNnm1Q0AH5+Xa5Rp5m9zqa8PlsP9B5VHb22/rii73tXUa9Hj06q6Dg7CaPMTUUAAAAAOIMU0MBAAAAwAQbygMAAAAAPIqOIAAAAADEGaaGAgAAAIAJpoYCAAAAADyKjiAAAAAAxBmmhgIAAACAiaha2sS97TW/ZTwjggAAAAAQZ+gIAgAAAECcYWooAAAAAJhg1VAAAAAAgEfREQQAAACAOMPUUAAAAAAwwdRQAAAAAIBH0REEAAAAgDjD1FAAAAAAMFEnb00NbX5zezqCAAAAAHAM2759u2prG3ZQE6LRaLSd6mnZ6tVSdXXzx4cMkVaubLt6JCk93S5XIGCXa8MGs1RlvYc6xoTDUlmZc67wGy8YVLRf3WUTzHKVl5ulUjjYQhs9QpEqv2NMKCRFIs65QjUuXqAjkWT4NyPDF6AuM8ssl6/CzYl19wKU1YQMKtrv66/NUimts4vf0a2aGrNU1cGwY4zf3/JbQn1c4SaDig6wvOcbni/TXG64velb3lyzs81SVVbZfQvG8lZYVeUc4/qeX1EUe0GHy883S7Uuc4RZrt/8xiyVnnm6+VESSZLPJ9U5xBww5DS7Nrbyv+3uYZHUHLNcFrcdn09KSYk9j9dlZ7+sL77Y095l1OvR43gVFFysESNGaNu2bQ2OMSIIAAAAACa8uWrookWLGo0I0hEEAAAAgGNYt27dGj3GqqEAAAAAEGdcjwguW7ZM8+bNU35+voLBoM4//3zdeuutOv744+tjLrvsMn366aeNfnbUqFGaN2+eTcUAAAAA4El1ammlzrYX46qhy5Yt0+TJk9WvXz/dfvvt2r59uxYsWKDVq1dr0aJF8vl8ikaj2rhxo/Ly8jRy5MgGP9+9e/fY6gcAAAAAmHHVEbzvvvvUrVs3Pf300wocWO2yW7dumjlzpt59912dffbZKiwsVGVlpc4991yNHTu2VYsGAAAAABw9x+8I7tu3TyeddJImTJhQ3wmUpKFD928zsHbtWknShgNbGPTq1as16gQAAAAAjzu4obxX/sUwNbRTp056/PHHGz3++eefS5IyMjIkSevXr5d0qCNYWVmp5ORkp/QAAAAAgDZ2xKuGbtu2TUuWLNHvfvc75ebm6rzzzpO0vyN4/PHHa9asWRo8eLAGDx6svLw8LV261LxoAAAAAMDRS4hGo1G3weXl5Ro2bJgkqXPnznrsscfq//+SSy7RmjVrNGrUKF100UWKRCJasGCB8vPzNWfOHI0bN65VfgEAAAAA8ILs7Of0xRcV7V1GvR49gioouKLJY0fUEdy1a5fef/99VVdXa+HChfr88881d+5cjRo1Ss8++6zq6uo0ceLE+viqqipdeOGF2rt3r9555x0lJia6r3r1aqm6uvnjQ4ZIK1e6z2chPd0u12Hft4zZge9nWijrPdQxJhyWysqcc4XfeMGgov3qLptglqu83CyVwsEW2ugRilT5HWNCISkScc4VqnHxAh2JJNc7zTgzfAHqMrPMcvkq3JxYdy9AWU3IoKL9vv7aLJXSOrv4Hd2qqTFLVR0MO8b4/S2/JdTHFW4yqOgAy3u+4fkyzeWG25u+5c01O9ssVWWV3ZbJlrfCqirnGNf3/Iqi2As6XH6+Wap1mSPMcv3mN2ap9MzTDsv7+3xSnbstAIacZtfGVv633T0skppjlsvituPzSSkpsefxuo7UETyilnviiSdq9OjRGjdunBYtWqSMjAzNmjVLknTllVc26ARKUiAQ0NixY1VaWlq/mAwAAAAAoH0d9Z8wAoGAzjnnHG3fvl1lLfylMBze/5feysrKo30qAAAAAOgA2nuV0Kb+Nc2xI7hx40aNGDFCixYtanRsz549SkhI0N69ezVmzBg9/PDDjWI2b94sScrMzHR6KgAAAABAG3DsCPbo0UO7d+/Wc889p+rDvqCxbds2vf766zr99NPVvXt3RSIRLV68WBUVh+bEFhUVacmSJRo2bJi6dOnSOr8BAAAAAOCIOH7tOSkpSTNmzNC0adN0zTXX6OKLL9bOnTu1aNEi+Xw+/epXv5Ik3XXXXZoyZYquuOIKjR8/Xnv27NGiRYuUlJSku+66q9V/EQAAAABoXwc3lPeKGDaUl6SxY8fquOOO0/z58zVr1iwlJyfrjDPO0NSpU9WzZ09JUl5env7whz/oscce0/33369AIKChQ4fqtttuq99kHgAAAADQ/lwvhDx69GiNHj26xZi8vDzl5eXFXBQAAAAAoPUY7ohjq65v/xaP+yTVDRrimKegwKYeScoJ2u3BVRe022esZpDz3n9uhT98xznorLMUXu0izvCPAr4Cu311woaN4m81dvsjudk2KxSSioud4ypc7Mt2JEpL7XINyLbLZbmdWn6B8zU5YIC0yk1cpt0+jiUyfC3dbFzmUlFNV7NcGXKzMWlY/goXccFg7AUdZLgvXiSYYZYrmGKWSr7SHe4C23jvwqJiu33ZLLc37Jtit19fQYVzm3B7zw+l2H6cKxtk996Wabht8uzZdrmc9v5budL9/oArP3K336AbTzxlt/ff+eebpVJGisHq/wkJkjrHnsfz6tTSdMy213wtdndaAAAAAECH4NkRQQAAAADoWFreu6/txbCPIAAAAADg2EJHEAAAAADiDFNDAQAAAMBEx9lHkBFBAAAAAIgzdAQBAAAAIM4wNRQAAAAATLBqKAAAAADAo+gIAgAAAECcYWooAAAAAJhg1VAAAAAAgEfREQQAAACAOMPUUAAAAAAwwaqhAAAAAACPoiMIAAAAAHGGqaEAAAAAYKJOLa3U2fZYNRQAAAAAcIBnRwR9hVukmprmA3Jy5CvY5JgnJzPTrqjSCrtcwZBZqpZO05Hy9+/vLtBNXIXd+apMzzHLlRwImOUaWbHOLJeCQRdBGcoNFjlGVadmxF5Pg2d1fk7XDC+jJMPraEDAzWuZ6y6uys1r6U6a4bmvTLFrFxkbVpnlUpKLt6JwWCoudo7Lzo65nHoFBWapAobXpK/Yrk2sLHaua0hXaWVhV8e4r792jnFrWLctZrkyMlPMcinJLld2qsu4bOeYuiS7cy9JpRvscuX2thsZyc+3G79Y+d9OnyFzXMTs98RTdp9RJl9nd74m32h3vp6YXhh7kqQkKcfuXCF2nu0IAgAAAEDHwobyAAAAAACPoiMIAAAAAHGGqaEAAAAAYIIN5QEAAAAAHkVHEAAAAADiDFNDAQAAAMAEU0MBAAAAAB5FRxAAAAAA4gxTQwEAAADARFQtbeLe9qLNHmFEEAAAAADiDB1BAAAAAIgzTA0FAAAAABOsGgoAAAAA8Cg6ggAAAAAQZ5gaCgAAAAAmmBoKAAAAAPAoOoIAAAAAEGc8OzV0b5csRZvf/1DJkirTcxzzfPieXU2DBmWY5SovMEuligq7XP37hx1jfJLqUlzEffSRQUX7FffOMsuVU1polkunnWaWqqjY+e8yGZKK5NwOqwx/RUmqqLBr+/37m6WSryJilmtVVa5jzAC3cTVrDCrar7p3X7NcyVV256uu/wCzXF9+6RyTJqnkW87nIq14U+wFHVCZbXjui7eY5VpVbnc/DATs4ob0tmtfJXvtfse0mjKzXAoGzVL5N6xzDsrNlb/ARZzbF9Kl3BS7fFsKu5rlGnma3WsZSWr5M2RIUiTV+XOmJJ1/vkFBB0y+0W6M5on5dpua3zPb+b3PyYknSlOmGBTjeXXy1tTQ5tsBI4IAAAAAEGfoCAIAAABAnPHs1FAAAAAA6FhYNRQAAAAA4FF0BAEAAAAgzjA1FAAAAABM1KmllTrbHquGAgAAAAAOoCMIAAAAAHGGqaEAAAAAYIIN5QEAAAAAHkVHEAAAAADiDFNDAQAAAMAEG8oDAAAAADyKjiAAAAAAxBmmhgIAAACACaaGAgAAAAA8io4gAAAAAMQZpoYCAAAAgImOs6F8QjQajbZhJa4984xUUdH88Ztvlv74R+c8N9/Y/C9/pOoMB1B9G9aZ5VqnXLNcwaBzTEaGVFTkIi5pR+wFHZRk+DeL/Hy7XL172+UqL3eOyc2V1jm3nZIT7dqEJHXpYpersNAuV1am3fVdWeV8fScnS5WVzrl27zYo6IC0bSvNcq1KGmKWKyXFLJW719Hnk+pcxFVVxV5Qa+Ryc327lZpql6ugwDlmwABp1SrHsDdLB8RezwGDBpmlMn0ZM9Lt7jmRCud7TigkRSLOuUIFzq/PkVgXsHstc4vfMcul/v3NUpUp3OLxcFgqK3OXKxxw8cbgluGb5D3/bfdZ4M7pRm3fd+xPRszOvk1ffFHa3mXU69EjVQUFDzR57Nh/NQAAAAAADTA1FAAAAABMsGooAAAAAMCj6AgCAAAAQJxhaigAAAAAmKhTSyt1tr3ma2FEEAAAAADiDB1BAAAAAIgzTA0FAAAAABMdZ0N5RgQBAAAAIM7QEQQAAACAOMPUUAAAAAAwwYbyAAAAAACPoiMIAAAAAHGGqaEAAAAAYIJVQwEAAAAAHkVHEAAAAADiDFNDAQAAAMBEnVqajtn2mq/Fsx3B4493jgkGW7+OVpNkd+otX8TERMM4n+GAs2WuTp3sclnW5bZNuIizLMua2zbW1hIS7OJMz7/fb5bqOMNz79XX0fUL6UZ7XN9uWP6Oxx1nFhcIxFjLYSxPvVfbquU9x/Xr6JJlc/Xqe66bTK6fzvKaNDz5J55olgpHIDPzW+1dQgMH69m+fbtqaxt+dzEhGo1G26MoAAAAAEDrqqqq0llnnaVdu3Y1eJyOIAAAAAAcoyKRiCKRSKPH6QgCAAAAQJzx8DeJAAAAAACtgY4gAAAAAMQZOoIAAAAAEGfoCAIAAABAnKEjCAAAAABxho4gAAAAAMQZOoIAAAAAEGfoCAIAAABAnElq7wKO1NatWzVnzhytWLFCknTOOedo+vTpCofD7VzZse+yyy7Tp59+2ujxUaNGad68ee1Q0bHvV7/6lQoKCrRw4cIGj3MdtI3mzj/Xgr13331XjzzyiD777DP5fD4NHDhQt956qwYNGlQfQ7tvHW7OPW2+9Sxbtkzz5s1Tfn6+gsGgzj//fN166606/vjj62No+63Dzbmn7eNY1qE6gjt37tS1116r6upq3XjjjaqtrdXjjz+utWvXavHixfL7/e1d4jErGo1q48aNysvL08iRIxsc6969eztVdWxbvHixXnjhBQ0dOrTB41wHbaO588+1YG/FihW66aabdPLJJ2vq1KmqqanRM888o6uvvlrPPPOMBgwYQLtvJW7OPW2+9SxbtkyTJ09Wv379dPvtt2v79u1asGCBVq9erUWLFsnn89H2W4mbc0/bxzEv2oE88MAD0VNPPTW6YcOG+sfef//9aG5ubvT5559vx8qOfVu2bInm5uZGX3zxxfYu5ZhXU1MTfeihh6KnnHJKNDc3N3r11Vc3OM510Lqczj/Xgr2xY8dGzznnnGhlZWX9Y19++WX09NNPj1533XXRaJR231rcnHvafOu55JJLoj/4wQ+ie/furX/s6aefjubm5kbfeuutaDRK228tbs49bR/Hug71HcGlS5dq6NCh6tWrV/1jw4cPV8+ePbV06dJ2rOzYt2HDBklqcO5hb9++fbrkkkv00EMPaezYsUpLS2sUw3XQetycf64FW7t27VJ+fr7OP/98de7cuf7x1NRUnX766fr4448l0e5bg9tzT5tvHfv27dNJJ52kCRMmKBAI1D9+cBbC2rVrJdH2W4Pbc0/bx7Guw0wN3bVrl7Zu3apRo0Y1OtavXz+9/fbb7VBV/Fi/fr2kQzfDyspKJScnt2dJx6R9+/apoqJCc+fO1ejRozVixIgGx7kOWpfT+Ze4FqwFg0G99tprDToiB+3cuVOJiYm0+1bi5txLtPnW0qlTJz3++OONHv/8888lSRkZGbT9VuLm3Eu0fRz7OsyIYElJiSQ1+Rf6Ll26aPfu3dq9e3dblxU31q9fr+OPP16zZs3S4MGDNXjwYOXl5fHXSGPBYFB/+9vfNHr06CaPcx20LqfzL3EtWEtMTFR2dnajNp2fn6+VK1dq8ODBtPtW4ubcS7T5trJt2zYtWbJEv/vd75Sbm6vzzjuPtt9Gmjr3Em0fx74OMyK4Z88eSWryL5edOnWStP8vNSeccEKb1hUvNmzYoD179mj37t269957FYlEtGDBAt122236+uuvNW7cuPYu8Zjg8/nk8zX/9xmug9bldP4lroW2sGfPHv3yl7+UJN188820+zb0zXMv0ebbQnl5ef0MhM6dO2vGjBnq1KkTbb8NNHfuJdo+jn0dpiMYjUYdYxISEtqgkvg0YcIE1dXVaeLEifWPjRkzRhdeeKHuu+8+XXTRRfXTiNB6uA7aH9dC69q7d69+/OMfKz8/Xz/84Q81dOhQrVy50vHnaPexa+rcS7T5tpCQkKC5c+equrpaCxcu1PXXX6+5c+eqS5curn4WR6+5cz9q1CjaPo55HWZq6ME52fv27Wt07OBjwWCwTWuKJ1deeWWDG6EkBQIBjR07VqWlpfVfqEbr4jpof1wLrScSiWjy5Mlavny5Lr30Uk2dOlUS7b4tNHfuJdp8WzjxxBM1evRojRs3TosWLVJGRoZmzZpF228DzZ17ibaPY1+H6Qge/OLul19+2ejYjh07FAqF+AJvOzi4mW1lZWU7VxIfuA68i2shNl999ZUmTZqklStX6vLLL9fvfve7+pEO2n3raunct4Q23zoCgYDOOeccbd++XV27dpVE228rh5/7srKyZuNo+zhWdJiOYCgUUmZmpj777LNGx9asWaP+/fu3Q1XxoaSkRGPGjNHDDz/c6NjmzZslSZmZmW1dVlziOmhfXAuto6KiQjfccIM+//xzXXfddZo5c2aDjgjtvvU4nXvafOvZuHGjRowYoUWLFjU6tmfPHiUkJMjv99P2W4Gbc793717aPo55HaYjKEkjR47UsmXLtHHjxvrHPvjgA23evLnFVf4Qm7S0NEUiES1evFgVFRX1jxcVFWnJkiUaNmyYq+8xwAbXQfvhWmgdM2fO1Oeff65JkybpjjvuaDKGdt86nM49bb719OjRQ7t379Zzzz2n6urq+se3bdum119/XaeffrqCwSBtvxW4Offdu3en7eOYlxB1s/qER5SVlenCCy9UYmKiJk+erH379mn+/PnKysrSc889J7/f394lHrPeeOMNTZkyRSeffLLGjx+vPXv2aNGiRfr666/17LPPstlqKxkxYoS6d++uhQsX1j/GddB2mjr/XAu2Nm7cqNGjRysUCumOO+5ocuGFsWPH0u5bgdtzT5tvPS+99JKmTZumQYMG6eKLL9bOnTvrz+0zzzyj3Nxc2n4rcXPuafs41nWojqAkbdq0SbNmzdJHH32kQCCgs88+W9OmTaufr43W88Ybb+ixxx5Tfn6+AoGAhg4dqttuu40bYStqqiMicR20lebOP9eCnWeffVa/+c1vWoxZu3atJNq9tSM597T51vPqq69q/vz5WrdunZKTk3XGGWdo6tSp6tmzZ30Mbb91uDn3tH0cyzpcRxAAAAAAEJsO9R1BAAAAAEDs6AgCAAAAQJyhIwgAAAAAcYaOIAAAAADEGTqCAAAAABBn6AgCAAAAQJyhIwgAAAAAcYaOIAAAAADEGTqCAAAAABBn/n9J4553XrR7dAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_168_1.png" } }, "output_type": "display_data" } ], "source": [ "fig = plt.figure(figsize=(20, 14))\n", "im = plt.imshow(J_sk, **cmap_args)\n", "plt.title(\"LinearRegression from Scikit-learn\", fontsize=18)\n", "plt.xticks(fontsize=18)\n", "plt.yticks(fontsize=18)\n", "cb = fig.colorbar(im)\n", "cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The results agree perfectly with our previous discussion where we used our own code.\n", "\n", "\n", "Having explored the ordinary least squares we move on to ridge\n", "regression. In ridge regression we include a **regularizer**. This\n", "involves a new cost function which leads to a new estimate for the\n", "weights $\\boldsymbol{\\beta}$. This results in a penalized regression problem. The\n", "cost function is given by" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "6\n", "0\n", " \n", "<\n", "<\n", "<\n", "!\n", "!\n", "M\n", "A\n", "T\n", "H\n", "_\n", "B\n", "L\n", "O\n", "C\n", "K" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ ":10: UserWarning: FixedFormatter should only be used together with FixedLocator\n", " cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAM2CAYAAACjUj0CAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAB4jElEQVR4nO3dfXxU5Z3//3cmYRiGYYzZQCBkYww0ImAEVyOyLlpMwYIabAVR1Crqun7pb4uuteh6t66KVitdql/rFqpFgrfLqpXeuG7Xm65Yvj4oVbkVMIQQEoghhBBCSGZ+f0AGYm7OwfkkOWFez8eDR+ucTz7zyZnrnJlrrivXlRSNRqMCAAAAACQMX08XAAAAAADoXnQEAQAAACDB0BEEAAAAgARDRxAAAAAAEgwdQQAAAABIMHQEAQAAACDB0BEEgHbMmzdPp512Wpt/Z5xxhr75zW/q7rvvVlVVVaufmThxoq699lrH3G7jrD3//PM6//zzlZ+fryeeeKLbn9+NxsZG/fu//7suu+wyjRkzRmeddZa+853v6N///d918ODB487X8jq2+NnPfqbTTjtNZWVlHf7M8uXLddppp+lPf/pTq8e3b9/u+HxlZWU67bTT9LOf/ey4awUAoDul9HQBAOBld911l04++eTYf9fV1WnlypX6j//4D3322Wd67bXX5Pf7JUl33323+vXr11Oldmrjxo2aP3++xowZox/84AcaMWJET5fURlNTk2688UatWbNG06ZN05VXXqnm5mZ9/PHHevLJJ/WHP/xBS5YsiZ1vN6688kqdd955x1XHOeecox//+McaNmxY7LEbb7xRAwcO1KOPPnpcuQAA8Co6ggDQicLCQmVlZbV6bNasWXrggQf04osv6p133tGUKVNisV61adMmSdItt9yiiRMn9nA17fvtb3+rVatW6Wc/+5kmTZoUe/y6667TokWL9Pjjj+u1117T1Vdf7Trn2LFjNXbs2OOq46//+q/113/9160e++Mf/6jLL7/8uPIAAOBlTA0FgK+hpVPwl7/8pYcrcefQoUOSpP79+/dwJR3785//LEn627/92zbHrr76avXp00dr1qzp5qoAADgx0REEgK+hZQpoNBqNPdbe3/795je/UVFRkfLz83XJJZfov//7v9vN995772n69OkaM2aMLrroIhUXF+uf//mf24zebd68WXPmzNHZZ5+tM888UzNnztQHH3zQaa3XXnut7rrrLkmHR9da/mbu2muv1Y033qgFCxZo7NixOu+887Rx40ZJh6eS/p//83909tlnKz8/XzNmzNA777zTJu8tt9yid955R5dddpnOOOMMTZ06Ve+9957q6up033336ZxzztF5552n++67Tw0NDZ3W2dJJffnll9scCwaDWr16tX784x+3enzLli36wQ9+oHPPPVd/8zd/o2uvvVYff/xx7PhX/0awPQ888IBOO+00Pf/885Ja/41gy9/8SdJ//ud/tvu3g24sX75c06ZN0xlnnKFx48Zp3rx52rVrV6uYuro6/eQnP9HFF1+sM844Q2PHjtWMGTNatZmWep5//nldddVVGj16tK6//vrY46+//roWLFigCRMm6IwzztD06dP10UcfHXe9AIATH1NDAeBraOl8jRw5ssOY5cuX66677tLYsWP1wx/+UNu2bdPcuXOVlJSkoUOHxuL+53/+R3PmzFFeXp5uu+02VVZW6tFHH1UwGGw1grdx40ZdffXVSk9P1y233KI+ffrorbfe0t///d/rJz/5SWyK6lf9wz/8g0499VS9/PLL+od/+Afl5ubGjq1evVrbt2/XD3/4Q5WVlWn48OH65JNPdN111ykUCumGG25Q//799cYbb2jOnDm67777NGvWrNjPr127Vn/+85913XXXacCAAXr22Wc1d+5cnX766erXr59uv/12ffzxx3r55Zc1aNAgff/73+/wfF122WV67rnn9Nhjj2n58uUqLCzUeeedp7Fjx8rv97f528CSkhLNmDFDKSkpuuaaa5SWlqaXXnpJN9xwg4qLi5Wfn9/hc7X46U9/qhdffFG33Xabrr/++jbH09LS9OMf/1h33nmnzj77bM2YMaPV3w668dRTT+lnP/uZJk+erBkzZqiyslJLly7VqlWr9NprryktLU3RaFS33HKL1q1bp2uuuUbZ2dmqqKjQSy+9pO9///t6/fXXW3Vo/+3f/k0TJ07UpZdeqr59+7Z6vF+/fpo9e7YOHTqkX/7yl7rlllv07rvvtvpbVwAAFAUAtPGjH/0ompeXF127dm30yy+/jP3btm1bdOnSpdExY8ZEv/3tb0cbGxtjP/PNb34zes0110Sj0Wi0qakpet5550W/+93vtor5j//4j2heXl4sLhqNRgsLC6OTJk2KHjhwIPbYf/3Xf0Xz8vKi3/zmN2OPXXPNNdHCwsLo/v37Y48dOnQoevXVV0fHjx8fPXjwYIe/T8vzfvTRR63y5eXlRdesWdMqdvr06dExY8ZEd+7cGXusoaEhevnll0fz8/OjX375Zauf/8Mf/hCLW7p0aTQvLy86Y8aM2GORSCQ6YcKE6JVXXtlhfS3+53/+J3reeedF8/LyYv/GjBkTvf3226Nbt25tFfuDH/wgmp+fHy0pKYk9Vl1dHf2bv/mb6D/+4z9Go9Gjr2OLhQsXRvPy8qLbt2+P/upXv4rm5eVFn3zyScdzlZeXF/3Rj37kWP/27dujeXl50YULF0aj0Wi0tLQ0OmLEiOgTTzzRKm7jxo3RUaNGRR9++OFoNBqNrlmzJpqXlxd98cUXW8W9//770by8vOgvf/nLVvm//e1vRyORSJvnveCCC1q1jxUrVkTz8vKiL7/8smPtAIDEwtRQAOjE5ZdfrvPOOy/271vf+pYef/xxTZw4UcXFxerTp0+7P7d27Vp9+eWX+s53vtMqpqioSCeddFLsvzds2KDS0lLNnDlTgUAg9nhhYWGrkbs9e/Zo1apVuuCCC9TQ0KDq6mpVV1ertrZW3/rWt1RVVaVPP/30uH+/QCCgM844I/bfVVVV+stf/qKioiINHjw49njfvn114403qqGhQR9++GGrx//u7/4u9t+nnnqqJOmiiy6KPdYyArp7927Hei688EL9z//8jxYsWKCioiINHDhQ9fX1euutt1RUVKRVq1ZJkiKRiN577z1dcMEFOuWUU2I/f/LJJ2vZsmW65557On2eN998U4888oi+853v6LbbbnOs6+v6r//6L0UiEU2cODH2mlVXVys9PV2nn3663n33XUnSmWeeqf/3//6fvvOd78R+trm5WZFIRJK0f//+VnnPPvtsJSUltXm+Cy64QMFgMPbfLavDujn3AIDEwtRQAOjE448/rvT0dB06dEgffPCBiouL9e1vf1sPPPBAqyl5X7Vjxw5JUnZ2dqvHk5OTW3Vctm3bJkmtHmuRm5ur9evXSzq6h90LL7ygF154od3n3Llz53H8ZoelpqbK5zv6nWBL3S0dumO1TIksLy9v9fMpKUffSpKTkyVJf/VXf9XqZ5OTk1v9PWVn+vbtqylTpsSmuq5du1a//OUv9dZbb+n+++/Xb3/7W9XU1Ki+vr7d85aXl+f4HP/2b/8mn8+nTz/9VIcOHeqwQ9+Rmpqa2AI8LdLS0trElZaWSpJmzpzZbp5jnzclJUUvvfSSVq1apW3btqm0tDT2d5VfPXftPVd7j7dMp23pUAIA0IKOIAB04qyzzoptH9Ey+vTQQw+ppqZG//f//t92R2UkxR5vb4GUYz+UNzU1SVK7e+Md29Fsbm6WdHjrio62qRg+fLibX6mVlo5bi846ay11f7Xz0p6OzktH6uvr9eyzz2rUqFGtto6QpFGjRuknP/mJamtr9f7772vPnj2x83G8z9Pisssu0znnnKN7771Xixcv1j/8wz8c18//f//f/xcbnWzR3kJALefsmWeeaTXi+1XV1dWaPn26du3apb/927/VxIkTNWLECA0dOlTTp09vE//V163FsZ16AAA6Q0cQAI7Dtddeq5UrV+q///u/9atf/ardBUYkxfahaxnxaxGNRrVjxw594xvfaBVXUlKi888/v1VsSUlJ7P+3LC6TnJys8ePHt4rbvHmzysrKTDazb3merVu3tjn2xRdfSFKrKaNW+vbtq8WLF2vs2LFtOoIthg8frg8++ECBQEB9+vRRIBCIjbgda/Hixdq9e7fmzZvX4fP94Ac/0NChQ7V8+XI988wzmjp1apu9Azvzox/9SLW1ta0eGzhwYJspmC3nc8iQITr99NNbHXvvvfcUCoUkScuWLVNZWZmef/55nXfeebGY1atXu64JAIDjwVeHAHCcHnzwQZ100kn66U9/Gpuy+VUjR47U0KFD9eKLL+rAgQOxx1esWKE9e/bE/nv06NEaMmSIXnvtNTU2NsYeX7NmjdatWxf770GDBmn06NH6z//8T1VWVsYeP3TokO6++2794z/+Y2x0MR4DBw7U6NGj9eabb6qioiL2eGNjo5577jn5/f529/mLV3JysqZMmaJVq1bpjTfeaHO8pqZGv//97zV+/Hj169dPKSkp+tu//Vu99957rabE7t27V4sXL+7wdTlWUlKS7rvvPh06dEgPPPBAp7E+n6/VSO7o0aM1fvz4Vv/amyr8zW9+U5L07LPPthptXb9+vW699Vb96le/iv1+UutR3Wg0qqVLl0qSyWsLAMCxGBEEgOOUnp6uO+64Q/fee68eeOABLV68uE1MUlKS7r33Xs2ZM0dXXnmlvvvd76qyslLFxcVKTU2Nxfl8Ps2bN09z587VzJkzVVRUpOrqai1ZsqTNdNF77rlH3/ve9/Td735XV111lVJTU7VixQr95S9/0T/90z+ZbQ/Q8jxXXHGFrrrqKvXv319vvvmm1q5dq3vuuUfhcNjkeb5q3rx5+uSTT3TnnXfqzTff1N/93d8pFAqptLRUy5cv16FDh3TffffF4v/pn/5J06dP1/Tp0zVr1iyFQiG98sorqq+v19y5c10958iRIzVz5kwVFxdrxYoVmjp1artxaWlpWrVqlV555RWdf/75yszMdJU/Ly9P1157rV544QXV1NSosLBQNTU1Wrp0qfr3768f/OAHkqQJEybohRde0C233KIrrrhChw4d0m9/+1t99tln8vl8bRaLAQAgXowIAsDXMH36dP3N3/yN/vjHP+r1119vN+ab3/ymnn32WQUCAT355JN655139PDDD7daDVSSLr74Yi1YsEBNTU16/PHH9dZbb+muu+7S6NGjW3UGx44dqxdffFGjR4/Wc889p8cff1wHDhzQo48+qr//+783+91anmfUqFH65S9/qX/7t39T37599fTTT+vaa681e56vSktL0/Lly/WDH/xANTU1evrpp/XAAw/orbfe0qRJk/TrX/9aOTk5sfhhw4bp5Zdf1hlnnKFFixZp4cKFGjRokJYtWxabeuvG3Llz9Vd/9VeaP39+m+meLe644w41NTXpX//1X9v8baCTf/7nf9b999+v6upqPfbYY1q2bJnOPvtsLVu2LLYAz4QJE/TQQw/FXs9FixYpNTVVL7/8sk4//fSvtYk9AACdSYq6XcYNAGCuublZe/fubXcVyEsvvVThcFjFxcU9UBkAADiRMSIIAD2oublZEyZMaDXlUZI2btyozz//XPn5+T1UGQAAOJHxN4IA0IP8fr8uvvhivfbaa0pKStLo0aO1a9cuvfjiizr55JN1ww039HSJAADgBMTUUADoYQ0NDVq8eLHefPNN7dy5UwMGDNB5552nuXPnxvYwBAAAsERHEAAAAABOULW1te0uhkZHEAAAAADidKC6Wv3aWfytpzU0NGjChAnau3dvq8e92xE8ZuPedvl8zjGS/vVhu/Vw7v1Ro3OQSwea/c5BLlVXm6VSByunt3L66dL69S7ihtmdL0uVe+zOfUbTDrNcam52jsnOlkpLHcO2NmUbFHRUbv9K5yCXavpmmOVKDdq1sfc/cm4XEyZI77/vnOvIjgAmhjY7v95u7TvZrl0MWGe3nUFlzrmOMRkZUqWLZpgRPmBQ0WH7mvqZ5Wpnr/mvzb9ujVmuA6eNcYzp10864OK09uvr/J7cE3bstPscMHCgWSpX/H6p0cVtzr/H7h4tSY0n292nLWszff8oWdN5wJgx0hqHmBaWi3rt3GmWqnHgULNcHeySdFyCQemSS+LP43W/PP981ZaV9XQZMeGsLM3+4x+1c+dONX/ls+YJv1jMnj09XUH7LLvfbvoPbh06ZBvnRZbnyzRZU5NZnNtUrhn+ni6+v+kRDQ12caZtzPDFNP3a7+BBs1Ruz5erOMNf0qNfk7rrGbjk9nf07LlwwfR69Cov/5Jeff9wcx0ZXmuuefS1rKvr6Qp6j9qyMu3dtq2ny2hjyJAhbR5j+wgAAAAASDDmHcHt27fr+9//vgoKClRQUKA777xT1ZZzFwEAAADAg3we/NcR06mhe/bs0fe+9z01NjbqpptuUnNzsxYvXqyNGzfq1Vdfld9v97dZAAAAAICvx7Qj+Pzzz6uiokK//vWvNezIaglnnnmmbrjhBr3++uuaMWOG5dMBAAAAAL4G06mhK1asUEFBQawTKEnjx4/XqaeeqhUrVlg+FQAAAAB4Sk9PAz2eqaFmHcG9e/dq+/btGjVqVJtjo0aN0tq1a62eCgAAAAAQB7OOYOWRDZ4yMtru8TJw4EDt27dP+/bts3o6AAAAAMDXZPY3gvv375ck9evXdgPevkd20q2vr9eAAQPcJfS56KO6iHnySXdP547dYjdBw3Vzsg33Dneby93eqd5cHCgz0zKb7cbtruTmOobkmT+p3UlLM8skWbaxSZNs4+w4v95uhc0ySZowwSyV29bl7toNxlFJa6bny1JBgVkqt2cr6CrQmztSWb5H9gRX6+zZvrHZvnsb1mb6/uHmOjK81lwzbLCWr+Ps2YbJTnBO0zG7W7esGhp1sdtsUlKS+4ROu4b6fK52Fr39DruX4slH7TYWrW+yuzyrqsxSqabGOSY/X/rkExdxI3pgI1YXyqvszn1mU6lZLlcbh+fmSlu3OoZtarLrPEhSXqjcLFd1wPBDQciujb39rnO7mDRJevtt51wjRhgUdER2k/Pr7VZtumGncs37ZrnKhzt3KjMzpXIXzTAztd6gosNqm+w6lYGAWSr516wyy1U/2vmDbjAo1bs4rcGA5W7fdkrL7D4HDB5slsoVv9/dnub+Krt7tCQ1ptvdpy1rM33/2OxwHRUUSKtcXmtnnx1/QS3KysxSNQ6261QuXRp/jlBIYt1IbzG7OwaPfF148ODBNsdaHguFQlZPBwAAAAD4msxGBDOPDP3v3r27zbFdu3YpHA7HOosAAAAAcKJJkremhnY2H9OsznA4rKysrHZXB123bp1Gjx5t9VQAAAAAgDiYdlgnTZqklStXasuWLbHHPvzwQ33xxReaMmWK5VMBAAAAAL4ms6mhknTzzTfrjTfe0PXXX6/Zs2fr4MGDWrRokUaNGqWioiLLpwIAAAAAT+lNq4aa1pmWlqalS5dqxIgRWrhwoX71q1+psLBQixYtkt/V+scAAAAAgK5mOiIoSbm5ufrFL34Rd55/fdinPXs6Pv7kk+62hnjyCbvlrO+cZ9eZnTfPLJVSDF/FfLnYF0L5ruJqG1xtNuiKm20t3Mou+9Au2bhxZqnqG5zbc1BS/WAX+wjW7TKo6Kj6kOGS3YZLzJdX2F2Tbl9KN3ElJXGV0kr26ByzXOEmu+02SnPs9hHMdrUNS7a77VpS7Nb3D1fYbd1huSS8xowxS7VmjXPM+PEu40bUxFnNMQz328je/JFZrk0NE81y5Q13cy/0yZ/iHFfaZLuPYHaZXduP5NhtW1NTYpZKaW4+PLn8gLW1xG5cJSfHbsuHzRvMUmn29Vbv3V4aK4N5RxAAAAAAElHCTg0FAAAAAHgfHUEAAAAASDBMDQUAAAAAA0wNBQAAAAB4Fh1BAAAAAEgwTA0FAAAAAANJ8tZIW1Inx7xUJwAAAACgG9ARBAAAAIAEw9RQAAAAADDAqqEAAAAAAM9iRBAAAAAADCSp8wVauhuLxQAAAAAAYugIAgAAAECCYWooAAAAABhIPvLPKzqrhRFBAAAAAEgwdAQBAAAAIMEwNRQAAAAADCTJWyNtna0amhSNRqPdVsnxaGzs/Ljf7xwj6c57/EYFST9+NGKW69Y5dk3k4YfNUikUco5xeerlr9kVf0EtamrMUjXm5Jnl8qfYtYmIi9uGzydFXDxlSUn89Rwrt2mTWa5PGuzOf37DKrNcGjzYOSY7WyotdQxb9sdsg4IOu3pavVmuxpSgWa6GBrNUCpd84hyUny994iIuPT3+grpAbSjTLFc4ZHffUV2diycMS7W1jmHrysIGBdkbMcIul0+G597NRRQMSvUu7gFlZfHXc6xAwCzVh2V298OzzzZL5fwZZdAgaZfLzzEphuMqbq5Jlyr72p37AQPiz5GUJPXrF38eryvOyVHdtm09XUZM6JRTNKuDD4Ze6rACAAAAALoBU0MBAAAAwIBP3hpp66wWL9UJAAAAAOgGdAQBAAAAIMEwNRQAAAAADDA1FAAAAADgWXQEAQAAACDBMDUUAAAAAAwwNRQAAAAA4Fl0BAEAAAAgwTA1FAAAAAAMJMlbI21JnRzzUp0AAAAAgG5ARxAAAAAAEgxTQwEAAADAAKuGAgAAAAA8i44gAAAAACQYpoYCAAAAgIEkdb5SZ3dj1VAAAAAAQExSNBqN9nQR7TlwQOqssmBQqq93ztPQYFfTP/+zXa5nno6Y5XrkUbv+/N037XIOGjRI2uUc9+HmQQYVHTZ+jIsX263f/c4s1eqc75jlOqtplXNQQYG0yjkucnaBQUVH+TZvMsvVmJNnlstfVW6W65OqTMeY/Hzpk0+cc+WPtru+I4bf1zU1maWSP8XudyyvcP4dMzOlchcvd2ao1qCiw2oVNssVll1dkZBdXW74fFLExcvta2o0e851m/1muUaOsGurph8qUlxMyvL7pUbn82p5viRpZKrdvdXynNWm55rlCocc2oXbhm/N8EZdWmHXLrLr1sWfpE8f6RvfiD+Px/1nTo72b9vW02XE9D/lFF1eUtLuMaaGAgAAAICB5CP/vKKzWpgaCgAAAAAJho4gAAAAACQYpoYCAAAAgIEkeWukjVVDAQAAAAAxdAQBAAAAIMEwNRQAAAAADPjkrZG2zmrxUp0AAAAAgG5ARxAAAAAAEgxTQwEAAADAAFNDAQAAAACeRUcQAAAAABIMU0MBAAAAwABTQwEAAAAAnkVHEAAAAAASDFNDAQAAAMBAkrw10pbUyTEv1QkAAAAA6AZ0BAEAAAAgwXh2amh1tdTc3PHx7Gypqso5T4rhb/jww3a5HnnUrg9+97yIWa6rrxnkGLNsmXT1XBdxP91lUZIkqbzG+fncqhnxHbNcAbNMknJGu4sb7Rznq6mOs5iu4//ofbNcjeMmmOXKbyp1EZWt/FTnuIiy4y/oiLo6s1RqarLLlRayS5aa6ncZ5yIoxe6qDDfUmuVSTY1Zqj+uCZvlOv98s1S2DUzu2kR3e//joFmu4cOdYzIzpfIq53MxMrDVoKKjakO5ZrnC6Y1muSpKzFIpXPNx5wEFBdLHDjFHRM4uMKjosJo6u7Zv+f6xNTAy7hwpKTJ8d/QuVg0FAAAAAHgWHUEAAAAASDCenRoKAAAAAL1JkjpfqbO7sWooAAAAACCGjiAAAAAAJBimhgIAAACAgeQj/7yis1oYEQQAAACABENHEAAAAAASDFNDAQAAAMBAkrw10saqoQAAAACAGDqCAAAAAJBgmBoKAAAAAAZ88tZIW2e1eKlOAAAAAEA3oCMIAAAAAAmGqaEAAAAAYICpoQAAAAAAz6IjCAAAAAAJhqmhAAAAAGCgN00N9WxHsLZWOnSo4+PZ2VJNjXOefH1iVlPjiHyzXHfftMss19XXDDLLtWxpxEWUz1Xc7XfY1fWjH5ml0sgRbn5HlxoazFJtrQg6xuTmuosrK3OOOR4TRtvlaszJM8tVVWWWSk3KdozJllTqJq7K7voOpdtdRyUlZqmUlmLX9ssq/I4xeXlSWZlzrrz0OoOKDqtNSTPLVaOwWa4Uw3fuNWucY846y11cerrdfWfwYLNUqq2z+0iWmmqWSsnJhnGWhUkKhQyTlVWYpTrpJOf7r1vV6QWdHk+TVD2885hY7B/fN6joSK4xY8xyBXLs7jsVdi8jesjOnTvV3Nzc6jHPdgQBAAAAAPGbNWuWduzY0eoxOoIAAAAAYCBJ3poamnTkf4uLixkRBAAAAIBEMmTIkDaPeanDCgAAAADoBuYjgldccYU+/fTTNo9PnjxZCxcutH46AAAAAPCEhF01NBqNasuWLSosLNSkSZNaHRs6dKjlUwEAAAAAvibTjmBZWZnq6+t10UUXqaioyDI1AAAAAMCIaUdw8+bNkqRhw4ZZpgUAAAAAz0vS0ZU6vaCzWkynsH7++eeSjnYE6+vrLdMDAAAAAAyYdwT79++v+fPna+zYsRo7dqwKCwu1YsUKy6cBAAAAAMQhKRqNRq2SXX755Vq3bp0mT56sSy+9VLW1tVqyZIk2bNigxx57TNOmTbN6KgAAAADwlNU5OTq4bVtPlxHT95RTdFZJSbvHTDuCL774oiKRiGbNmhV7rKGhQZdccokOHDig999/X8nJya5yrV8vHTrU8fH8fOmTT5zz5MtFkEuNI/LNcvlrdpnlunruILNcy5ZGnIN8PiniHHf7HXYDzj/6kVkqZQx08Tu61dBglmprRdAxJjdX2rrVOVdZmUFBx5gwutosV2MozSxXVZVZKjU1OcdkZ0ulpS7iAnbXdyTd7vru4H3ga8lNrzXLtaki7BiTlydt2uScKy/drq3Wpti11Zoas1Sm13cg4Bxz1lnS6tXOcenp8dfTIhSyy5ViuBqC5TWUkeEuprLSRVwfu3YvSZFUu7bvK3Nx03Spsm+2Wa4+fTo/npYmVbs8rWmfvR9/QS3GjDFLVZ/ifG91q6Ii/hwpKYffR090vakjaDo19KqrrmrVCZSkQCCgoqIiVVVVxRaTAQAAAAD0HPMN5duTlnb4myUWjwEAAABwokqStzaU75ZVQysrKzV16lQ99dRTbY598cUXkqSsrCyrpwMAAAAAfE1mHcGMjAzV1tbq1VdfVV1dXezx8vJyLV++XOeee64GDhxo9XQAAAAAgK/JdGro/fffrzlz5mjmzJmaPn269u/fr+LiYqWkpOj++++3fCoAAAAA8BSfvDU1tLNaTOssLCzU008/rX79+umJJ57Qc889pzFjxujFF1+MbTIPAAAAAOhZpttHmGps7Py43+8cI6m2wW9UkBRusFsS/sPNdkvCjx9uV9ftjzrX9eST0u23O+d68gm7bRrunGf3ncU//ZNZKsflp49HWo2LfSFc7h+xqSnXoKKjBg+2yxUOOF+3rrnZ88EtN+v7Z2ZK5eXOcRs2xF1Oi9qzJ5rlCtfYLeNu2Sga5XyfdnnLl18ebV+Ge51EsuzWX/fJbssgy/Plpk24tWePWSplDLBb9C4ScN4yyO2pP+YvckxYbBXQIifHLpfllkGZf3yl84AZM6RXHGJaXHhh3PV0CcP3IpNtLZKSpAED4s/jcX/JyVGjh7aP8J9yis7sYPuIblk1FAAAAABOdAk7NRQAAAAA4H10BAEAAAAgwTA1FAAAAAAMMDUUAAAAAOBZdAQBAAAAIMEwNRQAAAAADCTJWyNtSZ0c81KdAAAAAIBuQEcQAAAAABIMU0MBAAAAwACrhgIAAAAAPIuOIAAAAAAkGKaGAgAAAICBJHW+Umd3Y9VQAAAAAEAMHUEAAAAASDBMDQUAAAAAA8mSIj1dxDGSOznGiCAAAAAAJBg6ggAAAACQYJKi0Wi0p4toV2Nj58f9fucYSaUVfqOCpOyGTWa5lJVllqq8JmiWK7mz8eMjMjKkykrnuJ/8JP56Wvz4UbtB9n+ca/f9x8yZZqk0eLBzTG6utHWrc1xOTtzltFJXZ5cr3LDLLlkoZJcrxcVMeZf3nU822N13DG8VSgvUm+UqrbK772Rnubi+fT4p4iLuo4/iL6jF8OF2uQzbauU+u3P/pz85x1x2mfTmmy7iLqyNv6AWTU1mqUrr0sxyZafa/Y6VB8KOMW7fbwcONCjoGL4SF280LlX2zzXL1a+fWSrHS9LtLUeSfGWl8Rd0RH16tlmuigqzVMpNN2j7SUnSgAHx5/G4LTk5atq2rafLiEk55RQNKylp9xgjggAAAACQYOgIAgAAAECCYdVQAAAAADDgk7dG2jqrxUt1AgAAAAC6AR1BAAAAAEgwTA0FAAAAAANMDQUAAAAAeBYjggAAAABggBFBAAAAAIBn0REEAAAAgATD1FAAAAAAMJAkb420JXVyzEt1AgAAAAC6AR1BAAAAAEgwTA0FAAAAAAM+SdGeLuIYrBoKAAAAAIihIwgAAAAACYapoQAAAABgIEmdr9TZ3Vg1FAAAAAAQkxSNRr3094wxlZVSc3PHxzMzpfJy5zyZJR+a1dR49nizXP63lpvlWjfiO2a5Ro6IOAf5fFLEOa5yt933DA8/bJZKC3/q4nd0acZMu9/xlZfszn3E+DuesjK7XCmG8xAyQ7V2yWpqnGOys6XSUue4rKy4y2lRW+fN7+vCIbvrqLTM+Xd0e+pDIYOCjkhNtctVV2eXK9xUbZfMzQnz+6XGRsew+ia/QUGHBRvsfsdNVWlmufJCLj54WHL5Yae0KdP0abMHO7/ebr3/kV27OP98s1SO12Q4LNW6fIsJf/yH+AtqMWKEXa6SErtcFm/efr80Zkz8eTxuZ06Omrdt6+kyYpJPOUVDOmgLTA0FAAAAAAPJPV3AV3RWjze/agYAAAAAdBk6ggAAAACQYJgaCgAAAAAGkuStkTZWDQUAAAAAxNARBAAAAIAEw9RQAAAAADDgtVG2zurxWq0AAAAAgC5GRxAAAAAAEgxTQwEAAADAgNdG2ZgaCgAAAACIoSMIAAAAAAmGqaEAAAAAYMBro2xMDQUAAAAAxNARBAAAAIAEw9RQAAAAADCQJG+NtCV1csxLdQIAAAAAugEdQQAAAABIMJ6dGprRtENqbu4kIluZTaXOicaNM6vJr4hZrtU53zHLFTDLJKmhwTkmGHQV16dP0KCgw2bONEulGTPtvv945SW7NvHIo8513X23u7g77rCo6KisLLtcvs2b7JKFBpulqk/PdowJuoyrqTAo6IhQyC5XVZVdrvDH75rlqkmf6BiTnS3V1Djnyg7sir+gFk2pZqlCIb9ZLtUZvnU3NTnH+P2u4hoa7H7HYIXdRZSXY/guWVJjlqo8daRjTKakcmU6xpm2e0n6aINZqtGjJ5jl8n28yixX+OyznZ5N4ZC79/gPA873MLfGq9wsl+VnYFc3YCe+xBh/8kmK9nQRx2BqKAAAAAAgho4gAAAAACQYz04NBQAAAIDehKmhAAAAAADPoiMIAAAAAAmGqaEAAAAAYICpoQAAAAAAz6IjCAAAAAAJhqmhAAAAAGAgSZ1Px/QSRgQBAAAAIMHQEQQAAACABMPUUAAAAAAw0JtG2XpTrQAAAAAAA3QEAQAAACDBMDUUAAAAAAz0plG23lQrAAAAAMAAHUEAAAAASDBMDQUAAAAAA71plM27HcHmZqmpqfMYp+OS6hvsXo5AwCyVzmpaZZcsZ7RZqq0VQceY3FyXcdpqUZIkafDgXLNcr7wUMcv1yKN27evueW7q8rmKs6xLclubO/VZeWa5gg3VdrlS6t1EKSjnuMBg5+vDrd27zVJp8GC7XOWBiWa50t3GuQm0vFGXlJil2n2SXbsfODBslstXUe4cFAxKNTWOYWlyjnGrPHWkWa5Ag1kqpRm2r8xQrYuosLu4lFDc9bRy/vlmqdLq3PyeLmVlmaWqrev8fTIcdo5p4ere5JZhGysts/sskN1QFX+SlBQpNTX+PDDj3Y4gAAAAACBuO3fuVHNzc6vH6AgCAAAAgIGkpCQpKamnyzjqSC2zZs3Sjh07Wh2iIwgAAAAAJ7Di4mJGBAEAAAAgkQwZMqTNY1+rI3jvvfeqpKREL7zwQqvHt2/frscee0yrVh1eCOXCCy/UvHnzlJaW9nWeBgAAAAB6j5SUw/+8opNajrvKV199Va+88ooKCgpaPb5nzx5973vfU2Njo2666SY1Nzdr8eLF2rhxo1599VX5/f7jLxwAAAAAYM51R7C5uVnPPPOMnnrqqXaPP//886qoqNCvf/1rDRs2TJJ05pln6oYbbtDrr7+uGTNm2FQMAAAAAIiLq47gwYMHNX36dG3cuFHTpk3TypUr28SsWLFCBQUFsU6gJI0fP16nnnqqVqxYQUcQAAAAwIktOdlbU0OTkzs85GqnyYMHD6qurk4LFizQY489ppSv/HJ79+7V9u3bNWrUqDY/O2rUKK1du/Y4KwYAAAAAdBVX3dVQKKS33367TQewRWVlpSQpIyOjzbGBAwdq37592rdvnwYMGBBHqQAAAAAAC646gj6fTz5fx4OH+/fvlyT169evzbG+fftKkurr64+vI5id7RyTm+sYEnT/jN3rK4vteIWLU3occS6TdWsmyeVAuCt3322WSq7r6uRabGFbl2R5zoKWF2WwB1YkdvEL2J0tqZ3v1zzB9HV0KTPTTVTY7gnDdrk8+jK6Panu44x077MdhzTbdyNXDNthj7Cs3zCXm0xun872JbJ7X8s2fYvMs0x2YjuRVw1tTzQadYxJOrKrvWulpVJTU8fHc3OlrVsd09QPtrtpBwJmqeT7eJVdstGjzVJtrXD+dOfy1CtXLoJc2mrZqcyJmOV65FHDTuU8F3X5fFLEOc6yLsllbS7VNxh2KhuqzXK5usCDQam+3jEsErDrJe3ebZZKlpMyamrscrmRmSmVl7uIC9XaPWlFhVmqypPsPkQNHGiWSr4KNyfV5ck3VG7YFbR8706rsXtfU3q6c0w4LNW6aNPWHzotT1pdnSdz1YY6b2NuT71keqtQXrrd+1ppnWGnsmFT/ElSUtyPOKBbmHwiCx75avjgwYNtjrU8FgqFLJ4KAAAAABAnk6+QMo9MGdndzlfXu3btUjgcjnUWAQAAAOCEdKKtGuokHA4rKyur3dVB161bp9GGUxcBAAAAAPEx+2OdSZMmaeXKldqyZUvssQ8//FBffPGFpkyZYvU0AAAAAIA4mY1b3nzzzXrjjTd0/fXXa/bs2Tp48KAWLVqkUaNGqaioyOppAAAAAMCbetGqoWYjgmlpaVq6dKlGjBihhQsX6le/+pUKCwu1aNEi+f1+q6cBAAAAAMTpa3VX//CHP7T7eG5urn7xi1/EVRAAAAAAoGt5aNyyta1N2Z1uI5gnaVOT814keXW77GqqGGSWK+dsuw3lfTV2e86UlbnbR7CszDlXk+EejsOHm6VSxHC77zvuMEvlau+/u+92GWe4758k3TrH7pzdeqtZKo0YYbdHUkODc0xYUm2T8zUSlt35zxjgojC3OrupHqdgjYubgEu1WSNdxbnahchyzzLDvdQykuzeiyQX+8+5lZpqFteYYrc6eMVnZqlcbdfnVprhBppu9jkOSqpPcd6tPFhValDRMSxPmuUmezk5ZqnCdU6fndIUbnL3+erASXbvRZFUu1xNNWapVJ0e/16oPp+UGn8p3pdoq4YCAAAAAHoPOoIAAAAAkGA8NG4JAAAAAL1YIq4aCgAAAADoHegIAgAAAECC8dC4JQAAAAD0YqwaCgAAAADwKjqCAAAAAJBgPDRuCQAAAAC9GKuGAgAAAAC8io4gAAAAACQYD41bAgAAAEAvxqqhAAAAAACvoiMIAAAAAAnGQ+OWAAAAANCLsWooAAAAAMCr6AgCAAAAQILx0LglAAAAAPRivWjV0KRoNBrtxlLcq6yUmps7Pp6ZKZWXO6apT800KylYtsksl2elpzvHpKVJ1dWOYbUpaQYF2aupscuVlWWXy6eIiyCfFHGOu3WO7WD/M0+7qM2lu++xq+2BB8xSyd9Q6xwUDku1LuKamuIvqIVlg01NNUvVGLK7vv0Vpc5B2dlSqYu4UCj+grog19Yyv1muQMAslTJT652DgkGp3kXcmjVx19OiPGe8Wa7MOsP3bsM20Zju/PnE75caG51zbdhgUNAx8kfb3fPrG+zu+cEUFyfDLaf7tNt2L+mTzUGDgg7LH7zLLNe6qkFmuYYPt8njt7sVetfUqdLOnT1dxVFDhkgrVrR7iKmhAAAAAJBgPDRuCQAAAAC9GKuGAgAAAAC8io4gAAAAACQYD41bAgAAAEAv1otWDWVEEAAAAAASDB1BAAAAAEgwHhq3BAAAAIBejFVDAQAAAABeRUcQAAAAABKMh8YtAQAAAKAXY9VQAAAAAIBX0REEAAAAgATjoXFLAAAAAOjFWDUUAAAAAOBVdAQBAAAAIMF4aNwSAAAAAHoxVg0FAAAAAHiVh7qrrdX0zVAk0vHxNEnVgUzHPGmBTpIcp08a8sxyjRhhlkr+j943y9WY4/w7+iU1htIc48JqNKjoiJoas1R1KYPMcvk2bzLLVZ/lfO6DQam+wfn7m1tvtajoqLvvsfvO6JGH7K7JGTPt6nrppbBjjE9SJOQmzu53VCBgl6ukxCyVXxVmuTR8uLu4wYOdYxoa4qvlGOs2+81yjQyVmuWKDM62y6WgY4xPUiTgIi41Nf6CjsgcbHgNfWbXJtzcp11rcg7x+6UmF3H5WdXx13OsBrv7TtBwZKS6zu6aTGuo6jwgGHT92SM/y66Nqc4u18gRdtfR1pL4329TUqRsu9sXDHi2IwgAAAAAvQqrhgIAAAAAvMpD3VUAAAAA6MVYLAYAAAAA4FV0BAEAAAAgwXho3BIAAAAAejEWiwEAAAAAeBUdQQAAAABIMB4atwQAAACAXoxVQwEAAAAAXkVHEAAAAAASjIfGLQEAAACgF2PVUAAAAACAV9ERBAAAAIAE46FxSwAAAADoxVg1FAAAAADgVXQEAQAAACDBeGjcEgAAAAB6sV60aqiHqmwtNdjoEOFXWsgpRiqv8NsUJCm/YZVZLlVlmaVqHDfBLFdVlXNMZqbLuNSm+AtqEQqZpcpsqjXLpdBgs1TBhmoXQWmu4kaMSDOo6KgHHrDLNWOm3USEV16KmOX65fPOdc2eLT3/vHOucePsfsecnKBZrqDlG1N6ul2ud991jpk0yVVc44WT4i6nK0Syss1y+T77xCxXbU6+Y0w4LNXVOecKG9QTs3mzXS7D94+g6s1yvf+x87U9YYL08cfOuSaMNijoGNUNdvedtFS7+3RayWqzXPUjzur0eFBSfWqmq1yWt1a/YXtVQ4NZqtycgFEmJiN6Ca8GAAAAACQYz44IAgAAAECvwqqhAAAAAACvoiMIAAAAAAnGQ+OWAAAAANCL9aJVQxkRBAAAAIAEQ0cQAAAAABKMh8YtAQAAAKAXY9VQAAAAAIBX0REEAAAAgATjoXFLAAAAAOjFWDUUAAAAAOBVdAQBAAAAIMF4aNwSAAAAAHoxVg0FAAAAAHgVHUEAAAAASDAeGrcEAAAAgF6MVUMBAAAAAF6VFI1Goz1dRHvef19qaOj4+KRJ0ttvO+cZN86upnBNqVmuT2qyzXLlp9rVVSrnurKzpVIXT5mdUm5Q0RHp6Xa5KirMUtWn272OQdW7CApK9c5xtU1Bg4qOCqvWLFckFDbL9fzzZqk0+/qIc5DPJ0Wc426/w+47tieeMEtlqqbGLlda0y7noEGDpF0u4pqa4i+oheV9x5Lh77iuxPleMXKktG6dc66cnPjraWHZvjpZJ+G4ZfzFxQcPt9x8QAmHpVoX91/r0QfDF6A6kGmWK63B8HNFamrnx12+30rS6g1277lnhTaZ5VrXlGeWKysr/hxJSdKAAfHn8bwnn7S9icUrNVW6/Xbt3LlTzc3NrQ55aNwSAAAAAHoxj64aOmvWLO3YsaPVIQ9VCQAAAACwVlxczIggAAAAACSSIUOGtHnsa3UE7733XpWUlOiFF15o9fgVV1yhTz/9tE385MmTtXDhwq/zVAAAAADQO/SiVUOPu8pXX31Vr7zyigoKClo9Ho1GtWXLFhUWFmrSpEmtjg0dOvR4nwYAAAAA0EVcdwSbm5v1zDPP6Kmnnmr3eFlZmerr63XRRRepqKjIrEAAAAAAgC1XHcGDBw9q+vTp2rhxo6ZNm6aVK1e2idm8ebMkadiwYbYVAgAAAEBv4NFVQ9vjarOrgwcPqq6uTgsWLNBjjz2mlHZ+uc8//1zS0Y5gvcu9VwAAAAAA3ctVdzUUCuntt99utwPY4vPPP1f//v01f/58/eY3v1F9fb3++q//WrfddpumTp1qVjAAAAAAID5J0Wg0erw/NHHiRA0dOrTVqqGXX3651q1bp8mTJ+vSSy9VbW2tlixZog0bNuixxx7TtGnTLOsGAAAAAG9ZtEiqre3pKo4Kh6Wbbmr3kNkE1hkzZigSiWjWrFmxx6ZOnapLLrlEjz/+uC699FIldzJH9avef19qaOj4+KRJ0ttvO+cZN871UzoK15Sa5fqkJtssV36qXV2lcq4rO1sqdfGU2SnlBhUdkZ5ul6uiwixVfbrd6xiUi+nUwaDkYtp1bVPQoKKjwrK7oUVCYbNczz9vlkqzr484B/l8UsQ57vY7XM26d+WJJ8xSmaqpscuV1rTLOWjQIGmXi7impvgLamF537Fk+DuuK3G+V4wcKa1b55wrJyf+elpYtq/j+OjhKOMvLj54uOXmA0o47O4DpfXfIxm+ANWBTLNcaQ2GnytSUzs/7vL9VpJWb7B7zz0rtMks17qmPLNcWVnx50hKkgYMiD8P7Jh9WrnqqqtadQIlKRAIqKioSFVVVbHFZAAAAAAAPavLl7RJS0uTxOIxAAAAAE5wJ9qqoU4qKys1derUdvcY/OKLLyRJWRZjygAAAACAuJl0BDMyMlRbW6tXX31VdXV1scfLy8u1fPlynXvuuRo4cKDFUwEAAAAA4mQ2bnn//fdrzpw5mjlzpqZPn679+/eruLhYKSkpuv/++62eBgAAAAC8KSXFW1NDO6nFbLGYwsJCPf300+rXr5+eeOIJPffccxozZoxefPHF2CbzAAAAAICe97W6q3/4wx/afbywsFCFhYVxFQQAAAAA6FoeGrdsbdgwqbm585gRI5zzlJSYlCNJ+uwzuz3jrp7pYs8ylyIu9v5zK7vKxT5dGqTsgIu4zzbEXU+LT9InmuXKH223cFGN3ZaECgx23ofIJykScI4Ly659SZJq7PYt8xnWNm6c3X59bvb+e/JJl3FP2P2O/zjX7nd86CGzVAqF7HKpyWUyF0/q5vpwy3LXo7ycRrtkhty8j7qN81XY7fEW7Gwj4eMVCNjlstyc2M3vGA67izO9IGW6V6VtaXavZXVD5/eKtKBzTIuzcqotSjqszu53HCm7faarm+L/rOmzezvztkRbNRQAAAAA0HvQEQQAAACABOOhcUsAAAAA6MUScdVQAAAAAEDvQEcQAAAAABKMh8YtAQAAAKAXY9VQAAAAAIBX0REEAAAAgATjoXFLAAAAAOjFWDUUAAAAAOBVdAQBAAAAIMF4aNwSAAAAAHoxVg0FAAAAAHgVHUEAAAAASDAeGrcEAAAAgF6MVUMBAAAAAF5FRxAAAAAAEoyHxi0BAAAAoBfrRauGJkWj0Wg3luJeaanU1NTx8dxcaetW5zw5OWYlqaHBLFUkEDTLVVdnlkqhkHOMzydFIs5xlnV11hSOl5euzWMdOOAck5EhVVa6iBtQH39Bx6qosMs1eLBZqnrZXUeBgHOM27Y/d27c5cQs/KmLJ3Tp21PtJoG88YZZKvnLXNzLXd7zK/vnGlR0WEafarNckdQ0s1zPP2+WSrMv2eUcNGiQtMs5bmvdIIOKDtu92yyVBgywyzVysF2bqA84t4lgUKp3cTsP1pQbVHQMNx8G3Copscs1fLhZKqf3D7fnXpJqauKvp0Wm7F7LyOBMs1y+NavjT+L3S6NHx5/H6/7rv9x9qOsu/fpJ3/pWu4eYGgoAAAAACcajYyMAAAAA0MuwaigAAAAAwKvoCAIAAABAgvHQuCUAAAAA9GK9aNVQRgQBAAAAIMHQEQQAAACABOOhcUsAAAAA6MVYNRQAAAAA4FV0BAEAAAAgwXho3BIAAAAAejFWDQUAAAAAeBUdQQAAAABIMB4atwQAAACAXiwlRWpu7ukqjmLVUAAAAABACzqCAAAAAJBgmBoKAAAAABZYNRQAAAAA4FVJ0Wg02tNFtGffPqmzysJhqbbWOU840GhWU6P8Zrks1dXZ5aqpcY7JzZW2bnURl1Iadz0x6elmqWqbgma5qqrMUmnwYOeYYFCqr3cR1+Ti4jgeTU12uSoq7HIZfuMWGZ7nGOPzSZGIcy7La/LKK+1y/XaFi+JdWrLU7nvE6wrLnYMyM6VyF3GbN8df0BGfpE4wy5UfcnHTdKk6NdcsV1qdi/t0drZUang/dyM11S6X4f2r8lCaWa4vv3SOGTlSWrfOOS4nJ+5yWrG85Vt+Dnv7XbvPYZPGObxPuv2gKZl+GKjsb3d9ZyTtMsv1ScWguHP06SOdfrpBMV63Zo3UaNfu4+b3S2PGtHvIQ+OWAAAAANCLpaS4+9a4u7BqKAAAAACgBR1BAAAAAEgwTA0FAAAAAAusGgoAAAAA8Co6ggAAAACQYDw0bgkAAAAAvVhKSud74HU3Vg0FAAAAALSgIwgAAAAACYapoQAAAABgITnZW1NDWTUUAAAAANCCjiAAAAAAJBimhgIAAACABS9tJi+xaigAAAAA4CiPdVkBAAAAoJfqZHGWHsFiMQAAAACAFnQEAQAAACDBMDUUAAAAACykpEhJST1dxVGdTA31bEdwwLo/SQcPdhwwYYLCa953zFOaM8GsptRUs1QKhyJmudJCTXa5UhpcRIWVm17rHBYYHHc9LUor/Ga5srPszn3443fNcpUHJjrGBINSTY1zrmBNWfwFHaNx+EizXH5VmOVSerpZKjfnNS3NXVwoFG81R73xhl2uJUvtJoFcd43ddXT3PZmOMY88It39lHPczJnOMW412d1apawss1R1hpfQms3ZjjETs6U/uIlLXW1RkiSpsq/z87nVp49ZKmX0qbbLlerm/TZTI1PLHaMiAbt2L0kbNtjlGplaZZZr3DjD39PNBe7yJlDZPzfOYo7K+HKdWa7ICLv37lBd/Dm8tpgmmBoKAAAAAAmHvjkAAAAAWEhO9tbUUF/H436MCAIAAABAgqEjCAAAAAAJhqmhAAAAAGAhJUWK2C2oFjemhgIAAAAAWtARBAAAAIAEw9RQAAAAALCQnNzpdMxu18kKph6qEgAAAADQHegIAgAAAECCYWooAAAAAFhISZGi0Z6u4iimhgIAAAAAWtARBAAAAIAEw9RQAAAAALCQnNzTFbjGiCAAAAAAJBg6ggAAAACQYJgaCgAAAAAWUnpP94oRQQAAAABIMEnRqJc2ujiqslJqbu74eGamVF7unCezqdSuqJoas1Tl6flmuVJTzVKprMw5Ji9P2rTJOS4nJ+5yYvwpEbNcpWV2338YNgmlpzvHuG33oVD89RwrXGN4HQ0ebJfr3Xftco0Z4xwzaJC0a5dznOULUFFhlysQMEt191OZZrkeecjF9e3zSRHnuPsesLu+H7x+q1ku0xt1XZ1dLjdtNS1Nqq52DIukphkUdJivod4sV+W+oFmujJMbzXJpzRrnmIICadUqx7DfVBXEX88xLrzQLlewyu79ozqUbZbLqen7/VKjy5fbX+XijdmtqiqzVJUZdp81zdq+32+Tx8tcvFd1O59PO3fuVPNXOle9Z+wSAAAAADws4sEJlz5Js2bN0o4dO1o9TkcQAAAAAE5gxcXFjAgCAAAAQCIZMmRIm8dcdwQ/+OADPfPMM1q7dq18Pp/OPPNMzZ07V2OO+bua7du367HHHtOqI/PZL7zwQs2bN09paXZ/NwAAAAAAXtTU1NMVtNXRn2a66giuWrVKN998s77xjW/otttuU1NTk5YtW6ZrrrlGy5YtU35+vvbs2aPvfe97amxs1E033aTm5mYtXrxYGzdu1Kuvvip/IvxxKAAAAAD0Aq46go888oiGDBmiV155Rf369ZMkTZs2TVOmTNGCBQv03HPP6fnnn1dFRYV+/etfa9iwYZKkM888UzfccINef/11zZgxo+t+CwAAAACAa47L2uzdu1cbNmzQxRdfHOsESlJ6errOOecc/fnPf5YkrVixQgUFBbFOoCSNHz9ep556qlasWNEFpQMAAACAdzQ3H54e6pV/nW3H5zgiGAqF9Lvf/a5VJ7DFnj17lJycrL1792r79u2aPHlym5hRo0bpvffeO74zCAAAAADoMo4jgsnJycrJyVFGRkarxzds2KDVq1dr7NixqqyslKQ2MZI0cOBA7du3T/v27TMqGQAAAAAQj6RoNBo93h/av3+/rr76am3cuFFLlixRnz59NHPmTD300EOaPn16q9gFCxbo5z//ud5///12O4oAAAAAcCLYt086/t5V10lKkgYMaP/Yce8jeODAAd16663asGGDbrnlFhUUFGj16tUuikg6rueprOx8TmtmplRe7pwns6n0uJ63UzU1ZqnK0/PNcqWmmqVSWZlzTF6etGmTc1xOTtzlxPhTIma5SsscB8JdM2wSSk93jnHb7kOh+Os5VrjG8DoaPNgu17vv2uU6ZiucDg0aJO3a5Rxn+QJUVNjlCgTMUt39VKZZrkcecnF9+3xSxDnuvgfsru8Hr99qlsv0Rl1XZ5fLTVtNS5Oqqx3DIql2W0X5GurNclXuC5rlyji50SyX1qxxjikokI5sydWZ31QVxF/PMS680C5XsMru/aM6lG2Wy6np+/1So8uX21/l4o3Zraoqs1SVGXafNc3aPrsIeMpxvWPW1tZq9uzZ+tOf/qTvfve7uu222yRJweDhm+zBgwfb/EzLYyHrT6YAAAAAgK/F9Yjgl19+qRtvvFHr16/XlVdeqX/5l3+JjfJlZh7+Znj37t1tfm7Xrl0Kh8OxziIAAAAAnIiam11NYOk2vk6G/Vx1BOvq6mKdwOuvv1533XVXq+PhcFhZWVlau3Ztm59dt26dRo8efXwVAwAAAAC6jKupoQ8++KDWr1+v6667rk0nsMWkSZO0cuVKbdmyJfbYhx9+qC+++EJTpkyxqRYAAAAAEDfHEcEtW7bojTfeUDgc1umnn6433nijTUxRUZFuvvlmvfHGG7r++us1e/ZsHTx4UIsWLdKoUaNUVFTUJcUDAAAAgFc0NZ1AU0NXHVmtqra2tsPRwKKiIqWlpWnp0qWaP3++Fi5cqEAgoMLCQt15553ys0IQAAAAAHiGY0fwqquu0lVXXeUqWW5urn7xi1/EXRQAAAAAoOt8rQ3lu8WBA53vxhgMSvUu9hlKOe6tEjtmuLeL6T5jhnuDudqfyuWeUqa/48cfm6WqHjHeLFdak4s95dxy8zqGw1JtrXOc5T5jkm0bM7wmGwNhs1yu9oFyuZFjZLDdHnvtLMb8tWVsfN8s1yepE8xyvfaac8yDD0r33eci7gG7+Th3zrPbk/DRR81S2baJyk+cg/LzpU9cxA0fHn9BR2wqs1tpPG/Dm2a5ZLj4XW16rmOM21t+uGydQUXHyMoyS7Wpwu4+nZdq+J7b0ND58exsqdTlHogefY90/B2Ph8Vnus52Nj+BOO2F3t2Sk6WMjPaP2b3LAQAAAAB6BTqCAAAAAJBgDMefAQAAACBxNTV5a2poZ39px4ggAAAAACQYOoIAAAAAkGCYGgoAAAAABpqbD08P7Q0YEQQAAACABENHEAAAAAASDFNDAQAAAMBAUxNTQwEAAAAAHkVHEAAAAAASDFNDAQAAAMCA11YNTUrq+BgjggAAAACQYOgIAgAAAECCYWooAAAAABjw2qqhTA0FAAAAAMTQEQQAAACABMPUUAAAAAAw4LVVQ32dDPslRaPRaPeV4t6+fVJnlYXDUm2tc55w1Va7ogIBs1S1oUyzXGG5OBEu1Srs/Hxuz31KvUFFR9TVmaWKpA8yy+VrajTLpZIS55i8PGnTJuc4w7YqSRo82CzVus1+s1yWRg538Vr6/VKjc9ymErvfMS+92izXJ2VpZrks3+TOSnVxn87NlbY6x93581yDig778aMRs1y3zrGbgPPDH5qlUk6Oc4zPJ0VcnIrdu+MuJ6ZfP7tc4YDdfTqSYndtu3r/cHnP0YYN8RfUVUaPNkv14Ud219H4cQ6N2m3Dl6SamrjriamoMEvVOHykWS5/k8FnuqQk24vboz77zN1l2138/o4vQ6aGAgAAAECCYWooAAAAABjw2qqhnU0NZUQQAAAAABIMHUEAAAAASDBMDQUAAAAAA15bNTQ5ueNjjAgCAAAAQIKhIwgAAAAACYapoQAAAABgwGurhjI1FAAAAAAQQ0cQAAAAABIMU0MBAAAAwIDXVg1N6aS3x4ggAAAAACQYOoIAAAAAkGCYGgoAAAAABry2amhntTAiCAAAAAAJho4gAAAAACQYpoYCAAAAgAGvrRra3NzxMUYEAQAAACDBeHZEsG9f55hAwEWisrK4a2lRO2aCWa6was1yqabGLpXCjjHhsLunDKsq/oJapKebpaqrM0ulUMhvlmv3SXmOMRmSKt3EJe0yqOiorWV2v+fIUKlZrkhWtlkuGX57l5fTaJYrkpJmliu/ZqtZLmVl2eWqS3UXl+oc9+ijcVXSyq1z7L4rfebpiFmub0+1q+vZZ51jsrPdvZVmD7Zr95Y36so9dtdQZaVZKqWnO99XMzOl8ioXcaGQRUkx75flmuWaUGf3eWfYMOfPKD2hMWTXxlJG2OXyV5Sb5TIZ4kpOloYOjT8PzHi2IwgAAAAAvQmrhgIAAAAAPIuOIAAAAAAkGKaGAgAAAIABVg0FAAAAAHgWHUEAAAAASDBMDQUAAAAAA6waCgAAAADwLDqCAAAAAJBgmBoKAAAAAAZYNRQAAAAA4Fl0BAEAAAAgwTA1FAAAAAAMsGooAAAAAMCz6AgCAAAAQIJhaigAAAAAGGDVUAAAAACAZ9ERBAAAAIAE49mpof51a6TGxo4DCgrkX7PKOdGYMVYlKRyKmOWKKGyW649r7HKluGgR2dlSWZlzXNa47PgLOmL3brNUyuhTbZeszu4SGjjQ3es4cKCbqPS4avmqgOEUh8hgu3bh++wTs1waPtw5xu/v9vkezy/1m+WaNi3XLFddhVkqZavOOSgtTapzjtt9KM2gosN++EOzVPr2VLvvXX+7wu696O57nOt65BHp5z93zvXQQ3Zt1ZeaapZrQINZKmUMtDv31TXu2kQg4BzTmG53bUvSGMO3kMoDdp9RBgwwSyXV1HR+PC3NOaZFyO6+46soN8v1SVWmWa78wbviT+JLjPEnVg0FAAAAAHgWHUEAAAAASDCenRoKAAAAAL0Jq4YCAAAAADyLjiAAAAAAJBimhgIAAACAAVYNBQAAAAB4Fh1BAAAAAEgwTA0FAAAAAAOsGgoAAAAA8CxGBAEAAADAAIvFAAAAAAA8i44gAAAAACQYpoYCAAAAgAEWiwEAAAAAeBYdQQAAAABIMEwNBQAAAAADrBoKAAAAAPAsz44IHjhtjKLRjo8HJdWPLnDMs2aNWUkaP7rWLlkobJbq/PPNUrk+X4GAc4xPkbhqOdaf/mT3ncVlF4fMcll+5eOrKHcOysx0F5eaGnc9rZ7WMF1EQbNctTn5ZrnKSpxjRo6U1pU41z9iRPz1tJh9yS67ZHUNZqnWbM42y5U9xuU1GXKOyyj7JM5qjoqMtmtfzz5rlkp332N3P3zkITf3aZ+ruOWv29VVWGiXy/abebu60gL1LqKC7uI2bI67nmP5Bw+2S9ZvkFmqYJ3h/dANl43HX+XifdmlTXWZZrlMPwpUVMSfo08fKT09/jz4Wnbu3Knmr6wc49mOIAAAAAD0Jl5dNXTWrFnasWNHq2N0BAEAAADgBFZcXMyIIAAAAAAkkiFDhrR5zHVH8IMPPtAzzzyjtWvXyufz6cwzz9TcuXM1ZsyYWMwVV1yhTz/9tM3PTp48WQsXLvx6VQMAAABAL9CbVg111RFctWqVbr75Zn3jG9/QbbfdpqamJi1btkzXXHONli1bpvz8fEWjUW3ZskWFhYWaNGlSq58fOnRoXL8AAAAAAMCOq47gI488oiFDhuiVV15Rv379JEnTpk3TlClTtGDBAj333HMqKytTfX29LrroIhUVFXVp0QAAAACAr8+xI7h3715t2LBBN9xwQ6wTKEnp6ek655xz9L//+7+SpM2bDy9dPGzYsC4qFQAAAAC8y6urhrbHsSMYCoX0u9/9rlUnsMWePXuUnJwsSfr8888lHe0I1tfXKxi02y8MAAAAAGDDcWfU5ORk5eTkKCMjo9XjGzZs0OrVqzV27FhJhzuC/fv31/z58zV27FiNHTtWhYWFWrFiRddUDgAAAAD4WpKi0Wj0eH9o//79uvrqq7Vx40YtWbJEBQUFuvzyy7Vu3TpNnjxZl156qWpra7VkyRJt2LBBjz32mKZNm9YF5QMAAACAN/zrv0p79vR0FUedfLJ0773tHzvufQQPHDigW2+9VRs2bNAtt9yigoICSdKMGTMUiUQ0a9asWOzUqVN1ySWX6PHHH9ell14am0bq7nmkzrqowaBUX++cZ80a10/paPzoWrNckVDYLJclN+frrLOk1atdxI2JxF1Pizffchy8du2yixvNcplOAq+pcY7JzJTKy53jUlPjrabLRAJ2U8br6sxSqazMOWbkSGndOue4ESPir6eFr2qXXbKGBrNUf9icbZZr4phq56C0NKnaRZybF9KlyOh8s1yGZennP7fL9chDLu7TPp8UcY5b/rrdfbqw0CyV6W06ELDLFZSLDzFuP+wcWafBzODBZqlqA4PMcoUbDO+HTgYNkna5fD7DRrapLtMsl2V7za75JP4kffpIp58efx6YOa67dm1trWbPnq0//elP+u53v6vbbrstduyqq65q1QmUpEAgoKKiIlVVVcUWkwEAAAAA9CzXI4JffvmlbrzxRq1fv15XXnml/uVf/kVJSUmOP5eWlibp8OIxAAAAAHCi6k2rhroaEayrq4t1Aq+//no9+OCDrTqBlZWVmjp1qp566qk2P/vFF19IkrKyso6zbAAAAABAV3DVEXzwwQe1fv16XXfddbrrrrvaHM/IyFBtba1effVV1R3zRzvl5eVavny5zj33XA0cONCuagAAAADA1+Y4NXTLli164403FA6Hdfrpp+uNN95oE1NUVKT7779fc+bM0cyZMzV9+nTt379fxcXFSklJ0f33398lxQMAAACAVzQ1eWtqaGe1OHYEV61aJenwQjHtjQZKhzuChYWFevrpp/Xss8/qiSeeUCAQUEFBgW6//fbYJvMAAAAAgJ7n2BG86qqrdNVVV7lKVlhYqELLNZ8BAAAAAOaOex/B7tKvr9OeRT4FA877Go0fUWNSjyStK0szyzVyuDf3sktPd7fHW3q6iyDDui670G7/s/omuz0cGxr8ZrnSVGOWqzHFbq8+SfJ//KFZLp/hHoeWu3Hm5Ix0Gecc46twsdejS1sb7PaUyk0pNcs1MdXFZqIuRVLPcozxSYqkOt+DfYYbZ+3ebZZK2YPt7vkPPWR333Gz9993vuMybprd3rEPPmS3J+F9V7jY/NOl6sHu7hNuBBtqXAQF3e0xa7l5qaTSCrs2FjL8pFnaYLcnYXbAm3u0Wm5Efu7pdvtf/2Fz/PuqBgLSeINavO6EWzUUAAAAAHDioCMIAAAAAAnGs1NDAQAAAKA36U2rhjIiCAAAAAAJho4gAAAAACQYpoYCAAAAgAFWDQUAAAAAeBYdQQAAAABIMEwNBQAAAAADrBoKAAAAAPAsOoIAAAAAkGCYGgoAAAAABlg1FAAAAADgWXQEAQAAACDBMDUUAAAAAAywaigAAAAAwLPoCAIAAABAgmFqKAAAAAAY6E2rhp74HcFAoKcraNe6zX7DbHa5Bg92FxcKOcc0Gtblb6ozyxVsqrbLVVFhlqs8daRjTKakcmU6xlV8ZlDQMQbnjDfLlTk4YpZLmzebpaqpcY4JBl3GNTTEW07M7t1mqZR7eqpZrsq+2Wa5MhrqnYOCQflcxG0qCxpUdJjb+6ErdXb3MF9qqlmuwkJ3E4MKC51jHnzIbpLRfffY3ScmXOh8b3Xr/bdqzXIpPd0srrbB8jOFu/d4t9Jk956bVrbBLJfGjXOOcfkaVacMirOYo84NlJvlqjzg/HnBrVGj4s/hYx6i5/CSAAAAAECCOfFHBAEAAACgG7BqKAAAAADAs+gIAgAAAECCYWooAAAAABjoTauGMiIIAAAAAAmGjiAAAAAAJBimhgIAAACAAVYNBQAAAAB4Fh1BAAAAAEgwTA0FAAAAAAOsGgoAAAAA8Cw6ggAAAACQYJgaCgAAAAAGWDUUAAAAAOBZdAQBAAAAIMEwNRQAAAAADLBqKAAAAADAszw7Irhjp6/THmx2tlRa5tyPzd78kVlNIy6caJbLp4hZLku1de6+G0hx0XL27ImzmGMcPJhmlquhwSyV8nICZrkCLusKuHjK9PT4avmqzLpNdsk+M3wBQiGzVMnJhnFuXiSXBgwwS2X6FWWfPmapVLkv6BiTEXQXl7fhTYuSDsu52CxV5R67e9gAw0vIbZNwE3ffFeviK+YYEy4caZbr/Xft3m/ffCtsluuSS5xjfJIiKX7HOOvRB8NbmGqb7Np+aNx4s1w1NZ0fT0uTqmvcfSaqqoq/ntjzDrZ7X8to2GWWq7xpkFkueIdnO4IAAAAA0JuwaigAAAAAwLPoCAIAAABAgmFqKAAAAAAYYNVQAAAAAIBn0REEAAAAgATD1FAAAAAAMMCqoQAAAAAAz6IjCAAAAAAJhqmhAAAAAGCAVUMBAAAAAJ5FRxAAAAAAEgxTQwEAAADAAKuGAgAAAAA8i44gAAAAACQYpoYCAAAAgAFWDQUAAAAAeBYdQQAAAABIMJ6dGjpwoHPM4MHOMZsaJsZfzBF5ipjlUkODWar3Pw6a5UpNdY7Jz5dKSlzEDa+Pt5yj+hmOsdfV2eUqqTFLlRYIuAjKVVrNVuewmpr4CzqWm4vNpfqsPLNcQdm1sYw/vu0cNGmSMv7iIm7cuPgLOmJkoNosV+WhNLNcGX3s6tLJIRdBfmWc3OgcNnp03OW0iKT4zXJVVpqlUsZAw/cil98Hu7k9VQ8eGWctR73/Vq1ZrjffCpvluuwSu3N/6xznc//MM9KcOc657r3XoKBjpBh+OrR8OwpXbDLLFXDxXuSm3UtSXqg8zmqOioQyzXIpZNf2M5tc3H9dsbuvehWrhgIAAAAAPIuOIAAAAAAkGM9ODQUAAACA3oRVQwEAAAAAnkVHEAAAAAASDFNDAQAAAMAAq4YCAAAAADyLjiAAAAAAJBimhgIAAACAAVYNBQAAAAB4Fh1BAAAAAEgwTA0FAAAAAAOsGgoAAAAA8Cw6ggAAAACQYJgaCgAAAAAGWDUUAAAAAOBZdAQBAAAAIMEwNRQAAAAADPSmVUOTotFotPtKca+xsfPjfr9zjCT5UyI2BUlSQ4NdrhS7Pnh5ld8sV3Kyc0xGhlRZ6Rw3cGD89bTYvdsuV0ZzuVmucmWa5coM1ToHhcNSrXNcfUrYoKKjDJur6c3x44/tck0YY3f+Le8V9aFBZrlKSsxSaWSq3XWksjLnmIICadUqx7DaEQUGBR0WDrh4k3HJ8j4dCJilUlqg3jkoGJTqXcTV1MRdT0x6ulmqSIrduZ8zxyyVnnnaxecTn0+KOMddfY3tBK+nnrLLlda0yyxXdYrd/dDpOnLb7CWpqir+eloYNn0FK7aa5arsnxt3Dp/P9rOhV516qrRtW09XcdQpp0hffCHt3LlTzV/5g0FGBAEAAADgBDZr1izt2LGj1WN0BAEAAADAQDQakZfmWx6uxafi4mJGBAEAAAAgkQwZMqTNY64nla9cuVJXXXWVxo4dq7/7u7/Tww8/rP3797eK2b59u77//e+roKBABQUFuvPOO1VdXR1/5QAAAAAAM65GBFeuXKnZs2dr1KhRuuOOO7Rz504tWbJEn332mYqLi+Xz+bRnzx5973vfU2Njo2666SY1Nzdr8eLF2rhxo1599VX5/XZ/rA0AAAAA3tPJDu49pv2xP1cdwccff1xDhgzR0qVLFTiyzNKQIUP04IMP6oMPPtAFF1yg559/XhUVFfr1r3+tYcOGSZLOPPNM3XDDDXr99dc1Y8YMo18EAAAAABAPx6mhBw8e1Mknn6wZM2bEOoGSVFBweInujRs3SpJWrFihgoKCWCdQksaPH69TTz1VK1assK4bAAAAADwmosOjgl751/EWNI4jgn379tXixYvbPL5+/XpJUmZmpvbu3avt27dr8uTJbeJGjRql9957z+lpAAAAAADd5LhXDd2xY4f+9Kc/6bHHHlNeXp6+9a1vaduRXRMzMjLaxA8cOFD79u3Tvn37NGDAgPgrBgAAAADE5bg6gjU1NZo4caIkqV+/frrnnnvUt2/f2Oqh/fr1a/Mzffv2lSTV19fTEQQAAABwAmuW5KGNBJXU4ZHj6ggmJSVpwYIFamxs1AsvvKAbbrhBCxYs0MCBA1397PFws8iou4VIXe+Q4SwYtMtlKDOz+5+zncHfXvR8difM9tSHXYY5x3mzpR5muYDwhAl2uSzPv6sYlyxfy5EjDZNZtn63N7Ejf5veGbszL0l2jbUn7tPuuGxhbt7/PPoeafgpQM88Y5jMbWU+57hly+IspUsNMsuUZpbJHbdNOju7a+v42nJzzVJ188c+dJPj6giedNJJmjJliiTp4osv1iWXXKL58+fr5z//uaTDC8t8VctjoVDouAprbOz8uN/vHCNJ/pSO/0DyuDU02OVKOe5ZuR0qr7L7sJKc7ByTkSFVVjrHufh+wLXdu+1yZTSXm+Uqt+xUhmqdg8JhqdY5rj7F9uOwYXNVU5Ndro8/tss1YYzd+be8V9SH7D5ElZSYpdLIVLvrSGVlzjEFBdKqVY5htSOcO4tuhQMu3mRcsrxPH7NuW9zSAvXOQcGgVO8irqYm7npi0tPNUkVS7M79nDlmqfTM0y4+n/h8UsQ57uprLLu70lNP2eVKa9pllqs6xe5+6HQduW32klRVFX89LQybvoIVW81yVfaPv1Pp89l+NkT8vvadIxAI6MILL9TOnTs1aNDhC3N3O5/Wd+3apXA4rKBHvykEAAAAABsRD/5rn2NHcMuWLZo4caKKi4vbHNu/f7+SkpLk9/uVlZWltWvXtolZt26dRo8e7fQ0AAAAAIBu4tgRPOWUU7Rv3z699NJLajxmLuaOHTv0+9//Xuecc45CoZAmTZqklStXasuWLbGYDz/8UF988UVsOikAAAAAoOc5/uVPSkqK7rnnHt1555269tprddlll2nPnj0qLi6Wz+fTvffeK0m6+eab9cYbb+j666/X7NmzdfDgQS1atEijRo1SUVFRl/8iAAAAANCzOp+O2f06HvdztQREUVGR+vTpo0WLFmn+/PkKBoMaN26cbrvtNp166qmSpLS0NC1dulTz58/XwoULFQgEVFhYqDvvvFN+y2UCAQAAAABxcb0W4JQpUxyneObm5uoXv/hF3EUBAAAAALqO4aLwAAAAAJDImuWtqaEdb27v2Y6gf0+l1NzccUBmpvxVzvtYlTbZ7fOW3eBiryuX1jXlmeUaGbDbJ0apqS6C0pTRp9oxqrbObutXy31nSssM20TAbn8kpbjca9PFpn7BqtI4i2ntkxq73XLzs5zbjlsTLBckTnG5OZubTRWPc9/UzgRr7Pbry8mxa/uRgF2u361xzjVF0m+qnPcInFK2zqCiIww3vcw0bBON6XabRGvDZueY/Hxps4u4ESPir+eI2ga7Pymx3Lv0yLIIJtzs/bdsmcu4pbYfOu+cZ7cv4U032e39l5Vllsr53hrMdH3/zZZdI4sEvLk7fcY2531cHfn90sAx8eeBGdsdSAEAAAAAnufZEUEAAAAA6F16z9RQRgQBAAAAIMHQEQQAAACABMPUUAAAAAAwEdHh6aHex4ggAAAAACQYOoIAAAAAkGCYGgoAAAAAJiLy1qqhSR0eYUQQAAAAABIMHUEAAAAASDBMDQUAAAAAE81i1VAAAAAAgCfREQQAAACABMPUUAAAAAAw4bUN5Vk1FAAAAABwBB1BAAAAAEgwTA0FAAAAABNeWzWUqaEAAAAAgCOSotFotKeLaE9jY+fH/X7nGEnyl221KUiSUgwHUA1z1YYyzXKFQs4xPp8UiTjHbd4cfz0t8lIMX8esLLtcH31kl+v8851j3J78hob46zlWIGCXy7C26oagWa60hnLnoMxMqdxFXFNT/AW1SE01S1WrsFmusjKzVMrJcY4JBqX6ehdxTbVx1xNTUmKW6v2afLNcY8aYpVK4YZdz0KBB0i7nuNKGQQYVHebmvcgty9uX5aXtJldamlRd7Rz36KPx13OsHz/q4n3GpX+cazfmsPAhw+vb6f170iTp7bddpaodN8mgoMOqqsxSKXewi5umS6VV8b/fJidLQ4caFONxOTkl2rbN8GYRp1NOSVFJSU67x5gaCgAAAAAmmBoKAAAAAPAoOoIAAAAAkGCYGgoAAAAAJqI6vKm8V3S8HAwjggAAAACQYOgIAgAAAECCYWooAAAAAJjw2qqhHY/7MSIIAAAAAAmGjiAAAAAAJBimhgIAAACACaaGAgAAAAA8io4gAAAAACQYpoYCAAAAgImIvDU1NLnDI4wIAgAAAECCoSMIAAAAAAmGqaEAAAAAYMJrq4Z2XAsjggAAAACQYOgIAgAAAECCSYpGo9GeLqJdlZVScyfDqpmZUnm5Y5rI4Eyzkj76yCyVxg/eapcsK8suV0WFc0x2tlRa6hjWODjboKDD9uwxS6WNG+1yjR5tlystpdY5KByWal3EuXkdj0N9Vp5ZrmBKo1kupdjNbq+ucf5eLC1Nqq52zhUKGRR0hH/DJ3bJRoywy1VVZZerqck5xuV9Z1OD3X0nb3jELJfq6sxSVR4Im+Xq1885xu1tx83L6FaaXFxoLtWmpJnlqqkxS6XswC7noEGDpF3OcZtqBhlUdNRTT9nlWvhTu+voyZ/ajV/MnNn5cZcfMyVJgUD89bRIC9m9R24q8ZvlyqtZFX8Sv18aMyb+PB6Xk/Nnbdtm+FknTqec4ldJydh2jzEiCAAAAAAJho4gAAAAACQYVg0FAAAAABNe21C+4+nZjAgCAAAAQIKhIwgAAAAACYapoQAAAABggg3lAQAAAAAeRUcQAAAAABIMU0MBAAAAwASrhgIAAAAAPIqOIAAAAAAkGKaGAgAAAIAJVg0FAAAAAHgUHUEAAAAASDBMDQUAAAAAExF1tlJn92PVUAAAAADAEXQEAQAAACDBMDUUAAAAAEz0ng3lk6LRaLQbK3GtpkaKdDK9Ni1Nqq52l8dKVpZdroYGu1wVFXa5TjrJOSYjQ6qsdI5rNrwGQiFv5vJ9vMoumZsGlpkplZc7x6Wnx19PF6mu85vlSitZbZZLgwc7x7g9/4FA/PV0Qa63/xg0yzVunFkqNTU5x7i956c17Yq/oCM+3DzILNewYWapNGCAXa5gnYvzNWiQtMs5rrTB7nxll31olisybrxZLt/mTWa5qtPzHGPctnvLW44kBZtqzXI9uShsluv2uXZ/d3Xd9Z1PiluyRLruOne5HnrIoKAjPvvMLteUcS4aj1sWH1yTkw9/iDzB5eS8p23bDvR0GTGnnNJPJSUXtHuMqaEAAAAAkGCYGgoAAAAAJthQHgAAAADgUXQEAQAAACDBMDUUAAAAAEwwNRQAAAAA4FF0BAEAAAAgwTA1FAAAAABMRNXZJu7dr+Mt4xkRBAAAAIAEQ0cQAAAAABIMU0MBAAAAwASrhgIAAAAAPIqOIAAAAAAkGKaGAgAAAIAJpoYCAAAAADyKjiAAAAAAJBimhgIAAACAiYi8NTW0483t6QgCAAAAwAls586dam5u3UFNikaj0R6qp3Nr1kiNjR0fLyiQVq1yzpNi2NfNyrLLlZ5ul+vjj81SVQ8vcIxJS5Oqq51zpb3zikFFh0WumGGWq67OLJXCoY6/ZTletXXOM7XDYam21jlXuMnFC3Q8AgG7XDU1ZqnqUzPNcgVV7yIoKNU7x1U3BA0qOszy1AebXDQet5qazFI1htIcY/z+zt8SYnEVpQYVHWF5z7dkeA25eh0HDZJ27bJ7TjcM3yOra+z+CsbyenTD5S1HwZpy2yf+7DOzVOWjJ5nlmjfPLJWWPO/w/u3zSRF37/Hjz7drYx++ZvdaWr5H7tsXfw6fTxo4MP48XpeT86a2bdvf02XEnHJKf5WUXKaJEydqx44drY4xIggAAAAAJry5amhxcXGbEUE6ggAAAABwAhsyZEibx1g1FAAAAAASjOsRwZUrV2rhwoXasGGDQqGQLr74Ys2dO1f9+/ePxVxxxRX69NNP2/zs5MmTtXDhQpuKAQAAAMCTIupspc7uF+eqoStXrtTs2bM1atQo3XHHHdq5c6eWLFmizz77TMXFxfL5fIpGo9qyZYsKCws1aVLrPwweOnRofPUDAAAAAMy46gg+/vjjGjJkiJYuXarAkSWzhgwZogcffFAffPCBLrjgApWVlam+vl4XXXSRioqKurRoAAAAAMDX5/g3ggcPHtTJJ5+sGTNmxDqBklRQcHibgY0bN0qSNm/eLEkaNmxYV9QJAAAAAB7XsqG8V/7FMTW0b9++Wrx4cZvH169fL0nKzDy8R8nnn38u6WhHsL6+XsGg3V5aAAAAAAAbx71q6I4dO7R8+XI9/PDDysvL07e+9S1JhzuC/fv31/z58zV27FiNHTtWhYWFWrFihXnRAAAAAICvLykajUbdBtfU1Ojcc8+VJPXr10/PPvts7L8vv/xyrVu3TpMnT9all16q2tpaLVmyRBs2bNBjjz2madOmdckvAAAAAABekJPzkrZtq+vpMmJOOSWkkpKZ7R47ro7g3r179b//+79qbGzUCy+8oPXr12vBggWaPHmyXnzxRUUiEc2aNSsW39DQoEsuuUQHDhzQ+++/r+TkZPdVr1kjNTZ2fLygQFq1yjlPiusdMpxlZdnlSk+3y/Xxx2apqocXOMakpUnV1c650t55xaCiwyJXzDDLVWd4bYZDdssD19Y5D9CHw1JtrXOucJOLF+h4HPP3wXGrqTFLVZ+aaZYrqHoXQUGp3jmuusFuWrzlqQ82uWg8bjU1maVqDKU5xvj9nb8lxOIqSg0qOsLynm/J8Bpy9ToOGiTt2mX3nG4YvkdW19htmWx5Pbrh8pajYE257RN/9plZqvLRk5yDXJo3zyyVljzv8P7t80kRd+/x48+3a2Mfvmb3Wlq+R+7bF38On08aODD+PF7XmzqCx9VyTzrpJE2ZMkXTpk1TcXGxMjMzNX/+fEnSVVdd1aoTKEmBQEBFRUWqqqqKLSYDAAAAAOhZX/srjEAgoAsvvFA7d+5UdSfDQ2lph7/prXfzlRYAAAAA9Fo9vUpoe//a59gR3LJliyZOnKji4uI2x/bv36+kpCQdOHBAU6dO1VNPPdUm5osvvpAkZXl1ig0AAAAAJBjHjuApp5yiffv26aWXXlLjMX+gsWPHDv3+97/XOeeco6FDh6q2tlavvvqq6o75A6zy8nItX75c5557rgYmwqRgAAAAAOgFHFdSSUlJ0T333KM777xT1157rS677DLt2bNHxcXF8vl8uvfeeyVJ999/v+bMmaOZM2dq+vTp2r9/v4qLi5WSkqL777+/y38RAAAAAOhZLRvKe0UcG8pLUlFRkfr06aNFixZp/vz5CgaDGjdunG677TadeuqpkqTCwkI9/fTTevbZZ/XEE08oEAiooKBAt99+e2yTeQAAAABAz3O9t8KUKVM0ZcqUTmMKCwtVWFgYd1EAAAAAgK5juMmesfx855izz3YM2Vpit7dLborx3mxGImc77/3nVtof33cOmjBBaZ+5iLvwwrjraeErs9sbLGy4lcmHgYlmudxsmxUOSxUVznEHTnLel+14VJbZ5crPajDLZblN6OrPnPf+O+ssafUGF3E5dveK8hq71zLYUGWWq7J/rlmujCoX+2ZlZsrvJs6je1662SvRNcNcrs6p5G6/wQa7a7s6ZZBZriq7Zq+8kN0eb6VNznu8ZWe7qz9bdvt6SlLtOLu9/wKGpT30kF0up73/PvzQ/f6AH/7Rbk/hZS/Z7f130UVmqZRxsouNXF3xG+Xxsog6m47Z/Tquxa6XBAAAAADoFbw7IggAAAAAvUrne/d1vzj2EQQAAAAAnFjoCAIAAABAgmFqKAAAAACY6D37CDIiCAAAAAAJho4gAAAAACQYpoYCAAAAgAlWDQUAAAAAeBQdQQAAAABIMEwNBQAAAAATrBoKAAAAAPAoOoIAAAAAkGCYGgoAAAAAJlg1FAAAAADgUXQEAQAAACDBMDUUAAAAAExE1NlKnd2PVUMBAAAAAEd4d0Rw506puZM/tMzOlsrKHNPk5GTb1VRWZ5crFDJLVVPnN8uVNmaMu0A3cQ0N8ZTSSn263esYTLFr9uNVbpZLgYCLoDTlpVc7RkVS0+Kv5xgZSbvsktXZtQu/4XV0VqjERVSezgptcg6rc/NaupOZYne+KvvnmuXK+HKdWS41NTnHZGZKVVXOcVlZ8dfToqLCLFXKCLtr0ldhd9/ZVJfpGJPnMm7PHoOCjjg3YPc7pg22u09EQs7nwa10l5d2erpzTCRg+FlHUlWJXa7crEazXB99ZPd558PXnNpYpouYw5a9ZNcurp5pN5I0+ya78Z5fziuJP0lKipRr9z6E+Hm3IwgAAAAAvQobygMAAAAAPIqOIAAAAAAkGKaGAgAAAIAJNpQHAAAAAHgUHUEAAAAASDBMDQUAAAAAE0wNBQAAAAB4FB1BAAAAAEgwTA0FAAAAABNRdbaJe/eLdniEEUEAAAAASDB0BAEAAAAgwTA1FAAAAABMsGooAAAAAMCj6AgCAAAAQIJhaigAAAAAmGBqKAAAAADAo+gIAgAAAECC8ezU0MaBQzs97pfUODjbMc/mDUYFSfqrv3J+PrcOVpilUl2dXa5ATtgxJiipPsVF3IY18Rd0REXWILNcuRUlZrk0bpxZqtIy5+9lstOk0ro0x7imGoOCjtHQYHf+R44w3GS1ocEs1bqmPMeYka7jSg0qOiwyONMsV0bVLrNckREjzXLt3u0ckyGpMiPfOa6hPP6Cjmgcbvc7+ivs6vqkyq5NpKa6iwsEnGPOPb02rlqOVXnAsN032LV7hZzf+9wKVmx1DsrNdRdnLHfwYLNcm0qCZrmmjKs2y1Uf6LyNBSXVp7prhxddZFDQEbNvshuj+eUiu/fbRx51fu9zctJJ0pw5BsV4XkTemhracTtgRBAAAAAAEgwdQQAAAABIMJ6dGgoAAAAAvQurhgIAAAAAPIqOIAAAAAAkGKaGAgAAAICJiDpbqbP7sWooAAAAAOAIOoIAAAAAkGCYGgoAAAAAJthQHgAAAADgUXQEAQAAACDBMDUUAAAAAEywoTwAAAAAwKPoCAIAAABAgmFqKAAAAACYYGooAAAAAMCj6AgCAAAAQIJhaigAAAAAmOg9G8p7tiP4+utSXV3Hx2fPlpYudc4z+/qOf/njVd9gN4Ca8eU6s1xbAyPNclVUOMfk5rqMGzMm7npiuVRrlktVhs2+psYsVXZDlYuoPGU3bHKMqk7Pi7+gY4RCdrm2lthdR7k5AbNcWVl2cdVN2fEVc4y0NavNcn2ScpZZrlAn9+fjlZvV6CLKr4yTXcQ12DVWf1O9WS41NZmlyh+8yyyXq5t5dr6yaz5xDPvD5nyDgg4bNcoslcqbBpnlymxy01bdqeyf6xiT4TZu2yqDio4qTXF+TrfyagxrC7m8Ubuw71Dnx4NBad8+d7lc3Ztc+uW8ErNcjzxq91ng7nlWn6eZjOglvBoAAAAAkGA8OyIIAAAAAL0Lq4YCAAAAADyKjiAAAAAAJBimhgIAAACAiYg6W6mz+3VcCyOCAAAAAJBg6AgCAAAAQIJhaigAAAAAmOg9G8ozIggAAAAACYaOIAAAAAAkGKaGAgAAAIAJNpQHAAAAAHgUHUEAAAAASDBMDQUAAAAAE6waCgAAAADwKDqCAAAAAJBgmBoKAAAAACYi6mw6ZvfruBbPdgSDQeeYUKjr6zhWUpJhsj59zFKl9MCr6Oo5TU+YIb/fLpfPcFDd7QvpIs6yLGs90V7dcNtc3cSZnn/D9ton2SyVZ19H0/uOZa5kw5Nv2cDcvhe5iAsE4qzlGF6+h1lx+zu6irN8X5NtczWtzbAwN+e1R9qh4c31pJPMUuE4ZGX9VU+X0EpLPTt37lRzc+u/XUyKRqPRnigKAAAAANC1GhoaNGHCBO3du7fV43QEAQAAAOAEVVtbq9ra2jaP0xEEAAAAgASTALPwAQAAAADHoiMIAAAAAAmGjiAAAAAAJBg6ggAAAACQYOgIAgAAAECCoSMIAAAAAAmGjiAAAAAAJBg6ggAAAACQYFJ6uoDjtX37dj322GNatWqVJOnCCy/UvHnzlJaW1sOVnfiuuOIKffrpp20enzx5shYuXNgDFZ347r33XpWUlOiFF15o9TjXQffo6PxzLdj74IMP9Mwzz2jt2rXy+Xw688wzNXfuXI0ZMyYWQ7vvGm7OPW2+66xcuVILFy7Uhg0bFAqFdPHFF2vu3Lnq379/LIa23zXcnHvaPk5kvaojuGfPHn3ve99TY2OjbrrpJjU3N2vx4sXauHGjXn31Vfn9/p4u8YQVjUa1ZcsWFRYWatKkSa2ODR06tIeqOrG9+uqreuWVV1RQUNDqca6D7tHR+edasLdq1SrdfPPN+sY3vqHbbrtNTU1NWrZsma655hotW7ZM+fn5tPsu4ubc0+a7zsqVKzV79myNGjVKd9xxh3bu3KklS5bos88+U3FxsXw+H22/i7g597R9nPCivciTTz4ZPf3006ObN2+OPfa///u/0by8vOjLL7/cg5Wd+EpLS6N5eXnR//iP/+jpUk54TU1N0Z/97GfR0047LZqXlxe95pprWh3nOuhaTuefa8FeUVFR9MILL4zW19fHHtu9e3f0nHPOiV5//fXRaJR231XcnHvafNe5/PLLo9/85jejBw4ciD22dOnSaF5eXvTdd9+NRqO0/a7i5tzT9nGi61V/I7hixQoVFBRo2LBhscfGjx+vU089VStWrOjByk58mzdvlqRW5x72Dh48qMsvv1w/+9nPVFRUpIyMjDYxXAddx83551qwtXfvXm3YsEEXX3yx+vXrF3s8PT1d55xzjv785z9Lot13BbfnnjbfNQ4ePKiTTz5ZM2bMUCAQiD3eMgth48aNkmj7XcHtuaft40TXa6aG7t27V9u3b9fkyZPbHBs1apTee++9HqgqcXz++eeSjt4M6+vrFQwGe7KkE9LBgwdVV1enBQsWaMqUKZo4cWKr41wHXcvp/EtcC9ZCoZB+97vfteqItNizZ4+Sk5Np913EzbmXaPNdpW/fvlq8eHGbx9evXy9JyszMpO13ETfnXqLt48TXa0YEKysrJandb+gHDhyoffv2ad++fd1dVsL4/PPP1b9/f82fP19jx47V2LFjVVhYyLeRxkKhkN5++21NmTKl3eNcB13L6fxLXAvWkpOTlZOT06ZNb9iwQatXr9bYsWNp913EzbmXaPPdZceOHVq+fLkefvhh5eXl6Vvf+hZtv5u0d+4l2j5OfL1mRHD//v2S1O43l3379pV0+JuaAQMGdGtdiWLz5s3av3+/9u3bpx//+Meqra3VkiVLdPvtt+vQoUOaNm1aT5d4QvD5fPL5Ov5+huugazmdf4lroTvs379fP/rRjyRJf//3f0+770ZfPfcSbb471NTUxGYg9OvXT/fcc4/69u1L2+8GHZ17ibaPE1+v6QhGo1HHmKSkpG6oJDHNmDFDkUhEs2bNij02depUXXLJJXr88cd16aWXxqYRoetwHfQ8roWudeDAAd16663asGGDbrnlFhUUFGj16tWOP0e7j197516izXeHpKQkLViwQI2NjXrhhRd0ww03aMGCBRo4cKCrn8XX19G5nzx5Mm0fJ7xeMzW0ZU72wYMH2xxreSwUCnVrTYnkqquuanUjlKRAIKCioiJVVVXF/qAaXYvroOdxLXSd2tpazZ49W3/605/03e9+V7fddpsk2n136OjcS7T57nDSSSdpypQpmjZtmoqLi5WZman58+fT9rtBR+deou3jxNdrOoItf7i7e/fuNsd27dqlcDjMH/D2gJbNbOvr63u4ksTAdeBdXAvx+fLLL3Xddddp9erVuvLKK/Xwww/HRjpo912rs3PfGdp81wgEArrwwgu1c+dODRo0SBJtv7sce+6rq6s7jKPt40TRazqC4XBYWVlZWrt2bZtj69at0+jRo3ugqsRQWVmpqVOn6qmnnmpz7IsvvpAkZWVldXdZCYnroGdxLXSNuro63XjjjVq/fr2uv/56Pfjgg606IrT7ruN07mnzXWfLli2aOHGiiouL2xzbv3+/kpKS5Pf7aftdwM25P3DgAG0fJ7xe0xGUpEmTJmnlypXasmVL7LEPP/xQX3zxRaer/CE+GRkZqq2t1auvvqq6urrY4+Xl5Vq+fLnOPfdcV3/HABtcBz2Ha6FrPPjgg1q/fr2uu+463XXXXe3G0O67htO5p813nVNOOUX79u3TSy+9pMbGxtjjO3bs0O9//3udc845CoVCtP0u4ObcDx06lLaPE15S1M3qEx5RXV2tSy65RMnJyZo9e7YOHjyoRYsWKTs7Wy+99JL8fn9Pl3jCeueddzRnzhx94xvf0PTp07V//34VFxfr0KFDevHFF9lstYtMnDhRQ4cO1QsvvBB7jOug+7R3/rkWbG3ZskVTpkxROBzWXXfd1e7CC0VFRbT7LuD23NPmu84bb7yhO++8U2PGjNFll12mPXv2xM7tsmXLlJeXR9vvIm7OPW0fJ7pe1RGUpK1bt2r+/Pn6+OOPFQgEdMEFF+jOO++MzddG13nnnXf07LPPasOGDQoEAiooKNDtt9/OjbALtdcRkbgOuktH559rwc6LL76oBx54oNOYjRs3SqLdWzuec0+b7zq/+c1vtGjRIm3atEnBYFDjxo3TbbfdplNPPTUWQ9vvGm7OPW0fJ7Je1xEEAAAAAMSnV/2NIAAAAAAgfnQEAQAAACDB0BEEAAAAgARDRxAAAAAAEgwdQQAAAABIMHQEAQAAACDB0BEEAAAAgARDRxAAAAAAEgwdQQAAAABIMP8/8YfugAZ5AvAAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_171_1.png" } }, "output_type": "display_data" } ], "source": [ "_lambda = 0.1\n", "clf_ridge = skl.Ridge(alpha=_lambda).fit(X_train, y_train)\n", "J_ridge_sk = clf_ridge.coef_.reshape(L, L)\n", "fig = plt.figure(figsize=(20, 14))\n", "im = plt.imshow(J_ridge_sk, **cmap_args)\n", "plt.title(\"Ridge from Scikit-learn\", fontsize=18)\n", "plt.xticks(fontsize=18)\n", "plt.yticks(fontsize=18)\n", "cb = fig.colorbar(im)\n", "cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the **Least Absolute Shrinkage and Selection Operator** (LASSO)-method we get a third cost function." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " C(\\boldsymbol{X}, \\boldsymbol{\\beta}; \\lambda) = (\\boldsymbol{X}\\boldsymbol{\\beta} - \\boldsymbol{y})^T(\\boldsymbol{X}\\boldsymbol{\\beta} - \\boldsymbol{y}) + \\lambda \\sqrt{\\boldsymbol{\\beta}^T\\boldsymbol{\\beta}}.\n", "\\label{_auto12} \\tag{12}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finding the extremal point of this cost function is not so straight-forward as in least squares and ridge. We will therefore rely solely on the function ``Lasso`` from **Scikit-Learn**." ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ ":9: UserWarning: FixedFormatter should only be used together with FixedLocator\n", " cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAM2CAYAAACjUj0CAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABNzElEQVR4nO3dfZxWdZkw8GtmCAZELFZCXnxBntAeSJlcsGxTRAQDDEpFDV8ALbfcp8Ra0j6UaSmJGi7qogZZ4PjGI+UWbm9bmquk22O7lQIhoSGSbyAvA4Lcc54/jFvHgZkbODP3mTnf7+czn81zfvfv/Oaac7te9++6r1ORJEkSAAAA5EZluRcAAABA65IIAgAA5IxEEAAAIGckggAAADkjEQQAAMgZiSAAAEDOSAQB3uGyyy6LI444Ip5//vlyLyVVjz32WHzsYx+LQYMGxac+9alyL2e3HnzwwTj33HNjyJAhcdRRR8XIkSPjm9/8Zrz00kt7PNeiRYviiCOOiMcffzwiIh5//PE44ogjYtGiRbt9zfPPPx9HHHFE3HTTTQ2Or169uqRrDh8+PM4999w9XisAtKYO5V4AAC2vvr4+vvjFL0ZVVVVcfvnlcdBBB5V7Sbs0a9asuPXWW2PYsGFx8cUXR3V1daxYsSL+7//9v7F48eK4995745BDDil5viFDhsTMmTOjf//+Jb+me/fuMXPmzDjiiCOKx/71X/81fvCDH8TPf/7zPfp9ACCrJIIAOfDyyy/HunXrYvLkyTFx4sRyL2eX1q5dG9/5znfi3HPPjenTpzc4N3bs2Jg4cWJ8+9vfjhtvvLHkOQ8++OA4+OCD92gdXbp0iXHjxjU4tmTJkigUCns0DwBkmdJQgBx44403IiJiv/32K/NKdu9//ud/olAoxEc+8pFG52pqauKoo46K//7v/279hQFAOyQRBNgH//7v/x7nnHNOHHPMMTFo0KAYPnx4zJw5M7Zv314cs3379rj66qvjpJNOikGDBsUJJ5wQV155ZWzYsKE4JkmSuPnmm2PUqFHxgQ98II477rj453/+51i7dm2D661fvz6+/vWvx0c/+tEYNGhQjBo1Km6//fYmd6tuuummOOmkkyIi4uabby5+Z+6mm26KD3zgA/Hzn/88PvKRj0RNTU0sXLiw5OvcdNNNUVNTE88880xMnjw5Bg8eHB/96EfjO9/5TiRJEvPmzYthw4ZFTU1NXHDBBc1+53JnkvqDH/ygQfx2mj9/fjz00EMNjm3evDmuueaaGDZsWBx99NFx6qmnFn+HiMbfEdyVxYsXx/vf//74/Oc/H4VCodF3BIcPHx5PPPFErFmzZpffHSzF7373u5g8eXLU1NRETU1NTJkyJX7/+983GJMkSdx9991x+umnR01NTXzgAx+IU045JW6//fZIkqQ4bvjw4TF9+vT4yle+EkcddVQcf/zxsW7duhg+fHh87WtfiwceeCDGjBkTH/jAB2LkyJFRW1u7x+sFoP1TGgqwlxYuXBjTp0+P4cOHx5e+9KV444034uc//3nMmzcvIiKmTZsWERFXXXVV/PjHP47zzjsvDj744FixYkXU1tbGc889F9/97ncjIuLWW2+NW265JSZOnFhsVDN//vz44x//GD/+8Y+jqqoqNmzYEGeddVasWbMmzjrrrOjXr188+uijccMNN8TTTz+925LJk08+Ofbff/+YMWNGnHzyyXHyySdH//7944knnogdO3bE1772tZg8eXJs3749jjnmmD26zhtvvBHnn39+jBgxIkaOHBn3339/XH/99fGb3/wm1qxZE5MmTYr169fH3Llz4/LLL48FCxbsNp7HHnts9O3bN37605/G//t//y9GjhwZH/nIR2LIkCFxwAEHRMeOHRuM3759e0ycODFWrFgREyZMiCOPPDIefvjhmD59emzdujXOO++8Zv+Gv/71r+PLX/5yHH/88XHDDTdEVVVVozFf+cpX4oYbboj169fH5Zdf3uC7g6V49NFH46KLLoojjzwyvvCFL8T27dtj0aJFMXHixLjjjjvi7//+7yMi4sYbb4xbb701PvGJT8SECROirq4ufvjDH8YNN9wQ++23X4OS3sWLF8fhhx8eX/nKV+KVV16J7t27R0TEI488Ej/5yU/inHPOiQMPPDDuvffeuOqqq6Jv375xwgkn7NG6AWjnEgAa+PKXv5wMGDAgWb16dZPjTjnllOTMM89M6uvri8feeOON5Pjjj0/Gjh1bPHbUUUclV155ZYPXzpo1K/nkJz+ZbN68OUmSJPnYxz6WfOYzn2kw5u67704+/vGPJ88991ySJEly3XXXJQMGDEh+/vOfNxj39a9/PRkwYEDy0EMP7Xatq1evTgYMGJDMnj27eGz27NnJgAEDkttuu63B2FKvs/P13/rWt4pjVqxYkQwYMCCpqalJXn311eLxL37xi8kRRxyRbNu2bbdrTJIk+fOf/5yMHz8+GTBgQPHn/e9/f3LuuecmjzzySIOxtbW1yYABA5J/+7d/Kx6rr69PPvWpTyUf+chHkkKhkNx///3JgAEDkt/85jdJkiTJb37zm2TAgAHJ/fffnzz55JPJ0UcfnZx//vnJ66+/3mSszjnnnOTEE09scu07nXjiick555yTJEmSFAqF5KSTTkrOOuusZMeOHcUxdXV1ycknn5yMGzcuSZIk2b59e/LBD34wmTp1aoO5Nm3alAwaNCi56KKLGsx/5JFHJn/9618bXfeII45Ili5dWjz20ksvJUcccURy6aWXlrR2APJDaSjAXvq3f/u3uP3226OioqJ47NVXX41u3brFli1biscOOuigePDBB2PRokWxcePGiIi45JJL4v777y+WQx500EHx+OOPx/e///145ZVXIiLirLPOigceeKDYJfOXv/xl9O/fP0aMGNFgHZ/73OciIuI//uM/9ur3GDJkSIN/3tPrvH3cYYcdFhERH/zgB4u7VBERffv2jSRJir/b7vTr1y8WLVoU8+fPj/POOy/69+8fhUIhHn/88bjgggvi9ttvL4596KGHonv37jF27NjisYqKipg5c2bU1tY2+Lu804oVK+Kiiy6Kvn37xpw5c6JTp05NrmtvPf3007F69eoYMWJEbNiwIdatWxfr1q2L119/PU488cRYunRpvPjii/Gud70rHnvssbjqqqsavH79+vXRtWvXBvdTRMQhhxwSPXv2bHS9fv36xZFHHln85x49esSBBx7YbNwByB+loQB76V3velf813/9V/z4xz+OP//5z/GXv/wlXn311YiI6NOnT3Hc17/+9bjkkkvi8ssvj69+9asxePDgOPnkk+O0006L/fffPyLeLCP97Gc/G9dcc03MmDEjBg4cGMOHD48JEyZEjx49IuLN59t99KMfbbSOHj16RLdu3WLNmjV79Xv83d/9XYN/3tPrHHjggcX/3aFDh13OubPksr6+vtn1VFRUxLHHHhvHHntsRES88MILcf/998dtt90W//Iv/xLjxo2Lnj17xpo1a+KQQw5plPC9Pfa7893vfjcqKyvj9ddfj5dffnmPHkkREVFXV9coOdt///2jurq6wbG//OUvERExc+bMmDlz5i7neuGFF6Jnz57xrne9Kx566KH4j//4j1i1alU899xzxe+RJm/7jmBE4/ju9Pbke6eOHTuWFHcA8kUiCLCXvvGNb8Sdd94Z//t//+8YPHhwjBs3LmpqauIb3/hGgyYvH/7wh+NXv/pV8efRRx+NGTNmxPe+971YtGhRdO/ePY488sj46U9/Go888kj86le/ikceeSRmz54dd9xxR9x7773Rv3//RsnA29XX18e73vWuvfo9KisbFofs6XV29b26pnbjdmfBggWxbdu2uPDCCxsc7927d/yf//N/olOnTnHDDTfEf//3f8eoUaOiUCjs1XUiIo444oj42te+FpMmTYorr7yy+L3OUn33u9+Nm2++ucGxGTNmxCc/+ckGx3YmYF/4whdi8ODBu5zr8MMPjyRJ4nOf+1z86le/imOOOSZqamrizDPPjCFDhsT555/f6DW7inlE478lAOyORBBgL6xZsybuvPPOGDduXKOdnreX4W3fvj2WLl0aBx10UIwZMybGjBkT9fX1cccdd8TMmTNj8eLF8alPfSqWLVsWXbt2jZNOOqnY4fPBBx+MqVOnxsKFC+Oyyy6LPn36xKpVqxqt5eWXX47NmzdHr169UvndWus67/SLX/wifv/738enPvWp6NKlS6PzAwYMiIgo7rr17t07li9f3mjcww8/HA8++GD88z//826vNWnSpPj7v//7mDRpUnznO9+JH//4xw1KTJszfvz4OOaYYxoc+1//6381Grdzd7JLly5x3HHHNTj3+9//PjZs2BDV1dXx29/+Nn71q1/F5z73ufjCF75QHLNjx4547bXX9vhZiADQHB8dAuyFnSV77/yP/4cffjieffbZ2LFjR0S8+R2vM888M2677bbimMrKyvjABz5Q/N+FQiHOO++8uOaaaxrMdfTRRxfHRESceOKJsXLlyvjFL37RYNzO780NGzYsld+tta7zTqeeemps2bIlvvWtbzUqZayvr4+FCxdGt27dit9pPP744+OVV16Jn//85w3Gfv/734+HHnoo3vOe9zR7zc997nPRq1evmDFjRvH7m7tSWVnZYE0HH3xwHHfccQ1+3vve9zZ63aBBg6JHjx6xYMGCqKurKx7fvHlzsVy4qqoqXnvttYhofD/dd999sXXr1uL9BABpsSMIsBuzZs3a5QPYP/axj8UxxxwTvXv3jltvvTW2bdsWBx10UPz+97+PH/zgB9GpU6fif/T37NkzTj311Ljrrrti69atUVNTE6+99lrceeedceCBB8bHPvax6NixY5x77rkxZ86cuPjii+OjH/1ovP7663HvvfdG586d47TTTouIiIsuuih+9rOfxSWXXBJnn312HHbYYfGb3/wmfvazn8XIkSNTezxAa13nnT75yU/GI488Evfee2/87ne/i1NOOSUOOuigePXVV+Pf//3fY/ny5XHDDTcUdwvPOuusuP/++2Pq1KkxceLE6NevXzz00EPx6KOPxjXXXLPb8sm369KlS1x22WXxhS98Ia6//vpGzVp26t69e/zXf/1XfPe7341jjjmmmKQ3513veldMnz49pk6dGp/85Cfj9NNPj06dOsXChQvjhRdeiOuvvz46dOgQNTU10bVr15gxY0asWbMmDjjggHj88cfjwQcfbHA/AUBaJIIAu/HjH/94l8cPP/zw+PCHPxy33357fOtb34r58+dHkiRxyCGHxFe+8pXYsWNHXH311fHHP/4xBg0aFN/4xjfi4IMPjsWLF8fixYujc+fO8eEPfzimTp1abO7x+c9/Pt797nfH/fffH9dee21UVVXFBz/4wbjuuuuif//+ERHx7ne/O+6999648cYb48EHH4yNGzfGwQcfHNOmTYtJkyal9nu31nXeqbKyMm688cZ44IEH4oEHHog777wzNm3aFAcccEAcc8wxceWVV8ZRRx1VHF9dXR0LFiyIG2+8MRYvXhybNm2K/v37x4033hgf+9jHSr7uKaecEv/wD/8Q9913X3ziE58oNud5uwsvvDCWL18e3/72t+OTn/xkyYngzvkPOOCAmDNnTvzrv/5rVFZWxvve976YM2dOnHjiiRHxZsOd22+/Pa6//vqYM2dOdOzYMfr16xff/va34/e//33Mnz8/XnnllQaNeQBgX1QkTXUFAAAAoN3xHUEAAICckQgCAADkjEQQAAAgZzSLAQAAaKc2bty4y0ckaRYDAACwj7auWxed/9YNPEtef/31OP7444vPQN4pu4ngOx4m3EhlZfNjIuKww29rdkypnv3zRanN1aaVGHtagNiXl/iXj9iXj9iXj9iXj9g3cNjh39nnOfr27Rr/+Z8TU1hNtn33H/4hNj7/fLmXUdStb9+Y8p//GWvXro1CodDgXLsvDX3uuQ3NDwIAAHbpuecalxWyaxuffz42PPdcuZfRSK9evRod0ywGAAAgZ1JPBFevXh3/9E//FEOHDo2hQ4fGtGnTYt26dWlfBgAAIFMqM/izO6mWhq5fvz7OP//82L59e1x44YVRKBRi3rx5sXz58li4cGF07NgxzcsBAACwF1JNBL/3ve/FX//61/jRj34U/fv3j4iIo48+OiZPnhw//OEPY8KECWleDgAAgL2Qamno4sWLY+jQocUkMCLiuOOOi379+sXixYvTvBQAAECmlLsMdE9KQ1NLBDds2BCrV6+OgQMHNjo3cODAeOqpp9K6FAAAAPsgtUTwxRdfjIiInj17NjrXo0eP2LRpU2zatCmtywEAALCXUvuOYF1dXUREdO7cudG5Tp06RUTEli1bYv/99y9twsoSctQSxiTJZaVdjz1Tyt+HliH25SX+5SP25SP25SP25SP2RUnyz+VeQpvRXDlma2uVrqFJkjQ7pqKiovQJ6+ubPl9Z2fyYiKiomln6NZuRFKalNlebVmLsaQFiX17iXz5iXz5iXz5iXz5i30BF1Q37PMehh3aLZ5+9KIXVkJbUEtYuXbpERMS2bdsandt5rGvXrmldDgAAgL2U2o5g7969IyLi5ZdfbnTupZdeim7duhWTRQAAgPamIrJVGtpUPWZq6+zWrVv07dt3l91Bn3766Rg0aFBalwIAAGAfpJqwjhw5MpYsWRIrV64sHnvsscdi1apVMXr06DQvBQAAwF5KrTQ0IuLTn/50PPDAAzFp0qSYMmVKbNu2LebOnRsDBw6McePGpXkpAACATGlLXUNTXWf37t3jzjvvjCOPPDJmz54d3//+92PEiBExd+7c6NixY5qXAgAAYC9VJKU896EMNm2KaGpl3bpFbNzY/DzduqbX+reiakZqcyWFy1Obq9VpqVw+Yl9e4l8+Yl8+Yl8+Yl8+Yt8ycvBsxtsOOyw2PvdcuZdR1O3QQ+OiZ5/d5blUS0MBAADyKreloQAAAGSfRBAAACBnlIYCAACkQGkoAAAAmSURBAAAyBmloQAAACmoiGzttFU0cS5L6wQAAKAVSAQBAAByRmkoAABACnQNBQAAILPsCAIAAKSgIppu0NLaNIsBAACgSCIIAACQM0pDAQAAUlD1t5+saGotdgQBAAByRiIIAACQM0pDAQAAUlAR2dppa6praGYTwf33q29mRGV069rcmIgX/prenyIpXJ7aXBVV01KbKynMTG0uAACg/ctSwgoAAEAryOyOIAAAQFtSGdnaaWtqLVlaJwAAAK1AIggAAJAzSkMBAABSoDQUAACAzJIIAgAA5IzSUAAAgBQoDQUAACCzJIIAAAA5ozQUAAAgBRWRrZ22iibOZWmdAAAAtAKJIAAAQM4oDQUAAEiBrqEAAABklkQQAAAgZ5SGAgAApKAimu7U2dp0DQUAAKCo3e8I9j6oPrW56lPMm5PCzNTmqqj6fGpzJYXZqc0FAABkU7tPBAEAAFpD1d9+sqKptSgNBQAAyBmJIAAAQM4oDQUAAEhBRWRrp03XUAAAAIokggAAADmjNBQAACAFlZGtnbam1pKldQIAANAKJIIAAAA5ozQUAAAgBUpDAQAAyCyJIAAAQM4oDQUAAEiB0lAAAAAySyIIAACQM0pDAQAAUlAR2dppq2jiXJbWCQAAQCuQCAIAAOSM0tA9UBn1qc215fX0cvCkMDu1uSqqLm7+esmc0sYVbkljSQAA0CboGgoAAEBmSQQBAAByRmkoAABACiqi6U6drU3XUAAAAIokggAAADmjNBQAACAFVX/7yYqm1mJHEAAAIGckggAAADmjNBQAACAFFZGtnTZdQwEAACiSCAIAAOSM0lAAAIAUVEa2dtqaWkuW1gkAAEArkAgCAADkjNJQAACAFCgNBQAAILMkggAAADmjNBQAACAFbak0VCJYJl2q61Ob68WX07vdksItqY2rqJq2r8t52/VmpjYXAADkydq1a6NQKDQ4JhEEAABoxyZOnBhr1qxpcEwiCAAAkIKKyFZpaMXf/m9tba0dQQAAgDzp1atXo2NZSlgBAABoBanvCJ5++unxhz/8odHxUaNGxezZs9O+HAAAQCbktmtokiSxcuXKGDFiRIwcObLBuT59+qR5KQAAAPZSqong888/H1u2bImTTjopxo0bl+bUAAAApCTVRPCZZ56JiIj+/funOS0AAEDmVcRbnTqzoKm1pFrCumLFioh4KxHcsmVLmtMDAACQgtQTwf322y9mzJgRNTU1UVNTEyNGjIjFixeneRkAAAD2QeqloXV1dbFp06aYOXNmbNy4MebPnx+XXnppvPHGGzF+/PjSJ6ssIUctZUwO9OyZ5mwlxrSE2CfJ9fu4FnbJfV9e4l8+Yl8+Yl8+Yl8+Ys9eqPrbT1Y0tZaKJEmStC509913R319fUycOLF47PXXX4+xY8fG1q1b49e//nVUVZUYmvr6ps9XVjY/JidefDm9f1H17FFCTEuMfUXVtBRW9KakMDO1udo09315iX/5iH35iH35iH35iH3LyEFy/eRhh8W2554r9zKKOh16aHzw2Wd3eS7Vv8bZZ5/dIAmMiKiuro5x48bFK6+8UmwmAwAAQPmk/kD5XenevXtEaB4DAAC0XxWRrQfKt0rX0BdffDHGjBkTN998c6Nzq1atioiIvn37pnU5AAAA9lJqiWDPnj1j48aNsXDhwti8eXPx+AsvvBCLFi2KY489Nnr06JHW5QAAANhLqZaGXnHFFXHxxRfHWWedFWeccUbU1dVFbW1tdOjQIa644oo0LwUAAJAplZGt0tCm1pLqOkeMGBG33HJLdO7cOa6//vq44447YvDgwXH33XcXHzIPAABAeaX6+IhUeXxEWax7rfnPBrp3j1i3rvm5ur87vb9PRdXFqc2VFG5Jba5W574vL/EvH7EvH7EvH7EvH7FvGTl4fMT/HHZYbM/Q4yM6HnpoHL2bx0e0StdQAACA9i63paEAAABkn0QQAAAgZ5SGAgAApEBpKAAAAJklEQQAAMgZpaEAAAApqIhs7bRVNHEuS+sEAACgFUgEAQAAckZpKAAAQAp0DQUAACCzJIIAAAA5ozQUAAAgBRXRdKfO1qZrKAAAAEUSQQAAgJxRGgoAAJCCqoioL/ci3qaqiXN2BAEAAHJGIggAAJAzSkNpoPu7S9nMrixp3PYd6X3OkBRuSW2uiqrpqc2VFL6Z2lwAALRtFZGtnTZdQwEAACiSCAIAAOSM0lAAAIAUVEa2dtqaWkuW1gkAAEArkAgCAADkjNJQAACAFCgNBQAAILPsCAIAAKTAjiAAAACZJREEAADIGaWhAAAAKaiIbO20VTRxLkvrBAAAoBVIBAEAAHJGaSgAAEAKKiMiKfci3kbXUAAAAIokggAAADmjNBQAACAFFdF0p87WpmsoAAAARXYEaTEdO9SnNteLL6f3mUVS+GZqc1VU/WNqcyWFW1ObCwAAmiIRBAAASEFVuRfwDk2tR2koAABAzkgEAQAAckZpKAAAQAoqIls7bbqGAgAAUCQRBAAAyBmloQAAACnI2i5bU+vJ2loBAABoYRJBAACAnFEaCgAAkIKs7bIpDQUAAKBIIggAAJAzSkMBAABSkLVdNqWhAAAAFEkEAQAAckZpKAAAQAoqIls7bRVNnMvSOgEAAGgFEkEAAICcURpKm9CzR31qc9Wn+PlHUrg1tbkqqr7Y/PWSWaWNK9yQxpIAANgDlRGRlHsRb6M0FAAAgCKJIAAAQM4oDQUAAEiB0lAAAAAySyIIAACQM0pDAQAAUqA0FAAAgMySCAIAAOSM0lAAAIAUVETT5ZhZYkcQAAAgZySCAAAAOaM0FAAAIAVtaZetLa0VAACAFEgEAQAAckZpKAAAQAra0i5bW1orAAAAKZAIAgAA5IzSUAAAgBS0pV02iSC5Uxn1qc21fUd6b/ekcENq4yqqpu7rct5xzVmpzgcAQHlJBAEAANqxtWvXRqFQaHBMIggAAJCCioqKiIqKci/jLX9by8SJE2PNmjUNTkkEAQAA2rHa2lo7ggAAAHnSq1evRsf2KhH86le/Gs8++2wsWLCgwfHVq1fHtddeG0888URERAwbNiwuu+yy6N69+95cBgAAoO3o0OHNn6xoYi17vMqFCxfGfffdF0OHDm1wfP369XH++efH9u3b48ILL4xCoRDz5s2L5cuXx8KFC6Njx457vnAAAABSV3IiWCgUYs6cOXHzzTfv8vz3vve9+Otf/xo/+tGPon///hERcfTRR8fkyZPjhz/8YUyYMCGdFQMAALBPSkoEt23bFmeccUYsX748xo8fH0uWLGk0ZvHixTF06NBiEhgRcdxxx0W/fv1i8eLFEkEAAKB9q6rKVmloVdVuT5X0NOxt27bF5s2bY9asWXHttddGh3f8chs2bIjVq1fHwIEDG7124MCB8dRTT+3higEAAGgpJaWrXbt2jZ/97GeNEsCdXnzxxYiI6NmzZ6NzPXr0iE2bNsWmTZti//3334elAgAAkIaSEsHKysqorNz95mFdXV1ERHTu3LnRuU6dOkVExJYtW/YsEWziens0hpYh9hERkW4PpBJjWkLsk+Rf9nEt7JZ7v3zEvnzEvnzEvnzEnr3RnruG7kqSJM2OqfjbU+1LVl/f9PnKyubH0DLEvmj7jvT+n0THDiXEtMTYV1RNTWFFb0kKs1Kdr81y75eP2JeP2JeP2JeP2LcMyXWmpPLX6NKlS0S8+V3Cd9p5rGvXrmlcCgAAgH2Uyo5g7969IyLi5ZdfbnTupZdeim7duhWTRQAAgHapvXUNbU63bt2ib9++u+wO+vTTT8egQYPSuAwAAAApSK1Qd+TIkbFkyZJYuXJl8dhjjz0Wq1atitGjR6d1GQAAAPZRavuWn/70p+OBBx6ISZMmxZQpU2Lbtm0xd+7cGDhwYIwbNy6tywAAAGRTG+oamtqOYPfu3ePOO++MI488MmbPnh3f//73Y8SIETF37tzomG6PfQAAAPbBXqWrv/zlL3d5/PDDD4/vfOc7+7QgAAAAWlaG9i2h7Snp2X8lqi9hg76yxHFpP/evouqLqc2VFG5IbS4AgEzJW9dQAAAA2g6JIAAAQM5kaN8SAACgDctj11AAAADaBokgAABAzmRo3xIAAKAN0zUUAACArJIIAgAA5EyG9i0BAADaMF1DAQAAyCqJIAAAQM5kaN8SAACgDdM1FAAAgKySCAIAAORMhvYtAQAA2jBdQwEAAMgqiSAAAEDOZGjfEgAAoA1rQ11DM7RKyLfKqC9pVGnj0pUUbkhtroqqy1ObKynMSG0uAIA8URoKAACQM3YEAQAA0qBrKAAAAFklEQQAAMiZDO1bAgAAtGFtqGuoHUEAAICckQgCAADkTIb2LQEAANowXUMBAADIKokgAABAzmRo3xIAAKAN0zUUAACArJIIAgAA5EyG9i0BAADaMF1DAQAAyCqJIAAAQM5kaN8SAACgDdM1FAAAgKzKULoKZFV9ip8ZJYUZqc1VUXVuanMlhQWpzQUAkHUSQQAAgDToGgoAAEBWZShdBQAAaMM0iwEAACCrJIIAAAA5k6F9SwAAgDZMsxgAAACySiIIAACQMxnatwQAAGjDdA0FAAAgqySCAAAAOZOhfUsAAIA2TNdQAAAAskoiCAAAkDMZ2rcEAABow3QNBQAAIKskggAAADmToX1LAACANqwNdQ3N0CqBrKqM+nIvYZeSwoLU5qqo+mbz10u+Vtq4wvQ0lgQA0GKUhgIAAOSMHUEAAIA06BoKAABAVkkEAQAAciZD+5YAAABtWBvqGmpHEAAAIGckggAAADmToX1LAACANkzXUAAAALJKIggAAJAzGdq3BAAAaMN0DQUAACCrJIIAAAA5k6F9SwAAgDZM11AAAACySiIIAACQMxnatwQAAGjDdA0FAAAgqzKUrgLsmfoUP8tKCtNTG1dRdfm+Ludt15uR2lwAQD6tXbs2CoVCg2MSQQAAgDRktGvoxIkTY82aNQ1OZWiVAAAApK22ttaOIAAAQJ706tWr0bG9SgS/+tWvxrPPPhsLFixocPz000+PP/zhD43Gjxo1KmbPnr03lwIAAGgb2lDX0D1e5cKFC+O+++6LoUOHNjieJEmsXLkyRowYESNHjmxwrk+fPnt6GQAAAFpIyYlgoVCIOXPmxM0337zL888//3xs2bIlTjrppBg3blxqCwQAACBdJSWC27ZtizPOOCOWL18e48ePjyVLljQa88wzz0RERP/+/dNdIQAAQFuQ0a6hu1LSQ7i2bdsWmzdvjlmzZsW1114bHXbxy61YsSIi3koEt2zZsjdLBQAAoIWVlK527do1fvazn+0yAdxpxYoVsd9++8WMGTPiwQcfjC1btsTBBx8cU6dOjTFjxqS2YAAAAPZNSYlgZWVlVFY2vXn4zDPPRF1dXWzatClmzpwZGzdujPnz58ell14ab7zxRowfP37PVtbM9UoeQ8sQ+/IR+6J0I1HibCXEP0mu3ce1sEvu/fIR+/IR+/IRe/ZGe+4aujsTJkyI+vr6mDhxYvHYmDFjYuzYsXHdddfFqaeeGlVN1Kg2Ul/f9PnKyubH0DLEvnzEvoH6FFPByighriXGv6Lq8hRW9KakMCO1udo09375iH35iH35iH3LkFxnSmp/jbPPPrtBEhgRUV1dHePGjYtXXnml2EwGAACA8mrxfcvu3btHhOYxAABAO9feuoY258UXX4wxY8bs8hmDq1atioiIvn37pnEpAAAA9lEqiWDPnj1j48aNsXDhwti8eXPx+AsvvBCLFi2KY489Nnr06JHGpQAAANhHqe1bXnHFFXHxxRfHWWedFWeccUbU1dVFbW1tdOjQIa644oq0LgMAAJBNbahraGrNYkaMGBG33HJLdO7cOa6//vq44447YvDgwXH33XcXHzIPAABA+e1VuvrLX/5yl8dHjBgRI0aM2KcFAQAA0LIytG8JsGdKevZfiTZubr5Aolu30sal+ew/zyQEgDYkb11DAQAAaDskggAAADmToX1LAACANiyPXUMBAABoGySCAAAAOZOhfUsAAIA2TNdQAAAAskoiCAAAkDMZ2rcEAABow3QNBQAAIKskggAAADmToX1LAACANkzXUAAAALJKIggAAJAzGdq3BAAAaMN0DQUAACCrJIIAAAA5k6F9SwAAgDasDXUNzdAqAcqnW9f6EkZVljRu+470ii2SwozU5qqo+nxqcyWF2anNBQC0PqWhAAAAOWNHEAAAIA26hgIAAJBVEkEAAICcydC+JQAAQBvWhrqG2hEEAADIGYkgAABAzmRo3xIAAKAN0zUUAACArJIIAgAA5EyG9i0BAADaMF1DAQAAyCqJIAAAQM5kaN8SAACgDevQIaJQKPcq3qJrKAAAADtJBAEAAHJGaSgAAEAadA0FAAAgqzKUrgK0Dx071Kc217rX0vu8LinMTm2uiqorU5srKVyR2lwAQGkkggAAAGno0CGiPr0PhPeZrqEAAADsJBEEAADIGaWhAAAAadA1FAAAgKySCAIAAORMhvYtAQAA2rAOHSKSpNyreIuuoQAAAOwkEQQAAMgZpaEAAABpqKrKVmmorqEAAADsJBEEAADIGaWhAAAAacjSw+QjdA0FAADgLRlLWQEAANqoJpqzlIVmMQAAAOwkEQQAAMgZpaEAAABp6NAhoqKi3Kt4SxOloRJBgAzr/u761ObaviO9IpCkcEVqc1VUTW/+esk1pY0rfDONJQFAu6c0FAAAIGfsCAIAAKShqipbpaGVu9/3syMIAACQMxJBAACAnFEaCgAAkIYOHSLq02v0ts+UhgIAALCTRBAAACBnlIYCAACkoaqqyXLMVtdEB9MMrRIAAIDWIBEEAADIGaWhAAAAaejQISJJyr2KtygNBQAAYCeJIAAAQM4oDQUAAEhDVVW5V1AyO4IAAAA5IxEEAADIGaWhAAAAaejQdtIrO4IAAAA5U5EkWXrQxdvU1zd9vrKy+TG0DLEvH7EvL/Ev2vJ6ep8jdqkuIaYlxr6ianoKK3pTUvhmanO1ae778hH78hH7llGZgz2oLN43lZWxdu3aKBQKDQ63nb1LAACADKvPYMFlZURMnDgx1qxZ0+C4RBAAAKAdq62ttSMIAACQJ7169Wp0rORE8JFHHok5c+bEU089FZWVlXH00UfHJZdcEoMHDy6OWb16dVx77bXxxBNPRETEsGHD4rLLLovu3bvv++oBAAAybMeOcq+gsY4dd328pGYxTzzxRJx33nnxvve9L0477bTYsWNH3HXXXfHSSy/FXXfdFUcddVSsX78+TjvttNi+fXucd955USgUYt68edGnT59YuHBhdNzdCnZHs5jsEvvyEfvyEv8izWJyxH1fPmJfPmLfMnLQLGb79nKvoLHdpWEl7Qhec8010atXr7jvvvuic+fOERExfvz4GD16dMyaNSvuuOOO+N73vhd//etf40c/+lH0798/IiKOPvromDx5cvzwhz+MCRMmpPObAAAAsE+aTcs3bNgQy5Yti1NOOaWYBEZEHHjggTFkyJD43e9+FxERixcvjqFDhxaTwIiI4447Lvr16xeLFy9ugaUDAABkR6HwZnloVn7e0R+mgWZ3BLt27Ro/+clPGiSBO61fvz6qqqpiw4YNsXr16hg1alSjMQMHDoyHH354zyIIAABAi2l2R7CqqioOO+yw6NmzZ4Pjy5YtiyeffDJqamrixRdfjIhoNCYiokePHrFp06bYtGlTSksGAABgX+zV4yPq6uriy1/+ckREfOYzn4m6urqIiF3uGnbq1CkiIrZs2RL7779/6Rcp5cukOfjCaWaJffmIfXmJf0REdOmS5mwlxrSE2CfJNfu4FnbJfV8+Yl8+Ys9e2LEjovlWnK2nomL35/Y4Edy6dWt89rOfjWXLlsVFF10UQ4cOjSeffLKERTSxil3RNTS7xL58xL68xL9I19Accd+Xj9iXj9i3DMl1puzRX2Pjxo0xZcqUePzxx+O0006LqVOnRkREl799NLxt27ZGr9l5rGvXrvu6VgAAAFJQ8o7gq6++GhdccEEsXbo0zjzzzLjyyiuLu3y9e/eOiIiXX3650eteeuml6NatWzFZBAAAaI8KhWxtJje1CVtSIrh58+ZiEjhp0qS4/PLLG5zv1q1b9O3bN5566qlGr3366adj0KBBe7ZiAAAAWkxJpaFXXXVVLF26NM4777xGSeBOI0eOjCVLlsTKlSuLxx577LFYtWpVjB49Op3VAgAAsM8qkqTpvjYrV66M0aNHR7du3eLyyy+PqqqqRmPGjRsX69ati7Fjx0ZVVVVMmTIltm3bFnPnzo1DDjkk7rnnnujYseOerUyzmOwS+/IR+/IS/yLNYnLEfV8+Yl8+Yt8yctAs5pVXsnXrVFZGHHjgrs81Wxr6xBNPRMSbjWJ2txs4bty46N69e9x5550xY8aMmD17dlRXV8eIESNi2rRpe54EAgAA0GKa3REsGzuC2SX25SP25SX+RXYEc8R9Xz5iXz5i3zLsCLa6fdoRBIB3Kil5K1F9CV9XryxxXJrJW0XVP6Y2V1K4NbW5AMiuQuHNn6zYxbf6itp/Wg4AAEADEkEAAICcURoKAACQgh07slUa2lQ3GDuCAAAAOSMRBAAAyBmloQAAACkoFN4sD20L7AgCAADkjEQQAAAgZ5SGAgAApGDHDqWhAAAAZJREEAAAIGeUhgIAAKQga11DKyp2f86OIAAAQM5IBAEAAHJGaSgAAEAKstY1VGkoAAAARRJBAACAnFEaCgAAkIKsdQ2tbGLbTyIIQFlVRn1Jo0obl56kcGtqc1VUTU9trqTwzdTmAiC/lIYCAADkjB1BAACAFGSta2hTpaF2BAEAAHJGIggAAJAzSkMBAABSkLWuoVVVuz9nRxAAACBnJIIAAAA5ozQUAAAgBVnrGqo0FAAAgCKJIAAAQM4oDQUAAEhB1rqGdmgi27MjCAAAkDMSQQAAgJxRGgoAAJCCrHUNbWotdgQBAAByRiIIAACQM0pDAQAAUpC1rqGFwu7P2REEAADIGTuCALAL23ek91lpUvhmanNVVF2Q2lxJYV5qcwHQtkgEAQAAUqBrKAAAAJklEQQAAMgZpaEAAAAp0DUUAACAzJIIAgAA5IzSUAAAgBToGgoAAEBmSQQBAAByRmkoAABACnQNBQAAILMkggAAADmjNBQAACAFuoYCAACQWRJBAACAnFEaCgAAkAJdQwEAAMgsiSAAAEDOKA0FgF3o2KE+tbnqU/zcNSnMS22uiqppzV8vub60cYWZaSwJoE3TNRQAAIDMkggCAADkjNJQAACAFOgaCgAAQGZJBAEAAHJGaSgAAEAKdA0FAAAgsySCAAAAOaM0FAAAIAW6hgIAAJBZdgQBAABSoFkMAAAAmSURBAAAyBmloQAAACnQLAYAAIDMkggCAADkjNJQAACAFOgaCgAAQGbZEQSAFlYZ9anNVZ/iZ7hJYWZq4yqqvrmvy3nb9aanNhcAEWvXro3COzrHSAQBAABSkNWuoRMnTow1a9Y0OCcRBAAAaMdqa2vtCAIAAORJr169Gh0rORF85JFHYs6cOfHUU09FZWVlHH300XHJJZfE4MGDi2NOP/30+MMf/tDotaNGjYrZs2fv3aoBAADagLbUNbSkRPCJJ56IT3/60/G+970vpk6dGjt27Ii77rorzjnnnLjrrrviqKOOiiRJYuXKlTFixIgYOXJkg9f36dNnn34BAAAA0lNSInjNNddEr1694r777ovOnTtHRMT48eNj9OjRMWvWrLjjjjvi+eefjy1btsRJJ50U48aNa9FFAwAAsPeaTQQ3bNgQy5Yti8mTJxeTwIiIAw88MIYMGRKPPvpoREQ888wzERHRv3//FloqAABAdmW1a+iuNJsIdu3aNX7yk580SAJ3Wr9+fVRVVUVExIoVKyLirURwy5Yt0aVLl71ZLwAAAC2o2afSVlVVxWGHHRY9e/ZscHzZsmXx5JNPRk1NTUS8mQjut99+MWPGjKipqYmampoYMWJELF68uGVWDgAAwF7Zq8dH1NXVxZe//OWIiPjMZz4TEW+WhtbV1cWmTZti5syZsXHjxpg/f35ceuml8cYbb8T48eP37CKVzeaopY2hZYh9+Yh9eYl/+Yh9RJTwCW5LzFZC7JPka/u4FnbJfV8+Ys9eaHddQ99u69at8dnPfjaWLVsWF110UQwdOjQiIiZMmBD19fUxceLE4tgxY8bE2LFj47rrrotTTz21WEZakvr6ps9XVjY/hpYh9uUj9uUl/uUj9kX1KaaClVFCTEuMfUXVN1NY0ZuSwvTU5mrT3PflI/YtQ3KdKXv019i4cWNMmTIlHn/88TjttNNi6tSpxXNnn312gyQwIqK6ujrGjRsXr7zySrGZDAAAAOVV8o7gq6++GhdccEEsXbo0zjzzzLjyyiujoqKi2dd17949It5sHgMAANBetaWuoSXtCG7evLmYBE6aNCmuuuqqBkngiy++GGPGjImbb7650WtXrVoVERF9+/bdw2UDAADQEkpKBK+66qpYunRpnHfeeXH55Zc3Ot+zZ8/YuHFjLFy4MDZv3lw8/sILL8SiRYvi2GOPjR49eqS3agAAAPZas6WhK1eujAceeCC6desW73//++OBBx5oNGbcuHFxxRVXxMUXXxxnnXVWnHHGGVFXVxe1tbXRoUOHuOKKK1pk8QAAAFnRrrqGPvHEExHxZqOYXe0GRryZCI4YMSJuueWWuO222+L666+P6urqGDp0aFx66aXFh8wDAABQfhVJkiTlXsQueXxEdol9+Yh9eYl/+Yh9kcdH5Ij7vnzEvmXk4PER06dHrFtX7lW8pXv3iG/u5l/Pe/VAeQCgPEpK3kq0fUfz/1HWsWNp49JM3iqqPp/aXElhdmpzATSn3XUNBQAAoP2QCAIAAOSM0lAAAIAUtKWuoXYEAQAAckYiCAAAkDNKQwEAAFKgaygAAACZJREEAADIGaWhAAAAKdA1FAAAgMySCAIAAOSM0lAAAIAU6BoKAABAZkkEAQAAckZpKAAAQAp0DQUAACCzJIIAAAA5ozQUAAAgBW2pa6hEEAByqmOH+hJGVZY0rj7FIqOkMDu1uSqqLkhtrqQwL7W5AMpNaSgAAEDO2BEEAABIga6hAAAAZJZEEAAAIGeUhgIAAKSgLXUNtSMIAACQMxJBAACAnFEaCgAAkAJdQwEAAMgsiSAAAEDOKA0FAABIga6hAAAAZJZEEAAAIGeUhgIAAKRA11AAAAAySyIIAACQM0pDAQAAUqBrKAAAAJllRxAA2GeVUZ/aXPUpfk6dFOalNldF1Q2pzZUUvpjaXAB7QyIIAACQAl1DAQAAyCyJIAAAQM4oDQUAAEiBrqEAAABklkQQAAAgZ5SGAgAApEDXUAAAADJLIggAAJAzSkMBAABSoGsoAAAAmSURBAAAyBmloQAAACnQNRQAAIDMkggCAADkjNJQAACAFOgaCgAAQGZJBAEAAHJGaSgAkCmVUZ/aXFteT+8z76TwxdTmqqi6uPnrJXNKG1e4JY0lASnQNRQAAIDMkggCAADkjNJQAACAFOgaCgAAQGZJBAEAAHJGaSgAAEAKdA0FAAAgsySCAAAAOaM0FAAAIAW6hgIAAJBZEkEAAICcURoKAACQAl1DAQAAyCyJIAAAQM4oDQUAAEiBrqEAAABklkQQAAAgZ5SGAgAApKAtdQ2VCAIA7VaX6vrU5qpPsZAqKdyS2riKqn/c1+W845q3pjofUH5r166Nwju+MCgRBAAAaMcmTpwYa9asaXBMIggAAJCCJKmPJCn3Kt7y5loqo7a21o4gAABAnvTq1avRsZKL3ZcsWRJnn3121NTUxEc/+tG4+uqro66ursGY1atXxz/90z/F0KFDY+jQoTFt2rRYt27dvq8cAACA1JS0I7hkyZKYMmVKDBw4ML70pS/F2rVrY/78+fHHP/4xamtro7KyMtavXx/nn39+bN++PS688MIoFAoxb968WL58eSxcuDA6duzY0r8LAABAGTXxBPey2fXeX0mJ4HXXXRe9evWKO++8M6qrqyPize3Fq666Kh555JE44YQT4nvf+1789a9/jR/96EfRv3//iIg4+uijY/LkyfHDH/4wJkyYkNIvAgAAwL5otjR027Zt8Z73vCcmTJhQTAIjIoYOHRoREcuXL4+IiMWLF8fQoUOLSWBExHHHHRf9+vWLxYsXp71uAACAjKmPN3cFs/Kz+0foNLsj2KlTp5g3b16j40uXLo2IiN69e8eGDRti9erVMWrUqEbjBg4cGA8//HBzlwEAAKCV7HHX0DVr1sTjjz8e1157bQwYMCBOPvnkeO655yIiomfPno3G9+jRIzZt2hSbNm2K/ffff99XDAAAwD7Zo0Twtddei+HDh0dEROfOnWP69OnRqVOnYvfQzp07N3pNp06dIiJiy5YtEkEAAKAdK0REhh4kGBW7PbNHiWBFRUXMmjUrtm/fHgsWLIjJkyfHrFmzokePHiW9do9UlvBki1LG0DLEvnzEvrzEv3zEvnzEPiL24Jlbac5WQuyT5PZ9XAu75L6nndujRPCAAw6I0aNHR0TEKaecEmPHjo0ZM2bErbfeGhFvNpZ5p53Hunbtumcrq9/9Fxsj4s03Z3NjaBliXz5iX17iXz5iXz5iX1SfYipY2UQDh7cGlRb7iqp/TGFFb0kKt6Y6X5vkvm8ZkutM2eu/RnV1dQwbNizWrl0b733veyMi4uWXX2407qWXXopu3bpFly5d9n6VAAAAmVefwZ9dazYRXLlyZQwfPjxqa2sbnaurq4uKioro2LFj9O3bN5566qlGY55++ukYNGhQc5cBAACglTSbCB566KGxadOmuOeee2L79u3F42vWrImf/vSnMWTIkOjatWuMHDkylixZEitXriyOeeyxx2LVqlXFclIAAADKryJJkmbb2jzwwAMxbdq0GDx4cHz84x+P9evXR21tbbzxxhtx1113xYABA2LdunUxduzYqKqqiilTpsS2bdti7ty5ccghh8Q999wTHTt23LOV+Y5gdol9+Yh9eYl/+Yh9+Yh9ke8I5oj7vmXk4DuChx22MZ57Ljv3zqGHVsazz3bb5bmSEsGIiAcffDDmzp0bf/rTn6JLly7xoQ99KKZOnRr9+vUrjvnzn/8cM2bMiN/+9rdRXV0dJ5xwQkybNi26d+++56uWCGaX2JeP2JeX+JeP2JeP2BdJBHPEfd8yJIKtLpVEsNVJBLNL7MtH7MtL/MtH7MtH7Iskgjnivm8ZEsFW11QiuEePjwAAAGB3CtFUp87Wt/s9P4kgAEAJStrFK1Epu4uVJY5LewevouqLqc2VFG5IbS4gXe1/fxYAAIAG7AgCAACkou2UhtoRBAAAyBmJIAAAQM4oDQUAAEhFfbxZHpp9dgQBAAByRiIIAACQM0pDAQAAUlEf2eoaWrHbM3YEAQAAckYiCAAAkDNKQwEAAFJRCF1DAQAAyCSJIAAAQM4oDQUAAEhF1h4or2soAAAAfyMRBAAAyBmloQAAAKnIWtdQpaEAAAD8jR1BAIBWVhn1JY0qZdyW19P9XD8p3JDaXBVV/5jaXEnh1tTmAiSCAAAAKVEaCgAAQEZJBAEAAHJGaSgAAEAqkoiSvgPcWpLdnrEjCAAAkDMSQQAAgJxRGgoAAJCKrHUN3f2+nx1BAACAnJEIAgAA5IzSUAAAgFQoDQUAACCjJIIAAAA5ozQUAAAgFfWRrdLQqt2esSMIAACQMxJBAACAnFEaCgAAkIqsdQ3d/VrsCAIAAOSMRBAAACBnlIYCALRhXarrU52vPsV9gqRwa2pzVVR9MbW5ksINqc0FDdX/7Scrdr8WO4IAAAA5IxEEAADIGaWhAAAAqcjaA+WVhgIAAPA3EkEAAICcURoKAACQCg+UBwAAIKMkggAAADmjNBQAACAVuoYCAACQURJBAACAnFEaCgAAkApdQwEAAMgoiSAAAEDOKA0FAABIRX001amz9ekaCgAAwN9IBAEAAHJGaSgAAEAq2s4D5SWCAAAUVab4/ab6FIvPksINqc1VUTW16Wsl/9LsmOLYwqw0lgStTmkoAABAztgRBAAASIUHygMAAJBREkEAAICcURoKAACQCqWhAAAAZJREEAAAIGeUhgIAAKQiiaYe4t76kt2esSMIAACQMxJBAACAnFEaCgAAkApdQwEAAMgoiSAAAEDOKA0FAABIhdJQAAAAMkoiCAAAkDNKQwEAAFJRH9kqDd39w+0lggAAAO3Y2rVro1BomKBKBAEAaBGVTexG7Kn6FL/RlBRmpTImIqKi6uJ9Xc7brnlLanPB202cODHWrFnT4JhEEAAAIBXZ7BpaW1trRxAAACBPevXq1eiYrqEAAAA5U/KO4JIlS2L27NmxbNmy6Nq1a5xyyilxySWXxH777Vccc/rpp8cf/vCHRq8dNWpUzJ49O50VAwAAZFJ9NNWps/XtY9fQJUuWxJQpU2LgwIHxpS99KdauXRvz58+PP/7xj1FbWxuVlZWRJEmsXLkyRowYESNHjmzw+j59+uzb+gEAAEhNSYngddddF7169Yo777wzqqurI+LNOtOrrroqHnnkkTjhhBPi+eefjy1btsRJJ50U48aNa9FFAwAAsPea/Y7gtm3b4j3veU9MmDChmARGRAwdOjQiIpYvXx4REc8880xERPTv378l1gkAAJBxOx8on5WffSgN7dSpU8ybN6/R8aVLl0ZERO/evSMiYsWKFRHxViK4ZcuW6NKlS3PTAwAA0Mr2uGvomjVrYtGiRXH11VfHgAED4uSTT46INxPB/fbbL2bMmBE1NTVRU1MTI0aMiMWLF6e+aAAAAPbeHj1H8LXXXovhw4dHRETnzp1j+vTp0alTp4h4szS0rq4uNm3aFDNnzoyNGzfG/Pnz49JLL4033ngjxo8fv2crqywhRy1lDC1D7MtH7MtL/MtH7MtH7MtH7IvSjUR6/52ZJHP2cS20L9l8oPyuVCRJkpQ6zYYNG+LRRx+N7du3x4IFC2Lp0qUxa9asGDVqVNx9991RX18fEydOLI5//fXXY+zYsbF169b49a9/HVVVVaWvub6ZtquVlc2PoWWIffmIfXmJf/mIffmIffmIfQP1KaaClc2199+D2FdUXZzCit6UFG5Jba5MysEHG4cddk8899zmci+j6NBDu8azz561y3N79Nc44IADYvTo0TF+/Piora2N3r17x4wZMyIi4uyzz26QBEZEVFdXx7hx4+KVV14pNpMBAACgvPY6La+uro5hw4bF2rVrY926dbsd171794h4s3kMAABA+1XuLqG7+tm1ZhPBlStXxvDhw6O2trbRubq6uqioqIitW7fGmDFj4uabb240ZtWqVRER0bdv3+YuBQAAQCtoNhE89NBDY9OmTXHPPffE9u3bi8fXrFkTP/3pT2PIkCHRp0+f2LhxYyxcuDA2b36rJvaFF16IRYsWxbHHHhs9evRomd8AAACAPdJs19AOHTrE9OnTY9q0aXHuuefGxz/+8Vi/fn3U1tZGZWVlfPWrX42IiCuuuCIuvvjiOOuss+KMM86Iurq6qK2tjQ4dOsQVV1zR4r8IAABAee18oHxW7L7pUcldQx988MGYO3du/OlPf4ouXbrEhz70oZg6dWr069evOOYXv/hF3HbbbbFs2bKorq6OoUOHxqWXXlp8yPyerVnX0MwS+/IR+/IS//IR+/IR+/IR+wZ0DW0HctE1dEE899ymci+j6NBD949nnz13l+f26PERrUoimF1iXz5iX17iXz5iXz5iXz5i34BEsB2QCLa6phLBPXqgPAAAlEOzydseaC6prCxhzE5pJm8VVdNSmyspzExtLvZEfTRVjtn6dr+W9p+WAwAA0IAdQQAAgFQ0/ey+1rcPzxEEAACgfZEIAgAA5IzSUAAAgFS0necI2hEEAADIGYkgAABAzigNBQAASIWuoQAAAGSURBAAACBnlIYCAACkQtdQAAAAMkoiCAAAkDNKQwEAAFKhaygAAAAZJREEAADIGaWhAAAAqaiPpjp1tj5dQwEAAPgbO4IAAORKZbM7NpUljHlTfYr7KklhZmpzVVR9PrW5ksLs1OYiOySCAAAAqfBAeQAAADJKIggAAJAzSkMBAABS4YHyAAAAZJREEAAAIGeUhgIAAKRCaSgAAAAZJREEAADIGaWhAAAAqUiiqYe4t75kt2fsCAIAAOSMRBAAACBnlIYCAACkQtdQAAAAMkoiCAAAkDNKQwEAAFKhNBQAAICMkggCAADkjNJQAADYS5UpPjy8PsU9mqQwO7W5Kqq+uM9zHHroe+LZZ7+Wwmqyrj6yVRq6+/vTjiAAAEDOSAQBAAByRmkoAABAKnQNBQAAIKMkggAAADmjNBQAACAV9dFUp87Wp2soAAAAfyMRBAAAyBmloQAAAKnwQHkAAAAySiIIAACQM0pDAQAAUuGB8gAAAGSURBAAACBnlIYCAACkQmkoAAAAGSURBAAAyBmloQAAAKloOw+Uz2wi+JfnK2PHjt2fP/zwiD8/2/yG5uGH7f6XBwCArKhs4j/a91R9ioV/SeGG1OYiO5SGAgAA5ExmdwQBAADaFl1DAQAAyCiJIAAAQM4oDQUAAEhFfTTVqbP17X4tdgQBAAByRiIIAACQM0pDAQAAUtF2HihvRxAAACBnJIIAAAA5ozQUAAAgFR4oDwAAQEZJBAEAAHJGaSgAAEAqdA0FAAAgoySCAAAAOaM0FAAAIBX10VQ5Zuvb/VoymwhWVTU/pkNmVw8AAORN375/V+4lNLBzPWvXro1CoeF3FyuSJEnKsSgAAABa1uuvvx7HH398bNiwocFxiSAAAEA7tXHjxti4cWOj4xJBAACAnNE1FAAAIGckggAAADkjEQQAAMgZiSAAAEDOSAQBAAByRiIIAACQMxJBAACAnJEIAgAA5EyHci9gT61evTquvfbaeOKJJyIiYtiwYXHZZZdF9+7dy7yy9u/000+PP/zhD42Ojxo1KmbPnl2GFbV/X/3qV+PZZ5+NBQsWNDjufdA6dhd/74X0PfLIIzFnzpx46qmnorKyMo4++ui45JJLYvDgwcUx7vuWUUrs3fMtZ8mSJTF79uxYtmxZdO3aNU455ZS45JJLYr/99iuOce+3jFJi796nPWtTieD69evj/PPPj+3bt8eFF14YhUIh5s2bF8uXL4+FCxdGx44dy73EditJkli5cmWMGDEiRo4c2eBcnz59yrSq9m3hwoVx3333xdChQxsc9z5oHbuLv/dC+p544on49Kc/He973/ti6tSpsWPHjrjrrrvinHPOibvuuiuOOuoo930LKSX27vmWs2TJkpgyZUoMHDgwvvSlL8XatWtj/vz58cc//jFqa2ujsrLSvd9CSom9e592L2lDvv3tbyfvf//7k2eeeaZ47NFHH00GDBiQ3HvvvWVcWfv3l7/8JRkwYEBy//33l3sp7d6OHTuSm266KTniiCOSAQMGJOecc06D894HLau5+HsvpG/cuHHJsGHDki1bthSPvfzyy8mQIUOSSZMmJUnivm8ppcTePd9yPvGJTyQnnnhisnXr1uKxO++8MxkwYEDy0EMPJUni3m8ppcTevU9716a+I7h48eIYOnRo9O/fv3jsuOOOi379+sXixYvLuLL275lnnomIaBB70rdt27b4xCc+ETfddFOMGzcuevbs2WiM90HLKSX+3gvp2rBhQyxbtixOOeWU6Ny5c/H4gQceGEOGDInf/e53EeG+bwmlxt493zK2bdsW73nPe2LChAlRXV1dPL6zCmH58uUR4d5vCaXG3r1Pe9dmSkM3bNgQq1evjlGjRjU6N3DgwHj44YfLsKr8WLFiRUS89S/DLVu2RJcuXcq5pHZp27ZtsXnz5pg1a1aMHj06hg8f3uC890HLai7+Ed4LaevatWv85Cc/aZCI7LR+/fqoqqpy37eQUmIf4Z5vKZ06dYp58+Y1Or506dKIiOjdu7d7v4WUEvsI9z7tX5vZEXzxxRcjInb5CX2PHj1i06ZNsWnTptZeVm6sWLEi9ttvv5gxY0bU1NRETU1NjBgxwqeRKevatWv87Gc/i9GjR+/yvPdBy2ou/hHeC2mrqqqKww47rNE9vWzZsnjyySejpqbGfd9CSol9hHu+taxZsyYWLVoUV199dQwYMCBOPvlk934r2VXsI9z7tH9tZkewrq4uImKXn1x26tQpIt78pGb//fdv1XXlxTPPPBN1dXWxadOmmDlzZmzcuDHmz58fl156abzxxhsxfvz4ci+xXaisrIzKyt1/PuN90LKai3+E90JrqKuriy9/+csREfGZz3zGfd+K3hn7CPd8a3jttdeKFQidO3eO6dOnR6dOndz7rWB3sY9w79P+tZlEMEmSZsdUVFS0wkryacKECVFfXx8TJ04sHhszZkyMHTs2rrvuujj11FOLZUS0HO+D8vNeaFlbt26Nz372s7Fs2bK46KKLYujQofHkk082+zr3/b7bVewj3POtoaKiImbNmhXbt2+PBQsWxOTJk2PWrFnRo0ePkl7L3ttd7EeNGuXep91rM6WhO2uyt23b1ujczmNdu3Zt1TXlydlnn93gX4QREdXV1TFu3Lh45ZVXil+opmV5H5Sf90LL2bhxY0yZMiUef/zxOO2002Lq1KkR4b5vDbuLfYR7vjUccMABMXr06Bg/fnzU1tZG7969Y8aMGe79VrC72Ee492n/2kwiuPOLuy+//HKjcy+99FJ069bNF3jLYOfDbLds2VLmleSD90F2eS/sm1dffTXOO++8ePLJJ+PMM8+Mq6++urjT4b5vWU3Fvinu+ZZRXV0dw4YNi7Vr18Z73/veiHDvt5a3x37dunW7Hefep71oM4lgt27dom/fvvHUU081Ovf000/HoEGDyrCqfHjxxRdjzJgxcfPNNzc6t2rVqoiI6Nu3b2svK5e8D8rLe6FlbN68OS644IJYunRpTJo0Ka666qoGiYj7vuU0F3v3fMtZuXJlDB8+PGpraxudq6uri4qKiujYsaN7vwWUEvutW7e692n32kwiGBExcuTIWLJkSaxcubJ47LHHHotVq1Y12eWPfdOzZ8/YuHFjLFy4MDZv3lw8/sILL8SiRYvi2GOPLel7DKTD+6B8vBdaxlVXXRVLly6N8847Ly6//PJdjnHft4zmYu+ebzmHHnpobNq0Ke65557Yvn178fiaNWvipz/9aQwZMiS6du3q3m8BpcS+T58+7n3avYqklO4TGbFu3boYO3ZsVFVVxZQpU2Lbtm0xd+7cOOSQQ+Kee+6Jjh07lnuJ7dYvfvGLuPjii+N973tfnHHGGVFXVxe1tbXxxhtvxN133+1hqy1k+PDh0adPn1iwYEHxmPdB69lV/L0X0rVy5coYPXp0dOvWLS6//PJdNl4YN26c+74FlBp793zLeeCBB2LatGkxePDg+PjHPx7r168vxvauu+6KAQMGuPdbSCmxd+/T3rWpRDAi4s9//nPMmDEjfvvb30Z1dXWccMIJMW3atGK9Ni3nF7/4Rdx2222xbNmyqK6ujqFDh8all17qX4QtaFeJSIT3QWvZXfy9F9Jz9913x9e//vUmxyxfvjwi3Pdp25PYu+dbzoMPPhhz586NP/3pT9GlS5f40Ic+FFOnTo1+/foVx7j3W0YpsXfv0561uUQQAACAfdOmviMIAADAvpMIAgAA5IxEEAAAIGckggAAADkjEQQAAMgZiSAAAEDOSAQBAAByRiIIAACQMxJBAACAnPn/YovtX1B9RgoAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_175_1.png" } }, "output_type": "display_data" } ], "source": [ "clf_lasso = skl.Lasso(alpha=_lambda).fit(X_train, y_train)\n", "J_lasso_sk = clf_lasso.coef_.reshape(L, L)\n", "fig = plt.figure(figsize=(20, 14))\n", "im = plt.imshow(J_lasso_sk, **cmap_args)\n", "plt.title(\"Lasso from Scikit-learn\", fontsize=18)\n", "plt.xticks(fontsize=18)\n", "plt.yticks(fontsize=18)\n", "cb = fig.colorbar(im)\n", "cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is quite striking how LASSO breaks the symmetry of the coupling\n", "constant as opposed to ridge and OLS. We get a sparse solution with\n", "$J_{j, j + 1} = -1$.\n", "\n", "\n", "\n", "\n", "We see how the different models perform for a different set of values for $\\lambda$." ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\r", " 0%| | 0/10 [00:00" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_177_13.png" } }, "output_type": "display_data" } ], "source": [ "lambdas = np.logspace(-4, 5, 10)\n", "\n", "train_errors = {\n", " \"ols_sk\": np.zeros(lambdas.size),\n", " \"ridge_sk\": np.zeros(lambdas.size),\n", " \"lasso_sk\": np.zeros(lambdas.size)\n", "}\n", "\n", "test_errors = {\n", " \"ols_sk\": np.zeros(lambdas.size),\n", " \"ridge_sk\": np.zeros(lambdas.size),\n", " \"lasso_sk\": np.zeros(lambdas.size)\n", "}\n", "\n", "plot_counter = 1\n", "\n", "fig = plt.figure(figsize=(32, 54))\n", "\n", "for i, _lambda in enumerate(tqdm.tqdm(lambdas)):\n", " for key, method in zip(\n", " [\"ols_sk\", \"ridge_sk\", \"lasso_sk\"],\n", " [skl.LinearRegression(), skl.Ridge(alpha=_lambda), skl.Lasso(alpha=_lambda)]\n", " ):\n", " method = method.fit(X_train, y_train)\n", "\n", " train_errors[key][i] = method.score(X_train, y_train)\n", " test_errors[key][i] = method.score(X_test, y_test)\n", "\n", " omega = method.coef_.reshape(L, L)\n", "\n", " plt.subplot(10, 5, plot_counter)\n", " plt.imshow(omega, **cmap_args)\n", " plt.title(r\"%s, $\\lambda = %.4f$\" % (key, _lambda))\n", " plot_counter += 1\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see that LASSO reaches a good solution for low\n", "values of $\\lambda$, but will \"wither\" when we increase $\\lambda$ too\n", "much. Ridge is more stable over a larger range of values for\n", "$\\lambda$, but eventually also fades away.\n", "\n", "\n", "To determine which value of $\\lambda$ is best we plot the accuracy of\n", "the models when predicting the training and the testing set. We expect\n", "the accuracy of the training set to be quite good, but if the accuracy\n", "of the testing set is much lower this tells us that we might be\n", "subject to an overfit model. The ideal scenario is an accuracy on the\n", "testing set that is close to the accuracy of the training set." ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAABKUAAANBCAYAAAAxzw1RAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAEAAElEQVR4nOz9eZxkZ33ffX/PqX3r6nW6p6eXGY1mAQmDBQwgL9iYxWAb2Y4gsQkIYUnJ7fA4eeyXZfvl5blj374dO84rsZwER0hGLHawgYCMBcYLAS/CRBgDRkI9o5np7uqZ6X2rfTvn+aNnqvucqp61u04tn/fr9j1zrlNV/JSaWs63rut3GbZt2wIAAAAAAACayPS6AAAAAAAAAHQfQikAAAAAAAA0HaEUAAAAAAAAmo5QCgAAAAAAAE1HKAUAAAAAAICmI5QCAAAAAABA0xFKAQAAAAAAoOn8XhfQStbWsrIs2+sy0OUGBuJaWcl4XQbQ1XgdAt7iNQh4j9ch4K1OeQ2apqG+vtiu5wmldrAsm1AKLYF/h4D3eB0C3uI1CHiP1yHgrW54DbJ8DwAAAAAAAE1HKAUAAAAAAICmI5QCAAAAAABA0xFKAQAAAAAAoOkIpQAAAAAAANB0hFIAAAAAAABoOkIpAAAAAAAANB2hFAAAAAAAAJqOUAoAAAAAAABNRygFAAAAAACApiOUAgAAAAAAQNMRSgEAAAAAAKDpCKUAAAAAAADQdIRSAAAAAAAAaDpCKQAAAAAAADQdoRQAAAAAAACajlAKAAAAAAAATUcoBQAAAAAAgKYjlAIAAAAAAEDTEUoBAAAAAACg6QilAAAAAAAA0HSEUgAAAAAAAGg6QikAAAAAAAA0HaEUAAAAAAAAmo5QCgAAAAAAAE3XkqHUL//yL+ud73zndd02lUrpve99r06dOqVTp07p4Ycf1urq6j5XCAAAAAAAgFvh97oAt4997GP64z/+Y506deqat11bW9N9992nUqmkBx54QNVqVY8//rimpqb0sY99TMFgsAkVAwAAAAAA4Ea1TChVrVb1vve9T//1v/7X677PE088ofn5eX3605/W0aNHJUkvfelLdf/99+tTn/qU3v72t+9XuQAAAAAAALgFLbF8r1gs6kd+5Ef0u7/7u7rnnns0PDx8Xfd76qmndOrUqVogJUl33323jhw5oqeeemq/ygUAAAAAAMAtaomZUsViUZlMRv/5P/9nveUtb9HrXve6a95nY2NDqVRKb3rTm+rO3XHHHfriF7+4H6W2tEsbi/rm4kWVrarXpeAWBM75VS5XJEmWLc2VbmwZqnH5z5BhazhYbnibS6WAyrZRN14/Un9+PFRqeG617FfWurmc+2CwrKBp142nq6bWKr6GdVxLv7+iuM+qGy9ahhbLgWvev9H/RtxXVZ+//vVl2dKFHc+T37AVNi1FTFsRs6qYz1LMtBT3VRU1LYVMW8b1/EfAM+XNkHLZotdlYL/t6QuxNR/LaMn/xms/jpWLKJutyDB9Mgy/DNMvw9j6uxxj/u3b7DzHmywAAG2hJUKpeDyuP//zP5fff/3lLCwsSFLDWVVDQ0NKp9NKp9NKJBJ7Vmcrm99c0vtOL6uimNeloIWcLkT2/DHPF8Nt8Zhn9/wR95otQ7b8qiipjIJGWQGVFVBFwct/FhSUKUthFRVWSVEVFDHyCqmkkLF1m4DK8ski5NpjG14XAHS59Vu8v121pSv/V7FlW9t/V9V2nq+6xna5Td19Ktdxm/rfRoC2cdrrAoAulxo7pIF/8S8VPfkir0vZVy0RSpmmKdO8sRkW2WxWkhSJ1F90h0IhSVIul+uaUOrc6oIqCnldBoDrZsiWobKCWla/VD9RrLGGt7NlypIpSz5Z8quiXm06gquAKgoaZZm2paBKChulrXNGWcEdtwmoItO43mIAoDUZPkPybaf1XuX2tTDsKsGVvSMIu2oYVncbSVVLqkr25T9VuXxsOW+vKu/rANBu8nMXtPDhJ3Tk13/T61L2VUuEUjfDtq/94XqjU7cHBuI3W47nvl2T+rOlOVXa9ykFcNOMy3GUTxVJRYWUVaw+wLJ3+XvDR6vKp2otqAqqrJhyChnlrZlcRsU1s6usYN1YRT5VmcUFoGsZpiGZhrRj1bhnAVmj2WNXCcOuZ4aZ+zF3jtnr5a0xAMBN8/lMDQ119kSbtk0wotGopK1+VG5XxuLxGwuZVlYysqz2/PCMKKH7Ixl9feasylXmirebsumXZZgKVkuOL6u2DF2Kj24P3MA32UC1pMH8suOxrliKHlDFvJGX/5X72jqYuVR31jak9VCf8oHdlgvuXrgtaTC3qIBVqTuXDcS0GUpe87EavWp7i2uKVPJ14yUzqJXo0K71XE20nFWyuFH3v2fJ0MKO56lq+lQx/aoYflmmT1XDlG2YW89BGyQ0tkxVZKqigOreYW3Xn9d8MEs+qyK/VVXAKilglRWslBS0igpVigpaJQWqZQWssgLVkoKX/ww4/ixv3bdalnnd/8PAHmnFlkzX+0DNawG1949lXJ7p5DO2Zz35Lx+b2393nPcZkk8y/C2xj0/LafbsMbtkqfIPa6p+lQXRAHAzImNjGvgX79DSUtrrUm6JaRpXnQDUtqHU6OjWBeDS0lLducXFRfX09NSCq25x5M5X6sidr/S6DNyEv5lf02dTy4r7fbp9IK6X9ET1ot72nbmH3VUsW/lKVZvlitLlqmRLsYBPRctSqWqpWLVUtLb+/OrypkpVW2XbUsWyVbXt9luBYZiq+oKq+qSibr3Hmc+QgqapkM9U2Of8M+QzFTJNBXeMX7nt1jlDId/l86apgGnsOqN2aCjR9l8AgHZ2K69B27Yluyrbrsi2rvxZkX15TFbFde7yn5f/rtrtr5zb+RgVyXX7nX9qx/+ObVV0/Ql+5zGCpgKvGVDiTa/SwOQPy+fvru/lnYDPQsBb3fIabNtQqqenR2NjY3r22Wfrzj333HO68847PagKuDmpTEGSlKlU9bWFDY2FbmzHPbQPv2koEfQrEbz22+9rD/Y3HC9blgpVS4XK1p/ZylbAlStXla1s7Q54tCfaIOiy9fWVTW2W23eHzqot5auW8nswI9SQFLwcVIV9vsvhlaGQz6e78kWdCAdltsHMNgBOhmFIhl+G/FL9Bq5NZdvWdsBlVxsEYjvPucYahF91t7GrDcKyat1jyPbufb+w+YLmn39Ug4f/mULxcc/qAAC0prYNpSTpjW98oz70oQ/p7NmzOnr0qCTp6aef1vnz5/UTP/ETHlcHXL8rodQVE7G9340OnSNgmgqYphKBa9/W7c3jg7JtW2XL3gq2qpYK1Wrt7/nKVrCVLVeVq2wdF6qWSpal7xrp2xF02bXxs5s5rZfql1+2OltS0bJVtKp1Qd2zaxn9wPigvmOkz5viAHQEwzBl+IKSvP2xaffZY9shlnadPbYVatWHZY0Csa3bl4srsqvbC8Cr5U0tnPmgekdfp8SB19xw31cAQOdqm1AqlUrpq1/9qu666y6Nj2/9yvLggw/qySef1Lvf/W695z3vUbFY1GOPPaY77rhD99xzj8cVA9dno1TWRnn7gj5gGhqJspMi9o9hGAr6DAV9pnr24PHWimVtlipXCbm2/sxfmbV1OcwqW7a+Z6RPfp+5PX55VtdCvqjVordB19/MrxFKAegIzZ49ViltaHn6Eypl53aMWlq/+JcqZmc1MHGPTP+tL+kGALS/tgmlnnnmGf3CL/yCfuM3fqMWSvX39+sjH/mIfuM3fkOPPPKIwuGwXv/61+vhhx9WMMjyJ7SHWdcsqclkVH6TXxDRPvpCAfWFbnzalmVvtd9v9It5KlPQ2c2cM+iqWJeX7m39vXg52HK7ozeuZMjvCMCKVUubpYrWbmBG12a5qmLVUshH02QAuBH+YFLDx+7T+sW/Unrx7x3n8hundWnqUQ0evleh2CGPKgQAtArDtu3u7cDo0s6776F9fWZ2SX+7sF47fuORA/qeQfeOcwAasWxbxVpwtfV/Q+GA4oH631xSmYL+8sJK3YyuRsHWFe+8fUQv6uvsbXiBVtQtzV27QW5jSiszT8quOn+Ek2Gqb/QNig+dYjlfi+J1CHirU16DHbv7HtApUlnnl7TbemMeVQK0H9MwFPH7FPFfez3KeDys+0/U/ypftWwVra3G8b/77IyKO0KqZ9eyhFIAcAuiyRMKnnhIy9MfVyl3cfuEbWntwudUyM5qYOKHZPropwkA3Yg1CYCHKpatC9miY+wIoRTQVD7TUNTvU384oOGIs5/bdCbvUVUA0Dn8oV4NH7tf8aFTdefy69/S/PPvVyl3yYPKAABeI5QCPDSfL6qyYwVtT8Cv/gj90ACvHO1xNt5dL1VUZVk3ANwyw/Spf+z7NXjkbTJM5w8AldKa5k//vtJLz4jOIgDQXQilAA+5m5yPx5m6DnjpFUM9iuxobG7Z0qVc8Sr3AADciGjvizRy8kEFIiPOE3ZVa3Of1cr0/5JV5X0XALoFoRTgoZQrlJoglAI81RcK6mhP1DHGEj4A2FuBUL9Gjr9H8cFX1J3LrT+r+an3q5Sb96AyAECzEUoBHnI3OZ+IEUoBXpt0hcMzhFIAsOcM06/+8bdo4PCPyjCdrQsqxVUtnP59ZZa/ynI+AOhwhFKARzLlilaL5dqxaUijsdBV7gGgGQ4nnH2lptMFLooAYJ/E+u7UyIkHFQgPO8Ztu6LV1J9qZeZTsqolj6oDAOw3QinAI+5ZUgcjIQVMXpKA10aiIYV29JXKVqpa2REgAwD2ViA8oOET71Fs4NvrzuXW/knzU4+plF/0oDIAwH7jChjwyFgsrHuPDOvUUFIHoyFNumZnAPCGzzB0tC/mGJvezHlUDQB0B9MMaGDihzQw+cMyzIDjXKW4rIWpx5RZ+bpH1QEA9ovf6wKAbpUI+HXXYI/uGuyRJJYHAS0k7Pc5jr+1kdMrDvR6UwwAdJFY/7cpGDmo5emPq1xYqo3bdkWrs0+qmJlR3/ibZbqCKwBAe2KmFNAiDMPwugQAl60XnMv1Zl07ZQIA9k8gMqTh4z+hWP9L685lV7+mhanHVS4se1AZAGCvEUoBAODy4sGE4zhbqSpdrnhUDQB0H9MX1MDkPeqfeKsMw7m4o1xY1PzU+5Vd/SePqgMA7BVCKQAAXI71x+vGZtLMlgKAZosPvEzDJ35C/tCAY9y2ylqZ+aRWZ/9UtsWPBgDQrgilAABwOZyM1Y2dpdk5AHgiGBnWyIkHFO27s+5cZuWrmj/9+yoXVz2oDABwqwilAA/87fyavrWeUYblQEBLigZ86gk4l4ucTRNKAYBXTF9IA5M/or7xH5AM52YU5fy85p9/VLm15zyqDgBws9h9D2iyfKWqz6S2m3MOhAL6d3dOymfS6BxoJYcTYX1jNVM7XimUVapaCvr4PQcAvGAYhhKDL1coOqrl6U+osmN2lG2VtDz9ccUzr1TfoTfIMLnMAYB2wDdroMlSWWdfmoBpEEgBLehIIuo4tlX/+gUANF8welAjJx5UtPfFdecyy89o4fQHVCmueVAZAOBGEUoBTZZybS0/Hg97VAmAq2n02jxPXykAaAmmL6SBw/9MfWNvrlvOV8pf0qWpR5Vbf96j6gAA14tQCmgy90yLiRihFNCKhiNB+VyTGM+m894UAwCoYxiGEkOv1PDx++UL9jrO2dWils//sdbmPifbqnpTIADgmgilgCaybLvBTKmIR9UAuBqfYWgkGnKMzWULqtq2RxUBABoJRUd18MRDiiRP1J1LL31ZC2eeUKW03vzCAADXRCgFNNFKoax81aodh32mBsMBDysCcDW3uULjqi3N54oeVQMA2I3pD2vwyNvVe+iNcl/ilHIXNP/8o8pvnPamOADArgilgCZyL90bj4VlGjQ5B1pVo75SMxmanQNAKzIMQz0HXq3h4++WL5B0nLOqBS2d+6jWLvyFbJvlfADQKgilgCaapck50FYaLa89R7NzAGhpodiYRk4+pHDPsbpz6cUvafHMh1QpbXpQGQDAjVAKaKJGM6UAtK5k0K+Y37mr02wmL5u+UgDQ0nz+iIZu+xfqHf0+Sc5Z6cVsSvNTjyq/+YI3xQEAagilgCYpVa26XjTMlAJa37+9Y0IBc/uCJlOxtFaseFgRAOB6GIahnuHv0IFj98kXSDjOWZWcls7+odYvfl62be3yCACA/UYoBTTJXLagnXMrBsMBRV0zMAC0nnjQXzercTqT96gaAMCNCscnNHLiIYUTR+vObS78rRZf+LCq5bQHlQEACKWAJmHpHtC+JhPO3lIzhFIA0FZ8gZiGjv64kge/V3XL+TIzuvT8oyqkz3lTHAB0MUIpoEncTc4nWLoHtI3DrtfrdJod+ACg3RiGoeTId+nA7e+U6Y87zlmVrBZf+IjWL32B5XwA0ESEUkCTzOfd/aTqd/UC0Jom4hHH7+pLhZKyZbYUB4B2FE4c1sGTDykUP1J3bnP+r7X4wh+oWs54UBkAdB9CKaBJfvrOw/o3Lx7XWyeHdNdgQsORoNclAbhOIZ+pg9GQY4wlfADQvnyBuA7c/g4lR15bd66YOX95Od908wsDgC5DKAU0ic80dCgW1qsP9OreIyPyGca17wSgJaQyBW2WnDvuEUoBQHszDFPJg6/VgaP/UqY/5jhnVTJafOHD2pj/G9m2vcsjAABuFaEUAADXEPQZylScy/XObxJKAUAnCPfcppGTDykUn3SdsbVx6X9r6ewfqlrJeVIbAHQ6QikAAK5hKBxUyHTObryYK6pUpRkuAHQCfyChA7e/Uz3D31l3rpA+q/nn/4eKmVkPKgOAzkYoBQDANZiGoXHXDnyWpLksu/ABQKcwDFO9o6/T0NEfl+lzbkhTLae1cOaD2lx4muV8ALCHCKUAALgOY7Fw3dhMhlAKADpNpOd2jZx8SMHYmOuMrfWLf6mlcx9VtcISbgDYC4RSwD77ytKGPnzmor5wcVVnN3MqstwHaEsT8UahFBclANCJ/MGkho/dp8SB19SdK2ye0fzzj6qYnfOgMgDoLIRSwD47s5HTt9az+vMLK3p86oK+srThdUkAbkKjmVLT6bwslnEAQEcyDJ/6Dr1Bg7f9c5k+52dAtbyhhTNPaHPx71nOBwC3gFAK2GcpV8+ZiXhkl1sCaGXxgF99Qb9jrGTZWsiXPKoIANAM0eSJreV80VHnCdvS+oU/1/L5j8mqsJwbAG4GoRSwjzZLFa2XKrVjn2HoYDToYUUAboW72bm0NVsKANDZ/MFeDR+7X4mhV9Wdy288r0tTj6qYu+hBZQDQ3gilgH0062qCPBoNyW/ysgPa1XjDZueEUgDQDQzTp76xN2nwyNtl+EKOc9XSuhZOf0DppWdYzgcAN4CrY2Af1S/dq7+gBdA+Gs+UKnABAgBdJNp7UgdPPKRg5KDzhF3V2txntTL9CVnVojfFAUCbIZQC9lHKNYOi0SwLAO3jYDQkn+Ec2yw7l+kCADqfP9Sn4eP3Kz74yrpzufXnND/1fpVy8x5UBgDthVAK2CdVy9aFnPNXMmZKAe0tYJo6GA3VjbOEDwC6j2H61T/+Zg0c/mcyTGfP0EpxVfOnH1dm+R+YTQsAV0EoBeyT+XxRZWv7S0gi4FPStXMXgPbTaMbjdJpdlwCgW8X67tDIiQcViAw7T9hVraae0srMJ2VV2akVABohlAL2ibuf1HgsLMMwdrk1gHZxW09Uh1yzpZgpBQDdLRAe0PDx9yg+cFfdudzaNzU/9ZhK+UUPKgOA1kYoBewT9857LN0DOsMdfXG958Qh7YyYF/Il5StVz2oCAHjPNAPqn/hBDUz+iAwz4DhXKS5rYeoxZVa+5k1xANCiCKWAfZJyhVLj8YhHlQDYaxG/TyMRZ/+QmQxL+AAAUqz/JVvL+cJDjnHbrmh19k+0MvOkLKvsUXUA0FoIpYB9kC1XtVLc/rJhSnXLfQC0t8mEM2ieSbOEDwCwJRAe1PCJBxTrf1nduezq17Uw9ZjKhaXmFwYALYZQCtgH7n5SI9GQgj5ebkAnmXTNfpymrxQAYAfTDGhg8q3qn3irDMO52U25sKT5qceUXf2GR9UBQGvgKhnYB7ZsjUZDtRfYOP2kgI5zOOF8Xc9liypblkfVAABaVXzgZRo+8YD8oUHHuG2VtTLzKa3M/inL+QB0LfanB/bBi3rjelFvXKWqpQu5oiLMkgI6TiLgV8LvU/pyg/OqbetCtqjDCfrHAQCcgpEDGjnxgFZTTym39k+Oc9mVr6qUvaDBI/cqEB7wqEIA8AZXysA+CvpMHUlENEI/KaDj/PfnUrVA6ooZlvABAHZh+oIamPxh9Y//oGT4HOfKhQXNT71f2bVnPaoOALxBKAUAwE1w774nSdM0OwcAXIVhGIoP3qWR4z8hf6jfcc62SlqZ/oRWU5+RbVU8qhAAmotQCgCAmzDWoFfcTKYgy7Y9qAYA0E6C0RGNnHhQ0d476s5llr+i+dMfULm46kFlANBchFIAANyE8Vh9KFWoWlrMlzyoBgDQbkxfSAOHf1R9Y2+uX86Xv6T559+v3Pq3PKoOAJqDUArYYzlXjxkAnWkkElLANOrG6SsFALhehmEoMfRKjRy/X/5gn+OcbRW1fP5jWpv7nGyL75cAOhOhFLCHCpWqfv0fz+m3vzGtPzo7ry8trMtmKQ/QkXymodEGmxjMpAseVAMAaGfB6KhGTjyoSPJk3bn00pe1cOYDqhTXm18YAOwzQilgD81li7IlrRbL+vpqWl9e2pBh1M+kANAZGi3hm2amFADgJpj+sAaPvE29h94kGc7LtFLuoi5NParcxpRH1QHA/iCUAvbQbNY5Q2KiwQUrgM4x3qDZ+XqpovVi2YNqAADtzjAM9Rx4lYaP3S9fMOk4Z1cLWj73R1q78BeybZbzAegMhFLAHkq5Zkg0umAF0DkazZSStnbhAwDgZoVihzRy4iFFeo7XnUsvfkkLZz6oSmnDg8oAYG8RSgF7xLZtpVwzpXa7YAXQGZJBvxIBX904zc4BALfK549o8LZ/rt7RN0hytoMoZec0//yjym+c8aY4ANgjhFLAHlkplpWrWLXjkGnqQCToYUUA9pthGA3D55k0oRQA4NYZhqGe4ddo+Ni75Qv0OM5Z1byWzv1PrV/8K9m2tcsjAEBrI5QC9kjKtVxnPB6SSZNzoOM1WqY7ny+pUKHfBwBgb4Ti4xo5+ZDCPbfXndtc+DstvvAhVcppDyoDgFtDKAXsEXeT8/FYxKNKADRTo5lSturfEwAAuBU+f1RDt/2YkgdfJ/dyvmJmVvPP/w/lN896UxwA3CRCKWCP1M+Uop8U0A0OxcJqNCdymiV8AIA9ZhiGkiPfqQPH3iWfP+44Z1VyWjr7B1q/9AWW8wFoG4RSwB4oVS3N54qOMZqcA90h5DP1/eODunvYuXU3O/ABAPZLOD6pkZP/SuHEkbpzm/N/rcUXPqJqOeNBZQBwYwilgD1wIVfUzt+jBkIBxRrsyAWgM33XSJ++Y7jPMZbKFFSxbI8qAgB0Ol8gpqGj71By5LV154qZaV16/lEV0uc9qAwArh+hFLAHWLoHoDfoVzLgrx1XbFsXc8yWAgDsH8MwlTz4Wh24/V/K9Mcc56xKRosvfEQb838t2+ZHEgCtiVAK2AOprLN3DEv3gO5jGIYmE87X/nSaUAoAsP/Cidt08ORDCsUnXWdsbVz6gpbO/oGq5awntQHA1RBKAbfItm3NumZKTTBTCuhKk3HnrpszGZqdAwCawxdI6MDt71TP8HfVnSukz2l+6lEVMrMeVAYAuyOUAm6RLel7R/v17QMJDYYDCpiGRiIhr8sC4IHDifpQymLJBACgSQzDVO/o92ro6I/L9Ecd56rltBbPfFCbC3/Hcj4ALcN/7ZsAuBrTMPTqA7169YGt42LVks9stEE8gE6WKVc0nyvKZ0jVy9/1cxVLy4WyDkSC3hYHAOgqkZ7bNXLiIa1Mf0LFbGrHGVvrF/9KhcysBibvkc8VXAFAszFTCthjIR8vK6AbpTIFfez8Qi2QuoIlfAAAL/iDPTpw7D71HLi77lxh84zmn39UxeycB5UBwDaungEA2ANju/SSm04TSgEAvGEYpnoPvV5Dt/0LmT7nEvNqeVMLp5/Q5uKXWM4HwDOEUgAA7IFEwK/eYP2q+JkMO/ABALwVSR7XyMmHFIwecp2xtH7hL7R8/o9lVfgRBUDzEUoBALBHxmP1s6VWi2VtlioeVAMAwDZ/MKnhY+9WYujVdefyG1O6NPV+FbMXPKgMQDcjlAJuwVymoLObORWrltelAGgB47ss4aOvFACgFRimT31jb9TgkbfL8Dl3i66W1rVw5gNKL/0flvMBaBp23wNuwd8urOkbqxkZkoYjQb15fFDHkjGvywLgkUYzpSRpOl3QS/oTTa4GAIDGor0nFYwMa3n6EyrlLm6fsC2tzf2ZCpkZDfT9mHcFAugazJQCbkHqcq8YW9J8vsTOe0CXG42FZBr148yUAgC0Gn+oT8PH3q340Km6c/n1b+lbX/5dWRX6IgLYX1xBAzcpXa5obUefGJ8hHYyGrnIPAJ0uYJo6GKl/H7iUK7LMFwDQcgzTr/6x79fg4XtlmEHHuWJuSRvzX/SoMgDdglAKuEkp145aB6MhBUxeUkC3a9RXypY0y2wpAECLiva9WCMnH1IgMuIYz60/T38pAPuKK2jgJrlDqfFYxKNKALSS3fpKzWRYAgEAaF2BUL+Gj71bhrHddrha3lA5v+BhVQA6HaEUcJNms84LzIlddt0C0F1224FvOs1MKQBAazN9QYUTtznG8punPaoGQDcglAJuQtW2NecKpXa7EAXQXQZCAUUabHqQyhZUtVgCAQBobZHkccdxfoNQCsD+IZQCbsJCvqTyjovLuN+nvqD/KvcA0C0Mw2gYUpctW5dyRQ8qAgDg+rlDqVLuoirltEfVAOh0hFLATUi5GhaPx8MyjAb7wAPoSmOX+0r5XG8L0zQ7BwC0OF8grmB01DFW2DjjUTUAOh2hFHAT3E3O6ScFYKdXDiX10y+Z1PePDTrG6SsFAGgHkeQJx3FuY8qjSgB0OkIp4Ca4m5zvttsWgO6UDPo1GA7qcMK5K+dMpsDW2gCAludewldMn5dllT2qBkAnI5QCblCuUtVyYftD2ZB0iFAKQAMj0ZCC5vYavmylqpUiX+oBAK0tED6gYLivdmzbFRXS5zysCECnIpQCbpB76d5IJKhQg522AMBnGJqIO2dLsYQPANDqDMNQ8sCLHWPswgdgP3AlDdyglHvpnuuCEwB2mnT1nJtxBdsAALSi3qH6UIol6AD2GnvYAzfou0f6dLQnqlSmoFQ2r9t7CKUA7M7dV4qZUgCAdhDvu02GGZJtFSVJViWrUu6CQrExjysD0EkIpYAbFPSZOpKI6EgiIqnvmrcH0L3+6sKKzm7mHGMrxbLS5YoSAT6CAQCtyzT9ivQcVW79udpYfuM0oRSAPcXyPQAA9smZjZymGyzXm0mzhA8A0PoiyROOY/pKAdhrhFIAAOyT8XjjnTlnMizhAwC0vnDP7draa3pLubCoSnHNu4IAdBxCKQAA9sl4jFAKANC+fP6IQvEJxxizpQDsJUIpAAD2yW4zpS5miypVrSZXAwDAjYskjzuOc4RSAPYQoRRwnYpVSx89e0l/N7+mVKagisUFJYCr6w36FfP76sYtSaksfaUAAK0v0uMMpYqZGVlVPsMA7A1CKeA6zWUL+sZqRk+llvW+b6X0359LeV0SgBZnGIYmdpktNZ1mCR8AoPUFwgPyhwZ3jFjKb571rB4AnYVQCrhOKdcOWqOxkEeVAGgnu/eV4ldmAEB7cC/hy29MeVQJgE5DKAVcp1nXUpvxWMSjSgC0k7FdZkrNZvKq2naTqwEA4MbVhVKbL8i2qx5VA6CTEEoB18G27bqZUrstyQGAncZioR2baW8rWbbmc8Wm1wMAwI0KxcZk+rZ/kLWrBRUztLIAcOsIpYDrsFasKFvZ/jUoaBo6EAl6WBGAdhH2+TS0y/sFS/gAAO3AMEyW8AHYF4RSwHWYzTobEo/FwvIZjeY+AEC93fpK0ewcANAu6kOp07JZhg7gFhFKAdfBvXRvnKV7AG7Abst9ZzJ5vtADANpCOHGbZPhqx5XSmirFZQ8rAtAJCKWA6zDr7ie1y6wHAGhkbJf3jHS5qtViucnVAABw40xfSOH4YcdYfuO0N8UA6BiEUsA1lC1Ll/LOZsS77aYFAI0MR4KK+306nIgoGfQ7ztFXCgDQLhot4QOAW0EoBVzDxWxR1o7VNX0hvxIB/+53AAAX0zD08y87oodOjunlgz2Oc/SVAgC0C3coVcymVC1nPaoGQCcglAKuYTbr6ifF0j0AN8G8vDnCpGum5UyGUAoA0B78waQCkRHHWH7zBY+qAdAJCKWAa3A3OZ+IRzyqBEAnmIhHtHPvzqVCWZlyxbN6AAC4EfVL+KY8qgRAJyCUAq6hLpRiphSAWxDymToYDTnG3JspAADQqqKuUKqQPivb4scVADeHUAq4ikK1qoBve06D3zA04rqYBIAbNemacckSPgBAuwhEDsoXSNSObausQmbau4IAtDW6NQNXEfb59NMvOaxcpaq5bEEbpYr8pnHtOwLALmzb1mDY+fE7nWamFACgPRiGoUjPcWVW/qE2lt84rUjP7R5WBaBdEUoB1yHq9+l4MuZ1GQDa3DdXM3pyZlHZStUxfjFXUKlqKehjAjMAoPVFkvWhlD32ZhkGP94CuDF8+wUAoEliAV9dICVJVVuayzJbCgDQHsKJIzLMQO24Wt5UOT/vYUUA2hWhFAAATXIoGtr1g3eGZucAgDZhmH6FE7c5xvIbpz2qBkA7I5QCAKBJgj5z180SaHYOAGgnkeQJx3GOUArATSCUAnZh2bbXJQDoQGOxcMPxmUyB9x0AQNuI9BxzHJfzl1QpbXpUDYB2RSgF7OIT5xf0O9+c0f86v6CvLG0oU654XRKADjARbxxKFauWFvKlJlcDAMDN8QViCsbGHGP5TWZLAbgxhFLALmYyBS3kS/rK8qb+1/Silgtlr0sC0AF2myklSdNplvABANpHpOe445i+UgBuFKEU0ECmXNFqcTuEMg1pdJc+MABwIwbDAYV9jT9+6SsFAGgnkaQzlCqkz8uqMusXwPUjlAIaSLm2Zj8YCSm4y0UkANwI0zA0vstsqel0QTZ9pQAAbSIQHpI/2Lc9YFdVSJ/zriAAbadlrrJTqZTe+9736tSpUzp16pQefvhhra6uXvN+3/zmN3X//ffrZS97me666y7963/9r3XuHG+EuDUp19bs47v0gAGAmzG2y3vKZrmi9RL96wAA7cEwjLrZUvmNKY+qAdCO/F4XIElra2u67777VCqV9MADD6harerxxx/X1NSUPvaxjykYDDa837lz5/TOd75TkUhEP/mTPylJ+sAHPqAf//Ef15NPPqnh4eFm/megg8y6Q6mr9IABgBt1tfeU6XRefaFAE6sBAODmRZLHlV76cu04v3lGtm3JMFpm/gOAFtYSodQTTzyh+fl5ffrTn9bRo0clSS996Ut1//3361Of+pTe/va3N7zfBz/4QeVyOf3BH/yBXvziF0uSXv3qV+ttb3ubnnjiCf3cz/1c0/4b0Dks29aca/nebrtlAcDNuFooNZPJ69sHe5pYDQAANy8Un5DhC8muFiVJViWnUvaCQvFxjysD0A5aIr5+6qmndOrUqVogJUl33323jhw5oqeeemrX+83Nzamvr68WSEnSt33bt6m3t1enT7PzA27OYr6kkrXd0yXq96mfWQsA9lAssPv7yrRrpiYAAK3MMHyK9BxzjLGED8D18jyU2tjYUCqV0h133FF37o477tCzzz67630nJye1sbHh6D21vr6udDqtAwcO7Eu96HzupXsTsbAMw/CoGgCdarfZUov5knKVapOrAQDg5kV6XH2lNs94VAmAduN5KLWwsCBJDfs/DQ0NKZ1OK51ON7zvAw88oJGREf30T/+0nn/+eU1NTelnfuZnFAgE9M53vnNf60bncu+8R5NzAPvhxX0xvfpAUm87Mqwh16yp2Uzeo6oAALhxkZ6j2nlpWS4sqVy89qZVAOB5T6lsNitJikQidedCoZAkKZfLKZFI1J0fHR3Vv/pX/0q/9mu/pnvuuUeS5PP59MgjjziW9AE3oq7JOaEUgH3wkv6EXtK/9dmWyha0tLhROzedLuhkb9yr0gAAuCGmP6JQfELFzHRtLL9xWoEDr/auKABtwfNQyrbta95mt6VT/+W//Be9733v06lTp/T2t79d1WpVH/3oR/Xv/t2/0yOPPKLXve51N1TLwAAXAN0uV65oqVCqHRuSXjYxqEjA19Q6hobqQ1gAzdXM1+FLymX9/Y5Q6mKxxPsAuh6vAcB7N/I6tHLfprmp6dpxNXdWQ0Nv2IeqgO7RDZ+FnodS0WhUklQsFuvOXRmLx+vDos3NTT3++OO688479cQTT8jn2woNfuAHfkD33nuvfvmXf1nf+Z3fqWAweN21rKxkZFnXDsnQuc5sZB3HByJBZdZzyjSxhqGhhJaWGi9ZBdAczX4d9rs+es6v53RxYUMB0/NV9oAn+CwEvHejr8Oqb8JxnF47p4VLizL99StiAFxbp3wWmqZx1QlAnn/bHR0dlSQtLS3VnVtcXFRPT08tuNppenpapVJJP/iDP1gLpCQpEAjoh37oh7S8vKxz587tX+HoSHVL966ybTsA7JVkMKDe4PbvRFXb1oVs/Y81AAC0qkCoX4Hw0I4RW/nNs57VA6A9eB5K9fT0aGxsrOEue88995zuvPPOhve7MgOqWq3fociyLMefwPUajoR0R19cPZeX603QTwpAkxyOO39JnqHZOQCgzUSSrl34NqY8qgRAu/A8lJKkN77xjfrSl76ks2e3k/Snn35a58+f11ve8paG9zl27JgOHDigT37yk46lf8ViUZ/61KfU19enY8eO7Xvt6Cx39sf1jtsP6udfdpt+7qWHdUcffcYA7K90uaLn1jLKViqO8ek0oRQAoL3UhVLpF2Tb9ZMIAOAKz3tKSdKDDz6oJ598Uu9+97v1nve8R8ViUY899pjuuOOO2q56qVRKX/3qV3XXXXdpfHxcPp9Pv/Irv6Kf+qmf0r333qt7771XlmXpE5/4hM6dO6ff+q3fUiAQuMb/MrC7ZJB/PwD2l23beuSbs8pW6r+wz2QKsmxb5i6bfQAA0GqC0UMy/VFZlZwkya4WVczMKpw44nFlAFpVS8yU6u/v10c+8hGdPHlSjzzyiD74wQ/q9a9/vR577LHaMr1nnnlGDz/8sJ555pna/d7whjfo93//99Xb26v//J//s37nd35HPT09evTRR/XWt77Vq/8cAACui2EYu/auK1QtLeZLDc8BANCKDMNUpMe9hO+0R9UAaAeGbdtsN3cZu++hFXTKLgtAO2vm6/DzF1f1lxdWGp67Z3JIrzrQ25Q6gFbCZyHgvZt9HebWn9fy+T+uHfuCvRp98f9HBjN/gRvSKZ+FLb/7HgAA3Ww8Ftr13Ey6sOs5AABaUThxm2Rs745eLa2rXKjfaR0AJEIpQJJUtiylMgVV2LERQJON7bJ8T5Km2YEPANBmTF+wrocUS/gA7KYlGp0DXktlCnps6oJ8hqHRaEgv7ovptQf7vS4LQBeI+H0aCge1VKjvH7Veqmi9WFZviI0XAADtI5I8ocLmC7Xj/MaUkiPf6WFFAFoVM6UASans1hKZqm0rlS3QXBhAU43Hr7KEL8MSPgBAe4n0HHMcl3IXVC1nPKoGQCsjlAIkzbou+sbjuy+nAYC9ttsOfJI0wxI+AECb8Qd7FIwcdIzlN894VA2AVkYoha5n27ZSrlBq4ioXiACw164WSk2nCaUAAO0nkjzuOKavFIBGCKXQ9dZLFWUq1dpxwDQ0HN19KQ0A7LXhaEgBs/FW2Qv5kvI73qMAAGgH7lCqkD4nyyp7VA2AVkUoha7nXrp3KBaWz2h8cQgA+8FnGDq0Sxhuq/59CgCAVheIjMgX6Kkd21ZZxfR5DysC0IoIpdD1rjQ5v4KlewC8cLVedvSVAgC0G8MwWMIH4JoIpdD13P2kaHIOwAtjV+srxUwpAEAbqgulNs/Itm2PqgHQigil0NUqlqWLuaJj7GoNhwFgv0zsCMQDriXEc5mCKpbV7JIAALgl4fhhGWawdlwtp1XKX/KwIgCthlAKXe1irqjqjl9reoN+9QT9HlYEoFslgwH9s8MH9N47JvTLd92mZGD7vahi23UBOgAArc4w/QonbnOM5TemPKoGQCsilEJXczcPZukeAC+9fCip0WhIftPUZML5fjSdZgkfAKD9RJInHMf5jTMeVQKgFRFKoau5+0nR5BxAq5iMRxzHNDsHALSjSM/tkraXpZfz86qUNrwrCEBLIZRCV3PvvMdMKQCt4nCiPpSyaA4LAGgzvkBModiYY4xd+ABcQSiFrrVZqmi9VKkd+wxDo9GQhxUBwLbhSFAh3/bHdK5iablQ9rAiAABuTt0ufIRSAC4jlELXivhNvfv4qL5vtF/Hk1Ed7YnIb/KSANAaTMPQRMwZlLOEDwDQjtx9pQqZ87KqbOABQGKbMXStgGnqeDKm48mY16UAQM1Cvqhn17KayxQ07ep7N53O65VDSY8qAwDg5vhDA/KH+lUprm4N2JYKm2cV7Xuxt4UB8BzTQgAAaCGpTEF/eWFFz29kVbacPaRmMuzABwBoP4ZhKNLjWsK3yRI+AIRSAAC0lKttuLBaLGtzRy88AADaRX1fqTOybcujagC0CkIpAABayFA4qNBV+tvRVwoA0I5C8QmZvu0fXqxqXsXsnIcVAWgFhFIAALQQ0zA0Ft99J9DpNEv4AADtxzBMhXuOOcbyG1MeVQOgVRBKoSt9ZWlDzyxtaD5XlGXb174DADTReGz3JXzMlAIAtKv6JXz0lQK6HbvvoSt98dKaVoplSVLINPXQi8Z0MLr7zAQAaKarhVKXckUVq5ZCPn5XAgC0l0jPUW3Ni9jqJVUprqhcWFEgPOBpXQC8wzdadJ1cpVoLpCSpbFkaCAU8rAgAnMau0uzcljTLbCkAQBsyfWGFE5OOMWZLAd2NUApdJ+XaUn0kGlKQGQcAWkgi4FdfcPfJzNMZ+koBANpTJHnCcZzfpK8U0M24EkfXmc06L+autkwGALxytdlSM2lmSgEA2lOkx9lXqphJqVrJeVQNAK8RSqHrpFzLXsavcuEHAF6ZuEpgnsoWVLXYpAEA0H78oV4Fwgd2jNgqbL7gWT0AvEUoha5i2bZSmaJjbIJQCkALcgfmxo6/ly1bF3PO9zIAANoFu/ABuIJQCl1lqVBS0bJqxxGfSZNzAC3pYDQk344kyj0vaoZm5wCANlUXSm2+INuqelQNAC8RSqGruJucj8fDMgxjl1sDgHcCpqmD0dCu56fpKwUAaFPB6CGZ/ljt2LZKKmSmvSsIgGcIpdBVZl2hFEv3ALSysVhYpiEdioZ0Z1/ccW4mU5Bt01cKANB+DMNgCR8ASdLu+00DHSjFznsA2sjrDw3ozeODCpimqrat0189q9LlBufZSlUrxbIGw0GPqwQA4MZFkseVXfnH2nF+87Rs+/tZxQB0GWZKoWsUKlUt5ku1Y0OEUgBaW9TvU8Dc+qj2GYYm4hHHeZbwAQDaVThxmwxje45EtbShcmHRw4oAeIFQCl1jLlt0NAoeCgcV9vs8qwcAbtSka8nxjGtJMgAA7cI0AwoljjjG8htTHlUDwCuEUugas+6le/STAtBmDieYKQUA6BzR5AnHMX2lgO5DKIWusVIoOY5pcg6g3YzHwo4P7pViWelyxbN6AAC4FZHkMcdxKXdR1XLao2oAeIFG5+gab7ttRG8cG1AqU1AqW9AR14wDAGh1RctSfzig5UK5NjaTLujO/vhV7gUAQGvyBRIKRkdVyl2sjeU3zig+eJeHVQFoJkIpdJVkMKBkf0B39ie8LgUArotl2/qjc/NKZQpaL9XPiprJ5AmlAABtK5I87gqlThNKAV2E5XsAALQw0zB0KVdsGEhJW6EUAADtKuLqK1VIn5NllXe5NYBOQygFAECLG4/t3gPvYraoUtVqYjUAAOydQPiAfIFk7di2Kyqkz3lYEYBmIpQCAKDFuXcLDZpG7e+WpJRrd1EAANqFYRiKJI87xtiFD+gehFIAALQ490wp23V+Os0SPgBA+2oUStm2+9MOQCei0Tk63kw6r8+kljUeD2s8FtZkPKzeUMDrsgDguo1EQvIbhiqXv6CXLecXdfpKAQDaWTh+WIYZlG2VJElWJatS7oJCsTGPKwOw35gphY43kykolS3o6YV1/dG5ef353IrXJQHADfGZhg7FQruen80UVOUXZQBAmzJMnyI9tzvGWMIHdAdCKXS8VNY5g8DdmwUA2oF7CV/A2O4rVbJszeeKzS4JAIA9Q18poDsRSqGj2bat2YyzAfAEoRSANjTmeu8K+AzHMX2lAADtLNxzu6Ttz7ZyYVGV4rpn9QBoDkIpdLSNUkXpcrV27DcMjUR2XwIDAK1qwjVTqlC1HMczGXbgAwC0L58/qlB83DGW32S2FNDpCKXQ0WZd26QfioXkM41dbg0ArSsZ9CsR8NWOXb3ONZPJs1MRAKCtRXpOOI7zG1MeVQKgWQil0NFSLN0D0CEMw9CYa7bUzow9Xa5qtVhuclUAAOwdd1+pQmZGVpWZwEAnI5RCR3OHUu5GwQDQTtzBetTncxyzhA8A0M4C4QH5QwPbA7al/OZZ7woCsO8IpdCxKpali67dqMbjEY+qAYBb554pZcm5XI9m5wCAdscufEB38XtdALBfLuVKquzor5IM+pUM8k8eQPsai4V1IBzUWDyk8VhYti39yexS7fxMhlAKANDeIsnjSi9+qXZc2Dwj27ZkGMynADoRV+joWKksS/cAdJaQz9S/e8lk7bhYtfTp2aXafKmlQlmZckXxAB/vAID2FIqNy/RFZFW3fmixqgUVM7MKJw57WxiAfUHcjI4165oxQJNzAJ0m5DN1MBpyjM3SVwoA0MYMw1QkecwxxhI+oHMRSqFjMVMKQDeYdPXKYwkfAKDdRXpcfaU2T8u27V1uDaCdEUqhI6XLFa0VK7VjnyGNxkJXuQcAtKfDCWfgPp1mphQAoL2Fe45KxvYOs5XiqirFFQ8rArBfCKXQkYpVS8eTUUV8W//ED0ZDCpj8cwfQedwzpS7mCipVLY+qAQDg1pm+kMLxScdYfmPKo2oA7Cc6oaIjDYaDevfxQ7JtWyvFsvIVLtAAdJZCpaq5bFGz2YISAZ/S5aokqWpLc9mCbuuJelwhAAA3L5I8oUL6XO04v3FaPcPf4WFFAPYDoRQ6mmEYGgwHvS4DAPbU38yv6c9Sy7Vd94bCgVooJUkzGUIpAEB7iySPa23us7XjYnZO1XJWvkDMw6oA7DXWMwEA0GYGQgHtbPdasZzNX2l2DgBod/5gUoHI8I4RW/nNFzyrB8D+IJQCAKDNjMedzc03yxXH8UymIItdigAAbS6SrN+FD0BnIZQCAKDNJAJ+9Qa3V+BXbSnkM2rHxaql+XzJi9IAANgzkeQJx3Fh8wXZVmWXWwNoR4RS6DgbpQozBAB0vLGYc7ZUbzDgOJ5Js4QPANDegpGD8vnjtWPbKquQmfauIAB7jlAKHefxqTn96lfP6rHn5/Tnc8vKVarXvhMAtBn3Ej5ThuN4mr5SAIA2ZxhG/RK+DZbwAZ2EUAodJVeparlQVsmydS6d1xcvrclnGNe+IwC0mQnXTKlsxdVXKp2XzaxRAECbaxRK8fkGdA5CKXSUuWzBcTwSCSrk4585gM4zGgvJ3JG5b5ar8sl5vF6i7wYAoL2FEkdkGDv6KJY3Vc7Pe1gRgL3E1To6ymzGGUq5l7cAQKcImKZGIiHH2GAk6Diepq8UAKDNmWZA4Z6jjjGW8AGdg1AKHSVVF0pFPKoEAPbfuGsJX9g1M3SGvlIAgA5AXymgcxFKoWNYtq2Ua/me+4INADqJezZo2XL22Jh2BfUAALSjSM8xx3Epf0mV0qZH1QDYS4RS6BjLhbIKVat2HPaZGgwHrnIPAGhv7uB9pVh2HC/mS+xACgBoe75AXMHoIcdYfvOMR9UA2EuEUugYKdcylfFYWCY77wHoYAPhgGPJXrFqaSjkDONnWcIHAOgAkeQJx3F+Y8qjSgDsJUIpdIxZ99I9mpwD6HCmYThmS5mG1OeaITqdZgkfAKD9uftKFdLnZVVLHlUDYK/4r30ToD24m5xPEEoB6AKvGOrRsWRU47GwRmMhPbeW1emNXO08zc4BAJ0gEB6SL9iraml9a8CuqpA+p2jvSU/rAnBrmCmFjlCsWlrIO38pock5gG7wkv6EvnOkT5OJiAKmqcMJ53vfXLaosmXtcm8AANqDYRiK1i3hYxc+oN0RSqEjzGUL2rnn1FA4oIjf51k9AOCVZDCg3uD2ROiqbetCtuhhRQAA7I1I0rkLX37ztGybH16AdkYohY7gXrpHPykA3exwIuI4ZgkfAKAThOKTMnyh2rFVyamUu+BhRQBuFaEUOkJdk/NYZJdbAkDnm4w73wOn04RSAID2Zxg+RRK3O8ZYwge0N0IpdIRXH0jqu0b6dDgeVsA0aHIOoKu5+0rNZAqybHuXWwMA0D7cu/ARSgHtjd330BGOJ2M6noxJkqqWLcPwuCAAaCLLtpXKFJTKbv3fRrGiiM9UvrrVZ6NQtbSYL2kkGrrGIwEA0NoiPbdLMqTLHWXLhSWVi6sKhPo9rQvAzWGmFDqOzzRkkkoB6CKWLT0+dUGfSS3rn1Yzms0WNOoKoKbpKwUA6ACmP6JQfMIxxmwpoH0RSgEA0Ob8pqGDrhAqEXTuQDqTdvbeAwCgXUWSJxzHhFJA+yKUAgCgA7h3HXW3kGIHPgBAp3D3lSpmZmRV+JwD2hGhFAAAHWA85gylNkoV+XcsZV4vVbReLDe7LAAA9lwg1C9/eHDHiK385lnP6gFw8wil0NZWCiV9fSWt1WJZNjtLAehi7plSF3NFjUaDjrGZDEv4AACdIdrj2oVvkyV8QDti9z20tefWs/psalmSFPP79N0H+/RdI30eVwUAzdcX9Cvm9ylbqUqSSpatoXBQs9li7TbTmbxeOpDwqkQAAPZMJHlCm4tP147zm2dk21UZhu8q9wLQapgphbaW2vGrf7ZSlY9d9wB0KcMw6mZLBXzOj/mZNP02AACdIRg7JNMfrR3b1aKKmVkPKwJwMwil0NZSrqUo7p4qANBN3O+BhYrlOF7Il5S/PJMKAIB2ZhimIj3HHGPswge0H0IptK2NUlkb5Urt2G/Ub4kOAN3EHUpdzBc1HNnuK2VLmqWvFACgQ0SSJxzHuY0p+swCbYZQCm3LPUtqNBqS32T5HoDuNRYLaee74FK+pLGoM6iaybCEDwDQGcKJ26QdPaSqpXWVC0seVgTgRhFKoW2lss5QaiLO0j0A3S3s92ko7JwZFQ86P+qnmSkFAOgQpi+ocOKIY4wlfEB7IZRC23IvQXE3+AWAbuR+L7RcqxjmMgVVLGevKQAA2lUkedxxnN8klALaCaEU2lLFsnVhxzbnEk3OAUCSxlzvhUv5kpIBf+24Ytu6mCu67wYAQFuK9DhDqVJ2TtVy1qNqANwoQim0pfl8UZUdTQx7Aj4lg/6r3AMAusPOpcxD4YAGw0FNJpxB1XSaJXwAgM7gD/YoEDnoGMtvnvGoGgA3iqt4tCV3k/PxeFiGQZNzADgQCer+46Mai4UV8W81f/37xXV9YzVTu81Ws/M+jyoEAGBvRZPHtZG/VDvOb0wpPvAy7woCcN2YKYW2VBdKxSIeVQIArcVnGDqWjNUCKUmajDvfI2cyeVlsmQ0A6BDuvlKF9DnZVsWjagDcCEIptKVZdt4DgOs2HAkq7Nv+yM9VLC0Xyh5WBADA3glERuQLJGrHtlVWIX3ew4oAXC9CKbSdTLmi1eL2xZRpSKPRkIcVAUBrMw2jLrzfWsIHAED7MwxDkeQJx1h+g134gHZAKIW2M+fade9gJKSgj3/KAHA1h11L+KbThFIAgM4R6TnmOM5vnpbNUnWg5dHoHG3nRDKqn37JpFKZgmYzBfWFAl6XBAAtrVCp6qBrRuk0M6UAAB0knDgiwwzItrZWVFTLaZXzlxSMjnpcGYCrIZRC2zEMQ4PhoAbDQX37YI/X5QBASzqzkdU3VjNKZQpaKpT05vFB+QypevlH47ViRRulipJBvgoAANqfYfoVThxVfuP52lhu4zShFNDiWPMEAEAHupQr6R+WN7VYKMmWdDFX1KEofaUAAJ3LvQsffaWA1kcoBQBABxp3NTZPZQqaTDj7Ss3QVwoA0EHcfaXK+XlVShseVQPgehBKAQDQgQ5FQ44P+ZViWQcjQcdtZjKF5hYFAMA+8gViCsbGHGPMlgJaG6EUAAAdKOgzNexqbm4YhuP4Uq6oQrXazLIAANhX0eQJxzGhFNDaCKXQVj5+fl6fm1vWt9YzypQrXpcDAC1tPOYMpZYKJQ2Ft2dL2dpa1gcAQKdw95UqZKZlVYseVQPgWgil0Dbylaq+upzWFy+t6cNnLuk3vnZexarldVkA0LLG484eUqlMQYcTzl5T04RSAIAO4g8Nyh/s2x6wqyqkz3lXEICrIpRC25jLOi+cDkSCCvn4JwwAuxmPOQOouWxBE64xmp0DADqJYRjswge0Ea7o0TZmXb/muy+2AABOg+GAwjvC+3zVUjIYcNwmlS2oatnNLg0AgH0TadBXyrZZYQG0IkIptI2Ua6bURJxQCgCuxjQMjbn6Sm2WykoEfLXjsmXrYo5eGwCAzhGKj8vwbV8rWNW8itk5DysCsBtCKbQF27brmvGOE0oBwDWNx1x9pXJFTbp6Tc1kWMIHAOgchuFTpOd2xxhL+IDWRCiFtrBSLCu/o6l5yGc6dpACADQ2HnfOlNpqdu4MpabpKwUA6DCNlvABaD2EUmgLjfpJmYbhUTUA0D7GXP335vNFHYo6Q/2ZTEG2TV8pAEDniPQc1c7L3UpxWeXCincFAWiIUAptoS6UYukeAFyXeMCv/tB2c3PLlqq2FDK3vwJkK1WtFMtelAcAwL4wfWGF4pOOsfwms6WAVkMohbZQ1+ScnfcA4LqNxULqDfp1Z19cbxkf1EA4ULdZBEv4AACdJpo87jhmCR/QevxeFwBcS6lqad61MxQzpQDg+t17ZFh+0/k71GQirDObudrxTKagVwwlm10aAAD7JpI8rrULn6sdFzOzqlby8vkjV7kXgGZiphRa3ly2oJ2dTgZCAUX9vl1vDwBwcgdSkup24GOmFACg0/hDfQqED+wYsVXYfMGzegDUI5RCy6tbuscsKQC4ZVsbRmwfrxTLSpcr3hUEAMA+iNQt4ZvyqBIAjRBKoeWlaHIOAHsu6DM1Gg05xmaYLQUA6DB1odTmC7KtqkfVAHAjlELLMw1DgR0/54/T5BwA9sRh1xK+GdePAAAAtLtg9JBMf6x2bFslFTMzHlYEYCcanaPl/fjtB1W1bC3ki5rNFjQSCV37TgCAhsqWpUu5ojZKFU0mIvrbhfXaOfpKAQA6jWEYivQcU3b1a7Wx3OZphXtu864oADWEUmgLPtPQaCysUWZJAcBNyVeq+v2pC5rPF1W1paBp6KdfMum4zaVcUcWqpZCPidQAgM4RSZ5whFL5jdOyD71JhmHsficATdEy3zpTqZTe+9736tSpUzp16pQefvhhra6uXvN+q6ur+qVf+iXdfffduuuuu/TOd75TX/va1/a/YAAA2kjYZ2qtVFH18namJctWrmJpMByo3cZS/eYSAAC0u3DiiAxjez5GtbSucmHRw4oAXNESodTa2pruu+8+fe1rX9MDDzyg+++/X5///Od1//33q1Qq7Xq/TCajd7zjHfrsZz+rH/uxH9O//bf/VgsLC7rvvvt0+vTpJv4XAADQ2gzD0HjMufw5lS1o0t1XiiV8AIAOY/qCCiWOOMbyG1wvAq2gJZbvPfHEE5qfn9enP/1pHT16VJL00pe+VPfff78+9alP6e1vf3vD+73//e/X+fPn9eEPf1ivfOUrJUlvectb9PrXv16PPfaYfuu3fqtp/w0AALS68XhYUxu52nEqU9DheFj/sLxZG5vJEEoBADpPNHlchc0zteP8xmklR77Lw4oASC0yU+qpp57SqVOnaoGUJN199906cuSInnrqqYb3sW1bn/zkJ/U93/M9tUBKkoaGhvTwww/rFa94xb7Xjf1VtWzZtu11GQDQMdy7l85lC5pMOGdKzWYKqvLeCwDoMOHkccdxKXdB1XLGo2oAXOF5KLWxsaFUKqU77rij7twdd9yhZ599tuH95ubmtLCwoLvvvlvSVkiVzWYlSe94xzt2nV2F9vH3i+v6f792Xh86c1FfuLiq+VzR65IAoK2NuUKphXxJcb9PMb+vNlaybN5vAQAdxx9IKBgddYyxhA/wnueh1MLCgiRpeHi47tzQ0JDS6bTS6XTduZmZGUnSwMCAfvM3f1OveMUrdNddd+kNb3iDPv/5z+9v0WiK2WxB2UpVz69n9ecXVnR2M3ftOwEAdhXx+zS0o7G5LelCrqjDCWdYNU1fKQBAB4q4ZkvlNwmlAK95Hkpdmd0UiUTqzoVCWw1Zc7n6MGJzc6v/xe/8zu/oi1/8on7xF39Rv/mbv6lwOKx/82/+jZ5++ul9rBrNkMo4d4Aaj4d3uSUA4Ho1XMLnbnaeYQc+AEDnifQ4Q6nC5jlZVtmjagBILdDo/Hp6BhmGUTd2ZVe+zc1Nfe5zn1MymZQkve51r9Mb3vAG/af/9J9qS/uu18BA/IZuj/2zXihrvVSpHftNQ982MaiAz/MctSmGhhJelwB0vU59Hb4oV9BXV7ZnIC+UK3rL0RF9JrVcG0vlChocjDf8/AWapVNfg0A76bTXoW3HtTLTq3Jh/fJxRSFjXr1DL/a2MGAXnfYabMTzUCoajUqSisX6/hVXxuLx+rDoyv3e+MY31gIpSerp6dHrXvc6ffKTn1Q2m1UsFrvuWlZWMrIsmru2gufWnE0HD0ZCWl/NelRNcw0NJbS0VL9kFUDzdPLrsNf1MXd2NaPwoYoCpqHy5c/AjWJFU3OrGggHPagQ6OzXINAuOvV1GIofU7nwTO14PvV1lY1xDysCGuuU16BpGledAOT5tJPR0a1mc0tLS3XnFhcX1dPTUwugdrrSg6q/v7/uXH9/v2zbbrjsD+1hlqV7ALAvRiIh+XfMgEqXq8pUqnXL+ljCBwDoRHV9pTZOs+M34CHPQ6menh6NjY013GXvueee05133tnwfseOHVMwGNQLL7xQd25ubk6hUKhhYIX2MJt1hVIxQikA2As+09ChWMgxlsoWdDjh7CtFs3MAQCcKxydlmNszga1KRqXcRQ8rArqb56GUtLUE70tf+pLOnj1bG3v66ad1/vx5veUtb2l4n2g0qte97nX6whe+oDNnztTGU6mUPv/5z+v7vu/75PP5Gt4Xra1q27rgCqUmmCkFAHvGHfSnMgUdrmt2TigFAOg8hulXuOeoYyy/wS58gFdaIpR68MEHlUwm9e53v1sf+MAH9Hu/93v6qZ/6Kd1xxx265557JG2FTU8++aRSqVTtfj/7sz+rRCKhd73rXXrf+96nxx57TO94xzsUDof10z/901795+AWLeSKtb4mkpQI+NQb9Lz9GQB0DPeS6FS2oPF42PGlYKlQVqZcEQAAnSaaPOE4JpQCvNMSoVR/f78+8pGP6OTJk3rkkUf0wQ9+UK9//ev12GOPKRjcmlr5zDPP6OGHH9Yzz2w3pRsbG9Mf//Ef65WvfKUef/xxve9979OLXvQiffSjH9X4OM3q2lWjpXvsAAUAe2ciHtaJZFSvP9Sv+4+P6l3HRhXymToYdS7rc/f3AwCgE4R7bpe0fX1RLiyoUlr3rB6gmxk2Xd1q2H2vNXzs3Lz+ccd25W8aG9BrD3ZPf7BO2WUBaGfd+jr809klPb2wXjv+rpFevXl8yLuC0LW69TUItJJOfx0unH5Cxexs7bhv7PuVGDrlYUWAU6e8Blt+9z3ArW7nPZqcA0BTTLqW9U2nmSkFAOhMjXbhA9B8hFJoKblKVSvFcu3YlDRGKAUATTHpanZ+IVdQqWp5VA0AAPvHHUoVMtOyqkWPqgG6F6EUWkrKNUtqJBpS0Mc/UwBohp6gX/2hQO3YsqW5LLOlAACdJxAelD80sD1gWypsnt39DgD2BVf7aCmJoF+vPpDUoWhIpli6BwDNdti9hI9m5wCADuWeLZXbmPKoEqB7+b0uANhpNBrSWycPSJJKVUsli2UjALBfqratxXxJqUxBqWxBrxjs0WQioq/u2GxiJp33sEIAAPZPJHlc6cUv1Y4Lm2dk25YMg7kbQLMQSqFlBX0mS/cAYB99emZJ/2dpo3bcG/TrJf0Jx21mMwVZti3TMNx3BwCgrYVi4zJ9EVnVrR9grGpBxWxK4fikx5UB3YMrfgAAutRYLOQ4nssWNBQOKOrf/npQtCzN50vNLg0AgH1nGKbCPbc7xvIs4QOailAKAIAu5d7dNHW5qbl7Fz6W8AEAOlU0ecJxnN847VElQHcilAIAoEsdiAQVNLeX5eUqllaLZR12hVLTGUIpAEBnCvcclXb0kKoUV1UuLHtYEdBdCKXQMl7YyCldrnhdBgB0DdMwGs6Wmkw4x2bSedm23czSAABoCtMXUjh+2DHGbCmgeWh0jpZQqFb1gdMXZEvqC/k1EYvobbcN01gXAPbZeDysczuW56UyBd3Zl5DfMFS5HERtlqtaL1XUFwp4VSYAAPsmkjyuQvpc7Ti/cVo9w3d7WBHQPZgphZYwly3qym/wa8WKLuWKBFIA0ATjDWZK+U1D43Hn+DR9pQAAHSqSPO44LmZTqlZyHlUDdBdCKbSEVKbgOHZfDAEA9of7/fZSrqiyZWnSNT5DXykAQIfyB3sVCA/vGLGV3zjjWT1ANyGUQktwh1IThFIA0BSJgF+9we3V/FV7K5g6nHA3Oy+47woAQMdwz5bKb9JXCmgGQil4zrZtzWZdM6VihFIA0Cx1S/gyBU3Ew9q5iHoxX1KuUm1uYQAANIk7lCpsnpVtsQkTsN8IpeC51WLZcaETNA0diAQ9rAgAuot7CV8qW1DY59NINOQYn2UJHwCgQwWjozL98dqxbZVUyEx7VxDQJQil4LlZ15KQsViYJucA0ESNmp1LqusrNZ1mCR8AoDMZhlG/hI++UsC+I5SC51JZ+kkBgJdGYyGZO34LWCtWlClXdDju7CtFs3MAQCeL1oVSU7Jte5dbA9gLhFLwHE3OAcBbAdPUwcj2Ur3eoF8bpYomE87347ns1s58AAB0olDiiAxjx+Yf5U2V8wseVgR0Pv+1bwLsn1LV0qV80TE2RpNzAGi67x3tl7T1HtyzYze+vqBfa6WtRq9V29Zctqgjrp35AADoBKYZULjnNuU3tnfey2+eVjA64mFVQGdjphQ8dTFXlLVjRmx/KKB4gKwUAJrtxX1xvbgv7gikJGnSFUDNpFnCBwDoXJHkCcfxzoAKwN4jlIKn6pbuMUsKAFrKJH2lAABdJNJzzHFcyl1UpZz2qBqg8xFKwVOzribn7m3JAQDeOuzqKzWTKcii6SsAoEP5AnEFo4ccYwVmSwH7hlAKnnLPlCKUAoDWMhQOKuLb/rpQqFpazJc8rAgAgP0Vce3ClyOUAvYNoRQ8Y9u27js+qnsmh3TXQEIjkaBGduz+BADwlmXbMg2jbgnfNEv4AAAdzB1KFdLnZFX5QQbYD3SUhmcMw9DBaEgHoyG96oDX1QAAilVL31rPaDZT0Nzl5dU/+eIJTSbCen4jW7vdTLqgV/O+DQDoUIHwAfmCvaqW1rcG7KoK6fOK9p646v0A3DhCKQAAIEmq2rb++NxC7diQVKpaOkyzcwBAFzEMQ5HkcWWW/k9tLL8xRSgF7ANCqQ6TXnpGm4tfuqn79o+9WZHksbrxQmZGKzNP3tRjJgZfrp7h76gbtyp5XZp6/009Zig2rsHDP9Lw3MLpD9zU7himL6yDJx9qeG419RnlN1+44ceUpOFj98kfTNaNX+15mjcNVa3dmwjzPLXG83QtPE/t/Tyl187pwrP/86Ye82rP0/zUYzf1mKH4uAYmf7jhuYUzT6haurnnaeTkg46xqN+ngVBAK8WyJMmW9PXn/qdGzVX9mK+8fcOqNPdsQKZh1D3ugdvfJX+wp248vfwVpRf//obrlKS+se9XpOf2uvFCZlars5++jkcwLv9/2/XGBr5dPQdeXXdLq1rQwpkP1e5nSFKD/05deawd50LRQ+obe1PDCpbPf0LVSmb7flce31Dd2PZfDZlmUINH7m34mJsLf6didu6qNRmNHtsw1HfoTfIFYnWPmd84rdz6c1etaedY7f9vSPGBuxSMHqx7zHJhWenlr+xS0866tv9eWo+oUDQVjB5UOHGk7jEBYL9Fe1yh1OYZ2bYto+FnAoCbRSjVYaxqcXua6Q1a/OOPaPLBf183bluVm37M5U9/XD0P1F+c2bJv+jHTL1xUXN+u8OHDdedK6UXZZvGGH7NSqGr9i19Q72u/p+5ceWNR1fL6jRcqafkTf6yRH3uwbvxqz1P1Go/J89Qaz9O18Dy1+fNULe/L81Qprd3UYxa/dkEx+2WNn6fNhZt6nuxdnqfxeLgWSknSxVJQw+aakq7v4FZJsho87vIn/qjx81QpqFJcveE6JWnxjz68y+uprEpx5aYec/lP/kg9D9SHUrZtqZyfv6nHzD9/WpHKiYbPU371hZt+nvyzgw1fT/mlMyqWZ2+iUqnypZWGz1Mpv6Ds6jdu6jHTf/Z/Gj5PldKG48Luuh7ryn2/uanb3vnbdedt21bqK78uK1uUClXZJUsqWrKLllSwZBer28dF13HJ0sQv/t8Nn6f1L35Bix9+4oZqveLAO9/d8HkqTE9r9v/5v2/qMZPf/VoNv+v+hudOP/Dum3rM0MSkJn+l/nmSpJlf/f+pODtzU497/LEnGo4vfOgD2vjrL97UY078Es9TI816nq63tXc3PU9n5Lx/KzxP16ubnie3TnieOhmhFAAAqBmPhfW1le2ZV4v2gIfVtJndJ7l2uVv4f5hi459qbKso+S2ZyYCUDNzQQ1qXCrueKwbn5Ht571bQVbTqg62S1TiNBQAAN4VQCp6Y2shrynqxhrWiA8aKgkbF65IAANqaKbUToZTHujzosouNEyCrsnuwdM3HLO2eKpWCFxR4df+17++afWVnd/8eU7VyUsS3FbARaAEA4EAo1WESg6/Q4m8+cVP3DQ4eajgeik/I/suqShcv3PBj7vbF75/WS/q69dLLN7L1HV/+rI6f+6fre8yqLb208blY9qVa+eTHb7xOSfrhxucS0Vdp83f/9oYfU5KC3z7W+DHb5HkyfWEFnz+o9P/58o0/Js9Tw3Gep/Z4nuJ9R2T/1U0+T7tcRJu+sIJTN/k8Va7xPD1548+TbEn31A+PRELyyVb1cm+frKJa/cSaYvnr61sVfGnj5yk++HIt/scP3nidkoL9ow3HQ7Fx2Z+vqnTp4u53Nlx/XmYXdn+eAmeGlfnHf7h8v539lHZ57Ct/L1q7Pk+R/Emt/8Vnd9zHcPyx698tSd/T+DGj4TuUfvL/7Li9sfvjuY6Do43/7UeSx7X8kT/acR/Xg1zl8f3VxmFOIDwo+5uWKmur9Y+52+P5DBkhn+yFxksereotNNvfZfaVJNnX8SOZETSloCkjsf012s5Udg2c0rm/V/g9k1u3K++cfdV4Npbj70aXp5IAgI5n2LbNp91lKysZWVdpMI298x+/cV5rxe0vfv/q5JgmE5Gr3KN7DA0ltLR0402LAeydbn8d/t63UprNbM9E+fGjI7qzP6FHvjmj+XypNn7fsVGd6K1vlg3cqmu9Bm3bklXJy6rmZVULl/9e2PF/eVmVHX+vFmrHsf5vU//4mxs+Zupr/89N1RsIH9DBF/3rhucWTn9AxWzqph730Et+Vj5//fejzPI/KLcxJdMXkekLy/SHt/68cuwLy/RHamOGGaA5M25Yt38WXrFw5oMqZrZ7DvUeepN6DrzKw4rQLTrlNWiahgYG4rueZ6YUmi5drjgCKZ8hjcZCHlYEANhpPBZ2hFKpbFF39ic0GY84QqmZTJ5QCp4wDFO+QKzhDoLXsuvvsbal5MHXXTXYsquNlw2avnDDcWlrR8ebZfoafz8q5RdUuKGdTM1acDV4+EcVjNbPPqwU11TKz2+HWr6ITH9Yhhki0EJXiySPO0Kp/MYUoRSwhwil0HRzGeeXs4PRkAKm6VE1AAC38ZjzAjuV2VoqNZkI68tLG7Xx6czNX2wDXtktYDFMv5Ij33nV+9q2Lbta3J59dTm4MnYJjyTVAp6tcOr6Z+QbvpAMo/H3I6tyo8sXLVmVnKxKTlLjx8ynz2ot9ZlGlcj0hVyzspwzssI9xxSMHLjBmoD2EOk5rvULf1E7LmZmZVUKMv27h9EArh+hFJpuNuu8iHFf/AAAvOVudn4hV1TVtnU47lxGNJcpqGJZ8vPDArqEYRgy/OEbuhgdPr61tblt27KtUoPlhldmZOVl7Qi8DGP3r+m3NPuqwXJA6WrN4+1arbsZCCQIpdCxAuEB+UODqhSXL49YyqdfUKzvTk/rAjoFoRSaLuX6Zd198QMA8FZv0K+436dMZashdNmytZAvaTQaUjLo10Zpawl2xbZ1MVfURJyegMC1GIYhwxfaWpIXTN7SY/WOfp8q5VfsCLMKjrDLdh/b220TdltqeGvLDHf/Lpdd/SdFkidk+oI3/fiA1yLJ40ovLteO8xunCaWAPUIohaaq2rbmXDOlJmJczABAKzEMQ+PxsL61nq2NpTJ5jUZDmoyH9Y3VTG18Op0nlAKaLBgdUVAj131726psz8AyG4dDgfCgIsnjjqbwVjUv2ypf8/F3C6Vy69/Syswn5Q8NavDIvcymQtuKJo8rvfh07Ti/+YJsuyrD8HlYFdAZCKXQVIv5kko7djiM+X3qC/HPEABazXjMFUplC3qVpMOJiDOUyhT03R7UB+D6GaZfPjMhXyCx623iAy9TfOBldeO2VXXuYlgLrbb7avmCvXX3KxdXtTLzJ5KkSnFZC1OPqW/8BxQfeOle/WcBTROMjcn0Ry/3ZZPsakHFzKzCiSMeVwa0P9IANNWsa+neRDzMji4A0IKuLK2O+k2Nx8K1Wa2TrllRM+m8LNuWyXs50JEM0yefeeM7HW5c+oJsq1g7tu2KVmefVDEzo77xN8s0A3tdKrBvDMNUpOeYsqtfr43lN04TSgF7gFAKTZXKOneLock5ALSmiXhYP/OSSfWHAo4fD4YjQYV9pgpVS5KUr1paKpQ0HNl99zEA3ad//AdlGD7HRbwkZVe/plLuogaP3KtAeNCj6oAbF0kerwuleg+9kR/YgVvEdjloKpqcA0B7CJimBsLBui/bpmFowvXePZO++QbJADqT6QtqYPIe9U+8tW4nwXJhUfNT71d29Z88qg64ceHEUWlHD6lKaU2VwvJV7gHgehBKoWnylaqWCtvNMg1JY8yUAoC2c9i9hC+T3+WWALpdfOBlGj7xE/KHBhzjtlXWyswntTr7lGyrssu9gdZh+oIKxw87xvKbp70pBugghFJompRr173hSFAhH/8EAaDdTCacodQ0oRSAqwhGhjVy4gFF++6sO5dZ+QfNn/59lYurHlQG3JhI8oTjOLcx5VElQOcgEUDTrBXLMnesAmHpHgC0p7FYSL4d7+drxYo2Ssx0ALA70xfSwOSPqG/8BxxLoCSpnJ/X/POPKrf2nEfVAdcnkjzmOC5l51QtZ3e5NYDrQaNzNM2rDvTqrsEeXcgWlcoUNBqjKS4AtAvLtrWQL6k36FfE79OhaFizO2bAzmTy+rb+3bebBwDDMJQYfLlC0VEtT39ClR2zo2yrpOWZ/6XR2CH5g0kPqwR25w8mFYiMqJyfr43lN88oPvAy74oC2hyhFJoqYJo6nIjosGvpBwCgNT2ztKGvraR1IVtQybL1z28b0UsHEppMRJyhVJpQCsD1CUYPauTEA1qZ/bTy69+qjfeNvoFACi0vkjzuDKU2ThNKAbeA5XsAAGBXK4WyzqfzKlm2pO3+gIfdO/Bl2IEPwPUzfWENHr5XfWPfLxmmIr0vUnzolNdlAdcUTR53HBfSZ2nWD9wCZkoBAIBdufv/pS6HTxOuHfgu5YoqVKsK+5y9YgBgN4ZhKDF0SqHYmPyhfhmGce07AR4LRA7KF0ioWk5L2tpJspA+X9dvCsD1YaYUAADY1XjMGUpdzBVVsSzFAj4NhYO1cVvbgRUA3IhgdFSmr/EGOLZV0ersU6qUNppcFdCYYRiK9DhnS+U3T3tUDdD+CKUAAMCueoJ+JYPbE6urtq1LuZIk6XDCeRE5TSgFYI+tXfgLZVb+QfPPP6r8xhmvywEkbfWV2im/cVq2bXtUDdDeCKWw74pVS7/9jWl99Owl/d38Gr+kA0Cbcc+W2u4r5VzCN5PON60mAJ0vt/acMsvPSJKsal5L5/6n1i/+lWzb8rgydLtw4ogMM1A7rpbTjubnAK4foRT23Vy2oNViWd9Yzeip1LI+fp43bABoJ3Wh1OUfFyZdO6mmsgVVLX4pBrA3cjt25rtic+HvtPjCh1S53M8H8IJh+hVO3OYYy21MeVQN0N4IpbDv3DOj3E1zAQCtbczd7PzyTKm+oF89ge3G5mXL1sVcsam1AehcA4d/VMmDr5PkbIBezMxq/vn/ocLmOW8KAyRFkiccx/kN+koBN4NQCvvuysXLFeOxyC63BAC0okPRkOMLw2qxrGy5KsMwNOlewpdhCR+AvWEYhpIj36kDx94lnz/uOGdVclo8+xGtX/oCy/ngiUiPc7e9cn5eldKmR9UA7WvXUOrP/uzP9Ou//uv6+Mc/rkql4jj30EMP7Xth6Ay2bWvWNVNqgplSANBWgj5TI9GQY2wu23gJ3zR9pQDssXB8UiMn/5XCiSN15zbn/1qLL/yBquWMB5Whm/kCMQVjY44xZksBN65hKPWRj3xEv/Zrv6ZCoaDHH39cP/ZjP6b19fXa+a985SvNqg9tbq1UUbZSrR0HTUMHIsGr3AMA0IrGXH2lZmvNzp3jM5kCOxAB2HO+QExDR9+h5Mhr684VM+d16flHVUhPN78wdLVIj3sXPvpKATdq11Dq8ccf16/92q/p05/+tF70ohfpvvvuqwVTfNnE9XL3kzoUC8tnGLvcGgDQqtz9AOcuv78PR0MKmdtfJ7KVqpYL5abWBqA7GIap5MHX6sDt/1KmP+Y4Z1UyWnzhw9qY/xuuVdA0UVdfqUJmWla15FE1QHtqGEotLS3p5MmTkiS/369f/dVf1atf/Wq9613v0tramgxCBVynuqV7MZbuAUA7qtuBL1uQZdvyGUbdsmz6SgHYT+HEbTp48iGF4pOuM7YyK/8o22LDBTSHPzwof7Bve8CuqpA+611BQBtqGEr19fUplUo5xn7hF35Br3rVq/Sud71L1Wq10d2AOqms88KEnfcAoD0NhgMK+7a/NhSqllYuz4iaTDjf26cJpQDsM18goQO3v1M9w9+1PWj4NHjkXpk+vm+iOQzDUCTpXsJHXyngRjQMpV7zmtfok5/8ZN34L/7iL+pVr3qVikV+fcC1lS1Ll1xbgxNKAUB7Mg1DR3siOpKI6LtH+vSO2w+qJ+iXpPod+NKFRg8BAHvKMEz1jn6vho7+uEx/VH2H3qBQdNTrstBl6kKpzTPsCAncAH+jwV/+5V/edTbUL/3SL+k973nPvhaFznAxW1R1x5L+vpBfiUDDf3IAgDbwjtsbX+yNx8IyDcm6/J6/UiwrXa7wng+gKSI9t+vgyf9Lpj+6621s26YFCfZFKD4hwxeSXd36Md6q5FTKzikUn/C4MqA9NJwpFQwGFYlEGp2SJI2O8gsEri2Vdf5S7u5HAgDoDEGfqdFoyDE2k2YJH4Dm8QViu4ZOleKa5qceVTE71+Sq0A0Mw6dIzzHHGEv4gOvXMJQC9kJdk/P47kEnAKC9HXYv4cuwhA+A92yroqXzH1c5v6CFM09oc/HL7M6HPedewpfbJJQCrtd1hVJ/+Zd/qZ/6qZ/Se97zHv3X//pflclkGt7uK1/5iv7Df/gPe1og2hczpQCge0wmnKHUNDOlALSAtQt/rnL+0taBbWn9wue0fP5jsioE59g7kcTt2nlpXSksq1xc9a4goI1cs9nDZz7zGf3Mz/xM7ReFp59+Wp/97Gf10Y9+VIlEQn//93+vp556Sp///Oe1urr1wvv5n//5/a0aLa9i2TraE1EqU9BSoSy/Yeiga2kHAKBzTLo2sriUK6pYtRTyMSkbgDe2+kj56sbzG89rfmpBg0fuVTB60IPK0GlMf1ih+ISKmenaWH7jtAIHXu1dUUCbuGYo9YEPfECDg4P67d/+bU1MTOjzn/+8/uN//I969NFH9bWvfU1f+cpXZNu2Dhw4oLe97W36nu/5niaUjVbnNw3de2REkpSvVLVUKMlv0lwSADqBbdtaLZaVyhZUsWy9YiipeMCvwXBAy4WyJMnS1ozZ23t2bzwMAPvJMAz1jb1JofiEVmb/pNaIWpIqpTXNn/599R16k+KDL6cJOm5ZJHnCFUpNqYdQCrima4ZS58+f10MPPaRXvepVkqR3vOMdymazeuSRR2RZlt72trfp7W9/u+688859LxbtKeL30U8KADrEcqGk3/tWSrnK1nbXyaBfrxhKSpIm45FaKCVtNTsnlALgtWjvixSMjFzuLXVp+4Rd1drcZ1TMzKh/4gdl+pjVj5sXTR7X+oXP1Y6LmVlZlbxMP9dBwNVcc059JpPRyMiIY+z7vu/7VKlU9MADD+hXf/VXCaQAAOgSvUG/itXtJsEbpYo2SxVJ0mHXEr6ZDH2lALQGf6hPI8fvV3zwFXXncuvPan7qMZXyCx5Uhk7hD/UpEB7aMWIrv/mCZ/UA7eK6Gj24p7P29/dLkl7+8pfvfUUAAKBl+U1To64egXOXN7Y47Gp2PpspqMouVwBahGH61T/+Fg0c/mcyzKDjXKW4ooWpx5VZ/iq78+GmuXfhy2+wCx9wLdcVSn3hC1/Q//7f/1sLC85fD4LB4C73AAAAnWrMtZvqbGYrlOoPBRT3bzcVLlm25nNFAUArifXdoZETDyoQHnaM23ZFq6k/1crMk7KtikfVoZ3VhVKbL8i2qh5VA7SHa/aUkqSnnnpKn/nMZyRtzZI6evSoDMPQc889p9tuu03Dw8PXeAR0k41SRSHTUNhfv9sJAKD9jcdD+tLi9nHq8kwpwzA0mYjo2bVM7dx0Oq9DrhALALwWCA9o+MR7tDb3Z8qu/KPjnG2VpAa79gHXEowekumPyapkJUm2VVQxO6Nw4jaPKwNa1zVDqa985St67rnn9Nxzz+mb3/ymnnvuudqOe7/927+t3/7t31ZfX59OnjypkydP6sUvfrF+8Ad/sBm1o0V9bm5ZX19JaygS1EQsrNcM9+pglMaRANApJmLOZXoXsgVZti3TMHQ4HnaEUjOZgr6j2QUCwHUwzYAGJn5I4fikVlNPybbK8gV7NTDxVnbjw00xDFORnmPKrn6tNpbbOE0oBVzFNUOpeDyuU6dO6dSpU7WxXC6nb33rW3r22Wf13HPP6dlnn9WXv/xlPf300zIMg1Cqy6UyBdmSFvMlLeZL+vbBHq9LAgDsob6QX1G/T7nK1pKEkmVrIV/SwWhIk66+UjOZvGzb5gIPQMuK9X+bgpGDWp75lAYmfkCmn9mduHmR5HFHKJXfOC370Jv4HAR2cV3L99yi0ahe/vKXOxqdFwoFPf/883r22Wf3rDi0n2y5qpXi9nbgpqRDzJICgI5iGIbGYyFNbeRqY3PZgg5GQzoYDSloGipZW42C0+WqVotlDYTpQwmgdQUiQxo58cBVg4NqJSefP9rEqtCOwonbtpZ/2ls/3FRL6yoXlhSMHPC4MqA1XVej8+sRDof1spe9TO94xzv26iHRhlJZ5/bfI9GQgr49+2cGAGgR43HnjKjU5WbnPsPQeNw5y2D68jkAaGVXC6Ry69/SxWcfUXbtm02sCO3I9AUVThxxjOU3pjyqBmh9pAXYU7OuC4+JONOfAaATjcecs2CvNDuXpElXYDWTdv5gAQDtpFxc1crMn8i2SlqZ/l9aTX2G3flwVZHkCcdxfuO0R5UArY9QCntq50WJJI2z4xIAdKQx1/v7Yr6kQnVrqcJhVyg1nSGUAtCebKuq5fMfl20Va2OZ5a9o/vTvq1xc9bAytLJIzzHHcSl3QdVyZpdbA92NUAp7xrJtzWWKjjFmSgFAZ4r4fRra0SfKlnQhu/UZMB4PO75gLBfKypSZVQCgDRmm4oN3bfUI2qGcn9f88+9Xbv1bHhWGVuYP9igYOegYy2+e8agaoLURSmHPLOZLKlpW7TjqN9UfCnhYEQBgP43HXUv4Li/hDvlMHXRtcuFe3g0A7cAwDCUGX6GR4++RP9jnOGdbRS2f/5jW5j4n26p6VCFaVSR53HFMXymgMUIp7JlGS/fY+hQAOpd7ifZ8fnu27GTCtYSPvlIA2lgwelAjJx9UpPdFdefSS1/WwpkPqFJcb35haFnuvlKFzXOyrPIutwa6F6EU9oz7V3D3zkwAgM5yJBHVaw4k9fbbhvUzL5nUP79tpHZu0rV8e4aZUgDanOkLa/Dwveob+37JcF5GlXIXdWnqUeWYDYPLApFh+QI9tWPbrqiYPu9hRUBrIpTCnkmx8x4AdJUDkaB+aPKAXjbQo4Fw0DE71r0D34VcQaWq5X4IAGgrhmEoMXRKw8fuly+YdJyzqwUtn/sjrV34C9k2y/m6nWEYdUv4cuzCB9QhlMKeyFeqWiyUaseGpDHXduEAgO7RE/Q7+gpatjSXZbYUgM4Qih3SyImHFOk5XncuvfglLb7wEdm27UFlaCXuUKqwcZp/F4ALoRT2hPtC40AkqLDPt8utAQDd4LBrxuw0S/gAdBCfP6LB2/65ekffoK2fZLdFkyfprQqF44dlmNs71VYrGZVyFz2sCGg9hFLYE42anAMAupu72fkMzc4BdBjDMNQz/BoNH3t3rX9QpPdFig+d8rgytALD9Cvcc9Qxlt9kCR+wE6EU9sQdfXG9ZXxQd/bFlQz46ScFAKjrKzWbKchi2QKADhSKj2vk5EOKD9ylgYkfYpYUatxLPPP0lQIc/F4XgM4wHAlpOLLdQ4qLDgDoHulyRalMQbOZglLZgn5gYkij0ZCGwgFF/T7lKlsNf4uWpfl8SaNReg4C6Dw+f1T9Ez+463nbqqiUu6hQfKKJVcFrkeQxbS3v3Lo+KucXVCmtyx/s9bIsoGUwUwr7wuTXIQDoGn8ys6iPvHBJfz2/pvPpfG2ZnmEYdX2lWMIHoFutXfgLLZx5QhuXvijbZjfSbuHzRxWKjTnG8htnPKoGaD2EUgAA4Ja4+wju3PzC3VdqOkMoBaD75NaeU2b5GUnSxvwXtXT2D1QtZzyuCs0SSZ5wHOc3pjyqBGg9hFIAAOCWjLlCqdkdu+wdjtc3O2c7bADdxKoWtJL6U8dYIX1e888/qkJ62pui0FSRpLOvVCEzLata9KgaoLUQSgEAgFtyKBZ2bIa+UizX+kgdjIYUMLfPbparWi9VmlwhAHjH9IU1dPhemf6YY7xayWjxhQ9rY/5vCOs7nD80IH+of3vAtlTYPOtdQUALIZTCLSlVLf3t/JpmM3lVLNbGA0A3CvlMjUSCjrErS/j8plE3k2qavlIAuky45zaNnHxIofik64ytjUv/W0tn/1DVSs6T2rD/DMOomy2VYxc+QBKhFG7RhVxRn0kt6/e+Nad//9Vz+p9nL3ldEgDAA2PxG1jCR18pAF3IH0jowO3vVM/wd9adK6TPav75R1XMpDyoDM0Q6XEt4ds8Q8N7QIRSuEWzOy4sqrYtP7vuAUBXunqzc/dMqYIAoBsZhqne0ddp6OiPy/Q5A/tqeVMLZz6ozYWnWc7XgULxCZm+7c9Dq5pXMUsICRBK4ZakMs4Li3HXL+UAgO7gfv9PZQq1i6qJuLPn1GKhVOs5BQDdKNJzu0ZOPqRgbMx1xtL6xb/U8vk/klVhVmknMQxT4Z5jjrE8S/gAQincPNu2lco6Q6mJGKEUAHSjoXBQId/214p81dJKsSxJCvt8GomGHLdnCR+AbucPJjV87D4lDrym7lx+47SWzn/Mg6qwn9x9pQilAEIp3IL1UkXp8vYv3QHT0LDrogMA0B1Mw9BYzPkZsHM27aRrJtUMS/gAQIbhU9+hN2jwtn/uWNolw1Tv6Pd5Vxj2RaTnqGRsX4JXiisqF5Y9rAjwHqEUbpp76d6hWFg+ekoBQNdy95XaOZvW3ex8mplSAFATTZ7QyImHFIyOSpL6Dr1Rodghj6vCXjN9YYVdOzAyWwrdjlAKN42lewCAnRr1lbrC3ez8QragssWuQwBwhT/Uq+Fj96t/4q2KD77S63KwTyLJE45jQil0O0Ip3LRZmpwDAHZwz5S6lC/WgqdkMKC+oL92rmpLc9liU+sDgFZnmD7FB14mY5fVB5XimjLL/8DufG0s0uPsK1XMplSt5DyqBvAeoRRuSsWydDHnvJhwX4wAALpLPOBXX2greAqahibjEWV39B6cTDiX8M2kWcIHANfLtipaPv9xraae0srMJ2VVCfbbkT/Uq0D4wI4RW4XNFzyrB/Ca/9o3AepdypVU3fELTW/Qr54g/5wAoNv9yOFhxfw+HYgE6/oMTsYj+tpKunbMDnwAcP3WLvyFSvlLkqTc2jdVyl3S4JF7FYwMe1wZblQkeVzlwmLtOLcxpVj/t3lYEeAdZkrhpsy6LiSYJQUAkKTbe6I6GA013PjisKuv1EymIIslKABwTZXSprKrX3eOFVe0MPW4Miv/yHK+NuPuK1XYPCvbqnhUDeAtQinclNks/aQAADdmKBxUxLf91aNQtbSYL3lYEQC0B3+wRyMnHnAt+5Jsu6LV2U9rdfZJWVXeT9tFMDoq0x+vHdtWSYXMjIcVAd4hlMJNSbmanE8QSgEArsE0tvpM7TTNEj4AuC6B8KCGT/yEYgPfXncuu/oNLZx+XOX8kgeV4UYZhqFI8phjjF340K0IpXDDMuWK1kvb00t9hqHRaMjDigAA7aJuCV+6sMstAQBuphnQwMQPqX/iHhlmwHGuXFjS/OnHlF39hkfV4UZEks5d+PIbp1mGia5EZ2rcsHjAr1/+9ts0ly1oNlNQoWrJb5JvAgDq2batYtVS2O+TpLqZUjQ7B4AbFx94qYLRg1qe/rgqheXauG2VtTLzKRUyM+ob+36ZruAKrSOcuE2G4Zdtb/3YXy1vqJxfUDA64nFlQHMRSuGmRPw+HUvGdCwZ87oUAECL2SxV9MzShuayBaWyBY1EQnrg5Jgk6VAsJL9hqHL51+D1UkXrxbJ6Q1w4AcCNCEYOaOT4A1pNfUa5NefsqOzKP6qUvajBI/cqEB7wqEJcjWkGFE7cpvzm9rK9/OZpQil0Haa3AACAPVW2LP3VxVVNbeSUq1i6kC3Wdtnzm6bGYs4l3zMZlvABwM0wfUENTN6j/vEflGE45xuUCwvamP9rjyrD9Wi0hA/oNoRSAABgT/WHAor6t79iFC3nLnuTCZqdA8BeMQxD8cG7NHziJ+QP9dfGfcFe9Y+92cPKcC3uZuel3EVVymmPqgG8QSgFAAD2lGEYGos5G5rPZbdnQx1295VKE0oBwK0KRoY1cuJBRXvvkAyfho7cK9PPDtmtzBdIKBgddYwVNs54VA3gDUIpAACw58ZdoVRqRyg1EQ/L2HFuIV9SvlJtUmUA0LlMX0gDh39UIycerAs70JoiyROO49zGlEeVAN4glMIN+Yu5FX3i/IKeWdrQfG67RwgAADuNx12h1I6+URG/T8ORYO3YljRLXykA2BOGYSgYObDr+dz6t7Q29znZFj8GtAJ3X6li+rwsq+xRNUDzEUrhhvzTWlr/sLypT04v6pFnZzXNkgsAQAPumVIL+ZKKVat2PBmnrxQANFu5uKqVmT9ReunLWjjzhCqlda9L6nqB8AH5gsnasW1XVEif87AioLkIpXDdcpWqlgvbqb0h1fUMAQBA2poNNRgO1I5tSRd2LOGbTDg/P+grBQD7y7YqWj7/cdlWUZJUyl3Q/POPsuObxwzDUKSHXfjQvQilcN1SrqUVI9GQgj7+CQEAGrtaXyl3s/O5bFEVyxIAYH9Uypuyqs4fAKxqQUvnPqq1C38h22Y5n1fcS/jyG6dl0yYFXYJEAddtNusMpdwXGwAA7HS1vlK9oYCSQX/tuGLbupAtNq02AOg2gVC/Rk48pHDPsbpz6cUvafHMh1QpbXpQGcLxwzLM7V6LViWrUu6ChxUBzUMohevmnik1ESeUAgDsrtFMqZ2//E66Pkdm6CsFAPvK549o6LZ/od7R10uOfVClYjal+alHld98wZviuphh+hTpud0xxhI+dAtCKVwXy7Ydyy4kZkoBAK5uJBKS39i+6EmXq9ooVWrHhxPuZufswAcA+80wDPUM363hY/fJF0g4zlmVnJbO/qHWL/5v2TZLqpup0RI+oBsQSuG6LBWcuyZFfKajgS0AAG4+09BoLOQY2/kDh3sHvpl0XhY9NACgKULxia3lfImjdec2F/5GZ77yqKxqyYPKutPWssrtH3LKhUVVimveFQQ0CaEUrot76d54PCzDMHa5NQAAWyZcs2rndoRSw5Ggwjs2zMhXLS0VuAACgGbxBWIaOvrjSh78XrmX86XXzmr94l94U1gX8vkjCsXHHWPMlkI3aJlQKpVK6b3vfa9OnTqlU6dO6eGHH9bq6uoNPcbzzz+vO++8U7/7u7+7T1V2L5buAQBuxlg8LJ8hjcVCevWBpG7vidbOmYZR159wJs0SPgBoJsMwlBz5Lh24/Z0y/XHHuczyV1XKL3hUWfeJ9JxwHOcIpdAF/Ne+yf5bW1vTfffdp1KppAceeEDValWPP/64pqam9LGPfUzBYPCaj1GpVPQLv/ALKpfLTai4+8zS5BwAcBNe1BvTr9x1VAGz8e9gh+MRnd7I1Y5nMnmdOpBsVnkAgMvCicM6ePIhLZz+gCqlrWVj/lCfrCo7ozZLJHncMTutmJmRVS3I9HHthc7VEqHUE088ofn5eX3605/W0aNba5pf+tKX6v7779enPvUpvf3tb7/mY/yP//E/dObMmf0utSsVqlUt5p3LKcaYKQUAuA67hVFXTNY1O2cHPgDwii8QV+/YG7Uy/UmNHn2DzOhLZZgtccnYFQLhAflDA6oUVy6PWMpvnlWs7w5P6wL2U0ss33vqqad06tSpWiAlSXfffbeOHDmip5566pr3n5qa0vve9z795E/+5H6W2bXmskXtbDs7FA4q4vd5Vg8AoHOMxULy7ehRuFasOHboAwA0V6TnuEbv+LcaOfI9BFIeqN+Fb8qjSoDm8DyU2tjYUCqV0h131Ke/d9xxh5599tmr3v/Ksr3v+I7v0Fvf+tb9KrOruZucs3QPALBXAqapQ64d+maYLQUAnjEMQz5/5No3xL6IJJ19pfKbL8i2qx5VA+w/z0OphYWtxnnDw8N154aGhpROp5VOp3e9//vf/37NzMzo3//7f79vNXY7nyElg9u/khBKAQD20uG48+JnJk0oBQDoTqHYmEzf9ueiXS2omEl5WBGwvzyfj5nNZiVJkUh9Gh8Kbf1ymsvllEgk6s6fOXNG/+2//Tf9yq/8ikZGRjQ3N7e/xXap7z7Yr+8+2K+NUkWpTEHj8dC17wQAQAObpYrmsgUd7Ykq5Nv6bWwyEZbmt28zk2EHPgBoRVa1KNPHtcB+MgxTkeQxZVe/URvLb0wpnDjsXVHAPvI8lLJt+5q3MXb0mriiWq3q53/+5/Xyl7/8uhqhX4+Bgfi1b9TFhiTd7nURXWJoqD6EBdBcvA731sefn9MzF9e0WtjaJff/+8rb9eLL/28cSUb04TOXare9lCsq3htVJED/wm7GaxDw3pXXoWVVtDjzN7p0/vM6/vKHFEuOe1xZZ/NbL9W5HaFUKfOCBgf/WcPrYnS2bvgs9DyUikajkqRisX6r0Stj8Xh9WPT4449rampKf/iHf6jV1VVJ0ubmpiQpn89rdXVVvb29Mq+x689OKysZWda1QzJgPw0NJbS0tPuSVQD7j9fh3lvZLNQCKUn6p4urGtL2l+sD4aAWC1s7vdqS/nFmSceSsWaXiRbBaxDw3pXXYSF9TquzT6lSWpMknf/mJ3Xg2LsJSPaRZY9Khk+63EuqmF/RpbnzCoSHPK4MzdQpn4WmaVx1ApDnPaVGR0clSUtLS3XnFhcX1dPTUwuudvqbv/kblctlve1tb9NrXvMaveY1r9GP/MiPSNoKrF7zmtfo4sWL+1s8AAC4LuOufoTuTTQmE87z02mW8AFAK7BtuxZISVIxm1Ju/eqbUeHWmL6QwvHDjrH8xmlvigH2meczpXp6ejQ2NtZwl73nnntOd955Z8P7/dzP/VxtZtQVy8vL+tmf/Vndc889+uEf/mENDZEkAwDQCsZjrlAqW5Rt27Vf2g/HI3pmaftzfZod+ACgJUR6jirSc1z5ze1QZP3CXyqSPCHTDHhYWWeLJI+rkD5bO85vnFbP8Hd4WBGwPzwPpSTpjW98oz70oQ/p7NmzOnr0qCTp6aef1vnz5/UTP/ETDe/TKKy60uh8fHxcd9999/4V3CWqti3LthW4gSWQAAA0ciASVNA0VLq8TD5XqWq1WNZAOChJmkw4NzyZyxZUsWz5TZaHAIDXeg+9Qfn0C5JtSZKq5U2lF55W8uBrPa6sc0WSx7U299nacTGbUrWclS/A0nZ0lpZIGx588EElk0m9+93v1gc+8AH93u/9nn7qp35Kd9xxh+655x5JUiqV0pNPPqlUiu0wmyWVKehXv3pW//25WX16ZknPr2e9LgkA0KZMw9BY3Wyp7SV6fUG/enY0Ni9bti7l6vtNAgCaLxAeUGLoVY6xzYW/U6W04VFFnc8fTCoQGXGM5TfPeFQNsH9aIpTq7+/XRz7yEZ08eVKPPPKIPvjBD+r1r3+9HnvsMQWDW7+gPvPMM3r44Yf1zDPPeFxt90hlCqra0ly2qC8truvrK+3fZA0A4J26JXyZ7dDJMAxNxp2zpVjCBwCtIznyXTL9271+bbui9Yt/5WFFnS+SPO44pq8UOlFLLN+TpNtuu03vf//7dz3/oz/6o/rRH/3Rqz7G2NiYpqam9rq0rjWbdTaZdTepBQDgRtQ1O886Q6fJRET/tJapHc+k8/qukb6m1AYAuDrTF1bvwddpNfWntbHc2jdVHHyFQvEJDyvrXJHkcW3O/3XtuJA+K9uqyDBb5jIeuGUtMVMKrcm9M9JEjFAKAHDz3Mv3LuVKqlhW7fiwK7SazhRk23ZTagMAXFts4GV1S8rW5j7He/U+CUYOyhdI1I5tq6xCZtq7goB9QCiFhjZKZW2WK7Vjv2FoJBrysCIAQLvrCfrVG9z+dbdq27qUK9WOh6MhhXZsrpGrVLVcKDe1RgDA7gzDVN/YmxxjpfwlZVe/7lFFnc0wDEV6jjnGWMKHTkMohYZmXbOkDsVC7IAEALhlV2t27jMMTbhmS83QVwoAWko4Pqlo74sdY+sXPy+ryuYU+6FRXylmpqGTEEqhIffSPXdzWgAAboa7r9Rsxt1Xyr2Ej1AKAFpN7+jrZRjbM1+tSkabC3/rYUWdK5Q4IsMM1I6r5U2V8/MeVgTsLUIpNESTcwDAfnD3J5zLOn9Zd+/AN5N2fh4BALznD/UqceA1jrHMytdkWSy53mumGVA4cZtjjCV86CSEUqhTsWxddF0kuJdTAABwM0ZjIe1cDb5aLCuzo4fheCzsOL9SLCu94zwAoDX0DH/H5SbchuKDr9DBF/1fMnfM6MHecS/hyxFKoYOwlyTqzOeKquxYp9wT8CsZ5AMGAHDrAqapO3rjCvpMjcXCmoiHFfX7aueDPlOHomFHr6mZdF539icaPRwAwCOmL6iBiXtkBmIKRoa9LqejuZudl/OXVCltyh/s8agiYO8QSqGOe+kes6QAAHvpx24/eNXzk3FXKJUpEEoBQAsK99x27RvhlvkCcQWjh1TKXaiN5TdPKzH4Cg+rAvYGy/dQp67JOaEUAKCJDiecfaWm0zQ7BwB0t0jyhOOYvlLoFIRSqJNyz5Ri5z0AQBO5Z+heyhVVrFoeVQMAgPfcfaUK6fOyqiWPqgH2DqEUHDLlilaL27tmmMZWU1oAAJolHvBrMLzdy9BS/Q8mAIDWZNu2cmvPaWP+b70upaMEwkPyBXu3B+yqCulzntUD7BVCKdR5/aF+nUhGFfGZOhgJKWDyzwQA0FyH484lfDMs4QOAllfKXdLimQ9qefrj2rj0BZULS16X1DEMw1C0bgnflEfVAHuHRudwiAf8et3ogKStXznyLJcAAOyTfKWquWxBqWxBUb9Prz7QWzs3mYjoK8ubteOZDKEUALQy27a0fP7jqpTWLo9YWpv7cx24/R2e1tVJIsnjSi99uXac3zwj27ZkGEwiQPsilMKuDMNwbNMNAMBeObuZ0+NT27sIHYyGHKHUYVdfqdlMQVXbls8wmlUiAOAGGIap3tHv0/L0x2tjhfRZ5TfOKJI85mFlnSMUn5DhC8muFiVJViWnUvaCQvFxjysDbh6RKgAAaLqDUWe/woVcUaUds3P7QwHFd/wwUrJsXcoVm1YfAODGRXpfpFB80jG2duHPZVtVjyrqLIbhUyRxu2OMJXxod4RSAACg6aJ+nwZCzmbmF3aEToZhaDJBXykAaCeGYajv0JscY5XiitLLz3hUUeeJuPtKbZ7xqBJgbxBKAQAAT4zHnEv0UhnnDnvuJXzTGXbgA4BWF4yOKD5wl2NsY/6LqpazHlXUWSI9RyVtL2UvF5ZULq56VxBwiwilUHNuM6dLuaIs2/a6FABAFxh3hU6p/z979x0eVZn+Dfx7ps9kZjKTXiFUaYIoIALSpCrKimLvXXFdldXV1VVXXyw/1o5dVxRdLChNEUQUEAFBmoiETvqkzkxmMr28f0QnOSmQkHIyyfdzXV5e5z7tZsIwk/s8z/1UiYtODY2UCvMzioiow4tNnQBBVjNNOxz0wl60XrqEOhGZQltviqTbflCibIhajkUpilieU4JX9+XiyZ1H8G52Piq8fqlTIiKiTqzuSKn8OiOhUnVqqGQ1T4OdgSA/m4iIooBcGYPYlLGimLN8J3wui0QZdS7a2L6ibRalKJqxKEUAqpflLvVUf9H3hcI45nAjhivvERFRG0rRqaGotZqe3R+A3ReIbMsFod5oKk7hIyKKDobEEVCo42pFwrAWrOGI11ZQtyjldeYgFGDfRYpOLEoRgPpTJpK1Kqjl/OtBRERtRyET6q3CV7evVHc9m50TEUUjQSaHOX2KKOZ15sBtz5Yoo85DqY6DUpNYKxKGu/KwZPkQtQSrDgSg/i8BdZ9MExERtYVudT5v8qvqNjsXF6WOO1mUIiKKFhpjH2gMvUQxa8FahEOBRs6gpuIUPuosWJQiAPVHSnWLYVGKiIjaXkadz5vcqvoPSWp/WSnz+OH085cZIqJoIAjCH6OlaqZqB302VJZskS6pTkJrrFOUchxGOBSUKBuiU8eiFCEUDjcwUkrbyNFEREStp+7I3IIqD4K1+o2o5bJ6U/xy2VeKiChqKLWJ0CcOj2xrDL2gM/WTMKPOQRWTDplCF9kOB73wVuVImBHRqWFRilDm8cMdDEW2NXIZEjRKCTMiIqKuwqxSiBbW8IfCKHH7RMd0N9SZwse+UkREUSU2ZRxUMRlI7HkFEntdVacfEp0KQZBBa+wjirnthyTKhujUsShF9abuZcZoIKu1GhIREVFbERpYYa/u6N2sOvtzOFKKiCiqyBVapPS9CdrYvhD4e0ar0caeJtp22Q9wdUOKOixKEfLqNI1lk3MiImpPmbX6SskFwFGnZ1TdkVIFLg98tUb4EhERdUUaQ09AqBltHPTZ4PeUSpgRUfMppE6ApFevnxSbnBMRUTvqZ4qBUiagm16DVJ0aSpn4mZlBqUC8Wolyrx8AEApXr9LX06hr6HJERERdgkyugsbQA57Kw5GY234QKm2ShFkRNQ9HSnVx3mAIljq9OzhSioiI2lOqTo0xKWZ002vrFaT+1N0g/mw6zil8RESdgreqAKGg7+QHUoO0sXVW4bMfkCgTolPDolQXV1DlQe1ZxwkaJXS1Gs4SERF1BFl1VoXNYbNzIqKoFvA7UJ6zDMUH30NlyWap04laWqO4KOVzFSDod0qUDVHzsSjVxdVdVrsbp+4REVEH1L1OUSrX6UGIzVyJiKKSx3EMRb8vQFXFrwAAR/FmBHw2aZOKUgqVEUptqijmruQqfBQ9WJTq4uqtvMepe0RE1AHVHcnrDdWffk5ERNFBpUuDIFNFtsPhAGwF6yTMKLrp6k3hOyhRJkTNx6JUFzezexKu7p2KsSlmZBm06FbnSTQREVFHIAgCsuo8ODnOKXxERFFJJlfDlDZRFHPZ9sHjzJEoo+hWt6+Ux3EUoZBfomyImodFqS7OqFJgoFmPaZkJuK1fBlJ1aqlTIiKiLirP6cEmixX/O1yE/9tzDBVe8Rfq7gb2lSIi6ixi4oZAqU0Rxaz5axAOhyTKKHoptSmQK42R7XDID6/jmIQZETUdi1JERETUIazOL8OqvDL8ZnXC5gsgr07fw3rNzp1uhNlXiogoKgmCDOaMqaKY321BVcUeiTKKXoIgNLAKH6fwUXRgUYqIiIg6hMw6i23k1+l7mKpTQykTItuV/iCsvkC75EZERK1Po+8OnWmAKGYr/B6hoFeijKJXvaJU5SE+uKGowKIUERERdQgZdYpSdVeIVciEesdwCh8RUXQzpU+CICgi26FAFeyWjRJmFJ00+iwIMmVkO+h3wOcukjAjoqZhUYqIiIg6hG51GpkXubwIhMRPeetO4TvuZFGKiCiaKVQmGJLPEcUcpT/D7ymXKKPoJMgU0Bh6iWJu+wGJsiFqOhaluqgilxcrckqwu7wSFR4/h3YSEZHkjCoFYpU1T8sD4TAsLvEUju6GuiOlxKOpiIgo+hiTRkOuNNQEwiHYCtdKl1CU0saeJtp22w9JlAlR07Eo1UUdqXRha4kdnx0txn/2HsfS4yVSp0RERISMOqOl8ur0leqm10CotV3i8cEVCLZDZkRE1FZkchVMaZNEMbf9IDyVRyXKKDppjb1F2363BQGvTZpkiJqIRakuqm6fjmStSqJMiIiIanSr0zOq7gp8GrkcKTq1KJbDKXxERFFPZx4EVUyGKGYtWINwOCRRRtFHroyBOiZTFLNbNkiUDVHTsCjVRdV/8qxt5EgiIqL2c7KRUgDQXc8pfEREnY0gCDCnT60XD/orJcgmesXEDRZtV1XsgbeqQKJsiE6ORakuyO4LwF5rCW25ICBVx5FSREQkvXSdWvTlpNzrrzc9j83OiYg6J3VMOmLihkAm18KcMR0p/W6HQmWSOq2oEhM/FEpNoihmzV/NHsLUYbEo1QXVnQqRplNDIeNfBSIikp5KLkNynel5dT+3uhvERamCKg/8IU7vICLqDExpk5A64G4YEodDEPg7SnMJgqzeiDOfqwAu628SZUR0YnyXd0ENNY0lIiLqKDLr9pWq87kVq1LArK5ZpS8YBvKrxKv0ERFRdJIrYyBXsLVIS2iMPeutxGcr/A6hoE+ijIgax6JUF5RbZ5pDJotSRETUgdT9XKo7UgqoP4Uvx8EpfERERH8ypU8GBHlkO+h3oLLkJwkzImoYi1JdTDAURkGdp8l1VzoiIiKSUt2RUvlVnnq9MLrXLUqxrxQRUafHlfiaTqmOgzHxbFHMUbwFAa9NmoSIGsGiVBdjcXsRqPXF3qCUI1alOMEZRERE7StBo4RGLoMAIEmjwgCzHr5QnaKUoc4KfE4PQmziSkTUKYVCftiK1sNy4F2EQ8GTn0AAAGPKuZAp9JHtcDgAW+F3EmZEVB+rEV1Mbp0pEJkxGgiCIFE2RERE9ckEATeflo54tRIahbzBYxI1KmjlMriD1U/NPcEQStw+pNRpkk5ERNGtyroPtoK1CPorAQCOsm0wJp0jcVbRQSZXw5Q2ERW5KyIxl+13eBzHoTFkSZcYUS0cKdXFsMk5ERFFg/QYTaMFKaC6cFV3Fb7jnMJHRNTpeB3HIwUpALAXbUTQXyVhRtElJm4IVLo0UcxasIZTIanDYFGqi6nbLDZTz5UtiIgoOmXVebCS46jfEJ2IiKJbbOp4CPKaUbDhkBf2oh8kzCi6CIIAc/pUUczvLkZV+S6JMiISY1GqC6nyB1Hu9Ue2ZQDSOc2BiIiiVN1m5xwpRUTU+ciVMYhNGSeKOct3wueySJRR9FHrM6EzDxLFbEU/IBTgwxySHotSXUiF1w+douZHnqJTQyXnXwEiIopO6TFqKGr1RbT7ArDVevhCRESdgyFhOBTqeFGsegoaF7hoKlPaJAgyZWQ7FHDBbtkgYUZE1ViR6EIy9Ro8ckZPzD29O2b3SMbYFLPUKRERETWJOxCsV3BSyGTIiBGP+OVoKSKizkeQyWFOnyKKeZ05cNuzJcoo+ihURhiTR4tijtLt8HvKJMqIqBqLUl2MIAiI16gwNMGIwfEGqdMhIiJqlMXlxZJjFry49zie2nUU3+TV/+Jct9k5+0oREXVO2tg+0Bh6iWLWgrUIhwISZRR9DEnnQK6MrRUJwVrwrWT5EAEsShEREVEH5QuFsLPMgVJP9QipuivIAkBWnb5SORwpRUTUaZkzpqD2r7BBnw2VJVukSyjKyGRKmNMni2KeysNw2w9JlBERi1JERETUQaXq1JDXtIyCzReAwy9+It5Nr0GtQ1Ds9sEdCLZPgkRE1K6UmkQYEoeLYpXFmxDwOyTKKPpoTf2h1ncTxawF3yIc4mcnSYNFKSIiIuqQlDIZUuusEpvnFI+W0irkSNaqItthALlOTuEjIuqsYlPGQiavGSUbDvlhL1wnYUbRRRAEmNOniWIBbzkcZdslyoi6OhalugiuTEFERNEoM0Yj2m5oCl/3OlP42OyciKjzkim0iE2dIIpVVfwKb1W+RBlFH5UuBfr4M0Uxu2UDgv4qiTKiroxFqS7izf35eCc7H2vyyvC71Ql/KCR1SkRERCeVqa9TlGpgFFR3g/iYHAeLUkREnZk+4UwoNUmimN3yo0TZRKfY1AkQZDWjkcNBL+xFP0iYEXVVLEp1AZ5AEPlVHhxzuLHBYsXHh4sQDHHkFBERdXx1R0rlV3kQqjP6t26z8/wqLwJ8+EJE1GkJguyPpucAIIMhaSQSsi6WNKdoI1fGIDZ1rCjmLN8Jn8siUUbUVbEo1QXkVXlQ++t7olYFjUIuWT5ERERNFadWQlfrM8sXCqPE7RMdY1IrEatSRLYD4TAKqrztliMREbU/jaEnYlMnIrX/HTCnT4FMrjn5SSRiSBgBhTpeFLMWrGbrF2pXLEp1AXX7b3SL4T/YREQUHQRBQGZMnWbnDfSVqjtaKod9pYiIOr3YlDFQahKkTiNqCTI5zOlTRDGvMxdu236JMqKuiEWpLqBu/426/TmIiIg6slPpK3WcK/ARERGdlDa2DzTG3qKYtWAtQiG/RBlRV8OiVCcXDofrLY1dtz8HERFRR9ZQX6m66o2Ucrjr9Z4iIiKi+qpHS9WUBoJ+OxwlW6RLiLoUFqU6uXKvH+5gTbNXtVyGJK1KwoyIiIiaJ6NOUarY7YM3KG5knqRVQSOv+VrjDoZQ6hH3niIios4vHA7DWb4HXmee1KlEDaUmAYbE4aJYZfFPCPgqJcqIuhIWpTq5elP3YtSQCYJE2RARETWfViFHokYZ2Q6j/mgpmSCge51pfrvLHe2RHhERdRDeqjwUH3wPFbnLUZH/DcJhrsTaVLEp4yBT6CLb4ZAftsJ1EmZEXQWLUp1c/al72kaOJCIi6riaMoWvt1En2v7JYoPVy54YRERdgd9TjuKD78PnKqzedltQVb5b2qSiiEyhgSl1gijmsu7liDNqcyxKdXJ1Vyhik3MiIopGGXoNzCoFTo/T4/zMBPQzxdQ7ZlhiLAxKeWQ7EA5jTX5Ze6ZJREQSUWrioTMNFMVsRT8gFOTCF00VEz8USm2yKGYtWIMwezRSG2JRqhPzBUOwuLyiGJucExFRNBqRGIsHhvTAlb1SMSbFjGStut4xarkMU9LjRbFfK5zIdbrbK00iIpKQKX0SBEER2Q4FqmC3/ChhRtFFEGQwp08VxXyuQlRV7JEoI+oKWJTqxApcXtSeRR2vViKm1hNkIiKiaNHUfohDE4xI1YkLVl/nlvEpLxFRF6BQxcKQPEoUc5T+DL+nXKKMoo/GkAWtqb8oZitch1DQ28gZRC3DolQnllfnyTCn7hERUWcnEwRckJkgiuVVefBrhVOijIiIqD0Zk0ZBrjTWBMIh2ArWSpdQFDKnTQaEmsEMoUAVKjnijNoIi1KdWP0m5yxKERFR59fTqMOAOj2n1uSXwR/iKkxERJ2dTK6CKe08UcxdeRDuyiMSZRR9FGoTjEniEWeVpT/D762QKCPqzFiU6sS66TXoYdBCKRMi20RERF3BtMwEyGvN+LP5AvjJYpMsHyIiaj868yCoYjJEMVvBtwiH+XCiqYzJoyFXGmoC4SBHnFGbYFGqExubGodb+2XgsTN74e6B3ZDSQFNYIiKiaOMPhZDjcGOTxYpgqOFeUQkaFUYmmUSx9UUVcPgD7ZAhERFJSRCEeg27/Z5SOMt+kSij6FM94mySKOa2H4Cn8qhEGVFnxaJUFyAXBKTp1JDLmtYkloiIqKP68GAh/r3zCN7KzseqvDJY3I03Xp2YFgetvOarji8UxncFbHZLRNQVqGPSERM3RBSzF61HMOCSKKPo09CIM2vBGo44o1bFohQRERFFDZkA1B4clVflafRYrUKO89LjRbFfSitR5OIKQkREXYEpbSIEmSqyHQp6YC/aIGFG0aXxEWc7JMqIOiMWpYiIiChq1F1JNs/ZeFEKAM5OjEWCRhnZDgNYlVeKcLjhaX9ERNR5yJUGGJPHiGLOsl/gc5dIlFH0aXjE2Q8ccUathkUpIiIiihp1V5I90UgpAJDLBJyfmSCKHal044C9qtVzIyKijseYNBJylSmyrY09DTK5qvETqB6OOKO2xKJUJ+T0B1Du8fEpMBERdTrpMRrU7pBY5vHDHQie8JzTYmPQy6gVxVbllTXaJJ2IiDoPQaaAOX0KlJokJPW+Fok9L4OiVpGKTo4jzqgtsSjVCe0pd+D5vTmYt/sYPjxYiH1Wp9QpERERtQq1XIZkrfgJ98lGSwmCgPMzE+sVs34utbdBhkRE1NFoY09DSr/boDH0kDqVqGVMGgmFylwrEoY1fw0HQlCLsSjVCeX+8eXcFQgi214Fq9cvcUZEREStp7l9pQAgVafGsESjKLauoPyko6yIiCj6CYIAQeCvvi0hyBQwpU8WxbzOY3DbD0qUEXUWfGd2QnW/nHer8+WdiIgomjW3r9SfJqXHQyWrGS/lDobwfWFFq+ZGRETUWWljT4NaLx5tZiv4FuFQQKKMqDNgUaqTqfQFYPPV/KMgF6qfDhMREXUWdUdK5Vd5mjR9wKBUYHxqnCi2tcSGMo+vVfMjIqLo4ffy4URTCYIAc8ZUoNaE+IDPCkfpz9IlRVGPRalOJr/O0+JUnRpKGX/MRETUeSRqVFDX+mxzBUKoaOJU9dEpJphUish2MAyszitr9RyJiKhjC3htKDu2BEW/vwafq0jqdKKGSpsEfcIwUcxu+RFBP/sY06lhtaKTya0zdS8zRtvIkURERNFJJghIjxGPAq77+dcYpUyGqRkJotjvtiocqXS1Wn5ERNSxOUq3o3D/a3DZfgcQhrWADbubIzZ1HGTymlHL4ZAPtsLvJcyIohmLUp1M3b4a7CdFRESdUd3Pt6b2lQKAwXH6en2pVuWVIcRfSIiIugSFygSEaxa68Dpz4bbtly6hKCNX6BCbOl4Uq6rYDW9VgTQJUVRjUaoTCYbD9abvsShFRESdUd2iUt3PvxMRBAEXdBOPlipyebGrrLJVciMioo5NG9sHGmNvUcxauBahEFctbyp9wllQahJFMY44o1PBolQnUuz2wR+q+UdAr5CL+mYQERF1Fhl1HrpYXD4EQk3/ItxNr8XgOL0o9m1BObzBUKvkR0REHZs5fQpq/zoc9NnhKNkiXUJRRhDkMKVPEcV8VflwWX+TKCOKVixKdSJ5TrdoO1OvgSAIjRxNREQUvQxKBc5KMGJyejxu6puOf57RAwpZ8z7zpmYkQFHrc9LhD2KjxdraqRIRUQek1CTAkDhcFKss/gkBH0fNNpXW2Ava2L6imK3wO4SCXNWWmo5FqU4kz8mpe0RE1HVc0iMZE9Li0DtWB41C3uzzzWolRqeYRLFNFitsTVzJj4iIoltsyljIFLrIdjjkZ8PuZjKlTwGEWiPO/A5UlvwkYUYUbViU6kRyq+quvMeiFBER0YmMT42DvlZByx8K49uCcgkzIiKi9iJTaOs17HZZf4W3Kl+ahKKQUh0HQ+JIUcxRvAUBr02ahCjqsCjVSbgCQZR5ap7sCgDSWZQiIiI6IbVchskZ8aLY7nJHvdHHRETUOenjz4RSkyyKWfPZsLs5YlPOhUwRE9kOhwOwFX4nYUYUTViU6iTqfnlO0amhlvPHS0REdDJnJRiRolWJYqvySvkLCRFRFyAIMpgz6jTsdhXAZd0rUUbRRyZXw5R2nijmsv0OjzNHoowomrBq0Umkx6hxWc9kjEyKRbpOjSz2kyIiImoSmSDg/EzxstY5Tg9+szolyoiIiNqTxtAD2th+opitcB0bdjdDTNwQqLSpolj1iDOuaksnxqJUJ6FXKnBGvBEXdU/CnIHdMKNb4slPIiIiinLBcBiFLi9+LrFjyTELthTbTuk6vWN16BcbI4qtzi+DP8Qv00REXYE5fTIg1PQYDPodqCzeJGFG0UUQBJgzpolifrcFVeW7pUmIogaLUp2UIDRvWWwiIqJo9Gu5Awv25WJ5Tgl2ljnwu+3URzdNz0yArNbHp9UbOOUiFxERRReF2gxjkrhht8d5nFO5m0Gtz4TOPEgUsxV9j1CAfRqpcSxKERERUdTKrDNdPb/Ki9Ap/gKRqFXh7MRYUeyHIiuc/sAp50dERNHDmDwGMoUeMkUM4rpdiOQ+N/BhfzOZ0s6DICgi26GAC3bLRgkzoo6ORSkiIiKKWvFqJbS1FvbwBkMo9Zx6D5Dz0uOhqXO97woqWpQjERFFB5lcjcSelyNtwN3Qxw+FIPDX5eZSqGJhTB4tijlKt8HvKZMoI+ro+C4jIiKiqCUIQr3RUnVXpG0OnUKOiWlxotj2UjuK3d5TviYREUUPdUw6ZHK11GlENUPyKMiVtUceh2At+FayfKhjY1GKiIiIolpGTJ2iVFXLeleMTDIhXq2MbIcBrMrlE14iIqKmkMmUMKVPEsU8lYfhth+SKCPqyFiUIiIioqjWrW5fqRaMlAIAhUzA9MwEUexQpQsHbFUtui4REVFXoTMNgDqmmyhmLfgW4VBQooyoo2JRioiIiKJa3ZFSFrcPvmCoRdfsb4pBD4NWFFuVV4YgV2EiIupyQkEvrAXfwVb4vdSpRA1BEGDOmCqKBbzlcJRtlygj6qhYlCIiIqKoplPI6023y2/hFD5BEHBBZgJqr7lU6vFhe6m9RdclIqLoEQ6H4SzfhcLfF8BRshmVJZvh95RLnVbUUOlSERN/pihmt2xA0M+Rx1SDRSkiIiKKenWbnbe0KAUAaTEanJlgFMW+K6iAO8CpB0REXUE4HIC9aANCgT+KKGE27G4uU+oECLKaxvHhoBf2ovXSJUQdDotSREREFPUyW7nZ+Z8mp8dDJasZL+UKBLG+qKJVrk1ERB1bww27D8FdeViijKKPXBmD2JSxopizfAd8LotEGVFHw6IUERERRb26I6XynN5Wua5RpcDYVLMotrnYjnKPr1WuT0REHZvONBDqmExRzFrwLcJhjpptKkPiCCjUcaKYtWANwuzTSGBRioiIiDqBFK0aCqFmRFOlPwC7z98q1x6TbEasUhHZDobDWJ3PniJERF1Bgw27PWVwlu2QKKPoI8jkMKdPEcW8zhy4bfslyog6EhaliIiIKOopZALSdGpR7KDd1SrXVsllmJIRL4rtszpxzOFulesTEVHHptKlISbuDFHMXrQewUDrfM50BRpjH2gMvUQxa+FahEKt8wCJoheLUkRERNQpdDdUT+ETAIxNMWNovKHVrj0k3oCMGHHRa1VuKUKcekBE1CWY0iZCkKki26GgB/aiDRJmFF2qR5xNQe0SRNBnh6Nki3RJUYfAohQRERF1CiMSY5GkUeHWfhmYlpkAhaz1vubIBAHnZyaKYgUuL3aXO1rtHkRE1HHJlXrEppwrijnLfoHPXSJRRtFHqUmEIXG4KFZZ/BMCvkqJMqKOgEUpIiIi6hTiNSrcM6gbsgzaNrl+lkGLQWa9KPZtfjl8wVCb3I+IiDoWQ+LZUKhqL34RhjWfDbubIzZlLGQKXWQ7HPLDVrhOwoxIaixKERERUachq9XsvC1My0yAvE5D9R8t1ja9JxERdQyCTAFT+mRRzOs8Brf9oEQZRR+ZQovY1AmimMu6F96qPIkyIqmxKEVERETURHFqJUYnm0SxjRYr7L6ANAkREVG70saeBrU+SxSzFXyLcIifA02ljx8KpSZZFOOIs66LRSkiIiLq1ELhMH4ssuL7wopWud74VDNiFPLItj8Uxtr8sla5NhERdWzVDbunonpZjWoBnxWOsh3SJRVlBEH2x2tYw+cqRFXFrxJlRFJiUYqIiIg6LavXj3cPFOCb/DJ8X1COPKenxdfUKOSYlB4niu0sd6CgquXXJiKijk+lTYY+4azqDUEOY/IY6OOHSptUlNEYsqA19RfFbIXrEAp6JcqIpMKiFBEREXVKoXAY7x8swHGHu3obwGdHLa3SmHxYYiyStCpR7Ou8Mk49ICLqImJTxyMmbgjS+s+BKW0iZHLVyU8iEXPaJECoGXkcCjhRWbxJwoxICixKERERUackEwRckJkoipV7/fimFabayQUB52cmiGLHHW78bqtq8bWJiKjjkyt0iO8+Ewq1SepUopZCbYYx6RxRrLJkK/ze1pluT9GBRSkiIiLqtE4zxWBEYqwo9nOJHQftLS8e9Y2NQd9YnSj2TV4ZAqGWj8QiIiLqCozJYyBXGmoC4SBsBWulS4jaHYtSRERE1Kmdn5mAOLVSFPviWDFcgWCLrz09M0H0ZarC68eWEnuLr0tERNQVyOQqmNLOE8Xc9gPwVB6VKCNqbyxKERERUaemkstwWc/kWuskAQ5/EMtzSlp87WStGsOTxCOxfiisQJW/5QUvIiKKPuFwCM7y3QiF/FKnEjV05tOh0qWLYtaCNQiHOfK4K2BRioiIiDq9bnotxqWaRbG9FU7sKXe0+NrnpcVBI6/5SuUJhrCusLzF1yUioujicRyDJfttVOSugKNki9TpRA1BEGDOmCaK+T2lcJbtkCgjak8sShEREVGXMDEtHmk6tSi2PKcEdl/LnmbrlQpMSI0TxbaV2FHi9rXoukREFD2c5btRcngR/J7qUbiVxT8h4KuUOKvooY5JR0zcEFHMXrQewYBbooyovXSYolReXh7uvvtujBgxAiNGjMCDDz6IioqTd93/8ccfcdVVV2HIkCEYOnQobrjhBuzevbvtEyYiIqKoopAJmN0zGQqhZiKfJxjCF8dKEAqHW3Ttc5JjRX2rQgC+yStt0TWJiCh66GL7QaaoWfwiHPLDVrhOwoyijyltIgSZKrIdCrpht2yQMCNqDx2iKGW1WnH99ddj9+7duOWWW3DjjTfi+++/x4033gifr/GnjNu2bcOtt94Kh8OB++67D3PmzEFubi6uueYa/Prrr+34JyAiIqJokKxVY0pGvCh2uNKFn1vYnFwhk2FanesesLtwqBVW+SMioo5PptAgNnWCKOay7oW3Kk+ijKKPXGmAMXmMKOYs3Q6fu+U9IKnj6hBFqYULF8JiseCDDz7AbbfdhjvvvBOvvPIKsrOzsWzZskbPe/rpp5GamorPPvsMN9xwA2655RZ89tln0Ol0ePHFF9vvD0BERERRY1SyCT0MWlFsdX4ZSls43W6gWY8svUYUW5VXhmALR2EREVF00McPhVKTLIpZ89cgzM+BJjMmjYRcZaoVCcNW8C1fw06sQxSlvv76a4wYMQK9evWKxEaNGoUePXrg66+/bvAcu92O7OxsTJs2DVptzRfLhIQEDB8+HLt27WrzvImIiCj6yAQBl/ZIhlpW8zVIKRNg9wdadF1BEHB+t0RRrNjtw45S9hQhIuoKBEEGc8ZUUcznKkRVBWfxNJUgU8CcPkUU8ziOwl15UKKMqK1JXpSy2+3Iy8vDwIED6+0bOHAg9u3b1+B5er0eq1evxg033FBvn9VqhVwub+1UiYiIqJMwq5WY0b26gNQvNgZ/G9QdvY26k5x1chkxGgyNN4hiawvK4QkGW3xtIiLq+DSGLGhN/UUxe+E6hIJc/KKptLGnQa3PEsVs+d8iHGrZwyPqmCQvShUXFwMAkpOT6+1LTEyEw+GAw1F/uWa5XI6srKx652VnZ2Pnzp0YOnRo2yRMREREncKZ8Qbc0DcN1/ZJhUGpaLXrTsmIh1JW00y9KhDEhkJrq12fiIg6NnPaJECoGSQRDDhRWbxJwoyiiyAIf4w4q/ksDfiscJT+LF1S1GZa7xvYKaqqqm4AWnsK3p/U6uplm10uFwwGQ739DV3rH//4BwDgtttua3Yu8fH6kx5js9lQWlp2wgbsRC1R0sZ9/ORyOYxGA+Lj4yPvMSKqLzHx5J87FP2Skoytfs1EANOqPFh52BKJ/VRiw7R+aUjQ8d/dpuJ7kEh6fB+eKgNC7nGwHPs+EnGUbkW3PudCrYuTMK9oYkCwaiRK87ZEIpXFm9Ctzygo1a3/2d1RdYX3oORFqaY0LBNqLd3cGLfbjTvvvBPZ2dm4/fbbMWLEiGbnUl7uRCjUeD5+vw9WawlMpgTExqqblBdRcykUMgQCoTa5djgcRjAYhMdThcOHjyIuLhkKhfLkJxJ1MYmJBpSW1h+lS9RUZxljsEEpR6W/etpeIBTG//bk4MreqRJnFh34HiSSHt+HLaMwjIBcsQ3BgBMAEA4FcOS3ZUjsMVvizKKHyjQassJdCAU9AIBQ0Isje1civvtFEmfWPjrLe1AmE044AEjy6Xs6XXX/Bq/XW2/fnzG9/sQjmCorK3HTTTfh559/xiWXXIL77ruv9RMF4HDYoNfHQqXSsCBFUUkQBCgUCuj1sdDpDKiqYvNdIqKGuAJBFFZ5Tvl8lVyGKRkJotheqxM5DndLUyMioiggk6sQm3aeKOa27YfHcVyahKKQXKFDbOp4UayqYje8rkJpEqI2IXlRKi0tDQBQWlpab19JSQmMRmOkcNWQ8vJyXHfdddi5cycuv/xyzJs3r80KRoGAD2p1/WmGRNFIo4mB18tfjoiI6jpgq8LLv+Vg0aEiuAOn3qD8jHgD0upM1/s6rxQhLmtNRNQlxMQNhkqXJopZC9YgHG6bWRGdkT7hLCg14pVtrfmrmzTjiqKD5EUpo9GIjIyMBlfZ+/333zFo0KBGz3U6nbj55puxf/9+3HDDDXjyySfbdARTKBSETMZV/ahzkMvlCIW4GhQR0Z/C4TCWHy/BB4cK4fAHYfcHsDK3/kOzppIJAi7oJv4inV/lxa8V0T8Un4iITq6mYXcNpSYR4ZBfooyijyDIYUqfIor5qvLhstavH1B0krwoBQBTpkzBli1bcOTIkUhs8+bNOHbsGM4///xGz3vyySexf/9+XHfddXj44YfbI1VO26NOg3+XiYjEBEGAWi7+arS73IG9LSgi9TBoMdAcI4qtyS+HL8in5EREXYE6JhM68+lQ6dKQ3PdGJGTNgkzORS+aQ2vsBa2xryhmK/wOoSAXH+sMJG90DgC33norli9fjhtuuAE33XQTvF4v3n33XQwcOBAzZ84EAOTl5WHnzp0488wzkZmZiSNHjmD58uUwGo3o378/li9fXu+6f55LRERE1BST0uNw0F4Fi7vmi+6y4yXortfCqDq1r03TMhKQbatC8I+ZBnZfAD8V2zAhjSswERF1BXGZ50OQqfhQuAVM6ZPhdhwG/pj6GPRXorJkM0x1ek5R9OkQRam4uDh89NFHeOaZZ/DKK69Ao9Fg0qRJePDBB6FSqQAA27dvx8MPP4xnnnkGmZmZ2LZtG4DqJueNjZJiUYqIiIiaQyGTYXbPFLz+ex6Cf/SrcAdD+PJ4Ma7vk3ZKv1DEa1Q4J8mETcW2SGxDUQXOSjCecqGLiIiiB0dGtZxSEw9D4tlwlGyJxBzFm6GPPwMKlUm6xKjFhDA7hEWUlzsRCjX+clgsOUhJ6d6OGbW/efOewDfffHXS46ZPn4FHHnmiRfdatWolnn7633jllTdx5pnDWnStjnzPui699EKkpKRiwYK3G9yvUMgQCLTPtI6u8Hea6FR0liV46dRtLKrA6vxyUWxm9yScnRR7StdzB4J4fu9xuGr9+z4swYhZPZJblGdnxfcgkfT4PqSOJhT0ovD3BQgFqiIxnWkAEnpcKmFWbaezvAdlMgHx8fpG9/PxHInMnDkLw4aNiGzv2bMLK1YsxUUXXYwhQ4ZG4unpGS2+15AhQ/Gvfz2JrKweLb4WERFRaxqTYsZ+WxVynJ5IbFVeKXobtYjXqJp9Pa1CjvPS4kWN03eUVWJksqneCn1ERNR1hAJuyBRc4b0pZHI1TGkTUZG7MhJz2X6Hx5kDjZ4P2qMVi1IkMmjQYAwaNDiyHQwGsWLFUgwaNBhTpzbedP5UpKdntEpxi4iIqLXJBAGze6TglX058P0xitofCuPzo8W4tX8G5KcwjW9EYiy2lthQ6qledSkMYFVuKW4+LZ19RoiIuhi/pxzWgm8R8FqR2v92CAJXeW+KmLgz4Cz9BT53USRmzV+DlNNugSB0iHXcqJn4UyMiIiJqQJxGiQu6JYpiuVUe/FhkPaXryWUCzs8UX++ow41sW1UjZxARUWcTDodhLViLouw34Kk8hIC3DI7SX6ROK2oIggBzxlRRzO+2oKp8tzQJUYuxKEWn7L333sLEiaOwYcMPuOiiqZg8eSy++moZAODAgWw88sgDuPDCKRg37mzMmDEZTzzxCEpKiiPnr1q1EmPGDMPOnb+Itg8dOognnngE06ZNwOTJ5+Lhh+eiqKjwpPl4PB68+eYCXHrphRg/fiQuvfRCvPHGq/B4PCc8b/36dbjlluswefJYTJ06Dvfeexd+/XX3Kb0mR44cxv33340ZMyZh4sTRuOmmq/HVV/VXhqytvLwMl102ExdeOAW5ucdP6b5ERNQ2hiUY0c8UI4qtKyxHoct7StfrG6tDH6NOFFuVV4bACXpaEhFR5yEIAkJBT2QVOQCwWzYgGHBJmFV0Ueu7QWceJIrZir6vfl0p6nD6Xgu5svej5KMP4bMUnfzgdqRKSUXSNddB169/m94nEAhg/vynccUVV8Pn82Hw4DNw5Mhh3HXXzcjI6IZrr70BarUGe/fuwZo1q1BQkId33vnwhNd86KH7kZXVE7ffPgcFBfn4/PPFKCsrPeF5fr8f9913F377bS/OP/9C9Os3AL///hs+/vgD/Prrbrz66ltQKOr/dd+1awcef/yfGDlyFGbMmAmPx40vvvgc9947B4sWfdqs6YU2mw333z8HsbEmXHfdzVCpVPjuuzV49tmnoFKpMWXKtHrnOBwO3H//3XA6nXj11bfQrVtWk+9HRERtTxAEXJyVhJd/y4UrEAQABMPAZ0ctmDMgE0pZ857vCYKA6ZkJOLwvF3+Wocq9fvxcYsPoFHMrZ09ERB2RKXUCXNZ9CId8AIBw0AN70XrEZbZuu5TOzJR2Hty2bITDAQBAKOCC3bIR5vQpEmdGzcWiVAsVL1oIf3HxyQ9sZz5LEYoXLUSPec+16X1CoRCuuOJqXHPNDZHYf/7zDARBwKuvvgmjsXqVopkzZ8Hv92Pdum9RWWmPxBvSr19/zJs3P7Lt8bixbNkXyMvLRWZmtwbP+frr5di791fcc8/9uOyyqwAAF198KXr06InXX38FK1YsxaxZs+udt27dWqjVGjz77AuRfh7Dh4/Eo48+iIMHs5tVlNqxYzvKy8vx3HMvol+/AQCACy64CLfffiOOHj1c73iv14MHH7wXxcUWvPTSG+jVq3eT70VERO3HoFTg4qwkfHy45gFUiduHbSX2UyokpejUGJ5oxLbSykjs+8IKDE0wQqdgTxEios5OrtQjNmUsbIXfRWLOsh3QJ5wFlZarsjaFQhULY/Jo2C0bIjFHyTbo48+EUpMgYWbUXJy+Ry12xhlnirbnzn0In3++UlR4qqpyQq2uXl3I5XKf8HoTJ04Wbffu3RcAUFFR3tDhAIBNmzYiJiYGs2ZdJorPnn0lYmJisGnThgbPS0pKgstVhZdemo/jx48BAHr16o3Fi7/EhAmTTphnQ9cCgDfeWIA9e3YhGAxCqVTiv//9CHfccbfo2GAwgEceeRB79+7BM888j35tPKKNiIhaZqBZjzPjDQAAmQBMSo/DyGTTKV/vvPR4qGuNsnIHQ/i+sKKlaRIRUZQwJI6AQlX7wUYY1vw1CIc5nbupDMmjIFcaa0VCsBaslSwfOjUcKdVCydfegJKPF8HXhJ5H7UmVmoakq69tl3uZzXGibUEQUFlpx0cfvY/Dhw+jsDAfFktR5B/YcK350w0xmcRPnVWq6qW3g8Fgo+cUFRUiLS293hQ9pVKJtLR0WBqZXnnJJZdh27at+OKLz/DFF58hNTUdo0ePwQUXzESfPn1PmGddp58+BLNnX4klSz7Bjh3bYDTGYsSIkZgyZTpGjRojOnbv3l8h++OXkb1792Do0LOadS8iImp/M7olwhkIYlJ6PDJiNC26lkGpwPg0M9bk1zxw2Vpiw9mJsUjUqlqaKhERdXCCTAFTxhSUHf00EvM6j8NtPwCdqZ+EmUUPmUwJU/pklB//IhLzVB6C234I2tg+EmZGzcGiVAvp+vVH1lNPS52GpGR1+mmsW7cWTz75KOLjE3DWWcMxcuQo9OvXH9u2bcWiRe+f9Hqnsiz2iZ4ohEJhKJXKBvfFxOixYMHb+O23vfjxx/XYunUzliz5FF9++TkeffTJBvtAncjf/jYXl156OdavX4etWzdj/fp1+O67NZg5cxYeeOCfkeOUSiWeeuo5fPLJR/jgg/dw3nlTmjVVkIiI2p9GIccNfdNb7Xqjkk3YVmKH1fdHP4wwsDq/DNf2SWu1exARUcelNfaFxtADHsexSMxWsBZaY28IMv6q3hQ60wA4Y7bBW5UXiVkLvoXG2BOCwCnx0YDT96jVvfnmAmRkZOLjj5fgkUeewJVXXoOhQ8+CzWZrs3umpKShsLAAgUBAFPf7/SgqKkRSUsNzs3Nzc7B//z4MGnQ67rzzr/jgg8VYtOgzGAwGfPLJR83KoaKiHDt2bEd6egauvvp6vPrqW1i+fDUGDz4DK1YshdPpjBw7cODpGDNmLO6//0EEAgE8//yzzf9DExFRVFPKZJiaKe57sd9WhSOVXIGJiKgrEAQBpvSpAGoeygd8VjhKf5YuqSgjCALMGeKBBAFvORyl2yXKiJqLRSlqdZWVNiQnp0Kr1UZixcUWbNz4PYATT8M7VaNHn4uqqip8+eVnovjSpZ/D5arCqFHnNnjeSy/9Bw89dD9crppfALp3z4Jeb4Bc3ry3x9dfr8Tf/nYnsrN/j8RiY03IyMiEIAj1RpQBQM+evXHppZdj27atWLt2dbPuR0REHcup9AE53axHN714KuDXuaUIsacIEVGXoNImQZ8wTBSzW35E0O9s5AyqS6VLRUz8UFHMbtmAoL9KooyoOTgmkFrdyJGjsG7dWsyf/zT69x+AgoICrFy5DG63BwDgcrX+Pw4XXvgXrF79FV599UUcOXIY/foNQHb271i1aiUGDjwdF174lwbPu+KKq/H3v9+DOXNuwfTpM6BSqbBx4wYUFOTjxhv/3awcpk+fgU8//RgPPngfLr74UiQkJOLAgf1YvfprTJ8+AzqdrsHzbr75dnz33bd49dUXMXLkaJjNja9MSEREHU8wFMYPRRUodntxVa/UZk1DFwQBF2Qm4o39NdMOLG4fdpZVYlgiPw+IiLqC2NRxcFn3IhSs/n0pHPLBVvg94rtfJHFm0cOUOgEu6+8Ih7wAgHDQC3vResR1u0DizOhkOFKKWt3cuQ9jxoyZ2LRpA158cT7Wr1+HadMuwMsvvw4A2LHjl1a/p0qlwssvv4HLL78a27f/jFdeeR67du3AtdfeiFdeeaNeA/Q/jRgxEs8++wI0Gi3ef/9dvPrqi3A47HjiiXmYNq15/4AlJCTglVfexOmnD8ayZV/ghReewy+/bMdNN92Gv//94UbP0+licPfd96Kiohxvvvlqs+5JRETSKnH78Ob+PHxfWIF91irsKKts9jUy9RoMiTOIYt/ml8MbPPHCIERE1DnIFTrEpo4XxaoqdsPr6liLaXVkcqUesSljRTFn+U74XBaJMqKmEsJcczKivNyJUKjxl8NiyUFKSvd2zIi6IoVChkCgfX4R4d9pooYlJhpQWuqQOg2KAu9m5+Oowx3ZVskE3DOwO+I0DS+w0Rib148X9uYgUOtr2fhUM6ZkJJzgrM6L70Ei6fF92L7C4SAs2W/D7ymNxPQJwxGXOV3CrKJLOBREUfYbCHgrIjG1vjuSel93SotpSa2zvAdlMgHx8frG97djLkRERESdysVZSVDKar7o+kJhfH7M0uyeUCa1EuemmEWxTRYbbF5/q+RJREQdmyDIYUqfAgCQK42I7z6rXgNvOjFBJof5j9fwT15nDtz2bIkyoqZgTymiE3A6nfB6PSc9TiaTw2w2n/Q4IiLqXOI1KpyfmYDlOTVPtnOcHmyyWDE2Na5Z1xqbasYvZXY4/NULggTCYazJL8flvVJaNWciIuqYtMZeiO/+F2hj+0EmV0mdTlTSGPtAY+gFj+NIJGYtWAuNsTdksuaNYqb2waIU0Qm8/PJ/8M03X530uJSUVCxZsrIdMiIioo5mRGIs9tuqcNBes5Lr2oIK9ImNQapO3eTrqOUyTE6Px5fHSyKxPRUOjEo2IbPOCn1ERNQ5xcQNljqFqCYIAswZU1C0/00A1aOWgz4bHCVbEZvS8IrsJC0WpYhO4KqrrsOUKSefx61WN/2XDiIi6lwEQcCsrGS8/FsO3H80Jw+Gw/j8qAV3DciEQtb0bglnJhixpcSOIpc3Evs6txS398+Iyn4YRERE7U2pSYQhcQQcpT9HYpXFmxATNwQKlVHCzKghLEoRnUCPHj3Ro0dPqdMgIqIOzqhSYGZWEj45UrPKj8Xtw3cFFZiW2fRm5TJBwPmZCXjvQEEkllvlwd4KJwbHG05wJhEREf0pNmUsqip+RShYvRhJOOSHrXAdErIuljgzqouNzomIiIhaweA4A4bEiQtHP1qsOF5rdb6m6GXUob8pRhRbnV8Gf6h9VmYlIqKOJRhwoyJ/NbzOXKlTiRoyhRaxaRNEMZd1L7xVeRJlRI1hUYqIiIiolVzUPRFGZc1A9DCAz49Z4A02r6A0PTMBtRb1g80XwOZiW+skSUREUSEcDsFRuh1Fvy+As3QbrPlrEG7m6q5dmT7+TCg1yaIYX8OOh0UpIiIiolaiVchxaY86X4C9AXydW9rIGQ1L0KgwMskkiq0vtMLhD7Q0RSIiihK+qgJY87+JTEHzuYtQVbFH4qyihyDIYM6YIor5XIWoqvhVooyoISxKEREREbWi3rE6nJMUK4r9UlaJ/TZns64zMS0OWnnNVzVvKITvCspbJUciIur41PpMaE39RTFb4fcIBb2NnEF1aQw9GngN1/E17EBYlCIiIiJqZVMzEpCgUUa2DUo5lELzvnbpFHKclx4viv1SWgmLi1+kiYi6CnPaZECQR7ZDAScqizdJmFH0MadN4mvYgbEoRURERNTKVHIZLuuRAhmA0816/G1Qd/SO1TX7OmcnxoqKW2EAq/JK2Q+DiKiLUKhNMCadI4pVlmyF31shUUbRR6E2N/gaBrxWiTKi2liUIiIiImoDGXoN/jqoG67snQqdQn7yExoglwmYnpkgih2udOOA3dUaKRIRURQwJo+BXKGvCYSDsBV8J11CUaih19BasFa6hCiCRSkSmTfvCYwZM+yk/82b90SL77Vq1UqMGTMMO3f+0vLEG7Fz5y8YM2YYVq1a2SrHdRR3330bLr30QqnTICKik0jWqlt8jX6xMehp0Ipi3+SVIhjiaCkioq5AJlfBlD5JFHPbs+FxHJMoo+jT+Gt4VKKM6E+Kkx9CXcnMmbMwbNiIyPaePbuwYsVSXHTRxRgyZGgknp6e0eJ7DRkyFP/615PIyurR4ms1JiurB/71rycxaNDgNrsHERFRWxIEARd0S8SCfbn4swxV6vFjW6kd5ySbpEyNiIjaic58Ohyl2+BzFUZi1vw1SOl3G4Rm9izsqqpfw+3wuQoiMWv+t3wNJcaiFIkMGjRYVMAJBoNYsWIpBg0ajKlTz2/Ve6WnZ7RKcetE4uLiWz1vIiKilqr0BZBf5cEAs/7kBwNI1alxVoIRv5RVRmLrCstxRrwB2lOcGkhERNFDEASYM6ah+OB/IzG/pwTO8p0wJAyTMLPoUf0aTq3/GpbtgCFxuISZdW0sBxIRERG1o70VDrz8Ww4+OWJBsbvpK+lNzoiHSiZEtl2BEH4oZKNbIqKuQh2TAZ35dFHMXvgDQgG3RBlFH3VMBmLixLNo7EXrEeRrKBkWpeiUvffeW5g4cRQ2bPgBF100FZMnj8VXXy0DABw4kI1HHnkAF144BePGnY0ZMybjiSceQUlJceT8uj2l/tw+dOggnnjiEUybNgGTJ5+Lhx+ei6KiwoZSqHet9evXYfbsi3DeeaPx3ntvNdgryu124+WXn8fMmdMwadIYPPzw31FWVlbvmoFAAO+88wZmzboA5503GnfffRsOHTqIcePOxnvvvVXv/jfeeBUmThyFGTMmYd68Jxq8ZlNYLEV45JEHMHPmVEycOArXXDMbH3/8AUKhUKPnuFwu3Hrr9Zg8+Vzs2bP7lO5LRERtb0VOCRYfscAdDCEQDuPzo8UINLE3lEGpwLjUOFFsS4kNZR5fW6RKREQdkCntPAiymlVZQ0E37JaNEmYUfWIbfA03SJhR18bpey3kcRxDRd43CHhPrQDRVhTqBMRlTofG0Hb9moDqws38+U/jiiuuhs/nw+DBZ+DIkcO4666bkZHRDddeewPUag327t2DNWtWoaAgD++88+EJr/nQQ/cjK6snbr99DgoK8vH554tRVlZ60vMA4Nlnn8Ill1yOmJgYDBw4GMFgQLQ/HA7jH/+4D7t378RFF12MHj164ocf1mH+/KfrXevf/34UP/zwHaZPn4F+/QZg8+ZNuOeeO+oVh/7737fx3/++jfHjz8NFF81CSUkxvvzyM+zatQPvvrsIJpPp5C/kHwKBAO699254PB5cfvnV0OsN2LLlJ7zxxqsIBoO47rqb6p3j9/vxz3/+HUeOHMb8+S9hyJAzmnw/IiJqX931WmwtsUe2C11efF9YjikZCSc4q8aYFBO2ldph91V/vgXDwOq8MlzTJ61N8iUioo5FoTLCmDwa9qL1kZijdDv0CWdBqWnaZ0lXp1AaYEw+F/ai7yMxZ+l26OPPhEqbJGFmXROLUi1Ukfc1At6ON3Q+4C1DRd7XSBtwd5veJxQK4YorrsY119wQif3nP89AEAS8+uqbMBpjAVQ3UPf7/Vi37ltUVtoj8Yb069cf8+bNj2x7PG4sW/YF8vJykZnZ7YT5TJo0Fbfeemdku+7Kfps3b8LOnb/gnnvux2WXXfVHbpdg7tx7sGPHtshxe/bswg8/fIfrrrsJt912FwBg1qzZeOSRB7Fx4w+R4woK8rFw4bu45pobcMcdNa/15MlTcdNN1+DDD9/DPffMPWHOtR08mI3jx4/hqaeexYQJ1atDXHjhXzB37j3Izc2pd3woFMK///0I9uzZhaefno+zzuJcaCKijmxwnB6/W/XYa3VGYhuKrOhnikE3vfYEZ1ZTymSYmhGPz47WjDz+3VaFo5Uu9DTq2iRnIiLqWAxJ58BZvgtBnx2CTAlj8hgoVCap04oqxqSRcJbvRNBn+yMShq3gWyT2uhqCIJzoVGplnL5HLXbGGWeKtufOfQiff75SVHiqqnJCra5eFtvlOvF83YkTJ4u2e/fuCwCoqCg/aS61VwhsyNatmyGTyTBjxl8iMYVCgVmzZouO+7PwdMUVV0digiDg6quvr3PceoRCIYwZMxY2my3yX1xcAvr0OQ2bN286ac61JSQkQhAELFr0Pn7+eQv8fj8EQcALL7yKRx/9d73j589/BuvXf48HH3wE55wzpln3IiKi9icIAmZmJcGgrGlOHgbw+dFi+IKNT9OubUicAZkxGlFsVV4ZQuGmTQMkIqLoJpMpYU6bjJi4wUgdcDdiU86FION4k+YQZAqY08W/d3ocR+GuPChRRl0X/+a2UFzmBajI/wYBTwebvqdJQFzG9Ha5l9ks7m8hCAIqK+346KP3cfjwYRQW5sNiKUL4jy/L4fCJv3SbTGbRtkqlAlC9EmBzc6nLYimE2RwHnU78NLl79yzRdl5eHozG2HojuuoeV1CQDwC444760+oAQKlUNhhvTFJSMu6++294/fVXMXfuX6HV6jBs2HBMnDgZEydOhlxe80uMxVIU6eH16697MH36jGbdi4iIpKFTyHFJj2QsPFjTL7Hc68c3eWWYmXXyaQOCIOCCbgl4c39+JFbo8mJXuQNnJRjbJGciIupYdOYB0JkHSJ1GVNPG9oNanwWv83gkZitYC62hF4t87YivdAtpDD2Q1v8uqdOQlEwmHnC3bt1aPPnko4iPT8BZZw3HyJGj0K9ff2zbthWLFr1/0uu1ZLhk3VwaurbXW3+lo7p9ogKBQIMFpT8LZDXnVRfKnn32hchIsJa6+urrcN55U7Fhw/fYsuUnbNu2FT/+uAGrV6/C88+/EjlOEAT8/e8P49dfd+Orr5Zh+vQLMHjwGa2SAxERta2+sTE4OzEWP5fW9Jf6udSO/uYY9I2NOen53fRanB6nx96KmmmA3+aX4XSzHio5B8ITERGdjCAIMGdMhSX7bVSPWwYC3go4SrfBmDxK2uS6EH5roVb35psLkJGRiY8/XoJHHnkCV155DYYOPQs2m03q1JCWlg6n01Evl8LCgnrHWa0VqKpyiuL5+bmi7dTU6sayycnJGD78bNF/gYC/2YWqyko7duzYDqMxFpdccjn+859X8NVX32H8+PPw88+bceTI4cixyckpmDlzFubM+RtiYmIwf/7TCAQCJ7g6ERF1JNMzExCvFj8A+eJYMVyBk48MBoBpGQlQ1HqQ4/AHsdFibdUciYiIOjOVNhn6hLNEMbtlI4J+ZyNnUGtjUYpaXWWlDcnJqdBqaxq2FhdbsHFj9eoGTZmG11bGjp0AAFi8eFEkFg6HsXTpEtFx48aNRygUqhf/8svPRdujR58LAFi0aGFkeiIAHDp0AA89NBeffba4Wflt27YVc+bcjp9+qlnWVavVomfPXgAaHgkWFxePm2++A8eOHRX9uYiIqGNTyWWY3TMZtccHO/xBLD9eIvpMaYxZrcToZJMo9qPFCrvP37qJEhFR1AiHgvA4jkqdRlSJTR0PQV7TqzEc8sFW+P0JzqDWxOl71OpGjhyFdevWYv78p9G//wAUFBRg5cplcLs9AACXq0qy3M48cxgmTpyMjz/+AOXlZRgwYBA2bdqIgwf3i44bPnwkRo8+F2++uQC5uTno338gtm//GVu3bgZQM8WwZ8/euPTSK7BkySew2+0YO3YcKisr8cUXn0Kr1YlWAmyK0aPHonv3LDz77FM4cCAbGRkZyMnJwRdffIazzhqBHj16NnjerFmz8fXXK7Bw4bs477wpSEtLP4VXh4iI2ls3vRbjUs1YX1Qzwmmv1Yn+FQ6cEX/y/lDj0sz4pawSVX+MrvKHwvg2vxyze6a0Wc5ERNQxue2HYC34FgFvBVL63QaVNlnqlKKCXKGDKWUcrAVrIrGqit3QJw6DWpcmYWZdA0dKUaubO/dhzJgxE5s2bcCLL87H+vXrMG3aBXj55dcBADt2/CJpfo899hRuuOEW7Nz5CxYseAnhcAiPPz6v3nH//vczuOyyK7F162a8+uoLcDodePLJpwEASmVNb6m//W0u5s59CDabFa+99jK+/PJzDB48FK+//m69xugno9Vq8fLLr2Hs2An49ttv8Pzzz+H779fi4osvxdNP/1+j58nlcsyd+w/4fD48//xzzbonERFJa2JaPNJ04uneK3JKmzTiSSOXY3J6vCi2q9yBfKenVXMkIqKOrSJvFUqPLkbAWw4gDGv+miaNuqVq+sRhUGgSRDFr/mq+hu1ACPNVjigvdyIUavzlsFhykJLSvR0zIqk4nU4olcp6PaGys/fjlluuxUMP/QszZsxsk3srFDIEAk1bFryl+HeaqGGJiQaUljqkToO6kGK3F6/ty0Og1teysSlmTMtMOMFZ1ULhMBbsy4XF7YvEuus1uK1fRosWD5ES34NE0uP7MLq47AdQdvRTUSyhx2XQmfpJlFH0cVceRumR/4li8d1nISZukCT5dJb3oEwmID5e3/j+dsyFKGps2PA9Jk8+F3v37hHF1637FgAwYMBAKdIiIqJOKlmrxpSM6hFPCkHA+ZkJke2TkQkCptcpXuU4PfjNyiatRERdhdbYFxqDuNWHNX8N/J4yiTKKPlpjb2iMfUQxW+F3CAV9jZxBrYE9pYgaMGrUuYiJ0ePxx/+Jiy+ejdjYWOzbtxerVq3E1KnT0bNn7yZfy+Vywe12NenY+PiTPxEnIqLOaVSyCRVeP85OikWytnmrt/aJjcFpsTocsNd83qzOL0N/UwwUDSySQUREnYsgCDCnT0VR9psAqkfdBv12WLLfhiljCvTxZ0Xt6Nn2ZE6fgiLHESBcPXMl6K9EZclmmFLHS5tYJ8aiFFEDzGYz3njjPfz3v29jyZJP4HA4kJqaittvn4Mrr7y2WddavHgR3n//nSYdu2mTtP22iIhIOjJBwEXdk075/OmZiThkz8GfE8Ct3gA2F9sxNtXcOgkSEVGHptQmQp84HM7SbZFYOByANW8VPPbDiOt2IeTKGAkz7PiUmngYEs+Go2RLJOYo3gx9/FAoVLESZtZ5sShF1IisrB548slnWnydadMuwODBZ7Q8ISIiohNI0qowIikWW0vskdgPRRU4M8EAvZJf+YiIugJz2iSE/E64bL+L4u7KgyjKfhPx3S6CNrZPI2cTAMSmnIuqil8RClSvGh8OB2Ar+A4JPS6ROLPOid9QiNpYenoG0tMzpE6DiIiinD8UgvIkU/Empcdjd7kDnmD1eClvMIR1BRWYmXXqI7CIiCh6CDIF4rMugaaid/XqcaGafkihQBVKjy6GPmEYTOmTIZMpJcy045LJNTClTURF7spIzGXbB49zGDR6LhLV2thkgIiIiKgD84dCWJVbigX78uALnnh1Vp1CjolpcaLYtlI7it3etkyRiIg6EEEQoI8/Ayn9boMqpv7DcWfZL7AceAc+d4kE2UWHmLghUGpTRTFr/hqEw+2zSnpXwqIUERERUQdV6PLi9d/zsKnYhlKPD2vyT76K0sgkE+LVNU+/wwC+yePqS0REXY1SHYfkPjcgNmUcAHGT86DPAZlMJU1iUUAQZDBnTBXF/G4Lqsp3S5NQJ8aiFBEREVEHta3EhmJ3zdSLLSV2HLJXnfAchUzAtEzxaq4H7S4cPMl5RETU+QiCDLGp45Dc90YoVDULX8RlTodCbZIusSig0XeDzjRQFLMVfY9Q0CNRRp0Ti1JEREREHdT0zESY1eIWoF8cK4E7EDzheQNMMehh0Ipiq3LLEAyHWz1HIiLq+NQxGUjpdxti4s6AzjQQOvPpUqcUFUzpkyAINZ/DoYALdstGCTPqfFiUIiIiIuqg1HIZZvdIEU26qPQHsDKn9ITnCYKA8zMTROeVeHzYXmpv9BwiIurcZHI14rtfhPisiyEIQoPH+NwlHAlUi0IVC0PyKFHMUbINfg+nxbcWFqWIiIiIOrAsgxbnpphFsd0VDuytcJzwvPQYDYYmGESx7woq4DnJKCsiIurcBKHhMkAo6EHp0cUoyn4LHmduO2fVcRmTR0OuNNaKhGAtWCtZPp0Ni1JEREREHdyk9DikaMUNaZcdL0GlL3DC86akJ0Apq3ka7goE8UORtU1yJCKi6FaR9w2CPjuCPjtKDn0AW+H3CIf5IEMmU8KUPkkU81QegrvysEQZdS4sSpHIvHlPYMyYYSf9b968J1p8r1WrVmLMmGHYufOXDnWtaHL33bfh0ksvlDoNIiJqYwqZDJf1TIG81nQLdzCEL48XI3yCPlFGlQJj64yy2lxsQ4XH32a5EhFR9HHZD8Bl3VsrEkZl8SYUH3wffk+5ZHl1FDrTQKhjMkUxa/63LNq1AsXJD6GuZObMWRg2bERke8+eXVixYikuuuhiDBkyNBJPT89o8b2GDBmKf/3rSWRl9WjxtYiIiDq7FJ0ak9PjsTq/po/FQbsL20rtODvJ1Oh556aYsb20EpX+6lFVwXAYq/PLcFXv1LZOmYiIooTW0AuGxLPhKP1ZFPe5CmE58DbM6VMREz+00V5UnZ0gCDBnTIXlwLuRWMBbBkfpLzAmnS1hZtGPRSkSGTRoMAYNGhzZDgaDWLFiKQYNGoypU89v1Xulp2e0SnGLiIioqxiTYkK2zYnjzpomtKvyytDLqEOCRtXgOSq5DFMz4vH5seJI7DerE8cdbmTVWaGPiIi6JkGmgDljKjTG3ijPWY5QwBnZFw75UZH3FdyVhxDX7ULIFToJM5WOSpeGmPihqCrfFYnZLesRYx4EuTJGwsyiG6fvEREREUUJmSDg0p4pUNXqE+UPhfH50WIETzCNb0i8Aek6tSj2dW4pQic4h4iIuh6tsRdS+98Bbexp9fa57QdQtP/NLt1LyZQ6AYKs5iFQOOiF3bJeuoQ6ARal6JS9995bmDhxFDZs+AEXXTQVkyePxVdfLQMAHDiQjUceeQAXXjgF48adjRkzJuOJJx5BSUnNU9q6faD+3D506CCeeOIRTJs2AZMnn4uHH56LoqLCZueXn5+H//f/HsfFF5+P8eNHYvr0iXjwwftw9OgR0XHr16/DLbdch8mTx2Lq1HG499678Ouvu0XH7Nq1A3Pm3Ipp08Zj8uRzceedN2HTpo317vnVV8twww1XYeLEUZgxYxL+/e9HTyl3ALBYLHjkkQcwc+ZUTJw4CtdcMxsff/wBQqFQo+e4XC7ceuv1mDz5XOzZs7vR44iIKHrFqZWY0S1RFMur8mDjCRqYywQB59c5p8DlxZ7yE6/gR0REXY9coUNCj8sQ1+1CCDKlaF8o4ETpkf+hIn81QqGu159QrtQjNmWsKOYs2wmfyyJRRtGP0/da6EilCytySlDawRqGJmqUuKh7EnoZ23ZoZSAQwPz5T+OKK66Gz+fD4MFn4MiRw7jrrpuRkdEN1157A9RqDfbu3YM1a1ahoCAP77zz4Qmv+dBD9yMrqyduv30OCgry8fnni1FWVnrS82qrqCjH7bffAJ1Oj0suuQyxsSYcOnQAK1cuw8GD2ViyZCUUCgV27dqBxx//J0aOHIUZM2bC43Hjiy8+x733zsGiRZ8iPT0DubnH8eCD96JPn9Nw221zEA6HsXLlMjz88FwsWPAOhgw5AwDw2msvY/HiRTjrrBG46657UFZWhi+++BTbt/+Md975AKmpac16Xf/+97/C4/Hg8suvhl5vwJYtP+GNN15FMBjEddfdVO8cv9+Pf/7z7zhy5DDmz38pkhcREXU+ZyUYsd9Whf22KgCAWaU46VS8HgYtBpn1+M1aMyXj2/xyDDTroZLzOSUREdUQBAH6+KFQx3RDec5S+FziB+3O0m3wOo4hPmsWVNpkibKUhiHxbDjLdyLgrfgjEoa1YA2Sel/XZXtutQSLUi207HgJyr0dqyAFAKUeP5YdL8HcwVltep9QKIQrrrga11xzQyT2n/88A0EQ8Oqrb8JojAVQ3UDd7/dj3bpvUVlpj8Qb0q9ff8ybNz+y7fG4sWzZF8jLy0VmZrcm5bVq1UpUVlbi9dffQ/fuWZG4TheDjz5aiCNHDuO00/ph3bq1UKs1ePbZFyL/gAwfPhKPPvogDh7MRnp6Bn78cQPcbjeefvo/MJlMAIBJk6bgjjtuwqFD2Rgy5AwcO3YUn3zyEcaOnYB58/4vcq1zzx2PO+64Ea+//gqeeurZJuUOAAcPZuP48WN46qlnMWFC9fKjF174F8ydew9yc3PqHR8KhfDvfz+CPXt24emn5+Oss4Y3+V5ERBR9BEHAxVlJyP0tF/1NMTi/WwI0cvlJz5uWkYD9tqrIVD+7P4AfLVaclx7f1ikTEVEUUmrikdz3RtgtG1Fp2QSgZtq331OKqoq9UKV3raKUIJPDnD4FpUc/icS8zhy47dnQmfpLmFl04mMxarEzzjhTtD137kP4/POVosJTVZUTanV1LwuXy33C602cOFm03bt3XwDVo5+a6pprbsCKFWtEBSmv1wOZrPqvvNvtAgAkJSXB5arCSy/Nx/HjxwAAvXr1xuLFX0aKQYmJ1f/Ivvjic8jO3g8AiI01YfHiL3HppVcAADZv/hHhcBjXXHO9qDo+cOAgDB8+Elu2bEIgEGhy/gkJiRAEAYsWvY+ff94Cv98PQRDwwguv4tFH/13v+Pnzn8H69d/jwQcfwTnnjGnyfYiIKHrplQr8bVA3zOqR3KSCFADEaZQYlWwSxTZarKj0Nf0zioiIuhZBkMOUOgHJfa6HXGWKxJXaFJhSx0uWl5Q0xj7QGHqKYtaCtV1ySmNLcaRUC/0lKwkrckpR6vFJnYpIokaFi7onnvzAVmA2x4m2BUFAZaUdH330Pg4fPozCwnxYLEUI//FUNhxuvCcSAJhMZtG2SlXdSC4YDDYrL7/fj7fffh0HDmSjoCAPRUWFkWv82Zfpkksuw7ZtW/HFF5/hiy8+Q2pqOkaPHoMLLpiJPn2qi2ETJ07Cxo0/YN26tVi3bi3i4xNwzjmjMX36DAwZMhQAIn2junXLqpdHVlYWtm3bArvdhvj4hCblnpSUjDvvvAdvvbUAc+f+FVqtDsOGDcfEiZMxceJkyGv98mGxFEV6ef366x5Mnz6jWa8TERFFL72y+V/lJqSasaOsEq5A9WeiPxTGtwVluLRHSmunR0REnYha3w2p/W6HNf8buKy/I6H7xRBkXbOkIAgCzOlTUZT9Jv4cPRb02eAo2YrYlHOlTS7KdM2/Qa2ol1GH+07vLnUakvpz9NGf1q1biyeffBTx8Qk466zhGDlyFPr1649t27Zi0aL3T3q91piHu2fPLtx//93QanUYPvxsnHHGRejbtx8KCvLxwgvPRY6LidFjwYK38dtve/Hjj+uxdetmLFnyKb788nM8+uiTmDJlGhQKBf7f/3sOR44cxoYN32Pr1s1YtWolvvpqOW6//W5ce+0NkYJbQ0Kh6n1KpbLRYxpy1VXXYsqUadiw4Xts2fITtm3bih9/3IDVq1fh+edfiRwnCAL+/veH8euvu/HVV8swffoFGDz4jGbdi4iIug6NQo5J6XFYkVMaie0qc+CcJBPSYzQSZkZERB2dTK5GfPe/IDZlHBRqc4PHhMNhhEN+yOSqBvd3FkptIvSJw+Es3RaJVRZvQkzcEChURgkziy6cvket7s03FyAjIxMff7wEjzzyBK688hoMHXoWbDZbu+Xw3ntvQa3WYNGiz/D44/8P1157I84++xw4neJVhnJzc7B//z4MGnQ67rzzr/jgg8VYtOgzGAwGfPLJRwCqV8Hbs2c3evXqjZtuug1vv70QS5asREZGNyxevAgAkJJS3cQ8J+d4vVxyc3Og1WphMDT9H6bKSjt27vwFRmMsLrnkcvznP6/gq6++w/jx5+HnnzfjyJGaZViTk1Mwc+YszJnzN8TExGD+/KebNVWQiIg6H6vXjx9PsBrf8MRYJGlqLWkNYFVe2QkfshAREf2psYIUADhKtsCS/Ra8zrx2zEgappRxkMlrFhoJh/ywFX4vYUbRh0UpanWVlTYkJ6dCq615cxYXW7BxY/Wbs7nT8E6F3W6H2WyG2Vzzj6XT6cSqVV+Jcnjppf/goYfuh8vlihzXvXsW9HoD5H+sRLRo0X9x7713orS0JHJMUlIyEhMTI6PERo+uHqL58ccfiL7QHziQjV9++RnnnDOmWSPAtm3binvuuQM//bQxEtNqtejZsxeA+qPTACAuLh4333wHjh07GimWERFR1xIOh7GjrBKv/JaLb/LL8FuFs8Hj5IKA87uJp5Qfc7gjq/kRERGdCp/LAlvR9wj4rCg+tBC2ovUnbd8SzWQKLWJTJ4hiLuuv8FblS5RR9OH0PWp1I0eOwrp1azF//tPo338ACgoKsHLlMrjdHgCAy9X2X3hHjhyFjz/+AP/610MYMWIkysvL8NVXy1FRUfFHDtVFqCuuuBp///s9mDPnFkyfPgMqlQobN25AQUE+bryxuqH4rFmXYfXqrzFnzq2YOXMWDAYjduzYjl27duCWW+4AAPTs2QuXXnoFliz5BPfeOwdjx45DWVkZvviietTVHXfc3az8R48ei27duuPZZ5/CgQPZyMjIQE5ODr744jOcddYI9OjRs8HzZs2aja+/XoGFC9/FeedNQVpa+qm+hEREFIVW55fhR4stsr0spwTdDRoYGug91Tc2Bn1jdThor3kw801eGfrGxkAh45LWRETUPKGQH2U5XwKRIlQYlZaN8FQeQXzWxVCq4054frTSJ5wJZ9kv8HtqBjFY81cjue/NrdKaprPjSClqdXPnPowZM2Zi06YNePHF+Vi/fh2mTbsAL7/8OgBgx45f2jyHm266DVdeeS327duLF1+cj1WrVmL48LOxcOHHkMlk2LlzOwBgxIiRePbZF6DRaPH+++/i1VdfhMNhxxNPzMO0aRcAqF6N76WXXkdGRiYWL/7oj5X6juK++x7A9dffHLnn3/42F/ff/w9YreVYsOAlfP31CowdOx7vvfdRs4tDWq0WL7zwGsaOnYBvv/0Gzz//HL7/fi0uvvhSPP30/zV6nlwux9y5/4DP58Pzzz/X6HFERNQ5DYkzQF7r+68rEMTSYyWNTsubnpkg+jJY7vVja4mtTXMkIqLOSYBQb0U6APC5CmDJfgvO8l2dcpq4IMhgzpgqivlchaiq+FWijKKLEO6MfytOUXm5M9KUuiEWSw5SUrp2U3NqewqFDIFA+wxx5d9pooYlJhpQWuo4+YFEHdD6wgp8W1Auil2clYThibENHr/8eAl+LrVHtjVyGf4+OAs6hbzB49sD34NE0uP7kE6V234I5bkrEArUnyGjje2HuG4zIFfoJMisbZUe/Qxue3ZkW67QI3XAHMjk6lO6Xmd5D8pkAuLj9Y3vb8dciIiIiKiNnZtqRrc6q+h9nVuKCo+/wePPS4+DWl7zldATDOG7OkUtIiKiptLG9kFqvzugje1bb5/bng3L/jfhrjwiQWZty5w+GRBqHugEA05UFm+SMKPowKIUUTtxuVwoLy9r0n9ERESnSi4ImN0zGapafaF8oTA+P2ZBqIEB8nqlAhNSxX0+tpXYUeL2tXmuRETUOcmVMUjocTniMi+AIFOK9gUDTpQe+RjW/DUIhzrPquEKtRnGpJGiWGXJVgS8ja+GS2x0TtRuFi9ehPfff6dJx27a1PZ9t4iIqPOK16gwPTMRy3Nqmq7mOD3YZLFibGr9RrOjkmPxc6kNVm/1LwchAN/kleL6vlwwg4iITo0gCNAnnAW1PgvlOUvhcxWK9jtKf4bHcQzxWRdDpU2WKMvWZUweg6ryPQgG/lj9NhyEtWAtEnteJm1iHRiLUkTtZNq0CzB48BknPU4u5wBGIiJquRGJRmTbnDhQa3W9tQUV6BMbg1SduL+FQibDtIwELD5iicQO2F04ZK9Cn9iYdsuZiIg6H6UmHsl9b4S9aAMqi38CUDNq1+8pQemR/yFtwF8hyKK/PCGTqxGbdh4qcpdHYm57NjyOY9AYekiYWccV/T91oiiRnp6B9PSMkx7Xno3OiYio8xIEARdnJeOVfTlw/fG5EgyH8flRC+4akAmFTPwQZJBZj+56DXKcnkjsm7wy9DLqIOOS1kRE1AKCIIcpbSI0xl4oz1mGoK9mgQ1zxvROUZD6U0zcYDjLtotGhlnz1yCl320QBA5AqIuvCBEREVEnZVQpMLN7kihmcfvwXUFFvWMFQcAFmYn1jt1RVtmmORIRUdeh0XdHar/boTOfDgCIiR8KnamfxFm1LkEQYM6YJor5PSVwlu2UKKOOjUUpIiIiok7s9DgDzog3iGI/Wqw45nDXOzZDr8HQOsd+m18OTzDYpjkSEVHXIZNrkJB1MRJ6zIY5farU6bQJdUwGdObBopi96AcEA/U/e7s6FqWIiIiIOrkLuyUiVlkzNSIMYE+5o8Fjp2TEQ1lr5b6qQBAbirhyEBERtS6dqT9kclWD+4IBF4oPLYS3Kr+ds2o9prSJopUHQ0E37JYNEmbUMbEoRURERNTJaRVyXNKzemUjlUzAX7onYWb3xAaPjVUpcW6KWRT7yWKD1etv8zyJiIjC4TAqcr+C15mL4oPvw160AeFw9PXcVaiMMCaPEcWcpdvhd5dKlFHHxKIUERERURfQ26jDhd0S8deB3TAiKRbCCZqXn5tihkEpj2wHwmGsyS9rjzSJiKiLq6rYDbc9+4+tMOyWDSg+tBABb/SN2jUmnQO5ylQrEoa1YA3C4XBjp3Q5LEoRERERdRHnJJsQr2l4qkRtarkMUzISRLFfK5zIdbIXBhERtS2/p/5DEF9VPoqy34KzfE9UFXQEmQLm9MmimMdxFO7KgxJl1PGwKEVERERE9QyNNyBNpxbFvs4tQyiKfhkgIqLoY06fjMSeV0CmiBHFwyEfKnKXo+z4kqhqGK6N7Qe1vrsoZitYi3AoIFFGHQuLUkREREQEh1/85VgmCDg/UzxaKq/Kg70VzvZMi4iIuiBtbF+k9rsDGmOfevvctv2wZL8Jj+OYBJk1nyAIf6wyWDNtPuCtgKN0m3RJdSAsSpHIvHlPYMyYYSf9b968J1r1vi5XFazW1pkj/OefQSpFRYUYM2YY3nvvLclyICIiaipvMIRlx4vxwt6ces3Mexp1GGASP6lenV8Gfyj6Gs4SEVF0kStjkNjzCpgzzocgKET7gn4HSg4vgrXg26gYcaTSpUCfcKYoZrdsRNDPBz2Kkx9CXcnMmbMwbNiIyPaePbuwYsVSXHTRxRgyZGgknp6e0Wr3zM7ej4ceuh+PPfYUzGbpiklERERdTa7Tjc+OFqPij2LUkmPFuPm0dMhqNUGfnpmAA/YqBP+YtWf3BfCTxYbxaXFSpExERF2IIAgwJA6DxpCFsuNL4XcXifY7SrbC4ziG+O4XQ6VNkijLpolNGY8q6z6Egx4A1dMRbUU/IL7bhRJnJi0WpUhk0KDBGDRocGQ7GAxixYqlGDRoMKZOPb9N7nn06GGUlXFZTCIiovZWUOWNFKQA4JjDjZ+KbTg3xRyJxWtUOCfJhE3FtkhsfVEFzko0wqDkV0kiImp7Sk0CUvreBLtlPSqLfxLt87uLUXzgXST1uQ7qmNYbPNHa5MoYxKaMg61gTSRWVb4LhoSzoNKlSZiZtDh9j4iIiKiLOjspFr2NOlHs2/xyWFxeUWxCWhx0ipqvjb5QGGvzy9slRyIiIgAQZHKY0s5DUu/rIFcaRfuU2uSoKOwYEodBoRb3a7Tmr4mqFQVbG4tSdMp+++1X3HvvXZg8eSwmTx6L++6bg99//010TGVlJebNewKzZl2ACRPOwWWXzcSbby6A11v9Zfe9997C00//GwBwzz134NJLTzx00WIpwlNP/QszZkzCxImjcP31V2LFiqUnPCccDuP999/BlVfOwsSJo3DhhVPw1FP/QnGx5ZT+3OvXr8Mtt1yHyZPHYurUcbj33rvw66+7T3jO7t07MXHiaNx5581wu6NnpQgiIurcZIKAS3okQyOv+UoYDIfx+VELAqGaL8hahRznpcWLzt1RVomiOsUrIiKitqYxZCG13x3QmQcBAASZCvFZF0MQOn55QxDkMGdMEcW8VXlw2fZJlJH0OOa6lRy85YZTOk/drTu6P/bvBvflPPk4vLk5p3Tdvu8uPKXzmmr79q144IF70adPX9x66x3w+XxYtWol7r77Nrz44muR/lOPPfYQDh06gNmzr0R8fAJ+++1XfPTRQtjtdvzjH49g3LiJKC8vw4oVS3HttTeif/+Bjd6zsLAAt912A3w+Hy655DLEx8djw4Yf8H//Nw/5+bm4666/NXjehx/+F++//w5mzboMvXv3RmFhIT7//BNkZ+/Hhx9+Crlc3uQ/965dO/D44//EyJGjMGPGTHg8bnzxxee49945WLTo0wZ7bR08mI1//OM+9OzZC//5z8vQarVNvh8REVFbi1UpMLN7Ej49WvOwpsjtw7rCckzNqHmaOyIxFltLbCj1VE/3CwP4OrcUN5+WDqFWDyoiIqK2JlNokJA1C1V/rM6nVEdPn0OtsTc0xj7wVB6KxGwF30EbexpkMqWEmUmDRSlqtlAohPnzn0H//gOxYMHbkaLOJZdcjhtvvAovvTQf77//P1itFfjll224666/4aqrrgUAXHjhXxAOh1FYWAAA6N27DwYNGowVK5Zi+PCzceaZjTc6f+utBaistOOddz7Eaaf1AwDMmnUZHnpoLhYv/gjTps1Az5696p23du1qjBw5Cvfe+/dILCkpGcuWfQGLpahZTdvXrVsLtVqDZ599IfIFfPjwkXj00Qdx8GB2vWvl5eVi7tx7kJKShhdeeBUxMfom34uIiKi9DIk34HebE3sralYB2lhkRb/YGHQ3VD9MkcsETM9MxIeHCiPHHHW4kW2vQn8TP9+IiKj9xcSdfsL9jrJfoNKmQh2T3k4ZNY05fQqKHEeAcPVqtkF/JSqLf4Ipdby0iUmg449vow7n4MEDKCwswLnnjofD4YDNZoPNZoPX68Xo0efi0KGDKC0tQUyMHlqtDkuXLsH69esi09b++c/H8fLLrzfrnsFgEJs3/4QRI0ZGClIAIJPJcN11NyEcDuOnnzY2eG5iYhJ27vwFn322GBUV1f0v/vKXS7Bw4f+avYpgUlISXK4qvPTSfBw/fgwA0KtXbyxe/CUmTJgkOrasrBT33TcHAPDSS6/BaIxt1r2IiIja08zuSTAqa0YPhwF8fqwY3mAoEjstVofeRvGI32/yykRT/YiIiDoCb1U+rHnfoPjg+7BbfkQ4HDr5Se1EqYmHIXGEKOYo3oyAzy5RRtJhUYqaraAgHwDw+usvY8aMSaL/Pv30fwCA4mILVCoVHnjgn7Bay/Hoo//ABRech/vvvxvLl38Z6SnVVHa7DW63C926da+3LyurB4DqflMNmTPnXsTGmvDKK89j5sxpuOWW67Bw4bsoLy9rVg4AcMkll+GMM87EF198hmuumY3Zs2fipZfm49Chg/WOXblyGUpKimGzWZGXl9vsexEREbUnnUKOWT2SRbEKrx+r8mpWyBUEAednJqL2ZL0yjx/bSrvel2giIuq4QkEfyo8vRfUjlhDsRT+g5NAHCHitUqcWEZsyFjJFzWIj4XAAtoLvJMxIGpy+10raoodTY72mpBYKBQEAt9xyBwYObHi4ZLduWQCAKVOmYeTIc7Bx43ps2bIJv/yyDdu2bcXSpUvw9tsLoVKpmnTPE61GEApVV7yVyobn3/bu3QeffLIUP/+8GT/99CN+/nkL3n33TXzyyUd4662F6N49q0k5AEBMjB4LFryN337bix9/XI+tWzdjyZJP8eWXn+PRR5/ElCnTIscmJSXjqaeewwMP/A3z5z+N99//HxQKvuWIiKjj6hsbg7OTYvFzSU2RaXtpJQaY9DjNFAMASNGpMSzRiO2llZFj1hWU44x4A3SKpvdpJCIiaitu+wEEfOIClLcqD0XZbyEu83zozKdL3g9RJtfAlDoRFXlfRWIu2z54nMOg0dcfjNFZcaQUNVtqavVSmzqdDsOHny36T6/XIxQKQa1Ww+VyYc+e3QAEzJgxE/PmzcdXX32H2bOvxOHDB7Ft29Ym39NkMkOr1SInp37j99w/msEnJSXX2xcMBnHgQDaKiy0YM2Yc/vGPR/Hll1/j3/9+Bk6n86Qr9zV0r/3792HQoNNx551/xQcfLMaiRZ/BYDDgk08+Eh17wQUXYeDAQbjttjtx7NhRLF68qFn3IiIiksL0jATEq8UPer48XowqfzCyPSk9HipZzZd5dzCEHwor2i1HIiKiE4mJOx0JPS8XjUQCgHDIh/KcZSg//iVCAelXRY+JPwNKbYooZs1f06GmGrY1FqWo2fr1G4D4+AR8/vmncLlckXhVlROPPfYwnn7635DL5Th69AjmzLkFX321PHKMUqlE376nAQDkfyw/LZNV//9Eo6HkcjnOPnsUtm/figMHsiPxcDiMjz/+AIIg4JxzxtQ7LxQK4Z57bscrrzwvig8cOEiUQ1O99NJ/8NBD94v+3N27Z0GvNzR6rYsumoV+/QZg4cJ3I1MfiYiIOiqVXIbZPZNFU/Qc/iBW5JZEtg1KBcanilc62lJiQ5nH105ZEhERnZgu9jSk9rsDGmPvevtctn0oyn4LHsfx9k+sFkGQwZwxVRTzuy2oqtgjUUbtj3OJqNkUCgXuvffvePzxf+Kmm67BhRfOhEqlxsqVS2GxFOGxx56CQqHAwIGDMGTIULzzzusoKbGgV68+KCkpxpIln6J79ywMG3Y2gOpRUACwdOkSlJeXi6bA1XbnnX/Fzp2/4K9/vR2XXHIZEhISsHHjeuzYsR2XX341evToWe8cpVKJSy+9Ah988B4efvjvOPvsc+D1erBixVJoNBpccMHMZv3Zr7jiavz97/dgzpxbMH36DKhUKmzcuAEFBfm48caGp1vKZDLcf/+DuOOOm/D888/hhRdebdY9iYiI2ls3vRbjU+PwQ1H16KdEjRLnpphFx4xOMWFbqR02XwAAEApXNz2/tk9au+dLRETUELlSj8SeV8JZ9gtsBWsRDgci+4L+SpQc/hCGpFEwpU6AIJNmCrpG3x0600C4bPsiMVvh9wj2Gi5JPu2NRSk6JRMmTILBYMSHH/4XCxe+B5lMQM+evfDssy9g9OhzAVQ3Q33mmf/gv/99Bz/99CNWrFgKg8GA8eMn4pZb7oj0gBo2bAQmTpyMn37aiB07tmPcuAlQq9X17pmenoG3316Id955/Y9m6R50794DDz30L8yY0Xhx6eabb4fRaMTXX6/Aa6/9DLlcjtNPH4J//eupZvWTAoARI0bi2WdfwKJF7+P999+Fz+dFz5698MQT8zBp0tRGzxswYBBmzJiJFSuW4rvv1pzwWCIioo5gYlocDtqr0E2vxbTMeChl4hHBSpkM0zIS8MlRSyS231aFI5Uu9DLq6l6OiIhIEoIgwJA4HBp9FspylsLvtoj2O0o2w+M4ioSsi6HUJEqSoyl9Etz2A5GiWShQhaKj66COGydJPu1JCJ9ozlQXU17uROgESxpbLDlISek6DcdIGgqFDIFA+8wh5t9pooYlJhpQWuqQOg0iyflDoXrFqNrC4TDe2p+P3CpPJJaqU2POgEzIWtBAlu9BIunxfUidUTgUgK3oBzhKttTbJwgKmDMvgD5+iASZAbai9ai0bKyVjxwp/e6AUhMvST6tRSYTEB+vb3x/O+ZCRERERFHkRAUpoPrp8/ndEkSxIpcXO8sqGzmDiIhIOoJMAXP6ZCT1vhZypVG0LxwOQK6MkSgzwJg8WpRTOByErfA7yfJpL5y+R12e1WpFKBQ86XFqtQZ6feMVXiIioq6om16LwXF6/FrhjMTWFpTj9DgD1M1cUISIiKg9aAw9kNrvdlTkfQ2X7XcAgD5xBLQNNEVvLzKZEqa0SSjP+TIS81QeQTgcgiB03s9TFqWoy7v11utgsRSd9Ljp02fgkUeeaPuEiIiIOrgStw+bLFbM7J4EuUzAtIwE/G6tQuCPrhAOfxAbi6yYnBHdUw6IiKjzkim0iM+6BFprXzjKfoEp7TypU4LOPBDO8p3wOo8DAJTaJACnPh0+GrAoRV3eY489Ba/Xe9LjEhKkaXpHRETUUYTCYWwtsWN1XhkC4TCMKgUmpcfDpFZiTIoJ64uskWN/tFgxPNEIk1opYcZERESNEwQBMXGDoTOfDqGRXohBvxNBfyVUurZfXVYQBCT2vAKVJT9BoxagMJzVaF6dBYtS1OUNHnyG1CkQERFFhY1FVnxbUB7ZXl9YgdNiY5Cp12Bcahx+Ka2EM1A9JT4QDuPb/HJc1itFqnSJiIiapLHCTzgcRnnOcngcxxCbOh7G5FFtPpVOJlfBlDqhyyw20HknJhIRERFRqxqRFAuDUh7ZDgH4/JgFvmAIarms3nS93RUO5Dk9ICIiikbOsu3wOI4ACMFe9D1KDn+IgM8mdVqdCotSRERERNQkOoUcs7KSRbEyjx+r88sAAGclGJGiVYn2f51XivAfvaaIiIiiRdDvgK1AvPqd15mLouy3UFWxV6KsOh8WpYiIiIioyU4zxWBEongZ7a0ldhyyV0EmCDi/m7gHY67Tg71WJ4iIiKKJXGlAfNYsyORaUTwc9KI8ZynKjn+JUICjgVuKRSkiIiIiapbpmYmIq9PA/ItjxXAFguht1KGfKUa0b01eGfyhUHumSERE1GI6Uz+k9r8DGkOvevtc1t9QlP0WPM4cCTLrPFiUIiIiIqJmUctlmN0jWbRIdaU/iBU5JQCA6RkJkNXaafUFsLnY1q45EhERtQa50oDEXlfBnD4VEOSifUG/HSWHPoCtcB3CoaBEGUY3FqWIiIiIqNm6G7QYm2oWxX6tcGJPuQOJWhXOTjSJ9q0vtMLpD7RjhkRERK1DEAQYks5Gymm3QqlJrre/svgnFB/8L/yeMgmyi24sShERERHRKTkvLR6pOrUotiKnBHZfAOelx0Err/mq6Q2F8F1BeXunSERE1GpU2iSknHYzDEkj6+3zuYtgyX4bjrJfuMBHM7AoRSLz5j2BMWOGnfS/efOeaNX7ulxVsFqtp3Tue++9hTFjhqGoqLBVjuso2uJ1JiIiak0KmYDZPZIhF2rm6rmDIXx5rBhauQwT0+JEx28vrYTF5W3vNImIiFqNIFPAnD4FSb2vgVxpEO0LhwNwWfdJlFl0UkidAHUsM2fOwrBhIyLbe/bswooVS3HRRRdjyJChkXh6ekar3TM7ez8eeuh+PPbYUzCbhzX7/HHjJiIjIxMmk/nkBxMREVGrStGpMSUjHt/k1UxZOFTpws+ldpydZMLWEjvKvX4AQBjAqrwy3Ng3DYIgNHJFIiKijk9j6ImUfnfAmvc1XLbfAQCCXIP47n/hZ1wzsChFIoMGDcagQYMj28FgECtWLMWgQYMxder5bXLPo0cPo6ys9JTP7927D3r37tOKGREREVFzjE42IdtWhWMOdyRWUOXFyCQB0zMT8NHhokj8cKULB+0unFZnhT4iIqJoI1doEZ91CTQVfWDN/wZxmRdAoYqVOq2owul7RERERNQiMkHApT2SoZbJoJHLcHnPFFzSo7oRbH9TDHoatKLjV+WVIhhivw0iIop+giBAHz8EaQP+ihjzwEaPC/qd7ZhV9OBIKTplv/32K959903s2/cbAGDQoNNx6613YsCAQZFjKisr8eqrL2DHju2wWiuQmJiEiRMn48Ybb4VarcZ7772F999/BwBwzz13ICUlFUuWrGzwfvPmPYF9+/bi0kuvwNtvvw4AeOKJedi3by/ef/8dfP75CqSmpgEACgry8frrL2PHjl8gl8swffqFUCqV9a5ZVlaKN954BT//vAV+vx9jxozF+PHn4eGH/45XXnkTZ55ZPZ3Q6/Xigw/ew9q1q1FaWoLExGRMnTod119/c4PXPZldu3bg3XffxJEjhxAMBtG7dx9cffUNGDNmbKPnHD9+DHPm3ILYWBMWLHgbcXHxzb4vERFRWzGrlbiydwqStSrEqmo+GwVBwPndEvHavlz8WYYq9fixrdSOc5JNkuRKRETU2uTKxkcAexzHUXLkY5hSx8OQdA4EgeOD/sSiVCvJ3fXkKZ2n1KYitd+tDe4ryn4HfndRg/tOptvQx07pvKbavn0rHnjgXvTp0xe33noHfD4fVq1aibvvvg0vvvhapP/UY489hEOHDmD27CsRH5+A3377FR99tBB2ux3/+McjGDduIsrLy7BixVJce+2N6N+/8coyABQXW/DBB+/hpptuQ1lZKQYOPB379u0VHVNRUY477rgJfr8fl19+FdRqNZYuXQKbzSY6zuWqwpw5t6K8vAyzZ18Jk8mElSuXY8uWzaLjgsEgHnzwPuzduwcXXXQxsrKykJ29Hx9++F8cPHgAzz33QrPmDOfmHseDD96LPn1Ow223zUE4HMbKlcvw8MNzsWDBOzjrrDPrnWOxWHD//XdDp9Pj5ZffYEGKiIg6pL6xDX8hT9OpcWaCETvKKiOxdYXlOCPeAK1C3l7pERERtbtQwI3ynGVAOAhb4Tq4Kw8jvvtfOM3vDyxKUbOFQiHMn/8M+vcfiAUL3oZcXv1l8pJLLseNN16Fl16aj/ff/x+s1gr88ss23HXX33DVVdcCAC688C8Ih8MoLCwAUN0PatCgwVixYimGDz87MjKpMV6vF//85+M477wpjR7zv/8tgs1mxbvvLsJpp/UDAEyfPgPXXns53G5X5LjPPluMgoJ8vPjiaxg+/OxIftdeezkqK+2R49asWYUdO7bh+edfxdlnnxOJ9+8/EPPnP41Nmzbg3HPHN/n1+/HHDXC73Xj66f/AZDIBACZNmoI77rgJhw5l1ytKWa1W3HffXQCAV155A4mJSU2+FxERUUcxOT0eeysc8P0xbc8VCOGHwgqc3y1R4syIiIjaTkX+Nwj6ax7KeJ05KMp+C3GZF5xwul9XwTFj1GwHDx5AYWEBzj13PBwOB2w2G2w2G7xeL0aPPheHDh1EaWkJYmL00Gp1WLp0CdavXwe3u7r56T//+Thefvn1U77/kCH1RxLVtnXrZvTrNyBSkAIAszkOkyZNFR23ceMP6NWrd6QgBQA6XQwuvvhS0XHr138Pk8mM007rH/mz2mw2nHPOaMjlcmzevKlZ+ScmVvfYePHF55CdvR8AEBtrwuLFX+LSS68QHVtVVYW5c/8Ki6UIL730emR6IhERUbSpCgQxNjVOFNtSYkO5xydRRkRERG1PZ+oPmVzcWzEc9KD8+BcoO74MoaBHosw6Bo6UomYrKMgHALz++st4/fWXGzymuNiCxMQkPPDAP/F///f/8Oij/4BKpcIZZ5yJceMmYtq0C6BWq0/p/maz+YT7LZZCjBkzrl68e/cs0XZeXh5GjDj7pMcVFubDZrNixoxJDd6vuNhy4oTrmDhxEjZu/AHr1q3FunVrER+fgHPOGY3p02dEpj3+aePGHyCTyRAKhZCdvR/dunVv1r2IiIik5g4EsTKnFLsrHLi+TypiVQrYfQEAQDAMrM4vw9W9+dCFiIg6J52pP1S6dFTkLofHcUy0z2X9Fd6qHMR3vxgafTeJMpQWi1KtpC16ODXWa0pqoVAQAHDLLXdg4MDTGzymW7csAMCUKdMwcuQ52LhxPbZs2YRfftmGbdu2YunSJXj77YVQqVTNvv+f0wUbIwgCvF5vA3mHRNvBYABKZf37q1TqOscFkZHRDXPn/qPB+xkMxpOlLKJQKPD//t9zOHLkMDZs+B5bt27GqlUr8dVXy3H77Xfjxhtvihyr1+vx3HMv4Zln/o0FC17EyJGjYDQ2735ERERSOeZw47MjFtj91UWoZcdLMSHNjGU5pZFj9lmrcLTShZ5GnVRpEhERtSmFyojEXtfAUfozbIXrgHAwsi/os6Pk0AcwJo9GbOo4CELX6rXI6XvUbH9OIdPpdBg+/GzRf3q9HqFQCGq1Gi6XC3v27AYgYMaMmZg3bz6++uo7zJ59JQ4fPoht27a2SX5paenIz8+tF/+zj1Xt4/Ly6h9X99zU1DRUVtpx1lnDRX/WoUPPQmWlHTqdtt41TsRisWDPnt3o1as3brrpNrz99kIsWbISGRndsHjxItGx5547HkOGnIF7730AFRXlePPNV5t1LyIiIqlV/lGQAgC7P4BjDg8yYsQPgFbllSEUDtc9lYiIqNMQBAHGpJFIOe0WKDV1+wSHUVm8CcUH34ffUy5JflJhUYqarV+/AYiPT8Dnn38Kl6umcXhVlROPPfYwnn7635DL5Th69AjmzLkFX321PHKMUqlE376nAQDk8uq/fjJZ9f/DrfRldOzYCTh27Ci2bq1ZRc/pdGLNmlWi4849dzwOHszGb7/VrN7n8/lE+QLA6NFjUVlpx9KlS0TxZcuW4PHH/4nt27c1K79Fi/6Le++9E6WlJZFYUlIyEhMTI69FXSNHjsLYsROwcuUy7N27p1n3IyIikkoPgxZjUsTT7vdUONC/zip9hS4vdpc72jM1IiIiSai0yUg57RYYEuu3kvG5CmE58DacZTtb7ffjjo7T96jZFAoF7r3373j88X/ippuuwYUXzoRKpcbKlUthsRThsceegkKhwMCBgzBkyFC8887rKCmxoFevPigpKcaSJZ+ie/csDBtW/SY0maq/rC5dugTl5eWYMmVai/K78sprsHbtajzyyAO47LKrYDabsXz5lwDCdY67FmvWrMJ9983B7NlXwGQyY/Xqr5GTkwOgupINVK/It3r1V3jppfk4eDAb/fsPxNGjh7F8+Zfo27cfLrjgomblN2vWZVi9+mvMmXMrZs6cBYPBiB07tmPXrh245ZY7Gj3vnnvmYtu2LZg//2n8978fQ6Hg25eIiDq+yelxOGivQrG7pqH5TyU29IvVIdte83Dr2/wyDDLroZLzmSkREXVugkwBc8ZUaIy9UZGzHMGAM7IvHPKjIu8rJGf0AWCQLsl2wk99OiUTJkzCCy8sQFJSEhYufA/vvvsGYmJi8OyzL2Dy5OqikiAIeOaZ/2DmzEvw00+b8OKL/4cVK77E+PET8corb0KpVAIAhg0bgYkTJ2PLlupjGuoH1Rw6XQxee+0djB9/HpYv/xL//e/bGDJkKG644RbRcUajEa+99g6GDz8bS5Z8ivfeexO9e/fBrbfeCQCRflMqlQovv/wGrrjiGuzYsR0vv/wfbN68CRdffClefHEBNBpNs/Lr1as3XnrpdWRkZGLx4o/w0kvzcfz4Udx33wO4/vqbGz0vJSUF119/M44ePYLFiz9q5qtCREQkDYVMhst6pkAu1MRcgRC8obDoi2ilP4iNFmu750dERCQVrbEXUvrfAW1sP1HcmDwaOkPXWARECHeQMWF5eXl47rnnsG1b9VSo8ePH46GHHkJcXFybekw7kgAAHAhJREFUnNeQ8nInQqHGXw6LJQcpKVz9rLOw2WwwGAz1GqcvXvwRXnvtJXz66TKkp2e0e14KhQyBQOjkB7YC/p0malhiogGlpZxKRNSaNhRVYE2+uE9GH6MOhyprRkspZQLuPz0LvdPNfA8SSYyfhUTtJxwOo6piN6z5q6FQJyCl701ISjZ1ivegTCYgPl7f6P4OMf/HarXi+uuvh8/nwy233IJgMIj33nsPBw4cwOeff97oCm2neh4RACxY8CJ++ulHLFu2Cmp19WinYDCIH374DiaTOdLQnYiIiFru3BQzsm1VyHF6IrHjTje0chncweqHMf5QGN/ml6F3urmxyxAREXU6giBAHz8Uan13IByGIOs6K/B1iKLUwoULYbFYsHLlSvTq1QsAMGTIENx4441YtmwZLrvsslY9jwgApk6djjVrVuGvf70DU6dOByBgw4bv8fvvv+Ef/3i00abjDamstMPv95/0OKVSCaMxtgVZExERRSeZIODSHsl4dV8ufH+MTPeHwjCoFZGiFADsKnfguK0KMY1diIiIqJNSqps/4yvadYjpe5MmTUJGRgYWLlwoik+bNg3Jycn44IMPWvW8xnD6XtezdetmfPTRQhw5chiBQAC9evXGlVdeg3HjJjbrOnfffRt279550uPOOONMLFjw9gmP4fQ9IulxygJR29leasfS4yWimF4hhzMQjGz3MetxQ6+UyKIjRNT++FlIJK3O8h7s8NP37HY78vLyMHXq1Hr7Bg4ciA0bNrTqeUS1jRw5CiNHjmrxde6++z44HJUnPc5gMLb4XkRERNFsWIIR+61VyLZXRWKuWgUpADhkdWJFbikMSsm/qhJ1WTF2J6qqfCc/kIjaRJbfjwyZvNOvSiv5J31xcTEAIDk5ud6+xMREOBwOOBwOGAyGVjmPqC3069df6hSIiIiigiAIuLhHEl7+LQeuP0YGhwBo5DJ4ak3j+7nELlGGREREHUBBOfqbYnBtn87d61jykltVVfVTMq1WW2+fWq0GALhcrnr7TvU8IiIiIpKWQanAX7rXPFhM06kxu0ey9F9MiYiIOpCDtUYVd1aSj5RqSkurhvoJnOp5J3KieY4AUFIig0LBr0vU9trr75lMJkNiIkcTEjWE7w2itjUh0YAcrw+xaiUu7JMChUyGslAI3xwtljo1IiKiDmFAgrHTfyeVvCil0+kAAF6vt96+P2N6ff1i0amedyIna3QeCoXarQE1dV3t2eg8FAp1iuZ5RK2tszSWJOrozk8xQxAEWMurnwSPiTPAJAiwA3C56n/HI6L2o9Op+T4kklBWohHdFYqo/07a4Rudp6VVz48sLS2tt6+kpARGozFSgGqN84iIiIioY6g7ql0QBJweZ2BhmKgD4PuQSFpd5T0oeVHKaDQiIyMD+/btq7fv999/x6BBg1r1PCIiIiLq2PIqXfjmeDH8JxjB3pBkrQrjUuMa3Pd1bimq6qzy1xQqmYC/ZNVfWAcANhfbkF/lafY1AeD8zAToG1hdcL/Vib1W5yld85wkEzL1mnpxi8uLjRbrKV2znykGg+PqTx3xBkNYnlNyStfkzyk6fk49K6swzBjT4D7+nDrOz4nvp877c7rZ3DUG2UhelAKAKVOm4MMPP8SRI0fQq1cvAMDmzZtx7Ngx3Hzzza1+HhERERF1XGEARyvdKPf6m3Veb6MO41Ib3ve71QmrL9DsXLRyWaO/nB1zuLHvFH+RmpQe32C82O3D7vJTezLe3xSDTNT/5czhD5zyNWOVigZ/OQuGw6d8Tf6couPn5BPQaFGKP6eO83Pi+6nz/pwCzXwwE606RFHq1ltvxfLly3HDDTfgpptugtfrxbvvvouBAwdi5syZAIC8vDzs3LkTZ555JjIzM5t8HhERERFFh1A4jI1FVqwrrECwCYvaEBERUXTrEEWpuLg4fPTRR3jmmWfwyiuvQKPRYNKkSXjwwQehUqkAANu3b8fDDz+MZ555JlKUasp51Dzz5j2Bb7756qTHTZ8+A4888kSr3dflqoLX64PZbD5pbps2/dJq940GY8YMa/XXm4iIqCPaZ3Xi24JyqdMgIiKidtIhilIA0LNnT7zzzjuN7p81axZmzZrV7POoeWbOnIVhw0ZEtvfs2YUVK5bioosuxpAhQyPx9PSMVrtndvZ+PPTQ/XjssadgNg9rtesSERFRdDk9zoAEjQpVCgGOyub3LDE00KvkT9MzE+EPNX91W3mdZuy1nZMUiwGmhqc3nUyMQt5gvJ8pBrGqU/uKnhFTfwoLACRr1Zjdo+GpOCeTpG34Qa9KJpzyNflzio6fU0aioXoubQP4c+o4Pye+nzrvz0ktl8F9SneLLh2mKEUdw6BBgzFo0ODIdjAYxIoVSzFo0GBMnXp+m9zz6NHDKCurv4oiERERdT2pOnX1ikOq1l1xaFBc48tRn6qextZvQpuiUyNFp27VaxpVCgxNMLbqNRUyWatfE+DPqSP9nBITGl/5iz+njvNzOhH+nKL756SUy1r9Xh1R1/hTEhERERERERFRh8KRUnTKfvvtV7z77pvYt+83AMCgQafj1lvvxIABgyLHVFZW4tVXX8COHdthtVYgMTEJEydOxo033gq1Wo333nsL779fPf3ynnvuQEpKKpYsWdnkHHbs2I7//W8R9u/fh6oqJ8zmOIwaNQZ33nkPDIbqFRPC4TAWLnwX3377DYqLLYiJ0WPEiLNx221zkJycErnWsmVLsHTpEuTn50Gt1mDIkKG49dY70bNnr8gxHo8HCxe+i+++W4OyslIkJCTivPOm4MYbb4VG0/DQ0RPZtWsH3n33TRw5cgjBYBC9e/fBddfdiHPOObfRc44fP4Y5c25BbKwJCxa8jbi4hlexICIiIiIiIurIWJRqJf/cfuiUzkvTqXH3wG4N7luwLxeFLu8pXffp4X1O6bym2r59Kx544F706dMXt956B3w+H1atWom7774NL774WqT/1GOPPYRDhw5g9uwr/3979x4U5X3vcfwDLFDlUhsFrSbgXXRJUI7iLdboMTajsV6I1jqlBmJqE41RY3RqNZ0MuWjiFQgaUEKUeIMGBYkWTlLjJSpVoyZeohF7koigB29cFFh2zx9O6FDRiOI+u/B+zTCOuz/YLzt8mMePz/N71Lx5C3399VGlpCTr6tWrmjPnLxo4cLCKiv5PGRnpioiIVNeu5rueITd3n2bNmqZHHw3Rc89Nlqurq3Jz9ykjI13FxcWKjl4gSVqzJkkffJCoMWPGqWPHjsrPz1dq6gadPHlCa9ZslJubm7Kzt2nRogV66qnhCg//ra5cuaxNm9Zr2rTJ2rBhs7y9vVVZWakZM17U119/pWHDRigoqJuOH/9aH330oY4ePazY2PdlMt19pL777l+aPXu6OnXqoj/+cYpsNpsyMzdr9uyZiotLVEhI91s+p6CgQDNnTlXTpt5avnwFhRQAAAAAwGlRSqHOrFar3n33bXXtalZcXILc3G5uWBce/ltFRk7QsmXv6oMP1uny5Us6cCBXL774siZMiJAkjRgxSjabTfn55yRJHTt2UnDwY8rISFevXr0VGnr3G51v3LhO/v4ttWxZvNzd3SVJo0c/o8mTI7V//97qdTk529WnTz9Nnz6r+jF//5bavPlvKig4rzZtHlZ29ja1a9de8+a9Xr2mU6fOeu+9GOXlfavHHuuurKwt+uqro5o2babGjZtQ/Xrt2rVXfHyMMjLSNWbM2Luef9euz3X9+nW99dYiNWvWTJI0ZMhQvfDCczp9+uQtpdTly5c1Y8aLkqSYmBXy8/O/69cCAAAAAMDRsKcU6uzUqW+Un39OAwY8oeLiYl25ckVXrlxReXm5+vcfoNOnT+nixQvy8vJWkyZNlZ6eph07PtX16zfvHTB37l+1fHn8fc/xzjtLtXr12upCSpKuXr0iLy8vXb9eVv2Yn5+/Dh06oE2b1uvSpZu3mR41KlzJyeuq7yLo59dS3333v0pKStD58/mSpL59H1dKyiY99lh3SdLu3Tvl5eWlMWPG1Zhj7NjfycvLS7t3f16n+f38bt65YenShTp58oQk6ec/b6ZNm9L1zDPja6wtLS3VK6+8pIKC81q2LF6//GXrOr0WAAAAAACOhjOlUGfnzv0gSYqPX674+OW1riksLJCfn79efXWu3nnnDc2bN0ceHh7q3j1UAwcO1lNPDZen5/3dYcHNzU35+eeUmLhS//pXns6d+0EXL164Zd2UKdM1Z84MxcQsVmzsEnXp0lWPP/4rjRgxSs2bt5AkRUZO0rFjR5WUlKCkpAS1bdu+es2PxdX58/lq3brNLZfoubu7q3XrNiooOF+n+QcPHqKdO/+hTz/N0aef5qh58xbq27e/nn56hIKDu9dYu3PnP+Tq6iqr1aqTJ08oICCwTq8FAAAAAICjoZSqJw9iD6fb7TVlNKu1SpI0adKfZDY/WuuagIC2kqShQ59Snz59tXPnDu3du1sHDuQqN3ef0tPTlJCQLA8Pj3ueY926tYqPX66AgECFhPTQwIGD1a1bsP72t43Kzt5Wva5jx07asCFd+/d/oT17dmn//r1atWqlNmxI0fvvJyswsK38/VsqOXm9Dh06oN27P9e+fXuVkpKsjRs/0pIlcerR479ks9nu8J7YapyxdTdMJpPeeGOhzpz5Vp9//pn27ftCn3ySqa1bt2jy5KmKiHi2eq23t7cWLlymt99+XXFxS9WnTz/5+tb/7UgBAAAAALAXLt9Dnf146VjTpk3Vq1fvGh/e3t6yWq3y9PRUWVmZjhw5LMlFTz89Um+++a62bv0fjR37O3377Snl5u675xnKy8uVlPS+QkN7as2ajZozZ57Gjh0vszlYly9fql5XVVWlb745qcLCAj3++EDNmTNPH3+cpddff1slJSXKyEiXJJ05863Ons1Tz55hmj79VW3Y8LHi41fJZrMpLW2DJKlVq9bKzz8ni8VSY5bKykqdP58vf/+WdfoeCgoKdOTIYXXo0FFRUX9UQkKy0tIy9cgjAVq/fm2NtQMGPKGQkO6aPv1VXbpUpJUrY+/lbQMAAAAAwGFQSqHOgoK6qXnzFkpN3aiysn/v3VRaWqLXXvuz3nrrdbm5uSkv74ymTJmkrVu3VK9xd3dX585dJElubjd//Fxdb/55pzOR/lN5eblu3LihRx4JqHE53enT3+jw4UOSJIvFIqvVqmnTJismZnGNzzebg2vMMH/+HEVHv6aqqqrqNZ07B8nd3V2urjc3cu/ff4BKS0v18cebanyt9PRUlZWVql+/AXc9vyStXZuk6dNfqHHJob9/S/n5+Ve/J/+pT59++tWvBikzc7O++upInV4PAAAAAABHwuV7qDOTyaTp02fpr3+dq6io32vEiJHy8PBUZma6CgrO67XXomUymWQ2ByskpIcSE+N14UKBOnTopAsXCpWWtlGBgW3Vs2dvSVKzZr+QJKWnp6moqEhDhz71kzP4+vqqW7dgZWVlyMvLSwEBgcrLO6PMzC1ycblZ6JSVlcnX11fPPDNeH364Wn/+8yz17t1X5eU3lJGRrp/97GcaPnykJGnChAgtWPCGXn75BQ0aNESSTdu3f6KKigqNHv2MpJt3Dty+fatiY5fqzJlvFRTUTSdPHtcnn2TKbH5UI0aMqtP7OGbMOG3fnqUpU57XyJFj5OPjq4MH/6lDhw5o0qQ/3fbzpk17Rbm5e/Xuu28pKemjW/a4AgAAAADAGfCvWdyTQYOGyMfHV2vWJCk5ebVcXV3Uvn0HLViwRP373zxjyMXFRW+/vUhJSYnas2eXMjLS5ePjoyeeGKxJk/5UvQdTz55hGjz4Se3Zs1MHD/5TAwcOuqtN0KOjFyg2dqmysjJUUVGpVq1a6fe/n6i2bdvpL3+ZrUOH/qknnvhvPffcZPn6+iorK0Pvvbdfbm5uevTREM2fH63AwLaSpKefHiU3N5PS0jYqIeE9Wa1WdenSVYsWLVdoaE9JkoeHh5YvX6GkpER99lmOsrO3yc/PXxERkZo4MarO5VCHDh21bFm8PvggUevXp6isrFSPPBKgV16ZrVGjxt7281q1aqWJE5/T+++/p/XrU2rsPQUAAAAAgLNwsdXlmqkGrqioRFbr7d+OgoL/VatW3PUMD5bJ5CqLxWqX1+JnGqidn5+PLl4sNnoMoNEig4DxyCFgrIaSQVdXFzVv7n375+04CwAAAAAAACCJy/eAenXt2lVVVlb+5Dp3d3f5+v7cDhMBAAAAAOCYKKWAejR37qvVd/+7k+7dQxUXl2CHiQAAAAAAcEyUUkA9mjp1hoqLr/3kOh8fXztMAwAAAACA46KUAupRUFBXo0cAAAAAAMApsNE5AAAAAAAA7I5Sqo5sNpvRIwD1gp9lAAAAAICRKKXqwNXVTVZrldFjAPWiqqpKrq5uRo8BAAAAAGikKKXqwGTyUHn5daPHAOrFjRul8vRsYvQYAAAAAIBGilKqDnx8mqmk5KoqKm5w6ROcks1mk8ViUUnJVZWVFcvLi7sAAgAAAACMwd336sDd3UM+Pr/QtWuXZLFUGj0OGihXV1dZrdYH+PXd5OnZRA891FImk/sDex0AAAAAAO6EUqqOmjTxUpMmXkaPgQbMz89HFy8WGz0GAAAAAAAPFJfvAQAAAAAAwO4opQAAAAAAAGB3lFIAAAAAAACwO0opAAAAAAAA2B2lFAAAAAAAAOyOUgoAAAAAAAB2ZzJ6AEfi6upi9AiAJH4WAUdADgFjkUHAeOQQMFZDyOBPfQ8uNpvNZqdZAAAAAAAAAElcvgcAAAAAAAADUEoBAAAAAADA7iilAAAAAAAAYHeUUgAAAAAAALA7SikAAAAAAADYHaUUAAAAAAAA7I5SCgAAAAAAAHZHKQUAAAAAAAC7o5QCnFhWVpZ+/etfq0ePHgoPD9fBgweNHglotP7+979rwoQJRo8BNDpkD7A/jkEB461evVrBwcHq0aNH9UdhYaHRY9UZpRTgpPLy8jR//nwtWbJEX375pcaPH6+XX37Z6LGARsdmsyk1NVWzZs2SzWYzehyg0SB7gDE4BgUcw4kTJzR37lx9+eWX1R8tW7Y0eqw6o5QCnFT79u21a9cumc1mVVRU6Nq1a2rWrJnRYwGNzqJFi5Senq6oqCijRwEaFbIHGINjUMAxnDx5UkFBQUaPcd8opQAHVllZqWvXrt3ycf36dUmSl5eXjh8/rpCQEC1dulRz5swxeGKg4fmpHE6cOFHr1q1TYGCgwZMCjQvZA4zDMShgrPLycp09e1YJCQnq27evfvOb32jHjh1Gj3VPTEYPAOD2srOzNXPmzFseHz16tBYsWCBJ6tSpk44ePaqMjAxNmzZNOTk5atGihb1HBRqsn8qhv7+/AVMBIHuAsTgGBYxTVFSk0NBQRUREKCYmRnv27NGMGTOUlpamDh06GD1enVBKAQ5s+PDhGj58+B3XuLu7S5LCw8OVnJys3NxcDRs2zB7jAY3C3eQQAIDGhmNQwDitW7fW2rVrq/8+aNAg9e7dW7t27XK6UorL9wAntWPHDk2ePLnGYxUVFfLx8TFoIgAAADR0HIMCxjtx4oQSExNrPFZRUVFdFjsTSinASZnNZh08eFA5OTmyWCxKSUmRxWJRz549jR4NAAAADRTHoIDxmjZtqri4OH322WeyWq3atm2bDh8+rCeffNLo0eqMUgp4wObPn6+IiIhan/v+++81depUhYWFKSwsTLNnz9alS5fu6uv6+fkpLi5OsbGx6tOnj3JycpSYmKgmTZrU5/hAg/Cgcgjg3pFLwDj3kz+OQYH6cT85DAwM1JIlS7R48WKFhoZq5cqVWrFihVPut+his9lsRg8BNFSpqamaN2+ewsLCalzzK0mXL19WeHi4Kioq9Ic//EFVVVVavXq12rRpo9TUVHl4eBg0NdCwkEPA8ZBLwDjkDzAeOfw3NjoHHoCqqiqtWLFCcXFxt12TnJysgoICZWZmVm9GFxISosjISG3evFnjxo2z17hAg0QOAcdDLgHjkD/AeOTwVly+B9Sz8vJyjR49WrGxsRo5cqRatmxZ67qsrCyFhYXVuDtCv3791K5dO2VlZdlrXKBBIoeA4yGXgHHIH2A8clg7SimgnpWXl6ukpERLly7VwoULZTLdekLi1atX9f3338tsNt/ynNls1rFjx+wxKtBgkUPA8ZBLwDjkDzAeOawdl+8B9czb21vZ2dm1/pL5UWFhoSTV2o77+fmpuLhYxcXF3FoXuEfkEHA85BIwDvkDjEcOa8eZUkA9c3V1veMvGkkqLS2VpFrvUuLp6SlJKisrq//hgEaCHAKOh1wCxiF/gPHIYe0opQAD3M1NL11cXOwwCdB4kUPA8ZBLwDjkDzBeY8whpRRggKZNm0q6eV3xf/rxMW9vb7vOBDQ25BBwPOQSMA75A4zXGHNIKQUYoHXr1pKkixcv3vLchQsX5OvrW/0LCcCDQQ4Bx0MuAeOQP8B4jTGHlFKAAXx9ffXwww/XeveE48ePKzg42ICpgMaFHAKOh1wCxiF/gPEaYw4ppQCDDB06VHv37tWZM2eqH/viiy909uxZDRs2zMDJgMaDHAKOh1wCxiF/gPEaWw7vvPU7gAfm+eef15YtW/Tss88qKipK5eXlWrVqlcxms0aOHGn0eECjQA4Bx0MuAeOQP8B4jS2HnCkFGOShhx5SSkqKgoKCFBMTow8//FBDhgzRqlWr5OHhYfR4QKNADgHHQy4B45A/wHiNLYcutru55yAAAAAAAABQjzhTCgAAAAAAAHZHKQUAAAAAAAC7o5QCAAAAAACA3VFKAQAAAAAAwO4opQAAAAAAAGB3lFIAAAAAAACwO0opAAAAAAAA2B2lFAAAAAAAAOyOUgoAAMBJjB8/XsHBwRo3bpx++OEHo8cBAAC4L5RSAAAATiIyMlKjRo3SkSNHlJSUZPQ4AAAA98XFZrPZjB4CAAAAd8disahXr17q3LmzNm7caPQ4AAAA94wzpQAAAJyIyWRS586dderUKfF/iwAAwJlRSgEAADgRm82myspKlZWVsa8UAABwapRSAAAATmTdunU6duyYJOnUqVMGTwMAAHDvKKUAAACcRGFhoZYsWSI/Pz9JlFIAAMC5UUoBAAA4iejoaFksFsXExEiilAIAAM6NUgoAAMAJ5OTkKCcnRy+99JJCQ0PVvHlznT592uixAAAA7hmlFAAAgIMrKSlRdHS0zGazIiMjJUldunTR2bNnVVFRYfB0AAAA94ZSCgAAwMEtXrxYRUVFevPNN+Xm5ibpZillsViUl5dn8HQAAAD3hlIKAADAgR0+fFgbNmxQVFSUunbtWv14ly5dJLGvFAAAcF6UUgAAAA6qsrJS8+fPV0BAgKZOnVrjOUopAADg7ExGDwAAAIDarVq1SqdPn9aaNWvk6elZ47mOHTvKZDJRSgEAAKflYrPZbEYPAQAAAAAAgMaFy/cAAAAAAABgd5RSAAAAAAAAsDtKKQAAAAAAANgdpRQAAAAAAADsjlIKAAAAAAAAdkcpBQAAAAAAALujlAIAAAAAAIDdUUoBAAAAAADA7iilAAAAAAAAYHeUUgAAAAAAALC7/wfMu5KHomb2kQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_179_0.png" } }, "output_type": "display_data" } ], "source": [ "fig = plt.figure(figsize=(20, 14))\n", "\n", "colors = {\n", " \"ols_sk\": \"r\",\n", " \"ridge_sk\": \"y\",\n", " \"lasso_sk\": \"c\"\n", "}\n", "\n", "for key in train_errors:\n", " plt.semilogx(\n", " lambdas,\n", " train_errors[key],\n", " colors[key],\n", " label=\"Train {0}\".format(key),\n", " linewidth=4.0\n", " )\n", "\n", "for key in test_errors:\n", " plt.semilogx(\n", " lambdas,\n", " test_errors[key],\n", " colors[key] + \"--\",\n", " label=\"Test {0}\".format(key),\n", " linewidth=4.0\n", " )\n", "plt.legend(loc=\"best\", fontsize=18)\n", "plt.xlabel(r\"$\\lambda$\", fontsize=18)\n", "plt.ylabel(r\"$R^2$\", fontsize=18)\n", "plt.tick_params(labelsize=18)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From the above figure we can see that LASSO with $\\lambda = 10^{-2}$\n", "achieves a very good accuracy on the test set. This by far surpasses the\n", "other models for all values of $\\lambda$.\n", "\n", "\n", "\n", "\n", "\n", "\n", "## Exercises and Projects\n", "\n", "\n", "\n", "The main aim of this project is to study in more detail various\n", "regression methods, including the Ordinary Least Squares (OLS) method,\n", "The total score is **100** points. Each subtask has its own final score.\n", "\n", "\n", "We will first study how to fit polynomials to a specific\n", "two-dimensional function called [Franke's\n", "function](http://www.dtic.mil/dtic/tr/fulltext/u2/a081688.pdf). This\n", "is a function which has been widely used when testing various\n", "interpolation and fitting algorithms. Furthermore, after having\n", "established the model and the method, we will employ resamling\n", "techniques such as cross-validation and/or bootstrap in order to perform a\n", "proper assessment of our models. We will also study in detail the\n", "so-called Bias-Variance trade off.\n", "\n", "\n", "The Franke function, which is a weighted sum of four exponentials reads as follows" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{align*}\n", "f(x,y) &= \\frac{3}{4}\\exp{\\left(-\\frac{(9x-2)^2}{4} - \\frac{(9y-2)^2}{4}\\right)}+\\frac{3}{4}\\exp{\\left(-\\frac{(9x+1)^2}{49}- \\frac{(9y+1)}{10}\\right)} \\\\\n", "&+\\frac{1}{2}\\exp{\\left(-\\frac{(9x-7)^2}{4} - \\frac{(9y-3)^2}{4}\\right)} -\\frac{1}{5}\\exp{\\left(-(9x-4)^2 - (9y-7)^2\\right) }.\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The function will be defined for $x,y\\in [0,1]$. Our first step will\n", "be to perform an OLS regression analysis of this function, trying out\n", "a polynomial fit with an $x$ and $y$ dependence of the form $[x, y,\n", "x^2, y^2, xy, \\dots]$. We will also include bootstrap first as\n", "a resampling technique. After that we will include the cross-validation technique. As in homeworks 1 and 2, we can use a uniform\n", "distribution to set up the arrays of values for $x$ and $y$, or as in\n", "the example below just a set of fixed \n", "values for $x$ and $y$ with a given step\n", "size. We will fit a\n", "function (for example a polynomial) of $x$ and $y$. Thereafter we\n", "will repeat much of the same procedure using the Ridge and Lasso\n", "regression methods, introducing thus a dependence on the bias\n", "(penalty) $\\lambda$.\n", "\n", "Finally we are going to use (real) digital terrain data and try to\n", "reproduce these data using the same methods. We will also try to go\n", "beyond the second-order polynomials metioned above and explore \n", "which polynomial fits the data best.\n", "\n", "\n", "The Python code for the Franke function is included here (it performs also a three-dimensional plot of it)" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAScAAADsCAYAAAA2AmCCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABxNklEQVR4nO19eZgcZfX1qb33njUJJIjsISSAGDYfPyKyiQZEieyyJhDzwwD+AAGDEHYwsogfkYQdFUTZjEKIoEEEN/gEBBFRQNYkM5mZ3pfavj/efqurqqu6qnuqZyZJnefpJ+np6uq3t9P33vfccxld13WECBEixAQDO94LCBEiRAgnhOQUIkSICYmQnEKECDEhEZJTiBAhJiRCcgoRIsSEBN/sxoGB3FitI0SILRb9/cnxXsKERBg5hQgRYkIiJKcQIUJMSITkFCJEiAmJkJxChAgxIRGSU4gQISYkQnIKESLEhERITiFChJiQCMkpRIgQExIhOYUIEWJCIiSnECFCTEiE5BQiRIgJiZCcQoQIMSERklOIECEmJEJyChEixIRESE4hQoSYkGjq5xSic9B1DQADAGAYZnwXEyLEBERITuMAjgNUVYWqqgAAhuHAMCxoIBuSVYgQITmNKRiGEBPDMIhEoqhUZFSrVWiaCoZR6VFgGNYgq5CoQmypCMlpjMCyhJh4nkMsFkGlUkU0KiGdTkDTNIOoKhUZ3d0J5HIFyHIFIVmF2FIRktMYgK+9ypGIBEHgkc+XUa1WoChq7XYOoigiGo0inU4CYBCLxVAqlVGtytA0BYSTGIRkFWJLQUhOHQRN41iWQTwehaZpyOUKsA+AVxQVilJCsVgCAPT2dkHXdcTjUXR1JaGqmhFV1cmKklJIViE2T4Tk1CGwLLmIIo9oVEKpVEW1Kvu6r6ZpKJcrxvGCwEMUhRpZpaAoCqpVGZVKFbIsQ9dV6Lq9ZsXV/h+SVYhNEyE5BQwaLQFALBYBz7PI50tQVa3hOL+QZQWyrKBQIJGVKAoQRQHJZBw8z1vIihBaSFYhNn2E5BQgaLTEcSzi8SgURUE2W3Q5un2SqFZlSxTmRFaUqChZCQILlmVQLlfBMExIViEmPEJyCghEIgBIkoBIRESxWIEsK2Py2GayYhjGSANTqTg4joMsK9B1HboOlEoVADBFVqypZhWSVYiJg5CcRgmzdikWi4BlGeRyRWia7n3nDkDXdYOs8nmyLlEUEItFIAg8Jk/uhSyba1YKAM1UpA/JKsTEQEhOowDPM4hEBCiKing8gmpVRqFQHfV5db21mlTzc+moVKrgOBaKoiKfLxppYDqdBMexqFYVYzdQUUKyCjExEJJTm6ASAUkSIEkCCoWyoVvygiDwiMXiUBTViHKsO3k6RlOTagZKVpUKIVGWZQyyIpEfa6ynUqnWnlNIViHGHiE5tQhzGhePR8AwDDKZAnS7eMkFsZgEnueRyxWgqhpEUUAiEQPP85BlQgpj+YXXNB3lchXlMiUr1kRWUbAsUyMqomC3kxXLctB1JiSrEIEjJKcWQHfjSOQjoVKRwTCCL2JiWRbxeASapiGbLUDTVKP2A9QL2ZIkQhB4dHUlDdFlnRQ6D6qxKpcrxropWSUSUQAMqtWqEVmlUnGUSmUjEgsjqxBBISQnHzBrl6JR2oJSgqbpkCTB8/5+hJjmQjbHsSiXq9B1HaIooKsraUu3ZMPRoNOwkxXHmckqBpYlqnSWZVGpVGt6rjANDDF6hOTkARotmSMf2oLi54tGhZi5XAmapnkeD6D2xa7XhnI5awQTj0cBwFKvsos8OwVV1VAqVQxJQk9PCoqiQhRFJBJxC8lWKtXac7aSFcsSjVVIViGaISSnJqDaJffIx71wzbIsEokIFEVrIsT0j+YRjJUUiA3L2EgZdB0mZTrAcZyh9Uql4tA03ZIGappmI2kOLEuiqpCsQpgRkpMDzGlcPB4BxzlHPm5b/qIoIBoVUSpVUK22I8T03q2zRzDE2aBOCqRZ2GknsLNQVRXFoopisWxbl4RUymwPQ8hK11VoWj1FDY33QlCE5GRDay0ojWgnjQsCxNnATgoiYrEI0ukEdF03dgdJs3Awj8swTNMNAbd1RaNkXWYSJZEVMd4TRRG6rkFR9JCstlCE5GTCaFpQ2iWzTsFuw5JMxiEIvCFboM3CYx1Z2ddFW20oiVJ7mLo4tGRyCbVGViFRbd4IyQmEkGIxAQADQeDAMP5bUBiGaYvMJEmsaYiUhsgjSIU4BSmYK8jlCgBg2XHjec6QNVSr8pj1BAKNjguUrCIRAZIkIRKRLPYwoZfVloMtnpyofS7LspAkwSJI9ALDwNjub4XM4vEIAPJFTKcFmz9TJ4mhvj57szAlq1QqYWlpqVblMdNYAXWyIg3LspGKutnD6HroErq5YosmJ2qfK0lirRaj+CYmmsYBQC7nL43jeQ7xeASVioxisQxZJuRQJwbiIqDrOhiGfFHHghjsLS2UrCTJWSVuli141ZzaBSH+1u1hQrLafLBFklNdu0Tsc3VdR6lUAc/7mzFqTuNoFOSFSER07cGzuwh0dSXAsqxFfOlEDJ2ClawKhsZKkqhKvJE0ggcDc6RH4ccexhqFhsZ7myq2OHKiRW/aglIuk258UeThtX1PalOkOZamcVSM6RY91HvwgGy26Bll0F01WVZRKpUdicHc1uJXzzSa76GzxkqEKIrgeQ7d3SlL9BKExopGTs3gZg9jdlxwIqvu7iSGhkYQktXExhZDTlb7XNJ8a7fPbfbZpGkcKd6a0zj3bxDHsUgkoqhWFUOP5AfmgngjMVhFjnY9UzPyCyr7IhqrMkqlMgShC9lsATzPGVqmVtbkjtaJwi09tdvDCAIPjuNr9jBhZDVRsUWQk7kFpa7aLliOafb9kSQRkYhQqxP5qwHVUz//9/GDRpEjX6sNka14dxuWToGBqpLHpGtqlAeoRrTnV2NFNxtGA2d7GJJed3fb+xVpc7WZrMK+wPHEZk9O9RYUL9V2oyrbbIvilpLRKIfe5JT6dRKKokBRGocf2CUCLMt2pHDtBCd5gCSJDRqrzu9OWkFbaXRdx+DgSMv2MCFZjS02W3Kyt6B4kYVdW0R31qpVGaVSazt4jamf+xqD5gs3iUAkIoFlGfA8F6izgZ/vJyUrerwgCC5FbOrE2bldQKB+3lbtYULHhbHFZklOZu1SK2RB0WxnrREk4pIkviUhpvfnePQfdHNao2kaWJaBLKsNzgb1YZ3t7QS2wiFmeUBjEZtqrEikRwrabS3JFc1Iz8seBoAlDQzJqrPY7MhJklhEIiJUVYMk+a8T0V03uiPmZ2eN3i8Wk1pSlZOorO4k0EgOnUm/dB2Ou251BwHNtBPot5A9ui+gm21wKkW0TMlkwtiZDMIappVo1d5cbd6l9LKH6e5Oo1SSIctj11+5ucGfsGcTAMMQUSXDMOB5HjzPIZst+i5GcxxjGgJQ8vXF5Djy667rum9ikiQR8XgEhUIJw8MZVKsKIhERfX1d6O3tQjIZrxm4+Vr2qEB33UZGctiwYQgjIzmoqoZYLIL+/m709qaRSMQgil6GesGRKbUN1jQNw8NZbNw4gmq1ClEU0NOTRn9/N9LpRC1Nbf3jO5p0kb5emUwOAwNDGB7OQlHI+9ff343+/h6kUgm89tqreP3113ynzfl8HnPnzsUHH3zgeszatWvx+c9/3riezWZx5pln4vDDD8eJJ56IgYGBtp7TRMZmETnR3ThaJwJ05PMl3/enaRwJ6/3Vl2iBnX6ZvGAulGezRWiaClXVoChkSx6o77xJUgSRiARJEse0cOzUlGsuZFOPc3P/XSfqZgSERDTNHr1YrWE0TbNEn17EE2Sq5WYP8/OfP4S1a9ciFothzz33woUXLkFfX5/jOV5//TV8//vX4N1333V9nMHBQVx//fWWv918882YPXs2VqxYgcceewxXX301br755qCe2oTAJh050WiJZYl9bjweQbFY9v1loWkcz3MtkVk8HoEk0X46zfMDz3EskklivJbLuaeLdNetXK4gny8inyd1slQqgUmTetDVlapZsnC+1zoayLKCfL6IoaEMBgaGUCyWwLKsaT3J2vMLfj1upKeqakO0pygqolEa7ZHoU5JE1/elU7uW1Bpm6dKr8cc//hFLllyO7bffESzr/vlYtepRXHbZZZg0aZLrMUuWLMHZZ59t+dvatWtxxBFHAADmzp2L3//+90Y71OaCTTZysregaJpe0y4xvlIic59buVytfYCa35HopIgtit9+OhphtTMBuF44LhruB6IoBlrM9gtSGyKPB9RrQ5IkoqsrCYYx72wFsR7n9hU77H5Rdo2VXffVuV1A08oZBj09PTjwwINwwAGfb3rsRRddiv7+pOvt9913H2bMmIE99tjD8vcNGzagv78fAIm4E4kEhoaGMHny5NE/gQmCTZKc3FpQAIBhdM9IJholjb7m3Tgvm5K6Va9/d8vRGM/Zn4OuW0c42YvZwaiy/YOms+k0MDg4bNrZMheL62TV6nraTRftGiur7ouHpqnQdR2CwHcsVaZv3Wjfgn/9619Ys2YN7rnnHqxbt87z+HZqcBMZmxQ5WVtQyBff3oLS7ANB0zhd133vxpkfy41k7KRGozlVbeYf7s6ExJXAj01vY73KHjGMVb3K3TY4YmppIT8iQTpxesGu+0okohBF0bCGkWXFJroMDqP9gVi9ejUGBgZw9NFHQ5ZlbNiwASeccAJ++tOfYtKkSRgcHMSUKVOgKAry+Ty6urqCWfgEwSZDTlb73OaDA5y+2ILAIRaLWKIsM5wIwc+QAvIBrN+PRnPNxkB1Am5Kcfol1DQNqqqB57kxsWFxS7fMKnHaLOxEnp1Iv2hTdbUqI5crWDRWQY7fCqrovnjxYixevBgA8MEHH+Dkk0/GT3/6UwDAnDlz8Nhjj2HhwoX4zW9+g8MOOwyC4D2mbFPCJkFOdvtcr9TK/qE2z5rzq5PxM2vOjnYep1Ow16tSKeJ/1NWVMupDQdSr/H4RzemWVSXuHMF0TsBYr2WZNVZk/BZT0zGRdhZzHa0VjRVde7vcumDBAixevBizZs1yPeacc87BRRddhGOPPRb3L78dhx9+eHsPNoHB6E1+ngYGcmO5lgY4taAUCiVPPVE6nUA2m6/1xkWhaZqvXbyurgRGRvKGa0Gh4E0y8XgEsqxAkgToOlAolHx9KCMRYuDvtAUej5MvBt2tCwLmc9L6EDXZa098STB5ci/Wr984qrWZze1EkdgXk37GfCDCSzNaeW3NCnEycEG31PXcSF0QePT0pDE4mPe1pmYFcb/48xe/jvJH3nUpv4hsPQX7PnF/YOdrBxM2chJFFhzHQNdR63FrpQWFFDyjUck1jXNDMhkzRob7A4NoNIJKxZ+9L617EEJQHGtEnYZTfUiSxHGrVzmZ2/X3dxsz+ciA0bpKfDTN1K2ki40KcftMPmdSZ9nO7wjaIQ9tgDwYHDlxkfEvrk84cqLREvnC8LVoyU+Pm+UsiEalltIrQSAvBf2w+UEkIkIQOJTLsi9iMssXSqUKZLnqWCPieQ6qqhof/k6PHqfiy8a1kMbcsfYT13Uduq4jkyGRR50U7H5RVTgNiGgGhmHaTmOdRZeiZXLMunXr8Mwza7D33rPR1TXFV3qaz+dx3HHH4Uc/+hGmTZtmue3pp5/GrbfeCl3XMW3aNFx77bVIp9N48cUXcc0116C7uxt33nknuAgLLhocoYTkZAMtejMMg0hEaGpV4nx/pha2A9ms/+17WivSdfgSsjEMjMfxW7Ohv7huRGuuEcXjUQiCAJ7nDU0TjRza2ZZvFU6NudRP3Fyv6tSum11GYCeFupYpinSab8nDKkhFu5Oift269Vi5cgWuuGIpenv7MG/esfj6109zPUczhXg+n8fll1+Ohx9+GJMnT8Ytt9yCW2+9FUuWLMHFF1+M5cuXY8cddwQA8FEWQiw4MSwfING1vYbxXgBFXbtEdtVkWW3JcMy8SyZJgi8hZl3AqSGXKyCVisNL/Ge2RSmVKohGJc/HovUy/83EOlRVNcY41SOH+ra82Ra3k7CnXOZ6VSoVB8OQmXjB6quavwfNtUxWH3H7DwGJZDpD7rKsYJtttsXjj6/Cxo2D+N3vnkM8nmh6H6oQv/DCCx3OJ+Pyyy83hJW77LILVq1aBQB44oknIAhkQKogCGB5DqwQHDmxY9SF0AzjTk7mord5t4tET6Kvc9htdyXJe0vVScDphVbV3nUpgupbUe4Ep8ih3vPG1Xa6mqddQe1+meswDMOgv78bmqYFWq9qNbpx87ByGhBh9nPqFFiWxbRp03DkkV/xPLaZQry7uxsHH3wwAKBcLmPFihX4+te/DoDsdr755pu4+OKL8cgjj4CXOGiRACMnaQsnp8YWFBLB6Dp89Y+52e56iRjdtvztrpZmxGIRcJybYZ2TropGcv4V5X5hNW9zT7taGYDQDuhrVSgEXa9qP7pplAdYB0SwLGu0KnWq7SdojVYul8OiRYswffp0fOUrdcLbZZdd8MgjjwAAWJ4NOHLagtO6un2us57Ii2Da0SHVd8p0gwStoILK+g1mAnSKfuiMOTNGq3dqZeKvc9rV2NbCssyYaK/81Ku8+u+CrAvZDeR6elJQFHWUHlbNQchp1KcBQHrozjjjDOy333645JJLAACVSgXPPfecEVUBACew4IQAC+IBnqtdjDk50TSOmLS5t4U0+4J6tZM4ERvdKWuWxtkfs1W1t3marzP5dR72tpb6eG9S9xIE3rUe0zqaRzhu9SpRFGsuDcETg581l8v1tNxNRjGael5QUgJVVbFw4UIcfvjhWLRokfF3nuexdOlSTJkyBTNnziSPKXDgxOC+zkFGYe1iTMnJbp+rKEqT3rPGgQN+WlecQBt98/my7235VqMfu8uBFziOA8dJNQdF+we58bm3C5oC0lRDURSIomjUY8y7gJ12NvCrr1JVtWNEZf/Bc5NR2Ivrbm02zo/BYDQvJVWIr1u3Dv/4xz+gqiqeeuopAMDMmTNx9dVX46abbsJ3v/td9Pb2YuXKleB4FlqQkdMESOvGTCFeH/0t+PLaJi0XMWQypJbkPT2ljlgsAkVRIMuqEckUCmXPD3wiEUWlIiMSEX2ryiMRMliS4/zrsehrUK3KEEWhYfeNEKOAbNafwtgPEokYdF03voQA+ZEgqmyy+9ZqJMNxLLq70xgcHA5kjfVhDGKtXiV7FvpbRW9vFzKZnK/zkbSUN1paqL+5V+9dd3cKiqKhWPQXeQWhEP/Poq9DHlg/6vNQCP2TscNtm7lC3KxdomOW/FjamlOz+vQUv9olHRzHIRqVfEcyAFljvTnY330EgQfL+vcPp88lny+iXK5A0/SG3TdV1aDr5Dl0UoBpd5l0imSaNecGDXM6FY9HUSyWW65XeaGVYrWbh5XdU8u+prHwjLIjTOtaPTnPIJEgX/ZWUh4zUql4S+ZuAEmZaB+e319c8mvNolSq+JIW0EI56bdSPYnJbKNify723bdEIgZJEtDTkwIwdgJMpxRHkurNueYteZLqdkYzRFOvTtSrRlNspx5WVk8t85p0/OxnP0OhkMenPjUbn/jEDp4eS4VCHqeddryjOvyNN97AkiVLkM/nMXv2bCxduhSZTAann366cUwsFsMDDzwATmChb2YF8Y6sgNrnArpRiykUyi0RE9UqlUoVFIv+RnnT3TiGYXwXfM1WvbKs+Ip+BIFHMhlFuSz7kgnwPIdkMoZqtT4V1w26rhtkNTAwjKGhTM1E3zp0gLbbdBLEWqSIjRtHMDg4jHK5ClHkjUEDyWQMnRmB1Bh50HpVJuM8jKGnx9/rEmRU07imLIrFIh599FGcdNLxOOKIQ/Dmm/90vf/rr7+GRYvmu/qHX3DBBbj00kvx1FNPQdd1PPTQQ+jt7cXjjz+Oxx9/HI8++iimTJkCAGA5jggxg7q0Yb3cbFDDG2+8gaOPPhqHHXYYvvOd7xjzCZshcHKqF72Z2oeXjFnyG8GQ1pAIRJF0+ft5EgAhgFQqZhRV/YDjWKRSMaiq6ttDPBqVjL498ovdfNs/EiHTVvL5cksNyBSqSmpfIyNZbNgwZOi5Uqm44eNNNVidBIkaKshk8jXSzEJRVLAs0+KkFm/4iW6IV1QJw8PkdaEuA8kkeV26u1OIxaINerlOKsQVRcURRxyFX//61/jVr1bjggsuwVZbbe16/KpVj+Jb3/q2o3/4hx9+iHK5jD333BMA8NWvfhWrV6+2HPPwww8jEiE1VZrWBXVpNa175ZVXcPzxx7dEtF4I9OfXyT43GpV8/1LZJ+am097tJEDjEExJEuHlWFovzNfn2jUjGneZgPvOWqttK3W4n5NGVVRHZPYVJwMr6z5NnUwBVVVFuVyBJInYuHGkLdW6O1onECeVOKlXpSz1KqBT02LqIMr5yfjc5w5qetxFF13qepvZIxwA+vv7sX59veCtqiqWL1+OlStXksfkODABDpqg5/r4448b6p6pVAqpVMryt4ceesi1DceJaH/wgx/ghBNOaLqGwHMDeytJJCL6CqWddvGaKbYBK2FYCUBHs6DQfTy5MylwHIdEwn/NjA5CaHXScKuw+4pTa1yngnZnUCeR5qp1WOpVXqnzaEWYzfRVANDX19VRfVUQqaPT/c3p83PPPYftttsOO+ywAwCA5VjoAfbDsbVI/MQTT8SHH35oue3ss8/GN7/5Tcvfrr76atdzeRGtGwIlp2SS+nObW0m8CAaIxaKOO171HbvGOzcjDLcIqJ1eNy83AXfhZvttK60oxM2wW+PabVgURTXmvAWhFnd7X/2p1usRnsOZEWTqRWtD5XIVktSNkZGcxeokCOElRd0Fc3Trnzx5MgYHB43rAwMDlvTv6aefxhe/+EXjOivyQJC7dbVz/eQnP3GMnFqBF9G6IVByyufLaPxQNbaEUFCCIUZy/oreAJmaG4kITXRFjRGQn+kpdlJwj7CcEY2KEATBl3CTbBpwgffdmWG2YUkmY2BZzjBwowQyFupsN9W6m4tAkO0rZtDz2q1OnISX7Q49GK1FL8XUqVMhSRJeeuklfPrTn8Zjjz2GAw44wLj95ZdfxoIFC+qPy3FgAoycaFq31VZbjfpcXkTrhkDJSdP0hlqPWxRA60Tmmo8d9jYUc5TVrI5jf0x7qtkcDFiWqc2nc+6nc7pPIkF0L82GZlJwHGv0+LEsMyYqbbq5QGUCbpqmSkX2vQnRLqx+4lYXAYYhwxg0TQPLsoG+Hm5RuHe9auxU9Gb/8GXLlmHJkiUoFAqYMWMGTj75ZOO4999/39ipA2ppWIA1JzbADRYvonVDx/ejGwmGijHtdSKn+9ZJxlos9xdlmd0O/NjuEuEji0gk1kI/HfGgqlSqKJW861E0gisWKygWiwDqRe1kkqQ8qqo2nRIbBMyaJjpwQJJEpNNE0+SHMIOqrdhdBKijQV9fV8D9d97rdbIMpip68zw+t02HdtK63/72t8b/aYEbAKZPn45f/OIXjvd55ZVXLNdZIeC0LgCpil+idUOg7StUDW5GPB41FMat9p9R73CWZXy1vFDwPFGHsyzTkl9TIhEFx3HI54v4w5RPQ5frL81nNrzYcDwhmkjN6sU7wjL362maBlluLA6TmlUEkkSKt9RypFKpjqpO5NS+4ob6F5JEt9TZwF5cF0UB8XgUw8PZttflBPMQAhrhiaIAQeBHpVrneQ7pdBIbN460vTZqzStJ9fWY61U8z6Ovr8vXd2fNmtW47747oWkqTj31VJx44omW25999lksW7YMALDzzjvjiiuuQDweRzabxfnnn4/3338fO++8M2655RZkblsCLTPU9vOyg033IL3oqsDO1w7GoPGXWIrYt/t93VOHYTjnt+4DwOiDyuVKvto/GIbBC1vPBgBwUQ66rFmICQBemERupyRFiaZUKntqe1pxK5BlBeVyBQzDIpPJGX1vYykVaGxr4Wt+SNYaUadSHHNE5qxaF5uo1v2dt100q1d98MH7uPDCCzFr1izsttun8OlP741EwtkJc2BgA1auvA133nk/ttqqB8cddxz23Xdfw3Y3m83ioosuwv33348dd9wRK1euxE033YQlS5bg5ptvxuzZs7FixQqsWbOGPDdeABvg3DqGH/8ZeIGSk9vODWmkbW3KLtnhIb9MfgWS5v49mh41w1923K/+eFGSr+uyVrtOQkC1ZP3AvzBpNrr3SGH2Cy8gmy2A47imOw80Ha1W/bfu1Hc4zSmGu1Sg03Ui88BOc40oFiMbBul0IlBzO4aBa1e/uT5E58xR4vSTcgVN6Ob1SFIchx56KP7yl7/gkUcewbbbboc773Runn3xxb9gr71mI5VKIxaL4bDDDsPq1atx9tlnAwDeffddbL311gZZHXjggZg/fz6WLFmCtWvX4ic/+QkA4POf/zx5bjwLBFkQnwCuBB3urSNfplYIBqhv38uy4jvKMqeMskxGLjXDO/MahxBSYjKDi7INBDX8ShZ/m/NZ7PzE02i25W2uLwXROOskFZAkEV1dCTAMOyYOmGbCrFZlxGISqlXFIhMYvb85A8A7KqOqdWok50TeZsvgTu0CUnAch2OOOQ5nnHEG1q/PoFRy/8wPDg6gt7fPuD5p0iS8+uqrxvVPfvKTWLduHf75z39i+vTpePLJJ40dL7NuiK/ZfTAsF2hBnGE348Zfmsa1OmfMvH0vioIvPYRdi0SHMrrhnXmHY+Bl/zYfNIrq3s2q7/jXFw/Grk/91ukuRtrn30mhdVACoIVk80y1sRqCoOtokAmMVineLom467xIgZ1OEu6k2wPxctIhCELT8eBe2p9UKoXrr78el156KTRNwzHHHNP0fIzAAQH2WzKboyuBXbUtioKvnSea/rTqQNCKFolGS07E5BQ1mdH3qS4AgFq1HvfGYZ9Heloak1c8DKBVN8zgxIb2OpGdJDRNg6Ko4Di2o3a9bkrx1kZcBfO62C2Dnd0ego00/TZB9/dPwiuv/M24vmHDBov2R1VVTJkyBT//+c8BAK+//jq22WYbACTKGhwcxJQpU6AoCnie3ywjp0ATS55njebbfL5U+/B5v+mkoBhFuWx1IGjmI06aduM1P3C7srxRW9WMmLxAiQkAONH5JVt/5tHgOBbJZL2R2OvX38tOYzQg/XdFDA1lMDAwXIsoWfT0pNHX141kMg5JEkflKOAV4dAUMJst2BwWpKYOC51Iv6gLaLVqd3sQ0dfXjd7erkAal2nk5IXZs/fBSy/9FcPDwyiVSlizZo1F+8MwDE4//XSsX78euq7jrrvuMhThc+bMwWOPPQYA+N3vfkeOr/XWBXkZbwQaOSmK3mCF6zWooB0fca9ajv0xzfWlvt27mj6HgZe8t2M5kbVEUJkPMkhPS+OjM76CaXc/7llfIr/ikZrI0KzUrkLXvV+zVkHn4FWrCorFUsOk2mZz3pqjtQiHOizYR1zVJ7WQSIZh2JbO63u1poK421pGqxL3W3Tv75+EBQsWYfHis6DrGubNm4fdd9/dog264oorMH/+fFSrVey///4444wzAADnnHMOLrroInzpS1/C9ttvj0MOOQQMzwOB7taN+9S44G167c+JtCrwKBSsPkbmqSZuHkeiyIPnecvtRO3NIZ8vN63ldHUlMDKSR/mqs/Dxy+8hOSWF7EcZX8/BTFDmqMkMe3oHwJLeuYGqwysVGblcwbLjJAi8sQkgijw2bvS3Xj8gEZ1ubIFTkAk4dU1TK4X1SESCJAnG2PDRwOywEI1KNdKuBCqbMOunvNZC61U0uvTrwkkGjXLI5/23YwVh01t+/DboheA+L0w8jciXF3kf2EGMuUIc8NfnRu5bj5ysam9/NSkzMbWC/k/3kMdv8iG0R08AiaBw5tGuBEWbgs0Rn1k3Q4iCfDl5nkd/f3fNJrYaiDraKRrRdRhWtLkcHJt06e6cPSIMMv0yOyyQwQsVwzY5uGGd/qJRq2q9mQsnfV/sj9FhTxYHMGyw7StMB0sOfjEmsVtjnxvnaxeLEls703nXX/h15NeNGMTkN2oyo2e7Xgy9s9H1dieCAkj9yU5QtCnY/rzvXNvdcP+Fh+TAskSE6dz/FsRYJ2c4NenaU6/69NzOgGFgqNKdHBZaFV+a0Q7BN06N4Q07mHQ6abwvGzcOQdcVpNNdnuek6nBZlnHGGadb1OFvvPEGLrroIuP60NAQ0uk0fvWrX+HFF1/ENddcA1mWMXXqVNxwww3EJYAXyCUoTAARZsfTOo5jEYtFUCiUDA9tL6va+rk4Q69UKPgfUBmLRfDe6V+2/K0dcurelhBHM4ICGlO89LQ0AGDyiodrzp5kp6pQqBfJVzwTB0DqdK7nVXWcedCIcZ2mGdb0y5+3OO3bs6d1rcCcglJvJE3TkM8XGiKI0aCnJ418vugqgbCvw6/DAnkNVN+fPz8w9yVef/21eOihh7DLLtMxe/a+OPbYE9Dd3dNwn4GBDVi0aD7uvPN+CIKIs8+ejxtvvNEQXJpRKpXwta99DZdffjlmz56NQw45BMuXL8eOO+6IZcuWoaurC/Pnz0dlzd3Qi8FNS2JiSUiHnhbY+dpB4JGTvYit67ph2dvqdN5oVALDMMhm874++DT1y1zzjTZXXwclJsA7gqLYZr/tjf9nP9iI9WcejZ0eXA1ZVoxfXaBOTAAZAgFYSUpVddOxXcb/zzxoxNA11dMvCamU37RndOxhFz3S3S0aQYzGaqRhpU3ecCfxpR+HhSAn8dbXWZcsnHnmIsydOxd/+MPz+OtfX8S///0v7L33fg33MavDATSow824/fbbsffee2P2bNI+9cQTT0AQBMiyjPXr19eHIjAcEOT2P7OZ7dY5IRIhBJPLFX1HPlTtXa0qEAR/dQ16n+GrFmLoTavBejtRkx3NCGrbz+4EzeXLWC7X62p3PUv7rBqfEM8zFlJywopnuqCqOr5xaKYh/bKnPfVaVefU4qR5WUEuV7BZn6RM7petT41ptZbl1n9nd1igLUGdAs/z2GuvvbDrrrPw9a+718W81OEU2WwWDz30EFatWmX8TRAEvPnmmzjttNPA8zwuuOAC+uABp3Xjv1vXsRWYC9gAfBOTuUFY0zQIQsz3fYavWjiqNVOYoyYvTNtnB8e/p6b1ItrXherN38Jdu91muY1GS0A9YjITk9fty9ekjf9/41BCvGZjOaoWp7UiOgevk4VapyIyMQVsJbKjGF1R2dp/V3dYiETq7gZkHcGlohR+pAR+nSFXrVqFgw8+GL29vZa/77LLLnjhhRfw4IMP4sorr8Stt95KoqYgtUkTQITZEXKyF7C7upw7s81wMpJjGKapXa25nkOJyR41xfqSiPU1btWyPIeP/t9/W3hWzaMnludco6fTX1+E+3a/DU430+fH84xj/YmhmyYuWdLyNWmomo6zv1C3LXFSiyeTcUQiEqLRiCWiGV1rjTuJ2HVEzgVt5zUEuQtofi0YJlWLoBhXF87Rwg85eanDKZ5++mmcddZZxvVKpYLnnnsOBx98MADgyCOPxH333UduDCMnb8Ridc8iv9GSm5Gcl0LcuM8Njbl69w7EXrTSRIOz9V7bAkBLJGUmKLeoiaI0OIJoXxcA4OuvLMT9e/wIAKCocEzh7PUn1ZSO8YIpmpIbb//h6rpcwkxUQL2lpFwmE1NoMZkWyZ28mvygFRKxRnZmN4E4dF2zSCaC9hA3r5dGcI0unPUpLaNpnvZT15o9ex/cddcKDA8PIxqNYs2aNbjyyistx+i6jtdffx2f+tSnjL/xPI+lS5diypQpmDlzJp588knMnDmTHM9y0IOMdjbHyElRyBfA/AZRknH6RRFFAdGofyM5+30qN55TfxxFNUipGVibtcTWe21rEJSflK5nu17E+rscz+sWPTG1EEjVqH94YyG8fiz5l+cYKE4kJjAGQTmBEpWdpACnYrLVqykoc7tmcCto0zUwDINIJIJKpRKoZML+GXTTM/kfxOD2GM2PMavDZVnBcccd06AOHxoaqu0CSsb9OI7DTTfdhO9+97tQVRWTJ082pp7oTMDkNAEK4oFLCZzcMFOpOPL5xsZc2rpCnCGdl0GV3vX7EJ1U5d4rwIgSim+/CwCQ+npQXjdgua9b1GQnJzNKG/25OjqRE4WdoGj0pOsa7pn1I9f7VSruZEBJyo2UzFGU4tDEfOkJuuc2ulmlLUlCTaBZdRQbAv4V162AYRj09XWjWq1CEAQwTHDj2Ht7u5DJ5HwTHh3EIEmiL4cFhmEweXIvhobynpsbZgShEC+9+CT0SoDvgxRDdHajrdBYYowSSx3mUN08QMBL7U2lCaTbnxTYqw9cbxCT1Ed0JHZiagfRST2ITurB0BvvNj2udyZJ50rrveUFZjAMi0XFG3BbrHHwoKbqEGrRlNxE++QWNXE154dKxfmLd+VPGQA8zj3CfX1Oc/DIFr1ZKkBIQlXVjpi3kSgbyGYLhqc7KaxH2iisW9Fqu6LTIAbzLD430uykZ5QbwrSuTZi1T/W5bn5rHDp4vl5gZx65EQAsxOSEdqImip5dP+lJUM1gT+/MtafS2+/izIN/hXs2zgUAlMuNn2Q7SdlTO1p/MpOUotQcPDlym9Mvtyyr+N4jZAPhgq96izGtQxAa7U+oDUvwJFX/IfMqrLc2uab9dTrN4jPvRpZKJdx4402YMWNX7Lbbp9Db29/0fM0U4gDwwx/+EA8//LAxI+6YY47BXnvt1aAc32mnnXDXXXdBZ9iA07otpH2F1pzMBv+t1DNiMQmFQhn84zcDAJhEwkJMQURNdrgRFI2aACA6udd39BTdaQeU3voPAIB/+x9QE4SchBrRyE1qSM1qT3JVcyQiSlIAUC43RhitkBTg9OXkkErFIQikBzDI1ppmhXYnyYR5ck2z4n6QIkw7aWqaimw2i2XLlmFoaAh77rkXbr31dscNHbN/OFWIm/3DAeC1117DjTfeaCmIA8Djjz8OoK4cP/fccwEAOidA5wK0aubGv32lIwrxxr/piMUkqKruw4CNgMoESE2jZCEm5aOPgl00SEpnh58Iyo2gzNFTz/6fttxWevtdnLX9Dbg9UU/vBNNuXLHUSNx8jWzMJCVXvaOlSkVtOq7oyh+Tj8ClJ7X2wVZVtbYNTya61C2Dzbte7dWJ/DboOkkmGodjmtPQzokwWZbDRRddgu7uFP74x5cwODjo+jz8KMRfe+01rFy5Eu+//z723ntvfPvb37YUx6lyfPfddwcA6GzAkdOW0PjL85xhBWK3TXGDWSZAUwamNsXCTkxOUdNoUrogYSel6E47APGa5iubwVl5K0HRgEOsGdpVHZqKeY6BLOtGGmeGmaScak/29KtcrEcX7ZCU2VitbhlccKwTtTLOqV0Cca4RiUYaSv/Wqck1RJfHYOedp2Pnnd2P81KIFwoF7Lrrrvj2t7+NqVOn4qKLLsJtt92G8847D4CzclxnOGgB7rCxE2C3rqPkRJXb5EPjL9S3G8klElGIv73T131jnyR9RtHaF2DkH/9ub+EmmKMnc0pnhlP01LVPLRx3+xKk0kA2A1EwpWy2l8hOUubUj69Nx3AiKbeiOFCPSkoFZ1eB1kmq8fk51YkkyVwnctcSBSXAtE+uoUM67f137cy/c4NfF0wvhXg8HrcM1zz99NNxySWXGOTkpBzXWB5akKkYuxmKMIFGH/FIRPS1U+Jkp6L+ut76YY+a2IhkEJITumZYu7wZgUfm9bcajnNK6cwYbYHcgkK+Hj2l0jht+HqsTJHoiTrEtqKFNJNUsejvS1bI14nJzd79OyvJF+jqBaN35Ky3kxRdBzHUd986I8Ck7qx0AKh9/l0QI+FbccFsphD/6KOP8MILL2DevHkACJnxJsW2XTkOBL9bF2iK2CYCTyzrPtqa4SPuZTvLMMS1gDgQFI0PR/TZuxuO5fv7jYsTdJdfQabmU53ebSekd9up1aeFrb/4uaa3RyfXf8WMqAnwtX99Wt+vLNdFwURUptROEBhLbcqMalUziKoZzMQEAJpOLmaUS3V2/M5K3SAqO9qJcmidaGQkhw0bhpDNkhQ8lUpg0qQepNOEuIP2V7cTB01BN24cweDgCKrVKkRRRF9fV9t+4n7JyewfXi6XG/zDI5EIvve97+H999+Hruv4yU9+gkMOOQSAs3IcADSGhVZL7YK5bIY1J13XUSpZ1d66DtcJLNRNwG4kZyGmaBzIjlgIScuNbgR2eredHKOoZpA+sQ0q773f9BgLMbnBFj0BhKDuHiQ7ePSlYxhAktgGcaZ5h88eLfE865jqAUBmuATeZeSPppMoykxMZjhHUqOPcmidiLa1RCISRFFAX19XWwptNzQjDq/JNX5V8353A/0oxK+44gp84xvfgCzL2GuvvXDaacRbyUk5DtTSOjZARf8ESOsCV4gDjT2DkiSAZVmLpxHgPqI8+uzdhJAosiMNj+FETl5RkxuqA95ygMh2nwQAT3KStv2E8w0N8up6M7TSNxUA8OORukKy6KB/MpNUqVR/vZwK53aCygzXJQNuBJUdKgAAIjHR8fZysR51ff8cou+RZbnhfR0NOI5Fd3cag4PDDQrtdt0v7edtBU6qebepwolEFIIgIpttzcwuCIX4hv/8A6ocnDspJ4iYtMOMwM7XDsYkdrOndVQmIAg8stmilZj+8pCVmDoMtqsHkZ2ap3mUmAASPblBmrU7kOry98CF+o4iP/ghAOCkrvruSyzCIBaxRpuSxEKSWAsxAfXCuRk8z4LnWWSGSxZiAgDFYXOCEhNgJSG3v/3vLRUsuHKjr5mEraEejdGdNzreijQt8+jp6UJfXxeSybjv1KtdoShVzWezeQwMDGNkJAtF0RCLRdDf342ennRtN1BHPp/39Rhr1qzGSSd9DcceexQefvgh1+PWrl1rjBsHyC7dmWeeicMPPxwnnngiBgbqO9Uaw0JjueAuEyCtGyNyqpdeSE2KmMTnckXvN9Nn1DRaeBHUqOFTu2MmKKBOUuWyZlwiEQ6RiDX6EUW2gaSGNxZdycNMUGZioigXqwYhOZEVxfylG3HOsgLOWVYY9cw3wL2ORUkik8ljYGAIIyM5aJqGRCKGSZN60N2dQiwWAefqaRRMoZ1MFC5heDiLDRuGDDHo448/ggMP/By+/vWTcNddK4zR4XZQAeZtt92Be+55AL/85aP4978bd5UHBwdx/fXXW/528803Y/bs2XjyySfxta99zWj6BQCV4QO/jDfGNHISRd4YnumUCkT/YvsVcSAm18doI6Vju6y7dH4Jyil6kmbtXr8yiugJAOZu9xokEahUdeMSjzW+VXaCAghJDW8sYnhjvWfRD0G5oRkx2fGNa0ZwzrKCiSja+Xj5IxFFUS1RVbFYBs/z6OlJoa+vG6mUdWhoJwZ1AvXC+pe+9GXcfvsKzJ49G88//xz+9Kc/OB5vFmBGo1EceOBBWL16dcNxS5YsabDtXbt2LY44gqT+c+fOxe9//3vIMqnF6QEXxPUJEDmNWW8dx7GIRCTXqSsNxOSCTkRNZkR22gnlt+qFcnNKZ4af4ngDPL4hpcQkRPMb0DX8Dg6ZCvzmQ+LVk82T+1CCKhTrrx8lqHJZxfBwTVcU4VG1taywrLMGZ91/BxBLuruNFjL1qCqetqbbissPwsnf+Ri6ruOBG7YxnA38+kW1QyJ2TZPT0FDzoNdOgOd57LPPPpg9e2+cfrq7h71dgNnb24d33vmX5Zj77rsPM2bMwB577GH5+4YNG9Bf2xTieR6JRAIjIyPo7++HyvJQA+QTZgIUxDuyAnMaR2aPSQAYY9vYDi9iYo78pmVMOYX06LLRLtURdoLygiVqokh1eUZ+5Z2Jab0sRB1vP2Tqa/jNhzORSpAX04mk8nlngvBDUIMfkeGhxVzRkaDMxGS+Hk/HXYkJqIsMj7+wTt4/vnaqraXEuagdxKTjxlmAAqLRaAdmAVrBst4+8F4CzH/9619Ys2YN7rnnHqxbt87HY5LPgqaz0PTgtEmavplHTvVBBTKEJqOSS/scY/zfrhAXRR48nD+wla+cb7JfUVEsViA+dJ1xeyspnR2RnXYClOa/9KOJnigxAYAglwyCMkdPI93bYdbWw3h7KI2NGRZiTUIwklWNU5mRTErI5eok3oygKDFRuBGUEwqZAqSY5Hib25f9pIvraes9V05BPB51jaqCJAw6NJQWxPP5AiRJRDTafFJLO2jHonfjxkGLAHP16tUYGBjA0UcfDVmWsWHDBpxwwgn46U9/ikmTJmFwcBBTpkyBoijI5/OGa4EGFlqA4lVtbCo+TdExciK9VUQmoKqar2IpdS0wp372UVNmOGmkqscQSwmOY8H/4oa216/1bw32Y2/7XvaAQ4Fhl184h+iptPNsMB4f4FJiEgoSceScon0I9AAA0UNtzLDoSnEGQSWT5C3M5ZTadX8E5YRijtSpYslYQ9RkhiIrUDL1c9rTPS+ceil5vW67OG3RE3Uy/aKTeJ1ba0R0dSUso9grlWpbDcutWvSuXftbXHttvbC9ePFiLF68GADwwQcf4OSTT8ZPf/pTAMCcOXPw2GOPYeHChXjiiScwe/Zs40df0zm04G/nCTbAKKxddIScEokIWJY1BhUAzcN12u6i62hwLXBTl5vJz8miQ9d1SKdchmy20HL6p/VvTf7daltfBNUqdIZpIChz9GSGwooGQX2cT6I3XScoABaS8kNQ69+r7yIxHiTlF5TIuFpjtVtUZceia8nkGE3R8H8vSddm4EVqIszulmpV3nCOauoNy3BorVENovLTf+c3cjILMI844ssNAkw3nHPOObjooovwpS99CclkEsuW1T/Xm2Pk1BERZiTCN4TIdrtdCo5jkUhEUanIhgOjGTzPIRIRkc/XtTrxOCE/2h7jBJLuxZDNFhCPR6D8+Kr6bR4pHSUn43gXgpI/uavxf8EtegKM6KlkSuUANBCUnZxo9MRrVQwwU6BqHD7OJzGQIb8pw1ly/1iEwYfr6q+dVvsJNRMUALz/L+sa3chpZANJ+dJ9jX7qbrUmronjQzOy0mxi0TuW9kGSJBQKxdp0YzEQb3MqM8jl3CNCO6hSXJJEX/13kyb1IJstQXawSW6GIESYb7wzCNmlM6AdCDyLXbfr8z6wg+hI5FStKr7sYOqDCsqurgXmyInOwlNVopFqBpoOplIxUlf4yvkA2iuiBxFB2YnJCfboKV4ZRkHqhsKKgA5wrIoIr6I/DQxkeHSnGAxndRTLOrq7BAyP1Ga1cQyGB+uvz/Ag+VGIpWIommyRdU13JSgAyAwOOxJUKxBEoYGA2FoPoP3vADD/snpk98OLU4b9CVVpx+Mxh2kt3mhHhGltrWk0tjOPQKePMV5QdQ6KHtzjsxOgIN6RyMlpyEE6HbekeX6GG5BzsUgkIigWK449eG7geQ6JRBSlUqXh+Ojv7nK9nz1qMtZhIydz1EThFj1lt9kDguz8i+03egKAYZ1EfIPFJAZyIpJRtSGKAmCQlBNBFR08280ERaMmO9J93S1HTUKbokz74/zou9ZIl3qbtxJVJRIx6LpuTAUeLWhUJYoCOI7FxRdfgr6+XnzqU/tgt912b7oBZLboPeaYE7Bw4RmW259++mnceuut0HUd06ZNw7XXXot0Oo0XX3wR11xzDWRZxtSpU3H99dcjnSa1yFf/k0E1wMhJ5FnsvkPa9/GrVq3C8uXLIcsyTj31VF+2w/Zj7BgzcqITWHSd9CBpmubLfI5liWMBANf6kh20bsAwjGMqCbgTlBs5AVaCciInwJmgstsQvYoTQTkVx1shKAAYypLUzomggDpJeRGUGzFRxNPO6YcTOQVFTGbYSQqo974RohBdo6pkMgZNC46c7Pj1r3+JZ555Gi+//DKSySR+/vNViEYba4gDAxuwaNF8w6J34cLT8YMf3GxY9ObzeXzhC1/Aww8/jMmTJ+OWW25BLpfDkiVLcMghh2D58uXYcccdsWzZMrAsi29961sAgJf/k0W1idVzqxAFBnvukPI+EMD69etx/PHH45FHHoEoijjuuONw4403WmyHFy5ciLPOOqvBTaEZxlBppYPnOUSjku/oByA7eAzDIJMp+ArLYzGpVlsoIpVy30UqHXh6A0E1IyagvfSOEpMb/BTHC0giDvJDoWocOFZFXywHgJKFiKGsDqnWrzucUSFJ5NdhZLiCSExAuSijuy+B4cF8Q3oHkBSvGTRFRW7jSMPfu0xWMRTlIknFeI+GaztUVQXDMq5rWXBZvZds5VIiRnSbGGN2FKhUqmAYFroeoMe2DXPnfhlnnHE63nnnQ7z77n8RiUQcj7Nb9FKFOFWDy7KMyy+/HJMnTwZARo9Tx8snnngCgiBAlmWsX78eu+yyi3FeVWehBCjB4Gop4scff9ywg5pKpYwICABeeOEF7Lfffujq6gLQnu2wE8YwsSQDDgqFsi9iohETfb29iMnsCZXLNc7Is0MQOIhHfdP36im0rbZ1jZoAQO6eYvzfTkyy4H/LfYP0CeMCEIIaVMkXUtU4jJTj4GsWGf3JKnpSQE/t89Kd5tCdJtFMV7eErm4JkZiASExAdx9xQ4ilrJqmoY83QJVb3xUrF0sNl2icnFuRFePiBfMXgGGZhlqY/Quy4LIBC1lR2NtayuWyoZ2Lx6MtNQu3AirPSCbTmDlzlmv9yUkhvn79euN6d3e3MW68XC5jxYoVxnVBEPDmm29izpw5+POf/4wvfelLxv2ICDPYCwCceOKJOOiggyyXe++91/KczMp1gNgOm5+T2Xb40UcfRTabxW233QYvdEwhbkY8HgHDMCgW/aVldv2SKCaaHu+240eL4vb1UBlCPl+CeuDpAJrXoToNe/S0LrkjOKhQbVqTKFdGSSW/yEmxjFw1Ap7VMFySEBF1DOVY9KR0DNU6fLrTHIYz5PXu6ia/UiMAtvpEN9548W10TerGB2++a3kMVZbB2eolblOMIw6iTUpMZnhFUG7aJkpQWpPduQWXDUCtkd9d11inPdujKvrZs0dV7ViwNKy1yRAJ+5rc7mtGLpfDokWLMH36dHzlK18x/r7LLrvghRdewIMPPojzzjsPDz74IABA1ZlAdU5qLXL6yU9+4hg5mTFa22E3dDRyYlkGqRSJfvyqb8lctIjvCIs2ExeLFQcpgg7Y1OVER0OsWswfyFKNpLyQ790OpcTkpsfI3VNc0zm36OmjxM5Yl9wR65L1PJ1jGr+0Ua5ep0uK5P/d0Qq6oxX0JDVERB0RiUFEYlAu64hKLKISC7mqQa5qiMcFxOMCdtrjkxjZMIxEd2PR008EFRQxAcTf28lNQFM1g5hYlnV0x1RNUdnpl3yM0y/52PExGIZxjarasWBxOr9fi96hobp/mF0hDsBQhU+fPt1wHqhUKnj66aeNY4488ki8+eabxnVVY6EEeFE18lpvtdVWmDZtmuViJ6fJkydbXBicbId/8YtfGNfttsNu6FjNSRA4xGIRY3hmLBbxdA2hnex+0jLAWVFuhjlyYhjGKMR7yRD8oJSYjGh+vevtRTGFWHX0Tcoc0xhBmUEjKICQ1HBJQk+Svha1ulNGQypZf6uzOQXpLgmz5+yCzAjRQv1t7SuW86qyjJXLtkVEd36tFNb6JT7/xkYiFaTGL7qTfIDCTFCyi0TATFByxdkx4fRLPjZSyfu+RxwkqEKcorFWxUOSBFNU1ZqxnV8XTC+FuKqqWLhwIQ4//HAsWrTI+DvP81i6dCmmTJmCmTNn4sknn8Ree+1l3K5pgKYFJyVoxUb9M5/5DG699VYMDQ0hGo1izZo1uPLKK43bqe3wvvvui2nTpllsh5uhI7t1ksQjkZAswzNjMQmKorkOO2y2g2eXIVCzOgAoFEruA05SMeTzZTAMmgo9LWt/rdG+giLfu53luhs5bewh1ivNyMm8c/dxvG7VwjLOnwo7QZXUCIqK1bGyULVez5QIIeVLtZ68TP3cqgZks/X34vgDhiEw9etupMTo1vXJnLWoSUnKiZjscCMqxWLx3PjmKrbIzl5Ad6pxrbpjNxQKJZ/OCP52AM2gwxKGHLyx7FizZjXuv/8uQyF+7rlnGwrxdevW4Zvf/Kal2D1z5kxcffXVhpRAVVVMnjwZV1xxBaZMITXOta/LKAVnhImoCHxuN/9R5KpVq3D77bdDlmXMmzcPCxYssKjen3rqKdx6662G7fDSpUshis6OqxQdkxJwnDXMjUYlaJrWkKq5eYibQWUImqbXdE9kpp2XPWwyGUO1KiMSEY1G4ubrJkX18vMPO95uJyfAmaAoOQHeBGUmJmMdDgRlJqesXE8NZc05qnIjqkyeEFV/l4Ztu0YAAAnO+oXyS0xm2Enquyvr192iIDMoUXm5HdiJyXK7pvsqvtNoyi9oVGW3C65UqkbEHolISCRiGB5uPSoPQiH+zGtK4OR00MzxtU3pCDkBjT7ikQj5spgjF6pH8tIv0QiI45haqlhBter9IUzXGlL9jD83OyjwPI/qHx+x3O5ETBRmgjITE4UbQbmRE+BMUMNVd92JnaTchN9R3kroZmJSdQ4i4074otZcl1ZiE5B166/ttXda39dmREWJxSui0l1I0hJxOZQF7IX9H9/0Sde1uMEpqvrww4/x5z//Gfvvvx8ikebCRbsA8+ijj7GQ0xtvvIElS5Ygn89j9uzZWLp0KTKZDE4/vV4TzeVyGB4ext/+Vnc3+M2rauDkdMju49v8O2bkJEkiWJYxoh1aXyoUmivEARIBKYoKUeR9EQ1Adgh5nkexWPIc6ElJMpcrQVUVQ/kr/+lR45hm5ATUCaoVcvpI2BYi41JbsZFTM2Iyw60+ZSYlvbYTE+MaBYnNyMk4xoGkSmx9R9VOUJlqArfdn2m4DyUqt2jHTFJOx5hJyu0clKTcdhwp2iEqgERV//rXGzj//P/Fxo0bscMOO+KUU+bj858/uOFYJwHm5ZdfjX32qW+ezJ07F1dddRX23HNPXHLJJZg5cyZOOOEE43ZN03DKKafgmGOOMVwxAWD1yxpaMC31REwEvrDn+LawjOGj62AYsx4JvgvfLMtAELiGHTa3Y4k+Soeqqp5FymhUgiQJyGYLUFW5NiaojEwmh/Ju3kU7M5yICSDFcTs+ErYFAFR157zebPbll5g0nQUDveES4arQdca4AO0TEwBU2QiqbF1kaCYmAJbaFQCkxTwWfb0xohBEAQzDuKrJWZ4Fy7OuxMMwLBFXNvkMEVGn1rSHUFVVHL/4Pzh+8X9cj3GDoijYfvudsHr1U3jwwZ/hwAMPhuCyQ+lk0bt27TPG7R9++CHK5TL23HNPAMBXv/rVBgvfhx9+GNFo1EJMAKDqpI4Y2KUDlsatYsySStLAyyKViqFclg1L1Wagjb4AUCxWPLdq7fUrqq9ygrmons0WoGlaw/k1TUdx14OhbfD+0L7Xs5eh4g4Kms4iIzfXeJmPdYLENUZmoyEmM6psBCU9BhGN7yUlKBpFUYK66+ESysVqA+FQgjKnfTRyMssR7PfTahocutNn1+SYIyZKUGYysx9vJqgHfuA8ft4JHMdh111nYNo09/s4CTD/8Y/Xjet2MWN/f79FzKiqKpYvX47ly5c3nFvTGIxSqmU7X3DnahdjFjnxPAdB4Gr6JW9i4nnOKGgriurZ8S2K/vVRNLqi7gZOxGQ599Y7e64XICpuN5ijJxo1UbhFT4OVtGvB24zRElO7KOlE11TV3XddzFFUWszj9KOj+MpRWyGaiCCaaGzxEETB0cmAghd48AIPTVUNYjKDaqY4jnNN5agC3a12RdFKJOVH5+R0u9n4z0vM+Nxzz2G77baz7ORRKGrwl/FGx8jJ/DrHYhJ4nq95O3s/azvRNHPDBEhqRmpG1hl4TvejpFcuyygWS97EJApIpfxFL14oiqkGYqKwE9RgpZ4GNSOoiipA1jjjQuFETE4oqDFUdBE5bXQ7Rl4ENVhOY7BMyFbkdCw4hmxW2ElKlVWosmqQkJOIU5EVsBwH1mUMFG2ZYVgWjIt3DyU2lmWMix2tRE5+dE5OAsy+vnqkZBczDgwMWMSMTz/9NL74xS86nlvTmMAv442ORk7mfrdCoezL7yYWkxCJCDaiaVR6k/MT/RIRbhYc6lfW+1HSy+VKqFQqtW1g909UNBpBPB5FJpMHO8n9gzogTjX+3yx6AoCS4t7sSAnKTEwUTgRVURsjLkpSZVVsuOhgUFBjlosZOS1puWRV91oXjZqs628kqMFKNwYrVk+oVKSKjcUovjqXpDiKrEKQBFdtlJmk7GkdJSlKVE71KUpSDMu6RlwALCT12MoZjop0N7Csd+Q0e/Y+eOmlv2J4mKjT1679Lfbdd3/j9qlTp0KSJLz00ktkDY89hgMOOMC4/eWXX8bs2c6+YFrANScfpeCOo2PkxPMcUqlYbWprGV6zyKgQk2EYZLPWQrlTBMSyrGH6RRwxG89pvh+NrkhRXXFUlJuRSMQgigJGRnJGXcKJoMzEROFGUB/LUxz/boYTMVGYCcqJmChoQ7Af8KzLLletcJ5VU01Jyg4zQZlJSWBt/VkRktp/dW4fkl11ooskIsbFDEo6zdphdE1vMlST1J9oEb0ZHl2xa80quAu9vV1IJGKuRW6KVi16Tz31BBxyyGGYMWMmFixYgL///e8AgGXLluHaa6/F4YcfjlKphJNPPtm4//vvv2+ILu3YHNO6jkkJurtjKJUqRvRjts21g+NYxONRVKvOCm67RooWvr30TvR+fM1viAg5m6dxDMMglUrU2lyc1b7mArkTOVGYC+R2YrLrjYzzFVOQeG8Nl1udyY2Y3NI8L3KyI8VlHaMmO7JV5x5CewSYLZP3qKoyeOb3WeRGGkWM+WFnTy5zlKS67OjRH5ZmUgJz7emBH2wPQIeukx83nucRiUgWq17aLGz+Ae3tTaNcVlAue793dgQhwvzJs0DO2x7NN5IR4MQ5wZ2vHXQscspmves/AHEUpI6VXq0lQL0xOJ8vewoxGYYc77fwzXEsurqSkGW5qdc0jaCaEZMXnNK7gSKJUCqK+690URZQlAWUFQ5lZXQiOTdiaoasmkKm2rwGN1hKoaq6uGO6RFAip2OvT3UhEhMRiYmQK7JxkWKSow+536Zi3SNKppHUz364E1iWA8NwYFkWHMdC0xQUCgUMDQ1jYGAYlUoVkYiIvr5u9PSkEY9HMTIyglwu56u3Ttd1/PCHN+OEE47GSSd9Da+++rLrsfl8HnPnzsUHH3xg/O2FF17AEUccgUMPPRQ33XST8fdQSjAKOE1RiUSI2NFLWKnrpG2llcZgajqmKAqKxTJ0vTkxCQKPZDKOQqHkT+YwaQdgpPlPFTWJc0vnSopkRFCUmCgqCu8rgqIEFeHVlqMmN7hFTQBQVEi6lakmkBadIxqKqspB5BwaglnVsYY2uUvFvvv24d9v13cUc8P1HwlKUJXagFVz5MTViMoeQdF2FycZgRk/+2Fdo0Y+p/Q1YMAweo14dFSrZVQqZeg68VeKRER8//vX4/nnn8fuu++B/ff/LI455gTXrvu1a5/Bf//7Dn7845/jgw/exwUXnIM5c55qOP6VV17BkiVL8O677xp/K5fLuOSSS3D//fdjq622wllnnYU//elP2G+//aAoOkY5ds8Ccq7xLYqPyW6dExKJKHieOFZ6CSt1nVij+BVu0sI3kRQwnsXKSISY1mezBV/E1AreKXn3cdmJicIeQRVl9zpTscojWxYtF6D1dK4VOEVQgyXrc3GLoMyg0RNACGrH7aPYdvsu9PbHkewmFzOoONMJnMAbROXUh0dlBGZR5oO3um92MAwDhmFrli3WqEpVZeTzBVx22VLccccd2HXX3fDcc2uxfr37JJ4//vF5HHTQoWBZFp/4xLaYMmUrSxsKxUMPPYTLLrvMslv36quvYtttt8U222wDnudxxBFH4NlnnwVAdEmBFsQngM5pzDv76IReWVY9G3cBkmpFo2LN/9k7qaY2KsTFQDMsWxmGQbVKusrNDcDxeBSCQArfXkVyOyZ3RbDeI3oCgLIiIMI7k0Sz3TugTlBqG5M1smURPOdMaAz0mtWvFX6iJjPajaDs0VMqUjUIFQCm9PNYB6BYFJHZWIAUJbflTcM+xWj9+KqpsUw1iufkubs1CzMsgwdu2b7p2hvu4xBVpVJJ7L///pgxYw/PsVBOQkynsePUx8kMJ8fJ559/HgARDGsBFrHJV2F8I6cxJSdd15FMxgyPJy8IAo9YTKqNmvJ+oeLxKBgGyGbzRn2pWCyhWCyB49ja0EYiPZBlBSzLQtd1ZDK5tkdgNyOoDaUu4/9uBDVYJMXlCO/+ycpVyJcsJrr0oLloUnjO+TkxtZ3TwWJjIbY32pxonEAJyh41eYES1PtDdb/0dEzF5C4Vf/83ISigXnzPbCwgQZu5bROJKVGVHIY3uJFUq8RkB8MwEEUByWQChULF17w6ZyGmvwSmmUhTUXT4MGTwjSBTxHYxZuQkSWJN7+TdiAuQVEuSSD2KYRhj580JNBpTFBX5vHN9SVU1lEoVlEoVcBxrCCsZhkMyGa/NH6t6poxOcCIoMzFR2AmKEhO5jWtKUABJ3QArSQUpllM0FusLhGAmx1szyvMqkjtFT++PNO7ofThIoqmeLmDDoAqWYSCKnGEzDDiTlFzrCuBrWinFoUuAktT9y1qzTHGDKPIGMZVK7j+2d9zxI/zhD78HABQKeU8nTDc4OU729pIBE5oebCo2EXROY0JOtJCtaZqvLz+d6EsN5vgm02Q5jkMiETFcDb0K30QhnkC5XDbSSlEUalFVCqqqGUTViq+03xSPwkxMFE4ERaMmM4pV3jWKonCLmvzCTlJOKZ0ZgwVyezriXrMzE5QTMQFAIgbka8FPTxeHoREVPT0RDA2VIYochgcKpFY5UhuBXtM1ybD5hLmQVFDERDZQEigWmxMTAMyfvxDz5y8EAPz2t0/j17/+JQ4++DB8/PFHeP/995qOIDdjjz32wDvvvIP//ve/mDZtGn71q1/hpJNOAgAoitaBgvj4WqZ0jJzM1rh0Kz/p4Dtthv34+rkad/oAGFM1CoUyZFmuEV/zVpREIoZ8vmhJK81TWwWBt7SsUGMxPx7olKCcoiaKZvUncnudoJyIiYJGUV7Rlh2MhxjWDkpSScnfRkGmLDYlqLcHExB42ygsXoes1N/fZgTV3R/H8EAByS5CbpSkIjFCjuWi9QfCTFLBEROHVIoQU7HY2k7ogQcehH/84zWccsrxAICLLroUkUgE69evx5lnnonHH3/c9b6SJOG6667DN7/5TVQqFcyZMwdz5hAxkqbqcBG+t4Ug61ftomMiTEFgkUpFLQ6XiQS57tRfRyeoODkW0Km/WVM9IRoVIQhCbfdO9SxmE4V4BNls3nXaR+OaOIiiAEkSwLKsQWJe9bK/f+wVZUiICM3XG+HVpuQEABXZStjpaK0Q3CRqciMnRXOvexSq5Bd0StK5aZhGTZa1uBDUB8PRBnKiMBMUUCcoABgaIe/Z0BAhn+GBes2JEpQZZpK6+2pvZb5f8DyHdDqJUqmKQiGYnd0gRJjfe6gCl/mxbaErAVxwTPPNmk6jY5ET7aezCjGbR0DFYtmlHmXtkaNWKMTqRPUsZlPT+kwm21JNSVVVlEoqSqUyWJaBKIo1O9Y4ZLlOVPbH333rCl79yPmNHSyQv5dltilBrc+SGltMcl6vnZiAuhWvrgN9SYdt9BajJjvW5aKuBNWwliYRlKwwrgTlBqcICoBjFAWQSKpcLE94YgoKqqJDUYIrFKnK+O7UAR0kJ1lWYW9zclKJe01QMd/PnPaR0eberSjJZBy6rmNkZHReS5qmo1yuoFyu1HZpeIiiiHg8WqtT1SPEdDqBfaNV/NnmuEGJicKNoDLF+gtXrDANBOVETBT05RjMWaMuJ7Ki8BM1UazLkZ01SlJOUROFnaA+GK7vyjkRVLP0zgxKUACMNA8Akl1xg6DuvnISqlW5rQ0OJ/A8i1QqiXJ54hETAKiaDjVAWXeQ3lDtYox1TtYIKJEgH9ZcruAh2tRrPW8x34VvlmWRTidQqRBrlCCh63qtx8pcpxLR1RWp6alIRDVrK9UzxbMTlJmYKIqVmnulSxTlBxuydbKanBr9l8tvFEUJykxMzWAnKIpT55TB8zyy2Tx+9JvGNOiG/yHPj2G6IQg8eJ5HLEYm+tD3w49djxPI7m4SlUoV+fzEIyaA7EYHSk7qZhw5NXMJaFWISaX9xWLZ9GsYXCvKaCHLimF8XygUazWyOBiGwX5xGX/6t9YQNZnhleJRFCsMuCaSGL9SLZoyAqMjqnW5KHi2/S+EV3p3+IxhACQt5zge2SzpXzvrYLvMoU68uq5b6oI8z0EURSQScbBs/YdDlmVfrxfHsUink6hWlQlLTAAhp0DTus2ZnJyg6zo4jkUk4l+ISfvvANSIqfmXWJJIqpXLFTxHQQWFSERCNBpBJlMvtheLZbAsC0kS8NnpAh57qfk5yjLbNF2joDPoElH/H8Rmmc0HQ4Q0p3Q1vhf2lM6OkTxhyr5U84jknx9GkXDZqHUiKEpKACUm1iCmVkEMDokQl2WJEJfWDRVFMcjK6XPFcYxBTLlc6zbGYwlN0ZsOLG39fOM73AAYY3LiOBaCQCao+Amx63qnAmIxCV1dqaaF6FgsAkkSkcnkWtIojQa0/SWTaWx/IcMSiPDz8zsx+O1b7grqXLH+YRAF529hqVInLztJtSlwN7BuhPwAOJGUFwaznCdB5YvwRVAH7zxi/D2RiIFlWWQywWxDaZpmqhuSxl2ib4tA03RLVMXzLNLpFGRZQS5Xxni3cniBSAmCi5y0CWBLMGbkRKx6OV+5v7nwnc0WoOvEW4m2C5DG3pjpl6+KeDwGlmUwMtJ+K0qrSCZJquCn/UXXdRy4Ywa/+3ejmZyZmACgKjMNBGUmJjMoScUjrT9np9oOJalkrDm506iJwo2gPt5Yj76aERRgJyby2mazAe6Pm6DrcEj/BIgih0MPPQif/OQn8X/+zwHYb7//g09+cnRtLmMBRVWhBBg5KROAnDoeu9UdLtlafan5LxApPsZQrSooFkuWwjcpRFeRyxUwNDSCcrkCQeDR3Z2GIPCoVmVfVsBBPKd0mhRlM5l8S2R44I6N89ucUJUZVGtpnhsx1Y8FhnOMcTGj3c2q9UMs1g+19vEYzLavKDYTEyX9ThGTExRFRbFYRrlcxcqVKzF79mw88cQTOPvsM1tuCB8PaIoW+GW80dHIqT46XEapVIUg8E0HFdBG32KxYupzc/92qapWG5xZhqIoxo4ZCdHJ1r5fwWUrzymVSqBabX8X0BxB2aMmO6o+6lB2UILqTrq/dk5RkxPWD7GY3GP9oNqjJjPMEZQ5aqJwip7MNaZkMm40b481WJb86CQSM7D11tvhtNO+AVVVW/ISHy9omh5oKWMiDDjooAgTSCajFitdNxEmUG/0Jf7h3opvp1YU4lde36FJpYg4r5UWlGagfXmlUhnl8ugKpAfumMEvX+32PK5QqhNMRGp87ZrtKQzUvvO9Xa2tzdYB4khQzeBVg3JL75JJ8n45WTl3GgwDpFJJqKqOTKYEGuE38ySfSFBkFYqPhnr/5wvsVG2jo1KCjM3Wgvbb2UEL35lMAbquegrn6O5YNpt3rF+Zd2iIVYqIRCJqtKBUKtWWd/IEQUAy2diXNxocuftwU4IyExMAlCu6haD8LmPjSP3/rRIVBU3xJHdzCAMbBlVsGAS63Wc1GAR14mc1VKsxMmeuiW97J8EwQDqdgq7rtRap8Y8aWoWmuU+Vae98gZ2qbXQ0rWtUhFu/bLQepWn1wrdX/SYeJ5MwnHbHnECsUsq1FhTW2J3hOA6yrNSIqrnmJRIREYtFXclwNDhydxLe2EnKTkwU5Qr5u1MUZYab5oUSldsoPnvUZEel2pygNgz6f30OnzGMkRGSJrMsA47jjJS5XfuaVkGIKVnz9So2NdubyFBlLdDISW2jnBA0xlznRCMn2uhbqcgolyu+W1EAIJPJtrV1bt1KZho0L2SqhlWiQOUJ7ThltgKvKMqO4QxZSzzWWA/xEuNVqjoGhsgx/T2t1VMyWfIFmNTnne4MZ1R0p5sfl0iQXdd8vmjb3k8Z6u5O1A6BeipHovxNl5gAQNXUQF8jdXOuOTmBRlKtFr5pEVqWZRQKwbSi0J2/SqVqfCmogFNVVVQqMgSBA8tyYyZPOHL3YUiSiHt+39xaplSur6VQdCcpJ1Sq1ucxMKT5JihKTACJkOwE5RQ1uRHU12YPGyO48rUGusbtfWpfEwfAGBFVEOJaSkwMw2BkZNMmJgDQA95h0zf33Tp7Wkcjp1hM8l34JsM5E7Vt3s6odO1fCqqjYlkGqqoiEpFaNp9rB9FoBJGIiBP3z+Inf2zN8paSlCS2/iUbGGqN4CicCMoJdoI6Zm9CTLSB2w2KotSm55htliPgOL6pGNcLxPc7ZSKmlu4+IaEoquO04/bPN/5kPaaRU6xmCOa38D0erSgsyyAWi6JaraJQKIHneUhS3XyuUiG/3kHXnojSnNbSdBy/7wgA4IE/d1mOM0dNTsjmybpSCYfx5dXm9x3cSMi5r7fRR8ocNZlBCcqr1mQmqFQqCUVRUSi4E5MdZptlqxi37gpRrco+fkB0JJMkYiKpnO8lTGg0G7Pe3vnGXz4xJuRkLnwDpGep6vFFGY9WFFqQNUsF6K93oVACx3GQJMFo6g0qzUgk4uA4xlHQefy+IwZBeRFTpVp/nZqRlBNKpfoHe3Cj7EhQbvBbBB/OqLjgq7rxerYLc0oOmN1LkyCz5Wgbiv19IZNSOI7FyEhxQuxIBQVN0wKtOWkOcwXHGh0nJ/Oo8VKpAlHkEI1GkEjEXJ0liUKYHdNWFOpk0EwqoKoqikXV0tRLp7n4dcm0I5VK1HaK3EWHNIq661n3vXkzMZmRzatIJTjPqMkOcxTlFjWZMTJcRVe3t85gtMTkBFlWahq3kuFeWn9fFAwMDGDDhvWYOXMmOI7DyEhhsyImAFCrClSPCditnW8zj5wEgQwfMBe+y2UV5XLV5iwZgyyTPrlIRIKmachkRmcO1wpo+tiKVMDc1NuKSyYF8adKQFXVpnUXM06fk2lKUG7I5lWUSyrSaedoyBw12TG4UYYgNP+gjgxXjX+bEdS5RwS3oeGGRvdSAc888xtce+21mDZtGvbf/7OYN+84TJ06raPrGGsEv1s3/ibiHY6cmJoDgdKwI2d3loxEyPBLAKhWyTDMdoqdrSKI9LHRJdO5OZnW2OotMFUUvYRFNpw+h/TmmUnKLWqiKNfIJ5OhTp3+U7ZCnvwa+4mKgOYEFbTpnxfo+3L88SfggAPm4Mkn1+D555/D//t/L2525NSX5gPdYetLj/m83QZ0bMABAESjnKFVaQazOVy1KhtfbkEQXPVHQYB4BXHIZltr3m0F9LmIogBV1SDLCiRJDKQFBiAk1Yycyk2ionRaaBo1AXVyorATD42a7LAfd+ZBrc3BCwqJRByiyGNkpBiIpciaNatx3313QpZlHHPMCTj66GMst7/55j/xve9dA1mWMXnyZFx66ZVIJpsPMAhiwMHmiI4mljxPSKenJ41EIgZR5GHXM0UiEpLJOLLZPCqVqqPzgCgK6O5OIZVKIBKRfE3/bQaicUn4tjsZDapVGfl8EUNDGVQqFUSjEhiGPO9YLNJ0Jp8f0EiqHdBIyg12YgLcyajd4zoJ+pkLipgGBjZg5crbcNttd+Ceex7AL3/5KN55523LMbfcsgxnnHEW7r33AWyzzbZ44IH7R/24Wyo6Sk65XAUbN+aQz1fAMCySyQR6erqRSMRRrZZx9913olwuYWQk51rrqVblGlFlUC5XwPMcurpSSKeTiEalljvGaec59YoaK9AibTabx9BQBvl8AQBRvXd3pw0pQTv4xiE5fOOQxii3WdQEAOWygmy2imy2NSIx15j8HDceUVM8HoMoCshkgiEmAHjxxb9gr71mI5VKIxqN4sADD8Latc9YjtE0DcUi+VxVKmVI0viOV9qU0fHEUtcZVCoKKhUFDKNDFHkMDq7H4sXfhCzLOO2008DzHGRZ9VTp2odfShK1SNEM/VGzuhE1qjdP+x0LOBXcnZqT4/F6c3I7O3+UoJY7DACwo1y2RkWUoFIpko45RU1mtBJBERGrt8VyUIjHY7U6YiFQX+3BwQH09vYZ13t7+/CPf7xuOebss8/Deef9D37wg+8jEolixYp7Anv8LQ1jul9IiErFZZctRTKZxsqV9yCRSCKRiKOnpxvJZBySJDT1fKKQZcVIlwqFEliWQSqVRFdXymjsNUMQeKTTSRSLxTElpmiUpG+ZjHt0SJuTR0ZyRhQZjUro6elq6TWh+MYhOVz41faIwG8Ulc2UjUszfOd4tRbt1t+b0aayzRCLRRGJiMhkioESEwDH9N9cYqhUyrjuuitxyy234fHHn8JXvjIPV111WaBr2JIwLiX5pUuvQSxGitHEOL4MUeQhSTzi8RgSCQbVqoJqtVIrhDc/n1nnYvdyqlRIIT0alcZUaQ7Y/cX9fVGcmpNJ5OXenGwH1U6dd2QRN/2ysU/PHjXZse7j2hy4pHNKYiekbKaMVLpxBNaF88qQZUCWrZNQSAM3Y6i6g3pPYrEoolGpRkzBR2n9/ZPwyit/M65v3DiIvr5+4/rbb/8HkiRhxoyZAIAvf/lo3HHHjwJfx5aCcVFaJZNJW2TDoFpVazWqAjKZIjRNQzweQ09PV60QLvqKHojdagnDw1lkswXwPGe0zYiiYIyZ6jTIlGG+JWKyg24OZLMFDA9bNwecam7UPtjsi3TekUWcd6T/NhEzWpk44hVBAfb3hrg8xGJR9PSkkUzGIYqtRYhmRKMRg5hkuTPp4+zZ++Cll/6K4eFhlMtlrF37W+y77/7G7VOnboMNG9bjvffeBQA899yzmD59RkfWsiWgo1KCICAILCSJr6U2TE2sWa3phprfl44hz2bztSiEuG2yLGvUqDoRSdHIpZPGabTmJoqCIdcQRcGIIN1w7UPN9UrFgnOdi0ZRXiREI6gL5/nXb5kjRJ7na9qwqu+JvdFoBLFYBNlsCdVqZ8WDa9asxv333wVZVnDEEV/GiSeegvPPX4z58xdi+vQZ+OMfn8ePfvRDADq6unrw7W9/B1tvPbXpOUMpgTMmPDmZIQhsLf0TwHFszSyu4vghpo26uVy+IS2krSeiKBqtJ+24Y9rRjuo7CIgij0SCpLF0GnGz5mQ3gnIjJopkUvIVIaXSkZbIyQyzpxPVhlERq9NmB6npRceEmDqFkJycsUmRkxk8z0KSuBpRcUZE9frr/8B7772LuXOP8EUQtPWEnqfdnTJakG9H9T0a0LHrpRKpU9HmZFEUwbJM0wjRTlJe5FTMlRGNe2+NX3VGa8+hGWhTr3mwarlcRT6fr0kwYps0MQEhOblhkyUnM3iegSTx+Mtf/ozzzjsXRx11FM4//4Lar63/eg8dKS6KJL2QZdmXjS+VKASl+vYL+rjFovPYdacI0Yl4r31I9EVMZriRVJDEZAcl3mefXYtvfetb+PSnP43PfvYAzJlzCPr6+rxPMEERkpMzNgtyAoCRkRF8+cuH4YQTTsI555yLaJQM8aS9bZVKa2ZxjXUQuRaFWHfKqBleoeBMEJ0CfVy/AxdoE6yZeM3NyZfe3bwSbScnwJmgOklOFILA4403XseTTz6FZ59di1mz9sB3v3tl5x+4QwjJyRmbDTkBwHvv/Ref+MS2xnWOY4xiOiEqtTbPrlWiAkRRtPT70UbeRCI25hIF2ouYyxWNbfpWYG5Orj8fkv59587G452IyQxKUmNBTNRPK58vo1xWjB+K0QxT9eqXe++9d3HDDdcgl8uht7cXl19+DVKp1pxKmyEkJ2eMv2lLgDATEwCoqo5iUcbwcBEbN+ZQLlchCAK6u9Po7qaCQO+XQNfR0O8nSaQnUNM0cBw36n4/v6gTU6EtYgKcJyfTtqAfnGsdy+JFTABQKoxNKktmFdaJCSCkNBpi8uqX03Ud3/72t3DSSafi3nsfwE477YIf//ie0T6VED4w/r4IYwRNA0olBaWSApYlv8DULI4ONPBrv8txLFiWwchIFizL1c6TMuxiK5XOtGrQQaJBj6iytwXdeHYEoiji7Bv998R1OmoSRepcUfEUkbYCc78cAKNfbrvttgdAXAai0Sj22+8zAICTTz4NudzYTyPeErHFkJMZhKhklEoyWBaGPCEalYxR5mSHq5EAYrForaE0Z4yAphEM2Vlqrd/PLySJzM7LZPIdGZNEYVbbX7uAQyTSjbOuGW56n+8v7mxzKyGmBAqFCkqlYEfRevXLffjh++jp6cVVV12Gt956E9tvvyPOO++CQNcQwhmbVVrXDjSNtHNkMiVs3JhHsVipeYkn0dOTNoZ45nI5vP32W5YhBHaQL3Wx5jrg3e/nF9RehRjijd2WOa1N/eB/467HdJqYSBqbQLEYPDEB3v1yqqrib397CfPmHYt7730QW289FbfeelPg6wjRiC2enMzQdUpUZWzcmEehUAHLsiiVCvjmNxfhhz/8IYrFki//J+qVPTxct0dJpeK1WlfUd/NrvS2js0M97aC1rWyWFPu/v1hyJKJ6c7I4qtqP8xroWLAKisXgiQkg/XJDQxuN6/Z+uZ6eXkyb9gmjDeXggw/DG2+83nCeEMEjJCcXUKLauDGH4447HrFYHN///o1IpRLo6elyNc9zgr3fD9CRSHj7OMViUUiSUJs2PHYzjOrElIeiWOs7ZpL6/mIJw8MZo3Wmuztt6oMcHVERYkqiVKp2jJgA7365WbN2x8jIMN56618AgOef/z122WV6x9YToo7NSkrQCaiqit/8ZjU+97mDEIlEDE8qSeIN1TKtLVWrVQD+v5TUx4n2+5nbaOLxel/gWE2gAUZfdKdaKlHk294g4HkO6TQhpkKh89oxr365119/DTfffANKpTImTZqESy+9At3dPYE9figlcEYg5OSlE3nrrTdx/fVXI5/PY889P4Xzz794zNwBOgsrUTEMDL2QH6sXM8xqbp7noGk6CgV/AsugEPRuIN0gkKR6c3KlIjetm/E8i1QqhUqlinx+/K1+xwIhOTlj1GmdH1/lK664FOeeewEefPAR6LqOVaseG+3DThCYrV5yyGZL0HUYVi/JZMK3URwdNaVpZPevVCrXDOfqdiKdBCWmTCY4mYJ1g6BYa4wmdbd4PNrwA0XbcbYkYgrhjlGTk5ev8rp1H6NSqWDmzFkAgC9+8Qj87ndPj/ZhJyCsnlTZbKnBk4oUjd3PQJ0UstkCyuUKMpk8hoezxjw/a/E5uJWbialTu4Fk/Hi97qbrOhIJ4uX097+/gl/+8jGUy0VUq0pITCEABEBOTjqRDRs2+L59c0W1qiKfrxrmeaqqIR6PWszzqE+crusNJnEUdcO5vK34bCa89plKksQaMY2dTIFMTq7bEq9fvx53330XPve5z+G4447Dq6++PCbrCDGxMWpy8tKJeN2+JUCWNQtRKYqGaDSK7u4urFv3EY4++ij8+c9/8rR4MbedWJ0x6S6Z1BJR1YWd7Q8UHS0YRscxx3wNa9b8Brfffjf23nvfcVlHiImHUZOTl07E6/YtDbKsoVCoYmiogD/96SWcdtqp2HPPT2GfffYxWe96E4yum8dm1fvjqIUvme/n/vZSYspmx4+YiEg1BVkmUeZuu83CmWcuwu677zmq865ZsxonnfQ1HHvsUXj44Ydcj3vhhT/ga187clSPFaJzGDU5eelEpkzZCqIoGqH66tW/NvqUtnS88cYb+MIXvoRLLlmKQoFICGhtiRIVx/mLhMzDO4vFkjHxhJwnYiGqSGT8I6b6/EAV2WwJrUgwmsHPBg0ADA1txP/9vzePqUwjRGsY9X5+f/8kLFiwCIsXn2XoRGbMmGnRiXz3u1fhhhuuQrFYxE477YJ5847zdW4vicJzz63FnXeugK7r2HrrrXHxxZcFamXRaRxxxFHG/xVFh6LIKBRkk9WLhHg81rInFe2PA+pOkl1dSWiaXnNRYMdccW6GebBpJhMcMQHejbwU1113FU47bUHN7zvEREQgYqNDD/0CDj30C5a/LVv2A+P/O+20M1auvK+lc9JfwDvvvB+CIGLhwtOx116zjQ9ZoZDHsmXX4Y477kN//yTcccePcNddK3DuueeP/gmNM6jVS7FoJioS7bTqSWVu5KUz3XQdSKcTxpipIB0OvMCyQCqVhKrqgRMT4G/w5c9//iB22WU6dtttVqCPHSJYTNj2FS+JgqIo+N//vQj9/ZMAADvssCPWr183XsvtGMyeVENDeZTLclueVNGoVGuFyWJ4OFPbFaTj0KnuqHPDLgFi2pdKpaDrOrLZIoImJsB7A+btt/+NZ5/9LU45ZQyc8UKMChNWpu31C5hOd+GAAz4HgExa/fGP78W8eceO9TLHFKqqW6xe/HpSRaOR2hTceo+edRw6ZzhMMkzwwy4BQkzpdBK6riOTKXqOnm8XXoMvf/e7ZzA4OIj580+GosgYHBzAokXzcdttd3RkPSHax4SNnPxKEPL5PM4//xzsuONOOPzwuWOxtAkB6kk1MlLCxo05lEoyeJ6MXKcNxQyj4+GHf45isdB0uGddd0SHXerGsMtEIubamOwXJGJKQtfRUWICvDdozjjjLDz44CO4556f4nvfuwV9ff0hMU1QTFhy8iNBGBwcxP/8z3zsuOPOuOiiS8d6iRMGlKjMnlSapmHp0stx9913oVQqg+P8vdWqqqFUKiOTIQJJRVEtRNVqGw0lJoZhkMmUOkpMgHWD5tRTT8AhhxxmbND885//6OhjhwgWE9aVYGBgAxYtmo8VK+5FNBrFwoWn48ILLzHm0KuqigULTsEBB3wOp546f9zWOVGxYsVtWLXqMSxfvgK77TYdgsBD1/W2B4jS+X50pLufsVkMoyOVSoFhGIyMFFtqhN6SEDb+OmPCkhPQ3Mpi/fr1WLLkQuyww07G8dOn77pFR1BmfPjhBxAEAZMmTQaAmtWLUHNQqBMVrS+1Upy2j82yj5ki0JFKJcGyLDKZoufo+C0ZITk5Y0KTU6fgpZ+ieOGFP+Cmm27Az3/+yzFeYWcRpCeVfczU+++/j0cffRRz5hyAWbNmIZMphcTkgZCcnDFhd+s6BS/9FMXmrCDWdQaViopKRQVQNogqmYwDiLfkSUX7/ehA0Ww2hz/84TksX34bJk+egnnzjsPxx5/U8ecUYvPDhC2Idwpe+ikKqiDe/FG3ehkcNHtSxU0WLf48qQAdO++8E372s4fwyCO/wrHHngBNG73A06tX7rnn1uLUU0/AKaccj4sv/l9ks/5HWoWYuNjiIqdQQdwMhKiqVRVABaLIQRQ5xOMxJBIMZFmpqcqrDhGVjmQyAZ7nkckUMWnSFBxzzAmjXtGW3CmwpWOLi5xCBbF/+PGkohFVIpGojc0qQlWDS4XDToEtF1scOXnpp8wK4gsuOMdQEG/psHtSqSrxpOrp6UJ3dwqiyGNkJFhiArzNCp06Bej1EJs2tjhyChXEowclqqGhAkZGiHleNlsKnJiAsFNgS8YWR06dVhB7FW/fe+9dnH32mTjllOPxrW+dvckXbwkxlSHLndELhJ0CWy62SJ1Tp0BV7ebi7eWXX20Ub3VdxwknHI1zzjkf++33GSxffit0XceiRYvHeeUTF1tCp0Coc3LGFrdb10l4GZ29+eY/EY1GDSfQk08+DblcftzWuynAy8xw/fr1eOutN6FpGtau/S2AsFNgc0FITgHCS6bw4Yfvo6enF1dddRneeutNbL/9jjjvvAvGY6mbFJqZGU6fPgPPPffX8VhWiA5ji6s5dRJexVtVVfG3v72EefOOxb33Poitt56KW2+9aSyXGCLEJoOQnAKEV/G2p6cX06Z9AtOnzwAAHHzwYXjjjdcbzhMiRIiQnAKFl0xh1qzdMTIyjLfe+hcA4Pnnf49ddpk+XssNEWJCI9ytCxjNbF6mT5+B119/DTfffANKpTImTZqESy+9At3dPeO97BDjiHC3zhkhOW0i8LJ5efPNf+J737sGsixj8uTJuPTSK5FMhh/6TQEhOTkjTOs2AfgZFHnLLctwxhln4d57H8A222yLBx64f5xW6w0voepbb72J+fNPxnHHfRXXXXclFCW4QQshNh2E5LQJwI/Ni6ZpKBYLAEiPmSRJ47FUT/gh2iuuuBTnnnsBHnzwEei6jlWrHhufxYYYV4TktAnAq/kVAM4++zxcd91V+PKXD8Nf//pnHHXU0WO9TF/wItp16z5GpVLBzJnEruaLXzwCv/vd0+O13BDjiJCcNgF46acqlTKuu+5K3HLLbXj88afwla/Mw1VXXTaWS/QNL6L1Q8Qhtgw0VYiHhbqJge23/wRefPFF4/0olbL4xCemGtdfffUdxONRzJlDZAtnnHEK7rzz9gn5/sViIhhGNdaWTEYQi4nG9XQ6CkHgjOvFYtxyPcSWgzBy2gTwmc98Bn/84x8xNDSEUqmENWvW4IADDjBu33bbbbFu3Tq8/Tap3TzzzDOYNWtiunhOnjwZg4ODxvUNGzZg0qRJrrcPDAxYbg+x5SAkp00AkydPxnnnnYeTTz4ZRx11FObOnYvdd98dCxYswN///nek02lce+21OPfcc3HEEUfg4YcfxjXXXDPey3aEF9FOnToVkiThpZdeAgA89thjlttDbDloqnMKEaITWLVqFW6//XbIsox58+ZhwYIFWLBgARYvXoxZs2bhn//8J5YsWYJCoYAZM2bg2muvhSiK473sEGOMkJxChAgxIRGmdSFChJiQCMkpRIgQExIhOYUIEWJCIiSnECFCTEiE5BQiRIgJiZCcQoQIMSERklOIECEmJP4/uij4BsdHgT0AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/mhjensen/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_183_0.png" } }, "output_type": "display_data" } ], "source": [ "from mpl_toolkits.mplot3d import Axes3D\n", "import matplotlib.pyplot as plt\n", "from matplotlib import cm\n", "from matplotlib.ticker import LinearLocator, FormatStrFormatter\n", "import numpy as np\n", "from random import random, seed\n", "\n", "fig = plt.figure()\n", "ax = fig.gca(projection='3d')\n", "\n", "# Make data.\n", "x = np.arange(0, 1, 0.05)\n", "y = np.arange(0, 1, 0.05)\n", "x, y = np.meshgrid(x,y)\n", "\n", "\n", "def FrankeFunction(x,y):\n", " term1 = 0.75*np.exp(-(0.25*(9*x-2)**2) - 0.25*((9*y-2)**2))\n", " term2 = 0.75*np.exp(-((9*x+1)**2)/49.0 - 0.1*(9*y+1))\n", " term3 = 0.5*np.exp(-(9*x-7)**2/4.0 - 0.25*((9*y-3)**2))\n", " term4 = -0.2*np.exp(-(9*x-4)**2 - (9*y-7)**2)\n", " return term1 + term2 + term3 + term4\n", "\n", "\n", "z = FrankeFunction(x, y)\n", "\n", "# Plot the surface.\n", "surf = ax.plot_surface(x, y, z, cmap=cm.coolwarm,\n", " linewidth=0, antialiased=False)\n", "\n", "# Customize the z axis.\n", "ax.set_zlim(-0.10, 1.40)\n", "ax.zaxis.set_major_locator(LinearLocator(10))\n", "ax.zaxis.set_major_formatter(FormatStrFormatter('%.02f'))\n", "\n", "# Add a color bar which maps values to colors.\n", "fig.colorbar(surf, shrink=0.5, aspect=5)\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise: Ordinary Least Square (OLS) on the Franke function\n", "\n", "We will generate our own dataset for a function\n", "$\\mathrm{FrankeFunction}(x,y)$ with $x,y \\in [0,1]$. The function\n", "$f(x,y)$ is the Franke function. You should explore also the addition\n", "of an added stochastic noise to this function using the normal\n", "distribution $N(0,1)$.\n", "\n", "*Write your own code* (using either a matrix inversion or a singular\n", "value decomposition from e.g., **numpy** ) or use your code from\n", "homeworks 1 and 2 and perform a standard least square regression\n", "analysis using polynomials in $x$ and $y$ up to fifth order. Find the\n", "[confidence intervals](https://en.wikipedia.org/wiki/Confidence_interval) of the parameters (estimators) $\\beta$ by computing their\n", "variances, evaluate the Mean Squared error (MSE)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "MSE(\\hat{y},\\hat{\\tilde{y}}) = \\frac{1}{n}\n", "\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and the $R^2$ score function. If $\\tilde{\\hat{y}}_i$ is the predicted\n", "value of the $i-th$ sample and $y_i$ is the corresponding true value,\n", "then the score $R^2$ is defined as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "R^2(\\hat{y}, \\tilde{\\hat{y}}) = 1 - \\frac{\\sum_{i=0}^{n - 1} (y_i - \\tilde{y}_i)^2}{\\sum_{i=0}^{n - 1} (y_i - \\bar{y})^2},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where we have defined the mean value of $\\hat{y}$ as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\bar{y} = \\frac{1}{n} \\sum_{i=0}^{n - 1} y_i.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Your code has to include a scaling of the data (for example by\n", "subtracting the mean value), and\n", "a split of the data in training and test data. For this exercise you can\n", "either write your own code or use for example the function for\n", "splitting training data provided by the library **Scikit-Learn** (make\n", "sure you have installed it). This function is called\n", "$train\\_test\\_split$. **You should present a critical discussion of why and how you have scaled or not scaled the data**.\n", "\n", "It is normal in essentially all Machine Learning studies to split the\n", "data in a training set and a test set (eventually also an additional\n", "validation set). There\n", "is no explicit recipe for how much data should be included as training\n", "data and say test data. An accepted rule of thumb is to use\n", "approximately $2/3$ to $4/5$ of the data as training data.\n", "\n", "\n", "You can easily reuse the solutions to your exercises from week 35 and week 36.\n", "\n", "\n", "\n", "### Exercise: Bias-variance trade-off and resampling techniques\n", "\n", "Our aim here is to study the bias-variance trade-off by implementing the **bootstrap** resampling technique.\n", "\n", "With a code which does OLS and includes resampling techniques, \n", "we will now discuss the bias-variance trade-off in the context of\n", "continuous predictions such as regression. However, many of the\n", "intuitions and ideas discussed here also carry over to classification\n", "tasks and basically all Machine Learning algorithms. \n", "\n", "Before you perform an analysis of the bias-variance trade-off on your test data, make\n", "first a figure similar to Fig. 2.11 of Hastie, Tibshirani, and\n", "Friedman. Figure 2.11 of this reference displays only the test and training MSEs. The test MSE can be used to \n", "indicate possible regions of low/high bias and variance. You will most likely not get an\n", "equally smooth curve!\n", "\n", "With this result we move on to the bias-variance trade-off analysis.\n", "\n", "Consider a\n", "dataset $\\mathcal{L}$ consisting of the data\n", "$\\mathbf{X}_\\mathcal{L}=\\{(y_j, \\boldsymbol{x}_j), j=0\\ldots n-1\\}$.\n", "\n", "Let us assume that the true data is generated from a noisy model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{y}=f(\\boldsymbol{x}) + \\boldsymbol{\\epsilon}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here $\\epsilon$ is normally distributed with mean zero and standard\n", "deviation $\\sigma^2$.\n", "\n", "In our derivation of the ordinary least squares method we defined then\n", "an approximation to the function $f$ in terms of the parameters\n", "$\\boldsymbol{\\beta}$ and the design matrix $\\boldsymbol{X}$ which embody our model,\n", "that is $\\boldsymbol{\\tilde{y}}=\\boldsymbol{X}\\boldsymbol{\\beta}$.\n", "\n", "The parameters $\\boldsymbol{\\beta}$ are in turn found by optimizing the means\n", "squared error via the so-called cost function" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "C(\\boldsymbol{X},\\boldsymbol{\\beta}) =\\frac{1}{n}\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2=\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right].\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here the expected value $\\mathbb{E}$ is the sample value. \n", "\n", "Show that you can rewrite this as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\frac{1}{n}\\sum_i(f_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2+\\frac{1}{n}\\sum_i(\\tilde{y}_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2+\\sigma^2.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Explain what the terms mean, which one is the bias and which one is\n", "the variance and discuss their interpretations.\n", "\n", "Perform then a bias-variance analysis of the Franke function by\n", "studying the MSE value as function of the complexity of your model.\n", "\n", "Discuss the bias and variance trade-off as function\n", "of your model complexity (the degree of the polynomial) and the number\n", "of data points, and possibly also your training and test data using the **bootstrap** resampling method.\n", "\n", "Note also that when you calculate the bias, in all applications you don't know the function values $f_i$. You would hence replace them with the actual data points $y_i$.\n", "\n", "\n", "### Exercise: Cross-validation as resampling techniques, adding more complexity\n", "\n", "The aim here is to write your own code for another widely popular\n", "resampling technique, the so-called cross-validation method. Again,\n", "before you start with cross-validation approach, you should scale your\n", "data.\n", "\n", "Implement the $k$-fold cross-validation algorithm (write your own\n", "code) and evaluate again the MSE function resulting\n", "from the test folds. You can compare your own code with that from\n", "**Scikit-Learn** if needed. \n", "\n", "Compare the MSE you get from your cross-validation code with the one\n", "you got from your **bootstrap** code. Comment your results. Try $5-10$\n", "folds. You can also compare your own cross-validation code with the\n", "one provided by **Scikit-Learn**.\n", "\n", "\n", "### Exercise: Ridge Regression on the Franke function with resampling\n", "\n", "Write your own code for the Ridge method, either using matrix\n", "inversion or the singular value decomposition as done in the previous\n", "exercise. Perform the same bootstrap analysis as in the\n", "Exercise 2 (for the same polynomials) and the cross-validation in exercise 3 but now for different values of $\\lambda$. Compare and\n", "analyze your results with those obtained in exercises 1-3. Study the\n", "dependence on $\\lambda$.\n", "\n", "Study also the bias-variance trade-off as function of various values of\n", "the parameter $\\lambda$. For the bias-variance trade-off, use the **bootstrap** resampling method. Comment your results. \n", "\n", "### Exercise: Lasso Regression on the Franke function with resampling\n", "\n", "This exercise is essentially a repeat of the previous two ones, but now\n", "with Lasso regression. Write either your own code (difficult and optional) or, in this case,\n", "you can also use the functionalities of **Scikit-Learn** (recommended). \n", "Give a\n", "critical discussion of the three methods and a judgement of which\n", "model fits the data best. Perform here as well an analysis of the bias-variance trade-off using the **bootstrap** resampling technique and an analysis of the mean squared error using cross-validation. \n", "\n", "### Exercise: Analysis of real data\n", "\n", "With our codes functioning and having been tested properly on a\n", "simpler function we are now ready to look at real data. We will\n", "essentially repeat in this exercise what was done in exercises 1-5. However, we\n", "need first to download the data and prepare properly the inputs to our\n", "codes. We are going to download digital terrain data from the website\n", ",\n", "\n", "Or, if you prefer, we have placed selected datafiles at \n", "\n", "In order to obtain data for a specific region, you need to register as\n", "a user (free) at this website and then decide upon which area you want\n", "to fetch the digital terrain data from. In order to be able to read\n", "the data properly, you need to specify that the format should be **SRTM\n", "Arc-Second Global** and download the data as a **GeoTIF** file. The\n", "files are then stored in *tif* format which can be imported into a\n", "Python program using" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "ename": "NameError", "evalue": "name 'scipy' is not defined", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mscipy\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmisc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mimread\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[0;31mNameError\u001b[0m: name 'scipy' is not defined" ] } ], "source": [ "scipy.misc.imread" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here is a simple part of a Python code which reads and plots the data\n", "from such files" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "\"\"\"\n", "import numpy as np\n", "from imageio import imread\n", "import matplotlib.pyplot as plt\n", "from mpl_toolkits.mplot3d import Axes3D\n", "from matplotlib import cm\n", "\n", "# Load the terrain\n", "terrain1 = imread('SRTM_data_Norway_1.tif')\n", "# Show the terrain\n", "plt.figure()\n", "plt.title('Terrain over Norway 1')\n", "plt.imshow(terrain1, cmap='gray')\n", "plt.xlabel('X')\n", "plt.ylabel('Y')\n", "plt.show()\n", "\"\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you should have problems in downloading the digital terrain data,\n", "we provide two examples under the data folder of project 1. One is\n", "from a region close to Stavanger in Norway and the other Møsvatn\n", "Austfjell, again in Norway.\n", "Feel free to produce your own terrain data.\n", "\n", "\n", "Alternatively, if you would like to use another data set, feel free to do so. This could be data close to your reseach area or simply a data set you found interesting. See for example [kaggle.com](https://www.kaggle.com/datasets) for examples.\n", "\n", "\n", "Our final part deals with the parameterization of your digital terrain\n", "data (or your own data). We will apply all three methods for linear regression, the same type (or higher order) of polynomial\n", "approximation and cross-validation as resampling technique to evaluate which\n", "model fits the data best.\n", "\n", "At the end, you should present a critical evaluation of your results\n", "and discuss the applicability of these regression methods to the type\n", "of data presented here (either the terrain data we propose or other data sets)." ] } ], "metadata": { "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 4 }