{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Resampling Methods\n", "\n", "## Introduction\n", "\n", "Resampling methods are an indispensable tool in modern\n", "statistics. They involve repeatedly drawing samples from a training\n", "set and refitting a model of interest on each sample in order to\n", "obtain additional information about the fitted model. For example, in\n", "order to estimate the variability of a linear regression fit, we can\n", "repeatedly draw different samples from the training data, fit a linear\n", "regression to each new sample, and then examine the extent to which\n", "the resulting fits differ. Such an approach may allow us to obtain\n", "information that would not be available from fitting the model only\n", "once using the original training sample.\n", "\n", "Two resampling methods are often used in Machine Learning analyses,\n", "1. The **bootstrap method**\n", "\n", "2. and **Cross-Validation**\n", "\n", "In addition there are several other methods such as the Jackknife and the Blocking methods. We will discuss in particular\n", "cross-validation and the bootstrap method. \n", "\n", "\n", "Resampling approaches can be computationally expensive, because they\n", "involve fitting the same statistical method multiple times using\n", "different subsets of the training data. However, due to recent\n", "advances in computing power, the computational requirements of\n", "resampling methods generally are not prohibitive. In this chapter, we\n", "discuss two of the most commonly used resampling methods,\n", "cross-validation and the bootstrap. Both methods are important tools\n", "in the practical application of many statistical learning\n", "procedures. For example, cross-validation can be used to estimate the\n", "test error associated with a given statistical learning method in\n", "order to evaluate its performance, or to select the appropriate level\n", "of flexibility. The process of evaluating a model’s performance is\n", "known as model assessment, whereas the process of selecting the proper\n", "level of flexibility for a model is known as model selection. The\n", "bootstrap is widely used.\n", "\n", "\n", "* Our simulations can be treated as *computer experiments*. This is particularly the case for Monte Carlo methods\n", "\n", "* The results can be analysed with the same statistical tools as we would use analysing experimental data.\n", "\n", "* As in all experiments, we are looking for expectation values and an estimate of how accurate they are, i.e., possible sources for errors.\n", "\n", "## Reminder on Statistics\n", "\n", "\n", "* As in other experiments, many numerical experiments have two classes of errors:\n", "\n", " * Statistical errors\n", "\n", " * Systematical errors\n", "\n", "\n", "* Statistical errors can be estimated using standard tools from statistics\n", "\n", "* Systematical errors are method specific and must be treated differently from case to case. \n", "\n", "The\n", "advantage of doing linear regression is that we actually end up with\n", "analytical expressions for several statistical quantities. \n", "Standard least squares and Ridge regression allow us to\n", "derive quantities like the variance and other expectation values in a\n", "rather straightforward way.\n", "\n", "\n", "It is assumed that $\\varepsilon_i\n", "\\sim \\mathcal{N}(0, \\sigma^2)$ and the $\\varepsilon_{i}$ are\n", "independent, i.e.:" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{align*} \n", "\\mbox{Cov}(\\varepsilon_{i_1},\n", "\\varepsilon_{i_2}) & = \\left\\{ \\begin{array}{lcc} \\sigma^2 & \\mbox{if}\n", "& i_1 = i_2, \\\\ 0 & \\mbox{if} & i_1 \\not= i_2. \\end{array} \\right.\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The randomness of $\\varepsilon_i$ implies that\n", "$\\mathbf{y}_i$ is also a random variable. In particular,\n", "$\\mathbf{y}_i$ is normally distributed, because $\\varepsilon_i \\sim\n", "\\mathcal{N}(0, \\sigma^2)$ and $\\mathbf{X}_{i,\\ast} \\, \\boldsymbol{\\beta}$ is a\n", "non-random scalar. To specify the parameters of the distribution of\n", "$\\mathbf{y}_i$ we need to calculate its first two moments. \n", "\n", "Recall that $\\boldsymbol{X}$ is a matrix of dimensionality $n\\times p$. The\n", "notation above $\\mathbf{X}_{i,\\ast}$ means that we are looking at the\n", "row number $i$ and perform a sum over all values $p$.\n", "\n", "\n", "The assumption we have made here can be summarized as (and this is going to be useful when we discuss the bias-variance trade off)\n", "that there exists a function $f(\\boldsymbol{x})$ and a normal distributed error $\\boldsymbol{\\varepsilon}\\sim \\mathcal{N}(0, \\sigma^2)$\n", "which describe our data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{y} = f(\\boldsymbol{x})+\\boldsymbol{\\varepsilon}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We approximate this function with our model from the solution of the linear regression equations, that is our\n", "function $f$ is approximated by $\\boldsymbol{\\tilde{y}}$ where we want to minimize $(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2$, our MSE, with" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{\\tilde{y}} = \\boldsymbol{X}\\boldsymbol{\\beta}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can calculate the expectation value of $\\boldsymbol{y}$ for a given element $i$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{align*} \n", "\\mathbb{E}(y_i) & =\n", "\\mathbb{E}(\\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta}) + \\mathbb{E}(\\varepsilon_i)\n", "\\, \\, \\, = \\, \\, \\, \\mathbf{X}_{i, \\ast} \\, \\beta, \n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "while\n", "its variance is" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{align*} \\mbox{Var}(y_i) & = \\mathbb{E} \\{ [y_i\n", "- \\mathbb{E}(y_i)]^2 \\} \\, \\, \\, = \\, \\, \\, \\mathbb{E} ( y_i^2 ) -\n", "[\\mathbb{E}(y_i)]^2 \\\\ & = \\mathbb{E} [ ( \\mathbf{X}_{i, \\ast} \\,\n", "\\beta + \\varepsilon_i )^2] - ( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta})^2 \\\\ &\n", "= \\mathbb{E} [ ( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta})^2 + 2 \\varepsilon_i\n", "\\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta} + \\varepsilon_i^2 ] - ( \\mathbf{X}_{i,\n", "\\ast} \\, \\beta)^2 \\\\ & = ( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta})^2 + 2\n", "\\mathbb{E}(\\varepsilon_i) \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta} +\n", "\\mathbb{E}(\\varepsilon_i^2 ) - ( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta})^2 \n", "\\\\ & = \\mathbb{E}(\\varepsilon_i^2 ) \\, \\, \\, = \\, \\, \\,\n", "\\mbox{Var}(\\varepsilon_i) \\, \\, \\, = \\, \\, \\, \\sigma^2. \n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Hence, $y_i \\sim \\mathcal{N}( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta}, \\sigma^2)$, that is $\\boldsymbol{y}$ follows a normal distribution with \n", "mean value $\\boldsymbol{X}\\boldsymbol{\\beta}$ and variance $\\sigma^2$ (not be confused with the singular values of the SVD). \n", "\n", "\n", "With the OLS expressions for the parameters $\\boldsymbol{\\beta}$ we can evaluate the expectation value" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}(\\boldsymbol{\\beta}) = \\mathbb{E}[ (\\mathbf{X}^{\\top} \\mathbf{X})^{-1}\\mathbf{X}^{T} \\mathbf{Y}]=(\\mathbf{X}^{T} \\mathbf{X})^{-1}\\mathbf{X}^{T} \\mathbb{E}[ \\mathbf{Y}]=(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\mathbf{X}^{T}\\mathbf{X}\\boldsymbol{\\beta}=\\boldsymbol{\\beta}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This means that the estimator of the regression parameters is unbiased.\n", "v\n", "We can also calculate the variance\n", "\n", "The variance of $\\boldsymbol{\\beta}$ is" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{eqnarray*}\n", "\\mbox{Var}(\\boldsymbol{\\beta}) & = & \\mathbb{E} \\{ [\\boldsymbol{\\beta} - \\mathbb{E}(\\boldsymbol{\\beta})] [\\boldsymbol{\\beta} - \\mathbb{E}(\\boldsymbol{\\beta})]^{T} \\}\n", "\\\\\n", "& = & \\mathbb{E} \\{ [(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y} - \\boldsymbol{\\beta}] \\, [(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y} - \\boldsymbol{\\beta}]^{T} \\}\n", "\\\\\n", "% & = & \\mathbb{E} \\{ [(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y}] \\, [(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y}]^{T} \\} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n", "% \\\\\n", "% & = & \\mathbb{E} \\{ (\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\mathbf{Y} \\, \\mathbf{Y}^{T} \\, \\mathbf{X} \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1} \\} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n", "% \\\\\n", "& = & (\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\, \\mathbb{E} \\{ \\mathbf{Y} \\, \\mathbf{Y}^{T} \\} \\, \\mathbf{X} \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n", "\\\\\n", "& = & (\\mathbf{X}^{T} \\mathbf{X})^{-1} \\, \\mathbf{X}^{T} \\, \\{ \\mathbf{X} \\, \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T} \\, \\mathbf{X}^{T} + \\sigma^2 \\} \\, \\mathbf{X} \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n", "% \\\\\n", "% & = & (\\mathbf{X}^T \\mathbf{X})^{-1} \\, \\mathbf{X}^T \\, \\mathbf{X} \\, \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^T \\, \\mathbf{X}^T \\, \\mathbf{X} \\, (\\mathbf{X}^T % \\mathbf{X})^{-1}\n", "% \\\\\n", "% & & + \\, \\, \\sigma^2 \\, (\\mathbf{X}^T \\mathbf{X})^{-1} \\, \\mathbf{X}^T \\, \\mathbf{X} \\, (\\mathbf{X}^T \\mathbf{X})^{-1} - \\boldsymbol{\\beta} \\boldsymbol{\\beta}^T\n", "\\\\\n", "& = & \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T} + \\sigma^2 \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1} - \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T}\n", "\\, \\, \\, = \\, \\, \\, \\sigma^2 \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1},\n", "\\end{eqnarray*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where we have used that $\\mathbb{E} (\\mathbf{Y} \\mathbf{Y}^{T}) =\n", "\\mathbf{X} \\, \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T} \\, \\mathbf{X}^{T} +\n", "\\sigma^2 \\, \\mathbf{I}_{nn}$. From $\\mbox{Var}(\\boldsymbol{\\beta}) = \\sigma^2\n", "\\, (\\mathbf{X}^{T} \\mathbf{X})^{-1}$, one obtains an estimate of the\n", "variance of the estimate of the $j$-th regression coefficient:\n", "$\\boldsymbol{\\sigma}^2 (\\boldsymbol{\\beta}_j ) = \\boldsymbol{\\sigma}^2 \\sqrt{\n", "[(\\mathbf{X}^{T} \\mathbf{X})^{-1}]_{jj} }$. This may be used to\n", "construct a confidence interval for the estimates.\n", "\n", "\n", "In a similar way, we can obtain analytical expressions for say the\n", "expectation values of the parameters $\\boldsymbol{\\beta}$ and their variance\n", "when we employ Ridge regression, allowing us again to define a confidence interval. \n", "\n", "It is rather straightforward to show that" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E} \\big[ \\boldsymbol{\\beta}^{\\mathrm{Ridge}} \\big]=(\\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I}_{pp})^{-1} (\\mathbf{X}^{\\top} \\mathbf{X})\\boldsymbol{\\beta}^{\\mathrm{OLS}}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see clearly that \n", "$\\mathbb{E} \\big[ \\boldsymbol{\\beta}^{\\mathrm{Ridge}} \\big] \\not= \\boldsymbol{\\beta}^{\\mathrm{OLS}}$ for any $\\lambda > 0$. We say then that the ridge estimator is biased.\n", "\n", "We can also compute the variance as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mbox{Var}[\\boldsymbol{\\beta}^{\\mathrm{Ridge}}]=\\sigma^2[ \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1} \\mathbf{X}^{T} \\mathbf{X} \\{ [ \\mathbf{X}^{\\top} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}\\}^{T},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and it is easy to see that if the parameter $\\lambda$ goes to infinity then the variance of Ridge parameters $\\boldsymbol{\\beta}$ goes to zero. \n", "\n", "With this, we can compute the difference" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mbox{Var}[\\boldsymbol{\\beta}^{\\mathrm{OLS}}]-\\mbox{Var}(\\boldsymbol{\\beta}^{\\mathrm{Ridge}})=\\sigma^2 [ \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}[ 2\\lambda\\mathbf{I} + \\lambda^2 (\\mathbf{X}^{T} \\mathbf{X})^{-1} ] \\{ [ \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}\\}^{T}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The difference is non-negative definite since each component of the\n", "matrix product is non-negative definite. \n", "This means the variance we obtain with the standard OLS will always for $\\lambda > 0$ be larger than the variance of $\\boldsymbol{\\beta}$ obtained with the Ridge estimator. This has interesting consequences when we discuss the so-called bias-variance trade-off below. \n", "\n", "\n", "\n", "## Resampling methods\n", "\n", "With all these analytical equations for both the OLS and Ridge\n", "regression, we will now outline how to assess a given model. This will\n", "lead us to a discussion of the so-called bias-variance tradeoff (see\n", "below) and so-called resampling methods.\n", "\n", "One of the quantities we have discussed as a way to measure errors is\n", "the mean-squared error (MSE), mainly used for fitting of continuous\n", "functions. Another choice is the absolute error.\n", "\n", "In the discussions below we will focus on the MSE and in particular since we will split the data into test and training data,\n", "we discuss the\n", "1. prediction error or simply the **test error** $\\mathrm{Err_{Test}}$, where we have a fixed training set and the test error is the MSE arising from the data reserved for testing. We discuss also the \n", "\n", "2. training error $\\mathrm{Err_{Train}}$, which is the average loss over the training data.\n", "\n", "As our model becomes more and more complex, more of the training data tends to used. The training may thence adapt to more complicated structures in the data. This may lead to a decrease in the bias (see below for code example) and a slight increase of the variance for the test error.\n", "For a certain level of complexity the test error will reach minimum, before starting to increase again. The\n", "training error reaches a saturation.\n", "\n", "\n", "\n", "Two famous\n", "resampling methods are the **independent bootstrap** and **the jackknife**. \n", "\n", "The jackknife is a special case of the independent bootstrap. Still, the jackknife was made\n", "popular prior to the independent bootstrap. And as the popularity of\n", "the independent bootstrap soared, new variants, such as **the dependent bootstrap**.\n", "\n", "The Jackknife and independent bootstrap work for\n", "independent, identically distributed random variables.\n", "If these conditions are not\n", "satisfied, the methods will fail. Yet, it should be said that if the data are\n", "independent, identically distributed, and we only want to estimate the\n", "variance of $\\overline{X}$ (which often is the case), then there is no\n", "need for bootstrapping. \n", "\n", "\n", "The Jackknife works by making many replicas of the estimator $\\widehat{\\beta}$. \n", "The jackknife is a resampling method where we systematically leave out one observation from the vector of observed values $\\boldsymbol{x} = (x_1,x_2,\\cdots,X_n)$. \n", "Let $\\boldsymbol{x}_i$ denote the vector" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{x}_i = (x_1,x_2,\\cdots,x_{i-1},x_{i+1},\\cdots,x_n),\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "which equals the vector $\\boldsymbol{x}$ with the exception that observation\n", "number $i$ is left out. Using this notation, define\n", "$\\widehat{\\beta}_i$ to be the estimator\n", "$\\widehat{\\beta}$ computed using $\\vec{X}_i$." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Runtime: 0.184895 sec\n", "Jackknife Statistics :\n", "original bias std. error\n", " 100.109 100.099 0.148768\n" ] } ], "source": [ "from numpy import *\n", "from numpy.random import randint, randn\n", "from time import time\n", "\n", "def jackknife(data, stat):\n", " n = len(data);t = zeros(n); inds = arange(n); t0 = time()\n", " ## 'jackknifing' by leaving out an observation for each i \n", " for i in range(n):\n", " t[i] = stat(delete(data,i) )\n", "\n", " # analysis \n", " print(\"Runtime: %g sec\" % (time()-t0)); print(\"Jackknife Statistics :\")\n", " print(\"original bias std. error\")\n", " print(\"%8g %14g %15g\" % (stat(data),(n-1)*mean(t)/n, (n*var(t))**.5))\n", "\n", " return t\n", "\n", "\n", "# Returns mean of data samples \n", "def stat(data):\n", " return mean(data)\n", "\n", "\n", "mu, sigma = 100, 15\n", "datapoints = 10000\n", "x = mu + sigma*random.randn(datapoints)\n", "# jackknife returns the data sample \n", "t = jackknife(x, stat)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Bootstrap\n", "\n", "Bootstrapping is a nonparametric approach to statistical inference\n", "that substitutes computation for more traditional distributional\n", "assumptions and asymptotic results. Bootstrapping offers a number of\n", "advantages: \n", "1. The bootstrap is quite general, although there are some cases in which it fails. \n", "\n", "2. Because it does not require distributional assumptions (such as normally distributed errors), the bootstrap can provide more accurate inferences when the data are not well behaved or when the sample size is small. \n", "\n", "3. It is possible to apply the bootstrap to statistics with sampling distributions that are difficult to derive, even asymptotically. \n", "\n", "4. It is relatively simple to apply the bootstrap to complex data-collection plans (such as stratified and clustered samples).\n", "\n", "Since $\\widehat{\\beta} = \\widehat{\\beta}(\\boldsymbol{X})$ is a function of random variables,\n", "$\\widehat{\\beta}$ itself must be a random variable. Thus it has\n", "a pdf, call this function $p(\\boldsymbol{t})$. The aim of the bootstrap is to\n", "estimate $p(\\boldsymbol{t})$ by the relative frequency of\n", "$\\widehat{\\beta}$. You can think of this as using a histogram\n", "in the place of $p(\\boldsymbol{t})$. If the relative frequency closely\n", "resembles $p(\\vec{t})$, then using numerics, it is straight forward to\n", "estimate all the interesting parameters of $p(\\boldsymbol{t})$ using point\n", "estimators. \n", "\n", "\n", "\n", "In the case that $\\widehat{\\beta}$ has\n", "more than one component, and the components are independent, we use the\n", "same estimator on each component separately. If the probability\n", "density function of $X_i$, $p(x)$, had been known, then it would have\n", "been straight forward to do this by: \n", "1. Drawing lots of numbers from $p(x)$, suppose we call one such set of numbers $(X_1^*, X_2^*, \\cdots, X_n^*)$. \n", "\n", "2. Then using these numbers, we could compute a replica of $\\widehat{\\beta}$ called $\\widehat{\\beta}^*$. \n", "\n", "By repeated use of (1) and (2), many\n", "estimates of $\\widehat{\\beta}$ could have been obtained. The\n", "idea is to use the relative frequency of $\\widehat{\\beta}^*$\n", "(think of a histogram) as an estimate of $p(\\boldsymbol{t})$.\n", "\n", "\n", "But\n", "unless there is enough information available about the process that\n", "generated $X_1,X_2,\\cdots,X_n$, $p(x)$ is in general\n", "unknown. Therefore, [Efron in 1979](https://projecteuclid.org/euclid.aos/1176344552) asked the\n", "question: What if we replace $p(x)$ by the relative frequency\n", "of the observation $X_i$; if we draw observations in accordance with\n", "the relative frequency of the observations, will we obtain the same\n", "result in some asymptotic sense? The answer is yes.\n", "\n", "\n", "Instead of generating the histogram for the relative\n", "frequency of the observation $X_i$, just draw the values\n", "$(X_1^*,X_2^*,\\cdots,X_n^*)$ with replacement from the vector\n", "$\\boldsymbol{X}$. \n", "\n", "\n", "The independent bootstrap works like this: \n", "\n", "1. Draw with replacement $n$ numbers for the observed variables $\\boldsymbol{x} = (x_1,x_2,\\cdots,x_n)$. \n", "\n", "2. Define a vector $\\boldsymbol{x}^*$ containing the values which were drawn from $\\boldsymbol{x}$. \n", "\n", "3. Using the vector $\\boldsymbol{x}^*$ compute $\\widehat{\\beta}^*$ by evaluating $\\widehat \\beta$ under the observations $\\boldsymbol{x}^*$. \n", "\n", "4. Repeat this process $k$ times. \n", "\n", "When you are done, you can draw a histogram of the relative frequency\n", "of $\\widehat \\beta^*$. This is your estimate of the probability\n", "distribution $p(t)$. Using this probability distribution you can\n", "estimate any statistics thereof. In principle you never draw the\n", "histogram of the relative frequency of $\\widehat{\\beta}^*$. Instead\n", "you use the estimators corresponding to the statistic of interest. For\n", "example, if you are interested in estimating the variance of $\\widehat\n", "\\beta$, apply the etsimator $\\widehat \\sigma^2$ to the values\n", "$\\widehat \\beta^*$.\n", "\n", "Before we proceed however, we need to remind ourselves about a central\n", "theorem in statistics, namely the so-called **central limit theorem**.\n", "This theorem plays a central role in understanding why the Bootstrap\n", "(and other resampling methods) work so well on independent and\n", "identically distributed variables.\n", "\n", "\n", "Suppose we have a PDF $p(x)$ from which we generate a series $N$\n", "of averages $\\langle x_i \\rangle$. Each mean value $\\langle x_i \\rangle$\n", "is viewed as the average of a specific measurement, e.g., throwing \n", "dice 100 times and then taking the average value, or producing a certain\n", "amount of random numbers. \n", "For notational ease, we set $\\langle x_i \\rangle=x_i$ in the discussion\n", "which follows. \n", "\n", "If we compute the mean $z$ of $m$ such mean values $x_i$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "z=\\frac{x_1+x_2+\\dots+x_m}{m},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "the question we pose is which is the PDF of the new variable $z$.\n", "\n", "\n", "The probability of obtaining an average value $z$ is the product of the \n", "probabilities of obtaining arbitrary individual mean values $x_i$,\n", "but with the constraint that the average is $z$. We can express this through\n", "the following expression" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\tilde{p}(z)=\\int dx_1p(x_1)\\int dx_2p(x_2)\\dots\\int dx_mp(x_m)\n", " \\delta(z-\\frac{x_1+x_2+\\dots+x_m}{m}),\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where the $\\delta$-function enbodies the constraint that the mean is $z$.\n", "All measurements that lead to each individual $x_i$ are expected to\n", "be independent, which in turn means that we can express $\\tilde{p}$ as the \n", "product of individual $p(x_i)$. The independence assumption is important in the derivation of the central limit theorem.\n", "\n", "\n", "\n", "If we use the integral expression for the $\\delta$-function" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\delta(z-\\frac{x_1+x_2+\\dots+x_m}{m})=\\frac{1}{2\\pi}\\int_{-\\infty}^{\\infty}\n", " dq\\exp{\\left(iq(z-\\frac{x_1+x_2+\\dots+x_m}{m})\\right)},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and inserting $e^{i\\mu q-i\\mu q}$ where $\\mu$ is the mean value\n", "we arrive at" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\tilde{p}(z)=\\frac{1}{2\\pi}\\int_{-\\infty}^{\\infty}\n", " dq\\exp{\\left(iq(z-\\mu)\\right)}\\left[\\int_{-\\infty}^{\\infty}\n", " dxp(x)\\exp{\\left(iq(\\mu-x)/m\\right)}\\right]^m,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "with the integral over $x$ resulting in" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\int_{-\\infty}^{\\infty}dxp(x)\\exp{\\left(iq(\\mu-x)/m\\right)}=\n", " \\int_{-\\infty}^{\\infty}dxp(x)\n", " \\left[1+\\frac{iq(\\mu-x)}{m}-\\frac{q^2(\\mu-x)^2}{2m^2}+\\dots\\right].\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The second term on the rhs disappears since this is just the mean and \n", "employing the definition of $\\sigma^2$ we have" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\int_{-\\infty}^{\\infty}dxp(x)e^{\\left(iq(\\mu-x)/m\\right)}=\n", " 1-\\frac{q^2\\sigma^2}{2m^2}+\\dots,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "resulting in" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\left[\\int_{-\\infty}^{\\infty}dxp(x)\\exp{\\left(iq(\\mu-x)/m\\right)}\\right]^m\\approx\n", " \\left[1-\\frac{q^2\\sigma^2}{2m^2}+\\dots \\right]^m,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and in the limit $m\\rightarrow \\infty$ we obtain" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\tilde{p}(z)=\\frac{1}{\\sqrt{2\\pi}(\\sigma/\\sqrt{m})}\n", " \\exp{\\left(-\\frac{(z-\\mu)^2}{2(\\sigma/\\sqrt{m})^2}\\right)},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "which is the normal distribution with variance\n", "$\\sigma^2_m=\\sigma^2/m$, where $\\sigma$ is the variance of the PDF $p(x)$\n", "and $\\mu$ is also the mean of the PDF $p(x)$. \n", "\n", "\n", "Thus, the central limit theorem states that the PDF $\\tilde{p}(z)$ of\n", "the average of $m$ random values corresponding to a PDF $p(x)$ \n", "is a normal distribution whose mean is the \n", "mean value of the PDF $p(x)$ and whose variance is the variance\n", "of the PDF $p(x)$ divided by $m$, the number of values used to compute $z$.\n", "\n", "The central limit theorem leads to the well-known expression for the\n", "standard deviation, given by" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\sigma_m=\n", "\\frac{\\sigma}{\\sqrt{m}}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The latter is true only if the average value is known exactly. This is obtained in the limit\n", "$m\\rightarrow \\infty$ only. Because the mean and the variance are measured quantities we obtain \n", "the familiar expression in statistics" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\sigma_m\\approx \n", "\\frac{\\sigma}{\\sqrt{m-1}}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In many cases however the above estimate for the standard deviation,\n", "in particular if correlations are strong, may be too simplistic. Keep\n", "in mind that we have assumed that the variables $x$ are independent\n", "and identically distributed. This is obviously not always the\n", "case. For example, the random numbers (or better pseudorandom numbers)\n", "we generate in various calculations do always exhibit some\n", "correlations.\n", "\n", "\n", "\n", "The theorem is satisfied by a large class of PDFs. Note however that for a\n", "finite $m$, it is not always possible to find a closed form /analytic expression for\n", "$\\tilde{p}(x)$.\n", "\n", "\n", "The following code starts with a Gaussian distribution with mean value\n", "$\\mu =100$ and variance $\\sigma=15$. We use this to generate the data\n", "used in the bootstrap analysis. The bootstrap analysis returns a data\n", "set after a given number of bootstrap operations (as many as we have\n", "data points). This data set consists of estimated mean values for each\n", "bootstrap operation. The histogram generated by the bootstrap method\n", "shows that the distribution for these mean values is also a Gaussian,\n", "centered around the mean value $\\mu=100$ but with standard deviation\n", "$\\sigma/\\sqrt{n}$, where $n$ is the number of bootstrap samples (in\n", "this case the same as the number of original data points). The value\n", "of the standard deviation is what we expect from the central limit\n", "theorem." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Bootstrap Statistics :\n", "original bias std. error\n", " 99.7134 15.0572 99.7151 0.151986\n" ] } ], "source": [ "%matplotlib inline\n", "\n", "import numpy as np\n", "from time import time\n", "from scipy.stats import norm\n", "import matplotlib.pyplot as plt\n", "\n", "# Returns mean of bootstrap samples \n", "# Bootstrap algorithm\n", "def bootstrap(data, datapoints):\n", " t = np.zeros(datapoints)\n", " n = len(data)\n", " # non-parametric bootstrap \n", " for i in range(datapoints):\n", " t[i] = np.mean(data[np.random.randint(0,n,n)])\n", " # analysis \n", " print(\"Bootstrap Statistics :\")\n", " print(\"original bias std. error\")\n", " print(\"%8g %8g %14g %15g\" % (np.mean(data), np.std(data),np.mean(t),np.std(t)))\n", " return t\n", "\n", "# We set the mean value to 100 and the standard deviation to 15\n", "mu, sigma = 100, 15\n", "datapoints = 10000\n", "# We generate random numbers according to the normal distribution\n", "x = mu + sigma*np.random.randn(datapoints)\n", "# bootstrap returns the data sample \n", "t = bootstrap(x, datapoints)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see that our new variance and from that the standard deviation, agrees with the central limit theorem.\n", "\n", "We plot then the histogram together with a best fit for the data set." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEGCAYAAACO8lkDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAArj0lEQVR4nO3dd3xV9f3H8dcnixUCJIEwFVFELW0VIjJUElQqyFJw4F5FWrXYn7ZatVbt0NraugfuCQ6QERCZEVAQAWWJDBUVQSGAkDAzPr8/zoXGJCQ349zvHZ/n43Eed5177vtLEj73nPP9fo+oKsYYY0xpca4DGGOMCT9WHIwxxpRjxcEYY0w5VhyMMcaUY8XBGGNMOQmuA1RXenq6tm/f3nUMdu/eTaNGjVzHqLFIzw/WhnAR6W2I9PwQXBuWLFmSp6rNg91mxBWH9u3bs3jxYtcxyM3NJSsry3WMGov0/GBtCBeR3oZIzw/BtUFEvq7ONu2wkjHGmHKsOBhjjCnHioMxxphyrDgYY4wpx4qDMcaYcqw4GGOMKceKgzHGmHKsOBhjjCnHioMxxphyIm6EtDFhITu7euvPmeNPDmN8YnsOxhhjyrE9B2MOo6gINm2Cb76Bb7/9321+Pty9tyVHNfjedURjfGN7DsZUICcHmjWDnj3hj3+ECRMgLw86dYKjj4buSx9n6rZTXMc0xje252BMGTk5cPXVMGsWdOtW8Tp93vkLF312F9e2msJd7V8mTjS0IY3xme05GFPKwcKQk3P4wgBwatOVLO56HXN+PIlzVtzH9sLGoQtpTAhYcTAmINjCcFDLejuY+cubOaHh13Rd8jRL8zv6H9KYELHiYAzVLwwHJcYV8+AxT/JAh6f51fIHWFFwlH8hjQkhKw4m5tW0MJR2fov3+dtRzzNy7f9RolK3AY1xwLfiICLtRGSOiKwWkVUiMqqCdbJEZKeIfBpY7vIrjzEVWbas9oXhoF+3yqEE4bnN/esmnDEO+dlbqQi4WVWXikhjYImIzFDVz8qsN09VB/iYw5gKqcItt8Ddd9e+MADEifL0sf/hzGX/ZnD6B7RI+rH2GzXGEd/2HFR1s6ouDdzPB1YDbfz6PGOq6733vIFtv/513W3zF8lfckXL97j5i9/U3UaNcUBU/e+fLSLtgblAZ1XdVer5LGAcsBHYBNyiqqsqeP8IYARARkZG17Fjx/qeuSoFBQUkJye7jlFjkZ4fateG4mLh2mszueaaLzn11G3V38DatYd9ae/+BK6653z+cNn7dD1+k/fkscdWuG6s/xzCQaTnh+DakJ2dvURVM4Pdpu/FQUSSgfeBv6vq+DKvpQAlqlogIv2Bh1W10v6AmZmZunjxYv8CByk3N5esrCzXMWos0vNDDdpQarK8Zzf155Uf+pJ74k3I4c4fVzZZXhUT703O68HNX/yG5ZnXUD++8LDbismfQ5iJ9PwQXBtEpFrFwdfeSiKSiLdn8FrZwgCgqrtUtSBwfyqQKCLpfmYypqCoPn/ZcCX/PvrJwxeGWhqYvoCfNdrA/d9c7M8HGOMzP3srCfAcsFpV/3OYdVoG1kNEugXy1GAf35jgPbjxAno3Xc7JKWt8/ZxHjnmUx747l7V72vr6Ocb4wc/eSr2Ay4AVIvJp4LnbgSMAVPUpYBjwGxEpAvYCF2koToKYmLV5fyqPbDyPJV2v8/2z2tXfyh1Hvspv1v6emYpveynG+MG34qCq84FK/xxU9THgMb8yGFPWXzZcxdWt3qV9gx+qXrm6F/SpwI1txvP89/2YNg369av15owJGRshbWLGyoL2TMjrxe1HvBayz0yIK+GWdm/y8MMh+0hj6oRN2W1ixh+/vI7bj3iNZokFIf3cC5vP5tbZI/i8200c1+jbn774l7+ENIsxwbI9BxMT5s+HNXva8ds2E0P+2fXjCxnROodHvzsv5J9tTE1ZcTAx4YknYFTb8STFFTn5/JGtJ/H6ljP4sbCRk883prrssJKJPmVOJG890ISpH73K492nOwoEretto1/qIl74vh+/b/e2sxzGBMv2HEzUe/H7szm3+byQn2soa1TbcTz63bkUq/3ZmfBnv6UmqpWo8PSmgVzXarLrKJySsprmiTuZsq276yjGVMmKg4lqs3ecRKP4fZySstp1FAB+13Y8j2y0E9Mm/FlxMFHt6c0DGdl6UtiMTj6/eS6f7TmSlQXtXUcxplJWHEzU+n5/M2bu6MolGTNdRzkkKa6Ika0nW7dWE/ast5KJWs9/35/zm79PSsIe11F+4rrWkzlu0Uvc1+EZ77oQ99xTfqXKpgs3JgRsz8FEpWKNY/SmAVzX2v2J6LIyknYwKO1Dnt18jusoxhyWFQcTlaZvzyQ9cSddGx/+im0u/a7teB7/bjDFxWFyMsSYMqw4mKj09CbvRHS46tp4LW3q5bFw5RGuoxhTISsOJups3JfO3J2/4KIWs11HqdQVLacza9ExrmMYUyErDibqPPd9f4a3mE1ywj7XUSo1rPn7LFrZjvyiBq6jGFOOFQcTVYqK4JkwPRFdVlriLn7ecTMT83q5jmJMOVYcTFSZNg3a1d/CL5K/dB0lKH1O/oIxW85wHcOYcqw4mKgydixcljHDdYygnfrLDczf2Zm8AymuoxjzE1YcTNTYuxemTIGhzee6jhK0BvWL6J/2EW9v7e06ijE/YcXBRI1p06BLF2+QWSQZ3mI2r9uhJRNmrDiYqPHGG3DBBa5TVN/ZqYtYtbs93+xr4TqKMYdYcTBRYc8eePddOC8C57NLiivivObzeGNLdtUrGxMiVhxMVJg6FU45BZo3d52kZi5uMcsOLZmwYsXBRIU33oALL3SdouZOb7qcLQeasnq3TadhwoMVBxPxCgpg+nQYMsR1kpqLlxIubJFrYx5M2LDiYCJeTg707Alpaa6T1M7FLWYyZksfVF0nMcaKg4kCb74Z2YeUDuraeC2Csji/k+soxlhxMJEtPx9mzYLBg10nqT0RG/NgwocVBxPRJk2C00+HZs1cJ6kbw1vM4o0t2RQXu05iYp1vxUFE2onIHBFZLSKrRGRUBeuIiDwiIutFZLmIdPErj4lOb74ZmQPfDue4Rt/SMmk777/vOomJdX7uORQBN6vq8UB34HoROaHMOv2AjoFlBPCkj3lMlCkoSCA3NzoOKZV2QfNcxo93ncLEOt+Kg6puVtWlgfv5wGqgTZnVBgMvq2ch0FREWvmVyUSXD8YVkp04n5TB2ZBdaolwQ9LnM2EC1mvJOJUQig8RkfbAScBHZV5qA3xb6vHGwHOby7x/BN6eBRkZGeTm5voVNWgFBQVhkaOmIj0/wMzlx/GrIZ+R22246yg1VpCaSu7w8vnj/raH0aNX06lTvoNU1RPpv0uRnh/8aYPvxUFEkoFxwE2quqvsyxW8pdz3JVUdDYwGyMzM1KysrLqOWW25ubmEQ46aivT8O3bA6s8OMLvhgzT+Yq/rODWWO3w4WWPGlHv+4otH8O23XbnuOgehqinSf5ciPT/40wZfeyuJSCJeYXhNVSs6iroRaFfqcVtgk5+ZTHSYMAG6HP8djRMitzBUZsgQr43GuOJnbyUBngNWq+p/DrPaJODyQK+l7sBOVd18mHWNOWTCBDjtpK9cx/BNt26wfTusW+c6iYlVfu459AIuA/qIyKeBpb+IjBSRkYF1pgJfAuuBZ4Df+pjHRIk9e2DOHDil87dVrxyh4uK8XlgTJ7pOYmKVb+ccVHU+FZ9TKL2OAtf7lcFEp5kzITMTUhrtdx3FV0OGwL33wi23uE5iYpGNkDYRZ+LE6BvbUJHsbPjsM/j+e9dJTCyy4mAiSnGxNwvroEGuk/gvKQnOPhsmT3adxMQiKw4m/JUa4Lbw5BvJ2P0FR10d+YPdgmG9lowrVhxMRJmU15PB6R+4jhEy/frBvHne7LPGhJIVBxNRJub1YnBaDBSHwJ5SyuBseiV+xLTud0fF1CAmclhxMBFjzZ525Bc3oGvjta6jhNSQ9A+YkHeq6xgmxlhxMBFjYl4vBqV/iFTaQTr6DE6bz7vbu3GgJCRToRkDWHEwESRmDimV0bLeDo5r+A3v//hL11FMDLHiYCLClgNNWbW7PdnNPnUdxYkh6R/wTt5prmOYGGLFwUSEnG096Ju6mHpxha6jODEkfT4T83pSUuI6iYkVVhxMRJiY14tBMXhI6aBjG26kScJuFi92ncTECisOJuztKa7HnB9PpH9a2WtFxZYh6R/YRHwmZKw4mLA3c0dXMhuvITUxtkeCDUz70KbSMCFjxcGEvYl5vRic/qHrGM51S/mc77+Hr792ncTEAisOJqwVF0POtu4xfb7hoHgpoX9/m4jPhIYVBxPWFi6EjKQdHNXA5q0GGDjQioMJDSsOJqxNmgSD0uyQ0kF9+8KCBTYRn/FfUMVBRAaIiBUSE3KTJ8MgO99wSOPG0KMHzJjhOomJdsH+h38RsE5EHhCR4/0MZMxBX3wB27dDZuM1rqOEFTu0ZEIhqOKgqpcCJwFfAC+IyAIRGSEijX1NZ2JaTg6ccw7EibqOElYGDIApU7yT9cb4JehDRaq6CxgHjAVaAecCS0XkRp+ymRg3ebL3Ldn8VPv2kJEBixa5TmKiWVBzAIvIIOAq4GjgFaCbqm4RkYbAauBR/yKamFHqYjY7ixqxaMEbnHVgGMQ7zBSmDh5a6tHDdRITrYLdcxgG/FdVf6Gq/1LVLQCquge42rd0Jma9t/1kTm2ygkbx+1xHCUsDB3qH3YzxS7DFYbOqzi39hIj8E0BVZ9V5KhPzcrb1YEDaQtcxwla3bthoaeOrYIvDWRU8168ugxhzULHG8e72bgxIW+A6StiKj8dGSxtfVVocROQ3IrICOE5ElpdavgKWhyaiiTULdp5Am6Q8jqi/xXWUsGZdWo2fqjoh/TrwLnAfcFup5/NVdbtvqUxMm7ytJwNt4FuV+vaFq67yRks3tk7lpo5VdVhJVXUDcD2QX2pBRFL9jWZilZ1vCM7B0dLTp7tOYqJRVcXh9cDtEmBx4HZJqcfG1Kkv97ZiW2EKJzf+3HWUiGCHloxfKj2spKoDArdHhSaOiXWTt/XknLSFNio6SAMGwL33eqOl4208iKlDVZ2Q7lLZUsV7nxeRLSKy8jCvZ4nIThH5NLDcVZuGmOgwOa8HA20W1qDZaGnjl6pOSD9YyWsK9Knk9ReBx4CXK1ln3sG9E2N2FTXko/zjmdDsTtdRIsrAgd7U5jZa2tSlqg4rZVf2ehXvnSsi7Wv6fhN7Do6KTk6wUdGHlV3+T3LQzuO59sAT3Hefgzwmaonq4Y/tikgfVZ0tIudV9Lqqjq90415xyFHVzhW8loU3kd9GYBNwi6quOsx2RgAjADIyMrqOHTu2so8NiYKCApKTk13HqLFwzP+PP7XmhKO2MCTrs6DWL0hNJXl7ZPeoros2lJTAsNuv5LHHltK6degLazj+LlVHpOeH4NqQnZ29RFUzg91mVYeVegOzgYrmxlSg0uJQhaXAkapaICL9gQlAx4pWVNXRwGiAzMxMzcrKqsXH1o3c3FzCIUdNhVv+4mJYungnz/Jnjtgc3OC33OHDyRozxudk/qqrNpx33gi2bu3OxRfXQahqCrffpeqK9PzgTxuqOqz0l8DtVXX6qRyaAvzg/aki8oSIpKtqXl1/lgl/CxdCm3o2KrqmBg2Chx+GUaNcJzHRItjLhKaJyCMislRElojIwyKSVpsPFpGWIiKB+90CWbbVZpsmck2ejPVSqoUzz4SPP4YdO1wnMdEi2In3xgJbgaF403dvBd6o7A0iMgZYAHQSkY0ico2IjBSRkYFVhgErRWQZ8AhwkVZ2AsREtUmT7FrRtdGwIWRlwbRprpOYaBHUxX6AVFX9a6nHfxORIZW9QVWHV/H6Y3hdXU2MW7cOfvwRMlvYtaJrY9Agr8gOr/Qvz5jgBFsc5ojIRcCbgcfDgCn+RDJRr0x3zInfXMAg2tio6Fo65xz4wx/gwAFISnKdxkS6qkZI54vILuA6vHmWDgSWscDv/Y9nYsHEbb3skFIdaNUKjj0W5s1zncREg0qLg6o2VtWUwG2cqiYEljhVTQlVSBO9th5owvKCDvRputR1lKhw8NCSMbUV7AlpRKSZiHQTkdMPLn4GM7FhyrbunNVsCfXjC11HiQoHi4N17TC1FWxX1muBucB7wD2B27v9i2VixcRtvRic/oHrGFGjc2AugpUVTndpTPCC3XMYBZwMfB2Yb+kkvO6sxtTY3uIkZu84iXPswj61l50N2dlIn2wGFY5j0uBnK5yHyZhgBVsc9qnqPgARqaeqnwOd/ItlYsHMHV3p0ngdqYn5rqNElUHpHzI5r6frGCbCBduVdaOINMWb/2iGiOzAmyzPmBqbmNeLQTYqus6d3mQZa/a24/v9zWjpOoyJWEHtOajquar6o6reDfwZeA4Y4mMuE+VKVMjZ1sPON/ggMa6Ys1MXkbPNLvBgaq46vZW6iMjvgF8AG1X1gH+xTLT7aNfxNE/6kQ4NNruOEpUGpX3IpG12aMnUXLC9le4CXgLSgHTgBRGxy3WZGpuY14vBabbX4JezUxeR++OJ7NnjOomJVMHuOQwHTlbVvwSm8e4OXOJfLBPtrAurv5olFnBy4zXMnOk6iYlUwRaHDUD9Uo/rAV/UeRoTE9buacvOokZ0bbzWdZSoNij9QyZMcJ3CRKpKeyuJyKN4V3zbD6wSkRmBx2cB8/2PZ6LRpLyeDEr70Cba89mQ9Pn8bfINFBVBQrD9Eo0JqOpXZnHgdgnwTqnnc31JY2LCxG29uP2I11zHiHpH1v+B9u1h7lzo08d1GhNpqrpM6EsH74tIEnBs4OEaVbXJcEy1bd0KKwo60KfZJ66jxITzzoNx46w4mOoLtrdSFrAOeBx4AlhrE++ZmsjJgbNSF1Mvzr5bhMLQofDOO1BS4jqJiTTBnpB+EOirqr1V9XTgV8B//YtlotXEiVgX1hA69lhIS4MFC1wnMZEm2OKQqKqHruGoqmuBRH8imWi1axfMmQMD0ux/qlAaOtQ7tGRMdQRbHJaIyHMikhVYnsE7SW1M0HJy4LTToGnibtdRYsrQoTB+vF3jwVRPsMVhJLAK+B3e9N2fBZ4zJmhvvQXnn+86Rezp3Nm7pvQS+zpnqqHK3s8iEgcsUdXOwH/8j2SiUX4+zJ4Nzz8PvOg6TQzJzkaAofuvZdxQyOzwrPf8nDlOY5nwV+Weg6qWAMtE5IgQ5DFRKicHevWCZs1cJ4lNQ9PnMm7r6XZoyQQt2HGTrfBGSC8CDh0wVtVBvqQyUccOKbnVtfFaDpQksnL3Ufw8+SvXcUwECLY43ONrChPVCgpg1ix49lnXSWKXCJzXfB7jtp5uxcEEpdLDSiJSX0RuAs4HjgM+UNX3Dy6hCGgi35Qp0LMnpKa6ThLbhjafy/i801zHMBGiqj2Hl4BCYB7QDzgBr7eSMVULXOD+rZV3c37aR5D9ruNAsa1HyiryCpuwbk8bOroOY8JeVSekT1DVS1X1aWAYYF87TLXsLq7PjB1dGZJuk/i6FifKuenzGbfVZr4xVauqOByaAEdVi3zOYqLQlG3d6ZHyGamJ+a6jGALnHfKsOJiqVXVY6ZcisitwX4AGgccCqKqm+JrORLy3tvbm/BZ2eipc9G7yKRv2teTrr+HII12nMeGs0j0HVY1X1ZTA0lhVE0rdr7QwiMjzIrJFRFYe5nURkUdEZL2ILBeRLrVpiAk/u4vrM317JoPT7JBSuEiIK2FQ2oeMH+86iQl3wU6fURMvAmdX8no/oGNgGQE86WMW48DUbadwSspq0pN2Vb2yCZlhzd/nzTddpzDhzrfioKpzge2VrDIYeFk9C4GmItLKrzwm9N7amsX5ze2QUrg5s9kSvvwSvrCrwJtKiPo4nl5E2gM5gXmZyr6WA9yvqvMDj2cBt6rq4grWHYG3d0FGRkbXsWPH+pY5WAUFBSQnJ7uOUWN+59+3L45h53bntb+PoUnyfl8+oyA1leTtlX3/CH+u2vDItP40aVLIFVd8Xett2d+Ce8G0ITs7e4mqZga7TZeXHZcKnquwUqnqaGA0QGZmpmZlZfkYKzi5ubmEQ46a8jv/229DjwaLGTz5Rd8+I3f4cLLGjPFt+6Hgqg2NHhjBxRfD888fhVT0l1gN9rfgnh9t8POcQ1U2Au1KPW4LbHKUxdSxt97CDimFscxMiIuDRYtcJzHhymVxmARcHui11B3YqaqbHeYxdWTnTnjvPTgvfa7rKOYwRODSS+HVV10nMeHKt+IgImOABUAnEdkoIteIyEgROXiRoKnAl8B64Bngt35lMaH11lvQpw/WSynMXXIJvPEGFBZWva6JPb6dc1DV4VW8rsD1fn2+cefFF+EPfwAechzEVKpDB+jY0dvLGzDAdRoTblweVjJRaP16WLsW+vd3ncQE47LL7NCSqZjL3komCr38Mlx8MSQmuk5iKhWYMff8whRuXfgau067kJSEPXb5UHOI7TmYOlNSAi+9BFdc4TqJCVZa4i6ym37K+K024bL5KSsOps68/z40bQonnug6iamOSzNm8MoPfV3HMGHGioOpMwf3Gmo7qMqE1oC0BXxScAwb96W7jmLCiBUHUycKCmDCBK97pIks9eMLGdp8LmO2nOE6igkjVhxMnRg3Dk47DTIyXCcxNXFpxkxe/eEs1zFMGLHiYOrEiy/aiehIdlqT5ewoSmb5ctdJTLiw4mBqbcMGWLECBg50ncTUVJwol2TMtDEP5hArDqbWXnkFLrwQ6tVzncTUxuUZ03nlFThwwHUSEw6sOJhaUbWxDdHi+Ebf0KmT17HAGBshbWrlgw8gcfPXnPzHKyu+QoeJKNdfD489Bhdc4DqJcc32HEytvPQSXNlymo1tiBJDhsC6dd45JBPbrDiYGtu50+vCemnGTNdRTB1JTIQRI+DJJ10nMa5ZcTA19sIL0LcvtKmX5zqKqUMjRsCYMbDLLscR06w4mBopLoZHHoGbbnKdxNS11q3hzDO9XmgmdllxMDWSkwPNm0P37q6TGD9cfz08/rjXG83EJisOpkYefhhGjXKdwvild29vAsXcXNdJjCtWHEy1LV8Oa9bAsGGukxi/iMBvfwtPPOE6iXHFioOptocf9v7jSEpyncT46bLLYNYs+O4710mMCzYIzlTL1q0wfrx3nWgThQKXDwVIAS5qcBPP9NrB3RuudBbJuGF7DqZann4ahg71Tkab6Hd9mwmM3jyAwkLXSUyoWXEwQTtwwBscZSeiY8fPGm3g2AYbeecd10lMqFlxMEF76y047jj4+c9dJzGh9Ns2E3nsMdcpTKjZOQcTFFV4+DerufPIVyH7Q9dxTAidmz6P276FefO8q/2Z2GB7DiYoCxfC9sIUzklb6DqKCbHEuGLuuAPuucd1EhNKVhxMUB56CG5s+w7xUuI6inHg8svhiy+8vQcTG6w4mCqtWQOzZ8NVLd91HcU4kpgId95pew+xxIqDqdJdd8HNN0NKwh7XUYxDtvcQW6w4mEotXer9Z3Djja6TGNds7yG2+FocRORsEVkjIutF5LYKXs8SkZ0i8mlgucvPPKb67rjDWxo1cp3EhIPLL4cvv7S9h1jgW1dWEYkHHgfOAjYCH4vIJFX9rMyq81R1gF85TM3Nneudb5g40XUSEy5K7z3MtAsARjU/9xy6AetV9UtVPQCMBQb7+HmmDqnCn/7k/SdgE+yZ0i67zPYeYoGoT1fzEJFhwNmqem3g8WXAKap6Q6l1soBxeHsWm4BbVHVVBdsaAYwAyMjI6Dp27FhfMldHQUEBycnJrmPUWFX5FyxIY/ToDjz77MfExweeDLPZ9gpSU0nevt11jFqJmDYce+xPHk6d2pJZszJ48MFlUf+3EAmCaUN2dvYSVc0Mdpt+jpCWCp4rW4mWAkeqaoGI9AcmAB3LvUl1NDAaIDMzU7Oysuo2aQ3k5uYSDjlqqrL8JSXe/EkPPQRnnFFqnTA7E5k7fDhZY8a4jlErkdqGXiXxvP3Zy8THZ5GcHL1/C5HCjzb4eVhpI9Cu1OO2eHsHh6jqLlUtCNyfCiSKSLqPmUwQxo6Fhg1h0CDXSUy4Sowr5s4jX+Xuu+1SotHKz+LwMdBRRI4SkSTgImBS6RVEpKWISOB+t0CebT5mMlUo7H0md13zHf/Y+3ukT7Y3v//BxZhSLsuYzqZNMH++fZ+LRr4VB1UtAm4A3gNWA2+q6ioRGSkiIwOrDQNWisgy4BHgIvXrJIgJynOb+9OhwSaym33qOooJc4lxxTz5JDz66DHk57tOY+qar7OyBg4VTS3z3FOl7j8G2GTAYSI/H/769WVM7Hyn6ygmQmRlQdeuO7jrrlb897+u05i6ZCOkzSG33gq/Sl1MZkp49Uoy4W3kyC95/XVvNL2JHlYcDABz5sDkyfCfox93HcVEmCZNCrn/frjuOigudp3G1BUrDoaCArjmGnjqKWiauNt1HBOBrrzS6+H2xBOuk5i6YsXBcPvtcOqpcM45rpOYiJOdDWvXIn2yeWrvFdx784981/N8690WBaw4xLi5c+Htt70Bb8bUxvGNvmFk60nctP5611FMHbDiEMP27PEOJz35JKSmuk5josHtR7zGJ/kdmbKtu+soppasOMSwO++Ek0+GwTYdoqkjDeIP8OSx/+X6taPYudN1GlMbvo5zMOFrxdSdjHl8Gysyr4bsXa7jmChyVuoS+qd9xOWXD+addyDOvoJGJPuxxaDdu+FfL/fmsY6PkJ5khcHUvYeOeYxt2+Cvf3WdxNSU7TnEmOJiuOQS+FmHHxhab67rOCZKJcUV8fYb3mHLk06ySRwjke05xJg//hF27oT/u9Su1GL81bKl1xPu2mvh889dpzHVZcUhhjzxBEyZAuPGQWJCies4Jgaccgrcdx8MGYKdoI4wVhyiXWC67am/uI17b9rOlKaXkDrUBiiZ0LnmGujTx7u8aIl9J4kYds4hBiwrOJorPr+NCZ3/zNENNlX9BmPqQqlR0g+VJHDGsge5t8NS7t5wpbtMJmi25xDlNu1PY+CKv/PoMY/Qq8lK13FMjEqKK+KtE+7m+e/78dRTVa9v3LPiEMV27YKBK/7ByNaTuShjjus4Jsa1rLeDOSf+Hw88AP/8p+s0pipWHKLUxo1w2mnQI2UVfzriNddxjAHg6AabmDcPXnrJm/DRrvsYvqw4RKFly6BnT288w6MdH8G7Srcx4aFNG2/Cx+nT4YYb7CR1uLLiEGXeew/OOgv+9S9vTIMVBhN2srNJPz+bWQ0GsOK1ZVzRejpFvc9wncqUYcUhijz7LFxxBYwfDxde6DqNMZVrkrCbab+4lW2FKQxbdQ979rhOZEqz4hAFVOGOO+D++73d9VNPdZ3ImOA0jN/PhM5/pknCbrp0gUWLXCcyB9k4hwi3Zg2MHAmFS1ewoPOfaX6dDUM1kSUproiXjr+fN7csZOCpNzKidQ5/PvIVkuKKvBXmWE87F2zPIULt3w/33gu9enlTE7x/0k00T7LCYCLXBS1y+TTz13yS35FTlj7ByoL2riPFNCsOEWjuXDjxRFiyBD75BEaNgnixLh8m8rWqt53JP7+dG9u8Q/ay//LANxdRXOw6VWyy4hBBtmyBa1tN4ZK+W/hH/J+ZsDObdpdn28XcTVQRgatbvcvHXUYyfXsmxx8PL7wAhYWuk8UWKw7hKjBhHtnZLD/5Gq5u9S6d2uSTHL+XVSdfxbnN51s3VRPV2jf4gRm/vIVnnoHXX4djjvFmFt63z3Wy2GDFIUyVqDA5rwdnfPog/ZbfzzENvmNdt0t5qOPjpCRYnz8TG0Sgd2+YMQPefBOmTYMOHeDf/7YpwP1mvZXCiCqsWgWTJ8Pzi16mSfxuft/ubc5vnvu/nhvGxJrAYdNTgEnAsjZHc98DF/PX27pxetPlDGs+l8Fp82mauNtb33o31QkrDo7t3w+5uZCT4y2qMHAgvNDpn/RqstIOHRlTxi+Tv2Dsz/7KrqKGTN7Wk7e3ns7v1t3AqU1WMqz5+/T/ATIyXKeMfFYcQkgVvv4ali71liVL4MMZBXRutIGBaR8yOX0BP2u0AVkJNHWd1pjwlpKwh0syZnJJxkzyixqQs60Hb2/tzc3HQ5Mm0KMHdO/uLSeeCElJrhNHFl+Lg4icDTwMxAPPqur9ZV6XwOv9gT3Alaq61M9MobBrF2zY8L/lq69g5UqvINSvD126eMvIkfDyrkttfIIxtdQ4YS/DM2YzPGM2JSqs29uWhZ+cwILcE3h+1wms29uG4xp+yzENviu36N9vdB0/LPlWHEQkHngcOAvYCHwsIpNU9bNSq/UDOgaWU4AnA7dOFRV5h3v27YPdu6Gg4KdLfj58NDmR3CtfJK+wyaFla2FTvk06mgMHoH37ny59Z93KSZ3W0bLeDigA5gYW+zZjTJ2KE6VTw2/p1PBbrmj5HgAFRfX5bE971u9tw/q9bZjz40k8s/kc1u1ty86+ybSM/4GMpB20SPwxcLuD5qMuJiWFnyyNG3tLgwbeF7369aFeveic4NLPPYduwHpV/RJARMYCg4HSxWEw8LKqKrBQRJqKSCtV3VzXYbZvhz5HrKdI4ytc9msi+0u8RYCkuELqxRWSHL/30NI4fk/g/j72ntCSVOC4ht+QnriTtMRdpCfupG29raQn7vR+WXYDqwJLWl23yBgTrOSEfXRL+ZxuKZ+Xe23a0Es57tUZ/FCYypYDTfnhQDO2FDZj40NvkV/UkF3FjcgvbsCuIu82v7gh+0qS2FeSxN7iehzQBJKkiHpxB0iUYhLjikiQYhLl4G0x8VJMHEq8lBAnJcTj3cb9vDMiEBfnFZhD95cuAUBEERQBBOVv07uRmRmafzNRn662ISLDgLNV9drA48uAU1T1hlLr5AD3q+r8wONZwK2qurjMtkYAIwIPOwFrfAldPelAnusQtRDp+cHaEC4ivQ2Rnh+Ca8ORqto82A36uedQ0Y5W2UoUzDqo6mhgdF2EqisislhVQ1TD616k5wdrQ7iI9DZEen7wpw1+DoLbCLQr9bgtsKkG6xhjjAkxP4vDx0BHETlKRJKAi/DGsJQ2CbhcPN2BnX6cbzDGGFM9vh1WUtUiEbkBeA+vK+vzqrpKREYGXn8KmIrXjXU9XlfWq/zK44OwOsxVA5GeH6wN4SLS2xDp+cGHNvh2QtoYY0zkson3jDHGlGPFwRhjTDlWHMoQkVEislJEVonITYHnfikiC0RkhYhMFpGUCt7XTkTmiMjqwHtHhTz8/7LUqA2l3h8vIp8ExqE4UZs2BAZTvi0inwd+Hj1CGp5a5/994H0rRWSMiNQPUebnRWSLiKws9VyqiMwQkXWB22alXvuTiKwXkTUi8qvDbPOw74+gNvwr8Lu0XETeEZGmkdaGUuveIiIqIulVBlFVWwIL0BlYCTTEO1k/E29qj4+B3oF1rgb+WsF7WwFdAvcbA2uBEyKpDaW28X/A60BOpP0cAq+9BFwbuJ8ENI2U/EAb4CugQeDxm3hzjoUi9+lAF2BlqeceAG4L3L8N+Gfg/gnAMqAecBTwBRBfwTYrfH+EtaEvkBC4/89IbENg3XZ4HYS+BtKrymF7Dj91PLBQVfeoahHwPnAu3qjsuYF1ZgBDy75RVTdrYNJAVc0HVuP9oYdajdsAICJtgXOAZ0OQ9XBq3IbAt/HTgecAVPWAqv4YitCl1OpngFdQGohIAl6BCcnYH1WdC2wv8/RgvGJL4HZIqefHqup+Vf0Kr8dhtwo2e7j3+8KPNqjq9MDPEWAh3ngs3/j0cwD4L/BHKhhoXBErDj+1EjhdRNJEpCFeN9t2gecHBdY5n58O3CtHRNoDJwEf+Rf1sGrbhofwfoFKfM5Zmdq0oQOwFXghcGjsWRFpFIrQpdQ4v6p+B/wb+AbYjDf2Z3pIUlcsQwNjjwK3LQLPtwG+LbXeRir+MnS494dSbdtQ2tXAu3WesGq1aoOIDAK+U9VlwX6gFYdSVHU13m7jDGAa3u5aEd4vxPUisgTvkNGBw21DRJKBccBNqrrL99Bl1KYNIjIA2KKqS0KXuLxa/hwS8HbJn1TVk/CmP7wtFLkPquXPoBnet8GjgNZAIxG5NETRqyOoqW/CXLXaICJ34P0cX/MtUfVV2YbAF5Q7gLuqs2ErDmWo6nOq2kVVT8fbtVunqp+ral9V7QqMwTuuV46IJOIVhtdUdXzoUv9ULdrQCxgkIhuAsUAfEXk1ZMFLqUUbNgIbVfXgXtvbeMUipGqR/0zgK1XdqqqFwHigZ+iSl/ODiLQCCNxuCTwf7NQ3h3t/KNW2DYjIFcAA4BINHMAPsdq04Wi8LxvLAn/bbYGlItKysg+04lCGiLQI3B4BnAeMKfVcHHAn8FQF7xO849yrVfU/oUtcXk3boKp/UtW2qtoeb7qT2arq5FtrLdrwPfCtiHQKPHUGP50mPiRqmh/vcFJ3EWkY+J06A+/8lSuTgCsC968AJpZ6/iIRqSciR+GdcF9UjfeHUq3aIN5Fy24FBqnqnhDkrUiN26CqK1S1haq2D/xtb8TrPPN9pZ/o51n3SFyAeXj/mSwDzgg8Nwqv99Fa4H7+N7K8NTA1cP9UvN255cCngaV/JLWhzDaycNRbqbZtAE4EFgd+FhOAZhGW/x7gc7xzFK8A9UKUeQzeeY7CwH8g1+BdiWQWsC5wm1pq/Tvw9n7WAP1KPf8skBm4f9j3R1Ab1uMd1z/4d/1UpLWhzPY3EERvJZs+wxhjTDl2WMkYY0w5VhyMMcaUY8XBGGNMOVYcjDHGlGPFwRhjTDlWHIwxxpRjxcEYY0w5VhyMqSUROTkw1399EWkUuBZDZ9e5jKkNGwRnTB0Qkb8B9YEGeHM73ec4kjG1YsXBmDogIkl4F/PZB/RU1WLHkYypFTusZEzdSAWS8abiDsllPY3xk+05GFMHRGQS3jTnRwGtVPUGx5GMqZUE1wGMiXQicjlQpKqvi0g88KGI9FHV2a6zGVNTtudgjDGmHDvnYIwxphwrDsYYY8qx4mCMMaYcKw7GGGPKseJgjDGmHCsOxhhjyrHiYIwxppz/B/ogcq8bAbBTAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_47_0.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# the histogram of the bootstrapped data (normalized data if density = True)\n", "n, binsboot, patches = plt.hist(t, 50, density=True, facecolor='red', alpha=0.75)\n", "# add a 'best fit' line \n", "y = norm.pdf(binsboot, np.mean(t), np.std(t))\n", "lt = plt.plot(binsboot, y, 'b', linewidth=1)\n", "plt.xlabel('x')\n", "plt.ylabel('Probability')\n", "plt.grid(True)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The bias-variance tradeoff\n", "\n", "\n", "We will discuss the bias-variance tradeoff in the context of\n", "continuous predictions such as regression. However, many of the\n", "intuitions and ideas discussed here also carry over to classification\n", "tasks. Consider a dataset $\\mathcal{L}$ consisting of the data\n", "$\\mathbf{X}_\\mathcal{L}=\\{(y_j, \\boldsymbol{x}_j), j=0\\ldots n-1\\}$. \n", "\n", "Let us assume that the true data is generated from a noisy model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{y}=f(\\boldsymbol{x}) + \\boldsymbol{\\epsilon}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $\\epsilon$ is normally distributed with mean zero and standard deviation $\\sigma^2$.\n", "\n", "In our derivation of the ordinary least squares method we defined then\n", "an approximation to the function $f$ in terms of the parameters\n", "$\\boldsymbol{\\beta}$ and the design matrix $\\boldsymbol{X}$ which embody our model,\n", "that is $\\boldsymbol{\\tilde{y}}=\\boldsymbol{X}\\boldsymbol{\\beta}$. \n", "\n", "Thereafter we found the parameters $\\boldsymbol{\\beta}$ by optimizing the means squared error via the so-called cost function" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "C(\\boldsymbol{X},\\boldsymbol{\\beta}) =\\frac{1}{n}\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2=\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right].\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can rewrite this as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\frac{1}{n}\\sum_i(f_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2+\\frac{1}{n}\\sum_i(\\tilde{y}_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2+\\sigma^2.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The three terms represent the square of the bias of the learning\n", "method, which can be thought of as the error caused by the simplifying\n", "assumptions built into the method. The second term represents the\n", "variance of the chosen model and finally the last terms is variance of\n", "the error $\\boldsymbol{\\epsilon}$.\n", "\n", "To derive this equation, we need to recall that the variance of $\\boldsymbol{y}$ and $\\boldsymbol{\\epsilon}$ are both equal to $\\sigma^2$. The mean value of $\\boldsymbol{\\epsilon}$ is by definition equal to zero. Furthermore, the function $f$ is not a stochastics variable, idem for $\\boldsymbol{\\tilde{y}}$.\n", "We use a more compact notation in terms of the expectation value" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathbb{E}\\left[(\\boldsymbol{f}+\\boldsymbol{\\epsilon}-\\boldsymbol{\\tilde{y}})^2\\right],\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and adding and subtracting $\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]$ we get" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathbb{E}\\left[(\\boldsymbol{f}+\\boldsymbol{\\epsilon}-\\boldsymbol{\\tilde{y}}+\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2\\right],\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "which, using the abovementioned expectation values can be rewritten as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathbb{E}\\left[(\\boldsymbol{y}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2\\right]+\\mathrm{Var}\\left[\\boldsymbol{\\tilde{y}}\\right]+\\sigma^2,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "that is the rewriting in terms of the so-called bias, the variance of the model $\\boldsymbol{\\tilde{y}}$ and the variance of $\\boldsymbol{\\epsilon}$." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Error: 0.013121573975499602\n", "Bias^2: 0.012073649439965807\n", "Var: 0.0010479245355337968\n", "0.013121573975499602 >= 0.012073649439965807 + 0.0010479245355337968 = 0.013121573975499604\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABahklEQVR4nO29d5xU1f3//zzTdmc72yuwVOlFVCJYUbEiFuxfNRZCjDExiZGYfBRJfglqotEYo8SYnigxiliJwQpIaAvLAtJX2N5n++6U8/vjzsxO31mY7ef5ePhw594z9565zLzu+77PuwgpJQqFQqEY/Oj6ewIKhUKhiAxK0BUKhWKIoARdoVAohghK0BUKhWKIoARdoVAohgiG/jpxamqqHD16dH+dXqFQKAYlO3bsqJFSpgXa12+CPnr0aLZv395fp1coFIpBiRDiq2D7lMtFoVAohghK0BUKhWKIoARdoVAohgj95kMPhNVqpaSkhPb29v6eyrAhOjqa3NxcjEZjf09FoVCcIgNK0EtKSoiPj2f06NEIIfp7OkMeKSW1tbWUlJSQn5/f39NRKBSnyIAS9Pb2diXmfYgQgpSUFKqrq/t7KoohytqCUp5af4Cyhjayk8w8tHAii2fl9Pe0hiwDStABJeZ9jLreit5ibUEpP3pjD21WOwClDW386I09AErUewm1KKpQKHqFp9YfcIu5izarnafWH+inGQ19BpyFrlAoBg6n4jIpa2jr0XbFqaMsdB+ee+45Jk2axK233sratWtZuXJlyPE/+MEP+Oijj/podgpF3+FymZQ2tCHpcpmsLSgN6/3ZSeYebVecOkrQfXjhhRd47733+Pvf/86TTz7JfffdF3L8t7/9bVatWtVHs1Mo+o5TdZk8tHAiZqPea5vZqOehhRMjNkeFNwPW5fL423vZV9YY0WNOzk7gsaumBN2/bNkyjh49yqJFi7jtttuIiooiNTUVgKuvvprrrruO22+/nZdeeonPPvuMv//974waNYra2loqKirIzMyM6HwViv7kVF0mLteMinLpOwasoPcHL774Ih988AEff/wxb7/9NrNnz3bvW716NfPmzSM/P59f/epXbNmyxb1v9uzZbNq0ieuuu64/pq1Q9ArZSWZKA4h3T1wmi2flKAHvQwasoIeypPuC8vJy0tK6KlRmZGSwcuVKLrjgAt58802Sk5Pd+9LT0ykrK+uPaSoUvcZDCyfyw9cL6bQ73NuUy2RgM2AFvb8xm81YLBavbXv27CElJcVPvNvb2zGb1UKPYmixeFYOH31Zxbrd2vc9KyGahy87LaTFrRKJ+he1KBqESZMmcfjwYffrrVu38v7771NQUMAvf/lLjh075t538OBBpk6d2h/TVCh6FYOuK/Hs93fM6VbMTyUqRnHqKEEPwrnnnktBQQFSSjo6Orj33nt55ZVXyM7O5le/+hV33XUXUkqsViuHDx9mzpw5/T1lhSLi7K9oIj0+CoATda0hx6pEov5HCboPxcXFpKamEhMTw0UXXcSGDRuIiopi9+7d7kXSRYsW8fHHHyOE4J133uH666/HYFDeK8XQwmp3cKSqmYsnZwBwvBtBV4lE/U+3gi6EeEUIUSWEKAox5nwhxC4hxF4hxKeRnWL/8cgjj9DaGvpLbLPZ+P73v99HM1Io+o5jNS102h2cMTqZpBhjt4KuEon6n3As9D8BlwbbKYRIAl4AFkkppwBLIjKzAUBGRgaLFi0KOWbJkiUkJSX1zYQUij5kf7mWB3JaVjx5I2I4UR/a0laJRP1Pt34CKeVnQojRIYbcArwhpTzuHF8VobkpFIp+5EBFEwadYExqHCOTY9hXHjrRTyUS9T+RcPxOAIxCiE+AeOBZKeVfInBchULRj3xZ0cS49DhMBh15yTH8Z18FdodEr/MuuaxCFQcOkRB0A3A6sAAwA18IIbZIKQ/6DhRCLAWWAowcOTICp1YoFL3Fl+WNnJmvJdCNTI7BapdUNLaT4+ETVzXPBxaRiHIpAT6QUrZIKWuAz4AZgQZKKVdLKedIKed4ZmEOJPR6PTNnzmTKlCnMmDGDp59+GofDEfI9xcXF/OMf/+j1ud1zzz3s27cv5Ji1a9d2O0ah6A5Lm5UySzsTMxMATdDBP3Tx8bf3qlDFAUQkBP0t4BwhhEEIEQOcBeyPwHH7BbPZzK5du9i7dy8ffvgh7733Ho8//njI9/SVoL/88stMnjw55Bgl6IpIcKCiCdAWRAHykjWr3DPSZW1BKfWt1oDvV6GK/UM4YYv/BL4AJgohSoQQdwshlgkhlgFIKfcDHwCFwFbgZSll0BDHSLK2oJR5qz4if/m7zFv1UcQz0tLT01m9ejXPP/88UkqKi4s555xzmD17NrNnz2bz5s0ALF++nM8//5yZM2fyzDPPBB3nSXFxMaeddhp33HEH06dP5/rrr3eHSG7YsIFZs2Yxbdo07rrrLjo6OgA4//zz2b59OwBxcXH8+Mc/ZsaMGcydO5fKyko2b97MunXreOihh5g5cyZHjhzhueeeY/LkyUyfPp2bbropotdHMXTw/S29tu04AKdlaoKenWRGJ7wt9FBWeE9CFb+saOSOV7bS2mk7ydkrXHQr6FLKm6WUWVJKo5QyV0r5Bynli1LKFz3GPCWlnCylnCql/HWvzthJX6UZjxkzBofDQVVVFenp6Xz44Yfs3LmT1157jQceeACAVatWcc4557Br1y4efPDBoON8OXDgAEuXLqWwsJCEhAReeOEF2tvbufPOO3nttdfYs2cPNpuN3/3ud37vbWlpYe7cuezevZtzzz2X3//+95x99tksWrSIp556il27djF27FhWrVpFQUEBhYWFvPjiiwFmoRjuBPotrd1VRoxJT2ZCNABGvY7sJLOXhR7KCu9JqOJ7eyr49GA1u443nOxHUDgZtJmifZlmLKUEwGq1cu+99zJt2jSWLFkS1LUR7ri8vDzmzZsHwG233cbGjRs5cOAA+fn5TJgwAYA77riDzz77zO+9JpOJK6+8EoDTTz+d4uLigOeYPn06t956K3/7299UNusw5nBVEw6HDLgv0G/J7pDY7NKrifjI5BgvCz2YFW7QaccM98l5X5lWBG9vhPsfDEcGraD3VZrx0aNH0ev1pKen88wzz5CRkcHu3bvZvn07nZ2dAd8T7jjPH4vrtevm0R1Go9H9fr1ej80W+HH13Xff5Vvf+hY7duzg9NNPDzpOMXR5p7CMi57+jE8PVgfcH+w341k2FyBvRAzH67rGPrRwIia9t4QIwOagR0/ORaWakHcX567onkEr6H2RZlxdXc2yZcu4//77EUJgsVjIyspCp9Px17/+Fbtds2ri4+Npampyvy/YOF+OHz/OF198AcA///lP5s+fz2mnnUZxcbG70uNf//pXzjvvvLDn7DkXh8PBiRMnuOCCC3jyySdpaGigubn5pK6FYnBS39LJY2/tBaCkPnDqfrDfTEK09xPdyJQYapo73L7uxbNyOC+1kWg62GP6Oluiv8Nlpt1+xwn15Fzd1EFFYzsAe8ssAccowmfQCnpvpRm3tbW5wxYvuugiLrnkEh577DEA7rvvPv785z8zd+5cDh48SGxsLKC5NQwGAzNmzOCZZ54JOs6XSZMm8ec//5np06dTV1fHN7/5TaKjo/njH//IkiVLmDZtGjqdjmXLloU9/5tuuomnnnqKWbNmcejQIW677TamTZvGrFmzePDBB1WZgmHGT9/dh6VNi0SpaQ78pBjotwSa28XTss5zhy5qVnpHwRq2VgoW6rYTr+sgk2p+K57gEt02sqnhWNQt7DAtZZFuY9CnAJeInzk6mSPVLbRbAxs/ivAYtE7V3kozDmZNA4wfP57CwkL361/84heA5v7YsGGD19hA43zR6XQBFyoXLFhAQUGB3/ZPPvnE/benpX399ddz/fXXAzBv3jwvn/3GjRuDfh7F4CdUluanB6t5Y2cp918wjn9sPU5Nc0fAY7jGr1i3l4a2rjDElk47G998gUv+829i2ioYGTMX+DYn6lqZmBnPJx+8joU7WKzf5H6PEJAvKvjEmYqSomvmKeNLJBtNwBV+53b5zZfMyWVrcR1fVjQxMy8pAldmeDJoBR1Uv0LF8CZUlubZ41J45I09jEmL5f4Lx7F+bwW1QSx00H5LT60/4CXol4itrBSriWnT3pfXokUjHy/axN6kBTzbeB4pWJiv2+N1rCTRTCcm2ogihg6ihJ3HbL+GZ/4NCx6F6Te4xxaVWhidEsPcMSkA7CtrVIJ+CgxqQR/MjB49mqKiPgnXVwxRgkV6rVi3F4eUtNsc/OOes4g26kmNiwpqobvwdYv8xPB3YkTXTSCZJmJp46Vdnfy0YCOJpPEz48sYhfccktHWcOqIJwbtnALAcgLedobwOkW9qMzC9NwkckeYiY82ePnRLa1Wfr3hIN+7eALx0cawr8twZtD60BWK4U4wv3RDm5VJWQm8/51zmDNaq8WSEmeitiW4hQ7a4ugi3UYu0W0jjyrydN5RMULABFFCvcPM0nPH8Ol1cKV+q99xkoTmDmyQcf4nsbbBhpWAJtgn6toQwPwnPqap3caa7Sfcfvu/b/2KP24qZsvRupDzVnShLHSFYpCSnWSmNICoJ5qNvLp0rldYbGpcFDVNISz0d77HxvZXwCi5sfP/yBY1+ETVAvCS6WmIzyb9si3AJKjYAttfAbrCbZOFZqHXBxJ0AEsJ0LUg+p+9le4QSatdsvzfhUgpWbPtBIA7CkbRPcpCVygGKcGiU1ZcNdkvxyE1zkRThy1wFMk734Ptf0AgEQJKZSrZojbgOdNNnaRf8r2uDVc+DdeuhsQ85wbBCA+XiyeHHDm8YLsKR4I2tsgp6L7x7u02B//fe/sprtXCLCstStDDRVnoCsUgxTfSS68TjEuP45rZuX5jU+K0Rs91LZ3+cec7/uT+0y4FFSQHEHQBibl+i5qA9tpjW1JzB/zsvzRMXwoHd2luFuB1+zm8ZF9E3MhruZ2uhKJA1DR3Eh9lwGjQKQu9BygL3QdX+dypU6eyZMmSbnuKhuLOO+/k9ddfj+DsFApvFs/KYdPyCyl6fCEOKblkSmbAcalOQQ+4MCq7rPYqRmBHT7ao6dqfmAcrGuDBIn8xD0CSWVvArEuaClc957TeBTXGbAB+vtvMsZoW9pZZiDYEliABLJqZTd4IM5VK0MNGCboPrvK5RUVFmEwmvzjxUHHqCkV/safUgkPCrCAhfylxJoDAoYuiy21TJrXwQbeFbjRrVnkPMOh1JEQbaGjt1G4ADxbBigaqR15GXrKZKIOe77xawNGaFs4/LT2g20gCN56RR0ZCNBXK5RI2g1vQC9fAM1NhRZL2/8I1ET38Oeecw+HDh/nkk0+44IILuOWWW5g2bRp2u52HHnqIM844g+nTp/PSSy8BWhGv+++/n8mTJ3PFFVdQVaXaqyr6hl0nGgCYnpsYcH9qrGahVwey0E+/0/1nqUwFIEfUgDFWs7DDsMp9SY41+dVKr2nqYEJ6PCuvnkJhiQUpYcnpufzi2mnkJJkRQIxJE/dJWQlMy0kkKzFauVx6wOD1oReu0WJanf65QDGup4LNZuP999/n0ksvBWDr1q0UFRWRn5/P6tWrSUxMZNu2bXR0dDBv3jwuueQSCgoKOHDgAHv27KGyspLJkydz1113nfJcFIru2HW8gZHJMW5fuS+p8SEs9Cuf1v6/409uCz1r9hWw+KmTnk9SjIn6Vu9z1TR3MC0nkUUzslm/t4L3iyqYlpNIevE6FkethOgS/mq8nv/rvIYb5+QihCAjMZqmdhutnTZiTINXrvqKwXuFNqzsEnMXrhjXUxB0Vy0X0Cz0u+++m82bN3PmmWeSn58PwH/+8x8KCwvd/nGLxcKhQ4f47LPPuPnmm9Hr9WRnZ3PhhRee9DwUip6wu6TBHXMeiBiTAbNRT22w5KIrn4Yrn6bsrSISCkqJPwUxBxgRY/R6GnA4JLUtnaTGmxBC8MslM7hrXj7pxeu8DLOFnR+y15jCddEtQL67HnuFpZ0xaUHCIBVuBq+gO2NZw94eJi4fui+eBbaklPzmN79h4cKFXmPee+89v3AxhaK3qWxsp9zSHjxlvnANvP8wqbZHqdn8Kex9FS57IqDhU9bQHpGKpSNiTRys7Ko3VN/aid0hSXM+QcSYDNoN6BlvwyxdNLBK/yJ8lgen39Al6I1K0MNh8PrQE/1Ds0JujyALFy7kd7/7HVar5iM8ePAgLS0tnHvuubz66qvY7XbKy8v5+OOPe30uCkWBs9NPQEEvXANr74O2OlKwUEsCtNXBW98KuObkKvJ1qozwcbm4Kj2mxvu4hLoxzDISNUFXkS7hMXgt9AWPevvQ4aRW5E+Ge+65h+LiYmbPno2UkrS0NNauXcs111zDRx99xLRp05gwYUKP6pgrFOHiW2HxtKx4DDrBlOwE/8EbVoJDMzxShYUSmaZtt3cGdE+WWdqYPSrplOeYHGuitdNOu9VOtFHvDpdM9fXxJ+Zq61++OA2zLpdL6Do0Co1uBV0I8QpwJVAlpZwaYtwZwBbgRill7wdfu76IG1Zqd/NgSQ89JFADiPPPP5/zzz/f/Vqn0/Hzn/+cn//8535jn3/++VM6v0IRikAVFsstbeQkmYkOEP7naQGnikZ2O8YG3AfQ0mGjodUaEQs9KUaLRW9otZKZqKe6KYigd2OYxUYZiI8yKAs9TMKx0P8EPA/8JdgAIYQeeAJYH5lphYlPhppCMdgJVd8cAldYdEgtAzQgHhZwCo3UkYBDCnRC+rknyy2aqOZEyOUCmu88MzHabaGn+bpcwjDMMhJVLHq4dCvoUsrPhBCjuxn2beDfwBmRmJRCMRwJVd/cJerBKiy2dAZJeFvwqOZDd1hJFRbs6GkgjkTRjt7HPVnaoIlmpHzogNuPXt3cgcmZcORHN4ZZZoKKRQ+XU14UFULkANcA/q13/McuFUJsF0Jsr64O3LA23CbJisigrvfAIVh9c89+nMHENt3X8nUx/Qa2zfz/qJNxpKDVTjnqyORh+zdYa5/nNdR1s4hMlIvmcqlvcba/a+okNc50UlFgGQnRyuUSJpGIcvk18LCUstuceCnlainlHCnlnLS0NL/90dHR1NbWKpHpI6SU1NbWEh0d3d9TURDc+vbcHqjCok7AI5dPCnrc7+4bz+yO1Txg+zYA11sf5/XOs/0aN5c1tKETkBHs5tADAlnofhEuYZKZGEVVUwd2h9KF7ohElMsc4FXnnTcVuFwIYZNSru3pgXJzcykpKSGY9a6IPNHR0eTm9n6op6J7gtU397SYPSssusbe/rXRIVsxhrpRFJY0MCollkSzkdKGNjITojHoT93Ocy2K1jt9+zVNHWQlnpzhkJkQjd0hqWnuICNBGR+hOGVBl1Lmu/4WQvwJeOdkxBy0ZsuubEyFYrjx0MKJXj50ALNRz0MLJ3qNc/XSXfLiZsot7fzkiuDWOQS/UaQnRHH1bzeRnWhmyem5vFNYTqfNwbxVH51yw/Uog55Yk95dz8WV9n8yZCZqN7QKS7sS9G7o9lYshPgn8AUwUQhRIoS4WwixTAixrPenp1AMHxbPyvEqVJWTZOYX107zEta1BaXMW/URo5e/y7bieuaMGoFh7+shi9QFctOYjXrOn5COlNDUbuXXGw7RadMaTbgWY12t4E6WEbEmGlo7vdL+TwbPbFFFaMKJcrk53INJKe88pdkoFMMcl/UdCN8oGAC5901sh17CYHeKXZAidVEGnft9Jr3gF9dOY/3eCjITohFAY7vN61yuxdhTsdJHxJioa+10p/37xaCHSUai9j61MNo9gzf1X6EYZgSKgnlU/KFLzF14NGJ23QQa2rpK2docErtDsvFwDedOSA1q+QbzvYdLUoyR+larO+3fLwY9TFJjozDohIpFDwMl6ArFIMFXYBeIbSQL/8xmwJ0FGiwR6Rfv76ep3ca5E9KChimeavhicqyJ+pbO4Gn/YaLTCdLjo5TLJQyUoCsUgwRfgX3c+FeChnU7s0CDWdk1zZ3oBMwflxrUx+67GNtTXAW6gqb994CMRBWLHg5K0BWKQYKv8Hr1/fTFmQUazMo26gXTc5NIijGFtRh7MoyIMdHUbnNb1mmnIOiZqhVdWAzeaosKxTBj8awcmjts/GRtEQBVIo1MAuRsmJPdC6KBQiEBbHbJueNTvY59qgLuiytb9HBVs5b2bz55uclIiObzQyFuYApACbpCMajIS44B4B8jXiKzrRoQaC2VnRjNWvMKJ56JSGUNbSSajTS0WZHAuRP8s7UjSZIzW/RQZdNJp/27yEyMprnDRnOHjbgoQ7dFzIYrStAVikFE4X//CYxlWutWTcuRuEU9MS9gCWlP6/ujLyu560/biY82BO9wdJL4iuyiGdkAHKpqZlz6qXUb8mxFV1Rq6baI2XBFCbpCMUDo1uosXENhST1jRBnxwnOx0ynmDxZ1ew7XwuS8sakRSfH3nLuvyL6y6RgArZ32U1oQBdwZouWWtpBFzIa7oKtFUYViALC2oJSH/rWb0oY2JJogPvSv3d7ZmhtWUugYw3Rx1P8AYfbSzUkyY9LrWDg1IzITdxJIZDucmadwaguiAJOy4jEb9by5s5SyhjYW6Tay0fQAR6NuYaPpARbpNp5y3PxQQFnoCsUAYMW6vVh9qglaHZIV6/a6rc7KhmYqSWa6LoCgh9lLNyUuik3LLyQ17uTS8IPRnZiebNq/i6QYE7fNHckfNh7jmpjd/Mz+MjFCS1jKFTWsMr5MstEEXHFK5xnsKAtdoRgAeGZyBtteGK31j/EXdNGjXrpp8VGntEAZiO6SkE7V5QKw9NyxmAw67LZOt5i7iBGd/ND4WljHWVtQyscHqk55PgMRJegKxSBhT94t6HAwRRR7bBUw565+b8UYLDkpyayFLkZC0NPio7jtrFG80zmbYoe/yyimrSKs4zzz34O89OkR92tXwbP85e8yb9VHp1yUrD9Rgq5QDABGOOuHh9peaM1hQhKYkzIAoS2EXrsarny6byYZgmDJSVlOy/1k67j4svS8MRhw8Lx9sf/OMNxOUkoqG9s5Uae5iFyLuZ5rF5GoNNlfKB+6QjEAeOyqKfzgX7vwWEcE4AeXaOn3UkoKSyxcNGkkXN99NEt/ECg5ac12rUF1JCx0gPT4aG6dCH8+MJ9v699klM7pOjGaw3I7NbbZaLc6KLO00WlzDLmIGWWhKxQDgMWzcnh80VT36/T4KPRCsun9f1D12ChKf3UOdS2dTMtN8nvvQHYZjIjVFkNPNcrFk6XXXoodPe+YLsX9pHLVc2G5nSqbtPIBUmrWeDht/wYTykJXKAYIs0aOAOCFW2dzufycZ1//kGc6FrOe3zC+UxPpGW3bgFHu9wSK/x5ISTYpsSZMhlNL+/clMzGaiRnxbEm4jW/d/ZsevdezwNeJutaw2v4NJpSFrlAMECoaNWHJSoyGDSv5jm4N/zX9gG/o36FRxpBMIxMLfur1nlAug4HAnWeP5tkbZ0Y8quasMcns+Koeq93poypcE7Jrk4vKxg7338frWnut0mR/oSx0hWKAUO6sJpiVaHYnCo3TlfFD3Wv8wLCGTgxENXp3FhroLoMxaXGMSTu1tP9AnJWfwl+++IqiUguzGj7UujRZnZ85SNcm6LLQjXrBibpWfnS51o91qNSF6VbQhRCvAFcCVVLKqQH23wo87HzZDHxTSrk7orNUKIYB5Q3t6HVCiwhJzNWEyYlOSKKxav5iD4aayyBczsxPBmDL0TpmFazsEnMXrq5NPoJe1dhOotlISpyJ43WtQO9UmuwvwnG5/Am4NMT+Y8B5UsrpwE+B1RGYl2IAMpAX34YC5ZZ2MqLt6J+d5hRzHzdFgEiOoeYyCJe0+CjGpsXyv2O1wcseBNhe2dhBRkIUI5Nj3II+lOhW0KWUnwF1IfZvllLWO19uAcLLQVYMKoZavO5ApOJIIZlthz0sc4kDLSKjgjS2TXs8YCXF3mhOMRiYOyaF7cX1NEZnB9zfas7021bZ1E5GQjQjk2M4MRwFvYfcDbwfbKcQYqkQYrsQYnt1dYDC/IoBy0BffBsMrHx7H8/+91DgnYVrKG9sJ0t42046oFSmMrf9WW7fNirgDXTxrBw2Lb+QY6uuYNPyC4eFmAOcNSaF5g4bj3TcSav0rhXTKk3c0PwDPj3orTNVjR2kx2uC3thuw9IauOTCYCVigi6EuABN0B8ONkZKuVpKOUdKOSctrXeL6ysiy0BffBsMbPiykrd2B36ike89TLlMJkvU+u3Ldm5TN1Bv5jr96O+0TWG59R5KHKk4pKDEkcp3O79JkTWLf23vWodwOLQs0YyEKHJHaI1Cjte1DilXYkSiXIQQ04GXgcuklP7fSEW/EonuLsNh8a23u+DUNnfS0mmjtdNGjMn7p9fY1kEb0WQKf+9mmUzp+lvdQN2kJ0STnxpLeUMb62zzWdc532/MB0UVrC0oZfGsHOpaO7E5pNvlAvDvnSW8tu3EgI3j7ymnbKELIUYCbwD/T0p58NSnpIgkkfJ9D/XFt95eI2i32mnusCEl7C9v8ttfLjVr09flIiU8aevymw+lG2gkmDsmGQREG7qkbJFuI9frPgFA5+jk8zdeYG1BqTtkMSMhirxk7Tq+sbNkSLkSuxV0IcQ/gS+AiUKIEiHE3UKIZUKIZc4hjwIpwAtCiF1CiO29OF9FD4mU73uoL7719hpBXUtXudd95Y1++8tNWvanr4XeQjTrHJrlOZRuoJHirPwU2q0O7r9wHDlJZhbpNrLK+DL7pXY9OzFyr3iTXe+upsqZVJSeEE18tJHkWBON7baAxx2sT0LdulyklDd3s/8e4J6IzUgRUSLp+x5K8bq+BLsepQ1tXPvCJp68fsYp9cWsbfYQ9LIAgj75btja5S8HsAsjvzQsQ3Qw6BNeeouzxmhPNnqdjk3LL6Tk0XuxoWe/HMU5ukI+d0ynVKZzT+ff2Nh4N9DVzi4vOYaWDptXZyUXg/VJSGWKDnEGg++7srGdpBgjUQZ994N7iWDXKSXWxM7jDRQcrz8pQXf55V3HjjLoAlroFXFT0IlDpCXGQ2M9JOaiX/AoK6bfwIoen3X4kJVoZvbIJN4sKGHZeWPI1tXyqX06DnTcrP+Izx3TOSxzuEC/y5327yoUNjI5htL6VnQddq+ns8H8JKRquQxxTtX33dsRAFJKLv31Z7zw8ZHuB/ciwa7TuRO0aCxLkI5CofD0y7uw2h3sK7Ngs3tbhWWWdtLjzRi+VwgrGrSGz/3ctGKwcP3peRysbGZPqYV2cybbHKdhwMb5ut2k0sBhmU27OZPKpnZ3sTCAvBFmGlqt/Gzx1CHjSlQW+hCl3WpHrxPuL+bJRG/0RSU/S5uV+lYrO76q735wLxLsOm0r1nzajSch6IH88g4JDrvkWE0L4zPi3dsrLO1kJUWfwicYvlwxPYvH397L6ztKWHnZSra+WsoUUUyM6GCcrpRDMpeYy1ZSVdDudreAZqHbHJIz85PZtPzCfvwEkUMJ+hDllt9vITk2it/ffvpJ+777ovh/TbP2GLy3zIKUMuJV+XpCoOv02jYtjjnY4lkoQq1T7Ctv9BL0cksbEzPjg45XBCfRbGThlEze2lXGDxZeQyHrudP8GTgE46Iaecs2FzntCio/3URGQldddlfo4om6VvKcfw92lMtlCCKlZF95I//dX8mG/T1rhuvpYgnkU4bIRgBUN2mLhfWtVne1wYHE0Zpm4ORcLqHWKX78ZpHbfSWlpNzSTmbCwFnXGGxcd3ouljYrT//nIJ0OwRnXfx9WNDDuortpsgqqmzqcSUVdFrpLxIdSTRcl6EOQmuZO2q2aj3blO/to97Gyg+Ebix2MSC6ouix0gL0Boj+647Vtx1n6l+294udvare6F9JORtAD+eVdNHfY3HHuje02WjvtZCuXy0kzf1wqGQlR/OWLYgDOGK01CxmXrj31fPnCzdQ0tZG+74/uWulZidEYdIIT9UrQFQOYEucX9M6zR3O8rpU/bDwWcJzvgufjb+/1c7H4EukIAG9Bt/T4/a9sLObDfZW9khB0rKYFACFOTtA9Y/cB0qljo+kBlug+JokmLrJ9ylPrD1DhfDLJTFSCfrLodYJrZ+fikDAxI56kGK22y7jajwHY0pSGAx0ZHcVarfTCNRj0OrKTzByvG5wx54FQgj4EKanXvqA3nzmShVMyeP6jw35ukkCZkfUhChX1VgRATXMHep1gTGrsSVnoR6qb/Z4mIpUQdKRac7eMS4s7KUGHrsJZZtqZp9tLrq6GqbqvaCCeh4yvcXrjh5RbPDoVKU6a62ZrhV7PyB/h3pax+XHiaGWTY4r2WjR01UoHxqXHUVQa2pAYTLVelKAPQVyCnjvCzE+umIxDSn76zj6vMYEWPIORk2TutUp+NU2dpMSamJKTGDDhpjtsjsDOoUj4+d/fUwHAoapmjlY3h/1D/sPGY2wv7sr4lFLiQEe6aABgsu4rAA7LPFYY/+LdqUhx0oxLj+M3N8/im+ePc28TjSWMFWXskWMAyBDOaCpnrfRzx6dyrKaFr2pbAh5zsJWNVoI+AIi0BVBS30pyrInYKAN5yTE8sGA87xdV8G5huXtMuIIXZdD1apJFTXMHqXFRTMlO0J4SPFLku6MzQIafi1P1868tKPVaUHZIwvohOxySVe/v569bvnJva+m004GJFKHdsE4TxwHYJ0cxgmbKLe3ohNa0QXFqXDUj2+3iAiAxl7GiDIdT6tyCnqhZ8+dPTAfgkwOBy3kPtrLRStD7kbUFpcx8/D9897VdEbUATtS3kTui60v9jXPHMD03kR/+ezdzf76B/OXvogsSHphkNqLz2HX6qBG9mmRR09xBarwm6BC4zkmo9wYiEn7+p9YfwC69rf9wfsg1LR1Y7ZLi2q6FtlrnPF2CHi/aGCUq+KftQr5p/Q5v7SolLT4Ko179HCPOgkcZZ9BuzDocpGDx6vw0OjWW/NRYPj4QOBpssJWNVt+gfsL1KNcQwDd7qhZASX2rl6Ab9Doum5pJS4edisZ2JPiJFWhC+JMrJuGQ8L2LJ3DjnDx2Hq+noTV8q7mn1DR3khpnYkp2ItCzhdGqpi5Bz4iPiqif/2RDNssaNPeJ+xG+cA01Ly8BIJmum9Wt+g3EizaOkotJr2PxzMGZmTjgmX4DjDkfgDQaqBapfp2fzp+YxhdHamnr9HdBBnvSG0ilMzxRgt5PdOfDDiUcJfWtWO2B3Q1SSkrr29wF/F38bcvxgOP1QngJ4VljtNrbmQnRfH3+aNqtDv659UTA954qUkqqmztIi4siOdZEdmJ0jxZGqxq74tZf+foZEfXzZwRxf3T3Qy53/rs1tFpp2LYG3n6A2mZtnqmi67MtNbzLu9H/x39uSuLD753n7j6viCxrC0p59rDmVqkkOWDnpwsmptNhc7DlqH8rh8FWNloJej/RnaXnKxy1zR38YeMxLnv2c+Y/8TF/3lwc8H3VzR102BxeFnqo8zmk9BLCClfN6MRoTstM4OyxKfzli2K/2iORoKnDRqfNQaqzWNLk7MRuIw48qfZwuTSfRCZnKALdFEz67tcTyjySo4rfeRKsbdRJLRY6RXh8tsQ89Ne8oOq19DJPrT/gV03xYvunzH3rPFiRBM9M5cyWjzEb9XwSwO0y2MpGq9T/fiJYdT/wtwAa261c8sxn1LZ0Mj03kbT4KL44Uss954zpelPhGtiwkpL6KGAlue/dAVtq3b7CL6IfIV1W0yhjeN6+mD/aL0OHg/iYWK9zu2OinRl1t39tFMv+tpPPD9dwgXMBKVLUOF0mqfFazPCU7AQ2fFkZsKNPIFz1rQFaOiMr6K6MwsyEaPdN7o6zR3X7Q/a8cX7lSGOm/hC1aO6kZFyNLYRWfEvR6/gaMq566TE43YiWE0S/9wBnp73ExweqWRGg/MRgKhutLPR+IlgW4YgYo58F8GV5E7UtnTx700zW3T+fCyamUXCiASmlJuRP5MMb94LlBCekVh0wT1Rr3ePX3gdvfYtMqtEJSNK18hPjP3jP9CPGiHJMbRXuzDnA3dXFJegXnJZOUoyRf+8oifg1qHHWCHdZ6FOyE4J29AmEpw+9KYSFvuOren70RmGPnjKOVDeTaDbyxY8u5OMfnA/ApKyEbt9XbmkjR1ePwEGxzACgRiYQTyvRwrle4oywUPQ+vk+6PzSsIUb4rAlZ25jXsJbjda3k/+i9AR9rHgol6P2E61FO77QGYkx6fn3jTB67agpPrT/gFcJ4sFITuDmjtWL+s0aOoK6lk682v65lvbV1xTyXOAU9R9RoGxxWsPsvak7UlXCV/gsqZDKt/75fuykUrqGysZ1oo44Es2YhRxn0TMtO5J3Cckb3MKyyucPGN/+2I2iMrytKxS3oOZoluy/MhdHqpg5SYk3ucwXj9R0n+OfWE6zbXRbWcQGOVrcwJi0WIQSJZiMQXrZoWUM7+ZSQRR1fOTIBqJUJ7ggXzwgLRe/jazhlu34XPlxk+9z990CPNQ+FEvR+5KLJGe5ok9kjtey2QEkM6/dWEGvSk+3MJHSN3fnpOi3rzYMSmUYKFmJE4JA+T0YLLXGmWGZoN4W3H6Di+CEyE6Ldj51rC0rZeqzrhtGTL/unB6p5v6iCj78MHBLmK+jZidEkmo3srwjPQq9uaic/VXMZhfKh73H65X/z0eGwrfSjNc2MSdUaWiREaze38AS9jSxTO6N0lW4LvZZELcJF6OGq55TfvA/x9YFXiNSA43T0PER1IKIEvR85UqWllkcbdVQ0tgdNYth6rI5xGfFukR2XHkdclIGClhS/Y5bINHKDWCG+5DsF/ZjM0jZY26gsLSa9sci9YLTr3dV0+IhguF/2zUe0ebhqovhS09SBEJDstLLf2lVGa6eNf/zveFhPAlVNHYxKiUUIaOmwBUzQ6rDZOVDRxGmZ8RyraQnLSncV5Rqbrt0sDHodcVGGbgW90+agurmDrHEzGK2v4SuXoMt4UvQtcM2LSsz7AVf5hWOrrmC14TZapclrf6s0eTXidjFQY81DEU6T6FeEEFVCiICrOELjOSHEYSFEoRBiduSnOTQ57BT0M0YnU2lpD/oF6rA5GO/R/kyvE8zMS2KnmOw3tlSmkis8st50RtCb/MaBp4We6d5WYY8n014OSLCc4IfWF1ik2+h/noa2bkV38xEtDOxoEEGvbu4kOcaEXifccflWu3QfP9STgMMhqWnuICMhirgoA7tONAR8uln96VGsdsm3LhjHpKyEsKx01w3IZaGDVnM7qKAXroFnplK5cgJSQk6sZPSUr1FLIo0yhloxgtRRU5WYDwD+3Hwmy633UOJIxSEFJY5UllvvcTfi9mSgxpqHIhwL/U/ApSH2XwaMd/63FPjdqU9reHCoqhmDTjB3TApNHbaQ1fbG+/SznDUyiS/t2bQaEt3bHFJoFrrOaaEn5sHiF+Dq32p/A4iuf/JY0UEGdRx1+nqlhAo5gkxXejQQIzp52vgiX5juQ4eD8XTFs5c2tPHga7v4ydo9fvMta2jjWE0Lep2gOIQP3eVu6WmKdUObFatdkhavCfr2r+oDvt9VaXJGbhLfWTA+LCvdVR97dGpXLH+C2UhjWwC3TqEWa47lBGVoT0xZhb9lVLx20yi+dz91JJIyUsWZDwSyk8ysc8xnfudzjOn4O/M7nwNgo+kBfmV4AYALxY4BHWseim4FXUr5GVAXYsjVwF+kxhYgSQiRFakJDmUOVzUzOjXWHTN+59mj/SJfTM508AkZ3t1sZo8cgV0KCs962inWgur4SXRiJPfK5bDC0tWXcvoN2t8rLPBYPVz7ezBrC6z5unK3hd5AHJ2YyBDe/9wG4SBL18A5ukKaieEx/SvufRL4+5bjfpa0yzpfcFo6pfVtdNj8k6i0tH/t6aGnKdZVTVo0Tnp8NHFRBloDZPmBJvyJZiN5yWYumZzhttLtQYp6QZeg53kkZyWaDf5t6ArXwJvL3OsY5VIT9Gx7GaP3anbN7hMN2B2SlLjAT0mKvsV3kXSRbiO/NL5Erq6GkTrtyfZWwwb+csZXgyZU0ZNI+NBzAM9UwhLnNj+EEEuFENuFENurqwMXwxlOHKluZnx6nDvmeUp2ol8Sw+JZ2QB+Hedn5iUBUGCc5RTrBkqWvAvgl1Tkx/Qb4OFjcO3vyY9q1nzoiXlUmEYDeFnonlyr30g5qUzTfcWxqFs4FnULO6OWcpVuo58lvflIDcmxJi6dmolDam2+fPG00HuaYu2KQU9PiCIu2kCUIfBX2agXTM1JQAiBTie4be5IjtW0uGvGB+JEXSspzuJmLhKifVwuLstcdt1IyqR2k8wStYxs0Z5aXL1SXesEiv7Fd5F0pemvmIT2b+hyVZbLFM7Y83g/zvLkiURiUaAqTwHNHynlamA1wJw5c0I1xRnydNjsfFXbwpXTs9wx3xWN7Vx/eq6XZfD423sxG/XeFeSAEbEmxqTGsvN4l/i6yubmdSfoLqbfQH7D6dS99yWWbxRQ8dk6+AQyReAHsgt0BRix8aFjDnP0hwBIppmnjS/y/UYArdGulJLNh2v52tgUxqRpN6Kj1S0UlTZ6NWGubuoS9IcWTvRqSA2hU6yrnTHoaXGayyUzMZqqxg6v90cbdFgdkmk5Se5to1O0hc7ShjZGpXgnVbk4HqDHpJ8PfcNKvwijMplKIs3Eig5IzCOjLYqdxxuArkgeRf/jlSi0oiuiKoN6TFi1XA5ri3bTHmTrHpGw0EuAPI/XuUD4Ab/DlOKaVhxSs7xdvvNKj9okroW2Q1+8zXhZjK7oX37HmDkyiYLjzgQjugQ9Jyn8hrf5zoW/Y7UtVKWcAbiyJIUWZudBgmhjrm4f/3HMwbO2l0E4+LXpBXhmKhSu4VhNCxWN7Zw9NoV8p2i+U1jut2jZaXe4P7PLcnKFZsZFGUKmWLuSitITooiPNmDS6/yebu67YBx2h2RaTtc6g+vGWFofPILhRF2bu4Gwi0SzEUtLm/YZVyRpSVs+lMsUskStO9Z8VEqs232jXC4DH52Q5IgaSqQzI9rZBGMwEQlBXwfc7ox2mQtYpJTl3b1puOOKcBmbFke0UU9SjNHduYZ3vgdvLAXLCQ45chjnOOZum+XJ7JEjqGnuYO7PNzB6+bv80un2uOjpT8NOinDFcR+raabC4hTJ734GKxq0MDufB7BLdNs5JrM4IrO9tgvQRO6NpWx69g4A5tm2khhjJDnWxH/3VwYsRrbxUFeI5eJZOWz+0QLyU2M5b2JaSB9mVVM7cVEGYkwGYk0GmjtsLJ6Vw0MLJ5KdZKasoY1XnAuinoLuunm6qiL6YrM7KG1oIy/Z+yknsfEAbXZBZ0MZQR5AKZPJ5Ig6d6z56JSum0JKrLLQByTOtSQXuaLanW3taoIxmAgnbPGfwBfARCFEiRDibiHEMiHEMueQ94CjwGHg98B9vTbbIcShqiaE0AQdnDVDLB2aaG9/BZBYZAyVJDNeV+rVNsuFq6xtpdNadclMT5J/RibHoBNwrFqzqlPjTJhc/ujpN8Ccu/AU9Yv0OwFY75gT5IiSzY4pZFPDqI+/DYVryE+NxdxZx86opW7f+7+MjwHwjPVn7ixVF3nJMQF97p5UN3W4G0LERRtobrf5dZdxlSbe8VWXCynaqCctPorShsDHL7e0Y3dIfwv92HsAWAjspgEok2lkje0KT3S5dITQSjooBiCXPeH1Mk9UubOtB2OJhnCiXG6WUmZJKY1Sylwp5R+klC9KKV907pdSym9JKcdKKadJKbf3/rQHP4ermskdYcZs0twamYnRVDS2wfsP45Lmw1KzUCcIp6XgYzH843+BS+JC+Mk/JoOO3BExHKttpbKx3b1A6+bKp+Ha1W5LJkvUMUMc4UP76QGP55CCLxyTOVu/F2HTbkKjdVXosZMsmhFCEzhXwao0nUXLUl17n/Zk8sxURh75B8dLy/yeSDyp8hD0+CgDzZ02nvzgy4BPAb/8z0Gv1zkhCqO5biR5DVu1G82KRFiRSGK7du0tsmtx+nP7VBqlGRC0JozBQixZY6a697v89SNiTBhU84qByfQbYM7duIyWXFFNHQm0GJIGZYkG9S3rJw5XNTMurUscshKjqaht8KrLcsihCfp44bS0fSyGcktgt4GLcDPd8lNjnS6XdvcCrRceUTEYY7lEv51dcjyVMslv6AGZSwPxfE3n7GFqKWFM9QaqSKZFdrkdaqQm6KmukrIOq/ZkYjnBSFFFg4zFsu7hoKJe3dRBuoeFLqV36dpQ1yEnyRzY5VK4huOvPQRA3ucPe/1bJKDF0rss9BqZwO3W5TyluxtWNFB22+fuY7twxbGrCJcBjstoScwjz5llfeLcXw66BVFQgt7nrC0o5exfbODLiia2Fde73SIZCdHUtOvolF0LkYdkLtF0OMOphJ/F0F0mW7iZbvmpsRyrbqGysZ30QILuYvoN8OMysqaeB8CH9tPxzbl0lRFw9c0kMZfRHdqTgmdGao3TQk/x6OLjejIZKbTaLyc6E4IuTFU1trst9Njq3QBsibqPI1G3cizqFt4y/hiA5w3P8kX0d7xuDDkjNAvd4fCoVrkiEd64l+OtRgzYyPKJ9EkUmqA3Sk2kDzpykeh4te0sRi9/l0ue/hSAn727z/1v6nK5pChBH/g4czXylr4KwInUc/t5QieHEvQ+xOXjdVmSzR02fvTGHrate4nMzSsAqGKEe/xBmcs4UYZAcmTUjX4WQ7ASvNCzrir5qbG0dNqpbekMbKH7fIZHirQF0Z/Y7ua7nfdRKlOdUiwolVrxo1xR4472yI/X9h7zFHSZSBJNGIW/iyTPKejHZXrAhamWDhstnXbS46OhcA1xe/4EQLOMQS8kQuDu8j5Dd5RMqrVF5hWJ8MxUclr202lzUPvEDK3ssIclfkKmkSNqMAjvW1Wi00JvJBYQ7DFMAcCKdv1do2uaO93rF3FRBtLio0hVzZ8HDa6Q3+7WcAYqStD7kEDp7RfbP2XGzkfItGphcJWyS9APO3IYL0qok3HcXnmj3/E8kyQAdynennZVcUW6AGQmhhafp9YfoN2jA8w6x3zmdTzH/Og32Tb7CQ7KPOJopZUYd+/G0RfdC/hY6DKxy90CeCZujvQU9AALU64Y9PT4KNiwkni7dpwWum5Gh2UOsbR51LVxnsByguwiLYuztM0/DeO4THef35MEp4Vuic6FFQ08a78+8AVCW79YsW4v81Z9RHVTB9uO1Q3KUqzDkeRYEzEmvTsE2BU+7CpWF2pdZyCgOhb1IYF82j80rMGEzZ2dWe7MNmySZspJYbSo4HHb7UH94b7dVNYWlPLU+gM8+Nounlp/gIcWTuxW2D0F3W9RNIzPAFpkze3bRtFm1wT4ax3PYt6m5xd5pSyefQOZb7/NUftoQKsZUy0TScWClFAn43jHMZcl+s+IEZ3EizaSaeS4yIYFV/udyzMGHUsJcUJzgzTLLhdTlRxBhqhHBEh7y5FaUbJSmcpMjnjtOyHTWajb5vcel4Vuyb8cIGipARcNbVZ3lE1VUwc/ekPLHB2M6eTDCSEEuSPMnKhv7coGdiWQWU5or2HA+teVhd6H+Pq0F+k2uhtRuHy2FU5B3+TQoiW+cExinWN+WP5w37C9cMMXs5PM7poxoQqEBfoMLvRChCyulZ+VxrHMhbDCwnzzm+yQE9kip5Df8Q9O71zNY7a7eNJ4n7suTZ6hgRMp5wT84bizROOjIDGXOLQfXBOegp5EepASBjlCqzNTJr3LDzfLaOpICGihm6KiMeslloSJSCkDpkeHYrDW1x6O5I1whs0GyAbG6opEG5goQe9DHlo4kWhnjPdIKlhlfNltQSbQQjQdbkF/3X4uKVjYIqeG7Q/vacVCF3qdYJQzCaY7H3qwLuh2GSTZxmnRj06NpdhZljbQZzEb9cy8Yqm7Ls3IKXM5bo33GwfehblY8ChxRu3czZ6CThJpBO58lEALcbS6/f0u3O37qMIunf8wiXladM8jZSTGmrG0WalobEcCRl3PZH0w1tcejuQlx1BS34ZsCJJY1FY3YF0vStD7kMWzcnjw4gkA3Kz/2Ku3oRCalV4hk6mWCXzsmMkImnrkD+9pxUJPRqfGEmXQuduthfoMnn57s1Hv9doXl0Xf2mGjvtXK6OXv8thbewGtE1CwTuojk82U1rcFrF1e1dSBUS9IMhth+g3ELdSif1qkGYQOKTULPU00ICV+kTjCYCJH1FLiIehSwnGH1pBijf08zjW/4V2xkq56LgecHZWWnTfW73PnJJmDJhENxvraw5HcEWaaO2w0JEwIPmiAlgVQPvQ+xlWk6UL9Lr99GaKOSjmCt3QXY0fP7x68hfEZga3UQGQHSZgJR0ium53r7qHZHS6//f/7w/+wtFndQvzwvwvp8FgwdT1ZrC0o5f2iCvd2S7sVnYAfXTaJm88aGfAcI5NjsDkk5ZZ2v0JZrqJeOqeFHDv7OnjjA5oX/AIu+BevbT1O+xt7eMV+Oa/YL2eRbiMPG9eQLWoRibmw4FGyP4mirEqHRFAmU3jCeoO7ycFmZhLfaSN/+btkJ5nd6xAuQT9UqZVtuGt+Pt93Pm28sbOEOaOSGZkS43Z9hVtoTDGwcH3fSk7/ISM+uTvgGIelhHUFpQNuTUQJeh+z60QDsSY945IEXiHYQCb1bBeTeT3xDGak6Hok5hB+xULXwqmr6qFLsC6dmul7yJCMToll7a5SpJQsnpVDUamFl531U3I8jjtv1Ud0+ljaDgk/enMPz398OODCretHdcJZ+fBQZRPvF1Ww+UgNO79qYEpOgntslEGPyaCjydlX9OkPvTND1znms65jPjlJZjY9qFWEzDm6h531ZsRjDWwrKGXH+gPguhkKqG/VFjRd6xAArZ02vqxo4n/H6tAJ+PRgtXve187uisZxbQt0jRUDH1f56RMp85hgTCLK2uA3xiEFG998AbhvQP27KkHvY3YdPMp0eQx943G0dOMu33OmoZGSzmSoaOKnV0/p8bHDERJf69FTsHr6xRyVEkNTu42GVisjYk3kOH8I239ykVe52FAun2Dnd9VSOVzdzJZjdbzw8WHsUjI5K4E7zh7FNbO8wxnjogw0d3RFlQTCcx45STFY2qzuol6LZ+Xw9T9uZePhGncbPBdtVjuPvFFIh0261wockpDXzTf6SDF4cBkTHxRV8FHHbawUq93u0WJHBs1EM1X3FStZzZPvGlg8a+DUTleC3oe071zD/roo7tXvRctHkbhFPTGPzNwFsEPrUnTVjOzQBwtCd0ISauG0pwLkqlVSXNvCiFgTpfVtRBt1fpmRwVxBoc6flWjGoBP89J19WO2Sa2fl8MgVk4LWFY+L0gp0gVYIy2Vh+86j629X1cU2dzeo43WtfmLuotXq78s/2eumGNgkRBtJNBvZsL+Sls6zsQgDM3VHed9xFkUyHxNWCqKWEis6uKfzb8DAEXS1KNqHFH34Z2wYmKnzjH3WxJwHi8icdDYAF01OJymmd9LFT2Xh1BdXrZKvarWsutKGNq0euY8fPlRGa7Dz63WCSVkJjIgx8fLtc3j6xpkhm0RoFrp2ozp3Qprffl/Xk+ux2lUX3eGQlNS3ERfVMxtHRa4MTfKSzbQ4cw0+lGfylP0mDNhYov+ETozskfkAZOtq+3GW/igLvQ/Z1aT5fWfpDnvvcKa3T8iIQ68T3HrWqF6bw6ksnPqSOyIGIXA3gS6pbyNnRIzfOE9XUDBLPdD5/3bPWZj0OndFylDERXe5XDISojHoBOnxUZRb2gO6nlxNQFzzqW7uoMPm4Krp2by+M/w62CpyZWgyMjmGotKuRa7XjI9zlv4AtTKef9nPZ7djHHN1X9JuziT8djK9j7LQ+5AC/TRyqCZdNHjvcKa3j0mLY8+KS5g3LtX/zREiWBz5yURgRBv1ZCeavSz0YP1MF8/KYdPyC/n1jTPDPn+i2RiWmIPLQtdcLtVNHWQkRLP5Rws4tuoKNi2/0M8tkhYfhUEn3IJ+tFq7KV05Iyvs2uUqcmXo8sCC8dw4J8+dN/J3+wJapYkU0USeqGKXYywSiGkrH1AlAZSg9yG7DNOYaSj23ugsYOUixtS7D02+TXJ7WvfFl1EpMRTXttDaaaOupTNoPHpvnd+Fpw+9qqldKwsQAr1OkJUUTWm9VnXx2Q0HiY8yMD03iceumhLwpnP+xC5XTqTmrRiYnJaZwBPXT2fVddPJSTLztmM+Txrvo9WcxUxxmN2OsV3Zwq6SAANA1JXLpY+oamqntEXHnbOmQ1me5mZxxkT3dV2ISEZgjEqJ5YOicrcvOpiF3lvnd6G5XDSfZ1VjB2PSgncWcpHjbFX3j63H2XK0jlXXTiM51hQ0Wih3hJlPDlRzwcQ0/vj1MyM6f8XAxPu7egXwODN+toy3m1Od5SUatF2ujmL9XONFCXofscvZ/X3WWefD6KJ+nUskyU+Nob7Vyr5yzd8YjqD3BvE+YYtnjUkOOX5tQSm7T1hos9rZ8VU9EzLiuPGMrl7ngW46hyq1DNEJPcwPUAwtZnXuAK5il2Msl+h3uLcPhGSjsFwuQohLhRAHhBCHhRDLA+xPFEK8LYTYLYTYK4T4euSnOrCob+kMuMDXbrUHjHxYv7eS+CgDUz0aFg8FXE0cNh/WVvtdi419TVyUgXarg5YOG5Y2q1bnJQi+sfgSLVLnrV1lIc+RlWQmPT6qV9c4FAOfKUlWDNjY7Rjrtb3MkRJ2L9/eIpwm0Xrgt8BlwGTgZiHEZJ9h3wL2SSlnAOcDvxJCDOk2LT9ZW8TVz2+i3Sem+2fv7uOipz+ltrkruaW108b7ReVcMT2L6G7C91ysLShl3qqPyF/+LvNWfTRg62m7YtE3Hq7BqBfutnB9Tawz3PCYswBYqHkEisXvsDm6LWIWF2Vg648vChgWqRg+RF/0CKfpStglx7m3SQlm2rnY/ilPfvCl+2murwnHQj8TOCylPCql7AReBXyLVEsgXmgByHFAHWCL6EwHEA6HZPORGmqaO3hrV5fQ1jR3sGZ7Ca2ddv7u0cB5/d4KWjvtXunhoTjZMrj9gSujs9Tpa9b1sAJhpIiL9hH0EIuikYzFVwxDpt/AjDE57HaMxe7Qvu9CQIqumVXGl8luLODiZz5jx1eByzf3JuEIeg5wwuN1iXObJ88Dk4AyYA/wHSmlX2qdEGKpEGK7EGJ7dXW17+5Bw5HqZupbreh1gpc/P4Z0poP/9Yuv6LQ5mJSVwF++KHZb7//eUUpespk5o0aEOqybky2D2x+YTXp3yd3+8p+D5kOHrvDDUC6XYLHjKqZcES4zZp5OM2aOkeW1PUZ0kqHT1pPW7ep7AywcQQ9kcvnmRy8EdgHZwEzgeSFEgs8YpJSrpZRzpJRz0tIG72Pr1mKtGcW3LhjHoapmPjlYTbvVzl+3fMWC09L5yRWTqGnu5K1dpZRb2th0pIZrZ+WGtF49XSzBkm8GqgXpqqXeXchib+Ky0I/WaJUQQ7lcIhmLrxiezMpLAmCX9PajSwk77Zor5s2CUuyOwKUkeotwBL0EyPN4nYtmiXvydeANqXEYOAacFpkpDjy2HqsjLT6KkSPMCAFf/+M2Tvu/D6hr6WRSVgJnj01hUlYCL39+jDd2liIlXDs7+Mq3r4slGAPRglxbUMqeUq2RxPtFFf3mFvL0oQuh9YYMRm/FwiuGD3tKLMTQzi7HOK/tR2Q25WidsBrbbTz9Yd8+VYcTtrgNGC+EyAdKgZuAW3zGHAcWAJ8LITKAicDRSE50oCClZOuxOnKSzPzozT34Nup58ZPDjEuP495z8vnemt288PFhzhg9wh0NEohALhZfBqIF6Rst0tRu67femZ4ul5TYKAz60LaKqoaoOFnWFpTy47VFxNJOgY+gf+yY4fX6T5uKeWhh39m23VroUkobcD+wHtgPrJFS7hVCLBNCLHMO+ylwthBiD7ABeFhKWdNbk+5PSurbKLe0c6ymJWBlPpvUBPrK6dlkJETREsZiaChXykC2IAeSr9/lcmnusPVbpI1ieOD63teQxD45iv32PBxSUOuIY5NjGnlUsdH0ANlU09Jpx2Z39FnUWliJRVLK94D3fLa96PF3GXBJZKc2MNnm9J9bnB3dF+k2kkgrW+Rkfm/4JU/br+fthvmYDDqWnTeW5zYc4vJpWaEOGbRgVk6SmU3LL4z8h4gQAylaxLNKYndp/wrFqeD5/ZbouMz6BIt0G/mZ4RV2OCZwpX4LuboaHja8ynds3+ZX/znInzYXR6QHQXeoTNEesq24joRoA3FRBuY0/ZdVxpe9eoOu0r1MstEEXMGdZ4/m1rNGYTKEfhAKt9PQQCOSlRtPlViPGjg9sdCDdW9SKIIR6Ht/n34dh8ijiRjm6zSxXqjfToytnT9/URyxHgTdoYpzdYPvo9KG/VWcMTqZH156GiuMf/ESc9DCln5ofA0AIUS3Yg6Dd5FuIEWL6HSCWGdlxrQwBX0wxfsrBg6BvvfrHWew0TEVgYOzdVoT9Ghh5RLddlo7A6+P9caTrLLQQxCoXRtAjEnPYv0mpGgO+L6YtoqA20MxGBfpBlrvzLhoAy2d9pAx6J5EsnuTYvjg+72PMuj4g/0K8qhkqihmhIcuXB5/hLWW+QGP0xtPskrQQxAs+mTL0TqoXBkwQB9w1zcfDgykG1FclIFKOsJ2uQykNQDF4MLze19UauHK32xkL/ks06/rGmQ0M+2CJbAWjDqB1SMmvbeeZJXLJQTBftjVzR3uLkMB8ahvrug74qK1xhThLoqqjFFFJJiak8jFkzMAOCe+AhBaW8mrniPzrOuJMen52tiUPnGpKgs9BKGiT4jK1Qrb+2JO7veayMOVuCjNr+nrcgm28DlYF6MVA48fXz6JnCQzZ1z+JnismwlgTFosQog+iVhTFnoIAi1+mPQ67Qe/4FGt25AnRjNc9kQfzlDhiSt00XNRNNTC52BdjFYMPEanxrJi0ZSAQRBj0+I4Uh14vS3SKAs9BK4f9pMffEmZpR2jXvDEda4fvNMK37CyX7sPKbqIizISH23wKlHc3cLnQFoDUAxNxqbFsW53GW2d9rB75J4sStC7YfGsHMwmPd/46w5+dcNMFs3I7to5/QYl4AOIr88bzbkTvJtPqIVPRX8zJi0WKbU6Q5Oz/WoWRhQl6N0gpeTFT4+Ql2zm8qmZ/T0dRQim5iT6dYQaSMlPiuHDtnUvkbfzKdJlNYlyBvAwR2uae13QlQ89ALXNHewts1Db3MHWDf+m4HgD9zb+DsNz0wdEZ29F+Ayk5CfF8GDbupeYuuMnZFKNTsAZYh8CB599/kmvn1tZ6D4UlVq45fdbaGx3NVwyk0wjS/SfgKUT3n5A26xcLYOCgZb8pBj65O18CrNHBnm0sJIramgoDRAVF2GUoNMV1lba0IYQkGQ28uxNM6l793EqWwVzdfu6/oGsbdpCqBL0QYNa+FT0Jemy2q8t0BhRRqlM7vVzD3tB9+sAL6G1046U8HXrGjAGaDkRKqlIoVAMa6pEGpl4t9gcK8r4n2MSDofs1b67w96HHiisbaHjM+a+dR7+nfacDKPUfoVC0TNOzH6INundMWukqKKdKCoa23v13MNe0H3D1xbpNrLK+LLfHdaN0axS+xUKRVDOWPQNik7/GRWk4ZCCCtJg4hUAvZ5gNOxdLr5hbT80rPEriesmMU8lDykUim45Y9E3YNE3AMgELm9qZ8XeDRypauac8Wm9dt5hb6E/tHAiBg+fVrYI1jlPwINFSswVCkWPSYuLIj7awJHqll49T1iCLoS4VAhxQAhxWAixPMiY84UQu4QQe4UQn0Z2mr3H4lk5TM1JRK8TCLQFjYAov7lCoTgJtq17icrHxzO2Yz/7t37ItnUv9dq5unW5CCH0wG+Bi4ESYJsQYp2Ucp/HmCTgBeBSKeVxIUR6L80XiHzbMEublYsnZfDi/zsdClu0WHOrh29d+c0VCsVJ4EoyMotOxohyNjumcPqOb/Pqzg1EX/1Mv7SgOxM4LKU8KqXsBF4FrvYZcwvwhpTyOICUsiqis/Qg0m3DmtqtHKtpYYorJXf6DXDVc5q/3KOusXK1KBSKnuKZZDRWV0YFKbQSzY1yPS1vfifi7Q7DEfQcwDPFqcS5zZMJwAghxCdCiB1CiNsjNUFfQlXPOxn2lTUCMHXbcliRBM9M1XY8WAQrGpTfXKFQnDTpsitabqwoA+CozEIIuEV8iH3d9yJ6vnAEPVAUvG+AtgE4HbgCWAj8nxBigt+BhFgqhNguhNheXR0kLLAbIl09b+92zd0/pW0bILWmFW8/oGq2KBSKU8ZzTW6sKMeIjQpnxqgQcI3jg4hqTTiCXgLkebzOBcoCjPlAStkipawBPgNm+B5ISrlaSjlHSjknLe3kQnci3TasaF8R6dSTLixdG61tVLzxCPnL32Xeqo9UF3iFQnFSnJj9EK5WomNFGfuj7uQS/Q73fp1AKyUSIcIR9G3AeCFEvhDCBNwErPMZ8xZwjhDCIISIAc4C9kdslh5Eunre3o50puiK/bany5qI+OgVCsXw5YxF3+DY6JuQgE5IDMLhPyiCpUS6FXQppQ24H1iPJtJrpJR7hRDLhBDLnGP2Ax8AhcBW4GUpZVHEZulBOG3Dqhrbmf3TD9mwvzLksdqtdg7LbKaKYr99ZTLF/fep+OgVCsXwZuzXX0LMuTtYIRFazZHrsxBWpqiU8j3gPZ9tL/q8fgp4KmIzC0F31fM+O1RDXUsnT60/wAUT04MWw/myogk7eqYYva3vVmniSZv3QqjqcKNQKE6aK5/mjR2lXOP4AE85apUmnrTeyIoInWZIZopuOVoLaIL93xBWelGp5jefcsld7jDFCtJYbr2HdY75XmNVhxuFQnEq/KDtdr5rvY8SRyoOKShxpLLceg9/bj4zYucYkrVcvjhSy0WTMjhU1cRvPjrMxZMzEMLfSt9bZiHRbCT3a9fD2UsA2FJQyodv7AFHV2ik6nCjUChOlewkM+sa5rOu09tYzImgsTjkLPQTda2UNrQxf1wK3zp/HHtKLXxyUAuRlFJScLye/+6r5D97K9h6rI6pOQleYh+Oj16hUCh6Sl+0QxxyFvoXTnfL18amMiYtlmc3HOK5DYfQHfucZzdVstM6ymv8FdOy/I6hOtwoFIpI0xftEIecoG85WktyrInx6XHodIJvnj+Wn6wt4o7j0eQQw08NrzBTdwRhMKE/7yHGzR/f31NWKBTDhN42FoeUoEsp2XKklrljkt2RLUvm5LJ3/R+YZt3D9fpPMQmnb9wB7HwczlvSfxNWKBSKCDKkBP1EXRtllnaWjemKIY8y6Pm5fA5h8I8ClZaSgHUNFAqFYjAyaAU9UAndDptmfX9tTAq88z3Y8SeQ9qDHqCSVyIX0KxQKRf8yKAXdVULXVXXRlZ4/JTuB1DgT496/FYq7emwIQEqtGI6LVmniF9YlPNvHc1coFIreYlAKerASugXHG7g0z4oo9m+YJIQm6hJBmUzhSdsN7Ei4uK+mrFAoFL3OoBT0YGn4dimZW/430AfcDcCYjr8DWvznL1SykEKhGEIMSkHPTjJT6iPqp4liztAd4irdpqDvswsdwvn+SMd/KhQKRX8zKAX9oYUTvXzoAH8wPU2OqAn5PsOcr3Psyit6e3oKhULRLwzK1H9Xer5Jr00/J8lMtqgNOl4ClSlz4cqn+2iGCoVC0fcMSkEHTdTjog3cfOZINi2/EJGYG3CcTer4Tud9nF/1oGpSoVAohjSDVtDrWjqpa+lkbFqstmHBo2D0rlrWKk18z7qMdY75qkmFQqEY8gxKHzrA0epmAMamxWkbpjsbUmxYiaOhxB2a6FnXXDWpUCgUQ5lBK+hHfAUdNFGffgPnrPrILwoGVJMKhUIxtBm0Lpcj+3ZgwkrOcznwzFQoXOPe1xd1hxUKhWKgEZagCyEuFUIcEEIcFkIsDzHuDCGEXQhxfeSmGIDCNRw5uI8xohy9cIDlBLz9gFvUVZMKhUIxHOnW5SKE0AO/BS4GSoBtQoh1Usp9AcY9AazvjYl6sWElRxwPMkUUd22ztsGGlW5fumpSoVAohhvhWOhnAoellEellJ3Aq8DVAcZ9G/g3UBXB+QWko6GC4zKdsaLMe4elpLdPrVAoFAOWcBZFc4ATHq9LgLM8BwghcoBrgAuBM4IdSAixFFgKMHLkyJ7O1c1XcTNwdOgYq/MRdI9Y9EDldZXFrlAohjLhWOiBekD4dov4NfCwlCGKjwNSytVSyjlSyjlpaWlhTtGfo5PvA/C20I1mLRadrvK6pQ1tSLrK66rEIoVCMZQJR9BLgDyP17mAj2nMHOBVIUQxcD3wghBicSQmGIgjMTMAyE/UAwIS8+Cq59z+82DldVVikUKhGMqE43LZBowXQuQDpcBNwC2eA6SU+a6/hRB/At6RUq6N3DS9OVLVTHZiNLHf3xlwf7AEIpVYpFAohjLdWuhSShtwP1r0yn5gjZRyrxBimRBiWW9PMBBHqpsZmx4XdH9SjDHgdpVYpFAohjJhZYpKKd8D3vPZ9mKQsXee+rQCs7aglCc/+JIySzuxJj1rC0r9FjrXFpTS3G7ze69RL1RikUKhGNIMmkxR10JnmaUdgG/Y/8mitVNofeI0ryzRp9YfwOrwXbOFWJNBRbkoFIohzaARdNdC59fEHgDm6A6iE5KYtnKvLNFgfnJLm7XP5qpQKBT9waARdJdQL9F/ziRR7B2D7soSJbifXPnPFQrFUGfQCLpLkBfrN/J+1CNkiAbvAc4sUVWYS6FQDFcGjaC7hLpMpgYekJjrzg5ts9rRCy0fShXmUigUw4VBI+iuCoovm26jVZq8dxrNbBv7bXd2KIBdSrdlrsRcoVAMBwaNoIMm6it+8jgx1/1Wyw71yBL97r7xKjtUoVAMawZnxyJnZyJPyv7xbsChKjtUoVAMFwaVhR4KFd2iUCiGO0NG0FV0i0KhGO4MTpdLAFwLn6oGukKhGK4MLkEvXKMlEFlKtGYWCx718qWrtnMKhWI4M3gEvXCNluJvdS5yuhpDg98CqUKhUAxHBo8PfcPKLjF34ZHyr1AoFMOdwSPowRpAq8bQCoVCAQwmQfdoAB3WdoVCoRhmDB5BX/Co1gjaE4/G0AqFQjHcGTyCPv0GrRG0T8q/WhBVKBQKjbCiXIQQlwLPAnrgZSnlKp/9twIPO182A9+UUu6O5ESBgCn/CoVCodDo1kIXQuiB3wKXAZOBm4UQk32GHQPOk1JOB34KrI70RBUKhUIRmnBcLmcCh6WUR6WUncCrwNWeA6SUm6WU9c6XWwC1UqlQKBR9TDiCngOc8Hhd4twWjLuB9wPtEEIsFUJsF0Jsr66uDn+WCoVCoeiWcARdBNgmAw4U4gI0QX840H4p5Wop5Rwp5Zy0tLTwZ6lQKBSKbglnUbQEyPN4nQuU+Q4SQkwHXgYuk1LWRmZ6CoVCoQgXIWVAY7trgBAG4CCwACgFtgG3SCn3eowZCXwE3C6l3BzWiYWoBr46yXmnAjUn+d7eZqDOTc2rZ6h59ZyBOrehNq9RUsqALo5uLXQppU0IcT+wHi1s8RUp5V4hxDLn/heBR4EU4AWhNWe2SSnndHPck/a5CCG2d3f8/mKgzk3Nq2eoefWcgTq34TSvsOLQpZTvAe/5bHvR4+97gHsiOTGFQqFQ9IzBkymqUCgUipAMVkEfyIlLA3Vual49Q82r5wzUuQ2beXW7KKpQKBSKwcFgtdAVCoVC4YMSdIVCoRgiDApBF0IsEULsFUI4hBBBw3yEEJcKIQ4IIQ4LIZb30dyShRAfCiEOOf8/Isi4YiHEHiHELiHE9l6aS8jPLzSec+4vFELM7o15nOTczhdCWJzXZ5cQotcL3QshXhFCVAkhioLs78/r1d3c+uN65QkhPhZC7Hf+Hr8TYEy/XLMw59Yf1yxaCLFVCLHbOa/HA4yJ3DWTUg74/4BJwETgE2BOkDF64AgwBjABu4HJfTC3J4Hlzr+XA08EGVcMpPbiPLr9/MDlaHV2BDAX+F8f/fuFM7fzgXf6+Ht1LjAbKAqyv1+uV5hz64/rlQXMdv4dj5ZwOFC+Y+HMrT+umQDinH8bgf8Bc3vrmg0KC11KuV9KeaCbYd1Whewlrgb+7Pz7z8DiPjhnIML5/FcDf5EaW4AkIUTWAJlbnyOl/AyoCzGkv65XOHPrc6SU5VLKnc6/m4D9+Bfq65drFubc+hzndWh2vjQ6//ONRInYNRsUgh4mPa0KGSkypJTloH2pgPQg4yTwHyHEDiHE0l6YRzifv7+uUbjn/Zrz0fR9IcSUPphXd/TX9QqXfrteQojRwCw0i9OTfr9mIeYG/XDNhBB6IcQuoAr4UErZa9csrEzRvkAI8V8gM8CuH0sp3wrnEAG2RSQmM9TcenCYeVLKMiFEOvChEOJLpxUWKcL5/L12jbohnPPuRKtR0SyEuBxYC4zv7Yl1Q39dr3Dot+slhIgD/g18V0rZ6Ls7wFv67Jp1M7d+uWZSSjswUwiRBLwphJgqpfRcG4nYNRswgi6lvOgUDxFWVciTIdTchBCVQogsKWW58zGpKsgxypz/rxJCvInmhoikoIfz+XvtGnVDt+f1/PFJKd8TQrwghEiVUvZnUaX+ul7d0l/XSwhhRBPMv0sp3wgwpN+uWXdz6+/vmJSyQQjxCXAp4CnoEbtmQ8nlsg0YL4TIF0KYgJuAdX1w3nXAHc6/7wD8niaEELFCiHjX38AleP+DRoJwPv864HbnqvpcwOJyF/Uy3c5NCJEphFbZTQhxJtp3s7/LMPfX9eqW/rhezvP9AdgvpXw6yLB+uWbhzK2frlma0zJHCGEGLgK+9BkWuWvWlyu+J/sfcA3aXawDqATWO7dnA+95jLscbXX7CJqrpi/mlgJsAA45/5/sOze06I7dzv/29tbcAn1+YBmwTHatuP/WuX8PQSKG+mlu9zuvzW60NoZn98Gc/gmUA1bn9+vuAXS9uptbf1yv+WiugEJgl/O/ywfCNQtzbv1xzaYDBc55FQGPOrf3yjVTqf8KhUIxRBhKLheFQqEY1ihBVygUiiGCEnSFQqEYIihBVygUiiGCEnSFQqEYIihBVygUiiGCEnSFQqEYIvz/VyoyLfeoOGMAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_61_1.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "from sklearn.linear_model import LinearRegression, Ridge, Lasso\n", "from sklearn.preprocessing import PolynomialFeatures\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.pipeline import make_pipeline\n", "from sklearn.utils import resample\n", "\n", "np.random.seed(2018)\n", "\n", "n = 500\n", "n_boostraps = 100\n", "degree = 18 # A quite high value, just to show.\n", "noise = 0.1\n", "\n", "# Make data set.\n", "x = np.linspace(-1, 3, n).reshape(-1, 1)\n", "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2) + np.random.normal(0, 0.1, x.shape)\n", "\n", "# Hold out some test data that is never used in training.\n", "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2)\n", "\n", "# Combine x transformation and model into one operation.\n", "# Not neccesary, but convenient.\n", "model = make_pipeline(PolynomialFeatures(degree=degree), LinearRegression(fit_intercept=False))\n", "\n", "# The following (m x n_bootstraps) matrix holds the column vectors y_pred\n", "# for each bootstrap iteration.\n", "y_pred = np.empty((y_test.shape[0], n_boostraps))\n", "for i in range(n_boostraps):\n", " x_, y_ = resample(x_train, y_train)\n", "\n", " # Evaluate the new model on the same test data each time.\n", " y_pred[:, i] = model.fit(x_, y_).predict(x_test).ravel()\n", "\n", "# Note: Expectations and variances taken w.r.t. different training\n", "# data sets, hence the axis=1. Subsequent means are taken across the test data\n", "# set in order to obtain a total value, but before this we have error/bias/variance\n", "# calculated per data point in the test set.\n", "# Note 2: The use of keepdims=True is important in the calculation of bias as this \n", "# maintains the column vector form. Dropping this yields very unexpected results.\n", "error = np.mean( np.mean((y_test - y_pred)**2, axis=1, keepdims=True) )\n", "bias = np.mean( (y_test - np.mean(y_pred, axis=1, keepdims=True))**2 )\n", "variance = np.mean( np.var(y_pred, axis=1, keepdims=True) )\n", "print('Error:', error)\n", "print('Bias^2:', bias)\n", "print('Var:', variance)\n", "print('{} >= {} + {} = {}'.format(error, bias, variance, bias+variance))\n", "\n", "plt.plot(x[::5, :], y[::5, :], label='f(x)')\n", "plt.scatter(x_test, y_test, label='Data points')\n", "plt.scatter(x_test, np.mean(y_pred, axis=1), label='Pred')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Polynomial degree: 0\n", "Error: 0.3214960170351912\n", "Bias^2: 0.3123314713548606\n", "Var: 0.009164545680330616\n", "0.3214960170351912 >= 0.3123314713548606 + 0.009164545680330616 = 0.3214960170351912\n", "Polynomial degree: 1\n", "Error: 0.08426840630693411\n", "Bias^2: 0.07968918676726029\n", "Var: 0.004579219539673836\n", "0.08426840630693411 >= 0.07968918676726029 + 0.004579219539673836 = 0.08426840630693413\n", "Polynomial degree: 2\n", "Error: 0.10398646080125035\n", "Bias^2: 0.10077114273548984\n", "Var: 0.003215318065760509\n", "0.10398646080125035 >= 0.10077114273548984 + 0.003215318065760509 = 0.10398646080125035\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Polynomial degree: 3\n", "Error: 0.06547790180152357\n", "Bias^2: 0.06208238634231953\n", "Var: 0.0033955154592040944\n", "0.06547790180152357 >= 0.06208238634231953 + 0.0033955154592040944 = 0.06547790180152363\n", "Polynomial degree: 4\n", "Error: 0.06844519414009438\n", "Bias^2: 0.06453579006728315\n", "Var: 0.003909404072811231\n", "0.06844519414009438 >= 0.06453579006728315 + 0.003909404072811231 = 0.06844519414009438\n", "Polynomial degree: 5\n", "Error: 0.05227921801205692\n", "Bias^2: 0.04818727730430296\n", "Var: 0.0040919407077539514\n", "0.05227921801205692 >= 0.04818727730430296 + 0.0040919407077539514 = 0.05227921801205691\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Polynomial degree: 6\n", "Error: 0.03781367141738885\n", "Bias^2: 0.033657685071527485\n", "Var: 0.004155986345861374\n", "0.03781367141738885 >= 0.033657685071527485 + 0.004155986345861374 = 0.03781367141738886\n", "Polynomial degree: 7\n", "Error: 0.027609773491022314\n", "Bias^2: 0.02299949826036602\n", "Var: 0.004610275230656294\n", "0.027609773491022314 >= 0.02299949826036602 + 0.004610275230656294 = 0.027609773491022314\n", "Polynomial degree: 8\n", "Error: 0.017355848195591845\n", "Bias^2: 0.01033172130665515\n", "Var: 0.007024126888936694\n", "0.017355848195591845 >= 0.01033172130665515 + 0.007024126888936694 = 0.01735584819559184\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Polynomial degree: 9\n", "Error: 0.026605727637176654\n", "Bias^2: 0.010018312644139347\n", "Var: 0.016587414993037307\n", "0.026605727637176654 >= 0.010018312644139347 + 0.016587414993037307 = 0.026605727637176654\n", "Polynomial degree: 10\n", "Error: 0.02159270458799264\n", "Bias^2: 0.010516485576652856\n", "Var: 0.011076219011339788\n", "0.02159270458799264 >= 0.010516485576652856 + 0.011076219011339788 = 0.021592704587992645\n", "Polynomial degree: 11\n", "Error: 0.07160048164248561\n", "Bias^2: 0.014436800088969727\n", "Var: 0.05716368155351588\n", "0.07160048164248561 >= 0.014436800088969727 + 0.05716368155351588 = 0.07160048164248561\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Polynomial degree: 12\n", "Error: 0.11547777218940905\n", "Bias^2: 0.016285782696075054\n", "Var: 0.099191989493334\n", "0.11547777218940905 >= 0.016285782696075054 + 0.099191989493334 = 0.11547777218940906\n", "Polynomial degree: 13\n", "Error: 0.22842468702288576\n", "Bias^2: 0.01975416527179247\n", "Var: 0.20867052175109335\n", "0.22842468702288576 >= 0.01975416527179247 + 0.20867052175109335 = 0.22842468702288582\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA+QElEQVR4nO3deXxU5dn/8c81M9nIQsjGFkKAhH0ngAoqKiK4FFoXRKWutdha28fH/tS22pa2T9fHWq2Wuu8i+pS6ASpuQJUloLIvAQJkIwkhZN9m7t8fZxImCzBAkpNMrvfrNa+ZOcvMNSzfOXOf+9y3GGNQSikVuBx2F6CUUqptadArpVSA06BXSqkAp0GvlFIBToNeKaUCnMvuAloSFxdnkpOT7S5DKaU6jY0bNxYaY+JbWtchgz45OZn09HS7y1BKqU5DRA6caJ023SilVIDToFdKqQCnQa+UUgGuQ7bRK6UCX21tLVlZWVRVVdldSqcSGhpKYmIiQUFBfu+jQa+UskVWVhaRkZEkJycjInaX0ykYYzhy5AhZWVkMGDDA7/206UYpZYuqqipiY2M15E+DiBAbG3vav4I06JVSttGQP31n8mcWMEFf5/bwxKcZrNpdYHcpSinVoQRM0DsdwlOr9rFiW57dpSilOgmn08nYsWMbbn/4wx/sLqlNBMzJWBEhJSGCjPwyu0tRSnUSYWFhfP311yfdxu1243Q6T/jc3/3sFDBH9ACD4sPZV6BBr5Q6O8nJySxcuJCpU6fy5ptvNnv++uuvM2rUKEaOHMn999/fsF9ERAQPP/wwkydP5ssvv7TxEzQWMEf0ACkJESxJz6K4oobobsF2l6OU8tOv393G9pySVn3N4X2i+OVVI066TWVlJWPHjm14/uCDDzJ37lzA6q++Zs0aAB544IGG5zk5OZxzzjls3LiRHj16MGPGDP79738zZ84cysvLGTlyJAsXLmzVz3K2Ai7oATLyy0hLjrG5GqVUR3eyppv6wG/6fMOGDUybNo34eGugyBtvvJFVq1YxZ84cnE4nV199dZvWfCYCJ+jddZz32Txudw5jb8EoDXqlOpFTHXnbITw8vMXnxpgT7hMaGtph2uV9BU4bvdNFSOVhxjj36wlZpVSbmTx5Mp9//jmFhYW43W5ef/11LrzwQrvLOqnAOaIHJH4ow8v2sVSDXinlh6Zt9DNnzjxlF8vevXvz+9//nosuughjDJdffjmzZ89u40rPTkAFPXFDSNq7mr35rXtSRykVmNxud4vLMzMzT/r8hhtu4IYbbmi2X1lZxzzIDJymG4D4IQSbakzxIapqW/4LVEqprsavoBeRmSKyS0QyROSBFtbPFpHNIvK1iKSLyFR/921V8UMASJFs9hWUt+lbKaVUZ3HKoBcRJ/AEMAsYDswTkeFNNvsYGGOMGQvcBjxzGvu2nrjBgBX0GXrhlFJKAf4d0U8CMowx+4wxNcBioNGZB2NMmTne5ygcMP7u26q6xWDCE0h1ZLNXT8gqpRTgX9D3BQ75PM/yLmtERL4tIjuB97GO6v3etzVJ/BCGB+XpEb1SSnn5E/QtDX7c7IoBY8xSY8xQYA7wm9PZF0BE7vS276cXFJzFUMPxQxhoDrH3cOmZv4ZSSgUQf4I+C+jn8zwRyDnRxsaYVcAgEYk7nX2NMU8ZY9KMMWn1lxafkfihdDMVlB3Jwu058RVsSimVmZnJyJEjmy2/44472L59uw0VtQ1/+tFvAFJFZACQDVwPNOpAKiIpwF5jjBGR8UAwcAQoPtW+rc57Qra/5xBZRyvoHxt+ih2UUqqxZ555xu4SWtUpj+iNMXXA3cAHwA5giTFmm4gsEJEF3s2uBraKyNdYvWzmGkuL+7bB5zgufigAKZKjQyEopU6prq6Om2++mdGjR3PNNddQUVHBtGnTSE9PB+Cuu+4iLS2NESNG8Mtf/rJhvwceeIDhw4czevRo7rvvPrvK94tfV8YaY5YBy5osW+Tz+I/AH/3dt01FJOAJ6U5qXRZ7C8q4ZFjPdntrpdQZWv4A5G1p3dfsNQpmnXrGqF27dvHss88yZcoUbrvtNp588slG63/3u98RExOD2+3mkksuYfPmzSQmJrJ06VJ27tyJiFBcXNy6tbeywLoyFkAER8JQhrly9YheKXVK/fr1Y8qUKQDcdNNNDWPQ11uyZAnjx49n3LhxbNu2je3btxMVFUVoaCh33HEH//rXv+jWrZsdpfstsMa6qRc3mJTsdzToleos/DjybisicsLn+/fv5y9/+QsbNmygR48e3HLLLVRVVeFyuVi/fj0ff/wxixcv5u9//zuffPJJe5fut8A7ogeIH0p3zzGO5OeedOxopZQ6ePBgw7R/r7/+OlOnNozgQklJCeHh4XTv3p3Dhw+zfPlywBq87NixY1x++eU8+uijp5x31m4BGvTWmDcJ1QcoLKuxuRilVEc2bNgwXnzxRUaPHk1RURF33XVXw7oxY8Ywbtw4RowYwW233dbQxFNaWsqVV17J6NGjufDCC/nrX/9qV/l+CdimG4AURzYZ+WXER4bYXJBSqiNKTk5usb/8Z5991vD4hRdeaHHf9evXt1FVrS8wj+i798PjCiNVstmrQyEopbq4wAx6hwOJH8xgp/alV0qpwAx6rGkFhzhz9YheKdXlBWzQEzeYeE8BuYfPYoA0pZQKAIEb9N6hELqV7qO8us7mYpRSyj4BHPRWF0s9IauU6uoCN+h7DMA4gkhxaNArpZqbNm0aH3zwQaNljz76KD/4wQ/82v/hhx9m5cqVbVFaqwvcoHe6MLGDSPX2pVdKKV/z5s1j8eLFjZYtXryYefPmnXJft9vNwoULmT59eluV16oCN+gBR/xQhjp1cDOlVHPXXHMN7733HtXV1YA1CUlOTg6vvfZai8MSJycns3DhQqZOncqbb77JLbfcwltvvQXAwoULmThxIiNHjuTOO+9sGHpl2rRp3H///UyaNInBgwezevVqwPqiuO+++xg1ahSjR4/m8ccfB2Djxo1ceOGFTJgwgcsuu4zc3NxW+ayBeWVsvfgh9Nn+Dgfzj9pdiVLqJP64/o/sLNrZqq85NGYo90+6/4TrY2NjmTRpEitWrGD27NksXryYuXPn8uCDDzYblnj06NEAhIaGNoxuuWLFiobXuvvuu3n44YcBmD9/Pu+99x5XXXUVYI13v379epYtW8avf/1rVq5cyVNPPcX+/fv56quvcLlcFBUVUVtby49+9CPefvtt4uPjeeONN/j5z3/Oc889d9Z/FgF9RE/8EBx4cBRlUOv22F2NUqqD8W2+qW+2aWlY4npz585t8XU+/fRTJk+ezKhRo/jkk0/Ytu34/Erf+c53AJgwYQKZmZkArFy5kgULFuByWcfaMTEx7Nq1i61bt3LppZcyduxYfvvb35KVldUqnzOwj+jjrJ43A00WB45UkJIQYXNBSqmWnOzIuy3NmTOHe++9l02bNlFZWUmPHj1aHJa4Xnh486lJq6qq+MEPfkB6ejr9+vXjV7/6VaN9QkKssbacTid1dVZXb2NMs+GRjTGMGDGiYSTN1hTYR/SxKRhxkOLI0Z43SqlmIiIimDZtGrfddhvz5s074bDEJ1Mf6nFxcZSVlTW025/MjBkzWLRoUUPwFxUVMWTIEAoKChqCvra2ttEvg7MR2EEfFIqJ7k+KZOkJWaVUi+bNm8c333zD9ddff8JhiU8mOjqa733ve4waNYo5c+YwceLEU+5zxx13kJSUxOjRoxkzZgyvvfYawcHBvPXWW9x///2MGTOGsWPH8sUXX7TGR0Q64sQcaWlppn5i3rP2+jz27t7CE0Nf4ZG5Y1vnNZVSZ23Hjh0MGzbM7jI6pZb+7ERkozEmraXtA/uIHiBuMP1NLvvzj9ldiVJK2SLwgz5+KC7qqC3cp9MKKqW6JL+CXkRmisguEckQkQdaWH+jiGz23r4QkTE+6zJFZIuIfC0irdQecxrirdmm+tYeIK+k6hQbK6Xakx58nb4z+TM7ZdCLiBN4ApgFDAfmicjwJpvtBy40xowGfgM81WT9RcaYsSdqP2pT3mkFB0k2e/PL2/3tlVItCw0N5ciRIxr2p8EYw5EjRwgNDT2t/fzpRz8JyDDG7AMQkcXAbKDhKgJjjO+p4bVA4mlV0ZZCInFH9iW1OJuM/FKmpsbZXZFSCkhMTCQrK4uCAp0z4nSEhoaSmHh6EetP0PcFDvk8zwImn2T72wHfzqcG+FBEDPBPY0zTo/0250gYypDSfbymfemV6jCCgoIYMGCA3WV0Cf4EvbSwrMXfWiJyEVbQT/VZPMUYkyMiCcBHIrLTGLOqhX3vBO4ESEpK8qMs/0n8EAbtXcPewyWt+rpKKdUZ+HMyNgvo5/M8EchpupGIjAaeAWYbY47ULzfG5Hjv84GlWE1BzRhjnjLGpBlj0uLj4/3/BP6IH0II1ZQXHGjd11VKqU7An6DfAKSKyAARCQauB97x3UBEkoB/AfONMbt9loeLSGT9Y2AGsLW1ivebd8ybmIp9HKusbfe3V0opO52y6cYYUycidwMfAE7gOWPMNhFZ4F2/CHgYiAWe9A7UU+ftYdMTWOpd5gJeM8asaOFt2pZ3WsEUySEjv4wJ/Xu0ewlKKWUXv0avNMYsA5Y1WbbI5/EdwB0t7LcPGNN0ebvrFoM7LI7UOmtaQQ16pVRXEvhXxno5EoaQ6shmrw5uppTqYrpM0Ev8UFIdOWQcLrW7FKWUalddJuiJH0Ik5RzNP3TqbZVSKoB0qaAHCDu2l6pat83FKKVU++k6Qe/tYjlIssg8omPeKKW6jq4T9JG9cAdHkSI5OriZUqpL6TpBL4LEDybVka3TCiqlupSuE/RYg5sNduSQoYObKaU6mJfXHuDeJV9T6/a0+mt3qaAnbgixFHM4L9fuSpRSqpG30g+x53AZQc7Wj+WuFfTxQwFwFu3G49HJDpRSHcOhogq+yTrGFaN7t8nrd7Ggt2ab6u85RHZxpc3FKKWUZdkWq5XhilEa9GevexJuZygpkq3t9EqpDmPZllxGJ3anX0y3Nnn9rhX0DgcmNpVU0TFvlFIdQ32zzeVtdDQPXS3oAVfPYQx25mgXS6VUh9DWzTbQBYOe+MH0ppDswzohsVLKfm3dbANdMuitnjeegt2n2FAppdpWezTbQFcMeu+YNz2rMzlSVm1zMUqprqw9mm2gKwZ9zAA8jiBSHDnsLdAxb5RS9nm/HZptoCsGvTMId/QAUkXHvFFK2edQUQWbs461+dE8dMWgx+p5k+rQnjdKKfu87222qW+fX5+7nnf3vkudp67V36tLBr3ED6GfHOZgfpHdpSiluqhlW3IZ49Ns8+L2F3n8q8dxiI510zrih+DEQ/XhPXZXopTqguqbbeqP5o9VH+OL7C+YmTxTg77VeKcV7F62l4qa1v+ZpJRSJ9O02WblgZXUmTpmDpjZJu/nV9CLyEwR2SUiGSLyQAvrbxSRzd7bFyIyxt99bRGbgkFIcWSzT3veKKXaWdNmm+WZy+kf1Z9hMcPa5P1OGfQi4gSeAGYBw4F5IjK8yWb7gQuNMaOB3wBPnca+7S8ojNqoJGtaQR3cTCnVjg4eadxsU1hZyIa8DcxMnomItMl7+nNEPwnIMMbsM8bUAIuB2b4bGGO+MMYc9T5dCyT6u69dnD2HahdLpVS7W7a1cbPNh5kf4jEeZg2Y1Wbv6U/Q9wUO+TzP8i47kduB5ae7r4jcKSLpIpJeUND249A4E4YywJHL/vxjbf5eSilV7/3NTZpt9i8ntUcqg6IHtdl7+hP0Lf2WaHF6JhG5CCvo7z/dfY0xTxlj0owxafHx8X6UdZbihhBMHeV5GW3/XkophdVssyX7+ExSOWU5fF3wNZcPuLxN39efoM8C+vk8TwRymm4kIqOBZ4DZxpgjp7OvLbyDm4UUZ1DXBpPxKqVUU/W9bWaNtIL+g8wPALgs+bI2fV9/gn4DkCoiA0QkGLgeeMd3AxFJAv4FzDfG7D6dfW0TlwrAQJPFoaM6raBSqu0t25LLmH7RjZptRsWNol9kv1PseXZOGfTGmDrgbuADYAewxBizTUQWiMgC72YPA7HAkyLytYikn2zfNvgcpy80iprw3gxy6AlZpVTba2i2GdULgMxjmewo2sHM5LbpO+/L5c9GxphlwLImyxb5PL4DuMPffTsKR/wQUksP8J/8Mi4d3tPucpRSAaxps82KzBUI0ubNNtBVr4z1cvUcRoojl335JXaXopQKcL7NNsYYlu9fzvie4+kZ3vYHmV066IkfTDeqKM7bb3clSqkA1rTZZvfR3ew7tq/Ne9vU6+JBb/W8cR7ZgzEt9vpUSqmz1nRsmxWZK3CKk+n9p7fL+3ftoPdOK9i37gD5pTqtoFKqbby/JYcx/aJJ7HG82eac3ucQExrTLu/ftYM+PJaakBhrzBvteaOUagMHjpSzNbuEK71H81sLt5Jdlt1mI1W2pGsHPUDcEFId2WTo4GZKqTbQ0NvG2z6/PHM5QY4gLk66uN1q6PJBH9RrmDW42eFSu0tRSgWgZVtyGetttvEYDx/s/4CpfacSFRzVbjV0+aCX+CF0l3IKDx869cZKKXUa6ptt6icA33R4E/mV+W06UmVLunzQ1882RcEue+tQSgWcZs02+5cT5grjwsQL27UODXpv0MdWZlJSVWtzMUqpQOLbbFPrqeWjAx8xLXEa3YK6tWsdGvSRval1RZAiOq2gUqr1NG22WZ+7nqPVR9u1t009DXoR3LGpOtuUUqpVtdRsExkUydS+U9u9Fg16ILjXcFIcORr0SqlW8/7m4802Ne4aPj74MRcnXUywM7jda9GgBxwJQ0iQYnLzcu0uRSkVADILy9mWU8KV3pmk1mSvoay2rN1729TToIeGMW88+TttLkQpFQiON9t4x7bZv4IeIT2Y1HuSLfVo0APEDQYgonQvNXU6raBS6uws25LLuKRo+kaHUVFbwWdZnzEjeQZBjiBb6tGgB4hOwu0IYSDZHDiiPW+UUmeuvtmmvrfN51mfU1lX2S4zSZ2IBj2Aw0lNjxRStOeNUuosNW22Wb5/OQlhCYzvOd62mjTovYJ6DrUGN9OgV0qdhfc3H2+2KakpYU32Gi4bcBkOsS9uNei9XD2HkSiFHDpcYHcpSqlOKrOwnO25x5ttPjn4CbWeWmYl29Pbpp4Gfb1464Rs9WEd80YpdWZa6m3TN6IvI+NG2lmWBn0DbxfLkKMZeDw6raBS6vT5NtsUVRWxNnctswbMQkRsrcuvoBeRmSKyS0QyROSBFtYPFZEvRaRaRO5rsi5TRLaIyNcikt5ahbe6mIF4xEV/z0FyS6rsrkYp1cnsb9Js81HmR7iN27aLpHy5TrWBiDiBJ4BLgSxgg4i8Y4zZ7rNZEXAPMOcEL3ORMabwLGttW84gqqOSSSmyhkLoGx1md0VKqU5kWZMJwJdnLmdQ90GkRqfaWRbg3xH9JCDDGLPPGFMDLAZm+25gjMk3xmwAOvU4v46EIdrFUil1Rt7fnMv4pGj6RIeRV57HpsObmDlgpu3NNuBf0PcFfKdfyvIu85cBPhSRjSJy54k2EpE7RSRdRNILCuzp+RLcaxj9HYfJPFxky/srpTqn+mab+qP5DzM/xGBsvUjKlz9B39LX0emcrZxijBkPzAJ+KCIXtLSRMeYpY0yaMSYtPj7+NF6+9Uj8UFx4qMjdbcv7K6U6p6bNNisyVzAsZhjJ3ZNtrOo4f4I+C+jn8zwRyPH3DYwxOd77fGApVlNQx+SdbcpVpEGvlPLfez7NNodKD7GlcEuHOAlbz5+g3wCkisgAEQkGrgfe8efFRSRcRCLrHwMzgK1nWmybi0vFIPSqPsDR8hq7q1FKdQL7CsrY4dNss2L/CoAO02wDfvS6McbUicjdwAeAE3jOGLNNRBZ41y8SkV5AOhAFeETkJ8BwIA5Y6j0Z4QJeM8asaJNP0hqCwqgKTySlJJu9BWWkhcfYXZFSqoNrqbfNuIRx9I7obWdZjZwy6AGMMcuAZU2WLfJ5nIfVpNNUCTDmbApsd/FDSCndxTf5ZaQla9ArpU7u/S15Dc02GUcz2HN0Dw9OetDushrRK2ObCOk9nIGSx778YrtLUUp1cPXNNleM7gNYJ2Ed4mBG8gybK2tMg74JR8IQQqSWkpwMu0tRSnVwx5ttemGMYUXmCib2mkhcWJzNlTWmQd+Ud8wbCrXnjVLq5OqbbXp3D2NH0Q4OlBywfaTKlmjQNxVnXa7co2I/VbVum4tRSnVUzZpt9q/A5XAxvf90mytrToO+qdDuVIb2JEWy2Veg0woqpVrm22zjMR6WZy5nSp8pdA/pbnNlzWnQt8AdO9ga86ZAx7xRSrXsvc25TOjfg97dw/im4BvyyvOYOaDj9J33pUHfgtA+w0iRbPYeLrW7FKVUB7S3oIydeaXH+87vX06IM4SL+l1kc2Ut06BvgSthKOFSzZHcfXaXopTqgJZtPt5sU+ep44PMD7gg8QLCg8JtrqxlGvQt8fa8Mfk6raBSqrn3txxvtkk/nE5RVVGHGtumKQ36lngHNwsvycCt0woqpXzUN9tc4TO2TTdXN87ve77NlZ2YBn1LwuOoDoom2WSRdbTC7mqUUh1IfbPNrFG9qHXX8tGBj7g46WJCXaE2V3ZiGvQnUBMzmBRHjs42pZRqxLfZ5oucLyipKenQzTagQX9Cwb2GkirZZGjPG6WUV9Nmm+WZy+ke0p1ze59rc2Unp0F/AiG9h9NDysjLy7K7FKVUB+HbbFNZV8mnBz9letJ0gpxBNld2chr0JxI3GIC6vB02F6KU6gg8HsN7m3NJ8zbbrM5aTUVdRYdvtgEN+hPzdrEMKc7AGO15o1RX99eVu9l1uJTr0qyZVVdkriA2NJa0nmk2V3ZqGvQnEtWHGmc4iXUHKSzTaQWV6sreTD/E459kMDetH9emJVJWU8aqrFVclnwZTofT7vJOSYP+RESoik6xxrzRnjdKdVlfZBTy4L+2MCUllt9+eyQiwqeHPqXaXd0pmm1Ag/6kXAlDSHXo4GZKdVUZ+aUseGUjA+LCefLGCQQ5rchcvn85fcL7MCa+c8yUqkF/EmF9R9BTisnOybW7FKVUOyssq+bWFzYQ7HLw3C0T6R5m9awprirmy5wvuWzAZYiIzVX6R4P+JMQ7FEK19rxRqkupqnXzvZfSyS+p5pmbJ9IvplvDupUHV1Jn6jrkTFInokF/Mt6gdxXtadO3qaxx897mHEqqatv0fZRSp+bxGP57yTd8faiYR+eOZWy/6EbrV+xfQXJUMkNjhtpT4BnwK+hFZKaI7BKRDBF5oIX1Q0XkSxGpFpH7TmffDi26P3USTHxVJmXVda3+8jV1Hl5ee4AL//wpd7/2Fd99dj2lGvZK2erPH+7i/S25PDhrKLO8V8DWK6goYH3eemYOmNlpmm3Aj6AXESfwBDALGA7ME5HhTTYrAu4B/nIG+3ZcDicVUQO90wq23glZt8fwr01ZXPLIZzz0762kxjh5esJBMrNzueX5DW3ypaKUOrU3NhzkH5/t5YbJSXzv/IHN1n944EMMhpnJHXMmqRPx54h+EpBhjNlnjKkBFgOzfTcwxuQbYzYATQ9HT7lvRyfxQ6wxb1qhi6UxhhVb85j56CruXfIN3UOcLLsol1eq7ubSbQ/waa/HyDiUw20vbKCiRsNeqfa0Zk8hP1+6lQsGx7PwWyNaPGJfvn85g3sMZlD0IBsqPHP+BH1f4JDP8yzvMn/4va+I3Cki6SKSXlBQ4OfLt71ufUfQVwo5mHfmNRljWLW7gNlP/IcFr2zEbQyvX2Z4N+xXDP/yv5GwGJj+K3oUb+OTXk+wPTOHO15Mp7LG3YqfRCl1IrsPl3LXKxsZFB/BEzeMw+VsHo0vb3+Zbwq+YfagTnWsCvgX9C01RPk7JoDf+xpjnjLGpBlj0uLj4/18+bbnTBiCQwxlOWfW82bjgSKuf2ot331uPUfKanhyVjQr+z7DuZ/fiJTmwZxFcOfnMPW/4OpniT36DZ/2+Qdf7cvhzpfTqarVsFeqLRWUVnPr8xsIDXby3K0TiQxtPkDZ8v3L+dOGP3Fp/0u5cdiNNlR5dlx+bJMF9PN5ngjk+Pn6Z7Nvx+Ad88ZRuPu0dtuWc4z//XA3n+zMJy4ihN/PSuTaitdxff4MOIPhop/DuXdD8PFuW4yYA+5a4v/1PT7t+xQX7vk+d72ykUXzJxDi6viXWSvV2VTWuLnjpXSOlFez5Pvn0jc6rNk263LX8bM1P2NCzwn8/vzfd4ohD5ryJ+g3AKkiMgDIBq4HbvDz9c9m344hZiAenESX76fW7Wm4Mu5E9hWU8chHu3lvcy5RoS4emDGQ24I/JnjN7VBdAuNuskI+slfLLzD6WvDU0uvfP+CTfi4u2nUHP3zVwZM3jifYpb1hlWotHo/hv974ms1ZxSy6aQKjE6ObbbOzaCc//vTHJEcl89jFjxHiDGn/QlvBKYPeGFMnIncDHwBO4DljzDYRWeBdv0hEegHpQBTgEZGfAMONMSUt7dtGn6VtuIIpi+jPwGPZHDhSQUpCRIubZRdX8tjKPby1KYsQl4MfXTSIBT13EL7qBijaBwMvghm/hV4jT/2eY28Adw193/0xK5NcXLzjVn68WHhs3rhTftEopfzzxxU7WbEtj19cMYzLRjQ/8Mouy+aulXcRGRzJP6b/g6jgKBuqbB3+HNFjjFkGLGuybJHP4zysZhm/9u1sPLGppJZsZk9+WbOgLyit5snPMnh17UEAbj43mXuGlhC9+r/gyy+spp8b34KU6XA6/W4n3ALuWpKW3cdHSU6mb72Ze5c4+Ot1Y1o8UaSU8t+r6w7wz1X7mH9Of26fOqDZ+qNVR1nw0QJq3DU8M+MZeoWf4Bd4J+FX0Hd13fqOICLzIz48fBRGWn/hxypqeWr1Xp5bk0mN28O1ExL5ycQwem34E7y6BMLj4cq/wrjvgvMM/5gnfQ/cNQz44Ges6B/EZd/ciMsh/OXaMTgdnediDaU6ks93F/Dw29uYNiSeX141vFk3yoraCu7++G5yy3N5esbTna4rZUs06P0Q3GsYiIeS7J1U1KTy/H8y+efneympquOqMX2494JeDNjxFLz0pLXD+f8NU34Coa3wU+/cH0JdNakf/5r3+7u4/Ku5uBzCH68ejUPDXqnTsjOvhB++uonUhAj+fsP4Zr+O6zx1/HTVT9l6ZCuPTHuEcQnjbKq0dWnQ+8M75k1R5lYu+FMkhWXVXDI0gXunD2RE7r/htd9DeQGMug4ueRii+5389U7X+feCu5Zhn/0P7/R38q2N1+ByOvjdnJEa9kr5Kb+kitue30B4iJPnb51IREjj+DPGsPDLhazKWsVD5zzEJUmX2FRp69Og90dsKgYhoTqTQUnT+edN45hQsxHevhwKdkLSeXDDG9B3QtvVcOH/A3c1o1b/L//X38V31n8bl0NYOLvlK/iUUsdV1NRx+4vpFFfWsuT759K7e/NulE98/QRLM5ayYMwCrhtynQ1Vth0Nen8Ed8N078ftcTXce2kE8uFtsO8ziBkIc1+BoVee3onWMyECFz8E7hrGf/E4b/R3MXftVbicwsNXNm9nVEpZ3B7Djxd/zbacYzz93TRG9u3ebJs3dr7BPzf/k6tTr+YHY35gQ5VtS4PeT46EoURnroRF70Jod5j5B0i7HVzB7VeECFz6G3DXMnndIl5JDuKm/8wk2OnggVlDNeyVasH/LNvBR9sP86urhnPJsJ7N1n984GN+t+53XJh4Ib845xcB+f9Ig95fiZNg76fWydEL7oOwHvbUIWJ9ydRVM3Xj8zzXP4jbVgkup3DfjCEB+Y9UqTP10peZPLtmP7ecl8wtU5p3o9x0eBP/b9X/Y1T8KP584Z9xOQIzEgPzU7WFqf8Fk7/fOj1pzpYIXPEIuGu5+Ovn+Geyk+9/CkFOBz+ZPtju6pTqED7dmc+v3tnG9GEJPHRl89HRM45mcPcnd9Mnog9PXPwEYa7m7faBQoPeX04XODtAyNdzOOBbj4Gnlss2P83j/YP40Uor7H94UYrd1Sllq205x7j7tU0M6x3F364f1+y6k7zyPBasXECoM5RFly4iOjTankLbiQZ9Z+ZwwuwnwV3DVduepLa/i3s/gCCncOcFnf8iD6XORN6xKm5/IZ2osCCeu2Ui4U26UR6rPsZdK++ivLacF2a+QN8If0dd77w06Ds7pwu+8zS4a/nOzseoTnLy4DJwORzc1sKl3UoFsvLqOm5/cQOlVbW8ueA8ekaFNlpfVVfFPZ/cw4GSAyyavoghMUNsqrR9adAHAmcQXPM8vHET8/b8ldqkn/Lwe9aR/fxzk+2uTql2YYzh/v/bzI7cEp69eSLD+zRuanV73Dyw+gE25W/izxf8mUm9J9lUafvToA8UrmC47iVYPI/5e/9CTb/7eehtcDoc3DA5qVXfqrLGTVFFDUVlNRRV1FBb52FKShxhwZ1vnG4VOF5Zd5D3Nufy08uGcNHQhEbrjDH8fv3v+fjgx9w/8X5mDuhcc76eLQ36QBIUCte/hrx6Lbcf+BM1iT/jZ0vB5RSuS2t5WIZat4ejFTUUlVu3o+W1FFXUcLTcZ1lF/Tor2KtqPc1eJ7pbENdPTGL+uf1bnLxBqba0NfsYv3l3OxcOjueuC5ufn3p6y9O8sesNbh15KzcNv8mGCu0lxvg7K2D7SUtLM+np6XaX0XnVlMMr12AOreNvMT/nbzlDuX5iP2rdpll4l1adeBLyyFAXMeHB9OgW7HMfREx4CDHhQQ3LK2rcvLbuIB9uzwNgxvBe3DIlmckDYrRfv2pzJVW1XPnYGmrqPCz78fnEhDe+iHHpnqU8/MXDXDnwSn439Xc4JDCH+RaRjcaYtJbW6RF9IAoOhxuXIC9/mx/n/B5H0i958isHMd2C6RFuhXNSTDdivI97hAdb68KEeFcVMY4KoqSCoJpjUFUEVcegshiqiq3H+T6PK4vBuLlgxHfIvfAmXtzmZvGGg6zYlsfQXpHcOiWZ2WP7EhqkzTqq9RljuP+tzWQXV/LGnec0C/lVWav49Ze/5rw+57HwvIUBG/Knokf0gayyGF6aDfnb4YKfgjFWQFcWWyHd9HFN2clfzxkModEQFm0NA1H/uLoU9nwICAz/FtXjv8fSwkRe+PIAO/NKie4WxLxJSdx0jjbrqNb1wn/286t3t/PArKEsaNJks7lgM7d/cDsDowfy3GXPER4UblOV7eNkR/Qa9IGuoghengO531jPgyOtkA6LtoK6xcfdWw50V+iJB287egA2PA2bXrK+OHqPxUxewLpu03hhXQ4fbs9DRJgxvCe3nJfMJG3WUWfpm0PFXLPoC85PjeeZ76Y1GrI781gm85fPJyIogpcvf5m4sDgbK20fGvRdnccDlUet0D7T2a78VVMO37wO6/4Jhbshoiek3U5OyvW8uKWCxesPcayylmG9o7j1vGS+NbaPNuuo03asopYrHl+Nx2N4/57z6eHTZFNQUcD85fOprKvk5VkvkxTVur3OOioNetX+PB7Y9wmsXQQZH1nNPiOvoWrCnfw7L5YXvshkZ14pPboFcf2kJOaf058+2qyj/GCM4fsvb+STnfksWXAu45OODzBYVlPGLStu4WDpQZ6/7HlGxI2wsdL2pUGv7FW4xzrC//o1qC2HpPMwk7/P2uBzeWHtIT7afhgR4bIRPbn5XG3WUSf37Jr9/Oa97fziimHccf7AhuVuj5sfffIjvsz5kscveZypfafaWGX706BXHUNlMXz1Cqz/JxQfhO79YNL3yB54LS99fUybddQpfXXwKNcu+pKLhibw1PwJjQ4I/rrxrzy39Tl+MfkXzB0618Yq7XHWQS8iM4G/AU7gGWPMH5qsF+/6y4EK4BZjzCbvukygFHADdScqxJcGfYDzuGHXcli3CDJXQ1A3GHM9VeO/x9KsCF70adaZNymJGyYnkdijm91VK5sVV9RwxWNrEIH3f3Q+3bsFNax7b997PLj6Qa4bfB0PnfuQjVXa56yCXkScwG7gUiAL2ADMM8Zs99nmcuBHWEE/GfibMWayd10mkGaMKfS3YA36LiRvixX4m98EdzUMuhgzeQFrHeN54csDfLT9MAaYmhLHtWn9mDG8px7ld0HGGL73Ujqf7y7grQXnMaZfdMO6bYXbuHnFzYyMG8nTlz5NkDPoxC8UwM72gqlJQIYxZp/3xRYDs4HtPtvMBl4y1rfGWhGJFpHexpjcs6xdBbpeo2D2EzD917DxedjwLPLadZwbm8K5k75PzmXfZsnmo7yZnsU9r39FVKiLOeP6cl1avxbn/lSB6enV+1i5I59fXjW8UcgXVBRwz6f3EBsayyPTHumyIX8q/lwm1hc45PM8y7vM320M8KGIbBSRO0/0JiJyp4iki0h6QUGBH2WpgBIeZ13U9ePNcPWzVt/95T+lz7Pj+UnN06z+biyv3jGZi4YmsHjDIa58fA2X/201L/xnP0fLa+yuXrWhjQeK+OOKXcwa2YtbzktuWF7truYnn/2E0ppSHrv4MWJCY+wrsoPz54i+pe4PTdt7TrbNFGNMjogkAB+JyE5jzKpmGxvzFPAUWE03ftSlApErGEZdY92y0q1mnY0v4Fj/FFN6j2HKuPksnDGHd3aVsyQ9i1+9u53/WbaTS0f0ZG5aP6akxDWbTUh1XkXlNdz92lf0jQ7jj9eMbjj5aozhN1/+hs0Fm3lk2iNdZlz5M+VP0GcBvkMfJgI5/m5jjKm/zxeRpVhNQc2CXqlmEtMg8RmY9SfY8iZsehmW3Ud31y+YP+wq5l8xn+0hU1iyMZt/f53N+5tz6dM9lGsmJHLNhH4kxeoJ3M7M4zHcu+RrjpTV8K8fnEdU6PFmmZe3v8zbe9/mrjF3cWn/S22ssnPw52SsC+tk7CVANtbJ2BuMMdt8trkCuJvjJ2MfM8ZMEpFwwGGMKfU+/ghYaIxZcbL31JOx6oRyvra6aG5ZYg21EN0fxt1E9ci5rMwOZkn6IVbtKcAYOG9QLNel9WPmyF56ArcTevKzDP60Yhe/mT2i0QQ6X2R/wV0f38VF/S7ikWmPdNmByppqje6VlwOPYnWvfM4Y8zsRWQBgjFnk7V75d2AmVvfKW40x6SIyEFjqfRkX8Jox5nenej8NenVKtZWw4z346mXY/zkgMOhiGD+fnJ4X8X/fFPDmxiwOFlUQGeriW2P6MHdiP0b17a4XY3UC6/cXMe/ptcwa2YvH541r+Ds7UHKAee/Po1d4L16Z9QrdgvRXWz29YEoFtqOZ8NWr8PWrUJINYTEwei6esTexrqI3b6YfYtnWXKpqPQztFcm1af2YM7YPsREhdleuWlBYVs0Vj62mW7CLd+6eQqS3yaa0ppQbl93I0aqjvH7F6yRGJtpcaceiQa+6Bo8b9n1qteXvfB88tdBnPIy7iZLUOby7q4wl6Vl8c6iYIKcwfVhPrh6fyHkpsXQL1qkZOgKPx3Dz8+tZt7+IpT84jxF9rC60bo+bez69hy+yv+CpGU8xsddEmyvteHTiEdU1OJyQMt26lR+x2vE3vQzv30uU62fcOHw2N14+n12h57NkYxZLv8pm+dY8gpzChP49OD81nqkpcYzs21177tjkiU8zWL2nkP/59qiGkAd47KvHWJW1il9M/oWG/BnQI3oV2IyBnE3eE7hvQXUJ9BgA426kZuQ81h0JYc2eQlbvKWR7bglgzX87ZVAcU1PjmJoSR78YbQduD1/sLeSmZ9Zx1Zg+PDp3bEO7/Pv73ueB1Q9w7eBrefjch22usuPSphulAGoqYMc7VuhnrgZxwMBpMHgmpEynMCSR/2RYob9mTyF5JVUADIgLZ2qKFfznDopt1M1PtY6C0mouf2w1kaEu3r17KuEhVmND/fAGI2JH8MyMZ/TK15PQoFeqqSN7rZO325ZC0T5rWY8BVrNP6qWY/lPIKDZW6GcUsnbfESpq3Dgdwth+0UxNieOCwXGMSYzG5dTufWfD7THMf3Ydmw4e5d8/nMLQXlGANbzB9e9fj0tcvH7l63rl6ylo0Ct1MkX7IONjyFgJ+1dBbQU4Q6D/eQ3BXxOdwqZDxVYzT0YhW7KK8RiIDHFxzqBYLkiNY2pqPMmx3bT75mn660e7+dvHe/jT1aO5bqJ13WWNu4ZbP7iVPUf38PKsl/XKVz9o0Cvlr9oqOPilFfoZK6Fgp7W8ez9IuQRSLoUBF1DsCeWLvUdYvaeQ1XsKyDpaCUDf6DDOT43j/NR4pqTEEt0t+CRvptbsKWT+c+v49ri+/O+1YxARjDE89J+HeHvv2zwy7RG98tVPGvRKnaniQ8dDf9/nUFMKDhcknesN/umYhBEcKKpkdUYha/YU8MXeI5RW1SECo/t2Z6o3+Mcn9SDY1bGbeYwxHDhSwfbcEuIiQkiK6UZCZEijibdbS35JFZc/tpoe3YJ5++4pDV1cX97+Mn/a8CcWjFnAD8f+sNXfN1Bp0CvVGupqIGu9Ffp7VsLhLdbyiF7eJp7pMHAadcHd+SbrGGv2FLImo4BNB4txewzdgp2cMzDWe8Qfx6D4CNubeYwx7C0oY+2+ItbvL2Ld/iMcLqlutE2wy0G/HmEkxXSzbrHhDY/7xYSd0TUIdW4PNz6zjs1Zx3jn7imk9owEdHiDs6FBr1RbKMmFvd62/b2fWGPviAMSJ3n7818CvcdSUuNm7d4jrPH26NlfWA5A7+6hnO9t25+aEkdMeNs383g8ht35pazbZ4X6+v1FFJZZwzwnRIYweWAskwbEMCaxO0crajlYVMGhogoOHqngYJF1K6uua/Sa1pF/mE/4138hdKNnZGiLvwb+8sEu/v5pBv977RiunmBd4arDG5wdDXql2pq7DrI3ept5PoKcr6zlUYkwYg6M+Db0nQAiHCqq8IZ+AWv2FFLibeYZ2ae+mSeOCf17EOI6+4HY3B7DjtwS1u47wrr9RWzILKK4ohaAPt1DmTwwlskDYpg8MNavE8nGGIq9XwD1t0M+j3OKK/H4REqwy0FijzD6+3wJiAi/fX87105I5E/XjAGs4Q1uWnYTRVVFOrzBGdKgV6q9lRVYob/9beuo310D3ZNgxGwr9PuMBxHcHsOW7GOs3l3A6j2FbDp4lDqPISzIyeSBMZyfGs/5qXGkJvjXzFPr9rA1+xjr9ltNMRsyiyitso7Ak2K6NYT65AExbXIhWK3bQ05xZaMvgoZfA0cqKPX+GhjSM5J//3AKYcFOHd4ArAv7asqgshii+51y85Zo0Ctlp8piazL0bUutJh5PLUQnWYE/4tvQeyx4Q7ysuo61e4+weo8V/Pu8zTw9o0KYmhLPBYPjmJISR5x3QLbqOjebs46xfn8Ra/cdYeOBo1TUuAEYGB/O5AGxnDMwhkkDYujdPcyOT9/AGMOxylqyjlaSkhDRMHT0oxsf5dmtz/LzyT/n+qHX21pjq6upgLLDUF5g3ZflW7dy773vsrpK63zPfbvO6K006JXqKCqPws5lVujv+xQ8ddAj+Xjo9xrdEPoAWUcrGvru/yejsKHZZXjvKLqHBbHp4FGq6zyAdZQ82RvqkwbEkBAZekYlGmMoqCygR2gPghxteyWq7/AGD53zkO0np/1SV90ksH3CulGoF1i9tFrSLRbCEyAiASJ6eu8TILI3jL7ujMrSoFeqI6ooskbZ3LYU9n0Gxg0xA4+Hfs+RjULf7TFszT7GmoxCVu0uoLymjonJMUweYJ1APdOTucYYMksy2ZC3gfV569mQt4GiqiKc4qRvRF+SopJIjkomKSqJ/lH96R/Vn17deuF0nN05hHYb3sBdB7XlUF1mNY/UlHkfl7f8vLrUZ1154+fVZScO79DoxqHdEOTeMA+P997HQRt8Vg16pTq68iOw8z0r9PevskI/NuV46CcMbxT6Z8MYQ1ZpFuvz1rM+bz3peenkV+YDkBCWwMTeExkVN4qiqiIOlBzgYMlBMksyqayrbHiNIEcQSZFJjcK//hYfFn/KI/PCykLmvjcXpzh5/YrXiQ2L9f8DeNzW0XNp7vFbic/j+iPp+vD2qfuUgsIhJAKCwyE4wro1fR5efzTeEyLij4e4y975DTTolepMygthx7tW6GeuBuOBuME+oT/stF8ypyyn4Wh9fd568srzAIgNjWVSr0mk9UpjUq9J9I/q32JIG2MorCwksySTgyUHOVBywPoSKD3IwZKD1HhqGrYNc4WRFNn8CyApKokeIT2o9dRy2we3sfvobl6a9RJDY4bWv4k1umhJLpTmQGkelHjvS3OPPy47bH0R+hKnFbiRvaz7kEifgI607kMiWghvn3VB4eDovP32NeiV6qzKCqwRN7cthQP/sUI/fqgV+EnnWFfpisN7c3rvhcPVxaw/upMNR3ewvmg72d4j9h7BUaTFjmRS3GgmxY9hQGQy4nD6vIa1P+Kw3stTZx1BG7fPY5/lnjrcnloOVxSSWZ7DwYo8DlTkcaAynwOV+WRXFeHG0/BxIp2hRDlDya4p5n+jJzKjzuET6LnWOENNhUZbbddRva37Zo/7WEfUZ9mU1Nl1maD/+1d/JyIogj4RfegT0Yfe4b2JCY3pHCd4lDqV0sPe0P+3Ffoc/79b6HSwITSU9aEhbAgL5UCQ1QYc5XaTVlXNpKoqJlZWk1JbS3ses9YCOS4XB4JcHAgK4kCQi0MuFxdXVDK3otY6Ao/qY91H9vF53tu7rDcE64VT/ugSM0x5jIfFuxZzrPpYo+WhzlB6hfdqCP76L4E+4dZ9fFj8WZ9UUqo9mIgEqsfPp2TkbEqP7mdv9lrWF+9iQ/Fu9lUeBiDCGcqEyAFcGzWQSZEDGByWgBOso3BjvPfeW/3ReaObzzb1R/oOp/eXg9PnscO6dzi9y11Ws0fDY2t5kMNJf4eT/r77iwPCelg3PQhrFwET9A5xsOb6NZTUlJBblktOWQ455TnklOWQW24931m0k6Kqokb7ucRFz/Cezb4Eekf0pk94H3qF9yLYqSMQqtZR56mjrKaMkpoSSmtKOVZzjNKaUkprShuWlVR772tLKK32WV5TQq2nttHrdXN1Y3zP8czudQOTek1iaMxQXI6A+W+tWknA/YuICo4iKibqhONXV9ZVkluea30ZeL8I6r8M1uWuo6CyAI853qYoCHFhcfSO6E1CWAKhrlBCnCGN7kOd1uMwVxghzhBCXCGEOcMIcYU0rAt1hTbap637J5+Kx3hwe9y4jc/Nc/zeYzzUmboTbucxHuo8dQ3bYcBg8BgPxtukYMzx58baAA8ejDENy4wxzZ83ua/n+7q+z1ta56vp9r77eYyn4eY27obP2+i5afx5658328bT+LnbuCmrKaO0trQhwCvqWmiD9uESF5HBkUSFRBEZFElkcCS9I3pby4KjGu6jgqPoHdGb4bHDbf+3pDo+v4JeRGYCfwOcwDPGmD80WS/e9ZcDFcAtxphN/uzb3sJcYQzsPpCB3Qe2uL7WU8vh8sMNvwLqfxnkluWSWZJJVV0VVe4qquuqqXJXNTvC8pdTnI2+KIKdwQ1jcQPHA7Jp8PkTjt7ArQ/f+ue+Aa784xQnDnHgFCdOx/HHLd63sD48KJx+Ef2IDI5sCPD6oG5Y5hPgYa4wPaekWt0pg15EnMATwKVAFrBBRN4xxmz32WwWkOq9TQb+AUz2c98OJcgRRGJkot+DKrk9bqrd1Q3hX+mupLqummp3NZV1lQ3rquqOfzlUu6ubfWFUu6sxxiAiOHCAWL8m6p/X/+cXBIdYzwVrmUMcDds2fe67jdPhxCWuhsdO8bl5Q8olLhwOh9/b1S+vf6+Ge2+dDZ+j/jk0bFNfZ9NtGi2r//PwahqC9Z/Pd139spYCs+m6+vc8UVgrFQj8OaKfBGQYY/YBiMhiYDbgG9azgZeMdTi6VkSiRaQ3kOzHvp2a0+Gkm6ObDqmqlOqw/Dlk6Qsc8nme5V3mzzb+7AuAiNwpIukikl5QUOBHWUoppfzhT9C31GDY9IzXibbxZ19roTFPGWPSjDFp8fHxfpSllFLKH/403WQBvgMkJwI5fm4T7Me+Siml2pA/R/QbgFQRGSAiwcD1wDtNtnkH+K5YzgGOGWNy/dxXKaVUGzrlEb0xpk5E7gY+wOoi+ZwxZpuILPCuXwQsw+pamYHVvfLWk+3bJp9EKaVUiwJqrBullOqqTjbWjXYUVkqpAKdBr5RSAa5DNt2ISAFw4Ax3jwMKW7Gc9tRZa++sdYPWbhetvfX1N8a02De9Qwb92RCR9BO1U3V0nbX2zlo3aO120drblzbdKKVUgNOgV0qpABeIQf+U3QWchc5ae2etG7R2u2jt7Sjg2uiVUko1FohH9EoppXxo0CulVIALmKAXkZkisktEMkTkAbvr8ZeI9BORT0Vkh4hsE5Ef213T6RIRp4h8JSLv2V3L6fBOkPOWiOz0/vmfa3dN/hKR//L+e9kqIq+LSKjdNZ2IiDwnIvkistVnWYyIfCQie7z3PeyssSUnqPvP3n8vm0VkqYhE21ii3wIi6H2mLJwFDAfmichwe6vyWx3w38aYYcA5wA87Ue31fgzssLuIM/A3YIUxZigwhk7yGUSkL3APkGaMGYk1YOD19lZ1Ui8AM5ssewD42BiTCnzsfd7RvEDzuj8CRhpjRgO7gQfbu6gzERBBj890h8aYGqB+ysIOzxiTWz+RujGmFCtsWpyFqyMSkUTgCuAZu2s5HSISBVwAPAtgjKkxxhTbWtTpcQFhIuICutGB53kwxqwCiposng286H38IjCnPWvyR0t1G2M+NMbUeZ+uxZpjo8MLlKD3e8rCjkxEkoFxwDqbSzkdjwL/D/DYXMfpGggUAM97m52eEZFwu4vyhzEmG/gLcBDIxZr/4UN7qzptPb1zVuC9T7C5njNxG7Dc7iL8EShB7/eUhR2ViEQA/wf8xBhTYnc9/hCRK4F8Y8xGu2s5Ay5gPPAPY8w4oJyO2XzQjLc9ezYwAOgDhIvITfZW1bWIyM+xml1ftbsWfwRK0Psz3WGHJSJBWCH/qjHmX3bXcxqmAN8SkUys5rKLReQVe0vyWxaQZYyp//X0FlbwdwbTgf3GmAJjTC3wL+A8m2s6XYdFpDeA9z7f5nr8JiI3A1cCN5pOciFSoAR9p52yUEQEq514hzHmEbvrOR3GmAeNMYnGmGSsP/NPjDGd4sjSGJMHHBKRId5FlwDbbSzpdBwEzhGRbt5/P5fQSU4k+3gHuNn7+GbgbRtr8ZuIzATuB75ljKmwux5/BUTQe0+O1E9ZuANY0ommLJwCzMc6Gv7ae7vc7qK6iB8Br4rIZmAs8D/2luMf76+Qt4BNwBas/8cd9rJ8EXkd+BIYIiJZInI78AfgUhHZA1zqfd6hnKDuvwORwEfe/6uLbC3STzoEglJKBbiAOKJXSil1Yhr0SikV4DTolVIqwGnQK6VUgNOgV0qpAKdBr5RSAU6DXimlAtz/B3OvpEtWwzcRAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_62_5.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "from sklearn.linear_model import LinearRegression, Ridge, Lasso\n", "from sklearn.preprocessing import PolynomialFeatures\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.pipeline import make_pipeline\n", "from sklearn.utils import resample\n", "\n", "np.random.seed(2018)\n", "\n", "n = 40\n", "n_boostraps = 100\n", "maxdegree = 14\n", "\n", "\n", "# Make data set.\n", "x = np.linspace(-3, 3, n).reshape(-1, 1)\n", "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)\n", "error = np.zeros(maxdegree)\n", "bias = np.zeros(maxdegree)\n", "variance = np.zeros(maxdegree)\n", "polydegree = np.zeros(maxdegree)\n", "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2)\n", "\n", "for degree in range(maxdegree):\n", " model = make_pipeline(PolynomialFeatures(degree=degree), LinearRegression(fit_intercept=False))\n", " y_pred = np.empty((y_test.shape[0], n_boostraps))\n", " for i in range(n_boostraps):\n", " x_, y_ = resample(x_train, y_train)\n", " y_pred[:, i] = model.fit(x_, y_).predict(x_test).ravel()\n", "\n", " polydegree[degree] = degree\n", " error[degree] = np.mean( np.mean((y_test - y_pred)**2, axis=1, keepdims=True) )\n", " bias[degree] = np.mean( (y_test - np.mean(y_pred, axis=1, keepdims=True))**2 )\n", " variance[degree] = np.mean( np.var(y_pred, axis=1, keepdims=True) )\n", " print('Polynomial degree:', degree)\n", " print('Error:', error[degree])\n", " print('Bias^2:', bias[degree])\n", " print('Var:', variance[degree])\n", " print('{} >= {} + {} = {}'.format(error[degree], bias[degree], variance[degree], bias[degree]+variance[degree]))\n", "\n", "plt.plot(polydegree, error, label='Error')\n", "plt.plot(polydegree, bias, label='bias')\n", "plt.plot(polydegree, variance, label='Variance')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The bias-variance tradeoff summarizes the fundamental tension in\n", "machine learning, particularly supervised learning, between the\n", "complexity of a model and the amount of training data needed to train\n", "it. Since data is often limited, in practice it is often useful to\n", "use a less-complex model with higher bias, that is a model whose asymptotic\n", "performance is worse than another model because it is easier to\n", "train and less sensitive to sampling noise arising from having a\n", "finite-sized training dataset (smaller variance). \n", "\n", "\n", "\n", "The above equations tell us that in\n", "order to minimize the expected test error, we need to select a\n", "statistical learning method that simultaneously achieves low variance\n", "and low bias. Note that variance is inherently a nonnegative quantity,\n", "and squared bias is also nonnegative. Hence, we see that the expected\n", "test MSE can never lie below $Var(\\epsilon)$, the irreducible error.\n", "\n", "\n", "What do we mean by the variance and bias of a statistical learning\n", "method? The variance refers to the amount by which our model would change if we\n", "estimated it using a different training data set. Since the training\n", "data are used to fit the statistical learning method, different\n", "training data sets will result in a different estimate. But ideally the\n", "estimate for our model should not vary too much between training\n", "sets. However, if a method has high variance then small changes in\n", "the training data can result in large changes in the model. In general, more\n", "flexible statistical methods have higher variance.\n", "\n", "\n", "You may also find this recent [article](https://www.pnas.org/content/116/32/15849) of interest." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "============================\n", "Underfitting vs. Overfitting\n", "============================\n", "\n", "This example demonstrates the problems of underfitting and overfitting and\n", "how we can use linear regression with polynomial features to approximate\n", "nonlinear functions. The plot shows the function that we want to approximate,\n", "which is a part of the cosine function. In addition, the samples from the\n", "real function and the approximations of different models are displayed. The\n", "models have polynomial features of different degrees. We can see that a\n", "linear function (polynomial with degree 1) is not sufficient to fit the\n", "training samples. This is called **underfitting**. A polynomial of degree 4\n", "approximates the true function almost perfectly. However, for higher degrees\n", "the model will **overfit** the training data, i.e. it learns the noise of the\n", "training data.\n", "We evaluate quantitatively **overfitting** / **underfitting** by using\n", "cross-validation. We calculate the mean squared error (MSE) on the validation\n", "set, the higher, the less likely the model generalizes correctly from the\n", "training data.\n", "\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAykAAAFLCAYAAADSycMSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAACUPElEQVR4nOzddXiV5RvA8e9zzroH26jBBozu7g4BFURyNAqIitiKP1uxWxFJCUFCygTpDuluGB2DsbGx3t7fH+/ZGGPNthO7P9e1C3beek7de+8nlaZpCCGEEEIIIYSlMJi7AEIIIYQQQgiRliQpQgghhBBCCIsiSYoQQgghhBDCokiSIoQQQgghhLAokqQIIYQQQgghLIokKUIIIYQQQgiLIkmKEEIIIYQQwqJIkmKFlFIhSqkYpVSkUipcKbVVKTVaKWXx76dSykEptcj0HDSlVFtzl0mIosya40laSql3TTGlo7nLIkRRYc3xI7v7EaXUe0qpBKVUVJqfCuYpbdFk8R8ikalHNU1zBwKAT4HXgekFcSGllDGfT7kZGARczefzCiHyxprjCUqpikBv4Ep+n1sIkS1rjh/Z3Y8s0DTNLc3PmXy+vsiCJClWTtO0CE3T/gD6AUOVUjUBlFKOSqkvlVLnlVLXlFKTlFLOKccppV5TSl1RSl1WSo0w1SIEmbbNVEr9pJT6Ryl1B2inlCqtlFqslApVSp1VSo1Ncy6DUmqcUuq0UuqmUmqhUqpYJuWN1zTtW03TNgNJBfnaCCFyx9riSRoT0G+M4vP7NRFC5Iy1xQ+5H7F8kqTYCE3T/gMuAq1MD30GVAbqAkFAGeAdAKVUF+AloKNpW5sMTjkA+AhwB7YCfwL7TefpALyglHrItO9Y4DHTeUoDt4Af8/HpCSEKkTXFE6VUHyBe07R/8vJchRD5y5riRw48qpQKU0odVko9/QDnEXkgSYptuQwUU0opYCTwoqZpYZqmRQIfA/1N+/UFZmiadljTtGjg/QzO9bumaVs0TUsGagG+mqZ9YKp5OANMTXO+p4A3NU27qGlaHPAe0FspZVdQT1QIUeAsPp4opdxMZXkhn56zECJ/WHz8yIGFQDXA1/Qc3lFKBefhPCKP5CbStpQBwtC/UC7Abj0+AKCAlL6cpYFdaY67kMG50j4WAJRWSoWnecwIbEqzfalSKjnN9iSgBHAp189CCGEJrCGevA/8omna2Rw8HyFE4bGG+JElTdOOpPl1q1LqO/Sxb/Nycx6Rd5Kk2AilVCP0oLAZuAHEADU0TcvoS3kF8E/ze9kM9tHS/P8CcFbTtEqZXP4C8ISmaVtyXXAhhMWxonjSAfBXSj1j+t0XWKiU+kzTtM9ycLwQIp9ZUfzILQ09wRKFRLp7WTmllIdS6hFgPjBH07SDpibRqcA3Sik/035l0vTZXAgMV0pVU0q5YOobmoX/gNtKqdeVUs5KKaNSqqYpEAFMAj5SSgWYruWrlOqRRZkdlVJOpl8dlFJOKk0VixDCPKwwnnQAaqL3da+L3sXkKWRMnBCFzgrjR5b3I0qpHkopb6VrjD7e5ffcvi4i7yRJsV5/KqUi0WsN3gS+Boan2f46cArYrpS6DawGqgBomrYc+B5YZ9pnm+mYuIwupGlaEvAo+k3AWfSakWmAp2mX74A/gJWmMm0HmmRR9uPoNStlgH9N/w/I2dMWQhQAq4wnmqbd1DTtasoPereOW5qmReX2BRBC5JlVxg+TrO5H+pvKFAnMBj7TNG1Wlq+EyFdK07Ts9xI2TSlVDTgEOGqalmju8gghrJfEEyFEXkn8EGlJS0oRpZTqqfTVVr3Rpwf8UwKCECIvJJ4IIfJK4ofIjCQpRddTQChwGr2LhMz/LYTIK4knQoi8kvghMiTdvYQQQgghhBAWRVpShBBCCCGEEBZFkhQhhBBCCCGERZEkReSZaX7xI0qpkuYuizVSSi1RSnUxdzmEsAQSTx6MxBMh7lJKdVZKLTN3OWyJUmqsUurTwrym1SYpSqkQpVS8Uson3eP7lFKaUirQ9Lu/UmqxUuqGUipCKXVQKTXMtC3QtG9Uup9+BVTmSkqpWKXUnGz2e1EpddVU3p+VUo5ptgUqpf5RSt0y7TNBKWX3AGXK6lpjlFK7lFJxSqmZGRw+CthoWp8gL9c+oZSqnMtjZpjes6BMtvsppeYppS6bntMWpVSTNNvbKqWS073fQ/NSftP5ApVS65RS0UqpY0qpjmm2lVJK/WEqS+pnMo1PgY/yem2Rf2w1niil+iuljpvKel0pNUsp5WHa5qiUmq6UOqeUilRK7VVKdX3AMmUYT3J4rUKJJ0qp/6V7f2JMMcEnk/0z/Y6btj+nlDqrlLptipct81L+7K6llHpYKbVZKRVueo2nKqXc0xwu8aSQWVPcUNn/PU+7r1JKjVdKXTKVd71Sqkaa7XOUUldMn/kTSqkR+VlW0zWKKaWWKqXumOLGgHTb+yqljpriyRGl1GPpTvEx+nciL9durpTamsN9Z5o+A2nfO2MOjrvvXsb0nBeYPic3lFJzU+J1fjHF4p9N791VpdRL6ba3V0rtMW0/o5QalWbzFGCQMi3KWSg0TbPKHyAEfRGe59I8Vsv0mAYEmh5bB3wLuAJ2QD2gq2lboGlfu0Iq80pgE/pKrJnt8xBwDagBeAPrgU/TbP8HmAk4ASWBg8DYPJYnu2s9DjwG/ATMzOD4Q0CLTM7dFlifxbUrAqdyWd6WwEbTexaUyT4VgJeAUoAR/cbnBuCWplwX8/E93Ya+cJUz0AsIB3xN20oAzwDN0n4m0x1/EmhYGJ8/+cnyfbTVeFIW8DH93w2YC3xv+t0VeM9UbgPwCPqiZYF5LE+m8SQn1yrseJLm2PeAtVlsz+o73gS4AzQAFPqsRKGAMY9lyepaA4AugIvp9V0OTEp3vMSTQvyxprhBNn/P0+3bF7iM/vfUCHwC7EmzvQb6OiYAVYGrQIM8lOk94L1Mts0DFqDHrZZABFDDtK0MEA90NX3vHgaiAT/T9kbAySyuOxMYlsX28cBbOXwOM4HxuXzeGd7LABPR47oH+uKUq4Gv8/C6DsvsPTa9l5tMMaSa6b3rYtpmb3qdnzK9ro2AKKBOmuOnAq8U5Gc17Y/VtqSY/AIMSfP7UPRVQdNqhP5m3dE0LVHTtL2avsJpoVJK9Uf/g7Mmm12HAtM1TTusadot4EP0D1yK8sBCTdNiNb3GcQV6wEi5ziOmWpxwpdRWpVTtvF5L07QlmqYtA25m8HzKod8Y7Mjm+WTmYfSEK0eU3lr0AzAmq/00TTujadrXmqZd0TQtSdO0KYADptVtc3AdT1ON7xVTLdL4zGpFTLW29YF3NU2L0TRtMXrS2MtUlmuapk0EdmZxyfXor4UwP5uLJ5qmXdA07Uaah5KAINO2O5qmvadpWoimacmapv2FvoJzgzTXyZd4kt21CjuepFBKKWAwkOEq0tl9x9FvMA9rmrZb0/+CzwZ8AD/T8fkZT37VNG2FpmnRptd3KtAi3WnWI/GksFlF3Mjq73kGygObTX9Pk4A5QPU05zqsaVrKivCa6adiyvZcxo37KKVc0T/3b2uaFqVp2mb0VeQHm3bxB8I1TVuu6f5GryxIKUNXYENurplON/IQT3Iim3uZ8sAyTdNua5oWASzl3vu7pqbXM1wptV8p1TYPRRgCfKhp2i1N046ix5Fhpm3F0BOkX0yv607gKGneewo5xlh7krId8FBKVTMF/n7oX6b0+/yo9G4P5R7kYkqpiaYPR0Y/B7I4zgP4AHg5B5epAexP8/t+oIRSqrjp9++A/kopF6VUGfQv4wrTdeoDP6NnwcWBycAfKk0XrlxeKyu1gDNa3hdc6gb8nYv9X0TvCpLp65wRpVRd9CTlVJqH/ZRS15TeReMbU0BMMQtIRL+Rqwd0BjJryq6B/hpEpnlsP2mCSg4cBerkYn9RcGwxnqCUaqmUikBvueiFXqOb0X4lgMrAYdPvBRZP0l+Lwo8nKVqht3guzmR7dt/x5YBRKdXE9Jl5AtiHXjsJBRtPWnP39Ush8aTwWUXcyKX5QJBSqrJSyh498VqRQTmigWPAFUw39XmIGxmpDCRpmnYizWNpvwu7gKNKqe5KKaPSu3rFASnPP6U1K9eUUqXQY8LeXBz2jFIqTCm1WynVK5t9s7qX+RF4RCnlrfSFLXuhxxhM93t/o7fyFANeARYrpXxzWkjTOUtzf5yuAXrFKnoL1nDT69oMCAA2p9m/UGOMtScpcLcWoxP6l+VSuu190Ju23gbOmrL7Run2uZHui14towtpmvaMpmlemfxkVVPwIXoN44UcPB839Oa2FCn/T+l7vAH9A3UbuIj+ZV1m2jYSmKxp2g5TK8Is9C9u0zxeKyte6Dc9uaaUckGvWcpRTYdSqix6wHsnl9fxQP98vG+qlQD9M1IXvTtYe/Sa3K9N+5dAT/peMNV4XQe+Afpncon0rx+m33Py+qWIRH8thWWwtXiCpmmbNU3zRK99/AK9i8o9TDcic4FZmqYdMz1cIPEkk2t5UUjxJJ2hwCJN06Iy2Z7ddzwSPcHZjP7avAuM0jRNK8h4opTqZCp7+pgo8cQ8rCFu5MYVU3mPAzGm8r+Yvhzon81WwBL0zz/kPm5kJMvvgql1Zzbwq+ncvwJPaZp2x7SvF3mMJ+gVHitMLaM58T1QCb319G1gplIqfQsnkKN7mT3olao3TT9J6F3AAAYB/2ia9o+pNXoV+v1ftxyWE/TXFe6P02ljzDxT+eLQPwNvpvtbE4neFa1Q2EqSMgC9uSp9EyumJq1xmqbVQM+O9wHLlFIqzW4+6b7oR/OrcKaa/I7of5xyIgq9uS1Fyv8jlVIG4F/0gOCK3q3AG/jMtE8A8HLaQIfeJ720UmqgujuoK6WZOdNr5aCct7j/5mNcmuv+BbRMV5YUHYCtmqbFKqXKpSlXZjcK3wIfpEk0sqWUcgb+BLZrmvZJyuOapl3VNO2I6Ut+FngN6G3aHIDeJ/NKmjJP5m7XjcNpytqK+18/TL/nJji6o3fbEZbB1uJJKk3TLqHXhs5Pd04D+vOO594uCPkeT7K4VmHGk5TzO6PffGXY1SuT55TyvFKe0wj01pMa6DcXg4C/lFKlKaB4opRqin5T1jtdTTNIPDEXi44befAueuJfFn386/vAWlOFQCpTErIZvQIkZZX4TOMGgFLqrzSPjwPGpdn3L9M5svwuKH1Cic/Rx6o5AG2Aaab4CBnHkwNprjsASNsiNTHNrqldvdS9k2xMyuiF0jRtj6ZpNzW9G98/6BUwj2e0L9nfy/wGnDCV3QM4zd1WuQCgT7rXtSV6hes9LWzoic2ANPumtNqkxMT0cTrlda2KPg5oCPrrWgN4TSmVtnuXO/cnkAVHK6TBL/n9g14b2NH0//XoLQspg9IyHKRs2remaXtxcjlgDZiE/iZn9HM4k2NeQO8redX0E4VeM7Enk/1/BT5K83t74Krp/z6m8nqm2f4YcMj0/8noWW9OX8NMr5Vuv/GkG4SFHpRiMnvtyGKgK/rAvdG5KGc4+oDclNdQQx+cOiCT/R3Rk7lfAUM2524ChJn+Xyqr55TBsZWBWMA9zWMb0z+3rD6T6P1B3y2M74z8ZPle2mQ8yeD4lkBEmt8VMAN9YK9zun3zNZ5kc61Ciydpjhtoet9VFvtk+R0HJgDfpDtmH3rFR77HE/QuY9eBRzM5h8STQvyxlriR7vj7/p5nsM9fwPPpHgsnk0kZgGnAd6b/5zhukMnAedNrGA9USvPYbO5OxPEKsDTdMcswDegG3gKmZnHdmWQwcB69UuEG4PEAn4mfyGSwO9ncy3D/IPW6QJTp/29k9ZzSXWdYZu8x+oQIndL8/gEw3/T/3sDedPt/C0xI8/tAYF1eX59cv56FdaF8L/i9waFiypcnfXBAb2WoaXrcHb3P30nTtlwFhzyW0wV9Fq6Uny+BRZhmbMlg/y6mD2919FaStdw749YZ9NoHO/QmzaXAXNO2hsAF9BtvZfqiP0yaP3q5vJYdei3KJ+g1RU5pXyv0/p/NMzl3WzK/qQgByuXiNfRL9xpq6E3Hzhnsa4/egrIso/fVVK5yptenLPoN04w0239HH/fjgd7SWBFok0XZtpveUyegJ2lm4zFtdzK9Dxr64H2ndMefABqb+/tU1H9sOJ4MTPN5D0DvErUkzfZJps+wWwbH5nc8yfRapu2FEk/SHLcSvVYzu/0y/Y6jd7k6gT4LkkLv7hMNVDVtz7d4YvrcXQP6ZXG8xJNC/LGWuJGmTJn+PU+377voXRhLmD63g9ErR7zQ/x73R+86ZESf1e8O0MN0bI7jBlnP7jUfveuRK/oEEWln92qDnkzUNf1eD717VGfT7/WBE1m8FjPJOElpTxYz/WVyrt6m18KAPuYsEmibyb5Z3sug34/8gD67nzN6i8gW07ay6PH1IdPr7oQeF/0zuM4wMk9SPkX/O+CNPjPbFe7O7lURPVFqb3rvKqKP5x2Z5vgpwGuF9h0rrAvle8HTBId0j6cPDj+gT8sYhZ6x/gVUM20LNO2bvjbipQIs93ukmTIU/QYiijR/ZNGn0L2GXiszA9NUf6ZtddFrbG6ZvqS/YZp2z7S9C/psUuGmD99vZHJTkYNrvcfdmTtSft5Ls/1Z4KdMztuWDG4q0AP1oQd8DTXunbZvEqapONGDl4Z+o5D2PW2V5vleMm2/YPp8pK259ESvCbmIHhT3Av2zKEug6f2IQe+/2zGDst7zk2ZbI9LVWsiPeX5sNZ6gr5txEf0m4iL6H5jipm0BpvLGpivvwDTny5d4ksNrFVo8QZ/CNJEMpjJPG0/SvK8ZfsfR/5B/AJxHvzk5CgxOsz3f4onp9Uwmk5pziSeF/2NNcYMs/p5zf9xwQk+krpi+y3u4eyPri36TG27adpA0N7GmfXIUN8g6SSmGXtl4x/T9GpBu+xj0G+hI9Mrbl9Nt3wk0yeTcM8k4SfmSXE6viz5uI8L0WuxP//0mzf1HBsemv5cpj17JehMIQ++em7Y1qYnptQ8zfY7+JoMKGrJOUhzRJza4jR6rX0q3vS/6dPCR6HHrM0w9Ukyfi4tAicL6jinThYXINaXP1rEX6KBp2pUcHvMaet/b1wq0cFZAKbUYfQB0gUx1KIQ1kXjyYCSeCHGXUqoz8IymaY/l4pgj6GO9jhRYwayYUuo5oGxhxltJUkShUkr1BQ5q5h0UKISwARJPhBD5QSnlgN6qkKdV6kXBkCRFCCGEEEIIYVFsYQpiIYQQQgghhA2RJEUIIYQQQghhUewK4qQ+Pj5aYGBgQZxaCJFHu3fvvqFpmq+5y5FbEk+EsDy2EE9CbtwhMVkjyM8t64OEEAUmq1hSIElKYGAgu3btKohTCyHySCl1ztxlyAuJJ0JYHluIJ8Nn/MeNqHj+fK6lmUslRNGVVSyR7l5CCCGEKHKUUmjI5EFCWCpJUoQQQghR5ChAJjgVwnJJkiKEEEKIIkcpSVKEsGQFMiZFFC0JCQlcvHiR2NhYcxdFAE5OTvj7+2Nvb2/uogiRKxJLLI9txxMlnb1smMQTy5KXWCJJinhgFy9exN3dncDAQJRS5i5OkaZpGjdv3uTixYuUL1/e3MURIlckllgWW48nekuKpCm2SuKJ5chrLJHuXuKBxcbGUrx4cQkCFkApRfHixaXmSFgliSWWxdbjiXzKbJvEE8uR11giSYrIFxIELIe8F8KayefXstjy+yFjUmyfLX9+rU1e3gtJUoRNUEoxePDg1N8TExPx9fXlkUceydV5AgMDuXHjxgPvI4SwThJLig6FTEEsCpbEkwcjSYqwCa6urhw6dIiYmBgAVq1aRZkyZcxcKiGEtZFYUnRIS4ooaBJPHowkKcJmdO3alb///huAefPmERwcnLotLCyMxx57jNq1a9O0aVMOHDgAwM2bN+ncuTP16tXjqaeeumcQ5Zw5c2jcuDF169blqaeeIikpqXCfkBDCLCSWFA1KIe0oosBJPMk7SVKEzejfvz/z588nNjaWAwcO0KRJk9Rt7777LvXq1ePAgQN8/PHHDBkyBID333+fli1bsnfvXrp378758+cBOHr0KAsWLGDLli3s27cPo9HI3LlzzfK8hBCFS2JJ0aBQMruXKHAST/JOpiAW+er9Pw9z5PLtfD1n9dIevPtojWz3q127NiEhIcybN49u3brds23z5s0sXrwYgPbt23Pz5k0iIiLYuHEjS5YsAeDhhx/G29sbgDVr1rB7924aNWoEQExMDH5+fvn5tIQQWZBYIgqctKQUGRJPrJMkKcKmdO/enVdeeYX169dz8+bN1Mczqi1LmWkioxknNE1j6NChfPLJJwVXWCGExZJYYvsUSJYiCoXEk7yRJEXkq5zUKhSkJ554Ak9PT2rVqsX69etTH2/dujVz587l7bffZv369fj4+ODh4ZH6+FtvvcXy5cu5desWAB06dKBHjx68+OKL+Pn5ERYWRmRkJAEBAWZ6ZkIULRJLREFTSlacLyoknlgnSVKETfH39+f555+/7/H33nuP4cOHU7t2bVxcXJg1axag9wcNDg6mfv36tGnThnLlygFQvXp1xo8fT+fOnUlOTsbe3p4ff/zRZgOBEOJeEktsn0JWnBeFQ+JJ3qiC+II2bNhQ27VrV76fV1imo0ePUq1aNXMXQ6SR0XuilNqtaVpDMxUpzySeFB0SSyyTrcaT5+fvZd+FcDa82s7MpRIFQeKJ5cltLJHZvYQQQghR5OgtKeYuhRAiM5KkCCGEEKLI0cekSJYihKWSJEUIIYQQRY60pAhh2SRJEUIIIUTRoyRJEcKSSZIihBBCiCJHcf86FEIIyyFJihBCCCGKHKVkCmIhLJkkKcLq3bx5k7p161K3bl1KlixJmTJlUn+Pj4/Pl2ts2rSJGjVqULduXWJiYvLlnAAff/zxPb83b948384thMgdiSVFi0IWnBcFR+LJg5MkRVi94sWLs2/fPvbt28fo0aN58cUXU393cHAgMTHxga8xd+5cXnnlFfbt24ezs3M+lFqXPhBs3bo1384thMgdiSVFi5IxKaIASTx5cJKkCJs0bNgwXnrpJdq1a8frr7/Oe++9x5dffpm6vWbNmoSEhAAwZ84cGjduTN26dXnqqadISkq651zTpk1j4cKFfPDBBwwcOJD169fzyCOPpG4fM2YMM2fOBCAwMJB3332X+vXrU6tWLY4dOwZAVFQUw4cPp1atWtSuXZvFixczbtw4YmJiqFu3LgMHDgTAzc0N0LsgvPrqq9SsWZNatWqxYMECANavX0/btm3p3bs3VatWZeDAgdJdQYgCJLHEdilkCmJRuCSe5I4kKcJmnThxgtWrV/PVV19lus/Ro0dZsGABW7ZsYd++fRiNRubOnXvPPiNGjKB79+588cUX923LiI+PD3v27OHpp59ODT4ffvghnp6eHDx4kAMHDtC+fXs+/fRTnJ2d2bdv333nXbJkCfv27WP//v2sXr2aV199lStXrgCwd+9evv32W44cOcKZM2fYsmVLbl8aIUQuSCyxTdKSIsxB4knO2T3wGYRIa/k4uHowf89ZshZ0/TTXh/Xp0wej0ZjlPmvWrGH37t00atQIgJiYGPz8/PJUzBSPP/44AA0aNGDJkiUArF69mvnz56fu4+3tneU5Nm/eTHBwMEajkRIlStCmTRt27tyJh4cHjRs3xt/fH4C6desSEhJCy5YtH6jMQlgciSUSSwqYUpY9JiUuMYltp2+y9th1yvu4MrxFeXMXyXpJPLHKeCJJirBZrq6uqf+3s7MjOTk59ffY2FhAb7ocOnQon3zySY7Pm9m5Ujg6OgJgNBpT+5xqmoZSOZ/uMqtm0pTzp7+GEKJgSCyxVcpiW1Ji4pN46NuNnA+LBsDOoOhSsySlPPNv3IEwD4knOSdJishfeahVKAyBgYH89ddfAOzZs4ezZ88C0KFDB3r06MGLL76In58fYWFhREZGEhAQkOm5AgICOHLkCHFxccTGxrJmzZpsaws6d+7MhAkT+PbbbwG4desW3t7e2Nvbk5CQgL29/T37t27dmsmTJzN06FDCwsLYuHEjX3zxRWo/UiFsnsSSDEksyT/6vZllZin/Hr7K+bBoPu5Zi8bli/HQtxuZsSWE/3WrZu6iWSeJJxmy9HgiY1JEkdCrVy/CwsKoW7cuP/30E5UrVwagevXqjB8/ns6dO1O7dm06deqU2r8yM2XLlqVv377Url2bgQMHUq9evWyv/9Zbb3Hr1i1q1qxJnTp1WLduHQCjRo1KPU9aPXv2pHbt2tSpU4f27dvz+eefU7JkyTw+eyFEfpFYYjsUljsmZfGei5TxcqZ/o7IE+bnRrVYpft1xntuxCeYumshHEk+ypgpiNo+GDRtqu3btyvfzCst09OhRqlWT2h1LktF7opTarWlaQzMVKc8knhQdEkssk63Gk7eWHeSfg1fZ83YnM5fqXlciYmj+6VqeaxfES52rAHDwYgSPTtjM/7pVZVTrimYuoXWQeGJ5chtLpCVFCCGEEEWOQlnktMtL915C0+Dx+v6pj9Xy96RZheL8vDmEhKTkLI4WwnZIkiKEEEIIYQE0TWPx7os0DPAm0Mf1nm2DmgZw9XYsBy5GmKl0QhQuSVKEEEIIUeRY4hTEBy5GcDr0Dr0a+N+3rVF5fXrYvedvFXaxhDALSVKEEEIIUeRY4sD5zaduAPBQjfsHI/u5O+Hv7cweSVJEESFJihBCCCGKHKUsb0zKvgvhlPdxpZirQ4bb65fzZs+58MItlBBmIkmKEEIIIYokS0pRNE1j34Vw6pX1ynSf+uW8uHo7lsvhMYVXMCHMRJIUYRM++ugjatSoQe3atalbty47duwosGu1bdsWmRJXCNsl8aRoUAqLylIuR8QSGhlH3XJeme5TP0AflyJdvqyHxJO8kxXnhdXbtm0bf/31F3v27MHR0ZEbN24QHx9v7mIJIayQxJOiQ6EsKUdJHRBfN4uWlGqlPHCyN7DnXDiP1C5dSCUTeSXx5MFIS4owi9BQ2LlT//dBXblyBR8fHxwdHQHw8fGhdOnSfPDBBzRq1IiaNWsyatSo1L7Hbdu25cUXX6R169ZUq1aNnTt38vjjj1OpUiXeeustAEJCQqhatSpDhw6ldu3a9O7dm+jo6PuuvXLlSpo1a0b9+vXp06cPUVFRAIwbN47q1atTu3ZtXnnllQd/kkKIDOVnLAGJJ0WJUljUmJR958NxtDNQtaRHpvvYGw3ULuMlLSkFROKJZZEkRRS6efM0AoOS6NrnDoFBScyb92B/JDp37syFCxeoXLkyzzzzDBs2bABgzJgx7Ny5k0OHDhETE8Nff/2VeoyDgwMbN25k9OjR9OjRgx9//JFDhw4xc+ZMbt68CcDx48cZNWoUBw4cwMPDg4kTJ95z3Rs3bjB+/HhWr17Nnj17aNiwIV9//TVhYWEsXbqUw4cPc+DAgdTAIoTIX/kdS0DiSVFiYb292HchnJplPHGwy/rWrF6AF4cvRxCbkFRIJSsaJJ5YHklSRKEKDYURo5Px7LUFt/7r8ey1hRGjkx+o1sLNzY3du3czZcoUfH196devHzNnzmTdunU0adKEWrVqsXbtWg4fPpx6TPfu3QGoVasWNWrUoFSpUjg6OlKhQgUuXLgAQNmyZWnRogUAgwYNYvPmzfdcd/v27Rw5coQWLVpQt25dZs2axblz5/Dw8MDJyYkRI0awZMkSXFxc8v7khBAZKohYAhJPihK9JcXcpdAlJCVz8FJEloPmU9Qv501Cksbhy7KoY36ReGKZZEyKKFQhIeDsHYuDXyQADn6ROHvFEhLiiq9v3s9rNBpp27Ytbdu2pVatWkyePJkDBw6wa9cuypYty3vvvUdsbGzq/ilNrwaDIfX/Kb8nJiYC+vSUaaX/XdM0OnXqxLx58+4rz3///ceaNWuYP38+EyZMYO3atXl/ckKI+xRULAGJJ0WFUgrNQtpSjl2JJC4xOctB8ynqmfbZdyGCBgHFCrZgRYTEE8skLSmiUAUGQswtJ+KvuwMQf92dmHAnAgPzfs7jx49z8uTJ1N/37dtHlSpVAL3/Z1RUFIsWLcr1ec+fP8+2bdsAmDdvHi1btrxne9OmTdmyZQunTp0CIDo6mhMnThAVFUVERATdunXj22+/Zd++fXl8ZkKIzBRELAGJJ0WJJS3muO9C9oPmU/i5O1HM1YFT16MKuFRFh8QTyyQtKaJQ+frCtEkGRoxugbNXLDHhTkybZHigmoqoqCiee+45wsPDsbOzIygoiClTpuDl5UWtWrUIDAykUaNGuT5vtWrVmDVrFk899RSVKlXi6aefTvdcfJk5cybBwcHExcUBMH78eNzd3enRowexsbFomsY333yT9ycnhMhQQcQSkHhSpCjLGZOy70IEPm6OlPFyztH+FX1dOS1JSr6ReGKZVEHMbNGwYUPNluZpFlk7evQo1apVy9UxoaF682pgIA8cBApCSEgIjzzyCIcOHTJ3UfIko/dEKbVb07SGZipSnkk8KTpsMZaAxBNLkjaefLr8GD9vPsuJj7qauVTQ7btN+Lo7MuuJxjnaf9ziA6w6co3db3cq4JJZL4knlie3sURaUoRZ+PpabgAQQlgPiSUir5TCIsakJCYlcyo0ilaVfHJ8TJCfG/N3XuDWnXi8XR0KsHRFi8QTyyJjUoTIQGBgoFXWUgghLI/EE8tkKWNSQm5GE5+YTOUS7jk+pqKvGwCnQ6XLV1FTlOKJJClCCCGEKHKUhYxJOXFNn1GqSklJUoRIS5IUkS8sadXeok7eC2HN5PNrWWz5/VAoi3h+x65GYlB6F66cKuPtjKOdQWb4yoYlvL9Cl5f3QpIU8cCcnJy4efOmBAMLoGkaN2/exMnJydxFESLXJJZYFluPJxbTknI1kkAfV5zsjTk+xmhQlPdx5XTonQIsmXWTeGI58hpLZOC8eGD+/v5cvHiR0AddmlXkCycnJ/z9/c1dDCFyTWKJ5bHleGIpY1KOX4ukai66eqWo6OfGoUuy6nxmJJ5YlrzEEklSxAOzt7enfPny5i6GEMLKSSwRhSrdKt3mEJuQRMjNO/SoWzrXxwb5urH84BViE5Jy1QpTVEg8sX4W390rNBR27tT/FUKIByHxRAiRIiVFyWt3oPyIJyevRaFpUCUXM3ulqOjnRrIGITely5ewTRadpMybpxEYlETXPncIDEpi3jwLaJcVQlgliSdCiLRSGlLykqPkVzw5bprZq3Jeunv5ugJw+rokKcI2WWx3r9BQGDE6Gc9eW3Dwi8ThujsjRregY0ejLLQjhMgViSdCiPSUqS0lt+lFfsaT41dv42BnILC4ay5LARV83FAKmeFL2CyLbUkJCQFn71gc/PRaBge/SJy9YgkJMWuxhBBWSOKJECK9uy0puUtTQkLAuVh0vsST49eiqOTnhtGQ+/Exzg5Gyng5y1opwmYVSpKSl36bgYEQc8uJ+Ot6E2j8dXdiwp0IDCyQIgohrITEEyFEfoiO1v/N7ZiSzTfO4Np7E8nx+mD1+OtueY4nx6/eztUijulV8HXjzA1JUoRtKvAkJa/9Nn19YdokAxGLWxA1ry0Ri1swbZJBumYIUYRJPBFC5Id58zS++lqPH0HVcjem5GTYLZwdjCTfdgEgZm9QnuJJeHQ8127HUTkPg+ZTlCvmzIWwmDwfL4QlK9AkJW2/Tbf+6/HstYURo5NzXGsRHKwIOWVk+SJXQk4ZCQ42/3SBQgjzkHgihMgPKbHEscY5ADx7bs1VLLkQFkOjCl7sHN8SR6ORUePC8xRPTlzTW0AepCWlXDEXImISiIhJyPM5hLBUBZqk5Ec/cF9faNQIqfEUooiTeCKEyA8pscTOLR4AB7+oXMWSC7eiKVvMhVIlDdQL8OTI9Vt5KkfKzF55mX44RbliemvOhbDoPJ9DCEtVoEmK9AMXQuQXiSdCiPyQEksSoxwBiA/N+ZiSyNgEwqMTUpODhgHFOHLlNnfiEnNdjhNXI3F3tKOUp1Ouj03h7y1JirBdBZqkSD9wIUR+iIxNIMkhlm9+SOD2X42ImtdG4okQIk9S7k1i9wcCcPuPJjmOJSnjP8qakoMGgd4kJWvsvxCe63IcvxZJ5ZLuKJX3rqflipuSlFuSpAjbU+DrpAQHKzp2NBIS4kpgoHSzEELkzqFLEdR6b2Xq7z5PgL3BQFUPJ1bHuXF9tRcNArxpVN4bRzujGUsqhLAWwcGKG8WMfLUO9u02EFQuZ4lCSjJQtpgzAPXLeaMU7Dp3i+ZBPjm+vqZpnLgWSdeapXJf+DQ8nOzxdLbnvLSkCBtUKIs5+vpKciKEyJvibo78r1tV3BztSdY04hKTuX47lkvhMRy7Gsna49fRNHB1MNKqki+P1StD+6p+ONhZ7DJQQggL4OmhJyZexXI+s1dKt6qUlhRPZ3sq+7mz61zuxqWERsYRHp1AlRJuuTouI+WKuXBeZvgSNqhgkpTQ4/BzF7B3Blc/cC8JxcqDb1XwqwZOngVyWSGE7SmVcJ5Rp8eCg6seS9xLQdlKUL8K+DQjMlHx39kw1h67zqoj11hx+CrFXR3o37gsw5qXx9fd0dxPQQhhKW6cgJmPgLMXje+48ZQR7E4mQkBt8A68u8JjJi6ERePmaIeXi33qYw0Cvflz32WSkrUcL8qYMmi+8gPM7JWiXDEXjl65/cDnEcLSFEySYrQDgx3EhMONkxB1DZLiTRsV+FWHck0hqCNUaKPffAghREaUEdAg8jJc3gt3QvXfAeyccC9dnw6BLejQsDPvP9KGTadvMe+/80xcf5qpm87Sr2FZnmsfhJ9H3genCiFshDJAUgKEnqBC+GXesI+E3+fp21yKQ9kmUL41VH4IilW47/ALt2Lw93a+ZxxJg3Le/LrjPCeuRVKtlEeOinH86oPP7JXCv5gzq45cy1WSJIQ1KJgkpVhFGPbX3d+TkyHiAoQeg8v74MIOOLAQdk0HoyNU6gS1+0LlLmAntZ5CiDSKV4Th/9z9PTEObp6G60fg0h64sB02fQ0bv8DOxYd2NR6jXbt+nO3ahimbzjDvv/Ms2n2Rka3KM7ptRVwcCqWXqxDCEhUPgif/BWDZzgt8uHg7a4aUxC/6FFzcCee3wfF/YMU4vfdHrd5Qqy94BwB6S0p5n3srVhsGegP6uJScJiknrkXi4+ZAcbcHv+cpV8yF+KRkrt2OpbSX8wOfTwhLUTh/rQ0G/QvuHaDXTgAkxpuCwXI4vASO/QXOxaD+EGj0JHiVK5SiCSGsjJ0jlKiu/9TqrT8WcwtOrYFjf8PeObBzGuV9q/JJ45GMbtqdL9Zf5Pu1p1i85xLvd69Bx+olzPschBBmZzAoInEhtkQDKN4KGg7XN4SdhZMr4fAyWDte/6nYAa3xKC7eSqJ15XsH2ZYr5oKPmyN7z91icNOAHF37+LWoB1ppPq2yaaYhliRF2BLzjSy1c9C7enX9FF48AoMWQ2AL2Po9fFcHlozSx7YIIUR2nL31hKXPDHjlJHSfAHZO8PfLBMxuyIRS/7J4WHVcHY2MmL2LZ+fuIexOfPbnFULYLKPpDihJSzdwvlh5aPIUPLEcnj8Abd+Aa4dR8/rxp3qFNjFrIOnuuihKKer4e3LwUkSOrpucrHHyWmS+JSkpa7bIDF/C1ljG9DdGO318Sr85ekBo+gwc/RN+bKInK+HnzV1CIYS1cPKA+oNh1Hp4chUEtIQNn9JgSSuW19vBGx3LsfLIVR76diPrjl03d2mFEGZiMI0rSUrOYnYv7wBoOw5eOMiZNt+RgJHWh9+CHxvrLS2mBKeWvyenQqNytKjjpfAYouOTqJIPg+YBSns5Y1D6eBkhbIllJClpeZWFhz6CFw5Ci7Fw5Hf4oSGsehfiosxdOiGEtVAKyjaG4F9h9Bao0Abj+o94an8f1ne6SnEXe4bP3MnH/xwlISnZ3KUVQhSylEHmyelbUjJi58BB7050i/+ES12mgdEBfhsK0zvBpT3UKuOJpsHhy9nPspUyE1d+taQ42Bko5eksq84Lm2N5SQoQGgo7j/gQWvcDeG431OwFW77Vay6O/JFacyGEEFkJDYWdOyHUWBP6z4Un/gWPMpRZ/yL/eH7KS3USmbLxDP0mb+NqRKy5iyuEKER2piQlMSln9xTHLkSjYSC53OPw9Ba9W2n4eZjanmbHPsaDqBx1+TpwMQKjQVE9h4Psc6JsMWfp7iVsjsUlKfPmaQQGJdG1zx0Cg5KY908Z6PkTPLFS73e+cDAsHAJRoeYuqhDCgt0XS+Zp+tTnT66CR7/HEHqUsSefZEW97Zy+eovuEzaz53zuFmQTQlivlO5eOWlJmTdP4+tp0STH2FO1GsxbYNC7lY7ZCU1G43JgNqudxpF0bHm259p3IZyqJd1xdjA+8HNIUdbbRVpShM2xqCQlNBRGjE7Gs9cW3Pqvx7PXFkaMTiY0FCjXBEZtgA7vwokVMLGJPm5FCCHSyTKWGAzQYCiM2QXVHqXq0e/ZUeIzKhkv03/ydpbuvWju4gshCkFKd68sx6RwN544VryCwTnh3nji5KlPADRyLfH2noy6+AYsfRriIjM8V3Kyxv6L4dQp65Wvz6VcMReuR8YRE5+Ur+cVwpwsKkkJCQFn71gc/PQvt4NfJM5esYSEmHYw2kGrl+CpTeBZFhYMgr9ehAQZLCaEuCvbWALg6qPPBtZ3Nk53LjEn6XVe9tnBiwv28cOak2jSrVQIm2ZISVKy+a6nxBODg54AZBhPStfj98ZzmZD4GNqB+TC5tb6OUzpnbtwhMjaRuvmcpPgX06cevhQu90PCdlhUkhIYCDG3nIi/rg8mi7/uTky4E4GB6Xb0q6p32Wg+Fnb9DFPb64u7CSEEuYglANV7wOgtKP+GPBXxDYv9ZjBx1QHeWHKQRBlQL4TNMqZ098qmJSUlnmjJ+v6ZxZPq5Xz5MrEvRzr/qq8FN70T7Jh8zzja/RfCAfI/STGtlSJJirAlFpWk+PrCtEkGIha3IGpeWyIWt2DaJAO+vhnsbOcAnT/U11eJvApT2sLRvzLYUQhR1OQqlgB4lILBy6DdW9S/vYaN3h+xfdd/jPl1L3GJ0n1CCFuU0+5evr4w9ScFGsQfKZdpPKlVxhOArQlV4OnNENQJlr8GS0ZC/B1AH4/i5mhHRV+3fH0uZUyLOF68JeNShO2wqCQFIDhYEXLKyPJFroScMhIcrLI+IKgjPLUBileEBQNhwxeEXtf0GX1kbL0QRVauY4nBCG1eRQ1ajC9h/Ov6HlFHVzF4yi42bUuUeCKEjUldJyUHXTsf76OhjBrD+jhnGk983Bwp7emkz/Dl7A39f4X2b8HBRfDzQxBxkV1nwwnw8CTsZjbxKJdKeDhhZ1BckrVShA2xuCQF9FqLRo3IvNYzPa9yMHwF1O4P68azbuyT9Ox/4+6MPkKIIinXsQQgqAOM2oBjsXLMcvicKhd/ZcDU7ZSvFifxRAgbkrpOSg56dUaZFmmsGmSXZTyp5e/J/ovh+i8GA7R+FQb+BmEhxHzfDoere9i7wTXf70+MBkUpLyfp7iVsikUmKXli70Roy0m8s/kdeldZwtKBXQnss/zuDBxCCJFT3gHc6PEvK0534kOHmbxdeiLFh6xm5HPxEk+EsBHGHA6cB1JXknd1sMtyv6YVinPuZjQhN+7cfbBSJ8IeX8W1CEcWOH3AIzUX3ztDWD4p4+UsLSnCpthOkgKEnFNMPDWaZxPGUlud5Y/Sr1DZ/+S9M3AIIUQOnL3szvDNU5mR+BCj7P5mguv3lOixhaMnE81dNCFEPrg7JiX7ppTIWP177+aUdZLSqXoJAFYduXbP46cjq/HIkWkc08rxk/23PFFqyf0zhD2gMl4uXJQkRdgQm0pSUmbg+P1qRwbHj6M4t/mnaw8quh81d9GEEFYmMBDuhLny5uVn+TghmEeN25lV+m0m7d4oaxEIYQNSZvfKySR+Kd293B2zTlL8vV2oXsqDlUeu3vN4YCDc9DDQP+Yt1iXXZbz9DF6o+h2BAfnX5auMtzPXImOJT5RZCYVtsOokJTSUewbIp53RZ+2vT9Npzr94ekKx37vCpd3mLawQwqJlHk9a8vWcrxi5YiLNjEd5JfRVXpy5Vmb9EsLKGUx3QNnN7gUQlcOWFNBbU3afu8XqzXGp8STJIRbnwBuE7a/KwF9+55fDwbzV7DN89/wvZ4NicsDf2xlNg6sRsflyPiHMzWqTlHnzNAKDkuja5849A9DSzujzz45auDy7HBw9YFYPOLfNzKUWQliinMSTj/8ciLHfbOrYneP5iy/x5pz1so6KEFYsdeB8TsakxJuSlGxaUgCSz/uRrMGgty+kxpOFuy6QjMaKCQH89ZsnXX6cCE2ehu0T4a/n8yVR8ZdpiIWNscokJTQURoxOxrPXFtz6r79vANo9M/oUqwBPrAD3EjCnF4RsMWvZhRCWJVfxpNojGAcupJLdNUaeGcvHv22QlemFsFJ3u3tl/x1OHZOSTZISGgrvvOSGlqRwanJcjydPJzF3+3laBvlQv7KrHk/8DNDlE332rz2z4Y8xkPxgrbNlvE1JiszwJWyEVSYpISHg7B2Lg18kAA5+kVkPQPMoDcP+Bs8yMLc3hGwurKIKISxcruNJxfbYDfqNCnahBB95lh//3FpYRRVC5CNDLlpSUsakZNfdS48ncSijfk4Hv0jcql3mWmQsA5qUu3dnpfR1VNq+Afvmwu/PPlCLSilPZ5RCZvgSNsMqk5SUAfLx190BiL/uTky4E4GBWRzkXtKUqJSFuX3h/I7CKKoQwsLlKZ5UaIPd4EUEGG/QYddTzF+/rxBKKoTIT7lpSYmKTcSgwNnemOV+KfEkIcwFAE0DhwYnKObiQMdqJTI+qO04aPcm7J/3QF2/HOwMlHCXtVKE7bDKJCXtAPmoeW2JWNyCaZMM2S/Y5uYHQ//Qu37N7S2D6YUQeY4nqnxrjAMXEGS8Ss21Q1m1+3jhFFgIkS/uTkGcs5YUN0c7lMp6pfiUeBK+oDXR/zQk5mA5PDw1RrQuj4NdFrdcbV672/Vr+Wt6dpMHZbydZUyKsBnZjwCzUMHBio4djYSEuBIYmIsVpd1LwtA/YUY3fYzK8OXgV60giyqEsHB5jSfGoHYk9Z1DlQUDif9jEHs9l1EvqEyBllUIkT9ym6S4O9nn6Lx340kJAgNL4OtbK2cFavcmJMbC1h/A0R06vpuz49Io4+XM3gu3cn2cEJbIKltSUtwzoDU3PP1hyO9gdIRfesKtkIIonhDCiuQ1njhU60Lso5Opo04RPWcA567LDYIQ1iA3K85HxSbi6ph1V6+08hRPlIJOH0KD4bD5a9jyXS4O1vl7O3MlPDZHiZcQls6qk5QHUqw8DF4KCTEw+zGIum7uEgkhrJR7g96EtfucFuzj9JRBRNyJM3eRhBDZMJi6biXnortXgVMKHv4KajwOq96BPb/k6vAy3s4kJmtcj5S1UoT1K7pJCkCJ6jBwEURdg7l9IC7S3CUSQlgp3zYjOd9gHO0TN7P1p1EkyGKPQli03HT3ioxLxC2H3b0emMEIPSdDxQ7w5/NwfEWODy2TulaKDJ4X1q9oJykAZRtBn5lw9SAsHAKJ8eYukRDCSpV7ZBwnyg+ma9QyVk9/U9ZQEcKCpc7ulYOv6Z24RNwLoyUlhZ0D9J0NpWrDb8Pgws4cHeZvWitFpiEWtkCSFIDKD8Gj38HptfDXC3meVUMIUcQpReXB33OkWEe6XvmJTUsnmbtEQohMGEx3QDnq7hVbSN290nJ0gwG/6RP+zOsHN09ne0gZL33qY5mGWNgCSVJS1B98d0GljV+YuzRCCGtlMFB19FyOO9Wmyf63OLh1ublLJITIQK4Gzscl4lrYSQqAmy8MWqxXns7tA9FhWe7u7GCkuKuDTEMsbEKBJClRcYnW2c2hzetQJxjWfQT7F5i7NEIIK2VwcKL0U4u5bixB2ZUjuHTqgLmLJIRIx5DDxRyTkzV94Hw2q80XmOIVIXgeRFyEecGQkPWgeH9vZxmTImxCgSQpZ2/coefEraw6ci1HzagWQyl49HsIbAV/jIHz281dIiGElXL39sMwaDHJGNB+7ced8FBzF0kIkYadIWeze92JTwQo3DEp6ZVrCj0nwYXt8OfYLLull/F2ljEpwiYUSJJSxsuZm3fiGDl7F12/28Tv+y6RmJRcEJfKfymD1TzLwvyBsoaKECLPylSoxoVOU/FNus7Fyb3RZGIOISxGSnevxOySlDh9pj6ztaSkqPk4tHsLDiyATV9lupu/twuXwmOss0eLEGkUSJJSzNWBdS+35Zt+dUjWNJ6fv4/2X23g1x3nibOGaTldisGAhZCcAL/2h9jb5i6REMJK1WnRha3V36FKzD6OTH9KJuYQwkIopVAKkrP5TkbFJQAU/sD5jLR+BWr1hbUfwpHfM9yljJczcYnJhEbJek3CuhXYwHk7o4Ge9fz594XWTB7cAG9XB/639CCtPlvH1I1nuBOXWFCXzh8+QXqLyo0TsGQUJFtJS5AQwuK07TuWlcUGUOPKEk798625iyOEMDEqle2YlMhY/X7FIpIUpaD7D+DfCJaOhquH7ttFpiEWtqLAZ/cyGBQP1SjJsmeaM3dEE4L83Pjon6M0/3Qt36w6wa07Ftz9oUJb6PIJnFiuD6YXQog8UErR8qnv2G7XiICdH3L9wGpzF0kIgX6Pkt3sXlGmSlWzd/dKYe8E/eaAkyfMD4Y7N+/ZXCYlSZFpiIWVK7QpiJVStAjy4deRTVnyTHMaBRbjuzUnafHZWj76+wjXbmc9W4XZNB4F9QbDpi/h8FJzl0YIYaVcHB0o+cQczlMSx6XDiQs9a+4iCVHkGZXKduB8lCW1pKRwLwn950LkNfhtKCQlpG6SVeeFrTDLOin1y3kzbWhD/n2hNZ2ql2D65rO0+mwd/1t6kPM3LWxub6Xg4a+gbBNY9gxcO2zuEgkhrFRg6ZJc7ToTkhO5Ob0PxFtYvBOiiDEaFNnN65PakmJJSQpAmQbQ/XsI2QQr30592N3JHk9ne+nuJayeWRdzrFLSne/612PdK23p3dCfRbsu0vbLdbwwfy/Hr0aas2j3snPUx6c4eugzfsXcMneJhBBWqkWTxqyqOp6SMac4N2uEDKQXwowMORo4b5qC2FK6e6VVpz80eRp2/AT756c+XMbLWRZ0FFbPIlacDyjuysc9a7Hp9XY82bI8K49c46FvNzJy9i72nreQhMC9pJ6oRFyExSNlIL0QIs8e6zuc3zyHEHDpb66s/NrcxRGiyNJbUnLW3cssK87nROcPIaAl/Pk8XNkP6IPnZUyKsHYWkaSkKOHhxJsPV2fruPa80LES/50No+fErQyYup3NJ2+Yf87vck2g62dwahVs/Ny8ZRFCWC07o4EOIz9nvWqM77aPuHNyk7mLJESRZDQYsl0nJSouEUc7A/ZGi7plustoD31mgnMxWDAYYm5RxrTqvNnvm4R4ABb5jfNyceCFjpXZMq49b3arxqnrUQyavoPHJm7l38NXzbuKfcMnoM4AWP8pnFhpvnIIIayaj7sTHsFTuaD5kjh/CNrtK+YukhBFjtGQ/YrzUXGJltnVKy03X723x+3LsGQU/l5ORMcnER6dkP2xQlgoi0xSUrg52jGydQU2vtaOj3vW4tadeJ76ZTddvtvIsr1mWsU+ZSB9iZqwZISsSC+EyLP6lQPZ2fh77BPvcO3nAZBk4etHCWFjjCpnUxBb3KD5jJRtpC+bcHIlra/MAGSGL2HdLDpJSeFkb2RAk3KsfbkN3/ari0LxwoJ9tPtqPXO2nyM2oZBXsXdwgX6zQQMWDoVEWdVVCJE3fbp1Zo7fy5QM38O1ZW+auzhCFCkGQ86mILaYNVKy02gE1O5H0JEJtDAc5FK4DJ4X1ssqkpQUdkYDj9Urw/LnWzF1SEOKuzry1rJDtPp8HZM3nE6dgaNQFKsAPX+CK/tgxRuFd10hhE1RStF3+IssMXahxMFJ3Nn/u7mLJESRYczBYo6RcYm4OlhJkqIUPPINyT5V+N5+AmFXQsxdIiHyzKqSlBQGg6JT9RIsfaY5v45oQpUS7nyy/BjNP1nD14W5in3Vh6H5WNg1HQ78VjjXFELYHC8XByoM+o4DyRVQvz+NFiYLPQpRGIwqZ7N7WfyYlLQcXDH0m40z8bTc99o9Cz0KYU2sMklJoZSieZAPc0Y0YdmzLWhSoTjfm1axH//XEa5GFMIq9h3ehXLN4K8X4MbJgr+eEMIm1S1fkiMtfyAhSePGzAHSjVSIQmAwqGzXSbkTbyVjUtJQvlX41mUs5e4cgLUfmrs4QuSJVScpadUt68XUIQ1Z+WJrHqpRkhlbQ2j9+TreWHKAkBt3Cu7CRjvoNV1f8HHhUFlBWgiRZ/06teCXEq/je/sIoUteM3dxhLB5OW1JsZoxKWmcKfkQfzl0hS3fwfEV5i6OELlmM0lKisol3PmmX13Wv9KWPg39WbznEu2/Ws/YeXs5dvV2wVzUswz0nALXD8NyubEQQuSNUopBw55hvvFRfI/M5M6+JeYukhA2zWBQZDdRaGRcIm6O9oVToHxUtpgLb8UOQCtZG5aNhvAL5i6SELlic0lKirLFXPioZy02v9aOka0rsOboNbp8u4knZ+5k97kCWMW+Ukdo+RLs/UXGpwgh8szLxYHKA79kX3JF1B9jZHyKEAXIzqBISs48S4lPTCY+MRk3R2Mhlip/lCvmQni8kVsPT9GnN1/8pIxPEVbFZpOUFH4eTrzRtRpbx3XgpU6V2X3+Fr1+2kr/KdvYdDI0f1djbfcmlG1qGp9yKv/OK4QoUupXKMnBZt+QmKRxc9YgSCykyUCEKGIMBkVSFrcBd0yzhlrbmBTQkxSAEK0kPPotXNgB6z4yb6GEyAWbT1JSeLrYM7ZDJba83p63Hq7G2Rt3GDz9P3r8uIUVh/JpFXujHfSeDkZ7WDQMEgph4L4QwiYNfKg1M3xewSfiEDd/l2nOhSgIRpX1ivMpSxu4OVlfd6+A4nqSciEsGmr1hgbDYPM3cGq1eQsmRA4VmSQlhaujHSNa6avYf/J4LcKjExg9ZzcPfbuRJXsukvCgq9h7+sNjk+DqQVj1Tv4UWghR5BgMioHDx7DQ0IXiB6cRe+QfcxdJCJtjNGQ9cD4yNqUlxfq6e/l760nK+ZumCX26fAp+1WHJUxB51YwlEyJnilySksLRzkhwY30V++/618WgFC8t3E+7L9fzy4OuYl+lCzR9Bv6bDMf+zr9CCyGKFB83R/z7fc2R5AASF4+G25fNXSQhbIpBZb2YY4zpXsDJ3vqSFCd7IyU8HDkfZkpS7J2h9wyIvwNLRkEWY3GEsARFNklJYWc00KOuvor9tCEN8XV35O1lh2j52TombThNZGweB5l1fA9K1YHfn4WIi/laZiFE0dG8Shm21/8CQ2IsobOHQvIDVKAIIe5hNKgsu3vFWXGSAvq4lHNhaZZG8KsKXT+Dsxtg89fmK5gQOVDkk5QUBoOiY/USLHm6OfNGNqVaKXc+XX6MFp+u5euVxwnL7Sr2do56jUVSAiweKTcWQog8G/JoJ372fBbfG/8RvvJTcxdHCJthNGTdkhKbqP/tdrbSJKVsMRd9TEpa9YdAjcdh3cdw4T/zFEyIHJAkJR2lFM0qFueXJ5vw+7MtaF7Rh+/XnqLFp2v54M8jXImIyfnJileEh7+C81th45cFV2ghhE2zMxroOfxV/qYl7tu/JOHsVnMXSQibYFBZt6TEJuhdoqy5JeXq7dh7u7ArBY98o6/xtuhJiAk3W/mEyIokKVmoU9aLSYMbsOrF1nStWZJZ2/RV7MctzsUq9nX6Q+1+sOFTOCc3FkKIvCnj7YJDj++4lFyc6HnDIaYA1nsSooixMygSs0xSUrp7WeftUrliLmgaXApPV8Hq7AW9psPtS/qyCfm5HIMQ+cQ6v3WFrFIJd742rWLfr1FZluzVV7F/bt5ejl7JwSr2D38FXgF6ty+5sRBC5FGnekH8U+VjXOJCuf7r03JjIcQDMmQzu5cttKQAdwfPp1W2MbT7HxxeCnvnFHLJhMieJCm5ULaYC+Mfq8Xm1/VV7Ncdu07X7zbxxMyd7D4XlvmBju76+ilRV+HP5+XGQgiRZ8P69uIX54H4XVjO7a0/m7s4Qlg1o1IkZzUmJaUlxc5Kk5S0a6VkpOWLENgKlr8GN04WYsmEyJ4kKXng566vYr/l9fa83Kkye8/fotdP2+g3eRsbT2Syin2ZBtD+bTjyO+yZXfiFFkLYBCd7I62Hj2ebVgOH1W+QfP24uYskhNXKbp2UlIHzjlba3cvXzREne8PdtVLSMxjh8Sn6ZD+LnoDEuMItoBBZsM5vnYXwdLHnuQ6V2DKuPW8/Up1zN6MZ8vN/dJ+whRWHrtw/GK/5WKjQFpa/DqEnzFJmIYT1CyrhybUO3xOdbM/N2YPlxkKIPDIYFFnkKMTGJ6EUONpZ5+2SUopyxVwy7u6VwqM09JgIVw/Amg8Kr3BCZMM6v3UWxsXBjidblmfDa2359PFa3I5NYPScPXT6ZgOLdqdZxd5ggJ6TwcEFFkuNhRAi73q0asDCMuPwjTrO1aVvmrs4QlgloyKblpRkHO0MKKUKsVT5K9skBaBqN2g0ArZNgFNrCqdgQmRDkpR85GhnpH/jcqx9uS0/BNfD3mjgld/20/aL9czaGqL3bXUvCT1+hKsHpcZCCJFnSikGDBnNEmMXSh6eyp0jK81dJCGsTvYD55OsdtB8irKmJCXDruhpdR4PvlVh6WiICi2cwgmRBUlSCoDRoHi0TmmWP9+Kn4c1pKSnE+/+cZiWn61l4vpT3A7oCI1GmmosVpu7uEIIK+XhZE/5gd9yMrkMiYtHoUVdN3eRhI3TNI0v/j1m7mLkm5wMnLfWQfMpAoq5EB2fRGhkNr037J31aYljI+D3Z2WSH2F2kqQUIKUU7auWYNHoZswf1ZTqpT35fMVxWny6lm/VYBKLV4WlT0uNhRAiz+pVKMWexl/jlBjF5VlPyo2FKFDrj4fy47rT5i5Gvsl24HxCstWukZKiop8bAKdDc7C+W8ma0PlDOPkv/De1gEsmRNay/eYppcYopbwLozC2SilF0wrFmf1EY/4c05KWQT58t+kij19/gsTocGIXPyU3FqJgJcabuwSAxJOC0qfbQyzwHkmZ0I1cW/ODuYsjbFRyssaXK49Twdve3EUB8ieeZJ+kWH93r4q+KUlKVM4OaDwKKnWGlW/BtcMFWDIhspaT6oGSwE6l1EKlVBdlzaPHLEAtf09+GqSvYh9UqwnjEwbgdHYNv095l7M5XcVeiNzQNH2iBssg8aQAGAyKrk+8yybVAO/NHxB78aC5iyRs0L+Hr3Lx8mX+UC+ZuygpHjieGA2KpKy6eyUm42jlSUpJDydcHIw5T1KU0mf7cvKERU9CQkz2xwiRF2c3Zbk52yRF07S3gErAdGAYcFIp9bFSqmJ+lK+oCvJz5+u+dXnyxY857tGcLpcn8szXsxnz6x4OX44wd/GELdkzG47+ae5SABJPCpKvhxP2j08kQnMh/JfBcmMh8lVSssZXK48zwW0GrrFXzV0cIH/iiUHloCXFSqcfTmEwKCr4uuasu1cKN1947CcIPQqr3im4womiKzoMlozKcpccffM0fUqIq6afRMAbWKSU+vxBy1jUlS3uSpVRs7F39WK25xS2Hb/Ew99vZviM/9gVksUq9kJkIjQUdu7U/yX0BKwYB+XbmLtYqSSeFJymtaqyrtoHlIw7y9l5FlPbLaxYSjyZs+kSDcL+olXiNlQHy7lpfdB4YjSo+9c0SyPOBrp7gd7l6/T1HLakpKjUEZo+C/9NgeMrCqZgosi4595E0+DPsXAn6zHZORmTMlYptRv4HNgC1NI07WmgAdArH8ot3HwxPD4J35gzbK2/llc6V2b/xQh6T9pG30nbWH/8evZTBwoBzJunERiURNc+d6hcJZqwaU+CnZO+Po8FkHhS8Hr2GcIfzj0pf+ZXQnctM3dxhBVLjSf9Ivl50Vres5+NVr4tNBtj7qIB+RNPsu3uZQMD50FPUi6FxxATn5S7Azu+CyVqwe/PQKRltKAJ65P23iQwKIn/Js3Se3hkU+GRk2+eD/C4pmkPaZr2m6ZpCQCapiUDjzx40QUAQR2h2Rgc9/7MmDKn2Px6O959tDoXbkUzbMZOHvlhM/8czGAVeyFMQkNhxOhkPHttwa3/ej4Z9jTF4g4Q0XYCeJQyd/FSSDwpYPZGA/Wf+IajBOL493PE37pk7iIJK5Q2nhTru4Yfvb8iOs6ZsDaT9IWJLcMDxxODUiQnZ749NtF2WlIAztzIZWuKnSP0ng7x0fr6KVm9WEJkIP29SYN+86l5cRzx/m2zrfDIyZiUdzRNO5fJtqN5K7LIUId3oGQtWPYMLrGhDG9Rng2vtuPzXrWJjk/imbl76PjNBn7bdeHuKvZCmISEgLN3LA5+kbQyHGC0xzKmHx3KCcPD5i5aKoknhcPf15sbnSdinxzLxZ+Hyo2FyLW08eQVu4XUNIQwdtO3nAm1mAqPfIknRgPZtKQk4WwLSYqfK5DDaYjT860CXT6BM+tg+4/5XDJh69LGEgcS+KnkZ0QnOXOkyuRsKzwspjpEoNdY9PpZH/C6TK+xcLAz0LdRWVa/1IYJA+rhaGfk1UUHaPvFemZuOZv7plthswIDIeaWE243kvnKfhLH4svx2qqPCQw0d8mEObRq3oJV5V6kQuROTv7+ibmLI6xMSjxpEX+EUXZ/M+NON5Yd7G5z8cRoGjifWZdqvbuX9ScpgcVdUYrcj0tJ0WAYVHsUVr8Pl/flZ9GEjUuJJYlRDrxqt4AahnOM/ncCZaqWzPZYSVIsjW9l6PopnFkP2+6ud2A0KB6pXZp/xrZkxrBGlPJ04r0/j9Dys7X8uO4Ut2MTzFdmYRF8fWHaT4ov4mfjkRTDkPm/MuFHV3x9zV0yYS6dB7/GFvvmBO7/itDj281dHGFFfH1h4reX+MbtB47FBfLK9KlMm2SwuXhiNNXkZtaTOjYhCUcbGJPiZG+krLdLzqchTk8pePR7cPODxU9CXB7PI4ocX1+Y/BO0s9/HSLt/mB3fmcde7ZKjWGL93zxbVH+oXmOx5gO4tOeeTUop2lX1Y9HTzVkwqik1y3jyxb/HafHJWr749xg3o+LMVGhhCYIrTKFrhZVcqzOev3fUIjhYliEpypwc7PAfOpUbmhcJC58gIea2uYskrIWm0SRsJJ4qmqvNp3PsuJtNxhOj6S4oo2mIk5M14hKTcbKz/pYUgIq5nYY4PZdi+iQsN0/Ditfzr2DC5pUIOsLXHt9z3liej5IHUK9tzpJcSVIsUWqNRQlTjUVkhrs1qVCcWU805q/nWtKqsg8T15+mxWdree+Pw1wOlzUSipyrB2HV21C5KwG9R9pcjafImwB/f063+pqSiZc5On20uYsjrMT11d9SNXI7a8qNpe2jDW02nhgMeuKVnEF3r7hEfSyXLXT3An3w/JnQqAebgKd8K2j1MuydA4cW51/hhO1KTsZn9Qt4qFiMfacThwN7zofn6FBJUiyVSzF4fCrcCoG/X8ly15plPJk4sAGrXmzDI7VLM2f7OVp/vo7XFu3nTF6bdoV1ib8Di54A52LQ40c90RXCpGXHx1hfYgi1b/zN4X+nm7s4wtJd2Y/3lo9YRwOa9bPtGnOjKVZm1JISm6CP+bSFKYgBKvq5EZeYzKUHrcRsOw78G8GfL+j3KEJk4dqqb6gXv4udlV+idOX6+Lg5sPf8rRwdaxvfPFsV2AJavwoH5sP+BdnuHuTnxpd96rDhtXYMahrAH/sv0+HrDTw7dw+HLskq9jZtxTi4cRIenwyuxc1dGmGBmj/5BUeMVSm37U2unTtm7uIISxUXRfSvQ7mpuXGuxed4uzmau0QFymhqSclohq/YxJQkxXZaUgBO5XXwfAqjPfSapv9/8QhIkjGxIhOX9+Gz/WNWJjWgeveXUEpRr5w3e6UlxUa0fg3KNYO/X4KwMzk6pIyXM+91r8Hm19vzdJuKbDwRyiM/bGboz//x31lZxd7mHFoMe2ZDyxehQltzl0ZYKCdHR9wGzETTFBFzhpAQL+PXxP2S/3kNp8gQPnF6ieB29cxdnAJnMLWkZNQFKjYhpbuXbdwqVS6hJynHrmbchTxXvAPh0W/h4k5YL7MHigzERcKiJwhTXvxa8jWKmSo86pXz4uyNO4Tdic/2FLbxzbNlRju925fBqHfnScz8TQ0NhZ079X8BfNwcea1LVba80Z5XH6rCoUsR9J28jT6TtrJOVrG3DbdC4M8X9ab3dv8zd2mEhStXsRrHm3xE5YTj7Jzxcpb7po8nogg4uAjD/rn8mNiDro/2wdFGBoxnJbUlJavuXjbyOni5OODv7Zx/PStq9oJ6g2DT13BmQ5a7Sjwpgv55De3WWcbEPE3jGkGpD9cv5w3AvgvZd/mSJMUaeJWF7hPg8l5Y+0GGu8ybpxEYlETXPncIDEpi3ry7AdfDyZ5n2wWx+fX2vPdodS7dimG4aRX7vw9cyTA4CyuQlACLngQ0vendaG/uEgkr0KjbE+ws/ijNr/zC7jWLMtwnq3gibFTYGbQ/n2cfVdhWdiQP1ch+DQNbYMiqu1eCbXX3AqhVxpOD+dn9u+vn4FMJloyEqIwzEIknRdD++bD/Vw5XHMUOrRodqpZI3VTb3xOjQbHnXHi2p7ErwCKK/FS9OzR8Erb+wImENng37pQ620poKIwYnYxnry36ip7X3RkxugUdOxrvmZHF2cHIsBblGdAkgGX7LjFp/Wme/XUPFXxcGd22Io/VLYODneStVmPteLi0C/rM1Jvehcih2iN+4vyX+wnc+BKrEhtSt35gruOJsCGJ8bDoCWKTFGPinmHyo7VQRWTyDWNqd6/7t6V097KFdVJS1CzjyfJDV4mITsDTJR8qthxcoffPMLUDcQufZkPF2cw+fpjNZ65jUODqYMfllZXx7HUeB78oiSdFwY1T8NdLUK45PyQ9ThmvO6ldDQFcHOyoWtKdPTkYPG8737wiYGHEeA6FVqfYplE0rXMxtTYiJAScvWNx8NP7mTr4ReLoHss//2TctOpgZ6Bvw7KseqkNPw6oj7ODkdcWHaDtF+uYIavYW4dTa2DLt/oqwDV6SlO6yBVHZ3e2lJiJK9E4rh9C+SqxmcYTo1sc9s7x7N1rxgKLgrXmfbi8l5diR9C4VgOiL3kWmVhiZ2pJScwgS7G1gfOg12IDHLqcj60pJWuxs/hHOJ5fxc5/XmXT8Ru08Ank2XZBBHh44t7mCA5++mB9iSc2LiEWFg0HOwdie0xiw8lb1Czmx40b91Z6lPdx5WpEbLankyTFSoSGwvBnHHkmYQwujnf4ZUg/Rj2dQGgoBAZCzC0n4q+7AxCxvTzh15x46Z2sm1aNBsXDtUvx13MtmTm8Ef7eLryfZhX7iBiZscMS3Qy5SsLCUSQWqwYPfSJN6SLXQkNhxKt1efv2CFo77uftkS8wYnTyffHkztFSXJrShjvxyfTsLZ8tWxSxcwVsm8AKp0dZT3OmvBBUpGJJ6jopGbSkxNnYmBSAmqX1JCU/u3yFhkKn73uwIqkRrznNo2bMeZZ9WJUh9aowfXATorZVQdMgKcqBS1NaSzyxYTF/vA1XDxDRbiKfLnImNjGJZTM87osnLg5GonNQIS5JipVIqd08X8yLtxOG09zpEG+3+YSQEPD1hWmTDEQsbsHtOW2J2FKFkoO34B68Hs9eW1JvPjKjlKJtFT8Wjm7GwqeaUctfX8W+5adr+XzFMW7IKvYWY/6viRz6YAQJ0VE0+WY6k2c4p3bNceufs/dbiJR48ptjS/5IasYrTvPp2HbJPfEk/LcW3Fxem5IDtlNm1Ho8e8tny9Ysm32RxEWjORhZhefDe3F1TSU8HtlZpGJJ6orzGY5Jsa3ZvQC8XfXB8/mZpOw7FofHQ/t5LWEk1/Bmsu/nlPS7SkgIlPBT/DC6IpEbqmN0i6f0k5skntioTVN+x/ngFH46PIpSHR5i4h9X0TRwbnPwvnji4mBHTIIkKTYjbe3m4uTWLIxqzyt1v6GSnT6jRnCwIuSUkW8+csW79L1dv5y9YgkJydl1Gpcvxszh+ir2rav48tOG07T4dC3v/n6Ii7eiC+bJiRwJDYVTsz6nTblNvKsN4Uq7mzz/cjJOXnl/v0XRdDeeePC/hCc5n1yCSXVeI8lwFtDjybLFRjxLxMlny0aFXkvEd9OTODrGMtZhFNExLkSd9C1y77chR4s52k5LCpgGz1/MnyQlKVljysG9KPtEbt70Y0z8WEpwi68bv0ZggP6aBgcr5r0fSMIFH+zc9UrPovL5Kipunj5HrTNj2BtXia8rtsSl/V6cqlxOXVc6/fvtZG/M0dACSVKsRNrWkqh5bRkzcyaRjpXwWj0CIq+l7tOtG8SG3+36FX/dnZhwJwIDc3e9mmU8+XFAfVa/1IYedUszd8d52n6xnld+2//gC0GJPLm5awP/a/o5i5Na8ltSGxz8InHxiiX65oO/36JoSRtPrs7rxqClsyiubpP895OE39H7CderB/G35bNlqxJXjqdFme28mTics1opUJAUWfTe75QpiJOLyOxeoP99Px8WTUT0g3fp/vW/8+w8f5PegTUIX9iazb+O4J2N79Kj0l/4npmcul/9+oqIVbXRkvTXO/66W5H4fBUJifE4/jEMZYCxPEMCdrhUvIHBIYmEMBfg/nji4mAkPimZxKQM+lmmIUmKFUlpLVm+yJXDRz3xfHKWvljO4ichWQ+m6ZOZiMUtmDbJkOdZNCr6uvF57zpsNK1i/9eBy3T6ZgPPzN0tq9gXpshrVD4wghNhQbx+eSygiL/uTlykE99+nX/vtyg60saTZeuacq35uzRP3s2/U/9HUrKW77FEWJATKyl1+htmnAhmWVJLAJJuO2Nvp4hYVLTeb2MWLSkxNtjdC/Jv8HxEdAJfrzxO0wrF+PLZsqnx5KUFY6FyV1j5FlzcDej3JlO/c+L26joAxB6oUCQ+X0XCqndwi9jDUysmcPqaaT0Ug0bCdQ/C57fKMJ44mxL/7Lp8yRTEVsbXlzRf6urw8Jfw+7Ow4bPUxfyCgxUdOxoJCXElMJB8CQKlTavYj2kfxIwtZ5m97Rz/HLxK68q+PNu2Ik0qFH/wi4iMJSfB4icxJERypsEyrjxfFWevWGLCnZg2yUBwsOLxnvn7foui4Z540vk5QkK20OvyDH5b0pj+vfsXSCwRZhZxEZY+hVaiJr+qfmhhCdz5pzEx192ZOd1Ax46qSL3fhiK0mGOKtIPnWwT55Pk83689SXhMAu88UgOlVJp4ouCxiTC5Dfw2DEZvBGdvgoMV7TuUpt/0MyT2PkXvvmX0fYX1OvIH7PgJmjxNjyrd+XN0Mq6VruLScR+96wfwxmd2hITY3RdPnB1MSUo2Xb5sq3qgKKo3COoMgA2fw6nVqQ/7+kKjRvn/R8bHzZFXH6rKlnHtea1LFQ5fiqDflO30/mkr647JKvYFYt3HELIJHv6KbsNqpNZWhZwyEhysB/iCer9FEaIUAUOncMuxNO0Ovs7aXYcA+WzZlMR4/aYxKYE/q3zMyZho3n2sGst/9UqNJ0Xt/U5dJyWj7l6JSTjYGVITGVuRMnh+/4XwPJ/jTGgUs7aG0L9RWaqX9rh/B5di0GcGRF6BpaNTp08r4ad4+7HKXAyPZvHui3m+vrAAN0/rleRlGkCnD1Jb5zsMu4aHkz0fjSqTaTzJaUuKJCm24OGvwK86LB6p15IVAg8ne55pq69i/373GlwOj2H4zJ10+34zfx24LKvY55cT/8KmL6HeYKg3EJCbRlFwlJMnHkPm4aWicfnzKU5eCTd3kUR+WvUOXNxJeOdveHNDLM0rFueJtv5FOp4YjSnrpNz/NysuIRknG13guFmF4mw5dSPbMQGZ+XzFcZzsjbzUqUrmO/k3hIc+ghMrYOt3qQ+3r+pH3bJefL/mJHGJsi6bVUqIgYVDwWDUF5S2cwBgX+g1tl+6QnCTsqmtJRlxMW3Lbhpi2/z2FTUOLtB3NiQl6LVkifGFdmlnByNDmwey/tV2fNG7NnGJSYz5dS8dv97Agp3niU/MWwAUQPh5WDIKStSCbl+YuzSiiHD0r01M589pqg6x4+eX82VwrbAAh5fq3TKaPsMbxyoQn5TMxz2Lzsrymbm74nzG3b1sbdB8inZV/bgdm8i+PLSmHLwYwYrDVxnRqjy+7o5Z79x4FNToCWs+gLObAH3Zgxc7VeZyRCx/7b+Sh9ILs/v7Fbh2EB6fCl7lADh29TYvzN9LrTKevNixcpaHOzlIS0rR4hMEPSbAxZ2w8s1Cv7yDnYE+Dcuy6sU2TBxYH1dHI68vPkibL9bx8+azRMcnFnqZrEWGq8UnxMKCwaAlQ99ZYO9stvKJoser+XBCK/VlUMIipv88UVpGrUiG8ST0OPw+Bvwb82/pp1l+6CpjO1Qi0MfVbOW0FMZsxqTYapLSspIPRoNi3fHruT7261XH8XKx54mW5bPfWSno/gMUq6ivRH77MgCtK/lQyc+NGVvPSjdxC5ZhPNk9C/bNgdavQqVOABy9cpsRs3bh6mjHlMENs/3euNjLmJSip8Zj0GwM/DcF9i8wSxGMBkW3WqX4c0xLZj3RmLLFXPjgryO0/GwdP6w5KavYp5PpavHLX4Ur+6DnZChe0axlFEWTb98fCPOoxojQT5m8dKW5iyNyIMN4EhcJCwaBvTPhj0zjzT9OUL2UB6NaVzB3cS1C6jopmSzmaGsze6XwcLKnQYA3647lbjXF3efCWHc8lKdaV8TDyT5nBzm6Q785EB+d2ttDKcWwFoEcunSb3edu5f4JiAKXYTy5tBv+eQUqtkdrM44DF8N5du4eun63idsxCUwZ0pCSnk7ZnlsGzhdVHd+DgBbw5/Nw9ZDZiqGUok1lXxY+1YxFo5tRt6wXX606QYtP1/LZimOERsoq9qGhZLhafOSG2bBnNrR6Bap2M3cxRVFl70Sx4Quws7Oj3f5XWLLjhLlLJLKQcTxJIm7hs/oA194zeH/DLcKj4/myTx3sjfLnH9Ksk5JBz+TYRNttSQFoV8WPI1duczUiNkf7a5rGl/+ewMfNkaHNA3J3Mb+qem+PCztSe3v0rFcGDyc7ZmwNyWXJRUHLKJ68/kIoSfOGorn58av/23T+bgvdJ2xh/fHrPNc+iE2vtaduWa8cnT91TIp09ypijPbQewY4e8GCgRAdlnFzXSFqGFiMn4c14u+xLWlbxZdJG07T8rO1vFPEV7EPCQFn73tXi28etA3XDS9DhXapU0oLYTbeATj0nU4VwwXs/n6B1XtvmjWWiMxlFE9ea/k1jqd/h47vsjqmMkv3XuLZdkEZz8ZURKXkahm3pCTZ3PTDabWrqs+WsOFEzrp8rT8eyrYzNxnTriIuDnlYwaLm43d7e+ybx50IO9qULceKg1e5EhGT+/OJApM+njj7hfNLj+Fw5xpjEl/kf/9ewc3Jjo961mTrGx14uXMVPF1y2LLG3QVSY6UlpQhyLwF9f4GIS1yZ8CQVKsXf353IDGqU9mTCgPqsMa1iP+8/fRX7lxfu59T1SLOVy1wCAyHm1t0Vnj1vJDKz7Qg0t1LQ+2d91gwhzMyuSmfiWv+P7oYtbJ33Pt2G3TB7LBH3Sx9Pmt0+yVsNPiE26HHC6oxm3JKDVC3pzrPtgsxbUAtjyHLgfDKONtrdC6BKCXdKeTrlqMtXYlIynyw/SmBxFwY0yWUrSlod34fAViT+/gKPNd/N/K99SUzS+N/0c3k/p8h36ePJK/FLaFN6M6/HD2dfUgWmDmnI0mdaMLBJAJ7OOU9OUqQkudmNV7bdb19RV7YRka2/pFT0Wj4ZPvqe7kTmrgWtYFrFfsOr7RjcLIC/D16m0zcbGf3Lbg5eLDqr2Kdd0TtuQXN+5CdKeIRjHDBXn2NeCAsRVfNVloV05U3n2XTqMR3P3pstIpaIu9LGk+J/ludHj2+IdKqKY58feHPZIW7HJPBNv7o42OiUunlVVAfOg94tu20VPzafukFUXNY3i4v3XOTEtShe61L1wT5DRjtutJ/J5QgfFvbrS7lHV6IlGFl99iJXrslsoJYibTzptD2cpz2WMiuxE+GV+7DqpdZ0ql7igc6fsk6KdPcqwo65DmPm8cGM8VxMd8NWHPwicfaKJSTE3CXTlfZy5t1Ha7Dl9fY82zaILadv8OiEzQyevoPtZ24WiRk/goMVIScN7H/7PVr6b8Pu8QlQstY9+5i7u54QIecUz2/+nlNaGX60/54gv9M4e0dbTCwRuuBgxbkj0Wwa/gRenuA1ei5LD4ez/NBVXupcmWqlPCSepJPSkpLhOimJyTadpAD0a1SWqLhEZm45m+k+d+IS+WrlCeqX86JrzZIPfM2z13wYtn4axY0RTHT4DkfHOIyucSzdnvuZxkTBCQ5WXNx6gIntXmJHclW2VHyZHwfWT20FeZBY4mRvQCnp7lWkBQbCCyu+YHtsDT63n0zV8KvEhDsRGGjukt2ruJsjrzxUha2mVeyPXrlN/ynb6T1pG2uPXbP5ZMU3ZBp+52dCyxehVu97tmU6+5cQhSgwEG6EFmfY1XcAmGb/JZ51DhEQIJ9Hi5KcjM/GUbjcOYGh3ywuUJJ3fz9M48BijGxVQeJJBuyMWaw4n5Bks4s5pqhb1ouO1fyYsvFMprNvvvvHYUKj4njz4er5sq5OYCDsON2Yl24+TxPDMd5NnEPSHQe2XrvwwOcW+SjyGm7/BHM92Y3ppd7jh8GNcTSN0XrQWKKUwtneKIs5FmW+vvDTRCf6z1xC6B0/prt+wZwJ1/H1tczaefc0q9h/2KMGVyNieWLmLrp+t4k/9tvoKvZnN8KKcVDpIWj/9j2bMpv9y5LeM1E0pDT9H1jQmyf+nU4FdZVJtf/HsiMnAcuMJ0XSuvFw/B946GMSA1rz/Py9AHzVtw5hN5XEkwykLOZYFLt7pXixU2VuxyYyfdOZ+7Yt3XuRRbsv8ly7IBoEeOfL9VLiyaxf/se3B8YwyG0ln5TZztaz17kcHiPxxBIkxJI8fyBJd27yivF1xg9sl5qg5Ne9ibO9URZzLOqCgxU7D5Ykost8SntH0DNmAAvnRlt0bZqTvZHBzQJZ/2pbvuxTh4SkZMbO20v7r9Yz77/zxCVm/aG2GjdO6Qs2Fg+CXlPvGyif0Ww9ltRdTxQtwcGKkFNG/vdDF6Jbf0In4x601e8xbuIli44nRcb++bDpK6g3GJo8xfdrTrLnfDgfPV6LssVcJJ5kwmDIvCUlJiHJZtdJSatGaU+61SrJ9M1nuRR+d5atY1dv89bSQzQOLMbYDpXy9Zop8aTFuA+IC+zKwLivaaX28/aMCxJPzE3T4I/nMFzayUvxoxnV7zH8PO6ufZJfscTZwZjtOil5mENOWBtfX/DtWBvKTEVbMAjjidF49XoCe787OFx3Z8ToFnTsaMTX19wlvZe90UDvBv48Xq8MK49cY+L6U7yx5CDfrj7ByFYVGNCkXN6mQbQE0WEwr5+emATPByfP+3ZJmV3D4bo7Dn6RxF93t8jueqLo8PXVf9BGkRR9jKd2/8xrF8rg1b8r9t4xFh1PbNr57fDHcxDQEh7+mm1nwpiw7hS9G/jTvU5pQOJJZjJrSdE0zbSYo+23pAC81Kkya45ep9PXG3i6TUVu3olnzvZzeDjb823/utgVwLo6ejwxQp2p8HMXfrr+A4+ElMaz1xUc/KIknpjLpi/h4EK+SuhNiWb9aVfF757N+RVLctKSYqV3eCJPqj3Cxarv0Yt3uZyo+Cqxb5oM2NVig4DBoOhSsyQP1SjB5lM3+HHdKcb/fZQf153iiRblGdIsMFfzc5tdYjwsHALh52HIH1CsfIa7pTSJjxjdAmevWGLCnZg2yWCx75MoQpTC2O1zws6eYfzNn7lQ3JdtyTWsIp7YnLCzMH8gePpDv1+4HpPM2Pl7CfRx5f3uNVJ3k3iSscxm94pL1GeaKipJSpCfOytfbM0n/xzjq1UnMBoU/RqV5YWOlfBzz34F8Qfi6A7B89AmtGOm53h6On/ATTwlnpjD4aWwdjyr7Vqz1GkAKx+qct8u+RVLXByyH5MiSUoR49T+eWb+dpLnas3hvObH3CuPWk1tmlKKVpV8aVXJl93nwpi47jRfrTrB5I1nGNi0HE+2LF/wwfRBaRr8ORZCNkHPyRDQLMvdg4MVHTsaCQlxJTAQCdTCchjtSX58Fqc/78Qk72/oFf8eR0KrWE08sQnRYTC3D2hJMGAhSU7ePD9tB5GxCfzyZGNcHe/9Ey/x5H6ZdfeKS9CTFEcbHzifVkBxVyYNbsDhyxG4OtgR6ONaeBf3Kkf0o/PxXdKNqXZfE5z4JpHXi0s8KUznd8CSp7jkXodnQ59gyhO1M+2tkh+xxEnGpIj0fP0UTr2+Yc25tnxs/JkGO+OtsjatQUAxpg9rxPLnW9Guqh9TN56h5WfreGvZQS6EWfAq9hs+g/3zoO3/oE7/HB3i6wuNGskNhbA8Pv5enGi4iNhYV2bYf0EJhzC++SFBPquFITEOFgyC8HPQ/1fwqcQ3q06w7cxNPuhRk6olM15VXuLJve5297r38VjT2Ednh6LRkpJWjdKehZugmPjWacQU5w+oazzFV6Hzub24mVXen1ilm6dhXn/i3UrRM+wZutYNpE3lrF/4B40lLjkYkyJJShHUf4ADdcb/QoJnFX7vP4zgtgfNXaQ8q1bKgx+C67H25bb0ql+GBTsv0PbL9by0cB8nr1nYKvZ7ZsP6T6DOAGjzmrlLI0S+6DEkAIchCyllF8Uc3w9YEbYh06lMRT5JToalT8G5LdBjIgQ059/DV5mw7hR9G/rTt2FZc5fQahhMd0FJ6VpSYk01vE52RS9JMadHnxzMR4kDeaTsCq7PeIvgnNXliQcRFQpzeqEBrzi8TZxDMd56pHqBX9bZQVpSRCZ8ynjgMnIRBmdPmNsbboWYu0gPJNDHlU8er83G19oxrHkgyw9epdM3G3nql10cuBj+wOd/4CkRj6+AP1+Aih2g+/eQD3PNC2EpvGvUx67/bKobzvNi2HhGzdjKnWxWsC7KHiieaBr8+z+973inD6B2H05dj+Llhfup4+/JBz1q5nt5bZmdKUtJSteUEptQtMakWIoKvm4cDhjEArtHcTkwCbZ8Z+4iWbwHiidxUfo9YORV1tSfwB8XnHmja1V83BzzvZzpOdvbSUuKyIJHaRi0WO828MvjejZt5Up5OvP2I9XZMq49YztUYtvpm3SfsIXB03ew7XTeVrF/4AXQLvwHvw3TV5LvOxuMVjTIX4icqtwZ1f17WhkOMODKp4yc+V9qbbS464HjyZZvYcdP0ORpaD6WiJgERv2yC0c7Az8NaiA31bmU2t0r3duQ2pJSBKYgtjTBjcsxLqof18o9DKvfhX2/mrtIFuuB4kliPCwcDFcPEtl9Gq9us6dhgHehtcQ6OxikJUVkw68qDFgAty/DnMchNsLcJcoXxVwdeKlTZba+0YE3ulbl6JVIgqdu5/GftrL6SM5XsX/gRYuuHtJrKTxKwcDfwNEt709KCEtXbxB0eIcexq08dPEbnpq9y3bWNcoHDxxPds+E1e9BzV76go3JGmN+3cOFsGgmDqxPaS/nAiy9bUrp7pWcnEl3L0n6Cl3XmqUo7ubMO+pZqNAWfh8DR/8yd7EszgPFk+QkWDISTq+F7t/z9pEyRMYm8lHPWqmTSRQ0Fwc7ouOzbnGXJEVAuabQ7xe4fgTmBUNCTPbHWAk3RzuealORza+348MeNQiNjGPEbH0V+9/3XSIx/WjJdB5o0aKwM3riZ+8KQ34HN7/sjxHC2rV8CZo/x1DjSuqf/Ymn5+yRRMXkgeLJoSV6l9GgTvDYJDAYeP/PI2w6eYOPHqtFkwrFC7Dktit1CuL0Y1JSpyCW26TC5mBnYEDjsqw8Ec6FTtOgdD1YNBzObDB30SxKnuOJpsFfL8KRZdD5I1bYd2TZvss82y6IKiXdC7jUdznZG1O7VWZGvn1CV6kTPD4Fzm3V59xPjDN3ifJVyir2615py9d965CYrPH8/H20/2pDlqvYpyxaFH9d/+LmeNGi8PMwqzskJcCQZeBVDsiHsS1CWDqloNOHUG8wz9stpfLJaTwjiQrwAPHk+HK91rNcU73LqJ0D0zef5Zft53i0UgXaBcpA+bwyZLKYY0pLiqMMnDeLAU0CMCjF7D2hei+E4kF6Jer57eYumsXIUzzRNFjxBuyZBa1e5mbtkby59CA1SnvQv1ZQod6fuORg5jxJUsRdNXtB9x/g9BpYOFTvr2hj7I0GHq/vz8oXWjNpUAO8XOx5Y8lBWn++jmmbztw32Ddl0aKIxS2ImteWiMUtsp8S8fZlmPUoxN2GwUvBV18M6YH7ogthLZSCR7+Dmr0YZz+fcidn8dQvu4v8GJU8xZNTq/XFX0vW1rvmOrjw94ErjP/rCHFnSjBvfFmJJw8gpSVFuntZlpKeTnSpUZIFOy8QY+cJg5fp3abn9IaLu81dPIuQ63iiaXp3UdOYNq3dW/xv6UEiYxPp6FaHoCpaod6fOOfguyVJirhX/cHw8FdwYjksfsImExW4u4r978+2YM6TTSjv48r4v4/S4rO1fLf6JOHRd593cLAi5JSR5YtcCTllJDg4i/6aKQnKnZswaAmUrgvkQ190IayNwagvWFrtUd61/4Vyp+YyfMbOIj/rV67iyem1esu2bxUYvAScPNlx5iYvLNhH/BUvlNsd3PptkHjyAO4OnM94MUfp7mU+Q5oFcDs2kSV7L4J7CRjyB7gUgzk94dIecxfPIuQ4nmgarPtIn3ij4RPQ5RMmbTzLv4ev8XTLKrz1omuh35/kZA0i+faJ+zUaAV0+g6N/6rNS2WiiAvoq9i0r+TB/VDMWP92chgHefLP6BC0+Xcsn/xzl+u1YIIeLFkVcgpkPQ+RVGLQI/BumbnqgvuhCWCujPfT6Gap04wP7mVQ7P4fB03cQEV2011HJUTw5uRp+7Q/FKuq1yM7eHLwYwZOzduHn4kzctho4+EUBEk8ehCGzlpREaUkxt8bli1Hb35PJG87o40c9y8DQP8HJE2Y/Ji0qJtnGE02DNR/Axi+g3mDo9hXrT4Ty+b/HeLROaVoUL2+W+xPp7iXyrulo6PYlHP9bX9U4IdbcJSpwDQK8mTa0ESteaEWHaiWYuukMLT9fx5tLc7CKffh5mNmN5MhQjjRcSqhz03s257kvuhDWzs4B+syCao/yjt0vNL4yl76Tt6VWAIgMHF+BNi+YO66VufHIn+Dqw6nrkQyd8R+ezvZMDm5CzDV3iSf5xGhQJKZLUlLWb5AkxXyUUoxpF8T5sGj+2H9Zf9A7AIb9Ay7e8MtjcH6HWcto8TQNVr0Nm7/mernhhDb7nqPXohg7by9VSrjzWa9alC+vzHJ/It29xINpPBIe+QZOrtSn0Y3L3Qru1jpIvGpJD75PXcXen992XaTtl+t5ccE+TmS0in3oCfi5C/ERt2g3ezGtR9W4r09nnvqiC2Er7Byg9wyo0ZNxxrn0CJ9Jr5+2cCY0KkeHW2ssyZODi0ieN5A9l2tQZ9J8Amp58f3MSAZO24HRoJg7ogk1KzhLPMlHRoPKYMV5U3cvO7lNMqdO1UtQtaQ7E9aduju5gVdZPVFx9dUTldNrc3XOIhNPkpPgrxdg6w9M2jeC6h+MJ6hZBI9P2I6rox1ThzTExcHObPcn0t1LPLiGT+j9ys9t1Werig7L0WG2MEhcX8W+Fhtfa8fw5oH8e/gqnb/ZyMjZu9h3IVzf6fI+mNGF5IR4Ws/6k5Mt4jLt05mrvuhC2BqjPfSaDvUG84xazLOxU+k9cTO7z2UdU2whluTYrp/RFo9gy8XGBCeMI/Hx/Xj23cQXu3YQn6Ax58kmBPq4AhJP8pNRqQy7e9kZFHZGuU0yJ6UUz7WvxJnQOyw/dOXuBs8y8MQKKFYBfu0HR37P0fmKTDxJjNdnBNw9k893vMQHxbrj1n8DXr22EhVuZGLvZpQt5pK6uzniibSkiPxRpx/0mwPXDsP0TnArJMvdbW2QeElPJ956pDpbXtdXsf/vbBiP/biFTydMIOnnbmj2zhxq8i+nkoKy7dOZo77oQtgqg1GfQbDZGPpry/nS8D3Dpm7i7wNXMtzd1mJJpjQN1n0Mf71IhG8ngtf9Qryv3t3IofgdlF0yb7Voet8aBhJP8ofRoEi/ZFZsQlKObqJEwetSsyRBfm58s+oE8Ylp3ig3Pxj2F5Sqq89I+t/ULM9TZOJJbITe++XQYi5UfZ/Pj76cOn5NGSB2XR0Sw13uO6yw44mLg122+0iSInKmajd9QcI7N2BaJ7i8N9NdbXWQuLdpFfst49ozo+4JXrnxDsfjfXjC+AknHF2JueUofcSFyI5S0Hk8dB5P+6QtzHf+nDd+3ch3q0+ipetyY6ux5B5JCfqK2hs+g7qDSOj1K7dueJMQpt9EaEmK8N8b07pO4S2yVtQYFCRn0N3LUZIUi2A0KN7oWpXToXeYvvnsvRudvfV7kypd4Z9XYNU7kJzxAoFFIp7cvgwzusG5LfDYJG43Ho1946Opm+ND3Yi+5G0R9ybSkiLyV0AzeHIl2DnBz10zbV616UHiycm4bfqIdsfeQwW25GDneZyKdeOV33dT88UtxB4oT9T81tJHXIisKAXNn4Ne06mefILVHh+ybM0Gxszbe88UxTYdS0DvPvtLT9g3B9q8Dj0m4FvSnje+vIXRLY7kKCfC5rdk8seeEksKkN6Skn4K4iSZftiCdKhWgs7VS/D9mpNcvJVuIhsHF+j7CzR8ErZ8B78Ngfg7953D5uPJpT0wpR3cCiGp/wJm3GlKv1kbcK14g6itVYia14aIRS0t5t4kJ2NSsm9rESIt3yowco0+d//CIdDuLWj9CqE3FCEhehBIGYQ1YnQLnL1iiQl3yvBLERrKPcdYvLhIWDoajv0FDYZh7PYl/Yz29GpWjT8PXOan9afx6HiAEq4uPNGsAj3b+ANSEydEpmr1Rnn64zt/ICtc32PkoTE8fi2KTx5uQFKEK4GBNhpLAEKPw7z+EHERek7Ru9UCS/ZcZFbIASqWcuX1Jk1o8K6T9TwnK5XhwPnEJJnZy8K8270GHb/awPt/HmHqkIb3bjTa6Wu8FQ+ClW/Cz10geB54+t8TH2w2nhxaAsueAVdf9nVayOt/J3H82hFaV/blwx41cHnJ1eKekyQpomC4+elzlf85FtaN58LOvTT+7EcSXB2IuaV/6YODFR07GgkJcc3wSzFvnsaI0ck4e8fec4zFCj0BCwah3TzJheqf4txkNL5Gvbx2RgM96/nTo04ZVh+9xo/rT/PJ6kNM33GSka0qMKBJOVwd5asmRIbKNUWNXIvjvP7MSvqMr8P689hP0cTtrELUwXJMm2Qg5JQNxRKAw8vg92dJNjpzrMnf+JZuTPFkje/XnuTb1SdpVqE4kwY3wNPZ3twlLRIMGQycj4mXlhRLU8bLmRc6VuKT5ceYseUsw1uUv3cHpaDZM3qisugJmNyatcV+5tGXW90TH86cNLBydxyHbofwe8RtJn0RS3xiMtVKeaBuebDwizI4OijriCdJibD6Xdg2gXDPxoz3eINFi8Px93Zm0qAGPFSjBMq0YKmlJCcpctLdS6XvA5wfGjZsqO3atSvfzyvM754aBh+NqDWTcNzwFueT/XgueQz7rtUiYnELQk4ZM/1ChIZCYFASnr224OAXSfx192yPMatDi+GPscQmOdFz/nR23mmUZfDSNI2tp2/y47pTbD19Ey8Xe4Y1D2Ros0C8XR3M8AR0SqndmqY1zH5PyyLxxHbdE088ooj97TmcTi3h34SGvJY0irBob8LmtCXkuEOGscHqYkliPKx5H7ZN4IZTI5r98DO37IsTG2VHm5cPcjjiGo/XL8Onj9fGwcKnvrWleNLskzW0DPLhiz51Uh8LnrKdhKRkFj3dvLCLKLKQnKzx9NzdrDxyjSmDG9KpeonUbffEE3WKxLkDUTdP8MXtAUxx6kLsdQ/iT/hTuds5Lt+OwdHOQK0ynpTycsbOoDhw4TanrkeC0vOd+FA3Iha1tNx4cvsyLB4J5zYz9cwAPiv+CAl2io6lK/LjmCCraAms/NZyTn7ULdNYItW7IscyqrE8fXo0q5dWYUGv4SzlHT4rFcz3XvUJCXHL9Eud+eA1V8sKBHFRsPx12DeHhBKNqDX+Z+48FIKb33ocrrszYnQLOna8P3gppWgR5EOLIB/2nr/FxPWn+Xb1SaZsPMPAJuUY0aoCJTyczPOchLAQ98cTV06fnU7Yhrp82vp9ltuN4wX1LNv62bNuX336dvK87xxWE0sAbp7Wa3ev7COm1kiCRnyIS8+duPkdxCXRwKFbGi+1r8bYh8qn1nyKwpHRmJTYxCTcpAXc4hgMim/71aP/lG2MnbeXaUMb0iLIJ4N4UpHzt1dT7uQYxlWZQ+ukPbzo9wzX/KLwdCjGq/0q06l6yXve4507odvQm7h23w6Ag28ULmXDCAnxtbx4cnw5LHuG5IRYnjv9Nn+XqQZA4k1n5k8J4tNBRpwsrcwZyG7VefkGihxJO3Wfg18kDtfdeXJUC9AgNrElHS5N4NtSX/GO/S80b3eRCsUnAP4Znitl8JrDdffU2k+LG7x2fjssexrCzkKrV9jvNo5b9vG4+R0Ecn4zVK+cN1OHNOT41Uh+Wn+Kn7eEMGvrOXo18Gd0mwoEFHctpCckhOXIOp48zZEqbvzk9xnzHD5iskd33lsXwTVDDUa3rYh9mnUrrCKWJCfD7p9h5Tv6WjH95nIo6hHsPKNSkyuDXTIxa+vQvI8/kp8UvswWcyzuavk10UWRs4ORqUMb0n/KdgZO20H/+oH88GwlPHttvyeeGD007Cq/xp5AP953mMUKh3G8sv1DPp3TDV+/+79ogYEQfckLe1M80ZIVzh33EmbXAChe6M8zQ7G39RXkd88kya8WT98Zy8oyd+8j7ItHW25FTQay6/IlSYrIkYxqLB1c41H2ybjWP87RuQ/xsGcrRgQu4Jsu7+GwoJk+zWi9wWC4t9tCTgfWm0VclL5ewfaJ4FkWhv4B5VsTEAoxtwx5vhmqUtKdb/vX46VOVZi88TS/7brIgp3nebROaZ5uW5GqJT0K9GkJYUmyiyerZg2moU8XPm/wMc/UnUNPh708s/oJ/j5Yn89716a2vxdg4bEE9EqOP8fC2Y1QoS30+BE8/fE8F4ddw+Opu8XfcCXqeCnLSq6KEKOS2b2sjZ+7E38915LPlh9j1rYQfIZfQNmb1hbyi6RYvy3Y+USiaTD7dG9Wn+/OpFrvML35i7BuLXT7EtxL3nPO9PEkLimZqk/uZuzi/5hXvCn1y3mb46nedWoN/Pk83L5EXKNnCT7TiQO3Yri9piZOtUIst6ImC9kNnpckReRIRjWW8XccUAqci0dReuQGYs4WZ8b6EXwwpzPFN43R/zjvmwsPfw0la95zvuwG1hc6TYOjf8KKN+D2RX0qw07vg6M+VWF+3QyVK+7CRz1rMbZDJaZvPsuc7ef4fd9lOlYrwbPtKlLP3EFQiEKQ03jy3Jrv6fVWT0ptHMvS+HdZdrsjQ3/sQ7fGNXj1oSp4uThYXiwBSIiFrd/Dpq/AYAePfAsNhqEB/xy4wju/H8K1UgIR2ypBSGlibjlbVnJVxBgMKoN1UmR2L0vn4mDH+z1q0qJcaQa/fQnHqhcx2CWjJSmS4+yI2lYZh0qXcCh5mwsxxem4YAVXfpuI238fw5kN0O5NaDRCnxnMJH08MTg3o+fErYyctYulz7SgXPH7F0EscLcvw7//g8NLoXglbgf/Rf/lyZy6HsXkwQ24XsuPEaPLWGZFTTaya0mRgfMix1L7fKb5IgD3PRYcrPQuDvvn6c2SMeFQfwi0fQPcS2R9EXO4sh9Wvg1nN4BfDXjkayjXNMNd83tqwvDoeGZuDWHm1hDCoxNoXrE4z7YLonnF4vneL92WBroK65ereBIXCes/Rdv+EzEGV76O68Ef9l15umN1BjYJsJxB5pqm30iseR9uhUCNnvDQx+BRmpAbd3j3j8NsOBFKzTIefNmnDsWNHhY3LWhO2VI8eeibjQT6uDB58N2nU++DlTxcuxTjH6tV2EUUeZCreHLztL7w4+m14FcdOn0AQR3JrK/l6dAoHp+4leJuDix9ugWeLoU06178Hdg6QV/7JTkRWr9CbONnGTRzPwcuRTB1SEPaVNYDh1VOmwz0mbSVRU+3yDSWSJIiciWjL0KWX47oMFj/KeyaDkZHaDoamj4LrhbQvzP0OGz8Ag4uAmcvfTG1RiP0fuOFLCoukXk7zjN10xmuR8ZRp6wXz7atSMdqJTAY8idZsaWbCmEbch1Prh2Gf9+EM+u4bizJF7Hd2eXRmbGdq/Fo7dLYGc2UrGganFwFGz6FS7v1yo6HxkPF9ty6E8+P604xe9s5HOwMvNSpMkOaBZivrPnEluJJ1+82UcbLmWlD7z6dam+vYFDTcrz5cPXCLqLIo1zFE03TF6Re/R7cOgvlW0ObcRDYIsNz7zhzk4HTdtCxWgl+GlS/YCe3SIiBPbNh09cQdRWqdYdO75PsVZ7nF+zjz/2X+XFAfR6uXargylBIBk/fwZwRTWV2L5E/fH3vv3HI6LFULsWg2+fQ5ClY+6H+pds+CRoMg8YjoVj5TA4sIJoGF3boY06O/AH2ztDieWj5op6omImbox0jW1dgSPMAFu2+yOQNZxj1y24ql3DjmbZBPFK7lNXf1AiRXq7jSYkaMGQZnFqD75r3+eLKFK7FLuOnRV2YuqoLQ9vVokfdMoXXTScxXr/R2faD3iLrWQ56TIQ6/QmPTWLW6pNM23yGO3GJ9Krvz6sPVcFPZvazOEYD93T30jRNFnO0QrmKJ0pBjcegSjfY9bPeNXNmNyjXHJqPgcpdwHD3/W9SoTivdanCx/8cY872cwxuFpj/T+DOTdgzC7b/BHeuQ0AL6PcLlG0MwLerTvDn/su83qWqTSQoILN7iQKS66bF4hWhz0xoc0wPBjsm6YlCpc5QbyBUegjsC/CP952bcHiJXjtx9QA4ekKrlyynVcfE0c7IwCYB9GtYlr8PXmHiutO8sGAfX606zqjWFenTwF/+cAqbkqduCkEdUBXbw8lV+G38gvcuziY6+jeW/N6c0cvbU6tJR/o1Loe/dwH1Hw89AQfmw945EHUNilWE7hOgTn/O3opnzj/Hmf/fee7EJ9GxWglefagKVUq6F0xZxANLP3A+PikZTUNirRXKdTyxc9B7eDQYCrtn6WPJ5g/QKxzqD4HafcE7AIARLSuw9fRNPvz7KA0CilG9dD5MeJOcDCGb9O7xh5ZAUhxUaAetZ97TqrPm6DW+X3OSPqaZQW2Fi0PWaYh09xK5li8rPN++DLtm6LUGUdfAyVOvuaj8EFRsD875MID81jk4uRJO/Atn1ul9Ov1qQOMRUKsvOLplebgl9PFMTtZYc+w6P647xb4L4fi6OzKiZXkGNg3I9Rz+ttQ9Q9iGfFst/tJutP+mkHxoKcakOEK0EqxOasC1km2p2qgjHWqVxcvlARZSTU7SW0pOroTj/+j/Vwa9H3vjUdz2b83KI6Es23uJzaduYGdQPFy71D0z91lCPMlPthRPek7cgquDHXNGNAEgIiaBOu+v5K2HqzGile3cENq6fIknSYlw/G/4b6qePACUbZJ6f3LTpSJdv9+Ml4s9f4xpmbdENj4azm3R702O/wO3L4GDO9TuA41HgV+1e3a/EBbNw99vomwxFxY/3ZzIcKPNxJI3lhzk0161ZUyKyB/5vsJzUqI+YP3gb/oXNiZMf9y3mt7EWaIG+FbRpwN2LwUO6WpGNQ1iIyDyqt6vNPQYXD0I53fos3QBeJeHqg9Dnf5QMmeDIPPt5imfaJrGtjP6KvZbTt3E09meoc0DGd4856vY29JNhbB+BbJafOxtOPonMXsXYn9hC3ZaAnGaPQe0Clxzr4mzf03KVqpDYIXKOHiWvH/8WXIS3LkBkVfg5ik9nlzaAxd3QtxtQIF/Q5KrP8aZEl3YeMXIhhOhbDtzk/jEZPy9nenXsCz9GpfFz/1uy7ClxZP8YEvxpM+krdgZDMwbpU+Ycu12LE0+XsP4x2oyqGmAOYopcqlA4smtc3Bwod41/OoB/THnYtwoVpdfQrwJqFafxzu0Bo/S4FzsvuUWSIzTK2EjLsGN43D9qB5LruzXK03tXfRWk1q9oHLX++9vgLjEJHr/tI2Qm3f4+7lWbFnpbFOx5MO/jvDOozVkTIrIH/m+wrPRDoI66D/JSfoX+OwmuLCd5MO/Y9gzK93+jnq3MIM9JMXrA8ySE+7dx8MfyjaCss/p5y0elOmsHRlJu9Cc0S0OzhbnyVF1MlxdvrAopWhe0YfmFX3YdyGcietO8f2ak0zbdIYBjcsxsrWsYi+sS4GsFu/kAfUG4lxvIMRFoZ1Zz+3D6ygTso06UX/gcHwx3F2ehHjlRLLREWUwYJccjyEpBqUlp27XlJFYryBulOlKiGtttmp12H/LnoP/RhAZp5+ogq8rg5oE8Mj/27v72KrqO47jn9859/b24bYU7C2lDCjysEFFeZCKwEwUlTicGo1TmBADTJjb3OJiNieZurk4N93+MNsCwzmfVt3cTMwydVvc1IFgDeA2URBGERChICCl97b34eyP20tbKC1tb3vO7Xm/EhJCe29/IT3f/D73+3u4YISmjSo9bUNtpp6EF7wlK5iUHbe1fGWNq/UEHVmm42WOsXj6vg2We+WOfqknQ8dIl9yV/vPpR+l7Sj7coLI9G/TN4D9k7fijtKP1e42d3uMaCElOKn0MeSLa8f0CBVLlVGn2N6Qxc6Wqud0uc3/opW36z75jWr14hgpShVq+MumpuUlfcZkjsqpfb3i27PTRv6NnpT95/HFSI4buU1Vwp8aP3K3hgcO6+YvHNGlCczqgBPLTBaEokr6UqXR0uuuSP6RPw8gUu/jhsA7UzlJgSFTxZker1zhadY/7n1hMHVWqNUsu1PYDx/Wrf+7U4+vr9eSbu3XDjJFacck4VZVxiz28r99viw+FZSZdrcikq1Vb62jp8rjGlG3TlMmbNKpsr84fv0+2jsuJRWUppZjy1KSQGpxSHXRKVe9UaJczQi3RoLQ//ZbhUJPGRYp0zdRKTRs9VDVVw7q9N6G+XrJCcR16YYYCQ6JKHCtQYVFc9fW5O7EYbGzLqCXRFk5j8fTfu5tAwTv6vZ6UVErTF0vTF6u21tFX7z2mmkXPa2xwv4o2DdOyGw6remJUSsRaA0t+eglXcUX6tWUT0vtcTu22dOHvWw/oN+t26dbZVZpfXaG6Om/PTXqDyxyRVQNxw3PbJ4912vXCDDUt3K0d5eVqOThOD3+/j+3bs1BVJTUdzlfslfNUsWjDyYL34ENztOI270wsJg4v1s9vmqo7r5io1a/v1O/f3qvn6vbo6vMrdful3GIPbxuo2+IbGqRlt6XUnJQ+nXdEdeVlWndwrFY/Mkf1O22FiuP66GhUh4636JOmFo2IJ9WSTGmeZSkUtBQOBVRenK/hJSFFikM9Pno0HJZOHAmqYnHbMpSPn5qjcNdb4jCAbOtMnRROVMwVA11PYvFCvdsyTtvDI5ScGtTqVStVvzN784P9x6K66/l3VF1Zoru/8DlJuTM36Qk6Kci6/r7hOdPJsIJJBYZE+9y+7emG1UhE+t53LN3/s1Nax0P72DruJ6OGFeqB66bojssm6LF1u/T0m7v14jsf6fJJ5br90vGazi328KiBuC2+vl7KC7co6bR0eJ6DJenneebMoEoqglJF9+/Vm83vjY1ScXnHWhKOxNTYSMfTKyxjlEqx3CvXDXQ9CQ5rkiTZBXEVTt6j+vqqrMxN4smU7qjdrOZESo8unKZQIP17mGtzk7PR3RHEfEyAXolEpJkz+6cIZNq2qbitxLECtRxMH93Zm/Ztba2jqvFJXXXjCVWNT6q29uwOilixwigvkd+nnz3QykvydfdVk7T+u/N05xUT9fbuI7r+l+u1cM0G/euDQ24PD+hUf9YSKV1PWhrzFD/asZbEPx2YWlJVJSUaO9aS5Alv1xK/Oa2T0rr0i05K7nGjnqTilgou2q5wWeys36erevLwK9tUV39ED14/RedGOrZcc3Fu0hWWeyHntLVta1RYFE8vjYjElDzRs/Zt+w3weeXHlXewWMtXzjmrTWaRiLR2df+3jvvDkMKg7pg3QcvmjlXtW+lb7G95bKPbwwJcEYlIj62xdOsy6eOnZ8suapbdkq/H1w5gLRmAZSjoPcsYJdu2pCjaku6kZD7BBjI6qyd5RQlVLnlT976yWc985SIFu7l4uat6sqXhgFa//j/dMmu0rp06stOfn6tzk87MmzS8y68TUuBJbW1bW+Gw1NjY8/ZtX0/7GIjWcX8qCgW0/PPnavHFY/SnTfu06CG3RwS4I/Msb94sSQFNm0YtQRvbUoflXs2JdEjp7lNe+FNn9WTd3in61nNb9JOX39c9CyZ3+foz1ZPX3knqvje26LyRJVrVxXsMpnrS3X1vhBR4ViTSt4cvG6d99HUMXhAK2FpYM1qL3B4I4KJIRLryyt69lloyuAUsS4lU+9O92JOCrp1aT66LjNSmD4/o12/s0pTPlOqaCyrP+NrO6klzi/TTjXUKhwJau2Rmt797fqknhBQMWiyzAJAN1JLBzbKM2jVSTh5BnB9gTwrO3qoFk/X+/uO687ktKgzaunxy50uZTq0nsail87/2to42x/WHlRerYgh3nmXwBGJQW7jQqH6HrZeeL1L9Djunb2YF4B5qyeBlGynJ6V7oo7yApbW3XqjqyhLd/swmvba94Yzfm6knTz5hNOPbG3Uo3qhffHm6qiv7ds/bYENIwaDX36d9APAHasngZFnmlJCSOd2LkIKeKckP6omlNRpfHtbS39bpkb9u63BRaHvvHWnQ/evX65Nos55aWqNLP1s+wKP1PpZ7AQAA37KNUardEcTReFJB28i26Jah50oL8/Tsilm6/8WtevTVHfrb1gO68cJRumRCmSRpZ8MJPbNxt9744JDGnFOoJ5bWcPnyGRBSAACAb9mndVKSdFHQJyX5QT3ypQt0ZfVw/eTl9/XDP2/t8PUhBUGtWjBJiy8ew1HXXSCkAAAA30pvnO94BDEhBdkwv7pC86srtOeTJq3feUihgK2xZUWaMDyswjym4N3hfwgAAPiWbU7fk8Jt88imUcMKddOw0W4PI+fwFAIAAN+yLaPEqcu9WIIDuI6QAgAAfMu2TIcb56PsSQE8gZACAAB8y7aMkk7HTkoBIQVwHSEFAAD4lmWMUu2usojFUwqxJwVwHU8hAADwLdvSaZ0UlnsB7iOkAAAA3zr1dK/mRIqQAngAIQUAAPiW1XqzfGbzfPp0L6ZHgNt4CgEAgG/ZJh1SMku+ON0L8AZCCgAA8K1MJyXZrpNSkEdIAdxGSAEAAL5ltwspjuOkb5xnuRfgOp5CAADgWwGrbblXcyJ9FnGI5V6A6wJuDwAAAMAtlmnbON/cemEKe1IA9xFSAACAb7Vf7pU4GVJYaAK4jacQAAD4ltVuuVe0JSlJKqCTAriOkAIAAHzLPrncS4ol0iGF5V6A+wgpAADAt+zWmVCy9WQvieVegBfwFAIAAN9qv3E+Fm/tpATopABuI6QAAADfymycT7QLKRxBDLiPkAIAAHyr/elemZDCxnnAfYQUAADgW5mQkmJPCuApPIUAAMC3Mqd7te+kcLoX4D5CCgAA8C2rk+VehBTAfYQUAADgWyfvSXEcxRIs9wK8gqcQAAD4Vmcb5zmCGHAfIQUAAPiW1W7jfDSeVF7AOvlvANxDSAEAAL7VtnFeao6nlB9gagR4AU8iAADwLat1JpRIpRSLJ9k0D3gEIQUAAPhWoDWlpFIipAAeQkgBAAC+ZbfOhJKtlzlyshfgDTyJAADAt6zMEcQpR7FEUgV0UgBPIKQAAADfan8EcbQlqRAhBfAEQgoAAPCtTCcl2XqZI3tSAG8gpAAAAN/KdFI+PNyk/UejHEEMeETA7QEAAAC4JRNSfvSX91SUZ+vmmlEujwiAREgBAAA+dk5RnvKDluaOj+gH11arsrTA7SEBECEFAAD42DnhkP5733zZlpFp3Z8CwH2EFAAA4GsBm30ogNfwVAIAAADwFEIKAAAAAE8hpAAAAADwFEIKAAAAAE8hpAAAAADwFEIKAAAAAE8hpAAAAADwFEIKAAAAAE8hpAAAAADwFEIKAAAAAE8hpAAAAADwFEIKAAAAAE8hpAAAAADwFEIKAAAAAE8hpAAAAADwFEIKAAAAAE8hpAAAAADwFOM4Tvbf1JgGSbuz/sYA+mKM4zgRtwfRU9QTwJOoJwCy4Yy1pF9CCgAAAAD0Fsu9AAAAAHgKIQUAAACApxBSAAAAAHgKIQUAAACApxBScBpjzExjzL+NMfnGmCJjzLvGmPPcHheA3EM9AZAt1BN/4XQvdMoY84CkfEkFkvY6jvOgy0MCkKOoJwCyhXriH4QUdMoYkyepTlJM0mzHcZIuDwlAjqKeAMgW6ol/sNwLZzJMUlhSsdKfWABAb1FPAGQL9cQn6KSgU8aYFyU9K2mspBGO43zd5SEByFHUEwDZQj3xj4DbA4D3GGOWSEo4jvM7Y4wtab0x5jLHcV51e2wAcgv1BEC2UE/8hU4KAAAAAE9hTwoAAAAATyGkAAAAAPAUQgoAAAAATyGkAAAAAPAUQgoAAAAATyGkAAAAAPAUQgoAAAAATyGkAAAAAPCU/wO6Y8gz+WAVDwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_64_1.png" } }, "output_type": "display_data" } ], "source": [ "\"\"\"\n", "============================\n", "Underfitting vs. Overfitting\n", "============================\n", "\n", "This example demonstrates the problems of underfitting and overfitting and\n", "how we can use linear regression with polynomial features to approximate\n", "nonlinear functions. The plot shows the function that we want to approximate,\n", "which is a part of the cosine function. In addition, the samples from the\n", "real function and the approximations of different models are displayed. The\n", "models have polynomial features of different degrees. We can see that a\n", "linear function (polynomial with degree 1) is not sufficient to fit the\n", "training samples. This is called **underfitting**. A polynomial of degree 4\n", "approximates the true function almost perfectly. However, for higher degrees\n", "the model will **overfit** the training data, i.e. it learns the noise of the\n", "training data.\n", "We evaluate quantitatively **overfitting** / **underfitting** by using\n", "cross-validation. We calculate the mean squared error (MSE) on the validation\n", "set, the higher, the less likely the model generalizes correctly from the\n", "training data.\n", "\"\"\"\n", "\n", "print(__doc__)\n", "\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from sklearn.pipeline import Pipeline\n", "from sklearn.preprocessing import PolynomialFeatures\n", "from sklearn.linear_model import LinearRegression\n", "from sklearn.model_selection import cross_val_score\n", "\n", "\n", "def true_fun(X):\n", " return np.cos(1.5 * np.pi * X)\n", "\n", "np.random.seed(0)\n", "\n", "n_samples = 30\n", "degrees = [1, 4, 15]\n", "\n", "X = np.sort(np.random.rand(n_samples))\n", "y = true_fun(X) + np.random.randn(n_samples) * 0.1\n", "\n", "plt.figure(figsize=(14, 5))\n", "for i in range(len(degrees)):\n", " ax = plt.subplot(1, len(degrees), i + 1)\n", " plt.setp(ax, xticks=(), yticks=())\n", "\n", " polynomial_features = PolynomialFeatures(degree=degrees[i],\n", " include_bias=False)\n", " linear_regression = LinearRegression()\n", " pipeline = Pipeline([(\"polynomial_features\", polynomial_features),\n", " (\"linear_regression\", linear_regression)])\n", " pipeline.fit(X[:, np.newaxis], y)\n", "\n", " # Evaluate the models using crossvalidation\n", " scores = cross_val_score(pipeline, X[:, np.newaxis], y,\n", " scoring=\"neg_mean_squared_error\", cv=10)\n", "\n", " X_test = np.linspace(0, 1, 100)\n", " plt.plot(X_test, pipeline.predict(X_test[:, np.newaxis]), label=\"Model\")\n", " plt.plot(X_test, true_fun(X_test), label=\"True function\")\n", " plt.scatter(X, y, edgecolor='b', s=20, label=\"Samples\")\n", " plt.xlabel(\"x\")\n", " plt.ylabel(\"y\")\n", " plt.xlim((0, 1))\n", " plt.ylim((-2, 2))\n", " plt.legend(loc=\"best\")\n", " plt.title(\"Degree {}\\nMSE = {:.2e}(+/- {:.2e})\".format(\n", " degrees[i], -scores.mean(), scores.std()))\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 1\n", "Mean squared error on training data: 439230.69504801\n", "Mean squared error on test data: 481979.17861098\n", "Degree of polynomial: 2\n", "Mean squared error on training data: 115822.95008046\n", "Mean squared error on test data: 123711.53703498\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 3\n", "Mean squared error on training data: 9011.85263220\n", "Mean squared error on test data: 10913.84780262\n", "Degree of polynomial: 4\n", "Mean squared error on training data: 303.47610036\n", "Mean squared error on test data: 426.30787294\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 5\n", "Mean squared error on training data: 3.80354994\n", "Mean squared error on test data: 5.98822371\n", "Degree of polynomial: 6\n", "Mean squared error on training data: 3.66204648\n", "Mean squared error on test data: 8.14812206\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 7\n", "Mean squared error on training data: 0.47075725\n", "Mean squared error on test data: 2.00607783\n", "Degree of polynomial: 8\n", "Mean squared error on training data: 0.04912436\n", "Mean squared error on test data: 0.21596432\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 9\n", "Mean squared error on training data: 0.02522069\n", "Mean squared error on test data: 0.08576932\n", "Degree of polynomial: 10\n", "Mean squared error on training data: 0.02511518\n", "Mean squared error on test data: 1.20015436\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 11\n", "Mean squared error on training data: 0.01640891\n", "Mean squared error on test data: 1.35533774\n", "Degree of polynomial: 12\n", "Mean squared error on training data: 0.00813803\n", "Mean squared error on test data: 0.17446471\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 13\n", "Mean squared error on training data: 0.00759119\n", "Mean squared error on test data: 1.08131003\n", "Degree of polynomial: 14\n", "Mean squared error on training data: 0.00472199\n", "Mean squared error on test data: 0.81333805\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 15\n", "Mean squared error on training data: 0.00410478\n", "Mean squared error on test data: 92.09149881\n", "Degree of polynomial: 16\n", "Mean squared error on training data: 0.00315593\n", "Mean squared error on test data: 234.39095416\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 17\n", "Mean squared error on training data: 0.00242999\n", "Mean squared error on test data: 1270.94548496\n", "Degree of polynomial: 18\n", "Mean squared error on training data: 0.00228741\n", "Mean squared error on test data: 108.28590743\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 19\n", "Mean squared error on training data: 0.00156379\n", "Mean squared error on test data: 1378.43761347\n", "Degree of polynomial: 20\n", "Mean squared error on training data: 0.00137835\n", "Mean squared error on test data: 1954.37992857\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 21\n", "Mean squared error on training data: 0.00118527\n", "Mean squared error on test data: 14818.20320502\n", "Degree of polynomial: 22\n", "Mean squared error on training data: 0.00092646\n", "Mean squared error on test data: 871.17339342\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 23\n", "Mean squared error on training data: 0.00085884\n", "Mean squared error on test data: 5566.16660817\n", "Degree of polynomial: 24\n", "Mean squared error on training data: 0.00084705\n", "Mean squared error on test data: 1314.42631342\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 25\n", "Mean squared error on training data: 0.00079123\n", "Mean squared error on test data: 127043.53189647\n", "Degree of polynomial: 26\n", "Mean squared error on training data: 0.00076925\n", "Mean squared error on test data: 18526.05756733\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 27\n", "Mean squared error on training data: 0.00069103\n", "Mean squared error on test data: 2470.53697476\n", "Degree of polynomial: 28\n", "Mean squared error on training data: 0.00062595\n", "Mean squared error on test data: 4022.12945452\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Degree of polynomial: 29\n", "Mean squared error on training data: 0.00060705\n", "Mean squared error on test data: 3384.63675140\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/626635268.py:73: RuntimeWarning: divide by zero encountered in log10\n", " plt.plot(polynomial, np.log10(trainingerror), label='Training Error')\n", "/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/626635268.py:74: RuntimeWarning: divide by zero encountered in log10\n", " plt.plot(polynomial, np.log10(testerror), label='Test Error')\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEHCAYAAACjh0HiAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA/NklEQVR4nO3dd3hUZfrw8e8zM+kNUoBAEkroAaSEFgQpYgF7Z9UFe6+77vpb3V3LruvaXhcVXSzYyypWBBSQgHRCB2kCSajSUyB9nvePZ4AQ0jOTk5m5P9c118ycOXPOfRgy9zxdaa0RQgjhv2xWByCEEMJakgiEEMLPSSIQQgg/J4lACCH8nCQCIYTwc5IIhBDCzzmsPLlSqhnwFtAD0MDNWuvFVe0fGxur27Vr1zjBCSGEj1ixYsVBrXVcVa9bmgiA/wAztdZXKaUCgdDqdm7Xrh0ZGRmNE5kQQvgIpVRWda9blgiUUpHAMGACgNa6GCi2Kh4hhPBXVrYRdAAOAFOUUquUUm8ppcIq7qSUul0plaGUyjhw4EDjRymEED7OykTgAPoCr2ut+wDHgEcr7qS1nqy1TtVap8bFVVnFJYQQop6sTAS7gF1a66Wu519gEoMQQohGZFki0FrvA3Yqpbq4No0CfrEqHiGE8FdW9xq6D/jI1WNoO3CTxfEIIYTfsTQRaK1XA6lWxiCEEP7OP0YW78qABS9bHYUQQjRJfpEIds6bArP/jl7zmdWhCCFEk+MXieCLmLtZXNadsm/uNaUDIYQQJ/lFInjgvO581ekZdpc1o+jD6yBnt9UhCSFEk+EXicBmUzx53TCeb/4EJQX5FHxwDRQfszosIYRoEvwiEQCEBNr52y1X8kTAQwQd3EDh53eA02l1WEIIYTm/SQQALSKCufnmu3jReT3BW7+jeO6zVockhBCW86tEANC9dSR9r/srX5QNI/Dnf+Nc/5XVIQkhhKX8LhEAjOreitxznyfD2ZmyL++APautDkkIISzjl4kA4KZhXfixxwv8VhbB8fevgbx9VockhBCW8NtEoJTikSuH8lrLp6DgKPnvXQMlBVaHJYQQjc5vEwFAgN3GoxOu4dmQhwg/uIb8z+8Gra0OSwghGpVfJwKAqNAAbrntfl5V1xG+5UsK575odUhCCNGo/D4RALSNCaP/Df9kmnMwgfP/Qekv06wOSQjhSZtnwqqPrI6iyZBE4DIwOZbii15lg7MtBV/eDyWFVockhPCUOU/Bt/fB/o1WR9IkSCIo54oBHVma/AARpYfIXfqB1eEIITzh+GHYvwF0Gcz4s7QLIongDKPHXsM6ZztKF0wEZ5nV4Qgh3C1rkbnvcRXsmAcbv7M2niZAEkEFbWPDWdTqRqILsylaL/9BhPA5mQvAEQKXvAItUuCHx/y+67gkgkr0Pu/3ZDpbkjfneSk2CuFrshZAYn8IDIULn4WcbFg40eqoLCWJoBIDkuP4PuJKYnPW49zxs9XhCCHcpeAI7FsPbc82z9sPg+6XwYKX4Gi2paFZSRJBJZRSJIy4hQM6kiOznrc6HCGEu2QvATS0G3Jq23lPAwp+/KtVUVlOEkEVLuzdgS/sY4nZOx/2rbM6HCGEO2QuAHsQtEk9ta1ZEpz9EPzyNfhpDYAkgioEOmwEDLqdfB1M7hwZbSyET8haCAmpEBB8+vYh90NUkulOWlZqTWwWkkRQjSuH9OBzPYqwrd/CkSyrwxFCNERhLuxdA22HnPlaQAic/w8zvmDFlMaPzWKSCKrRPCyQ31JuwamhYL5/9yoQwuvtXAraeXr7QHndLjGNxz/9A44datzYLCaJoAZXjRjIN84hONZ84Hf/OYTwKZkLwBYACQMqf10puODfUJQHc//RuLFZTBJBDTq2CGdlwu8JcBZRuuQNq8MRQtRX1kJo08+MH6hKy+4w4DbImGKqkfyEJIJauHDkcGaV9aVsyX+h+JjV4Qgh6qooH3avrLpaqLzhj0JotF/NQySJoBbO7hjL9MhrCSrJQa983+pwhBB1tXOpmWSusobiikKaw6i/QfZiWD/V87E1AZIIakEpxaDhY1ju7EzxzxOhrMTqkIQQdZG1EJQdEgfWbv8+N0KrXmaQmR/UAkgiqKVLe7fhI8cVBB3bAxu+sjocIURdZC6E1n0gKLx2+9vsMOZ5yNsDP7/k2diaAEkEtRQcYCdp0OVscbahaN5LflN3KITXKz4Ou1fUrn2gvKRB0PMaWDQRDm/3TGxNhCSCOrhxcHvedl5M0KGN8Otsq8MRQtTGruXgLDk10VxdjH7SdDn18XmIJBHUQVxEEPS8mr06htL5vl9cFMInZC0EZTO/8OsqsrWZh2jTNDMOwUdJIqij8UM781bpBTh2LoJdGVaHI0TTcCQTXh8Ce1ZZHcmZMheaht/gyPq9f/A9ENnGLGDjdLo3tiZCEkEddW8dyY6kq8glDOeCl60OR4imYdbf4bf1sPg1qyM5XUmhqRpqV49qoRMCQ0130r2rYd3nbgutKbE8ESil7EqpVUqpaVbHUlvXD0vh/dJzUZumwcGtVocjhLV2LjNTOIfGwC/fmMXhm4rdGVBW1LBEAKbROL43zHnSND77GMsTAfAAsNHqIOpiRJcWzI28ghIc6Kb2C0iIxqS1qTIJbwXjPoWyYljzqdVRnZK5EFCQNLhhx7HZ4Px/Qu5uWOJ7f/OWJgKlVAIwFnjLyjjqymZTXDa0N9+XDaBs3ZdQWmx1SEJY45evYdcyGPkYJA4wC76seLfpdK/OWgCtekBIs4Yfq93Z0GUsLHgZ8n5r+PGaEKtLBC8DfwKqbIFRSt2ulMpQSmUcOHCg0QKryZX9EphjH4qjOAe2zbE6HCEaX2kRzH4CWqRA7+vNtn4T4OBm15KQFisthp3L69dttCqjn4LSQkh/xn3HbAIsSwRKqYuA/VrrFdXtp7WerLVO1VqnxsXFNVJ0NQsNdNCsx/kc1eGUrfXNBiQhqrXsTdNb6LynzUhcgB5XQGCEKRVYbc9KKC2o+0Cy6sR2hP63wsr34bdf3Hdci1lZIhgCXKKUygQ+BUYqpT60MJ46u+CsJKaXDUBvmu4X85EIcdLxwzD/OUgeBR1HndoeGAa9rjZVRgVHLAsPONXvPynNvcc9588QFAGzfGeQmWWJQGv9f1rrBK11O+A64Cet9Q1WxVMfgzpEMzdwGI6yAtg8w+pwhGg8818wC7icV8kCLv0mmOqTtf9r9LBOk7UQWnSHsBj3Hjc0GoY9YmYX8JEZBqxuI/BqDruNuJTh/KabU7b2C6vDEaJxHNoGyyZDnxvMQi4VxZ9lJnhzR6Px8cOw4eu6H6esBLKX1m7a6foYcDs0b2emnnCWeeYcjahJJAKtdbrW+iKr46iPMb0S+a5sEOrX2dYXhYVoDHOeBHsgjHis6n36TYD9v5jBXPWlNXx1B3w+HpbWcXXAvWug5Jh72wfKcwTBuU+aa1z1gWfO0YiaRCLwZoM6RJMeeA42XQIbv7M6HCE8K3upGTQ25AGIaFX1fj2uhMDwhjUab/oetv5opnf48XHIWlT792b+bO49VSIA6H4pJA6Cn/5pqsm8mCSCBnLYbSSmpJGtW0r1kPBtWsOPj0FEPKTdW/2+QREmGaz/Egpz6n6u4mMw81HTNfXOBdCsLXw+AfL21e79mQshtguEt6j7uWtLKTPI7Nh+WPifur23rAS2/WRGZTeBkdgOqwPwBWN6tebrVYO5L/NbM9AkoqXVIQnhfhu+MlU9l75megfVpN8EWPmeaTQecFvdzjX/ecjZCTfNNI2z134Ib40yyWD8d2APqPq9ZaVmHEOvq+t2zvpISDUJb9Gr5nqjEqrf/9ghWDEFlr9tFr05ITQGYjpBrOsW0wliO0PzttVfq5tIicANBneIYV7AMBRO021OCF9zYvBYyx5w1rjavad1HzPrZ10bjQ9sNl+sva+Htq6pIVp2h0teMesIz/pb9e/ftxaK8zxbLVTeqL+DdsKcp6uJaT18cy/8v+7w09MQ18Ukt3GfmZ5XXS8yU2VvmWmu79Nx8Go/+GcreCUVPhkHu1d67BKkROAGDruNTj36s3ltEp3Wfo5t4B1WhySEey2bDEez4MavTg0eq4lS5lfy9w+bL7GEfjW/R2v4/g+mxDH6qdNf63mVKZEsmQRt+pnnlclaaO4bOtFcbTVvC4PugoUvw6A7TQIE05to8wzT0J35MzhCTBIdeCe06FruABecfryCo3DoVzOh5cEtcGgrHPzVJBsPkUTgJmN6xvP1yjT+vPtTOJJl/nMI4QuOHzZVNR1HQ/LIur2359WmoXfFlNolgvVTzZfm2JcgLPbM10c/DXtWw7f3QcsUaNHtzH0yF0J0cvWN2e429GHTe+iHx+G6j2DVh6eSZ1SiSWp9bjTVXDUJaWaqnBJSPR72CVI15CaDk2OYFzjUPFk/1dpghHCnec+5Bo9VU/VRleBIM+3E+i+hMLf6fQtz4Ie/QOu+piRRGUcgXP2u6ZH02Q1nNkQ7yyB7kee6jVYlOAqG/5+Z5O6FzqZRPbINXPM+3L/a9LKqTRKwiCQCNwmw2+iZ0ovVujPOddJ7SPiIQ9tg+ZvQ9/eV//qujX43mT7962v4u5j7DOTvh7EvVl/9FBlvksHhHfD13ae3P/y2wSQHd040V1v9bjKzk/a4Eu6YDzfPMF1M7U2/4kUSgRuN6RXPV6WDse3fAPs3WR2OEA1TcNT8QrcHwfC/1P84bfqZRubqxhTsXWuqUvrfAm361nzMdkNMCWXTNFM3f8LJ9oFGLhGA+cIf9zFc/roZXe1FJBG4UVpyDD8Hno0TW82/foRoio7uhKX/hfcugeeTTS+Wcx5pWJfoE43Ge9dUvqax02kalEOiYeTjtT/uoLsh5XKY8xRsTzfbMl1jDmrqxilOI4nAjQLsNlJTurBUp5jqoaayOIcQVdHa/BpPfxbeGAov94AZf4K8vTD4XrhlNgx5sOHn6Xm16TWz4r0zX1v9oekNdN4/IKR57Y+pFFzyqulz/8XNJollLWy83kI+pOlXXnmZMT3j+XLVYAYfmWzmQ29Ti54SQjSmshIzXcPm6bBpOuRkA8qsMDb6KVPPHdvRvecMaWZ+va/73HzhB4Wb7ccPm4Xvk9LgrOvqftygcNMf/82R8N5FZr6vxho/4EOkROBmQzrGsjggjVIVAOuk95BogNIi909kWFYKb46A9y8xdfYtU8xArT9ugVt+NL1b3J0ETug3AYrzT+9VN/sJ07g79kXzC78+4jrDZZPMIjkgJYJ6kETgZgF2G2k9OjDPeRZ6/Zc+MUWtsEBZKUwZA/89x/yCd5dN02DfOvOr/E/b4Xefmh5BnpyT54TEARDX7VSj8c7lZgqKwXdXPp11XXS/xDRotz9HxvDUgyQCDxjTM56vSgaj8vfWbcZEIU5YNBF2Z5gBSe4cl7L4NTOP/qC7azdfkDudaDTes9I0Gn//MES0hnMedc/xh/8Zxn/rnmP5GUkEHjCkYyzLAwdSpEKk95Cou/2bIP1f0O1i8wt64UT3dDzYlQG7lsHAu2o/TYS79brGdEf97EYzJ9AF/zrVXiAsI4nAAwLsNoalJDHb2Q/9yzdQWmx1SMJblJXC13eZkbNjX4K0+2D/Btg2p+HHXvwaBEVBn+sbfqz6Co2GlMvMzKLJI82AK2E5SQQeMqZXPFNLBqEKjsD2uVaHI7zFoomm6mTsC6bevufVZv7/hRMbdtyjO82CMv1+b9YKsNLge8wAszEv1L+BWLiVJAIPGZIcy+rAvhy3RYBMOSFqo3yVUMoVZpsj0MxWuWOemWytvpb919wPaAIz48afBXcthJhkqyMRLpIIPCTQYWNkSgLTywagN30PxcetDkk0ZRWrhMr/Uk69CQIjTGmhPoryzECu7pdCs0T3xCt8iiQCDxrb01U9VHLMDNUXoiqLXzm9Sqi84CjoNx42fG2mOK+rVR9BUa4ZKSxEJSQReNCQjrFsDOxJjiNGpqYWVdu/ycy8Wb5KqKJBd5tSwpJJdTu2swyWvg6JA2u3HoDwS5IIPCjQYWNU99Z8WzoQvfVHM5ujEOVVVyVUXlQb03C88v26LXa+eboZcTvobreEK3yTJAIPG9urFVOLBqHKis2oTiHKq65KqKK0+6DkOGS8XYfjT4JmSWZNXCGqIInAw87uGMe2oC4ccbSArT9aHY5oSmpTJVReyxToeC4snQwlhTXvv3ulWa1r4J1esTiKsI4kAg8LdNg4r3s86aXd0Tvmy9xDwigrhW/urrlKqKK0++HYflj7ac37Lplkehv1ubFhsQqfJ4mgEYzt1Yr04u5mcNm+tVaHI5qCxa/A7hW1qxIqr/0w0w9/0atmQZeq5OyGDV+ZCeWCIxser/Bpkggawdkd41gd0Ns82T7P0lhEE1DXKqHylDKlgkNbYcuMqvdbNhm0EwY2gQFkosmTRNAIAh02OrbvwHZb21NL6gn/VN8qofK6X2YagBf+p/LXi/JhxRSTaGRKZlELkggayeDkGNKLu+HMXlK7hj7he5xlMPPR+lUJlWd3mMFhO5dC9tIzX1/ziVnsZdA9DYtX+I1qE4FSqm8tbj0bK1hvlpYcywJnD2ylBWYqYOFfCnPg42th+ZvmC7quVUIV9bnBrO9bcdoJp9M0ErdJNQvBCFELNfUpmwcsB6orv7YH2rkrIF/VtVUEW4J6Uqbt2Lenm0Y/4R8ObYNPxsHhbXDRy2buoIYKDIP+t8L8F+Dgr6eWl9wyEw5vh6sel5k9Ra3VlAiWa61HVreDUuonN8bjs2w2Rc/kRDZs60jP7fNQo6yOSDSKHfPhf783j2/8GtoPdd+xB9xupqde/Apc7GovWDIJohKhm8zzL2qv2qqhmpJAbfcRRlpyDHNLUsxIUpluwvctfxs+uBzCW8JtP7k3CYBpY+g9DlZ/Avn7Ye8ayPzZJAgZQCbqoKY2ghvKPR5S4TWZyrCOBifHsrAsBaWdkLnA6nCEp5SVwvRHzJq8ySPhllkQ3cEz5xp8H5QVm+6iiyeZ3kh9f++ZcwmfVVOvoYfLPX6lwms3N+TESqlEpdRcpdRGpdQGpdQDDTmeN0iOC2NXWApFKtgsNCJ8T8ER+OhK88Wcdh+M+9SzA7piO0LXseZ866e6GpGbee58wifVlAhUFY8re15XpcAftNbdgEHAPUqp7g08ZpOmlGJAx1Zk0A0t4wl8z4Et8OYoyFwIl06C8/7ROIvEp91veiU5S2UAmaiXmhKBruJxZc/rRGu9V2u90vU4D9gItGnIMb1BWnIsc4u7ow5ugdw9Vocj3OXXOfDWueYLecK0xl0gPmkgdBwNZ43zXBWU8Gk1tSh1VUqtxfz6T3Y9xvXcbf/jlFLtgD7AGaNjlFK3A7cDJCUlueuUlhmcHMMUZw/zZPs809gnvNuOn+Gjq6BFdxj3iRn129hukHWxRf3VlAi6eToApVQ4MBV4UGudW/F1rfVkYDJAampqg0ohTUFidCjHmnUmryiKiO3pkgh8wS9fgyMEbp4JQRFWRyNEndXUfTSr/A3IB/oCsa7nDaKUCsAkgY+01l829HjeYlByHAudKegd80B7fW4TWYvNKF5JAsJL1dR9dJpSqofrcTywHtNb6AOl1IMNObFSSgFvAxu11i815FjeJi05lrklKai8vXBwi9XhiIYoOAL7f4G2aVZHIkS91dRY3F5rvd71+CZgltb6YmAgDew+CgwBbgRGKqVWu25jGnhMrzA4OYaFzhTzRHoPebfspYCGpMFWRyJEvdXURlBS7vEo4E0wvXyUUtWsilEzrfUCGt4F1Su1jAwmKLY9+wviabF9nnT582bZi8AWAAmpVkciRL3VVCLYqZS6Tyl1OaZtYCaAUioECPB0cL4sLTmW9JLu6Mz5ZiSq8E5Zi6F1HwgIsToSIeqtpkRwC5ACTACu1VofdW0fBEzxXFi+Ly05hnklKaiiPNizyupwRH2UFJjPrq1UCwnvVm3VkNZ6P3BnJdvnAnM9FZQ/GNQhhr84XQOpd6RDYn9L4xH1sCsDnCWQJA3FwrtVmwiUUt9W97rW+hL3huM/mocFEh+fQGZ+Mu22z4Nhj1gdkqir7MWAMiN7hfBiNTUWDwZ2Ap9gRv36ZeOupwxOjmH2sm7csvNHVPFxCAy1OiRRF1mLzGjikOZWRyJEg9TURtAK+AvQA/gPMBo4qLWep7WW6TMbKC05hvmlKaiyYtevS+E1ykph13JpHxA+oaaRxWVa65la6/GYBuJfgXSl1H2NEp2PG9A+mpV0pUw5ZDyBt9m3ForzZfyA8Ak1LmOklAoCxgLjMGsTTwT8ZjoIT4oIDqBjm5ZsPNqNHrI+gXc5UYKTEcXCB9Q0xcR7wCLMGIIntdb9tdZPa613N0p0fiAtOYZZBV3Re9fCsUNWhyNqK2sRNGsLka2tjkSIBqupjeBGoDPwALBIKZXruuUppc6YKVTUXVpyLD+XpaDQkDnf6nBEbWgN2UukNCB8Rk1tBDatdYTrFlnuFqG19uD6e/6jX9vmbLR1osgWatYnEE3fwa1w/KC0DwifUVOJQHhYSKCdnkkxrLb3kAZjb5G9yNxLiUD4iJraCFbWdIDa7COql5Ycw8yCrnBkBxxp8DIPwtOyFkNYHMR0tDoSIdyixhXKyi1PWRkFRLkxHr+UlhzLY3N6mE9jxzxo/nurQxLVyV4ESYNAyfhK4RtqXLO4Fscoc0cg/qx3YjN2OhLJc8SY5Sv7SiJosnJ2w9FsGHiX1ZEI4TY1TTqXBaCUagm0ATSwR2v9WyPE5jcCHTb6t4th6b6enLt9HjidYJPmmybp5PgBaSgWvqOmNoLeSqklQDrwHPA8ME8ptUQp1bcR4vMbacmxzDzexfRG2f+L1eGIqmQtgsBwaNnT6kiEcJuafna+Czygte6mtT7XdesKPIisR+BWackxLCzrYZ7UpvdQSSFsmwsLJ0JRnkdjE+Vkuxaqt9c4KF8Ir1HT/+YwrfXSihu11kuUUmEeiskvpbSOJD+4JfuDkmixYx6k3Xv6DlqbksK2n8wtaxGUFprXshfDtR9JdZKnHT9sPoOUK6yORAi3qikRzFBKfQ+8j5mOGiAR+D2uZSuFezjsNga2j2bhzhQuz5wPpcVQeNSUDrb9ZH795+8zO8d1hdSbIXkk7N8Is/4K85+D4Y9aeQm+b6frN5G0DwgfU1Nj8f1KqQuBSzGNxQrYBbymtZ7eCPH5lcHJsczc3JXLA2fApEFweJt5ISQakkeYL/4OIyCqzak3dTzXJIP0f0GrntB1rDXB+4Ms10L1bfpZHYkQblVjRafWegYwoxFi8XtpyTG87EwhN7wDkRHx0Pt35ss/vnfV1T5KwUX/Dw5sgi9vh1vnQIva9PoVdZa9GNr0lYXqhc+pd6WyUmqyOwMR0KVlBAFhzXgiaQrc9D0M+6P54qmp7j8gGK790HxBffo7KDjaKPH6leLjZqF6mV9I+KCauo9GV3GLAcY0Uox+w2ZTDO4Qw+Jth9Ba1+3NUW3gmg/gaBZ8eRs4ZZyfW+3OAGepzC8kfFJNJYIDQAawotwtw3Vr4dnQ/NPg5Bj25hSydMdh9uYUcORYMceLS3E6a5EY2g6GC5+DrT/C3Gc8H6w/yXItVJ8oC9UL31NTG8F2YJTWOrviC0qpnZXsLxpoSMdYAK6bvOSM1wLtNoIcNoIC7AQHmMdxEUE8d+VZJMW4Fr5PvRn2roGfXzCNxymXNWL0Pix7EbRMgZBmVkcihNvVlAheBpoDZyQCzEhj4WbtY8P45LZB7MstoLDESWFJGUWlp98XljgpKi2jqMTJ3M37eWb6Rt640dWTRSkY87zpSfT13RDbyXyBiforK4Wdy03jvRA+qKbuo69V89or7g9HgKkeqq2Jc7by0qwtZGQeJrVdtNnoCIJr3ofJw03j8W1zITTaM8H6g31roOSYjB8QPqtWvYaUUldUchullJJ2AovdOrQ9LSKCeGb6xtMbmCPjTU+i3D0w9RZpPG6ILNdEc0nSUCx8U227j94CvAVc77q9CTwMLFRK3eih2EQthAY6eHh0Z1ZmH2Xm+n2nv5jYH8a8YEYmz3nSmgB9QfZiaN7OJFchfFBtE4ET6Ka1vlJrfSXQHSgCBgJ/9lRwonau6pdA55bh/HvmJopLnae/2G889L8VFv4H1n1hTYDeTGuTCKQ0IHxYbRNBuwprEOwHOmutDwMl7g9L1IXDbuPRC7uSeeg4Hy+tZKnL8/9lvsi+uVeWwqyrg1vg+CFpHxA+rbaJ4Gel1DSl1Hil1HjgW2C+awbSox6LTtTaiC4tGNwhhok//UpuYYXc7AiEK98EXWa6lYray3ItVC8lAuHDapsI7sGsP9Ab6AO8B9yjtT6mtR7hodhEHSil+MuYbhw+Vswb6dvO3CEqAfrdBKs+gsM7Gj9Ab5V9YqH6ZKsjEcJjapUItOmOsgD4CZgNzNd1ngNBeFrPhCgu7d2atxfsYM/RgjN3OPshsAfA/CZcKtDazPv/2wYoPmZ1NKbHUNJgWahe+LRaLbOklLoGs0xlOmYq6leUUo9orRvU+qiUugD4D2AH3tJaP9uQ4wn443ldmLFuHy/N2sILV591+ouR8Wbk8dL/wtCHG/9XrtMJ+b9B3h7TrTV3D+Tuhty9px7n7T214E7nC+B3nzVujOXl7IKcbBh8t3UxCNEIarve3mNAf631fgClVBymZFDvRKCUsgOvAaMxaxwsV0p9q7WWBXsbIDE6lPFpbXlrwQ5uHtKe7q0jT99hyIOQMQXmPw+Xv9G4wX15K6yfevo2eyBExENkazPTakQ8RLaBQ1sh4x3YMR/aD3NfDFrX/tf9yfED0lAsfFttE4HtRBJwOUQDprB2GQD8qrXeDqCU+hSzAI4kgga6d0Qn/pexi3/N2MgHt1SYJC2iJfS/BZZMgqF/hNiOjRPU/k0mCZz1O+h+ifnij2wDoTGVfzGXFMLWWfDj43BbunuW4dy/Cd4615y7/VBodza0GwphsZXvn70IAiPMnE1C+LDa/nXNVEr9oJSaoJSaAHwPNHSFsjacWv4STKmgTcWdlFK3K6UylFIZBw4caOAp/UNUaAD3jezIz1sPMn9LJf9mQx4ARzDM+3fjBbVsMtiD4LynocuFEH+W+QKu6td5QDCM/KuZQG+9G8Y/aA3THgKbHZolwupP4PMJ8HwyTBoM0x+BX7417RMnZLkWqrfZG35+IZqw2jYWPwJMBnoBZwGTtdYNHUhW2TfAGQ3QWuvJWutUrXVqXFxcA0/pP24c3JaE5iH8a8YmyipOYR3ewgwyW/8FHNjs+WAKjsKaT6HnVVX/+q5Mz6uhVS+Y87QpITTE6o/NL/zznoYbpsKjWXDLbBj1N4hoBas+hP/dCM+1h9eHmMRwYKOMHxB+odblba31VK31w1rrh7TWX7nh3LuAxHLPE4A9bjiuAIIcdh45vwsb9+by1ardZ+4w5AFwhDROqWD1R2bStgG31+19Npv54s7JNiWK+jp+2FQxJQ6C3jeYbfYAMwXH0D/AjV/Bn7Pg5h9g5OOmumrl+2a/DtI7Wvi+mlYoy1NK5VZyy1NK5Tbw3MuBTkqp9kqpQOA6zEA14SYX92pNzzZRvPjjZgpLKkw6FxYLA2+H9V+aKas9xemEZW+aL+HWvev+/g7DoeNoMxCufLVNXcz6KxTlwkUvVd3W4AiEpEEw7BEY/y08mg33r4aE1PqdUwgvUm0i0FpHaK0jK7lFaK0jq3tvTbTWpcC9wA/ARuB/WusNDTmmOJ3NZgaZ7c0p5J2FlQwiS7sfAsM8Wyr4dRYc2WGSTn2NfgqK8uDnF+v+3qxFptpn8D11W5fBEQTR7et+PiG8kBu6YtSf1nq61rqz1jpZa/1PK2PxVYOTYxjVtQWvz93G4WPFp78YGg0D74QNX5kBXJ6w9A3TJbTbJfU/RsvuZlGYZZPhSGbt31daDNMehqgkOEfmRhSiKpYmAtE4Hr2wK8eKS5k4Z+uZLw6+B4IiId0DY/kObjVTYKfeYurkG2LEY6DsMOep2r9nyWumwXfM86bkI4SolCQCP9CpZQTX9k/kwyVZZB86fvqLodEw6C7Y+C3sXeveEy+bbAaM9ZvQ8GNFtoa0e81YhN0rat7/SBak/xu6XgRdLmj4+YXwYZII/MQDozpjU4rX5v565ouD7oagKPe2FRTmmi6bKVdAuJu6/abdD6Gx8OPfzLiAqmhtun8qG1zYiGMlhPBSkgj8RKuoYMYNSGTqyl3sPFyhVBDSzFQRbZoGe1a754SrP4bi/IY1ElcUHAnDH4WsBbBlZtX7bZoGW3+AEX8xs64KIaolicCP3DW8YzWlgjshOMo9bQVOp6kWSugPbfo1/Hjl9ZsAMR1h1t+grPTM14vyYPqfoGVP0xAuhKiRJAI/0ioqmOsGJPLFikpKBcFRkHYfbJkBu1c27ETb5sDhbTDgjoYdpzL2ADj3SbNy2KoPznx97r/MDKYX/T+w13YqLSH8myQCP3PX8GRsSjEpvZJSwYA7IKQ5pP+rYSdZ+l8IbwndL23YcarSdawZoDb3GSjKP7V97xpY+ropNST298y5hfBBkgj8THxUCNcNSOTzjMpKBZGmQXbrj7Aro34nOLTNDCJLvdmM1vUEpeC8f8Cx/bDoFbPNWWYmlQuJhnP/7pnzCuGjJBH4oepLBbebuXZ+etrU9dfVsslgCzDLYnpSYn/ofhksmgh5+2DFu6Zb6fnPmFKNEKLWJBH4ofioEK7tb0oFu45UKBUEhZtRuNvT4bPrTTfQ2irKM2sip1xm1j3wtHP/DmUlpqvo7CfNAja9rvH8eYXwMZII/NSJUsFrcytZ6H7A7TDmBdjyA7x9HhzeXruDrvkUivMar7dOdAezyM7Gb6G0AMa+JGsLC1EPkgj8VOtmplTwxYqdZ5YKlIIBt5npmfP3wZsjTQmhOie6jLbu27gzdg77k5lLaMRjENup8c4rhA+RRODH7hpuFq+flF5JqQCgwzlw208Q3go+uAKWTq56RO/2uaZL50APdBmtTlgMPLAGzn6wcc8rhA+RRODHTpQKPs/Yye6jBZXvFN0Bbp0Fnc+HGY/Ad/ebWT0rWjYZwuIg5XLPBl0Zd6xnLIQfk78gP3f3cLN4/aTKRhufEBQB135kFrtf+T68fwnkl1sL+fB2057Q7yYzj78QwqtIIvBzrZuFcE1qIv+rrlQA5lf3qL/ClW/DnlUwefip2UqXvWUWeE+9uVFiFkK4lyQCwd0jalEqOKHnVXDzTECbHkWrPjIrgHW/FCLjPRuoEMIjJBEI2pQrFeyprlRwQus+cNtcaNUTvrkbinI8M6+QEKJRSCIQQLlSQWWjjSsT0RImTIP+t0KPKyFxgAejE0J4kiQCAZhSwdWpifxv+a7alQrANAyPfRGuekcGcgnhxSQRiJPuHp6MRvN6VeMKhBA+SRKBOCmheShXpyby2fKd7M2pZalACOH1JBGI00ipQAj/I0s4idOcKBV8sCSLkjInj5zflegwD60rIIRoEiQRiDM8PrYboQF23l2Uyfdr9/LH87vwuwFJOOxSgBTCF8lftjhDaKCDxy/qzowHhtIzIYq/fbOBi15ZwLIdh60OTQjhAZIIRJU6tYzgw1sGMun6vuQWlHDNfxfzwKer2JdTaHVoQgg3kkQgqqWUYkzPeOb8YTj3j+zIjPX7GPliOm/M20ZxaT2WshRCNDmSCESthATaefi8Lsx+6BzSkmN5dsYmLnh5PvO2HKj5zUKIJk3pqhYaaYJSU1N1RkaG1WEIYO7m/Tz13S/sOHiMrq0iCA6woxQoTCnC3IPCbDzxvH1sGFf2TaBf2+YoGY0sRKNQSq3QWle5dKAkAlFvRaVlTFmYyaJthzjx/0hr0GhzX/4xoLVmw55cjheX0T42jKv6JXBF3zbER4VYeyFC+DhJBKJJOVZUyvR1e/l8xS6W7TiMTcHZneK4ql8C53VvSXCA3eoQhfA5kghEk5V16BhTV+xi6srd7D5aQGSwg0t6t+aqfomclRAlVUdCuIkkAtHkOZ2aRdsO8cWKncxYv4+iUiedWoRz+7AOXNUvQRKCEA0kiUB4ldzCEqat2cvHy7JYvzuXAe2jeebyHnRsEWF1aEJ4rZoSgXQfFU1KZHAAvxuYxLf3nM2zV/Rk8748LvzPz7z442YKS8qsDk8In2RJIlBKPa+U2qSUWquU+kop1cyKOETTZbMprhuQxJw/nMPFvVrzyk+/cv7L8/l5q4xbEMLdrCoRzAJ6aK17AVuA/7MoDtHExYYH8dK1vfno1oHYlOLGt5dx/yer2J8n01wI4S6WzD6qtf6x3NMlwFX1PVZJSQm7du2isFC+GJqS4OBgEhISCAgIcMvxhnSMZcYDQ5mUvo030reRvnk/f76wK+P6J2GzSWOyEA1heWOxUuo74DOt9YdVvH47cDtAUlJSv6ysrNNe37FjBxEREcTExEjvkiZCa82hQ4fIy8ujffv2bj/+tgP5PPbVOpZsP0zfpGY8c0VPuraKdPt5hPAVlvUaUkrNBlpV8tJjWutvXPs8BqQCV+haBFJZr6GNGzfStWtXSQJNjNaaTZs20a1bN48d/8uVu/nH97+QW1hKclwYkcEBRAQ7iAwJIDI4gMgQh2vbqcfNQwPpFh8haysIv1JTIvBY1ZDW+tzqXldKjQcuAkbVJgnUcKyGvF14gKc/E6UUV/ZLYGTXFrwxfxvZh46TW1jCwfxith88Rm5BCbmFpZQ5z/yv1aZZCNcPSuLa1ERiwoM8GqcQ3sCSNgKl1AXAn4FztNbHrYhB+IbmYYH834WVlzq01hSUlJFbUEpuYQm5BSXsPlrA/zJ28tzMzbw8eysX92rN+LS29Epo1riBC9GEWLVU5atAEDDL9ctxidb6TotiaZBDhw4xatQoAPbt24fdbicuLg6AZcuWERhY9Xq/GRkZvP/++0ycOLHac6SlpbFo0aIGx5qens6ll156Wr39Cy+8wLnnVlt481pKKUIDHYQGOmgVFQyYeshLe7dh6295fLAkyzXFxS7OSmzG+MFtGdsrniCHzHck/IvljcV1UVUbgafqoevqiSeeIDw8nD/+8Y8nt5WWluJwNI2lodPT03nhhReYNm1alftordFaY7PZKn1elbKyMuz2079Am9JnU5W8whK+XLmb9xZnsv3AMWLCArluQCLXD2xL62YyK6rwDZa1EVjhye828MueXLces3vrSP5+cUqd3jNhwgSio6NZtWoVffv25dprr+XBBx+koKCAkJAQpkyZQpcuXU77Yn7iiSfIzs5m+/btZGdn8+CDD3L//fcDEB4eTn5+Punp6TzxxBPExsayfv16+vXrx4cffohSiunTp/Pwww8TGxtL37592b59e7Vf+OVlZmZy4YUXMmLECBYvXszLL7/MnXfeefL5119/zauvvsqMGTNQSvH4449z7bXXkp6ezpNPPkl8fDyrV6/ml19+qfO/r9UiggMYn9aO3w9uy8JfD/He4kxeT9/G6+nbGNm1BSmto2jTPISE5iEkNg+lVVQwAdLQLHyMTyWCpmTLli3Mnj0bu91Obm4u8+fPx+FwMHv2bP7yl78wderUM96zadMm5s6dS15eHl26dOGuu+46ox/+qlWr2LBhA61bt2bIkCEsXLiQ1NRU7rjjDubPn0/79u0ZN25clXH9/PPP9O7d++TzqVOnYrfb2bx5M1OmTGHSpElkZmae9nzq1KmsXr2aNWvWcPDgQfr378+wYcMAU/21fv16j3QTbUxKKc7uFMvZnWLZefg4Hy3N5rs1e5izaT/lC802Ba0ig0loHkqCK0G0aR5C25gweiVEERoof1LC+/jU/9q6/nL3pKuvvvpkVUlOTg7jx49n69atKKUoKSmp9D1jx44lKCiIoKAgWrRowW+//UZCQsJp+wwYMODktt69e5OZmUl4eDgdOnQ4+WU8btw4Jk+eXOk5hg4dekZJITMzk7Zt2zJo0KCT28o/X7BgAePGjcNut9OyZUvOOeccli9fTmRkJAMGDPD6JFBRYnQoj17YlUcv7EpxqZN9OYXsOnKcXUcK2HW04OTjpTsO8/XqAk50TLLbFD1aR9KvbTT92zWnX7vmtIgItvZihKgFn0oETUlYWNjJx3/9618ZMWIEX331FZmZmQwfPrzS9wQFnerKaLfbKS0trdU+7mjnKR9vxefVHb/i+3xNoMNGUkwoSTGhlb5eUmYSxa/781mRdYTlmYf5aGkW7yzcAUDbmFBSXYkhtV00yXFh0t1ZNDmSCBpBTk4Obdq0AeDdd991+/G7du3K9u3byczMpF27dnz22WduPf6wYcP473//y/jx4zl8+DDz58/n+eefZ9OmTW49jzcKsNtIjA4lMTqUEV1bAFBc6mTDnhwyMk1imLt5P1NX7gKgeWgA/dpGk9quOf3bNadHmyjppSQsJ4mgEfzpT39i/PjxvPTSS4wcOdLtxw8JCWHSpElccMEFxMbGMmDAgCr3rdhG8Pjjj5OaWmVnAgAuv/xyFi9ezFlnnYVSiueee45WrVpJIqhCoMNGn6Tm9Elqzm3DOqC1ZsfBYycTQ0bWEWZv/O3kvr3aRJHaLprUts3p17Y5zcOq7nIshCdI91EfkZ+fT3h4OFpr7rnnHjp16sRDDz1kaUzy2VTtYH4RK7KOkOFKDOt351BSZv4WO7YIN20MbaPp2iqC8CAH4cEOwoMcBDlsUrUk6syvuo/6szfffJP33nuP4uJi+vTpwx133GF1SKIaseFBnJ/SivNTzHRchSVlrNl5lIysI6zIOsL3a/fyybKdZ7zPYVMnk8LJm+t555YRjO0VT3JceGNfjvByUiIQHiOfTf05nZpfD+Sz4+AxjhWVcqyolLyiUvILS8kvct3KPc4rLCXz0DG0hq6tIrj4rNaM7RlPu1jfbswXtSMlAiG8kM2m6Nwygs4ta79W876cQmas38u0tXt5/ofNPP/DZlJaR3JRL5MUqur5JIQkAiF8RKuoYG4a0p6bhrRnz9ECpq8zSeHfMzfx75mb6JUQxUW94hnTM56E5pIUxCmSCITwQa2bhXDr0A7cOrQDOw8fZ/q6vXy/bi/PTN/EM9M30TIyiFZRIbSOCiY+KoTWzYJpVe5xXHiQrNngRyQRCOHjEqNDueOcZO44J5msQ8eYuX4fv+7PZ29OIZt/yyN98wEKSspOe4/dpmgREUR8VDCJ0aEkucZKtI02g+taRgTLEqE+RBJBAzVkGmowM4IGBgaSlpZ2xmvvvvsujzzyyMnBaAAff/wx3bt3d+MVCH/SNiaMO85JPm2b1prcglL25BSwL6fw1P3RQvYcLWBl9hGmrd172iI/gQ4bCc1DTGJwJYl2MWH0aBN1cspv4T0kETRQTEwMq1evBiqfhrom6enphIeHV5oIAK699lpeffXVKt9fcfrnyqaDrkxTmh5bWEspRVRoAFGhAXSLr3zt55IyJ3uOFpB9+DhZh46z8/Dxk4+XZx4hv+jUdCjxUcH0SWpGn8Tm9ElqRo82UQQHyOjppsy3vglmPAr71rn3mK16woXP1uktK1as4OGHHyY/P5/Y2Fjeffdd4uPjmThxIm+88QYOh4Pu3bvz7LPP8sYbb2C32/nwww955ZVXGDp0aI3Hrzj986RJk057vnLlSu666y4yMjJwOBy89NJLjBgxgnfffZfvv/+ewsJCjh07xk8//VTffxXhZwLsNtrGhNE2JoyhnU5/TWvN0eMlbD+Yz5qdOazaeZRV2UeYvm4fYMY+dG8dSZ/EZq4R181Iig6VgXFNiG8lgiZAa819993HN998Q1xcHJ999hmPPfYY77zzDs8++yw7duwgKCiIo0eP0qxZM+68885qSxGfffYZCxYsOPl88eLFwOnTP6enp5/2/MUXXwRg3bp1bNq0ifPOO48tW7acfP/atWuJjo728L+E8BdKKZqHBdIvLJp+bU/9vzqQV8RqV1JYlX2Uz1fs4r3FWQBEBjuIjQgiKiTgtFuzkAAiyz8PDSQ8yIHDrrAphd2msCuFzcbJx3abudlsikC7TUof9eBbiaCOv9w9oaioiPXr1zN69GjAVNXEx8cD0KtXL66//nouu+wyLrvsslodr6qqoYrTP5d/vmDBAu677z7ATEjXtm3bk4lg9OjRkgREo4iLCGJ095aM7t4SgDKnZstveazKPsove3M4ctysI30ov5jtB46RU1BCbmEJDR3jGmi3ERniICI4gMhgcx8R7CDyxH2IuQ8LdBDgUATa7QTYFYEOG4F2G4EOGwF2czuxLTLEQVRIgM+WYnwrETQBWmtSUlJO/nIv7/vvv2f+/Pl8++23PP3002zYsKHe55Fpo4W3sdsU3eIjq2yHADOiOq+wlJyCkpO3vMISyrSmzKlxak2Z0+xXpjWlTm0eu14rKnWSW1hCXmEpuQXmPq+whH25heQVlpBbUHpGD6naCrTbiIsIIjYiiBYRQcRFBBEXHkSLSHMfFxFEi8hgYsICva5UIonAzYKCgjhw4ACLFy9m8ODBlJSUsGXLFrp168bOnTsZMWIEZ599Nh9//DH5+flERESQm+ve5TWHDRvGRx99xMiRI9myZQvZ2dl06dKFlStXuvU8QribzXaq4dpTSsqc5BWWcry4lJIyTXGpk5IyJ0Wu+xPPT2wrLnWSU1DCgfwiDuQWcSC/iJ2Hj7Mi6wiHjxVXeo6IIFP1FRMWSGx4EDHh5j42/MRzsy000H6yFHKi9GFFqUMSgZvZbDa++OIL7r//fnJycigtLeXBBx+kc+fO3HDDDeTk5KC15qGHHqJZs2ZcfPHFXHXVVXzzzTeVNhZXbCOYNGlSjTHcfffd3HnnnfTs2ROHw8G777572oI2QvizALuN6LBAot0w3XdJmZND+cXszyvkQF4R+/OKOJRfxMH8Yg7mF3Ewv4htB/JZuqOII8crX5mwokC77VRVVbkE8a8rejGgvWeqdWXSOeEx8tkIcUppmZPDx4pPJolDx4ooLDEljuJSJ8XlSiDmedlpr90zoiMpraPqdW6ZdE4IIZoAh91Gi8hgWkQ2vQF3MpmIEEL4OZ9IBN5UveUv5DMRwnt4fSIIDg7m0KFD8sXThGitOXToEMHBTa8ILIQ4k9e3ESQkJLBr1y4OHDhgdSiinODgYBISEqwOQwhRC16fCAICAk4bYSuEEKJuvL5qSAghRMNIIhBCCD8niUAIIfycV40sVkodALLKbYoFDloUjqf56rXJdXkfX702f7qutlrruKre4FWJoCKlVEZ1w6a9ma9em1yX9/HVa5PrOkWqhoQQws9JIhBCCD/n7YlgstUBeJCvXptcl/fx1WuT63Lx6jYCIYQQDeftJQIhhBANJIlACCH8nNcmAqXUBUqpzUqpX5VSj1odj7sopTKVUuuUUquVUhk1v6PpUkq9o5Tar5RaX25btFJqllJqq+u+uZUx1kcV1/WEUmq363NbrZQaY2WM9aGUSlRKzVVKbVRKbVBKPeDa7tWfWTXX5QufWbBSaplSao3r2p50ba/TZ+aVbQRKKTuwBRgN7AKWA+O01r9YGpgbKKUygVSttdcPdFFKDQPygfe11j1c254DDmutn3Ul8OZa6z9bGWddVXFdTwD5WusXrIytIZRS8UC81nqlUioCWAFcBkzAiz+zaq7rGrz/M1NAmNY6XykVACwAHgCuoA6fmbeWCAYAv2qtt2uti4FPgUstjklUoLWeDxyusPlS4D3X4/cwf5BepYrr8npa671a65Wux3nARqANXv6ZVXNdXk8b+a6nAa6bpo6fmbcmgjbAznLPd+EjHyzmQ/xRKbVCKXW71cF4QEut9V4wf6BAC4vjcad7lVJrXVVHXlV9UpFSqh3QB1iKD31mFa4LfOAzU0rZlVKrgf3ALK11nT8zb00EqpJt3lfHVbkhWuu+wIXAPa5qCNH0vQ4kA72BvcCLlkbTAEqpcGAq8KDWOtfqeNylkuvyic9Ma12mte4NJAADlFI96noMb00Eu4DEcs8TgD0WxeJWWus9rvv9wFeYajBf8purzvZE3e1+i+NxC631b64/SCfwJl76ubnqmacCH2mtv3Rt9vrPrLLr8pXP7ASt9VEgHbiAOn5m3poIlgOdlFLtlVKBwHXAtxbH1GBKqTBXYxZKqTDgPGB99e/yOt8C412PxwPfWBiL25z4o3O5HC/83FwNj28DG7XWL5V7yas/s6quy0c+szilVDPX4xDgXGATdfzMvLLXEICrq9fLgB14R2v9T2sjajilVAdMKQDMMqIfe/N1KaU+AYZjpsX9Dfg78DXwPyAJyAau1lp7VcNrFdc1HFPFoIFM4I4TdbTeQil1NvAzsA5wujb/BVOf7rWfWTXXNQ7v/8x6YRqD7Zgf9v/TWj+llIqhDp+Z1yYCIYQQ7uGtVUNCCCHcRBKBEEL4OUkEQgjh5yQRCCGEn5NEIIQQfk4SgWhylFJlrtkg1yulPldKhVaz7wSl1KuNGV+5cz+llDq3hn3eVUpdVcM+7crPZCpEY5NEIJqiAq11b9fMnsXAnVYHVBmt9d+01rOtjqM818y8QtSJJALR1P0MdHTNr/61a4KwJa6BNCcppSKUUjtcUwmglIpUZm2HAKVUulLq365527copYa69glWSk1RZv2HVUqpEa7tE1zn+s51zHuVUg+79lmilIp27Xfy175S6m9KqeWuUsxk12jWKiml+rnmkF8M3FNuu10p9bzrWGuVUne4ttuUUpOUmXN+mlJqerlzZ7rOvwC4Wil1nlJqsVJqpatEFV7unPNcExr+UGFkrfBjkghEk6WUcmAm31sHPAms0lr3wowKfb/8vq7phdOBsa5N1wFTtdYlrucOrfUA4EHMSGBwfQFrrXtiRpm+p5QKdr3WA/gdZv6ZfwLHtdZ9gMXA7ysJ91WtdX9XKSYEuKiGy5sC3K+1Hlxh+y1Ajta6P9AfuE0p1R4zv3w7oCdwK1DxfYVa67OB2cDjwLmuyQszgIddCfIV4CqtdT/gHdd1CYHD6gCEqESIa1pdMCWCtzHTHFwJoLX+SSkVo5SKqvC+t4A/YaaxuAm4rdxrJyZQW4H5QgU4G/PliNZ6k1IqC+jsem2uK7nkKaVygO9c29cBp5VGXEYopf4EhALRwIZy7zmNK+5mWut5rk0fYBIemPmlepVrV4gCOrli/dw1Qdo+pdTcCof9zHU/COgOLHQVSgIxyasLJrnNcm23Y2bcFEISgWiSClzT6p5URVXLafOjaK0XuhpezwHsWuvyDbBFrvsyTv2/r676pqjcY2e5504q/N24ShGTMCvL7VRmtbJgqqYqxl7htfu01j9UOMfYKvY/4Vi598/SWo+r8P6ewIZKSiBCSNWQ8BrzgesBlFLDgYNVzJX/PvAJpuqlLsfsjJmga3M9YjvxpX/QVR9fbS8h13TBOa7J0DgRg8sPwF3l2jo6KzMT7QLgSldbQUvMJHeVWQIMUUp1dL0/1HVtm4E4pdRg1/YApVRKHa9T+CgpEQhv8QQwRSm1FjjOqSl2K/oI+AcmGdRkEvCGUmodUApM0FoX1dDOewat9VGl1JuYaqNMzDTpNbkJeEcpdRzz5X/CW5iqq5WuUtABzDKDU4FRmKmSt2CqynIqieWAUmoC8IlSKsi1+XGt9RZXddNEV9WUAzN774a6XKvwTTL7qPApri+7S7XWN1odi7sppcJdi5THAMswq9ntszou4f2kRCB8hlLqFUyj6xirY/GQacosQhIIPC1JQLiLlAiEEMLPSWOxEEL4OUkEQgjh5yQRCCGEn5NEIIQQfk4SgRBC+Ln/Dz+f7VBFxXRrAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_65_16.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Common imports\n", "import os\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from sklearn.linear_model import LinearRegression, Ridge, Lasso\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.utils import resample\n", "from sklearn.metrics import mean_squared_error\n", "# Where to save the figures and data files\n", "PROJECT_ROOT_DIR = \"Results\"\n", "FIGURE_ID = \"Results/FigureFiles\"\n", "DATA_ID = \"DataFiles/\"\n", "\n", "if not os.path.exists(PROJECT_ROOT_DIR):\n", " os.mkdir(PROJECT_ROOT_DIR)\n", "\n", "if not os.path.exists(FIGURE_ID):\n", " os.makedirs(FIGURE_ID)\n", "\n", "if not os.path.exists(DATA_ID):\n", " os.makedirs(DATA_ID)\n", "\n", "def image_path(fig_id):\n", " return os.path.join(FIGURE_ID, fig_id)\n", "\n", "def data_path(dat_id):\n", " return os.path.join(DATA_ID, dat_id)\n", "\n", "def save_fig(fig_id):\n", " plt.savefig(image_path(fig_id) + \".png\", format='png')\n", "\n", "infile = open(data_path(\"EoS.csv\"),'r')\n", "\n", "# Read the EoS data as csv file and organize the data into two arrays with density and energies\n", "EoS = pd.read_csv(infile, names=('Density', 'Energy'))\n", "EoS['Energy'] = pd.to_numeric(EoS['Energy'], errors='coerce')\n", "EoS = EoS.dropna()\n", "Energies = EoS['Energy']\n", "Density = EoS['Density']\n", "# The design matrix now as function of various polytrops\n", "\n", "Maxpolydegree = 30\n", "X = np.zeros((len(Density),Maxpolydegree))\n", "X[:,0] = 1.0\n", "testerror = np.zeros(Maxpolydegree)\n", "trainingerror = np.zeros(Maxpolydegree)\n", "polynomial = np.zeros(Maxpolydegree)\n", "\n", "trials = 100\n", "for polydegree in range(1, Maxpolydegree):\n", " polynomial[polydegree] = polydegree\n", " for degree in range(polydegree):\n", " X[:,degree] = Density**(degree/3.0)\n", "\n", "# loop over trials in order to estimate the expectation value of the MSE\n", " testerror[polydegree] = 0.0\n", " trainingerror[polydegree] = 0.0\n", " for samples in range(trials):\n", " x_train, x_test, y_train, y_test = train_test_split(X, Energies, test_size=0.2)\n", " model = LinearRegression(fit_intercept=False).fit(x_train, y_train)\n", " ypred = model.predict(x_train)\n", " ytilde = model.predict(x_test)\n", " testerror[polydegree] += mean_squared_error(y_test, ytilde)\n", " trainingerror[polydegree] += mean_squared_error(y_train, ypred) \n", "\n", " testerror[polydegree] /= trials\n", " trainingerror[polydegree] /= trials\n", " print(\"Degree of polynomial: %3d\"% polynomial[polydegree])\n", " print(\"Mean squared error on training data: %.8f\" % trainingerror[polydegree])\n", " print(\"Mean squared error on test data: %.8f\" % testerror[polydegree])\n", "\n", "plt.plot(polynomial, np.log10(trainingerror), label='Training Error')\n", "plt.plot(polynomial, np.log10(testerror), label='Test Error')\n", "plt.xlabel('Polynomial degree')\n", "plt.ylabel('log10[MSE]')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Cross-validation\n", "\n", "When the repetitive splitting of the data set is done randomly,\n", "samples may accidently end up in a fast majority of the splits in\n", "either training or test set. Such samples may have an unbalanced\n", "influence on either model building or prediction evaluation. To avoid\n", "this $k$-fold cross-validation structures the data splitting. The\n", "samples are divided into $k$ more or less equally sized exhaustive and\n", "mutually exclusive subsets. In turn (at each split) one of these\n", "subsets plays the role of the test set while the union of the\n", "remaining subsets constitutes the training set. Such a splitting\n", "warrants a balanced representation of each sample in both training and\n", "test set over the splits. Still the division into the $k$ subsets\n", "involves a degree of randomness. This may be fully excluded when\n", "choosing $k=n$. This particular case is referred to as leave-one-out\n", "cross-validation (LOOCV). \n", "\n", "\n", "* Define a range of interest for the penalty parameter.\n", "\n", "* Divide the data set into training and test set comprising samples $\\{1, \\ldots, n\\} \\setminus i$ and $\\{ i \\}$, respectively.\n", "\n", "* Fit the linear regression model by means of ridge estimation for each $\\lambda$ in the grid using the training set, and the corresponding estimate of the error variance $\\boldsymbol{\\sigma}_{-i}^2(\\lambda)$, as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{align*}\n", "\\boldsymbol{\\beta}_{-i}(\\lambda) & = ( \\boldsymbol{X}_{-i, \\ast}^{T}\n", "\\boldsymbol{X}_{-i, \\ast} + \\lambda \\boldsymbol{I}_{pp})^{-1}\n", "\\boldsymbol{X}_{-i, \\ast}^{T} \\boldsymbol{y}_{-i}\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "* Evaluate the prediction performance of these models on the test set by $\\log\\{L[y_i, \\boldsymbol{X}_{i, \\ast}; \\boldsymbol{\\beta}_{-i}(\\lambda), \\boldsymbol{\\sigma}_{-i}^2(\\lambda)]\\}$. Or, by the prediction error $|y_i - \\boldsymbol{X}_{i, \\ast} \\boldsymbol{\\beta}_{-i}(\\lambda)|$, the relative error, the error squared or the R2 score function.\n", "\n", "* Repeat the first three steps such that each sample plays the role of the test set once.\n", "\n", "* Average the prediction performances of the test sets at each grid point of the penalty bias/parameter. It is an estimate of the prediction performance of the model corresponding to this value of the penalty parameter on novel data. It is defined as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{align*}\n", "\\frac{1}{n} \\sum_{i = 1}^n \\log\\{L[y_i, \\mathbf{X}_{i, \\ast}; \\boldsymbol{\\beta}_{-i}(\\lambda), \\boldsymbol{\\sigma}_{-i}^2(\\lambda)]\\}.\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For the various values of $k$\n", "\n", "1. shuffle the dataset randomly.\n", "\n", "2. Split the dataset into $k$ groups.\n", "\n", "3. For each unique group:\n", "\n", "a. Decide which group to use as set for test data\n", "\n", "b. Take the remaining groups as a training data set\n", "\n", "c. Fit a model on the training set and evaluate it on the test set\n", "\n", "d. Retain the evaluation score and discard the model\n", "\n", "\n", "5. Summarize the model using the sample of model evaluation scores\n", "\n", "The code here uses Ridge regression with cross-validation (CV) resampling and $k$-fold CV in order to fit a specific polynomial." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEGCAYAAABvtY4XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA3pklEQVR4nO3dd3hU1dbH8e9KIwkJhBRqQAIEIUBCBwEpgiCIiuIFRBTQKyoq9mu7dq9iuRa8iqIiiihFURFQQDpKCy303kNJKIE0Uma/fyTwIiSQxJw5M5P1eZ48kJkze/+GkJWdffbZR4wxKKWU8jxedgdQSillDS3wSinlobTAK6WUh9ICr5RSHkoLvFJKeSgfuwOcLzw83NSuXdvuGEop5TZWrVqVbIyJKOg5lyrwtWvXJj4+3u4YSinlNkRkb2HP6RSNUkp5KC3wSinlobTAK6WUh3KpOfiCZGdnc+DAATIzM+2OokrA39+fyMhIfH197Y6iVJnj8gX+wIEDBAcHU7t2bUTE7jiqGIwxHDt2jAMHDhAVFWV3HKXKHJefosnMzCQsLEyLuxsSEcLCwvS3L6Vs4vIFHtDi7sb0a6eUfVx+ikYp5VoOr9/G3i+/w5w6Rblb+tC0Zwf9Qe6itMArpYrEOBwsH/YkzcZ9SNXcbACGpASTecCH//ZrSo2QAJsTqgu5xRSNurw9e/bQuHFju2MoD2UcDpb3GUzbL95lY4uOHFyZQPrxk3R7eBAbD55i/F3/JnH1RrtjqgvoCP5vysnJwcen7P0zltX3XVZNWLaXPal+OG4eylXff4545Y0NB3WoSIsQb6q+04+TS38kY/M6AkKCbU6rznKr79CXf9nIpsRTpdpmTPUKvHhDo0se8/XXX/POO+8gIsTGxuLt7U1oaChr1qyhefPm3HHHHdx3332kp6dTt25dxo4dS6VKlRg1ahSffPIJPj4+xMTEMHHiRBYuXMjDDz8M5J2AXLRoEcHBF39D9O/fn8GDB9OrVy8AhgwZwg033ECLFi244447SEtLA+B///sf7dq1u+z73LhxI0OHDiUrKwuHw8EPP/xAdHT0Re9t/Pjx7N27l7vuuoukpCQiIiL48ssvqVWrFkOGDPnL+x4+fDgPPPAASUlJBAYG8tlnn9GgQYPifgmUi9uZlMprv26h1e3DeHZoa8Trr/PtDRtHsf7Dz2hyd3+W97+bNrMm25RUXcjSAi8iDwP3AAJ8Zox538r+rLBx40b+85//8McffxAeHs7x48d57LHH2LZtG7///jve3t7Exsby4Ycf0qlTJ1544QVefvll3n//fUaOHMnu3bspV64cJ0+eBOCdd97ho48+on379qSmpuLv719gvwMGDGDSpEn06tWLrKws5s6dy+jRozHGMGfOHPz9/dm+fTu33XZbkTZo++STT3j44Ye5/fbbycrKIjc3t8D3BvDggw9y5513MnjwYMaOHcuIESP46aefAP7yvrt27conn3xCdHQ0y5cvZ/jw4cybN69U/t2V69h/6yB6R8Tw5JMv4OVV8MnUJnf1Y9n0X2n74zi2/DSbBn26OzmlKpAxxpIPoDGwAQgk7wfJ70D0pV7TokULc6FNmzZd9JgzjRo1yjz77LN/eWzw4MFm3LhxxhhjTp48aWrWrHnuuR07dphmzZoZY4zp0aOH6du3rxk/frw5ffq0McaYN954w7Ru3dp88MEHZv/+/YX2m5GRYSIjI01mZqb56aefzMCBA8/1N2jQINO4cWMTFxdnAgICjDHG7N692zRq1KjQ9iZMmGBiYmLMyJEjzbZt2wp9b8YYExYWZrKysowxxmRlZZmwsLCL3vfp06eNv7+/iYuLO/fRoEGDAvu2+2uoSm79Vz8YA2bp/U9f9tjTScfN0eBQs61WA5ObneOEdMoYY4B4U0hNtfIka0NgmTEm3RiTAywEbrawP0sYYwpcAla+fPnLvnbGjBk88MADrFq1ihYtWpCTk8PTTz/N559/TkZGBm3btmXLli0Fvtbf35/OnTsza9YsJk2axIABAwB47733qFKlCuvWrSM+Pp6srKwivY+BAwcybdo0AgIC6NGjB/PmzSv0vV3o/GPOvm+Hw0FISAhr164997F58+YiZVHuw+fllzgcUpmmb/77sscGhVdi66vvMrLVP5iz5agT0qnLsbLAbwA6ikiYiAQCvYCaFx4kIsNEJF5E4pOSkiyMUzJdu3Zl8uTJHDt2DODcNMZZFStWpFKlSixevBiA8ePH06lTJxwOB/v376dLly689dZbnDx5ktTUVHbu3EmTJk146qmnaNmyZaEFHvKmab788ksWL15Mjx49AEhJSaFatWp4eXkxfvx4cnNzi/Q+du3aRZ06dRgxYgQ33ngjCQkJhb63du3aMXHiRAAmTJhAhw4dLmqvQoUKREVFMWXKFCDvB+G6deuKlEW5hy1Tf6PBrvXsHToc/+DLD2gA2j54BzvadGHUvB1nf5NXNrJsDt4Ys1lE3gTmAKnAOiCngOPGAGMAWrZs6XL/Ixo1asRzzz1Hp06d8Pb2plmzZhcd89VXX507yVqnTh2+/PJLcnNzGTRoECkpKRhjePTRRwkJCeH5559n/vz5eHt7ExMTQ8+ePQvtu3v37tx5553ceOON+Pn5ATB8+HD69u3LlClT6NKlS5F+kwCYNGkS33zzDb6+vlStWpUXXniB0NDQi97buHHjGDVqFHfddRdvv/32uZOsBZkwYQL3338/r732GtnZ2QwYMIC4uLgi5VGuL+P1N0nxD6LJ848U+TU+3l6MaFudxKdeJKFiEnGDb7EuoLoscdZPWRF5HThgjPm4sGNatmxpLjxhuHnzZho2bGh1PGUh/Rq6n6OnMnl5yCv0qh3M9e8+U6zXZqVncrpyNfbVj6XZ6oUWJVRnicgqY0zLgp6z9EInEamc/2ct4BbgOyv7U0qVjp/WHmRG/fY0eOahYr/WL9Cfbb37Ebt2CYfXb7MgnSoqq69k/UFENgG/AA8YY05Y3J/bWb9+PU2bNv3LR5s2bUrc3qxZsy5q7+ab3e7ctrKRcTjIHPURXStkUzciqERt1H72UcQYdo98v3TDqWKxdB28MeZqK9v3BE2aNGHt2rWl1l6PHj3OnZBVqiR2/LqQET+8x/LoCKBPidqoFtuAtXHtiZ42kaz0d/ELLPh6D2Ut3YtGKfUXxz8ewxlvXxqM+OffasfcP5xlkY1ZvmZHKSVTxeVWWxUopazlyMml7qLZbGx+Nc2rRfytthrd3Z+hR8LotD8b/VXeHjqCV0qds236XMJTj+O4qc/fbsvPx4teTaqxbdEq0k+W7h5Sqmi0wBdBUND/n2iaOXMm0dHR7Nu3j5deeokaNWqcO5n59NNPF9rGpbbz7dy5c5H2k1HKanvmLiXTx4/oIf1Lpb2BcoRfR9/DptHjS6U9VTw6RVMMc+fO5aGHHmL27NnUqlULgEcffZQnnnjC5mRKlY63andiynsd+aJG5VJpL6Z3F45UjMBn0kR45oFSaVMVnY7gi2jx4sXcc889zJgxg7p16xZ6nDGGJ598ksaNG9OkSRMmTZp00TEZGRkMGDCA2NhY+vfvT0ZGhpXRlSqSHUdT2ZWURqcWUaXWppePN7u79iZm/TJSDrneViSezv1G8J07X/xYv34wfDikp0P+/ul/MWRI3kdyMtx661+fW7Dgsl2eOXOGm266iQULFly03/l7773HN998A8Cbb75Jamoqa9euZd26dSQnJ9OqVSs6duz4l9eMHj2awMBAEhISSEhIoHnz5pfNoJTVjr70H76dMZPaj5bu1acht/fHb+qXJIybTEsdxTuVjuCLwNfXl3bt2vHFF19c9Nyjjz56bjfFHj16sGTJEm677Ta8vb2pUqUKnTp1YuXKlX95zaJFixg0aBAAsbGxxMbGOuV9KHUpFefNoWpOGtWrhJRqu/Vv7EZyUCXk559LtV11ee43gr/UiDsw8NLPh4cXacR+IS8vLyZPnky3bt14/fXXefbZZws9tqh7++hd6JUrST+RQvT2dazucwd1SrltLx9vJj83ivHHyrEgJ5dyPt6l3IMqjI7giygwMJDp06czYcKEAkfyZ3Xs2JFJkyaRm5tLUlISixYtonXr1hcdM2HCBAA2bNhAQkKCpdmVupzt38/Ez5FD+d7XWdJ+w5u7c8g7kD93HrOkfVUw9xvB2yg0NJTffvuNjh07Eh4eXuAxN998M0uXLiUuLg4R4a233qJq1ars2bPn3DH3338/Q4cOJTY2lqZNm170A0ApZ8uY/itnvH2J7lvAOaxS0K5uGEM2zCL9jfkw7h1L+lAXc9p2wUWh2wV7Jv0aur63Bj5LwxMHuOHXry3rY3WbbkRuXkvEyaOIl04elBbbtgtWSrm+wymZfFyrPYlPv2hpPzk9elL59DF2zV1qaT/u5ptle3liyjocjtIfbGuBV6qMW/3HeipmnKZDdMHTjqWlzqC8uzsdnfKTpf24m/Dnn6b+hDF4eZX+wgu3KPCuNI2kike/dq4v/L9vsOCze2lYJdjafupHsbNGNBUX/G5pP+4kJyubDot+Ju5MsiXtu3yB9/f359ixY1oo3JAxhmPHjuHvr3uBu7JqCSvZ06ApXt7Wl4PDna7lRLbhVFqm5X25gz3zlxKUlYH3BRdDlhZLV9GIyKPAPwEDrAeGGmOK9ZWNjIzkwIEDJCXpZc7uyN/fn8jISLtjqEIkb9tNzeQDHOx3h1P68331FfqNWcboXSfo2aSaU/p0Zcm/zqUeEHlDd0vat6zAi0gNYAQQY4zJEJHJwABgXHHa8fX1JSqq9PbGUEr9v70/zyIcCO3ZzSn9Nb+iEsH+PizamKgFHvBd+ieHQypTtXG0Je1b/TuZDxAgIj5AIJBocX9KqWLIWbCQdF9/6lzbwSn9+Xh7MTLhBx4a3hvjcDilT1dljGGPKce2NtdY1odlBd4YcxB4B9gHHAJSjDGzLzxORIaJSLyIxOs0jFLO9WnTG/jk3lfwKefntD4rN6pP9ROH2T1/mdP6dEV7j6Xz+DX3s++lkZb1YVmBF5FKwE1AFFAdKC8igy48zhgzxhjT0hjTMiLi790iTClVdCfTs5iXWxG/m/s4td+a/W8C4OhPM53ar6uJ33EEgNZRoZb1YeUUTTdgtzEmyRiTDUwF2lnYn1KqGLb+/ie3bJhLq8rOXeVUtXE0+yJqErBovlP7dTW1nnuCX755nHoRQZc/uISsLPD7gLYiEih5Wyd2BTZb2J9Sqhgckybz1swPaFKjgtP7PtSqPdGbV5OVXnaXS1ZbH09uRGVLLnA6y8o5+OXA98Bq8pZIegFjrOpPKVU8QWvi2VOjLoGVKjq/89sG8l77gSTsOur8vl1A8o491EzaT2abqyztx9JVNMaYF40xDYwxjY0xdxhjzljZn1KqaHKzc4jauZFjjZvZ0n+DvtfxRdtbWHSobI7g90yZAUDYDdZsz3yWy1/JqpQqffv/XE1QVjpeba0dQRamYoAv7UOE09N/taV/uzl+n8sp/yDqdL/a0n60wCtVBh1elLejY9UenWzLMHztLzz34WOcPlr2bgIytWZzZva9F29fa2/JoQVeqTLop5gudHvsWyJbx9mWIbj3dfgYBzunlK3lkvuPpzOxajMy77f+BuRa4JUqg9bsO0lk47q23nij3k3XkuFTjszfZtmWwQ7rf19K3WP7aV83zPK+tMArVcacPpLMiDHP0fP0Xltz+AcFsv3KZlRd+YetOZyt8qj/Mmnis9SrbN3697O0wCtVxuz5dSHXb1lCQ2u3fy+S9Ks7UfvIHpK37bY7ilMYh4Pa65azJ7aNU3570gKvVBmTunAJAFdc19neIEDFe++m8z2fsjjV2pONrmLfn6sJTz2Oo0sXp/SnBV6pMiZg7Sr2Vq5FxWr27/10ZWw9UiJrs2THcbujOMXhqXnr3yP79nZKf1rglSpjqu/cRFL9xnbHAMDLS7gzey9XvftCmdg+uNy83zkQVoPqzWKc0p8WeKXKkKOHj3EkMISsVq3tjnJO+9zj3Lp8GvuWrrE7iqUys3MZ1u0hpv/7faf1qQVeqTJkw8kcbhjyAT4PPWR3lHNq5E9XHP5hus1JrLV893GO+pTnyp7W3H+1IFrglSpD1h84hQg0qu78HSQLU6NFIxJDq+G3cIHdUSyV+t4o7l4znavqWL/+/Swt8EqVIW1eGMGnc0ZRvpxrrVo50LwddTeuJOdMlt1RLBP7/ThuPLQOf19vp/WpBV6pMqTOxnhC/Z1XYIrKu3s3jpavxJbVW+2OYomD8eupmXyAjK7dndqva/0YV0pZJmnnXiqfSmZXs+Z2R7lI1P1DaX6sJk9k+OMa63tK14Fvp1IDiLztFqf2qyN4pcqIxN/zLnCq0L6NzUkuFhpUjphqFViyPcnuKJbwnzub/eGRRLaOdWq/WuCVKiPSl67EgVCrWwe7oxTo/r1L+PDx3mScOGV3lFKVfiab4+nZHGjf1el9W1bgReRKEVl73scpEXnEqv6UUpe2PiCCmW16ERReye4oBbqiUR0i0k6wY6pn3QRk4bZkhvZ9Ed55x+l9W3lP1q3GmKbGmKZACyAd+NGq/pRSlza2ZlvmPPKq3TEKVe+W68jy9iHt19l2RylV89bsoVKgL62iQp3et7OmaLoCO40x9u5PqlQZlZSUQmrScZrUsOEG20UUWKkiO+o2IXz5YrujlJqs9Eyeu68Hr22diY+382fEndXjAOC7gp4QkWEiEi8i8UlJnnmCRSm7JU6dwYb3+9E2aYfdUS4ppV0n6h3Yzom9iXZHKRVbvvuZkIzT1OjQwpb+LS/wIuIH3AhMKeh5Y8wYY0xLY0zLiAj7d7dTyhOlL1uBA6F2x1Z2R7mkkAF9+aRNX1bt9IzBXsbkH0jzC6DBIOcujzzLGSP4nsBqY8wRJ/SllCqAf8I6DkZEuuwJ1rOiu7Xjox73MPek+y/wy83Ooe6fv7O1aXv8g8vbksEZ/4q3Ucj0jFLKOars3sLRug3tjnFZPt5edKgVTMrvC+yO8rdt/3kO4akncPTpY1sGSwu8iAQC1wJTrexHKVW4lENJVD9xmKwmzr3IpqSGbJjNx6NHcHDVRruj/C3T0gIZ2fVurrx7gG0ZLC3wxph0Y0yYMSbFyn6UUoXbfjSNf197P969r7c7SpFUzd8++KAbbx+ck+tg0r4s9g29n+DKzts98kLuP9GllLqkDamGb5pfzxWd29odpUhqtWvO0eAwfOb+bneUElv34+90WDGLPjHhtubQAq+Uhzu1ZClxZ5KpHFzO7ihFIl5e7G3RnroJy9x2+2DHqFG8OucTOtW3d2WgFnilPFzPj17htVkfISJ2Rykyr149qZiZyo4Z8+yOUmwZJ08Ts3weWzt0p1z5QFuzaIFXyoPlnMmiVuIu0ho0sjtKsdS74x/0u+NtZgbUtDtKsW0a8w3lszIIGHKn3VG0wCvlyQ4sW0u53Gx8mjezO0qxVKwahmnXjvk7Ttgdpdi8v/uOo8FhNOzf2+4oWuCV8mTJS5YDENa+tc1Jiu/GoAxu/uptkna6zxZWR4+dxudQIjt79MHb1/77KWmBV8qD5axZS5a3D5FXudcIHuCqCF/uWjWNPd+4z2U0UxKO0Hvw+1R9/y27owBa4JXyaN+27cPL972Nr797rKA5X91rO5AUHIrXb7/ZHaVIHDm5/LxkK23rhhFVw/lbAxdEC7xSHsoYwx8Z/mR16mJ3lBIRLy92t7iaemv/dIvlkhsn/MzU1/7BfQHH7I5yjhZ4pTxU8t5Eblw4hZZeqXZHKTGf3tdTMTOVbT/NsTvKZWV98im5Xt607eM6P1C1wCvloQ79vpgX535G4yzXGVEWV/07+3IkKJSt8ZvsjnJJJ/Ym0mTlPDZ3u8m2nSMLogVeKQ+VtmIVAJEd3WOLgoIERYTyzPszeDeiBcYYu+MUauur/8UvN4eqT4ywO8pfaIFXykP5bVjP4ZDKVKxR2e4of0v3JtXZfyydzXtd8zeRrMws6kz+ivUxrandybWWo2qBV8pDhe/cwqGoBnbH+Nu61anI/M/uJeXFV+yOUqCZm5O489aXyHztDbujXEQLvFIeKDM1nWrJBzkT09juKH9beHhFMkPDqfz7r3ZHuYgxhrF/7CarUWNa3OQ6J1fP0gKvlAfaeiKLuIcnknr/g3ZHKRUp1/WmbuIOEte41snWLVNncdfofzO8QXm8vFxvMzct8Ep5oE2HTpHh50/9BlfYHaVU1Bw6EIB9X0ywOclfZb7+Jl12raJX+yvtjlIgq2/ZFyIi34vIFhHZLCJXWdmfUipP0NjPeHz5JCIrBdgdpVTUaNGInTWiCZ3+o91Rztk190+arV7ApluHEBhSwe44BbJ6N5wPgN+MMbeKiB9g7+bISpUR9X6fRjS45LRBSW0f/jg/rtzHM0mp1I4IsjsOJ599gVS/QGJGPm93lEJZNoIXkQpAR+ALAGNMljHmpFX9KaXyOHJyqbl/BylXxtgdpVTFPXAnsxu0Y1rCIbujsGfBcpqvmMv6W+506WWoVk7R1AGSgC9FZI2IfC4iF13iJSLDRCReROKTkpIsjKNU2XBo7SaCstKRuKZ2RylV1SoGcH3QGbw+/ADjcNia5dMtqYy9qi8N33zB1hyXY2WB9wGaA6ONMc2ANODpCw8yxowxxrQ0xrSMiLD3/oVKeYKjS1YAENKulc1JSt+dadt58OeP2PHbYtsybD18mol7Mzny/KuE1KpmW46isLLAHwAOGGOW53/+PXkFXylloaP7j5AcWJFaHV3rqsrScOUDQ8ny9uH4R5/almHXkPvoeHgL93Wsa1uGorKswBtjDgP7ReTs+qGugGstYlXKA02J7c6AV37Cv4L9JyJLW8UalUloey0N504j4+Rpp/e/Ydz39Jz1LfcFJlOpvJ/T+y8uq9fBPwRMEJEEoCnwusX9KVXmbT50iobVK9odwzKBD95PhTNprH/vM6f2m5udQ8Bzz5BYqSrN3nTdlTPns7TAG2PW5s+vxxpj+hhj3O8Oukq5kZSDR/n83bvpuSfe7iiWadjvenZWq8PulRuc2u+q10blXU371Av4B7nHim+9klUpD3JgwVIaJu0hMiLY7iiWES8vFkyYyVOxfdmUeMopfZ7cf5jod15ha1Qjmj8+zCl9loYiF3gR6SAiQ/P/HiEiUdbFUkqVxOn8PeCrd3LfPeCL4tY2dQj082byT0ud0t/IPw/ySdtb8fviM7x8vJ3SZ2koUoEXkReBp4Bn8h/yBb6xKpRSqmS8EhI4Vj6EsHqesQdNYSoG+vL28WU899D1JK7dbGlfy3YmM3HNYbyefIKoLu6120pRR/A3AzeSt5YdY0wi4Lm/AyrlpkJ3bCbxivqIeM4WBYVpOfx2DML+f79mWR+Zp1Lx796N/ofXMOKaaMv6sUpRC3yWybtflgEo6IpUpZS9snMdrAmrzeH219gdxSmqxESztuP1xM76geQdeyzpY91t99B01zruuCaGAD/3mZo5q6gFfrKIfAqEiMg9wO+Ac9coKaUuaVdSGk9eN4I0D9kDviiqv/kK3o5cdt3/RKm3vXb0eNrMnMiyPoNpfOctpd6+MxSpwBtj3iHvStQfgCuBF4wxH1oZTClVPJv3JYExNKzmmlvXWiGydSyrbxhI9JLZbN2yr9TaTVyziSueeJBd1evS7JuPS61dZyvqSdbywDxjzJPkjdwDRMTX0mRKqWKp+uZrLBs9hDphnrEHfFE1GP0OfR76gleWJJI3k/z3ZGTl8vtz7+LtcOD741TKlXePNe8FKeoUzSKgnIjUIG96ZigwzqpQSqniK79lI6kVw/D1tfo2D64lpFoEd9/cij+3JzFr0ty/1VZOroNHJ63lxdg+bJixgJqtY0sppT2KWuDFGJMO3AJ8aIy5GfCszaaVcmPGGKrv3cbxeg3tjmKLQW2u4MM139Fh8E0cWJFQojaMw8GiW+9h+8KV/Lt3I9pd06KUUzpfkQt8/u32bgdm5D9WtoYJSrmwYzv2EpZ2EkdcnN1RbOHlJTR/63lyvL3JuekmUpOLtytKzpksVlzXn2t+GstLjm3c3cEzruMsaoF/mLy93KcaYzbmX8U6z7pYSqniSFy4DIDg1u4/6iyp6k0bsv/jL6l5ZB9723cr8m6TacdOsv6qa2kz53uW9h9Gh8/fsTip8xS1wKcDDuC2/J0hpwFdLEullCqWjd4V+bT1LUR2dq8rLUtbkyF9WfOfD2iwfS0JHXuReDLjksevX7KGk/VjiFuzmOWPv8xVEz9FvDxni66iTrNMAJ4ANpBX6JVSLuRP/6qsvvkB7q2ud0Vr+cyDrClfno9WH2fNe4t4tHklboqtRlhUTSDvnrUb/1jLmEQvfluzj09r1OfUx2No07+3zclLX1ELfJIx5hdLkyilSixz9Vpi69ezO4bLaDZiKC8np/Hq9E2cfus9Kv05kYOhVcny8yfi+BGuAJY89i33XHMlrV6ZQ7C/Z676lqKsGxWRrsBtwFzgzNnHjTFTSzNMy5YtTXy85+5jrZQVMk+l4htSkZW33UvbCe57UY5Vdi9ZxeEvxuO3dQteWWfIqlIV7w7tafDAUIIquP+uKyKyyhjTsqDnijqCHwo0IG8XybNTNAa4ZIEXkT3AaSAXyCkshFKq5PYtWkF948Cvhd7yuCBRHVoQ1aFsnnwuaoGPM8Y0KWEfXYwxySV8rVLqMk4uXQlA5avb2JxEuZqini5eJiJ6YZNSLsisXUeaXwDVm+m3qPqrohb4DsBaEdkqIgkisj5/ueTlGGC2iKwSkQLvcyUiw0QkXkTik5KSippbKZUveNsm9tWMdqs7DSnnKOoUzXUlbL+9MSZRRCoDc0RkizFm0fkHGGPGAGMg7yRrCftRqkwyxjCywx10jqpI2dykQF1KkQq8MWZvSRrPv/MTxpijIvIj0Jq8jcuUUqXgwIkMFlVpQI8eje2OolyQZZdsiUh5EQk++3egO3kXSimlSsmeJavotn05jSr52R1FuSArr8mtAiwRkXXACmCGMeY3C/tTqszxmTKZMVNfo36E++5Zrqxj2Y6QxphdQNnc2k4pJ/HftIGDEZHUrFTR7ijKBXnOrjpKlUFVdm/haJ0GdsdQLkoLvFJuKuVQEtVPHCa7cUmvQVSeTgu8Um7qwLw/AQhs28rmJMpVaYFXyk0tqxFD17tHU613d7ujKBelBV4pN7X+cCppdaKJqBpqdxTlorTAK+Wmmo/9gP7HN9odQ7kwvXG2Um4o7XgKg+Z8zfJqFeyOolyYjuCVckP75i/FC0NAaz3BqgqnBV4pN5Tyx3IAql/TzuYkypVpgVfKDXmvWc2x8iFEXBlldxTlwrTAK+WGzJGjHIxqiHjpt7AqnJ5kVcrNZGbncluf53mgXU1i7Q6jXJr++FfKzWw9fJpchyGmdrjdUZSL0wKvlJs5/dV4xkx9jcbB+u2rLk2naJRyM76LFtFm/wYq1IywO4pycToEUMrNhG7dwP6oBnqCVV2W/g9Ryo1kZZzhioM7SI3R06vq8iwv8CLiLSJrRGS61X0p5en2LVyGX24Ovq1b2h1FuQFnzME/DGwGdNMMpf6mHfuOkVK9AdW6d7Y7inIDlo7gRSQSuB743Mp+lCor5oXW5Z77RlEtTm/Tpy7P6ima94F/AY7CDhCRYSISLyLxSUlJFsdRyr2t33ucuMiKiIjdUZQbsKzAi0hv4KgxZtWljjPGjDHGtDTGtIyI0GVfShUmNfkE3z/Ti9vXz7Y7inITVo7g2wM3isgeYCJwjYh8Y2F/Snm0vXMWUz47k4gr69gdRbkJywq8MeYZY0ykMaY2MACYZ4wZZFV/Snm604vybrJds3tHm5Mod6Hr4JVyE36rV5EYWo3QqEi7oyg34ZQCb4xZYIzp7Yy+lPJU1bat51B0Y7tjKDeie9Eo5QaSUjIYH9uDpj30Dk6q6HSKRik3kJB4io+v6kfIbf+wO4pyI1rglXIDe5etpVLmaRrX0AvCVdFpgVfKDbR++99M+eEFAv10VlUVnRZ4pVxczpksonZu5FhsC7ujKDejBV4pF7dnwTLKZ2Xg3aGD3VGUm9ECr5SLS569AIAa13e1N4hyO1rglXJxPsuWcrRCOFUb17c7inIzesZGKRf33zb9aXf1DTykt+hTxaT/Y5RyYYdSMljqF0H53j3tjqLckBZ4pVzY9hkLuGXDXFpWCbA7inJDWuCVcmF+303gtdkf07BmJbujKDekBV4pFxa2Lp7dUY3w9S9ndxTlhrTAK+WiUpNPEHVgO6eat7I7inJTWuCVclE7f/gVH+MgqMe1dkdRbkoLvFIu6sSfK8jy9qFen+52R1FuStfBK+Wi/tvyVsY37s4XIcF2R1FuyrIRvIj4i8gKEVknIhtF5GWr+lLK06RkZLMxMYVGTevZHUW5MSunaM4A1xhj4oCmwHUi0tbC/pTyGDsm/Mj/pr5Ox6Bsu6MoN2bZFI0xxgCp+Z/65n8Yq/pTypNk/zKda3bFQ5Mou6MoN2bpSVYR8RaRtcBRYI4xZnkBxwwTkXgRiU9KSrIyjlJuIyJ+KTvqNcE/KNDuKMqNWVrgjTG5xpimQCTQWkQuuiW8MWaMMaalMaZlRESElXGUcgsn9x2ibuIOTre92u4oys05ZZmkMeYksAC4zhn9KeXOdk+dCUCl63V5pPp7rFxFEyEiIfl/DwC6AVus6k8pT7Hh0GnWRjakTq8udkdRbs7KdfDVgK9ExJu8HySTjTHTLexPKbdnjGF0SBOavP41nwb62x1HuTkrV9EkAM2sal8pT7Rj92GSj52mc9dou6MoD6BbFSjlQo69M4rVHw6kc4S33VGUB9ACr5QLCZo/h+TQKlSLqmF3FOUBdC8a5bKSd+xh79RfyV63Ad/dO0k1Xrw76Dn8vIVBc7+hsncu/u3aUm/AjQRXDrM77t+WmnyC+tvWsbrPHdS2O4zyCFrglUvZcfQ0P61JpOFLT3D9sumEA7nixeFKVdhdtzEVA3zJysklZNsmWm5Ygu+kMZx5zJc1Tdvj9+TjxPS7HhGx+22UyLYJP9LckUNQnxvsjqI8hBZ4ZbvMU6mse+N/VPxuPP1vep7TgcE8HNeWZXExhN1wHVd0uYoagf7UADqcfdGwBWSeSmX7zPmc+nYy9ef9wsyRH/Pi8VBeuCGG2MgQ+95QCTm+n8rJgGCu/Ecvu6MoD6EFXtnmxN5EtrzwJlf+8BVt0lLYWSOaF5tXpMMtXagcfP1lX+9fIYiYATfAgBvIPJWK79Id7I1P5sXnxvJ08kpiJ4whwE222s3KcfB2w+u4vlUn7tTb86lSoidZldPtSU5j5LiFlIuuw1Vfj2JfvSZs/PZn6uzbwi2De1I5uPjrv/0rBNG/R1PmPt6JYX5HaTNzIkcaxrJn4QoL3kHpW7brGMtDo6j2zzvtjqI8iI7gldNsmTqLhKlzeKpmF3y9vGg08CFi7+pH046ld8/RCv6+9PxsJOvbt6DaQ/cS0L0zCR+NJfafA0qtDysc+mQsnZKFq6N1Nw9VeiRvV1/X0LJlSxMfH293DFWKcrNzSBg9noAP3qPBrvUcD6zI19/OY+A1MSUaqRfH0c07OX1tT65I3Mn8j7/j2vv6WdpfSTlycjleqTL7YprTfPkcu+MoNyMiq4wxLQt6TqdolCXSzuQw89MfOFTtCpo9fBcVTiSx7JEXKbd/L4/c1Nzy4g5QuWFdqqxdzrSb/sl9u/z5ZOFOy/ssiU3fTSM89Tim7y12R1EeRqdoVKk6smk70xZv5cNDvgQeTWVsSDirnnqeuBF3Ub2cn9PzBIVX4sbvRzN/8jq+mPInDX6ZSOd3nnN6jktJH/sVqX6BNLrvDrujKA+jBV79bbnZOWz4YiKOzz6nyZrFNLgijqtf+5y7hrYm5uPBdsfDx9uL9/rFMX/ih3Se9iXL0lNp+/EbdscCIOPkaWL+nMOmdtfSukKQ3XGUh9ECr0psd3IaO19+m9ivPyLuVDLHyoewsu9Q6j73OB81bWh3vL/w8faiy+RPWNX5IG1Hj2Rl5QhavfSY3bFYPmspjXz8CLx7iN1RlAfSAq+K5eDqTez9dBzv1OvG6mPZDNu4n8q16nHwrjdofN8grgpw3S1ufcr50WTuz6xv1YlmrzzJusoRxA23d1rkq/QQtj8ziUUDutqaQ3kmLfDqkrIzz7D9p9mk/PgLlf9YQN2D26kBxA6vSq8Bfej11IdUr+Q+9w31C/QnauFv7G7RDp9XXyH+up60rBNuS5aDB5JYvOUww7rUx8tHd49UpU+XSaq/OJOWzu5Zi1mfksts78okLV3FT5/cR7aXN9vrNuFU1x7Uuncw1V1sCqa4ju8+wNCvV7GHAH64vx31Kjt//nvpgHupPmsaPps2UqNaqNP7V57hUsskLRvBi0hN4GugKuAAxhhjPrCqP1V8mdm5bDtymswxn2NWr6HClg3U2bOZBrnZrGtyLdvueIZ23a5iVaOviP5HL2Kq2DPStUJoVCQfPhhK/w/ns+qWIYR8+Tbh0VFO6z/zdBoNfpnEngbNaKbFXVnEyimaHOBxY8xqEQkGVonIHGPMJgv7VBfIzM4lKWELx1cnkLl5G2bHDvz37WFfYCiPdBqGw8D8Me9SJfUE+yLrseaGgZTrdDVdb7iW/nVr5bcSa+t7sEqtsEDGdahEzddnkHhNAv5rlhMUXskpfa9/9zNapadw8OGHnNKfKpucNkUjIj8D/zPGFHqpnk7RXF7m6TROHzpKatIJkmvU5mR6Nn6LFlBu0wbM0aP4HDlMuaQjZDiEewe8zMn0bL797lna7UsAIM3XnyMRNdjfoCmrnvoPDatVoJF3BpH1ryiz88DrPvuORvcOYlOj1jRcuQBfizf7Mg4Hu2peiU9uNrUSdyFeer2hKjlbpmguCFCbvPuzLi/guWHAMIBatWpd+HSRpJzKwJw8iTEG43DgcDjAOHAEV8ThH4DJzECSkjHGgcl1gDGY3FxyIqrgKF8ec/oU3omJeY878trAOMisWRtHYCBex0/gs39v/vOOcx8ZV8aQGxCIz5FD+OzYgSMrC5OVfe7PxKs6caZcIEFbNhCybhVkZ2NycvL+zM4mvs9gMv3KUfOPeUStWIBXejremRl4Z2bgk5nJcw+8S1qucPuMz7lx+S8EnsnAPycLf6CCty9dnvgRgP/OGEPfDfPIES+OVQgjJSScjOq16B1bjSrB/mRGv8TmCuWIaBZDWJ1a1PHyog7QqWRfTo8Td89trNx3kFavPcmK6/rRat6PlhbdVZ9PomXiDuJffo8rtLgrC1le4EUkCPgBeMQYc+rC540xY4AxkDeCL0kf/3zkU6Z8+ehFjz90w5P8EtOJdnvW8u2kf1/0/NBbX2R+3VZ037aUMT/+56Ln+w0cyYqajemzcT7vT//vRc9fP+QDNlapy6A1M3lt9scXPf/YPZ+yJ7QG9y7/nmcWjLvo+REBzTleIZQHl63k2hVzOOPnz5ly/mT7BZBdzp/K5bzwDiqPV+NGbPc9gykfBJUqIaGh+ISH8vVNrQgp70elu5uQEuBHcNUIqvh4UyW//TbnetIbOF9Oq1efYOm+fcRMGssX4+fxz8HdLOnHGMOLadWJHfo6rz413JI+lDrL0ikaEfEFpgOzjDHvXu74kk7RTJ6xklpzpoMI4u0FCHgJh1tfTVrNKAKTj1Djz3ng5Y0575jk1u05U7U6AUcTCVu9Im/U5iUgXniJcLzVVWSHhhNw+CAVt2wA8UK8AC9v8BJONW+NCa5AuSOJBO3djXc5P7zK+eHl54t3OT9y69XHN9Af3/Q0/DLS8PLzw7ucH77+fviUK4dPOT+8vHUE5yqMw8Ebn81lzO4sXr+5CQPblOw3ykv5LSGR+75dw7v94rileWSpt6/KnktN0VhW4CXvvmlfAceNMY8U5TU6B6/slp3r4J6vVhLz9Wi69+9K0/tL70KoM6lp7K8fx9Sr+/LYt2/goz/cVSmwazfJ9sAdwDUisjb/Q+9Fplyar7cXH/VtxM27l3PliHvY8tPsUmt7zcPPU+/QTnr2aKHFXTmFXuikVAGSd+7jTOu2VEhL4eC3P9Dglr93I449C1dQrWsHNja/muYr5pZSSqV0P3ilii28bi28Fy7kZHAotQbczMbvppW4rcxTqeTePog0/0CumDSu9EIqdRla4JUqRNXG0QT8sYgjYVX56rtFTFuXWOw2jDF8/fJnXJG4k/3v/I+wqJoWJFWqYLrZmFKXEF4/CtmYwO7J65n83RpOLPiD2+65Ab/Ay++a6XAYXpi2gW9861N+/Gxuv113jFTOpSN4pS4jLDSYCf9sy/BawoDHB7K/fizbZy685GtOHUkmvn1Pdk2cxn2d6jJw4DVOSqvU/9MCr1QR+Pl48a/hvdj03meEnEwi+vrOrGneifVjJ5N1Jvvcccd3H2DZIy+SVa8+zZfP4cGq2Tx13ZXkrRpWyrl0FY1SxXTqSDIbn36NmIlfEJyZRtzj31MxIoQHp33EgD+nArC5bize779H/d46clfWsuVCp5LQAq/cSeapVDZO+ZW5NZpwKCWTNgt+5oqc01TudzN1ul6lm4gpp9ACr5RSHkrXwSulVBmkBV4ppTyUFnillPJQWuCVUspDaYFXSikPpQVeKaU8lBZ4pZTyUFrglVLKQ7nUhU4ikgTsLeHLw4HkUoxTWjRX8Wiu4tFcxeOJua4wxkQU9IRLFfi/Q0TiC7uay06aq3g0V/ForuIpa7l0ikYppTyUFnillPJQnlTgx9gdoBCaq3g0V/ForuIpU7k8Zg5eKaXUX3nSCF4ppdR5tMArpZSH8qgCLyKvikiCiKwVkdkiUt3uTAAi8raIbMnP9qOIhNidCUBE/iEiG0XEISK2Lh0TketEZKuI7BCRp+3Mcj4RGSsiR0Vkg91ZziciNUVkvohszv8aPmx3JgAR8ReRFSKyLj/Xy3ZnOktEvEVkjYhMtzvL+URkj4isz69bpXrHI48q8MDbxphYY0xTYDrwgs15zpoDNDbGxALbgGdsznPWBuAWYJGdIUTEG/gI6AnEALeJSIydmc4zDrjO7hAFyAEeN8Y0BNoCD7jIv9kZ4BpjTBzQFLhORNraG+mch4HNdocoRBdjTNPSXgvvUQXeGHPqvE/LAy5xBtkYM9sYk5P/6TIg0s48ZxljNhtjttqdA2gN7DDG7DLGZAETgZtszgSAMWYRcNzuHBcyxhwyxqzO//tp8gpXDXtTgcmTmv+pb/6H7d+HIhIJXA98bncWZ/KoAg8gIv8Rkf3A7bjOCP58dwG/2h3CxdQA9p/3+QFcoFi5CxGpDTQDltscBTg3FbIWOArMMca4Qq73gX8BDptzFMQAs0VklYgMK82G3a7Ai8jvIrKhgI+bAIwxzxljagITgAddJVf+Mc+R96v1BFfK5QKkgMdsH/W5AxEJAn4AHrngN1jbGGNy86dJI4HWItLYzjwi0hs4aoxZZWeOS2hvjGlO3hTlAyLSsbQa9imthpzFGNOtiId+C8wAXrQwzjmXyyUig4HeQFfjxIsPivHvZacDQM3zPo8EEm3K4jZExJe84j7BGDPV7jwXMsacFJEF5J3DsPMkdXvgRhHpBfgDFUTkG2PMIBsznWOMScz/86iI/EjelGWpnBdzuxH8pYhI9Hmf3ghssSvL+UTkOuAp4EZjTLrdeVzQSiBaRKJExA8YAEyzOZNLExEBvgA2G2PetTvPWSIScXaVmIgEAN2w+fvQGPOMMSbSGFObvP9b81yluItIeREJPvt3oDul+MPQowo8MDJ/+iGBvH8ol1g6BvwPCAbm5C+F+sTuQAAicrOIHACuAmaIyCw7cuSfgH4QmEXeycLJxpiNdmS5kIh8BywFrhSRAyJyt92Z8rUH7gCuyf8/tTZ/hGq3asD8/O/BleTNwbvUskQXUwVYIiLrgBXADGPMb6XVuG5VoJRSHsrTRvBKKaXyaYFXSikPpQVeKaU8lBZ4pZTyUFrglVLKQ2mBVy5JRFIvf1Shr30wf1dKIyLh5z0uIjIq/7kEEWl+3nMBIrIw/zL72qW1e6SIjBORW4tw3ILL7egpIhMvuNZDqUvSAq880R/kXWCz94LHewLR+R/DgNHnPXcXMNUYk+uUhCUzmrz9VJQqEi3wyqXlj7rfzr+Abb2I9M9/3EtEPs7fc3y6iMw8O1I2xqwxxuwpoLmbgK/zdzxcBoSISLX8524Hfi6g/9oislhEVud/tMt/vHP+iH+yiGwTkZEicrvk7YW+XkTqntdMt/w2tuXvi3L2N4aJ+b9JTAICzutztIjEy8X7qS/Ob8vtthhR9tD/KMrV3ULevuJxQDiwUkQWkXclZ22gCVCZvCtgx16mrQJ3rRSRY0CdQn4oHAWuNcZk5k+PfAecnUqJAxqSt53wLuBzY0xrybv5xkPAI/nH1QY6AXXJu8qzHnA/kG6MiRWRWGD1eX0+Z4w5Lnn75M8VkVhjTIIxxiEiO/L7ddWNs5QL0RG8cnUdgO/ydyg8AiwEWuU/PsUY4zDGHAbmF6GtwnatDAdOFvIaX+AzEVkPTCHvhiRnrczfl/0MsBOYnf/4evKK+lmT83NuJ+8HQQOgI/ANgDEmAUg47/h+IrIaWAM0uqDPo4BL3KlMuT4dwStXV1BRvtTjl1LYrpWZ5O0yWJBHgSPkjZq98o8968x5f3ec97mDv35vXbgfiCnkcUQkCngCaGWMOSEi4y7I5g9kFJJVqb/QEbxydYuA/vmrWyLIG/muAJYAffPn4qsAnYvQ1jTgzvx5/bZASv4I/ATgLSIFFfmKwCFjjIO8zb28S/Ae/pGfsy5QB9ia/75uB5C8/dJj84+tAKQBKfnvq+cFbdUHXGIjNuX6tMArV/cjedMX64B5wL/yp2R+IG9EvgH4lLy7GaUAiMiI/F0yI4EEETl7m7aZ5E2R7AA+A4af189s8qZ9LvQxMFhElpFXXNNK8B62kje19CtwnzEmk7wVMUH5uy7+i7wfWhhj1pE3NbORvHMKf5xtJL/gZxhjDpUggyqDdDdJ5bZEJMgYkyoiYeQVyPb5xb8kbTUDHjPG3FGqIUuRiDwKnDLGfGF3FuUedA5eubPp+TeX8ANeLWlxh7yllSIyX0S8XXgt/ElgvN0hlPvQEbxSSnkonYNXSikPpQVeKaU8lBZ4pZTyUFrglVLKQ2mBV0opD/V/11dakQaqEecAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_71_0.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from sklearn.model_selection import KFold\n", "from sklearn.linear_model import Ridge\n", "from sklearn.model_selection import cross_val_score\n", "from sklearn.preprocessing import PolynomialFeatures\n", "\n", "# A seed just to ensure that the random numbers are the same for every run.\n", "# Useful for eventual debugging.\n", "np.random.seed(3155)\n", "\n", "# Generate the data.\n", "nsamples = 100\n", "x = np.random.randn(nsamples)\n", "y = 3*x**2 + np.random.randn(nsamples)\n", "\n", "## Cross-validation on Ridge regression using KFold only\n", "\n", "# Decide degree on polynomial to fit\n", "poly = PolynomialFeatures(degree = 6)\n", "\n", "# Decide which values of lambda to use\n", "nlambdas = 500\n", "lambdas = np.logspace(-3, 5, nlambdas)\n", "\n", "# Initialize a KFold instance\n", "k = 5\n", "kfold = KFold(n_splits = k)\n", "\n", "# Perform the cross-validation to estimate MSE\n", "scores_KFold = np.zeros((nlambdas, k))\n", "\n", "i = 0\n", "for lmb in lambdas:\n", " ridge = Ridge(alpha = lmb)\n", " j = 0\n", " for train_inds, test_inds in kfold.split(x):\n", " xtrain = x[train_inds]\n", " ytrain = y[train_inds]\n", "\n", " xtest = x[test_inds]\n", " ytest = y[test_inds]\n", "\n", " Xtrain = poly.fit_transform(xtrain[:, np.newaxis])\n", " ridge.fit(Xtrain, ytrain[:, np.newaxis])\n", "\n", " Xtest = poly.fit_transform(xtest[:, np.newaxis])\n", " ypred = ridge.predict(Xtest)\n", "\n", " scores_KFold[i,j] = np.sum((ypred - ytest[:, np.newaxis])**2)/np.size(ypred)\n", "\n", " j += 1\n", " i += 1\n", "\n", "\n", "estimated_mse_KFold = np.mean(scores_KFold, axis = 1)\n", "\n", "## Cross-validation using cross_val_score from sklearn along with KFold\n", "\n", "# kfold is an instance initialized above as:\n", "# kfold = KFold(n_splits = k)\n", "\n", "estimated_mse_sklearn = np.zeros(nlambdas)\n", "i = 0\n", "for lmb in lambdas:\n", " ridge = Ridge(alpha = lmb)\n", "\n", " X = poly.fit_transform(x[:, np.newaxis])\n", " estimated_mse_folds = cross_val_score(ridge, X, y[:, np.newaxis], scoring='neg_mean_squared_error', cv=kfold)\n", "\n", " # cross_val_score return an array containing the estimated negative mse for every fold.\n", " # we have to the the mean of every array in order to get an estimate of the mse of the model\n", " estimated_mse_sklearn[i] = np.mean(-estimated_mse_folds)\n", "\n", " i += 1\n", "\n", "## Plot and compare the slightly different ways to perform cross-validation\n", "\n", "plt.figure()\n", "\n", "plt.plot(np.log10(lambdas), estimated_mse_sklearn, label = 'cross_val_score')\n", "plt.plot(np.log10(lambdas), estimated_mse_KFold, 'r--', label = 'KFold')\n", "\n", "plt.xlabel('log10(lambda)')\n", "plt.ylabel('mse')\n", "\n", "plt.legend()\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "More examples of the application of cross-validation follow here." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10\n", " plt.plot(polynomial, np.log10(estimated_mse_sklearn), label='Test Error')\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEGCAYAAABsLkJ6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAusUlEQVR4nO3dd3xUZfb48c9JJyEkEEKAVJAOoUak2BG7WFEsq7C6rmtb1911i/tbdf2ua11d+1oQdS0I2HsQECkqCSWBAKEFkgBJCKSTOs/vjxkwRJKZJDNzM8l5v168MnPnztxzc8k9c5/nuecRYwxKKaW6Hj+rA1BKKWUNTQBKKdVFaQJQSqkuShOAUkp1UZoAlFKqiwqwOgBX9O7d2yQlJVkdhlJK+ZT09PQDxpjo5l73iQSQlJREWlqa1WEopZRPEZHdLb2uTUBKKdVFaQJQSqkuShOAUkp1UT7RB3A8dXV15OXlUV1dbXUoPi0kJIS4uDgCAwOtDkUp5WU+mwDy8vIIDw8nKSkJEbE6HJ9kjKG4uJi8vDwGDBhgdThKKS/z2Sag6upqoqKi9OTfDiJCVFSUXkUp1UX5bAIA9OTvBvo7VKrr8ukEoJRSndWhylru/3gT5dV1HtuGz/YBWK24uJhp06YBsH//fvz9/YmOtt9w9+OPPxIUFNTi+5ctW0ZQUBBTpkz52Wvz5s3jj3/8I7GxsUeXvf3224wYMcKNe6CU6oiMMXyeuZ/7Pt5ISVUdJw/qzVkjYjyyLU0AbRQVFcX69esBuP/+++nevTt/+MMfXH7/smXL6N69+3ETAMBVV13Fs88+2+z7Gxoa8Pf3b/Z5c+rr6wkI0MOuVEdUWFbN3z7cyNdZBSTHRvDGL09iRP8eHtueNgG5UXp6OqeddhoTJkzgnHPOYd++fQA8/fTTjBgxgtGjRzNr1ixycnJ48cUXefLJJxk7dizfffedS5+/bNkyzjjjDK655hqSk5N/9ry6upo5c+aQnJzMuHHjWLp0KWC/opg5cyYXXXQRZ599tsf2XynVNsYY3kvL5ax/f8u32UX8+bxhfHDrFI+e/KGTXAE88MkmsvaWufUzR/TvwX0XjXR5fWMMd9xxBx999BHR0dHMnz+fe++9l7lz5/Lwww+za9cugoODKSkpITIykltuuaXFq4b58+ezYsWKo89Xr14N2JuXNm7cyIABA1i2bNkxz5944gkAMjMz2bJlC2effTbZ2dlH35+RkUGvXr3a+itRSnlA7sEq/vpBJt9tO8DEpF48fHkyA6O7e2XbnSIBdAQ1NTVs3LiR6dOnA/YmmX79+gEwevRorr32Wi655BIuueQSlz6vuSagiRMnHjNmv/HzFStWcMcddwAwbNgwEhMTjyaA6dOn68lfqQ7EZjO8sTqHR7/aigAPXjySa09KxM/PeyPzOkUCaM03dU8xxjBy5Mij39Qb++yzz1i+fDkff/wxDz74IJs2bWrzdsLCwpp9boxx+X1KKetsL6zgz4sySNt9iNOGRPPQZcnERnbzehzaB+AmwcHBFBUVHU0AdXV1bNq0CZvNRm5uLmeccQaPPvooJSUlVFRUEB4eTnl5uVtjOPXUU3nrrbcAyM7OZs+ePQwdOtSt21BKtZ0xhhe/3cH5T3/HtsIKnpg5hnlzTrTk5A+aANzGz8+PhQsX8qc//YkxY8YwduxYVq1aRUNDA9ddd93Rjtnf/e53REZGctFFF/HBBx802wk8f/58xo4de/TfqlWrnMZw66230tDQQHJyMldddRXz5s0jODjYE7urlGqDRWvzefiLLZw5tA+L7z6NyyfEWXozprTUbNBRpKSkmKYTwmzevJnhw4dbFFHnor9LpTwvv+Qw5z65nOH9e/DuryZ5pa1fRNKNMSnNva5XAEop5WE2m+GehRtoMIbHrxjj1Y7elmgCUEopD/vfD7tZub2Yv10wgoSoUKvDOcqnE4AvNF91dPo7VMqzdh2o5KHPN3P60GiunhhvdTjH8NkEEBISQnFxsZ7A2uHIfAAhISFWh6JUp9RgM/z+vfUEB/jzyOWjO1z1XZ+9DyAuLo68vDyKioqsDsWnHZkRTCnlfv9dvoO1e0r4z6yxxPToeF+0fDYBBAYG6ixWSqkOa8v+Mp5Mzeb85L7MGNPf6nCOy2ebgJRSqqOqrbfxu/kbiOgWxP9dktzhmn6O8NkrAKWU6qie/mYbm/eV8fL1KfQKa3luECvpFYBSbdRgM2zd795yHsr3rdtziOeXbeeKCXFM99BELu6iCUCpNnp3zR7OeWo5936QSW29zepwVAdwuLaB37+3gX4R3fj7RR1/Bj9NAEq1UXrOIQL9hbd+2MM1L39PYXm11SEpiz361RZ2HqjksStG0yMk0OpwnNIEoFQbZeSXctqQaJ65ehyb9pYx45mVrM8tsTosZZFVOw7w2socZk9JYsqg3laH4xKPJQARmSsihSKy8Tiv/UFEjIj4xm9JqSYqaurZUVRBcmwkF43pz6LfTCHAX7jyv6t5Ly3X6vC8pqy6joe/2ELuwSqrQ7FUeXUdf1yQwYDeYfzp3GFWh+MyT44Cmgc8C7zReKGIxAPTgT0e3LZSHrUxvxRjYHRcBGCfQvST20/m9nfWcs/CDDbll/K3C0cQ6N95L7KNMfzhvQ18nVXAki0FLPrNFMI7aLOHMYZDVXXkFFdyoLyGqtoGKmvrqaqx/6ysqaeytoGqIz9r66msacAYQ1CAn/2fv+NngP/Rx8GO17L2lrGv9DALfzOFbkH+Vu+uyzyWAIwxy0Uk6TgvPQncA3zkqW0r5WmZeaUAjIqNOLqsZ1gQr8+ZyCNfbuHl73axeX85z187nt7dO+ecDK98t4uvswq4bHwsH63fy13vruel61Pwt6jSpTGGwvIacg5Usru4it0HK8kprmJ3sf15eXV9s+/tFuhPWLA/oUEBhAb5ExYcQHhIACJCbX0D1XU2yg7XU1tvo7bBRm29jZp6G3WOxw02wx/OGcr4hJ5e3OP28+p9ACIyA8g3xmxwdmOEiNwM3AyQkJDgheiUcl1Gfin9I0KIDj/25B7g78e9F4xgVGwE9yzMYMYzK/jvL1JIjoto5pN805qcgzz85RbOG9WXJ2aOYVx8JP/vo008+tUW/nKed+eWeC8tl7krdpFTXEl13U+jsQL8hLie3UiICmN8Qk8So8JI7BVKTI8QwoLtJ/nQIPtJ36qkZTWvJQARCQXuBc52ZX1jzEvAS2CfEMaDoSnVapl5JYyOi2z29YvHxnJCdHd+/WY6V7y4in9dlsxl4ztHzaWi8hpue2stCb1CefQKe4GzX0xOYmtBOf/9didD+oRz+QTv7Osbq3P4+0ebGBMXwXUnJZIYFUpiVBhJUWH0jwwhoBM3wbmDN68ATgAGAEe+/ccBa0VkojFmvxfjUKpdSqvqyCmuYmZKy6V9R8VG8PHtU7n97XXc/d4GNuaX8dfzh/n0SanBZvjtu+soPVzH67+ceEyb/30XjWRHYSV/eT+TpN5hTEj0bHPIvJW7uP+TLKaPiOG5a8YTFOC7v1ereO03ZozJNMb0McYkGWOSgDxgvJ78la/JzLe3/492oVknqnswb944kTlTk5i7chezX1tDSVWt22IpqaqlsMx79x88mZrNqh3F/N8loxjer8cxrwX6+/H8tePpFxnCr99MI7/ksMfimLvCfvI/Z6Se/NvDk8NA3wFWA0NFJE9EbvTUtpTypoz8EgCSY11r1w/w9+O+i0by6BWj+XHXQS5+biXZBe0rIWGM4b01uUx9eAkTH/qGi59dwXNLt7OtoNxjc2Qs2VLAs0u3c1VKfLNXPz3Dgnj1hhRq6mzc9HoalTXNd7y21Svf7eQfn2Zx3qi+PKsn/3bx2UnhlbLKb/6Xzqa9ZSy/54xWvzd99yFu+V86VTX1PDVrXJtqxRyoqOEv72eSmlXApIG9OHlQb1I3F7LBcRPagN5hTB8Rw9kjYhiX0NMtHZx5h6q44OkV9I/sxge3TiEksOWhjku3FnLjvDWcPaIvz1873m1z4L68fCf//Hwz5yf35T+zxnXqYbbu4GxSeE0ASrXS1IeXMDYhkueuGd+m9+8rPcyv30wnM7+U308fwm1nDHK5XHBqVgF/eT+Dsup67jlnKL+cOuDoyXV/aTWpmwtIzSpg9Y4D1DUYencP4qzhMUwfEcPUQb2dnriPp6a+gStfXM3Ooko+ueNkknqHufS+V77byf99tpk7zxzE3WcPbfV2m/rvtzv41xdbuGB0P566aqye/F3gLAFoOWilWqG4oob8ksPcMCWxzZ/RL6Ib7/16Mn9elMHjX2ezeX85j10xmtCg5v8cK2rqefCTLOan5TK8Xw/eumksQ/uGH7NO34gQfjEpkV9MSqSsuo5lW4v4etN+Ps3Yx7trcgkN8ueckX259qQEJiT2dDnp/POzzWzIK+XF6ya4fPIHuPHkAWQXlPP0ku0MjgnnonZMivL8su08+uVWLhrTnyevHOPTHekdiSYApVrhSAdwcmxkuz4nJNCfJ68ay/B+PXj4yy3sKqrk5RtSiI3s9rN11+Qc5O731pN/6DC3nn4Cd501xGm7d4+QQGaM6c+MMf2pqW/g+50H+XLjfj7ZsJcP1uUzJKY710xM4NLxcUR0a/7u3Y/W5/PG6t386pQBnDuqb6v2UUR48JJR7DpQyR8WbCAxKrTFobPNeW7pdh77aisXj+3PEzP15O9O2gSkVCs8/c02/p2aTeb9Z7ut7MHSLYXc+c46ggL8eOG6CUwc0AuwN708mbqN/y7fQXzPUP595RhSknq1a1tVtfV8smEvb/2wh4y8UkIC/bhwdH+uOSmBcfGRx1wVbC8sZ8azKxnRrwfv3DypzU0uBypquPjZldTbbHx028n0jXB9btxnvtnGE6nZXDoulsdnjumyN2y1lfYBKOVGN72exs4DFSz5/elu/dzthRXc/EYauYeqeGDGKCYk9uSu+evZvK+MqyfGc+8FI+ge7N4L9o35pbz1wx4+Wp9PVW0Dw/qGc+2kRC4Z2x8/ES5+biWHKmv57M5TWnXSPp4t+8u4/PlVJESFcd6ovoQFBxDmKLnQPTjg6F25Rx53Dw7gpeU7eXJxNpeNj+WxK/Tk3xaaAJRyo5MeWszkgVE8NWuc2z+79HAdd76zjm+zi/D3E3qGBvLI5aOZNtyzs0qVV9fx0fq9vP3DHrL2lREa5E9Cr1C2FpTzvxtPYqqbShsvzirg9ws2UHq4zuX3XDEhjkcuH60n/zbSTmCl3KSgrJqCshqS29CO7YqIboHMnX0iTy3OJr/kMPeeP5woLxSSCw8J5LpJiVx7UgIb8kp5+4fdfJqxj3vOGea2kz/AWSNi2HDf2dQ12I6pwllRU09VbQMVNT9V5aysqSeyWyBXpsS7bQip+jlNAEq56EgF0DEeLOzm7yf83g1DJttCRBgbH8nY+EgeuXy0y6OEWivQ34+IUD8iQjtm6eiuRLvTlXJRRn4pfmKv/d/ZeerkrzoWTQBKuSgjr4TBfcJbHK+vlC/RBKCUC4wxZOaVdrq6/qpr0wSglAv2llZTXFnrUgVQpXyFJgClXJCZVwK4XgFUKV+gCUApF2TklRLgJz+rga+UL9MEoJQLMvNLGdo3vE3VNJXqqDQBKOWEMYaMvFJt/1edjiYApZzYc7CK0sN17a4AqlRHowlAKScy8lyfA1gpX6IJQCknMvNLCfL3Y0hMuPOVlfIhmgCUciIjr4Th/Xvo5OOq09H/0Uq1wGYzbMwvY7SO/1edkCYApVqw80AlFTX1WgJCdUqaAJRqQWZ+CaAdwKpz0gSgVAuOzJs7KLq71aEo5XaaAJRqQWZeKSP7RxDQxgnRlerI9H+1Us2ob7CxaW+ZNv+oTksTgFLN2FFUyeG6Bk0AqtPyWAIQkbkiUigiGxste0xEtohIhoh8ICKRntq+Uu214WgJ6EhL41DKUzx5BTAPOLfJslRglDFmNJAN/MWD21eqXTLzSgkL8mdg7zCrQ1HKIzyWAIwxy4GDTZZ9bYypdzz9Hojz1PaVaq+M/FJGxUbg56cTpKvOyco+gF8CXzT3oojcLCJpIpJWVFTkxbCUgtp6G5v3lTEmPtLqUJTyGEsSgIjcC9QDbzW3jjHmJWNMijEmJTo62nvBKQVkF5RTW2/TKSBVpxbg7Q2KyA3AhcA0Y4zx9vaVckVmvpaAVp2fVxOAiJwL/Ak4zRhT5c1tK9UaGXkl9AgJIKFXqNWhKOUxnhwG+g6wGhgqInkiciPwLBAOpIrIehF50VPbV6o97FNARiKiHcCq8/LYFYAx5urjLH7VU9tTqiUbcksY1Kc7YcHO/8tX1zWwdX85vzp1oBciU8o6Xu8DUMrbftx1kCv/u5qeoYHcdMpArp+cSHhIYLPrb9lfTr3NMEbb/1Unp6UgVKf32spdRHQLZGx8JI99tZWTH1nK099so/Rw3XHXzzxyB3BcpPeCVMoCmgBUp5Z3qIqvNu3n6okJvDZnIh/fPpUTk3rx79RsTn5kCf9Ozaa06thEkJFXSlRYEP0jQiyKWinv0ASgOrU3V+9GRLh+ciIAo+MieeWGFD6942SmnBDF099sY+ojS3j8q60cqqwF7AkgOS5CO4BVp6d9AKrTqqqt550f93DuyL70j+x2zGujYiP47y9S2LyvjGeXbOe5Zdt5beUurp2UyLbCcs4ZGWNR1Ep5jyYA1Wl9sC6fsup6Zk9Nanad4f168Ny148kuKOeZJdt5+budGKPt/6pr0ASgOiVjDPNW5jAqtgcpiT2drj8kJpxnrh7Hb6cN5tvsIk4fquVHVOfXYgIQkfEufEadMSbTTfEo5RYrth9gW2EFT8wc06q2/EF9ujOoj87/q7oGZ1cA3wJrgJb+ggYASe4KSCl3mLcyh97dg7hwTD+rQ1Gqw3KWANYYY85saQURWeLGeJRqt5wDlSzZWsgdZw4mOMDf6nCU6rBaHAbq7OTv6jpKedO8VTkE+AnXTUqwOhSlOrQWE4CIXNfo8dQmr93uqaCUaqvy6joWpudx4ej+9AnXG7mUaomzG8HubvT4mSav/dLNsSjVbgvT86ioqWf2lCSrQ1Gqw3OWAKSZx8d7rpSlbDbD66tyGJ8QqVM5KuUCZwnANPP4eM+VstTSrYXkFFcxZ+oAq0NRyic4GwU0TEQysH/bP8HxGMdzLZauOpTXVubQt0cI547qa3UoSvkEZwlguFeiUKqdthWUs2L7Af54zlAC/bXGoVKuaDEBGGN2N34uIlHAqcAeY0y6JwNTqjVeW5VDcIAfV0/UoZ9KucrZMNBPRWSU43E/YCP20T9vishdng9PKedKqmp5f20el4yNpVdYkNXhKOUznF0rDzDGbHQ8ngOkGmMuAk5Ch4GqDuLdNblU19larPqplPo5Zwmg8VRJ04DPAYwx5YDNU0Ep5ar6Bhtvrt7NpIG9GN6vh9XhKOVTnCWAXBG5Q0QuBcYDXwKISDeg+Vm1lfKS1KwC8ksO69BPpdrAWQK4ERgJzAauMsaUOJZPAl7zXFhKuea1lTnE9+rGWcN1Bi+lWsvZKKBC4JbjLF8KLPVUUEq5YmN+KT/mHORvFwzH309vTFeqtZxNCPNxS68bY2a4NxylXDdvVQ6hQf7MTIm3OhSlfJKzG8EmA7nAO8APaP0f1UGUVNXy8Ya9XJUST0Q37Y5Sqi2cJYC+wHTgauAa4DPgHWPMJk8HplRLPt6wl9p6m974pVQ7OJsQpsEY86Ux5gbsHb/bgWUicoezDxaRuSJSKCIbGy3rJSKpIrLN8dP5bN1KHceCtDxG9OvBiP469FOptnJaNEVEgkXkMuB/wG3A08D7Lnz2PODcJsv+DHxjjBkMfON4rlSrbNlfRmZ+KTNT4qwORSmf5qwT+HVgFPAF8ECju4KdMsYsF5GkJosvBk53PH4dWAb8ydXPVApgYVoegf7CxWNjrQ5FKZ/mrA/gF0AlMAS4U+RoH7AAxhjT2uvvGGPMPuxv3icifZpbUURuBm4GSEjQdl5lV9dg48P1+UwbFqN1f5RqJ2f3AVhWV9cY8xLwEkBKSopOPqMAWLa1iAMVtVwxQZt/lGovb5/gCxxVRY9UFy308vaVj1uQlkvv7sGcPjTa6lCU8nnOykGvdfYBrqzTyMfADY7HNwAfteK9qosrrqhhyZZCLhsfS4BO+qJUuzmdEazRNJDHI0DEcV8QeQd7h29vEckD7gMeBt4TkRuBPcDMVkesuqwP1++l3ma0+UcpN3E6J7ALn9FwvIXGmKubWX+aC5+p1M8sTM9jTFwEQ2LCrQ5FqU7BpSkhRSQGiAUMsNcYU+CF2JQ6amN+KZv3lfHgxSOtDkWpTsPZfQBjgRexN/PkOxbHiUgJcKsxpjXt/0q12cL0PIL8/ZgxRsf+K+UuzpqA5gG/Nsb80HihiByZD2CMh+JS6qia+gY+XJ/P9JExRIRq4Tel3MXZUIqwpid/AGPM90CYZ0JS6lhLNhdSUlXHTO38VcqtnF0BfCEinwFvYC8LDRAPXI9jekilPG1heh4xPYI5ZbCO/VfKnZx1At8pIudhr+ETi33YZx7wnDHmcy/Ep7q4wvJqlmUXcfOpA3XWL6XczNkVAMaYL7AXg1PK6z5cl0+Djv1XyiPafDuliLzkzkCUasoYw4K0PMYnRHJCdHerw1Gq03E2DLRXcy8B57s/HKV+kpFXyrbCCv51WbLVoSjVKTlrAioCdnPsXMDG8bzZUs5KucOC9FxCAv24YHQ/q0NRqlNylgB2AtOMMXuaviAiucdZXym3qK5r4OP1ezl3ZF96hOjYf6U8wVkfwFNAc/P2PureUJT6SWpWAWXV9VwxId7qUJTqtJwNA32uhdeecX84StktSM+jf0QIU06IsjoUpTotp8NAARyTwjdVCmQaY3RSF+VW+0urWbGtiNvOGISfjv1XymNcSgDAjcBkYKnj+enA98AQEfmHMeZND8SmuqhFa/OwGXTsv1Ie5moCsAHDj5SBdpSHfgE4CVgOaAJQbmGMYVF6HhMH9CIxSstNKeVJrt4IltRkDoBCYIgx5iBQ5/6wVFe1ds8hdh6o1G//SnmBq1cA34nIp8ACx/MrgOUiEgaUeCIw1TUtSMsjNMifC5J17L9SnuZqArgNuAw4GftNYK8Di4wxBjjDQ7GpLqTBZvjP4mzeXZPLrBPjCQt29b+mUqqtXPorM8YYEVkB1GK/E/hHx8lfqXY7VFnLb+evZ3l2ETMnxHH/DJ32USlvcHUY6JXAY8Ay7FcAz4jIH40xCz0Ym+oCMvNKueV/6RSV1/Cvy5KZdWI8Ijr0UylvcPU6+17gxCNj/kUkGlgMaAJQbTZ/zR7+30eb6B0WxIJbJjMmPtLqkJTqUlxNAH5Nbvgqph2lpFXXVl3XwH0fbWJ+Wi6nDO7Nf2aNo1dYkNVhKdXluJoAvhSRr4B3HM+vAnRGMNVquQeruPWttWTml3L7GYP43fQhOtOXUhZxtRP4jyJyOTAVex/AS8aYDzwamep0vs0u4rfvrqOhwfDy9SlMHxFjdUhKdWkuj7UzxiwCFnkwFtVJ2WyGZ5du58nF2QyNCeeF6yYwoLfe5auU1ZzNCFaOfdjnz17CPjq0R1s2KiK/A25yfHYmMMcYU92Wz1Id3/2fbOKN1bu5dFws/7x0FKFBOsZfqY6gxY5cY0y4MabHcf6Ft+PkHwvcCaQYY0YB/sCstnyW6viq6xpYkJbHZeNi+feVY/Tkr1QHYtVIngCgm4gEAKHAXoviUB62ascBDtc1MGNsfx3fr1QH4/UEYIzJBx4H9gD7gFJjzNdN1xORm0UkTUTSioqKvB2mcpPUrAK6BwcwWSd2UarD8XoCEJGewMXAAKA/ECYi1zVdzxjzkjEmxRiTEh0d7e0wlRvYbIbFmws5bUg0wQH+VoejlGrCiiags4BdxpgiY0wd8D4wxYI4lIdtyCuhqLyGs0b0sToUpdRxWJEA9gCTRCRU7I3C04DNFsShPCw1qwB/P+GMoZoAlOqIrOgD+AF7DaG12IeA+gEveTsO5XmpWQVMTOpFZKiWeVCqI7JkTJ4x5j7gPiu2rbwj50Al2wormDUxwepQlFLN0IJuyiMWb7bPIHq2lntQqsPSBKA84uusAob1DSe+V6jVoSilmqEJQLndocpa0nIOctZw/favVEemCUC53ZIthdgMWu1TqQ5OE4Byu9SsAmJ6BJMcG2F1KEqpFmgCUG5VXdfA8m1FTBseg59O9KJUh6YJQLnV6h3FVNU2aPOPUj5AE4Byq6+zCggL8meKFn9TqsPTBKDcxmYzfLO5gFO1+JtSPkETgHKbjPxSCstrtPlHKR+hCUC5TWrWfvz9hDOHafE3pXyBJgDlNouzCklJ7KnF35TyEZoAlFvsKa5ia0G5Nv8o5UM0ASi3SHUUf9MEoJTv0ASg3CI1az9DYrqTGBVmdShKKRdpAlDtVlJVy5qcQ/rtXykfowlAtdvSrYU02AzTR/S1OhSlVCtoAlDtlppVQJ/wYEZr8TelfIomANUuNfUNfLtVi78p5Ys0Aah2Wb2jmMraBp36USkfpAlAtUtqVgGhQf5M1uJvSvkcTQCqzWw2w+LNBZw6OJqQQC3+ppSv0QSg2mzj3lIKyrT4m1K+ShOAarPUrAL8BM7Q4m9K+SRNAKrNUrMKSEnqRa8wLf6mlC/SBKDaJPdgFVv2l+voH6V8WIDVAaiOY0NuCS8t3wkCAX6Cvwj+fkKAv+AnYl/m54e/H+wsqgTgrOGaAJTyVZYkABGJBF4BRgEG+KUxZrUVsaifPPT5ZjLzS+kXEUKDzVBvM9gcPxtshgZjaGhwPDeGKSdEkdRbi78p5ausugL4D/ClMeYKEQkCQi2KQzlszC/lh10H+ev5w7j51BOsDkcp5QVeTwAi0gM4FZgNYIypBWq9HYc61tyVuwgN8ueqExOsDkUp5SVWdAIPBIqA10RknYi8IiI/a0cQkZtFJE1E0oqKirwfZRdSWFbNJxv2cmVKPBHdAq0ORynlJVYkgABgPPCCMWYcUAn8uelKxpiXjDEpxpiU6Ohob8fYpbz5/W7qbYbZU5KsDkUp5UVW9AHkAXnGmB8czxdynATgDqt2HGBPcRVJvcNIigqjT3iwVqxsorqugbd+2MO0YTHaoatUF+P1BGCM2S8iuSIy1BizFZgGZHliWx+uy+e9tLyjz0MC/UiKCiMxKtTxM4yk3vbHfXuEdMnk8OG6fA5W1nLjyQOsDkUp5WVWjQK6A3jLMQJoJzDHExv512WjuePMweQUV5JTXMXuA5XkFFeyo6iSpVuKqG2wHV03KMCPyQOj+Mv5wxjWt4cnwulwjDHMXbmLEf16MGlgL6vDUUp5mSUJwBizHkjx9Hb8/YT4XqHE9wrllMHHvtZgM+wvqybHkRR2FlWyaG0e5//nO645KYG7pw/t9CUOvtt2gOyCCp6YOQaRrnf1o1RX12XvBPb3E2IjuxEb2Y2pg3oDcMeZg3hq8Tbe/H43H6/fy2/PGsL1kxMJ9O+cFTNeXbGL3t2DuXBMP6tDUUpZoHOe2dooMjSI+2eM5MvfnsKY+Ege/DSLc55aztKthVaH5nbbC8v5NruI6ycnEhygtfyV6oo0ARzH4Jhw3vjlRF69IQVjYM5ra5j92o9sL6ywOjS3mbsyh6AAP649SW/8Uqqr0gTQDBFh2vAYvrrrVP52wXDScw5x7lPLeeCTTZRW1VkdXrscqqzl/bV5XDYulqjuwVaHo5SyiCYAJ4IC/LjplIEs/ePpzEyJZ96qHE5/fClfbtxvdWht9vaPe6iuszFnqg79VKor0wTgot7dg/nXZcl8dscpxPUM5a7568guKLc6rFarrbfx+qocThncm6F9w60ORyllIU0ArTSifw9enZ1C9+AAbntrLYdrG6wOqVU+z9xHYXkNv9Qbv5Tq8jQBtEGf8BCevGos24sqeOCTTVaH4zJjDK+u2MUJ0WGcNljrKynV1WkCaKNTBkfzm9NO4N01uXy0Pt/qcFyyJucQmfmlzJk6oEuWvVBKHUsTQDvcPX0IKYk9+ev7meQcqLQ6HKdeXbGTyNBALh8fZ3UoSqkOQBNAOwT4+/Gfq8cR4O/H7e+spabeO/0Bxhg25pe2ant7iqv4OquAayYm0C1Ib/xSSmkCaLfYyG48PnMMG/PL+NfnW7yyzRe+3cGFz6xg0kPf8OCnWWwvdD4aad6qHPxFuH5ykucDVEr5hC5bC8idpo+IYc7UJF5bmcPkE6I4Z2Rfj21r6ZZCHvtqK2cO60NIoB+vr8rh1RW7SEnsyayJCVyQ3O9n3/DLq+t4Ly2XC0b3o29EiMdiU0r5Fk0AbvLn84axJucg9yzMYFRsBLGR3dy+jZ1FFdz57jqG9+3Bc9eMp1uQPwcqaliUnsf8Nbn8YcEGHvhkE5eMjWXWxHhG9o8AYP6aXCpq6rXmv1LqGGKMsToGp1JSUkxaWprVYTiVc6CSC59ZwdC+4bx78yS3VhEtr67jkudWcqiqjo9vn0pcz9BjXjfG8MOug7z74x4+37if2nobo+MiuOrEeF5YtoN+ESEsuGWK2+JRSnV8IpJujGm29L72AbhRUu8wHrosmfTdh/h3arbbPtdmM/xu/npyiqt47prxPzv5g7120aSBUTw1axw//nUa9180gtp6G/d+sJG8Q4f1279S6me0CcjNZozpz+odB3hh2Q4mD4zi1CHtv+HqycXZLN5cyAMzRjL5hCin60eGBjF76gBumJLE+twSsvaVMX2E5/ollFK+Sa8APODvF45kSEx37n5vPYVl1e36rC8y9/HMku1clRLP9ZMTW/VeEWFcQk+uPSkRf73xSynVhCYAD+gW5M9z14ynoqaeu+avp8HWtn6WLfvL+P2CDYxPiOQfl4zUaRuVUm6lCcBDBseE848Zo1i1o5gbX1/Duj2HWvX+Q5W1/OqNNMJDAnjxugk6a5dSyu20D8CDZqbEcbCqlheW7eDS51cxaWAvbj19EKcM7t3it/n6Bhu3vb2WgtIa5v96En166Nh9pZT76RWAB4kIt5x2Aqv+fCZ/u2A4OQequH7uj1z07Ao+y9jXbNPQQ59vYdWOYv556SjGJfT0ctRKqa5CE4AXhAUHcNMpA/n2ntN59PLRVNU0cNvba5n2xDLe+XHPMTV9FqXnMXflLmZPSWJmSryFUSulOju9EcwCDTbD15v28/yyHWTml9InPJibThnAqP4RzJ63hgkJPXnjxoluvZFMKdX1OLsRTPsALODvJ5yX3I9zR/Vl5fZiXvh2Ow85CsnF9ezGc9eO15O/UsrjNAFYSEQ4eXBvTh7cmw25JSxMz+MXkxPpFRZkdWhKqS5AE0AHMSY+kjHxkVaHoZTqQixrZxARfxFZJyKfWhWDUkp1ZVY2NP8W2Gzh9pVSqkuzJAGISBxwAfCKFdtXSill3RXAU8A9gK25FUTkZhFJE5G0oqIirwWmlFJdhdcTgIhcCBQaY9JbWs8Y85IxJsUYkxId3f6SykoppY5lxRXAVGCGiOQA7wJnisj/LIhDKaW6NK8nAGPMX4wxccaYJGAWsMQYc52341BKqa5ObzdVSqkuyidqAYlIEbC70aLewAGLwvG0zrpvul++p7PuW1far0RjTLOdqD6RAJoSkbSWChz5ss66b7pfvqez7pvu10+0CUgppbooTQBKKdVF+WoCeMnqADyos+6b7pfv6az7pvvl4JN9AEoppdrPV68AlFJKtZMmAKWU6qJ8LgGIyLkislVEtovIn62Ox11EJEdEMkVkvYj49ATIIjJXRApFZGOjZb1EJFVEtjl+9rQyxrZoZr/uF5F8x3FbLyLnWxljW4hIvIgsFZHNIrJJRH7rWO7Tx6yF/eoMxyxERH4UkQ2OfXvAsbxVx8yn+gBExB/IBqYDecAa4GpjTJalgbmBozZSijHG529QEZFTgQrgDWPMKMeyR4GDxpiHHYm7pzHmT1bG2VrN7Nf9QIUx5nErY2sPEekH9DPGrBWRcCAduASYjQ8fsxb260p8/5gJEGaMqRCRQGAF9jlWLqMVx8zXrgAmAtuNMTuNMbXYi8ldbHFMqgljzHLgYJPFFwOvOx6/jv0P0ac0s18+zxizzxiz1vG4HPtETbH4+DFrYb98nrGrcDwNdPwztPKY+VoCiAVyGz3Po5McUOwH72sRSReRm60OxgNijDH7wP6HCfSxOB53ul1EMhxNRD7VTNKUiCQB44Af6ETHrMl+QSc4Zo5pddcDhUCqMabVx8zXEoAcZ5nvtGG1bKoxZjxwHnCbo7lBdXwvACcAY4F9wBOWRtMOItIdWATcZYwpszoedznOfnWKY2aMaTDGjAXigIkiMqq1n+FrCSAPiG/0PA7Ya1EsbmWM2ev4WQh8gL25qzMpcLTJHmmbLbQ4HrcwxhQ4/hBtwMv46HFztCMvAt4yxrzvWOzzx+x4+9VZjtkRxpgSYBlwLq08Zr6WANYAg0VkgIgEYZ9P4GOLY2o3EQlzdFIhImHA2cDGlt/lcz4GbnA8vgH4yMJY3ObIH5vDpfjgcXN0KL4KbDbG/LvRSz59zJrbr05yzKJFJNLxuBtwFrCFVh4znxoFBOAYsvUU4A/MNcb809qI2k9EBmL/1g8QALzty/slIu8Ap2MvT1sA3Ad8CLwHJAB7gJnGGJ/qUG1mv07H3pRggBzg10faYH2FiJwMfAdk8tM83X/F3l7us8eshf26Gt8/ZqOxd/L6Y/8i/54x5h8iEkUrjpnPJQCllFLu4WtNQEoppdxEE4BSSnVRmgCUUqqL0gSglFJdlCYApZTqojQBqA5DRBoc1Rk3isgCEQltYd3ZIvKsN+NrtO1/iMhZTtaZJyJXOFknqXFlUaW8TROA6kgOG2PGOipt1gK3WB3Q8Rhj/m6MWWx1HI05KuUq1SqaAFRH9R0wyFHf/ENH4a7vHTfAHCUi4SKyy3HLPyLSQ+xzKwSKyDIRecRRNz1bRE5xrBMiIq+Jff6FdSJyhmP5bMe2PnF85u0icrdjne9FpJdjvaPf7kXk7yKyxnHV8pLj7tNmicgERw331cBtjZb7i8hjjs/KEJFfO5b7icjzYq/5/qmIfN5o2zmO7a8AZorI2SKyWkTWOq6gujfa5reOQoNfNbkTVnVhmgBUhyMiAdiL4mUCDwDrjDGjsd/F+UbjdR1lfpcBFzgWzQIWGWPqHM8DjDETgbuw37kLjhOvMSYZ+12hr4tIiOO1UcA12OvD/BOoMsaMA1YD1x8n3GeNMSc6rlq6ARc62b3XgDuNMZObLL8RKDXGnAicCPxKRAZgr++eBCQDNwFN31dtjDkZWAz8DTjLUVQwDbjbkRifAa4wxkwA5jr2SykCrA5AqUa6Ocrbgv0K4FXs5QguBzDGLBGRKBGJaPK+V4B7sJebmAP8qtFrRwqbpWM/kQKcjP2kiDFmi4jsBoY4XlvqSCrlIlIKfOJYngkcc/XhcIaI3AOEAr2ATY3ecwxH3JHGmG8di97EnujAXv9pdKN+gwhgsCPWBY7CZftFZGmTj53v+DkJGAGsdFyEBGFPWkOxJ7VUx3J/7BUwldIEoDqUw47ytkc106RyTP0SY8xKR4fqaYC/MaZxx2qN42cDP/1/b6mZpqbRY1uj5zaa/L04rhqexz6TW67YZwcLoXnSNPYmr91hjPmqyTYuaGb9IyobvT/VGHN1k/cnA5uOc8WhlDYBqQ5vOXAtgIicDhxoplb9G8A72JtYWvOZQ7AXztrahtiOnOwPONrbWxz14yjbW+ooUsaRGBy+An7TqC9jiNgrw64ALnf0BcRgLz53PN8DU0VkkOP9oY592wpEi8hkx/JAERnZyv1UnZReAaiO7n7gNRHJAKr4qdRtU28B/4c9CTjzPPCiiGQC9cBsY0yNk/7bnzHGlIjIy9ibh3Kwlyt3Zg4wV0SqsJ/0j3gFexPVWsdVTxH26fwWAdOwlyzOxt4kVnqcWIpEZDbwjogEOxb/zRiT7WhWetrRBBWAvZruptbsq+qctBqo6hQcJ7mLjTG/sDoWdxOR7o7Jv6OAH7HPHrff6riU79MrAOXzROQZ7J2p51sdi4d8KvbJP4KAB/Xkr9xFrwCUUqqL0k5gpZTqojQBKKVUF6UJQCmluihNAEop1UVpAlBKqS7q/wOSyeLBFSEwrQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_73_1.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Common imports\n", "import os\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from sklearn.linear_model import LinearRegression, Ridge, Lasso\n", "from sklearn.metrics import mean_squared_error\n", "from sklearn.model_selection import KFold\n", "from sklearn.model_selection import cross_val_score\n", "\n", "\n", "# Where to save the figures and data files\n", "PROJECT_ROOT_DIR = \"Results\"\n", "FIGURE_ID = \"Results/FigureFiles\"\n", "DATA_ID = \"DataFiles/\"\n", "\n", "if not os.path.exists(PROJECT_ROOT_DIR):\n", " os.mkdir(PROJECT_ROOT_DIR)\n", "\n", "if not os.path.exists(FIGURE_ID):\n", " os.makedirs(FIGURE_ID)\n", "\n", "if not os.path.exists(DATA_ID):\n", " os.makedirs(DATA_ID)\n", "\n", "def image_path(fig_id):\n", " return os.path.join(FIGURE_ID, fig_id)\n", "\n", "def data_path(dat_id):\n", " return os.path.join(DATA_ID, dat_id)\n", "\n", "def save_fig(fig_id):\n", " plt.savefig(image_path(fig_id) + \".png\", format='png')\n", "\n", "infile = open(data_path(\"EoS.csv\"),'r')\n", "\n", "# Read the EoS data as csv file and organize the data into two arrays with density and energies\n", "EoS = pd.read_csv(infile, names=('Density', 'Energy'))\n", "EoS['Energy'] = pd.to_numeric(EoS['Energy'], errors='coerce')\n", "EoS = EoS.dropna()\n", "Energies = EoS['Energy']\n", "Density = EoS['Density']\n", "# The design matrix now as function of various polytrops\n", "\n", "Maxpolydegree = 30\n", "X = np.zeros((len(Density),Maxpolydegree))\n", "X[:,0] = 1.0\n", "estimated_mse_sklearn = np.zeros(Maxpolydegree)\n", "polynomial = np.zeros(Maxpolydegree)\n", "k =5\n", "kfold = KFold(n_splits = k)\n", "\n", "for polydegree in range(1, Maxpolydegree):\n", " polynomial[polydegree] = polydegree\n", " for degree in range(polydegree):\n", " X[:,degree] = Density**(degree/3.0)\n", " OLS = LinearRegression(fit_intercept=False)\n", "# loop over trials in order to estimate the expectation value of the MSE\n", " estimated_mse_folds = cross_val_score(OLS, X, Energies, scoring='neg_mean_squared_error', cv=kfold)\n", "#[:, np.newaxis]\n", " estimated_mse_sklearn[polydegree] = np.mean(-estimated_mse_folds)\n", "\n", "plt.plot(polynomial, np.log10(estimated_mse_sklearn), label='Test Error')\n", "plt.xlabel('Polynomial degree')\n", "plt.ylabel('log10[MSE]')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that we have kept the intercept in the first column of design matrix $\\boldsymbol{X}$. When we call the corresponding **Scikit-Learn** function we need thus to set the intercept to **False**. Libraries like **Scikit-Learn** normally scale the design matrix and does not fit intercept. See the discussions below.\n", "\n", "## More on Rescaling data\n", "\n", "We end this chapter by adding some words on scaling and how to deal with the intercept for regression cases.\n", "\n", "When you are comparing your own code with for example **Scikit-Learn**'s\n", "library, there are some technicalities to keep in mind. The examples\n", "here demonstrate some of these aspects with potential pitfalls.\n", "\n", "The discussion here focuses on the role of the intercept, how we can\n", "set up the design matrix, what scaling we should use and other topics\n", "which tend confuse us.\n", "\n", "The intercept can be interpreted as the expected value of our\n", "target/output variables when all other predictors are set to zero.\n", "Thus, if we cannot assume that the expected outputs/targets are zero\n", "when all predictors are zero (the columns in the design matrix), it\n", "may be a bad idea to implement a model which penalizes the intercept.\n", "Furthermore, in for example Ridge and Lasso regression, the default solutions\n", "from the library **Scikit-Learn** (when not shrinking $\\beta_0$) for the unknown parameters\n", "$\\boldsymbol{\\beta}$, are derived under the assumption that both $\\boldsymbol{y}$ and\n", "$\\boldsymbol{X}$ are zero centered, that is we subtract the mean values.\n", "\n", "\n", "If our predictors represent different scales, then it is important to\n", "standardize the design matrix $\\boldsymbol{X}$ by subtracting the mean of each\n", "column from the corresponding column and dividing the column with its\n", "standard deviation. Most machine learning libraries do this as a default. This means that if you compare your code with the results from a given library,\n", "the results may differ. \n", "\n", "The\n", "[Standardscaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html)\n", "function in **Scikit-Learn** does this for us. For the data sets we\n", "have been studying in our various examples, the data are in many cases\n", "already scaled and there is no need to scale them. You as a user of different machine learning algorithms, should always perform a\n", "survey of your data, with a critical assessment of them in case you need to scale the data.\n", "\n", "If you need to scale the data, not doing so will give an *unfair*\n", "penalization of the parameters since their magnitude depends on the\n", "scale of their corresponding predictor.\n", "\n", "Suppose as an example that you \n", "you have an input variable given by the heights of different persons.\n", "Human height might be measured in inches or meters or\n", "kilometers. If measured in kilometers, a standard linear regression\n", "model with this predictor would probably give a much bigger\n", "coefficient term, than if measured in millimeters.\n", "This can clearly lead to problems in evaluating the cost/loss functions.\n", "\n", "\n", "\n", "Keep in mind that when you transform your data set before training a model, the same transformation needs to be done\n", "on your eventual new data set before making a prediction. If we translate this into a Python code, it would could be implemented as follows" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "data": { "text/plain": [ "'\\n#Model training, we compute the mean value of y and X\\ny_train_mean = np.mean(y_train)\\nX_train_mean = np.mean(X_train,axis=0)\\nX_train = X_train - X_train_mean\\ny_train = y_train - y_train_mean\\n\\n# The we fit our model with the training data\\ntrained_model = some_model.fit(X_train,y_train)\\n\\n\\n#Model prediction, we need also to transform our data set used for the prediction.\\nX_test = X_test - X_train_mean #Use mean from training data\\ny_pred = trained_model(X_test)\\ny_pred = y_pred + y_train_mean\\n'" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\"\"\"\n", "#Model training, we compute the mean value of y and X\n", "y_train_mean = np.mean(y_train)\n", "X_train_mean = np.mean(X_train,axis=0)\n", "X_train = X_train - X_train_mean\n", "y_train = y_train - y_train_mean\n", "\n", "# The we fit our model with the training data\n", "trained_model = some_model.fit(X_train,y_train)\n", "\n", "\n", "#Model prediction, we need also to transform our data set used for the prediction.\n", "X_test = X_test - X_train_mean #Use mean from training data\n", "y_pred = trained_model(X_test)\n", "y_pred = y_pred + y_train_mean\n", "\"\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us try to understand what this may imply mathematically when we\n", "subtract the mean values, also known as *zero centering*. For\n", "simplicity, we will focus on ordinary regression, as done in the above example.\n", "\n", "The cost/loss function for regression is" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "C(\\beta_0, \\beta_1, ... , \\beta_{p-1}) = \\frac{1}{n}\\sum_{i=0}^{n} \\left(y_i - \\beta_0 - \\sum_{j=1}^{p-1} X_{ij}\\beta_j\\right)^2,.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Recall also that we use the squared value. This expression can lead to an\n", "increased penalty for higher differences between predicted and\n", "output/target values.\n", "\n", "What we have done is to single out the $\\beta_0$ term in the\n", "definition of the mean squared error (MSE). The design matrix $X$\n", "does in this case not contain any intercept column. When we take the\n", "derivative with respect to $\\beta_0$, we want the derivative to obey" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\frac{\\partial C}{\\partial \\beta_j} = 0,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "for all $j$. For $\\beta_0$ we have" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\frac{\\partial C}{\\partial \\beta_0} = -\\frac{2}{n}\\sum_{i=0}^{n-1} \\left(y_i - \\beta_0 - \\sum_{j=1}^{p-1} X_{ij} \\beta_j\\right).\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Multiplying away the constant $2/n$, we obtain" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\sum_{i=0}^{n-1} \\beta_0 = \\sum_{i=0}^{n-1}y_i - \\sum_{i=0}^{n-1} \\sum_{j=1}^{p-1} X_{ij} \\beta_j.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us specialize first to the case where we have only two parameters $\\beta_0$ and $\\beta_1$.\n", "Our result for $\\beta_0$ simplifies then to" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "n\\beta_0 = \\sum_{i=0}^{n-1}y_i - \\sum_{i=0}^{n-1} X_{i1} \\beta_1.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We obtain then" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1}y_i - \\beta_1\\frac{1}{n}\\sum_{i=0}^{n-1} X_{i1}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If we define" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mu_{\\boldsymbol{x}_1}=\\frac{1}{n}\\sum_{i=0}^{n-1} X_{i1},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and the mean value of the outputs as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mu_y=\\frac{1}{n}\\sum_{i=0}^{n-1}y_i,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "we have" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\beta_0 = \\mu_y - \\beta_1\\mu_{\\boldsymbol{x}_1}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the general case with more parameters than $\\beta_0$ and $\\beta_1$, we have" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1}y_i - \\frac{1}{n}\\sum_{i=0}^{n-1}\\sum_{j=1}^{p-1} X_{ij}\\beta_j.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can rewrite the latter equation as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1}y_i - \\sum_{j=1}^{p-1} \\mu_{\\boldsymbol{x}_j}\\beta_j,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where we have defined" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mu_{\\boldsymbol{x}_j}=\\frac{1}{n}\\sum_{i=0}^{n-1} X_{ij},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "the mean value for all elements of the column vector $\\boldsymbol{x}_j$.\n", "\n", "\n", "\n", "Replacing $y_i$ with $y_i - y_i - \\overline{\\boldsymbol{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "C(\\boldsymbol{\\beta}) = (\\boldsymbol{\\tilde{y}} - \\tilde{X}\\boldsymbol{\\beta})^T(\\boldsymbol{\\tilde{y}} - \\tilde{X}\\boldsymbol{\\beta}).\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If we minimize with respect to $\\boldsymbol{\\beta}$ we have then" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\hat{\\boldsymbol{\\beta}} = (\\tilde{X}^T\\tilde{X})^{-1}\\tilde{X}^T\\boldsymbol{\\tilde{y}},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $\\boldsymbol{\\tilde{y}} = \\boldsymbol{y} - \\overline{\\boldsymbol{y}}$\n", "and $\\tilde{X}_{ij} = X_{ij} - \\frac{1}{n}\\sum_{k=0}^{n-1}X_{kj}$.\n", "\n", "For Ridge regression we need to add $\\lambda \\boldsymbol{\\beta}^T\\boldsymbol{\\beta}$ to the cost function and get then" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\hat{\\boldsymbol{\\beta}} = (\\tilde{X}^T\\tilde{X} + \\lambda I)^{-1}\\tilde{X}^T\\boldsymbol{\\tilde{y}}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What does this mean? And why do we insist on all this? Let us look at some examples.\n", "\n", "\n", "This code shows a simple first-order fit to a data set using the above transformed data, where we consider the role of the intercept first, by either excluding it or including it (*code example thanks to Øyvind Sigmundson Schøyen*). Here our scaling of the data is done by subtracting the mean values only.\n", "Note also that we do not split the data into training and test." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "True beta: [2, 0.5, 3.7]\n", "Fitted beta: [2.08376632 0.19569961 3.97898392]\n", "Sklearn fitted beta: [2.08376632 0.19569961 3.97898392]\n", "MSE with intercept column\n", "0.0041136346174431284\n", "MSE with intercept column from SKL\n", "0.004113634617443141\n", "Manual intercept: 2.0837663229239016\n", "Fitted beta (wiothout intercept): [0.19569961 3.97898392]\n", "Sklearn intercept: 2.0837663229239025\n", "Sklearn fitted beta (without intercept): [0.19569961 3.97898392]\n", "MSE with Manual intercept\n", "0.00411363461744314\n", "MSE with Sklearn intercept\n", "0.004113634617443135\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA7G0lEQVR4nO3deVhV1frA8e8GQY+CoqKIQ+KQEzIIDqhpqDlr4VCa5lC3zJ+38lpqmpnKbaAwK+t2G03tWlqOZYMjpDlUECBOpCSKSIooCMjM+v2BnESmIx4453Dez/OcJzl77b3fxaGXxdp7v0tTSiGEEMJ82Zg6ACGEEOWTRC2EEGZOErUQQpg5SdRCCGHmJFELIYSZq1UVB3V2dlZubm6V2jcjI4N69eoZNyAzJ32u+aytvyB9vl3h4eGXlVJNSttWJYnazc2NsLCwSu0bGhqKv7+/cQMyc9Lnms/a+gvS59uladrZsrbJ1IcQQpg5SdRCCGHmJFELIYSZq5I56tLk5uZy/vx5srKyym3XoEEDTpw4UU1RmQfps2WoU6cOLVu2xM7OztShCCtTbYn6/PnzODo64ubmhqZpZbZLS0vD0dGxusIyC9Jn86eUIjk5mfPnz9OmTRtThyOsTLVNfWRlZdG4ceNyk7QQ5krTNBo3blzhX4RCVIVqnaOWJC0smfz8ClOptqkPIYSoqbZGJPD9lrX41z1Hn4N5zB/emYBuLYx2fKu668PW1hZvb2/9Ky4ujj59+gAQFxfHF198YeIIhRCWZmtEAu9s2sPII6updzqUlGupLNwczdaIBKOdw6oStU6nIzIyUv9yc3Pj4MGDgCRqIUTlrPwhgvdtgunncY0znR/mOnXIzM0neEeM0c5hVYm6NA4ODgAsWLCA/fv34+3tzVtvvWXiqIQQliA3J5t/RC+ifk4is/OfwbaZl37bhZRMo53HJHPUy749xvEL10rdlp+fj62t7W0fs0vz+iwZ7V5um8zMTLy9vQFo06YNW7Zs0W8LCgpi+fLlbN++/bbPLYSwTptm+OMTkcMGnSf7m3rSkzz9tuZOOqOdx6ouJhZNfQghxJ36avFEvA6nEN21Dh+5Pgv5f2/T2dkyb2hHo53LJIm6vJGvpT0IIYSwPrvXvkrHzVHEtdAY8eke7P/MvDEnnUYLJx3zhnY06l0fVjWiLo+joyNpaWmmDkMIYeYuno8ld+3npNWDju+uxqFBIwK6QUC3FoSGhvL0ZH+jn9PqLyYW8fT0pFatWnh5ecnFRCFEqTIz0rj22UN49UrBdsl82nbpWS3ntaoRdXp6epnv2dnZsWfPnuoOSQhhIfLz8vjm2UGMahrLqYEf0HvQxGo7t0Ejak3TnDRN26hp2klN005omta7qgMTQghz8vXMAXj+lMb2zD54V2OSBsNH1O8APyqlxmuaZg/UrcKYhBDCrGwKnIrXz5c53qU244O+qfbzV5ioNU2rD/QHpgMopXKAnKoNSwghzEPI+hW0++o3zrlq3PfJj9jWqv4ZY00pVX4DTfMGPgKOA15AODBbKZVxS7sZwAwAFxcX3/Xr1xc7ToMGDWjfvn2FAVX2gRdLJn22HKdPnyY1NfW290tPT9c/BWstakKfM1Iuolu+lNrZcHH20zRq2aXc9nfS5wEDBoQrpbqXts2QRN0dOAz0VUr9omnaO8A1pdTisvbp3r27unUV8hMnTtC5c+cKg7XG+6ilz5bD0J/jW8mK3JYnKzODs28OpCD1PBd7zMf/odkV7nOHq5CXmagNuZh4HjivlPrlxtcbAZ9KRSKEEBYgPy+P714czt25J8m671WDknRVqjBRK6X+AuI1TSt6HnIQhdMgQghRI3311GC6fJfExtx76TZ0mqnDMfiBl6eBdZqmHQG8gVerLKIq9Morr+Du7o6npyfe3t788kvhHwlubm5cvny5RHtTzK9t3bqVwMBAAJKSkujVqxfdunVj//79jBgxgpSUFFJSUnj//ffLPc6FCxcYP358hed79VXTfZSVKS3r5uaGh4eHvqZ4UZnastqW9rlWZO7cuezdu/e29xM1w5agGXiG/sXJjnaMC9xs6nAKKaWM/vL19VW3On78eIn3SnPt2jWD2t2ugwcPKj8/P5WVlaWUUiopKUklJCQopZRq3bq1SkpKKrFPvXr1jHLuvLy8crff3OfevXvrY/nyyy/V1KlTS7Q/c+aMcnd3N0psleljRf0xxLVr11RISIgaOXLkbe1X1md1p21vFhcXpwYPHlzqNkN/jm8VEhJSqf0smSX2ed+m91S4Rye1s39ndeXS+dve/076DISpMnKqaZ5M/GEB/BVd6iZdfh7YViKsZh4wPKjMzYmJiTg7O1O7dm0AnJ2dS7TJzMxkzJgxjBs3jieeeKLYtuDgYL766iuys7MZM2YMy5YtAyAgIID4+HiysrKYPXs2M2bMAApH488++yw7duzgzTffZNiwYcyePZvt27ej0+nYtm0bLi4uxc7xxx9/ULt2bZydnYmMjGT+/Pn60qyHDh2ic+fOhIWFsWDBAmJjY/H29mbw4MEEBweX6EtcXByjRo3i6NGjrF69mm+++Ybr168TGxvLmDFjeOONN1iwYIH++O7u7qxbt47//e9/rFy5kpycHHr16sX777+Pra1tif78+eefLF++HE3T8PT05PPPPycpKYmZM2dy7tw5AN5++2369u3L0qVLiY2NJSEhgfj4eObPn8/EiRNZsGABJ06cwNvbm2nTpjFnzpzb+MD/VtZnUCQjI4OHHnqI8+fPk5+fz+LFi5kwYQLh4eE8++yzpKen4+zszOrVq3F1daV169YkJyfz119/0axZs0rFJCzPX/GnUG+8R7Y9tFy+koZNjFdU6U5ZTa2PIUOGEB8fT4cOHZg1axY//fRTse3p6emMHj2aSZMmlUjSO3fu5NSpU/z6669ERkYSHh7Ovn37AFi1ahXh4eGEhYWxcuVKkpOTgcLk0LVrV3755RfuueceMjIy8PPzIyoqiv79+/Pxxx+XiPHAgQP4+BRep/X29iYwMJAJEyYQGRmJTvd3bdugoCDatWtHZGRkqUm6NJGRkWzYsIHo6Gg2bNhAfHw8QUFB+tKv69at48SJE2zYsIEDBw4QGRmJra0t69atK9Gfhg0b8sorr7B3716ioqJ45513AJg9ezZz5szht99+Y9OmTTz++OP68x85coTvvvuOQ4cOERgYSGJiIkFBQfTr14/IyEjmzJlDTExMsaXSbn6lpKTojzVgwAC8vb3p1atXuZ9BkR9//JHmzZsTFRXF0aNHGTZsGLm5uTz99NNs3LiR8PBwHnvsMRYtWqTfx8fHhwMHDhj0vRWWLyc7iytrp2PbPYOM2dPo3OM+U4dUjGlG1OWMfDOr6LYtBwcHwsPD2b9/PyEhIUyYMIGgoCCmT58OwAMPPMD8+fOZPHlyiX137tzJzp076datG1CY1E+dOkX//v1ZuXKlfgGC+Ph4Tp06RePGjbG1tWXcuHH6Y9jb2zNq1CgAfH192bVrV4nzJCYm0qRJE2N3HYBBgwbRoEEDALp06cLZs2dp1apVsTZ79uwhPDycHj16AIV/YTRt2hSgWH/27t3L+PHj9X+VNGrUCIDdu3dz/Pjf15mvXbumr0j4wAMPoNPp0Ol0DBgwgPDwcFxdXYudv2PHjgbVCw8JCSn2F1FZn0ERDw8P5s6dy/PPP8+oUaPo168fR48e5ejRowwePBgovK/75niaNm3KhQsXKoxFWL78vDx2vjqGUdpRwoYvp+/IJyreqZpZVVEmW1tb/P398ff3x8PDgzVr1ugTdd++ffnhhx+YNGkSmqYV208pxcKFC3nyySeLvR8aGsru3bs5dOgQdevWxd/fn6ysLADq1KlT7IEOOzs7/XFtbW3Jy8vjVjqdrlIPUxiiaMqnvPMrpZg2bRqvvfZaiW0390cpVeJ7BFBQUMChQ4eKjf6L3Nq+tP1jYmKYMGFCqfGHhobi5ORU6vtlfQZFOnToQHh4ON9//z0LFy5kyJAhjBkzBnd3dw4dOlTq+bKyskrth6h5vp4zAo/d8WyZOJgxZpikwYqmPmJiYjh16pT+68jISFq3bq3/OjAwkMaNGzNr1qwS+w4dOpRVq1bpK+0lJCRw6dIlUlNTadiwIXXr1uXkyZMcPnz4jmLs3Lkzp0+frrCdMWtn29nZkZubCxSOujdu3MilS5cAuHLlCmfPni2xz6BBg/jqq6/0UwxXrlwBCqeX3nvvPX27m0fH27ZtIysri+TkZEJDQ/Hx8SnRj6IRdWmv0pI0YNBncOHCBerWrcsjjzzC3Llz+f333+nYsSNJSUn6RJ2bm8uxY8f0+/zxxx907drVkG+hsGDb3nwaj93xnGpXi1EL15k6nDJZTaJOT09n2rRpdOnSBU9PT44fP87SpUuLtXn77bfJyspi/vz5xd4fMmQIkyZNonfv3nh4eDB+/HjS0tIYNmwYeXl5eHp6snjxYvz8/O4oxv79+xMREYGq4GnRxo0b07dvX7p27cq8efPu6JwzZszA09OTyZMn06VLF15++WWGDBmCp6cngwcPJjExscQ+7u7uLFq0iHvvvRcvLy+effZZoHAKIiwsDE9PT7p06cIHH3yg36dnz56MHDkSPz8/Fi9ejKurq1FqgBvyGURHR9OzZ0+8vb155ZVXePHFF7G3t2fjxo08//zzeHl5FbvVLzc3l9OnT9O9e6kPiYka4uB3n9FyzW7+coY+H27Fzr52xTuZSlm3g9zJyxxvzzNnN/f5mWeeUbt27TJhNMa3ZMkSFRwcXOw9c/6cN2/erF588cVSt8nteYYz5z4n/Hlc/dSrkzrcrZOK3L/NaMetqtvzrGZEbSleeOEFrl+/buowrFpeXh7PPfecqcMQVSQ3J5vkL2eQ0SmblH9OwOue+00dUoWs6mKiJXBxceH++w3/wYmOjmbKlCnF3qtdu7b+qUtzcOsUk7l78MEHTR2CqEI/vT2V+3KOkBXwCj0CnjJ1OAaRRG3hPDw8DLqlTQgB6+eM4O69Z/huwjBGWkiSBiu6mCiEsG7b35tL1x/PkNDclvvmfGbqcG6LJGohRI33284vafbJdyQ1gh4fbKS2zrJWE5RELYSo0RLOHCdjaSAFGtQPDMS1dSdTh3TbrCpR29raFqsfERcXR58+fYCKS24mJibqHwE3J9OnT2fjxo0l3n/ppZfYvXt3ufuGhoaWWya0qq1evbrYY9oTJ04s9lCSEHcqLzeHxHVPkeaST9KMAHwHWeaFYqtK1EUFiIpebm5u+kRVUaJesWJFiWJN5iwwMJD77iu/sExlEnVpj55X1q2J+v/+7/944403jHZ8IQ5+8CTd88JxmTyTkf9XsjSCpbCqRF2aosUBFixYwP79+/H29i71KblNmzYxbNgwoDDBBAQEMHr0aNq0acN7773HihUr6NatG35+fvpHqj/++GN69OiBl5cX48aN098fPX36dJ555hn69OlD27Zt2bp1K1CYOG8etT/11FOsXr0aKEy8PXr0oGvXrsyYMaPCpxdvHmm7ubmxZMkSfHx88PDw4OTJk8TFxfHBBx/w1ltv4e3tzf79+0lKSmLcuHH06NGDHj166KvHLV26lBkzZjBkyBCmTp3KxYsXGTNmDF5eXnh5eemT/f/+9z/9E4BPPvkk+fn5+u/xc889h4+PD4MGDSIpKYmtW7cSFhbG5MmT8fb2JjMzk379+rF7926j/jIQ1mvD/AD44hC76g6n5/hnTR3OHTHZ7XmP/vhoifeGug1lZIuRZOZlMmt3yZobD7R/gID2AVzNusqzocW/8Z8Nq/gqblHtZYA2bdroK65BYenQ5cuXs3379hL7nTlzhoYNGxYrbHT06FEiIiLIysqiffv2vP7660RERDBnzhzWrl3Lv/71L8aOHasfhb/44ot8+umnPP3000DhVMrPP//MyZMnGTVqVIl7oW/11FNP8dJLLwEwZcoUtm/fzujRoyvscxFnZ2d+//133n//fZYvX84nn3zCzJkzcXBwYO7cuQBMmjSJOXPmcM8993Du3DmGDh3KiRMnAAgPD+fnn39Gp9MxYcIE7r33XrZs2UJ+fj7p6enFSqTa2dkxa9Ys1q1bx9SpU8nIyMDHx4c333yTwMBAli1bxmuvvcann37K8uXLiz2q3b59e6KiovD19TW4b0IU2RqRQPCOGLqdX8eTe2M429KWQf/8yNRh3TGruo+6aOrjdpVWfnTAgAE4Ojri6OhIgwYN9EnTw8ODI0eOAIXJ/MUXXyQlJYX09HSGDh2q3z8gIAAbGxu6dOlCUlJShTGEhITwxhtvcP36da5cuYK7u/ttJeqxY8cChSVWN28ufXmh8sqU3n///fpqcnv37mXt2rVA4bx/gwYN+Pzzz8sskWpjY6OvivfII4/oYylNUXlRSdTidm2NSGDh5mjaX/+NqQcOcqUBBPs8Q94faQR0q2/q8O6IyRJ1WSPgtLQ0dLV05Y6QG9ZpaNAI2lh0Ol2J0pk3j65tbGz0X9vY2Oj/dJ8+fTpbt27Fy8uL1atXExoaWur+RdMYtWrVoqCgQP9+0TmzsrKYNWsWYWFhtGrViqVLl5aIpyJF5yurxCmUX6a0Xr165R5flVMi9VallTgtIuVFRWUF74ihfuY55hz+ApsCeNdvHGdpSfCOGAK6mc9qLZVh9XPURcorHdqhQwfi4uJu+5hpaWm4urqSm5urXymlPK1bt+b48eNkZ2eTmprKnj17gL8TtrOzM+np6aXe5VEZt/a5vDKlNxs0aBD//e9/gcKC+9euXSu3RGpBQYE+5i+++IJ77rmn1PNDYXlRd3d3o/RPWJfklBSC7D4iz17xxT2+hNXuDcCFlEwTR3bnJFHfUF7JzXr16tGuXTuDakXf7N///je9evVi8ODBdOpU8b2brVq14qGHHtKXHS1aUcbJyYknnngCDw8PAgIC9NMLd2r06NFs2bJFfzGxvDKlN3vnnXcICQnBw8MDX19fjh07Vm6J1Hr16nHs2DF8fX3Zu3evfq59+vTpzJw5U38x8eLFi+h0uhIrvwhRkYy0VN61fZt77U+z1W8cmx0f1m9r7mT5f6FpFd09UBndu3dXYWFhxd47ceIEnTt3rnDftCpaiutObdmyhfDwcF5++WWjH9tc+2wsDg4O+kUXipTW57feeov69evzj3/8ozrDuy2G/hzfKjQ0FH9/f+MHZMaqq8+5Odl8+1BPdJlZHOw+nM/zB+u36exseW2sR7VNfdxJnzVNC1dKlVoEXUbUBhozZgxubm6mDqNGc3JyYtq0aaYOQ1iQ/Lw8tkzrS+eTOaS0bobv+Hm0cNKhAS2cdNWapKuSVd31caduXlVbGO7W0XRZHn205C2bQpTn65kD8IrI4IivAw+9vwfbWrVqRGK+lYyohRAWacOzI/H6+TLHu9Rm7Gc/Y1ur5o47JVELISxO2PaP8NDCOe5ux4jP95n3eodGIIlaCGFRQr5cjtdvC6Bxe0Z+fhBdPct+mMUQkqiFEBZjx6dLafjKp3x7vjktZ22jTl0HU4dULSRRCyEswr5N79F45QZSHcHjmQ+p79TY1CFVG6tK1K+88gru7u54enri7e2tXwDWzc2Ny5cvl2hfVFmvOm3dupXAwEAAkpKS6NWrF926dWP//v2MGDGClJQUUlJSeP/998s9zoULFxg/fnyF53v11VeNEndlVFRa9la9evXC29ubu+66iyZNmhSrK34npA62+Qvf8zX2r/6HHDtoEPQqd3v1NXVI1UspZfSXr6+vutXx48dLvFeaa9euGdTudh08eFD5+fmprKwspZRSSUlJKiEhQSmlVOvWrVVSUlKJferVq2eUc+fl5ZW7/eY+9+7dWx/Ll19+qaZOnVqi/ZkzZ5S7u7tRYqtMHyvqjyGuXbumQkJC1MiRI297388++0z985//LPF+bm5upWIJDQ1Vjz/+uEFtDf05vlVISEil9rNkxurz+TMnVKhfJ/WrVyd1YNtHRjlmVbmTPgNhqoycarIR9dkpU0u8rtwYXRVkZpa6PWVzYVnSvKtXS2yrSGJiIs7OzvriRM7OzjRv3rxYm8zMTIYNG8bHH39cYv/g4GB69OiBp6cnS5Ys0b8fEBCAr68v7u7ufPTR3+UUHRwceOmll+jVqxeHDh3CwcGBRYsW4eXlhZ+fHxcvXixxjj/++IPatWvj7OxMZGQk8+fP5/vvv9c/Yl008l+wYAGxsbF4e3szb968UvsbFxdH165dgcL62WPHjmXYsGHcfffdzJ8/HyiswV1U+nXy5MlA+TWlb+7P2rVr8fT0xMvLS1+itbx61lOmTGHgwIHcfffd+u9vRTXADXFrrezVq1fz1FN/ry49atQofTGsnTt30rt3b3x8fHjwwQf193dLHWzzlXr1MtlrJ5DvncnVf02mz/2Ws3iHMVnN1MeQIUOIj4+nQ4cOzJo1i59++qnY9vT0dEaPHs2kSZNKrOSyc+dOTp06xa+//kpkZCTh4eHs27cPgFWrVhEeHk5YWBgrV64kOTkZgIyMDLp27covv/zCPffcQ0ZGBn5+fkRFRdG/f/9SfxkcOHAAHx8fALy9vQkMDGTChAlERkYWqygXFBREu3btiIyMJDg42KD+R0ZGsmHDBqKjo9mwYQPx8fEEBQXpS7+uW7euWE3pyMhIbG1t9cWkbu5Pw4YNeeWVV9i7dy9RUVG88847AMyePZs5c+bw22+/sWnTpmIPCB05coTvvvuOQ4cOERgYSGJiIkFBQfTr14/IyEjmzJlDTExMsaXSbn6lpKSU2bfw8HC2bdtW7jTK5cuXefnll9m9eze///473bt3Z8WKFUBhxcOiOtjCfFy5GM+hl4fQMj+eJg+/xZDpL5o6JJMx2R3irT9fW+r7aWlp2Oh0ZW4HqNWwYbnbS+Pg4EB4eDj79+8nJCSECRMmEBQUxPTp0wF44IEHmD9/vn5kebOdO3eyc+dOfZGk9PR0Tp06Rf/+/Vm5cqV+AYL4+HhOnTpF48aNsbW1Zdy4cfpj2Nvb61dv8fX1ZdeuXSXOU1rda2MZNGgQDRo0AKBLly6cPXuWVq1aFWuzZ8+eMmtK39yfvXv3Mn78eJydnQFo1KgRUH496wceeACdTodOp2PAgAGEh4eXKL7UsWPHStULv7lWdlkOHz7M8ePH6du3cG4zJyeH3r1767dLHWzzkpGWyoHpw7nrXD4/L5zLwP5jTB2SSdXcR3lKYWtri7+/P/7+/nh4eLBmzRp9ou7bty8//PADkyZNKlEvWSnFwoULefLJJ4u9Hxoayu7duzl06BB169bF399fX5K0Tp062Nra6tva2dnpj1tWTWidTkdqaqoxu6x3c/3rss6vyqkpfXN/lFKl1pQur571re1L2z8mJka/wMCtQkNDcXJyKnXbzbWyy6rprZRi8ODBfPnll6UeQ+pgm4/cnGx2TOlP5zP5RI1qz8RHXjB1SCZnNVMfMTExxa7sR0ZG0rp1a/3XgYGBNG7cmFmzSi4BNnToUFatWqWf00xISODSpUukpqbSsGFD6taty8mTJzl8+PAdxdi5c2eDSqmWVzv7dtnZ2ZGbmwtQbk3pmw0aNIivvvpKP81TtEZkefWst23bRlZWFsnJyYSGhuLj41OiH0Uj6tJeZSXpW7m5uREZGUlBQQHx8fH8+uuvAPj5+XHgwAH99/f69ev88ccf+v2kDrZ5yM/LY8vUwiJLUfe6MHH5t6YOySxYTaJOT09n2rRpdOnSBU9PT44fP87SpUuLtXn77bfJysrSX2wrMmTIECZNmkTv3r3x8PBg/PjxpKWlMWzYMPLy8vD09GTx4sX4+fndUYz9+/cnIiKiwoVrGzduTN++fenatWuZFxMNNWPGDH396/JqSt/M3d2dRYsWce+99+Ll5cWzzxauX1lePeuePXsycuRI/Pz8WLx4Ma6uruXWAK+svn370qZNGzw8PJg7d65+zr9JkyasXr2ahx9+GE9PT/z8/Dh58iSA1ME2I5tfGotHZAZHfB158D+7TR2O+SjrdpA7eZnj7Xnm7OY+P/PMM2rXrl0mjMb4lixZooKDg4u9Z06f84oVK9Qnn3xiUFu5Pc9wt9vnw1++qvJfqq++/lc/lVfJWy1NrcbdnidK98ILL3D9+nVTh2FVpA626W0MnELT398iql4fAt7YXaMr4VWGQd8NTdPigDQgH8hTZaxCIO6ci4sL999/v8Hto6Oj9fcxF6ldu7b+qUtzcOsUk7mROtimtXX5LDquDyOqXVOGfbWRWnb2pg7J7NzOr60BSqmSz1kLk/Lw8KjULW1CmIMdnyzBbXUIfzlDr/c2UkdX/mr31kqmPoQQJrFv43s0fvcrUh3B7d1PcG1d8QLQ1sqgxW01TTsDXAUU8KFS6qNS2swAZgC4uLj4rl+/vtj2Bg0a0L59+wrPlZ+fX+z+Y2sgfbYcp0+frtS97unp6SYp8mVK5fU540oCBZ/+m8aXNBJmPkaTdj2qObqqcSef84ABA8pc3NbQRN1cKXVB07SmwC7gaaXUvrLa18RVyKuS9NlyyCrkhiurz5cvnCX748GQd52Evq/Rc9iUkjtbKJOuQq6UunDjv5eALUDPSkViYra2tnh7e+Pu7o6XlxcrVqwo9hRbaW63FKcQomznTkWx75nh2OakkjHmfzUqSVelChO1pmn1NE1zLPo3MAQ4WtWBVYWiAkTHjh1j165dfP/99yxbtqzcfSRRC2EcVy7Gc/zJh2l/QhHe6jE6+PibOiSLYciI2gX4WdO0KOBX4Dul1I9VGxZsjUigb9Be2iz4jr5Be9kakWDU4zdt2pSPPvqI9957D6UUcXFx9OvXDx8fH3x8fDh48CBQshRnWe2EEGVLT73CgWnDaJWoODWxJyNnlqwnI8pW4e15Sqk/Aa9qiEVva0QCCzdHk5lbWAs5ISWThZujAQjo1sJo52nbti0FBQVcunSJpk2bsmvXLurUqcOpU6d4+OGHCQsLIygoiOXLl7N9+3agsEZEae2EEKXLzclm51R/OscVcGR0ByYsXmPqkCyOWT7+E7wjRp+ki2Tm5hO8I8aoiRrQ19XIzc3lqaee0tdhvrlgz80MbSeEAFVQQMhbU3BNyCXSvxkPB28zdUgWySwT9YWUzNt6v7L+/PNPbG1tadq0KcuWLcPFxYWoqCgKCgqoU6dOqfu89dZbBrUTwlptjUggeEcME1ulcXzZE8zSdrD3kTFMfGaVqUOzWGaZqJs76UgoJSk3dzJeveCkpCRmzpzJU089haZppKam0rJlS2xsbFizZo1+CapbS3GW1U4IUXzaUhe7gmYpF/iqYz/s+v0bzUaer6sss/zOzRvaEZ1d8YchdHa2zBva8Y6OW7Q+oLu7O/fddx9DhgzRr384a9Ys1qxZg5+fH3/88Ye+GP2tpTjLaieE+HvacubllfTdf4ncjNosypnO8l2yyvudMMsRddE8dPCOGC6kZNLcSce8oR3veH66vNHv3XffzZEjR/RfF61yYmdnx549e4q1La2dEKJwevLpi28y4lAisXdpLO66kFwbndGnLa2NWSZqKEzWxr5wKISoWnMvvs7AQ0nEtLElbvzzpJ9yAow7bWmNzHLqQwhhWVRBAYc+e55udc9wtEMtnvdYgm2d+oBxpi2tXbUmakPqighhruTnt3T5eXl888oEep/9ALv2fbB9aTvOjRoD0MJJx2tjPeSv4ztUbVMfderUITk5mcaNG5e6ArUQ5kwpRXJystyOeYvcnGy2TOuLe2QG3zx4D6Oe+YIetraM7d6a0NBQnp7sb+oQa4RqS9QtW7bk/PnzJCUlldsuKyvL6v5nkD5bhjp16tCyZUtTh2E2sjOv8+2UvngczeJId0fGv7QNGwssXWsJqi1R29nZ0aZNmwrbhYaG0q1bt2qIyHxIn4WlyUhLZceU/rifzCGqTyMe/OgnWeewCsnFRCHEbcnOus4PLwyl88kcIv1dmLjqgCTpKiaJWghhsKzr6cS8fT9jG53g5LSePPxBqKlDsgqSqIUQBrmUEMuOh3phn3SUMM+ljFkoVfCqiyRqIUSF4mOPEvHIaNrFFhDtMJCe4+aYOiSrIolaCFGuP4//yslHH6T5RUXMI714cJmseFTdJFELIcr0R9R+zj4xjaZXIPaxAYxbtNrUIVkluVQrhCjVX/Gnsdk8g+tOBWRMup+AfwabOiSrJYlaCFHC7yGbqLv7eVrUSidvyed06jnY1CFZNUnUQohiDn73GSx5g78a18L29a/p5N3P1CFZPUnUQgi9fRvfo/ar/wHA8cmnuVuStFmQRC2EAGD350E4vbmGnFpg8+8F9Bs+zdQhiRskUQshOHbwO2w+WkNmbXAIepluA8aZOiRxE0nUQli56H3baLfnCTL7NqZOwAq6+g03dUjiFpKohbBiW4NnYnt4LzpPF9z+9T3OzVqZOiRRCnngRQgrtSlwKu0++4k6l2tRZ/IXkqTNmIyohbBCXy16EPfNR4l31fD4dCMt2nQxdUiiHJKohaihtkYkELwjhgspmTR30jFvaEcCurVg/bz78fj2FHGtbOi1djvOrhUv6CFMSxK1EDXQ1ogEFm6OJjM3H4CElEwWbo5GHfoPbbPDOd2+If0/20HDJrLorCWQOWohaqDgHTH6JF3k0bQPGHPpP9Rq5cPwTWGSpC2IJGohaqALKZl/f6HyefHsMkb+GMNnVzzw/Ncm7Gtb1sLC1k4StRA1UHMnXeE/VD7L/lxK34g0orrY80nTRdSyszdtcOK2SaIWogaaN7QjjjbZvHpqMT2jM/nNQ0dg51eYN9LT1KGJSpCLiULUQANa27Ms7RU6H8/hQDdHPu36MkEj3AnoJvPSlkgStRA1TOyxX7DfOJ1RjS7yw4wJPP7sf3nc1EGJOyKJWogaZPt7c2m05jvs+meR/tCXPNBrqKlDEkYgiVqIGmLDc6Nw/z6WpEZQZ2QwnSVJ1xiSqIWwcJkZ1/j28YF4RmRwuo0t3T/YiGvrTqYOSxiRwYla0zRbIAxIUEqNqrqQhBCGSr16mZ3zh+ARkckRX0cCPgmltq6uqcMSRnY7t+fNBk5UVSBCiNtz7o8jpLx7LyOaxXFskg8T1v0qSbqGMihRa5rWEhgJfFK14QghDPHdf57nj8cnkJeVRtzw/zH+pXWmDklUIU0pVXEjTdsIvAY4AnNLm/rQNG0GMAPAxcXFd/369ZUKKD09HQcHh0rta6mkzzWfMfsbt/FVfPfGk+wEF6dPxaVDb6Mc19is7TOGO+vzgAEDwpVS3UvdqJQq9wWMAt6/8W9/YHtF+/j6+qrKCgkJqfS+lkr6XPMZo79Z1zPUhknd1fGOndS3Q7qo838eu/PAqpC1fcZK3VmfgTBVRk415GJiX+B+TdNGAHWA+pqm/U8p9Uilfm0IIW7btZRkdjw1CI/wbKJ9HBj98R509eqbOixRTSqco1ZKLVRKtVRKuQETgb2SpIWoPgl/nuDKynvpd1cCRwI689AXv0mStjJyH7UQZuz7D16g4NvN9PK5xsWRq5jQd7SpQxImcFuJWikVCoRWSSRCiGI2zA+gy/YYkp00zg9+n26SpK2WjKiFMDPZmdfZ+oQ/nmFpxN5lg9d/N9CqXVdThyVMSBK1EGYkLfUKOx73xzM6l2jveoz+dK/MRwtJ1EKYiwtxMWSvfRD31pc4cld3Jry53dQhCTMhK7wIUYW2RiTQN2gv0Qmp9A3ay9aIhFLb/fjxYn5beD8N8y+Td/+HkqRFMTKiFqKKbI1IYOHm6MLVwFtBQkomCzdHAxRbaeWrhePovO04VxrYcOHZNXj0GGyqkIWZkhG1EFUkeEdMYZK+SWZuPsE7YoDCi4YbpvTCY8txzrW0oc3a9XSRJC1KISNqIarIhZTMMt9Pv3aVnVP64xmTx1EvHSM+CaGeY4NqjlBYChlRC1FFmjvpSn3f0zGVS2/707RFClHDWvPght8lSYtySaIWoorMG9oRnZ1tsfdGZn7DzLglOBdcosGEd5j49o8mik5YEpn6EKKKFF0wLJyTTmPW1f8w7OczXGlQi4tPb8HD08+0AQqLIYlaiCoU0K0FI92d+eqxfviEpXGmpQ1d3l2DW+fSyw4LURpJ1EJUoQtnY/h9xlh8zhZwzKMOw1eFyny0uG0yRy1EFfn9x9XU+2wAmks2v9zXijFf/iZJWlSKjKiFMLJLCbHsmz2eu5pf4lLT5ngu/pTYhKvY1pL/3UTlyIhaCCPa/XkQJ8eNovPRLP7Mbofb/AO0utvL1GEJCye/4oUwguzM62yePZyuP1/iWj2Imx3AxP97zdRhiRpCErUQdyjxbAy/BE/Ee18WJzva4bv8c/rKKFoYkUx9CHEHdq8NpN5n/gxslMiJ6X24f9Pv3CVJWhiZjKiFqITEsyc5OGcibqezOXp/C1o/9jlj23Y2dViihpJELcRt2rFqGXU/XE+naxDdpzEPLPxeVmERVUoStRAGysnOYvOs+/A4mMxVRzg/dwIT/7HU1GEJKyCJWggDJPx5grQvHsXhSjInO9Wl91vrae4mUx2iekiiFqIc+Xl5bFryMB2yDtPWMY/UaYvpFTDL1GEJKyOJWogyxMceJezZSXjE5HKiSwNc3tpEr9YdTR2WsEKSqIUoxfcfvECDVVtonw6R/Zsy9p0fqK2ra+qwhJWSRC3ETXJzstkW+CCdN54i2QkuLpjCw1NfMHVYwspJohbihjMnw8ndOJP7tT/Y0rsjA17+kqYt2pk6LCEkUQuRn5fHxhfG4/JzDG0HXOPoPW8zYfijpg5LCD1J1MKqnT0ZTsTc6XiezuNMKxvSA1bj02OwqcMSohhJ1MJqbX9vLo1Wf0e76xA10JVxb/+AnX1tU4clRAmSqIXVycnO4vfPnsX22z3k2NuQ9Nw/mPjwXFOHJUSZJFELq3Lwu8+wPfgOfnZn2HvfULo+8jrOrm1MHZYQ5ZJELaxCduZ1Ni8YS+c9ZznfWhEx+z8MHPKIqcMSwiCSqEWN9/0HL1B73Ra8kyC2tQ3tA9+lk+9AU4clhMEkUYsa6+L5WPa9PoWuu65ypT5sHtiJjx0fpcUumGeTQEC3FqYOUQiDSKIWNU5KciKH1r7EvSnb8K8PP/S5m3dd/o8k1RCAhJRMFm6OBpBkLSyCLMUlaoz8vDy2vP4k0SMGotu4nxN1fcl9fB8fdvi3PkkXyczNJ3hHjIkiFeL2yIha1AiHf1hD0jtv0CmugL8aQfpDw/Cf/TYAF1JiS93nQkpmNUYoROVVmKg1TasD7ANq32i/USm1pKoDE8IQ6deusmvFo7T/Kga7WhB1X0tGv7aReo4N9G2aO+lIKCUpN3fSVWeoQlSaIVMf2cBApZQX4A0M0zTNr0qjEqICuTnZ7Pj4BbJWeDOqIITj3R1wWPshE9/bVSxJA8wb2hGdnW2x93R2tswbKrWlhWWocEStlFJA+o0v7W68VFUGJUR59n6xnNyPV+GUqogf7cyVgDVM8PEvs33RBcPgHTFcSMmkuZOOeUM7yoVEYTEMmqPWNM0WCAfaA/9RSv1SpVEJUYrY6INEBP6TztFZpNeFuCGdGLtovUH1OQK6tZDELCyWVjhgNrCxpjkBW4CnlVJHb9k2A5gB4OLi4rt+/fpKBZSeno6Dg0Ol9rVU0ufyFeTncemXdbTfcJjauXDUsy4O4+fg6NyyiqM0HvmMrcOd9HnAgAHhSqnupW5USt3WC1gCzC2vja+vr6qskJCQSu9rqaTPZftp07vqz2UeSi2pr756xFMd2PZR1QZWReQztg530mcgTJWRUw2566MJkKuUStE0TQfcB7xeqV8ZQhjoyIFviX19EW1jc7kyOp+rA95j/EuT0Wzk1n9hfQyZo3YF1tyYp7YBvlJKba/asIS1SklO5MfnH6LT4cu0BU70dmboc+tp2ETml4X1qnB4opQ6opTqppTyVEp1VUoFVkdgombaGpFA36C9RCek0jdoL1sjEgBQBQX8sv0TokYNxOvny8S1qYX6bxATP9kvSVpYPXkyUVSbrREJLNwcTWZuPrT6u+ZGyvFd+MSvpVdWOFs6tiSt+3ACnlpu6nCFMBuSqEW1Cd4RU5ikb2iS/xf/PPM+vpuukz7kOod7zWf0onnUsrM3YZRCmB9J1KLaFNXWsFE52B/4L+8ejsfxOkR1tqfnY5/S3rOPiSMUwjxJohbVxrV+bTqk/8LYiHXcfUZxzlXjk3sGcKL5OCZJkhaiTJKoRZXLSEvlhzdnsUpF0ck+nm9dXdjXti2vOzxJbXsdr0nNDSHKJYlaVJnLiWfY/dpMWh4+h/s1ODPYhmivRbxr78mDrbNxjXeUmhtCGEAStTC6xLOn+CnwUdr+noxXJpxtoZE0rh+j5qzEzr42DwKhoaE8Pdnf1KEKYREkUQujiT32C5dDPqTLxW9xPd6YRNdaXB07jmGPLzN1aEJYNEnU4o6Ffv0uV9Z9QvNzOXQccYkTzkNweXcG93YfYOrQhKgRJFGLSsnPy+OHDxfCN9/T7mwB9e3hlKcjzhM+paeH3MEhhDFJoha3JT8vj6hdn5OydyXtvs0itS5E3etC33nv4tPew9ThCVEjSaIWBklJTmTHqzOwv3SaMe0ucN7BlUMPejL4X+/g19jV1OEJUaNJohblio89yoE3nsLtt4t4Xi+8gyNswgq6DZvGg7Xkx0eI6iD/p4lSXUo4Q+hbM2m/8xxeORDb2obLo0cwfOZr2EqCFqJayf9xophD2z/lavhm7is4hI9mx5E2LjSa9BijJvzL1KEJYbUkUQsAdqxaRtbGr+nwZz7J7QuIGBZAq5HzGdumk6lDE8LqSaK2YgX5+Xz7zmxqb99L6wuK9DoQ1acxPea8QTu5xU4IsyGJ2gqdijrAxaMhtIzdQK2T13FMtydqyF3ct/ADeri2MXV4QohbSKK2EudifufQJ8twiDqNW3wB2sB08ps1xmnMP/Ee9g/6OjYwdYhCiDJIoq7BrqUkE/H9h1xbuw63cwV4FkCSE0T7NaTNqAW0HTqFdrKqtxBmTxJ1DXMpIZbQDxbBpdMENDtDP/LYkdec493r02T4Q9z74Gy+jb7IsztiuPDTDzR30kmpUSHMnCTqGuDKxXj2friIWr9G0CYuD488uNAEwr3G0qjnwwxb4o92Y+RcbIFZ/l5gFpBkLYSZkkRtoVKSE/nzt12oY1tIOHQM96O2pNaFmK71qD9oOIOmvYidfe0S+926wCxAZm4+wTtiJFELYaYkUVuQ9NQr7P5oEfkHD+IWm4P9fSm0rG/LX57diB3QicGPLcNPV7fcYxQtMGvo+0II05NEbebycnP49YfPuLDmA9xis+iYBRm14czdtWng9U86PTyPkaWMnMvS3ElHQilJubmTzphhCyGMSBK1GcrOvM7uz5aRGRfOwLox+OZfQ51rztk29tj27s19T75K9waNKnXseUM7FpujBtDZ2TJPFpgVwmxJojYTuTnZ7FnzMtf2/MBdpzJomwEJTRVnHvBF6zoO7+fvp57Dnd/rXDQPHbwjhgspmXLXhxAWQBK1CaWnXiH+ZBi5v67h+3dm0uGERk4tOONWi/PdvfB/8t84V8GTggHdWkhiFsKCSKKuJvl5eRw58C2n936NOn2KBhcyaHZJUX9kMvfq8tjRvjNH3VvSb8bLeN11t6nDFUKYEUnUVSTx7EnCv/mYWgXpNMs+y+Wzp2mxx46uQHYt+MtF45iXA9E2g3B168OGi82YN7Qjze6Ska4QojhJ1EaQn5dHTHgI0V+/i+2ZczT+K5umydAOOOeXRaO2Oq627ELUiHya9hhIr1GPE3c6nX/fuKj3nH2ePHgihCiTJOpKOPHbbo7v/IK8mOPU0mUxvNkFWhfkUPCdK9frwMVmtlzs3BCdezd8Rj9Bq/Ye3HXLMYJ3hMuDJ0IIg5hNot4akUDwjhgmtkpjUdDearsToei8Zd0BkZaazPmTYaSeOsj5rVtxPZtDozToAuTZwCl3G6I9RmF7V09sBzXF9577DVqqSh48EUIYyiwSdbH6E62qr/7ErXUvEq+m8vX//kPy2mPUvxBPw8RMNBSDBiYWts9vzqXmdsTf1YxGvv3oOfpxPCq5Arc8eCKEMJRZJOrqqj+ReuUiZ44d5lJsNBl/nSPu9Bn+lZlG7dbNaWt7iZSjSdx9orB4UaY9JDazIa1FAyL6LKaVZ38CmrUyWizy4IkQwlBmkajvZBogIy2V1Ct/kXH1EueOHSD5yC/kX72CbVoGdhk51MnIw9VP0dYuhYOxOtx+q01Rui1aDdDurgSytYbEtGzGiUa1iXDsxudvBOJzG49m366bHzyBNFrIgydCiDKYRaK+dRqglsqhVd5ZOtdK4Ju3d+LQsDF1tVwuxR0j7/dYal/Po25GAfWug2Mm5I5IwbP+dc5fcMJjX2FRouv2kF4PrtfVSLJrQXaTPuTqcjnieo3aTZrh2LwtH5+sRVR2CzKVA+QA9QpfLZx0pVaeM7aiB09CQ0N5erJ/lZ9PCGGZzCJRzxvakQWbonj7+nyc3sxnWybYqL+3Jw9Mx6/pNQ5fbUBuYj2u19VIbViLyy3tyHeoR8MOo8hv60UDuzpkTbbFrUtPGjapeGSaFpFA1OZokOkHIYQZM4tEXfTnfsqXzUh1SyK9Th1snBpQ39WVes3uonU3fzI6+uLn6FQl55W6F0IIc1ZhotY0rRWwFmgGFAAfKaXeMXYgAd1aQLfdhIaGMsHf39iHL/e8kpiFEObMkBF1HvCcUup3TdMcgXBN03YppY5XcWxCCCGACpegVkolKqV+v/HvNOAEIENQIYSoJppSquJWRY01zQ3YB3RVSl27ZdsMYAaAi4uL7/r16ysVUHp6Og4ODpXa11JJn2s+a+svSJ9v14ABA8KVUt1L3aiUMugFOADhwNiK2vr6+qrKCgkJqfS+lkr6XPNZW3+Vkj7fLiBMlZFTK5z6ANA0zQ7YBKxTSm2u1K8LIYQQlVJhotY0TQM+BU4opVZUfUhCCCFuZsiIui8wBRioaVrkjdeIKo5LCCHEDRXenqeU+hnQqiEWIYQQpbituz4MPqimJQFnK7m7M3DZiOFYAulzzWdt/QXp8+1qrZRqUtqGKknUd0LTtDBV1i0qNZT0ueaztv6C9NmYDLrrQwghhOlIohZCCDNnjon6I1MHYALS55rP2voL0mejMbs5aiGEEMWZ44haCCHETSRRCyGEmTNJotY0bZimaTGapp3WNG1BKds1TdNW3th+RNM0H1PEaUwG9Hnyjb4e0TTtoKZpXqaI05gq6vNN7Xpompavadr46oyvKhjSZ03T/G884XtM07SfqjtGYzPgZ7uBpmnfapoWdaPPj5oiTmPRNG2VpmmXNE07WsZ24+evsqo1VdULsAVigbaAPRAFdLmlzQjgBwqfiPQDfqnuOE3Q5z5Awxv/Hm4Nfb6p3V7ge2C8qeOuhs/ZCTgO3HXj66amjrsa+vwC8PqNfzcBrgD2po79DvrcH/ABjpax3ej5yxQj6p7AaaXUn0qpHGA98MAtbR4A1qpChwEnTdNcqztQI6qwz0qpg0qpqze+PAy0rOYYjc2QzxngaQorM16qzuCqiCF9ngRsVkqdA1BKWXq/DemzAhxvFHhzoDBR51VvmMajlNpHYR/KYvT8ZYpE3QKIv+nr85RcMcaQNpbkdvvzDwp/I1uyCvusaVoLYAzwQTXGVZUM+Zw7AA01TQvVNC1c07Sp1RZd1TCkz+8BnYELQDQwWylVUD3hmYTR85cpViEvrcDTrfcIGtLGkhjcH03TBlCYqO+p0oiqniF9fht4XimVXzjYsniG9LkW4AsMAnTAIU3TDiul/qjq4KqIIX0eCkQCA4F2wC5N0/arW1aJqkGMnr9MkajPA61u+rolhb9pb7eNJTGoP5qmeQKfAMOVUsnVFFtVMaTP3YH1N5K0MzBC07Q8pdTWaonQ+Az92b6slMoAMjRN2wd4AZaaqA3p86NAkCqcwD2tadoZoBPwa/WEWO2Mnr9MMfXxG3C3pmltNE2zByYC39zS5htg6o2rp35AqlIqsboDNaIK+6xp2l3AZmCKBY+ublZhn5VSbZRSbkopN2AjMMuCkzQY9rO9DeinaVotTdPqAr0oXDDaUhnS53MU/gWBpmkuQEfgz2qNsnoZPX9V+4haKZWnadpTwA4KrxivUkod0zRt5o3tH1B4B8AI4DRwncLfyBbLwD6/BDQG3r8xwsxTFlx5zMA+1yiG9FkpdULTtB+BI0AB8IlSqtTbvCyBgZ/zv4HVmqZFUzgt8LxSymLLn2qa9iXgDzhrmnYeWALYQdXlL3mEXAghzJw8mSiEEGZOErUQQpg5SdRCCGHmJFELIYSZk0QthBBmThK1EEKYOUnUQghh5v4fObcLQEsi2YwAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_107_1.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "from sklearn.linear_model import LinearRegression\n", "\n", "\n", "np.random.seed(2021)\n", "\n", "def MSE(y_data,y_model):\n", " n = np.size(y_model)\n", " return np.sum((y_data-y_model)**2)/n\n", "\n", "\n", "def fit_beta(X, y):\n", " return np.linalg.pinv(X.T @ X) @ X.T @ y\n", "\n", "\n", "true_beta = [2, 0.5, 3.7]\n", "\n", "x = np.linspace(0, 1, 11)\n", "y = np.sum(\n", " np.asarray([x ** p * b for p, b in enumerate(true_beta)]), axis=0\n", ") + 0.1 * np.random.normal(size=len(x))\n", "\n", "degree = 3\n", "X = np.zeros((len(x), degree))\n", "\n", "# Include the intercept in the design matrix\n", "for p in range(degree):\n", " X[:, p] = x ** p\n", "\n", "beta = fit_beta(X, y)\n", "\n", "# Intercept is included in the design matrix\n", "skl = LinearRegression(fit_intercept=False).fit(X, y)\n", "\n", "print(f\"True beta: {true_beta}\")\n", "print(f\"Fitted beta: {beta}\")\n", "print(f\"Sklearn fitted beta: {skl.coef_}\")\n", "ypredictOwn = X @ beta\n", "ypredictSKL = skl.predict(X)\n", "print(f\"MSE with intercept column\")\n", "print(MSE(y,ypredictOwn))\n", "print(f\"MSE with intercept column from SKL\")\n", "print(MSE(y,ypredictSKL))\n", "\n", "\n", "plt.figure()\n", "plt.scatter(x, y, label=\"Data\")\n", "plt.plot(x, X @ beta, label=\"Fit\")\n", "plt.plot(x, skl.predict(X), label=\"Sklearn (fit_intercept=False)\")\n", "\n", "\n", "# Do not include the intercept in the design matrix\n", "X = np.zeros((len(x), degree - 1))\n", "\n", "for p in range(degree - 1):\n", " X[:, p] = x ** (p + 1)\n", "\n", "# Intercept is not included in the design matrix\n", "skl = LinearRegression(fit_intercept=True).fit(X, y)\n", "\n", "# Use centered values for X and y when computing coefficients\n", "y_offset = np.average(y, axis=0)\n", "X_offset = np.average(X, axis=0)\n", "\n", "beta = fit_beta(X - X_offset, y - y_offset)\n", "intercept = np.mean(y_offset - X_offset @ beta)\n", "\n", "print(f\"Manual intercept: {intercept}\")\n", "print(f\"Fitted beta (wiothout intercept): {beta}\")\n", "print(f\"Sklearn intercept: {skl.intercept_}\")\n", "print(f\"Sklearn fitted beta (without intercept): {skl.coef_}\")\n", "ypredictOwn = X @ beta\n", "ypredictSKL = skl.predict(X)\n", "print(f\"MSE with Manual intercept\")\n", "print(MSE(y,ypredictOwn+intercept))\n", "print(f\"MSE with Sklearn intercept\")\n", "print(MSE(y,ypredictSKL))\n", "\n", "plt.plot(x, X @ beta + intercept, \"--\", label=\"Fit (manual intercept)\")\n", "plt.plot(x, skl.predict(X), \"--\", label=\"Sklearn (fit_intercept=True)\")\n", "plt.grid()\n", "plt.legend()\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The intercept is the value of our output/target variable\n", "when all our features are zero and our function crosses the $y$-axis (for a one-dimensional case). \n", "\n", "Printing the MSE, we see first that both methods give the same MSE, as\n", "they should. However, when we move to for example Ridge regression,\n", "the way we treat the intercept may give a larger or smaller MSE,\n", "meaning that the MSE can be penalized by the value of the\n", "intercept. Not including the intercept in the fit, means that the\n", "regularization term does not include $\\beta_0$. For different values\n", "of $\\lambda$, this may lead to differeing MSE values. \n", "\n", "To remind the reader, the regularization term, with the intercept in Ridge regression, is given by" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\lambda \\vert\\vert \\boldsymbol{\\beta} \\vert\\vert_2^2 = \\lambda \\sum_{j=0}^{p-1}\\beta_j^2,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "but when we take out the intercept, this equation becomes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\lambda \\vert\\vert \\boldsymbol{\\beta} \\vert\\vert_2^2 = \\lambda \\sum_{j=1}^{p-1}\\beta_j^2.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For Lasso regression we have" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\lambda \\vert\\vert \\boldsymbol{\\beta} \\vert\\vert_1 = \\lambda \\sum_{j=1}^{p-1}\\vert\\beta_j\\vert.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It means that, when scaling the design matrix and the outputs/targets,\n", "by subtracting the mean values, we have an optimization problem which\n", "is not penalized by the intercept. The MSE value can then be smaller\n", "since it focuses only on the remaining quantities. If we however bring\n", "back the intercept, we will get a MSE which then contains the\n", "intercept.\n", "\n", "\n", "Armed with this wisdom, we attempt first to simply set the intercept equal to **False** in our implementation of Ridge regression for our well-known vanilla data set." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Beta values for own Ridge implementation\n", "[ 1.03032441e+00 6.28336218e-02 -6.24175744e-01 5.21169159e-02\n", " 2.80847477e-01 2.12552073e-01 8.13220608e-02 -1.69634577e-02\n", " -6.50846112e-02 -7.38962192e-02 -5.94226022e-02 -3.50227564e-02\n", " -9.80609615e-03 1.08299273e-02 2.41882037e-02 2.93492130e-02\n", " 2.64742912e-02 1.63249532e-02 -5.01831130e-05 -2.15098090e-02]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[ 1.03032441e+00 6.28336218e-02 -6.24175744e-01 5.21169159e-02\n", " 2.80847477e-01 2.12552073e-01 8.13220608e-02 -1.69634577e-02\n", " -6.50846112e-02 -7.38962192e-02 -5.94226022e-02 -3.50227564e-02\n", " -9.80609615e-03 1.08299273e-02 2.41882037e-02 2.93492130e-02\n", " 2.64742912e-02 1.63249532e-02 -5.01831200e-05 -2.15098090e-02]\n", "MSE values for own Ridge implementation\n", "4.3632959170548605e-07\n", "MSE values for Scikit-Learn Ridge implementation\n", "4.363295916414895e-07\n", "Beta values for own Ridge implementation\n", "[ 1.03630548 -0.01963611 -0.37900111 -0.07062318 0.12182967 0.16343471\n", " 0.13003291 0.07490892 0.02365049 -0.01449782 -0.03814292 -0.04909093\n", " -0.05009826 -0.04389027 -0.03279636 -0.01866537 -0.00289724 0.01348565\n", " 0.02976145 0.04543942]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[ 1.03630548 -0.01963611 -0.37900111 -0.07062318 0.12182967 0.16343471\n", " 0.13003291 0.07490892 0.02365049 -0.01449782 -0.03814292 -0.04909093\n", " -0.05009826 -0.04389027 -0.03279636 -0.01866537 -0.00289724 0.01348565\n", " 0.02976145 0.04543942]\n", "MSE values for own Ridge implementation\n", "5.194042826653172e-06\n", "MSE values for Scikit-Learn Ridge implementation\n", "5.194042826815498e-06\n", "Beta values for own Ridge implementation\n", "[ 1.04220758 -0.10931453 -0.17641709 -0.06020587 0.02208512 0.05789007\n", " 0.06491736 0.05785343 0.04537385 0.03196357 0.01969145 0.00934499\n", " 0.00107405 -0.00526348 -0.00992331 -0.01318643 -0.01531845 -0.01655318\n", " -0.01708852 -0.01708781]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[ 1.04220758 -0.10931453 -0.17641709 -0.06020587 0.02208512 0.05789007\n", " 0.06491736 0.05785343 0.04537385 0.03196357 0.01969145 0.00934499\n", " 0.00107405 -0.00526348 -0.00992331 -0.01318643 -0.01531845 -0.01655318\n", " -0.01708852 -0.01708781]\n", "MSE values for own Ridge implementation\n", "2.0940821989673748e-05\n", "MSE values for Scikit-Learn Ridge implementation\n", "2.0940821989624095e-05\n", "Beta values for own Ridge implementation\n", "[ 1.01219292 -0.06043581 -0.10391807 -0.05651951 -0.01898855 0.00312361\n", " 0.01463049 0.01975848 0.02123176 0.02068067 0.01905883 0.01691985\n", " 0.01458337 0.01223198 0.00996754 0.00784393 0.00588657 0.00410387\n", " 0.00249435 0.00105081]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[ 1.01219292 -0.06043581 -0.10391807 -0.05651951 -0.01898855 0.00312361\n", " 0.01463049 0.01975848 0.02123176 0.02068067 0.01905883 0.01691985\n", " 0.01458337 0.01223198 0.00996754 0.00784393 0.00588657 0.00410387\n", " 0.00249435 0.00105081]\n", "MSE values for own Ridge implementation\n", "0.0003153514830958126\n", "MSE values for Scikit-Learn Ridge implementation\n", "0.0003153514830958081\n", "Beta values for own Ridge implementation\n", "[ 8.38916861e-01 1.31276579e-01 8.97497404e-03 -1.72271878e-02\n", " -2.11744554e-02 -1.91492986e-02 -1.57201944e-02 -1.23002365e-02\n", " -9.30466214e-03 -6.81048318e-03 -4.78184120e-03 -3.15130074e-03\n", " -1.84923989e-03 -8.13661243e-04 7.46984697e-06 6.56636616e-04\n", " 1.16805821e-03 1.56912044e-03 1.88168312e-03 2.12318726e-03]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[ 8.38916861e-01 1.31276579e-01 8.97497404e-03 -1.72271878e-02\n", " -2.11744554e-02 -1.91492986e-02 -1.57201944e-02 -1.23002365e-02\n", " -9.30466214e-03 -6.81048318e-03 -4.78184120e-03 -3.15130074e-03\n", " -1.84923989e-03 -8.13661243e-04 7.46984697e-06 6.56636616e-04\n", " 1.16805821e-03 1.56912044e-03 1.88168312e-03 2.12318726e-03]\n", "MSE values for own Ridge implementation\n", "0.015072388895177109\n", "MSE values for Scikit-Learn Ridge implementation\n", "0.015072388895177088\n", "Beta values for own Ridge implementation\n", "[0.37396662 0.14174745 0.0764924 0.04892055 0.03447512 0.02586427\n", " 0.02024962 0.01633913 0.01347916 0.0113104 0.0096208 0.00827728\n", " 0.00719176 0.00630331 0.00556826 0.0049544 0.00443743 0.0039987\n", " 0.0036237 0.003301 ]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[0.37396662 0.14174745 0.0764924 0.04892055 0.03447512 0.02586427\n", " 0.02024962 0.01633913 0.01347916 0.0113104 0.0096208 0.00827728\n", " 0.00719176 0.00630331 0.00556826 0.0049544 0.00443743 0.0039987\n", " 0.0036237 0.003301 ]\n", "MSE values for own Ridge implementation\n", "0.2640931530791005\n", "MSE values for Scikit-Learn Ridge implementation\n", "0.2640931530791003\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAArZUlEQVR4nO3deXhU5f338fc3ISGBICCgbCKooCIkLAFEBUTZjYAkVtyVeqGt1FrFqj+fy+rP9pHWPnWrldKq3dzaBBRlEVQoKiIJAhFEBJFqBAVRdgJJ5n7+mEk6hAkJISdnZvJ5eeXKnP1zYsh37vucuY855xAREakswe8AIiISnVQgREQkIhUIERGJSAVCREQiUoEQEZGIGvkdoC61bt3ade7c2e8YIiIxY8WKFd8659pEWhZXBaJz584UFBT4HUNEJGaY2X+qWqYuJhERiUgFQkREIlKBEBGRiOLqGkQkJSUlFBUVUVxc7HcUiSIpKSl07NiRpKQkv6OIRK24LxBFRUU0a9aMzp07Y2Z+x5Eo4Jxjx44dFBUV0aVLF7/jiEStuO9iKi4uplWrVioOUsHMaNWqlVqVItWI+wIBqDjIEfQ7IVK9BlEgRETi1ZznH+CxX19GSfH+Ot+3CkQ9MDOuvfbaiunS0lLatGlDVlYWAN988w1ZWVlkZGTQvXt3xowZA8DmzZtJTU2lV69eFV9/+9vf6jV7eIbu3btz3XXXUVJSAkBBQQG33XZbxO06d+7Mt99+e1zHfu655yrOOzk5mZ49e9KrVy/uueeeGu/jscceY//+uv+HIxItniz4Pb/f9jqNklPqfN9xf5E6GjRt2pQ1a9Zw4MABUlNTWbhwIR06dKhYfv/99zN8+HB++tOfAlBYWFix7PTTT2fVqlX1Hfkw5RnKysoYPnw4//znP7n66qvJzMwkMzPTs+PeeOON3HjjjUCw4CxatIjWrVsf0z4ee+wxrrnmGpo0aeJFRBFffb/1c95qtoM7AwOwhLp/v68WRD0ZPXo0c+bMAeDFF1/kyiuvrFi2detWOnbsWDGdnp5+TPt+66236N27Nz179mTSpEkcPHiQ5cuXM2HCBABeffVVUlNTOXToEMXFxZx22mkAXHjhhdx9993079+fbt268c477xz1OImJifTv35+vvvoKgMWLF1e0gnbs2MGIESPo3bs3N998M+FPKnzooYc466yzGD58OFdeeSW//e1vAfjss88YNWoUffv2ZdCgQXzyySc1Ot9HHnmEfv36kZ6ezi9+8QsA9u3bxyWXXEJGRgY9evTg5Zdf5oknnmDLli0MHTqUoUOHHsNPVCQ2zJ75fylNhOwhP/Jk/w2rBXH77VDX78Z79YLHHqt2tYkTJ/K///u/ZGVlUVhYyKRJkyr+IN96661cccUV/P73v2fYsGHceOONtG/fHgj+Ee3Vq1fFfp588kkGDRpUMV1cXMwNN9zAW2+9Rbdu3bjuuut4+umnmTJlCitXrgTgnXfeoUePHuTn51NaWsqAAQMqti8tLWX58uXMnTuXBx98kDfffLPKcyguLuaDDz7g8ccfP2LZgw8+yAUXXMD999/PnDlzmDFjBhDshsrLy2PlypWUlpbSp08f+vbtC8DkyZOZPn06Xbt25YMPPuDHP/4xb7/99lF/jgsWLGDDhg0sX74c5xxjx45lyZIlbN++nfbt21cU4V27dtG8eXN+97vf1arlIRILcjfOplOjRDIvvrb6lWuhYRUIH6Wnp7N582ZefPHFimsM5UaOHMmmTZuYP38+8+bNo3fv3qxZswaovotp/fr1dOnShW7dugFw/fXX89RTT3H77bdzxhlnsG7dOpYvX84dd9zBkiVLKCsrO6zAlLcy+vbty+bNmyMeo7xIbdiwgZycnIgtnCVLljBz5kwALrnkElq2bAnAu+++y7hx40hNTQXg0ksvBWDv3r0sXbqUyy+/vGIfBw8erPI8yy1YsIAFCxbQu3fviv1s2LCBQYMGMXXqVO6++26ysrIOO0eReLR7exEL0rZxa1kfT7qXoKEViBq80/fS2LFjmTp1KosXL2bHjh2HLTvxxBO56qqruOqqq8jKymLJkiUV77SPJrwrp7JBgwYxb948kpKSGDZsGDfccANlZWUVXTwAjRs3BoLdR6WlpRH3U16ktm7dyoUXXsjs2bMZO3bsEetFunW0qnyBQIAWLVoc8/UV5xz33nsvN9988xHLVqxYwdy5c7n33nsZMWIE999//zHtWySWvJ73MIcaQc7AyZ4dQ9cg6tGkSZO4//776dmz52Hz33777Yo7bfbs2cNnn31Gp06darTPs846i82bN7Nx40YA/v73vzNkyBAABg8ezGOPPcbAgQNp06YNO3bs4JNPPuGcc86pVf527doxbdo0Hn744SOWDR48mOeffx6AefPm8f333wNwwQUX8Nprr1FcXMzevXsruoBOOOEEunTpwr/+9S8g+Id/9erV1WYYOXIkzz77LHv37gXgq6++Ytu2bWzZsoUmTZpwzTXXMHXqVD788EMAmjVrxp49e2p1viLRLHf9LNrvS+DckT/07BgNqwXhs44dO1bcqRRuxYoVTJkyhUaNGhEIBLjpppvo168fmzdvPuIaxKRJkw67tTQlJYXnnnuOyy+/nNLSUvr168ctt9wCwIABA/jmm28YPHgwEOzmOumkk47rQ2Ljx4/ngQceOOKC9i9+8QuuvPJK+vTpw5AhQyoKXL9+/Rg7diwZGRmceuqpZGZm0rx5cwCef/55fvSjH/HLX/6SkpISJk6cSEZGxlGPP2LECNatW8fAgQMBSEtL4x//+AcbN27krrvuIiEhgaSkJJ5++mkgeJ1j9OjRtGvXjkWLFtX6vEWiyd7vvmZe063cVNqThETv/ozb0boojnvnZqOAx4FE4M/OuWmVll8N3B2a3Av8yDm3OrRsM7AHKANKnXPV3k+ZmZnpKj8waN26dZx99tnHeSZyPPbu3UtaWhr79+9n8ODBzJgxgz59+vgdS78bErP+9cwd/KDoURb3eowh445803kszGxFVX9fPSs9ZpYIPAUMB4qAfDOb7Zz7OGy1z4EhzrnvzWw0MAMYELZ8qHPu+D5tJb6bPHkyH3/8McXFxVx//fVRURxEYlnu2lxOSjIuGOPN7a3lvOxi6g9sdM5tAjCzl4BxQEWBcM4tDVt/GdARiTsvvPCC3xFE4saB3d8xJ/VLrik5m8SkZE+P5eVF6g7Al2HTRaF5VfkhMC9s2gELzGyFmVV5md7MJptZgZkVbN++/bgCi4hEuzdmPcK+ZMjpf4Pnx/KyBRHpSmjECx5mNpRggbggbPb5zrktZnYSsNDMPnHOLTlih87NINg1RWZmpncXVEREokBu4UucmGQMyZri+bG8bEEUAaeETXcEtlReyczSgT8D45xzFR8OcM5tCX3fBswi2GUlItJgHdy3m9cab2Z82RkkpXg/vpiXBSIf6GpmXcwsGZgIzA5fwcw6ATOBa51zn4bNb2pmzcpfAyOANR5mFRGJem++8jt2N4acPtfUy/E8KxDOuVJgCvAGsA74p3NurZndYma3hFa7H2gF/MHMVplZ+T2qJwPvmtlqYDkwxzk336usXvN6uO8LL7yQM888k4yMDPr163fYp5PHjBnDzp07j9jmgQceOOwT1bWxY8eOilxt27alQ4cOFdOHDh2q0T4WL17M0qVLq19RRMhb9QLNi+Hi8XfUy/E8/aCcc24uMLfSvOlhr28Cboqw3Sbg6J+YiiH1Mdz3888/T2ZmJs899xx33XUXCxcuBGDu3LnVbFl7rVq1qsj2wAMPkJaWxtSpU49pH4sXLyYtLY3zzjvPg4Qi8aOkeD+vNNrI2JLTSE5Nq5djaqiNeuLlcN/hBg4cWDEcNxz+4J5f/epXnHnmmQwbNoz169dXrJOfn096ejoDBw7krrvuokePHgCUlZVx1113VQyt/cc//rFGGVasWMGQIUPo27cvI0eOZOvWrQA88cQTdO/enfT0dCZOnMjmzZuZPn06jz76KL169ap2uHGRhmzR7Mf5PsWRnX5FvR2zQQ21cfv821n19ao63Wevtr14bNRj1a7n1XDflc2fP5/x48cfMX/FihW89NJLEYfdvvHGG5kxYwbnnXfeYU9re+aZZ2jevDn5+fkcPHiQ888/nxEjRtClS5cqj19SUsJPfvITXn31Vdq0acPLL7/Mfffdx7PPPsu0adP4/PPPady4MTt37qRFixbccssttWp5iDQ0eQV/Iy0JRoyvv38rDapA+Mmr4b7LXX311ezbt4+ysrKKgerCvfPOO1x22WUVT1YrH411586d7Nmzp6KL56qrruL1118HgkNrFxYWkpubCwSfsbBhw4ajFoj169ezZs0ahg8fDgRbIe3atav4GVx99dWMHz8+YhETkcjKSg4xy9aTVdyJ1BNOrLfjNqgCUZN3+l7yYrjvcs8//zwZGRncc8893HrrrRXPZgh3LMNxly978sknGTlyZI1zOOc455xzeP/9949YNmfOHJYsWcLs2bN56KGHWLt2bY33K9KQvTPnD2xv4sjumFOvx9U1iHrkxXDf4ZKSkvjlL3/JsmXLWLdu3WHLBg8ezKxZszhw4AB79uzhtddeA6Bly5Y0a9aMZcuWAfDSSy9VbDNy5EiefvppSkpKAPj000/Zt2/fUTOceeaZbN++vaJAlJSUsHbtWgKBAF9++SVDhw7lN7/5DTt37mTv3r0ajlukBnKXPUtqCYyecHf1K9chFYh6dLThvjMzMysuFJcP9w3/vQZR/vXEE08c9RipqanceeedR9zC2qdPH6644gp69epFdnb2YdcxnnnmGSZPnszAgQNxzlUMx33TTTfRvXt3+vTpQ48ePbj55purfKhQueTkZHJzc7n77rvJyMigV69eLF26lLKyMq655hp69uxJ7969+dnPfkaLFi249NJLmTVrli5Si1QhUFbKzMBaRu9vT9OWJ9XrsT0d7ru+abjv2ikfjhtg2rRpbN26NeJzp+ONfjckFrw352kuKPgxL7SdwpU3P1nn+/dluG+JHXPmzOHhhx+mtLSUU089lb/85S9+RxKRkNx3/0TjRnBJPXcvgQqEAFdccQVXXFF/91aLSM24QIC80kJGFJ/MCW3q/2kIDeIaRDx1o0nd0O+ExIL8N//Kl2ll5Jwx1pfjx32BSElJYceOHfqDIBWcc+zYsYOUlBS/o4gcVe6/n6ZRGVw64V5fjh/3XUwdO3akqKgIPUxIwqWkpBw2vIlItHGBAHkHVzKsrDUt21X94VQvxX2BSEpKOuonf0VEotGqJS+zqVkp/9PyEt8yxH2BEBGJRXlvP0WiwbjL/OleAhUIEZGo4wIB/rVvOReWtaT1KWf6liPuL1KLiMSate+/yqcnlJB9Ss3HQfOCCoSISJTJe/MJzMFlPnYvgbqYRESiTu6u9xkUaE7b02r/8LC6oBaEiEgUWZ8/jzXND5Ld/mK/o6hAiIhEk7z5jwIwYdw91azpPXUxiYhEkdzv3uVc0uh4Zj+/o6gFISISLTatXszKFgfIOelCv6MAKhAiIlEjb84jAGRfWv9De0eiLiYRkSiRt/3f9KUJnXtc4HcUQC0IEZGo8MXH7/NBi33ktBpU/cr1RAVCRCQKzHztNwBkj77T5yT/pS4mEZEokLf1bdItha59h/sdpYJaECIiPtv62Srea7GbnJbn+R3lMJ4WCDMbZWbrzWyjmR3xqQ8zu9rMCkNfS80so6bbiojEi1mvTMMZZI/4qd9RDuNZgTCzROApYDTQHbjSzLpXWu1zYIhzLh14CJhxDNuKiMSF3C/f4OxdyXQ/159nT1fFyxZEf2Cjc26Tc+4Q8BIwLnwF59xS59z3ocllQMeabisiEg+2f7GOfzffSXaz/n5HOYKXBaID8GXYdFFoXlV+CMw71m3NbLKZFZhZgZ47LSKx5pWZDxNIgJyLf+J3lCN4WSAswjwXcUWzoQQLRPnHB2u8rXNuhnMu0zmX2aZNm1oFFRHxS+7muZyxO4n0C3L8jnIELwtEEXBK2HRHYEvllcwsHfgzMM45t+NYthURiWXfbfmMt0/YQXZqHywh+m4q9TJRPtDVzLqYWTIwEZgdvoKZdQJmAtc65z49lm1FRGLd7JkPU5oIORf+2O8oEXn2QTnnXKmZTQHeABKBZ51za83sltDy6cD9QCvgD2YGUBrqLoq4rVdZRUT8kPvZa5yamEjfi67xO0pEnn6S2jk3F5hbad70sNc3ATfVdFsRkXixa9sXLEzbxpSyvlHZvQQaakNExBevz5zGoUaQc95kv6NUSQVCRMQHeetfoX1SAgNGTPI7SpWis10jIhLH9n73NfOabiU7oQcJidH7Pj16k4mIxKm5M39NcRLk9I94CTZqqECIiNSzvI/zOCnJOH/0zX5HOSp1MYmI1KMDu79jTuqXTOBsEpOS/Y5zVGpBiIjUozdmPcK+ZMjufb3fUaqlAiEiUo9yC1+iVZIxJGuK31GqpS4mEZF6cnDfbl5rvJnxZV1JSmnid5xqqQUhIlJP3nzld+xuDNk9r/Y7So2oQIiI1JPclc/TPAkuHn+H31FqRF1MIiL1oKR4P68mfcbYktNITk3zO06NqAUhIlIPFs1+nO9THDndJ/odpcZUIERE6kFuwd9IS4IRl93ld5QaUxeTiIjHSg8V84qtJ6u4EylpLfyOU2NqQYiIeOydOX9gexNHzik/8DvKMVELQkTEY7nLniO1BEbFUPcSqECIiHgqUFbKTPcxY/Z3oGnLk/yOc0zUxSQi4qGl82bwddMAOe0m+B3lmKkFISLiobz3/kzjUrgk+x6/oxwztSBERDwSKCslt7SQkcVtadaqvd9xjplaECIiHsl/868UpZWRfcalfkepFRUIERGP5C35I0llcOmEe/2OUivqYhIR8YALBMg9uJJhZW1o2a6L33FqRS0IEREPrFryMp83KyW7yxi/o9SaWhAiIh7Iffv3JBqMuyw2u5dABUJEpM65QIDcfflcWNaS1qec6XecWlMXk4hIHVv7/qt8ekIJOZ1G+R3luHhaIMxslJmtN7ONZnbEp0TM7Cwze9/MDprZ1ErLNpvZR2a2yswKvMwpIlKXchc+jjm47LL/8TvKcfGsi8nMEoGngOFAEZBvZrOdcx+HrfYdcBswvordDHXOfetVRhERL+TtXsagQHNO7tLD7yjHxcsWRH9go3Nuk3PuEPASMC58BefcNudcPlDiYQ4RkXrzyfK5rGl+kJwOw/yOcty8LBAdgC/DpotC82rKAQvMbIWZTa7TZCIiHsl741EAJoyLvbGXKvPyLiaLMM8dw/bnO+e2mNlJwEIz+8Q5t+SIgwSLx2SATp061S6piEgdyfvuPQa6NDp0y/Q7ynHzsgVRBJwSNt0R2FLTjZ1zW0LftwGzCHZZRVpvhnMu0zmX2aZNm+OIKyJyfDatXszKFgfIaXuR31HqhJcFIh/oamZdzCwZmAjMrsmGZtbUzJqVvwZGAGs8SyoiUgfy5jwCwISs2HpyXFU862JyzpWa2RTgDSAReNY5t9bMbgktn25mbYEC4AQgYGa3A92B1sAsMyvP+IJzbr5XWUVE6kLu9n+TSRM697jA7yh1wtNPUjvn5gJzK82bHvb6a4JdT5XtBjK8zCYiUpe++Ph9lrfYx8OJI/2OUmeO2sVkZteEvT6/0rIpXoUSEYk1M1/7DQDZo+/0OUndqe4axB1hr5+stGxSHWcREYlZuVvfImNnCl37Dvc7Sp2prkBYFa8jTYuINEhbNnzI0hZ7yG55nt9R6lR1BcJV8TrStIhIgzTr1Wk4g5yRP/M7Sp2q7iL1WWZWSLC1cHroNaHp0zxNJiISI3KLFtI9oTFnD8jyO0qdqq5AnF0vKUREYtS2zWtZ0nwn9zHI7yh17qgFwjn3n/BpM2sFDAa+cM6t8DKYiEgseOWVhwkkQM5Ft/kdpc5Vd5vr62bWI/S6HcFPM08C/h76UJuISIOWt3k+Z+xOouf5E/yOUuequ0jdxTlXPsTFjcBC59ylwAB0m6uINHA7ijbwVvMd5DTpiyXE3wM6qzuj8Oc0XEzoU9HOuT1AwKtQIiKxYPashylLgJwLb/U7iiequ0j9pZn9hODIrH2A+QBmlgokeZxNRCSq5W2aQ+fERvQZepXfUTxRXQvih8A5wA3AFc65naH55wLPeRdLRCS67dr2BQuabSO7cUZcdi9B9XcxbQNuiTB/EbDIq1AiItHu9ZnTKEmE7IHx+8DLoxYIMzvq8xucc2PrNo6ISGzIXf8KHRolMGBE/N6vU901iIEEnyv9IvABGn9JRIS9333N/KZbmVyWQUKip09N8FV1Z9YWGA5cCVwFzAFedM6t9TqYiEi0mjvz1xQnQXb/H/odxVNHvbLinCtzzs13zl1P8ML0RmBx6M4mEZEGKffjXE7en8D5o2/2O4qnqm0bmVlj4BKCrYjOwBPATG9jiYhEp/27vmVuahHXlnYnMSnZ7ziequ4i9V+BHsA84MGwT1WLiDRIb8x6hH3JkNPnBr+jeK66FsS1wD6gG3CbWcU1agOcc+4ED7OJiESd3I9eplWSMeTS+O9pr+5zEPH56Q8RkVo4uG83rzX+Dz8o6Uaj5BS/43gufu/PEhGpYwtn/ZY9jSEn/Vq/o9QLFQgRkRrKW/UizZPgonG3+x2lXqgLSUSkBg4d2MsrSZ8xruQ0klPT/I5TL9SCEBGpgUWzH2dniiOne3yO3BqJCoSISA3krfgHaUkw/LI7/Y5Sb9TFJCJSjdJDxcxKWM+lxaeSktbC7zj1Ri0IEZFqvDPnD3yb6sjudrnfUeqVCoSISDVylz1Hk0YwOvtuv6PUK0+7mMxslJmtN7ONZnZPhOVnmdn7ZnbQzKYey7YiIvUhUFbKTPcxYw50oEnz1n7HqVeeFQgzSwSeAkYD3YErzax7pdW+A24DfluLbUVEPLd03gy+bhog+6wJfkepd162IPoDG51zm5xzh4CXgHHhKzjntjnn8oGSY91WRKQ+5L73JxqXwiXZDa8jw8sC0YHg0+jKFYXm1em2ZjbZzArMrGD79u21CioiEkmgrJS80o8YubctzVq19ztOvfOyQER6PKmr622dczOcc5nOucw2bdrUOJyISHXy3/wrRWll5HRtmB0YXhaIIuCUsOmOwJZ62FZEpE7kLplOUhlcmn2v31F84WWByAe6mlkXM0sGJgKz62FbEZHj5gIB8g6uYtieNrQ4+VS/4/jCs89BOOdKzWwK8AaQCDzrnFtrZreElk83s7ZAAXACEDCz24Huzrndkbb1KquISGUrF7/I581K+T8tL/E7im88/aCcc24uMLfSvOlhr78m2H1Uo21FROpL3uI/kGgwbsL/+B3FN/oktYhIJS4Q4F/78hladiKtOnb1O45vNFifiEgla5bOYsMJJeScOsrvKL5SC0JEpJK8N5/EgPHjG+bdS+VUIEREKsndvYzBgeac3KWH31F8pS4mEZEwnyyfy9rmB8npMNzvKL5TgRARCZP3xqMAXDauYQ3tHYm6mEREwuR+9x7nuWZ06JbpdxTfqQUhIhLy2aq3WdXiANlth/odJSqoQIiIhOTNCT6aJvvSn/ucJDqoi0lEJCT32yX0oymnnnO+31GigloQIiLAf9a+R36LfWS3HuR3lKihAiEiAsx87REAssdM9TlJ9FAXk4gIkPf122RYCmf0vtjvKFFDLQgRafC++rSA91ruIaelrj2EU4EQkQZv1qu/BiBn1B0+J4ku6mISkQYv76s36Z7QmLP6j/E7SlRRC0JEGrRtm9eypPlOck4Y4HeUqKMWhIg0aK+88jCBBMi+6Cd+R4k6KhAi0qDlbp5H18Qkep4/we8oUUddTCLSYO0o2sDbzb8jp0kmlqA/h5WpBSEiDdbsWQ9TlgDZg3/sd5SopAIhIg1W7qbX6ZzYiD5Dr/I7SlRSm0pEGqRd275gYbPtZDfOUPdSFdSCEJEG6bW8hylJhJzzbvE7StRSgRCRBin301fo2CiR/sNv8DtK1FK7SkQanD07tjA/7WsmNOpBQqLeJ1dFPxkRaXDmzvw1BxtBTv+b/I4S1VQgRKTByVs3k5OTEjhv1GS/o0Q1T7uYzGyUma03s41mdk+E5WZmT4SWF5pZn7Blm83sIzNbZWYFXuYUkYZj/65vmZNaxATOJjEp2e84Uc2zFoSZJQJPAcOBIiDfzGY75z4OW2000DX0NQB4OvS93FDn3LdeZRSRhmf+zN+wPxly+t7od5So52ULoj+w0Tm3yTl3CHgJGFdpnXHA31zQMqCFmbXzMJOINHB5a/5JqwPG4Kxb/Y4S9bwsEB2AL8Omi0LzarqOAxaY2Qozq7Kj0Mwmm1mBmRVs3769DmKLSLw6uG83rzX+D5eVdaNRcorfcaKelwXCIsxzx7DO+c65PgS7oW41s8GRDuKcm+Gcy3TOZbZp06b2aUUk7i2c9Vv2NIbsvtf4HSUmeFkgioBTwqY7Altquo5zrvz7NmAWwS4rEZFay131Ai2KjYvG3e53lJjgZYHIB7qaWRczSwYmArMrrTMbuC50N9O5wC7n3FYza2pmzQDMrCkwAljjYVYRiXOHDuzl1aRNjCs5jeTUNL/jxATP7mJyzpWa2RTgDSAReNY5t9bMbgktnw7MBcYAG4H9QPltBScDs8ysPOMLzrn5XmUVkfi3aPbj7ExxZHe/0u8oMcPTD8o55+YSLALh86aHvXbAEbcSOOc2ARleZhORhiW34O80S4Lhl93pd5SYobGYRCTulR4q5pXET8k6eCopaS38jhMzNNSGiMS9Ja8/xbepjpwzf+B3lJiiAiEicS/3g+do0ghGTfi531FiirqYRCSulZUcYpZbx5gDHWjSvLXfcWKKWhAiEteWzp/B100D5LTP9jtKzFGBEJG4lrf0GVISYcyEu/2OEnNUIEQkbgXKSskr/YiRxW1p1qq933Fijq5BiEjcWr7wLxSllZHTbbzfUWKSCoSIxK28d/5IUhlkTTjieWVSA+piEpG45AIBcg+uYnhZG1qcfKrfcWKSWhAiEpdWLn6Rzc1KyTkty+8oMUsFQkTiUu6ip0gMwNjL7vU7SsxSF5OIxB0XCJC7v4CLyk6kVceufseJWWpBiEjcWbN0FhtOKCH71FF+R4lpakGISNzJffMJEhyMH6/upeOhAiEicSdv9wcMDrTg5C49/I4S09TFJCJxZd0Hr7O2+UGyOwzzO0rMU4EQkbiS98ZjAExQ99JxUxeTiMSVvO+Xcp5rRvuuffyOEvPUghCRuLFx5VusanGAnLYX+R0lLqgFISIxafsX6yjMf53VG96l8Nu1rC7bwsdpB7BEmHDpXX7HiwsqECIS1UqK97N+xRus/mghhUUrWL3vMwqTvmdr00DFOu0SEkgva8kI15PRva7k1HPO9zFx/FCBEJGosW3zWgoL5rB6wzsU7vg42CpoVkxJYnB5skF3UhkR6EJ643PI6HoB6f2yaNPpbH+DxykVCBGpd4cO7OWTgvkUrnkr1CrYRGHy93wd1ipon5BAetmJjApkkN6+LxnpI+jWZzhJKU18TN6wqECIiKe++XwNhQWvs3rjuxTuWMfqsi2sC2sVNE6AcyyVUYHTgq2CboNI75dF61PO9De4qECISN04dGAv6/LnUrjmbVZ/VUDhvs8pbLyTb5r8t1XQwRJIpxVjXC/S22aSkTGcbn1G0Cg5xcfkUhUVCBE5Zl9vKgxeKyhvFbitrEsrprRSq2BM4HTSU4Ktgp6Zl6hVEGNUIESkSgf37WZdwTwK17zF6q8+pHD/5xQ2/p5tTVzFOh0TEkkvPZEs15v0dplkZIyga+9hahXEARUIEcEFAnz9eXmr4D0Kvwu2Cj5JO1jRKkhJgHOsCVmBM0hP7UFGt8H0zLxEz1uIY54WCDMbBTwOJAJ/ds5Nq7TcQsvHAPuBG5xzH9ZkWxGpnYP7dvNx/hwK175d0SpYnbKTb1P/2yo4JSGR9NJWjOUM0ttnkp4+XK2CBsizAmFmicBTwHCgCMg3s9nOuY/DVhsNdA19DQCeBgbUcFuRuOQCAZwL1Py7Cxy+nXMVr/ft+pa1qxey+rPyVsHXfNLsIGWhQXZSEqCHNWFcWddgq+DMwfTsl8WJ7U/38Scg0cLLFkR/YKNzbhOAmb0EjAPC/8iPA/7mnHPAMjNrYWbtgM412LbO9P1ZEw5YmRe7jlqu+lWObX9WxzsEXB2ndBVfDmeHTwNHzDts2kLzKq9n//1ZHrZNhHWO9p2w717pFGoVjHNnkNGhf0WrIDEp2dsDS8zyskB0AL4Mmy4i2Eqobp0ONdwWADObDEwG6NSpU62Cnp1wModcaa22jWVGHf5FctTl3ioEeyHrcH+hszYXfGUWmsZC5xA2L3yd8GVVzCd8m/L/LMK8Kv7DETq2Hdv38mNjmCUcNr9xoxS6nzGQnv2yaNmuS53+LCX+eVkgIv3LrvyWsKp1arJtcKZzM4AZAJmZmbV6y/mP//d5bTYTEYlrXhaIIuCUsOmOwJYarpNcg21FRMRDXj4PIh/oamZdzCwZmAjMrrTObOA6CzoX2OWc21rDbUVExEOetSCcc6VmNgV4g+Ctqs8659aa2S2h5dOBuQRvcd1I8DbXG4+2rVdZRUTkSBa8gSg+ZGZmuoKCAr9jiIjEDDNb4ZzLjLRMjxwVEZGIVCBERCQiFQgREYlIBUJERCKKq4vUZrYd+E8tN28NfFuHcfwUL+cSL+cBOpdoFC/nAcd3Lqc659pEWhBXBeJ4mFlBVVfyY028nEu8nAfoXKJRvJwHeHcu6mISEZGIVCBERCQiFYj/muF3gDoUL+cSL+cBOpdoFC/nAR6di65BiIhIRGpBiIhIRCoQIiISkQpEBGY21cycmbX2O0ttmNlDZlZoZqvMbIGZtfc7U22Z2SNm9knofGaZWQu/M9WWmV1uZmvNLGBmMXd7pZmNMrP1ZrbRzO7xO09tmdmzZrbNzNb4neV4mdkpZrbIzNaFfrd+Wpf7V4GoxMxOAYYDX/id5Tg84pxLd871Al4H7vc5z/FYCPRwzqUDnwL3+pzneKwBJgBL/A5yrMwsEXgKGA10B640s+7+pqq1vwCj/A5RR0qBO51zZwPnArfW5f8XFYgjPQr8nCoecRoLnHO7wyabEtvnssC5igeGLyP4dMGY5Jxb55xb73eOWuoPbHTObXLOHQJeAsb5nKlWnHNLgO/8zlEXnHNbnXMfhl7vAdYBHepq/14+cjTmmNlY4Cvn3OryB8HHKjP7FXAdsAsY6nOcujIJeNnvEA1UB+DLsOkiYIBPWSQCM+sM9AY+qKt9NrgCYWZvAm0jLLoP+B9gRP0mqp2jnYdz7lXn3H3AfWZ2LzAF+EW9BjwG1Z1LaJ37CDann6/PbMeqJucSoyK9Y4rZlmm8MbM0IA+4vVIPwnFpcAXCOTcs0nwz6wl0AcpbDx2BD82sv3Pu63qMWCNVnUcELwBziOICUd25mNn1QBZwsYvyD+4cw/+XWFMEnBI23RHY4lMWCWNmSQSLw/POuZl1ue8GVyCq4pz7CDipfNrMNgOZzrmYG+3RzLo65zaEJscCn/iZ53iY2SjgbmCIc26/33kasHygq5l1Ab4CJgJX+RtJLPhu9hlgnXPud3W9f12kjk/TzGyNmRUS7DKr01vf6tnvgWbAwtBtu9P9DlRbZnaZmRUBA4E5ZvaG35lqKnSjwBTgDYIXQv/pnFvrb6raMbMXgfeBM82syMx+6Hem43A+cC1wUejfxyozG1NXO9dQGyIiEpFaECIiEpEKhIiIRKQCISIiEalAiIhIRCoQIiISkQqExCUz23sc204JjVh62Ii+FvREaFmhmfUJW5ZqZv82s0Qz61xXI4Wa2V/MLKcG6y2uboRYM3vJzLrWRS5pGFQgRI70HjAM+E+l+aOBrqGvycDTYcsmATOdc2X1krB2niY4EKVIjahASFwLvet/JPTBwY/M7IrQ/AQz+0NoDP3XzWxu+Tt159xK59zmCLsbB/zNBS0DWphZu9Cyq4EjxloKtSbeMbMPQ1/nheZfGGpx/NPMPjWzaWZ2tZktD+U8PWw3w0L7+NTMskLbp4ZaBIVm9jKQGnbMp82sIHRuD4bt553QvjSCgtSIflEk3k0AegEZQGsg38yWEPwEamegJ8EhVtYBz1azr0gjmnYwsx3AaVUUlW3AcOdccah750WgvCsoAzib4NDTm4A/O+f6hx768hPg9tB6nYEhwOnAIjM7A/gRsN85l25m6cCHYce8zzn3XegZDm+ZWbpzrtA5FzCzjaHjrqjmXEXUgpC4dwHwonOuzDn3DfBvoF9o/r+cc4HQYIyLarCvqkY0bQ3srGKbJOBPZvYR8C+CD9splx8az/8g8BmwIDT/I4JFodw/Qzk3ECwkZwGDgX8AOOcKgcKw9X9gZh8CK4FzKh1zGxCzTxiU+qUWhMS7qh7sUZsHflQ1omkxkFLFNj8DviH4rj0htG65g2GvA2HTAQ7/t1l5PBxXxXxCg+lNBfo55743s79UypYCHKgiq8hh1IKQeLcEuCJ0d1Ebgu+8lwPvAtmhaxEnAxfWYF+zgetC1zXOBXaFWgDfA4lmFqlINAe2OucCBAdVS6zFOVweynk6cBqwPnReVwOYWQ8gPbTuCcA+YFfovEZX2lc3ICYH2ZP6pwIh8W4Wwe6X1cDbwM9DXUp5BFsEa4A/EnwK1y4AM7stNOpqR6DQzP4c2tdcgl08G4E/AT8OO84Cgt1Wlf0BuN7MlhH847yvFuewnmDX2DzgFudcMcE7ktJCI/b+nGDRwzm3mmDX0lqC11TeK99JqGAccM5trUUGaYA0mqs0WGaW5pzba2atCP6BPb+2D4cys97AHc65a+s0ZB0ys58Bu51zz/idRWKDrkFIQ/a6mbUAkoGHjufJgc65lWa2yMwSo/izEDuBv/sdQmKHWhAiIhKRrkGIiEhEKhAiIhKRCoSIiESkAiEiIhGpQIiISET/H1GSr6ffdQ3uAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_115_1.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from sklearn.model_selection import train_test_split\n", "from sklearn import linear_model\n", "\n", "def MSE(y_data,y_model):\n", " n = np.size(y_model)\n", " return np.sum((y_data-y_model)**2)/n\n", "\n", "\n", "# A seed just to ensure that the random numbers are the same for every run.\n", "# Useful for eventual debugging.\n", "np.random.seed(3155)\n", "\n", "n = 100\n", "x = np.random.rand(n)\n", "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)\n", "\n", "Maxpolydegree = 20\n", "X = np.zeros((n,Maxpolydegree))\n", "#We include explicitely the intercept column\n", "for degree in range(Maxpolydegree):\n", " X[:,degree] = x**degree\n", "# We split the data in test and training data\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n", "\n", "p = Maxpolydegree\n", "I = np.eye(p,p)\n", "# Decide which values of lambda to use\n", "nlambdas = 6\n", "MSEOwnRidgePredict = np.zeros(nlambdas)\n", "MSERidgePredict = np.zeros(nlambdas)\n", "lambdas = np.logspace(-4, 2, nlambdas)\n", "for i in range(nlambdas):\n", " lmb = lambdas[i]\n", " OwnRidgeBeta = np.linalg.pinv(X_train.T @ X_train+lmb*I) @ X_train.T @ y_train\n", " # Note: we include the intercept column and no scaling\n", " RegRidge = linear_model.Ridge(lmb,fit_intercept=False)\n", " RegRidge.fit(X_train,y_train)\n", " # and then make the prediction\n", " ytildeOwnRidge = X_train @ OwnRidgeBeta\n", " ypredictOwnRidge = X_test @ OwnRidgeBeta\n", " ytildeRidge = RegRidge.predict(X_train)\n", " ypredictRidge = RegRidge.predict(X_test)\n", " MSEOwnRidgePredict[i] = MSE(y_test,ypredictOwnRidge)\n", " MSERidgePredict[i] = MSE(y_test,ypredictRidge)\n", " print(\"Beta values for own Ridge implementation\")\n", " print(OwnRidgeBeta)\n", " print(\"Beta values for Scikit-Learn Ridge implementation\")\n", " print(RegRidge.coef_)\n", " print(\"MSE values for own Ridge implementation\")\n", " print(MSEOwnRidgePredict[i])\n", " print(\"MSE values for Scikit-Learn Ridge implementation\")\n", " print(MSERidgePredict[i])\n", "\n", "# Now plot the results\n", "plt.figure()\n", "plt.plot(np.log10(lambdas), MSEOwnRidgePredict, 'r', label = 'MSE own Ridge Test')\n", "plt.plot(np.log10(lambdas), MSERidgePredict, 'g', label = 'MSE Ridge Test')\n", "\n", "plt.xlabel('log10(lambda)')\n", "plt.ylabel('MSE')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The results here agree when we force **Scikit-Learn**'s Ridge function to include the first column in our design matrix.\n", "We see that the results agree very well. Here we have thus explicitely included the intercept column in the design matrix.\n", "What happens if we do not include the intercept in our fit?\n", "Let us see how we can change this code by zero centering." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Beta values for own Ridge implementation\n", "[ 3.43579948e-02 -5.43330971e-01 -3.10141413e-03 2.47116868e-01\n", " 2.18613217e-01 1.02054837e-01 -4.25617657e-04 -5.90475506e-02\n", " -7.68534263e-02 -6.68929213e-02 -4.24906604e-02 -1.40927184e-02\n", " 1.11482289e-02 2.88529063e-02 3.67047975e-02 3.38135733e-02\n", " 2.02198702e-02 -3.46383925e-03 -3.63025821e-02]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[ 3.43579948e-02 -5.43330971e-01 -3.10141413e-03 2.47116868e-01\n", " 2.18613217e-01 1.02054837e-01 -4.25617654e-04 -5.90475506e-02\n", " -7.68534263e-02 -6.68929213e-02 -4.24906604e-02 -1.40927184e-02\n", " 1.11482289e-02 2.88529063e-02 3.67047975e-02 3.38135733e-02\n", " 2.02198702e-02 -3.46383925e-03 -3.63025821e-02]\n", "Intercept from own implementation:\n", "1.0330308045180234\n", "Intercept from Scikit-Learn Ridge implementation\n", "1.0330308045183163\n", "MSE values for own Ridge implementation\n", "3.1392559581788775e-06\n", "MSE values for Scikit-Learn Ridge implementation\n", "3.1392559584983597e-06\n", "Beta values for own Ridge implementation\n", "[-0.05807125 -0.29822833 -0.08551306 0.08156108 0.13679863 0.12333649\n", " 0.08251519 0.03815288 0.00111756 -0.02498832 -0.04010697 -0.04566964\n", " -0.04355837 -0.03562355 -0.02348765 -0.00848904 0.00831018 0.0260906\n", " 0.04423486]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[-0.05807125 -0.29822833 -0.08551306 0.08156108 0.13679863 0.12333649\n", " 0.08251519 0.03815288 0.00111756 -0.02498832 -0.04010697 -0.04566964\n", " -0.04355837 -0.03562355 -0.02348765 -0.00848904 0.00831018 0.0260906\n", " 0.04423486]\n", "Intercept from own implementation:\n", "1.0411487294305746\n", "Intercept from Scikit-Learn Ridge implementation\n", "1.0411487294305246\n", "MSE values for own Ridge implementation\n", "1.9601304850213484e-05\n", "MSE values for Scikit-Learn Ridge implementation\n", "1.960130485007934e-05\n", "Beta values for own Ridge implementation\n", "[-0.1416398 -0.14021063 -0.05383795 0.01367553 0.04784395 0.05796251\n", " 0.05447415 0.044613 0.03267527 0.02098261 0.01066519 0.00217499\n", " -0.00440346 -0.00917248 -0.01231917 -0.01405935 -0.0146081 -0.01416528\n", " -0.01290947]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[-0.1416398 -0.14021063 -0.05383795 0.01367553 0.04784395 0.05796251\n", " 0.05447415 0.044613 0.03267527 0.02098261 0.01066519 0.00217499\n", " -0.00440346 -0.00917248 -0.01231917 -0.01405935 -0.0146081 -0.01416528\n", " -0.01290947]\n", "Intercept from own implementation:\n", "1.0495569966278315\n", "Intercept from Scikit-Learn Ridge implementation\n", "1.0495569966278269\n", "MSE values for own Ridge implementation\n", "5.495916150938325e-05\n", "MSE values for Scikit-Learn Ridge implementation\n", "5.495916150936654e-05\n", "Beta values for own Ridge implementation\n", "[-0.13535942 -0.08593216 -0.03568439 -0.0036367 0.01397146 0.02229529\n", " 0.02503753 0.0245528 0.02228115 0.01908936 0.01549377 0.01179792\n", " 0.00817631 0.00472512 0.00149311 -0.00149956 -0.00424967 -0.00676387\n", " -0.00905423]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[-0.13535942 -0.08593216 -0.03568439 -0.0036367 0.01397146 0.02229529\n", " 0.02503753 0.0245528 0.02228115 0.01908936 0.01549377 0.01179792\n", " 0.00817631 0.00472512 0.00149311 -0.00149956 -0.00424967 -0.00676387\n", " -0.00905423]\n", "Intercept from own implementation:\n", "1.0399676689527968\n", "Intercept from Scikit-Learn Ridge implementation\n", "1.0399676689527975\n", "MSE values for own Ridge implementation\n", "7.571105947979439e-05\n", "MSE values for Scikit-Learn Ridge implementation\n", "7.571105947979395e-05\n", "Beta values for own Ridge implementation\n", "[-0.05100875 -0.04063602 -0.02723445 -0.01713366 -0.0100706 -0.00517114\n", " -0.00174276 0.00068734 0.00243186 0.00369758 0.00462287 0.0053018\n", " 0.00579953 0.006162 0.00642221 0.00660427 0.00672607 0.0068011\n", " 0.00683964]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[-0.05100875 -0.04063602 -0.02723445 -0.01713366 -0.0100706 -0.00517114\n", " -0.00174276 0.00068734 0.00243186 0.00369758 0.00462287 0.0053018\n", " 0.00579953 0.006162 0.00642221 0.00660427 0.00672607 0.0068011\n", " 0.00683964]\n", "Intercept from own implementation:\n", "0.9999555851685968\n", "Intercept from Scikit-Learn Ridge implementation\n", "0.999955585168597\n", "MSE values for own Ridge implementation\n", "0.0007698473260556339\n", "MSE values for Scikit-Learn Ridge implementation\n", "0.0007698473260556334\n", "Beta values for own Ridge implementation\n", "[-0.00834567 -0.00803064 -0.00673407 -0.00554552 -0.00458878 -0.0038335\n", " -0.00323332 -0.00274989 -0.0023548 -0.00202756 -0.00175331 -0.00152117\n", " -0.001323 -0.0011526 -0.00100519 -0.00087697 -0.00076495 -0.00066668\n", " -0.00058016]\n", "Beta values for Scikit-Learn Ridge implementation\n", "[-0.00834567 -0.00803064 -0.00673407 -0.00554552 -0.00458878 -0.0038335\n", " -0.00323332 -0.00274989 -0.0023548 -0.00202756 -0.00175331 -0.00152117\n", " -0.001323 -0.0011526 -0.00100519 -0.00087697 -0.00076495 -0.00066668\n", " -0.00058016]\n", "Intercept from own implementation:\n", "0.9637117593816477\n", "Intercept from Scikit-Learn Ridge implementation\n", "0.9637117593816477\n", "MSE values for own Ridge implementation\n", "0.0023813163025848865\n", "MSE values for Scikit-Learn Ridge implementation\n", "0.002381316302584885\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEKCAYAAADaa8itAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA0KUlEQVR4nO3dd3hUVfrA8e+bkEZCJygYheBGkRJaaCogClIF9Scr2EBwERVZUVxxXUEU1wIqioiLCi4WUNeGiIAKCBakCNK7CBGWElpCeub9/TFDNqQTZnInyft5nnmcO+ece98T4rw599x7rqgqxhhjjDcEOB2AMcaY8sOSijHGGK+xpGKMMcZrLKkYY4zxGksqxhhjvMaSijHGGK/xaVIRkR4isk1EdorImHzKRURe8ZSvF5FWRbUVkYkistVT/1MRqe75vIGIpIjIOs/rdV/2zRhjTF4+SyoiEghMBXoCjYGBItI4V7WeQIznNQyYVoy2XwNNVTUW2A48mmN/u1S1hec13Dc9M8YYUxBfjlTaAjtVdbeqpgNzgH656vQDZqnbCqC6iNQtrK2qLlLVTE/7FUCUD/tgjDHmLFTy4b4vAPbl2I4H2hWjzgXFbAswBPggx3a0iKwFTgL/UNXluRuIyDDcoyLCw8NbN2rUqFidMcYY47ZmzZojqhqZX5kvk4rk81nuNWEKqlNkWxF5DMgE3vN8dAC4SFUTRKQ18JmINFHVk2fsRHU6MB0gLi5OV69eXWRHjDHG/I+I/F5QmS+TSjxwYY7tKGB/MesEF9ZWRAYBfYBr1LN4maqmAWme92tEZBdwCWBZwxhjSokv51RWATEiEi0iwcAAYG6uOnOBOzxXgbUHTqjqgcLaikgP4BGgr6omn96RiER6JvgRkYa4J/93+7B/xhhjcvHZSEVVM0VkBLAQCARmqOomERnuKX8dmA/0AnYCycCdhbX17PpVIAT4WkQAVniu9OoEPCkimUAWMFxVj/qqf8YYY/KSirz0fX5zKhkZGcTHx5OamupQVMYfhYaGEhUVRVBQkNOhGOM4EVmjqnH5lflyTqVMio+Pp0qVKjRo0ADPSMhUcKpKQkIC8fHxREdHOx2OMX7NlmnJJTU1lVq1allCMdlEhFq1atno1ZhisKSSD0soJjf7nTCmeCypGGNMBZOZ5fLZvi2p+CER4fbbb8/ezszMJDIykj59+gBw8OBB+vTpQ/PmzWncuDG9evUCYM+ePYSFhdGiRYvs16xZs0o19pwxNG7cmDvuuIOMjAwAVq9ezciRI/Nt16BBA44cOXJOx545c2Z2v4ODg2nWrBktWrRgzJg8a5kWaPLkySQnJxdd0Zgy7MnZ8/ngu3W+2bmqVthX69atNbfNmzfn+ay0hYeHa4sWLTQ5OVlVVefPn6/NmzfX3r17q6rqsGHDdPLkydn1f/31V1VV/e2337RJkyalH3AOOWPIzMzULl266Lvvvltku/r16+vhw4e9FkdJ91dYO3/43TDGHwCrtYDvVRup+KmePXvy5ZdfAjB79mwGDhyYXXbgwAGiov63jmZsbOxZ7fvbb7+lZcuWNGvWjCFDhpCWlsbKlSu58cYbAfj8888JCwsjPT2d1NRUGjZsCMBVV13FI488Qtu2bbnkkktYvjzP0mpnCAwMpG3btvzxxx8ALF26NHu0lZCQwLXXXkvLli25++670RyXtj/11FM0atSIbt26MXDgQCZNmgTArl276NGjB61bt6Zjx45s3bq1WP2dOHEibdq0ITY2lnHjxgFw6tQpevfuTfPmzWnatCkffPABr7zyCvv376dLly506dLlLH6ixpQNR0+mMPiVtzh6MsVnx7CkUoSrrsr7eu01d1lycv7lb7/tLj9yJG9ZcQ0YMIA5c+aQmprK+vXradfuf+tp3nfffQwdOpQuXbrw9NNPs3///1a/2bVr1xmnv3J/8aempjJ48GA++OADNmzYQGZmJtOmTaNVq1asXbsWgOXLl9O0aVNWrVrFzz//fMaxMzMzWblyJZMnT2b8+PGF9iE1NZWff/6ZHj165CkbP348V155JWvXrqVv377s3bsXcJ8i+/jjj1m7di2ffPIJOe8jGjZsGFOmTGHNmjVMmjSJe++9t8if46JFi9ixYwcrV65k3bp1rFmzhmXLlrFgwQLq1avHr7/+ysaNG+nRowcjR46kXr16LFmyhCVLlhS5b2PKmpEz3uHfx+5i1uKffXYMu0/FT8XGxrJnzx5mz56dPWdyWvfu3dm9ezcLFizgq6++omXLlmzcuBGAiy++mHXr1hW4323bthEdHc0ll1wCwKBBg5g6dSoPPPAAf/rTn9iyZQsrV67kwQcfZNmyZWRlZdGxY8fs9qdHM61bt2bPnj35HuN0YtuxYwc33XRTviOpZcuW8cknnwDQu3dvatSoAcD3339Pv379CAsLA+C6664DICkpiR9//JH+/ftn7yMtLa3Afp62aNEiFi1aRMuWLbP3s2PHDjp27Mjo0aN55JFH6NOnzxl9NKY8ysxy8eG+F6jsas3Ivp19dhxLKkVYurTgssqVCy+vXbvw8qL07duX0aNHs3TpUhISEs4oq1mzJrfccgu33HILffr0YdmyZbRu3brIfeY8zZRbx44d+eqrrwgKCqJr164MHjyYrKys7NNPACEhIYD71FZmZma++zmd2A4cOMBVV13F3Llz6du3b556+V2mW1B8LpeL6tWrF5ow86OqPProo9x99915ytasWcP8+fN59NFHufbaaxk7duxZ7duYsmTse1+QUXU799SbQ0CA7y6Rt9NffmzIkCGMHTuWZs2anfH54sWLs69QSkxMZNeuXVx00UXF2mejRo3Ys2cPO3fuBOCdd96hc2f3Xy2dOnVi8uTJdOjQgcjISBISEti6dStNmjQpUfx169bl2Wef5ZlnnslT1qlTJ957z/3Ugq+++opjx44BcOWVV/LFF1+QmppKUlJS9rxS1apViY6O5qOPPgLcyeLXX38tMobu3bszY8YMkpKSAPjjjz84dOgQ+/fvp3Llytx2222MHj2aX375BYAqVaqQmJhYov4a48+m/jKJSokNeG7Q//n0ODZS8WNRUVH89a9/zfP5mjVrGDFiBJUqVcLlcnHXXXfRpk0b9uzZk33q6bQhQ4accRlvaGgoM2fOpH///mRmZtKmTRuGD3c/ebldu3YcPHiQTp06Ae5TcHXq1DmnG/+uv/56nnjiiTxzO+PGjWPgwIG0atWKzp07ZyfFNm3a0LdvX5o3b079+vWJi4ujWrVqALz33nvcc889TJgwgYyMDAYMGEDz5s0LPf61117Lli1b6NChAwARERG8++677Ny5k4cffpiAgACCgoKYNm0a4J636dmzJ3Xr1rV5FVNuxB8+SZak0bfOKEKDffu1bwtK5lpQcsuWLVx22WUORWTAPe8RERFBcnIynTp1Yvr06bRq1crpsOx3w5RpLpeSmeUiOCjwnPdlC0qaMmXYsGFs3ryZ1NRUBg0a5BcJxZiyauNvBwkJqkRMVC2CA849oRTFkorxO++//77TIRhTbgx8/Uk2Bczm0Jh4aler7PPj2US9McaUU9v2HWFj0Ez+lHlDqSQUsJGKMcaUW/fOnAZBKUzq81CpHdOSijHGlEPHk1JZcmoKddJ707d941I7riUVY4wph16euxitfJhHO4wu1ePanIof8vXS9/PmzaNly5bZ7f/1r38B8MQTT5xx93x+GjRoQLNmzYiNjaVz5878/vvv2WWXX355vm0GDx7Mf/7zn7P7IeSyYcOG7D7VrFmT6OhoWrRoQdeuXYu9j88++4zNmzefUxzGlBXjbunFoj7bfbokS35spOKHwsPD2bhxIykpKYSFhfH1119zwQUXZJePHTuWbt26Zd8YuX79+uyyotb+ysjIYNiwYaxcuZKoqCjS0tIKXMOrIEuWLKF27dqMGzeOCRMm8MYbbwDw448/ntV+zkazZs2y+zV48GD69OnDTTfddFb7+Oyzz+jTpw+NG5feqQBjnJCcmkHl0CC6tY4p9WPbSMVP+Wrp+8TERDIzM6lVqxbgXsvr0ksvLVGMHTp0yF7WHtx3q4N7CZURI0bQuHFjevfuzaFDh7LrzJ8/n0aNGnHllVcycuTI7NHXqVOnGDJkCG3atKFly5Z8/vnnxYph0aJFdOjQgVatWtG/f//s5VjGjBlD48aNiY2NZfTo0fz444/MnTuXhx9+mBYtWrBr164S9dmYsqDemG60/vuDjhzbRipFuOrtq/J89ucmf+beNveSnJFMr/d65Skf3GIwg1sM5kjyEW768My/ppcOXlqs4w4YMIAnn3ySPn36sH79eoYMGZK91Ml9993HzTffzKuvvkrXrl258847qVevHkCeZVqmTJlyxgq8NWvWpG/fvtSvX59rrrmGPn36MHDgQAICzv7viwULFnD99dfn+fzTTz9l27ZtbNiwgYMHD9K4cWOGDBlCamoqd999N8uWLSM6OvqMRPn0009z9dVXM2PGDI4fP07btm3p2rUr4eHhBR7/yJEjTJgwgW+++Ybw8HCee+45XnzxRUaMGMGnn37K1q1bERGOHz9O9erV6du3b4lGOMaUJW8uWMGJGt9RP+wGR45vScVP+Wrpe4A333yTDRs28M033zBp0iS+/vpr3j79EJhi6NKlCwcPHqROnTpMmDAhT/myZcsYOHAggYGB1KtXj6uvvhqArVu30rBhQ6KjowEYOHAg06dPB9wjjrlz52bP6aSmprJ3795Cl0VZsWIFmzdv5oorrgAgPT2dDh06ULVqVUJDQ7nrrrvo3bt39mjImIrgiUWTkJDqvHb/UEeOb0mlCIWNLCoHVS60vHbl2sUemeTHF0vfn9asWTOaNWvG7bffTnR09FkllSVLlhAeHs7gwYMZO3YsL774Yp46Z7Os/emyjz/++KxOxakq3bp1Y/bs2XnKVq5cybfffsucOXN49dVXWbx4cbH3a0xZtXjdLv6o+gkdssZwfs0IR2KwORU/5oul75OSklia4yEv69ato379+mcdW1hYGJMnT2bWrFkcPXr0jLJOnToxZ84csrKyOHDgQPZqv40aNWL37t3ZFwZ88MEH2W26d+/OlClTshPP6adQFqZ9+/b88MMP2cv4Jycns337dpKSkjhx4gS9evVi8uTJ2SM3W9belHcPzJkMWUG8Nvh+x2KwpOLHClv6Pi4ujtjYWDp06JC99D3kfZzwK6+8ckZbVeX555/n0ksvpUWLFowbN+6MUcqECROIiorKfhWmbt26DBw4kKlTp57x+Q033EBMTAzNmjXjnnvuyX5eS1hYGK+99ho9evTgyiuv5Lzzzste1v7xxx8nIyOD2NhYmjZtyuOPP17kzycyMpK3336bgQMHEhsbS/v27dm6dSuJiYn06dMn+7Lnl156CXDPU02cOJGWLVvaRL0pl94e+igPNniPFhfXdSwGW/relr4vVaeXtVdV7rvvPmJiYhg1apTTYRWL/W4Y41bY0vc2UjGl6o033qBFixY0adKEEydO5PuYX2PM2Tl6MoV6o27gzQUrnA7FJupN6Ro1alSZGZkYU1aMnPEOB6p/RlJq3tPlpc2SSj5U9ZweoWvKn4p8mtj4t8wsFx/ue4HKrtalviRLfuz0Vy6hoaEkJCTYl4jJpqokJCQQGhrqdCjG5DH2vS/IqLqdoZeNJiDA+T+GbaSSS1RUFPHx8Rw+fNjpUIwfCQ0NLfJqOGOcMPWXSQRWqs/zj/rHShE+TSoi0gN4GQgE3lTVZ3OVi6e8F5AMDFbVXwprKyITgeuAdGAXcKeqHveUPQoMBbKAkaq68GxjDgoKyr7j2xhj/Flmlosu599InSq1CA32jzGCzy4pFpFAYDvQDYgHVgEDVXVzjjq9gPtxJ5V2wMuq2q6wtiJyLbBYVTNF5DkAVX1ERBoDs4G2QD3gG+ASVc0qKMb8Lik2xhhTOKcuKW4L7FTV3aqaDswB+uWq0w+YpW4rgOoiUrewtqq6SFUzPe1XAFE59jVHVdNU9Tdgp2c/xhhT7iz9dTdDpszk5Kk0p0M5gy+TygXAvhzb8Z7PilOnOG0BhgBfncXxEJFhIrJaRFbbvIkxpqy6//0XmHn4bnb/92jRlUuRL5NKfpch5D7XVlCdItuKyGNAJvDeWRwPVZ2uqnGqGhcZGZlPE2OM8W/b9h1hY9BMYlJuc3RJlvz4cmYnHrgwx3YUsL+YdYILaysig4A+wDX6v0mh4hzPGGPKvHtnToOgFCb2esjpUPLw5UhlFRAjItEiEgwMAObmqjMXuEPc2gMnVPVAYW09V4U9AvRV1eRc+xogIiEiEg3EACt92D9jjCl1R0+msOTUFCKP96Lf5U2cDicPn41UPFdnjQAW4r4seIaqbhKR4Z7y14H5uK/82on7kuI7C2vr2fWrQAjwteeu9xWqOtyz7w+BzbhPi91X2JVfxhhTFq3d9Qeh6Rfw984POx1KvmyVYruk2BhTxrhc7u9tp+6gt1WKjTGmHFi4eju/HzxOQID4xZIs+fGPWzCNMcYU6c/vDiUt4DjJk9b7bVKxkYoxxpQBby5Ywcka39Pr/Lv8NqGAJRVjjCkTnlg0CUmtzqtDhzgdSqEsqRhjjJ9bvG4Xf1T9hHaVhlOvVhWnwymUJRVjjPFzr309H7KCmDZ4pNOhFMkm6o0xxs/95+H7Wb39Rr9bkiU/NlIxxhg/dvRkCgBxl+S3pq7/saRijDF+6nhSKpFPxdDv2RedDqXYLKkYY4yfGvHmLFwRf9D50pZOh1JsNqdijDF+KDPLxYf7XiDM1YoH+l3ldDjFZknFGGP80Lj355FRdTvD687265sdc7OkYowxfui1NS8RGHQRzw++yelQzoolFWOM8UML7p7Fyh2/ERpctr6my1a0xhhTQbS77ELaXXZh0RX9jF39ZYwxfmTxul3UfqA781dudTqUErGkYowxfuSBOS+RUHUJ59eo6nQoJWJJxRhj/MSO+AQ2VJpBTMpttIqp53Q4JWJzKsYY4yeGz3gNglKY2Oshp0MpMUsqxhjjB44npbLk1BQi03vS7/ImTodTYpZUjDHGD2RmuehRdTTXtbjC6VDOiSUVY4zxA7WrVWb+Y39zOoxzZhP1xhjjsJc//457pr1Lanqm06GcMxupGGOMw8YueZxTQb/zEgOcDuWcWVIxxhgHvbXwZ07WWM71oS+VuSVZ8mOnv4wxxkHjFk6C1GpMvWuo06F4hSUVY4xxyOJ1u/ijyie0r3QP9WpVcTocr7CkYowxDtl54BBhJ5sz9Y77nQ7Fa0RVnY7BMXFxcbp69WqnwzDGmDJFRNaoalx+ZTZSMcYYB7y18Gf2JyQ6HYbXWVIxxphSdjwplb8s7kvbp+90OhSvK/vXrxljTBkz8s130MqHGNXsXqdD8TqfjlREpIeIbBORnSIyJp9yEZFXPOXrRaRVUW1FpL+IbBIRl4jE5fi8gYikiMg6z+t1X/bNGGNKIjPLxQf7XiDseCtGXd/F6XC8zmcjFREJBKYC3YB4YJWIzFXVzTmq9QRiPK92wDSgXRFtNwI3Av/K57C7VLWFj7pkjDHnbNz780ivuo0R579PQIA4HY7X+XKk0hbYqaq7VTUdmAP0y1WnHzBL3VYA1UWkbmFtVXWLqm7zYdzGGOMzC7YsJTDpIp4bdJPTofiEL5PKBcC+HNvxns+KU6c4bfMTLSJrReQ7EemYXwURGSYiq0Vk9eHDh4uxS2OM8Z41/3yRbaPWUjk0yOlQfMKXSSW/cV3um2IKqlOctrkdAC5S1ZbAg8D7IpLnIc+qOl1V41Q1LjIysohdGmOM9+w9dAKAi+vVdDgS3/FlUokHLsyxHQXsL2ad4rQ9g6qmqWqC5/0aYBdwSYkiN8YYL1u8bhf1XzmfB9/6yOlQfMqXSWUVECMi0SISDAwA5uaqMxe4w3MVWHvghKoeKGbbM4hIpGeCHxFpiHvyf7d3u2SMMSXzwJzJEJDFbZ3K9pMdi+Kzq79UNVNERgALgUBghqpuEpHhnvLXgflAL2AnkAzcWVhbABG5AZgCRAJfisg6Ve0OdAKeFJFMIAsYrqpHfdU/Y4wprh3xCWwInMGfkm+lVUw9p8PxKVv7y9b+Msb4WNcnJ/CtPs5n3TbS7/ImTodzzgpb+8vuqDfGGB9Kz8hiSdI0IjN6louEUhRLKsYY40PBQYEsue17TqakOh1KqbCkYowxPtYpNtrpEEqNrVJsjDE+MvbdedQZ1ZuNvx10OpRSYyMVY4zxkZdXP8+poN9pWLf83uyYmyUVY4zxgbcW/szJGsvpF/JiuV2SJT92+ssYY3xg3MJJkFqN1/5yl9OhlCpLKsYY42WL1+3ijyqf0C5wOPVqVXE6nFJlp7+MMcbLos+ryTWB43n+tiFOh1LqLKkYY4yXRdetwTdj/+F0GI6w01/GGONFw6bOKvcrERfGkooxxnjJ8aRU3tz7MO9unOl0KI4pNKmIyG053l+Rq2yEr4Iyxpiy6K9vvYtWPsQjHUc7HYpjihqpPJjj/ZRcZRVvBsoYYwqQmeVizt5JhB1vxajruzgdjmOKmqiXAt7nt22MMRXW+Pe/JL3qNkac/z4BARX367GokYoW8D6/bWOMqbBEhJrHuvLcoJucDsVRhT6kS0SScT+VUYCLPe/xbDdU1XCfR+hD9pAuY4w5e+fykK7LfBCPMcaUK0/O/orhPTpRp0aZ/jvbKwo9/aWqv+d8AUlAK6C2Z9sYYyq0pb/uZtzWPtw0+TmnQ/ELRV1SPE9Emnre1wU24r7q6x0RecD34RljjH8bOfsl0EAm3zbc6VD8QlET9dGqutHz/k7ga1W9DmiHXVJsjKngdsQnsCFwBn9KvpVWMfWcDscvFJVUMnK8vwaYD6CqiYDLV0EZY0xZcO/M1yE4medveMjpUPxGURP1+0TkfiAe91zKAgARCQMqzlNnjDEmH5uOriGSntxwRVOnQ/EbRSWVocCTQFfgZlU97vm8PVBxF7cxxhhg/0ufcOjYKafD8CuFJhVVPQTkmX1S1SXAEl8FZYwx/iwzy8W2fUdo0qCOXUacS6FJRUTmFlauqn29G44xxvi/8e9/yYTt/ZnRcRl3XtvW6XD8SlGnvzoA+4DZwM/Yel/GGMOUNZMIDDqPmzu1dDoUv1NUUjkf6AYMBG4BvgRmq+omXwdmjDH+aOailZyosYzrQ1+icqhdr5RbUXfUZ6nqAlUdhHtyfiew1HNFmDHGVDhjF0yC1GpMvWuo06H4pSKfUS8iIUBv3KOVBsArwCe+DcsYY/zPpj2HiI/4lHauh6hXq4rT4filoibq/w00Bb4Cxue4u94YYyqcJg3qsKDvJqJqV3c6FL9V1EjlduAUcAkwUiR7nl4AVdWqPozNGGP8hsulBAQI3eMucToUv1bUnEqAqlbxvKrmeFWxhGKMqUh6/vNZ6o26keTUjKIrV2BFrf11TkSkh4hsE5GdIjImn3IRkVc85etFpFVRbUWkv4hsEhGXiMTl2t+jnvrbRKS7L/tmjKk4jiel8k3iy2SSald8FcFnSUVEAoGpQE+gMTBQRBrnqtYTiPG8hgHTitF2I3AjsCzX8RoDA4AmQA/gNc9+jDHmnPz1rXdxVT7IIx1HOx2K3/PlSKUtsFNVd6tqOjAH6JerTj9glrqtAKp7nttSYFtV3aKq2/I5Xj9gjqqmqepvuC9/tltdjTHnJDPLxZzfXyDseEtGXd/F6XD8ni+TygW478Y/Ld7zWXHqFKdtSY6HiAwTkdUisvrw4cNF7NIYU9E9OXs+6dW2cuelowkIsEVFiuLLpJLfT1+LWac4bUtyPFR1uqrGqWpcZGRkEbs0xlR0N1/Zlu6BzzBxcH+nQykTirz58RzEAxfm2I4C9hezTnAx2pbkeMYYc1aaNKjDgn/kuc7IFMCXI5VVQIyIRItIMO5J9NyrHs8F7vBcBdYeOKGqB4rZNre5wAARCRGRaNyT/yu92SFjTMVy9fgnGf/+fKfDKFN8llRUNRMYASwEtgAfquomERkuIqef0TIf2I17Uv0N4N7C2gKIyA0iEo97BeUvRWShp80m4ENgM+4nVN6nqlm+6p8xpnxb+utulrjG89XmZUVXNtlEtaipivIrLi5OV69e7XQYxhg/1HzMSNYHv86a2/fQKqae0+H4FRFZo6px+ZX59OZHY4wpi3btP8r6wLf4U/KtllDOki8n6o0xpkwa/tY0CE7m+V4POR1KmWMjFWOMySW65oVcdupubriiqdOhlDk2UjHGmFym33cHcIfTYZRJNlIxxhiPzCwX90x7l6MnU5wOpcyypGKMMR5Pzp7P64du54k5nzsdSpllp7+MMcbjldUTCQy6iGfv+D+nQymzLKkYYwwwc9FKTtRYRr+QF+2ZKefATn8ZYwwwbsELkFqN1/5yl9OhlGmWVIwxFV5yagbHdC/tAodTr1YVp8Mp0+z0lzGmwqscGsSJF34kOc2eP3+ubKRijKnQ4g+fZEd8AgEBQkRYsNPhlHmWVIwxFdptU1/kkmn12bTnkNOhlAuWVIwxFdZtk6fzHU9yQUoPmjSo43Q45YLNqRhjKqTrnnmBeemjiTzRi/Xj33E6nHLDRirGmArn0X9/xrz00USd6M/uf35KzaphTodUblhSMcZUOOMG9mFglWnsem62Tc57mSUVY0yFkJ6RRadxY/llx35Cgyvx/oPDCQ4KdDqscseSijGm3EtOzeCSMbezPOAp/vnZx06HU67ZRL0xplw7npTKZY/fzH+rz6V74DP85+H7nQ6pXLOkYowpt/57NInGT17PsRrfcnPEVOY8dK/TIZV7llSMMeVWYnIaKXKEv9T+t+dpjsbXLKkYY8qdHfEJnFcjgpioWhx7bjWhwfZVV1psot4YU66s3v4HTV/sSPOxQwEsoZQy+2kbY8qNZet/45pZ15AZeoQH27/udDgVkiUVY0y5MO/nLfT7uCsalMLbXb5lULc2TodUIVlSMcaUeekZWdz44fVQKYuPen/H/13ZzOmQKixLKsaYMi84KJB/dX+HqFo16NY6xulwKjRLKsaYMmvix9+ydNsvfPn3h7nz2rZOh2Owq7+MMWXU4+98wd/W9WbxkXc4ejLF6XCMhyUVY0yZc/+/ZjNh5w2EJ8WycfRSW7rej9jpL2NMmTLo5TeZdWwY1U50ZONjXxAVWdXpkEwOPh2piEgPEdkmIjtFZEw+5SIir3jK14tIq6LaikhNEflaRHZ4/lvD83kDEUkRkXWel12kbkw5FCABRJ7oyc7xX1lC8UM+SyoiEghMBXoCjYGBItI4V7WeQIznNQyYVoy2Y4BvVTUG+NazfdouVW3heQ33Tc+MMaXN5VK+XrMDgJkjh/DfF+ZRu1plh6My+fHlSKUtsFNVd6tqOjAH6JerTj9glrqtAKqLSN0i2vYD/u15/2/geh/2wRjjMJdLaff437j201g+/3ETAAEB4nBUpiC+TCoXAPtybMd7PitOncLanqeqBwA8/62To160iKwVke9EpOO5d8EY46T0jCyajrmH1cGTaJY5lN7tLnM6JFMEX07U5/enhBazTnHa5nYAuEhVE0SkNfCZiDRR1ZNnHFBkGO5TbVx00UVF7NIY45Tk1AwaPzaY36u+T/vMMfzwz3/aCKUM8OVIJR64MMd2FLC/mHUKa3vQc4oMz38PAahqmqomeN6vAXYBl+QOSlWnq2qcqsZFRkaWsGvGGF/7y7QZ/F71fa4N/Cc/PfWMJZQywpdJZRUQIyLRIhIMDADm5qozF7jDcxVYe+CE55RWYW3nAoM87wcBnwOISKRngh8RaYh78n+377pnjPGlf4/8CxMaLWDhPx51OhRzFnyWVFQ1ExgBLAS2AB+q6iYRGS4ip6/Mmo/7i38n8AZwb2FtPW2eBbqJyA6gm2cboBOwXkR+Bf4DDFfVo77qnzHG+/YeOkH9hwby0+a9VAoM4LGbuzsdkjlLolrUVEX5FRcXp6tXr3Y6DGMMsG3fEVq+2J2UKuv5R8wnPHX7dU6HZAogImtUNS6/Mruj3hjjuF927Ofyad1IC9/NE40+Z9wtvZwOyZSQJRVjjKN+2ryXTjO6kBl2iJfivuKB669yOiRzDmxBSWOMo+rVqkrVzIa82ekbSyjlgI1UjDGOmPfzFi6/rAH1z6tOwuSvnQ7HeImNVIwxpW76Vz/R97PL6fDU/U6HYrzMkooxplS98Mli7v6+G4HptZgx+HGnwzFeZknFGFNqxr47j9FrexGS0oBV9y7niib1nQ7JeJnNqRhjSsXRkyk8vf5uKmc0Y91DC4iJquV0SMYHLKkYY0pFzaphfHL9IlrHXGgP1yrHLKkYY3zqxudf5vCpIywf/xT9Lm/idDjGx2xOxRjjEy6X0vXJCXya8gC7EzeTnpHldEimFFhSMcZ4nculdBg7hm/1cRom3s6uZz8gOCjQ6bBMKbCkYozxulaPPcDKoOdpknwP2557m9BgO9NeUdi/tDHG666o347Q+Ef48Rl7uFZFY0nFGOMVJ0+lMfu7Ndzd63KmDr8FuMXpkIwDLKkYY87ZkRPJNHriRhIiltIiegftLruw6EamXLKkYow5J/GHT9Lk6T6crP49g2u+aQmlgrOkYowpsR3xCbR4oQfJVdcx8oLZvPyXm50OyTjMkooxpsQemDWD5IgNjLv0M564tbfT4Rg/YEnFGHPWXC4lIED4Ysxo5q3sTd/2jZ0OyfgJu0/FGHNWFq7eTtWHOrB43S4CAsQSijmDjVSMMcX2n+Xr+fO8bhCsHDmZ5HQ4xg/ZSMUYUyxvLfyZP8/vTIArmC9vWs6fOzV3OiTjh2ykYowp0qxvVnPXsq5UyqjDksHfcmXTBk6HZPyUjVSMMUW6tlUj/pR2Ez/fvdwSiimUJRVjTIGe+XAR/z2axPk1I9gxaSatYuo5HZLxc5ZUjDH5uuvVt/n7pp5c98J4p0MxZYglFWNMHv0nvspbCXdS88Q1fDn6CafDMWWITdQbY87QfcIzLMr6O3WPX8/Wp+dQNTzE6ZBMGWJJxRg/5HIpJ5PTOJmcRmJyGkkpabS5NIqAAOGnzXtZvyeeU2lpJKWmkpKeRlpmBi8O7Q/Asx99zfc7fyUtM420zDRSM1MJCgzihyefBqDfsy+y6uD3ZJJKpqaRSRrhUosDL33OjvgEvjk5hQZZt7LpmZlUDg1y8sdgyiBLKsbkkp6RxX+PJXHoeBIJJ0+RmJLKqdQ0rmlxKVGRVVm1LZ6PflxJSnoap9JTSc1IIzUzjaf+PJAmDerw5oIVvL7sI9JdaaRnpZGhaWS40pg34mViG57P/f+azdtbppAlabgkjSxJRQPS2DRyLZdeWJvOT4xjmTyZJ67DD5yidrXK3PvOi6wLffnMQhUmuW4iIECYufIDtke85f5cBCSEwJRIwJ1U9p3Yx1HZQaCGEkgIlQghLKAqADFRtfjxzp9pfckFVAq0s+Pm7FlSMWWay6UcOZHMf48lUrtaOPVqVSH+8ElmfPMDx04lcjw5iZOpSSSmJTHsqj7c1DGWuSs2c++HY0nVJNI1kQxJIjMwiafb/4u/3dSV8XO+5J+7++U51gtJi3nwhi7MWrqcV/+b9wFU/XZcTpMGdfhh+2bWyOsIoYiEEKAhBEgIRxOTAQiqVIlgwqlETYIIpRIhBBNKcCX3M9z7xnaG9WMJCQwhLCiUkEohhAWFZD+Sd3y/YWza24vKISGEh4QQHup+nbb87y+T6XqJ6uGhhAZXyvPkxV+eeanQn6ktXW/Ohaiq0zE4Ji4uTlevXu10GBWOy6X8uPl3Dh5PJCExiYTERI6dSqJ1w4bc3LkFR04kc8OLz5CUnkhyZhIpWUmkaiL9L72VqcNv4ect+7h8ZltcQUkQdArE/Tv8f5Vf4T8P38+nP2zkxm+a5Tnu4BpvMXPkED74bh2DPr+VShpBsFYhRCIIC6jCEz3/yh1d41i2/jdemP8pVUIjqBoaTnhwKGHBIdzauT2XXlib3w4cY9WOvUSEhhARGkLV8FCqhIVwYWQ1goMCS/vHaUypE5E1qhqXX5lPRyoi0gN4GQgE3lTVZ3OVi6e8F5AMDFbVXwprKyI1gQ+ABsAe4M+qesxT9igwFMgCRqrqQl/2rzzJzHKRnpGVfQ59y97DHE9KITktnVNp6aSkpVMzIpxurWMAeHjGx+w/cYSTqUkkpSeRmJ5I7PlNmHH/nQDUGdWLZI6QIUlkBSbiqpREE9ctbHhuKplZLjp+HJ0nhlY7RnFz5xa4XMr3/BMkggCJoJJUoZJGkJTm/kv//JpViNE+hGsVwjWCiKAIqoZG0L9dRwA6N7uYNxJ/IrJqFWpXi+D8GlU4r0YEEWHBANzcuQU3d95U4M+iU2w0nWIfLLA8um4NouvWKMFP2Zjyz2dJRUQCgalANyAeWCUic1V1c45qPYEYz6sdMA1oV0TbMcC3qvqsiIzxbD8iIo2BAUAToB7wjYhcoqpZvupjYdIzsjiZnEZyWganUtM5lZpOcmo6VzZtkD3ZumnvflLS00nNyCAlPZ2MzEyeuv06AF79Yjkrdm0hLSOd9Kx00rMyqBRQiS8efQiAIVNmsvqPX8hwpZPpSidD06lSqTobnpsKQLt/PMLWpJ9wSQZZpOOSdKppAw6+9AUAkaN6ciT0JwhMd78Csog41oHEyT8C0HJyF9KqnfnFW+vYtRxp7c7TL21+iKwqv+corczx3/sD7qQiBFKZ2oTQgDAiCNMIro65EoDgoED+UvvfVAmtTM2IKtSMiKBWlQiaXFQXgNrVKpM1LjPPaZvT6p9Xna0T3yjwZ1+zahh39Whf3H8qY4wX+XKk0hbYqaq7AURkDtAPyJlU+gGz1H0OboWIVBeRurhHIQW17Qdc5Wn/b2Ap8Ijn8zmqmgb8JiI7PTH85IvODXr5TWbvfQ5XQDoq6WhAOhqQwebhu7jsokiufmocPwQ+naddQoNkalYNy3+y1RXAU7hz4CvL3mZHxIwzy1OrAe6ksuz379hd6QuEYESCEQ0iPC3qf7tSFwEEEqRhBEowlQjm/LD62eWXn9eNvScupZIGEazBBBNM9MUXZZff1+QJjp46SWhQMMGVgggNCqZhm/Oyy5cOXkrlkGDqVI+gTvXwPKd9Tievgky/744CywpKJsYY/+fLpHIBsC/Hdjzu0UhRdS4oou15qnoAQFUPiEidHPtakc++ziAiw4BhABdddFHu4mK7qOZ51NvbliANppIEEUQwQQQTHuo5xRJ3LVU2VCEkMJjgSp4v5krB2ZOtT1z3F379vTthwcHZr9CgoOyHH3351+dJTHmKsJAgwkODiQgNPuPyzp2T3i40vlVPTyy0/PMxBZ/eAXhh6E2Fltv6T8aY/PgyqeT352buqwIKqlOctiU5Hqo6HZgO7on6IvZZoKduvy77VFV+7u/bifv7diqwvN/lTeh3eZMCy2OiapU0NGOMcYwvL0SPB3JemxgF7C9mncLaHvScIsPz30NncTxjjDE+5MuksgqIEZFoEQnGPYk+N1educAd4tYeOOE5tVVY27nAIM/7QcDnOT4fICIhIhKNe/J/pa86Z4wxJi+fnf5S1UwRGQEsxH1Z8AxV3SQiwz3lrwPzcV9OvBP3JcV3FtbWs+tngQ9FZCiwF+jvabNJRD7EPZmfCdzn1JVfxhhTUdnNj3bzozHGnJXCbn60xX2MMcZ4jSUVY4wxXmNJxRhjjNdYUjHGGOM1FXqiXkQOA78XWbFgtYEjXgrHSeWlH2B98UflpR9gfTmtvqpG5ldQoZPKuRKR1QVdAVGWlJd+gPXFH5WXfoD1pTjs9JcxxhivsaRijDHGayypnJvpTgfgJeWlH2B98UflpR9gfSmSzakYY4zxGhupGGOM8RpLKsYYY7zGkooXiMhoEVERqe10LCUlIk+JyHoRWScii0SkntMxlZSITBSRrZ7+fCoi1Z2OqSREpL+IbBIRl4iUyctYRaSHiGwTkZ0iMsbpeEpKRGaIyCER2eh0LOdCRC4UkSUissXzu/VXbx/Dkso5EpELgW64l+EvyyaqaqyqtgDmAWMdjudcfA00VdVYYDvwqMPxlNRG4EZgmdOBlISIBAJTgZ5AY2CgiDR2NqoSexvo4XQQXpAJPKSqlwHtgfu8/W9iSeXcvQT8jaIfd+zXVPVkjs1wynB/VHWRqmZ6NlfgfgpomaOqW1R1m9NxnIO2wE5V3a2q6cAcoJ/DMZWIqi4Djjodx7lS1QOq+ovnfSKwBbjAm8fw5TPqyz0R6Qv8oaq/iojT4ZwzEXkauAM4AXRxOBxvGQJ84HQQFdQFwL4c2/FAO4diMbmISAOgJfCzN/drSaUIIvINcH4+RY8BfweuLd2ISq6wvqjq56r6GPCYiDwKjADGlWqAZ6GovnjqPIZ7uP9eacZ2NorTjzIsv7+0yuwIuDwRkQjgY+CBXGcpzpkllSKoatf8PheRZkA0cHqUEgX8IiJtVfW/pRhisRXUl3y8D3yJHyeVovoiIoOAPsA16sc3Y53Fv0lZFA9cmGM7CtjvUCzGQ0SCcCeU91T1E2/v35JKCanqBqDO6W0R2QPEqWqZXMFURGJUdYdnsy+w1cl4zoWI9AAeATqrarLT8VRgq4AYEYkG/gAGALc4G1LFJu6/gN8Ctqjqi744hk3Um9OeFZGNIrIe9yk9r19qWIpeBaoAX3sukX7d6YBKQkRuEJF4oAPwpYgsdDqms+G5WGIEsBD3hPCHqrrJ2ahKRkRmAz8Bl4pIvIgMdTqmEroCuB242vP/xjoR6eXNA9gyLcYYY7zGRirGGGO8xpKKMcYYr7GkYowxxmssqRhjjPEaSyrGGGO8xpKKMR4iknQObUd4VuI9Y7VqcXvFU7ZeRFrlKAsTke9EJFBEGnhrBVwReVtEbipGvaVFrX4sInNEJMYbcZmKwZKKMd7xA9AV+D3X5z2BGM9rGDAtR9kQ4BNVzSqVCEtmGu4FU40pFksqxuTiGV1M9NwMukFEbvZ8HiAir3meQzFPROafHhGo6lpV3ZPP7voBs9RtBVBdROp6ym4F8qzv5Rm1LBeRXzyvyz2fX+UZ2XwoIttF5FkRuVVEVnrivDjHbrp69rFdRPp42od5Rh7rReQDICzHMaeJyGpP38bn2M9yz75s9Q1TLPaLYkxeNwItgOZAbWCViCzDfTdyA6AZ7iV6tgAzithXfiv1XiAiCUDDAhLRIaCbqqZ6Tj3NBk6fpmoOXIZ7GfbdwJuq2tbzsKX7gQc89RoAnYGLgSUi8ifgHiBZVWNFJBb4JccxH1PVo55noHwrIrGqul5VXSKy03PcNUX01RgbqRiTjyuB2aqapaoHge+ANp7PP1JVl2fR0CXF2FdBK/XWBo4X0CYIeENENgAf4X7A1WmrPM/ESAN2AYs8n2/AnUhO+9AT5w7cyacR0Al4F0BV1wPrc9T/s4j8AqwFmuQ65iGgzD4J1JQuG6kYk1dBD8cpyUNzClqpNxUILaDNKOAg7tFBgKfuaWk53rtybLs48//n3OsvaQGf41nwcTTQRlWPicjbuWILBVIKiNWYM9hIxZi8lgE3e67KisT9F/5K4Hvg/zxzK+cBVxVjX3OBOzzzNO2BE56RxjEgUETySyzVgAOq6sK9+F9gCfrQ3xPnxUBDYJunX7cCiEhTINZTtypwCjjh6VfPXPu6BCiTC0Ga0mdJxZi8PsV9auhXYDHwN8/pro9xjzw2Av/C/cS8EwAiMtKzonAUsF5E3vTsaz7u0087gTeAe3McZxHuU2q5vQYMEpEVuL/QT5WgD9twn7b7Chiuqqm4r+SK8KxE/TfciRJV/RX3aa9NuOeIfji9E0+SSVHVAyWIwVRAtkqxMWdBRCJUNUlEauH+Ur6ipA9lE5GWwIOqertXg/QiERkFnFTVt5yOxZQNNqdizNmZJyLVgWDgqXN5yqeqrhWRJSIS6Mf3qhwH3nE6CFN22EjFGGOM19icijHGGK+xpGKMMcZrLKkYY4zxGksqxhhjvMaSijHGGK/5f9MxGcam1D7XAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_117_1.png" }, "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from sklearn.model_selection import train_test_split\n", "from sklearn import linear_model\n", "from sklearn.preprocessing import StandardScaler\n", "\n", "def MSE(y_data,y_model):\n", " n = np.size(y_model)\n", " return np.sum((y_data-y_model)**2)/n\n", "# A seed just to ensure that the random numbers are the same for every run.\n", "# Useful for eventual debugging.\n", "np.random.seed(315)\n", "\n", "n = 100\n", "x = np.random.rand(n)\n", "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)\n", "\n", "Maxpolydegree = 20\n", "X = np.zeros((n,Maxpolydegree-1))\n", "\n", "for degree in range(1,Maxpolydegree): #No intercept column\n", " X[:,degree-1] = x**(degree)\n", "\n", "# We split the data in test and training data\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n", "\n", "#For our own implementation, we will need to deal with the intercept by centering the design matrix and the target variable\n", "X_train_mean = np.mean(X_train,axis=0)\n", "#Center by removing mean from each feature\n", "X_train_scaled = X_train - X_train_mean \n", "X_test_scaled = X_test - X_train_mean\n", "#The model intercept (called y_scaler) is given by the mean of the target variable (IF X is centered)\n", "#Remove the intercept from the training data.\n", "y_scaler = np.mean(y_train) \n", "y_train_scaled = y_train - y_scaler \n", "\n", "p = Maxpolydegree-1\n", "I = np.eye(p,p)\n", "# Decide which values of lambda to use\n", "nlambdas = 6\n", "MSEOwnRidgePredict = np.zeros(nlambdas)\n", "MSERidgePredict = np.zeros(nlambdas)\n", "\n", "lambdas = np.logspace(-4, 2, nlambdas)\n", "for i in range(nlambdas):\n", " lmb = lambdas[i]\n", " OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)\n", " intercept_ = y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data\n", " #Add intercept to prediction\n", " ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler \n", " RegRidge = linear_model.Ridge(lmb)\n", " RegRidge.fit(X_train,y_train)\n", " ypredictRidge = RegRidge.predict(X_test)\n", " MSEOwnRidgePredict[i] = MSE(y_test,ypredictOwnRidge)\n", " MSERidgePredict[i] = MSE(y_test,ypredictRidge)\n", " print(\"Beta values for own Ridge implementation\")\n", " print(OwnRidgeBeta) #Intercept is given by mean of target variable\n", " print(\"Beta values for Scikit-Learn Ridge implementation\")\n", " print(RegRidge.coef_)\n", " print('Intercept from own implementation:')\n", " print(intercept_)\n", " print('Intercept from Scikit-Learn Ridge implementation')\n", " print(RegRidge.intercept_)\n", " print(\"MSE values for own Ridge implementation\")\n", " print(MSEOwnRidgePredict[i])\n", " print(\"MSE values for Scikit-Learn Ridge implementation\")\n", " print(MSERidgePredict[i])\n", "\n", "\n", "# Now plot the results\n", "plt.figure()\n", "plt.plot(np.log10(lambdas), MSEOwnRidgePredict, 'b--', label = 'MSE own Ridge Test')\n", "plt.plot(np.log10(lambdas), MSERidgePredict, 'g--', label = 'MSE SL Ridge Test')\n", "plt.xlabel('log10(lambda)')\n", "plt.ylabel('MSE')\n", "plt.legend()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see here, when compared to the code which includes explicitely the\n", "intercept column, that our MSE value is actually smaller. This is\n", "because the regularization term does not include the intercept value\n", "$\\beta_0$ in the fitting. This applies to Lasso regularization as\n", "well. It means that our optimization is now done only with the\n", "centered matrix and/or vector that enter the fitting procedure. Note\n", "also that the problem with the intercept occurs mainly in these type\n", "of polynomial fitting problem.\n", "\n", "The next example is indeed an example where all these discussions about the role of intercept are not present.\n", "\n", "## More complicated Example: The Ising model\n", "\n", "The one-dimensional Ising model with nearest neighbor interaction, no\n", "external field and a constant coupling constant $J$ is given by" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " H = -J \\sum_{k}^L s_k s_{k + 1},\n", "\\label{_auto1} \\tag{1}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $s_i \\in \\{-1, 1\\}$ and $s_{N + 1} = s_1$. The number of spins\n", "in the system is determined by $L$. For the one-dimensional system\n", "there is no phase transition.\n", "\n", "We will look at a system of $L = 40$ spins with a coupling constant of\n", "$J = 1$. To get enough training data we will generate 10000 states\n", "with their respective energies." ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from mpl_toolkits.axes_grid1 import make_axes_locatable\n", "import seaborn as sns\n", "import scipy.linalg as scl\n", "from sklearn.model_selection import train_test_split\n", "import tqdm\n", "sns.set(color_codes=True)\n", "cmap_args=dict(vmin=-1., vmax=1., cmap='seismic')\n", "\n", "L = 40\n", "n = int(1e4)\n", "\n", "spins = np.random.choice([-1, 1], size=(n, L))\n", "J = 1.0\n", "\n", "energies = np.zeros(n)\n", "\n", "for i in range(n):\n", " energies[i] = - J * np.dot(spins[i], np.roll(spins[i], 1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we use ordinary least squares\n", "regression to predict the energy for the nearest neighbor\n", "one-dimensional Ising model on a ring, i.e., the endpoints wrap\n", "around. We will use linear regression to fit a value for\n", "the coupling constant to achieve this.\n", "\n", "A more general form for the one-dimensional Ising model is" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " H = - \\sum_j^L \\sum_k^L s_j s_k J_{jk}.\n", "\\label{_auto2} \\tag{2}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we allow for interactions beyond the nearest neighbors and a state dependent\n", "coupling constant. This latter expression can be formulated as\n", "a matrix-product" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " \\boldsymbol{H} = \\boldsymbol{X} J,\n", "\\label{_auto3} \\tag{3}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $X_{jk} = s_j s_k$ and $J$ is a matrix which consists of the\n", "elements $-J_{jk}$. This form of writing the energy fits perfectly\n", "with the form utilized in linear regression, that is" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " \\boldsymbol{y} = \\boldsymbol{X}\\boldsymbol{\\beta} + \\boldsymbol{\\epsilon},\n", "\\label{_auto4} \\tag{4}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We split the data in training and test data as discussed in the previous example" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "X = np.zeros((n, L ** 2))\n", "for i in range(n):\n", " X[i] = np.outer(spins[i], spins[i]).ravel()\n", "y = energies\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the ordinary least squares method we choose the cost function" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " C(\\boldsymbol{X}, \\boldsymbol{\\beta})= \\frac{1}{n}\\left\\{(\\boldsymbol{X}\\boldsymbol{\\beta} - \\boldsymbol{y})^T(\\boldsymbol{X}\\boldsymbol{\\beta} - \\boldsymbol{y})\\right\\}.\n", "\\label{_auto5} \\tag{5}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We then find the extremal point of $C$ by taking the derivative with respect to $\\boldsymbol{\\beta}$ as discussed above.\n", "This yields the expression for $\\boldsymbol{\\beta}$ to be" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{\\beta} = \\frac{\\boldsymbol{X}^T \\boldsymbol{y}}{\\boldsymbol{X}^T \\boldsymbol{X}},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "which immediately imposes some requirements on $\\boldsymbol{X}$ as there must exist\n", "an inverse of $\\boldsymbol{X}^T \\boldsymbol{X}$. If the expression we are modeling contains an\n", "intercept, i.e., a constant term, we must make sure that the\n", "first column of $\\boldsymbol{X}$ consists of $1$. We do this here" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "X_train_own = np.concatenate(\n", " (np.ones(len(X_train))[:, np.newaxis], X_train),\n", " axis=1\n", ")\n", "X_test_own = np.concatenate(\n", " (np.ones(len(X_test))[:, np.newaxis], X_test),\n", " axis=1\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Doing the inversion directly turns out to be a bad idea since the matrix\n", "$\\boldsymbol{X}^T\\boldsymbol{X}$ is singular. An alternative approach is to use the **singular\n", "value decomposition**. Using the definition of the Moore-Penrose\n", "pseudoinverse we can write the equation for $\\boldsymbol{\\beta}$ as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{\\beta} = \\boldsymbol{X}^{+}\\boldsymbol{y},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where the pseudoinverse of $\\boldsymbol{X}$ is given by" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{X}^{+} = \\frac{\\boldsymbol{X}^T}{\\boldsymbol{X}^T\\boldsymbol{X}}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using singular value decomposition we can decompose the matrix $\\boldsymbol{X} = \\boldsymbol{U}\\boldsymbol{\\Sigma} \\boldsymbol{V}^T$,\n", "where $\\boldsymbol{U}$ and $\\boldsymbol{V}$ are orthogonal(unitary) matrices and $\\boldsymbol{\\Sigma}$ contains the singular values (more details below).\n", "where $X^{+} = V\\Sigma^{+} U^T$. This reduces the equation for\n", "$\\omega$ to" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " \\boldsymbol{\\beta} = \\boldsymbol{V}\\boldsymbol{\\Sigma}^{+} \\boldsymbol{U}^T \\boldsymbol{y}.\n", "\\label{_auto6} \\tag{6}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that solving this equation by actually doing the pseudoinverse\n", "(which is what we will do) is not a good idea as this operation scales\n", "as $\\mathcal{O}(n^3)$, where $n$ is the number of elements in a\n", "general matrix. Instead, doing $QR$-factorization and solving the\n", "linear system as an equation would reduce this down to\n", "$\\mathcal{O}(n^2)$ operations." ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "def ols_svd(x: np.ndarray, y: np.ndarray) -> np.ndarray:\n", " u, s, v = scl.svd(x)\n", " return v.T @ scl.pinv(scl.diagsvd(s, u.shape[0], v.shape[0])) @ u.T @ y" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "beta = ols_svd(X_train_own,y_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When extracting the $J$-matrix we need to make sure that we remove the intercept, as is done here" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "J = beta[1:].reshape(L, L)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A way of looking at the coefficients in $J$ is to plot the matrices as images." ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/4162706317.py:7: UserWarning: FixedFormatter should only be used together with FixedLocator\n", " cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAM2CAYAAACjUj0CAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABNyklEQVR4nO39fYxc1Zkg/j9V5fW7ieJvHMvY4VV4ZtYo4N+KJsN+N7DEMcQ2MSxg4OsMAQeSReQPQFlCIiaZQZNhINkgEbIMGjNkgAYSawksaxahjBaFCQSUkN3lxSbYGNZYzVtw/NIGm66q3x/GTTrt7r5tn6461ffzkRBS3eN7n3ruubfq6XPq3Eqz2WwGAAAApVFtdwAAAAC0lkIQAACgZBSCAAAAJaMQBAAAKBmFIAAAQMkoBAEAAEpmQrsDAKDz7dmzJ7q7u2Pt2rXx8ssvR6VSicMOOyyWLl0a5513XsyYMaO/7TXXXBM//elP45//+Z9j3rx5w+630WjEj3/84/jpT38aGzZsiEajEXPnzo1FixbFJZdcMmC/AEBxCkEADsobb7wRl1xySbz00kuxaNGiOOuss6LZbMZvfvObuOmmm+LHP/5x3HbbbXHUUUeNet9XX311PPzww/G5z30uzjjjjKhWq/Hcc8/F6tWr45FHHon77rsvPvrRj47BuwKA8U0hCMAB27NnT1x22WXx2muvxR133BF//ud/3r/tC1/4Qlx44YXx5S9/OS699NL47//9v8eUKVMK7/uZZ56Jhx56KK655pq4+OKLB2z79Kc/HVdccUWsXr06/tN/+k/J3g8AlIXfCAJwwB544IF4/vnn4+tf//qAInCf4447Lr75zW/Ga6+9Frfffvuo9v2b3/wmIiL+7b/9t4O2fe5zn4vZs2fH//pf/+uA4gaAslMIAnDAHnjggZg6dWqcddZZQ7b5/Oc/H7NmzYqHHnpoVPueNm1aRET85Cc/iUajMWj7z372s+ju7h5dwABARCgEAThA9Xo9nn322fjX//pfx6RJk4ZsV6lU4sQTT4xXXnkl3nrrrcL7X7x4cXzkIx+Ju+66KxYtWhQ33HBD/PznP49du3ZFRMTEiRMP+j0AQFkpBAE4INu2bYs9e/bErFmzRmz78Y9/PCIi3nzzzcL7nzlzZvzDP/xDfOITn4gtW7bEP/7jP8all14aXV1d8R//43+M//N//s8Bxw4AZacQBOCANJvNiIio1Wojtp0wYcKAf1PUcccdF4888kjcdtttcd5558W8efPi/fffj//5P/9nnHfeeaOebgoA7GXVUAAOyMyZM+Nf/at/Fb/73e9GbLtvJHDfyOBoTJgwIU455ZQ45ZRTIiLi5ZdfjnvuuSfuuuuu+Ju/+Zv47Gc/G5MnTx71fgGgzIwIAnBAKpVKLFy4MJ599tnYvXv3kO2azWb8+te/jk984hOjKgRvueWWWLNmzaDXjzrqqLj22mvj/PPPj9///vexYcOGA4ofAMpMIQjAAVu+fHns3LkzfvzjHw/Z5p//+Z9j8+bNccYZZ4xq3w888EDcdtttQ04nnT9/fkTEqJ5NCADspRAE4ID9h//wH2LhwoXxn//zf45/+Zd/GbR93bp18Zd/+Zcxb968uOSSS0a17zPOOCM2b94cf//3fz9o2+7du+OBBx6II444Io466qgDjh8AyqrSHO0v9wHgD7z99ttx2WWXxbPPPhuLFy+OE088MWq1Wvzv//2/46GHHoo5c+bEf/kv/yWOOeaYiIi45ppr4qc//WksW7as/1mBf+hzn/tc/Pmf/3m89957sWrVqvj1r38dCxcujM985jMxc+bM6OnpiYceeihef/31+Md//Mf4N//m37T6LQNAx1MIAnDQ9uzZEw888EDcf//9sWnTpujr64vDDjsslixZEueff37MmDGjv+2+QnAo3/jGN+Kiiy6KiIi+vr64995743/8j/8RGzdujN7e3pg5c2acdNJJ8ZWvfCWOPPLIsX5rADAuKQQBAABKxm8EAQAASkYhCAAAUDIKQQAAgJKZ0O4AAAAAGBvbt2+P7du3D3rdYjEAAAAH6d133okpM2e2O4xB3nvvvfj0pz8d27ZtG/B6toVgozH89mp15DYREUcdVU8TUES88nIl2b4aCWflVqNAIlIqmPyOfo9tUCRfRft96nylPJedrF35T6WTz2PR3JdBrvf8lMrw+ZHynl8WKc/lSPkfTe5z7WMpHXHUwZcL8+ZF/Mu/1BJEk7d//H//39j+2mvtDqPfIfPmxap/+Zfo6emJen1gXTTup4a++mq7IwAAgM7l+3Rx2197LbZlmLA5c+YMem3cF4IAAACtUI28VuMcLpbkcW7evDm++tWvRldXV3R1dcXVV18d77zzTurDAAAAcICSjghu3bo1vvjFL8aePXvikksuiXq9Hrfffnu8+OKLsWbNmpg4cWLKwwEAAHAAkhaCP/rRj+L111+Phx56KI4++uiIiDjuuOPi4osvjgceeCBWrFiR8nAAAADZKO3U0LVr10ZXV1d/ERgRcdJJJ8WRRx4Za9euTXkoAAAADlCyQnDbtm2xefPmWLBgwaBtCxYsiOeeey7VoQAAADgIyaaGvvHGGxERMXv27EHbZs2aFTt37owdO3bEjBkzUh0SAAAgG500NTRZIdjb2xsREVOmTBm0bdKkSRERsWvXrsKFYLVABou0aTbzfHBl2g7Shu5WIPkd/x5brOg7LNLvZX/stCP/qeQZVXHFcl8Ged7zkx4u472lkvaeXxbpklFkT8VzP/5PUvPgnydPhpIVgo1GY8Q21VHczUbaXbU6cpuIiFqtXviYI2nWK8n21Uh6MyuQiJQKJr+j32MbFMlX0X6fOl8pz2Una1f+U+nk81g092WQ6z0/pTJ8fqS855dFynM5Uv5Hk/tc+1hKldrBV4KHHx7xyit5DtCUVbJCcNq0aRERsXv37kHb9r22rw0AAMB4U4m8xoiHG8ZKFuehhx4aERFvvfXWoG1vvvlmHHLIITF16tRUhwMAAOAAJSsEDznkkJg3b148//zzg7a98MILceyxx6Y6FAAAAAch6cjl4sWL48knn4yNGzf2v/bEE0/Epk2bYsmSJSkPBQAAkJVqhv8NJdlvBCMiLr300njwwQfjoosuilWrVsXu3btj9erVsWDBgli+fHnKQwEAAHCAko4Izpw5M+6+++740z/907j55pvjn/7pn2LRokWxevXqmDhxYspDAQAAcICSjghGRBx11FHxD//wDwn2U49XXx16e7NZK/RoiHo93TK11YTZavS1bgnk0Ui5BHIZllNufe6rhdqlfkxArueykx+H0A6tXHp9NHLu+6l08jVULdzO58do5Nzvc81/a+87xXKfs5T5qqd7Gtu410kPlM8pTgAAAFpAIQgAAFAyyaeGAgAAlJGpoQAAAGRLIQgAAFAypoYCAAAkUIm8Rtoqw2zLKU4AAABaQCEIAABQMqaGAgAAJGDVUAAAALKlEAQAACgZU0MBAAASqMTwK3W2mlVDAQAA6KcQBAAAKBlTQwEAABKoffBfLoaLxYggAABAySgEAQAASsbUUAAAgAQqkddI23CrhmZbCL7y8sgLrzbrI7epJnyHfX3p9lWpNZPtq15PtqtoFOi61YLtUqpGI9m+Wh17O6TMV87K0C9SxpUyX2XpY+NdsfNY7ejzneu13ck5HY1c85+rXPOVrr/m+f7KytkAAAAomWxHBAEAADpJNfIaaRsulpziBAAAoAUUggAAACVjaigAAEACpoYCAACQLYUgAABAyZgaCgAAkICpoQAAAGRLIQgAAFAypoYCAAAkUIm8Rtoqw2zLKU4AAABaQCEIAABQMqaGAgAAJGDVUAAAALKlEAQAACgZU0MBAAASqMTwK3W2mlVDAQAA6JftiGBjhBq1WqBNRESjr5EooohKrZlsX816ur8VVGr1ZPsqFlc1qpEur52sLHkocq0VlWvOisVVrO+nzFdKucZVRNF7fq79K9drqFhOW993cj2PtJd+MTryxUiyLQQBAAA6Se2D/3IxXCyd++dhAAAADohCEAAAoGRMDQUAAEigEnmNtFk1FAAAgH4KQQAAgJIxNRQAACCBauQ10jZcLDnFCQAAQAsoBAEAAErG1FAAAIAETA0FAAAgWwpBAACAkjE1FAAAIAFTQwEAAMiWQhAAAKBkTA0FAABIoBJ5jbRVhtmWU5wAAAC0gEIQAACgZLKdGlqNxogtRm4T0UhY69bryXYV1YSZTxlXpTbyzprNiEqtOXK7+nCD0e1TpN8UlbJ/FVEteMyU75H2yvVctrrvF5VrXCmlfI/F+lfrP29T6uR7/niQ6z0spdZfk8XkGtd4Z9VQAAAAsqUQBAAAKJlsp4YCAAB0kkoMv1Jnq1k1FAAAgH4KQQAAgJIxNRQAACCB2gf/5WK4WIwIAgAAlIwRQQAAgAQqkddIm8ViAAAA6KcQBAAAKBlTQwEAABKoRl4jbcPFklOcAAAAtIBCEAAAoGRMDQUAAEjA1FAAAACypRAEAAAoGVNDAQAAEjA1FAAAgGyN+xHBajTaHcJ+9fWlq8FrtXqyfdXrtWTtqgl7V19fun3l2idyljJnjYR/f2p1XNWC7VLKNV8pFYurmm38ReR6HnPt97RXGc53rveTXO8VdL6enp6o1wfWDOO+EAQAAGiFSuQ15bLywf9XrlwZW7ZsGbBNIQgAADCOdXd3GxEEAAAokzlz5gx6TSEIAACQQCetGpq8EDznnHPi2WefHfT6aaedFjfffHPqwwEAADBKSQvBZrMZGzdujEWLFsXixYsHbJs7d27KQwEAAHCAkhaCr732WuzatSs+85nPxPLly1PuGgAAIGuV+HClzhwMF0vSKawbNmyIiIijjz465W4BAABIKGkh+NJLL0XEh4Xgrl27Uu4eAACABJIXgtOmTYvrr78+Fi5cGAsXLoxFixbF2rVrUx4GAAAgO7UM/xtK0t8IbtiwIXp7e2PHjh1x4403xvbt2+POO++Mq666Kt5///0488wzi++sWqBGLdImUykjbzaHO8Vjo0jqG42xj+PApMt+O3pgsW6f77WRNrLWn8sOvu1Evn2/4N46OPn6/Wjlea5z7fd5Zit36b5n5pv/fCMjD5Vms9lMtbN77703Go1GrFy5sv+19957L5YtWxbvvvtu/PznP49arWDRMlIVUa3mXGmMqJHw4qzV6sn2Va+PfH6Kpn5Cwj8z9PWl21c10vWblOexiKK5T/keU0uZs1afyw6/7WTb9wvF1eHJ1+9HJ9d7WK79vtWfRePBiPkfRcfPNf9ZXkcd/Ae9op454ojY/eqr7Q6j36TDD4//3yuv7Hdb0rNxwQUXDCgCIyImT54cy5cvj7fffrt/MRkAAIDxphIfPlQ+h/9atmroUGbOnBkRFo8BAADIQbJC8I033oilS5fGLbfcMmjbpk2bIiJi3rx5qQ4HAADAAUpWCM6ePTt27NgRa9asiZ07d/a/3tPTE/fff3+ceOKJMWvWrFSHAwAAyEq7p4Lu77+hJF019Fvf+lZcfvnlcf7558e5554bvb290d3dHRMmTIhvf/vbKQ8FAADAAUr6G8FFixbFD3/4w5gyZUp873vfizvuuCOOP/74uPfee/sfMg8AAEB7JR0RjNhbDC5atCj1bg9YGZb0bdaHWw9odKoFekSjUezRECkf+ZDyERkp85VSsT5RLdQudb/PcgnqkihD7gs9wqBwuw5+REZBuX6u0T453ydyvY5SyvX6LkPuczTSdMxWGy6WnOIEAACgBRSCAAAAJZN8aigAAEAZmRoKAABAthSCAAAAJWNqKAAAQAKVyGukbbi18nOKEwAAgBZQCAIAAJSMqaEAAAAJWDUUAACAbCkEAQAASsbUUAAAgAQqMfxKna1m1VAAAAD6KQQBAABKxtRQAACABGoR0Wh3EH+gNsw2I4IAAAAloxAEAAAoGVNDAQAAEqhEXiNtw60aqhBkgL6+dO1qtfrBBfMHmvV0C/FWJ6S7PIvmq4hGgdtGtWC71FIes5rVzPkPFYurWqhdO85Rq7X+PBbLfa46+xpqfb/PNV+59sGc7zmdmrPRfN7m+h5TxpWqj+XbU8vJ+QAAACgZI4IAAAAJVCOvkbbhYskpTgAAAFpAIQgAAFAypoYCAAAkYGooAAAA2VIIAgAAlIypoQAAAAmYGgoAAEC2FIIAAAAlY2ooAABAApXIa6StMsy2nOIEAACgBRSCAAAAJWNqKAAAQALViGi2O4g/YNVQAAAA+ikEAQAASsbUUAAAgAQqMfxKna1m1VAAAAD6jfsRwWo02h3CmGskrOeL5ataqF2znu7vIZVaup/d1uvJdhW1Wrqd1eu1ZPvKud+3vr8WUySuauF2rY2rqFzjKqJo7tMeM8/rKNfc59q/Wp2volLmK3Vflf+9LdpxD8j1OmJ8GveFIAAAQCuk+7N+GsPF408FAAAAJaMQBAAAKBlTQwEAABKoRF4jbVYNBQAAoJ9CEAAAoGRMDQUAAEggt1G24eLJLVYAAADGmEIQAACgZEwNBQAASCC3UTZTQwEAAOinEAQAACgZU0MBAAASyG2UzdRQAAAA+ikEAQAASsbUUAAAgAQqkddIW2WYbTnFCQAAQAsoBAEAAEpm3E8NbSSsdavRSLavXONKKeV7rNeT7SpqtXQ7a9aHG3AfnUqBuJrNWqH46/VaipDGRK7XUa5yvb5TKvYeq6XIRavlmnvnenRS3wvLkP+RclYt0ObDtuM/XxRXjYhmu4P4A6aGAgAA0E8hCAAAUDLjfmooAABAK5gaCgAAQLYUggAAACVjaigAAEACpoYCAACQLYUgAABAyZgaCgAAkEAlhp+OmRMjggAAACWjEAQAACgZU0MBAAAS6KRRtk6KFQAAgAQUggAAACVjaigAAEACnTTK1kmxAgAAkIBCEAAAoGRMDQUAAEigk0bZsi0EGyOksVqgzd52jUQRpZUyriJ5KCplXLm+x3q9lmxflVo92b6KxlWk3YTEV3ajL8/rKKVi/bVaqF2u12TKuHKV63vM9d5K+xT7DtOePl2G6ygl+RqNPHNVVtkWggAAABy8np6eqNcHDmAoBAEAABKoVCoRlUq7w/jQB7GsXLkytmzZMmCTQhAAAGAc6+7uNiIIAAAwJiZMSL9Yw8H4IJY5c+YM2nRAv9i89tpr4y/+4i8Gvb558+b46le/Gl1dXdHV1RVXX311vPPOOwdyCAAAAMbIqMvVNWvWxJo1a6Krq2vA61u3bo0vfvGLsWfPnrjkkkuiXq/H7bffHi+++GKsWbMmJk6cmCxoAAAADlzhQrBer8ett94at9xyy363/+hHP4rXX389HnrooTj66KMjIuK4446Liy++OB544IFYsWJFmogBAAByVKvlNTW0NvRjxwpNDd29e3ecddZZ8YMf/CCWL18es2fPHtRm7dq10dXV1V8ERkScdNJJceSRR8batWsPIGoAAADGQuFCcOfOnXHTTTfFDTfcEBP+qMrdtm1bbN68ORYsWDDo3y5YsCCee+65NNECAABw0AqNW06fPj0effTRQQXgPm+88UZExH5HCmfNmhU7d+6MHTt2xIwZMw4iVAAAgIxlumrofjcV+ffVajWq1aEHD3t7eyMiYsqUKYO2TZo0KSIidu3aNapCcJjDjarNAS6M2lHSvsOCeyuW/GRyPYvN5tDzrsdKkdQ3GsmPmnqHSeTa99sSV0v3lFquuU8p38gKafE9vwyKZlTq/1Br74edn/uOfwOMsSTlaqPAt87hCsn973Ok/RX7sluN5N+Is9NIemMsktSCyU8o5XtMqVarj9yooHp95KKyaOpT/yGq0ZfndZRr3295XAXleh3lmvuUOvqzqA33/DIo0lelfqBW3g/HQ+6zvO90fnU9riT5ujht2rSI2Ptbwj+277V9bQAAAMal8bZq6EgOPfTQiIh46623Bm17880345BDDompU6emOBQAAAAHKUkheMghh8S8efPi+eefH7TthRdeiGOPPTbFYQAAAEgg2UTdxYsXx5NPPhkbN27sf+2JJ56ITZs2xZIlS1IdBgAAIE/7Vg3N6b+hQk31ni+99NJ48MEH46KLLopVq1bF7t27Y/Xq1bFgwYJYvnx5qsMAAABwkJKNCM6cOTPuvvvu+NM//dO4+eab45/+6Z9i0aJFsXr16pg4cWKqwwAAAHCQKs1ms9nuIPbH4yOKy3UJ/ZRyXRLe4yPaK9e+7/ERo5Nr7lPq6M+i8bCOfoY8PmL0PD5idLK875Th8RFdXRGvvdbuKD40b17E00/vd1NGa5sONHLnrRbq4L4U8MdS5r5I8VZUkaKy2awVapcyroiI6oR011GuRWVKuV7fZSgqU8r1PNI+rf8jT3u+6+Ta91ub/2K5z1mO9+n8Iio35wMAAKBksh0RBAAA6CgjrNTZcsPEYkQQAACgZBSCAAAAJZPRuCUAAEAHq9XymhpaG3oBQSOCAAAAJaMQBAAAKJmMxi0BAAA6mFVDAQAAyJVCEAAAoGQyGrcEAADoYFYNBQAAIFcKQQAAgJLJaNwSAACgg1k1FAAAgFwpBAEAAEomo3FLAACADmbVUAAAAHKVUbk6UGOEGrVaoM3edo1EERU7Xjv2xfjQrFeStasmvrL7+tLtq1JrJttXvZ5sV0nlen2nvB+m3BftU+xztLM/b1Nq9XssmvvUypD/lMqQr1zfIwcn20IQAACgo1g1FAAAgFwpBAEAAEomo3FLAACADmbVUAAAAHKlEAQAACiZjMYtAQAAOphVQwEAAMiVQhAAAKBkMhq3BAAA6GBWDQUAACBXCkEAAICSyWjcEgAAoINZNRQAAIBcKQQBAABKJqNxSwAAgA5m1VAAAABypRAEAAAomYzGLfNXjUayfTXU4KOSa+5TxpVSoy9tXNUJ6XJWrw89RWG0arV6sn0165Vk+0op1z6WUpFrslq43fjPV65y/VzTJ0Yv189cRkfu28SqoQAAAORKIQgAAFAyGY1bAgAAdDCrhgIAAJArhSAAAEDJZDRuCQAA0MGsGgoAAECuFIIAAAAlk9G4JQAAQAezaigAAAC5UggCAACUTEbjlgAAAB3MqqEAAADkSiEIAABQMhmNWwIAAHQwq4YCAACQK4UgAABAyWQ0bgkAANDBOmjV0IyiHKgajRFbjNwmopHpoGeR2DtdytynzFfKfeX6HlNr9KWLrVJrJttXvT70vPfRqtTqI7ZpNovF36zneS5zvR+mlOs1mWvui73HYp+3KeWar5RyzX1EOa4j3yvaZfxf253E2QAAACiZbEcEAQAAOopVQwEAAMiVQhAAAKBkMhq3BAAA6GAdtGqoEUEAAICSUQgCAACUTEbjlgAAAB3MqqEAAADkSiEIAABQMhmNWwIAAHQwq4YCAACQK4UgAABAyWQ0bgkAANDBrBoKAABArhSCAAAAJZPRuCUAAEAHs2ooAAAAucqoXB2oMUKNWi3QhtErktNOz30nx15U6vdYjUayfTXrlWT7qia8g9XrQ/+YerTtKrX6wYbTL2m+Ep7HlH2sWFzVpPG3WifH3g655qv1/b6YnD/Xco0t1++ZufZ9Ol9PT0/U6wO/n2RbCAIAAHSUTFcNXblyZWzZsmXApoyiBAAAILXu7m4jggAAAGUyZ86cQa8dUCF47bXXxquvvhp33XXXgNfPOeecePbZZwe1P+200+Lmm28+kEMBAAB0hg5aNXTUUa5ZsybWrFkTXV1dA15vNpuxcePGWLRoUSxevHjAtrlz5472MAAAAIyRwoVgvV6PW2+9NW655Zb9bn/ttddi165d8ZnPfCaWL1+eLEAAAADSKlQI7t69O84999x48cUX48wzz4wnn3xyUJsNGzZERMTRRx+dNkIAAIBOkOmqoftT6AEpu3fvjp07d8ZNN90UN9xwQ0zYz5t76aWXIuLDQnDXrl0HEioAAABjrFC5On369Hj00Uf3WwDu89JLL8W0adPi+uuvj4cffjh27doVn/jEJ+LKK6+MpUuXJgsYAACAg1OoEKxWq1GtDj94uGHDhujt7Y0dO3bEjTfeGNu3b48777wzrrrqqnj//ffjzDPPHFVgIxyucJt85Rl80ahan/t0B8wz8xGFIyuQ/PTvMc+sNRqtP2aRvt9sDj0NY7xI2yPS9X3GiNxHRL793tkZG+3p9s5mxxvPq4YOZcWKFdFoNGLlypX9ry1dujSWLVsW3/3ud+OMM86I2jBzVP/YSF/wqtX2fAlMpRp5Bt8ocANqR+5T5qvIe2yHQu+xYPJTv8dc+2t1Qrr32ddX4HgF+36tVh+5UUHNeiXZvlJK2cdS9n3GgNz3y7Xf5/q51sna1e1z/bxNxh+VspLsbFxwwQUDisCIiMmTJ8fy5cvj7bff7l9MBgAAgPYa83HLmTNnRoTFYwAAgHFuvK0aOpI33ngjli5dut9nDG7atCkiIubNm5fiUAAAABykJIXg7NmzY8eOHbFmzZrYuXNn/+s9PT1x//33x4knnhizZs1KcSgAAAAOUrJxy29961tx+eWXx/nnnx/nnntu9Pb2Rnd3d0yYMCG+/e1vpzoMAABAnjpo1dBki8UsWrQofvjDH8aUKVPie9/7Xtxxxx1x/PHHx7333tv/kHkAAADar9JsNpvtDmJ/PD6iPTw+on1yXko81/7q8RHtk+sy+owBue+Xa7/P9XOtk3l8xBgpw+Mj7rkn4g9+Ktd206dH/H//3343ZTRuOdDIF0K1oy8WN+3RafmHb0Ep4ypUhBdul/bayLW/NvrSvc8iRWWjUWy2R72e7oHylYRFZcq4Ovn+W1Su/b7Vit530h4zz/t0SmW55+d6LnP9nplr7hmf8rw7AgAAMGayHREEAADoKGVcLAYAAIDOoBAEAAAomYzGLQEAADpYrZbX1NDa0AvFGREEAAAoGYUgAABAyWQ0bgkAANDBrBoKAABArhSCAAAAJZPRuCUAAEAHs2ooAAAAuVIIAgAAlExG45YAAAAdzKqhAAAA5EohCAAAUDIZjVsCAAB0MKuGAgAAkKuMytWx0UhY61ajkWxfKbU+rmrLj5nyPHZ2n2h97sui0Vckr9VC7Sq15sEH9IFmvZJsX5VaPdm+6vWh/8I4Wrn26ZRx5XrfEVf75Pq5llrOsaWSa3/NMff5RVRu474QBAAAaAmrhgIAAJArhSAAAEDJZDRuCQAA0MGsGgoAAECuFIIAAAAlk9G4JQAAQAezaigAAAC5UggCAACUTEbjlgAAAB3MqqEAAADkSiEIAABQMhmNWwIAAHSwCRMi6vV2R/Ehq4YCAACwj0IQAACgZEwNBQAASMGqoQAAAORKIQgAAFAyGY1bjo1qNNodAgk4j+2VMv+NhH9/SrmvlO+xXh96GsZoVRPepZv1dO8xZVx9fSOfx2oUO9+59tWUco0rpVzPY66fRanjyjVnrfz8KHrPyVme/bWzc1rIhAkRjYxyb9VQAAAA9lEIAgAAlMy4nxoKAADQElYNBQAAIFcKQQAAgJLJaNwSAACgg02YENFstjuKD1k1FAAAgH0UggAAACVjaigAAEAKtVpeU0OtGgoAAMA+CkEAAICSMTUUAAAghZweJh9h1VAAAAA+pBAEAAAomczGLgEAADrUMKt0toVVQwEAANhHIQgAAFAypoYCAACkMGFCRKXS7ig+NMzU0HFfCDYSDnpWo5HlvlK+xyKqBY+Z8j3mKtf+lbpPlKHvF+vTre/7fX3p3mOl1ky2r2Y9YZ+YMPJ7bDSKrcjd6EsQUOY6+Rpqh1zzlbNcc1aGuFLKsb/mF1G5OR8AAAAlM+5HBAEAAFqiVstramh16HE/I4IAAAAloxAEAAAoGVNDAQAAUpgwYe8KZ7kwNRQAAIB9FIIAAAAlY2ooAABACrXasNMxW26YFUwzihIAAIBWUAgCAACUjKmhAAAAKUyYENFstjuKD5kaCgAAwD4KQQAAgJIxNRQAACCFWq3dERRmRBAAAKBkFIIAAAAlY2ooAABAChM6p7wyIggAAFAy2ZasjRFq1GqBNnvbNRJFlFaR2DtdyveY63nMNa6c5dr3i53LasvPecrj1evpfsBeTfjp0deXrl2llu7ZTc16ntd36++tre/3uWp9HorlPvV91fkenTJ836Hz9fT0RL1eH/BatoUgAABAJ8nxD97ViFi5cmVs2bJlwOuVZrOZ7s+nCTVG+INItTpym4h8/7KSYycpqmjukx4z0/PYcgWTn/Nfh3Pt+4XeYzs6f0Ipc5/yJxBFRvqKpr5Wq4/cqKBmvZJsXym1fPShw/t9R3PPH6CVcbWr2+ea+1Sq+YWUXI63y2rViCAAAEDpzJkzZ9BrhQvBxx9/PG699dZ4/vnno1qtxnHHHRdXXHFFHH/88f1tNm/eHDfccEM8/fTTERFxyimnxDXXXBMzZ848+OgBAAAyVvR37600ceL+Xy80NfTpp5+OCy+8MI455pg4++yzo6+vL+655554880345577olPfvKTsXXr1jj77LNjz549ceGFF0a9Xo/bb7895s6dG2vWrImJQ0UwBFND82VqaBuZJjRmyjBFztTQ0TE1dF+jzu73Hc09fwBTQ0cnx8/bMkwN3bOn3REMNlQZVuij/G//9m9jzpw58ZOf/CSmTJkSERFnnnlmLFmyJG666aa444474kc/+lG8/vrr8dBDD8XRRx8dERHHHXdcXHzxxfHAAw/EihUr0rwTAAAADsqIdfm2bdti/fr1cfrpp/cXgRERH/vYx+KEE06I3/zmNxERsXbt2ujq6uovAiMiTjrppDjyyCNj7dq1YxA6AABAPur1vTNdcvmvPsxEmRFHBKdPnx6PPPLIgCJwn61bt0atVott27bF5s2b47TTThvUZsGCBfHYY4+NKoEAAACMnRFHBGu1WhxxxBExe/bsAa+vX78+nnnmmVi4cGG88cYbERGD2kREzJo1K3bu3Bk7duxIFDIAAAAH44B+7t/b2xtf//rXIyLiy1/+cvT29kZE7HfUcNKkSRERsWvXrpgxY0bhYxT5MWmxH5zm+avUPKMqrvU/9u30jCVUIPnps5Xyx/65KhhZB//SPWXkbVlEocAbaDZrYx9Im6VeFqRYs87t9x3PPX9M9lZkT+3p9rnmnqL6+iJyekp7ZZh1z0ZdCL777rtx2WWXxfr16+MrX/lKdHV1xa9//esR/111lFeTVUPzZdXQNrKC3Jgpw+qJVg0dHauG7mvU2f2+o7nnD2DV0NHJ8fPW35TyMqrTsX379li1alU89dRTcfbZZ8eVV14ZERHTpk2LiIjdu3cP+jf7XtvXBgAAgPYq/Dfd3/3ud/GlL30p1q1bF+edd1789V//dVQ+GGs89NBDIyLirbfeGvTv3nzzzTjkkENi6tSpiUIGAADIT72e1ySK4UZhCxWCO3fu7C8CL7roovjGN74xYPshhxwS8+bNi+eff37Qv33hhRfi2GOPHV3EAAAAjJlCU0Ovu+66WLduXVx44YWDisB9Fi9eHE8++WRs3Lix/7UnnngiNm3aFEuWLEkTLQAAAAet0mwOv67Nxo0bY8mSJTFjxoz45je/GbXa4BXZli9fHu+8804sW7YsarVarFq1Knbv3h2rV6+Oww47LO67776YOHHiqAKzWEy+LBbTRhYOGDNlWDTDYjGjY7GYfY06u993NPf8ASwWMzo5ft6WYbGYt9/O65ZZrUZ87GP73zZiIXjvvffGX/3VXw17gBdffDEiIl5++eW4/vrr41e/+lVMnjw5Tj755Lj66qtj5syZow5aIZgvhWAb+VIwZsrwhVghODoKwX2NOrvfdzT3/AEUgqOT4+etQrD1DqoQbBeFYL4Ugm3kS8GYKcMXYoXg6CgE9zXq7H7f0dzzB1AIjk6On7cKwdYbrhBM+FGepxwvgojxf6HnrFM/lPYerz3nO9ecMTpJ+2uB4q2oSm3kv0c2m7VCRV69nu6B8pWERWXKuFJKed9xbY9OWXKfa2wjx1XNNvai8ox//H9nrdf3/peL/fyqr9/4PxsAAAAMoBAEAAAomXE/NRQAAKAV+vrymho63GowRgQBAABKRiEIAABQMqaGAgAAJFCvF3ssUg6MCAIAAJSMQhAAAKBkTA0FAABIoK/P1FAAAAAypRAEAAAoGVNDAQAAEsht1dBKZehtRgQBAABKxoggAABAArktFmNEEAAAgH4KQQAAgJIxNRQAACCB3BaLqQ4z7GdEEAAAoGTG/YhgNRrtDqGjFMtXteV5bST8m0UZ+kTq95gy/2UgX6PTrBfrr836ML94/0ClVj/YcPrV67Vk+6oljKtIHtIqds/P9T6d6/WY6+dtRKfnrJiR3mO1QJuxkGvfL8N3pzIa94UgAABAK+S2aqipoQAAAPRTCAIAAJSMqaEAAAAJ5LZqaG2Yn7sbEQQAACgZhSAAAEDJmBoKAACQQG6rhpoaCgAAQD+FIAAAQMmYGgoAAJBAbquGThim2jMiCAAAUDIKQQAAgJIxNRQAACCB3FYNHS4WI4IAAAAloxAEAAAoGVNDAQAAEsht1dB6fehtRgQBAABKRiEIAABQMqaGjgPVaCTbV6PA3waqhduliyvlvnJV7D1W25KLXPNfpB8Wlet7ZHTq9VqyfdVqw8ynGaWUcVUSxtWsV5LtK1e5XtspP29Ta/X3inbsqwxyPI9lOINWDQUAACBbCkEAAICSMTUUAAAgAauGAgAAkC2FIAAAQMmYGgoAAJCAVUMBAADIlkIQAACgZEwNBQAASMCqoQAAAGRLIQgAAFAypoYCAAAkYNVQAAAAsqUQBAAAKBlTQwEAABKwaigAAADZUggCAACUjKmhAAAACXTSqqEKwXGgkXBgtxqNQq2KtUsn5XvMVatzSmdI2S9af68opkhc1YLtUmrWK8n2VakN8yONUarXa8n2VSSuZjOiUmuO2G6436GMlnv+h62KtEudr06+VzA20p1H5zAnzgYAAEDJGBEEAABIwKqhAAAAZEshCAAAUDKmhgIAACTQSauGGhEEAAAoGYUgAABAyZgaCgAAkIBVQwEAAMiWQhAAAKBkTA0FAABIwKqhAAAAZEshCAAAUDKmhgIAACRg1VAAAACypRAEAAAoGVNDAQAAErBqKAAAANnKdkSwGo0RW4zcJqKRsNYtcryiOjmuasF2Kcn9vuMVbZcurtRyzX+uWn2tFdX689j6e37KfTXr6fpqpTbML/9HqV6vJWtXa0NcreaeP3qde88vds8pi1TnMc9PtHLo6emJ+h+tHJNtIQgAANBJcl01dOXKlbFly5YB2xSCAAAA41h3d7cRQQAAgDKZM2fOoNcKF4KPP/543HrrrfH8889HtVqN4447Lq644oo4/vjj+9ucc8458eyzzw76t6eddlrcfPPNBxY1AABAB+ikVUMLFYJPP/10XHrppXHMMcfElVdeGX19fXHPPffEF77whbjnnnvik5/8ZDSbzdi4cWMsWrQoFi9ePODfz50796DeAAAAAOkUKgT/9m//NubMmRM/+clPYsqUKRERceaZZ8aSJUvipptuijvuuCNee+212LVrV3zmM5+J5cuXj2nQAAAAHLgRC8Ft27bF+vXr4+KLL+4vAiMiPvaxj8UJJ5wQv/jFLyIiYsOGDRERcfTRR49RqAAAAPnKddXQ/RmxEJw+fXo88sgjA4rAfbZu3Rq12t5n/bz00ksR8WEhuGvXrpg6deqBxAsAAMAYGvG5jrVaLY444oiYPXv2gNfXr18fzzzzTCxcuDAi9haC06ZNi+uvvz4WLlwYCxcujEWLFsXatWvHJnIAAAAOyAE9PqK3tze+/vWvR0TEl7/85YjYOzW0t7c3duzYETfeeGNs37497rzzzrjqqqvi/fffjzPPPHN0B6mOWKMWalNgL6OQbm+dHleR05OW3Pe3K9Sw5SeosFzzX/yQrb7vlEHBjHV07tNF1mwm21VhRe47zWZt7ANpO/f80eroe37rv+xkSyaKG3erhv6hd999Ny677LJYv359fOUrX4murq6IiFixYkU0Go1YuXJlf9ulS5fGsmXL4rvf/W6cccYZ/dNIC2k0ht9erY7cJiIaSW/aIx+vqE6Oq2Dqk5L7D45XMPcp40ot1/wXO2Dr7ztlUOg8dnjuU/bVSi1dJVivj/y5XPS+U6sN80OUUSoSVzu4549ex97z2/FlJ2OpzqPaOi+jOh3bt2+PVatWxVNPPRVnn312XHnllf3bLrjgggFFYETE5MmTY/ny5fH222/3LyYDAABAexUeEfzd734XX/rSl2LdunVx3nnnxV//9V9HpVIZ8d/NnDkzIvYuHgMAADBeddKqoYVGBHfu3NlfBF500UVx3XXXDSgC33jjjVi6dGnccsstg/7tpk2bIiJi3rx5owwbAACAsVCoELzuuuti3bp1ceGFF8Y3vvGNQdtnz54dO3bsiDVr1sTOnTv7X+/p6Yn7778/TjzxxJg1a1a6qAEAADhgI04N3bhxYzz44IMxY8aM+LM/+7N48MEHB7VZvnx5fOtb34rLL788zj///Dj33HOjt7c3uru7Y8KECfHtb397TIIHAADIxbhaNfTpp5+OiIgdO3bsdzQwYm8huGjRovjhD38Yt912W3zve9+LyZMnR1dXV1x11VX9D5kHAACg/SrNZjueSFSAx0cUluty1inJ/QfHs5T4AB4fMT54fMToeHxE+7jnj17H3vM9PmIAj48o7tprI955p91RfGjmzIi/+Zv9bzugB8p3klxvjrkWNimVIfcpFYurmm38RXV6/K1WhntFSrn2r5S5H24FuNEqUrw1m7VC7VIWbxMSfjvJaYrWH8r5np9r8ZZrXGWQLl/j/3No3K0aCgAAwPihEAQAACiZcT81FAAAoBU6adVQI4IAAAAloxAEAAAoGVNDAQAAErBqKAAAANlSCAIAAJSMqaEAAAAJWDUUAACAbCkEAQAASsbUUAAAgASsGgoAAEC2FIIAAAAlY2ooAABAAlYNBQAAIFsKQQAAgJIxNRQAACCBTlo1dNwXgo2Eg57VaCTbV65xFT1iq4/Z2fkqpsh7rBZul/Y95pr/XOPKtY/lmvsyyDX39XotWbsJCb9RpPwSVasN8y1olJr1SrJ9lUWu9+lc5XpvLUPuyyjP3gYAAMCYGfcjggAAAK1g1VAAAACypRAEAAAoGVNDAQAAEuikVUONCAIAAJSMQhAAAKBkTA0FAABIwKqhAAAAZMuIIAAAQAIWiwEAACBbCkEAAICSMTUUAAAgAYvFAAAAkC2FIAAAQMmYGgoAAJCAVUMBAADIlkIQAACgZMb91NBqNJLtq6Fubptcz2PKuIrtq5r0mEW145hF5BpXrn0s17hyJV+jk3IqVK02zFymUarXa8n2VU34ralIvqrRnu8eZeivuZL7zmfVUAAAALKlEAQAACiZcT81FAAAoBWsGgoAAEC2FIIAAAAlY2ooAABAAlYNBQAAIFsKQQAAgJIxNRQAACABq4YCAACQLYUgAABAyZgaCgAAkIBVQwEAAMiWQhAAAKBkTA0FAABIwKqhAAAAZEshCAAAUDKmhgIAACTQSauGZlsINkYYrKwWaLO3XSNRRPkqkoeUiuY+7THH/3lMKfX5KUP+i91PWt/3c5UyD7n2L3F9eMRWH7NeryXb14SE33RSfrmr1Yb54c4Hms1asXb1SoqQ+pXh+s5VrrlPFZdP0Lw4HwAAACWT7YggAABAJ7FqKAAAANlSCAIAAJSMqaEAAAAJdNKqoUYEAQAASkYhCAAAUDKmhgIAACRg1VAAAACypRAEAAAoGVNDAQAAErBqKAAAANlSCAIAAJSMqaEAAAAJWDUUAACAbCkEAQAASsbUUAAAgASsGgoAAEC2xv2IYCNhrVuNRrJ9dbZqR+ci19iL9NVq4XZp32PK6yillO+z2L6K9f2U+SpD7lPK9Z6f63ksoh33nZT76utLl/sJCb811eu1ZO2qib/NpRzNyLXvl+H6zvN+mGeuyqCnpyfqf7RyzLgvBAEAAFqh2WxEs9nuKD60N5ZqrFy5MrZs2TJgm0IQAABgHOvu7jYiCAAAUCZz5swZ9FrhibpPPvlkXHDBBbFw4cL4d//u38V3vvOd6O3tHdBm8+bN8dWvfjW6urqiq6srrr766njnnXcOPnIAAIDs1TP8b/8KjQj+8pe/jFWrVsWCBQvia1/7WvT09MSdd94Zzz33XHR3d0e1Wo2tW7fGF7/4xdizZ09ccsklUa/X4/bbb48XX3wx1qxZExMnTiyUOgAAAMZWoULwxhtvjDlz5sTdd98dkydPjoi9w4vXXXddPP7443HyySfHj370o3j99dfjoYceiqOPPjoiIo477ri4+OKL44EHHogVK1aM3bsAAACgsBGnhu7evTs++tGPxooVK/qLwIiIrq6uiIh48cUXIyJi7dq10dXV1V8ERkScdNJJceSRR8batWtTxw0AAJCZRrR/Kugf/jf0oz9GHBGcNGlS3H777YNeX7duXUREHHroobFt27bYvHlznHbaaYPaLViwIB577LGRDgMAAECLjHrV0C1btsRTTz0VN9xwQ8yfPz8++9nPxquvvhoREbNnzx7UftasWbFz587YsWNHzJgx4+AjBgAA4KCMqhD8/e9/H6eeempEREyZMiWuvfbamDRpUv/qoVOmTBn0byZNmhQREbt27VIIAgAA41g9IjJ6onxUhtwyqkKwUqnETTfdFHv27Im77rorLr744vj+978fs2bNGvHfVquFn1TxQfs0bdJq+QHz1frkj3tFM1os9WnPT75nuw2RFTgB+eYrpTLkPt3eOr1PtOO+k0rKqBpD/9RmzBTJfTvi6nzpvmjm2fNTK8e7LJtRFYIf+chHYsmSJRERcfrpp8eyZcvi7/7u7+Lv//7vI2LvwjJ/bN9r06ZNG1VgI93UqtXW3/iqw/zYslTakfwSaBS4yRZNfeq+WiS2dmj5NVnwBOSar5TKkPuU77GT+0S77juppMz9hFH/oGZofX0jtyma+5RxRRSLrdON2F9H8V2nk6/vopJd3wYSsnLAZ2Py5MlxyimnRE9PT3z84x+PiIi33nprULs333wzDjnkkJg6deqBRwkAAJC9Rob/7d+IheDGjRvj1FNPje7u7kHbent7o1KpxMSJE2PevHnx/PPPD2rzwgsvxLHHHjvSYQAAAGiREQvBww8/PHbs2BH33Xdf7Nmzp//1LVu2xKOPPhonnHBCTJ8+PRYvXhxPPvlkbNy4sb/NE088EZs2beqfTgoAAED7VZrN5ojL2jz44INx9dVXx/HHHx+f//znY+vWrdHd3R3vv/9+3HPPPTF//vx45513YtmyZVGr1WLVqlWxe/fuWL16dRx22GFx3333xcSJE0cVmN8IZsxvBMeE3wiOXhl+p5arMuTebwT38hvBD/mN4PjhN4Kj4zeCxR1xxPZ49dV87oeHH16NV145ZL/bChWCEREPP/xwrF69On7729/G1KlT41Of+lRceeWVceSRR/a3efnll+P666+PX/3qVzF58uQ4+eST4+qrr46ZM2eOOmiFYMYUgmNCITh6ZShGclWG3CsE91IIfkghOH4oBEdHIVjcuCwEW00hmDGF4JhQCI5eGYqRXJUh9wrBvRSCH1IIjh8KwdFRCBbXSYVg4lsHAABAWdVjuJU6W2/oMb9xXwiW4S+6rX6P1cLtxn/uaa9cR39y7fudPGLTjvtOGRTLV1VeP9DoS3ifmDByf240io32pR7Ba/XIZzuMdD8pes9JLdfPD8YnPQQAAKBkxv2IIAAAQGt0ztRQI4IAAAAloxAEAAAoGVNDAQAAkmjE3umh+TMiCAAAUDIKQQAAgJIxNRQAACCJRuS1amhlyC1GBAEAAEpGIQgAAFAypoYCAAAkUQ+rhgIAAJAlhSAAAEDJmBoKAACQRG4PlLdqKAAAAB9QCAIAAJSMqaEAAABJ5LZqqKmhAAAAfMCIYJtUo5FsX40S1PMp85WrMpzHiHz7fpF9VRMfs9VSxp7yPBbbV7VQu1zPT679PqVc48o193196dpNSPxtrmhsRaSMLWVcKeX6HSXXuMiHQhAAACAJU0MBAADIlEIQAACgZEwNBQAASKIZkdXvM5tDbjEiCAAAUDIKQQAAgJIxNRQAACCJ3FYNHXrcz4ggAABAySgEAQAASsbUUAAAgCRMDQUAACBTCkEAAICSMTUUAAAgiUbkNTW0NuQWI4IAAAAloxAEAAAoGVNDAQAAksht1dChYzEiCAAAUDIKQQAAgJIxNXQcqEYj2b4aCf82kHJfub7HlIq9x2rSXLRDGfqFuNKrFjxmru+xk89jO3Kfq1z7V19fsl1FRMSEhN8OU8ZWq6WbbtesV0Zo0Z7P206+VxSR5zew1Bof/JeLoWMpx/kAAACgnxFBAACAJHJ7jqARQQAAAD6gEAQAACgZU0MBAACS8BxBAAAAMqUQBAAAKBlTQwEAAJKwaigAAACZUggCAACUjKmhAAAASVg1FAAAgEwpBAEAAErG1FAAAIAkGjHcSp2tZ9VQAAAAPqAQBAAAKBlTQwEAAJLwQHkAAAAyle2IYHXEH1lWC7SJaKh1R6VITovmvgxyzUPqfp/r+8xVrvedlHG1vk+0/p6fa7/Xv8aHlJ+3qftEX1+6fdVq6UZG6vVasn1VRoir2Yyo1JqF9pUyrjL0ffKRbSEIAADQWTxQHgAAgEwpBAEAAErG1FAAAIAkTA0FAAAgUwpBAACAkjE1FAAAIIlmDPcQ99Yb+jEoRgQBAABKRiEIAABQMqaGAgAAJGHVUAAAADKlEAQAACgZU0MBAACSMDUUAACATCkEAQAASsbUUAAAgCQakdfU0KEfbm9EEAAAoGSMCAIAAIxjPT09Ua8PHKkc94VgdZjhUDpHI+Hgdco+kTKuIqoFj6nft1eu+U/ZX3Pt+ym1+njtUKyvVrPt0+SvWa8k21ellm66Xb1eS9ImIqKWMK6U+Uop3T1g/N9Xc101dOXKlbFly5YBW8Z9IQgAAFBm3d3d5RsRBAAAKLM5c+YMek0hCAAAkEQjhlups/WGjqVwIfjkk0/GzTffHOvXr4/p06fH6aefHldccUVMmzatv80555wTzz777KB/e9ppp8XNN988yqABAAAYC4UKwV/+8pexatWqWLBgQXzta1+Lnp6euPPOO+O5556L7u7uqFar0Ww2Y+PGjbFo0aJYvHjxgH8/d+7cMQkeAACA0StUCN54440xZ86cuPvuu2Py5MkRsXee6XXXXRePP/54nHzyyfHaa6/Frl274jOf+UwsX758TIMGAADIzzh6oPzu3bvjox/9aKxYsaK/CIyI6OrqioiIF198MSIiNmzYEBERRx999EGFCgAAwNgacURw0qRJcfvttw96fd26dRERceihh0ZExEsvvRQRHxaCu3btiqlTpyYLFAAAgDRGvWroli1b4qmnnoobbrgh5s+fH5/97GcjYm8hOG3atLj++uvj4Ycfjl27dsUnPvGJuPLKK2Pp0qXJAwcAAMhLng+U359RFYK///3v49RTT42IiClTpsS1114bkyZNioi9U0N7e3tjx44dceONN8b27dvjzjvvjKuuuiref//9OPPMM0cXc3XEWavF2jA2Wpz7tEdLt7d29MBiqXdtjFbRjHVy/vOMqji3/LFQMKmS3z4Fcp/32UkXXbOZbFeFFO32zWZtbAOBMVJpNotfVtu2bYtf/OIXsWfPnrjrrrti3bp18f3vfz9OP/30uPfee6PRaMTKlSv727/33nuxbNmyePfdd+PnP/951GqjuFAaIzx/o1oduQ1jow25byQt3tLFnjKuIoqmPuV7LIsi57LT89/q/pqSW/7YKNRXJb99CuY+52s75f2wUktXCdbrw38nHU23r9XSjf4065Vk+8pSCf6odMQR98Wrr+5sdxj9Dj98erzyyvn73TaqQvAP7Svy+vr64rHHHhuy3Q9+8IO45ZZb4r/9t/8Wf/Inf1L8AArBfCkE+ykExw+FYN7c8seGQjBzCsEBFILjQCkKwe4MC8GV+912wGdj8uTJccopp0RPT0+88847Q7abOXNmROxdPAYAAID2G7EQ3LhxY5x66qnR3d09aFtvb29UKpV49913Y+nSpXHLLbcMarNp06aIiJg3b16CcAEAADhYIxaChx9+eOzYsSPuu+++2LNnT//rW7ZsiUcffTROOOGEmDt3buzYsSPWrFkTO3d+OBTa09MT999/f5x44okxa9assXkHAAAAWdj3QPlc/ht6jvOIq4ZOmDAhrr322rj66qvjL/7iL+Lzn/98bN26Nbq7u6NSqcRf/uVfRkTEt771rbj88svj/PPPj3PPPTd6e3uju7s7JkyYEN/+9reL5Q0AAIAxV3ixmIcffjhWr14dv/3tb2Pq1KnxqU99Kq688so48sgj+9v87Gc/i9tuuy3Wr18fkydPjq6urrjqqqv6HzI/KhaLyZfFYvpZLGb8sFhM3tzyx4bFYjJnsZgBLBYzDpRisZi74tVXd7Q7jH6HHz4jXnnlL/a77YBXDR1zCsF8KQT7KQTHD4Vg3tzyx4ZCMHMKwQEUguNAKQrBf8qwEPzifreN6oHy5CnXIimlXOMqi1y/ZOgX7dP63Fdbfsxc+z3jQ6E/PhVuV4574UjF22iMVLw1m7XCBV7KuCoJi8qUcZWlj5WNTzkAAICSMSIIAACQxL7VOnMxdCxGBAEAAEpGIQgAAFAypoYCAAAkse+B8rkYeqEfI4IAAAAloxAEAAAoGVNDAQAAkrBqKAAAAJlSCAIAAJSMqaEAAABJWDUUAACATCkEAQAASsbUUAAAgCSsGgoAAECmFIIAAAAlY2ooAABAEo0YbqXO1rNqKAAAAB8wIjgKjYR1czXhXwpS7qvIe6wWbJdSyveYUqtzz9gpdi6rhdqV4V5RBvLVPrleQ+7To5fruWzWK0naRERUaukWBqnXa8n2VcswLldQXhSCAAAASXigPAAAAJlSCAIAAJSMqaEAAABJeKA8AAAAmVIIAgAAlIypoQAAAEmYGgoAAECmFIIAAAAlY2ooAABAEs0Y7iHurdcccosRQQAAgJJRCAIAAJSMqaEAAABJWDUUAACATCkEAQAASsbUUAAAgCRMDQUAACBTCkEAAICSyXZqaGOEGrVaoE1q1aweDjk2ir3HaqF2rT4/RaWMK2WfyDn3Zej7KaXMV679tfX332LHzPU9liGuMsj1POZ8zy9DH6vXa8n2Vaulm1KYW1yHHx7xyivpYspXI/KaGjr0NZjnN3UAAADGjEIQAACgZLKdGgoAANBZrBoKAABAphSCAAAAJWNqKAAAQBKNGG6lztazaigAAAAfUAgCAACUjKmhAAAASXigPAAAAJlSCAIAAJSMqaEAAABJeKA8AAAAmVIIAgAAlIypoQAAAEmYGgoAAECmjAgCAAAk4TmCAAAAZCrbEcGjjqrHq68Ovb3ZrEWtNnK1Xa/XEkY1/jUK/G2gWrBdSq0+XlEp46oO8xebdivL+8yRfLVPytzneg2lvOenjCvXft/681jNNhdF5fr5nWt/Tfm9tcj35KKa9UqyfZGPbAtBAACAzmKxGAAAADKlEAQAACgZU0MBAACSaMRwK3W2nlVDAQAA+IBCEAAAoGRMDQUAAEjCA+UBAADIlEIQAACgZEwNBQAASMID5QEAAMiUQhAAAKBkTA0FAABIwqqhAAAAZEohCAAAUDKmhgIAACTRiOGmY7aeqaEAAAB8INsRwXnzRm5z+OFjHwcAAHQa35PbY968/6fdIQywL56enp6o1wcuYlNpNpvNdgQFAADA2Hrvvffi05/+dGzbtm3A6wpBAACAcWr79u2xffv2Qa8rBAEAAErGYjEAAAAloxAEAAAoGYUgAABAySgEAQAASkYhCAAAUDIKQQAAgJJRCAIAAJSMQhAAAKBkJrQ7gNHavHlz3HDDDfH0009HRMQpp5wS11xzTcycObPNkY1/55xzTjz77LODXj/ttNPi5ptvbkNE49+1114br776atx1110DXncdtMZQ+XctpPf444/HrbfeGs8//3xUq9U47rjj4oorrojjjz++v41+PzaK5F6fHztPPvlk3HzzzbF+/fqYPn16nH766XHFFVfEtGnT+tvo+2OjSO71fcazjioEt27dGl/84hdjz549cckll0S9Xo/bb789XnzxxVizZk1MnDix3SGOW81mMzZu3BiLFi2KxYsXD9g2d+7cNkU1vq1ZsybWrFkTXV1dA153HbTGUPl3LaT39NNPx6WXXhrHHHNMXHnlldHX1xf33HNPfOELX4h77rknPvnJT+r3Y6RI7vX5sfPLX/4yVq1aFQsWLIivfe1r0dPTE3feeWc899xz0d3dHdVqVd8fI0Vyr+8z7jU7yPe///3mn/3ZnzU3bNjQ/9ovfvGL5vz585s//vGP2xjZ+Pd//+//bc6fP7/5X//rf213KONeX19f8wc/+EHzT/7kT5rz589vfuELXxiw3XUwtkbKv2shveXLlzdPOeWU5q5du/pfe+utt5onnHBC86KLLmo2m/r9WCmSe31+7Jx11lnNf//v/33z3Xff7X/t7rvvbs6fP7/52GOPNZtNfX+sFMm9vs9411G/EVy7dm10dXXF0Ucf3f/aSSedFEceeWSsXbu2jZGNfxs2bIiIGJB70tu9e3ecddZZ8YMf/CCWL18es2fPHtTGdTB2iuTftZDWtm3bYv369XH66afHlClT+l//2Mc+FieccEL85je/iQj9fiwUzb0+PzZ2794dH/3oR2PFihUxefLk/tf3zUJ48cUXI0LfHwtFc6/vM951zNTQbdu2xebNm+O0004btG3BggXx2GOPtT6oEnnppZci4sOb4a5du2Lq1KntDGlc2r17d+zcuTNuuummWLJkSZx66qkDtrsOxtZI+Y9wLaQ2ffr0eOSRRwYUIvts3bo1arWafj9GiuQ+Qp8fK5MmTYrbb7990Ovr1q2LiIhDDz1U3x8jRXIfoe8z/nXMiOAbb7wREbHfv9DPmjUrdu7cGTt27Gh1WKXx0ksvxbRp0+L666+PhQsXxsKFC2PRokX+GpnY9OnT49FHH40lS5bsd7vrYGyNlP8I10JqtVotjjjiiEF9ev369fHMM8/EwoUL9fsxUiT3Efp8q2zZsiXuv//++M53vhPz58+Pz372s/p+i+wv9xH6PuNfx4wI9vb2RkTs9y+XkyZNioi9f6mZMWNGS+Mqiw0bNkRvb2/s2LEjbrzxxti+fXvceeedcdVVV8X7778fZ555ZrtDHBeq1WpUq0P/fcZ1MLZGyn+Ea6EVent74+tf/3pERHz5y1/W71voj3Mfoc+3wu9///v+GQhTpkyJa6+9NiZNmqTvt8BQuY/Q9xn/OqYQbDQaI7YZ6QscB27FihXRaDRi5cqV/a8tXbo0li1bFt/97nfjjDPO6J9GxNhxHbSfa2Fsvfvuu3HZZZfF+vXr4ytf+Up0dXXFr3/96xH/nX5/8PaX+wh9vhUqlUrcdNNNsWfPnrjrrrvi4osvju9///sxa9asEf+tvn9whsr96aefru8z7nXM3WPfM1127949aNu+1/7wuS+kdcEFFwy4EUZETJ48OZYvXx5vv/12/w+qGVuug/ZzLYyd7du3x6pVq+Kpp56Ks88+O6688sqI0O9bYajcR+jzrfCRj3wklixZEmeeeWZ0d3fHoYceGn/3d3+n77fAULmP0PcZ/zqmENz3w9233npr0LY333wzDjnkED/gbYN9D7PdtWtXmyMpB9dBvlwLB+d3v/tdXHjhhfHMM8/EeeedF9/5zneiUqlEhH4/1obL/XD0+bExefLkOOWUU6Knpyc+/vGPR4S+3yp/mPt33nlnyHb6PuNFxxSChxxySMybNy+ef/75QdteeOGFOPbYY9sQVTm88cYbsXTp0rjlllsGbdu0aVNERMybN6/VYZWS66C9XAtjY+fOnfGlL30p1q1bFxdddFFcd911AwoR/X7sjJR7fX7sbNy4MU499dTo7u4etK23tzcqlUpMnDhR3x8DRXL/7rvv6vuMex1TCEZELF68OJ588snYuHFj/2tPPPFEbNq0adhV/jg4s2fPjh07dsSaNWti586d/a/39PTE/fffHyeeeGKh3zGQhuugfVwLY+O6666LdevWxYUXXhjf+MY39ttGvx8bI+Venx87hx9+eOzYsSPuu+++2LNnT//rW7ZsiUcffTROOOGEmD59ur4/Borkfu7cufo+416l2Ww22x1EUe+8804sW7YsarVarFq1Knbv3h2rV6+Oww47LO67776YOHFiu0Mct372s5/F5ZdfHsccc0yce+650dvbG93d3fH+++/Hvffe62GrY+TUU0+NuXPnxl133dX/muugdfaXf9dCWhs3bowlS5bEjBkz4pvf/OZ+F15Yvny5fj8GiuZenx87Dz74YFx99dVx/PHHx+c///nYunVrf27vueeemD9/vr4/RorkXt9nvOuoQjAi4uWXX47rr78+fvWrX8XkyZPj5JNPjquvvrp/vjZj52c/+1ncdtttsX79+pg8eXJ0dXXFVVdd5UY4hvZXiES4DlplqPy7FtK5995746/+6q+GbfPiiy9GhH6f2mhyr8+PnYcffjhWr14dv/3tb2Pq1KnxqU99Kq688so48sgj+9vo+2OjSO71fcazjisEAQAAODgd9RtBAAAADp5CEAAAoGQUggAAACWjEAQAACgZhSAAAEDJKAQBAABKRiEIAABQMgpBAACAklEIAgAAlMz/H48PFhamBTDjAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_148_1.png" } }, "output_type": "display_data" } ], "source": [ "fig = plt.figure(figsize=(20, 14))\n", "im = plt.imshow(J, **cmap_args)\n", "plt.title(\"OLS\", fontsize=18)\n", "plt.xticks(fontsize=18)\n", "plt.yticks(fontsize=18)\n", "cb = fig.colorbar(im)\n", "cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is interesting to note that OLS\n", "considers both $J_{j, j + 1} = -0.5$ and $J_{j, j - 1} = -0.5$ as\n", "valid matrix elements for $J$.\n", "In our discussion below on hyperparameters and Ridge and Lasso regression we will see that\n", "this problem can be removed, partly and only with Lasso regression. \n", "\n", "In this case our matrix inversion was actually possible. The obvious question now is what is the mathematics behind the SVD?\n", "\n", "\n", "\n", "\n", "\n", "Let us now \n", "focus on Ridge and Lasso regression as well. We repeat some of the\n", "basic parts of the Ising model and the setup of the training and test\n", "data. The one-dimensional Ising model with nearest neighbor\n", "interaction, no external field and a constant coupling constant $J$ is\n", "given by" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " H = -J \\sum_{k}^L s_k s_{k + 1},\n", "\\label{_auto7} \\tag{7}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $s_i \\in \\{-1, 1\\}$ and $s_{N + 1} = s_1$. The number of spins in the system is determined by $L$. For the one-dimensional system there is no phase transition.\n", "\n", "We will look at a system of $L = 40$ spins with a coupling constant of $J = 1$. To get enough training data we will generate 10000 states with their respective energies." ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from mpl_toolkits.axes_grid1 import make_axes_locatable\n", "import seaborn as sns\n", "import scipy.linalg as scl\n", "from sklearn.model_selection import train_test_split\n", "import sklearn.linear_model as skl\n", "import tqdm\n", "sns.set(color_codes=True)\n", "cmap_args=dict(vmin=-1., vmax=1., cmap='seismic')\n", "\n", "L = 40\n", "n = int(1e4)\n", "\n", "spins = np.random.choice([-1, 1], size=(n, L))\n", "J = 1.0\n", "\n", "energies = np.zeros(n)\n", "\n", "for i in range(n):\n", " energies[i] = - J * np.dot(spins[i], np.roll(spins[i], 1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A more general form for the one-dimensional Ising model is" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " H = - \\sum_j^L \\sum_k^L s_j s_k J_{jk}.\n", "\\label{_auto8} \\tag{8}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we allow for interactions beyond the nearest neighbors and a more\n", "adaptive coupling matrix. This latter expression can be formulated as\n", "a matrix-product on the form" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " H = X J,\n", "\\label{_auto9} \\tag{9}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where $X_{jk} = s_j s_k$ and $J$ is the matrix consisting of the\n", "elements $-J_{jk}$. This form of writing the energy fits perfectly\n", "with the form utilized in linear regression, viz." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " \\boldsymbol{y} = \\boldsymbol{X}\\boldsymbol{\\beta} + \\boldsymbol{\\epsilon}.\n", "\\label{_auto10} \\tag{10}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We organize the data as we did above" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "X = np.zeros((n, L ** 2))\n", "for i in range(n):\n", " X[i] = np.outer(spins[i], spins[i]).ravel()\n", "y = energies\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.96)\n", "\n", "X_train_own = np.concatenate(\n", " (np.ones(len(X_train))[:, np.newaxis], X_train),\n", " axis=1\n", ")\n", "\n", "X_test_own = np.concatenate(\n", " (np.ones(len(X_test))[:, np.newaxis], X_test),\n", " axis=1\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will do all fitting with **Scikit-Learn**," ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "clf = skl.LinearRegression().fit(X_train, y_train)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When extracting the $J$-matrix we make sure to remove the intercept" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "J_sk = clf.coef_.reshape(L, L)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And then we plot the results" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/3777801602.py:7: UserWarning: FixedFormatter should only be used together with FixedLocator\n", " cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAM2CAYAAACjUj0CAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAB7UElEQVR4nO3de3xU1bn/8W+GECZhGGMaSAhpCAEjAuXiwWCpRYsRFNRgq3hBUPHSerAesR6qHo62HCtgPdoi/qwtVguiVo5U2+KvtdZa7/KzHoqIQbkECCGBGEIyhBCTmd8fkEjMZW+cJ8kO83m/Xrx8OfvJM8/s2ZdZs9asFReJRCICAAAAAMQMX1cXAAAAAADoXDQEAQAAACDG0BAEAAAAgBhDQxAAAAAAYgwNQQAAAACIMTQEAQAAACDG0BAE0GFuv/12nXzyySouLnYV51WrV6/WySef3OLf8OHD9fWvf1033HCDPvjgg64us1O8++67Ovnkk7V69epOfd5du3Zp5syZGjlypMaNG6eKiopOfX63PvjgA9188836xje+oREjRuiMM87Q3Llzv9TxUVxcrJNPPlkPPfRQ02Mnn3yybr/99nb/buLEiZo5c2azxz799FPV1NQ4PqfXz0UAgJ34ri4AAC699FJ9/etf7+oyHF166aX6l3/5l6b/r6ur06ZNm/TMM8/o//2//6cXXnhBWVlZXVhhxxs8eLDuu+8+nXrqqZ36vIsXL9Z7772nm266SX379lVKSkqnPr8br732mr73ve9pyJAhmjVrllJSUrR7924999xz+tOf/qSHHnpI+fn5rvOlpKTovvvuO+aG2Z133qnExMSm///73/+u2267Tb/73e+UlJR0TLkAAMcvGoIAutyYMWM0ZsyYri7D0ejRo1VQUNDi8VNPPVVz587Vr3/9a/3oRz/q/MI6UWpqaqv7oKNt2rRJp5xyiubMmdPpz+3WT37yEw0dOlS//e1v1bNnz6bHZ82apYKCAi1YsEBnnXWW4uPd3XqTkpK+1L7+YmNz/fr1qqqqOuY8AIDjG0NDASBK5513nhITE/XPf/6zq0s5bn322Wfq3bt3V5fRpoqKChUVFWncuHHNGoGSlJycrGnTpqm8vNxxmDQAAJ2FhiCALvfF3yXdfvvtOvfcc7V+/XpdeeWVGjVqlMaPH6977rlHtbW1zf62tLRU8+bN0+mnn66vfe1rmjZtmn7/+9+3eI63335b1113ncaNG6fhw4frm9/8pu66665mPSWNz7ty5UqddtppOu200/Taa6851h8XF6devXopEok0e3zz5s2aM2eOxo4dq1GjRumyyy7T66+/3uLv//nPf2rWrFkaM2aMvvnNb+qhhx7S0qVLW90nrdXmZh9EIhEtXbpUkydP1te+9jWNHz9e//7v/67du3c3i3v66ad1wQUXaNSoURo3bpzmzJmjTz75pGl7a78RbGho0LJlyzR58uSm38XdfffdzX7H1/h3b775pn784x/r61//ukaNGqWrrrpKhYWFbe7bxr/btWuX1q5d2/SbucbHf/e73+mCCy7Q1772Nd1xxx3HXM9bb72l//iP/9Bpp52mf/mXf9Edd9yhmpoa/f3vf1dBQYFGjRqlgoICvf32223WKEmJiYnq0aOH/vrXv2rv3r0ttn//+9/Xhx9+qOzs7GbvyfLly3X++edr5MiRmjhxou6//34dPHhQUuu/EfyiLVu2aNy4cZo8ebLKy8slNf+N4O23366lS5dKks4+++wWvx10ozPOsWM55wEANhgaCsCTKioqdO211+q8887ThRdeqNdee00rVqxQQkKC5s2bJ0kqKyvTJZdcokgkopkzZ+qEE07QX//6V/37v/+79uzZo+uuu06S9MYbb+j666/XqaeeqptvvllxcXF688039dvf/lafffaZFi5c2PS8u3fv1sMPP6ybbrpJe/bs0ahRo/TXv/613Vo/+OADVVZWauLEiU2Pbdq0SVdccYVSU1P13e9+Vz179tQf//hH3XDDDfrv//5vTZkyRZK0YcMGzZo1S6mpqZozZ44OHjyo5cuXy+dr+T1da7W53Qe/+MUv9PDDD2vGjBlNE/gsX75cGzZs0B//+Ef16NFDv//97/WjH/1I06ZN08yZM1VRUaHf/OY3mjlzpv7yl7+oT58+rb7+uXPn6s9//rMmTZqkWbNmadu2bXr66af1zjvvaNWqVQoGg02x8+fPV79+/fSv//qv2r9/v5YtW6brr79ef/vb31odMtn4m8SFCxfqxBNP1Pe+9z2dfPLJ2r9/vyRpwYIFKigo0CWXXKKMjIxjruf222/XkCFD9IMf/EBr167V6tWrVVpaqo0bN2rmzJnq06ePfvnLX+rf/u3f9PLLLzf726MlJiZqypQp+sMf/qD8/HxNnDhRZ5xxhk4//XQNGDCg1df24x//WE8//bS+9a1v6fLLL9e2bdv061//WkVFRU2Nt/aUlJRo9uzZCgQC+s1vfqPU1NQWMZdeeqlCoZD+8pe/6I477tBJJ53kmPdonXWOvfjii67OeQCAoQgAdJAf/vCHkdzc3MjOnTtdxX3x/5cvX94s7rzzzoucccYZzeLy8vIiZWVlzeJuvfXWyIgRIyLl5eWRSCQSufbaayPf+ta3IocOHWoWN3369MiYMWNaPO9zzz3XLO65556L5ObmRlasWBH59NNPm/6VlJRE/vKXv0Ty8/MjI0eOjGzdurXpb6688spIfn5+5MCBA02PffbZZ5ErrrgiMn78+KZaZs2aFRk7dmzk008/bYr78MMPI0OHDm11n3yxNrf74LzzzovccMMNzWKefvrpyIUXXhjZvn17JBKJRK677rrI1KlTm8W8+uqrkSlTpkTee++9SCQSibzzzjvN6vj73/8eyc3Njdxzzz3N/u7FF1+M5ObmRu67775mf/ed73wnUl9f3xT36KOPRnJzcyNvvPFGpD3f+ta3IldeeWXT/zfmO/qxL1tPQ0NDJBKJRBoaGiLf+MY3Irm5uZG///3vTX/77LPPuqoxFApFbrnllkhubm6zf1OnTo08+eSTTc8TiUQin3zySeTkk0+OzJ8/v1mOBx54IJKbmxv55JNPIjt37ozk5uZGlixZ0rQ9Nzc38sMf/jDy6aefRiZNmhSZMGFCi/Pri/tqyZIlrs7DSKT1c7EzzjG35zwAwA5DQwF41nnnndfs/4cOHapPP/1UkhQOh/Xyyy9r7Nixio+PV0VFRdO/SZMmqa6uTm+++aYk6dFHH9Vzzz2nhISEplz79u1TIBBodUr9M844o9V6/uu//ktf//rXm/6dddZZ+v73v69+/frpmWee0aBBg5pyr127VmeeeaZqa2ub6qqqqtI555yj8vJyffDBB9q/f7/Wrl2rgoKCZrNgDhs2TN/4xjdareHo2o5lH6Snp+vdd9/Vb37zm6YhhJdddlmzmU7T09O1detWLV26tOm3bGeeeabWrFnTbLbUo73yyiuSpO9+97vNHj/vvPM0aNAgvfzyy80enzRpknr06NH0/6eccooktTqc0o0vvlfHWs/ZZ5/d1Pvq8/n01a9+VX6/XxMmTGiKyczMdFVj79699eCDD+rFF1/U97//fY0ZM0bx8fH65JNPtGDBAv3rv/6rGhoaJEmvvvpqUy/b0a699lr9/ve/b3f22QMHDui6667Trl279MQTTzTVZ60rzrH2znkAgC2GhgLwrC8uEZCQkND0QXrfvn2qrq7Wyy+/3OLDfaPG37/16NFDO3fu1M9//nNt3rxZO3bsUFlZWZvP+5WvfKXVx6+99lqdccYZCofDKiws1C9/+UsNHDhQP/3pT5uGJUrSzp07JUkrVqzQihUr2qytV69eCofDGjhwYIvtOTk5rf6e8OjajmUfzJs3TzfeeKPuvfdeLVy4UMOHD9fEiRM1ffp09e3bV5I0Z84crVu3Tg899JAeeughDRkyRBMnTtQll1zSZsOkuLhYwWCw1WGJgwcPbvEby9beU+lwo+PL+GK+Y63ni3Hx8fEtcjY2FN3WOHjwYN1000266aabVF1drT/96U/6+c9/rr/97W/685//rClTpmjXrl2S1Ow3g5IUDAbbHH7a6KWXXpLP51M4HNaGDRuavoBwq66urmlobSO/399i6G9XnGPtnfMAAFs0BAF4Vmu/k2vU+OFw8uTJuuyyy1qN+epXvypJeuaZZ3T33Xdr0KBBGjt2rCZNmqRRo0ZpxYoV+sMf/tDi747usTrakCFDNH78eEmHezTGjRunK664QjNnztSqVauaPsQ21jZjxow2140bMmSISkpKJKlZL0qjXr16tfp3R9d2LPtg6NCh+vOf/6zXX39df/vb3/T6669ryZIleuKJJ/TMM89o8ODBSk9P1wsvvKB3331Xf/3rX/X666/rl7/8pR5//HH9+te/Vl5eXov8kS9MkHO0cDjcYgbN9t7TL+OL79Wx1tPaex0XF3fMdbz66qt688039e///u/N3s8+ffrokksuUW5urqZPn65//OMfmjJlSlSNmz59+ugXv/iF7rzzTi1atEgTJkzQCSec4Prv//d//1ezZs1q9thFF12kRYsWNXusK84x6+MDANA2GoIAuqWUlBQlJiaqvr6+qXHWqKSkRBs3blRiYqIOHTqkRYsWady4cfr1r3/dbNKOn//851HV8LWvfU0/+MEPtHDhQs2fP1//5//8H0nSgAEDJB3+sPvF2jZv3qzi4mIlJiY2fYguKipqkXv79u2Oz+92HzQ0NKiwsFCBQEBnn322zj77bEnSiy++qLlz52rVqlW6/fbbtWnTJklqGvoqSf/4xz901VVXacWKFa02BAcMGKA33nhD5eXlLXrXtm3bpv79+zu+DktdVc+HH36o5cuX65xzzml1PzVO0uL3+yWpqQd5586dGjx4cFNcWVmZFi5cqCuvvFLp6emtPld+fr7Gjh2r+fPn6/rrr9d///d/a8GCBa5rHTp0qB5//PFmj/Xr169FnBfOMQBAx+GrNwDdUnx8vCZMmKC///3vLZYfWLRokebMmaN9+/aptrZWBw8eVHZ2drMPqB999JHWrl0rSaqvr//SdVx11VU69dRT9de//lUvvviipMMfqkeMGKHf/e53zYbHffbZZ7rzzjt18803q76+Xl/5ylc0ZswY/fGPf2w2VG/nzp2ulq1wuw8aGho0a9Ys3Xvvvc1iRo0aJenzXph/+7d/07x585r1Vg0bNkw9e/Zss6emcabURx99tNnjL7/8srZt26azzjrL8XVY6qp6pk6dKp/Pp8WLF7e6ePtvf/tbSWpqhJ955pmSDi/XcbTVq1fr//7f/6tAIOD4nBMmTNA555yjZ599Vu+//36bcY3vXWNv6QknnKDx48c3+zdkyJAWf+eVcwwA0DHoEQTQ4R588MFWFwM/77zzmnqevozbbrtN7777rmbMmKEZM2YoIyNDr776qv72t7/p0ksvbeqFGTVqlFavXq1AIKBBgwbpk08+0apVq5o+IB84cOCYhtYdLS4uTgsWLNBFF12kn/zkJ/rGN76hE044QfPnz9dVV12l73znO7r88suVnJysNWvW6J///Kd+8IMf6MQTT5Qk/fCHP9TMmTN18cUX67LLLlNdXZ1WrFjR7hDHL7MPZs6cqUceeURz5szRN7/5TdXW1uq3v/2tEhMT9Z3vfEfS4d9Azp8/X1dffbXOPfdcRSIRvfDCCzp06JCuuOKKVp//zDPP1Nlnn63ly5errKxM48aNU1FRkZ5++ml99atfbTFpS0frqnqys7N1xx136N57721a/iAnJ0e1tbV688039be//U0zZ87UqaeeKunwJDmXXHKJVqxYoT179ujrX/+6Nm/erGeeeUbTpk3T0KFDXS0+f+edd+qNN97Q3XffrdWrV7cY+ip9/ru7ZcuWacKECU2NUTe8cI4BADoGDUEAHe6Pf/xjq4/n5ORE1RDMysrSs88+qyVLlujZZ59VTU2NvvrVr+qOO+5oNhvjz3/+cy1cuFDPPfec6urqNGDAAN1www0aPHiwvv/97+udd97R5MmTv3QdJ510kq699lr94he/0KJFi7Rw4UKNGTNGTz/9tB566CE9/vjjqq+v16BBg7Ro0SJddNFFTX87ZswYLVu2TA8++KB+9rOfKTk5WTNnztSWLVv05z//2Wwf3HzzzUpOTtZzzz2nxYsXq0ePHjr11FP105/+tGlo4iWXXKKePXtq+fLleuCBBxQOhzVixAj96le/0rhx41p9/ri4OP385z/Xr371Kz3//PN65ZVX9JWvfEWXXnqpvv/97ztOfGKtK+uZNWuWhg0bppUrVzati+f3+zV06FA98MADmjp1arP4BQsWKDs7W6tWrdIrr7yijIwMzZkzp2ltPjcyMjJ044036oEHHtDjjz+uG264oUXM1KlT9dJLL2n16tVau3btMTUEvXKOAQDsxUXcfu0MADC3d+/eplk7j/a9731PhYWFevXVVzu/KAAAcNzjN4IA0IWmT5+ua6+9ttlj5eXlevfddzVy5MguqgoAABzvGBoKAF3owgsv1C9+8Qv94Ac/0Lhx41RVVaVnn31W4XBYc+bM6eryAADAcYqhoQDQhcLhsFauXKlnn31WO3fuVK9evXTqqafq3/7t3zR06NCuLg8AABynaAgCAAAAwHGqqqqq1aWNaAgCAAAAQJQOVlQo8ciSPV5SW1urCRMmNFuzWPJyQ7C2VmqvtMRE6eBBxzT/dX+iWUn/eWO5Wa69kVSzXOGwWSpVVDjHnHKK9NFHLuIG10VfUKN4u5+zlu21myMprX6XWS4dtYh3m7KypB07HMO21mcZFPS5nF52r7Oy9wCzXMlBu4P/tTecj4sJEyQX67zr5JMNCjrC8hirDtrt+z4b3zXLVZbd+tIUR0tLk8rKnHOlBZ3vC25V19vdP1pZRvNL861fZ5br4MmjHWNc3m7Vq1f09TTyye7c3rXb7prf2Z/v3O77xArDe5Gkur5214qEfS5OXJcqe6WZ5UouWtd+wOjR0jqHmEaWE3vt3m2WyvJ9/J//iT5H795SQUH0ebzu12ecoSoX68B2lmBmpma/8YZ2796thi981vTuZDGRSPsNwcYYB/v2GdUjmba4wobNbzftB7c++8w2zoss95dpsvp6szi3qVyLt3udll9cWKqttYvz6jFm+rXfoUNmqdy+RFdxhi/So1+TSnV2X7K5fY2e3RcuWJ6PXbEfXD2n6UXHmGFtpvcPN+eR4bnmmkffy1CoqyvoPqqKi7V/+/auLqOF/v37t3jMuw1BAAAAAOhGfPLW+nzt1WJe586dO3XTTTcpLy9PeXl5mjdvnircjDcEAAAAAHQK0x7Bffv26aqrrlJdXZ2uu+46NTQ06LHHHtOmTZu0atUqJSQkWD4dAAAAAOBLMG0IPvHEEyotLdUf/vAHDR48WJI0atQoXXPNNXr++ec1ffp0y6cDAAAAAM+I2aGha9asUV5eXlMjUJLGjx+vQYMGac2aNZZPBQAAAAD4kswagvv379fOnTs1fPjwFtuGDx+uDRs2WD0VAAAAACAKZkNDy44s8JSW1nKNl759+yoUCqm6ulp9+vSxekoAAAAA8IzuNDTUrCF44MABSVJiYssFeHsdWWW2pqbGfUOwlTwtJCU5hjzwgLunc6efWSa7JVFtZWS4i3O3dqo3Jwdy+xrdsV243ZWcHMeQXPMntXudtusx211qJ02yjbNjt++DZpkkTZhglsrtKenu3HW+L7hlur8s5eWZpXK7t1zcbo3ZndtZXXCZtuRq3xu/SNO7t+FN1/T+4eY8MjzXXDN8Ly3fxxtuMEwGzzBrCIZdrPLp8x3Dhf3gwfZXUU1KkmpqHNPcOt/u7vXA7XvMcpVF7BqVlmuPlpc7x4wcKa1f7yJuqOFCrPF28xqVlNp9wMio32GWy9Uq8Dk50tatjmEf1zs3Fo9Frt/udVYEDBuVyXarC7/0svNxMWmS9NJLzrlGjDAo6AjLY6wq2bBRue41s1wlQ5wblRkZUkmJc66MZOf7gltV9Xb3j0DALJV87601y1UzwvmDrsvbrfx+g4KO8Mnu3N5RbHfNT001S+WK232fVG54L5JUl27YGCl3ceK6VOE3bFRudjiP8vKktS7PtbFjoy+oUXGxWSrL9/GJJ6LPEQhIV1wRfR7YMft03bt3b0nSoUOHWmxrfKwxBgAAAACON3Hy1tDQuHa2mdWZcaTrf+/evS227dmzR8FgUEmdP7YEAAAAAPAFZg3BYDCozMxMffjhhy22bdy4USMsx0oBAAAAAL40057LSZMm6e2339aWLVuaHnvrrbe0bds2TZkyxfKpAAAAAMBTfB781xa7GTgkXX/99XrhhRd09dVXa/bs2Tp06JCWLVum4cOHq6CgwPKpAAAAAABfkmmPYEpKip588kkNHTpUS5Ys0W9+8xvl5+dr2bJlSkjw5lICAAAAABBrTHsEJSknJ0e/+tWvos6z8GeJ2rev7e333SfN+5Hz5DP33x91KU3m3W635MMPfmCWytRIuVgXQiNdxVXVulps0JXKSrNUyip+yy7Z6aebpaqrd/5eJkFSXaaLdQRDFQYVfa7GbzcFdbLhFPOW08K7nf3bTVxRUVSlNJMxOtMsVzBUZZZra6bdOoI5cjO9fIYy3MT506Oup1Fw88dmuVRaapfL8Df377zjHDNxosu40YbXHcO1KLI2uyjepY9rJ5rlyh3iZokMn5L8znE7jNe0zSq1W47CcgmDSruVFZTiZlkql0tXbS2yuxdlZ9vtr8INZql0w3VWS7p4aT7NjtGdFpT3Up0AAAAAgE5AQxAAAAAAYoz50FAAAAAAiEUMDQUAAAAAeBYNQQAAAACIMQwNBQAAAAADcfJWT1tcO9u8VCcAAAAAoBPQEAQAAACAGMPQUAAAAAAwwKyhAAAAAADPoiEIAAAAADGGoaEAAAAAYCBO7c/U2dmYNRQAAAAA0ISGIAAAAADEGIaGAgAAAICBHkf+eUV7tdAjCAAAAAAxhoYgAAAAAMQYhoYCAAAAgIE4eaunrb1ZQ+MikUik0yo5BuFw+9t9PucYSZo/36YeSbrnHrtct9xil+tHP7LLlZzsHON23/tKS6Kup0llpVmqmuxhZrmS/C52hEthF5cNt/u+uNigoKNkhTaa5Vpfb7f/R4beMsul9HTnmJwcaetWx7Cn3skxKOiwK6bVmOWqi08yy1Vba5ZKwc3vOwedeqr0vou4zMzoC2pUX2+WqiqQYZYrEDBLJV+oyjkoGJSqnOPWFwUNKjos3vBr6iFD7HIlxNtd8xUKOce43PfmF32/3yzVW6V218OxY81SKaFyT/sB/fpJexxiGlkesG6OC5fKemWZ5erTJ/occXFSYmL0ebxuZXa2Qtu3d3UZTQIDB2pGUVGr27zUYAUAAAAAdAKGhgIAAACAAZ+81dPWXi1eqhMAAAAA0AloCAIAAABAjGFoKAAAAAAYYGgoAAAAAMCzaAgCAAAAQIxhaCgAAAAAGGBoKAAAAADAs2gIAgAAAECMYWgoAAAAABiIk7d62uLa2ealOgEAAAAAnYCGIAAAAADEGIaGAgAAAIABZg0FAAAAAHgWDUEAAAAAiDEMDQUAAAAAA3Fqf6bOzsasoQAAAACAJnGRSCTS1UW0pq6u/e0JCc4xkhQK2dQjSbffbpfrl78Im+W6d5Fde/7O71U4B6WkSBXOcW8VphhUdNj40TVmufSnP5mlWj/k22a5RtaudQ7Ky5PWOseFx+YZVPQ5X9FWs1x1mTlmuRIq95jlWl/azzFm5Ehp/XrnXCNGGBR0hE9214q6ertrRbzheJLiYueYrCxpxw4Xcal214qq+iSzXEFVmeUKB4Jmudzw+aSwi8PQV+/ipuzSxs0JZrmGDbGrS/X1drncnEQuP+xY7i9JGpZcYpesttYsVUWy3f0jJdnhoHZ74FszPMZ2lNodF1mhjdEn6dlTOumk6PN43O+ys3Vg+/auLqNJ74EDdVFRUavbGBoKAAAAAAZ6HPnnFe3VwtBQAAAAAIgxNAQBAAAAIMYwNBQAAAAADMTJWz1tzBoKAAAAAGhCQxAAAAAAYgxDQwEAAADAgE/e6mlrrxYv1QkAAAAA6AQ0BAEAAAAgxjA0FAAAAAAMMDQUAAAAAOBZNAQBAAAAIMYwNBQAAAAADDA0FAAAAADgWTQEAQAAACDGMDQUAAAAAAzEyVs9bXHtbPNSnQAAAACATkBDEAAAAABijGeHhpaWSvX1bW/PyZGKi53zxBu+wkWL7HLdu8iuDX777WapdMWVKY4xTz0lXXGTi7if7bEoSZJUUtnPLFf5kG+b5TI1YoRZnC9UFWUxX9DeyXiMEt55zSxXzdgJZrlGJu9wEZXlKi6srOgLOqKi0u5a8dlnZqmU1qfGLFdqapLLOLOndCUow/OostIs1Rvrgma5zjjDLJXpdaK+PsEsl+UHgdfesatryBDnmIwMqaTc+TmH+bcaVPS5qkCOWa5gst21otzF5z63UgrfaT9g/HjpHYeYI8Knjzeo6LDKkN0xZnjZkZKHRZ2iRw9pgEEpXsesoQAAAAAAz6IhCAAAAAAxxrNDQwEAAACgO4lT+zN1djZmDQUAAAAANKEhCAAAAAAxhqGhAAAAAGCgx5F/XtFeLfQIAgAAAECMoUcQAAAAAAzEyVs9bUwWAwAAAABoQkMQAAAAAGIMQ0MBAAAAwIBP3uppa68WL9UJAAAAAOgENAQBAAAAIMYwNBQAAAAADDA0FAAAAADgWTQEAQAAACDGMDQUAAAAAAwwNBQAAAAA4Fme7RE8cED67LP2Y0Ih5zwj4zfaFCSpLn2YWa47v1dhluuKK1PMcj35pF3czbf0i66Yo/zHf5il0sgRYbtkbg5Cl7aWBh1jcnKkraVJjnHFxRYVfW7C0FqzXDWZuWa5qqvNUqkyPssxJkNSiZu48j0GFR0WSLY7j0yPiz52qdzUlZvrMi7Z7pysCdjt+3I5n99uxRveudetc4459VR3campztcmtzIzzVKpotLuO+9AwCyVevQwjEtOjqaUFixfp4rLzVKdcILz9detitTx7W5PkVQxtP2Yptg3XjOo6EiuESPMcgWG2n0+LC01S4Uusnv3bjU0NDR7zLMNQQAAAADoTuLkrSGXcUf+O2PGDO3atavZNhqCAAAAAHAcW7lyJT2CAAAAABBL+vfv3+IxGoIAAAAAYKA7zRpq3hC8+OKL9cEHH7R4fPLkyVqyZIn10wEAAAAAjpFpQzASiWjLli3Kz8/XpEmTmm0bMGCA5VMBAAAAAL4k04ZgcXGxampqdPbZZ6ugoMAyNQAAAAB4Wpw+n6nTC9qrxXQI6+bNmyVJgwcPtkwLAAAAADBk2hD85JNPJH3eEKypqbFMDwAAAAAwYN4Q7N27txYuXKgxY8ZozJgxys/P15o1ayyfBgAAAAA8p4cH/7UlLhKJRExetaSLLrpIGzdu1OTJk3XBBReoqqpKy5cvV2FhoRYvXqxp06ZZPRUAAAAAeMr72dk6tH17V5fRpNfAgTq1qKjVbaYNwaefflrhcFgzZsxoeqy2tlbnn3++Dh48qNdee009erTXLv3cRx9Jn33W9vaRI6X1653zjIzf6Or53KgbMswsV0KowizXFTelmOV68knnGJ9PCoed4265JepymvzHf9jlSuvroni3QiGzVFvLg44xOTnS1q3OuYqLDQo6yoShe8xy1QT6meWqrjZLpYYG55iMDKmkxEVcvN3+qku221+Wx0VOut3Q/4+LkxxjcnOljz92zpWb7M1jtbzcLJXp++j3O8eceqr0/vvOcamp0dfTKBCwy2Wpjc9SX4qbydTT0qSyMhdxPe0+U0hSONnuc4WveIdZrrJeWWa5evZsf3tKilThcrembHgt+oIajRhhlqouYPc+lpZGn6NHD3fHfXfXnRqCpkNDL7/88maNQEny+/0qKChQeXl502QyAAAAAHC8idPni8p74V+nzRralpSUw99IMHkMAAAAAHQ9s4ZgWVmZpk6dqqVLl7bYtm3bNklSZmam1dMBAAAAAL4ks4ZgWlqaqqurtWrVKoWO+t3U7t27tXr1ao0bN059+/a1ejoAAAAA8JSuHgra2r+2xJu84iPuuusuzZkzR5dddpkuueQSHThwQCtXrlR8fLzuvvtuy6cCAAAAAHxJpr8RzM/P18MPP6zExETdf//9evzxxzV69Gg9/fTTTYvMAwAAAAC6lunyEabq6trfnpDgHCOppj7BqCApqdLFvPEuvVWUYZZrfLZdXTcvcq5ryRLp5pudcy35md0yDbfeZvedxQ9/aJbKVNoBF+tCuFw/4uP6HIOKPpeebpcr6Hc+b7uEm/n93a4fUVgYfT1HVIyeaJYrpdLFMeaW4W++6+R8nXZ5yVeCDI+v2lq7XJWVZqnCmXZT6LvhdskgX73dvndzTLhluMqPUuKrzHKFA85LBrnd95avUbJd7sRyegiLJQwaZb23uv2Ab39bWu0Q0+iss6Kup0l9vV0uw3uRybIWPp+UnBx9Ho/7IDtbdR5aPiJh4EB9rTOWjwAAAAAAeB8NQQAAAACIMaaTxQAAAABArHKaqbOztVeLl+oEAAAAAHQCGoIAAAAAEGMYGgoAAAAABuLkrZ62uHa2ealOAAAAAEAnoCEIAAAAADGGoaEAAAAAYIBZQwEAAAAAnkVDEAAAAABiDENDAQAAAMBAnNqfqbOzMWsoAAAAAKAJDUEAAAAAiDEMDQUAAAAAAz0khbu6iKP0aGcbPYIAAAAAEGNoCAIAAABAjGFoKAAAAAAYiJO3etramzU0LhKJRDqtkmNx8KDUXmlJSVJNjWOaHeVJZiVl1X5slkvZ2WapdpQmmOXq1cs5Ji1NKitzjvvv/46+nkaLFtnluuUWu1yXXWaXKzPTOSYrS9qxwybXsaistMuVUr/HLpnfb5cr3sX3Yi6vO+s32113LN/LlPgqs1w7KoNmubIyXfyawueTwi7i3nkn+oIaDRlilysQMEtVVm13fL37rnPMhRdKv/+9i7iz7I4v1debpdoRSjHLlZVs9xpLQs7nUEaGVFLinCs93aCgo/iKtprlKuudY5YrMdEslYIBh+uJ22uOJBUVRV1Po5p0u/1VXm6WyubYj4uT+vSJPo/HbcnOVv327V1dRpP4gQM1uI1j1EsNVgAAAABAJ2BoKAAAAAAY8MlbPW3t1eKlOgEAAAAAnYCGIAAAAADEGIaGAgAAAIABhoYCAAAAADyLhiAAAAAAxBiGhgIAAACAAYaGAgAAAAA8i4YgAAAAAMQYhoYCAAAAgIE4eaunLa6dbV6qEwAAAADQCWgIAgAAAECMYWgoAAAAABjwSYp0dRFHYdZQAAAAAEATGoIAAAAAEGMYGgoAAAAABuLU/kydnY1ZQwEAAAAATeIikYiXfs/YpKxMamhoe3tGhlRS4pwnY/NrZjXVjJ1glivp5d+b5do45EKzXMOGhp2DfD4p7BxXttfue4bFi81S6YH7XbxGl759sd1rXP0/dvve2o5iu9fZq5dZKqX1qbFLVl7uHJOVJe3Y4RyXmRl9PUdUhbz5fV3QX2eWa0dpgmOM211veXz17WuXKxSyyxWsr7BLFgg4xyQkSHXO73dNvfP76FZSaI9Zrq2hfma5cvwuPnhYcvlhZ2tthunT5qTbXVtfey/JLNcZZ5ilcjwng0GpqspdruB7r0RfUKOhQ+1yFRXZ5Yo3GESYkCCNHh19Ho/bnZ2thu3bu7qMJj0GDlT/No4FhoYCAAAAgIEeXV3AF7RXjze/agYAAAAAdBgaggAAAAAQYxgaCgAAAAAG4uStnjZmDQUAAAAANKEhCAAAAAAxhqGhAAAAAGDAa71s7dXjtVoBAAAAAB2MhiAAAAAAxBiGhgIAAACAAa/1sjE0FAAAAADQhIYgAAAAAMQYhoYCAAAAgAGv9bIxNBQAAAAA0ISGIAAAAADEGIaGAgAAAICBOHmrpy2unW1eqhMAAAAA0AloCAIAAABAjPHs0NA0lUlqaCciQxkqcU50+ulWJSkpPmyW6/3MC81yxdebpZJqa51jkpJcxfXsmWRQ0GEXX2yWSt++2O77j9X/Y3dMLP6pc10//KG7uO9/36Kiz2Vm2uXyFW60S9Yz3SxVXXqWY0yCy7h9ew0KOiIx0S5XebldrmDRG2a5QukT3cWFnGOy/HuirOYotQGzVIGA3fVQIcNbd72LG0hCgqu42toEg4IOS6qsNMuVk2n3Pqqo0ixVSfIwx5gMSSXKcIzLCRge95L0zgazVCNGuzu/3fC9t9YsV3DsWKdnUzDg7h7/Wrzda5xQu9Usl+VnYJMbiC82+p98kiJdXcRRGBoKAAAAAGhCQxAAAAAAYoxnh4YCAAAAQHfC0FAAAAAAgGfREAQAAACAGMPQUAAAAAAwwNBQAAAAAIBn0RAEAAAAgBjD0FAAAAAAMBCn9odjegk9ggAAAAAQY2gIAgAAAECMYWgoAAAAABjoTr1s3alWAAAAAIABGoIAAAAAEGMYGgoAAAAABrpTL1t3qhUAAAAAYICGIAAAAADEGIaGAgAAAICB7tTLFheJRCJdXUSrduyQ6uvb3p6TI23d6pimJj3HrCS/3yyVfO+ttUs2YoRZqq2lSY4xLne9cuQiyKWtsnsfc7LDZrnuXWR3ut95u4u6fD4p7BxnWZfksjaXamrtakuqrTDL5eoET0qSampscrlUttduf514olkqlZfb5XIjI0MqKXERF6iye9LSUrNUZSfkmuWyfB8Tyt3sVJc739CO+gyzXIGAWSqlVNrd15Sa6hwTDEpVLo7peOPv9S3z1dZ6MleVv1+7293uesn0UqHcVLv72o5QilmurNqPo08SH3/4Q+RxLpKdLW3f3tVlfG7gQMUVFbW6iR5BAAAAADiO7d69Ww0NDc0eoyEIAAAAAAbi4uKkuLiuLuNzR2qZMWOGdu3a1WwTDUEAAAAAOI6tXLmSHkEAAAAA6BDx8fa/243GkVr69+/fYtOXmoVg/vz5mjlzZovHd+7cqZtuukl5eXnKy8vTvHnzVFFhOJkDAAAAACBqx9xcXbVqlVatWqW8vLxmj+/bt09XXXWV6urqdN1116mhoUGPPfaYNm3apFWrVikhIcGsaAAAAADAl+e6IdjQ0KBHHnlES5cubXX7E088odLSUv3hD3/Q4MGDJUmjRo3SNddco+eff17Tp0+3qRgAAAAAvKhHD28NDe3Ro81NroaGHjp0SBdddJEeeughFRQUKC0trUXMmjVrlJeX19QIlKTx48dr0KBBWrNmzZeoGgAAAADQEVw3BEOhkB588EEtXrxY8V9o5e7fv187d+7U8OHDW/zt8OHDtWHDBptqAQAAAABRc9VvGQgE9NJLL7VoADYqKyuTpFZ7Cvv27atQKKTq6mr16dMnilIBAAAAwMM8Omtoq5vc/L3P55PP13bn4YEDByRJiYmJLbb16tVLklRTU3NsDcGsLOeYnBzHkCT3z9i5vjDZjle42KXHEOcyWadmkr7kZLmtuvNOs1RyXVc752Ij27oky32WZHlSJqUYJnP7nJ17VWnl+zVPyMjw6nMG7Z4waJfLo2+j+zeyk99wF58AukaK7d3IFcPjsEtYThZouC/cZHL7dLZvkd19Lcv0FplrmQweYdJcDYfDjjHtNSRbtWOHVF/f9vacHGnrVsc0Nel2F22/3yyVfO+ttUs2YoRZqq2lzh9yXe565chFkEtbLRuV2c7Hq1v3LjJsVN7uoi6fT3JxvlnWJbmszaWaWsNGZa3h8jRuTvCkJKmmxiaXS2V77fbXiSeapVJ5uV0uNzIypJISF3GBKrsnLS01S1V2gt2HKMv3MaHczU51ufMN7ai3a3gGAmaplFJpd19TaqpzTDAoVbk4pq17Hyzz1dZ6MleVv1+7293uesn0UqHcVLv72o6QYaOy9uPok8THu+9xQKcwOdN79+4t6fBvCb+o8bHGGAAAAAA4Lh1vs4Y6yTgyZGTv3r0ttu3Zs0fBYFBJnTycCgAAAADQOpOGYDAYVGZmpj788MMW2zZu3KgRhkMXAQAAAADRMfvxyaRJk/T2229ry5YtTY+99dZb2rZtm6ZMmWL1NAAAAADgTY2zhnrpX1ulWr3m66+/Xi+88IKuvvpqzZ49W4cOHdKyZcs0fPhwFRQUWD0NAAAAACBKZj2CKSkpevLJJzV06FAtWbJEv/nNb5Sfn69ly5YpwXLqYAAAAABAVL5Uj+Arr7zS6uM5OTn61a9+FVVBAAAAANAtdaNZQz1UZXNb67PaXUYwV9LH9c5rkeSG9pjV9HFx+2vOHIshY+0WlPdV2q05U1zsbh3B4mLnXPWGazgOGWKWytQtt9jlcrP23513uowzXPdPkm6cY7eW3Y03mqXSkCF2ayS1d71pFJRUVe98jgRlt//TTqwzy+XqRbqUUb7ZLFdV9khXca7WgwuFoivmaIbrQaY12K3DF443XNw9Odksri7ebnbw8g1mqSwPe6UYHl9u1jlOklQT77xaeVL5DoOKjuJmjUO3LBfZy842SxUMOX12SlGw3t3nq4Mn2N2LwsmG97VKs1Qma6H6fFJfg1pgx3bVaQAAAACA53m2RxAAAAAAuhWHmTo7XTu10CMIAAAAADGGhiAAAAAAxBgP9VsCAAAAQDfWjWYNpUcQAAAAAGIMDUEAAAAAiDEe6rcEAAAAgG6MWUMBAAAAAF5FQxAAAAAAYoyH+i0BAAAAoBtj1lAAAAAAgFfREAQAAACAGOOhfksAAAAA6MaYNRQAAAAA4FU0BAEAAAAgxnio3xIAAAAAujFmDQUAAAAAeFVcJBKJdHURrSorkxoa2t6ekSGVlDimqUnOMCspqfhjs1ymLL91SE52jklJkSoqHMOq4lOir+eI+nqzVKqstMuVnW2Xy6ewiyCfFHaOu3GO7Xc8jzzsojaX7pxvV9s995ilkq+2xjkoKUmqcRFXWxt9QY3Ky+1yuTm/XaoJ9DPLlVTpfC13e82X3x99QR2Qa2tpklkuy5eYkWx43K9bF3U9jUqyx5vlyqjdapbL8n4bzsxyjHF5ydeGDQYFHWXk0DqzXFW1CWa5gn67uhw/WLg97iWt32x3fo9M32OWa32p3XV66FCbPAl2h4N3TZ0q7d7d1VV8rn9/ac2aVjd5qN8SAAAAALoxZg0FAAAAAHgVDUEAAAAAiDEe6rcEAAAAgG6MWUMBAAAAAF5FQxAAAAAAYoyH+i0BAAAAoBtj1lAAAAAAgFfREAQAAACAGOOhfksAAAAA6MaYNRQAAAAA4FU0BAEAAAAgxnio3xIAAAAAujFmDQUAAAAAeBUNQQAAAACIMR7qtwQAAACAboxZQwEAAAAAXkVDEAAAAABijIf6LZur7JWmcLjt7SmSKvwZjnlS/O0kOUbra3PNcg0ZYpZKSeveMstVl5njGJMgqS6Q4hgXVJ1BRUdUVpqlqvX3M8vl2/yxWa6aTOfjKylJqql1/v7mxhstKvrcnfPtvjO69x67c3L6ZXZ1PfuM312g30Wcmxi3AgG7XJs3m6VKMjwnlZ3tLi411TmmtjaqUo62sSjJLNewwA6zXOH0LLtccn6NPklhv4u45OToCzoiI93uOqENIbNUNUNGmuWSi0M1KcndIT0ysyL6eo5Wb3cNCxpeDitCCWa5UkKl7QdkZUnl5a5yjcw0vE6H7K5hI0fYnUdbi6K/38bHH96txz1mDQUAAAAAeBUNQQAAAACIMR7qtwQAAACAboxZQwEAAAAAXkVDEAAAAABijIf6LQEAAACgG2PWUAAAAACAV9EQBAAAAIAY46F+SwAAAADoxpg1FAAAAADgVTQEAQAAACDGeKjfEgAAAAC6MWYNBQAAAAB4FQ1BAAAAAIgxHuq3BAAAAIBujFlDAQAAAABeRUMQAAAAAGKMh/otAQAAAKAb60azhnqoyuaSk+ocIhKUEnCKkUpKE2wKkjSydq1ZLoWyzVLVjB5vlqt6n3NMWpq0z03cidHX08TvN0uVoSqzXPKnmqVKqq1wEZTiKm7IkBSDij53zz12uaZfZjcQ4dlnwma5fv2Ec12zZ7uLO+MMi4oOy8y0u4YlWd6YUu2Ofb36qnPMpEmu4urOmhR1OR0hnJlllsu37n2zXFVDTnWMCQalUMg5V9CgniabN9vlCgTMUiXFO3/ucOu1d5zP7QkTpPfec841YYRBQUcpq04yy5XW1+46nVJkd+zXDG3/2E+SVJPq7ry1vLQmJCfbJXNz4rqUk211HjEY0Ut4NwAAAAAgxni2RxAAAAAAuhVmDQUAAAAAeBUNQQAAAACIMR7qtwQAAACAbqwbzRpKjyAAAAAAxBgaggAAAAAQYzzUbwkAAAAA3RizhgIAAAAAvIqGIAAAAADEGA/1WwIAAABAN8asoQAAAAAAr6IhCAAAAAAxxkP9lgAAAADQjTFrKAAAAADAq2gIAgAAAECM8VC/JQAAAAB0Y8waCgAAAADwqrhIJBLp6iJa87e/SQcPtr19yhTpxRed85xxhl1NwfKtZrk21uaY5RoW2GGWqyQ+yzEmI0MqKXHOlRG/x6CiI5KT7XKVlpqlqkt33l9uJdTXOAclJUk1znFV9UkGFX0uGO+iNrf8frNUv37C7rus2VeHnYN8PinsHHfrbXZ13X+/WSpTlZV2uVJq3VxQXF54LKWnd+7zuRUKmaXaWBx0jBk2TNq40TlXdnb09TSyPL7amSfhmKX98yW7ZKef7hwTDEpVVc5x1r0Phm9AhT/DLFdKveHnCqd7kdt9L2l9kfN55NZI/8dmuTbW55rlysyMPkdcnNSnT/R5PO+BB2wvYtFKTpZuvVW7d+9WQ0NDs00e6rcEAAAAgG7Mo7OGzpgxQ7t27Wq2yUNVAgAAAACsrVy5kh5BAAAAAIgl/fv3b/HYl2oIzp8/X9u3b9eKFSuaPX7xxRfrgw8+aBE/efJkLVmy5Ms8FQAAAAB0D91o1tBjrnLVqlVatWqV8vLymj0eiUS0ZcsW5efna9KkSc22DRgw4FifBgAAAADQQVw3BBsaGvTII49o6dKlrW4vLi5WTU2Nzj77bBUUFJgVCAAAAACw5aoheOjQIV1yySXatGmTpk2bprfffrtFzObNmyVJgwcPtq0QAAAAALoDj84a2hpXi10dOnRIoVBIDz74oBYvXqz4Vl7cJ598IunzhmCNi7XOAAAAAACdz1VzNRAI6KWXXmq1Adjok08+Ue/evbVw4UK9+OKLqqmp0Ve/+lXNnTtXU6dONSsYAAAAABCduEgkEjnWP5o4caIGDBjQbNbQiy66SBs3btTkyZN1wQUXqKqqSsuXL1dhYaEWL16sadOmWdYNAAAAAN6ybJlUVdXVVXwuGJSuu67VTWYDWKdPn65wOKwZM2Y0PTZ16lSdf/75+ulPf6oLLrhAPdoZo/pFf/ubdPBg29unTJFefNE5zxlnuH5KR8HyrWa5NtbmmOUaFthhlqskPssxJiNDKilxzpURv8egoiOSk+1ylZaapapLd95fbiXUuxhOnZQkuRh2XVWfZFDR54LxhkO9/X6zVL9+wtXodldmXx12DvL5pLBz3K232dV1//1mqUxVVtrlSql1c0FxeeGxlJ7euc/nVihklmpjcdAxZtgwaeNG51zZ2dHX08jy+DqGjx6O0v75kl2y0093jgkG3X2gtP49kuEbUOHPMMuVUm/4ucLpXuR230taX+R8Hrk10v+xWa6N9blmuTIzo88RFyf16RN9Htgx+7Ry+eWXN2sESpLf71dBQYHKy8ubJpMBAAAAAHStDp/SJiUlRRKTxwAAAAA4zh1vs4Y6KSsr09SpU1tdY3Dbtm2SpEyLPmUAAAAAQNRMGoJpaWmqrq7WqlWrFDrqtwu7d+/W6tWrNW7cOPXt29fiqQAAAAAAUTLrt7zrrrs0Z84cXXbZZbrkkkt04MABrVy5UvHx8br77rutngYAAAAAvCk+3ltDQ9upxWyymPz8fD388MNKTEzU/fffr8cff1yjR4/W008/3bTIPAAAAACg632p5uorr7zS6uP5+fnKz8+PqiAAAAAA6Ja60WQxHqqyudxcqaGh/ZgRI5zzFBWZlCNJ2rDBbu2/yy4zS6Ww7Nayyyh3s0ZPP3drBG7YEHU9jdanTjTLNWKE3f7at9csldL6ulxfz8U6fEG5WBPvWFTW2uUyXEfQcp1QN2v/PfCAy7j77fb/zbfYrUm4aJFZKgUCdrkU7zKZiycNB+zW87K8f+Rk1tklM/yAMWSIXVxCud06j0m13rzmuFr7zy03rzEYdBdn+Rolqb7eLJVpabV2x37ZwfavFWlB55hGIzMrLEo6LGS3w4bV261/XVEf/Wdgn93tDEZ4SwAAAAAgxni2RxAAAAAAupVYnCwGAAAAANA90BAEAAAAgBjjoX5LAAAAAOjGutGsofQIAgAAAECMoSEIAAAAADHGQ/2WAAAAANCNMWsoAAAAAMCraAgCAAAAQIzxUL8lAAAAAHRjzBoKAAAAAPAqGoIAAAAAEGM81G8JAAAAAN0Ys4YCAAAAALyKhiAAAAAAxBgP9VsCAAAAQDfGrKEAAAAAAK+Ki0Qika4uolU7dkj19W1vz8mRtm51zpOdbVaSamvNUoX9SWa5QiGzVAoEnGN8Pikcdo6rrIy6nCaWX6y0d1gdK8u6Dh50jklLk8rKXMSdWBd9QUcrKrLLZXhO1tQnmOXy+51j3B77t9wSdTlNlvzMxRO6dOE0u+/+/ud/zFIpoehj56DcXOlj57iyE3INKjosrWeFWa5wcopZriefNEulWefucQ7q10/a4xy3NdTPoKLDiovNUik52S7XyHQX+8ulKr/z/goGpaoq51zBWru6JLm7ILplef8YMsQsVVV9+5/D3O57yfZzWIZKzHKF0zPMcvnWvR99koQEacSI6PN43V/+4u5DXWdJTJTOOafVTR7qtwQAAACAboxZQwEAAAAAXkVDEAAAAABijIf6LQEAAACgG2PWUAAAAACAV9EQBAAAAIAY46F+SwAAAADoxpg1FAAAAADgVTQEAQAAACDGeKjfEgAAAAC6MWYNBQAAAAB4FQ1BAAAAAIgxHuq3BAAAAIBuLD5eamjo6io+x6yhAAAAAIBGNAQBAAAAIMYwNBQAAAAALDBrKAAAAADAq2gIAgAAAECMiYtEIpGuLqI11dVSe5UFg1JVlXOeoL/OrKY6JZjlqq83S6XqartcBw44x+TkSFu3uoiL3xF9QY1SU81S1SjJLFd5uVkqpac7xyQkSHUuDumE+proCzpaKGSXy3KnGQ69CA/JdYzx+aRw2DlXba1BQUdcdpldrt8/76J4l556xu57xCvy9zgH9esn7XERV1gYfUFHbEydYJZrWMDuelgRyDLLlRJyUVdWlrTD8HruhuE13/KELPssxSzXp586xwwbJm3c6ByXnR11Oc1YfkYJBuyuO6+8anfdmTjW4UOk2w+akul9rSI5xyxXiirMcq0vjv7Y79lTOuUUg2K8bsMGdx/WOktCgjRiRKub6BEEAAAAgBhDQxAAAAAAYoyHprQBAAAAgG6MWUMBAAAAAF5FQxAAAAAAYoyH+i0BAAAAoBuLj29/6YPO1s4wVXoEAQAAACDG0BAEAAAAgBjD0FAAAAAAsNCjh7eGhjJrKAAAAACgEQ1BAAAAAIgxDA0FAAAAAAteWkxeYtZQAAAAAMDnaAgCAAAAQIzxWN8lAAAAAHRT7czS2SWYNRQAAAAA0IiGIAAAAADEGIaGAgAAAICF+HgpLq6rq/hcO0NDPdsQ7PPBW1JtbdsBEycq+N4rjnl2DJloVlNqqlkq+f12uZJUY5esj7tnzEl38Zzx6dFW02RHaYJZrqzMsF2uza+a5SqJdz5WMzKk8nLnXBnlmw0q+lzNkJFmuZIqK81yWZ6UbspKSXEXFwhEW83n/ud/7HI99YzdIJArLrM7j+6c388x5t57pTt/5hx32WXOMW7Vt3MLOmZD7K6HoVKzVFq3OcsxZmKW9IqbuOT3LUqSJFUEnJ/PvSSzTGk9K+xypda7iOqnYal7nMP8hh9QJG0stLtWDEu2O2BHj84wy6V6F/vfTYykst45URbzubTSjWa5wkOHmeVKDkWfw2s/nQNDQwEAAAAg5ni2RxAAAAAAupUePbw1NNTXdr8fPYIAAAAAEGNoCAIAAABAjGFoKAAAAABYiI+XwnYTqkWNoaEAAAAAgEY0BAEAAAAgxjA0FAAAAAAs9OjR7nDMTtfODKYeqhIAAAAA0BloCAIAAABAjGFoKAAAAABYiI+XIpGuruJzDA0FAAAAADSiIQgAAAAAMYahoQAAAABgoUePrq7ANXoEAQAAACDG0BAEAAAAgBjD0FAAAAAAsBDffZpX9AgCAAAAQIyJi0S8tNDF5/bulcLhtrenpUllZc550g7tsCuqstIs1Y7kkWa5UlPNUqm42DkmN1f6+GPnuOzsqMtpkhDfzsFwjHYU233/YXhIuHofMzKkkhLnuEAg+nqOFgy5eFK3LA/YV1+1yzVihHNMV7wBpaV2uZKTzVLd+bN+ZrnuvcfF+e3ztX9TOGLBPXbn911XbjXLZbnvFQrZ5XJTVzAoVVU5hoUDwejrOcJXW2OWq6w6ySxX2ol1Zrm0bp1zTF6etHatY9iL5XnR13OUM86wyxWstPscVhHIMsvldJlOSJDqXL7dCeWG98jycrNUZWl2nzXNjv2EBJs8XubiXtXpfD7t3r1bDQ0NzR7uPn2XAAAAAOBhYQ8OuPRJmjFjhnbt2tXscRqCAAAAAHAcW7lyJT2CAAAAABBL+vfv3+Ix1w3B119/XY888og+/PBD+Xw+jRo1SrfccotGjx7dFLNz504tXrxYa4+MZz/rrLN0++23KyUlJfrqAQAAAMDD6uu7uoKW2vpppquG4Nq1a3X99dfrpJNO0ty5c1VfX6+nnnpKV155pZ566imNHDlS+/bt01VXXaW6ujpdd911amho0GOPPaZNmzZp1apVSoiFH4cCAAAAQDfgqiF47733qn///nr22WeVmJgoSZo2bZqmTJmiBx98UI8//rieeOIJlZaW6g9/+IMGDx4sSRo1apSuueYaPf/885o+fXrHvQoAAAAAgGuO09rs379fhYWFOvfcc5sagZKUmpqq0047Tf/7v/8rSVqzZo3y8vKaGoGSNH78eA0aNEhr1qzpgNIBAAAAwDsaGg4PD/XKvy/MD9OMY49gIBDQn/70p2aNwEb79u1Tjx49tH//fu3cuVOTJ09uETN8+HC9arnWFwAAAAAgKo49gj169FB2drbS0tKaPV5YWKj3339fY8aMUdmRld2/GCNJffv2VSgUUnV1tVHJAAAAAIBoxEUikcix/tGBAwd0xRVXaNOmTVq+fLl69uypyy67TPfcc48uueSSZrEPPvigfvGLX+i1115rtaEIAAAAAMeD6mrp2FtXHScuTurTp/Vtx7yO4MGDB3XjjTeqsLBQ3/3ud5WXl6d//OMfjn/n8zl2Pjazd68UDre9PS1NOtIR2a60QzuO6XnbVVlplmpH8kizXKmpZqlUXOwck5srffyxc1x2dtTlNEmIb+dgOEY7io/tWGyP4SHh6n3MyJBKSpzjAoHo6zlaMOTiSd2yPGAth52PGOEc0xVvQGmpXa7kZLNUd/6sn1mue+9xcX77fO3fFI5YcI/d+X3XlVvNclnue4VCdrnc1BUMSlVVjmHhQDD6eo7w1daY5SqrTjLLlXZinVkurVvnHJOXJx1Zkqs9L5bnRV/PUc44wy5XsNLuc1hFIMssl9NlOiFBqnP5dieUG94jy8vNUpWl2X3WNDv2WUXAU47pjllVVaXZs2fr3Xff1Xe+8x3NnTtXktS7d29J0qFDh1r8TeNjjTEAAAAAgK7lukfw008/1bXXXquPPvpIl156qX784x8rLi5OkpSRkSFJ2rt3b4u/27Nnj4LBoJKS7L6RAwAAAACvaWhwNYCl07Q3KNNVQzAUCjU1Aq+++mrdcccdzbYHg0FlZmbqww8/bPG3Gzdu1Ag3Q64AAAAAAJ3C1dDQBQsW6KOPPtKsWbNaNAIbTZo0SW+//ba2bNnS9Nhbb72lbdu2acqUKTbVAgAAAACi5tgjuGXLFr3wwgvq06ePTjnlFL3wwgstYgoKCnT99dfrhRde0NVXX63Zs2fr0KFDWrZsmYYPH66CgoIOKR4AAAAAvKK+/jgaGrr2yGxV1dXVbfYGFhQUKCUlRU8++aQWLlyoJUuWyO/3Kz8/X/PmzVMCMwQBAAAAgGc4NgQvv/xyXX755a6S5eTk6Fe/+lXURQEAAAAAOs6XWlC+Uzgt3nIsC7xYsVw0znqhNyu1tc4xKSlSRYVznOVrfO89s1Rlg8eb5Urr6WI/uBXvYu4ml+t5ma4zJkl+v10uN6/TpTq/3bplrtaBcrmOYDg9w6Ciw1qZjPlLS9v0mlmu9ckTzHI9/7xzzF13SQsWuIibbzce59bb7NYkvP9+s1S2x0TZeuegkSOl9S7iDBeP/bjU7tzOLfy9WS5X6426VJWa4xjj9pIfLN5oUNFRMjPNUpm+l8l7zHI5ft7JypJ2uFwD0aP3SNPPAhZrAMfFSYmJ0efxuLKywzOHekWPHofXX2+N3V0OAAAAANAt0BAEAAAAgBhj2P8MAAAAALGrvt5bQ0Pb+xEgPYIAAAAAEGNoCAIAAABAjGFoKAAAAAAYaGg4PDy0O6BHEAAAAABiDA1BAAAAAIgxDA0FAAAAAAP19QwNBQAAAAB4FA1BAAAAAIgxDA0FAAAAAANemzU0Lq7tbfQIAgAAAECMoUcQAAAAAAx4bbIYegQBAAAAAE1oCAIAAABAjGFoKAAAAAAY8NpkMb52uv3oEQQAAACAGBMXiUQiXV1Ea6qrpfYqCwalqirnPMHyrXZF+f1mqaoCGWa5gnKxI1yqUtD5+dzue8O6LL9aCSenmOXy1deZ5VJRkXNMbq708cfOcYbHqiQpNdUs1caiJLNcloYNDTsH+XxS2Dlua5Hdd2w5yRVmuTaW2h37tbVmqXRqsovrdE6OtNU57talOQYVHfbA/S6OCZdunGN3TNxxh1kqZWY6x7g87FVZGXU5TeINxysF/XbX6XB8glkuV/ePhASpzkXchg3RF9RRRowwS/XWe3b7f/xYh/3qdt9Ltgd/aalZqrqhI81yJdTXRJ8kLk5KTIw+j8dt2OD+0OkMCQltn4YMDQUAAAAAA16bNZShoQAAAACAJjQEAQAAACDGMDQUAAAAAAx4bdbQHj3a3kaPIAAAAADEGBqCAAAAABBjGBoKAAAAAAa8NmsoQ0MBAAAAAE1oCAIAAABAjGFoKAAAAAAY8NqsofHttPboEQQAAACAGENDEAAAAABiDENDAQAAAMCA12YNba8WegQBAAAAIMbQEAQAAACAGMPQUAAAAAAw4LVZQxsa2t5GjyAAAAAAxBgaggAAAAAQYzw7NLRXL+cYv99FouLiqGtpVDV6glmuoKrMcqmy0i6Vgo4xwaC7pwzKRZBb6elmqUIhs1QKBBLMcu09IdcxJk1SmZu4hhKDij63tTTJLNewwA6zXOHMLLNclnIy68xyheNTzHINC9ntew2xOycVSnYXl+wcd//9UVXSzI1z7L4rfeThsFmuC6fZ1bV0qXNMVpa7W2lWut1xb3mhrgjZnUOlpWaplJzsfP/IyJBKyl3EuTg3jsVrxTlmuSbU15jlGjzY7p5rqSbQzyyXf4RdroRSw88CtbXR54iPP3xBOc4xaygAAAAAwLNoCAIAAABAjPHs0FAAAAAA6E6YNRQAAAAA4Fk0BAEAAAAgxjA0FAAAAAAMMGsoAAAAAMCzaAgCAAAAQIxhaCgAAAAAGGDWUAAAAACAZ9EQBAAAAIAYw9BQAAAAADDArKEAAAAAAM+iIQgAAAAAMYahoQAAAABggFlDAQAAAACeRUMQAAAAAGIMQ0MBAAAAwEB3mjXUsw3BhI3rpLq6tgPy8pSwbq1zotGjrUpSIGCWSmEFzXK9sc4uV7yLIyIrSyoudo7LPD0r+oKOKC01S6WM+D12yeQ3y3Tiie7exxNPdI4Jx2dEWU1zfsP9H063Oy586943y6UhQ5xjgkEpFHKOc3MiufTkMwlmuc4/327fhwyPiSy52KcpKa72/d7PUgwqOuyOO8xS6cJpdgNwfv982CzXvNud67rvPmnpUudcixbZHau+5GSzXP5as1QaNtRu31dUujsm/C5uM+H0nCiraW50ql2usuoks1x9+pilcr6euLzmSFJ8wO664ystMcu1vtzus8DIdIPPTj4GInoN7wgAAAAAxBjP9ggCAAAAQHfCrKEAAAAAAM+iIQgAAAAAMYahoQAAAABgoDvNGkqPIAAAAADEGBqCAAAAABBjGBoKAAAAAAaYNRQAAAAA4Fk0BAEAAAAgxjA0FAAAAAAMMGsoAAAAAMCzaAgCAAAAQIxhaCgAAAAAGGDWUAAAAACAZ9EQBAAAAIAYw9BQAAAAADDArKEAAAAAAM/ybI/gwZNHKxJpe3uSpJoReY553nvPrqYJo6vMctX5g2a5zjjDLJXWrXMX5/c7x/gUjqqWo733nt13Fheen2qWS7W1ZqkSykucgzIy3MUlJ0ddT7OnNUwXVpJZrqohp5rlKi52jhk2TNpY7HzuDhliUNARs87dY5csZHe8rtucZZYra2yyu0AXx3Va0fqoajlaeMRIs1xLl5ql0rzb7a6H9y1yc532uYpb/bxdXfn5drlsv5m3qysl3s1niqC7uA1F0ZbT/FnT0+2SJfYzS5VU6eL+51a8i4/ALg8eV/dllzZWZpjlCgTMUkmlpdHn6NlTSjX8DIZjsnv3bjV8YeYYzzYEAQAAAKA78eqsoTNmzNCuXbuabaMhCAAAAADHsZUrV9IjCAAAAACxpH///i0ec90QfP311/XII4/oww8/lM/n06hRo3TLLbdo9OjRTTEXX3yxPvjggxZ/O3nyZC1ZsuTLVQ0AAAAA3UB3mjXUVUNw7dq1uv7663XSSSdp7ty5qq+v11NPPaUrr7xSTz31lEaOHKlIJKItW7YoPz9fkyZNavb3AwYMiOoFAAAAAADsuGoI3nvvverfv7+effZZJSYmSpKmTZumKVOm6MEHH9Tjjz+u4uJi1dTU6Oyzz1ZBQUGHFg0AAAAA+PIcG4L79+9XYWGhrrnmmqZGoCSlpqbqtNNO05tvvilJ2rx5syRp8ODBHVQqAAAAAHiXV2cNbY1jQzAQCOhPf/pTs0Zgo3379qlHjx6SpE8++UTS5w3BmpoaJSXZrRcGAAAAALDhuDJqjx49lJ2drbS0tGaPFxYW6v3339eYMWMkHW4I9u7dWwsXLtSYMWM0ZswY5efna82aNR1TOQAAAADgS4mLRCKRY/2jAwcO6IorrtCmTZu0fPly5eXl6aKLLtLGjRs1efJkXXDBBaqqqtLy5ctVWFioxYsXa9q0aR1QPgAAAAB4w3/9l7RvX1dX8bkTT5T+8z9b33bM6wgePHhQN954owoLC/Xd735XeXl5kqTp06crHA5rxowZTbFTp07V+eefr5/+9Ke64IILmoaRunseqb0malKSVFPjnOe991w/paMJo6vMctX5g2a54g1Xg1y3zjnm1FOl9993ETc6HHU9jX7/R8fOa9cuPN+uLtXW2uWqrHSOyciQSkqc45KTo62mw4T9dkPGQyGzVCoudo4ZNkzauNE5bsiQ6OtplFC5xy6Z4fH6yuYss1wTx7q4tgaDUpWLuKKiqOtpFB4x0iyXm+PLraVL7XLdt8jF9dDnk8LOcauft7tO5+ebpTL9rY7fb5crqd6bx70kKT3dLFWVv59ZrmDIxf3PLacPT/36SXtcXn8ND7KNlRlmuSyP15zQ+uiT9OwpnXJK9Hlg5piu2lVVVZo9e7beffddfec739HcuXObtl1++eXNGoGS5Pf7VVBQoPLy8qbJZAAAAAAAXct1X9Knn36qa6+9Vh999JEuvfRS/fjHP1ZcXJzj36WkpEg6PHkMAAAAAByvutOsoa56BEOhUFMj8Oqrr9aCBQuaNQLLyso0depULW1lvMq2bdskSZmZmcdYNgAAAACgI7hqCC5YsEAfffSRZs2apTvuuKPF9rS0NFVXV2vVqlUKHfWjnd27d2v16tUaN26c+vbta1c1AAAAAOBLcxwaumXLFr3wwgvq06ePTjnlFL3wwgstYgoKCnTXXXdpzpw5uuyyy3TJJZfowIEDWrlypeLj43X33Xd3SPEAAAAA4BX19d4aGtpeLY4NwbVr10qSqqurW+0NlA43BPPz8/Xwww/r0Ucf1f333y+/36+8vDzdeuutTYvMAwAAAAC6nmND8PLLL9fll1/uKll+fr7yLed8BgAAAACYM1yBzlZiL6c1i3xK8juvazRhaLlNQZI2FtuthTMs23AWVcPu59RUd2u8paa6CDLsF7/wLLv1z2pq7dZwrK21WxMvRZVmueri7eqSpIT33jLL5TNc49DunZSys4e5jHOOSSi3W+tqa63dmlI58TvMck1MdrGYqEvhwKmOMT5J4YDzO+5z8wa55GZpT7ey0uvMci1alGCWy83af9/+tsu4aXZrtN67yG5NwjunuVj806WKdHfXCTeSal0shBoMulswdejQ6As6ytZiu2OsdztrQh+rynq762GWXFyn3X6OMVyjtbraLJWGZdqtf/1KUfTrqvr90niDWrzuuJs1FAAAAABw/KAhCAAAAAAxxrNDQwEAAACgO+lOs4bSIwgAAAAAMYaGIAAAAADEGIaGAgAAAIABZg0FAAAAAHgWDUEAAAAAiDEMDQUAAAAAA8waCgAAAADwLBqCAAAAABBjGBoKAAAAAAaYNRQAAAAA4Fk0BAEAAAAgxjA0FAAAAAAMMGsoAAAAAMCzaAgCAAAAQIxhaCgAAAAAGOhOs4Z6tiEYduis9LmIkSRfIGBUkaRyu1Qbi5LMclkebJmZ7uLc7NY6JURXzFES6kNmuZJq99jlKrc7KHYEhjnGZEnaUZ/hGFe+waCgo6RnjzfLlZEeNsulzZvNUlVWOsckJbmMq62NtpwmxcVmqZQzNtUsV0UgyyxXSm2Nc1BSknwu4j4uDRpUdFh6ulkqKWR3DfMlJ5vlys93NzAoP9855t5FdoOM7rzd7jox/gzna6tbb/2xwiyX3L6PLuKqau3ut5KUanepULDecJ9tt7vma+xY5xiXF4GyvXbH/jh/iVmukpDz5wW3Ro+OPoePcYiew1sCAAAAADHGsz2CAAAAANCdMGsoAAAAAMCzaAgCAAAAQIxhaCgAAAAAGOhOs4bSIwgAAAAAMYaGIAAAAADEGIaGAgAAAIABZg0FAAAAAHgWPYIAAAAAYIDJYgAAAAAAnkVDEAAAAABiDENDAQAAAMAAk8UAAAAAADyLhiAAAAAAxBiGhgIAAACAAWYNBQAAAAB4Fg1BAAAAAIgxnh0aunt3+12ZWVlScbFznqzN75jVNPSsiWa5fAqb5bJUFXL33UC8iyNn374oiznKoUMpZrksu+tzMv1muQIu6woEnGOshyRk1G61S7YhZJfLzc5wqUcPwzi/3XGRnGyWSqqtNUyWZJaprNo5V1qSu7jcwt9blHRY9rlmqSpCdtcwv+Hb6PZa4SbuzmkboyvmKOPPGGaW66037O63q5+3ex+/Pc1lXS6uJwf3RlnMF5x4ol2uqlq7fRYYm2eWq7Ky/e0pKVJFpbvPRPv3R19Po7R0u/taRu0es1wltf2izuH2PtvdMWsoAAAAAMCzaAgCAAAAQIzx7NBQAAAAAOhOmDUUAAAAAOBZNAQBAAAAIMYwNBQAAAAADDBrKAAAAADAs2gIAgAAAECMYWgoAAAAABhg1lAAAAAAgGfREAQAAACAGMPQUAAAAAAwwKyhAAAAAADPoiEIAAAAADGGoaEAAAAAYIBZQwEAAAAAnkVDEAAAAABiDENDAQAAAMBAd5o1NC4SiUQ6rxT36ura356Q4BwjSZs329QjScOGhu2S1daapXrtvSSzXIGAc8ypp0rvv+8ibmhN9AU1sjyjDPe9ysvtcvn9zjE5OdLWrc5xoVD09RwtOdksVU1qllmupHgXFwG3Xn3VOWbSJOmll5zjTj896nKaGB6vZZF+ZrnSelaY5XJ14XF70S8ujr6eI8LZOWa5CgvNUpnei2pqnQcGJSVJNS4u55aX1hTZHV+rX00xy/XtaXb7/uZbnPf9kiXSzTc757r9doOCjuLmlHSrstIuV1btx2a5ajJz293u9riXpKTKEoOKjkhPN0sVNhz455PRse87/gcj5uWZ3oqilpkprV3b+rbj/90AAAAAADTD0FAAAAAAMMCsoQAAAAAAz6IhCAAAAAAxhqGhAAAAAGCgO80aSo8gAAAAAMQYGoIAAAAAEGMYGgoAAAAABpg1FAAAAADgWTQEAQAAACDGMDQUAAAAAAwwaygAAAAAwLNoCAIAAABAjGFoKAAAAAAYYNZQAAAAAIBn0RAEAAAAgBjD0FAAAAAAMMCsoQAAAAAAz4qLRCKRri6iNXV17W9PSHCOkaSE+LBNQZJUW2uXy+83S1W2t3Pb82lpUlmZc1zfvnbPWVpqlytDJWa5SpRhlisjUOUcFAxKVc5xNfFBg4o+Z3i4mp5G771nl2vCaLv9b/kiq/z9zHIVF5ul0rDUPXbJioqcY/LypLVrHcOqhuZFX88RQb+Lm4xLJeUJZrksz8eUeMPjPhSKvqBGycl2uQx32M232N1vl/zMxecTn08KO8ddcaXt54ClS+1ypdTbXSsq4u2uh/EOY+LcHvaSVF4efT2N0tPtciWVbjXLVZGcE3UOn8/21PaqQYOk7du7uorPDRwobdsm7d69Ww1fmDmGoaEAAAAAYCASCctL3WyHa/FpxowZ2rVrV7NtNAQBAAAA4Di2cuVKegQBAAAAIJb079+/xWOuB5W//fbbuvzyyzVmzBh985vf1E9+8hMdOHCgWczOnTt10003KS8vT3l5eZo3b54qKiqirxwAAAAAPK/Bg/9a56pH8J133tHs2bM1fPhw3Xbbbdq9e7eWL1+uDRs2aOXKlfL5fNq3b5+uuuoq1dXV6brrrlNDQ4Mee+wxbdq0SatWrVJCgt0P5QEAAAAAX56rhuB9992n/v3768knn5T/yOxb/fv314IFC/T666/rzDPP1BNPPKHS0lL94Q9/0ODBgyVJo0aN0jXXXKPnn39e06dP77hXAQAAAABwzXFo6KFDh3TiiSdq+vTpTY1AScrLOzxF96ZNmyRJa9asUV5eXlMjUJLGjx+vQYMGac2aNdZ1AwAAAIDHhNX1Q0GP/tf2EjSOPYK9evXSY4891uLxjz76SJKUkZGh/fv3a+fOnZo8eXKLuOHDh+vVV191ehoAAAAAQCc55llDd+3apXfffVeLFy9Wbm6uzjnnHG0/smpiWlpai/i+ffsqFAqpurpaffr0ib5iAAAAAEBUjqkhWFlZqYkTJ0qSEhMTNX/+fPXq1atp9tDExMQWf9OrVy9JUk1NDQ1BAAAAAMexBkkeWlFecW1uOaaGYFxcnB588EHV1dVpxYoVuuaaa/TAAw+ob9++jn/r87leqUKS5GaSUXcTkR7b87YrKckul6FWOmKPu+fMyDDN5sFMkhR0GeYc580j9TDL02jCBLtclvvfVYxLdpmkYcMMk6mfYSqXuY78Nr09lvtLspvt2vYaZsmbx71XLVlimc3l5xMXn5+eeirKUjqU3bUixSyTO24Pac8e+jk5Zqk6e9+jcxxTQ/CEE07QlClTJEnnnnuuzj//fC1atEi/+MUvJB2eWOaLGh/r3bv3MRVWV9f+9oQE5xhJSohv+weSx6y21i7XURPvRKtsr2Fj14W0NKmszDnOxfcDrpWW2uXKUIlZrhLLRmWgyjkoGJSqnONq4m3vSoaHq+lp9N57drkmjLbb/5Yvsspv9yGquNgslYal7rFLVlTkHJOXJ61d6xhWNdS5sehW0O/iJuNSSbldo9LyfEyJNzzuQ6HoC2qUnGyXy3CH3XyL3f12yc9cfD7x+aSwc9wVV9p+Dli61C5XSr3dtaIi3u56GO/wCdjtYS9J5eXR19MoPd0uV1LpVrNcFcnRNyp9PttTG9H70lcOv9+vs846S7t371a/I9/m7t27t0Xcnj17FAwGleTR3jQAAAAAsBH24L/WOTYEt2zZookTJ2rlypUtth04cEBxcXFKSEhQZmamPvzwwxYxGzdu1IgRI5yeBgAAAADQSRwbggMHDlR1dbWeeeYZ1R01FnPXrl166aWXdNpppykQCGjSpEl6++23tWXLlqaYt956S9u2bWsaTgoAAAAA6HqOvxGMj4/X/PnzNW/ePM2cOVMXXnih9u3bp5UrVyouLk7/+Z//KUm6/vrr9cILL+jqq6/W7NmzdejQIS1btkzDhw9XQUFBh78QAAAAAOha7Q/H7Hxt9/u5miymoKBAPXv21LJly7Rw4UIlJSXp9NNP19y5czVo0CBJUkpKip588kktXLhQS5Yskd/vV35+vubNm6cEd9N7AgAAAAA6getZQ6dMmeI4xDMnJ0e/+tWvoi4KAAAAANBxjmn5CAAAAABAWxrkraGhbS9uHxeJRNre2pV27ZIaGtrenpUl7djhmKYkPsuspIzQx2a5NtbnmuUa5rdbJ0apqc4xXbCWneW6WW6WLHMrJ2C4llog4ByTlCTV1DjHWS5qJGl9pd15NDKzwiyXKTcHmdv9X18ffT2NDNckrAnYrcGV5Le7yb34J+eVjKZMkV580TnXlOyNBhUdYbnopeHiWeFsu0WifRvWOweNHCmtdxE3dGj0BR1RVWv3k5KDB81Stfux5FjddptzzFNPSVdc4SLuSdsPnfNut1uX8LrrzFIpM9MuV1Klw5rCGRlSid26w26F0+3WJ/YVO39Ods1iQeeEBGn06OjzeFx29j5t3+6dhuDAgT4VFZ3Y6rbOXYkcAAAAANDlGBoKAAAAACa6z9BQegQBAAAAIMbQEAQAAACAGMPQUAAAAAAwEdbh4aHeR48gAAAAAMQYGoIAAAAAEGMYGgoAAAAAJsLy1qyhcW1uoUcQAAAAAGIMDUEAAAAAiDEMDQUAAAAAEw1i1lAAAAAAgCfREAQAAACAGMPQUAAAAAAw4bUF5Zk1FAAAAABwBA1BAAAAAIgxDA0FAAAAABNemzWUoaEAAAAAgCPiIpFIpKuLaE043P52n885RpJ8RVttCpKkeMMOVMNcVYEMs1yBgHOM232/eXP09TTKjTd8H9PT7XK9845drjPOcI5JSJDq6pzj6uujr+dolse+YW1l1UlmudIaSpyDMjKkEhdxlvs/OdksVZWCZrmKi81SKTPTOSYYlKqqXMTJRZBbhhex10KnmuUaPdoslYK1e5yD+vWT9jjHbQ31M6josNRUs1Ty++1y1dba5XJzmUhJkSoqnOMWLYq+nqPdt8jFTd6lm2+x63NY8iMXO8Otdeva3z5xovTKK65SVY2dGH09R5SXm6VSTqrd9XBrefT3j/h4KSvLoBiPy84u0vbtxp/DojBwYLyKirJb3cbQUAAAAAAwwdBQAAAAAIBH0RAEAAAAgBjD0FAAAAAAMBHR4UXlvaLt6WDoEQQAAACAGENDEAAAAABiDENDAQAAAMCE12YNbbvfjx5BAAAAAIgxNAQBAAAAIMYwNBQAAAAATDA0FAAAAADgUTQEAQAAACDGMDQUAAAAAEyE5a2hoT3a3EKPIAAAAADEGBqCAAAAABBjGBoKAAAAACa8Nmto27XQIwgAAAAAMYaGIAAAAADEmLhIJBLp6iJaVVYmNbTTrZqRIZWUOKapS80wK+m998xSaXz6Vrtk6el2uSornWNc7vuaZLt9X11tlkqbNtnlGjHCLldKfJVzUDAoVbmIKy2NvqCjVKXnmuUK+uvMcinebnR7RaXz92IpKVJFhXMuv9+goCOSNq+3S2Z5wFoeY/X1zjFZWdKOHY5hH9dmGRR0WG624bHq5jW6VFadZJYrMdE5xu1l5+DB6OtplNbTxYnmUlV8ilkuN7dIt7L8e5yD+vWT9jjHfVzZz6Cizy1dapdryc/CZrke+Jld/8XFF7e/3eUlR5KUnBx1OU0s75EfFyWY5cotfyv6JL16Sf/yL9Hn8bjs7P/V9u2G948oDRyYoKKiMa1uo0cQAAAAAGIMk8UAAAAAgAmvrSPYdq88PYIAAAAAEGNoCAIAAABAjGFoKAAAAACYYB1BAAAAAIBH0RAEAAAAgBjD0FAAAAAAMMGsoQAAAAAAj6IhCAAAAAAxhqGhAAAAAGCCWUMBAAAAAB5FQxAAAAAAYgxDQwEAAADARFjtzdTZ+Zg1FAAAAABwBA1BAAAAAIgxDA0FAAAAABMsKA8AAAAA8Ki4SCQS6eoiWlNZKYXb+Z1lSopUUeEuj5XMTLtcoZBdLsvX2Lu3c0xamlRW5hx36FD09TRKTrbLFQjY5fK9t9YuWXa2c0y/ftKePc5xljvMWEUowSxXStH7ZrlcneBu93+84WALv98s1SvvJJnlGj3aLJUrbq/5KfUu3h+X3trczyzX4MFmqdSnj12upMoS56CMDKnEOW5HfYZBRYdlldpdW8Nj88xy+TZ/bJarIjXXMcbtcW94mZAkJdW6eFKXHngixSzXrbfYTcAx6+r2+0KWL5dmzXKX6557DAo6orDQLteksXbvo2pro8/Ro8fhD5HHuezsv2v79oNdXUaTgQMTVVR0ZqvbGBoKAAAAACZYUB4AAAAA4FE0BAEAAAAgxjA0FAAAAABMMDQUAAAAAOBRNAQBAAAAIMYwNBQAAAAATETU3iLuna/tlQLpEQQAAACAGENDEAAAAABiDENDAQAAAMAEs4YCAAAAADyKhiAAAAAAxBiGhgIAAACACYaGAgAAAAA8ioYgAAAAAMQYhoYCAAAAgImwvDU0tO3F7ekRBAAAAIAYQ48gAAAAABzHdu/erYaG5j2VcZFIJNJF9bRv3Tqprq7t7Xl50tq1znn8frOSlJ5ulys52S7XunVmqSqG5DnGpKRIFRXOuVJeftagoiMuvtgsVVXIriM8GGi7u/1YuakrGJSqqpxzBetdvEHHwvI8Ki83S1WTmmWWK6nezY519waUHQwaVHRYYqJZKgXl4jW6VV9vlqoukOIYk5DQ/i2hKa50h0FFR1he8y2FQna53LyP/fpJe/bY5HLLcN9XVNpd8y0vhW52l+trfqgk+oKOVlholmrHkIlmuebPN0ul5U843L99Pins7h6fd7rdMbb2ebv3siY5wyxXdXX0OXw+qW/f6PN4XXb277R9+4GuLqPJwIG9VVR0kSZOnKhdu3Y120aPIAAAAAAcx1auXNmiR5CGIAAAAAAcx/r379/iMRqCAAAAAGAirPZm6ux8bdfiuiH49ttva8mSJSosLFQgENC5556rW265Rb17926Kufjii/XBBx+0+NvJkydryZIlx1g0AAAAAKAjuGoIvvPOO5o9e7aGDx+u2267Tbt379by5cu1YcMGrVy5Uj6fT5FIRFu2bFF+fr4mTZrU7O8HDBjQIcUDAAAAAI6dq4bgfffdp/79++vJJ5+U/8iUWf3799eCBQv0+uuv68wzz1RxcbFqamp09tlnq6CgoEOLBgAAAADvOY4WlD906JBOPPFETZ8+vakRKEl5eYeXGdi0aZMkafPmzZKkwYMHR1UqAAAAAKBjOfYI9urVS4899liLxz/66CNJUkbG4TVKPvnkE0mfNwRramqUlJRkVigAAAAAwMYxzxq6a9cuvfvuu1q8eLFyc3N1zjnnSDrcEOzdu7cWLlyoF198UTU1NfrqV7+quXPnaurUqeaFAwAAAIC3NMhbQ0PbriUuEolE3KaprKzUuHHjJEmJiYl69NFHm/7/oosu0saNGzV58mRdcMEFqqqq0vLly1VYWKjFixdr2rRp0b0GAAAAAPCw7OxntH17qKvLaDJwYEBFRZe1uu2YGoL79+/Xm2++qbq6Oq1YsUIfffSRHnjgAZ177rl6+umnFQ6HNWPGjKb42tpanX/++Tp48KBee+019ejRw33V69ZJdXVtb8/Lk9audc5z1O8ao5aebpcrOdku17p1ZqkqhuQ5xqSkSBUVzrlSXn7WoKIjLr7YLFVVyPGnsa4FA3brxLipKxiUqqqccwXrXbxBx8LyPCovN0tVk5plliup3s2OdfcGlB0MGlR0WGKiWSoF5eI1ulVfb5aqLpDiGJOQ0P4toSmudIdBRUdYXvMthQw/YLh5H/v1k/bsscnlluG+r6i0u+ZbXgrd7C7X1/xQSfQFHa2w0CzVjiETzXLNn2+WSsufcLh/+3xS2N09Pu90u2Ns7fN272VNcoZZrurq6HP4fFLfvtHn8bru1BA8piP3hBNO0JQpUzRt2jStXLlSGRkZWrRokSTp8ssvb9YIlCS/36+CggKVl5c3TSYDAAAAAMenBg/+a92X/grD7/frrLPO0u7du1XRTvdQSsrhb3pramq+7FMBAAAAAAw5NgS3bNmiiRMnauXKlS22HThwQHFxcTp48KCmTp2qpUuXtojZtm2bJCkzM9OgXAAAAABAtBwbggMHDlR1dbWeeeYZ1R31A41du3bppZde0mmnnaYBAwaourpaq1atUuio3y7s3r1bq1ev1rhx49Q3FgYFAwAAAIhhjQvKe+Vf2791dVw+Ij4+XvPnz9e8efM0c+ZMXXjhhdq3b59WrlypuLg4/ed//qck6a677tKcOXN02WWX6ZJLLtGBAwe0cuVKxcfH6+6773a33wAAAAAAHc7VOoIFBQXq2bOnli1bpoULFyopKUmnn3665s6dq0GDBkmS8vPz9fDDD+vRRx/V/fffL7/fr7y8PN16661Ni8wDAAAAALqe6wXlp0yZoilTprQbk5+fr/z8/KiLAgAAAIDuJ6z2hmN2viiGhnaZkSOdY8aOdQzZWmS3tkuO324NrnB8glkujXVe+8+tlDdecw6aMEEpG1zEnXVW1PU0KSoySxUsLjbL9ZommOVys2xWMCiVljrHHTzBeV22Y1Fmt8s0MjNglive8Aq2frPz2n8jR0rri1zEZdqt41gSsnsvg7V2aziW9c4xy5VW7mLdrIwMJbiJs1zorbLSLFVNoJ9ZrngX6y665WqfSu4Wvautja6Yo5Tttbt3799vlkq5Abs13rbWOq/xFgy6W3o1aHjYS1LVWLu1/5LNMkn33GOXy2ntv7Vr3a8PuPYduw/9y5+0W/vPsm8mI9XFQq6uGH7+RdTsrrQAAAAAgG7Buz2CAAAAANCttL+Ie+frgAXlAQAAAADdEw1BAAAAAIgxDA0FAAAAABONC8p7RduTGdEjCAAAAAAxhoYgAAAAAMQYhoYCAAAAgAlmDQUAAAAAeBQNQQAAAACIMQwNBQAAAAATzBoKAAAAAPAoGoIAAAAAEGMYGgoAAAAAJpg1FAAAAADgUTQEAQAAACDGMDQUAAAAAEyE1d5MnZ2PWUMBAAAAAEd4t0dw926poZ0fWmZlScXFjmmys7PsaiquNEvlCwTMclVU2rXnU0aPdhfoJi4UiqaUZuoyc8xyJfj9Zrkm1O8wyyVXx0SKclMrHKPCySnR13OUtLg9dslCtWapEpKTzXKN9G92EZWrkf6PncNCdsdYRrzd/qpItjuP0ko3muVSfb1zTEaGVF7uHJeZGX09jVzcY9zyj+hnlstXWmKWa2NlhmPMsAx3cdXVFhUdNs5v9xrT0u3utwqkm6VKd3lqp7t4yrDf+f05FuVFdrlyMuvMcr1TmGCWa+3zTsdYhouYw5Y/abf/Z11p15M0+zq7z4e/vr0o+iTx8VKO3X0I0fNuQxAAAAAAuhUWlAcAAAAAeBQNQQAAAACIMQwNBQAAAAATLCgPAAAAAPAoGoIAAAAAEGMYGgoAAAAAJhgaCgAAAADwKBqCAAAAABBjGBoKAAAAACYiam8R984XaXMLPYIAAAAAEGNoCAIAAABAjGFoKAAAAACYYNZQAAAAAIBH0RAEAAAAgBjD0FAAAAAAMMHQUAAAAACAR9EQBAAAAIAY49mhoXV9B7S7PUFSXXqWY57CDUYFScrMdH4+t0LFZqlUWWmXKzA06BiTIKnO7yJu3broCzqiuDbDLFdOaZFZLp1+ulmqHcXO38tkpUg7QimOcfWVBgUdJRTqZ5Zr5AjDRVZDIbNUG+tzHWOGuY2r32pQ0WFhw+tOSmWFWa7w0GFmufbudY5Jk1SWNtI5LrQj+oKOqBvq/HxuJZSWmOVaX253PQwE3MX5/c4xwzKroivmKCUhu9eYUbvHLFc44Hzvcyup1MV1IifHXVy87ce5nNRks1wfF9nts0lj7a5hNf72j7EkSTXJ7o7D/HyDgo6YfZ1dH82vl9ndb+9d5Hzvc3LCCdKcOQbFeF5Y3hoa2vZxQI8gAAAAAMQYGoIAAAAAEGM8OzQUAAAAALoXZg0FAAAAAHgUDUEAAAAAiDEMDQUAAAAAE2G1N1Nn52PWUAAAAADAETQEAQAAACDGMDQUAAAAAEywoDwAAAAAwKNoCAIAAABAjGFoKAAAAACYYEF5AAAAAIBH0RAEAAAAgBjD0FAAAAAAMMHQUAAAAACAR9EjCAAAAAAmWEcQAAAAAOBRnu0RfP55KRRqe/vs2dKTTzrnmX1lnVlNNfUJZrlSSjea5aoPDDPLVVrqHJOV5TJuxIjoCzoiJ77KLJfKDQ/78nKzVFm1lS6icpVV+7FjVNkJuVHXc7TMTLtcW4vsvn/KyQ6Y5XL7Gt3EVdTnRFfMUVLWvW+Wa338qWa5ktu5Ph+rrHQ31+kEpZ3oIq4+Nep6mp6xvsYsl2przVKNTN9jlsvVxVwjlRNa7xj1StHI6Os5YvRos1Qqqe1nliujnW/Wj1VFsvN1IsVt3Oa1BhV9bmt9llmu3PK3zHIpkG2Wqvqz9rcnJUnV1e5yZaTafdb89e1FZrkW3GP3WeCu+VbHPn1QXuLZhiAAAAAAdC9MFgMAAAAA8CgaggAAAAAQYxgaCgAAAAAmwmpvps7Ox6yhAAAAAIAjaAgCAAAAQIxhaCgAAAAAmGBBeQAAAACAR9EQBAAAAIAYw9BQAAAAADDBgvIAAAAAAI+iIQgAAAAAMYahoQAAAABggllDAQAAAAAeRUMQAAAAAGIMQ0MBAAAAwERY7Q3H7HwMDQUAAAAAHOHZHsGkJOeYQKDj6zhaXJxhsp49zVLFG76LPXoYxvkMv2ew3PkJCXa5LF+j2zfSRZxlWdYsj1dLbg8xN3Gm+9/weO3p8vx2w+21otOZXqgNWR74lgeY23uRizi/P8pajmL5Er16rLp9ja7iLO9rMr5O9+pll8vwzXSzX7vkXmq48084wSwVjkFm5le6uoRmGuvZvXu3GhqaT2ITF4lEIl1RFAAAAACgY9XW1mrChAnav39/s8dpCAIAAADAcaqqqkpVVVUtHqchCAAAAAAxxsO/JAIAAAAAdAQaggAAAAAQY2gIAgAAAECMoSEIAAAAADGGhiAAAAAAxBgaggAAAAAQY2gIAgAAAECMoSEIAAAAADEmvqsLOFY7d+7U4sWLtXbtWknSWWedpdtvv10pKSldXNnx7+KLL9YHH3zQ4vHJkydryZIlXVDR8W/+/Pnavn27VqxY0exxzoPO0db+51yw9/rrr+uRRx7Rhx9+KJ/Pp1GjRumWW27R6NGjm2I47juGm33PMd9x3n77bS1ZskSFhYUKBAI699xzdcstt6h3795NMRz7HcPNvufYx/GsWzUE9+3bp6uuukp1dXW67rrr1NDQoMcee0ybNm3SqlWrlJCQ0NUlHrcikYi2bNmi/Px8TZo0qdm2AQMGdFFVx7dVq1Zp1apVysvLa/Y450HnaGv/cy7YW7t2ra6//nqddNJJmjt3rurr6/XUU0/pyiuv1FNPPaWRI0dy3HcQN/ueY77jvPPOO5o9e7aGDx+u2267Tbt379by5cu1YcMGrVy5Uj6fj2O/g7jZ9xz7OO5FupEHHnggcsopp0Q2b97c9Nibb74Zyc3Njfz2t7/twsqOfzt27Ijk5uZGnnvuua4u5bhXX18feeihhyInn3xyJDc3N3LllVc228550LGc9j/ngr2CgoLIWWedFampqWl6bO/evZHTTjstcvXVV0ciEY77juJm33PMd5yLLroo8q1vfSty8ODBpseefPLJSG5ubuTVV1+NRCIc+x3Fzb7n2Mfxrlv9RnDNmjXKy8vT4MGDmx4bP368Bg0apDVr1nRhZce/zZs3S1KzfQ97hw4d0kUXXaSHHnpIBQUFSktLaxHDedBx3Ox/zgVb+/fvV2Fhoc4991wlJiY2PZ6amqrTTjtN//u//yuJ474juN33HPMd49ChQzrxxBM1ffp0+f3+pscbRyFs2rRJEsd+R3C77zn2cbzrNkND9+/fr507d2ry5Mkttg0fPlyvvvpq5xcVQz755BNJn18Ma2pqlJSU1JUlHZcOHTqkUCikBx98UFOmTNHEiRObbec86FhO+1/iXLAWCAT0pz/9qVlDpNG+ffvUo0cPjvsO4mbfSxzzHaVXr1567LHHWjz+0UcfSZIyMjI49juIm30vcezj+NdtegTLysokqdVv6Pv27atQKKTq6urOLitmfPLJJ+rdu7cWLlyoMWPGaMyYMcrPz+fbSGOBQEAvvfSSpkyZ0up2zoOO5bT/Jc4Faz169FB2dnaLY7qwsFDvv/++xowZw3HfQdzse4ljvrPs2rVLq1ev1k9+8hPl5ubqnHPO4djvJK3te4ljH8e/btMjeODAAUlq9ZvLXr16STr8TU2fPn06ta5YsXnzZh04cEDV1dW67777VFVVpeXLl+vWW2/VZ599pmnTpnV1iccFn88nn6/t72c4DzqW0/6XOBc6w4EDB/TDH/5QknTDDTdw3HeiL+57iWO+M1RWVjaNQEhMTNT8+fPVq1cvjv1O0Na+lzj2cfzrNg3BcDjsGOP0AQ5f3vTp0xUOhzVjxoymx6ZOnarzzz9fP/3pT3XBBRc0DSNCx+E86HqcCx3r4MGDuvHGG1VYWKjvfve7ysvL0z/+8Q/Hv+O4j15r+17imO8McXFxevDBB1VXV6cVK1bommuu0QMPPKC+ffs6/i3HfnTa2vfnnnsuxz6Oe93m6tG4psuhQ4dabGt87Oh1X2Dr8ssvb3YhlCS/36+CggKVl5c3/aAaHYvzoOtxLnScqqoqzZ49W++++66+853vaO7cuZI47jtDW/te4pjvDCeccIKmTJmiadOmaeXKlcrIyNCiRYs49jtBW/te4tjH8a/bNAQbf7i7d+/eFtv27NmjYDDID3i7QONitjU1NV1cSWzgPPAuzoXofPrpp5o1a5bef/99XXrppfrJT36iuLg4SRz3Ha29fd8ejvmO4ff7ddZZZ2n37t3q16+fJI79znL0vq+oqGgzjmMfx4tu0xAMBoPKzMzUhx9+2GLbxo0bNWLEiC6oKjaUlZVp6tSpWrp0aYtt27ZtkyRlZmZ2dlkxifOga3EudIxQKKRrr71WH330ka6++motWLCgWUOE477jOO17jvmOs2XLFk2cOFErV65sse3AgQOKi4tTQkICx34HcLPvDx48yLGP4163aQhK0qRJk/T2229ry5YtTY+99dZb2rZtW7uz/CE6aWlpqq6u1qpVqxQKhZoe3717t1avXq1x48a5+h0DbHAedB3OhY6xYMECffTRR5o1a5buuOOOVmM47juG077nmO84AwcOVHV1tZ555hnV1dU1Pb5r1y699NJLOu200xQIBDj2O4CbfT9gwACOfRz34iKRSKSri3CroqJC559/vnr06KHZs2fr0KFDWrZsmbKysvTMM88oISGhq0s8br388suaM2eOTjrpJF1yySU6cOCAVq5cqc8++0xPP/00i612kIkTJ2rAgAFasWJF02OcB52ntf3PuWBry5YtmjJlivr06aM777yz1YkXCgoKOO47gNt9zzHfcV544QXNmzdPo0eP1oUXXqh9+/Y17dunnnpKubm5HPsdxM2+59jH8a5bNQQlaevWrVq4cKHee+89+f1+nXnmmZo3b17TeG10nJdfflmPPvqoCgsL5ff7lZeXp1tvvZULYQdqrSEicR50lrb2P+eCnaefflo/+tGP2o3ZtGmTJI57a8ey7znmO86LL76oZcuW6eOPP1ZSUpJOP/10zZ07V4MGDWqK4djvGG72Pcc+jmfdriEIAAAAAIhOt/qNIAAAAAAgejQEAQAAACDG0BAEAAAAgBhDQxAAAAAAYgwNQQAAAACIMTQEAQAAACDG0BAEAAAAgBhDQxAAAAAAYgwNQQAAAACIMf8fGtSTvKcG+dUAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_166_1.png" } }, "output_type": "display_data" } ], "source": [ "fig = plt.figure(figsize=(20, 14))\n", "im = plt.imshow(J_sk, **cmap_args)\n", "plt.title(\"LinearRegression from Scikit-learn\", fontsize=18)\n", "plt.xticks(fontsize=18)\n", "plt.yticks(fontsize=18)\n", "cb = fig.colorbar(im)\n", "cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The results agree perfectly with our previous discussion where we used our own code.\n", "\n", "\n", "Having explored the ordinary least squares we move on to ridge\n", "regression. In ridge regression we include a **regularizer**. This\n", "involves a new cost function which leads to a new estimate for the\n", "weights $\\boldsymbol{\\beta}$. This results in a penalized regression problem. The\n", "cost function is given by" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "6\n", "0\n", " \n", "<\n", "<\n", "<\n", "!\n", "!\n", "M\n", "A\n", "T\n", "H\n", "_\n", "B\n", "L\n", "O\n", "C\n", "K" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/438060758.py:10: UserWarning: FixedFormatter should only be used together with FixedLocator\n", " cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAM2CAYAAACjUj0CAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAB4rUlEQVR4nO3de3yU5Zn/8W8mYRiGIcZsIBDSGIFGBBrB1Yisi4oRXVCDrSCKWkVd64/uVlzraT2t61mrLerPusVDkeABl1Urbdd1u1ZbD/xcSylyEhBCgARiCCGEEJKZ3x+QQMzhuXGuJE+Yz/v14uXLea5cc+WZ+3lm7rnv3HdSLBaLCQAAAACQMALdXQAAAAAAoGvREQQAAACABENHEAAAAAASDB1BAAAAAEgwdAQBAAAAIMHQEQQAAACABENHEADacOutt+q4445r9S8/P19nnnmmbr/9dlVUVLT4mQkTJujyyy/3zO0aZ+3FF1/Uaaedpvz8fD322GNd/vwu6uvr9W//9m+64IILNHr0aJ144on67ne/q3/7t3/T3r17Dztf0+vY5Mknn9Rxxx2n0tLSdn9m0aJFOu644/TJJ5+0eHzTpk2ez1daWqrjjjtOTz755GHXCgBAV0rp7gIAwM9uu+02HX300c3/X1NTo48++kj//u//ruXLl+v1119XMBiUJN1+++3q06dPd5XaodWrV+vBBx/U6NGj9aMf/UjDhw/v7pJaaWho0NVXX62lS5dqypQpuvjii9XY2KhPP/1Ujz/+uH73u99p3rx5zefbxcUXX6xTTz31sOo4+eST9cgjj2jo0KHNj1199dXq37+/HnroocPKBQCAX9ERBIAOFBYWKjs7u8VjM2bM0D333KOXX35Z7777riZNmtQc61dr1qyRJF133XWaMGFCN1fTtt/85jdasmSJnnzySU2cOLH58SuuuEJz587Vo48+qtdff12XXnqpc84xY8ZozJgxh1XHt771LX3rW99q8dgf/vAHXXjhhYeVBwAAP2NqKAB8A02dgj//+c/dXImbffv2SZL69u3bzZW0709/+pMk6W/+5m9aHZsxY4Z69eqlpUuXdnFVAAAcmegIAsA30DQFNBaLNT/W1t/+/frXv1ZRUZHy8/N13nnn6eOPP24z3+9//3tNnTpVo0eP1llnnaX58+frn//5n1uN3q1du1azZs3SSSedpBNOOEHTp0/XBx980GGtl19+uW677TZJ+0fXmv5m7vLLL9fVV1+tJ554QmPGjNGpp56q1atXS9o/lfT//J//o5NOOkn5+fmaNm2a3n333VZ5r7vuOr377ru64IIL9J3vfEeTJ0/W73//e9XU1Oiuu+5SQUGBTj31VN11112qq6vrsM6mTuqrr77a6lifPn302Wef6ZFHHmnx+Lp16/SjH/1Ip5xyiv76r/9al19+uT799NPm41//G8G23HPPPTruuOP04osvSmr5N4JNf/MnSf/xH//R5t8Ouli0aJGmTJmi73znOxo7dqxuvfVWbdu2rUVMTU2NfvKTn+jcc8/Vd77zHY0ZM0bTpk3Tf//3fzfHNNXz4osv6pJLLtGoUaN05ZVXNj/+xhtv6IknntD48eP1ne98R1OnTm23zQEAEhtTQwHgG2jqfI0YMaLdmEWLFum2227TmDFj9OMf/1gbN27UD37wA0WjUQ0ePLg57n/+5380a9Ys5eXlafbs2SovL9fDDz+scDjcYgRv9erVuvTSS5WRkaHrrrtOvXr10ttvv62///u/109+8pPmKapf94Mf/EDHHnusXn31Vf3gBz/QkCFDmo999tln2rhxo3784x+rtLRUw4YN07Jly3TFFVcoEonoqquuUt++ffXmm29q1qxZuuuuuzRjxozmn//888/1pz/9SVdccYX69eunZ599VjfccIOOP/549enTR7Nnz9ann36qV199VQMGDNAPf/jDds/XBRdcoBdeeEEPP/ywFi1apMLCQp166qkaM2aMgsFgq78N3LBhg6ZNm6aUlBRddtllSk9P1yuvvKKrrrpKxcXFys/Pb/e5mvz0pz/Vyy+/rNmzZ+vKK69sdTw9PV2PPPKIbr75Zp100kmaNm1ai78ddPHUU0/pySef1DnnnKNp06apvLxc8+fP15IlS/T6668rPT1dsVhM1113nVasWKHLLrtMOTk5Kisr0yuvvKJ/+Id/0H/+53+2mK76s5/9TKeffrrOP/989e7du8Xjffr00cyZM7Vv3z49//zzuu666/Tee++1+FtXAAAUAwC0csstt8Ty8vJin3/+eeyrr75q/rdx48bY/PnzY6NHj4793d/9Xay+vr75Z84888zYZZddFovFYrGGhobYqaeeGvve977XIubf//3fY3l5ec1xsVgsVlhYGJs4cWJsz549zY/913/9VywvLy925plnNj922WWXxQoLC2O7d+9ufmzfvn2xSy+9NDZu3LjY3r172/19mp73448/bpHv64/FYrHY1KlTY6NHj45t3bq1+bG6urrYhRdeGMvPz4999dVXLX7+d7/7XXPc/PnzY3l5ebFp06Y1PxaNRmPjx4+PXXzxxe3W1+R//ud/YqeeemosLy+v+d/o0aNjN954Y2z9+vUtYn/0ox/F8vPzYxs2bGh+rLKyMvbXf/3XsX/8x3+MxWIHX8cmc+bMieXl5cU2bdoU++UvfxnLy8uLPf74457nKi8vL3bLLbd41r9p06ZYXl5ebM6cObFYLBYrKSmJDR8+PPbYY4+1iFu9enVs5MiRsfvvvz8Wi8ViS5cujeXl5cVefvnlFnHvv/9+LC8vL/b888+3yH/22We3aFdNj59++ukt2sfixYtjeXl5sVdffdWzdgBAYmFqKAB04MILL9Spp57a/O/ss8/Wo48+qjPPPFPFxcXq1atXmz/3+eef66uvvtJ3v/vdFjFFRUU66qijmv9/1apVKikp0fTp0xUKhZofLywsbDHytGPHDi1ZskSnn3666urqVFlZqcrKSlVXV+vss89WRUWF/vKXvxz27xcKhXTyySc3/39FRYX+/Oc/q6ioSAMHDmx+vHfv3rr66qtVV1enDz/8sMXjf/u3f9v8/8cee6wk6ayzzmp+LCkpSYMHD9b27ds96znjjDP0P//zP3riiSdUVFSk/v37q7a2Vm+//baKioq0ZMkSSVI0GtXvf/97nX766TrmmGOaf/7oo4/WggULdMcdd3T4PG+99ZYeeOABffe739Xs2bM96/qm/uu//kvRaFQTJkxofs0qKyuVkZGh448/Xu+9954k6YQTTtD/+3//T9/97nebf7axsVHRaFSStHv37hZ5x44d22bbO/300xUOh5v/v2l1WJdzDwBILEwNBYAOPProo8rIyNC+ffv0wQcfqLi4WH/3d3+ne+65p8WUvK/bvHmzJCknJ6fF48nJyS06Lhs3bpSkFo81OfbYY7Vy5UpJB/ewe+mll/TSSy+1+Zxbt249jN9sv7S0NAUCB78TbKq7qUN3qKaO6ZYtW1r8fErKwbeS5ORkSdJf/dVftfjZ5OTkFn9P2ZHevXtr0qRJzVNdP//8cz3//PN6++23dffdd+s3v/mNqqqqVFtb2+Z5y8vL83yOn/3sZwoEAlq2bJn27dvXboe+PVVVVc0L8DRJT09vFVdSUiJJmj59ept5Dn3elJQUvfLKK1qyZIk2btyokpKS5r+r/Pq5a+u52nq8aTptU4cSAIAmdAQBoAMnnnhi8/YRTaNP9913n6qqqvR//+//VVJSUps/1/R4W5ugH/qhvKGhQZLa3Bvv0I5mY2OjpP2rZ7a3TcWwYcNcfqUWmjpuTTrqrDXV/fXOS1vaOy/tqa2t1bPPPquRI0e22DpCkkaOHKmf/OQnqq6u1vvvv68dO3Y0n49DO7GH44ILLtDJJ5+sO++8U88995x+8IMfHNbP/8M//EPz6GSTQxd1adJ0zp555pkWI75fV11drenTp2vTpk36m7/5G02YMEHDhw/X4MGDNXXq1FbxX3/dmnzT8wEASDx0BAHgMFx++eX66KOP9N///d/65S9/2eYCI5KaF/bYsGFDi8djsZg2b96sb3/7263iTjvttBaxh/5s0+IyycnJGjduXIu4tWvXqrS01GQz+6bnWb9+fatjX375pSS1mDJqpXfv3nruuec0ZsyYVh3BJsOGDdMHH3ygUCikXr16KRQKNY+oHuq5555TRUWFbrnllnaf70c/+pEGDx6sRYsW6ZlnntHkyZNb7R3YkVtuuUXV1dUtHuvfv3+rKZhN53PQoEE6/vjjWxz7/e9/r0gkIkmaN2+e1q1bpxdffFGnnnpqc8xnn33mXBMAAIeDrw4B4DDde++9Ouqoo/TTn/60ecrm140YMUKDBw/Wyy+/rD179jQ/vnjxYu3YsaP5/0eNGqVBgwbp9ddfV319ffPjS5cu1YoVK5r/f8CAARo1apT+4z/+Q+Xl5c2P79u3T7fffrv+8R//sXl0MR79+/fXqFGj9NZbb6msrKz58fr6er3wwgsKBoNt7vMXr+TkZE2aNElLlizRm2++2ep4VVWV/vM//1Pjxo1Tnz59lJKSor/5m7/R73//+xZTYnfu3KnnnnuueUpmR5KSknTXXXdp3759uueeezqMDQQCLUZyR40apXHjxrX419ZU4TPPPFOS9Oyzz7YYbV25cqWuv/56/fKXv2z+/aSWo7qxWEzz58+XJJPXFgCAQzEiCACHKSMjQzfddJPuvPNO3X333Xr++edbxSQlJenOO+/UrFmzdPHFF+t73/ueysvLVVxcrLS0tOa4QCCgW2+9VTfccIOmT5+uoqIiVVZWat68ea2mi95xxx36/ve/r+9973u65JJLlJaWpsWLF+vPf/6z/umf/slse4Cm57nooot0ySWXqG/fvnrrrbf0+eef64477lBqaqrJ83zdrbfeqmXLlunmm2/WW2+9pb/9279VJBJRSUmJFi1apH379umuu+5qjv+nf/onTZ06VVOnTtWMGTMUiUT02muvqba2VjfccIPTc44YMULTp09XcXGxFi9erMmTJ7cZl56eriVLlui1117TaaedpqysLKf8eXl5uvzyy/XSSy+pqqpKhYWFqqqq0vz589W3b1/96Ec/kiSNHz9eL730kq677jpddNFF2rdvn37zm99o+fLlCgQCrRaLAQAgXowIAsA3MHXqVP31X/+1/vjHP+qNN95oM+bMM8/Us88+q1AopMcff1zvvvuu7r///hb7+EnSueeeqyeeeEINDQ169NFH9fbbb+u2227TqFGjWnQGx4wZo5dfflmjRo3SCy+8oEcffVR79uzRQw89pL//+783+92anmfkyJF6/vnn9bOf/Uy9e/fW008/rcsvv9zseb4uPT1dixYt0o9+9CNVVVXp6aef1j333KO3335bEydO1K9+9Svl5uY2xw8dOlSvvvqqvvOd72ju3LmaM2eOBgwYoAULFjRPvXVxww036K/+6q/04IMPtpru2eSmm25SQ0OD/vVf/7XV3wZ6+ed//mfdfffdqqys1MMPP6wFCxbopJNO0oIFC5oX4Bk/frzuu+++5tdz7ty5SktL06uvvqrjjz/+G21iDwBAR5Jirsu4AQDMNTY2aufOnW2uAnn++ecrNTVVxcXF3VAZAAA4kjEiCADdqLGxUePHj28x5VGS1qxZoy+++EL5+fndVBkAADiS8TeCANCNgsGgzj33XL3++utKSkrSqFGjtG3bNr388ss6+uijddVVV3V3iQAA4AjE1FAA6GZ1dXV67rnn9NZbb2nr1q3q16+fTj31VN1www3NexgCAABYoiMIAAAAAEeo6urqNhdDoyMIAAAAAHHaU1mpPm0s/tbd6urqNH78eO3cubPF4/7tCB6ycW+bAgHvGEn/er/dejh33lLvHeRoT2PQO8hRZaVZKrWzcnoLxx8vrVzpEDfU7nxZKt9hd+4zGzab5VJjo3dMTo7ksFH2+oYcg4IOGtK33DvIUVXvTLNcaWG7Nvb+x97tYvx46f33vXMd2BHAxOBG79fb1a6j7dpFvxV22xmU557iGZOZKZU7NMPM1D0GFe23q6GPWa429pr/xoIrlprl2nPcaM+YPn2kPQ6ntU9v7/fk7rB5q93ngP79zVI5CQaleofbXHCH3T1akuqPtrtPW9Zm+v6xYWnHAaNHS0s9YppYLuq1datZqvr+g81ytbNL0mEJh6Xzzos/j989f9ppqi4t7e4ymqVmZ2vmH/6grVu3qvFrnzWP+MViduzo7graZtn9duk/uNq3zzbOjyzPl2myhgazONdUzgx/T4fvb7pFXZ1dnGkbM3wxTb/227vXLJXr+XKKM/wlffo1qVvPwJHr7+jbc+HA9Hr0Kz//kn59/3C5jgyvNWc+fS1rarq7gp6jurRUOzdu7O4yWhk0aFCrx474jiAAAAAAdIWA/LU/X0e1mNe5adMm/fCHP1RBQYEKCgp08803q9Jy7iIAAAAAIC6mI4I7duzQ97//fdXX1+uaa65RY2OjnnvuOa1evVoLFy5UMGj3t1kAAAAAgG/GtCP44osvqqysTL/61a809MBqCSeccIKuuuoqvfHGG5o2bZrl0wEAAACAbyTs1NDFixeroKCguRMoSePGjdOxxx6rxYsXWz4VAAAAAOAbMusI7ty5U5s2bdLIkSNbHRs5cqSWL19u9VQAAAAAgDiYTQ0tP7DBU2Zm6z1e+vfvr5qaGu3atUv9+vWzekoAAAAA8I2eNDXUrCO4e/duSVKfPq034O19YCfd2tpa945gwOEUOsQ8/rjb07mxW+wmbLhuTo7h3uGuudz2TvXn4kBZWZbZbDdudzJkiGdInvmT2p20dLNMkmUbmzjRNs6O9+vtKtUsk6Tx481SubYut2s3HEclLZmeL0sFBWapXM9W2CnQTx99DrJ8j+wOTuvs2b6x2b57G9Zm+v7hch0ZXmvODBus5es4c6ZhMviGWUcw6rDLZ8Clc3cwoVcyp51Fb7zJ7o3p8YfsNhatbbC7PCsqzFKpqso7Jj9fWrbMIW54N2zE6mBLhd25z2ooMcvltHH4kCHS+vWeYWsa7DoPkpQX2WKWqzJk+KEgYtfG3nnPu11MnCi98453ruHDDQo6IKfB+/V2VZ1h2Klc+r5Zri3DvDuVWVnSFodmmJVWa1DRftUNdp3KUMgslYJLl5jlqh3l/UE3HJZqHU5rOGS527edklK7zwEDB5qlchIMuu1pHqywu0dLUn2G3X3asjbT94+1HtdRQYG0xPFaO+mk+AtqUlpqlqp+oF2ncv78+HNEIhLrRvqLWUewb9++kqS9e/e2Otb0WFMMAAAAABxpkuSv+RFJHRwzqzPrwND/9u3bWx3btm2bUlNTFXabWwIAAAAA6ERmHcHU1FRlZ2fr888/b3VsxYoVGjVqlNVTAQAAAADiYDpyOXHiRH300Udat25d82MffvihvvzyS02aNMnyqQAAAADAVwI+/Nces78RlKRrr71Wb775pq688krNnDlTe/fu1dy5czVy5EgVFRVZPhUAAAAA4BsyHRFMT0/X/PnzNXz4cM2ZM0e//OUvVVhYqLlz5yrotP4xAAAAAKCzmY4IStKQIUP0i1/8Iu48/3p/QDt2tH/88cfdtoZ4/DG75axvvtWuM3vrrWaplGL4KubLYV8I5TvFVdc5bTboxGVbC1c5pR/aJRs71ixVbZ13ew5Lqh3osI9gzTaDig6qjRgu2W24xPyWMrtr0vWldInbsCGuUlrIGZVrliu1wW67jZJcu30Ec5y2Yclx264lxW59/9Qyu607LJeE1+jRZqmWLvWOGTfOMW54VZzVHMJwv42ctR+b5VpTN8EsV94wl3thQMEU77iSBtt9BHNK7dp+NNdu25qqDWaplO7y4cnxA9b6DXbjKrm5dls+rF1llkozr7R67/bTepqdoydtKO+nOgEAAAAAXYCOIAAAAAAkGPOpoQAAAACQiJgaCgAAAADwLTqCAAAAAJBgmBoKAAAAAAaS5K+RtqQOjvmpTgAAAABAF6AjCAAAAAAJhqmhAAAAAGCAVUMBAAAAAL5FRxAAAAAAEgxTQwEAAADAQJI6Xqmzq7FqKAAAAACgGR1BAAAAAEgwTA0FAAAAAAPJB/75RUe1MCIIAAAAAAmGjiAAAAAAJBimhgIAAACAgST5a6Sto1VDk2KxWKzLKjkc9fUdHw8GvWMk3XxH0Kgg6ZGHoma5rp9l10Tuv98slSIR7xjHU69g1bb4C2pSVWWWqj43zyxXMMWuTUQdbhuBgBR1eMoNG+Kv51BDGtaY5VpWZ3f+8+uWmOXSwIHeMTk5UkmJZ9iCP+QYFLTfpVNqzXLVp4TNctXVmaVS6oZl3kH5+dIyh7iMjPgL6gTVkSyzXKkRu/uOamocnjBVqq72DFtRmmpQkL3hw+1yBWR47l0uonBYqnW4B5SWxl/PoUIhs1QfltrdD086ySyV92eUAQOkbY6fY1IMx1VcrklH5b3tzn2/fvHnSEqS+vSJP4/fFefmqmbjxu4uo1nkmGM0o50Phn7qsAIAAAAAugBTQwEAAADAQED+GmnrqBY/1QkAAAAA6AJ0BAEAAAAgwTA1FAAAAAAMMDUUAAAAAOBbdAQBAAAAIMEwNRQAAAAADDA1FAAAAADgW3QEAQAAACDBMDUUAAAAAAwkyV8jbUkdHPNTnQAAAACALkBHEAAAAAASDFNDAQAAAMAAq4YCAAAAAHyLjiAAAAAAJBimhgIAAACAgSR1vFJnV2PVUAAAAABAs6RYLBbr7iLasmeP1FFl4bBUW+udp67OrqZ//me7XM88HTXL9cBDdv3526/Z5h00YIC0zTvuw7UDDCrab9xohxfb1W9/a5bqs9zvmuU6sWGJd1BBgbTEOy56UoFBRQcF1q4xy1Wfm2eWK1ixxSzXsoosz5j8fGnZMu9c+aPsru+o4fd1DQ1mqRRMsfsdt5R5/45ZWdIWh5c7K1JtUNF+1Uo1y5Uqu7qiEbu6XAQCUtTh5Q401Js954q1QbNcI4bbtVXTDxUpDpOygkGp3vu8Wp4vSRqRZndvtTxn1RlDzHKlRjzahWvDt2Z4oy4ps2sXOTUr4k/Sq5f07W/Hn8fn/iM3V7s3buzuMpr1PeYYXbhhQ5vHmBoKAAAAAAaSD/zzi45qYWooAAAAACQYOoIAAAAAkGCYGgoAAAAABpLkr5E2Vg0FAAAAADSjIwgAAAAACYapoQAAAABgICB/jbR1VIuf6gQAAAAAdAE6ggAAAACQYJgaCgAAAAAGmBoKAAAAAPAtOoIAAAAAkGCYGgoAAAAABpgaCgAAAADwLTqCAAAAAJBgmBoKAAAAAAaS5K+RtqQOjvmpTgAAAABAF6AjCAAAAAAJxrdTQysrpcbG9o/n5EgVFd55Ugx/w/vvt8v1wEN2ffDbb42a5br0sgGeMQsWSJfe4BD3020WJUmStlR5P5+rquHfNcsVMsskKXeUW9wo77hAVWWcxXSe4Mfvm+WqHzveLFd+Q4lDVI7y07zjosqJv6ADamrMUqmhwS5XesQuWVpa0DHOISjF7qpMras2y6WqKrNUf1iaapbrtNPMUtk2MLm1ia72/qdhs1zDhnnHZGVJWyq8z8WI0HqDig6qjgwxy5WaUW+Wq2yDWSqlVn3acUBBgfSpR8wB0ZMKDCrar6rGru1bvn+sD42IO0dKigzfHf2LVUMBAAAAAL5FRxAAAAAAEoxvp4YCAAAAQE+SpI5X6uxqrBoKAAAAAGhGRxAAAAAAEgxTQwEAAADAQPKBf37RUS2MCAIAAABAgmFEEAAAAAAMJMlfI20sFgMAAAAAaEZHEAAAAAASDFNDAQAAAMBAQP4aaeuoFj/VCQAAAADoAnQEAQAAACDBMDUUAAAAAAwwNRQAAAAA4Ft0BAEAAAAgwTA1FAAAAAAMMDUUAAAAAOBbvh0RrK6W9u1r/3hOjlRV5Z0nX8vMaqofnm+W6/ZrtpnluvSyAWa5FsyPOkQFnOJuvMmurltuMUulEcNdfkdHdXVmqdaXhT1jhgxxiyst9Y45HONH2eWqz80zy1VRYZZKDcrxjMmRVOISV2F3fUcy7K6jDRvMUik9xa7tl5YFPWPy8qTSUu9ceRk1BhXtV52SbparSqlmuVIM37mXLvWOOfFEt7iMDLv7zsCBZqlUXWP3nXdamlkqJScbxlkWJikSMUxWWmaW6qijvO+/riozCjo8ni6pcljHMc2xf3jfoKIDuUaPNssVyrW775TZvYzoJlu3blVjY2OLx3zbEQQAAACAniRJ/ppymXTgvzNmzNDmzZtbHKMjCAAAAABHsOLiYkYEAQAAACCRDBo0qNVjdAQBAAAAwEBPWjXUvCN40UUX6S9/+Uurx8855xzNmTPH+ukAAAAAAIfJtCMYi8W0bt06FRYWauLEiS2ODR482PKpAAAAAADfkGlHsLS0VLW1tTrrrLNUVFRkmRoAAAAAfC1JB1fq9IOOajGdwrp27VpJ0tChQy3TAgAAAAAMmXYEv/jiC0kHO4K1tbWW6QEAAAAABsw7gn379tWDDz6oMWPGaMyYMSosLNTixYstnwYAAAAAfCfZh//akxSLxWImv7WkCy+8UCtWrNA555yj888/X9XV1Zo3b55WrVqlhx9+WFOmTLF6KgAAAADwlc9yc7V348buLqNZ72OO0YkbNrR5zLQj+PLLLysajWrGjBnNj9XV1em8887Tnj179P777ys5uaN+6UErV0r79rV/PD9fWrbMO0++HIIc1Q/PN8sVrNpmluvSGwaY5VowP+odFAhIUe+4G2+yG3C+5RazVMrs7/A7uqqrM0u1vizsGTNkiLR+vXeu0lKDgg4xflSlWa76SLpZrooKs1RqaPCOycmRSkoc4kJ213c0w+76bud94BsZklFtlmtNWapnTF6etGaNd668DLu2Wp1i11arqsxSmV7foZB3zIknSp995h2XkRF/PU0iEbtcKYbL4lleQ5mZbjHl5Q5xvezavSRF0+zafqDU4abpqLx3jlmuXr06Pp6eLlU6ntb05e/HX1CT0aPNUtWmeN9bXZWVxZ8jJWX/++iRrid1BE2nhl5yySUtOoGSFAqFVFRUpIqKiubFZAAAAADgSJOkg5vK++Ffl60a2p709P3fLLF4DAAAAAB0P7OOYHl5uSZPnqynnnqq1bEvv/xSkpSdnW31dAAAAACAb8isI5iZmaldu3Zp4cKFqqmpaX5869atWrRokU455RT179/f6ukAAAAAwFe6eypoW//aY/gn1NJdd92lWbNmafr06Zo6dap2796t4uJipaSk6O6777Z8KgAAAADAN2T6N4KFhYV6+umn1adPHz322GN64YUXNHr0aL388svNm8wDAAAAALqX6fYRpurrOz4eDHrHSKquCxoVJKXW2S0J/+FauyXhxw2zq+vGh7zrevxx6cYbvXM9/pjdNg0332r3ncU//ZNZKs/lpw9HepXDvhCO+0esaRhiUNFBAwfa5UoNeV+3zlz2fHDlsr5/Vpa0ZYt33KpVcZfTpPqkCWa5UqvslnG3bBT18r5PO97yFZRP25fhXifRbLv11wOy2zLI8ny5tAlXO3aYpVJmP7tF76Ih7y2DXE/9IX+RY8Jiq4Amubl2uSy3DMr6w2sdB0ybJr3mEdPkjDPirqdTGL4XmWxrkZQk9esXfx6f+0turup9tH1E8Jhj9J2u2D4CAAAAAOB/dAQBAAAAIMGYLhYDAAAAAInKa6XOrtZRLX6qEwAAAADQBegIAgAAAECCYWooAAAAABhIkr9G2pI6OOanOgEAAAAAXYCOIAAAAAAkGKaGAgAAAIABVg0FAAAAAPgWHUEAAAAASDBMDQUAAAAAA0nqeKXOrsaqoQAAAACAZnQEAQAAACDBMDUUAAAAAAwkS4p2dxGHSO7gGCOCAAAAAJBg6AgCAAAAQIJhaigAAAAAGEiSv0baOlo1NCkWi8W6rJLDUV/f8fFg0DtGUklZ0KggKadujVkuZWebpdpSFTbLldzRROIDMjOl8nLvuJ/8JP56mjzykN1s63+8we7ynD7dLJUGDvSOGTJEWr/eOy43N+5yWqipscuVWrfNLlkkYpcrxeF7Mcf7zrJVdvcdw1uF0kO1ZrlKKuzuOznZDtd3ICBFHeI+/jj+gpoMG2aXy7Ctlu+yO/effOIdc8EF0ltvOcSdUR1/QU0aGsxSldSkm+XKSbP7Hcv3pHrGuL7f9u9vUNAhAhsc3mgclfcdYparTx+zVJ6XpOstR5ICpSXxF3RAbUaOWa6yMrNUGpJh0PaTkqR+/eLP43PrcnPVsHFjd5fRLOWYYzR0w4Y2j/mpwwoAAAAA6AJMDQUAAAAAAwH5a6Sto1r8VCcAAAAAoAvQEQQAAACABMPUUAAAAAAwwNRQAAAAAIBv0REEAAAAgATD1FAAAAAAMMDUUAAAAACAb9ERBAAAAIAEw9RQAAAAADCQJH+NtCV1cMxPdQIAAAAAugAdQQAAAABIMEwNBQAAAAADAUmx7i7iEKwaCgAAAABoRkcQAAAAABIMU0MBAAAAwECSOl6ps6uxaigAAAAAoFlSLBbz098zNisvlxob2z+elSVt2eKdJ2vDh2Y11Z80zixX8O1FZrlWDP+uWa4Rw6PeQYGAFPWOK99u9z3D/febpdKcnzr8jo6mTbf7HV97xe7cR42/4ykttcuVYjgPIStSbZesqso7JidHKinxjsvOjrucJtU1/vy+LjVidx2VlHr/jq6nPhIxKOiAtDS7XDU1drlSGyrtkrmcsGBQqq/3DKttCBoUtF+4zu53XFORbpYrL+LwwcOS44edkoYs06fNGej9ert6/2O7dnHaaWapPK/J1FSp2vEtJvXT38VfUJPhw+1ybdhgl8vizTsYlEaPjj+Pz23NzVXjxo3dXUaz5GOO0aB22gJTQwEAAADAQHJ3F/A1HdXjz6+aAQAAAACdho4gAAAAACQYpoYCAAAAgIEk+WukjVVDAQAAAADN6AgCAAAAQIJhaigAAAAAGPDbKFtH9fitVgAAAABAJ6MjCAAAAAAJhqmhAAAAAGDAb6NsTA0FAAAAADSjIwgAAAAACYapoQAAAABgwG+jbEwNBQAAAAA0oyMIAAAAAAmGqaEAAAAAYCBJ/hppS+rgmJ/qBAAAAAB0ATqCAAAAAJBgfDs1NLNhs9TY2EFEjrIaSrwTjR1rVlNQUbNcn+V+1yxXyCyTpLo675hw2CmuV6+wQUH7TZ9ulkrTptt9//HaK3Zt4oGHvOu6/Xa3uJtusqjooOxsu1yBtWvskkUGmqWqzcjxjAk7xlWVGRR0QCRil6uiwi5X6qfvmeWqypjgGZOTI1VVeefKCW2Lv6AmDWlmqSKRoFku1Ri+dTc0eMcEg05xdXV2v2O4zO4iyss1fJfcUGWWakvaCM+YLElblOUZZ9ruJenjVWapRo0ab5Yr8OkSs1ypJ53k9WxKjbi9x38Y8r6HuRqnLWa5LD8DO92AvQQSY/wpICnW3UUcgqmhAAAAAIBmdAQBAAAAIMH4dmooAAAAAPQkTA0FAAAAAPgWHUEAAAAASDBMDQUAAAAAA0wNBQAAAAD4Fh1BAAAAAEgwTA0FAAAAAANJ6ng6pp8wIggAAAAACYaOIAAAAAAkGKaGAgAAAICBnjTK1pNqBQAAAAAYoCMIAAAAAAmGqaEAAAAAYKAnjbL1pFoBAAAAAAboCAIAAABAgmFqKAAAAAAY6EmjbP7tCDY2Sg0NHcd4HZdUW2f3coRCZql0YsMSu2S5o8xSrS8Le8YMGeIYp/UWJUmSBg4cYpbrtVeiZrkeeMiufd1+q0tdAac4y7ok19rc1GbnmeUK11Xa5UqpdYlSWN5xoYHe14er7dvNUmngQLtcW0ITzHJluMa5BFreqDdsMEu1/Si7dt+/f6pZrkDZFu+gcFiqqvIMS5d3jKstaSPMcoXqzFIp3bB9ZUWqHaJS3eJSInHX08Jpp5mlSq9x+T0dZWebpaqu6fh9MjXVO6aJ073JlWEbKym1+yyQU1cRf5KUFCktLf48MOPfjiAAAAAAIG5bt25VY2Nji8foCAIAAACAgaSkJCkpqbvLOOhALTNmzNDmzZtbHKIjCAAAAABHsOLiYkYEAQAAAKBTpKTs/+cXB2oZNGhQq0Pf6K9I77jjDl1++eWtHt+0aZN++MMfqqCgQAUFBbr55ptVWWm3mAMAAAAAIH6H3V1duHChFi5cqIKCghaP79ixQ9///vdVX1+va665Ro2NjXruuee0evVqLVy4UMFg0KxoAAAAAMA359wRbGxs1DPPPKOnnnqqzeMvvviiysrK9Ktf/UpDhw6VJJ1wwgm66qqr9MYbb2jatGk2FQMAAACAHyUn+2tqaHJyu4ecpobu3btXF154oZ588kkVFRUpMzOzVczixYtVUFDQ3AmUpHHjxunYY4/V4sWLv0HVAAAAAIDO4NwRrKmp0RNPPKGHH35YKV/r5e7cuVObNm3SyJEjW/3syJEjtXz5cptqAQAAAABxcxq3jEQieuedd1p1AJuUl5dLUpsjhf3791dNTY127dqlfv36xVEqAAAAAPiYT1cNbfOQy88HAgEFAu0PHu7evVuS1KdPn1bHevfuLUmqra09vI5gTo53zJAhniFh92fsWl9bbMcvHE7pYcQ5JuvSTNI3XCy3TbffbpZKznV1cC02sa1LsjxnYcuLMpxumMz1Ob1/AbuzJbXx/ZovmL6OjrKyXKJS7Z4w1S6XT19G15PqHmeka5/tMKTbvhs5MWyH3cKyfsNcLplcn872JbJ7X8sxfYvMs0wGnzDprkajUc+YjjqSbSopkRoa2j8+ZIi0fr1nmtqBdjftUMgslQKfLrFLNmqUWar1Zd6f7hxPvYbIIcjRestOZa53e3X1wEOGncpbHeoKBCSH682yLsmxNke1dYadyjrD7WlcLvBwWKqt9QyLhux6Sdu3m6WS5aSMqiq7XC6ysqQtWxziItV2T1pWZpaq/Ci7D1H9+5ulUqDM5aQ6nnxDWwy7gpbv3elVdu9rysjwjklNlaod2rT16IPlSaup8WWu6kjHbcz11EumtwrlZdi9r5XUGHYq69bEnyQlxX3EAV3C5M7Rt29fSfv/lvDrmh5rigEAAACAI9KRtmqol6wDU0a2t/HV9bZt25Samqpwd8wjAgAAAAC0YtIRTE1NVXZ2tj7//PNWx1asWKFRhlMXAQAAAADxMftjnYkTJ+qjjz7SunXrmh/78MMP9eWXX2rSpElWTwMAAAAA/tS0aqif/rVXqtXvfO211+rNN9/UlVdeqZkzZ2rv3r2aO3euRo4cqaKiIqunAQAAAADEyWxEMD09XfPnz9fw4cM1Z84c/fKXv1RhYaHmzp2rYDBo9TQAAAAAgDh9oxHB3/3ud20+PmTIEP3iF7+IqyAAAAAA6JF60KqhPqqypfUNOR1uI5gnaU2D914keTXb7GoqG2CWK/ckuw3lA1V2e86UlrrtI1ha6p2rwXAPx2HDzFIparjd9003maVy2vvv9tsd4wz3/ZOk62fZnbPrrzdLpeHD7fZIqqvzjkmVVN3gfY2kyu78Z/ZzKMxVRzfVwxSucrgJOKrOHuEUF4k4BFnuWWa4l1pmkt17keSw/5yrtDSzuPoUu9XBy5abpXLars9VuuEGmi77HIcl1aZ471YerigxqOgQlifNcpO93FyzVKk1Xp+d0pXa4Pb5as9Rdu9F0TS7XA1VZqlUmRH/XqiBgJQWfykwZLvrNAAAAADA93w7IggAAAAAPYrHSp1droNaGBEEAAAAgARDRxAAAAAAEoyPxi0BAAAAoAfrQauGMiIIAAAAAAmGjiAAAAAAJBgfjVsCAAAAQA/GqqEAAAAAAL+iIwgAAAAACcZH45YAAAAA0IOxaigAAAAAwK/oCAIAAABAgvHRuCUAAAAA9GCsGgoAAAAA8Cs6ggAAAACQYHw0bgkAAAAAPRirhgIAAAAA/CopFovFuruINpWXS42N7R/PypK2bPFMU5uWZVZSuHSNWS7fysjwjklPlyorPcOqU9INCrJXVWWXKzvbLldAUYeggBT1jrt+lu13PM887VCbo9vvsKvtnnvMUilYV+0dlJoqVTvENTTEX1ATywablmaWqj5id30Hy0q8g3JypBKHuEgk/oI6Idf60qBZrlDILJWy0mq9g8JhqdYhbunSuOtpsiV3nFmurBrD927DNlGf4f35JBiU6uu9c61aZVDQIfJH2d3za+vs7vnhFIeT4crrPu3a7iUtWxs2KGi//IHbzHKtqBhglmvYMJs8QbtboX9Nnixt3drdVRw0aJC0eHGbh3w0bgkAAAAAPRirhgIAAAAA/IqOIAAAAAAkGB+NWwIAAABAD8aqoQAAAAAAv6IjCAAAAAAJxkfjlgAAAADQg7FqKAAAAADAr+gIAgAAAECC8dG4JQAAAAD0YKwaCgAAAADwKzqCAAAAAJBgfDRuCQAAAAA9GKuGAgAAAAD8io4gAAAAACQYH41bAgAAAEAPxqqhAAAAAAC/oiMIAAAAAAnGR+OWLVX1zlQ02v7xdEmVoSzPPOmhDpIcpmV1eWa5hg83S6Xgx++b5arP9f4dg5LqI+mecamqN6jogKoqs1Q1KQPMcgXWrjHLVZvtfe7DYam2zvv7m+uvt6jooNvvsPvO6IH77K7JadPt6nrllVTPmICkaMQlzu53VChkl2vDBrNUQZWZ5dKwYW5xAwd6x9TVxVfLIVasDZrlGhEpMcsVHZhjl0thz5iApGjIIS4tLf6CDsgaaHgNLbdrEy73aWcN3iHBoNTgEJefXRl/PYeqs7vvhA2nyFXW2F2T6XUVHQeEw86fPfKz7dqYauxyjRhudx2t3xD/+21KipRjd/vyL1YNBQAAAAD4FR1BAAAAAEgwPhq3BAAAAIAejFVDAQAAAAB+RUcQAAAAABKMj8YtAQAAAKAHY9VQAAAAAIBf0REEAAAAgATjo3FLAAAAAOjBWDUUAAAAAOBXdAQBAAAAIMH4aNwSAAAAAHowVg0FAAAAAPgVHUEAAAAASDA+GrcEAAAAgB6MVUMBAAAAAH5FRxAAAAAAEoyPxi0BAAAAoAfrQauG+qjKltLC9R4RQaVHvGKkLWVBm4Ik5dctMculimyzVPVjx5vlqqjwjsnKcoxLa4i/oCaRiFmqrIZqs1yKDDRLFa6rdAhKd4obPjzdoKKD7rnHLte06XYTEV57JWqW6/kXveuaOVN68UXvXGPH2v2Oublhs1xhyzemjAy7XO+95x0zcaJTXP0ZE+MupzNEs3PMcgWWLzPLVZ2b7xmTmirV1HjnSjWop9natXa5DN8/wqo1y/X+p97X9vjx0qefeucaP8qgoENU1tndd9LT7O7T6Rs+M8tVO/zEDo+HJdWmZTnlsry1Bg3bq+rqzFINyQ0ZZWIyop/wagAAAABAgvHtiCAAAAAA9CisGgoAAAAA8Cs6ggAAAACQYHw0bgkAAAAAPVgPWjWUEUEAAAAASDB0BAEAAAAgwfho3BIAAAAAejBWDQUAAAAA+BUdQQAAAABIMD4atwQAAACAHoxVQwEAAAAAfkVHEAAAAAASjI/GLQEAAACgB2PVUAAAAACAX9ERBAAAAIAE46NxSwAAAADowVg1FAAAAADgV0mxWCzW3UW05f33pbq69o9PnCi98453nrFj7WpKrSoxy7WsKscsV36aXV0l8q4rJ0cqcXjKnJQtBhUdkJFhl6uszCxVbYbd6xhWrUNQWKr1jqtuCBtUdFCqqs1yRSOpZrlefNEslWZeGfUOCgSkqHfcjTfZfcf22GNmqUxVVdnlSm/Y5h00YIC0zSGuoSH+gppY3ncsGf6OKzZ43ytGjJBWrPDOlZsbfz1NLNtXB+skHLbMPzt88HDl8gElNVWqdrj/Wo8+GL4AlaEss1zpdYafK9LSOj7u+H4rSZ+tsnvPPTGyxizXioY8s1zZ2fHnSEqS+vWLP4/vPf647U0sXmlp0o03auvWrWpsbGxxyEfjlgAAAADQg/l01dAZM2Zo8+bNLQ75qEoAAAAAgLXi4mJGBAEAAAAgkQwaNKjVY9+oI3jHHXdo48aNeumll1o8ftFFF+kvf/lLq/hzzjlHc+bM+SZPBQAAAAA9Qw9aNfSwq1y4cKEWLlyogoKCFo/HYjGtW7dOhYWFmjhxYotjgwcPPtynAQAAAAB0EueOYGNjo5555hk99dRTbR4vLS1VbW2tzjrrLBUVFZkVCAAAAACw5dQR3Lt3r6ZOnarVq1drypQp+uijj1rFrF27VpI0dOhQ2woBAAAAoCfw6aqhbXHa7Grv3r2qqanRE088oYcfflgpbfxyX3zxhaSDHcFax71XAAAAAABdy6m7GolE9M4777TZAWzyxRdfqG/fvnrwwQf161//WrW1tfrWt76l2bNna/LkyWYFAwAAAADikxSLxWKH+0MTJkzQ4MGDW6waeuGFF2rFihU655xzdP7556u6ulrz5s3TqlWr9PDDD2vKlCmWdQMAAACAv8ydK1VXd3cVB6WmStdc0+Yhswms06ZNUzQa1YwZM5ofmzx5ss477zw9+uijOv/885XcwRzVr3v/famurv3jEydK77zjnWfsWOen9JRaVWKWa1lVjlmu/DS7ukrkXVdOjlTi8JQ5KVsMKjogI8MuV1mZWaraDLvXMSyH6dThsOQw7bq6IWxQ0UGpsruhRSOpZrlefNEslWZeGfUOCgSkqHfcjTc5zbp38thjZqlMVVXZ5Upv2OYdNGCAtM0hrqEh/oKaWN53LBn+jis2eN8rRoyQVqzwzpWbG389TSzb12F89PCU+WeHDx6uXD6gpKa6faC0/nskwxegMpRlliu9zvBzRVpax8cd328l6bNVdu+5J0bWmOVa0ZBnlis7O/4cSUlSv37x54Eds08rl1xySYtOoCSFQiEVFRWpoqKieTEZAAAAAED36vQlbdLT0yWxeAwAAACAI9yRtmqol/Lyck2ePLnNPQa//PJLSVK2xZgyAAAAACBuJh3BzMxM7dq1SwsXLlRNTU3z41u3btWiRYt0yimnqH///hZPBQAAAACIk9m45V133aVZs2Zp+vTpmjp1qnbv3q3i4mKlpKTo7rvvtnoaAAAAAPCnlBR/TQ3toBazxWIKCwv19NNPq0+fPnrsscf0wgsvaPTo0Xr55ZebN5kHAAAAAHS/b9Rd/d3vftfm44WFhSosLIyrIAAAAADokXrQYjE+qrKloUOlxsaOY4YP986zYYNJOZKk5cvt9oy7dLrDnmWOog57/7nKqXDYp0sDlBNyiFu+Ku56mizLmGCWK3+U3cJFVXZbEio00HsfooCkaMg7LlV27UuSVGW3b1nAsLaxY+3263PZ++/xxx3jHrP7Hf/xBrvf8b77zFIpErHLpQbHZA5P6nJ9uLLc9Sgvt94umSGX91HXuECZ3R5v4Y42Ej5coZBdLsvNiV1+x9RUtzjTC1Kme1Xalmb3WlbWdXyvSA97xzQ5MbfSoqT9aux+xxGy22e6siH+z5oBu7czGOElAQAAAIAE49sRQQAAAADoURJxsRgAAAAAQM9ARxAAAAAAEoyPxi0BAAAAoAfrQauGMiIIAAAAAAmGjiAAAAAAJBgfjVsCAAAAQA/GqqEAAAAAAL+iIwgAAAAACcZH45YAAAAA0IOxaigAAAAAwK/oCAIAAABAgvHRuCUAAAAA9GCsGgoAAAAA8Cs6ggAAAACQYHw0bgkAAAAAPRirhgIAAAAA/CopFovFuruINpWUSA0N7R8fMkRav947T26uWUmqqzNLFQ2FzXLV1JilUiTiHRMISNGod5xlXR01hcPlpy9pDrVnj3dMZqZUXu4Q1682/oIOVVZml2vgQLNUtbK7jkIh7xjXtn/DDXGX02zOTx2e0NHfTbb77u/NN81SKVjqcC93vOeX9x1iUNF+mb0qzXJF09LNcr34olkqzTxvm3fQgAHSNu+49TUDDCrab/t2s1Tq188u14iBdm2iNuTdJsJhqdbhdh6u2mJQ0SFcPgy42rDBLtewYWapvN4/XM+9JFVVxV9PkyzZvZbRgVlmuQJLP4s/STAojRoVfx6/+6//cvtQ11X69JHOPrvNQz79SAwAAAAAPQyrhgIAAAAA/IqOIAAAAAAkGB+NWwIAAABAD8aqoQAAAAAAv6IjCAAAAAAJxkfjlgAAAADQg7FqKAAAAADAr+gIAgAAAECC8dG4JQAAAAD0YKwaCgAAAADwKzqCAAAAAJBgfDRuCQAAAAA9WEqK1NjY3VUcxKqhAAAAAIAmdAQBAAAAIMEwNRQAAAAALLBqKAAAAADAr+gIAgAAAECCSYrFYrHuLqItu3ZJHVWWmipVV3vnSQ3Vm9VUr6BZLks1NXa5qqq8Y4YMkdavd4hLKYm7nmYZGWapqhvCZrkqKsxSaeBA75hwWKqtdYhrcLg4DkdDg12usjK7XIZTL6LD8jxjAgEpGvXOZXlNXnyxXa7fLHYo3tG8+XbfI15RuMU7KCtL2uIQt3Zt/AUdsCxtvFmu/IjDTdNRZdoQs1zpNQ736ZwcqcTwfu4iLc0ul+H9q3xfulmur77yjhkxQlqxwjsuNzfuclqwvOVbfg575z27z2ETx3q8T7p+0JRMPwyU97W7vjOTtpnlWlY2IO4cvXpJxx9vUIzfLV8u1du1+7gFg9KoUW0eYkQQAAAAABIMHUEAAAAASDA+WtIGAAAAAHowVg0FAAAAAPgVHUEAAAAASDA+GrcEAAAAgB4sJaXjrQ+6WgfTVBkRBAAAAIAEQ0cQAAAAABIMU0MBAAAAwEJysr+mhrJqKAAAAACgCR1BAAAAAEgwTA0FAAAAAAt+2kxeYtVQAAAAAMBBdAQBAAAAIMH4bOwSAAAAAHqoDlbp7BasGgoAAAAAaEJHEAAAAAASDFNDAQAAAMBCSoqUlNTdVRzUwdRQ33YE+634RNq7t/2A8eOVuvR9zzwluePNakpLM0ul1EjULFd6pMEuV0qdQ1SqhmRUe4eFBsZdT5OSsqBZrpxsu3Of+ul7Zrm2hCZ4xoTDUlWVd65wVWn8BR2iftgIs1xBlZnlUkaGWSqX85qe7hYXicRbzUFvvmmXa958u0kgV1xmdx3dfkeWZ8wDD0i3P+UdN326d4yrBrtbq5SdbZaqxvASWro2xzNmQo70O5e4tM8sSpIklff2fj5XvXqZpVJmr0q7XGku77dZGpG2xTMqGrJr95K0apVdrhFpFWa5xo41/D1dLnDHm0B53yFxFnNQ5lcrzHJFh9u9d0dq4s/ht10VwNRQAAAAAEg49M0BAAAAwEJysr+mhgbaH/djRBAAAAAAEgwdQQAAAABIMEwNBQAAAAALKSlS1G5BtbgxNRQAAAAA0ISOIAAAAAAkGKaGAgAAAICF5OQOp2N2uQ5WMPVRlQAAAACArkBHEAAAAAASDFNDAQAAAMBCSooUi3V3FQcxNRQAAAAA0ISOIAAAAAAkGKaGAgAAAICF5OTursAZI4IAAAAAkGDoCAIAAABAgmFqKAAAAABYSOk53StGBAEAAAAgwSTFYn7a6OKg8nKpsbH941lZ0pYt3nmyGkrsiqqqMku1JSPfLFdamlkqlZZ6x+TlSWvWeMfl5sZdTrNgStQsV0mp3fcfhk1CGRneMa7tPhKJv55DpVYZXkcDB9rleu89u1yjR3vHDBggbdvmHWf5ApSV2eUKhcxS3f5UllmuB+5zuL4DASnqHXfXPXbX971XrjfLZXqjrqmxy+XSVtPTpcpKz7BoWrpBQfsF6mrNcpXvCpvlyjy63iyXli71jikokJYs8Qz7dUVB/PUc4owz7HKFK+zePyojOWa5vJp+MCjVO77cwQqHN2ZXFRVmqcoz7T5rmrX9YNAmj585vFd1uUBAW7duVePXOlc9Z+wSAAAAAHws6sMJlwFJM2bM0ObNm1s8TkcQAAAAAI5gxcXFjAgCAAAAQCIZNGhQq8ecO4IffPCBnnnmGX3++ecKBAI64YQTdMMNN2j0IX9Xs2nTJj388MNacmA++xlnnKFbb71V6el2fzcAAAAAAH7U0NDdFbTW3p9mOnUElyxZomuvvVbf/va3NXv2bDU0NGjBggW67LLLtGDBAuXn52vHjh36/ve/r/r6el1zzTVqbGzUc889p9WrV2vhwoUKJsIfhwIAAABAD+DUEXzggQc0aNAgvfbaa+rTp48kacqUKZo0aZKeeOIJvfDCC3rxxRdVVlamX/3qVxo6dKgk6YQTTtBVV12lN954Q9OmTeu83wIAAAAA4MxzWZudO3dq1apVOvfcc5s7gZKUkZGhk08+WX/6058kSYsXL1ZBQUFzJ1CSxo0bp2OPPVaLFy/uhNIBAAAAwD8aG/dPD/XLv4624/McEYxEIvrtb3/bohPYZMeOHUpOTtbOnTu1adMmnXPOOa1iRo4cqfcs9/oCAAAAAMTFc0QwOTlZubm5yszMbPH4qlWr9Nlnn2nMmDEqLy+XpFYxktS/f3/V1NRo165dRiUDAAAAAOKRFIvFYof7Q7t379all16q1atXa968eerVq5emT5+u++67T1OnTm0R+8QTT+jnP/+53n///TY7igAAAABwJNi1Szr83lXnSUqS+vVr+9hh7yO4Z88eXX/99Vq1apWuu+46FRQU6H//9389fy4Q8Bx8bKG8vOM5rVlZ0pYt3nmyGkoO63k7VFVllmpLRr5ZrrQ0s1QqLfWOycuT1qzxjsvNjbucZsGUqFmuktLDa4sdMWwSysjwjnFt95FI/PUcKrXK8DoaONAul+W080O2wmnXgAHStm3ecZYvQFmZXa5QyCzV7U9lmeV64D6H6zsQkKLecXfdY3d933vlerNcpjfqmhq7XC5tNT1dqqz0DIum2W0VFairNctVvitslivz6HqzXFq61DumoEA6sCVXR35dURB/PYc44wy7XOEKu/ePykiOWS6vph8MSvWOL3ewwuGN2VVFhVmq8ky7z5pmbZ9dBHzlsN4xq6urNXPmTH3yySf63ve+p9mzZ0uS+vbtK0nau3dvq59peqwpBgAAAADQvZxHBL/66itdffXVWrlypS6++GL9y7/8i5KSkiRJWVn7vxnevn17q5/btm2bUlNTFQ7bfSMHAAAAAH7T2Og0gaXLdDQp06kjWFNT09wJvPLKK3Xbbbe1OJ6amqrs7Gx9/vnnrX52xYoVGjVq1OFVDAAAAADoNE5TQ++9916tXLlSV1xxRatOYJOJEyfqo48+0rp165of+/DDD/Xll19q0qRJNtUCAAAAAOLmOSK4bt06vfnmm+rXr5+OP/54vfnmm61iioqKdO211+rNN9/UlVdeqZkzZ2rv3r2aO3euRo4cqaKiok4pHgAAAAD8oqHhCJoauuTAalW7du1qdzSwqKhI6enpmj9/vh588EHNmTNHoVBIhYWFuvnmmxVkhSAAAAAA8A3PjuAll1yiSy65xCnZkCFD9Itf/CLuogAAAAAAnecbbSjfJfbs6Xg3xnBYqnXYZyjlsLdKbJ/h3i6m+4wZ7g3mtD+V455Spr/jp5+apaocPs4sV3qDw55yrlxex9RUqbraO85ynzHJto0ZXpP1oVSzXE77QDlu5BgdaLfHXhuLMX9jmavfN8u1LG28Wa7XX/eOufde6a67HOLusZuPc/OtdnsSPvSQWSrbNlG+zDsoP19a5hA3bFj8BR2wptRupfG8VW+Z5ZLh4nfVGUM8Y1xv+amlKwwqOkR2tlmqNWV29+m8NMP33Lq6jo/n5Egljnsg+vQ90vN3PBwWn+k62tn8COK1F3pXS06WMjPbPmb3LgcAAAAA6BHoCAIAAABAgjEcfwYAAACAxNXQ4K+poR39pR0jggAAAACQYOgIAgAAAECCYWooAAAAABhobNw/PbQnYEQQAAAAABIMHUEAAAAASDBMDQUAAAAAAw0NTA0FAAAAAPgUHUEAAAAASDBMDQUAAAAAA35bNTQpqf1jjAgCAAAAQIJhRBAAAAAADPhtsRhGBAEAAAAAzegIAgAAAECCYWooAAAAABjw22IxgQ6G/RgRBAAAAIAEkxSLxWLdXURbdu2SOqosNVWqrvbOk1qx3q6oUMgsVXUkyyxXqhxOhKNqpXo/n+u5T6k1qOiAmhqzVNGMAWa5Ag31Zrm0YYN3TF6etGaNd5xhW5UkDRxolmrF2qBZLksjhjm8lsGgVO8dt2aD3e+Yl1FplmtZabpZLstvO09Mc7hPDxkirfeOu/nnQwwq2u+Rh6Jmua6fZfe9649/bJZKubneMYGAFHU4Fdu3x11Osz597HKlhuzu09EUu2vb6f3D8Z6jVaviL6izjBpllurDj+2uo3FjPRq1a8OXpKqquOtpVlZmlqp+2AizXMEGg890SUm2F7dPLV/udtl2lWCw/cuQqaEAAAAAYMBvq4YyNRQAAAAA0IyOIAAAAAAkGKaGAgAAAIABv60ampzc/jFGBAEAAAAgwdARBAAAAIAEw9RQAAAAADDgt1VDmRoKAAAAAGhGRxAAAAAAEgxTQwEAAADAgN9WDU3poLfHiCAAAAAAJBg6ggAAAACQYJgaCgAAAAAG/LZqaEe1MCIIAAAAAAmGjiAAAAAAJBimhgIAAACAAb+tGtrY2P4xRgQBAAAAIMHQEQQAAACABOPbqaG9e3vHhEIOiUpL466lSfXo8Wa5UlVtlktVVXaplOoZk5rq9pSpqoi/oCYZGWapamrMUikSCZrl2n5UnmdMpqRyl7ikbQYVHbS+1O73HBEpMcsVzc4xyyXDaRx5ufVmuaIp6Wa58qvWm+VSdrZdrpo0t7g077iHHoqrkhaun2X3XekzT0fNcv3dZLu6nn3WOyYnx+2tNGegXbu3vFGX77C7hsrLzVIpI8P7vpqVJW2pcIiLRCxKavZ+6RCzXONr7D7vDB3q/RmlO9RH7NpYynC7XMGyLWa5TOY6JidLgwfHn8fnWDUUAAAAAOBbdAQBAAAAIMH4dmooAAAAAPQkrBoKAAAAAPAtOoIAAAAAkGCYGgoAAAAABlg1FAAAAADgW3QEAQAAACDBMDUUAAAAAAywaigAAAAAwLfoCAIAAABAgmFqKAAAAAAYYNVQAAAAAIBv0REEAAAAgATD1FAAAAAAMMCqoQAAAAAA36IjCAAAAAAJhqmhAAAAAGCgJ60a6tuOYHDFUqm+vv2AggIFly7xTjR6tFVJSo1EzXJFlWqW6w9L7XKlOLSInByptNQ7LntsTvwFHbB9u1kqZfaqtEtWY3cJ9e/v9jr27+8SlRFXLV8XMryhRQfatYvA8mVmuTRsmHdMMNjld/cX5wfNck2ZMsQsV02ZWSrlqMY7KD1dqvGO274v3aCi/X78Y7NU+rvJdhNwfrPY7r3o9ju863rgAennP/fOdd99dm01kJZmlqtfnVkqZfa3O/eVVW5tIhTyjqnPsLu2JWm04VtI+R67zyj9+pmlkqqqOj6enu4d0yRid98JlG0xy7WsIsssV/7AbfEnCTAR0W94RQAAAAAgwfh2RBAAAAAAehJWDQUAAAAA+BYdQQAAAABIMEwNBQAAAAADPWnVUEYEAQAAACDB0BEEAAAAgATD1FAAAAAAMMCqoQAAAAAA36IjCAAAAAAJhqmhAAAAAGCAVUMBAAAAAL5FRxAAAAAAEgxTQwEAAADAAKuGAgAAAAB8i44gAAAAACQYpoYCAAAAgAFWDQUAAAAA+JZvRwT3HDdasVj7x8OSakcVeOZZutSsJI0bVW2XLJJqluq008xSOZ+vUMg7JqBoXLUc6pNP7L6zuODciFkuy698AmVbvIOystzi0tLirqfF0xqmiypslqs6N98sV+kG75gRI6QVG7zrHz48/nqazDxvm12ymjqzVEvX5pjlyhnteE1GvOMyS5fFWc1B0VF27evZZ81S6fY77O6HD9zncp8OOMUtesOursJCu1y238zb1ZUeqnWICrvFrVobdz2HCg4caJeszwCzVOEaw/uhC8fGE6xweF92tKYmyyyX6UeBsrL4c/TqJWVkxJ8H38jWrVvV+LWVY3zbEQQAAACAnsSvq4bOmDFDmzdvbnGMjiAAAAAAHMGKi4sZEQQAAACARDJo0KBWjzl3BD/44AM988wz+vzzzxUIBHTCCSfohhtu0OjRo5tjLrroIv3lL39p9bPnnHOO5syZ882qBgAAAIAeoCetGurUEVyyZImuvfZaffvb39bs2bPV0NCgBQsW6LLLLtOCBQuUn5+vWCymdevWqbCwUBMnTmzx84MHD47rFwAAAAAA2HHqCD7wwAMaNGiQXnvtNfXp00eSNGXKFE2aNElPPPGEXnjhBZWWlqq2tlZnnXWWioqKOrVoAAAAAMA359kR3Llzp1atWqWrrrqquRMoSRkZGTr55JP1xz/+UZK0du3+pYuHDh3aSaUCAAAAgH/5ddXQtnh2BCORiH7729+26AQ22bFjh5KTkyVJX3zxhaSDHcHa2lqFw3b7hQEAAAAAbHjujJqcnKzc3FxlZma2eHzVqlX67LPPNGbMGEn7O4J9+/bVgw8+qDFjxmjMmDEqLCzU4sWLO6dyAAAAAMA3khSLxWKH+0O7d+/WpZdeqtWrV2vevHkqKCjQhRdeqBUrVuicc87R+eefr+rqas2bN0+rVq3Sww8/rClTpnRC+QAAAADgD//6r9KOHd1dxUFHHy3deWfbxw57H8E9e/bo+uuv16pVq3TdddepoKBAkjRt2jRFo1HNmDGjOXby5Mk677zz9Oijj+r8889vnkbq9jxSR13UcFiqrfXOs3Sp81N6Gjeq2ixXNJJqlsuSy/k68UTps88c4kZH466nyVtvew5eO7vg3HqzXKaTwKuqvGOysqQtW7zj0tLirabTREN2U8ZrasxSqbTUO2bECGnFCu+44cPjr6dJoGKbXbK6OrNUv1ubY5ZrwuhK76D0dKnSIc7lhXQUHZVvlsuwLP3853a5HrjP4T4dCEhR77hFb9jdpwsLzVKZ3qZDIbtcYTl8iHH9sHNgnQYzAweapaoODTDLlVpneD/0MmCAtM3x+Qwb2ZqaLLNclu01p2pZ/El69ZKOPz7+PDBzWHft6upqzZw5U5988om+973vafbs2c3HLrnkkhadQEkKhUIqKipSRUVF82IyAAAAAIDu5Twi+NVXX+nqq6/WypUrdfHFF+tf/uVflJSU5Plz6enpkvYvHgMAAAAAR6qetGqo04hgTU1Ncyfwyiuv1L333tuiE1heXq7JkyfrqaeeavWzX375pSQpOzv7MMsGAAAAAHQGp47gvffeq5UrV+qKK67Qbbfd1up4Zmamdu3apYULF6rmkD/a2bp1qxYtWqRTTjlF/fv3t6saAAAAAPCNeU4NXbdund58803169dPxx9/vN58881WMUVFRbrrrrs0a9YsTZ8+XVOnTtXu3btVXFyslJQU3X333Z1SPAAAAAD4RUODv6aGdlSLZ0dwyZIlkqRdu3a1ORoo7e8IFhYW6umnn9azzz6rxx57TKFQSAUFBbrxxhubN5kHAAAAAHQ/z47gJZdcoksuucQpWWFhoQot13wGAAAAAJg77H0Eu0qf3l57FgUUDnnvazRueJVJPZK0ojTdLNeIYf7cyy4jw22Pt4wMhyDDui44w27/s9oGuz0c6+qCZrnSVWWWqz7Fbq8+SQp++qFZroDhHoeWu3Hm5o5wjPOOCZQ57PXoaH2d3Z5SQ1JKzHJNSHPYTNRRNO1Ez5iApGia9z04YLhx1vbtZqmUM9Dunn/ffXb3HZe9/777Xce4KXZ7x957n92ehHdd5LD5p6PKgW73CRfhuiqHoLDbHrOWm5dKKimza2MRw0+aJXV2exLmhPy5R6vlRuSnHG+3//Xv1sa/r2ooJI0zqMXvjrhVQwEAAAAARw46ggAAAACQYHw7NRQAAAAAepKetGooI4IAAAAAkGDoCAIAAABAgmFqKAAAAAAYYNVQAAAAAIBv0REEAAAAgATD1FAAAAAAMMCqoQAAAAAA36IjCAAAAAAJhqmhAAAAAGCAVUMBAAAAAL5FRxAAAAAAEgxTQwEAAADAAKuGAgAAAAB8i44gAAAAACQYpoYCAAAAgIGetGrokd8RDIW6u4I2rVgbNMxml2vgQLe4SMQ7pt6wrmBDjVmucEOlXa6yMrNcW9JGeMZkSdqiLM+4suUGBR1iYO44s1xZA6NmubR2rVmqqirvmHDYMa6uLt5ymm3fbpZKQ45PM8tV3jvHLFdmXa13UDisgEPcmtKwQUX7ud4PndTY3cMCaWlmuQoL3SYGFRZ6x9x7n90ko7vusLtPjD/D+97q6v23q81yKSPDLK66zvIzhdt7vKt02b3nppeuMsulsWO9Yxxfo8qUAXEWc9ApoS1mucr3eH9ecDVyZPw5AsxD9B1eEgAAAABIMEf+iCAAAAAAdAFWDQUAAAAA+BYdQQAAAABIMEwNBQAAAAADPWnVUEYEAQAAACDB0BEEAAAAgATD1FAAAAAAMMCqoQAAAAAA32JEEAAAAAAMsFgMAAAAAMC36AgCAAAAQIJhaigAAAAAGGCxGAAAAACAb9ERBAAAAIAEw9RQAAAAADDAqqEAAAAAAN+iIwgAAAAACca3U0M3bw10OJSZkyOVlHr3Y3PWfmxW0/AzJpjlCihqlstSdY3bdwMpDi1nx444iznE3r3pZrnq6sxSKS83ZJYr5FhXyOEpMzLiq+XrsmrW2CVbbvgCRCJmqZKTDeNcXiRH/fqZpTKdq9Krl1kqle8Ke8Zkht3i8la9ZVHSfrnnmqUq32F3D+tneAm5NgmXuLsuWhFfMYcYf8YIs1zvv2f3fvvW26lmuc47zzsmICmaEvSMs56GZngLU3WDXduPjB1nlquqquPj6elSZZXbZ6KKivjraX7egXbva5l128xybWkYYJbrSMeqoQAAAAAA36IjCAAAAAAJxrdTQwEAAACgJ2HVUAAAAACAb9ERBAAAAIAEw9RQAAAAADDAqqEAAAAAAN+iIwgAAAAACYapoQAAAABggFVDAQAAAAC+RUcQAAAAABIMU0MBAAAAwACrhgIAAAAAfIuOIAAAAAAkGKaGAgAAAIABVg0FAAAAAPgWHUEAAAAASDBMDQUAAAAAAz1p1VDfdgT79/eOGTjQO2ZN3YT4izkgT1GzXKqrM0v1/qdhs1xpad4x+fnShg0OccNq4y3noD6GV1RNjV2uDVVmqdJDIYegIUqvWu8dVlUVf0GHcrnYHNVm55nlCsuujWX+4R3voIkTlflnh7ixY+Mv6IARoUqzXOX70s1yZfayq0tHRxyCgso8ut47bNSouMtpEk0JmuUqLzdLpcz+hu9FjhODXG5PlQNHxFnLQe+/XW2W6623U81yXXCe3bm/fpb3uX/mGWnWLO9cd95pUNAhUgw/HVq+HaWWrTHLFXJ4L3Jp95KUF9kSZzUHRSNZZrkUsWv7WQ0O918ndvdVxI+poQAAAACQYHw7IggAAAAAPQmrhgIAAAAAfIuOIAAAAAAkGKaGAgAAAICBnrRqKCOCAAAAAJBg6AgCAAAAQIJhaigAAAAAGGDVUAAAAACAb9ERBAAAAIAEw9RQAAAAADDAqqEAAAAAAN+iIwgAAAAACYapoQAAAABggFVDAQAAAAC+RUcQAAAAABIMU0MBAAAAwACrhgIAAAAAfCspFovFuruIttTXd3w8GPSOkaRgStSmIEmqq7PLlWI3GLulImiWKznZOyYzUyov947r3z/+epps326XK7Nxi1muLcoyy5UVqfYOSk2Vqr3jalNSDSo6yLC5mn5L9umndrnGj7Y7/5b3itrIALNcGzaYpdKINLvrSKWl3jEFBdKSJZ5h1cMLDAraLzXk8CbjyPI+HQqZpVJ6qNY7KByWah3iqqrirqdZRoZZqmiK3bmfNcsslZ552uHzSSAgRb3jLr3M9nv9p56yy5XesM0sV2WK3f3Q6zpybfaSVFERfz1NDJu+wmXrzXKV9x0Sd45AwPazoV8de6y0cWN3V3HQMcdIX34pbd26VY1fWzmGqaEAAAAAYCAWi8pPw2z7awloxowZ2rx5c4tjdAQBAAAA4AhWXFzMiCAAAAAAJJJBgwa1esx5UvlHH32kSy65RGPGjNHf/u3f6v7779fu3btbxGzatEk//OEPVVBQoIKCAt18882qrKyMv3IAAAAA8L1GH/5rm9OI4Mcff6yZM2dq5MiRuummm7R161bNmzdPy5cvV3FxsQKBgHbs2KHvf//7qq+v1zXXXKPGxkY999xzWr16tRYuXKhg0O6PtQEAAAAA35xTR/CRRx7RoEGDNH/+fIUOLLM0aNAg3Xvvvfrggw90+umn68UXX1RZWZl+9atfaejQoZKkE044QVdddZXeeOMNTZs2rfN+CwAAAACAM8+poXv37tXRRx+tadOmNXcCJamgYP8S3atXr5YkLV68WAUFBc2dQEkaN26cjj32WC1evNi6bgAAAADwmai6fyroof/a34LGc0Swd+/eeu6551o9vnLlSklSVlaWdu7cqU2bNumcc85pFTdy5Ei99957Xk8DAAAAAOgih71q6ObNm/XJJ5/o4YcfVl5ens4++2xtPLBrYmZmZqv4/v37q6amRrt27VK/fv3irxgAAAAAEJfD6ghWVVVpwoQJkqQ+ffrojjvuUO/evZtXD+3Tp0+rn+ndu7ckqba2lo4gAAAAgCNYoyQf7SivpHaPHFZHMCkpSU888YTq6+v10ksv6aqrrtLjjz+u/v37e/5sIOC8U4UkyWWRUbeFSA/veTsUDtvlMpSV1fXP2cbgbw96PrsTZnvqUx3DvOP82VL3s1xAePx4u1yW598pxpHlazlihGEyy9bvehM78LfpHbE785Jk11i74z7txrGFubz/+fQ90vBTgJ55xjCZa2UOn58WLIizlE41wCxTulkmN65NOienc+v4xoYMMUvVxR/70EUOqyN41FFHadKkSZKkc889V+edd54eeugh/fznP5e0f2GZr2t6rG/fvodVWH19x8eDQe8YSQqmtP8Hkoetrs4uV8phz8pt15YKuw8rycneMZmZUnm5d5zD9wPOtm+3y5XZuMUs1xbLTmWk2jsoNVWq9o6rTbH9OGzYXNXQYJfr00/tco0fbXf+Le8VtRG7D1EbNpil0og0u+tIpaXeMQUF0pIlnmHVw707i65SQw5vMo4s79OHrNsWt/RQrXdQOCzVOsRVVcVdT7OMDLNU0RS7cz9rllkqPfO0w+eTQECKesddeplld1d66im7XOkN28xyVabY3Q+9riPXZi9JFRXx19PEsOkrXLbeLFd53/g7lYGA7WdDxO8b3zlCoZDOOOMMbd26VQMG7L8wt7fxaX3btm1KTU1V2KffFAIAAACAjagP/7XNsyO4bt06TZgwQcXFxa2O7d69W0lJSQoGg8rOztbnn3/eKmbFihUaNWqU19MAAAAAALqIZ0fwmGOO0a5du/TKK6+o/pC5mJs3b9Y777yjk08+WZFIRBMnTtRHH32kdevWNcd8+OGH+vLLL5unkwIAAAAAup/nX/6kpKTojjvu0M0336zLL79cF1xwgXbs2KHi4mIlJSXpzjvvlCRde+21evPNN3XllVdq5syZ2rt3r+bOnauRI0eqqKio038RAAAAAOheHU/H7Hrtj/s5LQFRVFSkXr16ae7cuXrwwQcVDoc1duxYzZ49W8cee6wkKT09XfPnz9eDDz6oOXPmKBQKqbCwUDfffLOClssEAgAAAADi4rwW4KRJkzyneA4ZMkS/+MUv4i4KAAAAANB5DBeFBwAAAIBE1ih/TQ1tf3N733YEgzvKpcbG9gOyshSs8N7HqqTBbp+3nDqHva4crWjIM8s1ImS3T4zS0hyC0pXZq9IzqrrGbutXy31nSkoN20TIbn8kpUQc47wv23BFSZzFtLSsym633Pxs77bjarzlgsQpjpuzuWyqGHF8LR2Eq+z268vNtWv70ZBdrt8u9c41SdKvK7z3CJxUusKgogMMN73MMmwT9Rl2m0Rr1VrvmPx8aa1D3PDh8ddzQHWd3Z+UWO5demBZBBMue/8tWOAYN9/2Q+fNt9rtS3jNNXZ7/2Vnm6XyvreGs5zvvzmya2TRkD93p8/c6L2Pq6dgUOo/Ov48MGO7AykAAAAAwPd8OyIIAAAAAD1Lz5kayoggAAAAACQYOoIAAAAAkGCYGgoAAAAAJqLaPz3U/xgRBAAAAIAEQ0cQAAAAABIMU0MBAAAAwERU/lo1NKndI4wIAgAAAECCoSMIAAAAAAmGqaEAAAAAYKJRrBoKAAAAAPAlOoIAAAAAkGCYGgoAAAAAJvy2oTyrhgIAAAAADqAjCAAAAAAJhqmhAAAAAGDCb6uGMjUUAAAAAHBAUiwWi3V3EW2pr+/4eDDoHSNJwdL1NgVJUorhAKphrupIllmuSMQ7JhCQolHvuLVr46+nSV6K4euYnW2X6+OP7XKddpp3jOvJr6uLv55DhUJ2uQxrq6wLm+VKr9viHZSVJW1xiGtoiL+gJmlpZqmqlWqWq7TULJVyc71jwmGpttYhrqE67nqabdhglur9qnyzXKNHm6VSat0276ABA6Rt3nEldQMMKtrP5b3IleXty/LSdsmVni5VVnrHPfRQ/PUc6pGHHN5nHP3jDXZjDnPuM7y+vd6/J06U3nnHKVX12IkGBe1XUWGWSkMGOtw0HZVUxP9+m5wsDR5sUIzP5eZu0MaNhjeLOB1zTIo2bMht8xhTQwEAAADABFNDAQAAAAA+RUcQAAAAABIMU0MBAAAAwERM+zeV94v2l4NhRBAAAAAAEgwdQQAAAABIMEwNBQAAAAATfls1tP1xP0YEAQAAACDB0BEEAAAAgATD1FAAAAAAMMHUUAAAAACAT9ERBAAAAIAEw9RQAAAAADARlb+mhia3e4QRQQAAAABIMHQEAQAAACDBMDUUAAAAAEz4bdXQ9mthRBAAAAAAEgwdQQAAAABIMEmxWCzW3UW0qbxcauxgWDUrS9qyxTNNdGCWWUkff2yWSuMGrrdLlp1tl6uszDsmJ0cqKfEMqx+YY1DQfjt2mKXS6tV2uUaNssuVnlLtHZSaKlU7xLm8joehNjvPLFc4pd4sl1LsZrdXVnl/L5aeLlVWeueKRAwKOiC4apldsuHD7XJVVNjlamjwjnG876yps7vv5A2LmuVSTY1ZqvI9qWa5+vTxjnG97bi8jK7S5XChOapOSTfLVVVllko5oW3eQQMGSNu849ZUDTCo6KCnnrLLNeendtfR4z+1G7+YPr3j444fMyVJoVD89TRJj9i9R67ZEDTLlVe1JP4kwaA0enT8eXwuN/dP2rjR8LNOnI45JqgNG8a0eYwRQQAAAABIMCwWAwAAAAAm/LaPYPuj8owIAgAAAECCoSMIAAAAAAmGqaEAAAAAYIJ9BAEAAAAAPkVHEAAAAAASDFNDAQAAAMAEq4YCAAAAAHyKjiAAAAAAJBimhgIAAACACVYNBQAAAAD4FB1BAAAAAEgwTA0FAAAAABNRdbRSZ9dj1VAAAAAAwAF0BAEAAAAgwTA1FAAAAABMsKE8AAAAAMCnkmKxWKy7i2hLVZUU7eDvLNPTpcpKtzxWsrPtctXV2eUqK7PLddRR3jGZmVJ5uXdco+GXIZGIP3MFPl1il8ylgWVlSVu2eMdlZMRfTyeprAma5Urf8JlZLg0c6B3jev5Dofjr6YRc7/whbJZr7FizVGpo8I5xveenN2yLv6ADPlw7wCzX0KFmqdSvn12ucI3D+RowQNrmHVdSZ3e+cko/NMsVHTvOLFdg7RqzXJUZeZ4xru3e8pYjSeGGarNcj89NNct14w12C3BccWXHYyHz5klXXOGW6777DAo6YPlyu1yTxjo0HlcWH1yTk/d/iDzC5eb+Xhs37unuMpodc0wfbdhwepvHmBoKAAAAACbYUB4AAAAA4FN0BAEAAAAgwTA1FAAAAABMMDUUAAAAAOBTdAQBAAAAIMEwNRQAAAAATMTU0SbuXa/9nQIZEQQAAACABENHEAAAAAASDFNDAQAAAMAEq4YCAAAAAHyKjiAAAAAAJBimhgIAAACACaaGAgAAAAB8io4gAAAAACQYpoYCAAAAgImo/DU1tP3N7RkRBAAAAIAEw4ggAAAAABzBtm7dqsbGliOVSbFYLNZN9XRs6VKpvr794wUF0pIl3nlSDPu62dl2uTIy7HJ9+qlZqsphBZ4x6elSZaV3rvR3XzOoaL/oRdPMctXUmKVSaqT94fbDVV3jPUCfmipVV3vnSm1weIEORyhkl6uqyixVbVqWWa6wah2CwlKtd1xlXdigov0sT324waHxuGpoMEtVH0n3jAkGO35LaI4rKzGo6ADLe74lw2vI6XUcMEDats3uOV0YvkdWVtlNfrK8Hl043nIUrtpi+8TLl5ul2jJqolmuW281S6V5L3q8fwcCUtTtPX7caXZt7MPX7V5Ly/fIXbvizxEISP37x5/H73Jz/0MbN+7u7jKaHXNMX23YcKEmTJigzZs3tzjGiCAAAAAAHMGKi4tbjQjSEQQAAACAI9igQYNaPUZHEAAAAABMRNXRSp1dr/1anDuCH330kebMmaNVq1YpEono3HPP1Q033KC+ffs2x1x00UX6y1/+0upnzznnHM2ZM+cwiwYAAAAAdAanjuDHH3+smTNnauTIkbrpppu0detWzZs3T8uXL1dxcbECgYBisZjWrVunwsJCTZzY8g+DBw8e3CnFAwAAAAAOn1NH8JFHHtGgQYM0f/58hQ4smTVo0CDde++9+uCDD3T66aertLRUtbW1Ouuss1RUVNSpRQMAAACA/xxBG8rv3btXRx99tKZNm9bcCZSkgoL92wysXr1akrR27VpJ0tChQ+MqFQAAAADQuTxHBHv37q3nnnuu1eMrV66UJGVl7d+j5IsvvpB0sCNYW1urcNhuLy0AAAAAgI3DXjV08+bN+uSTT/Twww8rLy9PZ599tqT9HcG+ffvqwQcf1K9//WvV1tbqW9/6lmbPnq3JkyebFw4AAAAA/tIof00Nbb+WpFgsFnNNU1VVpVNOOUWS1KdPHz377LPN/3/hhRdqxYoVOuecc3T++eerurpa8+bN06pVq/Twww9rypQp8f0OAAAAAOBjubmvaOPGmu4uo9kxx0S0YcP0No8dVkdw586d+uMf/6j6+nq99NJLWrlypR5//HGde+65evnllxWNRjVjxozm+Lq6Op133nnas2eP3n//fSUnJ7tXvXSpVF/f/vGCAmnJEu88KYZbJWZn2+XKyLDL9emnZqkqhxV4xqSnS5WV3rnS333NoKL9ohdNM8tVY3htpkbs9omprvH8k12lpkrV1d65UhscXqDDccjfB8etqsosVW1allmusGodgsJSrXdcZZ3dtHjLUx9ucGg8rhoazFLVR9I9Y4LBjt8SmuPKSgwqOsDynm/J8Bpyeh0HDJC2bbN7TheG75GVVd73VleW16MLx1uOwlVbbJ94+XKzVFtGTfQOcnTrrWapNO9Fj/fvQECKur3HjzvNro19+Lrda2n5HrlrV/w5AgGpf//48/hdT+oIHlbLPeqoozRp0iRNmTJFxcXFysrK0kMPPSRJuuSSS1p0AiUpFAqpqKhIFRUVzYvJAAAAAMCRqdGH/9r2jb/CCIVCOuOMM7R161ZVdjA8lJ6+/5veWpevtAAAAAAAnc6zI7hu3TpNmDBBxcXFrY7t3r1bSUlJ2rNnjyZPnqynnnqqVcyXX34pScr26xQbAAAAAEgwnh3BY445Rrt27dIrr7yi+kP+QGPz5s165513dPLJJ2vw4MHatWuXFi5cqJpD/gBr69atWrRokU455RT1T4RJwQAAAAASWNOG8n751/7funqupJKSkqI77rhDN998sy6//HJdcMEF2rFjh4qLi5WUlKQ777xTknTXXXdp1qxZmj59uqZOnardu3eruLhYKSkpuvvuu93OGwAAAACg0zktqVlUVKRevXpp7ty5evDBBxUOhzV27FjNnj1bxx57rCSpsLBQTz/9tJ599lk99thjCoVCKigo0I033ti8yTwAAAAAoPs5760wadIkTZo0qcOYwsJCFRYWxl0UAAAAAPQ8UXU0HbPrxTE1tNvk53vHnHSSZ8j6DXZ7uwxJMd6bzUj0JO+9/1yl/+F976Dx45W+3CHujDPirqdJoNRub7BUw61MPgxNMMvlsm1WaqpUVuYdt+co733ZDkd5qV2u/Ow6s1yW24R+ttx7778TT5Q+W+UQl2t3r9hSZfdahusqzHKV9x1iliuzwmHfrKwsBV3ifLrnpcteic4MczmdU8ltv8E6u2u7MmWAWa4Ku2avvIjdHm8lDd57vOXkuNWfI7t9PSWpeqzd3n8hw9Luu88ul9fefx9+6L4/4Id/sPvQv+AVu73/zjrLLJUyj3bYyNVJ0CgPLNj1kgAAAAAAPYJ/RwQBAAAAoEfpeBP3rtcJG8oDAAAAAHomOoIAAAAAkGCYGgoAAAAAJpo2lPeL9hczYkQQAAAAABIMHUEAAAAASDBMDQUAAAAAE6waCgAAAADwKTqCAAAAAJBgmBoKAAAAACZYNRQAAAAA4FN0BAEAAAAgwTA1FAAAAABMsGooAAAAAMCn6AgCAAAAQIJhaigAAAAAmIiqo5U6ux6rhgIAAAAADvDviODWrVJjB39omZMjlZZ6psnNzbGrqbTGLlckYpaqqiZolit99Gi3QJe4urp4SmmhNsPudQyn2DX7cdpilkuhkENQuvIyKj2jomnp8ddziMykbXbJauzaRdDwOjoxssEhKk8nRtZ4h9W4vJZuslLszld53yFmuTK/WmGWSw0N3jFZWVJFhXdcdnb89TQpKzNLlTLc7poMlNndd9bUZHnG5DnG7dhhUNABp4Tsfsf0gXb3iWjE+zy4ynC8tDMyvGOiIcPPOpIqNtjlGpJdb5br44/tPu98+LpXG8tyiNlvwSt27eLS6XYjSTOvsRvvef7WDfEnSUmRhti9DyF+/u0IAgAAAECPwobyAAAAAACfoiMIAAAAAAmGqaEAAAAAYIIN5QEAAAAAPkVHEAAAAAASDFNDAQAAAMAEU0MBAAAAAD5FRxAAAAAAEgxTQwEAAADAREwdbeLe9WLtHmFEEAAAAAASDB1BAAAAAEgwTA0FAAAAABOsGgoAAAAA8Ck6ggAAAACQYJgaCgAAAAAmmBoKAAAAAPApOoIAAAAAkGB8OzW0vv/gDo8HJdUPzPHMs3aVUUGS/uqvvJ/P1d4ys1SqqbHLFcpN9YwJS6pNcYhbtTT+gg4oyx5glmtI2QazXBo71ixVSan39zI56VJJTbpnXEOVQUGHqKuzO/8jhhtuslpXZ5ZqRUOeZ8wI57gSg4r2iw7MMsuVWbHNLFd0+AizXNu3e8dkSirPzPeOq9sSf0EH1A+z+x2DZXZ1LauwaxNpaW5xoZB3zCnHV8dVy6HK9xi2+zq7dq+I93ufq3DZeu+gIUPc4owNGTjQLNeaDWGzXJPGVprlqg113MbCkmrT3NrhWWcZFHTAzGvsxmien2v3fvvAQ97vfV6OOkqaNcugGN+Lyl9TQ9tvB4wIAgAAAECCoSMIAAAAAAnGt1NDAQAAAKBnYdVQAAAAAIBP0REEAAAAgATD1FAAAAAAMBFVRyt1dj1WDQUAAAAAHEBHEAAAAAASDFNDAQAAAMAEG8oDAAAAAHyKjiAAAAAAJBimhgIAAACACTaUBwAAAAD4FB1BAAAAAEgwTA0FAAAAABNMDQUAAAAA+BQjggAAAABggn0EAQAAAAA+5dsRwTfekGpq2j8+c6Y0f753nplXtt8LPly1dXb95syvVpjlWh8aYZarrMw7ZsgQx7jRo+OupzmXqs1yqcKw2VdVmaXKqatwiMpTTt0az6jKjLz4CzpEJGKXa/0Gu+toSG7ILFd2tl1cZUNOfMUcIn3pZ2a5lqWcaJYr0sH9+XANya53iAoq82iHuDq7xhpsqDXLpYYGs1T5A7eZ5XK6mefkK6dqmWfY79bmGxS038iRZqm0pWGAWa6sBpe26qa87xDPmEzXuI1LDCo6qCTF+zld5VUZ1hZxvFE72LWv4+PhsLRrl1sup3uTo+dv3WCW64GH7D4L3H6r1edpxqD8xLcdQQAAAADoWVgsBgAAAADgU3QEAQAAACDBMDUUAAAAAExE1dFKnV2PVUMBAAAAAAfQEQQAAACABMPUUAAAAAAwwYbyAAAAAACfoiMIAAAAAAmGqaEAAAAAYIIN5QEAAAAAPkVHEAAAAAASDFNDAQAAAMAEq4YCAAAAAHyKjiAAAAAAJBimhgIAAACAiag6mo7Z9ZgaCgAAAAA4wLcjguGwd0wk0vl1HCopyTBZr15mqVK64VV0ek7TE2YoGLTLFTD8LsX1hXSIsyzLWne0VxeuzdUlzvT8G7bXXslmqXz7OpredyxzJRuefMsG5vpe5BAXCsVZyyH8fA+z4vo7OsVZvq/Jtrma1mZYmMt57ZZ2aHhzPeoos1Q4DNnZf9XdJbTQVM/WrVvV2NhyEZukWCwW646iAAAAAACdq66uTuPHj9fOnTtbPE5HEAAAAACOUNXV1aqurm71OB1BAAAAAEgwCTALHwAAAABwKDqCAAAAAJBg6AgCAAAAQIKhIwgAAAAACYaOIAAAAAAkGDqCAAAAAJBg6AgCAAAAQIKhIwgAAAAACSaluws4XJs2bdLDDz+sJUuWSJLOOOMM3XrrrUpPT+/myo58F110kf7yl7+0evycc87RnDlzuqGiI98dd9yhjRs36qWXXmrxONdB12jv/HMt2Pvggw/0zDPP6PPPP1cgENAJJ5ygG264QaNHj26Ood13DpdzT5vvPB999JHmzJmjVatWKRKJ6Nxzz9UNN9ygvn37NsfQ9juHy7mn7eNI1qM6gjt27ND3v/991dfX65prrlFjY6Oee+45rV69WgsXLlQwGOzuEo9YsVhM69atU2FhoSZOnNji2ODBg7upqiPbwoULtXDhQhUUFLR4nOuga7R3/rkW7C1ZskTXXnutvv3tb2v27NlqaGjQggULdNlll2nBggXKz8+n3XcSl3NPm+88H3/8sWbOnKmRI0fqpptu0tatWzVv3jwtX75cxcXFCgQCtP1O4nLuafs44sV6kMcffzx2/PHHx9auXdv82B//+MdYXl5e7NVXX+3Gyo58JSUlsby8vNi///u/d3cpR7yGhobYk08+GTvuuONieXl5scsuu6zFca6DzuV1/rkW7BUVFcXOOOOMWG1tbfNj27dvj5188smxK6+8MhaL0e47i8u5p813ngsvvDB25plnxvbs2dP82Pz582N5eXmx9957LxaL0fY7i8u5p+3jSNej/kZw8eLFKigo0NChQ5sfGzdunI499lgtXry4Gys78q1du1aSWpx72Nu7d68uvPBCPfnkkyoqKlJmZmarGK6DzuNy/rkWbO3cuVOrVq3Sueeeqz59+jQ/npGRoZNPPll/+tOfJNHuO4PruafNd469e/fq6KOP1rRp0xQKhZofb5qFsHr1akm0/c7geu5p+zjS9ZipoTt37tSmTZt0zjnntDo2cuRIvffee11fVAL54osvJB28GdbW1iocDndnSUekvXv3qqamRk888YQmTZqkCRMmtDjOddC5vM6/xLVgLRKJ6Le//W2LjkiTHTt2KDk5mXbfSVzOvUSb7yy9e/fWc8891+rxlStXSpKysrJo+53E5dxLtH0c+XrMiGB5ebkktfkNff/+/VVTU6Ndu3Z1dVkJ44svvlDfvn314IMPasyYMRozZowKCwv5NtJYJBLRO++8o0mTJrV5nOugc3mdf4lrwVpycrJyc3NbtelVq1bps88+05gxY2j3ncTl3Eu0+a6yefNmLVq0SPfff7/y8vJ09tln0/a7SFvnXqLt48jXY0YEd+/eLUltfnPZu3dvSfu/qenXr1+X1pUo1q5dq927d2vXrl165JFHVF1drXnz5unGG2/Uvn37NGXKlO4u8YgQCAQUCLT//QzXQefyOv8S10JX2L17t2655RZJ0t///d/T7rvQ18+9RJvvClVVVc0zEPr06aM77rhDvXv3pu13gfbOvUTbx5Gvx3QEo9GoZ4zXBzh8c9OmTVM0GtWMGTOaH5s8ebLOO+88Pfroozr//PObpxGh83AddD+uhc61Z88eXX/99Vq1apWuu+46FRQU6H//9389f452H7+2zr1Em+8KSUlJeuKJJ1RfX6+XXnpJV111lR5//HH179/f82dp+/Fp79yfe+65tH0c8XrM3aNpT5e9e/e2Otb02KH7vsDWJZdc0uJGKEmhUEhFRUWqqKho/oNqdC6ug+7HtdB5qqurNXPmTH3yySf63ve+p9mzZ0ui3XeF9s69RJvvCkcddZQmTZqkKVOmqLi4WFlZWXrooYdo+12gvXMv0fZx5OsxHcGmP9zdvn17q2Pbtm1Tamoqf8DbDZo2s62tre3mShID14F/cS3E56uvvtIVV1yhzz77TBdffLHuv/9+JSUlSaLdd7aOzn1HaPOdIxQK6YwzztDWrVs1YMAASbT9rnLoua+srGw3jraPI0WP6QimpqYqOztbn3/+eatjK1as0KhRo7qhqsRQXl6uyZMn66mnnmp17Msvv5QkZWdnd3VZCYnroHtxLXSOmpoaXX311Vq5cqWuvPJK3XvvvS06IrT7zuN17mnznWfdunWaMGGCiouLWx3bvXu3kpKSFAwGafudwOXc79mzh7aPI16P6QhK0sSJE/XRRx9p3bp1zY99+OGH+vLLLztc5Q/xyczM1K5du7Rw4ULV1NQ0P75161YtWrRIp5xyitPfMcAG10H34VroHPfee69WrlypK664QrfddlubMbT7zuF17mnzneeYY47Rrl279Morr6i+vr758c2bN+udd97RySefrEgkQtvvBC7nfvDgwbR9HPGSYrFYrLuLcFVZWanzzjtPycnJmjlzpvbu3au5c+cqJydHr7zyioLBYHeXeMR69913NWvWLH3729/W1KlTtXv3bhUXF2vfvn16+eWX2Wy1k0yYMEGDBw/WSy+91PwY10HXaev8cy3YWrdunSZNmqR+/frp9ttvb3PhhaKiItp9J3A997T5zvPmm2/q5ptv1ujRo3XBBRdox44dzed2wYIFysvLo+13EpdzT9vHka5HdQQlaf369XrwwQf16aefKhQK6fTTT9fNN9/cPF8bnefdd9/Vs88+q1WrVikUCqmgoEA33ngjN8JO1FZHROI66CrtnX+uBTsvv/yy7rnnng5jVq9eLYl2b+1wzj1tvvP8+te/1ty5c7VmzRqFw2GNHTtWs2fP1rHHHtscQ9vvHC7nnraPI1mP6wgCAAAAAOLTo/5GEAAAAAAQPzqCAAAAAJBg6AgCAAAAQIKhIwgAAAAACYaOIAAAAAAkGDqCAAAAAJBg6AgCAAAAQIKhIwgAAAAACYaOIAAAAAAkmP8PGZYR3soopNMAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_169_1.png" } }, "output_type": "display_data" } ], "source": [ "_lambda = 0.1\n", "clf_ridge = skl.Ridge(alpha=_lambda).fit(X_train, y_train)\n", "J_ridge_sk = clf_ridge.coef_.reshape(L, L)\n", "fig = plt.figure(figsize=(20, 14))\n", "im = plt.imshow(J_ridge_sk, **cmap_args)\n", "plt.title(\"Ridge from Scikit-learn\", fontsize=18)\n", "plt.xticks(fontsize=18)\n", "plt.yticks(fontsize=18)\n", "cb = fig.colorbar(im)\n", "cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the **Least Absolute Shrinkage and Selection Operator** (LASSO)-method we get a third cost function." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "\n", "$$\n", "\\begin{equation}\n", " C(\\boldsymbol{X}, \\boldsymbol{\\beta}; \\lambda) = (\\boldsymbol{X}\\boldsymbol{\\beta} - \\boldsymbol{y})^T(\\boldsymbol{X}\\boldsymbol{\\beta} - \\boldsymbol{y}) + \\lambda \\sqrt{\\boldsymbol{\\beta}^T\\boldsymbol{\\beta}}.\n", "\\label{_auto12} \\tag{12}\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Finding the extremal point of this cost function is not so straight-forward as in least squares and ridge. We will therefore rely solely on the function ``Lasso`` from **Scikit-Learn**." ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/3544313922.py:9: UserWarning: FixedFormatter should only be used together with FixedLocator\n", " cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAM2CAYAAACjUj0CAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABNwklEQVR4nO3de5xWdbko8GdmiDtUHAkRvBAnsANeyA2W7bwgAgEGpqKGV7TcZafEirJDlu6SJBU3appJGjhe4khZwmlbu03b43V32adUIEQlpMkbCMOAIO+s84fy6jjMzAusmXfNrO/385lPtdbv/a3fPKyXeN7f8z6rIkmSJAAAAMiNynIvAAAAgLYlEQQAAMgZiSAAAEDOSAQBAAByRiIIAACQMxJBAACAnJEIArzD1772tRg6dGg8//zz5V5Kqh5++OH4+Mc/HsOHD49PfepT5V5Ok5YuXRpnnXVWjBw5Mg499NAYO3ZsfPvb344XX3xxt+davHhxDB06NB577LGIiHjsscdi6NChsXjx4iZf8/zzz8fQoUPj+uuvb3B87dq1JV1z9OjRcdZZZ+32WgGgLXUq9wIAaH319fXxpS99KaqqquLSSy+Nfffdt9xL2qW5c+fGzTffHMcee2xcdNFF0bVr11i1alX87//9v2PJkiVxzz33xAEHHFDyfCNHjow5c+bE4MGDS35Nnz59Ys6cOTF06NDise9///vx05/+NH71q1/t1u8DAFklEQTIgZdeeinWr18f5513XkybNq3cy9mlmpqa+OEPfxhnnXVWzJo1q8G5SZMmxbRp0+Laa6+N6667ruQ5999//9h///13ax3du3ePyZMnNzj2yCOPRKFQ2K15ACDLlIYC5MDrr78eERE9evQo80qa9v/+3/+LQqEQH/3oRxudGzFiRBx66KHxX//1X22/MADogCSCAHvhl7/8ZZx55plxxBFHxPDhw2P06NExZ86c2L59e3HM9u3b4zvf+U4cf/zxMXz48DjmmGPi8ssvj40bNxbHJEkSN9xwQ4wbNy4OOeSQOOqoo+IrX/lK1NTUNLjehg0b4lvf+lZ87GMfi+HDh8e4cePilltuaXa36vrrr4/jjz8+IiJuuOGG4nfmrr/++jjkkEPiV7/6VXz0ox+NESNGxKJFi0q+zvXXXx8jRoyIp59+Os4777w4/PDD42Mf+1j88Ic/jCRJYv78+XHcccfFhz70oTj//PNb/M7lziT1pz/9aYP47bRgwYJYtmxZg2ObN2+OK6+8Mo499tg47LDD4sQTTyz+DhGNvyO4K0uWLIkPfvCD8YUvfCEKhUKj7wiOHj06Hn/88Vi3bt0uvztYij/+8Y9x3nnnxYgRI2LEiBExffr0+NOf/tRgTJIkcdddd8Upp5wSI0aMiEMOOSTGjx8ft9xySyRJUhw3evTomDVrVnz961+PQw45JI4++uhYv359jB49Oi677LK47777YuLEiXHIIYfE2LFjo7q6erfXC0DHpzQUYA8tWrQoZs2aFaNHj44vf/nL8frrr8evfvWrmD9/fnTv3j0+//nPR0TEFVdcEffff3+cffbZsf/++8eqVauiuro61qxZEz/60Y8iIuLmm2+OG2+8MaZNm1ZsVLNgwYJ44okn4v7774+qqqrYuHFjnH766bFu3bo4/fTTY9CgQfHQQw/FNddcE0899VSTJZMnnHBC9OrVK2bPnh0nnHBCnHDCCTF48OB4/PHHY8eOHTFr1qw4//zzY/v27XHEEUfs1nVef/31OOecc2LMmDExduzYuPfee+Pqq6+ORx99NNatWxfnnHNObNiwIW699da49NJLY+HChU3G88gjj4yBAwfGv/7rv8bvf//7GDt2bHz0ox+NkSNHxrvf/e7o3Llzg/Hbt2+PadOmxapVq2Lq1Klx8MEHx29/+9uYNWtWbN26Nc4+++wW/wwffPDB+OpXvxpHH310XHPNNVFVVdVozNe//vW45pprYsOGDXHppZc2+O5gKR566KG48MIL4+CDD44vfvGLsX379li8eHFMmzYtbrvttviHf/iHiIi47rrr4uabb46TTjoppk6dGnV1dfGzn/0srrnmmujbt2+cdNJJxTmXLFkSgwYNiv/1v/5XvPzyy9GnT5/i77Pzw4l99tkn7rnnnrjiiiti4MCBccwxx+zWugHo4BIAGvjqV7+aDBkyJFm7dm2z48aPH5+cdtppSX19ffHY66+/nhx99NHJpEmTiscOPfTQ5PLLL2/w2rlz5yaf/OQnk82bNydJkiQf//jHk8985jMNxtx1113JJz7xiWTNmjVJkiTJ9773vWTIkCHJr371qwbjvvWtbyVDhgxJli1b1uRa165dmwwZMiSZN29e8di8efMaHdud6+x8/Xe/+93imFWrViVDhgxJRowYkbzyyivF41/60peSoUOHJtu2bWtyjUmSJM8880wyZcqUZMiQIcWfD37wg8lZZ52VPPjggw3GVldXJ0OGDEl+/vOfF4/V19cnn/rUp5KPfvSjyY4dO5J77703GTJkSPLoo48mSZIkjz76aDJkyJDk3nvvTf7whz8khx12WHLOOeckr732WrOxOvPMM5Pjjjuu2bXvdNxxxyVnnnlmkiRJUigUkuOPPz45/fTTkx07dhTH1NXVJSeccEIyefLkJEmSZPv27cmHPvShZMaMGQ3mqq2tTYYPH55ceOGFDeY/+OCDi/fF248PHTo0Wb58efHYiy++mAwdOjS55JJLSlo7APmhNBRgD/385z+PW265JSoqKorHXnnllejdu3ds2bKleGzfffeNpUuXxuLFi2PTpk0REXHxxRfHvffeWyyH3HfffeOxxx6LH//4x/Hyyy9HRMTpp58e9913X7FL5m9+85sYPHhwjBkzpsE6Pve5z0VExL/927/t0e/xj//4jw3+9+5e5+3jDjrooIiI+NCHPlTcpYqIGDhwYCRJUvzdmjJo0KBYvHhxLFiwIM4+++wYPHhwFAqFeOyxx+L888+PW265pTh22bJl0adPn5g0aVLxWEVFRcyZMyeqq6ujsrLp/4tbtWpVXHjhhTFgwIC46aabokuXLs2ua0899dRTsXbt2hgzZkxs3Lgx1q9fH+vXr4/XXnstjjvuuFi+fHn8/e9/j3e9613x8MMPxxVXXNHg9Rs2bIiePXs2uJ8iIg444IBddk8dNGhQHHzwwcX/3bdv39hnn31ajDsA+aM0FGAPvetd74r//M//jPvvvz+eeeaZ+Otf/xqvvPJKREQMGDCgOO5b3/pWXHzxxXHppZfGN77xjTj88MPjhBNOiJNPPjl69eoVEREzZ86Mz372s3HllVfG7NmzY9iwYTF69OiYOnVq9O3bNyLeeL7dxz72sUbr6Nu3b/Tu3TvWrVu3R7/Hf/tv/63B/97d6+yzzz7F/96pU6ddzrmz5LK+vr7F9VRUVMSRRx4ZRx55ZERE/O1vf4t77703fvCDH8S//Mu/xOTJk6Nfv36xbt26OOCAAxok4hENY9+UH/3oR1FZWRmvvfZavPTSS7v1SIqIiLq6ukbJWa9evaJr164Njv31r3+NiIg5c+bEnDlzdjlXTU1N7LvvvvGud70rli1bFv/2b/8Wzz77bKxZs6b4PdLkbd8RjGgc353ennzv1Llz55LiDkC+SAQB9tA111wTt9xyS/yP//E/4vDDD4/JkyfHiBEj4p//+Z8bNHn5yEc+Ev/+7/9e/HnooYdi9uzZcfvtt8fixYujT58+cfDBB8e//uu/xoMPPhj//u//Hg8++GDMmzcvbr/99rj77rtj8ODBjZKBt6uvr493vetde/R7vHPnbHevs6vv1b0zOSvFwoULY9u2bXHBBRc0OL7ffvvF//yf/zO6dOkS11xzTfzXf/1XjBs3LgqFwh5dJyJi6NChcdlll8W5554bl19+ecyfP3+3Xv+jH/0obrjhhgbHZs+eHZ/85CcbHNuZgH3xi1+Mww8/fJdzvf/9748kSeIrX/lK3H///XHEEUfEiBEj4rTTTouRI0fGOeec0+g1u4p5ROM/SwBoikQQYA+sW7cubrnllpg8eXKjnZ63l+Ft3749li9fHvvuu29MnDgxJk6cGPX19XHbbbfFnDlzYsmSJfGpT30qVqxYET179ozjjz++2OFz6dKlMWPGjFi0aFF87WtfiwEDBsQzzzzTaC0vvfRSbN68Ofr375/K79ZW13mnX//61/GnP/0pPvWpT0X37t0bnR8yZEhERHHXbb/99ouVK1c2Gvfb3/42li5dGl/5yleavNa5554b//AP/xDnnntu/PCHP4z777+/QYlpS6ZMmRJHHHFEg2P//b//90bjdu5Odu/ePY466qgG5/70pz/Fxo0bo2vXrvG73/0u7r///vjc5z4XX/ziF4tjduzYEa+++upuPwsRAFrio0OAPbCzZO+d//j/7W9/G88991zs2LEjIiJeffXVOO200+IHP/hBcUxlZWUccsghxf9eKBTi7LPPjiuvvLLBXIcddlhxTETEcccdF88880z8+te/bjBu5/fmjj322FR+t7a6zjudeOKJsWXLlvjud7/bqJSxvr4+Fi1aFL17946RI0dGRMTRRx8dL7/8cvzqV79qMPbHP/5xLFu2LN773ve2eM3Pfe5z0b9//5g9e3bx+5u7UllZ2WBN+++/fxx11FENft73vvc1et3w4cOjb9++sXDhwqirqyse37x5c7FcuKqqKl599dWIaHw//eQnP4mtW7cW7ycASIsdQYAmzJ07d5cPYP/4xz8eRxxxROy3335x8803x7Zt22LfffeNP/3pT/HTn/40unTpUvxH//ve97448cQT484774ytW7fGiBEj4tVXX4077rgj9tlnn/j4xz8enTt3jrPOOituuummuOiii+JjH/tYvPbaa3HPPfdEt27d4uSTT46IiAsvvDAeeOCBuPjii+OMM86Igw46KB599NF44IEHYuzYsak9HqCtrvNOn/zkJ+PBBx+Me+65J/74xz/G+PHjY999941XXnkl/s//+T+xcuXKuOaaa4q7haeffnrce++9MWPGjJg2bVoMGjQoli1bFg899FBceeWVTZZPvl337t3ja1/7Wnzxi1+Mq6++ulGzlp369OkT//mf/xm33XZbfOhDHyom6S1517veFd/4xjfi4osvjk9+8pNxyimnRJcuXWLRokXxt7/9La6++uro1KlTjBgxInr27BmzZ8+Ov/3tb9G7d+947LHHYunSpQ3uJwBIi0QQoAn333//Lo+///3vj4985CNxyy23xHe/+91YsGBBJEkSBxxwQHz961+PHTt2xHe+85144oknYvjw4fHP//zPsf/++8eSJUtiyZIl0a1bt/jIRz4SM2bMKDb3+MIXvhDvec974t57742rrroqqqqq4kMf+lB873vfi8GDB0dExHve856455574rrrroulS5fGpk2bYv/994+ZM2fGueeem9rv3VbXeafKysq47rrr4r777ov77rsv7rjjjqitrY13v/vdccQRR8Tll18ehx56aHF8165dY+HChXHdddfFkiVLora2NgYPHhzXXXddfPzjHy/5uuPHj49//Md/jJ/85Cdx0kknFZvzvN0FF1xQTEQ/+clPlpwIRkSMGzcufvSjH8VNN90U3//+96OysjI+8IEPxE033RTHHXdcRLzRcOeWW26Jq6++Or7//e9H586dY9CgQXHttdfGn/70p1iwYEG8/PLLDRrzAMDeqEia6woAAABAh+M7ggAAADkjEQQAAMgZiSAAAEDOaBYDAADQQW3atGmXj0jSLAYAAGAvbV2/Prq92Q08S1577bU4+uiji89A3im7ieA7HibcSGVly2Mi4qD3/6DFMaV67pkLU5urXSsx9rQCsS8v8S8fsS8fsS8fsS8fsW/goPf/cK/nGDiwZ/zf/zsthdVk24/+8R9j0/PPl3sZRb0HDozp//f/Rk1NTRQKhQbnOnxp6Jo1G1seBAAA7NKaNY3LCtm1Tc8/HxvXrCn3Mhrp379/o2MdPhEEAABoC5WRrW6cza0l9XWuXbs2Pv/5z8eoUaNi1KhRMXPmzFi/fn3alwEAAGAPpbojuGHDhjjnnHNi+/btccEFF0ShUIj58+fHypUrY9GiRdG5c+c0LwcAAMAeSDURvP322+Pvf/97/OIXv4jBgwdHRMRhhx0W5513XvzsZz+LqVOnpnk5AACAzMhtaeiSJUti1KhRxSQwIuKoo46KQYMGxZIlS9K8FAAAAHsotURw48aNsXbt2hg2bFijc8OGDYsnnngirUsBAACwF1IrDX3hhRciIqJfv36NzvXt2zc2b94ctbW10atXr7QuCQAAkBntqTQ0tUSwrq4uIiK6devW6FyXLl0iImLLli2lJ4KVJYSwhDFJ8rXSrsfuKeXPh9Yh9uUl/uUj9uUj9uUj9uUj9kVJ8pVyL4FWkFoiWF9f3+KYyt15Q7U0X2Vly2MioqJqTunXbEFSmJnaXO1aibGnFYh9eYl/+Yh9+Yh9+Yh9+Yh9AxVV1+z1HAce2Duee+7CFFZDWlJLBHv06BEREdu2bWt0buexnWMAAAA6morIVmloRTPnUlvnfvvtFxERL730UqNzL774YvTu3Tu6d++e1uUAAADYQ6klgr17946BAwfGk08+2ejcU089FcOHD0/rUgAAAOyFVHcux44dG4888kisXr26eOzhhx+OZ599NiZMmJDmpQAAADKlMoM/TUntO4IREZ/+9Kfjvvvui3PPPTemT58e27Zti1tvvTWGDRsWkydPTvNSAAAA7KFUdwT79OkTd9xxRxx88MExb968+PGPfxxjxoyJW2+9NTp37pzmpQAAANhDFUmSJOVexK7U1kY0t7LevSM2bWp5nt4902v9W1E1O7W5ksKlqc3V5rRULh+xLy/xLx+xLx+xLx+xLx+xbx05eDbjDw86KDatWVPuZRT1PvDA+PRzz+3yXMf/0wAAAKABiSAAAEDOpNosBgAAIK9a6tTZ1ppbS5bWCQAAQBuQCAIAAOSM0lAAAIAUVES2dtoqmjmXpXUCAADQBiSCAAAAOaM0FAAAIAW6hgIAAJBZEkEAAICcURoKAACQgopovlNnW9M1FAAAgCKJIAAAQM4oDQUAAEhB1Zs/WdHcWuwIAgAA5IxEEAAAIGeUhgIAAKSgIrK109Zc19DMJoK9etS3MKIyevdsaUzE3/6e3h9FUrg0tbkqqmamNldSmJPaXAAAQMeXpYQVAACANpDZHUEAAID2pDKytdPW3FqytE4AAADagEQQAAAgZ5SGAgAApEBpKAAAAJklEQQAAMgZpaEAAAApUBoKAABAZkkEAQAAckZpKAAAQAoqIls7bRXNnMvSOgEAAGgDEkEAAICcURoKAACQAl1DAQAAyCyJIAAAQM4oDQUAAEhBRTTfqbOt6RoKAABAUYffEdxv3/rU5qpPMW9OCnNSm6ui6gupzZUU5qU2FwAAkE0dPhEEAABoC1Vv/mRFc2tRGgoAAJAzEkEAAICcURoKAACQgorI1k6brqEAAAAUSQQBAAByRmkoAABACiojWzttza0lS+sEAACgDUgEAQAAckZpKAAAQAqUhgIAAJBZEkEAAICcURoKAACQAqWhAAAAZJZEEAAAIGeUhgIAAKSgIrK101bRzLksrRMAAIA2IBEEAADIGaWhu6Ey6lOba8tr6eXgSWFeanNVVF3U8vWSm0obV7gxjSUBAEC7oGsoAAAAmSURBAAAyBmloQAAACmoiOY7dbY1XUMBAAAokggCAADkjNJQAACAFFS9+ZMVza3FjiAAAEDO2BEEAABIQUVka6dNsxgAAACKJIIAAAA5ozQUAAAgBZWRrZ225taSpXUCAADQBiSCAAAAOaM0FAAAIAVKQwEAAMgsiSAAAEDOKA0FAABIgdJQAAAAMsuOYJl071qf2lwvvJRePp8UbkxtXEXVzL1dztuuNye1uQAAIE9qamqiUCg0OCYRBAAASEFFZKvksuLN/5w2bVqsW7euwTmJIAAAQAdWXV1tRxAAACBP+vfv3+iYRBAAACAF7alraOqJ4CmnnBJ//vOfGx0fN25czJs3L+3LAQAAsJtSTQSTJInVq1fHmDFjYuzYsQ3ODRgwIM1LAQAAsIdSTQSff/752LJlSxx//PExefLkNKcGAADItIp4q1NnFjS3llRLWJ9++umIiBg8eHCa0wIAAJCiVBPBVatWRcRbieCWLVvSnB4AAIAUpJ4I9ujRI2bPnh0jRoyIESNGxJgxY2LJkiVpXgYAACBzqjL405RUvyP49NNPR11dXdTW1sacOXNi06ZNsWDBgrjkkkvi9ddfjylTppQ+WWUJOWopY3KgX780ZysxpiXEPkmu3su1sEvu+/IS//IR+/IR+/IR+/IRezq4iiRJkrQmu+uuu6K+vj6mTZtWPPbaa6/FpEmTYuvWrfEf//EfUVXVXF76NvX1zZ+vrGx5TE688FJ6f1H161tCTEuMfUXVzBRW9IakMCe1udo19315iX/5iH35iH35iH35iH3ryEFy/YeDDopta9aUexlFXQ48MD703HO7PJfqn8YZZ5zRIAmMiOjatWtMnjw5Xn755WIzGQAAgI6mIt56qHwWftqsa2hT+vTpExGaxwAAAGRBaongCy+8EBMnTowbbrih0blnn302IiIGDhyY1uUAAADYQ6klgv369Yva2tpYtGhRbN68uXi8pqYmFi9eHEceeWT07ds3rcsBAABkSrlLQXf105RUu4ZedtllcdFFF8Xpp58ep556atTV1UV1dXV06tQpvvnNb6Z5KQAAAPZQqt8RHDNmTNx4443RrVu3uPrqq+O2226Lww8/PO66667iQ+YBAAAor1QfH5Eqj48oi/WvtvzZQJ8+EevXtzxXn/ek9+dTUXVRanMlhRtTm6vNue/LS/zLR+zLR+zLR+zLR+xbRw4eH/Hngw6K7Rl6fETnAw+MQ9ri8REAAABkn0QQAAAgZ1JtFgMAAJBXLXXqbGvNrSVL6wQAAKANSAQBAAByRmkoAABACioiWzttFc2cy9I6AQAAaAMSQQAAgJxRGgoAAJACXUMBAADILIkgAABAzigNBQAASEFFNN+ps63pGgoAAECRRBAAACBnlIYCAACkoCoi6su9iLepauacHUEAAICckQgCAADkjNJQAACAFFREtnbamusaKhGkgT7vKaWqubKkcdt3pPc2SAo3pjZXRdWs1OZKCt9ObS4AAGgrWUpYAQAAaAN2BAEAAFJQGdnaaWtuLVlaJwAAAG1AIggAAJAzSkMBAABSoDQUAACAzJIIAgAA5IzSUAAAgBQoDQUAACCzJIIAAAA5ozQUAAAgBRWRrZ22imbOZWmdAAAAtAGJIAAAQM4oDQUAAEhBZUQk5V7E2+gaCgAAQJFEEAAAIGeUhgIAAKSgIprv1NnWdA0FAACgyI4graZzp/rU5nrhpfQ+s0gK305troqqf0ptrqRwc2pzAQBAcySCAAAAKagq9wLeobn1KA0FAADIGYkgAABAzigNBQAASEFFZGunTddQAAAAiiSCAAAAOaM0FAAAIAVZ22Vrbj1ZWysAAACtTCIIAACQM0pDAQAAUpC1XTaloQAAABRJBAEAAHJGaSgAAEAKsrbLpjQUAACAIokgAABAzigNBQAASEFFZGunraKZc1laJwAAAG1AIggAAJAzSkNpF/r1rU9trvoUP/9ICjenNldF1Zdavl4yt7RxhWvSWBIAALuhMiKSci/ibZSGAgAAUCQRBAAAyBmloQAAAClQGgoAAEBmSQQBAAByRmkoAABACpSGAgAAkFkSQQAAgJxRGgoAAJCCimi+HDNL7AgCAADkjEQQAAAgZ5SGAgAApKA97bK1p7UCAACQAokgAABAzigNBQAASEF72mVrT2sFAAAgBRJBAACAnFEaCgAAkIL2tMsmESR3KqM+tbm270jv7Z4UrkltXEXVjL1dzjuuOTfV+QAAKC+JIAAAQAdWU1MThUKhwTGJIAAAQAoqKioiKirKvYy3vLmWadOmxbp16xqckggCAAB0YNXV1XYEAQAAWkWnTm/8ZMWba+nfv3+jU3vU6WLWrFlx1llnNTq+du3a+PznPx+jRo2KUaNGxcyZM2P9+vV7cgkAAABayW6nq4sWLYpFixbFqFGjGhzfsGFDnHPOObF9+/a44IILolAoxPz582PlypWxaNGi6Ny5c2qLBgAAYM+VnAgWCoW46aab4oYbbtjl+dtvvz3+/ve/xy9+8YsYPHhwREQcdthhcd5558XPfvazmDp1ajorBgAAyKKqqmyVhlZVNXmqpNLQbdu2xUknnRTXX399TJ48Ofr169dozJIlS2LUqFHFJDAi4qijjopBgwbFkiVL9mDVAAAAtIaSE8HNmzfH3Llz46qrropO78hyN27cGGvXro1hw4Y1eu2wYcPiiSeeSGe1AAAA7LWS9i179uwZDzzwQKMEcKcXXnghImKXO4V9+/aNzZs3R21tbfTq1WsvlgoAAJBhGe0austTpby+srIyKiub3jysq6uLiIhu3bo1OtelS5eIiNiyZcvuJYLNXG+3xtA6xD4iItLtgVRiTEuIfZL8y16uhSa598tH7MtH7MtH7MtH7OngUklX6+vrWxzTXCLZxKQtTdjyGFqH2Bdt35He/0l07lRCTEuMfUXVjBRW9JakMDfV+dot9375iH35iH35iH35iH3rkFxnSiqJYI8ePSLije8SvtPOYzvHAAAAdEgdrWtoS/bbb7+IiHjppZcanXvxxRejd+/e0b179zQuBQAAwF5KJRHs3bt3DBw4MJ588slG55566qkYPnx4GpcBAAAgBakV6o4dOzYeeeSRWL16dfHYww8/HM8++2xMmDAhrcsAAABk086uoVn6aWqpaf3On/70p+O+++6Lc889N6ZPnx7btm2LW2+9NYYNGxaTJ09O6zIAAADspdR2BPv06RN33HFHHHzwwTFv3rz48Y9/HGPGjIlbb701OqfbYx8AAIC9sEc7gr/5zW92efz9739//PCHP9yrBQEAALRL7ahraIZWCe1PSc/+K1F9CRv0lSWOS/u5fxVVX0ptrqRwTWpzAQCwZzzVEQAAIGfsCAIAAKShhU6dba6ZtdgRBAAAyBmJIAAAQM5kaN8SAACgHWtHXUPtCAIAAOSMRBAAACBnMrRvCQAA0I7pGgoAAEBWSQQBAAByJkP7lgAAAO2YrqEAAABklUQQAAAgZzK0bwkAANCO6RoKAABAVkkEAQAAciZD+5YAAADtmK6hAAAAZFWG0lXIt8qoL2lUKePqU/6MJylck9pcFVWXpjZXUpid2lwAAHkiEQQAAEiDrqEAAABklUQQAAAgZzK0bwkAANCO6RoKAABAVkkEAQAAciZD+5YAAADtmK6hAAAAZJVEEAAAIGcytG8JAADQjukaCgAAQFZJBAEAAHImQ/uWAAAA7ZiuoQAAAGSVRBAAACBnMrRvCQAA0I7pGgoAAEBWSQQBAAByJkP7lkBaKqM+1fnqU/zMKCnMTm2uiqqzUpsrKSxMbS4AIKd0DQUAACCrJIIAAAA5k6F9SwAAgHZM11AAAACySiIIAACQMxnatwQAAGjHdA0FAAAgqySCAAAAOZOhfUsAAIB2TNdQAAAAskoiCAAAkDMZ2rcEAABox3QNBQAAIKskggAAADmToX1LAACAdkzXUAAAALJKIggAAJAzGdq3BAAAaMfaUdfQDK0SyKrKqC/3EnYpKSxMba6Kqm+3fL3kstLGFWalsSQAgFajNBQAACBn7AgCAACkQddQAAAAskoiCAAAkDMZ2rcEAABox9pR11A7ggAAADkjEQQAAMiZDO1bAgAAtGO6hgIAAJBVEkEAAICcydC+JQAAQDumaygAAABZJREEAADImQztWwIAALRjuoYCAACQVRJBAACAnMnQviUAAEA7pmsoAAAAWZWhdBVg99Sn+FlWUpiV2riKqkv3djlvu97s1OYCAPKppqYmCoVCg2MSQQAAgDRktGvotGnTYt26dQ1OZWiVAAAApK26utqOIAAAQJ7079+/0bE9SgRnzZoVa9asiYULFzY4fsopp8Sf//znRuPHjRsX8+bN25NLAQAAtA/tqGvobq9y0aJFsWjRohg1alSD40mSxOrVq2PMmDExduzYBucGDBiwu5cBAACglZScCBYKhbjpppvihhtu2OX5559/PrZs2RLHH398TJ48ObUFAgAAkK6SEsFt27bFqaeeGitXrowpU6bEI4880mjM008/HRERgwcPTneFAAAA7UFGu4buSkkP4dq2bVts3rw55s6dG1dddVV02sUvt2rVqoh4KxHcsmXLniwVAACAVlZSutqzZ8944IEHdpkA7rRq1aro0aNHzJ49O5YuXRpbtmyJ/fffP2bMmBETJ05MbcEAAADsnZISwcrKyqisbH7z8Omnn466urqora2NOXPmxKZNm2LBggVxySWXxOuvvx5TpkzZvZW1cL2Sx9A6xL58xL4o3UiUOFsJ8U+Sq/ZyLeySe798xL58xL58xJ490ZG7hjZl6tSpUV9fH9OmTSsemzhxYkyaNCm+973vxYknnhhVzdSoNlJf3/z5ysqWx9A6xL58xL6B+hRTwcooIa4lxr+i6tIUVvSGpDA7tbnaNfd++Yh9+Yh9+Yh965BcZ0pqfxpnnHFGgyQwIqJr164xefLkePnll4vNZAAAACivVt+37NOnT0RoHgMAAHRwHa1raEteeOGFmDhx4i6fMfjss89GRMTAgQPTuBQAAAB7KZVEsF+/flFbWxuLFi2KzZs3F4/X1NTE4sWL48gjj4y+ffumcSkAAAD2Umr7lpdddllcdNFFcfrpp8epp54adXV1UV1dHZ06dYpvfvObaV0GAAAgm9pR19DUmsWMGTMmbrzxxujWrVtcffXVcdttt8Xhhx8ed911V/Eh8wAAAJTfHqWrv/nNb3Z5fMyYMTFmzJi9WhAAAEC71I6axWRolQC7p6Rn/5Vo0+aWCyR69y5tXJrP/vNMQgCgNXiqIwAAQM7YEQQAAEhDHpvFAAAA0D5IBAEAAHImQ/uWAAAA7Vg76hpqRxAAACBnJIIAAAA5k6F9SwAAgHZM11AAAACySiIIAACQMxnatwQAAGjHdA0FAAAgqySCAAAAOZOhfUsAAIB2TNdQAAAAskoiCAAAkDMZ2rcEAABox3QNBQAAIKsylK4ClE/vnvUljKosadz2Hel9xpYUZqc2V0XVF1KbKynMS20uAKDtSQQBAADSoGsoAAAAWSURBAAAyJkM7VsCAAC0Y7qGAgAAkFUSQQAAgJzJ0L4lAABAO6ZrKAAAAFklEQQAAMiZDO1bAgAAtGO6hgIAAJBVEkEAAICcydC+JQAAQDvWqVNEoVDuVbxF11AAAAB2kggCAADkjNJQAACANOgaCgAAQFZJBAEAAHImQ/uWAB1D5071qc21/tX0Pq9LCvNSm6ui6vLU5koK30xtLgAoq06dIurT+3fAXtM1FAAAgJ0kggAAADmjNBQAACANuoYCAACQVRJBAACAnMnQviUAAEA71qlTRJKUexVv0TUUAACAnSSCAAAAOaM0FAAAIA1VVdkqDdU1FAAAgJ0kggAAADmjNBQAACANWXqYfISuoQAAALxFIggAAJAzGdu7BAAAaKea6dJZFrqGAgAAsJNEEAAAIGeUhgIAAKShU6eIiopyr+ItzZSGSgQBMqzPe+pTm2v7jvSKQJLCN1Obq6JqVsvXS64sbVzh22ksCQA6PKWhAAAAOWNHEAAAIA1VVdkqDa1set/PjiAAAEDOSAQBAAByRmkoAABAGjp1iqhPr9HbXlMaCgAAwE4SQQAAgJxRGgoAAJCGqqpmyzHbXDMdTDO0SgAAANqCRBAAACBnlIYCAACkoVOniCQp9yreojQUAACAnSSCAAAAOaM0FAAAIA1VVeVeQcnsCAIAAOSMRBAAACBnlIYCAACkoVP7Sa/sCAIAAORMRZJk6UEXb1Nf3/z5ysqWx9A6xL58xL68xL9oy2vpfY7YvWsJMS0x9hVVs1JY0RuSwrdTm6tdc9+Xj9iXj9i3jsoc7EFl8b6prIyampooFAoNDrefvUsAAIAMq89gwWVlREybNi3WrVvX4LhEEAAAoAOrrq62IwgAAJAn/fv3b3Ss5ETwwQcfjJtuuimefPLJqKysjMMOOywuvvjiOPzww4tj1q5dG1dddVU8/vjjERFx7LHHxte+9rXo06fP3q8eAAAgw3bsKPcKGuvcedfHS2oW8/jjj8fZZ58dH/jAB+Lkk0+OHTt2xJ133hkvvvhi3HnnnXHooYfGhg0b4uSTT47t27fH2WefHYVCIebPnx8DBgyIRYsWReemVtAUzWKyS+zLR+zLS/yLNIvJEfd9+Yh9+Yh968hBs5jt28u9gsaaSsNK2hG88soro3///vGTn/wkunXrFhERU6ZMiQkTJsTcuXPjtttui9tvvz3+/ve/xy9+8YsYPHhwREQcdthhcd5558XPfvazmDp1ajq/CQAAAHulxbR848aNsWLFihg/fnwxCYyI2GeffWLkyJHxxz/+MSIilixZEqNGjSomgRERRx11VAwaNCiWLFnSCksHAADIjkLhjfLQrPy8oz9MAy3uCPbs2TN++ctfNkgCd9qwYUNUVVXFxo0bY+3atTFu3LhGY4YNGxbLli3brQACAADQelrcEayqqoqDDjoo+vXr1+D4ihUr4g9/+EOMGDEiXnjhhYiIRmMiIvr27RubN2+O2tralJYMAADA3tijx0fU1dXFV7/61YiI+MxnPhN1dXUREbvcNezSpUtERGzZsiV69epV+kVK+TJpDr5wmlliXz5iX17iHxER3bunOVuJMS0h9kly5V6uhV1y35eP2JeP2LMHduyIaLkVZ9upqGj63G4nglu3bo3PfvazsWLFirjwwgtj1KhR8fvf/77F11Xu7ptJ19DsEvvyEfvyEv8iXUNzxH1fPmJfPmLfOiTXmbJbfxqbNm2K6dOnx2OPPRYnn3xyzJgxIyIievToERER27Zta/Sancd2jgEAAKC8St4RfOWVV+L888+P5cuXx2mnnRaXX355VLy517jffvtFRMRLL73U6HUvvvhi9O7dO7qnW0cEAACQKYVCtjaTm9uELSkR3Lx5czEJPPfcc+PSSy9tcL53794xcODAePLJJxu99qmnnorhw4fv3ooBAABoNSWVhl5xxRWxfPnyOPvssxslgTuNHTs2HnnkkVi9enXx2MMPPxzPPvtsTJgwIZ3VAgAAsNcqkqT5vjarV6+OCRMmRK9eveLrX/96VFVVNRozefLkWL9+fUyaNCmqqqpi+vTpsW3btrj11lvjgAMOiLvvvjs6d+68eyvTLCa7xL58xL68xL9Is5gccd+Xj9iXj9i3jhw0i3n55WzdOpWVEfvss+tzLZaGPv744xERUVtb2+Ru4OTJk6NPnz5xxx13xOzZs2PevHnRtWvXGDNmTMycOXP3k0AAAABaTYs7gmVjRzC7xL58xL68xL/IjmCOuO/LR+zLR+xbhx3BNrdXO4IA8E4lJW8lqi/h6+qVJY5LM3mrqPqn1OZKCjenNhcA2VUovPGTFbv4Vl9Rx0/LAQAAaEAiCAAAkDNKQwEAAFKwY0e2SkOb6wZjRxAAACBnJIIAAAA5ozQUAAAgBYXCG+Wh7YEdQQAAgJyRCAIAAOSM0lAAAIAU7NihNBQAAICMkggCAADkjNJQAACAFGSta2hFRdPn7AgCAADkjB1BAACAFGStWYwdQQAAAIokggAAADmjNBQAACAFWWsWU9nMtp8dQQAAgJyxIwhAWVVGfUmjShuXnqRwc2pzVVTNSm2upPDt1OYCIL8kggAAACnIWtdQpaEAAAAUSQQBAAByRmkoAABACrLWNbSqqulzdgQBAAByRiIIAACQM0pDAQAAUpC1rqFKQwEAACiSCAIAAOSM0lAAAIAUZK1raKdmsj07ggAAADkjEQQAAMgZpaEAAAApyFrX0ObWYkcQAAAgZySCAAAAOaM0FAAAIAVZ6xpaKDR9zo4gAABAzkgEAQAAckZpKADswvYd6X1WmhS+ndpcFVXnpzZXUpif2lwA6BoKAABAhkkEAQAAckZpKAAAQAp0DQUAACCzJIIAAAA5ozQUAAAgBbqGAgAAkFkSQQAAgJxRGgoAAJACXUMBAADILIkgAABAzigNBQAASIGuoQAAAGSWRBAAACBnlIYCAACkQNdQAAAAMksiCAAAkDNKQwEAAFLQnrqGSgQBYBc6d6pPba76FAtwksL81OaqqJrZ8vWSq0sbV5iTxpIAaCNKQwEAAHLGjiAAAEAKdA0FAAAgsySCAAAAOaM0FAAAIAXtqWuoHUEAAICckQgCAADkjNJQAACAFOgaCgAAQGZJBAEAAHJGaSgAAEAKdA0FAAAgsySCAAAAOaM0FAAAIAW6hgIAAJBZEkEAAICcURoKAACQAl1DAQAAyCw7ggDQyiqjvtxL2KWkMCe1cRVV397b5bzterNSmwuAiJqamii8o3OMRBAAACAFWe0aOm3atFi3bl2DcxJBAACADqy6utqOIAAAQJ7079+/0bGSE8EHH3wwbrrppnjyySejsrIyDjvssLj44ovj8MMPL4455ZRT4s9//nOj144bNy7mzZu3Z6sGAABoB9pT19CSEsHHH388Pv3pT8cHPvCBmDFjRuzYsSPuvPPOOPPMM+POO++MQw89NJIkidWrV8eYMWNi7NixDV4/YMCAvfoFAAAASE9JieCVV14Z/fv3j5/85CfRrVu3iIiYMmVKTJgwIebOnRu33XZbPP/887Fly5Y4/vjjY/Lkya26aAAAAPZci4ngxo0bY8WKFXHeeecVk8CIiH322SdGjhwZDz30UEREPP300xERMXjw4FZaKgAAQHZltWvorrSYCPbs2TN++ctfNkgCd9qwYUNUVVVFRMSqVasi4q1EcMuWLdG9e/c9WS8AAACtqLKlAVVVVXHQQQdFv379GhxfsWJF/OEPf4gRI0ZExBuJYI8ePWL27NkxYsSIGDFiRIwZMyaWLFnSOisHAABgj+zR4yPq6uriq1/9akREfOYzn4mIN0pD6+rqora2NubMmRObNm2KBQsWxCWXXBKvv/56TJkyZfcuUtlijlraGFqH2JeP2JeX+JeP2JdPCbFPksvaYCE55L4vH7FnD3S4rqFvt3Xr1vjsZz8bK1asiAsvvDBGjRoVERFTp06N+vr6mDZtWnHsxIkTY9KkSfG9730vTjzxxGIZaUnq65s/X1nZ8hhah9iXj9iXl/iXj9iXT4mxr6j6dmqXTAqzUpurXXPfl4/Ytw7Jdabs1p/Gpk2bYvr06fHYY4/FySefHDNmzCieO+OMMxokgRERXbt2jcmTJ8fLL79cbCYDAABAeZW8I/jKK6/E+eefH8uXL4/TTjstLr/88qioqGjxdX369ImIN5rHAAAAdFTtqWtoSTuCmzdvLiaB5557blxxxRUNksAXXnghJk6cGDfccEOj1z777LMRETFw4MDdXDYAAACtoaRE8Iorrojly5fH2WefHZdeemmj8/369Yva2tpYtGhRbN68uXi8pqYmFi9eHEceeWT07ds3vVUDAACwx1osDV29enXcd9990atXr/jgBz8Y9913X6MxkydPjssuuywuuuiiOP300+PUU0+Nurq6qK6ujk6dOsU3v/nNVlk8AABAVnSorqGPP/54RETU1tbucjcw4o1EcMyYMXHjjTfGD37wg7j66quja9euMWrUqLjkkkuKD5kHAACg/CqSJEnKvYhd8viI7BL78hH78hL/8hH78vH4iPJx35eP2LeOHDw+YtasiPXry72Kt/TpE/HtJv563qMHygMA7d/2HS3/o6xz59LGpZm8VVR9IbW5ksK81OYCaEmH6xoKAABAxyERBAAAyBmloQAAACloT11D7QgCAADkjEQQAAAgZ5SGAgAApEDXUAAAADJLIggAAJAzSkMBAABSoGsoAAAAmSURBAAAyBmloQAAACnQNRQAAIDMkggCAADkjNJQAACAFOgaCgAAQGZJBAEAAHJGaSgAAEAK2lPXUIkgAORU5071JYyqLGlcfYpFRklhXmpzVVSdn9pcSWF+anMBlJvSUAAAgJyxIwgAAJACXUMBAADILIkgAABAzigNBQAASEF76hpqRxAAACBnJIIAAAA5ozQUAAAgBbqGAgAAkFl2BAEAAFKgWQwAAACZJREEAADIGaWhAAAAKdAsBgAAgMySCAIAAOSM0lAAAIAU6BoKAABAZkkEAQAAckZpKACw1yqjPrW56lP8nDopzE9troqqa1KbKyl8KbW5gOzQNRQAAIDMkggCAADkjNJQAACAFOgaCgAAQGZJBAEAAHJGaSgAAEAKdA0FAAAgsySCAAAAOaM0FAAAIAW6hgIAAJBZEkEAAICcURoKAACQAl1DAQAAyCyJIAAAQM4oDQUAAEiBrqEAAABklkQQAAAgZ5SGAgAApKA9dQ2VCAIAmVIZ9anNteW19IqfksKXUpurouqilq+X3FTauMKNaSwJyBmloQAAADljRxAAACAFuoYCAACQWRJBAACAnFEaCgAAkIL21DXUjiAAAEDOSAQBAAByRmkoAABACnQNBQAAILMkggAAADmjNBQAACAFuoYCAACQWRJBAACAnFEaCgAAkAJdQwEAAMgsiSAAAEDOKA0FAABIga6hAAAAZJYdQQCgw+retT61uepT/Pw8KdyY2riKqn/a2+W845o3pzofUH41NTVReEfnGIkgAABACpKkPpKk3Kt4yxtrqYxp06bFunXrGpyTCAIAAHRg1dXVdgQBAADypH///o2OlVzs/sgjj8QZZ5wRI0aMiI997GPxne98J+rq6hqMWbt2bXz+85+PUaNGxahRo2LmzJmxfv36vV85AABA5hUy+LNrJe0IPvroozF9+vQYNmxYfPnLX46amppYsGBBPPHEE1FdXR2VlZWxYcOGOOecc2L79u1xwQUXRKFQiPnz58fKlStj0aJF0blz55JCBwAAQOsqKRGcM2dO9O/fP+64447o2rVrRLyxvXjFFVfEgw8+GMccc0zcfvvt8fe//z1+8YtfxODBgyMi4rDDDovzzjsvfvazn8XUqVNb77cAAACgZC2Whm7bti3e+973xtSpU4tJYETEqFGjIiJi5cqVERGxZMmSGDVqVDEJjIg46qijYtCgQbFkyZK01w0AAJAx9VH+UtC3/zT9CJ0WdwS7dOkS8+fPb3R8+fLlERGx3377xcaNG2Pt2rUxbty4RuOGDRsWy5Yta+kyAAAAtJHd7hq6bt26eOyxx+Kqq66KIUOGxAknnBBr1qyJiIh+/fo1Gt+3b9/YvHlz1NbWRq9evfZ+xQAAAOyV3UoEX3311Rg9enRERHTr1i1mzZoVXbp0KXYP7datW6PXdOnSJSIitmzZIhEEAAA6sEJEZOiJ8lHR5JndSgQrKipi7ty5sX379li4cGGcd955ce2110bfvn1bfG1lZclPqtj5gnTG0DrEvnzEvrzEv3zEvnzEPiJ245lbac5WQuyT5Ja9XAu75L6ng9utRPDd7353TJgwISIixo8fH5MmTYrvfve7cfPNN0fEG41l3mnnsR49euzeyuqb/mJjRLzx5mxpDK1D7MtH7MtL/MtH7MtH7IvqU0wFK5tp4PDWoNJiX1H1Tyms6C1J4eZU52uX3PetQ3KdKXv8p9G1a9c49thjo6amJt73vvdFRMRLL73UaNyLL74YvXv3ju7du+/5KgEAADKvPoM/u9ZiIrh69eoYPXp0VFdXNzpXV1cXFRUV0blz5xg4cGA8+eSTjcY89dRTMXz48JYuAwAAQBtpMRE88MADo7a2Nu6+++7Yvn178fi6devigQceiJEjR0bPnj1j7Nix8cgjj8Tq1auLYx5++OF49tlni+WkAAAAlF9FkiQttrW57777YubMmXH44YfHJz7xidiwYUNUV1fH66+/HnfeeWcMGTIk1q9fH5MmTYqqqqqYPn16bNu2LW699dY44IAD4u67747OnTvv3sp8RzC7xL58xL68xL98xL58xL7IdwRzxH3fOnLwHcGDDtoUa9Zk59458MDKeO653rs8V1IiGBGxdOnSuPXWW+Mvf/lLdO/ePT784Q/HjBkzYtCgQcUxzzzzTMyePTt+97vfRdeuXeOYY46JmTNnRp8+fXZ/1RLB7BL78hH78hL/8hH78hH7IolgjrjvW4dEsM2lkgi2OYlgdol9+Yh9eYl/+Yh9+Yh9kUQwR9z3rUMi2OaaSwR36/ERAAAANKUQzXXqbHtN7/lJBAEASlDSLl6JStldrCxxXNo7eBVVX0ptrqRwTWpzAenq+PuzAAAANGBHEAAAIBXtpzTUjiAAAEDOSAQBAAByRmkoAABAKurjjfLQ7LMjCAAAkDMSQQAAgJxRGgoAAJCK+shW19CKJs/YEQQAAMgZiSAAAEDOKA0FAABIRSF0DQUAACCTJIIAAAA5ozQUAAAgFVl7oLyuoQAAALxJIggAAJAzSkMBAABSkbWuoUpDAQAAeJMdQQCANlYZ9SWNKmXcltfS/Vw/KVyT2lwVVf+U2lxJ4ebU5gIkggAAAClRGgoAAEBGSQQBAAByRmkoAABAKpKIkr4D3FaSJs/YEQQAAMgZiSAAAEDOKA0FAABIRda6hja972dHEAAAIGckggAAADmjNBQAACAVSkMBAADIKIkgAABAzigNBQAASEV9ZKs0tKrJM3YEAQAAckYiCAAAkDNKQwEAAFKRta6hTa/FjiAAAEDOSAQBAAByRmkoAEA71r1rfarz1ae4T5AUbk5troqqL6U2V1K4JrW5oKH6N3+youm12BEEAADIGTuCAAAAqcjacwTtCAIAAPAmiSAAAEDOKA0FAABIhecIAgAAkFESQQAAgJxRGgoAAJAKXUMBAADIKIkgAABAzigNBQAASIWuoQAAAGSURBAAACBnlIYCAACkoj6a69TZ9nQNBQAA4E0SQQAAgJxRGgoAAJAKD5QHAAAgo+wIAgBQVJmpRhdvSQrXpDZXRdWM5q+V/EuLY4pjC3PTWBK0OYkgAABAKjxQHgAAgIySCAIAAOSM0lAAAIBUKA0FAAAgoySCAAAAOaM0FAAAIBVJNPcQ97aXNHnGjiAAAEDOSAQBAAByRmkoAABAKnQNBQAAIKMkggAAADmjNBQAACAVSkMBAADIKIkgAABAzigNBQAASEV9ZKs0tOmH29sRBAAAyBk7ggAAAB1YTU1NFAoNdyolggAAZF59ioVsSWFuKmMiIiqqLtrb5bztmjemNhflks2uodOmTYt169Y1OCMRBAAA6MCqq6vtCAIAAORJ//79Gx2TCAIAAKSiPprr1Nn2ml5LyYngI488EvPmzYsVK1ZEz549Y/z48XHxxRdHjx49imNOOeWU+POf/9zotePGjYt58+bt5qIBAABoDSUlgo8++mhMnz49hg0bFl/+8pejpqYmFixYEE888URUV1dHZWVlJEkSq1evjjFjxsTYsWMbvH7AgAGtsngAAAB2X0mJ4Jw5c6J///5xxx13RNeuXSPijTrTK664Ih588ME45phj4vnnn48tW7bE8ccfH5MnT27VRQMAAGRPB3qg/LZt2+K9731vTJ06tZgERkSMGjUqIiJWrlwZERFPP/10REQMHjx4r5YKAABA62pxR7BLly4xf/78RseXL18eERH77bdfRESsWrUqIt5KBLds2RLdu3dPbaEAAACkY7e7hq5bty4ee+yxuOqqq2LIkCFxwgknRMQbiWCPHj1i9uzZsXTp0tiyZUvsv//+MWPGjJg4cWLqCwcAAMiWbD5Qfld2KxF89dVXY/To0RER0a1bt5g1a1Z06dIlIt4oDa2rq4va2tqYM2dObNq0KRYsWBCXXHJJvP766zFlypTdW3Nli1WrpY2hdYh9+Yh9eYl/+Yh9+Yh9+Yh9UbqRSO/fmUly016uBcqjIkmSpNTBGzdujIceeii2b98eCxcujOXLl8e1114b48ePj7vuuivq6+tj2rRpxfGvvfZaTJo0KbZu3Rr/8R//EVVVVaWvrL6F529UVrY8htYh9uUj9uUl/uUj9uUj9uUj9g3Up5gKVrb0nLfdiH1F1UUprOgNSeHG1ObKpBx8sHHQQXfHmjWby72MogMP7BnPPXf6Ls/tViL4djuTvB07dsSyZcuaHHf99dfHDTfcED//+c9j6NChpV9AIphdYl8+Yl9e4l8+Yl8+Yl8+Yt+ARLADyEUiWJ3BRHDaLs/t8Z9G165d49hjj42amppYv359k+P69OkTEW80jwEAAKD8WkwEV69eHaNHj47q6upG5+rq6qKioiK2bt0aEydOjBtuuKHRmGeffTYiIgYOHJjCcgEAANhbLSaCBx54YNTW1sbdd98d27dvLx5ft25dPPDAAzFy5MgYMGBA1NbWxqJFi2Lz5re2QmtqamLx4sVx5JFHRt++fVvnNwAAAMiEnQ+Uz8pP0yXOLXYN7dSpU8yaNStmzpwZZ511VnziE5+IDRs2RHV1dVRUVMQ3vvGNiIi47LLL4qKLLorTTz89Tj311Kirq4vq6uro1KlTfPOb3ywtbgAAALS6kpvFLF26NG699db4y1/+Et27d48Pf/jDMWPGjBg0aFBxzK9//ev4wQ9+ECtWrIiuXbvGqFGj4pJLLik+ZH63aBaTXWJfPmJfXuJfPmJfPmJfPmLfgGYxHUAumsUsjDVrasu9jKIDD+wVzz131i7P7XHX0FYnEcwusS8fsS8v8S8fsS8fsS8fsW9AItgB5CIR/HEGE8Fzdnlutx4oDwAA5dBi8rYbWkoqK0sYs1OayVtF1czU5koKc1Kbi46p46flAAAANGBHEAAAIBU7u3VmRdNrsSMIAACQMxJBAACAnFEaCgAAkIqdD5TPiqabLNkRBAAAyBmJIAAAQM4oDQUAAEiFrqEAAABklEQQAAAgZ5SGAgAApELXUAAAADJKIggAAJAzSkMBAABSoWsoAAAAGSURBAAAyBmloQAAAKmoj+Y6dbY9XUMBAAB4kx1BAABypbLFHZvKEsa8oT7FfZWkMCe1uSqqvpDaXElhXmpzkR0SQQAAgFR4oDwAAAAZJREEAADIGaWhAAAAqfBAeQAAADJKIggAAJAzSkMBAABSoTQUAACAjJIIAgAA5IzSUAAAgFQk0dxD3Nte0uQZO4IAAAA5IxEEAADIGaWhAAAAqdA1FAAAgIySCAIAAOSM0lAAAIBUKA0FAAAgoySCAAAAOaM0FAAA9lBlig8Pr09xjyYpzEttroqqL+31HAce+N547rnLUlhN1tVHtkpDm74/7QgCAADkjEQQAAAgZ5SGAgAApELXUAAAADJKIggAAJAzSkMBAABSUR/Ndepse7qGAgAA8CaJIAAAQM4oDQUAAEiFB8oDAACQURJBAACAnFEaCgAAkAoPlAcAACCjJIIAAAA5ozQUAAAgFUpDAQAAyCg7ggAAAKnwHEEAAAAyKrM7gn99vjJ27Gj6/PvfH/HMcy3nse8/qOksGAAAsqKymd2b3VWf4n5PUrgmtbnIjswmggAAAO2LZjEAAABklEQQAAAgZ5SGAgAApKI+muvU2fZ0DQUAAOBNEkEAAICcURoKAACQCg+UBwAAIKMkggAAADmjNBQAACAVHigPAABARkkEAQAAckZpKAAAQCp0DQUAACCjJIIAAAA5ozQUAAAgFfXRXDlm21MaCgAAwJsyuyNYVdXymE6ZXT0AAJA3Awf+t3IvoYGd66mpqYlCoWETm4okSZJyLAoAAIDW9dprr8XRRx8dGzdubHBcIggAANBBbdq0KTZt2tTouEQQAAAgZzSLAQAAyBmJIAAAQM5IBAEAAHJGIggAAJAzEkEAAICckQgCAADkjEQQAAAgZySCAAAAOdOp3AvYXWvXro2rrroqHn/88YiIOPbYY+NrX/ta9OnTp8wr6/hOOeWU+POf/9zo+Lhx42LevHllWFHHN2vWrFizZk0sXLiwwXHvg7bRVPy9F9L34IMPxk033RRPPvlkVFZWxmGHHRYXX3xxHH744cUx7vvWUUrs3fOt55FHHol58+bFihUromfPnjF+/Pi4+OKLo0ePHsUx7v3WUUrs3ft0ZO0qEdywYUOcc845sX379rjggguiUCjE/PnzY+XKlbFo0aLo3LlzuZfYYSVJEqtXr44xY8bE2LFjG5wbMGBAmVbVsS1atCgWLVoUo0aNanDc+6BtNBV/74X0Pf744/HpT386PvCBD8SMGTNix44dceedd8aZZ54Zd955Zxx66KHu+1ZSSuzd863n0UcfjenTp8ewYcPiy1/+ctTU1MSCBQviiSeeiOrq6qisrHTvt5JSYu/ep8NL2pFrr702+eAHP5g8/fTTxWMPPfRQMmTIkOSee+4p48o6vr/+9a/JkCFDknvvvbfcS+nwduzYkVx//fXJ0KFDkyFDhiRnnnlmg/PeB62rpfh7L6Rv8uTJybHHHpts2bKleOyll15KRo4cmZx77rlJkrjvW0spsXfPt56TTjopOe6445KtW7cWj91xxx3JkCFDkmXLliVJ4t5vLaXE3r1PR9euviO4ZMmSGDVqVAwePLh47KijjopBgwbFkiVLyriyju/pp5+OiGgQe9K3bdu2OOmkk+L666+PyZMnR79+/RqN8T5oPaXE33shXRs3bowVK1bE+PHjo1u3bsXj++yzT4wcOTL++Mc/RoT7vjWUGnv3fOvYtm1bvPe9742pU6dG165di8d3ViGsXLkyItz7raHU2Lv36ejaTWnoxo0bY+3atTFu3LhG54YNGxbLli1r+0XlyKpVqyLirb8Mt2zZEt27dy/nkjqkbdu2xebNm2Pu3LkxYcKEGD16dIPz3getq6X4R3gvpK1nz57xy1/+skEistOGDRuiqqrKfd9KSol9hHu+tXTp0iXmz5/f6Pjy5csjImK//fZz77eSUmIf4d6n42s3O4IvvPBCRMQuP6Hv27dvbN68OWpra9t6WbmxatWq6NGjR8yePTtGjBgRI0aMiDFjxvg0MmU9e/aMBx54ICZMmLDL894Hraul+Ed4L6StqqoqDjrooEb39IoVK+IPf/hDjBgxwn3fSkqJfYR7vq2sW7cuFi9eHN/5zndiyJAhccIJJ7j328iuYh/h3qfjazc7gnV1dRERu/zkskuXLhHxxic1vXr1atN15cXTTz8ddXV1UVtbG3PmzIlNmzbFggUL4pJLLonXX389pkyZUu4ldgiVlZVRWdn05zPeB62rpfhHeC+0hbq6uvjqV78aERGf+cxn3Pdt6J2xj3DPt4VXX321WIHQrVu3mDVrVnTp0sW93waain2Ee5+Or90kgvX19S2OaekfcOy5qVOnRn19fUybNq14bOLEiTFp0qT43ve+FyeeeGKxjIjW431Qft4LrWvr1q3x2c9+NlasWBEXXnhhjBo1Kn7/+9+3+Dr3/d7bVewj3PNtoaKiIubOnRvbt2+PhQsXxnnnnRfXXntt9O3bt8XXuvf3TlOxHz9+vHufDq/d/O2x85ku27Zta3Ru57G3P/eFdJ1xxhkN/iKMiOjatWtMnjw5Xn755eIXqmld3gfl573QejZt2hTTp0+Pxx57LE4++eSYMWNGRLjv20JTsY9wz7eFd7/73TFhwoSYMmVKVFdXx3777Rff/e533fttoKnYR7j36fjaTSK484u7L730UqNzL774YvTu3dsXeMtg58Nst2zZUuaV5IP3QXZ5L+ydV155Jc4+++z4wx/+EKeddlp85zvfiYqKiohw37e25mLfHPd86+jatWsce+yxUVNTE+973/siwr3fVt4e+/Xr1zc5zr1PR9FuEsHevXvHwIED48knn2x07qmnnorhw4eXYVX58MILL8TEiRPjhhtuaHTu2WefjYiIgQMHtvWycsn7oLy8F1rH5s2b4/zzz4/ly5fHueeeG1dccUWDRMR933pair17vvWsXr06Ro8eHdXV1Y3O1dXVRUVFRXTu3Nm93wpKif3WrVvd+3R47SYRjIgYO3ZsPPLII7F69erisYcffjieffbZZrv8sXf69esXtbW1sWjRoti8eXPxeE1NTSxevDiOPPLIkr7HQDq8D8rHe6F1XHHFFbF8+fI4++yz49JLL93lGPd962gp9u751nPggQdGbW1t3H333bF9+/bi8XXr1sUDDzwQI0eOjJ49e7r3W0EpsR8wYIB7nw6vIkmSpNyLKNX69etj0qRJUVVVFdOnT49t27bFrbfeGgcccEDcfffd0blz53IvscP69a9/HRdddFF84AMfiFNPPTXq6uqiuro6Xn/99bjrrrs8bLWVjB49OgYMGBALFy4sHvM+aDu7ir/3QrpWr14dEyZMiF69esXXv/71XTZemDx5svu+FZQae/d867nvvvti5syZcfjhh8cnPvGJ2LBhQzG2d955ZwwZMsS930pKib17n46uXSWCERHPPPNMzJ49O373u99F165d45hjjomZM2cW67VpPb/+9a/jBz/4QaxYsSK6du0ao0aNiksuucRfhK1oV4lIhPdBW2kq/t4L6bnrrrviW9/6VrNjVq5cGRHu+7TtTuzd861n6dKlceutt8Zf/vKX6N69e3z4wx+OGTNmxKBBg4pj3Puto5TYu/fpyNpdIggAAMDeaVffEQQAAGDvSQQBAAByRiIIAACQMxJBAACAnJEIAgAA5IxEEAAAIGckggAAADkjEQQAAMgZiSAAAEDO/H9hfQV66bOjCgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_173_1.png" } }, "output_type": "display_data" } ], "source": [ "clf_lasso = skl.Lasso(alpha=_lambda).fit(X_train, y_train)\n", "J_lasso_sk = clf_lasso.coef_.reshape(L, L)\n", "fig = plt.figure(figsize=(20, 14))\n", "im = plt.imshow(J_lasso_sk, **cmap_args)\n", "plt.title(\"Lasso from Scikit-learn\", fontsize=18)\n", "plt.xticks(fontsize=18)\n", "plt.yticks(fontsize=18)\n", "cb = fig.colorbar(im)\n", "cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is quite striking how LASSO breaks the symmetry of the coupling\n", "constant as opposed to ridge and OLS. We get a sparse solution with\n", "$J_{j, j + 1} = -1$.\n", "\n", "\n", "\n", "\n", "We see how the different models perform for a different set of values for $\\lambda$." ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\r", " 0%| | 0/10 [00:00" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_175_13.png" } }, "output_type": "display_data" } ], "source": [ "lambdas = np.logspace(-4, 5, 10)\n", "\n", "train_errors = {\n", " \"ols_sk\": np.zeros(lambdas.size),\n", " \"ridge_sk\": np.zeros(lambdas.size),\n", " \"lasso_sk\": np.zeros(lambdas.size)\n", "}\n", "\n", "test_errors = {\n", " \"ols_sk\": np.zeros(lambdas.size),\n", " \"ridge_sk\": np.zeros(lambdas.size),\n", " \"lasso_sk\": np.zeros(lambdas.size)\n", "}\n", "\n", "plot_counter = 1\n", "\n", "fig = plt.figure(figsize=(32, 54))\n", "\n", "for i, _lambda in enumerate(tqdm.tqdm(lambdas)):\n", " for key, method in zip(\n", " [\"ols_sk\", \"ridge_sk\", \"lasso_sk\"],\n", " [skl.LinearRegression(), skl.Ridge(alpha=_lambda), skl.Lasso(alpha=_lambda)]\n", " ):\n", " method = method.fit(X_train, y_train)\n", "\n", " train_errors[key][i] = method.score(X_train, y_train)\n", " test_errors[key][i] = method.score(X_test, y_test)\n", "\n", " omega = method.coef_.reshape(L, L)\n", "\n", " plt.subplot(10, 5, plot_counter)\n", " plt.imshow(omega, **cmap_args)\n", " plt.title(r\"%s, $\\lambda = %.4f$\" % (key, _lambda))\n", " plot_counter += 1\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see that LASSO reaches a good solution for low\n", "values of $\\lambda$, but will \"wither\" when we increase $\\lambda$ too\n", "much. Ridge is more stable over a larger range of values for\n", "$\\lambda$, but eventually also fades away.\n", "\n", "\n", "To determine which value of $\\lambda$ is best we plot the accuracy of\n", "the models when predicting the training and the testing set. We expect\n", "the accuracy of the training set to be quite good, but if the accuracy\n", "of the testing set is much lower this tells us that we might be\n", "subject to an overfit model. The ideal scenario is an accuracy on the\n", "testing set that is close to the accuracy of the training set." ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAABKUAAANBCAYAAAAxzw1RAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAEAAElEQVR4nOz9e5ycd33f/b+v65rzYWeP2tVqDzpYkkEmEAMCO8cSDglJcJMa2oSCbWLcNuWmvZNHnOTRJL82d9s0vdM7jZOW1NjBJCQlAQqEGEoOlBxqQk0IENtYkiXt7qykXa32POfDdf3+WGl2r2tmJa20O9ccXs9HiHR9r5nxx56dmb3e8/1+vobjOI4AAAAAAACAJjL9LgAAAAAAAADdh1AKAAAAAAAATUcoBQAAAAAAgKYjlAIAAAAAAEDTEUoBAAAAAACg6QilAAAAAAAA0HSEUgAAAAAAAGi6gN8FtJLl5axs2/G7DHS5gYGEFhczfpcBdDVeh4C/eA0C/uN1CPirU16Dpmmory++7XlCqS1s2yGUQkvg5xDwH69DwF+8BgH/8ToE/NUNr0GW7wEAAAAAAKDpCKUAAAAAAADQdIRSAAAAAAAAaDpCKQAAAAAAADQdoRQAAAAAAACajlAKAAAAAAAATUcoBQAAAAAAgKYjlAIAAAAAAEDTEUoBAAAAAACg6QilAAAAAAAA0HSEUgAAAAAAAGg6QikAAAAAAAA0HaEUAAAAAAAAmo5QCgAAAAAAAE1HKAUAAAAAAICmI5QCAAAAAABA0xFKAQAAAAAAoOkIpQAAAAAAANB0hFIAAAAAAABoOkIpAAAAAAAANB2hFAAAAAAAAJqOUAoAAAAAAABNRygFAAAAAACApiOUAgAAAAAAQNMRSgEAAAAAAKDpWjKU+rmf+zm9613vuqnbptNpve9979PJkyd18uRJPfroo1paWtrjCgEAAAAAAHA7An4X4PWxj31MH/vYx3Ty5Mkb3nZ5eVkPPPCASqWSHn74YVWrVT355JM6deqUPvaxjykUCjWhYgAAAAAAAOxUy4RS1WpVH/jAB/Qbv/EbN32fp556SnNzc/rMZz6jI0eOSJJe+cpX6qGHHtKnPvUpveMd79ircgEAAAAAAHAbWmL5XrFY1A/90A/p13/913XfffdpeHj4pu739NNP6+TJk7VASpLuvfdeHTp0SE8//fRelQsAAAAAAIDb1BIzpYrFojKZjH71V39Vb33rW/WGN7zhhvdZXV1VOp3WW97ylrpzJ06c0Be/+MU9qLS1XVq9rOcuX1TZrvpdCm5D8FxA5XJFkmQ70mxpZ8tQjat/hg1Hw6Fyw9tcKgVVdoy68fqR+vPj4VLDc0vlgLL2reXc+0NlhUynbny9amq5YjWs40b6AxUlLLtuvGgbulwO3vD+jf4ZCauqvkD968t2pAtbnqeA4Shi2oqajqJmVXHLVty0lbCqipm2wqYj42b+JeCb8lpYuWzR7zKw13b1hdiaj2W05L/jjR/HzkWVzVZkmJYMIyDDDMgwNv4u11hg8zZbz/EmCwBAW2iJUCqRSOiP//iPFQjcfDnz8/OS1HBW1dDQkDKZjNbX15VMJnetzlY2t7agD5y+oorifpeCFnK6EN31xzxfjLTFY57d9UfcbY4MOQqoopQyChllBVVWUBWFrv5ZUEimbEVUVEQlxVRQ1MgrrJLCxsZtgirLkk3ItctW/S4A6HIrt3l/p+pI1/5XceTYm39X1XGfr3rGtrlN3X0qN3Gb+u9GgLZx2u8CgC6XHjuggX/0jxW782V+l7KnWiKUMk1TprmzGRbZbFaSFI3WX3SHw2FJUi6X65pQ6tzSvCoK+10GgJtmyJGhskK6on6pfqJYYw1v58iULVO2LNkKqKJerbmCq6AqChllmY6tkEqKGKWNc0ZZoS23Caoi07jZYgCgNRmWIVmbab1fuX0tDLtOcOVsCcKuG4bV3UZS1ZaqknP1T1WuHtvu26vK+zoAtJv87AXN/85TOvTvftnvUvZUS4RSt8K2b/zV006DroGBxK2W47tv1aT+58KsKu37lAK4ZcbVOMpSRVJRYWUVrw+wnG3+3vDRqrJUrQVVIZUVV05ho7wxk8uoeGZ2lRWqG6vIUpVZXAC6lmEakmlIW1aN+xaQNZo9dp0w7GZmmHkfc+uYs1LeGAMA3DLLMjU01NkTbdo2wYjHN5apFYv1PT+ujV27zc1aXMzIttvzwzOqpB6KZvT16bMqV5kr3m7KZkC2YSpULbl+WXVk6FJidHNgB7/JBqslDeavuB7rmoXYPlXMnbz8r93X0f7MpbqzjiGthPuUD263XHD7wh1Jg7nLCtqVunPZYFxr4dQNH6vRq7a3uKxoJV83XjJDWowNbVvP9cTKWaWKq3X/PFuG5rc8T1XTUsUMqGIEZJuWqoYpxzA3noM2SGgcmarIVEVB1b3DOp4/b/hgtiy7ooBdVdAuKWiXFaqUFLKLCleKCtklBatlBe2ygtWSQlf/DLr+LG/ct1qWedP/YGCXtGJLppt9oOa1gNr9xzKuznSyjM1ZT4Grx+bm313nLUOyJCPQEvv4tJxmzx5zSrYqf7Os6ldZEA0AtyI6NqaBf/ROLSys+13KbTFN47oTgNo2lBod3bgAXFhYqDt3+fJl9fT0KBaLNbssXx2667U6dNdr/S4Dt+Av55b1ufQVJQKW7hhI6BU9Mb2st31n7mF7FdtRvlLVWrmi9XJVcqR40FLRtlWq2ipWbRXtjT+/emVNpaqjsmOrYjuqOk77rcAwTFWtkKqWVNTt9zizDClkmgpbpiKW+8+wZSpsmgptGb92241zhsLW1fOmqaBpbNsMeWgo2fa/AADt7HZeg47jSE5VjlORY1/7syLn6pjsiufc1T+v/l212187t/UxKpLn9lv/1JZ/jmNXdPMJfucxQqaC9wwo+ZbXaWDy78sKdNfv5Z2Az0LAX93yGmzbUKqnp0djY2N6/vnn68698MILuuuuu3yoCrg16UxBkpSpVPW1+VWNhXe24x7aR8A0lAwFlAzd+O33u/b3Nxwv27YKVVuFysaf2cpGwJUrV5WtbOwOeKQn1iDocvT1xTWtldt3h86qI+WrtvK7MCPUkBS6GlRFLOtqeGUobFm6O1/U8UhIZhvMbAPgZhiGZARkKCDVb+DaVI5jbwZcTrVBILb1nGesQfhVdxun2iAsq9Y9hhz/3vcLay9p7sXHNXjwHyicGPetDgBAa2rbUEqS3vzmN+u3f/u3dfbsWR05ckSS9Mwzz+j8+fP6sR/7MZ+rA27etVDqmon47u9Gh84RNE0FTVPJ4I1v6/V944NyHEdl29kItqq2CtVq7e/5ykawlS1XlatsHBeqtkq2re8Y6dsSdDm18bNrOa2U6pdftjpHUtF2VLSrdUHd88sZff/4oL5tpM+f4gB0BMMwZVghSf5+2bT97LHNEEvbzh7bCLXqw7JGgdjG7cvFRTnVzQXg1fKa5s98WL2jb1By3z3bzlIFAHSftgml0um0vvrVr+ruu+/W+PjGtyzvfe979elPf1oPPvig3vOe96hYLOqJJ57QiRMndN999/lcMXBzVktlrZY3L+iDpqGRGDspYu8YhqGQZShkmerZhcdbLpa1VqpcJ+Ta+DN/bdbW1TCrbDv67pE+BSxzc/zqrK75fFFLRX+Drr+cWyaUAtARmj17rFJa1ZWpT6iUnd0yamvl4p+qmJ3RwMR9MgO3v6QbAND+2iaUevbZZ/WzP/uz+qVf+qVaKNXf36+PfOQj+qVf+iU99thjikQieuMb36hHH31UoRDLn9AeZjyzpCZTMQVMvkFE++gLB9UX3vm0LdvZaL/f6BvzdKags2s5d9BVsa8u3dv4e/FqsOV1ojehVDjgCsCKVVtrpYqWdzCja61cVbFqK2zRNBkAdiIQSmn46ANaufhnWr/8165z+dXTunTqcQ0evF/h+AGfKgQAtArDcZzu7cDo0c6776F9fXZmQX81v1I7fvOhffruQe+OcwAasR1HxVpwtfG/oUhQiWD9dy7pTEF/emGxbkZXo2DrmnfdMaKX9XX2NrxAK+qW5q7dILd6SovTn5ZTdX8JJ8NU3+iblBg6yXK+FsXrEPBXp7wGO3b3PaBTpLPuX9IO98Z9qgRoP6ZhKBqwFA3ceD3KeCKih47XfytftR0V7Y3G8b/+/LSKW0Kq55ezhFIAcBtiqeMKHX9EV6Y+rlLu4uYJx9byhc+rkJ3RwMQPyrTopwkA3Yg1CYCPKrajC9mia+wQoRTQVJZpKBaw1B8Jajjq7uc2lcn7VBUAdI5AuFfDRx9SYuhk3bn8yjc19+IHVcpd8qEyAIDfCKUAH83li6psWUHbEwyoP0o/NMAvR3rcjXdXShVVWdYNALfNMC31j32vBg+9XYbp/gKgUlrW3Onf0vrCs6KzCAB0F0IpwEfeJufjCaauA356zVCPolsam9uOdClXvM49AAA7Eet9mUbufK+C0RH3Caeq5dnPaXHqf8iu8r4LAN2CUArwUdoTSk0QSgG+6guHdKQn5hpjCR8A7K5guF8jx96jxOBr6s7lVp7X3KkPqpSb86EyAECzEUoBPvI2OZ+IE0oBfpv0hMPThFIAsOsMM6D+8bdq4OAPyzDdrQsqxSXNn/4tZa58leV8ANDhCKUAn2TKFS0Vy7Vj05BG4+Hr3ANAMxxMuvtKTa0XuCgCgD0S77tLI8ffq2Bk2DXuOBUtpf9Ii9Ofkl0t+VQdAGCvEUoBPvHOktofDSto8pIE/DYSCyu8pa9UtlLV4pYAGQCwu4KRAQ0ff4/iA99ady63/HeaO/WESvnLPlQGANhrXAEDPhmLR3T/oWGdHEppfyysSc/sDAD+sAxDR/rirrGptZxP1QBAdzDNoAYmflADk39fhhl0nasUr2j+1BPKLH7dp+oAAHsl4HcBQLdKBgO6e7BHdw/2SBLLg4AWEglYruNvrub0mn29/hQDAF0k3v8tCkX368rUx1UuLNTGHaeipZlPq5iZVt/498n0BFcAgPbETCmgRRiG4XcJAK5aKbiX6814dsoEAOydYHRIw8d+TPH+V9adyy59TfOnnlS5cMWHygAAu41QCgAAj5cPJl3H2UpV6+WKT9UAQPcxrZAGJu9T/8TbZBjuxR3lwmXNnfqgskt/51N1AIDdQigFAIDH0f5E3dj0OrOlAKDZEgOv0vDxH1MgPOAad+yyFqc/qaWZP5Jj86UBALQrQikAADwOpuJ1Y2dpdg4AvghFhzVy/GHF+u6qO5dZ/KrmTv+WysUlHyoDANwuQinAB381t6xvrmSUYTkQ0JJiQUs9QfdykbPrhFIA4BfTCmtg8ofUN/79kuHejKKcn9Pci48rt/yCT9UBAG4Vu+8BTZavVPXZ9GZzzoFwUP/yrklZJo3OgVZyMBnRN5YytePFQlmlqq2Qxfc5AOAHwzCUHHy1wrFRXZn6hCpbZkc5dklXpj6uROa16jvwJhkmlzkA0A74zRposnTW3ZcmaBoEUkALOpSMuY4d1b9+AQDNF4rt18jx9yrW+/K6c5krz2r+9IdUKS77UBkAYKcIpYAmS3u2lh9PRHyqBMD1NHptnqevFAC0BNMKa+DgP1Df2PfVLecr5S/p0qnHlVt50afqAAA3i1AKaDLvTIuJOKEU0IqGoyFZnkmMZ9fz/hQDAKhjGIaSQ6/V8LGHZIV6XeecalFXzv+Blmc/L8eu+lMgAOCGCKWAJrIdp8FMqahP1QC4HsswNBILu8ZmswVVHcenigAAjYRjo9p//BFFU8frzq0vfFnzZ55SpbTS/MIAADdEKAU00WKhrHzVrh1HLFODkaCPFQG4nsOe0LjqSHO5ok/VAAC2YwYiGjz0DvUeeLO8lzil3AXNvfi48qun/SkOALAtQimgibxL98bjEZkGTc6BVtWor9R0hmbnANCKDMNQz77Xa/jYg7KCKdc5u1rQwrmPavnCn8hxWM4HAK2CUApoohmanANtpdHy2nM0OweAlhaOj2nkzkcU6Tlad2798pd0+cxvq1Ja86EyAIAXoRTQRI1mSgFoXalQQPGAe1enmUxeDn2lAKClWYGohg7/I/WOfo8k96z0YjatuVOPK7/2kj/FAQBqCKWAJilV7bpeNMyUAlrfvzgxoaC5eUGTqdhaLlZ8rAgAcDMMw1DP8Ldp39EHZAWTrnN2JaeFs7+nlYtfkOPY2zwCAGCvEUoBTTKbLWjr3IrBSFAxzwwMAK0nEQrUzWqcyuR9qgYAsFORxIRGjj+iSPJI3bm1+b/S5Zd+R9Xyug+VAQAIpYAmYeke0L4mk+7eUtOEUgDQVqxgXENHflSp/X9Pdcv5MtO69OLjKqyf86c4AOhihFJAk3ibnE+wdA9oGwc9r9epdXbgA4B2YxiGUiPfoX13vEtmIOE6Z1eyuvzSR7Ry6Yss5wOAJiKUAppkLu/tJ1W/qxeA1jSRiLq+V18olJQts6U4ALSjSPKg9t/5iMKJQ3Xn1ub+Qpdf+l1VyxkfKgOA7kMoBTTJT9x1UP/85eN62+SQ7h5Majga8rskADcpbJnaHwu7xljCBwDtywomtO+Odyo18l1154qZ81eX8001vzAA6DKEUkCTWKahA/GIXr+vV/cfGpFlGDe+E4CWkM4UtFZy77hHKAUA7c0wTKX2f5f2HfnHMgNx1zm7ktHll35Hq3N/KcdxtnkEAMDtIpQCAOAGQpahTMW9XO/8GqEUAHSCSM9hjdz5iMKJSc8ZR6uX/pcWzv6eqpWcL7UBQKcjlAIA4AaGIiGFTffsxou5okpVmuECQCcIBJPad8e71DP87XXnCutnNffif1MxM+NDZQDQ2QilAAC4AdMwNO7Zgc+WNJtlFz4A6BSGYap39A0aOvKjMi33hjTV8rrmz3xYa/PPsJwPAHYRoRQAADdhLB6pG5vOEEoBQKeJ9tyhkTsfUSg+5jnjaOXin2rh3EdVrbCEGwB2A6EUsMe+srCq3zlzUV+8uKSzazkVWe4DtKWJRKNQiosSAOhEgVBKw0cfUHLfPXXnCmtnNPfi4ypmZ32oDAA6C6EUsMfOrOb0zZWs/vjCop48dUFfWVj1uyQAt6DRTKmp9bxslnEAQEcyDEt9B96kwcP/UKbl/gyollc1f+YprV3+a5bzAcBtIJQC9lja03NmIhHd5pYAWlkiGFBfKOAaK9mO5vMlnyoCADRDLHV8YzlfbNR9wrG1cuGPdeX8x2RXWM4NALeCUArYQ2ulilZKldqxZRjaHwv5WBGA2+Ftdi5tzJYCAHS2QKhXw0cfUnLodXXn8qsv6tKpx1XMXfShMgBob4RSwB6a8TRBHo2FFTB52QHtarxhs3NCKQDoBoZpqW/sLRo89A4ZVth1rlpa0fzpD2l94VmW8wHADnB1DOyh+qV79Re0ANpH45lSBS5AAKCLxHrv1P7jjygU3e8+4VS1PPs5LU59Qna16E9xANBmCKWAPZT2zKBoNMsCQPvYHwvLMtxja2X3Ml0AQOcLhPs0fOwhJQZfW3cut/KC5k59UKXcnA+VAUB7IZQC9kjVdnQh5/6WjJlSQHsLmqb2x8J14yzhA4DuY5gB9Y9/nwYO/gMZprtnaKW4pLnTTypz5W+YTQsA10EoBeyRuXxRZXvzl5Bk0FLKs3MXgPbTaMbj1Dq7LgFAt4r3ndDI8fcqGB12n3CqWko/rcXpT8quslMrADRCKAXsEW8/qfF4RIZhbHNrAO3icE9MBzyzpZgpBQDdLRgZ0PCx9ygxcHfdudzyc5o79YRK+cs+VAYArY1QCtgj3p33WLoHdIYTfQm95/gBbY2Y5/Ml5StV32oCAPjPNIPqn/gBDUz+kAwz6DpXKV7R/KknlFn8mj/FAUCLIpQC9kjaE0qNJ6I+VQJgt0UDlkai7v4h0xmW8AEApHj/KzaW80WGXOOOU9HSzB9qcfrTsu2yT9UBQGshlAL2QLZc1WJx85cNU6pb7gOgvU0m3UHz9DpL+AAAG4KRQQ0ff1jx/lfVncsufV3zp55QubDQ/MIAoMUQSgF7wNtPaiQWVsji5QZ0kknP7Mcp+koBALYwzaAGJt+m/om3yTDcm92UCwuaO/WEskvf8Kk6AGgNXCUDe8CRo9FYuPYCG6efFNBxDibdr+vZbFFl2/apGgBAq0oMvErDxx9WIDzoGnfsshanP6XFmT9iOR+ArsX+9MAeeFlvQi/rTahUtXUhV1SUWVJAx0kGA0oGLK1fbXBedRxdyBZ1MEn/OACAWyi6TyPHH9ZS+mnllv/OdS67+FWVshc0eOh+BSMDPlUIAP7gShnYQyHL1KFkVCP0kwI6zn99IV0LpK6ZZgkfAGAbphXSwOTfV//4D0iG5TpXLsxr7tQHlV1+3qfqAMAfhFIAANwC7+57kjRFs3MAwHUYhqHE4N0aOfZjCoT7Xeccu6TFqU9oKf1ZOXbFpwoBoLkIpQAAuAVjDXrFTWcKsh3Hh2oAAO0kFBvRyPH3KtZ7ou5c5spXNHf6QyoXl3yoDACai1AKAIBbMB6vD6UKVVuX8yUfqgEAtBvTCmvg4A+rb+z76pfz5S9p7sUPKrfyTZ+qA4DmIJQCdlnO02MGQGcaiYYVNI26cfpKAQBulmEYSg69ViPHHlIg1Oc659hFXTn/MS3Pfl6Oze+XADoToRSwiwqVqv7d357Tr3xjSr9/dk5fml+Rw1IeoCNZpqHRBpsYTK8XfKgGANDOQrFRjRx/r6KpO+vOrS98WfNnPqRKcaX5hQHAHiOUAnbRbLYoR9JSsayvL63rywurMoz6mRQAOkOjJXxTzJQCANwCMxDR4KG3q/fAWyTDfZlWyl3UpVOPK7d6yqfqAGBvEEoBu2gm654hMdHgghVA5xhv0Ox8pVTRSrHsQzUAgHZnGIZ69r1Ow0cfkhVKuc451YKunPt9LV/4EzkOy/kAdAZCKWAXpT0zJBpdsALoHI1mSkkbu/ABAHCrwvEDGjn+iKI9x+rOrV/+kubPfFiV0qoPlQHA7iKUAnaJ4zhKe2ZKbXfBCqAzpEIBJYNW3TjNzgEAt8sKRDV4+B+qd/RNktztIErZWc29+Ljyq2f8KQ4AdgmhFLBLFotl5Sp27ThsmtoXDflYEYC9ZhhGw/B5ep1QCgBw+wzDUM/wPRo++qCsYI/rnF3Na+Hcf9fKxT+T49jbPAIAtDZCKWCXpD3LdcYTYZk0OQc6XqNlunP5kgoV+n0AAHZHODGukTsfUaTnjrpza/P/W5df+m1Vyus+VAYAt4dQCtgl3ibn4/GoT5UAaKZGM6Uc1b8nAABwO6xATEOHf0Sp/W+QdzlfMTOjuRf/m/JrZ/0pDgBuEaEUsEvqZ0rRTwroBgfiETWaEznFEj4AwC4zDEOpkW/XvqPvlhVIuM7ZlZwWzv6uVi59keV8ANoGoRSwC0pVW3O5omuMJudAdwhbpr53fFD3Dru37mYHPgDAXokkJjVy5z9RJHmo7tza3F/o8ksfUbWc8aEyANgZQilgF1zIFbX1+6iBcFDxBjtyAehM3zHSp28b7nONpTMFVWzHp4oAAJ3OCsY1dOSdSo18V925YmZKl158XIX18z5UBgA3j1AK2AUs3QPQGwooFQzUjiuOo4s5ZksBAPaOYZhK7f8u7bvjH8sMxF3n7EpGl1/6iFbn/kKOw5ckAFoToRSwC9JZd+8Ylu4B3ccwDE0m3a/9qXVCKQDA3oskD2v/nY8onJj0nHG0eumLWjj7u6qWs77UBgDXQygF3CbHcTTjmSk1wUwpoCtNJty7bk5naHYOAGgOK5jUvjvepZ7h76g7V1g/p7lTj6uQmfGhMgDYHqEUcJscSX9vtF/fOpDUYCSooGloJBr2uywAPjiYrA+lbJZMAACaxDBM9Y7+PQ0d+VGZgZjrXLW8rstnPqy1+f/Ncj4ALSNw45sAuB7TMPT6fb16/b6N42LVlmU22iAeQCfLlCuayxVlGVL16u/6uYqtK4Wy9kVD/hYHAOgq0Z47NHL8ES1OfULFbHrLGUcrF/9MhcyMBibvk+UJrgCg2ZgpBeyysMXLCuhG6UxBHzs/XwukrmEJHwDAD4FQj/YdfUA9++6tO1dYO6O5Fx9XMTvrQ2UAsImrZwAAdsHYNr3kptYJpQAA/jAMU70H3qihw/9IpuVeYl4tr2n+9FNau/wllvMB8A2hFAAAuyAZDKg3VL8qfjrDDnwAAH9FU8c0cucjCsUOeM7YWrnwJ7py/g9kV/gSBUDzEUoBALBLxuP1s6WWimWtlSo+VAMAwKZAKKXhow8qOfT6unP51VO6dOqDKmYv+FAZgG5GKAXchtlMQWfXcipWbb9LAdACxrdZwkdfKQBAKzBMS31jb9bgoXfIsNy7RVdLK5o/8yGtL/wflvMBaBp23wNuw1/NL+sbSxkZkoajIX3f+KCOpuJ+lwXAJ41mSknS1HpBr+hPNrkaAAAai/XeqVB0WFemPqFS7uLmCcfW8uz/VCEzrYG+H/GvQABdg5lSwG1IX+0V40iay5fYeQ/ocqPxsEyjfpyZUgCAVhMI92n46INKDJ2sO5df+aa++eVfl12hLyKAvcUVNHCL1ssVLW/pE2MZ0v5Y+Dr3ANDpgqap/dH694FLuSLLfAEALccwA+of+14NHrxfhhlynSvmFrQ69+c+VQagWxBKAbco7dlRa38srKDJSwrodo36SjmSZpgtBQBoUbG+l2vkzkcUjI64xnMrL9JfCsCe4goauEXeUGo8HvWpEgCtZLu+UtMZlkAAAFpXMNyv4aMPyjA22w5Xy6sq5+d9rApApyOUAm7RTNZ9gTmxza5bALrLdjvwTa0zUwoA0NpMK6RI8rBrLL922qdqAHQDQingFlQdR7OeUGq7C1EA3WUgHFS0waYH6WxBVZslEACA1hZNHXMd51cJpQDsHUIp4BbM50sqb7m4TAQs9YUC17kHgG5hGEbDkLpsO7qUK/pQEQAAN88bSpVyF1Upr/tUDYBORygF3IK0p2HxeCIiw2iwDzyArjR2ta+U5XlbmKLZOQCgxVnBhEKxUddYYfWMT9UA6HSEUsAt8DY5p58UgK1eO5TST7xiUt87Nugap68UAKAdRFPHXce51VM+VQKg0xFKAbfA2+R8u922AHSnVCigwUhIB5PuXTmnMwW21gYAtDzvEr7i+nnZdtmnagB0MkIpYIdylaquFDY/lA1JBwilADQwEgsrZG6u4ctWqlos8ks9AKC1BSP7FIr01Y4dp6LC+jkfKwLQqQilgB3yLt0biYYUbrDTFgBYhqGJhHu2FEv4AACtzjAMpfa93DXGLnwA9gJX0sAOpb1L9zwXnACw1aSn59y0J9gGAKAV9Q7Vh1IsQQew29jDHtih7xzp05GemNKZgtLZvO7oIZQCsD1vXylmSgEA2kGi77AMMyzHLkqS7EpWpdwFheNjPlcGoJMQSgE7FLJMHUpGdSgZldR3w9sD6F5/dmFRZ9dyrrHFYlnr5YqSQT6CAQCtyzQDivYcUW7lhdpYfvU0oRSAXcXyPQAA9siZ1ZymGizXm15nCR8AoPVFU8ddx/SVArDbCKUAANgj44nGO3NOZ1jCBwBofZGeO7Sx1/SGcuGyKsVl/woC0HEIpQAA2CPjcUIpAED7sgJRhRMTrjFmSwHYTYRSAADske1mSl3MFlWq2k2uBgCAnYumjrmOc4RSAHYRoRRwk4pVWx89e0n/e25Z6UxBFZsLSgDX1xsKKB6w6sZtSeksfaUAAK0v2uMOpYqZadlVPsMA7A5CKeAmzWYL+sZSRk+nr+gD30zrv76Q9rskAC3OMAxNbDNbamqdJXwAgNYXjAwoEB7cMmIrv3bWt3oAdBZCKeAmpT07aI3Gwz5VAqCdbN9Xim+ZAQDtwbuEL796yqdKAHQaQingJs14ltqMx6M+VQKgnYxtM1NqJpNX1XGaXA0AADtXF0qtvSTHqfpUDYBOQigF3ATHcepmSm23JAcAthqLh7dspr2pZDuayxWbXg8AADsVjo/JtDa/kHWqBRUztLIAcPsIpYCbsFysKFvZ/DYoZBraFw35WBGAdhGxLA1t837BEj4AQDswDJMlfAD2BKEUcBNmsu6GxGPxiCyj0dwHAKi3XV8pmp0DANpFfSh1Wg7L0AHcJkIp4CZ4l+6Ns3QPwA5st9x3OpPnF3oAQFuIJA9LhlU7rpSWVSle8bEiAJ2AUAq4CTPeflLbzHoAgEbGtnnPWC9XtVQsN7kaAAB2zrTCiiQOusbyq6f9KQZAxyCUAm6gbNu6lHc3I95uNy0AaGQ4GlIiYOlgMqpUKOA6R18pAEC7aLSEDwBuB6EUcAMXs0XZW1bX9IUDSgYD298BADxMw9DPvOqQHrlzTK8e7HGdo68UAKBdeEOpYjatajnrUzUAOgGhFHADM1lPPymW7gG4BebVzREmPTMtpzOEUgCA9hAIpRSMjrjG8msv+VQNgE5AKAXcgLfJ+UQi6lMlADrBRCKqrXt3LhTKypQrvtUDAMBO1C/hO+VTJQA6AaEUcAN1oRQzpQDchrBlan8s7BrzbqYAAECrinlCqcL6WTk2X64AuDWEUsB1FKpVBa3NOQ0Bw9CI52ISAHZq0jPjkiV8AIB2EYzulxVM1o4du6xCZsq/ggC0Nbo1A9cRsSz9xCsOKlepajZb0GqpooBp3PiOALANx3E0GHF//E6tM1MKANAeDMNQtOeYMot/UxvLr55WtOcOH6sC0K4IpYCbEAtYOpaK+10GgDb33FJGn56+rGyl6hq/mCuoVLUVspjADABofdFUfSjljH2fDIMvbwHsDL/9AgDQJPGgVRdISVLVkWazzJYCALSHSPKQDDNYO66W11TOz/lYEYB2RSgFAECTHIiFt/3gnabZOQCgTRhmQJHkYddYfvW0T9UAaGeEUgAANEnIMrfdLIFm5wCAdhJNHXcd5wilANwCQilgG7bj+F0CgA40Fo80HJ/OFHjfAQC0jWjPUddxOX9JldKaT9UAaFeEUsA2PnF+Xr/23LT+x/l5fWVhVZlyxe+SAHSAiUTjUKpYtTWfLzW5GgAAbo0VjCsUH3ON5deYLQVgZwilgG1MZwqaz5f0lStr+h9Tl3WlUPa7JAAdYLuZUpI0tc4SPgBA+4j2HHMd01cKwE4RSgENZMoVLRU3QyjTkEa36QMDADsxGAkqYjX++KWvFACgnURT7lCqsH5edpVZvwBuHqEU0EDaszX7/mhYoW0uIgFgJ0zD0Pg2s6Wm1gty6CsFAGgTwciQAqG+zQGnqsL6Of8KAtB2WuYqO51O633ve59OnjypkydP6tFHH9XS0tIN7/fcc8/poYce0qte9Srdfffd+qf/9J/q3DneCHF70p6t2ce36QEDALdibJv3lLVyRSsl+tcBANqDYRh1s6Xyq6d8qgZAOwr4XYAkLS8v64EHHlCpVNLDDz+sarWqJ598UqdOndLHPvYxhUKhhvc7d+6c3vWudykajerHf/zHJUkf+tCH9KM/+qP69Kc/reHh4Wb+a6CDzHhDqev0gAGAnbree8rUel594WATqwEA4NZFU8e0vvDl2nF+7Ywcx5ZhtMz8BwAtrCVCqaeeekpzc3P6zGc+oyNHjkiSXvnKV+qhhx7Spz71Kb3jHe9oeL8Pf/jDyuVy+t3f/V29/OUvlyS9/vWv19vf/nY99dRT+umf/umm/Tugc9iOo1nP8r3tdssCgFtxvVBqOpPXtw72NLEaAABuXTgxIcMKy6kWJUl2JadS9oLCiXGfKwPQDloivn766ad18uTJWiAlSffee68OHTqkp59+etv7zc7Oqq+vrxZISdK3fMu3qLe3V6dPs/MDbs3lfEkle7OnSyxgqZ9ZCwB2UTy4/fvKlGemJgAArcwwLEV7jrrGWMIH4Gb5Hkqtrq4qnU7rxIkTdedOnDih5557btv7Tk5OanV11dV7amVlRevr69q3b9+e1IvO5126NxGPyDAMn6oB0Km2my11OV9SrlJtcjUAANy6aI+nr9TaGZ8qAdBufA+l5ufnJalh/6ehoSFlMhmtr683vO/DDz+skZER/cRP/IRefPFFnTp1Sj/5kz+pYDCod73rXXtaNzqXd+c9mpwD2Asv74vr9ftSevuhYQ15Zk3NZPI+VQUAwM5Fe45o66VlubCgcvHGm1YBgO89pbLZrCQpGo3WnQuHw5KkXC6nZDJZd350dFT/5J/8E/0//8//o/vuu0+SZFmWHnvsMdeSPmAn6pqcE0oB2AOv6E/qFf0bn23pbEELl1dr56bWC7qzN+FXaQAA7IgZiCqcmFAxM1Uby6+eVnDf6/0rCkBb8D2Usm37hrcxzcYTuv7zf/7P+sAHPqCTJ0/qHe94h6rVqj760Y/qX/7Lf6nHHntMb3jDG3ZUy8AAFwDdLleuaKFQqh0bkl41Maho0GpqHUND9SEsgOZq5uvwFeWy/npLKHWxWOJ9AF2P1wDgv528Du3ct2j21FTtuJo7q6GhN+1BVUD36IbPQt9DqXg8LkkqFot1566NXbvNVmtra3ryySd111136amnnpJlbYQG3//936/7779fP//zP69v//ZvVygUuulaFhczsrc0uEb3ObOadR3vi4aUWckp08QahoaSWlhovGQVQHM0+3XY7/noOb+S08X5VQW3+VIG6HR8FgL+2+nrsGpNuI7Xl89p/tJlmYH6FTEAbqxTPgtN07juBCDff9sdHR2VJC0sLNSdu3z5snp6ehSLxerOTU1NqVQq6Qd+4AdqgZQkBYNB/eAP/qCuXLmic+fO7V3h6Eh1S/eus207AOyWVCio3tDm90RVx9GFbP2XNQAAtKpguF/ByNCWEUf5tbO+1QOgPfgeSvX09GhsbEzPP/983bkXXnhBd911V8P7XZsBVa3W71B0bUngzSwNBLYajoZ1oi+hnqvL9SboJwWgSQ4m3N8kT9PsHADQZqIpzy58q6d8qgRAu/A9lJKkN7/5zfrSl76ks2c3k/RnnnlG58+f11vf+taG9zl69Kj27dunT37yk66lf8ViUZ/61KfU19eno0eP7nnt6Cx39Sf0zjv262dedVg//cqDOtFHnzEAe2u9XNELyxllKxXX+NQ6oRQAoL3UhVLrL8lx6icRAMA1vveUkqT3vve9+vSnP60HH3xQ73nPe1QsFvXEE0/oxIkTtV310um0vvrVr+ruu+/W+Pi4LMvSL/zCL+j973+/7r//ft1///2ybVuf+MQndO7cOf3H//gfFQwGb/BPBraXCvHzA2BvOY6jx56bUbZS/wv7dKYg23FkGoYPlQEAsHOh2AGZgZjsSk6S5FSLKmZmFEke8rkyAK2qJWZK9ff36yMf+YjuvPNOPfbYY/rwhz+sN77xjXriiSdqy/SeffZZPfroo3r22Wdr93vTm96k3/qt31Jvb69+9Vd/Vb/2a7+mnp4ePf7443rb297m178OAAA3xTCMbXvXFaq2LudLDc8BANCKDMNUtMe7hO+0T9UAaAeG4zhsN3cVu++hFXTKLgtAO2vm6/ALF5f0pxcWG567b3JIr9vX25Q6gFbCZyHgv1t9HeZWXtSV839QO7ZCvRp9+f8lg5m/wI50ymdhy+++BwBANxuPh7c9N71e2PYcAACtKJI8LBmbu6NXSysqF+p3WgcAiVAKkCSVbVvpTEEVdmwE0GRj2yzfk6QpduADALQZ0wrV9ZBiCR+A7bREo3PAb+lMQU+cuiDLMDQaC+vlfXF91/5+v8sC0AWiAUtDkZAWCvX9o1ZKFa0Uy+oNs/ECAKB9RFPHVVh7qXacXz2l1Mi3+1gRgFbFTClAUjq7sUSm6jhKZws0FwbQVOOJ6yzhy7CEDwDQXqI9R13HpdwFVcsZn6oB0MoIpQBJM56LvvHE9stpAGC3bbcDnyRNs4QPANBmAqEehaL7XWP5tTM+VQOglRFKoes5jqO0J5SauM4FIgDstuuFUlPrhFIAgPYTTR1zHdNXCkAjhFLoeiulijKVau04aBoajm2/lAYAdttwLKyg2Xir7Pl8Sfkt71EAALQDbyhVWD8n2y77VA2AVkUoha7nXbp3IB6RZTS+OASAvWAZhg5sE4Y7qn+fAgCg1QWjI7KCPbVjxy6ruH7ex4oAtCJCKXS9a03Or2HpHgA/XK+XHX2lAADtxjAMlvABuCFCKXQ9bz8pmpwD8MPY9fpKMVMKANCG6kKptTNyHMenagC0IkIpdLWKbetirugau17DYQDYKxNbAvGgZwnxbKagim03uyQAAG5LJHFQhhmqHVfL6yrlL/lYEYBWQyiFrnYxV1R1y7c1vaGAekIBHysC0K1SoaD+wcF9et+JCf383YeVCm6+F1Ucpy5ABwCg1RlmQJHkYddYfvWUT9UAaEWEUuhq3ubBLN0D4KdXD6U0GgsrYJqaTLrfj6bWWcIHAGg/0dRx13F+9YxPlQBoRYRS6GreflI0OQfQKiYTUdcxzc4BAO0o2nOHpM1l6eX8nCqlVf8KAtBSCKXQ1bw77zFTCkCrOJisD6VsmsMCANqMFYwrHB9zjbELH4BrCKXQtdZKFa2UKrVjyzA0Ggv7WBEAbBqOhhS2Nj+mcxVbVwplHysCAODW1O3CRygF4CpCKXStaMDUg8dG9T2j/TqWiulIT1QBk5cEgNZgGoYm4u6gnCV8AIB25O0rVcicl11lAw8AEtuMoWsFTVPHUnEdS8X9LgUAaubzRT2/nNVspqApT9+7qfW8XjuU8qkyAABuTSA8oEC4X5Xi0saAY6uwdlaxvpf7WxgA3zEtBACAFpLOFPSnFxb14mpWZdvdQ2o6ww58AID2YxiGoj2eJXxrLOEDQCgFAEBLud6GC0vFsta29MIDAKBd1PeVOiPHsX2qBkCrIJQCAKCFDEVCCl+nvx19pQAA7SicmJBpbX7xYlfzKmZnfawIQCsglAIAoIWYhqGxxPY7gU6ts4QPANB+DMNUpOeoayy/esqnagC0CkIpdKWvLKzq2YVVzeWKsh3nxncAgCYaj2+/hI+ZUgCAdlW/hI++UkC3Y/c9dKU/v7SsxWJZkhQ2TT3ysjHtj20/MwEAmul6odSlXFHFqq2wxfdKAID2Eu05oo15ERu9pCrFRZULiwpGBnytC4B/+I0WXSdXqdYCKUkq27YGwkEfKwIAt7HrNDt3JM0wWwoA0IZMK6JIctI1xmwpoLsRSqHrpD1bqo/Ewgox4wBAC0kGA+oLbT+ZeSpDXykAQHuKpo67jvNr9JUCuhlX4ug6M1n3xdz1lskAgF+uN1tqep2ZUgCA9hTtcfeVKmbSqlZyPlUDwG+EUug6ac+yl/HrXPgBgF8mrhOYp7MFVW02aQAAtJ9AuFfByL4tI44Kay/5Vg8AfxFKoavYjqN0pugamyCUAtCCvIG5seXvZdvRxZz7vQwAgHbBLnwAriGUQldZKJRUtO3acdQyaXIOoCXtj4VlbUmivPOipml2DgBoU3Wh1NpLcuyqT9UA8BOhFLqKt8n5eCIiwzC2uTUA+CdomtofC297foq+UgCANhWKHZAZiNeOHbukQmbKv4IA+IZQCl1lxhNKsXQPQCsbi0dkGtKBWFh39SVc56YzBTkOfaUAAO3HMAyW8AGQJG2/3zTQgdLsvAegjbzxwIC+b3xQQdNU1XF0+qtnVbra4DxbqWqxWNZgJORzlQAA7Fw0dUzZxb+tHefXTstxvpdVDECXYaYUukahUtXlfKl2bIhQCkBriwUsBc2Nj2rLMDSRiLrOs4QPANCuIsnDMozNORLV0qrKhcs+VgTAD4RS6Bqz2aKrUfBQJKRIwPKtHgDYqUnPkuNpz5JkAADahWkGFU4eco3lV0/5VA0AvxBKoWvMeJfu0U8KQJs5mGSmFACgc8RSx13H9JUCug+hFLrGYqHkOqbJOYB2Mx6PuD64F4tlrZcrvtUDAMDtiKaOuo5LuYuqltd9qgaAH2h0jq7x9sMjevPYgNKZgtLZgg55ZhwAQKsr2rb6I0FdKZRrY9PrBd3Vn7jOvQAAaE1WMKlQbFSl3MXaWH71jBKDd/tYFYBmIpRCV0mFgkr1B3VXf9LvUgDgptiOo98/N6d0pqCVUv2sqOlMnlAKANC2oqljnlDqNKEU0EVYvgcAQAszDUOXcsWGgZS0EUoBANCuop6+UoX1c7Lt8ja3BtBpCKUAAGhx4/Hte+BdzBZVqtpNrAYAgN0TjOyTFUzVjh2nosL6OR8rAtBMhFIAALQ4726hIdOo/d2WlPbsLgoAQLswDEPR1DHXGLvwAd2DUAoAgBbnnSnleM5PrbOEDwDQvhqFUo7j/bQD0IlodI6ON72e12fTVzSeiGg8HtFkIqLecNDvsgDgpo1EwwoYhipXf0Ev2+5f1OkrBQBoZ5HEQRlmSI5dkiTZlaxKuQsKx8d8rgzAXmOmFDredKagdLagZ+ZX9Pvn5vTHs4t+lwQAO2KZhg7Ew9uen8kUVOUbZQBAmzJMS9GeO1xjLOEDugOhFDpeOuueQeDtzQIA7cC7hC9obPaVKtmO5nLFZpcEAMCuoa8U0J0IpdDRHMfRTMbdAHiCUApAGxrzvHcFLcN1TF8pAEA7i/TcIWnzs61cuKxKccW3egA0B6EUOtpqqaL1crV2HDAMjUS3XwIDAK1qwjNTqlC1XcfTGXbgAwC0LysQUzgx7hrLrzFbCuh0hFLoaDOebdIPxMOyTGObWwNA60qFAkoGrdqxp9e5pjN5dioCALS1aM9x13F+9ZRPlQBoFkIpdLQ0S/cAdAjDMDTmmS21NWNfL1e1VCw3uSoAAHaPt69UITMtu8pMYKCTEUqho3lDKW+jYABoJ95gPWZZrmOW8AEA2lkwMqBAeGBzwLGVXzvrX0EA9hyhFDpWxbZ10bMb1Xgi6lM1AHD7vDOlbLmX69HsHADQ7tiFD+guAb8LAPbKpVxJlS39VVKhgFIhfuQBtK+xeET7IiGNJcIaj0fkONIfzizUzk9nCKUAAO0tmjqm9ctfqh0X1s7IcWwZBvMpgE7EFTo6VjrL0j0AnSVsmfqXr5isHRertj4zs1CbL7VQKCtTrigR5OMdANCewvFxmVZUdnXjixa7WlAxM6NI8qC/hQHYE8TN6FgznhkDNDkH0GnClqn9sbBrbIa+UgCANmYYpqKpo64xlvABnYtQCh2LmVIAusGkp1ceS/gAAO0u2uPpK7V2Wo7jbHNrAO2MUAodab1c0XKxUju2DGk0Hr7OPQCgPR1MugP3qXVmSgEA2luk54hkbO4wWykuqVJc9LEiAHuFUAodqVi1dSwVU9Ta+BHfHwsraPLjDqDzeGdKXcwVVKraPlUDAMDtM62wIolJ11h+9ZRP1QDYS3RCRUcajIT04LEDchxHi8Wy8hUu0AB0lkKlqtlsUTPZgpJBS+vlqiSp6kiz2YIO98R8rhAAgFsXTR1XYf1c7Ti/elo9w9/mY0UA9gKhFDqaYRgajIT8LgMAdtVfzi3rf6av1HbdG4oEa6GUJE1nCKUAAO0tmjqm5dnP1Y6L2VlVy1lZwbiPVQHYbaxnAgCgzQyEg9ra7rViu5u/0uwcANDuAqGUgtHhLSOO8msv+VYPgL1BKAUAQJsZT7ibm6+VK67j6UxBNrsUAQDaXDRVvwsfgM5CKAUAQJtJBgPqDW2uwK86UtgyasfFqq25fMmP0gAA2DXR1HHXcWHtJTl2ZZtbA2hHhFLoOKulCjMEAHS8sbh7tlRvKOg6nl5nCR8AoL2FovtlBRK1Y8cuq5CZ8q8gALuOUAod58lTs/rFr57VEy/O6o9nryhXqd74TgDQZrxL+EwZruMp+koBANqcYRj1S/hWWcIHdBJCKXSUXKWqK4WySrajc+t5/fmlZVmGceM7AkCbmfDMlMpWPH2l1vNymDUKAGhzjUIpPt+AzkEohY4ymy24jkeiIYUtfswBdJ7ReFjmlsx9rVyVJffxSom+GwCA9hZOHpJhbOmjWF5TOT/nY0UAdhNX6+goMxl3KOVd3gIAnSJomhqJhl1jg9GQ63iKvlIAgDZnmkFFeo64xljCB3QOQil0lHRdKBX1qRIA2HvjniV8Ec/M0Gn6SgEAOgB9pYDORSiFjmE7jtKe5XveCzYA6CTe2aBl291jY8oT1AMA0I6iPUddx6X8JVVKaz5VA2A3EUqhY1wplFWo2rXjiGVqMBK8zj0AoL15g/fFYtl1fDlfYgdSAEDbs4IJhWIHXGP5tTM+VQNgNxFKoWOkPctUxuMRmey8B6CDDUSCriV7xaqtobA7jJ9hCR8AoANEU8ddx/nVUz5VAmA3EUqhY8x4l+7R5BxAhzMNwzVbyjSkPs8M0al1lvABANqft69UYf287GrJp2oA7JbAjW8CtAdvk/MJQikAXeA1Qz06moppPB7RaDysF5azOr2aq52n2TkAoBMEI0OyQr2qllY2BpyqCuvnFOu909e6ANweZkqhIxSrtubz7m9KaHIOoBu8oj+pbx/p02QyqqBp6mDS/d43my2qbNvb3BsAgPZgGIZidUv42IUPaHeEUugIs9mCtu45NRQJKhqwfKsHAPySCgXVG9qcCF11HF3IFn2sCACA3RFNuXfhy6+dluPwxQvQzgil0BG8S/foJwWgmx1MRl3HLOEDAHSCcGJShhWuHduVnEq5Cz5WBOB2EUqhI9Q1OY9Ht7klAHS+yYT7PXBqnVAKAND+DMNSNHmHa4wlfEB7I5RCR3j9vpS+Y6RPBxMRBU2DJucAupq3r9R0piDbcba5NQAA7cO7Cx+hFNDe2H0PHeFYKq5jqbgkqWo7MgyfCwKAJrIdR+lMQensxv9WixVFLVP56kafjULV1uV8SSOx8A0eCQCA1hbtuUOSIV3tKFsuLKhcXFIw3O9rXQBuDTOl0HEs05BJKgWgi9iO9OSpC/ps+or+bimjmWxBo54Aaoq+UgCADmAGogonJlxjzJYC2hehFAAAbS5gGtrvCaGSIfcOpNPr7t57AAC0q2jquOuYUApoX4RSAAB0AO+uo94WUuzABwDoFN6+UsXMtOwKn3NAOyKUAgCgA4zH3aHUaqmiwJalzCulilaK5WaXBQDArguG+xWIDG4ZcZRfO+tbPQBuHaEU2tpioaSvL65rqViWw85SALqYd6bUxVxRo7GQa2w6wxI+AEBniPV4duFbYwkf0I7YfQ9t7YWVrD6XviJJigcsfef+Pn3HSJ/PVQFA8/WFAooHLGUrVUlSyXY0FAlpJlus3WYqk9crB5J+lQgAwK6Jpo5r7fIzteP82hk5TlWGYV3nXgBaDTOl0NbSW771z1aqsth1D0CXMgyjbrZU0HJ/zE+v028DANAZQvEDMgOx2rFTLaqYmfGxIgC3glAKbS3tWYri7akCAN3E+x5YqNiu4/l8SfmrM6kAAGhnhmEq2nPUNcYufED7IZRC21otlbVartSOA0b9lugA0E28odTFfFHD0c2+Uo6kGfpKAQA6RDR13HWcWz1Fn1mgzRBKoW15Z0mNxsIKmCzfA9C9xuJhbX0XXMiXNBZzB1XTGZbwAQA6QyR5WNrSQ6paWlG5sOBjRQB2ilAKbSuddYdSEwmW7gHobpGApaGIe2ZUIuT+qJ9iphQAoEOYVkiR5CHXGEv4gPZCKIW25V2C4m3wCwDdyPteaHtWMcxmCqrY7l5TAAC0q2jqmOs4v0YoBbQTQim0pYrt6MKWbc4lmpwDgCSNed4LF/IlpYKB2nHFcXQxV/TeDQCAthTtcYdSpeysquWsT9UA2ClCKbSluXxRlS1NDHuCllKhwHXuAQDdYetS5qFIUIORkCaT7qBqap0lfACAzhAI9SgY3e8ay6+d8akaADvFVTzakrfJ+XgiIsOgyTkA7IuG9NCxUY3FI4oGNpq//vXlFX1jKVO7zUaz8z6fKgQAYHfFUse0mr9UO86vnlJi4FX+FQTgpjFTCm2pLpSKR32qBABai2UYOpqK1wIpSZpMuN8jpzN52WyZDQDoEN6+UoX1c3Lsik/VANgJQim0pRl23gOAmzYcDSlibX7k5yq2rhTKPlYEAMDuCUZHZAWTtWPHLquwft7HigDcLEIptJ1MuaKl4ubFlGlIo7GwjxUBQGszDaMuvN9YwgcAQPszDEPR1HHXWH6VXfiAdkAohbYz69l1b380rJDFjzIAXM9BzxK+qXVCKQBA54j2HHUd59dOy2GpOtDyaHSOtnM8FdNPvGJS6UxBM5mC+sJBv0sCgJZWqFS13zOjdIqZUgCADhJJHpJhBuXYGysqquV1lfOXFIqN+lwZgOshlELbMQxDg5GQBiMhfetgj9/lAEBLOrOa1TeWMkpnCloolPR944OyDKl69Uvj5WJFq6WKUiF+FQAAtD/DDCiSPKL86ou1sdzqaUIpoMWx5gkAgA50KVfS31xZ0+VCSY6ki7miDsToKwUA6FzeXfjoKwW0PkIpAAA60LinsXk6U9Bk0t1Xapq+UgCADuLtK1XOz6lSWvWpGgA3g1AKAIAOdCAWdn3ILxbL2h8NuW4znSk0tygAAPaQFYwrFB9zjTFbCmhthFIAAHSgkGVq2NPc3DAM1/GlXFGFarWZZQEAsKdiqeOuY0IpoLURSqGtfPz8nD4/e0XfXMkoU674XQ4AtLTxuDuUWiiUNBTZnC3laGNZHwAAncLbV6qQmZJdLfpUDYAbIZRC28hXqvrqlXX9+aVl/c6ZS/qlr51XsWr7XRYAtKzxhLuHVDpT0MGku9fUFKEUAKCDBMKDCoT6Ngecqgrr5/wrCMB1EUqhbcxm3RdO+6IhhS1+hAFgO+NxdwA1my1owjNGs3MAQCcxDINd+IA2whU92saM59t878UWAMBtMBJUZEt4n6/aSoWCrtukswVVbafZpQEAsGeiDfpKOQ4rLIBWRCiFtpH2zJSaSBBKAcD1mIahMU9fqbVSWcmgVTsu244u5ui1AQDoHOHEuAxr81rBruZVzM76WBGA7RBKoS04jlPXjHecUAoAbmg87ukrlStq0tNrajrDEj4AQOcwDEvRnjtcYyzhA1oToRTawmKxrPyWpuZhy3TtIAUAaGw84Z4ptdHs3B1KTdFXCgDQYRot4QPQegil0BYa9ZMyDcOnagCgfYx5+u/N5Ys6EHOH+tOZghyHvlIAgM4R7TmirZe7leIVlQuL/hUEoCFCKbSFulCKpXsAcFMSwYD6w5vNzW1HqjpS2Nz8FSBbqWqxWPajPAAA9oRpRRROTLrG8mvMlgJaDaEU2kJdk3N23gOAmzYWD6s3FNBdfQm9dXxQA5Fg3WYRLOEDAHSaWOqY65glfEDrCfhdAHAjpaqtOc/OUMyUAoCbd/+hYQVM9/dQk8mIzqzlasfTmYJeM5RqdmkAAOyZaOqYli98vnZczMyoWsnLCkSvcy8AzcRMKbS82WxBWzudDISDigWsbW8PAHDzBlKS6nbgY6YUAKDTBMJ9Ckb2bRlxVFh7ybd6ANQjlELLq1u6xywpALhtGxtGbB4vFstaL1f8KwgAgD0QrVvCd8qnSgA0QiiFlpemyTkA7LqQZWo0FnaNTTNbCgDQYepCqbWX5NhVn6oB4EUohZZnGoaCW77OH6fJOQDsioOeJXzTni8BAABod6HYAZmBeO3YsUsqZqZ9rAjAVjQ6R8v70Tv2q2o7ms8XNZMtaCQavvGdAAANlW1bl3JFrZYqmkxG9VfzK7Vz9JUCAHQawzAU7Tmq7NLXamO5tdOK9Bz2rygANYRSaAuWaWg0HtEos6QA4JbkK1X91qkLmssXVXWkkGnoJ14x6brNpVxRxaqtsMVEagBA54imjrtCqfzqaTkH3iLDMLa/E4CmaJnfOtPptN73vvfp5MmTOnnypB599FEtLS3d8H5LS0v6uZ/7Od177726++679a53vUtf+9rX9r5gAADaSMQytVyqqHp1O9OS7ShXsTUYCdZuY6t+cwkAANpdJHlIhrE5H6NaWlG5cNnHigBc0xKh1PLysh544AF97Wtf08MPP6yHHnpIX/jCF/TQQw+pVCpte79MJqN3vvOd+tznPqcf+ZEf0b/4F/9C8/PzeuCBB3T69Okm/hsAANDaDMPQeNy9/DmdLWjS21eKJXwAgA5jWiGFk4dcY/lVrheBVtASy/eeeuopzc3N6TOf+YyOHDkiSXrlK1+phx56SJ/61Kf0jne8o+H9PvjBD+r8+fP6nd/5Hb32ta+VJL31rW/VG9/4Rj3xxBP6j//xPzbt3wEAgFY3nojo1GqudpzOFHQwEdHfXFmrjU1nCKUAAJ0nljqmwtqZ2nF+9bRSI9/hY0UApBaZKfX000/r5MmTtUBKku69914dOnRITz/9dMP7OI6jT37yk/ru7/7uWiAlSUNDQ3r00Uf1mte8Zs/rxt6q2o4cx/G7DADoGN7dS2ezBU0m3TOlZjIFVXnvBQB0mEjqmOu4lLugajnjUzUArvE9lFpdXVU6ndaJEyfqzp04cULPPfdcw/vNzs5qfn5e9957r6SNkCqbzUqS3vnOd247uwrt468vr+jff+28fvvMRX3x4pLmckW/SwKAtjbmCaXm8yUlApbiAas2VrId3m8BAB0nEEwqFBt1jbGED/Cf76HU/Py8JGl4eLju3NDQkDKZjNbX1+vOTU9PS5IGBgb0y7/8y3rNa16ju+++W29605v0hS98YW+LRlPMZAvKVqp6cSWrP76wqLNruRvfCQCwrWjA0tCWxuaOpAu5og4m3WHVFH2lAAAdKOqZLZVfI5QC/OZ7KHVtdlM0Gq07Fw5vNGTN5erDiLW1jf4Xv/Zrv6Y///M/17/6V/9Kv/zLv6xIJKJ//s//uZ555pk9rBrNkM64d4AaT0S2uSUA4GY1XMLnbXaeYQc+AEDnifa4Q6nC2jnZdtmnagBILdDo3LbtG97GNOuzs2u78q2trenzn/+8UqmUJOkNb3iD3vSmN+k//af/VFvad7MGBhI7uj32zkqhrJVSpXYcMA19y8SggpbvOWpTDA0l/S4B6Hqd+jp8Wa6gry5uzkCeL1f01iMj+mz6Sm0snStocDAhwzD8KBGQ1LmvQaCddNrr0HESWpzuVbmwcvW4orAxp96hl/tbGLCNTnsNNuJ7KBWPxyVJxWJ9/4prY9dus1UsFpMkvfnNb64FUpLU09OjN7zhDfrkJz+pbDbb8L7bWVzMyLZp7toKXlh2Nx3cHw1rZSnrUzXNNTSU1MJC/ZJVAM3Tya/DXs/H3NmljCIHKgqahspXPwNXixWdml3SQCTkQ4VAZ78GgXbRqa/DcOKoyoVna8dz6a+rbIz7WBHQWKe8Bk3TuO4EIN+nnYyObjSbW1hYqDt3+fJl9fT01AKora71oOrv768719/fL8dxGi77Q3uYYekeAOyJkWhYgS0zoNbLVWUq1bplfSzhAwB0orq+Uqun2fEb8JHvoVRPT4/Gxsb0/PPP15174YUXdNdddzW839GjRxUKhfTSSy/VnZudnVU4HG4YWKE9zGQ9oVScUAoAdoNlGjoQD7vG0tmCDibdfaVodg4A6ESRxKQMc3MmsF3JqJS76GNFQHfzPZSSNpbgfelLX9LZs2drY88884zOnz+vt771rQ3vE4vF9IY3vEFf/OIXdebMmdp4Op3WF77wBX3P93yPLMtqeF+0tqrj6IInlJpgphQA7Bpv0J/OFHSwrtk5oRQAoPMYZkCRniOusfwqu/ABfmmJUOq9732vUqmUHnzwQX3oQx/Sb/7mb+r973+/Tpw4ofvuu0/SRtj06U9/Wul0una/n/qpn1IymdS73/1ufeADH9ATTzyhd77znYpEIvqJn/gJv/51cJvmc8VaXxNJSgYt9YZ8b38GAB3DuyQ6nS1oPBFx/VKwUCgrU64IAIBOE0sddx0TSgH+aYlQqr+/Xx/5yEd055136rHHHtOHP/xhvfGNb9QTTzyhUGhjauWzzz6rRx99VM8+u9mUbmxsTH/wB3+g1772tXryySf1gQ98QC972cv00Y9+VOPjNKtrV42W7rEDFADsnolERMdTMb3xQL8eOjaqdx8dVdgytT/mXtbn7e8HAEAniPTcIWnz+qJcmFeltOJbPUA3Mxy6utWw+15r+Ni5Of3tlu3K3zI2oO/a3z39wTpllwWgnXXr6/CPZhb0zPxK7fg7Rnr1feND/hWErtWtr0GglXT663D+9FMqZmdqx31j36vk0EkfKwLcOuU12PK77wFedTvv0eQcAJpi0rOsb2qdmVIAgM7UaBc+AM1HKIWWkqtUtVgs145NSWOEUgDQFJOeZucXcgWVqrZP1QAAsHe8oVQhMyW7WvSpGqB7EUqhpaQ9s6RGYmGFLH5MAaAZekIB9YeDtWPbkWazzJYCAHSeYGRQgfDA5oBjq7B2dvs7ANgTXO2jpSRDAb1+X0oHYmGZYukeADTbQe8SPpqdAwA6lHe2VG71lE+VAN0r4HcBwFajsbDeNrlPklSq2irZLBsBgL1SdRxdzpeUzhSUzhb0msEeTSaj+uqWzSam1/M+VggAwN6Jpo5p/fKXaseFtTNyHFuGwdwNoFkIpdCyQpbJ0j0A2EOfmV7Q/1lYrR33hgJ6RX/SdZuZTEG248g0DO/dAQBoa+H4uEwrKru68QWMXS2omE0rkpj0uTKge3DFDwBAlxqLh13Hs9mChiJBxQKbvx4UbVtz+VKzSwMAYM8ZhqlIzx2usTxL+ICmIpQCAKBLeXc3TV9tau7dhY8lfACAThVLHXcd51dP+1QJ0J0IpQAA6FL7oiGFzM1lebmKraViWQc9odRUhlAKANCZIj1HpC09pCrFJZULV3ysCOguhFJoGS+t5rRervhdBgB0DdMwGs6Wmky6x6bX83Icp5mlAQDQFKYVViRx0DXGbCmgeWh0jpZQqFb1odMX5EjqCwc0EY/q7YeHaawLAHtsPBHRuS3L89KZgu7qSypgGKpcDaLWylWtlCrqCwf9KhMAgD0TTR1TYf1c7Ti/elo9w/f6WBHQPZgphZYwmy3q2nfwy8WKLuWKBFIA0ATjDWZKBUxD4wn3+BR9pQAAHSqaOuY6LmbTqlZyPlUDdBdCKbSEdKbgOvZeDAEA9ob3/fZSrqiybWvSMz5NXykAQIcKhHoVjAxvGXGUXz3jWz1ANyGUQkvwhlIThFIA0BTJYEC9oc3V/FVnI5g6mPQ2Oy947woAQMfwzpbKr9FXCmgGQin4znEczWQ9M6XihFIA0Cx1S/gyBU0kItq6iPpyvqRcpdrcwgAAaBJvKFVYOyvHZhMmYK8RSsF3S8Wy60InZBraFw35WBEAdBfvEr50tqCIZWkkFnaNz7CEDwDQoUKxUZmBRO3YsUsqZKb8KwjoEoRS8N2MZ0nIWDxCk3MAaKJGzc4l1fWVmlpnCR8AoDMZhlG/hI++UsCeI5SC79JZ+kkBgJ9G42GZW74LWC5WlClXdDDh7itFs3MAQCeL1YVSp+Q4zja3BrAbCKXgO5qcA4C/gqap/dHNpXq9oYBWSxVNJt3vx7PZjZ35AADoROHkIRnGls0/ymsq5+d9rAjofIEb3wTYO6WqrUv5omtsjCbnANB0f2+0X9LGe3DPlt34+kIBLZc2Gr1WHUez2aIOeXbmAwCgE5hmUJGew8qvbu68l187rVBsxMeqgM7GTCn46mKuKHvLjNj+cFCJIFkpADTby/sSenlfwhVISdKkJ4CaXmcJHwCgc0VTx13HWwMqALuPUAq+qlu6xywpAGgpk/SVAgB0kWjPUddxKXdRlfK6T9UAnY9QCr6a8TQ5925LDgDw10FPX6npTEE2TV8BAB3KCiYUih1wjRWYLQXsGUIp+Mo7U4pQCgBay1AkpKi1+etCoWrrcr7kY0UAAOytqGcXvhyhFLBnCKXgG8dx9MCxUd03OaS7B5IaiYY0smX3JwCAv2zHkWkYdUv4pljCBwDoYN5QqrB+TnaVL2SAvUBHafjGMAztj4W1PxbW6/b5XQ0AoFi19c2VjGYyBc1eXV794y+f0GQyohdXs7XbTa8X9HretwEAHSoY2Scr1KtqaWVjwKmqsH5esd7j170fgJ0jlAIAAJKkquPoD87N144NSaWqrYM0OwcAdBHDMBRNHVNm4f/UxvKrpwilgD1gOA7dSq9ZXMzIttv7P8f6wrNafOEPb+m+xjdNTf7Tf1M3XshMa/7rT8mpVHb8mNXn1nTkn/9a3bhdyWv2b/4/2YVCg3tdn3OpoP2v/+eKHDxYd+7Cs/9Z5dzijh9TRVu94Tep97u+u+7UwgsfVXbh+Z0/pqTIxQmN/Mh768Z5nnieeJ52rpnPUySwoFNf/K+39jw9f53n6Su38Tzds93z9GsqZ6/s+DFVtNUbqX+e/tM3prRYLNeOfzD3P7XfvnzTDxu5NKGRH23wPF35ihb/7lM7r1OS8aKlyX/W6PU0o/m//ZCccrnBvbZw6v9efWFNR97/WN1N7WpBF778q6pmMpv3a/SrkuP5U5J9uajR7/6/Gj5PF//PB1Scn/E8hrNtfTUlW33Db234elp88Y+09vxfNqjJ8yANHj9aOaaRdz5c95j51dOa++wHa7er3XXrY3prvHpsLcU1+S/qn6dy4YouPv0bqmbWG/x3a/C4jiTbkVO05SwUdce/+c26x5Sk+d/+kFb/4s8bnruRiZ/71w2fp5U//6Iu/85Tt/SY+971YMPnqTA1pZl/+69v6TFT3/ldGn73Qw3PnX74wVt6zPDEpCZ/of55kqTpX/z/qTgzfUuPe+yJpxqO8zw9eEuPyfPE89QIz9ODt/SY7fo8DQ0ltbDQ/js/mqahgYHEtueZKdVh7GpRZk/wlu7rWNXG43ZFRsyQoZ0/rh1u3LbMkSOFqjJDO3/M6sr2Fx+2eWv//k6hKm2zTNy2C7f831QXt3lMnieeJ56nnWvm81Qty4jvwfMUrsoM38LztHq956kgM7V7z9N4IuIKpS4nRnTAXL75B55rPGxXCjJ7QzuscoMT2O71VJaRMGRo549rx6zGj+nYcqJlmbfQ49Cp2Nueq1prssai257f9jGv83qqVK7IOhzf8WNKkl5s/CVcKT8v687kLT2k85XGz1OltCrjsKmAUjt+zMpza43/WY6j4ssuKDQxLhWqckq2VLTlFG2pYMspVjePi57j0vbPEwAAaC5CKQAAUDMej+hri5vfyl12Bnysps2092TrPXQb/2GK2wWSRSlgbwSyOwxl7Uvbz1YshmZlvbp3I+gq2vXBVsmWyLQAANg1hFLwxanVvE7ZL9ewFrXPWFTI2PkSGQDA7htPRFzHhFI+6/Kgyyk2ToDsys6XwdYe8zozpUqhCwq+vv/G9/fMvnKy2/8eU7VzUtTaCNgItAAAcKGn1Bad0FPKrhRkV2+tAa0ZiMu06pdA2HZZdjlzS49pWBFZgfrlCr9/9pK+vrTxmIak+8YSurs/Une7ho9pBmUFG69JrZRWJecWfuMzDAVCvQ1PVcsZOfYN+pVswwr1yDDql4hc73nq709oaWn7/97NfJ4cx1a1tHprj9nhz9ON8Dz1NjzVLs/TQH9Y85e2WYd2A9d7nirXdvHZ6WOaQQWCjZdUVYorcm7hSteQoUC4r/7xbEe/+NWXVNnycfgTd/YqbEr/9YW067Y//vJxRSz3csVAuLfh81St5GVXcjuuU5KsYEKmVb+czq6WVC03Xt61ybn6f+7GSlYg3vBn33FslfPzqt3DadCQacvjXvu7I8m0IgpFG29LWMyk5TiVzcdoUJMa/fMMS9GeI40fMzurau39pP7+dY/vbN4umjre8Ge/lJ9XKXepvi7XYzvu/3+tT1XPkcY/U6UV5VdOue7VqKatjx2NWsqsrSqaOq5I8mB9nblLmjv1wbrxmxHru0uDB3+44bn01/+DHHvn275bwaQO3PV/Nzx3+ex/V2HtjCTJMAIyA1GZVuTq/6IyAxEZW4+tiMzAxt/DsQMyzMZLTYG91in9bG5HtZLXhb/7FW19nxp9+f/V8L0O2G2d8hqkp1SXMQMbv8js6mOaQZm7/MY7k938htORNJzoVSC8814bXoHQzvtV3Mh2F+y343rPUziWVCC78/4we/E8GYa5Jx+6nfA83fJj8jzt+mPuyfNkhXb9v6lhmAqGrz8D41YEwr27+3imodF4RDOZzffpuXJEd/UnFY3mNJffvGC/WI7pePzmehpZgWjDsO52mFZIpjW4q49pGKZCsf27+piSFE6M7/5jxsd2/TFD0WGFosO7+piBUK+S+163o/vc6BfxYHRYB+76SdnVvOxqQXbl6p+1/+WvBtaFLbfZODatxj+HjmPfUiAladvHlCSnuuV3Hqeianld1fLNXWQceMVPyTLrHztz5W+UWz3lCbG2hFpWZEv4FZVhBmUYxs7/xYAuZwWiCicmVMxsNsLOrZ5Wzw7f0wBsj1AKTbdermi5uDnN3TKk0fjOG8oCAPbGuCeUSmeLuqs/qclE1BVKTWfyOt57i422gdtgGKasYFxWcOc/f9suEnBspfa/4brB1taAaSvT2j4Yt7e5z81oNENQ2pjRVlh7aSePVAuuBg/+sEKx0bpbVIrLKuXn6mdxmWECLXS1aOqYK5TKr54ilAJ2EaEUmm424/7lbH8srKDZeLcqAEDzjcfdF9jpzMbyyMlkRF9e2FwqOpW59YttwC/bBSyGGVBq5Nuve1/HceRUi5uzr64GV8Y24ZGkWsCzEU7dfJsIwwrLMBr/fmRXdrpk2ZZdyV1dQtv4MfPrZ7Wc/myjSmRa4bqlhVtnZEV6jm67dBVod9GeY1q58Ce142JmZmNH2V2epQ10K0IpNN3WpXtS/cUPAMBf3mbnF3JFVR1HBxPuZUSzmYIqtq0AXyygSxiGIWOHS4aHjz0k6WqgZZcaLDfM13rj2VsCL8PY/tf025p9tc0y2u2bxzu1WrczEEwSSqFjBSMDCoQHVSleuTpiK7/+kuJ9d/laF9ApCKXQdGnPN+veix8AgL96QwElApYylaokqWw7ms+XNBoLKxUKaLW0sQS74ji6mCtqIrG7vaKATmQYhgwrvLEk7zZ79vWOfo8q5ddsCbMKrrDL8R47m20TtltqeHvLDLf/XS679HfbNtYH2kU0dUzrl6/UjvOrpwmlgF1CKIWmqjqOZj0zpSbiXMwAQCsxDEPjiYi+uZKtjaUzeY3GwppMRPSNLTuETq3nCaWAJgvFRhTSyE3f3rErmzOwzMbhUDAyqGjqmKspvF3N39SOqduFUrmVb2px+pMKhAc1eOh+ZlOhbcVSx7R++ZnacX7tJTlOteFuswB2hlAKTXU5X1LJ3uynEA9Y6gvzYwgArWY87gmlsgW9TtLBZNQdSmUK+k4f6gNw8wwzIMtMygomt71NYuBVSgy8qm7csavuXQxrodVmXy0r1Ft3v3JxSYvTfyhJqhSvaP7UE+ob/34lBl65W/9aQNOE4mMyA7Grfdk2dtUsZmYUSR7yuTKg/ZEGoKlmPEv3JhIRdnQBgBZ0bWl1LGBqPB6pzWqd9MyKml7Py3YcmbyXAx3JMC1Z5s53Oly99EU5drF27DgVLc18WsXMtPrGv0+mGdztUoE9Yximoj1HlV36em0sv3qaUArYBYRSaKp01r1bDE3OAaA1TSQi+slXTKo/HHR9eTAcDSlimSpUbUlSvmproVDScHT73ccAdJ/+8R+QYViui3hJyi59TaXcRQ0eul/ByKBP1QE7F00dqwuleg+8mS/YgdvEdjloKpqcA0B7CJqmBiKhul+2TcPQhOe9e3r91hskA+hMphXSwOR96p94W91OguXCZc2d+qCyS3/nU3XAzkWSR6QtPaQqpWVVCleucw8AN4NQCk2Tr1S1UNhslmlIGmOmFAC0nYPeJXyZ/Da3BNDtEgOv0vDxH1MgPOAad+yyFqc/qaWZp+XYlW3uDbQO0wopkjjoGsuvnfanGKCDEEqhadKeXfeGoyGFLX4EAaDdTCbdodQUoRSA6whFhzVy/GHF+u6qO5dZ/BvNnf4tlYtLPlQG7Ew0ddx1nFs95VMlQOcgEUDTLBfLMresAmHpHgC0p7F4WNaW9/PlYkWrJWY6ANieaYU1MPlD6hv/ftcSKEkq5+c09+Ljyi2/4FN1wM2Jpo66jkvZWVXL2W1uDeBm0OgcTfO6fb26e7BHF7JFpTMFjcZpigsA7cJ2HM3nS+oNBRQNWDoQi2hmywzY6Uxe39K//XbzAGAYhpKDr1Y4NqorU59QZcvsKMcu6cr0/9Bo/IACoZSPVQLbC4RSCkZHVM7P1cbya2eUGHiVf0UBbY5QCk0VNE0dTEZ10LP0AwDQmp5dWNXXFtd1IVtQyXb0Dw+P6JUDSU0mo+5Qap1QCsDNCcX2a+T4w1qc+YzyK9+sjfeNvolACi0vmjrmDqVWTxNKAbeB5XsAAGBbi4Wyzq/nVbIdSZv9AQ96d+DLsAMfgJtnWhENHrxffWPfKxmmor0vU2LopN9lATcUSx1zHRfWz9KsH7gNzJQCAADb8vb/S18NnyY8O/BdyhVVqFYVsdy9YgBgO4ZhKDl0UuH4mALhfhmGceM7AT4LRvfLCiZVLa9L2thJsrB+vq7fFICbw0wpAACwrfG4O5S6mCuqYtuKBy0NRUK1cUebgRUA7EQoNirTarwBjmNXtDTztCql1SZXBTRmGIaiPe7ZUvm10z5VA7Q/QikAALCtnlBAqdDmxOqq4+hSriRJOph0X0ROEUoB2GXLF/5EmcW/0dyLjyu/esbvcgBJG32ltsqvnpbjOD5VA7Q3QinsuWLV1q98Y0ofPXtJ/3tumW/SAaDNeGdLbfaVci/hm17PN60mAJ0vt/yCMleelSTZ1bwWzv13rVz8MzmO7XNl6HaR5CEZZrB2XC2vu5qfA7h5hFLYc7PZgpaKZX1jKaOn01f08fO8YQNAO6kLpa5+uTDp2Uk1nS2oavNNMYDdkduyM981a/P/W5df+m1VrvbzAfxgmAFFkoddY7nVUz5VA7Q3QinsOe/MKG/TXABAaxvzNju/OlOqLxRQT3CzsXnZdnQxV2xqbQA618DBH1Zq/xskuRugFzMzmnvxv6mwds6fwgBJ0dRx13F+lb5SwK0glMKeu3bxcs14PLrNLQEArehALOz6hWGpWFa2XJVhGJr0LuHLsIQPwO4wDEOpkW/XvqPvlhVIuM7ZlZwun/2IVi59keV88EW0x73bXjk/p0ppzadqgPa1bSj1uc99Tv/u3/07feITn1ClUnGde+SRR/a8MHQGx3E045kpNcFMKQBoKyHL1Egs7BqbzTZewjdFXykAuyySmNTInf9EkeShunNrc3+hyy/9rqrljA+VoZtZwbhC8THXGLOlgJ1rGEp9+MMf1r/9t/9W2WxWjz/+uH7kR35EKysrtfNf+cpXmlUf2txyqaJspVo7DpmG9kVD17kHAKAVjXn6Ss3Ump27x6czBXYgArDrrGBcQ0feqdTId9WdK2bO69KLj6uwPtX8wtDVoj3eXfjoKwXsVMNQ6vd+7/f05JNP6t//+3+vz372szpx4oQeeOCBWjDFL5u4Wd5+UgfiEVmGsc2tAQCtytsPcPbq+/twLKywufnrRLZS1ZVCuam1AegOhmEqtf+7tO+OfywzEHedsysZXX7pd7Q695dcq6BpYp6+UoXMlOxqyadqgPbUMJRaWFjQnXfeKUmyLEv/+l//a91zzz1697vfreXlZRmECrhJdUv34izdA4B2VLcDX7Yg23FkGUbdsmz6SgHYS5HkYe2/8xGFE5OeM44yi38rx2bDBTRHIDKoQKhvc8CpqrB+1r+CgDbUMJTq7+9XOp12jf3Mz/yMXve61+nd7363qtVqo7sBddJZ94UJO+8BQHsajAQVsTZ/bShUbS1enRE1mXS/t08RSgHYY1YwqX13vEs9w9+xOWhYGjx0v0yL3zfRHIZhKJryLuGjrxSwEw1DqXvuuUef/OQn68b/1b/6V3rd616nYpFvH3BjZdvWJc/W4IRSANCeTMPQkZ6oDiWj+s6RPr3zjv3qCQUkqX4HvvVCo4cAgF1lGKZ6R/+eho78qMxATH0H3qRwbNTvstBl6kKptTPsCAnsQKDR4M///M9vOxvq537u5/Se97xnT4tCZ7iYLaq6ZUl/XzigZLDhjxwAoA28847GF3vj8YhMQ7KvvucvFstaL1d4zwfQFNGeO7T/zn8mMxDb9jaO49CCBHsinJiQYYXlVDe+jLcrOZWyswonJnyuDGgPDWdKhUIhRaPRRqckSaOjfAOBG0tn3d+Ue/uRAAA6Q8gyNRoLu8am11nCB6B5rGB829CpUlzW3KnHVczONrkqdAPDsBTtOeoaYwkfcPMahlLAbqhrcp7YPugEALS3g94lfBmW8AHwn2NXtHD+4yrn5zV/5imtXf4yu/Nh13mX8OXWCKWAm3VTodSf/umf6v3vf7/e85736Dd+4zeUyWTqbuM4jr7yla/oP/yH/7DrRaI9MVMKALrHZNIdSk0xUwpAC1i+8Mcq5y9tHDi2Vi58XlfOf0x2heAcuyeavENbL60rhSsqF5f8KwhoIzds9vDZz35WP/mTP1n7RuGZZ57R5z73OX30ox9VMpnUX//1X+vpp5/WF77wBS0tbbzwfuZnfmZvq0bLq9iOjvRElc4UtFAoK2AY2u9Z2gEA6ByTno0sLuWKKlZthS0mZQPwx0YfKatuPL/6ouZOzWvw0P0Kxfb7UBk6jRmIKJyYUDEzVRvLr55WcN/r/SsKaBM3DKU+9KEPaXBwUL/yK7+iiYkJfeELX9D/+//+v3r88cf1ta99TV/5ylfkOI6Gh4f19re/Xd/93d/dhLLR6gKmofsPjUiS8pWqFgolBUyaSwJAJ3AcR0vFstLZgiq2o9cMpZQIBjQYCepKoSxJsrUxY/aOnu0bDwPAXjIMQ31jb1E4MaHFmT+sNaKWpEppWXOnf0t9B96ixOCraYKO2xZNHfeEUqfUQygF3NANQ6nz58/rkUce0ete9zpJ0jvf+U5ls1k99thjsm1bb3/72/WOd7xDd911154Xi/YUDVj0kwKADnGlUNJvfjOtXGVju+tUKKDXDKUkSZOJaC2UkjaanRNKAfBbrPdlCkVHrvaWurR5wqlqefazKmam1T/xAzItZvXj1sVSx7Ry4fO142JmRnYlLzPAdRBwPTecU5/JZDQyMuIa+57v+R5VKhU9/PDD+sVf/EUCKQAAukRvKKBidbNJ8GqporVSRZJ00LOEbzpDXykArSEQ7tPIsYeUGHxN3bncyvOaO/WESvl5HypDpwiE+xSMDG0ZcZRfe8m3eoB2cVONHrzTWfv7+yVJr371q3e/IgAA0LICpqlRT4/A2asbWxz0NDufyRRUZZcrAC3CMAPqH3+rBg7+AxlmyHWuUlzU/KknlbnyVXbnwy3z7sKXX2UXPuBGbiqU+uIXv6j/9b/+l+bn3d8ehEKhbe4BAAA61ZhnN9WZzEYo1R8OKhHYbCpcsh3N5YoCgFYS7zuhkePvVTAy7Bp3nIqW0n+kxelPy7ErPlWHdlYXSq29JMeu+lQN0B5u2FNKkp5++ml99rOflbQxS+rIkSMyDEMvvPCCDh8+rOHh4Rs8ArrJaqmisGkoEqjf7QQA0P7GE2F96fLmcfrqTCnDMDSZjOr55Uzt3NR6Xgc8IRYA+C0YGdDw8fdoefZ/Krv4t65zjl2SGuzaB9xIKHZAZiAuu5KVJDl2UcXstCLJwz5XBrSuG4ZSX/nKV/TCCy/ohRde0HPPPacXXnihtuPer/zKr+hXfuVX1Nvbq5e97GW688479fKXv1w/8AM/0Iza0aI+P3tFX19c11A0pIl4RPcM92p/jMaRANApJuLuZXoXsgXZjiPTMHQwEXGFUtOZgr6t2QUCwE0wzaAGJn5QkcSkltJPy7HLskK9Gph4G7vx4ZYYhqloz1Fll75WG8utniaUAq7jhqFUIpHQyZMndfLkydpYLpfTN7/5TT3//PN64YUX9Pzzz+vLX/6ynnnmGRmGQSjV5dKZghxJl/MlXc6X9K2DPX6XBADYRX3hgGIBS7nKxpKEku1oPl/S/lhYk56+UtOZvBzH4QIPQMuK93+LQtH9ujL9KQ1MfL/MALM7ceuiqWOuUCq/elrOgbfwOQhs46aW73nFYjG9+tWvdjU6LxaLtaAK3StbrmqxuLkduCnpALOkAKCjGIah8XhYp1ZztbHZbEH7Y2Htj4UVMg2V7I1GwevlqpaKZQ1E6EMJoHUFo0MaOf7wdYODaiUnKxBrYlVoR5Hk4Y3ln87GFzfV0orKhQWFovt8rgxoTTfV6PxmhMNhvepVr9I73/nO3XpItKF01r3990gsrJC1az9mAIAWMZ5wz4hKX212bhmGxhPuWQZTV88BQCu7XiCVW/mmLj7/mLLLzzWxIrQj0wopkjzkGsuvnvKpGqD1kRZgV814LjwmEkx/BoBONB53z4K91uxckiY9gdX0uvsLCwBoJ+Xikhan/1COXdLi1P/QUvqz7M6H64qmjruO86unfaoEaH2EUthVWy9KJGmcHZcAoCONed7fL+dLKlQ3lioc9IRSUxlCKQDtybGrunL+43LsYm0sc+Urmjv9WyoXl3ysDK0s2nPUdVzKXVC1nNnm1kB3I5TCrrEdR7OZomuMmVIA0JmiAUtDW/pEOZIuZDc+A8YTEdcvGFcKZWXKzCoA0IYMU4nBuzd6BG1Rzs9p7sUPKrfyTZ8KQysLhHoUiu53jeXXzvhUDdDaCKWway7nSyradu04FjDVHw76WBEAYC+NJzxL+K4u4Q5bpvZ7NrnwLu8GgHZgGIaSg6/RyLH3KBDqc51z7KKunP+Ylmc/L8eu+lQhWlU0dcx1TF8poDFCKeyaRkv32PoUADqXd4n2XH5ztuxk0rOEj75SANpYKLZfI3e+V9Hel9WdW1/4subPfEiV4krzC0PL8vaVKqydk22Xt7k10L0IpbBrvN+Ce3dmAgB0lkPJmO7Zl9I7Dg/rJ18xqX94eKR2btKzfHuamVIA2pxpRTR48H71jX2vZLgvo0q5i7p06nHlmA2Dq4LRYVnBntqx41RUXD/vY0VAayKUwq5Js/MeAHSVfdGQfnByn1410KOBSMg1O9a7A9+FXEGlqu19CABoK4ZhKDl0UsNHH5IVSrnOOdWCrpz7fS1f+BM5Dsv5up1hGHVL+HLswgfUIZTCrshXqrpcKNWODUljnu3CAQDdoycUcPUVtB1pNstsKQCdIRw/oJHjjyjac6zu3PrlL+nySx+R4zg+VIZW4g2lCqun+bkAPAilsCu8Fxr7oiFFLGubWwMAusFBz4zZKZbwAeggViCqwcP/UL2jb9LGV7KbYqk76a0KRRIHZZibO9VWKxmVchd9rAhoPYRS2BWNmpwDALqbt9n5NM3OAXQYwzDUM3yPho8+WOsfFO19mRJDJ32uDK3AMAOK9BxxjeXXWMIHbEUohV1xoi+ht44P6q6+hFLBAP2kAAB1faVmMgXZLFsA0IHCiXGN3PmIEgN3a2DiB5klhRrvEs88faUAl4DfBaAzDEfDGo5u9pDiogMAusd6uaJ0pqCZTEHpbEHfPzGk0VhYQ5GgYgFLucpGw9+ibWsuX9JojJ6DADqPFYipf+IHtj3v2BWVchcVTkw0sSr4LZo6qo3lnRvXR+X8vCqlFQVCvX6WBbQMZkphT5h8OwQAXeMPpy/rIy9d0l/MLev8er62TM8wjLq+UizhA9Ctli/8iebPPKXVS38ux2E30m5hBWIKx8dcY/nVMz5VA7QeQikAAHBbvH0Et25+4e0rNZUhlALQfXLLLyhz5VlJ0urcn2vh7O+qWs74XBWaJZo67jrOr57yqRKg9RBKAQCA2zLmCaVmtuyydzBR3+yc7bABdBO7WtBi+o9cY4X185p78XEV1qf8KQpNFU25+0oVMlOyq0WfqgFaC6EUAAC4LQfiEddm6IvFcq2P1P5YWEFz8+xauaqVUqXJFQKAf0wroqGD98sMxF3j1UpGl1/6Ha3O/SVhfYcLhAcUCPdvDji2Cmtn/SsIaCGEUrgtpaqtv5pb1kwmr4rN2ngA6EZhy9RINOQau7aEL2AadTOppugrBaDLRHoOa+TORxROTHrOOFq99L+0cPb3VK3kfKkNe88wjLrZUjl24QMkEUrhNl3IFfXZ9BX95jdn9W++ek7//ewlv0sCAPhgLLGDJXz0lQLQhQLBpPbd8S71DH973bnC+lnNvfi4ipm0D5WhGaI9niV8a2doeA+IUAq3aWbLhUXVcRRg1z0A6ErXb3bunSlVEAB0I8Mw1Tv6Bg0d+VGZljuwr5bXNH/mw1qbf4blfB0onJiQaW1+HtrVvIpZQkiAUAq3JZ1xX1iMe74pBwB0B+/7fzpTqF1UTSTcPacuF0q1nlMA0I2iPXdo5M5HFIqPec7YWrn4p7py/vdlV5hV2kkMw1Sk56hrLM8SPoBQCrfOcRyls+5QaiJOKAUA3WgoElLY2vy1Il+1tVgsS5IilqWRWNh1e5bwAeh2gVBKw0cfUHLfPXXn8quntXD+Yz5Uhb3k7StFKAUQSuE2rJQqWi9vftMdNA0Ney46AADdwTQMjcXdnwFbZ9NOemZSTbOEDwBkGJb6DrxJg4f/oWtplwxTvaPf419h2BPRniOSsXkJXikuqly44mNFgP8IpXDLvEv3DsQjsugpBQBdy9tXautsWm+z8ylmSgFATSx1XCPHH1EoNipJ6jvwZoXjB3yuCrvNtCKKeHZgZLYUuh2hFG4ZS/cAAFs16it1jbfZ+YVsQWWbXYcA4JpAuFfDRx9S/8TblBh8rd/lYI9EU8ddx4RS6HaEUrhlMzQ5BwBs4Z0pdSlfrAVPqVBQfaFA7VzVkWazxabWBwCtzjAtJQZeJWOb1QeV4rIyV/6G3fnaWLTH3VeqmE2rWsn5VA3gP0Ip3JKKbetizn0x4b0YAQB0l0QwoL7wRvAUMg1NJqLKbuk9OJl0L+GbXmcJHwDcLMeu6Mr5j2sp/bQWpz8pu0qw344C4V4FI/u2jDgqrL3kWz2A3wI3vglQ71KupOqWb2h6QwH1hPhxAoBu90MHhxUPWNoXDdX1GZxMRPW1xfXaMTvwAcDNW77wJyrlL0mScsvPqZS7pMFD9ysUHfa5MuxUNHVM5cLl2nFu9ZTi/d/iY0WAf5gphVsy47mQYJYUAECS7uiJaX8s3HDji4OevlLTmYJslqAAwA1VSmvKLn3dPVZc1PypJ5VZ/FuW87UZb1+pwtpZOXbFp2oAfxFK4ZbMZOknBQDYmaFISFFr81ePQtXW5XzJx4oAoD0EQj0aOf6wZ9mX5DgVLc18Rkszn5Zd5f20XYRiozIDidqxY5dUyEz7WBHgH0Ip3JK0p8n5BKEUAOAGTGOjz9RWUyzhA4CbEowMavj4jyk+8K1157JL39D86SdVzi/4UBl2yjAMRVNHXWPswoduRSiFHcuUK1opbU4vtQxDo7GwjxUBANpF3RK+9cI2twQAeJlmUAMTP6j+iftkmEHXuXJhQXOnn1B26Rs+VYediKbcu/DlV0+zDBNdic7U2LFEMKCf/9bDms0WNJMpqFC1FTDJNwEA9RzHUbFqKxKwJKluphTNzgFg5xIDr1Qotl9Xpj6uSuFKbdyxy1qc/pQKmWn1jX2vTE9whdYRSR6WYQTkOBtf9lfLqyrn5xWKjfhcGdBchFK4JdGApaOpuI6m4n6XAgBoMWulip5dWNVstqB0tqCRaFgP3zkmSToQDytgGKpc/TZ4pVTRSrGs3jAXTgCwE6HoPo0ce1hL6c8qt+yeHZVd/FuVshc1eOh+BSMDPlWI6zHNoCLJw8qvbS7by6+dJpRC12F6CwAA2FVl29afXVzSqdWcchVbF7LF2i57AdPUWNy95Hs6wxI+ALgVphXSwOR96h//ARmGe75BuTCv1bm/8Kky3IxGS/iAbkMoBQAAdlV/OKhYYPNXjKLt3mVvMkmzcwDYLYZhKDF4t4aP/5gC4f7auBXqVf/Y9/lYGW7E2+y8lLuoSnndp2oAfxBKAQCAXWUYhsbi7obms9nN2VAHvX2l1gmlAOB2haLDGjn+XsV6T0iGpaFD98sMsEN2K7OCSYVio66xwuoZn6oB/EEoBQAAdt24J5RKbwmlJhIRGVvOzedLyleqTaoMADqXaYU1cPCHNXL8vXVhB1pTNHXcdZxbPeVTJYA/CKWwI38yu6hPnJ/Xswurmstt9ggBAGCr8YQnlNrSNyoasDQcDdWOHUkz9JUCgF1hGIZC0X3bns+tfFPLs5+XY/NlQCvw9pUqrp+XbZd9qgZoPkIp7MjfLa/rb66s6ZNTl/XY8zOaYskFAKAB70yp+XxJxapdO55M0FcKAJqtXFzS4vQfan3hy5o/85QqpRW/S+p6wcg+WaFU7dhxKiqsn/OxIqC5CKVw03KVqq4UNlN7Q6rrGQIAgLQxG2owEqwdO5IubFnCN5l0f37QVwoA9pZjV3Tl/Mfl2EVJUil3QXMvPs6Obz4zDEPRHnbhQ/cilMJNS3uWVozEwgpZ/AgBABq7Xl8pb7Pz2WxRFdsWAGBvVMprsqvuLwDsakEL5z6q5Qt/IsdhOZ9fvEv48qun5dAmBV2CRAE3bSbrDqW8FxsAAGx1vb5SveGgUqFA7bjiOLqQLTatNgDoNsFwv0aOP6JIz9G6c+uXv6TLZ35bldKaD5Uhkjgow9zstWhXsirlLvhYEdA8hFK4ad6ZUhMJQikAwPYazZTa+s3vpOdzZJq+UgCwp6xAVEOH/5F6R98oufZBlYrZtOZOPa782kv+FNfFDNNStOcO1xhL+NAtCKVwU2zHcS27kJgpBQC4vpFoWAFj86JnvVzVaqlSOz6Y9DY7Zwc+ANhrhmGoZ/heDR99QFYw6TpnV3JaOPt7Wrn4v+Q4LKlupkZL+IBuQCiFm7JQcO+aFLVMVwNbAAC8LNPQaDzsGtv6BYd3B77p9bxsemgAQFOEExMby/mSR+rOrc3/pc585XHZ1ZIPlXWnjWWVm1/klAuXVSku+1cQ0CSEUrgp3qV744mIDMPY5tYAAGyY8Myqnd0SSg1HQ4ps2TAjX7W1UOACCACaxQrGNXTkR5Xa//fkXc63vnxWKxf/xJ/CupAViCqcGHeNMVsK3aBlQql0Oq33ve99OnnypE6ePKlHH31US0tLO3qMF198UXfddZd+/dd/fY+q7F4s3QMA3IqxRESWIY3Fw3r9vpTu6InVzpmGUdefcHqdJXwA0EyGYSg18h3ad8e7ZAYSrnOZK19VKT/vU2XdJ9pz3HWcI5RCFwjc+CZ7b3l5WQ888IBKpZIefvhhVatVPfnkkzp16pQ+9rGPKRQK3fAxKpWKfvZnf1blcrkJFXefGZqcAwBuwct64/qFu48oaDb+HuxgIqrTq7na8XQmr5P7Us0qDwBwVSR5UPvvfETzpz+kSmlj2Vgg3Ce7ys6ozRJNHXPNTitmpmVXCzItrr3QuVoilHrqqac0Nzenz3zmMzpyZGNN8ytf+Uo99NBD+tSnPqV3vOMdN3yM//bf/pvOnDmz16V2pUK1qst593KKMWZKAQBuwnZh1DWTdc3O2YEPAPxiBRPqHXuzFqc+qdEjb5IZe6UMsyUuGbtCMDKgQHhAleLi1RFb+bWzived8LUuYC+1xPK9p59+WidPnqwFUpJ077336tChQ3r66adveP9Tp07pAx/4gH78x398L8vsWrPZora2nR2KhBQNWL7VAwDoHGPxsKwtPQqXixXXDn0AgOaK9hzT6Il/oZFD300g5YP6XfhO+VQJ0By+h1Krq6tKp9M6caI+/T1x4oSee+65697/2rK9e++9V29729v2qsyu5m1yztI9AMBuCZqmDnh26JtmthQA+MYwDFmB6I1viD0RTbn7SuXXXpLjVH2qBth7vodS8/MbjfOGh4frzg0NDSmTyWh9fX3b+3/wgx/U9PS0fvEXf3HPaux2liGlQpvfkhBKAQB208GE++Jnep1QCgDQncLxMZnW5ueiUy2omEn7WBGwt3yfj5nNZiVJ0Wh9Gh8Ob3xzmsvllEwm686fOXNG/+W//Bf9wi/8gkZGRjQ7O7u3xXap79zfr+/c36/VUkXpTEHjifCN7wQAQANrpYpmswUd6YkpbG18NzaZjEhzm7eZzrADHwC0IrtalGlxLbCXDMNUNHVU2aVv1Mbyq6cUSR70ryhgD/keStm2fcPbmA2apFarVf3sz/6sXv3qV99UI/SbMTCQuPGNutiQpDv8LqJLDA3Vh7AAmovX4e76+IuzevbispYKG7vk/t+vvUMvv/rfOJqK6nfOXKrd9lKuqERvTNEg/Qu7Ga9BwH/XXoe2XdHl6b/UpfNf0LFXP6J4atznyjpbwH6lzm0JpUqZlzQ4+A9kbOnBiO7QDZ+FvodS8XhcklQs1m81em3s2m22evLJJ/Xiiy/q937v97S0tCRJWltbkyTl83ktLS2pt7e3YaC1ncXFjGzbufENgT00NJTUwsL2S1YB7D1eh7tvca1QC6Qk6e8uLmlIm79c74uEdLmwsdOrI+lvpxd0NFX/+Y/uwGsQ8N+112Fh/ZyWZp5WpbQsSTr/3Ce17+iDBCR7yHZGJcOSrvaSKuYXdWn2vIKRIZ8rQzN1ymehaRrXnQDke0+p0dFRSdLCwkLducuXL6unp0exWKzu3F/+5V+qXC7r7W9/u+655x7dc889+qEf+iFJG4HVPffco4sXL+5t8QAA4KaMe/oRejfRmEy6z0+ts4QPAFqB4zi1QEqSitm0civP+1hR5zOtsCKJg66x/Oppf4oB9pjvM6V6eno0Njam55+vf2N74YUXdNdddzW830//9E/XZkZdc+XKFf3UT/2U7rvvPv39v//3NTREkgwAQCsYj3tCqWxRjuPUvmk/mIjq2YXNz/UpduADgJYQ7TmiaM8x5dc2Q5GVC3+qaOq4TDPoY2WdLZo6psL62dpxfvW0eoa/zceKgL3heyglSW9+85v127/92zp79qyOHDkiSXrmmWd0/vx5/diP/VjD+zQKq641Oh8fH9e99967dwV3iarjyHYcBXewBBIAgEb2RUMKmYZKV5fJ5ypVLRXLGoiEJEmTSfeGJ7PZgiq2o4DJ8hAA8FvvgTcpv/6S5Gz0A66W17Q+/4xS+7/L58o6VzR1TMuzn6sdF7NpVctZWUGWtqOztETa8N73vlepVEoPPvigPvShD+k3f/M39f73v18nTpzQfffdJ0lKp9P69Kc/rXSa7TCbJZ0p6Be/elb/9YUZfWZ6QS+uZP0uCQDQpkzD0FjdbKnNJXp9oYB6tjQ2L9uOLuXq+00CAJovGBlQcuh1rrG1+f+tSmnVp4o6XyCUUjA64hrLr53xqRpg77REKNXf36+PfOQjuvPOO/XYY4/pwx/+sN74xjfqiSeeUCi08Q3qs88+q0cffVTPPvusz9V2j3SmoKojzWaL+tLlFX19sf2brAEA/FO3hC+zGToZhqHJhHu2FEv4AKB1pEa+Q2Zgs9ev41S0cvHPfKyo80VTx1zH9JVCJ2qJ5XuSdPjwYX3wgx/c9vwP//AP64d/+Iev+xhjY2M6derUbpfWtWay7iaz3ia1AADsRF2z86w7dJpMRvV3y5na8fR6Xt8x0teU2gAA12daEfXuf4OW0n9UG8stP6fi4GsUTkz4WFnniqaOaW3uL2rHhfWzcuyKDLNlLuOB29YSM6XQmrw7I03ECaUAALfOu3zvUq6kim3Xjg96QqupTEGO4zSlNgDAjcUHXlW3pGx59vO8V++RUHS/rGCyduzYZRUyU/4VBOwBQik0tFoqa61cqR0HDEMjsbCPFQEA2l1PKKDe0Oa3u1XH0aVcqXY8HAsrvGVzjVylqiuFclNrBABszzBM9Y29xTVWyl9SdunrPlXU2QzDULTnqGuMJXzoNIRSaGjGM0vqQDzMDkgAgNt2vWbnlmFowjNbapq+UgDQUiKJScV6X+4aW7n4BdlVNqfYC436SjEzDZ2EUAoNeZfueZvTAgBwK7x9pWYy3r5S3iV8hFIA0Gp6R98ow9ic+WpXMlqb/ysfK+pc4eQhGWawdlwtr6mcn/OxImB3EUqhIZqcAwD2grc/4WzW/c26dwe+6XX35xEAwH+BcK+S++5xjWUWvybbZsn1bjPNoCLJw64xlvChkxBKoU7FdnTRc5HgXU4BAMCtGI2HtXU1+FKxrMyWHobj8Yjr/GKxrPUt5wEAraFn+NuuNuE2lBh8jfa/7J/J3DKjB7vHu4QvRyiFDsJekqgzlyuqsmWdck8woFSIDxgAwO0LmqZO9CYUskyNxSOaSEQUC1i18yHL1IFYxNVrano9r7v6k40eDgDgE9MKaWDiPpnBuELRYb/L6WjeZufl/CVVSmsKhHp8qgjYPYRSqONduscsKQDAbvqRO/Zf9/xkwhNKZQqEUgDQgiI9h298I9w2K5hQKHZApdyF2lh+7bSSg6/xsSpgd7B8D3XqmpwTSgEAmuhg0t1XamqdZucAgO4WTR13HdNXCp2CUAp10t6ZUuy8BwBoIu8M3Uu5oopV26dqAADwn7evVGH9vOxqyadqgN1DKAWXTLmipeLmrhmmsdGUFgCAZkkEAxqMbPYytFX/hQkAoDU5jqPc8gtanfsrv0vpKMHIkKxQ7+aAU1Vh/Zxv9QC7hVAKdd54oF/HUzFFLVP7o2EFTX5MAADNdTDhXsI3zRI+AGh5pdwlXT7zYV2Z+rhWL31R5cKC3yV1DMMwFKtbwnfKp2qA3UOjc7gkggG9YXRA0sa3HHmWSwAA9ki+UtVstqB0tqBYwNLr9/XWzk0mo/rKlbXa8XSGUAoAWpnj2Lpy/uOqlJavjthanv1j7bvjnb7W1UmiqWNaX/hy7Ti/dkaOY8swmESA9kUohW0ZhuHaphsAgN1ydi2nJ09t7iK0PxZ2hVIHPX2lZjIFVR1HlmE0q0QAwA4Yhqne0e/RlamP18YK62eVXz2jaOqoj5V1jnBiQoYVllMtSpLsSk6l7AWFE+M+VwbcOiJVAADQdPtj7n6F87miSltm5/aHg0ps+WKkZDu6lCs2rT4AwM5Fe1+mcGLSNbZ84Y/l2FWfKuoshmEpmrzDNcYSPrQ7QikAANB0sYClgbC7mfmFLaGTYRiaTNJXCgDaiWEY6jvwFtdYpbio9SvP+lRR54l6+0qtnfGpEmB3EEoBAABfjMfdS/TSGfcOe94lfFMZduADgFYXio0oMXC3a2z1/8/efYc5VaZ/A/+e9GSSTDK9wlClCaI0AQHpKMqK3bV3xXVV1ra6rq4/LC9rxV5WFF1WQWmKgKKASAdBQIbO9ExNMsmkl/eP0QxnCsww5Uxmvp/r8vI692k3GUIy93me+7GsR9BfJVFGHYvW2ANAzVR2v6cUfm+FdAkRNROLUhRxrNKFIpcXoXBY6lSIiKgTyKxVdMqrEhed6hspFeZnFBFRuxebeiEEWc007XDQC3vROukS6kBkCm2dKZJu+yGJsiFqPhalKGJZTgnm7c/Fv3YdxQfZ+ajw+qVOiYiIOrDaI6Xya42EStWpoZLVPA12BoL8bCIiigJyZQxiU8aIYs7yXfC5LBJl1LFoY3uLtlmUomjGohQBqF6Wu9RT/UXfFwrjuMONGK68R0RErShFp4bipNX07P4A7L5AZFsuCHVGU3EKHxFRdDAkDoNCHXdSJAxrwWqOeG0BtYtSXmcOQgH2XaToxKIUAag7ZSJZq4Jazr8eRETUehQyoc4qfLX7SnXVs9k5EVE0EmRymNMni2JeZw7c9myJMuo4lOo4KDWJJ0XCcFcekSwfouZg1YEA1P0loPaTaSIiotbQpdbnTX5V7Wbn4qLUCSeLUkRE0UJj7AWNoYcoZi34DuFQoIEzqLE4hY86ChalCEDdkVJdYliUIiKi1pdR6/Mmt6ruQ5KTv6yUefxw+vnLDBFRNBAE4ffRUjVTtYM+GypLNkuXVAehNdYqSjmOIBwKSpQN0ZljUYoQCofrGSmlbeBoIiKillN7ZG5BlQfBk/qNqOWyOlP8ctlXiogoaii1idAnDo1saww9oDP1kTCjjkEVkw6ZQhfZDge98FblSJgR0ZlhUYpQ5vHDHQxFtjVyGRI0SgkzIiKizsKsUogW1vCHwihx+0THdDXUmsLHvlJERFElNmUsVDEZSOx+DRJ7XFerHxKdCUGQQWvsJYq57YclyobozLEoRXWm7mXGaCA7aTUkIiKi1iLUs8Je7dG7WbX253CkFBFRVJErtEjpfSu0sb0h8PeMFqONPUu07bIf5OqGFHVYlCLk1WoayybnRETUljJP6islFwBHrZ5RtUdKFbg88J00wpeIiKgz0hi6A0LNaOOgzwa/p1TCjIiaTiF1AiS9Ov2k2OSciIjaUB9TDJQyAV30GqTq1FDKxM/MDEoF4tVKlHv9AIBQuHqVvu5GXX2XIyIi6hRkchU0hm7wVB6JxNz2Q1BpkyTMiqhpOFKqk/MGQ7DU6t3BkVJERNSWUnVqjE4xo4teW6cg9YeuBvFn0wlO4SMi6hC8VQUIBX2nP5DqpY2ttQqf/aBEmRCdGRalOrmCKg9OnnWcoFFCd1LDWSIiovYgq9aqsDlsdk5EFNUCfgfKc5ai+NCHqCzZJHU6UUtrFBelfK4CBP1OibIhajoWpTq52stqd+HUPSIiaoe61ipK5To9CLGZKxFRVPI4jqPotzdQVfErAMBRvAkBn03apKKUQmWEUpsqirkruQofRQ8WpTq5OivvceoeERG1Q7VH8npDdaefExFRdFDp0iDIVJHtcDgAW8FaCTOKbro6U/gOSZQJUdOxKNXJzeiahD/3TMWYFDOyDFp0qfUkmoiIqD0QBAFZtR6cnOAUPiKiqCSTq2FKGy+KuWz74XHmSJRRdKvdV8rjOIZQyC9RNkRNw6JUJ2dUKdDfrMfUzATc2ScDqTq11CkREVEnlef0YKPFiv8eKcL/23McFV7xF+quBvaVIiLqKGLiBkGpTRHFrPmrEQ6HJMooeim1KZArjZHtcMgPr+O4hBkRNR6LUkRERNQurMovw8q8MuyzOmHzBZBXq+9hnWbnTjfC7CtFRBSVBEEGc8YUUczvtqCqYo9EGUUvQRDqWYWPU/goOrAoRURERO1CZq3FNvJr9T1M1amhlAmR7Up/EFZfoE1yIyKilqfRd4XO1E8UsxX+gFDQK1FG0atOUaryMB/cUFRgUYqIiIjahYxaRanaK8QqZEKdYziFj4goupnSJ0IQFJHtUKAKdssGCTOKThp9FgSZMrId9DvgcxdJmBFR47AoRURERO1Cl1qNzItcXgRC4qe8tafwnXCyKEVEFM0UKhMMyeeLYo7SrfB7yiXKKDoJMgU0hh6imNt+UKJsiBqPRalOqsjlxfKcEuwur0SFx8+hnUREJDmjSoFYZc3T8kA4DItLPIWjq6H2SCnxaCoiIoo+xqRRkCsNNYFwCLbC76RLKEppY88SbbvthyXKhKjxWJTqpI5WurClxI4vjhXj33tPYMmJEqlTIiIiQkat0VJ5tfpKddFrIJy0XeLxwRUItkFmRETUWmRyFUxpE0Uxt/0QPJXHJMooOmmNPUXbfrcFAa9NmmSIGolFqU6qdp+OZK1KokyIiIhqdKnVM6r2CnwauRwpOrUolsMpfEREUU9nHgBVTIYoZi1YjXA4JFFG0UeujIE6JlMUs1vWS5QNUeOwKNVJ1X3yrG3gSCIiorZzupFSANBVzyl8REQdjSAIMKdPqRMP+islyCZ6xcQNFG1XVeyBt6pAomyITo9FqU7I7gvAftIS2nJBQKqOI6WIiEh66Tq16MtJuddfZ3oem50TEXVM6ph0xMQNgkyuhTljGlL63AWFyiR1WlElJn4wlJpEUcyav4o9hKndYlGqE6o9FSJNp4ZCxr8KREQkPZVchuRa0/Nqf251NYiLUgVVHvhDnN5BRNQRmNImIrXffTAkDoUg8HeUphIEWZ0RZz5XAVzWfRJlRHRqfJd3QvU1jSUiImovMmv3lar1uRWrUsCsrlmlLxgG8qvEq/QREVF0kitjIFewtUhzaIzd66zEZyv8HqGgT6KMiBrGolQnlFtrmkMmi1JERNSO1P5cqj1SCqg7hS/HwSl8REREfzClTwIEeWQ76HegsuRnCTMiqh+LUp1MMBRGQa2nybVXOiIiIpJS7ZFS+VWeOr0wutYuSrGvFBFRh8eV+BpPqY6DMXG4KOYo3oyA1yZNQkQNYFGqk7G4vQic9MXeoJQjVqU4xRlERERtK0GjhEYugwAgSaNCP7MevlCtopSh1gp8Tg9CbOJKRNQhhUJ+2IrWwXLwA4RDwdOfQAAAY8oFkCn0ke1wOABb4fcSZkRUF6sRnUxurSkQmTEaCIIgUTZERER1yQQBt52Vjni1EhqFvN5jEjUqaOUyuIPVT809wRBK3D6k1GqSTkRE0a3Kuh+2gu8Q9FcCABxl22BMOl/irKKDTK6GKW08KnKXR2Iu22/wOE5AY8iSLjGik3CkVCfDJudERBQN0mM0DRakgOrCVe1V+E5wCh8RUYfjdZyIFKQAwF60AUF/lYQZRZeYuEFQ6dJEMWvBak6FpHaDRalOpnaz2Ew9V7YgIqLolFXrwUqOo25DdCIiim6xqeMgyGtGwYZDXtiLfpQwo+giCALM6VNEMb+7GFXlv0iUEZEYi1KdSJU/iHKvP7ItA5DOaQ5ERBSlajc750gpIqKOR66MQWzKWFHMWb4LPpdFooyij1qfCZ15gChmK/oRoQAf5pD0WJTqRCq8fugUNT/yFJ0aKjn/ChARUXRKj1FDcVJfRLsvANtJD1+IiKhjMCQMhUIdL4pVT0HjAheNZUqbCEGmjGyHAi7YLeslzIioGisSnUimXoMnzumO2Wd3xZXdkjEmxSx1SkRERI3iDgTrFJwUMhkyYsQjfjlaioio4xFkcpjTJ4tiXmcO3PZsiTKKPgqVEcbkUaKYo3Q7/J4yiTIiqsaiVCcjCALiNSoMTjBiYLxB6nSIiIgaZHF5sfi4Ba/sPYFnfzmGb/PqfnGu3eycfaWIiDombWwvaAw9RDFrwXcIhwISZRR9DEnnQ66MPSkSgrVgjWT5EAEsShEREVE75QuFsKvMgVJP9Qip2ivIAkBWrb5SORwpRUTUYZkzJuPkX2GDPhsqSzZLl1CUkcmUMKdPEsU8lUfgth+WKCMiFqWIiIionUrVqSGvaRkFmy8Ah1/8RLyLXoOTDkGx2wd3INg2CRIRUZtSahJhSBwqilUWb0TA75Aoo+ijNfWFWt9FFLMWrEE4xM9OkgaLUkRERNQuKWUypNZaJTbPKR4tpVXIkaxVRbbDAHKdnMJHRNRRxaaMgUxeM0o2HPLDXrhWwoyiiyAIMKdPFcUC3nI4yrZLlBF1dixKdRJcmYKIiKJRZoxGtF3fFL6utabwsdk5EVHHJVNoEZt6oShWVfErvFX5EmUUfVS6FOjjzxXF7Jb1CPqrJMqIOjMWpTqJdw7k4/3sfKzOK8NvVif8oZDUKREREZ1Wpr5WUaqeUVBdDeJjchwsShERdWT6hHOh1CSJYnbLTxJlE51iUy+EIKsZjRwOemEv+lHCjKizYlGqE/AEgsiv8uC4w431Fis+O1KEYIgjp4iIqP2rPVIqv8qDUK3Rv7WbnedXeRHgwxciog5LEGS/Nz0HABkMSSOQkHWZpDlFG7kyBrGpY0QxZ/ku+FwWiTKizopFqU4gr8qDk7++J2pV0CjkkuVDRETUWHFqJXQnfWb5QmGUuH2iY0xqJWJVish2IBxGQZW3zXIkIqK2pzF0R2zqeKT2vRvm9MmQyTWnP4lEDAnDoFDHi2LWglVs/UJtikWpTqB2/40uMfwHm4iIooMgCMiMqdXsvJ6+UrVHS+WwrxQRUYcXmzIaSk2C1GlELUEmhzl9sijmdebCbTsgUUbUGbEo1QnU7r9Ruz8HERFRe3YmfaVOcAU+IiKi09LG9oLG2FMUsxZ8h1DIL1FG1NmwKNXBhcPhOktj1+7PQURE1J7V11eqtjojpRzuOr2niIiIqK7q0VI1pYGg3w5HyWbpEqJOhUWpDq7c64c7WNPsVS2XIUmrkjAjIiKipsmoVZQqdvvgDYobmSdpVdDIa77WuIMhlHrEvaeIiKjjC4fDcJbvgdeZJ3UqUUOpSYAhcagoVln8MwK+Sokyos6ERakOrs7UvRg1ZIIgUTZERERNp1XIkahRRrbDqDtaSiYI6Fprmt/uckdbpEdERO2EtyoPxYc+REXuMlTkf4twmCuxNlZsyljIFLrIdjjkh61wrYQZUWfBolQHV3fqnraBI4mIiNqvxkzh62nUibZ/tthg9bInBhFRZ+D3lKP40EfwuQqrt90WVJXvljapKCJTaGBKvVAUc1n3csQZtToWpTq42isUsck5ERFFowy9BmaVAmfH6XFRZgL6mGLqHDMkMRYGpTyyHQiHsTq/rC3TJCIiiSg18dCZ+otitqIfEQpy4YvGiokfDKU2WRSzFqxGmD0aqRWxKNWB+YIhWFxeUYxNzomIKBoNS4zFw4O64doeqRidYkayVl3nGLVchsnp8aLYrxVO5DrdbZUmERFJyJQ+EYKgiGyHAlWwW36SMKPoIggymNOniGI+VyGqKvZIlBF1BixKdWAFLi9OnkUdr1Yi5qQnyERERNGisf0QBycYkaoTF6y+yS3jU14iok5AoYqFIXmkKOYo3Qq/p1yijKKPxpAFramvKGYrXItQ0NvAGUTNw6JUB5ZX68kwp+4REVFHJxMEXJyZIIrlVXnwa4VTooyIiKgtGZNGQq401gTCIdgKvpMuoShkTpsECDWDGUKBKlRyxBm1EhalOrC6Tc5ZlCIioo6vu1GHfrV6Tq3OL4M/xFWYiIg6OplcBVPaBFHMXXkI7sqjEmUUfRRqE4xJ4hFnlaVb4fdWSJQRdWQsSnVgXfQadDNooZQJkW0iIqLOYGpmAuQnzfiz+QL42WKTLB8iImo7OvMAqGIyRDFbwRqEw3w40VjG5FGQKw01gXCQI86oVbAo1YGNSY3DHX0y8NS5PXBf/y5IqacpLBERUbTxh0LIcbix0WJFMFR/r6gEjQojkkyi2LqiCjj8gTbIkIiIpCQIQp2G3X5PKZxlOyTKKPpUjzibKIq57QfhqTwmUUbUUbEo1QnIBQFpOjXkssY1iSUiImqvPjlUiGd2HcW72flYmVcGi7vhxqvj0+Kgldd81fGFwvi+gM1uiYg6A3VMOmLiBoli9qJ1CAZcEmUUfeobcWYtWM0RZ9SiWJQiIiKiqCETgJMHR+VVeRo8VquQY0J6vCi2o7QSRS6uIERE1BmY0sZDkKki26GgB/ai9RJmFF0aHnG2U6KMqCNiUYqIiIiiRu2VZPOcDRelAGB4YiwSNMrIdhjAyrxShMP1T/sjIqKOQ640wJg8WhRzlu2Az10iUUbRp/4RZz9yxBm1GBaliIiIKGrUXkn2VCOlAEAuE3BRZoIodrTSjYP2qhbPjYiI2h9j0gjIVabItjb2LMjkqoZPoDo44oxaE4tSHZDTH0C5x8enwERE1OGkx2hwcofEMo8f7kDwlOecFRuDHkatKLYyr6zBJulERNRxCDIFzOmTodQkIannDUjsfhUUJxWp6PQ44oxaE4tSHdCecgde2puDObuP45NDhdhvdUqdEhERUYtQy2VI1oqfcJ9utJQgCLgoM7FOMWtrqb0VMiQiovZGG3sWUvrcCY2hm9SpRC1j0ggoVOaTImFY81dzIAQ1G4tSHVDu71/OXYEgsu1VsHr9EmdERETUcpraVwoAUnVqDEk0imJrC8pPO8qKiIiinyAIEAT+6tscgkwBU/okUczrPA63/ZBEGVFHwXdmB1T7y3mXWl/eiYiIollT+0r9YWJ6PFSymvFS7mAIPxRWtGhuREREHZU29iyo9eLRZraCNQiHAhJlRB0Bi1IdTKUvAJuv5h8FuVD9dJiIiKijqD1SKr/K06jpAwalAuNS40SxLSU2lHl8LZofERFFD7+XDycaSxAEmDOmACdNiA/4rHCUbpUuKYp6LEp1MPm1nhan6tRQyvhjJiKijiNRo4L6pM82VyCEikZOVR+VYoJJpYhsB8PAqryyFs+RiIjat4DXhrLji1H025vwuYqkTidqqLRJ0CcMEcXslp8Q9LOPMZ0ZVis6mNxaU/cyY7QNHElERBSdZIKA9BjxKODan38NUcpkmJKRIIr9ZqvC0UpXi+VHRETtm6N0OwoPvAmX7TcAYVgL2LC7KWJTx0Imrxm1HA75YCv8QcKMKJqxKNXB1O6rwX5SRETUEdX+fGtsXykAGBinr9OXamVeGUL8hYSIqFNQqExAuGahC68zF27bAekSijJyhQ6xqeNEsaqK3fBWFUiTEEU1FqU6kGA4XGf6HotSRETUEdUuKtX+/DsVQRBwcRfxaKkilxe/lFW2SG5ERNS+aWN7QWPsKYpZC79DKMRVyxtLn3AelJpEUYwjzuhMsCjVgRS7ffCHav4R0Cvkor4ZREREHUVGrYcuFpcPgVDjvwh30WsxME4viq0pKIc3GGqR/IiIqH0zp0/Gyb8OB312OEo2S5dQlBEEOUzpk0UxX1U+XNZ9EmVE0YpFqQ4kz+kWbWfqNRAEoYGjiYiIopdBqcB5CUZMSo/Hrb3T8fdzukEha9pn3pSMBChO+px0+IPYYLG2dKpERNQOKTUJMCQOFcUqi39GwMdRs42lNfaANra3KGYr/B6hIFe1pcZjUaoDyXNy6h4REXUel3dLxoVpcegZq4NGIW/y+Wa1EqNSTKLYRosVtkau5EdERNEtNmUMZApdZDsc8rNhdxOZ0icDwkkjzvwOVJb8LGFGFG1YlOpAcqtqr7zHohQREdGpjEuNg/6kgpY/FMaagnIJMyIiorYiU2jrNOx2WX+FtypfmoSikFIdB0PiCFHMUbwZAa9NmoQo6rAo1UG4AkGUeWqe7AoA0lmUIiIiOiW1XIZJGfGi2O5yR53Rx0RE1DHp48+FUpMsilnz2bC7KWJTLoBMERPZDocDsBV+L2FGFE1YlOogan95TtGpoZbzx0tERHQ65yUYkaJViWIr80r5CwkRUScgCDKYM2o17HYVwGXdK1FG0UcmV8OUNkEUc9l+g8eZI1FGFE1Ytegg0mPUuKp7MkYkxSJdp0YW+0kRERE1ikwQcFGmeFnrHKcH+6xOiTIiIqK2pDF0gza2jyhmK1zLht1NEBM3CCptqihWPeKMq9rSqbEo1UHolQqcE2/EpV2TMKt/F0zvknj6k4iIiKJcMBxGocuLrSV2LD5uweZi2xldp2esDn1iY0SxVfll8If4ZZqIqDMwp08ChJoeg0G/A5XFGyXMKLoIggBzxlRRzO+2oKp8tzQJUdRgUaqDEoSmLYtNREQUjX4td+CN/blYllOCXWUO/GY789FN0zITIDvp49PqDZxxkYuIiKKLQm2GMUncsNvjPMGp3E2g1mdCZx4gitmKfkAowD6N1DAWpYiIiChqZdaarp5f5UXoDH+BSNSqMDwxVhT7scgKpz9wxvkREVH0MCaPhkyhh0wRg7gulyC518182N9EprQJEARFZDsUcMFu2SBhRtTesShFREREUSterYT2pIU9vMEQSj1n3gNkQno8NLWu931BRbNyJCKi6CCTq5HY/Wqk9bsP+vjBEAT+utxUClUsjMmjRDFH6Tb4PWUSZUTtHd9lREREFLUEQagzWqr2irRNoVPIMT4tThTbXmpHsdt7xtckIqLooY5Jh0yuljqNqGZIHgm58uSRxyFYC9ZIlg+1byxKERERUVTLiKlVlKpqXu+KEUkmxKuVke0wgJW5fMJLRETUGDKZEqb0iaKYp/II3PbDEmVE7RmLUkRERBTVutTuK9WMkVIAoJAJmJaZIIodrnThoK2qWdclIiLqLHSmflDHdBHFrAVrEA4FJcqI2isWpYiIiCiq1R4pZXH74AuGmnXNvqYYdDNoRbGVeWUIchUmIqJOJxT0wlrwPWyFP0idStQQBAHmjCmiWMBbDkfZdokyovaKRSkiIiKKajqFvM50u/xmTuETBAEXZybg5DWXSj0+bC+1N+u6REQUPcLhMJzlv6DwtzfgKNmEypJN8HvKpU4raqh0qYiJP1cUs1vWI+jnyGOqwaIUERERRb3azc6bW5QCgLQYDc5NMIpi3xdUwB3g1AMios4gHA7AXrQeocDvRZQwG3Y3lSn1Qgiymsbx4aAX9qJ10iVE7Q6LUkRERBT1Mlu42fkfJqXHQyWrGS/lCgSxrqiiRa5NRETtW/0Nuw/DXXlEooyij1wZg9iUMaKYs3wnfC6LRBlRe8OiFBEREUW92iOl8pzeFrmuUaXAmFSzKLap2I5yj69Frk9ERO2bztQf6phMUcxasAbhMEfNNpYhcRgU6jhRzFqwGmH2aSSwKEVEREQdQIpWDYVQM6Kp0h+A3edvkWuPTjYjVqmIbAfDYazKZ08RIqLOoN6G3Z4yOMt2SpRR9BFkcpjTJ4tiXmcO3LYDEmVE7QmLUkRERBT1FDIBaTq1KHbI7mqRa6vkMkzOiBfF9ludOO5wt8j1iYiofVPp0hATd44oZi9ah2CgZT5nOgONsRc0hh6imLXwO4RCLfMAiaIXi1JERETUIXQ1VE/hEwCMSTFjcLyhxa49KN6AjBhx0WtlbilCnHpARNQpmNLGQ5CpItuhoAf2ovUSZhRdqkecTcbJJYigzw5HyWbpkqJ2gUUpIiIi6hCGJcYiSaPCHX0yMDUzAQpZy33NkQkCLspMFMUKXF7sLne02D2IiKj9kiv1iE25QBRzlu2Az10iUUbRR6lJhCFxqChWWfwzAr5KiTKi9oBFKSIiIuoQ4jUq3D+gC7IM2la5fpZBiwFmvSi2Jr8cvmCoVe5HRETtiyFxOBSqkxe/CMOaz4bdTRGbMgYyhS6yHQ75YStcK2FGJDUWpYiIiKjDkJ3U7Lw1TM1MgLxWQ/WfLNZWvScREbUPgkwBU/okUczrPA63/ZBEGUUfmUKL2NQLRTGXdS+8VXkSZURSY1GKiIiIqJHi1EqMSjaJYhssVth9AWkSIiKiNqWNPQtqfZYoZitYg3CInwONpY8fDKUmWRTjiLPOi0UpIiIi6tBC4TB+KrLih8KKFrneuFQzYhTyyLY/FMZ3+WUtcm0iImrfqht2T0H1shrVAj4rHGU7pUsqygiC7PfXsIbPVYiqil8lyoikxKIUERERdVhWrx8fHCzAt/ll+KGgHHlOT7OvqVHIMTE9ThTbVe5AQVXzr01ERO2fSpsMfcJ51RuCHMbk0dDHD5Y2qSijMWRBa+oritkK1yIU9EqUEUmFRSkiIiLqkELhMD46VIATDnf1NoAvjllapDH5kMRYJGlVotg3eWWcekBE1EnEpo5DTNwgpPWdBVPaeMjkqtOfRCLmtImAUDPyOBRworJ4o4QZkRRYlCIiIqIOSSYIuDgzURQr9/rxbQtMtZMLAi7KTBDFTjjc+M1W1exrExFR+ydX6BDfdQYUapPUqUQthdoMY9L5olhlyRb4vS0z3Z6iA4tSRERE1GGdZYrBsMRYUWxriR2H7M0vHvWOjUHvWJ0o9m1eGQKh5o/EIiIi6gyMyaMhVxpqAuEgbAXfSZcQtTkWpYiIiKhDuygzAXFqpSj25fFiuALBZl97WmaC6MtUhdePzSX2Zl+XiIioM5DJVTClTRDF3PaD8FQekygjamssShEREVGHppLLcFX35JPWSQIc/iCW5ZQ0+9rJWjWGJolHYv1YWIEqf/MLXkREFH3C4RCc5bsRCvmlTiVq6MxnQ6VLF8WsBasRDnPkcWfAohQRERF1eF30WoxNNYtieyuc2FPuaPa1J6TFQSOv+UrlCYawtrC82dclIqLo4nEchyX7PVTkLoejZLPU6UQNQRBgzpgqivk9pXCW7ZQoI2pLLEoRERFRpzA+LR5pOrUotiynBHZf855m65UKXJgaJ4ptK7GjxO1r1nWJiCh6OMt3o+TIAvg91aNwK4t/RsBXKXFW0UMdk46YuEGimL1oHYIBt0QZUVtpN0WpvLw83HfffRg2bBiGDRuGRx55BBUVp++6/9NPP+G6667DoEGDMHjwYNx8883YvXt36ydMREREUUUhE3Bl92QohJqJfJ5gCF8eL0EoHG7Wtc9PjhX1rQoB+DavtFnXJCKi6KGL7QOZombxi3DID1vhWgkzij6mtPEQZKrIdijoht2yXsKMqC20i6KU1WrFTTfdhN27d+P222/HLbfcgh9++AG33HILfL6GnzJu27YNd9xxBxwOBx588EHMmjULubm5uP766/Hrr7+24Z+AiIiIokGyVo3JGfGi2JFKF7Y2szm5QibD1FrXPWh34XALrPJHRETtn0yhQWzqhaKYy7oX3qo8iTKKPnKlAcbk0aKYs3Q7fO7m94Ck9qtdFKXmz58Pi8WCjz/+GHfeeSfuuecevP7668jOzsbSpUsbPO+5555DamoqvvjiC9x88824/fbb8cUXX0Cn0+GVV15puz8AERERRY2RySZ0M2hFsVX5ZSht5nS7/mY9svQaUWxlXhmCzRyFRURE0UEfPxhKTbIoZs1fjTA/BxrNmDQCcpXppEgYtoI1fA07sHZRlPrmm28wbNgw9OjRIxIbOXIkunXrhm+++abec+x2O7KzszF16lRotTVfLBMSEjB06FD88ssvrZ43ERERRR+ZIOCKbslQy2q+BillAuz+QLOuKwgCLuqSKIoVu33YWcqeIkREnYEgyGDOmCKK+VyFqKrgLJ7GEmQKmNMni2IexzG4Kw9JlBG1NsmLUna7HXl5eejfv3+dff3798e+ffvqPU+v12PVqlW4+eab6+yzWq2Qy+UtnSoRERF1EGa1EtO7VheQ+sTG4K8DuqKnUXeas04vI0aDwfEGUey7gnJ4gsFmX5uIiNo/jSELWlNfUcxeuBahIBe/aCxt7FlQ67NEMVv+GoRDzXt4RO2T5EWp4uJiAEBycnKdfYmJiXA6nXA46i7XLJfLkZWVVee87Oxs7Nq1C4MHD26dhImIiKhDODfegJt7p+GGXqkwKBUtdt3JGfFQymqaqVcFglhfaG2x6xMRUftmTpsICDWDJIIBJyqLN0qYUXQRBOH3EWc1n6UBnxWO0q3SJUWtpuW+gZ2hqqrqBqAnT8H7g1pdvWyzy+WCwWCos7++az366KMAgDvvvLPJucTH6097jM1mQ2lp2SkbsBM1R0kr9/GTy+UwGg2Ij4+PvMeIqK7ExNN/7lD0S0oytvg1EwFMrfJgxRFLJPZziQ1T+6QhQcd/dxuL70Ei6fF9eKYMCLnHwnL8h0jEUboFXXpdALUuTsK8ookBwaoRKM3bHIlUFm9El14joVS3/Gd3e9UZ3oOSF6VCodBpj5HJTj+gy+1245577kF2djbuuusuDBs2rMm5lJc7EQo13EDN7/fBai2ByZSA2Fg1hJOWlCZqKQqFDIHA6d8XZyIcDiMYDMLjqcKRI8cQF5cMhUJ5+hOJOpnERANKS+uO0iVqrPOMMVivlKPSXz1tLxAK4797cnBtz1SJM4sOfA8SSY/vw+ZRGIZBrtiGYMAJAAiHAji6bykSu10pcWbRQ2UaBVnhLwgFPQCAUNCLo3tXIL7rpRJn1jY6yntQJhNOOQBI8ul7MTExAACv11tn3x+xP45pSGVlJW699VZs3boVl19+OR588MGWTxSAw2GDXh8LlUrDghRFJUEQoFAooNfHQqczoKqKzXeJiOrjCgRRWOU54/NVchkmZySIYnutTuQ43M1NjYiIooBMrkJs2gRRzG07AI/jhDQJRSG5QofY1HGiWFXFbnhdhdIkRK1C8qJUWloaAKC0tLTOvpKSEhiNRuh0DTceLS8vx4033ohdu3bh6quvxpw5c1qtYBQI+KBW151mSBSNNJoYeL385YiIqLaDtiq8ti8HCw4XwR048wbl58QbkFZrut43eaUIcVlrIqJOISZuIFS6NFHMWrAa4XDrzIroiPQJ50GpEa9sa81fhTA/SzsMyYtSRqMRGRkZ2L9/f519v/32GwYMGNDguU6nE7fddhsOHDiAm2++Gf/6179adQRTKBSETMZV/ahjkMvlCIW4GhQR0R/C4TCWnSjBx4cL4fAHYfcHsCK37kOzxpIJAi7uIv4inV/lxa8V0T8Un4iITq+mYXcNpSYR4ZBfooyijyDIYUqfLIr5qvLhstatH1B0krwoBQCTJ0/G5s2bcfTo0Uhs06ZNOH78OC666KIGz/vXv/6FAwcO4MYbb8Tjjz/eFqly2h51GPy7TEQkJggC1HLxV6Pd5Q7sbUYRqZtBi/5mcRuC1fnl8AX5lJyIqDNQx2RCZz4bKl0aknvfgoSsmZDJuehFU2iNPaA19hbFbIXfIxTk4mMdgeSNzgHgjjvuwLJly3DzzTfj1ltvhdfrxQcffID+/ftjxowZAIC8vDzs2rUL5557LjIzM3H06FEsW7YMBoMBffv2xbJly+pc949ziYiIiBpjYnocDtmrYHHXfNFdeqIEXfVaGFVn9rVpakYCsm1VCP4+08DuC+DnYhsuTOMKTEREnUFc5kUQZCo+FG4GU/okuB1HgN+nPgb9lags2QRTrZ5TFH3aRVEqLi4On376KZ5//nm8/vrr0Gg0mDhxIh555BGoVCoAwPbt2/H444/j+eefR2ZmJrZt2wYAcDgcDY6SYlGKiIiImkIhk+HK7il467c8BH/vV+EOhvDViWLc1CvtjH6hiNeocH6SCRuLbZHY+qIKnJdgPONCFxERRQ+OjGo+pSYehsThcJRsjsQcxZugjz8HCpVJusSo2YQwO4RFlJc7EQo1/HJYLDlISenahhm1vTlznsa333592uOmTZuOJ554uln3WrlyBZ577hm8/vo7OPfcIc26Vnu+Z21XXHEJUlJS8cYb79W7X6GQIRBom2kdneHvNNGZ6ChL8NKZ21BUgVX55aLYjK5JGJ4Ue0bXcweCeGnvCbhO+vd9SIIRM7slNyvPjorvQSLp8X1I7U0o6EXhb28gFKiKxHSmfkjodoWEWbWejvIelMkExMfrG9zPx3MkMmPGTAwZMiyyvWfPL1i+fAkuvfQyDBo0OBJPT89o9r0GDRqMf/zjX8jK6tbsaxEREbWk0SlmHLBVIcfpicRW5pWip1GLeI2qydfTKuSYkBYvapy+s6wSI5JNdVboIyKiziMUcEOm4ArvjSGTq2FKG4+K3BWRmMv2GzzOHGj0fNAerViUIpEBAwZiwICBke1gMIjly5dgwICBmDKl4abzZyI9PaNFiltEREQtTSYIuLJbCl7fnwPf76Oo/aEwFh0rxh19MyA/g2l8wxJjsaXEhlJP9apLYQArc0tx21np7DNCRNTJ+D3lsBasQcBrRWrfuyAIXOW9MWLizoGzdAd87qJIzJq/Giln3Q5BaBfruFET8adGREREVI84jRIXd0kUxXKrPPipyHpG15PLBFyUKb7eMYcb2baqBs4gIqKOJhwOw1rwHYqy34an8jAC3jI4SndInVbUEAQB5owpopjfbUFV+W5pEqJmY1GKztiHH76L8eNHYv36H3HppVMwadIYfP31UgDAwYPZeOKJh3HJJZMxduxwTJ8+CU8//QRKSooj569cuQKjRw/Brl07RNuHDx/C008/galTL8SkSWPw+ON/Q1FR4Wnz8Xg8eOedN3DFFZdg3LgRuOKKS/D22/Pg8XhOed66dWtx++03YtKkMZgyZSweeOBe/Prr7jN6TY4ePYKHHroP06dPxIQJo3DrrX/G11/XXRnyZOXlZbjqqhm45JLJyM09cUb3JSKi1jEkwYg+phhRbG1hOQpd3jO6Xu9YHXoZdaLYyrwyBE7R05KIiDoOQRAQCnoiq8gBgN2yHsGAS8Ksoota3wU68wBRzFb0Q/XrSlGH0/eayZV9ACWffgKfpej0B7chVUoqkq6/Ebo+fVv1PoFAAC+++H+47rob4PP5MHDgOTh69Ajuvfc2ZGR0wQ033Ay1WoO9e/dg9eqVKC8vw7x5757ymo899hCysrrjrrtmoaAgH4sWLURZWQnef/+TBs/x+/148MF7sW/fXlx00SXo06cffvttHz777GP8+utuzJv3LhSKun/df/llJ/75z79jxIiRmD59BjweN778chEeeGAWFiz4vEnTC202Gx56aBZiY0248cbboFKp8P33q/HCC89CpVJj8uSpdc5xOBx46KG/wOl0Yt68d9GlS1aj70dERK1PEARclpWE1/blwhUIAgCCYeCLYxbM6pcJpaxpz/cEQcC0zAQc2Z+LP8pQ5V4/tpbYMCrF3MLZExFRe2RKvRAu636EQz4AQDjogb1oHeIyW7ZdSkdmSpsAty0b4XAAABAKuGC3bIA5fbLEmVFTsSjVTMUL5sNfXHz6A9uYz1KE4gXz0W3Oi616n1AohMsvvwrXX39zJPbvfz8PQRAwb947MBqrVymaMWMm/H4/1q5dg8pKeyRenz59+mLOnLmRbY/HjaVLv0ReXi4yM7vUe8433yzD3r2/4v77H8JVV10HALjssivQrVt3vPXW61ixYikuu6zuqgxr134HtVqDF154OdLPY+jQEXjyyUdw6FB2k4pSO3duR3l5OV588RX06dMPAHDxxZfirrtuwbFjR+oc7/V68MgjD6C4uAivvvo2evTo2eh7ERFR2zEoFbgsKwmfHal5AFXi9mFbif2MCkkpOjWGJhqxrbQyEvuhsAKDE4zQKdhThIioo5Mr9YhNGQNb4feRmLNsJ/QJ50Gl5aqsjaFQxcKYPAp2y/pIzFGyDfr4c6HUJEiYGTUVp+9Rsw0ffr5oe/bsx7Bo0QpR4amqygm1unp1IZfLfcrrjR8/SbTds2dvAEBFRXl9hwMANm7cgJiYGMyceZUofuWV1yImJgY//bSu3vOSkpLgclXh1Vfn4sSJ4wCAHj16YuHCr3DhhRNPmWd91wKAt99+A3v2/IJgMAilUon//OdT3H33faJjg8EAnnzyUezduwfPP/8S+rTyiDYiImqe/mY9zo03AABkAjAxPQ4jkk1nfL0J6fFQnzTKyh0M4YfCiuamSUREUcKQOAwK1ckPNsKw5q9GOMzp3I1lSB4JudJ4UiQEa8F3kuVDZ4YjpZop+YabUfLZAvga0fOoLalS05D05xva5F5mc5xoWxAEVFba8emnH+HIkSMoLMyHxVIU+Qc2fNL86fqYTOKnzipV9dLbwWCwwXOKigqRlpZeZ4qeUqlEWlo6LA1Mr7z88quwbdsWfPnlF/jyyy+QmpqOUaNG4+KLZ6BXr96nzLO2s88ehCuuuAZffvk5du7cBqMxFsOGjcDkydMwcuRo0bF79/4K2e+/jOzduweDB5/XpHsREVHbm94lEc5AEBPT45ERo2nWtQxKBcalmbE6v+aBy5YSG4YnxiJRq2puqkRE1M4JMgVMGZNRduzzSMzrPAG3/SB0pj4SZhY9ZDIlTOmTUH7iy0jMU3kYbvthaGN7SZgZNQWLUs2k69MXWc8+J3UakpLV6qexadNGPP74bMTHJ+C884ZixIiR6NOnL7Zt24IFCz467fXOZFnsUz1RCIXCUCqV9e6LidHjjTfew759e/HTT+uwZcsmLF78Ob76ahGefPJf9faBOpUHHvgbrrzyGqxbtxZbtmzCunVr8f33qzFjxkw8/PDfI8cplUo8++yL+N//PsXHH3+ICRMmN2mqIBERtT2NQo6be6e32PVGJpuwrcQOq+/3fhhhYFV+GW7oldZi9yAiovZLa+wNjaEbPI7jkZit4DtojT0hyPiremPoTP3gjNkGb1VeJGYtWAONsTsEgVPiowGn71GLe+WVucjIyMRnny3GE088jWuvvR6DB58Hm83WavdMSUlDYWEBAoGAKO73+1FUVIikpPrnZufm5uDAgf0YMOBs3HPPX/DxxwuxYMEXMBgM+N//Pm1SDhUV5di5czvS0zPw5z/fhHnz3sWyZaswcOA5WL58CZxOZ+TY/v3PxujRY/DQQ48gEAjgpZdeaPofmoiIoppSJsOUTHHfiwO2Khyt5ApMRESdgSAIMKVPAVDzUD7gs8JRulW6pKKMIAgwZ4gHEgS85XCUbpcoI2oqFqWoxVVW2pCcnAqtVhuJFRdbsGHDDwBOPQ3vTI0adQGqqqrw1VdfiOJLliyCy1WFkSMvqPe8V1/9Nx577CG4XDW/AHTtmgW93gC5vGlvj2++WYG//vUeZGf/FonFxpqQkZEJQRDqjCgDgO7de+KKK67Gtm1b8N13q5p0PyIial/OpA/I2WY9uujFUwG/yS1FiD1FiIg6BZU2CfqEIaKY3fITgn5nA2dQbSpdKmLiB4tidst6BP1VEmVETcExgdTiRowYibVrv8Pcuc+hb99+KCgowIoVS+F2ewAALlfL/+NwySV/wqpVX2PevFdw9OgR9OnTD9nZv2HlyhXo128ALrnkT/Wed801f8bf/nY/Zs26HdOmTYdKpcKGDetRUJCPW255pkk5TJs2HZ9//hkeeeRBXHbZFUhISMTBgwewatU3mDZtOnQ6Xb3n3XbbXfj++zWYN+8VjBgxCmZzwysTEhFR+xMMhfFjUQWK3V5c1yO1SdPQBUHAxZmJePtAzbQDi9uHXWWVGJLIzwMios4gNnUsXNa9CAWrf18Kh3ywFf6A+K6XSpxZ9DClXgiX9TeEQ14AQDjohb1oHeK6XCxxZnQ6HClFLW727McxffoMbNy4Hq+8Mhfr1q3F1KkX47XX3gIA7Ny5o8XvqVKp8Nprb+Pqq/+M7du34vXXX8Ivv+zEDTfcgnnz3qnTAP0Pw4aNwAsvvAyNRouPPvoA8+a9AofDjqefnoOpU5v2D1hCQgJef/0dnH32QCxd+iVefvlF7NixHbfeeif+9rfHGzxPp4vBffc9gIqKcrzzzrwm3ZOIiKRV4vbhnQN5+KGwAvutVdhZVtnka2TqNRgUZxDF1uSXwxs89cIgRETUMcgVOsSmjhPFqip2w+tqX4tptWdypR6xKWNEMWf5LvhcFokyosYSwlxzMqK83IlQqOGXw2LJQUpK1zbMiDojhUKGQKBtfhHh32mi+iUmGlBa6pA6DYoCH2Tn45jDHdlWyQTc378r4jT1L7DREJvXj5f35iBw0teycalmTM5IOMVZHRffg0TS4/uwbYXDQViy34PfUxqJ6ROGIi5zmoRZRZdwKIii7LcR8FZEYmp9VyT1vPGMFtOSWkd5D8pkAuLj9Q3vb8NciIiIiDqUy7KSoJTVfNH1hcJYdNzS5J5QJrUSF6SYRbGNFhtsXn+L5ElERO2bIMhhSp8MAJArjYjvOrNOA286NUEmh/n31/APXmcO3PZsiTKixmBPKaJTcDqd8Ho9pz1OJpPDbDaf9jgiIupY4jUqXJSZgGU5NU+2c5webLRYMSY1rknXGpNqxo4yOxz+6gVBAuEwVueX4+oeKS2aMxERtU9aYw/Ed/0TtLF9IJOrpE4nKmmMvaAx9IDHcTQSsxZ8B42xJ2Sypo1iprbBohTRKbz22r/x7bdfn/a4lJRULF68og0yIiKi9mZYYiwO2KpwyF6zkut3BRXoFRuDVJ260ddRy2WYlB6Pr06URGJ7KhwYmWxCZq0V+oiIqGOKiRsodQpRTRAEmDMmo+jAOwCqRy0HfTY4SrYgNqX+FdlJWixKEZ3CddfdiMmTTz+PW61u/C8dRETUsQiCgJlZyXhtXw7cvzcnD4bDWHTMgnv7ZUIha3y3hHMTjNhcYkeRyxuJfZNbirv6ZkRlPwwiIqK2ptQkwpA4DI7SrZFYZfFGxMQNgkJllDAzqg+LUkSn0K1bd3Tr1l3qNIiIqJ0zqhSYkZWE/x2tWeXH4vbh+4IKTM1sfLNymSDgoswEfHiwIBLLrfJgb4UTA+MNpziTiIiI/hCbMgZVFb8iFKxejCQc8sNWuBYJWZdJnBnVxkbnRERERC1gYJwBg+LEhaOfLFacOGl1vsboYdShrylGFFuVXwZ/qG1WZiUiovYlGHCjIn8VvM5cqVOJGjKFFrFpF4piLuteeKvyJMqIGsKiFBEREVELubRrIozKmoHoYQCLjlvgDTatoDQtMwEnLeoHmy+ATcW2lkmSiIiiQjgcgqN0O4p+ewPO0m2w5q9GuImru3Zm+vhzodQki2J8DdsfFqWIiIiIWohWIccV3Wp9AfYG8E1uaQNn1C9Bo8KIJJMotq7QCoc/0NwUiYgoSviqCmDN/zYyBc3nLkJVxR6Js4oegiCDOWOyKOZzFaKq4leJMqL6sChFRERE1IJ6xupwflKsKLajrBIHbM4mXWd8Why08pqvat5QCN8XlLdIjkRE1P6p9ZnQmvqKYrbCHxAKehs4g2rTGLrV8xqu5WvYjrAoRURERNTCpmQkIEGjjGwblHIohaZ97dIp5JiQHi+K7SithMXFL9JERJ2FOW0SIMgj26GAE5XFGyXMKPqY0ybyNWzHWJQiIiIiamEquQxXdUuBDMDZZj3+OqAresbqmnyd4YmxouJWGMDKvFL2wyAi6iQUahOMSeeLYpUlW+D3VkiUUfRRqM31voYBr1WijOhkLEoRERERtYIMvQZ/GdAF1/ZMhU4hP/0J9ZDLBEzLTBDFjlS6cdDuaokUiYgoChiTR0Ou0NcEwkHYCr6XLqEoVN9raC34TrqEKIJFKRKZM+dpjB495LT/zZnzdLPvtXLlCowePQS7du1ofuIN2LVrB0aPHoKVK1e0yHHtxX333YkrrrhE6jSIiOg0krXqZl+jT2wMuhu0oti3eaUIhjhaioioM5DJVTClTxTF3PZseBzHJcoo+jT8Gh6TKCP6g+L0h1BnMmPGTAwZMiyyvWfPL1i+fAkuvfQyDBo0OBJPT89o9r0GDRqMf/zjX8jK6tbsazUkK6sb/vGPf2HAgIGtdg8iIqLWJAgCLu6SiDf25+KPMlSpx49tpXacn2ySMjUiImojOvPZcJRug89VGIlZ81cjpc+dEJrYs7Czqn4Nt8PnKojErPlr+BpKjEUpEhkwYKCogBMMBrF8+RIMGDAQU6Zc1KL3Sk/PaJHi1qnExcW3eN5ERETNVekLIL/Kg35m/ekPBpCqU+O8BCN2lFVGYmsLy3FOvAHaM5waSERE0UMQBJgzpqL40H8iMb+nBM7yXTAkDJEws+hR/RpOqfsalu2EIXGohJl1biwHEhEREbWhvRUOvLYvB/87akGxu/Er6U3KiIdKJkS2XYEQfixko1sios5CHZMBnflsUcxe+CNCAbdEGUUfdUwGYuLEs2jsResQ5GsoGRal6Ix9+OG7GD9+JNav/xGXXjoFkyaNwddfLwUAHDyYjSeeeBiXXDIZY8cOx/Tpk/D000+gpKQ4cn7tnlJ/bB8+fAhPP/0Epk69EJMmjcHjj/8NRUWF9aVQ51rr1q3FlVdeigkTRuHDD9+tt1eU2+3Ga6+9hBkzpmLixNF48slH4HQ661wzEAjg/fffxsyZF2PChFG47747cfjwIYwdOxwffvhunfvfcst1GD9+JKZPn4g5c55GWVnZGb2uFksRnnjiYcyYMQXjx4/E9ddfic8++xihUKjBc1wuF+644yZMmnQBfv119xndl4iIWt/ynBIsPGqBOxhCIBzGomPFCDSyN5RBqcDY1DhRbHOJDWUeX2ukSkRE7ZApbQIEWc2qrKGgG3bLBgkzij6x9b6G6yXMqHPj9L1m8jiOoyLvWwS8Z1aAaC0KdQLiMqdBY2i9fk1AdeHmxRf/D9dddwN8Ph8GDjwHR48ewb333oaMjC644YaboVZrsHfvHqxevRLl5WWYN+/dU17zscceQlZWd9x11ywUFORj0aKFKCsrwfvvf3LafJ577l+44oqrodfr0b//QASDAdH+cDiMRx99ELt378Kll16Gbt2648cf1+K5556uc61nnnkSP/74PaZNm44+ffph06aNuP/+u+sUh/7zn/fwn/+8h3HjJuDSS2eipKQYX331BX75ZSc++GABTCbTafP+QyAQwAMP3AePx4Orr/4z9HoDNm/+GW+/PQ/BYBA33nhrnXP8fj/+/ve/4ejRI5g791UMHHhOo+9HRERtq6teiy0l9sh2ocuLHwrLMTkj4RRn1RidYsK2UjvsvurPt2AYWJVXhut7pbVKvkRE1L4oVEYYk0fBXrQuEnOUboc+4TwoNY37LOnsFEoDjMkXwF70QyTmLN0Offy5UGmTJMysc2JRqpkq8r5BwNv+hs4HvGWoyPsGaf3ua9X7hEIhXH75Vbj++psjsX//+3kIgoB5896B0RgLoLqBut/vx9q1a1BZaY/E69OnT1/MmTM3su3xuLF06ZfIy8tFZmaXU+YzduyFuPPOeyPbtVf227RpI3bt2oH7738IV1113e+5XY7Zs+/Hzp3bIsft2fMLfvzxe9x4462R682ceSWeeOIRbNjwY+S4goJ8zJ//Aa6//mbcfXfNaz1p0hTceuv1+OSTD3H//bNPmfPJDh3KxokTx/Hssy/gwgurV4e45JI/Yfbs+5Gbm1Pn+FAohGeeeRJ79vyC556bi/PO41xoIqL2bGCcHr9Z9dhrrRmhu77Iij6mGHTRa09xZjWlTIYpGfH44ljNyOPfbFU4VulCd6OuVXImIqL2xZB0PpzlvyDos0OQKWFMHg2FyiR1WlHFmDQCzvJdCPpsv0fCsBWsQWKPP0MQhFOdSi2M0/eo2YYPP1+0PXv2Y1i0aIWo8FRV5YRaXb0stst16vm648dPEm337NkbAFBRUX7aXIYNG3HK/Vu2bIJMJsP06X+KxBQKBWbOvFJ03B+Fp2uu+XMkJggC/vznm2odtw6hUAijR4+BzWaL/BcXl4Bevc7Cpk0bT5vzyRISEiEIAhYs+Ahbt26G3++HIAh4+eV5ePLJZ+ocP3fu81i3bi0eeeQJnH/+6Cbdi4iI2p4gCJiRlQSDsqY5eRjAomPF8AUbnqZ9skFxBmTGaESxlXllCIUbNw2QiIiim0ymhDltEmLiBiK1332ITbkAgozjTZpCkClgThf/3ulxHIO78pBEGXVe/JvbTHGZF6Mi/1sEPO1s+p4mAXEZ09rkXmazuL+FIAiorLTj008/wpEjR1BYmA+LpQjh378sh8On/tJtMplF2yqVCkD1SoBNzaU2i6UQZnMcdDrx0+SuXbNE23l5eTAaY+uM6Kp9XEFBPgDg7rvrTqsDAKVSWW+8IUlJybjvvr/irbfmYfbsv0Cr1WHIkKEYP34Sxo+fBLm85pcYi6Uo0sPr11/3YNq06U26FxERSUOnkOPybsmYf6imX2K5149v88owI+v00wYEQcDFXRLwzoH8SKzQ5cUv5Q6cl2BslZyJiKh90Zn7QWfuJ3UaUU0b2wdqfRa8zhORmK3gO2gNPVjka0N8pZtJY+iGtL73nv7ADkwmEw+427RpIx5/fDbi4xNw3nlDMWLESPTp0xfbtm3BggUfnfZ6zRkuWTuX+q7t89VtCFu7T1QgEKi3oPRHgazmvOpC2QsvvBwZCdZcf/7zjZgwYQrWr/8Bmzf/jG3btuCnn9Zj1aqVeOml1yPHCYKA2bMfw969e/D110sxbdrF7CdFRBQlesfGYHhiLLaW1vSX2lpqR19zDHrHxpz2/C56Lc6O02NvRc00wDX5ZTjbrIdKzoHwREREpyMIAswZU2DJfg/V45aBgLcCjtJtMCaPlDa5ToTfWqjFvfLKXGRkZOKzzxbjiSeexrXXXo/Bg8+DzWaTOjWkpaXD4aisk0thYUGd46zWClRViVfly8/PFW2nplY3lk1OTsbQocNF/wUC/iYXqior7di5czuMxlhcfvnV+Pe/X8fXX3+PceMmYOvWTTh69Ejk2OTkFPzpT5dj1qy/IiYmBnPnPodAIHCKqxMRUXsyLTMB8WrxA5AvjxfDFTj9yGAAmJqRAMVJD3Ic/iA2WKwtmiMREVFHptImQ59wnihmt2xA0F93dXZqHSxKUYurrLQhOTkVWm1Nw9biYgs2bKhe3aAx0/Bay5gxFwIAFi5cEImFw2EsWbJYdNzYseMQCoXqxL/6apFoe9SoCwAACxbMj0xPBIDDhw/iscdm44svFjYpv23btmDWrLvw8881y7pqtVp0794DQP0jweLi4nHbbXfj+PFjoj8XERG1byq5DFd2T8bJ44Md/iCWnSgRfaY0xKxWYlSySRT7yWKF3edv2USJiChqhENBeBzHpE4jqsSmjoMgr+nVGA75YCv84RRnUEvi9D1qcSNGjMTatd9h7tzn0LdvPxQUFGDFiqVwuz0AAJerSrLczj13CMaPn4TPPvsY5eVl6NdvADZu3IBDhw6Ijhs6dARGjboA77zzBnJzc9C3b39s374VW7duAlAzxbB795644oprsHjx/2C32zFmzFhUVlbiyy8/h1arwx133NOk/EaNGoOuXbPwwgvP4uDBbGRkZCAnJwdffvkFzjtvKLp1617veTNnXolvvlmO+fM/wIQJk5GWln4Grw4REbW1Lnotxqaasa6oZoTTXqsTfSscOCf+9P2hxqaZsaOsElW/j67yh8JYk1+OK7untFrORETUPrnth2EtWIOAtwIpfe6ESpssdUpRQa7QwZQyFtaC1ZFYVcVu6BOHQK1LkzCzzoEjpajFzZ79OKZPn4GNG9fjlVfmYt26tZg69WK89tpbAICdO3dImt9TTz2Lm2++Hbt27cAbb7yKcDiEf/5zTp3jnnnmeVx11bXYsmUT5s17GU6nA8888xwAQKms6S3117/OxuzZj8Fms+LNN1/DV18twsCBg/HWWx/UaYx+OlqtFq+99ibGjLkQa9Z8i5deehE//PAdLrvsCjz33NwGz5PL5Zg9+1H4fD689NKLTbonERFJa3xaPNJ04uney3NKGzXiSSOXY1J6vCj2S7kD+U5Pi+ZIRETtW0XeSpQeW4iAtxxAGNb81Y0adUvV9IlDoNAkiGLW/FV8DduAEOarHFFe7kQo1PDLYbHkICWlaxtmRFJxOp1QKpV1ekJlZx/A7bffgMce+wemT5/RKvdWKGQIBBq3LHhz8e80Uf0SEw0oLXVInQZ1IsVuL97cn4fASV/LxqSYMTUz4RRnVQuFw3hjfy4s7pqFPLrqNbizT0azFg+REt+DRNLj+zC6uOwHUXbsc1EsodtV0Jn6SJRR9HFXHkHp0f+KYvFdZyImboAk+XSU96BMJiA+Xt/w/jbMhShqrF//AyZNugB79+4RxdeuXQMA6NevvxRpERFRB5WsVWNyRvWIJ4Ug4KLMhMj26cgEAdNqFa9ynB7ss7JJKxFRZ6E19obGIG71Yc1fDb+nTKKMoo/W2BMaYy9RzFb4PULBuqu3U8thTymieowceQFiYvT45z//jssuuxKxsbHYv38vVq5cgSlTpqF7956NvpbL5YLb7WrUsfHxp38iTkREHdPIZBMqvH4MT4pFsrZpq7f2io3BWbE6HLTXfN6syi9DX1MMFPUskkFERB2LIAgwp09BUfY7AKpH3Qb9dliy34MpYzL08edF7ejZtmROn4wix1EgXD1zJeivRGXJJphSx0mbWAfGohRRPcxmM95++0P85z/vYfHi/8HhcCA1NRV33TUL1157Q5OutXDhAnz00fuNOnbjRmn7bRERkXRkgoBLuyad8fnTMhNx2J6DPyaAW70BbCq2Y0yquWUSJCKidk2pTYQ+cSicpdsisXA4AGveSnjsRxDX5RLIlTESZtj+KTXxMCQOh6NkcyTmKN4EffxgKFSxEmbWcbEoRdSArKxu+Ne/nm/2daZOvRgDB57T/ISIiIhOIUmrwrCkWGwpsUdiPxZV4NwEA/RKfuUjIuoMzGkTEfI74bL9Joq7Kw+hKPsdxHe5FNrYXg2cTQAQm3IBqip+RShQvWp8OByAreB7JHS7XOLMOiZ+QyFqZenpGUhPz5A6DSIiinL+UAjK00zFm5gej93lDniC1eOlvMEQ1hZUYEbWmY/AIiKi6CHIFIjPuhyaip7Vq8eFavohhQJVKD22EPqEITClT4JMppQw0/ZLJtfAlDYeFbkrIjGXbT88ziHQ6LlIVEtjkwEiIiKidswfCmFlbine2J8HX/DUq7PqFHKMT4sTxbaV2lHs9rZmikRE1I4IggB9/DlI6XMnVDF1H447y3bAcvB9+NwlEmQXHWLiBkGpTRXFrPmrEQ63zSrpnQmLUkRERETtVKHLi7d+y8PGYhtKPT6szj/9KkojkkyIV9c8/Q4D+DaPqy8REXU2SnUcknvdjNiUsQDETc6DPgdkMpU0iUUBQZDBnDFFFPO7Lagq3y1NQh0Yi1JERERE7dS2EhuK3TVTLzaX2HHYXnXKcxQyAVMzxau5HrK7cOg05xERUccjCDLEpo5Fcu9boFDVLHwRlzkNCrVJusSigEbfBTpTf1HMVvQDQkGPRBl1TCxKEREREbVT0zITYVaLW4B+ebwE7kDwlOf1M8Wgm0Eriq3MLUMwHG7xHImIqP1Tx2Qgpc+diIk7BzpTf+jMZ0udUlQwpU+EINR8DocCLtgtGyTMqONhUYqIiIionVLLZbiyW4po0kWlP4AVOaWnPE8QBFyUmSA6r8Tjw/ZSe4PnEBFRxyaTqxHf9VLEZ10GQRDqPcbnLuFIoJMoVLEwJI8UxRwl2+D3cFp8S2FRioiIiKgdyzJocUGKWRTbXeHA3grHKc9Lj9FgcIJBFPu+oAKe04yyIiKijk0Q6i8DhIIelB5biKLsd+Fx5rZxVu2XMXkU5ErjSZEQrAXfSZZPR8OiFBEREVE7NzE9DilacUPapSdKUOkLnPK8yekJUMpqnoa7AkH8WGRtlRyJiCi6VeR9i6DPjqDPjpLDH8NW+APCYT7IkMmUMKVPFMU8lYfhrjwiUUYdC4tSJDJnztMYPXrIaf+bM+fpZt9r5coVGD16CHbt2tGurhVN7rvvTlxxxSVSp0FERK1MIZPhqu4pkJ803cIdDOGrE8UIn6JPlFGlwJhao6w2FdtQ4fG3Wq5ERBR9XPaDcFn3nhQJo7J4I4oPfQS/p1yyvNoLnak/1DGZopg1fw2Ldi1AcfpDqDOZMWMmhgwZFtnes+cXLF++BJdeehkGDRociaenZzT7XoMGDcY//vEvZGV1a/a1iIiIOroUnRqT0uOxKr+mj8UhuwvbSu0YnmRq8LwLUszYXlqJSn/1qKpgOIxV+WW4rmdqa6dMRERRQmvoAUPicDhKt4riPlchLAffgzl9CmLiBzfYi6qjEwQB5owpsBz8IBILeMvgKN0BY9JwCTOLfixKkciAAQMxYMDAyHYwGMTy5UswYMBATJlyUYveKz09o0WKW0RERJ3F6BQTsm1OnHDWNKFdmVeGHkYdEjSqes9RyWWYkhGPRceLI7F9VidOONzIqrVCHxERdU6CTAFzxhRojD1RnrMMoYAzsi8c8qMi72u4Kw8jrsslkCt0EmYqHZUuDTHxg1FV/kskZresQ4x5AOTKGAkzi26cvkdEREQUJWSCgCu6p0B1Up8ofyiMRceKETzFNL5B8Qak69Si2De5pQid4hwiIup8tMYeSO17N7SxZ9XZ57YfRNGBdzp1LyVT6oUQZDUPgcJBL+yWddIl1AGwKEVn7MMP38X48SOxfv2PuPTSKZg0aQy+/nopAODgwWw88cTDuOSSyRg7djimT5+Ep59+AiUlNU9pa/eB+mP78OFDePrpJzB16oWYNGkMHn/8bygqKmxyfvn5efi///snLrvsIowbNwLTpo3HI488iGPHjoqOW7duLW6//UZMmjQGU6aMxQMP3Itff90tOuaXX3Zi1qw7MHXqOEyadAHuuedWbNy4oc49v/56KW6++TqMHz8S06dPxDPPPHlGuQOAxWLBE088jBkzpmD8+JG4/vor8dlnHyMUCjV4jsvlwh133IRJky6o82cgIqKOIU6txPQuiaJYXpUHG07RwFwmCLio1jkFLi/2lJ96BT8iIup85AodErpdhbgul0CQKUX7QgEnSo/+FxX5qxAKdb7+hHKlHrEpY0QxZ9ku+FwWiTKKfpy+10xHK11YnlOC0nbWMDRRo8SlXZPQw9i6QysDgQBefPH/cN11N8Dn82HgwHNw9OgR3HvvbcjI6IIbbrgZarUGe/fuwerVK1FeXoZ589495TUfe+whZGV1x113zUJBQT4WLVqIsrISvP/+J43Oq6KiHHfddTN0Oj0uv/wqxMaacPjwQaxYsRQ5OcexcOFXkMlk+OWXnfjnP/+OESNGYvr0GfB43Pjyy0V44IFZWLDgc6SnZyA39wQeeeQB9Op1Fu68cxbC4TBWrFiKxx+fjTfeeB+DBp0DAHjzzdewcOECnHfeMNx77/0oKyvDl19+ju3bt+L99z9Gampak17Xv/3tL/B4PLj66j9Drzdg8+af8fbb8xAMBnHjjbfWOcfv9+Pvf/8bjh49grlzX8XAgec0+n5ERBRdzksw4oCtCgdsVQAAs0px2ql43QxaDDDrsc9aMyVjTX45+pv1UMn5nJKIiGoIggB9/GCoY7qgPGcJfC7xg3Zn6TZ4HccRnzUTKm2yRFlKw5A4HM7yXQh4K36PhGEtWI2knjd22p5bzcGiVDMtPVGCcm/7KkgBQKnHj6UnSjB7YFar3icUCuHyy6/C9dffHIn9+9/PQxAEzJv3DozGWADVDdT9fj/Wrl2Dykp7JF6fPn36Ys6cuZFtj8eNpUu/RF5eLjIzuzQqr5UrV8But+Ottz5E165ZkbhOF4NPP52Pw4cP4ayz+mDt2u+gVmvwwgsvR/4BGTp0BJ588hEcOpSN9PQM/PTTerjdbjz33L9hMpkAABMnTsbdd9+Kw4ezMWjQOThx4jj+979PMWbMhZgz5/9FrnXBBeNw99234O235+Ff/3q+UbkDwKFD2Thx4jieffYFXHhh9fKjl1zyJ8yefT9yc3PqHB8KhfDMM09iz55f8Nxzc3HeeUMbfS8iIoo+giDgsqwk5O7LRV9TDC7qkgCNXH7a86ZmJOCArSoy1c/uD+AnixUT0uNbO2UiIopCSk08knvfArtlAyotGwHUTPv2e0pRVbEXqvTOVZQSZHKY0yej9Nj/IjGvMwduezZ0pr4SZhad+FiMmm348PNF27NnP4ZFi1aICk9VVU6o1dW9LFwu9ymvN378JNF2z569AVSPfmqs66+/GStWrBEVpLxeD2Sy6r/ybrcLAJCUlASXqwqvvjoXJ04cBwD06NETCxd+FSkGJSZW/yP7yisvIjv7AAAgNtaEhQu/whVXXAMA+PnnDQiHw7j++ptE1fH+/Qdg6NAR2LTpJwQCgUbnn5CQCEEQsGDBR9i6dTP8fj8EQcDLL8/Dk08+U+f4uXOfx7p1a/HII0/g/PNHN/o+REQUvfRKBf46oAtmdktuVEEKAOI0SoxMNoliGyxWVPoa/xlFRESdiyDIYUq9EMm9boJcZYrEldoUmFLHSZaXlDTGXtAYuoti1oLvOuWUxubiSKlm+lNWEpbnlKLU45M6FZFEjQqXdk08/YEtwGyOE20LgoDKSjs+/fQjHDlyBIWF+bBYihD+/alsONxwTyQAMJnMom2VqrqRXDAYbFJefr8f7733Fg4ezEZBQR6Kigoj1/ijL9Pll1+Fbdu24Msvv8CXX36B1NR0jBo1GhdfPAO9elUXw8aPn4gNG37E2rXfYe3a7xAfn4Dzzx+FadOmY9CgwQAQ6RvVpUtWnTyysrKwbdtm2O02xMcnNCr3pKRk3HPP/Xj33Tcwe/ZfoNXqMGTIUIwfPwnjx0+C/KRfPiyWokgvr19/3YNp06Y36XUiIqLopVc2/avchalm7CyrhCtQ/ZnoD4WxpqAMV3RLaen0iIioA1HruyC1z12w5n8Ll/U3JHS9DIKsc5YUBEGAOX0KirLfwR+jx4I+GxwlWxCbcoG0yUWZzvk3qAX1MOrw4NldpU5DUn+MPvrDpk0b8fjjsxEfn4DzzhuKESNGok+fvti2bQsWLPjotNdriXm42dkH8Je/3Am1WoMhQ4bh4osvRe/efVBQkI+XX34xclxMjB5vvPEe9u3bi59+WoctWzZh8eLP8dVXi/Dkk//C5MlToVAo8H//9yKOHj2C9et/wJYtm7By5Qp8/fUy3HXXfbjhhpsjBbf6hELV+5RKZYPH1Oe6627A5MlTsX79D9i8+Wds27YFP/20HqtWrcRLL70eOU4QBMye/Rj27t2Dr79eimnTLmY/KSIiapBGIcfE9DgszymNxH4pc+D8JBPSYzQSZkZERO2dTK5GfNc/ITZlLBRqc73HhMNhhEN+yOSqevd3FEptIvSJQ+Es3RaJVRZvREzcIChURgkziy6cvkct7pVX5iIjIxOffbYYTzzxNK699noMHnwebDZbm+Xw1luvQalUYcGCL/D003Nwww23YPjw8+F0ilcZys3NwYED+zFgwNm4556/4OOPF2LBgi9gMBjwv/99CqB6Fbw9e3ajR4+euPXWO/Hee/OxePEKZGR0wcKFCwAAKSnVTcxzck7UySU3NwdarRYGQ+P/YaqstGPXrh0wGmNx+eVX49//fh1ff/09xo2bgK1bN+Ho0ZplWJOTU/CnP12OWbP+ipiYGMyd+1yTpgoSEVHHY/X68dMpVuMbmhiLJM1JS1oDWJlXdsqHLERERH9oqCAFAI6SzbBkvwuvM68NM5KGKWUsZPKahUbCIT9shT9ImFH0YVGKWlxlpQ3JyanQamvenMXFFmzYUP3mbOo0vDNht9thNpthNtf8Y+l0OrFy5deiHF599d947LGH4HK5Isd17ZoFvd4A+e8rES1Y8B888MA9KC0tiRyTlJSMpKSkyCixUaOqh2h+9tnHoi/0Bw9mY8eOrTj//NFNGgG2bdsW3H//3fj55w2RmFarRffuPQDUHZ0GAHFx8bjttrtx/PixSLGMiIg6l3A4jJ1llXh9Xy6+zS/DvgpnvcfJBQEXdRFPKT/ucEdW8yMiIjoTPpcFtqIfEPBZUXx4PmxF607bviWayRRaxKZeKIq5rL/CW5UvUUbRh9P3qMWNGDESa9d+h7lzn0Pfvv1QUFCAFSuWwu32AABcrtb/wjtixEh89tnH+Mc/HsOwYSNQXl6Gr79ehoqKit9zqC5CXXPNn/G3v92PWbNux7Rp06FSqbBhw3oUFOTjlluqG4rPnHkVVq36BrNm3YEZM2bCYDBi587t2LVrB26//W4AQPfuPXDFFddg8eL/4YEHZmHMmLEoKyvDl19Wj7q6++77mpT/qFFj0KVLV7zwwrM4eDAbGRkZyMnJwZdffoHzzhuKbt2613vezJlX4ptvlmP+/A8wYcJkpKWln+lLSEREUWhVfhl+stgi20tzStDVoIGhnt5TvWNj0DtWh0P2mgcz3+aVoXdsDBQyLmlNRERNEwr5UZbzFRApQoVRadkAT+VRxGddBqU67pTnRyt9wrlwlu2A31MziMGavwrJvW9rkdY0HR1HSlGLmz37cUyfPgMbN67HK6/Mxbp1azF16sV47bW3AAA7d+5o9RxuvfVOXHvtDdi/fy9eeWUuVq5cgaFDh2P+/M8gk8mwa9d2AMCwYSPwwgsvQ6PR4qOPPsC8ea/A4bDj6afnYOrUiwFUr8b36qtvISMjEwsXfvr7Sn3H8OCDD+Omm26L3POvf52Nhx56FFZrOd5441V8881yjBkzDh9++GmTi0NarRYvv/wmxoy5EGvWfIuXXnoRP/zwHS677Ao899zcBs+Ty+WYPftR+Hw+vPTSiw0eR0REHdOgOAPkJ33/dQWCWHK8pMFpedMyE0RfBsu9fmwpsbVqjkRE1DEJEOqsSAcAPlcBLNnvwln+S4ecJi4IMpgzpohiPlchqip+lSij6CKEO+LfijNUXu6MNKWuj8WSg5SUzt3UnFqfQiFDINA2Q1z5d5qofomJBpSWOk5/IFE7tK6wAmsKykWxy7KSMDQxtt7jl50owdZSe2RbI5fhbwOzoFPI6z2+LfA9SCQ9vg/pTLnth1GeuxyhQN0ZMtrYPojrMh1yhU6CzFpX6bEv4LZnR7blCj1S+82CTK4+o+t1lPegTCYgPl7f8P42zIWIiIiIWtkFqWZ0qbWK3je5pajw+Os9fkJ6HNTymq+EnmAI39cqahERETWWNrYXUvvcDW1s7zr73PZsWA68A3flUQkya13m9EmAUPNAJxhworJ4o4QZRQcWpYjaiMvlQnl5WaP+IyIiOlNyQcCV3ZOhOqkvlC8UxqLjFoTqGSCvVypwYaq4z8e2EjtK3L5Wz5WIiDomuTIGCd2uRlzmxRBkStG+YMCJ0qOfwZq/GuFQx1k1XKE2w5g0QhSrLNmCgLfh1XCJjc6J2szChQvw0UfvN+rYjRtbv+8WERF1XPEaFaZlJmJZTk3T1RynBxstVoxJrdtodmRyLLaW2mD1Vv9yEALwbV4pburNBTOIiOjMCIIAfcJ5UOuzUJ6zBD5XoWi/o3QrPI7jiM+6DCptskRZtixj8mhUle9BMPD76rfhIKwF3yGx+1XSJtaOsShF1EamTr0YAweec9rj5HIOYCQiouYblmhEts2JgyetrvddQQV6xcYgVSfub6GQyTA1IwELj1oisYN2Fw7bq9ArNqbNciYioo5HqYlHcu9bYC9aj8rinwHUjNr1e0pQevS/SOv3Fwiy6C9PyORqxKZNQEXuskjMbc+Gx3EcGkM3CTNrv6L/p04UJdLTM5CennHa49qy0TkREXVcgiDgsqxkvL4/B67fP1eC4TAWHbPg3n6ZUMjED0EGmPXoqtcgx+mJxL7NK0MPow4yLmlNRETNIAhymNLGQ2PsgfKcpQj6ahbYMGdM6xAFqT/ExA2Es2y7aGSYNX81UvrcCUHgAITa+IoQERERdVBGlQIzuiaJYha3D98XVNQ5VhAEXJyZWOfYnWWVrZojERF1Hhp9V6T2uQs689kAgJj4wdCZ+kicVcsSBAHmjKmimN9TAmfZLokyat9YlCIiIiLqwM6OM+CceIMo9pPFiuMOd51jM/QaDK517Jr8cniCwVbNkYiIOg+ZXIOErMuQ0O1KmNOnSJ1Oq1DHZEBnHiiK2Yt+RDBQ97O3s2NRioiIiKiDu6RLImKVNVMjwgD2lDvqPXZyRjyUJ63cVxUIYn0RVw4iIqKWpTP1hUyuqndfMOBC8eH58Fblt3FWLceUNl608mAo6Ibdsl7CjNonFqWIiIiIOjitQo7Lu1evbKSSCfhT1yTM6JpY77GxKiUuSDGLYj9bbLB6/a2eJxERUTgcRkXu1/A6c1F86CPYi9YjHI6+nrsKlRHG5NGimLN0O/zuUokyap9YlCIiIiLqBHoadbikSyL+0r8LhiXFQjhF8/ILUswwKOWR7UA4jNX5ZW2RJhERdXJVFbvhtmf/vhWG3bIexYfnI+CNvlG7xqTzIVeZToqEYS1YjXA43NApnQ6LUkRERESdxPnJJsRr6p8qcTK1XIbJGQmi2K8VTuQ62QuDiIhal99T9yGIryofRdnvwlm+J6oKOoJMAXP6JFHM4zgGd+UhiTJqf1iUIiIiIqI6BscbkKZTi2Lf5JYhFEW/DBARUfQxp09CYvdrIFPEiOLhkA8VuctQdmJxVDUM18b2gVrfVRSzFXyHcCggUUbtC4tSRERERASHX/zlWCYIuChTPFoqr8qDvRXOtkyLiIg6IW1sb6T2uRsaY686+9y2A7BkvwOP47gEmTWdIAi/rzJYM20+4K2Ao3SbdEm1IyxKkcicOU9j9Oghp/1vzpynW/S+LlcVrNaWmSP8x59BKkVFhRg9egg+/PBdyXIgIiJqLG8whKUnivHy3pw6zcy7G3XoZxI/qV6VXwZ/KPoazhIRUXSRK2OQ2P0amDMugiAoRPuCfgdKjiyAtWBNVIw4UulSoE84VxSzWzYg6OeDHsXpD6HOZMaMmRgyZFhke8+eX7B8+RJceullGDRocCSenp7RYvfMzj6Axx57CE899SzMZumKSURERJ1NrtONL44Vo+L3YtTi48W47ax0yE5qgj4tMwEH7VUI/j5rz+4L4GeLDePS4qRImYiIOhFBEGBIHAKNIQtlJ5bA7y4S7XeUbIHHcRzxXS+DSpskUZaNE5syDlXW/QgHPQCqpyPain5EfJdLJM5MWixKkciAAQMxYMDAyHYwGMTy5UswYMBATJlyUavc89ixIygr47KYREREba2gyhspSAHAcYcbPxfbcEGKORKL16hwfpIJG4ttkdi6ogqcl2iEQcmvkkRE1PqUmgSk9L4Vdss6VBb/LNrndxej+OAHSOp1I9QxLTd4oqXJlTGITRkLW8HqSKyq/BcYEs6DSpcmYWbS4vQ9IiIiok5qeFIsehp1otia/HJYXF5R7MK0OOgUNV8bfaEwvssvb5MciYiIAECQyWFKm4CknjdCrjSK9im1yVFR2DEkDoFCLe7XaM1fHVUrCrY0FqXojO3b9yseeOBeTJo0BpMmjcGDD87Cb7/tEx1TWVmJOXOexsyZF+PCC8/HVVfNwDvvvAGvt/rL7ocfvovnnnsGAHD//XfjiitOPXTRYinCs8/+A9OnT8T48SNx003XYvnyJac8JxwO46OP3se1187E+PEjccklk/Hss/9AcbHljP7c69atxe2334hJk8ZgypSxeOCBe/Hrr7tPec7u3bswfvwo3HPPrXC7o2elCCIi6thkgoDLuyVDI6/5ShgMh7HomAWBUM0XZK1Cjglp8aJzd5ZVoqhW8YqIiKi1aQxZSO1zN3TmAQAAQaZCfNZlEIT2X94QBDnMGZNFMW9VHly2/RJlJD2OuW4hh26/+YzOU3fpiq5PPVPvvpx//RPe3Jwzum7vD+af0XmNtX37Fjz88APo1as37rjjbvh8PqxcuQL33XcnXnnlzUj/qaeeegyHDx/ElVdei/j4BOzb9ys+/XQ+7HY7Hn30CYwdOx7l5WVYvnwJbrjhFvTt27/BexYWFuDOO2+Gz+fD5Zdfhfj4eKxf/yP+3/+bg/z8XNx771/rPe+TT/6Djz56HzNnXoWePXuisLAQixb9D9nZB/DJJ59DLpc3+s/9yy878c9//h0jRozE9Okz4PG48eWXi/DAA7OwYMHn9fbaOnQoG48++iC6d++Bf//7dWi12kbfj4iIqLXFqhSY0TUJnx+reVhT5PZhbWE5pmTUPM0dlhiLLSU2lHqqp/uFAXyTW4rbzkqHcFIPKiIiotYmU2iQkDUTVb+vzqdUR0+fQ62xJzTGXvBUHo7EbAXfQxt7FmQypYSZSYNFKWqyUCiEuXOfR9++/fHGG+9FijqXX341brnlOrz66lx89NF/YbVWYMeObbj33r/iuutuAABccsmfEA6HUVhYAADo2bMXBgwYiOXLl2Do0OE499yGG52/++4bqKy04/33P8FZZ/UBAMyceRUee2w2Fi78FFOnTkf37j3qnPfdd6swYsRIPPDA3yKxpKRkLF36JSyWoiY1bV+79juo1Rq88MLLkS/gQ4eOwJNPPoJDh7LrXCsvLxezZ9+PlJQ0vPzyPMTE6Bt9LyIiorYyKN6A32xO7K2oWQVoQ5EVfWJj0NVQ/TBFLhMwLTMRnxwujBxzzOFGtr0KfU38fCMiorYXE3f2Kfc7ynZApU2FOia9jTJqHHP6ZBQ5jgLh6tVsg/5KVBb/DFPqOGkTk0D7H99G7c6hQwdRWFiACy4YB4fDAZvNBpvNBq/Xi1GjLsDhw4dQUlKMmBg9tFodlixZjHXr1kamrf397//Ea6+91aR7BoNBbNr0M4YNGxEpSAGATCbDjTfeinA4jJ9/3lDvuYmJSdi1awe++GIhKiqq+1/86U+XY/78/zZ5FcGkpCS4XFV49dW5OHHiOACgR4+eWLjwK1x44UTRsWVlpXjoofsAAK+++iaMxtgm3YuIiKgtzeiaBKOyZvRwGMCi48XwBkOR2FmxOvQ0ikf8fptXJprqR0RE1B54q/JhzfsWxYc+gt3yE8Lh0OlPaiNKTTwMicNEMUfxJgR8dokykg6LUtRkBQX5AIC33noN06dPFP33+ef/BQCUlBRDpVLh4Yf/Dqu1HE8++SguvngCHnroPixb9lWkp1Rj2e02uN0udOnStc6+rKxuAKr7TdVn1qwHEBtrwuuvv4QZM6bi9ttvxPz5H6C8vKxJOQDA5ZdfhXPOORdffvkFrr/+Slx55Qy8+upcHD58qM6xK1YsRXGxBTabFXl5uU2+FxERUVvSKeSY2S1ZFKvw+rEyr2aFXEEQcFFmIk6erFfm8WNbaef7Ek1ERO1XKOhD+YklqH7EEoK96EeUHP4YAa9V6tQiYlPGQKaoWWwkHA7AVvC9hBlJg9P3Wkhr9HBqqNeU1EKhIADg9tvvRv/+9Q+X7NIlCwAwefJUjBhxPjZsWIfNmzdix45t2LZtC5YsWYz33psPlUrVqHueajWCUKi64q1U1j//tmfPXvjf/5Zg69ZN+Pnnn7B162Z88ME7+N//PsO7736Erl2zGpUDAMTE6PHGG+9h3769+OmnddiyZRMWL/4cX321CE8++S9Mnjw1cmxSUjKeffZFPPzwXzF37nP46KP/QqHgW46IiNqv3rExGJ4Ui60lNUWm7aWV6GfS4yxTDAAgRafGkEQjtpdWRo5ZW1COc+IN0Cka36eRiIiotbjtBxHwiQtQ3qo8FGW/i7jMi6Azny15P0SZXANT6nhU5H0dibls++FxDoFGX3cwRkfFkVLUZKmp1Utt6nQ6DB06XPSfXq9HKBSCWq2Gy+XCnj27AQiYPn0G5syZi6+//h5XXnktjhw5hG3btjT6niaTGVqtFjk5dRu/5/7eDD4pKbnOvmAwiIMHs1FcbMHo0WPx6KNP4quvvsEzzzwPp9Nx2pX76rvXgQP7MWDA2bjnnr/g448XYsGCL2AwGPC//30qOvbiiy9F//4DcOed9+D48WNYuHBBk+5FREQkhWkZCYhXix/0fHWiGFX+YGR7Yno8VLKaL/PuYAg/Fla0WY5ERESnEhN3NhK6Xy0aiQQA4ZAP5TlLUX7iK4QC0q+KHhN/DpTaFFHMmr+6XU01bG0sSlGT9enTD/HxCVi06HO4XK5IvKrKiaeeehzPPfcM5HI5jh07ilmzbsfXXy+LHKNUKtG791kAAPnvy0/LZNX/P9VoKLlcjuHDR2L79i04eDA7Eg+Hw/jss48hCALOP390nfNCoRDuv/8uvP76S6J4//4DRDk01quv/huPPfaQ6M/dtWsW9HpDg9e69NKZ6NOnH+bP/yAy9ZGIiKi9UslluLJ7smiKnsMfxPLcksi2QanAuFTxSkebS2wo8/jaKEsiIqJT08WehdQ+d0Nj7Flnn8u2H0XZ78LjONH2iZ1EEGQwZ0wRxfxuC6oq9kiUUdvjXCJqMoVCgQcffBhPPfU4br31elxyyQyoVGqsWLEEFksRnnrqWSgUCvTvPwCDBg3G+++/hZISC3r06IWSkmIsXvw5unbNwpAhwwFUj4ICgCVLFqO8vFw0Be5k99zzF+zatQN/+ctduPzyq5CQkIANG9Zh587tuPrqP6Nbt+51zlEqlbjiimvw8ccf4vHH/4bhw8+H1+vB8uVLoNFocPHFM5r0Z7/mmj/jb3+7H7Nm3Y5p06ZDpVJhw4b1KCjIxy231D/dUiaTYfbsR3HXXbfgpZdexMsvz2vSPYmIiNpaF70W41Lj8GNR9einRI0SF6SYRceMSjFhW6kdNl8AABAKVzc9v6FXWpvnS0REVB+5Uo/E7tfCWbYDtoLvEA4HIvuC/kqUHPkEhqSRMKVeCEEmzRR0jb4rdKb+cNn2R2K2wh8Q7DFUknzaGotSdEbGjZuAl19+A5988h/Mn/8hZDIB3bv3wAsvvIxRoy4AUN0M9fnn/43//Od9/PzzT1i+fAkMBgPGjRuP22+/O9IDasiQYRg/fhJ+/nkDdu7cjrFjL4Rara5zz/T0DLz33ny8//5bvzdL96Br12547LF/YPr0hotLt912F4xGI775ZjnefHMr5HI5zj57EP7xj2eb1E8KAIYNG4EXXngZCxZ8hI8++gA+nxfdu/fA00/PwcSJUxo8r2/f/pg+fQaWL1+C779ffcpjiYiI2oPxaXE4ZK9CF70WUzPjoZSJRwQrZTJMzUjA/45ZIrEDtiocrXShh1FX+3JERESSEAQBhsSh0OizUJazBH63RbTfUbIJHscxJGRdBqUmUZIcTekT4bYfjBTNQoEqFB1bC3XcWEnyaUtC+FRzpjqZ8nInQqdY0thiyUFKSudpOEbSUChkCATaZg4x/04T1S8x0YDSUofUaRBJzh8K1SlGnSwcDuPdA/nIrfJEYqk6NWb1y4SsGQ1k+R4kkh7fh9QRhUMB2Ip+hKNkc519gqCAOfNi6OMHSZAZYCtah0rLhpPykSOlz91QauIlyaelyGQC4uP1De9vw1yIiIiIKIqcqiAFVD99vqhLgihW5PJiV1llA2cQERFJR5ApYE6fhKSeN0CuNIr2hcMByJUxEmUGGJNHiXIKh4OwFX4vWT5thdP3qNOzWq0IhYKnPU6t1kCvb7jCS0RE1Bl10WsxME6PXyuckdh3BeU4O84AdRMXFCEiImoLGkM3pPa5CxV538Bl+w0AoE8cBm09TdHbikymhCltIspzvorEPJVHEQ6HIAgd9/OURSnq9O6440ZYLEWnPW7atOl44omnWz8hIiKidq7E7cNGixUzuiZBLhMwNSMBv1mrEPi9K4TDH8SGIismZUT3lAMiIuq4ZAot4rMuh9baG46yHTClTZA6JejM/eEs3wWv8wQAQKlNAnDm0+GjAYtS1Ok99dSz8Hq9pz0uIUGapndERETtRSgcxpYSO1bllSEQDsOoUmBiejxMaiVGp5iwrsgaOfYnixVDE40wqZUSZkxERNQwQRAQEzcQOvPZEBrohRj0OxH0V0Kla/3VZQVBQGL3a1BZ8jM0agEKw3kN5tVRsChFnd7AgedInQIREVFU2FBkxZqC8sj2usIKnBUbg0y9BmNT47CjtBLOQPWU+EA4jDX55biqR4pU6RIRETVKQ4WfcDiM8pxl8DiOIzZ1HIzJI1t9Kp1MroIp9cJOs9hAx52YSEREREQtalhSLAxKeWQ7BGDRcQt8wRDUclmd6Xq7KxzIc3pAREQUjZxl2+FxHAUQgr3oB5Qc+QQBn03qtDoUFqWIiIiIqFF0CjlmZiWLYmUeP1bllwEAzkswIkWrEu3/Jq8U4d97TREREUWLoN8BW4F49TuvMxdF2e+iqmKvRFl1PCxKEREREVGjnWWKwbBE8TLaW0rsOGyvgkwQcFEXcQ/GXKcHe61OEBERRRO50oD4rJmQybWieDjoRXnOEpSd+AqhAEcDNxeLUkRERETUJNMyExFXq4H5l8eL4QoE0dOoQx9TjGjf6rwy+EOhtkyRiIio2XSmPkjtezc0hh519rms+1CU/S48zhwJMus4WJQiIiIioiZRy2W4sluyaJHqSn8Qy3NKAADTMhIgO2mn1RfApmJbm+ZIRETUEuRKAxJ7XAdz+hRAkIv2Bf12lBz+GLbCtQiHghJlGN1YlCIiIiKiJutq0GJMqlkU+7XCiT3lDiRqVRieaBLtW1dohdMfaMMMiYiIWoYgCDAkDUfKWXdAqUmus7+y+GcUH/oP/J4yCbKLbixKEREREdEZmZAWj1SdWhRbnlMCuy+ACelx0Mprvmp6QyF8X1De1ikSERG1GJU2CSln3QZD0og6+3zuIliy34OjbAcX+GgCFqVIZM6cpzF69JDT/jdnztMtel+XqwpWq/WMzv3ww3cxevQQFBUVtshx7UVrvM5EREQtSSETcGW3ZMiFmrl67mAIXx0vhlYuw/i0ONHx20srYXF52zpNIiKiFiPIFDCnT0ZSz+shVxpE+8LhAFzW/RJlFp0UUidA7cuMGTMxZMiwyPaePb9g+fIluPTSyzBo0OBIPD09o8XumZ19AI899hCeeupZmM1Dmnz+2LHjkZGRCZPJfPqDiYiIqEWl6NSYnBGPb/NqpiwcrnRha6kdw5NM2FJiR7nXDwAIA1iZV4ZbeqdBEIQGrkhERNT+aQzdkdLnbljzvoHL9hsAQJBrEN/1T/yMawIWpUhkwICBGDBgYGQ7GAxi+fIlGDBgIKZMuahV7nns2BGUlZWe8fk9e/ZCz569WjAjIiIiaopRySZk26pw3OGOxAqqvBiRJGBaZgI+PVIUiR+pdOGQ3YWzaq3QR0REFG3kCi3isy6HpqIXrPnfIi7zYihUsVKnFVU4fY+IiIiImkUmCLiiWzLUMhk0chmu7p6Cy7tVN4Lta4pBd4NWdPzKvFIEQ+y3QURE0U8QBOjjByGt318QY+7f4HFBv7MNs4oeHClFZ2zfvl/xwQfvYP/+fQCAAQPOxh133IN+/QZEjqmsrMS8eS9j587tsForkJiYhPHjJ+GWW+6AWq3Ghx++i48+eh8AcP/9dyMlJRWLF6+o935z5jyN/fv34oorrsF7770FAHj66TnYv38vPvrofSxatBypqWkAgIKCfLz11mvYuXMH5HIZLr10JhSKun/dy8pK8fbbr2Pr1s3w+/0YPXoMxo2bgMcf/xtef/0dnHtu9XRCr9eLjz/+EN99twqlpSVITEzGlCnTcNNNt0GpVDb5tfvll5344IN3cPToYQSDQfTs2Qt//vPNGD16TIPnnDhxHLNm3Q6jMRZvvvk+4uLim3xfIiKi1mJWK3FtzxQka1WIVdV8NgqCgIu6JOLN/bn4owxV6vFjW6kd5yebJMmViIiopcmVDY8A9jhOoOToZzCljoMh6XwIAscH/YFFqRaS+8u/zug8pTYVqX3uqHdfUfb78LuL6t13Ol0GP3VG5zXW9u1b8PDDD6BXr96444674fP5sHLlCtx335145ZU3I/2nnnrqMRw+fBBXXnkt4uMTsG/fr/j00/mw2+149NEnMHbseJSXl2H58iW44YZb0Ldvw5VlACgutuCjj97HrbfeibKyUvTrNwD79+8VHVNRUY67774Vfr8fV199HdRqNZYsWQyHo1J0nMtVhVmz7kB5eRmuvPJamEwmrFixDJs3bxIdFwwG8cgjD2Lv3j249NLLkJWVhezsA/jkk//g0KGDePHFl5s0Zzg39wQeeeQB9Op1Fu68cxbC4TBWrFiKxx+fjTfeeB/nnXdunXMsFgseeug+6HR6vPba2yxIERFRu9Q7tv4v5Gk6Nc5NMGJnWc1n8drCcpwTb4BWIW+r9IiIiNpcKOBGec5SIByErXAt3JVHEN/1T5zm9zsWpajJQqEQ5s59Hn379scbb7wHubz6y+Tll1+NW265Dq++OhcfffRfWK0V2LFjG+6996+47robAACXXPInhMNhFBYWAKjuBzVgwEAsX74EQ4cOj4xMaojX68Xs2Y/hoosuafCY//53AWw2Kz74YAHOOqsPAGDatOm44YarRcd98cVCFBTk45VX3sTQocMj+d1ww9WorLRHjlu9eiV27tyGl16ah+HDz4/E+/btj7lzn8PGjetxwQXjGvnqAT/9tB5utxvPPfdvmEwmAMDEiZNx99234vDh7DpFKavVigcfvBcA8PrrbyMpKbnR9yIiImovJqXHY2+FA77fp+25AiH8WFiBi7okSpwZERFR66nI/xZBf81DGa8zB0XZ7yIu8+JTTvfrLDhmjJrs0KGDKCwswAUXjIPD4YDNZoPNZoPX68WoURfg8OFDKCkpRkyMHlqtDkuWLMa6dWvhdlc3P/373/+J115764zvP2zY+afcv2XLJvTp0y9SkAIAszkOEydOER23YcOP6NGjZ6QgBQA6XQwuu+wK0XHr1v0Ak8mMs87qG/mz2mw2nH/+KMjlcmzatLFJ+ScmVheVXnnlRWRnHwAAxMaasHDhV7jiimtEx7pcVfjb3+6HxVKEV199KzI9kYiIKNpUBYIYkxonim0usaHc45MoIyIiotanM/WFTC7urRgOelB+4kuUnViKUNAjUWbtA0dKUZMVFOQDAN566zW89dZr9R5TUlKMpKRkPPzw3/H//t//4cknH4VKpcI555yLsWPHY+rUi6FWq8/o/maz+ZT7LZZCjB49tk68a9cs0XZeXh6GDRt+2uMKC/Nhs1kxffrEeu9XXGw5dcK1jB8/ERs2/Ii1a7/D2rXfIT4+AeefPwrTpk2PTHv8w/r1P0ImkyEUCiE7+wC6dOnapHsRERFJzR0IYkVOKXZXOHBTr1TEqhSw+wIAgGAYWJVfhj/35EMXIiLqmHSmvlDp0lGRuwwex3HRPpf1V3irchDf9TJo9F0kylBaLEq1kNbo4dRQrymphUJBAMDtt9+N/v3PrveYLl2yAACTJ0/FiBHnY8OGddi8eSN27NiGbdu2YMmSxXjvvflQqVRNvv8f0wUbIggCfL66T11DoZBoOxgMQKmse3+VSl3ruCAyMrpg9uxH672fwWA8XcoiCoUC//d/L+Lo0SNYv/4HbNmyCStXrsDXXy/DXXfdh1tuuTVyrF6vx4svvornn38Gb7zxCkaMGAmjsWn3IyIikspxhxtfHLXA7q8uQi09UYoL08xYmlMaOWa/tQrHKl3obtRJlSYREVGrUqiMSOxxPRylW2ErXAuEg5F9QZ8dJYc/hjF5FGJTx0IQOlevRU7foyb7YwqZTqfD0KHDRf/p9XqEQiGo1Wq4XC7s2bMbgIDp02dgzpy5+Prr73HlldfiyJFD2LZtS6vkl5aWjry8nDrxP/pYiY/LrXNcfr44lpqahspKO847b6jozzp48HmorLRDp9PWucapWCwW7NmzGz169MStt96J996bj8WLVyAjowsWLlwgOvaCC8Zh0KBz8MADD6OiohzvvDOvSfciIiKSWuXvBSkAsPsDOO7wICNG/ABoZV4ZQuFw7VOJiIg6DEEQYEwagZSzbodSk1RrbxiVxRtRfOgj+D3lkuQnFRalqMn69OmH+PgELFr0OVwuVyReVeXEU089jueeewZyuRzHjh3FrFm34+uvl0WOUSqV6N37LACAXF79108mq/5/uIW+jI4ZcyGOHz+GLVtqVtFzOp1YvXql6LgLLhiHQ4eysW9fzep9Pp9PlC8AjBo1BpWVdixZslgUX7p0Mf75z79j+/ZtTcpvwYL/4IEH7kFpaUkklpSUjKSkpMhrUduIESMxZsyFWLFiKfbu3dOk+xEREUmlm0GL0Sniafd7KhzoW2uVvkKXF7vLHW2ZGhERkSRU2mSknHU7DIl1W8n4XIWwHHwPzrJdLfb7cXvH6XvUZAqFAg8++DCeeupx3Hrr9bjkkhlQqdRYsWIJLJYiPPXUs1AoFOjffwAGDRqM999/CyUlFvTo0QslJcVYvPhzdO2ahSFDqt+EJlP1l9UlSxajvLwckydPbVZ+1157Pdas+RZPPPEwrrrqOpjNZixb9hWAcK3jbsDq1Svx4IOzcOWV18BkMmPVqm+Qm1s9ykoQBADVK/KtWvU1Xn11Lg4dykbfvv1x7NgRLFv2FXr37oOLL760SfnNnHkVVq36BrNm3YEZM2bCYDBi587t2LVrB26//e4Gz7v//tnYtm0z5s59Dv/5z2dQKPj2JSKi9m9SehwO2atQ7K6ZWv9ziQ19YnXIttc83FqTX4YBZj1Ucj4zJSKijk2QKWDOmAKNsScqcpYhGHBG9oVDflTkfY3kjF4ADNIl2Ub4qU9nZNy4CXj55TeQlJSE+fM/xAcfvI2YmBi88MLLmDSpuqgkCAKef/7fmDHjcvz880a88sr/w/LlX2HcuPF4/fV3oFQqAQBDhgzD+PGTsHlz9TFer7dZuel0MXjrrQ8wbtwELFv2Ff7zn/cwaNBg3Hzz7aLjjEYj3nzzfQwdOhyLF3+ODz98Bz179sLtt98DAJF+UyqVCq+99jauueZ67Ny5Ha+99m9s2rQRl112BV555Q1oNJom5dejR0+8+upbyMjIxMKFn+LVV+fiS1ettAAAHH1JREFUxIljePDBh3HTTbc1eF5KSgpuuuk2HDt2FAsXftrEV4WIiEgaCpkMV3VPgVyoibkCIXhDYdEX0Up/EBss1jbPj4iISCpaYw+k9L0b2tg+orgxeRR0hs6xCIgQbidjwvLy8vDiiy9i27bqqVDjxo3DY489hri4uFY5rz7l5U6EQg2/HBZLDlJSuPpZR2Gz2WAwGOo0Tl+48FO8+ear+PzzpUhPz2jzvBQKGQKB0OkPbAH8O01Uv8REA0pLOZWIqCWtL6rA6nxxn4xeRh0OV9aMllLKBDx0dhZ6ppv5HiSSGD8LidpOOBxGVcVuWPNXQaFOQErvW5GUbOoQ70GZTEB8vL7B/e1i/o/VasVNN90En8+H22+/HcFgEB9++CEOHjyIRYsWNbhC25meRwQAb7zxCn7++ScsXboSanX1aKdgMIgff/weJpM50tCdiIiImu+CFDOybVXIcXoisRNON7RyGdzB6ocx/lAYa/LL0DPd3NBliIiIOhxBEKCPHwy1visQDkOQdZ4V+NpFUWr+/PmwWCxYsWIFevToAQAYNGgQbrnlFixduhRXXXVVi55HBABTpkzD6tUr8Ze/3I0pU6YBELB+/Q/47bd9ePTRJxtsOl6fyko7/H7/aY9TKpUwGmObkTUREVF0kgkCruiWjHn7c+H7fWS6PxSGQa2IFKUA4JdyB07YqhDT0IWIiIg6KKW66TO+ol27mL43ceJEZGRkYP78+aL41KlTkZycjI8//rhFz2sIp+91Plu2bMKnn87H0aNHEAgE0KNHT1x77fUYO3Z8k65z3313YvfuXac97pxzzsUbb7x3ymM4fY9IepyyQNR6tpfaseREiSimV8jhDAQj273MetzcIyWy6AgRtT1+FhJJq6O8B9v99D273Y68vDxMmTKlzr7+/ftj3bp1LXoe0clGjBiJESNGNvs69933IByOytMeZzAYm30vIiKiaDYkwYgD1ipk26siMddJBSkAOGx1YnluKQxKyb+qEnVaMXYnqqp8pz+QiFpFlt+PDJm8w69KK/knfXFxMQAgOTm5zr7ExEQ4nU44HA4YDIYWOY+oNfTp01fqFIiIiKKCIAi4rFsSXtuXA9fvI4NDADRyGTwnTePbWmKXKEMiIqJ2oKAcfU0xuKFXx+51LHnJraqq+imZVquts0+tVgMAXC5XnX1neh4RERERScugVOBPXWseLKbp1LiyW7L0X0yJiIjakUMnjSruqCQfKRUKnb53Tn0Np8/0vFM51TxHACgpkUGh4Nclan1t9fdMJpMhMZGjCYnqw/cGUeu6MNGAHK8PsWolLumVAoVMhrJQCN8eK5Y6NSIionahX4Kxw38nlbwoFRNTvbaK1+uts++P2B/HtMR5p3K6RuehUKjNGlBT59WWjc5DoVCHaJ5H1NI6SmNJovbuohQzBEGAtbz6SfDoOANMggA7AJer7nc8Imo7Op2a70MiCWUlGtFVoYj676TtvtF5Wlr1/MjS0tI6+0pKSmA0GqHT6VrsPCIiIiJqH2qvricIAs6OM7AwTNQO8H1IJK3O8h6UvChlNBqRkZGB/fv319n322+/YcCAAS16HhERERG1b3mVLnx7ohj+U4xgr0+yVoWxqXH17vsmtxRVtVb5awyVTMCfsuourAMAm4ptyK/yNPmaAHBRZgL09awueMDqxF6r84yueX6SCZl6TZ24xeXFBov1jK7ZxxSDgXF1p454gyEsyyk5o2vy5xQdP6fulVUYYqx/5gl/Tu3n58T3U8f9Od1m7hyDbCQvSgHA5MmT8cknn+Do0aPo0aMHAGDTpk04fvw4brvtthY/j4iIiIjarzCAY5VulHv9TTqvp1GHsan17/vN6oTVF2hyLlq5rMFfzo473Nh/hr9ITUyPrzde7PZhd/mZPRnva4pBJur+cubwB874mrFKRb2/nAXD4TO+Jn9O0fFz8glosCjFn1P7+Tnx/dRxf06BJj6YiVbtoih1xx13YNmyZbj55ptx6623wuv14oMPPkD//v0xY8YMAEBeXh527dqFc889F5mZmY0+j4iIiIiiQygcxoYiK9YWViAY7hxfxomIiDqzdlGUiouLw6effornn38er7/+OjQaDSZOnIhHHnkEKpUKALB9+3Y8/vjjeP755yNFqcacR00zZ87T+Pbbr0973LRp0/HEE0+32H1drip4vT6YzebT5rZx444Wu280GD16SIu/3kRERO3RfqsTawrKpU6DiIiI2ki7KEoBQPfu3fH+++83uH/mzJmYOXNmk8+jppkxYyaGDBkW2d6z5xcsX74El156GQYNGhyJp6dntNg9s7MP4LHHHsJTTz0Ls3lIi12XiIiIosvZcQYkaFSoUghwVDa9Z4mhnl4lf5iWmQh/qOmr28prNWM/2flJsehnatpqz3+IUcjrjfcxxSBWdWZf0TNi6k5hAYBkrRpXdqt/Ks7pJGnrf9CrkglnfE3+nKLj55SRaKieS1sP/pzaz8+J76eO+3NSy2Vwn9Hdoku7KUpR+zBgwEAMGDAwsh0MBrF8+RIMGDAQU6Zc1Cr3PHbsCMrK6q6iSERERJ1Pqk5dveKQqmVXHBoQ1/By1Gequ7Hlm9Cm6NRI0alb9JpGlQKDE4wtek2FTNbi1wT4c2pPP6fEhIZX/uLPqf38nE6FP6fo/jkp5bIWv1d71Dn+lERERERERERE1K5wpBSdsX37fsUHH7yD/fv3AQAGDDgbd9xxD/r1GxA5prKyEvPmvYydO7fDaq1AYmISxo+fhFtuuQNqtRoffvguPvqoevrl/fffjZSUVCxevKLROezcuR3//e8CHDiwH1VVTpjNcRg5cjTuued+GAzVKyaEw2HMn/8B1qz5FsXFFsTE6DFs2HDceecsJCenRK61dOliLFmyGPn5eVCrNRg0aDDuuOMedO/eI3KMx+PB/Pkf4PvvV6OsrBQJCYmYMGEybrnlDmg09Q8dPZVfftmJDz54B0ePHkYwGETPnr1w44234PzzL2jwnBMnjmPWrNthNMbizTffR1xc/atYEBEREREREbVnLEq1kL9vP3xG56X9//buPSrqOv/j+IuZAdIB1s1AVxPvgkJ5+Sma5npZ9XQsjxeSbT25hZlWuiRuarnqb10rsxRviCaIpJTXREHUo9l6qbykpqZGmrhbiaA/VBJRYJj5/eGJPSxmgjjfgXk+zpljznyGecvh5fn66vv9fGt7a2xI4G1fiz35vbIKCiv1dd/u1LJS77tbX365XxMmjFPLlq304osvqaioSFu2pGns2FGaO3dR6f5T06a9rjNnvtXQoX9S3boP6cSJ40pOTlJeXp4mTfqbevTordzc/1NqaoqGD49U69Yhdz3DwYP79dprUXrkkbZ64YXRMplMOnhwv1JTU2Sz2TR58v9KklasSNTy5fEaMiRCLVq0UFZWltatW62MjG+0YsUamc1mbd++VbNnv6MnnnhS4eF/1NWrV7R27SpFRY3W6tUb5ePjo+LiYkVHv6ITJ75W//4DFBzcRqdOndCHH36g48ePauHC92Wx3H2kvv/+X5o4cZxatgzSqFFj5HA4lJa2URMnjldsbLzatm1X7j3Z2dkaP36satf20fz5iymkAAAAAADVFqUUKsxut+u992aqdesQxcYuldl8a8O68PA/KjJymObNe0/Ll3+kK1cu69Chg3rllVc1bNhwSdKAAYPkcDiUlXVektSiRUuFhj6q1NQUderUWR063P1G52vWfKSAgHqaNy9Onp6ekqTBg5/W6NGR2rXr09JSaseOberSpavGjXut9L0BAfW0cePHys6+oIYNH9b27VvVtGkzTZkyvXRNy5attGjRAmVmfqdHH22n9PRN+vrr44qKGq+IiGGln9e0aTPFxS1QWtpGDR789F3Pv3fvbt24cUNvvz1bderUkST16dNPL7/8gs6cyShXSl25ckXR0a9IkhYsWKyAgMptsgcAAAAAgCtgTylU2OnT3yor67y6d++pa9eu6erVq7p69aoKCwvVrVt3nTlzWhcv5shq9VGtWrWVkrJeu3bt1I0bt+4dMHny/2r+/Lh7nuPdd+dq2bKVpYWUJOXlXZXVatWNGwWlz/n7B+jIkUNau3aVLl++dZvpQYPClZT0UeldBP396+n77/+txMSlunAhS5L02GOPKzl5rR59tJ0k6bPP9shqtWrIkIgycwwd+idZrVbt3burQvP7+98qlebOnaWMjG8kSb/5TR2tXZuip59+pszagoLreu21KGVnX9C8eXH63e8aVOizAAAAAABwNZwphQo7f/5HSVJc3HzFxc2/7ZqLF3MUEFBPEyZM1rvvvqkpUybJy8tL7dp1UI8evfXEE0/K2/ve7rBgNpuVlXVe8fFL9K9/Zer8+R916dLFcuvGjBmnSZOitWDBHC1cGKOgoNZ6/PHfa8CAQapb9yFJUmTkSJ08eVyJiUuVmLhUTZo0K13zc3F14UKWGjRoWO4SPU9PTzVo0FDZ2RcqNH/v3n20Z88/tXPnDu3cuUN16z6kxx7rpqeeGqDQ0HZl1u7e/U+ZTCbZ7XZlZHyjwMDGFfosAAAAAABcDaVUFbkfezj90l5TRrPbSyRJI0e+pJCQR267JjCwiSSpX78n1KXLY9qzZ5f27ftMhw4d1MGD+5WSsl5LlybJy8ur0nNs3PixZs+eqcDAxmrbtr169OitNm1C9fHHa7R9+9bSdS1atNTq1Sk6cOALff75Xh04sE8JCUu0evWHev/95WrcuIkCAuopKWmVjhw5pM8+2639+/cpOTlJa9Z8qJiYWLVv/z9yOBx3+J44ypyxdTcsFovefHOWzp79Trt3f6r9+7/Qli1p2rx5k0aPHqvhw58vXevj46NZs+Zp5szpio2dqy5dusrPr+pvRwoAAAAAgLNw+R4q7OdLx2rXrq1OnTqXefj4+Mhut8vb21sFBQU6duyoJA899dRAvfXWe9q8+RMNHfonfffdaR08uL/SMxQWFio2dq46dOioFSvWaNKkKRo69BmFhITqypXLpetKSkr07bcZysnJ1uOP99CkSVO0YUO6pk+fqfz8a0pNTZEknT37nc6dy1THjmEaN26CVq/eoLi4BDkcDq1fv1qSVL9+A2VlnZfNZiszS3FxsS5cyKrwHk/Z2dk6duyomjdvoREjRmnp0iStX5+mRo0CtWrVyjJru3fvqbZt22ncuAm6fDlXS5YsrMy3DQAAAAAAl0EphQoLDm6junUf0rp1a1RQ8J+9m65fz9e0aW/o7beny2w2KzPzrMaMGanNmzeVrvH09FSrVkGSJLP51o+fyXTr1zudifTfCgsLdfPmTTVqFFjmcrozZ77V0aNHJEk2m012u11RUaO1YMGcMu8PCQktM8PUqZM0Y8Y0lZSUlK5p1SpYnp6eMplubeTerVt3Xb9+XRs2rC3ztVJS1qmg4Lq6du1+1/NL0sqViRo37uUylxwGBNRTQEBA6ffkv3Xp0lW//30vpaVt1NdfH6vQ5wEAAAAA4Eq4fA8VZrFYFB09QdOmvaERI57VgAED5eXlrbS0FGVnX9C0aTNksVgUEhKqtm3bKz4+ThcvZqt585a6eDFH69evUePGTdSxY2dJUp06v5UkpaSsV25urvr1e+JXZ/Dz81ObNqFKT0+V1WpVYGBjZWaeVVraJnl43Cp0CgoK5Ofnp6effkYffLBMb7zxmjp3fkyFhTeVmpqiBx54QE8+OVCSNGzYcL3zzpt69dWX1atXH0kObdu2RUVFRaV31BswYJC2bdushQvn6uzZ7xQc3EYZGae0ZUua2rQJ1YABgyr0fRwyJELbtqVrzJgXNXDgEPn6+unw4S91+PAhjRz50i++Lyrqrzp4cJ/ee+9tJSZ+WG6PKwAAAAAAqgP+NYtK6dnzD4qJidWKFYlKSlomk8lDzZo11zvvxKhbt1tnDHl4eGjmzNlKTIzX55/vVWpqinx9fdWzZ2+NHPlS6R5MHTuGqXfvvvr88z06fPhL9ejR6642QZ8x4x0tXDhX6empKioqVv369fXss8+pSZOm+tvfJurIkS/Vs+cf9MILo+Xn56f09FQtWnRAZrNZjzzSVlOnzlDjxk0kSU89NUhms0Xr16/R0qWLZLfbFRTUWrNnz1eHDh0lSV5eXpo/f7ESE+P16ac7tH37Vvn7B2j48Eg999yICpdDzZu30Lx5cVq+PF6rViWroOC6GjUK1F//OlGDBg39xffVr19fzz33gt5/f5FWrUous/cUAAAAAADVhYejItdM1XC5ufmy23/525Gd/W/Vr89dz3B/WSwm2Wx2p3wWP9PA7fn7++rSpWtGjwG4LTIIGI8cAsaqKRk0mTxUt67PL7/uxFkAAAAAAAAASVy+B1Spn37KU3Fx8a+u8/T0lJ/fb5wwEQAAAAAArolSCqhCkydPKL373520a9dBsbFLnTARAAAAAACuiVIKqEJjx0br2rWffnWdr6+fE6YBAAAAAMB1UUoBVSg4uLXRIwAAAAAAUC2w0TkAAAAAAACcjlKqghwOh9EjAFWCn2UAAAAAgJEopSrAZDLLbi8xegygSpSUlMhkMhs9BgAAAADATVFKVYDF4qXCwhtGjwFUiZs3r8vbu5bRYwAAAAAA3BSlVAX4+tZRfn6eiopucukTqiWHwyGbzab8/DwVFFyT1cpdAAEAAAAAxuDuexXg6eklX9/f6qefLstmKzZ6HNRQJpNJdrv9Pn59s7y9a+nBB+vJYvG8b58DAAAAAMCdUEpVUK1aVtWqZTV6DNRg/v6+unTpmtFjAAAAAABwX3H5HgAAAAAAAJyOUgoAAAAAAABORykFAAAAAAAAp6OUAgAAAAAAgNNRSgEAAAAAAMDpKKUAAAAAAADgdBajB3AlJpOH0SMAkvhZBFwBOQSMRQYB45FDwFg1IYO/9mfwcDgcDifNAgAAAAAAAEji8j0AAAAAAAAYgFIKAAAAAAAATkcpBQAAAAAAAKejlAIAAAAAAIDTUUoBAAAAAADA6SilAAAAAAAA4HSUUgAAAAAAAHA6SikAAAAAAAA4HaUUUI2lpqaqX79+at++vcLDw3X48GGjRwLc1tatWzVs2DCjxwDcDtkDnI9jUMB4y5YtU2hoqNq3b1/6yMnJMXqsCqOUAqqpzMxM/f3vf1dMTIy++uorRURE6NVXXzV6LMDtlJSUKDExURMmTJDD4TB6HMBtkD3AGByDAq7hm2++0aRJk/TVV1+VPurVq2f0WBVGKQVUU82aNdPevXsVGhqqoqIi5eXlqU6dOkaPBbid2bNna+fOnRo1apTRowBuhewBxuAYFHANGRkZCg4ONnqMe2YxegAAlWe1WnXq1CmFh4fLbDZryZIlRo8EuJ3IyEgFBARow4YNRo8CuBWyBxiHY1DAWIWFhTp37pwSEhIUFRUlf39/jR8/Xj179jR6tArjTCnAhW3btk1BQUHlHq+//nrpmpYtW+r48eOaPn26oqKilJuba+DEQM3zazkMCAgweELAPZE9wFgcgwLGyc3NVfv27fXss89q9+7dio6OVnR0tM6ePWv0aBXGmVKAC+vbt6+OHDlS7nlPT89y/x0eHq6kpCQdOHBA/fv3d9qMQE13NzkEAMDdcAwKGKdBgwZKTk4u/X2vXr3UuXNn7dmzR82bNzdwsorjTCnAhZnNZlmt1nIPLy8v7dq1q9w+GkVFRfL19TVoWqBmulMOAQBwNxyDAsbLyMhQfHx8meeKioqq5fEppRRQTYWEhOjIkSPatm2bbDabkpOTZbPZ1KlTJ6NHAwAAQA3FMShgPKvVqkWLFmnnzp2y2+3aunWrjh49qr59+xo9WoVRSgH32ZQpUzR8+PDbvvbDDz9o7NixCgsLU1hYmCZOnKjLly/f1df19/dXbGys4uLi1LlzZ+3YsUPx8fF64IEHqnJ8oEa4XzkEUHnkEjDOveSPY1CgatxLDhs1aqQ5c+YoJiZGHTp00JIlS7R48eJqud+ih8PhcBg9BFBTrVu3TlOmTFFYWJhWrlxZ5rUrV64oPDxcRUVF+vOf/6ySkhItW7ZMDRs21Lp166rlqZeAKyKHgOshl4BxyB9gPHL4H2x0DtwHJSUlWrx4sWJjY39xTVJSkrKzs5WWlla6GV3btm0VGRmpjRs3KiIiwlnjAjUSOQRcD7kEjEP+AOORw/K4fA+oYoWFhRo8eLAWLlyogQMHql69erddl56errCwsDJ3R+jatauaNm2q9PR0Z40L1EjkEHA95BIwDvkDjEcOb49SCqhihYWFys/P19y5czVr1ixZLOVPSMzLy9MPP/ygkJCQcq+FhIToxIkTzhgVqLHIIeB6yCVgHPIHGI8c3h6X7wFVzMfHR9u3b7/tXzI/y8nJkaTbtuP+/v7Kz8/XtWvXuLUuUEnkEHA95BIwDvkDjEcOb48zpYAqZjKZ7vgXjSRdv35dklSrVq1yr3l7e0uSCgoKqn44wE2QQ8D1kEvAOOQPMB45vD1KKcAAdrv9V9eYTMQTuJ/IIeB6yCVgHPIHGM8dc1iz/jRANWG1WiXduq74v/383M9rANwf5BBwPeQSMA75A4znjjmklAIM0KBBA0nSpUuXyr128eJF+fn5qXbt2s4eC3Ar5BBwPeQSMA75A4znjjmklAIM4Ofnp4cfflgnT54s99qpU6cUGhpqwFSAeyGHgOshl4BxyB9gPHfMIaUUYJB+/fpp3759Onv2bOlzX3zxhc6dO6f+/fsbOBngPsgh4HrIJWAc8gcYz91yeOet3wHcNy+++KI2bdqk559/XiNGjFBhYaESEhIUEhKigQMHGj0e4BbIIeB6yCVgHPIHGM/dcsiZUoBBHnzwQSUnJys4OFgLFizQBx98oD59+ighIUFeXl5Gjwe4BXIIuB5yCRiH/AHGc7ccejgcDofRQwAAAAAAAMC9cKYUAAAAAAAAnI5SCgAAAAAAAE5HKQUAAAAAAACno5QCAAAAAACA01FKAQAAAAAAwOkopQAAAAAAAOB0lFIAAAAAAABwOkopAAAAAAAAOB2lFAAAQDXxzDPPKDQ0VBEREfrxxx+NHgcAAOCeUEoBAABUE5GRkRo0aJCOHTumxMREo8cBAAC4Jx4Oh8Nh9BAAAAC4OzabTZ06dVKrVq20Zs0ao8cBAACoNM6UAgAAqEYsFotatWql06dPi/+3CAAAqjNKKQAAgGrE4XCouLhYBQUF7CsFAACqNUopAACAauSjjz7SyZMnJUmnT582eBoAAIDKo5QCAACoJnJychQTEyN/f39JlFIAAKB6o5QCAACoJmbMmCGbzaYFCxZIopQCAADVG6UUAABANfDJJ59ox44d+stf/qIOHTqobt26OnPmjNFjAQAAVBqlFAAAgIvLz8/XP/7xD4WEhCgyMlKSFBQUpHPnzqmoqMjg6QAAACqHUgoAAMDFzZkzR7m5uXrrrbdkNpsl3SqlbDabMjMzDZ4OAACgciilAAAAXNjRo0e1evVqjRgxQq1bty59PigoSBL7SgEAgOqLUgoAAMBFFRcXa+rUqQoMDNTYsWPLvEYpBQAAqjuL0QMAAADg9hISEnTmzBmtWLFC3t7eZV5r0aKFLBYLpRQAAKi2PBwOh8PoIQAAAAAAAOBeuHwPAAAAAAAATkcpBQAAAAAAAKejlAIAAAAAAIDTUUoBAAAAAADA6SilAAAAAAAA4HSUUgAAAAAAAHA6SikAAAAAAAA4HaUUAAAAAAAAnI5SCgAAAAAAAE5HKQUAAAAAAACn+3/ov5EOAvuUaQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_177_0.png" } }, "output_type": "display_data" } ], "source": [ "fig = plt.figure(figsize=(20, 14))\n", "\n", "colors = {\n", " \"ols_sk\": \"r\",\n", " \"ridge_sk\": \"y\",\n", " \"lasso_sk\": \"c\"\n", "}\n", "\n", "for key in train_errors:\n", " plt.semilogx(\n", " lambdas,\n", " train_errors[key],\n", " colors[key],\n", " label=\"Train {0}\".format(key),\n", " linewidth=4.0\n", " )\n", "\n", "for key in test_errors:\n", " plt.semilogx(\n", " lambdas,\n", " test_errors[key],\n", " colors[key] + \"--\",\n", " label=\"Test {0}\".format(key),\n", " linewidth=4.0\n", " )\n", "plt.legend(loc=\"best\", fontsize=18)\n", "plt.xlabel(r\"$\\lambda$\", fontsize=18)\n", "plt.ylabel(r\"$R^2$\", fontsize=18)\n", "plt.tick_params(labelsize=18)\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From the above figure we can see that LASSO with $\\lambda = 10^{-2}$\n", "achieves a very good accuracy on the test set. This by far surpasses the\n", "other models for all values of $\\lambda$.\n", "\n", "\n", "\n", "\n", "\n", "\n", "## Exercises and Projects\n", "\n", "\n", "\n", "The main aim of this project is to study in more detail various\n", "regression methods, including the Ordinary Least Squares (OLS) method,\n", "The total score is **100** points. Each subtask has its own final score.\n", "\n", "\n", "We will first study how to fit polynomials to a specific\n", "two-dimensional function called [Franke's\n", "function](http://www.dtic.mil/dtic/tr/fulltext/u2/a081688.pdf). This\n", "is a function which has been widely used when testing various\n", "interpolation and fitting algorithms. Furthermore, after having\n", "established the model and the method, we will employ resamling\n", "techniques such as cross-validation and/or bootstrap in order to perform a\n", "proper assessment of our models. We will also study in detail the\n", "so-called Bias-Variance trade off.\n", "\n", "\n", "The Franke function, which is a weighted sum of four exponentials reads as follows" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\begin{align*}\n", "f(x,y) &= \\frac{3}{4}\\exp{\\left(-\\frac{(9x-2)^2}{4} - \\frac{(9y-2)^2}{4}\\right)}+\\frac{3}{4}\\exp{\\left(-\\frac{(9x+1)^2}{49}- \\frac{(9y+1)}{10}\\right)} \\\\\n", "&+\\frac{1}{2}\\exp{\\left(-\\frac{(9x-7)^2}{4} - \\frac{(9y-3)^2}{4}\\right)} -\\frac{1}{5}\\exp{\\left(-(9x-4)^2 - (9y-7)^2\\right) }.\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The function will be defined for $x,y\\in [0,1]$. Our first step will\n", "be to perform an OLS regression analysis of this function, trying out\n", "a polynomial fit with an $x$ and $y$ dependence of the form $[x, y,\n", "x^2, y^2, xy, \\dots]$. We will also include bootstrap first as\n", "a resampling technique. After that we will include the cross-validation technique. As in homeworks 1 and 2, we can use a uniform\n", "distribution to set up the arrays of values for $x$ and $y$, or as in\n", "the example below just a set of fixed \n", "values for $x$ and $y$ with a given step\n", "size. We will fit a\n", "function (for example a polynomial) of $x$ and $y$. Thereafter we\n", "will repeat much of the same procedure using the Ridge and Lasso\n", "regression methods, introducing thus a dependence on the bias\n", "(penalty) $\\lambda$.\n", "\n", "Finally we are going to use (real) digital terrain data and try to\n", "reproduce these data using the same methods. We will also try to go\n", "beyond the second-order polynomials metioned above and explore \n", "which polynomial fits the data best.\n", "\n", "\n", "The Python code for the Franke function is included here (it performs also a three-dimensional plot of it)" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/3980313467.py:9: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().\n", " ax = fig.gca(projection='3d')\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAScAAADsCAYAAAA2AmCCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABxOUlEQVR4nO19ebQcZbX9rrl6vlMGIBiUKUACimFy+UCUSSQogjwICBIIkxiFnyAiEMMUwQgqyhRmVBANGKMQ80DDQ3EAnoAgkwIyJrljz0N1Vf3++PqrrqquqftW33tJaq/VK+nb1dXVQ+0653z77MPouq4jQoQIEaYY2Mk+gAgRIkRwQkROESJEmJKIyClChAhTEhE5RYgQYUoiIqcIESJMSfBeDw4O5ifqOCJE2GIxbVpqsg9hSiKKnCJEiDAlEZFThAgRpiQicooQIcKUREROESJEmJKIyClChAhTEhE5RYgQYUoiIqcIESJMSUTkFCFChCmJiJwiRIgwJRGRU4QIEaYkInKKECHClEREThEiRJiSiMgpQoQIUxIROUWIEGFKIiKnCBEiTEl4+jlF6B50XQPAAAAYhpncg4kQYQoiIqdJAMcBqqpCVVUAAMNwYBgWNJCNyCpChIicJhQMQ4iJYRjIcgzVqoJarQZNU8EwKt0KDMMaZBURVYQtFRE5TRBYlhATz3OIx2VUqzXEYhIymSQ0TTOIqlpV0NubRD5fhKJUEZFVhC0VETlNAPjGpyzLEgSBR6FQQa1WRb2uNh7nIIoiYrEYMpkUAAbxeBzlcgW1mgJNq4NwEoOIrCJsKYjIqYugaRzLMkgkYtA0Dfl8EfYB8PW6inq9jFKpDADo7++BrutIJGLo6UlBVTUjqmqSFSWliKwibJ6IyKlLYFlyE0UesZiEcrmGWk0J9FxN01CpVI3tBYGHKAoNskqjXq+jVlNQrdagKAp0XYWu22tWXOP/EVlFeH8iIqeQQaMlAIjHZfA8i0KhDFXVWrYLCkWpQ1HqKBZJZCWKAkRRQCqVAM/zFrIihBaRVYT3PyJyChE0WuI4FolEDPV6HblcyWXrzkmiVlMsUZgTWVGiomQlCCxYlkGlUgPDMBFZRZjyiMgpJBCJACBJAmRZRKlUhaLUJ+S1zWTFMIyRBqbTCXAcB0WpQ9d16DpQLlcBwBRZsaaaVURWEaYOInIaJ8zapXhcBssyyOdL0DTd/8ldgK7rBlkVCuS4RFFAPC5DEHjMmNEPRTHXrOoANFORPiKrCFMDETmNAzzPQJYF1OsqEgkZtZqCYrE27v3qens1Ke996ahWa+A4FvW6ikKhZKSBmUwKHMeiVqsbq4H1ekRWEaYGInLqEFQiIEkCJElAsVgxdEt+EAQe8XgC9bpqRDnWlTwd46lJeYGSVbVKSJRlGYOsSOTHGsdTrdYa7ykiqwgTj4ic2oQ5jUskZDAMg2y2CN0uXnJBPC6B53nk80WoqgZRFJBMxsHzPBSFkMJEnvCapqNSqaFSoWTFmsgqBpZlGkRFFOx2smJZDrrORGQVIXRE5NQG6GociXwkVKsKGEYIREwsyyKRkKFpGnK5IjRNNWo/QLOQLUkiBIFHT0/KEF02SaH7oBqrSqVqHDclq2QyBoBBrVYzIqt0OoFyuWJEYlFkFSEsROQUAGbtUixGW1DK0DQdkiT4Pj+IENNcyOY4FpVKDbquQxQF9PSkbOmWYjgadBt2suI4M1nFwbJElc6yLKrVWkPPFaWBEcaPiJx8QKMlc+RDW1CCnGhUiJnPl6Fpmu/2ABondrM2lM9bI5hEIgYAlnqVXeTZLaiqhnK5akgS+vrSqNdViKKIZDJhIdlqtdZ4z1ayYlmisYrIKoIXInLyANUuuUc+7oVrlmWRTMqo1zUPIWZweEcwVlIgNiwTI2XQdZiU6QDHcYbWK51OQNN0SxqoaZqNpDmwLImqIrKKYEZETg4wp3GJhAyOc4583Jb8RVFALCaiXK6iVutEiOm/WmePYIizQZMUSLOw00pgd6GqKkolFaVSxXZcEtJpsz0MIStdV6FpzRQ1Mt6LQBGRkw3ttaC0opM0LgwQZwM7KYiIx2VkMknoum6sDpJm4XBel2EYzwUBt+OKxchxmUmURFbEeE8URei6hnpdj8hqC0VETiaMpwWlUzLrFuw2LKlUAoLAG7IF2iw80ZGV/bhoqw0lUWoP0xSHlk0uodbIKiKqzRsROYEQUjwuAGAgCBwYJngLCsMwHZGZJIkNDVG9JfIIUyFOQQrmdeTzRQCwrLjxPGfIGmo1ZcJ6AoFWxwVKVrIsQJIkyLJksYeJvKy2HGzx5ETtc1mWhSQJFkGiHxgGxnJ/O2SWSMgAyImYyQg2f6ZuEkPz+OzNwpSs0umkpaWlVlMmTGMFNMmKNCwrRirqZg+j65FL6OaKLZqcqH2uJImNWkw9MDHRNA4A8vlgaRzPc0gkZFSrCkqlChSFkEOTGIiLgK7rYBhyok4EMdhbWihZSZKzStwsW/CrOXUKQvzt28NEZLX5YIskp6Z2idjn6rqOcrkKng82Y9ScxtEoyA+yLLr24NldBHp6kmBZ1iK+dCKGbsFKVkVDYyVJVCXeShrhg4E50qMIYg9jjUIj4733K7Y4cqJFb9qCUqmQbnxR5OG3fE9qU6Q5lqZxVIzpFj00e/CAXK7kG2XQVTVFUVEuVxyJwdzWElTPNJ7z0FljJUIURfA8h97etCV6CUNjRSMnL7jZw5gdF5zIqrc3hZGRMURkNbWxxZCT1T6XNN/a7XO9fps0jSPFW3Ma534GcRyLZDKGWq1u6JGCwFwQbyUGq8jRrmfyIr+wsi+isaqgXK5AEHqQyxXB85yhZWrnmNzRPlG4pad2exhB4MFxfMMeJoqspiq2CHIyt6A0VdtFyzZe548kiZBloVEnClYDaqZ+wZ8TBK0iR75RGyJL8e42LN0CA1Ulr0mPqVUeoBrRXlCNFV1sGA+c7WFIet3ba+9XpM3VZrKK+gInE5s9OTVbUPxU262qbLMtiltKRqMc+pBT6tdN1Ot11Outww/sEgGWZbtSuHaCkzxAksQWjVX3VyetoK00uq5jaGisbXuYiKwmFpstOdlbUPzIwq4toitrtZqCcrm9FbzW1M/9GMPmCzeJgCxLYFkGPM+F6mwQ5PykZEW3FwTBpYhNnTi7twoINPfbrj1M5LgwsdgsycmsXWqHLCi8VtZaQSIuSeLbEmL6/47H/0M3pzWapoFlGSiK2uJs0BzW2dlKYDscYpYHtBaxqcaKRHqkoN3RIbnCi/T87GEAWNLAiKy6i82OnCSJhSyLUFUNkhS8TkRX3eiKWJCVNfq8eFxqS1VOorKmk0ArOXQn/dJ1OK66NR0ENNNKYNBC9vhOQDfb4HSaaJlSqaSxMhmGNUw70aq9udq8SulnD9Pbm0G5rEBRJq6/cnNDMGHP+wAMQ0SVDMOA53nwPIdcrhS4GM1xjGkIQDnQiclx5Oqu63pgYpIkEYmEjGKxjNHRLGq1OmRZxMBAD/r7e5BKJRoGboEOe1ygq25jY3ls2jSCsbE8VFVDPC5j2rRe9PdnkEzGIYp+hnrhkSm1DdY0DaOjOQwPj6FWq0EUBfT1ZTBtWi8ymWQjTW3/5zuedJF+XtlsHoODIxgdzaFeJ9/ftGm9mDatD+l0Es8//xxeeOH5wGlzoVDAEUccgbffftt1m/Xr1+OTn/ykcf/dd9/FCSecgMMOOwxnnXUWisWi63Pfr9gsIie6GkfrRICOQqEc+Pk0jSNhfbD6Ei2w05PJD+ZCeS5XgqapUFUN9TpZkgeaK2+SJEOWJUiSOKGFY6emXHMhm3qcm/vvulE3IyAkomn26MVqDaNpmiX69COeMFMtN3uYX/zifqxfvx7xeBwf/vCeuOCCizEwMOC4jxdeeB7f+95VeOONN1xfZ2hoCFdffbXlb8uWLcPChQvxmc98Bj/+8Y9xww034Pzzzw/tvU0FvK8jJxotsSyxz00kZJRKlcAnC03jeJ5ri8wSCRmSRPvpNN8fPMexSKWI8Vo+754u0lW3SqWKQqGEQoHUydLpJKZP70NPT7phycIFPtbxQFHqKBRKGBnJYnBwBKVSGSzLmo4n1Xh/4R+PG+mpqtoS7dXrKmIxGu2R6FOSRNfvpVurltQaZtmyK/HnP/8ZF1/8bXzoQzuAZd1/H2vWPIilS5di+vTprttcfPHFOOecc4z7iqLgySefxKGHHgoA+PznP4+1a9eG90amCN63kZO9BUXT9IZ2iQmUEpn73CqVWuMH5P1EopMitihB++lohNXJBOBm4bhkuB+IohhqMTsoSG2IvB7QrA1JkoienhQYxryyFcbxOLev2GH3i7JrrOy6r+6tApqOnGHQ19eHAw/8FPbf/5Oe21544SWYNi3l+vjdd9+NXXfdFXvssYfxt9HRUSSTSfCN5tBp06Zh48aN4Rz8FML7kpzcWlAAgGF030gmFiONvubVOD+bkqZVb3B3y/EYz9nfg65bRzjZi9nhqLKDg6azmQwwNDRqWtkyF4ubZNXu8XSaLto1VlbdFw9NU6HrOgSB71qqTL+68X4Fr7zyCtatW4c777wTGzZsMP5OGsNbNXmbG95X5GRtQSEnvr0FxesHQdM4XdcDr8aZX8uNZOy/CxrNqaqXf7j7j8npx2eHuYUEcFeKT1S9yt02WDa1tJCLSJhOnH6w676SyRhEUTSsYRSlbhNdhofxXiDWrl2LwcFBHH300VAUBZs2bcLChQtx1113IZ/PQ1VVcByHwcFBz7Tw/Yr3DTlZ7XO9Bwc4ndiCwCEely1RlhlOhBBkSAH5ATafR6M5rzFQ3YCbUpyehJqmQVU18Dw3ITYsbumWWSVOm4WdyLMb6Rdtqq7VFOTzRYvGKszxW2FFMUuWLMGSJUsAAG+//TZOOukk/OxnPwMAzJ8/Hw899BAWLFiAhx9+GAsWLAjlNacS3hfkZLfP9Uut7D9q86y5oDqZILPm7OjkdboFe70qnSb+Rz09aaM+FEa9KuiJaE63rCpx5wime2lKs5Zl1liR8VtMQ8dE2lnMdbR2NFb02Dvl1sWLF2PJkiWYN2+e6zZLly7FhRdeiHvuuQc/ufkWiL09nb3YFAaje1yeBgfzE3ksLXBqQSkWy756okwmiVyu0OiNi0HTtECreD09SYyNFQzXgmLRn2QSCRmKUockCdB1oFgsB/pRyjIx8HdaAk8kyIlBV+vCgHmftD5ETfY6E18SzJjRj40bh8d1bGZzO1Ek9sWkn7EQivDSjHY+W7NCnAxc0C11PTdSFwQefX0ZDA0VAh2TV0E8KJ5c8EVU3guvKC5vNQN7rbkntP11gikbOYkiC45joOto9Li104JCCp6xmOSaxrkhlYobI8ODgUEsJqNaDWbvS+sehBDqjjWibsOpPiRJ4qTVq5zM7aZN6zVm8pEBo02V+HiaqdtJF1sV4vaZfM6kzrLdXxG0QxkdhDK0wX/DgODkyVcZTTlyotESOWH4RrQUpMfNshfEYlJb6ZUgkI+C/tiCQJZFCAKHSkUJRExm+UK5XIWi1BxrRDzPQVVV48ff7dHjVHzZeiykMXei/cR1XYeu68hmSeTRJAW7X1QNTgMivMAwTMdprLPoUrRMjtmwYQMefXQd9tprPnp6ZgZKTwuFAo477jjcdNNNmDVrluWxRx55BNdffz10XcesWbOwfPlyZDIZPPXUU7jqqqvQ29uL2267DVycBZ8MT2/GxSNysoAWvRmGgSwLnlYlzs9nGmE7kMsFX76ntSJdh+Hr7QWGgfE6QWs29IrrRrTmGlEiEYMgCOB53tA00cihk2X5duHUmEv9xM31qm6tutllBHZSaGqZYshk+LY8rMJUtDsp6jds2IiVK2/BZZctQ3//AI455r/xxS+e4roPL4V4oVDAt7/9baxatQozZszAD37wA1x//fW4+OKL8c1vfhM33ngjdthhBwAAx7PghPAIhQtoWd1NTBlyamqXyKqaoqhtGY6ZV8kkSQgkxGwKODXk80Wk0wn4if/MtijlchWxmOT7WrReFryZWIeqqsYYp2bk0FyWN9vidhP2lMtcr0qnE2AYMhMvXH2V93fgrWWy+ojbLwQkkukOuStKHdtuOxurV6/B8PAQ/vCHx5FIJD2fQxXiF1xwgcP+FCxduhQzZswAAOy8885Ys2YNAOChhx6CIJABqYIggBM5cGKIkVOI++oUk05O5qK3ebWLRE9ioH3YbXclya9R1VnA6Yd21d5NKYIaWFHuBKfIodnzxjVWurzTrrBWv8x1GIZhMG1aLzRNC7Ve1W504+Zh5TQgwuzn1C2wLItZs2bhyCOP8t3WSyHe29uLgw8+GABQqVRwyy234Itf/CIAstr58ssv45vf/CYeeOABcBILXg6RnKQtPHJqbUEhEYyuI1D/mJvtrp+I0W3J3+5qaUY8LoPj3AzrnHRVNJILrigPCqt5m3va1c4AhE5AP6tiMex6VefRTas8wDoggmVZo1WpW20/YWu08vk8vvzlL2POnDk46qgm4e2888544IEHAAAsx4ENseeS7UK/ZLuYNHJq2uc664n8CKYTHVJzpUw3SNAKKqhsPmAmQKfoh86YM2O8eqd2Jv46p12tbS0sy0yI9ipIvcqv/y7MupDdQK6vL416XR2nh5U3CDmNezcAgE2bNuHUU0/Fvvvui4suuggAUK1W8fjjj+Oggw4ytmOFcGtObIj76hQTTk40jSMmbe5tIV4nqF87iROx0ZUyrzTO/prtqr3N03ydya/7sLe1NMd7k7qXIPCu9Zj24R3huNWrRFFsuDSETwxBjrlSaablbjKK8dTzwpISqKqKM888E5/+9Kdx9tlnG3/neR7Lli3DzJkzMXfuXPI3kYMuhXc681tazclun1uv1z16z1oHDgRpXXECbfQtFCqBl+XbjX7sLgd+4DgOHCc1HBTtP+TW994paApIU416vQ5RFI16jHkVsNvOBkH1Vaqqdo2o7Bc8NxmFvbju1mbj/BoMxvNRUoX4hg0b8M9//hOqquJ3v/sdAGDu3Lm48sorcd111+HSSy9Ff38/Vq5cuVmmdROmEG+O/hYCeW2Tlos4sllSS/KfntJEPC6jXq9DUVQjkikWK74/+GQyhmpVgSyLgVXlskwGS3JccD0W/QxqNQWiKLSsvhFiFJDLBVMYB0EyGYeu68ZJCJCLBFFlk9W3diMZjmPR25vB0NBoKMfYHMYgNupVim+hv1309/cgm80H2h9JS3mjpYX6m/v13vX2plGvayiVgkVeYSjE/33OSagPhqcQ56fNwPY/uju0/XV0DN1+AbN2iY5ZCmJpa07NmtNTgmqXdHAch1hMChzJAOQYm83BwZ4jCDxYNrh/OH0vhUIJlUoVmqa3rL6pqgZdJ++hmwJMu8ukUyTj1ZwbNszpVCIRQ6lUabte5Yd2itVuHlZ2Ty37MU2EZ5QdvMgBUVrXxs55BskkOdnbSXnMSKcTbZm7ASRlon14Qa+45GrNolyuBpIW0EI56bdSfYnJbKNify/21bdkMg5JEtDXlwYwcQJMpxRHkprNueYleZLqdkczRFOvbtSrxlNspx5WVk8t8zHp+PnPf45isYCPfGQ+PvCB7X19zovFAk455XhHdfiLL76Ib33rWygWi5g/fz6WLVuGbDaLRYsWGdvE43Hce++9m2Va15WSPLXPBXSjFlMsVtoiJqpVKperKJWCjfKmq3EMwwQu+JqtehWlHij6EQQeqVQMlYoSSCbA8xxSqThqteZUXDfoum6Q1eDgKEZGsg0TfevQAdpu000Qa5EShofHMDQ0ikqlBlHkjUEDqVQc3RmB1Bp50HpVNus8jKGvL9jnEmZU03pMOZRKJTz44IM48cTjsWDBwXj55Zdcn//CC8/j7LNPc/UPP//883HppZfid7/7HXRdx/3334/+/n6sXr0aq1evxoMPPoiZM2cCAFiRAyfyod3YDiInr0ENL774Ij7/+c/j0EMPxbe+9S1jPqEXQienZtGbafx4yZiloBEMaQ2RIYqkyz/ImwAIAaTTcaOoGgQcxyKdjkNV1cAe4rGYZPTtkSu297K/LJNpK4VCpa0GZApVJbWvsbEcNm0aMfRc6XTC8PGmGqxugkQNVWSzhQZp5lCvq2BZps1JLf4IEt0Qr6gyRkfJ50JdBlIp8rn09qYRj8da9HLdVIjX6yoWLPgcfvvb3+I3v1mL88+/CFtttbXr9mvWPIjzzvuGo1HcO++8g0qlgg9/+MMAnH3CV61aBVkmNVUaOYV2azNyevbZZ3H88ce3RbR+CPXy62SfG4tJga9U9om5mYx/OwnQOgRTkkT4TQ1qFuabc+28iMZdJuC+stZu20oT7vukURXVEZl9xcnAyqZPUzdTQFVVUalUIUkihofHOlKtu6N9AnFSiZN6VdpSrwK6NS2mCaKcn4FPfOJTnttdeOElro9t2rQJ06ZNM+7bfcJVVcVNN92EW265hbwmx4IJ8QJF9/Xee++11D3T6TTS6bTlb/fff79rG44T0f7whz/EwoULPY8h9NzA3koiy2KgUNppFc9LsQ1YCcNKADq8gkL38eTOpMBxHJLJ4DUzOgih3UnD7cLuK06tcZ0K2t1Bk0S8Veuw1Kv8UufxijC99FUAMDDQ01V9VRipo32qj1279/jjj2O77bbD9ttvDwDgBB4IIWql4Brp8QknnIB33nnH8tg555yDr3zlK5a/XXnlla778iNaN4RKTqkU9ec2t5L4EQwQj8ccV7yaX0jrk70Iwy0C6qTXzc9NwF242XnbSjsKcTPs1rh2G5Z6XTXmvIWhFnf7XoOp1psRnsOeEWbqRWtDlUoNktSLsbG8xeokDOElRdMFc3zHP3PmTAwODhr3h4aGLOnfI488gsMPP7z5uhwLNkQnARo5/fSnP3WMnNqBH9G6IVRyKhQqaP1RtbaEUFCCIUZywYreAJmaK8uCh66oNQIKMj3FTgruEZYzYjERgiAEEm6SRQMu9L47M8w2LKlUHCzLGQZulEAmQp3tplp3cxEIs33FDLpfu9WJk/Cy06EH47Xopdhmm20gSRKefvppfPSjH8Xq1aux//77G48/88wzWLx4cfN1ORZMiCtslJy22mqrce/Lj2jdECo5aZreUutxiwJonchc87HDzrDmKMurjmN/TXuq6Q0GLMs05tM599M5PSeZJLoXr6GZFBzHGj1+LMtMiEqbLi5QmYCbpqlaVQIvQnQKq5+41UWAYcgwBk3TwLJsqJ+HWxTuX6+aOBW92T98xYoVuPjii1EoFLDbbrvhpJNOMrZ76623jJU6AGAFIdS0jhXC25cf0bqh6+vRrQRDxZj2OpHTc5skYy2WB4uyzG4HQWx3ifCRhSzH2+inIx5U1WoN5bJ/PYpGcKVSFaVSCUCzqJ1KkZRHVVXPKbFhwKxpogMHJElEJkM0TUEIM4zaipOLAHU0GBjoCbn/zv94nSyDqYrePI/PbdGhk7Tu97//vfH/lStXGv+fM2cOfvnLXzo+59lnn7XcZzm26T0UAtgQiutBidYNobavUDW4GYlEzFAYt9t/Rr3DWZYJ1PJCwfNEHc6yTFt+TclkDBzHoVAo4Y8zPwpdaX40H9v0VMv2hGjkhtWLf4Rl7tfTNA2K0locJjUrGZJEirfUcqRarY2rTuTUvuKG5glJolvqbGAvrouigEQihtHRXMfH5QTzEAIa4YmiAEHgx6Va53kOmUwKw8NjHR8bteaVpObxmOtVPM9jYKAn0Lmzbt1a3H33bdB1DSeffDJOOOEEy+OPPfYYVqxYAQDYaaedcNlllyGRSODdd9/F+eefj+HhYXz0ox/FlVdeidyNl0LLjXT8vuxg031In3VZaPvrBBPQ+EssRezL/YGeqcMwnAta9wFg9EHl8+VA7R8Mw+CJrecDALgYB13RLMQEAE9MJ49TkqJEUy5XfLU97bgVKEodlUoVDMMim80bfW8TKRVobWvhG35I1hpRt1Icc0TmrFoXPVTrwfbbKbzqVW+//RYuuOACzJs3D7vt9hF89KN7IZl0dsIcHNyElStvwG233YOtturDcccdh3322cew3c3lcsbopx122AErV67Eddddh4svvhjLli3DwoUL8ZnPfAZ330363xiBBxuiMJeZAJGvH0I9AreVG9JI296UXbLCQ65MQQWS5v49mh554W877Nt8vRgJiXVFa9wnIaBatv7gn5g+H717pDH/iSeQyxXBcZznygNNR2u14K07zRVOc4rhLhXodp3IPLDTXCOKx8mCQSaTDNXcjmHg2tVvrg/ROXOUOIOkXGETuvl4JCmBQw45BH/729/wwAMPYPbsD+K225zHKz311N+w557zkU5nEI/Hceihh2Lt2rU455xzAABvvPEGtt56a4OsDjzwQJx22mn4xje+gSeffBI//vGPAQCHHXYYeW9suGkd4ycUnAB0ubeOnEztEAzQXL5XlHrgKMucMioKGbnkhdeP+XTL3ygxmcHF2BaCGn02h78f8HHs9NAj8FryNteXwmicdZIKSJKInp4kGIadEAdMM2HWagricQm1Wt0iExi/vzkDwD8qo6p1aiTnRN5my+BurQJScByHY489Dqeeeio2bsyiXHb/zQ8NDaK/f8C4P336dDz33HPG/e222w4bNmzASy+9hDlz5uDhhx/G0NAQRkdHkUwmwTdsPvr6+gAADM8BIfbWMSHuq1N0jZxoGtfunDHz8r0oCoH0EHYtEh3K6IbXj/k0Bp8JbvNBo6je3az6jlcOPwi7/O73Tk8x0r7gTgrtgxIALSSbZ6pN1BAEXUeLTGC8SvFOScRd50UK7HSScDfdHoiXkw5BECB4rHj5aX/S6TSuvvpqXHLJJdA0DcceeywEQXDVCDEcD/DhrbAx3GaW1gGtqm1RFAKtPNH0p10Hgna0SDRaciImp6jJjIGP9AAA1Jp1uxcP/SQyszKYccsqAO26YYYnNrTXiewkoWka6nUVHMd21a7XTSne3oircD4Xu2Wws9tDuJFm0Cbo6dNn4Nln/27cHxwctGh/VFXFzJkz8Ytf/AIA8Nxzz2HbbbdFX18f8vk8VFUFx3EYGRnB9OnTN8vIKdTEkudZo/m2UCg3fnz+XzopKMZQqVgdCLyUpKRpN9HwA7cry1u1VV7E5AdKTADAic4f2cbTjwbHsUilmo3Efld/PzuN8YD035UwMpLF4OBoI6Jk0deXwcBAL1KpBCRJHJejgF+EQ1PAXK5oc1iQPB0WupF+URfQWs3u9iBiYKAX/f09oTQu08jJD/Pn742nn34So6OjKJfLWLdunUX7wzAMFi1ahI0bN0LXddx55504/PDDIQgC5s+fj4ceeggAsG7dOvIEKiUI7baZ1Zzqdb3FCtdPqt6Jj7hfLcf+mub60sDuPZ7vYfBp/+VYTmQtEVT27SwyszJ499SjMOuO1b71JXIVlxsiQ7NSuwZdDy7vDwo6B69Wq6NUKrdMqvWa8+aN9iIc6rBgH3HVnNRCIhmGYdvab+CjNRXE3Y5lvCrxoEX3adOmY/His7FkyRnQdQ3HHHMMdt99d4s26LLLLsNpp52GWq2G/fbbD6eeeioAYOnSpbjwwgtx4403Gh7iDCeEnNaFt6+OjyFsm17eRnekVYFHsWj1MTJPNXHzOBJFHjzPWx4nam8OhULFs5bT05PE2FgBlSvOwHvPvInUzDRy72YDvQczQZmjJjPs6R0AS3rnBqoOr1YV5PNFy4qTIPDGIoAo8hgeDna8QUAiOt1YAqcgE3CamqZ2CuuyLEGSBGNs+HhgdliIxaQGaVdDlU2Y9VN+x0LrVTS6DOrCSQaNcigUgrdjhWHTW/31jdCL4enNmEQa0pFnhba/TjDhCnEgWJ8beW4zcrKqvYPVpMzE1A6mfZSsgOgeP0J79ASQCAqnH+1KULQp2BzxmXUzhCjIycnzPKZN623YxNZCUUc7RSO6DsOKNp+nEg6xpbDuNCQzzPTL7LBABi9UDdvk8IZ1BotGrap1LxdO+r3YX2NiLXoBABzfGhmMd3+TjAk5gtY+Ny7QKhYltk6m82684IsobBgziClo1GRG3wf7MfL6sOvjTgQFkPqTnaBoU7D9fd+2vrfl+WcenAfLEhGmc/9bGGOdnOHUpGtPvZrTc7sDhoGhSndyWGhXfGlGJwTfOjWGN+xgMpmU8b0MD49A1+vIZHp890nV4fV6HYsWnWJRh7/44ou48MILjfsjIyPIZDL4zW9+g6eeegpXXXUVFEXBNttsg2uuuYa4BHBcuIQyBWx6u57WcRyLeFxGsVg2PLT9rGqb++IMvVKxGHxAZTwu481Fn7X8rRNy6p1NiMOLoIDWFC8zKwMAmHHLqoazJ1mpKhabRfJbHk0AIHU61/2qOk7/1Jhxn6YZ1vQrmLc47duzp3XtwJyCUm8kTdNQKBRbIojxoK8vg0Kh5CqBsB9HUIcF8hmogX9/QWDuS7z66uW4//77sfPOczB//j747/9eiN7evpbnDA5uwtlnn4bbbrsHgiDinHNOw7XXXmsILs0ol8v4whe+gG9/+9uYP38+Dj74YNx4443YYYcdsGLFCvT09OC0005Ddd1d0MvhTUtiYilIh5wc2v46QeiRk72Ireu6Ydnb7nTeWEwCwzDI5QqBfvg09cteNf5cmRIT4B9BUWy774eM/+feHsbG04/GjvethaLUjasu0CQmgAyBAKwkpaq6adse4/+nf2rM0DU10y8J6XTQtGe8BmhW0SNd3aIRxHisRlqO1OMLdxJfBnFYCHMSb/M4m5KF008/G0cccQT++Mc/4cknn8K//vUK9tpr35bnmNXhAFrU4WbcfPPN2GuvvTB/PmmfeuihhyAIAhRFwcaNG5tDEfiQ07ow99XpIXT7BWSZEEw+Xwoc+VC1d61WhyAEq2vQ54xecSZGXrYarHcSNdnhRVCzP74jNJeTsVJp1tVuf4z2WbW+IZ5nLKTkhFse7YGq6jjrkGxL+mVPe5q1qu6pxUnzch35fNFmfZI2uV+2PzWm3VqWW/+d3WGBtgR1CzzPY88998Quu8zDF7/oXhfzU4dT5PN53H///VizZo3xN0EQ8PLLL+OUU04Bz/M4//zzyQObYVrXNXIyF7ABBCYmc4OwpmkQhHjg54xecea4jpnCHDX5Ydbe2zv+PT2rH7GBHtS+fx5u3+0Gy2M0WgKaEZOZmPwev3Fdxvj/WYcQ4jUby1G1OK0V0Tl43SzUOhWRiSlgO5EdxfiKytb+u6bDgiw33Q3IcYSXilIEkRIEdYb89a9/jYMOOgj9/f2Wv++888544okncN999+Hyyy/H9ddfDzAcwIZIKMxmSk72AnZPj3NnthlORnIMw3ja1ZrrOZSY7FFTfCCF+EDrUi3Lc3j3//7Txrvyjp5YnnONnha9cDbu3v0GOD1M3x/PM471J4Zq4VyypBvXZaBqOs45rLmM7KQWT6USkGUJsZhsiWjG11rjTiJ2HZFzQdv5GMJcBTR/FgyTbkRQjKsL53gRhJz81OEUjzzyCM444wzjfrVaxeOPP46DDjoIAHDkkUcargQ6z0EPNa3bDMkpHm96FgWNltyM5PwU4sZzrmnN1Xu3J/aiVQ8NztZ7zgaAtkjKTFBuURNFeWgMsYEeAMAXnz0T9+xxEwCgrsIxhbPXn1RTOsYLpmhKaX38R2ubcgkzUQHNlpJKhUxMocVkWiR38moKgnZIxBrZmd0EEtB1zSKZCNtD3Hy8NIJrdeFsTmkZT/N0kLrW/Pl74/bbb8Ho6ChisRjWrVuHyy+/3LKNrut44YUX8JGPfMT4G8/zWLZsGWbOnIm5c+fi4YcfNkSYYHlyCwth7qtDhH4E9To5AcxfECUZpyuKKAqIxYIbydmfU732q83XqasGKXnBPhl16z1nGwQVJKXr+2A/4tN6HPfrFj0xjRBI1ah/eGshvLkt+ZfnGNSdSExgDIJyAiUqO0kBTsVkq1dTWOZ2XnAraNNjYBgGsiyjWq2GKpmw/wbd9EzBBzG4vYb3NmZ1uKLUcdxxx7aow0dGRhqrgJLxPI7jcN111+HSSy+FqqqYMWOGMfVEZ1jooaZ1k9++ErqUwMkNM51OoFBobcylrSvEGdL5MKjSu/kcopOq3nUZGFFC6bU3AADSQB8qGwYtz3WLmrzGNpeHg6lsnciJwk5QNHrSdQ13zrvJ9XnVqjsZUJJyIyVzFFV3aGK+ZKHuu4xuVmlLktAQaNYcxYZAcMV1O2AYBgMDvajVahAEAQwT3jj2/v4eZLP5wIRHBzFIkhjIYYFhGMyY0Y+RkYLv4oYZYSjES0+vhV4N8XuQ4oh/9LDQ9tcJJih202EO1c0DBPzU3lSaQLr9SYG9du/VBjFJA0RHYiemThCb3ofY9D6MvPiG53b9c0k6V97oLy8wg2FYnF26BjfEWwcPaqoOoRFNKR7aJ7eoiWs4P1Srzife5T9jAPD42gL343Oag0eW6M1SAUISqqp2xbyNRNlALlc0PN1JYV3uoLBuRbvtik6DGMyz+NxIs5ueUW7QWQ56qGndZlhzcoJZ+9Sc6xa0xqGD55sFduaBawHAQkxO6CRqoujbZTtfgvKCPb0z157Kr72B0w/6De4cPgIAUKm0/pLtJGVP7Wj9yUxS9XrDwZMjjzlduRVFxXcfIAsI53/eX4xpHYLQan9CbVjCJ6nmhcyvsN7e5JrOj9NpFp95NbJcLuPaa6/Drrvugt12+wj6+6d57s9LIQ4AP/rRj7Bq1SpjRtyxxx6Lww47DIsWLTK2yefzkGUZDz30EHSGCzmt22LISTdEle0WywGSyhWLFfCrvw8AYJJJCzGFETXZ4UZQNGoCgNiM/sDRU2zH7VF+9d8AAP61f0JNEnISGkSjeNSQvGpPSk1zJCJKUgBQqbRGGO2QFOB0cnJIpxMQBNIDGGZrjVeh3UkyYZ5c41XcD1OEaSdNTVORy+WwYsUKjIyM4MMf3hPXX3+z44KO2T+cKsTN/uEA8Pzzz+Paa6+1FMQBYPXq1Y3XI0MRTj6ZqLh1joMe6Zy84eYjHo9LUFU9gAEbAZUJkJpG2UJM9XffDfegQVI6O4JEUG4EZY6e+vb7qOWx8mtv4IwPXYObk830TjCtxpXKrcTNN8jGTFJKzT9aqlZVz3FFl/+E/AQuObG9FElV1cYyPJno0rQMNq96dVYnCtqg6ySZaB2OaU5DuyfCZFkOF154EXp70/jzn5/G0NCQ6/sIohB//vnncfPNN+Odd97BXnvthW984xuW4viqVasQi8UMWYHOcNDDjHa2hMiJ5znDCsRum+IGs0yApgxMY4qFnZicoqbxpHRhwk5KsR23BxINzVcuizMKVoKiAYfYMLSrOTQV8xwDRdGNNM4MM0k51Z7s6Vel1IwuOiEps7Fa0zK46FgnamecU6cE4lwjEo00lP6tW5NriC6PwU47zcFOO7lv56cQLxaL2GWXXXD++edj9uzZuPDCC3HDDTfg3HPPBUAuDDfddBNuuKEp7tVZHlqIHkzsFJASdHW9UJZFJBJy40cTLNQXRR7JZMy4Iuq6DvH3twV6bny7WYhvNws9u+6Anl1bmyg7Qd8u2xn/N6d0ZsRm9Lf8rWfvj3hXYBtXTVFgHFc4AUJSosl5U1F0I/3jeRY87/z1uRXFgeYJZCYmMy7/CW8QVTC0nuQ05Rkby2HTphFD25ROJzF9eh8ymSRkWXK0bw5LgNl04Sw0nC/JKmw8LmPatF709WWQSMRaXDjHg6AumH4K8UQigZUrV2L77bcHz/NYtGgRHnvsMePxxx9/HNtttx123nnn5j5YLvTbZKMr9Gj3EZdlMdBKiZOdivrb5tXBHjWxsoT4drNc92cnKEbgkX3h1ZbtnFI6M8ZbILegWGhGT+kMThm9GivTJHqiDrHtaCEpQRHTvmBRT7HQtDtxs3f/1kpykl25ePyOnM12kpLrIIbm6lt3BJjUnZUOALXPvwtjJHzQhQE/hfi7776LJ554AscccwwAQl68Sf39yCOP4PDDD7fsU2M4aJtZWhd65NT00dYMH3E/21mGIa4FxIGgZPw4Yo/d0bItP22acXOC7pI20CGBmd12RGa3Hdt9W9j68E94Pm6Onnr2NhUxA7DyKQO/sdwXTWPvzamdIDCW2pQZtZrmGkmZYSYmANB0cjOjUm6y47dW6gZR2dFJlEPrRGNjeWzaNIJcjqTg5qgKCN9f3U4cNAUdHh7D0NAYarUaRFHEwEBPx37iQcnJ7B9eqVRa/MNlWcZ3v/tdvPXWW9B1HT/96U9x8MEHG48/88wzhksBhcby0FghxNvkp3VdKIjrKJetam9dh+sEFuomYDeSsxBTLAHkxiyEpOXHZ0ma2W1HxyjKC9IHtkX1zbc8t7EQkxts0RNACOqOIbKCRz86hgEkiW0RZ5pX+OzREs+zjvUoAMiOlsELzldETSdRlJmYzHCOpMYf5dA6EU39ZFmCKAoYGOjpSKHtBi/i8JtcE1Q1H3Q1MIhC/LLLLsNZZ50FRVGw55574pRTTjGe/9Zbb2HmzJmWfeosBy3EVIyZAmld6ApxoNUKRpIEsCxr8TQC3EeUxx67gxASRW6s5TWcyMkvanJDbdBfDiB/cDsA8CUnafYHnB9okVc3m6HrA9sAAH4y1lRIlhz0T2aSKpebn5dT4dxOUNnRpmTAjaByI0UAgBwXHR+vlJpR1/e+SvQ9iqK0fK/jAcex6O3NYGhotEWh3an7pX2/7cBJNe82VTiZjEEQRORy7ZnZhaEQ3/jvF6HWw5tPyPECZmy/S2j76wQT0kBjT+uoTEAQeORyJSsx/e1+KzF1GWxPH+QdvdM8SkwAiZ7cIM3bHUj3BHvhYnNFkR96BwBwYk/TtycuM4jL1mhTklhIEmshJgCWormxz0bBPDtathATANQdFicoMQFWEnL72//7QRWLLx8ONJOwPTSjMbryRsdbkaZlHn19PRgY6EEqlQicenUqFKWqeVpYHxvLoV7XWgrrgI5CoRDoNdatW4sTT/wCjjvuKKxadb/rduvXr8cnP/lJ4/67776LE044AYcddhjOOussFIvN70xl+dBvk40JIqdm6YXUpIhJfD5f8v8yA0ZN44UfQY0bAbU7ZoICmiRVqWjGTZY5yLI1+rGv7AHA6HDJlTzMBGUmJopKqWYQkhNZUZy2bBhfXVHEV1cUxz3zDXCvY1GSyGYLGBwcwdhYHpqmIZmMY/r0PvT2phGPy+BcxYPhFNrJROEyRkfNK5EsVq9+AAce+Al88Ysn4vbbb8HQ0JDj86kA84YbbsUdd/wMv/71g/jXv/7Vst3Q0BCuvvpqy9+WLVuGhQsXYu3atZg7d65FSqAxrFEUD+c2+Y2/Exo5UZlApVJ1TAVif7NdRRyIyfU1Okjp2B7rKl1QgnKKnqR5uzfvjCN6AoAjPvg8JBGo1nTjloi3flV2ggIISY0OlzA63OxZDEJQbvAiJjvOumoMX11RNBFFJz+vYCRSr6uWqKpUqoDnefT1pTEw0It02jo0tBuDOoFmYf0zn/ksbr75FsyfPx9/+tPj+Mtf/ui4vVmAGYvFcOCBn8LatWtbtrv44ostokxFUfDkk0/i0EMPBQB8/vOftzxPD5WYQhZ0dogJ663jOBayLLlOXWkhJhd0I2oyQ95xR1RebRbKzSmdGUGK4y3wOUPKyemIFTahZ/R1HLwN8D/vEK+eXIE8hxJUsdT8/ChBVSoqRkcb/Wcyj5qtZYVlnTU4G/4ziHjK3W20mG1GVYmMNd2uu1wQTvrWe9B1Hfdes63hbBDUL6oTErG21sBxaKh50Gs3wPM89t57b8yfvxcWLXL3sLcLMPv7B/D6669Ytrn77rux6667Yo899jD+Njo6imQyaUgKpk2bho0bNxqPq+CghijBYLCZkpM5jSOzxyQAjLFsbIcfMTFHfsUyppxCenDFeA/VEXaC8oMlaqJI9/hGfpWdyHKwIsQcHz94m+fxP+/MRTpJPkwnkioUnAkiCEENvUuGh5byJUeCMhOT+X4ik3AlJqCp8D7+giZ5/2T5NraWEueidhiTjltnAQqIxWJdmAVoBcv6+8D7CTBfeeUVrFu3DnfeeSc2bNjguh1g/aw0sNBCJBRtYpIqT3Q1cmoOKlAgCO71iPLexxr/t48aF0UePJx/sNWjvm6yX1FRKlUh3v8d4/F2Ujo75B13BHxWP8YTPVFiAgBBKRsEZY6exno/iHlbj+K1kQyGsyzEhoRgLKcauzIjlZKQzzdJ3IugKDFRuBGUE4rZIqS45PiY28l+4jebaeudl89EIhFzjarCJAw6NJQWxAuFIiRJRCzmPamlE3Ri0TsyMmwRYK5duxaDg4M4+uijoSgKNm3ahIULF+Kuu+5CPp+HqqrgOK5FuKmDhRZi5KRvzuREequITEBVtUDFUupaYE797KOmzHDSSNWOJcMIOY4F/8trOj5+bdrWYN/zt+9l9z8EGN3g/KBD9FTeaT4Ynx9wOTkdRYk4cs7U3gH6AIDooYazLHrSnEFQqRT5CvP5euN+MIJyQilP6lTxVLwlajKjrtRRzzb3aU/3/PClS8jndcM3MxY9UTfTLzqJ18mChTQsJ2EexV6t1jpqWG7Xonf9+t9j+fIrjceXLFmCJUuWAADefvttnHTSSfjZz37WeO58PPTQQ1iwYAF+9atfWYSbqs6hDX87X7D6ZprWJZMyWJY1BhUA3uE6bXfRdbS4Fripy83k52TRoes6pJOXIpcrtp3+adO2Jv9uNTsQQbULnWFaCMocPZlRZ0WDoN4rpNCfaRIUAAtJBSGojW82V5EYH5IKCkpkXKOx2i2qsuPs5WRyjFbX8OOLMo0ZeHJDhNnbVq3KH85RTbNhGQ6tNapBVEEaloNETnYB5oIFn20RYLph6dKluPDCC3HjjTdiq622wrXXXms8poEJNRXTXLKViURXRJiyzLeEyHa7XQqOY5FMxlCtKoYDoxk8z0GWRRQKTa1OIkHIj7bHOIGke3HkckUkEjLqP7mi+ZhPSkfJydjehaCU7ZoiNcEtegKM6Km8k7XlwE5QdnKi0ROv1TDIzISqcXivkMJgllxTRnPk+XGZwTsbmp+d1riEmgkKAN56xXqMbuQ0tomkfJmBVj91t1oT5+H44EVWmk0seuuyAUiShGKx1JhuLLal0nYDlRnk8+4RoR1UKS5JYqD+u+nT+5DLlaE42CR7IQwR5j9fH0bNpTOgE4g8i10/2NrQPpHoSuRUq9Udu+ztaA4qqLi6FpgjJzoLT1WJRsoLNB1Mp+OkrnDU1wF0VkQPI4KyE5MT7NFTojqKotSLOisCOsCxKmRexbQMMJjl0ZtmMJrTUaro6O0RMDrWmNXGMRgdan4+o0PkohBPx1Ey2SLrmu5KUACQHRp1JKh2IIhCCwGxjR5A+98B4LSlzcjuR99MG/YnVKWdSMQdprX4oxMRprW1ptXYzjwCnb7GZIGkdeG9vqpPfs2pK5GTkwVIJpOwpHlBhhuQfbFIJmWUSlXHHjw38DxnWK/Yt4/94XbX59mjJuM4bORkjpoo3KKn3LZ7QFCcr9hBoycAGNVJxDdUSmEwLyIVU1uiKAAGSTkRVMnBs91MUDRqsiMz0Nt21CR0KMq0v85Nl1ojXept3k5UlUzGoeu6MRV4vKBRlSgK4DgW3/zmRRgY6MdHPrI3dtttd88FILNF7xe+cDzOPPNUy+OPPPIIrr/+eui6jlmzZmH58uXIZDJ46qmncNVVV0FRFGyzzTa4+uqrkcmQWuSz/86FHjntsX3af8MG1qxZgxtvvBH1eh0nn3xyINth+zZ2TBg50Qksuk56kDRNC2Q+x7LEsQCAa33JDlo3YBjGMZUE3AnKjZwAK0E5kRPgTFC5bYlexYmgnIrj7RAUAIzkSGrnRFBAk6T8CMqNmCgSGef0w4mcwiImM+wkBTR73whRiK5RVSoVh6aFR052/Pa3v8ajjz6CZ555BqlUCr/4xRrEYq01xMHBTTj77NMMi94zz1yEH/7w+4ZFb6FQwGGHHYZVq1ZhxowZ+MEPfoB8Po+LL74YBx98MG688UbssMMOWLFiBViWxXnnnQcA+L9/5VH1sHpuF5LAYM8dgqWbGzduxPHHH48HHngAoijiuOOOw7XXXmuxHT7zzDNxxhlntNgOe2ECG2h08DyHWEwKHP0AZAWPYRhks8VAYXk8LjVqCyWk0+6rSOUDF7UQlBcxAZ2ld5SY3BCkOF5ECgmQC4WqceBYFQPxPAD64xExktMhNfp1R7MqJIlcHcZGq5DjAiolBb0DSYwOFVrSO4CkeF7Q6iryw2Mtf+9xMNqrlEgqxrdp5KaqKhiWcT2WxUubrqcrlxGHCreJMWZHgWq1BoZhoevjkwp44YgjPotTT12E119/B2+88R/Isuy4nd2ilyrEqRpcURQsXboUM2bMAEBGj69ZQ1qaHnroIQiCAEVRsHHjRovZnKaz0EKUYGiNFPG9995rWUFNp9NGBAQATzzxBPbdd1/09PQA6Mx22AkTmFiSAQfFYiUQMdGIiX7efsRk9oTK51tn5NkhCBzEz30l8NFTaFvNdo2aAEDpbVpZ2IlJEYIvuW+SPmDcAEJQQyo5IVWNw1glAZ4lYfy0VA19aaCv8XvpzXDozZBopqdXQk+vBDkuQI4L6B0gbgjxtFXTNPLeJqhK+6tilVK55RZLkH3Xlbpx84P5BGBYpqUWZj9BFi8dtJAVhb2tpVKpGNq5RCLWVrNwO6DyjFQqg7lz57nWn5wU4mald29vr+HdVKlUcMsttxg+4YIg4OWXX8YBBxyAv/71r/jMZz5jPE/VmdBvAHDCCSfgU5/6lOV21113Wd7Tpk2bMM1kZzR9+nTLezLbDj/44IPI5XKWvkA3dE0hbkYiIYNhGJRKwdIyu35JFJOe27ut+NGiuP14qAyhUChDPZCM2vGqQ3Ub9uhpQ2oHcFCh2rQmMa6CskquyCmxgnxNBs9qGC1LkEUdI3kWfWkdDUda9GY4jGbJ593TS65SYwC2+kAvXnzqNfRM78XbL79heQ1VUcDZ6iVuU4xlB9EmJSYz/CIoN20TJSjNY3Vu8dJBqA3yu/0q67Rne1RFf3v2qKoTC5aWY/UYImGGn0KcIp/P48tf/jLmzJmDo446yvj7zjvvjCeeeAL33Xcfzj33XNx3332N/bKWwarjhaaRY/rpT3/qGDm1856o7TDFokWLcNFFFxme6G7oauREfKNJ9BNUfUvmosmBIyzaTFwqVR2kCDpg02sQHQ2xajH/IMsHLkIQFPo/iHJyhuc2Su9M13TOLXp6N7kTNqR2wIZUM0/nmNaTNsY163Qpkfy/N1ZFb6yKvpQGWdQhSwxkiUGloiMmsYhJLJSaBqWmIZEQkEgI2HGP7TC2aRTJ3kzLawSJoMIiJoCM2XZyE9BUzSAmlmUd3TFVU1S26KL3sOii9xxfg2EY16iqEwsWp/0HtegdHm6uSNoV4gAMVfjOO+9sjBuvVqt45JFHjG2OPPJIvPzyy8Z9Eu2wId7IebPVVlth1qxZlpudnGbOnInBwWYU62Q7/Mtf/tK4b7cddkPXak6CwCEel43hmfG47OsaQjvZg6RlgLOi3Axz5MQwjFGI95MhBEE5OQOxwkbXx0tiGvHa+JuUOaY1gjKDRlAAIanRsoS+FP0sGnWnrIZ0qvlV5/J1ZHokzD9gZ2THiBbq7+uftexXVRSsXDEbsu78WdVZ60n89WtbiVSQWk90J/kAhZmgFBeJgJmglKqzY8Kii94zUsm7v0scJKhCnKK1VsVDkgRTVNWesV1QF0w/hbiqqjjzzDPx6U9/Gmeffbbxd57nsWzZMsycORNz587Fww8/jD333NN4XNOZFrvl8aCdfX3sYx/D9ddfj5GREcRiMaxbtw6XX3658Ti1Hd5nn30wa9asFtthN3RltU6SeCSTkmV4ZjwuoV7XXIcdeq3g2WUI1KwOAIrFsuuPIp2Oo1CogGHgKfS0HPvzrfYVFIX+D1ruu5HTcB+xXvEiJ/PK3XuJplULyzifCHaCKqsySnWrY2WxZr2fLRNCKpQbPXnZ5r5VDcjlmt/F8fuPQmCa991IidGtx6dw1qImJSknYrLDjajqFovn1i+3bovs7AV0pxrXmlt3Q7FYDuiMEGwF0Aw6LGHEwRvLjnXr1uKee243FOJf+9o5hkJ8w4YN+MpXvmIpds+dOxdXXnmlISVQVRUzZszAZZddZtj1/uF5BeXg7ja+iInAgXODR5Fr1qzBzTffDEVRcMwxx2Dx4sUW1fvvfvc7XH/99Ybt8LJlyyCKzo6rFF2TEnCcNcyNxSRomtaSqrl5iJtBZQiapjd0T2SmnZ89bCoVR62mQJZFo5HY+7hJUb3yp1WOj9vJCXAmKEpOgD9BmYnJOA4HgjKTU05ppoaK5hxVuRFVtkCIalqPhtk9YwCAJGc9oYISkxl2krp0ZfO+WxRkBiUqP7cDOzFZHtf0QMV3Gk0FBY2q7HbB1WrNiNhlWUIyGcfoaPtReRgK8Uf+UQ+dnA6aN7lumF0hJ6DVR1yWyclijlyoHslPv0QjII5jGqliFbWa/48w02hIDTL+3OygwPM8an9+wPK4EzFRmAnKTEwUbgTlRk6AM0GN1txFcXaSchN+x3groZuJSdU5iIw74Yuaty6tzCah6Nar7fLbrN+rF1FRYvGLqHQXkrREXA55ib2w/5PrtnM9Fjc4RVXvvPMe/vrXv2K//faFLLfW8MywCzCPPvpYCzm9+OKL+Na3voVisYj58+dj2bJlyGazWLSoWRPN5/MYHR3F3//edDdY96yKNnwBfREXgUP2mNzm3wkjJ0kSwbKMEe3Q+lKx6K0QB0gEVK+rEEU+ENEAZIWQ53mUSmXfgZ6UJPP5MlS1bih/lb88aGzjRU5Ak6DaIad3hdkQGZfaio2cvIjJDLf6lJmU9EaxM861ChK9yMnYxoGkymxzRdVOUNlaEjfck215DiUqt2jHTFJO25hJym0flKTcVhwpOiEqgERVr7zyIr7+9f+H4eFhbL/9Djj55NPwyU8e1LKtkwDz29++Envv3Vw8OeKII3DFFVfgwx/+MC666CLMnTsXCxcuNB7XNA0nn3wyjj32WCxY0ByI8dDftdDJ6fCPTG4LywS+ug6GMeuRELjwzbIMBIFrWWFz25boo3SoqupbpIzFJEiSgFyuCFVVGmOCKshm86js5l+0M8OJmABSHLfjXWE2AKCmO+f1mqm3KSgxaToLBnrLTeZq0HXGuAGdExMA1FgZNbYpMjQTEwBL7QoAMmIBZ3+xNaIQRAEMw7iqyVmeBcuzrsTDMCwRV3r8hoioU/PsIVRVFccv+TeOX/Jv123cUK/X8aEP7Yi1a3+H++77OQ488CDXKcJOFr3r1z9qPP7OO++gUqngwx/+MIBWK14AWLVqFWKxmIWYgMb8QS3EWxcsjdvFhCWVRPvAIp2Oo1JRDEtVL9BGXwAolaq+S7X2+hXVVznBXFTP5YrQNK1l/5qmo7TLQdA2+f9o3+zb01BxhwVNZ5FVvDVe5m2dIHGtkdl4iMmMGiujrMchovW7pARFoyhKULevKqNSqrUQDiUoc9pHIyezHMH+PK2hwaErfXZNjjliogRlJjP79maCuveHzuPnncBxHHbZZVfMmuX+HCcB5j//+YJx3y5mbLHiVVXcdNNNjgJGTWMwTqmWbX/h7atTTFjkxPMcBIFr6Jf8iYnnOaOgXa+rvh3fohhcH0WjK+pu4ERMln1vvZPv8QJExe0Gc/REoyYKt+hpqJpxLXibMV5i6hRlneiaarr7qos5isqIBSw6OoajPrcVYkkZsWRri4cgCo5OBhS8wIMXeGiqahCTGVQzxXGcaypHFehutSuKdiKpIDonJ7Gi2fjPT8z4+OOPY7vttrOs5FHUVSb022Sja+Rk/p7icQk8zze8nf0V4nai8XLDBEhqRmpG1hl4Ts+jpFepKCiVyv7EJApIp4NFL34oiekWYqKwE9RQtZkGeRFUVRWgaJxxo3AiJicU1Tiquoi8Nr4VIz+CGqpkMFQhZCtyOhYfSxYr7CSlKipURTVIyEnEWVfqYDkOrMsYKNoyw7AsGBfvHkpsLMsYNzvaiZyC6JycBJgDA81IyS5mHBoasogZH3nkERx++OGO+9Z0Ig8J6zYV0rquRk7mfrdisRLI7yYelyDLgo1oWpXeZP9Ev0SEm0WH+pX1eZT08vkyqtVqYxnY/VuIxWQkEjFkswWw091/qIPiNsb/vaInACjX3ZsdKUGZiYnCiaCqamvERUmqoootNx0MimrccjMjr6Ust5zqXuuiUZP1+FsJaqjai6Gq1RMqLdcwXIrh80eQFKeuqBAkwVUbZSYpe1pHSYoSlVN9ipIUw7KuERcAC0n9auWujop0N7Csf+Q0f/7eePrpJzE6StTp69f/Hvvss5/x+DbbbANJkvD0008DAFavXm2x4n3mmWcwf76zL1hdDf822egaOfE8h3Q63pjaWoHfLDIqxGQYBrmctVDuFAGxLGuYfhFHzNZ9mp9HoytSVK87KsrNSCbjEEUBY2N5oy7hRFBmYqJwI6j3lJmOfzfDiZgozATlREwUtCE4CHjWZZWrUTjPqWlPkrLDTFBmUhJYW3+WTFL7zx8xgFRPk+jkpGzczKCk49UOo2u6x1BNUn+iRXQvPHjLLg2r4B709/cgmYy7Frkp2rXo/dKXFuLggw/FrrvOxeLFi/GPf/wDALBixQosX74chx12GEqlEk466STj+W+99ZYhurRDVcO/TTa6JiXo7Y2jXK4a0Y/ZNtcOjmORSMRQqzkruO0aKVr49tM70efxDb8hIuT0TuMYhkE6nWy0uTirfc0FcidyojAXyO3EZNcbGfsrpSHx/houtzqTGzG5pXl+5GRHmss5Rk125GrOPYT2CDBXId9RTWXw6P/mkB9rFTEWRp09ucxRkuqyokcvLF5SAnPt6d4ffgiADl0nFzee5yHLksWqlzYLmy+g/f0ZVCp1VCr+350dYYgwf/IYkA/RqioVA048ILz9dYKuRU65nH/9ByCOgtSx0q+1BGg2BhcKFV8hJsOQ7YMWvjmORU9PCoqieHpN0wjKi5j84JTeDZZIhFKtu1+lS4qAkiKgUudQqY9PJOdGTF7IqWlka941uKFyGjXVxR3TJYISOR17fqQHclyEHBehVBXjJsUlRx/yoE3Fuk+UTCOpn/9oR7AsB4bhwLIsOI6FptVRLBYxMjKKwcFRVKs1yLKIgYFe9PVlkEjEMDY2hnw+H6i3Ttd1/OhH38fChUfjxBO/gOeee8Z120KhgCOOOAJvv/228bcnnngCCxYswCGHHILrrrvO+Htd1UO/TTYmbLXOyRpClkXEYqQHz6u1hD6XTubI50u+I4So6ZiqqiiVKtB1b2ISBB6ZTMoyNsgLXjUoCpreuaVzZoKixEThRVBmmEmq3ajJDW5REwCU6iTd8iMoAIEJimJGj4p99hlA/7QEUr3kZoadpMyREyfw4ByL5w1/bwePKDN+/qOmRo1hGCP9o0RF/q+jVqsgm81h06Zh5PNFsCyD733vanz84x/H6acvws9+drenA8f69Y/iP/95HT/5yS9w1VUrcNVVyxy3f/bZZ3H88cfjjTfeMP5WqVRw0UUX4YYbbsBDDz2E559/Hn/5y18AAFrIKZ02BdK6CVmtc0IyGQPPcw2i8b6y6TqxRgkq3KSFbyIpYHyLlbJMTOtzuWIgmUM7eL3s38dlJyYKO0GVFPc6U6nGI1cRLTeg/XSuHTgR1FDZ+l7cCMoMGj0BhKB2+FAMsz/U40pSVJzpBDNJOfXhUZIyE9V917tfaChREcsWa1SlqgoKhSKWLl2GW2+9Fbvsshsef3w9Nm50n8Tz5z//CZ/61CFgWRYf+MBszJgx09KGQnH//fdj6dKlltW65557DrNnz8a2224LnuexYMECPPbYYwAANeTVuikQOE2kTS8BndCrKKpv4y5AUq1YTGz4P/tHNNRGhbgYaIZlK8MwqNVIV7k5SkskYhAEUvj2K5LbMaNHxsYx/2Oq1AXIvDNJeK3eAU2C6mSyRq4iguecCY2B3rD6tSJI1GRGtpZERnSuCVHUVA4iZ70UC6xqqT+l5ZpBqAAwcxqPDQBKJRHZ4SKkGHmsYBr2Kcaa29dMXa+qUTwn792tWZhhGdz7gw95HnvLcxgGzRVgBgyjI51OYb/99sOuu+7hOxbKSYhpHjtOQX2czHBynPzTn/4EAKjXdYxzYLEFZF+Tq3WaUHLSdR2pVNzwePKDIPCIx6XGqCn/DyqRiIFhgFyuYNSXSqUySqUyOI5tDG0k0gNFqYNlWei6jmw23/EIbC+C2lTuMf7vRlBDJVJclnn3ODpfJSdZXHTpQdOcPxuec35PTGPldKjUWojtj3kTjRMoQdmjJj9QgnprpOmXnomrmNGj4h//IgQFNIvv2eEikrSZ2zaRmBJV2WF4gxtJtUtMdjAMA1EUkEolUSxWA82rcxZiBktgvESamqqHusI2FdK6CSMnSRIbeif/RlyApFqSRKx0GYYxVt6cQKOxel1FoeBcX1JVDeVyFeVyFRzHGsJKhuGQSiUa88dqvimjE5wIykxMFHaCosREHuM8CQogqRtgJSk3YuoEdY3FxiIhmBmJ9ozy/GpQTtHTW2OtK3rvDJFoqq8H2DSkgmUYiCJn2AwDziSlNLoC+IZWqu7QJUBJ6p4V7VmmuEEUeYOYymX3i+2tt96EP/7xfwEAxWLB1wnTDU6Ok/39ZMCEquuhtq9sMWkddSDQNC3QyU8n+lKDOd5jmizHcUgmZcPV0K/wTRTiSVQqFSOtFEWhEVWloaqaQVTt+EoHTfEozMRE4URQNGoyo1TjXaMoCreoKSjsJOWU0pkxVCSPZ2T3mp2ZoJyICQCScaDQCH76ejiMjKno65MxMlKBKHIYHSySWuVYYwR6Q9ekwOYT5kJSYRGTIBBiKpW8iQkATjvtTJx22pkAgN///hH89re/xkEHHYr33nsXb731pucIcjP22GMPvP766/jPf/6DWbNm4Te/+Q1OPPFEAICq6KiHMbW9ATXEfXWKrpGT2RqXLuWnHHynzbBv39yXswk8napRLFagKEqD+LxbUZLJOAqFkiWtNE9tFQTe0rJCjcWCeKBTgnKKmii86k/k8SZBORETBY2i/KItOxgfMawdlKRSUrCFgmxF9CSo14aSEHjbKCxeh1Jvfr9eBNU7LYHRwSJSPYTcKEnJcUKOFdtKq5mkwiMmDuk0IaZSqb2z+MADP4V//vN5nHzy8QCACy+8BLIsY+PGjTj99NOxevVq1+dKkoTvfOc7+MpXvoJqtYoDDjgABxxAxEibY+TUNRGmILBIp2MWh8tkktx36q+jE1ScHAvo1N+cqZ4Qi4kQBKGxeqf6FrOJQlxGLlfwlSE0j4mDKAqQJAEsyxok5lcv+8d7flGGBFnwPl6ZVz3JCQCqipWwM7FGIdgjanIjp7rmXvco1kh0MjPlrPKjUZPlWFwI6u3RWAs5UZgJCmgSFACMjJHvbGSEkM/oYLPmRAnKDDNJ3XGlvzI/KHieQyaTQrlcQ7EYzspuGCLMa35ehcv82I7QkwQu+G/vxZpuo2uRE+2nswoxvSOgUqniUo+y9shRKxRidaL6FrOpaX02m2urpqSqKsplFeVyBSzLQBTFhh1rAorSJCr76+++dRXPvev8xQ4Vyd8rCutJUBtzpMYWl5yP105MQNOKV9eBgZTDMnqbUZMdG/IxV4JqORaPCEqpM64E5QanCAqAYxQFkEiqUqpMeWIKC2pdRz3Eib9qfXJX6oAukpOiqLC3OTmpxP0mqJifZ077yGhz/1aUVCoBXdcxNjY+ryVN01GpVFGpVBurNDxEUUQiEWvUqZoRYiaTxD6xGv5qc9ygxEThRlDZUvODK1WZFoJyIiYK+nEM5a1RlxNZUQSJmig25MnKGiUpp6iJwk5Qb482V+WcCMorvTODEhQAI80DgFRPwiCoOy6fjlpN6WiBwwk8zyKdTqFSmXrEBACqpoU6t04NcaGlU0ywzskaASWT5Meazxd9RJt6o+ctHrjwzbIsMpkkqlVijRImdF1v9FiZ61Qienrkhp6KRFTztlJ9Uzw7QZmJiaJUbbhXukRRQbAp1ySrGenxn1xBoyhKUGZi8oKdoCi+dEAFPM8jlyvgpv9pTYOu+TJ5fwzTC0HgwfM84nEy0Yd+H0HsepxAVndTqFZrKBSmHjEBVEoQ4lDNzVlK4OUS0K4Qkw7gK5UqpquhdytKKpVAsVgOXfHtBEWpG8b3xWKpUSNLgGEY7JtQ8Jd/aS1Rkxl+KR5FqcqA85DEBJVq0ZQRGB9RbcjHwLOdnxB+6d2ndx0FQNJyjuORy5H+tTMOssscmsSr67qlLsjzHERRRDKZAMs2LxyKogT6vDiORSaTQq1Wn7LEBJBpxmGmdfXNOa1zgq7r4DgWshxciCnLojGFlRCT90ksSSTVyueLvqOgwoIsS4jFZGSzzWJ7qVQBy7KQJAEfnyPgV09776OisJ7pGgWdQZeMBf8hekX7b48Q0pzZ0/pd2FM6O8YKhCkH0t6X2ZfeiSHpslDrRFCUlABKTKxBTO2CGBwSIS7LEiEurRvW63WDrJx+VxzHGMSUz7dvYzyRID1xIaZ1U8AJc0LJieNYCAKZoBIkxG7qnYqIxyX09KQ9C9HxuAxJEpHN5tvSKI0HtP0lm21tfyHDEojw85M7Mvj9q+4K6nypGRKJgvOPrFxt/mDsJNWhwN3AhjFyAXAiKT8M5ThfgiqUEIigDtppzPh7MhkHy7LIZsNZhtI0zVQ3BASB6ttkaJpuiap4nkUmk4ai1JHPVzDZrRx+UOsq6gEU6sH3N/laggkjJ2LVywXK/c2F71yuCF0n3kq0XYA09sZNV74aEok4WJbB2FjnrSjtIpUiqUKQ9hdd13HgDln84V+tZnJmYgKAmsK0EJSZmMygJJWQ23/PTrUdSlKpuPcPnUZNFG4E9d5wM/ryIijATkzks83lQlwfN0HX4ZD+CRBFDocc8ilst912+K//2h/77vtf2G678bW5TAQ0VYMW4gV5KtScum6Z0nS4ZBv1Je8rECk+xlGr1VEqlS2Fb1KIriGfL2JkZAyVShWCwKO3NwNB4FGrKYGsgMN4T5kMKcpms4W2yPDAHVrntzmhpjCoNdI8N2JqbguM5hnjZkanCzgbR1hsHGnv5zGU69xfykxMlPS7RUxOqNeJtU6lUsPKlSsxf/58PPTQQzjnnNPbbgifDGiaBlUN7zYV3nNXI6fm6HAF5XINgsB7Diqgjb6lUtXU5+Z+dqmq1hicWUG9XjdWzEiITpb2gwou23lP6XQStVrnq4DmCMoeNdlRC1CHsoMSVG/K/bNzipqcsHGExYw+6w/VHjWZYY6gzFEThVP0ZK4xpVIJo3l7osGy5KKTTO6Krbf+IE455SyoqtqWl/hkoa5oqAfoWQ2+v9B21TG6KMIEUqmYxUrXTYQJNBt9iX+4v+LbqRWF+JU3V2jSaSLOa6cFxQu0L69crqBSGV+B9MAdsvj1c72+2xXLTYKRpdbPzmtNYbBxzvf3tHdsdq89J4Lygl8Nyi29S6XI9+Vk5dxtMAyQTqegqjqy2TJohO/lST6VoGkhp3Wbs85J14GszdaC9tvZQQvf2WwRuq76Cufo6lguV3CsX5lXaIhViohkMma0oFSrtbZX8gRBQCrV2pc3Hhy5+6gnQZmJCQAqVd1CUEEPY3is+f92iYqCpniSuzmEgU1DKjYNAb3usxoMgjrh4xpqtTiZM+fh295NMAyQyaSh63qjRWryT8x2odVVqCGOTNEmZqHbE11N61oV4daTjdajNK1Z+Par3yQSZBKG0+qYE4hVSqXRgsIaqzMcx0FR6g2i8ta8yLKIeDzmSobjwZG7k/DGTlJ2YqKoVMnfnaIoM+ouqy2UqNxG8fk5FFdr3gS1aSj45/PpXUcxNkbSZJZlwHGckTJ3al/TLggxpRq+XiVPs72pDLWuQQ0xrdus21ecYE7raKNvtaqgUqkGbkUBgGw219HSuXUpmWnRvJCpGlaJApUndOKU2Q78oig7RrPkWBLx1nqIGzFRVGs6BkfINtP62qunZHPkBJg+4J/ujGZV9Ga8t0smyaproVCyLe+nDXV3N2qHQDOVI1H++5eYANIHGuZntMXpnGgk1W7hmxahFUVBsRhOKwpd+atWa8ZJQQWcqqqiWlUgCBxYlpswecKRu49CkkTc+b/e1jLlSvNYiiV3knJCtWZ9H4MjWmCCosQEkAjJTlBOUZMbQX1h/qgxgqvQaKBrXd6n9jUJAIwRUYUhrqXExDAMxsbe38QEAHrIUgJ9gnSCXpjQtK45RUUKXPgmwzmTjWXe7qh07ScF1VGxLANVVSHLUtvmc50gFpMhyyJO2C+Hn/65PctbSlKS2P5JNjjSHsFROBGUE+wEdexehJhoA7cb6vU66vW6zWZZBsfxnmJcPxDf77SJmNp6+pREvV539UrvbH+h7apjTGjkFG8YggUtfE9GKwrLMojHY6jVaigWy+B5HpLUNJ+rVsnVO+zaE1Ga01qajuP3GQMA3PvXHst25qjJCbkCOa500mF8ec37uUPD5Mc90N/qI2WOmsygBOVXazITVDqdQr2uolh0JyY7zDbLVjFu0xWiVlMCXEB0pFIkYiKpXOBDmNLQVA1aPUwR5mYeOVGYC98A6Vmq+Zwok9GKQguyZqkAvXoXi2VwHAdJEoym3rDSjGQyAY5jHAWdx+8zZhCUHzFVa83PyYuknFAuN8llaFhxJCg3BC2Cj2ZVnP953fg8O4U5JQfM7qUpAOY2FPv3QialcByLsbESpoDOMDRoqhZqzUkLMNKr2+g6OZlHjZfLVYgih1hMRjIZd3WWJAphdkJbUaiTgZdUgAzoVC1NvXSaS1CXTDvS6WRjpchddEijqNsfc1+bNxOTGbmCinSS842a7DBHUW5RkxljozX09PrrDMZLTE5QlHpD41Y23Eub30sdg4OD2LRpI+bOnQuO4zA2VtysiAkA1LoCNcS0brNfrRMEMnzAXPiuVFRUKjWbs2QcikL65GRZgqZpyGbHZw7XDmj62I5UwNzU245LJgXxp0pCVVXPuosZiw7IehKUG3IFFZWyikzGORoyR012DA0rEATvWtTYaM3414ugvrYgvAUNN7S6lwp49NH/wfLlyzFr1izst9/Hccwxx2GbbWZ19TgmGlpdC1nnNPns3eXIiWk4ENRbVuTszpKyTIZfAkCtRoZhdlLsbBdhpI+tLpnOzcm0xtZsgakFGn1uxqIDSG+emaTcoiaKSoN8slnq1Bk8ZSsWSGoUJCoCvAkqbNM/P9Dv5fjjF2L//Q/Aww+vw5/+9Dj+7/+e2uzIqS/DQa2Hl4r1+UhAJgJdG3AAALEYZ2hVvGA2h6vVFOPkFgTBVX8UBohXEIdcrr3m3XZA34soClBVDYpShySJobTAAISkvMip4hEVZTKCZ9QENMmJwk48NGqyw77d6Z9qbw5eWEgmExBFHmNjpVD8jtatW4u7774N9XodX/jC8Tj66GMtj7/88kv47nevgqIomDFjBi655HKkUt4DDMIYcLA5oqsdjTxPSKevL4NkMg5R5GHXM8myhFQqgVyugGq15ug8IIoCenvTSKeTkGUp0PRfLxCNSzKw3cl4UKspKBRKGBnJolqtIhaTwDDkfcfjsudMviCgkVQnoJGUG+zEBLiTUafbdRP0NxcWMQ0ObsLKlTfghhtuxR13/Ay//vWDeP311yzb/OAHK3DqqWfgrrvuxbbbzsa9994z7tfdUtFVcsrnqxgezqNQqIJhWKRSSfT19SKZTKBWq+COO25DpVLG2FjetdZTqykNosqiUqmC5zn09KSRyaQQi0ltd4zTznPqFTVRoEXaXK6AkZEsCoUiAKJ67+3NGFKCTnDWwXmcdXBrlOsVNQFApVJHLldDLtcekZhrTEG2m4yoKZGIQxQFZLPhEBMAPPXU37DnnvORTmcQi8Vw4IGfwvr1j1q20TQNpRL5XVWrFUjS5I5Xej+j66t1us6gWq2jWq2DYXSIIo+hoY1YsuQrUBQFp5xyCnieg6Kovipd+/BLSaIWKZqhP/KqG1GjevO034mAU8HdqTk5kWg2J3ey8kcJ6kaHAQB2VCrWqIgSVDpN0jGnqMmMdiIoImL1t1gOC4lEvFFHLPq28rSDoaFB9PcPGPf7+wfwz3++YNnmnHPOxXnnnYMf/vB7kOUYbrnlztBef0vDhBrVEKJSsXTpMqRSGaxceSeSyRSSyQT6+nqRSiUgSYKn5xOFotSNdKlYLINlGaTTKfT0pI3GXjMEgUcmk0KpVJpQYorFSPqWzbpHh7Q5eWwsb0SRsZiEvr6etj4TirMOzuOCz3dGBEGjqFy2Yty88K3j1Ua02/xuxpvKeiEej0GWRWSzpVCJCSBRkdlVQ9d1S4mhWq3gO9+5HN///o+xevXvcNRRx+CKK5aGegxbEiZ4NBTBsmVXIR4nxWhiHF+BKPKQJB6JRBzJJINarY5ardoohHvvz6xzsXs5VaukkB6LSROqNAfs/uLBThSn5mQSebk3J9tBtVPnHlnCdb9u7dOzR012bHivMQcu5ZyS2Akpl60gnWkdgXXBMRUoCqAo1kkopIGbMVTdYX0n8XgMsZjUIKbwo7Tp02fg2Wf/btwfGRnGwMA04/5rr/0bkiRh113nAgA++9mjceutN4V+HFsKJsXiL5VK2SIbBrWa2qhRFZHNlqBpGhKJOPr6ehqFcDFQ9EDsVssYHc0hlyuC5zmjbUYUBWPMVLdBpgzzbRGTHXRxIJcrYnTUujjgVHOj9sFmX6Rzjyzh3CODt4mY0c7EEb8ICrB/N8TlIR6Poa8vg1QqAVFsL0I0IxaTDWJSQjT6N2P+/L3x9NNPYnR0FJVKBevX/x777LOf8fg222yLTZs24s033wAAPP74Y5gzZ9euHMuWgK5KCcKAILCQJL6R2jANsWatoRvyfi4dQ57LFRpRCHHbZFnWqFF1I5KikUs3jdNozU0UBUOuIYqCEUG6Yfn93nqlUtG5zkWjKD8SohHUBccE12+ZI0Se5xvasFrgib2xmIx4XEYuV0at1l1n/nXr1uKee26HotSxYMFnccIJJ+PrX1+C0047E3Pm7Io///lPuOmmHwHQ0dPTh29841vYeuttPPcZSQmcMeXJyQxBYBvpnwCOYxtmcVXHHzFt1M3nCy1pIW09EUXRaD3pxB3Tjk5U32FAFHkkkySNpdOIvZqT3QjKjZgoUikpUISUzshtkZMZZk8nqg2jIlanxQ5S04tNCDF1CxE5OeN9RU5m8DwLSeIaRMUZEdULL/wTb775Bo44YkEggqCtJ3Q/na6U0YJ8J6rv8YCOXS+XSZ2KNieLogiWZTwjRDtJ+ZFTKV9BLOG/NH7Fqe29By/Qpl7zYNVKpYZCodCQYMTf18QEROTkhvctOZnB8wwkicff/vZXnHvu1/C5z30OX//6+Y2rbfB6Dx0pLookvVAUJZCNL5UohKX6Dgr6uqWS89h1pwjRiXiX3y8GIiYz3EgqTGKygxLvY4+tx3nnnYePfvSj+PjH98cBBxyMgYEB/x1MUUTk5IzNgpwAYGxsDJ/97KFYuPBEfPWrX0MsRoZ40t62arU9s7jWOojSiEKsK2XUDK9YdCaIboG+btCBC7QJ1ky85ubkS+7wrkTbyQlwJqhukhOFIPB48cUX8PDDv8Njj63HvHl74NJLL+/+C3cJETk5Y7MhJwB4883/4AMfmG3c5zjGKKYTolIb8+zaJSpAFEVLvx9t5E0m4xMuUaC9iPl8yVimbwfm5uTm+yHp37dua93eiZjMoCQ1EcRE/bQKhQoqlbpxoRjPMFW/frk333wD11xzFfL5PPr7+/Htb1+FdLo9p1IvROTkjKk/LbANmIkJAFRVR6mkYHS0hOHhPCqVGgRBQG9vBr29VBDo/xHoOlr6/SSJ9ARqmgaO48bd7xcUTWIqdkRMgPPkZNoW9MOvWcey+BETAJSLE5PKklmFTWICCCmNh5j8+uV0Xcc3vnEeTjzxS7jrrnux44474yc/uXO8byVCAEyKCHMyoGlAuVxHuVwHy5IrMDWLowMNgtrvchwLlmUwNpYDy3KN/aQNu9hqtTutGnSQaNgjquxtQdeeI0MURZxzbfCeuG5HTaJInSuqviLSdmDulwNg9Mt98IMfAkBcBmKxGPbd92MAgJNOOgX5/MRPI94SscWQkxmEqBSUywpYFoY8IRaTjFHmZIWrlQDi8VijoZSIK4kNSvPEbo5ED9bvFxSSRGbnZbOFroxJojCr7Zcv5iDLvTjjqlHP53xvSXebWwkxJVEsVlEuhzsn269f7p133kJfXz+WL78Mr776MmbP/iDOPff8UI8hgjM2q7SuE2gaaefIZssYHi6gVKo2vMRT6OvLGEM88/k8XnvtVcsQAjvISV1quA749/sFBbVXIYZ4E7dkTmtTP/x/Cddtuk1MJI1NolQKn5gA/345VVXx978/jaOOOga33/5TbL31Nrj++utCP44IrdjiyckMXadEVcHwcAHFYhUsy6JcLuIrXzkbP/rRj1AqlQP5P1Gv7NHRpj1KOp1o1LpigZtfm20Z3R3qaQetbeVypNj/vSWSIxE1m5PFcdV+nI+BjgWrolQKn5gA0i83PDxk3Lf3y/X19WPWrA8YbSgHHXQoXnzxhZb9RAgfETm5gBLV8HAexx13POLxBL73vWuRTifR19fjap7nBHu/H6AjmfT3cYrHY5AkoTFteOJmGDWJqYC6bYCZmaS+t0TC6GjWaJ3p7c2Y+iDHR1SEmFIol2tdIybAv19u3rzdMTY2ildffQUA8Kc//S923nlO144nQhOblZSgG1BVFf/zP2vxiU98CrIsG55UksQbqmVaW6rVagCCn5TUx4n2+5nbaBKJZl/gRE2gAcZfdKdaKlHkO14g4HkOmQwhpmKx+9oxv365F154Ht///jUolyuYPn06LrnkMvT29oX2+pGUwBmhkJOfTuTVV1/Gd75zBYrFIj784Y/g61//5oS5A3QXVqJiGBh6oSBWL2aY1dw8z0HTdBSLwQSWYSHs1UC6QCBJzebkalXxrJvxPIt0Oo1qtYZCYfKtficCETk5Y9xpXRBf5csuuwTnnnsB7rvvAei6jjVrfjXel50iMFu95JHLlaHrMKxeUqlkYKM4OmpK08jqX7lcaRjONe1EuglKTNlseDIF6wJBqdEYTepuiUSs5QJF23G2JGKK4I5xk5Ofr/KGDe+hWq1i7tx5AIDDD1+AP/zhkfG+7BSE1ZMqlyu3eFKRorH7HqiTQi5XRKVSRTZbwOhozpjnZy0+h3fkZmLq1mogGT/erLvpuo5kkng5/eMfz+LXv/4VKpUSarV6REwRAIRATk46kU2bNgV+fHNFraaiUKgZ5nmqqiGRiFnM86hPnK7rLSZxFE3DuYKt+GwmvM6ZSpLEBjFNnEyBTE5u2hJv3LgRd9xxOz7xiU/guOOOw3PPPTMhxxFhamPc5OSnE/F7fEuAomgWoqrXNcRiMfT29mDDhndx9NGfw1//+hdfixdz24nVGZOukkltEVVT2Nn5QNHxgmF0HHvsF7Bu3f/g5pvvwF577TMpxxFh6mHcVWk/X2U/HcmWBkXRoChkFepf/3oZS5achf32+xj23ntvMAxrNCb7SQd03dp2Qht5aRsN9Rp3WyWjxJTLTR4xEZFqGopCoszddpuH3XabN+79+i3QUDzxxB9x3XXX4Be/+PW4XzNC+Bh35OSnE5k5cyuIomiE6mvXPmT0KW3pePHFF3HYYZ/BRRctQ7FIJAS0tkQ9wjkuWCRkHt5ZKpWNiSdkP7LFa1yWJz9ias4PVJHLldGOBMMLQRZoAHKR/PGPvz+hMo0I7WHckdO0adOxePHZWLLkDEMnsuuucy06kUsvvQLXXEOkBDvtNAfHHHNcoH37XQEff3w9brvtFui6jq233hrf/ObSUK0suo0FCz5n/L9e11GvKygWFZPVi4REIt62JxXtjwOaTpI9PSlomt5wUWAnXHFuhnmwaTYbHjEB/o28FN/5zhU45ZTFDb/vCFMRoYiNDjnkMBxyyGGWv61Y8UPj/zvuuBNWrry7rX3SK+Btt90DQRBx5pmLsOee840fWbFYwIoV38Gtt96NadOm49Zbb8Ltt9+Cr33t6+N/Q5MMavVSKpmJikQ77XpSmRt56Uw3XQcymaSR+oXpcOAHlgXS6RRUVQ+dmIBggy9/8Yv7sPPOc0JJISN0D1O2fcVPolCv13Heed/AtGnTAQDbb78DNm7cMFmH2zWYPalGRgqoVJSOPKliManRCpPD6Gi2sSpIx6FT3VH3hl0CxLQvnU5D13XkciWETUyA/wLMa6/9C4899nucfPIEOONFGBemrEzb7wqYyfTggAMOBEAmrf7kJ3fhmGP+e8KPcyKhqrrF6iWoJ1UsJjem4DZ79Kzj0DnDYZJhwh92CRBiymRS0HUd2WzJd/R8p/BboPnDHx7F0NAQTjvtJNTrCoaGBnH22afhhhtu7crxROgcUzZyCipBKBQKOP/8r2GHHXbEpz99xEQe4qSCelKNjZUxPJxHuayA58nIddpQzDA6Vq36BUqloudwz6buiA671I1hl8lk3LUxOShIxJSCrqOrxAT4L9CceuoZuO++B3DnnT/Dd7/7AwwMTIuIaYpiypJTEAnC0NAQvvzl07D99jviwgsvmehDnDKgRGX2pNI0DcuWfRt33HE7yuUKOC7YV62qGsrlCrJZIpCs11ULUbXbRkOJiWEYZLPlrhITYF2g+dKXFuLggw81FmheeumfXX3tCOFiyroSDA5uwtlnn4ZbbrkLsVgMZ565CBdccJExh15VVZx++pfwX/91AL70pdMm7TinKm655QasWfMr3HjjLdhttzkQBB66rnc8QJTO96Mj3YOMzWIYHel0GgzDYGys1FYj9JaEqPHXGVOWnABvK4uNGzfi4osvwPbb72hsP2fOLlt0BGXGO++8DUEQMH36DABoWL0IDQeFJlHR+lI7xWn72Cz7mCkCHel0CizLIpst+Y6O35IRkZMzpjQ5dQtbuoI4TE8q+5ipt956Cw8++CAOOGB/zJs3D9lsOSImH0Tk5Iwpu1rXLfjppyg2ZwWxrjOoVlVUqyqAikFUqVQCQKItTyra70cHiuZyefzxj4/jxhtvwIwZM3HMMcfh+ONP7Pp7irD5YYsjp0hBbAexeiF2L02iSiQSSCbbNc/TsdNOO+LnP78fL730b6xf//uOZ+uZsbl3CkRwxhZHTpGC2AtNogKqEEUOosghkYgjmWSgKPWGqrzmQFQ6UqkkeJ5HNlvC9OkzceyxC8d9RFtyp8CWjikrJegWIgVxcATxpKIfZTKZbIzNKkFVw0uFo06BLRdbHDn56afMCuLzz/+qoSDe0mH3pFJV4knV19eD3t40RJHH2Fi4xAT4mxU6dQrsv/8nQj2GCJODLY6cIgXx+EGJamSkiLExYp6Xy5VDJyYg6hTYkrHFkVO3FcTr1q3FiSd+AccddxRWrbq/5fE333wD55xzOk4++Xicd945yOVy437NyQQhpgoUpTt6gahTYMvFFqlz6haoqt1cvP32t680ire6rmPhwqPx1a9+Hfvu+zHceOP10HUdZ5+9ZJKPfOpiS+gUiHROztjiVuu6CT+Zwssvv4RYLGY4gZ500inI5wuTdrzvB/iZGW7cuBGvvPISVFXF+vW/BxB1CmwuiMgpRPjJFN555y309fVj+fLL8OqrL2P27A/i3HPPn4xDfV/By8xwzpxd8fjjT07GYUXoMra4mlM34Ve8VVUVf//70zjqqGNw++0/xdZbb4Prr79uMg41QoQpj4icQoRf8bavrx+zZn0Ac+bsCgA46KBD8eKLL7TsJ0KECBE5hQo/mcK8ebtjbGwUr776CgDgT3/6X+y885zJOtwIEaY0otW6kOFl8zJnzq544YXn8f3vX4NyuYLp06fjkksuQ29v32QfdoRJRLRa54yInN5H8GuAffnll/Dd714FRVEwY8YMXHLJ5Uiloh/+VEdETs6I0rr3CYIMi/zBD1bg1FPPwF133Yttt52Ne++9Z5KO1ht+QtVXX30Zp576RRx33Ofxne9cjno9vEELEd4/iMjpfQK/BliArBaWSkUApM9MkqTJOFRPBCHZyy67BOeeewHuu+8B6LqONWt+NTkHG2FSEZHT+wR+DbAAcM455+Lqq6/EZz97KJ588q/43OeOnujD9IUfyW7Y8B6q1SrmziV2NYcfvgB/+MMjk3W4ESYRETm9T+CnoapWK/jOdy7H97//Y6xe/TscddQxuOKKpZNxqJ7wI9kgJBxhy4CnQjwq1E0dbL/9bDz11FPGd1Kp5PGBD2xj3H/uudeRSMRwwAFEunDqqSfjtttunnLfYSwmgGU147iSSQnxuGjcT6dliCJv3C8W4xAEbsq9jwjdRxQ5vU/wsY99DH/+858xMjKCcrmMdevWYf/99zcenz17NjZs2IDXXiP1m0cffRTz5k09J8+ZM2dicHDQuD84OIjp06e7Pj40NGR5PMKWg4ic3ieYMWMGzj33XJx00kn43Oc+hyOOOAK77747Fi9ejH/84x/IZDJYvnw5vva1r2HBggVYtWoVrrrqqsk+7Bb4kew222wDSZLw9NNPAwBWr15teTzClgNPnVOECN3AmjVrcPPNN0NRFBxzzDFYvHgxFi9ejCVLlmDevHl46aWXcPHFF6NQKGC33XbD8uXLIYriZB92hAlGRE4RIkSYkojSuggRIkxJROQUIUKEKYmInCJEiDAlEZFThAgRpiQicooQIcKUREROESJEmJKIyClChAhTEv8f3UV1QpuQl9wAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "filenames": { "image/png": "/Users/MortenImac/Teaching/MachineLearning/doc/LectureNotes/_build/jupyter_execute/chapter3_181_1.png" } }, "output_type": "display_data" } ], "source": [ "from mpl_toolkits.mplot3d import Axes3D\n", "import matplotlib.pyplot as plt\n", "from matplotlib import cm\n", "from matplotlib.ticker import LinearLocator, FormatStrFormatter\n", "import numpy as np\n", "from random import random, seed\n", "\n", "fig = plt.figure()\n", "ax = fig.gca(projection='3d')\n", "\n", "# Make data.\n", "x = np.arange(0, 1, 0.05)\n", "y = np.arange(0, 1, 0.05)\n", "x, y = np.meshgrid(x,y)\n", "\n", "\n", "def FrankeFunction(x,y):\n", " term1 = 0.75*np.exp(-(0.25*(9*x-2)**2) - 0.25*((9*y-2)**2))\n", " term2 = 0.75*np.exp(-((9*x+1)**2)/49.0 - 0.1*(9*y+1))\n", " term3 = 0.5*np.exp(-(9*x-7)**2/4.0 - 0.25*((9*y-3)**2))\n", " term4 = -0.2*np.exp(-(9*x-4)**2 - (9*y-7)**2)\n", " return term1 + term2 + term3 + term4\n", "\n", "\n", "z = FrankeFunction(x, y)\n", "\n", "# Plot the surface.\n", "surf = ax.plot_surface(x, y, z, cmap=cm.coolwarm,\n", " linewidth=0, antialiased=False)\n", "\n", "# Customize the z axis.\n", "ax.set_zlim(-0.10, 1.40)\n", "ax.zaxis.set_major_locator(LinearLocator(10))\n", "ax.zaxis.set_major_formatter(FormatStrFormatter('%.02f'))\n", "\n", "# Add a color bar which maps values to colors.\n", "fig.colorbar(surf, shrink=0.5, aspect=5)\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise: Ordinary Least Square (OLS) on the Franke function\n", "\n", "We will generate our own dataset for a function\n", "$\\mathrm{FrankeFunction}(x,y)$ with $x,y \\in [0,1]$. The function\n", "$f(x,y)$ is the Franke function. You should explore also the addition\n", "of an added stochastic noise to this function using the normal\n", "distribution $N(0,1)$.\n", "\n", "*Write your own code* (using either a matrix inversion or a singular\n", "value decomposition from e.g., **numpy** ) or use your code from\n", "homeworks 1 and 2 and perform a standard least square regression\n", "analysis using polynomials in $x$ and $y$ up to fifth order. Find the\n", "[confidence intervals](https://en.wikipedia.org/wiki/Confidence_interval) of the parameters (estimators) $\\beta$ by computing their\n", "variances, evaluate the Mean Squared error (MSE)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "MSE(\\hat{y},\\hat{\\tilde{y}}) = \\frac{1}{n}\n", "\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2,\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "and the $R^2$ score function. If $\\tilde{\\hat{y}}_i$ is the predicted\n", "value of the $i-th$ sample and $y_i$ is the corresponding true value,\n", "then the score $R^2$ is defined as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "R^2(\\hat{y}, \\tilde{\\hat{y}}) = 1 - \\frac{\\sum_{i=0}^{n - 1} (y_i - \\tilde{y}_i)^2}{\\sum_{i=0}^{n - 1} (y_i - \\bar{y})^2},\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "where we have defined the mean value of $\\hat{y}$ as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\bar{y} = \\frac{1}{n} \\sum_{i=0}^{n - 1} y_i.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Your code has to include a scaling of the data (for example by\n", "subtracting the mean value), and\n", "a split of the data in training and test data. For this exercise you can\n", "either write your own code or use for example the function for\n", "splitting training data provided by the library **Scikit-Learn** (make\n", "sure you have installed it). This function is called\n", "$train\\_test\\_split$. **You should present a critical discussion of why and how you have scaled or not scaled the data**.\n", "\n", "It is normal in essentially all Machine Learning studies to split the\n", "data in a training set and a test set (eventually also an additional\n", "validation set). There\n", "is no explicit recipe for how much data should be included as training\n", "data and say test data. An accepted rule of thumb is to use\n", "approximately $2/3$ to $4/5$ of the data as training data.\n", "\n", "\n", "You can easily reuse the solutions to your exercises from week 35 and week 36.\n", "\n", "\n", "\n", "### Exercise: Bias-variance trade-off and resampling techniques\n", "\n", "Our aim here is to study the bias-variance trade-off by implementing the **bootstrap** resampling technique.\n", "\n", "With a code which does OLS and includes resampling techniques, \n", "we will now discuss the bias-variance trade-off in the context of\n", "continuous predictions such as regression. However, many of the\n", "intuitions and ideas discussed here also carry over to classification\n", "tasks and basically all Machine Learning algorithms. \n", "\n", "Before you perform an analysis of the bias-variance trade-off on your test data, make\n", "first a figure similar to Fig. 2.11 of Hastie, Tibshirani, and\n", "Friedman. Figure 2.11 of this reference displays only the test and training MSEs. The test MSE can be used to \n", "indicate possible regions of low/high bias and variance. You will most likely not get an\n", "equally smooth curve!\n", "\n", "With this result we move on to the bias-variance trade-off analysis.\n", "\n", "Consider a\n", "dataset $\\mathcal{L}$ consisting of the data\n", "$\\mathbf{X}_\\mathcal{L}=\\{(y_j, \\boldsymbol{x}_j), j=0\\ldots n-1\\}$.\n", "\n", "Let us assume that the true data is generated from a noisy model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\boldsymbol{y}=f(\\boldsymbol{x}) + \\boldsymbol{\\epsilon}.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here $\\epsilon$ is normally distributed with mean zero and standard\n", "deviation $\\sigma^2$.\n", "\n", "In our derivation of the ordinary least squares method we defined then\n", "an approximation to the function $f$ in terms of the parameters\n", "$\\boldsymbol{\\beta}$ and the design matrix $\\boldsymbol{X}$ which embody our model,\n", "that is $\\boldsymbol{\\tilde{y}}=\\boldsymbol{X}\\boldsymbol{\\beta}$.\n", "\n", "The parameters $\\boldsymbol{\\beta}$ are in turn found by optimizing the means\n", "squared error via the so-called cost function" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "C(\\boldsymbol{X},\\boldsymbol{\\beta}) =\\frac{1}{n}\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2=\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right].\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here the expected value $\\mathbb{E}$ is the sample value. \n", "\n", "Show that you can rewrite this as" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$$\n", "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\frac{1}{n}\\sum_i(f_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2+\\frac{1}{n}\\sum_i(\\tilde{y}_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2+\\sigma^2.\n", "$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Explain what the terms mean, which one is the bias and which one is\n", "the variance and discuss their interpretations.\n", "\n", "Perform then a bias-variance analysis of the Franke function by\n", "studying the MSE value as function of the complexity of your model.\n", "\n", "Discuss the bias and variance trade-off as function\n", "of your model complexity (the degree of the polynomial) and the number\n", "of data points, and possibly also your training and test data using the **bootstrap** resampling method.\n", "\n", "Note also that when you calculate the bias, in all applications you don't know the function values $f_i$. You would hence replace them with the actual data points $y_i$.\n", "\n", "\n", "### Exercise: Cross-validation as resampling techniques, adding more complexity\n", "\n", "The aim here is to write your own code for another widely popular\n", "resampling technique, the so-called cross-validation method. Again,\n", "before you start with cross-validation approach, you should scale your\n", "data.\n", "\n", "Implement the $k$-fold cross-validation algorithm (write your own\n", "code) and evaluate again the MSE function resulting\n", "from the test folds. You can compare your own code with that from\n", "**Scikit-Learn** if needed. \n", "\n", "Compare the MSE you get from your cross-validation code with the one\n", "you got from your **bootstrap** code. Comment your results. Try $5-10$\n", "folds. You can also compare your own cross-validation code with the\n", "one provided by **Scikit-Learn**.\n", "\n", "\n", "### Exercise: Ridge Regression on the Franke function with resampling\n", "\n", "Write your own code for the Ridge method, either using matrix\n", "inversion or the singular value decomposition as done in the previous\n", "exercise. Perform the same bootstrap analysis as in the\n", "Exercise 2 (for the same polynomials) and the cross-validation in exercise 3 but now for different values of $\\lambda$. Compare and\n", "analyze your results with those obtained in exercises 1-3. Study the\n", "dependence on $\\lambda$.\n", "\n", "Study also the bias-variance trade-off as function of various values of\n", "the parameter $\\lambda$. For the bias-variance trade-off, use the **bootstrap** resampling method. Comment your results. \n", "\n", "### Exercise: Lasso Regression on the Franke function with resampling\n", "\n", "This exercise is essentially a repeat of the previous two ones, but now\n", "with Lasso regression. Write either your own code (difficult and optional) or, in this case,\n", "you can also use the functionalities of **Scikit-Learn** (recommended). \n", "Give a\n", "critical discussion of the three methods and a judgement of which\n", "model fits the data best. Perform here as well an analysis of the bias-variance trade-off using the **bootstrap** resampling technique and an analysis of the mean squared error using cross-validation. \n", "\n", "### Exercise: Analysis of real data\n", "\n", "With our codes functioning and having been tested properly on a\n", "simpler function we are now ready to look at real data. We will\n", "essentially repeat in this exercise what was done in exercises 1-5. However, we\n", "need first to download the data and prepare properly the inputs to our\n", "codes. We are going to download digital terrain data from the website\n", ",\n", "\n", "Or, if you prefer, we have placed selected datafiles at \n", "\n", "In order to obtain data for a specific region, you need to register as\n", "a user (free) at this website and then decide upon which area you want\n", "to fetch the digital terrain data from. In order to be able to read\n", "the data properly, you need to specify that the format should be **SRTM\n", "Arc-Second Global** and download the data as a **GeoTIF** file. The\n", "files are then stored in *tif* format which can be imported into a\n", "Python program using" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "collapsed": false, "editable": true }, "outputs": [ { "ename": "NameError", "evalue": "name 'scipy' is not defined", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m/var/folders/jy/g42mrgv128v34gnnhxwk9nrc0000gp/T/ipykernel_42456/1950915150.py\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mscipy\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmisc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mimread\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[0;31mNameError\u001b[0m: name 'scipy' is not defined" ] } ], "source": [ "scipy.misc.imread" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here is a simple part of a Python code which reads and plots the data\n", "from such files" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "editable": true }, "outputs": [], "source": [ "\"\"\"\n", "import numpy as np\n", "from imageio import imread\n", "import matplotlib.pyplot as plt\n", "from mpl_toolkits.mplot3d import Axes3D\n", "from matplotlib import cm\n", "\n", "# Load the terrain\n", "terrain1 = imread('SRTM_data_Norway_1.tif')\n", "# Show the terrain\n", "plt.figure()\n", "plt.title('Terrain over Norway 1')\n", "plt.imshow(terrain1, cmap='gray')\n", "plt.xlabel('X')\n", "plt.ylabel('Y')\n", "plt.show()\n", "\"\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you should have problems in downloading the digital terrain data,\n", "we provide two examples under the data folder of project 1. One is\n", "from a region close to Stavanger in Norway and the other Møsvatn\n", "Austfjell, again in Norway.\n", "Feel free to produce your own terrain data.\n", "\n", "\n", "Alternatively, if you would like to use another data set, feel free to do so. This could be data close to your reseach area or simply a data set you found interesting. See for example [kaggle.com](https://www.kaggle.com/datasets) for examples.\n", "\n", "\n", "Our final part deals with the parameterization of your digital terrain\n", "data (or your own data). We will apply all three methods for linear regression, the same type (or higher order) of polynomial\n", "approximation and cross-validation as resampling technique to evaluate which\n", "model fits the data best.\n", "\n", "At the end, you should present a critical evaluation of your results\n", "and discuss the applicability of these regression methods to the type\n", "of data presented here (either the terrain data we propose or other data sets)." ] } ], "metadata": { "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.12" } }, "nbformat": 4, "nbformat_minor": 4 }