diff --git a/doc/pub/week37/html/._week37-bs001.html b/doc/pub/week37/html/._week37-bs001.html
index a2e0eeb5c..900223e79 100644
--- a/doc/pub/week37/html/._week37-bs001.html
+++ b/doc/pub/week37/html/._week37-bs001.html
@@ -307,6 +307,8 @@ MathJax.Hub.Config({
+ Video of Lecture
+ Whiteboard notes
Resampling techniques, Bootstrap and cross validation and bias-variance tradeoff
Statistical interpretation of Ridge and Lasso regression
Readings and Videos:
diff --git a/doc/pub/week37/html/week37-reveal.html b/doc/pub/week37/html/week37-reveal.html
index 56a0c5bee..a0614cd1e 100644
--- a/doc/pub/week37/html/week37-reveal.html
+++ b/doc/pub/week37/html/week37-reveal.html
@@ -221,6 +221,10 @@ MathJax.Hub.Config({
+
Video of Lecture
+
+
Whiteboard notes
+
Resampling techniques, Bootstrap and cross validation and bias-variance tradeoff
Statistical interpretation of Ridge and Lasso regression
diff --git a/doc/pub/week37/html/week37-solarized.html b/doc/pub/week37/html/week37-solarized.html
index 4ee9ecde5..b03440269 100644
--- a/doc/pub/week37/html/week37-solarized.html
+++ b/doc/pub/week37/html/week37-solarized.html
@@ -279,6 +279,8 @@ MathJax.Hub.Config({
Material for the lecture on Thursday September 7
+ Video of Lecture
+ Whiteboard notes
Resampling techniques, Bootstrap and cross validation and bias-variance tradeoff
Statistical interpretation of Ridge and Lasso regression
Readings and Videos:
diff --git a/doc/pub/week37/html/week37.html b/doc/pub/week37/html/week37.html
index d95278055..e4b915ced 100644
--- a/doc/pub/week37/html/week37.html
+++ b/doc/pub/week37/html/week37.html
@@ -356,6 +356,8 @@ MathJax.Hub.Config({
Material for the lecture on Thursday September 7
+ Video of Lecture
+ Whiteboard notes
Resampling techniques, Bootstrap and cross validation and bias-variance tradeoff
Statistical interpretation of Ridge and Lasso regression
Readings and Videos:
diff --git a/doc/pub/week37/ipynb/ipynb-week37-src.tar.gz b/doc/pub/week37/ipynb/ipynb-week37-src.tar.gz
index f3d856a93..0bccf2ac5 100644
Binary files a/doc/pub/week37/ipynb/ipynb-week37-src.tar.gz and b/doc/pub/week37/ipynb/ipynb-week37-src.tar.gz differ
diff --git a/doc/pub/week37/ipynb/week37.ipynb b/doc/pub/week37/ipynb/week37.ipynb
index 9da9a1db5..96c6b8d8c 100644
--- a/doc/pub/week37/ipynb/week37.ipynb
+++ b/doc/pub/week37/ipynb/week37.ipynb
@@ -2,8 +2,10 @@
"cells": [
{
"cell_type": "markdown",
- "id": "36ef4820",
- "metadata": {},
+ "id": "9d1d9c58",
+ "metadata": {
+ "editable": true
+ },
"source": [
"\n",
@@ -12,8 +14,10 @@
},
{
"cell_type": "markdown",
- "id": "eb429a32",
- "metadata": {},
+ "id": "c1bdb9de",
+ "metadata": {
+ "editable": true
+ },
"source": [
"# Week 37: Statitsitcal interpretations and Resampling Methods\n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and Facility for Rare Isotope Beams, Michigan State University\n",
@@ -27,8 +31,10 @@
},
{
"cell_type": "markdown",
- "id": "108e644b",
- "metadata": {},
+ "id": "ccaa7ab9",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Plans for week 37\n",
"\n",
@@ -47,6 +53,10 @@
" \n",
"**Material for the lecture on Thursday September 7.**\n",
"\n",
+ " * [Video of Lecture](https://youtu.be/YOBBr_toYxc)\n",
+ "\n",
+ " * [Whiteboard notes](https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2023/NotesSep14.pdf)\n",
+ "\n",
" * Resampling techniques, Bootstrap and cross validation and bias-variance tradeoff\n",
"\n",
" * Statistical interpretation of Ridge and Lasso regression\n",
@@ -64,16 +74,20 @@
},
{
"cell_type": "markdown",
- "id": "ccf47e3c",
- "metadata": {},
+ "id": "1c5c27bf",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Material from last week and relevant for the weekly exercises"
]
},
{
"cell_type": "markdown",
- "id": "957d3950",
- "metadata": {},
+ "id": "656c8db6",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Linking the regression analysis with a statistical interpretation\n",
"\n",
@@ -99,8 +113,10 @@
},
{
"cell_type": "markdown",
- "id": "055a6564",
- "metadata": {},
+ "id": "4c35f924",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\begin{align*} \n",
@@ -113,8 +129,10 @@
},
{
"cell_type": "markdown",
- "id": "71981487",
- "metadata": {},
+ "id": "9087bc37",
+ "metadata": {
+ "editable": true
+ },
"source": [
"The randomness of $\\varepsilon_i$ implies that\n",
"$\\mathbf{y}_i$ is also a random variable. In particular,\n",
@@ -130,8 +148,10 @@
},
{
"cell_type": "markdown",
- "id": "600fc3dd",
- "metadata": {},
+ "id": "8a7bb057",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Assumptions made\n",
"\n",
@@ -142,8 +162,10 @@
},
{
"cell_type": "markdown",
- "id": "6811f1d0",
- "metadata": {},
+ "id": "68392e55",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\boldsymbol{y} = f(\\boldsymbol{x})+\\boldsymbol{\\varepsilon}\n",
@@ -152,8 +174,10 @@
},
{
"cell_type": "markdown",
- "id": "6964c438",
- "metadata": {},
+ "id": "27cd8fb1",
+ "metadata": {
+ "editable": true
+ },
"source": [
"We approximate this function with our model from the solution of the linear regression equations, that is our\n",
"function $f$ is approximated by $\\boldsymbol{\\tilde{y}}$ where we want to minimize $(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2$, our MSE, with"
@@ -161,8 +185,10 @@
},
{
"cell_type": "markdown",
- "id": "5a0e80d9",
- "metadata": {},
+ "id": "f0344126",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\boldsymbol{\\tilde{y}} = \\boldsymbol{X}\\boldsymbol{\\beta}.\n",
@@ -171,8 +197,10 @@
},
{
"cell_type": "markdown",
- "id": "10aa4aba",
- "metadata": {},
+ "id": "d23667d2",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Expectation value and variance\n",
"\n",
@@ -181,8 +209,10 @@
},
{
"cell_type": "markdown",
- "id": "b7fcd7a7",
- "metadata": {},
+ "id": "63687c52",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\begin{align*} \n",
@@ -195,8 +225,10 @@
},
{
"cell_type": "markdown",
- "id": "7596f904",
- "metadata": {},
+ "id": "54e07f66",
+ "metadata": {
+ "editable": true
+ },
"source": [
"while\n",
"its variance is"
@@ -204,8 +236,10 @@
},
{
"cell_type": "markdown",
- "id": "9f5e6648",
- "metadata": {},
+ "id": "de6492ed",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\begin{align*} \\mbox{Var}(y_i) & = \\mathbb{E} \\{ [y_i\n",
@@ -225,8 +259,10 @@
},
{
"cell_type": "markdown",
- "id": "0f86ff41",
- "metadata": {},
+ "id": "a487e7d3",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Hence, $y_i \\sim \\mathcal{N}( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta}, \\sigma^2)$, that is $\\boldsymbol{y}$ follows a normal distribution with \n",
"mean value $\\boldsymbol{X}\\boldsymbol{\\beta}$ and variance $\\sigma^2$ (not be confused with the singular values of the SVD)."
@@ -234,8 +270,10 @@
},
{
"cell_type": "markdown",
- "id": "86654915",
- "metadata": {},
+ "id": "dbac7b3f",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Expectation value and variance for $\\boldsymbol{\\beta}$\n",
"\n",
@@ -244,8 +282,10 @@
},
{
"cell_type": "markdown",
- "id": "8a52c0e3",
- "metadata": {},
+ "id": "edc35885",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\mathbb{E}(\\boldsymbol{\\hat{\\beta}}) = \\mathbb{E}[ (\\mathbf{X}^{\\top} \\mathbf{X})^{-1}\\mathbf{X}^{T} \\mathbf{Y}]=(\\mathbf{X}^{T} \\mathbf{X})^{-1}\\mathbf{X}^{T} \\mathbb{E}[ \\mathbf{Y}]=(\\mathbf{X}^{T} \\mathbf{X})^{-1} \\mathbf{X}^{T}\\mathbf{X}\\boldsymbol{\\beta}=\\boldsymbol{\\beta}.\n",
@@ -254,8 +294,10 @@
},
{
"cell_type": "markdown",
- "id": "b98da759",
- "metadata": {},
+ "id": "5b79b613",
+ "metadata": {
+ "editable": true
+ },
"source": [
"This means that the estimator of the regression parameters is unbiased.\n",
"\n",
@@ -266,8 +308,10 @@
},
{
"cell_type": "markdown",
- "id": "929e6445",
- "metadata": {},
+ "id": "b435eaee",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\begin{eqnarray*}\n",
@@ -295,8 +339,10 @@
},
{
"cell_type": "markdown",
- "id": "f38c673a",
- "metadata": {},
+ "id": "5aaad09e",
+ "metadata": {
+ "editable": true
+ },
"source": [
"where we have used that $\\mathbb{E} (\\mathbf{y} \\mathbf{y}^{T}) =\n",
"\\mathbf{X} \\, \\boldsymbol{\\beta} \\, \\boldsymbol{\\beta}^{T} \\, \\mathbf{X}^{T} +\n",
@@ -315,8 +361,10 @@
},
{
"cell_type": "markdown",
- "id": "b5f6be3f",
- "metadata": {},
+ "id": "276770f2",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\mathbb{E} \\big[ \\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}} \\big]=(\\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I}_{pp})^{-1} (\\mathbf{X}^{\\top} \\mathbf{X})\\boldsymbol{\\beta}.\n",
@@ -325,8 +373,10 @@
},
{
"cell_type": "markdown",
- "id": "6ed11037",
- "metadata": {},
+ "id": "ac99320c",
+ "metadata": {
+ "editable": true
+ },
"source": [
"We see clearly that \n",
"$\\mathbb{E} \\big[ \\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}} \\big] \\not= \\hat{\\boldsymbol{\\beta}}^{\\mathrm{OLS}}$ for any $\\lambda > 0$.\n",
@@ -336,8 +386,10 @@
},
{
"cell_type": "markdown",
- "id": "6af4a27a",
- "metadata": {},
+ "id": "9bb35322",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\mbox{Var}[\\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}}]=\\sigma^2[ \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1} \\mathbf{X}^{T} \\mathbf{X} \\{ [ \\mathbf{X}^{\\top} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}\\}^{T},\n",
@@ -346,8 +398,10 @@
},
{
"cell_type": "markdown",
- "id": "10761191",
- "metadata": {},
+ "id": "83113c02",
+ "metadata": {
+ "editable": true
+ },
"source": [
"and it is easy to see that if the parameter $\\lambda$ goes to infinity then the variance of Ridge parameters $\\boldsymbol{\\beta}$ goes to zero. \n",
"\n",
@@ -356,8 +410,10 @@
},
{
"cell_type": "markdown",
- "id": "82e2b53d",
- "metadata": {},
+ "id": "2dea05a4",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\mbox{Var}[\\hat{\\boldsymbol{\\beta}}^{\\mathrm{OLS}}]-\\mbox{Var}(\\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}})=\\sigma^2 [ \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}[ 2\\lambda\\mathbf{I} + \\lambda^2 (\\mathbf{X}^{T} \\mathbf{X})^{-1} ] \\{ [ \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}\\}^{T}.\n",
@@ -366,8 +422,10 @@
},
{
"cell_type": "markdown",
- "id": "d97d0d83",
- "metadata": {},
+ "id": "f8a821f1",
+ "metadata": {
+ "editable": true
+ },
"source": [
"The difference is non-negative definite since each component of the\n",
"matrix product is non-negative definite. \n",
@@ -378,16 +436,20 @@
},
{
"cell_type": "markdown",
- "id": "41ce5dee",
- "metadata": {},
+ "id": "ff9cbfd6",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Material for lecture Thursday September 14"
]
},
{
"cell_type": "markdown",
- "id": "b77c72bf",
- "metadata": {},
+ "id": "2592817d",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Deriving OLS from a probability distribution\n",
"\n",
@@ -407,8 +469,10 @@
},
{
"cell_type": "markdown",
- "id": "25955a18",
- "metadata": {},
+ "id": "b134aee7",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"y_i\\sim \\mathcal{N}(\\boldsymbol{X}_{i,*}\\boldsymbol{\\beta}, \\sigma^2)=\\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\exp{\\left[-\\frac{(y_i-\\boldsymbol{X}_{i,*}\\boldsymbol{\\beta})^2}{2\\sigma^2}\\right]}.\n",
@@ -417,8 +481,10 @@
},
{
"cell_type": "markdown",
- "id": "48f228a3",
- "metadata": {},
+ "id": "2040959c",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Independent and Identically Distrubuted (iid)\n",
"\n",
@@ -428,8 +494,10 @@
},
{
"cell_type": "markdown",
- "id": "8a118bf2",
- "metadata": {},
+ "id": "130fdf44",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(y_i, \\boldsymbol{X}\\vert\\boldsymbol{\\beta})=\\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\exp{\\left[-\\frac{(y_i-\\boldsymbol{X}_{i,*}\\boldsymbol{\\beta})^2}{2\\sigma^2}\\right]},\n",
@@ -438,8 +506,10 @@
},
{
"cell_type": "markdown",
- "id": "ac35066f",
- "metadata": {},
+ "id": "2e162d22",
+ "metadata": {
+ "editable": true
+ },
"source": [
"which reads as finding the likelihood of an event $y_i$ with the input variables $\\boldsymbol{X}$ given the parameters (to be determined) $\\boldsymbol{\\beta}$.\n",
"\n",
@@ -448,8 +518,10 @@
},
{
"cell_type": "markdown",
- "id": "34924f1f",
- "metadata": {},
+ "id": "a2e8bbf0",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(\\boldsymbol{y},\\boldsymbol{X}\\vert\\boldsymbol{\\beta})=\\prod_{i=0}^{n-1}\\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\exp{\\left[-\\frac{(y_i-\\boldsymbol{X}_{i,*}\\boldsymbol{\\beta})^2}{2\\sigma^2}\\right]}=\\prod_{i=0}^{n-1}p(y_i,\\boldsymbol{X}\\vert\\boldsymbol{\\beta}).\n",
@@ -458,8 +530,10 @@
},
{
"cell_type": "markdown",
- "id": "aae0235b",
- "metadata": {},
+ "id": "92c5755a",
+ "metadata": {
+ "editable": true
+ },
"source": [
"We will write this in a more compact form reserving $\\boldsymbol{D}$ for the domain of events, including the ouputs (targets) and the inputs. That is\n",
"in case we have a simple one-dimensional input and output case"
@@ -467,8 +541,10 @@
},
{
"cell_type": "markdown",
- "id": "8d20762a",
- "metadata": {},
+ "id": "f3b2d261",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\boldsymbol{D}=[(x_0,y_0), (x_1,y_1),\\dots, (x_{n-1},y_{n-1})].\n",
@@ -477,8 +553,10 @@
},
{
"cell_type": "markdown",
- "id": "a7b26374",
- "metadata": {},
+ "id": "9d4d6500",
+ "metadata": {
+ "editable": true
+ },
"source": [
"In the more general case the various inputs should be replaced by the possible features represented by the input data set $\\boldsymbol{X}$. \n",
"We can now rewrite the above probability as"
@@ -486,8 +564,10 @@
},
{
"cell_type": "markdown",
- "id": "3528de48",
- "metadata": {},
+ "id": "9a18f7aa",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(\\boldsymbol{D}\\vert\\boldsymbol{\\beta})=\\prod_{i=0}^{n-1}\\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\exp{\\left[-\\frac{(y_i-\\boldsymbol{X}_{i,*}\\boldsymbol{\\beta})^2}{2\\sigma^2}\\right]}.\n",
@@ -496,16 +576,20 @@
},
{
"cell_type": "markdown",
- "id": "eed312f3",
- "metadata": {},
+ "id": "ead724f7",
+ "metadata": {
+ "editable": true
+ },
"source": [
"It is a conditional probability (see below) and reads as the likelihood of a domain of events $\\boldsymbol{D}$ given a set of parameters $\\boldsymbol{\\beta}$."
]
},
{
"cell_type": "markdown",
- "id": "9978550f",
- "metadata": {},
+ "id": "7aeab260",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Maximum Likelihood Estimation (MLE)\n",
"\n",
@@ -533,8 +617,10 @@
},
{
"cell_type": "markdown",
- "id": "9d73a0e7",
- "metadata": {},
+ "id": "43afa9cd",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## A new Cost Function\n",
"\n",
@@ -543,8 +629,10 @@
},
{
"cell_type": "markdown",
- "id": "c07f3eec",
- "metadata": {},
+ "id": "2bbe6c3a",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"C(\\boldsymbol{\\beta}=-\\log{\\prod_{i=0}^{n-1}p(y_i,\\boldsymbol{X}\\vert\\boldsymbol{\\beta})}=-\\sum_{i=0}^{n-1}\\log{p(y_i,\\boldsymbol{X}\\vert\\boldsymbol{\\beta})},\n",
@@ -553,16 +641,20 @@
},
{
"cell_type": "markdown",
- "id": "284ef375",
- "metadata": {},
+ "id": "1106ec56",
+ "metadata": {
+ "editable": true
+ },
"source": [
"which becomes"
]
},
{
"cell_type": "markdown",
- "id": "977e59d1",
- "metadata": {},
+ "id": "cc4efbb2",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"C(\\boldsymbol{\\beta}=\\frac{n}{2}\\log{2\\pi\\sigma^2}+\\frac{\\vert\\vert (\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta})\\vert\\vert_2^2}{2\\sigma^2}.\n",
@@ -571,16 +663,20 @@
},
{
"cell_type": "markdown",
- "id": "68bc1696",
- "metadata": {},
+ "id": "031ab659",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Taking the derivative of the *new* cost function with respect to the parameters $\\beta$ we recognize our familiar OLS equation, namely"
]
},
{
"cell_type": "markdown",
- "id": "456b2ed2",
- "metadata": {},
+ "id": "09f14189",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\boldsymbol{X}^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right) =0,\n",
@@ -589,16 +685,20 @@
},
{
"cell_type": "markdown",
- "id": "6b5375bf",
- "metadata": {},
+ "id": "775ed960",
+ "metadata": {
+ "editable": true
+ },
"source": [
"which leads to the well-known OLS equation for the optimal paramters $\\beta$"
]
},
{
"cell_type": "markdown",
- "id": "2200d55a",
- "metadata": {},
+ "id": "f0e242cf",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\hat{\\boldsymbol{\\beta}}^{\\mathrm{OLS}}=\\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}!\n",
@@ -607,16 +707,20 @@
},
{
"cell_type": "markdown",
- "id": "41a23c50",
- "metadata": {},
+ "id": "aa1d0f4e",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Before we make a similar analysis for Ridge and Lasso regression, we need a short reminder on statistics."
]
},
{
"cell_type": "markdown",
- "id": "a5d06190",
- "metadata": {},
+ "id": "04b1875f",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## More basic Statistics and Bayes' theorem\n",
"\n",
@@ -633,8 +737,10 @@
},
{
"cell_type": "markdown",
- "id": "faadacb9",
- "metadata": {},
+ "id": "fd5fd7d3",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(X \\cup Y)= p(X)+p(Y)-p(X \\cap Y).\n",
@@ -643,16 +749,20 @@
},
{
"cell_type": "markdown",
- "id": "5a383325",
- "metadata": {},
+ "id": "dbb5f8dd",
+ "metadata": {
+ "editable": true
+ },
"source": [
"**The product rule (aka joint probability) is given by.**"
]
},
{
"cell_type": "markdown",
- "id": "83e4e576",
- "metadata": {},
+ "id": "7cda70f2",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(X \\cup Y)= p(X,Y)= p(X\\vert Y)p(Y)=p(Y\\vert X)p(X),\n",
@@ -661,8 +771,10 @@
},
{
"cell_type": "markdown",
- "id": "db8b3492",
- "metadata": {},
+ "id": "ec080569",
+ "metadata": {
+ "editable": true
+ },
"source": [
"where we read $p(X\\vert Y)$ as the likelihood of obtaining $X$ given $Y$.\n",
"\n",
@@ -671,8 +783,10 @@
},
{
"cell_type": "markdown",
- "id": "a9586794",
- "metadata": {},
+ "id": "3b12ffc4",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Marginal Probability\n",
"\n",
@@ -681,8 +795,10 @@
},
{
"cell_type": "markdown",
- "id": "24631159",
- "metadata": {},
+ "id": "b347871f",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(X)=\\sum_{i=0}^{n-1}p(X,Y=y_i)=\\sum_{i=0}^{n-1}p(X\\vert Y=y_i)p(Y=y_i)=\\sum_{i=0}^{n-1}p(X\\vert y_i)p(y_i).\n",
@@ -691,8 +807,10 @@
},
{
"cell_type": "markdown",
- "id": "ea5c8438",
- "metadata": {},
+ "id": "7af990c8",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Conditional Probability\n",
"\n",
@@ -701,8 +819,10 @@
},
{
"cell_type": "markdown",
- "id": "f8e818fc",
- "metadata": {},
+ "id": "ff6fb728",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(X\\vert Y)= \\frac{p(X,Y)}{p(Y)}=\\frac{p(X,Y)}{\\sum_{i=0}^{n-1}p(Y\\vert X=x_i)p(x_i)}.\n",
@@ -711,8 +831,10 @@
},
{
"cell_type": "markdown",
- "id": "39f88f08",
- "metadata": {},
+ "id": "9b9c9cc4",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Bayes' Theorem\n",
"\n",
@@ -721,8 +843,10 @@
},
{
"cell_type": "markdown",
- "id": "d932650c",
- "metadata": {},
+ "id": "294e9ad9",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(X\\vert Y)= \\frac{p(X,Y)}{p(Y)},\n",
@@ -731,16 +855,20 @@
},
{
"cell_type": "markdown",
- "id": "713135cf",
- "metadata": {},
+ "id": "8021f77b",
+ "metadata": {
+ "editable": true
+ },
"source": [
"which we can rewrite as"
]
},
{
"cell_type": "markdown",
- "id": "604a3543",
- "metadata": {},
+ "id": "78fbc86a",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(X\\vert Y)= \\frac{p(X,Y)}{\\sum_{i=0}^{n-1}p(Y\\vert X=x_i)p(x_i)}=\\frac{p(Y\\vert X)p(X)}{\\sum_{i=0}^{n-1}p(Y\\vert X=x_i)p(x_i)},\n",
@@ -749,16 +877,20 @@
},
{
"cell_type": "markdown",
- "id": "6118ab84",
- "metadata": {},
+ "id": "ab52abe5",
+ "metadata": {
+ "editable": true
+ },
"source": [
"which is Bayes' theorem. It allows us to evaluate the uncertainty in in $X$ after we have observed $Y$. We can easily interchange $X$ with $Y$."
]
},
{
"cell_type": "markdown",
- "id": "269b0de5",
- "metadata": {},
+ "id": "1f809753",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Interpretations of Bayes' Theorem\n",
"\n",
@@ -774,8 +906,10 @@
},
{
"cell_type": "markdown",
- "id": "0b5d3215",
- "metadata": {},
+ "id": "908364e7",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Example of Usage of Bayes' theorem\n",
"\n",
@@ -793,8 +927,10 @@
},
{
"cell_type": "markdown",
- "id": "0efca62b",
- "metadata": {},
+ "id": "6e5c83b1",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(X=1\\vert Y=1) =0.8.\n",
@@ -803,8 +939,10 @@
},
{
"cell_type": "markdown",
- "id": "2ffcb4f3",
- "metadata": {},
+ "id": "5653360e",
+ "metadata": {
+ "editable": true
+ },
"source": [
"This obviously sounds scary since many would conclude that if the test is positive, there is a likelihood of $80\\%$ for having cancer.\n",
"It is however not correct, as the following Bayesian analysis shows."
@@ -812,8 +950,10 @@
},
{
"cell_type": "markdown",
- "id": "a2d92d4d",
- "metadata": {},
+ "id": "e617edc1",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Doing it correctly\n",
"\n",
@@ -823,8 +963,10 @@
},
{
"cell_type": "markdown",
- "id": "6a0102e8",
- "metadata": {},
+ "id": "ecb9f426",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(Y=1) =0.004.\n",
@@ -833,16 +975,20 @@
},
{
"cell_type": "markdown",
- "id": "cea754a6",
- "metadata": {},
+ "id": "eb2347e9",
+ "metadata": {
+ "editable": true
+ },
"source": [
"We need also to account for the fact that the test may produce a false positive result (false alarm). Let us here assume that we have"
]
},
{
"cell_type": "markdown",
- "id": "cd39d2de",
- "metadata": {},
+ "id": "ce975ccd",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(X=1\\vert Y=0) =0.1.\n",
@@ -851,16 +997,20 @@
},
{
"cell_type": "markdown",
- "id": "d454e003",
- "metadata": {},
+ "id": "4b0ad706",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Using Bayes' theorem we can then find the posterior probability that the person has breast cancer in case of a positive test, that is we can compute"
]
},
{
"cell_type": "markdown",
- "id": "73666b6a",
- "metadata": {},
+ "id": "249c79a0",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(Y=1\\vert X=1)=\\frac{p(X=1\\vert Y=1)p(Y=1)}{p(X=1\\vert Y=1)p(Y=1)+p(X=1\\vert Y=0)p(Y=0)}=\\frac{0.8\\times 0.004}{0.8\\times 0.004+0.1\\times 0.996}=0.031.\n",
@@ -869,16 +1019,20 @@
},
{
"cell_type": "markdown",
- "id": "8f36b626",
- "metadata": {},
+ "id": "6497aa62",
+ "metadata": {
+ "editable": true
+ },
"source": [
"That is, in case of a positive test, there is only a $3\\%$ chance of having breast cancer!"
]
},
{
"cell_type": "markdown",
- "id": "09261fa6",
- "metadata": {},
+ "id": "36ac6a6c",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Bayes' Theorem and Ridge and Lasso Regression\n",
"\n",
@@ -889,8 +1043,10 @@
},
{
"cell_type": "markdown",
- "id": "e4d61df8",
- "metadata": {},
+ "id": "0667ce18",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\boldsymbol{D}=[(x_0,y_0), (x_1,y_1),\\dots, (x_{n-1},y_{n-1})],\n",
@@ -899,16 +1055,20 @@
},
{
"cell_type": "markdown",
- "id": "854de83d",
- "metadata": {},
+ "id": "23e6bbe1",
+ "metadata": {
+ "editable": true
+ },
"source": [
"is given by"
]
},
{
"cell_type": "markdown",
- "id": "d47d42b1",
- "metadata": {},
+ "id": "1303e4fa",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(\\boldsymbol{D}\\vert\\boldsymbol{\\beta})=\\prod_{i=0}^{n-1}\\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\exp{\\left[-\\frac{(y_i-\\boldsymbol{X}_{i,*}\\boldsymbol{\\beta})^2}{2\\sigma^2}\\right]}.\n",
@@ -917,16 +1077,20 @@
},
{
"cell_type": "markdown",
- "id": "5296f7ff",
- "metadata": {},
+ "id": "07a72af1",
+ "metadata": {
+ "editable": true
+ },
"source": [
"In Bayes' theorem this function plays the role of the so-called likelihood. We could now ask the question what is the posterior probability of a parameter set $\\boldsymbol{\\beta}$ given a domain of events $\\boldsymbol{D}$? That is, how can we define the posterior probability"
]
},
{
"cell_type": "markdown",
- "id": "a8dd9dd6",
- "metadata": {},
+ "id": "a0fdb3bd",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(\\boldsymbol{\\beta}\\vert\\boldsymbol{D}).\n",
@@ -935,16 +1099,20 @@
},
{
"cell_type": "markdown",
- "id": "bc10a10a",
- "metadata": {},
+ "id": "0c191fc5",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Bayes' theorem comes to our rescue here since (omitting the normalization constant)"
]
},
{
"cell_type": "markdown",
- "id": "0d35f771",
- "metadata": {},
+ "id": "05a216b4",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(\\boldsymbol{\\beta}\\vert\\boldsymbol{D})\\propto p(\\boldsymbol{D}\\vert\\boldsymbol{\\beta})p(\\boldsymbol{\\beta}).\n",
@@ -953,16 +1121,20 @@
},
{
"cell_type": "markdown",
- "id": "5195b5eb",
- "metadata": {},
+ "id": "21d80a1c",
+ "metadata": {
+ "editable": true
+ },
"source": [
"We have a model for $p(\\boldsymbol{D}\\vert\\boldsymbol{\\beta})$ but need one for the **prior** $p(\\boldsymbol{\\beta}$!"
]
},
{
"cell_type": "markdown",
- "id": "2721688e",
- "metadata": {},
+ "id": "06073a94",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Ridge and Bayes\n",
"\n",
@@ -975,8 +1147,10 @@
},
{
"cell_type": "markdown",
- "id": "fc227b08",
- "metadata": {},
+ "id": "aaf3ac26",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(\\boldsymbol{\\beta})=\\prod_{j=0}^{p-1}\\exp{\\left(-\\frac{\\beta_j^2}{2\\tau^2}\\right)}.\n",
@@ -985,16 +1159,20 @@
},
{
"cell_type": "markdown",
- "id": "9f6c54d1",
- "metadata": {},
+ "id": "5ff662ce",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Our posterior probability becomes then (omitting the normalization factor which is just a constant)"
]
},
{
"cell_type": "markdown",
- "id": "7cf8bdb0",
- "metadata": {},
+ "id": "2c210fdb",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(\\boldsymbol{\\beta\\vert\\boldsymbol{D})}=\\prod_{i=0}^{n-1}\\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\exp{\\left[-\\frac{(y_i-\\boldsymbol{X}_{i,*}\\boldsymbol{\\beta})^2}{2\\sigma^2}\\right]}\\prod_{j=0}^{p-1}\\exp{\\left(-\\frac{\\beta_j^2}{2\\tau^2}\\right)}.\n",
@@ -1003,8 +1181,10 @@
},
{
"cell_type": "markdown",
- "id": "b759bd30",
- "metadata": {},
+ "id": "ebbc15d5",
+ "metadata": {
+ "editable": true
+ },
"source": [
"We can now optimize this quantity with respect to $\\boldsymbol{\\beta}$. As we\n",
"did for OLS, this is most conveniently done by taking the negative\n",
@@ -1014,8 +1194,10 @@
},
{
"cell_type": "markdown",
- "id": "0819637c",
- "metadata": {},
+ "id": "b9123830",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"C(\\boldsymbol{\\beta})=\\frac{\\vert\\vert (\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta})\\vert\\vert_2^2}{2\\sigma^2}+\\frac{1}{2\\tau^2}\\vert\\vert\\boldsymbol{\\beta}\\vert\\vert_2^2,\n",
@@ -1024,16 +1206,20 @@
},
{
"cell_type": "markdown",
- "id": "eb658f59",
- "metadata": {},
+ "id": "9fb1e84e",
+ "metadata": {
+ "editable": true
+ },
"source": [
"and replacing $1/2\\tau^2$ with $\\lambda$ we have"
]
},
{
"cell_type": "markdown",
- "id": "ae7a6f59",
- "metadata": {},
+ "id": "c48c43d0",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"C(\\boldsymbol{\\beta})=\\frac{\\vert\\vert (\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta})\\vert\\vert_2^2}{2\\sigma^2}+\\lambda\\vert\\vert\\boldsymbol{\\beta}\\vert\\vert_2^2,\n",
@@ -1042,16 +1228,20 @@
},
{
"cell_type": "markdown",
- "id": "5b4d9b86",
- "metadata": {},
+ "id": "ed19a572",
+ "metadata": {
+ "editable": true
+ },
"source": [
"which is our Ridge cost function! Nice, isn't it?"
]
},
{
"cell_type": "markdown",
- "id": "4f966872",
- "metadata": {},
+ "id": "65ea42da",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Lasso and Bayes\n",
"\n",
@@ -1060,8 +1250,10 @@
},
{
"cell_type": "markdown",
- "id": "363e1852",
- "metadata": {},
+ "id": "587ea365",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(\\boldsymbol{\\beta})=\\prod_{j=0}^{p-1}\\exp{\\left(-\\frac{\\vert\\beta_j\\vert}{\\tau}\\right)}.\n",
@@ -1070,16 +1262,20 @@
},
{
"cell_type": "markdown",
- "id": "46d56ecc",
- "metadata": {},
+ "id": "55422904",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Our posterior probability becomes then (omitting the normalization factor which is just a constant)"
]
},
{
"cell_type": "markdown",
- "id": "4e43ec1c",
- "metadata": {},
+ "id": "55e99fd7",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"p(\\boldsymbol{\\beta}\\vert\\boldsymbol{D})=\\prod_{i=0}^{n-1}\\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\exp{\\left[-\\frac{(y_i-\\boldsymbol{X}_{i,*}\\boldsymbol{\\beta})^2}{2\\sigma^2}\\right]}\\prod_{j=0}^{p-1}\\exp{\\left(-\\frac{\\vert\\beta_j\\vert}{\\tau}\\right)}.\n",
@@ -1088,8 +1284,10 @@
},
{
"cell_type": "markdown",
- "id": "b4358bc6",
- "metadata": {},
+ "id": "fb8e8c1e",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Taking the negative\n",
"logarithm of the posterior probability and leaving out the\n",
@@ -1098,8 +1296,10 @@
},
{
"cell_type": "markdown",
- "id": "2762ae09",
- "metadata": {},
+ "id": "72c3b3dc",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"C(\\boldsymbol{\\beta}=\\frac{\\vert\\vert (\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta})\\vert\\vert_2^2}{2\\sigma^2}+\\frac{1}{\\tau}\\vert\\vert\\boldsymbol{\\beta}\\vert\\vert_1,\n",
@@ -1108,16 +1308,20 @@
},
{
"cell_type": "markdown",
- "id": "59f5920e",
- "metadata": {},
+ "id": "348550b2",
+ "metadata": {
+ "editable": true
+ },
"source": [
"and replacing $1/\\tau$ with $\\lambda$ we have"
]
},
{
"cell_type": "markdown",
- "id": "a458cbaa",
- "metadata": {},
+ "id": "6c3f6965",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"C(\\boldsymbol{\\beta}=\\frac{\\vert\\vert (\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta})\\vert\\vert_2^2}{2\\sigma^2}+\\lambda\\vert\\vert\\boldsymbol{\\beta}\\vert\\vert_1,\n",
@@ -1126,16 +1330,20 @@
},
{
"cell_type": "markdown",
- "id": "0d6393f7",
- "metadata": {},
+ "id": "24532f00",
+ "metadata": {
+ "editable": true
+ },
"source": [
"which is our Lasso cost function!"
]
},
{
"cell_type": "markdown",
- "id": "61d3f0f4",
- "metadata": {},
+ "id": "7634be1d",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Why resampling methods\n",
"\n",
@@ -1151,8 +1359,10 @@
},
{
"cell_type": "markdown",
- "id": "e174c24e",
- "metadata": {},
+ "id": "5dcc9cb2",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Resampling methods\n",
"Resampling methods are an indispensable tool in modern\n",
@@ -1177,8 +1387,10 @@
},
{
"cell_type": "markdown",
- "id": "8fafe562",
- "metadata": {},
+ "id": "0971b192",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Resampling approaches can be computationally expensive\n",
"\n",
@@ -1201,8 +1413,10 @@
},
{
"cell_type": "markdown",
- "id": "710b8f05",
- "metadata": {},
+ "id": "00c350e4",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Why resampling methods ?\n",
"**Statistical analysis.**\n",
@@ -1216,8 +1430,10 @@
},
{
"cell_type": "markdown",
- "id": "7584aa1d",
- "metadata": {},
+ "id": "aa3988c6",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Statistical analysis\n",
"\n",
@@ -1234,8 +1450,10 @@
},
{
"cell_type": "markdown",
- "id": "d34611c4",
- "metadata": {},
+ "id": "9868a98b",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Resampling methods\n",
"\n",
@@ -1261,8 +1479,10 @@
},
{
"cell_type": "markdown",
- "id": "5ccf4082",
- "metadata": {},
+ "id": "5015585e",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Resampling methods: Bootstrap\n",
"Bootstrapping is a [non-parametric approach](https://en.wikipedia.org/wiki/Nonparametric_statistics) to statistical inference\n",
@@ -1284,8 +1504,10 @@
},
{
"cell_type": "markdown",
- "id": "812c5c00",
- "metadata": {},
+ "id": "92d9273d",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## The Central Limit Theorem\n",
"\n",
@@ -1302,8 +1524,10 @@
},
{
"cell_type": "markdown",
- "id": "188b97ba",
- "metadata": {},
+ "id": "cd12bb37",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"z=\\frac{x_1+x_2+\\dots+x_m}{m},\n",
@@ -1312,16 +1536,20 @@
},
{
"cell_type": "markdown",
- "id": "6c308a4b",
- "metadata": {},
+ "id": "cd8fc890",
+ "metadata": {
+ "editable": true
+ },
"source": [
"the question we pose is which is the PDF of the new variable $z$."
]
},
{
"cell_type": "markdown",
- "id": "8b9a07db",
- "metadata": {},
+ "id": "265ab8d1",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Finding the Limit\n",
"\n",
@@ -1333,8 +1561,10 @@
},
{
"cell_type": "markdown",
- "id": "24fa8320",
- "metadata": {},
+ "id": "cdcb3e67",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\tilde{p}(z)=\\int dx_1p(x_1)\\int dx_2p(x_2)\\dots\\int dx_mp(x_m)\n",
@@ -1344,8 +1574,10 @@
},
{
"cell_type": "markdown",
- "id": "7ebaab95",
- "metadata": {},
+ "id": "89a036fe",
+ "metadata": {
+ "editable": true
+ },
"source": [
"where the $\\delta$-function enbodies the constraint that the mean is $z$.\n",
"All measurements that lead to each individual $x_i$ are expected to\n",
@@ -1355,8 +1587,10 @@
},
{
"cell_type": "markdown",
- "id": "81419b24",
- "metadata": {},
+ "id": "db8fc317",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Rewriting the $\\delta$-function\n",
"\n",
@@ -1365,8 +1599,10 @@
},
{
"cell_type": "markdown",
- "id": "ce150f6c",
- "metadata": {},
+ "id": "8f878995",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\delta(z-\\frac{x_1+x_2+\\dots+x_m}{m})=\\frac{1}{2\\pi}\\int_{-\\infty}^{\\infty}\n",
@@ -1376,8 +1612,10 @@
},
{
"cell_type": "markdown",
- "id": "416547dc",
- "metadata": {},
+ "id": "705c200b",
+ "metadata": {
+ "editable": true
+ },
"source": [
"and inserting $e^{i\\mu q-i\\mu q}$ where $\\mu$ is the mean value\n",
"we arrive at"
@@ -1385,8 +1623,10 @@
},
{
"cell_type": "markdown",
- "id": "b3f00e54",
- "metadata": {},
+ "id": "594730bc",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\tilde{p}(z)=\\frac{1}{2\\pi}\\int_{-\\infty}^{\\infty}\n",
@@ -1397,16 +1637,20 @@
},
{
"cell_type": "markdown",
- "id": "85890d2e",
- "metadata": {},
+ "id": "50cedd29",
+ "metadata": {
+ "editable": true
+ },
"source": [
"with the integral over $x$ resulting in"
]
},
{
"cell_type": "markdown",
- "id": "a5e6936c",
- "metadata": {},
+ "id": "6d7b9a46",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\int_{-\\infty}^{\\infty}dxp(x)\\exp{\\left(iq(\\mu-x)/m\\right)}=\n",
@@ -1417,8 +1661,10 @@
},
{
"cell_type": "markdown",
- "id": "a040e4f8",
- "metadata": {},
+ "id": "ffeff7ca",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Identifying Terms\n",
"\n",
@@ -1428,8 +1674,10 @@
},
{
"cell_type": "markdown",
- "id": "7756f6b9",
- "metadata": {},
+ "id": "caf853d3",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\int_{-\\infty}^{\\infty}dxp(x)e^{\\left(iq(\\mu-x)/m\\right)}=\n",
@@ -1439,16 +1687,20 @@
},
{
"cell_type": "markdown",
- "id": "a98a8f7f",
- "metadata": {},
+ "id": "b0237a6a",
+ "metadata": {
+ "editable": true
+ },
"source": [
"resulting in"
]
},
{
"cell_type": "markdown",
- "id": "8d194e5a",
- "metadata": {},
+ "id": "4475d0be",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\left[\\int_{-\\infty}^{\\infty}dxp(x)\\exp{\\left(iq(\\mu-x)/m\\right)}\\right]^m\\approx\n",
@@ -1458,16 +1710,20 @@
},
{
"cell_type": "markdown",
- "id": "ecbd1cac",
- "metadata": {},
+ "id": "1ba48e74",
+ "metadata": {
+ "editable": true
+ },
"source": [
"and in the limit $m\\rightarrow \\infty$ we obtain"
]
},
{
"cell_type": "markdown",
- "id": "4ef2da67",
- "metadata": {},
+ "id": "6561a4db",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\tilde{p}(z)=\\frac{1}{\\sqrt{2\\pi}(\\sigma/\\sqrt{m})}\n",
@@ -1477,8 +1733,10 @@
},
{
"cell_type": "markdown",
- "id": "f6b31b60",
- "metadata": {},
+ "id": "a0416060",
+ "metadata": {
+ "editable": true
+ },
"source": [
"which is the normal distribution with variance\n",
"$\\sigma^2_m=\\sigma^2/m$, where $\\sigma$ is the variance of the PDF $p(x)$\n",
@@ -1487,8 +1745,10 @@
},
{
"cell_type": "markdown",
- "id": "7c8fc04f",
- "metadata": {},
+ "id": "5b86aaee",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Wrapping it up\n",
"\n",
@@ -1504,8 +1764,10 @@
},
{
"cell_type": "markdown",
- "id": "60e5f2a3",
- "metadata": {},
+ "id": "4620502c",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\sigma_m=\n",
@@ -1515,8 +1777,10 @@
},
{
"cell_type": "markdown",
- "id": "a8fac659",
- "metadata": {},
+ "id": "03867622",
+ "metadata": {
+ "editable": true
+ },
"source": [
"The latter is true only if the average value is known exactly. This is obtained in the limit\n",
"$m\\rightarrow \\infty$ only. Because the mean and the variance are measured quantities we obtain \n",
@@ -1525,8 +1789,10 @@
},
{
"cell_type": "markdown",
- "id": "df92a53f",
- "metadata": {},
+ "id": "c4908940",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\sigma_m\\approx \n",
@@ -1536,8 +1802,10 @@
},
{
"cell_type": "markdown",
- "id": "f96a2ca7",
- "metadata": {},
+ "id": "c74f36e7",
+ "metadata": {
+ "editable": true
+ },
"source": [
"In many cases however the above estimate for the standard deviation,\n",
"in particular if correlations are strong, may be too simplistic. Keep\n",
@@ -1554,8 +1822,10 @@
},
{
"cell_type": "markdown",
- "id": "659bc37e",
- "metadata": {},
+ "id": "e497bcf9",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Confidence Intervals\n",
"\n",
@@ -1575,8 +1845,10 @@
},
{
"cell_type": "markdown",
- "id": "da802422",
- "metadata": {},
+ "id": "60541de0",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Standard Approach based on the Normal Distribution\n",
"\n",
@@ -1588,8 +1860,10 @@
},
{
"cell_type": "markdown",
- "id": "2685818c",
- "metadata": {},
+ "id": "62737ab8",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\left(\\mu_{\\beta}\\pm \\frac{z\\sigma_{\\beta}}{\\sqrt{n}}\\right),\n",
@@ -1598,8 +1872,10 @@
},
{
"cell_type": "markdown",
- "id": "ca68cc22",
- "metadata": {},
+ "id": "96722498",
+ "metadata": {
+ "editable": true
+ },
"source": [
"where $z$ defines the level of certainty (or confidence). For a normal\n",
"distribution typical parameters are $z=2.576$ which corresponds to a\n",
@@ -1616,8 +1892,10 @@
},
{
"cell_type": "markdown",
- "id": "3abf49cc",
- "metadata": {},
+ "id": "c1242f98",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Resampling methods: Bootstrap background\n",
"\n",
@@ -1634,8 +1912,10 @@
},
{
"cell_type": "markdown",
- "id": "a1768cab",
- "metadata": {},
+ "id": "9d6890f4",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Resampling methods: More Bootstrap background\n",
"\n",
@@ -1656,8 +1936,10 @@
},
{
"cell_type": "markdown",
- "id": "eb39b3e2",
- "metadata": {},
+ "id": "eaebe344",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Resampling methods: Bootstrap approach\n",
"\n",
@@ -1675,8 +1957,10 @@
},
{
"cell_type": "markdown",
- "id": "ba467e0a",
- "metadata": {},
+ "id": "17ee9799",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Resampling methods: Bootstrap steps\n",
"\n",
@@ -1703,8 +1987,10 @@
},
{
"cell_type": "markdown",
- "id": "07eb7309",
- "metadata": {},
+ "id": "ea87812a",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Code example for the Bootstrap method\n",
"\n",
@@ -1725,19 +2011,12 @@
{
"cell_type": "code",
"execution_count": 1,
- "id": "1f8a2b73",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Bootstrap Statistics :\n",
- "original bias std. error\n",
- " 99.9942 14.9454 99.992 0.14958\n"
- ]
- }
- ],
+ "id": "a21ecc6d",
+ "metadata": {
+ "collapsed": false,
+ "editable": true
+ },
+ "outputs": [],
"source": [
"%matplotlib inline\n",
"\n",
@@ -1771,16 +2050,20 @@
},
{
"cell_type": "markdown",
- "id": "afc2dfd8",
- "metadata": {},
+ "id": "79da869e",
+ "metadata": {
+ "editable": true
+ },
"source": [
"We see that our new variance and from that the standard deviation, agrees with the central limit theorem."
]
},
{
"cell_type": "markdown",
- "id": "6d6ebddb",
- "metadata": {},
+ "id": "ce9b96bc",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Plotting the Histogram"
]
@@ -1788,22 +2071,12 @@
{
"cell_type": "code",
"execution_count": 2,
- "id": "5a3c437b",
- "metadata": {},
- "outputs": [
- {
- "data": {
- "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAArnklEQVR4nO3dd3xUVfrH8c+TipAgPQkYigpIkRoRddXEQlEQCwpYKAsiKCp20d+qqCuuHURXEASC0qQIlhVQieAiICAlgFRpAtJLKJKE8/vjDhKzIXXunCnP+/W6r8zcuTPzPWHCM+fec88VYwxKKaVCV5jtAEoppezSQqCUUiFOC4FSSoU4LQRKKRXitBAopVSIi7AdoKgqVapkatasaTuG1xw9epQyZcrYjuET2tbgpG0NDEuWLNlrjKmc12MBVwhq1qzJ4sWLbcfwmrS0NJKTk23H8Alta3DStgYGEdlytsd015BSSoU4LQRKKRXitBAopVSI00KglFIhTguBUkqFOC0ESikV4rQQKKVUiNNCoJRSIU4LgVJKhbiAO7NYKb+WkuL87NIFBg48s37OHDt5lCoE7REopVSI00KglFIhTguBUkqFOD1GoFRxnD4WUNLt9diB8gPaI1BKqRCnPQKl9Nu6CnHaI1DKC7JNGEuO1OFwVmnbUZQqMu0RKFVMxsCiI/UY9/u1TNqTTNnwo+w4WYlGZTZRO+YPIg+tpEXsGiLDsm1HVSpfWgiUKqI1a+DjTT2ZsDuFCMmmS5XvSGvyCHVLb+N4dhQ/HLqYkSfvod/6h9h0PIHkcst57YJh1C29zXZ0pfKkhUCpIvjoI3j6aehWKoJPGwykacx6RM48fk74Sa6vsITI2+qQfHI8u0+W45Pfr+Pqn9/h0wYvcGW5lfbCK3UWWgiUKqQhQ+DNN2HePKjbZ1ihnlMl6iCPJE6mQZnN3LZqIO/WfpdOVfQgtPIvWgiUKoAx8MorMHo0zJ0LNWoU/TVaVVjM7MZP0H7lP9l8Io4nEyc4PQkdsaT8gI4aUiofxsCAATB+fPGLwGmNYzYyv2k/xv1+Hfev70/WKf3zU/5BP4lKncWpU/DAA/Dtt/D995CQUPLXPK/UXuY1fYiNx6tyc/rLHM0uVfIXVaqEtBAolYdTRujRA9LTnUJQsaL3XrtsxDG+vHgApcP/oP+GB7z3wkoVkxYCpfIw9LdbWLcOvv4aypb1/utHhmUzou7rfHugGdP3XuH9N1CqCLQQKJXL+mPVeHFzV1JTobSLJwqXjTjG2HqDuG/to+z6o7x7b6RUAVwrBCKSKCJzRGS1iKwSkYfz2CZZRA6JyDLP8pxbeZQqjGwTRo9fnuIfNcdSu7b773fFuen0SviSnmufxBj330+pvLjZI8gCHjPG1AdaAg+ISP08tptnjGniWV50MY9SBRqy/VbC5BQPVpvqs/d8vuYYfj9Zng923OSz91QqJ9cKgTFmpzFmqef2EWANUM2t91OqpNYeS+SfW+5m1EWvESa++3oeGZbNx/X+yXObe7D2WKLP3lep08T4oD8qIjWBuUBDY8zhHOuTgSnAdmAH8LgxZlUez+8N9AaIi4trPmHCBNcz+0pGRgYxMTG2Y/iE37Z13TqyTwkPv96ea1ts5JaU//kIFllGhQrE7N9fpOd8llafr+fXYehT04mo54P9Ul7it/+uLgjktqakpCwxxiTl9ZjrhUBEYoDvgX8aY6bmeqwscMoYkyEiNwCDjTH5/gUkJSWZxYsXuxfYx9LS0khOTrYdwyf8tq0pKbyx9Q6+3N+Sbxs/5pXeQFqXLiSPH1+k5xgDN64cRPPYdby0uWuJM/iK3/67uiCQ2yoiZy0Ero4aEpFInG/8n+QuAgDGmMPGmAzP7a+ASBGp5GYmpXL75Wgi/9rWhZF1X/fpLqHcROCjuq/x4Y52LFxoLYYKQW6OGhJgJLDGGPPWWbaJ92yHiLTw5NnnVialcsvOhu6/PM3AmqM5/5ydtuMQH32AV88fzuOPo6OIlM+42SO4ArgHuCbH8NAbRKSPiPTxbNMRSBeR5cAQoLPxxUELpTzGjoXIsCz6VJ1hO8qf7omfzb59zslsSvmCa7OPGmN+AKSAbYYCQ93KoFR+MjPhxRdhTK0RVncJ5RYup3jpJXj2WWjdGsL0tE/lMv2IqZA1ahRceCF+ebGYW2+F8HCYPNl2EhUKtBCokPTHH/Dyy06PwB+JONdA+Mc/ICvLdhoV7LQQqJD04Ydw8cXQsqXtJGd33XVQtSqMGWM7iQp2WghUyDl+HAYN8t/ewGkiTs6BA+HECdtpVDDTQqBCzgcfQIsW0Ly57SQFa9kSmjZ1MivlFr1msQopR4/Ca6/BzJm2kxTeyy/D9ddDz54QG2s7jQpGWghU6EhJYejWzlyVXYdGD/v5fqEcLr7YOV7wzjvOwWOlvE13DamQcTirNG9uu4MXao62HaXIBg6EwYNhn553r1yghUCFjMHbb6NV+cXUK7PVdpQiu+AC6NgR3n7bdhIVjLQQqJBw8KBTCJ6vGbhjMR97DIYPd0Y9KeVNWghUSBg+HNpWXEjt0r/ZjlJstWs7o4g+/th2EhVstBCooJeVBe+9B/3Pm2I7Son17+8cNNapGZU3aSFQQW/6dEhMhOax62xHKbGUFGcOotmzbSdRwUQLgQp6Q4bAQw/ZTuEdImd6BUp5ixYCFdSWLYNNm+CWW2wn8Z4774SlS+GXX2wnUcFCTyhTQW3IELj/foiMtJ2kiFJS8l4/Zw6lSsF99znnFfz7376NpYKT9ghU0NqzB6ZNg3vvtZ3E+/r2hQkTYP9+20lUMNBCoILW8OFw221QqZLtJN4XHw8dOjjTaStVUloIVFDKzIT334cHH7SdxD0PPwxDhzptVaoktBCooDRlinMCVuPGtpO4p2lTZ+qJKYF/eoSyTAuBCkpDhjjfmINd//7O/EN6gpkqCS0EKuj89BPs2AE33WQ7ifvat4e9e2HBAttJVCDT4aMq6AxpP4t+ZTYSft0k21G8L9ew0nDgQbmNoUP7cdlldiKpwKc9AhVUdu2CL/ZdRs+Er2xH8Zmu8bP48ksdSqqKTwuBCiojR8Ltlb+nfGSG7Sg+UyHyCDfcoLOSquLTXUMqcOXaTXLKCB8tHMuE+l9YCmTPvfc68yk9+KAzH5FSRaE9AhU00g42ISb8BEmxa21H8bnkZDhxAhYutJ1EBSItBCpojNh5I70SvgzJb8Qi0LMnjBhhO4kKRK4VAhFJFJE5IrJaRFaJyP+M6hbHEBHZICIrRKSZW3lUcNufGctX+y7lrrhvbEexpnt35+SyI0dsJ1GBxs0eQRbwmDGmPtASeEBE6ufapi1Q27P0BnQuRVUsn/x+HTdUXEiFyND9XzA+3jlsMn687SQq0LhWCIwxO40xSz23jwBrgGq5NusApBrHAqCciCS4lUkFJ2PO7BYKdb166e4hVXQ+GTUkIjWBpkDuQ1nVgG057m/3rNuZ6/m9cXoMxMXFkZaW5lZUn8vIyAiq9uTH623t0gWAtZsrsWdVZehzEWlhF3nv9Usgo0IF0jz5fMLze42Ohi1bWjJixEouvPCoT95aP8OBz/VCICIxwBSgvzHmcHFewxgzHBgOkJSUZJKTk70X0LK0tDSCqT358XpbBw4EYMLaR3ggdhLXTPSffSJpXbqQ7Mt9NHPm/Hmzb19YvvwSevXyzVvrZzjwuTpqSEQicYrAJ8aYqXls8huQmOP+eZ51ShXK0exSTNqTTPf4mbaj+I0ePWDcODh+3HYSFSjcHDUkwEhgjTHmrbNsNgPo6hk91BI4ZIzZeZZtlfofk/dczeVlV1Eteq/tKH6jRg1o0QImT7adRAUKN3sEVwD3ANeIyDLPcoOI9BGRPp5tvgI2ARuAD4H7XcyjgtCInTfoQeI83HuvHjRWhefaMQJjzA9Avqf2GGMM8IBbGVRwW3sskQ3Hq3FjRZ2DObd27eD++2HtWqhb13Ya5e90riEVsEbuvIGucbOIDMu2HcW+XPMuRQHdonozYkQXXn/dTiQVOHSKCRWQMjMhdVerkJpuuqj+nvAfPv4YsrJsJ1H+TguBCkhffAF1S2+jTunttqP4rbqlt1GjBsyaZTuJ8ndaCFRAGj0aesT/x3YMv9etG4wZYzuF8ndaCFTA2bMH5s6F2yrPtR3F73XqBDNnwoEDtpMof6aFQAWcceOci7bHRugZUwWpUAFatYJJQXj5ZuU9WghUwBkzBrp2tZ0icHTr5uxKU+pstBCogLJypbNrKNdoSZWP1q3h11+dcwqUyosWAhVQUlPhnnsgPNx2ksAREQF33eX87pTKixYCFTCysuCTT5xdHapounWDsWPh1CnbSZQ/KtSZxSLSHvjSGKMfI2XN7NlQvbpOmVAknn1ojYBKB4Yxp9kwri2/9C/TVitV2B5BJ2C9iLwmIv5x5Q8VcsaM0d5ASXSLn8XoXa1tx1B+qFCFwBhzN84VxjYCo0XkRxHpLSKxrqZTyuPgQfj6a2dcvCqeO6t8w+d7L+dI1jm2oyg/U+hjBJ6ri00GJgAJwC3AUhF50KVsSv1p0iS47jpnXLwqnspRh7i63HIm77nadhTlZwpVCESkg4hMA9KASKCFMaYt0Bh4zL14Sjl0t5B3dIufyRjdPaRyKWyP4FbgbWPMxcaY140xuwGMMceAnq6lUwrYsMFZ2rSxnSTwtav4I6uO1eTXX20nUf6ksIVglzHmLxO7iMi/AIwx33o9lVI5pKbCnXdCZKTtJIEvKiyLzlW+03MK1F8UthBcn8e6tt4MolReTp2C1Nd20XXevc5QyJyLKpZucTNJTQVjbCdR/iLfQiAifUVkJXCRiKzIsfwKrPBNRBXK5s6FshFHaRKzwXaUoNE8dh3R0TB/vu0kyl8UdELZOOA/wCDg6Rzrjxhj9ruWSimP1FToGjcLyffq16ooRKDrieGk3p7AFXXf+uuDeqJZSCpo15AxxmzGucD8kRwLIqID+ZSrjh2DadPgrrhvbEcJOnfFfcPkPVdxIlsPvKiCC8E4z88lwGLPzyU57ivlms8+g5YtISFaO5/ellhqD01jNvD5vsttR1F+IN9CYIxp5/lZyxhzvufn6eV830RUoSo1Vc8dcFPX+Fmk/t7KdgzlB/I9RiAizfJ73Biz1LtxlHLs2AGLFjm7hhhmO01wurXSXB5a/yC7T5ajStRB23GURQUdLH4zn8cMcI0Xsyj1p08+gVtvhXN0WhzXxESc4KZK8xm/+1oePm+K7TjKonwLgTFGB2srnzPGmVLi/fdtJwl+XeNm8tSm+7QQhLiCdg1dY4z5TkRuzetxY8xUd2KpULZsGRw9Cn/7m+0kwS+l/DJ+P1me9IyaNIzZbDuOsqSgXUNXA98B7fN4zABaCJTXnb4cZZheP8914XKKu+NmM/b3VvwrZrjtOMqSgnYNPe/52aOoLywiHwHtgN3GmIZ5PJ4MTAdOT3811RjzYlHfRwWXzEwYNw5++MF2ktBxT/xsWi1/jVfOH4FeCjo0FXYa6ooiMkRElorIEhEZLCIVC3jaaKCg+SLnGWOaeBYtAopZs+CCC6B2bdtJQkeDMptJiNrPdwea2o6iLCls53sCsAe4DejouT0xvyd4ZivVM4FUkaSmQteutlOEHj2nILSJKcQUhCKSnnv3joisNMZcXMDzagJf5LNraAqwHdgBPG6MWXWW1+kN9AaIi4trPmHChAIzB4qMjAxiYmJsx/CJgtqakRFB584tGTduAWXLZp15YN06H6TzrowKFYjZHzjfgw4cLsU9z3Vi0uRFlC6dXaTn6mc4MKSkpCwxxiTl9VhBB4tPmyUinYFJnvsdgZklzLUUqGGMyRCRG4DPgDx3CBhjhgPDAZKSkkxycnIJ39p/pKWlEUztyU9Bbf3wQ+fiMzfdlGu40MCB7gZzQVqXLiSPH287RpFcU/oC9uy5sshnc+tnOPAVNA31ERE5DNyLM+/QSc8yAc839OIyxhw2xmR4bn8FRIpIpZK8pgpsulvIrq5xsxgzxnYKZUNBcw3FGmPKen6GGWMiPEuYMaZsSd5YROJFnMmFRaSFJ8u+krymClwbN8LatXo5SpvaVfyRFStgyxbbSZSvFXbXECJSHmfXTanT63JfvjLX9uOBZKCSiGwHnse58D3GmA9wdi/1FZEs4DjQ2RTmgIUKSmPHQueoqUS1ftd2lJBVKjyTTp2cntk//mE7jfKlQhUCEekFPAycBywDWgI/ks9cQ8aYLvm9pjFmKDC0sEFV8Dp1yplSYnJ8SQ87qZLq0QM6dYJnn9UT+kJJYf+pHwYuAbZ45h9qChx0K5QKLd9/D7Gx0Cwm8EYHBZvmzZ2J/vSEvtBS2F1DJ4wxJ0QEEYk2xvwiInVdTaZCxqhRzjdRmWE7iZJrUuh+/A5Gd6rJVRe9duYBvYRlUCtsj2C7iJTDGeI5W0SmA3pISZXY4cMwYwbcfbftJOq0u+NmM23v38jIKlXwxiooFKoQGGNuMcYcNMa8APwDGAnc7GIuFSImTYKUFKhc2XYSdVp89AH+du5Kpuy92nYU5SNFGTXUDPgbzqyj/zXGnHQtlQpuKWcuczFq6bs8VX08pMy3GEjl1j1+JkN/u5luegA/JBR20rnngDFARaASMEpE/s/NYCr4rT2WyMbjVWlbYaHtKCqX9hXnk360FpuOJ9iOonygsMcI7gIuMcY875mauiVwj3uxVCgYvas1d8fNJjKsaHPbKPdFhWXRpcp3pO7SiehCQWELwQ5ynEgGRAO/eT+OChXZJozUXa3oHv+17SjqLLrHf83oXW04ZcR2FOWygi5V+S7OMYFDwCoRme25fz2wyP14KljN3t+cqtH79PKIfqxpzHrOjTjK9wcboxcvD24FHSxe7Pm5BJiWY32aK2lUyBi1qy09tDfg10TO9Aq0EAS3gi5V+edchCISBdTx3F1rjMl0M5gKXvszY5m5/xI+qPOW7SiqAHfFfcPAzd04csQ5+1sFp8KOGkoG1gPvAe8D60TkKvdiqWA2fve1tKmwiPKRGbajqAJUiTpIcrllfPqp7STKTYU9WPwm0MoYc7Ux5iqgNfC2e7FUMBu1sw09EnS3UKDoHv81o0bZTqHcVNhCEGmMWXv6jjFmHZ4ppZUqipUr4ffM8lxXfontKKqQbqy4gPXr4ZdfbCdRbilsIVgiIiNEJNmzfMiZA8lKFdqoUc6VsMLllO0oqpAiw7Lp0cO5lKgKToUtBH2A1cBDnmU10NetUCo4nTwZxscfQ4/4/9iOooqoVy/ngjUnTthOotxQ4FxDIhIOLDfGXAToMA9VbHPnVqJxY7gwa4ftKKqILrgAmjSBadOgS76XnFKBqMAegTEmG1grItV9kEcFsc8/r8p999lOoYqrd28YPtx2CuWGwu4aKo9zZvG3IjLj9OJmMBVc1qyB7dvPoUMH20lUcXXoAKtXwzq9kFzQKew01Hopa1Uiw4dDmza7iIysYTuKKqaoKOje3Tlo/PrrttMob8q3RyAipUSkP3A7cBHOdQi+P734IqAKfMePw9ix0K7dTttRVAn16gVjxsAff9hOorypoF1DY4AkYCXQFufEMqWK5NNPISkJEhJ0yEmgq10bLr4YPvvMdhLlTQUVgvrGmLuNMcOAjsCVPsikgsywYdCnj+0Uylvuu08PGgebggrBnxPLGWOyXM6iglB6OmzeDO3a2U6iSiQl5c/l5veuJ33eftZferftVMpLCioEjUXksGc5AjQ6fVtEDvsioApsw4ZBz54QUeirYyt/FxWWRbf4WYzYeaPtKMpLCpqGOtxXQVTwOXYMxo2Dn3+2nUR5W6+EL7ny58G8dNJ2EuUNhT2PQKkimzgRLrsMquupiEGnTuntNCizmenTbSdR3qCFQLlm2DD0TOIg1jvhC4YNs51CeYNrhUBEPhKR3SKSfpbHRUSGiMgGEVkhIs3cyqJ8b/ly+O03aNvWdhLlllsq/0B6OmzdWtp2FFVCbvYIRgNt8nm8LVDbs/QG/u1iFuVjw4ZBr4hRRFx/ZrQJ69Y5P1VQiA7LpHdvmDKlmu0oqoRcG8thjJkrIjXz2aQDkGqMMcACESknIgnGGD39NMAdPAgTJsDKi76yHUW5rG9fqFu3CgcOQPnyttOo4rI5qK8asC3H/e2edf9TCESkN06vgbi4ONLS0nyRzycyMjKCqj0AEycm0qxZDOvvuJ71OdZnVKhAWojMYRwybV2bRvPmFzJgwB907ryt4O0DXDD+vYLdQlBoxpjhwHCApKQkk5ycbDeQF6WlpRFM7cnKgm7dYMoUSHpi/F8eS+vSheTx48/yzOASMm2dM4e1axfzyitJDB16QdCfLxJsf6+n2Rw19BuQmOP+eZ51KoBNneoMF01Ksp1E+UrduhkkJur8Q4HMZiGYAXT1jB5qCRzS4wOB7+234ZFHbKdQvta/PwwebDuFKi43h4+OB34E6orIdhHpKSJ9ROT09GNfAZuADcCHwP1uZVG+sWAB/P47evGZEHTzzbB1KyxZYjuJKg43Rw3le6TMM1roAbfeX/ne22/DQw9BuE5MEnIiIqBfP6dXkJpqO40qKj2zWHnF1q3wzTfw97/bTqJ8Ksf5Ib2mt+fz8UfYeflter5IgNFCoLzi3Xed0UJly9pOomwpH5lB5yrf8cGOm2xHUUWkhUCVWEYGfPSRs1tIhbaHqk3lgx3tOZEdaTuKKgItBKrERo1y9gTUrGk7ibKtXpmtNI3ZwITd19iOoopAC4Eqkexs5wChDhlVp/U/bzLvbO+IMbaTqMLSQqBK5IsvoEIFuPxy20mUv2hVYTGZJoJZs2wnUYWlhUCVyFtvOb0BEdtJlL8IE8OzNT7mpZfQXkGA0EKgim3ePNi2DTp2tJ1E+ZtOVeawezcE4fxsQSnIp4hSbho4EJ59FiJb6Zhx9VfhcopnnoGXXtJTCgKB9ghUsfzwA2zcCF272k6i/NVdd8Hmzc5nRfk3LQSqWP7sDehwcXUWkZEwYIDTK1D+TQuBKrL582H9eu0NqIJ16wZr1sCiRbaTqPxoIVBFdro3EBVlO4nyd1FR8NRT2ivwd3qwWBXJjz/C2rXONz2l8uU5StwzO5JXFn7Cz0nP0DR2A8yZYzmYyk17BKrwUlIYeOMinol+k6jWKc4fug4JUQUoFZ7JE4kTeWnLPbajqLPQQqAKbeHheqw5Vp3u8V/bjqICTO+qXzD/UENWZtSyHUXlQQuBKrSBm7syoPo4osKybEdRAaZ0+B88mvgp/9xyt+0oKg9aCFShLFoE6Udr0SNBewOqePpWnc53B5uSnm47icpNC4EqlIEDYUD1cUSHZdqOogJUbMRxnq3xCY8/bjuJyk0LgSrQd9/B6tXw94T/2I6iAlzfqtPZtAm+1o6lX9FCoPKVlQX9+8Mbb6C9AVViUWFZvBH9LI/dtpmsq689M/JMR59ZpYVA5WvECOd6A7feajuJChbtK84nLmo/H+5sZzuK8tBCoM7qwAF44QV45x293oDyHhF464L3Gbi5G4eyytiOo9BCoPLx4ovQoQM0aWI7iQo2TWI3cmPFBTqc1E9oIVB5+uUX+PhjnSNGueflWiMZubMtm44n2I4S8nSuIfVXnoN2j64YxIDyS6nS6VPLgVSwSojezyPnTeapTb35tMFA23FCmvYI1P/4at+lbDxelX7VptmOooLcY4mTWHi4Hj8cbGg7SkjTQqD+IvNUOI9uuJ83L/i3TiWhXHdO+EkGnT+CRzY+wKlTttOELlcLgYi0EZG1IrJBRJ7O4/HuIrJHRJZ5ll5u5lEFe++3m6lZahc3VlxgO4oKEV2qfEuEZDNihO0kocu1QiAi4cB7QFugPtBFROrnselEY0wTz6IfBYt27oR/br2bty58X4eLKp8JE8OIum/w7LOwZYvtNKHJzYPFLYANxphNACIyAegArHbxPVUxGQO9ekHfqjOoX0b/GpVvNSizmUdjP6RX06bMavTEX7+I6IVsXOfmrqFqwLYc97d71uV2m4isEJHJIpLoYh6Vj5EjnR7B/9UYazuKClFPJE7gUFYZhu9sbztKyBFjjDsvLNIRaGOM6eW5fw9wqTGmX45tKgIZxpg/ROQ+oJMx5po8Xqs30BsgLi6u+YQJE1zJbENGRgYxMTFWM+zcWYq+fZvx9tvLqZX5s2vvk1GhAjH797v2+v5E21o8m3eUo/+b7flgwDTiK2U4K+vU8cpre4M//L0WV0pKyhJjTFJej7lZCC4DXjDGtPbcHwBgjBl0lu3Dgf3GmHPze92kpCSzePFib8e1Ji0tjeTkZGvvf+qUc+pAu3bwxBO4OvlXWpcuJI8f79rr+xNta/G9uqUL3xxozuzGjzu7iPxo15Dtv9eSEJGzFgI3dw39BNQWkVoiEgV0BmbkCpbzlMKbgDUu5lF5eOcdpxg8+qjtJEo5Hk+cyJHs0gzbobuIfMW1g8XGmCwR6QfMBMKBj4wxq0TkRWCxMWYG8JCI3ARkAfuB7m7lUf9r9WoYNAgWLoTwcNtplHJEhJ1i1EX/4upl79CmwiJq2g4UAlydYsIY8xXwVa51z+W4PQAY4GYGlbfMTOjaFV5+Gc4/33Yapf6qfpktPJ44kZ5rn2T2KQjTU19dpb/eEPXKK1C5MvTubTuJUnl77LxJHM0uxZtv2k4S/HTSuRA0bx68/8/9LE26D7lmr+04SuUpIuwUkxoM5NK3JtKoEbRubTtR8NIeQYj59Ve44w4YW28Q1aK1CCj/Vr3UbiZOdHZjbthgO03w0kIQQo4cgZtugmeegVYVgmcIrgpuVz2fwvPnvkOHxps5cuUNeo1jF2ghCBHZ2XDnnXDFFdCvX8HbK+VP+ladzhXnpnPPmmc4ZXQiLG/TQhAiBgyAjAx49129/rAKPCIwtPZg9mSW48XNXW3HCTp6sDhY5NNVHr2zNVO33MPC5vcT2eqwD0Mp5T1RYVlMafAclyz9gMYxG7nFdqAgoj2CIPffQw15ctN9fH7xM1SM1CKgAlt89AGmNniO3useY8UK22mCh/YIgtiGY1W5fdXzpF40iHplttqOo5RXXFJ2Le/VHkzrS/oxu9HjNIzZ/NcN/GhuokChhSBIrT5ag1bLX+elWqNoU/En23GU8qo7qqSRbcK4fsUbzGr0BBfH/Go7UkDTQhCElmdcQNsVr/La+cO4O/4b23GUckWXuO8IE0OrFa8zs9GTNIrZZDtSwNJCEGR+OlyXditf4b3ag+lYZa7tOEq5qlOVOQiGVstfZ2bjJ2kcs9F2pICkhSCI/PdQQ25Jf5GRdV+nfaUfbcdRyifuqJKGYGi9/DW+bvQkTWwHCkBaCILEnANN6LT6OT6u94qeNaxCzu1VvidMDG1WvMYXiyEpz8uvqLPRQhBo8jhf4NPdV/PA+uf4tMFAri633EIopey7rfJcIiSbtm1f5vXXoXt324kChxaCAPbHqUge39iHr/ZdyteNnqJZ7HrbkZSyqkOl/1J7ItxyCyxYAIMHQ3S07VT+T08oC1C/Ho/nbz8P4bc/KrGk+X1aBJTyqP9ACj9VuZHdU+ZyVeU1bLvsDp2krgBaCALQ9L1X0HLpe9xV5RumNHiecpFHbUdSyq+UjTjGlAbP07Hy97RY+m++PdDMdiS/pruGAkhmJgzY0IdP91zN9Ib/R8tz19iOpJTfEoEnqk+keew67lrzLL2fdyZfLFXKdjL/oz2CADF7NjRpAuuOn8fSpPu0CChVSNeU/5mfmvVh5Upo2BC++qrg54QaLQR+buNG6NAB+vaFQYNgesP/08njlCqi80rtZepUGDoUHn4Ybr4ZNm+2ncp/aCHwU0euvIGnq4/j0osOcXn6MFZVa8VNb6fotQSUKoE2bSA9HS65xDnX4OWX4cQJ26ns00Jg27p1Zy69l5JCxpVtea/OYC5aNIadJyuy4pKePFV9AtFhmbaTKhUUoqPh2WdhyRJnqVMH3nwTDodwR1sLgZ/YfDyOxzf0oeaCCXx7oCnTGj7HmHqvUjV6n+1oSgWlGjVg2jSYOhUWL4ZateCJJ2DbNtvJfE9HDVlkDCxfF8+76QNJO9iY7vEz+al5H2qds8t2NKWCz1nOJUiaM4fx42HLFucEtMaN4YYbnGMJSUmhcWlXLQQ+Zgz8/LPzLWTKFDh64CqeKj+GMRcNIiZCd1Yq5XOeAlEDeAt4rn4Zhv/Yns4/3ocIdOwIt98OzYL4VAQtBD6QnQ3zkx5i6t4rmbbnSiLDsri10jxGV57H0YebcM3Ez2xHVEp5lIs8ypPVJ/CEmcDPGbX5dPzVdBqcjAFaXFWOc86B5s0hIoj+9wyipvjY2U5ZnzOHo0dh0SKYPx/++1/48UeokfkQt1aexxcXD6BBmc1/djfTwpr4LLJSqvBEoFnseprFrueVWiNYlnEhb4TdT8+eznGEli3hyivhqqugRYvAPlFNC0EJGAO/nyzPmmM1WH2sJquP1mDRJbB6tbOf8Yor4N57YdQoiOt8r+24SqliEoGmsRu495afSO7dlH374IcfYN485wDzqlXQqJGzNGhwZqlSJTCOMbhaCESkDTAYCAdGGGNezfV4NJAKNAf2AZ2MMZvdzFQUxsChQ7B9u7Ns23bm59ql77L6WA0iJJt6pbdQr/RW6pXewp3ZD9K8+VpKhWfCYpxliO2WKKW8qWJF50TPDh2c+xkZzsij9HRnmTTJKQ5hYVC/vjMiKTERqlc/syQmQkyMfxQK1wqBiIQD7wHXA9uBn0RkhjFmdY7NegIHjDEXikhn4F9AJzfyZGbCsmVw5IizZGScuX3kCOzb97/L3r0Q9scxEqN3kxi9h/Oi95AYvZvLo/fQ/fzt1Cu9hcpRh9yIq5TyV3nsFo4Bkj0LABFgGsGukxVYc7gGW5LfYutWZ5fx5MmwdauzGAOVK0OlSmeWihWh3OepxIYfJzb8GGUjjhEb7ixN5rxN+fLeb5KbPYIWwAZjzCYAEZkAdAByFoIOwAue25OBoSIixhjj7TBHj8L91/5CbPgxYsKP//lLjgk/TmzEMS6KOELFyMPOEnGIigmHqVT9ELERx70dRSkVAkQgIXo/CdH7ITVX8UhwlqPZpdibea6z/H4ue7efy56T53JYyrAn81w2HU/gcHYZjmSfw5Gs0ry61jk24fWsLvyf67ywSEegjTGml+f+PcClxph+ObZJ92yz3XN/o2ebvbleqzfQ23O3LrDWldB2VAL2FrhVcNC2Bidta2CoYYypnNcDAXGw2BgzHBhuO4cbRGSxMSYkrrCqbQ1O2tbA5+YUE78BiTnun+dZl+c2IhIBnItz0FgppZSPuFkIfgJqi0gtEYkCOgMzcm0zA+jmud0R+M6N4wNKKaXOzrVdQ8aYLBHpB8zEGT76kTFmlYi8CCw2xswARgJjRWQDsB+nWISaoNzldRba1uCkbQ1wrh0sVkopFRh0GmqllApxWgiUUirEaSFwkYg8LCLpIrJKRPp71jUWkR9FZKWIfC4iZc/y3HIiMllEfhGRNSJymU/DF1EJ2/qI53npIjJeRPxq+i4R+UhEdnvOezm9roKIzBaR9Z6f5T3rRUSGiMgGEVkhInlOXiwizT2/lw2e7f1gogHvt1VESovIl57P8SoReTX3Nra48e+a43Vm5Hxdv2eM0cWFBWgIpAOlcQ7KfwNciDOa6mrPNn8HXjrL88cAvTy3o4ByttvkRluBasCvwDme+5OA7rbblCvjVUAzID3HuteApz23nwb+5bl9A/AfQICWwMKzvOYiz+Pi2b6t7Xa60VbPZyLFczsKmBesbc3xGrcC43K+rr8v2iNwTz2cD8sxY0wW8D3OB6QOMNezzWzgttxPFJFzcT6kIwGMMSeNMQd9EbqYit1WjwjgHM+5JKWBHS7nLRJjzFycUW05dcAp1nh+3pxjfapxLADKiUhCzid67pc1xiwwzv8cqTmeb5W32+r5TMzx3D4JLMU5p8g6b7cVQERigEeBl10J7RItBO5JB64UkYoiUhrnG0UisArnQwVwO3896e60WsAeYJSI/CwiI0SkjC9CF1Ox22qM+Q14A9gK7AQOGWNm+SR1ycQZY3Z6bu8C4jy3qwE5r3q73bMup2qe9flt409K0tY/iUg5oD3wrQsZvaWkbX0JeBM45lpCF2ghcIkxZg3ObKqzgK+BZUA2zi6S+0VkCRALnMzj6RE4XdZ/G2OaAkdxuql+qSRt9eyD7YBT/KoCZUTkbt8k9w7Pt/qQGIdd3LZ6envjgSHGMxGlvytqW0WkCXCBMWaaa6FcooXARcaYkcaY5saYq4ADwDpjzC/GmFbGmOY4fxgb83jqdmC7MWah5/5knMLgt0rQ1uuAX40xe4wxmcBU4HLfJS+230/vGvD83O1ZX9ipVc4rYBt/UpK2njYcWG+MecetkF5SkrZeBiSJyGbgB6COiKS5mtZLtBC4SESqeH5Wx3MAKce6MOD/gA9yP88YswvYJiJ1Pauu5a/Td/ud4rYVZ5dQS8/oEsFp6xrfpC6RnNOjdAOm51jf1TPKpCXOrq6dOZ/ouX9YRFp62tw1x/P9UbHbCiAiL+PMI9bfB1lLqiT/rv82xlQ1xtQE/obzZSjZN7FLyPbR6mBecEZIrAaWA9d61j0MrPMsr3Lm7O6qwFc5ntsE5/pmK4DPgPK22+NiWwcCv+AcaxgLRNtuT662jcc5fpGJ01vrCVTE2de9HmeUVAXPtoJzQaaNwEogKcfrLMtxO8nT3o3A0NO/G9uLt9uK883Z4BT3ZZ6ll+12uvXvmmNdTQJo1JBOMaGUUiFOdw0ppVSI00KglFIhTguBUkqFOC0ESikV4rQQKKVUiNNCoJRSIU4LgVJKhTgtBEqVkIhc4pmjvpSIlPHMu9/Qdi6lCktPKFPKCzzTKJQCzsGZJ2qQ5UhKFZoWAqW8QESicC7EcwK43BiTbTmSUoWmu4aU8o6KQAzOdNt+dalNpQqiPQKlvEBEZgATcK6rkGCM6Wc5klKFFmE7gFKBTkS6ApnGmHEiEg7MF5FrjDHf2c6mVGFoj0AppUKcHiNQSqkQp4VAKaVCnBYCpZQKcVoIlFIqxGkhUEqpEKeFQCmlQpwWAqWUCnH/D0TrHLt4poahAAAAAElFTkSuQmCC\n",
- "text/plain": [
- ""
- ]
- },
- "metadata": {
- "needs_background": "light"
- },
- "output_type": "display_data"
- }
- ],
+ "id": "1e35dddb",
+ "metadata": {
+ "collapsed": false,
+ "editable": true
+ },
+ "outputs": [],
"source": [
"# the histogram of the bootstrapped data (normalized data if density = True)\n",
"n, binsboot, patches = plt.hist(t, 50, density=True, facecolor='red', alpha=0.75)\n",
@@ -1818,8 +2091,10 @@
},
{
"cell_type": "markdown",
- "id": "786479ae",
- "metadata": {},
+ "id": "3bdc7fb8",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## The bias-variance tradeoff\n",
"\n",
@@ -1834,8 +2109,10 @@
},
{
"cell_type": "markdown",
- "id": "63793f63",
- "metadata": {},
+ "id": "8854ebad",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\boldsymbol{y}=f(\\boldsymbol{x}) + \\boldsymbol{\\epsilon}\n",
@@ -1844,8 +2121,10 @@
},
{
"cell_type": "markdown",
- "id": "90ad5852",
- "metadata": {},
+ "id": "677d1f32",
+ "metadata": {
+ "editable": true
+ },
"source": [
"where $\\epsilon$ is normally distributed with mean zero and standard deviation $\\sigma^2$.\n",
"\n",
@@ -1859,8 +2138,10 @@
},
{
"cell_type": "markdown",
- "id": "db9e2c41",
- "metadata": {},
+ "id": "4ffc6245",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"C(\\boldsymbol{X},\\boldsymbol{\\beta}) =\\frac{1}{n}\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2=\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right].\n",
@@ -1869,16 +2150,20 @@
},
{
"cell_type": "markdown",
- "id": "2b457128",
- "metadata": {},
+ "id": "6059ec46",
+ "metadata": {
+ "editable": true
+ },
"source": [
"We can rewrite this as"
]
},
{
"cell_type": "markdown",
- "id": "c4da8a10",
- "metadata": {},
+ "id": "1d29cec4",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\frac{1}{n}\\sum_i(f_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2+\\frac{1}{n}\\sum_i(\\tilde{y}_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2+\\sigma^2.\n",
@@ -1887,8 +2172,10 @@
},
{
"cell_type": "markdown",
- "id": "c1cb56f7",
- "metadata": {},
+ "id": "7ef17636",
+ "metadata": {
+ "editable": true
+ },
"source": [
"The three terms represent the square of the bias of the learning\n",
"method, which can be thought of as the error caused by the simplifying\n",
@@ -1902,8 +2189,10 @@
},
{
"cell_type": "markdown",
- "id": "a63e4646",
- "metadata": {},
+ "id": "17963e0a",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathbb{E}\\left[(\\boldsymbol{f}+\\boldsymbol{\\epsilon}-\\boldsymbol{\\tilde{y}})^2\\right],\n",
@@ -1912,16 +2201,20 @@
},
{
"cell_type": "markdown",
- "id": "89e31458",
- "metadata": {},
+ "id": "58693061",
+ "metadata": {
+ "editable": true
+ },
"source": [
"and adding and subtracting $\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]$ we get"
]
},
{
"cell_type": "markdown",
- "id": "011de0ba",
- "metadata": {},
+ "id": "ed10ae30",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathbb{E}\\left[(\\boldsymbol{f}+\\boldsymbol{\\epsilon}-\\boldsymbol{\\tilde{y}}+\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2\\right],\n",
@@ -1930,16 +2223,20 @@
},
{
"cell_type": "markdown",
- "id": "cedb2b42",
- "metadata": {},
+ "id": "2ca576c8",
+ "metadata": {
+ "editable": true
+ },
"source": [
"which, using the abovementioned expectation values can be rewritten as"
]
},
{
"cell_type": "markdown",
- "id": "2c5213f2",
- "metadata": {},
+ "id": "de708fa8",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathbb{E}\\left[(\\boldsymbol{y}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2\\right]+\\mathrm{Var}\\left[\\boldsymbol{\\tilde{y}}\\right]+\\sigma^2,\n",
@@ -1948,16 +2245,20 @@
},
{
"cell_type": "markdown",
- "id": "4f613fae",
- "metadata": {},
+ "id": "45ce65a1",
+ "metadata": {
+ "editable": true
+ },
"source": [
"that is the rewriting in terms of the so-called bias, the variance of the model $\\boldsymbol{\\tilde{y}}$ and the variance of $\\boldsymbol{\\epsilon}$."
]
},
{
"cell_type": "markdown",
- "id": "a098c5b5",
- "metadata": {},
+ "id": "ca73b735",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## A way to Read the Bias-Variance Tradeoff\n",
"\n",
@@ -1970,8 +2271,10 @@
},
{
"cell_type": "markdown",
- "id": "dd5ba4e9",
- "metadata": {},
+ "id": "a460cbc4",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Example code for Bias-Variance tradeoff"
]
@@ -1979,8 +2282,11 @@
{
"cell_type": "code",
"execution_count": 3,
- "id": "3e700d8e",
- "metadata": {},
+ "id": "520c4af0",
+ "metadata": {
+ "collapsed": false,
+ "editable": true
+ },
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
@@ -2041,8 +2347,10 @@
},
{
"cell_type": "markdown",
- "id": "14ecf8d3",
- "metadata": {},
+ "id": "e3e8854d",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Understanding what happens"
]
@@ -2050,98 +2358,12 @@
{
"cell_type": "code",
"execution_count": 4,
- "id": "62b66d83",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Polynomial degree: 0\n",
- "Error: 0.2937910450030775\n",
- "Bias^2: 0.2929212799917661\n",
- "Var: 0.0008697650113114114\n",
- "0.2937910450030775 >= 0.2929212799917661 + 0.0008697650113114114 = 0.2937910450030775\n",
- "Polynomial degree: 1\n",
- "Error: 0.06894146856540673\n",
- "Bias^2: 0.06832043024896824\n",
- "Var: 0.0006210383164384982\n",
- "0.06894146856540673 >= 0.06832043024896824 + 0.0006210383164384982 = 0.06894146856540674\n",
- "Polynomial degree: 2\n",
- "Error: 0.06106765054837855\n",
- "Bias^2: 0.06054765422099532\n",
- "Var: 0.0005199963273832353\n",
- "0.06106765054837855 >= 0.06054765422099532 + 0.0005199963273832353 = 0.06106765054837855\n",
- "Polynomial degree: 3\n",
- "Error: 0.03346202229536658\n",
- "Bias^2: 0.033140956468054594\n",
- "Var: 0.00032106582731199066\n",
- "0.03346202229536658 >= 0.033140956468054594 + 0.00032106582731199066 = 0.033462022295366586\n",
- "Polynomial degree: 4\n",
- "Error: 0.03352778717048319\n",
- "Bias^2: 0.033116075385773665\n",
- "Var: 0.00041171178470952977\n",
- "0.03352778717048319 >= 0.033116075385773665 + 0.00041171178470952977 = 0.03352778717048319\n",
- "Polynomial degree: 5\n",
- "Error: 0.025517151530854775\n",
- "Bias^2: 0.02496889020925645\n",
- "Var: 0.0005482613215983166\n",
- "0.025517151530854775 >= 0.02496889020925645 + 0.0005482613215983166 = 0.025517151530854765\n",
- "Polynomial degree: 6\n",
- "Error: 0.019946076068427937\n",
- "Bias^2: 0.01950207688986865\n",
- "Var: 0.00044399917855928643\n",
- "0.019946076068427937 >= 0.01950207688986865 + 0.00044399917855928643 = 0.019946076068427937\n",
- "Polynomial degree: 7\n",
- "Error: 0.018695928655417776\n",
- "Bias^2: 0.017979840090002395\n",
- "Var: 0.0007160885654153815\n",
- "0.018695928655417776 >= 0.017979840090002395 + 0.0007160885654153815 = 0.018695928655417776\n",
- "Polynomial degree: 8\n",
- "Error: 0.010736105188369705\n",
- "Bias^2: 0.010376602508045108\n",
- "Var: 0.0003595026803245951\n",
- "0.010736105188369705 >= 0.010376602508045108 + 0.0003595026803245951 = 0.010736105188369703\n",
- "Polynomial degree: 9\n",
- "Error: 0.011013290652731648\n",
- "Bias^2: 0.010539027867198339\n",
- "Var: 0.0004742627855333097\n",
- "0.011013290652731648 >= 0.010539027867198339 + 0.0004742627855333097 = 0.011013290652731648\n",
- "Polynomial degree: 10\n",
- "Error: 0.010972468815260695\n",
- "Bias^2: 0.010593565969982321\n",
- "Var: 0.0003789028452783743\n",
- "0.010972468815260695 >= 0.010593565969982321 + 0.0003789028452783743 = 0.010972468815260695\n",
- "Polynomial degree: 11\n",
- "Error: 0.010840555937749019\n",
- "Bias^2: 0.010348475861970803\n",
- "Var: 0.0004920800757782156\n",
- "0.010840555937749019 >= 0.010348475861970803 + 0.0004920800757782156 = 0.010840555937749019\n",
- "Polynomial degree: 12\n",
- "Error: 0.010192472149420886\n",
- "Bias^2: 0.009610568640077932\n",
- "Var: 0.0005819035093429566\n",
- "0.010192472149420886 >= 0.009610568640077932 + 0.0005819035093429566 = 0.010192472149420888\n",
- "Polynomial degree: 13\n",
- "Error: 0.010312285920735095\n",
- "Bias^2: 0.009802534263878348\n",
- "Var: 0.0005097516568567488\n",
- "0.010312285920735095 >= 0.009802534263878348 + 0.0005097516568567488 = 0.010312285920735097\n"
- ]
- },
- {
- "data": {
- "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAApO0lEQVR4nO3deXRc9X338fd3RrvkRZs3SbZkvNt4AWGSQAMkrAnBNCEBJ+TACSknaUjTLKd2mudAwtM8JWma0NPQFhooedqCQ3iyOAmEQnYCxJYds3iLjVd5lSVbXrRrvs8fMxLjsWSN7ZGvZubzOkdn7vK7o+9cjz/36i6/a+6OiIhkrlDQBYiIyPBS0IuIZDgFvYhIhlPQi4hkOAW9iEiGywm6gEQVFRVeW1sbdBkiImllzZo1h9y9cqB5Iy7oa2traWhoCLoMEZG0YmY7B5unQzciIhkuqaA3s+vNbLOZbTWz5QPM/4SZvW5m68zsRTObEzfvi7HlNpvZdaksXkREhjZk0JtZGHgIuAGYAyyND/KYJ9z9QndfCHwd+GZs2TnAbcBc4HrgX2LvJyIi50kyx+gXA1vdfRuAma0AlgAb+hq4+9G49sVAX78KS4AV7t4JbDezrbH3ezkFtYtIGuvu7qaxsZGOjo6gS0krBQUFVFdXk5ubm/QyyQR9FbA7brwRuDSxkZl9CvgckAe8K27ZVxKWrRpg2buBuwEmT56cTN0ikuYaGxsZNWoUtbW1mFnQ5aQFd6e5uZnGxkbq6uqSXi5lJ2Pd/SF3vwBYBvyvM1z2EXevd/f6ysoBrw4SkQzT0dFBeXm5Qv4MmBnl5eVn/FdQMkG/B6iJG6+OTRvMCuDms1xWRLKIQv7Mnc06SyboVwPTzazOzPKInlxdmfCLp8eNvhfYEhteCdxmZvlmVgdMB1adcZVJaG3r5p9e2MJruw8Px9uLiKStIYPe3XuAe4DngI3AU+6+3szuN7ObYs3uMbP1ZraO6HH6O2LLrgeeInri9ufAp9y9N/UfA8In9vHh372bQy8+PhxvLyIZKBwOs3Dhwv6fBx54IOiShkVSd8a6+zPAMwnT7o0b/sxplv0q8NWzLTBZJWWTyLcT0Lxl6MYiIkBhYSHr1q07bZve3l7C4fCg48kuF6TMuTM2nMOBnEkUH90WdCUikuZqa2tZtmwZF110Ed///vdPGX/yySe58MILmTdvHsuWLetfrqSkhM9//vMsWLCAl18eOVeRj7i+bs5Fa3EtFa1v4u46ySOSRr7yk/Vs2Ht06IZnYM6k0dz3vrmnbdPe3s7ChQv7x7/4xS9y6623AlBeXs7atWsBWL58ef/43r17edvb3saaNWsoLS3l2muv5Uc/+hE333wzJ06c4NJLL+Uf//EfU/pZzlVGBX1P6TRqjvyelqMnKB9TEnQ5IjLCne7QTV/gJ46vXr2aK6+8kr5LwT/ykY/w29/+lptvvplwOMwHPvCBYa35bGRU0OeNn0nejl72bN9E+cL6oMsRkSQNtecdhOLi4tOOD6SgoGDEHJePlznH6IGyKfMAOLJ7wxAtRUTOzuLFi/nNb37DoUOH6O3t5cknn+SKK64IuqzTyqg9+sra6F5B94FNAVciIukg8Rj99ddfP+QllhMnTuSBBx7gqquuwt1573vfy5IlS4a50nOTUUEfKiqlxcaSd+TNoEsRkTTQ2zvwbT07duw47fjSpUtZunTpKcsdP348VaWlVEYdugE4VDCFsW2DPmhFRCTrZFzQd4yeSlXvbjp7huUGXBGRtJNxQW+VMyiz4zQ2NgZdiojIiJBxQT+qajYATTveCLgSEZGRIeOCflzdhQC07d0YcCUiIiNDxgV90bg6OslV52YiIjEZF/SEwhzMqaLk2PagKxGREW7Hjh3MmzfvlOkf//jH2bAhc268zKjr6Pu0FtdR2bpJnZuJyFn5zne+E3QJKZV5e/RAb9k0avwATa3Hgi5FREa4np4ePvKRjzB79mxuueUW2trauPLKK2loaADgk5/8JPX19cydO5f77ruvf7nly5czZ84c5s+fzxe+8IWgyk9KRu7R50+cSc72CHu3bWTcRZcGXY6IDOXZ5bD/9dS+54QL4Yahnxi1efNmHn30US677DI+9rGP8S//8i8nzf/qV79KWVkZvb29vPvd7+a1116jqqqKH/7wh2zatAkz48iRI6mtPcUyco++r3Oz1sbMOcYmIsOjpqaGyy67DIDbb7+dF1988aT5Tz31FBdddBGLFi1i/fr1bNiwgTFjxlBQUMBdd93FD37wA4qKioIoPWkZuUdfMXkOAD0HNgdciYgkJYk97+GSeB4vfnz79u184xvfYPXq1ZSWlnLnnXfS0dFBTk4Oq1at4he/+AVPP/003/72t/nlL395vktPWkbu0YcKx3DIysg/sjXoUkRkhNu1a1f/Y/+eeOIJLr/88v55R48epbi4mDFjxnDgwAGeffZZINp5WWtrK+95z3v41re+xauvvhpI7cnKyKAHaC6sZWz7rqDLEJERbubMmTz00EPMnj2bw4cP88lPfrJ/3oIFC1i0aBGzZs3iwx/+cP8hnmPHjnHjjTcyf/58Lr/8cr75zW8GVX5SMvLQDUDH6DpqTzxLR1cPBXkZ+zFF5BzU1tayadOpz6/49a9/3T/8+OOPD7jsqlWrhqmq1MvYPfrQuJmMsTZ279FevYhkt4wN+lHV0ROyh7arczMRyW4ZG/Tj66KXWLarczMRyXJJBb2ZXW9mm81sq5ktH2D+58xsg5m9Zma/MLMpcfN6zWxd7GdlKos/ncLyKXSQh7XoyhsRyW5DnqU0szDwEHAN0AisNrOV7h5/N9IfgXp3bzOzTwJfB26NzWt394WpLTsJoRAHcqspObbtvP9qEZGRJJk9+sXAVnff5u5dwArgpEeeu/uv3L0tNvoKUJ3aMs/OsZI6JnTtxt2DLkVEJDDJBH0VsDtuvDE2bTB3Ac/GjReYWYOZvWJmNw+0gJndHWvT0NTUlERJyektm84kDnLwcGvK3lNEMsNVV13Fc889d9K0Bx988KTr6E/n3nvv5YUXXhiO0lIupSdjzex2oB74h7jJU9y9Hvgw8KCZXZC4nLs/4u717l5fWVmZsnoKJswkbM6+betT9p4ikhmWLl3KihUrTpq2YsUKli5dOuSyvb293H///Vx99dXDVV5KJRP0e4CauPHq2LSTmNnVwJeAm9y9s2+6u++JvW4Dfg0sOod6z0h5bfSxgq2NuvJGRE52yy238LOf/Yyuri4g+hCSvXv38uSTTw7YLXFtbS3Lli3joosu4vvf/z533nknTz/9NAD3338/l1xyCfPmzePuu+/uP1x85ZVXsmzZMhYvXsyMGTP43e9+B0Q3FF/4wheYN28e8+fP55//+Z8BWLNmDVdccQUXX3wx1113Hfv27UvJZ03mltHVwHQzqyMa8LcR3TvvZ2aLgIeB6939YNz0UqDN3TvNrAK4jOiJ2vOifHL0QeHq3ExkZPvaqq+xqeXUO1TPxayyWSxbvGzQ+WVlZSxevJhnn32WJUuWsGLFCj70oQ/xt3/7t6d0Szx//nwAysvLWbt2LQA///nP+9/rnnvu4d577wXgox/9KD/96U953/veB0T7u1+1ahXPPPMMX/nKV3jhhRd45JFH2LFjB+vWrSMnJ4eWlha6u7v59Kc/zY9//GMqKyv53ve+x5e+9CUee+yxc14XQwa9u/eY2T3Ac0AYeMzd15vZ/UCDu68keqimBPh+rOe3Xe5+EzAbeNjMIkT/engg4WqdYWX5JTSFKiloffN8/UoRSSN9h2/6gv7RRx/lqaee4pFHHqGnp4d9+/axYcOG/qC/9dZbB3yfX/3qV3z961+nra2NlpYW5s6d2x/073//+wG4+OKL2bFjBwAvvPACn/jEJ8jJiUZwWVkZb7zxBm+88QbXXHMNEN3rnzhxYko+Z1KdwLj7M8AzCdPujRse8ECVu78EXHguBZ6r5oIplLbvDLIEERnC6fa8h9OSJUv47Gc/y9q1a2lra6OsrGzAbon7FBcXn/IeHR0d/OVf/iUNDQ3U1NTw5S9/+aRl8vPzAQiHw/T09Axai7szd+7c/p40Uylj74zt0zlmKjWRPbR3Dr6CRSQ7lZSUcNVVV/Gxj32MpUuXDtot8en0hXpFRQXHjx/vP25/Otdccw0PP/xwf/C3tLQwc+ZMmpqa+oO+u7ub9etTcyFJxgd9eNxMRlk7u3dvD7oUERmBli5dyquvvsrSpUsH7Zb4dMaOHctf/MVfMG/ePK677jouueSSIZf5+Mc/zuTJk5k/fz4LFizgiSeeIC8vj6effpply5axYMECFi5cyEsvvZSKj4iNtJuJ6uvrve+hvKmwc/UzTPnZUl66/HHecfWfp+x9ReTcbNy4kdmzZwddRloaaN2Z2ZrYpeynyPg9+vFTo52bdexL7Rl9EZF0kfFBX1BWQxsFhFq2BF2KiEggMj7oMeNgbg2jjusYvchIM9IOHaeDs1lnmR/0wNFR6txMZKQpKCigublZ/y/PgLvT3NxMQUHBGS2XFQ9T9bLpTGx+nv3NLUysKA+6HBEBqquraWxsJJUdGWaDgoICqqvPrIPgrAj6womzCG119m/bwMSKPwu6HBEBcnNzqaurC7qMrJAVh27Ka+cC0Np43npfEBEZMbIi6MtqZhNxI3JQnZuJSPbJiqC3vCKawuMoaNVjBUUk+2RF0AO0FE6hrEOdm4lI9smaoO8cewGTI3to6+wKuhQRkfMqa4I+Z9xMiqyT3TvUN72IZJesCfoxNXMAaN75RsCViIicX1kT9OPqYp2b7deVNyKSXbIm6PPHTuI4RYRbtgZdiojIeZU1QY8ZB/PUuZmIZJ/sCXrg+Kg6JnbvIhJRJ0oikj2yKugj5TOYaC3sazoUdCkiIudNVgV90cSZABzYritvRCR7ZFXQV9ZdCMDRxo0BVyIicv5kVdCPrZpJL0ak6U9BlyIict4kFfRmdr2ZbTazrWa2fID5nzOzDWb2mpn9wsymxM27w8y2xH7uSGXxZ8pyCzgYnkDhUd0dKyLZY8igN7Mw8BBwAzAHWGpmcxKa/RGod/f5wNPA12PLlgH3AZcCi4H7zKw0deWfucOFtZR37AqyBBGR8yqZPfrFwFZ33+buXcAKYEl8A3f/lbu3xUZfAfqec3Ud8Ly7t7j7YeB54PrUlH52usZOZXJkD8c71LmZiGSHZIK+CtgdN94YmzaYu4Bnz2RZM7vbzBrMrGG4nx+ZM24WBdbN7u3qCkFEskNKT8aa2e1APfAPZ7Kcuz/i7vXuXl9ZWZnKkk4xdnL0qNPhXXqsoIhkh2SCfg9QEzdeHZt2EjO7GvgScJO7d57JsufTuNgllh37NgVZhojIeZNM0K8GpptZnZnlAbcBK+MbmNki4GGiIX8wbtZzwLVmVho7CXttbFpg8kaP4ygl5BxW52Yikh1yhmrg7j1mdg/RgA4Dj7n7ejO7H2hw95VED9WUAN83M4Bd7n6Tu7eY2f8murEAuN/dW4blkyTLjKb8Gkaf2BFoGSIi58uQQQ/g7s8AzyRMuzdu+OrTLPsY8NjZFjgcjo+aysSml+iNOOGQBV2OiMiwyqo7Y/tVTGe8HWbfgYNDtxURSXNZGfRFE2cDsH/76wFXIiIy/LIy6Ps6NzveqEssRSTzZWXQj62aQTdh/NCWoEsRERl2WRn0hHNjnZttC7oSEZFhl51BDxwpqqVCnZuJSBbI2qDvHnsBNb6Xo20dQZciIjKssjboc8fPIt96aFTnZiKS4bI26Ps7N9u5PuBKRESGV9YGfV/nZp0H1LmZiGS2rA363FEVHLHR5BzWYwVFJLNlbdADNOVPZuyJ7UGXISIyrLI66NtGTWVSTyM9vZGgSxERGTZZHfRUTKfCWtm7b1/QlYiIDJusDvriSdHOzQ7ueCPgSkREhk9WB/24unkAHN+zMeBKRESGT1YH/eiJ0+kmBz/0p6BLEREZNlkd9IRzOBCeRPFRXXkjIpkru4MeOFJcS0WnOjcTkcyV9UHfU3oB1b6PI8dOBF2KiMiwyPqgzx0/kzzrpXG7ukIQkcyU9UFfOjl65c2R3XqsoIhkpqwP+r5LLLv2a49eRDJT1gd9TnEpLTaW3CPq3ExEMlPWBz3AofwpjG3bEXQZIiLDIqmgN7PrzWyzmW01s+UDzH+nma01sx4zuyVhXq+ZrYv9rExV4anUNrqOqp7ddKtzMxHJQEMGvZmFgYeAG4A5wFIzm5PQbBdwJ/DEAG/R7u4LYz83nWO9w8IqZlBmx9mzpzHoUkREUi6ZPfrFwFZ33+buXcAKYEl8A3ff4e6vAWm5S1xSpc7NRCRzJRP0VcDuuPHG2LRkFZhZg5m9YmY3D9TAzO6OtWloamo6g7dOjcqp0ccKtqlzMxHJQOfjZOwUd68HPgw8aGYXJDZw90fcvd7d6ysrK89DSScbPX4qneRC85bz/rtFRIZbMkG/B6iJG6+OTUuKu++JvW4Dfg0sOoP6zo9QmAM5VRQf3RZ0JSIiKZdM0K8GpptZnZnlAbcBSV09Y2alZpYfG64ALgNG5C2orcW1VHapczMRyTxDBr279wD3AM8BG4Gn3H29md1vZjcBmNklZtYIfBB42MzWxxafDTSY2avAr4AH3H1EBn1P6TSq/QAtR48HXYqISErlJNPI3Z8BnkmYdm/c8Gqih3QSl3sJuPAcazwv8ifMImdHhL3bNlC2cHHQ5YiIpIzujI0pmzwXgCO71w/RUkQkvSjoYyrrokHffUCPFRSRzKKgjwkXjuGQlZF3ZGvQpYiIpJSCPs6hgimMbdsZdBkiIimloI/TMXoq1b2NdHX3Bl2KiEjKKOjjWOUMxtgJ9uzR9fQikjkU9HFGVUc7N2tS52YikkEU9HHG1cU6N9urzs1EJHMo6OOUVNbSQR6mzs1EJIMo6OOFQuzPrab42PagKxERSRkFfYKjxXWM69qFuwddiohISijoE/SWTaPKD9LcejToUkREUkJBnyB/wizC5ux9U33eiEhmUNAnKJ8S7fOmtVFX3ohIZlDQJ6iMBX3Pgc0BVyIikhoK+gShghIOWiX5rW8GXYqISEoo6AfQXDiF0nZ1biYimUFBP4COMdHOzTq7e4IuRUTknCnoBxCunMEoa6dx946gSxEROWcK+gH0dW52aPvrAVciInLuFPQDGB/r3Kx9/6aAKxEROXcK+gEUVUymjQJCh/RYQRFJfwr6gZhxILeakuPq3ExE0p+CfhDHSuoYr87NRCQDKOgH0Vs2nYkcoqnlcNCliIick6SC3syuN7PNZrbVzJYPMP+dZrbWzHrM7JaEeXeY2ZbYzx2pKny4FU6cRcicfds2BF2KiMg5GTLozSwMPATcAMwBlprZnIRmu4A7gScSli0D7gMuBRYD95lZ6bmXPfzKpswDoLVRQS8i6S2ZPfrFwFZ33+buXcAKYEl8A3ff4e6vAZGEZa8Dnnf3Fnc/DDwPXJ+CuoddxeTZRNyIHFTnZiKS3pIJ+ipgd9x4Y2xaMpJa1szuNrMGM2toampK8q2HVyi/iIPhcercTETS3og4Gevuj7h7vbvXV1ZWBl1Ov5bCKZSpczMRSXPJBP0eoCZuvDo2LRnnsmzgOsdcQHVkDx1d6txMRNJXMkG/GphuZnVmlgfcBqxM8v2fA641s9LYSdhrY9PSQnjcDIqtk907dfhGRNLXkEHv7j3APUQDeiPwlLuvN7P7zewmADO7xMwagQ8CD5vZ+tiyLcD/JrqxWA3cH5uWFkZXRy8uat75RsCViIicvZxkGrn7M8AzCdPujRteTfSwzEDLPgY8dg41Bmb81GjnZh379PxYEUlfI+Jk7EhVWDqJ4xQRalbnZiKSvhT0p2PGgbwaRp1Q52Yikr4U9EM4XlLHhK7d6txMRNKWgn4IkfLpTLRmDh5Km3PIIiInUdAPoXDiLAD2bdNjBUUkPSnoh1BRG+3c7NgeXXkjIulJQT+E8ppZ9LrRe/BPQZciInJWFPRDsNwCDoYnUKDOzUQkTSnok9BSVEt5hzo3E5H0pKBPQteYqdRE9tDW2RV0KSIiZ0xBn4Sc8TMpsG4ad+g4vYikHwV9EkbX9HVupscKikj6UdAnYXxdtHOzTnVuJiJpSEGfhIIx4zlKCeHDuvJGRNJPUt0UZz0zDubVUHV4Fd968AHy697GzOmzWDSljLLivKCrExE5LQV9kkZf/CFKX3mAzx75e/gj7F9byiuR6ewomkdkUj3jZlzKoqnjuaCyhFDIgi5XRKSfjbReGevr672hoSHoMgbW2w37X6d75x84uuUlcvc1MLpjLwCdnsN6r+X10CyOViyiaOrbmTVjJgtqxlKSr+2piAwvM1vj7vUDzlPQn6Nj+/Hd0eDv3vEHxhxZT65Hr7ff4+WsjUxnb8mFePViJs66hIvqxlNdWoiZ9vpFJHUU9OdTTxfsf52O7S9xdMtLFOxfw+iuAwB0eC6v+VQ25cyirfIiSqa9g9kzpjOvajT5OeGACxeRdKagD1rrHnp3r+LI5heJ7FpF6dGN5Hg3ALsilaxjBgfGXsTbP/BXzJsyLuBiRSQdKehHmu4O2P8ax7f+nuNbX6b44FpGdTfxql9A4zUP897LLwm6QhFJMwr6kc6dY+t+SHjlp2iL5PCT6V/l9qW3kxvWbQ4ikpzTBb2SZCQwY9Si95P3iV9BYSkf3fIZnnjwbzh4tD3oykQkAyjoR5Cc8bOo+OzvOVh1NXcc+3deffADrHuzMeiyRCTNJRX0Zna9mW02s61mtnyA+flm9r3Y/D+YWW1seq2ZtZvZutjPv6W4/syTP4pJf/EUBxZ/kXdFXqbo/17HT375u6CrEpE0NmTQm1kYeAi4AZgDLDWzOQnN7gIOu/s04FvA1+LmvenuC2M/n0hR3ZnNjPHvWU77h55iYvgoV/zmg3z38X+ls6c36MpEJA0ls0e/GNjq7tvcvQtYASxJaLME+G5s+Gng3aY7gs5ZyZxrKLrnRdpKJnPHjuX88Jv3sO/IiaDLEpE0k0zQVwG748YbY9MGbOPuPUArUB6bV2dmfzSz35jZnw30C8zsbjNrMLOGpqamM/oAmS5cNoUJf/0bGqf8Obe1PcGb/3QjDRu3BV2WiKSR4T4Zuw+Y7O6LgM8BT5jZ6MRG7v6Iu9e7e31lZeUwl5SGcgupvvM/OPjOv+dSf41xK67jxz//OSPt0lgRGZmSCfo9QE3ceHVs2oBtzCwHGAM0u3unuzcDuPsa4E1gxrkWnZXMGPeuv6Troz9lVE4v1778Uf77379BR7eO24vI6SUT9KuB6WZWZ2Z5wG3AyoQ2K4E7YsO3AL90dzezytjJXMxsKjAd0HGHc1B8wdsZ85mXaB4zj9v3/h0//8ad7G5qDbosERnBhgz62DH3e4DngI3AU+6+3szuN7ObYs0eBcrNbCvRQzR9l2C+E3jNzNYRPUn7CXdvSfFnyDqh0ROo/sz/sGvGndzcuZKmh65j1et6zKGIDExdIKS5g7//T0Y//zmOeDGv1H+TJTf+ubpAFslC6gIhg4277KNE7nqBcF4B72n4ON/71y/T1tkddFkiMoIo6DNAUc0CKj77MvvK385tBx/kxW/cys79h4IuS0RGCAV9hrCiUqbc8xN2Xvhpru3+BSf+7WpeXrM26LJEZARQ0GeSUIgpH/g7mm78LpPtILNWvo8fPf2fRCIj6zyMiJxfemp1Bqqsv5mO6jkc/Y8P8b7XP80P965j3gfvIxw2+s6990X/W+N+8njC9PhpieOFeSHqKkoIh3QSWGQk0lU3Gcw7j7H9sbuYeuA5jnsBERKD+NRgHujb4AntEsfbyGc90zlYuohw7TuYMmcxC2srKMrTfoTI+aInTGUzd/b95lHad7/61jSLTre+wO576fsunDz5JJawKTDA21sYdXAtpd37ATjh+fzRp7OzeAE9VZcybs47WHRBDRPGFKTsY4nIyRT0cn60NtK29fe0bPoteXv+QEXbVkI4PR5ivdeyKXcux8fXM2rG5cydMZ1ZE0brcI9IiijoJRgdrfTs/APNG35DZOfLlLe+Tp53AbA9Mp51NpvmskXk1r2DC2YtYuGUUkrydbhH5Gwo6GVk6OnC962jdfPvaN/6IqOb1lDcG+2np9lH0RCZye6S+URq3sbE2Zdycd14Jo0tDLhokfSgoJeRyR0ObaF924u0bvodBftWMbYj+ozcds9jXWQam/Lm0F4+h9zxsyirnsXUiWVMG1fCqILcgIsXGVkU9JI+ju2nd8fLHN70W9j1CqXHNhEmAkCvG7t8HFu9iv25NbSPuYBQxQxG1cylpmoS08aVUFmSr75+JCsp6CV9dZ2A5q30HtjE0cYNdO7fRO7hLYxp20WOv9WnT5OPZptPYmeomuMlU4mUT6do0mwmTJ7GtHFjqC4tJKQTv5LBFPSSeSK9cGQn3vQnjjWup23vRqx5C6OObaOo92h/s3bPY5tPZDtVHC6qpbt0OvkTZlE2eTYXTKqgamwhhblhbQQk7SnoJbucOASH/kTb3o0c3b2eSNNmClu3MaZrH6HYfQARNxq9gkavpI18OqyQzlAB3aECukOFdIcL6c0pojdcSCS3CM8pIpJbDHlFWF4xofxiLK+YcEExOXnFFOTlUJgXoiAnTEFumPzcECX5OUwfN4rCvHDAK0SywemCXteySeYproDiCoqmvIOit8dN726H5q107t/EkV3rCR3YTN3xvYR72sjpPURObzu5vR3k9bQT7umFzuR+XcSNdvJoI592z+cEBbSTzxEv5L+YwqHSBeRPuZRZM2Zw8ZRSxo/WjWNyfmmPXiSRO/R2Rc8PdLdBVxt0n4iO9w+34V3H6ek4QW/HcXo7TxDpPEGk8zgeW846DlN85E/95xL2eDl/jExje/5seifVUzljMYumTmTmhFG6cUzOmfboRc6EGeTkR38oG7wZkBv7GVRPJ+x/nZ5dqyja+jJX7G3gxo4/wK7H6doZZoNPYYXNoLV8IQVTL2XGjAt145iknPboRc63YwfwxtUc2/oynTtXMbrldfIj7QAc8tGsi0yjsXgekap6xs96OwumVVM1tlCXjcpp6WSsyEjW2wNNG+nY/gpHtrxM7r41lLfvAKLH/zd7NZtzZnK8YhFFU9/GBXMuYk7VWHLDqXucRCTiRNyJOESSyIT4bY7FdX+XuC2KH43fUFmsrTZeqaOgF0k37Yfp3d1A8+bf07VzNaUtr1IcOQbAUS/kdaZzJLcy1q+0x3oe9bjh6E1mxKaHPEJ/J9TuGNGfvvkW63w6FOvMuq+X0sTeShlknp0yL669ndreMSIe/a0RDDcjQihWSSj2SaLVRGLT+itLaBux6PS+dm4hei2HiOUSCeUQiR8O5eKhXDyUA6FcPBwd7xu22LiFcyGcC+E8LJyL5eQRio1HP0C0MvfoOjSPRJ/d4A4e6V/HEMEjff8u0TYWmx9dHbHh2Gvp2FI+cMO1yX9PTlrPOkYvkl4KSwnPuIZxM66JjrtD85sc2fJ7jvzpJabuX0th96t4LOQI2Vtx3TctForxr/SP9/01kDAtbtk+ftJed98mITrt1E6r459XEP8e8cvTv2Gy2AbIPILR0x+g8fMt7pW+1/5p0bbGW+Ea8l7C3kMo0kO4t7v/zup0sDVvFpxl0J+Ogl4kHZhBxTTGVkxj7NvvCLqa9BKJQKQbervfeu3tjl5ZFemJvvZ2nzx8UttoO+/pxHt7ou9poehhJzOsbwNpIaIbzvhxBp53Srvoe00rGDssq0BBLyKZLRSCUN9VVGfv5L9z0oseDi4ikuGSCnozu97MNpvZVjNbPsD8fDP7Xmz+H8ysNm7eF2PTN5vZdSmsXUREkjDkoRszCwMPAdcAjcBqM1vp7hvimt0FHHb3aWZ2G/A14FYzmwPcBswFJgEvmNkMd+9N9QfpjnSzuWUzZkaIEKHYMbQQ0dfE6YYRsth4bPiU6XHLxNYFhvW/QuzSMuOk8b5LxuKHJb25O44T8eiVFhEibw17hAhxw7GrKyIe6f9+hS1MKBTq/z6FQ+H+4b7vmoxMfVcm+imnnmP/z9Pg3y6ZY/SLga3uvg3AzFYAS4D4oF8CfDk2/DTwbYt++iXACnfvBLab2dbY+72cmvLfcrTzKEt/tjTVb5tSg20k+ob7QiF+g9M/Hr/R4q3x6AUXb22sYOBxiH5R4y+n7fvi9oVY4nDfeP+ycctHLyXjlHmJ73PSdH+rbcQjJ7eLax/f7pT1N8iGNfE/Xfx4/HIDLZNMeCfWkmp9GwOz2EbB3toIDDbdsEHXN3DKukwMrNMtE1fYgOsscSfmlB2cAdZ3/Lz+Ggb5fsTXd8pwkssM9B79y7/15T3lfePX0dkY9Lv41goZcLqZMbd8Lt+94btn/bsHk0zQVwG748YbgUsHa+PuPWbWCpTHpr+SsGxV4i8ws7uBuwEmT56cbO0nGZU3im+/69uD7nX1TY/f2zplemyZoab3iQ8pGPgLdMp/qtMEbf8XOOH3xYdNfG3ASZ/nlHGnfx0kBh+cuqE55T8qiTe5nBqqJ98s81aoJv5Vk7hs4katL7wHC+6T1nHi+o7f4AwQCAOu67jpfQF6yl+DcRvMweb3bXwT/zrs27gC9Hpv/3cp4pGTxnu9F3c/6XWwtonTEv9dhvo3GiyoB1vmdOus/zscv4FO+HeKn5843P/9i//eDLLBTqz1rJYZIFwT33Oo9RUbiH3wvpdTP+9A4/3ra7B2sdcJRRMYDiPiqht3fwR4BKI3TJ3Ne+SF87ii5oqU1iUikgmSORm7B6iJG6+OTRuwjZnlAGOA5iSXFRGRYZRM0K8GpptZnZnlET25ujKhzUqg7y6OW4BfevRvk5XAbbGrcuqA6cCq1JQuIiLJGPLQTeyY+z3Ac0AYeMzd15vZ/UCDu68EHgX+M3aytYXoxoBYu6eInrjtAT41HFfciIjI4NSpmYhIBjhdp2a6M1ZEJMMp6EVEMpyCXkQkwynoRUQy3Ig7GWtmTcDOc3iLCuBQiso5n9K1blDtQVHtwRiptU9x98qBZoy4oD9XZtYw2JnnkSxd6wbVHhTVHox0rF2HbkREMpyCXkQkw2Vi0D8SdAFnKV3rBtUeFNUejLSrPeOO0YuIyMkycY9eRETiKOhFRDJcxgT9UA8wH6nMrMbMfmVmG8xsvZl9JuiazpSZhc3sj2b206BrORNmNtbMnjazTWa20czeHnRNyTCzz8a+K2+Y2ZNmVhB0TadjZo+Z2UEzeyNuWpmZPW9mW2KvpUHWOJBB6v6H2PflNTP7oZmNDbDEpGVE0NtbDzC/AZgDLLXog8nTQQ/weXefA7wN+FQa1d7nM8DGoIs4C/8E/NzdZwELSIPPYGZVwF8B9e4+j2jX4bcFW9WQHgeuT5i2HPiFu08HfhEbH2ke59S6nwfmuft84E/AF893UWcjI4KeuAeYu3sX0PcA8xHP3fe5+9rY8DGiYXPKc3VHKjOrBt4LfCfoWs6EmY0B3kn0WQq4e5e7Hwm0qOTlAIWxp7kVAXsDrue03P23RJ9TEW8J0PcU7O8CN5/PmpIxUN3u/j/u3hMbfYXoU/NGvEwJ+oEeYJ42YdnHzGqBRcAfAi7lTDwI/A0QGaLdSFMHNAH/ETvs9B0zKw66qKG4+x7gG8AuYB/Q6u7/E2xVZ2W8u++LDe8HxgdZzFn6GPBs0EUkI1OCPu2ZWQnw/4C/dvejQdeTDDO7ETjo7muCruUs5AAXAf/q7ouAE4zMwwcniR3LXkJ0QzUJKDaz24Ot6tzEHjuaVtd5m9mXiB52/e+ga0lGpgR9Wj+E3MxyiYb8f7v7D4Ku5wxcBtxkZjuIHi57l5n9V7AlJa0RaHT3vr+eniYa/CPd1cB2d29y927gB8A7Aq7pbBwws4kAsdeDAdeTNDO7E7gR+IinyY1ImRL0yTzAfEQyMyN6nHiju38z6HrOhLt/0d2r3b2W6Dr/pbunxd6lu+8HdpvZzNikdxN9tvFItwt4m5kVxb477yYNTiIPYCVwR2z4DuDHAdaSNDO7nuihypvcvS3oepKVEUEfOznS9wDzjcBT7r4+2KqSdhnwUaJ7w+tiP+8Juqgs8Wngv83sNWAh8H+CLWdosb9AngbWAq8T/T88om/JN7MngZeBmWbWaGZ3AQ8A15jZFqJ/pTwQZI0DGaTubwOjgOdj/1f/LdAik6QuEEREMlxG7NGLiMjgFPQiIhlOQS8ikuEU9CIiGU5BLyKS4RT0IiIZTkEvIpLh/j/5eQioAf7jBQAAAABJRU5ErkJggg==\n",
- "text/plain": [
- ""
- ]
- },
- "metadata": {
- "needs_background": "light"
- },
- "output_type": "display_data"
- }
- ],
+ "id": "642a16bf",
+ "metadata": {
+ "collapsed": false,
+ "editable": true
+ },
+ "outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
@@ -2153,7 +2375,7 @@
"\n",
"np.random.seed(2018)\n",
"\n",
- "n = 400\n",
+ "n = 40\n",
"n_boostraps = 100\n",
"maxdegree = 14\n",
"\n",
@@ -2193,8 +2415,10 @@
},
{
"cell_type": "markdown",
- "id": "f174845b",
- "metadata": {},
+ "id": "143e1777",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Summing up\n",
"\n",
@@ -2229,8 +2453,10 @@
},
{
"cell_type": "markdown",
- "id": "9fa4672c",
- "metadata": {},
+ "id": "0ddb1890",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Another Example from Scikit-Learn's Repository"
]
@@ -2238,47 +2464,12 @@
{
"cell_type": "code",
"execution_count": 5,
- "id": "7fc282db",
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "\n",
- "============================\n",
- "Underfitting vs. Overfitting\n",
- "============================\n",
- "\n",
- "This example demonstrates the problems of underfitting and overfitting and\n",
- "how we can use linear regression with polynomial features to approximate\n",
- "nonlinear functions. The plot shows the function that we want to approximate,\n",
- "which is a part of the cosine function. In addition, the samples from the\n",
- "real function and the approximations of different models are displayed. The\n",
- "models have polynomial features of different degrees. We can see that a\n",
- "linear function (polynomial with degree 1) is not sufficient to fit the\n",
- "training samples. This is called **underfitting**. A polynomial of degree 4\n",
- "approximates the true function almost perfectly. However, for higher degrees\n",
- "the model will **overfit** the training data, i.e. it learns the noise of the\n",
- "training data.\n",
- "We evaluate quantitatively **overfitting** / **underfitting** by using\n",
- "cross-validation. We calculate the mean squared error (MSE) on the validation\n",
- "set, the higher, the less likely the model generalizes correctly from the\n",
- "training data.\n",
- "\n"
- ]
- },
- {
- "data": {
- "image/png": "iVBORw0KGgoAAAANSUhEUgAAAykAAAFLCAYAAADSycMSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAACUOklEQVR4nOzddXiV5RvA8e9zzro32KjBBozu7hQUMJBmkiIgKmL8DOzCVixAQFBAWkpRUbpDuhtGx9jY2MZ67++P92yMsWbbid2f69oFO289p+6995NK0zSEEEIIIYQQwlIYzF0AIYQQQgghhEhPkhQhhBBCCCGERZEkRQghhBBCCGFRJEkRQgghhBBCWBRJUoQQQgghhBAWRZIUIYQQQgghhEWRJEUIIYQQQghhUSRJsUJKqRClVKxSKkopFaGU2qqUGqWUsvj3UynloJRaZHoOmlKqvbnLJERxZs3xJD2l1LummNLJ3GURoriw5viR0/2IUup9pVSiUio63U8l85S2eLL4D5HI0qOaprkDAcBnwOvA9MK4kFLKWMCn3AwMBK4W8HmFEPljzfEEpVRloA9wpaDPLYTIkTXHj5zuRxZomuaW7udMAV9fZEOSFCunaVqkpml/AP2AIUqp2gBKKUel1FdKqfNKqWtKqclKKefU45RSrymlriilLiulhptqEYJM22YopX5USv2tlIoBOiilyiqlFiulQpVSZ5VSY9Kdy6CUGquUOq2UClNKLVRK+WRR3gRN077VNG0zkFyYr40QIm+sLZ6kMxH9xiihoF8TIUTuWFv8kPsRyydJio3QNO0/4CLQxvTQZ0BVoD4QBJQD3gVQSnUBXgY6mba1z+SUTwAfA+7AVmA5sN90ngeAF5VSD5n2fR54HGgHlAVuot80CCGskDXFE6VUHyBe07S/8/FUhRAFzJriRy48qpQKV0odVko9cx/nEfkgSYptuQz4KKUUMBJ4SdO0cE3TooBPgP6m/foCv2iadljTtNvA+5mc63dN07ZompYC1AF8NU370FTzcAb4Kd35RgFvaZp2UdO0eNP5eiul7ArpeQohCp/FxxOllLupLC8U0HMWQhQMi48fubAQqAH4AiOAd5VSwfk4j8gnuYm0LeWAcPQvlAuwW48PACggtS9nWWBXuuMuZHKu9I8FAGWVUhHpHjMCm9JtX6qUSkm3PRkoBVzK87MQQlgCa4gn7wO/apoWkuOzEUIUJWuIH9nSNO1Iul+3KqW+A3oD8/JyHpF/kqTYCKVUE/SgsBm4AcQCtTRNy+xLeQXwT/d7+Uz20dL9/wJwVtO0Kllc/gIwTNO0LXkuuBDC4lhRPHkA8FdKPWv63RdYqJT6XNO0z3NxvBCigFlR/MgrDT3BEkVEuntZOaWUh1LqEWA+MFvTtIOmJtGfgG+UUn6m/cql67O5EHhSKVVDKeUCvJPDZf4DopRSryulnJVSRqVUbVMgApgMfKyUCjBdy1cp1T2bMjsqpZxMvzoopZxUuioWIYR5WGE8eQCojd7XvT56F5OnkTFxQhQ5K4wf2d6PKKW6K6W8la4pMAb4Pa+vi8g/SVKs13KlVBR6rcFbwHjgyXTbXwdOAduVUreA1UA1AE3TVgDfA+tS9zEdE5/ZhTRNSwYeQb8JOIteMzIN8DTt8h3wB7DSVKbtQLNsyn4cvWalHPCv6f8BuXvaQohCYJXxRNO0ME3Trqb+oHfruKlpWnReXwAhRL5ZZfwwye5+pL+pTFHALOBzTdNmZvtKiAKlNE3LeS9h05RSNYBDgKOmaUnmLo8QwnpJPBFC5JfED5GetKQUU0qpHqZmTm/gc2C5BAQhRH5IPBFC5JfED5EVSVKKr6eB68Bp9C4SMv+3ECK/JJ4IIfJL4ofIlHT3EkIIIYQQQlgUaUkRQgghhBBCWBRJUoQQQgghhBAWRZIUcV+UUjWVUrtknZO8U0o9qpRaYO5yCGEJJJbkn1KqlFLqqFLK0dxlEcISSDwpeEqp55VSRbpIrlUnKUqpEKVUglKqZIbH9yqlNKVUoOl3f6XUYqXUDaVUpFLqkFJqqGlboGnf6Aw//QqpzFWUUnFKqdnZ7KOUUp8rpcJMP5+n/6IppToqpfYopW4ppc4opUbeR3lyutZUpdRxpVRK6muWwUfAV1o+BjcppcoqpS7m8RgH0x/jLI9TSj2slNqslIpQSl1VSk1TSrmn2z7D9LlJ/34b81r+dOd7yXSdW0qpn9PfKCilPlJKHVRKJSml3k9/nKZpy4FaSqm6+b22KBg2HEteMsWIW0qpy0qpb5RSdqZtfkqpeabHI5VSW5RS2a0nkFN5sowlSqmqSqnflVKhSqlwpdS/SqlqGU5RJLFEKbUiw/uToJQ6mM3+Dyiljimlbiul1inTInGmbT5KqQWm53tDKTVHKeWR1/Ln8lpfKaVOKqWiTPsMTt2mado19HUm8v23QOSdNcUNpdRopd+0xyulZuSwr1JKjVNKXTKVd71Sqla67V8opS6Y4so5pdSbBVlW0zUclf739JbS/76+nGF7X6XfC0QppY4opR7PcIqiiifvK6USM7x3lXJx3M+m9z0o3WOBSqm/lVI3Tc95Qmq8Lij3+br+BAxQpkU5i4JVJykmZ4Hg1F+UUnUAlwz7/Iq+yFAAUAIYBFzLsI+Xpmlu6X4Kq4Z7IrAzh31GAo8D9YC6wKPos1+glLIHlgJT0Bcv6geMV0rVy2d5sryWyX7gWWBPxgOVUmWADsCyzE5s+vK+n821uwH/5LG8rwKhOezjCYwDygI10Bdp+jLDPl9keL+T81gOAJS+au5Y9JWvA4BKwAfpdjkFvAb8lcUp5iE3FpbCFmPJH0BDTdM80Fdmr4e+ajKAm+n4RoAPMBP4Synlls/yZBdLvExlqQaUQl81Om3l5qKMJZqmdU3//gBbgd+yuG5JYAn6Ktg+wC4g/fs5DvAGKgKVTc8tu3JmKRfXikF/TT2BIcB3SqmW6bbP4e7YLYqGtcSNy+if159zsW8fYBjQBv2zuA39OaSaDlQ3xZWW6DeuPfNaIKXU0GwSpveBKuivWQfgNaVUF9Nx5YDZwMuAB/p9wVx1Z0X7or43WZDhvTuT3c5Kqdbo8SKjSeiznJVBX6yyHfr9V54U1uuqaVocsAIYfO9pC4ctJCm/cvcLNgR9ZdD0mgAzNE2L0TQtSdO0vaZVTouUUqo/EAGsyWHXIcDXmqZd1DTtEvA1MNS0zQf9w/OrptsJHAVqprvOMFMmfFPpNZbZreae3bXQNG2ipmlrgLhMju0M7DF9cPOjG/B3bndWSlUEBgKfZrefpmlzNU37R9O025qm3UTP/lvl4TqPKKX2Kb0lZqvKvqVjCDBd07TDpmt9xN2v30zTZy0qi+PXAw/ntmyiUNlcLNE07bSmaRGphwEpQJBp2xlN08ZrmnZF07RkTdOmAg6YVoI2XadAYommaf9pmjZd07RwTdMSgW+AakqpEqZjizSWpDLVdLfh3vc5VU/gsKZpv5nK9j5QTylV3bS9IrBM07RbmqZFolcgpa9xrq6UWqX01qPjSqm+2RQn22tpmvaepmnHNE1L0TRtB7AJaJHu+B1ApRzeI1HwrCJuaJq2RNO0ZUBYLnavCGw2xYhk9BvXtHsMTdOOa5oWk27/tLgCoJRqbvrbGaGU2q+Uap+PIg8BPtI07aamaUfR/44PNW3zByI0TVthug/6Cz2JT73xN0s8yQ1Ty8gPwPOZbK4ILNQ0LU7TtKvoiVL6eGLu1xWK+J7FFpKU7YCHUqqG0rvs9Ef/QmXcZ6JSqr9SqsL9XEwpNcn0Acns50A2x3kAH6JnqDmphd6CkWq/6bHUZv15wJNKKaNSqgV6RrzZdJ3uwJvof/B80f+QzcvPtXKhDnA8l/vexdQi1BZYlYfDfkB/brF5vFxb4HCGx5413TjsVkr1SleuBug1TU+j13hNAf5QWff1zuz1K5Xu5isnR4FAdR9dRESBscVYglLqCaXULeAGeivHlCz2q4+epJwy/V6YsaQtcFXTtNQbpqKOJakGA5s0TQvJYvtdz8l0Y3aaO89rIvCIUspb6QvR9UKvaUQp5Woq01zAD/3zNEkpVZPM5XStNEopZ/Qb38Pp9k9Cf+/y26ou8scq4kYezQcqK72bpj36je1dLQtKqbFKqWjgIuCK/jlPrY3/C73Vxgd4BVislPLN7cVN36UyZB1PdgFHlVKPme6DHgfigdTnX9Tx5FHT/cRhpVROa7y8BGzUNC2z9+pboL9SysX0OnbF9LpbyOsK+j1LkcUYW0hS4E5NRmf0F/BShu190P/AvgOcNdWSN8mwz40MX/YamV1I07RnNU3zyuInuxr3j9Br3HPTz9ENiEz3eyTgplTaWJF5wLvoH55NwFuapl0wbRsFfKpp2lHTH61PgPrZ1K7ldK3seJF1C0FO2gL7NU3L1fFKqR6AUdO0pXm5iFKqM3qAfTfdw9+jN3f6oX8mZiilUltaRgJTNE3bYapdnon+OjfP4hKZvX4A7pnsm5nU5++Vy/1F4bK1WJLasugBVAUmc283k9TE51fgA1OLABRSLFFK+aPf3KdPsrwooliSwWBgRjbbMz4nTL+nfr/3oCd2YaafZPQuGwCPACGapv2SWnsOLEb/DOXnWulNRr+5+DfD41FILDEHa4gbeXEFveLzOHqlYB/0m+v05fgM/bPZEP35p352BwJ/a5r2t6nVbxX6zW+3PFw/tctpxnjibrp2Mnpr1Vz0v89zgafTte54UXTxZCF6t3JfYATwrlIqOLMdlVLl0StA381sO7ARPWG4hZ787eJOlzVLeF1Bf10983DN+2JLScoT6E1W9zTbm5q1xmqaVgu9z/A+YFmGP54lM3zZjxZU4Uw1lJ3QuzjkRjR6l65UHkC0pmma0pv+56MHRAf0D/RrSqnU5rcA9L7KEUqpCCAcvZtHOaXUm+rOwK7JOV0rF+W8SYY/oEqpP9NdeywwNl3Q/TPdrmnNqUqpAenKdU8TuKlG8gvu9KXPFaVUc/QvWW9N006kPq5p2h5N08JMNw5/o/flTu1PGwD8L/0fC6A8UDaLcmb2+kHuA2Tq6xeRl+cmCo2txZI0mqadRK95n5T+cVOt/HJgu6Zp6btSFngsMdX6rQQmaZqWvlWmSGJJhvO3BkoDi7LZLeNzSn1eqd/vhcAJU9k90Fs+UmvRA4BmGWLJAKC0UqpCunJG5/JaqeX+En18Ud9M4rQ7EkvMwaLjRj68i95SVx5wQh9nuVYpdddYG1OXoL3oiUzqWMwAoE+Gz31r9Br8u1qC0GPRE+n2Ta2xT/1OZIwnUaZzdEK/J2iPfh/UDphmio9QhPFE07QjmqZdNlVqbgW+A3pnti96S8mH6SqC0pfPgN5qsgS9Zaok+ni31Nm0LOF1Bf11vaf8hUbTNKv9AUKATqb/r0fPPl0BO0ADArM4rrZpewkg0PR/u1xeczL6G53Zz+EsjnkRvV/fVdNPNPqXek8W+28FRqT7fRj6DQToH/69Gfb/Fphg+v+/wIA8vIZZXivDfpuBoRkeGwisyubc7wPvZ7HtKFAjl2WsDySme/3C0Wssr2bzHjdAH4D2aC7O/yMw3vT/KegtU7l9/eYCH6f7vSN6N5aM+83O7LVAHytztjC+H/KT+x9bjSWZHD8QvZYw9XdHU8yYAxgy7FugsQT9D+5e4LMsylXosSTDcT8Bs3LYZySwJd3vrsBt9EHDmF7/eum210dPzEAfTJ3lc8rrtUyPfQAcAkpkcrydaf+Agv5+yE+W75lVxI0Mx49DHx+T3T5/Ai9keCwCaJzF/m8Dv5v+/wbwUy6fy9CsyoI+0L9zut8/BOab/v8KsDTD/suAV0z/L/J4ku7414ElWWyLQG/JTo3fGvpEQE+gJyUa4Jlu/8eBQ5byupp+HwCsy+/rk+fXs6guVCiFvztAVE79AmUMEOiZaG3T4+7oXQ1OmrblKUDks5wu6DV2qT9fodfe+Wax/yjTF6Uc+gxVh4FR6Z5nNPrNsDL9fgoYadreA/2PWC3T755An2zKluW1TNsd0GtStqA3ZTphuplBrxEKA5yyOHemgQB9cNiZPLx+dhlev56mL1pp9C5gGfevbQoE/bI4X2/0Zk8D8CB6LUJ707bG6LOwNDO9vq7og8TcszhXF/RgUxO9iXkt6W7CAHvTazYX/Y+DU/oyo/f5n2Tu71Jx/7HhWDIc8DP9v6bp+52akNujt6Asy6zMBRlL0Gvr/sNUmZLJsUUSS9Id54xeG9gxh/18Tfv1Mn13P+fuxGsd+lg5Z9PPJGCraZs7cA59Jid7008TsrgBysW13gBOAqWzOL4lcMTc36Xi9GMtcSNdmZzQJ5751fT/TK8JvIdeMVkK/e/kIPTKES/T70+jVzoooCl697AxpmPLo/9NfAgwmq7THvDP5DpDyfpm+jNgg+k61U3X6GLa1g59jF190+8NTPHjQdPvRRZPgO4ZXotLwJAs9vXj7vitoXcldzZtP4PeymNneq2XAnMt5XU1PTYVeK3IvmNFdaFCKXy6AJHh8YwB4gf04B6NnrX+iekPBXcCRMYaiZcLsdzvA7PT/d4GU+2b6XeF3uQWbvr5AlDptvdFv3mIQu+3+DnpakHRA8pB9FqdC8DP2ZQlp2utN70+6X/ap9v+G1knA++TeSAYTRY3K7l8/doDFzM8Fg20Mf3/F/TZRjKtYULvGxxpen32A/0znKsL+tSsEaYv8G9kkaSY9n8ZPSm6Zbq2Y7ptMzJ5/Yam236QdDWx8mOeHxuOJb+YPpsxpuf4JaY/3Oh/kDT02vf05W2T7vgCiSXo48I0UznSX6tCuuOLLJagt3KcI12sS7ftMOlakNC71x1Db7FaT7racfSbmuXof8jD0btrVEm3vRr6YNdQ0z5rMd0AZFGu7K6lofcTT//6vZlu+0RMN4ryUzQ/1hQ3TN+hjH+L3jdtq5D++4h+AzwR/e/fLfSxV6k3sqndksJNx5xAr2xLf9/QDP1GONz0fP9K/11Pt99Qsr6ZdkSfxOYWegx7OcP20egVtFHoN/f/y7C9SOIJ+hjhMNNrcSzjd5AMMTXDNg0ISvd7fdP3/iZ6srAQKGUpr6vpc3ExfZkK+yf1D4gQ+aL0mWpmAk21XH6YlFJ/oweCQpniz1oopR4FBmmalt20pEIUCxJL8k/p6xhsABpo+Z92VQibIfGk4CmlngfKa5r2WpFdU5IUUdSUUq8BP2ialtephIUQIo3EEiFEQZF4YnkkSRFCCCGEEEJYFFuZglgIIYQQQghhIyRJEUIIIYQQQlgUu8I4acmSJbXAwMDCOLUQIp927959Q9M0X3OXI68knghheWwhnoTciCEpRSPIzy37g4QQhSa7WFIoSUpgYCC7du0qjFMLIfJJKXXO3GXID4knQlgeW4gnT/7yHzeiE1j+fGszl0qI4iu7WCLdvYQQQghR7Cil0JDJg4SwVJKkCCGEEKLYUYBMcCqE5ZIkRQghhBDFjlKSpAhhyQplTIooXhITE7l48SJxcbLQsSVwcnLC398fe3t7cxdFiDyRWGJ5bDueKOnsZcMknliW/MQSSVLEfbt48SLu7u4EBgailDJ3cYo1TdMICwvj4sWLVKxY0dzFESJPJJZYFluPJ3pLiqQptkriieXIbyyR7l7ivsXFxVGiRAkJAhZAKUWJEiWk5khYJYkllsXW44l8ymybxBPLkd9YIkmKKBASBCyHvBfCmsnn17LY8vshY1Jsny1/fq1Nft4LSVKETVBKMXDgwLTfk5KS8PX15ZFHHsnTeQIDA7lx48Z97yOEsE4SS4oPhUxBLAqXxJP7I0mKsAmurq4cOnSI2NhYAFatWkW5cuXMXCohhLWRWFJ8SEuKKGwST+6PJCnCZnTr1o2//voLgHnz5hEcHJy2LTw8nMcff5y6devSvHlzDhw4AEBYWBgPPvggtWrVYvjw4XcNopw9ezZNmzalfv36PP300yQnJxftExJCmIXEkuJBKaQdRRQ6iSf5J0mKsBn9+/dn/vz5xMXFceDAAZo1a5a27b333qNBgwYcOHCATz75hMGDBwPwwQcf0Lp1aw4fPkyPHj04f/48AEePHmXBggVs2bKFffv2YTQamTNnjlmelxCiaEksKR4USmb3EoVO4kn+yRTEokB9sPwwRy7fKtBz1izrwXuP1spxv7p16xISEsK8efPo1q3bXds2b97M4sWLAejYsSNhYWHcunWLjRs3smTJEgAefvhhvL29AVizZg27d++mSZMmAMTGxuLn51eQT0sIkQ2JJaLQSUtKsSHxxDpJkiJsymOPPcYrr7zC+vXrCQsLy/d5NE1jyJAhfPrppwVYOiGEtZBYYvsUSJYiioTEk/yRJEUUqNzUKhSmYcOG4eXlRZ06dVi/fn3a423atGHOnDm88847rF+/npIlS+Lh4UHbtm2ZO3cub7/9NitWrODmzZsAPPDAA3Tv3p2XXnoJPz8/wsPDiYqKIiAgwEzPTIjiRWKJKGxKyYrzxYXEE+skSYqwKf7+/owZM+aex99//32GDRtG3bp1cXFxYebMmYDeHzQ4OJhatWrRsmVLKlSoAEDNmjUZN24cDz74ICkpKdjb2zNx4kSbDQRCiLtJLLF9CllxXhQNiSf5owrjC9q4cWNt165dBX5eYZmOHj1KjRo1zF0MkU5m74lSaremaY3NVKR8k3hSfEgssUy2Gk9emL+XfRci2PBqBzOXShQGiSeWJ6+xRGb3EkIIIUSxo7ekmLsUQoisSJIihBBCiGJHH5MiWYoQlkqSFCGEEEIUO9KSIoRlkyRFCCGEEMWPkiRFCEsmSYoQQgghih2lr5QihLBQkqQIIYQQothRSqYgFsKSSZIirF5YWBj169enfv36lC5dmnLlyqX9npCQUCDX2LRpE7Vq1aJ+/frExsYWyDkBPvnkk7t+b9myZYGdWwiRNxJLiheFLDgvCo/Ek/snSYqweiVKlGDfvn3s27ePUaNG8dJLL6X97uDgQFJS0n1fY86cObzxxhvs27cPZ2fnAii1LmMg2Lp1a4GdWwiRNxJLihclY1JEIZJ4cv8kSRE2aejQoYwaNYpmzZrx2muv8f777/PVV1+lba9duzYhISEAzJ49m6ZNm1K/fn2efvppkpOT7zrXtGnTWLhwIe+88w4DBgxg/fr1PPLII2nbR48ezYwZMwAIDAzkvffeo2HDhtSpU4djx44BEB0dzZNPPkmdOnWoW7cuixcvZuzYscTGxlK/fn0GDBgAgJubG6B3QXj11VepXbs2derUYcGCBQCsX7+e9u3b07t3b6pXr86AAQOku4IQhUhiie1SyBTEomhJPMkbSVKEzbp48SJbt25l/PjxWe5z9OhRFixYwJYtW9i3bx9Go5E5c+bctc/w4cN57LHH+PLLL+/ZlpmSJUuyZ88ennnmmbTg89FHH+Hp6cnBgwc5cOAAHTt25LPPPsPZ2Zl9+/bdc94lS5awb98+9u/fz+rVq3n11Ve5cuUKAHv37uXbb7/lyJEjnDlzhi1btuT1pRFC5IHEEtskLSnCHCSe5J7dfZ9BiPRWjIWrBwv2nKXrQNfP8nxYnz59MBqN2e6zZs0adu/eTZMmTQCIjY3Fz88vX8VM1bNnTwAaNWrEkiVLAFi9ejXz589P28fb2zvbc2zevJng4GCMRiOlSpWiXbt27Ny5Ew8PD5o2bYq/vz8A9evXJyQkhNatW99XmYWwOBJLJJYUMqUse0xKfFIy28+Es/boNSqWdGVoq4rmLpL1knhilfFEkhRhs1xdXdP+b2dnR0pKStrvcXFxgN50OWTIED799NNcnzerc6VydHQEwGg0Fkif04xSz1+Y1xBC3CGxxFYpi21JiU1I5qFvN3I+/DYAdgbFQ7VLU8az4MYdCPOQeJJ7kqSIgpWPWoWiEBgYyJ9//gnAnj17OHv2LAAPPPAA3bt356WXXsLPz4/w8HCioqIICAjI8lwBAQEcOXKE+Ph4YmNjWbNmTY61BZ07d2bixIl8++23ANy8eRNvb2/s7e1JTEzE3t7+rv3btGnDlClTGDJkCOHh4WzcuJEvv/wyrR+pEDZPYkmmJJYUHKXAUttS/j18lfPht/mkRx2aVvThoW838suWEN7sVsPcRbNOEk8yZenxRMakiGKhV69ehIeHU6tWLSZMmEDVqlUBqFmzJuPGjePBBx+kbt26dO7cOa1/ZVbKly9P3759qV27Nn379qVBgwY5Xv/tt9/m5s2b1K5dm3r16rFu3ToARo4cSd26ddMGp6Xq0aMHdevWpV69enTs2JEvvviC0qVL5/PZCyEKisQS26Gw3DEpi/dcpJyXM/2blCfIz41udcowd8d5bsUlmrtoogBJPMmeKozZPBo3bqzt2rWrwM8rLNPRo0epUUNqdyxJZu+JUmq3pmmNzVSkfJN4UnxILLFMthpP3l52kL8PXmXPO53NXKq7XYmMpeVna3m+QxAvP1gNgIMXI3l0wmbe7FadkW0rm7mE1kHiieXJayyRlhQhhBBCFDsKZZHTLi/dewlNg54N/dMeq+PvSYtKJfh5cwiJySnZHC2E7ZAkRQghhBDCAmiaxuLdF2kc4E1gSde7tg1sHsDVW3EcuBhpptIJUbQkSRFCCCFEsWOJUxAfuBjJ6dAYejXyv2dbk4r69LB7z98s6mIJYRaSpAghhBCi2LHEgfObT90A4KFa9w5G9nN3wt/bmT2SpIhiQpIUIYQQQhQ7SlnemJR9FyKoWNIVH1eHTLc3rODNnnMRRVsoIcxEkhQhhBBCFEuWlKJomsa+CxE0KO+V5T4NK3hx9VYclyNii65gQpiJJCnCJnz88cfUqlWLunXrUr9+fXbs2FFo12rfvj0yJa4QtkviSfGgFBaVpVyOjCM0Kp76Fbyy3KdhgD4uRbp8WQ+JJ/knK84Lq7dt2zb+/PNP9uzZg6OjIzdu3CAhIcHcxRJCWCGJJ8WHQllSjpI2IL5+Ni0pNcp44GRvYM+5CB6pW7aISibyS+LJ/ZGWFGEWoaGwc6f+7/26cuUKJUuWxNHREYCSJUtStmxZPvzwQ5o0aULt2rUZOXJkWt/j9u3b89JLL9G4cWNq1KjBzp076dmzJ1WqVOHtt98GICQkhOrVqzNgwABq1KhB7969uX379j3XXrlyJS1atKBhw4b06dOH6OhoAMaOHUvNmjWpW7cur7zyyv0/SSFEpgoyloDEk+JEKSxqTMq+8xE42hmoXtojy33sjQbqlvOSlpRCIvHEskiSIorcvHkagUHJdO0TQ2BQMvPm3d8fiQcffJALFy5QtWpVnn32WTZs2ADA6NGj2blzJ4cOHSI2NpY///wz7RgHBwd27drFqFGj6N69OxMnTuTQoUPMmDGDsLAwAI4fP86zzz7L0aNH8fDwYNKkSXdd98aNG4wbN47Vq1ezZ88eGjduzPjx4wkLC2Pp0qUcPnyYAwcOpAUWIUTBKuhYAhJPihML6+3FvgsR1C7niYNd9rdmDQK8OHw5krjE5CIqWfEg8cTySJIiilRoKAwflYJnry249V+PZ68tDB+Vcl+1Fm5ubuzevZupU6fi6+tLv379mDFjBuvWraNZs2bUqVOHtWvXcvjw4bRjHnvsMQDq1KlDrVq1KFOmDI6OjlSqVIkLFy4AUL58eVq1agXAwIED2bx5813X3b59O0eOHKFVq1bUr1+fmTNncu7cOTw9PXFycuKpp55iyZIluLi45P/JCSEyVRixBCSeFCd6S4q5S6FLTE7h4KXIbAfNp2pYwZvEZI3Dl2VRx4Ii8cQyyZgUUaRCQsDZOw4HvygAHPyicPaKIyTEFV/f/J/XaDTSvn172rdvT506dZgyZQoHDhxg165dlC9fnvfff5+4uLi0/VObXg0GQ9r/U39PSkoC9Okp08v4u6ZpdO7cmXnz5t1Tnv/++481a9awaNEiJkyYwNq1a/P/5IQQ9yisWAIST4oLpRSahbSlHLsSRXxSSraD5lM1MO2z70IkjQJ8CrdgxYTEE8skLSmiSAUGQuxNJxKuuwOQcN2d2AgnAgPzf87jx49z8uTJtN/37dtHtWrVAL3/Z3R0NIsWLcrzec+fP8+2bdsAmDt3Lq1bt75re/PmzdmyZQunTp0CICYmhhMnThAdHU1kZCTdunXjm2++Yf/+/fl9akKILBRGLAGJJ8WJJS3muO9CzoPmU/m5O+Hj6sCp69GFXKriQ+KJZZKWFFGkfH1h2mQDw0e1wtkrjtgIJ6ZNNtxXTUV0dDTPP/88ERER2NnZERQUxNSpU/Hy8qJ27dqULl2aJk2a5Pm81apVY+LEiQwbNoyaNWvyzDPPZHguvsyYMYPg4GDi4+MBGDduHO7u7nTv3p24uDg0TWP8+PH5f3JCiEwVRiwBiSfFirKcMSn7LkRS0s2Rcl7Oudq/sq8rpyVJKTASTyyTKoyZLRo3bqzZ0jzNIntHjx6lRo0aeTomNFRvXg0M5L6DQGEICQnhkUce4dChQ+YuSr5k9p4opXZrmtbYTEXKN4knxYctxhKQeGJJ0seTz1Yc4+fNZznxcVczlwq6fbcJX3dHZg5rmqv9xy4+wKoj19j9TudCLpn1knhiefIaS6QlRZiFr6/lBgAhhPWQWCLySyksYkxKUnIKp0KjaVOlZK6PCfJzY/7OC9yMScDb1aEQS1e8SDyxLDImRYhMBAYGWmUthRDC8kg8sUyWMiYlJOw2CUkpVC3lnutjKvu6AXA6VLp8FTfFKZ5IkiKEEEKIYkdZyJiUE9f0GaWqlZYkRYj0JEkRBcKSVu0t7uS9ENZMPr+WxZbfD4WyiOd37GoUBqV34cqtct7OONoZZIavHFjC+yt0+XkvJEkR983JyYmwsDAJBhZA0zTCwsJwcnIyd1GEyDOJJZbF1uOJxbSkXI0isKQrTvbGXB9jNCgqlnTldGhMIZbMukk8sRz5jSUycF7cN39/fy5evEjo/S7NKgqEk5MT/v7+5i6GEHkmscTy2HI8sZQxKcevRVE9D129UlX2c+PQJVl1PisSTyxLfmKJJCnivtnb21OxYkVzF0MIYeUklogilWGVbnOIS0wmJCyG7vXL5vnYIF83Vhy8Qlxicp5aYYoLiSfWz+K7e4WGws6d+r9CCHE/JJ4IIVKlpij57Q5UEPHk5LVoNA2q5WFmr1SV/dxI0SAkTLp8Cdtk0UnKvHkagUHJdO0TQ2BQMvPmWUC7rBDCKkk8EUKkl9qQkp8cpaDiyXHTzF5V89Pdy9cVgNPXJUkRtsliu3uFhsLwUSl49tqCg18UDtfdGT6qFZ06GWWhHSFEnkg8EUJkpExtKXlNLwoynhy/egsHOwOBJVzzWAqoVNINpZAZvoTNstiWlJAQcPaOw8FPr2Vw8IvC2SuOkBCzFksIYYUkngghMrrTkpK3NCUkBJx9bhdIPDl+LZoqfm4YDXkfH+PsYKScl7OslSJsVpEkKfnptxkYCLE3nUi4rjeBJlx3JzbCicDAQimiEMJKSDwRQhSE27f1f/M6pmTzjTO49t5ESoI+WD3hulu+48nxq7fytIhjRpV83ThzQ5IUYZsKPUnJb79NX1+YNtlA5OJWRM9rT+TiVkybbJCuGUIUYxJPhBAFYd48ja/H6/EjqEbexpScDL+Js4ORlFsuAMTuDcpXPIm4ncC1W/FUzceg+VQVfJy5EB6b7+OFsGSFmqSk77fp1n89nr22MHxUSq5rLYKDFSGnjKxY5ErIKSPBweafLlAIYR4ST4QQBSE1ljjWOgeAZ4+teYolF8JjaVLJi53jWuNoNDJybES+4smJa3oLyP20pFTwcSEyNpHI2MR8n0MIS1WoSUpB9AP39YUmTZAaTyGKOYknQoiCkBpL7NwSAHDwi85TLLlw8zblfVwoU9pAgwBPjly/ma9ypM7slZ/ph1NV8NFbcy6E3873OYSwVIWapEg/cCFEQZF4IoQoCKmxJCnaEYCE0NyPKYmKSyTidmJactA4wIcjV24RE5+U53KcuBqFu6MdZTyd8nxsKn9vSVKE7SrUJEX6gQshCkJUXCLJDnF880Mit/5sQvS8dhJPhBD5knpvErc/EIBbfzTLdSxJHf9R3pQcNAr0JjlFY/+FiDyX4/i1KKqWdkep/Hc9rVDClKTclCRF2J5CXyclOFjRqZORkBBXAgOlm4UQIm8OXYqkzvsr034vOQzsDQaqezixOt6N66u9aBTgTZOK3jjaGc1YUiGEtQgOVtzwMfL1Oti320BQhdwlCqnJQHkfZwAaVvBGKdh17iYtg0rm+vqapnHiWhRda5fJe+HT8XCyx9PZnvPSkiJsUJEs5ujrK8mJECJ/Srg58ma36rg52pOiacQnpXD9VhyXImI5djWKtcevo2ng6mCkTRVfHm9Qjo7V/XCws9hloIQQFsDTQ09MvHxyP7NXareq1JYUT2d7qvq5s+tc3salhEbFE3E7kWql3PJ0XGYq+LhwXmb4EjaocJKU0OPwcxewdwZXP3AvDT4Vwbc6+NUAJ89CuawQwvaUSTzPyNNjwMFVjyXuZaB8FWhYDUq2ICpJ8d/ZcNYeu86qI9f45/BVSrg60L9peYa2rIivu6O5n4IQwlLcOAEzHgFnL5rGuPG0EexOJkFAXfAOvLPCYxYuhN/GzdEOLxf7tMcaBXqzfN9lklO0XC/KmDpovup9zOyVqoKPC0ev3Lrv8whhaQonSTHagcEOYiPgxkmIvgbJCaaNCvxqQoXmENQJKrXTbz6EECIzyghoEHUZLu+FmFD9dwA7J9zLNuSBwFY80PhBPnikHZtO32Tef+eZtP40P206S7/G5Xm+YxB+HvkfnCqEsBHKAMmJEHqCShGXecM+Cn6fp29zKQHlm0HFtlD1IfCpdM/hF27GUt7H5a5xJI0qeDN3x3lOXIuiRhmPXBXj+NX7n9krlb+PM6uOXMtTkiSENSicJMWnMgz9887vKSkQeQFCj8HlfXBhBxxYCLumg9ERqnSGun2hahewk1pPIUQ6JSrDk3/f+T0pHsJOw/UjcGkPXNgOm8bDxi+xcylJh1qP06FDP852bcfUTWeY9995Fu2+yIg2FRnVvjIuDkXSy1UIYYlKBMFT/wKwbOcFPlq8nTWDS+N3+xRc3Annt8Hxv+GfsXrvjzq9oU5f8A4A9JaUiiXvrlhtHOgN6ONScpuknLgWRUk3B0q43f89TwUfFxKSU7h2K46yXs73fT4hLEXR/LU2GPQvuHeAXjsBkJRgCgYr4PASOPYnOPtAw8HQ5CnwqlAkRRNCWBk7RyhVU/+p01t/LPYmnFoDx/6CvbNh5zQq+lbn06YjGNX8Mb5cf5Hv155i8Z5LfPBYLTrVLGXe5yCEMDuDQRGFC3GlGkGJNtD4SX1D+Fk4uRIOL4O14/Sfyg+gNR3JxZvJtK169yDbCj4ulHRzZO+5mwxqHpCrax+/Fn1fK82nVz7dNMSSpAhbYr6RpXYOelevrp/BS0dg4GIIbAVbv4fv6sGSkfrYFiGEyImzt56w9PkFXjkJj00AOyf4638EzGrMhDL/snhoTVwdjQyftYvn5uwhPCYh5/MKIWyW0XQHlKxlGDjvUxGaPQ3DVsALB6D9G3DtMGpeP5arV2gXuwaS76yLopSinr8nBy9F5uq6KSkaJ69FFViSkrpmi8zwJWyNZUx/Y7TTx6f0m60HhObPwtHlMLGZnqxEnDd3CYUQ1sLJAxoOgpHr4alVENAaNnxGoyVtWNFgB290qsDKI1d56NuNrDt23dylFUKYicE0riQ5JZvZvbwDoP1YePEgZ9p9RyJG2h5+GyY21VtaTAlOHX9PToVG52pRx0sRsdxOSKZaAQyaByjr5YxB6eNlhLAllpGkpOdVHh76GF48CK3GwJHf4YfGsOo9iI82d+mEENZCKSjfFILnwqgtUKkdxvUf8/T+PqzvfJUSLvY8OWMnn/x9lMTkFHOXVghRxFIHmadkbEnJjJ0DB7070y3hUy51mQZGB/htCEzvDJf2UKecJ5oGhy/nPMtW6kxcBdWS4mBnoIyns6w6L2yO5SUpQGgo7DxSktD6H8Lzu6F2L9jyrV5zceSPtJoLIYTITmgo7NwJocba0H8ODPsXPMpRbv1L/O35GS/XS2LqxjP0m7KNq5Fx5i6uEKII2ZmSlKTk3N1THLtwGw0DKRV6wjNb9G6lEefhp460OPYJHkTnqsvXgYuRGA2KmrkcZJ8b5X2cpbuXsDkWl6TMm6cRGJRM1z4xBAYlM+/vctDjRxi2Uu93vnAQLBwM0aHmLqoQwoLdE0vmafrU50+tgke/xxB6lDEnn+KfBts5ffUmj03YzJ7zeVuQTQhhvVK7e+WmJWXePI3x026TEmtP9Rowb4FB71Y6eic0G4XLgVmsdhpL8rEVOZ5r34UIqpd2x9nBeN/PIVV5bxdpSRE2x6KSlNBQGD4qBc9eW3Drvx7PXlsYPiqF0FCgQjMYuQEeeA9O/AOTmunjVoQQIoNsY4nBAI2GwOhdUONRqh/9nh2lPqeK8TL9p2xn6d6L5i6+EKIIpHb3ynZMCnfiiWPlKxicE++OJ06e+gRAI9aSYO/JyItvwNJnID4q03OlpGjsvxhBvfJeBfpcKvi4cD0qntiE5AI9rxDmZFFJSkgIOHvH4eCnf7kd/KJw9oojJMS0g9EO2rwMT28Cz/KwYCD8+RIkymAxIcQdOcYSANeS+mxgfWfhFHOJ2cmv87+SO3hpwT5+WHMSTbqVCmHTDKlJSg7f9dR4YnDQE4BM40nZBvzedA4Tkh5HOzAfprTV13HK4MyNGKLikqhfwEmKv48+9fClCLkfErbDopKUwECIvelEwnV9MFnCdXdiI5wIDMywo191vctGyzGw62f4qaO+uJsQQpCHWAJQszuM2oLyb8zTkd+w2O8XJq06wBtLDpIkA+qFsFnG1O5eObSkpMYTLUXfP6t4UrOCL18l9eXIg3P1teCmd4YdU+4aR7v/QgRAwScpprVSJEkRtsSikhRfX5g22UDk4lZEz2tP5OJWTJtswNc3k53tHODBj/T1VaKuwtT2cPTPTHYUQhQ3eYolAB5lYNAy6PA2DW+tYaP3x2zf9R+j5+4lPkm6Twhhi3Lb3cvXF376UYEGCUcqZBlP6pTzBGBrYjV4ZjMEdYYVr8GSEZAQA+jjUdwc7ajs61agz6WcaRHHizdlXIqwHRaVpAAEBytCThlZsciVkFNGgoNV9gcEdYKnN0CJyrBgAGz4ktDrmj6jj4ytF6LYynMsMRih3auogYvxJZx/Xd8n+ugqBk3dxaZtSRJPhLAxaeuk5KJrZ88+GsqoMbSPc5bxpKSbI2U9nfQZvpy9of9c6Pg2HFwEPz8EkRfZdTaCAA9PwsNyiEd5VMrDCTuD4pKslSJsiMUlKaDXWjRpQta1nhl5VYAn/4G6/WHdONaNeYoe/W/cmdFHCFEs5TmWAAQ9ACM34OhTgZkOX1Dt4lye+Gk7FWvESzwRwoakrZOSi16d0aZFGqsH2WUbT+r4e7L/YoT+i8EAbV+FAb9BeAix33fA4eoe9m5wLfD7E6NBUcbLSbp7CZtikUlKvtg7Edp6Mu9ufpfe1ZawdEBXAvusuDMDhxBC5JZ3ADe6/8s/pzvzkcMM3ik7iRKDVzPi+QSJJ0LYCGMuB84DaSvJuzrYZbtf80olOBd2m5AbMXcerNKZ8J6ruBbpyAKnD3mk9uK7ZwgrIOW8nKUlRdgU20lSgJBzikmnRvFc4hjqqrP8UfYVqvqfvHsGDiGEyIWzl915cvNP/JL0ECPt/mKC6/eU6r6FoyeTzF00IUQBuDMmJeemlKg4/Xvv5pR9ktK5ZikAVh25dtfjp6Nq8MiRaRzTKvCj/bcMK7Pk3hnC7lM5LxcuSpIibIhNJSmpM3D8frUTgxLGUoJb/N21O5Xdj5q7aEIIKxMYCDHhrrx1+Tk+SQzmUeN2ZpZ9h8m7N8paBELYgNTZvXIziV9qdy93x+yTFH9vF2qW8WDlkat3PR4YCGEeBvrHvs26lPqMs/+FF6t/R2BAwXX5KuftzLWoOBKSZFZCYRusOkkJDeWuAfLpZ/RZO/cZOs/+F09P8Pm9K1zabd7CCiEsWtbxpDXjZ3/NiH8m0cJ4lFdCX+WlGWtl1i8hrJzBdAeU0+xeANG5bEkBvTVl97mbrN4cnxZPkh3icA68Qfj+6gz49Xd+PRzM2y0+x3fPm7kbFJML/t7OaBpcjYwrkPMJYW5Wm6TMm6cRGJRM1z4xdw1ASz+jz9876uDy3Apw9ICZ3eHcNjOXWghhiXITTz5ZPgBjv1nUszvHCxdf5q3Z62UdFSGsWNrA+dyMSUkwJSk5tKQApJz3I0WDge9cSIsnC3ZeIAWNfyYE8OdvnnSZOAmaPQPbJ8GfLxRIouKfOg1xhExDLGyDVSYpoaEwfFQKnr224NZ//T0D0O6a0cenEgz7B9xLwexeELLFrGUXQliWPMWTGo9gHLCQKnbXGHFmDJ/8tkFWphfCSt3p7pXzdzhtTEoOSUpoKLz7shtassKp2XE9njyTzNwd52kdVJKGVV31eOJngC6f6rN/7ZkFf4yGlPtrnS3nnbpWioxLEbbBKpOUkBBw9o7DwS8KAAe/qOwHoHmUhaF/gWc5mNMbQjYXVVGFEBYuz/GkckfsBv5GJbtQgo88x8TlW4uqqEKIAmTIQ0tK6piUnLp76fEkHmXUz+ngF4Vbjctci4rjiWYV7t5ZKX0dlfZvwL458Ptz99WiUsbTGaWQGb6EzbDKJCV1gHzCdXcAEq67ExvhRGBgNge5lzYlKuVhTl84v6MoiiqEsHD5iieV2mE3aBEBxhs8sOtp5q/fVwQlFUIUpLy0pETHJWFQ4GxvzHa/1HiSGO4CgKaBQ6MT+Lg40KlGqcwPaj8WOrwF++fdV9cvBzsDpdxlrRRhO6wySUk/QD56XnsiF7di2mRDzgu2ufnBkD/0rl9zestgeiFEvuOJqtgW44AFBBmvUnvtEFbtPl40BRZCFIg7UxDnriXFzdEOpbJfKT41nkQsaMvtvxsTe7ACHp4aw9tWxMEum1uudq/d6fq14jU9u8mHct6yVoqwHTmPALNQwcGKTp2MhIS4EhiYhxWl3UvDkOXwSzd9jMqTK8CvRmEWVQhh4fIbT4xBHUjuO5tqCwaQ8MdA9nouo0FQuUItqxCiYOQ1SXF3ss/Vee/Ek1IEBpbC17dO7grU4S1IioOtP4CjO3R6L3fHpVPOy5m9F27m+TghLJFVtqSkumtAa154+sPg38HoCL/2gJshhVE8IYQVyW88cajRhbhHp1BPneL27Cc4d11uEISwBnlZcT46LglXx+y7eqWXr3iiFHT+CBo9CZvHw5bv8nCwzt/bmSsRcblKvISwdFadpNwXn4owaCkkxsKsxyH6urlLJISwUu6NehPe4QtasY/TUwcSGRNv7iIJIXJgMHXdSslDd69CpxQ8/DXU6gmr3oU9v+bp8HLeziSlaFyPkrVShPUrvkkKQKmaMGARRF+DOX0gPsrcJRJCWCnfdiM432gsHZM2s/XHkSTKYo9CWLS8dPeKik/CLZfdve6bwQg9pkDlB2D5C3D8n1wfWs5LpiEWtqN4JykA5ZtAnxlw9SAsHAxJCeYukRDCSlV4ZCwnKg6ia/QyVk9/S9ZQEcKCpc3ulYuvaUx8Eu5F0ZKSys4B+s6CMnXht6FwYWeuDvM3rZUig+eFLZAkBaDqQ/Dod3B6Lfz5Yr5n1RBCFHNKUXXQ9xzx6UTXKz+yaelkc5dICJEFg+kOKFfdveKKqLtXeo5u8MRv+oQ/8/pB2OkcDynnpU99LNMQC1sgSUqqhoPuLKi08Utzl0YIYa0MBqqPmsNxp7o02/82B7euMHeJhBCZyNPA+fgkXIs6SQFw84WBi/XK0zl94HZ4trs7Oxgp4erAxZu3i6iAQhSeQklSouOTrLObQ7vXoV4wrPsY9i8wd2mEEFbK4OBE2acXc91YivIrh3Pp1AFzF0kIkYEhl4s5pqRo+sD5HFabLzQlKkPwPIi8CPOCITH7QfH+3s4yJkXYhEJJUs7eiKHHpK2sOnItV82oFkMpePR7CGwDf4yG89vNXSIhhJVy9/bDMHAxKRjQ5vYjJiLU3EUSQqRjZ8jd7F4xCUkARTsmJaMKzaHHZLiwHZaPybZbejlvZ+nuJWxCoSQp5bycCYuJZ8SsXXT9bhO/77tEUnJKYVyq4KUOVvMsD/MHyBoqQoh8K1epBhc6/4Rv8nUuTumNJhNzCGExUrt7JeWUpMTrM/WZrSUlVe2e0OFtOLAANn2d5W7+3i5cuhlrnT1ahEinUJIUH1cH1v2vPd/0q0eKpvHC/H10/HoDc3ecJ94apuV08YEnFkJKIsztD3G3zF0iIYSVqteqC1trvku12H0cmf60TMwhhIVQSqEUpOTwnYyOTwQo+oHzmWn7CtTpC2s/giO/Z7pLOS9n4pNSuBEtlSLCuhXawHk7o4EeDfz598W2TBnUCG9XB95cepA2n6/jp41niIlPKqxLF4ySQXqLyo0TsGQkpFhJS5AQwuK07zuGlT5PUOvKEk79/a25iyOEMDEqleOYlKg4/X7FIpIUpeCxH8C/CSwdBVcP3bNL6jTEMnheWLtCn93LYFA8VKs0y55tyZzhzQjyc+Pjv4/S8rO1fLPqBDdjLDjTr9QeunwKJ1bog+mFECIflFK0fvo7tts1IWDnR1w/sNrcRRJCoN+j5DS7V7SpUtXs3b1S2TtBv9ng5AnzgyEm7K7N5VLXSpFxKcLKFdkUxEopWgWVZO6I5ix5tiVNAn34bs1JWn2+lo//OsK1W9nPVmE2TUdCg0Gw6Ss4vNTcpRFCWCkXRwdKD5vNeUrjuPRJ4kPPmrtIQhR7RqVyHDgfbUktKancS0P/ORB1DX4bAsmJaZtk1XlhK8yyTkrDCt5MG9KYf19sS+eapZi++SxtPl/Hm0sPcj7MwponlYKHv4byzWDZs3DtsLlLJISwUoFlS3O16wxISSJseh9IsLB4J0QxYzQocprXJ60lxZKSFIByjeCx7yFkE6x8J+1hdyd7PJ3tZdV5YfXMuphjtdLufNe/AeteaU/vxv4s2nWR9l+t48X5ezl+NcqcRbubnaM+PsXRQ5/xK/amuUskhLBSrZo1ZVX1cZSOPcW5mcNlIL0QZmTI1cB50xTEltLdK716/aHZM7DjR9g/P+3hcl7OMiZFWD2LWHE+oIQrn/Sow6bXO/BU64qsPHKNh77dyIhZu9h73kISAvfSeqISeREWj5CB9EKIfHu875P85jmYgEt/cWXleHMXR4hiS29JyV13L7OsOJ8bD34EAa1h+QtwZT+gD56XMSnC2llEkpKqlIcTbz1ck61jO/Jipyr8dzacHpO28sRP29l88ob55/yu0Ay6fg6nVsHGL8xbFiGE1bIzGnhgxBesV03x3fYxMSc3mbtIQhRLRoMhx3VSouOTcLQzYG+0qFumO4z20GcGOPvAgkEQe5NyplXnzX7fJMR9sMhvnJeLAy92qsqWsR15q1sNTl2PZuD0HTw+aSv/Hr5q3lXsGw+Dek/A+s/gxErzlUMIYdVKujvhEfwTFzRfkuYPRrt1xdxFEqLYMRpyXnE+Oj7JMrt6pefmq/f2uHUZlozE38uJ2wnJRNxOzPlYISyURSYpqdwc7RjRthIbX+vAJz3qcDMmgad/3U2X7zaybK+ZVrFPHUhfqjYsGS4r0gsh8q1h1UB2Nv0e+6QYrv38BCRb+PpRQtgYo8rdFMQWN2g+M+Wb6MsmnFxJ2yu/ADLDl7BuFp2kpHKyN/JEswqs/V87vu1XH4XixQX76PD1emZvP0dcYhGvYu/gAv1mgQYsHAJJ8UV7fSGEzejT7UFm+/2P0hF7uLbsLXMXR4hixWDI3RTEFrNGSk6aDIe6/Qg6MoFWhoNcipDB88J6WUWSksrOaODxBuVY8UIbfhrcmBKujry97BBtvljHlA2n02bgKBI+laDHj3BlH/zzRtFdVwhhU5RS9H3yJZYYu1Dq4GRi9v9u7iIJUWwYc7GYY1R8Eq4OVpKkKAWPfENKyWp8bz+B8Csh5i6REPlmVUlKKoNB0blmKZY+25K5w5tRrZQ7n644RstP1zC+KFexr/4wtBwDu6bDgd+K5ppCCJvj5eJApYHfcSClEur3Z9DCZaFHIYqCUeVudi+LH5OSnoMrhn6zcCaB1vteu2uhRyGsiVUmKamUUrQMKsns4c1Y9lwrmlUqwfemVezH/XmEq5FFsIr9A+9BhRbw54tw42ThX08IYZPqVyzNkdY/kJiscWPGE9KNVIgiYDCoHNdJiUmwkjEp6SjfanzrMoYKMQdg7UfmLo4Q+WLVSUp69ct78dPgxqx8qS0P1SrNL1tDaPvFOt5YcoCQGzGFd2GjHfSari/4uHCIrCAthMi3fp1b8Wup1/G9dYTQJa+ZuzhC2LzctqRYzZiUdM6Ufog/HbrClu/g+D/mLo4QeWYzSUqqqqXc+aZffda/0p4+jf1ZvOcSHb9ez5h5ezl29VbhXNSzHPSYCtcPwwq5sRBC5I9SioFDn2W+8VF8j8wgZt8ScxdJCJumL+aY/T5R8Um4OdoXTYEKUHkfF96OewKtdF1YNgoiLpi7SELkic0lKanK+7jwcY86bH6tAyPaVmLN0Wt0+XYTT83Yye5zhbCKfZVO0Ppl2PurjE8RQuSbl4sDVQd8xb6Uyqg/Rsv4FCEKkZ6kZJ2lJCSlkJCUgpujsQhLVTAq+LgQkWDk5sNT9enNFz8l41OEVbHZJCWVn4cTb3StwdaxD/By56rsPn+TXj9upf/UbWw6GVqwq7F2eAvKNzeNTzlVcOcVQhQrDSuV5mCLb0hK1gibORCSimgyECGKGYNBkZzNbUCMadZQaxuTAnqSAhCilYZHv4ULO2Ddx+YtlBB5YPNJSipPF3vGPFCFLa935O2Ha3D2RgyDpv9H94lb+OdQAa1ib7SD3tPBaA+LhkJiEQzcF0LYpAEPteWXkq9QMvIQYb/LNOdCFAajyn7F+dSlDdycrK+7V0AJPUm5EH4b6vSGRkNh8zdwarV5CyZELhWbJCWVq6Mdw9voq9h/2rMOEbcTGTV7Nw99u5Eley6SeL+r2Hv6w+OT4epBWPVuwRRaCFHsGAyKAU+OZqGhCyUOTiPuyN/mLpIQNkfv7pV1khIVl9qSYn3dvfy99STlfJhpQp8un4FfTVjyNERdNWPJhMidYpekpHK0MxLcVF/F/rv+9TEoxcsL99Phq/X8er+r2FfrAs2fhf+mwLG/Cq7QQohipaSbI/79xnMkJYCkxaPg1mVzF0kIm2JQ2S/mGGu6F3Cyt74kxcneSCkPR86Hm5IUe2fo/QskxMCSkZDNWBwhLEGxTVJS2RkNdK+vr2I/bXBjfN0deWfZIVp/vo7JG04TFZfPQWad3ocy9eD35yDyYoGWWQhRfLSsVo7tDb/EkBRH6KwhkHIfFShCiLsYDSrb7l7xVpykgD4u5Vx4uqUR/KpD18/h7AbYPN58BRMiF4p9kpLKYFB0qlmKJc+0ZN6I5tQo485nK47R6rO1jF95nPC8rmJv56jXWCQnwuIRcmMhhMi3wY925mfP5/C98R8RKz8zd3GEsBlGQ/YtKXFJ+t9uZytNUsr7uOhjUtJrOBhq9YR1n8CF/8xTMCFyQZKUDJRStKhcgl+fasbvz7WiZeWSfL/2FK0+W8uHy49wJTI29ycrURke/hrOb4WNXxVeoYUQNs3OaKDHk6/yF61x3/4ViWe3mrtIQtgEg8q+JSUuUe8SZc0tKVdvxd3dhV0peOQbfY23RU9BbITZyidEdiRJyUa98l5MHtSIVS+1pWvt0szcpq9iP3ZxHlaxr9cf6vaDDZ/BObmxEELkTzlvFxy6f8ellBLcnvckxBbCek9CFDN2BkVStklKancv67xdquDjgqbBpYgMFazOXtBrOty6pC+bUJDLMQhRQKzzW1fEqpRyZ7xpFft+TcqzZK++iv3z8/Zy9EouVrF/+GvwCtC7fcmNhRAinzo3COLvap/gEh/K9bnPyI2FEPfJkMPsXrbQkgLcGTyfXvmm0OFNOLwU9s4u4pIJkTNJUvKgvI8L4x6vw+bX9VXs1x27TtfvNjFsxk52nwvP+kBHd339lOirsPwFubEQQuTb0L69+NV5AH4XVnBr68/mLo4QVs2oFCnZjUlJbUmxs9IkJf1aKZlp/RIEtoEVr8GNk0VYMiFyJklKPvi566vYb3m9I//rXJW952/S68dt9JuyjY0nsljFvlwj6PgOHPkd9swq+kILIWyCk72Rtk+OY5tWC4fVb5By/bi5iySE1cppnZTUgfOOVtrdy9fNESd7w521UjIyGKHnVH2yn0XDICm+aAsoRDas81tnITxd7Hn+gSpsGduRdx6pybmw2wz++T8em7CFfw5duXcwXssxUKk9rHgdQk+YpcxCCOsXVMqTaw98z+0Ue8JmDZIbCyHyyWBQZJOjEJeQjFLgaGedt0tKKSr4uGTe3SuVR1noPgmuHoA1HxZd4YTIgXV+6yyMi4MdT7WuyIbX2vNZzzrciktk1Ow9dP5mA4t2p1vF3mCAHlPAwQUWS42FECL/urdpxMJyY/GNPs7VpW+ZuzhCWCWjIoeWlBQc7QwopYqwVAUrxyQFoHo3aDIctk2AU2uKpmBC5ECSlALkaGekf9MKrP1fe34IboC90cArv+2n/Zfrmbk1RO/b6l4auk+EqwelxkIIkW9KKZ4YPIolxi6UPvwTMUdWmrtIQlidnAfOJ1vtoPlU5U1JSqZd0dN7cBz4VoeloyA6tGgKJ0Q2JEkpBEaD4tF6ZVnxQht+HtqY0p5OvPfHYVp/vpZJ609xK6ATNBlhqrFYbe7iCiGslIeTPRUHfMvJlHIkLR6JFn3d3EUSNk7TNL7895i5i1FgcjNw3loHzacK8HHhdkIyoVE59N6wd9anJY6LhN+fk0l+hNlJklKIlFJ0rF6KRaNaMH9kc2qW9eSLf47T6rO1fKsGkVSiOix9RmoshBD51qBSGfY0HY9TUjSXZz4lNxaiUK0/HsrEdafNXYwCk+PA+cQUq10jJVVlPzcATofmYn230rXhwY/g5L/w30+FXDIhspfjN08p9bxSyrsoCmOrlFI0r1SCWcOasnx0a1oHleS7TRfpeX0YSbcjiFv8tNxYiMKVlGDuEgASTwpLn24PscB7BOVCN3JtzQ/mLo6wUSkpGl+tPE4lb3tzFwUomHiSc5Ji/d29KvumJinRuTug6Uio8iCsfBuuHS7EkgmRvdxUD5QCdiqlFiqluihrHj1mAer4e/LjQH0V+6A6zRiX+AROZ9fw+9T3OJvbVeyFyAtN0ydqsAwSTwqBwaDoOuw9NqlGeG/+kLiLB81dJGGD/j18lYuXL/OHetncRUl13/HEaFAkZ9fdKykFRytPUkp7OOHiYMx9kqKUPtuXkycsegoSY3M+Roj8OLsp2805Jimapr0NVAGmA0OBk0qpT5RSlQuifMVVkJ874/vW56mXPuG4R0u6XJ7Es+NnMXruHg5fjjR38YQt2TMLji43dykAiSeFydfDCfuek4jUXIj4dZDcWIgClZyi8fXK40xw+wXXuKvmLg5QMPHEoHLRkmKl0w+nMhgUlXxdc9fdK5WbLzz+I4QehVXvFl7hRPF1OxyWjMx2l1x98zR9Soirpp8kwBtYpJT64n7LWNyVL+FKtZGzsHf1YpbnVLYdv8TD32/myV/+Y1dINqvYC5GF0FDYuVP/l9AT8M9YqNjO3MVKI/Gk8DSvU511NT6kdPxZzs6zmNpuYcVS48nsTZdoFP4nbZK2oR6wnJvW+40nRoO6d02zdOJtoLsX6F2+Tl/PZUtKqiqdoPlz8N9UOP5P4RRMFBt33ZtoGiwfAzHZj8nOzZiUF5RSu4EvgC1AHU3TngEaAb0KoNzCzRdDz8n4xp5ha8O1vPJgVfZfjKT35G30nbyN9cev5zx1oBDAvHkagUHJdO0TQ9Vqtwmf9hTYOenr81gAiSeFr0efwfzh3IOKZ+YSumuZuYsjrFhaPOkXxc+L1vK+/Sy0iu2hxWhzFw0omHiSY3cvGxg4D3qScikiltiE5Lwd2Ok9KFUHfn8WoiyjBU1Yn/T3JoFByfw3eabewyOHCo/cfPN8gJ6apj2kadpvmqYlAmialgI8cv9FFwAEdYIWo3Hc+zOjy51i8+sdeO/Rmly4eZuhv+zkkR828/fBTFaxF8IkNBSGj0rBs9cW3Pqv59Ohz+ATf4DI9hPAo4y5i5dK4kkhszcaaDjsG44SiONfz5Nw85K5iySsUPp44tN3DRO9v+Z2vDPh7SbrCxNbhvuOJwalSEnJentcku20pACcuZHH1hQ7R+g9HRJu6+unZPdiCZGJjPcmjfrNp/bFsST4t8+xwiM3Y1Le0zTtXBbbjuavyCJTD7wLpevAsmdxiQvlyVYV2fBqB77oVZfbCck8O2cPnb7ZwG+7LtxZxV4Ik5AQcPaOw8EvijaGA4zyWMb0o0M4YXjY3EVLI/GkaPj7enPjwUnYp8Rx8echcmMh8ix9PHnFbiG1DSGM2fQtZ0ItpsKjQOKJ0UAOLSnJONtCkuLnCuRyGuKMfKtBl0/hzDrYPrGASyZsXfpY4kAiP5b+nNvJzhypNiXHCg+LqQ4R6DUWvX7WB7wu02ssHOwM9G1SntUvt2PCEw1wtDPy6qIDtP9yPTO2nM17062wWYGBEHvTCbcbKXxtP5ljCRV4bdUnBAaau2TCHNq0bMWqCi9RKWonJ3//1NzFEVYmNZ60SjjCSLu/+CWmG8sOPmZz8cRoGjifVZdqvbuX9ScpgSVcUYq8j0tJ1Wgo1HgUVn8Al/cVZNGEjUuNJUnRDrxqt4BahnOM+ncC5aqXzvFYSVIsjW9V6PoZnFkP2+6sd2A0KB6pW5a/x7Tml6FNKOPpxPvLj9D687VMXHeKW3GJ5iuzsAi+vjDtR8WXCbPwSI5l8Py5TJjoiq+vuUsmzOXBQa+xxb4lgfu/JvT4dnMXR1gRX1+Y9O0lvnH7gWPxgbwy/SemTTbYXDwxmmpys+pJHZeYjKMNjElxsjdS3tsl99MQZ6QUPPo9uPnB4qcgPp/nEcWOry9M+RE62O9jhN3fzEp4kMdf7ZKrWGL93zxb1HCIXmOx5kO4tOeuTUopOlT3Y9EzLVkwsjm1y3ny5b/HafXpWr789xhh0fFmKrSwBMGVptK10kqu1RvHXzvqEBwsy5AUZ04OdvgP+YkbmheJC4eRGHvL3EUS1kLTaBY+Ak91m6stp3PsuJtNxhOj6S4os2mIU1I04pNScLKz/pYUgMp5nYY4IxcffRKWsNPwz+sFVzBh80oFHWG8x/ecN1bk45QnaNA+d0muJCmWKK3GopSpxiIq092aVSrBzGFN+fP51rSpWpJJ60/T6vO1vP/HYS5HyBoJxc7Vg7DqHajalYDeI2yuxlPkT4C/P6fbjKd00mWOTh9l7uIIK3F99bdUj9rOmgpjaP9oY5uNJwaDnnilZNLdKz5JH8tlC929QB88fyY0+v4m4KnYBtr8D/bOhkOLC65wwnalpFBy9Yt4qDiMfacTjwN7zkfk6lBJUiyViw/0/AluhsBfr2S7a+1ynkwa0IhVL7Xjkbplmb39HG2/WMdri/ZzJr9Nu8K6JMTAomHg7APdJ+qJrhAmrTs9zvpSg6l74y8O/zvd3MURlu7Kfry3fMw6GtGin23XmBtNsTKzlpS4RH3Mpy1MQQxQ2c+N+KQULt1vJWb7seDfBJa/qN+jCJGNa6u+oUHCLnZWfZmyVRtS0s2Bvedv5upY2/jm2arAVtD2VTgwH/YvyHH3ID83vupTjw2vdWBg8wD+2H+ZB8Zv4Lk5ezh0SVaxt2n/jIUbJ6HnFHAtYe7SCAvU8qkvOWKsToVtb3Ht3DFzF0dYqvhobs8dQpjmxrlWX+Dt5mjuEhUqo6klJbMZvuKSUpMU22lJATiV38HzqYz20Gua/v/FwyFZxsSKLFzeR8ntn7AyuRE1H3sZpRQNKnizV1pSbETb16BCC/jrZQg/k6tDynk58/5jtdj8ekeeaVeZjSdCeeSHzQz5+T/+Oyur2NucQ4thzyxo/RJUam/u0ggL5eToiNsTM9A0ReTswSQmyPg1ca+Uv1/DKSqET51eJrhDA3MXp9AZTC0pmXWBiktM7e5lG7dKVUvpScqxq5l3Ic8T70B49Fu4uBPWy+yBIhPxUbBoGOHKi7mlX8PHVOHRoIIXZ2/EEB6TkOMpbOObZ8uMdnq3L4NR786TlPWbGhoKO3fq/wKUdHPktS7V2fJGR159qBqHLkXSd8o2+kzeyjpZxd423AyB5S/pTe8d3jR3aYSFq1C5BsebfUzVxOPs/OV/2e6bMZ6IYuDgIgz75zAxqTtdH+2Do40MGM9OWktKdt29bOR18HJxwN/bueB6VtTuBQ0GwqbxcGZDtrtKPCmG/n4N7eZZRsc+Q9NaQWkPN6zgDcC+Czl3+ZIkxRp4lYfHJsDlvbD2w0x3mTdPIzAoma59YggMSmbevDsB18PJnuc6BLH59Y68/2hNLt2M5UnTKvZ/HbiSaXAWViA5ERY9BWh607vR3twlElagSbdh7CzxKC2v/MruNYsy3Se7eCJsVPgZtOUvsI9qbCs/godq5byGgS0wZNfdK9G2unsB1CnnycGC7P7d9QsoWQWWjIDozDMQiSfF0P75sH8uhyuPZIdWgweql0rbVNffE6NBsedcRI6nsSvEIoqCVPMxaPwUbP2BE4nt8G7aOW22ldBQGD4qBc9eW/QVPa+7M3xUKzp1Mt41I4uzg5GhrSryRLMAlu27xOT1p3lu7h4qlXRlVPvKPF6/HA52krdajbXj4NIu6DNDb3oXIpfqDv+R81/tJ3Djy6xKakz9hoF5jifChiQlwKJhxCUrRsc/y5RH66CKyeQbxrTuXvduS+3uZQvrpKSqXc6TFYeuEnk7EU+XAqjYcnCF3j/DTw8Qv/AZNlSexazjh9l85joGBa4OdlxeWRXPXudx8IuWeFIc3DgFf74MFVryQ3JPynnFpHU1BHBxsKN6aXf25GLwvO1884qBhZHjOBRaE59NI2le72JabURICDh7x+Hgp/czdfCLwtE9jr//zrxp1cHOQN/G5Vn1cjsmPtEQZwcjry06QPsv1/GLrGJvHU6tgS3f6qsA1+ohTekiTxyd3dlSagau3MZx/WAqVovLMp4Y3eKxd05g714zFlgUrjUfwOW9vBw3nKZ1GnH7kmexiSV2ppaUpEyyFFsbOA96LTbAocsF2JpSug47S3yM4/lV7Pz7VTYdv0GrkoE81yGIAA9P3NsdwcFPH6wv8cTGJcbBoifBzoG47pPZcPImtX38uHHj7kqPiiVduRoZl+PpJEmxEqGh8OSzjjybOBoXxxh+HdyPkc8kEhoKgYEQe9OJhOvuAERur0jENSdefjf7plWjQfFw3TL8+XxrZjzZBH9vFz5It4p9ZKzM2GGJwkKukrhwJEk+NeChT6UpXeRZaCgMf7U+79waTlvH/bwz4kWGj0q5J57EHC3DpantiElIoUdv+WzZosid/8C2Cfzj9CjracnUF4OKVSxJWyclk5aUeBsbkwJQu6yepBRkl6/QUOj8fXf+SW7Ca07zqB17nmUfVWdwg2pMH9SM6G3V0DRIjnbg0tS2Ek9sWOwf78DVA0R2mMRni5yJS0pm2S8e98QTFwcjt3NRIS5JipVIrd087+PFO4lP0tLpEO+0+5SQEPD1hWmTDUQubsWt2e2J3FKN0oO24B68Hs9eW9JuPrKilKJ9NT8WjmrBwqdbUMdfX8W+9Wdr+eKfY9yQVewtxvy5SRz6cDiJt6Np9s10pvzinNY1x61/7t5vIVLjyW+OrfkjuQWvOM2nU/sld8WTiN9aEbaiLqWf2E65kevx7C2fLVuzbNZFkhaN4mBUNV6I6MXVNVXweGRnsYolaSvOZzomxbZm9wLwdtUHzxdkkrLvWDweD+3ntcQRXMObKb5fUNrvKiEhUMpP8cOoykRtqInRLYGyT22SeGKjNk39HeeDU/nx8EjKPPAQk/64iqaBc7uD98QTFwc7YhMlSbEZ6Ws3F6e0ZWF0R16p/w1V7PQZNYKDFSGnjHzzsSveZe/u+uXsFUdISO6u07SiDzOe1Fexb1vNlx83nKbVZ2t57/dDXLx5u3CenMiV0FA4NfML2lXYxHvaYK50COOF/6Xg5JX/91sUT3fiiQdvJj7F+ZRSTK73GsmGs4AeT5YtNuJZKl4+WzYq9FoSvpuewtExjjEOI7kd60L0Sd9i934bcrWYo+20pIBp8PzFgklSklM0ph7ci7JPIizMj9EJYyjFTcY3fY3AAP01DQ5WzPsgkMQLJbFz1ys9i8vnq7gIO32OOmdGsze+CuMrt8al416cql1OW1c64/vtZG/M1dACSVKsRPrWkuh57Rk9YwZRjlXwWj0coq6l7dOtG8RF3On6lXDdndgIJwID83a92uU8mfhEQ1a/3I7u9csyZ8d52n+5nld+23//C0GJfAnbtYE3m3/B4uTW/JbcDge/KFy84rgddv/vtyhe0seTq/O6MXDpTEqoW6T89RQRMXo/4QYNIOGWfLZsVdLKcbQqt523kp7krFYGFCRHFb/3O3UK4pRiMrsX6H/fz4ffJvL2/XfpnvvfeXaeD6N3YC0iFrZl89zhvLvxPbpX+RPfM1PS9mvYUBG5qi5asv56J1x3Kxafr2IhKQHHP4aiDDCGZ0nEDpfKNzA4JJMY7gLcG09cHIwkJKeQlJxJP8t0JEmxIqmtJSsWuXL4qCeeT83UF8tZ/BSk6ME0YzITubgV0yYb8j2LRmVfN77oXY+NplXs/zxwmc7fbODZObtlFfuiFHWNqgeGcyI8iNcvjwEUCdfdiY9y4tvxBfd+i+IjfTxZtq4511q+R8uU3fz705skp2gFHkuEBTmxkjKnv+GXE8EsS24NQPItZ+ztFJGLitf7bcymJSXWBrt7QcENno+8ncj4lcdpXsmHr54rnxZPXl4wBqp2hZVvw8XdgH5v8tN3TtxaXQ+AuAOVisXnq1hY9S5ukXt4+p8JnL5mWg/FoJF43YOI+W0yjSfOpsQ/py5fMgWxlfH1Jd2XuiY8/BX8/hxs+DxtMb/gYEWnTkZCQlwJDKRAgkBZ0yr2ozsG8cuWs8zado6/D16lbVVfnmtfmWaVStz/RUTmUpJh8VMYEqM402gZV16ojrNXHLERTkybbCA4WNGzR8G+36J4uCuePPg8ISFb6HX5F35b0pT+vfsXSiwRZhZ5EZY+jVaqNnNVP7TwRGL+bkrsdXdmTDfQqZMqVu+3oRgt5pgq/eD5VkEl832e79eeJCI2kXcfqYVSKl08UfD4JJjSDn4bCqM2grM3wcGKjg+Upd/0MyT1PkXvvuX0fYX1OvIH7PgRmj1D92qPsXxUCq5VruLSaR+9Gwbwxud2hITY3RNPnB1MSUoOXb5sq3qgOGowEOo9ARu+gFOr0x729YUmTQr+j0xJN0defag6W8Z25LUu1Th8KZJ+U7fT+8etrDsmq9gXinWfQMgmePhrug2tlVZbFXLKSHCwHuAL6/0WxYhSBAyZyk3HsnQ4+Dprdx0C5LNlU5IS9JvG5ESWV/uEk7G3ee/xGqyY65UWT4rb+522Tkpm3b2SknGwM6QlMrYidfD8/gsR+T7HmdBoZm4NoX+T8tQs63HvDi4+0OcXiLoCS0elTZ9Wyk/xzuNVuRhxm8W7L+b7+sIChJ3WK8nLNYLOH6a1zj8w9BoeTvZ8PLJclvEkty0pkqTYgoe/Br+asHiEXktWBDyc7Hm2vb6K/QeP1eJyRCxPzthJt+838+eBy7KKfUE58S9s+goaDIIGAwC5aRSFRzl54jF4Hl7qNi7Ln+bklQhzF0kUpFXvwsWdRDz4DW9tiKNl5RIMa+9frOOJ0Zi6Tsq9f7PiE1NwstEFjltUKsGWUzdyHBOQlS/+OY6TvZGXO1fLeif/xvDQx3DiH9j6XdrDHav7Ub+8F9+vOUl8kqzLZpUSY2HhEDAY9QWl7RwA2Bd6je2XrhDcrHxaa0lmXEzbcpqG2Da/fcWNgwv0nQXJiXotWVJCkV3a2cHIkJaBrH+1A1/2rkt8UjKj5+6l0/gNLNh5noSk/AVAAUSchyUjoVQd6PaluUsjiglH/7rEPvgFzdUhdvz8vwIZXCsswOGlereM5s/yxrFKJCSn8EmP4rOyfFburDifeXcvWxs0n6pDdT9uxSWxLx+tKQcvRvLP4asMb1MRX3fH7HduOhJq9YA1H8LZTYC+7MFLnatyOTKOP/dfyUfphdn99QpcOwg9fwKvCgAcu3qLF+fvpW45T17qVDXbw50cpCWleCkZBN0nwMWdsPKtIr+8g52BPo3Ls+qldkwa0BBXRyOvLz5Iuy/X8fPms9xOSCryMlmLTFeLT4yDBYNAS4G+M8He2WzlE8WPV8snCa3Sl4GJi5j+8yRpGbUimcaT0OPw+2jwb8q/ZZ9hxaGrjHmgCoElXc1WTkthzGFMiq0mKa2rlMRoUKw7fj3Px45fdRwvF3uGta6Y885KwWM/gE9lfSXyW5cBaFulJFX83Phl61npJm7BMo0nu2fCvtnQ9lWo0hmAo1duMXzmLlwd7ZgyqHGO3xsXexmTUvzUehxajIb/psL+BWYpgtGg6FanDMtHt2bmsKaU93Hhwz+P0Przdfyw5qSsYp9BlqvFr3gVruyDHlOgRGWzllEUT759fyDcowbDQz9jytKV5i6OyIVM40l8FCwYCPbORDwyjbf+OEHNMh6MbFvJ3MW1CGnrpGSxmKOtzeyVysPJnkYB3qw7lrfVFHefC2fd8VCeblsZDyf73B3k6A79ZkPC7bTeHkophrYK5NClW+w+dzPvT0AUukzjyaXd8PcrULkjWruxHLgYwXNz9tD1u03cik1k6uDGlPZ0yvHcMnC+uOr0PgS0guUvwNVDZiuGUop2VX1Z+HQLFo1qQf3yXny96gStPlvL5/8cIzRKVrEPDSXT1eKjNsyCPbOgzStQvZu5iymKK3snfJ5cgJ2dHR32v8KSHSfMXSKRjczjSTLxC5/TB7j2/oUPNtwk4nYCX/Wph71R/vxDunVSMumZHJdkuy0pAB2q+XHkyi2uRsblan9N0/jq3xOUdHNkSMuAvF3Mr7re2+PCjrTeHj0alMPDyY5ftobkseSisGUWT15/MZTkeUPQ3PyY6/8OD363hccmbGH98es83zGITa91pH55r1ydP21MinT3KmaM9tD7F3D2ggUD4HZ45s11RahxoA8/D23CX2Na076aL5M3nKb152t5t5ivYh8SAs7ed68W3zJoG64b/geVOqRNKS2E2XgH4NB3OtUMF7D760VW7w0zaywRWcssnrzWejyOp3+HTu+xOrYqS/de4rkOQZnPxlRMpeZqmbekJNvc9MPpdaiuz5aw4UTuunytPx7KtjNhjO5QGReHfKxgUbvnnd4e++YRE2lHu/IV+OfgVa5Exub9fKLQZIwnzn4R/Nr9SYi5xuikl3jz3yu4OdnxcY/abH3jAf73YDU8XXLZssadBVLjpCWlGHIvBX1/hchLXJnwFJWqJNzbncgMapX1ZMITDVljWsV+3n/6Kvb/W7ifU9ejzFYucwkMhNibd1Z49ryRxIz2w9HcykDvn/VZM4QwM7tqDxLf9k0eM2xh67wP6Db0htljibhXxnjS4tZJ3m70KXFBPQmvN4qxSw5SvbQ7z3UIMm9BLYwh24HzKTjaaHcvgGql3Cnj6ZSrLl9JySl8uuIogSVceKJZHltR0uv0AQS2Ien3F3m85W7mj/clKVnjzenn8n9OUeAyxpNXEpbQruxmXk94kn3JlfhpcGOWPtuKAc0C8HTOfXKSKjXJzWm8su1++4q78k2IavsVZW6v5dMnR93VncjctaCVTKvYb3i1A4NaBPDXwct0/mYjo37dzcGLxWcV+/QrescvaMlEfqSURwTGJ+boc8wLYSGia7/KspCuvOU8i87dp+PZe7NFxBJxR/p4UmJ5RSZ6fEOUU3Uc+/zAW8sOcSs2kW/61cfBRqfUza/iOnAe9G7Z7av5sfnUDaLjs79ZXLznIieuRfNal+r39xky2nGj4wwuR5ZkYb++VHh0JVqikdVnL3LlmswGainSx5PO2yN4xmMpM5M6E1G1D6tebkvnmqXu6/yp66RId69i7JjrUGYcH8Roz8U8ZtiKg18Uzl5xhISYu2S6sl7OvPdoLba83pHn2gex5fQNHp2wmUHTd7D9TFixmPEjOFgRctLA/nfep7X/Nux6ToDSde7ax9zd9YQIOad4YfP3nNLKMdH+e4L8TuPsfdtiYonQBQcrzh25zaYnh+HlCV6j5rD0cAQrDl3l5QerUqOMh8STDFJbUjJdJyUpxaaTFIB+TcoTHZ/EjC1ns9wnJj6Jr1eeoGEFL7rWLn3f1zx7rSRD10+jhDGSSQ7f4egYj9E1nqXb8z7TmCg8wcGKi1sPMKnDy+xIqc6Wyv9j4oCGaa0g9xNLnOwNKCXdvYq1wEB48Z8v2R5Xiy/sp1A94iqxEU4EBpq7ZHcr4ebIKw9VY6tpFfujV27Rf+p2ek/extpj12w+WfENmYbf+RnQ+iWo0/uubVnO/iVEEQoMhBuhJRh69V0Aptl/hWe9QwQEyOfRoqSkUHLjSFxiTmDoN5MLlOa93w/TNNCHEW0qSTzJhJ0xmxXnE5NtdjHHVPXLe9Gphh9TN57JcvbN9/44TGh0PG89XLNA1tUJDIQdp5vyctgLNDMc472k2STHOLD12oX7PrcoQFHXcPs7mOspbkwv8z4/DGqKo2mM1v3GEqUUzvZGWcyxOPP1hR8nOdF/xhJCY/yY7volsydcx9fXMmvn3dOtYv9R91pcjYxj2IxddP1uE3/st9FV7M9uhH/GQpWHoOM7d23KavYvS3rPRPGQ2vR/YEFvhv07nUrqKpPrvsmyIycBy4wnxdK6cXD8b3joE5IC2vLC/L0AfN23HuFhSuJJJlIXcyyO3b1SvdS5Krfikpi+6cw925buvcii3Rd5vkMQjQK8C+R6qfFk5q9v8u2B0Qx0W8mn5baz9ex1LkfESjyxBIlxpMwfQHJMGK8YX2fcgA5pCUpB3Zs42xtlMcfiLjhYsfNgaSK7zKesdyQ9Yp9g4ZzbFl2b5mRvZFCLQNa/2p6v+tQjMTmFMfP20vHr9cz77zzxSdl/qK3GjVP6go0lgqDXT/cMlM9sth5L6q4nipfgYEXIKSNv/tCF220/pbNxD9rq9xk76ZJFx5NiY/982PQ1NBgEzZ7m+zUn2XM+go971qG8j4vEkywYDFm3pMQmJtvsOinp1SrrSbc6pZm++SyXIu7MsnXs6i3eXnqIpoE+jHmgSoFeMzWetBr7IfGBXRkQP542aj/v/HJB4om5aRr88TyGSzt5OWEUI/s9jp/HnbVPCiqWODsYc1wnJR9zyAlr4+sLvp3qQrmf0BYMxHhiFF69hmHvF4PDdXeGj2pFp05GfH3NXdK72RsN9G7kT88G5Vh55BqT1p/ijSUH+Xb1CUa0qcQTzSrkbxpES3A7HOb10xOT4Png5HnPLqmzazhcd8fBL4qE6+4W2V1PFB++vvoP2kiSbx/j6d0/89qFcnj174q9d6xFxxObdn47/PE8BLSGh8ez7Uw4E9adoncjfx6rVxaQeJKVrFpSNE0zLeZo+y0pAC93rsqao9fpPH4Dz7SrTFhMArO3n8PD2Z5v+9fHrhDW1dHjiRHq/QQ/d+HH6z/wSEhZPHtdwcEvWuKJuWz6Cg4u5OvE3pRq0Z8O1fzu2lxQsSQ3LSlWeocn8qXGI1ys/j69eI/LSYqvk/qmy4BdLTYIGAyKLrVL81CtUmw+dYOJ604x7q+jTFx3imGtKjK4RWCe5uc2u6QEWDgYIs7D4D/Ap2Kmu6U2iQ8f1QpnrzhiI5yYNtlgse+TKEaUwtjtC8LPnmFc2M9cKOHLtpRaVhFPbE74WZg/ADz9od+vXI9NYcz8vQSWdOWDx2ql7SbxJHNZze4Vn6TPNFVckpQgP3dWvtSWT/8+xterTmA0KPo1Kc+Lnarg557zCuL3xdEdguehTejADM9x9HD+kDA8JZ6Yw+GlsHYcq+3astTpCVY+VO2eXQoqlrg45DwmRZKUYsap4wvM+O0kz9eZzXnNjzlXHrWa2jSlFG2q+NKmii+7z4Uzad1pvl51gikbzzCgeQWeal2x8IPp/dI0WD4GQjZBjykQ0CLb3YODFZ06GQkJcSUwEAnUwnIY7UnpOZPTX3Rmsvc39Ep4nyOh1awmntiE2+Ewpw9oyfDEQpKdvHlh2g6i4hL59ammuDre/Sde4sm9suruFZ+oJymONj5wPr2AEq5MHtSIw5cjcXWwI7Cka9Fd3KsCtx+dj++SbvxkN57gpLeIul5C4klROr8DljzNJfd6PBc6jKnD6mbZW6UgYomTjEkRGfn6KZx6fcOac+35xPgzjXYmWGVtWqMAH6YPbcKKF9rQobofP208Q+vP1/H2soNcCLfgVew3fA7750H7N6Fe/1wd4usLTZrIDYWwPCX9vTjReBFxca78Yv8lpRzC+eaHRPmsFoWkeFgwECLOQf+5ULIK36w6wbYzYXzYvTbVS2e+qrzEk7vd6e519+NxprGPzg7FoyUlvVplPYs2QTHxrdeEqc4fUt94iq9D53NrcQurvD+xSmGnYV5/EtzK0CP8WbrWD6Rd1exf+PuNJS65GJMiSUox1P8JB+qN+5VEz2r83n8owe0PmrtI+VajjAc/BDdg7f/a06thORbsvED7r9bz8sJ9nLxmYavY75kF6z+Fek9Au9fMXRohCkT3wQE4DF5IGbtoZvt+yD/hG7KcylQUkJQUWPo0nNsC3SdBQEv+PXyVCetO0bexP30blzd3Ca2GwXQXlJyhJSXOVMPrZFf8khRzevSpQXycNIBHyv/D9V/eJjh3dXnifkSHwuxeaMArDu8Q7+DD24/ULPTLOjtIS4rIQslyHriMWITB2RPm9IabIeYu0n0JLOnKpz3rsvG1DgxtGciKg1fp/M1Gnv51FwcuRtz3+e97SsTj/8DyF6HyA/DY91AAc80LYSm8azXErv8sahrO81L4OEb+spWYHFawLs7uK55oGvz7pt53vPOHULcPp65H87+F+6nn78mH3WsXeHltmZ0pS0nO0JQSl1i8xqRYikq+bhwOGMgCu0dxOTAZtnxn7iJZvPuKJ/HR+j1g1FXWNJzAHxeceaNrdUq6ORZ4OTNytreTlhSRDY+yMHCx3m3g1556Nm3lyng6884jNdkytiNjHqjCttNhPDZhC4Om72Db6fytYn/fC6Bd+A9+G6qvJN93FhitaJC/ELlV9UHUY9/TxnCAJ658xogZ/6XVRos77juebPkWdvwIzZ6BlmOIjE1k5K+7cLQz8OPARnJTnUdp3b0yvA1pLSnFYApiSxPctAJjo/txrcLDsPo92DfX3EWyWPcVT5ISYOEguHqQqMem8eo2exoHeBdZS6yzg0FaUkQO/KrDEwvg1mWY3RPiIs1dogLh4+rAy52rsvWNB3ija3WOXoki+Kft9PxxK6uP5H4V+/tetOjqIb2WwqMMDPgNHN3y/6SEsHQNBsID79LduJWHLn7D07N22c66RgXgvuPJ7hmw+n2o3UtfsDFFY/TcPVwIv82kAQ0p6+VciKW3TandvVJSsujuJUlfketauwwl3Jx5Vz0HldrD76Ph6J/mLpbFua94kpIMS0bA6bXw2Pe8c6QcUXFJfNyjTtpkEoXNxcGO2wnZt7hLkiKgQnPo9ytcPwLzgiExNudjrISbox1Pt6vM5tc78FH3WoRGxTN8lr6K/e/7LpGUcbRkBve1aFH4GT3xs3eFwb+Dm1/Oxwhh7Vq/DC2fZ4hxJQ3P/sgzs/dIomJyX/Hk0BK9y2hQZ3h8MhgMfLD8CJtO3uDjx+vQrFKJQiy57UqbgjjjmJS0KYjlNqmoOdgZeKJpeVaeiOBC52lQtgEsehLObDB30SxKvuOJpsGfL8GRZfDgx/xj34ll+y7zXIcgqpV2L+RS3+Fkb0zrVpkV+fYJXZXO0HMqnNuqz7mfFG/uEhWo1FXs173SnvF965GUovHC/H10/HpDtqvYpy5alHBd/+LmetGiiPMw8zFIToTBy8CrAlAAY1uEsHRKQeePoMEgXrBbStWT03hWEhXgPuLJ8RV6rWeF5nqXUTsHpm8+y6/bz/FolUp0CJSB8vllyGIxx9SWFEcZOG8WTzQLwKAUs/aE6r0QSgTplajnt5u7aBYjX/FE0+CfN2DPTGjzP8LqjuCtpQepVdaD/nWCivT+xCUXM+dJkiLuqN0LHvsBTq+BhUP0/oo2xt5ooGdDf1a+2JbJAxvh5WLPG0sO0vaLdUzbdOaewb6pixZFLm5F9Lz2RC5ulfOUiLcuw8xHIf4WDFoKvvpiSPfdF10Ia6EUPPod1O7FWPv5VDg5k6d/3V3sx6jkK56cWq0v/lq6rt4118GFvw5cYdyfR4g/U4p548pLPLkPqS0p0t3LspT2dKJLrdIs2HmBWDtPGLRM7zY9uzdc3G3u4lmEPMcTTdO7i5rGtGkd3ubNpQeJikuik1s9gqppRXp/4pyL75YkKeJuDQfBw1/DiRWweJhNJipwZxX7359rxeynmlGxpCvj/jpKq8/X8t3qk0TcvvO8g4MVIaeMrFjkSsgpI8HB2fTXTE1QYsJg4BIoWx8ogL7oQlgbg1FfsLTGo7xn/ysVTs3hyV92FvtZv/IUT06v1Vu2favBoCXg5MmOM2G8uGAfCVe8UG4xuPXbIPHkPtwZOJ/5Yo7S3ct8BrcI4FZcEkv2XgT3UjD4D3Dxgdk94NIecxfPIuQ6nmgarPtYn3ij8TDo8imTN57l38PXeKZ1Nd5+ybXI709yswaRfPvEvZoMhy6fw9Hl+qxUNpqogL6KfesqJZk/sgWLn2lJ4wBvvll9glafreXTv49y/VYckMtFiyIvwYyHIeoqDFwE/o3TNt1XX3QhrJXRHnr9DNW68aH9DGqcn82g6TuIvF2811HJVTw5uRrm9gefynotsrM3By9G8tTMXfi5OBO/rRYOftGAxJP7YciqJSVJWlLMrWlFH+r6ezJlwxl9/KhnORiyHJw8Ydbj0qJikmM80TRY8yFs/BIaDIJuX7P+RChf/HuMR+uVpVWJima5P5HuXiL/mo+Cbl/B8b/0VY0T48xdokLXKMCbaUOa8M+LbXigRil+2nSG1l+s462luVjFPuI8zOhGSlQoRxovJdS5+V2b890XXQhrZ+cAfWZCjUd51+5Xml6ZQ98p29IqAEQmjv+DNi+YGNeq3HhkObiW5NT1KIb88h+ezvZMCW5G7DV3iScFxGhQJGVIUlLXb5AkxXyUUozuEMT58Nv8sf+y/qB3AAz9G1y84dfH4fwOs5bR4mkarHoHNo/neoUnCW3xPUevRTNm3l6qlXLn8151qFhRmeX+RLp7ifvTdAQ88g2cXKlPoxuftxXcrXWQePXSHnyftoq9P7/tukj7r9bz0oJ9nMhsFfvQE/BzFxIib9Jh1mLajqx1T5/OfPVFF8JW2DlA71+gVg/GGufQPWIGvX7cwpnQ6Fwdbq2xJF8OLiJl3gD2XK5FvcnzCajjxfczohgwbQdGg2LO8GbUruQs8aQAGQ0qkxXnTd297OQ2yZw61yxF9dLuTFh36s7kBl7l9UTF1VdPVE6vzdM5i008SUmGP1+ErT8wed9wan44jqAWkfScsB1XRzt+GtwYFwc7s92fSHcvcf8aD9P7lZ/bqs9WdTs8V4fZwiBxfRX7Omx8rQNPtgzk38NXefCbjYyYtYt9FyL0nS7vg1+6kJKYQNuZyznZKj7LPp156osuhK0x2kOv6dBgEM+qxTwX9xO9J21m97nsY4otxJJc2/Uz2uLhbLnYlODEsST13I9n3018uWsHCYkas59qRmBJV0DiSUEyKpVpdy87g8LOKLdJ5qSU4vmOVTgTGsOKQ1fubPAsB8P+AZ9KMLcfHPk9V+crNvEkKUGfEXD3DL7Y8TIf+jyGW/8NePXaSnSEkUm9W1DexyVtd3PEE2lJEQWjXj/oNxuuHYbpneFmSLa729og8dKeTrz9SE22vK6vYv/f2XAen7iFzyZMIPnnbmj2zhxq9i+nkoNy7NOZq77oQtgqg1GfQbDFaPprK/jK8D1Df9rEXweuZLq7rcWSLGkarPsE/nyJSN/OBK/7lQRfvbuRQ4kYlF0Kb7dqfs8aBhJPCobRoMi4ZFZcYnKubqJE4etSuzRBfm58s+oECUnp3ig3Pxj6J5Spr89I+t9P2Z6n2MSTuEi998uhxVyo/gFfHP1f2vg1ZYC4dfVIinC557CijicuDnY57iNJisid6t30BQljbsC0znB5b5a72uogcW/TKvZbxnbkl/oneOXGuxxPKMkw46eccHQl9qaj9BEXIidKwYPj4MFxdEzewnznL3hj7ka+W30SLUOXG1uNJXdJTtRX1N7wOdQfSGKvudy84U1iuH4ToSUrIn5vStt6RbfIWnFjUJCSSXcvR0lSLILRoHija3VOh8YwffPZuzc6e+v3JtW6wt+vwKp3ISXzBQKLRTy5dRl+6QbntsDjk7nVdBT2TY+mbU4IdeP2JW+LuDeRlhRRsAJawFMrwc4Jfu6aZfOqTQ8ST0nBbdPHdDj2PiqwNQcfnMepODde+X03tV/aQtyBikTPbyt9xIXIjlLQ8nnoNZ2aKSdY7fERy9ZsYPS8vXdNUWzTsQT07rO/9oB9s6Hd69B9Ar6l7Xnjq5sY3eJJiXYifH5rpnziKbGkEOktKRmnIE6W6YctyAM1SvFgzVJ8v+YkF29mmMjGwQX6/gqNn4It38FvgyEh5p5z2Hw8ubQHpnaAmyEk91/ALzHN6TdzA66VbxC9tRrR89oRuai1xdyb5GZMSs5tLUKk51sNRqzR5+5fOBg6vA1tXyH0hiIkRA8CqYOwho9qhbNXHLERTpl+KUJDuesYixcfBUtHwbE/odFQjN2+op/Rnl4tarD8wGV+XH8aj04HKOXqwrAWlejRzh+QmjghslSnN8rTH9/5A/jH9X1GHBpNz2vRfPpwI5IjXQkMtNFYAhB6HOb1h8iL0GOq3q0WWLLnIjNDDlC5jCuvN2tGo/ecrOc5WalMB84nJcvMXhbmvcdq0enrDXyw/Ag/DW5890ajnb7GW4kgWPkW/NwFgueBp/9d8cFm48mhJbDsWXD1ZV/nhbz+VzLHrx2hbVVfPupeC5eXXS3uOUmSIgqHm58+V/nyMbBuHBd27qXp5xNJdHUg9qb+pQ8OVnTqZCQkxDXTL8W8eRrDR6Xg7B131zEWK/QELBiIFnaSCzU/w7nZKHyNenntjAZ6NPCne71yrD56jYnrT/Pp6kNM33GSEW0q8USzCrg6yldNiExVaI4asRbHef2Zmfw548P78/iPt4nfWY3ogxWYNtlAyCkbiiUAh5fB78+RYnTmWLO/8C3blBIpGt+vPcm3q0/SolIJJg9qhKezvblLWiwYMhk4H5sgLSmWppyXMy92qsKnK47xy5azPNmq4t07KAUtntUTlUXDYEpb1vr8zKP/a3NXfDhz0sDK3fEcuhXC75G3mPxlHAlJKdQo44G66cHCL8vh6KCsI54kJ8Hq92DbBCI8mzLO4w0WLY7A39uZyQMb8VCtUijTgqWWkpykyk13L5WxD3BBaNy4sbZr164CP68wv7tqGEpqRK+ZjOOGtzmf4sfzKaPZd60OkYtbEXLKmOUXIjQUAoOS8ey1BQe/KBKuu+d4jFkdWgx/jCEu2Yke86ezM6ZJtsFL0zS2ng5j4rpTbD0dhpeLPUNbBjKkRSDerg5meAI6pdRuTdMa57ynZZF4Yrvuiice0cT99jxOp5bwb2JjXkseSfhtb8JntyfkuEOmscHqYklSAqz5ALZN4IZTE1r88DM37UsQF21Hu/8d5HDkNXo2LMdnPeviYOFT39pSPGnx6RpaB5Xkyz710h4LnrqdxOQUFj3TsqiLKLKRkqLxzJzdrDxyjamDGtO5Zqm0bXfFE3WKpDkDUGEn+PLWE0x16kLcdQ8STvhTtds5Lt+KxdHOQJ1ynpTxcsbOoNh/PpLTodGg9HwnIdSNyEWtLTee3LoMi0fAuc38dOYJPi/xCIl2ik5lKzNxdJBVtARWfXsFJz/ulmUskepdkWuZ1ViePj2K1UursaDXkyzlXT4vE8z3Xg0JCXHL8kud9eA1V8sKBPHRsOJ12DebxFJNqDPuZ2IeCsHNbz0O190ZPqoVnTrdG7yUUrQKKkmroJLsPX+TSetP8+3qk0zdeIYBzSowvE0lSnk4mec5CWEh7o0nrpw+O53wDfX5rO0HrLAby4vqObb1s2fdvob07ex5zzmsJpYAhJ3Wa3ev7CO2zgiChn+ES4+duPkdxCXJwKGbGi93rMGYhyqm1XyKopHZmJS4pGTcpAXc4hgMim/7NaD/1G2MmbeXaUMa0yqoZCbxpDLnb62mwsnRjK02m7bJe3jJ71mu+UXj6eDDq/2q0rlm6bve4507oduQMFwf2w6Ag280LuXDCQnxtbx4cnwFLHuWlMQ4nj/9Dn+VqwFAUpgz86cG8dlAI06WVuZM5LTqvHwDRa6kn7rPwS8Kh+vuPDWyFWgQl9SaBy5N4NsyX/Ou/a+07HCRSiUmAP6Znit18JrDdfe02k+LG7x2fjssewbCz0KbV9jvNpab9gm4+R0Ecn8z1KCCNz8Nbszxq1H8uP4UP28JYebWc/Rq5M+odpUIKOFaRE9ICMuRfTx5hiPV3PjR73PmOXzMFI/HeH9dJNcMtRjVvjL26datsIpYkpICu3+Gle/qa8X0m8Oh6Eew84xOS64MdinErq1Hyz7+SH5S9LJazLGEq+XXRBdHzg5GfhrSmP5TtzNg2g76Nwzkh+eq4Nlr+13xxOihYVf1NfYE+vGBw0z+cRjLK9s/4rPZ3fD1u/eLFhgIty95YW+KJ1qKwrnTXsLtGgElivx5Zirulr6C/O4ZJPvV4ZmYMawsd+c+wr7EbcutqMlETl2+JEkRuZJZjaWDawLKPgXXhsc5OuchHvZsw/DABXzT5X0cFrTQpxltMAgMd3dbyO3AerOIj9bXK9g+CTzLw5A/oGJbAkIh9qYh3zdD1Uq7823/BrzcuRpTNp7mt10XWbDzPI/WK8sz7StTvbRHoT4tISxJTvFk1cxBNC7ZhS8afcKz9WfTw2Evz64exl8HG/JF77rU9fcCLDyWgF7JsXwMnN0IldpD94ng6Y/nuXjsGh9P2y3hhivRx8tYVnJVjBiVzO5lbfzcnfjz+dZ8vuIYM7eFUPLJCyh709pCflH49NuCXckoNA1mne7N6vOPMbnOu0xv+RKsWwvdvgL30nedM2M8iU9OofpTuxmz+D/mlWhOwwre5niqd5xaA8tfgFuXiG/yHMFnOnPgZiy31tTGqU6I5VbUZCOnwfOSpIhcyazGMiHGAaXAuUQ0ZUdsIPZsCX5ZP5wPZz9IiU2j9T/O++bAw+OhdO27zpfTwPoip2lwdDn88wbcuqhPZdj5A3DUpyosqJuhCiVc+LhHHcY8UIXpm88ye/s5ft93mU41SvFch8o0MHcQFKII5DaePL/me3q93YMyG8ewNOE9lt3qxJCJfejWtBavPlQNLxcHy4slAIlxsPV72PQ1GOzgkW+h0VA04O8DV3j390O4VkkkclsVCClL7E1ny0quihmDQWWyTorM7mXpXBzs+KB7bVpVKMugdy7hWP0iBrsUtGRFSrwd0duq4lDlEg6lb3EhtgSdFvzDld8m4fbfJ3BmA3R4C5oM12cGM8kYTwzOLegxaSsjZu5i6bOtqFDi3kUQC92ty/Dvm3B4KZSowq3gP+m/IoVT16OZMqgR1+v4MXxUOcusqMlBTi0pMnBe5Fpan890XwTgnseCg5XexWH/PL1ZMjYCGg6G9m+Ae6nsL2IOV/bDynfg7AbwqwWPjIcKzTPdtaCnJoy4ncCMrSHM2BpCxO1EWlYuwXMdgmhZuUSB90u3pYGuwvrlKZ7ER8H6z9C2/0iswZXx8d35w74rz3SqyYBmAZYzyFzT9BuJNR/AzRCo1QMe+gQ8yhJyI4b3/jjMhhOh1C7nwVd96lHC6GFx04Lmli3Fk4e+2UhgSRemDLrzdBp8uJKH65Zh3ON1irqIIh/yFE/CTusLP55eC341ofOHENSJrPpang6NpuekrZRwc2DpM63wdCmiWfcSYmDrBH3tl5QkaPsKcU2fY+CM/Ry4FMlPgxvTrqoeOKxy2mSgz+StLHqmVZaxRJIUkSeZfRGy/XLcDof1n8Gu6WB0hOajoPlz4GoB/TtDj8PGL+HgInD20hdTazJc7zdexKLjk5i34zw/bTrD9ah46pX34rn2lelUoxQGQ8EkK7Z0UyFsQ57jybXD8O9bcGYd142l+TLuMXZ5PMiYB2vwaN2y2BnNlKxoGpxcBRs+g0u79cqOh8ZB5Y7cjElg4rpTzNp2Dgc7Ay93rsrgFgHmK2sBsaV40vW7TZTzcmbakDtPp8Y7/zCweQXeerhmURdR5FOe4omm6QtSr34fbp6Fim2h3VgIbJXpuXecCWPAtB10qlGKHwc2LNzJLRJjYc8s2DQeoq9Cjceg8wekeFXkhQX7WL7/MhOfaMjDdcsUXhmKyKDpO5g9vLnM7iUKhq/vvTcOmT2WxsUHun0BzZ6GtR/pX7rtk6HRUGg6AnwqZnFgIdE0uLBDH3Ny5A+wd4ZWL0Drl/RExUzcHO0Y0bYSg1sGsGj3RaZsOMPIX3dTtZQbz7YP4pG6Zaz+pkaIjPIcT0rVgsHL4NQafNd8wJdXpnItbhk/LurCT6u6MKRDHbrXL1d03XSSEvQbnW0/6C2ynhWg+ySo15+IuGRmrj7JtM1niIlPoldDf159qBp+MrOfxTEauKu7l6ZpspijFcpTPFEKaj0O1brBrp/1rpkzukGFltByNFTtAoY773+zSiV4rUs1Pvn7GLO3n2NQi8CCfwIxYbBnJmz/EWKuQ0Ar6PcrlG8KwLerTrB8/2Ve71LdJhIUkNm9RCHJc9NiicrQZwa0O6YHgx2T9UShyoPQYABUeQjsC/GPd0wYHF6i105cPQCOntDmZctp1TFxtDMyoFkA/RqX56+DV5i07jQvLtjH16uOM7JtZfo08pc/nMKm5KubQtADqMod4eQq/DZ+yfsXZ3H79m8s+b0lo1Z0pE6zTvRrWgF/70LqPx56Ag7Mh72zIfoa+FSGxyZAvf6cvZnA7L+PM/+/88QkJNOpRilefaga1Uq7F05ZxH3LOHA+ITkFTUNirRXKczyxc9B7eDQaArtn6mPJ5j+hVzg0HAx1+4J3AADDW1di6+kwPvrrKI0CfKhZtgAmvElJgZBNevf4Q0sgOR4qdYC2M+5q1Vlz9BrfrzlJH9PMoLbCxSH7NES6e4k8K5AVnm9dhl2/6LUG0dfAyVOvuaj6EFTuCM4FMID85jk4uRJO/Atn1ul9Ov1qQdPhUKcvOLple7gl9PFMSdFYc+w6E9edYt+FCHzdHRneuiIDmgfkeQ5/W+qeIWxDga0Wf2k32n9TSTm0FGNyPCFaKVYnN+Ja6fZUb9KJB+qUx8vlPhZSTUnWW0pOroTjf+v/Vwa9H3vTkdzyb8vKI6Es23uJzaduYGdQPFy3zF0z91lCPClIthRPekzagquDHbOHNwMgMjaReh+s5O2HazC8je3cENq6AoknyUlw/C/47yc9eQAo3yzt/iTMpTJdv9+Ml4s9f4xunb9ENuE2nNui35sc/xtuXQIHd6jbB5qOBL8ad+1+Ifw2D3+/ifI+Lix+piVREUabiSVvLDnIZ73qypgUUTAKfIXn5CR9wPrB3/QvbGy4/rhvDb2Js1Qt8K2mTwfsXgYcMtSMahrERULUVb1faegxuHoQzu/QZ+kC8K4I1R+Gev2hdO4GQRbYzVMB0TSNbWf0Vey3nArD09meIS0DebJl7lext6WbCmH9CmW1+LhbcHQ5sXsXYn9hC3ZaIvGaPQe0Slxzr42zf23KV6lHYKWqOHiWvnf8WUoyxNyAqCsQdkqPJ5f2wMWdEH8LUODfmJSaj3OmVBc2XjGy4UQo286EkZCUgr+3M/0al6df0/L4ud9pGba0eFIQbCme9Jm8FTuDgXkj9QlTrt2Ko9knaxj3eG0GNg8wRzFFHhVKPLl5Dg4u1LuGXz2gP+bsww2f+vwa4k1AjYb0fKAteJQFZ597llsgKV6vhI28BDeOw/Wjeiy5sl+vNLV30VtN6vSCql3vvb8B4pOS6f3jNkLCYvjr+TZsWelsU7Hkoz+P8O6jtWRMiigYBb7Cs9EOgh7Qf1KS9S/w2U1wYTsph3/HsGdmhv0d9W5hBntITtAHmKUk3r2Phz+UbwLln9fPWyIoy1k7MpN+oTmjWzycLcFTI+tlurp8UVFK0bJySVpWLsm+CxFMWneK79ecZNr/27v34KjKM47jv/ec3ewmWULAbAihQBCwhchdIl5nFC9jodbRsQqK4wAKta3t2HFaK1N1asfaavuH03agWuut0VbrjNOp2na8UfASh0utKAiyiIrc5BbYzd5O/9gsSSAkQDY5Z3O+nxlnHCHLOw7nmfPb533fZ/nHmtswQjedzxR7FJdemRYfrpCmXKfSKddJLc1yPn5N+99/VcNib2pS8wsqWf+c1DaeREkTVtYOyViWAtmkrExcxske/nXH2EpUjtGuYZcpVj5RK51JWrsnqPde3qcDLbkPOjVaruvPHKnZk4ZqyvDKow7U5utJZNY7soIZ2SlbCxc3uFpP0JFlOg5zTKRy8zbY7lU8eqWeDBopnX977p/9n+fmlHzylqq2vqXvB1+VtfE5aWPr7zV27oxrICQ52dw15Ol4x88LlEq1k6WzvyeNPFeqO7fbbe73v7he7322T0vnTVNptkwLF2c89W7SUwxzREH16oRny85d/TtiRu6bx19kNHTQZ6oLbtKYYVs0JLBb135jn8aNbckFlEA4VxDKo7mhTJUjcl2X8MAeLSNf7FK7I9reOEOBgXGlWhwtXeZoyZ3uf2MxeXillt1whjZsP6Dfv7ZJj66M6fE3t+iqacO06PzRqqtiij28r9enxYciMuNmKzputhobHc1fmNLIqvWaMH6Vhld9qoljPpOtA3IScVnKKqESHVJIO51K7XAqFXNqtNkZqmQ8KG3LfWQkdEijo+W6fHKtpowYpIa6wd3OTYjFJCuU0q7npykwMK70vlKVlacUixXvi0V/Y1tGyXRbOE2kcv/e3QsUvKPX60lFrTR1njR1nhobHX37rn1qmPusRgW3qXzVYC24arfqT4tL6URrYAnntnANqMn9bNXY3DmXI7stXfj3uu3644rNuvHsOl1aX6OmJm+/m5wMhjmioPpiwnPbN49N2vz8NB2as0Ubq6uV3DFaD/y0h+3b41BXJx3aHVbi5dNVM/etwwXvvvvP0aKbvfNicdqQAfrNNZN128Wnaekbm/SXdz/VM01bNXtirW65gCn28La+mha/c6e04OasWjLS/pl71FRdpRU7Rmnpg+cotslWaEBKn++Na9eBpL48lNTQVEbJTFYzLUuhoKVIKKDqAWENqQgpOiB0wlePRiLSwT1B1cxr24byxRPnKNL1kTj0Ids6VieFGxWLRV/Xk0SqTO8nR2tDZKgyk4NaumSxYpsK936wbV9ctz+7VvW1Fbrj61+TVDzvJieCTgoKrrcnPOc7GVYwo8DAeI/btyd6YDUalX7yI0v3/PqI1vGgHraOe8nwwWW694oJuvXCsXpkxWY9+eYWvbD2c100rlq3XDBGU5liD4/qi2nxsZhUEkkq4yQ7PM/BitzzPH16UBU1Qamm+886mcPvzc3SgOqOtSQSTai5mY6nV1jGKJtlu1ex6+t6Ehx8SJJkl6ZUNn6rYrG6grybpDJZ3dq4Wi3prB6aM0WhQO7vYbG9mxyP7q4g5msCnJRoVJo+vXeKQL5tm03ZSu8rVXJH7urOk2nfNjY6qhuT0WVXH1TdmIwaG4/voohFi4xK0uEe/dl9rboirDsuG6eVP56p2y4+Te9u2aMrf7dSc5a9pf98tMvt5QGd6s1aIuXqSbK5RKm9HWtJan/f1JK6Oind3LGWZA56u5b4zVGdlNatX3RSio8b9SSbslR65gZFqhLH/Tld1ZMHXl6vptge3XflBJ0a7dhyLcZ3k66w3QtFp61t26Cy8lRua0Q0oczBE2vftj8AX1J9QCU7Bmjh4nOO65BZNCo9vLT3W8e9YWBZULfOHKsF545S4zu5KfbXP/K228sCXBGNSo8ss3TjAumLJ8+WXd4iOxnWow/3YS3pg20oOHmWMcq0HUlRPJnrpOS/wQbyOqsnJeVp1d7wpu56ebWeuulMBbsZvNxVPVmzc7uWvvGxrp8xQt+cPKzTP79Y3006M3PckC5/nZACT2pr29qKRKTm5hNv3/b0to++aB33pvJQQAvPO1Xzzhqpv636THPvd3tFgDvyz/Lq1ZIU0JQp1BK0sS112O7Vks6FlO6+5YU/dVZPVnw6QT94Zo1++dKHunPW+C5//lj15PW1Gd29fI1OH1ahJV18Rn+qJ93NeyOkwLOi0Z49fIW47aOna/CCUMDWnIYRmuv2QgAXRaPSJZec3M9SS/q3gGUpnW1/uxdnUtC1I+vJFdFhWvXJHv1h+WZN+EqlLp9Ue8yf7ayetCSlX73dpEgooIdvmN7t3z2/1BNCCvottlkAKARqSf9mWUbtGimHryAOBziTguO3ZNZ4fbjtgG57Zo3KgrYuGt/5VqYj60kibmnid97V3paU/rr4LNUMZOZZHk8g+rU5c4xiG229+Gy5Yhvtop7MCsA91JL+yzZShtu90EMlAUsP33iG6msrdMtTq/T6hp3H/L35evL4Y0bTfvi2dqWa9dvrpqq+tmdz3vobQgr6vd6+7QOAP1BL+ifLMkeElPztXoQUnJiKcFCPzW/QmOqI5v+pSQ/+c32HQaHtfbBnp+5ZuVJfxlv0xPwGXfDV6j5erfex3QsAAPiWbYyy7a4gjqcyCtpGtkW3DCeusqxETy+aoXteWKeHXtmof63brqvPGK7zx1ZJkjbtPKin3t6i5R/t0shTyvTY/AaGLx8DIQUAAPiWfVQnJUMXBT1SEQ7qwW9N0iX1Q/TLlz7Uz/6+rsOvDywNasmscZp31kiuuu4CIQUAAPhW7uB8xyuICSkohEvra3RpfY22fnlIKzftUihga1RVucYOiaishFfw7vB/CAAA+JZtjj6TwrR5FNLwwWW6ZvAIt5dRdHgKAQCAb9mWUfrI7V5swQFcR0gBAAC+ZVumw8T5OGdSAE8gpAAAAN+yLaOM07GTUkpIAVxHSAEAAL5lGaNsu1EWiVRWIc6kAK7jKQQAAL5lWzqqk8J2L8B9hBQAAOBbR97u1ZLOElIADyCkAAAA37JaJ8vnD8/nbvfi9QhwG08hAADwLdvkQkp+yxe3ewHeQEgBAAC+le+kZNp1UkpLCCmA2wgpAADAtwLtQorjOLmJ82z3AlzHUwgAAHzLttq2e7Wkc3cRh9juBbgu4PYCAAAA3GKZtoPzLa0DUziTAriPkAIAAHzLbrfdK304pLDRBHAbTyEAAPAtq912r3gyI0kqpZMCuI6QAgAAfMs+vN1LSqRzIYXtXoD7CCkAAMC37NY3oUzrzV4S270AL+ApBAAAvtX+4Hwi1dpJCdBJAdxGSAEAAL6VPzifbhdSuIIYcB8hBQAA+Fb7273yIYWD84D7CCkAAMC38iEly5kUwFN4CgEAgG/lb/dq30nhdi/AfYQUAADgW1Yn270IKYD7CCkAAMC3Ds9JcRwl0mz3AryCpxAAAPhWZwfnuYIYcB8hBQAA+JbV7uB8PJVRScA6/N8AuIeQAgAAfKvt4LzUksoqHODVCPACnkQAAOBbVuubUDqbVSKV4dA84BGEFAAA4FuB1pSSzYqQAngIIQUAAPiW3fomlGkd5sjNXoA38CQCAADfsvJXEGcdJdIZldJJATyBkAIAAHyr/RXE8WRGIUIK4AmEFAAA4Fv5TkqmdZgjZ1IAbyCkAAAA38p3Uj7ZfUjb9sa5ghjwiIDbCwAAAHBLPqT8/B8fqLzE1rUNw11eEQCJkAIAAHzslPIShYOWzhsb1T2X16u2stTtJQEQIQUAAPjYKZGQ/nf3pQrYbPMCvIQnEgAA+BoBBfAenkoAAAAAnkJIAQAAAOAphBQAAAAAnkJIAQAAAOAphBQAAAAAnkJIAQAAAOAphBQAAAAAnkJIAQAAAOAphBQAAAAAnkJIAQAAAOAphBQAAAAAnkJIAQAAAOAphBQAAAAAnkJIAQAAAOAphBQAAAAAnkJIAQAAAOAphBQAAAAAnmIcxyn8hxqzU9KWgn8wgJ4Y6ThO1O1FnCjqCeBJ1BMAhXDMWtIrIQUAAAAAThbbvQAAAAB4CiEFAAAAgKcQUgAAAAB4CiEFAAAAgKcQUnAUY8x0Y8x/jTFhY0y5MeZ9Y8zpbq8LQPGhngAoFOqJv3C7FzpljLlXUlhSqaRPHce5z+UlAShS1BMAhUI98Q9CCjpljCmR1CQpIelsx3EyLi8JQJGingAoFOqJf7DdC8dyiqSIpAHKfWMBACeLegKgUKgnPkEnBZ0yxrwg6WlJoyQNdRznuy4vCUCRop4AKBTqiX8E3F4AvMcYc4OklOM4fzbG2JJWGmMudBznFbfXBqC4UE8AFAr1xF/opAAAAADwFM6kAAAAAPAUQgoAAAAATyGkAAAAAPAUQgoAAAAATyGkAAAAAPAUQgoAAAAATyGkAAAAAPAUQgoAAAAAT/k/79JoIoWXhgcAAAAASUVORK5CYII=\n",
- "text/plain": [
- ""
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
+ "id": "74c7f5dc",
+ "metadata": {
+ "collapsed": false,
+ "editable": true
+ },
+ "outputs": [],
"source": [
"\"\"\"\n",
"============================\n",
@@ -2355,8 +2546,10 @@
},
{
"cell_type": "markdown",
- "id": "2dbb5103",
- "metadata": {},
+ "id": "8f89de90",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Various steps in cross-validation\n",
"\n",
@@ -2378,8 +2571,10 @@
},
{
"cell_type": "markdown",
- "id": "a8dcb6de",
- "metadata": {},
+ "id": "9465a17d",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Cross-validation in brief\n",
"\n",
@@ -2404,8 +2599,10 @@
},
{
"cell_type": "markdown",
- "id": "10bf4343",
- "metadata": {},
+ "id": "d0e4b197",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Code Example for Cross-validation and $k$-fold Cross-validation\n",
"\n",
@@ -2414,23 +2611,13 @@
},
{
"cell_type": "code",
- "execution_count": 8,
- "id": "68e11e09",
- "metadata": {},
- "outputs": [
- {
- "data": {
- "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEGCAYAAACKB4k+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA2pUlEQVR4nO3dd1yVdf/H8deHIYgLRRwJKpobARVnOdJSs8zRUnNbVlpaNtS6292l1S/T6tYsNTXv0ixHaWqppebEAQ4cuBInLpyIwPf3x7n0JsUE5HCdA5/n43EeXNf3Gud9QPnwvcb3EmMMSimlFICH3QGUUkq5Di0KSimlrtKioJRS6iotCkoppa7SoqCUUuoqL7sD3IqSJUuaihUr2h1DKaXcyvr1648bYwIzWubWRaFixYpERUXZHUMppdyKiOy/0TI9fKSUUuoqLQpKKaWu0qKglFLqKrc+p6CUcp7Lly8THx9PUlKS3VFUNvn6+hIUFIS3t3emt9GioJTKUHx8PEWKFKFixYqIiN1xVBYZYzhx4gTx8fGEhIRkejunHT4SkWoisind64yIPCciJUTkVxHZZX0tbq0vIjJGROJEJEZE6jorm1Lq5pKSkggICNCC4KZEhICAgCz39JxWFIwxO4wxEcaYCKAecAGYBQwDFhtjqgCLrXmAe4Eq1qs/MNZZ2ZRSmaMFwb1l5+eXW4ePWgG7jTH7RaQD0MJqnwz8DgwFOgBTjGMs79Ui4i8iZY0xh3M6zPr9J4mf9gNBPlCibihlI8PwLVIop99GKaXcTm4VhS7At9Z06XS/6I8Apa3pcsCBdNvEW205XhSi9p0ifNwY6h3YAkAawqESZdhdK5Ko1z4iLKgYYcU8CLwtwxv+lFIqz3J6URCRAsADwPBrlxljjIhk6Sk/ItIfx+Elypcvn61MTzavzIXoFexeF8OpjVu4vGUbBXbv4rjxZsySXRgDv3/xBGkmhQOhkaS1aEHQg/dzW52a2Xo/pVTetm/fPu6//362bNlid5Rblhs9hXuBDcaYo9b80SuHhUSkLHDMaj8IBKfbLshq+xtjzHhgPEBkZGS2HxvnV7wYlVs3hdZNr7bVA1pfSmHboUSOXHwc7zWrCdm0ioA/f4F/D2XmHZ3ZMfwd2tQqQ92gYnh4eWb37ZVSOSAlJQUvr/x3EaUzP3dufDe78r9DRwBzgV7ACOvrnHTtz4jId0BDINEZ5xNuppCPF/VDAmD02wCYtDT2LY/iyOz57PApy9cr97Fk9jKmf/cKcXfdR6nBT1OpVZPcjqlUrnrrp61sO3QmR/dZ87aivNG+1k3XmzJlCh999BEiQlhYGJ6envj6+rJx40buuOMOevbsyVNPPcWFCxeoXLkyEydOpHjx4owZM4Zx48bh5eVFzZo1+e677/jjjz8YPHgw4DgJu2zZMooUKXLde3bp0oUePXpw3333AdC7d2/uv/9+IiMj6dGjB+fPnwfgs88+o0mTm///37p1K3369CE5OZm0tDR++OEHqlSpct1nmzp1Kvv27aNv374cP36cwMBAJk2aRPny5endu/ffPvfAgQMZOHAgCQkJ+Pn58eWXX1K9evWs/AgyJM58RrOIFAL+AioZYxKttgBgBlAe2A88Yow5KY7T5J8BbXFcqdTHGPOPo91FRkaa3B4Q72zSZdbN/xP/ke8Suv4PCqSmsCu4Gqd69iVs2DP4FvbL1TxKOUtsbCw1atQA7CsKW7dupVOnTqxcuZKSJUty8uRJhgwZwvHjx5kzZw6enp6EhYXx6aef0rx5c15//XXOnDnDJ598wm233cbevXvx8fHh9OnT+Pv70759e4YNG8Ydd9zBuXPn8PX1zfAv7lmzZjF79mwmT55McnIylStXZufOnRhj8PDwwNfXl127dtG1a1eioqJuevjo2WefpVGjRjz22GMkJyeTmprKnj17rvtsJUqUoH379jz00EP06tWLiRMnMnfuXGbPnk3v3r3/9rlbtWrFuHHjqFKlCmvWrGH48OEsWbLkuvdO/3O8QkTWG2MiMwxrjHHbV7169YydTh84bFY//6bZfVtlc9GrgGk5bKb5bMkuc/pCsq25lMoJ27ZtszuCGTNmjHnllVf+1tarVy/z9ddfG2OMOX36tAkODr66LC4uztSpU8cYY0ybNm3Mgw8+aKZOnWrOnj1rjDHm/fffNw0aNDCjR482Bw4cuOH7Xrx40QQHB5ukpCQze/Zs061bt6vv1717dxMaGmrCw8NNwYIFjTHG7N2719SqVeuG+5s2bZqpWbOmGTFihNm5c+cNP5sxxgQEBJjkZMfvkOTkZBMQEHDd5z579qzx9fU14eHhV1/Vq1fP8L0z+jkCUeYGv1d17KNbUCyoDA0/foOQAzvZOn8Z5aoE8+GC7WyMaMbqp4eRdPa83RGVypMKFbr5JeTz5s1j4MCBbNiwgfr165OSksKwYcP46quvuHjxInfccQfbt2/PcFtfX19atGjBwoULmT59Oo8++igAo0aNonTp0kRHRxMVFUVycnKm8nbr1o25c+dSsGBB2rVrl+Ff9Jlx5XOnpaXh7+/Ppk2brr5iY2Oztc9raVHIAeLhQb17GjKlbwN+6RtOcR+h0biRnA4OIer9zzFpaXZHVMottWzZku+//54TJ04AcPLkyb8tL1asGMWLF2f58uUATJ06lebNm5OWlsaBAwe46667GDlyJImJiZw7d47du3dTu3Zthg4dSv369W9YFAAeffRRJk2axPLly2nbti0AiYmJlC1bFg8PD6ZOnUpqamqmPseePXuoVKkSgwYNokOHDsTExNzwszVp0oTvvvsOgGnTptG0adPr9le0aFFCQkL4/vvvAccRn+jo6ExluRktCjmsRrVgwmNWsmXqLM4WLUHkK8+wOfxODu3Ya3c0pdxOrVq1ePXVV2nevDnh4eEMGTLkunUmT57MSy+9RFhYGJs2beL1118nNTWV7t27U7t2berUqcOgQYPw9/fnk08+ITQ0lLCwMLy9vbn33ntv+N6tW7fmjz/+4O6776ZAgQIADBgwgMmTJxMeHs727dsz1WMBmDFjBqGhoURERLBlyxZ69ux5w8/26aefMmnSpKsnnkePHp3hPqdNm8aECRMIDw+nVq1azJkzJ8P1ssqpJ5qdzY4TzVmRejmFdS+9Q/Fvp9CtzygGd4igR6MKOnSAcgsZnaBU7ierJ5q1p+BEnt5eNPrkLQptjSG06m38e+YGfm3Xg8SDx26+sVJK2SD/3fVhg6CSRfi6T31++WQHd33yHcdDf+P4jB+ofM8ddkdTKl/bvHkzPXr0+Fubj48Pa9asydb+Fi5cyNChQ//WFhISwqxZs7KdMbfp4aNctvOnxfh370Lhi+fYOWo8EQN73HwjpWygh4/yBj185OKqtm+FrF3DwbIVCXumFyte+cDuSEopdZUWBRsEVqtEUPRaVjTvwL+O+/P50ji7IymlFKBFwTYF/YvQZPEPhLesz4cLtvPjW2P1fgallO20KNjIy9ODjx+J4N2U7XR+cwCrnxp6842UykcKFy58dXr+/PlUrVqV/fv38+abb1KuXDkiIiKIiIhg2LBhN9zHvn37CA0NzXBZixYtcLfzks6mVx/ZzNND6DbyeaJWL6bxlx+xtmRJGrynxUGp9BYvXsygQYNYuHAhFSpUAOD555/nxRdftDlZ3qM9BRfg4eVJ+OLZRNduTL0Rr7BxzCS7IynlMpYtW8YTTzzBzz//TOXKlW+4njGGl156idDQUGrXrs306dOvW+fixYt06dKFGjVq0KlTJy5evOjM6G5JewouwtvXh6rLFrI7oiGVX36WHW3uplq14JtvqFRuadHi+rZHHoEBA+DCBWjX7vrlvXs7XsePw0MP/X3Z77/f9C0vXbpEx44d+f333697VsCoUaP45ptvABg5ciTnzp1j06ZNREdHc/z4cerXr0+zZs3+ts3YsWPx8/MjNjaWmJgY6tate9MM+Y32FFxIQf8iFF80j6Hd36LvrF2cOHfJ7khK2crb25smTZowYcKE65Y9//zzV0cIbdOmDStWrKBr1654enpSunRpmjdvzrp16/62zbJly+jevTsAYWFhhIWF5crncCfaU3AxgVVDeOqNfjz8xSpGv/01b/y7H57e+mNSLuCf/rL38/vn5SVLZqpncC0PDw9mzJhBq1ateO+993jllVeyvA+VNdpTcEHhwf6MrQFvf/gUax+/flRIpfITPz8/5s2bd3VU0Btp2rQp06dPJzU1lYSEBJYtW0aDBg3+tk6zZs3473//C8CWLVuIiYlxanZ3pH+CuqhWj91L1Lj7aDD1c7bdew81u7S3O5JStilRogQLFiygWbNmBAYGZrhOp06dWLVqFeHh4YgIH3zwAWXKlGHfvn1X13n66afp06cPNWrUoEaNGtSrVy+XPoH70LGPXNi546c4Xb023inJ+G6JoVhQGbsjqXxExz7KG3TsozykcMniXJwyjRJnT7Kr6+N2x1FK5QNOLQoi4i8iM0Vku4jEikhjESkhIr+KyC7ra3FrXRGRMSISJyIxIqLXigFV2jVnde/nmF2wPEtjj9odRymVxzm7pzAaWGCMqQ6EA7HAMGCxMaYKsNiaB7gXqGK9+gNjnZzNbTT4YiRr732UV2Zv4UzSZbvjqHzEnQ8vq+z9/JxWFESkGNAMmABgjEk2xpwGOgCTrdUmAx2t6Q7AFOOwGvAXkbLOyudOfLw8+eChcBqvnM/Gbk/ZHUflE76+vpw4cUILg5syxnDixAl8fX2ztJ0zrz4KARKASSISDqwHBgOljTGHrXWOAKWt6XLAgXTbx1tth9O1ISL9cfQkKF++vNPCu5qIYH+SPI/TYNZUds3rTpX77rI7ksrjgoKCiI+PJyEhwe4oKpt8fX0JCgrK0jbOLApeQF3gWWPMGhEZzf8OFQFgjDEikqU/Q4wx44Hx4Lj6KKfCuoNa4z/m5OKfSHv2WdLaROPh5Wl3JJWHeXt7ExISYncMlcuceU4hHog3xlx52OlMHEXi6JXDQtbXK0+xPwikH+wnyGpTliKlAtgz5F9U27uV9f/+1O44Sqk8yGlFwRhzBDggItWsplbANmAu0Mtq6wXMsabnAj2tq5AaAYnpDjMpS+Rrg9kRUouQj97m7PFTdsdRSuUxzr6j+VlgmogUAPYAfXAUohki0g/YDzxirTsfaAfEAResddU1PLw84T9jeXHCYsI2HGNI6+J2R1JK5SFOLQrGmE1ARnfNtcpgXQMMdGaevKJa26YUOlGIr1bspWeTipQs7GN3JKVUHqF3NLupF1pX5eklk9nea4DdUZRSeYgWBTdVKbAwDfxSaDB7Coc2brM7jlIqj9Ci4MZCRo8gzcODg8/pM52VUjlDi4IbK1WjMpvu70qd5fM5uH6r3XGUUnmAFgU3d/uI10n18OTgsNftjqKUygO0KLi5ktUqMX/Aa7xT4S4Onr5odxyllJvTopAHNHj7RWJLVWL8H7vtjqKUcnNaFPKAcv4F6RfsQcRrz3F89192x1FKuTEtCnlE97pl6bB5Cbve/MDuKEopN6ZFIY8IbhhOdJ2mVP/xG5LOnLM7jlLKTWlRyEMKvPQCxS8kEv3+53ZHUUq5KS0KeUjNR+9nV3A1ykz8D2kpqXbHUUq5IS0KeYh4eHBq0BB+LV+HZZv1hLNSKuucPXS2ymV1nn+cQcmVqboxgRZ19KlZSqms0Z5CHuPt6UGX+kGkLPqV+OjtdsdRSrkZLQp5ULfyBZgy43Xi3/nQ7ihKKTejRSEPKlUthJh6zan2y0ySzl2wO45Syo1oUcijvAcOoPiFM2wePcHuKEopN6JFIY+q1b0TB0oGUXjSV3ZHUUq5EacWBRHZJyKbRWSTiERZbSVE5FcR2WV9LW61i4iMEZE4EYkRkbrOzJbXeXh5cvCRHpQ4Gs/OrfvsjqOUchO50VO4yxgTYYyJtOaHAYuNMVWAxdY8wL1AFevVHxibC9nytGpvDeWuZ75myq6zdkdRSrkJOw4fdQAmW9OTgY7p2qcYh9WAv4iUtSFfnlG8ZDHuDgvi5w0HSLp4ye44Sik34OyiYIBFIrJeRPpbbaWNMYet6SNAaWu6HHAg3bbxVtvfiEh/EYkSkaiEhARn5c4zupdOZf4nvdj66US7oyil3ICzi8Kdxpi6OA4NDRSRZukXGmMMjsKRacaY8caYSGNMZGBgYA5GzZvqNauDh6cH3lOn2B1FKeUGnFoUjDEHra/HgFlAA+DolcNC1tdj1uoHgeB0mwdZbeoWeHp5srfdQ4RuWcPRbbvsjqOUcnFOKwoiUkhEilyZBloDW4C5QC9rtV7AHGt6LtDTugqpEZCY7jCTugXln38KDwx7Rn1hdxSllItzZk+hNLBCRKKBtcA8Y8wCYARwj4jsAu625gHmA3uAOOBLYIATs+Ur5SJrs61KBEGzp2PS0uyOo5RyYU4bJdUYswcIz6D9BNAqg3YDDHRWnvzu6AvDGbl4O8/uO0lkpZJ2x1FKuSi9ozmfaNC7M+uqN2DmxkN2R1FKuTAtCvlEIR8vugWmEjJ6BJfO6yB5SqmMaVHIR9oXPMeTy79l24TpdkdRSrkoLQr5SK0enThZqBhm2n/tjqKUclFaFPIRL58C7GrejpoblnH22Am74yilXJAWhXymWL9e+KYks32s3uGslLqeFoV8plrHe9hTuiJ7YuLsjqKUckFaFPIZ8fDgx0nzGV61HQlndeRUpdTfaVHIhzrUDSLNwMLVOhaSUurvnHZHs3JdVUoXYcLiMQRNj4e9W+2Oo5RyIdpTyKf8IutQbd82DkZttjuKUsqFaFHIp0IG9AXgry8m32RNpVR+okUhnypTuyo7QmoRuGCu3VGUUi5Ei0I+duq+jtwev4v4tTF2R1FKuQg90ZyPVRjQh5d2n6NKQhr9b766Uiof0J5CPla2RmXi7n+YObvP2h1FKeUitCjkc50qFiLyp2nEb9hmdxSllAvQopDPtQ72463fvuDA2Il2R1FKuQAtCvlcmdpV2V5Rr0JSSjk4vSiIiKeIbBSRn635EBFZIyJxIjJdRApY7T7WfJy1vKKzsymH0/d14Pb4XRxcp1chKZXf5UZPYTAQm25+JDDKGHM7cAroZ7X3A05Z7aOs9VQuqPhkLwAOjNfhtJXK75xaFEQkCLgP+MqaF6AlMNNaZTLQ0ZruYM1jLW9lra+crEztqsRWqs2pbTvtjqKUspmzewqfAC8DadZ8AHDaGJNizccD5azpcsABAGt5orW+ygUrx8/g6aZPsv/EebujKKVs5LSiICL3A8eMMetzeL/9RSRKRKISEhJyctf5Wpu65QGYvyne5iRKKTs5s6dwB/CAiOwDvsNx2Gg04C8iV+6kDgIOWtMHgWAAa3kx4LoHCRtjxhtjIo0xkYGBgU6Mn78EFffjk3XfcNfjD9odRSllI6cVBWPMcGNMkDGmItAFWGKMeQxYCjxkrdYLmGNNz7XmsZYvMcYYZ+VT1ytTrSLV923l0Ea9kU2p/MqO+xSGAkNEJA7HOYMJVvsEIMBqHwIMsyFbvlb+iR4A/KVXISmVb4k7/zEeGRlpoqKi7I6Rp+wqXx3j4UHVfdpbUCqvEpH1xpjIjJZluqcgIneKSB9rOlBEQnIqoHIdx9u2p+r+WA7HbLc7ilLKBpkqCiLyBo7DPsOtJm/gG2eFUvYJfrI3HzTryeL9OnKqUvlRZnsKnYAHgPMAxphDQBFnhVL2CapXi987P86s+Mt2R1FK2SCzRSHZuhLIAIhIIedFUnZ7oEoxyiyYy7Gd++yOopTKZZktCjNE5Asc9xg8AfwGfOm8WMpO7Yqn8Pnckez9zyS7oyilclmmioIx5iMc4xH9AFQDXjfGfOrMYMo+5RvXZW+ZEIrMm3PzlZVSeUpmTzQXwnEz2Us4eggFRcTbqcmUrY60vp/qcTEc37nX7ihKqVyU2cNHywAfESkHLAB6AF87K5SyX9m+3fHAsHu8XmSmVH6S2aIgxpgLQGdgrDHmYaCW82Ipu1VoGsn+UuVJXrHS7ihKqVyU6aIgIo2Bx4B5VpuncyIpVyAeHsz7bDq9Wgzk+LlLdsdRSuWSzBaFwTjGIvrRGLPVupt5ifNiKVdw1501SUNYtPWo3VGUUrkks0XhAo4H5XQVkRgcI5re5bRUyiVUL1OEt9dPp9Izfe2OopTKJV43XwWAacCLwBb+9xQ1lceJCFUCChK5eCmn9x/Cv8JtdkdSSjlZZnsKCcaYn4wxe40x+6+8nJpMuYTA3t3wMmns+nKa3VGUUrkgs0XhDRH5SkS6ikjnKy+nJlMuoXLrphwqURafObPsjqKUygWZPXzUB6iOY3TUK4ePDPCjM0Ip1yEeHuy/qx2Rs74m8eAxipUrZXckpZQTZbYo1DfGVHNqEuWySjzei2/2HaPklnjaa1FQKk/LbFFYKSI1jTH6OK58qGqbO+m76TLVD6fS3u4wSimnyuw5hUbAJhHZISIxIrLZujRV5QMiQrsagVxcvJQzx07YHUcp5USZ7Sm0dWoK5fIeTDvMq98MJap6QSJffdbuOEopJ8ns0Nn7M3r90zYi4isia0UkWkS2ishbVnuIiKwRkTgRmS4iBax2H2s+zlpe8ZY/ncoxVdu34mixQDx/1GsLlMrLMnv4KDsuAS2NMeFABNBWRBoBI4FRxpjbgVNAP2v9fsApq32UtZ5yER5enuxt1oaa0X9y7vgpu+MopZzEaUXBOJyzZr2tlwFa4nhgD8BkoKM13cGax1reSkTEWflU1hXt3hWf1MvsmPCd3VGUUk7izJ4CIuIpIpuAY8CvwG7gtDEmxVolHihnTZcDDgBYyxOBgAz22V9EokQkKiEhwZnx1TWqd25DQpESpPz0k91RlFJO4tSiYIxJNcZEAEFAAxw3wN3qPscbYyKNMZGBgYG3ujuVBR5ennz7/iT6NXuaC8kpN99AKeV2nFoUrjDGnAaWAo0BfxG5ctVTEHDQmj4IBANYy4sBev2ji6nf9g7OpXnw+w7tpSmVFzmtKIhIoIj4W9MFgXuAWBzF4SFrtV7AlafDz7XmsZYvMcYYZ+VT2dMgpATPb5hNgeFD7Y6ilHICZ/YUygJLrZvc1gG/GmN+BoYCQ0QkDsc5gwnW+hOAAKt9CI6H+igX4+khNJJEmiz6nqQz526+gVLKrWT25rUsM8bEAHUyaN+D4/zCte1JwMPOyqNyjl/XR/Fb9D0bv/6eOoP62B1HKZWDcuWcgspbanRtzym/oqTO+N7uKEqpHKZFQWWZl08BdjW+m+rr/iDp7Hm74yilcpAWBZUtvj0e4/dK9Vi5frfdUZRSOchp5xRU3laze0f6HihC/fjLtLQ7jFIqx2hPQWWLl6cH94fdxp4/N5B47KTdcZRSOUSLgsq2LgVPs3Bcf3aM+cruKEqpHKJFQWVbtVaNiS9ZjkIzZ9gdRSmVQ7QoqGwTDw8OtOtMjR0bOLptl91xlFI5QIuCuiXBAx/HA8PeTyfcfGWllMvToqBuSVCDMHZWqEHx+XNuvrJSyuVpUVC3bOu7n/BwpzfZceSs3VGUUrdIi4K6ZU07Nue8XxFmbzp485WVUi5Ni4K6ZSUL+/DM+e00GtKP1Mv68B2l3JkWBZUjmgX50Tx2Jdum/mh3FKXULdCioHJE6ICenC5YhOQv9SokpdyZFgWVI3wK+bGjZXtC1y0lMf6I3XGUUtmkRUHlmJKDnsIn9TKxo8bbHUUplU1aFFSOqdy6KYsatmPRGW+7oyilskmLgspRhz/+nIkBYWw9lGh3FKVUNjitKIhIsIgsFZFtIrJVRAZb7SVE5FcR2WV9LW61i4iMEZE4EYkRkbrOyqacp0PEbZS6dI61k2fbHUUplQ3O7CmkAC8YY2oCjYCBIlITGAYsNsZUARZb8wD3AlWsV39grBOzKSfx9yvA52u/pvPbz5B05pzdcZRSWeS0omCMOWyM2WBNnwVigXJAB2CytdpkoKM13QGYYhxWA/4iUtZZ+ZTz+A14imJJ54j5Pz3hrJS7yZVzCiJSEagDrAFKG2MOW4uOAKWt6XLAgXSbxVtt1+6rv4hEiUhUQkKC80KrbKvZtT37S1fA/2t9+I5S7sbpRUFECgM/AM8ZY86kX2aMMYDJyv6MMeONMZHGmMjAwMAcTKpyinh4cKRbH6r+FcvOn5fYHUcplQVOLQoi4o2jIEwzxlwZ/+DolcNC1tdjVvtBIDjd5kFWm3JDNYYO5HyBguyc/pPdUZRSWeDMq48EmADEGmM+TrdoLtDLmu4FzEnX3tO6CqkRkJjuMJNyM0VLl2TU+AW8UKE1py8k2x1HKZVJzuwp3AH0AFqKyCbr1Q4YAdwjIruAu615gPnAHiAO+BIY4MRsKhd0vieCSylp/Lh6j91RlFKZ5OWsHRtjVgByg8WtMljfAAOdlUflvpq3FeWDzT/ScNJvpB3cg4eXp92RlFI3oXc0K6eq3KI+FY79RfTYqXZHUUplghYF5VThg/pyqHgZfEePsjuKUioTtCgop/LyKcBfPftTY3cM22cvsjuOUuomtCgop6v92vOc8S3M+REf2R1FKXUTTjvRrNQVhQL8mfHqx4w6UoDvTpynQkAhuyPla8did3Ng3mKS129g4SMDSBUP6h7eQSWvZKp1eQDfIvrzyc+0p6ByRfNBPTheLJCJK/baHSVfSktJZf0HY9laPZJSNW+n3ktPUn/6l8xfv5+50Ye4PHYc4f27kVKyFGtbdiJ+bYzdkZVNtCioXFG6qC9PFztLyxf6cGLvgZtvoHLMn3HH+bDPm9QbOoBixw+zqucgdv60mNSzZ1n7bnui32hNm0XfEvPlt2y7sw21l/9CmUZ1WP5gPy4kp9gdX+Uycdwe4J4iIyNNVFSU3TFUJu3/cz1BTRuwrnNvGs2cYHecPO/c8VNMnvgLH54sRuVi3owosJ96g/ve9H6R4zv3snvwcOaYkqxs9SBjutYhLMg/d0KrXCEi640xkRku06KgclPUne2ouXYJF3fsIiAk+OYbqGzZ/+d66NwZ3wvnmPzNYga1q42vd9ZuHly95wQvzIgmct1v9G4SQp0XnnBSWpXb/qko6OEjlasCR76DT8pldr30pt1R8qzNX39PQMtmFDmfSMLYCbzcISLLBQGgUaUA5gxsQr8ti6jzYn/WvPyuE9IqV6NFQeWqCnfUY2OTNoTNncZJPbeQ49Z/MJZq/bpxrGRZklevI7R7x1vaX8kivlSN+oNNEU1p+OFrrBn2Xs4EVS5Li4LKdYEfvMuoOx9jwiZ9SFJOmrZ6H4cnfUtcpVoERK2iTGiVHNmvb5FC1FixgOjwO2g48lXWf6BPys3LtCioXFehSV0S+j/DxA3HOHYmye44ecL3q/by6uytzB3yPpXWr6BY2Zx9AJVPIT+q/7mI2MphrJ+9hPX7T+Xo/pXr0KKgbPHc3VVpveV3Nj/+nN1R3F7U+58T2qEl95USPu3TCN+ihZ3yPj6F/Ci9ZjnTOg2g/5QoDpy84JT3UfbSoqBsUbFkIbqmHqTFzC/Zv2yd3XHc1sZPvybi1UEYf38+6tsUHycPT14ioCgTe9en6oEdHGjRlqQz55z6fir3aVFQtqn22QecL1CQU4OG2B3FLW2fvYgaQ55kd4XqlF+5hIL+RXLlfSsHFmZYvRI02byc6If65Mp7qtyjRUHZpnj5smzrPZCI6BVs+Wa23XHcysH1WynV/RGOFytJ4NJFFC5ZPFffP/zJbqx6+HEa/jqTqPc/y9X3Vs6lRUHZKuLD1zlUvAyFX3yey0mX7I7jFhIvXGbI7G3EBVYk7eefKVGxnC056k/9nNjKYdR84yUORm22JYPKeVoUlK18ixTi8MhPGNmoC1+vibc7jstLvpDEU1PWsDGtMGbxYso3qmNbFi+fAhSfPZMUD0+iX3qb1DT3HR1B/Y/TioKITBSRYyKyJV1bCRH5VUR2WV+LW+0iImNEJE5EYkSkrrNyKddT9/FHuNSxM58s3sWRRL1E9UZMWhrRbR+i16iX+aBzKA0rBdgdiTKhVVg7dTbP1O/Bl8v32B1H5QBn9hS+Btpe0zYMWGyMqQIstuYB7gWqWK/+gN4dk4+ICG+2r0WPlT+w94FH7I7jslb3eY76y+dRrEkDOtZznXGjWj3UijZh5fh61lp2r9xodxx1i5xWFIwxy4CT1zR3ACZb05OBjunapxiH1YC/iJR1VjblesoH+NE8uBCNl/3Exk+/tjuOy1n3zmgaT/mUdc3b02jCx3bH+RsR4d0OtZj63avw8CMkX9DenjvL7XMKpY0xh63pI0Bpa7ockH4gnHirTeUj9T4fwe7bbif4lSEkxh+xO47L2DptDuFvvsjWavUInz8D8XC9U4EBRXw5/9pbVD4Ux4YnXrA7jroFtv3rMo4xu7N8ZkpE+otIlIhEJSTo2Dl5SQE/X5g0kWIXzrCr6+N2x3EJccfOMmLpXmIr1iRo8XzH98hFRQzswbpm9xP57TjiFvxhdxyVTbldFI5eOSxkfT1mtR8E0h8kDbLarmOMGW+MiTTGRAYG5uz4Lsp+lVs3Jarb09T58xeW/7Tc7ji2OnY8kd6T1hFbsRYl1q6kWLlSdke6qar/ncCpwv549u2rh5HcVG4XhblAL2u6FzAnXXtP6yqkRkBiusNMKp+p98WHDB0yloHrLxB/Kn+Or3PhVCKJ9RrReeFUJvauT3BAIbsjZUqxcqU4NPITYouUYfwCvXfBHTnzktRvgVVANRGJF5F+wAjgHhHZBdxtzQPMB/YAccCXwABn5VKur4CfL4OGd8cY+M+7k/PdTW0pl5LZedf9VDqwk5YPtnS7R2GGP92Dxe/8h1FRCWyOT7Q7jsoiL2ft2BjT9QaLWmWwrgEGOiuLcj/lA/z4tLYXzR4eyJr962k88yu7I+UKk5bG+o49aBi9gjUvvE3DgT3sjpQtr7evyf7VGzn0YFeqLpmJTyE/uyOpTHK9yxiUsrR4qBXr2jxE4x8mEPX+53bHyRWr+71AwwUzWPVQPxp+9JrdcbKtWEFvXqtWgDZrf2HD43o1kjvRoqBcWp0fp7Dt9nBCXx/Czp+X2B3HqSas2MvMY8KaFh1o+O0Xdse5ZeFPdmNdiweoP308u+YttTuOyiRxHLlxT5GRkSYqKsruGMrJTu6NJ6lefbxSLpO6eQtlK5SxO1KOm71wA88tPUzbWmX4rFsdvDzzxt9riYcTuFStBhcKFaFs3FY9jGQxaWkcO5NEfGIS57fE4vvnCkzyJczlFMTHB8+AElxu2YrbKpXjNv+CeOfwvwcRWW+MicxomdPOKSiVU0qEBLH3hx+ZPHIKv83czvdPlaBEoQJ2x8ox694ZTet3hvL4C5/yUte2eaYgABQrG0j0R2MIf/Ix/hj8Os2/+sjuSLZIPHiMPT/M49JvSyi2LYZyB/fw9ENvsKFcDTpsXcron//vum3u7TOG2FKVeHTzr/TfNI/joXXwaNKY4M73UaZ2Vadl1Z6Cchtr9pyg58S1tEk9yr+fb0+RUvYPCHer1rz8bxp++C+2VK9H5VVLc+1BOblt2uARvFegKt8Odr+rqbLrwPFzzN96lL1zFvLeqGfwwJDkVYC9Fapz5vZqHOn+OEXrR1CaZAJSL+JdsCCe3l5cvnCR88dOcCSgLPvPpSELfqH6dxOouHsLRS45LtH+PaIll76ZRpta2es1/1NPQYuCciu/r4sjvFldjpQLodyfiylauqTdkbJt9cBXaPSf94kOv4NqyxfiW8Q97kXIjsSLl2kzahmBninMfKZpnj2MdP7EabZ+/AX+0yaz8LYw/q9ZD+qW8ePFTbPxb3s3ldu3yvZnT0tJZf+KKI7OnMumCx7UGDaI5lWzdwPvPxUFjDFu+6pXr55R+c/6j8abSx5eZldQVXNy30G742RZamqa+fbdr4wBs77B3ebS+Yt2R8oVf0TFmd3FbzMrH+1vd5Qct2fJSrO69cPmXIGCxoDZUybELBn2gfnrxHm7o2UIiDI3+L2adw5eqnyj7gtPEDtuCkFH9nGmYRMObdxmd6RMu5CcwtPT1jP8TCl+GPAmYcvmufR4RjmpWb3KnIioT4MZX+WJK8mMMazec4I+k9ay88khhC2Zy7Ym97D9hwVUPBjHXe+/RHAJ9+sRaVFQbin8ia7smToT/8QTLHtqOKt2n7A70k3Fr40hNqwJsSujea19KJ0/ex0vn7xzwjwzqv33S44XDcCnXx/Onzhtd5xsMWlpbPpsCjsqhzF8xA/ExCdy+M33Sd67j/pL51C9cxuXHMk2s9w3ucr3anZpz5nlq5jceSA9Jqzhv7NWkZaSanesDEW9/zn+TRtz+/5YRjcqTt87QxARu2PluqJlAkn49AuCEuKJfaArJi3N7khZsuWb2eysXJuIZ3tRNPEEwyOK8uewlvTu1oJiQXnjUmktCsqtBUeGMmPwXbSpVJSGjz/MttoNObp1l92xrjq++y/WN25D5CvPEB9UmYtr11OnV2e7Y9kqtEcn1vYZDPv38+0S9zj0t/nAaTaG30loj074n0pg7asjKXVoH60HdMHX29PueDlKi4Jye0V8vfmsbxNOPPUslXZvxa9eBKuf/ZetA+mlpKbx7dq/WPDYYGqvW8Kq3oOpvHWdU68vdycNv/iQL96cwBtL/yJq37UPaHQd+6J3MHDaBtp//idri1dkzYDhFPtrDw3efTnPHvrToqDyBPHwoMG/h3JyxRr2Vgmn0Wf/5lDFaiz5PZq0tNy77DotJZWNYybx/PPjGP7jZn575GkO/7GaxpM+wdvXJ9dyuDoPL08+7FaPaj6pnLqvIwdWu9aznY9u28Xaux8kqE5Nzi38jUGtqtBtwSQafv4evkUL2x3PqfQ+BZXnmLQ0osdO48y4L+nV7iVuL12Ul0ud5877mzrt5rALpxLZPPJzykz6ggrH/mJxnZakTJ1G65ql8+W5g8yKX7eZQi2acsHXD9+oNQSEBN98Iyc6/ddhtj/3CnXmTkMwbGz3KLePft/2XDlN71NQ+dLllFQze2O86fjefJPoU8icLeBn1tz9oIn+8ltz8cy5W97/uaTLZtHWI+aXzk+Y896+xoDZGVzNrHt3jLmcdCkHPkH+EDtrobnoVcDsDK5mTscftSXD2aTLZvSi7WZ/8bImRTzM2ubtzcGN22zJkhv4h/sUtKeg8jyTlkbsjHmcH/cltVYuwu/yJS54+zC55zDOPdyFCH9PKp9LoFR4DQqXLJ7h9qcPHOHE9jhOr9tEysZNFNm8iW7tXyHRuyCPb1nAPclHKPpEH6o/2NatL0e0S/S4b6jxTB/+KluJwFV/5NqVPElnz7Pu3TE871eH4xdTefHSDu7veCcVmzfIlfe3iw6Ip/I18fCgZpf20KU9SWfOETPjZy7OmktckdLM/mMPTXavZ+qM1wFI8irAeR8/kr19+Ncjw1lfrgZtohYy8qePuVIukj292FeuCgOrF6JGi0gahtxLAS8tBLci/KnuRIsHJYYO4akvljPyufsoH+C8G78uJ11iw1ujCPnP/9H0zHEeHfwx97zQh4jg+5z2nu5CewoqX0u6nMqeLbu5+NsSkuP2ICdOIGfP4pF0kRWd+nA6pCplEw4QsXkVBcoHU7xeGEGNIvSksZOsiT3Ik99vxTstja9qGsIfuTdH93/xfBLRr42kwqSxlD19lO0hoaS8/Q6h3Tvm6Pu4Oh0QTynlNvYeP8+8J17hmdljWN2+O7UnfUqhAP9b2uexoyf5dvMJJq/YzYxPnyDFvziXXnqZsP7d8uXhvn8qCvnvu6GUcmkhJQvRb9K7rGnXlUY/fcPFipVY8/K7XDx9Nkv7Sb6QxKbPp7KxXgu8QkIY+0sMdSoGcOa3pVTfHUP4U93zZUG4GZfqKYhIW2A04Al8ZYwZ8U/ra09Bqbxt++xFmBdfosbuGH6r3oTf3x9H86qlqFcoleLBZf72Sz3pcirbj5zlwLK1lBs9kqrRqymcfIHjhYuzq21nyrz3BiFV8talpdnlFoePRMQT2AncA8QD64Cuxpgb3gevRUGpvM+kpbFt+s/M33WKCZdLUeLEEVaO7ctZHz/O+hUlzcOTQhfP8XbLx5lV6y7qHoxl3NyR7G10F76dHqBm74f1HNA13KUoNAbeNMa0seaHAxhj3r/RNloUlMpfLqWkErM2ltTpM5C4ODzOnkFSU0ktWpR9bTtRrFVzapUpQlAJPz009A/c5ZLUcsCBdPPxQMNrVxKR/kB/gPLly+dOMqWUS/Dx8qR+k1BoEnrdsut+WahscbtSaowZb4yJNMZEBgZm71F0SimlMuZKReEgkP4sUJDVppRSKpe4UlFYB1QRkRARKQB0AebanEkppfIVlzmnYIxJEZFngIU4LkmdaIzZanMspZTKV1ymKAAYY+YD8+3OoZRS+ZUrHT5SSillMy0KSimlrtKioJRS6iqXuaM5O0QkAdifzc1LAsdzME5O0VxZo7myzlWzaa6suZVcFYwxGd7o5dZF4VaISNSNbvO2k+bKGs2Vda6aTXNljbNy6eEjpZRSV2lRUEopdVV+Lgrj7Q5wA5orazRX1rlqNs2VNU7JlW/PKSillLpefu4pKKWUuoYWBaWUUlfl66IgIu+ISIyIbBKRRSJym92ZAETkQxHZbmWbJSL+dmcCEJGHRWSriKSJiO2X6IlIWxHZISJxIjLM7jwAIjJRRI6JyBa7s6QnIsEislREtlk/w8F2ZwIQEV8RWSsi0Vaut+zOlJ6IeIrIRhH52e4sV4jIPhHZbP3eyvFHT+brogB8aIwJM8ZEAD8Dr9uc54pfgVBjTBiO51YPtznPFVuAzsAyu4NYz/T+HLgXqAl0FZGa9qYC4Gugrd0hMpACvGCMqQk0Aga6yPfrEtDSGBMORABtRaSRvZH+ZjAQa3eIDNxljInQ+xRymDHmTLrZQoBLnHU3xiwyxqRYs6txPHDIdsaYWGPMDrtzWBoAccaYPcaYZOA7oIPNmTDGLANO2p3jWsaYw8aYDdb0WRy/6MrZmwqMwzlr1tt6ucT/QxEJAu4DvrI7S27K10UBQET+LSIHgMdwnZ5Cen2BX+wO4YIyeqa37b/k3IGIVATqAGtsjgJcPUSzCTgG/GqMcYlcwCfAy0CazTmuZYBFIrLeemZ9jsrzRUFEfhORLRm8OgAYY141xgQD04BnXCWXtc6rOLr901wpl3JfIlIY+AF47pqesm2MManWIdwgoIGIhNocCRG5HzhmjFlvd5YM3GmMqYvj0OlAEWmWkzt3qYfsOIMx5u5MrjoNxwN+3nBinKtulktEegP3A61MLt5MkoXvl930md5ZJCLeOArCNGPMj3bnuZYx5rSILMVxTsbuE/V3AA+ISDvAFygqIt8YY7rbnAtjzEHr6zERmYXjUGqOnefL8z2FfyIiVdLNdgC225UlPRFpi6Pb+oAx5oLdeVyUPtM7C0REgAlArDHmY7vzXCEigVeurhORgsA9uMD/Q2PMcGNMkDGmIo5/W0tcoSCISCERKXJlGmhNDhfQfF0UgBHWoZEYHN9cl7hMD/gMKAL8al12Ns7uQAAi0klE4oHGwDwRWWhXFutE/JVnescCM1zhmd4i8i2wCqgmIvEi0s/uTJY7gB5AS+vf1Cbrr2C7lQWWWv8H1+E4p+Ayl3+6oNLAChGJBtYC84wxC3LyDXSYC6WUUlfl956CUkqpdLQoKKWUukqLglJKqau0KCillLpKi4JSSqmrtCioPENEzt18rRtu+4w12qoRkZLp2kVExljLYkSkbrplZa+MnikiLXJqJE0R+T0zo9Bao2WWvMk6v4lI8ZzIpfIHLQpKOfwJ3A3sv6b9XqCK9eoPjE23bAjwZa6ky76pwAC7Qyj3oUVB5TnWX/cfWjcmbhaRR612DxH5jzieVfGriMwXkYcAjDEbjTH7MthdB2CKNZrnasBfRMpayx4ErrtxSEQaiMgqaxz+lSJSzWrvLSKzrffeZ/VOhljrrRaREul208O6wWyLiDSwtg8Qx3M/torIV4Cke8/Z1gBpW68ZJG0u0DW730uV/2hRUHlRZxxj84fj+Ov/Q+sXeWegIo7nL/TAcWf2zWQ4GquIhACnjDGXMthmO9DUGFMHx8i776VbFmrlqA/8G7hgrbcK6JluPT9rkLgBwESr7Q1ghTGmFjALKJ9u/b7GmHpAJDBIRAIAjDGnAJ8r80rdTJ4fEE/lS3cC3xpjUoGjIvIHjl/CdwLfG2PSgCPW4GvZVRZIuMGyYsBka2wtg+MZAVcstZ5ncFZEEoGfrPbNQFi69b4Fx/MZRKSoNT5QMxwFBWPMPBE5lW79QSLSyZoOxnG464Q1fwy4Ld28UjekPQWl/tmNRmO9iGP0zIy8g+OXfyjQ/pr10vcs0tLNp/H3P9KuHX/mhuPRiEgLHD2ixtYTzDZe856+Vl6lbkqLgsqLlgOPiuPhLYE4/sJei+Nk8oPWuYXSQItM7Gsu0NM6T9EISDTGHMbxmNSKN9imGP8bxrt3Nj/DlfMgd1rvmYhjeORuVvu9wJWriorhOJR1QUSq43jcJtZ6ApQB9mUzh8pntCiovGgWEANEA0uAl40xR3A8SyAe2AZ8A2wAEgFEZJA1AmwQEGOdyAXHMzb2AHE4rjQaAGCMOQ/sFpHbM3j/D4D3RWQj2T9Em2RtPw64MtLqW0AzEdmK4zDSX1b7AsBLRGKBETge4XpFPWB1use7KvWPdJRUla+ISGFjzDnrxOta4A6rYGRnX52AesaYf+VoyBwkIqOBucaYxXZnUe5BTzSr/OZn66RtAeCd7BYEAGPMLDe4qmeLFgSVFdpTUEopdZWeU1BKKXWVFgWllFJXaVFQSil1lRYFpZRSV2lRUEopddX/A0kFnLz3Kj/2AAAAAElFTkSuQmCC\n",
- "text/plain": [
- ""
- ]
- },
- "metadata": {
- "needs_background": "light"
- },
- "output_type": "display_data"
- }
- ],
+ "execution_count": 6,
+ "id": "8a0b7b89",
+ "metadata": {
+ "collapsed": false,
+ "editable": true
+ },
+ "outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
@@ -2451,7 +2638,7 @@
"## Cross-validation on Ridge regression using KFold only\n",
"\n",
"# Decide degree on polynomial to fit\n",
- "poly = PolynomialFeatures(degree = 10)\n",
+ "poly = PolynomialFeatures(degree = 6)\n",
"\n",
"# Decide which values of lambda to use\n",
"nlambdas = 500\n",
@@ -2525,8 +2712,10 @@
},
{
"cell_type": "markdown",
- "id": "2e882dc9",
- "metadata": {},
+ "id": "a41beafb",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## More examples on bootstrap and cross-validation and errors"
]
@@ -2534,8 +2723,11 @@
{
"cell_type": "code",
"execution_count": 7,
- "id": "1346747f",
- "metadata": {},
+ "id": "e3038a58",
+ "metadata": {
+ "collapsed": false,
+ "editable": true
+ },
"outputs": [],
"source": [
"# Common imports\n",
@@ -2620,16 +2812,20 @@
},
{
"cell_type": "markdown",
- "id": "229b085a",
- "metadata": {},
+ "id": "19f4f20f",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Note that we kept the intercept column in the fitting here. This means that we need to set the **intercept** in the call to the **Scikit-Learn** function as **False**. Alternatively, we could have set up the design matrix $X$ without the first column of ones."
]
},
{
"cell_type": "markdown",
- "id": "e7a622a2",
- "metadata": {},
+ "id": "98b035c3",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## The same example but now with cross-validation\n",
"\n",
@@ -2639,8 +2835,11 @@
{
"cell_type": "code",
"execution_count": 8,
- "id": "66531f63",
- "metadata": {},
+ "id": "6ce294be",
+ "metadata": {
+ "collapsed": false,
+ "editable": true
+ },
"outputs": [],
"source": [
"# Common imports\n",
@@ -2714,8 +2913,10 @@
},
{
"cell_type": "markdown",
- "id": "c01472b1",
- "metadata": {},
+ "id": "63a7497b",
+ "metadata": {
+ "editable": true
+ },
"source": [
"## Notes on scaling with examples\n",
"\n",
@@ -2739,8 +2940,11 @@
{
"cell_type": "code",
"execution_count": 9,
- "id": "a0321a24",
- "metadata": {},
+ "id": "938f039a",
+ "metadata": {
+ "collapsed": false,
+ "editable": true
+ },
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
@@ -2809,8 +3013,10 @@
},
{
"cell_type": "markdown",
- "id": "3f37a915",
- "metadata": {},
+ "id": "e732961c",
+ "metadata": {
+ "editable": true
+ },
"source": [
"In this example we do not include the intercept and we scale the data by subtracting the mean values. This follows the discussion in the [lecture material](https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/chapter3.html#more-on-rescaling-data).\n",
"see also the weekly slides [for week 36](https://compphysics.github.io/MachineLearning/doc/pub/week36/html/._week36-bs029.html).\n",
@@ -2827,8 +3033,10 @@
},
{
"cell_type": "markdown",
- "id": "db803554",
- "metadata": {},
+ "id": "4c89151d",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"C(\\beta_0, \\beta_1, ... , \\beta_{p-1}) = \\frac{1}{n}\\sum_{i=0}^{n} \\left(y_i - \\beta_0 - \\sum_{j=1}^{p-1} X_{ij}\\beta_j\\right)^2,.\n",
@@ -2837,8 +3045,10 @@
},
{
"cell_type": "markdown",
- "id": "7864f9d2",
- "metadata": {},
+ "id": "52318e4d",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Recall also that we use the squared value. This expression can lead to an\n",
"increased penalty for higher differences between predicted and\n",
@@ -2852,8 +3062,10 @@
},
{
"cell_type": "markdown",
- "id": "95755a96",
- "metadata": {},
+ "id": "b8dc7f6d",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\frac{\\partial C}{\\partial \\beta_j} = 0,\n",
@@ -2862,16 +3074,20 @@
},
{
"cell_type": "markdown",
- "id": "ab2d94ba",
- "metadata": {},
+ "id": "0ea24a0d",
+ "metadata": {
+ "editable": true
+ },
"source": [
"for all $j$. For $\\beta_0$ we have"
]
},
{
"cell_type": "markdown",
- "id": "fc0e8cf9",
- "metadata": {},
+ "id": "47719be0",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\frac{\\partial C}{\\partial \\beta_0} = -\\frac{2}{n}\\sum_{i=0}^{n-1} \\left(y_i - \\beta_0 - \\sum_{j=1}^{p-1} X_{ij} \\beta_j\\right).\n",
@@ -2880,16 +3096,20 @@
},
{
"cell_type": "markdown",
- "id": "a0451a39",
- "metadata": {},
+ "id": "4be82ca9",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Multiplying away the constant $2/n$, we obtain"
]
},
{
"cell_type": "markdown",
- "id": "0c138c00",
- "metadata": {},
+ "id": "778ad469",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\sum_{i=0}^{n-1} \\beta_0 = \\sum_{i=0}^{n-1}y_i - \\sum_{i=0}^{n-1} \\sum_{j=1}^{p-1} X_{ij} \\beta_j.\n",
@@ -2898,8 +3118,10 @@
},
{
"cell_type": "markdown",
- "id": "f2eee08a",
- "metadata": {},
+ "id": "c8690ebb",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Let us specialize first to the case where we have only two parameters $\\beta_0$ and $\\beta_1$.\n",
"Our result for $\\beta_0$ simplifies then to"
@@ -2907,8 +3129,10 @@
},
{
"cell_type": "markdown",
- "id": "289597ac",
- "metadata": {},
+ "id": "19422394",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"n\\beta_0 = \\sum_{i=0}^{n-1}y_i - \\sum_{i=0}^{n-1} X_{i1} \\beta_1.\n",
@@ -2917,16 +3141,20 @@
},
{
"cell_type": "markdown",
- "id": "4e7c637a",
- "metadata": {},
+ "id": "bc1ec19a",
+ "metadata": {
+ "editable": true
+ },
"source": [
"We obtain then"
]
},
{
"cell_type": "markdown",
- "id": "ee5b28ee",
- "metadata": {},
+ "id": "61370bdd",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1}y_i - \\beta_1\\frac{1}{n}\\sum_{i=0}^{n-1} X_{i1}.\n",
@@ -2935,16 +3163,20 @@
},
{
"cell_type": "markdown",
- "id": "74f6d0a5",
- "metadata": {},
+ "id": "9ff08be6",
+ "metadata": {
+ "editable": true
+ },
"source": [
"If we define"
]
},
{
"cell_type": "markdown",
- "id": "52d5f956",
- "metadata": {},
+ "id": "9cd5def5",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\mu_{\\boldsymbol{x}_1}=\\frac{1}{n}\\sum_{i=0}^{n-1} X_{i1},\n",
@@ -2953,16 +3185,20 @@
},
{
"cell_type": "markdown",
- "id": "38886105",
- "metadata": {},
+ "id": "b080c860",
+ "metadata": {
+ "editable": true
+ },
"source": [
"and the mean value of the outputs as"
]
},
{
"cell_type": "markdown",
- "id": "0abe36b0",
- "metadata": {},
+ "id": "10fc1a84",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\mu_y=\\frac{1}{n}\\sum_{i=0}^{n-1}y_i,\n",
@@ -2971,16 +3207,20 @@
},
{
"cell_type": "markdown",
- "id": "4987ef25",
- "metadata": {},
+ "id": "df6a1ef4",
+ "metadata": {
+ "editable": true
+ },
"source": [
"we have"
]
},
{
"cell_type": "markdown",
- "id": "aec91083",
- "metadata": {},
+ "id": "c62e999c",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\beta_0 = \\mu_y - \\beta_1\\mu_{\\boldsymbol{x}_1}.\n",
@@ -2989,16 +3229,20 @@
},
{
"cell_type": "markdown",
- "id": "5f8a4d1d",
- "metadata": {},
+ "id": "e16831f8",
+ "metadata": {
+ "editable": true
+ },
"source": [
"In the general case with more parameters than $\\beta_0$ and $\\beta_1$, we have"
]
},
{
"cell_type": "markdown",
- "id": "2df58f1a",
- "metadata": {},
+ "id": "863e58fa",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1}y_i - \\frac{1}{n}\\sum_{i=0}^{n-1}\\sum_{j=1}^{p-1} X_{ij}\\beta_j.\n",
@@ -3007,16 +3251,20 @@
},
{
"cell_type": "markdown",
- "id": "d8fbe238",
- "metadata": {},
+ "id": "b146fe2b",
+ "metadata": {
+ "editable": true
+ },
"source": [
"We can rewrite the latter equation as"
]
},
{
"cell_type": "markdown",
- "id": "4e7618f7",
- "metadata": {},
+ "id": "89ca9b85",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1}y_i - \\sum_{j=1}^{p-1} \\mu_{\\boldsymbol{x}_j}\\beta_j,\n",
@@ -3025,16 +3273,20 @@
},
{
"cell_type": "markdown",
- "id": "7a88a697",
- "metadata": {},
+ "id": "323f217d",
+ "metadata": {
+ "editable": true
+ },
"source": [
"where we have defined"
]
},
{
"cell_type": "markdown",
- "id": "524294e9",
- "metadata": {},
+ "id": "ce63593a",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\mu_{\\boldsymbol{x}_j}=\\frac{1}{n}\\sum_{i=0}^{n-1} X_{ij},\n",
@@ -3043,8 +3295,10 @@
},
{
"cell_type": "markdown",
- "id": "c99deb02",
- "metadata": {},
+ "id": "07c90e67",
+ "metadata": {
+ "editable": true
+ },
"source": [
"the mean value for all elements of the column vector $\\boldsymbol{x}_j$.\n",
"\n",
@@ -3053,8 +3307,10 @@
},
{
"cell_type": "markdown",
- "id": "93edef09",
- "metadata": {},
+ "id": "467273e3",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"C(\\boldsymbol{\\beta}) = (\\boldsymbol{\\tilde{y}} - \\tilde{X}\\boldsymbol{\\beta})^T(\\boldsymbol{\\tilde{y}} - \\tilde{X}\\boldsymbol{\\beta}).\n",
@@ -3063,16 +3319,20 @@
},
{
"cell_type": "markdown",
- "id": "7d5b2225",
- "metadata": {},
+ "id": "eaebf350",
+ "metadata": {
+ "editable": true
+ },
"source": [
"If we minimize with respect to $\\boldsymbol{\\beta}$ we have then"
]
},
{
"cell_type": "markdown",
- "id": "e8d16eeb",
- "metadata": {},
+ "id": "bdb72a09",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\hat{\\boldsymbol{\\beta}} = (\\tilde{X}^T\\tilde{X})^{-1}\\tilde{X}^T\\boldsymbol{\\tilde{y}},\n",
@@ -3081,8 +3341,10 @@
},
{
"cell_type": "markdown",
- "id": "4398395e",
- "metadata": {},
+ "id": "68e0b318",
+ "metadata": {
+ "editable": true
+ },
"source": [
"where $\\boldsymbol{\\tilde{y}} = \\boldsymbol{y} - \\overline{\\boldsymbol{y}}$\n",
"and $\\tilde{X}_{ij} = X_{ij} - \\frac{1}{n}\\sum_{k=0}^{n-1}X_{kj}$.\n",
@@ -3092,8 +3354,10 @@
},
{
"cell_type": "markdown",
- "id": "392021b8",
- "metadata": {},
+ "id": "9c298056",
+ "metadata": {
+ "editable": true
+ },
"source": [
"$$\n",
"\\hat{\\boldsymbol{\\beta}} = (\\tilde{X}^T\\tilde{X} + \\lambda I)^{-1}\\tilde{X}^T\\boldsymbol{\\tilde{y}}.\n",
@@ -3102,8 +3366,10 @@
},
{
"cell_type": "markdown",
- "id": "be7a0552",
- "metadata": {},
+ "id": "fa2668d8",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Now we try to implement this."
]
@@ -3111,8 +3377,11 @@
{
"cell_type": "code",
"execution_count": 10,
- "id": "e572b7ed",
- "metadata": {},
+ "id": "010ff40f",
+ "metadata": {
+ "collapsed": false,
+ "editable": true
+ },
"outputs": [],
"source": [
"\n",
@@ -3181,8 +3450,10 @@
},
{
"cell_type": "markdown",
- "id": "390485db",
- "metadata": {},
+ "id": "e857615a",
+ "metadata": {
+ "editable": true
+ },
"source": [
"Finally, instead of using our own function we repeat the same example\n",
"using the **standardscaler** functionality of the library\n",
@@ -3192,8 +3463,11 @@
{
"cell_type": "code",
"execution_count": 11,
- "id": "9c72207d",
- "metadata": {},
+ "id": "a3ad5ac3",
+ "metadata": {
+ "collapsed": false,
+ "editable": true
+ },
"outputs": [],
"source": [
"from sklearn import linear_model\n",
@@ -3258,25 +3532,7 @@
]
}
],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3 (ipykernel)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.16"
- }
- },
+ "metadata": {},
"nbformat": 4,
"nbformat_minor": 5
}
diff --git a/doc/src/week37/week37.do.txt b/doc/src/week37/week37.do.txt
index 3b742d382..c5a0bdd72 100644
--- a/doc/src/week37/week37.do.txt
+++ b/doc/src/week37/week37.do.txt
@@ -16,6 +16,8 @@ DATE: today
* For more discussions of Ridge regression and calculation of averages, "Wessel van Wieringen's":"https://arxiv.org/abs/1509.09169" article is highly recommended.
!eblock
!bblock Material for the lecture on Thursday September 7
+ * "Video of Lecture":"https://youtu.be/YOBBr_toYxc"
+ * "Whiteboard notes":"https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2023/NotesSep14.pdf"
* Resampling techniques, Bootstrap and cross validation and bias-variance tradeoff
* Statistical interpretation of Ridge and Lasso regression
* Readings and Videos: