diff --git a/doc/LectureNotes/exercisesweek34.ipynb b/doc/LectureNotes/exercisesweek34.ipynb index ccdcab057..5a9b9a23b 100644 --- a/doc/LectureNotes/exercisesweek34.ipynb +++ b/doc/LectureNotes/exercisesweek34.ipynb @@ -2,302 +2,312 @@ "cells": [ { "cell_type": "markdown", - "id": "7791881e", - "metadata": { - "editable": true - }, - "source": [ - "\n", - "" - ] - }, - { - "cell_type": "markdown", - "id": "fd145f2c", - "metadata": { - "editable": true - }, + "id": "232d1306", + "metadata": {}, "source": [ "# Exercises week 34\n", - "**FYS-STK3155/4155**\n", "\n", - "Date: **August 19-23, 2024**" + "## Coding Setup and Linear Regression" ] }, { "cell_type": "markdown", - "id": "5543700e", - "metadata": { - "editable": true - }, + "id": "9b66a351", + "metadata": {}, "source": [ - "## Exercises\n", + "Welcome to FYS-STK3155/4155!\n", "\n", - "Here are three possible exercises for week 34" + "In this first week will focus on getting you set up with the programs you are going to be using throughout this course. We expect that many of you will encounter some trouble with setting these programs up, as they can be extremely finnicky and prone to not working the same on all machines, so we strongly encourage you to not get discouraged, and to show up to the group-sessions where we can help you along. The group sessions are also the best place to find group partners for the projects and to be challenged on your understanding of the material, which are both essential to doing well in this course. We strongly encourage you to form groups of 2-3 participants. \n", + "\n", + "If you are unable to complete this week's exercises, don't worry, this will likely be the most frustrating week for many of you. You have time to get back on track next week, especially if you come to the group-sessions! Note also that this week's set of exercises does not count for the additional score. The deadline for the weekly exercises is set to Fridays, at midnight." ] }, { "cell_type": "markdown", - "id": "0e490470", - "metadata": { - "editable": true - }, + "id": "36d8750b", + "metadata": {}, "source": [ - "## Exercise 1: Setting up various Python environments\n", + "### Learning goals\n", "\n", - "The first exercise here is of a mere technical art. We want you to have \n", - "* git as a version control software and to establish a user account on a provider like GitHub. Other providers like GitLab etc are equally fine. You can also use the University of Oslo [GitHub facilities](https://www.uio.no/tjenester/it/maskin/filer/versjonskontroll/github.html). \n", + "After completing these exercises, you will know how to\n", "\n", - "* Install various Python packages\n", + "- Create and use a Github repository\n", + "- Set up and use a virtual environment in Python\n", + "- Fit an OLS model to data using scikit-learn\n", + "- Fit a model on training data and evaluate it on test data\n", "\n", - "We will make extensive use of Python as programming language and its\n", - "myriad of available libraries. You will find\n", - "IPython/Jupyter notebooks invaluable in your work. You can run **R**\n", - "codes in the Jupyter/IPython notebooks, with the immediate benefit of\n", - "visualizing your data. You can also use compiled languages like C++,\n", - "Rust, Fortran etc if you prefer. The focus in these lectures will be\n", - "on Python.\n", + "### Deliverables\n", "\n", - "If you have Python installed (we recommend Python3) and you feel\n", - "pretty familiar with installing different packages, we recommend that\n", - "you install the following Python packages via **pip** as \n", + "Complete the following exercises while working in a jupyter notebook. Exercises 1,2 and 3 require no writing in the notebook. Then, in canvas, include\n", + "- The jupyter notebook with the exercises completed\n", + "- An exported PDF of the notebook (https://code.visualstudio.com/docs/datascience/jupyter-notebooks#_export-your-jupyter-notebook)\n", + "- Optional: A link to your github repository, which must be set to public, include the notebook file, a README file, requirements file and gitignore file.\n", "\n", - "1. pip install numpy scipy matplotlib ipython scikit-learn sympy pandas pillow \n", - "\n", - "For **Tensorflow**, we recommend following the instructions in the text of \n", - "[Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow, O'Reilly](http://shop.oreilly.com/product/0636920052289.do)\n", - "\n", - "We will come back to **tensorflow** later. \n", - "\n", - "For Python3, replace **pip** with **pip3**.\n", - "\n", - "For OSX users we recommend, after having installed Xcode, to\n", - "install **brew**. Brew allows for a seamless installation of additional\n", - "software via for example \n", - "\n", - "1. brew install python3\n", - "\n", - "For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution,\n", - "you can use **pip** as well and simply install Python as \n", - "\n", - "1. sudo apt-get install python3 (or python for Python2.7)\n", - "\n", - "If you don't want to perform these operations separately and venture\n", - "into the hassle of exploring how to set up dependencies and paths, we\n", - "recommend two widely used distrubutions which set up all relevant\n", - "dependencies for Python, namely \n", - "\n", - "* [Anaconda](https://docs.anaconda.com/), \n", - "\n", - "which is an open source\n", - "distribution of the Python and R programming languages for large-scale\n", - "data processing, predictive analytics, and scientific computing, that\n", - "aims to simplify package management and deployment. Package versions\n", - "are managed by the package management system **conda**. \n", - "\n", - "* [Enthought canopy](https://www.enthought.com/product/canopy/) \n", - "\n", - "is a Python\n", - "distribution for scientific and analytic computing distribution and\n", - "analysis environment, available for free and under a commercial\n", - "license.\n", - "\n", - "We recommend using **Anaconda** if you are not too familiar with setting paths in a terminal environment." + "We require you to deliver a jupyter notebook so that we can evaluate the results of your code without needing to download and run the code of every student, as well as to teach you to use this useful tool." ] }, { "cell_type": "markdown", - "id": "5b1c397f", + "id": "2a9c7ef8", + "metadata": {}, + "source": [ + "## Exercise 1 - Github Setup\n" + ] + }, + { + "cell_type": "markdown", + "id": "1498aed1", + "metadata": {}, + "source": [ + "In this course, we require you to pay extra mind to the reproducibility of your results and the shareability of your code. The first step toward these goals is using a version control system like git and online repository like Github.\n", + "\n", + "**a)** Download git if you don't already have it on your machine, check with the terminal command ´git --version´ (https://git-scm.com/downloads).\n", + "\n", + "**b)** Create a Github account(https://github.com/), or log in to github with your UiO account (https://github.uio.no/login).\n", + "\n", + "**c)** Learn the basics of opening the terminal and navigating folders on your operating system. Things to learn: Opening a terminal, opening a terminal in a specific folder, listing the contents of the current folder, navigating into a folder, navigating out of a folder.\n", + "\n", + "**d)** Download the Github CLI tool and run ´gh auth login´ in your terminal to authenticate your local machine for some of the later steps. (https://github.com/cli/cli#installation). You might need to change file permissions to make it work, ask us or ChatGPT for help with these issues." + ] + }, + { + "cell_type": "markdown", + "id": "c56fbefa", + "metadata": {}, + "source": [ + "## Exercise 2 - Setting up a Github repository\n" + ] + }, + { + "cell_type": "markdown", + "id": "fb9b8acd", "metadata": { - "editable": true + "vscode": { + "languageId": "plaintext" + } }, "source": [ - "## Exercise 2: making your own data and exploring scikit-learn\n", + "**a)** Create an empty repository for your coursework in this course in your browser at github.com (or uio github).\n", "\n", - "We will generate our own dataset for a function $y(x)$ where $x \\in [0,1]$ and defined by random numbers computed with the uniform distribution. The function $y$ is a quadratic polynomial in $x$ with added stochastic noise according to the normal distribution $\\cal {N}(0,1)$.\n", - "The following simple Python instructions define our $x$ and $y$ values (with 100 data points)." + "**b)** Open a terminal in the location you want to create your local folder for this repository, like your desktop.\n", + "\n", + "**c)** Clone the repository to your laptop using the terminal command ´gh repo clone username/repository-name´. This creates a folder with the same name as the repository. Moving it or renaming it might require some extra steps.\n", + "\n", + "**d)** Download this jupyter notebook. Add the notebook to the local folder.\n", + "\n", + "**e)** Run the ´git add .´ command command in a terminal opened in the local folder to stage the current changes in the folder to be commited to the version control history. Run ´git status´ to see the staged files.\n", + "\n", + "**f)** Run the ´git commit -m \"Adding first weekly assignment file\"´ command to commit the staged changes to the version control history. Run ´git status´ to see that no files are staged.\n", + "\n", + "**g)** Run the ´git push\" command to upload the commited changes to the remote repository on Github.\n", + "\n", + "**h)** Add a file called README.txt to the repository at Github.com. Don't do this in your local folder. Add a suitable title for your repository and some inforomation to the file.\n", + "\n", + "**i)** Run the ´git fetch origin´ command to fetch the latest remote changes to your repository.\n", + "\n", + "**j)** Run the ´git pull´ command to download and update files to match the remote changes.\n" + ] + }, + { + "cell_type": "markdown", + "id": "f84d0db6", + "metadata": {}, + "source": [ + "## Exercise 3 - Setting up a Python virtual environment\n" + ] + }, + { + "cell_type": "markdown", + "id": "b5a4818a", + "metadata": {}, + "source": [ + "Following the themes from the previous exercises, another way of improving the reproducibility of your results and shareability of your code is having a good handle on which python packages you are using.\n", + "\n", + "There are many ways to manage your packages in Python, and you are free to use any approach you want, but in this course we encourage you to use something called a virtual environment. A virtual environemnt is a folder in your project which contains a Python runtime executable as well as all the packages you are using in the current project. In this way, each of your projects has its required set of packages installed in the same folder, so that if anything goes wrong while managing your packages it only affects the one project, and if multiple projects require different versions of the same package, you don't need to worry about messing up old projects. Also, it's easy to just delete the folder and start over if anything goes wrong.\n", + "\n", + "Virtual environments are typically created, activated, managed and updated using terminal commands, but for now we recommend that you let for example VS Code (a popular cross-paltform package) handle it for you to make the coding experience much easier. If you are familiar with another approach for virtual environments that works for you, feel free to keep doing it that way.\n" + ] + }, + { + "cell_type": "markdown", + "id": "0f6de364", + "metadata": {}, + "source": [ + "**a)** Open this notebook in VS Code (https://code.visualstudio.com/Download). Download the Python and Jupyter extensions.\n", + "\n", + "**b)** Press ´Cmd + Shift + P´, then search and run ´Python: Create Environment...´\n", + "\n", + "**c)** Select ´Venv´\n", + "\n", + "**d)** Choose the most up-to-date version of Python your have installed.\n", + "\n", + "**e)** Press ´Cmd + Shift + P´, then search and run ´Python: Select Interpreter´\n", + "\n", + "**f)** Selevet the (.venv) option you just created.\n", + "\n", + "**g)** Open a terminal in VS Code, the venv name should be visible at the beginning of the line. Run `pip list` to see that there are no packages install in the environment.\n", + "\n", + "**h)** In this terminal, run `pip install matplotlib numpy scikit-learn`. This will install the listed packages.\n", + "\n", + "**i)** To make these installations reproducible, which is important for reproducing results and sharing your code, run ´pip freeze > requirements.txt´ to create the file requirements.txt with all your dependencies.\n", + "\n", + "Now, anyone who wants to recreate your package setup can download your requirements.txt file and run ´pip install -r requirements.txt´ to install the correct packages and versions. To keep the requirements.txt file up to date with your environment, you will need to re-run the freeze command whenever you install a new package.\n", + "\n", + "**j)** Create a .gitignore file at the root of your project folder, and add the line ´.venv´ to it. This way, you won't try to upload a copy of all your python packages when you regularly push your changes to Github. Ignored files should not show up when you run ´git status´, and are not staged when running ´git add .´, try it!" + ] + }, + { + "cell_type": "markdown", + "id": "5d184ab1", + "metadata": {}, + "source": [ + "## Exercise 3 - Fitting an OLS model to data\n" + ] + }, + { + "cell_type": "markdown", + "id": "d19ebd67", + "metadata": {}, + "source": [ + "Great job on getting through all of that! Now it is time to do some actual machine learning!\n", + "\n", + "**a)** Complete the code below so that you fit a second order polynomial to the data. You will need to look up some scikit-learn documentation online (look at the imported functions for hints).\n", + "\n", + "**b)** Compute the mean square error for the line model and for the second degree polynomial model." ] }, { "cell_type": "code", "execution_count": 1, - "id": "eb74f73c", - "metadata": { - "collapsed": false, - "editable": true - }, + "id": "b58fb9bf", + "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", - "x = np.random.rand(100,1)\n", - "y = 2.0+5*x*x+0.1*np.random.randn(100,1)" - ] - }, - { - "cell_type": "markdown", - "id": "49257b1a", - "metadata": { - "editable": true - }, - "source": [ - "1. Write your own code (following the examples under the [regression notes](https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/chapter1.html)) for computing the parametrization of the data set fitting a second-order polynomial. \n", - "\n", - "2. Use thereafter **scikit-learn** (see again the examples in the regression slides) and compare with your own code. \n", - "\n", - "3. Using scikit-learn, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as" - ] - }, - { - "cell_type": "markdown", - "id": "3bfa5d7b", - "metadata": { - "editable": true - }, - "source": [ - "$$\n", - "MSE(\\boldsymbol{y},\\boldsymbol{\\tilde{y}}) = \\frac{1}{n}\n", - "\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2,\n", - "$$" - ] - }, - { - "cell_type": "markdown", - "id": "5df6889b", - "metadata": { - "editable": true - }, - "source": [ - "and the $R^2$ score function.\n", - "If $\\tilde{\\boldsymbol{y}}_i$ is the predicted value of the $i-th$ sample and $y_i$ is the corresponding true value, then the score $R^2$ is defined as" - ] - }, - { - "cell_type": "markdown", - "id": "2f021f71", - "metadata": { - "editable": true - }, - "source": [ - "$$\n", - "R^2(\\boldsymbol{y}, \\tilde{\\boldsymbol{y}}) = 1 - \\frac{\\sum_{i=0}^{n - 1} (y_i - \\tilde{y}_i)^2}{\\sum_{i=0}^{n - 1} (y_i - \\bar{y})^2},\n", - "$$" - ] - }, - { - "cell_type": "markdown", - "id": "7b91a34b", - "metadata": { - "editable": true - }, - "source": [ - "where we have defined the mean value of $\\boldsymbol{y}$ as" - ] - }, - { - "cell_type": "markdown", - "id": "58da66a3", - "metadata": { - "editable": true - }, - "source": [ - "$$\n", - "\\bar{y} = \\frac{1}{n} \\sum_{i=0}^{n - 1} y_i.\n", - "$$" - ] - }, - { - "cell_type": "markdown", - "id": "2d3dc22b", - "metadata": { - "editable": true - }, - "source": [ - "You can use the functionality included in scikit-learn. If you feel for it, you can use your own program and define functions which compute the above two functions. \n", - "Discuss the meaning of these results. Try also to vary the coefficient in front of the added stochastic noise term and discuss the quality of the fits." - ] - }, - { - "cell_type": "markdown", - "id": "4527a3e2", - "metadata": { - "editable": true - }, - "source": [ - "## Exercise 3: Split data in test and training data\n", - "\n", - "In this exercise we want you to to compute the MSE for the training\n", - "data and the test data as function of the complexity of a polynomial,\n", - "that is the degree of a given polynomial.\n", - "\n", - "The aim is to reproduce Figure 2.11 of [Hastie et al](https://github.com/CompPhysics/MLErasmus/blob/master/doc/Textbooks/elementsstat.pdf).\n", - "\n", - "Our data is defined by $x\\in [-3,3]$ with a total of for example $n=100$ data points. You should try to vary the number of data points $n$ in your analysis." + "import matplotlib.pyplot as plt\n", + "from sklearn.preprocessing import PolynomialFeatures # use the fit_transform method of the created object!\n", + "from sklearn.linear_model import LinearRegression\n", + "from sklearn.metrics import mean_squared_error" ] }, { "cell_type": "code", "execution_count": 2, - "id": "87ac310b", - "metadata": { - "collapsed": false, - "editable": true - }, + "id": "0208e9ca", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhYAAAGdCAYAAABO2DpVAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjUsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvWftoOwAAAAlwSFlzAAAPYQAAD2EBqD+naQAASZ5JREFUeJzt3QuczPX++PH37GItdpclLVIuiSSV08U1KUVJOnXSr4vLOV1Jkn+FOn6ojqXboZtup1Kd8lMhKioJSeIkHZduWJdEInZdF7vzf7y/s7PNzs7l+535zv31fDzmrJ35zsx3J8f3vZ/P++JwOp1OAQAAsEGaHS8CAACgCCwAAIBtCCwAAIBtCCwAAIBtCCwAAIBtCCwAAIBtCCwAAIBtCCwAAIBtqkiUlZaWyi+//CJZWVnicDii/fYAACAE2k9z37590rBhQ0lLS4ufwEKDisaNG0f7bQEAgA22bt0qJ5xwgj2BRZMmTWTz5s2V7h88eLA888wzpl5DVyrcJ5adnW3l7QEAQIwUFRUZCwPu67gtgcWKFSukpKSk/Ps1a9bIxRdfLNdcc43p13Bvf2hQQWABAEBiCZbGYCmwOO644yp8P2HCBGnevLl07do1tLMDAABJJeQciyNHjsgbb7whw4cPDxi9FBcXGzfPpRQAAJCcQi43nTVrluzdu1cGDhwY8Lj8/HzJyckpv5G4CQBA8nI4tX4kBD169JBq1arJnDlzAh7na8VCg4vCwkK/ORaax3H06NFQTgsJJj09XapUqULpMQDEOb1+6wJBoOt3yFshWhkyf/58mTFjRtBjMzIyjJtZ+/fvl59//tmol0VqqFGjhjRo0MAIVAEAiS2kwOKVV16R+vXrS69evWw9GV2p0KBCLzSaKMpvsclNg0fN1fntt9+koKBAWrRoEbDpCgAgCQML7ZypgcWAAQOMJWw76faHXmw0qMjMzLT1tRGf9L9z1apVjVUwDTKqV68e61MCAITB8q+HugWyZcsW+dvf/iaRwkpFamGVAgCSh+Ulh0suuYT8BwAA4kxJqVOWF/wuO/cdlvpZ1eXcprmSnhb9X9SjPisEAADYG0x8sm6HzFr1i/x+4Ej5Yw1yqsuY3q2lZ5sGEk0EFgAAJKB5a7bLuDnrZHvhYZ+P7yg8LIPeWClTbmwX1eCCzW0baJMwzQvRmyYiHn/88cYMlZdfftlIdjXr1Vdfldq1a0f0XAEAyRFUDHpjpd+gQrmTFjT40JWNaEnKwEI/wC837Jb3Vm0zvkbjA+3Zs6ds375dNm3aJHPnzpVu3brJXXfdJZdffrkcO3Ys4u8PAEhuJWXXtpnfbJP7Z64uDxwC0WM0+NDtkmipkgpLQ9HYZ9ImYHl5ecafGzVqJO3atZP27dvLRRddZKxE3HzzzfLEE08YpbobN26U3Nxc6d27tzzyyCNSq1YtWbhwofz1r3+tUBUzZswYGTt2rLz++usyefJk+eGHH6RmzZpy4YUXyqRJk4xeIgCA5DcvyLZHMJrQGS1pqbA05N5n0sejSQOAM844o7xDqZZVPvnkk7J27VqZOnWqLFiwQO677z7jsY4dOxrBgrZJ1ZUPvd1zzz3l/T0eeugh+fbbb40ZLboqEmxGCwAgdbY9gtEqkWipkkxLRBrN+Voa0vt0DUAfv7h1XlTLb1q1aiX//e9/jT8PGzas/P4mTZrIww8/LLfffrs8++yzRjtr7cGuqxXulQ83z54hzZo1M4KTc845x2h/rqsdAIDkVBLg2maGXu3yclylp9GSNCsWun8ULIkl2vtMxvs6neVbG9pcTLdGdKskKytL+vXrJ7t375aDBw8GfI2vv/7a2DY58cQTjed17drVuF8blQEAktfyINe2QNy/QmsqQDR/oU6awMLs/lE095nUd999J02bNjW2LzSRs23btvLuu+8awcIzzzxjHKOtrP05cOCAMUlWt0j+/e9/y4oVK2TmzJlBnwcASHw7w7hm6UpFtEtNk2orxOz+UTT3mTSHYvXq1XL33XcbgYSWnj7++OPlLaynT59e4XjdDtFBbJ6+//57Y1VjwoQJxrh59Z///CdqPwMAIHbq1TI/HdzTkG7N5e6LW8ak82bSrFjo/pFWf/j7CPX+BhHcZyouLpYdO3bItm3bZOXKlTJ+/Hjp06ePsUrRv39/Ofnkk40kzKeeesqoCtFKj+eee67Ca2jeheZNfPrpp7Jr1y5ji0S3PzTgcD9v9uzZRiInACAFOEN7WqeTj4tJUJFUgYV+gLqPpBwx2GeaN2+eNGjQwAgOtKfFZ599ZiRZvvfee5Kenm5Uh2i56cSJE6VNmzbGtkZ+fn6F19DKEE3mvPbaa40Jr1qKql+1XPXtt9+W1q1bGysXjz32WER+BgBAfNl1oNjS8ZH+JdrUOTijPFGsqKjIqH4oLCw08gY8HT58WAoKCoychFDHZ8eqjwVCZ8d/dwBIRl9u2C3XvbjM1LHuX5sjlVcR6PqdlDkWbvphaklpPEx4AwDAjm1+7cfkNJGsGQ+/RCddYKE0iOjQvG6sTwMAgLD9zzknyj/n/1jpfv11WYONv3VqYvxCHS+/RCdlYAEAQLK38c6LkxUKbwQWAADEaRtvp5/H7+7eQoZc2CIuViiStioEAIBUaOPtEJFpK7ZKvCKwAAAgjiyP0xEVZhFYAAAQR3bG6YgKswgsAACII/XjcESFFSRvAgAQ5RyK5QF6LQXrXRGLUehWsGIRBTo2fdasWZIsxo4dK2eeeabp43Wyq34Gq1atiuh5AUAiVHt0nrjA6KZ517RVxlf9Xu/37l3hL3lT77/ijAZxWRGSvCsWpSUim5eK7P9VpNbxIid1FElLj9jbDRw4UPbu3es3eNi+fbvUqVMnYu8PAIh//kpIdWVC79dW3CpQ7wq3FxYXyFkn1om7HhbJGVismy0yb4RI0S9/3JfdUKTnRJHWV8TklPLy8mLyvgCA+C8hdZZtb4yasVr2HDxq+jX19bTjZrytXKQlXVAxvX/FoEIVbXfdr4/HeCvEvS0wY8YM6datm9SoUcOYfPrll19WeM6SJUukS5cukpmZKY0bN5ahQ4fKgQMHgm5PvPzyy8ao9Vq1asngwYOlpKTEmJKqwU39+vXlH//4R4XnbdmyxRjvrsfrUJm+ffvKr7/+WuEYnah6/PHHS1ZWltx0003G0DBvL730kpx66qnGELFWrVrJs88+G+anBgCpVUK6x0JQEc8lp2lJtf2hKxV+40FdhxrpOi4OPPDAA3LPPfcYeQennHKKXHfddXLs2DHjsQ0bNhij16+++mr573//K//3f/9nBBpDhgwJ+Jr6vLlz5xoj3N966y3517/+Jb169ZKff/5ZFi1aZIxs//vf/y5fffWVcXxpaakRVPz+++/G45988ols3LjRGNvuNn36dCNoGT9+vPznP/8xRsN7Bw06Av5///d/jaDlu+++M44dPXq0TJ06NSKfHQAkmp0RKg2Nx5LT5NkK0ZwK75WKCpwiRdtcxzXtIrGmQYVe9NW4cePktNNOk/Xr1xu/7efn58sNN9wgw4YNMx5v0aKFPPnkk9K1a1eZMmWK39HiGijoioWuLLRu3dpYEfnhhx/kww8/lLS0NGnZsqURXHz22Wdy3nnnyaeffiqrV682Rpbrqoh67bXXjHNZsWKFnHPOOTJp0iRjlUJv6uGHH5b58+dXWLUYM2aMPP7443LVVVcZ3+v483Xr1snzzz8vAwYMiPhnCQDxrn6ESkPjseQ0eVYsNFHTzuMirG3btuV/1lUAtXPnTuPrt99+K6+++qqxPeG+9ejRwwgcNAjwp0mTJkZQ4abbFxpgaFDheZ/7fXR1QQMKd1Ch9PjatWsbj7mP0SDEU4cOHcr/rNszulKigYfn+WoAovcDAET+dFIdsTMVQl+qQZyWnCbPioVWf9h5XIRVrVq1/M+ac6E0cFD79++X2267zcir8Kb5E2Ze0/26vu5zv48d9FzViy++WCkASU+PXCUOACSSrzfvkVJ/9aMWueMTnWwab4mbyRVYaEmpVn9ooqa/liL6uB4X59q1a2dsJZx88skRfR9Ntty6datxc69a6Ptq6ayuXLiP0ZyM/v37lz9v2bJlFVZAGjZsaORm6PYNACC8XAgNFQLFIPE6Lj35AgvtU6ElpVr9Uek/S1lE13NCxPpZFBYWVmoAVbdu3QrbDGaNGDFC2rdvbyRr3nzzzVKzZk3jgq/JlU8//bRt59y9e3c5/fTTjYBAcyk0eVQrSTSX4+yzzzaOueuuu4w+Hfp9p06djETNtWvXSrNmzcpfR3NEdHUlJyfHSDotLi42Ej337Nkjw4cPt+18ASBR1TeZC6Hj0HVyqWcFSV52hlx37onSpF5Nn506403yBBZK+1T0fc1PH4sJEe1jsXDhQjnrrLMq3Kd5B1qGGUr+hVZpaOWIlpw6nU5p3rx5hWoNO+i2yHvvvSd33nmnnH/++UYuhgYGTz31VPkx+p6aK3HfffcZCZtaqTJo0CD56KOPyo/R4EfLZh999FG59957jUBIAxZ38ikApLpzTbbpHnJhC+MWqOV3vHM49aoVRUVFRcZvtvobvvZN8KQXLk1O1KoCf5UP8dh5E+Gx7b87ACRA503xvaZudN6M1+2NYNfv5F2xcNMgIg5KSgEAqafEz5AxDRo0ePBu2R3vORNWJWdgAQBAjFYlxnkFDg08Age9aRvuRN7qCIbAAgCAKA0Z69nGNZW0Q/O69p9AnKQBEFgAABCFIWPjIjk0LI4GcMZl580o55MixvjvDSAVhoxtj8TQMF2lWKitFvrFzQDOuAos3J0ajxw5EutTQRQdPHjQ+OrdJRQAkq0B1k47h4atnSXyaAuRheP9HBCbAZxxtRVSpUoVox/Cb7/9ZlxkPGdcIDlXKjSo0NklOp+EFuAAEqXCw/vxn37dF92hYR+PFln6pIkDoz+AM64CC23YpAO5tKfB5s2bY306iBINKvLy8mJ9GgBgqcLD1+P+uBtg2TI0bM0sk0FFbAZwxlVgoapVq2aMCWc7JDXoyhQrFQASrcLj1vObyguLCwLO9LB9aNixIyLLXxD59EHrz43iAM64CyyUboHQgREAEI8VHurFz80FFbY0wNL8iHdvFlk7M8h4MomLAZxxGVgAABCvFR7KzAj0Id1Olk4n1wuvAZZue8y6TeRYGEmfERzA6QuBBQAAEajcaHF8rfAaYZlO0PQju1HEB3D6QmABAEAEKjfqh/o6uvWx6JHwgoquI0W63kfnTQAAYu1PJ9WR3JrV5PcDoRcRpDlE9oTyfG1mNfc+kX3bQ35v6ThUpNsoiRUaRQAA4FEN0vXRz8IKKtw5GHe8udJ4PUtBhXbKDDWoqFFP5C9TRS55SGLJcmCxbds2ufHGG6Vu3bqSmZkpp59+uvznP/+JzNkBABDlElMzfSnMGjdnnVFlYmr7Q2d9WK76UA6RG2aI3POjSJsrJdYsbYXs2bNHOnXqJN26dZO5c+fKcccdJz/99JPUqVMncmcIAEAMS0xD5fSYDxI0iVM7Y3rP+jCr450iLS6SeGEpsJg4caI0btxYXnnllfL7mjZtGvA5xcXFxs2tqKgolPMEACCmJaa2VZkcOyKy4kWRPZtE6jQROeeWEDtjOlxBRYy3PsIKLGbPni09evSQa665RhYtWiSNGjWSwYMHyy233OL3Ofn5+TJu3Dg7zhUAgIiwdThYoOoQLSH98mkRZ6nHfX8XaW1xC6NJF5EbZ4hUqSbxxlKOxcaNG2XKlClGy+2PPvpIBg0aJEOHDpWpU6f6fc6oUaOksLCw/LZ161Y7zhsAANuEWhqaVb1Kectub46y2SLl80HcfSk8gwql36+dIVKtpkcD8AAJmtdMFRn4flwGFZZXLEpLS+Xss8+W8eNdI1rPOussWbNmjTz33HMyYMAAn8/JyMgwbgAAxCu9+GsQoLNArORZXPOnE+SVLzYZ4YAz0HyQY0dcKxWBHDlY9irer1bmgvtFzr8nJr0pIrZioZNHW7duXeG+U089VbZs2WL3eQEAEDV68dcgQFlpvn1x6zyZcmM7Yx6Ip7yc6sb95fNBNKfCe6WiEqfIGTeIZDeo3EGz7+siF4yI+6DC8oqFVoT88MMPFe778ccf5aSTTrL7vAAAiGpVSE5mNflbpyYyc9U2+f3AUdNj0DUo0QBDE0B37jtsbKtUmg+iiZpmZNQUGbbGVSWiCZ06lVQHiCVAQBFSYHH33XdLx44dja2Qvn37yvLly+WFF14wbgAAJGr/Ci019awK0c6bZzbOkQXf/xZ8m6NsxaO8pLS0RGTzkoqBgVZ/mKHHaRDRtIskKofT6bRUtvv+++8bCZnav0JLTYcPHx6wKsSblpvm5OQYiZzZ2dmhnDMAALY2xfK+ELqDh1vPbyqzv91eIehoEGgM+rrZrkZXnj0pdGz5Jf8QefemwNshjnSRB3bEbVKm2eu35cAiWicGAECktz86T1zgt3+Fe7tj0b3d5OvNe/xvc7hXKRY/JrJwvJ9XEpGWl4r88GHgGR9x1pMilOs3Q8gAACkpWFMsd+dMDSr8ds50BxTLnhU5vDfAKzlEtn8r0mGI61jPlQtdqehwR1wHFVYQWAAAUpLZplh+j9NtjzlDRQ7tMfEqTpGibSKn9BS5aEzlzptxuv0RCgILAEBKMtsUq9JxAbc9gtCETg0idIUiSRFYAABSkpmmWHVrVpM/nVSn4irF3PtCH21e63hJdpbHpgMAkCpNsXYfOCLdHpkvyxfMEpk3SmR6vxCDCoer0ZWWniY5AgsAQMrSklFfnTPdLk37SmYX/03OXTzAlXQZ1ptNSKhGV6FiKwQAIKkeXBw75pQ7p31TYUtkZPqbcluV98Vhpce3L7pSoUFF6yskFRBYAAAk1ZtkDZn2Tfn3VeSY5Fd5Uf6S/nn4L35BYgwOsxOBBQAgpZtkaTtvlSalMqnK03J5+jLx7n9lWWauSO/JKbNK4YnAAgAgqd4kq2faMvln1SmS6Qg8fCyozDoi5w1KuVUKTwQWAICUpc2vbMulSMFtD18ILAAAKavV7gVyRZX3w3uRFEvODIbAAgCQmkpL5JSvx4S+UqFbHq16uXpTpPgqhScCCwBAatq8VBwHd4f23DifRBpLBBYAgNSkczusqlFP5LLHRdpcGYkzSgoEFgCA1GR1bsdpV4lc/RLbHkEQWAAAkqovhZaQarWHTiXVQWM6E8QnzY3IbihS9EuQV3WIdLyTrQ+TCCwAAEnTQVObXWlfCjedXqqDxrRtdyW68tBzosj0/iL+5ps26SJy4wzXqHOYwhAyAEBSBBWD3lgpvxYelPZp6+SKtKXG152FB4379XGftES072uulQvvXIprpooMfJ+gwiJWLAAASdGW+5K05TKm6mvS0PF7+WO/OHPlwaP9Zdyc6nJx6zzf2yIaXGjZ6OalroROzb2ghDRkBBYAgISmORVt9y2WKVUnVXosT36XZ6tOkkH79LgzpUPzur5fRIOIpl0if7IpgK0QAEBC21l0wFipUN4LEu7vx1R93TgOkUdgAQBIaCcfXG1sf/gr/tD7Gzp2G8ch8ggsAAAJ7dSsg7Yeh/AQWAAAElpaVp6txyE8BBYAgMRW1ujKqY2sfDDu1wmkehwijsACAJDYyhpdaVjhHVzo98Y9Otac8tGoILAAACS+skZXjuyKHTYd2vhKG2Dp44gK+lgAAJIDja7iAoEFACB+lJaEFxjQ6CrmCCwAAPFh3WyReSMqThvVrQwdFMZWRsIgxwIAEB9BhU4Z9R5hXrTddb8+joRAYAEAiP32h65U+BxdXnbfvJGu4xD3CCwAALGlORXeKxUVOEWKtrmOQ9wjsAAAxJYmatp5HGKKwAIAEFta/WHncYgpqkIAADEtIS1p3EF2SV05zrnb54TSUqfITkddOa5xB6EjRfwjsAAAxLSEdPnmQnn1SD+ZUnWSEUR4Bhf6vRpzpJ8M3FwoHZrXjfZPAIvYCgEAxLSEdOe+w/JR6bky6Ogw2SG5FQ7dIXWN+/VxPQ7xjxULAEAMSkgdrhLSVr2kflZ1414NHj4pPlvOTfte6ste2Sm1ZXlpKykt+x3YfRziG4EFACCmJaTnNu0sDXKqy47Cw0YQsay0dYUjdWckL6e6nNu04moG4hNbIQCAmJaQpqc5ZExvVzDhnbvp/l4f1+MQ/wgsAAAxLyHt2aaBTLmxnbEy4Um/1/v1cSQGtkIAAPbTklKt/tBETZ95Fg7X43pcGQ0eLm6dJ8sLfjcSNTWnQrc/WKlILAQWAAD7aZ8KLSnV6g9jQ8MzuCgLFHpOqDQSXYMISkoTG1shAIDI0D4VfV8TZ3bFbQynrlT0fY1R6EmKFQsAQMTMKz1HHjo8WRof+ba8hHTr4TNkdOnp0jPWJ4eIILAAAETEvDXbZdAbK41NkG3yRwmpo+iocT9JmcmJrRAAgO1KSp0ybs46v+2xlD6uxyGFA4uxY8eKw+GocGvVqlXkzg4AkJC0smN7of8W3BpO6ON6HFJ8K+S0006T+fPn//ECVdhNAYBUmURqltm5Hsz/SD6WowINJPLy8kwfX1xcbNzcioqKrL4lACBOJpGaZXauB/M/ko/lHIuffvpJGjZsKM2aNZMbbrhBtmzZEvD4/Px8ycnJKb81btw4nPMFAMRwEqlZ2thK53/4a22l9+vjzP9IPg6n02k6c2bu3Lmyf/9+admypWzfvl3GjRsn27ZtkzVr1khWVpbpFQsNLgoLCyU7O9uenwIAEPrWx6YlIm/3Fzm0189BZV0yh622tC3irgoR3+2xqApJMHr91gWCYNdvS4GFt71798pJJ50kTzzxhNx00022nhgAIMIBxeLHRL56NkBA4WXA+yJNu1h6Gw0utPrDM5FTVyp0qBhBRWIxe/0OK/Oydu3acsopp8j69evDeRkAQDStnSUye4hI8b7ITCz1wPyP1BNWYKHbIhs2bJB+/frZd0YAgMj5eLTI0icjO7HUC/M/Uoul5M177rlHFi1aJJs2bZKlS5fKn//8Z0lPT5frrrsucmcIALDHmlkhBhWaY9GowiRSwJYVi59//tkIInbv3i3HHXecdO7cWZYtW2b8GQAQ5zkVHw4P4Yn+J5ECYQcW06ZNs3I4ACBeaNOrg7stP00nkTo0qGASKUyibSYAJNvKRMHnIpuXuGo8tYqjSWdLiZc6vqNQasngo0Nly+GzmEQKSwgsACBZaBOrOUNFDu35477PHxXJzBU573ZTL+FuQDDy6M3yZWkbJpHCMqabAkAyrFIsnCgyvV/FoMLt0O8iC8eLZNYJ+lJ7pJYMOjpMPio91/ieSaSwihULAEj0VYoP7xXZv8PEwY6AKxWzS9rL3ceGSKnX75yek0gpG0UwrFgAQELP9+hnMqgoW7m44H45lFlxkOQuZ7YMOjpU7jo2tFJQ4YlJpDCDFQsASNTtD82nsOjbg7ly1Z7H5Jy076W+7JWdUluWl7YKGFC4MYkUZhBYAEAi0soPX/kUQTz/zUEpkTRZVtra9HN0AyWPSaQwia0QAEhEWk5qUXGNBjJvXzNLz3FnZejQMOZ7wAwCCwBIRCEUaHzbZqSpLQ9PulJBqSmsILAAgERkZXy59rHo+7qUtOxt6S1G9zpVloy4kKAClpBjAQCJSLtpasCglR7+pFcXuW6aSLPzjTkf55Y6pUFOdaN01Ix6WRlsf8AyViwAIB5bcq9+x/VVv/dFB4L1nhz4ta5+UeTkbuXDwzRIuOIM86sPVIEgFKxYAEA89aWYN0Kk6Jc/7stuKNJzou8hYHpf39crPUeTNDWfoiSjk7FK4V51mLdmu7ywuCDoaVAFgnA4nE53Z/joKCoqkpycHCksLJTs7OxovjUAxHmzq/4+sjLLtiL6vuZ/wqiuamxeKt9+971RTqqVH+4kTd360IqOi1vnSeeJC4Jug7g3PkjYRKjXb7ZCACDWNDDQVQefpR5l980b6XdbRPtSTN6QJ30WN5QP951cofJDA4nb31gpTy9Ybyq3IrdmNYIKhIWtEACItc1LK25/VOIUKdrmOs6rGkS3N8bOXis7iooDvsXzizeYOpW/9zqVoAJhIbAAgFjb/2tIx2lQoSPNzexnHzziJwnUS15OprlzAfxgKwQAYq3W8ZaP0xHmOsrcSpKcI8hjmo9BwibCRWABALF2UkdX9YffS79DJLuR67gyOsLcbD8KN3cQ4v0utO2GnQgsACDWtM+ElpQGuuz3nFDejyKcEeY3dWpilJJ6om037ESOBQBEglZwbFoiUrBIZO/PIrVPEGlyviv50iNAqNiT4jU/fSwmVCo1DbV5VffWeXJ/r9bGiocGJ/o6uv3BSgXsQmABAHZbO0tk9hCR4n0V7//8cZHMOiK9n/Tf8KpVL1f1hyZqak6Fbn/4CET+dFId0Vig1GSShWfTKw0iOjSvG+pPBwREYAEAdvp4tMjSJ/0/fmiPyPR+ro6ZvoILDSJMDBj7evMeS0GFIocC0UCOBQDYZc2swEGFp7kj/M8BMcFKjgU5FIgmViwAwA4aJHw43Pzx+37x2fBKy0jN5D+YzbHQ0ecDOzVlpQJRQ2ABAKEqm9Fh5EPo7eBua8/30fBKe1N4lpG6Z314rzZowKGP7Sg87LOXhTungqAC0UZgAQB2TSK1yqPhlb8umho46P3eWxkaLGjAoY9p2OD5PHIqEEvkWABAqJNIwwkqshqWN7wK1EXTfZ8+rsd50kBDAw76UiCesGIBALZNIrXg0onlZaTBumjqO+njepx3magGDzoSnb4UiBcEFgBg6yTSIDJzRXpPrlBqarbCw99x9KVAPCGwAIBITCL1VCVT5OSLRc65yWfnTbMVHqF22wSiicACAIJVfHh2wDQ7ibTHeCmtWV++21dD1tc4Xepn1/S7RWG2woPJo0gEBBYAYKbiw5jZMdHVclv/XLTdT56FTiJtKPNq9ZFx7/9QljuxOmDpKBUeSCYOp9MZZgaSNUVFRZKTkyOFhYWSnZ0dzbcGAJMVH/38P66tuJVWhRgqhwHfdJgsV31Wr1LY4Q4L/FVsWOljAUSb2es3gQUAeG5/PHqyyKHfAydf3rte5PsPfKxqNJKSHvnSeXYtv1Ue7m2NJSMu9LkCYbbzJhBtZq/fbIUAgJuOOQ8UVCh9XI/zM4l0ecFe2V64LKTSUUWFBxIdgQUAuBV8bv64Zl19TiKdv26H7UPEgERCYAEgNfmq+jC74+DnOM2R+NcXm0y9BKWjSFYEFgBSj7+qj7PcCZlBnNS50l3uttzBUDqKZEdgASA153x412xo+eiiCSLVaokc2R84edNr+8NMW243fVdKR5HMGEIGIHUEnPOh9zlE0qsFfg1tx+3VOdNKzsTfOjWhdBRJjcACQOoIOufD6ar6uOB+kawGlaeRag8LjxkfoeRM6MAwIJmxFQIgdZIzzc75qNtc5O61vlt6+0FbbsCFwAJA6iRnthto7vkaSPgoJQ0kUFtuKfv+f85pbPr1gETFVgiA5FqlWDjR1ZLbe8tDkzMX5otk1vFfL2rM+WjkWp0IgeZOaLtuXZnw5Z/zf5LOExcYZalAsiKwAJA8qxT/PE1k4Xg/B7jXENzrCd7BRdn3PScE3PIwE1xou+67u5/i83HdKtFVDYILJCsCCwCJvUKxcZHI//V3rVLsC3ax9kjOzPZKztStkr6v+U3OtGraii3+zsCgPS+09wWQbMixAJC4KxSzh4oc3mP9uZqcOWyNpeRMK4L1tAg2LwRI2RWLCRMmiMPhkGHDhtl3RgAQzNpZrhWKUIIK7+TM0//i+mpTUGGlpwXzQpCMQl6xWLFihTz//PPStm1be88IAAJZM0vkHZPVHT6TMxuGnJxpltmeFswLQTIKacVi//79csMNN8iLL74odepohjUARGn7450BfjpnmhRmcqYnzZH4csNueW/VNuOrO2diz4EjEqhjtz6kPS/oaYFkFNKKxR133CG9evWS7t27y8MPPxzw2OLiYuPmVlRUFMpbAkh15e24Q6RlpBpUmEzO1CBBcyB0u0JXFjQI8JzvoVUdmoDpmUuhwcIVZzSQFxYXBA19mBeCZGU5sJg2bZqsXLnS2AoxIz8/X8aNGxfKuQFIdZ7dM/UWsB13AFoFcv49plcqNGgYO3ut7Cj645eivOwMGXvFaUY5qT6uJaPewYMGGc8vLgj42hpLPH3dWcwLQdKyFFhs3bpV7rrrLvnkk0+kenVze4OjRo2S4cOHV1ixaNyY7nMAQuieGeFVCqVBw+1vrKx0vwYZev+z158lD33wXcibMbpbUqdmRojPBpIssPj6669l586d0q5du/L7SkpKZPHixfL0008bWx7p6RV/I8jIyDBuABD2aHMruo4U6XqfpXwK3f4YOWN1wGPuffe/cqC4JPTzohoESc5SYHHRRRfJ6tUV/0/317/+VVq1aiUjRoyoFFQAgOVtD21yNW9keEFFx6Ei3UZZftqyDbtl78GjAY8JN6hQVIMgmVkKLLKysqRNmzYV7qtZs6bUrVu30v0AENVtD5WRJXLF0yKnXRnS07/cuEsiiQmnSAV03gSQ+NseOljsvEGWEjR9M1elUSsj3Vi5CHTG3hNO3a9MNQiSXdiBxcKFC+05EwCpt/VR8LnInKGhBRU9xrs6aNrYjlvbaz/92fqgx93cuZlM/vQnv8HDrec3ldnfbq9QiqorFRpUUA2CZMeKBYAE2/oo65553u22tuFW7ZvVldo1qgbMs6hTo6rceVELadUgq1IfC8/g4b6epwbsgwEkKwILAAm09WHPaHN/9MI/4arTfZabuuVfdbpxnAYPF7fO8xs86FcGjCEVEVgAiN7Wx6YlInPuDD2fQlcqLPalsEoDhudubCdjZ6+THUUVu2p6b2UQPACVEVgAiOOtD4dIjboiPfNFshrYOto8kGCrEQD8I7AAEKdbH2UX8cv/GdEVCn9YjQBCQ2ABIAqDw5xR3/YINkQMQGQQWACIHO2kaXX7Q3tSXDNVpEnnkLc9/E0epdwTiLy0KLwHgFSlE0lN09UEh0jvJ0WadQ0rqNDJo55BhdpReNi4Xx8HEDkEFgAiR5tXWdn66PtaWPkUuv2hKxW+Nl7c9+njehyAyGArBEDkaBWHBgxFukrg52KemSvyl1dEmnYJu+Jj2cbdlVYqPOkZ6OOae0FiJhAZrFgAiBwNFHpOLPvG4WfrY7JI8wvCDip0i+OOf/tvbOWJseVA5BBYAIgs3drQLY7sBrZvfXjnVew9FHjkuZtWieh2yJcbdst7q7YZX9keAezBVgiAyNPgoVUvV5WIJnTaODgsUF6Fv7Hlew4US+eJC6gaASKAwAJAdGgQoXkUNtKg4uUlBQHzKrxdcUYDuePNbyoFIu6qkSk3tiO4AMLAVgiAhKTbH+0e+lj+8eF3po7XqaXPXN/OGGdO1QgQOaxYAIhb/rpnalARaAKpL89c107S0hxUjQARRmABIC6DCM2DeOiD7yrlQYzudarcP3ON6dd151W0b15X3v+vuS6gVI0AoSOwABBzvlpw+6J5EIPf/Mby62tSpq50aMBihtnjAFRGYAEgptylomayGqxmPmhexYSrTi9PxtStFF310ADFGWB1Q48DEBqSNwHEjJVS0VBoXoVnhYeuWujqhb92XZ6rGwBCQ2ABpPJI84LPRVa/4/qq30eZ5lRYKRW1ok6NqkZehTcNNLSkVFcmPOn3lJoC4WMrBEhF62aLzBtRcaS5dsLU9ts2dMI0K5JJkv+4so3flQcNHi5uneez4gRAeAgsgFQMKqb3r5yxoIPC9H6b2mxHMklSL/+Btk9uO7+pXNa2YcDX0CCCklLAfgQWQCrQbQ5tp71vu8i8UX4uy3qfQ2TeSFf7bRvabQcTLJnSH3/H1sqoIo9c3VYua8t2BhAr5FgAyW7NLJHHWohMvVxkxi0iB3cFONgpUrTNFYREQaBkSl+C7VTUykiXHm3y7Dk5ACFhxQJIZh+PFln6pPXn6aCwKHEnU3r3sXA1w2otdWpWM/Igdu1zNcwKZEdRMV0zgRgjsACS1dpZoQUVSqePRpGZZEodb24GXTOB2CKwAJI1p+KD/xfCEx2u6hAdaR5lwZIp6ZoJJAZyLIBkpDkSAXMpfClbHeg5ISqJm6EmevpLs9D79XG6ZgKxRWABJGOjq1ByJHSlIoqlplbRNRNIDGyFAMnY6KrdQHPPz8gW6fW4SFYD1/ZHBFcqvKeX/umkOvL15j2WGlT5S/TUrpkaVNA1E4g9h9PpjFSbfp+KiookJydHCgsLJTs7O5pvDaRGoyt3+6jMXJFDvwd+jb9MFWlzpcRieqnGEKUep97AQnDgHaTQNROIn+s3KxZAom59zBkauNFVsB6VHYdGLajwNb3UM6hQ2iRLjzMzr4OumUD8IscCSLRVikltRF7vI3JoT4ADna7VigtGubZGPNWoJ3LNVJFLHoqr6aXuY/R4fR6AxMSKBZDwWx8B1G0uMmyNq0pEEzq1P0WEcynCmV6qP5keT5MrIHERWACJsv2hSZqWJmqUNbrSIKJpF4mFUJtV0eQKSFwEFkAi0BUHz8qPOG50ZUezKppcAYmLHAsgEVjqSxE/ja6CNbXyRpMrIPERWACJwMrsjjhqdGVleilNroDkQGABJALd0jCqOwJccDPriPSfLTJsdVwEFd5NrbSJlSfv2EEfN1NqCiC+kWMBJALd0ug5sawqxLs3RdkVuveTIs26SjzyNb00lM6bAOIfnTeBhG/h3ciVTxFHqxQAkg+dN4F4Lh0Nta+EBg+tesWsLwUABENgAcTD0DDd5jC74hDDvhQAEAzJm0C0O2d696Mo2u66Xx8HgARHYAHEvHNm2X3zRrqOA4AExlYIEI18ioJFQTpnOkWKtrmOtXmbgxHjAKKJwAKIZj6FrR02zY0s12mhnoPAtLOlNqGiXwSASGArBLB7haLgc5F5o0Sm97M438Nih00TQcWgN1ZWmi66o/Cwcb8+DgAxDSymTJkibdu2NepX9dahQweZO3eu7ScFJOwKxaQ2IlMvF1n2rMUn69CwRrYNDdPtD12pCJDRYTyuxwFAzLZCTjjhBJkwYYK0aNFCtK/W1KlTpU+fPvLNN9/IaaedZuuJAQm1SrH4MZGF40N8AfuHhmlOhfdKhScNJ/TxV78okHpZGeReAIifzpu5ubny6KOPyk033WTqeDpvIulWKebeJ7IvjG2FCHTOfG/VNrlr2ipLzyH3AkBMO2+WlJTI22+/LQcOHDC2RPwpLi42bp4nBiRVXwqfGw4mdLnXNdsjAp0zdQXCKnfuBYPAAEQ1eXP16tVSq1YtycjIkNtvv11mzpwprVu7xiL7kp+fb0Q47lvjxo3DOmEgLrY+Ni4SmXNniEFFWT5Ft1Gu0tIItOPWbQ1dgbCysUHuBYCYBBYtW7aUVatWyVdffSWDBg2SAQMGyLp16/weP2rUKGPZxH3bunVruOcMxD5B87UrRA7tjYt8Cl80V0K3NTze0RR37oXmaABATHIsunfvLs2bN5fnn3/e1PHkWCBltz5iMInUVx8LMyb/z5nS58xGETsvAIknatNNS0tLK+RQAKnXktuE9oNFWl5mSz6FlU6amitxceu88uN37SuWhz74LiI5GgBgObDQbY1LL71UTjzxRNm3b5+8+eabsnDhQvnoo4/4NJHctNW21WZXEVihCKWTpgYdHZrXLQ9KXlpSYCRq+gqRNDzJy3EFKwAQ8cBi586d0r9/f9m+fbuxHKLNsjSouPjii0N6cyDuZ3xoi23thhlKOekF94ucf49tuRTuTprOMKo53LkXerwGEZ6v5V7z0MfpZwEgZjkWVpFjgbh27IjInGEi62aKHD34x/016ooc3B2zPApdaeg8cYHfXAn3SsOSEReaCgqYIQIgbnMsgKTx8WiRpU/5zqMwE1Rk5or85ZWIlJCa7aSpx7m3PQLxzr2g8yYAuxBYILXplsemJSILHhL5eYXJJ/nZROg9WaT5BZE4S+PiH85x/hI+zQQhAGAFgQVSu3x0zlCRQ3usPc/YFtn1x/fZDSNeQmq2SsPXcWx7AIgmAgukcE+Kfsa6g+XF/575IlkN/kjstFhCaqVc1LuTptVqDjsSPgHACgILpOb2x5yhoQUVSoMKzaMIIYjYc+CIPPSB9dWDQNUcUvb9ZW1cORPuQCXY6HR9HX1ccy3IrQBgF6pCkHp0zoe25A6FVnwMW+13hcIzkNi064C8tXyL7CgK3EDOfUk3s3rga1tDYwLP0R7uQCUns5pc9+KyoD/SW7e0J9cCQFBUhQD++lL89HHorxNgxkeo7bOtrB54VnN8sm6HvPzFpgpBhec2x187NbE1MRQAzCCwQPIHFIsfE/nq2RCHhpWpVkvkyil+EzT95TKYZaVcVAMP3e4YPn2V39fS0OS9VeY6hdK+G4CdCCyQ3AHF0skiRw6E/DLGRfq0q0Sufing9oe/XAarzK4emOlrsfvAEcmtWVX2HDhK+24A8Ts2HYh7a2aJTDhRZOH4gEGFv0BAs470NsdxgZTev1PkmlcCVn0Eu8hbYXb1wGwA8ueyCaXemyu07wYQKQQWSL7ume8MEDmyP+ihgS6nzx+7XKpe/ZykV8uISo6Coyzp0uzqgdkApHvrPCMpVFcmPOn3lJoCiAS2QpA81s4SWfpkWC+xy5klj1e5TbpcfbNRVfHeqm1Be02Em6MQyuqBlb4W+pq07wYQLQQWSJ6cig/+X0hPLblkvGw4WEN2OmtLepNO0uXQMUu9JoJd5IPJC6ELptUppbTvBhAt9LFAcij4XGTq5WH3pfBX3RGo14Qmbz69YL38c/6PQd9OA5DRvU6VOjUzbFk9oF03gGihjwVSi/anCLMvRSidKoP1rtCL/P+cc6I0qVcjIlsQTCkFEG8ILJAcdGaHFTrivPdkKWnVW5Zv2G1clHftKzY1mnzZht2SluaQ+et2yL++2OT3+Lu7t5AhF7aI+EWebQ4A8YTAAslBB4HplNGiIE2hqtYU6XSXyPn3yLx1O2XcxAWWS0XveHOl7D10NOAxGkpMW7HVCCwAIJVQborkoNsZPScGLiLVRlejtopcMMIIKjSXIpT+E8GCCu9OmgCQSggskDy03Xbf11wrF55q1BO5Zmp5oys7O2UGwxwOAKmGrRAkxuAwzaHQ7Y4AHTDLg4tWvQI+z85OmcEwhwNAqiGwQOIMDtOVCN3u8DMIrJwGEU27+B1n/tOvwbtyhos5HABSFYEF4su62eKcc5c4DvnITSjaLjK9v2u7I1hwYcM481AxhwNAKiOwQHwFFRo4+B8P5rpszxvp2u4IMG3UvTpR8Nt+mfTpeltOr2ZGuhwsLgmamxFKJ00ASBYEFogPpSVyaM69Ut3pFEfAX/KdIkXbXDkUXtsdkV6duKVzM5n86U8+W2jr93/r1MRoVkWDKgCpjMAiDnn+xp3UnRQ9kjNL9v0qmYd2BB45GqTTpr923HaoU6Oq3HlRC2nVIKtS4MIKBQD8gcAizqTM7Id1s0XmjShvaBWk1iNop81Il5DmX3W6EdzRQhsAAiOwiCP+fuPWqZl6v68BWAkbVATMpfDPyLLQwWFaQuohUiWkvoI6WmgDgH8EFnEilAFYCdmPouZxInPv8xtU6KxdfzkWxhxeR8XBYZFqRFU7s6o8c0M7ad+sbuJ93gAQQwQWcSLYb9yeLaIT6rdlrfSYO0Ic+4LM8CgTKHFzr6OWZF/zrKT7KDW1qxGV++0nXH26dDq5ni2vCQCphJbeccLsb9wJ1SLaKB/tJ2IyqPBnj7OWPH70L/LV1V9J+ml9fB6jeQ66bWGWLkLc0qVJpedoImbSbDkBQAywYhEnzP7GnRAtonXro+BzOfbubZIeYGsjkAeP3ii7nLVlp9SWrbXOkNHXnB7wYq/bFaN7tZbBb6409fpPX9dOLmvbQEZe2ppETACwEYFFnHD/xq2Jms5EbhHtUe1h/OWyeI0udYrskLpyQo9hckZ2DUsX+zo1q5l6j7u7n2IEFYpETACwF1shcUIvcFp9oBxx1CJak0q/3LBb3lu1zfiq3wet9igrIbXK/dLjjvaTutk1pM+ZjYyLvtmf2ew2UZN6NUI6PwBAcKxYxBFd6tf9fTMNmOxuouXr9T5Zt8N8Tw3d/tCVijA6SehKhQYVH5WeKwND2PJJqu0kAEhQBBZRphfwZRt3G7/960W4Q7N60t7jt3IzDZjsbqLl6/Vq16gqew8erXSs354aWk4awkrFQ0evl9+cuUYuxfLSVuKUNONn0Z/ZavCUNNtJAJDAHE6n0R0gaoqKiiQnJ0cKCwslOztbUolewEfOWF3pgq0X8QlXBU5ODNZEy325tVrREEobbPcFesmIC/+40K9+R+Tdm0y/hv6t2yO15Ozi56S0bEfO82dQoQRP7p/HeA+vc3a/NhUfABC56zc5FlGiF7zb31jpcxVA79PH9JhAOQ3BmmjpbezstYHzIGxog+3ZU8Nfi+2Az3e6XuPhtNvLgwrPUk+lwYF3Xw/3aol+TsG2k/S1PFFGCgDRwVZIFOgFfOzsdUGPG/Huf43jdhT5/i3dTNvqHUXF8vSC9XJX9xbl763P09f8fX+x5NasJnk5mcZ2QLhtsCskS2qL7eyGIkV60Q8cqvzqyJXtHcbKoxf3l2u8tjpU54kLwupAyjwPAIgdAosocF/Ygyk8dMy4+ctpKD5Waur9/jn/R2mZV8v4s78R4hqwXNYmT8JRIQlSW2z3nGhUhTjFIQ6P0MD9pw3N+0txs57S6rweklfF9VfPu9RTV2ns6EBKGSkAxAaBRRSE0y3T87f0x645w/TzNJej8OBRv2sHenH+1xebgr5OmpTKuWnfS33ZWyHB0mcSZOsr5JsOk6Xhl+PkeNHkVJdfpa5s7zBGzuoxIDU7kAJACiGwiIJwyxvdv6XrH3Slwcz2ha9cDl90d8Cd8+CtR9pyGVP1NWno+COX4hdnrjx4tL9c2fv2SlsLRuLkZ/XEIZMrBCMrSltJ6WdpMqXR9qA5DpSMAkBiI3kzCvQ3+7zs8C+Euw4UlzfRsovmebpXRTz1TFsuU6pOkjyPoELp91OqTZaeaSv8JoJqQuay0tYyu7Sj8bWk7K+ZPh4ssdRdMuovG0Lvd5ejAgDiT0oGFpa6SdpAf7Mfe0X4AYH+lq5JiTUzKo4MD9ffOjUxtjZ026N92jrpk7ZExld72Zjx4f0XRL83LvrzRrqaYlmczvrqFwUBP+947UAKADAn5bZC7G4uZZa+9nM3tvPdxyKzihwpccrBI39cqL1prwt3JceBYv/HhUKDlV7pX0nT5WMkV4pMPMMpUrTN1RSraRdLOQ8PffCdvLSkIODnbaUDKQAgvqRUYOGvGZTfbpI2c5dBenfePKdprpz54McBn3u0rCLE7qRFDarqL3tYmv3oWqGwZP+vIeU8mPm8KRkFgMSUMoFFsOZSZvoj2EFfu9PJ9Yyb2xc/7Qq4WqEOHCmRZRt225q0qD/l02dskWbLXw7tBTyaYgVrpx3K503JKAAknpTJsTCbA1Chm2SUfLlxl+njgiU3mqWvMeWGM6Tttw8ar2VttcIhkt3I1RTLRG5EvH3eAIDISZnAIr77I5i9qjsqXMBDlVuzqiy6t5v0rFUgVYt/D+1ce05wNcUy0U47EPpRAEBySZnAIlb9EcxUoJhd7ncf576Aa4Bglrvi44q0pXLKoW/l64JdFXIkTNO23X1fM5ph+aLnpsPJRvc61dTL0Y8CAFI4xyI/P19mzJgh33//vWRmZkrHjh1l4sSJ0rJlS4l30Rqp7Tnqe9Oug/LW8i1+Z3+4aYdM3YoINGe2To2q0r7ZHwGIPv/CVsdL+/xP5fcDRwKek69GV4fefVGkvblppEVSQ2r9eZKkaVCh2x9eKxXedFVlYKemRvUHI8wBILVYWrFYtGiR3HHHHbJs2TL55JNP5OjRo3LJJZfIgQMHJN7Z0R8h2OqDVp3oAK3rXlwmd01bZczs8J4R4j2hU7/e8ebKgEGFyr/q9ErnVq1Kmoz/cxtXjoSfVYqh6e/Kc1UnSQOpuOVR/fBOkYXjRTJz/SZbGh05nSIb2o+XtDOudZWWBgkq3OhHAQCpyeF0Bruk+ffbb79J/fr1jYDj/PPPt3Wee7z1sQj2PH+lrBLgt3XNc+j66GcBk0r1uvv0dWfJZW0bWjo3XaXIr/qi5DoCBX0Okcw6Iof2iNMYHVaR/iwFp9wkza5/wsRPFV99QwAA9jJ7/Q6r3FRfXOXm+l/OLi4uNm6eJxZLofRHCNb/4pnr28lDH/guZQ1UEfH6l5uCzv3QRZE6NTMs/Uzp38+Ry76bZCIl1Cly6HeRC+4Xx8pXRYp+KX/kSEZdSe/9uDRr82eTP5W5c6MfBQAkt5ADi9LSUhk2bJh06tRJ2rRpEzAvY9y4cRJPrPRHMNP/YvR7a2R3kDwHXzb/ftC2yol0KZUOaetE0raLbJ4oTof5WhOp21xk2BpXJ01N6Kx1vFQzkUthFv0oACB1hBxYaK7FmjVrZMmSJQGPGzVqlAwfPrzCikXjxo3FTp4Jk3b/Rmym/0UoQYU6KbeGPZUT62aLzBtRYcXBYbXRlQYRZe25AQCIamAxZMgQef/992Xx4sVywgknBDw2IyPDuEVKpPfwI9FnwZ1j0a9Dk/AqJ3QI2KJHRBZNCP1kvBpdAQAQtaoQzfPUoGLmzJmyYMECadq0qcSSO/fBe0XBu/IiHGb7LGhPCSurBBr4aFVHyJUTa2eJTDgxvKDCT6MrAACiEljo9scbb7whb775pmRlZcmOHTuM26FDhyTaguU+KH083JHowVpo6/36+MN9XHkmwYILjRFuPb9p+WqKv26V+r3fIV0fjxZ5e4DIkf2h/VDGiaaL/GWq30ZXAABEvNzU4WegxCuvvCIDBw6Marmp9pHQfhHBvHVL+6CJg8FyNNwrI8rzw3If4Q4AfG3LePN+jtlzKLdmlsg7AyRs10wVOe3K8F8HAJASiiJRbhpGy4u4nf1hJkfDvargfVyej+OMsegbdhtNr/YeOmp6sqepygnNqfjwj0TYkGTmivSezEoFACAiqqTy7I9g/Sk8VxXM9mPQ79PSHD6DCl+TPS2VYWo56MHdYklmXZGz/+oKZ7Tqo0lncioAABGTsIFFuLM/zPSnCGlVIZKTVEMZGtZ7EqsTAICoSdjppuHOojDTn8K9qhDV1RTd7ti4SOTTh0UWPCyyYaHrPne/CbNIzgQAxEDCrlhYyX2I6qpCOKsp2uhqzlBjdscfHnXN8+j9pEirXq6x5R6NsPz6y8skZwIAoi6hA4twZlHYkaMRbDVF8zQcfipJKq2mBKr20EBjej+Rvq+L9JwoMr2/16t6qFZL5MoprFQAAGIiYbdCPLlzH/qc2cj4aqadt9n+FH67XgZhqT+FNrt6x0S57twRrlWLvq+5Vi48VcsSOX+kyMgtBBUAgJhJ+BWLUIW0qhCJ1RTd/tBmV2bs+8VVGaKBgwYYHkPDjLbcVHsAAGIsZQOLcHM0zApYSaJJmTo8LJTKEIaGAQDiUEoHFuHkaIRMg4lNS0QKPhcp3GIuEdOTlcoQAACiLOUDCyv9KcJmVH3cJXLIeglreR4Fk0gBAHGMwCLSdIVCcyF++FBk2bPhvdYVT5JHAQCIawQWkV6hmHufyL7wx7dLy8tE2lxlx1kBABAxBBaRDCq094QdOtwp0uNhe14LAIAIIrCIxNaHJmbOuj2816lWU6R1H5HLJ4tUqWbX2QEAEFEEFnavUmj5qNVKD09d7hVp1pW+FACAhERgYevWR4BW22ZkNRTpNoqAAgCQsJKipXfMlTe6CiOoUJdOJKgAACQ0Ags7aDlpONsfmbmuAWPM+AAAJDi2QuzgbrNtRZVMkQ6DRZqc72rNzUoFACAJEFjYIZQ221e9wAoFACDpEFiE0kXTe6KoftUx5kXbg+dZZDcS6TmBoAIAkJQILMIpJdVgoudEV5CgX42qEO8h7GXaD3Z1z6SMFACQxEjetFJK6p2gqSsUer8+rsFF39dEshtUXqHQxMye+eRSAACSHisWYZWS6n0OkXkjRVr1cgUX+tXXdgkAACmAwCLsUlKnSNE213HuFQn9CgBACmIrxK5S0lBKTgEASDKsWASq9rBSShpKySkAAEmGwCJYtUfQUlKH63E9DgCAFJeW0qPN540Smd4vcLWHrlxokGHQUlJPZd9rXwoSNAEASMHAQoOFSW1Epl4usuxZPweVrUxotYcGIX5LSRu67qfZFQAAKbgVYmm0uVe1B6WkAAAElTqBRaijzT2rPSglBQAgoNTZCgl1tDnVHgAAmJY6KxaW+0xQ7QEAgFWps2JhaeWBag8AAEKROoGFux9FpZJRH6j2AAAghbdCAnXOdHP3o2C0OQAAEVMl6TtnenL3o6h0fCPXtgcrFAAAhMXhdDot1l+Gp6ioSHJycqSwsFCys7Mj1JeibLvD33aGmRUOAABg+fpdJTn7Uuh9DlfnTG1q5WtbhH4UAADYLi15+1J4dM4EAABRkZb0fSks968AAACpF1iY7UtB50wAAKImLXn7UmjnzEZ0zgQAIIoSN7Bw96UweAcXdM4EACAWEjew8OxLkd2g4v10zgQAICYSt9zUTYMHLSmlLwUAADGX+IGFoi8FAABxIbG3QgAAQGIHFosXL5bevXtLw4YNxeFwyKxZsyJzZgAAIPkDiwMHDsgZZ5whzzzzTGTOCAAApE6OxaWXXmrcAAAAop68WVxcbNw8p6MBAIDkFPHkzfz8fGPMqvvWuHHjSL8lAABI1sBi1KhRxux2923r1q2RfksAAJCsWyEZGRnGDQAAJD/6WAAAgNitWOzfv1/Wr19f/n1BQYGsWrVKcnNz5cQTTwz6fKfTaXwliRMAgMThvm67r+N+OS367LPP9BUr3QYMGGDq+Vu3bvX5fG7cuHHjxo2bxP1Nr+OBOPR/JIpKS0vll19+kaysLKNzZziRk1aYaDJodna2recI3/jMo4/PPPr4zKOPzzwxPnMNF/bt22d03k5LS4ufIWR6MieccIJtr6cfCH8Ro4vPPPr4zKOPzzz6+Mzj/zPXthHBkLwJAABsQ2ABAABsk7CBhfbGGDNmDD0yoojPPPr4zKOPzzz6+MyT6zOPevImAABIXgm7YgEAAOIPgQUAALANgQUAALANgQUAALANgQUAAEiNwOKZZ56RJk2aSPXq1eW8886T5cuXBzz+7bffllatWhnHn3766fLhhx9G7VyThZXP/MUXX5QuXbpInTp1jFv37t2D/jdC+H/P3aZNm2a0xb/yyisjfo6p/pnv3btX7rjjDmnQoIFRnnfKKafw70sEP+9JkyZJy5YtJTMz02g7fffdd8vhw4ejdr6JbvHixdK7d2+j9bb+GzFr1qygz1m4cKG0a9fO+Pt98skny6uvvhr6CTjj1LRp05zVqlVzvvzyy861a9c6b7nlFmft2rWdv/76q8/jv/jiC2d6errzkUceca5bt87597//3Vm1alXn6tWro37uicrqZ3799dc7n3nmGec333zj/O6775wDBw505uTkOH/++eeon3uqfOZuBQUFzkaNGjm7dOni7NOnT9TONxU/8+LiYufZZ5/tvOyyy5xLliwxPvuFCxc6V61aFfVzT4XP+9///rczIyPD+Kqf9UcffeRs0KCB8+677476uSeqDz/80PnAAw84Z8yYYQwNmzlzZsDjN27c6KxRo4Zz+PDhxvXzqaeeMq6n8+bNC+n94zawOPfcc5133HFH+fclJSXOhg0bOvPz830e37dvX2evXr0q3Hfeeec5b7vttoifa7Kw+pl7O3bsmDMrK8s5derUCJ5lcgnlM9fPuWPHjs6XXnrJmCpMYBHZz3zKlCnOZs2aOY8cORLFs0zdz1uPvfDCCyvcpxe8Tp06Rfxck5GYCCzuu+8+52mnnVbhvmuvvdbZo0ePkN4zLrdCjhw5Il9//bWxtO45vEy///LLL30+R+/3PF716NHD7/EI/zP3dvDgQTl69Kjk5uZG8EyTR6if+YMPPij169eXm266KUpnmtqf+ezZs6VDhw7GVsjxxx8vbdq0kfHjx0tJSUkUzzx1Pu+OHTsaz3Fvl2zcuNHYdrrsssuidt6p5kubr59Rn25qxq5du4z/0+r/iT3p999//73P5+zYscPn8Xo/IvOZexsxYoSxp+f9FxT2feZLliyRf/3rX7Jq1aoonWVyCeUz1wvbggUL5IYbbjAucOvXr5fBgwcbQbS2RIa9n/f1119vPK9z587GmO5jx47J7bffLvfff3+Uzjr17PBz/dTR6ocOHTJyXayIyxULJJ4JEyYYyYQzZ840ErRgv3379km/fv2MpNl69erF+nRSRmlpqbFC9MILL8if/vQnufbaa+WBBx6Q5557LtanlpQ0iVBXhJ599llZuXKlzJgxQz744AN56KGHYn1qSOQVC/1HMz09XX799dcK9+v3eXl5Pp+j91s5HuF/5m6PPfaYEVjMnz9f2rZtG+EzTd3PfMOGDbJp0yYj29vzoqeqVKkiP/zwgzRv3jwKZ55af8+1EqRq1arG89xOPfVU47c8XeqvVq1axM87lT7v0aNHGwH0zTffbHyvFX4HDhyQW2+91QjodCsF9vJ3/czOzra8WqHi8r+Q/h9VfzP49NNPK/wDqt/rXqcver/n8eqTTz7xezzC/8zVI488YvwmMW/ePDn77LOjdLap+ZlrKfXq1auNbRD37YorrpBu3boZf9ayPNj/97xTp07G9oc7iFM//vijEXAQVNj/eWuulnfw4A7qmJkZGbZfP51xXKKkJUevvvqqUf5y6623GiVKO3bsMB7v16+fc+TIkRXKTatUqeJ87LHHjNLHMWPGUG4a4c98woQJRhnZO++849y+fXv5bd++fTH8KZL7M/dGVUjkP/MtW7YY1U5Dhgxx/vDDD87333/fWb9+fefDDz8cw58ieT9v/bdbP++33nrLKIP8+OOPnc2bNzcq/2CO/husbQD0ppf5J554wvjz5s2bjcf189bP3bvc9N577zWun9pGICnLTZXW0p544onGxUtLlpYtW1b+WNeuXY1/VD1Nnz7decoppxjHa+nMBx98EIOzTmxWPvOTTjrJ+EvrfdN/GBC5v+eeCCyi85kvXbrUKF/XC6SWnv7jH/8wyn5h/+d99OhR59ixY41gonr16s7GjRs7Bw8e7NyzZ0+Mzj7xfPbZZz7/bXZ/zvpVP3fv55x55pnGfyP9O/7KK6+E/P4O/R97FlMAAECqi8scCwAAkJgILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgNjl/wOB6vqgznVLNAAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "n = 100\n", + "x = np.random.rand(n, 1)\n", + "y = 2.0 + 5 * x**2 + 0.1 * np.random.randn(n, 1)\n", + "\n", + "line_model = LinearRegression().fit(x, y)\n", + "line_predict = line_model.predict(x)\n", + "#line_mse = ...\n", + "\n", + "#poly_features = ...\n", + "#poly_model = LinearRegression().fit(..., y)\n", + "#poly_predict = ...\n", + "#poly_mse = ...\n", + "\n", + "plt.scatter(x, y, label = \"Data\")\n", + "plt.scatter(x, line_predict, label = \"Line model\")\n", + "plt.legend()\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "id": "248d8931", + "metadata": {}, + "source": [ + "## Exercise 4 - The train-test split\n" + ] + }, + { + "cell_type": "markdown", + "id": "1efd3376", + "metadata": {}, + "source": [ + "Hopefully your model fit the data quite well, but to know how well the model actually generalizes to unseen data, which is most often what we care about, we need to split our data into training and testing data. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0f8d75fb", + "metadata": {}, "outputs": [], "source": [ - "np.random.seed()\n", - "n = 100\n", - "# Make data set.\n", - "x = np.linspace(-3, 3, n).reshape(-1, 1)\n", - "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)" + "from sklearn.model_selection import train_test_split" ] }, { "cell_type": "markdown", - "id": "2b923c6f", - "metadata": { - "editable": true - }, + "id": "edb213fc", + "metadata": {}, "source": [ - "where $y$ is the function we want to fit with a given polynomial." + "**a)** Complete the code below so that the polynomial features and the targets y get split into training and test data.\n", + "\n", + "**b)** What is the shape of X_test?\n", + "\n", + "**c)** Fit your model to X_train\n", + "\n", + "**d)** Compute the MSE when your model predicts on the training data and on the testing data, using y_train and y_test as targets for the two cases.\n", + "\n", + "**e)** Why do we not fit the model to X_test?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a03e0388", + "metadata": {}, + "outputs": [], + "source": [ + "polynomial_features = ...\n", + "\n", + "#X_train, X_test, y_train, y_test = train_test_split(polynomial_features, y, test_size=0.2)\n", + "\n" ] }, { "cell_type": "markdown", - "id": "e62af227", - "metadata": { - "editable": true - }, - "source": [ - "**a)**\n", - "Write a first code which sets up a design matrix $X$ defined by a fifth-order polynomial and split your data set in training and test data." - ] - }, - { - "cell_type": "markdown", - "id": "8df2cfd3", - "metadata": { - "editable": true - }, - "source": [ - "**b)**\n", - "Write thereafter (using either **scikit-learn** or your matrix inversion code using for example **numpy**)\n", - "and perform an ordinary least squares fitting and compute the mean squared error for the training data and the test data. These calculations should apply to a model given by a fifth-order polynomial." - ] - }, - { - "cell_type": "markdown", - "id": "01a227d5", - "metadata": { - "editable": true - }, - "source": [ - "**c)**\n", - "Add now a model which allows you to make polynomials up to degree $15$. Perform a standard OLS fitting of the training data and compute the MSE for the training and test data and plot both test and training data MSE as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?" - ] + "id": "22e7536e", + "metadata": {}, + "source": [] } ], - "metadata": {}, + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.15" + } + }, "nbformat": 4, "nbformat_minor": 5 }