diff --git a/doc/LectureNotes/_build/.doctrees/environment.pickle b/doc/LectureNotes/_build/.doctrees/environment.pickle index 7f75321f5..7b060ebdd 100644 Binary files a/doc/LectureNotes/_build/.doctrees/environment.pickle and b/doc/LectureNotes/_build/.doctrees/environment.pickle differ diff --git a/doc/LectureNotes/_build/.doctrees/exercisesweek34.doctree b/doc/LectureNotes/_build/.doctrees/exercisesweek34.doctree new file mode 100644 index 000000000..a652d192b Binary files /dev/null and b/doc/LectureNotes/_build/.doctrees/exercisesweek34.doctree differ diff --git a/doc/LectureNotes/_build/.doctrees/exercisesweek35.doctree b/doc/LectureNotes/_build/.doctrees/exercisesweek35.doctree new file mode 100644 index 000000000..beac0e7da Binary files /dev/null and b/doc/LectureNotes/_build/.doctrees/exercisesweek35.doctree differ diff --git a/doc/LectureNotes/_build/.doctrees/exercisesweek36.doctree b/doc/LectureNotes/_build/.doctrees/exercisesweek36.doctree new file mode 100644 index 000000000..4c09c2dc8 Binary files /dev/null and b/doc/LectureNotes/_build/.doctrees/exercisesweek36.doctree differ diff --git a/doc/LectureNotes/_build/.doctrees/intro.doctree b/doc/LectureNotes/_build/.doctrees/intro.doctree index 32f449485..0b4af0db7 100644 Binary files a/doc/LectureNotes/_build/.doctrees/intro.doctree and b/doc/LectureNotes/_build/.doctrees/intro.doctree differ diff --git a/doc/LectureNotes/_build/html/_images/cb756715680fbe8cbf1d070371bf3950695ddf24b29ecb476c48fb9e559af08e.png b/doc/LectureNotes/_build/html/_images/cb756715680fbe8cbf1d070371bf3950695ddf24b29ecb476c48fb9e559af08e.png new file mode 100644 index 000000000..a16a30cd2 Binary files /dev/null and b/doc/LectureNotes/_build/html/_images/cb756715680fbe8cbf1d070371bf3950695ddf24b29ecb476c48fb9e559af08e.png differ diff --git a/doc/LectureNotes/_build/html/_sources/exercisesweek34.ipynb b/doc/LectureNotes/_build/html/_sources/exercisesweek34.ipynb new file mode 100644 index 000000000..50e773ede --- /dev/null +++ b/doc/LectureNotes/_build/html/_sources/exercisesweek34.ipynb @@ -0,0 +1,315 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "232d1306", + "metadata": {}, + "source": [ + "# Exercises week 34\n", + "\n", + "## Coding Setup and Linear Regression" + ] + }, + { + "cell_type": "markdown", + "id": "9b66a351", + "metadata": {}, + "source": [ + "Welcome to FYS-STK3155/4155!\n", + "\n", + "In this first week will focus on getting you set up with the programs you are going to be using throughout this course. We expect that many of you will encounter some trouble with setting these programs up, as they can be extremely finnicky and prone to not working the same on all machines, so we strongly encourage you to not get discouraged, and to show up to the group-sessions where we can help you along. The group sessions are also the best place to find group partners for the projects and to be challenged on your understanding of the material, which are both essential to doing well in this course. We strongly encourage you to form groups of 2-3 participants. \n", + "\n", + "If you are unable to complete this week's exercises, don't worry, this will likely be the most frustrating week for many of you. You have time to get back on track next week, especially if you come to the group-sessions! Note also that this week's set of exercises does not count for the additional score. The deadline for the weekly exercises is set to Fridays, at midnight." + ] + }, + { + "cell_type": "markdown", + "id": "36d8750b", + "metadata": {}, + "source": [ + "### Learning goals\n", + "\n", + "After completing these exercises, you will know how to\n", + "\n", + "- Create and use a Github repository\n", + "- Set up and use a virtual environment in Python\n", + "- Fit an OLS model to data using scikit-learn\n", + "- Fit a model on training data and evaluate it on test data\n", + "\n", + "### Deliverables\n", + "\n", + "Complete the following exercises while working in a jupyter notebook. Exercises 1,2 and 3 require no writing in the notebook. Then, in canvas, include\n", + "- The jupyter notebook with the exercises completed\n", + "- An exported PDF of the notebook (https://code.visualstudio.com/docs/datascience/jupyter-notebooks#_export-your-jupyter-notebook)\n", + "- Optional: A link to your github repository, which must be set to public, include the notebook file, a README file, requirements file and gitignore file.\n", + "\n", + "We require you to deliver a jupyter notebook so that we can evaluate the results of your code without needing to download and run the code of every student, as well as to teach you to use this useful tool." + ] + }, + { + "cell_type": "markdown", + "id": "2a9c7ef8", + "metadata": {}, + "source": [ + "## Exercise 1 - Github Setup\n" + ] + }, + { + "cell_type": "markdown", + "id": "1498aed1", + "metadata": {}, + "source": [ + "In this course, we require you to pay extra mind to the reproducibility of your results and the shareability of your code. The first step toward these goals is using a version control system like git and online repository like Github.\n", + "\n", + "**a)** Download git if you don't already have it on your machine, check with the terminal command ´git --version´ (https://git-scm.com/downloads).\n", + "\n", + "**b)** Create a Github account(https://github.com/), or log in to github with your UiO account (https://github.uio.no/login).\n", + "\n", + "**c)** Learn the basics of opening the terminal and navigating folders on your operating system. Things to learn: Opening a terminal, opening a terminal in a specific folder, listing the contents of the current folder, navigating into a folder, navigating out of a folder.\n", + "\n", + "**d)** Download the Github CLI tool and run ´gh auth login´ in your terminal to authenticate your local machine for some of the later steps. (https://github.com/cli/cli#installation). You might need to change file permissions to make it work, ask us or ChatGPT for help with these issues.\n", + "\n", + "**e)** As an alternative to the above terminal based instructions, you could install GitHub Desktop (see https://desktop.github.com/download/) or if you prefer GitLab, GitLab desktop (see https://about.gitlab.com/install/). This sets up all communications between your PC/Laptop and the repository. This allows you to combine exercises 1 and 2 in an easy way if you don't want to use terminarl. Keep in mind that these GUIs (graphical user interfaces) are not text editors." + ] + }, + { + "cell_type": "markdown", + "id": "c56fbefa", + "metadata": {}, + "source": [ + "## Exercise 2 - Setting up a Github repository\n" + ] + }, + { + "cell_type": "markdown", + "id": "fb9b8acd", + "metadata": { + "vscode": { + "languageId": "plaintext" + } + }, + "source": [ + "**a)** Create an empty repository for your coursework in this course in your browser at github.com (or uio github).\n", + "\n", + "**b)** Open a terminal in the location you want to create your local folder for this repository, like your desktop.\n", + "\n", + "**c)** Clone the repository to your laptop using the terminal command ´gh repo clone username/repository-name´. This creates a folder with the same name as the repository. Moving it or renaming it might require some extra steps.\n", + "\n", + "**d)** Download this jupyter notebook. Add the notebook to the local folder.\n", + "\n", + "**e)** Run the ´git add .´ command command in a terminal opened in the local folder to stage the current changes in the folder to be commited to the version control history. Run ´git status´ to see the staged files.\n", + "\n", + "**f)** Run the ´git commit -m \"Adding first weekly assignment file\"´ command to commit the staged changes to the version control history. Run ´git status´ to see that no files are staged.\n", + "\n", + "**g)** Run the ´git push\" command to upload the commited changes to the remote repository on Github.\n", + "\n", + "**h)** Add a file called README.txt to the repository at Github.com. Don't do this in your local folder. Add a suitable title for your repository and some inforomation to the file.\n", + "\n", + "**i)** Run the ´git fetch origin´ command to fetch the latest remote changes to your repository.\n", + "\n", + "**j)** Run the ´git pull´ command to download and update files to match the remote changes.\n" + ] + }, + { + "cell_type": "markdown", + "id": "f84d0db6", + "metadata": {}, + "source": [ + "## Exercise 3 - Setting up a Python virtual environment\n" + ] + }, + { + "cell_type": "markdown", + "id": "b5a4818a", + "metadata": {}, + "source": [ + "Following the themes from the previous exercises, another way of improving the reproducibility of your results and shareability of your code is having a good handle on which python packages you are using.\n", + "\n", + "There are many ways to manage your packages in Python, and you are free to use any approach you want, but in this course we encourage you to use something called a virtual environment. A virtual environemnt is a folder in your project which contains a Python runtime executable as well as all the packages you are using in the current project. In this way, each of your projects has its required set of packages installed in the same folder, so that if anything goes wrong while managing your packages it only affects the one project, and if multiple projects require different versions of the same package, you don't need to worry about messing up old projects. Also, it's easy to just delete the folder and start over if anything goes wrong.\n", + "\n", + "Virtual environments are typically created, activated, managed and updated using terminal commands, but for now we recommend that you let for example VS Code (a popular cross-paltform package) handle it for you to make the coding experience much easier. If you are familiar with another approach for virtual environments that works for you, feel free to keep doing it that way.\n" + ] + }, + { + "cell_type": "markdown", + "id": "0f6de364", + "metadata": {}, + "source": [ + "**a)** Open this notebook in VS Code (https://code.visualstudio.com/Download). Download the Python and Jupyter extensions.\n", + "\n", + "**b)** Press ´Cmd + Shift + P´, then search and run ´Python: Create Environment...´\n", + "\n", + "**c)** Select ´Venv´\n", + "\n", + "**d)** Choose the most up-to-date version of Python your have installed.\n", + "\n", + "**e)** Press ´Cmd + Shift + P´, then search and run ´Python: Select Interpreter´\n", + "\n", + "**f)** Selevet the (.venv) option you just created.\n", + "\n", + "**g)** Open a terminal in VS Code, the venv name should be visible at the beginning of the line. Run `pip list` to see that there are no packages install in the environment.\n", + "\n", + "**h)** In this terminal, run `pip install matplotlib numpy scikit-learn`. This will install the listed packages.\n", + "\n", + "**i)** To make these installations reproducible, which is important for reproducing results and sharing your code, run ´pip freeze > requirements.txt´ to create the file requirements.txt with all your dependencies.\n", + "\n", + "Now, anyone who wants to recreate your package setup can download your requirements.txt file and run ´pip install -r requirements.txt´ to install the correct packages and versions. To keep the requirements.txt file up to date with your environment, you will need to re-run the freeze command whenever you install a new package.\n", + "\n", + "**j)** Create a .gitignore file at the root of your project folder, and add the line ´.venv´ to it. This way, you won't try to upload a copy of all your python packages when you regularly push your changes to Github. Ignored files should not show up when you run ´git status´, and are not staged when running ´git add .´, try it!" + ] + }, + { + "cell_type": "markdown", + "id": "5d184ab1", + "metadata": {}, + "source": [ + "## Exercise 3 - Fitting an OLS model to data\n" + ] + }, + { + "cell_type": "markdown", + "id": "d19ebd67", + "metadata": {}, + "source": [ + "Great job on getting through all of that! Now it is time to do some actual machine learning!\n", + "\n", + "**a)** Complete the code below so that you fit a second order polynomial to the data. You will need to look up some scikit-learn documentation online (look at the imported functions for hints).\n", + "\n", + "**b)** Compute the mean square error for the line model and for the second degree polynomial model." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "b58fb9bf", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "from sklearn.preprocessing import PolynomialFeatures # use the fit_transform method of the created object!\n", + "from sklearn.linear_model import LinearRegression\n", + "from sklearn.metrics import mean_squared_error" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "0208e9ca", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhYAAAGdCAYAAABO2DpVAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjUsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvWftoOwAAAAlwSFlzAAAPYQAAD2EBqD+naQAASZ5JREFUeJzt3QuczPX++PH37GItdpclLVIuiSSV08U1KUVJOnXSr4vLOV1Jkn+FOn6ojqXboZtup1Kd8lMhKioJSeIkHZduWJdEInZdF7vzf7y/s7PNzs7l+535zv31fDzmrJ35zsx3J8f3vZ/P++JwOp1OAQAAsEGaHS8CAACgCCwAAIBtCCwAAIBtCCwAAIBtCCwAAIBtCCwAAIBtCCwAAIBtCCwAAIBtqkiUlZaWyi+//CJZWVnicDii/fYAACAE2k9z37590rBhQ0lLS4ufwEKDisaNG0f7bQEAgA22bt0qJ5xwgj2BRZMmTWTz5s2V7h88eLA888wzpl5DVyrcJ5adnW3l7QEAQIwUFRUZCwPu67gtgcWKFSukpKSk/Ps1a9bIxRdfLNdcc43p13Bvf2hQQWABAEBiCZbGYCmwOO644yp8P2HCBGnevLl07do1tLMDAABJJeQciyNHjsgbb7whw4cPDxi9FBcXGzfPpRQAAJCcQi43nTVrluzdu1cGDhwY8Lj8/HzJyckpv5G4CQBA8nI4tX4kBD169JBq1arJnDlzAh7na8VCg4vCwkK/ORaax3H06NFQTgsJJj09XapUqULpMQDEOb1+6wJBoOt3yFshWhkyf/58mTFjRtBjMzIyjJtZ+/fvl59//tmol0VqqFGjhjRo0MAIVAEAiS2kwOKVV16R+vXrS69evWw9GV2p0KBCLzSaKMpvsclNg0fN1fntt9+koKBAWrRoEbDpCgAgCQML7ZypgcWAAQOMJWw76faHXmw0qMjMzLT1tRGf9L9z1apVjVUwDTKqV68e61MCAITB8q+HugWyZcsW+dvf/iaRwkpFamGVAgCSh+Ulh0suuYT8BwAA4kxJqVOWF/wuO/cdlvpZ1eXcprmSnhb9X9SjPisEAADYG0x8sm6HzFr1i/x+4Ej5Yw1yqsuY3q2lZ5sGEk0EFgAAJKB5a7bLuDnrZHvhYZ+P7yg8LIPeWClTbmwX1eCCzW0baJMwzQvRmyYiHn/88cYMlZdfftlIdjXr1Vdfldq1a0f0XAEAyRFUDHpjpd+gQrmTFjT40JWNaEnKwEI/wC837Jb3Vm0zvkbjA+3Zs6ds375dNm3aJHPnzpVu3brJXXfdJZdffrkcO3Ys4u8PAEhuJWXXtpnfbJP7Z64uDxwC0WM0+NDtkmipkgpLQ9HYZ9ImYHl5ecafGzVqJO3atZP27dvLRRddZKxE3HzzzfLEE08YpbobN26U3Nxc6d27tzzyyCNSq1YtWbhwofz1r3+tUBUzZswYGTt2rLz++usyefJk+eGHH6RmzZpy4YUXyqRJk4xeIgCA5DcvyLZHMJrQGS1pqbA05N5n0sejSQOAM844o7xDqZZVPvnkk7J27VqZOnWqLFiwQO677z7jsY4dOxrBgrZJ1ZUPvd1zzz3l/T0eeugh+fbbb40ZLboqEmxGCwAgdbY9gtEqkWipkkxLRBrN+Voa0vt0DUAfv7h1XlTLb1q1aiX//e9/jT8PGzas/P4mTZrIww8/LLfffrs8++yzRjtr7cGuqxXulQ83z54hzZo1M4KTc845x2h/rqsdAIDkVBLg2maGXu3yclylp9GSNCsWun8ULIkl2vtMxvs6neVbG9pcTLdGdKskKytL+vXrJ7t375aDBw8GfI2vv/7a2DY58cQTjed17drVuF8blQEAktfyINe2QNy/QmsqQDR/oU6awMLs/lE095nUd999J02bNjW2LzSRs23btvLuu+8awcIzzzxjHKOtrP05cOCAMUlWt0j+/e9/y4oVK2TmzJlBnwcASHw7w7hm6UpFtEtNk2orxOz+UTT3mTSHYvXq1XL33XcbgYSWnj7++OPlLaynT59e4XjdDtFBbJ6+//57Y1VjwoQJxrh59Z///CdqPwMAIHbq1TI/HdzTkG7N5e6LW8ak82bSrFjo/pFWf/j7CPX+BhHcZyouLpYdO3bItm3bZOXKlTJ+/Hjp06ePsUrRv39/Ofnkk40kzKeeesqoCtFKj+eee67Ca2jeheZNfPrpp7Jr1y5ji0S3PzTgcD9v9uzZRiInACAFOEN7WqeTj4tJUJFUgYV+gLqPpBwx2GeaN2+eNGjQwAgOtKfFZ599ZiRZvvfee5Kenm5Uh2i56cSJE6VNmzbGtkZ+fn6F19DKEE3mvPbaa40Jr1qKql+1XPXtt9+W1q1bGysXjz32WER+BgBAfNl1oNjS8ZH+JdrUOTijPFGsqKjIqH4oLCw08gY8HT58WAoKCoychFDHZ8eqjwVCZ8d/dwBIRl9u2C3XvbjM1LHuX5sjlVcR6PqdlDkWbvphaklpPEx4AwDAjm1+7cfkNJGsGQ+/RCddYKE0iOjQvG6sTwMAgLD9zzknyj/n/1jpfv11WYONv3VqYvxCHS+/RCdlYAEAQLK38c6LkxUKbwQWAADEaRtvp5/H7+7eQoZc2CIuViiStioEAIBUaOPtEJFpK7ZKvCKwAAAgjiyP0xEVZhFYAAAQR3bG6YgKswgsAACII/XjcESFFSRvAgAQ5RyK5QF6LQXrXRGLUehWsGIRBTo2fdasWZIsxo4dK2eeeabp43Wyq34Gq1atiuh5AUAiVHt0nrjA6KZ517RVxlf9Xu/37l3hL3lT77/ijAZxWRGSvCsWpSUim5eK7P9VpNbxIid1FElLj9jbDRw4UPbu3es3eNi+fbvUqVMnYu8PAIh//kpIdWVC79dW3CpQ7wq3FxYXyFkn1om7HhbJGVismy0yb4RI0S9/3JfdUKTnRJHWV8TklPLy8mLyvgCA+C8hdZZtb4yasVr2HDxq+jX19bTjZrytXKQlXVAxvX/FoEIVbXfdr4/HeCvEvS0wY8YM6datm9SoUcOYfPrll19WeM6SJUukS5cukpmZKY0bN5ahQ4fKgQMHgm5PvPzyy8ao9Vq1asngwYOlpKTEmJKqwU39+vXlH//4R4XnbdmyxRjvrsfrUJm+ffvKr7/+WuEYnah6/PHHS1ZWltx0003G0DBvL730kpx66qnGELFWrVrJs88+G+anBgCpVUK6x0JQEc8lp2lJtf2hKxV+40FdhxrpOi4OPPDAA3LPPfcYeQennHKKXHfddXLs2DHjsQ0bNhij16+++mr573//K//3f/9nBBpDhgwJ+Jr6vLlz5xoj3N966y3517/+Jb169ZKff/5ZFi1aZIxs//vf/y5fffWVcXxpaakRVPz+++/G45988ols3LjRGNvuNn36dCNoGT9+vPznP/8xRsN7Bw06Av5///d/jaDlu+++M44dPXq0TJ06NSKfHQAkmp0RKg2Nx5LT5NkK0ZwK75WKCpwiRdtcxzXtIrGmQYVe9NW4cePktNNOk/Xr1xu/7efn58sNN9wgw4YNMx5v0aKFPPnkk9K1a1eZMmWK39HiGijoioWuLLRu3dpYEfnhhx/kww8/lLS0NGnZsqURXHz22Wdy3nnnyaeffiqrV682Rpbrqoh67bXXjHNZsWKFnHPOOTJp0iRjlUJv6uGHH5b58+dXWLUYM2aMPP7443LVVVcZ3+v483Xr1snzzz8vAwYMiPhnCQDxrn6ESkPjseQ0eVYsNFHTzuMirG3btuV/1lUAtXPnTuPrt99+K6+++qqxPeG+9ejRwwgcNAjwp0mTJkZQ4abbFxpgaFDheZ/7fXR1QQMKd1Ch9PjatWsbj7mP0SDEU4cOHcr/rNszulKigYfn+WoAovcDAET+dFIdsTMVQl+qQZyWnCbPioVWf9h5XIRVrVq1/M+ac6E0cFD79++X2267zcir8Kb5E2Ze0/26vu5zv48d9FzViy++WCkASU+PXCUOACSSrzfvkVJ/9aMWueMTnWwab4mbyRVYaEmpVn9ooqa/liL6uB4X59q1a2dsJZx88skRfR9Ntty6datxc69a6Ptq6ayuXLiP0ZyM/v37lz9v2bJlFVZAGjZsaORm6PYNACC8XAgNFQLFIPE6Lj35AgvtU6ElpVr9Uek/S1lE13NCxPpZFBYWVmoAVbdu3QrbDGaNGDFC2rdvbyRr3nzzzVKzZk3jgq/JlU8//bRt59y9e3c5/fTTjYBAcyk0eVQrSTSX4+yzzzaOueuuu4w+Hfp9p06djETNtWvXSrNmzcpfR3NEdHUlJyfHSDotLi42Ej337Nkjw4cPt+18ASBR1TeZC6Hj0HVyqWcFSV52hlx37onSpF5Nn506403yBBZK+1T0fc1PH4sJEe1jsXDhQjnrrLMq3Kd5B1qGGUr+hVZpaOWIlpw6nU5p3rx5hWoNO+i2yHvvvSd33nmnnH/++UYuhgYGTz31VPkx+p6aK3HfffcZCZtaqTJo0CD56KOPyo/R4EfLZh999FG59957jUBIAxZ38ikApLpzTbbpHnJhC+MWqOV3vHM49aoVRUVFRcZvtvobvvZN8KQXLk1O1KoCf5UP8dh5E+Gx7b87ACRA503xvaZudN6M1+2NYNfv5F2xcNMgIg5KSgEAqafEz5AxDRo0ePBu2R3vORNWJWdgAQBAjFYlxnkFDg08Age9aRvuRN7qCIbAAgCAKA0Z69nGNZW0Q/O69p9AnKQBEFgAABCFIWPjIjk0LI4GcMZl580o55MixvjvDSAVhoxtj8TQMF2lWKitFvrFzQDOuAos3J0ajxw5EutTQRQdPHjQ+OrdJRQAkq0B1k47h4atnSXyaAuRheP9HBCbAZxxtRVSpUoVox/Cb7/9ZlxkPGdcIDlXKjSo0NklOp+EFuAAEqXCw/vxn37dF92hYR+PFln6pIkDoz+AM64CC23YpAO5tKfB5s2bY306iBINKvLy8mJ9GgBgqcLD1+P+uBtg2TI0bM0sk0FFbAZwxlVgoapVq2aMCWc7JDXoyhQrFQASrcLj1vObyguLCwLO9LB9aNixIyLLXxD59EHrz43iAM64CyyUboHQgREAEI8VHurFz80FFbY0wNL8iHdvFlk7M8h4MomLAZxxGVgAABCvFR7KzAj0Id1Olk4n1wuvAZZue8y6TeRYGEmfERzA6QuBBQAAEajcaHF8rfAaYZlO0PQju1HEB3D6QmABAEAEKjfqh/o6uvWx6JHwgoquI0W63kfnTQAAYu1PJ9WR3JrV5PcDoRcRpDlE9oTyfG1mNfc+kX3bQ35v6ThUpNsoiRUaRQAA4FEN0vXRz8IKKtw5GHe8udJ4PUtBhXbKDDWoqFFP5C9TRS55SGLJcmCxbds2ufHGG6Vu3bqSmZkpp59+uvznP/+JzNkBABDlElMzfSnMGjdnnVFlYmr7Q2d9WK76UA6RG2aI3POjSJsrJdYsbYXs2bNHOnXqJN26dZO5c+fKcccdJz/99JPUqVMncmcIAEAMS0xD5fSYDxI0iVM7Y3rP+jCr450iLS6SeGEpsJg4caI0btxYXnnllfL7mjZtGvA5xcXFxs2tqKgolPMEACCmJaa2VZkcOyKy4kWRPZtE6jQROeeWEDtjOlxBRYy3PsIKLGbPni09evSQa665RhYtWiSNGjWSwYMHyy233OL3Ofn5+TJu3Dg7zhUAgIiwdThYoOoQLSH98mkRZ6nHfX8XaW1xC6NJF5EbZ4hUqSbxxlKOxcaNG2XKlClGy+2PPvpIBg0aJEOHDpWpU6f6fc6oUaOksLCw/LZ161Y7zhsAANuEWhqaVb1Kectub46y2SLl80HcfSk8gwql36+dIVKtpkcD8AAJmtdMFRn4flwGFZZXLEpLS+Xss8+W8eNdI1rPOussWbNmjTz33HMyYMAAn8/JyMgwbgAAxCu9+GsQoLNArORZXPOnE+SVLzYZ4YAz0HyQY0dcKxWBHDlY9irer1bmgvtFzr8nJr0pIrZioZNHW7duXeG+U089VbZs2WL3eQEAEDV68dcgQFlpvn1x6zyZcmM7Yx6Ip7yc6sb95fNBNKfCe6WiEqfIGTeIZDeo3EGz7+siF4yI+6DC8oqFVoT88MMPFe778ccf5aSTTrL7vAAAiGpVSE5mNflbpyYyc9U2+f3AUdNj0DUo0QBDE0B37jtsbKtUmg+iiZpmZNQUGbbGVSWiCZ06lVQHiCVAQBFSYHH33XdLx44dja2Qvn37yvLly+WFF14wbgAAJGr/Ci019awK0c6bZzbOkQXf/xZ8m6NsxaO8pLS0RGTzkoqBgVZ/mKHHaRDRtIskKofT6bRUtvv+++8bCZnav0JLTYcPHx6wKsSblpvm5OQYiZzZ2dmhnDMAALY2xfK+ELqDh1vPbyqzv91eIehoEGgM+rrZrkZXnj0pdGz5Jf8QefemwNshjnSRB3bEbVKm2eu35cAiWicGAECktz86T1zgt3+Fe7tj0b3d5OvNe/xvc7hXKRY/JrJwvJ9XEpGWl4r88GHgGR9x1pMilOs3Q8gAACkpWFMsd+dMDSr8ds50BxTLnhU5vDfAKzlEtn8r0mGI61jPlQtdqehwR1wHFVYQWAAAUpLZplh+j9NtjzlDRQ7tMfEqTpGibSKn9BS5aEzlzptxuv0RCgILAEBKMtsUq9JxAbc9gtCETg0idIUiSRFYAABSkpmmWHVrVpM/nVSn4irF3PtCH21e63hJdpbHpgMAkCpNsXYfOCLdHpkvyxfMEpk3SmR6vxCDCoer0ZWWniY5AgsAQMrSklFfnTPdLk37SmYX/03OXTzAlXQZ1ptNSKhGV6FiKwQAIKkeXBw75pQ7p31TYUtkZPqbcluV98Vhpce3L7pSoUFF6yskFRBYAAAk1ZtkDZn2Tfn3VeSY5Fd5Uf6S/nn4L35BYgwOsxOBBQAgpZtkaTtvlSalMqnK03J5+jLx7n9lWWauSO/JKbNK4YnAAgAgqd4kq2faMvln1SmS6Qg8fCyozDoi5w1KuVUKTwQWAICUpc2vbMulSMFtD18ILAAAKavV7gVyRZX3w3uRFEvODIbAAgCQmkpL5JSvx4S+UqFbHq16uXpTpPgqhScCCwBAatq8VBwHd4f23DifRBpLBBYAgNSkczusqlFP5LLHRdpcGYkzSgoEFgCA1GR1bsdpV4lc/RLbHkEQWAAAkqovhZaQarWHTiXVQWM6E8QnzY3IbihS9EuQV3WIdLyTrQ+TCCwAAEnTQVObXWlfCjedXqqDxrRtdyW68tBzosj0/iL+5ps26SJy4wzXqHOYwhAyAEBSBBWD3lgpvxYelPZp6+SKtKXG152FB4379XGftES072uulQvvXIprpooMfJ+gwiJWLAAASdGW+5K05TKm6mvS0PF7+WO/OHPlwaP9Zdyc6nJx6zzf2yIaXGjZ6OalroROzb2ghDRkBBYAgISmORVt9y2WKVUnVXosT36XZ6tOkkH79LgzpUPzur5fRIOIpl0if7IpgK0QAEBC21l0wFipUN4LEu7vx1R93TgOkUdgAQBIaCcfXG1sf/gr/tD7Gzp2G8ch8ggsAAAJ7dSsg7Yeh/AQWAAAElpaVp6txyE8BBYAgMRW1ujKqY2sfDDu1wmkehwijsACAJDYyhpdaVjhHVzo98Y9Otac8tGoILAAACS+skZXjuyKHTYd2vhKG2Dp44gK+lgAAJIDja7iAoEFACB+lJaEFxjQ6CrmCCwAAPFh3WyReSMqThvVrQwdFMZWRsIgxwIAEB9BhU4Z9R5hXrTddb8+joRAYAEAiP32h65U+BxdXnbfvJGu4xD3CCwAALGlORXeKxUVOEWKtrmOQ9wjsAAAxJYmatp5HGKKwAIAEFta/WHncYgpqkIAADEtIS1p3EF2SV05zrnb54TSUqfITkddOa5xB6EjRfwjsAAAxLSEdPnmQnn1SD+ZUnWSEUR4Bhf6vRpzpJ8M3FwoHZrXjfZPAIvYCgEAxLSEdOe+w/JR6bky6Ogw2SG5FQ7dIXWN+/VxPQ7xjxULAEAMSkgdrhLSVr2kflZ1414NHj4pPlvOTfte6ste2Sm1ZXlpKykt+x3YfRziG4EFACCmJaTnNu0sDXKqy47Cw0YQsay0dYUjdWckL6e6nNu04moG4hNbIQCAmJaQpqc5ZExvVzDhnbvp/l4f1+MQ/wgsAAAxLyHt2aaBTLmxnbEy4Um/1/v1cSQGtkIAAPbTklKt/tBETZ95Fg7X43pcGQ0eLm6dJ8sLfjcSNTWnQrc/WKlILAQWAAD7aZ8KLSnV6g9jQ8MzuCgLFHpOqDQSXYMISkoTG1shAIDI0D4VfV8TZ3bFbQynrlT0fY1R6EmKFQsAQMTMKz1HHjo8WRof+ba8hHTr4TNkdOnp0jPWJ4eIILAAAETEvDXbZdAbK41NkG3yRwmpo+iocT9JmcmJrRAAgO1KSp0ybs46v+2xlD6uxyGFA4uxY8eKw+GocGvVqlXkzg4AkJC0smN7of8W3BpO6ON6HFJ8K+S0006T+fPn//ECVdhNAYBUmURqltm5Hsz/SD6WowINJPLy8kwfX1xcbNzcioqKrL4lACBOJpGaZXauB/M/ko/lHIuffvpJGjZsKM2aNZMbbrhBtmzZEvD4/Px8ycnJKb81btw4nPMFAMRwEqlZ2thK53/4a22l9+vjzP9IPg6n02k6c2bu3Lmyf/9+admypWzfvl3GjRsn27ZtkzVr1khWVpbpFQsNLgoLCyU7O9uenwIAEPrWx6YlIm/3Fzm0189BZV0yh622tC3irgoR3+2xqApJMHr91gWCYNdvS4GFt71798pJJ50kTzzxhNx00022nhgAIMIBxeLHRL56NkBA4WXA+yJNu1h6Gw0utPrDM5FTVyp0qBhBRWIxe/0OK/Oydu3acsopp8j69evDeRkAQDStnSUye4hI8b7ITCz1wPyP1BNWYKHbIhs2bJB+/frZd0YAgMj5eLTI0icjO7HUC/M/Uoul5M177rlHFi1aJJs2bZKlS5fKn//8Z0lPT5frrrsucmcIALDHmlkhBhWaY9GowiRSwJYVi59//tkIInbv3i3HHXecdO7cWZYtW2b8GQAQ5zkVHw4P4Yn+J5ECYQcW06ZNs3I4ACBeaNOrg7stP00nkTo0qGASKUyibSYAJNvKRMHnIpuXuGo8tYqjSWdLiZc6vqNQasngo0Nly+GzmEQKSwgsACBZaBOrOUNFDu35477PHxXJzBU573ZTL+FuQDDy6M3yZWkbJpHCMqabAkAyrFIsnCgyvV/FoMLt0O8iC8eLZNYJ+lJ7pJYMOjpMPio91/ieSaSwihULAEj0VYoP7xXZv8PEwY6AKxWzS9rL3ceGSKnX75yek0gpG0UwrFgAQELP9+hnMqgoW7m44H45lFlxkOQuZ7YMOjpU7jo2tFJQ4YlJpDCDFQsASNTtD82nsOjbg7ly1Z7H5Jy076W+7JWdUluWl7YKGFC4MYkUZhBYAEAi0soPX/kUQTz/zUEpkTRZVtra9HN0AyWPSaQwia0QAEhEWk5qUXGNBjJvXzNLz3FnZejQMOZ7wAwCCwBIRCEUaHzbZqSpLQ9PulJBqSmsILAAgERkZXy59rHo+7qUtOxt6S1G9zpVloy4kKAClpBjAQCJSLtpasCglR7+pFcXuW6aSLPzjTkf55Y6pUFOdaN01Ix6WRlsf8AyViwAIB5bcq9+x/VVv/dFB4L1nhz4ta5+UeTkbuXDwzRIuOIM86sPVIEgFKxYAEA89aWYN0Kk6Jc/7stuKNJzou8hYHpf39crPUeTNDWfoiSjk7FK4V51mLdmu7ywuCDoaVAFgnA4nE53Z/joKCoqkpycHCksLJTs7OxovjUAxHmzq/4+sjLLtiL6vuZ/wqiuamxeKt9+971RTqqVH+4kTd360IqOi1vnSeeJC4Jug7g3PkjYRKjXb7ZCACDWNDDQVQefpR5l980b6XdbRPtSTN6QJ30WN5QP951cofJDA4nb31gpTy9Ybyq3IrdmNYIKhIWtEACItc1LK25/VOIUKdrmOs6rGkS3N8bOXis7iooDvsXzizeYOpW/9zqVoAJhIbAAgFjb/2tIx2lQoSPNzexnHzziJwnUS15OprlzAfxgKwQAYq3W8ZaP0xHmOsrcSpKcI8hjmo9BwibCRWABALF2UkdX9YffS79DJLuR67gyOsLcbD8KN3cQ4v0utO2GnQgsACDWtM+ElpQGuuz3nFDejyKcEeY3dWpilJJ6om037ESOBQBEglZwbFoiUrBIZO/PIrVPEGlyviv50iNAqNiT4jU/fSwmVCo1DbV5VffWeXJ/r9bGiocGJ/o6uv3BSgXsQmABAHZbO0tk9hCR4n0V7//8cZHMOiK9n/Tf8KpVL1f1hyZqak6Fbn/4CET+dFId0Vig1GSShWfTKw0iOjSvG+pPBwREYAEAdvp4tMjSJ/0/fmiPyPR+ro6ZvoILDSJMDBj7evMeS0GFIocC0UCOBQDYZc2swEGFp7kj/M8BMcFKjgU5FIgmViwAwA4aJHw43Pzx+37x2fBKy0jN5D+YzbHQ0ecDOzVlpQJRQ2ABAKEqm9Fh5EPo7eBua8/30fBKe1N4lpG6Z314rzZowKGP7Sg87LOXhTungqAC0UZgAQB2TSK1yqPhlb8umho46P3eWxkaLGjAoY9p2OD5PHIqEEvkWABAqJNIwwkqshqWN7wK1EXTfZ8+rsd50kBDAw76UiCesGIBALZNIrXg0onlZaTBumjqO+njepx3magGDzoSnb4UiBcEFgBg6yTSIDJzRXpPrlBqarbCw99x9KVAPCGwAIBITCL1VCVT5OSLRc65yWfnTbMVHqF22wSiicACAIJVfHh2wDQ7ibTHeCmtWV++21dD1tc4Xepn1/S7RWG2woPJo0gEBBYAYKbiw5jZMdHVclv/XLTdT56FTiJtKPNq9ZFx7/9QljuxOmDpKBUeSCYOp9MZZgaSNUVFRZKTkyOFhYWSnZ0dzbcGAJMVH/38P66tuJVWhRgqhwHfdJgsV31Wr1LY4Q4L/FVsWOljAUSb2es3gQUAeG5/PHqyyKHfAydf3rte5PsPfKxqNJKSHvnSeXYtv1Ue7m2NJSMu9LkCYbbzJhBtZq/fbIUAgJuOOQ8UVCh9XI/zM4l0ecFe2V64LKTSUUWFBxIdgQUAuBV8bv64Zl19TiKdv26H7UPEgERCYAEgNfmq+jC74+DnOM2R+NcXm0y9BKWjSFYEFgBSj7+qj7PcCZlBnNS50l3uttzBUDqKZEdgASA153x412xo+eiiCSLVaokc2R84edNr+8NMW243fVdKR5HMGEIGIHUEnPOh9zlE0qsFfg1tx+3VOdNKzsTfOjWhdBRJjcACQOoIOufD6ar6uOB+kawGlaeRag8LjxkfoeRM6MAwIJmxFQIgdZIzzc75qNtc5O61vlt6+0FbbsCFwAJA6iRnthto7vkaSPgoJQ0kUFtuKfv+f85pbPr1gETFVgiA5FqlWDjR1ZLbe8tDkzMX5otk1vFfL2rM+WjkWp0IgeZOaLtuXZnw5Z/zf5LOExcYZalAsiKwAJA8qxT/PE1k4Xg/B7jXENzrCd7BRdn3PScE3PIwE1xou+67u5/i83HdKtFVDYILJCsCCwCJvUKxcZHI//V3rVLsC3ax9kjOzPZKztStkr6v+U3OtGraii3+zsCgPS+09wWQbMixAJC4KxSzh4oc3mP9uZqcOWyNpeRMK4L1tAg2LwRI2RWLCRMmiMPhkGHDhtl3RgAQzNpZrhWKUIIK7+TM0//i+mpTUGGlpwXzQpCMQl6xWLFihTz//PPStm1be88IAAJZM0vkHZPVHT6TMxuGnJxpltmeFswLQTIKacVi//79csMNN8iLL74odepohjUARGn7450BfjpnmhRmcqYnzZH4csNueW/VNuOrO2diz4EjEqhjtz6kPS/oaYFkFNKKxR133CG9evWS7t27y8MPPxzw2OLiYuPmVlRUFMpbAkh15e24Q6RlpBpUmEzO1CBBcyB0u0JXFjQI8JzvoVUdmoDpmUuhwcIVZzSQFxYXBA19mBeCZGU5sJg2bZqsXLnS2AoxIz8/X8aNGxfKuQFIdZ7dM/UWsB13AFoFcv49plcqNGgYO3ut7Cj645eivOwMGXvFaUY5qT6uJaPewYMGGc8vLgj42hpLPH3dWcwLQdKyFFhs3bpV7rrrLvnkk0+kenVze4OjRo2S4cOHV1ixaNyY7nMAQuieGeFVCqVBw+1vrKx0vwYZev+z158lD33wXcibMbpbUqdmRojPBpIssPj6669l586d0q5du/L7SkpKZPHixfL0008bWx7p6RV/I8jIyDBuABD2aHMruo4U6XqfpXwK3f4YOWN1wGPuffe/cqC4JPTzohoESc5SYHHRRRfJ6tUV/0/317/+VVq1aiUjRoyoFFQAgOVtD21yNW9keEFFx6Ei3UZZftqyDbtl78GjAY8JN6hQVIMgmVkKLLKysqRNmzYV7qtZs6bUrVu30v0AENVtD5WRJXLF0yKnXRnS07/cuEsiiQmnSAV03gSQ+NseOljsvEGWEjR9M1elUSsj3Vi5CHTG3hNO3a9MNQiSXdiBxcKFC+05EwCpt/VR8LnInKGhBRU9xrs6aNrYjlvbaz/92fqgx93cuZlM/vQnv8HDrec3ldnfbq9QiqorFRpUUA2CZMeKBYAE2/oo65553u22tuFW7ZvVldo1qgbMs6hTo6rceVELadUgq1IfC8/g4b6epwbsgwEkKwILAAm09WHPaHN/9MI/4arTfZabuuVfdbpxnAYPF7fO8xs86FcGjCEVEVgAiN7Wx6YlInPuDD2fQlcqLPalsEoDhudubCdjZ6+THUUVu2p6b2UQPACVEVgAiOOtD4dIjboiPfNFshrYOto8kGCrEQD8I7AAEKdbH2UX8cv/GdEVCn9YjQBCQ2ABIAqDw5xR3/YINkQMQGQQWACIHO2kaXX7Q3tSXDNVpEnnkLc9/E0epdwTiLy0KLwHgFSlE0lN09UEh0jvJ0WadQ0rqNDJo55BhdpReNi4Xx8HEDkEFgAiR5tXWdn66PtaWPkUuv2hKxW+Nl7c9+njehyAyGArBEDkaBWHBgxFukrg52KemSvyl1dEmnYJu+Jj2cbdlVYqPOkZ6OOae0FiJhAZrFgAiBwNFHpOLPvG4WfrY7JI8wvCDip0i+OOf/tvbOWJseVA5BBYAIgs3drQLY7sBrZvfXjnVew9FHjkuZtWieh2yJcbdst7q7YZX9keAezBVgiAyNPgoVUvV5WIJnTaODgsUF6Fv7Hlew4US+eJC6gaASKAwAJAdGgQoXkUNtKg4uUlBQHzKrxdcUYDuePNbyoFIu6qkSk3tiO4AMLAVgiAhKTbH+0e+lj+8eF3po7XqaXPXN/OGGdO1QgQOaxYAIhb/rpnalARaAKpL89c107S0hxUjQARRmABIC6DCM2DeOiD7yrlQYzudarcP3ON6dd151W0b15X3v+vuS6gVI0AoSOwABBzvlpw+6J5EIPf/Mby62tSpq50aMBihtnjAFRGYAEgptylomayGqxmPmhexYSrTi9PxtStFF310ADFGWB1Q48DEBqSNwHEjJVS0VBoXoVnhYeuWujqhb92XZ6rGwBCQ2ABpPJI84LPRVa/4/qq30eZ5lRYKRW1ok6NqkZehTcNNLSkVFcmPOn3lJoC4WMrBEhF62aLzBtRcaS5dsLU9ts2dMI0K5JJkv+4so3flQcNHi5uneez4gRAeAgsgFQMKqb3r5yxoIPC9H6b2mxHMklSL/+Btk9uO7+pXNa2YcDX0CCCklLAfgQWQCrQbQ5tp71vu8i8UX4uy3qfQ2TeSFf7bRvabQcTLJnSH3/H1sqoIo9c3VYua8t2BhAr5FgAyW7NLJHHWohMvVxkxi0iB3cFONgpUrTNFYREQaBkSl+C7VTUykiXHm3y7Dk5ACFhxQJIZh+PFln6pPXn6aCwKHEnU3r3sXA1w2otdWpWM/Igdu1zNcwKZEdRMV0zgRgjsACS1dpZoQUVSqePRpGZZEodb24GXTOB2CKwAJI1p+KD/xfCEx2u6hAdaR5lwZIp6ZoJJAZyLIBkpDkSAXMpfClbHeg5ISqJm6EmevpLs9D79XG6ZgKxRWABJGOjq1ByJHSlIoqlplbRNRNIDGyFAMnY6KrdQHPPz8gW6fW4SFYD1/ZHBFcqvKeX/umkOvL15j2WGlT5S/TUrpkaVNA1E4g9h9PpjFSbfp+KiookJydHCgsLJTs7O5pvDaRGoyt3+6jMXJFDvwd+jb9MFWlzpcRieqnGEKUep97AQnDgHaTQNROIn+s3KxZAom59zBkauNFVsB6VHYdGLajwNb3UM6hQ2iRLjzMzr4OumUD8IscCSLRVikltRF7vI3JoT4ADna7VigtGubZGPNWoJ3LNVJFLHoqr6aXuY/R4fR6AxMSKBZDwWx8B1G0uMmyNq0pEEzq1P0WEcynCmV6qP5keT5MrIHERWACJsv2hSZqWJmqUNbrSIKJpF4mFUJtV0eQKSFwEFkAi0BUHz8qPOG50ZUezKppcAYmLHAsgEVjqSxE/ja6CNbXyRpMrIPERWACJwMrsjjhqdGVleilNroDkQGABJALd0jCqOwJccDPriPSfLTJsdVwEFd5NrbSJlSfv2EEfN1NqCiC+kWMBJALd0ug5sawqxLs3RdkVuveTIs26SjzyNb00lM6bAOIfnTeBhG/h3ciVTxFHqxQAkg+dN4F4Lh0Nta+EBg+tesWsLwUABENgAcTD0DDd5jC74hDDvhQAEAzJm0C0O2d696Mo2u66Xx8HgARHYAHEvHNm2X3zRrqOA4AExlYIEI18ioJFQTpnOkWKtrmOtXmbgxHjAKKJwAKIZj6FrR02zY0s12mhnoPAtLOlNqGiXwSASGArBLB7haLgc5F5o0Sm97M438Nih00TQcWgN1ZWmi66o/Cwcb8+DgAxDSymTJkibdu2NepX9dahQweZO3eu7ScFJOwKxaQ2IlMvF1n2rMUn69CwRrYNDdPtD12pCJDRYTyuxwFAzLZCTjjhBJkwYYK0aNFCtK/W1KlTpU+fPvLNN9/IaaedZuuJAQm1SrH4MZGF40N8AfuHhmlOhfdKhScNJ/TxV78okHpZGeReAIifzpu5ubny6KOPyk033WTqeDpvIulWKebeJ7IvjG2FCHTOfG/VNrlr2ipLzyH3AkBMO2+WlJTI22+/LQcOHDC2RPwpLi42bp4nBiRVXwqfGw4mdLnXNdsjAp0zdQXCKnfuBYPAAEQ1eXP16tVSq1YtycjIkNtvv11mzpwprVu7xiL7kp+fb0Q47lvjxo3DOmEgLrY+Ni4SmXNniEFFWT5Ft1Gu0tIItOPWbQ1dgbCysUHuBYCYBBYtW7aUVatWyVdffSWDBg2SAQMGyLp16/weP2rUKGPZxH3bunVruOcMxD5B87UrRA7tjYt8Cl80V0K3NTze0RR37oXmaABATHIsunfvLs2bN5fnn3/e1PHkWCBltz5iMInUVx8LMyb/z5nS58xGETsvAIknatNNS0tLK+RQAKnXktuE9oNFWl5mSz6FlU6amitxceu88uN37SuWhz74LiI5GgBgObDQbY1LL71UTjzxRNm3b5+8+eabsnDhQvnoo4/4NJHctNW21WZXEVihCKWTpgYdHZrXLQ9KXlpSYCRq+gqRNDzJy3EFKwAQ8cBi586d0r9/f9m+fbuxHKLNsjSouPjii0N6cyDuZ3xoi23thhlKOekF94ucf49tuRTuTprOMKo53LkXerwGEZ6v5V7z0MfpZwEgZjkWVpFjgbh27IjInGEi62aKHD34x/016ooc3B2zPApdaeg8cYHfXAn3SsOSEReaCgqYIQIgbnMsgKTx8WiRpU/5zqMwE1Rk5or85ZWIlJCa7aSpx7m3PQLxzr2g8yYAuxBYILXplsemJSILHhL5eYXJJ/nZROg9WaT5BZE4S+PiH85x/hI+zQQhAGAFgQVSu3x0zlCRQ3usPc/YFtn1x/fZDSNeQmq2SsPXcWx7AIgmAgukcE+Kfsa6g+XF/575IlkN/kjstFhCaqVc1LuTptVqDjsSPgHACgILpOb2x5yhoQUVSoMKzaMIIYjYc+CIPPSB9dWDQNUcUvb9ZW1cORPuQCXY6HR9HX1ccy3IrQBgF6pCkHp0zoe25A6FVnwMW+13hcIzkNi064C8tXyL7CgK3EDOfUk3s3rga1tDYwLP0R7uQCUns5pc9+KyoD/SW7e0J9cCQFBUhQD++lL89HHorxNgxkeo7bOtrB54VnN8sm6HvPzFpgpBhec2x187NbE1MRQAzCCwQPIHFIsfE/nq2RCHhpWpVkvkyil+EzT95TKYZaVcVAMP3e4YPn2V39fS0OS9VeY6hdK+G4CdCCyQ3AHF0skiRw6E/DLGRfq0q0Sufing9oe/XAarzK4emOlrsfvAEcmtWVX2HDhK+24A8Ts2HYh7a2aJTDhRZOH4gEGFv0BAs470NsdxgZTev1PkmlcCVn0Eu8hbYXb1wGwA8ueyCaXemyu07wYQKQQWSL7ume8MEDmyP+ihgS6nzx+7XKpe/ZykV8uISo6Coyzp0uzqgdkApHvrPCMpVFcmPOn3lJoCiAS2QpA81s4SWfpkWC+xy5klj1e5TbpcfbNRVfHeqm1Be02Em6MQyuqBlb4W+pq07wYQLQQWSJ6cig/+X0hPLblkvGw4WEN2OmtLepNO0uXQMUu9JoJd5IPJC6ELptUppbTvBhAt9LFAcij4XGTq5WH3pfBX3RGo14Qmbz69YL38c/6PQd9OA5DRvU6VOjUzbFk9oF03gGihjwVSi/anCLMvRSidKoP1rtCL/P+cc6I0qVcjIlsQTCkFEG8ILJAcdGaHFTrivPdkKWnVW5Zv2G1clHftKzY1mnzZht2SluaQ+et2yL++2OT3+Lu7t5AhF7aI+EWebQ4A8YTAAslBB4HplNGiIE2hqtYU6XSXyPn3yLx1O2XcxAWWS0XveHOl7D10NOAxGkpMW7HVCCwAIJVQborkoNsZPScGLiLVRlejtopcMMIIKjSXIpT+E8GCCu9OmgCQSggskDy03Xbf11wrF55q1BO5Zmp5oys7O2UGwxwOAKmGrRAkxuAwzaHQ7Y4AHTDLg4tWvQI+z85OmcEwhwNAqiGwQOIMDtOVCN3u8DMIrJwGEU27+B1n/tOvwbtyhos5HABSFYEF4su62eKcc5c4DvnITSjaLjK9v2u7I1hwYcM481AxhwNAKiOwQHwFFRo4+B8P5rpszxvp2u4IMG3UvTpR8Nt+mfTpeltOr2ZGuhwsLgmamxFKJ00ASBYEFogPpSVyaM69Ut3pFEfAX/KdIkXbXDkUXtsdkV6duKVzM5n86U8+W2jr93/r1MRoVkWDKgCpjMAiDnn+xp3UnRQ9kjNL9v0qmYd2BB45GqTTpr923HaoU6Oq3HlRC2nVIKtS4MIKBQD8gcAizqTM7Id1s0XmjShvaBWk1iNop81Il5DmX3W6EdzRQhsAAiOwiCP+fuPWqZl6v68BWAkbVATMpfDPyLLQwWFaQuohUiWkvoI6WmgDgH8EFnEilAFYCdmPouZxInPv8xtU6KxdfzkWxhxeR8XBYZFqRFU7s6o8c0M7ad+sbuJ93gAQQwQWcSLYb9yeLaIT6rdlrfSYO0Ic+4LM8CgTKHFzr6OWZF/zrKT7KDW1qxGV++0nXH26dDq5ni2vCQCphJbeccLsb9wJ1SLaKB/tJ2IyqPBnj7OWPH70L/LV1V9J+ml9fB6jeQ66bWGWLkLc0qVJpedoImbSbDkBQAywYhEnzP7GnRAtonXro+BzOfbubZIeYGsjkAeP3ii7nLVlp9SWrbXOkNHXnB7wYq/bFaN7tZbBb6409fpPX9dOLmvbQEZe2ppETACwEYFFnHD/xq2Jms5EbhHtUe1h/OWyeI0udYrskLpyQo9hckZ2DUsX+zo1q5l6j7u7n2IEFYpETACwF1shcUIvcFp9oBxx1CJak0q/3LBb3lu1zfiq3wet9igrIbXK/dLjjvaTutk1pM+ZjYyLvtmf2ew2UZN6NUI6PwBAcKxYxBFd6tf9fTMNmOxuouXr9T5Zt8N8Tw3d/tCVijA6SehKhQYVH5WeKwND2PJJqu0kAEhQBBZRphfwZRt3G7/960W4Q7N60t7jt3IzDZjsbqLl6/Vq16gqew8erXSs354aWk4awkrFQ0evl9+cuUYuxfLSVuKUNONn0Z/ZavCUNNtJAJDAHE6n0R0gaoqKiiQnJ0cKCwslOztbUolewEfOWF3pgq0X8QlXBU5ODNZEy325tVrREEobbPcFesmIC/+40K9+R+Tdm0y/hv6t2yO15Ozi56S0bEfO82dQoQRP7p/HeA+vc3a/NhUfABC56zc5FlGiF7zb31jpcxVA79PH9JhAOQ3BmmjpbezstYHzIGxog+3ZU8Nfi+2Az3e6XuPhtNvLgwrPUk+lwYF3Xw/3aol+TsG2k/S1PFFGCgDRwVZIFOgFfOzsdUGPG/Huf43jdhT5/i3dTNvqHUXF8vSC9XJX9xbl763P09f8fX+x5NasJnk5mcZ2QLhtsCskS2qL7eyGIkV60Q8cqvzqyJXtHcbKoxf3l2u8tjpU54kLwupAyjwPAIgdAosocF/Ygyk8dMy4+ctpKD5Waur9/jn/R2mZV8v4s78R4hqwXNYmT8JRIQlSW2z3nGhUhTjFIQ6P0MD9pw3N+0txs57S6rweklfF9VfPu9RTV2ns6EBKGSkAxAaBRRSE0y3T87f0x645w/TzNJej8OBRv2sHenH+1xebgr5OmpTKuWnfS33ZWyHB0mcSZOsr5JsOk6Xhl+PkeNHkVJdfpa5s7zBGzuoxIDU7kAJACiGwiIJwyxvdv6XrH3Slwcz2ha9cDl90d8Cd8+CtR9pyGVP1NWno+COX4hdnrjx4tL9c2fv2SlsLRuLkZ/XEIZMrBCMrSltJ6WdpMqXR9qA5DpSMAkBiI3kzCvQ3+7zs8C+Euw4UlzfRsovmebpXRTz1TFsuU6pOkjyPoELp91OqTZaeaSv8JoJqQuay0tYyu7Sj8bWk7K+ZPh4ssdRdMuovG0Lvd5ejAgDiT0oGFpa6SdpAf7Mfe0X4AYH+lq5JiTUzKo4MD9ffOjUxtjZ026N92jrpk7ZExld72Zjx4f0XRL83LvrzRrqaYlmczvrqFwUBP+947UAKADAn5bZC7G4uZZa+9nM3tvPdxyKzihwpccrBI39cqL1prwt3JceBYv/HhUKDlV7pX0nT5WMkV4pMPMMpUrTN1RSraRdLOQ8PffCdvLSkIODnbaUDKQAgvqRUYOGvGZTfbpI2c5dBenfePKdprpz54McBn3u0rCLE7qRFDarqL3tYmv3oWqGwZP+vIeU8mPm8KRkFgMSUMoFFsOZSZvoj2EFfu9PJ9Yyb2xc/7Qq4WqEOHCmRZRt225q0qD/l02dskWbLXw7tBTyaYgVrpx3K503JKAAknpTJsTCbA1Chm2SUfLlxl+njgiU3mqWvMeWGM6Tttw8ar2VttcIhkt3I1RTLRG5EvH3eAIDISZnAIr77I5i9qjsqXMBDlVuzqiy6t5v0rFUgVYt/D+1ce05wNcUy0U47EPpRAEBySZnAIlb9EcxUoJhd7ncf576Aa4Bglrvi44q0pXLKoW/l64JdFXIkTNO23X1fM5ph+aLnpsPJRvc61dTL0Y8CAFI4xyI/P19mzJgh33//vWRmZkrHjh1l4sSJ0rJlS4l30Rqp7Tnqe9Oug/LW8i1+Z3+4aYdM3YoINGe2To2q0r7ZHwGIPv/CVsdL+/xP5fcDRwKek69GV4fefVGkvblppEVSQ2r9eZKkaVCh2x9eKxXedFVlYKemRvUHI8wBILVYWrFYtGiR3HHHHbJs2TL55JNP5OjRo3LJJZfIgQMHJN7Z0R8h2OqDVp3oAK3rXlwmd01bZczs8J4R4j2hU7/e8ebKgEGFyr/q9ErnVq1Kmoz/cxtXjoSfVYqh6e/Kc1UnSQOpuOVR/fBOkYXjRTJz/SZbGh05nSIb2o+XtDOudZWWBgkq3OhHAQCpyeF0Bruk+ffbb79J/fr1jYDj/PPPt3Wee7z1sQj2PH+lrBLgt3XNc+j66GcBk0r1uvv0dWfJZW0bWjo3XaXIr/qi5DoCBX0Okcw6Iof2iNMYHVaR/iwFp9wkza5/wsRPFV99QwAA9jJ7/Q6r3FRfXOXm+l/OLi4uNm6eJxZLofRHCNb/4pnr28lDH/guZQ1UEfH6l5uCzv3QRZE6NTMs/Uzp38+Ry76bZCIl1Cly6HeRC+4Xx8pXRYp+KX/kSEZdSe/9uDRr82eTP5W5c6MfBQAkt5ADi9LSUhk2bJh06tRJ2rRpEzAvY9y4cRJPrPRHMNP/YvR7a2R3kDwHXzb/ftC2yol0KZUOaetE0raLbJ4oTof5WhOp21xk2BpXJ01N6Kx1vFQzkUthFv0oACB1hBxYaK7FmjVrZMmSJQGPGzVqlAwfPrzCikXjxo3FTp4Jk3b/Rmym/0UoQYU6KbeGPZUT62aLzBtRYcXBYbXRlQYRZe25AQCIamAxZMgQef/992Xx4sVywgknBDw2IyPDuEVKpPfwI9FnwZ1j0a9Dk/AqJ3QI2KJHRBZNCP1kvBpdAQAQtaoQzfPUoGLmzJmyYMECadq0qcSSO/fBe0XBu/IiHGb7LGhPCSurBBr4aFVHyJUTa2eJTDgxvKDCT6MrAACiEljo9scbb7whb775pmRlZcmOHTuM26FDhyTaguU+KH083JHowVpo6/36+MN9XHkmwYILjRFuPb9p+WqKv26V+r3fIV0fjxZ5e4DIkf2h/VDGiaaL/GWq30ZXAABEvNzU4WegxCuvvCIDBw6Marmp9pHQfhHBvHVL+6CJg8FyNNwrI8rzw3If4Q4AfG3LePN+jtlzKLdmlsg7AyRs10wVOe3K8F8HAJASiiJRbhpGy4u4nf1hJkfDvargfVyej+OMsegbdhtNr/YeOmp6sqepygnNqfjwj0TYkGTmivSezEoFACAiqqTy7I9g/Sk8VxXM9mPQ79PSHD6DCl+TPS2VYWo56MHdYklmXZGz/+oKZ7Tqo0lncioAABGTsIFFuLM/zPSnCGlVIZKTVEMZGtZ7EqsTAICoSdjppuHOojDTn8K9qhDV1RTd7ti4SOTTh0UWPCyyYaHrPne/CbNIzgQAxEDCrlhYyX2I6qpCOKsp2uhqzlBjdscfHnXN8+j9pEirXq6x5R6NsPz6y8skZwIAoi6hA4twZlHYkaMRbDVF8zQcfipJKq2mBKr20EBjej+Rvq+L9JwoMr2/16t6qFZL5MoprFQAAGIiYbdCPLlzH/qc2cj4aqadt9n+FH67XgZhqT+FNrt6x0S57twRrlWLvq+5Vi48VcsSOX+kyMgtBBUAgJhJ+BWLUIW0qhCJ1RTd/tBmV2bs+8VVGaKBgwYYHkPDjLbcVHsAAGIsZQOLcHM0zApYSaJJmTo8LJTKEIaGAQDiUEoHFuHkaIRMg4lNS0QKPhcp3GIuEdOTlcoQAACiLOUDCyv9KcJmVH3cJXLIeglreR4Fk0gBAHGMwCLSdIVCcyF++FBk2bPhvdYVT5JHAQCIawQWkV6hmHufyL7wx7dLy8tE2lxlx1kBABAxBBaRDCq094QdOtwp0uNhe14LAIAIIrCIxNaHJmbOuj2816lWU6R1H5HLJ4tUqWbX2QEAEFEEFnavUmj5qNVKD09d7hVp1pW+FACAhERgYevWR4BW22ZkNRTpNoqAAgCQsJKipXfMlTe6CiOoUJdOJKgAACQ0Ags7aDlpONsfmbmuAWPM+AAAJDi2QuzgbrNtRZVMkQ6DRZqc72rNzUoFACAJEFjYIZQ221e9wAoFACDpEFiE0kXTe6KoftUx5kXbg+dZZDcS6TmBoAIAkJQILMIpJdVgoudEV5CgX42qEO8h7GXaD3Z1z6SMFACQxEjetFJK6p2gqSsUer8+rsFF39dEshtUXqHQxMye+eRSAACSHisWYZWS6n0OkXkjRVr1cgUX+tXXdgkAACmAwCLsUlKnSNE213HuFQn9CgBACmIrxK5S0lBKTgEASDKsWASq9rBSShpKySkAAEmGwCJYtUfQUlKH63E9DgCAFJeW0qPN540Smd4vcLWHrlxokGHQUlJPZd9rXwoSNAEASMHAQoOFSW1Epl4usuxZPweVrUxotYcGIX5LSRu67qfZFQAAKbgVYmm0uVe1B6WkAAAElTqBRaijzT2rPSglBQAgoNTZCgl1tDnVHgAAmJY6KxaW+0xQ7QEAgFWps2JhaeWBag8AAEKROoGFux9FpZJRH6j2AAAghbdCAnXOdHP3o2C0OQAAEVMl6TtnenL3o6h0fCPXtgcrFAAAhMXhdDot1l+Gp6ioSHJycqSwsFCys7Mj1JeibLvD33aGmRUOAABg+fpdJTn7Uuh9DlfnTG1q5WtbhH4UAADYLi15+1J4dM4EAABRkZb0fSks968AAACpF1iY7UtB50wAAKImLXn7UmjnzEZ0zgQAIIoSN7Bw96UweAcXdM4EACAWEjew8OxLkd2g4v10zgQAICYSt9zUTYMHLSmlLwUAADGX+IGFoi8FAABxIbG3QgAAQGIHFosXL5bevXtLw4YNxeFwyKxZsyJzZgAAIPkDiwMHDsgZZ5whzzzzTGTOCAAApE6OxaWXXmrcAAAAop68WVxcbNw8p6MBAIDkFPHkzfz8fGPMqvvWuHHjSL8lAABI1sBi1KhRxux2923r1q2RfksAAJCsWyEZGRnGDQAAJD/6WAAAgNitWOzfv1/Wr19f/n1BQYGsWrVKcnNz5cQTTwz6fKfTaXwliRMAgMThvm67r+N+OS367LPP9BUr3QYMGGDq+Vu3bvX5fG7cuHHjxo2bxP1Nr+OBOPR/JIpKS0vll19+kaysLKNzZziRk1aYaDJodna2recI3/jMo4/PPPr4zKOPzzwxPnMNF/bt22d03k5LS4ufIWR6MieccIJtr6cfCH8Ro4vPPPr4zKOPzzz6+Mzj/zPXthHBkLwJAABsQ2ABAABsk7CBhfbGGDNmDD0yoojPPPr4zKOPzzz6+MyT6zOPevImAABIXgm7YgEAAOIPgQUAALANgQUAALANgQUAALANgQUAAEiNwOKZZ56RJk2aSPXq1eW8886T5cuXBzz+7bffllatWhnHn3766fLhhx9G7VyThZXP/MUXX5QuXbpInTp1jFv37t2D/jdC+H/P3aZNm2a0xb/yyisjfo6p/pnv3btX7rjjDmnQoIFRnnfKKafw70sEP+9JkyZJy5YtJTMz02g7fffdd8vhw4ejdr6JbvHixdK7d2+j9bb+GzFr1qygz1m4cKG0a9fO+Pt98skny6uvvhr6CTjj1LRp05zVqlVzvvzyy861a9c6b7nlFmft2rWdv/76q8/jv/jiC2d6errzkUceca5bt87597//3Vm1alXn6tWro37uicrqZ3799dc7n3nmGec333zj/O6775wDBw505uTkOH/++eeon3uqfOZuBQUFzkaNGjm7dOni7NOnT9TONxU/8+LiYufZZ5/tvOyyy5xLliwxPvuFCxc6V61aFfVzT4XP+9///rczIyPD+Kqf9UcffeRs0KCB8+677476uSeqDz/80PnAAw84Z8yYYQwNmzlzZsDjN27c6KxRo4Zz+PDhxvXzqaeeMq6n8+bNC+n94zawOPfcc5133HFH+fclJSXOhg0bOvPz830e37dvX2evXr0q3Hfeeec5b7vttoifa7Kw+pl7O3bsmDMrK8s5derUCJ5lcgnlM9fPuWPHjs6XXnrJmCpMYBHZz3zKlCnOZs2aOY8cORLFs0zdz1uPvfDCCyvcpxe8Tp06Rfxck5GYCCzuu+8+52mnnVbhvmuvvdbZo0ePkN4zLrdCjhw5Il9//bWxtO45vEy///LLL30+R+/3PF716NHD7/EI/zP3dvDgQTl69Kjk5uZG8EyTR6if+YMPPij169eXm266KUpnmtqf+ezZs6VDhw7GVsjxxx8vbdq0kfHjx0tJSUkUzzx1Pu+OHTsaz3Fvl2zcuNHYdrrsssuidt6p5kubr59Rn25qxq5du4z/0+r/iT3p999//73P5+zYscPn8Xo/IvOZexsxYoSxp+f9FxT2feZLliyRf/3rX7Jq1aoonWVyCeUz1wvbggUL5IYbbjAucOvXr5fBgwcbQbS2RIa9n/f1119vPK9z587GmO5jx47J7bffLvfff3+Uzjr17PBz/dTR6ocOHTJyXayIyxULJJ4JEyYYyYQzZ840ErRgv3379km/fv2MpNl69erF+nRSRmlpqbFC9MILL8if/vQnufbaa+WBBx6Q5557LtanlpQ0iVBXhJ599llZuXKlzJgxQz744AN56KGHYn1qSOQVC/1HMz09XX799dcK9+v3eXl5Pp+j91s5HuF/5m6PPfaYEVjMnz9f2rZtG+EzTd3PfMOGDbJp0yYj29vzoqeqVKkiP/zwgzRv3jwKZ55af8+1EqRq1arG89xOPfVU47c8XeqvVq1axM87lT7v0aNHGwH0zTffbHyvFX4HDhyQW2+91QjodCsF9vJ3/czOzra8WqHi8r+Q/h9VfzP49NNPK/wDqt/rXqcver/n8eqTTz7xezzC/8zVI488YvwmMW/ePDn77LOjdLap+ZlrKfXq1auNbRD37YorrpBu3boZf9ayPNj/97xTp07G9oc7iFM//vijEXAQVNj/eWuulnfw4A7qmJkZGbZfP51xXKKkJUevvvqqUf5y6623GiVKO3bsMB7v16+fc+TIkRXKTatUqeJ87LHHjNLHMWPGUG4a4c98woQJRhnZO++849y+fXv5bd++fTH8KZL7M/dGVUjkP/MtW7YY1U5Dhgxx/vDDD87333/fWb9+fefDDz8cw58ieT9v/bdbP++33nrLKIP8+OOPnc2bNzcq/2CO/husbQD0ppf5J554wvjz5s2bjcf189bP3bvc9N577zWun9pGICnLTZXW0p544onGxUtLlpYtW1b+WNeuXY1/VD1Nnz7decoppxjHa+nMBx98EIOzTmxWPvOTTjrJ+EvrfdN/GBC5v+eeCCyi85kvXbrUKF/XC6SWnv7jH/8wyn5h/+d99OhR59ixY41gonr16s7GjRs7Bw8e7NyzZ0+Mzj7xfPbZZz7/bXZ/zvpVP3fv55x55pnGfyP9O/7KK6+E/P4O/R97FlMAAECqi8scCwAAkJgILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgNjl/wOB6vqgznVLNAAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "n = 100\n", + "x = np.random.rand(n, 1)\n", + "y = 2.0 + 5 * x**2 + 0.1 * np.random.randn(n, 1)\n", + "\n", + "line_model = LinearRegression().fit(x, y)\n", + "line_predict = line_model.predict(x)\n", + "#line_mse = ...\n", + "\n", + "#poly_features = ...\n", + "#poly_model = LinearRegression().fit(..., y)\n", + "#poly_predict = ...\n", + "#poly_mse = ...\n", + "\n", + "plt.scatter(x, y, label = \"Data\")\n", + "plt.scatter(x, line_predict, label = \"Line model\")\n", + "plt.legend()\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "id": "248d8931", + "metadata": {}, + "source": [ + "## Exercise 4 - The train-test split\n" + ] + }, + { + "cell_type": "markdown", + "id": "1efd3376", + "metadata": {}, + "source": [ + "Hopefully your model fit the data quite well, but to know how well the model actually generalizes to unseen data, which is most often what we care about, we need to split our data into training and testing data. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0f8d75fb", + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.model_selection import train_test_split" + ] + }, + { + "cell_type": "markdown", + "id": "edb213fc", + "metadata": {}, + "source": [ + "**a)** Complete the code below so that the polynomial features and the targets y get split into training and test data.\n", + "\n", + "**b)** What is the shape of X_test?\n", + "\n", + "**c)** Fit your model to X_train\n", + "\n", + "**d)** Compute the MSE when your model predicts on the training data and on the testing data, using y_train and y_test as targets for the two cases.\n", + "\n", + "**e)** Why do we not fit the model to X_test?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a03e0388", + "metadata": {}, + "outputs": [], + "source": [ + "polynomial_features = ...\n", + "\n", + "#X_train, X_test, y_train, y_test = train_test_split(polynomial_features, y, test_size=0.2)\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "22e7536e", + "metadata": {}, + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.15" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/doc/LectureNotes/_build/html/_sources/exercisesweek35.ipynb b/doc/LectureNotes/_build/html/_sources/exercisesweek35.ipynb new file mode 100644 index 000000000..8ce6a6c3d --- /dev/null +++ b/doc/LectureNotes/_build/html/_sources/exercisesweek35.ipynb @@ -0,0 +1,539 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "b4005770", + "metadata": {}, + "source": [ + "# Exercises week 35\n", + "\n", + "## Deriving and Implementing Ordinary Least Squares" + ] + }, + { + "cell_type": "markdown", + "id": "2ca1b589", + "metadata": {}, + "source": [ + "This week you will be deriving the analytical expressions for linear regression, building up the model from scratch. This will include taking several derivatives of products of vectors and matrices. Such derivatives are central to the optimization of many machine learning models. Although we will often use automatic differentiation in actual calculations, to be able to have analytical expressions is extremely helpful in case we have simpler derivatives as well as when we analyze various properties (like second derivatives) of the chosen cost functions.\n", + "\n", + "Vectors are always written as boldfaced lower case letters and matrices as upper case boldfaced letters. You will find useful the notes from week 35 on derivatives of vectors and matrices. See also the textbook of Faisal at al, chapter 5 and in particular sections 5.3-5.5 at " + ] + }, + { + "cell_type": "markdown", + "id": "41e92bf9", + "metadata": {}, + "source": [ + "### Learning goals\n", + "\n", + "After completing these exercises, you will know how to\n", + "- Take the derivatives of simple products between vectors and matrices\n", + "- Implement OLS using the analytical expressions\n", + "- Create a feature matrix from a set of data\n", + "- Create a feature matrix for a polynomial model\n", + "- Evaluate the MSE score of various model on training and test data, and comparing their performance\n", + "\n", + "### Deliverables\n", + "\n", + "Complete the following exercises while working in a jupyter notebook. Then, in canvas, include\n", + "- The jupyter notebook with the exercises completed\n", + "- An exported PDF of the notebook (https://code.visualstudio.com/docs/datascience/jupyter-notebooks#_export-your-jupyter-notebook)" + ] + }, + { + "cell_type": "markdown", + "id": "f7a9209d", + "metadata": {}, + "source": [ + "## How to take derivatives of Matrix-Vector expressions" + ] + }, + { + "cell_type": "markdown", + "id": "45f3712e", + "metadata": {}, + "source": [ + "In these exercises it is always useful to write out with summation indices the various quantities. Take also a look at the weekly slides from week 35 and the various examples included there.\n", + "\n", + "As an example, consider the function\n", + "\n", + "$$\n", + "f(\\boldsymbol{x}) =\\boldsymbol{A}\\boldsymbol{x},\n", + "$$\n", + "\n", + "which reads for a specific component $f_i$ (we define the matrix $\\boldsymbol{A}$ to have dimension $n\\times n$ and the vector $\\boldsymbol{x}$ to have length $n$)\n", + "\n", + "$$\n", + "f_i =\\sum_{j=0}^{n-1}a_{ij}x_j,\n", + "$$\n", + "\n", + "which leads to\n", + "\n", + "$$\n", + "\\frac{\\partial f_i}{\\partial x_j}= a_{ij},\n", + "$$\n", + "\n", + "and written out in terms of the vector $\\boldsymbol{x}$ we have\n", + "\n", + "$$\n", + "\\frac{\\partial f(\\boldsymbol{x})}{\\partial \\boldsymbol{x}}= \\boldsymbol{A}.\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "5fa8a4e6", + "metadata": {}, + "source": [ + "## Exercise 1 - Finding the derivative of Matrix-Vector expressions" + ] + }, + { + "cell_type": "markdown", + "id": "df7a2270", + "metadata": {}, + "source": [ + "**a)** Consider the expression\n", + "\n", + "$$\n", + "\\frac{\\partial (\\boldsymbol{a}^T\\boldsymbol{x})}{\\partial \\boldsymbol{x}},\n", + "$$\n", + "\n", + "Where $\\boldsymbol{a}$ and $\\boldsymbol{x}$ are column-vectors with length $n$.\n", + "\n", + "What is the *shape* of the expression we are taking the derivative of?\n", + "\n", + "What is the *shape* of the thing we are taking the derivative with respect to?\n", + "\n", + "What is the *shape* of the result of the expression?" + ] + }, + { + "cell_type": "markdown", + "id": "c0396734", + "metadata": {}, + "source": [ + "**b)** Show that\n", + "\n", + "$$\n", + "\\frac{\\partial (\\boldsymbol{a}^T\\boldsymbol{x})}{\\partial \\boldsymbol{x}} = \\boldsymbol{a}^T,\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "dc39d541", + "metadata": {}, + "source": [ + "**c)** Show that\n", + "\n", + "$$\n", + "\\frac{\\partial (\\boldsymbol{a}^T\\boldsymbol{A}\\boldsymbol{a})}{\\partial \\boldsymbol{a}} = \\boldsymbol{a}^T(\\boldsymbol{A}+\\boldsymbol{A}^T),\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "498d13ec", + "metadata": {}, + "source": [ + "## Exercise 2 - Deriving the expression for OLS" + ] + }, + { + "cell_type": "markdown", + "id": "f3f771de", + "metadata": {}, + "source": [ + "The ordinary least squares method finds the parameters $\\boldsymbol{\\theta}$ which minimizes the squared error between our model $\\boldsymbol{X\\theta}$ and the true values $\\boldsymbol{y}$.\n", + "\n", + "To find the parameters $\\boldsymbol{\\theta}$ which minimizes this error, we take the derivative of the squared error expression with respect to $\\boldsymbol{\\theta}$, and set it equal to 0." + ] + }, + { + "cell_type": "markdown", + "id": "49690237", + "metadata": {}, + "source": [ + "**a)** Very briefly explain why the approach above finds the parameters $\\boldsymbol{\\theta}$ which minimizes this error." + ] + }, + { + "cell_type": "markdown", + "id": "b7cccc9d", + "metadata": {}, + "source": [ + "We typically write the squared error as\n", + "\n", + "$$\n", + "\\vert\\vert\\boldsymbol{y} - \\boldsymbol{X\\theta}\\vert\\vert^2\n", + "$$\n", + "\n", + "which we can rewrite in matrix-vector form as\n", + "\n", + "$$\n", + "\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "8fbecf74", + "metadata": {}, + "source": [ + "**b)** If $\\boldsymbol{X}$ is invertible, what is the expression for the optimal parameters $\\boldsymbol{\\theta}$? (**Hint:** Don't compute any derivatives, but solve $\\boldsymbol{X\\theta}=\\boldsymbol{y}$ for $\\boldsymbol{\\theta}$)" + ] + }, + { + "cell_type": "markdown", + "id": "f37af8f0", + "metadata": {}, + "source": [ + "**c)** Show that\n", + "\n", + "$$\n", + "\\frac{\\partial \\left(\\boldsymbol{x}-\\boldsymbol{A}\\boldsymbol{s}\\right)^T\\left(\\boldsymbol{x}-\\boldsymbol{A}\\boldsymbol{s}\\right)}{\\partial \\boldsymbol{s}} = -2\\left(\\boldsymbol{x}-\\boldsymbol{A}\\boldsymbol{s}\\right)^T\\boldsymbol{A},\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "869fca4d", + "metadata": {}, + "source": [ + "**d)** Using the expression from **c)**, but substituting back in $\\boldsymbol{\\theta}$, $\\boldsymbol{y}$ and $\\boldsymbol{X}$, find the expression for the optimal parameters $\\boldsymbol{\\theta}$ in the case that $\\boldsymbol{X}$ is not invertible, but $\\boldsymbol{X^T X}$ is, which is most often the case.\n", + "\n", + "$$\n", + "\\boldsymbol{\\hat{\\theta}_{OLS}} = ...\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "57ca3d74", + "metadata": {}, + "source": [ + "## Exercise 3 - Creating feature matrix and implementing OLS using the analytical expression" + ] + }, + { + "cell_type": "markdown", + "id": "5dc179f7", + "metadata": {}, + "source": [ + "With the expression for $\\boldsymbol{\\hat{\\theta}_{OLS}}$, you now have what you need to implement OLS regression with your input data and target data $\\boldsymbol{y}$. But before you can do that, you need to set up you input data as a feature matrix $\\boldsymbol{X}$.\n", + "\n", + "In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, with the montly income and the number of children as columns.\n", + "\n", + "We typically also include an intercept in our models. The intercept is a value that is added to our prediction regardless of the value of the other features. The intercept tries to account for constant effects in our data that are not dependant on anything else. In our current example, the intercept could account for living expenses which are typical regardless of income or childcare expenses.\n", + "\n", + "We calculate the optimal intercept by including a feature with the constant value of 1 in our model, which is then multplied by some parameter $\\theta_0$ from the OLS method into the optimal intercept value (which will be $\\theta_0$). In practice, we include the intercept in our model by adding a column of ones to the start of our feature matrix." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "e5ff2a69", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "a3cf2792", + "metadata": {}, + "outputs": [], + "source": [ + "n = 20\n", + "income = np.array([116., 161., 167., 118., 172., 163., 179., 173., 162., 116., 101., 176., 178., 172., 143., 135., 160., 101., 149., 125.])\n", + "children = np.array([5, 3, 0, 4, 5, 3, 0, 4, 4, 3, 3, 5, 1, 0, 2, 3, 2, 1, 5, 4])\n", + "spending = np.array([152., 141., 102., 136., 161., 129., 99., 159., 160., 107., 98., 164., 121., 93., 112., 127., 117., 69., 156., 131.])\n" + ] + }, + { + "cell_type": "markdown", + "id": "5da61481", + "metadata": {}, + "source": [ + "**a)** Create a feature matrix $\\boldsymbol{X}$ for the features income and children, including an intercept column of ones at the start." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "5ad87a65", + "metadata": {}, + "outputs": [], + "source": [ + "X = np.zeros((n, 3))\n", + "#X[:, 0] = ...\n", + "#X[:, 1] = ...\n", + "#X[:, 2] = ..." + ] + }, + { + "cell_type": "markdown", + "id": "e0ddfac2", + "metadata": {}, + "source": [ + "**b)** Use the expression from **3d)** to find the optimal parameters $\\boldsymbol{\\hat{\\beta}_{OLS}}$ for predicting spending based on these features. Create a function for this operation, as you are going to need to use it a lot." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "8f3f68aa", + "metadata": {}, + "outputs": [], + "source": [ + "def OLS_parameters(X, y):\n", + " return ...\n", + "\n", + "#beta = OLS_parameters(X, y)" + ] + }, + { + "cell_type": "markdown", + "id": "0cb6da80", + "metadata": {}, + "source": [ + "## Exercise 4 - Fitting a polynomial" + ] + }, + { + "cell_type": "markdown", + "id": "71015064", + "metadata": {}, + "source": [ + "In this course, we typically do linear regression using polynomials, though in real world applications it is also very common to make linear models based on measured features like you did in the previous exercise.\n", + "\n", + "When fitting a polynomial with linear regression, we make each polynomial degree($x, x^2, x^3, ..., x^p$) its own feature." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "d7476c84", + "metadata": {}, + "outputs": [], + "source": [ + "n = 100\n", + "x = np.linspace(-3, 3, n)\n", + "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2) + np.random.normal(0, 0.1)" + ] + }, + { + "cell_type": "markdown", + "id": "8321451b", + "metadata": {}, + "source": [ + "**a)** Create a feature matrix $\\boldsymbol{X}$ for the features $x, x^2, x^3, x^4, x^5$, including an intercept column of ones at the start. Make this into a function, as you will do this a lot over the next weeks." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "91496e40", + "metadata": {}, + "outputs": [], + "source": [ + "def polynomial_features(x, p):\n", + " n = len(x)\n", + " X = np.zeros((n, p + 1))\n", + " #X[:, 0] = ...\n", + " #X[:, 1] = ...\n", + " #X[:, 2] = ...\n", + " # could this be a loop?\n", + "\n", + "#X = polynomial_features(x, 5)" + ] + }, + { + "cell_type": "markdown", + "id": "b84b1e31", + "metadata": {}, + "source": [ + "**b)** Use the expression from **3d)** to find the optimal parameters $\\boldsymbol{\\hat{\\beta}_{OLS}}$ for predicting $\\boldsymbol{y}$ based on these features. If you have done everything right so far, this code will not need changing." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "034f502c", + "metadata": {}, + "outputs": [], + "source": [ + "#beta = OLS_parameters(X, y)" + ] + }, + { + "cell_type": "markdown", + "id": "d703f788", + "metadata": {}, + "source": [ + "**c)** Like in exercise 4 last week, split your feature matrix and target data into a training split and test split." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "29171358", + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.model_selection import train_test_split\n", + "\n", + "#X_train, X_test, y_train, y_test = ..." + ] + }, + { + "cell_type": "markdown", + "id": "a0e3509f", + "metadata": {}, + "source": [ + "**d)** Train your model on the training data(find the parameters which best fit) and compute the MSE on both the training and test data." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "1e346f4c", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Ellipsis" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "..." + ] + }, + { + "cell_type": "markdown", + "id": "7e431889", + "metadata": {}, + "source": [ + "**e)** Do the same for each polynomial degree from 2 to 10, and plot the MSE on both the training and test data as a function of polynomial degree. The aim is to reproduce Figure 2.11 of [Hastie et al](https://github.com/CompPhysics/MLErasmus/blob/master/doc/Textbooks/elementsstat.pdf). Feel free to read the discussions leading to figure 2.11 of Hastie et al. " + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "ceb57457", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Ellipsis" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "..." + ] + }, + { + "cell_type": "markdown", + "id": "5e5b5954", + "metadata": {}, + "source": [ + "**f)** Interpret the graph. Why do the lines move as they do? What does it tell us about model performance and generalizability?" + ] + }, + { + "cell_type": "markdown", + "id": "ad2acfb9", + "metadata": {}, + "source": [] + }, + { + "cell_type": "markdown", + "id": "5994f0c5", + "metadata": {}, + "source": [ + "## Exercise 5 - Comparing your code with sklearn" + ] + }, + { + "cell_type": "markdown", + "id": "8f595b7a", + "metadata": {}, + "source": [ + "When implementing different algorithms for the first time, it can be helpful to double check your results with established implementations before you go on to add more complexity." + ] + }, + { + "cell_type": "markdown", + "id": "8ab310c1", + "metadata": {}, + "source": [ + "**a)** Make sure your `polynomial_features` function creates the same feature matrix as sklearns PolynomialFeatures.\n", + "\n", + "(https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "85b964d1", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "73c32c52", + "metadata": {}, + "source": [ + "**b)** Make sure your `OLS_parameters` function computes the same parameters as sklearns LinearRegression with fit_intercept set to False, since the intercept is included in the feature matrix. Use `your_model_object.coef_` to extract the computed parameters.\n", + "\n", + "(https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "35b04126", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.15" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/doc/LectureNotes/_build/html/_sources/exercisesweek36.ipynb b/doc/LectureNotes/_build/html/_sources/exercisesweek36.ipynb new file mode 100644 index 000000000..e3365bb50 --- /dev/null +++ b/doc/LectureNotes/_build/html/_sources/exercisesweek36.ipynb @@ -0,0 +1,383 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "9dd8f8d0", + "metadata": {}, + "source": [ + "# Weekly Exercises 3\n", + "\n", + "## Deriving and Implementing Ridge Regression" + ] + }, + { + "cell_type": "markdown", + "id": "e5cc5739", + "metadata": {}, + "source": [ + "## Learning goals\n", + "\n", + "After completing these exercises, you will know how to\n", + "- Take more derivatives of simple products between vectors and matrices\n", + "- Implement Ridge regression using the analytical expressions\n", + "- Scale data appropriately for linear regression\n", + "- Evaluate a model across two different hyperparameters" + ] + }, + { + "cell_type": "markdown", + "id": "427d9b9d", + "metadata": {}, + "source": [ + "## Exercise 1 - Choice of model and degrees of freedom" + ] + }, + { + "cell_type": "markdown", + "id": "ff6ebdb5", + "metadata": {}, + "source": [ + "**a)** How many degrees of freedom does an OLS model fit to the features $x, x^2, x^3$ and the intercept have?\n", + "\n", + "**b)** Why is it bad for a model to have too many degrees of freedom?\n", + "\n", + "**c)** Why is it bad for a model to have too few degrees of freedom?\n", + "\n", + "**d)** Read [chapter 3.4.1 of Hastie et al.'s book](https://link.springer.com/book/10.1007/978-0-387-84858-7). What is the expression for the effective degrees of freedom of the ridge regression fit?\n", + "\n", + "**e)** Why might we want to use Ridge regression instead of OLS?\n", + "\n", + "**f)** Why migth we want to use OLS instead of Ridge regression?" + ] + }, + { + "cell_type": "markdown", + "id": "a78b2ce2", + "metadata": {}, + "source": [ + "## Exercise 2 - Deriving the expression for Ridge Regression\n", + "\n", + "The aim here is to derive the expression for the optimal parameters using Ridge regression.\n", + "\n", + "The expression for the standard Mean Squared Error (MSE) which we used to define our cost function and the equations for the ordinary least squares (OLS) method, was given by the optimization problem" + ] + }, + { + "cell_type": "markdown", + "id": "93a7ad16", + "metadata": {}, + "source": [ + "$$\n", + "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in {\\mathbb{R}}^{p}}}\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\\right\\}.\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "6b182c01", + "metadata": {}, + "source": [ + "By minimizing the above equation with respect to the parameters $\\boldsymbol{\\beta}$ we could then obtain an analytical expression for the parameters $\\boldsymbol{\\hat\\beta_{OLS}}$.\n", + "\n", + "We can add a regularization parameter $\\lambda$ by\n", + "defining a new cost function to be optimized, that is" + ] + }, + { + "cell_type": "markdown", + "id": "fb065daf", + "metadata": {}, + "source": [ + "$$\n", + "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n", + "{\\mathbb{R}}^{p}}}\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2+\\lambda\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "89ae11e8", + "metadata": {}, + "source": [ + "which leads to the Ridge regression minimization problem. (One can require as part of the optimization problem that $\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\\le t$, where $t$ is a finite number larger than zero. We will not implement that in this course.)" + ] + }, + { + "cell_type": "markdown", + "id": "456d58d4", + "metadata": {}, + "source": [ + "### a) Expression for Ridge regression\n", + "\n", + "Show that the optimal parameters" + ] + }, + { + "cell_type": "markdown", + "id": "a5677efd", + "metadata": {}, + "source": [ + "$$\n", + "\\hat{\\boldsymbol{\\beta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "e7b08048", + "metadata": {}, + "source": [ + "with $\\boldsymbol{I}$ being a $p\\times p$ identity matrix.\n", + "\n", + "The ordinary least squares result is" + ] + }, + { + "cell_type": "markdown", + "id": "8bb0e6af", + "metadata": {}, + "source": [ + "$$\n", + "\\hat{\\boldsymbol{\\beta}}_{\\mathrm{OLS}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "2faaae78", + "metadata": {}, + "source": [ + "## Exercise 3 - Scaling data" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "8ed79466", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "from sklearn.model_selection import train_test_split\n", + "from sklearn.preprocessing import StandardScaler" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "id": "59d6f347", + "metadata": {}, + "outputs": [], + "source": [ + "n = 100\n", + "x = np.linspace(-3, 3, n)\n", + "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2) + np.random.normal(0, 0.1)" + ] + }, + { + "cell_type": "markdown", + "id": "c9fc9fe0", + "metadata": {}, + "source": [ + "**a)** Adapt your function from last week to only include the intercept column if the boolean argument `intercept` is set to true." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "id": "5aa9c86d", + "metadata": {}, + "outputs": [], + "source": [ + "def polynomial_features(x, p, intercept=False):\n", + " n = len(x)\n", + " X = np.zeros((n, p + 1))\n", + " #X[:, 0] = ...\n", + " #X[:, 1] = ...\n", + " #X[:, 2] = ...\n", + " # could this be a loop?" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "id": "2ebf249a", + "metadata": {}, + "outputs": [], + "source": [ + "def polynomial_features(x, p, intercept=False):\n", + " n = len(x)\n", + " X = np.zeros((n, p))\n", + " X[:, 0] = x[:]\n", + " X[:, 1] = x**2\n", + " X[:, 2] = x**3\n", + " return X" + ] + }, + { + "cell_type": "markdown", + "id": "c5924634", + "metadata": {}, + "source": [ + "**b)** Split your data into training and test data(80/20 split)" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "id": "676541e7", + "metadata": {}, + "outputs": [], + "source": [ + "X = polynomial_features(x, 3)" + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "id": "f895606f", + "metadata": {}, + "outputs": [], + "source": [ + "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n", + "x_train = X_train[:, 0] # These are used for plotting\n", + "x_test = X_test[:, 0] # These are used for plotting" + ] + }, + { + "cell_type": "markdown", + "id": "a58f05ad", + "metadata": {}, + "source": [ + "**c)** Scale your design matrix with the sklearn standard scaler, though based on the mean and standard deviation of the training data only." + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "id": "0c07995a", + "metadata": {}, + "outputs": [], + "source": [ + "scaler = StandardScaler()\n", + "scaler.fit(X_train)\n", + "X_train_s = scaler.transform(X_train)\n", + "X_test_s = scaler.transform(X_test)\n", + "y_offset = np.mean(y_train)" + ] + }, + { + "cell_type": "markdown", + "id": "63b86003", + "metadata": {}, + "source": [ + "## Exercise 4 - Implementing Ridge Regression" + ] + }, + { + "cell_type": "markdown", + "id": "a2f52ef2", + "metadata": {}, + "source": [ + "**a)** Implement a function for computing the optimal Ridge parameters using the expression from **2a)**." + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "id": "69f19bde", + "metadata": {}, + "outputs": [], + "source": [ + "def Ridge_parameters(X, y):\n", + " # Assumes X is scaled and has no intercept column\n", + " return np.linalg.inv(X.T @ X) @ X.T @ y\n", + "\n", + "beta = Ridge_parameters(X_train_s, y_train)" + ] + }, + { + "cell_type": "markdown", + "id": "626c81ba", + "metadata": {}, + "source": [ + "**b)** Fit a model to the data, and plot the prediction using both the training and test x-values extracted before scaling, and the y_offset." + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "id": "f9ba0655", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 82, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAXxtJREFUeJzt3Qd409X6B/Bv0r2hm0JpS8tGyixbQVmKuK8IKIjryhUX/3sv4ICLC/V6Fa8gXvdEwAWoyF4yy1Q2FAqFlm66IW2T/J9z0pSOtE1Kdr6f58nTX9JfmtNQkjfnvO97FFqtVgsiIiIiG1Ha6oGJiIiIBAYjREREZFMMRoiIiMimGIwQERGRTTEYISIiIptiMEJEREQ2xWCEiIiIbIrBCBEREdmUOxyARqNBRkYGAgICoFAobD0cIiIiMoLoq1pcXIyoqCgolUrHDkZEIBIdHW3rYRAREVEznD9/Hm3atHHsYETMiOh/mcDAQFsPh4iIiIxQVFQkJxP07+MOHYzol2ZEIMJghIiIyLE0lWLBBFYiIiKyKQYjREREZFMMRoiIiMimGIwQERGRTTEYISIiIptiMEJEREQ2xWCEiIiIbIrBCBEREdkUgxEiIiKyKQYjREREZFMMRoiIiMimGIwQERGRTTnERnlERESNUVWqcSCtAMmp+fByVyIxugWuax0EPy++zTkC/isREZFDyi6+gp/2p2P76TzsSc3H5Qp1re8rFUCHiAD0immJx6+PR9sQX5uNlcy8TLN161aMHTsWUVFRckvg5cuXN3kflUqF559/HjExMfDy8kJsbCw+/fRTUx+aiIhIWnMkEyPf2Yp5vx3H1pM5MhAJ9ffErd1b4eZukWgV5A2NFjieWYzFu9Nw87tbsXRPGrRara2HTuaYGSktLUViYiIeeugh3HXXXUbd595770VWVhY++eQTJCQk4OLFi9BoNKY+NBERubjL5Wq8/OtRGWAInSIDcE/vNhjcPhQdIwLkh2S9rKIr+ON8AT7eliqXb2b8cAjrjmZh3l3dERbgZcPfgupSaK8hTBT/6D/99BPuuOOOBs9ZvXo17rvvPpw5cwbBwcHNepyioiIEBQWhsLAQgYGBzR0uERE5sCMZhXjq2wM4nVMqr//1+nb4v5Ed4ene+CS/WqPFJ9vO4K01J1Gu1iDEzxNv3tMdN3WOsNLIXVeRke/fFq+mWblyJfr06YM333wTrVu3RocOHfD3v/8dly9fbnRZR/wCNS9EROS6tp3KxZ0Ld8hAJDzAC18/3A+zbuncZCAiuCkVeOz6eKyYNkjOpOSVluOxr/Zhy8kcq4ydmmbxYETMiGzbtg2HDx+Wsyjz58/H999/j7/97W8N3mfevHkyktJfoqOjLT1MIiKyU8cuFuHxr/fJWY2hHcOw+pnr5bKMqTq3CpQBye09ouRsyRPf7MeJzGKLjJnsLBgRuSFiOeebb75BUlISbrnlFrz99tv44osvGpwdmTVrlpzS0V/Onz9v6WESEZEdulh4GVM+24MSVSX6xQXjfw/0RrCfZ7N/npe7m1yiSYoLlj/zoc/3yKoccvJgpFWrVnJ5Rsxw6HXu3FlmNF+4cMHgfUTFjVhbqnkhIiLXUnSlAg9+ugeZRVfQPtwfHz7QRwYT10r8jP/d3xtxoX5IL7iMR7/chyt1yoLJyYKRQYMGISMjAyUlJdW3nTx5EkqlEm3atLH0wxMRkQMqr9Rg6tf7cCKrWFa+fDalL4J8Pcz281v6eeLTB/uiha+HrLiZvuwgNKIWmBwjGBFBxcGDB+VFSE1NlcdpaWnVSyyTJk2qPn/ChAkICQnBlClTcPToUdmn5B//+IcsDfbx8THn70JERE5i9orD2J6SBz9PN3z2YF+0aWn+hmViZkTMkHi4KbDqUCYWbTlt9scgCwUje/fuRc+ePeVFmD59ujyePXu2vC56iOgDE8Hf3x/r1q1DQUGBrKqZOHGibJr23//+19SHJiIiF7DxeBaW7DkvO6gunNgL3VpfXeY3t37tQvDKHd3k8X83nEJaXpnFHoss1GfEWthnhIjIdfJERr69VeaJPDokDs+P6WLxxxRvg/d/slvOxIhqHTETU7N5GjlBnxEiIiJjvfbrMRmIxIb4YvqIjlZ5TBF4vHR7N3i6KbH5RA5WH860yuPSVQxGiIjILvx+Kkcuzwhv3pMIH89rr5wxVnyYP/56Qzt5PPfno7Lsl6yHwQgREdmcePOf+cMheTx5QIzsA2JtTwxLQNtgXzkz8+76k1Z/fFfGYISIiGzujd+Oy54fbVr64J+jO9lkDN4ebph7e1d5/On2s7LzK1kHgxEiIrKpPWfz8dWuc/L4jbu7w8/L5A3lzWZYx3CM7hop28W/sPywTG4ly2MwQkRENiPe7F/99Zg8HtcnGoMSTN9zxtzm3NYFvp5u2HfuEjYcy7b1cFwCgxEiIrIZUbly8HyBfPP/v1EdbD0cqVWQDyYPjJXH721K4eyIFTAYISIim6hQa/DmmhPy+JEh7RAe4A178fDgOHh7KGWr+G0pubYejtNjMEJERDYhynhTc0sR4ueJx67XldXai1B/L0xIipHH721MsfVwnB6DESIisrpSVSXeXX9KHj91U3v42zBptSEiQBKN0JJT87H7TJ6th+PUGIwQEZHVffx7KnJLVIgJ8cX4pLawR5FB3vhLH93u8gs2cXbEkhiMEBGRVYkg5MOtuh1y/zGqIzzd7fet6PEb4uGmVOD3U7kyf4Qsw37/AoiIyCmJ3XFLy9VIbBOEMde1gj2LDvbFnT1by2POjlgOgxEiIrKai4WX8W1ymjyeeXNnh9gdd+rQeIhhrjuaxa6sFsJghIiIrOajramoUGvRv10wBsSHwBGITfT0MzgfbNEtL5F5MRghIiKryC8tr54V+dvQBDgSkTsi/HYoE3klKlsPx+kwGCEiIqv4fMdZXK5Qo1vrQAxpb/u276bo1joIidEtUK7W4Pt9F2w9HKfDYISIiCyuRFWJL3acrZ4VcYRckbomVpUgL05Og0bDFvHmxGCEiIgsbvHucyi8XIF2YX4Y1TUSjujWxFYI8HbHubwybD/NFvHmxGCEiIgsSlWplk3OavbtcES+nu64u5euCdo3u3S5L2QeDEaIiMiiftiXjuxiFVoFeeOOHrqeHY5qQj/dUs26Y1nIKrpi6+E4DQYjRERkMZVqDf5X1W310SHt7LrbqjE6RAQgKTYYao0WS/ect/VwnIZj/1UQEZFdW3U4U+ZYBPt54r6kaDiDif11syOiTFkEW3TtGIwQEZHFfLJNlysyeUCszLlwBqO7Rcrg6mLhFWw+kWPr4TgFBiNERGQR+9Muyc3lxNLM/VWzCc7Ay90N9/SuSmTdfc7Ww3EKDEaIiMgiPt+u6ytyW2IUQvy94EzGV/Uc2XwyB+fzy2w9HIfHYISIiMwus/AKVh26KI+nDIqFs4kL9cOAdiHQaoEVB9NtPRyHx2CEiIjM7utd51Cp0SIpLhhdo4LgjO7sqStTXn4wA1oRlVCzMRghIiKzulKhli3ThYeccFZEb/R1kTIfJiW7BEcyimw9HIfGYISIiMxq5cEMuUNv6xY+GN45As4q0NsDN3UKl8dcqrk2DEaIiMhsxHLFZ1Ub4k0aEAN3N+d+m7m9qqPsyj8yZCM0ah7n/ishIiKr2p2aj2MXi+Dj4Yb7+jpPOW9DhnUKQ6C3O7KKVNh9Js/Ww3FYDEaIiMhsPtuua3J2V6/WCPL1gLMTPUduua6VPF7OpRrrBSNbt27F2LFjERUVBYVCgeXLlxt93+3bt8Pd3R09evQw9WGJiMjOpRdcxrqjWfL4wYHOm7ja0FLNb4cyZfIuWSEYKS0tRWJiIhYuXGjS/QoKCjBp0iTcdNNNpj4kERE5gG93p0GkTQyMD0H7iAC4in5xwXJH4mJVJTafyLb1cFwjGLn55pvxyiuv4M477zTpfo8//jgmTJiAAQMGmPqQRERk58orNVhStYvt/f1j4EqUSgVu6xElj5cfyLD1cBySVXJGPvvsM5w5cwZz5swx6nyVSoWioqJaFyIisiMaNZD6O3Doe/l17eEM5Jao0MLXQy5V7DydJ6tLxEUci9JX8VUELTWvGzrHEatS7qhaqtl4PBuFlytsPRyHY/EtFE+dOoWZM2fi999/l/kixpg3bx7mzp1r6aEREZEpwce5HUBJFpB3Gtj3GVCsa/cufFn5EoAEFJRVYPqyP+RtIjARxG16SgXkUo6eoXNaB3rgnf5lSAqrBPwjgJiBgNIN9qxzq0B0jAjAiaxirD58EeNcoJLIYYIRtVotl2ZEYNGhQwej7zdr1ixMnz69+rqYGYmOjrbQKImIqFFHVwKrZwBFhpcgUjRRSK5MgBIaDFb8ia3aHvUCDL26kx51zxmlTMYc1ZeI2pp/9cbAKGDUPMA3RBcM2WmAcnvPKLy5+gRWHMxgMGJPwUhxcTH27t2LAwcOYNq0afI2jUYjm+KIWZK1a9fixhtvrHc/Ly8veSEiIusTyyTJqfnILr6CTpc2o8OWJ6BAw0snX6uHy683KvfjJY8vMFj1LjTNyAIQgcgij/n1vyGCoO8m175NBCij3wC63AZ7Mba7LhjZdSZPdqAN9vO09ZAchkWDkcDAQBw6dKjWbe+//z42btyI77//HnFxcZZ8eCIiMjH4OJtbhm+T05BZdEXOdGzzegFahRaKBu5bpvXCD+oh8vh+t/WIUuQhSXkcuzRdTBqDeKw5Hl/qjht6sBq0RReBZZOwJ2k+1B3Hyg353Iy5owVFB/uia1Sg3Kdm/dEs3NuXM/oWC0ZKSkqQkpJSfT01NRUHDx5EcHAw2rZtK5dY0tPT8eWXX0KpVKJbt2617h8eHg5vb+96txMRkfWJ/Ia5Px/FxcIr9b4ngoooRY3lEgN+Vg9AMfzQVpGF65W6D5/hKDB5HMY8Vk1ipkYs+bTZ/RIGbw1BRJAv5oztgtHddA3IbOXmbpEyGPnt8EUGI5YMRsSyy7Bhw6qv63M7Jk+ejM8//xwXL15EWpput0YiIrLnWZBSvLP+VIPnGhNU6JdoJrhtgFKhW8rJRguTx9WcAEZMhERBNxOzu7ALpn69H88M74DYUF+EB3jbZLZkdLdIvLX2JLan5KHoSoXcTI8sEIwMHTpU5nw0RAQkjfnXv/4lL0REZD+zIIY0FVT8oWmHQ9p28EQF/uK2Rc5UZCIEyZpOJo+tOQFMzUBG/670zvqT1beLRmTWni1JCA9AfJgfTueUYtPx7OrurNQ47k1DROTE9D08Xv75CB7/er/RgYgggooMbXC9Chi9b9S6jtq3KHejJYrl8dyKB5qVvNrUYzUnkMksvCJnS0QQZu3ZEWHNkUyrPq4jYzBCROSkxJvw4Dc2YvxHu/DJ9rMm318EFXMrJumO6wQJRVofmS8iTHDfAJVvJJ7z+CfWaJKqz2np61HdR0Sv7qqJ/pxaj2Xs+LRAhrbhmRht1WXmD4ewPSXXas3Ubq6aidl0PAeXy7lXjV00PSMiIvvLBzGWCC6mVjwjK12icDXBdIXHGFxWeSMhUIO+970FRewgvAolbq96fH3OhpBc47beMS2x79ylBs7pgVOXOqDDgVca7Gmip48rjJmJKbhcgYkf77baso2oqGndwkduHLj1VA5GddXNlFDDFNrGEkDshGh6FhQUhMLCQlkuTERE15YPYgpFVenttyPVsiuq1i8cN68EjmcWY/atXfDQ4DjLdXsVDc5K84C1s2oFKGJGRAQiNWdijPk9hEX397J4QPLyL0fxybZU3NWzNd4e57o71RcZ+f7NmREiIicIRERuhKU+WUZWzSgkVb2BH0y7hOOZO+DprsRdvSyQoCk6q8bpepdU6zJWBih/HDuOedsKsEfTCWoTMw3E8+MGDVauWIaRmigoAyIt1slVlPiKYGTdsSy5H494rqhhDEaIiBx4WWbX6TyZE2HOQCQy0Avjk9oiNtTPYIns4t269g23XtcKLXyt1GW0KkBJjBuCB9texLlmzALJVvNiuakiH/jRsp1ce7VtibAAL+QUq7DzTB5u6BBm1p/vbBiMEBG58LKMPsQwtj+H6J3x85+65ZLx/Wyz/4pYYhnRJbJefowYsdbkVvO6Tq6490uzBiRKpQIju0Tgm91pWH04k8FIExiMEBG58LKMfgnG2ByKFQfScaVCg/bh/ugT0xK2IoKlAfEh1dc7RgY0GJw13mpeC9HsvvyXf2K1qgfCA/3M1ixNVNWIYGTd0Uy8ckc3m7ert2cMRoiIHEFVUqemOBMrf86AAu2gbUZ3Bv3swUODYuXsgilvvKLeQby5CmIZR6GwnzdX/WyJWLZ6YvF+WUFjbKt50Vreq+wivv1uqdxTx1xVN/3aBSPIxwO5JeXYezYf/dpdDZ6oNmbUEBHZu6MrgfndgC9uhfLHR/B+xWxs83pKLj00Zybkg/t7YfbYrnJmwZRP6wfPF8gKGi93Je7u1Qb2Rvwug9qH4vW7r5NBl8LEVvP688zVLM3DTYmbOoXL4w3Hs6/pZzk7BiNERPYeiIichjp9NyKRL3MgGgtI9G/Izw7vgHfv64FvH+2PbTNubPYnfn3i6pjurRBUp5mZPRG/nyjfFYGXKa3m9efpl7/Ess+1Nkq7qXOE/LrhWNY1/Rxnx2UaIiJ7bV5WVIrR6/4BT5nVUJuY0BDvk3M8vsI6VR+Djb9MzQdpTM3E1QlJtklcbXaSa9F1UK37GJ5lWXJJpq66e+qIHBOxtBNeXIDjO6+g64DRzS7/vb5DKNyVCrlXjUi0FRVKVB+DESIiO62S6a88its9M43atVbkOui18PHAwom90L+dacswjVleI3FVdFF1BFeTXEMAr3/rZpjq1NzU7eRaXf6rzzFZtwDY3fzy3wBvD5k7InbxFUs1D5u7QZyT4DINEZGdVcnoK0JMzXXQL8uInIlBCaFmC0Rk4uou3RLNxH72lbhqNBFIiPLdwNqzRGJGRLS7F51c9eW/YgnMYPmvWDJrhhs76ZZqNh7nUk1DODNCRGQnSzNiRqTmIoKpuQ7mXJapaX/aJZzIKoa3hxJ32mHiqkkBSacx1VVJ037OwJridrKTa1PlvzLMWz1Td38Tl2xEEqtoD7/7TL5c7gr0tt98G1thMEJEZAeByOfbU+v1yBA5DBnaYPlJ3dAkh+yP4RuJ8bePw9Nm7I9Rl76c99buUbJU1aFVdXIVywK3KS/it6/3y9mkpsp/ZUBSlI5tG1bCrd31Jj3XIk8kPsxP5o38fjJXJgBTbVymISKy8dLM4Dc24uVfj9X7nshhmFsxqVZuw1UK+SbqdeubuL1nW5PLdI1VWFaBX//UlbhOsFHHVWtU3Ri7JLZs016M/2iX/DczpfS3uqqGSzUGMRghIrKTHBFDRC6DyGnIRHDtb4g9VczcwtyQH/ZfgKpSg06RAegZbdyykaMFJKLc+a9jBpq0JGZqL5Ibq/qNbD6Rc83lws6IyzRERDYo280svCxnQ4x5WxIByXpVH4wKOIMFYy2722zdxNXFyQ6euGoEMaMky3dF1YxIVjWi/Lcqi0Tm+YgS4qZmpUTr/EBvd+SXluPg+UvoHVMnuHRxDEaIiOx8cztF1ZLNbbffC6WZk1Mbs+fsJaRkl8DHww2392wNpyYCO1G+a0T5r564WfxbiuCy5j45hri7KTG0YzhW/pGBDceyGYzUwWUaIiI7WZJpiMhpELkN5q6Sacri3efk19sSo1yjAsSI8l9DtqcYt/RyU+eq1vDH2Bq+Ls6MEBHZoGzXWC+O6YwHB8VZfcfXS6XlWHU40ykTV40t/z15OgWzN+bKpRlDHW71Fmw6jR/2pzdZVn1DhzD57yjKpM/nlyE62NdCv4Tj4cwIEZENynabIkIPsXusLQIRfeJqeaUGXaMC0b1NEFxKVflv/I0P4lxAL6N2RzYmobWFr2d199pNJzg7UhODESIiG5TtNkYfeohP2rYIRDQabXVvkQlOnLjaFPHci38DoalnwNjN9fS7+K7nUk0tDEaIiCzAEXNE9LafzkVqbikCvNxxRw8nT1w1cQfgxtRMaG0qb2TX6TyUlVeadayOjDkjRER2kCMS7OeBF2/tishAb4t1UjXWVzt1iat3924DPy++Teh3AH5n3Uks2JTS4Hn63X49jqUDys4Gy6/jw/zRuoUP0gsuY9eZvOp9a1wd/8qIiMzcQ0RUVxg7I6IPOV678zqbzYTUJN4k1x/TdQm9v78LJa42QQSHYvPBhoKRWrv97oXuElh/t1+x5DW0Y5hcBttyIofBSBUGI0RENuwhYqnN7Zrr291psq/GwPgQJIQH2Ho4dkXMWImkYpGsWnPWS7/bbz363X7rdMoVVTUyGDmZY52BOwDmjBAR2Sg/RJTtilbk9hKIqCrVWLJHl7j6QP8YWw/HIRJam97tV/yBzAQ06upbByaEwsNNgbN5ZTibW2qVsds7BiNERFbOD7F12W5DVh/ORG5JOSICvTCiC5cPjElo1e/22/A/o263X9G3RM/fyx19qjqwcnZEh8s0RERW7CFi67JdYxJXJyTFyPbl1HhCq8gPksmqIj+kKSW1d+u9oWMYdp7Jw+YT2Zg8MBaujn9tRERW7CFi67LdhhzNKMLec5fgrlRgfFK0rYdj90QgKfaj6dO1s1HnHynyqdV/RCSxCiIguVJxdQnHVZkcjGzduhVjx45FVFSUzApevnx5o+f/+OOPGDFiBMLCwhAYGIgBAwZgzZo11zJmIiKHyxGZNiwB3z7a365yRGr6apduVmRUt0iEBzbdU4OqiPJdUTXTQFs0EX9kaEMw9meNDF71HVo7RgTI5bArFZpG+5K4CpODkdLSUiQmJmLhwoVGBy8iGFm1ahX27duHYcOGyWDmwIEDzRkvEZFD5Yjo80OeHdFBfpK2t6UZoehKBZYfSJfHTFxt5m6/Uu1/27q7/dZsGS8+zIuqGmEL80ZMzxm5+eab5cVY8+fXLnd67bXXsGLFCvz888/o2bOnqQ9PRGRT4lOsqT1E7DE/pKZle87jcoUaHSL80S+OW9s3e7ff1TOAooxau/2KQES/26+ITcRfgQhmRc7J0I7hWLb3ggxGXoRrs3oCq0ajQXFxMYKDG/6DV6lU8qJXVFRkpdERETVAlGae2wGPY8fQX1nY5E6u9thDxJBKtQafbT8rj6cMinPZfWjMtdvvkZ2r8b9fdyAbLQz+jdRsGS+aqIkgNSW7BBculaFNS9fdxdfqwchbb72FkpIS3HvvvQ2eM2/ePMydO9eq4yIiatDRldWfevsAWOIp8gCCMbdiUvWnXkM9ROytdNeQNUeyZNfVYD9P3NnTtfehuWZKN6T49cBKTdOnZhdfwQCfEPRq2wJ7zl6SsyMT+7nuEplVq2kWL14sg4xly5YhPFy3WZAhs2bNQmFhYfXl/Pnz1hwmEVHtQER00awx/S5EIl923RTdNx2hh0hDPtl2Rn69v19beHvU3keFTBceYFzy76msYuw8nYch7XV5I5tPuHbeiNVmRpYsWYJHHnkE3333HYYPH97ouV5eXvJCRGTzpRkxI2IgXVXEGSJBcY7HV1in6iOn4x0lR0Rvf9ol7E8rgKebEvcPcN1P5dZoGV/Xgk2n5SXU31Ne35GSi/JKDTzdXbPjhlV+62+//RZTpkyRX8eMGWONhyQiunaia2adGZGaRLwRpciTXTjtuYdIQz7Zliq/3tYjyuhP9GR6y/jGiI63Qmm5GnvPuW6Jr8kzIyLfIyXl6q6FqampOHjwoExIbdu2rVxiSU9Px5dfflm9NDN58mS8++676NevHzIzM+XtPj4+CAoKMufvQkRk1hLe06dT0MGIc/8+MAgVnfvLT8WOMCMiiIRJ0f5deGhQnK2H45Qt403dOHHziRwMjA+FKzJ5ZmTv3r2yJFdfljt9+nR5PHv2bHn94sWLSEvTbbQkfPjhh6isrMQTTzyBVq1aVV+efvppc/4eRERm7646e2OuUeeLLpz22kOkIV/sOCsDLrE7b5eoQFsPxykDEtHgTjS6Ew3vjLH2iC44dEUmz4wMHToUWm3DK2Gff/55reubN29u3siIiGzYXVW8ymWhk6yaEcmqhuMMha77pujC6UBKVJVYkqwrDHhkCGdFLN0yXlTOGONsXhlyS1QI9Xe9nEnXzJQhIjKiu6pIShXlu/K43mewquhk9Ou6LpwO5Lu951GsqkS7MD8M7dBwZSOZhyn5OL+7aDdWBiNE5PJEECLKLN9Zd7LeGr/oIzK14hlkok6jRjEjIrpuimZXDqRCralOXBVNzpQOtLTk6BU2xjzTz/10qHr/Gldi9aZnRET2RLzwN5VoKAISUb4rqmbCUYB7h/XB4Jtuc7gZEWHFwQxcuHRZlpTe06uNrYfjUhU2YvlPBCSNlfxertDg8a/34wMHqsoyB86MEJHLMmX3XbFks0vTBSs1A+HW7nqHDETEDND7m3TVkI8MaQcfT8f7HRy9wkaUfxtj7s9H5b+Xq2AwQkQuydTdd2t2VxXT7o7o10MXcSa3FC18PXA/d+e1WYWN2CqgKRer9q9xFQxGiMglmbL7ruBo3VXr0mi0WLgxpbqviL8XV+ltQfzthAYYVy2TbWQVjjPgXyMR2QXxwns6uxRl5ZUoK1fjcrkaqko12ob4IbFNEFr46tpmm2NGRAQiv5mYJOgIO/A2Zt2xLJzIKkaAlzsmD4y19XBcmrHVNeEu1BWXwQgR2UxaXhnWHMnE6iOZcp+URloYIS5UF5T0iQ2W7csDvT0skqxa17Rh8RiUEOZQ3VXrEr2h3tt4Sh6LQCTIx/Tnjqy7f02wn4fDLgc2B4MRIrL6G+Pao1l4d/0pHL1YVC/gCPB2h4+HG3w93eDupkRKdglSc0urL8sPZmDeqmP4S59oPDgwFrGhfiY3MzOGomo25NkRHR02CNHbfDIHh9OL5HP60GA2OXOE6pq4UD/88meGnB1x5EDYWAxGiMiqMyFzVh7Gpqrt0sULbL+4YIzuFomRXSIbrDQoKCvHHxcKcTCtQL5An8ouwec7zuKLnWdxU6dwTLuxPXpEtzBbsqqj54fUmxXZoJsVEUmrwX7mWe4iy+xf4+vpJpcp950rwL5zB+VtrRx8idAYCm1jvd3tRFFRkdxUr7CwEIGB3EOByNFcqVDjgy2n8f7m03KbdA83BR67vh0eGdwOLU18cxQvWdtScvHpttTqoEahACb2a4t/ju5kcPlGNDQb/9Euox/DmV78t53Kxf2f7IaXuxK/zxjmUnkIjkCfw5RdfAVnc8vwzvqT9c7Rh8OOtCO0qe/fnBkhIos6n1+GKZ/vkcstwuCEUMy9vSviw/yb9fMUCgWGtA+Tl9M5JbJC5McD6fh6VxrWHMnCv8Z2xS3XRcrz9C/221OM2/Bu0oAY3NytldNMi4sKmnm/HZPHE/q1ZSBix/vXqDVauTmjIWLGQPw1ilmUEV0ineJvsy4GI0RkMUcyCvHgZ3uQU6xCeIAXZo/tgjHXtaoOFK6VCGjeHtcD9/Rpgxd+Oix7aDyxeD+GdQzDG3d3l0mxpiSsikBEvDE4ixV/pONIRpGsoHnyxva2Hg5dQ6m5tkbvEWf6G9VjMEJEFiFmI/761T65Q2ynyAB88VASIgIt88l8YHwoVj09BIs2n5YXsXwz/O0tKLpSaVKyqjNVL4ilsbfW6Kb8pw6LZ66Incs2sqeIs/YeYdMzIjK7FQfT8eBnyTIQ6d8uGMseH2CxQETP28MNz47ogFVPD0aHCH+TAhFnSVatSST4phdcRlSQt2xyRvYt3MV7jzAYISKzWpKchqeXHESFWosx3VvJGZHm9ARproTwAMy6uel223piRsQREwMbk19aXt1t9f9GdpSBGjn2zr4KACF+nsgsvCwTsp1t3xou0xCR2fx+KgfPLz8sj0UPkNm3drHJFvVFVyqMbmjmDH1E6vrvhlMoVlWiS6tA3Nmzta2HQ2boPaIFkFdajmeX/eF0FV8CZ0aIyCxSsovxt2/2y09sd/VqLV8orR2IiMcWnxpPZekqd5oiOqs6WyByNrcUX+86J4+fH9PZJsEgWX5n38zCKzJwEc38nAFnRojomuWVqGT5bvGVSiTFBmPeXdeZrWLGUq3eHXn33ca8sfo4KjVaDO0YhkEJobYeDjUjIBnRJRK7z+Thr1/vk/+nXKHclzMjRHRNxGZ2omrmfP5lxIT44oMHesPL3bo5CvpW76bsOSM6vzr463c9649m4bfDmfL3mnlzJ1sPh5rJTanAwIRQJLZpuKtw3XJfR8dghIiaTXRDnfnDIew9d0nuKfPJ5L5WLyE1tdW7n6cuUBJ73Lyw/LDTJAIWXq7A88sPyeNHh7RDp0h2q3Z0MSG+LlPuy2CEiJpt2d7z+OlAuvwkt2hibySEN6+rqiWbRdVMVv320f7481+jZD6LWEX6Zncanvhmv+zJ4ejE5oFZRSq5wZoocSbHNzA+1GXKfRmMEFGzN7176eej8vgfozpicHvr5yfoWr3r9qdpSvuIANm5UgROUwbFYcH4XvB0U2L1kUzZE8XYChx7bTC3ZM95eSw6z7KU1zmM7hYJ90bWEhVOlPvEYISImhUE/N93B1FarpYJq2JZwNpEnojYy2PBptPN+vQoeqB8PqUv/L3csetMPsb9bxeyixxvurtUVYmZP/5ZvbeOM7wxkY4InK/vEOoSzfoYjBCRyT7+/Qz2nL0k8y/+c2+i1V8MTUlYbezTo0gSXPJYf4T6e+HYxSLctWgHzuQYVxZsL/695oRMHm7dwkfuWkzO5f7+MfKrW53qtIhALzwzvD1UlRqnaILG0l4iMol40/7PWt2eJ2Lju+hg45LsbJGwasynx26tg/Dj1IF44NPdOJdXhrsX7cAnD/ZFr7YtYe9EvswXO8/KY1FOLWZ5yLn0iwuBh5tCdjSeP66HzHU6m1uGb5PT8M76U9XnOXoTNM6MEJFJZbzPLj2IcrUGwzuH494+0XabsGpKq/e2Ib74YepAdG8ThEtlFZjw0S5ZJmvPMgouyyZzWi3wl95tcH2HMFsPiSzAz8sdPasCY7HXk5e7EvPXn0RmnSVFR2+CxmCEiIz23oYUHM8sluW78+7qbvXGZqaUMU4bloBtM240+pOiWKoR1TaiWdiVCg0e+2ovFu9Ogz26XK6W48stUckdkf91W1dbD4ksaEhV8zqx3UJDs4L628T3HXHJhsEIERklNbcU/9uqSxZ99Y5uCAvwsu4ANGog9Xd0zl2L/sqjUELT6Omi+6ipuSziU+hHk/rg3j5tIF7Pn/vpkOxoqrGjF3fR2+WfP/yJw+lFMigU4xXjJuc1uKpSbdup3EZnBR25CRr/gonIKC/9fESuW9/QIUyWHFrV0ZXA6hlAUQZEB40lnkCGNhhzKyZhjSap1qmKquWZ5laVeLgpZXlsZKA3/rsxBYs2n8aprGK8M64HAqy4+3BDFm05jZ//yJAln4sm9rJ6zg5Z33Wtg2RTwYZawztDEzTOjBBRkzYcy8KmEzkykU7XMExh3UBk2SQZiNQUiXws8piPUcpks5c7it9v+siOeGdcIjzdlVh/LBt3vb8D5/JKYUsij0VUzwhiaaZfuxCbjoesw91NiYHxxv9bO2ITNAYjRNQo0Z30pV90zc0eGhyHdmH+1l2aETMiBlbJZayhAOZ4fFW9ZGNswqqx7uzZBt/9dYAsozyVXYLbFmyXDcZsYcvJHDy15IBMWJ3Yr211ySe5hsHtdQnKnm6K6qDbmZqgMRghokZ9si1VlryGB3jhyRvbW/fBz+2oNyNS9wUsSpGHL2+qlMmnpiSsGisxugVWThssv4r9Xx74ZDf+veY4yisbz1kxp58OXMDDn+9BWblaVs3MGcuEVVdNYlVrr+7YW5O4LoLyd/sVw+3IDzK/SgbzzhqMbN26FWPHjkVUVJScyly+fHmT99m8eTN69eoFLy8vJCQk4PPPP2/ueInIii4WXsaCjSny+LlbOlu1j4WoCDh5WvfYTRkcqa5u9W4JEYHeWPpYf1lCK3JZF246jTvf346TWcWwtA+3nsazS/9ApUaL2xKj8PGkPnLpiFxv07w2LX3k/4snhsXLWcCaxvkfxOGWf0fS1snADw8DX9wKzO+mW+Z0ACb/RZeWliIxMRELFy406vzU1FSMGTMGw4YNw8GDB/HMM8/gkUcewZo1a5ozXiKyotdWHcflCjX6xLTE7T2irN7qffZGI5dE/CMsPSS538u//5Iok0Zb+nrgSEYRbn1vm+xGa4lqG/EzX/nlqPw3EB4eHCebXjEQcU0KhQKDq2ZHROm5mAUUs4Hv3tcDa0YVYF7lv+FzObP2nYou6vKtHCAgUWhFnVhz76xQ4KeffsIdd9zR4DkzZszAr7/+isOHD1ffdt9996GgoACrV6826nGKiooQFBSEwsJCBAZyW2wia9h3Lh93L9opOz7+PG2w7FRqzVbv2qpp521eT8lkVcOTHgogMAp45hCgtN7mcGIPmxk//CmTevXVDtNHdsDQDmFmSe49klEoNyHcXVWi+dwtnfDY9fHX/HPJsf3yZwamLT6AjhEBWPPs9bobxVKMmAFpcDnTNv9HTH3/tniIvXPnTgwfPrzWbaNGjZK3N0SlUslfoOaFiKxHfEZ5c7WuamNcn2irBSJ1W71roJTlu/K43semqjf90a9b/UU2PNAbnz7YF6/deZ3cn+dQeiGmfLYH93ywEztONz/BNa9EhVk/HsLY97bJQER023z73kQGIiQNjA+VHw5OZBVf3dSxibwqmWFSlK47z45ZPBjJzMxERETtKVRxXQQYly9fNnifefPmyUhKf4mOtn7LaSJXti0lV74Zerop8dRN7W3a6l30EZla8QwyUadCQHzau/dLoMttsAUxAzKhX1ts/ecwPHZ9Oxk47Dt3CRM+2o17P9iJz7enIi2vzKjA70RmMRZuSsHQtzbLPUdE4HVr91bY+PehuKtXG6v8PmT/gv080TVKN7uwXR/0lhi5bYGx59mIXTY9mzVrFqZPn159XQQuDEiIrEO8Ob5V1ctClI9GtfCx2mM31KxJBCTrVH2QpDyOcBTg3mF9MPim22wy7VxXiL+XTO59ZHCcDCgWJ6ch+Wy+vPzr56OID/PDsI7hiA31k/keXlWXosuVchZl++k85BSrqn+eeLMR1TKOWJ5Jljc4IUx23/39VK4sPTc6X8oKeVV2HYxERkYiK6t2RCaui7UjHx/DL3Ki6kZciMj61h7Nwh8XCuHr6Ya/DYu32vKMmBU5lVXS4DliyWaXpos8Ht+uv10EInWXbube3g2P3RCPX/7IwMbj2dh77hJO55TidE5qo/cVwYkIPsYmRuHuXm0sVhVEjm9I+1B8sOW0bA0vPjgoYgbqZglFsqrBXWuqckbEea4cjAwYMACrVq2qddu6devk7URkX0RQ8J+1ulmRKYNi5eZx1khYFXkixuzEe62t3q2hdQsf/PWGeHkRfUnEm8a2lBzkl5ZDVamR/UnERQQcfWJbyj10erVtKat1iJrSO6alDF6zi1WyEV+HiABg9Bu6qhn5P0RrF3lVFg9GSkpKkJKSUqt0V5TsBgcHo23btnKJJT09HV9++aX8/uOPP44FCxbgn//8Jx566CFs3LgRy5YtkxU2RGRfxJ4nJ7NKEOjtjseGxFu1cqYp5mr1bk1BPh4Y072VvBCZg7eHmwzGxTKNCHRlMCLypkT+VNX+TdXEjIgIRGyUV2XRYGTv3r2yZ4iePrdj8uTJspnZxYsXkZZ2ddvtuLg4GXg8++yzePfdd9GmTRt8/PHHsqKGiOxHhVqDt9edlMfiU32Qr4dVK2eaImZERCBi7g6rRI5mcEKoLhhJyZVbNEgi4Og0Rlc1I5JV/SOgjh6A5HOFyD6YLverEUGMvQbyJgcjQ4cOletUDTHUXVXc58CBA6aPjois5ru9F5CWX4ZQf088ODDWJpUzhkwbFo9BCWF2/UJKZE2D24cCvwG7zuTJJb/qRnhiKSZuyNXlz39vqfV/rJUdB/Rs5UdE8gVtwcZT8vhvQxPgZ4W278Zuc94+IsCird6JHE3nyECE+HnKvYoOni9ocPmzbrCfWXhF3i6+b28YjBARVhxMR0bhFZmwKnpnWJpYosmtUc7qbNuhE1mSUqnAwKrW8NtO6boAG7P8qb9NfF+cZ08YjBC5OPGitGjLaXn8yJA4i1d16PedefnXY42e58jboRNZaxff31NyTVr+FCGI+L44z57YZdMzIrKetUcycSanVFbQTLTwrIix1TOOWDlDZE2DRN4IgD/OF8gSclG5Zcryp7HnWQtnRohcmEhGf3+zblZk8sBYBHh72EX1jKicWXR/L7tMtCOyl3427cL85NYBO0/nmbysaW/LnwxGiFyYKA0Um7z5eLhhyqCqEkEbV8+8OKaz3B6dgQiRcUs1oqmenljWFMubCgdb/mQwQuTCxF4qwn1J0XITLksydlo4NMCLSzNERhjSPkx+FT1H9MT/HbG8KSgcaPmTwQiRixI7zO46kw8PNwUeHdLOosszYhq5sX1n7Hn6mMhe9Y8PgbtSgXN5ZbV2iBazimKZUyx3OsryJxNYiVzUos26WZG7erax2M68zrbvDJE98fdyl/saiR2if0/JwcSQmOrviYBjRJdIuTwqZiX1HVgF8eGg5m32MEvCYITIBZ3ILMb6Y9lQKETrd8vMijj7vjNE9tKNNVkEIydzMbHf1WBEEP+XRMPAxj4c2EtXVi7TELmgj38/I7/e3C0S7cL87WLfGXudPiayZ0OqSnx3nM5FpVrT4Hn23pWVMyNELianWIUVB3U7ez5ioVwR7jtDZB3d27SQPYKKrlTiz/RCuWxTV1NdWcX/PPF9saxjq/+HnBkhcjFf7TqHcrUGPdu2MPjCZQ7cd4bIOtyUCgyqbg1fuxurI3VlZTBC5EKuVKjxza5z8vhh/dbjFqmcKTbqfFbOEF27qyW+tfep0XOErqxcpiFysQ3x8krLZffG0V0jzfqzWTlDZNu8kQNpBSi+UlGvk7IjdGXlzAiRC7V+/2Rbqjx+cGAs3N3M99+/oeQ4Q1g5Q2Re0cG+iA3xRaVGK3sH1eUIXVkZjBC5CNGl8WRWCfw83TAuKdpsP5eVM0T2UeIrbDOwVOMIXVkZjBC5CP2syF/6RCPQjBviGV85k4BvH+3PfWeIrNQaviZ778rKnBEiFyASSreczJFNzqYMirVR5Yx/rQZMRGQ+A6qq0s7kluLCpTK0aelb7xxDXVl7x7SUW0OIfDJbdmRlMELkAj7drpsVGdE5AjEhfmZdosktVhl1LitniCwn0NsDPaJbyMBClPjel9TW4Hk1u7KKXK8b/r3JLjqycpmGyMkVlJXjx/3pZi/nFS9kg9/YiJd/PdboefaQHEfkSlU1vzewVGPPHVkZjBA5uWV7z0NVqUGXVoFmCwiMrZ6xl+Q4IlfKG9mWkitnLRvSVEdWQXy/sZ9hbgxGiJyYeDERHVeFSQNioBBJI1asnrGX5DgiV5DYJghBPh4ovFyBPy4UNHiePXZkZTBC5MS2nMzG+fzLcu+K23u0tmr1zItjOrNyhsiK3N2UGFzVGn7LCcPdWO21IyuDESIn9sUO3azIuL7R8PF0M8vPNPYFKjTAi0szRFZ2fQddMLK1gdbw9tqRlcEIkZNKzS2tLue9v3/MNf887jtDZP+u76DLG/njfIFMXjfEHjuyMhghclJfV+WKDOsYfs3lvPrKmfEf7cKCTacbPZfVM0S20yrIBx0i/CFyT0UiqyH22JGVwQiREyorr5RVNPrE1WvBfWeIHMv1VVU1jeWN2FtHVjY9I3JCyw9koPhKpdw8S//CZK19Z2zRMImIrrqhYxg+3pYq80bEBpkNVdEZ6sjKDqxEZBbixefLnWflscgVUV7DC4sp+84MSgi12QsZEV3VNzYY3h5KZBWp5OaYHSMD0JCaHVltics0RE5mz9lLOJ5ZDB8PN/yld7RV951hIEJke94ebujfLqS6vN8RMBghcjL6Jmd39IxCkO817M6rUSOh9CBuU+5Af+VRKKFp8FRWzhDZl+urlme3nmy6NbzDBiMLFy5EbGwsvL290a9fPyQnJzd6/vz589GxY0f4+PggOjoazz77LK5csV4zFSJXkVuiqt5TYmK/a0hcPboSmN8NXddNwH89F2CJ5yvY5vUURilr/19n5QyRfZf4Jqfmy4R2pwtGli5diunTp2POnDnYv38/EhMTMWrUKGRnG54KWrx4MWbOnCnPP3bsGD755BP5M5577jlzjJ+IahAVNBVqrdy9s1vroOYHIssmAUUZtW6ORD4WecyvDkhYOUNkv+LD/NC6hQ/K1RrsPmO9tu5WC0befvttPProo5gyZQq6dOmCDz74AL6+vvj0008Nnr9jxw4MGjQIEyZMkLMpI0eOxPjx45ucTSEi02g0WizenSaPJ/Zr28wfogZWz6ixXdZV+nhjjsdXcsmG+84Q2S+FQlE9OyKaHzpVMFJeXo59+/Zh+PDhV3+AUimv79y50+B9Bg4cKO+jDz7OnDmDVatW4ZZbbmnwcVQqFYqKimpdiKhxW07l4MIl3T40YxOjmvdDzu2oNyNSNyCJUuTh57FK7jtDZOduqApGtjpAMGJSaW9ubi7UajUiIiJq3S6uHz9+3OB9xIyIuN/gwYNlyWFlZSUef/zxRpdp5s2bh7lz55oyNCKX980u3azIPb2jZTZ9s5RkGXVa18DLV6dKiMguDUzQVbidyS3F+fwyRAf7wmWraTZv3ozXXnsN77//vswx+fHHH/Hrr7/i5ZdfbvA+s2bNQmFhYfXl/HldJ0kiMiy94DI2HtcFEhP7t232vjPbMo0MYvxrfyAhIvsT6O2B3m1byuPNdj47YtLMSGhoKNzc3JCVVfvTk7geGRlp8D4vvvgiHnjgATzyyCPy+nXXXYfS0lI89thjeP755+UyT11eXl7yQkTGWZqcJveiGNAuBPFh/ibdV1TfiC6rormZEu7Y5hWMSEV+A59UFEBgFBAz0EwjJyJLGtopDMln87HpeDYeMMOGmXYxM+Lp6YnevXtjw4YN1bdpNBp5fcCAAQbvU1ZWVi/gEAGNIJZtiOjaVKg1WLLnfLNmReruO6OBEnMrJsn8VRHc1Fa1LDP6dUDZzGUgIrKqYR3D5dcdp3NxpUINp1mmEWW9H330Eb744gtZqjt16lQ50yGqa4RJkybJZRa9sWPHYtGiRViyZAlSU1Oxbt06OVsibtcHJUTUfOuPZiG7WIVQfy+M7GJ4htKUfWfWaJIwteIZZKJO7xAxI3Lvl0CX28wzcCKyuE6RAbIX0JUKDXadyYO9MnlvmnHjxiEnJwezZ89GZmYmevTogdWrV1cntaalpdWaCXnhhRdkiZH4mp6ejrCwMBmIvPrqq+b9TYhc1DdV5bzj+raBp7vSLPvOiIBknaoPkpTH8dKNoegQn6BbmuGMCJFDUSgUGNoxHN8mp2HziRx5bI8UWgdYKxGlvUFBQTKZNTAw0NbDIbIbZ3NLMfStzRCbcm79xzCTsuVXHEzH00sONnneu/f1wO09Wl/jSInIVtYeycRjX+1D22BfbPnH0AZ38bXl+zf3piFyYOLTjjC0Q5hJgYhYosktVhl1LvedIXJsgxJC4eGmQFp+GVJzS+EUyzREZB9UlWp8t++CPB6f1LZZ1TONEZ+dRJdV7jtD5Nj8vNzRLy4E21JyselEDtqZWHFnDZwZIXJQa45kIb+0HJGB3rixU3izqmcawn1niJzL0I66bqybTxjeR87WGIwQOahvqxJX7+0bDXc3ZbOrZwzhvjNEzmVY1QcWsWleqcr+dvHlMg2RAzqTU4KdZ/JkR/ZxfaOvuXqmphfHdMaDg+I4I0LkRNqF+skEVpE3sj0lFyO7Gt8GwBo4M0LkyImrHcPlNuHGyC5uOhARQgO8GIgQORmFQoFhVUs1Im/E3jAYIXIwoovi91WJqxOMSFzV7ztzKqvYqJ/P6hki516q2Xwi2+46oHOZhsjBrDmSiUtlFbKroj4p7VorZwRWzxA5t/7tQuDtoZSvByeyitEp0n76dnFmhMjBLK7uuNp44qqxlTMCq2eInJ+3hxsGxofK403H7WuphsEIkQNJyS7B7tT8JhNXTamcEVg9Q+QahunzRo7bV4kvl2mIHDBxVfQVaRXkc82VM9OGJcjujGJphjMiRC6SN7LiCPaey8el0nK09POEPeDMCJEDJa7+sL8qcbVfW7NUzrSP8MeA+BAGIkQuok1LX3RuFQiNVlTV2M/sCIMRIgex+nAmCsoqZCnvDR0a7rjKfWeIqDEjOuteP9Yfy4K9YDBC5ICJqw3NZIik1cFvbMTLvx5r9GeJe4tqHFbOELmemzpHyK9bTuTIPa7sAYMRIgcgeoQkn82XQci9fQwnrnLfGSIyxnWtgxAe4IXScjV2ncmHPWAwQuQAFtdIXBWVL3Vx3xkiMpZSqaieHVl/1D6WahiMEDlA4uqP+9MbTVw1Zd+ZbTNuZCBC5OJGdNHljWw4lmUX3VgZjBDZuVWHLqLwsi5x9fr2hjuuct8ZIjKFaH7m4+GGjMIrOHqxCLbGYITIQRJXxyfVT1zlvjNE1NxurEPa67qxrj9q+xJfNj0jsmMns4qx99wlGYT8pU7iKvedIaJrMbxLBNYezZIlvk8Pbw9b4swIkQPMigzvHI6IwKuzGtx3hoiulUiIVyiAQ+mFuFh4GbbEYITITl0uF4mr+o6rMdW3c98ZIjKHUH8v9IxuIY83HLPtUg2XaYjs1C9/ZqDoSiXatPTBkATd2q7AfWeIyJxLNfvTCuRSzf39r37osTbOjBDZqW+qlmhEOa/oC6DHfWeIyFxGVPUb2ZGSh1JVJWyFwQiRHTqcXoiD5wvg4Va/46qxFTGsnCGipiSE+yMmxBflag1+P5ULW2EwQmTHHVdFnodY15U0aiD1d/Qr3YhbAlLgBo3B+3LfGSIylkKhwKiukegb2xJeHrYLCZgzQmRnSlSVWHFA13F1or7j6tGVwOoZQFGG/ATxPoAMr2C8VDEJqzVJ1fdl5QwRmWrWzZ1kUGJLnBkhsjPLD6TLDaziw/zQT8xuiEBk2SQZiNTUSnEJizznY5Qyufo2Vs4QkalsHYgInBkhsiNijwh94urEfjFQaDW6GREDhbwKaKGFAv9tsRSrRzyM8EA/Vs4QkUNiMEJkRw6cL8Cxi0Xwclfi7l5tgHM76s2I1A1IvMou4vYW54C4IVYdKxGRuXCZhsiOfLNLNysyNjEKQb4eQImR23sbex4RkR1iMEJkJwrKymWjM33iqui0eqTIx7g7++t6BRAROSIu0xDZie/3XYCqUoMurQKRWXgZf/tmP7IKNdjmFYxI5MNwKogCCIwCYgZaf8BERLacGVm4cCFiY2Ph7e2Nfv36ITn5aja/IQUFBXjiiSfQqlUreHl5oUOHDli1alVzx0zkdDSaq4mrPaKD8LdvDsiW7xooMbdiku6cejmsVdHJ6NcBpZuVR0xEZMNgZOnSpZg+fTrmzJmD/fv3IzExEaNGjUJ2tuFNdsrLyzFixAicPXsW33//PU6cOIGPPvoIrVu3Nsf4iZzCtpRcpOaWwt/LHRuOZ9eqnVmjScLUimeQiTpNzMSMyL1fAl1us/ZwiYjMSqEVtYQmEDMhffv2xYIFC+R1jUaD6OhoPPnkk5g5c2a98z/44AP8+9//xvHjx+Hh4dGsQRYVFSEoKAiFhYUIDAxs1s8gsmePfLFXblQlOiGuOZJp8BwlNEhSHkc4CvDXMQPRdcBozogQkV0z9v3bpJkRMcuxb98+DB8+/OoPUCrl9Z07dxq8z8qVKzFgwAC5TBMREYFu3brhtddeg1qtbvBxVCqV/AVqXoic1YVLZdh4XFcN0ztGt523IWLJZpemC1ZqBiLFrwcDESJyGiYFI7m5uTKIEEFFTeJ6ZqbhT3NnzpyRyzPifiJP5MUXX8R//vMfvPLKKw0+zrx582Qkpb+ImRciZ7V4d5rMB+kWFYiiy8btmslN8IjImVi8tFcs44SHh+PDDz9E7969MW7cODz//PNy+aYhs2bNklM6+sv58+ctPUwim1BVqvHVznPy+HBGERZsSmn0fG6CR0Rw9dLe0NBQuLm5ISurdoMlcT0yMtLgfUQFjcgVEffT69y5s5xJEcs+np6e9e4jKm7EhcjZvb7qGIpVxs2GcBM8InJWJs2MiMBBzG5s2LCh1syHuC7yQgwZNGgQUlJS5Hl6J0+elEGKoUCEyFWIpmZfV5XzGoOb4BGRszK56Zko6508eTL69OmDpKQkzJ8/H6WlpZgyZYr8/qRJk2TZrsj7EKZOnSorb55++mlZcXPq1CmZwPrUU0+Z/7chciBL95xHhbrpYrZpw+IxKCGMm+ARkdMyORgROR85OTmYPXu2XGrp0aMHVq9eXZ3UmpaWJits9ETy6Zo1a/Dss8+ie/fuMlARgcmMGWInUiLXteJgulHntY8IwID4EIuPh4jIYfqM2AL7jJCzyStRod9rG1BZv61qPd8+2p/BCBE5JIv0GSGia7f68EUMfWtzk4EIK2eIyFVwozwiKwcij3+9v8nzWDlDRK6EwQiRFatn5v581OjKGRGIsHKGiFwBgxEiK0lOzZc78TblxTGd8eCgOM6IEJHLYM4IkZVkFzcdiAihAV4MRIjIpXBmhMgKyzNiVuRUVrFR53PfGSJyNQxGiCycsCryRIxZnlFU5YqweoaIXA2DESILBiJTv94PYxr5sHqGiFwZgxEiC1bOGNtRkNUzROTKGIwQ2bByZtqwBAxKCOW+M0Tk0hiMEFlgVmR7So5R57aP8GerdyJyeQxGiGyUsCqwcoaIiMEIkc0SVlk5Q0Skw6ZnRFZOWGXlDBFRbZwZIbJiwqrAyhkiotoYjBBZsdW7qJ55dkQHzogQEdXAYITIiq3eRRkvAxEiotoYjBA1E1u9ExGZB4MRIgtXzugxYZWIyDAGI0QWbvXeigmrRESNYjBCZKHKGV9PN3xwf2/miRARNYHBCJGFKmdu6hyO6zuEWXw8RESOjk3PiExcosktVhl17p092lh8PEREzoAzI0QWqJ7x93LHDR05K0JEZAwGI0QWqJ6ZexsrZ4iIjMVlGiIzV8+M7hqBu3tHW3hURETOg8EIkRn3nfH2UOLVO6+z+JiIiJwJgxGihmjUQOrv8Dj2A/orj0IJTZN3mdgvBiH+XlYZHhGRs2DOCJEhR1cCq2cARRnoA2CJJ5ChDcbciklYo0kyeBd3pQKPXd/O6kMlInJ0nBkhMhSILJskA5GaIpGPRR7zMUqZbPBu4/pGIyLQ20qDJCJyHgxGiOouzYgZEQPpqvrimDkeX9VbshGVM4/fEG+tURIRORUGI0Q1ndtRb0akbkASpchDkvK4vO7hpotQxidFIzrY12rDJCKCqwcjCxcuRGxsLLy9vdGvXz8kJxuetq5ryZIlUCgUuOOOO5rzsESWV5Jl1Gl/HxiEGaM7okKthZe7Ek/e2N7iQyMiclYmByNLly7F9OnTMWfOHOzfvx+JiYkYNWoUsrOzG73f2bNn8fe//x1Dhgy5lvESWayXyM7TediW6WbU+b27dMKvhy7K48kDY5krQkRkzWDk7bffxqOPPoopU6agS5cu+OCDD+Dr64tPP/20wfuo1WpMnDgRc+fORbt2rDYg++uuOviNjRj/0S5M2uAuq2YaLuJVAIGtsbokHofTi2Tbd+aKEBFZMRgpLy/Hvn37MHz48Ks/QKmU13fu3Nng/V566SWEh4fj4YcfNupxVCoVioqKal2ILNnmXd/UTAOlLN8V+auaejmsuvwQ9cjX8Z/1KfL4ocFxCPbztPawiYhcNxjJzc2VsxwRERG1bhfXMzMzDd5n27Zt+OSTT/DRRx8Z/Tjz5s1DUFBQ9SU6mq21yXpt3kUfkakVzyATwbW/ERgF3Psllqt6ISW7BEE+HnhkSJw1h0xE5JQs2vSsuLgYDzzwgAxEQkNDjb7frFmzZF6KnpgZYUBC1mzzLgKSdao+smrmpRtD0SE+AYgZiHKNAvPf3izPEcszgd4eVh41EZGLByMioHBzc0NWVu2KA3E9MjKy3vmnT5+Wiatjx46tvk2j0a3Gu7u748SJE4iPr7/e7uXlJS9ElpwV2Z6S0+g5Yslml6YLjoX2QIe41vK2pbvP4nz+ZYT6e2HywBgrjZaIyLmZFIx4enqid+/e2LBhQ3V5rgguxPVp06bVO79Tp044dOhQrdteeOEFOWPy7rvvcraDbJYnIpZnjN38LjxAVylTeLkC76w/JY+fuikBvp7cTYGIyBxMfjUVyyeTJ09Gnz59kJSUhPnz56O0tFRW1wiTJk1C69atZd6H6EPSrVu3Wvdv0aKF/Fr3diJrJqzW769an0hXjQzyRlKcLndk4aYU5JeWIyHcHxOS2lp8rERErsLkYGTcuHHIycnB7NmzZdJqjx49sHr16uqk1rS0NFlhQ+QoCauGVHV+x5yxXWSr93N5pfhse6q87fkxneHuxr9xIiJzUWi1WmNem21KJLCKqprCwkIEBgbaejjkoERTM9FLxBitgrxlIDK6Wyt5ferX+/Db4UwMaR+KLx9Kkp2EiYjIPO/fXPQmpydmRETlzG+HdR1TmzJtWAKeHdFBzogIu8/kyUBEXH1hTBcGIkREZsZghJyaqcmqwqCE0OpARKPR4pVfj8nj8Ult0TEywGJjJSJyVQxGyGmZkqxqKGFV+OlAOg6lFyLAy13OlhARkfkxC4/g6smqhhJWhRJVJd5cc1weP3FjguwtQkRE5seZEXK57qqGRNZJWBXeXnsSWUUqxIT44sGBsRYaKRERMRghJ+2ummvUuZMGxODmbq3k0ox+RkQ4nF6Iz3foSnlfvr0bvD3cLDZeIiJXx2CEXDphVQQiA+JD6gUzz/90SO7aOzYxCtd3CLPQaImISGAwQk7jWrqr1vTN7nP444IuafXFMZ0tMlYiIrqKCawEV++uWlNW0RX8e/UJefzP0R0RHqjbl4aIiCyHMyPkcgmrhpJV9V7+5SiKVZVIbBOECf24Ky8RkTUwGCEX664aj2dHdKw3IyJsPpGNX/68KDutvnrndQbPISIi82MwQi7WXTXMYJBRWFaBmT8cksdTBsWhW+sgs46ViIgaxmCEXLa7ak3/+vkIMouuIC7UD/83kp1WiYisiQms5JLdVWv67dBF2fZdfOs/9ybC15MxOhGRNfFVl1yyu6pedvEVPPeTbnlm6tB49Grb0qxjJSKipjEYIYdiju6qelqtFs/9eAiXyirQuVUgnr6JyzNERLbAYIRcqrtqTd/tu4D1x7Lh6abEO+MS4enOVUsiIltgMEIu1V1VLzW3FC/9fFQeTx/ZAZ0iA804WiIiMgU/CpLLdFfVu1yuxtSv96FEVYmk2GA8OqSdWcdLRESm4cwI2X1DM5EjYo7uqnqzVxzG8cxihPp74r0JPdncjIjIxhiMkNM0NGusu6resj3nZa6IOOW/43signvPEBHZHIMRcviGZk11V9U7klGIF1cclsf/N7IjBsaHXuNIiYjIHBiMkEM3NDM2YbXwcgX+9s1+qCo1GNYxDFNviDfLeImI6NoxgZXsh0aN4ztXoW/xRvRXHoUSGrMkrFaqNXh26UGcyytD6xY+eGdcDyiZJ0JEZDc4M0L24ehKYPUMdC3KwH89dTdlaIMxt2IS1miSmp2wKhqbiZmWjcez4eWuxKL7e6GFb9UDEBGRXWAwQvYRiCybJEKHWjdHIh+LPOZjasUz9QISkawqckQa6q6q98m2VHy16xwUCmD+uB7o3qaFxX4NIiJqHi7TkG1p1HJGpG4gIuhjjDkeX1Uv2YibWgV5y6oZ0V21sUBEJMK+uuqYPH7+ls64+bqGy32JiMh2GIyQbZ3bARRlNPhtEWtEKfKQpDxuVH6I3oG0S3h6yUFotcAD/WPw8OA4Mw+ciIjMhcs0ZNOGZh7HjqGPEeeHo8CohmbC6ZwSPPrlXlk5c2OncHkfhVinISIiu8RghGza0Ky/shBLjMgn/euYgXhnwI1NzoiIPWfGf7gLuSXl6BoViPfG94S7GycAiYjsGYMRsmlDs2RNJ1k1I5JVDccZCiAwCl0HjL6aRNKAc3m6QCS7WIWOEQH48qEk+HnxT5yIyN7xIyNZZUlm5+k8/HQgHc/9dKhWqqoGSlm+K4/r5bBWBR+jXweUbo0+xvn8MhmIZBZdQftwf3zzaD+E+HuZ+TchIiJL4MdGsvkeM6JsV5TvzvH4ElHIv/qNwChdINLltkYf48KlMtz34S5kFF5BuzA/GYiEMhAhInLumZGFCxciNjYW3t7e6NevH5KTkxs896OPPsKQIUPQsmVLeRk+fHij55PzLckYs9mdCEgGq/6L+8pfwN4+/wYm/wI8c6jJQOREZjH+8sFOpBdcRlyoH759tD/CA7j5HRGRUwcjS5cuxfTp0zFnzhzs378fiYmJGDVqFLKzsw2ev3nzZowfPx6bNm3Czp07ER0djZEjRyI9Pd0c4ycn2mNGLNns0nRBRee7gbghTS7N7Didi3s+2CGDnfgwXSDCXXiJiByPQiv6ZZtAzIT07dsXCxYskNc1Go0MMJ588knMnDmzyfur1Wo5QyLuP2mSLlegKUVFRQgKCkJhYSECAwNNGS7ZKBD5fHsqXv5V13DM1A3vts1oumpmxcF0/P27P1Ch1qJvbEt8NKkP27wTEdkZY9+/TcoZKS8vx759+zBr1qzq25RKpVx6EbMexigrK0NFRQWCgxveYVWlUslLzV+GnCdHxBBjG5qJ2Pl/W8/g9d+Oy+u3XBeJt+/tAW+PxmdRiIjISZZpcnNz5cxGRERErdvF9czMTKN+xowZMxAVFSUDmIbMmzdPRlL6i5h5IefKEalLzIiITewaa2hWqqrEM0sPVgcioqvqgvG9GIgQETk4q1bTvP7661iyZInMIxHJrw0RMy8iL6XmzAgDEufKERFzH8F+nnhhTGdEBvk0ueHdqaxiTP1mP1KyS+R54n5TBrHFOxGRywUjoaGhcHNzQ1ZWVq3bxfXIyMhG7/vWW2/JYGT9+vXo3r17o+d6eXnJCzlOW/ftKTlGz4joQ45X7+zWZGt34acDF/Dcj4dxuUKNiEAvLJjQC31jG17mIyIiJw5GPD090bt3b2zYsAF33HFHdQKruD5t2rQG7/fmm2/i1VdfxZo1a9CnjzE7kZAz54cYu8dM0ZUKvPLLUSzbe0FeH5wQivn39WAPESIiV1+mEcsnkydPlkFFUlIS5s+fj9LSUkyZMkV+X1TItG7dWuZ9CG+88QZmz56NxYsXy94k+twSf39/eSHnaOturBfHdMaDg+KarJZZfzQLzy8/hKwiFcQed0/d2B5P3dS+yfsREZELBCPjxo1DTk6ODDBEYNGjRw+sXr26Oqk1LS1NVtjoLVq0SFbh3HPPPbV+juhT8q9//cscvwNZeVlm15k8zPyhdlt3Y8t2mwpE8kpUcrZl5R8Z8npsiC9ev7s7+rcLMcPoiYjIKfqM2AL7jDhH2W5j1TKVag2W7j2P/6w9ifzScrkn3qND2uHZER1YLUNE5KAs0meEXFdzl2WayhERsfDG49mY99txWSkjdIoMwBt3d0didAszjJyIiOwdgxFqmEYNnNsBTXEmVv6cAQXaQWtCa5ppwxIwKCG0wbLdg+cL8Ppvx7DrjG5zvJa+HjIvZGK/GHi6c0NpIiJXwWCEDDu6Elg9AyjKkOHH+wAyvIIxt2KS3NTOmPwQscRSNwgRMyG/n8rFB1tOY8fpPHmbCDweGhSHqUPjEeTjYdFfi4iI7A+DEarXM8TtxM/om/yMCB2q8z2ESORjkcd8TK14psGApKG27hVqDVYduoj/bTmDoxd17f3dlQrc3qM1nh3RHm1a+lr0dyMiIvvFYIRqJadmFZZhm9dcaMWCTJ2VFXFdowXmeHyFdao+cpfdpvJDRB7Isr3n8eP+C8gtKZe3+Xq64b6+bfHwkDi0buFjnV+QiIjsFoMRqpWc2l95HFEKXQ6HISIgiUIekpTHsUvTpfr2Fr4eWDi+F/rHh+BSWTm+TU7D9/suYN+5S9XniGZlkwbEyAt32CUiIrh6MKJfksguvoLwAO8m90ZxleTUcBQYdXf9efpn7J+jOuJEVjHe3XgKe8/myxkUQTynwzqGY1zfaAztGAYPNyamEhFRbS4ZjBjql9HKyBblzp6cmg3jymkztLq9YXw83WTS6XM/Ha71/etaB+GW61rh7l6tER7Y8KaIRERELtf0rKF+GeITvhIafDtSjaSwSsA/AogZCCjdXCI5VT+T8beKpzDb42uZrKqfKNJoFTirjcBBbQIOauKxH51xRNMWNf9yRMt2sXndqK6RGNU1ggmpREQENj0zcZv7kcpkzPH4ElFba+RLBEYBo98AutwGV0lOfdr9Jzxb/jeMcduFU5o2OKltg2PaGBTBr97PjQv1w8D4ENlLZEC7ELT0Yx4IERGZzqWCETEzYKiV+ShlsixZrafoIrTLJuHkDQtxvOVQx8ktqcoHQUkWknPc8cRaN6ihRD/FcfigHMe1bZGlCcZFbTDStaFy1iNNG4Fz2ojqoGN35dXkVMEL5ejmkYHEhLbo0aM3+sS0RBQrYYiIyAxcKhgRyap1iaUZMSMij+vFGGIOAQjY/CKeVb0rS1ltnVsiVtVUlRqUlatRqqpE0eUKJJ/NR2bhFZkcGlp6AsVH1qLwigaXtP7IQxDaK4JlwLFH2wk9VR82+Rg+UOEyvKCAFu0UGZjW0wO39oqHR7vbnGbZioiI7IdLBSNiZqOuOMVF7NB0hVYr3nqvRiPiWFv9VYEYRSZStVFyZuXxr/ejf7tgdIwIQHSwLjdC5E+otVpoxEUjvuqWhcR18VV+T6NFpbiodV/VGo08LldrZFOwCrVWfhXBRnml/qtafr1SocblcjUuV6ir8zsME8HCzY0+DyEoRKQiH60U+fJrjCILbRXZ1V8fqviHLNuNDPLBP8aOdY2kXiIishmXSmAVQcHgNzbKWQS7/6WNIGZ1NDKAEhcteitOIkxRgBaKUgSiDC0UxQhFEUIURQhGETQKhbwercg2MAukC7zKfSOxesRahAf6OcaSFBER2S0msBog3ljFEouoptG9fQNBKEFPZYr8vm4O5Or8yNW5EWCvpgMuwbhAyMdDiYRwfwT6eMDP013mVojHziq6Imc5ArzdEd3SV14XMx0tfDzk+WKZJS2/DGXllQjx90LXVoFIzStDiaoCkYE+SFLvh9/vr8KnJA2+uAI3hVaW2IqS3EL4Y4nnK02O7T8V9+BZ9+/l7ErtOEMhf0+vW9/E7V3aNuPZJSIiah6Xmhkx1GdEzC5s83pKLlcYascl3rQzEYLBVTkjzSG6kwoFZRX1qleMPUck2X7gqUuyNVSS+2nlzXjE47cmx/JU+TSo4K6rHKrZaTWwNTD6dYevHCIiIvvBmZFGiByIEV0iqzuwFl96Ba22PFH1XW29N/q5FQ80OxCpG2DU/dnGnKNPshVhY0MluXe4bzdqLKKpmcgHWa/qg77K45g1uAUSO3dyqp4qRETkWFwyGBHEssmA+JCqaxOBiIDqrqR6YkZEBCIN7VBrLUlG7BcjckFytQEIRrHBfBCN+H20IUjWdJLXw4N88eDY+5HI5FQiIrIxlw1G6hHLE53G1OrPMX6t2zXNiJiLsfvFLFcPwkNuqxvMByke+jLeadnLcfqlEBGRS2AwUpNYpogbIg/FXMjC8Pp72NiCsfvFrNf0wV5NJ8wW+SCo3UlWMfp1dOxyGzpabphERETNwmDEyNyS9Ucz8cn2szYZh1haEVUzNfeLMZRkK86LCPLFn7c+iij/VDnD42x77BARkfNxyWoac+72ay01W9bXDEj0hch7kuZD3XEsl1+IiMjh3r8ZjDRz91tRhSNyLy6VqvDyr8esEqCMVibrlmBYkktERA6Apb1WqcLRGdWtVXWAcja3DN8mpyGz6Gpw0tLXQxYMX0ufEeGPgOu5BENERE6HwYgFApRpNybUmj0RSydCzdt6x7TEvnOXTD5HtwTTxma/KxERkblxmYaIiIhs+v5t+yYaRERE5NIYjBAREZFNMRghIiIim2IwQkRERDbFYISIiIhsisEIEREROV4wsnDhQsTGxsLb2xv9+vVDcnJyo+d/99136NSpkzz/uuuuw6pVq5o7XiIiInL1YGTp0qWYPn065syZg/379yMxMRGjRo1Cdna2wfN37NiB8ePH4+GHH8aBAwdwxx13yMvhw4fNMX4iIiJytaZnYiakb9++WLBggbyu0WgQHR2NJ598EjNnzqx3/rhx41BaWopffvml+rb+/fujR48e+OCDD4x6TDY9IyIicjwW2ZumvLwc+/btw6xZs6pvUyqVGD58OHbu3GnwPuJ2MZNSk5hJWb58eYOPo1Kp5EVP/BL6X4qIiIgcg/59u6l5D5OCkdzcXKjVakRERNS6XVw/fvy4wftkZmYaPF/c3pB58+Zh7ty59W4XMzBERETkWIqLi+UMiUNtlCdmXmrOpoiloPz8fISEhEChEBvFmS9iEwHO+fPnufxjBD5fxuNzZTw+V8bjc2U8Plf28VyJGRERiERFRTV6nknBSGhoKNzc3JCVlVXrdnE9MjLS4H3E7aacL3h5eclLTS1atICliCeff6zG4/NlPD5XxuNzZTw+V8bjc2X756qxGZFmVdN4enqid+/e2LBhQ61ZC3F9wIABBu8jbq95vrBu3boGzyciIiLXYvIyjVg+mTx5Mvr06YOkpCTMnz9fVstMmTJFfn/SpElo3bq1zPsQnn76adxwww34z3/+gzFjxmDJkiXYu3cvPvzwQ/P/NkREROT8wYgo1c3JycHs2bNlEqoo0V29enV1kmpaWpqssNEbOHAgFi9ejBdeeAHPPfcc2rdvLytpunXrBlsTS0GiX0rdJSEyjM+X8fhcGY/PlfH4XBmPz5VjPVcm9xkhIiIiMifuTUNEREQ2xWCEiIiIbIrBCBEREdkUgxEiIiKyKQYjNdx2221o27YtvL290apVKzzwwAPIyMiw9bDsztmzZ+UuzHFxcfDx8UF8fLzMxBZ7F1F9r776qqwq8/X1tWjzPke0cOFCxMbGyv9zYhPO5ORkWw/JLm3duhVjx46VXSxFF+rG9vZydaKthNjMNSAgAOHh4XKX+BMnTth6WHZp0aJF6N69e3WzM9H/67fffrPJWBiM1DBs2DAsW7ZM/uH+8MMPOH36NO655x5bD8vuiH2IRLO7//3vfzhy5AjeeecduQOzKN2m+kSQ9pe//AVTp0619VDsytKlS2XfIhHI7t+/H4mJiXITzezsbFsPze6IXk7i+RHBGzVuy5YteOKJJ7Br1y7ZYLOiogIjR46UzyHV1qZNG7z++utyA1zR/+vGG2/E7bffLl/XrU6U9pJhK1as0CoUCm15ebmth2L33nzzTW1cXJyth2HXPvvsM21QUJCth2E3kpKStE888UT1dbVarY2KitLOmzfPpuOyd+Jl+6effrL1MBxGdna2fM62bNli66E4hJYtW2o//vhjqz8uZ0YaIDbm++abb+T0uoeHh62HY/cKCwsRHBxs62GQA80WiU9jw4cPr75NNEsU13fu3GnTsZHzvTYJfH1qnFqtlh3SxQySLbZrYTBSx4wZM+Dn5yd3CBbdZFesWGHrIdm9lJQUvPfee/jrX/9q66GQg8jNzZUvfvrOzXriuujsTGQOYjn5mWeewaBBg+yi67c9OnToEPz9/WX31ccffxw//fQTunTpYvVxOH0wMnPmTJnw1dhF5EDo/eMf/8CBAwewdu1auUOx2GvHVZrUmvpcCenp6Rg9erTMiXj00UfhKprzXBGRdYnckcOHD8tP/GRYx44dcfDgQezevVvmtYm9544ePQprc/p28GIfnby8vEbPadeundyRuK4LFy4gOjoaO3bscIldhk19rkSl0dChQ9G/f398/vnntfYkcnbN+bsSz5H4lFZQUABXJ5ZpRHXR999/L6sd9MQLoXh+OCPZMBHoik+vNZ83qm/atGny70hUIonKPzKOWCoVFZKiQMGuN8pzNGFhYfLS3Ck+QaVSwRWY8lyJGRFRfdS7d2989tlnLhWIXOvfFUEGaeJvZ8OGDdVvquL/m7gu3kSImkt8vn7yySdlwLZ582YGIiYS/w9t8Z7n9MGIscQU1Z49ezB48GC0bNlSlvW++OKLMkJ0hVkRU4hARMyIxMTE4K233pKzBHqRkZE2HZs9ErlHIiFafBV5EmJKVEhISJBrta5KlPWKmZA+ffogKSkJ8+fPl8lzU6ZMsfXQ7E5JSYnMzdJLTU2Vf0ciKVP0RqLaSzNip3gxKyJ6jehzkIKCgmRfJLpq1qxZuPnmm+XfUHFxsXzeRAC3Zs0aWJ3V63fs1J9//qkdNmyYNjg4WOvl5aWNjY3VPv7449oLFy7Yemh2WaIq/nQMXai+yZMnG3yuNm3apHV17733nrZt27ZaT09PWeq7a9cuWw/JLom/FUN/Q+Jvi2pr6LVJvG5RbQ899JA2JiZG/v8LCwvT3nTTTdq1a9dqbcHpc0aIiIjIvrnWQj8RERHZHQYjREREZFMMRoiIiMimGIwQERGRTTEYISIiIptiMEJEREQ2xWCEiIiIbIrBCBEREdkUgxEiIiKyKQYjREREZFMMRoiIiMimGIwQERERbOn/AVyTDpTVNbQPAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plt.plot(x, y)\n", + "plt.scatter(x_train, X_train_s @ beta + y_offset)\n", + "plt.scatter(x_test, X_test_s @ beta + y_offset)" + ] + }, + { + "cell_type": "markdown", + "id": "2993cfae", + "metadata": {}, + "source": [ + "## Exercise 4 - Testing multiple hyperparameters" + ] + }, + { + "cell_type": "markdown", + "id": "e6cdc89a", + "metadata": {}, + "source": [ + "**a)** Compute the MSE of your ridge model for polynomials of degrees 1 to 5 with lambda set to 0.01. Plot the MSE as a function of polynomial degree.\n", + "\n", + "**b)** Compute the MSE of your ridge model for a polynomial with degree 3, and with lambdas from $10^{-1}$ to $10^{-5}$ on a logarithmic scale. Plot the MSE as a function of lambda.\n", + "\n", + "**c)** Compute the MSE of your ridge model for polynomials of degrees 1 to 5, and with lambdas from $10^{-1}$ to $10^{-5}$ on a logarithmic scale. Plot the MSE as a function of polynomial degree and lambda using a [heatmap](https://matplotlib.org/stable/gallery/images_contours_and_fields/image_annotated_heatmap.html)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.15" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/doc/LectureNotes/_build/html/chapter1.html b/doc/LectureNotes/_build/html/chapter1.html index 5fced352b..df8db8de4 100644 --- a/doc/LectureNotes/_build/html/chapter1.html +++ b/doc/LectureNotes/_build/html/chapter1.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/chapter10.html b/doc/LectureNotes/_build/html/chapter10.html index 1c0921613..869a92480 100644 --- a/doc/LectureNotes/_build/html/chapter10.html +++ b/doc/LectureNotes/_build/html/chapter10.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/chapter11.html b/doc/LectureNotes/_build/html/chapter11.html index 9256ab85d..2874d8c2e 100644 --- a/doc/LectureNotes/_build/html/chapter11.html +++ b/doc/LectureNotes/_build/html/chapter11.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/chapter12.html b/doc/LectureNotes/_build/html/chapter12.html index 768f7564c..35bae2997 100644 --- a/doc/LectureNotes/_build/html/chapter12.html +++ b/doc/LectureNotes/_build/html/chapter12.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/chapter13.html b/doc/LectureNotes/_build/html/chapter13.html index 2bb917ddb..0498eab1d 100644 --- a/doc/LectureNotes/_build/html/chapter13.html +++ b/doc/LectureNotes/_build/html/chapter13.html @@ -62,7 +62,7 @@ - + @@ -223,10 +223,11 @@

Weekly material, notes and exercises

@@ -1731,7 +1732,7 @@ plot_results(results, plot_number)

next

diff --git a/doc/LectureNotes/_build/html/chapter2.html b/doc/LectureNotes/_build/html/chapter2.html index 1241fcb90..1ec0257b1 100644 --- a/doc/LectureNotes/_build/html/chapter2.html +++ b/doc/LectureNotes/_build/html/chapter2.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises
diff --git a/doc/LectureNotes/_build/html/chapter3.html b/doc/LectureNotes/_build/html/chapter3.html index 492052da1..86b3b7372 100644 --- a/doc/LectureNotes/_build/html/chapter3.html +++ b/doc/LectureNotes/_build/html/chapter3.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/chapter4.html b/doc/LectureNotes/_build/html/chapter4.html index 0372a062c..f16288205 100644 --- a/doc/LectureNotes/_build/html/chapter4.html +++ b/doc/LectureNotes/_build/html/chapter4.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/chapter5.html b/doc/LectureNotes/_build/html/chapter5.html index 786ded8d0..2b77dd203 100644 --- a/doc/LectureNotes/_build/html/chapter5.html +++ b/doc/LectureNotes/_build/html/chapter5.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/chapter6.html b/doc/LectureNotes/_build/html/chapter6.html index ca992d56c..dfd1a8477 100644 --- a/doc/LectureNotes/_build/html/chapter6.html +++ b/doc/LectureNotes/_build/html/chapter6.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/chapter7.html b/doc/LectureNotes/_build/html/chapter7.html index 7cbce3699..152c5dfc8 100644 --- a/doc/LectureNotes/_build/html/chapter7.html +++ b/doc/LectureNotes/_build/html/chapter7.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/chapter8.html b/doc/LectureNotes/_build/html/chapter8.html index eda68ae73..d5cb0ff3e 100644 --- a/doc/LectureNotes/_build/html/chapter8.html +++ b/doc/LectureNotes/_build/html/chapter8.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/chapter9.html b/doc/LectureNotes/_build/html/chapter9.html index 5f918c5e3..db1949333 100644 --- a/doc/LectureNotes/_build/html/chapter9.html +++ b/doc/LectureNotes/_build/html/chapter9.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/chapteroptimization.html b/doc/LectureNotes/_build/html/chapteroptimization.html index 89800d65c..907b9eb5f 100644 --- a/doc/LectureNotes/_build/html/chapteroptimization.html +++ b/doc/LectureNotes/_build/html/chapteroptimization.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/clustering.html b/doc/LectureNotes/_build/html/clustering.html index 12906f305..e586e5def 100644 --- a/doc/LectureNotes/_build/html/clustering.html +++ b/doc/LectureNotes/_build/html/clustering.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/exercisesweek34.html b/doc/LectureNotes/_build/html/exercisesweek34.html new file mode 100644 index 000000000..aaf73ada9 --- /dev/null +++ b/doc/LectureNotes/_build/html/exercisesweek34.html @@ -0,0 +1,663 @@ + + + + + + + + + + + Exercises week 34 — Applied Data Analysis and Machine Learning + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + +
+
+
+
+ + + Ctrl+K +
+
+ +
+ +
+ + +
+
+ + +
+
+ + + +
+ + + +
+ + + + +
+ +
+
+ + + + + +
+
+ +
+
+
+ + +
+
+ + + + +
+ +
+ + + +
+ +
+
+ +
+
+ +
+ +
+ +
+ + +
+ +
+ +
+ + + + + +
+ + +
+ + + + + + + + + + + + + +
+ +
+ +
+
+ + + +
+

Exercises week 34

+ +
+ +
+
+ + + + +
+ +
+

Exercises week 34#

+
+

Coding Setup and Linear Regression#

+

Welcome to FYS-STK3155/4155!

+

In this first week will focus on getting you set up with the programs you are going to be using throughout this course. We expect that many of you will encounter some trouble with setting these programs up, as they can be extremely finnicky and prone to not working the same on all machines, so we strongly encourage you to not get discouraged, and to show up to the group-sessions where we can help you along. The group sessions are also the best place to find group partners for the projects and to be challenged on your understanding of the material, which are both essential to doing well in this course. We strongly encourage you to form groups of 2-3 participants.

+

If you are unable to complete this week’s exercises, don’t worry, this will likely be the most frustrating week for many of you. You have time to get back on track next week, especially if you come to the group-sessions! Note also that this week’s set of exercises does not count for the additional score. The deadline for the weekly exercises is set to Fridays, at midnight.

+
+

Learning goals#

+

After completing these exercises, you will know how to

+
    +

  • Create and use a Github repository

    • +

  • Set up and use a virtual environment in Python

    • +

  • Fit an OLS model to data using scikit-learn

    • +

  • Fit a model on training data and evaluate it on test data

    • +
+
+
+

Deliverables#

+

Complete the following exercises while working in a jupyter notebook. Exercises 1,2 and 3 require no writing in the notebook. Then, in canvas, include

+ +

We require you to deliver a jupyter notebook so that we can evaluate the results of your code without needing to download and run the code of every student, as well as to teach you to use this useful tool.

+
+
+
+

Exercise 1 - Github Setup#

+

In this course, we require you to pay extra mind to the reproducibility of your results and the shareability of your code. The first step toward these goals is using a version control system like git and online repository like Github.

+

a) Download git if you don’t already have it on your machine, check with the terminal command ´git –version´ (https://git-scm.com/downloads).

+

b) Create a Github account(), or log in to github with your UiO account (https://github.uio.no/login).

+

c) Learn the basics of opening the terminal and navigating folders on your operating system. Things to learn: Opening a terminal, opening a terminal in a specific folder, listing the contents of the current folder, navigating into a folder, navigating out of a folder.

+

d) Download the Github CLI tool and run ´gh auth login´ in your terminal to authenticate your local machine for some of the later steps. (cli/cli). You might need to change file permissions to make it work, ask us or ChatGPT for help with these issues.

+

e) As an alternative to the above terminal based instructions, you could install GitHub Desktop (see https://desktop.github.com/download/) or if you prefer GitLab, GitLab desktop (see https://about.gitlab.com/install/). This sets up all communications between your PC/Laptop and the repository. This allows you to combine exercises 1 and 2 in an easy way if you don’t want to use terminarl. Keep in mind that these GUIs (graphical user interfaces) are not text editors.

+
+
+

Exercise 2 - Setting up a Github repository#

+

a) Create an empty repository for your coursework in this course in your browser at github.com (or uio github).

+

b) Open a terminal in the location you want to create your local folder for this repository, like your desktop.

+

c) Clone the repository to your laptop using the terminal command ´gh repo clone username/repository-name´. This creates a folder with the same name as the repository. Moving it or renaming it might require some extra steps.

+

d) Download this jupyter notebook. Add the notebook to the local folder.

+

e) Run the ´git add .´ command command in a terminal opened in the local folder to stage the current changes in the folder to be commited to the version control history. Run ´git status´ to see the staged files.

+

f) Run the ´git commit -m “Adding first weekly assignment file”´ command to commit the staged changes to the version control history. Run ´git status´ to see that no files are staged.

+

g) Run the ´git push” command to upload the commited changes to the remote repository on Github.

+

h) Add a file called README.txt to the repository at Github.com. Don’t do this in your local folder. Add a suitable title for your repository and some inforomation to the file.

+

i) Run the ´git fetch origin´ command to fetch the latest remote changes to your repository.

+

j) Run the ´git pull´ command to download and update files to match the remote changes.

+
+
+

Exercise 3 - Setting up a Python virtual environment#

+

Following the themes from the previous exercises, another way of improving the reproducibility of your results and shareability of your code is having a good handle on which python packages you are using.

+

There are many ways to manage your packages in Python, and you are free to use any approach you want, but in this course we encourage you to use something called a virtual environment. A virtual environemnt is a folder in your project which contains a Python runtime executable as well as all the packages you are using in the current project. In this way, each of your projects has its required set of packages installed in the same folder, so that if anything goes wrong while managing your packages it only affects the one project, and if multiple projects require different versions of the same package, you don’t need to worry about messing up old projects. Also, it’s easy to just delete the folder and start over if anything goes wrong.

+

Virtual environments are typically created, activated, managed and updated using terminal commands, but for now we recommend that you let for example VS Code (a popular cross-paltform package) handle it for you to make the coding experience much easier. If you are familiar with another approach for virtual environments that works for you, feel free to keep doing it that way.

+

a) Open this notebook in VS Code (https://code.visualstudio.com/Download). Download the Python and Jupyter extensions.

+

b) Press ´Cmd + Shift + P´, then search and run ´Python: Create Environment…´

+

c) Select ´Venv´

+

d) Choose the most up-to-date version of Python your have installed.

+

e) Press ´Cmd + Shift + P´, then search and run ´Python: Select Interpreter´

+

f) Selevet the (.venv) option you just created.

+

g) Open a terminal in VS Code, the venv name should be visible at the beginning of the line. Run pip list to see that there are no packages install in the environment.

+

h) In this terminal, run pip install matplotlib numpy scikit-learn. This will install the listed packages.

+

i) To make these installations reproducible, which is important for reproducing results and sharing your code, run ´pip freeze > requirements.txt´ to create the file requirements.txt with all your dependencies.

+

Now, anyone who wants to recreate your package setup can download your requirements.txt file and run ´pip install -r requirements.txt´ to install the correct packages and versions. To keep the requirements.txt file up to date with your environment, you will need to re-run the freeze command whenever you install a new package.

+

j) Create a .gitignore file at the root of your project folder, and add the line ´.venv´ to it. This way, you won’t try to upload a copy of all your python packages when you regularly push your changes to Github. Ignored files should not show up when you run ´git status´, and are not staged when running ´git add .´, try it!

+
+
+

Exercise 3 - Fitting an OLS model to data#

+

Great job on getting through all of that! Now it is time to do some actual machine learning!

+

a) Complete the code below so that you fit a second order polynomial to the data. You will need to look up some scikit-learn documentation online (look at the imported functions for hints).

+

b) Compute the mean square error for the line model and for the second degree polynomial model.

+
+
+
import numpy as np
+import matplotlib.pyplot as plt
+from sklearn.preprocessing import PolynomialFeatures # use the fit_transform method of the created object!
+from sklearn.linear_model import LinearRegression
+from sklearn.metrics import mean_squared_error
+
+
+
+
+
+
+
n = 100
+x = np.random.rand(n, 1)
+y = 2.0 + 5 * x**2 + 0.1 * np.random.randn(n, 1)
+
+line_model = LinearRegression().fit(x, y)
+line_predict = line_model.predict(x)
+#line_mse = ...
+
+#poly_features = ...
+#poly_model = LinearRegression().fit(..., y)
+#poly_predict = ...
+#poly_mse = ...
+
+plt.scatter(x, y, label = "Data")
+plt.scatter(x, line_predict, label = "Line model")
+plt.legend()
+plt.show()
+
+
+
+
+_images/c62786c19b580c6638248aa3cfe7dea30bdbc00a922e7e45a45a5a7a053bdb38.png +
+
+
+
+

Exercise 4 - The train-test split#

+

Hopefully your model fit the data quite well, but to know how well the model actually generalizes to unseen data, which is most often what we care about, we need to split our data into training and testing data.

+
+
+
from sklearn.model_selection import train_test_split
+
+
+
+
+

a) Complete the code below so that the polynomial features and the targets y get split into training and test data.

+

b) What is the shape of X_test?

+

c) Fit your model to X_train

+

d) Compute the MSE when your model predicts on the training data and on the testing data, using y_train and y_test as targets for the two cases.

+

e) Why do we not fit the model to X_test?

+
+
+
polynomial_features = ...
+
+#X_train, X_test, y_train, y_test = train_test_split(polynomial_features, y, test_size=0.2)
+
+
+
+
+
+
+ + + + +
+ + + + + + + + +
+ + + +
+ + +
+
+ Contents +
+
+ +
+ + +
+
+ +
+ +
+ +

+By Morten Hjorth-Jensen +

+ +
+ +
+ + +

+ + © Copyright 2023. +
+ +

+ +
+ +
+ +
+ +
+ +
+ +
+
+ + +
+
+
+ + + + + +
+
+ + \ No newline at end of file diff --git a/doc/LectureNotes/_build/html/exercisesweek35.html b/doc/LectureNotes/_build/html/exercisesweek35.html new file mode 100644 index 000000000..85b63cce4 --- /dev/null +++ b/doc/LectureNotes/_build/html/exercisesweek35.html @@ -0,0 +1,760 @@ + + + + + + + + + + + Exercises week 35 — Applied Data Analysis and Machine Learning + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + +
+
+
+
+ + + Ctrl+K +
+
+ +
+ +
+ + +
+
+ + +
+
+ + + +
+ + + +
+ + + + +
+ +
+
+ + + + + +
+
+ +
+
+
+ + +
+
+ + + + +
+ +
+ + + +
+ +
+
+ +
+
+ +
+ +
+ +
+ + +
+ +
+ +
+ + + + + +
+ + +
+ + + + + + + + + + + + + +
+ +
+ +
+
+ + + +
+

Exercises week 35

+ +
+ +
+
+ + + + +
+ +
+

Exercises week 35#

+
+

Deriving and Implementing Ordinary Least Squares#

+

This week you will be deriving the analytical expressions for linear regression, building up the model from scratch. This will include taking several derivatives of products of vectors and matrices. Such derivatives are central to the optimization of many machine learning models. Although we will often use automatic differentiation in actual calculations, to be able to have analytical expressions is extremely helpful in case we have simpler derivatives as well as when we analyze various properties (like second derivatives) of the chosen cost functions.

+

Vectors are always written as boldfaced lower case letters and matrices as upper case boldfaced letters. You will find useful the notes from week 35 on derivatives of vectors and matrices. See also the textbook of Faisal at al, chapter 5 and in particular sections 5.3-5.5 at CompPhysics/MachineLearning

+
+

Learning goals#

+

After completing these exercises, you will know how to

+
    +

  • Take the derivatives of simple products between vectors and matrices

    • +

  • Implement OLS using the analytical expressions

    • +

  • Create a feature matrix from a set of data

    • +

  • Create a feature matrix for a polynomial model

    • +

  • Evaluate the MSE score of various model on training and test data, and comparing their performance

    • +
+
+
+

Deliverables#

+

Complete the following exercises while working in a jupyter notebook. Then, in canvas, include

+ +
+
+
+

How to take derivatives of Matrix-Vector expressions#

+

In these exercises it is always useful to write out with summation indices the various quantities. Take also a look at the weekly slides from week 35 and the various examples included there.

+

As an example, consider the function

+
+\[ +f(\boldsymbol{x}) =\boldsymbol{A}\boldsymbol{x}, +\]
+

which reads for a specific component \(f_i\) (we define the matrix \(\boldsymbol{A}\) to have dimension \(n\times n\) and the vector \(\boldsymbol{x}\) to have length \(n\))

+
+\[ +f_i =\sum_{j=0}^{n-1}a_{ij}x_j, +\]
+

which leads to

+
+\[ +\frac{\partial f_i}{\partial x_j}= a_{ij}, +\]
+

and written out in terms of the vector \(\boldsymbol{x}\) we have

+
+\[ +\frac{\partial f(\boldsymbol{x})}{\partial \boldsymbol{x}}= \boldsymbol{A}. +\]
+
+
+

Exercise 1 - Finding the derivative of Matrix-Vector expressions#

+

a) Consider the expression

+
+\[ +\frac{\partial (\boldsymbol{a}^T\boldsymbol{x})}{\partial \boldsymbol{x}}, +\]
+

Where \(\boldsymbol{a}\) and \(\boldsymbol{x}\) are column-vectors with length \(n\).

+

What is the shape of the expression we are taking the derivative of?

+

What is the shape of the thing we are taking the derivative with respect to?

+

What is the shape of the result of the expression?

+

b) Show that

+
+\[ +\frac{\partial (\boldsymbol{a}^T\boldsymbol{x})}{\partial \boldsymbol{x}} = \boldsymbol{a}^T, +\]
+

c) Show that

+
+\[ +\frac{\partial (\boldsymbol{a}^T\boldsymbol{A}\boldsymbol{a})}{\partial \boldsymbol{a}} = \boldsymbol{a}^T(\boldsymbol{A}+\boldsymbol{A}^T), +\]
+
+
+

Exercise 2 - Deriving the expression for OLS#

+

The ordinary least squares method finds the parameters \(\boldsymbol{\theta}\) which minimizes the squared error between our model \(\boldsymbol{X\theta}\) and the true values \(\boldsymbol{y}\).

+

To find the parameters \(\boldsymbol{\theta}\) which minimizes this error, we take the derivative of the squared error expression with respect to \(\boldsymbol{\theta}\), and set it equal to 0.

+

a) Very briefly explain why the approach above finds the parameters \(\boldsymbol{\theta}\) which minimizes this error.

+

We typically write the squared error as

+
+\[ +\vert\vert\boldsymbol{y} - \boldsymbol{X\theta}\vert\vert^2 +\]
+

which we can rewrite in matrix-vector form as

+
+\[ +\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right) +\]
+

b) If \(\boldsymbol{X}\) is invertible, what is the expression for the optimal parameters \(\boldsymbol{\theta}\)? (Hint: Don’t compute any derivatives, but solve \(\boldsymbol{X\theta}=\boldsymbol{y}\) for \(\boldsymbol{\theta}\))

+

c) Show that

+
+\[ +\frac{\partial \left(\boldsymbol{x}-\boldsymbol{A}\boldsymbol{s}\right)^T\left(\boldsymbol{x}-\boldsymbol{A}\boldsymbol{s}\right)}{\partial \boldsymbol{s}} = -2\left(\boldsymbol{x}-\boldsymbol{A}\boldsymbol{s}\right)^T\boldsymbol{A}, +\]
+

d) Using the expression from c), but substituting back in \(\boldsymbol{\theta}\), \(\boldsymbol{y}\) and \(\boldsymbol{X}\), find the expression for the optimal parameters \(\boldsymbol{\theta}\) in the case that \(\boldsymbol{X}\) is not invertible, but \(\boldsymbol{X^T X}\) is, which is most often the case.

+
+\[ +\boldsymbol{\hat{\theta}_{OLS}} = ... +\]
+
+
+

Exercise 3 - Creating feature matrix and implementing OLS using the analytical expression#

+

With the expression for \(\boldsymbol{\hat{\theta}_{OLS}}\), you now have what you need to implement OLS regression with your input data and target data \(\boldsymbol{y}\). But before you can do that, you need to set up you input data as a feature matrix \(\boldsymbol{X}\).

+

In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, with the montly income and the number of children as columns.

+

We typically also include an intercept in our models. The intercept is a value that is added to our prediction regardless of the value of the other features. The intercept tries to account for constant effects in our data that are not dependant on anything else. In our current example, the intercept could account for living expenses which are typical regardless of income or childcare expenses.

+

We calculate the optimal intercept by including a feature with the constant value of 1 in our model, which is then multplied by some parameter \(\theta_0\) from the OLS method into the optimal intercept value (which will be \(\theta_0\)). In practice, we include the intercept in our model by adding a column of ones to the start of our feature matrix.

+
+
+
import numpy as np
+
+
+
+
+
+
+
n = 20
+income = np.array([116., 161., 167., 118., 172., 163., 179., 173., 162., 116., 101., 176., 178., 172., 143., 135., 160., 101., 149., 125.])
+children = np.array([5, 3, 0, 4, 5, 3, 0, 4, 4, 3, 3, 5, 1, 0, 2, 3, 2, 1, 5, 4])
+spending = np.array([152., 141., 102., 136., 161., 129.,  99., 159., 160., 107.,  98., 164., 121.,  93., 112., 127., 117.,  69., 156., 131.])
+
+
+
+
+

a) Create a feature matrix \(\boldsymbol{X}\) for the features income and children, including an intercept column of ones at the start.

+
+
+
X = np.zeros((n, 3))
+#X[:, 0] = ...
+#X[:, 1] = ...
+#X[:, 2] = ...
+
+
+
+
+

b) Use the expression from 3d) to find the optimal parameters \(\boldsymbol{\hat{\beta}_{OLS}}\) for predicting spending based on these features. Create a function for this operation, as you are going to need to use it a lot.

+
+
+
def OLS_parameters(X, y):
+    return ...
+
+#beta = OLS_parameters(X, y)
+
+
+
+
+
+
+

Exercise 4 - Fitting a polynomial#

+

In this course, we typically do linear regression using polynomials, though in real world applications it is also very common to make linear models based on measured features like you did in the previous exercise.

+

When fitting a polynomial with linear regression, we make each polynomial degree(\(x, x^2, x^3, ..., x^p\)) its own feature.

+
+
+
n = 100
+x = np.linspace(-3, 3, n)
+y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2) + np.random.normal(0, 0.1)
+
+
+
+
+

a) Create a feature matrix \(\boldsymbol{X}\) for the features \(x, x^2, x^3, x^4, x^5\), including an intercept column of ones at the start. Make this into a function, as you will do this a lot over the next weeks.

+
+
+
def polynomial_features(x, p):
+    n = len(x)
+    X = np.zeros((n, p + 1))
+    #X[:, 0] = ...
+    #X[:, 1] = ...
+    #X[:, 2] = ...
+    # could this be a loop?
+
+#X = polynomial_features(x, 5)
+
+
+
+
+

b) Use the expression from 3d) to find the optimal parameters \(\boldsymbol{\hat{\beta}_{OLS}}\) for predicting \(\boldsymbol{y}\) based on these features. If you have done everything right so far, this code will not need changing.

+
+
+
#beta = OLS_parameters(X, y)
+
+
+
+
+

c) Like in exercise 4 last week, split your feature matrix and target data into a training split and test split.

+
+
+
from sklearn.model_selection import train_test_split
+
+#X_train, X_test, y_train, y_test = ...
+
+
+
+
+

d) Train your model on the training data(find the parameters which best fit) and compute the MSE on both the training and test data.

+
+
+
...
+
+
+
+
+
Ellipsis
+
+
+
+
+

e) Do the same for each polynomial degree from 2 to 10, and plot the MSE on both the training and test data as a function of polynomial degree. The aim is to reproduce Figure 2.11 of Hastie et al. Feel free to read the discussions leading to figure 2.11 of Hastie et al.

+
+
+
...
+
+
+
+
+
Ellipsis
+
+
+
+
+

f) Interpret the graph. Why do the lines move as they do? What does it tell us about model performance and generalizability?

+
+
+

Exercise 5 - Comparing your code with sklearn#

+

When implementing different algorithms for the first time, it can be helpful to double check your results with established implementations before you go on to add more complexity.

+

a) Make sure your polynomial_features function creates the same feature matrix as sklearns PolynomialFeatures.

+

(https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html)

+

b) Make sure your OLS_parameters function computes the same parameters as sklearns LinearRegression with fit_intercept set to False, since the intercept is included in the feature matrix. Use your_model_object.coef_ to extract the computed parameters.

+

(https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html)

+
+
+ + + + +
+ + + + + + + + +
+ + + +
+ + +
+
+ Contents +
+
+ +
+ + +
+
+ +
+ +
+ +

+By Morten Hjorth-Jensen +

+ +
+ +
+ + +

+ + © Copyright 2023. +
+ +

+ +
+ +
+ +
+ +
+ +
+ +
+
+ + +
+
+
+ + + + + +
+
+ + \ No newline at end of file diff --git a/doc/LectureNotes/_build/html/exercisesweek36.html b/doc/LectureNotes/_build/html/exercisesweek36.html new file mode 100644 index 000000000..288b02bf3 --- /dev/null +++ b/doc/LectureNotes/_build/html/exercisesweek36.html @@ -0,0 +1,691 @@ + + + + + + + + + + + Weekly Exercises 3 — Applied Data Analysis and Machine Learning + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + + + + + + + +
+
+
+
+ + + Ctrl+K +
+
+ +
+ +
+ + +
+
+ + +
+
+ + + +
+ + + +
+ + + + +
+ +
+
+ + + + + +
+
+ +
+
+
+ + +
+
+ + + + +
+ +
+ + + +
+ +
+
+ +
+
+ +
+ +
+ +
+ + +
+ +
+ +
+ + + + + +
+ + +
+ + + + + + + + + + + + + +
+ +
+ +
+
+ + + +
+

Weekly Exercises 3

+ +
+ +
+
+ + + + +
+ +
+

Weekly Exercises 3#

+
+

Deriving and Implementing Ridge Regression#

+
+
+

Learning goals#

+

After completing these exercises, you will know how to

+
    +

  • Take more derivatives of simple products between vectors and matrices

    • +

  • Implement Ridge regression using the analytical expressions

    • +

  • Scale data appropriately for linear regression

    • +

  • Evaluate a model across two different hyperparameters

    • +
+
+
+

Exercise 1 - Choice of model and degrees of freedom#

+

a) How many degrees of freedom does an OLS model fit to the features \(x, x^2, x^3\) and the intercept have?

+

b) Why is it bad for a model to have too many degrees of freedom?

+

c) Why is it bad for a model to have too few degrees of freedom?

+

d) Read chapter 3.4.1 of Hastie et al.’s book. What is the expression for the effective degrees of freedom of the ridge regression fit?

+

e) Why might we want to use Ridge regression instead of OLS?

+

f) Why migth we want to use OLS instead of Ridge regression?

+
+
+

Exercise 2 - Deriving the expression for Ridge Regression#

+

The aim here is to derive the expression for the optimal parameters using Ridge regression.

+

The expression for the standard Mean Squared Error (MSE) which we used to define our cost function and the equations for the ordinary least squares (OLS) method, was given by the optimization problem

+
+\[ +{\displaystyle \min_{\boldsymbol{\beta}\in {\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}. +\]
+

By minimizing the above equation with respect to the parameters \(\boldsymbol{\beta}\) we could then obtain an analytical expression for the parameters \(\boldsymbol{\hat\beta_{OLS}}\).

+

We can add a regularization parameter \(\lambda\) by +defining a new cost function to be optimized, that is

+
+\[ +{\displaystyle \min_{\boldsymbol{\beta}\in +{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_2^2 +\]
+

which leads to the Ridge regression minimization problem. (One can require as part of the optimization problem that \(\vert\vert \boldsymbol{\beta}\vert\vert_2^2\le t\), where \(t\) is a finite number larger than zero. We will not implement that in this course.)

+
+

a) Expression for Ridge regression#

+

Show that the optimal parameters

+
+\[ +\hat{\boldsymbol{\beta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}, +\]
+

with \(\boldsymbol{I}\) being a \(p\times p\) identity matrix.

+

The ordinary least squares result is

+
+\[ +\hat{\boldsymbol{\beta}}_{\mathrm{OLS}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}, +\]
+
+
+
+

Exercise 3 - Scaling data#

+
+
+
import numpy as np
+import matplotlib.pyplot as plt
+from sklearn.model_selection import train_test_split
+from sklearn.preprocessing import StandardScaler
+
+
+
+
+
+
+
n = 100
+x = np.linspace(-3, 3, n)
+y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2) + np.random.normal(0, 0.1)
+
+
+
+
+

a) Adapt your function from last week to only include the intercept column if the boolean argument intercept is set to true.

+
+
+
def polynomial_features(x, p, intercept=False):
+    n = len(x)
+    X = np.zeros((n, p + 1))
+    #X[:, 0] = ...
+    #X[:, 1] = ...
+    #X[:, 2] = ...
+    # could this be a loop?
+
+
+
+
+
+
+
def polynomial_features(x, p, intercept=False):
+    n = len(x)
+    X = np.zeros((n, p))
+    X[:, 0] = x[:]
+    X[:, 1] = x**2
+    X[:, 2] = x**3
+    return X
+
+
+
+
+

b) Split your data into training and test data(80/20 split)

+
+
+
X = polynomial_features(x, 3)
+
+
+
+
+
+
+
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
+x_train = X_train[:, 0] # These are used for plotting
+x_test = X_test[:, 0] # These are used for plotting
+
+
+
+
+

c) Scale your design matrix with the sklearn standard scaler, though based on the mean and standard deviation of the training data only.

+
+
+
scaler = StandardScaler()
+scaler.fit(X_train)
+X_train_s = scaler.transform(X_train)
+X_test_s = scaler.transform(X_test)
+y_offset = np.mean(y_train)
+
+
+
+
+
+
+

Exercise 4 - Implementing Ridge Regression#

+

a) Implement a function for computing the optimal Ridge parameters using the expression from 2a).

+
+
+
def Ridge_parameters(X, y):
+    # Assumes X is scaled and has no intercept column
+    return np.linalg.inv(X.T @ X) @ X.T @ y
+
+beta = Ridge_parameters(X_train_s, y_train)
+
+
+
+
+

b) Fit a model to the data, and plot the prediction using both the training and test x-values extracted before scaling, and the y_offset.

+
+
+
plt.plot(x, y)
+plt.scatter(x_train, X_train_s @ beta + y_offset)
+plt.scatter(x_test, X_test_s @ beta + y_offset)
+
+
+
+
+
<matplotlib.collections.PathCollection at 0x113e21950>
+
+
+_images/cb756715680fbe8cbf1d070371bf3950695ddf24b29ecb476c48fb9e559af08e.png +
+
+
+
+

Exercise 4 - Testing multiple hyperparameters#

+

a) Compute the MSE of your ridge model for polynomials of degrees 1 to 5 with lambda set to 0.01. Plot the MSE as a function of polynomial degree.

+

b) Compute the MSE of your ridge model for a polynomial with degree 3, and with lambdas from \(10^{-1}\) to \(10^{-5}\) on a logarithmic scale. Plot the MSE as a function of lambda.

+

c) Compute the MSE of your ridge model for polynomials of degrees 1 to 5, and with lambdas from \(10^{-1}\) to \(10^{-5}\) on a logarithmic scale. Plot the MSE as a function of polynomial degree and lambda using a heatmap.

+
+
+ + + + +
+ + + + + + + + +
+ + + +
+ + +
+
+ Contents +
+
+ +
+ + +
+
+ +
+ +
+ +

+By Morten Hjorth-Jensen +

+ +
+ +
+ + +

+ + © Copyright 2023. +
+ +

+ +
+ +
+ +
+ +
+ +
+ +
+
+ + +
+
+
+ + + + + +
+
+ + \ No newline at end of file diff --git a/doc/LectureNotes/_build/html/genindex.html b/doc/LectureNotes/_build/html/genindex.html index 10764068d..9f60090ee 100644 --- a/doc/LectureNotes/_build/html/genindex.html +++ b/doc/LectureNotes/_build/html/genindex.html @@ -220,10 +220,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/intro.html b/doc/LectureNotes/_build/html/intro.html index a91b04569..0b745b6fe 100644 --- a/doc/LectureNotes/_build/html/intro.html +++ b/doc/LectureNotes/_build/html/intro.html @@ -224,10 +224,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/linalg.html b/doc/LectureNotes/_build/html/linalg.html index b83150d91..51a4d76bb 100644 --- a/doc/LectureNotes/_build/html/linalg.html +++ b/doc/LectureNotes/_build/html/linalg.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/objects.inv b/doc/LectureNotes/_build/html/objects.inv index a90c6d46d..99eb8b010 100644 Binary files a/doc/LectureNotes/_build/html/objects.inv and b/doc/LectureNotes/_build/html/objects.inv differ diff --git a/doc/LectureNotes/_build/html/schedule.html b/doc/LectureNotes/_build/html/schedule.html index 5ebaa40f4..f01e24174 100644 --- a/doc/LectureNotes/_build/html/schedule.html +++ b/doc/LectureNotes/_build/html/schedule.html @@ -221,10 +221,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/search.html b/doc/LectureNotes/_build/html/search.html index 303d77f62..f956c387e 100644 --- a/doc/LectureNotes/_build/html/search.html +++ b/doc/LectureNotes/_build/html/search.html @@ -222,10 +222,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/searchindex.js b/doc/LectureNotes/_build/html/searchindex.js index ee128239d..8a9f7b94f 100644 --- a/doc/LectureNotes/_build/html/searchindex.js +++ b/doc/LectureNotes/_build/html/searchindex.js @@ -1 +1 @@ -Search.setIndex({"alltitles": {"A Classification Tree": [[11, "a-classification-tree"]], "A Frequentist approach to data analysis": [[2, "a-frequentist-approach-to-data-analysis"], [23, "a-frequentist-approach-to-data-analysis"]], "A better approach": [[10, "a-better-approach"]], "A first summary": [[23, "a-first-summary"]], "A quick Reminder on Lagrangian Multipliers": [[10, "a-quick-reminder-on-lagrangian-multipliers"]], "A simple example": [[6, "a-simple-example"]], "A soft classifier": [[10, "a-soft-classifier"]], "A top-down perspective on Neural networks": [[3, "a-top-down-perspective-on-neural-networks"]], "ADAM optimizer": [[15, "adam-optimizer"]], "Activation functions": [[14, "activation-functions"]], "Adaptive boosting: AdaBoost, Basic Algorithm": [[12, "adaptive-boosting-adaboost-basic-algorithm"]], "Adding error analysis and training set up": [[23, "adding-error-analysis-and-training-set-up"], [24, "adding-error-analysis-and-training-set-up"]], "Adjust hyperparameters": [[3, "adjust-hyperparameters"]], "Algorithms for Setting up Decision Trees": [[11, "algorithms-for-setting-up-decision-trees"]], "An Overview of Ensemble Methods": [[12, "an-overview-of-ensemble-methods"]], "An extrapolation example": [[6, "an-extrapolation-example"]], "An optimization/minimization problem": [[23, "an-optimization-minimization-problem"]], "And finally \\boldsymbol{X}\\boldsymbol{X}^T": [[24, "and-finally-boldsymbol-x-boldsymbol-x-t"]], "And what about using neural networks?": [[23, "and-what-about-using-neural-networks"]], "Another example, the moons again": [[11, "another-example-the-moons-again"]], "Applied Data Analysis and Machine Learning": [[17, null]], "Autocorrelation function": [[20, "autocorrelation-function"]], "Automatic differentiation": [[15, "automatic-differentiation"]], "Back to Ridge and LASSO Regression": [[24, "back-to-ridge-and-lasso-regression"]], "Back to the Cancer Data": [[13, "back-to-the-cancer-data"]], "Bagging": [[12, "bagging"]], "Bagging Examples": [[12, "bagging-examples"]], "Basic Matrix Features": [[18, "basic-matrix-features"]], "Basic ideas of the Principal Component Analysis (PCA)": [[13, null]], "Basic math of the SVD": [[7, "basic-math-of-the-svd"], [24, "basic-math-of-the-svd"]], "Basics": [[9, "basics"]], "Basics of a tree": [[11, "basics-of-a-tree"]], "Batch Normalization": [[3, "batch-normalization"]], "Bayes\u2019 Theorem and Ridge and Lasso Regression": [[7, "bayes-theorem-and-ridge-and-lasso-regression"]], "Boosting, a Bird\u2019s Eye View": [[12, "boosting-a-bird-s-eye-view"]], "Bootstrap": [[8, "bootstrap"]], "Bringing it together, first back propagation equation": [[14, "bringing-it-together-first-back-propagation-equation"]], "Building a Feed Forward Neural Network": [[3, null]], "Building a tree, regression": [[11, "building-a-tree-regression"]], "Building neural networks in Tensorflow and Keras": [[3, "building-neural-networks-in-tensorflow-and-keras"]], "CNNs in more detail, building convolutional neural networks in Tensorflow and Keras": [[5, "cnns-in-more-detail-building-convolutional-neural-networks-in-tensorflow-and-keras"]], "Cancer Data again now with Decision Trees and other Methods": [[11, "cancer-data-again-now-with-decision-trees-and-other-methods"]], "Choose cost function and optimizer": [[3, "choose-cost-function-and-optimizer"]], "Classical PCA Theorem": [[13, "classical-pca-theorem"]], "Clustering and Unsupervised Learning": [[16, null]], "Code for SVD and Inversion of Matrices": [[7, "code-for-svd-and-inversion-of-matrices"]], "Codes and Approaches": [[16, "codes-and-approaches"]], "Codes for the SVD": [[7, "codes-for-the-svd"], [24, "codes-for-the-svd"]], "Coding Setup and Linear Regression": [[0, "coding-setup-and-linear-regression"]], "Collect and pre-process data": [[3, "collect-and-pre-process-data"]], "Communication channels": [[23, "communication-channels"]], "Compare Bagging on Trees with Random Forests": [[12, "compare-bagging-on-trees-with-random-forests"]], "Comparing with a numerical scheme": [[4, "comparing-with-a-numerical-scheme"]], "Computing the Gini index": [[11, "computing-the-gini-index"]], "Conjugate gradient method": [[15, "conjugate-gradient-method"]], "Convex functions": [[15, "convex-functions"]], "Convolution Examples: Polynomial multiplication": [[5, "convolution-examples-polynomial-multiplication"]], "Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)": [[5, "convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms"]], "Convolutional Neural Network": [[14, "convolutional-neural-network"]], "Convolutional Neural Networks": [[5, null]], "Correlation Function and Design/Feature Matrix": [[24, "correlation-function-and-design-feature-matrix"]], "Correlation Matrix": [[13, "correlation-matrix"], [24, "correlation-matrix"]], "Correlation Matrix with Pandas": [[24, "correlation-matrix-with-pandas"]], "Course Format": [[23, "course-format"]], "Course setting": [[19, null]], "Covariance Matrix Examples": [[24, "covariance-matrix-examples"]], "Covariance and Correlation Matrix": [[24, "covariance-and-correlation-matrix"]], "Cross-validation": [[8, "cross-validation"]], "Deadlines for projects (tentative)": [[23, "deadlines-for-projects-tentative"]], "Decision trees, overarching aims": [[11, null]], "Deep learning methods": [[23, "deep-learning-methods"]], "Define model and architecture": [[3, "define-model-and-architecture"]], "Defining the cost function": [[3, "defining-the-cost-function"]], "Deliverables": [[0, "deliverables"], [1, "deliverables"]], "Derivatives and the chain rule": [[14, "derivatives-and-the-chain-rule"]], "Derivatives, example 1": [[24, "derivatives-example-1"]], "Deriving OLS from a probability distribution": [[7, "deriving-ols-from-a-probability-distribution"]], "Deriving and Implementing Ordinary Least Squares": [[1, "deriving-and-implementing-ordinary-least-squares"]], "Deriving the Lasso Regression Equations": [[24, "deriving-the-lasso-regression-equations"]], "Deriving the Ridge Regression Equations": [[24, "deriving-the-ridge-regression-equations"]], "Deriving the back propagation code for a multilayer perceptron model": [[14, "deriving-the-back-propagation-code-for-a-multilayer-perceptron-model"]], "Developing a code for doing neural networks with back propagation": [[3, "developing-a-code-for-doing-neural-networks-with-back-propagation"]], "Diagonalize the sample covariance matrix to obtain the principal components": [[13, "diagonalize-the-sample-covariance-matrix-to-obtain-the-principal-components"]], "Different kernels and Mercer\u2019s theorem": [[10, "different-kernels-and-mercer-s-theorem"]], "Disadvantages": [[11, "disadvantages"]], "Discriminative Modeling": [[23, "discriminative-modeling"]], "Domains and probabilities": [[20, "domains-and-probabilities"]], "Dropout": [[3, "dropout"]], "Economy-size SVD": [[24, "economy-size-svd"]], "Elements of Probability Theory and Statistical Data Analysis": [[20, null]], "Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods": [[12, null]], "Entropy and the ID3 algorithm": [[11, "entropy-and-the-id3-algorithm"]], "Essential elements of ML": [[23, "essential-elements-of-ml"]], "Evaluate model performance on test data": [[3, "evaluate-model-performance-on-test-data"]], "Example 2": [[24, "example-2"]], "Example 3": [[24, "example-3"]], "Example 4": [[24, "example-4"]], "Example Matrix": [[24, "example-matrix"]], "Example of discriminative modeling, taken from Generative Deep Learning by David Foster": [[23, "example-of-discriminative-modeling-taken-from-generative-deep-learning-by-david-foster"]], "Example of generative modeling, taken from Generative Deep Learning by David Foster": [[23, "example-of-generative-modeling-taken-from-generative-deep-learning-by-david-foster"]], "Example of own Standard scaling": [[24, "example-of-own-standard-scaling"]], "Example relevant for the exercises": [[24, "example-relevant-for-the-exercises"]], "Example: Exponential decay": [[4, "example-exponential-decay"]], "Example: Population growth": [[4, "example-population-growth"]], "Example: The diffusion equation": [[4, "example-the-diffusion-equation"]], "Example: binary classification problem": [[3, "example-binary-classification-problem"]], "Examples": [[23, "examples"]], "Examples of likelihood functions used in logistic regression and neural networks": [[9, "examples-of-likelihood-functions-used-in-logistic-regression-and-neural-networks"]], "Exercise 1 - Finding the derivative of Matrix-Vector expressions": [[1, "exercise-1-finding-the-derivative-of-matrix-vector-expressions"]], "Exercise 1 - Github Setup": [[0, "exercise-1-github-setup"]], "Exercise 1: Setting up various Python environments": [[2, "exercise-1-setting-up-various-python-environments"]], "Exercise 2 - Deriving the expression for OLS": [[1, "exercise-2-deriving-the-expression-for-ols"]], "Exercise 2 - Setting up a Github repository": [[0, "exercise-2-setting-up-a-github-repository"]], "Exercise 2: making your own data and exploring scikit-learn": [[2, "exercise-2-making-your-own-data-and-exploring-scikit-learn"]], "Exercise 3 - Creating feature matrix and implementing OLS using the analytical expression": [[1, "exercise-3-creating-feature-matrix-and-implementing-ols-using-the-analytical-expression"]], "Exercise 3 - Fitting an OLS model to data": [[0, "exercise-3-fitting-an-ols-model-to-data"]], "Exercise 3 - Setting up a Python virtual environment": [[0, "exercise-3-setting-up-a-python-virtual-environment"]], "Exercise 3: Normalizing our data": [[2, "exercise-3-normalizing-our-data"]], "Exercise 4 - Fitting a polynomial": [[1, "exercise-4-fitting-a-polynomial"]], "Exercise 4 - The train-test split": [[0, "exercise-4-the-train-test-split"]], "Exercise 4: Adding Ridge Regression": [[2, "exercise-4-adding-ridge-regression"]], "Exercise 5 - Comparing your code with sklearn": [[1, "exercise-5-comparing-your-code-with-sklearn"]], "Exercise 5: Analytical exercises": [[2, "exercise-5-analytical-exercises"]], "Exercise: Cross-validation as resampling techniques, adding more complexity": [[8, "exercise-cross-validation-as-resampling-techniques-adding-more-complexity"]], "Exercise: Analysis of real data": [[8, "exercise-analysis-of-real-data"]], "Exercise: Bias-variance trade-off and resampling techniques": [[8, "exercise-bias-variance-trade-off-and-resampling-techniques"]], "Exercise: Lasso Regression on the Franke function with resampling": [[8, "exercise-lasso-regression-on-the-franke-function-with-resampling"]], "Exercise: Ordinary Least Square (OLS) on the Franke function": [[8, "exercise-ordinary-least-square-ols-on-the-franke-function"]], "Exercise: Ridge Regression on the Franke function with resampling": [[8, "exercise-ridge-regression-on-the-franke-function-with-resampling"]], "Exercises": [[2, "exercises"]], "Exercises and Projects": [[8, "exercises-and-projects"]], "Exercises week 34": [[0, null]], "Exercises week 35": [[1, null]], "Expectation values": [[20, "expectation-values"]], "Extremely useful tools, strongly recommended": [[23, "extremely-useful-tools-strongly-recommended"]], "Feed-forward neural networks": [[14, "feed-forward-neural-networks"]], "Feed-forward pass": [[3, "feed-forward-pass"]], "Final back propagating equation": [[14, "final-back-propagating-equation"]], "Fine-tuning neural network hyperparameters": [[3, "fine-tuning-neural-network-hyperparameters"]], "Fitting an Equation of State for Dense Nuclear Matter": [[2, "fitting-an-equation-of-state-for-dense-nuclear-matter"]], "Fixing the singularity": [[24, "fixing-the-singularity"]], "Frequently used scaling functions": [[24, "frequently-used-scaling-functions"]], "From one to many layers, the universal approximation theorem": [[14, "from-one-to-many-layers-the-universal-approximation-theorem"]], "Functionality in Scikit-Learn": [[24, "functionality-in-scikit-learn"]], "Further Dimensionality Remarks": [[5, "further-dimensionality-remarks"]], "Further properties (important for our analyses later)": [[7, "further-properties-important-for-our-analyses-later"], [24, "further-properties-important-for-our-analyses-later"]], "Gaussian Elimination": [[18, "gaussian-elimination"]], "General Features": [[11, "general-features"]], "General linear models and linear algebra": [[23, "general-linear-models-and-linear-algebra"]], "Generalizing the fitting procedure as a linear algebra problem": [[23, "generalizing-the-fitting-procedure-as-a-linear-algebra-problem"], [23, "id1"]], "Generative Adversarial Networks": [[6, "generative-adversarial-networks"]], "Generative Models": [[6, "generative-models"]], "Generative Versus Discriminative Modeling": [[23, "generative-versus-discriminative-modeling"]], "Geometric Interpretation and link with Singular Value Decomposition": [[13, "geometric-interpretation-and-link-with-singular-value-decomposition"]], "Gradient Boosting, Classification Example": [[12, "gradient-boosting-classification-example"]], "Gradient Boosting, Examples of Regression": [[12, "gradient-boosting-examples-of-regression"]], "Gradient Clipping": [[3, "gradient-clipping"]], "Gradient boosting: Basics with Steepest Descent/Functional Gradient Descent": [[12, "gradient-boosting-basics-with-steepest-descent-functional-gradient-descent"]], "Gradient descent": [[4, "gradient-descent"]], "Grading": [[21, "grading"], [21, "id2"], [23, "grading"]], "How to take derivatives of Matrix-Vector expressions": [[1, "how-to-take-derivatives-of-matrix-vector-expressions"]], "Hyperplanes and all that": [[10, "hyperplanes-and-all-that"]], "Important Matrix and vector handling packages": [[18, "important-matrix-and-vector-handling-packages"]], "Improving performance": [[3, "improving-performance"]], "In summary": [[21, "in-summary"]], "Including Stochastic Gradient Descent with Autograd": [[15, "including-stochastic-gradient-descent-with-autograd"]], "Incremental PCA": [[13, "incremental-pca"]], "Installing R, C++, cython or Julia": [[23, "installing-r-c-cython-or-julia"]], "Installing R, C++, cython, Numba etc": [[23, "installing-r-c-cython-numba-etc"]], "Instructor information": [[21, "instructor-information"]], "Interpretations and optimizing our parameters": [[23, "interpretations-and-optimizing-our-parameters"], [23, "id2"], [23, "id3"], [24, "interpretations-and-optimizing-our-parameters"], [24, "id1"], [24, "id2"]], "Interpreting the Ridge results": [[24, "interpreting-the-ridge-results"]], "Introducing JAX": [[15, "introducing-jax"]], "Introducing the Covariance and Correlation functions": [[13, "introducing-the-covariance-and-correlation-functions"], [24, "introducing-the-covariance-and-correlation-functions"]], "Introduction": [[2, "introduction"], [8, "introduction"], [17, "introduction"], [18, "introduction"]], "Iterative Fitting, Classification and AdaBoost": [[12, "iterative-fitting-classification-and-adaboost"]], "Iterative Fitting, Regression and Squared-error Cost Function": [[12, "iterative-fitting-regression-and-squared-error-cost-function"]], "Kernel PCA": [[13, "kernel-pca"]], "Kernels and non-linearity": [[10, "kernels-and-non-linearity"]], "LU Decomposition, the inverse of a matrix": [[18, "lu-decomposition-the-inverse-of-a-matrix"]], "Layers": [[3, "layers"]], "Layers used to build CNNs": [[5, "layers-used-to-build-cnns"]], "Learning goals": [[0, "learning-goals"], [1, "learning-goals"]], "Learning outcomes": [[17, "learning-outcomes"], [23, "learning-outcomes"]], "Lectures and ComputerLab": [[23, "lectures-and-computerlab"]], "Limitations of supervised learning with deep networks": [[3, "limitations-of-supervised-learning-with-deep-networks"]], "Linear Algebra, Handling of Arrays and more Python Features": [[18, null]], "Linear Regression": [[2, null]], "Linear Regression Problems": [[24, "linear-regression-problems"]], "Linear Regression, basic elements": [[2, "linear-regression-basic-elements"]], "Linking Bayes\u2019 Theorem with Ridge and Lasso Regression": [[7, "linking-bayes-theorem-with-ridge-and-lasso-regression"]], "Linking the regression analysis with a statistical interpretation": [[7, "linking-the-regression-analysis-with-a-statistical-interpretation"]], "Linking with the SVD": [[7, "linking-with-the-svd"], [24, "linking-with-the-svd"]], "Links to relevant courses at the University of Oslo": [[22, "links-to-relevant-courses-at-the-university-of-oslo"]], "Logistic Regression": [[9, null], [9, "id1"]], "MNIST and GANs": [[6, "mnist-and-gans"]], "Machine Learning": [[23, "machine-learning"]], "Machine learning": [[17, "machine-learning"]], "Main textbooks": [[23, "main-textbooks"]], "Making a tree": [[11, "making-a-tree"]], "Making your own Bootstrap: Changing the Level of the Decision Tree": [[12, "making-your-own-bootstrap-changing-the-level-of-the-decision-tree"]], "Making your own test-train splitting": [[24, "making-your-own-test-train-splitting"]], "Mathematical Interpretation of Ordinary Least Squares": [[7, "mathematical-interpretation-of-ordinary-least-squares"], [24, "mathematical-interpretation-of-ordinary-least-squares"]], "Mathematical optimization of convex functions": [[10, "mathematical-optimization-of-convex-functions"]], "Mathematics of CNNs": [[5, "mathematics-of-cnns"]], "Mathematics of the SVD and implications": [[7, "mathematics-of-the-svd-and-implications"], [24, "mathematics-of-the-svd-and-implications"]], "Matrices in Python": [[23, "matrices-in-python"]], "Matrix multiplication": [[3, "matrix-multiplication"]], "Matrix-vector notation and activation": [[14, "matrix-vector-notation-and-activation"]], "Meet the covariance!": [[20, "meet-the-covariance"]], "Meet the Covariance Matrix": [[7, "meet-the-covariance-matrix"], [24, "meet-the-covariance-matrix"]], "Meet the Hessian Matrix": [[24, "meet-the-hessian-matrix"]], "Meet the Pandas": [[23, "meet-the-pandas"]], "Min-Max Scaling": [[24, "min-max-scaling"]], "Momentum based GD": [[15, "momentum-based-gd"]], "More complicated Example: The Ising model": [[8, "more-complicated-example-the-ising-model"]], "More interpretations": [[24, "more-interpretations"]], "More on Dimensionalities": [[5, "more-on-dimensionalities"]], "More on Rescaling data": [[8, "more-on-rescaling-data"]], "More preprocessing": [[24, "more-preprocessing"]], "Multilayer perceptrons": [[14, "multilayer-perceptrons"]], "Network requirements": [[4, "network-requirements"]], "Neural Networks vs CNNs": [[5, "neural-networks-vs-cnns"]], "Neural networks": [[14, null]], "Note about SVD Calculations": [[24, "note-about-svd-calculations"]], "Numerical experiments and the covariance, central limit theorem": [[20, "numerical-experiments-and-the-covariance-central-limit-theorem"]], "Numpy and arrays": [[18, "numpy-and-arrays"], [23, "numpy-and-arrays"]], "Numpy examples and Important Matrix and vector handling packages": [[23, "numpy-examples-and-important-matrix-and-vector-handling-packages"]], "Optimization, the central part of any Machine Learning algortithm": [[15, null]], "Optimizing our parameters": [[23, "optimizing-our-parameters"]], "Optimizing our parameters, more details": [[23, "optimizing-our-parameters-more-details"]], "Optimizing the cost function": [[3, "optimizing-the-cost-function"]], "Organizing our data": [[2, "organizing-our-data"], [23, "organizing-our-data"]], "Other Matrix and Vector Operations": [[18, "other-matrix-and-vector-operations"]], "Other Types of Recurrent Neural Networks": [[6, "other-types-of-recurrent-neural-networks"]], "Other courses on Data science and Machine Learning at UiO": [[23, "other-courses-on-data-science-and-machine-learning-at-uio"]], "Other courses on Data science and Machine Learning at UiO, contn": [[23, "other-courses-on-data-science-and-machine-learning-at-uio-contn"]], "Other popular texts": [[23, "other-popular-texts"]], "Other techniques": [[13, "other-techniques"]], "Other types of networks": [[14, "other-types-of-networks"]], "Other ways of visualizing the trees": [[11, "other-ways-of-visualizing-the-trees"]], "Our model for the nuclear binding energies": [[23, "our-model-for-the-nuclear-binding-energies"]], "Overview of first week": [[23, "overview-of-first-week"]], "Own code for Ordinary Least Squares": [[23, "own-code-for-ordinary-least-squares"], [24, "own-code-for-ordinary-least-squares"]], "PCA and scikit-learn": [[13, "pca-and-scikit-learn"]], "Pandas AI": [[23, "pandas-ai"]], "Partial Differential Equations": [[4, "partial-differential-equations"]], "Plans for week 35": [[24, "plans-for-week-35"]], "Practical tips": [[15, "practical-tips"]], "Practicalities": [[21, "practicalities"], [21, "id1"]], "Predicting New Points With A Trained Recurrent Neural Network": [[6, "predicting-new-points-with-a-trained-recurrent-neural-network"]], "Preprocessing our data": [[24, "preprocessing-our-data"]], "Prerequisites": [[23, "prerequisites"]], "Prerequisites and background": [[17, "prerequisites-and-background"]], "Prerequisites: Collect and pre-process data": [[5, "prerequisites-collect-and-pre-process-data"]], "Probability Distribution Functions": [[20, "probability-distribution-functions"]], "Program for stochastic gradient": [[15, "program-for-stochastic-gradient"]], "Properties of PDFs": [[20, "properties-of-pdfs"]], "Pros and cons of trees, pros": [[11, "pros-and-cons-of-trees-pros"]], "Python installers": [[17, "python-installers"], [23, "python-installers"]], "RMS prop": [[15, "rms-prop"]], "Random Numbers": [[20, "random-numbers"]], "Random forests": [[12, "random-forests"]], "Randomized PCA": [[13, "randomized-pca"]], "Reading material": [[23, "reading-material"]], "Reading recommendations:": [[24, "reading-recommendations"]], "Reading suggestions week 34": [[23, "reading-suggestions-week-34"]], "Recurrent neural networks": [[14, "recurrent-neural-networks"]], "Recurrent neural networks: Overarching view": [[6, null]], "Reducing the number of degrees of freedom, overarching view": [[2, "reducing-the-number-of-degrees-of-freedom-overarching-view"], [24, "reducing-the-number-of-degrees-of-freedom-overarching-view"]], "Reformulating the problem": [[4, "reformulating-the-problem"]], "Regression Case": [[12, "regression-case"]], "Regression analysis, overarching aims": [[23, "regression-analysis-overarching-aims"]], "Regression analysis, overarching aims II": [[23, "regression-analysis-overarching-aims-ii"]], "Regularization": [[3, "regularization"]], "Reminder from last week": [[24, "reminder-from-last-week"]], "Reminder on Statistics": [[8, "reminder-on-statistics"]], "Replace or not": [[15, "replace-or-not"]], "Required Technologies": [[17, "required-technologies"]], "Resampling Methods": [[8, null]], "Resampling methods": [[8, "id1"]], "Residual Error": [[24, "residual-error"]], "Resources on differential equations and deep learning": [[4, "resources-on-differential-equations-and-deep-learning"]], "Revisiting our Linear Regression Solvers": [[15, "revisiting-our-linear-regression-solvers"]], "Rewriting the Covariance and/or Correlation Matrix": [[24, "rewriting-the-covariance-and-or-correlation-matrix"]], "Rewriting the fitting procedure as a linear algebra problem": [[23, "rewriting-the-fitting-procedure-as-a-linear-algebra-problem"]], "Rewriting the fitting procedure as a linear algebra problem, more details": [[23, "rewriting-the-fitting-procedure-as-a-linear-algebra-problem-more-details"]], "Ridge and LASSO Regression": [[24, "ridge-and-lasso-regression"]], "Ridge and Lasso Regression": [[7, null], [7, "id1"]], "Same code but now with momentum gradient descent": [[15, "same-code-but-now-with-momentum-gradient-descent"]], "Schedule first week": [[23, "schedule-first-week"]], "Schematic Regression Procedure": [[11, "schematic-regression-procedure"]], "Setting up the Back propagation algorithm": [[14, "setting-up-the-back-propagation-algorithm"]], "Setting up the Matrix to be inverted": [[24, "setting-up-the-matrix-to-be-inverted"]], "Setting up the network using Autograd; The full program": [[4, "setting-up-the-network-using-autograd-the-full-program"]], "Similar (second order function now) problem but now with AdaGrad": [[15, "similar-second-order-function-now-problem-but-now-with-adagrad"]], "Simple Python Code to read in Data and perform Classification": [[11, "simple-python-code-to-read-in-data-and-perform-classification"]], "Simple case": [[24, "simple-case"]], "Simple linear regression model using scikit-learn": [[2, "simple-linear-regression-model-using-scikit-learn"], [23, "simple-linear-regression-model-using-scikit-learn"]], "Software and needed installations": [[23, "software-and-needed-installations"]], "Solving Differential Equations with Deep Learning": [[4, null]], "Solving the one dimensional Poisson equation": [[4, "solving-the-one-dimensional-poisson-equation"]], "Solving the wave equation with Neural Networks": [[4, "solving-the-wave-equation-with-neural-networks"]], "Some famous Matrices": [[18, "some-famous-matrices"]], "Some simple problems": [[15, "some-simple-problems"]], "Some useful matrix and vector expressions": [[24, "some-useful-matrix-and-vector-expressions"]], "Splitting our Data in Training and Test data": [[2, "splitting-our-data-in-training-and-test-data"], [24, "splitting-our-data-in-training-and-test-data"]], "Standard steepest descent": [[15, "standard-steepest-descent"]], "Statistical analysis and optimization of data": [[17, "statistical-analysis-and-optimization-of-data"], [23, "statistical-analysis-and-optimization-of-data"]], "Steepest descent": [[15, "steepest-descent"]], "Stochastic Gradient Descent (SGD)": [[15, "stochastic-gradient-descent-sgd"]], "Stochastic variables and the main concepts, the discrete case": [[20, "stochastic-variables-and-the-main-concepts-the-discrete-case"]], "Support Vector Machines, overarching aims": [[10, null]], "Systematic reduction": [[5, "systematic-reduction"]], "Teachers": [[23, "teachers"]], "Teachers and Grading": [[21, null]], "Teaching Assistants Fall semester 2023": [[21, "teaching-assistants-fall-semester-2023"]], "Tentative deadllines for projects": [[21, "tentative-deadllines-for-projects"]], "Testing the Means Squared Error as function of Complexity": [[2, "testing-the-means-squared-error-as-function-of-complexity"], [24, "testing-the-means-squared-error-as-function-of-complexity"]], "Textbooks": [[22, null]], "The Algorithm before theorem": [[13, "the-algorithm-before-theorem"]], "The Breast Cancer Data, now with Keras": [[3, "the-breast-cancer-data-now-with-keras"]], "The CART algorithm for Classification": [[11, "the-cart-algorithm-for-classification"]], "The CART algorithm for Regression": [[11, "the-cart-algorithm-for-regression"]], "The CIFAR01 data set": [[5, "the-cifar01-data-set"]], "The Jacobian": [[24, "the-jacobian"]], "The MNIST dataset again": [[5, "the-mnist-dataset-again"]], "The RELU function family": [[3, "the-relu-function-family"]], "The SVD, a Fantastic Algorithm": [[24, "the-svd-a-fantastic-algorithm"]], "The Softmax function": [[3, "the-softmax-function"]], "The \\chi^2 function": [[2, "the-chi-2-function"], [23, "the-chi-2-function"], [23, "id4"], [23, "id5"], [23, "id6"], [23, "id7"], [23, "id8"]], "The bias-variance tradeoff": [[8, "the-bias-variance-tradeoff"]], "The code for solving the ODE": [[4, "the-code-for-solving-the-ode"]], "The complete code with a simple data set": [[24, "the-complete-code-with-a-simple-data-set"]], "The cost/loss function": [[24, "the-cost-loss-function"]], "The course has two central parts": [[17, "the-course-has-two-central-parts"]], "The equations for ordinary least squares": [[24, "the-equations-for-ordinary-least-squares"]], "The logistic function": [[9, "the-logistic-function"]], "The mean squared error and its derivative": [[24, "the-mean-squared-error-and-its-derivative"]], "The moons example": [[10, "the-moons-example"]], "The multilayer perceptron (MLP)": [[14, "the-multilayer-perceptron-mlp"]], "The network with one input layer, specified number of hidden layers, and one output layer": [[4, "the-network-with-one-input-layer-specified-number-of-hidden-layers-and-one-output-layer"]], "The plethora of machine learning algorithms/methods": [[23, "the-plethora-of-machine-learning-algorithms-methods"]], "The singular value decomposition": [[7, "the-singular-value-decomposition"], [24, "the-singular-value-decomposition"]], "The two-dimensional case": [[10, "the-two-dimensional-case"]], "To our real data: nuclear binding energies. Brief reminder on masses and binding energies": [[23, "to-our-real-data-nuclear-binding-energies-brief-reminder-on-masses-and-binding-energies"]], "Topics covered in this course: Statistical analysis and optimization of data": [[23, "topics-covered-in-this-course-statistical-analysis-and-optimization-of-data"]], "Towards the PCA theorem": [[13, "towards-the-pca-theorem"]], "Train and test datasets": [[3, "train-and-test-datasets"]], "Two-dimensional Objects": [[5, "two-dimensional-objects"]], "Type of problem": [[4, "type-of-problem"]], "Types of Machine Learning": [[23, "types-of-machine-learning"]], "Useful Python libraries": [[17, "useful-python-libraries"], [23, "useful-python-libraries"]], "Using Autograd": [[15, "using-autograd"]], "Using forward Euler to solve the ODE": [[4, "using-forward-euler-to-solve-the-ode"]], "Using gradient descent methods, limitations": [[15, "using-gradient-descent-methods-limitations"]], "Visualization": [[3, "visualization"], [3, "id1"]], "Visualizing the Tree, Classification": [[11, "visualizing-the-tree-classification"]], "Week 34: Introduction to the course, Logistics and Practicalities": [[23, null]], "Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression": [[24, null]], "What Is Generative Modeling?": [[23, "what-is-generative-modeling"]], "What does it mean?": [[24, "what-does-it-mean"]], "What is Machine Learning?": [[2, "what-is-machine-learning"]], "What is a good model?": [[2, "what-is-a-good-model"], [23, "what-is-a-good-model"]], "What is a good model? Can we define it?": [[23, "what-is-a-good-model-can-we-define-it"]], "Which activation function should I use?": [[3, "which-activation-function-should-i-use"]], "Why Linear Regression (aka Ordinary Least Squares and family)": [[23, "why-linear-regression-aka-ordinary-least-squares-and-family"]], "Wisconsin Cancer Data": [[9, "wisconsin-cancer-data"]], "Writing Our First Generative Adversarial Network": [[6, "writing-our-first-generative-adversarial-network"]], "Writing our own PCA code": [[13, "writing-our-own-pca-code"]], "XGBoost: Extreme Gradient Boosting": [[12, "xgboost-extreme-gradient-boosting"]], "scikit-learn implementation": [[3, "scikit-learn-implementation"]]}, "docnames": ["E1", "E2", "chapter1", "chapter10", "chapter11", "chapter12", "chapter13", "chapter2", "chapter3", "chapter4", "chapter5", "chapter6", "chapter7", "chapter8", "chapter9", "chapteroptimization", "clustering", "intro", "linalg", "schedule", "statistics", "teachers", "textbooks", "week34", "week35"], "envversion": {"sphinx": 62, "sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.intersphinx": 1}, "filenames": ["E1.ipynb", "E2.ipynb", "chapter1.ipynb", "chapter10.ipynb", "chapter11.ipynb", "chapter12.ipynb", "chapter13.ipynb", "chapter2.ipynb", "chapter3.ipynb", "chapter4.ipynb", "chapter5.ipynb", "chapter6.ipynb", "chapter7.ipynb", "chapter8.ipynb", "chapter9.ipynb", "chapteroptimization.ipynb", "clustering.ipynb", "intro.md", "linalg.ipynb", "schedule.md", "statistics.ipynb", "teachers.md", "textbooks.md", "week34.ipynb", "week35.ipynb"], "indexentries": {}, "objects": {}, "objnames": {}, "objtypes": {}, "terms": {"": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 13, 14, 15, 17, 18, 20, 21, 23, 24], "0": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 21, 23, 24], "00": [2, 3, 7, 13, 23, 24], "000": [3, 5], "00000000e": [], "001": [4, 10, 15], "004": 7, "00727646693": [2, 23], "0086649156": [2, 23], "01": [2, 3, 4, 7, 11, 13, 15, 22, 23, 24], "0110": 20, "01719003e": [], "02": [2, 6, 9, 14, 23], "02334824": [], "02857": 6, "02f": 8, "03077640549": 6, "03097597e": [], "031": 7, "04": 13, "0458": 11, "05": [6, 8], "062292565": 6, "062435": [], "06730814": [], "07": [], "0713": [2, 23], "07285": 5, "08": 20, "08078025e": [], "08336233266": 6, "0917": 11, "0n": [2, 23], "1": [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23], "10": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 19, 20, 21, 23, 24], "100": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 16, 18, 20, 21, 23, 24], "1000": [2, 3, 4, 6, 7, 10, 13, 15, 16, 17, 20, 23], "10000": [4, 7, 8, 12, 13, 15, 20], "100000": 10, "10001": 12, "1001": 20, "1002": 20, "1003": 20, "1005": 20, "1009": 20, "101": 1, "1011": 20, "1013": 20, "1013904243": 20, "1015": 20, "102": 1, "1023": 20, "1024": 5, "1026": 20, "1027": 20, "103": 3, "1030": 20, "1037": 20, "1038": 20, "1040": 20, "1047": 20, "107": 1, "108": [], "10th": 11, "10x": [2, 23], "11": [1, 2, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 20, 22, 23, 24], "110": [], "1100": 20, "1101": 20, "111": [3, 9, 14], "112": 1, "11340253": [], "11590451": [], "116": 1, "117": 1, "118": 1, "12": [2, 3, 4, 5, 6, 7, 8, 10, 11, 13, 14, 18, 20, 22, 23, 24], "120": 5, "121": [1, 10, 11, 12], "1215pm": [21, 23], "122": [10, 11, 12], "124": [2, 23], "125": 1, "127": [1, 6], "128": [5, 6, 15], "129": 1, "1298": 11, "12pm": [21, 23], "13": [2, 4, 11, 14, 18, 20, 23], "131": 1, "133": 9, "135": 1, "136": 1, "14": [2, 4, 6, 8, 10, 11, 12, 14, 18, 20, 22, 24], "141": 1, "143": 1, "1446729567": 6, "149": 1, "14g": 8, "15": [2, 4, 6, 8, 9, 10, 11, 14, 15, 20, 23], "150": [6, 10], "152": 1, "153760": [], "156": 1, "157": [], "158": [], "159": 1, "15g": 8, "15pm": 23, "16": [3, 4, 5, 6, 7, 10, 11, 12, 20, 23], "160": 1, "1603": 5, "161": 1, "162": 1, "16231451": 6, "163": 1, "16384": 5, "164": 1, "167": 1, "17": [3, 4, 10, 20], "172": 1, "173": 1, "176": 1, "178": 1, "179": 1, "1797": 3, "18": [4, 8, 9, 10, 11, 12, 20, 23], "1807": 6, "18392847": [], "19": [4, 20, 23], "1940": [], "1943": 14, "1970": [18, 23], "1973": 11, "1979": 8, "1_1": 14, "1_2": 14, "1_3": 14, "1cm": [2, 10, 12, 20, 23], "1d": [3, 4, 5], "1e": [4, 6, 15, 16], "1e10": 16, "1e4": 8, "1f": 3, "1k": 18, "1n": [2, 23], "1x": [2, 23], "2": [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22], "20": [1, 2, 3, 4, 8, 9, 10, 20, 21, 23, 24], "200": [2, 4, 5, 6, 10, 11, 12], "2000": [2, 24], "2004": 15, "2006": 22, "20072279": [], "2008": 23, "2010": 3, "2011": 3, "2014": 6, "2015": 3, "2016": [2, 23], "2018": [2, 8, 24], "2021": [8, 16], "2022": 23, "2025": [23, 24], "21": [2, 3, 7, 9, 11, 14, 18, 23, 24], "2116753732": 6, "215pm": [21, 23], "2167072": [], "22": [2, 3, 7, 14, 15, 18, 23, 24], "221": 10, "225": 6, "22948497": [], "23": [3, 14, 18], "24": [2, 3, 18, 23], "25": [4, 5, 6, 7, 8, 10, 11, 13, 24], "250": [4, 6, 9, 11], "25000": [], "250154": [], "253775": [], "255": 5, "256": 6, "26": [], "26303845": [], "264": [], "265": [], "265109911": 6, "266": [], "269": [], "27": 3, "270": [], "27n_": 20, "28": [3, 5, 6], "2830637392": 6, "2861": 20, "2873": 11, "2882": 20, "2886": 20, "2890": [2, 23], "2892": 20, "29": 24, "2915": 20, "2931": 23, "29364655": [], "294399745619595": [], "296247": [], "2968": 23, "2980": 23, "298273": [], "298375": [], "2990": 23, "2_": 14, "2_1": 14, "2_2": 14, "2_3": 14, "2_i": 14, "2_m": [8, 20], "2_t": 15, "2_x": 20, "2b": 20, "2cm": 10, "2d": [3, 5, 13, 14, 17, 23], "2e": 8, "2f": [2, 9, 11, 12, 13, 14, 23], "2g": 4, "2g_i": 4, "2k": 5, "2m": 8, "2n": [2, 4, 5, 23, 24], "2nd": 11, "2p": 20, "2pt": 6, "2x": [2, 5, 10, 15, 23], "2x_ix_jy_iy_j": 10, "2x_j": 10, "2y_i": 12, "2y_j": 10, "3": [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23], "30": [2, 3, 6, 8, 9, 12, 15, 21], "30000": [2, 23], "3072": 5, "31": [14, 18, 20], "315": 8, "3155": [2, 7, 8, 24], "32": [5, 6, 8, 14, 15, 18, 20], "3200": 3, "3250": 3, "3297": [], "33": [14, 18, 21], "3303": [], "3310": [], "332331": [], "333": 9, "3331": [], "3337": [], "34": 18, "3436": [2, 23], "3437": [2, 23], "35": [2, 8, 23], "3581341341": 6, "359": 7, "36": [2, 7, 8, 20], "370782966": 6, "38": 20, "39": [2, 21, 23], "3d": [1, 4, 5, 6, 8, 15], "3f": [3, 5, 11], "3n": 18, "3x": [4, 10], "3x_i": 4, "3y": 10, "4": [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23], "40": [3, 8, 21, 23], "400": 6, "4000": 23, "4050": [22, 23], "41": 18, "4155": [0, 4], "41589548": [], "42": [3, 6, 10, 11, 12, 18], "43": [2, 9, 18], "4310": 23, "436462435": 6, "44": [2, 18], "45": [21, 23], "46": [21, 23], "462": 9, "47": [21, 23], "479465113": 6, "47958494": [], "48": [], "48257387": [21, 23], "49": [7, 8, 13], "49152": 5, "4940954": [2, 23], "4990": 20, "4992": 20, "4997": 20, "4c4c7f": [11, 12], "4d": 5, "4f": 8, "4pm": [21, 23], "4y": 10, "4y_i": 12, "5": [0, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "50": [3, 4, 5, 6, 8, 9, 10, 12, 15, 23, 24], "500": [3, 5, 6, 8, 11, 12, 15], "5018": 20, "506": [], "507d50": [11, 12], "50j": 15, "50x10": 3, "51": 12, "510": 3, "512132": [], "5177783846": 6, "53": 11, "54": [8, 20], "5411205": [], "54894451": [], "55": 3, "56": 3, "56536": [2, 23], "569": 3, "57": [2, 10, 21, 23], "571": 7, "58": [12, 21, 23], "591317992": 6, "5cm": 20, "5f": 10, "5x": 10, "5y": 10, "6": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 18, 20, 21, 23, 24], "60": [3, 5], "60000": 6, "6019067271": 6, "606439": [], "625": 9, "63": 3, "64": [3, 5, 6, 15, 18, 23], "64x50": 3, "65": [3, 10, 11], "6887363571": 6, "69": [1, 20], "69069n_": 20, "691": [], "6n_": 20, "7": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 18, 20, 22, 23], "70": [3, 9], "70653767": 6, "71": 3, "724": 5, "73": [], "7304881": [], "75": [7, 8, 10, 13], "76": [21, 23], "765": 9, "77": [21, 23], "7718": 11, "7782028952": 6, "77893972": [], "78": [], "7d7d58": [11, 12], "8": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 18, 20, 21, 23], "80": [2, 3, 7, 10, 24], "800": [6, 9], "81": 3, "815am": [21, 23], "85": 3, "8702784034": 6, "88": 23, "8f": 8, "8g": 8, "8n": 18, "8x8": 3, "9": [2, 3, 4, 6, 7, 8, 9, 10, 11, 13, 14, 15, 18, 20, 23], "90": 3, "9040": 11, "91": [21, 23], "92": [21, 23], "93": 1, "931": [2, 23], "933": 7, "937": 20, "938": 20, "939": [2, 20, 23], "94": 20, "95": [3, 13], "954": 20, "955820c21e8b": 6, "96": 8, "960": 20, "961": 20, "962": 20, "9649652536": 6, "96611194e": [], "9780387310732": 22, "9780387848570": 22, "9781098134174": 23, "9781492032632": 22, "9781801819312": 23, "98": [1, 2, 3], "985": 20, "986": 20, "989": 20, "9898ff": [11, 12], "99": [1, 15], "991": 20, "992": 20, "993": 20, "996": 7, "999": [11, 20], "9x": 8, "9y": 8, "A": [0, 1, 4, 5, 7, 8, 9, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 24], "AND": 4, "And": [2, 5, 6, 7, 8, 11, 15, 17, 20], "As": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 14, 15, 18, 20, 23, 24], "At": [2, 6, 8, 15, 23], "BE": [2, 23], "Be": [4, 17, 23], "Being": 15, "But": [1, 2, 3, 4, 5, 7, 8, 11, 12, 20, 24], "By": [2, 5, 7, 8, 14, 15, 18, 23, 24], "For": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 23, 24], "IF": 8, "IN": 22, "If": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 20, 23, 24], "In": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 23, 24], "Ising": [7, 14, 24], "It": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 23, 24], "Its": [3, 4, 6, 13], "No": [8, 11, 23], "Not": [2, 3, 7, 8, 24], "OR": 20, "Of": 20, "On": [2, 5, 20, 21, 22, 23], "One": [2, 3, 5, 6, 7, 8, 9, 10, 13, 14, 15, 20, 24], "Or": [2, 3, 8, 23], "Such": [1, 2, 8, 14, 20], "That": [2, 7, 9, 12, 13, 14, 16, 20, 23], "The": [1, 6, 12, 15, 16, 18, 19, 20, 21, 22], "Then": [0, 1, 2, 3, 8, 10, 11, 12, 13, 14, 15, 16, 18, 23], "There": [0, 2, 5, 6, 7, 8, 10, 11, 13, 14, 16, 18, 20, 21, 23, 24], "These": [2, 5, 6, 7, 10, 11, 12, 13, 14, 15, 16, 18, 20, 21, 23, 24], "To": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 20, 24], "With": [1, 2, 7, 8, 10, 11, 12, 13, 14, 16, 18, 20, 23, 24], "_": [1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 23, 24], "_0": [7, 10, 12, 13, 15, 24], "_1": [4, 7, 8, 10, 12, 13, 14, 15, 16, 18, 24], "_2": [4, 7, 10, 13, 14, 15, 18, 24], "_3": 18, "_4": 18, "_9": 15, "__class__": 12, "__doc__": 8, "__future__": [10, 11], "__init__": 3, "__main__": 4, "__name__": [4, 12], "_auto1": [4, 5, 6, 7, 8, 9, 14, 15, 18, 20, 24], "_auto10": [8, 14], "_auto11": 8, "_auto12": 8, "_auto2": [4, 5, 6, 7, 8, 14, 15, 18, 20], "_auto3": [5, 6, 7, 8, 14, 15, 18], "_auto4": [6, 8, 14, 15, 18], "_auto5": [6, 8, 14, 15, 18], "_auto6": [6, 8, 14, 18], "_auto7": [6, 8, 14, 18], "_auto8": [8, 14], "_auto9": [8, 14], "_build": [2, 17, 22, 23], "_c": 3, "_compon": 13, "_depth": 11, "_export": [0, 1], "_fraction": 11, "_i": [2, 3, 4, 7, 8, 9, 10, 13, 14, 15, 23, 24], "_j": [2, 3, 4, 5, 7, 8, 10, 15, 24], "_k": 15, "_l": 14, "_lambda": 8, "_leaf": 11, "_m": 12, "_multilayer_perceptron": [], "_n": [4, 7, 10, 13, 15, 24], "_node": 11, "_p": [7, 10, 24], "_ratio": 13, "_sampl": 11, "_split": [8, 11], "_t": 15, "_test": 8, "_varianc": 13, "_weight": 11, "a0": 5, "a0faa0": [11, 12], "a1": [2, 23], "a2": [2, 23], "a3": [2, 23], "a4": [2, 23], "a_": [1, 2, 3, 18, 23, 24], "a_0": [2, 23], "a_1a": [2, 23], "a_2a": [2, 23], "a_3": [2, 23], "a_3a": [2, 23], "a_4": [2, 23], "a_4a": [2, 23], "a_h": 3, "a_i": [2, 3, 4, 14, 23], "a_j": [3, 14], "a_k": [2, 3, 14], "aaron": 22, "ab": [2, 4, 7, 15, 16, 23, 24], "ab_channel": 17, "abandon": 3, "abid": 20, "abil": [2, 12], "abl": [1, 2, 3, 6, 7, 8, 9, 12, 14, 15, 24], "about": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 21], "abov": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 22, 23, 24], "abovement": [8, 23], "abscissa": 15, "absolut": [2, 4, 7, 8, 15, 23, 24], "absorb": 24, "abstract": 3, "acceler": 15, "accept": [2, 5, 8, 11, 24], "access": [5, 13, 20, 23], "accid": [6, 8], "accompani": [2, 23, 24], "accomplish": [10, 11, 15], "accord": [2, 3, 4, 7, 8, 11, 14, 15, 16, 20, 23], "accordingli": 13, "account": [0, 1, 2, 5, 7, 15, 20, 23], "accumul": [14, 15, 20], "accur": [2, 5, 6, 8, 12, 15], "accuraci": [2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 14, 23, 24], "accuracy_scor": [2, 3, 12, 23], "accuracy_score_numpi": 3, "achiev": [2, 3, 7, 8, 10, 14, 18, 23], "aco": 20, "acquaint": 17, "acquir": [3, 17, 23], "acr": [], "across": [3, 5, 8, 11, 17, 23], "act": [3, 5, 18], "action": 20, "activ": [0, 2, 4, 5, 6, 11, 19, 21, 23], "actual": [0, 1, 2, 3, 6, 7, 8, 10, 13, 18, 20, 23, 24], "ad": [0, 1, 3, 5, 6, 7, 10, 15, 18], "ada_clf": 12, "adaboostclassifi": 12, "adadelta": 15, "adam": [3, 5, 6, 23], "adapt": [6, 8, 15, 22], "add": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14, 20, 21, 23, 24], "add_subplot": [3, 9, 14, 16], "addendum": 7, "addit": [0, 2, 4, 5, 7, 8, 9, 10, 11, 12, 14, 15, 17, 18, 20, 21, 22, 23, 24], "addition": [14, 15], "address": [3, 11, 13, 15, 23], "adjac": [5, 14], "adjoint": [7, 24], "adjust": [2, 7, 14, 15], "admir": [2, 23], "advanc": [6, 8, 14, 22, 23], "advantag": [3, 5, 7, 8, 12, 15, 18], "adversari": 23, "afecionado": 23, "affect": [0, 5], "affin": [2, 5, 10, 13, 24], "afford": 5, "aficionado": 23, "aforement": 16, "african": [], "after": [0, 1, 2, 3, 4, 6, 7, 8, 11, 13, 14, 15, 17, 18, 20, 23, 24], "afterward": [2, 23], "ag": [2, 9, 23, 24], "ag_0": 4, "again": [2, 3, 6, 7, 8, 9, 10, 12, 13, 14, 15, 20, 23, 24], "against": [3, 6, 9, 12], "agegroup": 9, "agegroupmean": 9, "aggreg": [11, 12], "agorithm": 12, "agre": [7, 8, 20, 24], "agreement": 15, "ahead": 11, "ai": [2, 22], "aid": 13, "aim": [1, 2, 3, 6, 8, 9, 13, 16, 17, 18, 24], "ainv": 7, "airplan": 5, "aka": 7, "al": [1, 2, 4, 6, 22, 23, 24], "alarm": [7, 9], "aldo": 24, "algebra": [2, 5, 7, 15, 17, 24], "algorithm": [1, 2, 3, 4, 6, 7, 8, 9, 10, 15, 16, 17, 18, 20, 22], "align": [2, 4, 7, 8, 9, 10, 15, 20, 23, 24], "all": [0, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24], "allevi": [3, 15], "alloc": [5, 18], "allow": [0, 2, 3, 4, 5, 7, 8, 10, 12, 15, 17, 18, 23, 24], "almost": [2, 3, 8, 10, 13, 15, 20], "alon": [4, 11], "along": [0, 4, 5, 6, 7, 8, 11, 12, 13, 17, 18, 23, 24], "alpha": [2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 15, 16, 20, 23, 24], "alpha_": 12, "alpha_0": 5, "alpha_1": 5, "alpha_2": 5, "alpha_i": [5, 15], "alpha_k": 15, "alpha_m": 12, "alpha_n": 5, "alpha_opt": 15, "alreadi": [0, 4, 5, 6, 7, 8, 12, 14, 17, 18, 20, 23, 24], "also": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 23, 24], "alter": 3, "altern": [0, 2, 3, 6, 7, 8, 10, 11, 13, 15, 18, 23, 24], "although": [1, 2, 3, 7, 8, 10, 12, 15, 23], "alwai": [1, 2, 5, 7, 8, 14, 15, 20, 23, 24], "am": 6, "ame2016": [2, 23], "american": [], "among": [2, 5, 7, 11, 12, 14, 18, 23, 24], "amongst": 7, "amount": [2, 3, 5, 6, 8, 10, 12, 16, 17], "an": [1, 3, 4, 5, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 20, 21, 22, 24], "an_": 20, "anaconda": [2, 3, 17, 23], "analogi": 15, "analys": 8, "analysi": [3, 5, 6, 9, 16, 18, 22], "analyt": [4, 5, 7, 8, 9, 14, 15, 17, 23, 24], "analyz": [1, 2, 3, 5, 6, 7, 8, 20, 24], "andrew": 3, "angl": [2, 5, 11, 24], "anharmon": 5, "ani": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 16, 20, 23, 24], "anim": [6, 14], "ann": 14, "annot": [2, 3, 5, 9, 10, 23], "announc": 23, "anoth": [0, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 18, 20, 23, 24], "ansatz": [2, 23], "answer": [2, 3, 5, 7, 8, 18, 21, 23], "antialias": [4, 8], "anticip": 6, "anymor": [3, 10], "anyon": [0, 6, 10], "anyth": [0, 1, 3, 20], "anytim": [21, 23], "apach": 3, "apart": [13, 15], "api": [3, 17, 23], "appar": 4, "appear": [2, 3, 5, 15, 18, 20], "append": [3, 5, 6, 10, 11, 15, 23], "appli": [2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 20, 22, 23, 24], "applic": [1, 2, 3, 5, 6, 7, 8, 9, 11, 14, 15, 18, 20, 22, 23, 24], "apply_gradi": 6, "approach": [0, 1, 3, 4, 6, 7, 8, 11, 12, 13, 14, 15, 17, 20, 22, 24], "appropri": [4, 8, 11, 14, 15, 17, 20], "approv": 23, "approx": [2, 4, 5, 8, 12, 13, 15, 20, 23], "approxim": [2, 3, 4, 5, 6, 7, 8, 9, 12, 13, 15, 20, 23, 24], "apt": [2, 17, 23], "aq": 20, "ar": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24], "aragorn": 23, "arang": [3, 5, 6, 8, 9, 11, 12, 14, 15, 23], "arbitrari": [3, 6, 8, 10, 14, 15, 20], "arbitrarili": [2, 3, 13, 23], "arc": 8, "architectur": [5, 6, 14], "area": [2, 5, 8, 22, 23], "argmax": [3, 13], "argmin": [6, 12, 16], "argsort": 13, "argu": [3, 15], "argument": [2, 4, 5, 7, 13, 14, 15, 23, 24], "aris": [2, 8, 14, 15, 20, 23], "arithmet": [2, 15, 18, 23], "arm": 8, "armadillo": 18, "around": [2, 3, 6, 7, 8, 13, 20, 23], "arrai": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 14, 15, 16, 17, 20, 24], "arrang": [5, 23], "arraybox": 15, "arriv": [2, 8, 11, 13, 18, 20, 23], "arrow": 14, "arrowprop": 10, "art": [2, 3, 17], "articl": [2, 5, 6, 8, 12, 23, 24], "artifici": [2, 4, 9, 14, 22, 23], "artificialneuron": 14, "arug": 15, "arxiv": [5, 6], "asarrai": [2, 8, 11], "asid": 24, "ask": [0, 7, 8, 13, 14], "aspect": [2, 8, 17, 23], "assembl": 5, "assembli": [2, 23], "assert": 6, "assess": [2, 8, 23], "assici": 6, "assign": [0, 2, 9, 10, 11, 14, 15, 16, 19, 21, 22, 23], "associ": [2, 8, 11, 14, 16, 20, 23], "assum": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "assumpt": [2, 5, 7, 8, 11, 13, 20, 23], "ast": [2, 7, 8, 23], "astyp": [6, 11, 12], "asymmetri": [2, 23], "asymptot": [6, 8], "atom": [2, 23], "attempt": [2, 6, 8, 9, 10, 12, 23], "attend": 23, "attent": [2, 18, 23], "attract": [2, 12, 23], "attribut": [2, 11, 23], "audi": [2, 23], "audio": [5, 6], "august": [23, 24], "aurelien": [2, 22, 23], "austfjel": 8, "auth": 0, "authent": 0, "author": [2, 3, 12, 20], "authour": 23, "auto": [11, 12, 20], "autocor": 20, "autocorrelation_tim": 20, "autocorrelform": 20, "autocovari": 20, "autoencod": [6, 17, 23], "autoencond": 17, "autograd": [17, 23], "autom": [2, 17, 22, 23], "automac": 18, "automag": 23, "automat": [1, 2, 3, 4, 5, 6, 13, 17, 18, 23], "automobil": 5, "autonom": 6, "avail": [2, 3, 6, 8, 12, 13, 17, 18, 19, 21, 22, 23], "averag": [2, 3, 5, 8, 11, 12, 15, 16, 20, 21, 23], "avoid": [2, 6, 7, 8, 11, 13, 15, 18, 24], "awai": [4, 5, 8, 24], "awar": [4, 12], "award": [21, 23], "ax": [2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 23], "axes3d": [4, 8, 15], "axes_grid1": 8, "axhlin": 10, "axi": [2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23], "axiom": 7, "axvlin": [6, 10], "axvspan": 6, "b": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 14, 15, 16, 20, 21, 23, 24], "b1": 10, "b2": 10, "b3": 10, "b_": [2, 3, 18], "b_0": 2, "b_1": [2, 4, 14, 15], "b_2": [2, 15], "b_5": 15, "b_group": 11, "b_i": [2, 3, 4, 14, 23], "b_ia_": [2, 23], "b_ia_i": 2, "b_index": 11, "b_j": [3, 14], "b_k": [2, 3, 14, 15], "b_m": 14, "b_score": 11, "b_valu": 11, "babcock": 23, "bachelor": [19, 21], "back": [0, 1, 2, 5, 6, 7, 8, 10, 11, 12, 18, 20, 23], "backbon": 18, "backend": [3, 6], "background": [22, 23], "backpropag": 3, "backtrack": 11, "backup": 18, "backward": [3, 4, 6, 14, 18], "bad": 8, "badli": 20, "bag": [11, 17, 23], "bag_clf": 12, "baggin": 23, "baggingboot": 12, "baggingclassifi": 12, "baggingtre": 12, "balanc": 8, "band": 18, "bandwidth": 18, "bar": [2, 8, 13, 23], "barber": 22, "bare": [6, 12], "base": [0, 1, 2, 3, 5, 6, 7, 9, 10, 11, 12, 16, 17, 20, 21, 22, 23, 24], "basi": [7, 9, 10, 12, 13, 14, 15, 18, 24], "basic": [0, 8, 10, 14, 15, 16, 17, 20, 23], "batch": [5, 6, 13, 14, 15], "batch_shap": 6, "batch_siz": [3, 5, 6], "batchnorm": 6, "bay": 9, "bayesian": [7, 17, 22, 23], "becaus": [2, 3, 4, 5, 6, 7, 8, 10, 11, 14, 15, 16, 23], "becom": [2, 3, 4, 7, 8, 9, 11, 14, 15, 20, 23, 24], "been": [2, 3, 4, 5, 6, 7, 8, 13, 14, 15, 17, 18, 23], "befor": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 14, 15, 16, 18, 20, 23, 24], "beforehand": [2, 20, 23], "begin": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 18, 20, 21, 23, 24], "behav": [3, 8, 15], "behavior": [2, 3, 15, 23], "behaviour": 14, "behind": [2, 3, 8, 10, 15, 23], "being": [2, 3, 4, 5, 6, 7, 9, 10, 12, 13, 14, 15, 20, 23, 24], "believ": [11, 18], "belong": [9, 10, 11, 15, 16], "below": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 20, 23, 24], "benchmark": 12, "benefici": [3, 15], "benefit": [2, 3, 6, 13, 15, 17, 23], "bengio": [3, 22, 23, 24], "benign": [3, 9], "besid": [6, 7], "bessel": [7, 24], "best": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 21, 23, 24], "beta": [1, 3, 5, 12, 13, 15, 23], "beta_": [5, 15], "beta_0": [3, 5, 15], "beta_1": [3, 5, 12, 15], "beta_1x_i": 15, "beta_2": [5, 15], "beta_3": 5, "beta_i": 5, "beta_j": 15, "beta_k": 15, "beta_linreg": 15, "beta_m": 12, "beta_mg_m": 12, "beta_n": 5, "better": [2, 3, 4, 5, 6, 8, 11, 12, 13, 14, 15, 23, 24], "between": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 20, 23, 24], "beyond": [2, 3, 7, 8, 10, 15, 23, 24], "bf": [15, 16, 18, 20], "bg": 23, "bgd": 15, "bia": [2, 3, 4, 5, 7, 10, 11, 12, 14, 15, 23, 24], "bias": [3, 4, 5, 7, 8, 11, 14], "big": [2, 3, 4, 7, 8, 16], "bigger": [3, 8], "bigr": 14, "bike": 11, "bilbo": 23, "billion": [5, 14, 17], "bin": [9, 20], "binari": [2, 5, 7, 9, 11, 12, 14, 23], "binarycrossentropi": 6, "bind": 2, "binomi": [17, 20, 23], "binsboot": 8, "bioinformat": 2, "biolog": [3, 14], "bios1100": [17, 23], "bird": [2, 5], "birth": 23, "bishop": [22, 23], "bit": [3, 6, 18, 20, 23], "bitwis": 20, "bivari": 4, "bk": 15, "bla": [18, 23], "black": [10, 11, 16], "block": [8, 12, 17, 18, 20, 23], "blog": 23, "blogpost": 6, "blue": [2, 5], "bmatrix": [2, 3, 5, 7, 9, 10, 13, 15, 18, 23, 24], "bmi": 3, "bodi": [2, 3, 6, 14], "bold": 3, "boldfac": [1, 2, 7, 24], "boldsymbol": [1, 2, 3, 4, 5, 7, 8, 9, 10, 12, 13, 15, 16, 23], "boltzmann": [14, 17, 23], "book": [22, 23, 24], "book1": 22, "boolean": 6, "boost": [3, 11, 17, 23], "boostrap": 12, "bootstrap": [3, 15, 17, 23], "borrow": 23, "boston_dataset": [], "bot": 10, "both": [0, 1, 2, 3, 6, 7, 8, 10, 11, 12, 15, 16, 17, 18, 20, 21, 23, 24], "bottl": 9, "bound": [10, 14], "boundari": [4, 6, 10, 13, 14], "box": [6, 11], "boyd": [10, 15], "bracket": [6, 20], "brain": [3, 9, 14], "branch": [11, 23], "break": [2, 6, 8, 13, 16, 23], "breast": [7, 9, 13], "breviti": 15, "brew": [2, 17, 23], "brg": 10, "brief": 24, "briefli": [1, 2, 23], "bring": [2, 7, 8, 12], "britt": [21, 23], "broad": 2, "broadli": 23, "brought": [15, 17, 23], "brownle": 6, "browser": [0, 23], "brute": [5, 7, 13, 24], "buffer_s": 6, "bui": 6, "build": [1, 2, 6, 7, 8, 12, 18, 20, 23], "built": [3, 5, 6, 8], "bunch": 13, "busi": [], "byte": [18, 23], "c": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24], "c1": [10, 13], "c2": [10, 13], "c_": [10, 11, 12, 15, 20], "c_0": 20, "c_1": 14, "c_2": 14, "c_3": 14, "c_4": 14, "c_i": [14, 15], "c_k": 20, "ca": [3, 23], "cach": 12, "cal": [2, 10, 12, 14, 15], "calcul": [1, 2, 3, 4, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23], "call": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 23, 24], "calor": [2, 24], "cambridg": [15, 22], "can": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 24], "cancel": [2, 15, 23, 24], "cancer": [7, 12], "cancerpd": 9, "candid": [10, 11, 12], "cannot": [2, 3, 6, 7, 8, 9, 10, 11, 20, 24], "canopi": [2, 17, 23], "canva": [0, 1, 23], "cap": 7, "capabl": [2, 3, 10, 15, 17, 23], "capac": [4, 21], "capita": [], "captur": [6, 13, 14, 23], "car": [5, 6], "card": [2, 9, 23], "cardin": 3, "care": [0, 13], "carefulli": 15, "carlo": [2, 8, 17, 20, 22, 23], "carri": [4, 8, 9], "cart": 12, "case": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 13, 14, 15, 16, 17, 18, 23], "casella": 22, "cast": 3, "cat": [5, 6], "catch": 2, "categor": [2, 3, 5, 11, 13, 23], "categori": [2, 3, 5, 9, 12, 14, 16, 23], "categorical_crossentropi": [3, 5], "caus": [2, 7, 8, 20, 23, 24], "causal": 2, "causat": [2, 23], "cax": 3, "cb": [8, 23], "cbar": 3, "cc": [2, 3, 7, 15, 23, 24], "ccc": [7, 14], "cdf": 20, "cdot": [2, 4, 8, 14, 15, 16, 18, 20, 23], "celebr": 15, "cell": 6, "center": [2, 3, 8, 9, 10, 11, 13, 16, 20, 23], "central": [1, 2, 5, 7, 8, 10, 18, 23, 24], "centroid": [16, 20], "centroid_differ": 16, "centuri": 5, "certain": [2, 5, 8, 9, 11, 20, 23, 24], "cg": 15, "cha": [], "chain": [2, 3, 15, 17, 20, 23], "challeng": 0, "chanc": [3, 7, 15, 20], "chang": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 13, 14, 15, 16, 18, 20, 23, 24], "channel": 5, "chapter": [1, 2, 8, 12, 13, 18, 22, 23, 24], "chapter3": 2, "charact": [2, 5, 7, 23, 24], "character": [10, 11, 12, 14, 20], "characterist": [2, 3, 5, 12, 15, 23], "charg": [2, 23], "charl": [], "chase": 6, "chatgpt": 0, "chd": 9, "chddata": 9, "cheap": [7, 24], "cheaper": [3, 15], "check": [0, 1, 3, 5, 6, 7, 13, 15, 18, 23], "checkmark": 5, "checkpoint": 6, "checkpoint_dir": 6, "checkpoint_prefix": 6, "chen": 12, "cheng": 24, "chiaramont": 4, "childcar": 1, "children": 1, "choic": [2, 3, 4, 5, 6, 8, 11, 14, 15, 16, 18, 23, 24], "choleski": [7, 18, 24], "choos": [0, 4, 5, 8, 11, 12, 13, 15, 16], "chosen": [1, 2, 3, 4, 8, 10, 11, 12, 15, 20, 23], "chosen_datapoint": 3, "christian": 22, "christoph": [22, 23], "cifar": 5, "cifar10": 5, "circ": [3, 14], "circl": [2, 10, 14, 24], "circuit": 5, "circumfer": 11, "circumv": [3, 7, 15, 24], "ckpt": 6, "clariti": 20, "class": [2, 3, 5, 6, 8, 9, 10, 11, 13, 14, 15, 20, 23], "class_nam": [5, 11], "class_val": 11, "class_valu": 11, "classic": [9, 11, 15], "classif": [2, 5, 7, 8, 9, 10, 13, 14, 17, 22, 23, 24], "classifi": [2, 3, 6, 9, 11, 12, 13, 23], "classificaton": 3, "classifii": 12, "clean": 3, "clear": [3, 7, 12, 14, 15], "clearli": [2, 5, 7, 8, 9, 10, 20], "clever": [3, 12], "clf": [2, 8, 10, 11, 12, 23, 24], "clf3": 2, "clf_lasso": 8, "clf_ridg": 8, "cli": 0, "clip": [5, 20], "clone": [0, 21], "close": [2, 3, 4, 6, 8, 10, 11, 13, 14, 15, 16, 20, 22, 23], "closer": [5, 7, 15, 24], "closest": [10, 13, 15, 16], "closur": [17, 23], "cloud": [17, 23], "cluster": [2, 3, 6, 8, 13, 17, 23], "cluster_label": 16, "cm": [3, 4, 5, 8, 10, 15], "cmap": [2, 3, 4, 5, 6, 8, 10, 11, 12, 23], "cmap_arg": 8, "cmd": [0, 11], "cn_": 20, "cnn": 14, "cnn_kera": 5, "cntk": [17, 23], "co": [2, 4, 5, 8, 11, 15, 23], "code": [2, 5, 6, 8, 9, 10, 17, 18, 20, 22], "coef": [2, 23], "coef0": 10, "coef_": [1, 2, 7, 8, 10, 11, 15, 23, 24], "coeff": 7, "coeffici": [2, 5, 7, 8, 9, 10, 11, 15, 18, 23], "coerc": [2, 8, 23], "coin": [12, 20], "coin_toss": 12, "col": [2, 13, 23, 24], "colab": [17, 23], "cold": 11, "colinear": [], "collaps": 10, "collect": [4, 8, 12, 13, 17, 20, 22, 23], "collinear": [7, 24], "color": [2, 5, 6, 8, 10, 11, 12, 20], "color_channel": 5, "color_cod": 8, "colorbar": [3, 8], "colsample_bytre": 12, "colsaobject": 12, "column": [1, 2, 3, 4, 7, 8, 9, 10, 11, 13, 14, 18, 23, 24], "columntransform": 11, "com": [0, 1, 6, 8, 17, 22, 23], "combin": [0, 3, 4, 7, 8, 9, 12, 20], "come": [0, 2, 3, 5, 6, 7, 14, 15, 16, 23, 24], "command": [0, 2, 3], "comment": [2, 6, 7, 8], "commerci": [2, 17, 23], "commit": 0, "commod": [2, 23], "common": [1, 2, 3, 5, 7, 8, 9, 11, 13, 15, 16, 20, 23, 24], "commonli": [2, 3, 6, 8, 9, 11, 15, 16, 24], "commun": [0, 2, 14], "commut": 5, "commutatitav": 5, "compact": [2, 3, 5, 7, 8, 9, 11, 13, 14, 15, 16, 23, 24], "compair": 2, "compar": [2, 5, 6, 7, 8, 13, 15, 18, 23, 24], "comparison": [4, 6, 15], "compat": 9, "compet": 2, "competit": 12, "compil": [2, 3, 5, 6, 15, 17, 18, 23], "complet": [0, 1, 2, 4, 5, 6, 11, 14, 23], "completenn": 14, "complex": [1, 3, 7, 10, 11, 13, 14, 15, 23], "complic": [2, 3, 11, 15, 23], "compoment": 24, "compon": [1, 2, 3, 5, 6, 7, 8, 9, 11, 16, 17, 23, 24], "components_": 13, "compos": [11, 14, 15, 16, 17, 23], "compphys": [1, 2, 8, 17, 19, 21, 22, 23], "compress": [2, 23, 24], "compris": 8, "compromis": [7, 24], "compulsori": [17, 23], "comput": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 17, 18, 19, 20, 22, 23, 24], "computation": [2, 5, 8, 11, 15, 20, 23], "computationalscienceuio": 23, "concaten": [4, 6, 8, 16], "concav": [3, 15, 24], "concentr": 12, "concept": [2, 4, 17, 23, 24], "conceptu": [14, 15], "concern": [2, 3, 6, 9, 23], "concic": 23, "conclud": [2, 7, 15], "conclus": 3, "cond": 4, "conda": [2, 3, 17, 23], "condis": 24, "condit": [2, 4, 6, 7, 8, 10, 11, 13, 15, 20, 23, 24], "conduct": 17, "condwav": 4, "confid": [2, 7, 8, 9, 10, 23, 24], "configur": 5, "confirm": [7, 14], "confus": [7, 8, 9, 12, 18], "confusion_matrix": 11, "congruenti": 20, "conjug": [6, 10], "conjugaci": 15, "conjunct": 5, "connect": [2, 3, 5, 6, 11, 13, 14, 15, 18, 23, 24], "consequ": [7, 8, 10, 12, 14, 15, 24], "conserv": [7, 16, 24], "consid": [1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 14, 15, 18, 20, 23, 24], "consider": [2, 3, 7, 15, 23, 24], "consist": [3, 4, 5, 6, 8, 14, 15, 20, 24], "constant": [1, 2, 4, 6, 7, 8, 10, 14, 15, 20, 23, 24], "constitu": [2, 23], "constitut": [4, 8], "constrain": [3, 5, 7, 9, 13], "constraint": [7, 8, 10, 15, 24], "construct": [2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 18, 20, 23, 24], "contact": [2, 23], "contain": [0, 2, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 18, 20, 22, 23, 24], "contemporari": 23, "content": [0, 3, 17, 18, 23], "context": [8, 12, 15], "contigu": 18, "continu": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 18, 20, 23, 24], "contour": [11, 12, 15], "contourf": [10, 11, 12], "contrast": [3, 6, 11, 12, 14, 23], "contribut": [2, 5, 7, 15, 20, 23, 24], "contributor": 2, "control": [0, 2, 3, 5, 11, 15, 17, 23], "conv": [5, 6], "conv2d": [5, 6], "conv2dtranspos": 6, "convei": 23, "conveni": [7, 8, 14, 15, 18, 23], "convent": [14, 24], "converg": [3, 4, 6, 7, 10, 15, 16, 24], "convergencewarn": [], "convert": [2, 3, 6, 7, 11, 13, 15, 18, 23, 24], "converttomatrix": 6, "convex": [6, 7, 9, 24], "convinc": 15, "convolut": [3, 6, 17, 23], "cool": [6, 11], "coolwarm": 8, "coordin": [7, 14, 16, 24], "coorel": [], "copi": [0, 2, 3, 16, 24], "core": 12, "corel": 23, "coronari": 9, "corr": [7, 9, 13, 24], "correalt": [13, 17], "correct": [0, 2, 3, 4, 5, 6, 7, 9, 15, 18, 20, 23, 24], "correctli": [3, 4, 8, 9, 12], "correl": [2, 3, 5, 7, 8, 9, 12, 14, 15, 17, 20, 23], "correlation_matrix": [7, 9, 13, 24], "correspond": [2, 5, 7, 8, 10, 11, 13, 14, 17, 18, 20, 23, 24], "cortex": 14, "cosin": [5, 8], "cost": [1, 2, 4, 5, 7, 8, 9, 10, 11, 14, 15, 23], "cost_deep_grad": 4, "cost_funct": 4, "cost_function_deep": 4, "cost_function_deep_grad": 4, "cost_function_grad": 4, "cost_grad": 4, "cost_sum": 4, "costol": 15, "could": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 20, 23, 24], "coulomb": [2, 23], "count": [0, 2, 11, 19, 20, 21, 23], "counterpart": 23, "countor": 15, "coupl": [6, 7, 8], "cours": [0, 1, 2, 3, 5, 7, 13, 21, 24], "coursework": 0, "courvil": [22, 23, 24], "cov": [7, 8, 13, 18, 20, 23, 24], "cov_xi": [7, 13, 24], "cov_xx": [7, 13, 24], "cov_yi": [7, 13, 24], "covari": [2, 9, 17, 18, 23], "covariance_matrix": [7, 13, 16], "cover": [2, 7, 17, 21, 22, 24], "covert": [2, 23], "covxi": 20, "covxx": 20, "covxz": 20, "covyi": 20, "covyz": 20, "covzz": 20, "cpu": 3, "craft": 5, "creat": [0, 3, 5, 6, 7, 11, 12, 13, 14, 17, 23], "create_biases_and_weight": 3, "create_convolutional_neural_network_kera": 5, "create_neural_network_kera": 3, "create_x": [7, 13], "credit": [2, 9, 21, 23], "crim": [], "crime": [], "criteria": [2, 6, 11, 12, 16, 20, 23], "criterion": [11, 12, 15], "critic": 8, "cross": [0, 2, 3, 5, 9, 11, 12, 15, 17, 20, 23, 24], "cross_entropi": 6, "cross_val_scor": 8, "cross_valid": [9, 12], "crossvalid": 8, "crucial": [3, 20], "cs231": 5, "csr_matrix": [18, 23], "csv": [2, 6, 8, 9, 11], "ctnk": 3, "cubic": 2, "cumbersom": 7, "cumsum": [12, 13, 23], "cumul": [9, 12, 20], "cumulative_heads_ratio": 12, "cup": 7, "current": [0, 1, 3, 4, 5, 6, 15, 16, 22], "curs": [2, 24], "curv": [8, 9, 12, 14], "curvatur": 15, "custom": [8, 16], "custom_cmap": [11, 12], "custom_cmap2": [11, 12], "cutpoint": 11, "cv": [8, 9, 12], "cvxbook": 15, "cvxopt": [7, 10, 24], "cycl": [3, 14], "d": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 20, 21, 23, 24], "d2_g_t": 4, "d_f": 15, "d_g_t": 4, "d_net_out": 4, "da": 5, "dagger": [7, 18, 24], "dai": [3, 11, 17], "damp": 5, "darget": 11, "darkr": 20, "dat": [2, 23], "dat_id": [2, 8, 9, 11, 23], "data": [1, 4, 6, 7, 10, 12, 14, 15, 16, 18, 22], "data1": 16, "data2": 16, "data3": 16, "data4": 16, "data_id": [2, 8, 9, 11, 23], "data_indic": 3, "data_panda": 23, "data_path": [2, 8, 9, 11, 23], "databas": 3, "datafil": [2, 8, 9, 11, 23], "datafram": [2, 6, 7, 9, 11, 13, 23, 24], "datapoint": [1, 3, 7, 8, 9, 13, 15], "datasci": [0, 1], "dataset": [1, 2, 6, 8, 9, 10, 11, 12, 13, 15, 16, 23], "datatyp": 6, "date": [0, 23, 24], "daughter": 12, "david": 22, "dbh": 3, "dbo": 3, "dcomposit": 18, "ddot": 4, "dead": 3, "deadlin": 0, "deal": [2, 3, 5, 7, 8, 10, 13, 15, 16, 18, 20, 23, 24], "dealt": 2, "debt": 9, "debug": [2, 7, 8, 24], "decad": [2, 5], "decai": [2, 15, 20, 23], "decemb": [21, 23], "decent": 12, "decid": [2, 4, 5, 7, 8, 11, 24], "decim": [2, 23], "decis": [2, 3, 10, 13, 17, 22, 23], "decision_funct": 10, "decision_tre": 11, "decisiontreeclassifi": [11, 12], "decisiontreeregressor": [2, 11, 12], "declar": [2, 6, 18, 23], "decompos": [7, 8, 18, 24], "decomposit": [2, 8, 14, 23], "decompost": [7, 24], "deconvolut": 5, "decorrel": [12, 15], "decreas": [3, 4, 6, 7, 8, 12, 13, 15], "deduc": [2, 23], "deep": [5, 9, 14, 15, 17, 22, 24], "deep_neural_network": 4, "deep_param": 4, "deep_tree_clf": [11, 12], "deep_tree_clf1": 11, "deep_tree_clf2": 11, "deepen": [7, 17, 23], "deeper": [2, 5, 6, 23], "deeplearningbook": [22, 23], "deer": 5, "def": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 20, 23, 24], "def_covari": 20, "default": [2, 3, 4, 6, 8, 9, 18, 23], "default_tim": 6, "defect": [7, 24], "defici": [7, 24], "defin": [1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 24], "definit": [3, 4, 7, 8, 9, 10, 12, 13, 14, 15, 18, 20, 24], "defint": 20, "degre": [0, 1, 5, 7, 8, 10, 11, 12, 13, 20, 23], "deisenroth": 24, "del": 3, "delet": [0, 8], "delimit": 6, "deliv": [0, 19, 23], "delta": [2, 4, 5, 8, 10, 14, 15, 16, 23], "delta_": [3, 18], "delta_0": 5, "delta_1": 5, "delta_2": 5, "delta_3": 5, "delta_4": 5, "delta_5": 5, "delta_h": [2, 3, 23], "delta_j": [5, 14], "delta_k": 14, "delta_l": [3, 5], "delta_momentum": 15, "delta_n": [2, 5, 23], "delug": 17, "delv": 2, "demand": 15, "demonstr": [2, 5, 7, 8, 9, 13, 14, 17, 23, 24], "den": 6, "denomin": [3, 7], "denot": [3, 4, 8, 9, 15, 20], "dens": [3, 5, 6], "densiti": [2, 4, 8, 20], "depart": [21, 23, 24], "depend": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 13, 14, 15, 17, 18, 20, 23, 24], "depict": 20, "deploy": [2, 17, 23], "depth": [2, 5, 11, 12, 18], "deriv": [2, 3, 4, 8, 9, 10, 12, 13, 15, 17, 23], "derivati": 15, "derivative_fn": 15, "descend": [7, 11, 13, 24], "descent": [2, 3, 5, 9, 10, 14, 23, 24], "describ": [2, 4, 6, 7, 8, 10, 12, 13, 14, 15, 18, 23], "descript": [2, 10, 11, 23], "design": [2, 3, 5, 6, 7, 8, 9, 12, 13, 14, 15, 23], "designmatrix": [2, 23], "desir": [2, 4, 6, 7, 15, 16, 23, 24], "desktop": 0, "despit": [3, 14], "destroi": 18, "det": [7, 18, 24], "detail": [2, 8, 13, 15, 16, 18, 24], "detect": [5, 10, 14], "determin": [2, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 18, 20, 23, 24], "determinist": [9, 15, 20], "dev": 3, "develop": [2, 5, 7, 10, 12, 13, 14, 17, 18, 23, 24], "deviat": [2, 3, 4, 6, 7, 8, 20, 23, 24], "devis": 14, "df": [6, 10, 13, 15, 23], "df1": 23, "di": [], "diag": [7, 10, 24], "diagnost": [3, 12], "diagon": [2, 7, 9, 15, 18, 20, 23, 24], "diagonaliz": [7, 24], "diagram": 12, "diagsvd": 8, "dice": [8, 20], "dict": [8, 10], "dictionari": [], "did": [1, 2, 3, 7, 8, 9, 12, 13, 16, 23], "die": 3, "diff": 4, "diff1": 4, "diff2": 4, "diff_ag": 4, "diffeent": 10, "differ": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 23, 24], "differenti": [1, 2, 5, 17, 18, 23, 24], "difficult": [2, 3, 8, 12, 15, 20, 23], "difficulti": [2, 3, 15, 23], "diffonedim": 4, "digit": [2, 3, 5, 6, 8, 21, 23], "dilemma": 15, "dilut": 3, "dim": [6, 13, 16, 18], "dimens": [1, 2, 3, 4, 5, 6, 7, 10, 13, 16, 18, 23, 24], "dimension": [2, 6, 7, 8, 11, 13, 15, 16, 17, 18, 23, 24], "dimensionless": [2, 5, 23], "diment": 18, "dimnsion": 6, "diod": 5, "direct": [2, 3, 4, 6, 13, 14, 15, 16, 23, 24], "directli": [3, 6, 7, 8, 20, 24], "disadvantag": [2, 23], "disappear": [5, 8], "disc_loss": 6, "disc_tap": 6, "discard": [8, 13], "disciplin": [2, 5, 14], "disclaim": 20, "discord": 23, "discourag": [0, 15], "discov": [2, 23], "discover": 7, "discret": [3, 5, 7, 9, 15], "discrimin": [6, 9, 12, 13], "discriminator_loss": 6, "discriminator_loss_list": 6, "discriminator_model": 6, "discriminator_optim": 6, "discuss": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 23, 24], "diseas": 9, "disguis": 8, "disord": [3, 9], "displai": [2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 20, 23, 24], "displaystyl": [2, 7, 23, 24], "disregard": [2, 23], "dissimilar": [13, 16], "dist": 16, "distanc": [10, 11, 13, 16, 20], "distance_list": 11, "distinct": [5, 9, 10, 11, 12, 16], "distinctli": 10, "distinguish": [2, 6, 9, 10, 20, 23], "distplot": [], "distribut": [2, 3, 6, 8, 9, 12, 13, 15, 16, 17, 18, 23, 24], "distrubut": [2, 17, 23], "dive": [2, 10, 18, 23], "diverg": [3, 15], "divid": [2, 3, 5, 7, 8, 9, 10, 11, 13, 14, 20, 23, 24], "divis": [8, 10, 11, 15, 18, 20], "dna": 9, "dnn": [2, 3, 4, 6, 14, 23], "dnn1": 6, "dnn2_gru2": 6, "dnn_kera": 3, "dnn_model": 3, "dnn_numpi": 3, "dnn_scikit": [2, 3, 23], "do": [0, 1, 2, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 18, 23, 24], "doc": [0, 1, 2, 17, 19, 21, 22, 23], "document": [0, 6, 15], "doe": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14, 15, 18, 20, 23], "doesn": [5, 11, 14, 23], "dog": [3, 5, 6], "domain": [7, 10, 15], "domin": [2, 23], "don": [0, 1, 2, 3, 5, 7, 8, 10, 13, 15, 17, 23, 24], "done": [1, 2, 4, 5, 6, 7, 8, 11, 12, 13, 15, 18, 23, 24], "dot": [2, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 20, 23, 24], "doubl": [1, 5, 6, 18, 23], "doubli": 3, "down": [2, 5, 8, 11, 13, 14, 15], "download": [0, 2, 3, 5, 7, 8, 18, 22, 23], "downsampl": 5, "dozen": 3, "dq": 8, "drag": 15, "dramat": 13, "drastic": 6, "draw": [6, 8, 12, 15], "drawback": [2, 3, 5, 15, 24], "drawn": [3, 6, 8, 9, 13, 20, 23], "drive": [5, 6], "driven": 5, "drop": [2, 3, 7, 8, 13, 15, 20, 23, 24], "dropna": [2, 8, 23], "dropout": 6, "dt": [4, 5, 15, 20], "dtype": [2, 3, 5, 6, 16, 18, 23], "dub": [2, 23], "due": [3, 4, 7, 8, 10, 12, 14, 15, 21, 23, 24], "dummi": [], "dure": [2, 3, 5, 6, 10, 11, 13, 17, 23], "dwell": [], "dwh": 3, "dwo": 3, "dx": [4, 5, 10, 20], "dx_1": 20, "dx_1p": 8, "dx_2p": 8, "dx_mp": 8, "dx_n": 20, "dxp": 8, "dy": [3, 10, 20], "dynam": 6, "dz": 10, "e": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 21, 23, 24], "e_": [2, 4, 23], "each": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24], "eapprox": [2, 23], "earli": [3, 15], "earlier": [2, 7, 9, 10, 11, 13, 14, 15, 23, 24], "earthexplor": 8, "eas": [8, 11, 16], "easi": [0, 2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 23, 24], "easier": [0, 7, 8, 10, 11, 15, 20, 23, 24], "easiest": 15, "easili": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 23, 24], "eastern": [21, 23], "ebind": [2, 23], "eblock": 11, "econometr": 23, "economi": 7, "ecosystem": [17, 23], "ect": 19, "edg": 5, "edgecolor": 8, "editor": 0, "edu": 15, "educ": [2, 23], "eff": 20, "effect": [1, 3, 6, 12, 15, 20], "effic": 3, "effici": [2, 5, 12, 15, 17, 18, 20, 23], "efron": 8, "egrad": 15, "eig": [7, 13, 15, 18, 20, 23, 24], "eigen": 20, "eigenpair": [7, 13, 24], "eigenvalu": [2, 7, 10, 13, 15, 18, 23, 24], "eigenvector": [7, 13, 15, 24], "eight": [18, 23], "eigval": [18, 20, 23], "eigvalu": [13, 15], "eigvec": [18, 20, 23], "eigvector": [13, 15], "eir": [21, 23], "eispack": [18, 23], "either": [3, 7, 8, 9, 10, 11, 12, 13, 15, 20, 23, 24], "eivind": 21, "eivinsto": 21, "ekstr\u00f8m": 6, "elabor": 20, "elarn": 5, "electr": [2, 5, 14, 23], "electron": 23, "eleg": 13, "element": [3, 4, 5, 6, 7, 8, 9, 10, 13, 14, 15, 17, 18, 22, 24], "elementari": [12, 15, 18], "elementwis": [5, 15], "elementwise_grad": [4, 15], "elessar": 23, "elif": 16, "elim": 18, "elimin": [5, 10], "elin": [21, 23], "ellipsi": 1, "els": [1, 3, 5, 6, 9, 11, 14, 15, 18], "elu": 3, "elus": [2, 23], "email": [19, 21, 23], "embed": [2, 13, 24], "embodi": 8, "emit": 20, "emner": 22, "emphas": [2, 12, 17, 23], "emphasi": [2, 17, 22, 23], "empir": [3, 13, 20], "emploi": [2, 3, 7, 8, 13, 15, 20, 23, 24], "employ": 2, "empti": [0, 8, 12], "emul": 14, "en": [17, 22], "enabl": 13, "enbodi": 8, "encod": [2, 5, 7, 11, 13, 16, 23, 24], "encompass": [2, 20], "encount": [0, 2, 3, 7, 9, 15, 20, 23, 24], "encourag": 0, "end": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 21, 23, 24], "endpoint": [5, 8], "energi": [2, 6, 8], "enforc": 14, "eng": 22, "engin": [2, 3, 5, 6, 17, 23], "enorm": 5, "enough": [2, 8, 15, 23], "ensembl": [3, 11, 23], "ensur": [2, 3, 4, 5, 7, 8, 13, 15, 20, 24], "entail": 23, "enter": [7, 8, 24], "enthought": [2, 17, 23], "entir": [3, 5, 9, 11, 17, 20, 23], "entiti": [11, 14, 18, 23], "entri": [2, 7, 10, 13, 14, 18, 23, 24], "entropi": [3, 5, 9, 12, 15, 23], "enumer": [2, 3, 4, 5, 6, 8, 10, 23], "env": 20, "environ": [4, 17, 23], "environemnt": 0, "eo": [2, 8], "eol": 2, "eosfit": 2, "epoch": [2, 3, 5, 6, 14, 15, 23], "epsilon": [2, 7, 8, 9, 15, 23, 24], "epsilon_": [2, 23], "epsilon_0": [2, 23], "epsilon_1": [2, 23], "epsilon_2": [2, 23], "epsilon_i": [2, 23, 24], "eq": [5, 15, 16, 18, 20], "eqnarrai": [5, 7, 8], "equal": [1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 13, 14, 15, 16, 18, 20, 23, 24], "equat": [3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 20, 23], "equilibrium": [4, 14], "equiv": [5, 15, 18, 20], "equival": [2, 3, 7, 9, 10, 13, 15, 17, 18, 23, 24], "erf": 20, "eriador": 23, "err": [2, 12], "err_": 8, "err_sqr": 4, "errat": 15, "erron": 4, "error": [0, 1, 3, 4, 6, 7, 8, 9, 11, 13, 14, 15, 17, 18, 20], "error_estimate_corr_tim": 20, "error_hidden": 3, "error_output": 3, "escap": 15, "especi": [0, 3, 5, 11, 14, 15], "essenti": [0, 2, 7, 8, 11, 12, 14, 16, 20, 24], "establish": [1, 2, 8, 12, 13], "estim": [2, 3, 7, 8, 9, 12, 13, 15, 17, 20, 23, 24], "estimated_mse_fold": 8, "estimated_mse_kfold": 8, "estimated_mse_sklearn": 8, "et": [1, 2, 4, 6, 22, 23, 24], "eta": [2, 3, 5, 10, 14, 15, 23], "eta0": [10, 15], "eta_": 15, "eta_t": 15, "eta_v": [2, 3, 5, 23], "etc": [2, 3, 5, 7, 9, 10, 11, 13, 14, 15, 16, 17, 18, 20, 24], "ethic": 17, "euclidean": [2, 16, 24], "evalu": [0, 1, 2, 4, 5, 6, 7, 8, 11, 15, 20, 23, 24], "evalut": 15, "even": [2, 3, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 23, 24], "evenli": 6, "event": [7, 9, 12, 20], "eventu": [2, 7, 8, 13, 14, 15, 21, 24], "everi": [0, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16, 17, 20, 21, 23, 24], "everyth": [1, 6, 14], "everywher": [6, 15], "evolv": 2, "exact": [2, 7, 13, 14, 15, 18, 20, 23, 24], "exactli": [2, 5, 6, 8, 14, 17, 24], "exam": 23, "examin": 8, "exampl": [0, 1, 2, 7, 13, 14, 15, 17, 18, 20, 22], "exce": [3, 14, 15], "excel": [2, 3, 6, 7, 12, 23, 24], "except": [5, 6, 8, 10, 11, 18], "excess": [2, 23], "excit": 2, "exclud": [3, 8, 14], "exclus": [2, 3, 5, 8, 20, 23], "execut": [0, 4, 7, 15, 24], "exemplifi": 15, "exercic": [21, 23], "exercis": [7, 17, 19, 21, 23], "exhaust": 8, "exhibit": [2, 7, 8, 10, 23, 24], "exist": [2, 3, 4, 5, 7, 8, 9, 10, 11, 15, 18, 23], "exit": [7, 18, 24], "exp": [1, 2, 3, 4, 7, 8, 9, 10, 12, 13, 14, 15, 20, 24], "exp_term": 3, "expand": [7, 9, 13, 15], "expans": [2, 5, 7, 10, 12, 14, 15, 23, 24], "expect": [0, 2, 3, 7, 8, 9, 13, 14, 15, 17, 23, 24], "expectation_value_of_h_wrt_p": 20, "expens": [1, 8, 12, 15], "experi": [0, 2, 3, 8, 10, 15, 17, 23, 24], "experiment": [2, 6, 8, 11, 20, 23], "expert": [3, 11], "explain": [1, 2, 8, 11, 12, 13, 15, 23], "explained_variance_ratio_": 13, "explanatori": [2, 23], "explicit": [2, 5, 8, 15, 18, 23, 24], "explicitli": [2, 6], "explod": 3, "exploit": [2, 5, 14, 15, 23], "explor": [3, 6, 8, 10, 15, 17, 23], "expon": 3, "exponenti": [2, 3, 7, 8, 12, 15, 20, 23], "export": [0, 1, 11], "export_graphviz": 11, "export_text": 11, "exporttext": 11, "expos": 17, "express": [2, 4, 5, 7, 8, 9, 12, 14, 15, 18, 20, 23], "exptmean": 20, "exptvari": 20, "extend": [2, 4, 9, 13, 15, 17, 23], "extens": [0, 2, 14, 17, 23], "extent": [2, 3, 8, 22], "extern": [5, 8, 11], "extra": [0, 3, 5, 7, 21, 23, 24], "extract": [1, 2, 5, 7, 8, 9, 10, 13, 15, 18, 23, 24], "extrapol": [2, 23], "extrem": [0, 1, 2, 3, 6, 7, 8, 9, 10, 11, 15, 18, 24], "extremum": 15, "extrins": 13, "ey": [2, 7, 8, 15, 16, 18, 23, 24], "f": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 14, 15, 16, 18, 20, 21, 23, 24], "f1": 15, "f11": [2, 23], "f12": [2, 23], "f13": [2, 23], "f1_grad": 15, "f1d": 15, "f2": 15, "f2_grad_x1": 15, "f2_grad_x1_analyt": 15, "f2_grad_x2": 15, "f2_grad_x2_analyt": 15, "f3": 15, "f3_grad": 15, "f3_grad_analyt": 15, "f4": 15, "f4_grad": 15, "f4_grad_analyt": 15, "f5": 15, "f5_grad": 15, "f6": 15, "f6_for": 15, "f6_for_grad": 15, "f6_grad_analyt": 15, "f6_while": 15, "f6_while_grad": 15, "f7": 15, "f7_grad": 15, "f7_grad_analyt": 15, "f8": 15, "f8_grad": 15, "f9": [2, 15, 23], "f9_altern": 15, "f9_alternative_grad": 15, "f9_grad": 15, "f_": 12, "f_0": [5, 12], "f_1": [12, 15], "f_2": [14, 15], "f_3": 14, "f_d": 20, "f_grad": 15, "f_grad_analyt": 15, "f_i": [1, 2, 8, 14], "f_m": [5, 12], "f_n": 5, "f_vec": 4, "face": [15, 23], "facecolor": [8, 10, 20], "facil": [2, 17], "facilit": 14, "fact": [2, 3, 5, 7, 11, 13, 14, 15, 23, 24], "factor": [2, 3, 5, 7, 8, 11, 12, 13, 15, 18, 20, 23, 24], "factori": 15, "fade": 8, "fafab0": [11, 12], "fail": [2, 8, 15, 21, 23], "failur": 9, "fairli": [3, 4, 20], "faisal": [1, 24], "fake": 6, "fake_loss": 6, "fake_output": 6, "fall": [10, 11, 19], "fals": [1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 16, 18, 23, 24], "famili": [2, 9, 10, 20, 24], "familiar": [0, 2, 5, 7, 8, 10, 17, 18, 20, 23], "famou": [8, 14], "far": [1, 2, 5, 6, 7, 8, 10, 13, 14, 15, 16, 23, 24], "fashion": [2, 11, 12, 23], "fast": [3, 5, 8, 12, 14, 15, 17, 20, 23], "faster": [3, 13, 15], "fastest": [15, 18], "favor": 9, "favorit": 20, "fc": 5, "featur": [0, 2, 3, 5, 7, 8, 9, 10, 12, 13, 14, 15, 17, 20, 23], "feature_nam": [3, 9, 11], "feautur": 11, "fed": 3, "feed": [2, 4, 5, 13, 17, 23], "feed_forward": 3, "feed_forward_out": 3, "feed_forward_train": 3, "feedback": [6, 23], "feeddorward": 6, "feedforward": [3, 6, 14], "feel": [0, 1, 2, 7, 8, 13, 15, 17, 21, 23], "feet": [], "fetch": [0, 8], "few": [3, 5, 6, 7, 11, 20, 23], "fewer": [2, 11, 13, 23], "ffnn": [3, 14], "field": [2, 5, 8, 14, 17], "fifth": [2, 8, 23], "fig": [2, 3, 4, 5, 6, 8, 9, 14, 15, 16, 23], "fig_id": [2, 8, 9, 11, 23], "figaxi": 20, "figsiz": [2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 23], "figur": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 23, 24], "figure_id": [2, 8, 9, 11, 23], "figurefil": [2, 8, 9, 11, 23], "file": [0, 2, 6, 7, 8, 9, 11, 23], "file_prefix": 6, "filenam": 23, "fill": [7, 11, 24], "filter": [5, 6], "final": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 19, 20, 21, 23], "financ": 2, "find": [0, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 20, 23, 24], "fine": [2, 16], "finish": 4, "finit": [5, 7, 8, 14, 15, 20, 24], "finnicki": 0, "first": [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 20, 21, 22, 24], "firsteigvector": 13, "fit": [3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 20, 24], "fit_intercept": [1, 2, 7, 8, 24], "fit_mod": 11, "fit_theta": 8, "fit_transform": [0, 2, 8, 10, 11, 13], "fiti": [2, 23], "five": [2, 11, 23, 24], "fix": [2, 5, 6, 8, 12, 13, 14, 15, 23], "flag": 6, "flat": [14, 15], "flatten": [3, 5, 6, 7, 18], "flexibl": [3, 8, 10, 12, 14, 23], "flip": [21, 23], "float": [2, 5, 6, 7, 11, 13, 15, 16, 18, 23, 24], "float32": [6, 11], "float64": [6, 18, 23], "flop": [7, 18, 24], "flow": [3, 6, 14], "fluctuat": 7, "fly": 13, "fm": 2, "fmax": 5, "fmesh": 15, "fn": 9, "focu": [0, 2, 5, 6, 7, 8, 17, 22, 23], "focus": [3, 8, 9, 18], "fold": [8, 11], "folder": [0, 2, 6, 8, 23], "follow": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 23, 24], "font": [9, 20, 23], "fontdict": 20, "fontsiz": [3, 8, 10, 11, 12, 20], "fontweight": 3, "footprint": 5, "foral": [10, 24], "forc": [2, 7, 8, 12, 13, 24], "forcast": 6, "forecast": [6, 14], "forest": [2, 3, 11, 17, 23], "forget": 13, "form": [0, 1, 2, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 17, 18, 20, 23, 24], "formal": [5, 6, 16, 20], "format": [2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 17, 20, 22], "format_data": 6, "formatstrformatt": [8, 15], "formul": [6, 8, 13, 16], "formula": [5, 15, 20], "forth": [6, 14], "fortran": [2, 17, 18, 23], "fortran2003": [17, 23], "fortran90": 20, "fortun": [2, 13, 24], "forward": [2, 5, 8, 17, 18, 23], "found": [3, 4, 6, 7, 8, 14, 15, 23, 24], "foundat": [17, 23], "four": [6, 7, 8, 10, 14, 18, 19, 21, 23], "fourier": [2, 23], "fourierdef1": 5, "fourierdef2": 5, "fourierseriessign": 5, "fourth": [14, 23, 24], "fp": 9, "frac": [1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "fraction": 11, "frame": 9, "framework": [3, 10, 12, 20], "frank": [7, 13], "frankefunct": [7, 8, 13], "fredli": [21, 23], "free": [0, 1, 2, 8, 13, 15, 17, 18, 20, 21, 22, 23], "freecodecamp": 17, "freedom": 7, "freeli": 2, "freez": 0, "frequenc": [5, 8, 9, 20], "frequent": [2, 10, 11, 15], "frequentist": 17, "fresh": 12, "fridai": [0, 21, 23], "friedman": [8, 22, 23], "friendli": 6, "frodo": 23, "frog": 5, "from": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 13, 15, 16, 17, 18, 20, 21, 22], "from_cod": 11, "from_logit": [5, 6], "from_tensor_slic": 6, "front": [2, 6, 7, 23, 24], "frustrat": 0, "fulfil": [4, 7, 14, 24], "full": [3, 5, 7, 9, 11, 12, 15, 20, 23, 24], "full_matric": [7, 24], "fulli": [5, 8, 14, 20], "fun": [17, 23], "func": 4, "function": [0, 1, 4, 5, 6, 7, 11, 16, 17, 18], "functionali": 13, "fundament": [2, 8, 17, 23], "funtion": 4, "further": [4, 9, 11, 23], "furthermor": [2, 5, 7, 8, 9, 13, 14, 15, 17, 23, 24], "futur": [2, 6, 10, 11, 23], "fy": [0, 19, 21, 22, 23], "fys5419": [22, 23], "fys5429": [22, 23], "f\u00f8470": [21, 23], "g": [0, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 15, 20, 23, 24], "g0": 4, "g_": [4, 11, 12], "g_0": 4, "g_1": [4, 12], "g_2": [4, 12], "g_analyt": 4, "g_dnn_ag": 4, "g_euler": 4, "g_i": 4, "g_m": [5, 12], "g_n": 5, "g_re": 4, "g_t": 4, "g_t_d2t": 4, "g_t_d2x": 4, "g_t_dt": 4, "g_t_hessian": 4, "g_t_hessian_func": 4, "g_t_jacobian": 4, "g_t_jacobian_func": 4, "g_trial": 4, "g_trial_deep": 4, "g_vec": 4, "gain": [3, 7, 9, 11, 12, 15, 24], "galleri": [2, 23], "game": 6, "gamge": 23, "gamma": [2, 4, 10, 11, 12, 13, 15, 23], "gamma1": 10, "gamma2": 10, "gamma_": [2, 23], "gamma_0": 12, "gamma_1": 12, "gamma_1x": 12, "gamma_i": [2, 10, 20, 23], "gamma_j": 15, "gamma_k": 15, "gamma_m": 12, "gamma_x": [2, 23], "gap": 10, "gate": [6, 14], "gather": [2, 3, 14, 24], "gaug": 14, "gaussbacksub": 18, "gaussian": [6, 7, 8, 10, 16, 20, 23], "gaussian_point": 16, "gaussian_rbf": 10, "gave": 15, "gavra": 23, "gbc": 23, "gca": [4, 8, 10, 15], "gd": 3, "gd_clf": 12, "gdclassiffiercgain": 12, "gdclassiffierconfus": 12, "gdclassiffierroc": 12, "gdm": 15, "gdregress": 12, "ge": [3, 7, 9, 20, 24], "gen_loss": 6, "gen_tap": 6, "gender": [2, 23], "genener": 6, "gener": [0, 1, 2, 3, 4, 5, 7, 8, 10, 12, 13, 14, 15, 16, 18, 20, 22, 24], "generaliz": 1, "generallay": 14, "generate_and_save_imag": 6, "generate_imag": 6, "generate_latent_point": 6, "generate_simple_clustering_dataset": 16, "generated_imag": 6, "generator_loss": 6, "generator_loss_list": 6, "generator_model": 6, "generator_optim": 6, "genom": 17, "geodes": 13, "geometr": [2, 15, 23], "geometri": 7, "georg": 22, "geotif": 8, "geq": [4, 7, 10, 11, 15, 24], "geron": [2, 22, 23], "get": [0, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 17, 18, 20, 21, 23, 24], "get_dummi": 11, "get_paramet": 4, "get_split": 11, "get_yaxi": 10, "get_yticklabel": 8, "gh": 0, "gibb": [17, 23], "gif": 6, "gini": 12, "gini_index": 11, "ginvers": 15, "git": [0, 2, 17, 23], "giter": 15, "github": [2, 17, 19, 21, 22, 23, 24], "gitignor": 0, "gitlab": [0, 2, 17, 23], "give": [2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 20, 23, 24], "given": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "global": [8, 9, 15], "glorot": 3, "gnew": 15, "go": [0, 1, 2, 3, 5, 7, 8, 10, 11, 13, 14, 15, 23, 24], "goal": [2, 9, 11, 23], "goe": [0, 2, 3, 4, 7, 8, 15, 16, 18, 23, 24], "golden": 15, "gone": [7, 24], "gong": 3, "good": [0, 3, 5, 6, 7, 8, 11, 12, 13, 15, 17, 20, 22, 24], "goodfellow": [6, 22, 23, 24], "googl": [3, 6, 17, 23], "got": [3, 8], "gotten": 23, "gov": 8, "govern": 23, "gp": 22, "gpu": [3, 15, 17, 23], "grad": [4, 15], "grad_analyt": 15, "grade": 19, "gradient": [2, 5, 6, 9, 10, 11, 14, 17, 23, 24], "gradientboostingclassifi": 12, "gradientboostingregressor": 12, "gradients_of_discrimin": 6, "gradients_of_gener": 6, "gradienttap": 6, "gradual": [3, 16], "grai": [6, 8], "graph": [1, 3, 11, 13, 14, 15], "graph_from_dot_data": 11, "graphic": [0, 2, 3, 11, 23], "grasp": 2, "gray_r": [3, 5], "grayscal": 5, "great": [0, 7, 15], "greater": [3, 9, 20, 24], "greatli": 15, "greedi": 11, "green": [2, 5, 11, 20], "grei": 6, "grid": [3, 5, 8, 9, 10, 14, 20], "grossli": 15, "ground": [2, 23], "group": [0, 2, 8, 9, 11, 16, 17, 19, 21, 23], "groupbi": [2, 23], "grow": [3, 5, 11, 12], "growth": [2, 23], "gru": 6, "guarante": [2, 6, 15, 20, 23, 24], "guess": [3, 6, 12, 15, 16], "guestrin": 12, "gui": 0, "guid": 3, "h": [0, 2, 3, 7, 8, 10, 15, 20, 21, 22, 23, 24], "h1": 4, "h_": [2, 15, 23], "h_1": [4, 15], "h_2": [4, 15], "h_m": 12, "ha": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "haanen": [21, 23], "habit": [2, 24], "had": [2, 3, 8, 9, 15, 23], "hadamard": [3, 14, 15], "half": [3, 10, 11], "halv": 12, "hand": [2, 3, 4, 5, 7, 13, 14, 15, 17, 18, 20, 21, 22, 23, 24], "handi": 5, "handl": [0, 2, 3, 4, 7, 11, 13, 17, 24], "handle_unknown": 11, "handsid": 14, "handwrit": 14, "handwritten": [3, 7], "happen": [3, 4, 5, 6, 7, 8, 12, 15, 20, 24], "hard": [3, 9, 10, 12, 15], "hardcopi": [17, 23], "harder": [2, 3, 24], "harmon": 5, "hasn": [], "hassl": [2, 17, 23], "hast": [17, 23], "hasti": [1, 2, 8, 22, 23, 24], "hat": [1, 2, 3, 7, 8, 9, 11, 12, 13, 14, 15, 18, 24], "have": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 23, 24], "haven": 3, "he": 9, "head": [6, 12, 20], "header": [2, 23], "heads_proba": 12, "health": [2, 24], "hear": [2, 15, 23], "heart": [2, 9, 23], "heatmap": [2, 3, 5, 9, 23], "heavili": 2, "heavisid": 3, "height": [3, 5, 8], "held": 15, "help": [0, 1, 2, 3, 6, 14, 15, 23], "helper": [6, 16], "henc": [2, 7, 8, 10, 11, 12, 14, 15, 23, 24], "henrik": [21, 23], "her": 9, "here": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 23, 24], "hereaft": [2, 10, 14, 23], "hermitian": 18, "hessenberg": 18, "hessian": [2, 4, 7, 15], "heterogen": [11, 12], "hi": 9, "hidden": [3, 5, 6, 14], "hidden_bia": 3, "hidden_bias_gradi": 3, "hidden_layer_s": [2, 3, 23], "hidden_neuron": 6, "hidden_weight": 3, "hidden_weights_gradi": 3, "hierarch": [7, 24], "high": [2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 15, 16, 17, 18, 23, 24], "higher": [2, 3, 5, 7, 8, 10, 15, 23, 24], "highest": [3, 4], "highli": [2, 5, 6, 12, 17, 18, 22, 23, 24], "highwai": [], "hing": 10, "hint": [0, 1, 15, 24], "hip": 17, "hire": 2, "hist": [6, 8, 9, 20], "histogram": [8, 9, 20], "histor": [9, 13], "histori": [0, 5, 6, 14], "hitherto": 7, "hjorth": [21, 23, 24], "hobbi": 20, "hoc": [7, 24], "hoff": 22, "hold": [3, 5, 8, 15, 16], "holder": [2, 23], "home": [], "homepag": 23, "homework": [8, 15], "homogen": [3, 5, 11, 12, 15], "honchar": 4, "hopefulli": [0, 2, 13, 20, 23], "horizont": 13, "horlyk": [21, 23], "hors": [5, 9, 23], "hot": [3, 11], "hour": [3, 17, 19, 20, 21, 23], "how": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 23, 24], "howev": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 23, 24], "hspace": [2, 6, 10, 12, 20, 23], "hstack": 3, "htf": 23, "html": [1, 2, 17, 19, 21, 22, 23], "http": [0, 1, 2, 5, 6, 8, 15, 17, 18, 19, 21, 22, 23, 24], "huang": [2, 23], "huber": [2, 23], "huge": [3, 5, 6, 17], "human": [2, 3, 5, 8, 11, 14], "humid": 11, "hundr": 3, "hungri": 3, "hybrid": 19, "hydrogen": [2, 23], "hyperbol": [3, 6, 14], "hyperparam": 10, "hyperparamet": [5, 6, 7, 8, 11, 15, 24], "hyperplan": 13, "h\u00f8rlyk": [21, 23], "i": [0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24], "i0": [2, 23], "i1": [2, 8, 10, 14, 23], "i2": [2, 10, 14, 23], "i3": [2, 14, 23], "i5": [2, 23], "i_": 15, "i_1": [7, 8], "i_2": [7, 8], "ian": 22, "ic": 3, "id": [9, 15], "ida": [21, 23], "idea": [2, 3, 4, 5, 6, 8, 11, 12, 14, 15, 18], "ideal": [2, 4, 8, 10, 15, 20, 23], "idem": 8, "ident": [7, 8, 14, 15, 18, 24], "identifi": [2, 3, 9, 11, 13, 14, 15, 16, 23, 24], "ieor": 20, "ifi": 22, "ifs": [17, 23], "ignor": [0, 2, 3, 5, 11, 24], "ii": [18, 20], "iii": [18, 23], "ij": [1, 2, 3, 5, 8, 10, 14, 16, 18, 20, 23, 24], "ik": [2, 18, 23, 24], "illustr": [7, 9, 12, 14, 15, 16, 17, 23], "im": 8, "imag": [3, 5, 6, 8, 11, 13, 14, 16, 22, 23], "image_at_epoch_": 6, "image_batch": 6, "image_height": 5, "image_path": [2, 8, 9, 11, 23], "image_width": 5, "imageio": 8, "images_from_seed_imag": 6, "imagin": 3, "immedi": [2, 5, 6, 8, 17, 23], "implement": [2, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 20, 23, 24], "impli": [5, 7, 8, 9, 15, 18, 24], "implicit": 5, "implicitli": [13, 20], "import": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20], "importantli": 5, "impos": [2, 8, 13, 14, 23], "imposs": [2, 7, 23, 24], "impress": [2, 14, 23], "improv": [0, 2, 6, 7, 11, 12, 13, 15, 24], "impur": 11, "imread": 8, "imshow": [3, 5, 6, 8], "in3050": [22, 23], "in3310": 23, "in4080": [22, 23], "in4300": [22, 23], "in4310": 22, "in5400": 5, "in5550": 22, "in_out_neuron": 6, "inaccur": 15, "inact": 14, "inadequ": [2, 23], "inch": 8, "includ": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 13, 14, 17, 20, 21, 22, 23, 24], "include_bia": [8, 11], "incom": [1, 14], "incorrect": 3, "incoveni": 10, "increas": [2, 3, 5, 6, 7, 8, 11, 14, 15, 20, 23], "increasingli": 20, "ind": 8, "inde": [2, 4, 6, 7, 8, 15, 23, 24], "indefinit": 6, "independ": [2, 7, 8, 9, 10, 14, 15, 20, 23, 24], "index": [2, 3, 5, 6, 12, 16, 17, 18, 20, 22, 23], "index_col": [2, 23], "indic": [1, 2, 3, 5, 6, 7, 8, 11, 12, 13, 15, 23, 24], "indispens": 8, "individu": [3, 8, 9, 12, 14, 20, 23, 24], "indu": [], "indx": 18, "indx1": 4, "indx2": 4, "indx3": 4, "ineffici": [5, 15], "inequ": [10, 15], "inertia": 15, "inf1000": [17, 23], "inf1100": [17, 23], "inf1100l": [17, 23], "inf1110": [17, 23], "inf3000": 23, "infeas": 11, "infer": [2, 3, 6, 8, 22, 23], "inferenc": 3, "infil": [2, 8, 9, 11, 23], "infin": [7, 8, 9, 13, 24], "infinit": 5, "infinitesim": 20, "influenc": [8, 12], "influenti": 3, "info": 23, "inform": [2, 3, 5, 6, 8, 11, 13, 14, 15, 16, 18, 22, 23], "inforom": 0, "infti": [5, 8, 15, 20], "ingeni": 15, "ingredi": [2, 11, 23], "inher": 8, "inherit": [18, 23], "initi": [2, 3, 4, 8, 12, 15, 16, 18, 20, 23], "inject": 16, "inlin": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "inner": [2, 15, 24], "inp": 6, "inplac": 15, "input": [1, 2, 3, 5, 6, 7, 8, 9, 10, 14, 15, 16, 20, 23, 24], "input_dim": 3, "input_shap": [5, 6], "inputs": 3, "inputs_shuffl": [2, 3, 24], "insert": [5, 7, 8, 10, 12, 20, 24], "insid": [6, 9], "insight": [2, 3, 7, 17, 23, 24], "insist": [8, 15], "inspir": [2, 3, 14, 23], "instabl": 4, "instal": [0, 2, 3, 7, 8, 11], "instanc": [1, 2, 3, 4, 6, 8, 11, 13, 15, 23, 24], "instanti": 12, "instead": [2, 3, 4, 5, 6, 7, 8, 10, 11, 13, 15, 16, 18, 20, 23, 24], "institut": 3, "instruct": [0, 2, 3], "int": [2, 3, 4, 5, 6, 7, 8, 13, 15, 16, 18, 20, 24], "int32": 12, "int_": [5, 8, 20], "int_0": 20, "int_a": 20, "intak": [2, 24], "integ": [3, 4, 15, 16, 18, 20, 23], "integer_vector": 3, "integr": [5, 8, 20, 23], "intellig": [2, 16, 22, 23], "intend": 12, "intens": 3, "intention": 16, "interact": [2, 8, 11, 14, 17, 23], "intercept": [1, 2, 8, 10, 13, 15, 23, 24], "intercept_": [2, 8, 10, 11, 15, 23], "interchang": [7, 14, 18], "interconnect": 3, "interest": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 14, 17, 20, 23, 24], "interfac": [0, 2, 3, 18, 24], "interior": [2, 11, 23], "intermedi": [18, 24], "intern": [3, 12, 14], "interpol": [3, 5, 6, 8, 14], "interpr": [7, 24], "interpret": [0, 1, 2, 3, 8, 11, 12, 14, 15, 18, 20], "interv": [2, 5, 7, 8, 9, 15, 20, 23, 24], "intial": 15, "intract": [2, 6, 24], "intrins": [5, 13, 18, 20, 23], "intro": [17, 22, 23], "introduc": [2, 3, 7, 8, 10, 12, 14, 18, 20, 23], "introduct": [3, 4, 6, 15, 22, 24], "introductori": [2, 6, 18, 22, 23, 24], "intuit": [2, 7, 8, 10, 14, 15, 23], "inv": [2, 7, 15, 23, 24], "invalu": [2, 15, 17, 23], "invari": 3, "invd": 7, "inver": 10, "invers": [2, 5, 8, 15, 23, 24], "inverse_transform": 10, "invert": [1, 2, 7, 9, 12, 15, 23], "invh": 15, "invok": 10, "involv": [2, 4, 8, 9, 13, 14, 23, 24], "io": [2, 17, 19, 21, 22, 23, 24], "ip": [2, 10, 20, 23], "ipca": 13, "ipynb": [17, 23], "ipython": [2, 7, 9, 11, 13, 16, 17, 23, 24], "iq": 8, "iri": [10, 11], "irreduc": 8, "irrelev": [7, 24], "irrespect": [2, 23], "isn": 7, "isnul": [], "isomap": 13, "issu": [0, 3, 11, 18], "it_arrai": 15, "item": [2, 15, 23], "items": [18, 23], "iter": [3, 4, 6, 8, 10, 15, 16, 20], "its": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 23], "itself": [7, 8, 14, 20, 23, 24], "j": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 13, 14, 15, 16, 18, 20, 22, 23, 24], "j1": 18, "j_": 8, "j_lasso_sk": 8, "j_ridge_sk": 8, "j_sk": 8, "jackknif": [8, 17, 23], "jacobian": [4, 15], "jason": 6, "jax": [17, 23], "jensen": [21, 23, 24], "jerom": 22, "ji": [14, 18], "jit": 15, "jj": [2, 7, 8, 23], "jk": [2, 3, 8, 14, 18, 23], "jl": [2, 23], "jm": 18, "jnp": 15, "job": [0, 4, 10, 12], "join": [2, 6, 8, 9, 11, 23], "joint": [6, 7], "judg": 15, "judgement": 8, "julia": [17, 18], "jump": 20, "junk": 6, "jupit": 23, "jupyt": [0, 1, 2, 17, 22, 23], "just": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 20, 23, 24], "justif": 2, "justifi": [5, 12], "k": [2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 23, 24], "k0": 9, "k1": 9, "kaggl": 8, "kappa_d": 20, "karl": [21, 23], "karush": 10, "katrin": [21, 23], "keep": [0, 2, 3, 6, 7, 8, 13, 15, 16, 18, 23, 24], "keepdim": [3, 8, 12, 18], "kei": [3, 5, 8, 14], "kept": [6, 8, 16], "kera": [2, 6, 17, 23], "kernel": [2, 3, 5, 17, 23, 24], "kernel_regular": [3, 5], "kernel_s": 6, "kernelpca": 13, "kev": [2, 23], "kevin": [22, 23], "keyword": [18, 23], "kfold": 8, "kg": 3, "ki": 18, "kick": [3, 15], "kiener": 4, "kilomet": 8, "kind": [2, 4, 5, 6, 10, 14, 15, 16, 23, 24], "kj": [8, 14, 18, 24], "kjm": [17, 23], "kkt": 10, "kl": 20, "km": [14, 23], "kmean": 16, "kmeanspoint": 16, "kn_k": 16, "know": [0, 1, 2, 3, 4, 7, 8, 10, 15, 17, 23, 24], "knowledg": [2, 17, 23], "known": [3, 5, 6, 7, 8, 9, 10, 11, 14, 18, 20, 22], "kondev": [2, 23], "kp": 20, "kpca": 13, "kroneck": 16, "kuhn": 10, "kvalsund": [21, 23], "kwown": [2, 23], "l": [2, 3, 4, 5, 7, 8, 9, 10, 12, 13, 14, 15, 18, 20, 23], "l0": 9, "l1": [2, 3, 5, 9, 23], "l1_l2": [3, 5], "l1regl": 7, "l2": [3, 5], "l_": 18, "l_1": 9, "l_2": [9, 15], "l_j": 14, "la": 15, "la_i": 14, "la_k": 14, "lab": [17, 23], "label": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 20, 23, 24], "labelencod": [9, 12], "labels": [8, 10, 11], "labels_shuffl": [2, 3, 24], "laboratori": 19, "lack": [2, 23], "lagari": 4, "lagrang": [10, 13], "lambda": [2, 3, 4, 5, 7, 8, 9, 10, 12, 14, 15, 20, 23, 24], "lambda_": 13, "lambda_0": 13, "lambda_1": [7, 10, 13, 24], "lambda_2": [10, 13], "lambda_i": [10, 13], "lambda_iy_i": 10, "lambda_jy_iy_j": 10, "lambda_k": 10, "lambda_n": [7, 10, 24], "lamda": 3, "land": 10, "landmark": 10, "landscap": 15, "langl": [2, 8, 13, 20, 23, 24], "languag": [2, 3, 6, 10, 17, 18, 22, 23], "lapack": [18, 23], "laplac": 7, "laptop": [0, 17], "larg": [2, 3, 4, 6, 7, 8, 10, 11, 12, 13, 15, 17, 18, 20, 22, 23, 24], "larger": [2, 5, 7, 8, 10, 12, 13, 15, 20, 23, 24], "largest": [6, 10, 13], "lasso": [2, 9, 17, 23], "lasso_sk": 8, "last": [1, 2, 3, 5, 6, 7, 8, 9, 10, 14, 18, 20, 21, 23], "latent": 6, "latent_dim": 6, "latent_point": 6, "latent_space_value_rang": 6, "later": [0, 2, 3, 6, 9, 10, 14, 15, 16, 17, 23], "latest": [0, 6, 17], "latest_checkpoint": 6, "latex": 23, "latter": [2, 5, 8, 9, 10, 13, 15, 18, 20, 23, 24], "lattic": 14, "law": 2, "layer": [2, 6, 15, 23], "lbfg": [9, 11, 12], "lcc": [7, 8], "lda": 13, "ldot": [2, 8, 13, 23], "le": [7, 9, 12, 15, 20, 24], "lead": [1, 2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 20, 23, 24], "leaf": 11, "leaki": 3, "leakyrelu": 6, "lear": 15, "learn": [5, 6, 7, 8, 9, 10, 11, 12, 14, 18, 21, 22], "learnabl": 5, "learner": 12, "learnig": 23, "learning_r": [10, 12], "learning_rate_init": [2, 3, 23], "learning_schedul": 15, "least": [2, 9, 10, 12, 13, 17, 18, 20], "leat": 15, "leav": [2, 3, 5, 7, 8, 11, 13, 23], "lectur": [2, 3, 7, 12, 13, 14, 15, 17, 18, 19, 21, 22, 24], "lecturenot": [2, 17, 22, 23], "left": [1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "leftarrow": [10, 14], "legend": [0, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 23, 24], "leinonen": 23, "len": [1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 18, 23, 24], "length": [1, 2, 3, 5, 6, 10, 11, 15, 17, 23, 24], "length_of_sequ": 6, "leq": [2, 7, 9, 10, 15, 16, 20, 23, 24], "less": [2, 3, 5, 6, 7, 8, 10, 11, 15, 17, 20, 23, 24], "lessen": 3, "let": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "letter": [1, 2, 18, 20, 23, 24], "level": [2, 3, 7, 8, 11, 17, 18, 19, 21, 23], "li": [10, 13], "lib": [], "liblinear": 12, "librari": [2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 18, 20, 22, 24], "licens": [2, 3, 17, 23], "lie": [2, 8, 13, 20, 23, 24], "life": [2, 3, 10, 14, 23], "lifetim": 15, "like": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 17, 18, 20, 23, 24], "likelihood": [2, 3, 7, 11, 23, 24], "lim_": 20, "limit": [2, 7, 8, 10, 14, 18, 23, 24], "lin_clf": 10, "lin_model": [], "lin_reg": 11, "linalg": [2, 4, 7, 8, 10, 13, 15, 18, 20, 23, 24], "line": [0, 1, 2, 5, 8, 10, 13, 15, 23], "line1": 10, "line2": 10, "line3": 10, "line_model": 0, "line_ms": 0, "line_predict": 0, "linear": [1, 3, 5, 7, 8, 9, 11, 12, 13, 14, 17, 20], "linear_model": [0, 1, 2, 7, 8, 9, 10, 11, 12, 13, 15, 23, 24], "linear_regress": 8, "linearli": [7, 24], "linearloc": [8, 15], "linearregress": [0, 1, 2, 8, 9, 11, 23, 24], "linearsvc": 10, "liner": [3, 5], "linerar": 12, "linewidth": [2, 4, 6, 8, 10, 11, 12], "link": [0, 2, 6, 11, 14, 17, 19, 21, 23], "linlag": 7, "linpack": [18, 23], "linreg": [2, 23], "linspac": [1, 2, 4, 5, 6, 8, 10, 11, 12, 15, 18, 20, 23, 24], "linu": 6, "linux": [2, 3, 17, 23], "liquid": [2, 23], "list": [0, 3, 4, 5, 6, 11, 17, 23], "listedcolormap": [11, 12], "literatur": [3, 9, 16, 22], "littl": [3, 5, 11, 14], "live": [1, 10], "ll": [2, 20, 23, 24], "lle": [2, 24], "lloyd": [6, 16], "lmb": [2, 4, 7, 8, 24], "lmbd": [2, 3, 5, 23], "lmbd_val": [2, 3, 5, 23], "lmbda": 15, "ln": [3, 15], "load": [3, 6, 8, 9, 11, 12], "load_boston": [], "load_breast_canc": [3, 9, 11, 12, 13], "load_data": [5, 6], "load_digit": [3, 5], "load_iri": [10, 11], "loc": [5, 8, 9, 10, 11, 12, 23], "local": [0, 2, 3, 5, 9, 14, 15, 24], "locat": [0, 4, 5, 10], "log": [0, 2, 3, 4, 6, 7, 8, 9, 11, 12, 13, 15, 18, 23], "log10": [2, 7, 8, 24], "log_": [2, 23], "log_clf": 12, "logarithm": [2, 7, 9, 18, 23], "logic": [2, 3, 11, 23], "login": 0, "logist": [2, 3, 4, 10, 11, 12, 13, 14, 15, 17, 24], "logisticregress": [9, 11, 12, 13], "logit": 9, "logreg": [9, 11, 12, 13], "logspac": [2, 3, 5, 7, 8, 23, 24], "long": [2, 3, 5, 6, 14, 15, 23], "longer": [4, 5, 10, 12, 16, 18, 20, 23], "loocv": 8, "look": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 18, 20, 23, 24], "loop": [1, 3, 6, 8, 12, 14, 16, 17, 18, 23], "lose": 3, "loss": [2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 15, 18, 23], "loss_fil": 6, "lossfil": 6, "lost": 6, "lot": [1, 3, 6, 8], "low": [2, 8, 11, 12, 13, 23, 24], "lower": [1, 2, 3, 5, 8, 11, 12, 18, 24], "lowercas": [18, 23], "lowest": [11, 15, 20], "lr": [3, 5, 6, 12], "lstat": [], "lstm": 6, "lstm_2layer": 6, "lstsq": [2, 23, 24], "lt": 8, "lu": [2, 7, 23, 24], "lubksb": 18, "luckili": 4, "ludcmp": 18, "lux": 18, "lvert": 3, "lw": [2, 23], "m": [0, 2, 3, 4, 5, 7, 8, 10, 11, 12, 13, 14, 15, 18, 20, 21, 22, 23, 24], "m_": [11, 14], "m_1": 16, "m_h": [2, 23], "m_k": 16, "m_l": 14, "m_n": [2, 23], "m_p": [2, 23], "m_t": 15, "ma": 13, "machin": [0, 1, 3, 5, 6, 7, 8, 9, 11, 12, 13, 14, 18, 22, 24], "machinelearn": [1, 2, 8, 17, 19, 21, 22, 23], "mackai": 22, "made": [2, 3, 5, 6, 7, 8, 9, 11, 13, 14, 23, 24], "mae": [2, 23], "magic": 6, "magnitud": [3, 8, 9, 15], "mai": [2, 3, 4, 5, 7, 8, 9, 10, 11, 13, 14, 15, 17, 18, 20, 23, 24], "mail": [19, 21], "main": [2, 3, 5, 6, 7, 8, 9, 11, 18, 22, 24], "mainli": [2, 7, 8, 9, 11, 23, 24], "maintain": 8, "major": [3, 8, 11, 12, 15, 18, 23], "make": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 13, 14, 15, 17, 18, 20, 22, 23], "make_axes_locat": 8, "make_moon": [10, 11, 12], "make_pipelin": [2, 8, 12, 24], "makedir": [2, 8, 9, 11, 23], "malcondit": 18, "malign": [3, 9, 11], "mammographi": 7, "manag": [0, 2, 4, 5, 17, 23], "mandatori": [21, 23], "mani": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 15, 16, 17, 18, 20, 22, 23, 24], "manifold": 13, "manner": 5, "manual": 8, "map": [2, 3, 4, 8, 9, 10, 13, 14, 16, 20, 23], "marc": 24, "margin": [2, 7, 10], "marit": [2, 23], "mark": 23, "marker": [9, 18, 23], "markov": [17, 23], "marsaglia": 20, "mass": [2, 3, 7, 15, 24], "massag": [2, 23], "masses2016": [2, 23], "masses2016ol": [2, 23], "masses2016tre": 2, "masseval2016": [2, 23], "master": [19, 21], "mat": [17, 23], "mat1100": [17, 23], "mat1110": [17, 23], "mat1120": [17, 23], "match": [0, 3, 6, 7, 15, 16, 24], "materi": [0, 6, 7, 9, 15, 18, 19, 21, 24], "math": [5, 9, 14, 15, 18, 20, 22, 23], "mathbb": [2, 6, 7, 8, 9, 10, 13, 14, 15, 16, 18, 20, 23, 24], "mathbf": [2, 7, 8, 9, 10, 15, 18, 23, 24], "mathcal": [3, 7, 8, 9, 15], "matheemat": 5, "mathemat": [2, 8, 13, 14, 15, 17, 18, 20, 22, 23], "mathemati": 23, "mathrm": [2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 23, 24], "matmul": [3, 4, 7], "matnat": 22, "matplotlib": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 23, 24], "matric": [1, 2, 3, 5, 6, 8, 9, 10, 13, 15, 17, 24], "matrix": [2, 4, 5, 6, 8, 9, 10, 12, 15, 20], "matshow": 3, "matter": [4, 5, 15, 24], "max": [2, 3, 4, 5, 6, 11, 12, 14, 15, 21, 23], "max_depth": [2, 11, 12], "max_diff": 4, "max_diff1": 4, "max_diff2": 4, "max_it": [2, 3, 10, 15, 23], "max_iter": 16, "max_leaf_nod": 12, "max_sampl": 12, "maxdegre": [2, 8, 12, 24], "maxdepth": 12, "maxim": [3, 6, 7, 9, 10, 13], "maximum": [2, 4, 5, 7, 9, 10, 11, 12, 15, 16, 23, 24], "maxpolydegre": [7, 8, 24], "maxpooling2d": 5, "mbox": [7, 8, 24], "mcculloch": 14, "md": 13, "mdoel": 6, "mean": [0, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 20, 23], "mean_absolute_error": [2, 23], "mean_divisor": 16, "mean_i": 20, "mean_matrix": 16, "mean_squared_error": [0, 2, 6, 8, 9, 12, 23, 24], "mean_squared_log_error": [2, 23], "mean_vector": 16, "mean_x": 20, "meaning": [2, 6, 9, 23], "meansquarederror": [2, 23], "meant": [5, 9, 12, 15], "measur": [1, 2, 3, 4, 7, 8, 11, 13, 14, 16, 20, 23, 24], "mechan": [2, 6, 20, 23], "median": [2, 23, 24], "medicin": 14, "medium": [6, 10, 15], "medv": [], "meet": [2, 21], "mehta": [2, 23, 24], "memori": [5, 6, 13, 14, 15, 18], "mention": [2, 14, 15, 20, 23], "mere": 2, "meshgrid": [4, 7, 8, 10, 11, 12, 13], "mess": 0, "messag": [7, 15], "messi": 4, "met": [2, 5, 10, 24], "meteorolog": 11, "meter": 8, "method": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 13, 14, 16, 17, 18, 20, 22, 24], "metion": 8, "metric": [0, 2, 3, 5, 8, 9, 11, 12, 16, 23, 24], "metropoli": [17, 23], "mev": [2, 20, 23], "mgd": 15, "mglearn": [17, 23], "mgrid": 15, "mhjensen": [], "mi": 12, "mia": [21, 23], "microsoft": 22, "mid": 3, "midel": 6, "midnight": 0, "midpoint": 11, "might": [0, 2, 3, 4, 6, 8, 11, 15, 24], "mild": 11, "millimet": 8, "million": [2, 23, 24], "mimic": 14, "min": [2, 4, 7, 10, 11], "min_": [2, 4, 7, 16, 23, 24], "min_samples_leaf": 11, "mind": [0, 2, 8, 15, 23, 24], "mindboard": 6, "mine": [17, 23], "mini": [3, 13, 14, 15], "minibatch": [3, 13, 15], "minibathc": 15, "miniforge3": [], "minim": [1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 24], "minima": [2, 3, 9, 15, 23], "minimum": [2, 3, 4, 8, 10, 11, 13, 15, 24], "minmaxscal": [2, 24], "minor": 20, "minst": 3, "minu": 9, "mirjalili": 23, "mirror": 11, "misc": 8, "misclassif": [10, 11, 12], "misclassifi": [10, 12], "miser": 2, "mismatch": 3, "miss": [9, 12], "mistak": 6, "mit": 22, "mix": [3, 4, 23], "mixtur": 15, "mk": [11, 18], "mkdir": [2, 8, 9, 11, 23], "ml": [2, 3, 12, 15, 18, 24], "mlab": 20, "mle": [7, 9], "mlp": 3, "mlpclassifi": 3, "mlpregressor": [2, 23], "mm": 18, "mml": 24, "mn": [14, 20], "mnist": [3, 13], "mod": 20, "mode": [19, 21, 23], "model": [1, 4, 5, 7, 9, 10, 11, 12, 13, 15, 16, 17, 20, 22, 24], "model_select": [0, 1, 2, 3, 5, 7, 8, 9, 11, 12, 13, 23, 24], "moder": 12, "modern": [2, 8, 9, 17, 23], "modif": [4, 14, 15], "modifi": [2, 3, 5, 7, 9, 10, 12, 14, 15, 23, 24], "modul": [1, 2, 18, 23], "modular": 20, "modulo": 20, "moe": [13, 24], "moment": [7, 8, 15, 20], "mondai": [21, 23], "monitor": 15, "monoton": [7, 14, 20], "mont": [2, 8, 17, 20, 22, 23], "montli": 1, "moor": [7, 8], "more": [1, 2, 3, 4, 6, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 20], "moreov": [2, 5], "morten": [21, 23, 24], "mortenhj": 23, "most": [0, 1, 2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 20, 23, 24], "mostli": [3, 13], "motion": [2, 15], "motiv": [3, 6], "move": [0, 1, 2, 6, 7, 8, 9, 11, 14, 15, 16, 20, 24], "mpl": [9, 23], "mpl_toolkit": [4, 8, 15], "mplot3d": [4, 8, 15], "mplregressor": 3, "mse": [0, 1, 2, 6, 7, 8, 11, 12, 23, 24], "mse_simpletre": 12, "mselassopredict": 7, "mselassotrain": 7, "mseownridgepredict": [8, 24], "msepredict": 7, "mseridgepredict": [2, 7, 8, 24], "msetrain": 7, "msle": [2, 23], "mt": [9, 14], "mu": [2, 8, 13, 15, 20, 23], "mu0": 20, "mu1": 20, "mu2": 20, "mu_": [8, 20], "mu_i": 8, "mu_n": 13, "mu_x": 20, "much": [0, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 18, 20, 23, 24], "multi": [2, 3, 5, 9, 17, 23], "multiclass": [3, 9], "multidimension": [13, 14, 23], "multilay": 3, "multinomi": 9, "multipl": [0, 4, 6, 7, 8, 9, 14, 15, 20, 24], "multipli": [5, 7, 8, 13, 15, 18, 20, 24], "multiplum": 10, "multivari": [2, 4, 12, 13, 17, 20, 23], "multivariate_norm": [13, 16], "multpli": 1, "murphi": [13, 22, 23], "must": [0, 3, 4, 7, 8, 10, 12, 14, 15, 16, 20, 24], "mutat": 9, "mutual": [3, 5, 8, 15], "mx_": 20, "my": 23, "myenv": [], "myriad": [2, 17, 23], "mz1": 20, "mz2": 20, "m\u00f8svatn": 8, "n": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 18, 20, 23, 24], "n1": 18, "n2": 18, "n_": [3, 4, 5, 10, 14, 20], "n_0": [14, 20], "n_boostrap": [8, 12], "n_bootstrap": 8, "n_categori": [3, 5], "n_cluster": 16, "n_compon": 13, "n_epoch": 15, "n_estim": 12, "n_examples_to_gener": 6, "n_featur": 3, "n_filter": 5, "n_hidden": 4, "n_hidden_neuron": [2, 3, 23], "n_i": 20, "n_input": [2, 3, 5, 24], "n_instanc": 11, "n_job": 12, "n_k": 16, "n_l": [14, 20], "n_layer": 3, "n_m": 11, "n_neuron": 3, "n_neurons_connect": 5, "n_neurons_layer1": 3, "n_neurons_layer2": 3, "n_point": 16, "n_sampl": [8, 10, 11, 12, 16], "n_split": 8, "n_step": 6, "n_t": 4, "n_x": 4, "nabla": [3, 15], "nabla_": [4, 15], "nabla_w": 15, "nag": 15, "naimi": [2, 23], "naiv": 9, "naive_kmean": 16, "name": [0, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 20, 21, 23, 24], "narrow": 15, "nation": [3, 7], "nativ": [17, 23], "natur": [2, 3, 6, 10, 11, 14, 15, 20, 22, 23], "navier": 14, "navig": 0, "nb": 20, "nb_": 18, "nbconvert": 23, "nd": 16, "ndarrai": 8, "ne": [11, 12, 18, 20, 24], "nearest": [3, 5, 8, 13], "nearli": 15, "neat": 23, "neccesari": 8, "necess": 4, "necessari": [2, 3, 5, 6, 10, 16, 23], "necessarili": [2, 6, 13, 20, 23], "necesserali": 7, "neck": 9, "need": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 24], "neg": [2, 3, 5, 7, 8, 9, 12, 15, 18, 20, 23], "neg_mean_squared_error": 8, "neglect": 20, "neglig": 20, "neighbor": [5, 8, 13], "neither": [6, 15], "neq": [15, 16, 20], "nervou": 14, "nest": [11, 14], "nesterov": 15, "net": [4, 6, 14], "netlib": [18, 23], "network": [2, 11, 15, 17, 22, 24], "neural": [2, 15, 17, 22, 24], "neural_network": [2, 3, 4, 23], "neuralnetwork": 3, "neuron": [3, 4, 5, 6, 14], "neutral": [2, 23], "neutron": [2, 23], "never": [3, 6, 8, 11, 20], "new": [0, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 23, 24], "new_chang": 15, "new_hobbit": 23, "newaxi": [2, 5, 8, 11], "newli": [2, 23], "newton": [3, 9, 10, 15, 20], "next": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 15, 16, 23, 24], "next_guess": 15, "next_input": 6, "ng": 3, "ni": 16, "nice": [2, 3, 7, 13, 23, 24], "niter": 15, "nitric": [], "nlambda": [2, 7, 8, 24], "nlp": 22, "nm": 20, "nm_n": [2, 23], "nmse": 8, "nn": [4, 7, 8, 14, 18, 23], "nn_model": 3, "nnmin": 4, "node": [3, 5, 11, 12, 14], "nois": [2, 6, 7, 8, 10, 11, 12, 15, 23, 24], "noise_dimens": 6, "noisi": [3, 8], "non": [2, 3, 5, 7, 8, 9, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "none": [2, 3, 4, 6, 7, 11, 12, 15, 20, 23, 24], "nonlinear": [5, 8, 10, 11, 13, 14], "nonneg": [8, 11, 15], "nonparametr": 8, "nonsens": 20, "nonsingular": 18, "nonumb": [5, 9, 10, 15, 18], "nor": [3, 6, 15], "norm": [2, 3, 7, 8, 10, 13, 15, 23, 24], "normal": [1, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 20, 23, 24], "normali": [18, 23], "norwai": [8, 23], "notat": [2, 4, 7, 8, 15, 16, 20, 23, 24], "note": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 14, 15, 16, 17, 18, 20, 22, 23], "notebook": [0, 1, 2, 3, 5, 11, 17, 23], "noth": [3, 4, 7, 10, 14, 16, 20, 24], "notic": [6, 7, 14, 15, 18, 20, 23], "notion": 5, "novel": [5, 8, 12, 23], "novemb": [3, 21, 23], "now": [0, 1, 2, 4, 6, 7, 8, 9, 10, 12, 13, 14, 16, 17, 18, 20, 23, 24], "nowadai": [2, 3, 5, 11, 17, 23], "nox": [], "np": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 20, 23, 24], "npr": 4, "nsampl": 8, "nt": 4, "nu": 20, "nuclear": [7, 24], "nuclei": [2, 20, 23], "nucleon": [2, 23], "nucleu": [2, 23], "num": 6, "num_coordin": 4, "num_hidden_neuron": 4, "num_it": 4, "num_neuron": 4, "num_neurons_hidden": 4, "num_point": 4, "num_tre": 12, "num_valu": 4, "number": [1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 21, 23], "numberid": 9, "numberparamet": 5, "numer": [2, 7, 8, 11, 12, 13, 14, 15, 17, 18, 22, 23, 24], "numpi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 20, 24], "nunmpi": [7, 24], "nx": 4, "ny": 20, "o": [2, 3, 6, 7, 8, 9, 10, 11, 13, 18, 21, 22, 23, 24], "obei": [8, 13, 15], "object": [0, 2, 3, 6, 10, 12, 18, 23], "obliqu": [7, 24], "observ": [2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 23], "obtain": [2, 3, 7, 8, 9, 10, 11, 12, 14, 15, 16, 18, 20, 23, 24], "obviou": [7, 8, 13, 20, 24], "obviouli": 23, "obvious": [2, 6, 7, 8, 18, 23], "oc": 24, "occupi": [], "occur": [2, 8, 10, 11, 18, 20, 23], "octob": [21, 23], "od": 2, "odd": [2, 5, 9, 23, 24], "odenum": 4, "odesi": 4, "oen": 2, "off": [3, 5, 6, 7, 11, 15, 20], "offer": [8, 13, 17, 18, 19, 21, 23], "offic": [21, 23], "offici": [19, 23], "often": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 23, 24], "ofter": [18, 23], "ol": [2, 15, 24], "old": [0, 3, 7, 12, 15], "ols_paramet": 1, "ols_sk": 8, "ols_svd": 8, "olstheta": [2, 7], "omega": [4, 5, 8], "omega_0": 5, "omit": [2, 7, 23, 24], "onc": [3, 8, 11, 13, 15], "one": [0, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 20, 21, 23, 24], "onehot": 3, "onehot_vector": 3, "onehotencod": 11, "ones": [1, 2, 4, 7, 8, 10, 11, 12, 13, 15, 18, 23, 24], "ones_lik": 6, "ong": 24, "onl": 5, "onli": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "onlin": [0, 13, 19], "onto": [7, 13, 24], "open": [0, 2, 3, 6, 8, 9, 11, 17, 19, 21, 23], "oper": [0, 1, 2, 3, 5, 7, 8, 12, 13, 14, 15, 17, 20, 23, 24], "operation": 20, "oplu": 20, "opmiz": 15, "opportun": 2, "oppos": [8, 15], "opposit": [3, 7, 10, 24], "opt": [3, 7, 23], "optim": [1, 2, 4, 5, 6, 7, 8, 9, 11, 12, 13, 16], "optimis": [3, 5], "option": [0, 2, 3, 5, 7, 8, 10, 13, 18, 24], "optmiz": [3, 10, 15, 24], "oral": 23, "orang": 2, "order": [0, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 18, 20, 23, 24], "ordinari": [2, 4, 5, 9, 13, 15, 17], "oreilli": [22, 23], "org": [1, 2, 5, 6, 17, 18, 22, 23], "organ": [8, 9, 12, 18], "orient": [3, 7, 20, 24], "origin": [0, 2, 5, 7, 8, 10, 13, 14, 15, 18, 23, 24], "orthogn": [7, 24], "orthogon": [2, 7, 8, 10, 13, 15, 18, 23, 24], "orthonorm": [7, 24], "os": [21, 23], "oscar": 3, "oscil": [5, 15], "oskar": 23, "oskarlei": 23, "oslo": [2, 17, 19, 21, 23, 24], "osx": [2, 17, 23], "other": [1, 2, 3, 4, 5, 7, 8, 9, 10, 12, 15, 16, 17, 19, 20, 21, 22, 24], "otherwis": [2, 3, 6, 9, 15, 18, 23], "ouput": [7, 9, 14], "our": [0, 1, 3, 4, 5, 8, 9, 10, 11, 12, 14, 16, 17, 18, 20], "ourmodel": 2, "ourselv": [2, 7, 8, 10, 13, 15, 23, 24], "out": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 20, 23, 24], "out_fil": 11, "outcom": [2, 9, 11, 12, 14, 20, 24], "outdoor": 11, "outer": [8, 14, 15], "outfil": 6, "outlier": [2, 10, 23, 24], "outlin": [8, 12, 13], "outlook": 11, "outperform": 12, "output": [2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 18, 20, 23, 24], "output_bia": 3, "output_bias_gradi": 3, "output_shap": 6, "output_weight": 3, "output_weights_gradi": 3, "outputlayer1": 14, "outputlayer2": 14, "outsid": 6, "over": [0, 1, 2, 3, 5, 6, 7, 8, 11, 12, 14, 15, 18, 23, 24], "over1": 15, "overal": [3, 12], "overcast": 11, "overcom": [14, 15], "overdetermin": [2, 23], "overfit": [2, 3, 5, 8, 11, 12, 15], "overflow": 7, "overhead": 14, "overlap": [5, 9, 10, 11], "overlin": [2, 7, 8, 11, 12, 13, 16, 18, 23, 24], "overst": 2, "overtrain": 6, "overview": 5, "own": [1, 6, 7, 8, 10, 14, 15, 17, 18], "owner": [], "ownmsepredict": 2, "ownmsetrain": 2, "ownridgetheta": [2, 8, 24], "ownypredictridg": 2, "ownytilderidg": 2, "oxid": [], "p": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 20, 23, 24], "p0": 4, "p1": 4, "p_": [4, 6, 10, 11], "p_hidden": 4, "p_i": [7, 20], "p_j": 20, "p_n": 20, "p_output": 4, "p_x": 20, "pack": [2, 23], "packag": [0, 2, 3, 5, 6, 7, 10, 13, 15, 17, 20, 24], "packtpub": 23, "packtpublish": 23, "pad": [5, 6], "page": [2, 17, 23], "pai": [0, 2, 3, 11, 15], "pair": [2, 4, 5, 11, 17, 20, 23], "paltform": 0, "panda": [2, 6, 7, 8, 9, 11, 13, 17], "panel": 23, "paper": 3, "paradigm": [2, 23], "parallel": [12, 15, 17, 18, 23], "param": 4, "paramat": 4, "paramet": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 20], "parameter": [2, 8, 12, 23, 24], "parametr": [2, 8, 23, 24], "paramt": [5, 7], "part": [2, 3, 5, 7, 8, 12, 18, 19, 20, 21, 23, 24], "partial": [1, 2, 3, 7, 8, 9, 10, 12, 13, 14, 15, 20, 23, 24], "particip": [0, 17, 19, 21, 23], "particl": [2, 6, 15, 20, 23], "particular": [1, 2, 3, 4, 5, 7, 8, 11, 12, 13, 14, 15, 20, 22, 23, 24], "particularli": [7, 8, 10, 13, 15, 20, 24], "partit": [3, 6, 11], "partli": [8, 23], "partner": 0, "pass": [4, 5, 14, 16], "past": [12, 20], "patch": [8, 20], "path": [2, 6, 8, 9, 11, 17, 23], "patient": 9, "patter": 6, "pattern": [2, 5, 6, 14, 22, 23], "pauli": [2, 23], "pc": [0, 13, 17], "pca": [2, 9, 17, 23, 24], "pd": [2, 6, 7, 8, 9, 11, 13, 23, 24], "pde": 4, "pdf": [0, 1, 2, 5, 6, 7, 8, 11, 22, 23], "pedagog": [2, 23, 24], "penal": 8, "penalti": [8, 15], "penros": [7, 8], "pentagon": 15, "peopl": [3, 11, 15, 17], "per": [2, 3, 8, 19, 21, 23], "percentag": [12, 13, 21], "perceptron": [2, 3, 9, 23], "peregrin": 23, "perfect": [2, 3, 15, 23], "perfectli": [6, 8], "perform": [1, 2, 4, 5, 6, 7, 8, 10, 12, 13, 14, 15, 16, 17, 18, 20, 23, 24], "performac": 6, "perhap": [2, 7, 15, 23, 24], "perimet": 3, "period": [3, 6, 20], "permiss": 0, "permut": 13, "persist": 15, "person": [1, 7, 8, 9, 19, 21, 23], "perspect": 22, "pertin": [14, 23], "petal": [10, 11], "peter": [22, 24], "phantom": 20, "phase": [8, 14], "phenomena": 20, "phi": 10, "phi_k": 10, "philosophi": 15, "phone": [21, 23], "photo": [6, 23], "phrase": [2, 23], "physic": [2, 3, 6, 9, 14, 15, 20, 21, 22, 23, 24], "pi": [4, 5, 7, 8, 9, 11, 14, 15, 20], "pick": [3, 11, 12, 13, 15, 16], "pickl": 3, "pictur": [2, 23], "pie": [17, 23], "piec": [13, 16], "pillow": [2, 17, 23], "pinv": [7, 8, 15, 24], "pip": [0, 2, 3, 17, 23], "pip3": [2, 3, 23], "pipelin": [2, 8, 10, 12, 24], "pippin": 23, "pit": 6, "pitfal": 8, "pitt": 14, "pixel": [3, 5, 6, 23], "pixel_height": [3, 5], "pixel_width": [3, 5], "place": [0, 2, 6, 8, 10, 15, 18, 23], "plai": [2, 5, 6, 7, 8, 10, 13, 17, 23, 24], "plain": [10, 12, 14, 15, 16], "plan": [8, 11, 21, 22, 23], "plane": [10, 11], "plateau": 7, "platform": [17, 23], "plausibl": 14, "pleas": [15, 21, 23], "plenti": 3, "plethora": [5, 14], "plot": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 23, 24], "plot_confusion_matrix": [9, 12], "plot_count": 8, "plot_cumulative_gain": [9, 12], "plot_data": 3, "plot_dataset": 10, "plot_decision_boundari": [11, 12], "plot_import": 12, "plot_max": 6, "plot_min": 6, "plot_model": 6, "plot_numb": 6, "plot_predict": 10, "plot_regression_predict": 11, "plot_result": 6, "plot_roc": [9, 12], "plot_surfac": [4, 8, 15], "plot_train": 11, "plot_tre": [11, 12], "plt": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "plu": [2, 5, 7, 9, 23, 24], "pm": 10, "pmatrix": 4, "pml": 22, "pn": 5, "png": [2, 6, 8, 9, 11, 23], "point": [2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 20, 21, 23, 24], "point_1": 6, "point_2": 6, "poisson": [17, 20, 23], "poli": [8, 10], "poly100_kernel_svm_clf": 10, "poly3": 2, "poly3_plot": 2, "poly_featur": [0, 10, 11], "poly_features10": 11, "poly_fit": 11, "poly_fit10": 11, "poly_kernel_svm_clf": 10, "poly_model": 0, "poly_ms": 0, "poly_predict": 0, "polydegre": [2, 7, 8, 12, 24], "polygon": 15, "polym": 14, "polynomi": [0, 2, 7, 8, 9, 10, 11, 12, 13, 23, 24], "polynomial_featur": [0, 1, 8], "polynomial_svm_clf": 10, "polynomialfeatur": [0, 1, 2, 8, 10, 11, 24], "polytrop": [2, 8], "pool": 5, "pool_siz": 5, "poor": [3, 15], "poorli": [2, 24], "popul": [2, 7, 23, 24], "popular": [0, 2, 3, 5, 8, 9, 10, 11, 13, 14, 17, 18, 20, 24], "popularli": [2, 23], "portabl": 12, "portion": [13, 15], "pose": [2, 6, 7, 8, 13, 20, 23], "posit": [2, 3, 4, 5, 7, 9, 10, 12, 13, 15, 16, 18, 20, 23, 24], "possibl": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 20, 21, 23, 24], "possibli": [8, 10, 15], "posterior": 7, "postpon": [2, 24], "postul": 7, "potenti": [2, 5, 7, 8, 14, 15], "pott": 14, "power": [2, 3, 7, 8, 10, 11, 14, 15, 23, 24], "pp": [7, 8], "practic": [1, 2, 7, 8, 9, 10, 20, 24], "practition": [2, 3, 5, 23], "pre": 23, "preced": [3, 13, 14, 20], "preceed": 6, "preceq": 10, "precis": [2, 4, 7, 13, 15, 18, 20, 23, 24], "pred": 8, "predicit": 2, "predict": [0, 1, 2, 3, 7, 8, 9, 10, 11, 12, 17, 22, 23, 24], "predict_prob": 3, "predict_proba": [9, 12], "predictor": [2, 7, 8, 9, 11, 12, 13, 23, 24], "prefer": [0, 2, 3, 8, 10, 11, 13, 15, 17, 23], "prepar": [2, 8, 18, 23, 24], "preprocess": [0, 1, 2, 6, 8, 9, 10, 11, 12, 13], "prerequisit": 2, "presenc": 15, "present": [2, 7, 8, 9, 11, 14, 15, 18, 20, 23, 24], "preserv": [5, 13, 18], "press": [0, 15, 22], "pretrain": [3, 6], "pretti": [2, 6, 10, 11, 17, 23], "prev_centroid": 16, "prevent": [15, 20], "previou": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14, 15, 18, 20, 24], "previous": [4, 5, 11, 12, 20], "price": [2, 6, 11, 15], "primal": 10, "primari": [2, 9, 23], "prime": 20, "princip": [2, 7, 9, 17, 23, 24], "principl": [2, 8, 9, 10, 16, 23], "print": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 20, 23, 24], "print_funct": [10, 11], "printout": [2, 23], "prior": [2, 7, 8, 23], "privat": 2, "prob": [3, 20], "probabilist": [2, 22, 23, 24], "probabl": [2, 3, 5, 6, 8, 9, 12, 15, 17, 23, 24], "problem": [2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 18, 20], "probml": 22, "proce": [2, 7, 8, 9, 10, 11, 12, 13, 15, 18, 23, 24], "procedur": [4, 6, 7, 8, 10, 12, 13, 15, 24], "proceed": 18, "process": [2, 4, 6, 8, 11, 12, 14, 15, 17, 18, 20, 22, 23], "prod": 22, "prod_": [3, 7, 9], "produc": [2, 5, 6, 7, 8, 11, 12, 13, 14, 15, 17, 18, 20, 23, 24], "product": [1, 2, 3, 5, 7, 8, 9, 10, 14, 15, 17, 18, 23, 24], "profess": [2, 23], "program": [0, 2, 3, 6, 7, 8, 10, 14, 16, 17, 18, 19, 20, 21, 23, 24], "programm": 18, "progress": [3, 6, 16], "prohibit": 8, "project": [0, 2, 3, 4, 5, 7, 13, 15, 17, 19, 24], "project_root_dir": [2, 8, 9, 11, 23], "promin": 14, "promis": 10, "promot": [21, 23], "prone": [0, 11], "pronounc": [15, 17, 23], "proof": [2, 13, 14, 15, 23], "propag": [4, 5, 15], "proper": [2, 4, 8, 9], "properli": [3, 8, 10, 12, 15], "properti": [1, 2, 3, 5, 14, 15, 18, 23], "proport": [2, 3, 7, 11, 13, 15, 20, 23, 24], "propos": [3, 6, 8, 12, 23], "propto": [7, 15], "proton": [2, 23], "prove": [5, 15], "provid": [2, 3, 5, 6, 7, 8, 10, 11, 12, 14, 15, 17, 18, 20, 23, 24], "proxi": [3, 15], "prune": 11, "pseudo": [18, 20], "pseudoinv": 7, "pseudoinvers": [7, 8], "pseudorandom": [8, 20], "psychologi": [2, 23], "pt": 15, "public": [0, 2, 17, 23], "pull": 0, "punish": [2, 3, 23], "pure": [5, 11, 20], "purest": 11, "puriti": 11, "purpos": [2, 5, 12, 14, 16, 23], "push": 0, "put": 3, "py": 7, "pycod": 23, "pydata": 17, "pydot": 11, "pyhton2": 23, "pylab": [9, 23], "pypi": 17, "pyplot": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "pythagora": 7, "python": [3, 4, 5, 7, 8, 10, 13, 14, 15, 16, 20, 24], "python2": 2, "python3": [2, 17, 23], "pytorch": [2, 17, 23], "q": [7, 8, 10, 13, 20], "qp": 10, "qquad": [4, 13, 15, 18], "qr": [7, 8, 18, 24], "quad": [3, 15, 18], "quadrat": [2, 10, 11, 15, 23], "qualit": [6, 11, 20], "qualiti": [2, 11, 17, 23, 24], "quantifi": 3, "quantil": 12, "quantit": [2, 8, 11, 23], "quantiti": [1, 2, 4, 7, 8, 9, 11, 12, 13, 14, 16, 18, 20, 23, 24], "quantum": [6, 14, 22, 23], "quartil": [2, 24], "quench": 7, "queri": 11, "question": [2, 7, 8, 11, 13, 14, 15, 21, 23, 24], "qugan": 6, "quick": [6, 20], "quickli": [3, 5, 11, 13, 15], "quit": [0, 3, 7, 8, 11, 12, 14, 24], "quot": 6, "r": [0, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 24], "r2": [2, 7, 8, 23, 24], "r2_score": [2, 23], "r2score": [2, 23], "r_1": 11, "r_2": 11, "r_j": 11, "r_m": 11, "rad": [], "radial": [10, 14], "radioact": 20, "radiu": [2, 3, 24], "rain": 11, "ramp": 3, "ran0": 20, "ran1": 20, "ran2": 20, "ran3": 20, "rand": [0, 2, 6, 7, 8, 11, 12, 15, 18, 23, 24], "randint": [8, 11, 15], "randn": [0, 2, 3, 4, 7, 8, 11, 13, 15, 23, 24], "random": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 15, 16, 17, 18, 23, 24], "random_forest_model": 12, "random_index": 15, "random_indic": [3, 5], "random_st": [9, 10, 11, 12, 13], "randomforestclassifi": 12, "randomli": [3, 8, 11, 15, 16], "rang": [2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "rangl": [2, 8, 13, 20, 23, 24], "rangle_x": 20, "rank": [7, 24], "rankdir": 6, "raphson": [3, 10, 15], "rapidli": 2, "rare": [3, 15], "raschka": [23, 24], "rasckha": 23, "rate": [3, 4, 5, 6, 10, 11, 12, 14, 15], "rather": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 20, 23, 24], "ratio": [6, 9, 11, 12, 13], "rational": [2, 23], "ravel": [7, 8, 9, 10, 11, 12, 13, 15, 18], "raw": 5, "rbf": [10, 13, 14], "rbf_kernel_svm_clf": 10, "rbf_pca": 13, "rc": 20, "rcond": [2, 23, 24], "rcparam": [3, 5, 9, 10, 11, 12, 20, 23], "re": [0, 4, 6, 15], "reach": [3, 6, 7, 8, 11, 12, 14, 15, 16], "read": [1, 2, 4, 5, 6, 7, 8, 9, 10, 13, 14, 18, 20, 22], "read_csv": [2, 8, 9, 11], "read_fwf": [2, 23], "reader": [2, 8, 18, 20, 23], "readi": [2, 3, 7, 8, 10, 12, 13, 14, 18, 23], "readili": 3, "readm": 0, "readthedoc": 17, "real": [1, 2, 3, 6, 9, 12, 13, 14, 18, 24], "real_loss": 6, "real_output": 6, "realist": [10, 23], "realiti": 20, "realiz": [3, 14], "realli": [2, 3, 23], "rearrang": 15, "reason": [2, 3, 5, 6, 12, 15, 22, 23], "reassign": 3, "recal": [7, 8, 11, 12, 13, 14, 18, 20, 23, 24], "recast": 5, "receiv": [3, 5, 12, 14, 20], "recent": [2, 8, 15, 22], "recept": [5, 14], "receptive_field": 5, "recip": [2, 8, 9, 18, 23, 24], "reciproc": 7, "recogn": [2, 6, 7, 12, 23], "recognit": [2, 3, 5, 14, 22, 23], "recommen": 23, "recommend": [0, 2, 4, 5, 6, 7, 8, 10, 15, 17, 18, 22], "reconsid": 11, "reconstruct": 13, "record": [12, 19, 21, 23], "recreat": 0, "rectangl": [11, 15], "rectangular": [7, 24], "rectifi": [3, 5, 14], "recur": [2, 17, 23], "recurr": [2, 3, 17, 23], "recurs": [11, 17, 18, 23], "red": [2, 5, 6, 8, 10, 11], "redefin": [2, 12, 23, 24], "reduc": [3, 5, 7, 8, 11, 12, 13, 15, 23], "reduct": [2, 12, 13, 17, 20, 23, 24], "refer": [2, 3, 4, 5, 7, 8, 13, 14, 15, 16, 18, 22, 23, 24], "referenc": 4, "refin": 14, "refit": 8, "reflect": [2, 3, 6, 7, 20, 23], "refresh": [17, 23], "refreshprogrammingskil": 23, "reg": [12, 13], "regard": [3, 11, 15], "regardless": [1, 14], "region": [5, 6, 8, 11, 14], "regist": [8, 20], "reglasso": 7, "regr_1": [2, 11], "regr_2": [2, 11], "regr_3": [2, 11], "regress": [1, 3, 10, 13, 14, 17, 18], "regressor": [2, 9, 12], "regridg": [2, 7, 8, 24], "regular": [2, 5, 6, 7, 8, 9, 11, 15, 21, 23, 24], "regularli": 0, "reilli": [2, 22, 23], "reinforc": [2, 10, 17, 23], "reiter": 3, "reject": 9, "rel": [2, 6, 8, 9, 11, 14, 15, 20, 23, 24], "relat": [2, 3, 5, 6, 7, 13, 15, 16, 18, 20, 23, 24], "relationship": [2, 6, 11, 23], "relativeerror": [2, 23, 24], "releas": [3, 17, 23], "relev": [2, 3, 7, 9, 13, 17, 20, 23], "reli": [2, 8, 10], "reliabl": [9, 20], "relu": [5, 6, 23], "remain": [3, 4, 6, 8, 14, 18, 20], "remaind": 20, "reman": 4, "remark": 3, "rememb": [2, 10, 15, 18, 23], "remind": [2, 7, 13, 15, 18, 20], "remot": 0, "remov": [6, 7, 8, 24], "renam": 0, "render": [2, 23, 24], "reorder": [7, 9, 24], "reorgan": [2, 23], "repeat": [2, 3, 5, 6, 7, 8, 11, 12, 13, 15, 16, 18, 20, 23, 24], "repeated": 23, "repeatedli": [2, 8, 12, 15], "repet": 5, "repetit": [8, 23, 24], "rephras": 15, "replac": [2, 3, 5, 6, 7, 8, 12, 14, 16, 17, 23, 24], "replica": 8, "repo": 0, "report": 23, "repositori": [6, 23], "repres": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 20, 23, 24], "represent": [2, 3, 5, 8, 20, 23], "representd": 5, "reproduc": [0, 1, 2, 7, 8, 11, 14, 17, 20, 23, 24], "repuls": [2, 23], "request": [2, 15], "requir": [0, 2, 3, 5, 6, 7, 8, 10, 11, 13, 14, 15, 18, 23, 24], "res1": 4, "res2": 4, "res3": 4, "res_analyt": 4, "res_analytical1": 4, "res_analytical2": 4, "res_analytical3": 4, "resaml": 8, "resampl": [2, 9, 12, 17, 23, 24], "rescal": [2, 13, 14, 24], "rescu": 7, "reseach": 8, "research": [2, 6, 15, 17, 22, 23], "resembl": [8, 20], "reserv": [3, 7, 8, 20], "reshap": [2, 3, 4, 5, 6, 8, 10, 11, 12, 18, 23, 24], "residenti": [], "residu": [2, 7, 15, 23], "resiz": [7, 24], "resourc": 23, "respect": [1, 2, 3, 4, 5, 7, 8, 9, 10, 12, 13, 14, 15, 16, 20, 23, 24], "respond": 14, "respons": [2, 9, 11, 14, 23, 24], "rest": [2, 7, 24], "restat": [2, 14, 23], "restor": 6, "restored_discrimin": 6, "restored_gener": 6, "restrict": [2, 5, 11, 14, 23], "result": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 23], "retail": [], "retain": [7, 8, 24], "return": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 16, 18, 20, 23, 24], "return_data": 16, "return_sequ": 6, "return_x_i": 11, "reus": [3, 5, 8], "reveal": [2, 14, 23], "revers": [3, 18], "review": [17, 18], "revisit": 16, "revolut": 23, "reward": [2, 6, 23], "rewrit": [1, 2, 5, 7, 8, 9, 10, 12, 13, 14, 15, 18, 20], "rewritten": [4, 8, 10, 12, 20], "rewrot": 15, "rf": 12, "rgb": 5, "rgoj5yh7evk": 17, "rh": 8, "rho": [2, 12, 15], "rho_1": 12, "rho_2": 12, "rho_m": 12, "rich": [2, 23], "ride": 11, "rideclass": 11, "ridedata": 11, "ridg": [9, 13, 15, 17, 23], "ridge_sk": 8, "ridgetheta": 7, "right": [1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 14, 15, 16, 18, 20, 23, 24], "right_sid": 4, "rightarrow": [2, 3, 7, 8, 10, 13, 14, 15, 20, 23, 24], "rigor": [2, 23, 24], "ring": 8, "rise": [2, 23], "risk": [2, 15, 23], "rival": 6, "river": [], "rlm": 23, "rm": 20, "rmse": [], "rmsporp": 15, "rmsprop": [3, 5, 6, 15], "rnd_clf": 12, "rng": 20, "rnn": [6, 14], "rnn1": 6, "rnn2": 6, "rnn_2layer": 6, "rnn_input": 6, "rnn_output": 6, "rnn_train": 6, "rntrick1": 20, "rntrick2": 20, "rntrick3": 20, "rntrick4": 20, "ro": [2, 15, 23], "robert": 22, "robust": [2, 23], "robustscal": [2, 24], "roc": [9, 12], "role": [2, 4, 7, 8, 10, 17, 23, 24], "roll": 8, "room": [2, 21, 23], "root": [0, 2, 7, 11, 15, 20, 24], "rot": 23, "rotat": [3, 10, 11, 12], "rotation_matrix": 11, "roughli": [3, 5], "round": [9, 11, 15], "routin": [15, 18, 23], "row": [1, 2, 3, 4, 7, 8, 11, 13, 18, 23, 24], "rr": [7, 24], "rrr": [7, 24], "rug": 15, "rule": [2, 3, 7, 8, 15, 23, 24], "run": [0, 2, 3, 4, 6, 7, 8, 10, 11, 13, 15, 17, 23, 24], "runtim": [0, 3, 8, 16], "rust": [2, 17, 18, 23], "rvert": 3, "rvert_2": 3, "s_": [5, 8], "s_1": 8, "s_i": [8, 9], "s_j": 8, "s_k": 8, "saddl": 15, "sai": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 20, 23, 24], "said": [8, 11, 15], "sake": [2, 7, 9, 13, 23, 24], "sale": [2, 23], "sam": 23, "same": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 13, 14, 16, 18, 20, 23, 24], "samm": 12, "sampl": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 16, 17, 18, 20, 23, 24], "sample_vari": 16, "sampleexptvari": 20, "samwis": 23, "sastri": 13, "satisfactori": [2, 23], "satisfi": [3, 4, 5, 8, 10, 15, 18, 20], "satur": [3, 8], "save": [2, 6, 8, 9, 11, 15, 23], "save_fig": [2, 8, 9, 11, 12, 23], "savefig": [2, 6, 8, 9, 11, 20, 23], "savetxt": 6, "saw": [7, 24], "scalabl": 12, "scalar": [4, 7, 8, 12, 24], "scale": [2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 21, 23], "scale_mean": 6, "scale_std": 6, "scaler": [2, 9, 10, 11, 12, 13, 24], "scan": [7, 9], "scari": 7, "scatter": [0, 2, 3, 8, 9, 10, 11, 16, 23], "scenario": [8, 15], "schedul": 15, "scheme": [3, 15], "schrage": 20, "sch\u00f8yen": 8, "scienc": [2, 3, 12, 14, 15, 17, 19, 20, 21, 22], "scientif": [2, 17, 23], "scientist": [2, 23], "scikit": [0, 1, 5, 7, 8, 10, 11, 12, 15, 17, 18, 22], "scikit_learn": 2, "scikitlearn": 23, "scikitplot": [9, 12], "scipi": [2, 5, 7, 8, 15, 17, 18, 23, 24], "scl": 8, "scm": 0, "score": [0, 1, 2, 3, 5, 8, 9, 11, 12, 13, 21, 23, 24], "scores_kfold": 8, "scratch": [1, 3, 15], "sdg": 15, "seaborn": [2, 3, 5, 8, 9, 23], "seamless": [2, 17, 23], "search": [0, 2, 3, 5, 7, 11, 15, 23], "sebastian": 23, "sebastianraschka": 23, "sec": 8, "second": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 16, 17, 18, 20, 21, 23, 24], "secondeigvector": 13, "secondli": 14, "section": [1, 6, 13, 18, 20, 24], "sector": 2, "see": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 17, 18, 20, 23, 24], "seed": [2, 3, 4, 5, 6, 7, 8, 10, 11, 13, 15, 16, 20, 23, 24], "seed_imag": 6, "seek": [3, 4, 10], "seem": [3, 5, 6], "seemingli": [2, 23], "seen": [2, 3, 5, 7, 12, 14, 20], "segment": 15, "seismic": 8, "seldomli": [2, 23], "select": [0, 3, 7, 8, 10, 11, 12, 13, 19, 20, 21, 22, 23, 24], "selevet": 0, "self": [3, 7, 24], "sell": 6, "semest": [9, 19], "semi": [10, 15], "semilogx": 8, "send": [7, 14, 15, 21, 23], "senior": [19, 21], "sens": [2, 6, 8, 10, 23], "sensibl": 5, "sensit": [2, 7, 8, 11, 15, 23, 24], "sent": 4, "sentenc": [6, 14], "separ": [2, 3, 4, 6, 8, 10, 11, 14, 16, 17, 20, 23], "septemb": 23, "sequenc": [5, 6, 9, 11, 12, 14, 15, 17, 18, 20, 23], "sequenti": [3, 5, 6, 12, 14, 20], "seri": [2, 3, 4, 5, 6, 7, 8, 12, 13, 14, 15, 18, 23, 24], "serif": [9, 20, 23], "serv": [2, 3, 4, 5, 7, 9, 15, 22, 23, 24], "session": [0, 3, 19, 21, 23], "set": [1, 3, 6, 7, 8, 9, 10, 12, 13, 15, 16, 17, 18, 20, 21], "set_major_formatt": 8, "set_major_loc": 8, "set_tick": [3, 10], "set_ticklabel": 3, "set_titl": [2, 3, 4, 5, 9, 14, 16, 23], "set_xlabel": [2, 3, 4, 5, 9, 14, 23], "set_xlim": [9, 14], "set_xticklabel": 3, "set_ylabel": [2, 3, 4, 5, 9, 23], "set_ylim": [9, 14], "set_ytick": 9, "set_yticklabel": [3, 8], "set_zlim": 8, "seth": 6, "setminu": 8, "setosa": [10, 11], "setosa_or_versicolor": 10, "setp": 8, "setup": [3, 6, 8, 10, 17, 23, 24], "sever": [1, 2, 5, 7, 8, 9, 10, 11, 13, 14, 15, 17, 18, 20, 23, 24], "sgd": [3, 5], "sgd_clf": 10, "sgdclassifi": 10, "sgdreg": 15, "sgdregressor": 15, "sgn": [7, 24], "shallow": 15, "shape": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 23, 24], "share": [0, 3, 5, 23], "shareabl": 0, "she": 9, "shift": [0, 3, 8, 14, 20], "ship": 5, "shire": 23, "short": [6, 7], "shortcom": 15, "shorten": 6, "shorter": 20, "shorthand": 23, "shortli": [18, 23], "should": [0, 2, 4, 5, 7, 8, 10, 11, 13, 14, 18, 20, 23, 24], "show": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "show_shap": 6, "shown": [2, 6, 7, 10, 14, 15, 18, 24], "shrink": [5, 7, 8, 10, 13, 24], "shrinkag": [7, 8, 24], "shrunk": 13, "shuffl": [2, 3, 6, 8, 15, 24], "side": [2, 4, 7, 10, 14, 15, 18, 23], "sigh": [17, 23], "sigma": [2, 3, 7, 8, 9, 12, 13, 14, 15, 18, 20, 23, 24], "sigma0": 20, "sigma1": 20, "sigma2": 20, "sigma_": [7, 18, 23, 24], "sigma_0": [7, 24], "sigma_1": [7, 24], "sigma_2": [7, 24], "sigma_fn": [9, 14], "sigma_i": [2, 7, 23, 24], "sigma_j": [7, 24], "sigma_m": [8, 20], "sigma_n": [13, 20], "sigma_t": 15, "sigma_x": 20, "sigmoid": [3, 4, 6, 9, 10, 12, 14], "sigmundson": 8, "sign": [3, 4, 9, 10, 12, 20, 21], "signal": [3, 5, 12, 14], "signifi": 6, "signific": 3, "significantli": [3, 15, 20], "sim": [6, 7, 8, 15, 20], "similar": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 18, 23], "similarli": [2, 3, 5, 7, 10, 12, 15, 20, 23, 24], "simpl": [1, 3, 4, 5, 7, 8, 9, 10, 12, 13, 14, 16, 17, 18, 20], "simplepredict": 12, "simpler": [1, 2, 3, 7, 8, 9, 15, 17, 23], "simplernn": 6, "simplest": [2, 3, 5, 6, 11, 12, 14, 16, 23], "simpletre": 12, "simpli": [2, 3, 4, 6, 7, 8, 10, 11, 12, 13, 14, 17, 18, 20, 23, 24], "simplic": [4, 7, 8, 9, 10, 11, 12, 13, 14, 16, 24], "simplicti": [7, 24], "simplifi": [2, 8, 11, 17, 23], "simplist": [5, 8, 20], "simul": 8, "simultan": 8, "sin": [2, 3, 4, 5, 6, 11, 14, 15, 18, 23], "sinc": [1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 15, 18, 20, 22, 23, 24], "sine": [5, 14], "singl": [2, 3, 4, 5, 7, 8, 9, 10, 11, 14, 15, 18, 20, 23, 24], "singular": [2, 8, 15, 18, 23], "sinusoid": 5, "site": [2, 19], "situat": [2, 6, 7, 9, 15, 20, 23, 24], "six": [5, 20], "size": [2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 18, 20, 23], "sketch": 12, "ski": 11, "skill": 2, "skip": 13, "skl": [2, 8, 23, 24], "sklearn": [0, 2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 15, 16, 23, 24], "skplt": [9, 12], "sl": 8, "slack": 10, "slice": [4, 18, 23], "slide": [1, 2, 5, 20, 23, 24], "slight": [8, 15], "slightli": [3, 4, 5, 7, 8, 9, 12, 20, 24], "slope": [10, 13, 14], "slow": [2, 4, 10, 15, 24], "slower": [7, 18, 23, 24], "slowest": 18, "slowli": 14, "slp": 3, "small": [2, 3, 4, 5, 7, 8, 10, 11, 12, 13, 14, 15, 17, 18, 20, 23, 24], "smaller": [2, 3, 4, 7, 8, 10, 11, 13, 15, 20, 23], "smallest": [2, 6, 16, 23], "smallest_row_index": 16, "smooth": [2, 5, 8, 15, 23], "sn": [2, 3, 5, 8, 9, 23], "sne": 13, "so": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 23, 24], "soar": 8, "social": 2, "soft": [3, 9, 12, 14], "soften": 10, "softmax": [5, 9], "softwar": [2, 10, 17, 18], "sol": 10, "sole": [2, 8, 23], "solid": [2, 9], "solut": [2, 3, 4, 5, 7, 8, 10, 12, 13, 15, 18, 20, 23, 24], "soluton": 4, "solv": [1, 2, 3, 5, 7, 8, 10, 12, 13, 14, 15, 18, 23, 24], "solve_expdec": 4, "solve_ode_deep_neural_network": 4, "solve_ode_neural_network": 4, "solve_pde_deep_neural_network": 4, "solveod": 4, "solveode_popul": 4, "solver": [4, 9, 10, 11, 12, 18, 23], "some": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 20, 23], "some_model": 8, "somehow": 6, "someon": 1, "someth": [0, 2, 3, 5, 6, 9, 11, 13, 20, 23, 24], "sometim": [2, 3, 13, 14, 15, 16, 24], "soon": [18, 21, 24], "sophist": [2, 23], "sopt": 15, "sort": [7, 8, 11, 13, 20], "sound": [5, 7], "sourc": [2, 3, 5, 8, 17, 18, 20, 23], "space": [2, 3, 6, 7, 10, 11, 13, 14, 15, 16, 20, 24], "span": [2, 5, 7, 11, 13, 18, 23, 24], "spare": 3, "spars": [5, 8, 18, 23], "sparse_mtx": [18, 23], "sparsecategoricalcrossentropi": 5, "sparsiti": 12, "spatial": [3, 4, 5, 14], "speak": 20, "special": [8, 9, 12, 14, 15, 18, 20, 23], "specif": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 17, 18, 20, 22, 23, 24], "specifi": [2, 5, 7, 8, 9, 11, 13, 15, 16, 20, 23], "specifici": [2, 12, 23], "spectacular": 5, "spectral": 3, "speech": [2, 3, 5, 6, 14], "speed": [3, 4, 6, 15], "spend": [1, 20], "sphere": [2, 24], "spin": 8, "spite": 2, "spline": 10, "split": [1, 3, 5, 6, 7, 8, 10, 11, 12, 13, 16, 20, 23], "splite": 2, "splitter": [3, 12], "spontan": 20, "spot": 5, "spread": [2, 13, 20, 23, 24], "springer": [22, 23], "spuriou": 15, "sqrsignal": 5, "sqrt": [5, 6, 7, 8, 10, 12, 13, 15, 20, 24], "squar": [0, 3, 4, 5, 6, 9, 10, 11, 13, 15, 16, 17, 18, 20], "squarederror": 12, "squaredeuclidean": 16, "squash": 14, "srtm": 8, "srtm_data_norway_1": 8, "stabil": 7, "stabl": [1, 2, 6, 7, 8, 11, 17, 23, 24], "stack": [5, 6], "stage": [0, 7, 15], "stai": [2, 4, 6, 7, 13, 23, 24], "stand": [2, 7, 11, 14, 23, 24], "standard": [2, 3, 6, 7, 8, 9, 10, 12, 14, 18, 20, 23], "standardscal": [2, 8, 9, 10, 11, 12, 13, 24], "stanford": 15, "start": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 18, 20, 21, 23, 24], "start_tim": 16, "stat": 8, "state": [3, 4, 6, 7, 8, 9, 10, 12, 13, 14, 15, 17, 20, 23, 24], "statement": [2, 9, 18, 23], "statist": [2, 3, 5, 6, 9, 11, 12, 13, 14, 15, 16, 18, 22, 24], "statu": [0, 2, 9, 23], "stavang": 8, "std": [2, 6, 8, 23, 24], "steep": 15, "step": [0, 2, 3, 4, 6, 8, 9, 11, 12, 13, 14, 15, 16, 18, 23], "step_fn": [9, 14], "step_length": 15, "steps_list": 11, "stereo": 5, "still": [2, 4, 5, 7, 8, 13, 15, 20, 24], "stimuli": 14, "stk": [22, 23], "stk2100": [22, 23], "stk3155": [0, 19, 21], "stk4021": [22, 23], "stk4051": [22, 23], "stk4155": [19, 21], "stk5000": 22, "stochast": [2, 3, 7, 8, 10, 13, 14], "stock": 6, "stoke": 14, "stone": [2, 9], "stop": [3, 6, 11, 15, 16], "storag": [7, 24], "store": [2, 3, 4, 5, 8, 13, 15, 20, 23], "storehaug": [21, 23], "str": [3, 5, 6], "straight": [2, 8, 10, 15, 23], "straightforward": [2, 4, 5, 7, 8, 10, 11, 12, 15, 18, 23, 24], "strategi": [2, 3, 11, 23], "stratifi": 8, "strength": [2, 7, 16, 24], "stretch": 13, "strict": [10, 15], "strictli": [10, 15], "stride": [6, 18], "strike": 8, "string": 3, "stroke": 9, "strong": [5, 8, 11, 12, 14, 18, 20], "strongli": [0, 2, 10, 17, 18], "stronli": [], "structur": [2, 3, 4, 5, 8, 11, 12, 14, 17, 23], "stuck": [3, 15], "student": [0, 2, 19, 21, 22, 23], "studi": [2, 5, 6, 7, 8, 9, 10, 13, 14, 15, 17, 22, 23, 24], "studier": 22, "style": [9, 11, 18, 23], "st\u00f8land": 21, "sub": [11, 14], "subdivid": [2, 18, 23], "subfield": 2, "subject": [8, 10, 20], "submit": 23, "subplot": [2, 3, 5, 6, 8, 9, 10, 11, 12, 16, 23], "subplots_adjust": [10, 20], "subprogram": [18, 23], "subract": [2, 24], "subroutin": [2, 23], "subscript": 3, "subsequ": [3, 6, 7, 8, 14, 18, 20, 24], "subset": [3, 8, 11, 14, 15, 17, 23], "subspac": [2, 10, 13, 24], "substanti": [11, 12], "substep": 13, "substitut": [1, 5, 8, 14, 18], "subsubset": 11, "subtask": 8, "subtl": 3, "subtract": [2, 6, 7, 8, 13, 15, 18, 20, 24], "subtre": 11, "succeed": [2, 6, 23], "success": [5, 9, 11, 15, 20], "successfulli": [6, 11], "sudo": [2, 17, 23], "suffer": [2, 3, 4, 7, 12, 23, 24], "suffici": [3, 8, 10, 13, 15], "suggest": [3, 15, 22], "suit": [10, 14], "suitabl": [0, 2, 20, 24], "sum": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "sum_": [1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "sum_i": [2, 4, 7, 8, 10, 15, 24], "sum_j": 8, "sum_ja_": 2, "sum_k": [8, 10, 14, 18], "sum_logist": 15, "sum_m": 5, "sum_n": 5, "sum_nx_": 5, "summar": [7, 8, 11], "summari": [3, 5, 6, 12, 19], "summat": [1, 2, 5, 24], "sunni": 11, "super": [7, 24], "superfici": 5, "superscript": [3, 14], "supervis": [2, 7, 8, 9, 11, 14, 17, 23, 24], "supplement": 9, "support": [2, 3, 11, 12, 13, 15, 17, 23, 24], "suppos": [2, 7, 8, 9, 10, 12, 13, 14, 15, 18, 23, 24], "suppress": [7, 15], "sure": [1, 2, 3, 6, 8], "surf": 8, "surfac": [2, 8, 23], "surpass": 8, "surpris": [2, 23], "surround": [5, 17], "survei": [2, 7, 8, 23], "svc": [10, 11, 12], "svd": [2, 8, 13, 23], "svdinv": 7, "svm": [10, 11, 12, 13], "svm_clf": [10, 12], "swath": [7, 24], "switch": 2, "sy": 15, "symbol": [3, 7, 13, 15, 17, 20, 23, 24], "symmeteri": 3, "symmetr": [2, 7, 10, 13, 14, 15, 18, 23, 24], "symmetri": 8, "sympi": [2, 17, 23], "synonim": 20, "syntax": 15, "system": [0, 2, 3, 5, 6, 8, 9, 11, 12, 14, 15, 17, 18, 23], "systemat": [6, 8], "t": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 23], "t0": [5, 8, 15], "t1": [4, 15], "t2": 4, "t3": 4, "t_": 4, "t_0": [4, 11, 15], "t_1": 15, "t_b": 12, "t_i": [3, 4, 7, 14, 24], "t_j": 14, "t_k": 11, "tabl": [11, 20, 21, 23], "tabul": [2, 23], "tabular": 23, "tackl": 6, "tag": [4, 5, 6, 7, 8, 9, 14, 15, 16, 18, 20, 24], "taht": [2, 23], "tail": 20, "tailor": [4, 10, 13, 23], "taiwan": [2, 23], "take": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 20, 23, 24], "taken": [2, 3, 5, 8, 12, 15, 18], "tan": 5, "tangent": [3, 6, 14, 15], "tanh": [3, 6, 9, 10, 14], "target": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 23, 24], "target_nam": 11, "task": [2, 3, 5, 8, 11, 13, 14, 16, 23], "tau": [5, 7, 20], "taught": 23, "tax": [], "taylor": [4, 15], "taylornr": 15, "tc": 10, "teach": [0, 19, 23], "team": 3, "teaser": 2, "technic": [2, 7, 8, 15], "techniqu": [2, 3, 10, 12, 15, 17, 20, 22, 23, 24], "technologi": [2, 3], "tell": [1, 2, 6, 8, 12, 13, 15, 20], "temp": 3, "temp1": 3, "temp2": 3, "temperatur": [2, 11, 23], "temporarili": 3, "ten": [5, 23], "tend": [5, 7, 8, 10, 11, 12, 14, 15, 16], "tendenc": [2, 23], "tension": 8, "tensor": 5, "tensorflow": [2, 4, 6, 10, 16, 17, 18, 22, 23, 24], "term": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 23, 24], "term1": [7, 8, 13], "term2": [7, 8, 13], "term3": [7, 8, 13], "term4": [7, 8, 13], "termin": [0, 2, 6, 7, 11, 12, 15, 24], "terminarl": 0, "terrain": 8, "terrain1": 8, "test": [1, 5, 6, 7, 8, 9, 10, 11, 12, 15, 18, 20, 23], "test_acc": 5, "test_accuraci": [3, 5], "test_error": 8, "test_imag": [5, 6], "test_ind": 8, "test_input": 6, "test_label": [5, 6], "test_loss": 5, "test_pr": 3, "test_predict": 3, "test_rnn": 6, "test_scor": [9, 12], "test_siz": [0, 2, 3, 5, 7, 8, 12, 24], "test_split": 11, "testerror": [2, 8, 24], "testi": 6, "testpredict": 6, "testx": 6, "text": [0, 2, 3, 4, 6, 7, 10, 11, 13, 15, 18, 20, 22, 24], "textbook": [1, 24], "textual": 11, "textur": 3, "tf": [3, 5, 6, 15, 16], "th": [2, 3, 4, 7, 8, 9, 11, 14, 15, 16, 18, 20, 23, 24], "than": [2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 17, 20, 23, 24], "thank": [6, 8], "theano": [3, 17, 23], "thei": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 18, 20, 23, 24], "them": [2, 3, 5, 6, 8, 10, 11, 12, 13, 14, 15, 18, 23, 24], "theme": [0, 2, 23], "themselv": [2, 20, 23], "thenc": 8, "theorem": [4, 8, 9, 24], "theoret": [2, 6, 12], "theori": [2, 3, 5, 10, 11, 14, 15, 17, 22, 23], "thereaft": [2, 7, 8, 13, 14, 18, 23], "therebi": [2, 7, 9, 13, 23, 24], "therefor": [2, 3, 4, 5, 6, 8, 9, 10, 13, 15, 20, 23, 24], "therein": 13, "thereof": [2, 8, 15, 23], "theta": [1, 2, 3, 6, 7, 8, 9, 15, 20, 23, 24], "theta_": [2, 3, 8, 9, 15, 23, 24], "theta_0": [1, 2, 7, 8, 9, 23, 24], "theta_0x_": [2, 23, 24], "theta_1": [2, 7, 8, 9, 23, 24], "theta_1x_": [2, 23, 24], "theta_1x_0": [2, 23], "theta_1x_1": [2, 9, 23], "theta_1x_2": [2, 23], "theta_1x_i": [9, 24], "theta_2": [2, 23, 24], "theta_2x_": [2, 23, 24], "theta_2x_0": [2, 23], "theta_2x_1": [2, 23], "theta_2x_2": [2, 9, 23], "theta_2x_i": 24, "theta_3x_i": 24, "theta_4x_i": 24, "theta_i": [2, 3, 7, 23, 24], "theta_j": [2, 7, 8, 23, 24], "theta_linreg": 15, "theta_p": 9, "theta_px_p": 9, "theta_t": 15, "thetavalu": 7, "thi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 24], "thing": [0, 1, 2, 3, 4, 6, 7, 9, 11, 20, 23], "think": [2, 3, 5, 6, 8, 11, 14, 15, 16, 20, 23, 24], "third": [2, 5, 8, 15, 21, 23], "thirti": 9, "thorughout": 23, "those": [2, 5, 7, 8, 10, 11, 12, 13, 18, 23, 24], "though": [1, 3, 4, 5, 6, 15, 18, 20], "thought": [8, 16, 20], "thousand": [2, 3, 24], "three": [2, 3, 5, 7, 8, 10, 11, 14, 18, 19, 20, 21, 23, 24], "threshold": [3, 5, 11, 12, 13, 14, 15], "through": [0, 2, 3, 4, 5, 6, 7, 8, 10, 13, 14, 15, 16, 17, 18, 20, 23, 24], "throughout": [0, 2, 6, 7, 16, 17, 18, 20, 23], "throw": [5, 8, 20], "thu": [2, 3, 4, 7, 8, 9, 10, 12, 13, 14, 15, 21, 23, 24], "thumb": [2, 8, 24], "thursdai": [], "tibshirani": [8, 22, 23], "tick_param": 8, "ticker": [8, 15, 20], "tif": 8, "tight_layout": [3, 9], "tightli": 13, "tild": [2, 7, 8, 9, 13, 20, 23, 24], "till": [2, 6, 9, 10, 11, 12, 14, 18, 23, 24], "time": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 23, 24], "timeit": 6, "timer": 6, "tini": 3, "tip": 5, "titl": [0, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 15, 20, 23], "tmp": 15, "tn": [4, 5, 9], "to_categor": [3, 5, 6], "to_categorical_numpi": 3, "to_numer": [2, 8, 23], "todai": 5, "togeth": [2, 5, 8, 10, 13, 15, 17, 23], "toi": 16, "told": 15, "toler": [4, 16], "tolist": 6, "tomographi": 14, "too": [2, 4, 6, 7, 8, 11, 13, 15, 20, 22, 24], "took": [10, 23], "tool": [0, 2, 3, 5, 8, 15, 17, 24], "toolbox": 10, "top": [2, 5, 7, 8, 11, 12, 17, 23], "topic": [2, 7, 8, 9, 10, 17, 24], "topolog": [5, 14], "topologi": [3, 14], "torkjellsdatt": [21, 23], "toss": [12, 20], "total": [2, 3, 4, 5, 6, 8, 9, 10, 12, 13, 14, 15, 16, 18, 20, 21, 23, 24], "total_loss": 6, "totalclustervari": 16, "totalscatt": 16, "toward": [0, 3, 4, 9, 14, 15], "town": [], "tp": [6, 9], "tpng": 11, "tpu": [15, 17, 23], "tqdm": 8, "tr": [], "track": [0, 5, 15, 16, 18, 24], "tract": [], "tractabl": [2, 23, 24], "trade": [7, 11], "tradeoff": [2, 7, 23, 24], "tradit": [2, 3, 6, 8, 23], "train": [1, 4, 5, 7, 8, 10, 11, 12, 13, 14, 15], "train_accuraci": [2, 3, 5, 23], "train_dataset": 6, "train_end": [2, 3, 24], "train_error": 8, "train_imag": [5, 6], "train_ind": 8, "train_label": [5, 6], "train_pr": 3, "train_siz": [2, 3, 5, 24], "train_step": 6, "train_test_split": [0, 1, 2, 3, 5, 7, 8, 9, 11, 12, 13, 23, 24], "train_test_split_numpi": [2, 3, 24], "trainable_vari": 6, "trained_model": 8, "trainerror": [2, 24], "traini": 6, "training_checkpoint": 6, "training_dataset": 6, "training_gradi": 15, "trainingerror": 8, "trainpredict": 6, "trainscor": 6, "trainx": 6, "trait": [2, 23], "trajectori": 6, "transfer": [11, 23], "transform": [2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 23, 24], "transit": [8, 14], "translat": [3, 6, 8, 12, 23], "transpos": [3, 7, 13, 18, 24], "travers": [2, 7], "treat": [2, 3, 5, 8, 14, 15, 20, 23, 24], "tree": [2, 3, 17, 23], "tree_clf": [11, 12], "tree_clf_": 11, "tree_clf_sr": 11, "tree_reg": 11, "tree_reg1": 11, "tree_reg2": 11, "trend": 20, "treue": 9, "trevor": 22, "tri": [1, 4, 5, 6, 11, 15], "triain": 2, "trial": [2, 4, 6, 8, 15, 20, 23], "triangl": 15, "triangular": 18, "trick": [5, 6, 10, 13, 15, 20], "trickier": 20, "tridiagon": 18, "trillion": 17, "trivial": [2, 3, 7, 13, 20, 23], "troubl": [0, 2, 10, 14, 24], "truck": 5, "true": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 18, 20, 23, 24], "true_fun": 8, "true_theta": 8, "truli": 23, "try": [0, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 20, 23], "tucker": 10, "tuesdai": [21, 23], "tumor": [9, 11], "tumour": 9, "tunabl": 3, "tune": [6, 11, 15, 18, 23], "turn": [2, 3, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 20, 23, 24], "tutori": [3, 6], "tv": 4, "tveito": 4, "tweak": [3, 6, 12, 20], "twice": 15, "twist": 13, "two": [0, 2, 3, 4, 6, 7, 8, 9, 11, 12, 13, 14, 15, 18, 19, 20, 22, 23, 24], "tx": 15, "tx_1": 15, "txt": [0, 6], "ty": 15, "type": [2, 3, 5, 8, 10, 12, 15, 18, 20, 24], "typic": [0, 1, 2, 3, 4, 5, 6, 7, 9, 11, 12, 14, 15, 20, 23, 24], "u": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 22, 23, 24], "u_": 18, "u_i": 14, "u_m": 12, "ua": [2, 23], "ubuntu": [2, 17, 23], "uci": [], "uio": [0, 21, 22], "un": 16, "unabl": 0, "unari": [18, 23], "unbalanc": [8, 11], "unbias": [2, 7, 8, 23], "uncent": 8, "uncertainti": [2, 7, 23], "uncertitud": 20, "unchang": [3, 5], "uncorrel": [12, 20], "undefin": [7, 24], "under": [2, 3, 7, 8, 12, 15, 17, 23], "underdetermin": [2, 23], "underfit": [3, 8], "underflowproblem": 7, "undergo": 7, "undergradu": [19, 21], "underli": [2, 3, 11, 15, 20, 23], "underset": [6, 16], "understand": [0, 2, 3, 5, 7, 8, 12, 15, 16, 17, 23, 24], "understood": [10, 15], "undesir": 10, "undetermin": [7, 10], "undo": 6, "unexpect": 8, "unexpected": 20, "unfair": 8, "unfortun": [3, 10, 11, 12], "unicode_liter": [10, 11], "uniform": [2, 3, 7, 8, 13, 15, 20, 23], "uniformli": [15, 20], "unifrompdf": 20, "unimport": 15, "union": [7, 8], "uniqu": [2, 4, 8, 15, 16, 18, 23], "unique_cluster_label": 16, "unit": [2, 3, 5, 6, 7, 12, 14, 20, 23, 24], "unitari": [7, 8, 18, 24], "unitarili": [18, 23], "uniti": 20, "univari": 20, "univers": [2, 3, 4, 15, 17, 19, 21, 23, 24], "unix": 3, "unknow": [2, 18, 23], "unknown": [2, 3, 5, 6, 7, 8, 10, 12, 15, 18, 23, 24], "unknowwn": 14, "unlabel": 3, "unless": [2, 5, 8, 13, 15, 23], "unlik": [3, 5, 10, 15], "unnecessarili": 11, "unord": 5, "unravel": 3, "unrol": [5, 13], "unseen": [0, 2, 9, 11], "unstabl": 3, "unsupervis": [2, 3, 6, 14, 17, 23], "unsymmetr": [18, 23], "until": [3, 4, 6, 11, 14, 15, 16], "untouch": 2, "unusu": 14, "up": [1, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 17, 18, 20, 21], "updat": [0, 3, 4, 12, 14, 15, 16], "uploa": 23, "upload": [0, 17, 22], "upon": [2, 3, 8, 9, 13, 18], "upper": [1, 2, 10, 11, 18, 24], "uppercas": [18, 23], "upsampl": 6, "upscal": 6, "url": [23, 24], "us": [0, 6, 7, 8, 10, 11, 12, 13, 14, 16, 18, 20, 22], "usag": [2, 10, 17, 23, 24], "usd": [], "usd10000": [], "use_bia": 6, "usecol": [2, 23], "useless": 3, "user": [0, 2, 3, 4, 6, 8, 9, 17, 18, 23], "usernam": 0, "usetex": 20, "usg": 8, "usr": 20, "usual": [2, 5, 6, 9, 14, 15, 16, 23], "ut": 7, "util": [3, 5, 6, 8, 9, 12, 16, 23], "ux": 18, "v": [0, 4, 6, 7, 8, 13, 15, 17, 24], "v0": 20, "v1": 20, "v2": 20, "v_0": 13, "va": 3, "vahid": 23, "val": 15, "val_accuraci": 5, "val_loss": 6, "vale": 4, "valid": [2, 3, 6, 9, 11, 12, 15, 17, 20, 23, 24], "validation_data": 5, "validation_split": 6, "valu": [1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 23], "valuat": 11, "valy": 6, "van": [2, 23, 24], "vandenbergh": [10, 15], "vandermond": [2, 23], "vanilla": [2, 8, 13, 16, 24], "vanish": [3, 6, 15, 20], "var": [7, 8, 12, 13, 20, 24], "var_x": 20, "varabl": 10, "varepsilon": [7, 8], "varepsilon_": [7, 8], "varepsilon_i": [7, 8], "vari": [2, 3, 5, 7, 8, 12, 23], "variabl": [2, 3, 4, 7, 8, 9, 10, 12, 13, 14, 15, 16, 18, 23, 24], "varianc": [2, 3, 7, 9, 11, 12, 13, 15, 16, 17, 18, 20, 23, 24], "variance_i": [7, 13, 24], "variance_x": [7, 13, 24], "variant": [2, 3, 8, 10, 14, 15, 23, 24], "variat": [5, 6, 13, 23], "varieti": [2, 5, 14, 17, 23], "variou": [1, 3, 5, 7, 8, 9, 10, 11, 13, 14, 15, 17, 18, 20, 23, 24], "varydimens": 6, "vastli": 5, "vaue": 3, "vault": 2, "vdot": [4, 15], "vec": 8, "vector": [2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 16, 17], "vector_mean": 16, "ventur": [2, 10, 17, 23], "venv": 0, "verbos": [3, 5, 6], "veri": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 20, 22, 23, 24], "verifi": [5, 13, 18, 23], "versatil": [10, 23], "versicolor": [10, 11], "version": [0, 2, 5, 12, 15, 16, 17, 18, 20, 23], "versu": 3, "vert": [1, 2, 3, 7, 8, 9, 10, 11, 13, 15, 23, 24], "vert_1": [7, 8, 24], "vert_2": [7, 8, 13, 24], "via": [2, 7, 8, 9, 10, 11, 12, 13, 14, 17, 18, 19, 20, 21, 23, 24], "vidal": 13, "video": [2, 3, 14, 17, 19, 21, 23], "view": [3, 5, 7, 8, 14, 15, 20, 22, 23], "violat": 10, "virginica": 11, "viridi": [2, 3, 4, 5, 23], "virtual": 3, "viscos": 15, "viscou": 15, "visibl": 0, "vision": [2, 5], "visual": [2, 5, 13, 14, 17, 23, 24], "visualis": 3, "visualstudio": [0, 1], "viz": [8, 10, 20], "vmap": 15, "vmax": [3, 8], "vmin": [3, 8], "voic": 5, "volum": [2, 5, 23], "vote": [12, 23], "voting_clf": 12, "votingclassifi": 12, "votingsimpl": 12, "vstack": [7, 13, 18, 20, 23, 24], "vt": [7, 24], "w": [2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 18, 20, 23, 24], "w1": 10, "w2": [10, 13], "w3": 10, "w_": [3, 14], "w_1": [10, 18], "w_1x_": 10, "w_1x_1": 10, "w_2": [10, 18], "w_2x_": 10, "w_2x_2": 10, "w_3": 18, "w_4": 18, "w_hidden": 4, "w_i": [3, 4, 12], "w_ix_i": 14, "w_j": 18, "w_m": 18, "w_output": 4, "w_px_": 10, "w_px_p": 10, "wa": [3, 5, 6, 7, 8, 9, 12, 13, 14, 16, 18, 23, 24], "wai": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 18, 20, 23, 24], "walk": 11, "walker": 20, "wang": [2, 23], "want": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 20, 23, 24], "warn": 6, "warrant": 8, "wast": 5, "watch": 17, "wave": 5, "wavelet": 10, "we": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 24], "weak": [11, 12, 16], "weather": [3, 14], "web": [17, 19, 21, 23], "webpag": 23, "websit": [8, 18, 19, 23], "wedg": [10, 20], "wednesdai": [21, 23], "wee": 13, "week": [2, 7, 8, 9, 19, 21], "weekli": [0, 1, 17, 19, 21, 22, 23], "weight": [3, 4, 5, 8, 9, 11, 12, 14, 15, 20], "weigth": 4, "welcom": [0, 10, 17], "well": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 17, 18, 20, 22, 23, 24], "went": 10, "were": [2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 20, 23], "wessel": [2, 23, 24], "what": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20], "whatev": 5, "when": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "whenev": [0, 15, 20], "where": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 23, 24], "wherea": [8, 20], "wherein": [3, 14], "whether": [2, 5, 7, 9, 11, 20, 23], "which": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24], "whichev": [3, 5], "while": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 20, 23, 24], "white": 11, "who": [0, 2], "whole": [3, 5, 6, 7, 11, 13, 15], "whose": [2, 8, 12, 20, 24], "whow": [13, 24], "why": [0, 1, 2, 3, 5, 8, 15], "wide": [2, 3, 5, 8, 9, 14, 17, 18, 23], "widehat": 8, "width": [2, 5, 10, 11, 23], "wieringen": [2, 23, 24], "win": 12, "wind": 11, "wing": [21, 23], "winther": 4, "wiothout": 8, "wiscons": 9, "wisconsin": 12, "wisdom": 8, "wise": [3, 7, 14, 15, 24], "wish": [2, 4, 7, 9, 10, 13, 15, 16, 18, 23, 24], "with_std": [2, 24], "wither": 8, "within": [2, 4, 5, 6, 9, 11, 14, 15, 16, 20, 22, 23], "withinclust": 16, "without": [0, 2, 3, 7, 8, 10, 11, 13, 14, 15, 23, 24], "won": [0, 2, 23], "wonder": 10, "word": [2, 3, 5, 6, 7, 8, 9, 16, 20, 23, 24], "work": [0, 1, 2, 3, 6, 8, 9, 10, 11, 15, 17, 19, 20, 21, 23, 24], "workshop": 23, "world": [1, 2, 10, 24], "worldwid": [2, 23], "worri": 0, "wors": [2, 3, 5, 6, 8, 23], "worth": 11, "would": [1, 2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 20, 23, 24], "wrap": [8, 18, 23], "write": [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 14, 15, 18, 23, 24], "written": [1, 2, 4, 5, 7, 13, 14, 15, 17, 18, 20, 23, 24], "wrong": [0, 3, 10], "wrongli": 12, "wrote": [7, 13, 24], "wrt": [12, 15], "wth": [12, 15], "www": [17, 18, 22, 23], "wx_1": 10, "x": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23], "x0": 10, "x1": [6, 10, 11, 12, 15], "x1_exampl": 10, "x1d": 10, "x2": [10, 11, 12, 15], "x2d": [10, 13], "x2d_train": 13, "x2dsl": 13, "x3": 10, "x_": [2, 4, 5, 7, 8, 10, 12, 13, 15, 16, 18, 20, 23, 24], "x_0": [2, 7, 13, 18, 23, 24], "x_1": [2, 4, 7, 8, 9, 10, 11, 12, 13, 15, 18, 20, 23, 24], "x_2": [2, 4, 7, 8, 9, 10, 11, 12, 13, 15, 18, 20, 23, 24], "x_3": [10, 18, 20], "x_4": 18, "x_center": 13, "x_data": 3, "x_data_ful": 3, "x_hidden": 4, "x_i": [2, 3, 4, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "x_input": 4, "x_ix_": [2, 23], "x_iy_i": 10, "x_j": [1, 2, 4, 10, 11, 14, 20, 24], "x_jy_j": 10, "x_k": [14, 16, 18, 20, 24], "x_l": 20, "x_m": [8, 14, 18, 20], "x_n": [2, 4, 5, 8, 10, 13, 14, 15, 18, 20, 23], "x_new": [11, 12], "x_offset": 8, "x_output": 4, "x_p": [5, 9, 11], "x_poli": 11, "x_poly10": 11, "x_pred": 6, "x_prev": 4, "x_reduc": 13, "x_scale": 10, "x_small": 15, "x_test": [0, 1, 2, 3, 5, 7, 8, 9, 11, 12, 13, 24], "x_test_own": 8, "x_test_scal": [2, 8, 9, 11, 12, 13, 24], "x_tot": 6, "x_train": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 23, 24], "x_train_mean": 8, "x_train_own": 8, "x_train_scal": [2, 8, 9, 11, 12, 13, 24], "x_val": 3, "xarrai": [17, 23], "xavier": 3, "xbnew": 15, "xcode": [2, 17, 23], "xdclassiffierconfus": 12, "xdclassiffierroc": 12, "xg_clf": 12, "xgb": 12, "xgbclassifi": 12, "xgboost": 11, "xgboot": 12, "xgbregressor": 12, "xgparam": 12, "xgtree": 12, "xi": [10, 15], "xi_": 10, "xi_1": 10, "xi_i": 10, "xk": 10, "xla": [15, 17, 23], "xlabel": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 20, 23, 24], "xlim": [8, 12], "xm": 11, "xmesh": 15, "xnew": [2, 15, 23], "xp": 20, "xpanda": [2, 24], "xpd": [7, 13, 24], "xplot": 2, "xscale": [2, 24], "xsr": 11, "xt_x": 15, "xtest": 8, "xtick": [5, 8, 10, 11], "xtrain": 8, "xu": [2, 23], "xx": [2, 18, 23], "xy": [2, 8, 10, 18, 23], "xytext": 10, "xz": [18, 23], "y": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "y1": 6, "y2": 6, "y3": 6, "y_": [2, 3, 7, 8, 12, 13, 18, 23, 24], "y_0": [2, 7, 13, 18, 23, 24], "y_1": [2, 7, 10, 11, 13, 15, 18, 23, 24], "y_1y_1": 10, "y_1y_1k": 10, "y_1y_2": 10, "y_1y_2k": 10, "y_1y_n": 10, "y_1y_nk": 10, "y_2": [2, 7, 10, 11, 13, 18, 23, 24], "y_2y_1": 10, "y_2y_1k": 10, "y_2y_2": 10, "y_2y_2k": 10, "y_3": [2, 11, 18], "y_4": 18, "y_data": [2, 3, 7, 8, 23, 24], "y_data_ful": 3, "y_decis": 10, "y_fit": [2, 24], "y_i": [2, 3, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 23, 24], "y_if_": 12, "y_ix_": [2, 23], "y_ix_i": [9, 10, 15, 24], "y_iy_jk": 10, "y_j": [8, 10, 14], "y_k": 14, "y_m": 18, "y_model": [2, 6, 7, 8, 23, 24], "y_n": [10, 15], "y_ny_1": 10, "y_ny_1k": 10, "y_ny_2": 10, "y_ny_2k": 10, "y_ny_n": 10, "y_ny_nk": 10, "y_offset": 8, "y_plot": 11, "y_pred": [2, 3, 6, 8, 9, 10, 11, 12, 24], "y_pred1": 11, "y_pred2": 11, "y_pred_rf": 12, "y_pred_tre": 12, "y_proba": [9, 12], "y_scaler": 8, "y_test": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 24], "y_test_onehot": 3, "y_test_predict": [], "y_tot": 6, "y_train": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 23, 24], "y_train_mean": 8, "y_train_onehot": 3, "y_train_predict": [], "y_train_scal": 8, "y_val": 3, "ye": [5, 8, 9], "year": [2, 17, 23], "yet": [2, 3, 8, 10, 13, 15, 23], "yi": 15, "yield": [2, 4, 7, 8, 10, 12, 14, 15, 16, 18, 20, 23], "yk": 10, "ylabel": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 20, 23, 24], "ylim": [5, 8], "ym": 11, "ymesh": 15, "yn": 2, "yo": [10, 11, 12], "yoshua": [3, 22], "you": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 13, 15, 17, 18, 20, 21, 22, 23, 24], "young": 2, "your": [0, 3, 4, 6, 7, 8, 10, 13, 15, 17, 18, 23], "your_model_object": 1, "yourself": [13, 15, 23], "youtub": 17, "ypred": 8, "ypredict": [2, 15, 23, 24], "ypredict2": 15, "ypredictlasso": 7, "ypredictol": [2, 7], "ypredictown": 8, "ypredictownridg": [8, 24], "ypredictridg": [2, 7, 8, 24], "ypredictskl": 8, "ytest": 8, "ytick": [5, 8, 10, 11], "ytild": [2, 8, 23, 24], "ytildelasso": 7, "ytildenp": [2, 23, 24], "ytildeol": [2, 7], "ytildeownridg": [8, 24], "ytilderidg": [7, 8, 24], "ytrain": 8, "yuxi": 23, "yx": [18, 23], "yy": [18, 23], "yz": [18, 23], "z": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 18, 20, 23, 24], "z_": [3, 4, 14, 18, 23], "z_0": [18, 23], "z_1": [18, 23], "z_2": [18, 23], "z_c": 3, "z_h": 3, "z_hidden": 4, "z_i": [3, 14], "z_j": [3, 14], "z_k": [14, 24], "z_m": 3, "z_mod": 11, "z_o": 3, "z_output": 4, "zaman": 20, "zaxi": 8, "zero": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 24], "zeros_lik": 6, "zeroth": 24, "zfill": 6, "zip": [6, 8], "zm_h": [2, 23], "zn": [], "zone": [], "zoom": 23, "zx": [18, 23], "zy": [18, 23], "zz": [18, 23], "\u00f8yvind": 8}, "titles": ["Exercises week 34", "Exercises week 35", "3. Linear Regression", "14. Building a Feed Forward Neural Network", "15. Solving Differential Equations with Deep Learning", "16. Convolutional Neural Networks", "17. Recurrent neural networks: Overarching view", "4. Ridge and Lasso Regression", "5. Resampling Methods", "6. Logistic Regression", "8. Support Vector Machines, overarching aims", "9. Decision trees, overarching aims", "10. Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods", "11. Basic ideas of the Principal Component Analysis (PCA)", "13. Neural networks", "7. Optimization, the central part of any Machine Learning algortithm", "12. Clustering and Unsupervised Learning", "Applied Data Analysis and Machine Learning", "2. Linear Algebra, Handling of Arrays and more Python Features", "Course setting", "1. Elements of Probability Theory and Statistical Data Analysis", "Teachers and Grading", "Textbooks", "Week 34: Introduction to the course, Logistics and Practicalities", "Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression"], "titleterms": {"": [10, 12], "1": [0, 1, 2, 24], "2": [0, 1, 2, 23, 24], "2023": 21, "3": [0, 1, 2, 24], "34": [0, 23], "35": [1, 24], "4": [0, 1, 2, 24], "5": [1, 2], "A": [2, 3, 6, 10, 11, 23], "And": [23, 24], "In": 21, "Ising": 8, "The": [0, 2, 3, 4, 5, 7, 8, 9, 10, 11, 13, 14, 17, 23, 24], "To": 23, "With": 6, "about": [23, 24], "activ": [3, 14], "ad": [2, 8, 23, 24], "adaboost": 12, "adagrad": 15, "adam": 15, "adapt": 12, "adjust": 3, "adversari": 6, "again": [5, 11], "ai": 23, "aim": [10, 11, 23], "aka": 23, "algebra": [18, 23], "algorithm": [11, 12, 13, 14, 23, 24], "algortithm": 15, "all": 10, "an": [0, 2, 6, 12, 23], "analys": [7, 24], "analysi": [2, 7, 8, 13, 17, 20, 23, 24], "analyt": [1, 2], "ani": 15, "anoth": 11, "appli": 17, "approach": [2, 10, 16, 23], "approxim": 14, "architectur": 3, "arrai": [18, 23], "assist": 21, "autocorrel": 20, "autograd": [4, 15], "automat": 15, "back": [3, 13, 14, 24], "background": 17, "bag": 12, "base": 15, "basic": [2, 7, 9, 11, 12, 13, 18, 24], "batch": 3, "bay": 7, "befor": 13, "better": 10, "bia": 8, "binari": 3, "bind": 23, "bird": 12, "boldsymbol": 24, "boost": 12, "bootstrap": [8, 12], "boston": [], "breast": 3, "brief": 23, "bring": 14, "build": [3, 5, 11], "c": 23, "calcul": 24, "can": 23, "cancer": [3, 9, 11, 13], "cart": 11, "case": [10, 12, 20, 24], "central": [15, 17, 20], "chain": 14, "chang": 12, "channel": 23, "chi": [2, 23], "choos": 3, "cifar01": 5, "classic": 13, "classif": [3, 11, 12], "classifi": 10, "clip": 3, "cluster": 16, "cnn": 5, "code": [0, 1, 3, 4, 7, 11, 13, 14, 15, 16, 23, 24], "collect": [3, 5], "commun": 23, "compar": [1, 4, 12], "complet": 24, "complex": [2, 8, 24], "complic": 8, "compon": 13, "comput": 11, "computerlab": 23, "con": 11, "concept": 20, "conjug": 15, "contn": 23, "convex": [10, 15], "convolut": [5, 14], "correl": [13, 24], "cost": [3, 12, 24], "cours": [17, 19, 22, 23], "covari": [7, 13, 20, 24], "cover": 23, "creat": 1, "cross": 8, "cython": 23, "data": [0, 2, 3, 5, 8, 9, 11, 13, 17, 20, 23, 24], "dataset": [3, 5], "david": 23, "deadlin": 23, "deadllin": 21, "decai": 4, "decis": [11, 12], "decomposit": [7, 13, 18, 24], "deeep": [], "deep": [3, 4, 23], "defin": [3, 23], "degre": [2, 24], "deliver": [0, 1], "dens": 2, "deriv": [1, 7, 14, 24], "descent": [4, 12, 15], "design": 24, "detail": [5, 23], "develop": 3, "diagon": 13, "differ": 10, "differenti": [4, 15], "diffus": 4, "dimension": [4, 5, 10], "disadvantag": 11, "discret": 20, "discrimin": 23, "distribut": [7, 20], "do": 3, "doe": 24, "domain": 20, "down": 3, "dropout": 3, "economi": 24, "element": [2, 20, 23], "elimin": 18, "energi": 23, "ensembl": 12, "entropi": 11, "environ": [0, 2], "equat": [2, 4, 14, 24], "error": [2, 12, 23, 24], "essenti": 23, "etc": 23, "euler": 4, "evalu": 3, "exampl": [3, 4, 5, 6, 8, 9, 10, 11, 12, 23, 24], "exercis": [0, 1, 2, 8, 24], "expect": 20, "experi": 20, "explor": 2, "exponenti": 4, "express": [1, 24], "extrapol": 6, "extrem": [12, 23], "ey": 12, "fall": 21, "famili": [3, 23], "famou": 18, "fantast": 24, "featur": [1, 11, 18, 24], "feed": [3, 14], "final": [14, 24], "find": 1, "fine": 3, "first": [6, 14, 23], "fit": [0, 1, 2, 12, 23], "fix": 24, "forc": 5, "forest": 12, "format": 23, "forward": [3, 4, 14], "foster": 23, "fourier": 5, "frank": 8, "freedom": [2, 24], "frequent": 24, "frequentist": [2, 23], "from": [7, 12, 14, 23, 24], "full": 4, "function": [2, 3, 8, 9, 10, 12, 13, 14, 15, 20, 23, 24], "further": [5, 7, 24], "gan": 6, "gaussian": 18, "gd": 15, "gener": [6, 11, 23], "geometr": 13, "gini": 11, "github": 0, "goal": [0, 1], "good": [2, 23], "grade": [21, 23], "gradient": [3, 4, 12, 15], "growth": 4, "ha": 17, "handl": [18, 23], "hessian": 24, "hidden": 4, "hous": [], "how": 1, "hyperparamet": 3, "hyperplan": 10, "i": [2, 3, 23], "id3": 11, "idea": 13, "ii": 23, "implement": [1, 3], "implic": [7, 24], "import": [7, 18, 23, 24], "improv": 3, "includ": 15, "increment": 13, "index": 11, "inform": 21, "input": 4, "instal": [17, 23], "instructor": 21, "interpret": [7, 13, 23, 24], "introduc": [13, 15, 24], "introduct": [2, 8, 17, 18, 23], "invers": [7, 18], "invert": 24, "iter": 12, "its": 24, "jacobian": 24, "jax": 15, "julia": 23, "jungl": 12, "kera": [3, 5], "kernel": [10, 13], "lagrangian": 10, "lasso": [7, 8, 24], "last": 24, "later": [7, 24], "layer": [3, 4, 5, 14], "learn": [0, 1, 2, 3, 4, 13, 15, 16, 17, 23, 24], "least": [1, 7, 8, 23, 24], "lectur": 23, "level": 12, "librari": [17, 23], "likelihood": 9, "limit": [3, 15, 20], "linear": [0, 2, 10, 15, 18, 23, 24], "link": [7, 13, 22, 24], "logist": [9, 23], "loss": 24, "lu": 18, "machin": [2, 10, 15, 17, 23], "main": [20, 23], "make": [2, 11, 12, 24], "mani": [12, 14], "mass": 23, "materi": 23, "math": [7, 24], "mathemat": [5, 7, 10, 24], "matric": [7, 18, 23], "matrix": [1, 3, 7, 13, 14, 18, 23, 24], "matter": 2, "max": 24, "mean": [2, 24], "meet": [7, 12, 20, 23, 24], "mercer": 10, "method": [8, 11, 12, 15, 23], "min": 24, "minim": 23, "ml": 23, "mlp": 14, "mnist": [5, 6], "model": [0, 2, 3, 6, 8, 14, 23], "momentum": 15, "moon": [10, 11], "more": [5, 8, 18, 23, 24], "multilay": 14, "multipl": [3, 5], "multipli": 10, "need": 23, "network": [3, 4, 5, 6, 9, 14, 23], "neural": [3, 4, 5, 6, 9, 14, 23], "new": 6, "non": 10, "normal": [2, 3], "notat": 14, "note": 24, "now": [3, 11, 15], "nuclear": [2, 23], "numba": 23, "number": [2, 4, 20, 24], "numer": [4, 20], "numpi": [18, 23], "object": 5, "obtain": 13, "od": 4, "off": 8, "ol": [0, 1, 7, 8], "one": [4, 14], "oper": 18, "optim": [3, 10, 15, 17, 23, 24], "order": 15, "ordinari": [1, 7, 8, 23, 24], "organ": [2, 23], "oslo": 22, "other": [6, 11, 13, 14, 18, 23], "our": [2, 6, 7, 13, 15, 23, 24], "outcom": [17, 23], "output": 4, "overarch": [2, 6, 10, 11, 23, 24], "overview": [12, 23], "own": [2, 12, 13, 23, 24], "packag": [18, 23], "panda": [23, 24], "paramet": [23, 24], "part": [15, 17], "partial": 4, "pass": 3, "pca": 13, "pdf": 20, "perceptron": 14, "perform": [3, 11], "period": 5, "perspect": 3, "plan": 24, "plethora": 23, "point": 6, "poisson": 4, "polynomi": [1, 5], "popul": 4, "popular": 23, "practic": [15, 21, 23], "pre": [3, 5], "predict": 6, "preprocess": 24, "prerequisit": [5, 17, 23], "princip": 13, "principl": 5, "pro": 11, "probabl": [7, 20], "problem": [3, 4, 15, 23, 24], "procedur": [11, 23], "process": [3, 5], "program": [4, 15], "project": [8, 21, 23], "prop": 15, "propag": [3, 14], "properti": [7, 20, 24], "python": [0, 2, 11, 17, 18, 23], "quick": 10, "r": 23, "random": [12, 13, 20], "read": [11, 23, 24], "real": [8, 23], "recommend": [23, 24], "recurr": [6, 14], "reduc": [2, 24], "reduct": 5, "reformul": 4, "regress": [0, 2, 7, 8, 9, 11, 12, 15, 23, 24], "regular": 3, "relat": [], "relev": [22, 24], "relu": 3, "remark": 5, "remind": [8, 10, 23, 24], "replac": 15, "repositori": 0, "requir": [4, 17], "resampl": 8, "rescal": 8, "residu": 24, "resourc": 4, "result": 24, "revisit": 15, "rewrit": [23, 24], "ridg": [2, 7, 8, 24], "rm": 15, "rule": 14, "same": 15, "sampl": 13, "scale": 24, "schedul": 23, "schemat": 11, "scheme": 4, "scienc": 23, "scikit": [2, 3, 13, 23, 24], "second": 15, "semest": 21, "set": [0, 2, 4, 5, 11, 14, 19, 23, 24], "setup": 0, "sgd": 15, "should": 3, "similar": 15, "simpl": [2, 6, 11, 15, 23, 24], "singl": 12, "singular": [7, 13, 24], "size": 24, "sklearn": 1, "soft": 10, "softmax": 3, "softwar": 23, "solv": 4, "solver": 15, "some": [15, 18, 24], "specifi": 4, "split": [0, 2, 24], "squar": [1, 2, 7, 8, 12, 23, 24], "standard": [15, 24], "state": 2, "statist": [7, 8, 17, 20, 23], "steepest": [12, 15], "stochast": [15, 20], "strongli": 23, "suggest": 23, "summari": [21, 23], "superposit": 5, "supervis": 3, "support": 10, "svd": [7, 24], "systemat": 5, "t": 24, "take": 1, "taken": 23, "teach": 21, "teacher": [21, 23], "techniqu": [8, 13], "technologi": 17, "tensorflow": [3, 5], "tent": [21, 23], "test": [0, 2, 3, 24], "text": 23, "textbook": [22, 23], "theorem": [7, 10, 13, 14, 20], "theori": 20, "thi": 23, "tip": 15, "togeth": 14, "tool": 23, "top": 3, "topic": 23, "toward": 13, "trade": 8, "tradeoff": 8, "train": [0, 2, 3, 6, 23, 24], "transform": 5, "tree": [11, 12], "tune": 3, "two": [5, 10, 17], "type": [4, 6, 14, 23], "uio": 23, "univers": [14, 22], "unsupervis": 16, "up": [0, 2, 4, 11, 14, 23, 24], "us": [1, 2, 3, 4, 5, 9, 15, 17, 23, 24], "v": 5, "valid": 8, "valu": [7, 13, 20, 24], "variabl": 20, "varianc": 8, "variou": 2, "vector": [1, 10, 14, 18, 23, 24], "versu": 23, "view": [2, 6, 12, 24], "virtual": 0, "visual": [3, 11], "wai": 11, "wave": 4, "we": 23, "week": [0, 1, 23, 24], "what": [2, 23, 24], "which": 3, "why": 23, "wisconsin": 9, "write": [6, 13], "x": 24, "xgboost": 12, "your": [1, 2, 12, 24]}}) \ No newline at end of file +Search.setIndex({"alltitles": {"A Classification Tree": [[9, "a-classification-tree"]], "A Frequentist approach to data analysis": [[0, "a-frequentist-approach-to-data-analysis"], [24, "a-frequentist-approach-to-data-analysis"]], "A better approach": [[8, "a-better-approach"]], "A first summary": [[24, "a-first-summary"]], "A quick Reminder on Lagrangian Multipliers": [[8, "a-quick-reminder-on-lagrangian-multipliers"]], "A simple example": [[4, "a-simple-example"]], "A soft classifier": [[8, "a-soft-classifier"]], "A top-down perspective on Neural networks": [[1, "a-top-down-perspective-on-neural-networks"]], "ADAM optimizer": [[13, "adam-optimizer"]], "Activation functions": [[12, "activation-functions"]], "Adaptive boosting: AdaBoost, Basic Algorithm": [[10, "adaptive-boosting-adaboost-basic-algorithm"]], "Adding error analysis and training set up": [[24, "adding-error-analysis-and-training-set-up"], [25, "adding-error-analysis-and-training-set-up"]], "Adjust hyperparameters": [[1, "adjust-hyperparameters"]], "Algorithms for Setting up Decision Trees": [[9, "algorithms-for-setting-up-decision-trees"]], "An Overview of Ensemble Methods": [[10, "an-overview-of-ensemble-methods"]], "An extrapolation example": [[4, "an-extrapolation-example"]], "An optimization/minimization problem": [[24, "an-optimization-minimization-problem"]], "And finally \\boldsymbol{X}\\boldsymbol{X}^T": [[25, "and-finally-boldsymbol-x-boldsymbol-x-t"]], "And what about using neural networks?": [[24, "and-what-about-using-neural-networks"]], "Another example, the moons again": [[9, "another-example-the-moons-again"]], "Applied Data Analysis and Machine Learning": [[18, null]], "Autocorrelation function": [[21, "autocorrelation-function"]], "Automatic differentiation": [[13, "automatic-differentiation"]], "Back to Ridge and LASSO Regression": [[25, "back-to-ridge-and-lasso-regression"]], "Back to the Cancer Data": [[11, "back-to-the-cancer-data"]], "Bagging": [[10, "bagging"]], "Bagging Examples": [[10, "bagging-examples"]], "Basic Matrix Features": [[19, "basic-matrix-features"]], "Basic ideas of the Principal Component Analysis (PCA)": [[11, null]], "Basic math of the SVD": [[5, "basic-math-of-the-svd"], [25, "basic-math-of-the-svd"]], "Basics": [[7, "basics"]], "Basics of a tree": [[9, "basics-of-a-tree"]], "Batch Normalization": [[1, "batch-normalization"]], "Bayes\u2019 Theorem and Ridge and Lasso Regression": [[5, "bayes-theorem-and-ridge-and-lasso-regression"]], "Boosting, a Bird\u2019s Eye View": [[10, "boosting-a-bird-s-eye-view"]], "Bootstrap": [[6, "bootstrap"]], "Bringing it together, first back propagation equation": [[12, "bringing-it-together-first-back-propagation-equation"]], "Building a Feed Forward Neural Network": [[1, null]], "Building a tree, regression": [[9, "building-a-tree-regression"]], "Building neural networks in Tensorflow and Keras": [[1, "building-neural-networks-in-tensorflow-and-keras"]], "CNNs in more detail, building convolutional neural networks in Tensorflow and Keras": [[3, "cnns-in-more-detail-building-convolutional-neural-networks-in-tensorflow-and-keras"]], "Cancer Data again now with Decision Trees and other Methods": [[9, "cancer-data-again-now-with-decision-trees-and-other-methods"]], "Choose cost function and optimizer": [[1, "choose-cost-function-and-optimizer"]], "Classical PCA Theorem": [[11, "classical-pca-theorem"]], "Clustering and Unsupervised Learning": [[14, null]], "Code for SVD and Inversion of Matrices": [[5, "code-for-svd-and-inversion-of-matrices"]], "Codes and Approaches": [[14, "codes-and-approaches"]], "Codes for the SVD": [[5, "codes-for-the-svd"], [25, "codes-for-the-svd"]], "Coding Setup and Linear Regression": [[15, "coding-setup-and-linear-regression"]], "Collect and pre-process data": [[1, "collect-and-pre-process-data"]], "Communication channels": [[24, "communication-channels"]], "Compare Bagging on Trees with Random Forests": [[10, "compare-bagging-on-trees-with-random-forests"]], "Comparing with a numerical scheme": [[2, "comparing-with-a-numerical-scheme"]], "Computing the Gini index": [[9, "computing-the-gini-index"]], "Conjugate gradient method": [[13, "conjugate-gradient-method"]], "Convex functions": [[13, "convex-functions"]], "Convolution Examples: Polynomial multiplication": [[3, "convolution-examples-polynomial-multiplication"]], "Convolution Examples: Principle of Superposition and Periodic Forces (Fourier Transforms)": [[3, "convolution-examples-principle-of-superposition-and-periodic-forces-fourier-transforms"]], "Convolutional Neural Network": [[12, "convolutional-neural-network"]], "Convolutional Neural Networks": [[3, null]], "Correlation Function and Design/Feature Matrix": [[25, "correlation-function-and-design-feature-matrix"]], "Correlation Matrix": [[11, "correlation-matrix"], [25, "correlation-matrix"]], "Correlation Matrix with Pandas": [[25, "correlation-matrix-with-pandas"]], "Course Format": [[24, "course-format"]], "Course setting": [[20, null]], "Covariance Matrix Examples": [[25, "covariance-matrix-examples"]], "Covariance and Correlation Matrix": [[25, "covariance-and-correlation-matrix"]], "Cross-validation": [[6, "cross-validation"]], "Deadlines for projects (tentative)": [[24, "deadlines-for-projects-tentative"]], "Decision trees, overarching aims": [[9, null]], "Deep learning methods": [[24, "deep-learning-methods"]], "Define model and architecture": [[1, "define-model-and-architecture"]], "Defining the cost function": [[1, "defining-the-cost-function"]], "Deliverables": [[15, "deliverables"], [16, "deliverables"]], "Derivatives and the chain rule": [[12, "derivatives-and-the-chain-rule"]], "Derivatives, example 1": [[25, "derivatives-example-1"]], "Deriving OLS from a probability distribution": [[5, "deriving-ols-from-a-probability-distribution"]], "Deriving and Implementing Ordinary Least Squares": [[16, "deriving-and-implementing-ordinary-least-squares"]], "Deriving and Implementing Ridge Regression": [[17, "deriving-and-implementing-ridge-regression"]], "Deriving the Lasso Regression Equations": [[25, "deriving-the-lasso-regression-equations"]], "Deriving the Ridge Regression Equations": [[25, "deriving-the-ridge-regression-equations"]], "Deriving the back propagation code for a multilayer perceptron model": [[12, "deriving-the-back-propagation-code-for-a-multilayer-perceptron-model"]], "Developing a code for doing neural networks with back propagation": [[1, "developing-a-code-for-doing-neural-networks-with-back-propagation"]], "Diagonalize the sample covariance matrix to obtain the principal components": [[11, "diagonalize-the-sample-covariance-matrix-to-obtain-the-principal-components"]], "Different kernels and Mercer\u2019s theorem": [[8, "different-kernels-and-mercer-s-theorem"]], "Disadvantages": [[9, "disadvantages"]], "Discriminative Modeling": [[24, "discriminative-modeling"]], "Domains and probabilities": [[21, "domains-and-probabilities"]], "Dropout": [[1, "dropout"]], "Economy-size SVD": [[25, "economy-size-svd"]], "Elements of Probability Theory and Statistical Data Analysis": [[21, null]], "Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods": [[10, null]], "Entropy and the ID3 algorithm": [[9, "entropy-and-the-id3-algorithm"]], "Essential elements of ML": [[24, "essential-elements-of-ml"]], "Evaluate model performance on test data": [[1, "evaluate-model-performance-on-test-data"]], "Example 2": [[25, "example-2"]], "Example 3": [[25, "example-3"]], "Example 4": [[25, "example-4"]], "Example Matrix": [[25, "example-matrix"]], "Example of discriminative modeling, taken from Generative Deep Learning by David Foster": [[24, "example-of-discriminative-modeling-taken-from-generative-deep-learning-by-david-foster"]], "Example of generative modeling, taken from Generative Deep Learning by David Foster": [[24, "example-of-generative-modeling-taken-from-generative-deep-learning-by-david-foster"]], "Example of own Standard scaling": [[25, "example-of-own-standard-scaling"]], "Example relevant for the exercises": [[25, "example-relevant-for-the-exercises"]], "Example: Exponential decay": [[2, "example-exponential-decay"]], "Example: Population growth": [[2, "example-population-growth"]], "Example: The diffusion equation": [[2, "example-the-diffusion-equation"]], "Example: binary classification problem": [[1, "example-binary-classification-problem"]], "Examples": [[24, "examples"]], "Examples of likelihood functions used in logistic regression and neural networks": [[7, "examples-of-likelihood-functions-used-in-logistic-regression-and-neural-networks"]], "Exercise 1 - Choice of model and degrees of freedom": [[17, "exercise-1-choice-of-model-and-degrees-of-freedom"]], "Exercise 1 - Finding the derivative of Matrix-Vector expressions": [[16, "exercise-1-finding-the-derivative-of-matrix-vector-expressions"]], "Exercise 1 - Github Setup": [[15, "exercise-1-github-setup"]], "Exercise 1: Setting up various Python environments": [[0, "exercise-1-setting-up-various-python-environments"]], "Exercise 2 - Deriving the expression for OLS": [[16, "exercise-2-deriving-the-expression-for-ols"]], "Exercise 2 - Deriving the expression for Ridge Regression": [[17, "exercise-2-deriving-the-expression-for-ridge-regression"]], "Exercise 2 - Setting up a Github repository": [[15, "exercise-2-setting-up-a-github-repository"]], "Exercise 2: making your own data and exploring scikit-learn": [[0, "exercise-2-making-your-own-data-and-exploring-scikit-learn"]], "Exercise 3 - Creating feature matrix and implementing OLS using the analytical expression": [[16, "exercise-3-creating-feature-matrix-and-implementing-ols-using-the-analytical-expression"]], "Exercise 3 - Fitting an OLS model to data": [[15, "exercise-3-fitting-an-ols-model-to-data"]], "Exercise 3 - Scaling data": [[17, "exercise-3-scaling-data"]], "Exercise 3 - Setting up a Python virtual environment": [[15, "exercise-3-setting-up-a-python-virtual-environment"]], "Exercise 3: Normalizing our data": [[0, "exercise-3-normalizing-our-data"]], "Exercise 4 - Fitting a polynomial": [[16, "exercise-4-fitting-a-polynomial"]], "Exercise 4 - Implementing Ridge Regression": [[17, "exercise-4-implementing-ridge-regression"]], "Exercise 4 - Testing multiple hyperparameters": [[17, "exercise-4-testing-multiple-hyperparameters"]], "Exercise 4 - The train-test split": [[15, "exercise-4-the-train-test-split"]], "Exercise 4: Adding Ridge Regression": [[0, "exercise-4-adding-ridge-regression"]], "Exercise 5 - Comparing your code with sklearn": [[16, "exercise-5-comparing-your-code-with-sklearn"]], "Exercise 5: Analytical exercises": [[0, "exercise-5-analytical-exercises"]], "Exercise: Cross-validation as resampling techniques, adding more complexity": [[6, "exercise-cross-validation-as-resampling-techniques-adding-more-complexity"]], "Exercise: Analysis of real data": [[6, "exercise-analysis-of-real-data"]], "Exercise: Bias-variance trade-off and resampling techniques": [[6, "exercise-bias-variance-trade-off-and-resampling-techniques"]], "Exercise: Lasso Regression on the Franke function with resampling": [[6, "exercise-lasso-regression-on-the-franke-function-with-resampling"]], "Exercise: Ordinary Least Square (OLS) on the Franke function": [[6, "exercise-ordinary-least-square-ols-on-the-franke-function"]], "Exercise: Ridge Regression on the Franke function with resampling": [[6, "exercise-ridge-regression-on-the-franke-function-with-resampling"]], "Exercises": [[0, "exercises"]], "Exercises and Projects": [[6, "exercises-and-projects"]], "Exercises week 34": [[15, null]], "Exercises week 35": [[16, null]], "Expectation values": [[21, "expectation-values"]], "Extremely useful tools, strongly recommended": [[24, "extremely-useful-tools-strongly-recommended"]], "Feed-forward neural networks": [[12, "feed-forward-neural-networks"]], "Feed-forward pass": [[1, "feed-forward-pass"]], "Final back propagating equation": [[12, "final-back-propagating-equation"]], "Fine-tuning neural network hyperparameters": [[1, "fine-tuning-neural-network-hyperparameters"]], "Fitting an Equation of State for Dense Nuclear Matter": [[0, "fitting-an-equation-of-state-for-dense-nuclear-matter"]], "Fixing the singularity": [[25, "fixing-the-singularity"]], "Frequently used scaling functions": [[25, "frequently-used-scaling-functions"]], "From one to many layers, the universal approximation theorem": [[12, "from-one-to-many-layers-the-universal-approximation-theorem"]], "Functionality in Scikit-Learn": [[25, "functionality-in-scikit-learn"]], "Further Dimensionality Remarks": [[3, "further-dimensionality-remarks"]], "Further properties (important for our analyses later)": [[5, "further-properties-important-for-our-analyses-later"], [25, "further-properties-important-for-our-analyses-later"]], "Gaussian Elimination": [[19, "gaussian-elimination"]], "General Features": [[9, "general-features"]], "General linear models and linear algebra": [[24, "general-linear-models-and-linear-algebra"]], "Generalizing the fitting procedure as a linear algebra problem": [[24, "generalizing-the-fitting-procedure-as-a-linear-algebra-problem"], [24, "id1"]], "Generative Adversarial Networks": [[4, "generative-adversarial-networks"]], "Generative Models": [[4, "generative-models"]], "Generative Versus Discriminative Modeling": [[24, "generative-versus-discriminative-modeling"]], "Geometric Interpretation and link with Singular Value Decomposition": [[11, "geometric-interpretation-and-link-with-singular-value-decomposition"]], "Gradient Boosting, Classification Example": [[10, "gradient-boosting-classification-example"]], "Gradient Boosting, Examples of Regression": [[10, "gradient-boosting-examples-of-regression"]], "Gradient Clipping": [[1, "gradient-clipping"]], "Gradient boosting: Basics with Steepest Descent/Functional Gradient Descent": [[10, "gradient-boosting-basics-with-steepest-descent-functional-gradient-descent"]], "Gradient descent": [[2, "gradient-descent"]], "Grading": [[22, "grading"], [22, "id2"], [24, "grading"]], "How to take derivatives of Matrix-Vector expressions": [[16, "how-to-take-derivatives-of-matrix-vector-expressions"]], "Hyperplanes and all that": [[8, "hyperplanes-and-all-that"]], "Important Matrix and vector handling packages": [[19, "important-matrix-and-vector-handling-packages"]], "Improving performance": [[1, "improving-performance"]], "In summary": [[22, "in-summary"]], "Including Stochastic Gradient Descent with Autograd": [[13, "including-stochastic-gradient-descent-with-autograd"]], "Incremental PCA": [[11, "incremental-pca"]], "Installing R, C++, cython or Julia": [[24, "installing-r-c-cython-or-julia"]], "Installing R, C++, cython, Numba etc": [[24, "installing-r-c-cython-numba-etc"]], "Instructor information": [[22, "instructor-information"]], "Interpretations and optimizing our parameters": [[24, "interpretations-and-optimizing-our-parameters"], [24, "id2"], [24, "id3"], [25, "interpretations-and-optimizing-our-parameters"], [25, "id1"], [25, "id2"]], "Interpreting the Ridge results": [[25, "interpreting-the-ridge-results"]], "Introducing JAX": [[13, "introducing-jax"]], "Introducing the Covariance and Correlation functions": [[11, "introducing-the-covariance-and-correlation-functions"], [25, "introducing-the-covariance-and-correlation-functions"]], "Introduction": [[0, "introduction"], [6, "introduction"], [18, "introduction"], [19, "introduction"]], "Iterative Fitting, Classification and AdaBoost": [[10, "iterative-fitting-classification-and-adaboost"]], "Iterative Fitting, Regression and Squared-error Cost Function": [[10, "iterative-fitting-regression-and-squared-error-cost-function"]], "Kernel PCA": [[11, "kernel-pca"]], "Kernels and non-linearity": [[8, "kernels-and-non-linearity"]], "LU Decomposition, the inverse of a matrix": [[19, "lu-decomposition-the-inverse-of-a-matrix"]], "Layers": [[1, "layers"]], "Layers used to build CNNs": [[3, "layers-used-to-build-cnns"]], "Learning goals": [[15, "learning-goals"], [16, "learning-goals"], [17, "learning-goals"]], "Learning outcomes": [[18, "learning-outcomes"], [24, "learning-outcomes"]], "Lectures and ComputerLab": [[24, "lectures-and-computerlab"]], "Limitations of supervised learning with deep networks": [[1, "limitations-of-supervised-learning-with-deep-networks"]], "Linear Algebra, Handling of Arrays and more Python Features": [[19, null]], "Linear Regression": [[0, null]], "Linear Regression Problems": [[25, "linear-regression-problems"]], "Linear Regression, basic elements": [[0, "linear-regression-basic-elements"]], "Linking Bayes\u2019 Theorem with Ridge and Lasso Regression": [[5, "linking-bayes-theorem-with-ridge-and-lasso-regression"]], "Linking the regression analysis with a statistical interpretation": [[5, "linking-the-regression-analysis-with-a-statistical-interpretation"]], "Linking with the SVD": [[5, "linking-with-the-svd"], [25, "linking-with-the-svd"]], "Links to relevant courses at the University of Oslo": [[23, "links-to-relevant-courses-at-the-university-of-oslo"]], "Logistic Regression": [[7, null], [7, "id1"]], "MNIST and GANs": [[4, "mnist-and-gans"]], "Machine Learning": [[24, "machine-learning"]], "Machine learning": [[18, "machine-learning"]], "Main textbooks": [[24, "main-textbooks"]], "Making a tree": [[9, "making-a-tree"]], "Making your own Bootstrap: Changing the Level of the Decision Tree": [[10, "making-your-own-bootstrap-changing-the-level-of-the-decision-tree"]], "Making your own test-train splitting": [[25, "making-your-own-test-train-splitting"]], "Mathematical Interpretation of Ordinary Least Squares": [[5, "mathematical-interpretation-of-ordinary-least-squares"], [25, "mathematical-interpretation-of-ordinary-least-squares"]], "Mathematical optimization of convex functions": [[8, "mathematical-optimization-of-convex-functions"]], "Mathematics of CNNs": [[3, "mathematics-of-cnns"]], "Mathematics of the SVD and implications": [[5, "mathematics-of-the-svd-and-implications"], [25, "mathematics-of-the-svd-and-implications"]], "Matrices in Python": [[24, "matrices-in-python"]], "Matrix multiplication": [[1, "matrix-multiplication"]], "Matrix-vector notation and activation": [[12, "matrix-vector-notation-and-activation"]], "Meet the covariance!": [[21, "meet-the-covariance"]], "Meet the Covariance Matrix": [[5, "meet-the-covariance-matrix"], [25, "meet-the-covariance-matrix"]], "Meet the Hessian Matrix": [[25, "meet-the-hessian-matrix"]], "Meet the Pandas": [[24, "meet-the-pandas"]], "Min-Max Scaling": [[25, "min-max-scaling"]], "Momentum based GD": [[13, "momentum-based-gd"]], "More complicated Example: The Ising model": [[6, "more-complicated-example-the-ising-model"]], "More interpretations": [[25, "more-interpretations"]], "More on Dimensionalities": [[3, "more-on-dimensionalities"]], "More on Rescaling data": [[6, "more-on-rescaling-data"]], "More preprocessing": [[25, "more-preprocessing"]], "Multilayer perceptrons": [[12, "multilayer-perceptrons"]], "Network requirements": [[2, "network-requirements"]], "Neural Networks vs CNNs": [[3, "neural-networks-vs-cnns"]], "Neural networks": [[12, null]], "Note about SVD Calculations": [[25, "note-about-svd-calculations"]], "Numerical experiments and the covariance, central limit theorem": [[21, "numerical-experiments-and-the-covariance-central-limit-theorem"]], "Numpy and arrays": [[19, "numpy-and-arrays"], [24, "numpy-and-arrays"]], "Numpy examples and Important Matrix and vector handling packages": [[24, "numpy-examples-and-important-matrix-and-vector-handling-packages"]], "Optimization, the central part of any Machine Learning algortithm": [[13, null]], "Optimizing our parameters": [[24, "optimizing-our-parameters"]], "Optimizing our parameters, more details": [[24, "optimizing-our-parameters-more-details"]], "Optimizing the cost function": [[1, "optimizing-the-cost-function"]], "Organizing our data": [[0, "organizing-our-data"], [24, "organizing-our-data"]], "Other Matrix and Vector Operations": [[19, "other-matrix-and-vector-operations"]], "Other Types of Recurrent Neural Networks": [[4, "other-types-of-recurrent-neural-networks"]], "Other courses on Data science and Machine Learning at UiO": [[24, "other-courses-on-data-science-and-machine-learning-at-uio"]], "Other courses on Data science and Machine Learning at UiO, contn": [[24, "other-courses-on-data-science-and-machine-learning-at-uio-contn"]], "Other popular texts": [[24, "other-popular-texts"]], "Other techniques": [[11, "other-techniques"]], "Other types of networks": [[12, "other-types-of-networks"]], "Other ways of visualizing the trees": [[9, "other-ways-of-visualizing-the-trees"]], "Our model for the nuclear binding energies": [[24, "our-model-for-the-nuclear-binding-energies"]], "Overview of first week": [[24, "overview-of-first-week"]], "Own code for Ordinary Least Squares": [[24, "own-code-for-ordinary-least-squares"], [25, "own-code-for-ordinary-least-squares"]], "PCA and scikit-learn": [[11, "pca-and-scikit-learn"]], "Pandas AI": [[24, "pandas-ai"]], "Partial Differential Equations": [[2, "partial-differential-equations"]], "Plans for week 35": [[25, "plans-for-week-35"]], "Practical tips": [[13, "practical-tips"]], "Practicalities": [[22, "practicalities"], [22, "id1"]], "Predicting New Points With A Trained Recurrent Neural Network": [[4, "predicting-new-points-with-a-trained-recurrent-neural-network"]], "Preprocessing our data": [[25, "preprocessing-our-data"]], "Prerequisites": [[24, "prerequisites"]], "Prerequisites and background": [[18, "prerequisites-and-background"]], "Prerequisites: Collect and pre-process data": [[3, "prerequisites-collect-and-pre-process-data"]], "Probability Distribution Functions": [[21, "probability-distribution-functions"]], "Program for stochastic gradient": [[13, "program-for-stochastic-gradient"]], "Properties of PDFs": [[21, "properties-of-pdfs"]], "Pros and cons of trees, pros": [[9, "pros-and-cons-of-trees-pros"]], "Python installers": [[18, "python-installers"], [24, "python-installers"]], "RMS prop": [[13, "rms-prop"]], "Random Numbers": [[21, "random-numbers"]], "Random forests": [[10, "random-forests"]], "Randomized PCA": [[11, "randomized-pca"]], "Reading material": [[24, "reading-material"]], "Reading recommendations:": [[25, "reading-recommendations"]], "Reading suggestions week 34": [[24, "reading-suggestions-week-34"]], "Recurrent neural networks": [[12, "recurrent-neural-networks"]], "Recurrent neural networks: Overarching view": [[4, null]], "Reducing the number of degrees of freedom, overarching view": [[0, "reducing-the-number-of-degrees-of-freedom-overarching-view"], [25, "reducing-the-number-of-degrees-of-freedom-overarching-view"]], "Reformulating the problem": [[2, "reformulating-the-problem"]], "Regression Case": [[10, "regression-case"]], "Regression analysis, overarching aims": [[24, "regression-analysis-overarching-aims"]], "Regression analysis, overarching aims II": [[24, "regression-analysis-overarching-aims-ii"]], "Regularization": [[1, "regularization"]], "Reminder from last week": [[25, "reminder-from-last-week"]], "Reminder on Statistics": [[6, "reminder-on-statistics"]], "Replace or not": [[13, "replace-or-not"]], "Required Technologies": [[18, "required-technologies"]], "Resampling Methods": [[6, null]], "Resampling methods": [[6, "id1"]], "Residual Error": [[25, "residual-error"]], "Resources on differential equations and deep learning": [[2, "resources-on-differential-equations-and-deep-learning"]], "Revisiting our Linear Regression Solvers": [[13, "revisiting-our-linear-regression-solvers"]], "Rewriting the Covariance and/or Correlation Matrix": [[25, "rewriting-the-covariance-and-or-correlation-matrix"]], "Rewriting the fitting procedure as a linear algebra problem": [[24, "rewriting-the-fitting-procedure-as-a-linear-algebra-problem"]], "Rewriting the fitting procedure as a linear algebra problem, more details": [[24, "rewriting-the-fitting-procedure-as-a-linear-algebra-problem-more-details"]], "Ridge and LASSO Regression": [[25, "ridge-and-lasso-regression"]], "Ridge and Lasso Regression": [[5, null], [5, "id1"]], "Same code but now with momentum gradient descent": [[13, "same-code-but-now-with-momentum-gradient-descent"]], "Schedule first week": [[24, "schedule-first-week"]], "Schematic Regression Procedure": [[9, "schematic-regression-procedure"]], "Setting up the Back propagation algorithm": [[12, "setting-up-the-back-propagation-algorithm"]], "Setting up the Matrix to be inverted": [[25, "setting-up-the-matrix-to-be-inverted"]], "Setting up the network using Autograd; The full program": [[2, "setting-up-the-network-using-autograd-the-full-program"]], "Similar (second order function now) problem but now with AdaGrad": [[13, "similar-second-order-function-now-problem-but-now-with-adagrad"]], "Simple Python Code to read in Data and perform Classification": [[9, "simple-python-code-to-read-in-data-and-perform-classification"]], "Simple case": [[25, "simple-case"]], "Simple linear regression model using scikit-learn": [[0, "simple-linear-regression-model-using-scikit-learn"], [24, "simple-linear-regression-model-using-scikit-learn"]], "Software and needed installations": [[24, "software-and-needed-installations"]], "Solving Differential Equations with Deep Learning": [[2, null]], "Solving the one dimensional Poisson equation": [[2, "solving-the-one-dimensional-poisson-equation"]], "Solving the wave equation with Neural Networks": [[2, "solving-the-wave-equation-with-neural-networks"]], "Some famous Matrices": [[19, "some-famous-matrices"]], "Some simple problems": [[13, "some-simple-problems"]], "Some useful matrix and vector expressions": [[25, "some-useful-matrix-and-vector-expressions"]], "Splitting our Data in Training and Test data": [[0, "splitting-our-data-in-training-and-test-data"], [25, "splitting-our-data-in-training-and-test-data"]], "Standard steepest descent": [[13, "standard-steepest-descent"]], "Statistical analysis and optimization of data": [[18, "statistical-analysis-and-optimization-of-data"], [24, "statistical-analysis-and-optimization-of-data"]], "Steepest descent": [[13, "steepest-descent"]], "Stochastic Gradient Descent (SGD)": [[13, "stochastic-gradient-descent-sgd"]], "Stochastic variables and the main concepts, the discrete case": [[21, "stochastic-variables-and-the-main-concepts-the-discrete-case"]], "Support Vector Machines, overarching aims": [[8, null]], "Systematic reduction": [[3, "systematic-reduction"]], "Teachers": [[24, "teachers"]], "Teachers and Grading": [[22, null]], "Teaching Assistants Fall semester 2023": [[22, "teaching-assistants-fall-semester-2023"]], "Tentative deadllines for projects": [[22, "tentative-deadllines-for-projects"]], "Testing the Means Squared Error as function of Complexity": [[0, "testing-the-means-squared-error-as-function-of-complexity"], [25, "testing-the-means-squared-error-as-function-of-complexity"]], "Textbooks": [[23, null]], "The Algorithm before theorem": [[11, "the-algorithm-before-theorem"]], "The Breast Cancer Data, now with Keras": [[1, "the-breast-cancer-data-now-with-keras"]], "The CART algorithm for Classification": [[9, "the-cart-algorithm-for-classification"]], "The CART algorithm for Regression": [[9, "the-cart-algorithm-for-regression"]], "The CIFAR01 data set": [[3, "the-cifar01-data-set"]], "The Jacobian": [[25, "the-jacobian"]], "The MNIST dataset again": [[3, "the-mnist-dataset-again"]], "The RELU function family": [[1, "the-relu-function-family"]], "The SVD, a Fantastic Algorithm": [[25, "the-svd-a-fantastic-algorithm"]], "The Softmax function": [[1, "the-softmax-function"]], "The \\chi^2 function": [[0, "the-chi-2-function"], [24, "the-chi-2-function"], [24, "id4"], [24, "id5"], [24, "id6"], [24, "id7"], [24, "id8"]], "The bias-variance tradeoff": [[6, "the-bias-variance-tradeoff"]], "The code for solving the ODE": [[2, "the-code-for-solving-the-ode"]], "The complete code with a simple data set": [[25, "the-complete-code-with-a-simple-data-set"]], "The cost/loss function": [[25, "the-cost-loss-function"]], "The course has two central parts": [[18, "the-course-has-two-central-parts"]], "The equations for ordinary least squares": [[25, "the-equations-for-ordinary-least-squares"]], "The logistic function": [[7, "the-logistic-function"]], "The mean squared error and its derivative": [[25, "the-mean-squared-error-and-its-derivative"]], "The moons example": [[8, "the-moons-example"]], "The multilayer perceptron (MLP)": [[12, "the-multilayer-perceptron-mlp"]], "The network with one input layer, specified number of hidden layers, and one output layer": [[2, "the-network-with-one-input-layer-specified-number-of-hidden-layers-and-one-output-layer"]], "The plethora of machine learning algorithms/methods": [[24, "the-plethora-of-machine-learning-algorithms-methods"]], "The singular value decomposition": [[5, "the-singular-value-decomposition"], [25, "the-singular-value-decomposition"]], "The two-dimensional case": [[8, "the-two-dimensional-case"]], "To our real data: nuclear binding energies. Brief reminder on masses and binding energies": [[24, "to-our-real-data-nuclear-binding-energies-brief-reminder-on-masses-and-binding-energies"]], "Topics covered in this course: Statistical analysis and optimization of data": [[24, "topics-covered-in-this-course-statistical-analysis-and-optimization-of-data"]], "Towards the PCA theorem": [[11, "towards-the-pca-theorem"]], "Train and test datasets": [[1, "train-and-test-datasets"]], "Two-dimensional Objects": [[3, "two-dimensional-objects"]], "Type of problem": [[2, "type-of-problem"]], "Types of Machine Learning": [[24, "types-of-machine-learning"]], "Useful Python libraries": [[18, "useful-python-libraries"], [24, "useful-python-libraries"]], "Using Autograd": [[13, "using-autograd"]], "Using forward Euler to solve the ODE": [[2, "using-forward-euler-to-solve-the-ode"]], "Using gradient descent methods, limitations": [[13, "using-gradient-descent-methods-limitations"]], "Visualization": [[1, "visualization"], [1, "id1"]], "Visualizing the Tree, Classification": [[9, "visualizing-the-tree-classification"]], "Week 34: Introduction to the course, Logistics and Practicalities": [[24, null]], "Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression": [[25, null]], "Weekly Exercises 3": [[17, null]], "What Is Generative Modeling?": [[24, "what-is-generative-modeling"]], "What does it mean?": [[25, "what-does-it-mean"]], "What is Machine Learning?": [[0, "what-is-machine-learning"]], "What is a good model?": [[0, "what-is-a-good-model"], [24, "what-is-a-good-model"]], "What is a good model? Can we define it?": [[24, "what-is-a-good-model-can-we-define-it"]], "Which activation function should I use?": [[1, "which-activation-function-should-i-use"]], "Why Linear Regression (aka Ordinary Least Squares and family)": [[24, "why-linear-regression-aka-ordinary-least-squares-and-family"]], "Wisconsin Cancer Data": [[7, "wisconsin-cancer-data"]], "Writing Our First Generative Adversarial Network": [[4, "writing-our-first-generative-adversarial-network"]], "Writing our own PCA code": [[11, "writing-our-own-pca-code"]], "XGBoost: Extreme Gradient Boosting": [[10, "xgboost-extreme-gradient-boosting"]], "a) Expression for Ridge regression": [[17, "a-expression-for-ridge-regression"]], "scikit-learn implementation": [[1, "scikit-learn-implementation"]]}, "docnames": ["chapter1", "chapter10", "chapter11", "chapter12", "chapter13", "chapter2", "chapter3", "chapter4", "chapter5", "chapter6", "chapter7", "chapter8", "chapter9", "chapteroptimization", "clustering", "exercisesweek34", "exercisesweek35", "exercisesweek36", "intro", "linalg", "schedule", "statistics", "teachers", "textbooks", "week34", "week35"], "envversion": {"sphinx": 62, "sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.intersphinx": 1}, "filenames": ["chapter1.ipynb", "chapter10.ipynb", "chapter11.ipynb", "chapter12.ipynb", "chapter13.ipynb", "chapter2.ipynb", "chapter3.ipynb", "chapter4.ipynb", "chapter5.ipynb", "chapter6.ipynb", "chapter7.ipynb", "chapter8.ipynb", "chapter9.ipynb", "chapteroptimization.ipynb", "clustering.ipynb", "exercisesweek34.ipynb", "exercisesweek35.ipynb", "exercisesweek36.ipynb", "intro.md", "linalg.ipynb", "schedule.md", "statistics.ipynb", "teachers.md", "textbooks.md", "week34.ipynb", "week35.ipynb"], "indexentries": {}, "objects": {}, "objnames": {}, "objtypes": {}, "terms": {"": [0, 1, 2, 3, 4, 5, 6, 7, 9, 11, 12, 13, 15, 16, 17, 18, 19, 21, 22, 24, 25], "0": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 21, 22, 24, 25], "00": [0, 1, 5, 11, 24, 25], "000": [1, 3], "00000000e": [], "001": [2, 8, 13], "004": 5, "00727646693": [0, 24], "0086649156": [0, 24], "01": [0, 1, 2, 5, 9, 11, 13, 17, 23, 24, 25], "0110": 21, "01719003e": [], "02": [0, 4, 7, 12, 24], "02334824": [], "02857": 4, "02f": 6, "03077640549": 4, "03097597e": [], "031": 5, "04": 11, "0458": 9, "05": [4, 6], "062292565": 4, "062435": [], "06730814": [], "07": [], "0713": [0, 24], "07285": 3, "08": 21, "08078025e": [], "08336233266": 4, "0917": 9, "0n": [0, 24], "0x113e21950": 17, "1": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 20, 21, 22, 23, 24], "10": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 19, 20, 21, 22, 24, 25], "100": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 13, 14, 15, 16, 17, 19, 21, 22, 24, 25], "1000": [0, 1, 2, 4, 5, 8, 11, 13, 14, 18, 21, 24], "10000": [2, 5, 6, 10, 11, 13, 21], "100000": 8, "10001": 10, "1001": 21, "1002": 21, "1003": 21, "1005": 21, "1009": 21, "101": 16, "1011": 21, "1013": 21, "1013904243": 21, "1015": 21, "102": 16, "1023": 21, "1024": 3, "1026": 21, "1027": 21, "103": 1, "1030": 21, "1037": 21, "1038": 21, "1040": 21, "1047": 21, "107": 16, "108": [], "10th": 9, "10x": [0, 24], "11": [0, 2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 19, 21, 23, 24, 25], "110": [], "1100": 21, "1101": 21, "111": [1, 7, 12], "112": 16, "11340253": [], "11590451": [], "116": 16, "117": 16, "118": 16, "12": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 19, 21, 23, 24, 25], "120": 3, "121": [8, 9, 10, 16], "1215pm": [22, 24], "122": [8, 9, 10], "124": [0, 24], "125": 16, "127": [4, 16], "128": [3, 4, 13], "129": 16, "1298": 9, "12pm": [22, 24], "13": [0, 2, 9, 12, 19, 21, 24], "131": 16, "133": 7, "135": 16, "136": 16, "14": [0, 2, 4, 6, 8, 9, 10, 12, 19, 21, 23, 25], "141": 16, "143": 16, "1446729567": 4, "149": 16, "14g": 6, "15": [0, 2, 4, 6, 7, 8, 9, 12, 13, 21, 24], "150": [4, 8], "152": 16, "153760": [], "156": 16, "157": [], "158": [], "159": 16, "15g": 6, "15pm": 24, "16": [1, 2, 3, 4, 5, 8, 9, 10, 21, 24], "160": 16, "1603": 3, "161": 16, "162": 16, "16231451": 4, "163": 16, "16384": 3, "164": 16, "167": 16, "17": [1, 2, 8, 21], "172": 16, "173": 16, "176": 16, "178": 16, "179": 16, "1797": 1, "18": [2, 6, 7, 8, 9, 10, 21, 24], "1807": 4, "18392847": [], "19": [2, 21, 24], "1940": [], "1943": 12, "1970": [19, 24], "1973": 9, "1979": 6, "1_1": 12, "1_2": 12, "1_3": 12, "1cm": [0, 8, 10, 21, 24], "1d": [1, 2, 3], "1e": [2, 4, 13, 14], "1e10": 14, "1e4": 6, "1f": 1, "1k": 19, "1n": [0, 24], "1x": [0, 24], "2": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 19, 21, 23], "20": [0, 1, 2, 6, 7, 8, 16, 17, 21, 22, 24, 25], "200": [0, 2, 3, 4, 8, 9, 10], "2000": [0, 25], "2004": 13, "2006": 23, "20072279": [], "2008": 24, "2010": 1, "2011": 1, "2014": 4, "2015": 1, "2016": [0, 24], "2018": [0, 6, 25], "2021": [6, 14], "2022": 24, "2025": [24, 25], "21": [0, 1, 5, 7, 9, 12, 19, 24, 25], "2116753732": 4, "215pm": [22, 24], "2167072": [], "22": [0, 1, 5, 12, 13, 19, 24, 25], "221": 8, "225": 4, "22948497": [], "23": [1, 12, 19], "24": [0, 1, 19, 24], "25": [2, 3, 4, 5, 6, 8, 9, 11, 25], "250": [2, 4, 7, 9], "25000": [], "250154": [], "253775": [], "255": 3, "256": 4, "26": [], "26303845": [], "264": [], "265": [], "265109911": 4, "266": [], "269": [], "27": 1, "270": [], "27n_": 21, "28": [1, 3, 4], "2830637392": 4, "2861": 21, "2873": 9, "2882": 21, "2886": 21, "2890": [0, 24], "2892": 21, "29": 25, "2915": 21, "2931": 24, "29364655": [], "294399745619595": [], "296247": [], "2968": 24, "2980": 24, "298273": [], "298375": [], "2990": 24, "2_": 12, "2_1": 12, "2_2": 12, "2_3": 12, "2_i": 12, "2_m": [6, 21], "2_t": 13, "2_x": 21, "2a": 17, "2b": 21, "2cm": 8, "2d": [1, 3, 11, 12, 18, 24], "2e": 6, "2f": [0, 7, 9, 10, 11, 12, 24], "2g": 2, "2g_i": 2, "2k": 3, "2m": 6, "2n": [0, 2, 3, 24, 25], "2nd": 9, "2p": 21, "2pt": 4, "2x": [0, 3, 8, 13, 24], "2x_ix_jy_iy_j": 8, "2x_j": 8, "2y_i": 10, "2y_j": 8, "3": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 19, 20, 21, 22, 24], "30": [0, 1, 4, 6, 7, 10, 13, 22], "30000": [0, 24], "3072": 3, "31": [12, 19, 21], "315": 6, "3155": [0, 5, 6, 25], "32": [3, 4, 6, 12, 13, 19, 21], "3200": 1, "3250": 1, "3297": [], "33": [12, 19, 22], "3303": [], "3310": [], "332331": [], "333": 7, "3331": [], "3337": [], "34": 19, "3436": [0, 24], "3437": [0, 24], "35": [0, 6, 24], "3581341341": 4, "359": 5, "36": [0, 5, 6, 21], "370782966": 4, "38": 21, "39": [0, 22, 24], "3d": [2, 3, 4, 6, 13, 16], "3f": [1, 3, 9], "3n": 19, "3x": [2, 8], "3x_i": 2, "3y": 8, "4": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 21, 24], "40": [1, 6, 22, 24], "400": 4, "4000": 24, "4050": [23, 24], "41": 19, "4155": [2, 15], "41589548": [], "42": [1, 4, 8, 9, 10, 19], "43": [0, 7, 19], "4310": 24, "436462435": 4, "44": [0, 19], "45": [22, 24], "46": [22, 24], "462": 7, "47": [22, 24], "479465113": 4, "47958494": [], "48": [], "48257387": [22, 24], "49": [5, 6, 11], "49152": 3, "4940954": [0, 24], "4990": 21, "4992": 21, "4997": 21, "4c4c7f": [9, 10], "4d": 3, "4f": 6, "4pm": [22, 24], "4y": 8, "4y_i": 10, "5": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 21, 24, 25], "50": [1, 2, 3, 4, 6, 7, 8, 10, 13, 24, 25], "500": [1, 3, 4, 6, 9, 10, 13], "5018": 21, "506": [], "507d50": [9, 10], "50j": 13, "50x10": 1, "51": 10, "510": 1, "512132": [], "5177783846": 4, "53": 9, "54": [6, 21], "5411205": [], "54894451": [], "55": 1, "56": 1, "56536": [0, 24], "569": 1, "57": [0, 8, 22, 24], "571": 5, "58": [10, 22, 24], "591317992": 4, "5cm": 21, "5f": 8, "5x": 8, "5y": 8, "6": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 19, 21, 22, 24, 25], "60": [1, 3], "60000": 4, "6019067271": 4, "606439": [], "625": 7, "63": 1, "64": [1, 3, 4, 13, 19, 24], "64x50": 1, "65": [1, 8, 9], "6887363571": 4, "69": [16, 21], "69069n_": 21, "691": [], "6n_": 21, "7": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 19, 21, 23, 24], "70": [1, 7], "70653767": 4, "71": 1, "724": 3, "73": [], "7304881": [], "75": [5, 6, 8, 11], "76": [22, 24], "765": 7, "77": [22, 24], "7718": 9, "7782028952": 4, "77893972": [], "78": [], "7d7d58": [9, 10], "8": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 19, 21, 22, 24], "80": [0, 1, 5, 8, 17, 25], "800": [4, 7], "81": 1, "815am": [22, 24], "85": 1, "8702784034": 4, "88": 24, "8f": 6, "8g": 6, "8n": 19, "8x8": 1, "9": [0, 1, 2, 4, 5, 6, 7, 8, 9, 11, 12, 13, 19, 21, 24], "90": 1, "9040": 9, "91": [22, 24], "92": [22, 24], "93": 16, "931": [0, 24], "933": 5, "937": 21, "938": 21, "939": [0, 21, 24], "94": 21, "95": [1, 11], "954": 21, "955820c21e8b": 4, "96": 6, "960": 21, "961": 21, "962": 21, "9649652536": 4, "96611194e": [], "9780387310732": 23, "9780387848570": 23, "9781098134174": 24, "9781492032632": 23, "9781801819312": 24, "98": [0, 1, 16], "985": 21, "986": 21, "989": 21, "9898ff": [9, 10], "99": [13, 16], "991": 21, "992": 21, "993": 21, "996": 5, "999": [9, 21], "9x": 6, "9y": 6, "A": [2, 3, 5, 6, 7, 10, 11, 12, 13, 15, 16, 18, 19, 20, 21, 22, 23, 25], "AND": 2, "And": [0, 3, 4, 5, 6, 9, 13, 18, 21], "As": [0, 1, 2, 3, 4, 5, 6, 8, 10, 12, 13, 15, 16, 19, 21, 24, 25], "At": [0, 4, 6, 13, 24], "BE": [0, 24], "Be": [2, 18, 24], "Being": 13, "But": [0, 1, 2, 3, 5, 6, 9, 10, 16, 21, 25], "By": [0, 3, 5, 6, 12, 13, 17, 19, 24, 25], "For": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 19, 21, 23, 24, 25], "IF": 6, "IN": 23, "If": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 19, 21, 24, 25], "In": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 21, 23, 24, 25], "Ising": [5, 12, 25], "It": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 19, 21, 24, 25], "Its": [1, 2, 4, 11], "No": [6, 9, 24], "Not": [0, 1, 5, 6, 25], "OR": 21, "Of": 21, "On": [0, 3, 21, 22, 23, 24], "One": [0, 1, 3, 4, 5, 6, 7, 8, 11, 12, 13, 17, 21, 25], "Or": [0, 1, 6, 24], "Such": [0, 6, 12, 16, 21], "That": [0, 5, 7, 10, 11, 12, 14, 21, 24], "The": [4, 10, 13, 14, 16, 17, 19, 20, 21, 22, 23], "Then": [0, 1, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 24], "There": [0, 3, 4, 5, 6, 8, 9, 11, 12, 14, 15, 19, 21, 22, 24, 25], "These": [0, 3, 4, 5, 8, 9, 10, 11, 12, 13, 14, 17, 19, 21, 22, 24, 25], "To": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 19, 21, 25], "With": [0, 5, 6, 8, 9, 10, 11, 12, 14, 16, 19, 21, 24, 25], "_": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 19, 24, 25], "_0": [5, 8, 10, 11, 13, 25], "_1": [2, 5, 6, 8, 10, 11, 12, 13, 14, 19, 25], "_2": [2, 5, 8, 11, 12, 13, 19, 25], "_3": 19, "_4": 19, "_9": 13, "__class__": 10, "__doc__": 6, "__future__": [8, 9], "__init__": 1, "__main__": 2, "__name__": [2, 10], "_auto1": [2, 3, 4, 5, 6, 7, 12, 13, 19, 21, 25], "_auto10": [6, 12], "_auto11": 6, "_auto12": 6, "_auto2": [2, 3, 4, 5, 6, 12, 13, 19, 21], "_auto3": [3, 4, 5, 6, 12, 13, 19], "_auto4": [4, 6, 12, 13, 19], "_auto5": [4, 6, 12, 13, 19], "_auto6": [4, 6, 12, 19], "_auto7": [4, 6, 12, 19], "_auto8": [6, 12], "_auto9": [6, 12], "_build": [0, 18, 23, 24], "_c": 1, "_compon": 11, "_depth": 9, "_export": [15, 16], "_fraction": 9, "_i": [0, 1, 2, 5, 6, 7, 8, 11, 12, 13, 24, 25], "_j": [0, 1, 2, 3, 5, 6, 8, 13, 25], "_k": 13, "_l": 12, "_lambda": 6, "_leaf": 9, "_m": 10, "_multilayer_perceptron": [], "_n": [2, 5, 8, 11, 13, 25], "_node": 9, "_p": [5, 8, 25], "_ratio": 11, "_sampl": 9, "_split": [6, 9], "_t": 13, "_test": 6, "_varianc": 11, "_weight": 9, "a0": 3, "a0faa0": [9, 10], "a1": [0, 24], "a2": [0, 24], "a3": [0, 24], "a4": [0, 24], "a_": [0, 1, 16, 19, 24, 25], "a_0": [0, 24], "a_1a": [0, 24], "a_2a": [0, 24], "a_3": [0, 24], "a_3a": [0, 24], "a_4": [0, 24], "a_4a": [0, 24], "a_h": 1, "a_i": [0, 1, 2, 12, 24], "a_j": [1, 12], "a_k": [0, 1, 12], "aaron": 23, "ab": [0, 2, 5, 13, 14, 24, 25], "ab_channel": 18, "abandon": 1, "abid": 21, "abil": [0, 10], "abl": [0, 1, 4, 5, 6, 7, 10, 12, 13, 16, 25], "about": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 19, 22], "abov": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 21, 23, 24, 25], "abovement": [6, 24], "abscissa": 13, "absolut": [0, 2, 5, 6, 13, 24, 25], "absorb": 25, "abstract": 1, "acceler": 13, "accept": [0, 3, 6, 9, 25], "access": [3, 11, 21, 24], "accid": [4, 6], "accompani": [0, 24, 25], "accomplish": [8, 9, 13], "accord": [0, 1, 2, 5, 6, 9, 12, 13, 14, 21, 24], "accordingli": 11, "account": [0, 3, 5, 13, 15, 16, 21, 24], "accumul": [12, 13, 21], "accur": [0, 3, 4, 6, 10, 13], "accuraci": [0, 1, 3, 4, 5, 6, 7, 9, 10, 11, 12, 24, 25], "accuracy_scor": [0, 1, 10, 24], "accuracy_score_numpi": 1, "achiev": [0, 1, 5, 6, 8, 12, 19, 24], "aco": 21, "acquaint": 18, "acquir": [1, 18, 24], "acr": [], "across": [1, 3, 6, 9, 17, 18, 24], "act": [1, 3, 19], "action": 21, "activ": [0, 2, 3, 4, 9, 15, 20, 22, 24], "actual": [0, 1, 4, 5, 6, 8, 11, 15, 16, 19, 21, 24, 25], "ad": [1, 3, 4, 5, 8, 13, 15, 16, 19], "ada_clf": 10, "adaboostclassifi": 10, "adadelta": 13, "adam": [1, 3, 4, 24], "adapt": [4, 6, 13, 17, 23], "add": [0, 1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 15, 16, 17, 21, 22, 24, 25], "add_subplot": [1, 7, 12, 14], "addendum": 5, "addit": [0, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 15, 18, 19, 21, 22, 23, 24, 25], "addition": [12, 13], "address": [1, 9, 11, 13, 24], "adjac": [3, 12], "adjoint": [5, 25], "adjust": [0, 5, 12, 13], "admir": [0, 24], "advanc": [4, 6, 12, 23, 24], "advantag": [1, 3, 5, 6, 10, 13, 19], "adversari": 24, "afecionado": 24, "affect": [3, 15], "affin": [0, 3, 8, 11, 25], "afford": 3, "aficionado": 24, "aforement": 14, "african": [], "after": [0, 1, 2, 4, 5, 6, 9, 11, 12, 13, 15, 16, 17, 18, 19, 21, 24, 25], "afterward": [0, 24], "ag": [0, 7, 24, 25], "ag_0": 2, "again": [0, 1, 4, 5, 6, 7, 8, 10, 11, 12, 13, 21, 24, 25], "against": [1, 4, 7, 10], "agegroup": 7, "agegroupmean": 7, "aggreg": [9, 10], "agorithm": 10, "agre": [5, 6, 21, 25], "agreement": 13, "ahead": 9, "ai": [0, 23], "aid": 11, "aim": [0, 1, 4, 6, 7, 11, 14, 16, 17, 18, 19, 25], "ainv": 5, "airplan": 3, "aka": 5, "al": [0, 2, 4, 16, 17, 23, 24, 25], "alarm": [5, 7], "aldo": 25, "algebra": [0, 3, 5, 13, 18, 25], "algorithm": [0, 1, 2, 4, 5, 6, 7, 8, 13, 14, 16, 18, 19, 21, 23], "align": [0, 2, 5, 6, 7, 8, 13, 21, 24, 25], "all": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 18, 19, 20, 21, 22, 23, 24, 25], "allevi": [1, 13], "alloc": [3, 19], "allow": [0, 1, 2, 3, 5, 6, 8, 10, 13, 15, 18, 19, 24, 25], "almost": [0, 1, 6, 8, 11, 13, 21], "alon": [2, 9], "along": [2, 3, 4, 5, 6, 9, 10, 11, 15, 18, 19, 24, 25], "alpha": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 13, 14, 21, 24, 25], "alpha_": 10, "alpha_0": 3, "alpha_1": 3, "alpha_2": 3, "alpha_i": [3, 13], "alpha_k": 13, "alpha_m": 10, "alpha_n": 3, "alpha_opt": 13, "alreadi": [2, 3, 4, 5, 6, 10, 12, 15, 18, 19, 21, 24, 25], "also": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 21, 24, 25], "alter": 1, "altern": [0, 1, 4, 5, 6, 8, 9, 11, 13, 15, 19, 24, 25], "although": [0, 1, 5, 6, 8, 10, 13, 16, 24], "alwai": [0, 3, 5, 6, 12, 13, 16, 21, 24, 25], "am": 4, "ame2016": [0, 24], "american": [], "among": [0, 3, 5, 9, 10, 12, 19, 24, 25], "amongst": 5, "amount": [0, 1, 3, 4, 6, 8, 10, 14, 18], "an": [1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 14, 16, 17, 18, 19, 21, 22, 23, 25], "an_": 21, "anaconda": [0, 1, 18, 24], "analogi": 13, "analys": 6, "analysi": [1, 3, 4, 7, 14, 19, 23], "analyt": [2, 3, 5, 6, 7, 12, 13, 17, 18, 24, 25], "analyz": [0, 1, 3, 4, 5, 6, 16, 21, 25], "andrew": 1, "angl": [0, 3, 9, 25], "anharmon": 3, "ani": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 14, 15, 16, 21, 24, 25], "anim": [4, 12], "ann": 12, "annot": [0, 1, 3, 7, 8, 24], "announc": 24, "anoth": [0, 1, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 19, 21, 24, 25], "ansatz": [0, 24], "answer": [0, 1, 3, 5, 6, 19, 22, 24], "antialias": [2, 6], "anticip": 4, "anymor": [1, 8], "anyon": [4, 8, 15], "anyth": [1, 15, 16, 21], "anytim": [22, 24], "apach": 1, "apart": [11, 13], "api": [1, 18, 24], "appar": 2, "appear": [0, 1, 3, 13, 19, 21], "append": [1, 3, 4, 8, 9, 13, 24], "appli": [0, 1, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 21, 23, 24, 25], "applic": [0, 1, 3, 4, 5, 6, 7, 9, 12, 13, 16, 19, 21, 23, 24, 25], "apply_gradi": 4, "approach": [1, 2, 4, 5, 6, 9, 10, 11, 12, 13, 15, 16, 18, 21, 23, 25], "appropri": [2, 6, 9, 12, 13, 17, 18, 21], "approv": 24, "approx": [0, 2, 3, 6, 10, 11, 13, 21, 24], "approxim": [0, 1, 2, 3, 4, 5, 6, 7, 10, 11, 13, 21, 24, 25], "apt": [0, 18, 24], "aq": 21, "ar": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25], "aragorn": 24, "arang": [1, 3, 4, 6, 7, 9, 10, 12, 13, 24], "arbitrari": [1, 4, 6, 8, 12, 13, 21], "arbitrarili": [0, 1, 11, 24], "arc": 6, "architectur": [3, 4, 12], "area": [0, 3, 6, 23, 24], "argmax": [1, 11], "argmin": [4, 10, 14], "argsort": 11, "argu": [1, 13], "argument": [0, 2, 3, 5, 11, 12, 13, 17, 24, 25], "aris": [0, 6, 12, 13, 21, 24], "arithmet": [0, 13, 19, 24], "arm": 6, "armadillo": 19, "around": [0, 1, 4, 5, 6, 11, 21, 24], "arrai": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 13, 14, 16, 18, 21, 25], "arrang": [3, 24], "arraybox": 13, "arriv": [0, 6, 9, 11, 19, 21, 24], "arrow": 12, "arrowprop": 8, "art": [0, 1, 18], "articl": [0, 3, 4, 6, 10, 24, 25], "artifici": [0, 2, 7, 12, 23, 24], "artificialneuron": 12, "arug": 13, "arxiv": [3, 4], "asarrai": [0, 6, 9], "asid": 25, "ask": [5, 6, 11, 12, 15], "aspect": [0, 6, 18, 24], "assembl": 3, "assembli": [0, 24], "assert": 4, "assess": [0, 6, 24], "assici": 4, "assign": [0, 7, 8, 9, 12, 13, 14, 15, 20, 22, 23, 24], "associ": [0, 6, 9, 12, 14, 21, 24], "assum": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 19, 21, 24, 25], "assumpt": [0, 3, 5, 6, 9, 11, 21, 24], "ast": [0, 5, 6, 24], "astyp": [4, 9, 10], "asymmetri": [0, 24], "asymptot": [4, 6], "atom": [0, 24], "attempt": [0, 4, 6, 7, 8, 10, 24], "attend": 24, "attent": [0, 19, 24], "attract": [0, 10, 24], "attribut": [0, 9, 24], "audi": [0, 24], "audio": [3, 4], "august": [24, 25], "aurelien": [0, 23, 24], "austfjel": 6, "auth": 15, "authent": 15, "author": [0, 1, 10, 21], "authour": 24, "auto": [9, 10, 21], "autocor": 21, "autocorrelation_tim": 21, "autocorrelform": 21, "autocovari": 21, "autoencod": [4, 18, 24], "autoencond": 18, "autograd": [18, 24], "autom": [0, 18, 23, 24], "automac": 19, "automag": 24, "automat": [0, 1, 2, 3, 4, 11, 16, 18, 19, 24], "automobil": 3, "autonom": 4, "avail": [0, 1, 4, 6, 10, 11, 18, 19, 20, 22, 23, 24], "averag": [0, 1, 3, 6, 9, 10, 13, 14, 21, 22, 24], "avoid": [0, 4, 5, 6, 9, 11, 13, 19, 25], "awai": [2, 3, 6, 25], "awar": [2, 10], "award": [22, 24], "ax": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 24], "axes3d": [2, 6, 13], "axes_grid1": 6, "axhlin": 8, "axi": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 21, 24], "axiom": 5, "axvlin": [4, 8], "axvspan": 4, "b": [0, 1, 3, 4, 5, 6, 8, 9, 10, 12, 13, 14, 15, 16, 17, 21, 22, 24, 25], "b1": 8, "b2": 8, "b3": 8, "b_": [0, 1, 19], "b_0": 0, "b_1": [0, 2, 12, 13], "b_2": [0, 13], "b_5": 13, "b_group": 9, "b_i": [0, 1, 2, 12, 24], "b_ia_": [0, 24], "b_ia_i": 0, "b_index": 9, "b_j": [1, 12], "b_k": [0, 1, 12, 13], "b_m": 12, "b_score": 9, "b_valu": 9, "babcock": 24, "bachelor": [20, 22], "back": [0, 3, 4, 5, 6, 8, 9, 10, 15, 16, 19, 21, 24], "backbon": 19, "backend": [1, 4], "background": [23, 24], "backpropag": 1, "backtrack": 9, "backup": 19, "backward": [1, 2, 4, 12, 19], "bad": [6, 17], "badli": 21, "bag": [9, 18, 24], "bag_clf": 10, "baggin": 24, "baggingboot": 10, "baggingclassifi": 10, "baggingtre": 10, "balanc": 6, "band": 19, "bandwidth": 19, "bar": [0, 6, 11, 24], "barber": 23, "bare": [4, 10], "base": [0, 1, 3, 4, 5, 7, 8, 9, 10, 14, 15, 16, 17, 18, 21, 22, 23, 24, 25], "basi": [5, 7, 8, 10, 11, 12, 13, 19, 25], "basic": [6, 8, 12, 13, 14, 15, 18, 21, 24], "batch": [3, 4, 11, 12, 13], "batch_shap": 4, "batch_siz": [1, 3, 4], "batchnorm": 4, "bay": 7, "bayesian": [5, 18, 23, 24], "becaus": [0, 1, 2, 3, 4, 5, 6, 8, 9, 12, 13, 14, 24], "becom": [0, 1, 2, 5, 6, 7, 9, 12, 13, 21, 24, 25], "been": [0, 1, 2, 3, 4, 5, 6, 11, 12, 13, 18, 19, 24], "befor": [0, 1, 2, 3, 4, 5, 6, 7, 8, 12, 13, 14, 16, 17, 19, 21, 24, 25], "beforehand": [0, 21, 24], "begin": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 19, 21, 22, 24, 25], "behav": [1, 6, 13], "behavior": [0, 1, 13, 24], "behaviour": 12, "behind": [0, 1, 6, 8, 13, 24], "being": [0, 1, 2, 3, 4, 5, 7, 8, 10, 11, 12, 13, 17, 21, 24, 25], "believ": [9, 19], "belong": [7, 8, 9, 13, 14], "below": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 19, 21, 24, 25], "benchmark": 10, "benefici": [1, 13], "benefit": [0, 1, 4, 11, 13, 18, 24], "bengio": [1, 23, 24, 25], "benign": [1, 7], "besid": [4, 5], "bessel": [5, 25], "best": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 15, 16, 22, 24, 25], "beta": [1, 3, 10, 11, 13, 16, 17, 24], "beta_": [3, 13, 17], "beta_0": [1, 3, 13], "beta_1": [1, 3, 10, 13], "beta_1x_i": 13, "beta_2": [3, 13], "beta_3": 3, "beta_i": 3, "beta_j": 13, "beta_k": 13, "beta_linreg": 13, "beta_m": 10, "beta_mg_m": 10, "beta_n": 3, "better": [0, 1, 2, 3, 4, 6, 9, 10, 11, 12, 13, 24, 25], "between": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 21, 24, 25], "beyond": [0, 1, 5, 6, 8, 13, 24, 25], "bf": [13, 14, 19, 21], "bg": 24, "bgd": 13, "bia": [0, 1, 2, 3, 5, 8, 9, 10, 12, 13, 24, 25], "bias": [1, 2, 3, 5, 6, 9, 12], "big": [0, 1, 2, 5, 6, 14], "bigger": [1, 6], "bigr": 12, "bike": 9, "bilbo": 24, "billion": [3, 12, 18], "bin": [7, 21], "binari": [0, 3, 5, 7, 9, 10, 12, 24], "binarycrossentropi": 4, "bind": 0, "binomi": [18, 21, 24], "binsboot": 6, "bioinformat": 0, "biolog": [1, 12], "bios1100": [18, 24], "bird": [0, 3], "birth": 24, "bishop": [23, 24], "bit": [1, 4, 19, 21, 24], "bitwis": 21, "bivari": 2, "bk": 13, "bla": [19, 24], "black": [8, 9, 14], "block": [6, 10, 18, 19, 21, 24], "blog": 24, "blogpost": 4, "blue": [0, 3], "bmatrix": [0, 1, 3, 5, 7, 8, 11, 13, 19, 24, 25], "bmi": 1, "bodi": [0, 1, 4, 12], "bold": 1, "boldfac": [0, 5, 16, 25], "boldsymbol": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 13, 14, 16, 17, 24], "boltzmann": [12, 18, 24], "book": [17, 23, 24, 25], "book1": 23, "boolean": [4, 17], "boost": [1, 9, 18, 24], "boostrap": 10, "bootstrap": [1, 13, 18, 24], "borrow": 24, "boston_dataset": [], "bot": 8, "both": [0, 1, 4, 5, 6, 8, 9, 10, 13, 14, 15, 16, 17, 18, 19, 21, 22, 24, 25], "bottl": 7, "bound": [8, 12], "boundari": [2, 4, 8, 11, 12], "box": [4, 9], "boyd": [8, 13], "bracket": [4, 21], "brain": [1, 7, 12], "branch": [9, 24], "break": [0, 4, 6, 11, 14, 24], "breast": [5, 7, 11], "breviti": 13, "brew": [0, 18, 24], "brg": 8, "brief": 25, "briefli": [0, 16, 24], "bring": [0, 5, 6, 10], "britt": [22, 24], "broad": 0, "broadli": 24, "brought": [13, 18, 24], "brownle": 4, "browser": [15, 24], "brute": [3, 5, 11, 25], "buffer_s": 4, "bui": 4, "build": [0, 4, 5, 6, 10, 16, 19, 21, 24], "built": [1, 3, 4, 6], "bunch": 11, "busi": [], "byte": [19, 24], "c": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25], "c1": [8, 11], "c2": [8, 11], "c_": [8, 9, 10, 13, 21], "c_0": 21, "c_1": 12, "c_2": 12, "c_3": 12, "c_4": 12, "c_i": [12, 13], "c_k": 21, "ca": [1, 24], "cach": 10, "cal": [0, 8, 10, 12, 13], "calcul": [0, 1, 2, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 16, 19, 21, 24], "call": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 19, 21, 22, 24, 25], "calor": [0, 25], "cambridg": [13, 23], "can": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23, 25], "cancel": [0, 13, 24, 25], "cancer": [5, 10], "cancerpd": 7, "candid": [8, 9, 10], "cannot": [0, 1, 4, 5, 6, 7, 8, 9, 21, 25], "canopi": [0, 18, 24], "canva": [15, 16, 24], "cap": 5, "capabl": [0, 1, 8, 13, 18, 24], "capac": [2, 22], "capita": [], "captur": [4, 11, 12, 24], "car": [3, 4], "card": [0, 7, 24], "cardin": 1, "care": [11, 15], "carefulli": 13, "carlo": [0, 6, 18, 21, 23, 24], "carri": [2, 6, 7], "cart": 10, "case": [0, 1, 2, 3, 4, 5, 6, 7, 11, 12, 13, 14, 15, 16, 18, 19, 24], "casella": 23, "cast": 1, "cat": [3, 4], "catch": 0, "categor": [0, 1, 3, 9, 11, 24], "categori": [0, 1, 3, 7, 10, 12, 14, 24], "categorical_crossentropi": [1, 3], "caus": [0, 5, 6, 21, 24, 25], "causal": 0, "causat": [0, 24], "cax": 1, "cb": [6, 24], "cbar": 1, "cc": [0, 1, 5, 13, 24, 25], "ccc": [5, 12], "cdf": 21, "cdot": [0, 2, 6, 12, 13, 14, 19, 21, 24], "celebr": 13, "cell": 4, "center": [0, 1, 6, 7, 8, 9, 11, 14, 21, 24], "central": [0, 3, 5, 6, 8, 16, 19, 24, 25], "centroid": [14, 21], "centroid_differ": 14, "centuri": 3, "certain": [0, 3, 6, 7, 9, 21, 24, 25], "cg": 13, "cha": [], "chain": [0, 1, 13, 18, 21, 24], "challeng": 15, "chanc": [1, 5, 13, 21], "chang": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 13, 14, 15, 16, 19, 21, 24, 25], "channel": 3, "chapter": [0, 6, 10, 11, 16, 17, 19, 23, 24, 25], "chapter3": 0, "charact": [0, 3, 5, 24, 25], "character": [8, 9, 10, 12, 21], "characterist": [0, 1, 3, 10, 13, 24], "charg": [0, 24], "charl": [], "chase": 4, "chatgpt": 15, "chd": 7, "chddata": 7, "cheap": [5, 25], "cheaper": [1, 13], "check": [1, 3, 4, 5, 11, 13, 15, 16, 19, 24], "checkmark": 3, "checkpoint": 4, "checkpoint_dir": 4, "checkpoint_prefix": 4, "chen": 10, "cheng": 25, "chiaramont": 2, "childcar": 16, "children": 16, "choic": [0, 1, 2, 3, 4, 6, 9, 12, 13, 14, 19, 24, 25], "choleski": [5, 19, 25], "choos": [2, 3, 6, 9, 10, 11, 13, 14, 15], "chosen": [0, 1, 2, 6, 8, 9, 10, 13, 16, 21, 24], "chosen_datapoint": 1, "christian": 23, "christoph": [23, 24], "cifar": 3, "cifar10": 3, "circ": [1, 12], "circl": [0, 8, 12, 25], "circuit": 3, "circumfer": 9, "circumv": [1, 5, 13, 25], "ckpt": 4, "clariti": 21, "class": [0, 1, 3, 4, 6, 7, 8, 9, 11, 12, 13, 21, 24], "class_nam": [3, 9], "class_val": 9, "class_valu": 9, "classic": [7, 9, 13], "classif": [0, 3, 5, 6, 7, 8, 11, 12, 18, 23, 24, 25], "classifi": [0, 1, 4, 7, 9, 10, 11, 24], "classificaton": 1, "classifii": 10, "clean": 1, "clear": [1, 5, 10, 12, 13], "clearli": [0, 3, 5, 6, 7, 8, 21], "clever": [1, 10], "clf": [0, 6, 8, 9, 10, 24, 25], "clf3": 0, "clf_lasso": 6, "clf_ridg": 6, "cli": 15, "clip": [3, 21], "clone": [15, 22], "close": [0, 1, 2, 4, 6, 8, 9, 11, 12, 13, 14, 21, 23, 24], "closer": [3, 5, 13, 25], "closest": [8, 11, 13, 14], "closur": [18, 24], "cloud": [18, 24], "cluster": [0, 1, 4, 6, 11, 18, 24], "cluster_label": 14, "cm": [1, 2, 3, 6, 8, 13], "cmap": [0, 1, 2, 3, 4, 6, 8, 9, 10, 24], "cmap_arg": 6, "cmd": [9, 15], "cn_": 21, "cnn": 12, "cnn_kera": 3, "cntk": [18, 24], "co": [0, 2, 3, 6, 9, 13, 24], "code": [0, 3, 4, 6, 7, 8, 18, 19, 21, 23], "coef": [0, 24], "coef0": 8, "coef_": [0, 5, 6, 8, 9, 13, 16, 24, 25], "coeff": 5, "coeffici": [0, 3, 5, 6, 7, 8, 9, 13, 19, 24], "coerc": [0, 6, 24], "coin": [10, 21], "coin_toss": 10, "col": [0, 11, 24, 25], "colab": [18, 24], "cold": 9, "colinear": [], "collaps": 8, "collect": [2, 6, 10, 11, 17, 18, 21, 23, 24], "collinear": [5, 25], "color": [0, 3, 4, 6, 8, 9, 10, 21], "color_channel": 3, "color_cod": 6, "colorbar": [1, 6], "colsample_bytre": 10, "colsaobject": 10, "column": [0, 1, 2, 5, 6, 7, 8, 9, 11, 12, 16, 17, 19, 24, 25], "columntransform": 9, "com": [4, 6, 15, 16, 18, 23, 24], "combin": [1, 2, 5, 6, 7, 10, 15, 21], "come": [0, 1, 3, 4, 5, 12, 13, 14, 15, 24, 25], "command": [0, 1, 15], "comment": [0, 4, 5, 6], "commerci": [0, 18, 24], "commit": 15, "commod": [0, 24], "common": [0, 1, 3, 5, 6, 7, 9, 11, 13, 14, 16, 21, 24, 25], "commonli": [0, 1, 4, 6, 7, 9, 13, 14, 25], "commun": [0, 12, 15], "commut": 3, "commutatitav": 3, "compact": [0, 1, 3, 5, 6, 7, 9, 11, 12, 13, 14, 24, 25], "compair": 0, "compar": [0, 3, 4, 5, 6, 11, 13, 19, 24, 25], "comparison": [2, 4, 13], "compat": 7, "compet": 0, "competit": 10, "compil": [0, 1, 3, 4, 13, 18, 19, 24], "complet": [0, 2, 3, 4, 9, 12, 15, 16, 17, 24], "completenn": 12, "complex": [1, 5, 8, 9, 11, 12, 13, 16, 24], "complic": [0, 1, 9, 13, 24], "compoment": 25, "compon": [0, 1, 3, 4, 5, 6, 7, 9, 14, 16, 18, 24, 25], "components_": 11, "compos": [9, 12, 13, 14, 18, 24], "compphys": [0, 6, 16, 18, 20, 22, 23, 24], "compress": [0, 24, 25], "compris": 6, "compromis": [5, 25], "compulsori": [18, 24], "comput": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 23, 24, 25], "computation": [0, 3, 6, 9, 13, 21, 24], "computationalscienceuio": 24, "concaten": [2, 4, 6, 14], "concav": [1, 13, 25], "concentr": 10, "concept": [0, 2, 18, 24, 25], "conceptu": [12, 13], "concern": [0, 1, 4, 7, 24], "concic": 24, "conclud": [0, 5, 13], "conclus": 1, "cond": 2, "conda": [0, 1, 18, 24], "condis": 25, "condit": [0, 2, 4, 5, 6, 8, 9, 11, 13, 21, 24, 25], "conduct": 18, "condwav": 2, "confid": [0, 5, 6, 7, 8, 24, 25], "configur": 3, "confirm": [5, 12], "confus": [5, 6, 7, 10, 19], "confusion_matrix": 9, "congruenti": 21, "conjug": [4, 8], "conjugaci": 13, "conjunct": 3, "connect": [0, 1, 3, 4, 9, 11, 12, 13, 19, 24, 25], "consequ": [5, 6, 8, 10, 12, 13, 25], "conserv": [5, 14, 25], "consid": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 16, 19, 21, 24, 25], "consider": [0, 1, 5, 13, 24, 25], "consist": [1, 2, 3, 4, 6, 12, 13, 21, 25], "constant": [0, 2, 4, 5, 6, 8, 12, 13, 16, 21, 24, 25], "constitu": [0, 24], "constitut": [2, 6], "constrain": [1, 3, 5, 7, 11], "constraint": [5, 6, 8, 13, 25], "construct": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 19, 21, 24, 25], "contact": [0, 24], "contain": [0, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 19, 21, 23, 24, 25], "contemporari": 24, "content": [1, 15, 18, 19, 24], "context": [6, 10, 13], "contigu": 19, "continu": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 19, 21, 24, 25], "contour": [9, 10, 13], "contourf": [8, 9, 10], "contrast": [1, 4, 9, 10, 12, 24], "contribut": [0, 3, 5, 13, 21, 24, 25], "contributor": 0, "control": [0, 1, 3, 9, 13, 15, 18, 24], "conv": [3, 4], "conv2d": [3, 4], "conv2dtranspos": 4, "convei": 24, "conveni": [5, 6, 12, 13, 19, 24], "convent": [12, 25], "converg": [1, 2, 4, 5, 8, 13, 14, 25], "convergencewarn": [], "convert": [0, 1, 4, 5, 9, 11, 13, 19, 24, 25], "converttomatrix": 4, "convex": [4, 5, 7, 25], "convinc": 13, "convolut": [1, 4, 18, 24], "cool": [4, 9], "coolwarm": 6, "coordin": [5, 12, 14, 25], "coorel": [], "copi": [0, 1, 14, 15, 25], "core": 10, "corel": 24, "coronari": 7, "corr": [5, 7, 11, 25], "correalt": [11, 18], "correct": [0, 1, 2, 3, 4, 5, 7, 13, 15, 19, 21, 24, 25], "correctli": [1, 2, 6, 7, 10], "correl": [0, 1, 3, 5, 6, 7, 10, 12, 13, 18, 21, 24], "correlation_matrix": [5, 7, 11, 25], "correspond": [0, 3, 5, 6, 8, 9, 11, 12, 18, 19, 21, 24, 25], "cortex": 12, "cosin": [3, 6], "cost": [0, 2, 3, 5, 6, 7, 8, 9, 12, 13, 16, 17, 24], "cost_deep_grad": 2, "cost_funct": 2, "cost_function_deep": 2, "cost_function_deep_grad": 2, "cost_function_grad": 2, "cost_grad": 2, "cost_sum": 2, "costol": 13, "could": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 19, 21, 24, 25], "coulomb": [0, 24], "count": [0, 9, 15, 20, 21, 22, 24], "counterpart": 24, "countor": 13, "coupl": [4, 5, 6], "cours": [0, 1, 3, 5, 11, 15, 16, 17, 22, 25], "coursework": 15, "courvil": [23, 24, 25], "cov": [5, 6, 11, 19, 21, 24, 25], "cov_xi": [5, 11, 25], "cov_xx": [5, 11, 25], "cov_yi": [5, 11, 25], "covari": [0, 7, 18, 19, 24], "covariance_matrix": [5, 11, 14], "cover": [0, 5, 18, 22, 23, 25], "covert": [0, 24], "covxi": 21, "covxx": 21, "covxz": 21, "covyi": 21, "covyz": 21, "covzz": 21, "cpu": 1, "craft": 3, "creat": [1, 3, 4, 5, 9, 10, 11, 12, 15, 18, 24], "create_biases_and_weight": 1, "create_convolutional_neural_network_kera": 3, "create_neural_network_kera": 1, "create_x": [5, 11], "credit": [0, 7, 22, 24], "crim": [], "crime": [], "criteria": [0, 4, 9, 10, 14, 21, 24], "criterion": [9, 10, 13], "critic": 6, "cross": [0, 1, 3, 7, 9, 10, 13, 15, 18, 21, 24, 25], "cross_entropi": 4, "cross_val_scor": 6, "cross_valid": [7, 10], "crossvalid": 6, "crucial": [1, 21], "cs231": 3, "csr_matrix": [19, 24], "csv": [0, 4, 6, 7, 9], "ctnk": 1, "cubic": 0, "cumbersom": 5, "cumsum": [10, 11, 24], "cumul": [7, 10, 21], "cumulative_heads_ratio": 10, "cup": 5, "current": [1, 2, 3, 4, 13, 14, 15, 16, 23], "curs": [0, 25], "curv": [6, 7, 10, 12], "curvatur": 13, "custom": [6, 14], "custom_cmap": [9, 10], "custom_cmap2": [9, 10], "cutpoint": 9, "cv": [6, 7, 10], "cvxbook": 13, "cvxopt": [5, 8, 25], "cycl": [1, 12], "d": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 19, 21, 22, 24, 25], "d2_g_t": 2, "d_f": 13, "d_g_t": 2, "d_net_out": 2, "da": 3, "dagger": [5, 19, 25], "dai": [1, 9, 18], "damp": 3, "darget": 9, "darkr": 21, "dat": [0, 24], "dat_id": [0, 6, 7, 9, 24], "data": [2, 4, 5, 8, 10, 12, 13, 14, 16, 19, 23], "data1": 14, "data2": 14, "data3": 14, "data4": 14, "data_id": [0, 6, 7, 9, 24], "data_indic": 1, "data_panda": 24, "data_path": [0, 6, 7, 9, 24], "databas": 1, "datafil": [0, 6, 7, 9, 24], "datafram": [0, 4, 5, 7, 9, 11, 24, 25], "datapoint": [1, 5, 6, 7, 11, 13, 16], "datasci": [15, 16], "dataset": [0, 4, 6, 7, 8, 9, 10, 11, 13, 14, 16, 24], "datatyp": 4, "date": [15, 24, 25], "daughter": 10, "david": 23, "dbh": 1, "dbo": 1, "dcomposit": 19, "ddot": 2, "dead": 1, "deadlin": 15, "deal": [0, 1, 3, 5, 6, 8, 11, 13, 14, 19, 21, 24, 25], "dealt": 0, "debt": 7, "debug": [0, 5, 6, 25], "decad": [0, 3], "decai": [0, 13, 21, 24], "decemb": [22, 24], "decent": 10, "decid": [0, 2, 3, 5, 6, 9, 25], "decim": [0, 24], "decis": [0, 1, 8, 11, 18, 23, 24], "decision_funct": 8, "decision_tre": 9, "decisiontreeclassifi": [9, 10], "decisiontreeregressor": [0, 9, 10], "declar": [0, 4, 19, 24], "decompos": [5, 6, 19, 25], "decomposit": [0, 6, 12, 24], "decompost": [5, 25], "deconvolut": 3, "decorrel": [10, 13], "decreas": [1, 2, 4, 5, 6, 10, 11, 13], "deduc": [0, 24], "deep": [3, 7, 12, 13, 18, 23, 25], "deep_neural_network": 2, "deep_param": 2, "deep_tree_clf": [9, 10], "deep_tree_clf1": 9, "deep_tree_clf2": 9, "deepen": [5, 18, 24], "deeper": [0, 3, 4, 24], "deeplearningbook": [23, 24], "deer": 3, "def": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 16, 17, 21, 24, 25], "def_covari": 21, "default": [0, 1, 2, 4, 6, 7, 19, 24], "default_tim": 4, "defect": [5, 25], "defici": [5, 25], "defin": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 19, 21, 25], "definit": [1, 2, 5, 6, 7, 8, 10, 11, 12, 13, 19, 21, 25], "defint": 21, "degre": [3, 5, 6, 8, 9, 10, 11, 15, 16, 21, 24], "deisenroth": 25, "del": 1, "delet": [6, 15], "delimit": 4, "deliv": [15, 20, 24], "delta": [0, 2, 3, 6, 8, 12, 13, 14, 24], "delta_": [1, 19], "delta_0": 3, "delta_1": 3, "delta_2": 3, "delta_3": 3, "delta_4": 3, "delta_5": 3, "delta_h": [0, 1, 24], "delta_j": [3, 12], "delta_k": 12, "delta_l": [1, 3], "delta_momentum": 13, "delta_n": [0, 3, 24], "delug": 18, "delv": 0, "demand": 13, "demonstr": [0, 3, 5, 6, 7, 11, 12, 18, 24, 25], "den": 4, "denomin": [1, 5], "denot": [1, 2, 6, 7, 13, 21], "dens": [1, 3, 4], "densiti": [0, 2, 6, 21], "depart": [22, 24, 25], "depend": [0, 1, 2, 4, 5, 6, 7, 8, 11, 12, 13, 15, 16, 18, 19, 21, 24, 25], "depict": 21, "deploy": [0, 18, 24], "depth": [0, 3, 9, 10, 19], "deriv": [0, 1, 2, 6, 7, 8, 10, 11, 13, 18, 24], "derivati": 13, "derivative_fn": 13, "descend": [5, 9, 11, 25], "descent": [0, 1, 3, 7, 8, 12, 24, 25], "describ": [0, 2, 4, 5, 6, 8, 10, 11, 12, 13, 19, 24], "descript": [0, 8, 9, 24], "design": [0, 1, 3, 4, 5, 6, 7, 10, 11, 12, 13, 17, 24], "designmatrix": [0, 24], "desir": [0, 2, 4, 5, 13, 14, 24, 25], "desktop": 15, "despit": [1, 12], "destroi": 19, "det": [5, 19, 25], "detail": [0, 6, 11, 13, 14, 19, 25], "detect": [3, 8, 12], "determin": [0, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 19, 21, 24, 25], "determinist": [7, 13, 21], "dev": 1, "develop": [0, 3, 5, 8, 10, 11, 12, 18, 19, 24, 25], "deviat": [0, 1, 2, 4, 5, 6, 17, 21, 24, 25], "devis": 12, "df": [4, 8, 11, 13, 24], "df1": 24, "di": [], "diag": [5, 8, 25], "diagnost": [1, 10], "diagon": [0, 5, 7, 13, 19, 21, 24, 25], "diagonaliz": [5, 25], "diagram": 10, "diagsvd": 6, "dice": [6, 21], "dict": [6, 8], "dictionari": [], "did": [0, 1, 5, 6, 7, 10, 11, 14, 16, 24], "die": 1, "diff": 2, "diff1": 2, "diff2": 2, "diff_ag": 2, "diffeent": 8, "differ": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 23, 24, 25], "differenti": [0, 3, 16, 18, 19, 24, 25], "difficult": [0, 1, 6, 10, 13, 21, 24], "difficulti": [0, 1, 13, 24], "diffonedim": 2, "digit": [0, 1, 3, 4, 6, 22, 24], "dilemma": 13, "dilut": 1, "dim": [4, 11, 14, 19], "dimens": [0, 1, 2, 3, 4, 5, 8, 11, 14, 16, 19, 24, 25], "dimension": [0, 4, 5, 6, 9, 11, 13, 14, 18, 19, 24, 25], "dimensionless": [0, 3, 24], "diment": 19, "dimnsion": 4, "diod": 3, "direct": [0, 1, 2, 4, 11, 12, 13, 14, 24, 25], "directli": [1, 4, 5, 6, 21, 25], "disadvantag": [0, 24], "disappear": [3, 6], "disc_loss": 4, "disc_tap": 4, "discard": [6, 11], "disciplin": [0, 3, 12], "disclaim": 21, "discord": 24, "discourag": [13, 15], "discov": [0, 24], "discover": 5, "discret": [1, 3, 5, 7, 13], "discrimin": [4, 7, 10, 11], "discriminator_loss": 4, "discriminator_loss_list": 4, "discriminator_model": 4, "discriminator_optim": 4, "discuss": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 18, 19, 21, 23, 24, 25], "diseas": 7, "disguis": 6, "disord": [1, 7], "displai": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 21, 24, 25], "displaystyl": [0, 5, 17, 24, 25], "disregard": [0, 24], "dissimilar": [11, 14], "dist": 14, "distanc": [8, 9, 11, 14, 21], "distance_list": 9, "distinct": [3, 7, 8, 9, 10, 14], "distinctli": 8, "distinguish": [0, 4, 7, 8, 21, 24], "distplot": [], "distribut": [0, 1, 4, 6, 7, 10, 11, 13, 14, 18, 19, 24, 25], "distrubut": [0, 18, 24], "dive": [0, 8, 19, 24], "diverg": [1, 13], "divid": [0, 1, 3, 5, 6, 7, 8, 9, 11, 12, 21, 24, 25], "divis": [6, 8, 9, 13, 19, 21], "dna": 7, "dnn": [0, 1, 2, 4, 12, 24], "dnn1": 4, "dnn2_gru2": 4, "dnn_kera": 1, "dnn_model": 1, "dnn_numpi": 1, "dnn_scikit": [0, 1, 24], "do": [0, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 24, 25], "doc": [0, 15, 16, 18, 20, 22, 23, 24], "document": [4, 13, 15], "doe": [0, 1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 15, 16, 17, 19, 21, 24], "doesn": [3, 9, 12, 24], "dog": [1, 3, 4], "domain": [5, 8, 13], "domin": [0, 24], "don": [0, 1, 3, 5, 6, 8, 11, 13, 15, 16, 18, 24, 25], "done": [0, 2, 3, 4, 5, 6, 9, 10, 11, 13, 16, 19, 24, 25], "dot": [0, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 19, 21, 24, 25], "doubl": [3, 4, 16, 19, 24], "doubli": 1, "down": [0, 3, 6, 9, 11, 12, 13], "download": [0, 1, 3, 5, 6, 15, 19, 23, 24], "downsampl": 3, "dozen": 1, "dq": 6, "drag": 13, "dramat": 11, "drastic": 4, "draw": [4, 6, 10, 13], "drawback": [0, 1, 3, 13, 25], "drawn": [1, 4, 6, 7, 11, 21, 24], "drive": [3, 4], "driven": 3, "drop": [0, 1, 5, 6, 11, 13, 21, 24, 25], "dropna": [0, 6, 24], "dropout": 4, "dt": [2, 3, 13, 21], "dtype": [0, 1, 3, 4, 14, 19, 24], "dub": [0, 24], "due": [1, 2, 5, 6, 8, 10, 12, 13, 22, 24, 25], "dummi": [], "dure": [0, 1, 3, 4, 8, 9, 11, 18, 24], "dwell": [], "dwh": 1, "dwo": 1, "dx": [2, 3, 8, 21], "dx_1": 21, "dx_1p": 6, "dx_2p": 6, "dx_mp": 6, "dx_n": 21, "dxp": 6, "dy": [1, 8, 21], "dynam": 4, "dz": 8, "e": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 21, 22, 24, 25], "e_": [0, 2, 24], "each": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 24, 25], "eapprox": [0, 24], "earli": [1, 13], "earlier": [0, 5, 7, 8, 9, 11, 12, 13, 24, 25], "earthexplor": 6, "eas": [6, 9, 14], "easi": [0, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 18, 19, 24, 25], "easier": [5, 6, 8, 9, 13, 15, 21, 24, 25], "easiest": 13, "easili": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 19, 24, 25], "eastern": [22, 24], "ebind": [0, 24], "eblock": 9, "econometr": 24, "economi": 5, "ecosystem": [18, 24], "ect": 20, "edg": 3, "edgecolor": 6, "editor": 15, "edu": 13, "educ": [0, 24], "eff": 21, "effect": [1, 4, 10, 13, 16, 17, 21], "effic": 1, "effici": [0, 3, 10, 13, 18, 19, 21, 24], "efron": 6, "egrad": 13, "eig": [5, 11, 13, 19, 21, 24, 25], "eigen": 21, "eigenpair": [5, 11, 25], "eigenvalu": [0, 5, 8, 11, 13, 19, 24, 25], "eigenvector": [5, 11, 13, 25], "eight": [19, 24], "eigval": [19, 21, 24], "eigvalu": [11, 13], "eigvec": [19, 21, 24], "eigvector": [11, 13], "eir": [22, 24], "eispack": [19, 24], "either": [1, 5, 6, 7, 8, 9, 10, 11, 13, 21, 24, 25], "eivind": 22, "eivinsto": 22, "ekstr\u00f8m": 4, "elabor": 21, "elarn": 3, "electr": [0, 3, 12, 24], "electron": 24, "eleg": 11, "element": [1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 18, 19, 23, 25], "elementari": [10, 13, 19], "elementwis": [3, 13], "elementwise_grad": [2, 13], "elessar": 24, "elif": 14, "elim": 19, "elimin": [3, 8], "elin": [22, 24], "ellipsi": 16, "els": [1, 3, 4, 7, 9, 12, 13, 16, 19], "elu": 1, "elus": [0, 24], "email": [20, 22, 24], "embed": [0, 11, 25], "embodi": 6, "emit": 21, "emner": 23, "emphas": [0, 10, 18, 24], "emphasi": [0, 18, 23, 24], "empir": [1, 11, 21], "emploi": [0, 1, 5, 6, 11, 13, 21, 24, 25], "employ": 0, "empti": [6, 10, 15], "emul": 12, "en": [18, 23], "enabl": 11, "enbodi": 6, "encod": [0, 3, 5, 9, 11, 14, 24, 25], "encompass": [0, 21], "encount": [0, 1, 5, 7, 13, 15, 21, 24, 25], "encourag": 15, "end": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 21, 22, 24, 25], "endpoint": [3, 6], "energi": [0, 4, 6], "enforc": 12, "eng": 23, "engin": [0, 1, 3, 4, 18, 24], "enorm": 3, "enough": [0, 6, 13, 24], "ensembl": [1, 9, 24], "ensur": [0, 1, 2, 3, 5, 6, 11, 13, 21, 25], "entail": 24, "enter": [5, 6, 25], "enthought": [0, 18, 24], "entir": [1, 3, 7, 9, 18, 21, 24], "entiti": [9, 12, 19, 24], "entri": [0, 5, 8, 11, 12, 19, 24, 25], "entropi": [1, 3, 7, 10, 13, 24], "enumer": [0, 1, 2, 3, 4, 6, 8, 24], "env": 21, "environ": [2, 18, 24], "environemnt": 15, "eo": [0, 6], "eol": 0, "eosfit": 0, "epoch": [0, 1, 3, 4, 12, 13, 24], "epsilon": [0, 5, 6, 7, 13, 24, 25], "epsilon_": [0, 24], "epsilon_0": [0, 24], "epsilon_1": [0, 24], "epsilon_2": [0, 24], "epsilon_i": [0, 24, 25], "eq": [3, 13, 14, 19, 21], "eqnarrai": [3, 5, 6], "equal": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 13, 14, 16, 19, 21, 24, 25], "equat": [1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 17, 19, 21, 24], "equilibrium": [2, 12], "equiv": [3, 13, 19, 21], "equival": [0, 1, 5, 7, 8, 11, 13, 18, 19, 24, 25], "erf": 21, "eriador": 24, "err": [0, 10], "err_": 6, "err_sqr": 2, "errat": 13, "erron": 2, "error": [1, 2, 4, 5, 6, 7, 9, 11, 12, 13, 15, 16, 17, 18, 19, 21], "error_estimate_corr_tim": 21, "error_hidden": 1, "error_output": 1, "escap": 13, "especi": [1, 3, 9, 12, 13, 15], "essenti": [0, 5, 6, 9, 10, 12, 14, 15, 21, 25], "establish": [0, 6, 10, 11, 16], "estim": [0, 1, 5, 6, 7, 10, 11, 13, 18, 21, 24, 25], "estimated_mse_fold": 6, "estimated_mse_kfold": 6, "estimated_mse_sklearn": 6, "et": [0, 2, 4, 16, 17, 23, 24, 25], "eta": [0, 1, 3, 8, 12, 13, 24], "eta0": [8, 13], "eta_": 13, "eta_t": 13, "eta_v": [0, 1, 3, 24], "etc": [0, 1, 3, 5, 7, 8, 9, 11, 12, 13, 14, 18, 19, 21, 25], "ethic": 18, "euclidean": [0, 14, 25], "evalu": [0, 2, 3, 4, 5, 6, 9, 13, 15, 16, 17, 21, 24, 25], "evalut": 13, "even": [0, 1, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 18, 19, 21, 24, 25], "evenli": 4, "event": [5, 7, 10, 21], "eventu": [0, 5, 6, 11, 12, 13, 22, 25], "everi": [0, 1, 2, 3, 4, 5, 6, 9, 10, 11, 12, 13, 14, 15, 18, 21, 22, 24, 25], "everyth": [4, 12, 16], "everywher": [4, 13], "evolv": 0, "exact": [0, 5, 11, 12, 13, 19, 21, 24, 25], "exactli": [0, 3, 4, 6, 12, 18, 25], "exam": 24, "examin": 6, "exampl": [0, 5, 11, 12, 13, 15, 16, 18, 19, 21, 23], "exce": [1, 12, 13], "excel": [0, 1, 4, 5, 10, 24, 25], "except": [3, 4, 6, 8, 9, 19], "excess": [0, 24], "excit": 0, "exclud": [1, 6, 12], "exclus": [0, 1, 3, 6, 21, 24], "execut": [2, 5, 13, 15, 25], "exemplifi": 13, "exercic": [22, 24], "exercis": [5, 18, 20, 22, 24], "exhaust": 6, "exhibit": [0, 5, 6, 8, 24, 25], "exist": [0, 1, 2, 3, 5, 6, 7, 8, 9, 13, 19, 24], "exit": [5, 19, 25], "exp": [0, 1, 2, 5, 6, 7, 8, 10, 11, 12, 13, 16, 17, 21, 25], "exp_term": 1, "expand": [5, 7, 11, 13], "expans": [0, 3, 5, 8, 10, 12, 13, 24, 25], "expect": [0, 1, 5, 6, 7, 11, 12, 13, 15, 18, 24, 25], "expectation_value_of_h_wrt_p": 21, "expens": [6, 10, 13, 16], "experi": [0, 1, 6, 8, 13, 15, 18, 24, 25], "experiment": [0, 4, 6, 9, 21, 24], "expert": [1, 9], "explain": [0, 6, 9, 10, 11, 13, 16, 24], "explained_variance_ratio_": 11, "explanatori": [0, 24], "explicit": [0, 3, 6, 13, 19, 24, 25], "explicitli": [0, 4], "explod": 1, "exploit": [0, 3, 12, 13, 24], "explor": [1, 4, 6, 8, 13, 18, 24], "expon": 1, "exponenti": [0, 1, 5, 6, 10, 13, 21, 24], "export": [9, 15, 16], "export_graphviz": 9, "export_text": 9, "exporttext": 9, "expos": 18, "express": [0, 2, 3, 5, 6, 7, 10, 12, 13, 19, 21, 24], "exptmean": 21, "exptvari": 21, "extend": [0, 2, 7, 11, 13, 18, 24], "extens": [0, 12, 15, 18, 24], "extent": [0, 1, 6, 23], "extern": [3, 6, 9], "extra": [1, 3, 5, 15, 22, 24, 25], "extract": [0, 3, 5, 6, 7, 8, 11, 13, 16, 17, 19, 24, 25], "extrapol": [0, 24], "extrem": [0, 1, 4, 5, 6, 7, 8, 9, 13, 15, 16, 19, 25], "extremum": 13, "extrins": 11, "ey": [0, 5, 6, 13, 14, 19, 24, 25], "f": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14, 15, 16, 17, 19, 21, 22, 24, 25], "f1": 13, "f11": [0, 24], "f12": [0, 24], "f13": [0, 24], "f1_grad": 13, "f1d": 13, "f2": 13, "f2_grad_x1": 13, "f2_grad_x1_analyt": 13, "f2_grad_x2": 13, "f2_grad_x2_analyt": 13, "f3": 13, "f3_grad": 13, "f3_grad_analyt": 13, "f4": 13, "f4_grad": 13, "f4_grad_analyt": 13, "f5": 13, "f5_grad": 13, "f6": 13, "f6_for": 13, "f6_for_grad": 13, "f6_grad_analyt": 13, "f6_while": 13, "f6_while_grad": 13, "f7": 13, "f7_grad": 13, "f7_grad_analyt": 13, "f8": 13, "f8_grad": 13, "f9": [0, 13, 24], "f9_altern": 13, "f9_alternative_grad": 13, "f9_grad": 13, "f_": 10, "f_0": [3, 10], "f_1": [10, 13], "f_2": [12, 13], "f_3": 12, "f_d": 21, "f_grad": 13, "f_grad_analyt": 13, "f_i": [0, 6, 12, 16], "f_m": [3, 10], "f_n": 3, "f_vec": 2, "face": [13, 24], "facecolor": [6, 8, 21], "facil": [0, 18], "facilit": 12, "fact": [0, 1, 3, 5, 9, 11, 12, 13, 24, 25], "factor": [0, 1, 3, 5, 6, 9, 10, 11, 13, 19, 21, 24, 25], "factori": 13, "fade": 6, "fafab0": [9, 10], "fail": [0, 6, 13, 22, 24], "failur": 7, "fairli": [1, 2, 21], "faisal": [16, 25], "fake": 4, "fake_loss": 4, "fake_output": 4, "fall": [8, 9, 20], "fals": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 14, 16, 17, 19, 24, 25], "famili": [0, 7, 8, 21, 25], "familiar": [0, 3, 5, 6, 8, 15, 18, 19, 21, 24], "famou": [6, 12], "far": [0, 3, 4, 5, 6, 8, 11, 12, 13, 14, 16, 24, 25], "fashion": [0, 9, 10, 24], "fast": [1, 3, 6, 10, 12, 13, 18, 21, 24], "faster": [1, 11, 13], "fastest": [13, 19], "favor": 7, "favorit": 21, "fc": 3, "featur": [0, 1, 3, 5, 6, 7, 8, 10, 11, 12, 13, 15, 17, 18, 21, 24], "feature_nam": [1, 7, 9], "feautur": 9, "fed": 1, "feed": [0, 2, 3, 11, 18, 24], "feed_forward": 1, "feed_forward_out": 1, "feed_forward_train": 1, "feedback": [4, 24], "feeddorward": 4, "feedforward": [1, 4, 12], "feel": [0, 5, 6, 11, 13, 15, 16, 18, 22, 24], "feet": [], "fetch": [6, 15], "few": [1, 3, 4, 5, 9, 17, 21, 24], "fewer": [0, 9, 11, 24], "ffnn": [1, 12], "field": [0, 3, 6, 12, 18], "fifth": [0, 6, 24], "fig": [0, 1, 2, 3, 4, 6, 7, 12, 13, 14, 24], "fig_id": [0, 6, 7, 9, 24], "figaxi": 21, "figsiz": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 24], "figur": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 18, 24, 25], "figure_id": [0, 6, 7, 9, 24], "figurefil": [0, 6, 7, 9, 24], "file": [0, 4, 5, 6, 7, 9, 15, 24], "file_prefix": 4, "filenam": 24, "fill": [5, 9, 25], "filter": [3, 4], "final": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 20, 21, 22, 24], "financ": 0, "find": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 21, 24, 25], "fine": [0, 14], "finish": 2, "finit": [3, 5, 6, 12, 13, 17, 21, 25], "finnicki": 15, "first": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 19, 21, 22, 23, 25], "firsteigvector": 11, "fit": [1, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 17, 21, 25], "fit_intercept": [0, 5, 6, 16, 25], "fit_mod": 9, "fit_theta": 6, "fit_transform": [0, 6, 8, 9, 11, 15], "fiti": [0, 24], "five": [0, 9, 24, 25], "fix": [0, 3, 4, 6, 10, 11, 12, 13, 24], "flag": 4, "flat": [12, 13], "flatten": [1, 3, 4, 5, 19], "flexibl": [1, 6, 8, 10, 12, 24], "flip": [22, 24], "float": [0, 3, 4, 5, 9, 11, 13, 14, 19, 24, 25], "float32": [4, 9], "float64": [4, 19, 24], "flop": [5, 19, 25], "flow": [1, 4, 12], "fluctuat": 5, "fly": 11, "fm": 0, "fmax": 3, "fmesh": 13, "fn": 7, "focu": [0, 3, 4, 5, 6, 15, 18, 23, 24], "focus": [1, 6, 7, 19], "fold": [6, 9], "folder": [0, 4, 6, 15, 24], "follow": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 21, 22, 23, 24, 25], "font": [7, 21, 24], "fontdict": 21, "fontsiz": [1, 6, 8, 9, 10, 21], "fontweight": 1, "footprint": 3, "foral": [8, 25], "forc": [0, 5, 6, 10, 11, 25], "forcast": 4, "forecast": [4, 12], "forest": [0, 1, 9, 18, 24], "forget": 11, "form": [0, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 16, 18, 19, 21, 24, 25], "formal": [3, 4, 14, 21], "format": [0, 1, 3, 4, 6, 7, 8, 9, 10, 11, 18, 21, 23], "format_data": 4, "formatstrformatt": [6, 13], "formul": [4, 6, 11, 14], "formula": [3, 13, 21], "forth": [4, 12], "fortran": [0, 18, 19, 24], "fortran2003": [18, 24], "fortran90": 21, "fortun": [0, 11, 25], "forward": [0, 3, 6, 18, 19, 24], "found": [1, 2, 4, 5, 6, 12, 13, 24, 25], "foundat": [18, 24], "four": [4, 5, 6, 8, 12, 19, 20, 22, 24], "fourier": [0, 24], "fourierdef1": 3, "fourierdef2": 3, "fourierseriessign": 3, "fourth": [12, 24, 25], "fp": 7, "frac": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 19, 21, 24, 25], "fraction": 9, "frame": 7, "framework": [1, 8, 10, 21], "frank": [5, 11], "frankefunct": [5, 6, 11], "fredli": [22, 24], "free": [0, 6, 11, 13, 15, 16, 18, 19, 21, 22, 23, 24], "freecodecamp": 18, "freedom": 5, "freeli": 0, "freez": 15, "frequenc": [3, 6, 7, 21], "frequent": [0, 8, 9, 13], "frequentist": 18, "fresh": 10, "fridai": [15, 22, 24], "friedman": [6, 23, 24], "friendli": 4, "frodo": 24, "frog": 3, "from": [0, 1, 2, 3, 4, 6, 7, 8, 9, 11, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23], "from_cod": 9, "from_logit": [3, 4], "from_tensor_slic": 4, "front": [0, 4, 5, 24, 25], "frustrat": 15, "fulfil": [2, 5, 12, 25], "full": [1, 3, 5, 7, 9, 10, 13, 21, 24, 25], "full_matric": [5, 25], "fulli": [3, 6, 12, 21], "fun": [18, 24], "func": 2, "function": [2, 3, 4, 5, 9, 14, 15, 16, 17, 18, 19], "functionali": 11, "fundament": [0, 6, 18, 24], "funtion": 2, "further": [2, 7, 9, 24], "furthermor": [0, 3, 5, 6, 7, 11, 12, 13, 18, 24, 25], "futur": [0, 4, 8, 9, 24], "fy": [15, 20, 22, 23, 24], "fys5419": [23, 24], "fys5429": [23, 24], "f\u00f8470": [22, 24], "g": [0, 1, 2, 3, 4, 6, 8, 9, 10, 11, 13, 15, 21, 24, 25], "g0": 2, "g_": [2, 9, 10], "g_0": 2, "g_1": [2, 10], "g_2": [2, 10], "g_analyt": 2, "g_dnn_ag": 2, "g_euler": 2, "g_i": 2, "g_m": [3, 10], "g_n": 3, "g_re": 2, "g_t": 2, "g_t_d2t": 2, "g_t_d2x": 2, "g_t_dt": 2, "g_t_hessian": 2, "g_t_hessian_func": 2, "g_t_jacobian": 2, "g_t_jacobian_func": 2, "g_trial": 2, "g_trial_deep": 2, "g_vec": 2, "gain": [1, 5, 7, 9, 10, 13, 25], "galleri": [0, 24], "game": 4, "gamge": 24, "gamma": [0, 2, 8, 9, 10, 11, 13, 24], "gamma1": 8, "gamma2": 8, "gamma_": [0, 24], "gamma_0": 10, "gamma_1": 10, "gamma_1x": 10, "gamma_i": [0, 8, 21, 24], "gamma_j": 13, "gamma_k": 13, "gamma_m": 10, "gamma_x": [0, 24], "gap": 8, "gate": [4, 12], "gather": [0, 1, 12, 25], "gaug": 12, "gaussbacksub": 19, "gaussian": [4, 5, 6, 8, 14, 21, 24], "gaussian_point": 14, "gaussian_rbf": 8, "gave": 13, "gavra": 24, "gbc": 24, "gca": [2, 6, 8, 13], "gd": 1, "gd_clf": 10, "gdclassiffiercgain": 10, "gdclassiffierconfus": 10, "gdclassiffierroc": 10, "gdm": 13, "gdregress": 10, "ge": [1, 5, 7, 21, 25], "gen_loss": 4, "gen_tap": 4, "gender": [0, 24], "genener": 4, "gener": [0, 1, 2, 3, 5, 6, 8, 10, 11, 12, 13, 14, 15, 16, 19, 21, 23, 25], "generaliz": 16, "generallay": 12, "generate_and_save_imag": 4, "generate_imag": 4, "generate_latent_point": 4, "generate_simple_clustering_dataset": 14, "generated_imag": 4, "generator_loss": 4, "generator_loss_list": 4, "generator_model": 4, "generator_optim": 4, "genom": 18, "geodes": 11, "geometr": [0, 13, 24], "geometri": 5, "georg": 23, "geotif": 6, "geq": [2, 5, 8, 9, 13, 25], "geron": [0, 23, 24], "get": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 13, 15, 18, 19, 21, 22, 24, 25], "get_dummi": 9, "get_paramet": 2, "get_split": 9, "get_yaxi": 8, "get_yticklabel": 6, "gh": 15, "gibb": [18, 24], "gif": 4, "gini": 10, "gini_index": 9, "ginvers": 13, "git": [0, 15, 18, 24], "giter": 13, "github": [0, 18, 20, 22, 23, 24, 25], "gitignor": 15, "gitlab": [0, 15, 18, 24], "give": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 14, 18, 21, 24, 25], "given": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 19, 21, 24, 25], "global": [6, 7, 13], "glorot": 1, "gnew": 13, "go": [0, 1, 3, 5, 6, 8, 9, 11, 12, 13, 15, 16, 24, 25], "goal": [0, 7, 9, 24], "goe": [0, 1, 2, 5, 6, 13, 14, 15, 19, 24, 25], "golden": 13, "gone": [5, 25], "gong": 1, "good": [1, 3, 4, 5, 6, 9, 10, 11, 13, 15, 18, 21, 23, 25], "goodfellow": [4, 23, 24, 25], "googl": [1, 4, 18, 24], "got": [1, 6], "gotten": 24, "gov": 6, "govern": 24, "gp": 23, "gpu": [1, 13, 18, 24], "grad": [2, 13], "grad_analyt": 13, "grade": 20, "gradient": [0, 3, 4, 7, 8, 9, 12, 18, 24, 25], "gradientboostingclassifi": 10, "gradientboostingregressor": 10, "gradients_of_discrimin": 4, "gradients_of_gener": 4, "gradienttap": 4, "gradual": [1, 14], "grai": [4, 6], "graph": [1, 9, 11, 12, 13, 16], "graph_from_dot_data": 9, "graphic": [0, 1, 9, 15, 24], "grasp": 0, "gray_r": [1, 3], "grayscal": 3, "great": [5, 13, 15], "greater": [1, 7, 21, 25], "greatli": 13, "greedi": 9, "green": [0, 3, 9, 21], "grei": 4, "grid": [1, 3, 6, 7, 8, 12, 21], "grossli": 13, "ground": [0, 24], "group": [0, 6, 7, 9, 14, 15, 18, 20, 22, 24], "groupbi": [0, 24], "grow": [1, 3, 9, 10], "growth": [0, 24], "gru": 4, "guarante": [0, 4, 13, 21, 24, 25], "guess": [1, 4, 10, 13, 14], "guestrin": 10, "gui": 15, "guid": 1, "h": [0, 1, 5, 6, 8, 13, 15, 21, 22, 23, 24, 25], "h1": 2, "h_": [0, 13, 24], "h_1": [2, 13], "h_2": [2, 13], "h_m": 10, "ha": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 21, 24, 25], "haanen": [22, 24], "habit": [0, 25], "had": [0, 1, 6, 7, 13, 24], "hadamard": [1, 12, 13], "half": [1, 8, 9], "halv": 10, "hand": [0, 1, 2, 3, 5, 11, 12, 13, 18, 19, 21, 22, 23, 24, 25], "handi": 3, "handl": [0, 1, 2, 5, 9, 11, 15, 18, 25], "handle_unknown": 9, "handsid": 12, "handwrit": 12, "handwritten": [1, 5], "happen": [1, 2, 3, 4, 5, 6, 10, 13, 21, 25], "hard": [1, 7, 8, 10, 13], "hardcopi": [18, 24], "harder": [0, 1, 25], "harmon": 3, "hasn": [], "hassl": [0, 18, 24], "hast": [18, 24], "hasti": [0, 6, 16, 17, 23, 24, 25], "hat": [0, 1, 5, 6, 7, 9, 10, 11, 12, 13, 16, 17, 19, 25], "have": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 24, 25], "haven": 1, "he": 7, "head": [4, 10, 21], "header": [0, 24], "heads_proba": 10, "health": [0, 25], "hear": [0, 13, 24], "heart": [0, 7, 24], "heatmap": [0, 1, 3, 7, 17, 24], "heavili": 0, "heavisid": 1, "height": [1, 3, 6], "held": 13, "help": [0, 1, 4, 12, 13, 15, 16, 24], "helper": [4, 14], "henc": [0, 5, 6, 8, 9, 10, 12, 13, 24, 25], "henrik": [22, 24], "her": 7, "here": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 18, 19, 21, 24, 25], "hereaft": [0, 8, 12, 24], "hermitian": 19, "hessenberg": 19, "hessian": [0, 2, 5, 13], "heterogen": [9, 10], "hi": 7, "hidden": [1, 3, 4, 12], "hidden_bia": 1, "hidden_bias_gradi": 1, "hidden_layer_s": [0, 1, 24], "hidden_neuron": 4, "hidden_weight": 1, "hidden_weights_gradi": 1, "hierarch": [5, 25], "high": [0, 1, 2, 3, 4, 5, 6, 9, 10, 11, 13, 14, 18, 19, 24, 25], "higher": [0, 1, 3, 5, 6, 8, 13, 24, 25], "highest": [1, 2], "highli": [0, 3, 4, 10, 18, 19, 23, 24, 25], "highwai": [], "hing": 8, "hint": [13, 15, 16, 25], "hip": 18, "hire": 0, "hist": [4, 6, 7, 21], "histogram": [6, 7, 21], "histor": [7, 11], "histori": [3, 4, 12, 15], "hitherto": 5, "hjorth": [22, 24, 25], "hobbi": 21, "hoc": [5, 25], "hoff": 23, "hold": [1, 3, 6, 13, 14], "holder": [0, 24], "home": [], "homepag": 24, "homework": [6, 13], "homogen": [1, 3, 9, 10, 13], "honchar": 2, "hopefulli": [0, 11, 15, 21, 24], "horizont": 11, "horlyk": [22, 24], "hors": [3, 7, 24], "hot": [1, 9], "hour": [1, 18, 20, 21, 22, 24], "how": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 21, 24, 25], "howev": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 19, 21, 24, 25], "hspace": [0, 4, 8, 10, 21, 24], "hstack": 1, "htf": 24, "html": [0, 16, 18, 20, 22, 23, 24], "http": [0, 3, 4, 6, 13, 15, 16, 18, 19, 20, 22, 23, 24, 25], "huang": [0, 24], "huber": [0, 24], "huge": [1, 3, 4, 18], "human": [0, 1, 3, 6, 9, 12], "humid": 9, "hundr": 1, "hungri": 1, "hybrid": 20, "hydrogen": [0, 24], "hyperbol": [1, 4, 12], "hyperparam": 8, "hyperparamet": [3, 4, 5, 6, 9, 13, 25], "hyperplan": 11, "h\u00f8rlyk": [22, 24], "i": [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25], "i0": [0, 24], "i1": [0, 6, 8, 12, 24], "i2": [0, 8, 12, 24], "i3": [0, 12, 24], "i5": [0, 24], "i_": 13, "i_1": [5, 6], "i_2": [5, 6], "ian": 23, "ic": 1, "id": [7, 13], "ida": [22, 24], "idea": [0, 1, 2, 3, 4, 6, 9, 10, 12, 13, 19], "ideal": [0, 2, 6, 8, 13, 21, 24], "idem": 6, "ident": [5, 6, 12, 13, 17, 19, 25], "identifi": [0, 1, 7, 9, 11, 12, 13, 14, 24, 25], "ieor": 21, "ifi": 23, "ifs": [18, 24], "ignor": [0, 1, 3, 9, 15, 25], "ii": [19, 21], "iii": [19, 24], "ij": [0, 1, 3, 6, 8, 12, 14, 16, 19, 21, 24, 25], "ik": [0, 19, 24, 25], "illustr": [5, 7, 10, 12, 13, 14, 18, 24], "im": 6, "imag": [1, 3, 4, 6, 9, 11, 12, 14, 23, 24], "image_at_epoch_": 4, "image_batch": 4, "image_height": 3, "image_path": [0, 6, 7, 9, 24], "image_width": 3, "imageio": 6, "images_from_seed_imag": 4, "imagin": 1, "immedi": [0, 3, 4, 6, 18, 24], "implement": [0, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 21, 24, 25], "impli": [3, 5, 6, 7, 13, 19, 25], "implicit": 3, "implicitli": [11, 21], "import": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 21], "importantli": 3, "impos": [0, 6, 11, 12, 24], "imposs": [0, 5, 24, 25], "impress": [0, 12, 24], "improv": [0, 4, 5, 9, 10, 11, 13, 15, 25], "impur": 9, "imread": 6, "imshow": [1, 3, 4, 6], "in3050": [23, 24], "in3310": 24, "in4080": [23, 24], "in4300": [23, 24], "in4310": 23, "in5400": 3, "in5550": 23, "in_out_neuron": 4, "inaccur": 13, "inact": 12, "inadequ": [0, 24], "inch": 6, "includ": [0, 1, 2, 3, 4, 5, 6, 7, 11, 12, 15, 16, 17, 18, 21, 22, 23, 24, 25], "include_bia": [6, 9], "incom": [12, 16], "incorrect": 1, "incoveni": 8, "increas": [0, 1, 3, 4, 5, 6, 9, 12, 13, 21, 24], "increasingli": 21, "ind": 6, "inde": [0, 2, 4, 5, 6, 13, 24, 25], "indefinit": 4, "independ": [0, 5, 6, 7, 8, 12, 13, 21, 24, 25], "index": [0, 1, 3, 4, 10, 14, 18, 19, 21, 23, 24], "index_col": [0, 24], "indic": [0, 1, 3, 4, 5, 6, 9, 10, 11, 13, 16, 24, 25], "indispens": 6, "individu": [1, 6, 7, 10, 12, 21, 24, 25], "indu": [], "indx": 19, "indx1": 2, "indx2": 2, "indx3": 2, "ineffici": [3, 13], "inequ": [8, 13], "inertia": 13, "inf1000": [18, 24], "inf1100": [18, 24], "inf1100l": [18, 24], "inf1110": [18, 24], "inf3000": 24, "infeas": 9, "infer": [0, 1, 4, 6, 23, 24], "inferenc": 1, "infil": [0, 6, 7, 9, 24], "infin": [5, 6, 7, 11, 25], "infinit": 3, "infinitesim": 21, "influenc": [6, 10], "influenti": 1, "info": 24, "inform": [0, 1, 3, 4, 6, 9, 11, 12, 13, 14, 19, 23, 24], "inforom": 15, "infti": [3, 6, 13, 21], "ingeni": 13, "ingredi": [0, 9, 24], "inher": 6, "inherit": [19, 24], "initi": [0, 1, 2, 6, 10, 13, 14, 19, 21, 24], "inject": 14, "inlin": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 21, 24, 25], "inner": [0, 13, 25], "inp": 4, "inplac": 13, "input": [0, 1, 3, 4, 5, 6, 7, 8, 12, 13, 14, 16, 21, 24, 25], "input_dim": 1, "input_shap": [3, 4], "inputs": 1, "inputs_shuffl": [0, 1, 25], "insert": [3, 5, 6, 8, 10, 21, 25], "insid": [4, 7], "insight": [0, 1, 5, 18, 24, 25], "insist": [6, 13], "inspir": [0, 1, 12, 24], "instabl": 2, "instal": [0, 1, 5, 6, 9, 15], "instanc": [0, 1, 2, 4, 6, 9, 11, 13, 16, 24, 25], "instanti": 10, "instead": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 13, 14, 17, 19, 21, 24, 25], "institut": 1, "instruct": [0, 1, 15], "int": [0, 1, 2, 3, 4, 5, 6, 11, 13, 14, 19, 21, 25], "int32": 10, "int_": [3, 6, 21], "int_0": 21, "int_a": 21, "intak": [0, 25], "integ": [1, 2, 13, 14, 19, 21, 24], "integer_vector": 1, "integr": [3, 6, 21, 24], "intellig": [0, 14, 23, 24], "intend": 10, "intens": 1, "intention": 14, "interact": [0, 6, 9, 12, 18, 24], "intercept": [0, 6, 8, 11, 13, 16, 17, 24, 25], "intercept_": [0, 6, 8, 9, 13, 24], "interchang": [5, 12, 19], "interconnect": 1, "interest": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 18, 21, 24, 25], "interfac": [0, 1, 15, 19, 25], "interior": [0, 9, 24], "intermedi": [19, 25], "intern": [1, 10, 12], "interpol": [1, 3, 4, 6, 12], "interpr": [5, 25], "interpret": [0, 1, 6, 9, 10, 12, 13, 15, 16, 19, 21], "interv": [0, 3, 5, 6, 7, 13, 21, 24, 25], "intial": 13, "intract": [0, 4, 25], "intrins": [3, 11, 19, 21, 24], "intro": [18, 23, 24], "introduc": [0, 1, 5, 6, 8, 10, 12, 19, 21, 24], "introduct": [1, 2, 4, 13, 23, 25], "introductori": [0, 4, 19, 23, 24, 25], "intuit": [0, 5, 6, 8, 12, 13, 24], "inv": [0, 5, 13, 17, 24, 25], "invalu": [0, 13, 18, 24], "invari": 1, "invd": 5, "inver": 8, "invers": [0, 3, 6, 13, 24, 25], "inverse_transform": 8, "invert": [0, 5, 7, 10, 13, 16, 24], "invh": 13, "invok": 8, "involv": [0, 2, 6, 7, 11, 12, 24, 25], "io": [0, 18, 20, 22, 23, 24, 25], "ip": [0, 8, 21, 24], "ipca": 11, "ipynb": [18, 24], "ipython": [0, 5, 7, 9, 11, 14, 18, 24, 25], "iq": 6, "iri": [8, 9], "irreduc": 6, "irrelev": [5, 25], "irrespect": [0, 24], "isn": 5, "isnul": [], "isomap": 11, "issu": [1, 9, 15, 19], "it_arrai": 13, "item": [0, 13, 24], "items": [19, 24], "iter": [1, 2, 4, 6, 8, 13, 14, 21], "its": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 21, 24], "itself": [5, 6, 12, 21, 24, 25], "j": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 13, 14, 15, 16, 19, 21, 23, 24, 25], "j1": 19, "j_": 6, "j_lasso_sk": 6, "j_ridge_sk": 6, "j_sk": 6, "jackknif": [6, 18, 24], "jacobian": [2, 13], "jason": 4, "jax": [18, 24], "jensen": [22, 24, 25], "jerom": 23, "ji": [12, 19], "jit": 13, "jj": [0, 5, 6, 24], "jk": [0, 1, 6, 12, 19, 24], "jl": [0, 24], "jm": 19, "jnp": 13, "job": [2, 8, 10, 15], "join": [0, 4, 6, 7, 9, 24], "joint": [4, 5], "judg": 13, "judgement": 6, "julia": [18, 19], "jump": 21, "junk": 4, "jupit": 24, "jupyt": [0, 15, 16, 18, 23, 24], "just": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 21, 24, 25], "justif": 0, "justifi": [3, 10], "k": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 18, 19, 21, 22, 24, 25], "k0": 7, "k1": 7, "kaggl": 6, "kappa_d": 21, "karl": [22, 24], "karush": 8, "katrin": [22, 24], "keep": [0, 1, 4, 5, 6, 11, 13, 14, 15, 19, 24, 25], "keepdim": [1, 6, 10, 19], "kei": [1, 3, 6, 12], "kept": [4, 6, 14], "kera": [0, 4, 18, 24], "kernel": [0, 1, 3, 18, 24, 25], "kernel_regular": [1, 3], "kernel_s": 4, "kernelpca": 11, "kev": [0, 24], "kevin": [23, 24], "keyword": [19, 24], "kfold": 6, "kg": 1, "ki": 19, "kick": [1, 13], "kiener": 2, "kilomet": 6, "kind": [0, 2, 3, 4, 8, 12, 13, 14, 24, 25], "kj": [6, 12, 19, 25], "kjm": [18, 24], "kkt": 8, "kl": 21, "km": [12, 24], "kmean": 14, "kmeanspoint": 14, "kn_k": 14, "know": [0, 1, 2, 5, 6, 8, 13, 15, 16, 17, 18, 24, 25], "knowledg": [0, 18, 24], "known": [1, 3, 4, 5, 6, 7, 8, 9, 12, 19, 21, 23], "kondev": [0, 24], "kp": 21, "kpca": 11, "kroneck": 14, "kuhn": 8, "kvalsund": [22, 24], "kwown": [0, 24], "l": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 13, 19, 21, 24], "l0": 7, "l1": [0, 1, 3, 7, 24], "l1_l2": [1, 3], "l1regl": 5, "l2": [1, 3], "l_": 19, "l_1": 7, "l_2": [7, 13], "l_j": 12, "la": 13, "la_i": 12, "la_k": 12, "lab": [18, 24], "label": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 18, 19, 21, 24, 25], "labelencod": [7, 10], "labels": [6, 8, 9], "labels_shuffl": [0, 1, 25], "laboratori": 20, "lack": [0, 24], "lagari": 2, "lagrang": [8, 11], "lambda": [0, 1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 17, 21, 24, 25], "lambda_": 11, "lambda_0": 11, "lambda_1": [5, 8, 11, 25], "lambda_2": [8, 11], "lambda_i": [8, 11], "lambda_iy_i": 8, "lambda_jy_iy_j": 8, "lambda_k": 8, "lambda_n": [5, 8, 25], "lamda": 1, "land": 8, "landmark": 8, "landscap": 13, "langl": [0, 6, 11, 21, 24, 25], "languag": [0, 1, 4, 8, 18, 19, 23, 24], "lapack": [19, 24], "laplac": 5, "laptop": [15, 18], "larg": [0, 1, 2, 4, 5, 6, 8, 9, 10, 11, 13, 18, 19, 21, 23, 24, 25], "larger": [0, 3, 5, 6, 8, 10, 11, 13, 17, 21, 24, 25], "largest": [4, 8, 11], "lasso": [0, 7, 18, 24], "lasso_sk": 6, "last": [0, 1, 3, 4, 5, 6, 7, 8, 12, 16, 17, 19, 21, 22, 24], "latent": 4, "latent_dim": 4, "latent_point": 4, "latent_space_value_rang": 4, "later": [0, 1, 4, 7, 8, 12, 13, 14, 15, 18, 24], "latest": [4, 15, 18], "latest_checkpoint": 4, "latex": 24, "latter": [0, 3, 6, 7, 8, 11, 13, 19, 21, 24, 25], "lattic": 12, "law": 0, "layer": [0, 4, 13, 24], "lbfg": [7, 9, 10], "lcc": [5, 6], "lda": 11, "ldot": [0, 6, 11, 24], "le": [5, 7, 10, 13, 17, 21, 25], "lead": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 19, 21, 24, 25], "leaf": 9, "leaki": 1, "leakyrelu": 4, "lear": 13, "learn": [3, 4, 5, 6, 7, 8, 9, 10, 12, 19, 22, 23], "learnabl": 3, "learner": 10, "learnig": 24, "learning_r": [8, 10], "learning_rate_init": [0, 1, 24], "learning_schedul": 13, "least": [0, 7, 8, 10, 11, 17, 18, 19, 21], "leat": 13, "leav": [0, 1, 3, 5, 6, 9, 11, 24], "lectur": [0, 1, 5, 10, 11, 12, 13, 18, 19, 20, 22, 23, 25], "lecturenot": [0, 18, 23, 24], "left": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 19, 21, 24, 25], "leftarrow": [8, 12], "legend": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 15, 24, 25], "leinonen": 24, "len": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 16, 17, 19, 24, 25], "length": [0, 1, 3, 4, 8, 9, 13, 16, 18, 24, 25], "length_of_sequ": 4, "leq": [0, 5, 7, 8, 13, 14, 21, 24, 25], "less": [0, 1, 3, 4, 5, 6, 8, 9, 13, 18, 21, 24, 25], "lessen": 1, "let": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 19, 21, 24, 25], "letter": [0, 16, 19, 21, 24, 25], "level": [0, 1, 5, 6, 9, 18, 19, 20, 22, 24], "li": [8, 11], "lib": [], "liblinear": 10, "librari": [0, 1, 2, 3, 4, 5, 6, 9, 10, 11, 19, 21, 23, 25], "licens": [0, 1, 18, 24], "lie": [0, 6, 11, 21, 24, 25], "life": [0, 1, 8, 12, 24], "lifetim": 13, "like": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 15, 16, 18, 19, 21, 24, 25], "likelihood": [0, 1, 5, 9, 24, 25], "lim_": 21, "limit": [0, 5, 6, 8, 12, 19, 24, 25], "lin_clf": 8, "lin_model": [], "lin_reg": 9, "linalg": [0, 2, 5, 6, 8, 11, 13, 17, 19, 21, 24, 25], "line": [0, 3, 6, 8, 11, 13, 15, 16, 24], "line1": 8, "line2": 8, "line3": 8, "line_model": 15, "line_ms": 15, "line_predict": 15, "linear": [1, 3, 5, 6, 7, 9, 10, 11, 12, 16, 17, 18, 21], "linear_model": [0, 5, 6, 7, 8, 9, 10, 11, 13, 15, 16, 24, 25], "linear_regress": 6, "linearli": [5, 25], "linearloc": [6, 13], "linearregress": [0, 6, 7, 9, 15, 16, 24, 25], "linearsvc": 8, "liner": [1, 3], "linerar": 10, "linewidth": [0, 2, 4, 6, 8, 9, 10], "link": [0, 4, 9, 12, 15, 18, 20, 22, 24], "linlag": 5, "linpack": [19, 24], "linreg": [0, 24], "linspac": [0, 2, 3, 4, 6, 8, 9, 10, 13, 16, 17, 19, 21, 24, 25], "linu": 4, "linux": [0, 1, 18, 24], "liquid": [0, 24], "list": [1, 2, 3, 4, 9, 15, 18, 24], "listedcolormap": [9, 10], "literatur": [1, 7, 14, 23], "littl": [1, 3, 9, 12], "live": [8, 16], "ll": [0, 21, 24, 25], "lle": [0, 25], "lloyd": [4, 14], "lmb": [0, 2, 5, 6, 25], "lmbd": [0, 1, 3, 24], "lmbd_val": [0, 1, 3, 24], "lmbda": 13, "ln": [1, 13], "load": [1, 4, 6, 7, 9, 10], "load_boston": [], "load_breast_canc": [1, 7, 9, 10, 11], "load_data": [3, 4], "load_digit": [1, 3], "load_iri": [8, 9], "loc": [3, 6, 7, 8, 9, 10, 24], "local": [0, 1, 3, 7, 12, 13, 15, 25], "locat": [2, 3, 8, 15], "log": [0, 1, 2, 4, 5, 6, 7, 9, 10, 11, 13, 15, 19, 24], "log10": [0, 5, 6, 25], "log_": [0, 24], "log_clf": 10, "logarithm": [0, 5, 7, 17, 19, 24], "logic": [0, 1, 9, 24], "login": 15, "logist": [0, 1, 2, 8, 9, 10, 11, 12, 13, 18, 25], "logisticregress": [7, 9, 10, 11], "logit": 7, "logreg": [7, 9, 10, 11], "logspac": [0, 1, 3, 5, 6, 24, 25], "long": [0, 1, 3, 4, 12, 13, 24], "longer": [2, 3, 8, 10, 14, 19, 21, 24], "loocv": 6, "look": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 16, 19, 21, 24, 25], "loop": [1, 4, 6, 10, 12, 14, 16, 17, 18, 19, 24], "lose": 1, "loss": [0, 1, 3, 4, 5, 6, 7, 8, 10, 11, 13, 19, 24], "loss_fil": 4, "lossfil": 4, "lost": 4, "lot": [1, 4, 6, 16], "low": [0, 6, 9, 10, 11, 24, 25], "lower": [0, 1, 3, 6, 9, 10, 16, 19, 25], "lowercas": [19, 24], "lowest": [9, 13, 21], "lr": [1, 3, 4, 10], "lstat": [], "lstm": 4, "lstm_2layer": 4, "lstsq": [0, 24, 25], "lt": 6, "lu": [0, 5, 24, 25], "lubksb": 19, "luckili": 2, "ludcmp": 19, "lux": 19, "lvert": 1, "lw": [0, 24], "m": [0, 1, 2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 15, 19, 21, 22, 23, 24, 25], "m_": [9, 12], "m_1": 14, "m_h": [0, 24], "m_k": 14, "m_l": 12, "m_n": [0, 24], "m_p": [0, 24], "m_t": 13, "ma": 11, "machin": [1, 3, 4, 5, 6, 7, 9, 10, 11, 12, 15, 16, 19, 23, 25], "machinelearn": [0, 6, 16, 18, 20, 22, 23, 24], "mackai": 23, "made": [0, 1, 3, 4, 5, 6, 7, 9, 11, 12, 24, 25], "mae": [0, 24], "magic": 4, "magnitud": [1, 6, 7, 13], "mai": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 13, 18, 19, 21, 24, 25], "mail": [20, 22], "main": [0, 1, 3, 4, 5, 6, 7, 9, 19, 23, 25], "mainli": [0, 5, 6, 7, 9, 24, 25], "maintain": 6, "major": [1, 6, 9, 10, 13, 19, 24], "make": [1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 15, 16, 18, 19, 21, 23, 24], "make_axes_locat": 6, "make_moon": [8, 9, 10], "make_pipelin": [0, 6, 10, 25], "makedir": [0, 6, 7, 9, 24], "malcondit": 19, "malign": [1, 7, 9], "mammographi": 5, "manag": [0, 2, 3, 15, 18, 24], "mandatori": [22, 24], "mani": [0, 1, 3, 4, 5, 6, 7, 8, 9, 11, 13, 14, 15, 16, 17, 18, 19, 21, 23, 24, 25], "manifold": 11, "manner": 3, "manual": 6, "map": [0, 1, 2, 6, 7, 8, 11, 12, 14, 21, 24], "marc": 25, "margin": [0, 5, 8], "marit": [0, 24], "mark": 24, "marker": [7, 19, 24], "markov": [18, 24], "marsaglia": 21, "mass": [0, 1, 5, 13, 25], "massag": [0, 24], "masses2016": [0, 24], "masses2016ol": [0, 24], "masses2016tre": 0, "masseval2016": [0, 24], "master": [20, 22], "mat": [18, 24], "mat1100": [18, 24], "mat1110": [18, 24], "mat1120": [18, 24], "match": [1, 4, 5, 13, 14, 15, 25], "materi": [4, 5, 7, 13, 15, 19, 20, 22, 25], "math": [3, 7, 12, 13, 19, 21, 23, 24], "mathbb": [0, 4, 5, 6, 7, 8, 11, 12, 13, 14, 17, 19, 21, 24, 25], "mathbf": [0, 5, 6, 7, 8, 13, 19, 24, 25], "mathcal": [1, 5, 6, 7, 13], "matheemat": 3, "mathemat": [0, 6, 11, 12, 13, 18, 19, 21, 23, 24], "mathemati": 24, "mathrm": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 21, 24, 25], "matmul": [1, 2, 5], "matnat": 23, "matplotlib": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 21, 24, 25], "matric": [0, 1, 3, 4, 6, 7, 8, 11, 13, 16, 17, 18, 25], "matrix": [0, 2, 3, 4, 6, 7, 8, 10, 13, 17, 21], "matshow": 1, "matter": [2, 3, 13, 25], "max": [0, 1, 2, 3, 4, 9, 10, 12, 13, 22, 24], "max_depth": [0, 9, 10], "max_diff": 2, "max_diff1": 2, "max_diff2": 2, "max_it": [0, 1, 8, 13, 24], "max_iter": 14, "max_leaf_nod": 10, "max_sampl": 10, "maxdegre": [0, 6, 10, 25], "maxdepth": 10, "maxim": [1, 4, 5, 7, 8, 11], "maximum": [0, 2, 3, 5, 7, 8, 9, 10, 13, 14, 24, 25], "maxpolydegre": [5, 6, 25], "maxpooling2d": 3, "mbox": [5, 6, 25], "mcculloch": 12, "md": 11, "mdoel": 4, "mean": [1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 21, 24], "mean_absolute_error": [0, 24], "mean_divisor": 14, "mean_i": 21, "mean_matrix": 14, "mean_squared_error": [0, 4, 6, 7, 10, 15, 24, 25], "mean_squared_log_error": [0, 24], "mean_vector": 14, "mean_x": 21, "meaning": [0, 4, 7, 24], "meansquarederror": [0, 24], "meant": [3, 7, 10, 13], "measur": [0, 1, 2, 5, 6, 9, 11, 12, 14, 16, 21, 24, 25], "mechan": [0, 4, 21, 24], "median": [0, 24, 25], "medicin": 12, "medium": [4, 8, 13], "medv": [], "meet": [0, 22], "mehta": [0, 24, 25], "memori": [3, 4, 11, 12, 13, 19], "mention": [0, 12, 13, 21, 24], "mere": 0, "meshgrid": [2, 5, 6, 8, 9, 10, 11], "mess": 15, "messag": [5, 13], "messi": 2, "met": [0, 3, 8, 25], "meteorolog": 9, "meter": 6, "method": [0, 1, 2, 3, 4, 5, 7, 8, 11, 12, 14, 15, 16, 17, 18, 19, 21, 23, 25], "metion": 6, "metric": [0, 1, 3, 6, 7, 9, 10, 14, 15, 24, 25], "metropoli": [18, 24], "mev": [0, 21, 24], "mgd": 13, "mglearn": [18, 24], "mgrid": 13, "mhjensen": [], "mi": 10, "mia": [22, 24], "microsoft": 23, "mid": 1, "midel": 4, "midnight": 15, "midpoint": 9, "might": [0, 1, 2, 4, 6, 9, 13, 15, 17, 25], "migth": 17, "mild": 9, "millimet": 6, "million": [0, 24, 25], "mimic": 12, "min": [0, 2, 5, 8, 9], "min_": [0, 2, 5, 14, 17, 24, 25], "min_samples_leaf": 9, "mind": [0, 6, 13, 15, 24, 25], "mindboard": 4, "mine": [18, 24], "mini": [1, 11, 12, 13], "minibatch": [1, 11, 13], "minibathc": 13, "miniforge3": [], "minim": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 25], "minima": [0, 1, 7, 13, 24], "minimum": [0, 1, 2, 6, 8, 9, 11, 13, 25], "minmaxscal": [0, 25], "minor": 21, "minst": 1, "minu": 7, "mirjalili": 24, "mirror": 9, "misc": 6, "misclassif": [8, 9, 10], "misclassifi": [8, 10], "miser": 0, "mismatch": 1, "miss": [7, 10], "mistak": 4, "mit": 23, "mix": [1, 2, 24], "mixtur": 13, "mk": [9, 19], "mkdir": [0, 6, 7, 9, 24], "ml": [0, 1, 10, 13, 19, 25], "mlab": 21, "mle": [5, 7], "mlp": 1, "mlpclassifi": 1, "mlpregressor": [0, 24], "mm": 19, "mml": 25, "mn": [12, 21], "mnist": [1, 11], "mod": 21, "mode": [20, 22, 24], "model": [2, 3, 5, 7, 8, 9, 10, 11, 13, 14, 16, 18, 21, 23, 25], "model_select": [0, 1, 3, 5, 6, 7, 9, 10, 11, 15, 16, 17, 24, 25], "moder": 10, "modern": [0, 6, 7, 18, 24], "modif": [2, 12, 13], "modifi": [0, 1, 3, 5, 7, 8, 10, 12, 13, 24, 25], "modul": [0, 16, 19, 24], "modular": 21, "modulo": 21, "moe": [11, 25], "moment": [5, 6, 13, 21], "mondai": [22, 24], "monitor": 13, "monoton": [5, 12, 21], "mont": [0, 6, 18, 21, 23, 24], "montli": 16, "moor": [5, 6], "more": [0, 1, 2, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 21], "moreov": [0, 3], "morten": [22, 24, 25], "mortenhj": 24, "most": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 21, 24, 25], "mostli": [1, 11], "motion": [0, 13], "motiv": [1, 4], "move": [0, 4, 5, 6, 7, 9, 12, 13, 14, 15, 16, 21, 25], "mpl": [7, 24], "mpl_toolkit": [2, 6, 13], "mplot3d": [2, 6, 13], "mplregressor": 1, "mse": [0, 4, 5, 6, 9, 10, 15, 16, 17, 24, 25], "mse_simpletre": 10, "mselassopredict": 5, "mselassotrain": 5, "mseownridgepredict": [6, 25], "msepredict": 5, "mseridgepredict": [0, 5, 6, 25], "msetrain": 5, "msle": [0, 24], "mt": [7, 12], "mu": [0, 6, 11, 13, 21, 24], "mu0": 21, "mu1": 21, "mu2": 21, "mu_": [6, 21], "mu_i": 6, "mu_n": 11, "mu_x": 21, "much": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 15, 19, 21, 24, 25], "multi": [0, 1, 3, 7, 18, 24], "multiclass": [1, 7], "multidimension": [11, 12, 24], "multilay": 1, "multinomi": 7, "multipl": [2, 4, 5, 6, 7, 12, 13, 15, 21, 25], "multipli": [3, 5, 6, 11, 13, 19, 21, 25], "multiplum": 8, "multivari": [0, 2, 10, 11, 18, 21, 24], "multivariate_norm": [11, 14], "multpli": 16, "murphi": [11, 23, 24], "must": [1, 2, 5, 6, 8, 10, 12, 13, 14, 15, 21, 25], "mutat": 7, "mutual": [1, 3, 6, 13], "mx_": 21, "my": 24, "myenv": [], "myriad": [0, 18, 24], "mz1": 21, "mz2": 21, "m\u00f8svatn": 6, "n": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 19, 21, 24, 25], "n1": 19, "n2": 19, "n_": [1, 2, 3, 8, 12, 21], "n_0": [12, 21], "n_boostrap": [6, 10], "n_bootstrap": 6, "n_categori": [1, 3], "n_cluster": 14, "n_compon": 11, "n_epoch": 13, "n_estim": 10, "n_examples_to_gener": 4, "n_featur": 1, "n_filter": 3, "n_hidden": 2, "n_hidden_neuron": [0, 1, 24], "n_i": 21, "n_input": [0, 1, 3, 25], "n_instanc": 9, "n_job": 10, "n_k": 14, "n_l": [12, 21], "n_layer": 1, "n_m": 9, "n_neuron": 1, "n_neurons_connect": 3, "n_neurons_layer1": 1, "n_neurons_layer2": 1, "n_point": 14, "n_sampl": [6, 8, 9, 10, 14], "n_split": 6, "n_step": 4, "n_t": 2, "n_x": 2, "nabla": [1, 13], "nabla_": [2, 13], "nabla_w": 13, "nag": 13, "naimi": [0, 24], "naiv": 7, "naive_kmean": 14, "name": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 18, 19, 21, 22, 24, 25], "narrow": 13, "nation": [1, 5], "nativ": [18, 24], "natur": [0, 1, 4, 8, 9, 12, 13, 21, 23, 24], "navier": 12, "navig": 15, "nb": 21, "nb_": 19, "nbconvert": 24, "nd": 14, "ndarrai": 6, "ne": [9, 10, 19, 21, 25], "nearest": [1, 3, 6, 11], "nearli": 13, "neat": 24, "neccesari": 6, "necess": 2, "necessari": [0, 1, 3, 4, 8, 14, 24], "necessarili": [0, 4, 11, 21, 24], "necesserali": 5, "neck": 7, "need": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 21, 25], "neg": [0, 1, 3, 5, 6, 7, 10, 13, 19, 21, 24], "neg_mean_squared_error": 6, "neglect": 21, "neglig": 21, "neighbor": [3, 6, 11], "neither": [4, 13], "neq": [13, 14, 21], "nervou": 12, "nest": [9, 12], "nesterov": 13, "net": [2, 4, 12], "netlib": [19, 24], "network": [0, 9, 13, 18, 23, 25], "neural": [0, 13, 18, 23, 25], "neural_network": [0, 1, 2, 24], "neuralnetwork": 1, "neuron": [1, 2, 3, 4, 12], "neutral": [0, 24], "neutron": [0, 24], "never": [1, 4, 6, 9, 21], "new": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 17, 19, 24, 25], "new_chang": 13, "new_hobbit": 24, "newaxi": [0, 3, 6, 9], "newli": [0, 24], "newton": [1, 7, 8, 13, 21], "next": [0, 1, 2, 3, 4, 5, 6, 8, 9, 13, 14, 15, 16, 24, 25], "next_guess": 13, "next_input": 4, "ng": 1, "ni": 14, "nice": [0, 1, 5, 11, 24, 25], "niter": 13, "nitric": [], "nlambda": [0, 5, 6, 25], "nlp": 23, "nm": 21, "nm_n": [0, 24], "nmse": 6, "nn": [2, 5, 6, 12, 19, 24], "nn_model": 1, "nnmin": 2, "node": [1, 3, 9, 10, 12], "nois": [0, 4, 5, 6, 8, 9, 10, 13, 24, 25], "noise_dimens": 4, "noisi": [1, 6], "non": [0, 1, 3, 5, 6, 7, 9, 10, 11, 12, 13, 14, 19, 21, 24, 25], "none": [0, 1, 2, 4, 5, 9, 10, 13, 21, 24, 25], "nonlinear": [3, 6, 8, 9, 11, 12], "nonneg": [6, 9, 13], "nonparametr": 6, "nonsens": 21, "nonsingular": 19, "nonumb": [3, 7, 8, 13, 19], "nor": [1, 4, 13], "norm": [0, 1, 5, 6, 8, 11, 13, 24, 25], "normal": [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 18, 19, 21, 24, 25], "normali": [19, 24], "norwai": [6, 24], "notat": [0, 2, 5, 6, 13, 14, 21, 24, 25], "note": [0, 1, 2, 3, 4, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16, 18, 19, 21, 23, 24], "notebook": [0, 1, 3, 9, 15, 16, 18, 24], "noth": [1, 2, 5, 8, 12, 14, 21, 25], "notic": [4, 5, 12, 13, 19, 21, 24], "notion": 3, "novel": [3, 6, 10, 24], "novemb": [1, 22, 24], "now": [0, 2, 4, 5, 6, 7, 8, 10, 11, 12, 14, 15, 16, 18, 19, 21, 24, 25], "nowadai": [0, 1, 3, 9, 18, 24], "nox": [], "np": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 19, 21, 24, 25], "npr": 2, "nsampl": 6, "nt": 2, "nu": 21, "nuclear": [5, 25], "nuclei": [0, 21, 24], "nucleon": [0, 24], "nucleu": [0, 24], "num": 4, "num_coordin": 2, "num_hidden_neuron": 2, "num_it": 2, "num_neuron": 2, "num_neurons_hidden": 2, "num_point": 2, "num_tre": 10, "num_valu": 2, "number": [1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 19, 20, 22, 24], "numberid": 7, "numberparamet": 3, "numer": [0, 5, 6, 9, 10, 11, 12, 13, 18, 19, 23, 24, 25], "numpi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 21, 25], "nunmpi": [5, 25], "nx": 2, "ny": 21, "o": [0, 1, 4, 5, 6, 7, 8, 9, 11, 19, 22, 23, 24, 25], "obei": [6, 11, 13], "object": [0, 1, 4, 8, 10, 15, 19, 24], "obliqu": [5, 25], "observ": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 21, 24], "obtain": [0, 1, 5, 6, 7, 8, 9, 10, 12, 13, 14, 17, 19, 21, 24, 25], "obviou": [5, 6, 11, 21, 25], "obviouli": 24, "obvious": [0, 4, 5, 6, 19, 24], "oc": 25, "occupi": [], "occur": [0, 6, 8, 9, 19, 21, 24], "octob": [22, 24], "od": 0, "odd": [0, 3, 7, 24, 25], "odenum": 2, "odesi": 2, "oen": 0, "off": [1, 3, 4, 5, 9, 13, 21], "offer": [6, 11, 18, 19, 20, 22, 24], "offic": [22, 24], "offici": [20, 24], "often": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 21, 24, 25], "ofter": [19, 24], "ol": [0, 13, 17, 25], "old": [1, 5, 10, 13, 15], "ols_paramet": 16, "ols_sk": 6, "ols_svd": 6, "olstheta": [0, 5], "omega": [2, 3, 6], "omega_0": 3, "omit": [0, 5, 24, 25], "onc": [1, 6, 9, 11, 13], "one": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 18, 19, 21, 22, 24, 25], "onehot": 1, "onehot_vector": 1, "onehotencod": 9, "ones": [0, 2, 5, 6, 8, 9, 10, 11, 13, 16, 19, 24, 25], "ones_lik": 4, "ong": 25, "onl": 3, "onli": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 21, 24, 25], "onlin": [11, 15, 20], "onto": [5, 11, 25], "open": [0, 1, 4, 6, 7, 9, 15, 18, 20, 22, 24], "oper": [0, 1, 3, 5, 6, 10, 11, 12, 13, 15, 16, 18, 21, 24, 25], "operation": 21, "oplu": 21, "opmiz": 13, "opportun": 0, "oppos": [6, 13], "opposit": [1, 5, 8, 25], "opt": [1, 5, 24], "optim": [0, 2, 3, 4, 5, 6, 7, 9, 10, 11, 14, 16, 17], "optimis": [1, 3], "option": [0, 1, 3, 5, 6, 8, 11, 15, 19, 25], "optmiz": [1, 8, 13, 25], "oral": 24, "orang": 0, "order": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 15, 19, 21, 24, 25], "ordinari": [0, 2, 3, 7, 11, 13, 17, 18], "oreilli": [23, 24], "org": [0, 3, 4, 16, 18, 19, 23, 24], "organ": [6, 7, 10, 19], "orient": [1, 5, 21, 25], "origin": [0, 3, 5, 6, 8, 11, 12, 13, 15, 19, 24, 25], "orthogn": [5, 25], "orthogon": [0, 5, 6, 8, 11, 13, 19, 24, 25], "orthonorm": [5, 25], "os": [22, 24], "oscar": 1, "oscil": [3, 13], "oskar": 24, "oskarlei": 24, "oslo": [0, 18, 20, 22, 24, 25], "osx": [0, 18, 24], "other": [0, 1, 2, 3, 5, 6, 7, 8, 10, 13, 14, 16, 18, 20, 21, 22, 23, 25], "otherwis": [0, 1, 4, 7, 13, 19, 24], "ouput": [5, 7, 12], "our": [1, 2, 3, 6, 7, 8, 9, 10, 12, 14, 15, 16, 17, 18, 19, 21], "ourmodel": 0, "ourselv": [0, 5, 6, 8, 11, 13, 24, 25], "out": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 18, 19, 21, 24, 25], "out_fil": 9, "outcom": [0, 7, 9, 10, 12, 21, 25], "outdoor": 9, "outer": [6, 12, 13], "outfil": 4, "outlier": [0, 8, 24, 25], "outlin": [6, 10, 11], "outlook": 9, "outperform": 10, "output": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 19, 21, 24, 25], "output_bia": 1, "output_bias_gradi": 1, "output_shap": 4, "output_weight": 1, "output_weights_gradi": 1, "outputlayer1": 12, "outputlayer2": 12, "outsid": 4, "over": [0, 1, 3, 4, 5, 6, 9, 10, 12, 13, 15, 16, 19, 24, 25], "over1": 13, "overal": [1, 10], "overcast": 9, "overcom": [12, 13], "overdetermin": [0, 24], "overfit": [0, 1, 3, 6, 9, 10, 13], "overflow": 5, "overhead": 12, "overlap": [3, 7, 8, 9], "overlin": [0, 5, 6, 9, 10, 11, 14, 19, 24, 25], "overst": 0, "overtrain": 4, "overview": 3, "own": [4, 5, 6, 8, 12, 13, 16, 18, 19], "owner": [], "ownmsepredict": 0, "ownmsetrain": 0, "ownridgetheta": [0, 6, 25], "ownypredictridg": 0, "ownytilderidg": 0, "oxid": [], "p": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 19, 21, 24, 25], "p0": 2, "p1": 2, "p_": [2, 4, 8, 9], "p_hidden": 2, "p_i": [5, 21], "p_j": 21, "p_n": 21, "p_output": 2, "p_x": 21, "pack": [0, 24], "packag": [0, 1, 3, 4, 5, 8, 11, 13, 15, 18, 21, 25], "packtpub": 24, "packtpublish": 24, "pad": [3, 4], "page": [0, 18, 24], "pai": [0, 1, 9, 13, 15], "pair": [0, 2, 3, 9, 18, 21, 24], "paltform": 15, "panda": [0, 4, 5, 6, 7, 9, 11, 18], "panel": 24, "paper": 1, "paradigm": [0, 24], "parallel": [10, 13, 18, 19, 24], "param": 2, "paramat": 2, "paramet": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 16, 17, 21], "parameter": [0, 6, 10, 24, 25], "parametr": [0, 6, 24, 25], "paramt": [3, 5], "part": [0, 1, 3, 5, 6, 10, 17, 19, 20, 21, 22, 24, 25], "partial": [0, 1, 5, 6, 7, 8, 10, 11, 12, 13, 16, 21, 24, 25], "particip": [15, 18, 20, 22, 24], "particl": [0, 4, 13, 21, 24], "particular": [0, 1, 2, 3, 5, 6, 9, 10, 11, 12, 13, 16, 21, 23, 24, 25], "particularli": [5, 6, 8, 11, 13, 21, 25], "partit": [1, 4, 9], "partli": [6, 24], "partner": 15, "pass": [2, 3, 12, 14], "past": [10, 21], "patch": [6, 21], "path": [0, 4, 6, 7, 9, 18, 24], "pathcollect": 17, "patient": 7, "patter": 4, "pattern": [0, 3, 4, 12, 23, 24], "pauli": [0, 24], "pc": [11, 15, 18], "pca": [0, 7, 18, 24, 25], "pd": [0, 4, 5, 6, 7, 9, 11, 24, 25], "pde": 2, "pdf": [0, 3, 4, 5, 6, 9, 15, 16, 23, 24], "pedagog": [0, 24, 25], "penal": 6, "penalti": [6, 13], "penros": [5, 6], "pentagon": 13, "peopl": [1, 9, 13, 18], "per": [0, 1, 6, 20, 22, 24], "percentag": [10, 11, 22], "perceptron": [0, 1, 7, 24], "peregrin": 24, "perfect": [0, 1, 13, 24], "perfectli": [4, 6], "perform": [0, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 14, 16, 18, 19, 21, 24, 25], "performac": 4, "perhap": [0, 5, 13, 24, 25], "perimet": 1, "period": [1, 4, 21], "permiss": 15, "permut": 11, "persist": 13, "person": [5, 6, 7, 16, 20, 22, 24], "perspect": 23, "pertin": [12, 24], "petal": [8, 9], "peter": [23, 25], "phantom": 21, "phase": [6, 12], "phenomena": 21, "phi": 8, "phi_k": 8, "philosophi": 13, "phone": [22, 24], "photo": [4, 24], "phrase": [0, 24], "physic": [0, 1, 4, 7, 12, 13, 21, 22, 23, 24, 25], "pi": [2, 3, 5, 6, 7, 9, 12, 13, 21], "pick": [1, 9, 10, 11, 13, 14], "pickl": 1, "pictur": [0, 24], "pie": [18, 24], "piec": [11, 14], "pillow": [0, 18, 24], "pinv": [5, 6, 13, 25], "pip": [0, 1, 15, 18, 24], "pip3": [0, 1, 24], "pipelin": [0, 6, 8, 10, 25], "pippin": 24, "pit": 4, "pitfal": 6, "pitt": 12, "pixel": [1, 3, 4, 24], "pixel_height": [1, 3], "pixel_width": [1, 3], "place": [0, 4, 6, 8, 13, 15, 19, 24], "plai": [0, 3, 4, 5, 6, 8, 11, 18, 24, 25], "plain": [8, 10, 12, 13, 14], "plan": [6, 9, 22, 23, 24], "plane": [8, 9], "plateau": 5, "platform": [18, 24], "plausibl": 12, "pleas": [13, 22, 24], "plenti": 1, "plethora": [3, 12], "plot": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 21, 24, 25], "plot_confusion_matrix": [7, 10], "plot_count": 6, "plot_cumulative_gain": [7, 10], "plot_data": 1, "plot_dataset": 8, "plot_decision_boundari": [9, 10], "plot_import": 10, "plot_max": 4, "plot_min": 4, "plot_model": 4, "plot_numb": 4, "plot_predict": 8, "plot_regression_predict": 9, "plot_result": 4, "plot_roc": [7, 10], "plot_surfac": [2, 6, 13], "plot_train": 9, "plot_tre": [9, 10], "plt": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 21, 24, 25], "plu": [0, 3, 5, 7, 24, 25], "pm": 8, "pmatrix": 2, "pml": 23, "pn": 3, "png": [0, 4, 6, 7, 9, 24], "point": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 19, 21, 22, 24, 25], "point_1": 4, "point_2": 4, "poisson": [18, 21, 24], "poli": [6, 8], "poly100_kernel_svm_clf": 8, "poly3": 0, "poly3_plot": 0, "poly_featur": [8, 9, 15], "poly_features10": 9, "poly_fit": 9, "poly_fit10": 9, "poly_kernel_svm_clf": 8, "poly_model": 15, "poly_ms": 15, "poly_predict": 15, "polydegre": [0, 5, 6, 10, 25], "polygon": 13, "polym": 12, "polynomi": [0, 5, 6, 7, 8, 9, 10, 11, 15, 17, 24, 25], "polynomial_featur": [6, 15, 16, 17], "polynomial_svm_clf": 8, "polynomialfeatur": [0, 6, 8, 9, 15, 16, 25], "polytrop": [0, 6], "pool": 3, "pool_siz": 3, "poor": [1, 13], "poorli": [0, 25], "popul": [0, 5, 24, 25], "popular": [0, 1, 3, 6, 7, 8, 9, 11, 12, 15, 18, 19, 21, 25], "popularli": [0, 24], "portabl": 10, "portion": [11, 13], "pose": [0, 4, 5, 6, 11, 21, 24], "posit": [0, 1, 2, 3, 5, 7, 8, 10, 11, 13, 14, 19, 21, 24, 25], "possibl": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 18, 19, 21, 22, 24, 25], "possibli": [6, 8, 13], "posterior": 5, "postpon": [0, 25], "postul": 5, "potenti": [0, 3, 5, 6, 12, 13], "pott": 12, "power": [0, 1, 5, 6, 8, 9, 12, 13, 24, 25], "pp": [5, 6], "practic": [0, 5, 6, 7, 8, 16, 21, 25], "practition": [0, 1, 3, 24], "pre": 24, "preced": [1, 11, 12, 21], "preceed": 4, "preceq": 8, "precis": [0, 2, 5, 11, 13, 19, 21, 24, 25], "pred": 6, "predicit": 0, "predict": [0, 1, 5, 6, 7, 8, 9, 10, 15, 16, 17, 18, 23, 24, 25], "predict_prob": 1, "predict_proba": [7, 10], "predictor": [0, 5, 6, 7, 9, 10, 11, 24, 25], "prefer": [0, 1, 6, 8, 9, 11, 13, 15, 18, 24], "prepar": [0, 6, 19, 24, 25], "preprocess": [0, 4, 6, 7, 8, 9, 10, 11, 15, 16, 17], "prerequisit": 0, "presenc": 13, "present": [0, 5, 6, 7, 9, 12, 13, 19, 21, 24, 25], "preserv": [3, 11, 19], "press": [13, 15, 23], "pretrain": [1, 4], "pretti": [0, 4, 8, 9, 18, 24], "prev_centroid": 14, "prevent": [13, 21], "previou": [0, 1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 15, 16, 19, 21, 25], "previous": [2, 3, 9, 10, 21], "price": [0, 4, 9, 13], "primal": 8, "primari": [0, 7, 24], "prime": 21, "princip": [0, 5, 7, 18, 24, 25], "principl": [0, 6, 7, 8, 14, 24], "print": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 19, 21, 24, 25], "print_funct": [8, 9], "printout": [0, 24], "prior": [0, 5, 6, 24], "privat": 0, "prob": [1, 21], "probabilist": [0, 23, 24, 25], "probabl": [0, 1, 3, 4, 6, 7, 10, 13, 18, 24, 25], "problem": [0, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 17, 18, 19, 21], "probml": 23, "proce": [0, 5, 6, 7, 8, 9, 10, 11, 13, 19, 24, 25], "procedur": [2, 4, 5, 6, 8, 10, 11, 13, 25], "proceed": 19, "process": [0, 2, 4, 6, 9, 10, 12, 13, 18, 19, 21, 23, 24], "prod": 23, "prod_": [1, 5, 7], "produc": [0, 3, 4, 5, 6, 9, 10, 11, 12, 13, 18, 19, 21, 24, 25], "product": [0, 1, 3, 5, 6, 7, 8, 12, 13, 16, 17, 18, 19, 24, 25], "profess": [0, 24], "program": [0, 1, 4, 5, 6, 8, 12, 14, 15, 18, 19, 20, 21, 22, 24, 25], "programm": 19, "progress": [1, 4, 14], "prohibit": 6, "project": [0, 1, 2, 3, 5, 11, 13, 15, 18, 20, 25], "project_root_dir": [0, 6, 7, 9, 24], "promin": 12, "promis": 8, "promot": [22, 24], "prone": [9, 15], "pronounc": [13, 18, 24], "proof": [0, 11, 12, 13, 24], "propag": [2, 3, 13], "proper": [0, 2, 6, 7], "properli": [1, 6, 8, 10, 13], "properti": [0, 1, 3, 12, 13, 16, 19, 24], "proport": [0, 1, 5, 9, 11, 13, 21, 24, 25], "propos": [1, 4, 6, 10, 24], "propto": [5, 13], "proton": [0, 24], "prove": [3, 13], "provid": [0, 1, 3, 4, 5, 6, 8, 9, 10, 12, 13, 18, 19, 21, 24, 25], "proxi": [1, 13], "prune": 9, "pseudo": [19, 21], "pseudoinv": 5, "pseudoinvers": [5, 6], "pseudorandom": [6, 21], "psychologi": [0, 24], "pt": 13, "public": [0, 15, 18, 24], "pull": 15, "punish": [0, 1, 24], "pure": [3, 9, 21], "purest": 9, "puriti": 9, "purpos": [0, 3, 10, 12, 14, 24], "push": 15, "put": 1, "py": 5, "pycod": 24, "pydata": 18, "pydot": 9, "pyhton2": 24, "pylab": [7, 24], "pypi": 18, "pyplot": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 21, 24, 25], "pythagora": 5, "python": [1, 2, 3, 5, 6, 8, 11, 12, 13, 14, 21, 25], "python2": 0, "python3": [0, 18, 24], "pytorch": [0, 18, 24], "q": [5, 6, 8, 11, 21], "qp": 8, "qquad": [2, 11, 13, 19], "qr": [5, 6, 19, 25], "quad": [1, 13, 19], "quadrat": [0, 8, 9, 13, 24], "qualit": [4, 9, 21], "qualiti": [0, 9, 18, 24, 25], "quantifi": 1, "quantil": 10, "quantit": [0, 6, 9, 24], "quantiti": [0, 2, 5, 6, 7, 9, 10, 11, 12, 14, 16, 19, 21, 24, 25], "quantum": [4, 12, 23, 24], "quartil": [0, 25], "quench": 5, "queri": 9, "question": [0, 5, 6, 9, 11, 12, 13, 22, 24, 25], "qugan": 4, "quick": [4, 21], "quickli": [1, 3, 9, 11, 13], "quit": [1, 5, 6, 9, 10, 12, 15, 25], "quot": 4, "r": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 19, 21, 25], "r2": [0, 5, 6, 24, 25], "r2_score": [0, 24], "r2score": [0, 24], "r_1": 9, "r_2": 9, "r_j": 9, "r_m": 9, "rad": [], "radial": [8, 12], "radioact": 21, "radiu": [0, 1, 25], "rain": 9, "ramp": 1, "ran0": 21, "ran1": 21, "ran2": 21, "ran3": 21, "rand": [0, 4, 5, 6, 9, 10, 13, 15, 19, 24, 25], "randint": [6, 9, 13], "randn": [0, 1, 2, 5, 6, 9, 11, 13, 15, 24, 25], "random": [0, 1, 2, 3, 4, 5, 6, 8, 9, 13, 14, 15, 16, 17, 18, 19, 24, 25], "random_forest_model": 10, "random_index": 13, "random_indic": [1, 3], "random_st": [7, 8, 9, 10, 11], "randomforestclassifi": 10, "randomli": [1, 6, 9, 13, 14], "rang": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 14, 19, 21, 24, 25], "rangl": [0, 6, 11, 21, 24, 25], "rangle_x": 21, "rank": [5, 25], "rankdir": 4, "raphson": [1, 8, 13], "rapidli": 0, "rare": [1, 13], "raschka": [24, 25], "rasckha": 24, "rate": [1, 2, 3, 4, 8, 9, 10, 12, 13], "rather": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 19, 21, 24, 25], "ratio": [4, 7, 9, 10, 11], "rational": [0, 24], "ravel": [5, 6, 7, 8, 9, 10, 11, 13, 19], "raw": 3, "rbf": [8, 11, 12], "rbf_kernel_svm_clf": 8, "rbf_pca": 11, "rc": 21, "rcond": [0, 24, 25], "rcparam": [1, 3, 7, 8, 9, 10, 21, 24], "re": [2, 4, 13, 15], "reach": [1, 4, 5, 6, 9, 10, 12, 13, 14], "read": [0, 2, 3, 4, 5, 6, 7, 8, 11, 12, 16, 17, 19, 21, 23], "read_csv": [0, 6, 7, 9], "read_fwf": [0, 24], "reader": [0, 6, 19, 21, 24], "readi": [0, 1, 5, 6, 8, 10, 11, 12, 19, 24], "readili": 1, "readm": 15, "readthedoc": 18, "real": [0, 1, 4, 7, 10, 11, 12, 16, 19, 25], "real_loss": 4, "real_output": 4, "realist": [8, 24], "realiti": 21, "realiz": [1, 12], "realli": [0, 1, 24], "rearrang": 13, "reason": [0, 1, 3, 4, 10, 13, 23, 24], "reassign": 1, "recal": [5, 6, 9, 10, 11, 12, 19, 21, 24, 25], "recast": 3, "receiv": [1, 3, 10, 12, 21], "recent": [0, 6, 13, 23], "recept": [3, 12], "receptive_field": 3, "recip": [0, 6, 7, 19, 24, 25], "reciproc": 5, "recogn": [0, 4, 5, 10, 24], "recognit": [0, 1, 3, 12, 23, 24], "recommen": 24, "recommend": [0, 2, 3, 4, 5, 6, 8, 13, 15, 18, 19, 23], "reconsid": 9, "reconstruct": 11, "record": [10, 20, 22, 24], "recreat": 15, "rectangl": [9, 13], "rectangular": [5, 25], "rectifi": [1, 3, 12], "recur": [0, 18, 24], "recurr": [0, 1, 18, 24], "recurs": [9, 18, 19, 24], "red": [0, 3, 4, 6, 8, 9], "redefin": [0, 10, 24, 25], "reduc": [1, 3, 5, 6, 9, 10, 11, 13, 24], "reduct": [0, 10, 11, 18, 21, 24, 25], "refer": [0, 1, 2, 3, 5, 6, 11, 12, 13, 14, 19, 23, 24, 25], "referenc": 2, "refin": 12, "refit": 6, "reflect": [0, 1, 4, 5, 21, 24], "refresh": [18, 24], "refreshprogrammingskil": 24, "reg": [10, 11], "regard": [1, 9, 13], "regardless": [12, 16], "region": [3, 4, 6, 9, 12], "regist": [6, 21], "reglasso": 5, "regr_1": [0, 9], "regr_2": [0, 9], "regr_3": [0, 9], "regress": [1, 8, 11, 12, 16, 18, 19], "regressor": [0, 7, 10], "regridg": [0, 5, 6, 25], "regular": [0, 3, 4, 5, 6, 7, 9, 13, 17, 22, 24, 25], "regularli": 15, "reilli": [0, 23, 24], "reinforc": [0, 8, 18, 24], "reiter": 1, "reject": 7, "rel": [0, 4, 6, 7, 9, 12, 13, 21, 24, 25], "relat": [0, 1, 3, 4, 5, 11, 13, 14, 19, 21, 24, 25], "relationship": [0, 4, 9, 24], "relativeerror": [0, 24, 25], "releas": [1, 18, 24], "relev": [0, 1, 5, 7, 11, 18, 21, 24], "reli": [0, 6, 8], "reliabl": [7, 21], "relu": [3, 4, 24], "remain": [1, 2, 4, 6, 12, 19, 21], "remaind": 21, "reman": 2, "remark": 1, "rememb": [0, 8, 13, 19, 24], "remind": [0, 5, 11, 13, 19, 21], "remot": 15, "remov": [4, 5, 6, 25], "renam": 15, "render": [0, 24, 25], "reorder": [5, 7, 25], "reorgan": [0, 24], "repeat": [0, 1, 3, 4, 5, 6, 9, 10, 11, 13, 14, 19, 21, 24, 25], "repeated": 24, "repeatedli": [0, 6, 10, 13], "repet": 3, "repetit": [6, 24, 25], "rephras": 13, "replac": [0, 1, 3, 4, 5, 6, 10, 12, 14, 18, 24, 25], "replica": 6, "repo": 15, "report": 24, "repositori": [4, 24], "repres": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 21, 24, 25], "represent": [0, 1, 3, 6, 21, 24], "representd": 3, "reproduc": [0, 5, 6, 9, 12, 15, 16, 18, 21, 24, 25], "repuls": [0, 24], "request": [0, 13], "requir": [0, 1, 3, 4, 5, 6, 8, 9, 11, 12, 13, 15, 17, 19, 24, 25], "res1": 2, "res2": 2, "res3": 2, "res_analyt": 2, "res_analytical1": 2, "res_analytical2": 2, "res_analytical3": 2, "resaml": 6, "resampl": [0, 7, 10, 18, 24, 25], "rescal": [0, 11, 12, 25], "rescu": 5, "reseach": 6, "research": [0, 4, 13, 18, 23, 24], "resembl": [6, 21], "reserv": [1, 5, 6, 21], "reshap": [0, 1, 2, 3, 4, 6, 8, 9, 10, 19, 24, 25], "residenti": [], "residu": [0, 5, 13, 24], "resiz": [5, 25], "resourc": 24, "respect": [0, 1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 13, 14, 16, 17, 21, 24, 25], "respond": 12, "respons": [0, 7, 9, 12, 24, 25], "rest": [0, 5, 25], "restat": [0, 12, 24], "restor": 4, "restored_discrimin": 4, "restored_gener": 4, "restrict": [0, 3, 9, 12, 24], "result": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 24], "retail": [], "retain": [5, 6, 25], "return": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 13, 14, 16, 17, 19, 21, 24, 25], "return_data": 14, "return_sequ": 4, "return_x_i": 9, "reus": [1, 3, 6], "reveal": [0, 12, 24], "revers": [1, 19], "review": [18, 19], "revisit": 14, "revolut": 24, "reward": [0, 4, 24], "rewrit": [0, 3, 5, 6, 7, 8, 10, 11, 12, 13, 16, 19, 21], "rewritten": [2, 6, 8, 10, 21], "rewrot": 13, "rf": 10, "rgb": 3, "rgoj5yh7evk": 18, "rh": 6, "rho": [0, 10, 13], "rho_1": 10, "rho_2": 10, "rho_m": 10, "rich": [0, 24], "ride": 9, "rideclass": 9, "ridedata": 9, "ridg": [7, 11, 13, 18, 24], "ridge_paramet": 17, "ridge_sk": 6, "ridgetheta": 5, "right": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 17, 19, 21, 24, 25], "right_sid": 2, "rightarrow": [0, 1, 5, 6, 8, 11, 12, 13, 21, 24, 25], "rigor": [0, 24, 25], "ring": 6, "rise": [0, 24], "risk": [0, 13, 24], "rival": 4, "river": [], "rlm": 24, "rm": 21, "rmse": [], "rmsporp": 13, "rmsprop": [1, 3, 4, 13], "rnd_clf": 10, "rng": 21, "rnn": [4, 12], "rnn1": 4, "rnn2": 4, "rnn_2layer": 4, "rnn_input": 4, "rnn_output": 4, "rnn_train": 4, "rntrick1": 21, "rntrick2": 21, "rntrick3": 21, "rntrick4": 21, "ro": [0, 13, 24], "robert": 23, "robust": [0, 24], "robustscal": [0, 25], "roc": [7, 10], "role": [0, 2, 5, 6, 8, 18, 24, 25], "roll": 6, "room": [0, 22, 24], "root": [0, 5, 9, 13, 15, 21, 25], "rot": 24, "rotat": [1, 8, 9, 10], "rotation_matrix": 9, "roughli": [1, 3], "round": [7, 9, 13], "routin": [13, 19, 24], "row": [0, 1, 2, 5, 6, 9, 11, 16, 19, 24, 25], "rr": [5, 25], "rrr": [5, 25], "rug": 13, "rule": [0, 1, 5, 6, 13, 24, 25], "run": [0, 1, 2, 4, 5, 6, 8, 9, 11, 13, 15, 18, 24, 25], "runtim": [1, 6, 14, 15], "rust": [0, 18, 19, 24], "rvert": 1, "rvert_2": 1, "s_": [3, 6], "s_1": 6, "s_i": [6, 7], "s_j": 6, "s_k": 6, "saddl": 13, "sai": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 19, 21, 24, 25], "said": [6, 9, 13], "sake": [0, 5, 7, 11, 24, 25], "sale": [0, 24], "sam": 24, "same": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 12, 14, 15, 16, 19, 21, 24, 25], "samm": 10, "sampl": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 14, 18, 19, 21, 24, 25], "sample_vari": 14, "sampleexptvari": 21, "samwis": 24, "sastri": 11, "satisfactori": [0, 24], "satisfi": [1, 2, 3, 6, 8, 13, 19, 21], "satur": [1, 6], "save": [0, 4, 6, 7, 9, 13, 24], "save_fig": [0, 6, 7, 9, 10, 24], "savefig": [0, 4, 6, 7, 9, 21, 24], "savetxt": 4, "saw": [5, 25], "scalabl": 10, "scalar": [2, 5, 6, 10, 25], "scale": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 18, 19, 22, 24], "scale_mean": 4, "scale_std": 4, "scaler": [0, 7, 8, 9, 10, 11, 17, 25], "scan": [5, 7], "scari": 5, "scatter": [0, 1, 6, 7, 8, 9, 14, 15, 17, 24], "scenario": [6, 13], "schedul": 13, "scheme": [1, 13], "schrage": 21, "sch\u00f8yen": 6, "scienc": [0, 1, 10, 12, 13, 18, 20, 21, 22, 23], "scientif": [0, 18, 24], "scientist": [0, 24], "scikit": [3, 5, 6, 8, 9, 10, 13, 15, 16, 18, 19, 23], "scikit_learn": 0, "scikitlearn": 24, "scikitplot": [7, 10], "scipi": [0, 3, 5, 6, 13, 18, 19, 24, 25], "scl": 6, "scm": 15, "score": [0, 1, 3, 6, 7, 9, 10, 11, 15, 16, 22, 24, 25], "scores_kfold": 6, "scratch": [1, 13, 16], "sdg": 13, "seaborn": [0, 1, 3, 6, 7, 24], "seamless": [0, 18, 24], "search": [0, 1, 3, 5, 9, 13, 15, 24], "sebastian": 24, "sebastianraschka": 24, "sec": 6, "second": [0, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 14, 15, 16, 18, 19, 21, 22, 24, 25], "secondeigvector": 11, "secondli": 12, "section": [4, 11, 16, 19, 21, 25], "sector": 0, "see": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15, 16, 18, 19, 21, 24, 25], "seed": [0, 1, 2, 3, 4, 5, 6, 8, 9, 11, 13, 14, 21, 24, 25], "seed_imag": 4, "seek": [1, 2, 8], "seem": [1, 3, 4], "seemingli": [0, 24], "seen": [0, 1, 3, 5, 10, 12, 21], "segment": 13, "seismic": 6, "seldomli": [0, 24], "select": [1, 5, 6, 8, 9, 10, 11, 15, 20, 21, 22, 23, 24, 25], "selevet": 15, "self": [1, 5, 25], "sell": 4, "semest": [7, 20], "semi": [8, 13], "semilogx": 6, "send": [5, 12, 13, 22, 24], "senior": [20, 22], "sens": [0, 4, 6, 8, 24], "sensibl": 3, "sensit": [0, 5, 6, 9, 13, 24, 25], "sent": 2, "sentenc": [4, 12], "separ": [0, 1, 2, 4, 6, 8, 9, 12, 14, 18, 21, 24], "septemb": 24, "sequenc": [3, 4, 7, 9, 10, 12, 13, 18, 19, 21, 24], "sequenti": [1, 3, 4, 10, 12, 21], "seri": [0, 1, 2, 3, 4, 5, 6, 10, 11, 12, 13, 19, 24, 25], "serif": [7, 21, 24], "serv": [0, 1, 2, 3, 5, 7, 13, 23, 24, 25], "session": [1, 15, 20, 22, 24], "set": [1, 4, 5, 6, 7, 8, 10, 11, 13, 14, 16, 17, 18, 19, 21, 22], "set_major_formatt": 6, "set_major_loc": 6, "set_tick": [1, 8], "set_ticklabel": 1, "set_titl": [0, 1, 2, 3, 7, 12, 14, 24], "set_xlabel": [0, 1, 2, 3, 7, 12, 24], "set_xlim": [7, 12], "set_xticklabel": 1, "set_ylabel": [0, 1, 2, 3, 7, 24], "set_ylim": [7, 12], "set_ytick": 7, "set_yticklabel": [1, 6], "set_zlim": 6, "seth": 4, "setminu": 6, "setosa": [8, 9], "setosa_or_versicolor": 8, "setp": 6, "setup": [1, 4, 6, 8, 18, 24, 25], "sever": [0, 3, 5, 6, 7, 8, 9, 11, 12, 13, 16, 18, 19, 21, 24, 25], "sgd": [1, 3], "sgd_clf": 8, "sgdclassifi": 8, "sgdreg": 13, "sgdregressor": 13, "sgn": [5, 25], "shallow": 13, "shape": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 19, 24, 25], "share": [1, 3, 15, 24], "shareabl": 15, "she": 7, "shift": [1, 6, 12, 15, 21], "ship": 3, "shire": 24, "short": [4, 5], "shortcom": 13, "shorten": 4, "shorter": 21, "shorthand": 24, "shortli": [19, 24], "should": [0, 2, 3, 5, 6, 8, 9, 11, 12, 15, 19, 21, 24, 25], "show": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 21, 24, 25], "show_shap": 4, "shown": [0, 4, 5, 8, 12, 13, 19, 25], "shrink": [3, 5, 6, 8, 11, 25], "shrinkag": [5, 6, 25], "shrunk": 11, "shuffl": [0, 1, 4, 6, 13, 25], "side": [0, 2, 5, 8, 12, 13, 19, 24], "sigh": [18, 24], "sigma": [0, 1, 5, 6, 7, 10, 11, 12, 13, 19, 21, 24, 25], "sigma0": 21, "sigma1": 21, "sigma2": 21, "sigma_": [5, 19, 24, 25], "sigma_0": [5, 25], "sigma_1": [5, 25], "sigma_2": [5, 25], "sigma_fn": [7, 12], "sigma_i": [0, 5, 24, 25], "sigma_j": [5, 25], "sigma_m": [6, 21], "sigma_n": [11, 21], "sigma_t": 13, "sigma_x": 21, "sigmoid": [1, 2, 4, 7, 8, 10, 12], "sigmundson": 6, "sign": [1, 2, 7, 8, 10, 21, 22], "signal": [1, 3, 10, 12], "signifi": 4, "signific": 1, "significantli": [1, 13, 21], "sim": [4, 5, 6, 13, 21], "similar": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 14, 18, 19, 24], "similarli": [0, 1, 3, 5, 8, 10, 13, 21, 24, 25], "simpl": [1, 2, 3, 5, 6, 7, 8, 10, 11, 12, 14, 16, 17, 18, 19, 21], "simplepredict": 10, "simpler": [0, 1, 5, 6, 7, 13, 16, 18, 24], "simplernn": 4, "simplest": [0, 1, 3, 4, 9, 10, 12, 14, 24], "simpletre": 10, "simpli": [0, 1, 2, 4, 5, 6, 8, 9, 10, 11, 12, 18, 19, 21, 24, 25], "simplic": [2, 5, 6, 7, 8, 9, 10, 11, 12, 14, 25], "simplicti": [5, 25], "simplifi": [0, 6, 9, 18, 24], "simplist": [3, 6, 21], "simul": 6, "simultan": 6, "sin": [0, 1, 2, 3, 4, 9, 12, 13, 19, 24], "sinc": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13, 16, 19, 21, 23, 24, 25], "sine": [3, 12], "singl": [0, 1, 2, 3, 5, 6, 7, 8, 9, 12, 13, 19, 21, 24, 25], "singular": [0, 6, 13, 19, 24], "sinusoid": 3, "site": [0, 20], "situat": [0, 4, 5, 7, 13, 21, 24, 25], "six": [3, 21], "size": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 13, 19, 21, 24], "sketch": 10, "ski": 9, "skill": 0, "skip": 11, "skl": [0, 6, 24, 25], "sklearn": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 17, 24, 25], "skplt": [7, 10], "sl": 6, "slack": 8, "slice": [2, 19, 24], "slide": [0, 3, 16, 21, 24, 25], "slight": [6, 13], "slightli": [1, 2, 3, 5, 6, 7, 10, 21, 25], "slope": [8, 11, 12], "slow": [0, 2, 8, 13, 25], "slower": [5, 19, 24, 25], "slowest": 19, "slowli": 12, "slp": 1, "small": [0, 1, 2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 18, 19, 21, 24, 25], "smaller": [0, 1, 2, 5, 6, 8, 9, 11, 13, 21, 24], "smallest": [0, 4, 14, 24], "smallest_row_index": 14, "smooth": [0, 3, 6, 13, 24], "sn": [0, 1, 3, 6, 7, 24], "sne": 11, "so": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 21, 22, 24, 25], "soar": 6, "social": 0, "soft": [1, 7, 10, 12], "soften": 8, "softmax": [3, 7], "softwar": [0, 8, 18, 19], "sol": 8, "sole": [0, 6, 24], "solid": [0, 7], "solut": [0, 1, 2, 3, 5, 6, 8, 10, 11, 13, 19, 21, 24, 25], "soluton": 2, "solv": [0, 1, 3, 5, 6, 8, 10, 11, 12, 13, 16, 19, 24, 25], "solve_expdec": 2, "solve_ode_deep_neural_network": 2, "solve_ode_neural_network": 2, "solve_pde_deep_neural_network": 2, "solveod": 2, "solveode_popul": 2, "solver": [2, 7, 8, 9, 10, 19, 24], "some": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 21, 24], "some_model": 6, "somehow": 4, "someon": 16, "someth": [0, 1, 3, 4, 7, 9, 11, 15, 21, 24, 25], "sometim": [0, 1, 11, 12, 13, 14, 25], "soon": [19, 22, 25], "sophist": [0, 24], "sopt": 13, "sort": [5, 6, 9, 11, 21], "sound": [3, 5], "sourc": [0, 1, 3, 6, 18, 19, 21, 24], "space": [0, 1, 4, 5, 8, 9, 11, 12, 13, 14, 21, 25], "span": [0, 3, 5, 9, 11, 19, 24, 25], "spare": 1, "spars": [3, 6, 19, 24], "sparse_mtx": [19, 24], "sparsecategoricalcrossentropi": 3, "sparsiti": 10, "spatial": [1, 2, 3, 12], "speak": 21, "special": [6, 7, 10, 12, 13, 19, 21, 24], "specif": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 15, 16, 18, 19, 21, 23, 24, 25], "specifi": [0, 3, 5, 6, 7, 9, 11, 13, 14, 21, 24], "specifici": [0, 10, 24], "spectacular": 3, "spectral": 1, "speech": [0, 1, 3, 4, 12], "speed": [1, 2, 4, 13], "spend": [16, 21], "sphere": [0, 25], "spin": 6, "spite": 0, "spline": 8, "split": [1, 3, 4, 5, 6, 8, 9, 10, 11, 14, 16, 17, 21, 24], "splite": 0, "splitter": [1, 10], "spontan": 21, "spot": 3, "spread": [0, 11, 21, 24, 25], "springer": [23, 24], "spuriou": 13, "sqrsignal": 3, "sqrt": [3, 4, 5, 6, 8, 10, 11, 13, 21, 25], "squar": [1, 2, 3, 4, 7, 8, 9, 11, 13, 14, 15, 17, 18, 19, 21], "squarederror": 10, "squaredeuclidean": 14, "squash": 12, "srtm": 6, "srtm_data_norway_1": 6, "stabil": 5, "stabl": [0, 4, 5, 6, 9, 16, 18, 24, 25], "stack": [3, 4], "stage": [5, 13, 15], "stai": [0, 2, 4, 5, 11, 24, 25], "stand": [0, 5, 9, 12, 24, 25], "standard": [0, 1, 4, 5, 6, 7, 8, 10, 12, 17, 19, 21, 24], "standardscal": [0, 6, 7, 8, 9, 10, 11, 17, 25], "stanford": 13, "start": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 21, 22, 24, 25], "start_tim": 14, "stat": 6, "state": [1, 2, 4, 5, 6, 7, 8, 10, 11, 12, 13, 18, 21, 24, 25], "statement": [0, 7, 19, 24], "statist": [0, 1, 3, 4, 7, 9, 10, 11, 12, 13, 14, 19, 23, 25], "statu": [0, 7, 15, 24], "stavang": 6, "std": [0, 4, 6, 24, 25], "steep": 13, "step": [0, 1, 2, 4, 6, 7, 9, 10, 11, 12, 13, 14, 15, 19, 24], "step_fn": [7, 12], "step_length": 13, "steps_list": 9, "stereo": 3, "still": [0, 2, 3, 5, 6, 11, 13, 21, 25], "stimuli": 12, "stk": [23, 24], "stk2100": [23, 24], "stk3155": [15, 20, 22], "stk4021": [23, 24], "stk4051": [23, 24], "stk4155": [20, 22], "stk5000": 23, "stochast": [0, 1, 5, 6, 8, 11, 12], "stock": 4, "stoke": 12, "stone": [0, 7], "stop": [1, 4, 9, 13, 14], "storag": [5, 25], "store": [0, 1, 2, 3, 6, 11, 13, 21, 24], "storehaug": [22, 24], "str": [1, 3, 4], "straight": [0, 6, 8, 13, 24], "straightforward": [0, 2, 3, 5, 6, 8, 9, 10, 13, 19, 24, 25], "strategi": [0, 1, 9, 24], "stratifi": 6, "strength": [0, 5, 14, 25], "stretch": 11, "strict": [8, 13], "strictli": [8, 13], "stride": [4, 19], "strike": 6, "string": 1, "stroke": 7, "strong": [3, 6, 9, 10, 12, 19, 21], "strongli": [0, 8, 15, 18, 19], "stronli": [], "structur": [0, 1, 2, 3, 6, 9, 10, 12, 18, 24], "stuck": [1, 13], "student": [0, 15, 20, 22, 23, 24], "studi": [0, 3, 4, 5, 6, 7, 8, 11, 12, 13, 18, 23, 24, 25], "studier": 23, "style": [7, 9, 19, 24], "st\u00f8land": 22, "sub": [9, 12], "subdivid": [0, 19, 24], "subfield": 0, "subject": [6, 8, 21], "submit": 24, "subplot": [0, 1, 3, 4, 6, 7, 8, 9, 10, 14, 24], "subplots_adjust": [8, 21], "subprogram": [19, 24], "subract": [0, 25], "subroutin": [0, 24], "subscript": 1, "subsequ": [1, 4, 5, 6, 12, 19, 21, 25], "subset": [1, 6, 9, 12, 13, 18, 24], "subspac": [0, 8, 11, 25], "substanti": [9, 10], "substep": 11, "substitut": [3, 6, 12, 16, 19], "subsubset": 9, "subtask": 6, "subtl": 1, "subtract": [0, 4, 5, 6, 11, 13, 19, 21, 25], "subtre": 9, "succeed": [0, 4, 24], "success": [3, 7, 9, 13, 21], "successfulli": [4, 9], "sudo": [0, 18, 24], "suffer": [0, 1, 2, 5, 10, 24, 25], "suffici": [1, 6, 8, 11, 13], "suggest": [1, 13, 23], "suit": [8, 12], "suitabl": [0, 15, 21, 25], "sum": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 21, 24, 25], "sum_": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 19, 21, 24, 25], "sum_i": [0, 2, 5, 6, 8, 13, 25], "sum_j": 6, "sum_ja_": 0, "sum_k": [6, 8, 12, 19], "sum_logist": 13, "sum_m": 3, "sum_n": 3, "sum_nx_": 3, "summar": [5, 6, 9], "summari": [1, 3, 4, 10, 20], "summat": [0, 3, 16, 25], "sunni": 9, "super": [5, 25], "superfici": 3, "superscript": [1, 12], "supervis": [0, 5, 6, 7, 9, 12, 18, 24, 25], "supplement": 7, "support": [0, 1, 9, 10, 11, 13, 18, 24, 25], "suppos": [0, 5, 6, 7, 8, 10, 11, 12, 13, 19, 24, 25], "suppress": [5, 13], "sure": [0, 1, 4, 6, 16], "surf": 6, "surfac": [0, 6, 24], "surpass": 6, "surpris": [0, 24], "surround": [3, 18], "survei": [0, 5, 6, 24], "svc": [8, 9, 10], "svd": [0, 6, 11, 24], "svdinv": 5, "svm": [8, 9, 10, 11], "svm_clf": [8, 10], "swath": [5, 25], "switch": 0, "sy": 13, "symbol": [1, 5, 11, 13, 18, 21, 24, 25], "symmeteri": 1, "symmetr": [0, 5, 8, 11, 12, 13, 19, 24, 25], "symmetri": 6, "sympi": [0, 18, 24], "synonim": 21, "syntax": 13, "system": [0, 1, 3, 4, 6, 7, 9, 10, 12, 13, 15, 18, 19, 24], "systemat": [4, 6], "t": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 24], "t0": [3, 6, 13], "t1": [2, 13], "t2": 2, "t3": 2, "t_": 2, "t_0": [2, 9, 13], "t_1": 13, "t_b": 10, "t_i": [1, 2, 5, 12, 25], "t_j": 12, "t_k": 9, "tabl": [9, 21, 22, 24], "tabul": [0, 24], "tabular": 24, "tackl": 4, "tag": [2, 3, 4, 5, 6, 7, 12, 13, 14, 19, 21, 25], "taht": [0, 24], "tail": 21, "tailor": [2, 8, 11, 24], "taiwan": [0, 24], "take": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17, 18, 19, 21, 24, 25], "taken": [0, 1, 3, 6, 10, 13, 19], "tan": 3, "tangent": [1, 4, 12, 13], "tanh": [1, 4, 7, 8, 12], "target": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 16, 24, 25], "target_nam": 9, "task": [0, 1, 3, 6, 9, 11, 12, 14, 24], "tau": [3, 5, 21], "taught": 24, "tax": [], "taylor": [2, 13], "taylornr": 13, "tc": 8, "teach": [15, 20, 24], "team": 1, "teaser": 0, "technic": [0, 5, 6, 13], "techniqu": [0, 1, 8, 10, 13, 18, 21, 23, 24, 25], "technologi": [0, 1], "tell": [0, 4, 6, 10, 11, 13, 16, 21], "temp": 1, "temp1": 1, "temp2": 1, "temperatur": [0, 9, 24], "temporarili": 1, "ten": [3, 24], "tend": [3, 5, 6, 8, 9, 10, 12, 13, 14], "tendenc": [0, 24], "tension": 6, "tensor": 3, "tensorflow": [0, 2, 4, 8, 14, 18, 19, 23, 24, 25], "term": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 21, 24, 25], "term1": [5, 6, 11], "term2": [5, 6, 11], "term3": [5, 6, 11], "term4": [5, 6, 11], "termin": [0, 4, 5, 9, 10, 13, 15, 25], "terminarl": 15, "terrain": 6, "terrain1": 6, "test": [3, 4, 5, 6, 7, 8, 9, 10, 13, 16, 19, 21, 24], "test_acc": 3, "test_accuraci": [1, 3], "test_error": 6, "test_imag": [3, 4], "test_ind": 6, "test_input": 4, "test_label": [3, 4], "test_loss": 3, "test_pr": 1, "test_predict": 1, "test_rnn": 4, "test_scor": [7, 10], "test_siz": [0, 1, 3, 5, 6, 10, 15, 17, 25], "test_split": 9, "testerror": [0, 6, 25], "testi": 4, "testpredict": 4, "testx": 4, "text": [0, 1, 2, 4, 5, 8, 9, 11, 13, 15, 19, 21, 23, 25], "textbook": [16, 25], "textual": 9, "textur": 1, "tf": [1, 3, 4, 13, 14], "th": [0, 1, 2, 5, 6, 7, 9, 12, 13, 14, 19, 21, 24, 25], "than": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 17, 18, 21, 24, 25], "thank": [4, 6], "theano": [1, 18, 24], "thei": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 16, 19, 21, 24, 25], "them": [0, 1, 3, 4, 6, 8, 9, 10, 11, 12, 13, 19, 24, 25], "theme": [0, 15, 24], "themselv": [0, 21, 24], "thenc": 6, "theorem": [2, 6, 7, 25], "theoret": [0, 4, 10], "theori": [0, 1, 3, 8, 9, 12, 13, 18, 23, 24], "thereaft": [0, 5, 6, 11, 12, 19, 24], "therebi": [0, 5, 7, 11, 24, 25], "therefor": [0, 1, 2, 3, 4, 6, 7, 8, 11, 13, 21, 24, 25], "therein": 11, "thereof": [0, 6, 13, 24], "theta": [0, 1, 4, 5, 6, 7, 13, 16, 21, 24, 25], "theta_": [0, 1, 6, 7, 13, 24, 25], "theta_0": [0, 5, 6, 7, 16, 24, 25], "theta_0x_": [0, 24, 25], "theta_1": [0, 5, 6, 7, 24, 25], "theta_1x_": [0, 24, 25], "theta_1x_0": [0, 24], "theta_1x_1": [0, 7, 24], "theta_1x_2": [0, 24], "theta_1x_i": [7, 25], "theta_2": [0, 24, 25], "theta_2x_": [0, 24, 25], "theta_2x_0": [0, 24], "theta_2x_1": [0, 24], "theta_2x_2": [0, 7, 24], "theta_2x_i": 25, "theta_3x_i": 25, "theta_4x_i": 25, "theta_i": [0, 1, 5, 24, 25], "theta_j": [0, 5, 6, 24, 25], "theta_linreg": 13, "theta_p": 7, "theta_px_p": 7, "theta_t": 13, "thetavalu": 5, "thi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 25], "thing": [0, 1, 2, 4, 5, 7, 9, 15, 16, 21, 24], "think": [0, 1, 3, 4, 6, 9, 12, 13, 14, 21, 24, 25], "third": [0, 3, 6, 13, 22, 24], "thirti": 7, "thorughout": 24, "those": [0, 3, 5, 6, 8, 9, 10, 11, 19, 24, 25], "though": [1, 2, 3, 4, 13, 16, 17, 19, 21], "thought": [6, 14, 21], "thousand": [0, 1, 25], "three": [0, 1, 3, 5, 6, 8, 9, 12, 19, 20, 21, 22, 24, 25], "threshold": [1, 3, 9, 10, 11, 12, 13], "through": [0, 1, 2, 3, 4, 5, 6, 8, 11, 12, 13, 14, 15, 18, 19, 21, 24, 25], "throughout": [0, 4, 5, 14, 15, 18, 19, 21, 24], "throw": [3, 6, 21], "thu": [0, 1, 2, 5, 6, 7, 8, 10, 11, 12, 13, 22, 24, 25], "thumb": [0, 6, 25], "thursdai": [], "tibshirani": [6, 23, 24], "tick_param": 6, "ticker": [6, 13, 21], "tif": 6, "tight_layout": [1, 7], "tightli": 11, "tild": [0, 5, 6, 7, 11, 21, 24, 25], "till": [0, 4, 7, 8, 9, 10, 12, 19, 24, 25], "time": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 24, 25], "timeit": 4, "timer": 4, "tini": 1, "tip": 3, "titl": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 13, 15, 21, 24], "tmp": 13, "tn": [2, 3, 7], "to_categor": [1, 3, 4], "to_categorical_numpi": 1, "to_numer": [0, 6, 24], "todai": 3, "togeth": [0, 3, 6, 8, 11, 13, 18, 24], "toi": 14, "told": 13, "toler": [2, 14], "tolist": 4, "tomographi": 12, "too": [0, 2, 4, 5, 6, 9, 11, 13, 17, 21, 23, 25], "took": [8, 24], "tool": [0, 1, 3, 6, 13, 15, 18, 25], "toolbox": 8, "top": [0, 3, 5, 6, 9, 10, 18, 24], "topic": [0, 5, 6, 7, 8, 18, 25], "topolog": [3, 12], "topologi": [1, 12], "torkjellsdatt": [22, 24], "toss": [10, 21], "total": [0, 1, 2, 3, 4, 6, 7, 8, 10, 11, 12, 13, 14, 19, 21, 22, 24, 25], "total_loss": 4, "totalclustervari": 14, "totalscatt": 14, "toward": [1, 2, 7, 12, 13, 15], "town": [], "tp": [4, 7], "tpng": 9, "tpu": [13, 18, 24], "tqdm": 6, "tr": [], "track": [3, 13, 14, 15, 19, 25], "tract": [], "tractabl": [0, 24, 25], "trade": [5, 9], "tradeoff": [0, 5, 24, 25], "tradit": [0, 1, 4, 6, 24], "train": [2, 3, 5, 6, 8, 9, 10, 11, 12, 13, 16, 17], "train_accuraci": [0, 1, 3, 24], "train_dataset": 4, "train_end": [0, 1, 25], "train_error": 6, "train_imag": [3, 4], "train_ind": 6, "train_label": [3, 4], "train_pr": 1, "train_siz": [0, 1, 3, 25], "train_step": 4, "train_test_split": [0, 1, 3, 5, 6, 7, 9, 10, 11, 15, 16, 17, 24, 25], "train_test_split_numpi": [0, 1, 25], "trainable_vari": 4, "trained_model": 6, "trainerror": [0, 25], "traini": 4, "training_checkpoint": 4, "training_dataset": 4, "training_gradi": 13, "trainingerror": 6, "trainpredict": 4, "trainscor": 4, "trainx": 4, "trait": [0, 24], "trajectori": 4, "transfer": [9, 24], "transform": [0, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17, 18, 19, 24, 25], "transit": [6, 12], "translat": [1, 4, 6, 10, 24], "transpos": [1, 5, 11, 19, 25], "travers": [0, 5], "treat": [0, 1, 3, 6, 12, 13, 21, 24, 25], "tree": [0, 1, 18, 24], "tree_clf": [9, 10], "tree_clf_": 9, "tree_clf_sr": 9, "tree_reg": 9, "tree_reg1": 9, "tree_reg2": 9, "trend": 21, "treue": 7, "trevor": 23, "tri": [2, 3, 4, 9, 13, 16], "triain": 0, "trial": [0, 2, 4, 6, 13, 21, 24], "triangl": 13, "triangular": 19, "trick": [3, 4, 8, 11, 13, 21], "trickier": 21, "tridiagon": 19, "trillion": 18, "trivial": [0, 1, 5, 11, 21, 24], "troubl": [0, 8, 12, 15, 25], "truck": 3, "true": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 16, 17, 19, 21, 24, 25], "true_fun": 6, "true_theta": 6, "truli": 24, "try": [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 18, 19, 21, 24], "tucker": 8, "tuesdai": [22, 24], "tumor": [7, 9], "tumour": 7, "tunabl": 1, "tune": [4, 9, 13, 19, 24], "turn": [0, 1, 5, 6, 7, 8, 9, 10, 11, 12, 13, 19, 21, 24, 25], "tutori": [1, 4], "tv": 2, "tveito": 2, "tweak": [1, 4, 10, 21], "twice": 13, "twist": 11, "two": [0, 1, 2, 4, 5, 6, 7, 9, 10, 11, 12, 13, 15, 17, 19, 20, 21, 23, 24, 25], "tx": 13, "tx_1": 13, "txt": [4, 15], "ty": 13, "type": [0, 1, 3, 6, 8, 10, 13, 19, 21, 25], "typic": [0, 1, 2, 3, 4, 5, 7, 9, 10, 12, 13, 15, 16, 21, 24, 25], "u": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 21, 23, 24, 25], "u_": 19, "u_i": 12, "u_m": 10, "ua": [0, 24], "ubuntu": [0, 18, 24], "uci": [], "uio": [15, 22, 23], "un": 14, "unabl": 15, "unari": [19, 24], "unbalanc": [6, 9], "unbias": [0, 5, 6, 24], "uncent": 6, "uncertainti": [0, 5, 24], "uncertitud": 21, "unchang": [1, 3], "uncorrel": [10, 21], "undefin": [5, 25], "under": [0, 1, 5, 6, 10, 13, 18, 24], "underdetermin": [0, 24], "underfit": [1, 6], "underflowproblem": 5, "undergo": 5, "undergradu": [20, 22], "underli": [0, 1, 9, 13, 21, 24], "underset": [4, 14], "understand": [0, 1, 3, 5, 6, 10, 13, 14, 15, 18, 24, 25], "understood": [8, 13], "undesir": 8, "undetermin": [5, 8], "undo": 4, "unexpect": 6, "unexpected": 21, "unfair": 6, "unfortun": [1, 8, 9, 10], "unicode_liter": [8, 9], "uniform": [0, 1, 5, 6, 11, 13, 21, 24], "uniformli": [13, 21], "unifrompdf": 21, "unimport": 13, "union": [5, 6], "uniqu": [0, 2, 6, 13, 14, 19, 24], "unique_cluster_label": 14, "unit": [0, 1, 3, 4, 5, 10, 12, 21, 24, 25], "unitari": [5, 6, 19, 25], "unitarili": [19, 24], "uniti": 21, "univari": 21, "univers": [0, 1, 2, 13, 18, 20, 22, 24, 25], "unix": 1, "unknow": [0, 19, 24], "unknown": [0, 1, 3, 4, 5, 6, 8, 10, 13, 19, 24, 25], "unknowwn": 12, "unlabel": 1, "unless": [0, 3, 6, 11, 13, 24], "unlik": [1, 3, 8, 13], "unnecessarili": 9, "unord": 3, "unravel": 1, "unrol": [3, 11], "unseen": [0, 7, 9, 15], "unstabl": 1, "unsupervis": [0, 1, 4, 12, 18, 24], "unsymmetr": [19, 24], "until": [1, 2, 4, 9, 12, 13, 14], "untouch": 0, "unusu": 12, "up": [1, 3, 4, 5, 6, 8, 10, 11, 13, 14, 16, 18, 19, 21, 22], "updat": [1, 2, 10, 12, 13, 14, 15], "uploa": 24, "upload": [15, 18, 23], "upon": [0, 1, 6, 7, 11, 19], "upper": [0, 8, 9, 16, 19, 25], "uppercas": [19, 24], "upsampl": 4, "upscal": 4, "url": [24, 25], "us": [4, 5, 6, 8, 9, 10, 11, 12, 14, 15, 17, 19, 21, 23], "usag": [0, 8, 18, 24, 25], "usd": [], "usd10000": [], "use_bia": 4, "usecol": [0, 24], "useless": 1, "user": [0, 1, 2, 4, 6, 7, 15, 18, 19, 24], "usernam": 15, "usetex": 21, "usg": 6, "usr": 21, "usual": [0, 3, 4, 7, 12, 13, 14, 24], "ut": 5, "util": [1, 3, 4, 6, 7, 10, 14, 24], "ux": 19, "v": [2, 4, 5, 6, 11, 13, 15, 18, 25], "v0": 21, "v1": 21, "v2": 21, "v_0": 11, "va": 1, "vahid": 24, "val": 13, "val_accuraci": 3, "val_loss": 4, "vale": 2, "valid": [0, 1, 4, 7, 9, 10, 13, 18, 21, 24, 25], "validation_data": 3, "validation_split": 4, "valu": [0, 1, 2, 3, 4, 6, 7, 8, 9, 10, 12, 13, 14, 16, 17, 18, 19, 24], "valuat": 9, "valy": 4, "van": [0, 24, 25], "vandenbergh": [8, 13], "vandermond": [0, 24], "vanilla": [0, 6, 11, 14, 25], "vanish": [1, 4, 13, 21], "var": [5, 6, 10, 11, 21, 25], "var_x": 21, "varabl": 8, "varepsilon": [5, 6], "varepsilon_": [5, 6], "varepsilon_i": [5, 6], "vari": [0, 1, 3, 5, 6, 10, 24], "variabl": [0, 1, 2, 5, 6, 7, 8, 10, 11, 12, 13, 14, 19, 24, 25], "varianc": [0, 1, 5, 7, 9, 10, 11, 13, 14, 18, 19, 21, 24, 25], "variance_i": [5, 11, 25], "variance_x": [5, 11, 25], "variant": [0, 1, 6, 8, 12, 13, 24, 25], "variat": [3, 4, 11, 24], "varieti": [0, 3, 12, 18, 24], "variou": [1, 3, 5, 6, 7, 8, 9, 11, 12, 13, 16, 18, 19, 21, 24, 25], "varydimens": 4, "vastli": 3, "vaue": 1, "vault": 0, "vdot": [2, 13], "vec": 6, "vector": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 13, 14, 17, 18], "vector_mean": 14, "ventur": [0, 8, 18, 24], "venv": 15, "verbos": [1, 3, 4], "veri": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 21, 23, 24, 25], "verifi": [3, 11, 19, 24], "versatil": [8, 24], "versicolor": [8, 9], "version": [0, 3, 10, 13, 14, 15, 18, 19, 21, 24], "versu": 1, "vert": [0, 1, 5, 6, 7, 8, 9, 11, 13, 16, 17, 24, 25], "vert_1": [5, 6, 25], "vert_2": [5, 6, 11, 17, 25], "via": [0, 5, 6, 7, 8, 9, 10, 11, 12, 18, 19, 20, 21, 22, 24, 25], "vidal": 11, "video": [0, 1, 12, 18, 20, 22, 24], "view": [1, 3, 5, 6, 12, 13, 21, 23, 24], "violat": 8, "virginica": 9, "viridi": [0, 1, 2, 3, 24], "virtual": 1, "viscos": 13, "viscou": 13, "visibl": 15, "vision": [0, 3], "visual": [0, 3, 11, 12, 18, 24, 25], "visualis": 1, "visualstudio": [15, 16], "viz": [6, 8, 21], "vmap": 13, "vmax": [1, 6], "vmin": [1, 6], "voic": 3, "volum": [0, 3, 24], "vote": [10, 24], "voting_clf": 10, "votingclassifi": 10, "votingsimpl": 10, "vstack": [5, 11, 19, 21, 24, 25], "vt": [5, 25], "w": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 19, 21, 24, 25], "w1": 8, "w2": [8, 11], "w3": 8, "w_": [1, 12], "w_1": [8, 19], "w_1x_": 8, "w_1x_1": 8, "w_2": [8, 19], "w_2x_": 8, "w_2x_2": 8, "w_3": 19, "w_4": 19, "w_hidden": 2, "w_i": [1, 2, 10], "w_ix_i": 12, "w_j": 19, "w_m": 19, "w_output": 2, "w_px_": 8, "w_px_p": 8, "wa": [1, 3, 4, 5, 6, 7, 10, 11, 12, 14, 17, 19, 24, 25], "wai": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 19, 21, 24, 25], "walk": 9, "walker": 21, "wang": [0, 24], "want": [0, 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 21, 24, 25], "warn": 4, "warrant": 6, "wast": 3, "watch": 18, "wave": 3, "wavelet": 8, "we": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23, 25], "weak": [9, 10, 14], "weather": [1, 12], "web": [18, 20, 22, 24], "webpag": 24, "websit": [6, 19, 20, 24], "wedg": [8, 21], "wednesdai": [22, 24], "wee": 11, "week": [0, 5, 6, 7, 17, 20, 22], "weekli": [15, 16, 18, 20, 22, 23, 24], "weight": [1, 2, 3, 6, 7, 9, 10, 12, 13, 21], "weigth": 2, "welcom": [8, 15, 18], "well": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 15, 16, 18, 19, 21, 23, 24, 25], "went": 8, "were": [0, 1, 3, 4, 5, 6, 7, 8, 10, 11, 12, 14, 21, 24], "wessel": [0, 24, 25], "what": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21], "whatev": 3, "when": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 19, 21, 24, 25], "whenev": [13, 15, 21], "where": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 24, 25], "wherea": [6, 21], "wherein": [1, 12], "whether": [0, 3, 5, 7, 9, 21, 24], "which": [0, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24, 25], "whichev": [1, 3], "while": [0, 1, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 15, 16, 21, 24, 25], "white": 9, "who": [0, 15], "whole": [1, 3, 4, 5, 9, 11, 13], "whose": [0, 6, 10, 21, 25], "whow": [11, 25], "why": [0, 1, 3, 6, 13, 15, 16, 17], "wide": [0, 1, 3, 6, 7, 12, 18, 19, 24], "widehat": 6, "width": [0, 3, 8, 9, 24], "wieringen": [0, 24, 25], "win": 10, "wind": 9, "wing": [22, 24], "winther": 2, "wiothout": 6, "wiscons": 7, "wisconsin": 10, "wisdom": 6, "wise": [1, 5, 12, 13, 25], "wish": [0, 2, 5, 7, 8, 11, 13, 14, 19, 24, 25], "with_std": [0, 25], "wither": 6, "within": [0, 2, 3, 4, 7, 9, 12, 13, 14, 21, 23, 24], "withinclust": 14, "without": [0, 1, 5, 6, 8, 9, 11, 12, 13, 15, 24, 25], "won": [0, 15, 24], "wonder": 8, "word": [0, 1, 3, 4, 5, 6, 7, 14, 21, 24, 25], "work": [0, 1, 4, 6, 7, 8, 9, 13, 15, 16, 18, 20, 21, 22, 24, 25], "workshop": 24, "world": [0, 8, 16, 25], "worldwid": [0, 24], "worri": 15, "wors": [0, 1, 3, 4, 6, 24], "worth": 9, "would": [0, 1, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16, 19, 21, 24, 25], "wrap": [6, 19, 24], "write": [0, 1, 2, 3, 5, 6, 7, 8, 12, 13, 15, 16, 19, 24, 25], "written": [0, 2, 3, 5, 11, 12, 13, 16, 18, 19, 21, 24, 25], "wrong": [1, 8, 15], "wrongli": 10, "wrote": [5, 11, 25], "wrt": [10, 13], "wth": [10, 13], "www": [18, 19, 23, 24], "wx_1": 8, "x": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 21, 24], "x0": 8, "x1": [4, 8, 9, 10, 13], "x1_exampl": 8, "x1d": 8, "x2": [8, 9, 10, 13], "x2d": [8, 11], "x2d_train": 11, "x2dsl": 11, "x3": 8, "x_": [0, 2, 3, 5, 6, 8, 10, 11, 13, 14, 19, 21, 24, 25], "x_0": [0, 5, 11, 19, 24, 25], "x_1": [0, 2, 5, 6, 7, 8, 9, 10, 11, 13, 19, 21, 24, 25], "x_2": [0, 2, 5, 6, 7, 8, 9, 10, 11, 13, 19, 21, 24, 25], "x_3": [8, 19, 21], "x_4": 19, "x_center": 11, "x_data": 1, "x_data_ful": 1, "x_hidden": 2, "x_i": [0, 1, 2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 19, 21, 24, 25], "x_input": 2, "x_ix_": [0, 24], "x_iy_i": 8, "x_j": [0, 2, 8, 9, 12, 16, 21, 25], "x_jy_j": 8, "x_k": [12, 14, 19, 21, 25], "x_l": 21, "x_m": [6, 12, 19, 21], "x_n": [0, 2, 3, 6, 8, 11, 12, 13, 19, 21, 24], "x_new": [9, 10], "x_offset": 6, "x_output": 2, "x_p": [3, 7, 9], "x_poli": 9, "x_poly10": 9, "x_pred": 4, "x_prev": 2, "x_reduc": 11, "x_scale": 8, "x_small": 13, "x_test": [0, 1, 3, 5, 6, 7, 9, 10, 11, 15, 16, 17, 25], "x_test_": 17, "x_test_own": 6, "x_test_scal": [0, 6, 7, 9, 10, 11, 25], "x_tot": 4, "x_train": [0, 1, 3, 4, 5, 6, 7, 9, 10, 11, 15, 16, 17, 24, 25], "x_train_": 17, "x_train_mean": 6, "x_train_own": 6, "x_train_scal": [0, 6, 7, 9, 10, 11, 25], "x_val": 1, "xarrai": [18, 24], "xavier": 1, "xbnew": 13, "xcode": [0, 18, 24], "xdclassiffierconfus": 10, "xdclassiffierroc": 10, "xg_clf": 10, "xgb": 10, "xgbclassifi": 10, "xgboost": 9, "xgboot": 10, "xgbregressor": 10, "xgparam": 10, "xgtree": 10, "xi": [8, 13], "xi_": 8, "xi_1": 8, "xi_i": 8, "xk": 8, "xla": [13, 18, 24], "xlabel": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 21, 24, 25], "xlim": [6, 10], "xm": 9, "xmesh": 13, "xnew": [0, 13, 24], "xp": 21, "xpanda": [0, 25], "xpd": [5, 11, 25], "xplot": 0, "xscale": [0, 25], "xsr": 9, "xt_x": 13, "xtest": 6, "xtick": [3, 6, 8, 9], "xtrain": 6, "xu": [0, 24], "xx": [0, 19, 24], "xy": [0, 6, 8, 19, 24], "xytext": 8, "xz": [19, 24], "y": [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 21, 24, 25], "y1": 4, "y2": 4, "y3": 4, "y_": [0, 1, 5, 6, 10, 11, 19, 24, 25], "y_0": [0, 5, 11, 19, 24, 25], "y_1": [0, 5, 8, 9, 11, 13, 19, 24, 25], "y_1y_1": 8, "y_1y_1k": 8, "y_1y_2": 8, "y_1y_2k": 8, "y_1y_n": 8, "y_1y_nk": 8, "y_2": [0, 5, 8, 9, 11, 19, 24, 25], "y_2y_1": 8, "y_2y_1k": 8, "y_2y_2": 8, "y_2y_2k": 8, "y_3": [0, 9, 19], "y_4": 19, "y_data": [0, 1, 5, 6, 24, 25], "y_data_ful": 1, "y_decis": 8, "y_fit": [0, 25], "y_i": [0, 1, 5, 6, 7, 8, 9, 10, 11, 12, 13, 19, 24, 25], "y_if_": 10, "y_ix_": [0, 24], "y_ix_i": [7, 8, 13, 25], "y_iy_jk": 8, "y_j": [6, 8, 12], "y_k": 12, "y_m": 19, "y_model": [0, 4, 5, 6, 24, 25], "y_n": [8, 13], "y_ny_1": 8, "y_ny_1k": 8, "y_ny_2": 8, "y_ny_2k": 8, "y_ny_n": 8, "y_ny_nk": 8, "y_offset": [6, 17], "y_plot": 9, "y_pred": [0, 1, 4, 6, 7, 8, 9, 10, 25], "y_pred1": 9, "y_pred2": 9, "y_pred_rf": 10, "y_pred_tre": 10, "y_proba": [7, 10], "y_scaler": 6, "y_test": [0, 1, 3, 4, 5, 6, 7, 9, 10, 11, 15, 16, 17, 25], "y_test_onehot": 1, "y_test_predict": [], "y_tot": 4, "y_train": [0, 1, 3, 4, 5, 6, 7, 9, 10, 11, 15, 16, 17, 24, 25], "y_train_mean": 6, "y_train_onehot": 1, "y_train_predict": [], "y_train_scal": 6, "y_val": 1, "ye": [3, 6, 7], "year": [0, 18, 24], "yet": [0, 1, 6, 8, 11, 13, 24], "yi": 13, "yield": [0, 2, 5, 6, 8, 10, 12, 13, 14, 19, 21, 24], "yk": 8, "ylabel": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 21, 24, 25], "ylim": [3, 6], "ym": 9, "ymesh": 13, "yn": 0, "yo": [8, 9, 10], "yoshua": [1, 23], "you": [0, 1, 3, 4, 5, 6, 8, 9, 10, 11, 13, 15, 16, 17, 18, 19, 21, 22, 23, 24, 25], "young": 0, "your": [1, 2, 4, 5, 6, 8, 11, 13, 15, 17, 18, 19, 24], "your_model_object": 16, "yourself": [11, 13, 24], "youtub": 18, "ypred": 6, "ypredict": [0, 13, 24, 25], "ypredict2": 13, "ypredictlasso": 5, "ypredictol": [0, 5], "ypredictown": 6, "ypredictownridg": [6, 25], "ypredictridg": [0, 5, 6, 25], "ypredictskl": 6, "ytest": 6, "ytick": [3, 6, 8, 9], "ytild": [0, 6, 24, 25], "ytildelasso": 5, "ytildenp": [0, 24, 25], "ytildeol": [0, 5], "ytildeownridg": [6, 25], "ytilderidg": [5, 6, 25], "ytrain": 6, "yuxi": 24, "yx": [19, 24], "yy": [19, 24], "yz": [19, 24], "z": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 19, 21, 24, 25], "z_": [1, 2, 12, 19, 24], "z_0": [19, 24], "z_1": [19, 24], "z_2": [19, 24], "z_c": 1, "z_h": 1, "z_hidden": 2, "z_i": [1, 12], "z_j": [1, 12], "z_k": [12, 25], "z_m": 1, "z_mod": 9, "z_o": 1, "z_output": 2, "zaman": 21, "zaxi": 6, "zero": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 19, 21, 24, 25], "zeros_lik": 4, "zeroth": 25, "zfill": 4, "zip": [4, 6], "zm_h": [0, 24], "zn": [], "zone": [], "zoom": 24, "zx": [19, 24], "zy": [19, 24], "zz": [19, 24], "\u00f8yvind": 6}, "titles": ["3. Linear Regression", "14. Building a Feed Forward Neural Network", "15. Solving Differential Equations with Deep Learning", "16. Convolutional Neural Networks", "17. Recurrent neural networks: Overarching view", "4. Ridge and Lasso Regression", "5. Resampling Methods", "6. Logistic Regression", "8. Support Vector Machines, overarching aims", "9. Decision trees, overarching aims", "10. Ensemble Methods: From a Single Tree to Many Trees and Extreme Boosting, Meet the Jungle of Methods", "11. Basic ideas of the Principal Component Analysis (PCA)", "13. Neural networks", "7. Optimization, the central part of any Machine Learning algortithm", "12. Clustering and Unsupervised Learning", "Exercises week 34", "Exercises week 35", "Weekly Exercises 3", "Applied Data Analysis and Machine Learning", "2. Linear Algebra, Handling of Arrays and more Python Features", "Course setting", "1. Elements of Probability Theory and Statistical Data Analysis", "Teachers and Grading", "Textbooks", "Week 34: Introduction to the course, Logistics and Practicalities", "Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression"], "titleterms": {"": [8, 10], "1": [0, 15, 16, 17, 25], "2": [0, 15, 16, 17, 24, 25], "2023": 22, "3": [0, 15, 16, 17, 25], "34": [15, 24], "35": [16, 25], "4": [0, 15, 16, 17, 25], "5": [0, 16], "A": [0, 1, 4, 8, 9, 24], "And": [24, 25], "In": 22, "Ising": 6, "The": [0, 1, 2, 3, 5, 6, 7, 8, 9, 11, 12, 15, 18, 24, 25], "To": 24, "With": 4, "about": [24, 25], "activ": [1, 12], "ad": [0, 6, 24, 25], "adaboost": 10, "adagrad": 13, "adam": 13, "adapt": 10, "adjust": 1, "adversari": 4, "again": [3, 9], "ai": 24, "aim": [8, 9, 24], "aka": 24, "algebra": [19, 24], "algorithm": [9, 10, 11, 12, 24, 25], "algortithm": 13, "all": 8, "an": [0, 4, 10, 15, 24], "analys": [5, 25], "analysi": [0, 5, 6, 11, 18, 21, 24, 25], "analyt": [0, 16], "ani": 13, "anoth": 9, "appli": 18, "approach": [0, 8, 14, 24], "approxim": 12, "architectur": 1, "arrai": [19, 24], "assist": 22, "autocorrel": 21, "autograd": [2, 13], "automat": 13, "back": [1, 11, 12, 25], "background": 18, "bag": 10, "base": 13, "basic": [0, 5, 7, 9, 10, 11, 19, 25], "batch": 1, "bay": 5, "befor": 11, "better": 8, "bia": 6, "binari": 1, "bind": 24, "bird": 10, "boldsymbol": 25, "boost": 10, "bootstrap": [6, 10], "boston": [], "breast": 1, "brief": 24, "bring": 12, "build": [1, 3, 9], "c": 24, "calcul": 25, "can": 24, "cancer": [1, 7, 9, 11], "cart": 9, "case": [8, 10, 21, 25], "central": [13, 18, 21], "chain": 12, "chang": 10, "channel": 24, "chi": [0, 24], "choic": 17, "choos": 1, "cifar01": 3, "classic": 11, "classif": [1, 9, 10], "classifi": 8, "clip": 1, "cluster": 14, "cnn": 3, "code": [1, 2, 5, 9, 11, 12, 13, 14, 15, 16, 24, 25], "collect": [1, 3], "commun": 24, "compar": [2, 10, 16], "complet": 25, "complex": [0, 6, 25], "complic": 6, "compon": 11, "comput": 9, "computerlab": 24, "con": 9, "concept": 21, "conjug": 13, "contn": 24, "convex": [8, 13], "convolut": [3, 12], "correl": [11, 25], "cost": [1, 10, 25], "cours": [18, 20, 23, 24], "covari": [5, 11, 21, 25], "cover": 24, "creat": 16, "cross": 6, "cython": 24, "data": [0, 1, 3, 6, 7, 9, 11, 15, 17, 18, 21, 24, 25], "dataset": [1, 3], "david": 24, "deadlin": 24, "deadllin": 22, "decai": 2, "decis": [9, 10], "decomposit": [5, 11, 19, 25], "deeep": [], "deep": [1, 2, 24], "defin": [1, 24], "degre": [0, 17, 25], "deliver": [15, 16], "dens": 0, "deriv": [5, 12, 16, 17, 25], "descent": [2, 10, 13], "design": 25, "detail": [3, 24], "develop": 1, "diagon": 11, "differ": 8, "differenti": [2, 13], "diffus": 2, "dimension": [2, 3, 8], "disadvantag": 9, "discret": 21, "discrimin": 24, "distribut": [5, 21], "do": 1, "doe": 25, "domain": 21, "down": 1, "dropout": 1, "economi": 25, "element": [0, 21, 24], "elimin": 19, "energi": 24, "ensembl": 10, "entropi": 9, "environ": [0, 15], "equat": [0, 2, 12, 25], "error": [0, 10, 24, 25], "essenti": 24, "etc": 24, "euler": 2, "evalu": 1, "exampl": [1, 2, 3, 4, 6, 7, 8, 9, 10, 24, 25], "exercis": [0, 6, 15, 16, 17, 25], "expect": 21, "experi": 21, "explor": 0, "exponenti": 2, "express": [16, 17, 25], "extrapol": 4, "extrem": [10, 24], "ey": 10, "fall": 22, "famili": [1, 24], "famou": 19, "fantast": 25, "featur": [9, 16, 19, 25], "feed": [1, 12], "final": [12, 25], "find": 16, "fine": 1, "first": [4, 12, 24], "fit": [0, 10, 15, 16, 24], "fix": 25, "forc": 3, "forest": 10, "format": 24, "forward": [1, 2, 12], "foster": 24, "fourier": 3, "frank": 6, "freedom": [0, 17, 25], "frequent": 25, "frequentist": [0, 24], "from": [5, 10, 12, 24, 25], "full": 2, "function": [0, 1, 6, 7, 8, 10, 11, 12, 13, 21, 24, 25], "further": [3, 5, 25], "gan": 4, "gaussian": 19, "gd": 13, "gener": [4, 9, 24], "geometr": 11, "gini": 9, "github": 15, "goal": [15, 16, 17], "good": [0, 24], "grade": [22, 24], "gradient": [1, 2, 10, 13], "growth": 2, "ha": 18, "handl": [19, 24], "hessian": 25, "hidden": 2, "hous": [], "how": 16, "hyperparamet": [1, 17], "hyperplan": 8, "i": [0, 1, 24], "id3": 9, "idea": 11, "ii": 24, "implement": [1, 16, 17], "implic": [5, 25], "import": [5, 19, 24, 25], "improv": 1, "includ": 13, "increment": 11, "index": 9, "inform": 22, "input": 2, "instal": [18, 24], "instructor": 22, "interpret": [5, 11, 24, 25], "introduc": [11, 13, 25], "introduct": [0, 6, 18, 19, 24], "invers": [5, 19], "invert": 25, "iter": 10, "its": 25, "jacobian": 25, "jax": 13, "julia": 24, "jungl": 10, "kera": [1, 3], "kernel": [8, 11], "lagrangian": 8, "lasso": [5, 6, 25], "last": 25, "later": [5, 25], "layer": [1, 2, 3, 12], "learn": [0, 1, 2, 11, 13, 14, 15, 16, 17, 18, 24, 25], "least": [5, 6, 16, 24, 25], "lectur": 24, "level": 10, "librari": [18, 24], "likelihood": 7, "limit": [1, 13, 21], "linear": [0, 8, 13, 15, 19, 24, 25], "link": [5, 11, 23, 25], "logist": [7, 24], "loss": 25, "lu": 19, "machin": [0, 8, 13, 18, 24], "main": [21, 24], "make": [0, 9, 10, 25], "mani": [10, 12], "mass": 24, "materi": 24, "math": [5, 25], "mathemat": [3, 5, 8, 25], "matric": [5, 19, 24], "matrix": [1, 5, 11, 12, 16, 19, 24, 25], "matter": 0, "max": 25, "mean": [0, 25], "meet": [5, 10, 21, 24, 25], "mercer": 8, "method": [6, 9, 10, 13, 24], "min": 25, "minim": 24, "ml": 24, "mlp": 12, "mnist": [3, 4], "model": [0, 1, 4, 6, 12, 15, 17, 24], "momentum": 13, "moon": [8, 9], "more": [3, 6, 19, 24, 25], "multilay": 12, "multipl": [1, 3, 17], "multipli": 8, "need": 24, "network": [1, 2, 3, 4, 7, 12, 24], "neural": [1, 2, 3, 4, 7, 12, 24], "new": 4, "non": 8, "normal": [0, 1], "notat": 12, "note": 25, "now": [1, 9, 13], "nuclear": [0, 24], "numba": 24, "number": [0, 2, 21, 25], "numer": [2, 21], "numpi": [19, 24], "object": 3, "obtain": 11, "od": 2, "off": 6, "ol": [5, 6, 15, 16], "one": [2, 12], "oper": 19, "optim": [1, 8, 13, 18, 24, 25], "order": 13, "ordinari": [5, 6, 16, 24, 25], "organ": [0, 24], "oslo": 23, "other": [4, 9, 11, 12, 19, 24], "our": [0, 4, 5, 11, 13, 24, 25], "outcom": [18, 24], "output": 2, "overarch": [0, 4, 8, 9, 24, 25], "overview": [10, 24], "own": [0, 10, 11, 24, 25], "packag": [19, 24], "panda": [24, 25], "paramet": [24, 25], "part": [13, 18], "partial": 2, "pass": 1, "pca": 11, "pdf": 21, "perceptron": 12, "perform": [1, 9], "period": 3, "perspect": 1, "plan": 25, "plethora": 24, "point": 4, "poisson": 2, "polynomi": [3, 16], "popul": 2, "popular": 24, "practic": [13, 22, 24], "pre": [1, 3], "predict": 4, "preprocess": 25, "prerequisit": [3, 18, 24], "princip": 11, "principl": 3, "pro": 9, "probabl": [5, 21], "problem": [1, 2, 13, 24, 25], "procedur": [9, 24], "process": [1, 3], "program": [2, 13], "project": [6, 22, 24], "prop": 13, "propag": [1, 12], "properti": [5, 21, 25], "python": [0, 9, 15, 18, 19, 24], "quick": 8, "r": 24, "random": [10, 11, 21], "read": [9, 24, 25], "real": [6, 24], "recommend": [24, 25], "recurr": [4, 12], "reduc": [0, 25], "reduct": 3, "reformul": 2, "regress": [0, 5, 6, 7, 9, 10, 13, 15, 17, 24, 25], "regular": 1, "relat": [], "relev": [23, 25], "relu": 1, "remark": 3, "remind": [6, 8, 24, 25], "replac": 13, "repositori": 15, "requir": [2, 18], "resampl": 6, "rescal": 6, "residu": 25, "resourc": 2, "result": 25, "revisit": 13, "rewrit": [24, 25], "ridg": [0, 5, 6, 17, 25], "rm": 13, "rule": 12, "same": 13, "sampl": 11, "scale": [17, 25], "schedul": 24, "schemat": 9, "scheme": 2, "scienc": 24, "scikit": [0, 1, 11, 24, 25], "second": 13, "semest": 22, "set": [0, 2, 3, 9, 12, 15, 20, 24, 25], "setup": 15, "sgd": 13, "should": 1, "similar": 13, "simpl": [0, 4, 9, 13, 24, 25], "singl": 10, "singular": [5, 11, 25], "size": 25, "sklearn": 16, "soft": 8, "softmax": 1, "softwar": 24, "solv": 2, "solver": 13, "some": [13, 19, 25], "specifi": 2, "split": [0, 15, 25], "squar": [0, 5, 6, 10, 16, 24, 25], "standard": [13, 25], "state": 0, "statist": [5, 6, 18, 21, 24], "steepest": [10, 13], "stochast": [13, 21], "strongli": 24, "suggest": 24, "summari": [22, 24], "superposit": 3, "supervis": 1, "support": 8, "svd": [5, 25], "systemat": 3, "t": 25, "take": 16, "taken": 24, "teach": 22, "teacher": [22, 24], "techniqu": [6, 11], "technologi": 18, "tensorflow": [1, 3], "tent": [22, 24], "test": [0, 1, 15, 17, 25], "text": 24, "textbook": [23, 24], "theorem": [5, 8, 11, 12, 21], "theori": 21, "thi": 24, "tip": 13, "togeth": 12, "tool": 24, "top": 1, "topic": 24, "toward": 11, "trade": 6, "tradeoff": 6, "train": [0, 1, 4, 15, 24, 25], "transform": 3, "tree": [9, 10], "tune": 1, "two": [3, 8, 18], "type": [2, 4, 12, 24], "uio": 24, "univers": [12, 23], "unsupervis": 14, "up": [0, 2, 9, 12, 15, 24, 25], "us": [0, 1, 2, 3, 7, 13, 16, 18, 24, 25], "v": 3, "valid": 6, "valu": [5, 11, 21, 25], "variabl": 21, "varianc": 6, "variou": 0, "vector": [8, 12, 16, 19, 24, 25], "versu": 24, "view": [0, 4, 10, 25], "virtual": 15, "visual": [1, 9], "wai": 9, "wave": 2, "we": 24, "week": [15, 16, 24, 25], "weekli": 17, "what": [0, 24, 25], "which": 1, "why": 24, "wisconsin": 7, "write": [4, 11], "x": 25, "xgboost": 10, "your": [0, 10, 16, 25]}}) \ No newline at end of file diff --git a/doc/LectureNotes/_build/html/statistics.html b/doc/LectureNotes/_build/html/statistics.html index aba8d0cc7..015066dc6 100644 --- a/doc/LectureNotes/_build/html/statistics.html +++ b/doc/LectureNotes/_build/html/statistics.html @@ -223,10 +223,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/teachers.html b/doc/LectureNotes/_build/html/teachers.html index de539ac85..615800cd9 100644 --- a/doc/LectureNotes/_build/html/teachers.html +++ b/doc/LectureNotes/_build/html/teachers.html @@ -221,10 +221,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/textbooks.html b/doc/LectureNotes/_build/html/textbooks.html index bef340138..91034731e 100644 --- a/doc/LectureNotes/_build/html/textbooks.html +++ b/doc/LectureNotes/_build/html/textbooks.html @@ -221,10 +221,11 @@

Weekly material, notes and exercises

diff --git a/doc/LectureNotes/_build/html/week34.html b/doc/LectureNotes/_build/html/week34.html index 520575b8b..b1ee5fc16 100644 --- a/doc/LectureNotes/_build/html/week34.html +++ b/doc/LectureNotes/_build/html/week34.html @@ -62,8 +62,8 @@ - - + + @@ -223,10 +223,11 @@

Weekly material, notes and exercises

@@ -2517,7 +2518,7 @@ Singular Value Decomposition (SVD) method discussed next week.

@@ -2526,7 +2527,7 @@ Singular Value Decomposition (SVD) method discussed next week.

next

diff --git a/doc/LectureNotes/_build/html/week35.html b/doc/LectureNotes/_build/html/week35.html index 0c1161977..353391878 100644 --- a/doc/LectureNotes/_build/html/week35.html +++ b/doc/LectureNotes/_build/html/week35.html @@ -62,7 +62,8 @@ - + + @@ -222,10 +223,11 @@

Weekly material, notes and exercises
@@ -2385,7 +2387,7 @@ C(\boldsymbol{X},\boldsymbol{\theta})=\frac{1}{n}\left\{(\boldsymbol{y}-\boldsym
@@ -2393,6 +2395,15 @@ C(\boldsymbol{X},\boldsymbol{\theta})=\frac{1}{n}\left\{(\boldsymbol{y}-\boldsym

Exercises week 35

 + +
+

next

+

Weekly Exercises 3

+
+ +
diff --git a/doc/LectureNotes/_build/jupyter_execute/cb756715680fbe8cbf1d070371bf3950695ddf24b29ecb476c48fb9e559af08e.png b/doc/LectureNotes/_build/jupyter_execute/cb756715680fbe8cbf1d070371bf3950695ddf24b29ecb476c48fb9e559af08e.png new file mode 100644 index 000000000..a16a30cd2 Binary files /dev/null and b/doc/LectureNotes/_build/jupyter_execute/cb756715680fbe8cbf1d070371bf3950695ddf24b29ecb476c48fb9e559af08e.png differ diff --git a/doc/LectureNotes/_build/jupyter_execute/exercisesweek34.ipynb b/doc/LectureNotes/_build/jupyter_execute/exercisesweek34.ipynb new file mode 100644 index 000000000..851509f7b --- /dev/null +++ b/doc/LectureNotes/_build/jupyter_execute/exercisesweek34.ipynb @@ -0,0 +1,315 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "232d1306", + "metadata": {}, + "source": [ + "# Exercises week 34\n", + "\n", + "## Coding Setup and Linear Regression" + ] + }, + { + "cell_type": "markdown", + "id": "9b66a351", + "metadata": {}, + "source": [ + "Welcome to FYS-STK3155/4155!\n", + "\n", + "In this first week will focus on getting you set up with the programs you are going to be using throughout this course. We expect that many of you will encounter some trouble with setting these programs up, as they can be extremely finnicky and prone to not working the same on all machines, so we strongly encourage you to not get discouraged, and to show up to the group-sessions where we can help you along. The group sessions are also the best place to find group partners for the projects and to be challenged on your understanding of the material, which are both essential to doing well in this course. We strongly encourage you to form groups of 2-3 participants. \n", + "\n", + "If you are unable to complete this week's exercises, don't worry, this will likely be the most frustrating week for many of you. You have time to get back on track next week, especially if you come to the group-sessions! Note also that this week's set of exercises does not count for the additional score. The deadline for the weekly exercises is set to Fridays, at midnight." + ] + }, + { + "cell_type": "markdown", + "id": "36d8750b", + "metadata": {}, + "source": [ + "### Learning goals\n", + "\n", + "After completing these exercises, you will know how to\n", + "\n", + "- Create and use a Github repository\n", + "- Set up and use a virtual environment in Python\n", + "- Fit an OLS model to data using scikit-learn\n", + "- Fit a model on training data and evaluate it on test data\n", + "\n", + "### Deliverables\n", + "\n", + "Complete the following exercises while working in a jupyter notebook. Exercises 1,2 and 3 require no writing in the notebook. Then, in canvas, include\n", + "- The jupyter notebook with the exercises completed\n", + "- An exported PDF of the notebook (https://code.visualstudio.com/docs/datascience/jupyter-notebooks#_export-your-jupyter-notebook)\n", + "- Optional: A link to your github repository, which must be set to public, include the notebook file, a README file, requirements file and gitignore file.\n", + "\n", + "We require you to deliver a jupyter notebook so that we can evaluate the results of your code without needing to download and run the code of every student, as well as to teach you to use this useful tool." + ] + }, + { + "cell_type": "markdown", + "id": "2a9c7ef8", + "metadata": {}, + "source": [ + "## Exercise 1 - Github Setup\n" + ] + }, + { + "cell_type": "markdown", + "id": "1498aed1", + "metadata": {}, + "source": [ + "In this course, we require you to pay extra mind to the reproducibility of your results and the shareability of your code. The first step toward these goals is using a version control system like git and online repository like Github.\n", + "\n", + "**a)** Download git if you don't already have it on your machine, check with the terminal command ´git --version´ (https://git-scm.com/downloads).\n", + "\n", + "**b)** Create a Github account(https://github.com/), or log in to github with your UiO account (https://github.uio.no/login).\n", + "\n", + "**c)** Learn the basics of opening the terminal and navigating folders on your operating system. Things to learn: Opening a terminal, opening a terminal in a specific folder, listing the contents of the current folder, navigating into a folder, navigating out of a folder.\n", + "\n", + "**d)** Download the Github CLI tool and run ´gh auth login´ in your terminal to authenticate your local machine for some of the later steps. (https://github.com/cli/cli#installation). You might need to change file permissions to make it work, ask us or ChatGPT for help with these issues.\n", + "\n", + "**e)** As an alternative to the above terminal based instructions, you could install GitHub Desktop (see https://desktop.github.com/download/) or if you prefer GitLab, GitLab desktop (see https://about.gitlab.com/install/). This sets up all communications between your PC/Laptop and the repository. This allows you to combine exercises 1 and 2 in an easy way if you don't want to use terminarl. Keep in mind that these GUIs (graphical user interfaces) are not text editors." + ] + }, + { + "cell_type": "markdown", + "id": "c56fbefa", + "metadata": {}, + "source": [ + "## Exercise 2 - Setting up a Github repository\n" + ] + }, + { + "cell_type": "markdown", + "id": "fb9b8acd", + "metadata": { + "vscode": { + "languageId": "plaintext" + } + }, + "source": [ + "**a)** Create an empty repository for your coursework in this course in your browser at github.com (or uio github).\n", + "\n", + "**b)** Open a terminal in the location you want to create your local folder for this repository, like your desktop.\n", + "\n", + "**c)** Clone the repository to your laptop using the terminal command ´gh repo clone username/repository-name´. This creates a folder with the same name as the repository. Moving it or renaming it might require some extra steps.\n", + "\n", + "**d)** Download this jupyter notebook. Add the notebook to the local folder.\n", + "\n", + "**e)** Run the ´git add .´ command command in a terminal opened in the local folder to stage the current changes in the folder to be commited to the version control history. Run ´git status´ to see the staged files.\n", + "\n", + "**f)** Run the ´git commit -m \"Adding first weekly assignment file\"´ command to commit the staged changes to the version control history. Run ´git status´ to see that no files are staged.\n", + "\n", + "**g)** Run the ´git push\" command to upload the commited changes to the remote repository on Github.\n", + "\n", + "**h)** Add a file called README.txt to the repository at Github.com. Don't do this in your local folder. Add a suitable title for your repository and some inforomation to the file.\n", + "\n", + "**i)** Run the ´git fetch origin´ command to fetch the latest remote changes to your repository.\n", + "\n", + "**j)** Run the ´git pull´ command to download and update files to match the remote changes.\n" + ] + }, + { + "cell_type": "markdown", + "id": "f84d0db6", + "metadata": {}, + "source": [ + "## Exercise 3 - Setting up a Python virtual environment\n" + ] + }, + { + "cell_type": "markdown", + "id": "b5a4818a", + "metadata": {}, + "source": [ + "Following the themes from the previous exercises, another way of improving the reproducibility of your results and shareability of your code is having a good handle on which python packages you are using.\n", + "\n", + "There are many ways to manage your packages in Python, and you are free to use any approach you want, but in this course we encourage you to use something called a virtual environment. A virtual environemnt is a folder in your project which contains a Python runtime executable as well as all the packages you are using in the current project. In this way, each of your projects has its required set of packages installed in the same folder, so that if anything goes wrong while managing your packages it only affects the one project, and if multiple projects require different versions of the same package, you don't need to worry about messing up old projects. Also, it's easy to just delete the folder and start over if anything goes wrong.\n", + "\n", + "Virtual environments are typically created, activated, managed and updated using terminal commands, but for now we recommend that you let for example VS Code (a popular cross-paltform package) handle it for you to make the coding experience much easier. If you are familiar with another approach for virtual environments that works for you, feel free to keep doing it that way.\n" + ] + }, + { + "cell_type": "markdown", + "id": "0f6de364", + "metadata": {}, + "source": [ + "**a)** Open this notebook in VS Code (https://code.visualstudio.com/Download). Download the Python and Jupyter extensions.\n", + "\n", + "**b)** Press ´Cmd + Shift + P´, then search and run ´Python: Create Environment...´\n", + "\n", + "**c)** Select ´Venv´\n", + "\n", + "**d)** Choose the most up-to-date version of Python your have installed.\n", + "\n", + "**e)** Press ´Cmd + Shift + P´, then search and run ´Python: Select Interpreter´\n", + "\n", + "**f)** Selevet the (.venv) option you just created.\n", + "\n", + "**g)** Open a terminal in VS Code, the venv name should be visible at the beginning of the line. Run `pip list` to see that there are no packages install in the environment.\n", + "\n", + "**h)** In this terminal, run `pip install matplotlib numpy scikit-learn`. This will install the listed packages.\n", + "\n", + "**i)** To make these installations reproducible, which is important for reproducing results and sharing your code, run ´pip freeze > requirements.txt´ to create the file requirements.txt with all your dependencies.\n", + "\n", + "Now, anyone who wants to recreate your package setup can download your requirements.txt file and run ´pip install -r requirements.txt´ to install the correct packages and versions. To keep the requirements.txt file up to date with your environment, you will need to re-run the freeze command whenever you install a new package.\n", + "\n", + "**j)** Create a .gitignore file at the root of your project folder, and add the line ´.venv´ to it. This way, you won't try to upload a copy of all your python packages when you regularly push your changes to Github. Ignored files should not show up when you run ´git status´, and are not staged when running ´git add .´, try it!" + ] + }, + { + "cell_type": "markdown", + "id": "5d184ab1", + "metadata": {}, + "source": [ + "## Exercise 3 - Fitting an OLS model to data\n" + ] + }, + { + "cell_type": "markdown", + "id": "d19ebd67", + "metadata": {}, + "source": [ + "Great job on getting through all of that! Now it is time to do some actual machine learning!\n", + "\n", + "**a)** Complete the code below so that you fit a second order polynomial to the data. You will need to look up some scikit-learn documentation online (look at the imported functions for hints).\n", + "\n", + "**b)** Compute the mean square error for the line model and for the second degree polynomial model." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "b58fb9bf", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "from sklearn.preprocessing import PolynomialFeatures # use the fit_transform method of the created object!\n", + "from sklearn.linear_model import LinearRegression\n", + "from sklearn.metrics import mean_squared_error" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "0208e9ca", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhYAAAGdCAYAAABO2DpVAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjUsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvWftoOwAAAAlwSFlzAAAPYQAAD2EBqD+naQAASZ5JREFUeJzt3QuczPX++PH37GItdpclLVIuiSSV08U1KUVJOnXSr4vLOV1Jkn+FOn6ojqXboZtup1Kd8lMhKioJSeIkHZduWJdEInZdF7vzf7y/s7PNzs7l+535zv31fDzmrJ35zsx3J8f3vZ/P++JwOp1OAQAAsEGaHS8CAACgCCwAAIBtCCwAAIBtCCwAAIBtCCwAAIBtCCwAAIBtCCwAAIBtCCwAAIBtqkiUlZaWyi+//CJZWVnicDii/fYAACAE2k9z37590rBhQ0lLS4ufwEKDisaNG0f7bQEAgA22bt0qJ5xwgj2BRZMmTWTz5s2V7h88eLA888wzpl5DVyrcJ5adnW3l7QEAQIwUFRUZCwPu67gtgcWKFSukpKSk/Ps1a9bIxRdfLNdcc43p13Bvf2hQQWABAEBiCZbGYCmwOO644yp8P2HCBGnevLl07do1tLMDAABJJeQciyNHjsgbb7whw4cPDxi9FBcXGzfPpRQAAJCcQi43nTVrluzdu1cGDhwY8Lj8/HzJyckpv5G4CQBA8nI4tX4kBD169JBq1arJnDlzAh7na8VCg4vCwkK/ORaax3H06NFQTgsJJj09XapUqULpMQDEOb1+6wJBoOt3yFshWhkyf/58mTFjRtBjMzIyjJtZ+/fvl59//tmol0VqqFGjhjRo0MAIVAEAiS2kwOKVV16R+vXrS69evWw9GV2p0KBCLzSaKMpvsclNg0fN1fntt9+koKBAWrRoEbDpCgAgCQML7ZypgcWAAQOMJWw76faHXmw0qMjMzLT1tRGf9L9z1apVjVUwDTKqV68e61MCAITB8q+HugWyZcsW+dvf/iaRwkpFamGVAgCSh+Ulh0suuYT8BwAA4kxJqVOWF/wuO/cdlvpZ1eXcprmSnhb9X9SjPisEAADYG0x8sm6HzFr1i/x+4Ej5Yw1yqsuY3q2lZ5sGEk0EFgAAJKB5a7bLuDnrZHvhYZ+P7yg8LIPeWClTbmwX1eCCzW0baJMwzQvRmyYiHn/88cYMlZdfftlIdjXr1Vdfldq1a0f0XAEAyRFUDHpjpd+gQrmTFjT40JWNaEnKwEI/wC837Jb3Vm0zvkbjA+3Zs6ds375dNm3aJHPnzpVu3brJXXfdJZdffrkcO3Ys4u8PAEhuJWXXtpnfbJP7Z64uDxwC0WM0+NDtkmipkgpLQ9HYZ9ImYHl5ecafGzVqJO3atZP27dvLRRddZKxE3HzzzfLEE08YpbobN26U3Nxc6d27tzzyyCNSq1YtWbhwofz1r3+tUBUzZswYGTt2rLz++usyefJk+eGHH6RmzZpy4YUXyqRJk4xeIgCA5DcvyLZHMJrQGS1pqbA05N5n0sejSQOAM844o7xDqZZVPvnkk7J27VqZOnWqLFiwQO677z7jsY4dOxrBgrZJ1ZUPvd1zzz3l/T0eeugh+fbbb40ZLboqEmxGCwAgdbY9gtEqkWipkkxLRBrN+Voa0vt0DUAfv7h1XlTLb1q1aiX//e9/jT8PGzas/P4mTZrIww8/LLfffrs8++yzRjtr7cGuqxXulQ83z54hzZo1M4KTc845x2h/rqsdAIDkVBLg2maGXu3yclylp9GSNCsWun8ULIkl2vtMxvs6neVbG9pcTLdGdKskKytL+vXrJ7t375aDBw8GfI2vv/7a2DY58cQTjed17drVuF8blQEAktfyINe2QNy/QmsqQDR/oU6awMLs/lE095nUd999J02bNjW2LzSRs23btvLuu+8awcIzzzxjHKOtrP05cOCAMUlWt0j+/e9/y4oVK2TmzJlBnwcASHw7w7hm6UpFtEtNk2orxOz+UTT3mTSHYvXq1XL33XcbgYSWnj7++OPlLaynT59e4XjdDtFBbJ6+//57Y1VjwoQJxrh59Z///CdqPwMAIHbq1TI/HdzTkG7N5e6LW8ak82bSrFjo/pFWf/j7CPX+BhHcZyouLpYdO3bItm3bZOXKlTJ+/Hjp06ePsUrRv39/Ofnkk40kzKeeesqoCtFKj+eee67Ca2jeheZNfPrpp7Jr1y5ji0S3PzTgcD9v9uzZRiInACAFOEN7WqeTj4tJUJFUgYV+gLqPpBwx2GeaN2+eNGjQwAgOtKfFZ599ZiRZvvfee5Kenm5Uh2i56cSJE6VNmzbGtkZ+fn6F19DKEE3mvPbaa40Jr1qKql+1XPXtt9+W1q1bGysXjz32WER+BgBAfNl1oNjS8ZH+JdrUOTijPFGsqKjIqH4oLCw08gY8HT58WAoKCoychFDHZ8eqjwVCZ8d/dwBIRl9u2C3XvbjM1LHuX5sjlVcR6PqdlDkWbvphaklpPEx4AwDAjm1+7cfkNJGsGQ+/RCddYKE0iOjQvG6sTwMAgLD9zzknyj/n/1jpfv11WYONv3VqYvxCHS+/RCdlYAEAQLK38c6LkxUKbwQWAADEaRtvp5/H7+7eQoZc2CIuViiStioEAIBUaOPtEJFpK7ZKvCKwAAAgjiyP0xEVZhFYAAAQR3bG6YgKswgsAACII/XjcESFFSRvAgAQ5RyK5QF6LQXrXRGLUehWsGIRBTo2fdasWZIsxo4dK2eeeabp43Wyq34Gq1atiuh5AUAiVHt0nrjA6KZ517RVxlf9Xu/37l3hL3lT77/ijAZxWRGSvCsWpSUim5eK7P9VpNbxIid1FElLj9jbDRw4UPbu3es3eNi+fbvUqVMnYu8PAIh//kpIdWVC79dW3CpQ7wq3FxYXyFkn1om7HhbJGVismy0yb4RI0S9/3JfdUKTnRJHWV8TklPLy8mLyvgCA+C8hdZZtb4yasVr2HDxq+jX19bTjZrytXKQlXVAxvX/FoEIVbXfdr4/HeCvEvS0wY8YM6datm9SoUcOYfPrll19WeM6SJUukS5cukpmZKY0bN5ahQ4fKgQMHgm5PvPzyy8ao9Vq1asngwYOlpKTEmJKqwU39+vXlH//4R4XnbdmyxRjvrsfrUJm+ffvKr7/+WuEYnah6/PHHS1ZWltx0003G0DBvL730kpx66qnGELFWrVrJs88+G+anBgCpVUK6x0JQEc8lp2lJtf2hKxV+40FdhxrpOi4OPPDAA3LPPfcYeQennHKKXHfddXLs2DHjsQ0bNhij16+++mr573//K//3f/9nBBpDhgwJ+Jr6vLlz5xoj3N966y3517/+Jb169ZKff/5ZFi1aZIxs//vf/y5fffWVcXxpaakRVPz+++/G45988ols3LjRGNvuNn36dCNoGT9+vPznP/8xRsN7Bw06Av5///d/jaDlu+++M44dPXq0TJ06NSKfHQAkmp0RKg2Nx5LT5NkK0ZwK75WKCpwiRdtcxzXtIrGmQYVe9NW4cePktNNOk/Xr1xu/7efn58sNN9wgw4YNMx5v0aKFPPnkk9K1a1eZMmWK39HiGijoioWuLLRu3dpYEfnhhx/kww8/lLS0NGnZsqURXHz22Wdy3nnnyaeffiqrV682Rpbrqoh67bXXjHNZsWKFnHPOOTJp0iRjlUJv6uGHH5b58+dXWLUYM2aMPP7443LVVVcZ3+v483Xr1snzzz8vAwYMiPhnCQDxrn6ESkPjseQ0eVYsNFHTzuMirG3btuV/1lUAtXPnTuPrt99+K6+++qqxPeG+9ejRwwgcNAjwp0mTJkZQ4abbFxpgaFDheZ/7fXR1QQMKd1Ch9PjatWsbj7mP0SDEU4cOHcr/rNszulKigYfn+WoAovcDAET+dFIdsTMVQl+qQZyWnCbPioVWf9h5XIRVrVq1/M+ac6E0cFD79++X2267zcir8Kb5E2Ze0/26vu5zv48d9FzViy++WCkASU+PXCUOACSSrzfvkVJ/9aMWueMTnWwab4mbyRVYaEmpVn9ooqa/liL6uB4X59q1a2dsJZx88skRfR9Ntty6datxc69a6Ptq6ayuXLiP0ZyM/v37lz9v2bJlFVZAGjZsaORm6PYNACC8XAgNFQLFIPE6Lj35AgvtU6ElpVr9Uek/S1lE13NCxPpZFBYWVmoAVbdu3QrbDGaNGDFC2rdvbyRr3nzzzVKzZk3jgq/JlU8//bRt59y9e3c5/fTTjYBAcyk0eVQrSTSX4+yzzzaOueuuu4w+Hfp9p06djETNtWvXSrNmzcpfR3NEdHUlJyfHSDotLi42Ej337Nkjw4cPt+18ASBR1TeZC6Hj0HVyqWcFSV52hlx37onSpF5Nn506403yBBZK+1T0fc1PH4sJEe1jsXDhQjnrrLMq3Kd5B1qGGUr+hVZpaOWIlpw6nU5p3rx5hWoNO+i2yHvvvSd33nmnnH/++UYuhgYGTz31VPkx+p6aK3HfffcZCZtaqTJo0CD56KOPyo/R4EfLZh999FG59957jUBIAxZ38ikApLpzTbbpHnJhC+MWqOV3vHM49aoVRUVFRcZvtvobvvZN8KQXLk1O1KoCf5UP8dh5E+Gx7b87ACRA503xvaZudN6M1+2NYNfv5F2xcNMgIg5KSgEAqafEz5AxDRo0ePBu2R3vORNWJWdgAQBAjFYlxnkFDg08Age9aRvuRN7qCIbAAgCAKA0Z69nGNZW0Q/O69p9AnKQBEFgAABCFIWPjIjk0LI4GcMZl580o55MixvjvDSAVhoxtj8TQMF2lWKitFvrFzQDOuAos3J0ajxw5EutTQRQdPHjQ+OrdJRQAkq0B1k47h4atnSXyaAuRheP9HBCbAZxxtRVSpUoVox/Cb7/9ZlxkPGdcIDlXKjSo0NklOp+EFuAAEqXCw/vxn37dF92hYR+PFln6pIkDoz+AM64CC23YpAO5tKfB5s2bY306iBINKvLy8mJ9GgBgqcLD1+P+uBtg2TI0bM0sk0FFbAZwxlVgoapVq2aMCWc7JDXoyhQrFQASrcLj1vObyguLCwLO9LB9aNixIyLLXxD59EHrz43iAM64CyyUboHQgREAEI8VHurFz80FFbY0wNL8iHdvFlk7M8h4MomLAZxxGVgAABCvFR7KzAj0Id1Olk4n1wuvAZZue8y6TeRYGEmfERzA6QuBBQAAEajcaHF8rfAaYZlO0PQju1HEB3D6QmABAEAEKjfqh/o6uvWx6JHwgoquI0W63kfnTQAAYu1PJ9WR3JrV5PcDoRcRpDlE9oTyfG1mNfc+kX3bQ35v6ThUpNsoiRUaRQAA4FEN0vXRz8IKKtw5GHe8udJ4PUtBhXbKDDWoqFFP5C9TRS55SGLJcmCxbds2ufHGG6Vu3bqSmZkpp59+uvznP/+JzNkBABDlElMzfSnMGjdnnVFlYmr7Q2d9WK76UA6RG2aI3POjSJsrJdYsbYXs2bNHOnXqJN26dZO5c+fKcccdJz/99JPUqVMncmcIAEAMS0xD5fSYDxI0iVM7Y3rP+jCr450iLS6SeGEpsJg4caI0btxYXnnllfL7mjZtGvA5xcXFxs2tqKgolPMEACCmJaa2VZkcOyKy4kWRPZtE6jQROeeWEDtjOlxBRYy3PsIKLGbPni09evSQa665RhYtWiSNGjWSwYMHyy233OL3Ofn5+TJu3Dg7zhUAgIiwdThYoOoQLSH98mkRZ6nHfX8XaW1xC6NJF5EbZ4hUqSbxxlKOxcaNG2XKlClGy+2PPvpIBg0aJEOHDpWpU6f6fc6oUaOksLCw/LZ161Y7zhsAANuEWhqaVb1Kectub46y2SLl80HcfSk8gwql36+dIVKtpkcD8AAJmtdMFRn4flwGFZZXLEpLS+Xss8+W8eNdI1rPOussWbNmjTz33HMyYMAAn8/JyMgwbgAAxCu9+GsQoLNArORZXPOnE+SVLzYZ4YAz0HyQY0dcKxWBHDlY9irer1bmgvtFzr8nJr0pIrZioZNHW7duXeG+U089VbZs2WL3eQEAEDV68dcgQFlpvn1x6zyZcmM7Yx6Ip7yc6sb95fNBNKfCe6WiEqfIGTeIZDeo3EGz7+siF4yI+6DC8oqFVoT88MMPFe778ccf5aSTTrL7vAAAiGpVSE5mNflbpyYyc9U2+f3AUdNj0DUo0QBDE0B37jtsbKtUmg+iiZpmZNQUGbbGVSWiCZ06lVQHiCVAQBFSYHH33XdLx44dja2Qvn37yvLly+WFF14wbgAAJGr/Ci019awK0c6bZzbOkQXf/xZ8m6NsxaO8pLS0RGTzkoqBgVZ/mKHHaRDRtIskKofT6bRUtvv+++8bCZnav0JLTYcPHx6wKsSblpvm5OQYiZzZ2dmhnDMAALY2xfK+ELqDh1vPbyqzv91eIehoEGgM+rrZrkZXnj0pdGz5Jf8QefemwNshjnSRB3bEbVKm2eu35cAiWicGAECktz86T1zgt3+Fe7tj0b3d5OvNe/xvc7hXKRY/JrJwvJ9XEpGWl4r88GHgGR9x1pMilOs3Q8gAACkpWFMsd+dMDSr8ds50BxTLnhU5vDfAKzlEtn8r0mGI61jPlQtdqehwR1wHFVYQWAAAUpLZplh+j9NtjzlDRQ7tMfEqTpGibSKn9BS5aEzlzptxuv0RCgILAEBKMtsUq9JxAbc9gtCETg0idIUiSRFYAABSkpmmWHVrVpM/nVSn4irF3PtCH21e63hJdpbHpgMAkCpNsXYfOCLdHpkvyxfMEpk3SmR6vxCDCoer0ZWWniY5AgsAQMrSklFfnTPdLk37SmYX/03OXTzAlXQZ1ptNSKhGV6FiKwQAIKkeXBw75pQ7p31TYUtkZPqbcluV98Vhpce3L7pSoUFF6yskFRBYAAAk1ZtkDZn2Tfn3VeSY5Fd5Uf6S/nn4L35BYgwOsxOBBQAgpZtkaTtvlSalMqnK03J5+jLx7n9lWWauSO/JKbNK4YnAAgAgqd4kq2faMvln1SmS6Qg8fCyozDoi5w1KuVUKTwQWAICUpc2vbMulSMFtD18ILAAAKavV7gVyRZX3w3uRFEvODIbAAgCQmkpL5JSvx4S+UqFbHq16uXpTpPgqhScCCwBAatq8VBwHd4f23DifRBpLBBYAgNSkczusqlFP5LLHRdpcGYkzSgoEFgCA1GR1bsdpV4lc/RLbHkEQWAAAkqovhZaQarWHTiXVQWM6E8QnzY3IbihS9EuQV3WIdLyTrQ+TCCwAAEnTQVObXWlfCjedXqqDxrRtdyW68tBzosj0/iL+5ps26SJy4wzXqHOYwhAyAEBSBBWD3lgpvxYelPZp6+SKtKXG152FB4379XGftES072uulQvvXIprpooMfJ+gwiJWLAAASdGW+5K05TKm6mvS0PF7+WO/OHPlwaP9Zdyc6nJx6zzf2yIaXGjZ6OalroROzb2ghDRkBBYAgISmORVt9y2WKVUnVXosT36XZ6tOkkH79LgzpUPzur5fRIOIpl0if7IpgK0QAEBC21l0wFipUN4LEu7vx1R93TgOkUdgAQBIaCcfXG1sf/gr/tD7Gzp2G8ch8ggsAAAJ7dSsg7Yeh/AQWAAAElpaVp6txyE8BBYAgMRW1ujKqY2sfDDu1wmkehwijsACAJDYyhpdaVjhHVzo98Y9Otac8tGoILAAACS+skZXjuyKHTYd2vhKG2Dp44gK+lgAAJIDja7iAoEFACB+lJaEFxjQ6CrmCCwAAPFh3WyReSMqThvVrQwdFMZWRsIgxwIAEB9BhU4Z9R5hXrTddb8+joRAYAEAiP32h65U+BxdXnbfvJGu4xD3CCwAALGlORXeKxUVOEWKtrmOQ9wjsAAAxJYmatp5HGKKwAIAEFta/WHncYgpqkIAADEtIS1p3EF2SV05zrnb54TSUqfITkddOa5xB6EjRfwjsAAAxLSEdPnmQnn1SD+ZUnWSEUR4Bhf6vRpzpJ8M3FwoHZrXjfZPAIvYCgEAxLSEdOe+w/JR6bky6Ogw2SG5FQ7dIXWN+/VxPQ7xjxULAEAMSkgdrhLSVr2kflZ1414NHj4pPlvOTfte6ste2Sm1ZXlpKykt+x3YfRziG4EFACCmJaTnNu0sDXKqy47Cw0YQsay0dYUjdWckL6e6nNu04moG4hNbIQCAmJaQpqc5ZExvVzDhnbvp/l4f1+MQ/wgsAAAxLyHt2aaBTLmxnbEy4Um/1/v1cSQGtkIAAPbTklKt/tBETZ95Fg7X43pcGQ0eLm6dJ8sLfjcSNTWnQrc/WKlILAQWAAD7aZ8KLSnV6g9jQ8MzuCgLFHpOqDQSXYMISkoTG1shAIDI0D4VfV8TZ3bFbQynrlT0fY1R6EmKFQsAQMTMKz1HHjo8WRof+ba8hHTr4TNkdOnp0jPWJ4eIILAAAETEvDXbZdAbK41NkG3yRwmpo+iocT9JmcmJrRAAgO1KSp0ybs46v+2xlD6uxyGFA4uxY8eKw+GocGvVqlXkzg4AkJC0smN7of8W3BpO6ON6HFJ8K+S0006T+fPn//ECVdhNAYBUmURqltm5Hsz/SD6WowINJPLy8kwfX1xcbNzcioqKrL4lACBOJpGaZXauB/M/ko/lHIuffvpJGjZsKM2aNZMbbrhBtmzZEvD4/Px8ycnJKb81btw4nPMFAMRwEqlZ2thK53/4a22l9+vjzP9IPg6n02k6c2bu3Lmyf/9+admypWzfvl3GjRsn27ZtkzVr1khWVpbpFQsNLgoLCyU7O9uenwIAEPrWx6YlIm/3Fzm0189BZV0yh622tC3irgoR3+2xqApJMHr91gWCYNdvS4GFt71798pJJ50kTzzxhNx00022nhgAIMIBxeLHRL56NkBA4WXA+yJNu1h6Gw0utPrDM5FTVyp0qBhBRWIxe/0OK/Oydu3acsopp8j69evDeRkAQDStnSUye4hI8b7ITCz1wPyP1BNWYKHbIhs2bJB+/frZd0YAgMj5eLTI0icjO7HUC/M/Uoul5M177rlHFi1aJJs2bZKlS5fKn//8Z0lPT5frrrsucmcIALDHmlkhBhWaY9GowiRSwJYVi59//tkIInbv3i3HHXecdO7cWZYtW2b8GQAQ5zkVHw4P4Yn+J5ECYQcW06ZNs3I4ACBeaNOrg7stP00nkTo0qGASKUyibSYAJNvKRMHnIpuXuGo8tYqjSWdLiZc6vqNQasngo0Nly+GzmEQKSwgsACBZaBOrOUNFDu35477PHxXJzBU573ZTL+FuQDDy6M3yZWkbJpHCMqabAkAyrFIsnCgyvV/FoMLt0O8iC8eLZNYJ+lJ7pJYMOjpMPio91/ieSaSwihULAEj0VYoP7xXZv8PEwY6AKxWzS9rL3ceGSKnX75yek0gpG0UwrFgAQELP9+hnMqgoW7m44H45lFlxkOQuZ7YMOjpU7jo2tFJQ4YlJpDCDFQsASNTtD82nsOjbg7ly1Z7H5Jy076W+7JWdUluWl7YKGFC4MYkUZhBYAEAi0soPX/kUQTz/zUEpkTRZVtra9HN0AyWPSaQwia0QAEhEWk5qUXGNBjJvXzNLz3FnZejQMOZ7wAwCCwBIRCEUaHzbZqSpLQ9PulJBqSmsILAAgERkZXy59rHo+7qUtOxt6S1G9zpVloy4kKAClpBjAQCJSLtpasCglR7+pFcXuW6aSLPzjTkf55Y6pUFOdaN01Ix6WRlsf8AyViwAIB5bcq9+x/VVv/dFB4L1nhz4ta5+UeTkbuXDwzRIuOIM86sPVIEgFKxYAEA89aWYN0Kk6Jc/7stuKNJzou8hYHpf39crPUeTNDWfoiSjk7FK4V51mLdmu7ywuCDoaVAFgnA4nE53Z/joKCoqkpycHCksLJTs7OxovjUAxHmzq/4+sjLLtiL6vuZ/wqiuamxeKt9+971RTqqVH+4kTd360IqOi1vnSeeJC4Jug7g3PkjYRKjXb7ZCACDWNDDQVQefpR5l980b6XdbRPtSTN6QJ30WN5QP951cofJDA4nb31gpTy9Ybyq3IrdmNYIKhIWtEACItc1LK25/VOIUKdrmOs6rGkS3N8bOXis7iooDvsXzizeYOpW/9zqVoAJhIbAAgFjb/2tIx2lQoSPNzexnHzziJwnUS15OprlzAfxgKwQAYq3W8ZaP0xHmOsrcSpKcI8hjmo9BwibCRWABALF2UkdX9YffS79DJLuR67gyOsLcbD8KN3cQ4v0utO2GnQgsACDWtM+ElpQGuuz3nFDejyKcEeY3dWpilJJ6om037ESOBQBEglZwbFoiUrBIZO/PIrVPEGlyviv50iNAqNiT4jU/fSwmVCo1DbV5VffWeXJ/r9bGiocGJ/o6uv3BSgXsQmABAHZbO0tk9hCR4n0V7//8cZHMOiK9n/Tf8KpVL1f1hyZqak6Fbn/4CET+dFId0Vig1GSShWfTKw0iOjSvG+pPBwREYAEAdvp4tMjSJ/0/fmiPyPR+ro6ZvoILDSJMDBj7evMeS0GFIocC0UCOBQDYZc2swEGFp7kj/M8BMcFKjgU5FIgmViwAwA4aJHw43Pzx+37x2fBKy0jN5D+YzbHQ0ecDOzVlpQJRQ2ABAKEqm9Fh5EPo7eBua8/30fBKe1N4lpG6Z314rzZowKGP7Sg87LOXhTungqAC0UZgAQB2TSK1yqPhlb8umho46P3eWxkaLGjAoY9p2OD5PHIqEEvkWABAqJNIwwkqshqWN7wK1EXTfZ8+rsd50kBDAw76UiCesGIBALZNIrXg0onlZaTBumjqO+njepx3magGDzoSnb4UiBcEFgBg6yTSIDJzRXpPrlBqarbCw99x9KVAPCGwAIBITCL1VCVT5OSLRc65yWfnTbMVHqF22wSiicACAIJVfHh2wDQ7ibTHeCmtWV++21dD1tc4Xepn1/S7RWG2woPJo0gEBBYAYKbiw5jZMdHVclv/XLTdT56FTiJtKPNq9ZFx7/9QljuxOmDpKBUeSCYOp9MZZgaSNUVFRZKTkyOFhYWSnZ0dzbcGAJMVH/38P66tuJVWhRgqhwHfdJgsV31Wr1LY4Q4L/FVsWOljAUSb2es3gQUAeG5/PHqyyKHfAydf3rte5PsPfKxqNJKSHvnSeXYtv1Ue7m2NJSMu9LkCYbbzJhBtZq/fbIUAgJuOOQ8UVCh9XI/zM4l0ecFe2V64LKTSUUWFBxIdgQUAuBV8bv64Zl19TiKdv26H7UPEgERCYAEgNfmq+jC74+DnOM2R+NcXm0y9BKWjSFYEFgBSj7+qj7PcCZlBnNS50l3uttzBUDqKZEdgASA153x412xo+eiiCSLVaokc2R84edNr+8NMW243fVdKR5HMGEIGIHUEnPOh9zlE0qsFfg1tx+3VOdNKzsTfOjWhdBRJjcACQOoIOufD6ar6uOB+kawGlaeRag8LjxkfoeRM6MAwIJmxFQIgdZIzzc75qNtc5O61vlt6+0FbbsCFwAJA6iRnthto7vkaSPgoJQ0kUFtuKfv+f85pbPr1gETFVgiA5FqlWDjR1ZLbe8tDkzMX5otk1vFfL2rM+WjkWp0IgeZOaLtuXZnw5Z/zf5LOExcYZalAsiKwAJA8qxT/PE1k4Xg/B7jXENzrCd7BRdn3PScE3PIwE1xou+67u5/i83HdKtFVDYILJCsCCwCJvUKxcZHI//V3rVLsC3ax9kjOzPZKztStkr6v+U3OtGraii3+zsCgPS+09wWQbMixAJC4KxSzh4oc3mP9uZqcOWyNpeRMK4L1tAg2LwRI2RWLCRMmiMPhkGHDhtl3RgAQzNpZrhWKUIIK7+TM0//i+mpTUGGlpwXzQpCMQl6xWLFihTz//PPStm1be88IAAJZM0vkHZPVHT6TMxuGnJxpltmeFswLQTIKacVi//79csMNN8iLL74odepohjUARGn7450BfjpnmhRmcqYnzZH4csNueW/VNuOrO2diz4EjEqhjtz6kPS/oaYFkFNKKxR133CG9evWS7t27y8MPPxzw2OLiYuPmVlRUFMpbAkh15e24Q6RlpBpUmEzO1CBBcyB0u0JXFjQI8JzvoVUdmoDpmUuhwcIVZzSQFxYXBA19mBeCZGU5sJg2bZqsXLnS2AoxIz8/X8aNGxfKuQFIdZ7dM/UWsB13AFoFcv49plcqNGgYO3ut7Cj645eivOwMGXvFaUY5qT6uJaPewYMGGc8vLgj42hpLPH3dWcwLQdKyFFhs3bpV7rrrLvnkk0+kenVze4OjRo2S4cOHV1ixaNyY7nMAQuieGeFVCqVBw+1vrKx0vwYZev+z158lD33wXcibMbpbUqdmRojPBpIssPj6669l586d0q5du/L7SkpKZPHixfL0008bWx7p6RV/I8jIyDBuABD2aHMruo4U6XqfpXwK3f4YOWN1wGPuffe/cqC4JPTzohoESc5SYHHRRRfJ6tUV/0/317/+VVq1aiUjRoyoFFQAgOVtD21yNW9keEFFx6Ei3UZZftqyDbtl78GjAY8JN6hQVIMgmVkKLLKysqRNmzYV7qtZs6bUrVu30v0AENVtD5WRJXLF0yKnXRnS07/cuEsiiQmnSAV03gSQ+NseOljsvEGWEjR9M1elUSsj3Vi5CHTG3hNO3a9MNQiSXdiBxcKFC+05EwCpt/VR8LnInKGhBRU9xrs6aNrYjlvbaz/92fqgx93cuZlM/vQnv8HDrec3ldnfbq9QiqorFRpUUA2CZMeKBYAE2/oo65553u22tuFW7ZvVldo1qgbMs6hTo6rceVELadUgq1IfC8/g4b6epwbsgwEkKwILAAm09WHPaHN/9MI/4arTfZabuuVfdbpxnAYPF7fO8xs86FcGjCEVEVgAiN7Wx6YlInPuDD2fQlcqLPalsEoDhudubCdjZ6+THUUVu2p6b2UQPACVEVgAiOOtD4dIjboiPfNFshrYOto8kGCrEQD8I7AAEKdbH2UX8cv/GdEVCn9YjQBCQ2ABIAqDw5xR3/YINkQMQGQQWACIHO2kaXX7Q3tSXDNVpEnnkLc9/E0epdwTiLy0KLwHgFSlE0lN09UEh0jvJ0WadQ0rqNDJo55BhdpReNi4Xx8HEDkEFgAiR5tXWdn66PtaWPkUuv2hKxW+Nl7c9+njehyAyGArBEDkaBWHBgxFukrg52KemSvyl1dEmnYJu+Jj2cbdlVYqPOkZ6OOae0FiJhAZrFgAiBwNFHpOLPvG4WfrY7JI8wvCDip0i+OOf/tvbOWJseVA5BBYAIgs3drQLY7sBrZvfXjnVew9FHjkuZtWieh2yJcbdst7q7YZX9keAezBVgiAyNPgoVUvV5WIJnTaODgsUF6Fv7Hlew4US+eJC6gaASKAwAJAdGgQoXkUNtKg4uUlBQHzKrxdcUYDuePNbyoFIu6qkSk3tiO4AMLAVgiAhKTbH+0e+lj+8eF3po7XqaXPXN/OGGdO1QgQOaxYAIhb/rpnalARaAKpL89c107S0hxUjQARRmABIC6DCM2DeOiD7yrlQYzudarcP3ON6dd151W0b15X3v+vuS6gVI0AoSOwABBzvlpw+6J5EIPf/Mby62tSpq50aMBihtnjAFRGYAEgptylomayGqxmPmhexYSrTi9PxtStFF310ADFGWB1Q48DEBqSNwHEjJVS0VBoXoVnhYeuWujqhb92XZ6rGwBCQ2ABpPJI84LPRVa/4/qq30eZ5lRYKRW1ok6NqkZehTcNNLSkVFcmPOn3lJoC4WMrBEhF62aLzBtRcaS5dsLU9ts2dMI0K5JJkv+4so3flQcNHi5uneez4gRAeAgsgFQMKqb3r5yxoIPC9H6b2mxHMklSL/+Btk9uO7+pXNa2YcDX0CCCklLAfgQWQCrQbQ5tp71vu8i8UX4uy3qfQ2TeSFf7bRvabQcTLJnSH3/H1sqoIo9c3VYua8t2BhAr5FgAyW7NLJHHWohMvVxkxi0iB3cFONgpUrTNFYREQaBkSl+C7VTUykiXHm3y7Dk5ACFhxQJIZh+PFln6pPXn6aCwKHEnU3r3sXA1w2otdWpWM/Igdu1zNcwKZEdRMV0zgRgjsACS1dpZoQUVSqePRpGZZEodb24GXTOB2CKwAJI1p+KD/xfCEx2u6hAdaR5lwZIp6ZoJJAZyLIBkpDkSAXMpfClbHeg5ISqJm6EmevpLs9D79XG6ZgKxRWABJGOjq1ByJHSlIoqlplbRNRNIDGyFAMnY6KrdQHPPz8gW6fW4SFYD1/ZHBFcqvKeX/umkOvL15j2WGlT5S/TUrpkaVNA1E4g9h9PpjFSbfp+KiookJydHCgsLJTs7O5pvDaRGoyt3+6jMXJFDvwd+jb9MFWlzpcRieqnGEKUep97AQnDgHaTQNROIn+s3KxZAom59zBkauNFVsB6VHYdGLajwNb3UM6hQ2iRLjzMzr4OumUD8IscCSLRVikltRF7vI3JoT4ADna7VigtGubZGPNWoJ3LNVJFLHoqr6aXuY/R4fR6AxMSKBZDwWx8B1G0uMmyNq0pEEzq1P0WEcynCmV6qP5keT5MrIHERWACJsv2hSZqWJmqUNbrSIKJpF4mFUJtV0eQKSFwEFkAi0BUHz8qPOG50ZUezKppcAYmLHAsgEVjqSxE/ja6CNbXyRpMrIPERWACJwMrsjjhqdGVleilNroDkQGABJALd0jCqOwJccDPriPSfLTJsdVwEFd5NrbSJlSfv2EEfN1NqCiC+kWMBJALd0ug5sawqxLs3RdkVuveTIs26SjzyNb00lM6bAOIfnTeBhG/h3ciVTxFHqxQAkg+dN4F4Lh0Nta+EBg+tesWsLwUABENgAcTD0DDd5jC74hDDvhQAEAzJm0C0O2d696Mo2u66Xx8HgARHYAHEvHNm2X3zRrqOA4AExlYIEI18ioJFQTpnOkWKtrmOtXmbgxHjAKKJwAKIZj6FrR02zY0s12mhnoPAtLOlNqGiXwSASGArBLB7haLgc5F5o0Sm97M438Nih00TQcWgN1ZWmi66o/Cwcb8+DgAxDSymTJkibdu2NepX9dahQweZO3eu7ScFJOwKxaQ2IlMvF1n2rMUn69CwRrYNDdPtD12pCJDRYTyuxwFAzLZCTjjhBJkwYYK0aNFCtK/W1KlTpU+fPvLNN9/IaaedZuuJAQm1SrH4MZGF40N8AfuHhmlOhfdKhScNJ/TxV78okHpZGeReAIifzpu5ubny6KOPyk033WTqeDpvIulWKebeJ7IvjG2FCHTOfG/VNrlr2ipLzyH3AkBMO2+WlJTI22+/LQcOHDC2RPwpLi42bp4nBiRVXwqfGw4mdLnXNdsjAp0zdQXCKnfuBYPAAEQ1eXP16tVSq1YtycjIkNtvv11mzpwprVu7xiL7kp+fb0Q47lvjxo3DOmEgLrY+Ni4SmXNniEFFWT5Ft1Gu0tIItOPWbQ1dgbCysUHuBYCYBBYtW7aUVatWyVdffSWDBg2SAQMGyLp16/weP2rUKGPZxH3bunVruOcMxD5B87UrRA7tjYt8Cl80V0K3NTze0RR37oXmaABATHIsunfvLs2bN5fnn3/e1PHkWCBltz5iMInUVx8LMyb/z5nS58xGETsvAIknatNNS0tLK+RQAKnXktuE9oNFWl5mSz6FlU6amitxceu88uN37SuWhz74LiI5GgBgObDQbY1LL71UTjzxRNm3b5+8+eabsnDhQvnoo4/4NJHctNW21WZXEVihCKWTpgYdHZrXLQ9KXlpSYCRq+gqRNDzJy3EFKwAQ8cBi586d0r9/f9m+fbuxHKLNsjSouPjii0N6cyDuZ3xoi23thhlKOekF94ucf49tuRTuTprOMKo53LkXerwGEZ6v5V7z0MfpZwEgZjkWVpFjgbh27IjInGEi62aKHD34x/016ooc3B2zPApdaeg8cYHfXAn3SsOSEReaCgqYIQIgbnMsgKTx8WiRpU/5zqMwE1Rk5or85ZWIlJCa7aSpx7m3PQLxzr2g8yYAuxBYILXplsemJSILHhL5eYXJJ/nZROg9WaT5BZE4S+PiH85x/hI+zQQhAGAFgQVSu3x0zlCRQ3usPc/YFtn1x/fZDSNeQmq2SsPXcWx7AIgmAgukcE+Kfsa6g+XF/575IlkN/kjstFhCaqVc1LuTptVqDjsSPgHACgILpOb2x5yhoQUVSoMKzaMIIYjYc+CIPPSB9dWDQNUcUvb9ZW1cORPuQCXY6HR9HX1ccy3IrQBgF6pCkHp0zoe25A6FVnwMW+13hcIzkNi064C8tXyL7CgK3EDOfUk3s3rga1tDYwLP0R7uQCUns5pc9+KyoD/SW7e0J9cCQFBUhQD++lL89HHorxNgxkeo7bOtrB54VnN8sm6HvPzFpgpBhec2x187NbE1MRQAzCCwQPIHFIsfE/nq2RCHhpWpVkvkyil+EzT95TKYZaVcVAMP3e4YPn2V39fS0OS9VeY6hdK+G4CdCCyQ3AHF0skiRw6E/DLGRfq0q0Sufing9oe/XAarzK4emOlrsfvAEcmtWVX2HDhK+24A8Ts2HYh7a2aJTDhRZOH4gEGFv0BAs470NsdxgZTev1PkmlcCVn0Eu8hbYXb1wGwA8ueyCaXemyu07wYQKQQWSL7ume8MEDmyP+ihgS6nzx+7XKpe/ZykV8uISo6Coyzp0uzqgdkApHvrPCMpVFcmPOn3lJoCiAS2QpA81s4SWfpkWC+xy5klj1e5TbpcfbNRVfHeqm1Be02Em6MQyuqBlb4W+pq07wYQLQQWSJ6cig/+X0hPLblkvGw4WEN2OmtLepNO0uXQMUu9JoJd5IPJC6ELptUppbTvBhAt9LFAcij4XGTq5WH3pfBX3RGo14Qmbz69YL38c/6PQd9OA5DRvU6VOjUzbFk9oF03gGihjwVSi/anCLMvRSidKoP1rtCL/P+cc6I0qVcjIlsQTCkFEG8ILJAcdGaHFTrivPdkKWnVW5Zv2G1clHftKzY1mnzZht2SluaQ+et2yL++2OT3+Lu7t5AhF7aI+EWebQ4A8YTAAslBB4HplNGiIE2hqtYU6XSXyPn3yLx1O2XcxAWWS0XveHOl7D10NOAxGkpMW7HVCCwAIJVQborkoNsZPScGLiLVRlejtopcMMIIKjSXIpT+E8GCCu9OmgCQSggskDy03Xbf11wrF55q1BO5Zmp5oys7O2UGwxwOAKmGrRAkxuAwzaHQ7Y4AHTDLg4tWvQI+z85OmcEwhwNAqiGwQOIMDtOVCN3u8DMIrJwGEU27+B1n/tOvwbtyhos5HABSFYEF4su62eKcc5c4DvnITSjaLjK9v2u7I1hwYcM481AxhwNAKiOwQHwFFRo4+B8P5rpszxvp2u4IMG3UvTpR8Nt+mfTpeltOr2ZGuhwsLgmamxFKJ00ASBYEFogPpSVyaM69Ut3pFEfAX/KdIkXbXDkUXtsdkV6duKVzM5n86U8+W2jr93/r1MRoVkWDKgCpjMAiDnn+xp3UnRQ9kjNL9v0qmYd2BB45GqTTpr923HaoU6Oq3HlRC2nVIKtS4MIKBQD8gcAizqTM7Id1s0XmjShvaBWk1iNop81Il5DmX3W6EdzRQhsAAiOwiCP+fuPWqZl6v68BWAkbVATMpfDPyLLQwWFaQuohUiWkvoI6WmgDgH8EFnEilAFYCdmPouZxInPv8xtU6KxdfzkWxhxeR8XBYZFqRFU7s6o8c0M7ad+sbuJ93gAQQwQWcSLYb9yeLaIT6rdlrfSYO0Ic+4LM8CgTKHFzr6OWZF/zrKT7KDW1qxGV++0nXH26dDq5ni2vCQCphJbeccLsb9wJ1SLaKB/tJ2IyqPBnj7OWPH70L/LV1V9J+ml9fB6jeQ66bWGWLkLc0qVJpedoImbSbDkBQAywYhEnzP7GnRAtonXro+BzOfbubZIeYGsjkAeP3ii7nLVlp9SWrbXOkNHXnB7wYq/bFaN7tZbBb6409fpPX9dOLmvbQEZe2ppETACwEYFFnHD/xq2Jms5EbhHtUe1h/OWyeI0udYrskLpyQo9hckZ2DUsX+zo1q5l6j7u7n2IEFYpETACwF1shcUIvcFp9oBxx1CJak0q/3LBb3lu1zfiq3wet9igrIbXK/dLjjvaTutk1pM+ZjYyLvtmf2ew2UZN6NUI6PwBAcKxYxBFd6tf9fTMNmOxuouXr9T5Zt8N8Tw3d/tCVijA6SehKhQYVH5WeKwND2PJJqu0kAEhQBBZRphfwZRt3G7/960W4Q7N60t7jt3IzDZjsbqLl6/Vq16gqew8erXSs354aWk4awkrFQ0evl9+cuUYuxfLSVuKUNONn0Z/ZavCUNNtJAJDAHE6n0R0gaoqKiiQnJ0cKCwslOztbUolewEfOWF3pgq0X8QlXBU5ODNZEy325tVrREEobbPcFesmIC/+40K9+R+Tdm0y/hv6t2yO15Ozi56S0bEfO82dQoQRP7p/HeA+vc3a/NhUfABC56zc5FlGiF7zb31jpcxVA79PH9JhAOQ3BmmjpbezstYHzIGxog+3ZU8Nfi+2Az3e6XuPhtNvLgwrPUk+lwYF3Xw/3aol+TsG2k/S1PFFGCgDRwVZIFOgFfOzsdUGPG/Huf43jdhT5/i3dTNvqHUXF8vSC9XJX9xbl763P09f8fX+x5NasJnk5mcZ2QLhtsCskS2qL7eyGIkV60Q8cqvzqyJXtHcbKoxf3l2u8tjpU54kLwupAyjwPAIgdAosocF/Ygyk8dMy4+ctpKD5Waur9/jn/R2mZV8v4s78R4hqwXNYmT8JRIQlSW2z3nGhUhTjFIQ6P0MD9pw3N+0txs57S6rweklfF9VfPu9RTV2ns6EBKGSkAxAaBRRSE0y3T87f0x645w/TzNJej8OBRv2sHenH+1xebgr5OmpTKuWnfS33ZWyHB0mcSZOsr5JsOk6Xhl+PkeNHkVJdfpa5s7zBGzuoxIDU7kAJACiGwiIJwyxvdv6XrH3Slwcz2ha9cDl90d8Cd8+CtR9pyGVP1NWno+COX4hdnrjx4tL9c2fv2SlsLRuLkZ/XEIZMrBCMrSltJ6WdpMqXR9qA5DpSMAkBiI3kzCvQ3+7zs8C+Euw4UlzfRsovmebpXRTz1TFsuU6pOkjyPoELp91OqTZaeaSv8JoJqQuay0tYyu7Sj8bWk7K+ZPh4ssdRdMuovG0Lvd5ejAgDiT0oGFpa6SdpAf7Mfe0X4AYH+lq5JiTUzKo4MD9ffOjUxtjZ026N92jrpk7ZExld72Zjx4f0XRL83LvrzRrqaYlmczvrqFwUBP+947UAKADAn5bZC7G4uZZa+9nM3tvPdxyKzihwpccrBI39cqL1prwt3JceBYv/HhUKDlV7pX0nT5WMkV4pMPMMpUrTN1RSraRdLOQ8PffCdvLSkIODnbaUDKQAgvqRUYOGvGZTfbpI2c5dBenfePKdprpz54McBn3u0rCLE7qRFDarqL3tYmv3oWqGwZP+vIeU8mPm8KRkFgMSUMoFFsOZSZvoj2EFfu9PJ9Yyb2xc/7Qq4WqEOHCmRZRt225q0qD/l02dskWbLXw7tBTyaYgVrpx3K503JKAAknpTJsTCbA1Chm2SUfLlxl+njgiU3mqWvMeWGM6Tttw8ar2VttcIhkt3I1RTLRG5EvH3eAIDISZnAIr77I5i9qjsqXMBDlVuzqiy6t5v0rFUgVYt/D+1ce05wNcUy0U47EPpRAEBySZnAIlb9EcxUoJhd7ncf576Aa4Bglrvi44q0pXLKoW/l64JdFXIkTNO23X1fM5ph+aLnpsPJRvc61dTL0Y8CAFI4xyI/P19mzJgh33//vWRmZkrHjh1l4sSJ0rJlS4l30Rqp7Tnqe9Oug/LW8i1+Z3+4aYdM3YoINGe2To2q0r7ZHwGIPv/CVsdL+/xP5fcDRwKek69GV4fefVGkvblppEVSQ2r9eZKkaVCh2x9eKxXedFVlYKemRvUHI8wBILVYWrFYtGiR3HHHHbJs2TL55JNP5OjRo3LJJZfIgQMHJN7Z0R8h2OqDVp3oAK3rXlwmd01bZczs8J4R4j2hU7/e8ebKgEGFyr/q9ErnVq1Kmoz/cxtXjoSfVYqh6e/Kc1UnSQOpuOVR/fBOkYXjRTJz/SZbGh05nSIb2o+XtDOudZWWBgkq3OhHAQCpyeF0Bruk+ffbb79J/fr1jYDj/PPPt3Wee7z1sQj2PH+lrBLgt3XNc+j66GcBk0r1uvv0dWfJZW0bWjo3XaXIr/qi5DoCBX0Okcw6Iof2iNMYHVaR/iwFp9wkza5/wsRPFV99QwAA9jJ7/Q6r3FRfXOXm+l/OLi4uNm6eJxZLofRHCNb/4pnr28lDH/guZQ1UEfH6l5uCzv3QRZE6NTMs/Uzp38+Ry76bZCIl1Cly6HeRC+4Xx8pXRYp+KX/kSEZdSe/9uDRr82eTP5W5c6MfBQAkt5ADi9LSUhk2bJh06tRJ2rRpEzAvY9y4cRJPrPRHMNP/YvR7a2R3kDwHXzb/ftC2yol0KZUOaetE0raLbJ4oTof5WhOp21xk2BpXJ01N6Kx1vFQzkUthFv0oACB1hBxYaK7FmjVrZMmSJQGPGzVqlAwfPrzCikXjxo3FTp4Jk3b/Rmym/0UoQYU6KbeGPZUT62aLzBtRYcXBYbXRlQYRZe25AQCIamAxZMgQef/992Xx4sVywgknBDw2IyPDuEVKpPfwI9FnwZ1j0a9Dk/AqJ3QI2KJHRBZNCP1kvBpdAQAQtaoQzfPUoGLmzJmyYMECadq0qcSSO/fBe0XBu/IiHGb7LGhPCSurBBr4aFVHyJUTa2eJTDgxvKDCT6MrAACiEljo9scbb7whb775pmRlZcmOHTuM26FDhyTaguU+KH083JHowVpo6/36+MN9XHkmwYILjRFuPb9p+WqKv26V+r3fIV0fjxZ5e4DIkf2h/VDGiaaL/GWq30ZXAABEvNzU4WegxCuvvCIDBw6Marmp9pHQfhHBvHVL+6CJg8FyNNwrI8rzw3If4Q4AfG3LePN+jtlzKLdmlsg7AyRs10wVOe3K8F8HAJASiiJRbhpGy4u4nf1hJkfDvargfVyej+OMsegbdhtNr/YeOmp6sqepygnNqfjwj0TYkGTmivSezEoFACAiqqTy7I9g/Sk8VxXM9mPQ79PSHD6DCl+TPS2VYWo56MHdYklmXZGz/+oKZ7Tqo0lncioAABGTsIFFuLM/zPSnCGlVIZKTVEMZGtZ7EqsTAICoSdjppuHOojDTn8K9qhDV1RTd7ti4SOTTh0UWPCyyYaHrPne/CbNIzgQAxEDCrlhYyX2I6qpCOKsp2uhqzlBjdscfHnXN8+j9pEirXq6x5R6NsPz6y8skZwIAoi6hA4twZlHYkaMRbDVF8zQcfipJKq2mBKr20EBjej+Rvq+L9JwoMr2/16t6qFZL5MoprFQAAGIiYbdCPLlzH/qc2cj4aqadt9n+FH67XgZhqT+FNrt6x0S57twRrlWLvq+5Vi48VcsSOX+kyMgtBBUAgJhJ+BWLUIW0qhCJ1RTd/tBmV2bs+8VVGaKBgwYYHkPDjLbcVHsAAGIsZQOLcHM0zApYSaJJmTo8LJTKEIaGAQDiUEoHFuHkaIRMg4lNS0QKPhcp3GIuEdOTlcoQAACiLOUDCyv9KcJmVH3cJXLIeglreR4Fk0gBAHGMwCLSdIVCcyF++FBk2bPhvdYVT5JHAQCIawQWkV6hmHufyL7wx7dLy8tE2lxlx1kBABAxBBaRDCq094QdOtwp0uNhe14LAIAIIrCIxNaHJmbOuj2816lWU6R1H5HLJ4tUqWbX2QEAEFEEFnavUmj5qNVKD09d7hVp1pW+FACAhERgYevWR4BW22ZkNRTpNoqAAgCQsJKipXfMlTe6CiOoUJdOJKgAACQ0Ags7aDlpONsfmbmuAWPM+AAAJDi2QuzgbrNtRZVMkQ6DRZqc72rNzUoFACAJEFjYIZQ221e9wAoFACDpEFiE0kXTe6KoftUx5kXbg+dZZDcS6TmBoAIAkJQILMIpJdVgoudEV5CgX42qEO8h7GXaD3Z1z6SMFACQxEjetFJK6p2gqSsUer8+rsFF39dEshtUXqHQxMye+eRSAACSHisWYZWS6n0OkXkjRVr1cgUX+tXXdgkAACmAwCLsUlKnSNE213HuFQn9CgBACmIrxK5S0lBKTgEASDKsWASq9rBSShpKySkAAEmGwCJYtUfQUlKH63E9DgCAFJeW0qPN540Smd4vcLWHrlxokGHQUlJPZd9rXwoSNAEASMHAQoOFSW1Epl4usuxZPweVrUxotYcGIX5LSRu67qfZFQAAKbgVYmm0uVe1B6WkAAAElTqBRaijzT2rPSglBQAgoNTZCgl1tDnVHgAAmJY6KxaW+0xQ7QEAgFWps2JhaeWBag8AAEKROoGFux9FpZJRH6j2AAAghbdCAnXOdHP3o2C0OQAAEVMl6TtnenL3o6h0fCPXtgcrFAAAhMXhdDot1l+Gp6ioSHJycqSwsFCys7Mj1JeibLvD33aGmRUOAABg+fpdJTn7Uuh9DlfnTG1q5WtbhH4UAADYLi15+1J4dM4EAABRkZb0fSks968AAACpF1iY7UtB50wAAKImLXn7UmjnzEZ0zgQAIIoSN7Bw96UweAcXdM4EACAWEjew8OxLkd2g4v10zgQAICYSt9zUTYMHLSmlLwUAADGX+IGFoi8FAABxIbG3QgAAQGIHFosXL5bevXtLw4YNxeFwyKxZsyJzZgAAIPkDiwMHDsgZZ5whzzzzTGTOCAAApE6OxaWXXmrcAAAAop68WVxcbNw8p6MBAIDkFPHkzfz8fGPMqvvWuHHjSL8lAABI1sBi1KhRxux2923r1q2RfksAAJCsWyEZGRnGDQAAJD/6WAAAgNitWOzfv1/Wr19f/n1BQYGsWrVKcnNz5cQTTwz6fKfTaXwliRMAgMThvm67r+N+OS367LPP9BUr3QYMGGDq+Vu3bvX5fG7cuHHjxo2bxP1Nr+OBOPR/JIpKS0vll19+kaysLKNzZziRk1aYaDJodna2recI3/jMo4/PPPr4zKOPzzwxPnMNF/bt22d03k5LS4ufIWR6MieccIJtr6cfCH8Ro4vPPPr4zKOPzzz6+Mzj/zPXthHBkLwJAABsQ2ABAABsk7CBhfbGGDNmDD0yoojPPPr4zKOPzzz6+MyT6zOPevImAABIXgm7YgEAAOIPgQUAALANgQUAALANgQUAALANgQUAAEiNwOKZZ56RJk2aSPXq1eW8886T5cuXBzz+7bffllatWhnHn3766fLhhx9G7VyThZXP/MUXX5QuXbpInTp1jFv37t2D/jdC+H/P3aZNm2a0xb/yyisjfo6p/pnv3btX7rjjDmnQoIFRnnfKKafw70sEP+9JkyZJy5YtJTMz02g7fffdd8vhw4ejdr6JbvHixdK7d2+j9bb+GzFr1qygz1m4cKG0a9fO+Pt98skny6uvvhr6CTjj1LRp05zVqlVzvvzyy861a9c6b7nlFmft2rWdv/76q8/jv/jiC2d6errzkUceca5bt87597//3Vm1alXn6tWro37uicrqZ3799dc7n3nmGec333zj/O6775wDBw505uTkOH/++eeon3uqfOZuBQUFzkaNGjm7dOni7NOnT9TONxU/8+LiYufZZ5/tvOyyy5xLliwxPvuFCxc6V61aFfVzT4XP+9///rczIyPD+Kqf9UcffeRs0KCB8+677476uSeqDz/80PnAAw84Z8yYYQwNmzlzZsDjN27c6KxRo4Zz+PDhxvXzqaeeMq6n8+bNC+n94zawOPfcc5133HFH+fclJSXOhg0bOvPz830e37dvX2evXr0q3Hfeeec5b7vttoifa7Kw+pl7O3bsmDMrK8s5derUCJ5lcgnlM9fPuWPHjs6XXnrJmCpMYBHZz3zKlCnOZs2aOY8cORLFs0zdz1uPvfDCCyvcpxe8Tp06Rfxck5GYCCzuu+8+52mnnVbhvmuvvdbZo0ePkN4zLrdCjhw5Il9//bWxtO45vEy///LLL30+R+/3PF716NHD7/EI/zP3dvDgQTl69Kjk5uZG8EyTR6if+YMPPij169eXm266KUpnmtqf+ezZs6VDhw7GVsjxxx8vbdq0kfHjx0tJSUkUzzx1Pu+OHTsaz3Fvl2zcuNHYdrrsssuidt6p5kubr59Rn25qxq5du4z/0+r/iT3p999//73P5+zYscPn8Xo/IvOZexsxYoSxp+f9FxT2feZLliyRf/3rX7Jq1aoonWVyCeUz1wvbggUL5IYbbjAucOvXr5fBgwcbQbS2RIa9n/f1119vPK9z587GmO5jx47J7bffLvfff3+Uzjr17PBz/dTR6ocOHTJyXayIyxULJJ4JEyYYyYQzZ840ErRgv3379km/fv2MpNl69erF+nRSRmlpqbFC9MILL8if/vQnufbaa+WBBx6Q5557LtanlpQ0iVBXhJ599llZuXKlzJgxQz744AN56KGHYn1qSOQVC/1HMz09XX799dcK9+v3eXl5Pp+j91s5HuF/5m6PPfaYEVjMnz9f2rZtG+EzTd3PfMOGDbJp0yYj29vzoqeqVKkiP/zwgzRv3jwKZ55af8+1EqRq1arG89xOPfVU47c8XeqvVq1axM87lT7v0aNHGwH0zTffbHyvFX4HDhyQW2+91QjodCsF9vJ3/czOzra8WqHi8r+Q/h9VfzP49NNPK/wDqt/rXqcver/n8eqTTz7xezzC/8zVI488YvwmMW/ePDn77LOjdLap+ZlrKfXq1auNbRD37YorrpBu3boZf9ayPNj/97xTp07G9oc7iFM//vijEXAQVNj/eWuulnfw4A7qmJkZGbZfP51xXKKkJUevvvqqUf5y6623GiVKO3bsMB7v16+fc+TIkRXKTatUqeJ87LHHjNLHMWPGUG4a4c98woQJRhnZO++849y+fXv5bd++fTH8KZL7M/dGVUjkP/MtW7YY1U5Dhgxx/vDDD87333/fWb9+fefDDz8cw58ieT9v/bdbP++33nrLKIP8+OOPnc2bNzcq/2CO/husbQD0ppf5J554wvjz5s2bjcf189bP3bvc9N577zWun9pGICnLTZXW0p544onGxUtLlpYtW1b+WNeuXY1/VD1Nnz7decoppxjHa+nMBx98EIOzTmxWPvOTTjrJ+EvrfdN/GBC5v+eeCCyi85kvXbrUKF/XC6SWnv7jH/8wyn5h/+d99OhR59ixY41gonr16s7GjRs7Bw8e7NyzZ0+Mzj7xfPbZZz7/bXZ/zvpVP3fv55x55pnGfyP9O/7KK6+E/P4O/R97FlMAAECqi8scCwAAkJgILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgG0ILAAAgNjl/wOB6vqgznVLNAAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "n = 100\n", + "x = np.random.rand(n, 1)\n", + "y = 2.0 + 5 * x**2 + 0.1 * np.random.randn(n, 1)\n", + "\n", + "line_model = LinearRegression().fit(x, y)\n", + "line_predict = line_model.predict(x)\n", + "#line_mse = ...\n", + "\n", + "#poly_features = ...\n", + "#poly_model = LinearRegression().fit(..., y)\n", + "#poly_predict = ...\n", + "#poly_mse = ...\n", + "\n", + "plt.scatter(x, y, label = \"Data\")\n", + "plt.scatter(x, line_predict, label = \"Line model\")\n", + "plt.legend()\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "id": "248d8931", + "metadata": {}, + "source": [ + "## Exercise 4 - The train-test split\n" + ] + }, + { + "cell_type": "markdown", + "id": "1efd3376", + "metadata": {}, + "source": [ + "Hopefully your model fit the data quite well, but to know how well the model actually generalizes to unseen data, which is most often what we care about, we need to split our data into training and testing data. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0f8d75fb", + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.model_selection import train_test_split" + ] + }, + { + "cell_type": "markdown", + "id": "edb213fc", + "metadata": {}, + "source": [ + "**a)** Complete the code below so that the polynomial features and the targets y get split into training and test data.\n", + "\n", + "**b)** What is the shape of X_test?\n", + "\n", + "**c)** Fit your model to X_train\n", + "\n", + "**d)** Compute the MSE when your model predicts on the training data and on the testing data, using y_train and y_test as targets for the two cases.\n", + "\n", + "**e)** Why do we not fit the model to X_test?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a03e0388", + "metadata": {}, + "outputs": [], + "source": [ + "polynomial_features = ...\n", + "\n", + "#X_train, X_test, y_train, y_test = train_test_split(polynomial_features, y, test_size=0.2)\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "22e7536e", + "metadata": {}, + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.15" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/doc/LectureNotes/_build/jupyter_execute/exercisesweek35.ipynb b/doc/LectureNotes/_build/jupyter_execute/exercisesweek35.ipynb new file mode 100644 index 000000000..6b559d91e --- /dev/null +++ b/doc/LectureNotes/_build/jupyter_execute/exercisesweek35.ipynb @@ -0,0 +1,539 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "b4005770", + "metadata": {}, + "source": [ + "# Exercises week 35\n", + "\n", + "## Deriving and Implementing Ordinary Least Squares" + ] + }, + { + "cell_type": "markdown", + "id": "2ca1b589", + "metadata": {}, + "source": [ + "This week you will be deriving the analytical expressions for linear regression, building up the model from scratch. This will include taking several derivatives of products of vectors and matrices. Such derivatives are central to the optimization of many machine learning models. Although we will often use automatic differentiation in actual calculations, to be able to have analytical expressions is extremely helpful in case we have simpler derivatives as well as when we analyze various properties (like second derivatives) of the chosen cost functions.\n", + "\n", + "Vectors are always written as boldfaced lower case letters and matrices as upper case boldfaced letters. You will find useful the notes from week 35 on derivatives of vectors and matrices. See also the textbook of Faisal at al, chapter 5 and in particular sections 5.3-5.5 at " + ] + }, + { + "cell_type": "markdown", + "id": "41e92bf9", + "metadata": {}, + "source": [ + "### Learning goals\n", + "\n", + "After completing these exercises, you will know how to\n", + "- Take the derivatives of simple products between vectors and matrices\n", + "- Implement OLS using the analytical expressions\n", + "- Create a feature matrix from a set of data\n", + "- Create a feature matrix for a polynomial model\n", + "- Evaluate the MSE score of various model on training and test data, and comparing their performance\n", + "\n", + "### Deliverables\n", + "\n", + "Complete the following exercises while working in a jupyter notebook. Then, in canvas, include\n", + "- The jupyter notebook with the exercises completed\n", + "- An exported PDF of the notebook (https://code.visualstudio.com/docs/datascience/jupyter-notebooks#_export-your-jupyter-notebook)" + ] + }, + { + "cell_type": "markdown", + "id": "f7a9209d", + "metadata": {}, + "source": [ + "## How to take derivatives of Matrix-Vector expressions" + ] + }, + { + "cell_type": "markdown", + "id": "45f3712e", + "metadata": {}, + "source": [ + "In these exercises it is always useful to write out with summation indices the various quantities. Take also a look at the weekly slides from week 35 and the various examples included there.\n", + "\n", + "As an example, consider the function\n", + "\n", + "$$\n", + "f(\\boldsymbol{x}) =\\boldsymbol{A}\\boldsymbol{x},\n", + "$$\n", + "\n", + "which reads for a specific component $f_i$ (we define the matrix $\\boldsymbol{A}$ to have dimension $n\\times n$ and the vector $\\boldsymbol{x}$ to have length $n$)\n", + "\n", + "$$\n", + "f_i =\\sum_{j=0}^{n-1}a_{ij}x_j,\n", + "$$\n", + "\n", + "which leads to\n", + "\n", + "$$\n", + "\\frac{\\partial f_i}{\\partial x_j}= a_{ij},\n", + "$$\n", + "\n", + "and written out in terms of the vector $\\boldsymbol{x}$ we have\n", + "\n", + "$$\n", + "\\frac{\\partial f(\\boldsymbol{x})}{\\partial \\boldsymbol{x}}= \\boldsymbol{A}.\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "5fa8a4e6", + "metadata": {}, + "source": [ + "## Exercise 1 - Finding the derivative of Matrix-Vector expressions" + ] + }, + { + "cell_type": "markdown", + "id": "df7a2270", + "metadata": {}, + "source": [ + "**a)** Consider the expression\n", + "\n", + "$$\n", + "\\frac{\\partial (\\boldsymbol{a}^T\\boldsymbol{x})}{\\partial \\boldsymbol{x}},\n", + "$$\n", + "\n", + "Where $\\boldsymbol{a}$ and $\\boldsymbol{x}$ are column-vectors with length $n$.\n", + "\n", + "What is the *shape* of the expression we are taking the derivative of?\n", + "\n", + "What is the *shape* of the thing we are taking the derivative with respect to?\n", + "\n", + "What is the *shape* of the result of the expression?" + ] + }, + { + "cell_type": "markdown", + "id": "c0396734", + "metadata": {}, + "source": [ + "**b)** Show that\n", + "\n", + "$$\n", + "\\frac{\\partial (\\boldsymbol{a}^T\\boldsymbol{x})}{\\partial \\boldsymbol{x}} = \\boldsymbol{a}^T,\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "dc39d541", + "metadata": {}, + "source": [ + "**c)** Show that\n", + "\n", + "$$\n", + "\\frac{\\partial (\\boldsymbol{a}^T\\boldsymbol{A}\\boldsymbol{a})}{\\partial \\boldsymbol{a}} = \\boldsymbol{a}^T(\\boldsymbol{A}+\\boldsymbol{A}^T),\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "498d13ec", + "metadata": {}, + "source": [ + "## Exercise 2 - Deriving the expression for OLS" + ] + }, + { + "cell_type": "markdown", + "id": "f3f771de", + "metadata": {}, + "source": [ + "The ordinary least squares method finds the parameters $\\boldsymbol{\\theta}$ which minimizes the squared error between our model $\\boldsymbol{X\\theta}$ and the true values $\\boldsymbol{y}$.\n", + "\n", + "To find the parameters $\\boldsymbol{\\theta}$ which minimizes this error, we take the derivative of the squared error expression with respect to $\\boldsymbol{\\theta}$, and set it equal to 0." + ] + }, + { + "cell_type": "markdown", + "id": "49690237", + "metadata": {}, + "source": [ + "**a)** Very briefly explain why the approach above finds the parameters $\\boldsymbol{\\theta}$ which minimizes this error." + ] + }, + { + "cell_type": "markdown", + "id": "b7cccc9d", + "metadata": {}, + "source": [ + "We typically write the squared error as\n", + "\n", + "$$\n", + "\\vert\\vert\\boldsymbol{y} - \\boldsymbol{X\\theta}\\vert\\vert^2\n", + "$$\n", + "\n", + "which we can rewrite in matrix-vector form as\n", + "\n", + "$$\n", + "\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "8fbecf74", + "metadata": {}, + "source": [ + "**b)** If $\\boldsymbol{X}$ is invertible, what is the expression for the optimal parameters $\\boldsymbol{\\theta}$? (**Hint:** Don't compute any derivatives, but solve $\\boldsymbol{X\\theta}=\\boldsymbol{y}$ for $\\boldsymbol{\\theta}$)" + ] + }, + { + "cell_type": "markdown", + "id": "f37af8f0", + "metadata": {}, + "source": [ + "**c)** Show that\n", + "\n", + "$$\n", + "\\frac{\\partial \\left(\\boldsymbol{x}-\\boldsymbol{A}\\boldsymbol{s}\\right)^T\\left(\\boldsymbol{x}-\\boldsymbol{A}\\boldsymbol{s}\\right)}{\\partial \\boldsymbol{s}} = -2\\left(\\boldsymbol{x}-\\boldsymbol{A}\\boldsymbol{s}\\right)^T\\boldsymbol{A},\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "869fca4d", + "metadata": {}, + "source": [ + "**d)** Using the expression from **c)**, but substituting back in $\\boldsymbol{\\theta}$, $\\boldsymbol{y}$ and $\\boldsymbol{X}$, find the expression for the optimal parameters $\\boldsymbol{\\theta}$ in the case that $\\boldsymbol{X}$ is not invertible, but $\\boldsymbol{X^T X}$ is, which is most often the case.\n", + "\n", + "$$\n", + "\\boldsymbol{\\hat{\\theta}_{OLS}} = ...\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "57ca3d74", + "metadata": {}, + "source": [ + "## Exercise 3 - Creating feature matrix and implementing OLS using the analytical expression" + ] + }, + { + "cell_type": "markdown", + "id": "5dc179f7", + "metadata": {}, + "source": [ + "With the expression for $\\boldsymbol{\\hat{\\theta}_{OLS}}$, you now have what you need to implement OLS regression with your input data and target data $\\boldsymbol{y}$. But before you can do that, you need to set up you input data as a feature matrix $\\boldsymbol{X}$.\n", + "\n", + "In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, with the montly income and the number of children as columns.\n", + "\n", + "We typically also include an intercept in our models. The intercept is a value that is added to our prediction regardless of the value of the other features. The intercept tries to account for constant effects in our data that are not dependant on anything else. In our current example, the intercept could account for living expenses which are typical regardless of income or childcare expenses.\n", + "\n", + "We calculate the optimal intercept by including a feature with the constant value of 1 in our model, which is then multplied by some parameter $\\theta_0$ from the OLS method into the optimal intercept value (which will be $\\theta_0$). In practice, we include the intercept in our model by adding a column of ones to the start of our feature matrix." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "e5ff2a69", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "a3cf2792", + "metadata": {}, + "outputs": [], + "source": [ + "n = 20\n", + "income = np.array([116., 161., 167., 118., 172., 163., 179., 173., 162., 116., 101., 176., 178., 172., 143., 135., 160., 101., 149., 125.])\n", + "children = np.array([5, 3, 0, 4, 5, 3, 0, 4, 4, 3, 3, 5, 1, 0, 2, 3, 2, 1, 5, 4])\n", + "spending = np.array([152., 141., 102., 136., 161., 129., 99., 159., 160., 107., 98., 164., 121., 93., 112., 127., 117., 69., 156., 131.])\n" + ] + }, + { + "cell_type": "markdown", + "id": "5da61481", + "metadata": {}, + "source": [ + "**a)** Create a feature matrix $\\boldsymbol{X}$ for the features income and children, including an intercept column of ones at the start." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "5ad87a65", + "metadata": {}, + "outputs": [], + "source": [ + "X = np.zeros((n, 3))\n", + "#X[:, 0] = ...\n", + "#X[:, 1] = ...\n", + "#X[:, 2] = ..." + ] + }, + { + "cell_type": "markdown", + "id": "e0ddfac2", + "metadata": {}, + "source": [ + "**b)** Use the expression from **3d)** to find the optimal parameters $\\boldsymbol{\\hat{\\beta}_{OLS}}$ for predicting spending based on these features. Create a function for this operation, as you are going to need to use it a lot." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "8f3f68aa", + "metadata": {}, + "outputs": [], + "source": [ + "def OLS_parameters(X, y):\n", + " return ...\n", + "\n", + "#beta = OLS_parameters(X, y)" + ] + }, + { + "cell_type": "markdown", + "id": "0cb6da80", + "metadata": {}, + "source": [ + "## Exercise 4 - Fitting a polynomial" + ] + }, + { + "cell_type": "markdown", + "id": "71015064", + "metadata": {}, + "source": [ + "In this course, we typically do linear regression using polynomials, though in real world applications it is also very common to make linear models based on measured features like you did in the previous exercise.\n", + "\n", + "When fitting a polynomial with linear regression, we make each polynomial degree($x, x^2, x^3, ..., x^p$) its own feature." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "d7476c84", + "metadata": {}, + "outputs": [], + "source": [ + "n = 100\n", + "x = np.linspace(-3, 3, n)\n", + "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2) + np.random.normal(0, 0.1)" + ] + }, + { + "cell_type": "markdown", + "id": "8321451b", + "metadata": {}, + "source": [ + "**a)** Create a feature matrix $\\boldsymbol{X}$ for the features $x, x^2, x^3, x^4, x^5$, including an intercept column of ones at the start. Make this into a function, as you will do this a lot over the next weeks." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "91496e40", + "metadata": {}, + "outputs": [], + "source": [ + "def polynomial_features(x, p):\n", + " n = len(x)\n", + " X = np.zeros((n, p + 1))\n", + " #X[:, 0] = ...\n", + " #X[:, 1] = ...\n", + " #X[:, 2] = ...\n", + " # could this be a loop?\n", + "\n", + "#X = polynomial_features(x, 5)" + ] + }, + { + "cell_type": "markdown", + "id": "b84b1e31", + "metadata": {}, + "source": [ + "**b)** Use the expression from **3d)** to find the optimal parameters $\\boldsymbol{\\hat{\\beta}_{OLS}}$ for predicting $\\boldsymbol{y}$ based on these features. If you have done everything right so far, this code will not need changing." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "034f502c", + "metadata": {}, + "outputs": [], + "source": [ + "#beta = OLS_parameters(X, y)" + ] + }, + { + "cell_type": "markdown", + "id": "d703f788", + "metadata": {}, + "source": [ + "**c)** Like in exercise 4 last week, split your feature matrix and target data into a training split and test split." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "29171358", + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.model_selection import train_test_split\n", + "\n", + "#X_train, X_test, y_train, y_test = ..." + ] + }, + { + "cell_type": "markdown", + "id": "a0e3509f", + "metadata": {}, + "source": [ + "**d)** Train your model on the training data(find the parameters which best fit) and compute the MSE on both the training and test data." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "1e346f4c", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Ellipsis" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "..." + ] + }, + { + "cell_type": "markdown", + "id": "7e431889", + "metadata": {}, + "source": [ + "**e)** Do the same for each polynomial degree from 2 to 10, and plot the MSE on both the training and test data as a function of polynomial degree. The aim is to reproduce Figure 2.11 of [Hastie et al](https://github.com/CompPhysics/MLErasmus/blob/master/doc/Textbooks/elementsstat.pdf). Feel free to read the discussions leading to figure 2.11 of Hastie et al. " + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "ceb57457", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Ellipsis" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "..." + ] + }, + { + "cell_type": "markdown", + "id": "5e5b5954", + "metadata": {}, + "source": [ + "**f)** Interpret the graph. Why do the lines move as they do? What does it tell us about model performance and generalizability?" + ] + }, + { + "cell_type": "markdown", + "id": "ad2acfb9", + "metadata": {}, + "source": [] + }, + { + "cell_type": "markdown", + "id": "5994f0c5", + "metadata": {}, + "source": [ + "## Exercise 5 - Comparing your code with sklearn" + ] + }, + { + "cell_type": "markdown", + "id": "8f595b7a", + "metadata": {}, + "source": [ + "When implementing different algorithms for the first time, it can be helpful to double check your results with established implementations before you go on to add more complexity." + ] + }, + { + "cell_type": "markdown", + "id": "8ab310c1", + "metadata": {}, + "source": [ + "**a)** Make sure your `polynomial_features` function creates the same feature matrix as sklearns PolynomialFeatures.\n", + "\n", + "(https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "85b964d1", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "73c32c52", + "metadata": {}, + "source": [ + "**b)** Make sure your `OLS_parameters` function computes the same parameters as sklearns LinearRegression with fit_intercept set to False, since the intercept is included in the feature matrix. Use `your_model_object.coef_` to extract the computed parameters.\n", + "\n", + "(https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "35b04126", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.15" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/doc/LectureNotes/_build/jupyter_execute/exercisesweek36.ipynb b/doc/LectureNotes/_build/jupyter_execute/exercisesweek36.ipynb new file mode 100644 index 000000000..843407984 --- /dev/null +++ b/doc/LectureNotes/_build/jupyter_execute/exercisesweek36.ipynb @@ -0,0 +1,383 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "9dd8f8d0", + "metadata": {}, + "source": [ + "# Weekly Exercises 3\n", + "\n", + "## Deriving and Implementing Ridge Regression" + ] + }, + { + "cell_type": "markdown", + "id": "e5cc5739", + "metadata": {}, + "source": [ + "## Learning goals\n", + "\n", + "After completing these exercises, you will know how to\n", + "- Take more derivatives of simple products between vectors and matrices\n", + "- Implement Ridge regression using the analytical expressions\n", + "- Scale data appropriately for linear regression\n", + "- Evaluate a model across two different hyperparameters" + ] + }, + { + "cell_type": "markdown", + "id": "427d9b9d", + "metadata": {}, + "source": [ + "## Exercise 1 - Choice of model and degrees of freedom" + ] + }, + { + "cell_type": "markdown", + "id": "ff6ebdb5", + "metadata": {}, + "source": [ + "**a)** How many degrees of freedom does an OLS model fit to the features $x, x^2, x^3$ and the intercept have?\n", + "\n", + "**b)** Why is it bad for a model to have too many degrees of freedom?\n", + "\n", + "**c)** Why is it bad for a model to have too few degrees of freedom?\n", + "\n", + "**d)** Read [chapter 3.4.1 of Hastie et al.'s book](https://link.springer.com/book/10.1007/978-0-387-84858-7). What is the expression for the effective degrees of freedom of the ridge regression fit?\n", + "\n", + "**e)** Why might we want to use Ridge regression instead of OLS?\n", + "\n", + "**f)** Why migth we want to use OLS instead of Ridge regression?" + ] + }, + { + "cell_type": "markdown", + "id": "a78b2ce2", + "metadata": {}, + "source": [ + "## Exercise 2 - Deriving the expression for Ridge Regression\n", + "\n", + "The aim here is to derive the expression for the optimal parameters using Ridge regression.\n", + "\n", + "The expression for the standard Mean Squared Error (MSE) which we used to define our cost function and the equations for the ordinary least squares (OLS) method, was given by the optimization problem" + ] + }, + { + "cell_type": "markdown", + "id": "93a7ad16", + "metadata": {}, + "source": [ + "$$\n", + "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in {\\mathbb{R}}^{p}}}\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\\right\\}.\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "6b182c01", + "metadata": {}, + "source": [ + "By minimizing the above equation with respect to the parameters $\\boldsymbol{\\beta}$ we could then obtain an analytical expression for the parameters $\\boldsymbol{\\hat\\beta_{OLS}}$.\n", + "\n", + "We can add a regularization parameter $\\lambda$ by\n", + "defining a new cost function to be optimized, that is" + ] + }, + { + "cell_type": "markdown", + "id": "fb065daf", + "metadata": {}, + "source": [ + "$$\n", + "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n", + "{\\mathbb{R}}^{p}}}\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2+\\lambda\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "89ae11e8", + "metadata": {}, + "source": [ + "which leads to the Ridge regression minimization problem. (One can require as part of the optimization problem that $\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\\le t$, where $t$ is a finite number larger than zero. We will not implement that in this course.)" + ] + }, + { + "cell_type": "markdown", + "id": "456d58d4", + "metadata": {}, + "source": [ + "### a) Expression for Ridge regression\n", + "\n", + "Show that the optimal parameters" + ] + }, + { + "cell_type": "markdown", + "id": "a5677efd", + "metadata": {}, + "source": [ + "$$\n", + "\\hat{\\boldsymbol{\\beta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "e7b08048", + "metadata": {}, + "source": [ + "with $\\boldsymbol{I}$ being a $p\\times p$ identity matrix.\n", + "\n", + "The ordinary least squares result is" + ] + }, + { + "cell_type": "markdown", + "id": "8bb0e6af", + "metadata": {}, + "source": [ + "$$\n", + "\\hat{\\boldsymbol{\\beta}}_{\\mathrm{OLS}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "2faaae78", + "metadata": {}, + "source": [ + "## Exercise 3 - Scaling data" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "8ed79466", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "from sklearn.model_selection import train_test_split\n", + "from sklearn.preprocessing import StandardScaler" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "id": "59d6f347", + "metadata": {}, + "outputs": [], + "source": [ + "n = 100\n", + "x = np.linspace(-3, 3, n)\n", + "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2) + np.random.normal(0, 0.1)" + ] + }, + { + "cell_type": "markdown", + "id": "c9fc9fe0", + "metadata": {}, + "source": [ + "**a)** Adapt your function from last week to only include the intercept column if the boolean argument `intercept` is set to true." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "id": "5aa9c86d", + "metadata": {}, + "outputs": [], + "source": [ + "def polynomial_features(x, p, intercept=False):\n", + " n = len(x)\n", + " X = np.zeros((n, p + 1))\n", + " #X[:, 0] = ...\n", + " #X[:, 1] = ...\n", + " #X[:, 2] = ...\n", + " # could this be a loop?" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "id": "2ebf249a", + "metadata": {}, + "outputs": [], + "source": [ + "def polynomial_features(x, p, intercept=False):\n", + " n = len(x)\n", + " X = np.zeros((n, p))\n", + " X[:, 0] = x[:]\n", + " X[:, 1] = x**2\n", + " X[:, 2] = x**3\n", + " return X" + ] + }, + { + "cell_type": "markdown", + "id": "c5924634", + "metadata": {}, + "source": [ + "**b)** Split your data into training and test data(80/20 split)" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "id": "676541e7", + "metadata": {}, + "outputs": [], + "source": [ + "X = polynomial_features(x, 3)" + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "id": "f895606f", + "metadata": {}, + "outputs": [], + "source": [ + "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n", + "x_train = X_train[:, 0] # These are used for plotting\n", + "x_test = X_test[:, 0] # These are used for plotting" + ] + }, + { + "cell_type": "markdown", + "id": "a58f05ad", + "metadata": {}, + "source": [ + "**c)** Scale your design matrix with the sklearn standard scaler, though based on the mean and standard deviation of the training data only." + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "id": "0c07995a", + "metadata": {}, + "outputs": [], + "source": [ + "scaler = StandardScaler()\n", + "scaler.fit(X_train)\n", + "X_train_s = scaler.transform(X_train)\n", + "X_test_s = scaler.transform(X_test)\n", + "y_offset = np.mean(y_train)" + ] + }, + { + "cell_type": "markdown", + "id": "63b86003", + "metadata": {}, + "source": [ + "## Exercise 4 - Implementing Ridge Regression" + ] + }, + { + "cell_type": "markdown", + "id": "a2f52ef2", + "metadata": {}, + "source": [ + "**a)** Implement a function for computing the optimal Ridge parameters using the expression from **2a)**." + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "id": "69f19bde", + "metadata": {}, + "outputs": [], + "source": [ + "def Ridge_parameters(X, y):\n", + " # Assumes X is scaled and has no intercept column\n", + " return np.linalg.inv(X.T @ X) @ X.T @ y\n", + "\n", + "beta = Ridge_parameters(X_train_s, y_train)" + ] + }, + { + "cell_type": "markdown", + "id": "626c81ba", + "metadata": {}, + "source": [ + "**b)** Fit a model to the data, and plot the prediction using both the training and test x-values extracted before scaling, and the y_offset." + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "id": "f9ba0655", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 82, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAXxtJREFUeJzt3Qd409X6B/Bv0r2hm0JpS8tGyixbQVmKuK8IKIjryhUX/3sv4ICLC/V6Fa8gXvdEwAWoyF4yy1Q2FAqFlm66IW2T/J9z0pSOtE1Kdr6f58nTX9JfmtNQkjfnvO97FFqtVgsiIiIiG1Ha6oGJiIiIBAYjREREZFMMRoiIiMimGIwQERGRTTEYISIiIptiMEJEREQ2xWCEiIiIbIrBCBEREdmUOxyARqNBRkYGAgICoFAobD0cIiIiMoLoq1pcXIyoqCgolUrHDkZEIBIdHW3rYRAREVEznD9/Hm3atHHsYETMiOh/mcDAQFsPh4iIiIxQVFQkJxP07+MOHYzol2ZEIMJghIiIyLE0lWLBBFYiIiKyKQYjREREZFMMRoiIiMimGIwQERGRTTEYISIiIptiMEJEREQ2xWCEiIiIbIrBCBEREdkUgxEiIiKyKQYjREREZFMMRoiIiMimGIwQERGRTTnERnlERESNUVWqcSCtAMmp+fByVyIxugWuax0EPy++zTkC/isREZFDyi6+gp/2p2P76TzsSc3H5Qp1re8rFUCHiAD0immJx6+PR9sQX5uNlcy8TLN161aMHTsWUVFRckvg5cuXN3kflUqF559/HjExMfDy8kJsbCw+/fRTUx+aiIhIWnMkEyPf2Yp5vx3H1pM5MhAJ9ffErd1b4eZukWgV5A2NFjieWYzFu9Nw87tbsXRPGrRara2HTuaYGSktLUViYiIeeugh3HXXXUbd595770VWVhY++eQTJCQk4OLFi9BoNKY+NBERubjL5Wq8/OtRGWAInSIDcE/vNhjcPhQdIwLkh2S9rKIr+ON8AT7eliqXb2b8cAjrjmZh3l3dERbgZcPfgupSaK8hTBT/6D/99BPuuOOOBs9ZvXo17rvvPpw5cwbBwcHNepyioiIEBQWhsLAQgYGBzR0uERE5sCMZhXjq2wM4nVMqr//1+nb4v5Ed4ene+CS/WqPFJ9vO4K01J1Gu1iDEzxNv3tMdN3WOsNLIXVeRke/fFq+mWblyJfr06YM333wTrVu3RocOHfD3v/8dly9fbnRZR/wCNS9EROS6tp3KxZ0Ld8hAJDzAC18/3A+zbuncZCAiuCkVeOz6eKyYNkjOpOSVluOxr/Zhy8kcq4ydmmbxYETMiGzbtg2HDx+Wsyjz58/H999/j7/97W8N3mfevHkyktJfoqOjLT1MIiKyU8cuFuHxr/fJWY2hHcOw+pnr5bKMqTq3CpQBye09ouRsyRPf7MeJzGKLjJnsLBgRuSFiOeebb75BUlISbrnlFrz99tv44osvGpwdmTVrlpzS0V/Onz9v6WESEZEdulh4GVM+24MSVSX6xQXjfw/0RrCfZ7N/npe7m1yiSYoLlj/zoc/3yKoccvJgpFWrVnJ5Rsxw6HXu3FlmNF+4cMHgfUTFjVhbqnkhIiLXUnSlAg9+ugeZRVfQPtwfHz7QRwYT10r8jP/d3xtxoX5IL7iMR7/chyt1yoLJyYKRQYMGISMjAyUlJdW3nTx5EkqlEm3atLH0wxMRkQMqr9Rg6tf7cCKrWFa+fDalL4J8Pcz281v6eeLTB/uiha+HrLiZvuwgNKIWmBwjGBFBxcGDB+VFSE1NlcdpaWnVSyyTJk2qPn/ChAkICQnBlClTcPToUdmn5B//+IcsDfbx8THn70JERE5i9orD2J6SBz9PN3z2YF+0aWn+hmViZkTMkHi4KbDqUCYWbTlt9scgCwUje/fuRc+ePeVFmD59ujyePXu2vC56iOgDE8Hf3x/r1q1DQUGBrKqZOHGibJr23//+19SHJiIiF7DxeBaW7DkvO6gunNgL3VpfXeY3t37tQvDKHd3k8X83nEJaXpnFHoss1GfEWthnhIjIdfJERr69VeaJPDokDs+P6WLxxxRvg/d/slvOxIhqHTETU7N5GjlBnxEiIiJjvfbrMRmIxIb4YvqIjlZ5TBF4vHR7N3i6KbH5RA5WH860yuPSVQxGiIjILvx+Kkcuzwhv3pMIH89rr5wxVnyYP/56Qzt5PPfno7Lsl6yHwQgREdmcePOf+cMheTx5QIzsA2JtTwxLQNtgXzkz8+76k1Z/fFfGYISIiGzujd+Oy54fbVr64J+jO9lkDN4ebph7e1d5/On2s7LzK1kHgxEiIrKpPWfz8dWuc/L4jbu7w8/L5A3lzWZYx3CM7hop28W/sPywTG4ly2MwQkRENiPe7F/99Zg8HtcnGoMSTN9zxtzm3NYFvp5u2HfuEjYcy7b1cFwCgxEiIrIZUbly8HyBfPP/v1EdbD0cqVWQDyYPjJXH721K4eyIFTAYISIim6hQa/DmmhPy+JEh7RAe4A178fDgOHh7KGWr+G0pubYejtNjMEJERDYhynhTc0sR4ueJx67XldXai1B/L0xIipHH721MsfVwnB6DESIisrpSVSXeXX9KHj91U3v42zBptSEiQBKN0JJT87H7TJ6th+PUGIwQEZHVffx7KnJLVIgJ8cX4pLawR5FB3vhLH93u8gs2cXbEkhiMEBGRVYkg5MOtuh1y/zGqIzzd7fet6PEb4uGmVOD3U7kyf4Qsw37/AoiIyCmJ3XFLy9VIbBOEMde1gj2LDvbFnT1by2POjlgOgxEiIrKai4WX8W1ymjyeeXNnh9gdd+rQeIhhrjuaxa6sFsJghIiIrOajramoUGvRv10wBsSHwBGITfT0MzgfbNEtL5F5MRghIiKryC8tr54V+dvQBDgSkTsi/HYoE3klKlsPx+kwGCEiIqv4fMdZXK5Qo1vrQAxpb/u276bo1joIidEtUK7W4Pt9F2w9HKfDYISIiCyuRFWJL3acrZ4VcYRckbomVpUgL05Og0bDFvHmxGCEiIgsbvHucyi8XIF2YX4Y1TUSjujWxFYI8HbHubwybD/NFvHmxGCEiIgsSlWplk3OavbtcES+nu64u5euCdo3u3S5L2QeDEaIiMiiftiXjuxiFVoFeeOOHrqeHY5qQj/dUs26Y1nIKrpi6+E4DQYjRERkMZVqDf5X1W310SHt7LrbqjE6RAQgKTYYao0WS/ect/VwnIZj/1UQEZFdW3U4U+ZYBPt54r6kaDiDif11syOiTFkEW3TtGIwQEZHFfLJNlysyeUCszLlwBqO7Rcrg6mLhFWw+kWPr4TgFBiNERGQR+9Muyc3lxNLM/VWzCc7Ay90N9/SuSmTdfc7Ww3EKDEaIiMgiPt+u6ytyW2IUQvy94EzGV/Uc2XwyB+fzy2w9HIfHYISIiMwus/AKVh26KI+nDIqFs4kL9cOAdiHQaoEVB9NtPRyHx2CEiIjM7utd51Cp0SIpLhhdo4LgjO7sqStTXn4wA1oRlVCzMRghIiKzulKhli3ThYeccFZEb/R1kTIfJiW7BEcyimw9HIfGYISIiMxq5cEMuUNv6xY+GN45As4q0NsDN3UKl8dcqrk2DEaIiMhsxHLFZ1Ub4k0aEAN3N+d+m7m9qqPsyj8yZCM0ah7n/ishIiKr2p2aj2MXi+Dj4Yb7+jpPOW9DhnUKQ6C3O7KKVNh9Js/Ww3FYDEaIiMhsPtuua3J2V6/WCPL1gLMTPUduua6VPF7OpRrrBSNbt27F2LFjERUVBYVCgeXLlxt93+3bt8Pd3R09evQw9WGJiMjOpRdcxrqjWfL4wYHOm7ja0FLNb4cyZfIuWSEYKS0tRWJiIhYuXGjS/QoKCjBp0iTcdNNNpj4kERE5gG93p0GkTQyMD0H7iAC4in5xwXJH4mJVJTafyLb1cFwjGLn55pvxyiuv4M477zTpfo8//jgmTJiAAQMGmPqQRERk58orNVhStYvt/f1j4EqUSgVu6xElj5cfyLD1cBySVXJGPvvsM5w5cwZz5swx6nyVSoWioqJaFyIisiMaNZD6O3Doe/l17eEM5Jao0MLXQy5V7DydJ6tLxEUci9JX8VUELTWvGzrHEatS7qhaqtl4PBuFlytsPRyHY/EtFE+dOoWZM2fi999/l/kixpg3bx7mzp1r6aEREZEpwce5HUBJFpB3Gtj3GVCsa/cufFn5EoAEFJRVYPqyP+RtIjARxG16SgXkUo6eoXNaB3rgnf5lSAqrBPwjgJiBgNIN9qxzq0B0jAjAiaxirD58EeNcoJLIYYIRtVotl2ZEYNGhQwej7zdr1ixMnz69+rqYGYmOjrbQKImIqFFHVwKrZwBFhpcgUjRRSK5MgBIaDFb8ia3aHvUCDL26kx51zxmlTMYc1ZeI2pp/9cbAKGDUPMA3RBcM2WmAcnvPKLy5+gRWHMxgMGJPwUhxcTH27t2LAwcOYNq0afI2jUYjm+KIWZK1a9fixhtvrHc/Ly8veSEiIusTyyTJqfnILr6CTpc2o8OWJ6BAw0snX6uHy683KvfjJY8vMFj1LjTNyAIQgcgij/n1vyGCoO8m175NBCij3wC63AZ7Mba7LhjZdSZPdqAN9vO09ZAchkWDkcDAQBw6dKjWbe+//z42btyI77//HnFxcZZ8eCIiMjH4OJtbhm+T05BZdEXOdGzzegFahRaKBu5bpvXCD+oh8vh+t/WIUuQhSXkcuzRdTBqDeKw5Hl/qjht6sBq0RReBZZOwJ2k+1B3Hyg353Iy5owVFB/uia1Sg3Kdm/dEs3NuXM/oWC0ZKSkqQkpJSfT01NRUHDx5EcHAw2rZtK5dY0tPT8eWXX0KpVKJbt2617h8eHg5vb+96txMRkfWJ/Ia5Px/FxcIr9b4ngoooRY3lEgN+Vg9AMfzQVpGF65W6D5/hKDB5HMY8Vk1ipkYs+bTZ/RIGbw1BRJAv5oztgtHddA3IbOXmbpEyGPnt8EUGI5YMRsSyy7Bhw6qv63M7Jk+ejM8//xwXL15EWpput0YiIrLnWZBSvLP+VIPnGhNU6JdoJrhtgFKhW8rJRguTx9WcAEZMhERBNxOzu7ALpn69H88M74DYUF+EB3jbZLZkdLdIvLX2JLan5KHoSoXcTI8sEIwMHTpU5nw0RAQkjfnXv/4lL0REZD+zIIY0FVT8oWmHQ9p28EQF/uK2Rc5UZCIEyZpOJo+tOQFMzUBG/670zvqT1beLRmTWni1JCA9AfJgfTueUYtPx7OrurNQ47k1DROTE9D08Xv75CB7/er/RgYgggooMbXC9Chi9b9S6jtq3KHejJYrl8dyKB5qVvNrUYzUnkMksvCJnS0QQZu3ZEWHNkUyrPq4jYzBCROSkxJvw4Dc2YvxHu/DJ9rMm318EFXMrJumO6wQJRVofmS8iTHDfAJVvJJ7z+CfWaJKqz2np61HdR0Sv7qqJ/pxaj2Xs+LRAhrbhmRht1WXmD4ewPSXXas3Ubq6aidl0PAeXy7lXjV00PSMiIvvLBzGWCC6mVjwjK12icDXBdIXHGFxWeSMhUIO+970FRewgvAolbq96fH3OhpBc47beMS2x79ylBs7pgVOXOqDDgVca7Gmip48rjJmJKbhcgYkf77baso2oqGndwkduHLj1VA5GddXNlFDDFNrGEkDshGh6FhQUhMLCQlkuTERE15YPYgpFVenttyPVsiuq1i8cN68EjmcWY/atXfDQ4DjLdXsVDc5K84C1s2oFKGJGRAQiNWdijPk9hEX397J4QPLyL0fxybZU3NWzNd4e57o71RcZ+f7NmREiIicIRERuhKU+WUZWzSgkVb2BH0y7hOOZO+DprsRdvSyQoCk6q8bpepdU6zJWBih/HDuOedsKsEfTCWoTMw3E8+MGDVauWIaRmigoAyIt1slVlPiKYGTdsSy5H494rqhhDEaIiBx4WWbX6TyZE2HOQCQy0Avjk9oiNtTPYIns4t269g23XtcKLXyt1GW0KkBJjBuCB9texLlmzALJVvNiuakiH/jRsp1ce7VtibAAL+QUq7DzTB5u6BBm1p/vbBiMEBG58LKMPsQwtj+H6J3x85+65ZLx/Wyz/4pYYhnRJbJefowYsdbkVvO6Tq6490uzBiRKpQIju0Tgm91pWH04k8FIExiMEBG58LKMfgnG2ByKFQfScaVCg/bh/ugT0xK2IoKlAfEh1dc7RgY0GJw13mpeC9HsvvyXf2K1qgfCA/3M1ixNVNWIYGTd0Uy8ckc3m7ert2cMRoiIHEFVUqemOBMrf86AAu2gbUZ3Bv3swUODYuXsgilvvKLeQby5CmIZR6GwnzdX/WyJWLZ6YvF+WUFjbKt50Vreq+wivv1uqdxTx1xVN/3aBSPIxwO5JeXYezYf/dpdDZ6oNmbUEBHZu6MrgfndgC9uhfLHR/B+xWxs83pKLj00Zybkg/t7YfbYrnJmwZRP6wfPF8gKGi93Je7u1Qb2Rvwug9qH4vW7r5NBl8LEVvP688zVLM3DTYmbOoXL4w3Hs6/pZzk7BiNERPYeiIichjp9NyKRL3MgGgtI9G/Izw7vgHfv64FvH+2PbTNubPYnfn3i6pjurRBUp5mZPRG/nyjfFYGXKa3m9efpl7/Ess+1Nkq7qXOE/LrhWNY1/Rxnx2UaIiJ7bV5WVIrR6/4BT5nVUJuY0BDvk3M8vsI6VR+Djb9MzQdpTM3E1QlJtklcbXaSa9F1UK37GJ5lWXJJpq66e+qIHBOxtBNeXIDjO6+g64DRzS7/vb5DKNyVCrlXjUi0FRVKVB+DESIiO62S6a88its9M43atVbkOui18PHAwom90L+dacswjVleI3FVdFF1BFeTXEMAr3/rZpjq1NzU7eRaXf6rzzFZtwDY3fzy3wBvD5k7InbxFUs1D5u7QZyT4DINEZGdVcnoK0JMzXXQL8uInIlBCaFmC0Rk4uou3RLNxH72lbhqNBFIiPLdwNqzRGJGRLS7F51c9eW/YgnMYPmvWDJrhhs76ZZqNh7nUk1DODNCRGQnSzNiRqTmIoKpuQ7mXJapaX/aJZzIKoa3hxJ32mHiqkkBSacx1VVJ037OwJridrKTa1PlvzLMWz1Td38Tl2xEEqtoD7/7TL5c7gr0tt98G1thMEJEZAeByOfbU+v1yBA5DBnaYPlJ3dAkh+yP4RuJ8bePw9Nm7I9Rl76c99buUbJU1aFVdXIVywK3KS/it6/3y9mkpsp/ZUBSlI5tG1bCrd31Jj3XIk8kPsxP5o38fjJXJgBTbVymISKy8dLM4Dc24uVfj9X7nshhmFsxqVZuw1UK+SbqdeubuL1nW5PLdI1VWFaBX//UlbhOsFHHVWtU3Ri7JLZs016M/2iX/DczpfS3uqqGSzUGMRghIrKTHBFDRC6DyGnIRHDtb4g9VczcwtyQH/ZfgKpSg06RAegZbdyykaMFJKLc+a9jBpq0JGZqL5Ibq/qNbD6Rc83lws6IyzRERDYo280svCxnQ4x5WxIByXpVH4wKOIMFYy2722zdxNXFyQ6euGoEMaMky3dF1YxIVjWi/Lcqi0Tm+YgS4qZmpUTr/EBvd+SXluPg+UvoHVMnuHRxDEaIiOx8cztF1ZLNbbffC6WZk1Mbs+fsJaRkl8DHww2392wNpyYCO1G+a0T5r564WfxbiuCy5j45hri7KTG0YzhW/pGBDceyGYzUwWUaIiI7WZJpiMhpELkN5q6Sacri3efk19sSo1yjAsSI8l9DtqcYt/RyU+eq1vDH2Bq+Ls6MEBHZoGzXWC+O6YwHB8VZfcfXS6XlWHU40ykTV40t/z15OgWzN+bKpRlDHW71Fmw6jR/2pzdZVn1DhzD57yjKpM/nlyE62NdCv4Tj4cwIEZENynabIkIPsXusLQIRfeJqeaUGXaMC0b1NEFxKVflv/I0P4lxAL6N2RzYmobWFr2d199pNJzg7UhODESIiG5TtNkYfeohP2rYIRDQabXVvkQlOnLjaFPHci38DoalnwNjN9fS7+K7nUk0tDEaIiCzAEXNE9LafzkVqbikCvNxxRw8nT1w1cQfgxtRMaG0qb2TX6TyUlVeadayOjDkjRER2kCMS7OeBF2/tishAb4t1UjXWVzt1iat3924DPy++Teh3AH5n3Uks2JTS4Hn63X49jqUDys4Gy6/jw/zRuoUP0gsuY9eZvOp9a1wd/8qIiMzcQ0RUVxg7I6IPOV678zqbzYTUJN4k1x/TdQm9v78LJa42QQSHYvPBhoKRWrv97oXuElh/t1+x5DW0Y5hcBttyIofBSBUGI0RENuwhYqnN7Zrr291psq/GwPgQJIQH2Ho4dkXMWImkYpGsWnPWS7/bbz363X7rdMoVVTUyGDmZY52BOwDmjBAR2Sg/RJTtilbk9hKIqCrVWLJHl7j6QP8YWw/HIRJam97tV/yBzAQ06upbByaEwsNNgbN5ZTibW2qVsds7BiNERFbOD7F12W5DVh/ORG5JOSICvTCiC5cPjElo1e/22/A/o263X9G3RM/fyx19qjqwcnZEh8s0RERW7CFi67JdYxJXJyTFyPbl1HhCq8gPksmqIj+kKSW1d+u9oWMYdp7Jw+YT2Zg8MBaujn9tRERW7CFi67LdhhzNKMLec5fgrlRgfFK0rYdj90QgKfaj6dO1s1HnHynyqdV/RCSxCiIguVJxdQnHVZkcjGzduhVjx45FVFSUzApevnx5o+f/+OOPGDFiBMLCwhAYGIgBAwZgzZo11zJmIiKHyxGZNiwB3z7a365yRGr6apduVmRUt0iEBzbdU4OqiPJdUTXTQFs0EX9kaEMw9meNDF71HVo7RgTI5bArFZpG+5K4CpODkdLSUiQmJmLhwoVGBy8iGFm1ahX27duHYcOGyWDmwIEDzRkvEZFD5Yjo80OeHdFBfpK2t6UZoehKBZYfSJfHTFxt5m6/Uu1/27q7/dZsGS8+zIuqGmEL80ZMzxm5+eab5cVY8+fXLnd67bXXsGLFCvz888/o2bOnqQ9PRGRT4lOsqT1E7DE/pKZle87jcoUaHSL80S+OW9s3e7ff1TOAooxau/2KQES/26+ITcRfgQhmRc7J0I7hWLb3ggxGXoRrs3oCq0ajQXFxMYKDG/6DV6lU8qJXVFRkpdERETVAlGae2wGPY8fQX1nY5E6u9thDxJBKtQafbT8rj6cMinPZfWjMtdvvkZ2r8b9fdyAbLQz+jdRsGS+aqIkgNSW7BBculaFNS9fdxdfqwchbb72FkpIS3HvvvQ2eM2/ePMydO9eq4yIiatDRldWfevsAWOIp8gCCMbdiUvWnXkM9ROytdNeQNUeyZNfVYD9P3NnTtfehuWZKN6T49cBKTdOnZhdfwQCfEPRq2wJ7zl6SsyMT+7nuEplVq2kWL14sg4xly5YhPFy3WZAhs2bNQmFhYfXl/Pnz1hwmEVHtQER00awx/S5EIl923RTdNx2hh0hDPtl2Rn69v19beHvU3keFTBceYFzy76msYuw8nYch7XV5I5tPuHbeiNVmRpYsWYJHHnkE3333HYYPH97ouV5eXvJCRGTzpRkxI2IgXVXEGSJBcY7HV1in6iOn4x0lR0Rvf9ol7E8rgKebEvcPcN1P5dZoGV/Xgk2n5SXU31Ne35GSi/JKDTzdXbPjhlV+62+//RZTpkyRX8eMGWONhyQiunaia2adGZGaRLwRpciTXTjtuYdIQz7Zliq/3tYjyuhP9GR6y/jGiI63Qmm5GnvPuW6Jr8kzIyLfIyXl6q6FqampOHjwoExIbdu2rVxiSU9Px5dfflm9NDN58mS8++676NevHzIzM+XtPj4+CAoKMufvQkRk1hLe06dT0MGIc/8+MAgVnfvLT8WOMCMiiIRJ0f5deGhQnK2H45Qt403dOHHziRwMjA+FKzJ5ZmTv3r2yJFdfljt9+nR5PHv2bHn94sWLSEvTbbQkfPjhh6isrMQTTzyBVq1aVV+efvppc/4eRERm7646e2OuUeeLLpz22kOkIV/sOCsDLrE7b5eoQFsPxykDEtHgTjS6Ew3vjLH2iC44dEUmz4wMHToUWm3DK2Gff/55reubN29u3siIiGzYXVW8ymWhk6yaEcmqhuMMha77pujC6UBKVJVYkqwrDHhkCGdFLN0yXlTOGONsXhlyS1QI9Xe9nEnXzJQhIjKiu6pIShXlu/K43mewquhk9Ou6LpwO5Lu951GsqkS7MD8M7dBwZSOZhyn5OL+7aDdWBiNE5PJEECLKLN9Zd7LeGr/oIzK14hlkok6jRjEjIrpuimZXDqRCralOXBVNzpQOtLTk6BU2xjzTz/10qHr/Gldi9aZnRET2RLzwN5VoKAISUb4rqmbCUYB7h/XB4Jtuc7gZEWHFwQxcuHRZlpTe06uNrYfjUhU2YvlPBCSNlfxertDg8a/34wMHqsoyB86MEJHLMmX3XbFks0vTBSs1A+HW7nqHDETEDND7m3TVkI8MaQcfT8f7HRy9wkaUfxtj7s9H5b+Xq2AwQkQuydTdd2t2VxXT7o7o10MXcSa3FC18PXA/d+e1WYWN2CqgKRer9q9xFQxGiMglmbL7ruBo3VXr0mi0WLgxpbqviL8XV+ltQfzthAYYVy2TbWQVjjPgXyMR2QXxwns6uxRl5ZUoK1fjcrkaqko12ob4IbFNEFr46tpmm2NGRAQiv5mYJOgIO/A2Zt2xLJzIKkaAlzsmD4y19XBcmrHVNeEu1BWXwQgR2UxaXhnWHMnE6iOZcp+URloYIS5UF5T0iQ2W7csDvT0skqxa17Rh8RiUEOZQ3VXrEr2h3tt4Sh6LQCTIx/Tnjqy7f02wn4fDLgc2B4MRIrL6G+Pao1l4d/0pHL1YVC/gCPB2h4+HG3w93eDupkRKdglSc0urL8sPZmDeqmP4S59oPDgwFrGhfiY3MzOGomo25NkRHR02CNHbfDIHh9OL5HP60GA2OXOE6pq4UD/88meGnB1x5EDYWAxGiMiqMyFzVh7Gpqrt0sULbL+4YIzuFomRXSIbrDQoKCvHHxcKcTCtQL5An8ouwec7zuKLnWdxU6dwTLuxPXpEtzBbsqqj54fUmxXZoJsVEUmrwX7mWe4iy+xf4+vpJpcp950rwL5zB+VtrRx8idAYCm1jvd3tRFFRkdxUr7CwEIGB3EOByNFcqVDjgy2n8f7m03KbdA83BR67vh0eGdwOLU18cxQvWdtScvHpttTqoEahACb2a4t/ju5kcPlGNDQb/9Euox/DmV78t53Kxf2f7IaXuxK/zxjmUnkIjkCfw5RdfAVnc8vwzvqT9c7Rh8OOtCO0qe/fnBkhIos6n1+GKZ/vkcstwuCEUMy9vSviw/yb9fMUCgWGtA+Tl9M5JbJC5McD6fh6VxrWHMnCv8Z2xS3XRcrz9C/221OM2/Bu0oAY3NytldNMi4sKmnm/HZPHE/q1ZSBix/vXqDVauTmjIWLGQPw1ilmUEV0ineJvsy4GI0RkMUcyCvHgZ3uQU6xCeIAXZo/tgjHXtaoOFK6VCGjeHtcD9/Rpgxd+Oix7aDyxeD+GdQzDG3d3l0mxpiSsikBEvDE4ixV/pONIRpGsoHnyxva2Hg5dQ6m5tkbvEWf6G9VjMEJEFiFmI/761T65Q2ynyAB88VASIgIt88l8YHwoVj09BIs2n5YXsXwz/O0tKLpSaVKyqjNVL4ilsbfW6Kb8pw6LZ66Incs2sqeIs/YeYdMzIjK7FQfT8eBnyTIQ6d8uGMseH2CxQETP28MNz47ogFVPD0aHCH+TAhFnSVatSST4phdcRlSQt2xyRvYt3MV7jzAYISKzWpKchqeXHESFWosx3VvJGZHm9ARproTwAMy6uel223piRsQREwMbk19aXt1t9f9GdpSBGjn2zr4KACF+nsgsvCwTsp1t3xou0xCR2fx+KgfPLz8sj0UPkNm3drHJFvVFVyqMbmjmDH1E6vrvhlMoVlWiS6tA3Nmzta2HQ2boPaIFkFdajmeX/eF0FV8CZ0aIyCxSsovxt2/2y09sd/VqLV8orR2IiMcWnxpPZekqd5oiOqs6WyByNrcUX+86J4+fH9PZJsEgWX5n38zCKzJwEc38nAFnRojomuWVqGT5bvGVSiTFBmPeXdeZrWLGUq3eHXn33ca8sfo4KjVaDO0YhkEJobYeDjUjIBnRJRK7z+Thr1/vk/+nXKHclzMjRHRNxGZ2omrmfP5lxIT44oMHesPL3bo5CvpW76bsOSM6vzr463c9649m4bfDmfL3mnlzJ1sPh5rJTanAwIRQJLZpuKtw3XJfR8dghIiaTXRDnfnDIew9d0nuKfPJ5L5WLyE1tdW7n6cuUBJ73Lyw/LDTJAIWXq7A88sPyeNHh7RDp0h2q3Z0MSG+LlPuy2CEiJpt2d7z+OlAuvwkt2hibySEN6+rqiWbRdVMVv320f7481+jZD6LWEX6Zncanvhmv+zJ4ejE5oFZRSq5wZoocSbHNzA+1GXKfRmMEFGzN7176eej8vgfozpicHvr5yfoWr3r9qdpSvuIANm5UgROUwbFYcH4XvB0U2L1kUzZE8XYChx7bTC3ZM95eSw6z7KU1zmM7hYJ90bWEhVOlPvEYISImhUE/N93B1FarpYJq2JZwNpEnojYy2PBptPN+vQoeqB8PqUv/L3csetMPsb9bxeyixxvurtUVYmZP/5ZvbeOM7wxkY4InK/vEOoSzfoYjBCRyT7+/Qz2nL0k8y/+c2+i1V8MTUlYbezTo0gSXPJYf4T6e+HYxSLctWgHzuQYVxZsL/695oRMHm7dwkfuWkzO5f7+MfKrW53qtIhALzwzvD1UlRqnaILG0l4iMol40/7PWt2eJ2Lju+hg45LsbJGwasynx26tg/Dj1IF44NPdOJdXhrsX7cAnD/ZFr7YtYe9EvswXO8/KY1FOLWZ5yLn0iwuBh5tCdjSeP66HzHU6m1uGb5PT8M76U9XnOXoTNM6MEJFJZbzPLj2IcrUGwzuH494+0XabsGpKq/e2Ib74YepAdG8ThEtlFZjw0S5ZJmvPMgouyyZzWi3wl95tcH2HMFsPiSzAz8sdPasCY7HXk5e7EvPXn0RmnSVFR2+CxmCEiIz23oYUHM8sluW78+7qbvXGZqaUMU4bloBtM240+pOiWKoR1TaiWdiVCg0e+2ovFu9Ogz26XK6W48stUckdkf91W1dbD4ksaEhV8zqx3UJDs4L628T3HXHJhsEIERklNbcU/9uqSxZ99Y5uCAvwsu4ANGog9Xd0zl2L/sqjUELT6Omi+6ipuSziU+hHk/rg3j5tIF7Pn/vpkOxoqrGjF3fR2+WfP/yJw+lFMigU4xXjJuc1uKpSbdup3EZnBR25CRr/gonIKC/9fESuW9/QIUyWHFrV0ZXA6hlAUQZEB40lnkCGNhhzKyZhjSap1qmKquWZ5laVeLgpZXlsZKA3/rsxBYs2n8aprGK8M64HAqy4+3BDFm05jZ//yJAln4sm9rJ6zg5Z33Wtg2RTwYZawztDEzTOjBBRkzYcy8KmEzkykU7XMExh3UBk2SQZiNQUiXws8piPUcpks5c7it9v+siOeGdcIjzdlVh/LBt3vb8D5/JKYUsij0VUzwhiaaZfuxCbjoesw91NiYHxxv9bO2ITNAYjRNQo0Z30pV90zc0eGhyHdmH+1l2aETMiBlbJZayhAOZ4fFW9ZGNswqqx7uzZBt/9dYAsozyVXYLbFmyXDcZsYcvJHDy15IBMWJ3Yr211ySe5hsHtdQnKnm6K6qDbmZqgMRghokZ9si1VlryGB3jhyRvbW/fBz+2oNyNS9wUsSpGHL2+qlMmnpiSsGisxugVWThssv4r9Xx74ZDf+veY4yisbz1kxp58OXMDDn+9BWblaVs3MGcuEVVdNYlVrr+7YW5O4LoLyd/sVw+3IDzK/SgbzzhqMbN26FWPHjkVUVJScyly+fHmT99m8eTN69eoFLy8vJCQk4PPPP2/ueInIii4WXsaCjSny+LlbOlu1j4WoCDh5WvfYTRkcqa5u9W4JEYHeWPpYf1lCK3JZF246jTvf346TWcWwtA+3nsazS/9ApUaL2xKj8PGkPnLpiFxv07w2LX3k/4snhsXLWcCaxvkfxOGWf0fS1snADw8DX9wKzO+mW+Z0ACb/RZeWliIxMRELFy406vzU1FSMGTMGw4YNw8GDB/HMM8/gkUcewZo1a5ozXiKyotdWHcflCjX6xLTE7T2irN7qffZGI5dE/CMsPSS538u//5Iok0Zb+nrgSEYRbn1vm+xGa4lqG/EzX/nlqPw3EB4eHCebXjEQcU0KhQKDq2ZHROm5mAUUs4Hv3tcDa0YVYF7lv+FzObP2nYou6vKtHCAgUWhFnVhz76xQ4KeffsIdd9zR4DkzZszAr7/+isOHD1ffdt9996GgoACrV6826nGKiooQFBSEwsJCBAZyW2wia9h3Lh93L9opOz7+PG2w7FRqzVbv2qpp521eT8lkVcOTHgogMAp45hCgtN7mcGIPmxk//CmTevXVDtNHdsDQDmFmSe49klEoNyHcXVWi+dwtnfDY9fHX/HPJsf3yZwamLT6AjhEBWPPs9bobxVKMmAFpcDnTNv9HTH3/tniIvXPnTgwfPrzWbaNGjZK3N0SlUslfoOaFiKxHfEZ5c7WuamNcn2irBSJ1W71roJTlu/K43semqjf90a9b/UU2PNAbnz7YF6/deZ3cn+dQeiGmfLYH93ywEztONz/BNa9EhVk/HsLY97bJQER023z73kQGIiQNjA+VHw5OZBVf3dSxibwqmWFSlK47z45ZPBjJzMxERETtKVRxXQQYly9fNnifefPmyUhKf4mOtn7LaSJXti0lV74Zerop8dRN7W3a6l30EZla8QwyUadCQHzau/dLoMttsAUxAzKhX1ts/ecwPHZ9Oxk47Dt3CRM+2o17P9iJz7enIi2vzKjA70RmMRZuSsHQtzbLPUdE4HVr91bY+PehuKtXG6v8PmT/gv080TVKN7uwXR/0lhi5bYGx59mIXTY9mzVrFqZPn159XQQuDEiIrEO8Ob5V1ctClI9GtfCx2mM31KxJBCTrVH2QpDyOcBTg3mF9MPim22wy7VxXiL+XTO59ZHCcDCgWJ6ch+Wy+vPzr56OID/PDsI7hiA31k/keXlWXosuVchZl++k85BSrqn+eeLMR1TKOWJ5Jljc4IUx23/39VK4sPTc6X8oKeVV2HYxERkYiK6t2RCaui7UjHx/DL3Ki6kZciMj61h7Nwh8XCuHr6Ya/DYu32vKMmBU5lVXS4DliyWaXpos8Ht+uv10EInWXbube3g2P3RCPX/7IwMbj2dh77hJO55TidE5qo/cVwYkIPsYmRuHuXm0sVhVEjm9I+1B8sOW0bA0vPjgoYgbqZglFsqrBXWuqckbEea4cjAwYMACrVq2qddu6devk7URkX0RQ8J+1ulmRKYNi5eZx1khYFXkixuzEe62t3q2hdQsf/PWGeHkRfUnEm8a2lBzkl5ZDVamR/UnERQQcfWJbyj10erVtKat1iJrSO6alDF6zi1WyEV+HiABg9Bu6qhn5P0RrF3lVFg9GSkpKkJKSUqt0V5TsBgcHo23btnKJJT09HV9++aX8/uOPP44FCxbgn//8Jx566CFs3LgRy5YtkxU2RGRfxJ4nJ7NKEOjtjseGxFu1cqYp5mr1bk1BPh4Y072VvBCZg7eHmwzGxTKNCHRlMCLypkT+VNX+TdXEjIgIRGyUV2XRYGTv3r2yZ4iePrdj8uTJspnZxYsXkZZ2ddvtuLg4GXg8++yzePfdd9GmTRt8/PHHsqKGiOxHhVqDt9edlMfiU32Qr4dVK2eaImZERCBi7g6rRI5mcEKoLhhJyZVbNEgi4Og0Rlc1I5JV/SOgjh6A5HOFyD6YLverEUGMvQbyJgcjQ4cOletUDTHUXVXc58CBA6aPjois5ru9F5CWX4ZQf088ODDWJpUzhkwbFo9BCWF2/UJKZE2D24cCvwG7zuTJJb/qRnhiKSZuyNXlz39vqfV/rJUdB/Rs5UdE8gVtwcZT8vhvQxPgZ4W278Zuc94+IsCird6JHE3nyECE+HnKvYoOni9ocPmzbrCfWXhF3i6+b28YjBARVhxMR0bhFZmwKnpnWJpYosmtUc7qbNuhE1mSUqnAwKrW8NtO6boAG7P8qb9NfF+cZ08YjBC5OPGitGjLaXn8yJA4i1d16PedefnXY42e58jboRNZaxff31NyTVr+FCGI+L44z57YZdMzIrKetUcycSanVFbQTLTwrIix1TOOWDlDZE2DRN4IgD/OF8gSclG5Zcryp7HnWQtnRohcmEhGf3+zblZk8sBYBHh72EX1jKicWXR/L7tMtCOyl3427cL85NYBO0/nmbysaW/LnwxGiFyYKA0Um7z5eLhhyqCqEkEbV8+8OKaz3B6dgQiRcUs1oqmenljWFMubCgdb/mQwQuTCxF4qwn1J0XITLksydlo4NMCLSzNERhjSPkx+FT1H9MT/HbG8KSgcaPmTwQiRixI7zO46kw8PNwUeHdLOosszYhq5sX1n7Hn6mMhe9Y8PgbtSgXN5ZbV2iBazimKZUyx3OsryJxNYiVzUos26WZG7erax2M68zrbvDJE98fdyl/saiR2if0/JwcSQmOrviYBjRJdIuTwqZiX1HVgF8eGg5m32MEvCYITIBZ3ILMb6Y9lQKETrd8vMijj7vjNE9tKNNVkEIydzMbHf1WBEEP+XRMPAxj4c2EtXVi7TELmgj38/I7/e3C0S7cL87WLfGXudPiayZ0OqSnx3nM5FpVrT4Hn23pWVMyNELianWIUVB3U7ez5ioVwR7jtDZB3d27SQPYKKrlTiz/RCuWxTV1NdWcX/PPF9saxjq/+HnBkhcjFf7TqHcrUGPdu2MPjCZQ7cd4bIOtyUCgyqbg1fuxurI3VlZTBC5EKuVKjxza5z8vhh/dbjFqmcKTbqfFbOEF27qyW+tfep0XOErqxcpiFysQ3x8krLZffG0V0jzfqzWTlDZNu8kQNpBSi+UlGvk7IjdGXlzAiRC7V+/2Rbqjx+cGAs3N3M99+/oeQ4Q1g5Q2Re0cG+iA3xRaVGK3sH1eUIXVkZjBC5CNGl8WRWCfw83TAuKdpsP5eVM0T2UeIrbDOwVOMIXVkZjBC5CP2syF/6RCPQjBviGV85k4BvH+3PfWeIrNQaviZ778rKnBEiFyASSreczJFNzqYMirVR5Yx/rQZMRGQ+A6qq0s7kluLCpTK0aelb7xxDXVl7x7SUW0OIfDJbdmRlMELkAj7drpsVGdE5AjEhfmZdosktVhl1LitniCwn0NsDPaJbyMBClPjel9TW4Hk1u7KKXK8b/r3JLjqycpmGyMkVlJXjx/3pZi/nFS9kg9/YiJd/PdboefaQHEfkSlU1vzewVGPPHVkZjBA5uWV7z0NVqUGXVoFmCwiMrZ6xl+Q4IlfKG9mWkitnLRvSVEdWQXy/sZ9hbgxGiJyYeDERHVeFSQNioBBJI1asnrGX5DgiV5DYJghBPh4ovFyBPy4UNHiePXZkZTBC5MS2nMzG+fzLcu+K23u0tmr1zItjOrNyhsiK3N2UGFzVGn7LCcPdWO21IyuDESIn9sUO3azIuL7R8PF0M8vPNPYFKjTAi0szRFZ2fQddMLK1gdbw9tqRlcEIkZNKzS2tLue9v3/MNf887jtDZP+u76DLG/njfIFMXjfEHjuyMhghclJfV+WKDOsYfs3lvPrKmfEf7cKCTacbPZfVM0S20yrIBx0i/CFyT0UiqyH22JGVwQiREyorr5RVNPrE1WvBfWeIHMv1VVU1jeWN2FtHVjY9I3JCyw9koPhKpdw8S//CZK19Z2zRMImIrrqhYxg+3pYq80bEBpkNVdEZ6sjKDqxEZBbixefLnWflscgVUV7DC4sp+84MSgi12QsZEV3VNzYY3h5KZBWp5OaYHSMD0JCaHVltics0RE5mz9lLOJ5ZDB8PN/yld7RV951hIEJke94ebujfLqS6vN8RMBghcjL6Jmd39IxCkO817M6rUSOh9CBuU+5Af+VRKKFp8FRWzhDZl+urlme3nmy6NbzDBiMLFy5EbGwsvL290a9fPyQnJzd6/vz589GxY0f4+PggOjoazz77LK5csV4zFSJXkVuiqt5TYmK/a0hcPboSmN8NXddNwH89F2CJ5yvY5vUURilr/19n5QyRfZf4Jqfmy4R2pwtGli5diunTp2POnDnYv38/EhMTMWrUKGRnG54KWrx4MWbOnCnPP3bsGD755BP5M5577jlzjJ+IahAVNBVqrdy9s1vroOYHIssmAUUZtW6ORD4WecyvDkhYOUNkv+LD/NC6hQ/K1RrsPmO9tu5WC0befvttPProo5gyZQq6dOmCDz74AL6+vvj0008Nnr9jxw4MGjQIEyZMkLMpI0eOxPjx45ucTSEi02g0WizenSaPJ/Zr28wfogZWz6ixXdZV+nhjjsdXcsmG+84Q2S+FQlE9OyKaHzpVMFJeXo59+/Zh+PDhV3+AUimv79y50+B9Bg4cKO+jDz7OnDmDVatW4ZZbbmnwcVQqFYqKimpdiKhxW07l4MIl3T40YxOjmvdDzu2oNyNSNyCJUuTh57FK7jtDZOduqApGtjpAMGJSaW9ubi7UajUiIiJq3S6uHz9+3OB9xIyIuN/gwYNlyWFlZSUef/zxRpdp5s2bh7lz55oyNCKX980u3azIPb2jZTZ9s5RkGXVa18DLV6dKiMguDUzQVbidyS3F+fwyRAf7wmWraTZv3ozXXnsN77//vswx+fHHH/Hrr7/i5ZdfbvA+s2bNQmFhYfXl/HldJ0kiMiy94DI2HtcFEhP7t232vjPbMo0MYvxrfyAhIvsT6O2B3m1byuPNdj47YtLMSGhoKNzc3JCVVfvTk7geGRlp8D4vvvgiHnjgATzyyCPy+nXXXYfS0lI89thjeP755+UyT11eXl7yQkTGWZqcJveiGNAuBPFh/ibdV1TfiC6rormZEu7Y5hWMSEV+A59UFEBgFBAz0EwjJyJLGtopDMln87HpeDYeMMOGmXYxM+Lp6YnevXtjw4YN1bdpNBp5fcCAAQbvU1ZWVi/gEAGNIJZtiOjaVKg1WLLnfLNmReruO6OBEnMrJsn8VRHc1Fa1LDP6dUDZzGUgIrKqYR3D5dcdp3NxpUINp1mmEWW9H330Eb744gtZqjt16lQ50yGqa4RJkybJZRa9sWPHYtGiRViyZAlSU1Oxbt06OVsibtcHJUTUfOuPZiG7WIVQfy+M7GJ4htKUfWfWaJIwteIZZKJO7xAxI3Lvl0CX28wzcCKyuE6RAbIX0JUKDXadyYO9MnlvmnHjxiEnJwezZ89GZmYmevTogdWrV1cntaalpdWaCXnhhRdkiZH4mp6ejrCwMBmIvPrqq+b9TYhc1DdV5bzj+raBp7vSLPvOiIBknaoPkpTH8dKNoegQn6BbmuGMCJFDUSgUGNoxHN8mp2HziRx5bI8UWgdYKxGlvUFBQTKZNTAw0NbDIbIbZ3NLMfStzRCbcm79xzCTsuVXHEzH00sONnneu/f1wO09Wl/jSInIVtYeycRjX+1D22BfbPnH0AZ38bXl+zf3piFyYOLTjjC0Q5hJgYhYosktVhl1LvedIXJsgxJC4eGmQFp+GVJzS+EUyzREZB9UlWp8t++CPB6f1LZZ1TONEZ+dRJdV7jtD5Nj8vNzRLy4E21JyselEDtqZWHFnDZwZIXJQa45kIb+0HJGB3rixU3izqmcawn1niJzL0I66bqybTxjeR87WGIwQOahvqxJX7+0bDXc3ZbOrZwzhvjNEzmVY1QcWsWleqcr+dvHlMg2RAzqTU4KdZ/JkR/ZxfaOvuXqmphfHdMaDg+I4I0LkRNqF+skEVpE3sj0lFyO7Gt8GwBo4M0LkyImrHcPlNuHGyC5uOhARQgO8GIgQORmFQoFhVUs1Im/E3jAYIXIwoovi91WJqxOMSFzV7ztzKqvYqJ/P6hki516q2Xwi2+46oHOZhsjBrDmSiUtlFbKroj4p7VorZwRWzxA5t/7tQuDtoZSvByeyitEp0n76dnFmhMjBLK7uuNp44qqxlTMCq2eInJ+3hxsGxofK403H7WuphsEIkQNJyS7B7tT8JhNXTamcEVg9Q+QahunzRo7bV4kvl2mIHDBxVfQVaRXkc82VM9OGJcjujGJphjMiRC6SN7LiCPaey8el0nK09POEPeDMCJEDJa7+sL8qcbVfW7NUzrSP8MeA+BAGIkQuok1LX3RuFQiNVlTV2M/sCIMRIgex+nAmCsoqZCnvDR0a7rjKfWeIqDEjOuteP9Yfy4K9YDBC5ICJqw3NZIik1cFvbMTLvx5r9GeJe4tqHFbOELmemzpHyK9bTuTIPa7sAYMRIgcgeoQkn82XQci9fQwnrnLfGSIyxnWtgxAe4IXScjV2ncmHPWAwQuQAFtdIXBWVL3Vx3xkiMpZSqaieHVl/1D6WahiMEDlA4uqP+9MbTVw1Zd+ZbTNuZCBC5OJGdNHljWw4lmUX3VgZjBDZuVWHLqLwsi5x9fr2hjuuct8ZIjKFaH7m4+GGjMIrOHqxCLbGYITIQRJXxyfVT1zlvjNE1NxurEPa67qxrj9q+xJfNj0jsmMns4qx99wlGYT8pU7iKvedIaJrMbxLBNYezZIlvk8Pbw9b4swIkQPMigzvHI6IwKuzGtx3hoiulUiIVyiAQ+mFuFh4GbbEYITITl0uF4mr+o6rMdW3c98ZIjKHUH8v9IxuIY83HLPtUg2XaYjs1C9/ZqDoSiXatPTBkATd2q7AfWeIyJxLNfvTCuRSzf39r37osTbOjBDZqW+qlmhEOa/oC6DHfWeIyFxGVPUb2ZGSh1JVJWyFwQiRHTqcXoiD5wvg4Va/46qxFTGsnCGipiSE+yMmxBflag1+P5ULW2EwQmTHHVdFnodY15U0aiD1d/Qr3YhbAlLgBo3B+3LfGSIylkKhwKiukegb2xJeHrYLCZgzQmRnSlSVWHFA13F1or7j6tGVwOoZQFGG/ATxPoAMr2C8VDEJqzVJ1fdl5QwRmWrWzZ1kUGJLnBkhsjPLD6TLDaziw/zQT8xuiEBk2SQZiNTUSnEJizznY5Qyufo2Vs4QkalsHYgInBkhsiNijwh94urEfjFQaDW6GREDhbwKaKGFAv9tsRSrRzyM8EA/Vs4QkUNiMEJkRw6cL8Cxi0Xwclfi7l5tgHM76s2I1A1IvMou4vYW54C4IVYdKxGRuXCZhsiOfLNLNysyNjEKQb4eQImR23sbex4RkR1iMEJkJwrKymWjM33iqui0eqTIx7g7++t6BRAROSIu0xDZie/3XYCqUoMurQKRWXgZf/tmP7IKNdjmFYxI5MNwKogCCIwCYgZaf8BERLacGVm4cCFiY2Ph7e2Nfv36ITn5aja/IQUFBXjiiSfQqlUreHl5oUOHDli1alVzx0zkdDSaq4mrPaKD8LdvDsiW7xooMbdiku6cejmsVdHJ6NcBpZuVR0xEZMNgZOnSpZg+fTrmzJmD/fv3IzExEaNGjUJ2tuFNdsrLyzFixAicPXsW33//PU6cOIGPPvoIrVu3Nsf4iZzCtpRcpOaWwt/LHRuOZ9eqnVmjScLUimeQiTpNzMSMyL1fAl1us/ZwiYjMSqEVtYQmEDMhffv2xYIFC+R1jUaD6OhoPPnkk5g5c2a98z/44AP8+9//xvHjx+Hh4dGsQRYVFSEoKAiFhYUIDAxs1s8gsmePfLFXblQlOiGuOZJp8BwlNEhSHkc4CvDXMQPRdcBozogQkV0z9v3bpJkRMcuxb98+DB8+/OoPUCrl9Z07dxq8z8qVKzFgwAC5TBMREYFu3brhtddeg1qtbvBxVCqV/AVqXoic1YVLZdh4XFcN0ztGt523IWLJZpemC1ZqBiLFrwcDESJyGiYFI7m5uTKIEEFFTeJ6ZqbhT3NnzpyRyzPifiJP5MUXX8R//vMfvPLKKw0+zrx582Qkpb+ImRciZ7V4d5rMB+kWFYiiy8btmslN8IjImVi8tFcs44SHh+PDDz9E7969MW7cODz//PNy+aYhs2bNklM6+sv58+ctPUwim1BVqvHVznPy+HBGERZsSmn0fG6CR0Rw9dLe0NBQuLm5ISurdoMlcT0yMtLgfUQFjcgVEffT69y5s5xJEcs+np6e9e4jKm7EhcjZvb7qGIpVxs2GcBM8InJWJs2MiMBBzG5s2LCh1syHuC7yQgwZNGgQUlJS5Hl6J0+elEGKoUCEyFWIpmZfV5XzGoOb4BGRszK56Zko6508eTL69OmDpKQkzJ8/H6WlpZgyZYr8/qRJk2TZrsj7EKZOnSorb55++mlZcXPq1CmZwPrUU0+Z/7chciBL95xHhbrpYrZpw+IxKCGMm+ARkdMyORgROR85OTmYPXu2XGrp0aMHVq9eXZ3UmpaWJits9ETy6Zo1a/Dss8+ie/fuMlARgcmMGWInUiLXteJgulHntY8IwID4EIuPh4jIYfqM2AL7jJCzyStRod9rG1BZv61qPd8+2p/BCBE5JIv0GSGia7f68EUMfWtzk4EIK2eIyFVwozwiKwcij3+9v8nzWDlDRK6EwQiRFatn5v581OjKGRGIsHKGiFwBgxEiK0lOzZc78TblxTGd8eCgOM6IEJHLYM4IkZVkFzcdiAihAV4MRIjIpXBmhMgKyzNiVuRUVrFR53PfGSJyNQxGiCycsCryRIxZnlFU5YqweoaIXA2DESILBiJTv94PYxr5sHqGiFwZgxEiC1bOGNtRkNUzROTKGIwQ2bByZtqwBAxKCOW+M0Tk0hiMEFlgVmR7So5R57aP8GerdyJyeQxGiGyUsCqwcoaIiMEIkc0SVlk5Q0Skw6ZnRFZOWGXlDBFRbZwZIbJiwqrAyhkiotoYjBBZsdW7qJ55dkQHzogQEdXAYITIiq3eRRkvAxEiotoYjBA1E1u9ExGZB4MRIgtXzugxYZWIyDAGI0QWbvXeigmrRESNYjBCZKHKGV9PN3xwf2/miRARNYHBCJGFKmdu6hyO6zuEWXw8RESOjk3PiExcosktVhl17p092lh8PEREzoAzI0QWqJ7x93LHDR05K0JEZAwGI0QWqJ6ZexsrZ4iIjMVlGiIzV8+M7hqBu3tHW3hURETOg8EIkRn3nfH2UOLVO6+z+JiIiJwJgxGihmjUQOrv8Dj2A/orj0IJTZN3mdgvBiH+XlYZHhGRs2DOCJEhR1cCq2cARRnoA2CJJ5ChDcbciklYo0kyeBd3pQKPXd/O6kMlInJ0nBkhMhSILJskA5GaIpGPRR7zMUqZbPBu4/pGIyLQ20qDJCJyHgxGiOouzYgZEQPpqvrimDkeX9VbshGVM4/fEG+tURIRORUGI0Q1ndtRb0akbkASpchDkvK4vO7hpotQxidFIzrY12rDJCKCqwcjCxcuRGxsLLy9vdGvXz8kJxuetq5ryZIlUCgUuOOOO5rzsESWV5Jl1Gl/HxiEGaM7okKthZe7Ek/e2N7iQyMiclYmByNLly7F9OnTMWfOHOzfvx+JiYkYNWoUsrOzG73f2bNn8fe//x1Dhgy5lvESWayXyM7TediW6WbU+b27dMKvhy7K48kDY5krQkRkzWDk7bffxqOPPoopU6agS5cu+OCDD+Dr64tPP/20wfuo1WpMnDgRc+fORbt2rDYg++uuOviNjRj/0S5M2uAuq2YaLuJVAIGtsbokHofTi2Tbd+aKEBFZMRgpLy/Hvn37MHz48Ks/QKmU13fu3Nng/V566SWEh4fj4YcfNupxVCoVioqKal2ILNnmXd/UTAOlLN8V+auaejmsuvwQ9cjX8Z/1KfL4ocFxCPbztPawiYhcNxjJzc2VsxwRERG1bhfXMzMzDd5n27Zt+OSTT/DRRx8Z/Tjz5s1DUFBQ9SU6mq21yXpt3kUfkakVzyATwbW/ERgF3Psllqt6ISW7BEE+HnhkSJw1h0xE5JQs2vSsuLgYDzzwgAxEQkNDjb7frFmzZF6KnpgZYUBC1mzzLgKSdao+smrmpRtD0SE+AYgZiHKNAvPf3izPEcszgd4eVh41EZGLByMioHBzc0NWVu2KA3E9MjKy3vmnT5+Wiatjx46tvk2j0a3Gu7u748SJE4iPr7/e7uXlJS9ElpwV2Z6S0+g5Yslml6YLjoX2QIe41vK2pbvP4nz+ZYT6e2HywBgrjZaIyLmZFIx4enqid+/e2LBhQ3V5rgguxPVp06bVO79Tp044dOhQrdteeOEFOWPy7rvvcraDbJYnIpZnjN38LjxAVylTeLkC76w/JY+fuikBvp7cTYGIyBxMfjUVyyeTJ09Gnz59kJSUhPnz56O0tFRW1wiTJk1C69atZd6H6EPSrVu3Wvdv0aKF/Fr3diJrJqzW769an0hXjQzyRlKcLndk4aYU5JeWIyHcHxOS2lp8rERErsLkYGTcuHHIycnB7NmzZdJqjx49sHr16uqk1rS0NFlhQ+QoCauGVHV+x5yxXWSr93N5pfhse6q87fkxneHuxr9xIiJzUWi1WmNem21KJLCKqprCwkIEBgbaejjkoERTM9FLxBitgrxlIDK6Wyt5ferX+/Db4UwMaR+KLx9Kkp2EiYjIPO/fXPQmpydmRETlzG+HdR1TmzJtWAKeHdFBzogIu8/kyUBEXH1hTBcGIkREZsZghJyaqcmqwqCE0OpARKPR4pVfj8nj8Ult0TEywGJjJSJyVQxGyGmZkqxqKGFV+OlAOg6lFyLAy13OlhARkfkxC4/g6smqhhJWhRJVJd5cc1weP3FjguwtQkRE5seZEXK57qqGRNZJWBXeXnsSWUUqxIT44sGBsRYaKRERMRghJ+2ummvUuZMGxODmbq3k0ox+RkQ4nF6Iz3foSnlfvr0bvD3cLDZeIiJXx2CEXDphVQQiA+JD6gUzz/90SO7aOzYxCtd3CLPQaImISGAwQk7jWrqr1vTN7nP444IuafXFMZ0tMlYiIrqKCawEV++uWlNW0RX8e/UJefzP0R0RHqjbl4aIiCyHMyPkcgmrhpJV9V7+5SiKVZVIbBOECf24Ky8RkTUwGCEX664aj2dHdKw3IyJsPpGNX/68KDutvnrndQbPISIi82MwQi7WXTXMYJBRWFaBmT8cksdTBsWhW+sgs46ViIgaxmCEXLa7ak3/+vkIMouuIC7UD/83kp1WiYisiQms5JLdVWv67dBF2fZdfOs/9ybC15MxOhGRNfFVl1yyu6pedvEVPPeTbnlm6tB49Grb0qxjJSKipjEYIYdiju6qelqtFs/9eAiXyirQuVUgnr6JyzNERLbAYIRcqrtqTd/tu4D1x7Lh6abEO+MS4enOVUsiIltgMEIu1V1VLzW3FC/9fFQeTx/ZAZ0iA804WiIiMgU/CpLLdFfVu1yuxtSv96FEVYmk2GA8OqSdWcdLRESm4cwI2X1DM5EjYo7uqnqzVxzG8cxihPp74r0JPdncjIjIxhiMkNM0NGusu6resj3nZa6IOOW/43signvPEBHZHIMRcviGZk11V9U7klGIF1cclsf/N7IjBsaHXuNIiYjIHBiMkEM3NDM2YbXwcgX+9s1+qCo1GNYxDFNviDfLeImI6NoxgZXsh0aN4ztXoW/xRvRXHoUSGrMkrFaqNXh26UGcyytD6xY+eGdcDyiZJ0JEZDc4M0L24ehKYPUMdC3KwH89dTdlaIMxt2IS1miSmp2wKhqbiZmWjcez4eWuxKL7e6GFb9UDEBGRXWAwQvYRiCybJEKHWjdHIh+LPOZjasUz9QISkawqckQa6q6q98m2VHy16xwUCmD+uB7o3qaFxX4NIiJqHi7TkG1p1HJGpG4gIuhjjDkeX1Uv2YibWgV5y6oZ0V21sUBEJMK+uuqYPH7+ls64+bqGy32JiMh2GIyQbZ3bARRlNPhtEWtEKfKQpDxuVH6I3oG0S3h6yUFotcAD/WPw8OA4Mw+ciIjMhcs0ZNOGZh7HjqGPEeeHo8CohmbC6ZwSPPrlXlk5c2OncHkfhVinISIiu8RghGza0Ky/shBLjMgn/euYgXhnwI1NzoiIPWfGf7gLuSXl6BoViPfG94S7GycAiYjsGYMRsmlDs2RNJ1k1I5JVDccZCiAwCl0HjL6aRNKAc3m6QCS7WIWOEQH48qEk+HnxT5yIyN7xIyNZZUlm5+k8/HQgHc/9dKhWqqoGSlm+K4/r5bBWBR+jXweUbo0+xvn8MhmIZBZdQftwf3zzaD+E+HuZ+TchIiJL4MdGsvkeM6JsV5TvzvH4ElHIv/qNwChdINLltkYf48KlMtz34S5kFF5BuzA/GYiEMhAhInLumZGFCxciNjYW3t7e6NevH5KTkxs896OPPsKQIUPQsmVLeRk+fHij55PzLckYs9mdCEgGq/6L+8pfwN4+/wYm/wI8c6jJQOREZjH+8sFOpBdcRlyoH759tD/CA7j5HRGRUwcjS5cuxfTp0zFnzhzs378fiYmJGDVqFLKzsw2ev3nzZowfPx6bNm3Czp07ER0djZEjRyI9Pd0c4ycn2mNGLNns0nRBRee7gbghTS7N7Didi3s+2CGDnfgwXSDCXXiJiByPQiv6ZZtAzIT07dsXCxYskNc1Go0MMJ588knMnDmzyfur1Wo5QyLuP2mSLlegKUVFRQgKCkJhYSECAwNNGS7ZKBD5fHsqXv5V13DM1A3vts1oumpmxcF0/P27P1Ch1qJvbEt8NKkP27wTEdkZY9+/TcoZKS8vx759+zBr1qzq25RKpVx6EbMexigrK0NFRQWCgxveYVWlUslLzV+GnCdHxBBjG5qJ2Pl/W8/g9d+Oy+u3XBeJt+/tAW+PxmdRiIjISZZpcnNz5cxGRERErdvF9czMTKN+xowZMxAVFSUDmIbMmzdPRlL6i5h5IefKEalLzIiITewaa2hWqqrEM0sPVgcioqvqgvG9GIgQETk4q1bTvP7661iyZInMIxHJrw0RMy8iL6XmzAgDEufKERFzH8F+nnhhTGdEBvk0ueHdqaxiTP1mP1KyS+R54n5TBrHFOxGRywUjoaGhcHNzQ1ZWVq3bxfXIyMhG7/vWW2/JYGT9+vXo3r17o+d6eXnJCzlOW/ftKTlGz4joQ45X7+zWZGt34acDF/Dcj4dxuUKNiEAvLJjQC31jG17mIyIiJw5GPD090bt3b2zYsAF33HFHdQKruD5t2rQG7/fmm2/i1VdfxZo1a9CnjzE7kZAz54cYu8dM0ZUKvPLLUSzbe0FeH5wQivn39WAPESIiV1+mEcsnkydPlkFFUlIS5s+fj9LSUkyZMkV+X1TItG7dWuZ9CG+88QZmz56NxYsXy94k+twSf39/eSHnaOturBfHdMaDg+KarJZZfzQLzy8/hKwiFcQed0/d2B5P3dS+yfsREZELBCPjxo1DTk6ODDBEYNGjRw+sXr26Oqk1LS1NVtjoLVq0SFbh3HPPPbV+juhT8q9//cscvwNZeVlm15k8zPyhdlt3Y8t2mwpE8kpUcrZl5R8Z8npsiC9ev7s7+rcLMcPoiYjIKfqM2AL7jDhH2W5j1TKVag2W7j2P/6w9ifzScrkn3qND2uHZER1YLUNE5KAs0meEXFdzl2WayhERsfDG49mY99txWSkjdIoMwBt3d0didAszjJyIiOwdgxFqmEYNnNsBTXEmVv6cAQXaQWtCa5ppwxIwKCG0wbLdg+cL8Ppvx7DrjG5zvJa+HjIvZGK/GHi6c0NpIiJXwWCEDDu6Elg9AyjKkOHH+wAyvIIxt2KS3NTOmPwQscRSNwgRMyG/n8rFB1tOY8fpPHmbCDweGhSHqUPjEeTjYdFfi4iI7A+DEarXM8TtxM/om/yMCB2q8z2ESORjkcd8TK14psGApKG27hVqDVYduoj/bTmDoxd17f3dlQrc3qM1nh3RHm1a+lr0dyMiIvvFYIRqJadmFZZhm9dcaMWCTJ2VFXFdowXmeHyFdao+cpfdpvJDRB7Isr3n8eP+C8gtKZe3+Xq64b6+bfHwkDi0buFjnV+QiIjsFoMRqpWc2l95HFEKXQ6HISIgiUIekpTHsUvTpfr2Fr4eWDi+F/rHh+BSWTm+TU7D9/suYN+5S9XniGZlkwbEyAt32CUiIrh6MKJfksguvoLwAO8m90ZxleTUcBQYdXf9efpn7J+jOuJEVjHe3XgKe8/myxkUQTynwzqGY1zfaAztGAYPNyamEhFRbS4ZjBjql9HKyBblzp6cmg3jymkztLq9YXw83WTS6XM/Ha71/etaB+GW61rh7l6tER7Y8KaIRERELtf0rKF+GeITvhIafDtSjaSwSsA/AogZCCjdXCI5VT+T8beKpzDb42uZrKqfKNJoFTirjcBBbQIOauKxH51xRNMWNf9yRMt2sXndqK6RGNU1ggmpREQENj0zcZv7kcpkzPH4ElFba+RLBEYBo98AutwGV0lOfdr9Jzxb/jeMcduFU5o2OKltg2PaGBTBr97PjQv1w8D4ENlLZEC7ELT0Yx4IERGZzqWCETEzYKiV+ShlsixZrafoIrTLJuHkDQtxvOVQx8ktqcoHQUkWknPc8cRaN6ihRD/FcfigHMe1bZGlCcZFbTDStaFy1iNNG4Fz2ojqoGN35dXkVMEL5ejmkYHEhLbo0aM3+sS0RBQrYYiIyAxcKhgRyap1iaUZMSMij+vFGGIOAQjY/CKeVb0rS1ltnVsiVtVUlRqUlatRqqpE0eUKJJ/NR2bhFZkcGlp6AsVH1qLwigaXtP7IQxDaK4JlwLFH2wk9VR82+Rg+UOEyvKCAFu0UGZjW0wO39oqHR7vbnGbZioiI7IdLBSNiZqOuOMVF7NB0hVYr3nqvRiPiWFv9VYEYRSZStVFyZuXxr/ejf7tgdIwIQHSwLjdC5E+otVpoxEUjvuqWhcR18VV+T6NFpbiodV/VGo08LldrZFOwCrVWfhXBRnml/qtafr1SocblcjUuV6ir8zsME8HCzY0+DyEoRKQiH60U+fJrjCILbRXZ1V8fqviHLNuNDPLBP8aOdY2kXiIishmXSmAVQcHgNzbKWQS7/6WNIGZ1NDKAEhcteitOIkxRgBaKUgSiDC0UxQhFEUIURQhGETQKhbwercg2MAukC7zKfSOxesRahAf6OcaSFBER2S0msBog3ljFEouoptG9fQNBKEFPZYr8vm4O5Or8yNW5EWCvpgMuwbhAyMdDiYRwfwT6eMDP013mVojHziq6Imc5ArzdEd3SV14XMx0tfDzk+WKZJS2/DGXllQjx90LXVoFIzStDiaoCkYE+SFLvh9/vr8KnJA2+uAI3hVaW2IqS3EL4Y4nnK02O7T8V9+BZ9+/l7ErtOEMhf0+vW9/E7V3aNuPZJSIiah6Xmhkx1GdEzC5s83pKLlcYascl3rQzEYLBVTkjzSG6kwoFZRX1qleMPUck2X7gqUuyNVSS+2nlzXjE47cmx/JU+TSo4K6rHKrZaTWwNTD6dYevHCIiIvvBmZFGiByIEV0iqzuwFl96Ba22PFH1XW29N/q5FQ80OxCpG2DU/dnGnKNPshVhY0MluXe4bzdqLKKpmcgHWa/qg77K45g1uAUSO3dyqp4qRETkWFwyGBHEssmA+JCqaxOBiIDqrqR6YkZEBCIN7VBrLUlG7BcjckFytQEIRrHBfBCN+H20IUjWdJLXw4N88eDY+5HI5FQiIrIxlw1G6hHLE53G1OrPMX6t2zXNiJiLsfvFLFcPwkNuqxvMByke+jLeadnLcfqlEBGRS2AwUpNYpogbIg/FXMjC8Pp72NiCsfvFrNf0wV5NJ8wW+SCo3UlWMfp1dOxyGzpabphERETNwmDEyNyS9Ucz8cn2szYZh1haEVUzNfeLMZRkK86LCPLFn7c+iij/VDnD42x77BARkfNxyWoac+72ay01W9bXDEj0hch7kuZD3XEsl1+IiMjh3r8ZjDRz91tRhSNyLy6VqvDyr8esEqCMVibrlmBYkktERA6Apb1WqcLRGdWtVXWAcja3DN8mpyGz6Gpw0tLXQxYMX0ufEeGPgOu5BENERE6HwYgFApRpNybUmj0RSydCzdt6x7TEvnOXTD5HtwTTxma/KxERkblxmYaIiIhs+v5t+yYaRERE5NIYjBAREZFNMRghIiIim2IwQkRERDbFYISIiIhsisEIEREROV4wsnDhQsTGxsLb2xv9+vVDcnJyo+d/99136NSpkzz/uuuuw6pVq5o7XiIiInL1YGTp0qWYPn065syZg/379yMxMRGjRo1Cdna2wfN37NiB8ePH4+GHH8aBAwdwxx13yMvhw4fNMX4iIiJytaZnYiakb9++WLBggbyu0WgQHR2NJ598EjNnzqx3/rhx41BaWopffvml+rb+/fujR48e+OCDD4x6TDY9IyIicjwW2ZumvLwc+/btw6xZs6pvUyqVGD58OHbu3GnwPuJ2MZNSk5hJWb58eYOPo1Kp5EVP/BL6X4qIiIgcg/59u6l5D5OCkdzcXKjVakRERNS6XVw/fvy4wftkZmYaPF/c3pB58+Zh7ty59W4XMzBERETkWIqLi+UMiUNtlCdmXmrOpoiloPz8fISEhEChEBvFmS9iEwHO+fPnufxjBD5fxuNzZTw+V8bjc2U8Plf28VyJGRERiERFRTV6nknBSGhoKNzc3JCVlVXrdnE9MjLS4H3E7aacL3h5eclLTS1atICliCeff6zG4/NlPD5XxuNzZTw+V8bjc2X756qxGZFmVdN4enqid+/e2LBhQ61ZC3F9wIABBu8jbq95vrBu3boGzyciIiLXYvIyjVg+mTx5Mvr06YOkpCTMnz9fVstMmTJFfn/SpElo3bq1zPsQnn76adxwww34z3/+gzFjxmDJkiXYu3cvPvzwQ/P/NkREROT8wYgo1c3JycHs2bNlEqoo0V29enV1kmpaWpqssNEbOHAgFi9ejBdeeAHPPfcc2rdvLytpunXrBlsTS0GiX0rdJSEyjM+X8fhcGY/PlfH4XBmPz5VjPVcm9xkhIiIiMifuTUNEREQ2xWCEiIiIbIrBCBEREdkUgxEiIiKyKQYjNdx2221o27YtvL290apVKzzwwAPIyMiw9bDsztmzZ+UuzHFxcfDx8UF8fLzMxBZ7F1F9r776qqwq8/X1tWjzPke0cOFCxMbGyv9zYhPO5ORkWw/JLm3duhVjx46VXSxFF+rG9vZydaKthNjMNSAgAOHh4XKX+BMnTth6WHZp0aJF6N69e3WzM9H/67fffrPJWBiM1DBs2DAsW7ZM/uH+8MMPOH36NO655x5bD8vuiH2IRLO7//3vfzhy5AjeeecduQOzKN2m+kSQ9pe//AVTp0619VDsytKlS2XfIhHI7t+/H4mJiXITzezsbFsPze6IXk7i+RHBGzVuy5YteOKJJ7Br1y7ZYLOiogIjR46UzyHV1qZNG7z++utyA1zR/+vGG2/E7bffLl/XrU6U9pJhK1as0CoUCm15ebmth2L33nzzTW1cXJyth2HXPvvsM21QUJCth2E3kpKStE888UT1dbVarY2KitLOmzfPpuOyd+Jl+6effrL1MBxGdna2fM62bNli66E4hJYtW2o//vhjqz8uZ0YaIDbm++abb+T0uoeHh62HY/cKCwsRHBxs62GQA80WiU9jw4cPr75NNEsU13fu3GnTsZHzvTYJfH1qnFqtlh3SxQySLbZrYTBSx4wZM+Dn5yd3CBbdZFesWGHrIdm9lJQUvPfee/jrX/9q66GQg8jNzZUvfvrOzXriuujsTGQOYjn5mWeewaBBg+yi67c9OnToEPz9/WX31ccffxw//fQTunTpYvVxOH0wMnPmTJnw1dhF5EDo/eMf/8CBAwewdu1auUOx2GvHVZrUmvpcCenp6Rg9erTMiXj00UfhKprzXBGRdYnckcOHD8tP/GRYx44dcfDgQezevVvmtYm9544ePQprc/p28GIfnby8vEbPadeundyRuK4LFy4gOjoaO3bscIldhk19rkSl0dChQ9G/f398/vnntfYkcnbN+bsSz5H4lFZQUABXJ5ZpRHXR999/L6sd9MQLoXh+OCPZMBHoik+vNZ83qm/atGny70hUIonKPzKOWCoVFZKiQMGuN8pzNGFhYfLS3Ck+QaVSwRWY8lyJGRFRfdS7d2989tlnLhWIXOvfFUEGaeJvZ8OGDdVvquL/m7gu3kSImkt8vn7yySdlwLZ582YGIiYS/w9t8Z7n9MGIscQU1Z49ezB48GC0bNlSlvW++OKLMkJ0hVkRU4hARMyIxMTE4K233pKzBHqRkZE2HZs9ErlHIiFafBV5EmJKVEhISJBrta5KlPWKmZA+ffogKSkJ8+fPl8lzU6ZMsfXQ7E5JSYnMzdJLTU2Vf0ciKVP0RqLaSzNip3gxKyJ6jehzkIKCgmRfJLpq1qxZuPnmm+XfUHFxsXzeRAC3Zs0aWJ3V63fs1J9//qkdNmyYNjg4WOvl5aWNjY3VPv7449oLFy7Yemh2WaIq/nQMXai+yZMnG3yuNm3apHV17733nrZt27ZaT09PWeq7a9cuWw/JLom/FUN/Q+Jvi2pr6LVJvG5RbQ899JA2JiZG/v8LCwvT3nTTTdq1a9dqbcHpc0aIiIjIvrnWQj8RERHZHQYjREREZFMMRoiIiMimGIwQERGRTTEYISIiIptiMEJEREQ2xWCEiIiIbIrBCBEREdkUgxEiIiKyKQYjREREZFMMRoiIiMimGIwQERERbOn/AVyTDpTVNbQPAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plt.plot(x, y)\n", + "plt.scatter(x_train, X_train_s @ beta + y_offset)\n", + "plt.scatter(x_test, X_test_s @ beta + y_offset)" + ] + }, + { + "cell_type": "markdown", + "id": "2993cfae", + "metadata": {}, + "source": [ + "## Exercise 4 - Testing multiple hyperparameters" + ] + }, + { + "cell_type": "markdown", + "id": "e6cdc89a", + "metadata": {}, + "source": [ + "**a)** Compute the MSE of your ridge model for polynomials of degrees 1 to 5 with lambda set to 0.01. Plot the MSE as a function of polynomial degree.\n", + "\n", + "**b)** Compute the MSE of your ridge model for a polynomial with degree 3, and with lambdas from $10^{-1}$ to $10^{-5}$ on a logarithmic scale. Plot the MSE as a function of lambda.\n", + "\n", + "**c)** Compute the MSE of your ridge model for polynomials of degrees 1 to 5, and with lambdas from $10^{-1}$ to $10^{-5}$ on a logarithmic scale. Plot the MSE as a function of polynomial degree and lambda using a [heatmap](https://matplotlib.org/stable/gallery/images_contours_and_fields/image_annotated_heatmap.html)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.15" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git a/doc/LectureNotes/_toc.yml b/doc/LectureNotes/_toc.yml index ad41afbdd..88cafbfde 100644 --- a/doc/LectureNotes/_toc.yml +++ b/doc/LectureNotes/_toc.yml @@ -41,7 +41,8 @@ parts: - caption: Weekly material, notes and exercises numbered: false chapters: - - file: E1.ipynb + - file: exercisesweek34.ipynb - file: week34.ipynb - - file: E2.ipynb - - file: week35.ipynb \ No newline at end of file + - file: exercisesweek35.ipynb + - file: week35.ipynb + - file: exercisesweek36.ipynb \ No newline at end of file diff --git a/doc/LectureNotes/exercisesweek36.ipynb b/doc/LectureNotes/exercisesweek36.ipynb new file mode 100644 index 000000000..e3365bb50 --- /dev/null +++ b/doc/LectureNotes/exercisesweek36.ipynb @@ -0,0 +1,383 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "9dd8f8d0", + "metadata": {}, + "source": [ + "# Weekly Exercises 3\n", + "\n", + "## Deriving and Implementing Ridge Regression" + ] + }, + { + "cell_type": "markdown", + "id": "e5cc5739", + "metadata": {}, + "source": [ + "## Learning goals\n", + "\n", + "After completing these exercises, you will know how to\n", + "- Take more derivatives of simple products between vectors and matrices\n", + "- Implement Ridge regression using the analytical expressions\n", + "- Scale data appropriately for linear regression\n", + "- Evaluate a model across two different hyperparameters" + ] + }, + { + "cell_type": "markdown", + "id": "427d9b9d", + "metadata": {}, + "source": [ + "## Exercise 1 - Choice of model and degrees of freedom" + ] + }, + { + "cell_type": "markdown", + "id": "ff6ebdb5", + "metadata": {}, + "source": [ + "**a)** How many degrees of freedom does an OLS model fit to the features $x, x^2, x^3$ and the intercept have?\n", + "\n", + "**b)** Why is it bad for a model to have too many degrees of freedom?\n", + "\n", + "**c)** Why is it bad for a model to have too few degrees of freedom?\n", + "\n", + "**d)** Read [chapter 3.4.1 of Hastie et al.'s book](https://link.springer.com/book/10.1007/978-0-387-84858-7). What is the expression for the effective degrees of freedom of the ridge regression fit?\n", + "\n", + "**e)** Why might we want to use Ridge regression instead of OLS?\n", + "\n", + "**f)** Why migth we want to use OLS instead of Ridge regression?" + ] + }, + { + "cell_type": "markdown", + "id": "a78b2ce2", + "metadata": {}, + "source": [ + "## Exercise 2 - Deriving the expression for Ridge Regression\n", + "\n", + "The aim here is to derive the expression for the optimal parameters using Ridge regression.\n", + "\n", + "The expression for the standard Mean Squared Error (MSE) which we used to define our cost function and the equations for the ordinary least squares (OLS) method, was given by the optimization problem" + ] + }, + { + "cell_type": "markdown", + "id": "93a7ad16", + "metadata": {}, + "source": [ + "$$\n", + "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in {\\mathbb{R}}^{p}}}\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\\right\\}.\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "6b182c01", + "metadata": {}, + "source": [ + "By minimizing the above equation with respect to the parameters $\\boldsymbol{\\beta}$ we could then obtain an analytical expression for the parameters $\\boldsymbol{\\hat\\beta_{OLS}}$.\n", + "\n", + "We can add a regularization parameter $\\lambda$ by\n", + "defining a new cost function to be optimized, that is" + ] + }, + { + "cell_type": "markdown", + "id": "fb065daf", + "metadata": {}, + "source": [ + "$$\n", + "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n", + "{\\mathbb{R}}^{p}}}\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2+\\lambda\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "89ae11e8", + "metadata": {}, + "source": [ + "which leads to the Ridge regression minimization problem. (One can require as part of the optimization problem that $\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\\le t$, where $t$ is a finite number larger than zero. We will not implement that in this course.)" + ] + }, + { + "cell_type": "markdown", + "id": "456d58d4", + "metadata": {}, + "source": [ + "### a) Expression for Ridge regression\n", + "\n", + "Show that the optimal parameters" + ] + }, + { + "cell_type": "markdown", + "id": "a5677efd", + "metadata": {}, + "source": [ + "$$\n", + "\\hat{\\boldsymbol{\\beta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "e7b08048", + "metadata": {}, + "source": [ + "with $\\boldsymbol{I}$ being a $p\\times p$ identity matrix.\n", + "\n", + "The ordinary least squares result is" + ] + }, + { + "cell_type": "markdown", + "id": "8bb0e6af", + "metadata": {}, + "source": [ + "$$\n", + "\\hat{\\boldsymbol{\\beta}}_{\\mathrm{OLS}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "id": "2faaae78", + "metadata": {}, + "source": [ + "## Exercise 3 - Scaling data" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "8ed79466", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "from sklearn.model_selection import train_test_split\n", + "from sklearn.preprocessing import StandardScaler" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "id": "59d6f347", + "metadata": {}, + "outputs": [], + "source": [ + "n = 100\n", + "x = np.linspace(-3, 3, n)\n", + "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2) + np.random.normal(0, 0.1)" + ] + }, + { + "cell_type": "markdown", + "id": "c9fc9fe0", + "metadata": {}, + "source": [ + "**a)** Adapt your function from last week to only include the intercept column if the boolean argument `intercept` is set to true." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "id": "5aa9c86d", + "metadata": {}, + "outputs": [], + "source": [ + "def polynomial_features(x, p, intercept=False):\n", + " n = len(x)\n", + " X = np.zeros((n, p + 1))\n", + " #X[:, 0] = ...\n", + " #X[:, 1] = ...\n", + " #X[:, 2] = ...\n", + " # could this be a loop?" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "id": "2ebf249a", + "metadata": {}, + "outputs": [], + "source": [ + "def polynomial_features(x, p, intercept=False):\n", + " n = len(x)\n", + " X = np.zeros((n, p))\n", + " X[:, 0] = x[:]\n", + " X[:, 1] = x**2\n", + " X[:, 2] = x**3\n", + " return X" + ] + }, + { + "cell_type": "markdown", + "id": "c5924634", + "metadata": {}, + "source": [ + "**b)** Split your data into training and test data(80/20 split)" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "id": "676541e7", + "metadata": {}, + "outputs": [], + "source": [ + "X = polynomial_features(x, 3)" + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "id": "f895606f", + "metadata": {}, + "outputs": [], + "source": [ + "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n", + "x_train = X_train[:, 0] # These are used for plotting\n", + "x_test = X_test[:, 0] # These are used for plotting" + ] + }, + { + "cell_type": "markdown", + "id": "a58f05ad", + "metadata": {}, + "source": [ + "**c)** Scale your design matrix with the sklearn standard scaler, though based on the mean and standard deviation of the training data only." + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "id": "0c07995a", + "metadata": {}, + "outputs": [], + "source": [ + "scaler = StandardScaler()\n", + "scaler.fit(X_train)\n", + "X_train_s = scaler.transform(X_train)\n", + "X_test_s = scaler.transform(X_test)\n", + "y_offset = np.mean(y_train)" + ] + }, + { + "cell_type": "markdown", + "id": "63b86003", + "metadata": {}, + "source": [ + "## Exercise 4 - Implementing Ridge Regression" + ] + }, + { + "cell_type": "markdown", + "id": "a2f52ef2", + "metadata": {}, + "source": [ + "**a)** Implement a function for computing the optimal Ridge parameters using the expression from **2a)**." + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "id": "69f19bde", + "metadata": {}, + "outputs": [], + "source": [ + "def Ridge_parameters(X, y):\n", + " # Assumes X is scaled and has no intercept column\n", + " return np.linalg.inv(X.T @ X) @ X.T @ y\n", + "\n", + "beta = Ridge_parameters(X_train_s, y_train)" + ] + }, + { + "cell_type": "markdown", + "id": "626c81ba", + "metadata": {}, + "source": [ + "**b)** Fit a model to the data, and plot the prediction using both the training and test x-values extracted before scaling, and the y_offset." + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "id": "f9ba0655", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 82, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvZiW1igAAAAlwSFlzAAAPYQAAD2EBqD+naQAAXxtJREFUeJzt3Qd409X6B/Bv0r2hm0JpS8tGyixbQVmKuK8IKIjryhUX/3sv4ICLC/V6Fa8gXvdEwAWoyF4yy1Q2FAqFlm66IW2T/J9z0pSOtE1Kdr6f58nTX9JfmtNQkjfnvO97FFqtVgsiIiIiG1Ha6oGJiIiIBAYjREREZFMMRoiIiMimGIwQERGRTTEYISIiIptiMEJEREQ2xWCEiIiIbIrBCBEREdmUOxyARqNBRkYGAgICoFAobD0cIiIiMoLoq1pcXIyoqCgolUrHDkZEIBIdHW3rYRAREVEznD9/Hm3atHHsYETMiOh/mcDAQFsPh4iIiIxQVFQkJxP07+MOHYzol2ZEIMJghIiIyLE0lWLBBFYiIiKyKQYjREREZFMMRoiIiMimGIwQERGRTTEYISIiIptiMEJEREQ2xWCEiIiIbIrBCBEREdkUgxEiIiKyKQYjREREZFMMRoiIiMimGIwQERGRTTnERnlERESNUVWqcSCtAMmp+fByVyIxugWuax0EPy++zTkC/isREZFDyi6+gp/2p2P76TzsSc3H5Qp1re8rFUCHiAD0immJx6+PR9sQX5uNlcy8TLN161aMHTsWUVFRckvg5cuXN3kflUqF559/HjExMfDy8kJsbCw+/fRTUx+aiIhIWnMkEyPf2Yp5vx3H1pM5MhAJ9ffErd1b4eZukWgV5A2NFjieWYzFu9Nw87tbsXRPGrRara2HTuaYGSktLUViYiIeeugh3HXXXUbd595770VWVhY++eQTJCQk4OLFi9BoNKY+NBERubjL5Wq8/OtRGWAInSIDcE/vNhjcPhQdIwLkh2S9rKIr+ON8AT7eliqXb2b8cAjrjmZh3l3dERbgZcPfgupSaK8hTBT/6D/99BPuuOOOBs9ZvXo17rvvPpw5cwbBwcHNepyioiIEBQWhsLAQgYGBzR0uERE5sCMZhXjq2wM4nVMqr//1+nb4v5Ed4ene+CS/WqPFJ9vO4K01J1Gu1iDEzxNv3tMdN3WOsNLIXVeRke/fFq+mWblyJfr06YM333wTrVu3RocOHfD3v/8dly9fbnRZR/wCNS9EROS6tp3KxZ0Ld8hAJDzAC18/3A+zbuncZCAiuCkVeOz6eKyYNkjOpOSVluOxr/Zhy8kcq4ydmmbxYETMiGzbtg2HDx+Wsyjz58/H999/j7/97W8N3mfevHkyktJfoqOjLT1MIiKyU8cuFuHxr/fJWY2hHcOw+pnr5bKMqTq3CpQBye09ouRsyRPf7MeJzGKLjJnsLBgRuSFiOeebb75BUlISbrnlFrz99tv44osvGpwdmTVrlpzS0V/Onz9v6WESEZEdulh4GVM+24MSVSX6xQXjfw/0RrCfZ7N/npe7m1yiSYoLlj/zoc/3yKoccvJgpFWrVnJ5Rsxw6HXu3FlmNF+4cMHgfUTFjVhbqnkhIiLXUnSlAg9+ugeZRVfQPtwfHz7QRwYT10r8jP/d3xtxoX5IL7iMR7/chyt1yoLJyYKRQYMGISMjAyUlJdW3nTx5EkqlEm3atLH0wxMRkQMqr9Rg6tf7cCKrWFa+fDalL4J8Pcz281v6eeLTB/uiha+HrLiZvuwgNKIWmBwjGBFBxcGDB+VFSE1NlcdpaWnVSyyTJk2qPn/ChAkICQnBlClTcPToUdmn5B//+IcsDfbx8THn70JERE5i9orD2J6SBz9PN3z2YF+0aWn+hmViZkTMkHi4KbDqUCYWbTlt9scgCwUje/fuRc+ePeVFmD59ujyePXu2vC56iOgDE8Hf3x/r1q1DQUGBrKqZOHGibJr23//+19SHJiIiF7DxeBaW7DkvO6gunNgL3VpfXeY3t37tQvDKHd3k8X83nEJaXpnFHoss1GfEWthnhIjIdfJERr69VeaJPDokDs+P6WLxxxRvg/d/slvOxIhqHTETU7N5GjlBnxEiIiJjvfbrMRmIxIb4YvqIjlZ5TBF4vHR7N3i6KbH5RA5WH860yuPSVQxGiIjILvx+Kkcuzwhv3pMIH89rr5wxVnyYP/56Qzt5PPfno7Lsl6yHwQgREdmcePOf+cMheTx5QIzsA2JtTwxLQNtgXzkz8+76k1Z/fFfGYISIiGzujd+Oy54fbVr64J+jO9lkDN4ebph7e1d5/On2s7LzK1kHgxEiIrKpPWfz8dWuc/L4jbu7w8/L5A3lzWZYx3CM7hop28W/sPywTG4ly2MwQkRENiPe7F/99Zg8HtcnGoMSTN9zxtzm3NYFvp5u2HfuEjYcy7b1cFwCgxEiIrIZUbly8HyBfPP/v1EdbD0cqVWQDyYPjJXH721K4eyIFTAYISIim6hQa/DmmhPy+JEh7RAe4A178fDgOHh7KGWr+G0pubYejtNjMEJERDYhynhTc0sR4ueJx67XldXai1B/L0xIipHH721MsfVwnB6DESIisrpSVSXeXX9KHj91U3v42zBptSEiQBKN0JJT87H7TJ6th+PUGIwQEZHVffx7KnJLVIgJ8cX4pLawR5FB3vhLH93u8gs2cXbEkhiMEBGRVYkg5MOtuh1y/zGqIzzd7fet6PEb4uGmVOD3U7kyf4Qsw37/AoiIyCmJ3XFLy9VIbBOEMde1gj2LDvbFnT1by2POjlgOgxEiIrKai4WX8W1ymjyeeXNnh9gdd+rQeIhhrjuaxa6sFsJghIiIrOajramoUGvRv10wBsSHwBGITfT0MzgfbNEtL5F5MRghIiKryC8tr54V+dvQBDgSkTsi/HYoE3klKlsPx+kwGCEiIqv4fMdZXK5Qo1vrQAxpb/u276bo1joIidEtUK7W4Pt9F2w9HKfDYISIiCyuRFWJL3acrZ4VcYRckbomVpUgL05Og0bDFvHmxGCEiIgsbvHucyi8XIF2YX4Y1TUSjujWxFYI8HbHubwybD/NFvHmxGCEiIgsSlWplk3OavbtcES+nu64u5euCdo3u3S5L2QeDEaIiMiiftiXjuxiFVoFeeOOHrqeHY5qQj/dUs26Y1nIKrpi6+E4DQYjRERkMZVqDf5X1W310SHt7LrbqjE6RAQgKTYYao0WS/ect/VwnIZj/1UQEZFdW3U4U+ZYBPt54r6kaDiDif11syOiTFkEW3TtGIwQEZHFfLJNlysyeUCszLlwBqO7Rcrg6mLhFWw+kWPr4TgFBiNERGQR+9Muyc3lxNLM/VWzCc7Ay90N9/SuSmTdfc7Ww3EKDEaIiMgiPt+u6ytyW2IUQvy94EzGV/Uc2XwyB+fzy2w9HIfHYISIiMwus/AKVh26KI+nDIqFs4kL9cOAdiHQaoEVB9NtPRyHx2CEiIjM7utd51Cp0SIpLhhdo4LgjO7sqStTXn4wA1oRlVCzMRghIiKzulKhli3ThYeccFZEb/R1kTIfJiW7BEcyimw9HIfGYISIiMxq5cEMuUNv6xY+GN45As4q0NsDN3UKl8dcqrk2DEaIiMhsxHLFZ1Ub4k0aEAN3N+d+m7m9qqPsyj8yZCM0ah7n/ishIiKr2p2aj2MXi+Dj4Yb7+jpPOW9DhnUKQ6C3O7KKVNh9Js/Ww3FYDEaIiMhsPtuua3J2V6/WCPL1gLMTPUduua6VPF7OpRrrBSNbt27F2LFjERUVBYVCgeXLlxt93+3bt8Pd3R09evQw9WGJiMjOpRdcxrqjWfL4wYHOm7ja0FLNb4cyZfIuWSEYKS0tRWJiIhYuXGjS/QoKCjBp0iTcdNNNpj4kERE5gG93p0GkTQyMD0H7iAC4in5xwXJH4mJVJTafyLb1cFwjGLn55pvxyiuv4M477zTpfo8//jgmTJiAAQMGmPqQRERk58orNVhStYvt/f1j4EqUSgVu6xElj5cfyLD1cBySVXJGPvvsM5w5cwZz5swx6nyVSoWioqJaFyIisiMaNZD6O3Doe/l17eEM5Jao0MLXQy5V7DydJ6tLxEUci9JX8VUELTWvGzrHEatS7qhaqtl4PBuFlytsPRyHY/EtFE+dOoWZM2fi999/l/kixpg3bx7mzp1r6aEREZEpwce5HUBJFpB3Gtj3GVCsa/cufFn5EoAEFJRVYPqyP+RtIjARxG16SgXkUo6eoXNaB3rgnf5lSAqrBPwjgJiBgNIN9qxzq0B0jAjAiaxirD58EeNcoJLIYYIRtVotl2ZEYNGhQwej7zdr1ixMnz69+rqYGYmOjrbQKImIqFFHVwKrZwBFhpcgUjRRSK5MgBIaDFb8ia3aHvUCDL26kx51zxmlTMYc1ZeI2pp/9cbAKGDUPMA3RBcM2WmAcnvPKLy5+gRWHMxgMGJPwUhxcTH27t2LAwcOYNq0afI2jUYjm+KIWZK1a9fixhtvrHc/Ly8veSEiIusTyyTJqfnILr6CTpc2o8OWJ6BAw0snX6uHy683KvfjJY8vMFj1LjTNyAIQgcgij/n1vyGCoO8m175NBCij3wC63AZ7Mba7LhjZdSZPdqAN9vO09ZAchkWDkcDAQBw6dKjWbe+//z42btyI77//HnFxcZZ8eCIiMjH4OJtbhm+T05BZdEXOdGzzegFahRaKBu5bpvXCD+oh8vh+t/WIUuQhSXkcuzRdTBqDeKw5Hl/qjht6sBq0RReBZZOwJ2k+1B3Hyg353Iy5owVFB/uia1Sg3Kdm/dEs3NuXM/oWC0ZKSkqQkpJSfT01NRUHDx5EcHAw2rZtK5dY0tPT8eWXX0KpVKJbt2617h8eHg5vb+96txMRkfWJ/Ia5Px/FxcIr9b4ngoooRY3lEgN+Vg9AMfzQVpGF65W6D5/hKDB5HMY8Vk1ipkYs+bTZ/RIGbw1BRJAv5oztgtHddA3IbOXmbpEyGPnt8EUGI5YMRsSyy7Bhw6qv63M7Jk+ejM8//xwXL15EWpput0YiIrLnWZBSvLP+VIPnGhNU6JdoJrhtgFKhW8rJRguTx9WcAEZMhERBNxOzu7ALpn69H88M74DYUF+EB3jbZLZkdLdIvLX2JLan5KHoSoXcTI8sEIwMHTpU5nw0RAQkjfnXv/4lL0REZD+zIIY0FVT8oWmHQ9p28EQF/uK2Rc5UZCIEyZpOJo+tOQFMzUBG/670zvqT1beLRmTWni1JCA9AfJgfTueUYtPx7OrurNQ47k1DROTE9D08Xv75CB7/er/RgYgggooMbXC9Chi9b9S6jtq3KHejJYrl8dyKB5qVvNrUYzUnkMksvCJnS0QQZu3ZEWHNkUyrPq4jYzBCROSkxJvw4Dc2YvxHu/DJ9rMm318EFXMrJumO6wQJRVofmS8iTHDfAJVvJJ7z+CfWaJKqz2np61HdR0Sv7qqJ/pxaj2Xs+LRAhrbhmRht1WXmD4ewPSXXas3Ubq6aidl0PAeXy7lXjV00PSMiIvvLBzGWCC6mVjwjK12icDXBdIXHGFxWeSMhUIO+970FRewgvAolbq96fH3OhpBc47beMS2x79ylBs7pgVOXOqDDgVca7Gmip48rjJmJKbhcgYkf77baso2oqGndwkduHLj1VA5GddXNlFDDFNrGEkDshGh6FhQUhMLCQlkuTERE15YPYgpFVenttyPVsiuq1i8cN68EjmcWY/atXfDQ4DjLdXsVDc5K84C1s2oFKGJGRAQiNWdijPk9hEX397J4QPLyL0fxybZU3NWzNd4e57o71RcZ+f7NmREiIicIRERuhKU+WUZWzSgkVb2BH0y7hOOZO+DprsRdvSyQoCk6q8bpepdU6zJWBih/HDuOedsKsEfTCWoTMw3E8+MGDVauWIaRmigoAyIt1slVlPiKYGTdsSy5H494rqhhDEaIiBx4WWbX6TyZE2HOQCQy0Avjk9oiNtTPYIns4t269g23XtcKLXyt1GW0KkBJjBuCB9texLlmzALJVvNiuakiH/jRsp1ce7VtibAAL+QUq7DzTB5u6BBm1p/vbBiMEBG58LKMPsQwtj+H6J3x85+65ZLx/Wyz/4pYYhnRJbJefowYsdbkVvO6Tq6490uzBiRKpQIju0Tgm91pWH04k8FIExiMEBG58LKMfgnG2ByKFQfScaVCg/bh/ugT0xK2IoKlAfEh1dc7RgY0GJw13mpeC9HsvvyXf2K1qgfCA/3M1ixNVNWIYGTd0Uy8ckc3m7ert2cMRoiIHEFVUqemOBMrf86AAu2gbUZ3Bv3swUODYuXsgilvvKLeQby5CmIZR6GwnzdX/WyJWLZ6YvF+WUFjbKt50Vreq+wivv1uqdxTx1xVN/3aBSPIxwO5JeXYezYf/dpdDZ6oNmbUEBHZu6MrgfndgC9uhfLHR/B+xWxs83pKLj00Zybkg/t7YfbYrnJmwZRP6wfPF8gKGi93Je7u1Qb2Rvwug9qH4vW7r5NBl8LEVvP688zVLM3DTYmbOoXL4w3Hs6/pZzk7BiNERPYeiIichjp9NyKRL3MgGgtI9G/Izw7vgHfv64FvH+2PbTNubPYnfn3i6pjurRBUp5mZPRG/nyjfFYGXKa3m9efpl7/Ess+1Nkq7qXOE/LrhWNY1/Rxnx2UaIiJ7bV5WVIrR6/4BT5nVUJuY0BDvk3M8vsI6VR+Djb9MzQdpTM3E1QlJtklcbXaSa9F1UK37GJ5lWXJJpq66e+qIHBOxtBNeXIDjO6+g64DRzS7/vb5DKNyVCrlXjUi0FRVKVB+DESIiO62S6a88its9M43atVbkOui18PHAwom90L+dacswjVleI3FVdFF1BFeTXEMAr3/rZpjq1NzU7eRaXf6rzzFZtwDY3fzy3wBvD5k7InbxFUs1D5u7QZyT4DINEZGdVcnoK0JMzXXQL8uInIlBCaFmC0Rk4uou3RLNxH72lbhqNBFIiPLdwNqzRGJGRLS7F51c9eW/YgnMYPmvWDJrhhs76ZZqNh7nUk1DODNCRGQnSzNiRqTmIoKpuQ7mXJapaX/aJZzIKoa3hxJ32mHiqkkBSacx1VVJ037OwJridrKTa1PlvzLMWz1Td38Tl2xEEqtoD7/7TL5c7gr0tt98G1thMEJEZAeByOfbU+v1yBA5DBnaYPlJ3dAkh+yP4RuJ8bePw9Nm7I9Rl76c99buUbJU1aFVdXIVywK3KS/it6/3y9mkpsp/ZUBSlI5tG1bCrd31Jj3XIk8kPsxP5o38fjJXJgBTbVymISKy8dLM4Dc24uVfj9X7nshhmFsxqVZuw1UK+SbqdeubuL1nW5PLdI1VWFaBX//UlbhOsFHHVWtU3Ri7JLZs016M/2iX/DczpfS3uqqGSzUGMRghIrKTHBFDRC6DyGnIRHDtb4g9VczcwtyQH/ZfgKpSg06RAegZbdyykaMFJKLc+a9jBpq0JGZqL5Ibq/qNbD6Rc83lws6IyzRERDYo280svCxnQ4x5WxIByXpVH4wKOIMFYy2722zdxNXFyQ6euGoEMaMky3dF1YxIVjWi/Lcqi0Tm+YgS4qZmpUTr/EBvd+SXluPg+UvoHVMnuHRxDEaIiOx8cztF1ZLNbbffC6WZk1Mbs+fsJaRkl8DHww2392wNpyYCO1G+a0T5r564WfxbiuCy5j45hri7KTG0YzhW/pGBDceyGYzUwWUaIiI7WZJpiMhpELkN5q6Sacri3efk19sSo1yjAsSI8l9DtqcYt/RyU+eq1vDH2Bq+Ls6MEBHZoGzXWC+O6YwHB8VZfcfXS6XlWHU40ykTV40t/z15OgWzN+bKpRlDHW71Fmw6jR/2pzdZVn1DhzD57yjKpM/nlyE62NdCv4Tj4cwIEZENynabIkIPsXusLQIRfeJqeaUGXaMC0b1NEFxKVflv/I0P4lxAL6N2RzYmobWFr2d199pNJzg7UhODESIiG5TtNkYfeohP2rYIRDQabXVvkQlOnLjaFPHci38DoalnwNjN9fS7+K7nUk0tDEaIiCzAEXNE9LafzkVqbikCvNxxRw8nT1w1cQfgxtRMaG0qb2TX6TyUlVeadayOjDkjRER2kCMS7OeBF2/tishAb4t1UjXWVzt1iat3924DPy++Teh3AH5n3Uks2JTS4Hn63X49jqUDys4Gy6/jw/zRuoUP0gsuY9eZvOp9a1wd/8qIiMzcQ0RUVxg7I6IPOV678zqbzYTUJN4k1x/TdQm9v78LJa42QQSHYvPBhoKRWrv97oXuElh/t1+x5DW0Y5hcBttyIofBSBUGI0RENuwhYqnN7Zrr291psq/GwPgQJIQH2Ho4dkXMWImkYpGsWnPWS7/bbz363X7rdMoVVTUyGDmZY52BOwDmjBAR2Sg/RJTtilbk9hKIqCrVWLJHl7j6QP8YWw/HIRJam97tV/yBzAQ06upbByaEwsNNgbN5ZTibW2qVsds7BiNERFbOD7F12W5DVh/ORG5JOSICvTCiC5cPjElo1e/22/A/o263X9G3RM/fyx19qjqwcnZEh8s0RERW7CFi67JdYxJXJyTFyPbl1HhCq8gPksmqIj+kKSW1d+u9oWMYdp7Jw+YT2Zg8MBaujn9tRERW7CFi67LdhhzNKMLec5fgrlRgfFK0rYdj90QgKfaj6dO1s1HnHynyqdV/RCSxCiIguVJxdQnHVZkcjGzduhVjx45FVFSUzApevnx5o+f/+OOPGDFiBMLCwhAYGIgBAwZgzZo11zJmIiKHyxGZNiwB3z7a365yRGr6apduVmRUt0iEBzbdU4OqiPJdUTXTQFs0EX9kaEMw9meNDF71HVo7RgTI5bArFZpG+5K4CpODkdLSUiQmJmLhwoVGBy8iGFm1ahX27duHYcOGyWDmwIEDzRkvEZFD5Yjo80OeHdFBfpK2t6UZoehKBZYfSJfHTFxt5m6/Uu1/27q7/dZsGS8+zIuqGmEL80ZMzxm5+eab5cVY8+fXLnd67bXXsGLFCvz888/o2bOnqQ9PRGRT4lOsqT1E7DE/pKZle87jcoUaHSL80S+OW9s3e7ff1TOAooxau/2KQES/26+ITcRfgQhmRc7J0I7hWLb3ggxGXoRrs3oCq0ajQXFxMYKDG/6DV6lU8qJXVFRkpdERETVAlGae2wGPY8fQX1nY5E6u9thDxJBKtQafbT8rj6cMinPZfWjMtdvvkZ2r8b9fdyAbLQz+jdRsGS+aqIkgNSW7BBculaFNS9fdxdfqwchbb72FkpIS3HvvvQ2eM2/ePMydO9eq4yIiatDRldWfevsAWOIp8gCCMbdiUvWnXkM9ROytdNeQNUeyZNfVYD9P3NnTtfehuWZKN6T49cBKTdOnZhdfwQCfEPRq2wJ7zl6SsyMT+7nuEplVq2kWL14sg4xly5YhPFy3WZAhs2bNQmFhYfXl/Pnz1hwmEVHtQER00awx/S5EIl923RTdNx2hh0hDPtl2Rn69v19beHvU3keFTBceYFzy76msYuw8nYch7XV5I5tPuHbeiNVmRpYsWYJHHnkE3333HYYPH97ouV5eXvJCRGTzpRkxI2IgXVXEGSJBcY7HV1in6iOn4x0lR0Rvf9ol7E8rgKebEvcPcN1P5dZoGV/Xgk2n5SXU31Ne35GSi/JKDTzdXbPjhlV+62+//RZTpkyRX8eMGWONhyQiunaia2adGZGaRLwRpciTXTjtuYdIQz7Zliq/3tYjyuhP9GR6y/jGiI63Qmm5GnvPuW6Jr8kzIyLfIyXl6q6FqampOHjwoExIbdu2rVxiSU9Px5dfflm9NDN58mS8++676NevHzIzM+XtPj4+CAoKMufvQkRk1hLe06dT0MGIc/8+MAgVnfvLT8WOMCMiiIRJ0f5deGhQnK2H45Qt403dOHHziRwMjA+FKzJ5ZmTv3r2yJFdfljt9+nR5PHv2bHn94sWLSEvTbbQkfPjhh6isrMQTTzyBVq1aVV+efvppc/4eRERm7646e2OuUeeLLpz22kOkIV/sOCsDLrE7b5eoQFsPxykDEtHgTjS6Ew3vjLH2iC44dEUmz4wMHToUWm3DK2Gff/55reubN29u3siIiGzYXVW8ymWhk6yaEcmqhuMMha77pujC6UBKVJVYkqwrDHhkCGdFLN0yXlTOGONsXhlyS1QI9Xe9nEnXzJQhIjKiu6pIShXlu/K43mewquhk9Ou6LpwO5Lu951GsqkS7MD8M7dBwZSOZhyn5OL+7aDdWBiNE5PJEECLKLN9Zd7LeGr/oIzK14hlkok6jRjEjIrpuimZXDqRCralOXBVNzpQOtLTk6BU2xjzTz/10qHr/Gldi9aZnRET2RLzwN5VoKAISUb4rqmbCUYB7h/XB4Jtuc7gZEWHFwQxcuHRZlpTe06uNrYfjUhU2YvlPBCSNlfxertDg8a/34wMHqsoyB86MEJHLMmX3XbFks0vTBSs1A+HW7nqHDETEDND7m3TVkI8MaQcfT8f7HRy9wkaUfxtj7s9H5b+Xq2AwQkQuydTdd2t2VxXT7o7o10MXcSa3FC18PXA/d+e1WYWN2CqgKRer9q9xFQxGiMglmbL7ruBo3VXr0mi0WLgxpbqviL8XV+ltQfzthAYYVy2TbWQVjjPgXyMR2QXxwns6uxRl5ZUoK1fjcrkaqko12ob4IbFNEFr46tpmm2NGRAQiv5mYJOgIO/A2Zt2xLJzIKkaAlzsmD4y19XBcmrHVNeEu1BWXwQgR2UxaXhnWHMnE6iOZcp+URloYIS5UF5T0iQ2W7csDvT0skqxa17Rh8RiUEOZQ3VXrEr2h3tt4Sh6LQCTIx/Tnjqy7f02wn4fDLgc2B4MRIrL6G+Pao1l4d/0pHL1YVC/gCPB2h4+HG3w93eDupkRKdglSc0urL8sPZmDeqmP4S59oPDgwFrGhfiY3MzOGomo25NkRHR02CNHbfDIHh9OL5HP60GA2OXOE6pq4UD/88meGnB1x5EDYWAxGiMiqMyFzVh7Gpqrt0sULbL+4YIzuFomRXSIbrDQoKCvHHxcKcTCtQL5An8ouwec7zuKLnWdxU6dwTLuxPXpEtzBbsqqj54fUmxXZoJsVEUmrwX7mWe4iy+xf4+vpJpcp950rwL5zB+VtrRx8idAYCm1jvd3tRFFRkdxUr7CwEIGB3EOByNFcqVDjgy2n8f7m03KbdA83BR67vh0eGdwOLU18cxQvWdtScvHpttTqoEahACb2a4t/ju5kcPlGNDQb/9Euox/DmV78t53Kxf2f7IaXuxK/zxjmUnkIjkCfw5RdfAVnc8vwzvqT9c7Rh8OOtCO0qe/fnBkhIos6n1+GKZ/vkcstwuCEUMy9vSviw/yb9fMUCgWGtA+Tl9M5JbJC5McD6fh6VxrWHMnCv8Z2xS3XRcrz9C/221OM2/Bu0oAY3NytldNMi4sKmnm/HZPHE/q1ZSBix/vXqDVauTmjIWLGQPw1ilmUEV0ineJvsy4GI0RkMUcyCvHgZ3uQU6xCeIAXZo/tgjHXtaoOFK6VCGjeHtcD9/Rpgxd+Oix7aDyxeD+GdQzDG3d3l0mxpiSsikBEvDE4ixV/pONIRpGsoHnyxva2Hg5dQ6m5tkbvEWf6G9VjMEJEFiFmI/761T65Q2ynyAB88VASIgIt88l8YHwoVj09BIs2n5YXsXwz/O0tKLpSaVKyqjNVL4ilsbfW6Kb8pw6LZ66Incs2sqeIs/YeYdMzIjK7FQfT8eBnyTIQ6d8uGMseH2CxQETP28MNz47ogFVPD0aHCH+TAhFnSVatSST4phdcRlSQt2xyRvYt3MV7jzAYISKzWpKchqeXHESFWosx3VvJGZHm9ARproTwAMy6uel223piRsQREwMbk19aXt1t9f9GdpSBGjn2zr4KACF+nsgsvCwTsp1t3xou0xCR2fx+KgfPLz8sj0UPkNm3drHJFvVFVyqMbmjmDH1E6vrvhlMoVlWiS6tA3Nmzta2HQ2boPaIFkFdajmeX/eF0FV8CZ0aIyCxSsovxt2/2y09sd/VqLV8orR2IiMcWnxpPZekqd5oiOqs6WyByNrcUX+86J4+fH9PZJsEgWX5n38zCKzJwEc38nAFnRojomuWVqGT5bvGVSiTFBmPeXdeZrWLGUq3eHXn33ca8sfo4KjVaDO0YhkEJobYeDjUjIBnRJRK7z+Thr1/vk/+nXKHclzMjRHRNxGZ2omrmfP5lxIT44oMHesPL3bo5CvpW76bsOSM6vzr463c9649m4bfDmfL3mnlzJ1sPh5rJTanAwIRQJLZpuKtw3XJfR8dghIiaTXRDnfnDIew9d0nuKfPJ5L5WLyE1tdW7n6cuUBJ73Lyw/LDTJAIWXq7A88sPyeNHh7RDp0h2q3Z0MSG+LlPuy2CEiJpt2d7z+OlAuvwkt2hibySEN6+rqiWbRdVMVv320f7481+jZD6LWEX6Zncanvhmv+zJ4ejE5oFZRSq5wZoocSbHNzA+1GXKfRmMEFGzN7176eej8vgfozpicHvr5yfoWr3r9qdpSvuIANm5UgROUwbFYcH4XvB0U2L1kUzZE8XYChx7bTC3ZM95eSw6z7KU1zmM7hYJ90bWEhVOlPvEYISImhUE/N93B1FarpYJq2JZwNpEnojYy2PBptPN+vQoeqB8PqUv/L3csetMPsb9bxeyixxvurtUVYmZP/5ZvbeOM7wxkY4InK/vEOoSzfoYjBCRyT7+/Qz2nL0k8y/+c2+i1V8MTUlYbezTo0gSXPJYf4T6e+HYxSLctWgHzuQYVxZsL/695oRMHm7dwkfuWkzO5f7+MfKrW53qtIhALzwzvD1UlRqnaILG0l4iMol40/7PWt2eJ2Lju+hg45LsbJGwasynx26tg/Dj1IF44NPdOJdXhrsX7cAnD/ZFr7YtYe9EvswXO8/KY1FOLWZ5yLn0iwuBh5tCdjSeP66HzHU6m1uGb5PT8M76U9XnOXoTNM6MEJFJZbzPLj2IcrUGwzuH494+0XabsGpKq/e2Ib74YepAdG8ThEtlFZjw0S5ZJmvPMgouyyZzWi3wl95tcH2HMFsPiSzAz8sdPasCY7HXk5e7EvPXn0RmnSVFR2+CxmCEiIz23oYUHM8sluW78+7qbvXGZqaUMU4bloBtM240+pOiWKoR1TaiWdiVCg0e+2ovFu9Ogz26XK6W48stUckdkf91W1dbD4ksaEhV8zqx3UJDs4L628T3HXHJhsEIERklNbcU/9uqSxZ99Y5uCAvwsu4ANGog9Xd0zl2L/sqjUELT6Omi+6ipuSziU+hHk/rg3j5tIF7Pn/vpkOxoqrGjF3fR2+WfP/yJw+lFMigU4xXjJuc1uKpSbdup3EZnBR25CRr/gonIKC/9fESuW9/QIUyWHFrV0ZXA6hlAUQZEB40lnkCGNhhzKyZhjSap1qmKquWZ5laVeLgpZXlsZKA3/rsxBYs2n8aprGK8M64HAqy4+3BDFm05jZ//yJAln4sm9rJ6zg5Z33Wtg2RTwYZawztDEzTOjBBRkzYcy8KmEzkykU7XMExh3UBk2SQZiNQUiXws8piPUcpks5c7it9v+siOeGdcIjzdlVh/LBt3vb8D5/JKYUsij0VUzwhiaaZfuxCbjoesw91NiYHxxv9bO2ITNAYjRNQo0Z30pV90zc0eGhyHdmH+1l2aETMiBlbJZayhAOZ4fFW9ZGNswqqx7uzZBt/9dYAsozyVXYLbFmyXDcZsYcvJHDy15IBMWJ3Yr211ySe5hsHtdQnKnm6K6qDbmZqgMRghokZ9si1VlryGB3jhyRvbW/fBz+2oNyNS9wUsSpGHL2+qlMmnpiSsGisxugVWThssv4r9Xx74ZDf+veY4yisbz1kxp58OXMDDn+9BWblaVs3MGcuEVVdNYlVrr+7YW5O4LoLyd/sVw+3IDzK/SgbzzhqMbN26FWPHjkVUVJScyly+fHmT99m8eTN69eoFLy8vJCQk4PPPP2/ueInIii4WXsaCjSny+LlbOlu1j4WoCDh5WvfYTRkcqa5u9W4JEYHeWPpYf1lCK3JZF246jTvf346TWcWwtA+3nsazS/9ApUaL2xKj8PGkPnLpiFxv07w2LX3k/4snhsXLWcCaxvkfxOGWf0fS1snADw8DX9wKzO+mW+Z0ACb/RZeWliIxMRELFy406vzU1FSMGTMGw4YNw8GDB/HMM8/gkUcewZo1a5ozXiKyotdWHcflCjX6xLTE7T2irN7qffZGI5dE/CMsPSS538u//5Iok0Zb+nrgSEYRbn1vm+xGa4lqG/EzX/nlqPw3EB4eHCebXjEQcU0KhQKDq2ZHROm5mAUUs4Hv3tcDa0YVYF7lv+FzObP2nYou6vKtHCAgUWhFnVhz76xQ4KeffsIdd9zR4DkzZszAr7/+isOHD1ffdt9996GgoACrV6826nGKiooQFBSEwsJCBAZyW2wia9h3Lh93L9opOz7+PG2w7FRqzVbv2qpp521eT8lkVcOTHgogMAp45hCgtN7mcGIPmxk//CmTevXVDtNHdsDQDmFmSe49klEoNyHcXVWi+dwtnfDY9fHX/HPJsf3yZwamLT6AjhEBWPPs9bobxVKMmAFpcDnTNv9HTH3/tniIvXPnTgwfPrzWbaNGjZK3N0SlUslfoOaFiKxHfEZ5c7WuamNcn2irBSJ1W71roJTlu/K43semqjf90a9b/UU2PNAbnz7YF6/deZ3cn+dQeiGmfLYH93ywEztONz/BNa9EhVk/HsLY97bJQER023z73kQGIiQNjA+VHw5OZBVf3dSxibwqmWFSlK47z45ZPBjJzMxERETtKVRxXQQYly9fNnifefPmyUhKf4mOtn7LaSJXti0lV74Zerop8dRN7W3a6l30EZla8QwyUadCQHzau/dLoMttsAUxAzKhX1ts/ecwPHZ9Oxk47Dt3CRM+2o17P9iJz7enIi2vzKjA70RmMRZuSsHQtzbLPUdE4HVr91bY+PehuKtXG6v8PmT/gv080TVKN7uwXR/0lhi5bYGx59mIXTY9mzVrFqZPn159XQQuDEiIrEO8Ob5V1ctClI9GtfCx2mM31KxJBCTrVH2QpDyOcBTg3mF9MPim22wy7VxXiL+XTO59ZHCcDCgWJ6ch+Wy+vPzr56OID/PDsI7hiA31k/keXlWXosuVchZl++k85BSrqn+eeLMR1TKOWJ5Jljc4IUx23/39VK4sPTc6X8oKeVV2HYxERkYiK6t2RCaui7UjHx/DL3Ki6kZciMj61h7Nwh8XCuHr6Ya/DYu32vKMmBU5lVXS4DliyWaXpos8Ht+uv10EInWXbube3g2P3RCPX/7IwMbj2dh77hJO55TidE5qo/cVwYkIPsYmRuHuXm0sVhVEjm9I+1B8sOW0bA0vPjgoYgbqZglFsqrBXWuqckbEea4cjAwYMACrVq2qddu6devk7URkX0RQ8J+1ulmRKYNi5eZx1khYFXkixuzEe62t3q2hdQsf/PWGeHkRfUnEm8a2lBzkl5ZDVamR/UnERQQcfWJbyj10erVtKat1iJrSO6alDF6zi1WyEV+HiABg9Bu6qhn5P0RrF3lVFg9GSkpKkJKSUqt0V5TsBgcHo23btnKJJT09HV9++aX8/uOPP44FCxbgn//8Jx566CFs3LgRy5YtkxU2RGRfxJ4nJ7NKEOjtjseGxFu1cqYp5mr1bk1BPh4Y072VvBCZg7eHmwzGxTKNCHRlMCLypkT+VNX+TdXEjIgIRGyUV2XRYGTv3r2yZ4iePrdj8uTJspnZxYsXkZZ2ddvtuLg4GXg8++yzePfdd9GmTRt8/PHHsqKGiOxHhVqDt9edlMfiU32Qr4dVK2eaImZERCBi7g6rRI5mcEKoLhhJyZVbNEgi4Og0Rlc1I5JV/SOgjh6A5HOFyD6YLverEUGMvQbyJgcjQ4cOletUDTHUXVXc58CBA6aPjois5ru9F5CWX4ZQf088ODDWJpUzhkwbFo9BCWF2/UJKZE2D24cCvwG7zuTJJb/qRnhiKSZuyNXlz39vqfV/rJUdB/Rs5UdE8gVtwcZT8vhvQxPgZ4W278Zuc94+IsCird6JHE3nyECE+HnKvYoOni9ocPmzbrCfWXhF3i6+b28YjBARVhxMR0bhFZmwKnpnWJpYosmtUc7qbNuhE1mSUqnAwKrW8NtO6boAG7P8qb9NfF+cZ08YjBC5OPGitGjLaXn8yJA4i1d16PedefnXY42e58jboRNZaxff31NyTVr+FCGI+L44z57YZdMzIrKetUcycSanVFbQTLTwrIix1TOOWDlDZE2DRN4IgD/OF8gSclG5Zcryp7HnWQtnRohcmEhGf3+zblZk8sBYBHh72EX1jKicWXR/L7tMtCOyl3427cL85NYBO0/nmbysaW/LnwxGiFyYKA0Um7z5eLhhyqCqEkEbV8+8OKaz3B6dgQiRcUs1oqmenljWFMubCgdb/mQwQuTCxF4qwn1J0XITLksydlo4NMCLSzNERhjSPkx+FT1H9MT/HbG8KSgcaPmTwQiRixI7zO46kw8PNwUeHdLOosszYhq5sX1n7Hn6mMhe9Y8PgbtSgXN5ZbV2iBazimKZUyx3OsryJxNYiVzUos26WZG7erax2M68zrbvDJE98fdyl/saiR2if0/JwcSQmOrviYBjRJdIuTwqZiX1HVgF8eGg5m32MEvCYITIBZ3ILMb6Y9lQKETrd8vMijj7vjNE9tKNNVkEIydzMbHf1WBEEP+XRMPAxj4c2EtXVi7TELmgj38/I7/e3C0S7cL87WLfGXudPiayZ0OqSnx3nM5FpVrT4Hn23pWVMyNELianWIUVB3U7ez5ioVwR7jtDZB3d27SQPYKKrlTiz/RCuWxTV1NdWcX/PPF9saxjq/+HnBkhcjFf7TqHcrUGPdu2MPjCZQ7cd4bIOtyUCgyqbg1fuxurI3VlZTBC5EKuVKjxza5z8vhh/dbjFqmcKTbqfFbOEF27qyW+tfep0XOErqxcpiFysQ3x8krLZffG0V0jzfqzWTlDZNu8kQNpBSi+UlGvk7IjdGXlzAiRC7V+/2Rbqjx+cGAs3N3M99+/oeQ4Q1g5Q2Re0cG+iA3xRaVGK3sH1eUIXVkZjBC5CNGl8WRWCfw83TAuKdpsP5eVM0T2UeIrbDOwVOMIXVkZjBC5CP2syF/6RCPQjBviGV85k4BvH+3PfWeIrNQaviZ778rKnBEiFyASSreczJFNzqYMirVR5Yx/rQZMRGQ+A6qq0s7kluLCpTK0aelb7xxDXVl7x7SUW0OIfDJbdmRlMELkAj7drpsVGdE5AjEhfmZdosktVhl1LitniCwn0NsDPaJbyMBClPjel9TW4Hk1u7KKXK8b/r3JLjqycpmGyMkVlJXjx/3pZi/nFS9kg9/YiJd/PdboefaQHEfkSlU1vzewVGPPHVkZjBA5uWV7z0NVqUGXVoFmCwiMrZ6xl+Q4IlfKG9mWkitnLRvSVEdWQXy/sZ9hbgxGiJyYeDERHVeFSQNioBBJI1asnrGX5DgiV5DYJghBPh4ovFyBPy4UNHiePXZkZTBC5MS2nMzG+fzLcu+K23u0tmr1zItjOrNyhsiK3N2UGFzVGn7LCcPdWO21IyuDESIn9sUO3azIuL7R8PF0M8vPNPYFKjTAi0szRFZ2fQddMLK1gdbw9tqRlcEIkZNKzS2tLue9v3/MNf887jtDZP+u76DLG/njfIFMXjfEHjuyMhghclJfV+WKDOsYfs3lvPrKmfEf7cKCTacbPZfVM0S20yrIBx0i/CFyT0UiqyH22JGVwQiREyorr5RVNPrE1WvBfWeIHMv1VVU1jeWN2FtHVjY9I3JCyw9koPhKpdw8S//CZK19Z2zRMImIrrqhYxg+3pYq80bEBpkNVdEZ6sjKDqxEZBbixefLnWflscgVUV7DC4sp+84MSgi12QsZEV3VNzYY3h5KZBWp5OaYHSMD0JCaHVltics0RE5mz9lLOJ5ZDB8PN/yld7RV951hIEJke94ebujfLqS6vN8RMBghcjL6Jmd39IxCkO817M6rUSOh9CBuU+5Af+VRKKFp8FRWzhDZl+urlme3nmy6NbzDBiMLFy5EbGwsvL290a9fPyQnJzd6/vz589GxY0f4+PggOjoazz77LK5csV4zFSJXkVuiqt5TYmK/a0hcPboSmN8NXddNwH89F2CJ5yvY5vUURilr/19n5QyRfZf4Jqfmy4R2pwtGli5diunTp2POnDnYv38/EhMTMWrUKGRnG54KWrx4MWbOnCnPP3bsGD755BP5M5577jlzjJ+IahAVNBVqrdy9s1vroOYHIssmAUUZtW6ORD4WecyvDkhYOUNkv+LD/NC6hQ/K1RrsPmO9tu5WC0befvttPProo5gyZQq6dOmCDz74AL6+vvj0008Nnr9jxw4MGjQIEyZMkLMpI0eOxPjx45ucTSEi02g0WizenSaPJ/Zr28wfogZWz6ixXdZV+nhjjsdXcsmG+84Q2S+FQlE9OyKaHzpVMFJeXo59+/Zh+PDhV3+AUimv79y50+B9Bg4cKO+jDz7OnDmDVatW4ZZbbmnwcVQqFYqKimpdiKhxW07l4MIl3T40YxOjmvdDzu2oNyNSNyCJUuTh57FK7jtDZOduqApGtjpAMGJSaW9ubi7UajUiIiJq3S6uHz9+3OB9xIyIuN/gwYNlyWFlZSUef/zxRpdp5s2bh7lz55oyNCKX980u3azIPb2jZTZ9s5RkGXVa18DLV6dKiMguDUzQVbidyS3F+fwyRAf7wmWraTZv3ozXXnsN77//vswx+fHHH/Hrr7/i5ZdfbvA+s2bNQmFhYfXl/HldJ0kiMiy94DI2HtcFEhP7t232vjPbMo0MYvxrfyAhIvsT6O2B3m1byuPNdj47YtLMSGhoKNzc3JCVVfvTk7geGRlp8D4vvvgiHnjgATzyyCPy+nXXXYfS0lI89thjeP755+UyT11eXl7yQkTGWZqcJveiGNAuBPFh/ibdV1TfiC6rormZEu7Y5hWMSEV+A59UFEBgFBAz0EwjJyJLGtopDMln87HpeDYeMMOGmXYxM+Lp6YnevXtjw4YN1bdpNBp5fcCAAQbvU1ZWVi/gEAGNIJZtiOjaVKg1WLLnfLNmReruO6OBEnMrJsn8VRHc1Fa1LDP6dUDZzGUgIrKqYR3D5dcdp3NxpUINp1mmEWW9H330Eb744gtZqjt16lQ50yGqa4RJkybJZRa9sWPHYtGiRViyZAlSU1Oxbt06OVsibtcHJUTUfOuPZiG7WIVQfy+M7GJ4htKUfWfWaJIwteIZZKJO7xAxI3Lvl0CX28wzcCKyuE6RAbIX0JUKDXadyYO9MnlvmnHjxiEnJwezZ89GZmYmevTogdWrV1cntaalpdWaCXnhhRdkiZH4mp6ejrCwMBmIvPrqq+b9TYhc1DdV5bzj+raBp7vSLPvOiIBknaoPkpTH8dKNoegQn6BbmuGMCJFDUSgUGNoxHN8mp2HziRx5bI8UWgdYKxGlvUFBQTKZNTAw0NbDIbIbZ3NLMfStzRCbcm79xzCTsuVXHEzH00sONnneu/f1wO09Wl/jSInIVtYeycRjX+1D22BfbPnH0AZ38bXl+zf3piFyYOLTjjC0Q5hJgYhYosktVhl1LvedIXJsgxJC4eGmQFp+GVJzS+EUyzREZB9UlWp8t++CPB6f1LZZ1TONEZ+dRJdV7jtD5Nj8vNzRLy4E21JyselEDtqZWHFnDZwZIXJQa45kIb+0HJGB3rixU3izqmcawn1niJzL0I66bqybTxjeR87WGIwQOahvqxJX7+0bDXc3ZbOrZwzhvjNEzmVY1QcWsWleqcr+dvHlMg2RAzqTU4KdZ/JkR/ZxfaOvuXqmphfHdMaDg+I4I0LkRNqF+skEVpE3sj0lFyO7Gt8GwBo4M0LkyImrHcPlNuHGyC5uOhARQgO8GIgQORmFQoFhVUs1Im/E3jAYIXIwoovi91WJqxOMSFzV7ztzKqvYqJ/P6hki516q2Xwi2+46oHOZhsjBrDmSiUtlFbKroj4p7VorZwRWzxA5t/7tQuDtoZSvByeyitEp0n76dnFmhMjBLK7uuNp44qqxlTMCq2eInJ+3hxsGxofK403H7WuphsEIkQNJyS7B7tT8JhNXTamcEVg9Q+QahunzRo7bV4kvl2mIHDBxVfQVaRXkc82VM9OGJcjujGJphjMiRC6SN7LiCPaey8el0nK09POEPeDMCJEDJa7+sL8qcbVfW7NUzrSP8MeA+BAGIkQuok1LX3RuFQiNVlTV2M/sCIMRIgex+nAmCsoqZCnvDR0a7rjKfWeIqDEjOuteP9Yfy4K9YDBC5ICJqw3NZIik1cFvbMTLvx5r9GeJe4tqHFbOELmemzpHyK9bTuTIPa7sAYMRIgcgeoQkn82XQci9fQwnrnLfGSIyxnWtgxAe4IXScjV2ncmHPWAwQuQAFtdIXBWVL3Vx3xkiMpZSqaieHVl/1D6WahiMEDlA4uqP+9MbTVw1Zd+ZbTNuZCBC5OJGdNHljWw4lmUX3VgZjBDZuVWHLqLwsi5x9fr2hjuuct8ZIjKFaH7m4+GGjMIrOHqxCLbGYITIQRJXxyfVT1zlvjNE1NxurEPa67qxrj9q+xJfNj0jsmMns4qx99wlGYT8pU7iKvedIaJrMbxLBNYezZIlvk8Pbw9b4swIkQPMigzvHI6IwKuzGtx3hoiulUiIVyiAQ+mFuFh4GbbEYITITl0uF4mr+o6rMdW3c98ZIjKHUH8v9IxuIY83HLPtUg2XaYjs1C9/ZqDoSiXatPTBkATd2q7AfWeIyJxLNfvTCuRSzf39r37osTbOjBDZqW+qlmhEOa/oC6DHfWeIyFxGVPUb2ZGSh1JVJWyFwQiRHTqcXoiD5wvg4Va/46qxFTGsnCGipiSE+yMmxBflag1+P5ULW2EwQmTHHVdFnodY15U0aiD1d/Qr3YhbAlLgBo3B+3LfGSIylkKhwKiukegb2xJeHrYLCZgzQmRnSlSVWHFA13F1or7j6tGVwOoZQFGG/ATxPoAMr2C8VDEJqzVJ1fdl5QwRmWrWzZ1kUGJLnBkhsjPLD6TLDaziw/zQT8xuiEBk2SQZiNTUSnEJizznY5Qyufo2Vs4QkalsHYgInBkhsiNijwh94urEfjFQaDW6GREDhbwKaKGFAv9tsRSrRzyM8EA/Vs4QkUNiMEJkRw6cL8Cxi0Xwclfi7l5tgHM76s2I1A1IvMou4vYW54C4IVYdKxGRuXCZhsiOfLNLNysyNjEKQb4eQImR23sbex4RkR1iMEJkJwrKymWjM33iqui0eqTIx7g7++t6BRAROSIu0xDZie/3XYCqUoMurQKRWXgZf/tmP7IKNdjmFYxI5MNwKogCCIwCYgZaf8BERLacGVm4cCFiY2Ph7e2Nfv36ITn5aja/IQUFBXjiiSfQqlUreHl5oUOHDli1alVzx0zkdDSaq4mrPaKD8LdvDsiW7xooMbdiku6cejmsVdHJ6NcBpZuVR0xEZMNgZOnSpZg+fTrmzJmD/fv3IzExEaNGjUJ2tuFNdsrLyzFixAicPXsW33//PU6cOIGPPvoIrVu3Nsf4iZzCtpRcpOaWwt/LHRuOZ9eqnVmjScLUimeQiTpNzMSMyL1fAl1us/ZwiYjMSqEVtYQmEDMhffv2xYIFC+R1jUaD6OhoPPnkk5g5c2a98z/44AP8+9//xvHjx+Hh4dGsQRYVFSEoKAiFhYUIDAxs1s8gsmePfLFXblQlOiGuOZJp8BwlNEhSHkc4CvDXMQPRdcBozogQkV0z9v3bpJkRMcuxb98+DB8+/OoPUCrl9Z07dxq8z8qVKzFgwAC5TBMREYFu3brhtddeg1qtbvBxVCqV/AVqXoic1YVLZdh4XFcN0ztGt523IWLJZpemC1ZqBiLFrwcDESJyGiYFI7m5uTKIEEFFTeJ6ZqbhT3NnzpyRyzPifiJP5MUXX8R//vMfvPLKKw0+zrx582Qkpb+ImRciZ7V4d5rMB+kWFYiiy8btmslN8IjImVi8tFcs44SHh+PDDz9E7969MW7cODz//PNy+aYhs2bNklM6+sv58+ctPUwim1BVqvHVznPy+HBGERZsSmn0fG6CR0Rw9dLe0NBQuLm5ISurdoMlcT0yMtLgfUQFjcgVEffT69y5s5xJEcs+np6e9e4jKm7EhcjZvb7qGIpVxs2GcBM8InJWJs2MiMBBzG5s2LCh1syHuC7yQgwZNGgQUlJS5Hl6J0+elEGKoUCEyFWIpmZfV5XzGoOb4BGRszK56Zko6508eTL69OmDpKQkzJ8/H6WlpZgyZYr8/qRJk2TZrsj7EKZOnSorb55++mlZcXPq1CmZwPrUU0+Z/7chciBL95xHhbrpYrZpw+IxKCGMm+ARkdMyORgROR85OTmYPXu2XGrp0aMHVq9eXZ3UmpaWJits9ETy6Zo1a/Dss8+ie/fuMlARgcmMGWInUiLXteJgulHntY8IwID4EIuPh4jIYfqM2AL7jJCzyStRod9rG1BZv61qPd8+2p/BCBE5JIv0GSGia7f68EUMfWtzk4EIK2eIyFVwozwiKwcij3+9v8nzWDlDRK6EwQiRFatn5v581OjKGRGIsHKGiFwBgxEiK0lOzZc78TblxTGd8eCgOM6IEJHLYM4IkZVkFzcdiAihAV4MRIjIpXBmhMgKyzNiVuRUVrFR53PfGSJyNQxGiCycsCryRIxZnlFU5YqweoaIXA2DESILBiJTv94PYxr5sHqGiFwZgxEiC1bOGNtRkNUzROTKGIwQ2bByZtqwBAxKCOW+M0Tk0hiMEFlgVmR7So5R57aP8GerdyJyeQxGiGyUsCqwcoaIiMEIkc0SVlk5Q0Skw6ZnRFZOWGXlDBFRbZwZIbJiwqrAyhkiotoYjBBZsdW7qJ55dkQHzogQEdXAYITIiq3eRRkvAxEiotoYjBA1E1u9ExGZB4MRIgtXzugxYZWIyDAGI0QWbvXeigmrRESNYjBCZKHKGV9PN3xwf2/miRARNYHBCJGFKmdu6hyO6zuEWXw8RESOjk3PiExcosktVhl17p092lh8PEREzoAzI0QWqJ7x93LHDR05K0JEZAwGI0QWqJ6ZexsrZ4iIjMVlGiIzV8+M7hqBu3tHW3hURETOg8EIkRn3nfH2UOLVO6+z+JiIiJwJgxGihmjUQOrv8Dj2A/orj0IJTZN3mdgvBiH+XlYZHhGRs2DOCJEhR1cCq2cARRnoA2CJJ5ChDcbciklYo0kyeBd3pQKPXd/O6kMlInJ0nBkhMhSILJskA5GaIpGPRR7zMUqZbPBu4/pGIyLQ20qDJCJyHgxGiOouzYgZEQPpqvrimDkeX9VbshGVM4/fEG+tURIRORUGI0Q1ndtRb0akbkASpchDkvK4vO7hpotQxidFIzrY12rDJCKCqwcjCxcuRGxsLLy9vdGvXz8kJxuetq5ryZIlUCgUuOOOO5rzsESWV5Jl1Gl/HxiEGaM7okKthZe7Ek/e2N7iQyMiclYmByNLly7F9OnTMWfOHOzfvx+JiYkYNWoUsrOzG73f2bNn8fe//x1Dhgy5lvESWayXyM7TediW6WbU+b27dMKvhy7K48kDY5krQkRkzWDk7bffxqOPPoopU6agS5cu+OCDD+Dr64tPP/20wfuo1WpMnDgRc+fORbt2rDYg++uuOviNjRj/0S5M2uAuq2YaLuJVAIGtsbokHofTi2Tbd+aKEBFZMRgpLy/Hvn37MHz48Ks/QKmU13fu3Nng/V566SWEh4fj4YcfNupxVCoVioqKal2ILNnmXd/UTAOlLN8V+auaejmsuvwQ9cjX8Z/1KfL4ocFxCPbztPawiYhcNxjJzc2VsxwRERG1bhfXMzMzDd5n27Zt+OSTT/DRRx8Z/Tjz5s1DUFBQ9SU6mq21yXpt3kUfkakVzyATwbW/ERgF3Psllqt6ISW7BEE+HnhkSJw1h0xE5JQs2vSsuLgYDzzwgAxEQkNDjb7frFmzZF6KnpgZYUBC1mzzLgKSdao+smrmpRtD0SE+AYgZiHKNAvPf3izPEcszgd4eVh41EZGLByMioHBzc0NWVu2KA3E9MjKy3vmnT5+Wiatjx46tvk2j0a3Gu7u748SJE4iPr7/e7uXlJS9ElpwV2Z6S0+g5Yslml6YLjoX2QIe41vK2pbvP4nz+ZYT6e2HywBgrjZaIyLmZFIx4enqid+/e2LBhQ3V5rgguxPVp06bVO79Tp044dOhQrdteeOEFOWPy7rvvcraDbJYnIpZnjN38LjxAVylTeLkC76w/JY+fuikBvp7cTYGIyBxMfjUVyyeTJ09Gnz59kJSUhPnz56O0tFRW1wiTJk1C69atZd6H6EPSrVu3Wvdv0aKF/Fr3diJrJqzW769an0hXjQzyRlKcLndk4aYU5JeWIyHcHxOS2lp8rERErsLkYGTcuHHIycnB7NmzZdJqjx49sHr16uqk1rS0NFlhQ+QoCauGVHV+x5yxXWSr93N5pfhse6q87fkxneHuxr9xIiJzUWi1WmNem21KJLCKqprCwkIEBgbaejjkoERTM9FLxBitgrxlIDK6Wyt5ferX+/Db4UwMaR+KLx9Kkp2EiYjIPO/fXPQmpydmRETlzG+HdR1TmzJtWAKeHdFBzogIu8/kyUBEXH1hTBcGIkREZsZghJyaqcmqwqCE0OpARKPR4pVfj8nj8Ult0TEywGJjJSJyVQxGyGmZkqxqKGFV+OlAOg6lFyLAy13OlhARkfkxC4/g6smqhhJWhRJVJd5cc1weP3FjguwtQkRE5seZEXK57qqGRNZJWBXeXnsSWUUqxIT44sGBsRYaKRERMRghJ+2ummvUuZMGxODmbq3k0ox+RkQ4nF6Iz3foSnlfvr0bvD3cLDZeIiJXx2CEXDphVQQiA+JD6gUzz/90SO7aOzYxCtd3CLPQaImISGAwQk7jWrqr1vTN7nP444IuafXFMZ0tMlYiIrqKCawEV++uWlNW0RX8e/UJefzP0R0RHqjbl4aIiCyHMyPkcgmrhpJV9V7+5SiKVZVIbBOECf24Ky8RkTUwGCEX664aj2dHdKw3IyJsPpGNX/68KDutvnrndQbPISIi82MwQi7WXTXMYJBRWFaBmT8cksdTBsWhW+sgs46ViIgaxmCEXLa7ak3/+vkIMouuIC7UD/83kp1WiYisiQms5JLdVWv67dBF2fZdfOs/9ybC15MxOhGRNfFVl1yyu6pedvEVPPeTbnlm6tB49Grb0qxjJSKipjEYIYdiju6qelqtFs/9eAiXyirQuVUgnr6JyzNERLbAYIRcqrtqTd/tu4D1x7Lh6abEO+MS4enOVUsiIltgMEIu1V1VLzW3FC/9fFQeTx/ZAZ0iA804WiIiMgU/CpLLdFfVu1yuxtSv96FEVYmk2GA8OqSdWcdLRESm4cwI2X1DM5EjYo7uqnqzVxzG8cxihPp74r0JPdncjIjIxhiMkNM0NGusu6resj3nZa6IOOW/43signvPEBHZHIMRcviGZk11V9U7klGIF1cclsf/N7IjBsaHXuNIiYjIHBiMkEM3NDM2YbXwcgX+9s1+qCo1GNYxDFNviDfLeImI6NoxgZXsh0aN4ztXoW/xRvRXHoUSGrMkrFaqNXh26UGcyytD6xY+eGdcDyiZJ0JEZDc4M0L24ehKYPUMdC3KwH89dTdlaIMxt2IS1miSmp2wKhqbiZmWjcez4eWuxKL7e6GFb9UDEBGRXWAwQvYRiCybJEKHWjdHIh+LPOZjasUz9QISkawqckQa6q6q98m2VHy16xwUCmD+uB7o3qaFxX4NIiJqHi7TkG1p1HJGpG4gIuhjjDkeX1Uv2YibWgV5y6oZ0V21sUBEJMK+uuqYPH7+ls64+bqGy32JiMh2GIyQbZ3bARRlNPhtEWtEKfKQpDxuVH6I3oG0S3h6yUFotcAD/WPw8OA4Mw+ciIjMhcs0ZNOGZh7HjqGPEeeHo8CohmbC6ZwSPPrlXlk5c2OncHkfhVinISIiu8RghGza0Ky/shBLjMgn/euYgXhnwI1NzoiIPWfGf7gLuSXl6BoViPfG94S7GycAiYjsGYMRsmlDs2RNJ1k1I5JVDccZCiAwCl0HjL6aRNKAc3m6QCS7WIWOEQH48qEk+HnxT5yIyN7xIyNZZUlm5+k8/HQgHc/9dKhWqqoGSlm+K4/r5bBWBR+jXweUbo0+xvn8MhmIZBZdQftwf3zzaD+E+HuZ+TchIiJL4MdGsvkeM6JsV5TvzvH4ElHIv/qNwChdINLltkYf48KlMtz34S5kFF5BuzA/GYiEMhAhInLumZGFCxciNjYW3t7e6NevH5KTkxs896OPPsKQIUPQsmVLeRk+fHij55PzLckYs9mdCEgGq/6L+8pfwN4+/wYm/wI8c6jJQOREZjH+8sFOpBdcRlyoH759tD/CA7j5HRGRUwcjS5cuxfTp0zFnzhzs378fiYmJGDVqFLKzsw2ev3nzZowfPx6bNm3Czp07ER0djZEjRyI9Pd0c4ycn2mNGLNns0nRBRee7gbghTS7N7Didi3s+2CGDnfgwXSDCXXiJiByPQiv6ZZtAzIT07dsXCxYskNc1Go0MMJ588knMnDmzyfur1Wo5QyLuP2mSLlegKUVFRQgKCkJhYSECAwNNGS7ZKBD5fHsqXv5V13DM1A3vts1oumpmxcF0/P27P1Ch1qJvbEt8NKkP27wTEdkZY9+/TcoZKS8vx759+zBr1qzq25RKpVx6EbMexigrK0NFRQWCgxveYVWlUslLzV+GnCdHxBBjG5qJ2Pl/W8/g9d+Oy+u3XBeJt+/tAW+PxmdRiIjISZZpcnNz5cxGRERErdvF9czMTKN+xowZMxAVFSUDmIbMmzdPRlL6i5h5IefKEalLzIiITewaa2hWqqrEM0sPVgcioqvqgvG9GIgQETk4q1bTvP7661iyZInMIxHJrw0RMy8iL6XmzAgDEufKERFzH8F+nnhhTGdEBvk0ueHdqaxiTP1mP1KyS+R54n5TBrHFOxGRywUjoaGhcHNzQ1ZWVq3bxfXIyMhG7/vWW2/JYGT9+vXo3r17o+d6eXnJCzlOW/ftKTlGz4joQ45X7+zWZGt34acDF/Dcj4dxuUKNiEAvLJjQC31jG17mIyIiJw5GPD090bt3b2zYsAF33HFHdQKruD5t2rQG7/fmm2/i1VdfxZo1a9CnjzE7kZAz54cYu8dM0ZUKvPLLUSzbe0FeH5wQivn39WAPESIiV1+mEcsnkydPlkFFUlIS5s+fj9LSUkyZMkV+X1TItG7dWuZ9CG+88QZmz56NxYsXy94k+twSf39/eSHnaOturBfHdMaDg+KarJZZfzQLzy8/hKwiFcQed0/d2B5P3dS+yfsREZELBCPjxo1DTk6ODDBEYNGjRw+sXr26Oqk1LS1NVtjoLVq0SFbh3HPPPbV+juhT8q9//cscvwNZeVlm15k8zPyhdlt3Y8t2mwpE8kpUcrZl5R8Z8npsiC9ev7s7+rcLMcPoiYjIKfqM2AL7jDhH2W5j1TKVag2W7j2P/6w9ifzScrkn3qND2uHZER1YLUNE5KAs0meEXFdzl2WayhERsfDG49mY99txWSkjdIoMwBt3d0didAszjJyIiOwdgxFqmEYNnNsBTXEmVv6cAQXaQWtCa5ppwxIwKCG0wbLdg+cL8Ppvx7DrjG5zvJa+HjIvZGK/GHi6c0NpIiJXwWCEDDu6Elg9AyjKkOHH+wAyvIIxt2KS3NTOmPwQscRSNwgRMyG/n8rFB1tOY8fpPHmbCDweGhSHqUPjEeTjYdFfi4iI7A+DEarXM8TtxM/om/yMCB2q8z2ESORjkcd8TK14psGApKG27hVqDVYduoj/bTmDoxd17f3dlQrc3qM1nh3RHm1a+lr0dyMiIvvFYIRqJadmFZZhm9dcaMWCTJ2VFXFdowXmeHyFdao+cpfdpvJDRB7Isr3n8eP+C8gtKZe3+Xq64b6+bfHwkDi0buFjnV+QiIjsFoMRqpWc2l95HFEKXQ6HISIgiUIekpTHsUvTpfr2Fr4eWDi+F/rHh+BSWTm+TU7D9/suYN+5S9XniGZlkwbEyAt32CUiIrh6MKJfksguvoLwAO8m90ZxleTUcBQYdXf9efpn7J+jOuJEVjHe3XgKe8/myxkUQTynwzqGY1zfaAztGAYPNyamEhFRbS4ZjBjql9HKyBblzp6cmg3jymkztLq9YXw83WTS6XM/Ha71/etaB+GW61rh7l6tER7Y8KaIRERELtf0rKF+GeITvhIafDtSjaSwSsA/AogZCCjdXCI5VT+T8beKpzDb42uZrKqfKNJoFTirjcBBbQIOauKxH51xRNMWNf9yRMt2sXndqK6RGNU1ggmpREQENj0zcZv7kcpkzPH4ElFba+RLBEYBo98AutwGV0lOfdr9Jzxb/jeMcduFU5o2OKltg2PaGBTBr97PjQv1w8D4ENlLZEC7ELT0Yx4IERGZzqWCETEzYKiV+ShlsixZrafoIrTLJuHkDQtxvOVQx8ktqcoHQUkWknPc8cRaN6ihRD/FcfigHMe1bZGlCcZFbTDStaFy1iNNG4Fz2ojqoGN35dXkVMEL5ejmkYHEhLbo0aM3+sS0RBQrYYiIyAxcKhgRyap1iaUZMSMij+vFGGIOAQjY/CKeVb0rS1ltnVsiVtVUlRqUlatRqqpE0eUKJJ/NR2bhFZkcGlp6AsVH1qLwigaXtP7IQxDaK4JlwLFH2wk9VR82+Rg+UOEyvKCAFu0UGZjW0wO39oqHR7vbnGbZioiI7IdLBSNiZqOuOMVF7NB0hVYr3nqvRiPiWFv9VYEYRSZStVFyZuXxr/ejf7tgdIwIQHSwLjdC5E+otVpoxEUjvuqWhcR18VV+T6NFpbiodV/VGo08LldrZFOwCrVWfhXBRnml/qtafr1SocblcjUuV6ir8zsME8HCzY0+DyEoRKQiH60U+fJrjCILbRXZ1V8fqviHLNuNDPLBP8aOdY2kXiIishmXSmAVQcHgNzbKWQS7/6WNIGZ1NDKAEhcteitOIkxRgBaKUgSiDC0UxQhFEUIURQhGETQKhbwercg2MAukC7zKfSOxesRahAf6OcaSFBER2S0msBog3ljFEouoptG9fQNBKEFPZYr8vm4O5Or8yNW5EWCvpgMuwbhAyMdDiYRwfwT6eMDP013mVojHziq6Imc5ArzdEd3SV14XMx0tfDzk+WKZJS2/DGXllQjx90LXVoFIzStDiaoCkYE+SFLvh9/vr8KnJA2+uAI3hVaW2IqS3EL4Y4nnK02O7T8V9+BZ9+/l7ErtOEMhf0+vW9/E7V3aNuPZJSIiah6Xmhkx1GdEzC5s83pKLlcYascl3rQzEYLBVTkjzSG6kwoFZRX1qleMPUck2X7gqUuyNVSS+2nlzXjE47cmx/JU+TSo4K6rHKrZaTWwNTD6dYevHCIiIvvBmZFGiByIEV0iqzuwFl96Ba22PFH1XW29N/q5FQ80OxCpG2DU/dnGnKNPshVhY0MluXe4bzdqLKKpmcgHWa/qg77K45g1uAUSO3dyqp4qRETkWFwyGBHEssmA+JCqaxOBiIDqrqR6YkZEBCIN7VBrLUlG7BcjckFytQEIRrHBfBCN+H20IUjWdJLXw4N88eDY+5HI5FQiIrIxlw1G6hHLE53G1OrPMX6t2zXNiJiLsfvFLFcPwkNuqxvMByke+jLeadnLcfqlEBGRS2AwUpNYpogbIg/FXMjC8Pp72NiCsfvFrNf0wV5NJ8wW+SCo3UlWMfp1dOxyGzpabphERETNwmDEyNyS9Ucz8cn2szYZh1haEVUzNfeLMZRkK86LCPLFn7c+iij/VDnD42x77BARkfNxyWoac+72ay01W9bXDEj0hch7kuZD3XEsl1+IiMjh3r8ZjDRz91tRhSNyLy6VqvDyr8esEqCMVibrlmBYkktERA6Apb1WqcLRGdWtVXWAcja3DN8mpyGz6Gpw0tLXQxYMX0ufEeGPgOu5BENERE6HwYgFApRpNybUmj0RSydCzdt6x7TEvnOXTD5HtwTTxma/KxERkblxmYaIiIhs+v5t+yYaRERE5NIYjBAREZFNMRghIiIim2IwQkRERDbFYISIiIhsisEIEREROV4wsnDhQsTGxsLb2xv9+vVDcnJyo+d/99136NSpkzz/uuuuw6pVq5o7XiIiInL1YGTp0qWYPn065syZg/379yMxMRGjRo1Cdna2wfN37NiB8ePH4+GHH8aBAwdwxx13yMvhw4fNMX4iIiJytaZnYiakb9++WLBggbyu0WgQHR2NJ598EjNnzqx3/rhx41BaWopffvml+rb+/fujR48e+OCDD4x6TDY9IyIicjwW2ZumvLwc+/btw6xZs6pvUyqVGD58OHbu3GnwPuJ2MZNSk5hJWb58eYOPo1Kp5EVP/BL6X4qIiIgcg/59u6l5D5OCkdzcXKjVakRERNS6XVw/fvy4wftkZmYaPF/c3pB58+Zh7ty59W4XMzBERETkWIqLi+UMiUNtlCdmXmrOpoiloPz8fISEhEChEBvFmS9iEwHO+fPnufxjBD5fxuNzZTw+V8bjc2U8Plf28VyJGRERiERFRTV6nknBSGhoKNzc3JCVlVXrdnE9MjLS4H3E7aacL3h5eclLTS1atICliCeff6zG4/NlPD5XxuNzZTw+V8bjc2X756qxGZFmVdN4enqid+/e2LBhQ61ZC3F9wIABBu8jbq95vrBu3boGzyciIiLXYvIyjVg+mTx5Mvr06YOkpCTMnz9fVstMmTJFfn/SpElo3bq1zPsQnn76adxwww34z3/+gzFjxmDJkiXYu3cvPvzwQ/P/NkREROT8wYgo1c3JycHs2bNlEqoo0V29enV1kmpaWpqssNEbOHAgFi9ejBdeeAHPPfcc2rdvLytpunXrBlsTS0GiX0rdJSEyjM+X8fhcGY/PlfH4XBmPz5VjPVcm9xkhIiIiMifuTUNEREQ2xWCEiIiIbIrBCBEREdkUgxEiIiKyKQYjNdx2221o27YtvL290apVKzzwwAPIyMiw9bDsztmzZ+UuzHFxcfDx8UF8fLzMxBZ7F1F9r776qqwq8/X1tWjzPke0cOFCxMbGyv9zYhPO5ORkWw/JLm3duhVjx46VXSxFF+rG9vZydaKthNjMNSAgAOHh4XKX+BMnTth6WHZp0aJF6N69e3WzM9H/67fffrPJWBiM1DBs2DAsW7ZM/uH+8MMPOH36NO655x5bD8vuiH2IRLO7//3vfzhy5AjeeecduQOzKN2m+kSQ9pe//AVTp0619VDsytKlS2XfIhHI7t+/H4mJiXITzezsbFsPze6IXk7i+RHBGzVuy5YteOKJJ7Br1y7ZYLOiogIjR46UzyHV1qZNG7z++utyA1zR/+vGG2/E7bffLl/XrU6U9pJhK1as0CoUCm15ebmth2L33nzzTW1cXJyth2HXPvvsM21QUJCth2E3kpKStE888UT1dbVarY2KitLOmzfPpuOyd+Jl+6effrL1MBxGdna2fM62bNli66E4hJYtW2o//vhjqz8uZ0YaIDbm++abb+T0uoeHh62HY/cKCwsRHBxs62GQA80WiU9jw4cPr75NNEsU13fu3GnTsZHzvTYJfH1qnFqtlh3SxQySLbZrYTBSx4wZM+Dn5yd3CBbdZFesWGHrIdm9lJQUvPfee/jrX/9q66GQg8jNzZUvfvrOzXriuujsTGQOYjn5mWeewaBBg+yi67c9OnToEPz9/WX31ccffxw//fQTunTpYvVxOH0wMnPmTJnw1dhF5EDo/eMf/8CBAwewdu1auUOx2GvHVZrUmvpcCenp6Rg9erTMiXj00UfhKprzXBGRdYnckcOHD8tP/GRYx44dcfDgQezevVvmtYm9544ePQprc/p28GIfnby8vEbPadeundyRuK4LFy4gOjoaO3bscIldhk19rkSl0dChQ9G/f398/vnntfYkcnbN+bsSz5H4lFZQUABXJ5ZpRHXR999/L6sd9MQLoXh+OCPZMBHoik+vNZ83qm/atGny70hUIonKPzKOWCoVFZKiQMGuN8pzNGFhYfLS3Ck+QaVSwRWY8lyJGRFRfdS7d2989tlnLhWIXOvfFUEGaeJvZ8OGDdVvquL/m7gu3kSImkt8vn7yySdlwLZ582YGIiYS/w9t8Z7n9MGIscQU1Z49ezB48GC0bNlSlvW++OKLMkJ0hVkRU4hARMyIxMTE4K233pKzBHqRkZE2HZs9ErlHIiFafBV5EmJKVEhISJBrta5KlPWKmZA+ffogKSkJ8+fPl8lzU6ZMsfXQ7E5JSYnMzdJLTU2Vf0ciKVP0RqLaSzNip3gxKyJ6jehzkIKCgmRfJLpq1qxZuPnmm+XfUHFxsXzeRAC3Zs0aWJ3V63fs1J9//qkdNmyYNjg4WOvl5aWNjY3VPv7449oLFy7Yemh2WaIq/nQMXai+yZMnG3yuNm3apHV17733nrZt27ZaT09PWeq7a9cuWw/JLom/FUN/Q+Jvi2pr6LVJvG5RbQ899JA2JiZG/v8LCwvT3nTTTdq1a9dqbcHpc0aIiIjIvrnWQj8RERHZHQYjREREZFMMRoiIiMimGIwQERGRTTEYISIiIptiMEJEREQ2xWCEiIiIbIrBCBEREdkUgxEiIiKyKQYjREREZFMMRoiIiMimGIwQERERbOn/AVyTDpTVNbQPAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plt.plot(x, y)\n", + "plt.scatter(x_train, X_train_s @ beta + y_offset)\n", + "plt.scatter(x_test, X_test_s @ beta + y_offset)" + ] + }, + { + "cell_type": "markdown", + "id": "2993cfae", + "metadata": {}, + "source": [ + "## Exercise 4 - Testing multiple hyperparameters" + ] + }, + { + "cell_type": "markdown", + "id": "e6cdc89a", + "metadata": {}, + "source": [ + "**a)** Compute the MSE of your ridge model for polynomials of degrees 1 to 5 with lambda set to 0.01. Plot the MSE as a function of polynomial degree.\n", + "\n", + "**b)** Compute the MSE of your ridge model for a polynomial with degree 3, and with lambdas from $10^{-1}$ to $10^{-5}$ on a logarithmic scale. Plot the MSE as a function of lambda.\n", + "\n", + "**c)** Compute the MSE of your ridge model for polynomials of degrees 1 to 5, and with lambdas from $10^{-1}$ to $10^{-5}$ on a logarithmic scale. Plot the MSE as a function of polynomial degree and lambda using a [heatmap](https://matplotlib.org/stable/gallery/images_contours_and_fields/image_annotated_heatmap.html)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.15" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}