lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2024-08-18 21:52:31 +02:00
parent 0150f76242
commit 1b5934da6e
25 changed files with 35 additions and 62651 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/exercisesweek34.ipynb
-302
View File
@@ -1,302 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "241c4a61",
"metadata": {
"editable": true
},
"source": [
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
"doconce format html exercisesweek34.do.txt -->\n",
"<!-- dom:TITLE: Exercises week 34 -->"
]
},
{
"cell_type": "markdown",
"id": "a33815b9",
"metadata": {
"editable": true
},
"source": [
"# Exercises week 34\n",
"**FYS-STK3155/4155**\n",
"\n",
"Date: **August 21-25, 2023**"
]
},
{
"cell_type": "markdown",
"id": "2990585c",
"metadata": {
"editable": true
},
"source": [
"## Exercises\n",
"\n",
"Here are three possible exercises for week 34"
]
},
{
"cell_type": "markdown",
"id": "be056de0",
"metadata": {
"editable": true
},
"source": [
"## Exercise 1: Setting up various Python environments\n",
"\n",
"The first exercise here is of a mere technical art. We want you to have \n",
"* git as a version control software and to establish a user account on a provider like GitHub. Other providers like GitLab etc are equally fine. You can also use the University of Oslo [GitHub facilities](https://www.uio.no/tjenester/it/maskin/filer/versjonskontroll/github.html). \n",
"\n",
"* Install various Python packages\n",
"\n",
"We will make extensive use of Python as programming language and its\n",
"myriad of available libraries. You will find\n",
"IPython/Jupyter notebooks invaluable in your work. You can run **R**\n",
"codes in the Jupyter/IPython notebooks, with the immediate benefit of\n",
"visualizing your data. You can also use compiled languages like C++,\n",
"Rust, Fortran etc if you prefer. The focus in these lectures will be\n",
"on Python.\n",
"\n",
"If you have Python installed (we recommend Python3) and you feel\n",
"pretty familiar with installing different packages, we recommend that\n",
"you install the following Python packages via **pip** as \n",
"\n",
"1. pip install numpy scipy matplotlib ipython scikit-learn sympy pandas pillow \n",
"\n",
"For **Tensorflow**, we recommend following the instructions in the text of \n",
"[Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow, O'Reilly](http://shop.oreilly.com/product/0636920052289.do)\n",
"\n",
"We will come back to **tensorflow** later. \n",
"\n",
"For Python3, replace **pip** with **pip3**.\n",
"\n",
"For OSX users we recommend, after having installed Xcode, to\n",
"install **brew**. Brew allows for a seamless installation of additional\n",
"software via for example \n",
"\n",
"1. brew install python3\n",
"\n",
"For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution,\n",
"you can use **pip** as well and simply install Python as \n",
"\n",
"1. sudo apt-get install python3 (or python for Python2.7)\n",
"\n",
"If you don't want to perform these operations separately and venture\n",
"into the hassle of exploring how to set up dependencies and paths, we\n",
"recommend two widely used distrubutions which set up all relevant\n",
"dependencies for Python, namely \n",
"\n",
"* [Anaconda](https://docs.anaconda.com/), \n",
"\n",
"which is an open source\n",
"distribution of the Python and R programming languages for large-scale\n",
"data processing, predictive analytics, and scientific computing, that\n",
"aims to simplify package management and deployment. Package versions\n",
"are managed by the package management system **conda**. \n",
"\n",
"* [Enthought canopy](https://www.enthought.com/product/canopy/) \n",
"\n",
"is a Python\n",
"distribution for scientific and analytic computing distribution and\n",
"analysis environment, available for free and under a commercial\n",
"license.\n",
"\n",
"We recommend using **Anaconda** if you are not too familiar with setting paths in a terminal environment."
]
},
{
"cell_type": "markdown",
"id": "f0f4ffae",
"metadata": {
"editable": true
},
"source": [
"## Exercise 2: making your own data and exploring scikit-learn\n",
"\n",
"We will generate our own dataset for a function $y(x)$ where $x \\in [0,1]$ and defined by random numbers computed with the uniform distribution. The function $y$ is a quadratic polynomial in $x$ with added stochastic noise according to the normal distribution $\\cal {N}(0,1)$.\n",
"The following simple Python instructions define our $x$ and $y$ values (with 100 data points)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "93d6a2b2",
"metadata": {
"collapsed": false,
"editable": true
},
"outputs": [],
"source": [
"x = np.random.rand(100,1)\n",
"y = 2.0+5*x*x+0.1*np.random.randn(100,1)"
]
},
{
"cell_type": "markdown",
"id": "45392145",
"metadata": {
"editable": true
},
"source": [
"1. Write your own code (following the examples under the [regression notes](https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/chapter1.html)) for computing the parametrization of the data set fitting a second-order polynomial. \n",
"\n",
"2. Use thereafter **scikit-learn** (see again the examples in the regression slides) and compare with your own code. \n",
"\n",
"3. Using scikit-learn, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as"
]
},
{
"cell_type": "markdown",
"id": "3eed315b",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"MSE(\\boldsymbol{y},\\boldsymbol{\\tilde{y}}) = \\frac{1}{n}\n",
"\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2,\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "26038071",
"metadata": {
"editable": true
},
"source": [
"and the $R^2$ score function.\n",
"If $\\tilde{\\boldsymbol{y}}_i$ is the predicted value of the $i-th$ sample and $y_i$ is the corresponding true value, then the score $R^2$ is defined as"
]
},
{
"cell_type": "markdown",
"id": "4c750d55",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"R^2(\\boldsymbol{y}, \\tilde{\\boldsymbol{y}}) = 1 - \\frac{\\sum_{i=0}^{n - 1} (y_i - \\tilde{y}_i)^2}{\\sum_{i=0}^{n - 1} (y_i - \\bar{y})^2},\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "30b4731e",
"metadata": {
"editable": true
},
"source": [
"where we have defined the mean value of $\\boldsymbol{y}$ as"
]
},
{
"cell_type": "markdown",
"id": "49afe51d",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\bar{y} = \\frac{1}{n} \\sum_{i=0}^{n - 1} y_i.\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "81f16b80",
"metadata": {
"editable": true
},
"source": [
"You can use the functionality included in scikit-learn. If you feel for it, you can use your own program and define functions which compute the above two functions. \n",
"Discuss the meaning of these results. Try also to vary the coefficient in front of the added stochastic noise term and discuss the quality of the fits."
]
},
{
"cell_type": "markdown",
"id": "cb71cb1c",
"metadata": {
"editable": true
},
"source": [
"## Exercise 3: Split data in test and training data\n",
"\n",
"In this exercise we want you to to compute the MSE for the training\n",
"data and the test data as function of the complexity of a polynomial,\n",
"that is the degree of a given polynomial.\n",
"\n",
"The aim is to reproduce Figure 2.11 of [Hastie et al](https://github.com/CompPhysics/MLErasmus/blob/master/doc/Textbooks/elementsstat.pdf).\n",
"\n",
"Our data is defined by $x\\in [-3,3]$ with a total of for example $n=100$ data points. You should try to vary the number of data points $n$ in your analysis."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "b8c919e5",
"metadata": {
"collapsed": false,
"editable": true
},
"outputs": [],
"source": [
"np.random.seed()\n",
"n = 100\n",
"# Make data set.\n",
"x = np.linspace(-3, 3, n).reshape(-1, 1)\n",
"y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)"
]
},
{
"cell_type": "markdown",
"id": "50629e14",
"metadata": {
"editable": true
},
"source": [
"where $y$ is the function we want to fit with a given polynomial."
]
},
{
"cell_type": "markdown",
"id": "0578f08c",
"metadata": {
"editable": true
},
"source": [
"**a)**\n",
"Write a first code which sets up a design matrix $X$ defined by a fifth-order polynomial and split your data set in training and test data."
]
},
{
"cell_type": "markdown",
"id": "125ec9a1",
"metadata": {
"editable": true
},
"source": [
"**b)**\n",
"Write thereafter (using either **scikit-learn** or your matrix inversion code using for example **numpy**)\n",
"and perform an ordinary least squares fitting and compute the mean squared error for the training data and the test data. These calculations should apply to a model given by a fifth-order polynomial."
]
},
{
"cell_type": "markdown",
"id": "e5bb2036",
"metadata": {
"editable": true
},
"source": [
"**c)**\n",
"Add now a model which allows you to make polynomials up to degree $15$. Perform a standard OLS fitting of the training data and compute the MSE for the training and test data and plot both test and training data MSE as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?"
]
}
],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 5
}
doc/LectureNotes/exercisesweek36.ipynb
-394
View File
@@ -1,394 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "fb4d27e1",
"metadata": {
"editable": true
},
"source": [
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
"doconce format html exercisesweek36.do.txt --no_mako -->\n",
"<!-- dom:TITLE: Exercises week 36 -->"
]
},
{
"cell_type": "markdown",
"id": "c3ef468d",
"metadata": {
"editable": true
},
"source": [
"# Exercises week 36\n",
"**September 4-8, 2023**\n",
"\n",
"Date: **Deadline is Sunday September 10 at midnight**"
]
},
{
"cell_type": "markdown",
"id": "b81273ff",
"metadata": {
"editable": true
},
"source": [
"## Overarching aims of the exercises this week\n",
"\n",
"This set of exercises form an important part of the first project. The\n",
"analytical exercises deal with the material covered last week on the\n",
"mathematical interpretations of ordinary least squares and of Ridge\n",
"regression. The numerical exercises can be seen as a continuation of\n",
"exercise 3 from week 35, with the inclusion of Ridge regression. This\n",
"material enters also the discussions of the first project."
]
},
{
"cell_type": "markdown",
"id": "53c4e245",
"metadata": {
"editable": true
},
"source": [
"## Exercise 1: Analytical exercises\n",
"\n",
"The aim here is to derive the expression for the optimal parameters\n",
"using Ridge regression. Furthermore, using the singular value\n",
"decomposition, we will analyze the difference between the ordinary\n",
"least squares approach and Ridge regression.\n",
"\n",
"The expression for the standard Mean Squared Error (MSE) which we used to define our cost function and the equations for the ordinary least squares (OLS) method, was given by the\n",
"optimization problem"
]
},
{
"cell_type": "markdown",
"id": "e5813e09",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in {\\mathbb{R}}^{p}}}\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\\right\\}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "f8af4862",
"metadata": {
"editable": true
},
"source": [
"which we can also write as"
]
},
{
"cell_type": "markdown",
"id": "8a114760",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n",
"{\\mathbb{R}}^{p}}}\\frac{1}{n}\\sum_{i=0}^{n-1}\\left(y_i-\\tilde{y}_i\\right)^2=\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2,\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "c4610a33",
"metadata": {
"editable": true
},
"source": [
"where we have used the definition of a norm-2 vector, that is"
]
},
{
"cell_type": "markdown",
"id": "a3608d5c",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\vert\\vert \\boldsymbol{x}\\vert\\vert_2 = \\sqrt{\\sum_i x_i^2}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "5dcf5daa",
"metadata": {
"editable": true
},
"source": [
"By minimizing the above equation with respect to the parameters\n",
"$\\boldsymbol{\\beta}$ we could then obtain an analytical expression for the\n",
"parameters $\\boldsymbol{\\beta}$.\n",
"\n",
"We can add a regularization parameter $\\lambda$ by\n",
"defining a new cost function to be optimized, that is"
]
},
{
"cell_type": "markdown",
"id": "0cbd93fd",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n",
"{\\mathbb{R}}^{p}}}\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2+\\lambda\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "6fba5d18",
"metadata": {
"editable": true
},
"source": [
"which leads to the Ridge regression minimization problem. One can require as part of the optimization problem \n",
"that $\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\\le t$, where $t$ is\n",
"a finite number larger than zero. We will not implement that here."
]
},
{
"cell_type": "markdown",
"id": "bec3e21d",
"metadata": {
"editable": true
},
"source": [
"### a) Expression for Ridge regression\n",
"\n",
"Show that the optimal parameters"
]
},
{
"cell_type": "markdown",
"id": "380e526b",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\hat{\\boldsymbol{\\beta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "97a04d2d",
"metadata": {
"editable": true
},
"source": [
"with $\\boldsymbol{I}$ being a $p\\times p$ identity matrix with the constraint that"
]
},
{
"cell_type": "markdown",
"id": "a378590f",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\sum_{i=0}^{p-1} \\beta_i^2 \\leq t,\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "f2d2d7ae",
"metadata": {
"editable": true
},
"source": [
"with $t$ a finite positive number. In the optimization, we will not require that the latter is satisfied.\n",
"\n",
"The ordinary least squares result is"
]
},
{
"cell_type": "markdown",
"id": "b2ffd80f",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\hat{\\boldsymbol{\\beta}}_{\\mathrm{OLS}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "3c3a9191",
"metadata": {
"editable": true
},
"source": [
"### b) The singular value decomposition\n",
"\n",
"Here we will use the singular value decomposition of an $n\\times p$ matrix $\\boldsymbol{X}$ (our design matrix)"
]
},
{
"cell_type": "markdown",
"id": "4e3f0bb8",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\boldsymbol{X}=\\boldsymbol{U}\\boldsymbol{\\Sigma}\\boldsymbol{V}^T,\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "040a6f88",
"metadata": {
"editable": true
},
"source": [
"to study properties of Ridge regression and ordinary least squares regression.\n",
"Here $\\boldsymbol{U}$ and $\\boldsymbol{V}$ are orthogonal matrices of dimensions\n",
"$n\\times n$ and $p\\times p$, respectively, and $\\boldsymbol{\\Sigma}$ is an\n",
"$n\\times p$ matrix which contains the singular values only. This material was discussed during the lectures of week 35.\n",
"\n",
"Show that you can write the \n",
"OLS solutions in terms of the eigenvectors (the columns) of the orthogonal matrix $\\boldsymbol{U}$ as"
]
},
{
"cell_type": "markdown",
"id": "0107597b",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\tilde{\\boldsymbol{y}}_{\\mathrm{OLS}}=\\boldsymbol{X}\\boldsymbol{\\beta} = \\sum_{j=0}^{p-1}\\boldsymbol{u}_j\\boldsymbol{u}_j^T\\boldsymbol{y}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "55fd1a9a",
"metadata": {
"editable": true
},
"source": [
"For Ridge regression, show that the corresponding equation is"
]
},
{
"cell_type": "markdown",
"id": "8a05634c",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\tilde{\\boldsymbol{y}}_{\\mathrm{Ridge}}=\\boldsymbol{X}\\boldsymbol{\\beta}_{\\mathrm{Ridge}} = \\boldsymbol{U\\Sigma V^T}\\left(\\boldsymbol{V}\\boldsymbol{\\Sigma}^2\\boldsymbol{V}^T+\\lambda\\boldsymbol{I} \\right)^{-1}(\\boldsymbol{U\\Sigma V^T})^T\\boldsymbol{y}=\\sum_{j=0}^{p-1}\\boldsymbol{u}_j\\boldsymbol{u}_j^T\\frac{\\sigma_j^2}{\\sigma_j^2+\\lambda}\\boldsymbol{y},\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "5d46ab65",
"metadata": {
"editable": true
},
"source": [
"with the vectors $\\boldsymbol{u}_j$ being the columns of $\\boldsymbol{U}$ from the SVD of the matrix $\\boldsymbol{X}$. \n",
"\n",
"Give an interpretation of the results. [Section 3.4 of Hastie et al's textbook gives a good discussion of the above results](https://link.springer.com/book/10.1007/978-0-387-84858-7)."
]
},
{
"cell_type": "markdown",
"id": "478a39d8",
"metadata": {
"editable": true
},
"source": [
"## Exercise 2: Adding Ridge Regression\n",
"\n",
"This exercise is a continuation of exercise 3 from week 35, see <https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/exercisesweek35.html>. We will use the same function to\n",
"generate our data set, still staying with a simple function $y(x)$\n",
"which we want to fit using linear regression, but now extending the\n",
"analysis to include the Ridge regression method.\n",
"\n",
"In this exercise you need to include the same elements from last week, that is\n",
"1. scale your data by subtracting the mean value from each column in the design matrix.\n",
"\n",
"2. perform a split of the data in a training set and a test set.\n",
"\n",
"The addition to the analysis this time is the introduction of the hyperparameter $\\lambda$ when introducing Ridge regression.\n",
"\n",
"Extend the code from exercise 3 from [week 35](https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/exercisesweek35.html) to include Ridge regression with the hyperparameter $\\lambda$. The optimal parameters $\\hat{\\beta}$ for Ridge regression can be obtained by matrix inversion in a similar way as done for ordinary least squares. You need to add to your code the following equations"
]
},
{
"cell_type": "markdown",
"id": "ed28dbd2",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\hat{\\boldsymbol{\\beta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "1939c401",
"metadata": {
"editable": true
},
"source": [
"The ordinary least squares result you encoded last week is given by"
]
},
{
"cell_type": "markdown",
"id": "10877c2d",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\hat{\\boldsymbol{\\beta}}_{\\mathrm{OLS}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "a7adfe54",
"metadata": {
"editable": true
},
"source": [
"Use these results to compute the mean squared error for ordinary least\n",
"squares and Ridge regression first for a polynomial of degree five\n",
"with $n=100$ data points and five selected values of\n",
"$\\lambda=[0.0001,0.001, 0.01,0.1,1.0]$. Compute thereafter the mean\n",
"squared error for the same values of $\\lambda$ for polynomials of degree ten\n",
"and $15$. Discuss your results for the training MSE and test MSE with\n",
"Ridge regression and ordinary least squares."
]
}
],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 5
}
doc/LectureNotes/exercisesweek37.ipynb
-268
View File
@@ -1,268 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "8b21a389",
"metadata": {
"editable": true
},
"source": [
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
"doconce format html exercisesweek37.do.txt -->\n",
"<!-- dom:TITLE: Exercises week 37 -->"
]
},
{
"cell_type": "markdown",
"id": "10a3b32a",
"metadata": {
"editable": true
},
"source": [
"# Exercises week 37\n",
"**September 11-15, 2023**\n",
"\n",
"Date: **Deadline is Sunday September 17 at midnight**"
]
},
{
"cell_type": "markdown",
"id": "940daa26",
"metadata": {
"editable": true
},
"source": [
"## Overarching aims of the exercises this week\n",
"\n",
"This exercise deals with various mean values and variances in linear\n",
"regression method (here it may be useful to look up chapter 3,\n",
"equation (3.8) of [Trevor Hastie, Robert Tibshirani, Jerome\n",
"H. Friedman, The Elements of Statistical Learning,\n",
"Springer](https://www.springer.com/gp/book/9780387848570)). The\n",
"exercise is also a part of project 1 and can be reused in the theory\n",
"part of the project.\n",
"\n",
"For more discussions on Ridge regression and calculation of\n",
"expectation values, [Wessel van\n",
"Wieringen's](https://arxiv.org/abs/1509.09169) article is highly\n",
"recommended.\n",
"\n",
"The assumption we have made is that there exists a continuous function\n",
"$f(\\boldsymbol{x})$ and a normal distributed error $\\boldsymbol{\\varepsilon}\\sim N(0,\n",
"\\sigma^2)$ which describes our data"
]
},
{
"cell_type": "markdown",
"id": "d51b0da4",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\boldsymbol{y} = f(\\boldsymbol{x})+\\boldsymbol{\\varepsilon}\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "40a14559",
"metadata": {
"editable": true
},
"source": [
"We then approximate this function $f(\\boldsymbol{x})$ with our model $\\boldsymbol{\\tilde{y}}$ from the solution of the linear regression equations (ordinary least squares OLS), that is our\n",
"function $f$ is approximated by $\\boldsymbol{\\tilde{y}}$ where we minimized $(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2$, with"
]
},
{
"cell_type": "markdown",
"id": "e4afb2ea",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\boldsymbol{\\tilde{y}} = \\boldsymbol{X}\\boldsymbol{\\beta}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "b0153681",
"metadata": {
"editable": true
},
"source": [
"The matrix $\\boldsymbol{X}$ is the so-called design or feature matrix."
]
},
{
"cell_type": "markdown",
"id": "17b98331",
"metadata": {
"editable": true
},
"source": [
"## Exercise 1: Expectation values for ordinary least squares expressions\n",
"\n",
"Show that the expectation value of $\\boldsymbol{y}$ for a given element $i$"
]
},
{
"cell_type": "markdown",
"id": "0e35faa1",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\mathbb{E}(y_i) =\\sum_{j}x_{ij} \\beta_j=\\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta},\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "c1c7837e",
"metadata": {
"editable": true
},
"source": [
"and that\n",
"its variance is"
]
},
{
"cell_type": "markdown",
"id": "0ab57c17",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\mbox{Var}(y_i) = \\sigma^2.\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "345dbeca",
"metadata": {
"editable": true
},
"source": [
"Hence, $y_i \\sim N( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta}, \\sigma^2)$, that is $\\boldsymbol{y}$ follows a normal distribution with \n",
"mean value $\\boldsymbol{X}\\boldsymbol{\\beta}$ and variance $\\sigma^2$.\n",
"\n",
"With the OLS expressions for the optimal parameters $\\boldsymbol{\\hat{\\beta}}$ show that"
]
},
{
"cell_type": "markdown",
"id": "fea2ddcf",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\mathbb{E}(\\boldsymbol{\\hat{\\beta}}) = \\boldsymbol{\\beta}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "683a72cf",
"metadata": {
"editable": true
},
"source": [
"Show finally that the variance of $\\boldsymbol{\\boldsymbol{\\beta}}$ is"
]
},
{
"cell_type": "markdown",
"id": "0dfd010a",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\mbox{Var}(\\boldsymbol{\\hat{\\beta}}) = \\sigma^2 \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "d4e4aa80",
"metadata": {
"editable": true
},
"source": [
"We can use the last expression when we define a [so-called confidence interval](https://en.wikipedia.org/wiki/Confidence_interval) for the parameters $\\beta$. \n",
"A given parameter $\\beta_j$ is given by the diagonal matrix element of the above matrix."
]
},
{
"cell_type": "markdown",
"id": "acb128a7",
"metadata": {
"editable": true
},
"source": [
"## Exercise 2: Expectation values for Ridge regression\n",
"\n",
"Show that"
]
},
{
"cell_type": "markdown",
"id": "83af53d9",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\mathbb{E} \\big[ \\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}} \\big]=(\\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I}_{pp})^{-1} (\\mathbf{X}^{\\top} \\mathbf{X})\\boldsymbol{\\beta}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "1bbd0f50",
"metadata": {
"editable": true
},
"source": [
"We see clearly that\n",
"$\\mathbb{E} \\big[ \\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}} \\big] \\not= \\mathbb{E} \\big[\\hat{\\boldsymbol{\\beta}}^{\\mathrm{OLS}}\\big ]$ for any $\\lambda > 0$.\n",
"\n",
"Show also that the variance is"
]
},
{
"cell_type": "markdown",
"id": "617deac1",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\mbox{Var}[\\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}}]=\\sigma^2[ \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1} \\mathbf{X}^{T}\\mathbf{X} \\{ [ \\mathbf{X}^{\\top} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}\\}^{T},\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "bee378b0",
"metadata": {
"editable": true
},
"source": [
"and it is easy to see that if the parameter $\\lambda$ goes to infinity then the variance of the Ridge parameters $\\boldsymbol{\\beta}$ goes to zero."
]
}
],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 5
}
doc/LectureNotes/exercisesweek38.ipynb
-183
View File
@@ -1,183 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "ce05d309",
"metadata": {
"editable": true
},
"source": [
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
"doconce format html exercisesweek38.do.txt -->\n",
"<!-- dom:TITLE: Exercises week 38 -->"
]
},
{
"cell_type": "markdown",
"id": "5b7c1442",
"metadata": {
"editable": true
},
"source": [
"# Exercises week 38\n",
"**September 18-22, 2023**\n",
"\n",
"Date: **Deadline is Sunday September 24 at midnight**"
]
},
{
"cell_type": "markdown",
"id": "02e09b06",
"metadata": {
"editable": true
},
"source": [
"## Overarching aims of the exercises this week\n",
"\n",
"The aim of the exercises this week is to derive the equations for the bias-variance tradeoff to be used in project 1 as well as testing this for a simpler function using the bootstrap method. The exercises here can be reused in project 1 as well.\n",
"\n",
"Consider a\n",
"dataset $\\mathcal{L}$ consisting of the data\n",
"$\\mathbf{X}_\\mathcal{L}=\\{(y_j, \\boldsymbol{x}_j), j=0\\ldots n-1\\}$.\n",
"\n",
"We assume that the true data is generated from a noisy model"
]
},
{
"cell_type": "markdown",
"id": "babb7346",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\boldsymbol{y}=f(\\boldsymbol{x}) + \\boldsymbol{\\epsilon}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "a5ee908c",
"metadata": {
"editable": true
},
"source": [
"Here $\\epsilon$ is normally distributed with mean zero and standard\n",
"deviation $\\sigma^2$.\n",
"\n",
"In our derivation of the ordinary least squares method we defined \n",
"an approximation to the function $f$ in terms of the parameters\n",
"$\\boldsymbol{\\beta}$ and the design matrix $\\boldsymbol{X}$ which embody our model,\n",
"that is $\\boldsymbol{\\tilde{y}}=\\boldsymbol{X}\\boldsymbol{\\beta}$.\n",
"\n",
"The parameters $\\boldsymbol{\\beta}$ are in turn found by optimizing the mean\n",
"squared error via the so-called cost function"
]
},
{
"cell_type": "markdown",
"id": "afa3df2c",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"C(\\boldsymbol{X},\\boldsymbol{\\beta}) =\\frac{1}{n}\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2=\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right].\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "9665df80",
"metadata": {
"editable": true
},
"source": [
"Here the expected value $\\mathbb{E}$ is the sample value. \n",
"\n",
"Show that you can rewrite this in terms of a term which contains the variance of the model itself (the so-called variance term), a\n",
"term which measures the deviation from the true data and the mean value of the model (the bias term) and finally the variance of the noise.\n",
"That is, show that"
]
},
{
"cell_type": "markdown",
"id": "25391aba",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathrm{Bias}[\\tilde{y}]+\\mathrm{var}[\\tilde{y}]+\\sigma^2,\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "b6e8652c",
"metadata": {
"editable": true
},
"source": [
"with"
]
},
{
"cell_type": "markdown",
"id": "77755270",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\mathrm{Bias}[\\tilde{y}]=\\mathbb{E}\\left[\\left(\\boldsymbol{y}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]\\right)^2\\right],\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "fa94db90",
"metadata": {
"editable": true
},
"source": [
"and"
]
},
{
"cell_type": "markdown",
"id": "4ecda624",
"metadata": {
"editable": true
},
"source": [
"$$\n",
"\\mathrm{var}[\\tilde{y}]=\\mathbb{E}\\left[\\left(\\tilde{\\boldsymbol{y}}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]\\right)^2\\right]=\\frac{1}{n}\\sum_i(\\tilde{y}_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2.\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "2dcb3e9a",
"metadata": {
"editable": true
},
"source": [
"Explain what the terms mean and discuss their interpretations.\n",
"\n",
"Perform then a bias-variance analysis of a simple one-dimensional (or other models of your choice) function by\n",
"studying the MSE value as function of the complexity of your model. Use ordinary least squares only.\n",
"\n",
"Discuss the bias and variance trade-off as function\n",
"of your model complexity (the degree of the polynomial) and the number\n",
"of data points, and possibly also your training and test data using the **bootstrap** resampling method.\n",
"You can follow the code example in the jupyter-book at <https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/chapter3.html#the-bias-variance-tradeoff>.\n",
"\n",
"See also the whiteboard notes from week 37 at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2023/NotesSep14.pdf>"
]
}
],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 5
}
doc/LectureNotes/exercisesweek39.ipynb
-59
View File
@@ -1,59 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "f35930ac",
"metadata": {
"editable": true
},
"source": [
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
"doconce format html exercisesweek39.do.txt -->\n",
"<!-- dom:TITLE: Exercises week 39 -->"
]
},
{
"cell_type": "markdown",
"id": "8cb567a0",
"metadata": {
"editable": true
},
"source": [
"# Exercises week 39\n",
"**September 25-29, 2023**\n",
"\n",
"Date: **Deadline is Sunday October 1 at midnight**"
]
},
{
"cell_type": "markdown",
"id": "934324b3",
"metadata": {
"editable": true
},
"source": [
"## Overarching aims of the exercises this week\n",
"\n",
"The aim of the exercises this week is to aid you in getting started\n",
"with writing the report. This will be discussed during the lab\n",
"sessions as well. One of the lab sessions will be recorded.\n",
"\n",
"A general guideline can be found at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>.\n",
"\n",
"Similarly, an example of an earlier project can be found at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/ReportSample.pdf>\n",
"\n",
"Your task this week is to\n",
"1. Write an abstract for your project\n",
"\n",
"2. Write an introduction\n",
"\n",
"3. Include references\n",
"\n",
"Ashort feedback to the this exercise will be available after the deadline. And you can reuse these elements in your final report."
]
}
],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 5
}
doc/LectureNotes/exercisesweek41.ipynb
-1144
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/exercisesweek42.ipynb
-73
View File
@@ -1,73 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "be117070",
"metadata": {
"editable": true
},
"source": [
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
"doconce format html exercisesweek42.do.txt -->\n",
"<!-- dom:TITLE: Exercises week 42 -->"
]
},
{
"cell_type": "markdown",
"id": "6f7356c3",
"metadata": {
"editable": true
},
"source": [
"# Exercises week 42\n",
"**October 9-13, 2023**\n",
"\n",
"Date: **Deadline is Sunday October 22 at midnight**\n",
"\n",
"You can hand in the exercises from week 41 and week 42 as one exercise and get a total score of two additional points."
]
},
{
"cell_type": "markdown",
"id": "aa378ef2",
"metadata": {
"editable": true
},
"source": [
"# Overarching aims of the exercises this week\n",
"\n",
"The aim of the exercises this week is to get started with implementing\n",
"gradient methods of relevance for project 2. The exercise this week is a simple\n",
"continuation from the previous week with the addition of automatic differentation.\n",
"Everything you develop here will be used in project 2. \n",
"\n",
"In order to get started, we will now replace in our standard ordinary\n",
"least squares (OLS) and Ridge regression codes (from project 1) the\n",
"matrix inversion algorithm with our own gradient descent (GD) and SGD\n",
"codes. You can use the Franke function or the terrain data from\n",
"project 1. **However, we recommend using a simpler function like**\n",
"$f(x)=a_0+a_1x+a_2x^2$ or higher-order one-dimensional polynomials.\n",
"You can obviously test your final codes against for example the Franke\n",
"function. Automatic differentiation will be discussed next week.\n",
"\n",
"You should include in your analysis of the GD and SGD codes the following elements\n",
"1. A plain gradient descent with a fixed learning rate (you will need to tune it) using automatic differentiation. Compare this with the analytical expression of the gradients you obtained last week. Feel free to use **Autograd** as Python package or **JAX**. You can use the examples form last week.\n",
"\n",
"2. Add momentum to the plain GD code and compare convergence with a fixed learning rate (you may need to tune the learning rate). Compare this with the analytical expression of the gradients you obtained last week.\n",
"\n",
"3. Repeat these steps for stochastic gradient descent with mini batches and a given number of epochs. Use a tunable learning rate as discussed in the lectures from week 39. Discuss the results as functions of the various parameters (size of batches, number of epochs etc)\n",
"\n",
"4. Implement the Adagrad method in order to tune the learning rate. Do this with and without momentum for plain gradient descent and SGD using automatic differentiation..\n",
"\n",
"5. Add RMSprop and Adam to your library of methods for tuning the learning rate. Again using automatic differentiation.\n",
"\n",
"The lecture notes from weeks 39 and 40 contain more information and code examples. Feel free to use these examples.\n",
"\n",
"We recommend reading chapter 8 on optimization from the textbook of [Goodfellow, Bengio and Courville](https://www.deeplearningbook.org/). This chapter contains many useful insights and discussions on the optimization part of machine learning."
]
}
],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 5
}
doc/LectureNotes/exercisesweek43.ipynb
-1940
View File
File diff suppressed because one or more lines are too long
doc/LectureNotes/exercisesweek47.ipynb
-214
View File
@@ -1,214 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "29616fbf",
"metadata": {},
"source": [
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
"doconce format html exercisesweek47.do.txt -->\n",
"<!-- dom:TITLE: Exercise week 47 -->"
]
},
{
"cell_type": "markdown",
"id": "e4f3fe3f",
"metadata": {},
"source": [
"# Exercise week 47\n",
"**November 20-24, 2023**\n",
"\n",
"Date: **Deadline is Sunday November 26 at midnight**"
]
},
{
"cell_type": "markdown",
"id": "c1f597b7",
"metadata": {},
"source": [
"# Overarching aims of the exercises this week\n",
"\n",
"The exercise this week is a simple course survey and feedback. This\n",
"is important for us in order to improve our teaching material, the\n",
"active learning format and anything else related to a succesful\n",
"mastering of central machine learning methods and their applications."
]
},
{
"cell_type": "markdown",
"id": "7497591b",
"metadata": {},
"source": [
"### Why did you choose this course?"
]
},
{
"cell_type": "markdown",
"id": "1c08143f",
"metadata": {},
"source": [
"### What was your programming knowledge before you started?\n",
"\n",
"And do you feel this course added to your programming competences and skills?"
]
},
{
"cell_type": "markdown",
"id": "9783d8a7",
"metadata": {},
"source": [
"### How do you judge your own level of knowledge on machine learning before and after this course?\n",
"\n",
"Here you can discuss your level of skill/knowledge at start of course\n",
"and at the end of the course and how these matched the level of\n",
"skill/knowledge needed to complete the projects."
]
},
{
"cell_type": "markdown",
"id": "8b0d5c47",
"metadata": {},
"source": [
"### Did the projects and the teaching material allow you to deepen your insights about Machine Learning?\n",
"\n",
"Feel free to comment here."
]
},
{
"cell_type": "markdown",
"id": "1ead8d47",
"metadata": {},
"source": [
"### Project based teaching and active learning\n",
"\n",
"This is a project based course and we as teachers would like to keep\n",
"it as it is since we see very clearly that people who attend this\n",
"course have a very good learning outcome. Project based courses are\n",
"however demanding (and expensive seen from the university admin) when\n",
"it comes to proper feedback and evaluations. Feel free to discuss\n",
"whether you found a project-based and active learning approach\n",
"useful. Feel also free to comment upon things we can improve upon or\n",
"alternative ways to assess whether the learning outcomes have been\n",
"achieved. Would you for example a standard 4 hours written exam\n",
"be something you would prefer? Or other alternatives to projects? We\n",
"would very much value your thoughts here since projects are an\n",
"essential part of this course."
]
},
{
"cell_type": "markdown",
"id": "746bd238",
"metadata": {},
"source": [
"### Usefulness of the weekly exercises\n",
"\n",
"Did the weekly exercises help in getting started with the projects?\n",
"How relevant where they for solving the projects? Feel free to\n",
"elaborate"
]
},
{
"cell_type": "markdown",
"id": "2ce96822",
"metadata": {},
"source": [
"### Active learning/lab sessions and lectures\n",
"\n",
"Was there a good link between lectures and active learning sessions?\n",
"Would you prefer an active learning environment only with no lectures\n",
"or would you prefer a more lecture based format with lab sessions only\n",
"(that is no discussion at the beginning of the lab sessions)? Feel\n",
"free to comment."
]
},
{
"cell_type": "markdown",
"id": "5dc0637a",
"metadata": {},
"source": [
"### How would you improve this course?\n",
"\n",
"Are there topics which are missing, topics which could have been\n",
"omitted and/or discussed in more depth? Feel free to add your comments\n",
"here such as how to improve to teaching material and more."
]
},
{
"cell_type": "markdown",
"id": "a98adf6c",
"metadata": {},
"source": [
"## Then some basic questions"
]
},
{
"cell_type": "markdown",
"id": "3a935279",
"metadata": {},
"source": [
"### Which is your preferred information chanel, Canvas, Discord, mail or other?"
]
},
{
"cell_type": "markdown",
"id": "41592552",
"metadata": {},
"source": [
"### Was the weekly update with plans etc useful?"
]
},
{
"cell_type": "markdown",
"id": "e79a2360",
"metadata": {},
"source": [
"### Was it easy to access the course material?"
]
},
{
"cell_type": "markdown",
"id": "8bc4dfc9",
"metadata": {},
"source": [
"### Which resources and tools did you use? Jupyter-notebooks, GitHub, the various textbooks we have recommended, etc etc"
]
},
{
"cell_type": "markdown",
"id": "bf415075",
"metadata": {},
"source": [
"### If you did not attend the lectures or the active learning/lab sessions, which resources did you use?"
]
},
{
"cell_type": "markdown",
"id": "db3db20a",
"metadata": {},
"source": [
"### Any other topics, impressions, ideas etc you would like to share with us?"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.10"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
doc/LectureNotes/week34.ipynb
-4962
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/week35.ipynb
-6044
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/week36.ipynb
-4242
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/week37.ipynb
-3538
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/week38.ipynb
-3014
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/week39.ipynb
-4889
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/week40.ipynb
-3545
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/week41.ipynb
-3537
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/week42.ipynb
-2643
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/week43.ipynb
-7528
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/week44.ipynb
-5951
View File
File diff suppressed because one or more lines are too long
doc/LectureNotes/week45.ipynb
-1997
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/week46.ipynb
-3131
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/week47.ipynb
-3018
View File
File diff suppressed because it is too large Load Diff
doc/src/week35/exercisesweek35.do.txt
+2 -2
View File
@@ -1,6 +1,6 @@
TITLE: Exercises week 35
AUTHOR: August 28-September 1, 2023
DATE: Deadline is Friday September 1 at midnight
AUTHOR: August 26-30, 2024
DATE: Deadline is Friday August 30 at midnight
===== Exercise: Analytical exercises =====
doc/LectureNotes/exercisesweek35.ipynb → doc/src/week35/exercisesweek35.ipynb
+33 -33
View File
@@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
"id": "053b96d5",
"id": "a15180da",
"metadata": {
"editable": true
},
@@ -14,20 +14,20 @@
},
{
"cell_type": "markdown",
"id": "a09cb811",
"id": "ac77b923",
"metadata": {
"editable": true
},
"source": [
"# Exercises week 35\n",
"**August 28-September 1, 2023**\n",
"**August 26-30, 2024**\n",
"\n",
"Date: **Deadline is Friday September 1 at midnight**"
"Date: **Deadline is Friday August 30 at midnight**"
]
},
{
"cell_type": "markdown",
"id": "8e83839e",
"id": "e8b90270",
"metadata": {
"editable": true
},
@@ -49,7 +49,7 @@
},
{
"cell_type": "markdown",
"id": "fe2b1c49",
"id": "2e2e8f1a",
"metadata": {
"editable": true
},
@@ -61,7 +61,7 @@
},
{
"cell_type": "markdown",
"id": "259f2ad1",
"id": "1cdc68da",
"metadata": {
"editable": true
},
@@ -71,7 +71,7 @@
},
{
"cell_type": "markdown",
"id": "5ac63202",
"id": "b87f9fdd",
"metadata": {
"editable": true
},
@@ -83,7 +83,7 @@
},
{
"cell_type": "markdown",
"id": "a52c6353",
"id": "6ab53932",
"metadata": {
"editable": true
},
@@ -93,7 +93,7 @@
},
{
"cell_type": "markdown",
"id": "910282dd",
"id": "1a7a8ec2",
"metadata": {
"editable": true
},
@@ -105,7 +105,7 @@
},
{
"cell_type": "markdown",
"id": "f3d0f2f5",
"id": "3bdb1514",
"metadata": {
"editable": true
},
@@ -120,7 +120,7 @@
},
{
"cell_type": "markdown",
"id": "5fe4bc49",
"id": "3e54eca9",
"metadata": {
"editable": true
},
@@ -132,7 +132,7 @@
},
{
"cell_type": "markdown",
"id": "55fee214",
"id": "3db49a1c",
"metadata": {
"editable": true
},
@@ -142,7 +142,7 @@
},
{
"cell_type": "markdown",
"id": "92b0b614",
"id": "5d4d3a0e",
"metadata": {
"editable": true
},
@@ -154,7 +154,7 @@
},
{
"cell_type": "markdown",
"id": "6b68973e",
"id": "90eecd1c",
"metadata": {
"editable": true
},
@@ -164,7 +164,7 @@
},
{
"cell_type": "markdown",
"id": "89e679e8",
"id": "7eb7605f",
"metadata": {
"editable": true
},
@@ -176,7 +176,7 @@
},
{
"cell_type": "markdown",
"id": "a35b6ab2",
"id": "b30dd90f",
"metadata": {
"editable": true
},
@@ -186,7 +186,7 @@
},
{
"cell_type": "markdown",
"id": "747d2724",
"id": "1167ec1f",
"metadata": {
"editable": true
},
@@ -198,7 +198,7 @@
},
{
"cell_type": "markdown",
"id": "911dc89a",
"id": "8f309e9e",
"metadata": {
"editable": true
},
@@ -216,7 +216,7 @@
{
"cell_type": "code",
"execution_count": 1,
"id": "09971906",
"id": "2df1063f",
"metadata": {
"collapsed": false,
"editable": true
@@ -230,7 +230,7 @@
},
{
"cell_type": "markdown",
"id": "de82ebe8",
"id": "0cb75ec5",
"metadata": {
"editable": true
},
@@ -244,7 +244,7 @@
},
{
"cell_type": "markdown",
"id": "21b2c91f",
"id": "f0c36bdb",
"metadata": {
"editable": true
},
@@ -257,7 +257,7 @@
},
{
"cell_type": "markdown",
"id": "3b542aa1",
"id": "8c3ce778",
"metadata": {
"editable": true
},
@@ -268,7 +268,7 @@
},
{
"cell_type": "markdown",
"id": "5393a93f",
"id": "f861d243",
"metadata": {
"editable": true
},
@@ -280,7 +280,7 @@
},
{
"cell_type": "markdown",
"id": "df1b8e23",
"id": "51500e4b",
"metadata": {
"editable": true
},
@@ -290,7 +290,7 @@
},
{
"cell_type": "markdown",
"id": "30ad9bb6",
"id": "fda9dadd",
"metadata": {
"editable": true
},
@@ -302,7 +302,7 @@
},
{
"cell_type": "markdown",
"id": "0e2c0d15",
"id": "6b691499",
"metadata": {
"editable": true
},
@@ -313,7 +313,7 @@
},
{
"cell_type": "markdown",
"id": "80eab896",
"id": "b6f10ab9",
"metadata": {
"editable": true
},
@@ -333,7 +333,7 @@
{
"cell_type": "code",
"execution_count": 2,
"id": "887378fa",
"id": "285159ae",
"metadata": {
"collapsed": false,
"editable": true
@@ -349,7 +349,7 @@
},
{
"cell_type": "markdown",
"id": "a059a330",
"id": "c2840fc9",
"metadata": {
"editable": true
},
@@ -359,7 +359,7 @@
},
{
"cell_type": "markdown",
"id": "e2614fe9",
"id": "9dbda275",
"metadata": {
"editable": true
},
@@ -370,7 +370,7 @@
},
{
"cell_type": "markdown",
"id": "8522c8f4",
"id": "824dba6f",
"metadata": {
"editable": true
},
@@ -382,7 +382,7 @@
},
{
"cell_type": "markdown",
"id": "c4e99901",
"id": "a3f059cf",
"metadata": {
"editable": true
},